2024-09-19T05:42:51.220 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T05:42:51.248 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T05:42:51.345 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911698' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFXHzwBr4ldgl5c9h/uBWOoETQ5GfoVQUdfIFULVffmtxAuwpTm3eSN4qwegOi+VM0fGz/WzE+qM4xvffORI258= smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFxCmDT9vHyCLJYaZOOtyEaii8Ok0Xd35eCCY0uRa8NifdvmDzlsIiLkQca5CC2mOTsQHaSrSO2OGAvmoIo//dQ= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBjC66aDIR2C8zjyduUCASjwRszpGwzXWjXB3K1TRC4g4G2dQDedhZNgSVhdEej7wdri5PgApha0qjjRVgnkyM8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T05:42:51.345 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T05:42:51.346 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T05:42:51.346 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T05:42:51.348 INFO:teuthology.task.internal:Checking packages... 2024-09-19T05:42:51.370 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T05:42:51.370 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T05:42:51.370 INFO:teuthology.packaging:ref: None 2024-09-19T05:42:51.370 INFO:teuthology.packaging:tag: None 2024-09-19T05:42:51.370 INFO:teuthology.packaging:branch: squid 2024-09-19T05:42:51.370 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:42:51.370 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T05:42:51.571 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T05:42:51.572 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T05:42:51.580 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T05:42:51.580 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T05:42:51.586 INFO:teuthology.task.internal:Saving configuration 2024-09-19T05:42:51.608 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T05:42:51.615 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T05:42:51.638 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 05:36:25.496271', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFXHzwBr4ldgl5c9h/uBWOoETQ5GfoVQUdfIFULVffmtxAuwpTm3eSN4qwegOi+VM0fGz/WzE+qM4xvffORI258='} 2024-09-19T05:42:51.663 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 05:36:25.495068', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFxCmDT9vHyCLJYaZOOtyEaii8Ok0Xd35eCCY0uRa8NifdvmDzlsIiLkQca5CC2mOTsQHaSrSO2OGAvmoIo//dQ='} 2024-09-19T05:42:51.684 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 05:36:25.493310', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBjC66aDIR2C8zjyduUCASjwRszpGwzXWjXB3K1TRC4g4G2dQDedhZNgSVhdEej7wdri5PgApha0qjjRVgnkyM8='} 2024-09-19T05:42:51.684 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T05:42:51.691 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T05:42:51.691 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T05:42:51.691 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T05:42:51.691 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T05:42:51.778 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faa6e60b520>, signals=[15]) 2024-09-19T05:42:51.778 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T05:42:51.785 INFO:teuthology.task.internal:Opening connections... 2024-09-19T05:42:51.785 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-09-19T05:42:51.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:42:51.863 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-09-19T05:42:51.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:42:51.939 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-09-19T05:42:51.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:42:52.014 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T05:42:52.021 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-09-19T05:42:52.039 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-09-19T05:42:52.040 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T05:42:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T05:42:52.099 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T05:42:52.099 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-09-19T05:42:52.112 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-09-19T05:42:52.131 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-09-19T05:42:52.131 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="CentOS Stream" 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="9" 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi049.stdout:ID="centos" 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="rhel fedora" 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="9" 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el9" 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi049.stdout:LOGO="fedora-logo-icon" 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://centos.org/" 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T05:42:52.191 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-09-19T05:42:52.212 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-09-19T05:42:52.231 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-09-19T05:42:52.231 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-09-19T05:42:52.289 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T05:42:52.290 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T05:42:52.291 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-09-19T05:42:52.312 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T05:42:52.322 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T05:42:52.346 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T05:42:52.346 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-09-19T05:42:52.348 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-09-19T05:42:52.351 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-09-19T05:42:52.368 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T05:42:52.375 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T05:42:52.375 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-09-19T05:42:52.405 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-09-19T05:42:52.409 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-09-19T05:42:52.495 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T05:42:52.572 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T05:42:52.588 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-19T05:42:52.588 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-19T05:42:52.588 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T05:42:52.589 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T05:42:52.589 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T05:42:52.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:42:52.607 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-09-19T05:42:52.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:42:52.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:42:52.609 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-09-19T05:42:52.609 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-503.el9.x86_64 2024-09-19T05:42:52.665 INFO:teuthology.task.kernel:Running kernel on smithi005: 5.14.0-503.el9.x86_64 2024-09-19T05:42:52.665 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum install -y kernel 2024-09-19T05:42:52.668 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-503.el9.x86_64 2024-09-19T05:42:52.668 INFO:teuthology.task.kernel:Running kernel on smithi049: 5.14.0-503.el9.x86_64 2024-09-19T05:42:52.668 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum install -y kernel 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-503.el9.x86_64 2024-09-19T05:42:52.669 INFO:teuthology.task.kernel:Running kernel on smithi157: 5.14.0-503.el9.x86_64 2024-09-19T05:42:52.669 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum install -y kernel 2024-09-19T05:42:55.077 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.3 MB 00:00 2024-09-19T05:42:56.845 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.3 MB 00:02 2024-09-19T05:42:57.294 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.3 MB 00:02 2024-09-19T05:42:59.078 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-19T05:42:59.512 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-19T05:43:01.187 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-19T05:43:02.680 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 19 kB 00:00 2024-09-19T05:43:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-19T05:43:03.577 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-19T05:43:03.905 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-19T05:43:05.630 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-19T05:43:05.787 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 19 kB 00:01 2024-09-19T05:43:06.776 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-19T05:43:06.833 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-19T05:43:09.181 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T05:43:11.154 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T05:43:11.154 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T05:43:11.253 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout:Install 4 Packages 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 79 M 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 126 M 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi005.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 415 kB/s | 36 kB 00:00 2024-09-19T05:43:11.766 INFO:teuthology.orchestra.run.smithi005.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-19T05:43:12.225 INFO:teuthology.orchestra.run.smithi005.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 37 MB/s | 35 MB 00:00 2024-09-19T05:43:12.264 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-19T05:43:12.348 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-19T05:43:12.634 INFO:teuthology.orchestra.run.smithi005.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 23 MB/s | 29 MB 00:01 2024-09-19T05:43:12.634 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:43:12.634 INFO:teuthology.orchestra.run.smithi005.stdout:Total 58 MB/s | 79 MB 00:01 2024-09-19T05:43:13.344 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-09-19T05:43:13.743 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-09-19T05:43:13.743 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-09-19T05:43:14.187 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T05:43:14.187 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T05:43:14.268 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-19T05:43:14.279 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-19T05:43:14.279 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-09-19T05:43:14.279 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-19T05:43:14.280 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-19T05:43:14.280 INFO:teuthology.orchestra.run.smithi157.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T05:43:14.280 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-19T05:43:14.280 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T05:43:14.280 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T05:43:14.280 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T05:43:14.280 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:43:14.280 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-19T05:43:14.280 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-19T05:43:14.280 INFO:teuthology.orchestra.run.smithi157.stdout:Install 4 Packages 2024-09-19T05:43:14.280 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:43:14.281 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 79 M 2024-09-19T05:43:14.281 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 126 M 2024-09-19T05:43:14.281 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-19T05:43:14.321 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T05:43:14.321 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi049.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi049.stdout:Install 4 Packages 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 79 M 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 126 M 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-19T05:43:14.608 INFO:teuthology.orchestra.run.smithi049.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 298 kB/s | 36 kB 00:00 2024-09-19T05:43:14.738 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-09-19T05:43:14.739 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-09-19T05:43:15.107 INFO:teuthology.orchestra.run.smithi157.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 62 kB/s | 36 kB 00:00 2024-09-19T05:43:15.333 INFO:teuthology.orchestra.run.smithi049.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-09-19T05:43:15.817 INFO:teuthology.orchestra.run.smithi049.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-19T05:43:15.907 INFO:teuthology.orchestra.run.smithi157.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-19T05:43:16.150 INFO:teuthology.orchestra.run.smithi049.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-19T05:43:16.151 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:43:16.151 INFO:teuthology.orchestra.run.smithi049.stdout:Total 46 MB/s | 79 MB 00:01 2024-09-19T05:43:16.582 INFO:teuthology.orchestra.run.smithi157.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-19T05:43:16.863 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-19T05:43:17.287 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-09-19T05:43:17.294 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-19T05:43:17.294 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-19T05:43:17.374 INFO:teuthology.orchestra.run.smithi157.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-19T05:43:17.374 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:43:17.375 INFO:teuthology.orchestra.run.smithi157.stdout:Total 25 MB/s | 79 MB 00:03 2024-09-19T05:43:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T05:43:18.025 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:43:18.058 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-19T05:43:18.299 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-19T05:43:18.300 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-19T05:43:18.418 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-19T05:43:18.418 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-19T05:43:19.393 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-19T05:43:19.393 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-19T05:43:20.787 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:43:20.868 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-19T05:43:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:43:21.538 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T05:43:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:43:21.839 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-19T05:43:22.569 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T05:43:22.650 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:43:24.507 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:43:24.654 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:43:25.418 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:43:25.546 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:43:26.041 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:43:26.079 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:43:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:43:35.286 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:10.600 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:10.996 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:10.996 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T05:44:10.996 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:44:10.996 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:44:11.803 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:11.803 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:44:11.803 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-09-19T05:44:11.803 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T05:44:11.803 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T05:44:11.803 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T05:44:11.803 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T05:44:11.803 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:44:11.803 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-09-19T05:44:12.131 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T05:44:12.214 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T05:44:12.214 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T05:44:12.215 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T05:44:12.215 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T05:44:12.215 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T05:44:12.215 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-19T05:44:12.215 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T05:44:12.215 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T05:44:12.274 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T05:44:12.274 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T05:44:12.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2024-09-19T05:44:12.317 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2024-09-19T05:44:12.317 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2024-09-19T05:44:12.317 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.06 2024-09-19T05:44:12.317 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 82.el9 2024-09-19T05:44:12.317 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2024-09-19T05:44:12.317 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T05:44:12.317 INFO:teuthology.orchestra.run.smithi005.stdout:Group : Unspecified 2024-09-19T05:44:12.317 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 8274670 2024-09-19T05:44:12.317 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2024-09-19T05:44:12.317 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : builder@centos.org 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : CentOS 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2024-09-19T05:44:12.318 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:44:12.319 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T05:44:12.319 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T05:44:12.319 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 5.14.0-508.el9.x86_64 2024-09-19T05:44:12.320 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T05:44:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:13.982 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:13.982 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T05:44:13.982 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:44:13.982 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:44:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T05:44:14.296 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-09-19T05:44:14.298 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-09-19T05:44:14.314 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.BezYsRsoX2 2024-09-19T05:44:14.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BezYsRsoX2 2024-09-19T05:44:14.386 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.BezYsRsoX2 2024-09-19T05:44:14.533 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.BezYsRsoX2 is 6KB 2024-09-19T05:44:14.583 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.BezYsRsoX2 2024-09-19T05:44:14.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T05:44:14.660 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T05:44:14.661 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T05:44:14.661 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T05:44:14.661 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T05:44:14.662 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T05:44:14.813 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-09-19T05:44:14.858 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:14.858 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:44:14.858 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-19T05:44:14.858 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T05:44:14.858 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T05:44:14.858 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T05:44:14.858 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T05:44:14.859 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:44:14.859 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-19T05:44:15.226 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T05:44:15.320 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T05:44:15.320 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T05:44:15.320 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T05:44:15.320 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T05:44:15.320 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T05:44:15.320 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-19T05:44:15.321 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T05:44:15.321 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T05:44:15.334 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:15.376 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:15.379 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T05:44:15.380 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T05:44:15.380 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -qi grub2-tools 2024-09-19T05:44:15.419 INFO:teuthology.orchestra.run.smithi157.stdout:Name : grub2-tools 2024-09-19T05:44:15.419 INFO:teuthology.orchestra.run.smithi157.stdout:Epoch : 1 2024-09-19T05:44:15.419 INFO:teuthology.orchestra.run.smithi157.stdout:Version : 2.06 2024-09-19T05:44:15.419 INFO:teuthology.orchestra.run.smithi157.stdout:Release : 82.el9 2024-09-19T05:44:15.419 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: x86_64 2024-09-19T05:44:15.419 INFO:teuthology.orchestra.run.smithi157.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T05:44:15.419 INFO:teuthology.orchestra.run.smithi157.stdout:Group : Unspecified 2024-09-19T05:44:15.420 INFO:teuthology.orchestra.run.smithi157.stdout:Size : 8274670 2024-09-19T05:44:15.420 INFO:teuthology.orchestra.run.smithi157.stdout:License : GPLv3+ 2024-09-19T05:44:15.420 INFO:teuthology.orchestra.run.smithi157.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T05:44:15.420 INFO:teuthology.orchestra.run.smithi157.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T05:44:15.420 INFO:teuthology.orchestra.run.smithi157.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T05:44:15.420 INFO:teuthology.orchestra.run.smithi157.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T05:44:15.420 INFO:teuthology.orchestra.run.smithi157.stdout:Packager : builder@centos.org 2024-09-19T05:44:15.420 INFO:teuthology.orchestra.run.smithi157.stdout:Vendor : CentOS 2024-09-19T05:44:15.420 INFO:teuthology.orchestra.run.smithi157.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T05:44:15.420 INFO:teuthology.orchestra.run.smithi157.stdout:Summary : Support tools for GRUB. 2024-09-19T05:44:15.420 INFO:teuthology.orchestra.run.smithi157.stdout:Description : 2024-09-19T05:44:15.420 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:44:15.421 INFO:teuthology.orchestra.run.smithi157.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T05:44:15.421 INFO:teuthology.orchestra.run.smithi157.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T05:44:15.421 INFO:teuthology.orchestra.run.smithi157.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T05:44:15.421 INFO:teuthology.orchestra.run.smithi157.stdout:hardware devices. 2024-09-19T05:44:15.421 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:44:15.421 INFO:teuthology.orchestra.run.smithi157.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T05:44:15.422 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T05:44:15.422 INFO:teuthology.task.kernel:Updating grub on smithi157 to boot 5.14.0-508.el9.x86_64 2024-09-19T05:44:15.422 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T05:44:15.681 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:15.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T05:44:15.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:44:15.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:44:16.063 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-19T05:44:16.849 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T05:44:16.932 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T05:44:16.932 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T05:44:16.932 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T05:44:16.932 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T05:44:16.932 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T05:44:16.932 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-19T05:44:16.933 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi049.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T05:44:16.933 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T05:44:16.993 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T05:44:16.993 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T05:44:16.993 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -qi grub2-tools 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi049.stdout:Name : grub2-tools 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:Epoch : 1 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:Version : 2.06 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:Release : 82.el9 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:Architecture: x86_64 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:Group : Unspecified 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:Size : 8274670 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:License : GPLv3+ 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:Packager : builder@centos.org 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:Vendor : CentOS 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:Summary : Support tools for GRUB. 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:Description : 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:hardware devices. 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T05:44:17.028 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T05:44:17.028 INFO:teuthology.task.kernel:Updating grub on smithi049 to boot 5.14.0-508.el9.x86_64 2024-09-19T05:44:17.028 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T05:44:17.376 INFO:teuthology.orchestra.run.smithi157.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T05:44:17.407 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-09-19T05:44:17.409 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-09-19T05:44:17.425 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.LWDsdKgRM5 2024-09-19T05:44:17.426 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LWDsdKgRM5 2024-09-19T05:44:17.500 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.LWDsdKgRM5 2024-09-19T05:44:17.646 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.LWDsdKgRM5 is 6KB 2024-09-19T05:44:17.694 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.LWDsdKgRM5 2024-09-19T05:44:17.709 DEBUG:teuthology.orchestra.run.smithi157:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T05:44:17.740 INFO:teuthology.orchestra.run.smithi049.stderr:Generating grub configuration file ... 2024-09-19T05:44:17.772 INFO:teuthology.orchestra.run.smithi157.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T05:44:17.772 INFO:teuthology.orchestra.run.smithi157.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T05:44:17.772 INFO:teuthology.orchestra.run.smithi157.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T05:44:17.772 INFO:teuthology.orchestra.run.smithi157.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T05:44:17.774 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T05:44:17.921 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2024-09-19T05:44:19.087 INFO:teuthology.orchestra.run.smithi049.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T05:44:19.117 INFO:teuthology.orchestra.run.smithi049.stderr:done 2024-09-19T05:44:19.119 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-09-19T05:44:19.135 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.Jt27islNaz 2024-09-19T05:44:19.135 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Jt27islNaz 2024-09-19T05:44:19.202 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.Jt27islNaz 2024-09-19T05:44:19.337 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.Jt27islNaz is 6KB 2024-09-19T05:44:19.385 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.Jt27islNaz 2024-09-19T05:44:19.403 DEBUG:teuthology.orchestra.run.smithi049:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T05:44:19.474 INFO:teuthology.orchestra.run.smithi049.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T05:44:19.475 INFO:teuthology.orchestra.run.smithi049.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T05:44:19.475 INFO:teuthology.orchestra.run.smithi049.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T05:44:19.475 INFO:teuthology.orchestra.run.smithi049.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T05:44:19.476 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T05:44:19.631 DEBUG:teuthology.orchestra.run.smithi049:> sudo shutdown -r now 2024-09-19T05:44:44.841 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-19T05:44:44.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-19T05:44:44.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:44:47.927 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-19T05:44:47.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-19T05:44:47.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:44:49.635 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-19T05:44:49.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-19T05:44:49.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:45:22.493 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-19T05:45:24.285 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-09-19T05:45:31.500 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-19T05:45:31.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:45:33.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-19T05:45:33.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:45:34.557 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-19T05:45:36.349 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-09-19T05:45:44.853 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T05:45:46.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-19T05:45:46.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:45:48.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-19T05:45:48.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:45:51.421 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-09-19T05:45:53.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-19T05:45:53.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:45:54.257 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-09-19T05:45:54.656 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-19T05:45:54.656 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-19T05:45:54.657 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-09-19T05:45:54.672 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-508.el9.x86_64 2024-09-19T05:45:54.673 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T05:45:54.673 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T05:45:54.673 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T05:45:55.674 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-19T05:45:55.674 DEBUG:teuthology.orchestra.run.smithi005:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T05:45:55.760 INFO:teuthology.orchestra.run.smithi005.stdout:ttyS1 2024-09-19T05:45:55.790 DEBUG:teuthology.parallel:result is None 2024-09-19T05:46:06.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-19T05:46:06.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:46:09.501 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-09-19T05:46:27.502 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-19T05:46:27.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:46:43.706 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-09-19T05:46:44.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-19T05:46:44.146 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-19T05:46:44.146 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-09-19T05:46:44.161 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-508.el9.x86_64 2024-09-19T05:46:44.161 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T05:46:44.161 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T05:46:44.161 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T05:46:45.162 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-19T05:46:45.162 DEBUG:teuthology.orchestra.run.smithi049:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T05:46:45.264 INFO:teuthology.orchestra.run.smithi049.stdout:ttyS1 2024-09-19T05:46:45.304 DEBUG:teuthology.parallel:result is None 2024-09-19T05:46:46.562 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T05:47:01.563 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-19T05:47:01.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:47:01.962 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-09-19T05:47:02.318 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-19T05:47:02.319 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-508.el9.x86_64"... 2024-09-19T05:47:02.319 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-09-19T05:47:02.335 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-508.el9.x86_64 2024-09-19T05:47:02.335 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T05:47:02.335 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T05:47:02.336 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T05:47:03.337 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-19T05:47:03.337 DEBUG:teuthology.orchestra.run.smithi157:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T05:47:03.445 INFO:teuthology.orchestra.run.smithi157.stdout:ttyS1 2024-09-19T05:47:03.479 DEBUG:teuthology.parallel:result is None 2024-09-19T05:47:03.479 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T05:47:03.486 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T05:47:03.487 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T05:47:03.490 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T05:47:03.493 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T05:47:03.512 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T05:47:03.519 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T05:47:03.526 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T05:47:03.526 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T05:47:03.550 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T05:47:03.555 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T05:47:03.597 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T05:47:03.603 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T05:47:03.603 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T05:47:03.617 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T05:47:03.623 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T05:47:03.652 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:47:03.655 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:47:03.663 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:47:03.667 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:47:03.668 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:47:03.679 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:47:03.681 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T05:47:03.687 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T05:47:03.687 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T05:47:03.712 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T05:47:03.716 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T05:47:03.749 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T05:47:03.758 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T05:47:03.758 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T05:47:03.782 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T05:47:03.784 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T05:47:03.805 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:47:03.878 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:47:03.931 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:47:04.012 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:47:04.058 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:47:04.059 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T05:47:04.125 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:47:04.168 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:47:04.221 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:47:04.305 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:47:04.347 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-19T05:47:04.347 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T05:47:04.414 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:47:04.460 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:47:04.521 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:47:04.566 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:47:04.608 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T05:47:04.609 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T05:47:04.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-09-19T05:47:04.678 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-09-19T05:47:04.680 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-09-19T05:47:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:47:04.734 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:47:04.763 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:47:05.177 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T05:47:05.184 INFO:teuthology.task.internal:Starting timer... 2024-09-19T05:47:05.184 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T05:47:05.199 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T05:47:05.208 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T05:47:05.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-09-19T05:47:05.272 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-09-19T05:47:05.274 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-09-19T05:47:05.346 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-09-19T05:47:05.348 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-09-19T05:47:05.405 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-09-19T05:47:05.407 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T05:47:05.407 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-09-19T05:47:05.434 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-09-19T05:47:05.434 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-09-19T05:47:05.454 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-09-19T05:47:05.455 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-09-19T05:47:05.479 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-09-19T05:47:05.479 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-09-19T05:47:05.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:47:05.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:47:05.509 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:47:05.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:47:05.537 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:47:05.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:47:05.564 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T05:47:05.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-09-19T05:47:05.601 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-09-19T05:47:05.644 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-09-19T05:47:05.693 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T05:47:05.707 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T05:47:05.721 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T05:47:05.722 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi049.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T05:50:57.604 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-09-19T05:50:57.605 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-19T05:50:57.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:50:57.688 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-09-19T05:50:57.727 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-19T05:50:57.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-19T05:50:57.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:50:57.803 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-09-19T05:50:57.889 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-19T05:50:57.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-19T05:50:57.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:50:57.968 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-09-19T05:50:58.052 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-19T05:50:58.053 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T05:50:58.066 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T05:50:58.066 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T05:50:58.067 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:50:58.070 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T05:50:58.070 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:50:58.073 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T05:50:58.073 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:50:58.100 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T05:50:58.102 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T05:50:58.116 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T05:50:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T05:50:58.124 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T05:50:58.139 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T05:50:58.141 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-09-19T05:50:58.145 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-09-19T05:50:58.153 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-09-19T05:50:58.156 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-09-19T05:50:58.164 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-09-19T05:50:58.168 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T05:50:58.172 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T05:50:58.175 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-09-19T05:50:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T05:50:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T05:50:58.189 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T05:50:58.204 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T05:50:58.247 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:50:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:50:58.251 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:50:58.252 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-09-19T05:50:58.252 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:50:58.252 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:50:58.252 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:50:58.252 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:50:58.253 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:50:58.254 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-09-19T05:50:58.254 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:50:58.254 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:50:58.254 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:50:58.254 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:50:58.264 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:50:58.267 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:50:58.268 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-09-19T05:50:58.268 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:50:58.268 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:50:58.268 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:50:58.268 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:50:58.269 INFO:teuthology.run_tasks:Running task install... 2024-09-19T05:50:58.277 DEBUG:teuthology.task.install:project ceph 2024-09-19T05:50:58.277 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T05:50:58.277 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T05:50:58.277 INFO:teuthology.task.install:Using flavor: default 2024-09-19T05:50:58.283 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T05:50:58.283 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T05:50:58.284 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T05:50:58.284 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:50:58.286 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T05:50:58.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:50:58.287 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T05:50:58.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:50:58.433 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T05:50:58.434 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T05:50:58.437 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T05:50:58.437 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T05:50:58.448 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T05:50:58.448 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T05:50:58.546 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T05:50:58.547 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:50:58.547 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T05:50:58.559 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T05:50:58.559 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-19T05:50:58.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T05:50:58.564 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T05:50:58.564 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T05:50:58.564 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T05:50:58.574 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T05:50:58.574 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T05:50:58.586 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T05:50:58.586 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T05:50:58.590 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T05:50:58.590 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T05:50:58.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T05:50:58.653 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T05:50:58.656 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T05:50:58.718 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T05:50:58.730 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T05:50:58.731 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T05:50:58.783 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-09-19T05:50:58.785 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-09-19T05:50:58.796 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-09-19T05:50:58.796 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-09-19T05:50:58.797 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-09-19T05:50:58.798 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-09-19T05:50:59.023 INFO:teuthology.orchestra.run.smithi005.stdout:45 files removed 2024-09-19T05:50:59.031 INFO:teuthology.orchestra.run.smithi049.stdout:45 files removed 2024-09-19T05:50:59.033 INFO:teuthology.orchestra.run.smithi157.stdout:45 files removed 2024-09-19T05:50:59.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T05:50:59.059 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T05:50:59.060 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T05:50:59.620 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-19T05:50:59.630 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 310 kB/s | 84 kB 00:00 2024-09-19T05:50:59.641 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 286 kB/s | 84 kB 00:00 2024-09-19T05:50:59.904 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T05:50:59.922 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-19T05:50:59.924 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T05:51:00.180 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T05:51:00.183 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 8.6 kB/s | 1.9 kB 00:00 2024-09-19T05:51:00.197 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T05:51:00.738 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.3 MB 00:00 2024-09-19T05:51:02.792 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.3 MB 00:02 2024-09-19T05:51:02.973 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.3 MB 00:02 2024-09-19T05:51:04.356 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 8.0 MB/s | 20 MB 00:02 2024-09-19T05:51:04.715 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-19T05:51:08.824 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.5 MB 00:00 2024-09-19T05:51:10.208 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 119 kB/s | 19 kB 00:00 2024-09-19T05:51:11.062 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-19T05:51:13.340 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.5 MB 00:04 2024-09-19T05:51:14.949 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-19T05:51:16.087 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-19T05:51:16.442 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-19T05:51:18.655 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:51:18.656 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:51:18.669 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T05:51:18.670 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T05:51:18.671 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T05:51:18.672 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T05:51:18.674 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T05:51:18.677 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T05:51:18.678 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T05:51:18.742 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-19T05:51:18.768 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-09-19T05:51:18.768 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-09-19T05:51:18.768 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-09-19T05:51:18.768 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-19T05:51:18.768 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T05:51:18.769 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T05:51:18.769 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T05:51:18.769 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T05:51:18.769 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T05:51:18.769 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T05:51:18.769 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T05:51:18.769 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T05:51:18.769 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T05:51:18.769 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T05:51:18.769 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T05:51:18.769 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T05:51:18.770 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T05:51:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T05:51:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T05:51:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T05:51:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T05:51:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T05:51:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T05:51:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T05:51:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T05:51:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T05:51:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T05:51:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T05:51:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T05:51:18.772 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-09-19T05:51:18.772 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T05:51:18.772 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T05:51:18.772 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T05:51:18.772 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T05:51:18.772 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T05:51:18.772 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T05:51:18.772 INFO:teuthology.orchestra.run.smithi157.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T05:51:18.772 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T05:51:18.772 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-19T05:51:18.772 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T05:51:18.772 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T05:51:18.773 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T05:51:18.773 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T05:51:18.773 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T05:51:18.773 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T05:51:18.773 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T05:51:18.773 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T05:51:18.773 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T05:51:18.773 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T05:51:18.773 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T05:51:18.774 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T05:51:18.775 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T05:51:18.775 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T05:51:18.775 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T05:51:18.775 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T05:51:18.775 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T05:51:18.775 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T05:51:18.775 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T05:51:18.775 INFO:teuthology.orchestra.run.smithi157.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T05:51:18.775 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T05:51:18.775 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T05:51:18.775 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T05:51:18.776 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T05:51:18.776 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T05:51:18.776 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T05:51:18.776 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T05:51:18.776 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T05:51:18.776 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T05:51:18.776 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T05:51:18.776 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T05:51:18.776 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T05:51:18.776 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T05:51:18.776 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T05:51:18.776 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T05:51:18.777 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T05:51:18.778 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T05:51:18.779 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T05:51:18.780 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T05:51:18.780 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T05:51:18.780 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T05:51:18.780 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T05:51:18.780 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T05:51:18.780 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T05:51:18.780 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T05:51:18.780 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T05:51:18.780 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T05:51:18.780 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T05:51:18.781 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T05:51:18.781 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T05:51:18.781 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T05:51:18.781 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T05:51:18.781 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T05:51:18.781 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T05:51:18.782 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T05:51:18.782 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T05:51:18.782 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T05:51:18.782 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T05:51:18.782 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T05:51:18.782 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T05:51:18.782 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T05:51:18.782 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T05:51:18.782 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2024-09-19T05:51:18.782 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T05:51:18.783 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:51:18.783 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-19T05:51:18.783 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-09-19T05:51:18.783 INFO:teuthology.orchestra.run.smithi157.stdout:Install 148 Packages 2024-09-19T05:51:18.783 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 8 Packages 2024-09-19T05:51:18.783 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:51:18.784 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 226 M 2024-09-19T05:51:18.784 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-19T05:51:19.369 INFO:teuthology.orchestra.run.smithi157.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 52 kB/s | 6.6 kB 00:00 2024-09-19T05:51:19.594 INFO:teuthology.orchestra.run.smithi157.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.9 MB/s | 896 kB 00:00 2024-09-19T05:51:19.678 INFO:teuthology.orchestra.run.smithi157.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.7 MB/s | 145 kB 00:00 2024-09-19T05:51:19.745 INFO:teuthology.orchestra.run.smithi157.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-19T05:51:19.895 INFO:teuthology.orchestra.run.smithi157.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 10 MB/s | 1.5 MB 00:00 2024-09-19T05:51:19.953 INFO:teuthology.orchestra.run.smithi157.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 8.7 MB/s | 2.4 MB 00:00 2024-09-19T05:51:20.179 INFO:teuthology.orchestra.run.smithi157.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-19T05:51:20.412 INFO:teuthology.orchestra.run.smithi157.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 18 MB/s | 21 MB 00:01 2024-09-19T05:51:20.454 INFO:teuthology.orchestra.run.smithi157.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 606 kB/s | 25 kB 00:00 2024-09-19T05:51:20.822 INFO:teuthology.orchestra.run.smithi157.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 17 MB/s | 11 MB 00:00 2024-09-19T05:51:20.871 INFO:teuthology.orchestra.run.smithi157.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T05:51:21.064 INFO:teuthology.orchestra.run.smithi157.stdout:(12/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 15 MB/s | 17 MB 00:01 2024-09-19T05:51:21.080 INFO:teuthology.orchestra.run.smithi157.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 129 kB/s | 27 kB 00:00 2024-09-19T05:51:21.105 INFO:teuthology.orchestra.run.smithi157.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 765 kB/s | 32 kB 00:00 2024-09-19T05:51:21.139 INFO:teuthology.orchestra.run.smithi157.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 13 MB/s | 751 kB 00:00 2024-09-19T05:51:21.165 INFO:teuthology.orchestra.run.smithi157.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.7 MB/s | 163 kB 00:00 2024-09-19T05:51:21.190 INFO:teuthology.orchestra.run.smithi157.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T05:51:21.215 INFO:teuthology.orchestra.run.smithi157.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 9.9 MB/s | 506 kB 00:00 2024-09-19T05:51:21.265 INFO:teuthology.orchestra.run.smithi157.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 903 kB/s | 45 kB 00:00 2024-09-19T05:51:21.332 INFO:teuthology.orchestra.run.smithi157.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.0 MB/s | 136 kB 00:00 2024-09-19T05:51:21.399 INFO:teuthology.orchestra.run.smithi157.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.4 MB/s | 161 kB 00:00 2024-09-19T05:51:21.508 INFO:teuthology.orchestra.run.smithi157.stdout:(22/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 5.3 MB 00:00 2024-09-19T05:51:21.608 INFO:teuthology.orchestra.run.smithi157.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 1.5 MB/s | 320 kB 00:00 2024-09-19T05:51:21.633 INFO:teuthology.orchestra.run.smithi157.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 2.4 MB/s | 300 kB 00:00 2024-09-19T05:51:21.684 INFO:teuthology.orchestra.run.smithi157.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.3 MB/s | 99 kB 00:00 2024-09-19T05:51:21.709 INFO:teuthology.orchestra.run.smithi157.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-19T05:51:21.767 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T05:51:21.768 INFO:teuthology.orchestra.run.smithi157.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-19T05:51:21.860 INFO:teuthology.orchestra.run.smithi157.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 18 MB/s | 3.1 MB 00:00 2024-09-19T05:51:21.885 INFO:teuthology.orchestra.run.smithi157.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 272 kB/s | 32 kB 00:00 2024-09-19T05:51:21.910 INFO:teuthology.orchestra.run.smithi157.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T05:51:22.185 INFO:teuthology.orchestra.run.smithi157.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 12 MB/s | 3.6 MB 00:00 2024-09-19T05:51:22.269 INFO:teuthology.orchestra.run.smithi157.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.9 MB/s | 249 kB 00:00 2024-09-19T05:51:22.403 INFO:teuthology.orchestra.run.smithi157.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-19T05:51:22.428 INFO:teuthology.orchestra.run.smithi157.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 311 kB/s | 49 kB 00:00 2024-09-19T05:51:22.453 INFO:teuthology.orchestra.run.smithi157.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 328 kB/s | 16 kB 00:00 2024-09-19T05:51:22.478 INFO:teuthology.orchestra.run.smithi157.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.5 MB/s | 285 kB 00:00 2024-09-19T05:51:22.521 INFO:teuthology.orchestra.run.smithi157.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 11 MB/s | 765 kB 00:00 2024-09-19T05:51:22.546 INFO:teuthology.orchestra.run.smithi157.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 272 kB/s | 18 kB 00:00 2024-09-19T05:51:22.997 INFO:teuthology.orchestra.run.smithi157.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 19 MB/s | 49 MB 00:02 2024-09-19T05:51:23.122 INFO:teuthology.orchestra.run.smithi157.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 133 kB/s | 80 kB 00:00 2024-09-19T05:51:23.214 INFO:teuthology.orchestra.run.smithi157.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-19T05:51:23.239 INFO:teuthology.orchestra.run.smithi157.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 790 kB/s | 191 kB 00:00 2024-09-19T05:51:23.298 INFO:teuthology.orchestra.run.smithi157.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-19T05:51:23.331 INFO:teuthology.orchestra.run.smithi157.stdout:(44/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 801 kB 00:00 2024-09-19T05:51:23.356 INFO:teuthology.orchestra.run.smithi157.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-19T05:51:23.382 INFO:teuthology.orchestra.run.smithi157.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-19T05:51:23.415 INFO:teuthology.orchestra.run.smithi157.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-19T05:51:23.457 INFO:teuthology.orchestra.run.smithi157.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.0 MB/s | 1.2 MB 00:00 2024-09-19T05:51:23.558 INFO:teuthology.orchestra.run.smithi157.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-09-19T05:51:23.591 INFO:teuthology.orchestra.run.smithi157.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-19T05:51:23.617 INFO:teuthology.orchestra.run.smithi157.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-19T05:51:23.641 INFO:teuthology.orchestra.run.smithi157.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-19T05:51:23.667 INFO:teuthology.orchestra.run.smithi157.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-09-19T05:51:23.700 INFO:teuthology.orchestra.run.smithi157.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-19T05:51:23.726 INFO:teuthology.orchestra.run.smithi157.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-19T05:51:23.751 INFO:teuthology.orchestra.run.smithi157.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T05:51:23.777 INFO:teuthology.orchestra.run.smithi157.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-19T05:51:23.802 INFO:teuthology.orchestra.run.smithi157.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-09-19T05:51:23.844 INFO:teuthology.orchestra.run.smithi157.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-19T05:51:23.885 INFO:teuthology.orchestra.run.smithi157.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 589 kB/s | 24 kB 00:00 2024-09-19T05:51:23.936 INFO:teuthology.orchestra.run.smithi157.stdout:(61/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-09-19T05:51:23.961 INFO:teuthology.orchestra.run.smithi157.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-09-19T05:51:23.986 INFO:teuthology.orchestra.run.smithi157.stdout:(63/156): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-09-19T05:51:24.012 INFO:teuthology.orchestra.run.smithi157.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-19T05:51:24.037 INFO:teuthology.orchestra.run.smithi157.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-19T05:51:24.038 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:51:24.040 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:51:24.053 INFO:teuthology.orchestra.run.smithi049.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T05:51:24.054 INFO:teuthology.orchestra.run.smithi049.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T05:51:24.054 INFO:teuthology.orchestra.run.smithi049.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T05:51:24.056 INFO:teuthology.orchestra.run.smithi049.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T05:51:24.058 INFO:teuthology.orchestra.run.smithi049.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T05:51:24.060 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T05:51:24.062 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T05:51:24.062 INFO:teuthology.orchestra.run.smithi157.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-19T05:51:24.129 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-19T05:51:24.138 INFO:teuthology.orchestra.run.smithi157.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-19T05:51:24.149 INFO:teuthology.orchestra.run.smithi049.stdout:====================================================================================== 2024-09-19T05:51:24.150 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-09-19T05:51:24.150 INFO:teuthology.orchestra.run.smithi049.stdout:====================================================================================== 2024-09-19T05:51:24.150 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-19T05:51:24.150 INFO:teuthology.orchestra.run.smithi049.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T05:51:24.150 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T05:51:24.150 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T05:51:24.150 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T05:51:24.150 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T05:51:24.150 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T05:51:24.150 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T05:51:24.150 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T05:51:24.151 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T05:51:24.152 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T05:51:24.153 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T05:51:24.154 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T05:51:24.155 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T05:51:24.156 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T05:51:24.157 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T05:51:24.158 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T05:51:24.159 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T05:51:24.160 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T05:51:24.161 INFO:teuthology.orchestra.run.smithi049.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T05:51:24.162 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T05:51:24.162 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T05:51:24.162 INFO:teuthology.orchestra.run.smithi049.stdout:Installing weak dependencies: 2024-09-19T05:51:24.162 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T05:51:24.162 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:51:24.162 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-19T05:51:24.162 INFO:teuthology.orchestra.run.smithi049.stdout:====================================================================================== 2024-09-19T05:51:24.162 INFO:teuthology.orchestra.run.smithi049.stdout:Install 148 Packages 2024-09-19T05:51:24.162 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 8 Packages 2024-09-19T05:51:24.162 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:51:24.163 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 226 M 2024-09-19T05:51:24.163 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-19T05:51:24.182 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 20 MB 00:20 2024-09-19T05:51:24.205 INFO:teuthology.orchestra.run.smithi157.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 26 MB/s | 4.9 MB 00:00 2024-09-19T05:51:24.247 INFO:teuthology.orchestra.run.smithi157.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-19T05:51:24.272 INFO:teuthology.orchestra.run.smithi157.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-19T05:51:24.348 INFO:teuthology.orchestra.run.smithi157.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-19T05:51:24.389 INFO:teuthology.orchestra.run.smithi157.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-09-19T05:51:24.414 INFO:teuthology.orchestra.run.smithi157.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-19T05:51:24.498 INFO:teuthology.orchestra.run.smithi157.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-19T05:51:24.515 INFO:teuthology.orchestra.run.smithi157.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 611 kB/s | 61 kB 00:00 2024-09-19T05:51:24.540 INFO:teuthology.orchestra.run.smithi157.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 5.8 MB/s | 245 kB 00:00 2024-09-19T05:51:24.565 INFO:teuthology.orchestra.run.smithi157.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-19T05:51:24.583 INFO:teuthology.orchestra.run.smithi157.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 602 kB/s | 26 kB 00:00 2024-09-19T05:51:24.599 INFO:teuthology.orchestra.run.smithi157.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 571 kB/s | 19 kB 00:00 2024-09-19T05:51:24.625 INFO:teuthology.orchestra.run.smithi157.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.1 MB/s | 172 kB 00:00 2024-09-19T05:51:24.650 INFO:teuthology.orchestra.run.smithi157.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-19T05:51:24.719 INFO:teuthology.orchestra.run.smithi049.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 54 kB/s | 6.6 kB 00:00 2024-09-19T05:51:24.734 INFO:teuthology.orchestra.run.smithi157.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-19T05:51:24.759 INFO:teuthology.orchestra.run.smithi157.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-19T05:51:24.784 INFO:teuthology.orchestra.run.smithi157.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-19T05:51:24.809 INFO:teuthology.orchestra.run.smithi157.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T05:51:24.835 INFO:teuthology.orchestra.run.smithi157.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-19T05:51:24.860 INFO:teuthology.orchestra.run.smithi157.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-19T05:51:24.885 INFO:teuthology.orchestra.run.smithi157.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-19T05:51:24.902 INFO:teuthology.orchestra.run.smithi049.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.8 MB/s | 896 kB 00:00 2024-09-19T05:51:24.910 INFO:teuthology.orchestra.run.smithi157.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-19T05:51:24.944 INFO:teuthology.orchestra.run.smithi157.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-09-19T05:51:24.969 INFO:teuthology.orchestra.run.smithi157.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 916 kB/s | 54 kB 00:00 2024-09-19T05:51:24.986 INFO:teuthology.orchestra.run.smithi049.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.7 MB/s | 145 kB 00:00 2024-09-19T05:51:25.053 INFO:teuthology.orchestra.run.smithi157.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-19T05:51:25.078 INFO:teuthology.orchestra.run.smithi157.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-19T05:51:25.086 INFO:teuthology.orchestra.run.smithi049.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-19T05:51:25.103 INFO:teuthology.orchestra.run.smithi157.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-09-19T05:51:25.129 INFO:teuthology.orchestra.run.smithi157.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-19T05:51:25.154 INFO:teuthology.orchestra.run.smithi157.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-09-19T05:51:25.170 INFO:teuthology.orchestra.run.smithi049.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 13 MB/s | 2.4 MB 00:00 2024-09-19T05:51:25.179 INFO:teuthology.orchestra.run.smithi157.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-19T05:51:25.205 INFO:teuthology.orchestra.run.smithi157.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T05:51:25.221 INFO:teuthology.orchestra.run.smithi049.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-19T05:51:25.230 INFO:teuthology.orchestra.run.smithi157.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T05:51:25.264 INFO:teuthology.orchestra.run.smithi157.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-19T05:51:25.289 INFO:teuthology.orchestra.run.smithi157.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-19T05:51:25.314 INFO:teuthology.orchestra.run.smithi157.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-19T05:51:25.406 INFO:teuthology.orchestra.run.smithi157.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-19T05:51:25.456 INFO:teuthology.orchestra.run.smithi157.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T05:51:25.545 INFO:teuthology.orchestra.run.smithi049.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-19T05:51:25.636 INFO:teuthology.orchestra.run.smithi157.stdout:(105/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-19T05:51:25.682 INFO:teuthology.orchestra.run.smithi157.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 1.3 MB/s | 308 kB 00:00 2024-09-19T05:51:25.707 INFO:teuthology.orchestra.run.smithi157.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 277 kB/s | 19 kB 00:00 2024-09-19T05:51:25.741 INFO:teuthology.orchestra.run.smithi157.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-09-19T05:51:25.766 INFO:teuthology.orchestra.run.smithi157.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-09-19T05:51:25.800 INFO:teuthology.orchestra.run.smithi157.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T05:51:25.812 INFO:teuthology.orchestra.run.smithi049.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 17 MB/s | 21 MB 00:01 2024-09-19T05:51:25.833 INFO:teuthology.orchestra.run.smithi157.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-09-19T05:51:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 607 kB/s | 25 kB 00:00 2024-09-19T05:51:25.858 INFO:teuthology.orchestra.run.smithi157.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-09-19T05:51:25.934 INFO:teuthology.orchestra.run.smithi157.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-19T05:51:25.959 INFO:teuthology.orchestra.run.smithi157.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-09-19T05:51:25.984 INFO:teuthology.orchestra.run.smithi157.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 868 kB/s | 43 kB 00:00 2024-09-19T05:51:26.010 INFO:teuthology.orchestra.run.smithi157.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-19T05:51:26.035 INFO:teuthology.orchestra.run.smithi157.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-19T05:51:26.102 INFO:teuthology.orchestra.run.smithi157.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-19T05:51:26.127 INFO:teuthology.orchestra.run.smithi157.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-09-19T05:51:26.152 INFO:teuthology.orchestra.run.smithi157.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-09-19T05:51:26.178 INFO:teuthology.orchestra.run.smithi157.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-19T05:51:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 17 MB/s | 17 MB 00:00 2024-09-19T05:51:26.203 INFO:teuthology.orchestra.run.smithi157.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-19T05:51:26.220 INFO:teuthology.orchestra.run.smithi157.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 648 kB/s | 11 kB 00:00 2024-09-19T05:51:26.245 INFO:teuthology.orchestra.run.smithi157.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-19T05:51:26.270 INFO:teuthology.orchestra.run.smithi157.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-19T05:51:26.295 INFO:teuthology.orchestra.run.smithi157.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-19T05:51:26.321 INFO:teuthology.orchestra.run.smithi157.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-09-19T05:51:26.331 INFO:teuthology.orchestra.run.smithi049.stdout:(11/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 14 MB/s | 11 MB 00:00 2024-09-19T05:51:26.347 INFO:teuthology.orchestra.run.smithi157.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-19T05:51:26.355 INFO:teuthology.orchestra.run.smithi049.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.4 MB/s | 228 kB 00:00 2024-09-19T05:51:26.372 INFO:teuthology.orchestra.run.smithi157.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T05:51:26.380 INFO:teuthology.orchestra.run.smithi049.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 540 kB/s | 27 kB 00:00 2024-09-19T05:51:26.406 INFO:teuthology.orchestra.run.smithi049.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 632 kB/s | 32 kB 00:00 2024-09-19T05:51:26.406 INFO:teuthology.orchestra.run.smithi157.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-19T05:51:26.430 INFO:teuthology.orchestra.run.smithi157.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T05:51:26.455 INFO:teuthology.orchestra.run.smithi049.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 9.8 MB/s | 751 kB 00:00 2024-09-19T05:51:26.481 INFO:teuthology.orchestra.run.smithi157.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-19T05:51:26.481 INFO:teuthology.orchestra.run.smithi049.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.1 MB/s | 163 kB 00:00 2024-09-19T05:51:26.506 INFO:teuthology.orchestra.run.smithi157.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-19T05:51:26.514 INFO:teuthology.orchestra.run.smithi049.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.1 MB/s | 127 kB 00:00 2024-09-19T05:51:26.532 INFO:teuthology.orchestra.run.smithi157.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-09-19T05:51:26.556 INFO:teuthology.orchestra.run.smithi157.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T05:51:26.559 INFO:teuthology.orchestra.run.smithi049.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 6.4 MB/s | 506 kB 00:00 2024-09-19T05:51:26.607 INFO:teuthology.orchestra.run.smithi157.stdout:(136/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-09-19T05:51:26.626 INFO:teuthology.orchestra.run.smithi049.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 678 kB/s | 45 kB 00:00 2024-09-19T05:51:26.632 INFO:teuthology.orchestra.run.smithi157.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-09-19T05:51:26.657 INFO:teuthology.orchestra.run.smithi157.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-09-19T05:51:26.684 INFO:teuthology.orchestra.run.smithi157.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-19T05:51:26.701 INFO:teuthology.orchestra.run.smithi049.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.8 MB/s | 136 kB 00:00 2024-09-19T05:51:26.708 INFO:teuthology.orchestra.run.smithi157.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-19T05:51:26.733 INFO:teuthology.orchestra.run.smithi157.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-19T05:51:26.758 INFO:teuthology.orchestra.run.smithi157.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-19T05:51:26.784 INFO:teuthology.orchestra.run.smithi157.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T05:51:26.809 INFO:teuthology.orchestra.run.smithi157.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T05:51:26.818 INFO:teuthology.orchestra.run.smithi049.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 18 MB/s | 5.3 MB 00:00 2024-09-19T05:51:26.834 INFO:teuthology.orchestra.run.smithi157.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-19T05:51:26.843 INFO:teuthology.orchestra.run.smithi049.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.1 MB/s | 161 kB 00:00 2024-09-19T05:51:26.859 INFO:teuthology.orchestra.run.smithi157.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T05:51:26.885 INFO:teuthology.orchestra.run.smithi049.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 4.7 MB/s | 320 kB 00:00 2024-09-19T05:51:26.893 INFO:teuthology.orchestra.run.smithi157.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-19T05:51:26.910 INFO:teuthology.orchestra.run.smithi049.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 4.4 MB/s | 300 kB 00:00 2024-09-19T05:51:26.936 INFO:teuthology.orchestra.run.smithi157.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T05:51:26.944 INFO:teuthology.orchestra.run.smithi049.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.7 MB/s | 99 kB 00:00 2024-09-19T05:51:26.969 INFO:teuthology.orchestra.run.smithi049.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-19T05:51:27.045 INFO:teuthology.orchestra.run.smithi049.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.2 MB/s | 171 kB 00:00 2024-09-19T05:51:27.170 INFO:teuthology.orchestra.run.smithi049.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 14 MB/s | 3.1 MB 00:00 2024-09-19T05:51:27.193 INFO:teuthology.orchestra.run.smithi157.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 10 MB/s | 3.4 MB 00:00 2024-09-19T05:51:27.195 INFO:teuthology.orchestra.run.smithi049.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 212 kB/s | 32 kB 00:00 2024-09-19T05:51:27.229 INFO:teuthology.orchestra.run.smithi049.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.5 MB/s | 146 kB 00:00 2024-09-19T05:51:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 5.7 MB/s | 2.3 MB 00:00 2024-09-19T05:51:27.388 INFO:teuthology.orchestra.run.smithi049.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 19 MB/s | 3.6 MB 00:00 2024-09-19T05:51:27.411 INFO:teuthology.orchestra.run.smithi157.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 703 kB/s | 41 kB 00:00 2024-09-19T05:51:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:(152/156): NetworkManager-libnm-1.51.0-1.el9.x8 6.8 MB/s | 1.8 MB 00:00 2024-09-19T05:51:27.471 INFO:teuthology.orchestra.run.smithi049.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.9 MB/s | 249 kB 00:00 2024-09-19T05:51:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.9 MB/s | 249 kB 00:00 2024-09-19T05:51:27.537 INFO:teuthology.orchestra.run.smithi157.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-19T05:51:27.546 INFO:teuthology.orchestra.run.smithi049.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 659 kB/s | 49 kB 00:00 2024-09-19T05:51:27.613 INFO:teuthology.orchestra.run.smithi049.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 247 kB/s | 16 kB 00:00 2024-09-19T05:51:27.697 INFO:teuthology.orchestra.run.smithi049.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-19T05:51:27.720 INFO:teuthology.orchestra.run.smithi157.stdout:(155/156): librbd1-19.1.1-361.g2f599a46.el9.x86 3.8 MB/s | 3.2 MB 00:00 2024-09-19T05:51:27.722 INFO:teuthology.orchestra.run.smithi049.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 2.6 MB/s | 285 kB 00:00 2024-09-19T05:51:27.762 INFO:teuthology.orchestra.run.smithi157.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 2.4 MB 00:00 2024-09-19T05:51:27.764 INFO:teuthology.orchestra.run.smithi049.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 11 MB/s | 765 kB 00:00 2024-09-19T05:51:27.767 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:51:27.767 INFO:teuthology.orchestra.run.smithi157.stdout:Total 25 MB/s | 226 MB 00:08 2024-09-19T05:51:27.789 INFO:teuthology.orchestra.run.smithi049.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 272 kB/s | 18 kB 00:00 2024-09-19T05:51:28.282 INFO:teuthology.orchestra.run.smithi049.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 20 MB/s | 49 MB 00:02 2024-09-19T05:51:28.307 INFO:teuthology.orchestra.run.smithi049.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 147 kB/s | 80 kB 00:00 2024-09-19T05:51:28.399 INFO:teuthology.orchestra.run.smithi049.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-19T05:51:28.424 INFO:teuthology.orchestra.run.smithi049.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-19T05:51:28.449 INFO:teuthology.orchestra.run.smithi049.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-19T05:51:28.475 INFO:teuthology.orchestra.run.smithi049.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-19T05:51:28.500 INFO:teuthology.orchestra.run.smithi049.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-19T05:51:28.534 INFO:teuthology.orchestra.run.smithi049.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-19T05:51:28.541 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-19T05:51:28.559 INFO:teuthology.orchestra.run.smithi049.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-19T05:51:28.601 INFO:teuthology.orchestra.run.smithi049.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-09-19T05:51:28.994 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-19T05:51:29.147 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-19T05:51:29.147 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-19T05:51:29.318 INFO:teuthology.orchestra.run.smithi049.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 149 kB/s | 107 kB 00:00 2024-09-19T05:51:29.402 INFO:teuthology.orchestra.run.smithi049.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 794 kB/s | 689 kB 00:00 2024-09-19T05:51:29.510 INFO:teuthology.orchestra.run.smithi049.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 992 kB/s | 943 kB 00:00 2024-09-19T05:51:29.869 INFO:teuthology.orchestra.run.smithi049.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 39 kB/s | 22 kB 00:00 2024-09-19T05:51:30.061 INFO:teuthology.orchestra.run.smithi049.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-19T05:51:30.100 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-19T05:51:30.186 INFO:teuthology.orchestra.run.smithi049.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 400 kB/s | 313 kB 00:00 2024-09-19T05:51:30.570 INFO:teuthology.orchestra.run.smithi049.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 36 kB/s | 18 kB 00:00 2024-09-19T05:51:30.703 INFO:teuthology.orchestra.run.smithi049.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-09-19T05:51:30.945 INFO:teuthology.orchestra.run.smithi049.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-09-19T05:51:30.965 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-19T05:51:31.303 INFO:teuthology.orchestra.run.smithi049.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 218 kB/s | 160 kB 00:00 2024-09-19T05:51:31.320 INFO:teuthology.orchestra.run.smithi049.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-09-19T05:51:31.378 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-19T05:51:31.378 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-19T05:51:31.829 INFO:teuthology.orchestra.run.smithi049.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 757 kB/s | 662 kB 00:00 2024-09-19T05:51:31.863 INFO:teuthology.orchestra.run.smithi049.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 45 kB/s | 24 kB 00:00 2024-09-19T05:51:32.154 INFO:teuthology.orchestra.run.smithi049.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 684 kB/s | 581 kB 00:00 2024-09-19T05:51:32.430 INFO:teuthology.orchestra.run.smithi049.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 79 kB/s | 47 kB 00:00 2024-09-19T05:51:32.713 INFO:teuthology.orchestra.run.smithi049.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 344 kB/s | 292 kB 00:00 2024-09-19T05:51:32.880 INFO:teuthology.orchestra.run.smithi049.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 260 kB/s | 188 kB 00:00 2024-09-19T05:51:32.980 INFO:teuthology.orchestra.run.smithi049.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 70 kB/s | 38 kB 00:00 2024-09-19T05:51:33.856 INFO:teuthology.orchestra.run.smithi049.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.3 MB/s | 4.9 MB 00:01 2024-09-19T05:51:34.022 INFO:teuthology.orchestra.run.smithi049.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-09-19T05:51:34.064 INFO:teuthology.orchestra.run.smithi049.stdout:(69/156): postgresql-13.16-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-09-19T05:51:34.781 INFO:teuthology.orchestra.run.smithi049.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 984 kB/s | 910 kB 00:00 2024-09-19T05:51:34.806 INFO:teuthology.orchestra.run.smithi049.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 175 kB/s | 137 kB 00:00 2024-09-19T05:51:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 5.0 MB/s | 5.8 MB 00:01 2024-09-19T05:51:35.507 INFO:teuthology.orchestra.run.smithi049.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 125 kB/s | 87 kB 00:00 2024-09-19T05:51:35.832 INFO:teuthology.orchestra.run.smithi049.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 983 kB/s | 1.0 MB 00:01 2024-09-19T05:51:36.158 INFO:teuthology.orchestra.run.smithi049.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 94 kB/s | 61 kB 00:00 2024-09-19T05:51:36.366 INFO:teuthology.orchestra.run.smithi049.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-19T05:51:36.538 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-19T05:51:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 294 kB/s | 245 kB 00:00 2024-09-19T05:51:36.917 INFO:teuthology.orchestra.run.smithi049.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 47 kB/s | 26 kB 00:00 2024-09-19T05:51:37.000 INFO:teuthology.orchestra.run.smithi049.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 295 kB/s | 248 kB 00:00 2024-09-19T05:51:37.142 INFO:teuthology.orchestra.run.smithi049.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 40 kB/s | 19 kB 00:00 2024-09-19T05:51:37.559 INFO:teuthology.orchestra.run.smithi049.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 62 kB/s | 35 kB 00:00 2024-09-19T05:51:37.648 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-19T05:51:37.659 INFO:teuthology.orchestra.run.smithi049.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 232 kB/s | 172 kB 00:00 2024-09-19T05:51:37.791 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T05:51:37.916 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T05:51:38.111 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T05:51:38.151 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T05:51:38.184 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T05:51:38.247 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T05:51:38.360 INFO:teuthology.orchestra.run.smithi049.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-09-19T05:51:38.385 INFO:teuthology.orchestra.run.smithi049.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 304 kB/s | 250 kB 00:00 2024-09-19T05:51:38.427 INFO:teuthology.orchestra.run.smithi049.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 290 kB/s | 222 kB 00:00 2024-09-19T05:51:38.583 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T05:51:38.702 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:51:38.807 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:51:38.836 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:51:38.874 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:51:38.875 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:51:38.887 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T05:51:38.888 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T05:51:38.889 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T05:51:38.889 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T05:51:38.891 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T05:51:38.894 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T05:51:38.895 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T05:51:38.911 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:51:38.945 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:51:38.961 INFO:teuthology.orchestra.run.smithi049.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 73 kB/s | 42 kB 00:00 2024-09-19T05:51:38.961 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-09-19T05:51:38.977 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-19T05:51:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-09-19T05:51:38.978 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-19T05:51:38.978 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-09-19T05:51:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T05:51:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T05:51:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T05:51:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T05:51:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T05:51:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T05:51:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T05:51:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T05:51:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T05:51:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T05:51:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T05:51:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T05:51:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T05:51:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T05:51:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T05:51:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T05:51:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T05:51:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T05:51:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T05:51:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T05:51:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T05:51:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T05:51:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T05:51:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T05:51:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T05:51:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T05:51:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T05:51:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T05:51:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T05:51:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T05:51:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T05:51:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T05:51:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T05:51:38.984 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T05:51:38.984 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T05:51:38.984 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T05:51:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T05:51:38.985 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-09-19T05:51:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T05:51:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T05:51:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T05:51:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T05:51:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T05:51:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T05:51:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T05:51:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T05:51:38.987 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-09-19T05:51:38.987 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T05:51:38.987 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T05:51:38.987 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T05:51:38.987 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T05:51:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T05:51:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T05:51:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T05:51:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T05:51:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T05:51:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T05:51:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T05:51:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T05:51:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T05:51:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T05:51:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T05:51:38.990 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T05:51:38.990 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T05:51:38.990 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T05:51:38.990 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T05:51:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T05:51:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T05:51:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T05:51:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T05:51:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T05:51:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T05:51:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T05:51:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T05:51:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T05:51:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T05:51:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T05:51:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T05:51:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T05:51:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T05:51:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T05:51:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T05:51:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T05:51:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T05:51:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T05:51:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T05:51:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T05:51:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T05:51:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T05:51:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T05:51:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T05:51:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T05:51:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T05:51:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T05:51:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T05:51:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T05:51:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T05:51:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T05:51:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T05:51:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T05:51:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T05:51:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T05:51:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T05:51:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T05:51:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T05:51:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T05:51:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T05:51:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T05:51:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T05:51:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T05:51:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T05:51:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T05:51:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T05:51:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T05:51:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T05:51:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T05:51:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T05:51:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T05:51:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T05:51:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T05:51:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T05:51:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T05:51:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T05:51:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T05:51:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T05:51:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T05:51:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T05:51:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T05:51:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T05:51:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T05:51:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T05:51:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T05:51:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T05:51:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T05:51:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T05:51:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T05:51:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T05:51:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T05:51:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T05:51:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T05:51:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T05:51:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T05:51:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T05:51:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T05:51:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T05:51:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T05:51:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T05:51:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T05:51:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T05:51:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T05:51:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T05:51:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T05:51:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T05:51:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T05:51:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T05:51:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T05:51:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T05:51:39.003 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2024-09-19T05:51:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T05:51:39.004 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:51:39.004 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-09-19T05:51:39.004 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-19T05:51:39.004 INFO:teuthology.orchestra.run.smithi005.stdout:Install 148 Packages 2024-09-19T05:51:39.004 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 8 Packages 2024-09-19T05:51:39.005 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:51:39.005 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 226 M 2024-09-19T05:51:39.006 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-09-19T05:51:39.052 INFO:teuthology.orchestra.run.smithi049.stdout:(87/156): python3-packaging-20.9-5.el9.noarch.r 112 kB/s | 77 kB 00:00 2024-09-19T05:51:39.127 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T05:51:39.179 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T05:51:39.194 INFO:teuthology.orchestra.run.smithi049.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 351 kB/s | 269 kB 00:00 2024-09-19T05:51:39.207 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:51:39.311 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:51:39.341 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:51:39.405 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:51:39.434 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:51:39.458 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:51:39.636 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:51:39.711 INFO:teuthology.orchestra.run.smithi049.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 212 kB/s | 159 kB 00:00 2024-09-19T05:51:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 50 kB/s | 6.6 kB 00:00 2024-09-19T05:51:39.788 INFO:teuthology.orchestra.run.smithi049.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 86 kB/s | 51 kB 00:00 2024-09-19T05:51:39.802 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:51:39.837 INFO:teuthology.orchestra.run.smithi049.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 356 kB/s | 279 kB 00:00 2024-09-19T05:51:39.940 INFO:teuthology.orchestra.run.smithi005.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.9 MB/s | 896 kB 00:00 2024-09-19T05:51:39.940 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T05:51:39.990 INFO:teuthology.orchestra.run.smithi005.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.8 MB/s | 145 kB 00:00 2024-09-19T05:51:40.044 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:51:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:51:40.249 INFO:teuthology.orchestra.run.smithi005.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 8.1 MB/s | 5.4 MB 00:00 2024-09-19T05:51:40.299 INFO:teuthology.orchestra.run.smithi005.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 7.7 MB/s | 2.4 MB 00:00 2024-09-19T05:51:40.320 INFO:teuthology.orchestra.run.smithi049.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 88 kB/s | 54 kB 00:00 2024-09-19T05:51:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:51:40.408 INFO:teuthology.orchestra.run.smithi005.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 9.6 MB/s | 1.5 MB 00:00 2024-09-19T05:51:40.462 INFO:teuthology.orchestra.run.smithi049.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-09-19T05:51:40.470 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T05:51:40.569 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T05:51:40.612 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T05:51:40.657 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T05:51:40.722 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T05:51:40.724 INFO:teuthology.orchestra.run.smithi005.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-19T05:51:40.766 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T05:51:40.816 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T05:51:40.866 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T05:51:40.908 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T05:51:40.935 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T05:51:41.003 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T05:51:41.057 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T05:51:41.088 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T05:51:41.096 INFO:teuthology.orchestra.run.smithi049.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 57 kB/s | 44 kB 00:00 2024-09-19T05:51:41.108 INFO:teuthology.orchestra.run.smithi005.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 14 MB/s | 21 MB 00:01 2024-09-19T05:51:41.139 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T05:51:41.150 INFO:teuthology.orchestra.run.smithi005.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 609 kB/s | 25 kB 00:00 2024-09-19T05:51:41.216 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T05:51:41.251 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T05:51:41.294 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T05:51:41.323 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi049.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-19T05:51:41.371 INFO:teuthology.orchestra.run.smithi049.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 243 kB/s | 221 kB 00:00 2024-09-19T05:51:41.400 INFO:teuthology.orchestra.run.smithi005.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 17 MB/s | 17 MB 00:00 2024-09-19T05:51:41.402 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T05:51:41.451 INFO:teuthology.orchestra.run.smithi005.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T05:51:41.454 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T05:51:41.529 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T05:51:41.576 INFO:teuthology.orchestra.run.smithi005.stdout:(12/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 12 MB/s | 11 MB 00:00 2024-09-19T05:51:41.601 INFO:teuthology.orchestra.run.smithi005.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 180 kB/s | 27 kB 00:00 2024-09-19T05:51:41.626 INFO:teuthology.orchestra.run.smithi005.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 631 kB/s | 32 kB 00:00 2024-09-19T05:51:41.660 INFO:teuthology.orchestra.run.smithi005.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 13 MB/s | 751 kB 00:00 2024-09-19T05:51:41.669 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T05:51:41.686 INFO:teuthology.orchestra.run.smithi005.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.7 MB/s | 163 kB 00:00 2024-09-19T05:51:41.711 INFO:teuthology.orchestra.run.smithi005.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T05:51:41.724 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T05:51:41.753 INFO:teuthology.orchestra.run.smithi005.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 7.4 MB/s | 506 kB 00:00 2024-09-19T05:51:41.805 INFO:teuthology.orchestra.run.smithi049.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 51 kB/s | 36 kB 00:00 2024-09-19T05:51:41.820 INFO:teuthology.orchestra.run.smithi005.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 676 kB/s | 45 kB 00:00 2024-09-19T05:51:41.903 INFO:teuthology.orchestra.run.smithi005.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.6 MB/s | 136 kB 00:00 2024-09-19T05:51:41.930 INFO:teuthology.orchestra.run.smithi049.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-09-19T05:51:41.987 INFO:teuthology.orchestra.run.smithi005.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 19 MB/s | 5.3 MB 00:00 2024-09-19T05:51:42.012 INFO:teuthology.orchestra.run.smithi005.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.5 MB/s | 161 kB 00:00 2024-09-19T05:51:42.038 INFO:teuthology.orchestra.run.smithi005.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 6.2 MB/s | 320 kB 00:00 2024-09-19T05:51:42.063 INFO:teuthology.orchestra.run.smithi005.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.8 MB/s | 300 kB 00:00 2024-09-19T05:51:42.088 INFO:teuthology.orchestra.run.smithi005.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T05:51:42.114 INFO:teuthology.orchestra.run.smithi005.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T05:51:42.155 INFO:teuthology.orchestra.run.smithi049.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 388 kB/s | 304 kB 00:00 2024-09-19T05:51:42.181 INFO:teuthology.orchestra.run.smithi005.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-19T05:51:42.239 INFO:teuthology.orchestra.run.smithi005.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 21 MB/s | 3.1 MB 00:00 2024-09-19T05:51:42.247 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T05:51:42.265 INFO:teuthology.orchestra.run.smithi005.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 381 kB/s | 32 kB 00:00 2024-09-19T05:51:42.290 INFO:teuthology.orchestra.run.smithi005.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.8 MB/s | 146 kB 00:00 2024-09-19T05:51:42.327 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T05:51:42.356 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T05:51:42.400 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T05:51:42.414 INFO:teuthology.orchestra.run.smithi049.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 95 kB/s | 58 kB 00:00 2024-09-19T05:51:42.437 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T05:51:42.480 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T05:51:42.590 INFO:teuthology.orchestra.run.smithi005.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 11 MB/s | 3.6 MB 00:00 2024-09-19T05:51:42.622 INFO:teuthology.orchestra.run.smithi049.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 138 kB/s | 96 kB 00:00 2024-09-19T05:51:42.647 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T05:51:42.658 INFO:teuthology.orchestra.run.smithi005.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.7 MB/s | 249 kB 00:00 2024-09-19T05:51:42.707 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T05:51:42.749 INFO:teuthology.orchestra.run.smithi005.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-19T05:51:42.762 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T05:51:42.783 INFO:teuthology.orchestra.run.smithi005.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 394 kB/s | 49 kB 00:00 2024-09-19T05:51:42.808 INFO:teuthology.orchestra.run.smithi005.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 281 kB/s | 16 kB 00:00 2024-09-19T05:51:42.823 INFO:teuthology.orchestra.run.smithi049.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 97 kB/s | 64 kB 00:00 2024-09-19T05:51:42.835 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T05:51:42.842 INFO:teuthology.orchestra.run.smithi005.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.8 MB/s | 285 kB 00:00 2024-09-19T05:51:42.866 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T05:51:42.885 INFO:teuthology.orchestra.run.smithi005.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 441 kB/s | 18 kB 00:00 2024-09-19T05:51:42.902 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T05:51:42.909 INFO:teuthology.orchestra.run.smithi005.stdout:(38/156): cephadm-19.1.1-361.g2f599a46.el9.noar 7.4 MB/s | 765 kB 00:00 2024-09-19T05:51:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T05:51:42.985 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T05:51:42.998 INFO:teuthology.orchestra.run.smithi049.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 78 kB/s | 45 kB 00:00 2024-09-19T05:51:43.028 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T05:51:43.066 INFO:teuthology.orchestra.run.smithi049.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 8.1 MB/s | 548 kB 00:00 2024-09-19T05:51:43.107 INFO:teuthology.orchestra.run.smithi049.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 8.9 MB/s | 308 kB 00:00 2024-09-19T05:51:43.133 INFO:teuthology.orchestra.run.smithi049.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-09-19T05:51:43.163 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T05:51:43.166 INFO:teuthology.orchestra.run.smithi049.stdout:(107/156): lua-devel-5.4.4-4.el9.x86_64.rpm 41 kB/s | 22 kB 00:00 2024-09-19T05:51:43.216 INFO:teuthology.orchestra.run.smithi049.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 529 kB/s | 26 kB 00:00 2024-09-19T05:51:43.241 INFO:teuthology.orchestra.run.smithi049.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T05:51:43.269 INFO:teuthology.orchestra.run.smithi049.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T05:51:43.300 INFO:teuthology.orchestra.run.smithi049.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-19T05:51:43.307 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T05:51:43.364 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T05:51:43.384 INFO:teuthology.orchestra.run.smithi049.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-19T05:51:43.405 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T05:51:43.426 INFO:teuthology.orchestra.run.smithi049.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-19T05:51:43.443 INFO:teuthology.orchestra.run.smithi049.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-19T05:51:43.476 INFO:teuthology.orchestra.run.smithi049.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-09-19T05:51:43.502 INFO:teuthology.orchestra.run.smithi049.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-09-19T05:51:43.509 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T05:51:43.527 INFO:teuthology.orchestra.run.smithi049.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-19T05:51:43.539 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T05:51:43.552 INFO:teuthology.orchestra.run.smithi049.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-19T05:51:43.586 INFO:teuthology.orchestra.run.smithi049.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-19T05:51:43.593 INFO:teuthology.orchestra.run.smithi005.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 20 MB/s | 49 MB 00:02 2024-09-19T05:51:43.596 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T05:51:43.611 INFO:teuthology.orchestra.run.smithi049.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-19T05:51:43.629 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T05:51:43.636 INFO:teuthology.orchestra.run.smithi049.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T05:51:43.654 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T05:51:43.670 INFO:teuthology.orchestra.run.smithi049.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-19T05:51:43.695 INFO:teuthology.orchestra.run.smithi049.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-19T05:51:43.721 INFO:teuthology.orchestra.run.smithi049.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 717 kB/s | 18 kB 00:00 2024-09-19T05:51:43.746 INFO:teuthology.orchestra.run.smithi049.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-19T05:51:43.748 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T05:51:43.771 INFO:teuthology.orchestra.run.smithi049.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-09-19T05:51:43.796 INFO:teuthology.orchestra.run.smithi049.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 796 kB/s | 20 kB 00:00 2024-09-19T05:51:43.847 INFO:teuthology.orchestra.run.smithi049.stdout:(128/156): protobuf-compiler-3.14.0-13.el9.x86_ 843 kB/s | 863 kB 00:01 2024-09-19T05:51:43.877 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T05:51:43.906 INFO:teuthology.orchestra.run.smithi049.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.9 MB/s | 2.0 MB 00:00 2024-09-19T05:51:43.914 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T05:51:43.931 INFO:teuthology.orchestra.run.smithi049.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.4 MB/s | 79 kB 00:00 2024-09-19T05:51:43.935 INFO:teuthology.orchestra.run.smithi005.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 76 kB/s | 80 kB 00:01 2024-09-19T05:51:43.949 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T05:51:43.957 INFO:teuthology.orchestra.run.smithi049.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-09-19T05:51:43.982 INFO:teuthology.orchestra.run.smithi049.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-19T05:51:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T05:51:44.024 INFO:teuthology.orchestra.run.smithi049.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-09-19T05:51:44.024 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T05:51:44.049 INFO:teuthology.orchestra.run.smithi049.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 244 kB/s | 16 kB 00:00 2024-09-19T05:51:44.064 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T05:51:44.074 INFO:teuthology.orchestra.run.smithi049.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T05:51:44.100 INFO:teuthology.orchestra.run.smithi049.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-19T05:51:44.102 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T05:51:44.133 INFO:teuthology.orchestra.run.smithi049.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 524 kB/s | 31 kB 00:00 2024-09-19T05:51:44.136 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T05:51:44.158 INFO:teuthology.orchestra.run.smithi049.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-19T05:51:44.170 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T05:51:44.183 INFO:teuthology.orchestra.run.smithi049.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-19T05:51:44.185 INFO:teuthology.orchestra.run.smithi005.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 628 kB/s | 801 kB 00:01 2024-09-19T05:51:44.209 INFO:teuthology.orchestra.run.smithi049.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-19T05:51:44.211 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T05:51:44.234 INFO:teuthology.orchestra.run.smithi049.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-19T05:51:44.246 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T05:51:44.259 INFO:teuthology.orchestra.run.smithi049.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 985 kB/s | 74 kB 00:00 2024-09-19T05:51:44.296 INFO:teuthology.orchestra.run.smithi049.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-19T05:51:44.318 INFO:teuthology.orchestra.run.smithi049.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-09-19T05:51:44.327 INFO:teuthology.orchestra.run.smithi005.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 261 kB/s | 191 kB 00:00 2024-09-19T05:51:44.333 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T05:51:44.343 INFO:teuthology.orchestra.run.smithi049.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-19T05:51:44.369 INFO:teuthology.orchestra.run.smithi049.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T05:51:44.377 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T05:51:44.402 INFO:teuthology.orchestra.run.smithi049.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-19T05:51:44.407 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T05:51:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T05:51:44.512 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T05:51:44.575 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T05:51:44.636 INFO:teuthology.orchestra.run.smithi049.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-19T05:51:44.712 INFO:teuthology.orchestra.run.smithi049.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-19T05:51:44.770 INFO:teuthology.orchestra.run.smithi049.stdout:(151/156): librados2-19.1.1-361.g2f599a46.el9.x 8.5 MB/s | 3.4 MB 00:00 2024-09-19T05:51:44.786 INFO:teuthology.orchestra.run.smithi005.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 298 kB/s | 253 kB 00:00 2024-09-19T05:51:44.787 INFO:teuthology.orchestra.run.smithi049.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 545 kB/s | 41 kB 00:00 2024-09-19T05:51:44.829 INFO:teuthology.orchestra.run.smithi049.stdout:(153/156): NetworkManager-libnm-1.51.0-1.el9.x8 9.4 MB/s | 1.8 MB 00:00 2024-09-19T05:51:44.879 INFO:teuthology.orchestra.run.smithi049.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.2 MB/s | 249 kB 00:00 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi049.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-09-19T05:51:44.971 INFO:teuthology.orchestra.run.smithi049.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 7.4 MB/s | 1.4 MB 00:00 2024-09-19T05:51:44.977 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:51:44.977 INFO:teuthology.orchestra.run.smithi049.stdout:Total 11 MB/s | 226 MB 00:20 2024-09-19T05:51:45.003 INFO:teuthology.orchestra.run.smithi005.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 106 kB 00:00 2024-09-19T05:51:45.153 INFO:teuthology.orchestra.run.smithi005.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-19T05:51:45.304 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T05:51:45.347 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T05:51:45.381 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T05:51:45.478 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T05:51:45.535 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T05:51:45.536 INFO:teuthology.orchestra.run.smithi005.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 180 kB/s | 135 kB 00:00 2024-09-19T05:51:45.589 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T05:51:45.662 INFO:teuthology.orchestra.run.smithi005.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-09-19T05:51:45.686 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T05:51:45.704 INFO:teuthology.orchestra.run.smithi005.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 178 kB/s | 125 kB 00:00 2024-09-19T05:51:45.728 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T05:51:45.769 INFO:teuthology.orchestra.run.smithi005.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 8.6 MB/s | 943 kB 00:00 2024-09-19T05:51:45.804 INFO:teuthology.orchestra.run.smithi005.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-09-19T05:51:45.830 INFO:teuthology.orchestra.run.smithi005.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 359 kB/s | 22 kB 00:00 2024-09-19T05:51:45.888 INFO:teuthology.orchestra.run.smithi005.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-19T05:51:45.914 INFO:teuthology.orchestra.run.smithi005.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-09-19T05:51:45.916 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T05:51:45.939 INFO:teuthology.orchestra.run.smithi005.stdout:(54/156): python3-urllib3-1.26.5-6.el9.noarch.r 273 kB/s | 215 kB 00:00 2024-09-19T05:51:45.948 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T05:51:45.981 INFO:teuthology.orchestra.run.smithi005.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 277 kB/s | 18 kB 00:00 2024-09-19T05:51:46.027 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T05:51:46.039 INFO:teuthology.orchestra.run.smithi005.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-09-19T05:51:46.065 INFO:teuthology.orchestra.run.smithi005.stdout:(57/156): libaio-devel-0.3.111-13.el9.x86_64.rp 89 kB/s | 11 kB 00:00 2024-09-19T05:51:46.090 INFO:teuthology.orchestra.run.smithi005.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-19T05:51:46.115 INFO:teuthology.orchestra.run.smithi005.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-09-19T05:51:46.166 INFO:teuthology.orchestra.run.smithi005.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-19T05:51:46.191 INFO:teuthology.orchestra.run.smithi005.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 325 kB/s | 24 kB 00:00 2024-09-19T05:51:46.233 INFO:teuthology.orchestra.run.smithi005.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-19T05:51:46.239 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-19T05:51:46.258 INFO:teuthology.orchestra.run.smithi005.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 513 kB/s | 47 kB 00:00 2024-09-19T05:51:46.292 INFO:teuthology.orchestra.run.smithi005.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-19T05:51:46.317 INFO:teuthology.orchestra.run.smithi005.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-19T05:51:46.345 INFO:teuthology.orchestra.run.smithi005.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 460 kB/s | 38 kB 00:00 2024-09-19T05:51:46.406 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-19T05:51:46.407 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-19T05:51:46.434 INFO:teuthology.orchestra.run.smithi005.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 35 MB/s | 4.9 MB 00:00 2024-09-19T05:51:46.485 INFO:teuthology.orchestra.run.smithi005.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-19T05:51:46.526 INFO:teuthology.orchestra.run.smithi005.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 9.7 MB/s | 910 kB 00:00 2024-09-19T05:51:46.552 INFO:teuthology.orchestra.run.smithi005.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-19T05:51:46.619 INFO:teuthology.orchestra.run.smithi005.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-19T05:51:46.661 INFO:teuthology.orchestra.run.smithi005.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-19T05:51:46.686 INFO:teuthology.orchestra.run.smithi005.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-19T05:51:46.694 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T05:51:46.778 INFO:teuthology.orchestra.run.smithi005.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-19T05:51:46.804 INFO:teuthology.orchestra.run.smithi005.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 523 kB/s | 61 kB 00:00 2024-09-19T05:51:46.814 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T05:51:46.846 INFO:teuthology.orchestra.run.smithi005.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-19T05:51:46.879 INFO:teuthology.orchestra.run.smithi005.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-09-19T05:51:46.957 INFO:teuthology.orchestra.run.smithi005.stdout:(78/156): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-09-19T05:51:46.980 INFO:teuthology.orchestra.run.smithi005.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 191 kB/s | 26 kB 00:00 2024-09-19T05:51:47.005 INFO:teuthology.orchestra.run.smithi005.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-09-19T05:51:47.030 INFO:teuthology.orchestra.run.smithi005.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-19T05:51:47.064 INFO:teuthology.orchestra.run.smithi005.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-09-19T05:51:47.107 INFO:teuthology.orchestra.run.smithi005.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-09-19T05:51:47.182 INFO:teuthology.orchestra.run.smithi005.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-09-19T05:51:47.215 INFO:teuthology.orchestra.run.smithi005.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-09-19T05:51:47.240 INFO:teuthology.orchestra.run.smithi005.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-09-19T05:51:47.274 INFO:teuthology.orchestra.run.smithi005.stdout:(87/156): python3-packaging-20.9-5.el9.noarch.r 460 kB/s | 77 kB 00:00 2024-09-19T05:51:47.299 INFO:teuthology.orchestra.run.smithi005.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-09-19T05:51:47.325 INFO:teuthology.orchestra.run.smithi005.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2024-09-19T05:51:47.350 INFO:teuthology.orchestra.run.smithi005.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-19T05:51:47.383 INFO:teuthology.orchestra.run.smithi005.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 607 kB/s | 51 kB 00:00 2024-09-19T05:51:47.417 INFO:teuthology.orchestra.run.smithi005.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 581 kB/s | 54 kB 00:00 2024-09-19T05:51:47.450 INFO:teuthology.orchestra.run.smithi005.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 623 kB/s | 42 kB 00:00 2024-09-19T05:51:47.476 INFO:teuthology.orchestra.run.smithi005.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 752 kB/s | 44 kB 00:00 2024-09-19T05:51:47.526 INFO:teuthology.orchestra.run.smithi005.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-19T05:51:47.551 INFO:teuthology.orchestra.run.smithi005.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 478 kB/s | 36 kB 00:00 2024-09-19T05:51:47.585 INFO:teuthology.orchestra.run.smithi005.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-09-19T05:51:47.636 INFO:teuthology.orchestra.run.smithi005.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.5 MB/s | 304 kB 00:00 2024-09-19T05:51:47.661 INFO:teuthology.orchestra.run.smithi005.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 764 kB/s | 58 kB 00:00 2024-09-19T05:51:47.853 INFO:teuthology.orchestra.run.smithi005.stdout:(100/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-19T05:51:47.878 INFO:teuthology.orchestra.run.smithi005.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 293 kB/s | 64 kB 00:00 2024-09-19T05:51:47.912 INFO:teuthology.orchestra.run.smithi005.stdout:(102/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 347 kB/s | 96 kB 00:00 2024-09-19T05:51:47.937 INFO:teuthology.orchestra.run.smithi005.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 537 kB/s | 45 kB 00:00 2024-09-19T05:51:47.996 INFO:teuthology.orchestra.run.smithi005.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T05:51:48.021 INFO:teuthology.orchestra.run.smithi005.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 156 kB/s | 22 kB 00:00 2024-09-19T05:51:48.063 INFO:teuthology.orchestra.run.smithi005.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 4.5 MB/s | 308 kB 00:00 2024-09-19T05:51:48.081 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T05:51:48.088 INFO:teuthology.orchestra.run.smithi005.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-19T05:51:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T05:51:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T05:51:48.164 INFO:teuthology.orchestra.run.smithi005.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.3 MB/s | 863 kB 00:00 2024-09-19T05:51:48.189 INFO:teuthology.orchestra.run.smithi005.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-19T05:51:48.219 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T05:51:48.248 INFO:teuthology.orchestra.run.smithi005.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-19T05:51:48.298 INFO:teuthology.orchestra.run.smithi005.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 839 kB 00:00 2024-09-19T05:51:48.324 INFO:teuthology.orchestra.run.smithi005.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T05:51:48.341 INFO:teuthology.orchestra.run.smithi005.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-19T05:51:48.389 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T05:51:48.416 INFO:teuthology.orchestra.run.smithi005.stdout:(116/156): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-19T05:51:48.424 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T05:51:48.441 INFO:teuthology.orchestra.run.smithi005.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 141 kB/s | 14 kB 00:00 2024-09-19T05:51:48.475 INFO:teuthology.orchestra.run.smithi005.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-09-19T05:51:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-09-19T05:51:48.525 INFO:teuthology.orchestra.run.smithi005.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-19T05:51:48.551 INFO:teuthology.orchestra.run.smithi005.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-19T05:51:48.576 INFO:teuthology.orchestra.run.smithi005.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.8 MB/s | 220 kB 00:00 2024-09-19T05:51:48.601 INFO:teuthology.orchestra.run.smithi005.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-19T05:51:48.626 INFO:teuthology.orchestra.run.smithi005.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-19T05:51:48.652 INFO:teuthology.orchestra.run.smithi005.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-19T05:51:48.677 INFO:teuthology.orchestra.run.smithi005.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 458 kB/s | 23 kB 00:00 2024-09-19T05:51:48.703 INFO:teuthology.orchestra.run.smithi005.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 385 kB/s | 19 kB 00:00 2024-09-19T05:51:48.716 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-19T05:51:48.717 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-19T05:51:48.728 INFO:teuthology.orchestra.run.smithi005.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-19T05:51:48.738 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T05:51:48.762 INFO:teuthology.orchestra.run.smithi005.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-19T05:51:48.786 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:51:48.787 INFO:teuthology.orchestra.run.smithi005.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T05:51:48.812 INFO:teuthology.orchestra.run.smithi005.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T05:51:48.854 INFO:teuthology.orchestra.run.smithi005.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-19T05:51:48.879 INFO:teuthology.orchestra.run.smithi005.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-19T05:51:48.883 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:51:48.916 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T05:51:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-19T05:51:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T05:51:48.952 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T05:51:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 977 kB/s | 90 kB 00:00 2024-09-19T05:51:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-19T05:51:49.023 INFO:teuthology.orchestra.run.smithi005.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T05:51:49.048 INFO:teuthology.orchestra.run.smithi005.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-19T05:51:49.073 INFO:teuthology.orchestra.run.smithi005.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-19T05:51:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-19T05:51:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T05:51:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T05:51:49.199 INFO:teuthology.orchestra.run.smithi005.stdout:(144/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-09-19T05:51:49.225 INFO:teuthology.orchestra.run.smithi005.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-09-19T05:51:49.250 INFO:teuthology.orchestra.run.smithi005.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T05:51:49.278 INFO:teuthology.orchestra.run.smithi005.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.7 MB/s | 427 kB 00:00 2024-09-19T05:51:49.329 INFO:teuthology.orchestra.run.smithi005.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T05:51:49.433 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T05:51:49.472 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:51:49.543 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:51:49.554 INFO:teuthology.orchestra.run.smithi005.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-19T05:51:49.582 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T05:51:49.614 INFO:teuthology.orchestra.run.smithi005.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 9.5 MB/s | 3.2 MB 00:00 2024-09-19T05:51:50.197 INFO:teuthology.orchestra.run.smithi005.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 70 kB/s | 41 kB 00:00 2024-09-19T05:51:50.355 INFO:teuthology.orchestra.run.smithi005.stdout:(152/156): NetworkManager-1.51.0-1.el9.x86_64.r 2.3 MB/s | 2.3 MB 00:01 2024-09-19T05:51:50.556 INFO:teuthology.orchestra.run.smithi005.stdout:(153/156): NetworkManager-libnm-1.51.0-1.el9.x8 1.8 MB/s | 1.8 MB 00:01 2024-09-19T05:51:50.964 INFO:teuthology.orchestra.run.smithi005.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 324 kB/s | 249 kB 00:00 2024-09-19T05:51:51.297 INFO:teuthology.orchestra.run.smithi005.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-19T05:51:51.589 INFO:teuthology.orchestra.run.smithi005.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-19T05:51:51.594 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:51:51.595 INFO:teuthology.orchestra.run.smithi005.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-19T05:51:52.226 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:51:52.262 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:51:52.807 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:51:52.834 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-09-19T05:51:52.848 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:51:52.940 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:51:52.940 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T05:51:52.940 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T05:51:52.940 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:51:52.974 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:51:53.003 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-09-19T05:51:53.003 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-09-19T05:51:55.141 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-19T05:51:55.243 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T05:51:55.332 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T05:51:55.369 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-09-19T05:51:55.369 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-09-19T05:51:55.538 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T05:51:55.603 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T05:51:55.628 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T05:51:55.687 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T05:51:56.037 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T05:51:56.145 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:51:56.270 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:51:56.296 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:51:56.397 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:51:56.432 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:51:56.625 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T05:51:56.687 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T05:51:56.717 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:51:56.807 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:51:56.835 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:51:56.881 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:51:56.911 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:51:56.938 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:51:57.131 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:51:57.302 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:51:57.437 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T05:51:57.550 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:51:57.619 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:51:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:51:57.951 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T05:51:58.059 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T05:51:58.099 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T05:51:58.143 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T05:51:58.218 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T05:51:58.269 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T05:51:58.309 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T05:51:58.352 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T05:51:58.385 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T05:51:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T05:51:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T05:51:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T05:51:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T05:51:58.617 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T05:51:58.697 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T05:51:58.744 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T05:51:58.789 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T05:51:58.816 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T05:51:58.892 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T05:51:58.940 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T05:51:59.021 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T05:51:59.174 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T05:51:59.236 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T05:51:59.801 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T05:51:59.892 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T05:51:59.933 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T05:51:59.977 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T05:52:00.015 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T05:52:00.057 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T05:52:00.234 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T05:52:00.284 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T05:52:00.341 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T05:52:00.416 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T05:52:00.460 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T05:52:00.496 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T05:52:00.545 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T05:52:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T05:52:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T05:52:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T05:52:00.836 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T05:52:00.883 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T05:52:00.924 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T05:52:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T05:52:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T05:52:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T05:52:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T05:52:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T05:52:01.276 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T05:52:01.418 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T05:52:01.450 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T05:52:01.479 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T05:52:01.516 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T05:52:01.560 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T05:52:01.600 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T05:52:01.629 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T05:52:01.673 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T05:52:01.706 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T05:52:01.757 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T05:52:01.799 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T05:52:01.890 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T05:52:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-09-19T05:52:01.946 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T05:52:01.977 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T05:52:02.069 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T05:52:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T05:52:02.108 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T05:52:02.146 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T05:52:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T05:52:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T05:52:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T05:52:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T05:52:02.731 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T05:52:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:52:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:52:02.931 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:52:02.941 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T05:52:02.983 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T05:52:03.006 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:52:03.017 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T05:52:03.044 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:52:03.123 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T05:52:03.173 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T05:52:03.218 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T05:52:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T05:52:03.291 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T05:52:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T05:52:03.326 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:52:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T05:52:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:52:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:52:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:52:03.552 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T05:52:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:52:03.584 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T05:52:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:52:03.664 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T05:52:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:52:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:52:04.111 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T05:52:04.233 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:52:04.312 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:52:04.366 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T05:52:04.478 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T05:52:04.530 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:52:04.649 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T05:52:04.750 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T05:52:04.785 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T05:52:04.828 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T05:52:04.911 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T05:52:04.935 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:52:04.935 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-09-19T05:52:04.936 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-09-19T05:52:04.936 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-09-19T05:52:04.936 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-09-19T05:52:04.936 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-09-19T05:52:04.936 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-09-19T05:52:04.936 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-09-19T05:52:04.936 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:52:04.963 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T05:52:05.002 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T05:52:05.046 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T05:52:05.087 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T05:52:05.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:52:05.114 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T05:52:05.126 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:52:05.126 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:05.126 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T05:52:05.126 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:52:05.186 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T05:52:05.238 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T05:52:05.276 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T05:52:05.320 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T05:52:05.398 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T05:52:05.430 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T05:52:05.474 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T05:52:05.668 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T05:52:05.668 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T05:52:05.668 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T05:52:05.703 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T05:52:05.847 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T05:52:05.851 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T05:52:05.913 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T05:52:05.954 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T05:52:05.982 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T05:52:06.015 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T05:52:06.351 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T05:52:06.395 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:52:06.463 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T05:52:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:52:06.516 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T05:52:06.551 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T05:52:06.553 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T05:52:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T05:52:06.605 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:52:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T05:52:06.631 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:52:06.632 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:06.632 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T05:52:06.632 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:52:06.632 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:52:06.632 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:52:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T05:52:06.710 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T05:52:06.856 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:52:06.879 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:52:06.879 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:06.879 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T05:52:06.880 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:52:06.880 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:52:06.880 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:52:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T05:52:06.946 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T05:52:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T05:52:07.062 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T05:52:07.083 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T05:52:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:52:07.121 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T05:52:07.158 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T05:52:07.161 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:52:07.191 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T05:52:07.205 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T05:52:07.240 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T05:52:07.284 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T05:52:07.323 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:52:07.326 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T05:52:07.346 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:52:07.346 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:07.347 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T05:52:07.347 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:52:07.347 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:52:07.347 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:52:07.452 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T05:52:07.467 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T05:52:07.510 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T05:52:07.556 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T05:52:07.564 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T05:52:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T05:52:07.752 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T05:52:07.807 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T05:52:07.884 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T05:52:07.974 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T05:52:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T05:52:08.128 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T05:52:08.175 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T05:52:08.206 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T05:52:08.238 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T05:52:08.254 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T05:52:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T05:52:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T05:52:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T05:52:08.466 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T05:52:08.606 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T05:52:08.638 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T05:52:08.666 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T05:52:08.670 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:52:08.699 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:52:08.700 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T05:52:08.730 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T05:52:08.740 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T05:52:08.787 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T05:52:08.792 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T05:52:08.817 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T05:52:08.822 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:52:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T05:52:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T05:52:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T05:52:08.961 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T05:52:09.051 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T05:52:09.100 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T05:52:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T05:52:09.255 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T05:52:09.316 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T05:52:09.643 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:52:09.679 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:52:09.889 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:52:09.923 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:52:10.071 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T05:52:10.129 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T05:52:10.166 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T05:52:10.257 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:52:10.267 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T05:52:10.299 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:52:10.330 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T05:52:10.376 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:52:10.376 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T05:52:10.376 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T05:52:10.377 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:10.383 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T05:52:10.417 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:52:10.473 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T05:52:10.510 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T05:52:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T05:52:10.733 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T05:52:10.813 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T05:52:10.887 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:52:10.915 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:52:11.017 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:52:11.187 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T05:52:11.209 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:52:11.233 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:52:11.233 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:11.233 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T05:52:11.233 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:52:11.234 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:52:11.234 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:52:11.284 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:52:11.297 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:52:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T05:52:11.444 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T05:52:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T05:52:11.626 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T05:52:12.373 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T05:52:12.404 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:52:12.428 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:52:12.428 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:12.428 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T05:52:12.428 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:52:12.428 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:52:12.428 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:52:12.471 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:52:12.494 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:52:12.495 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:12.495 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T05:52:12.495 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:52:12.538 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:52:12.562 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:52:12.562 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:12.562 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T05:52:12.562 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:52:12.562 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:52:12.562 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:52:12.864 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:52:12.887 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:52:12.888 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:12.888 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T05:52:12.888 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:52:12.888 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:52:12.888 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:52:12.937 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T05:52:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T05:52:13.066 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T05:52:13.107 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T05:52:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T05:52:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:52:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:52:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T05:52:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T05:52:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T05:52:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:52:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:52:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T05:52:16.720 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:52:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:52:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:52:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:52:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:52:17.459 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T05:52:17.459 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T05:52:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:17.492 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T05:52:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:52:17.534 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T05:52:17.575 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T05:52:17.607 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T05:52:17.634 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T05:52:17.829 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T05:52:17.881 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T05:52:17.911 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T05:52:17.941 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T05:52:17.987 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T05:52:18.612 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:52:18.652 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:52:18.672 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:52:19.525 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T05:52:19.614 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T05:52:19.666 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T05:52:19.762 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T05:52:19.800 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T05:52:19.826 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T05:52:19.852 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T05:52:19.852 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:52:19.944 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:52:19.944 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:52:20.028 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:52:20.053 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T05:52:20.058 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:52:20.059 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:52:20.255 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:52:20.255 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:52:20.331 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:52:20.395 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:52:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:52:22.750 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-09-19T05:52:22.751 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-09-19T05:52:22.751 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-09-19T05:52:22.751 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-09-19T05:52:22.751 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-09-19T05:52:22.751 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /root 2024-09-19T05:52:22.751 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-09-19T05:52:22.751 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:22.930 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:52:22.949 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:52:22.949 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:22.949 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T05:52:22.949 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:24.475 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:52:24.503 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:52:24.503 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:24.503 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T05:52:24.504 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:52:24.504 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:52:24.504 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:24.743 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:52:24.767 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:52:24.768 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:24.768 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T05:52:24.768 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:52:24.768 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:52:24.768 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:52:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:52:25.253 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:25.253 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T05:52:25.253 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:52:25.253 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:52:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T05:52:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T05:52:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T05:52:25.702 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T05:52:25.753 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T05:52:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T05:52:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T05:52:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T05:52:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T05:52:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T05:52:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T05:52:26.251 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:52:26.251 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:52:26.252 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T05:52:26.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T05:52:26.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T05:52:26.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T05:52:26.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T05:52:26.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T05:52:26.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T05:52:26.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T05:52:26.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T05:52:26.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T05:52:26.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T05:52:26.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T05:52:26.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T05:52:26.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T05:52:26.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T05:52:26.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T05:52:26.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T05:52:26.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T05:52:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T05:52:26.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T05:52:26.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T05:52:26.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T05:52:26.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T05:52:26.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T05:52:26.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T05:52:26.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T05:52:26.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T05:52:26.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T05:52:26.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T05:52:26.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T05:52:26.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T05:52:26.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T05:52:26.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T05:52:26.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T05:52:26.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T05:52:26.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T05:52:26.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T05:52:26.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T05:52:26.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T05:52:26.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T05:52:26.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T05:52:26.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T05:52:26.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T05:52:26.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T05:52:26.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T05:52:26.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T05:52:26.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T05:52:26.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T05:52:26.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T05:52:26.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T05:52:26.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T05:52:26.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T05:52:26.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T05:52:26.630 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:52:26.652 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:52:26.684 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T05:52:26.749 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T05:52:26.775 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:52:27.887 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:52:27.927 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:52:28.944 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:52:28.958 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:52:28.958 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:52:28.958 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-09-19T05:52:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T05:52:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T05:52:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T05:52:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T05:52:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T05:52:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T05:52:28.959 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-19T05:52:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T05:52:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T05:52:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T05:52:28.960 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T05:52:28.960 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.960 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.960 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.960 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.960 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:28.960 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.960 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.960 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.960 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:28.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:28.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:28.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:28.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:28.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:28.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:28.962 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:28.962 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.962 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.962 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:28.962 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T05:52:28.962 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T05:52:28.962 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T05:52:28.962 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T05:52:28.962 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T05:52:28.962 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T05:52:28.962 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T05:52:28.963 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T05:52:28.963 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T05:52:28.963 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T05:52:28.963 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T05:52:28.963 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.963 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.963 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.963 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T05:52:28.963 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T05:52:28.963 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T05:52:28.963 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T05:52:28.964 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T05:52:28.964 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.964 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.964 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T05:52:28.964 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.964 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T05:52:28.964 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T05:52:28.964 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T05:52:28.964 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T05:52:28.964 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T05:52:28.964 INFO:teuthology.orchestra.run.smithi157.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T05:52:28.964 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T05:52:28.965 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T05:52:28.965 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T05:52:28.965 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T05:52:28.965 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T05:52:28.965 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T05:52:28.965 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T05:52:28.965 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T05:52:28.965 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T05:52:28.965 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T05:52:28.965 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T05:52:28.965 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T05:52:28.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T05:52:28.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T05:52:28.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T05:52:28.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T05:52:28.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T05:52:28.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T05:52:28.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T05:52:28.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T05:52:28.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T05:52:28.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T05:52:28.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T05:52:28.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T05:52:28.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T05:52:28.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T05:52:28.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T05:52:28.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T05:52:28.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T05:52:28.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T05:52:28.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T05:52:28.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T05:52:28.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T05:52:28.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T05:52:28.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T05:52:28.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T05:52:28.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T05:52:28.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T05:52:28.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T05:52:28.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T05:52:28.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T05:52:28.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T05:52:28.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T05:52:28.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T05:52:28.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T05:52:28.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T05:52:28.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T05:52:28.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T05:52:28.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T05:52:28.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T05:52:28.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T05:52:28.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T05:52:28.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T05:52:28.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T05:52:28.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T05:52:28.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T05:52:28.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T05:52:28.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T05:52:28.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T05:52:28.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T05:52:28.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T05:52:28.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T05:52:28.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T05:52:28.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T05:52:28.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T05:52:28.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T05:52:28.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T05:52:28.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T05:52:28.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T05:52:28.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T05:52:28.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T05:52:28.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T05:52:28.972 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T05:52:28.972 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T05:52:28.972 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T05:52:28.972 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.972 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.972 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:28.973 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T05:52:28.973 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T05:52:28.973 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T05:52:28.973 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T05:52:28.973 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T05:52:28.973 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T05:52:28.974 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:52:28.974 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T05:52:28.974 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:52:28.974 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-19T05:52:29.080 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:52:29.256 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T05:52:29.288 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:52:29.313 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:52:29.314 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:29.314 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T05:52:29.314 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:52:29.314 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:52:29.314 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:29.363 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:52:29.375 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:52:29.393 DEBUG:teuthology.parallel:result is None 2024-09-19T05:52:29.477 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T05:52:29.538 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T05:52:29.926 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:52:29.926 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-09-19T05:52:29.926 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-09-19T05:52:29.926 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-09-19T05:52:29.926 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-09-19T05:52:29.926 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-09-19T05:52:29.926 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2024-09-19T05:52:29.926 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-09-19T05:52:29.926 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:30.116 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:52:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:52:30.133 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:30.133 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T05:52:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:30.515 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T05:52:30.549 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:52:30.574 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:52:30.574 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:30.574 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T05:52:30.574 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:52:30.575 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:52:30.575 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:30.616 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:52:30.640 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:52:30.641 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:30.641 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T05:52:30.641 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:30.692 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:52:30.717 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:52:30.717 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:30.717 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T05:52:30.717 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:52:30.717 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:52:30.717 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:31.043 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:52:31.068 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:52:31.068 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:31.068 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T05:52:31.068 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:52:31.068 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:52:31.068 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:52:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:52:31.680 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:31.681 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T05:52:31.681 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:52:31.681 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:52:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:52:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:52:31.944 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:31.945 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T05:52:31.945 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:52:31.945 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:52:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:52:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:52:32.437 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:32.437 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T05:52:32.437 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:52:32.437 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:52:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T05:52:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T05:52:32.682 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T05:52:32.877 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T05:52:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T05:52:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T05:52:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T05:52:33.278 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T05:52:33.339 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T05:52:33.374 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T05:52:33.398 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T05:52:33.821 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:52:33.856 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:52:33.888 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T05:52:33.952 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T05:52:33.979 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:52:35.086 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:52:35.122 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:52:35.913 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T05:52:35.955 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T05:52:35.996 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T05:52:36.027 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T05:52:36.054 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T05:52:36.135 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:52:36.164 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:52:36.264 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T05:52:36.275 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:52:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T05:52:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T05:52:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T05:52:36.451 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T05:52:36.457 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T05:52:36.483 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:52:36.508 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:52:36.508 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:36.508 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T05:52:36.508 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:52:36.508 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:52:36.508 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:36.559 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:52:36.570 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:52:36.673 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T05:52:36.730 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T05:52:37.094 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:52:37.146 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:52:37.168 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:52:37.707 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T05:52:37.736 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:52:37.761 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:52:37.761 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:37.761 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T05:52:37.761 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:52:37.762 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:52:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:52:37.828 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:52:37.828 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:37.828 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T05:52:37.828 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:52:37.912 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:52:37.912 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:37.912 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T05:52:37.913 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:52:37.913 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:52:37.913 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:38.090 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T05:52:38.184 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T05:52:38.230 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:52:38.234 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T05:52:38.255 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:52:38.255 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:52:38.255 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T05:52:38.255 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:52:38.255 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:52:38.255 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:38.330 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T05:52:38.362 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T05:52:38.396 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T05:52:38.439 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T05:52:38.439 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:52:38.531 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:52:38.531 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:52:38.615 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:52:38.640 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T05:52:38.646 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:52:38.646 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:52:38.842 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:52:38.842 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:52:38.935 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:52:39.003 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:52:43.085 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T05:52:43.126 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T05:52:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T05:52:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T05:52:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T05:52:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T05:52:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T05:52:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T05:52:43.548 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T05:52:43.590 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T05:52:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:52:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:52:44.265 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:52:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:52:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T05:52:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T05:52:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T05:52:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T05:52:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T05:52:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T05:52:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T05:52:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:52:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:52:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:52:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T05:52:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T05:52:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:52:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T05:52:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T05:52:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T05:52:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T05:52:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T05:52:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T05:52:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T05:52:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T05:52:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T05:52:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T05:52:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T05:52:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T05:52:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T05:52:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T05:52:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T05:52:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T05:52:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T05:52:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T05:52:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T05:52:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T05:52:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T05:52:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T05:52:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T05:52:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T05:52:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T05:52:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T05:52:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T05:52:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T05:52:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T05:52:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T05:52:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T05:52:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T05:52:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T05:52:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T05:52:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T05:52:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T05:52:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T05:52:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T05:52:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T05:52:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T05:52:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T05:52:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T05:52:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T05:52:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T05:52:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T05:52:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T05:52:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T05:52:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T05:52:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T05:52:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T05:52:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T05:52:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T05:52:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T05:52:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T05:52:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T05:52:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T05:52:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T05:52:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T05:52:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T05:52:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T05:52:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T05:52:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T05:52:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T05:52:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T05:52:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T05:52:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T05:52:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T05:52:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T05:52:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T05:52:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T05:52:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T05:52:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T05:52:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T05:52:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T05:52:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T05:52:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T05:52:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T05:52:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T05:52:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T05:52:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T05:52:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T05:52:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T05:52:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T05:52:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T05:52:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T05:52:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T05:52:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T05:52:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T05:52:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T05:52:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T05:52:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T05:52:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T05:52:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T05:52:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T05:52:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T05:52:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T05:52:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T05:52:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T05:52:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T05:52:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T05:52:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T05:52:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T05:52:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T05:52:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T05:52:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T05:52:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T05:52:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T05:52:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T05:52:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T05:52:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T05:52:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T05:52:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T05:52:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T05:52:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T05:52:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T05:52:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T05:52:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T05:52:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T05:52:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T05:52:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T05:52:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T05:52:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T05:52:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T05:52:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T05:52:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T05:52:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T05:52:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T05:52:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T05:52:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T05:52:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T05:52:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T05:52:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T05:52:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T05:52:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T05:52:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T05:52:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T05:52:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T05:52:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T05:52:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T05:52:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T05:52:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T05:52:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T05:52:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T05:52:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T05:52:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T05:52:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T05:52:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T05:52:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T05:52:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T05:52:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T05:52:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T05:52:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T05:52:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T05:52:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T05:52:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T05:52:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T05:52:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T05:52:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:52:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:52:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:52:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:52:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T05:52:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:52:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:52:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:52:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:52:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:52:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:52:47.382 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:52:47.382 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:47.383 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-09-19T05:52:47.383 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T05:52:47.383 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T05:52:47.383 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T05:52:47.383 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T05:52:47.383 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.383 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.383 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T05:52:47.383 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T05:52:47.383 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-19T05:52:47.383 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T05:52:47.383 INFO:teuthology.orchestra.run.smithi049.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T05:52:47.384 INFO:teuthology.orchestra.run.smithi049.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T05:52:47.384 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T05:52:47.384 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.384 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.384 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.384 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.384 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:47.384 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.385 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.385 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.385 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:47.385 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:47.385 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:47.385 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:47.385 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:47.385 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.385 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.385 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:47.385 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.385 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T05:52:47.386 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T05:52:47.387 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T05:52:47.387 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T05:52:47.387 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T05:52:47.387 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T05:52:47.387 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.387 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.387 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T05:52:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T05:52:47.389 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T05:52:47.389 INFO:teuthology.orchestra.run.smithi049.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T05:52:47.389 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T05:52:47.389 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T05:52:47.389 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T05:52:47.389 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T05:52:47.389 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T05:52:47.389 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T05:52:47.389 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T05:52:47.389 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T05:52:47.389 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T05:52:47.389 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T05:52:47.390 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T05:52:47.390 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T05:52:47.390 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T05:52:47.390 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T05:52:47.390 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T05:52:47.390 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T05:52:47.390 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T05:52:47.390 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.390 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.390 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.390 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T05:52:47.390 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T05:52:47.391 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T05:52:47.392 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T05:52:47.393 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T05:52:47.394 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T05:52:47.395 INFO:teuthology.orchestra.run.smithi049.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T05:52:47.396 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T05:52:47.396 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T05:52:47.396 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T05:52:47.396 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:52:47.396 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-19T05:52:47.797 DEBUG:teuthology.parallel:result is None 2024-09-19T05:52:51.641 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:52:51.641 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T05:52:51.642 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T05:52:51.642 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T05:52:51.642 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T05:52:51.642 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T05:52:51.642 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T05:52:51.642 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T05:52:51.642 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:52:51.642 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:52:51.642 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:52:51.642 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T05:52:51.642 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T05:52:51.642 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T05:52:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T05:52:51.644 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T05:52:51.645 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T05:52:51.646 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T05:52:51.647 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T05:52:51.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T05:52:51.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T05:52:51.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T05:52:51.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T05:52:51.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T05:52:51.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T05:52:51.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T05:52:51.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T05:52:51.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T05:52:51.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T05:52:51.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T05:52:51.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T05:52:51.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T05:52:51.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T05:52:51.651 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T05:52:51.652 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T05:52:51.653 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T05:52:51.654 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T05:52:51.654 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T05:52:51.654 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T05:52:51.654 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T05:52:51.654 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T05:52:51.654 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T05:52:51.654 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T05:52:51.654 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T05:52:54.610 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:52:54.610 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T05:52:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T05:52:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:52:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T05:52:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T05:52:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T05:52:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T05:52:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T05:52:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T05:52:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T05:52:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T05:52:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T05:52:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T05:52:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T05:52:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T05:52:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T05:52:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T05:52:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T05:52:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T05:52:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T05:52:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T05:52:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T05:52:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T05:52:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T05:52:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T05:52:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T05:52:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T05:52:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T05:52:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T05:52:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T05:52:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T05:52:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T05:52:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T05:52:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T05:52:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T05:52:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T05:52:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T05:52:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T05:52:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T05:52:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T05:52:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T05:52:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T05:52:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T05:52:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T05:52:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T05:52:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T05:52:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T05:52:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T05:52:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:52:54.623 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-09-19T05:52:55.188 DEBUG:teuthology.parallel:result is None 2024-09-19T05:52:55.188 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:52:55.332 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T05:52:55.358 INFO:teuthology.orchestra.run.smithi005.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T05:52:55.359 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T05:52:55.359 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T05:52:55.361 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:52:55.500 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T05:52:55.529 INFO:teuthology.orchestra.run.smithi049.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T05:52:55.529 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T05:52:55.529 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T05:52:55.531 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:52:55.670 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T05:52:55.696 INFO:teuthology.orchestra.run.smithi157.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T05:52:55.696 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T05:52:55.696 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T05:52:55.698 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T05:52:55.698 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:52:55.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T05:52:55.732 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-19T05:52:55.732 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T05:52:55.766 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T05:52:55.766 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T05:52:55.799 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T05:52:55.799 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:52:55.799 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T05:52:55.831 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T05:52:55.900 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-19T05:52:55.900 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T05:52:55.931 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T05:52:56.002 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T05:52:56.002 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T05:52:56.030 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T05:52:56.095 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T05:52:56.095 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:52:56.095 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T05:52:56.126 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T05:52:56.197 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-19T05:52:56.197 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T05:52:56.227 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T05:52:56.295 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T05:52:56.295 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T05:52:56.320 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T05:52:56.385 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T05:52:56.386 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:52:56.386 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T05:52:56.417 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T05:52:56.486 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-19T05:52:56.486 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T05:52:56.511 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T05:52:56.577 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T05:52:56.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T05:52:56.603 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T05:52:56.668 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T05:52:56.767 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-19T05:52:56.767 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:52:56.767 INFO:tasks.cephadm:Cluster fsid is 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:52:56.768 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T05:52:56.768 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.5', 'mon.b': '172.21.15.49', 'mon.c': '172.21.15.157'} 2024-09-19T05:52:56.768 INFO:tasks.cephadm:First mon is mon.a on smithi005 2024-09-19T05:52:56.768 INFO:tasks.cephadm:First mgr is x 2024-09-19T05:52:56.768 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T05:52:56.768 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname $(hostname -s) 2024-09-19T05:52:56.802 DEBUG:teuthology.orchestra.run.smithi049:> sudo hostname $(hostname -s) 2024-09-19T05:52:56.829 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2024-09-19T05:52:56.854 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T05:52:56.854 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:52:56.995 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T05:52:57.133 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:52:57.134 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T05:52:57.134 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T05:52:57.135 DEBUG:teuthology.orchestra.run.smithi005:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T05:52:57.513 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 05:52 /home/ubuntu/cephtest/cephadm 2024-09-19T05:52:57.514 DEBUG:teuthology.orchestra.run.smithi049:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T05:52:57.886 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 05:52 /home/ubuntu/cephtest/cephadm 2024-09-19T05:52:57.886 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T05:52:58.257 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 05:52 /home/ubuntu/cephtest/cephadm 2024-09-19T05:52:58.258 DEBUG:teuthology.orchestra.run.smithi005:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T05:52:58.278 DEBUG:teuthology.orchestra.run.smithi049:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T05:52:58.297 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T05:52:58.366 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T05:52:58.366 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T05:52:58.369 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T05:52:58.371 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T05:52:59.925 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T05:52:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T05:53:00.056 INFO:teuthology.orchestra.run.smithi049.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T05:53:27.966 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T05:53:27.967 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T05:53:27.967 INFO:teuthology.orchestra.run.smithi005.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T05:53:27.967 INFO:teuthology.orchestra.run.smithi005.stdout: "repo_digests": [ 2024-09-19T05:53:27.967 INFO:teuthology.orchestra.run.smithi005.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T05:53:27.967 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-19T05:53:27.967 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T05:53:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-09-19T05:53:29.182 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T05:53:29.182 INFO:teuthology.orchestra.run.smithi049.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T05:53:29.183 INFO:teuthology.orchestra.run.smithi049.stdout: "repo_digests": [ 2024-09-19T05:53:29.183 INFO:teuthology.orchestra.run.smithi049.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T05:53:29.183 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-09-19T05:53:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-09-19T05:53:32.243 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-09-19T05:53:32.244 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T05:53:32.244 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T05:53:32.244 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2024-09-19T05:53:32.244 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T05:53:32.244 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2024-09-19T05:53:32.244 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-09-19T05:53:32.269 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph 2024-09-19T05:53:32.304 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph 2024-09-19T05:53:32.338 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2024-09-19T05:53:32.366 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /etc/ceph 2024-09-19T05:53:32.397 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /etc/ceph 2024-09-19T05:53:32.427 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2024-09-19T05:53:32.452 INFO:tasks.cephadm:Writing seed config... 2024-09-19T05:53:32.454 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T05:53:32.454 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T05:53:32.454 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T05:53:32.454 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T05:53:32.454 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T05:53:32.454 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T05:53:32.454 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T05:53:32.454 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T05:53:32.454 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T05:53:32.454 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T05:53:32.455 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-19T05:53:32.456 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-19T05:53:32.457 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T05:53:32.457 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T05:53:32.457 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-19T05:53:32.457 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T05:53:32.457 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T05:53:32.457 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T05:53:32.457 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T05:53:32.457 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T05:53:32.457 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T05:53:32.457 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T05:53:32.457 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-19T05:53:32.458 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:53:32.458 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T05:53:32.477 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 6b5d55e8-764b-11ef-bceb-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T05:53:32.478 DEBUG:teuthology.orchestra.run.smithi005:mon.a> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T05:53:32.521 DEBUG:teuthology.orchestra.run.smithi005:mgr.x> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T05:53:32.564 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T05:53:32.565 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.5 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:53:32.784 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:53:32.784 INFO:teuthology.orchestra.run.smithi005.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', '6b5d55e8-764b-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.5', '--skip-admin-label'] 2024-09-19T05:53:32.784 INFO:teuthology.orchestra.run.smithi005.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T05:53:32.785 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying podman|docker is present... 2024-09-19T05:53:32.810 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 5.2.2 2024-09-19T05:53:32.810 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying lvm2 is present... 2024-09-19T05:53:32.810 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying time synchronization is in place... 2024-09-19T05:53:32.818 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T05:53:32.818 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T05:53:32.826 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T05:53:32.826 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-19T05:53:32.833 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-09-19T05:53:32.840 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-09-19T05:53:32.840 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-09-19T05:53:32.840 INFO:teuthology.orchestra.run.smithi005.stdout:Repeating the final host check... 2024-09-19T05:53:32.879 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 5.2.2 2024-09-19T05:53:32.879 INFO:teuthology.orchestra.run.smithi005.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T05:53:32.879 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl is present 2024-09-19T05:53:32.879 INFO:teuthology.orchestra.run.smithi005.stdout:lvcreate is present 2024-09-19T05:53:32.885 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T05:53:32.885 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T05:53:32.892 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T05:53:32.893 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-19T05:53:32.900 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-09-19T05:53:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-09-19T05:53:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-09-19T05:53:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:Host looks OK 2024-09-19T05:53:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:Cluster fsid: 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:53:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:Acquiring lock 140470055375872 on /run/cephadm/6b5d55e8-764b-11ef-bceb-c7b262605968.lock 2024-09-19T05:53:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:Lock 140470055375872 acquired on /run/cephadm/6b5d55e8-764b-11ef-bceb-c7b262605968.lock 2024-09-19T05:53:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 3300 ... 2024-09-19T05:53:32.909 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 6789 ... 2024-09-19T05:53:32.909 INFO:teuthology.orchestra.run.smithi005.stdout:Base mon IP(s) is [172.21.15.5:3300, 172.21.15.5:6789], mon addrv is [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-19T05:53:32.912 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.5 metric 100 2024-09-19T05:53:32.912 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.5 metric 100 2024-09-19T05:53:32.915 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T05:53:32.915 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-19T05:53:32.915 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1792sec hoplimit 64 pref medium 2024-09-19T05:53:32.919 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T05:53:32.919 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T05:53:32.919 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T05:53:32.919 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-19T05:53:32.920 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-09-19T05:53:32.920 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T05:53:32.920 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-09-19T05:53:32.920 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-09-19T05:53:32.921 INFO:teuthology.orchestra.run.smithi005.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T05:53:32.921 INFO:teuthology.orchestra.run.smithi005.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T05:53:32.922 INFO:teuthology.orchestra.run.smithi005.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T05:53:33.401 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T05:53:33.401 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T05:53:33.401 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T05:53:33.401 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T05:53:33.401 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T05:53:33.401 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T05:53:33.401 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T05:53:34.307 INFO:teuthology.orchestra.run.smithi005.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T05:53:34.307 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T05:53:34.307 INFO:teuthology.orchestra.run.smithi005.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T05:53:35.194 INFO:teuthology.orchestra.run.smithi005.stdout:stat: stdout 167 167 2024-09-19T05:53:35.194 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial keys... 2024-09-19T05:53:36.109 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBfvOtmtVCEHRAA9HDaH4xlCAfySTQ4+KSVbA== 2024-09-19T05:53:37.006 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBgvOtmsyfWFxAAbpYIorjc1O00xAuY3d4saQ== 2024-09-19T05:53:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBhvOtmrN3CFBAASGVoV4uAOK50spLMSqKKzg== 2024-09-19T05:53:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial monmap... 2024-09-19T05:53:41.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T05:53:41.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T05:53:41.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:53:41.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T05:53:41.567 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool for a [v2:172.21.15.5:3300,v1:172.21.15.5:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T05:53:41.567 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-09-19T05:53:41.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: set fsid to 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:53:41.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T05:53:41.567 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:53:41.567 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mon... 2024-09-19T05:53:49.973 INFO:teuthology.orchestra.run.smithi005.stdout:create mon.a on 2024-09-19T05:53:50.190 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T05:53:50.335 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T05:53:50.606 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-6b5d55e8-764b-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-6b5d55e8-764b-11ef-bceb-c7b262605968.target. 2024-09-19T05:53:50.606 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-6b5d55e8-764b-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-6b5d55e8-764b-11ef-bceb-c7b262605968.target. 2024-09-19T05:53:50.865 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.a 2024-09-19T05:53:50.865 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T05:53:51.030 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6b5d55e8-764b-11ef-bceb-c7b262605968.target.wants/ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@.service. 2024-09-19T05:53:51.513 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T05:53:51.514 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-19T05:53:51.520 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T05:53:51.520 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-19T05:53:51.520 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-19T05:53:51.520 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T05:53:51.521 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon to start... 2024-09-19T05:53:51.521 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon... 2024-09-19T05:53:51.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:51 smithi005 systemd[1]: Started Ceph mon.a for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T05:53:52.546 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:52 smithi005 ceph-mon[30588]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T05:53:52.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:52 smithi005 ceph-mon[30588]: monmap epoch 1 2024-09-19T05:53:52.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:52 smithi005 ceph-mon[30588]: fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:53:52.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:52 smithi005 ceph-mon[30588]: last_changed 2024-09-19T05:53:38.259181+0000 2024-09-19T05:53:52.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:52 smithi005 ceph-mon[30588]: created 2024-09-19T05:53:38.259181+0000 2024-09-19T05:53:52.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:52 smithi005 ceph-mon[30588]: min_mon_release 19 (squid) 2024-09-19T05:53:52.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:52 smithi005 ceph-mon[30588]: election_strategy: 1 2024-09-19T05:53:52.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:52 smithi005 ceph-mon[30588]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-19T05:53:52.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:52 smithi005 ceph-mon[30588]: fsmap 2024-09-19T05:53:52.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:52 smithi005 ceph-mon[30588]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T05:53:52.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:52 smithi005 ceph-mon[30588]: mgrmap e1: no daemons active 2024-09-19T05:53:52.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:52 smithi005 ceph-mon[30588]: from='client.? 172.21.15.5:0/240462990' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T05:53:52.554 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T05:53:52.554 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout id: 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:53:52.554 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T05:53:52.554 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-19T05:53:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout services: 2024-09-19T05:53:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.566035s) 2024-09-19T05:53:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T05:53:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T05:53:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-19T05:53:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout data: 2024-09-19T05:53:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T05:53:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T05:53:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T05:53:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T05:53:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-19T05:53:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.947+0000 7ff9f552b640 1 Processor -- start 2024-09-19T05:53:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.947+0000 7ff9f552b640 1 -- start start 2024-09-19T05:53:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.947+0000 7ff9f552b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 0x7ff9f01068f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:53:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.947+0000 7ff9f552b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0106ec0 con 0x7ff9f01064f0 2024-09-19T05:53:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.948+0000 7ff9eeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 0x7ff9f01068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:53:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.948+0000 7ff9eeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 0x7ff9f01068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41120/0 (socket says 172.21.15.5:41120) 2024-09-19T05:53:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.948+0000 7ff9eeffd640 1 -- 172.21.15.5:0/259421914 learned_addr learned my addr 172.21.15.5:0/259421914 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:53:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.948+0000 7ff9eeffd640 1 -- 172.21.15.5:0/259421914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f01076f0 con 0x7ff9f01064f0 2024-09-19T05:53:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.950+0000 7ff9eeffd640 1 --2- 172.21.15.5:0/259421914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 0x7ff9f01068f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009e80 tx=0x7ff9e802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6b95f110d70ac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:53:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.950+0000 7ff9edffb640 1 -- 172.21.15.5:0/259421914 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9e8036070 con 0x7ff9f01064f0 2024-09-19T05:53:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.950+0000 7ff9edffb640 1 -- 172.21.15.5:0/259421914 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e8031400 con 0x7ff9f01064f0 2024-09-19T05:53:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.950+0000 7ff9edffb640 1 -- 172.21.15.5:0/259421914 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9e8031700 con 0x7ff9f01064f0 2024-09-19T05:53:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.951+0000 7ff9f552b640 1 -- 172.21.15.5:0/259421914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 msgr2=0x7ff9f01068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.951+0000 7ff9f552b640 1 --2- 172.21.15.5:0/259421914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 0x7ff9f01068f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009e80 tx=0x7ff9e802a7a0 comp rx=0 tx=0).stop 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.951+0000 7ff9f552b640 1 -- 172.21.15.5:0/259421914 shutdown_connections 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.951+0000 7ff9f552b640 1 --2- 172.21.15.5:0/259421914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 0x7ff9f01068f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.951+0000 7ff9f552b640 1 -- 172.21.15.5:0/259421914 >> 172.21.15.5:0/259421914 conn(0x7ff9f0101ca0 msgr2=0x7ff9f01040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.951+0000 7ff9f552b640 1 -- 172.21.15.5:0/259421914 shutdown_connections 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.951+0000 7ff9f552b640 1 -- 172.21.15.5:0/259421914 wait complete. 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.952+0000 7ff9f552b640 1 Processor -- start 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.952+0000 7ff9f552b640 1 -- start start 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.952+0000 7ff9f552b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 0x7ff9f018f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.952+0000 7ff9f552b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0107c90 con 0x7ff9f01064f0 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.952+0000 7ff9eeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 0x7ff9f018f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.952+0000 7ff9eeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 0x7ff9f018f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41134/0 (socket says 172.21.15.5:41134) 2024-09-19T05:53:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.952+0000 7ff9eeffd640 1 -- 172.21.15.5:0/240462990 learned_addr learned my addr 172.21.15.5:0/240462990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.953+0000 7ff9eeffd640 1 -- 172.21.15.5:0/240462990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f018f5f0 con 0x7ff9f01064f0 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.953+0000 7ff9eeffd640 1 --2- 172.21.15.5:0/240462990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 0x7ff9f018f0b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8034040 tx=0x7ff9e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.953+0000 7ff9e7fff640 1 -- 172.21.15.5:0/240462990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9e8041070 con 0x7ff9f01064f0 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.953+0000 7ff9e7fff640 1 -- 172.21.15.5:0/240462990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e803c650 con 0x7ff9f01064f0 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.953+0000 7ff9f552b640 1 -- 172.21.15.5:0/240462990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f018d7e0 con 0x7ff9f01064f0 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.953+0000 7ff9e7fff640 1 -- 172.21.15.5:0/240462990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9e8036040 con 0x7ff9f01064f0 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.953+0000 7ff9f552b640 1 -- 172.21.15.5:0/240462990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f018dd20 con 0x7ff9f01064f0 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.953+0000 7ff9e7fff640 1 -- 172.21.15.5:0/240462990 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff9e8044070 con 0x7ff9f01064f0 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.953+0000 7ff9e7fff640 1 -- 172.21.15.5:0/240462990 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff9e8046c10 con 0x7ff9f01064f0 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.954+0000 7ff9f552b640 1 -- 172.21.15.5:0/240462990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f01082e0 con 0x7ff9f01064f0 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:51.956+0000 7ff9e7fff640 1 -- 172.21.15.5:0/240462990 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff9e803c950 con 0x7ff9f01064f0 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.018+0000 7ff9f552b640 1 -- 172.21.15.5:0/240462990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7ff9f018e420 con 0x7ff9f01064f0 2024-09-19T05:53:52.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.018+0000 7ff9e7fff640 1 -- 172.21.15.5:0/240462990 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7ff9e803cb30 con 0x7ff9f01064f0 2024-09-19T05:53:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.019+0000 7ff9f552b640 1 -- 172.21.15.5:0/240462990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 msgr2=0x7ff9f018f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:53:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.019+0000 7ff9f552b640 1 --2- 172.21.15.5:0/240462990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 0x7ff9f018f0b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8034040 tx=0x7ff9e8004a40 comp rx=0 tx=0).stop 2024-09-19T05:53:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.019+0000 7ff9f552b640 1 -- 172.21.15.5:0/240462990 shutdown_connections 2024-09-19T05:53:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.019+0000 7ff9f552b640 1 --2- 172.21.15.5:0/240462990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f01064f0 0x7ff9f018f0b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:53:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.019+0000 7ff9f552b640 1 -- 172.21.15.5:0/240462990 >> 172.21.15.5:0/240462990 conn(0x7ff9f0101ca0 msgr2=0x7ff9f007b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:53:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.019+0000 7ff9f552b640 1 -- 172.21.15.5:0/240462990 shutdown_connections 2024-09-19T05:53:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.019+0000 7ff9f552b640 1 -- 172.21.15.5:0/240462990 wait complete. 2024-09-19T05:53:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:mon is available 2024-09-19T05:53:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T05:53:53.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:53 smithi005 ceph-mon[30588]: from='client.? 172.21.15.5:0/2400599334' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T05:53:53.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:53 smithi005 ceph-mon[30588]: from='client.? 172.21.15.5:0/2400599334' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T05:53:53.639 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-19T05:53:53.646 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T05:53:53.646 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:53:53.646 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-19T05:53:53.646 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T05:53:53.646 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T05:53:53.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.926+0000 7fe13c810640 1 Processor -- start 2024-09-19T05:53:53.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.926+0000 7fe13c810640 1 -- start start 2024-09-19T05:53:53.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.926+0000 7fe13c810640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 0x7fe138107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:53:53.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.926+0000 7fe13c810640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1381066c0 con 0x7fe138107bb0 2024-09-19T05:53:53.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.926+0000 7fe13659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 0x7fe138107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:53:53.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.926+0000 7fe13659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 0x7fe138107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41136/0 (socket says 172.21.15.5:41136) 2024-09-19T05:53:53.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.926+0000 7fe13659c640 1 -- 172.21.15.5:0/2403231645 learned_addr learned my addr 172.21.15.5:0/2403231645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:53:53.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.927+0000 7fe13659c640 1 -- 172.21.15.5:0/2403231645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe138106840 con 0x7fe138107bb0 2024-09-19T05:53:53.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.927+0000 7fe13659c640 1 --2- 172.21.15.5:0/2403231645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 0x7fe138107fb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe128009820 tx=0x7fe12802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662af31502cfb41d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:53:53.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.928+0000 7fe135d9b640 1 -- 172.21.15.5:0/2403231645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe128037070 con 0x7fe138107bb0 2024-09-19T05:53:53.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.928+0000 7fe135d9b640 1 -- 172.21.15.5:0/2403231645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe128032440 con 0x7fe138107bb0 2024-09-19T05:53:53.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.928+0000 7fe135d9b640 1 -- 172.21.15.5:0/2403231645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe128032740 con 0x7fe138107bb0 2024-09-19T05:53:53.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.928+0000 7fe13c810640 1 -- 172.21.15.5:0/2403231645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 msgr2=0x7fe138107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:53:53.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.928+0000 7fe13c810640 1 --2- 172.21.15.5:0/2403231645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 0x7fe138107fb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe128009820 tx=0x7fe12802a270 comp rx=0 tx=0).stop 2024-09-19T05:53:53.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.928+0000 7fe13c810640 1 -- 172.21.15.5:0/2403231645 shutdown_connections 2024-09-19T05:53:53.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.928+0000 7fe13c810640 1 --2- 172.21.15.5:0/2403231645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 0x7fe138107fb0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:53:53.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.928+0000 7fe13c810640 1 -- 172.21.15.5:0/2403231645 >> 172.21.15.5:0/2403231645 conn(0x7fe1381013e0 msgr2=0x7fe138103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:53:53.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.928+0000 7fe13c810640 1 -- 172.21.15.5:0/2403231645 shutdown_connections 2024-09-19T05:53:53.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.928+0000 7fe13c810640 1 -- 172.21.15.5:0/2403231645 wait complete. 2024-09-19T05:53:53.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.929+0000 7fe13c810640 1 Processor -- start 2024-09-19T05:53:53.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.929+0000 7fe13c810640 1 -- start start 2024-09-19T05:53:53.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.929+0000 7fe13c810640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 0x7fe138106030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:53:53.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.929+0000 7fe13c810640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe138106de0 con 0x7fe138107bb0 2024-09-19T05:53:53.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.929+0000 7fe13659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 0x7fe138106030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:53:53.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.929+0000 7fe13659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 0x7fe138106030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41144/0 (socket says 172.21.15.5:41144) 2024-09-19T05:53:53.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.929+0000 7fe13659c640 1 -- 172.21.15.5:0/2400599334 learned_addr learned my addr 172.21.15.5:0/2400599334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:53:53.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.930+0000 7fe13659c640 1 -- 172.21.15.5:0/2400599334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13819dbe0 con 0x7fe138107bb0 2024-09-19T05:53:53.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.930+0000 7fe13659c640 1 --2- 172.21.15.5:0/2400599334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 0x7fe138106030 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe1280090b0 tx=0x7fe1280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:53:53.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.930+0000 7fe12f7fe640 1 -- 172.21.15.5:0/2400599334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe128041070 con 0x7fe138107bb0 2024-09-19T05:53:53.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.930+0000 7fe12f7fe640 1 -- 172.21.15.5:0/2400599334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe12803c7a0 con 0x7fe138107bb0 2024-09-19T05:53:53.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.930+0000 7fe13c810640 1 -- 172.21.15.5:0/2400599334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13819ddb0 con 0x7fe138107bb0 2024-09-19T05:53:53.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.930+0000 7fe12f7fe640 1 -- 172.21.15.5:0/2400599334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe128037040 con 0x7fe138107bb0 2024-09-19T05:53:53.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.930+0000 7fe13c810640 1 -- 172.21.15.5:0/2400599334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13819e150 con 0x7fe138107bb0 2024-09-19T05:53:53.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.930+0000 7fe12f7fe640 1 -- 172.21.15.5:0/2400599334 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe128044070 con 0x7fe138107bb0 2024-09-19T05:53:53.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.930+0000 7fe12f7fe640 1 -- 172.21.15.5:0/2400599334 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe12803d3f0 con 0x7fe138107bb0 2024-09-19T05:53:53.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.931+0000 7fe13c810640 1 -- 172.21.15.5:0/2400599334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe138107430 con 0x7fe138107bb0 2024-09-19T05:53:53.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.933+0000 7fe12f7fe640 1 -- 172.21.15.5:0/2400599334 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe128046450 con 0x7fe138107bb0 2024-09-19T05:53:53.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.993+0000 7fe13c810640 1 -- 172.21.15.5:0/2400599334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe13819e490 con 0x7fe138107bb0 2024-09-19T05:53:53.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.996+0000 7fe12f7fe640 1 -- 172.21.15.5:0/2400599334 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7fe128046630 con 0x7fe138107bb0 2024-09-19T05:53:53.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.997+0000 7fe12f7fe640 1 -- 172.21.15.5:0/2400599334 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe128056070 con 0x7fe138107bb0 2024-09-19T05:53:53.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.997+0000 7fe13c810640 1 -- 172.21.15.5:0/2400599334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 msgr2=0x7fe138106030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:53:53.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.997+0000 7fe13c810640 1 --2- 172.21.15.5:0/2400599334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 0x7fe138106030 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe1280090b0 tx=0x7fe1280046c0 comp rx=0 tx=0).stop 2024-09-19T05:53:53.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.997+0000 7fe13c810640 1 -- 172.21.15.5:0/2400599334 shutdown_connections 2024-09-19T05:53:53.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.997+0000 7fe13c810640 1 --2- 172.21.15.5:0/2400599334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe138107bb0 0x7fe138106030 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:53:53.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.998+0000 7fe13c810640 1 -- 172.21.15.5:0/2400599334 >> 172.21.15.5:0/2400599334 conn(0x7fe1381013e0 msgr2=0x7fe13818dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:53:53.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.998+0000 7fe13c810640 1 -- 172.21.15.5:0/2400599334 shutdown_connections 2024-09-19T05:53:53.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:52.998+0000 7fe13c810640 1 -- 172.21.15.5:0/2400599334 wait complete. 2024-09-19T05:53:53.651 INFO:teuthology.orchestra.run.smithi005.stdout:Generating new minimal ceph.conf... 2024-09-19T05:53:54.685 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:54 smithi005 ceph-mon[30588]: from='client.? 172.21.15.5:0/3279007749' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:53:54.691 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.020+0000 7f015ae49640 1 Processor -- start 2024-09-19T05:53:54.691 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.020+0000 7f015ae49640 1 -- start start 2024-09-19T05:53:54.691 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.020+0000 7f015ae49640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 0x7f01541090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:53:54.691 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.020+0000 7f015ae49640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0154109680 con 0x7f0154108cb0 2024-09-19T05:53:54.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.020+0000 7f0158be5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 0x7f01541090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:53:54.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.020+0000 7f0158be5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 0x7f01541090b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41158/0 (socket says 172.21.15.5:41158) 2024-09-19T05:53:54.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.020+0000 7f0158be5640 1 -- 172.21.15.5:0/2517773527 learned_addr learned my addr 172.21.15.5:0/2517773527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:53:54.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.021+0000 7f0158be5640 1 -- 172.21.15.5:0/2517773527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0154105550 con 0x7f0154108cb0 2024-09-19T05:53:54.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.021+0000 7f0158be5640 1 --2- 172.21.15.5:0/2517773527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 0x7f01541090b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0144009820 tx=0x7f014402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59cdd53649cce2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:53:54.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.022+0000 7f01537fe640 1 -- 172.21.15.5:0/2517773527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0144037070 con 0x7f0154108cb0 2024-09-19T05:53:54.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.022+0000 7f01537fe640 1 -- 172.21.15.5:0/2517773527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f01440328d0 con 0x7f0154108cb0 2024-09-19T05:53:54.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.022+0000 7f01537fe640 1 -- 172.21.15.5:0/2517773527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01440307d0 con 0x7f0154108cb0 2024-09-19T05:53:54.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.022+0000 7f015ae49640 1 -- 172.21.15.5:0/2517773527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 msgr2=0x7f01541090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:53:54.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.022+0000 7f015ae49640 1 --2- 172.21.15.5:0/2517773527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 0x7f01541090b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0144009820 tx=0x7f014402a270 comp rx=0 tx=0).stop 2024-09-19T05:53:54.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.022+0000 7f015ae49640 1 -- 172.21.15.5:0/2517773527 shutdown_connections 2024-09-19T05:53:54.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.022+0000 7f015ae49640 1 --2- 172.21.15.5:0/2517773527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 0x7f01541090b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:53:54.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.022+0000 7f015ae49640 1 -- 172.21.15.5:0/2517773527 >> 172.21.15.5:0/2517773527 conn(0x7f01541013e0 msgr2=0x7f0154103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:53:54.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.022+0000 7f015ae49640 1 -- 172.21.15.5:0/2517773527 shutdown_connections 2024-09-19T05:53:54.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.022+0000 7f015ae49640 1 -- 172.21.15.5:0/2517773527 wait complete. 2024-09-19T05:53:54.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.023+0000 7f015ae49640 1 Processor -- start 2024-09-19T05:53:54.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.023+0000 7f015ae49640 1 -- start start 2024-09-19T05:53:54.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.023+0000 7f015ae49640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 0x7f015419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:53:54.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.023+0000 7f015ae49640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f015410bc20 con 0x7f0154108cb0 2024-09-19T05:53:54.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.023+0000 7f0158be5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 0x7f015419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:53:54.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.023+0000 7f0158be5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 0x7f015419b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41168/0 (socket says 172.21.15.5:41168) 2024-09-19T05:53:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.023+0000 7f0158be5640 1 -- 172.21.15.5:0/3279007749 learned_addr learned my addr 172.21.15.5:0/3279007749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:53:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.024+0000 7f0158be5640 1 -- 172.21.15.5:0/3279007749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015419bba0 con 0x7f0154108cb0 2024-09-19T05:53:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.024+0000 7f0158be5640 1 --2- 172.21.15.5:0/3279007749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 0x7f015419b660 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f01440097f0 tx=0x7f0144030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:53:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.024+0000 7f0151ffb640 1 -- 172.21.15.5:0/3279007749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0144037040 con 0x7f0154108cb0 2024-09-19T05:53:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.024+0000 7f0151ffb640 1 -- 172.21.15.5:0/3279007749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0144032bb0 con 0x7f0154108cb0 2024-09-19T05:53:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.024+0000 7f015ae49640 1 -- 172.21.15.5:0/3279007749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015419be30 con 0x7f0154108cb0 2024-09-19T05:53:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.024+0000 7f0151ffb640 1 -- 172.21.15.5:0/3279007749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f014403a390 con 0x7f0154108cb0 2024-09-19T05:53:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.024+0000 7f015ae49640 1 -- 172.21.15.5:0/3279007749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015419eb20 con 0x7f0154108cb0 2024-09-19T05:53:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.024+0000 7f0151ffb640 1 -- 172.21.15.5:0/3279007749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f014403a580 con 0x7f0154108cb0 2024-09-19T05:53:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.024+0000 7f0151ffb640 1 -- 172.21.15.5:0/3279007749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0144044b50 con 0x7f0154108cb0 2024-09-19T05:53:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.025+0000 7f015ae49640 1 -- 172.21.15.5:0/3279007749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015410bda0 con 0x7f0154108cb0 2024-09-19T05:53:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.027+0000 7f0151ffb640 1 -- 172.21.15.5:0/3279007749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0144032d60 con 0x7f0154108cb0 2024-09-19T05:53:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.087+0000 7f015ae49640 1 -- 172.21.15.5:0/3279007749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f015419ef10 con 0x7f0154108cb0 2024-09-19T05:53:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.088+0000 7f0151ffb640 1 -- 172.21.15.5:0/3279007749 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f014403aa00 con 0x7f0154108cb0 2024-09-19T05:53:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.088+0000 7f015ae49640 1 -- 172.21.15.5:0/3279007749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 msgr2=0x7f015419b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:53:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.088+0000 7f015ae49640 1 --2- 172.21.15.5:0/3279007749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 0x7f015419b660 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f01440097f0 tx=0x7f0144030d00 comp rx=0 tx=0).stop 2024-09-19T05:53:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.089+0000 7f015ae49640 1 -- 172.21.15.5:0/3279007749 shutdown_connections 2024-09-19T05:53:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.089+0000 7f015ae49640 1 --2- 172.21.15.5:0/3279007749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0154108cb0 0x7f015419b660 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:53:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.089+0000 7f015ae49640 1 -- 172.21.15.5:0/3279007749 >> 172.21.15.5:0/3279007749 conn(0x7f01541013e0 msgr2=0x7f0154101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:53:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.089+0000 7f015ae49640 1 -- 172.21.15.5:0/3279007749 shutdown_connections 2024-09-19T05:53:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:54.089+0000 7f015ae49640 1 -- 172.21.15.5:0/3279007749 wait complete. 2024-09-19T05:53:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:Restarting the monitor... 2024-09-19T05:53:54.955 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:54 smithi005 systemd[1]: Stopping Ceph mon.a for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T05:53:55.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:54 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a[30584]: 2024-09-19T05:53:54.953+0000 7f310fd61640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:53:55.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:54 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a[30584]: 2024-09-19T05:53:54.953+0000 7f310fd61640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T05:53:55.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:55 smithi005 podman[30831]: 2024-09-19 05:53:55.158219447 +0000 UTC m=+0.302594188 container died aa23883239efe4713917940c14f15fb5bf9ed6f26fc34970741204fdfb8490fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T05:53:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:57 smithi005 podman[30831]: 2024-09-19 05:53:57.176175843 +0000 UTC m=+2.320550616 container cleanup aa23883239efe4713917940c14f15fb5bf9ed6f26fc34970741204fdfb8490fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T05:53:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:57 smithi005 bash[30831]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a 2024-09-19T05:53:57.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:57 smithi005 podman[30843]: 2024-09-19 05:53:57.39364925 +0000 UTC m=+2.231222755 container remove aa23883239efe4713917940c14f15fb5bf9ed6f26fc34970741204fdfb8490fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:53:57.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:57 smithi005 systemd[1]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T05:53:57.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:57 smithi005 systemd[1]: Stopped Ceph mon.a for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T05:53:57.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:57 smithi005 systemd[1]: Starting Ceph mon.a for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T05:53:58.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:57 smithi005 podman[30936]: 2024-09-19 05:53:57.819332792 +0000 UTC m=+0.113038564 container create 550e96d2d6843dafb63080c6849096e67598494006cedfd54fdd2e2d32350d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T05:53:58.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:57 smithi005 podman[30936]: 2024-09-19 05:53:57.722591598 +0000 UTC m=+0.016297371 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:53:58.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 podman[30936]: 2024-09-19 05:53:58.063393367 +0000 UTC m=+0.357099152 container init 550e96d2d6843dafb63080c6849096e67598494006cedfd54fdd2e2d32350d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T05:53:58.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 podman[30936]: 2024-09-19 05:53:58.066778431 +0000 UTC m=+0.360484204 container start 550e96d2d6843dafb63080c6849096e67598494006cedfd54fdd2e2d32350d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T05:53:58.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T05:53:58.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T05:53:58.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: pidfile_write: ignore empty --pid-file 2024-09-19T05:53:58.185 INFO:teuthology.orchestra.run.smithi005.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T05:53:58.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: load: jerasure load: lrc 2024-09-19T05:53:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: RocksDB version: 7.9.2 2024-09-19T05:53:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Git sha 0 2024-09-19T05:53:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T05:53:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: DB SUMMARY 2024-09-19T05:53:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: DB Session ID: 2DDEY723613W4X6T7GK1 2024-09-19T05:53:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: CURRENT file: CURRENT 2024-09-19T05:53:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T05:53:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T05:53:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T05:53:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105478 ; 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.error_if_exists: 0 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.create_if_missing: 0 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.paranoid_checks: 1 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.env: 0x558126fdc940 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.info_log: 0x5581295476e0 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.statistics: (nil) 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.use_fsync: 0 2024-09-19T05:53:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_log_file_size: 0 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.allow_fallocate: 1 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.use_direct_reads: 0 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.db_log_dir: 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.wal_dir: 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T05:53:58.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T05:53:58.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T05:53:58.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T05:53:58.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T05:53:58.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T05:53:58.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T05:53:58.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T05:53:58.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.write_buffer_manager: 0x558129556dc0 2024-09-19T05:53:58.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T05:53:58.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T05:53:58.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T05:53:58.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T05:53:58.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T05:53:58.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T05:53:58.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T05:53:58.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T05:53:58.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.unordered_write: 0 2024-09-19T05:53:58.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T05:53:58.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T05:53:58.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T05:53:58.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T05:53:58.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.row_cache: None 2024-09-19T05:53:58.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.wal_filter: None 2024-09-19T05:53:58.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T05:53:58.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T05:53:58.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.two_write_queues: 0 2024-09-19T05:53:58.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T05:53:58.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.wal_compression: 0 2024-09-19T05:53:58.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.atomic_flush: 0 2024-09-19T05:53:58.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T05:53:58.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T05:53:58.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T05:53:58.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.log_readahead_size: 0 2024-09-19T05:53:58.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_background_jobs: 2 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_background_compactions: -1 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_subcompactions: 1 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T05:53:58.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_open_files: -1 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_background_flushes: -1 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Compression algorithms supported: 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: kZSTD supported: 0 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: kXpressCompression supported: 0 2024-09-19T05:53:58.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: kBZip2Compression supported: 0 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: kLZ4Compression supported: 1 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: kZlibCompression supported: 1 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: kSnappyCompression supported: 1 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.merge_operator: 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_filter: None 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T05:53:58.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558129547320) 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks: 1 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_top_level_index_and_filter: 1 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_type: 0 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_index_type: 0 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_shortening: 1 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout: checksum: 4 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout: no_block_cache: 0 2024-09-19T05:53:58.390 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache: 0x55812953b1f0 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_name: BinnedLRUCache 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_options: 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: capacity : 536870912 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_shard_bits : 4 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: strict_capacity_limit : 0 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: high_pri_pool_ratio: 0.000 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_compressed: (nil) 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: persistent_cache: (nil) 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size: 4096 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size_deviation: 10 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_restart_interval: 16 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_block_restart_interval: 1 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: metadata_block_size: 4096 2024-09-19T05:53:58.391 INFO:journalctl@ceph.mon.a.smithi005.stdout: partition_filters: 0 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout: use_delta_encoding: 1 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout: filter_policy: bloomfilter 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout: whole_key_filtering: 1 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout: verify_compression: 0 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout: read_amp_bytes_per_bit: 0 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout: format_version: 5 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_index_compression: 1 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_align: 0 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout: max_auto_readahead_size: 262144 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout: prepopulate_block_cache: 0 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout: initial_auto_readahead_size: 8192 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T05:53:58.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compression: NoCompression 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.num_levels: 7 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T05:53:58.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T05:53:58.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T05:53:58.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T05:53:58.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.inplace_update_support: 0 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.bloom_locality: 0 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.max_successive_merges: 0 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T05:53:58.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.ttl: 2592000 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.enable_blob_files: false 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.min_blob_size: 0 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T05:53:58.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3be42137-040a-4bd6-b5d1-5af5d682b8fb 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726725238094278, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726725238095327, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100472, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98366, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14373, "raw_average_key_size": 49, "raw_value_size": 90630, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726725238, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3be42137-040a-4bd6-b5d1-5af5d682b8fb", "db_session_id": "2DDEY723613W4X6T7GK1", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726725238095426, "job": 1, "event": "recovery_finished"} 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T05:53:58.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558129568e00 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: DB pointer 0x558129664000 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** DB Stats ** 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-09-19T05:53:58.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: L0 2/0 99.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: Sum 2/0 99.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 147.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-19T05:53:58.401 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative compaction: 0.00 GB write, 29.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval compaction: 0.00 GB write, 29.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache BinnedLRUCache@0x55812953b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: starting mon.a rank 0 at public addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] at bind addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:53:58.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: mon.a@-1(???) e1 preinit fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: mon.a@-1(???).mds e1 new map 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: mon.a@-1(???).mds e1 print_map 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout: e1 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout: btime 2024-09-19T05:53:51:453912+0000 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout: legacy client fscid: -1 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout: No filesystems configured 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:53:58.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:53:58.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 ceph-mon[30955]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T05:53:58.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 bash[30936]: 550e96d2d6843dafb63080c6849096e67598494006cedfd54fdd2e2d32350d59 2024-09-19T05:53:58.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:58 smithi005 systemd[1]: Started Ceph mon.a for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T05:53:59.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:59 smithi005 ceph-mon[30955]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T05:53:59.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:59 smithi005 ceph-mon[30955]: monmap epoch 1 2024-09-19T05:53:59.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:59 smithi005 ceph-mon[30955]: fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:53:59.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:59 smithi005 ceph-mon[30955]: last_changed 2024-09-19T05:53:38.259181+0000 2024-09-19T05:53:59.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:59 smithi005 ceph-mon[30955]: created 2024-09-19T05:53:38.259181+0000 2024-09-19T05:53:59.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:59 smithi005 ceph-mon[30955]: min_mon_release 19 (squid) 2024-09-19T05:53:59.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:59 smithi005 ceph-mon[30955]: election_strategy: 1 2024-09-19T05:53:59.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:59 smithi005 ceph-mon[30955]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-19T05:53:59.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:59 smithi005 ceph-mon[30955]: fsmap 2024-09-19T05:53:59.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:59 smithi005 ceph-mon[30955]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T05:53:59.272 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:59 smithi005 ceph-mon[30955]: mgrmap e1: no daemons active 2024-09-19T05:53:59.272 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:53:59 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2324214788' entity='client.admin' 2024-09-19T05:53:59.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.633+0000 7fb845a46640 1 Processor -- start 2024-09-19T05:53:59.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.633+0000 7fb845a46640 1 -- start start 2024-09-19T05:53:59.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.633+0000 7fb845a46640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 0x7fb84010a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:53:59.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.633+0000 7fb845a46640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb84007be20 con 0x7fb8401080c0 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.634+0000 7fb83effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 0x7fb84010a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.634+0000 7fb83effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 0x7fb84010a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45662/0 (socket says 172.21.15.5:45662) 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.634+0000 7fb83effd640 1 -- 172.21.15.5:0/1894295081 learned_addr learned my addr 172.21.15.5:0/1894295081 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.634+0000 7fb83effd640 1 -- 172.21.15.5:0/1894295081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84010a9f0 con 0x7fb8401080c0 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.636+0000 7fb83effd640 1 --2- 172.21.15.5:0/1894295081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 0x7fb84010a4b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb838009e80 tx=0x7fb83802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df03a22f533d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.636+0000 7fb83dffb640 1 -- 172.21.15.5:0/1894295081 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb838036070 con 0x7fb8401080c0 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.636+0000 7fb83dffb640 1 -- 172.21.15.5:0/1894295081 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb838031890 con 0x7fb8401080c0 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.636+0000 7fb83dffb640 1 -- 172.21.15.5:0/1894295081 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb838007980 con 0x7fb8401080c0 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.637+0000 7fb845a46640 1 -- 172.21.15.5:0/1894295081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 msgr2=0x7fb84010a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.637+0000 7fb845a46640 1 --2- 172.21.15.5:0/1894295081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 0x7fb84010a4b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb838009e80 tx=0x7fb83802a7a0 comp rx=0 tx=0).stop 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.637+0000 7fb845a46640 1 -- 172.21.15.5:0/1894295081 shutdown_connections 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.637+0000 7fb845a46640 1 --2- 172.21.15.5:0/1894295081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 0x7fb84010a4b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:53:59.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.637+0000 7fb845a46640 1 -- 172.21.15.5:0/1894295081 >> 172.21.15.5:0/1894295081 conn(0x7fb840101d00 msgr2=0x7fb840104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:53:59.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.637+0000 7fb845a46640 1 -- 172.21.15.5:0/1894295081 shutdown_connections 2024-09-19T05:53:59.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.637+0000 7fb845a46640 1 -- 172.21.15.5:0/1894295081 wait complete. 2024-09-19T05:53:59.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.638+0000 7fb845a46640 1 Processor -- start 2024-09-19T05:53:59.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.638+0000 7fb845a46640 1 -- start start 2024-09-19T05:53:59.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.638+0000 7fb845a46640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 0x7fb840107940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:53:59.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.638+0000 7fb845a46640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb84010b970 con 0x7fb8401080c0 2024-09-19T05:53:59.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.638+0000 7fb83effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 0x7fb840107940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:53:59.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.638+0000 7fb83effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 0x7fb840107940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45672/0 (socket says 172.21.15.5:45672) 2024-09-19T05:53:59.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.638+0000 7fb83effd640 1 -- 172.21.15.5:0/2324214788 learned_addr learned my addr 172.21.15.5:0/2324214788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:53:59.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.638+0000 7fb83effd640 1 -- 172.21.15.5:0/2324214788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb840107e80 con 0x7fb8401080c0 2024-09-19T05:53:59.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.639+0000 7fb83effd640 1 --2- 172.21.15.5:0/2324214788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 0x7fb840107940 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb838031c70 tx=0x7fb838031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:53:59.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.639+0000 7fb844a44640 1 -- 172.21.15.5:0/2324214788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb838041020 con 0x7fb8401080c0 2024-09-19T05:53:59.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.639+0000 7fb844a44640 1 -- 172.21.15.5:0/2324214788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb83803c900 con 0x7fb8401080c0 2024-09-19T05:53:59.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.639+0000 7fb845a46640 1 -- 172.21.15.5:0/2324214788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb840106070 con 0x7fb8401080c0 2024-09-19T05:53:59.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.639+0000 7fb844a44640 1 -- 172.21.15.5:0/2324214788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb838036040 con 0x7fb8401080c0 2024-09-19T05:53:59.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.639+0000 7fb845a46640 1 -- 172.21.15.5:0/2324214788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8401065b0 con 0x7fb8401080c0 2024-09-19T05:53:59.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.639+0000 7fb844a44640 1 -- 172.21.15.5:0/2324214788 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb838044020 con 0x7fb8401080c0 2024-09-19T05:53:59.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.639+0000 7fb844a44640 1 -- 172.21.15.5:0/2324214788 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb838046c50 con 0x7fb8401080c0 2024-09-19T05:53:59.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.640+0000 7fb845a46640 1 -- 172.21.15.5:0/2324214788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb840108520 con 0x7fb8401080c0 2024-09-19T05:53:59.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.642+0000 7fb844a44640 1 -- 172.21.15.5:0/2324214788 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb83804b020 con 0x7fb8401080c0 2024-09-19T05:53:59.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.702+0000 7fb845a46640 1 -- 172.21.15.5:0/2324214788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fb840102fc0 con 0x7fb8401080c0 2024-09-19T05:53:59.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.704+0000 7fb844a44640 1 -- 172.21.15.5:0/2324214788 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fb83803cab0 con 0x7fb8401080c0 2024-09-19T05:53:59.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.704+0000 7fb845a46640 1 -- 172.21.15.5:0/2324214788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 msgr2=0x7fb840107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:53:59.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.704+0000 7fb845a46640 1 --2- 172.21.15.5:0/2324214788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 0x7fb840107940 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb838031c70 tx=0x7fb838031ca0 comp rx=0 tx=0).stop 2024-09-19T05:53:59.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.704+0000 7fb845a46640 1 -- 172.21.15.5:0/2324214788 shutdown_connections 2024-09-19T05:53:59.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.704+0000 7fb845a46640 1 --2- 172.21.15.5:0/2324214788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8401080c0 0x7fb840107940 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:53:59.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.704+0000 7fb845a46640 1 -- 172.21.15.5:0/2324214788 >> 172.21.15.5:0/2324214788 conn(0x7fb840101d00 msgr2=0x7fb8401028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:53:59.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.705+0000 7fb845a46640 1 -- 172.21.15.5:0/2324214788 shutdown_connections 2024-09-19T05:53:59.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:53:58.705+0000 7fb845a46640 1 -- 172.21.15.5:0/2324214788 wait complete. 2024-09-19T05:53:59.281 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T05:53:59.312 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:53:59.312 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mgr... 2024-09-19T05:53:59.313 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T05:53:59.313 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T05:53:59.579 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.x 2024-09-19T05:53:59.580 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T05:53:59.738 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6b5d55e8-764b-11ef-bceb-c7b262605968.target.wants/ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@.service. 2024-09-19T05:53:59.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:53:59 smithi005 systemd[1]: Starting Ceph mgr.x for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T05:54:00.276 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:53:59 smithi005 podman[31168]: 2024-09-19 05:53:59.963490871 +0000 UTC m=+0.108071969 container create 67f13a9e350e9d53f5be01555dc1d2c319be6bae6ecee696cc5d097ed887e459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T05:54:00.276 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:53:59 smithi005 podman[31168]: 2024-09-19 05:53:59.871471142 +0000 UTC m=+0.016052252 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:54:00.276 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:00 smithi005 podman[31168]: 2024-09-19 05:54:00.149030055 +0000 UTC m=+0.293611164 container init 67f13a9e350e9d53f5be01555dc1d2c319be6bae6ecee696cc5d097ed887e459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T05:54:00.276 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:00 smithi005 podman[31168]: 2024-09-19 05:54:00.151805735 +0000 UTC m=+0.296386836 container start 67f13a9e350e9d53f5be01555dc1d2c319be6bae6ecee696cc5d097ed887e459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T05:54:00.276 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:00 smithi005 bash[31168]: 67f13a9e350e9d53f5be01555dc1d2c319be6bae6ecee696cc5d097ed887e459 2024-09-19T05:54:00.290 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T05:54:00.290 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-19T05:54:00.298 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T05:54:00.298 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-19T05:54:00.298 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-19T05:54:00.298 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T05:54:00.307 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T05:54:00.307 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-19T05:54:00.315 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T05:54:00.315 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-19T05:54:00.315 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-19T05:54:00.316 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T05:54:00.316 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr to start... 2024-09-19T05:54:00.316 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr... 2024-09-19T05:54:00.551 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:00.273+0000 7f4b8bcb8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:54:00.551 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:00 smithi005 systemd[1]: Started Ceph mgr.x for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T05:54:00.551 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:00.342+0000 7f4b8bcb8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:54:00.551 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:00.500+0000 7f4b8bcb8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:54:00.835 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:00.573+0000 7f4b8bcb8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:54:00.835 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:00.730+0000 7f4b8bcb8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:54:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:00 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2178095233' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T05:54:01.129 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:00.895+0000 7f4b8bcb8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:54:01.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:01 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:01.228+0000 7f4b8bcb8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:54:01.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:01 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:01.291+0000 7f4b8bcb8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:54:01.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-19T05:54:01.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:54:01.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T05:54:01.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T05:54:01.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T05:54:01.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T05:54:01.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T05:54:01.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:01.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T05:54:01.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T05:54:01.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-19T05:54:01.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:54:01.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T05:54:01.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T05:54:01.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:54:01.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-19T05:54:01.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T05:54:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:54:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T05:54:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T05:54:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T05:54:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:54:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T05:54:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T05:54:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T05:54:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T05:54:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T05:54:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T05:54:01.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:01.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T05:54:01.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T05:54:01.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T05:54:01.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T05:54:01.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T05:54:01.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T05:54:01.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T05:54:01.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T05:54:01.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T05:54:01.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:01.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T05:54:01.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:54:01.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T05:53:51:453912+0000", 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T05:54:01.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T05:53:51.454524+0000", 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.749+0000 7fe66f86a640 1 Processor -- start 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.749+0000 7fe66f86a640 1 -- start start 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.749+0000 7fe66f86a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 0x7fe66810a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.749+0000 7fe66f86a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe668063490 con 0x7fe668108060 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.750+0000 7fe66d606640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 0x7fe66810a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:01.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.750+0000 7fe66d606640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 0x7fe66810a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45698/0 (socket says 172.21.15.5:45698) 2024-09-19T05:54:01.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.750+0000 7fe66d606640 1 -- 172.21.15.5:0/2201422348 learned_addr learned my addr 172.21.15.5:0/2201422348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:01.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.750+0000 7fe66d606640 1 -- 172.21.15.5:0/2201422348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66810a990 con 0x7fe668108060 2024-09-19T05:54:01.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.751+0000 7fe66d606640 1 --2- 172.21.15.5:0/2201422348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 0x7fe66810a450 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe664009820 tx=0x7fe66402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999c633a6dfe0785 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:01.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.751+0000 7fe65bfff640 1 -- 172.21.15.5:0/2201422348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe664037070 con 0x7fe668108060 2024-09-19T05:54:01.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.751+0000 7fe65bfff640 1 -- 172.21.15.5:0/2201422348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe6640328d0 con 0x7fe668108060 2024-09-19T05:54:01.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.751+0000 7fe65bfff640 1 -- 172.21.15.5:0/2201422348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6640307d0 con 0x7fe668108060 2024-09-19T05:54:01.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.752+0000 7fe66f86a640 1 -- 172.21.15.5:0/2201422348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 msgr2=0x7fe66810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:01.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.752+0000 7fe66f86a640 1 --2- 172.21.15.5:0/2201422348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 0x7fe66810a450 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe664009820 tx=0x7fe66402a270 comp rx=0 tx=0).stop 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.752+0000 7fe66f86a640 1 -- 172.21.15.5:0/2201422348 shutdown_connections 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.752+0000 7fe66f86a640 1 --2- 172.21.15.5:0/2201422348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 0x7fe66810a450 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.752+0000 7fe66f86a640 1 -- 172.21.15.5:0/2201422348 >> 172.21.15.5:0/2201422348 conn(0x7fe668101c00 msgr2=0x7fe668104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.752+0000 7fe66f86a640 1 -- 172.21.15.5:0/2201422348 shutdown_connections 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.752+0000 7fe66f86a640 1 -- 172.21.15.5:0/2201422348 wait complete. 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.753+0000 7fe66f86a640 1 Processor -- start 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.753+0000 7fe66f86a640 1 -- start start 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.753+0000 7fe66f86a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 0x7fe6681078e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.753+0000 7fe66f86a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe66810b910 con 0x7fe668108060 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.753+0000 7fe66d606640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 0x7fe6681078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.753+0000 7fe66d606640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 0x7fe6681078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45712/0 (socket says 172.21.15.5:45712) 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.753+0000 7fe66d606640 1 -- 172.21.15.5:0/2178095233 learned_addr learned my addr 172.21.15.5:0/2178095233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:01.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.753+0000 7fe66d606640 1 -- 172.21.15.5:0/2178095233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe668107e20 con 0x7fe668108060 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.753+0000 7fe66d606640 1 --2- 172.21.15.5:0/2178095233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 0x7fe6681078e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe664032e10 tx=0x7fe664032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.754+0000 7fe65a7fc640 1 -- 172.21.15.5:0/2178095233 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe664042070 con 0x7fe668108060 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.754+0000 7fe65a7fc640 1 -- 172.21.15.5:0/2178095233 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe664030c80 con 0x7fe668108060 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.754+0000 7fe66f86a640 1 -- 172.21.15.5:0/2178095233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe668106010 con 0x7fe668108060 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.754+0000 7fe66f86a640 1 -- 172.21.15.5:0/2178095233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe668106550 con 0x7fe668108060 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.754+0000 7fe65a7fc640 1 -- 172.21.15.5:0/2178095233 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe664037040 con 0x7fe668108060 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.754+0000 7fe65a7fc640 1 -- 172.21.15.5:0/2178095233 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe664045070 con 0x7fe668108060 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.754+0000 7fe65a7fc640 1 -- 172.21.15.5:0/2178095233 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe664047a50 con 0x7fe668108060 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.755+0000 7fe66f86a640 1 -- 172.21.15.5:0/2178095233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64c0052c0 con 0x7fe668108060 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.757+0000 7fe65a7fc640 1 -- 172.21.15.5:0/2178095233 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe66403c6f0 con 0x7fe668108060 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.833+0000 7fe66f86a640 1 -- 172.21.15.5:0/2178095233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe64c005880 con 0x7fe668108060 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.834+0000 7fe65a7fc640 1 -- 172.21.15.5:0/2178095233 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fe66403c8d0 con 0x7fe668108060 2024-09-19T05:54:01.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.835+0000 7fe66f86a640 1 -- 172.21.15.5:0/2178095233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 msgr2=0x7fe6681078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:01.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.835+0000 7fe66f86a640 1 --2- 172.21.15.5:0/2178095233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 0x7fe6681078e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe664032e10 tx=0x7fe664032e40 comp rx=0 tx=0).stop 2024-09-19T05:54:01.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.835+0000 7fe66f86a640 1 -- 172.21.15.5:0/2178095233 shutdown_connections 2024-09-19T05:54:01.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.835+0000 7fe66f86a640 1 --2- 172.21.15.5:0/2178095233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe668108060 0x7fe6681078e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:01.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.835+0000 7fe66f86a640 1 -- 172.21.15.5:0/2178095233 >> 172.21.15.5:0/2178095233 conn(0x7fe668101c00 msgr2=0x7fe6681027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:01.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.835+0000 7fe66f86a640 1 -- 172.21.15.5:0/2178095233 shutdown_connections 2024-09-19T05:54:01.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:00.835+0000 7fe66f86a640 1 -- 172.21.15.5:0/2178095233 wait complete. 2024-09-19T05:54:01.437 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (1/15)... 2024-09-19T05:54:01.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:01 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:01.487+0000 7f4b8bcb8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:54:02.781 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:02 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:02.410+0000 7f4b8bcb8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:54:02.781 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:02 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:02.480+0000 7f4b8bcb8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:54:03.371 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:03 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:03.244+0000 7f4b8bcb8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:54:03.371 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:03 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:03.369+0000 7f4b8bcb8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:54:03.629 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:03 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:03.526+0000 7f4b8bcb8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:54:03.629 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:03 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:03.592+0000 7f4b8bcb8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:54:03.928 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:03 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:03.752+0000 7f4b8bcb8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:54:03.928 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:03 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:03.826+0000 7f4b8bcb8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:54:03.928 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:03 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:03.893+0000 7f4b8bcb8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:54:04.214 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:03 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/807338248' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T05:54:04.214 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:04 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:04.073+0000 7f4b8bcb8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:54:04.214 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:04 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:04.141+0000 7f4b8bcb8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:54:04.214 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:04 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:04.212+0000 7f4b8bcb8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:54:04.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:04 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:04.299+0000 7f4b8bcb8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:54:04.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:04 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:04.361+0000 7f4b8bcb8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:54:04.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:04 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:04.423+0000 7f4b8bcb8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:54:04.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-19T05:54:04.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:54:04.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T05:54:04.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T05:54:04.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T05:54:04.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T05:54:04.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T05:54:04.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:04.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T05:54:04.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T05:54:04.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-19T05:54:04.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:54:04.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T05:54:04.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T05:54:04.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:54:04.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-19T05:54:04.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T05:54:04.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:54:04.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T05:54:04.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T05:54:04.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:04.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T05:54:04.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:54:04.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T05:54:04.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T05:53:51:453912+0000", 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T05:54:04.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T05:53:51.454524+0000", 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.840+0000 7fa28df64640 1 Processor -- start 2024-09-19T05:54:04.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.840+0000 7fa28df64640 1 -- start start 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.840+0000 7fa28df64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 0x7fa28810a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.840+0000 7fa28df64640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa288063490 con 0x7fa288108060 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.840+0000 7fa2877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 0x7fa28810a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.840+0000 7fa2877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 0x7fa28810a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45728/0 (socket says 172.21.15.5:45728) 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.840+0000 7fa2877fe640 1 -- 172.21.15.5:0/4088713967 learned_addr learned my addr 172.21.15.5:0/4088713967 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.841+0000 7fa2877fe640 1 -- 172.21.15.5:0/4088713967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28810a990 con 0x7fa288108060 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.841+0000 7fa2877fe640 1 --2- 172.21.15.5:0/4088713967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 0x7fa28810a450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa278009820 tx=0x7fa27802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e3477d8f4a3287 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.841+0000 7fa2867fc640 1 -- 172.21.15.5:0/4088713967 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa278037070 con 0x7fa288108060 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.842+0000 7fa2867fc640 1 -- 172.21.15.5:0/4088713967 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa2780328d0 con 0x7fa288108060 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.842+0000 7fa2867fc640 1 -- 172.21.15.5:0/4088713967 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2780307d0 con 0x7fa288108060 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.842+0000 7fa28df64640 1 -- 172.21.15.5:0/4088713967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 msgr2=0x7fa28810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.842+0000 7fa28df64640 1 --2- 172.21.15.5:0/4088713967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 0x7fa28810a450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa278009820 tx=0x7fa27802a270 comp rx=0 tx=0).stop 2024-09-19T05:54:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.842+0000 7fa28df64640 1 -- 172.21.15.5:0/4088713967 shutdown_connections 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.842+0000 7fa28df64640 1 --2- 172.21.15.5:0/4088713967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 0x7fa28810a450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.842+0000 7fa28df64640 1 -- 172.21.15.5:0/4088713967 >> 172.21.15.5:0/4088713967 conn(0x7fa288101c00 msgr2=0x7fa288104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.842+0000 7fa28df64640 1 -- 172.21.15.5:0/4088713967 shutdown_connections 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.842+0000 7fa28df64640 1 -- 172.21.15.5:0/4088713967 wait complete. 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.843+0000 7fa28df64640 1 Processor -- start 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.843+0000 7fa28df64640 1 -- start start 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.843+0000 7fa28df64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 0x7fa2881078e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.843+0000 7fa28df64640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810b910 con 0x7fa288108060 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.843+0000 7fa2877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 0x7fa2881078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.844+0000 7fa2877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 0x7fa2881078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45740/0 (socket says 172.21.15.5:45740) 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.844+0000 7fa2877fe640 1 -- 172.21.15.5:0/807338248 learned_addr learned my addr 172.21.15.5:0/807338248 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.844+0000 7fa2877fe640 1 -- 172.21.15.5:0/807338248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa288107e20 con 0x7fa288108060 2024-09-19T05:54:04.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.844+0000 7fa2877fe640 1 --2- 172.21.15.5:0/807338248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 0x7fa2881078e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2780097f0 tx=0x7fa278030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.844+0000 7fa284ff9640 1 -- 172.21.15.5:0/807338248 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa278037040 con 0x7fa288108060 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.844+0000 7fa284ff9640 1 -- 172.21.15.5:0/807338248 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa278032bb0 con 0x7fa288108060 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.844+0000 7fa28df64640 1 -- 172.21.15.5:0/807338248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa288106010 con 0x7fa288108060 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.844+0000 7fa284ff9640 1 -- 172.21.15.5:0/807338248 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa27803a390 con 0x7fa288108060 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.844+0000 7fa28df64640 1 -- 172.21.15.5:0/807338248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa288106550 con 0x7fa288108060 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.845+0000 7fa284ff9640 1 -- 172.21.15.5:0/807338248 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa27803a580 con 0x7fa288108060 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.845+0000 7fa284ff9640 1 -- 172.21.15.5:0/807338248 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa278044ba0 con 0x7fa288108060 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.845+0000 7fa28df64640 1 -- 172.21.15.5:0/807338248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2881084c0 con 0x7fa288108060 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.849+0000 7fa284ff9640 1 -- 172.21.15.5:0/807338248 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa278044400 con 0x7fa288108060 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.912+0000 7fa28df64640 1 -- 172.21.15.5:0/807338248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa288106890 con 0x7fa288108060 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.912+0000 7fa284ff9640 1 -- 172.21.15.5:0/807338248 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fa278032d60 con 0x7fa288108060 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.913+0000 7fa28df64640 1 -- 172.21.15.5:0/807338248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 msgr2=0x7fa2881078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.913+0000 7fa28df64640 1 --2- 172.21.15.5:0/807338248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 0x7fa2881078e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2780097f0 tx=0x7fa278030d00 comp rx=0 tx=0).stop 2024-09-19T05:54:04.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.913+0000 7fa28df64640 1 -- 172.21.15.5:0/807338248 shutdown_connections 2024-09-19T05:54:04.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.913+0000 7fa28df64640 1 --2- 172.21.15.5:0/807338248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa288108060 0x7fa2881078e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:04.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.913+0000 7fa28df64640 1 -- 172.21.15.5:0/807338248 >> 172.21.15.5:0/807338248 conn(0x7fa288101c00 msgr2=0x7fa2881027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:04.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.914+0000 7fa28df64640 1 -- 172.21.15.5:0/807338248 shutdown_connections 2024-09-19T05:54:04.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:03.914+0000 7fa28df64640 1 -- 172.21.15.5:0/807338248 wait complete. 2024-09-19T05:54:04.527 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (2/15)... 2024-09-19T05:54:05.629 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:05.254+0000 7f4b8bcb8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:54:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: Activating manager daemon x 2024-09-19T05:54:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: mgrmap e2: x(active, starting, since 0.00299342s) 2024-09-19T05:54:05.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: from='mgr.14100 172.21.15.5:0/3285484618' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T05:54:05.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: from='mgr.14100 172.21.15.5:0/3285484618' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T05:54:05.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: from='mgr.14100 172.21.15.5:0/3285484618' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T05:54:05.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: from='mgr.14100 172.21.15.5:0/3285484618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:54:05.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: from='mgr.14100 172.21.15.5:0/3285484618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T05:54:05.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: Manager daemon x is now available 2024-09-19T05:54:05.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: from='mgr.14100 172.21.15.5:0/3285484618' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:54:05.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: from='mgr.14100 172.21.15.5:0/3285484618' entity='mgr.x' 2024-09-19T05:54:05.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: from='mgr.14100 172.21.15.5:0/3285484618' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:54:05.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: from='mgr.14100 172.21.15.5:0/3285484618' entity='mgr.x' 2024-09-19T05:54:05.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:05 smithi005 ceph-mon[30955]: from='mgr.14100 172.21.15.5:0/3285484618' entity='mgr.x' 2024-09-19T05:54:07.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:07 smithi005 ceph-mon[30955]: mgrmap e3: x(active, since 1.00708s) 2024-09-19T05:54:07.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:07 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2372573136' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T05:54:07.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-19T05:54:07.734 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T05:54:07.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:54:07.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-19T05:54:07.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T05:54:07.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:54:07.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T05:54:07.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T05:54:07.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:07.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T05:54:07.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:54:07.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T05:54:07.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T05:54:07.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T05:54:07.744 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T05:54:07.744 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T05:54:07.744 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T05:54:07.744 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:07.744 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T05:53:51:453912+0000", 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T05:54:07.745 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:54:07.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T05:53:51.454524+0000", 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.921+0000 7f3bc9e46640 1 Processor -- start 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.921+0000 7f3bc9e46640 1 -- start start 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.921+0000 7f3bc9e46640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 0x7f3bc407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.921+0000 7f3bc9e46640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc407c8a0 con 0x7f3bc407de70 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.922+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 0x7f3bc407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.922+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 0x7f3bc407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45022/0 (socket says 172.21.15.5:45022) 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.922+0000 7f3bc37fe640 1 -- 172.21.15.5:0/2714098161 learned_addr learned my addr 172.21.15.5:0/2714098161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.922+0000 7f3bc37fe640 1 -- 172.21.15.5:0/2714098161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc407ca20 con 0x7f3bc407de70 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.923+0000 7f3bc37fe640 1 --2- 172.21.15.5:0/2714098161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 0x7f3bc407c2d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009820 tx=0x7f3bb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504d386d3a336f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:07.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.923+0000 7f3bc27fc640 1 -- 172.21.15.5:0/2714098161 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bb4037070 con 0x7f3bc407de70 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.923+0000 7f3bc27fc640 1 -- 172.21.15.5:0/2714098161 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3bb40328d0 con 0x7f3bc407de70 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.923+0000 7f3bc27fc640 1 -- 172.21.15.5:0/2714098161 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bb40307d0 con 0x7f3bc407de70 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.924+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2714098161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 msgr2=0x7f3bc407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.924+0000 7f3bc9e46640 1 --2- 172.21.15.5:0/2714098161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 0x7f3bc407c2d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009820 tx=0x7f3bb402a270 comp rx=0 tx=0).stop 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.924+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2714098161 shutdown_connections 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.924+0000 7f3bc9e46640 1 --2- 172.21.15.5:0/2714098161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 0x7f3bc407c2d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.924+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2714098161 >> 172.21.15.5:0/2714098161 conn(0x7f3bc4102d50 msgr2=0x7f3bc4105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.924+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2714098161 shutdown_connections 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.924+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2714098161 wait complete. 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.925+0000 7f3bc9e46640 1 Processor -- start 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.925+0000 7f3bc9e46640 1 -- start start 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.925+0000 7f3bc9e46640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 0x7f3bc419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.925+0000 7f3bc9e46640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc410db90 con 0x7f3bc407de70 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.925+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 0x7f3bc419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.925+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 0x7f3bc419d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45032/0 (socket says 172.21.15.5:45032) 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.925+0000 7f3bc37fe640 1 -- 172.21.15.5:0/2372573136 learned_addr learned my addr 172.21.15.5:0/2372573136 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.925+0000 7f3bc37fe640 1 -- 172.21.15.5:0/2372573136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc419dd00 con 0x7f3bc407de70 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.926+0000 7f3bc37fe640 1 --2- 172.21.15.5:0/2372573136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 0x7f3bc419d7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4030520 tx=0x7f3bb4032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.926+0000 7f3bc0ff9640 1 -- 172.21.15.5:0/2372573136 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bb4042070 con 0x7f3bc407de70 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.926+0000 7f3bc0ff9640 1 -- 172.21.15.5:0/2372573136 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3bb4030cc0 con 0x7f3bc407de70 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.926+0000 7f3bc0ff9640 1 -- 172.21.15.5:0/2372573136 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bb4037040 con 0x7f3bc407de70 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.926+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2372573136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc419df90 con 0x7f3bc407de70 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.926+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2372573136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc41a0c80 con 0x7f3bc407de70 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.927+0000 7f3bc0ff9640 1 -- 172.21.15.5:0/2372573136 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7f3bb4045070 con 0x7f3bc407de70 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.927+0000 7f3bc0ff9640 1 --2- 172.21.15.5:0/2372573136 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f3bac0390a0 0x7f3bac03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.927+0000 7f3bc0ff9640 1 -- 172.21.15.5:0/2372573136 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3bb406c0b0 con 0x7f3bc407de70 2024-09-19T05:54:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.927+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2372573136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc407c350 con 0x7f3bc407de70 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.927+0000 7f3bc2ffd640 1 --2- 172.21.15.5:0/2372573136 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f3bac0390a0 0x7f3bac03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.928+0000 7f3bc2ffd640 1 --2- 172.21.15.5:0/2372573136 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f3bac0390a0 0x7f3bac03b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80098c0 tx=0x7f3bb8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:06.932+0000 7f3bc0ff9640 1 -- 172.21.15.5:0/2372573136 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3bb4045360 con 0x7f3bc407de70 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:07.174+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2372573136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3bc41a0e50 con 0x7f3bc407de70 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:07.174+0000 7f3bc0ff9640 1 -- 172.21.15.5:0/2372573136 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f3bb403ca90 con 0x7f3bc407de70 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:07.178+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2372573136 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f3bac0390a0 msgr2=0x7f3bac03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:07.178+0000 7f3bc9e46640 1 --2- 172.21.15.5:0/2372573136 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f3bac0390a0 0x7f3bac03b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80098c0 tx=0x7f3bb8006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:07.178+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2372573136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 msgr2=0x7f3bc419d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:07.178+0000 7f3bc9e46640 1 --2- 172.21.15.5:0/2372573136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 0x7f3bc419d7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4030520 tx=0x7f3bb4032dd0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:07.178+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2372573136 shutdown_connections 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:07.178+0000 7f3bc9e46640 1 --2- 172.21.15.5:0/2372573136 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f3bac0390a0 0x7f3bac03b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:07.178+0000 7f3bc9e46640 1 --2- 172.21.15.5:0/2372573136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc407de70 0x7f3bc419d7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:07.178+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2372573136 >> 172.21.15.5:0/2372573136 conn(0x7f3bc4102d50 msgr2=0x7f3bc4103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:07.178+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2372573136 shutdown_connections 2024-09-19T05:54:07.751 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:07.178+0000 7f3bc9e46640 1 -- 172.21.15.5:0/2372573136 wait complete. 2024-09-19T05:54:07.751 INFO:teuthology.orchestra.run.smithi005.stdout:mgr is available 2024-09-19T05:54:08.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:08 smithi005 ceph-mon[30955]: mgrmap e4: x(active, since 2s) 2024-09-19T05:54:08.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-19T05:54:08.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T05:54:08.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:54:08.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-19T05:54:08.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T05:54:08.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T05:54:08.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.152+0000 7f4e74deb640 1 Processor -- start 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.152+0000 7f4e74deb640 1 -- start start 2024-09-19T05:54:08.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.152+0000 7f4e74deb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 0x7f4e701068f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:08.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.153+0000 7f4e74deb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70106ec0 con 0x7f4e701064f0 2024-09-19T05:54:08.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.153+0000 7f4e6e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 0x7f4e701068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:08.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.153+0000 7f4e6e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 0x7f4e701068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45046/0 (socket says 172.21.15.5:45046) 2024-09-19T05:54:08.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.153+0000 7f4e6e59c640 1 -- 172.21.15.5:0/1006447304 learned_addr learned my addr 172.21.15.5:0/1006447304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:08.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.153+0000 7f4e6e59c640 1 -- 172.21.15.5:0/1006447304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e701076f0 con 0x7f4e701064f0 2024-09-19T05:54:08.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.154+0000 7f4e6e59c640 1 --2- 172.21.15.5:0/1006447304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 0x7f4e701068f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009820 tx=0x7f4e6002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6370d4db7bb3896 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:08.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.154+0000 7f4e6d59a640 1 -- 172.21.15.5:0/1006447304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e60037070 con 0x7f4e701064f0 2024-09-19T05:54:08.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.154+0000 7f4e6d59a640 1 -- 172.21.15.5:0/1006447304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4e600328d0 con 0x7f4e701064f0 2024-09-19T05:54:08.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.154+0000 7f4e6d59a640 1 -- 172.21.15.5:0/1006447304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e600307d0 con 0x7f4e701064f0 2024-09-19T05:54:08.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.155+0000 7f4e74deb640 1 -- 172.21.15.5:0/1006447304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 msgr2=0x7f4e701068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:08.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.155+0000 7f4e74deb640 1 --2- 172.21.15.5:0/1006447304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 0x7f4e701068f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009820 tx=0x7f4e6002a270 comp rx=0 tx=0).stop 2024-09-19T05:54:08.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.155+0000 7f4e74deb640 1 -- 172.21.15.5:0/1006447304 shutdown_connections 2024-09-19T05:54:08.889 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.155+0000 7f4e74deb640 1 --2- 172.21.15.5:0/1006447304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 0x7f4e701068f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:08.889 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.155+0000 7f4e74deb640 1 -- 172.21.15.5:0/1006447304 >> 172.21.15.5:0/1006447304 conn(0x7f4e70101ca0 msgr2=0x7f4e701040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:08.889 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.155+0000 7f4e74deb640 1 -- 172.21.15.5:0/1006447304 shutdown_connections 2024-09-19T05:54:08.889 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.155+0000 7f4e74deb640 1 -- 172.21.15.5:0/1006447304 wait complete. 2024-09-19T05:54:08.889 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.156+0000 7f4e74deb640 1 Processor -- start 2024-09-19T05:54:08.889 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.156+0000 7f4e74deb640 1 -- start start 2024-09-19T05:54:08.889 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.156+0000 7f4e74deb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 0x7f4e7019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:08.889 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.156+0000 7f4e74deb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7010aaa0 con 0x7f4e701064f0 2024-09-19T05:54:08.889 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.156+0000 7f4e6e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 0x7f4e7019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:08.889 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.156+0000 7f4e6e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 0x7f4e7019b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45056/0 (socket says 172.21.15.5:45056) 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.156+0000 7f4e6e59c640 1 -- 172.21.15.5:0/165705682 learned_addr learned my addr 172.21.15.5:0/165705682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.156+0000 7f4e6e59c640 1 -- 172.21.15.5:0/165705682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7019bb10 con 0x7f4e701064f0 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.157+0000 7f4e6e59c640 1 --2- 172.21.15.5:0/165705682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 0x7f4e7019b5d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4e600097f0 tx=0x7f4e60030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.157+0000 7f4e5f7fe640 1 -- 172.21.15.5:0/165705682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e60037040 con 0x7f4e701064f0 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.157+0000 7f4e5f7fe640 1 -- 172.21.15.5:0/165705682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4e60032bb0 con 0x7f4e701064f0 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.157+0000 7f4e74deb640 1 -- 172.21.15.5:0/165705682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7019bda0 con 0x7f4e701064f0 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.157+0000 7f4e5f7fe640 1 -- 172.21.15.5:0/165705682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e6003a390 con 0x7f4e701064f0 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.157+0000 7f4e74deb640 1 -- 172.21.15.5:0/165705682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7019ea90 con 0x7f4e701064f0 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.158+0000 7f4e5f7fe640 1 -- 172.21.15.5:0/165705682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f4e6003a5d0 con 0x7f4e701064f0 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.158+0000 7f4e5f7fe640 1 --2- 172.21.15.5:0/165705682 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f4e58039160 0x7f4e5803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.158+0000 7f4e5f7fe640 1 -- 172.21.15.5:0/165705682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4e6006c410 con 0x7f4e701064f0 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.158+0000 7f4e74deb640 1 -- 172.21.15.5:0/165705682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7010ac20 con 0x7f4e701064f0 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.158+0000 7f4e6dd9b640 1 --2- 172.21.15.5:0/165705682 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f4e58039160 0x7f4e5803b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:08.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.158+0000 7f4e6dd9b640 1 --2- 172.21.15.5:0/165705682 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f4e58039160 0x7f4e5803b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4e640098c0 tx=0x7f4e64006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.163+0000 7f4e5f7fe640 1 -- 172.21.15.5:0/165705682 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4e60040210 con 0x7f4e701064f0 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.341+0000 7f4e74deb640 1 -- 172.21.15.5:0/165705682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f4e70106970 con 0x7f4e701064f0 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.342+0000 7f4e5f7fe640 1 -- 172.21.15.5:0/165705682 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+427 (secure 0 0 0) 0x7f4e6002c030 con 0x7f4e701064f0 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.345+0000 7f4e74deb640 1 -- 172.21.15.5:0/165705682 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f4e58039160 msgr2=0x7f4e5803b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.345+0000 7f4e74deb640 1 --2- 172.21.15.5:0/165705682 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f4e58039160 0x7f4e5803b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4e640098c0 tx=0x7f4e64006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.345+0000 7f4e74deb640 1 -- 172.21.15.5:0/165705682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 msgr2=0x7f4e7019b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.345+0000 7f4e74deb640 1 --2- 172.21.15.5:0/165705682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 0x7f4e7019b5d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4e600097f0 tx=0x7f4e60030d00 comp rx=0 tx=0).stop 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.345+0000 7f4e74deb640 1 -- 172.21.15.5:0/165705682 shutdown_connections 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.345+0000 7f4e74deb640 1 --2- 172.21.15.5:0/165705682 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f4e58039160 0x7f4e5803b620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.345+0000 7f4e74deb640 1 --2- 172.21.15.5:0/165705682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e701064f0 0x7f4e7019b5d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.345+0000 7f4e74deb640 1 -- 172.21.15.5:0/165705682 >> 172.21.15.5:0/165705682 conn(0x7f4e70101ca0 msgr2=0x7f4e70102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.345+0000 7f4e74deb640 1 -- 172.21.15.5:0/165705682 shutdown_connections 2024-09-19T05:54:08.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:08.345+0000 7f4e74deb640 1 -- 172.21.15.5:0/165705682 wait complete. 2024-09-19T05:54:08.892 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling cephadm module... 2024-09-19T05:54:09.520 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:09 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/165705682' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T05:54:10.599 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:10 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3818835009' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T05:54:10.600 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:10 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: ignoring --setuser ceph since I am not root 2024-09-19T05:54:10.600 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:10 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: ignoring --setgroup ceph since I am not root 2024-09-19T05:54:10.600 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:10 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:10.416+0000 7fa196965040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:54:10.600 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:10 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:10.475+0000 7fa196965040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:54:10.861 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:10 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:10.635+0000 7fa196965040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:54:10.861 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:10 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:10.704+0000 7fa196965040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:54:10.861 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:10 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:10.859+0000 7fa196965040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:54:10.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.285+0000 7fb042428640 1 Processor -- start 2024-09-19T05:54:10.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.285+0000 7fb042428640 1 -- start start 2024-09-19T05:54:10.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.285+0000 7fb042428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 0x7fb03c1079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:10.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.285+0000 7fb042428640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c107f90 con 0x7fb03c1075c0 2024-09-19T05:54:10.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.285+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 0x7fb03c1079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:10.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.286+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 0x7fb03c1079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45064/0 (socket says 172.21.15.5:45064) 2024-09-19T05:54:10.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.286+0000 7fb03bfff640 1 -- 172.21.15.5:0/3183611745 learned_addr learned my addr 172.21.15.5:0/3183611745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:10.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.286+0000 7fb03bfff640 1 -- 172.21.15.5:0/3183611745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c1087c0 con 0x7fb03c1075c0 2024-09-19T05:54:10.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.287+0000 7fb03bfff640 1 --2- 172.21.15.5:0/3183611745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 0x7fb03c1079c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009820 tx=0x7fb02c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1549d4ceef60eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:10.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.287+0000 7fb03affd640 1 -- 172.21.15.5:0/3183611745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb02c037070 con 0x7fb03c1075c0 2024-09-19T05:54:10.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.287+0000 7fb03affd640 1 -- 172.21.15.5:0/3183611745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb02c0328d0 con 0x7fb03c1075c0 2024-09-19T05:54:10.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.287+0000 7fb03affd640 1 -- 172.21.15.5:0/3183611745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb02c0307d0 con 0x7fb03c1075c0 2024-09-19T05:54:10.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.287+0000 7fb042428640 1 -- 172.21.15.5:0/3183611745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 msgr2=0x7fb03c1079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:10.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.287+0000 7fb042428640 1 --2- 172.21.15.5:0/3183611745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 0x7fb03c1079c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009820 tx=0x7fb02c02a270 comp rx=0 tx=0).stop 2024-09-19T05:54:10.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.287+0000 7fb042428640 1 -- 172.21.15.5:0/3183611745 shutdown_connections 2024-09-19T05:54:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.287+0000 7fb042428640 1 --2- 172.21.15.5:0/3183611745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 0x7fb03c1079c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.288+0000 7fb042428640 1 -- 172.21.15.5:0/3183611745 >> 172.21.15.5:0/3183611745 conn(0x7fb03c102d50 msgr2=0x7fb03c105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.288+0000 7fb042428640 1 -- 172.21.15.5:0/3183611745 shutdown_connections 2024-09-19T05:54:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.288+0000 7fb042428640 1 -- 172.21.15.5:0/3183611745 wait complete. 2024-09-19T05:54:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.288+0000 7fb042428640 1 Processor -- start 2024-09-19T05:54:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.288+0000 7fb042428640 1 -- start start 2024-09-19T05:54:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.289+0000 7fb042428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 0x7fb03c07db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.289+0000 7fb042428640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c10bb70 con 0x7fb03c1075c0 2024-09-19T05:54:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.289+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 0x7fb03c07db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.289+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 0x7fb03c07db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45078/0 (socket says 172.21.15.5:45078) 2024-09-19T05:54:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.289+0000 7fb03bfff640 1 -- 172.21.15.5:0/3818835009 learned_addr learned my addr 172.21.15.5:0/3818835009 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.289+0000 7fb03bfff640 1 -- 172.21.15.5:0/3818835009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c07e0a0 con 0x7fb03c1075c0 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.289+0000 7fb03bfff640 1 --2- 172.21.15.5:0/3818835009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 0x7fb03c07db60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb02c030520 tx=0x7fb02c032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.289+0000 7fb0397fa640 1 -- 172.21.15.5:0/3818835009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb02c042070 con 0x7fb03c1075c0 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.289+0000 7fb0397fa640 1 -- 172.21.15.5:0/3818835009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb02c030cc0 con 0x7fb03c1075c0 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.289+0000 7fb042428640 1 -- 172.21.15.5:0/3818835009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03c07c290 con 0x7fb03c1075c0 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.290+0000 7fb0397fa640 1 -- 172.21.15.5:0/3818835009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb02c037040 con 0x7fb03c1075c0 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.290+0000 7fb042428640 1 -- 172.21.15.5:0/3818835009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03c07c7d0 con 0x7fb03c1075c0 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.290+0000 7fb0397fa640 1 -- 172.21.15.5:0/3818835009 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fb02c045070 con 0x7fb03c1075c0 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.291+0000 7fb0397fa640 1 --2- 172.21.15.5:0/3818835009 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb0240391b0 0x7fb02403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.291+0000 7fb0397fa640 1 -- 172.21.15.5:0/3818835009 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb02c06c0b0 con 0x7fb03c1075c0 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.291+0000 7fb03b7fe640 1 --2- 172.21.15.5:0/3818835009 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb0240391b0 0x7fb02403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.291+0000 7fb042428640 1 -- 172.21.15.5:0/3818835009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb03c10bcf0 con 0x7fb03c1075c0 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.291+0000 7fb03b7fe640 1 --2- 172.21.15.5:0/3818835009 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb0240391b0 0x7fb02403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0300098c0 tx=0x7fb030006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.296+0000 7fb0397fa640 1 -- 172.21.15.5:0/3818835009 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb02c02f2a0 con 0x7fb03c1075c0 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:09.518+0000 7fb042428640 1 -- 172.21.15.5:0/3818835009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb03c07cee0 con 0x7fb03c1075c0 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:10.266+0000 7fb0397fa640 1 -- 172.21.15.5:0/3818835009 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb02c046dd0 con 0x7fb03c1075c0 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:10.268+0000 7fb0397fa640 1 -- 172.21.15.5:0/3818835009 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fb02c03ca50 con 0x7fb03c1075c0 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:10.270+0000 7fb042428640 1 -- 172.21.15.5:0/3818835009 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb0240391b0 msgr2=0x7fb02403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:10.270+0000 7fb042428640 1 --2- 172.21.15.5:0/3818835009 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb0240391b0 0x7fb02403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0300098c0 tx=0x7fb030006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:10.270+0000 7fb042428640 1 -- 172.21.15.5:0/3818835009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 msgr2=0x7fb03c07db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:10.270+0000 7fb042428640 1 --2- 172.21.15.5:0/3818835009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 0x7fb03c07db60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb02c030520 tx=0x7fb02c032dd0 comp rx=0 tx=0).stop 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:10.270+0000 7fb042428640 1 -- 172.21.15.5:0/3818835009 shutdown_connections 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:10.270+0000 7fb042428640 1 --2- 172.21.15.5:0/3818835009 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb0240391b0 0x7fb02403b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:10.270+0000 7fb042428640 1 --2- 172.21.15.5:0/3818835009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03c1075c0 0x7fb03c07db60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:10.270+0000 7fb042428640 1 -- 172.21.15.5:0/3818835009 >> 172.21.15.5:0/3818835009 conn(0x7fb03c102d50 msgr2=0x7fb03c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:10.270+0000 7fb042428640 1 -- 172.21.15.5:0/3818835009 shutdown_connections 2024-09-19T05:54:10.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:10.270+0000 7fb042428640 1 -- 172.21.15.5:0/3818835009 wait complete. 2024-09-19T05:54:11.129 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:11 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:11.010+0000 7fa196965040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:54:11.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:11 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3818835009' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T05:54:11.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:11 smithi005 ceph-mon[30955]: mgrmap e5: x(active, since 5s) 2024-09-19T05:54:11.410 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:11 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:11.344+0000 7fa196965040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:54:11.410 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:11 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:11.408+0000 7fa196965040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:54:11.848 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:11 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:11.613+0000 7fa196965040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:54:12.091 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:54:12.091 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T05:54:12.091 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T05:54:12.091 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T05:54:12.091 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T05:54:12.091 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:54:12.091 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.296+0000 7f91eb956640 1 Processor -- start 2024-09-19T05:54:12.091 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.297+0000 7f91eb956640 1 -- start start 2024-09-19T05:54:12.091 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.297+0000 7f91eb956640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 0x7f91e410a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.297+0000 7f91eb956640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91e40634b0 con 0x7f91e4108040 2024-09-19T05:54:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.297+0000 7f91e96f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 0x7f91e410a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.297+0000 7f91e96f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 0x7f91e410a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45104/0 (socket says 172.21.15.5:45104) 2024-09-19T05:54:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.297+0000 7f91e96f2640 1 -- 172.21.15.5:0/4017665590 learned_addr learned my addr 172.21.15.5:0/4017665590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.297+0000 7f91e96f2640 1 -- 172.21.15.5:0/4017665590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e410a970 con 0x7f91e4108040 2024-09-19T05:54:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.298+0000 7f91e96f2640 1 --2- 172.21.15.5:0/4017665590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 0x7f91e410a430 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009820 tx=0x7f91e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b00f15369c42c0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.298+0000 7f91dbfff640 1 -- 172.21.15.5:0/4017665590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91e0037070 con 0x7f91e4108040 2024-09-19T05:54:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.298+0000 7f91dbfff640 1 -- 172.21.15.5:0/4017665590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f91e00328d0 con 0x7f91e4108040 2024-09-19T05:54:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.298+0000 7f91dbfff640 1 -- 172.21.15.5:0/4017665590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91e00307d0 con 0x7f91e4108040 2024-09-19T05:54:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.299+0000 7f91eb956640 1 -- 172.21.15.5:0/4017665590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 msgr2=0x7f91e410a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.299+0000 7f91eb956640 1 --2- 172.21.15.5:0/4017665590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 0x7f91e410a430 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009820 tx=0x7f91e002a270 comp rx=0 tx=0).stop 2024-09-19T05:54:12.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.299+0000 7f91eb956640 1 -- 172.21.15.5:0/4017665590 shutdown_connections 2024-09-19T05:54:12.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.299+0000 7f91eb956640 1 --2- 172.21.15.5:0/4017665590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 0x7f91e410a430 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:12.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.299+0000 7f91eb956640 1 -- 172.21.15.5:0/4017665590 >> 172.21.15.5:0/4017665590 conn(0x7f91e4101c00 msgr2=0x7f91e4104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:12.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.299+0000 7f91eb956640 1 -- 172.21.15.5:0/4017665590 shutdown_connections 2024-09-19T05:54:12.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.299+0000 7f91eb956640 1 -- 172.21.15.5:0/4017665590 wait complete. 2024-09-19T05:54:12.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.299+0000 7f91eb956640 1 Processor -- start 2024-09-19T05:54:12.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.300+0000 7f91eb956640 1 -- start start 2024-09-19T05:54:12.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.300+0000 7f91eb956640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 0x7f91e419b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:12.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.300+0000 7f91eb956640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91e410b950 con 0x7f91e4108040 2024-09-19T05:54:12.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.300+0000 7f91e96f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 0x7f91e419b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:12.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.300+0000 7f91e96f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 0x7f91e419b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45106/0 (socket says 172.21.15.5:45106) 2024-09-19T05:54:12.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.300+0000 7f91e96f2640 1 -- 172.21.15.5:0/2774956310 learned_addr learned my addr 172.21.15.5:0/2774956310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:12.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.300+0000 7f91e96f2640 1 -- 172.21.15.5:0/2774956310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e419ba90 con 0x7f91e4108040 2024-09-19T05:54:12.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.300+0000 7f91e96f2640 1 --2- 172.21.15.5:0/2774956310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 0x7f91e419b550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f91e0032e10 tx=0x7f91e0032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:12.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.300+0000 7f91da7fc640 1 -- 172.21.15.5:0/2774956310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91e0042070 con 0x7f91e4108040 2024-09-19T05:54:12.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.300+0000 7f91da7fc640 1 -- 172.21.15.5:0/2774956310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f91e0030c80 con 0x7f91e4108040 2024-09-19T05:54:12.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.300+0000 7f91eb956640 1 -- 172.21.15.5:0/2774956310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e419bd20 con 0x7f91e4108040 2024-09-19T05:54:12.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.300+0000 7f91da7fc640 1 -- 172.21.15.5:0/2774956310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91e0037040 con 0x7f91e4108040 2024-09-19T05:54:12.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.301+0000 7f91eb956640 1 -- 172.21.15.5:0/2774956310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e419ea10 con 0x7f91e4108040 2024-09-19T05:54:12.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.301+0000 7f91da7fc640 1 -- 172.21.15.5:0/2774956310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f91e0045070 con 0x7f91e4108040 2024-09-19T05:54:12.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.302+0000 7f91da7fc640 1 --2- 172.21.15.5:0/2774956310 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f91d00391b0 0x7f91d003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:12.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.302+0000 7f91da7fc640 1 -- 172.21.15.5:0/2774956310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f91e006c170 con 0x7f91e4108040 2024-09-19T05:54:12.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.302+0000 7f91eb956640 1 -- 172.21.15.5:0/2774956310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e41084a0 con 0x7f91e4108040 2024-09-19T05:54:12.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.302+0000 7f91e8ef1640 1 -- 172.21.15.5:0/2774956310 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f91d00391b0 msgr2=0x7f91d003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/330367622 2024-09-19T05:54:12.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.302+0000 7f91e8ef1640 1 --2- 172.21.15.5:0/2774956310 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f91d00391b0 0x7f91d003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:54:12.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.306+0000 7f91da7fc640 1 -- 172.21.15.5:0/2774956310 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f91e0045360 con 0x7f91e4108040 2024-09-19T05:54:12.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.503+0000 7f91e8ef1640 1 -- 172.21.15.5:0/2774956310 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f91d00391b0 msgr2=0x7f91d003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/330367622 2024-09-19T05:54:12.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.503+0000 7f91e8ef1640 1 --2- 172.21.15.5:0/2774956310 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f91d00391b0 0x7f91d003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:54:12.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.533+0000 7f91eb956640 1 -- 172.21.15.5:0/2774956310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f91e419ebe0 con 0x7f91e4108040 2024-09-19T05:54:12.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.534+0000 7f91da7fc640 1 -- 172.21.15.5:0/2774956310 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f91e419ebe0 con 0x7f91e4108040 2024-09-19T05:54:12.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.536+0000 7f91eb956640 1 -- 172.21.15.5:0/2774956310 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f91d00391b0 msgr2=0x7f91d003b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:54:12.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.536+0000 7f91eb956640 1 --2- 172.21.15.5:0/2774956310 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f91d00391b0 0x7f91d003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:12.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.536+0000 7f91eb956640 1 -- 172.21.15.5:0/2774956310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 msgr2=0x7f91e419b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:12.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.536+0000 7f91eb956640 1 --2- 172.21.15.5:0/2774956310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 0x7f91e419b550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f91e0032e10 tx=0x7f91e0032e40 comp rx=0 tx=0).stop 2024-09-19T05:54:12.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.536+0000 7f91eb956640 1 -- 172.21.15.5:0/2774956310 shutdown_connections 2024-09-19T05:54:12.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.536+0000 7f91eb956640 1 --2- 172.21.15.5:0/2774956310 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7f91d00391b0 0x7f91d003b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:12.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.536+0000 7f91eb956640 1 --2- 172.21.15.5:0/2774956310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91e4108040 0x7f91e419b550 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:12.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.536+0000 7f91eb956640 1 -- 172.21.15.5:0/2774956310 >> 172.21.15.5:0/2774956310 conn(0x7f91e4101c00 msgr2=0x7f91e4102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:12.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.536+0000 7f91eb956640 1 -- 172.21.15.5:0/2774956310 shutdown_connections 2024-09-19T05:54:12.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:11.536+0000 7f91eb956640 1 -- 172.21.15.5:0/2774956310 wait complete. 2024-09-19T05:54:12.096 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-09-19T05:54:12.096 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 5... 2024-09-19T05:54:12.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:12 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2774956310' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T05:54:12.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:12 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:12.549+0000 7fa196965040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:54:12.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:12 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:12.620+0000 7fa196965040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:54:13.379 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:13.363+0000 7fa196965040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:54:13.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:13.483+0000 7fa196965040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:54:13.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:13.636+0000 7fa196965040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:54:13.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:13.700+0000 7fa196965040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:54:13.859 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:13.856+0000 7fa196965040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:54:14.129 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:13.927+0000 7fa196965040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:54:14.129 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:13.990+0000 7fa196965040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:54:14.446 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:14 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:14.159+0000 7fa196965040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:54:14.446 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:14 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:14.227+0000 7fa196965040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:54:14.446 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:14 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:14.296+0000 7fa196965040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:54:14.447 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:14 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:14.382+0000 7fa196965040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:54:14.447 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:14 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:14.444+0000 7fa196965040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:54:14.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:14 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:14.505+0000 7fa196965040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:54:15.629 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:15.332+0000 7fa196965040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:54:15.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: Active manager daemon x restarted 2024-09-19T05:54:15.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: Activating manager daemon x 2024-09-19T05:54:15.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T05:54:15.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: mgrmap e6: x(active, starting, since 0.00348841s) 2024-09-19T05:54:15.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:54:15.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T05:54:15.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T05:54:15.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T05:54:15.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T05:54:15.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: Manager daemon x is now available 2024-09-19T05:54:15.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' 2024-09-19T05:54:15.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' 2024-09-19T05:54:15.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:54:15.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:54:15.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:54:15.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:15 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:54:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:16 smithi005 ceph-mon[30955]: Found migration_current of "None". Setting to last migration. 2024-09-19T05:54:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:16 smithi005 ceph-mon[30955]: mgrmap e7: x(active, since 1.00561s) 2024-09-19T05:54:16.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:54:16.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T05:54:16.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T05:54:16.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:54:16.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.508+0000 7fb9eadbd640 1 Processor -- start 2024-09-19T05:54:16.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.508+0000 7fb9eadbd640 1 -- start start 2024-09-19T05:54:16.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.508+0000 7fb9eadbd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 0x7fb9e410a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:16.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.508+0000 7fb9eadbd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e4063510 con 0x7fb9e4107ff0 2024-09-19T05:54:16.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.508+0000 7fb9e8b59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 0x7fb9e410a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:16.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.508+0000 7fb9e8b59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 0x7fb9e410a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45116/0 (socket says 172.21.15.5:45116) 2024-09-19T05:54:16.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.508+0000 7fb9e8b59640 1 -- 172.21.15.5:0/1111169250 learned_addr learned my addr 172.21.15.5:0/1111169250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:16.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.509+0000 7fb9e8b59640 1 -- 172.21.15.5:0/1111169250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e410a920 con 0x7fb9e4107ff0 2024-09-19T05:54:16.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.509+0000 7fb9e8b59640 1 --2- 172.21.15.5:0/1111169250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 0x7fb9e410a3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009820 tx=0x7fb9d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d4ac3c799d6811 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:16.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.510+0000 7fb9e37fe640 1 -- 172.21.15.5:0/1111169250 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9d8037070 con 0x7fb9e4107ff0 2024-09-19T05:54:16.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.510+0000 7fb9e37fe640 1 -- 172.21.15.5:0/1111169250 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb9d80328d0 con 0x7fb9e4107ff0 2024-09-19T05:54:16.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.510+0000 7fb9e37fe640 1 -- 172.21.15.5:0/1111169250 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9d80307d0 con 0x7fb9e4107ff0 2024-09-19T05:54:16.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.510+0000 7fb9eadbd640 1 -- 172.21.15.5:0/1111169250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 msgr2=0x7fb9e410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:16.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.510+0000 7fb9eadbd640 1 --2- 172.21.15.5:0/1111169250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 0x7fb9e410a3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009820 tx=0x7fb9d802a270 comp rx=0 tx=0).stop 2024-09-19T05:54:16.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.510+0000 7fb9eadbd640 1 -- 172.21.15.5:0/1111169250 shutdown_connections 2024-09-19T05:54:16.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.510+0000 7fb9eadbd640 1 --2- 172.21.15.5:0/1111169250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 0x7fb9e410a3e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:16.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.510+0000 7fb9eadbd640 1 -- 172.21.15.5:0/1111169250 >> 172.21.15.5:0/1111169250 conn(0x7fb9e4101bf0 msgr2=0x7fb9e4104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:16.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.510+0000 7fb9eadbd640 1 -- 172.21.15.5:0/1111169250 shutdown_connections 2024-09-19T05:54:16.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.510+0000 7fb9eadbd640 1 -- 172.21.15.5:0/1111169250 wait complete. 2024-09-19T05:54:16.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.511+0000 7fb9eadbd640 1 Processor -- start 2024-09-19T05:54:16.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.511+0000 7fb9eadbd640 1 -- start start 2024-09-19T05:54:16.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.511+0000 7fb9eadbd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 0x7fb9e419b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:16.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.511+0000 7fb9eadbd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e410b960 con 0x7fb9e4107ff0 2024-09-19T05:54:16.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.512+0000 7fb9e8b59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 0x7fb9e419b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:16.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.512+0000 7fb9e8b59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 0x7fb9e419b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45128/0 (socket says 172.21.15.5:45128) 2024-09-19T05:54:16.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.512+0000 7fb9e8b59640 1 -- 172.21.15.5:0/2966612979 learned_addr learned my addr 172.21.15.5:0/2966612979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:16.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.512+0000 7fb9e8b59640 1 -- 172.21.15.5:0/2966612979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e419ba70 con 0x7fb9e4107ff0 2024-09-19T05:54:16.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.512+0000 7fb9e8b59640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 0x7fb9e419b530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80097f0 tx=0x7fb9d8030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:16.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.512+0000 7fb9e1ffb640 1 -- 172.21.15.5:0/2966612979 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9d8037040 con 0x7fb9e4107ff0 2024-09-19T05:54:16.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.512+0000 7fb9e1ffb640 1 -- 172.21.15.5:0/2966612979 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb9d8032bb0 con 0x7fb9e4107ff0 2024-09-19T05:54:16.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.512+0000 7fb9eadbd640 1 -- 172.21.15.5:0/2966612979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e419bd00 con 0x7fb9e4107ff0 2024-09-19T05:54:16.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.512+0000 7fb9e1ffb640 1 -- 172.21.15.5:0/2966612979 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9d803a390 con 0x7fb9e4107ff0 2024-09-19T05:54:16.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.512+0000 7fb9eadbd640 1 -- 172.21.15.5:0/2966612979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e419e9f0 con 0x7fb9e4107ff0 2024-09-19T05:54:16.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.513+0000 7fb9e1ffb640 1 -- 172.21.15.5:0/2966612979 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fb9d803a5d0 con 0x7fb9e4107ff0 2024-09-19T05:54:16.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.514+0000 7fb9e1ffb640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb9d40391b0 0x7fb9d403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:16.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.514+0000 7fb9e1ffb640 1 -- 172.21.15.5:0/2966612979 --> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb9d403bd80 con 0x7fb9d40391b0 2024-09-19T05:54:16.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.514+0000 7fb9e1ffb640 1 -- 172.21.15.5:0/2966612979 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb9d806c620 con 0x7fb9e4107ff0 2024-09-19T05:54:16.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.514+0000 7fb9e3fff640 1 -- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb9d40391b0 msgr2=0x7fb9d403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/330367622 2024-09-19T05:54:16.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.514+0000 7fb9e3fff640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb9d40391b0 0x7fb9d403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:54:16.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.715+0000 7fb9e3fff640 1 -- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb9d40391b0 msgr2=0x7fb9d403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/330367622 2024-09-19T05:54:16.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:12.715+0000 7fb9e3fff640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb9d40391b0 0x7fb9d403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:54:16.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:13.116+0000 7fb9e3fff640 1 -- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb9d40391b0 msgr2=0x7fb9d403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/330367622 2024-09-19T05:54:16.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:13.116+0000 7fb9e3fff640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb9d40391b0 0x7fb9d403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T05:54:16.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:13.917+0000 7fb9e3fff640 1 -- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb9d40391b0 msgr2=0x7fb9d403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/330367622 2024-09-19T05:54:16.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:13.917+0000 7fb9e3fff640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb9d40391b0 0x7fb9d403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T05:54:16.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:15.336+0000 7fb9e1ffb640 1 -- 172.21.15.5:0/2966612979 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7fb9d806b0e0 con 0x7fb9e4107ff0 2024-09-19T05:54:16.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:15.336+0000 7fb9e1ffb640 1 -- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb9d40391b0 msgr2=0x7fb9d403b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:54:16.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:15.336+0000 7fb9e1ffb640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb9d40391b0 0x7fb9d403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:16.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.337+0000 7fb9e1ffb640 1 -- 172.21.15.5:0/2966612979 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7fb9d8044780 con 0x7fb9e4107ff0 2024-09-19T05:54:16.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.337+0000 7fb9e1ffb640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fb9d403cae0 0x7fb9d403eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:16.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.337+0000 7fb9e1ffb640 1 -- 172.21.15.5:0/2966612979 --> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb9d403bd80 con 0x7fb9d403cae0 2024-09-19T05:54:16.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.337+0000 7fb9e3fff640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fb9d403cae0 0x7fb9d403eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:16.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.338+0000 7fb9e3fff640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fb9d403cae0 0x7fb9d403eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0003980 tx=0x7fb9d0006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:16.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.339+0000 7fb9e1ffb640 1 -- 172.21.15.5:0/2966612979 <== mgr.14118 v2:172.21.15.5:6800/255461399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fb9d403bd80 con 0x7fb9d403cae0 2024-09-19T05:54:16.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9eadbd640 1 -- 172.21.15.5:0/2966612979 --> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb9e4108450 con 0x7fb9d403cae0 2024-09-19T05:54:16.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9e1ffb640 1 -- 172.21.15.5:0/2966612979 <== mgr.14118 v2:172.21.15.5:6800/255461399 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fb9e4108450 con 0x7fb9d403cae0 2024-09-19T05:54:16.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9eadbd640 1 -- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fb9d403cae0 msgr2=0x7fb9d403eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:16.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9eadbd640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fb9d403cae0 0x7fb9d403eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0003980 tx=0x7fb9d0006e10 comp rx=0 tx=0).stop 2024-09-19T05:54:16.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9eadbd640 1 -- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 msgr2=0x7fb9e419b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:16.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9eadbd640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 0x7fb9e419b530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80097f0 tx=0x7fb9d8030d00 comp rx=0 tx=0).stop 2024-09-19T05:54:16.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9eadbd640 1 -- 172.21.15.5:0/2966612979 shutdown_connections 2024-09-19T05:54:16.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9eadbd640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fb9d403cae0 0x7fb9d403eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:16.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9eadbd640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:6800/330367622,v1:172.21.15.5:6801/330367622] conn(0x7fb9d40391b0 0x7fb9d403b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:16.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9eadbd640 1 --2- 172.21.15.5:0/2966612979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9e4107ff0 0x7fb9e419b530 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:16.904 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9eadbd640 1 -- 172.21.15.5:0/2966612979 >> 172.21.15.5:0/2966612979 conn(0x7fb9e4101bf0 msgr2=0x7fb9e4102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:16.904 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9eadbd640 1 -- 172.21.15.5:0/2966612979 shutdown_connections 2024-09-19T05:54:16.904 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:16.346+0000 7fb9eadbd640 1 -- 172.21.15.5:0/2966612979 wait complete. 2024-09-19T05:54:16.904 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 5 is available 2024-09-19T05:54:16.904 INFO:teuthology.orchestra.run.smithi005.stdout:Setting orchestrator backend to cephadm... 2024-09-19T05:54:17.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:17 smithi005 ceph-mon[30955]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T05:54:17.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:17 smithi005 ceph-mon[30955]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T05:54:17.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:17 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' 2024-09-19T05:54:17.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:17 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' 2024-09-19T05:54:17.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:17 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' 2024-09-19T05:54:17.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:17 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:54:18.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.318+0000 7fdc946d8640 1 Processor -- start 2024-09-19T05:54:18.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.318+0000 7fdc946d8640 1 -- start start 2024-09-19T05:54:18.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.319+0000 7fdc946d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 0x7fdc8c1079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:18.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.319+0000 7fdc946d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c107f90 con 0x7fdc8c1075c0 2024-09-19T05:54:18.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.319+0000 7fdc92474640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 0x7fdc8c1079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:18.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.319+0000 7fdc92474640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 0x7fdc8c1079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37050/0 (socket says 172.21.15.5:37050) 2024-09-19T05:54:18.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.319+0000 7fdc92474640 1 -- 172.21.15.5:0/251179973 learned_addr learned my addr 172.21.15.5:0/251179973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:18.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.319+0000 7fdc92474640 1 -- 172.21.15.5:0/251179973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c1087c0 con 0x7fdc8c1075c0 2024-09-19T05:54:18.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.320+0000 7fdc92474640 1 --2- 172.21.15.5:0/251179973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 0x7fdc8c1079c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009820 tx=0x7fdc8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899b92e163243261 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:18.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.320+0000 7fdc91472640 1 -- 172.21.15.5:0/251179973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc88037070 con 0x7fdc8c1075c0 2024-09-19T05:54:18.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.320+0000 7fdc91472640 1 -- 172.21.15.5:0/251179973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdc880328d0 con 0x7fdc8c1075c0 2024-09-19T05:54:18.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.320+0000 7fdc91472640 1 -- 172.21.15.5:0/251179973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc880307d0 con 0x7fdc8c1075c0 2024-09-19T05:54:18.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.321+0000 7fdc946d8640 1 -- 172.21.15.5:0/251179973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 msgr2=0x7fdc8c1079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:18.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.321+0000 7fdc946d8640 1 --2- 172.21.15.5:0/251179973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 0x7fdc8c1079c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009820 tx=0x7fdc8802a270 comp rx=0 tx=0).stop 2024-09-19T05:54:18.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.321+0000 7fdc946d8640 1 -- 172.21.15.5:0/251179973 shutdown_connections 2024-09-19T05:54:18.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.321+0000 7fdc946d8640 1 --2- 172.21.15.5:0/251179973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 0x7fdc8c1079c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:18.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.321+0000 7fdc946d8640 1 -- 172.21.15.5:0/251179973 >> 172.21.15.5:0/251179973 conn(0x7fdc8c102d50 msgr2=0x7fdc8c105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:18.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.321+0000 7fdc946d8640 1 -- 172.21.15.5:0/251179973 shutdown_connections 2024-09-19T05:54:18.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.321+0000 7fdc946d8640 1 -- 172.21.15.5:0/251179973 wait complete. 2024-09-19T05:54:18.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.322+0000 7fdc946d8640 1 Processor -- start 2024-09-19T05:54:18.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.322+0000 7fdc946d8640 1 -- start start 2024-09-19T05:54:18.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.322+0000 7fdc946d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 0x7fdc8c19b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:18.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.322+0000 7fdc946d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c10bb70 con 0x7fdc8c1075c0 2024-09-19T05:54:18.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.322+0000 7fdc92474640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 0x7fdc8c19b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:18.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.322+0000 7fdc92474640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 0x7fdc8c19b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37064/0 (socket says 172.21.15.5:37064) 2024-09-19T05:54:18.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.322+0000 7fdc92474640 1 -- 172.21.15.5:0/240238632 learned_addr learned my addr 172.21.15.5:0/240238632 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:18.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.322+0000 7fdc92474640 1 -- 172.21.15.5:0/240238632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c19bb30 con 0x7fdc8c1075c0 2024-09-19T05:54:18.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.322+0000 7fdc92474640 1 --2- 172.21.15.5:0/240238632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 0x7fdc8c19b5f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdc880097f0 tx=0x7fdc88030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:18.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.323+0000 7fdc837fe640 1 -- 172.21.15.5:0/240238632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc88037040 con 0x7fdc8c1075c0 2024-09-19T05:54:18.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.323+0000 7fdc837fe640 1 -- 172.21.15.5:0/240238632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdc88032bb0 con 0x7fdc8c1075c0 2024-09-19T05:54:18.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.323+0000 7fdc946d8640 1 -- 172.21.15.5:0/240238632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8c19bdc0 con 0x7fdc8c1075c0 2024-09-19T05:54:18.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.323+0000 7fdc837fe640 1 -- 172.21.15.5:0/240238632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc8803a390 con 0x7fdc8c1075c0 2024-09-19T05:54:18.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.323+0000 7fdc946d8640 1 -- 172.21.15.5:0/240238632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8c19eab0 con 0x7fdc8c1075c0 2024-09-19T05:54:18.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.323+0000 7fdc837fe640 1 -- 172.21.15.5:0/240238632 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7fdc8803a580 con 0x7fdc8c1075c0 2024-09-19T05:54:18.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.324+0000 7fdc837fe640 1 --2- 172.21.15.5:0/240238632 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fdc780390f0 0x7fdc7803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:18.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.324+0000 7fdc837fe640 1 -- 172.21.15.5:0/240238632 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc8806c390 con 0x7fdc8c1075c0 2024-09-19T05:54:18.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.324+0000 7fdc946d8640 1 -- 172.21.15.5:0/240238632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc8c10bcf0 con 0x7fdc8c1075c0 2024-09-19T05:54:18.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.324+0000 7fdc91c73640 1 --2- 172.21.15.5:0/240238632 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fdc780390f0 0x7fdc7803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:18.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.324+0000 7fdc91c73640 1 --2- 172.21.15.5:0/240238632 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fdc780390f0 0x7fdc7803b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0098c0 tx=0x7fdc7c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:18.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.329+0000 7fdc837fe640 1 -- 172.21.15.5:0/240238632 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdc88032dd0 con 0x7fdc8c1075c0 2024-09-19T05:54:18.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.521+0000 7fdc946d8640 1 -- 172.21.15.5:0/240238632 --> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fdc8c104020 con 0x7fdc780390f0 2024-09-19T05:54:18.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.525+0000 7fdc837fe640 1 -- 172.21.15.5:0/240238632 <== mgr.14118 v2:172.21.15.5:6800/255461399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdc8c104020 con 0x7fdc780390f0 2024-09-19T05:54:18.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.528+0000 7fdc946d8640 1 -- 172.21.15.5:0/240238632 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fdc780390f0 msgr2=0x7fdc7803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:18.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.528+0000 7fdc946d8640 1 --2- 172.21.15.5:0/240238632 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fdc780390f0 0x7fdc7803b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0098c0 tx=0x7fdc7c006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:18.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.528+0000 7fdc946d8640 1 -- 172.21.15.5:0/240238632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 msgr2=0x7fdc8c19b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:18.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.528+0000 7fdc946d8640 1 --2- 172.21.15.5:0/240238632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 0x7fdc8c19b5f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdc880097f0 tx=0x7fdc88030d00 comp rx=0 tx=0).stop 2024-09-19T05:54:18.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.528+0000 7fdc946d8640 1 -- 172.21.15.5:0/240238632 shutdown_connections 2024-09-19T05:54:18.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.528+0000 7fdc946d8640 1 --2- 172.21.15.5:0/240238632 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fdc780390f0 0x7fdc7803b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:18.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.528+0000 7fdc946d8640 1 --2- 172.21.15.5:0/240238632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c1075c0 0x7fdc8c19b5f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:18.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.528+0000 7fdc946d8640 1 -- 172.21.15.5:0/240238632 >> 172.21.15.5:0/240238632 conn(0x7fdc8c102d50 msgr2=0x7fdc8c103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:18.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.528+0000 7fdc946d8640 1 -- 172.21.15.5:0/240238632 shutdown_connections 2024-09-19T05:54:18.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:17.528+0000 7fdc946d8640 1 -- 172.21.15.5:0/240238632 wait complete. 2024-09-19T05:54:18.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:18.236+0000 7fa0fdc7f640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:05:54:18] ENGINE Error in HTTPServer.serve 2024-09-19T05:54:18.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: Traceback (most recent call last): 2024-09-19T05:54:18.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:54:18.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self._connections.run(self.expiration_interval) 2024-09-19T05:54:18.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:54:18.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self._run(expiration_interval) 2024-09-19T05:54:18.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:54:18.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:54:18.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:54:18.339 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:54:18.339 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:54:18.339 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: s = self.context.wrap_socket( 2024-09-19T05:54:18.339 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:54:18.339 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: return self.sslsocket_class._create( 2024-09-19T05:54:18.339 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:54:18.339 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self.do_handshake() 2024-09-19T05:54:18.339 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:54:18.339 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self._sslobj.do_handshake() 2024-09-19T05:54:18.340 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:54:18.340 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:19.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-mon[30955]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:54:19.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-mon[30955]: mgrmap e8: x(active, since 2s) 2024-09-19T05:54:19.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-mon[30955]: [19/Sep/2024:05:54:18] ENGINE Bus STARTING 2024-09-19T05:54:19.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-mon[30955]: [19/Sep/2024:05:54:18] ENGINE Serving on http://172.21.15.5:8765 2024-09-19T05:54:19.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-mon[30955]: [19/Sep/2024:05:54:18] ENGINE Serving on https://172.21.15.5:7150 2024-09-19T05:54:19.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-mon[30955]: [19/Sep/2024:05:54:18] ENGINE Bus STARTED 2024-09-19T05:54:19.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:54:19.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:18 smithi005 ceph-mon[30955]: [19/Sep/2024:05:54:18] ENGINE Error in HTTPServer.serve 2024-09-19T05:54:19.003 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-19T05:54:19.003 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:54:19.003 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-19T05:54:19.003 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:54:19.003 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-19T05:54:19.004 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:54:19.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T05:54:19.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.516+0000 7f48a434e640 1 Processor -- start 2024-09-19T05:54:19.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.516+0000 7f48a434e640 1 -- start start 2024-09-19T05:54:19.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.517+0000 7f48a434e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 0x7f489c106880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:19.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.517+0000 7f48a434e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f489c106e50 con 0x7f489c106480 2024-09-19T05:54:19.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.517+0000 7f48a20ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 0x7f489c106880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:19.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.517+0000 7f48a20ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 0x7f489c106880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37074/0 (socket says 172.21.15.5:37074) 2024-09-19T05:54:19.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.517+0000 7f48a20ea640 1 -- 172.21.15.5:0/1250173404 learned_addr learned my addr 172.21.15.5:0/1250173404 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:19.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.517+0000 7f48a20ea640 1 -- 172.21.15.5:0/1250173404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c107680 con 0x7f489c106480 2024-09-19T05:54:19.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.518+0000 7f48a20ea640 1 --2- 172.21.15.5:0/1250173404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 0x7f489c106880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4898009820 tx=0x7f489802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2260cee6f9dcaa56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:19.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.518+0000 7f48a10e8640 1 -- 172.21.15.5:0/1250173404 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4898037070 con 0x7f489c106480 2024-09-19T05:54:19.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.518+0000 7f48a10e8640 1 -- 172.21.15.5:0/1250173404 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f48980328d0 con 0x7f489c106480 2024-09-19T05:54:19.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.518+0000 7f48a10e8640 1 -- 172.21.15.5:0/1250173404 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48980307d0 con 0x7f489c106480 2024-09-19T05:54:19.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.519+0000 7f48a434e640 1 -- 172.21.15.5:0/1250173404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 msgr2=0x7f489c106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:19.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.519+0000 7f48a434e640 1 --2- 172.21.15.5:0/1250173404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 0x7f489c106880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4898009820 tx=0x7f489802a270 comp rx=0 tx=0).stop 2024-09-19T05:54:19.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.519+0000 7f48a434e640 1 -- 172.21.15.5:0/1250173404 shutdown_connections 2024-09-19T05:54:19.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.519+0000 7f48a434e640 1 --2- 172.21.15.5:0/1250173404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 0x7f489c106880 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:19.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.519+0000 7f48a434e640 1 -- 172.21.15.5:0/1250173404 >> 172.21.15.5:0/1250173404 conn(0x7f489c101bf0 msgr2=0x7f489c104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:19.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.519+0000 7f48a434e640 1 -- 172.21.15.5:0/1250173404 shutdown_connections 2024-09-19T05:54:19.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.519+0000 7f48a434e640 1 -- 172.21.15.5:0/1250173404 wait complete. 2024-09-19T05:54:19.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.519+0000 7f48a434e640 1 Processor -- start 2024-09-19T05:54:19.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.519+0000 7f48a434e640 1 -- start start 2024-09-19T05:54:19.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.520+0000 7f48a434e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 0x7f489c1916b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:19.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.520+0000 7f48a434e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f489c10aa30 con 0x7f489c106480 2024-09-19T05:54:19.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.520+0000 7f48a20ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 0x7f489c1916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:19.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.520+0000 7f48a20ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 0x7f489c1916b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37080/0 (socket says 172.21.15.5:37080) 2024-09-19T05:54:19.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.520+0000 7f48a20ea640 1 -- 172.21.15.5:0/2300838803 learned_addr learned my addr 172.21.15.5:0/2300838803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:19.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.520+0000 7f48a20ea640 1 -- 172.21.15.5:0/2300838803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c191bf0 con 0x7f489c106480 2024-09-19T05:54:19.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.520+0000 7f48a20ea640 1 --2- 172.21.15.5:0/2300838803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 0x7f489c1916b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f48980097f0 tx=0x7f4898030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:19.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.520+0000 7f48937fe640 1 -- 172.21.15.5:0/2300838803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4898037040 con 0x7f489c106480 2024-09-19T05:54:19.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.520+0000 7f48937fe640 1 -- 172.21.15.5:0/2300838803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4898032bb0 con 0x7f489c106480 2024-09-19T05:54:19.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.520+0000 7f48937fe640 1 -- 172.21.15.5:0/2300838803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f489803a390 con 0x7f489c106480 2024-09-19T05:54:19.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.520+0000 7f48a434e640 1 -- 172.21.15.5:0/2300838803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c18fde0 con 0x7f489c106480 2024-09-19T05:54:19.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.520+0000 7f48a434e640 1 -- 172.21.15.5:0/2300838803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c190320 con 0x7f489c106480 2024-09-19T05:54:19.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.521+0000 7f48937fe640 1 -- 172.21.15.5:0/2300838803 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f489803a5d0 con 0x7f489c106480 2024-09-19T05:54:19.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.521+0000 7f48937fe640 1 --2- 172.21.15.5:0/2300838803 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f48880391b0 0x7f488803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:19.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.521+0000 7f48937fe640 1 -- 172.21.15.5:0/2300838803 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f489806c460 con 0x7f489c106480 2024-09-19T05:54:19.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.521+0000 7f48a434e640 1 -- 172.21.15.5:0/2300838803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489c106900 con 0x7f489c106480 2024-09-19T05:54:19.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.522+0000 7f48a18e9640 1 --2- 172.21.15.5:0/2300838803 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f48880391b0 0x7f488803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:19.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.522+0000 7f48a18e9640 1 --2- 172.21.15.5:0/2300838803 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f48880391b0 0x7f488803b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f488c0098c0 tx=0x7f488c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:19.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.526+0000 7f48937fe640 1 -- 172.21.15.5:0/2300838803 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4898040210 con 0x7f489c106480 2024-09-19T05:54:19.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.706+0000 7f48a434e640 1 -- 172.21.15.5:0/2300838803 --> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f489c195410 con 0x7f48880391b0 2024-09-19T05:54:19.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.706+0000 7f48937fe640 1 -- 172.21.15.5:0/2300838803 <== mgr.14118 v2:172.21.15.5:6800/255461399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f489c195410 con 0x7f48880391b0 2024-09-19T05:54:19.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.708+0000 7f48a434e640 1 -- 172.21.15.5:0/2300838803 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f48880391b0 msgr2=0x7f488803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:19.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.708+0000 7f48a434e640 1 --2- 172.21.15.5:0/2300838803 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f48880391b0 0x7f488803b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f488c0098c0 tx=0x7f488c006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:19.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.708+0000 7f48a434e640 1 -- 172.21.15.5:0/2300838803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 msgr2=0x7f489c1916b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:19.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.708+0000 7f48a434e640 1 --2- 172.21.15.5:0/2300838803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 0x7f489c1916b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f48980097f0 tx=0x7f4898030d00 comp rx=0 tx=0).stop 2024-09-19T05:54:19.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.708+0000 7f48a434e640 1 -- 172.21.15.5:0/2300838803 shutdown_connections 2024-09-19T05:54:19.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.708+0000 7f48a434e640 1 --2- 172.21.15.5:0/2300838803 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f48880391b0 0x7f488803b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:19.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.708+0000 7f48a434e640 1 --2- 172.21.15.5:0/2300838803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489c106480 0x7f489c1916b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:19.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.708+0000 7f48a434e640 1 -- 172.21.15.5:0/2300838803 >> 172.21.15.5:0/2300838803 conn(0x7f489c101bf0 msgr2=0x7f489c102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:19.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.708+0000 7f48a434e640 1 -- 172.21.15.5:0/2300838803 shutdown_connections 2024-09-19T05:54:19.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:18.708+0000 7f48a434e640 1 -- 172.21.15.5:0/2300838803 wait complete. 2024-09-19T05:54:19.281 INFO:teuthology.orchestra.run.smithi005.stdout:Generating ssh key... 2024-09-19T05:54:19.865 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:19 smithi005 ceph-mon[30955]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:54:20.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: Generating public/private rsa key pair. 2024-09-19T05:54:20.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: Your identification has been saved in /tmp/tmpv657r0a3/key 2024-09-19T05:54:20.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: Your public key has been saved in /tmp/tmpv657r0a3/key.pub 2024-09-19T05:54:20.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: The key fingerprint is: 2024-09-19T05:54:20.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: SHA256:3c3jhCTzx/gbzP29MTiDhzhx95IfG9e21Zkq/wuLB5Q ceph-6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:54:20.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: The key's randomart image is: 2024-09-19T05:54:20.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: +---[RSA 3072]----+ 2024-09-19T05:54:20.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: | | 2024-09-19T05:54:20.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: | | 2024-09-19T05:54:20.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: | o o | 2024-09-19T05:54:20.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: | . E * | 2024-09-19T05:54:20.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: | S.o.=.B | 2024-09-19T05:54:20.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: | +.+B+o=| 2024-09-19T05:54:20.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: | o o.O*OB| 2024-09-19T05:54:20.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: | ..o.O+%| 2024-09-19T05:54:20.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: | .++oB=| 2024-09-19T05:54:20.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: +----[SHA256]-----+ 2024-09-19T05:54:20.807 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.674+0000 7f83120ad640 1 Processor -- start 2024-09-19T05:54:20.807 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.674+0000 7f83120ad640 1 -- start start 2024-09-19T05:54:20.807 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.674+0000 7f83120ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 0x7f830c1079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:20.807 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.674+0000 7f83120ad640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f830c107f90 con 0x7f830c1075c0 2024-09-19T05:54:20.807 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.674+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 0x7f830c1079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:20.807 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.674+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 0x7f830c1079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37094/0 (socket says 172.21.15.5:37094) 2024-09-19T05:54:20.808 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.674+0000 7f830b7fe640 1 -- 172.21.15.5:0/2628344295 learned_addr learned my addr 172.21.15.5:0/2628344295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:20.808 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.675+0000 7f830b7fe640 1 -- 172.21.15.5:0/2628344295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c1087c0 con 0x7f830c1075c0 2024-09-19T05:54:20.808 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.675+0000 7f830b7fe640 1 --2- 172.21.15.5:0/2628344295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 0x7f830c1079c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009820 tx=0x7f82fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f2782ec9982c6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:20.808 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.676+0000 7f830a7fc640 1 -- 172.21.15.5:0/2628344295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82fc037070 con 0x7f830c1075c0 2024-09-19T05:54:20.808 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.676+0000 7f830a7fc640 1 -- 172.21.15.5:0/2628344295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f82fc0328d0 con 0x7f830c1075c0 2024-09-19T05:54:20.808 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.676+0000 7f830a7fc640 1 -- 172.21.15.5:0/2628344295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82fc0307d0 con 0x7f830c1075c0 2024-09-19T05:54:20.808 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.676+0000 7f83120ad640 1 -- 172.21.15.5:0/2628344295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 msgr2=0x7f830c1079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:20.808 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.676+0000 7f83120ad640 1 --2- 172.21.15.5:0/2628344295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 0x7f830c1079c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009820 tx=0x7f82fc02a270 comp rx=0 tx=0).stop 2024-09-19T05:54:20.808 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.676+0000 7f83120ad640 1 -- 172.21.15.5:0/2628344295 shutdown_connections 2024-09-19T05:54:20.808 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.676+0000 7f83120ad640 1 --2- 172.21.15.5:0/2628344295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 0x7f830c1079c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.676+0000 7f83120ad640 1 -- 172.21.15.5:0/2628344295 >> 172.21.15.5:0/2628344295 conn(0x7f830c102d50 msgr2=0x7f830c105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.676+0000 7f83120ad640 1 -- 172.21.15.5:0/2628344295 shutdown_connections 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.676+0000 7f83120ad640 1 -- 172.21.15.5:0/2628344295 wait complete. 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.677+0000 7f83120ad640 1 Processor -- start 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.677+0000 7f83120ad640 1 -- start start 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.677+0000 7f83120ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 0x7f830c19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.677+0000 7f83120ad640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10bb70 con 0x7f830c1075c0 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.677+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 0x7f830c19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.677+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 0x7f830c19b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37102/0 (socket says 172.21.15.5:37102) 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.677+0000 7f830b7fe640 1 -- 172.21.15.5:0/659061100 learned_addr learned my addr 172.21.15.5:0/659061100 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.678+0000 7f830b7fe640 1 -- 172.21.15.5:0/659061100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c19bb00 con 0x7f830c1075c0 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.678+0000 7f830b7fe640 1 --2- 172.21.15.5:0/659061100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 0x7f830c19b5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f82fc030520 tx=0x7f82fc032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:20.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.678+0000 7f8308ff9640 1 -- 172.21.15.5:0/659061100 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82fc042070 con 0x7f830c1075c0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.678+0000 7f8308ff9640 1 -- 172.21.15.5:0/659061100 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f82fc030cc0 con 0x7f830c1075c0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.678+0000 7f83120ad640 1 -- 172.21.15.5:0/659061100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c19bd90 con 0x7f830c1075c0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.678+0000 7f8308ff9640 1 -- 172.21.15.5:0/659061100 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82fc037040 con 0x7f830c1075c0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.678+0000 7f83120ad640 1 -- 172.21.15.5:0/659061100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c19ea80 con 0x7f830c1075c0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.679+0000 7f8308ff9640 1 -- 172.21.15.5:0/659061100 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f82fc045070 con 0x7f830c1075c0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.679+0000 7f83120ad640 1 -- 172.21.15.5:0/659061100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f830c10bcf0 con 0x7f830c1075c0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.679+0000 7f8308ff9640 1 --2- 172.21.15.5:0/659061100 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f82f40391b0 0x7f82f403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.679+0000 7f8308ff9640 1 -- 172.21.15.5:0/659061100 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f82fc06c540 con 0x7f830c1075c0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.679+0000 7f830affd640 1 --2- 172.21.15.5:0/659061100 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f82f40391b0 0x7f82f403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.680+0000 7f830affd640 1 --2- 172.21.15.5:0/659061100 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f82f40391b0 0x7f82f403b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f83000098c0 tx=0x7f8300006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.684+0000 7f8308ff9640 1 -- 172.21.15.5:0/659061100 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f82fc046e00 con 0x7f830c1075c0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:19.862+0000 7f83120ad640 1 -- 172.21.15.5:0/659061100 --> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f830c103fe0 con 0x7f82f40391b0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:20.192+0000 7f8308ff9640 1 -- 172.21.15.5:0/659061100 <== mgr.14118 v2:172.21.15.5:6800/255461399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f830c103fe0 con 0x7f82f40391b0 2024-09-19T05:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:20.194+0000 7f83120ad640 1 -- 172.21.15.5:0/659061100 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f82f40391b0 msgr2=0x7f82f403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:20.194+0000 7f83120ad640 1 --2- 172.21.15.5:0/659061100 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f82f40391b0 0x7f82f403b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f83000098c0 tx=0x7f8300006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:20.194+0000 7f83120ad640 1 -- 172.21.15.5:0/659061100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 msgr2=0x7f830c19b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:20.194+0000 7f83120ad640 1 --2- 172.21.15.5:0/659061100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 0x7f830c19b5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f82fc030520 tx=0x7f82fc032dd0 comp rx=0 tx=0).stop 2024-09-19T05:54:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:20.194+0000 7f83120ad640 1 -- 172.21.15.5:0/659061100 shutdown_connections 2024-09-19T05:54:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:20.194+0000 7f83120ad640 1 --2- 172.21.15.5:0/659061100 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f82f40391b0 0x7f82f403b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:20.194+0000 7f83120ad640 1 --2- 172.21.15.5:0/659061100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c1075c0 0x7f830c19b5c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:20.194+0000 7f83120ad640 1 -- 172.21.15.5:0/659061100 >> 172.21.15.5:0/659061100 conn(0x7f830c102d50 msgr2=0x7f830c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:20.194+0000 7f83120ad640 1 -- 172.21.15.5:0/659061100 shutdown_connections 2024-09-19T05:54:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:20.194+0000 7f83120ad640 1 -- 172.21.15.5:0/659061100 wait complete. 2024-09-19T05:54:21.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:21 smithi005 ceph-mon[30955]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:54:21.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:21 smithi005 ceph-mon[30955]: Generating ssh key... 2024-09-19T05:54:21.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:21 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' 2024-09-19T05:54:21.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:21 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' 2024-09-19T05:54:21.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtqLgIeURvpD/Isv7Ln3I0mfg/xdl2R9z1grX5vfkoUMyw28IaOwPmpHnNRg24I0PvhvJkWh5qXy7KzYedgb4GhztYfVY++4aUZl8maj6uhU5POJohCG9DyxQ+w70S7+Cv3voRA/ZNG5+Klymxk/eN95uLGk8id/W3osznz0IsfvRVO4GMHoE+Ldy1WUQUEjOnujXhs4mkriBRhltYUjlIQn2SuXlGRVQzsDI8U2zV4vfvDKx/2E9qiee83qhn6x8zzkBJ6u/PW1XvghLnqosVlSIM/LWSkPb2TC31Immn/nr34l6bxzxRYAhZAx9qQGYcH+PzZa5Q9+FRBgBPR5IDDkLCatreX26Es41Ru+D1vCF8tJBBtQSby7qceeI2NIphPzvA96S9C3WIBOWifPVQ4WZGtySFcMc3xfx9nDWKZ3oJX2qh0Ixdozmmbe1Ih0hSQtHsz8QGWaOSYm5QOSSSQlX2A7IObzuZLrZjh3uLkqCkng1b5PsxyXYQJwynBR0= ceph-6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:54:21.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.153+0000 7fae4836e640 1 Processor -- start 2024-09-19T05:54:21.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.153+0000 7fae4836e640 1 -- start start 2024-09-19T05:54:21.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.154+0000 7fae4836e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 0x7fae401079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:21.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.154+0000 7fae4836e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae40107f90 con 0x7fae401075c0 2024-09-19T05:54:21.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.154+0000 7fae4610a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 0x7fae401079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:21.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.154+0000 7fae4610a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 0x7fae401079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37110/0 (socket says 172.21.15.5:37110) 2024-09-19T05:54:21.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.154+0000 7fae4610a640 1 -- 172.21.15.5:0/1416170934 learned_addr learned my addr 172.21.15.5:0/1416170934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:21.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.154+0000 7fae4610a640 1 -- 172.21.15.5:0/1416170934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae401087c0 con 0x7fae401075c0 2024-09-19T05:54:21.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.155+0000 7fae4610a640 1 --2- 172.21.15.5:0/1416170934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 0x7fae401079c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009820 tx=0x7fae3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c156a7bb3aeaab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:21.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.155+0000 7fae45108640 1 -- 172.21.15.5:0/1416170934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae3c037070 con 0x7fae401075c0 2024-09-19T05:54:21.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.155+0000 7fae45108640 1 -- 172.21.15.5:0/1416170934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fae3c0328d0 con 0x7fae401075c0 2024-09-19T05:54:21.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.155+0000 7fae45108640 1 -- 172.21.15.5:0/1416170934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0307d0 con 0x7fae401075c0 2024-09-19T05:54:21.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.156+0000 7fae4836e640 1 -- 172.21.15.5:0/1416170934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 msgr2=0x7fae401079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:21.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.156+0000 7fae4836e640 1 --2- 172.21.15.5:0/1416170934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 0x7fae401079c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009820 tx=0x7fae3c02a270 comp rx=0 tx=0).stop 2024-09-19T05:54:21.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.156+0000 7fae4836e640 1 -- 172.21.15.5:0/1416170934 shutdown_connections 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.156+0000 7fae4836e640 1 --2- 172.21.15.5:0/1416170934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 0x7fae401079c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.156+0000 7fae4836e640 1 -- 172.21.15.5:0/1416170934 >> 172.21.15.5:0/1416170934 conn(0x7fae40102d50 msgr2=0x7fae40105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.156+0000 7fae4836e640 1 -- 172.21.15.5:0/1416170934 shutdown_connections 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.156+0000 7fae4836e640 1 -- 172.21.15.5:0/1416170934 wait complete. 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.156+0000 7fae4836e640 1 Processor -- start 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae4836e640 1 -- start start 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae4836e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 0x7fae4019b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae4836e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae4010bb70 con 0x7fae401075c0 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae4610a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 0x7fae4019b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae4610a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 0x7fae4019b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37118/0 (socket says 172.21.15.5:37118) 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae4610a640 1 -- 172.21.15.5:0/13590213 learned_addr learned my addr 172.21.15.5:0/13590213 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae4610a640 1 -- 172.21.15.5:0/13590213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4019bb00 con 0x7fae401075c0 2024-09-19T05:54:21.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae4610a640 1 --2- 172.21.15.5:0/13590213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 0x7fae4019b5c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fae3c030520 tx=0x7fae3c032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:21.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae377fe640 1 -- 172.21.15.5:0/13590213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae3c042070 con 0x7fae401075c0 2024-09-19T05:54:21.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae4836e640 1 -- 172.21.15.5:0/13590213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4019bd90 con 0x7fae401075c0 2024-09-19T05:54:21.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae377fe640 1 -- 172.21.15.5:0/13590213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fae3c030cc0 con 0x7fae401075c0 2024-09-19T05:54:21.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae377fe640 1 -- 172.21.15.5:0/13590213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae3c037040 con 0x7fae401075c0 2024-09-19T05:54:21.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.157+0000 7fae4836e640 1 -- 172.21.15.5:0/13590213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4019ea80 con 0x7fae401075c0 2024-09-19T05:54:21.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.158+0000 7fae4836e640 1 -- 172.21.15.5:0/13590213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4010bcf0 con 0x7fae401075c0 2024-09-19T05:54:21.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.159+0000 7fae377fe640 1 -- 172.21.15.5:0/13590213 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fae3c045070 con 0x7fae401075c0 2024-09-19T05:54:21.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.159+0000 7fae377fe640 1 --2- 172.21.15.5:0/13590213 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fae2c0391b0 0x7fae2c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:21.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.159+0000 7fae377fe640 1 -- 172.21.15.5:0/13590213 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fae3c06c260 con 0x7fae401075c0 2024-09-19T05:54:21.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.159+0000 7fae45909640 1 --2- 172.21.15.5:0/13590213 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fae2c0391b0 0x7fae2c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.159+0000 7fae45909640 1 --2- 172.21.15.5:0/13590213 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fae2c0391b0 0x7fae2c03b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fae300098c0 tx=0x7fae30006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.163+0000 7fae377fe640 1 -- 172.21.15.5:0/13590213 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fae3c045360 con 0x7fae401075c0 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.342+0000 7fae4836e640 1 -- 172.21.15.5:0/13590213 --> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fae40103fe0 con 0x7fae2c0391b0 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.342+0000 7fae377fe640 1 -- 172.21.15.5:0/13590213 <== mgr.14118 v2:172.21.15.5:6800/255461399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fae40103fe0 con 0x7fae2c0391b0 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.346+0000 7fae4836e640 1 -- 172.21.15.5:0/13590213 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fae2c0391b0 msgr2=0x7fae2c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.346+0000 7fae4836e640 1 --2- 172.21.15.5:0/13590213 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fae2c0391b0 0x7fae2c03b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fae300098c0 tx=0x7fae30006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.346+0000 7fae4836e640 1 -- 172.21.15.5:0/13590213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 msgr2=0x7fae4019b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.346+0000 7fae4836e640 1 --2- 172.21.15.5:0/13590213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 0x7fae4019b5c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fae3c030520 tx=0x7fae3c032dd0 comp rx=0 tx=0).stop 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.346+0000 7fae4836e640 1 -- 172.21.15.5:0/13590213 shutdown_connections 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.346+0000 7fae4836e640 1 --2- 172.21.15.5:0/13590213 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fae2c0391b0 0x7fae2c03b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.346+0000 7fae4836e640 1 --2- 172.21.15.5:0/13590213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae401075c0 0x7fae4019b5c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.346+0000 7fae4836e640 1 -- 172.21.15.5:0/13590213 >> 172.21.15.5:0/13590213 conn(0x7fae40102d50 msgr2=0x7fae401038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:21.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.346+0000 7fae4836e640 1 -- 172.21.15.5:0/13590213 shutdown_connections 2024-09-19T05:54:21.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:21.346+0000 7fae4836e640 1 -- 172.21.15.5:0/13590213 wait complete. 2024-09-19T05:54:21.910 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T05:54:21.910 INFO:teuthology.orchestra.run.smithi005.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T05:54:21.910 INFO:teuthology.orchestra.run.smithi005.stdout:Adding host smithi005... 2024-09-19T05:54:22.550 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:22 smithi005 ceph-mon[30955]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:54:23.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:23 smithi005 ceph-mon[30955]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:54:24.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:24 smithi005 ceph-mon[30955]: Deploying cephadm binary to smithi005 2024-09-19T05:54:27.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Added host 'smithi005' with addr '172.21.15.5' 2024-09-19T05:54:27.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.353+0000 7fb69efb1640 1 Processor -- start 2024-09-19T05:54:27.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.354+0000 7fb69efb1640 1 -- start start 2024-09-19T05:54:27.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.354+0000 7fb69efb1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 0x7fb69810a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:27.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.354+0000 7fb69efb1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb6980634d0 con 0x7fb698108020 2024-09-19T05:54:27.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.354+0000 7fb69cd4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 0x7fb69810a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:27.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.354+0000 7fb69cd4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 0x7fb69810a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37120/0 (socket says 172.21.15.5:37120) 2024-09-19T05:54:27.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.354+0000 7fb69cd4d640 1 -- 172.21.15.5:0/3223760433 learned_addr learned my addr 172.21.15.5:0/3223760433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:27.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.354+0000 7fb69cd4d640 1 -- 172.21.15.5:0/3223760433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69810a950 con 0x7fb698108020 2024-09-19T05:54:27.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.355+0000 7fb69cd4d640 1 --2- 172.21.15.5:0/3223760433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 0x7fb69810a410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb680009820 tx=0x7fb68002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d518d27d4f4681 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:27.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.355+0000 7fb6977fe640 1 -- 172.21.15.5:0/3223760433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb680037070 con 0x7fb698108020 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.355+0000 7fb6977fe640 1 -- 172.21.15.5:0/3223760433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb6800328d0 con 0x7fb698108020 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.355+0000 7fb6977fe640 1 -- 172.21.15.5:0/3223760433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6800307d0 con 0x7fb698108020 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.356+0000 7fb69efb1640 1 -- 172.21.15.5:0/3223760433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 msgr2=0x7fb69810a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.356+0000 7fb69efb1640 1 --2- 172.21.15.5:0/3223760433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 0x7fb69810a410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb680009820 tx=0x7fb68002a270 comp rx=0 tx=0).stop 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.356+0000 7fb69efb1640 1 -- 172.21.15.5:0/3223760433 shutdown_connections 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.356+0000 7fb69efb1640 1 --2- 172.21.15.5:0/3223760433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 0x7fb69810a410 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.356+0000 7fb69efb1640 1 -- 172.21.15.5:0/3223760433 >> 172.21.15.5:0/3223760433 conn(0x7fb698101c00 msgr2=0x7fb698104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.356+0000 7fb69efb1640 1 -- 172.21.15.5:0/3223760433 shutdown_connections 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.356+0000 7fb69efb1640 1 -- 172.21.15.5:0/3223760433 wait complete. 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.357+0000 7fb69efb1640 1 Processor -- start 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.357+0000 7fb69efb1640 1 -- start start 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.357+0000 7fb69efb1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 0x7fb69819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.357+0000 7fb69efb1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb69810b930 con 0x7fb698108020 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.357+0000 7fb69cd4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 0x7fb69819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.357+0000 7fb69cd4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 0x7fb69819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37126/0 (socket says 172.21.15.5:37126) 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.357+0000 7fb69cd4d640 1 -- 172.21.15.5:0/692531262 learned_addr learned my addr 172.21.15.5:0/692531262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.357+0000 7fb69cd4d640 1 -- 172.21.15.5:0/692531262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69819dcf0 con 0x7fb698108020 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.357+0000 7fb69cd4d640 1 --2- 172.21.15.5:0/692531262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 0x7fb69819d7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb680032e10 tx=0x7fb680032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.358+0000 7fb695ffb640 1 -- 172.21.15.5:0/692531262 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb680042070 con 0x7fb698108020 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.358+0000 7fb695ffb640 1 -- 172.21.15.5:0/692531262 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb680030c80 con 0x7fb698108020 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.358+0000 7fb69efb1640 1 -- 172.21.15.5:0/692531262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69819df80 con 0x7fb698108020 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.358+0000 7fb695ffb640 1 -- 172.21.15.5:0/692531262 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb680037040 con 0x7fb698108020 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.358+0000 7fb69efb1640 1 -- 172.21.15.5:0/692531262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6981a0c70 con 0x7fb698108020 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.359+0000 7fb695ffb640 1 -- 172.21.15.5:0/692531262 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb680045070 con 0x7fb698108020 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.359+0000 7fb69efb1640 1 -- 172.21.15.5:0/692531262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb698108480 con 0x7fb698108020 2024-09-19T05:54:27.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.359+0000 7fb695ffb640 1 --2- 172.21.15.5:0/692531262 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fb684039200 0x7fb68403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:27.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.359+0000 7fb695ffb640 1 -- 172.21.15.5:0/692531262 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb68006c1a0 con 0x7fb698108020 2024-09-19T05:54:27.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.359+0000 7fb697fff640 1 --2- 172.21.15.5:0/692531262 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fb684039200 0x7fb68403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:27.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.360+0000 7fb697fff640 1 --2- 172.21.15.5:0/692531262 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fb684039200 0x7fb68403b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb68800ad30 tx=0x7fb6880093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:27.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.364+0000 7fb695ffb640 1 -- 172.21.15.5:0/692531262 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb680045360 con 0x7fb698108020 2024-09-19T05:54:27.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:22.547+0000 7fb69efb1640 1 -- 172.21.15.5:0/692531262 --> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}) -- 0x7fb698102e70 con 0x7fb684039200 2024-09-19T05:54:27.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:26.418+0000 7fb695ffb640 1 -- 172.21.15.5:0/692531262 <== mgr.14118 v2:172.21.15.5:6800/255461399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fb698102e70 con 0x7fb684039200 2024-09-19T05:54:27.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:26.421+0000 7fb69efb1640 1 -- 172.21.15.5:0/692531262 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fb684039200 msgr2=0x7fb68403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:27.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:26.421+0000 7fb69efb1640 1 --2- 172.21.15.5:0/692531262 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fb684039200 0x7fb68403b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb68800ad30 tx=0x7fb6880093f0 comp rx=0 tx=0).stop 2024-09-19T05:54:27.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:26.421+0000 7fb69efb1640 1 -- 172.21.15.5:0/692531262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 msgr2=0x7fb69819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:27.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:26.421+0000 7fb69efb1640 1 --2- 172.21.15.5:0/692531262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 0x7fb69819d7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb680032e10 tx=0x7fb680032e40 comp rx=0 tx=0).stop 2024-09-19T05:54:27.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:26.421+0000 7fb69efb1640 1 -- 172.21.15.5:0/692531262 shutdown_connections 2024-09-19T05:54:27.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:26.421+0000 7fb69efb1640 1 --2- 172.21.15.5:0/692531262 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7fb684039200 0x7fb68403b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:27.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:26.421+0000 7fb69efb1640 1 --2- 172.21.15.5:0/692531262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb698108020 0x7fb69819d7b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:27.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:26.421+0000 7fb69efb1640 1 -- 172.21.15.5:0/692531262 >> 172.21.15.5:0/692531262 conn(0x7fb698101c00 msgr2=0x7fb698102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:27.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:26.421+0000 7fb69efb1640 1 -- 172.21.15.5:0/692531262 shutdown_connections 2024-09-19T05:54:27.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:26.421+0000 7fb69efb1640 1 -- 172.21.15.5:0/692531262 wait complete. 2024-09-19T05:54:27.034 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mon service... 2024-09-19T05:54:27.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:27 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' 2024-09-19T05:54:27.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:27 smithi005 ceph-mon[30955]: Added host smithi005 2024-09-19T05:54:27.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:27 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:54:28.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T05:54:28.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.417+0000 7f430c38a640 1 Processor -- start 2024-09-19T05:54:28.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.417+0000 7f430c38a640 1 -- start start 2024-09-19T05:54:28.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.417+0000 7f430c38a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 0x7f430410a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:28.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.417+0000 7f430c38a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4304063510 con 0x7f4304107fe0 2024-09-19T05:54:28.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.417+0000 7f430a126640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 0x7f430410a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:28.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.417+0000 7f430a126640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 0x7f430410a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39800/0 (socket says 172.21.15.5:39800) 2024-09-19T05:54:28.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.417+0000 7f430a126640 1 -- 172.21.15.5:0/917190927 learned_addr learned my addr 172.21.15.5:0/917190927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.418+0000 7f430a126640 1 -- 172.21.15.5:0/917190927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430410a910 con 0x7f4304107fe0 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.418+0000 7f430a126640 1 --2- 172.21.15.5:0/917190927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 0x7f430410a3d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4300009820 tx=0x7f430002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f80bba33b3bac33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.419+0000 7f4309124640 1 -- 172.21.15.5:0/917190927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4300037070 con 0x7f4304107fe0 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.419+0000 7f4309124640 1 -- 172.21.15.5:0/917190927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f43000328d0 con 0x7f4304107fe0 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.419+0000 7f4309124640 1 -- 172.21.15.5:0/917190927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43000307d0 con 0x7f4304107fe0 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.420+0000 7f430c38a640 1 -- 172.21.15.5:0/917190927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 msgr2=0x7f430410a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.420+0000 7f430c38a640 1 --2- 172.21.15.5:0/917190927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 0x7f430410a3d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4300009820 tx=0x7f430002a270 comp rx=0 tx=0).stop 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.420+0000 7f430c38a640 1 -- 172.21.15.5:0/917190927 shutdown_connections 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.420+0000 7f430c38a640 1 --2- 172.21.15.5:0/917190927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 0x7f430410a3d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.420+0000 7f430c38a640 1 -- 172.21.15.5:0/917190927 >> 172.21.15.5:0/917190927 conn(0x7f4304101c00 msgr2=0x7f4304104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.420+0000 7f430c38a640 1 -- 172.21.15.5:0/917190927 shutdown_connections 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.420+0000 7f430c38a640 1 -- 172.21.15.5:0/917190927 wait complete. 2024-09-19T05:54:28.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.421+0000 7f430c38a640 1 Processor -- start 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.421+0000 7f430c38a640 1 -- start start 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.421+0000 7f430c38a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 0x7f430419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.421+0000 7f430c38a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f430410b950 con 0x7f4304107fe0 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.421+0000 7f430a126640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 0x7f430419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.421+0000 7f430a126640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 0x7f430419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39814/0 (socket says 172.21.15.5:39814) 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.421+0000 7f430a126640 1 -- 172.21.15.5:0/2492569557 learned_addr learned my addr 172.21.15.5:0/2492569557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.421+0000 7f430a126640 1 -- 172.21.15.5:0/2492569557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430419dc50 con 0x7f4304107fe0 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.421+0000 7f430a126640 1 --2- 172.21.15.5:0/2492569557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 0x7f430419d710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4300032e10 tx=0x7f4300032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.422+0000 7f42fb7fe640 1 -- 172.21.15.5:0/2492569557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4300042070 con 0x7f4304107fe0 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.422+0000 7f430c38a640 1 -- 172.21.15.5:0/2492569557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430419dee0 con 0x7f4304107fe0 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.422+0000 7f42fb7fe640 1 -- 172.21.15.5:0/2492569557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4300030c80 con 0x7f4304107fe0 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.422+0000 7f42fb7fe640 1 -- 172.21.15.5:0/2492569557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4300037040 con 0x7f4304107fe0 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.422+0000 7f430c38a640 1 -- 172.21.15.5:0/2492569557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43041a0bd0 con 0x7f4304107fe0 2024-09-19T05:54:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.422+0000 7f42fb7fe640 1 -- 172.21.15.5:0/2492569557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4300045070 con 0x7f4304107fe0 2024-09-19T05:54:28.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.423+0000 7f42fb7fe640 1 --2- 172.21.15.5:0/2492569557 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f42f00391b0 0x7f42f003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:28.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.423+0000 7f42fb7fe640 1 -- 172.21.15.5:0/2492569557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f430006c270 con 0x7f4304107fe0 2024-09-19T05:54:28.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.423+0000 7f430c38a640 1 -- 172.21.15.5:0/2492569557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4304108440 con 0x7f4304107fe0 2024-09-19T05:54:28.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.423+0000 7f4309925640 1 --2- 172.21.15.5:0/2492569557 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f42f00391b0 0x7f42f003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:28.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.423+0000 7f4309925640 1 --2- 172.21.15.5:0/2492569557 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f42f00391b0 0x7f42f003b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f42f40098c0 tx=0x7f42f4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:28.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.428+0000 7f42fb7fe640 1 -- 172.21.15.5:0/2492569557 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4300045360 con 0x7f4304107fe0 2024-09-19T05:54:28.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.639+0000 7f430c38a640 1 -- 172.21.15.5:0/2492569557 --> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4304102f10 con 0x7f42f00391b0 2024-09-19T05:54:28.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.643+0000 7f42fb7fe640 1 -- 172.21.15.5:0/2492569557 <== mgr.14118 v2:172.21.15.5:6800/255461399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4304102f10 con 0x7f42f00391b0 2024-09-19T05:54:28.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.645+0000 7f430c38a640 1 -- 172.21.15.5:0/2492569557 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f42f00391b0 msgr2=0x7f42f003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:28.467 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.645+0000 7f430c38a640 1 --2- 172.21.15.5:0/2492569557 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f42f00391b0 0x7f42f003b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f42f40098c0 tx=0x7f42f4006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:28.467 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.645+0000 7f430c38a640 1 -- 172.21.15.5:0/2492569557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 msgr2=0x7f430419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:28.467 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.645+0000 7f430c38a640 1 --2- 172.21.15.5:0/2492569557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 0x7f430419d710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4300032e10 tx=0x7f4300032e40 comp rx=0 tx=0).stop 2024-09-19T05:54:28.467 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.646+0000 7f430c38a640 1 -- 172.21.15.5:0/2492569557 shutdown_connections 2024-09-19T05:54:28.467 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.646+0000 7f430c38a640 1 --2- 172.21.15.5:0/2492569557 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f42f00391b0 0x7f42f003b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:28.467 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.646+0000 7f430c38a640 1 --2- 172.21.15.5:0/2492569557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4304107fe0 0x7f430419d710 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:28.467 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.646+0000 7f430c38a640 1 -- 172.21.15.5:0/2492569557 >> 172.21.15.5:0/2492569557 conn(0x7f4304101c00 msgr2=0x7f4304102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:28.467 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.646+0000 7f430c38a640 1 -- 172.21.15.5:0/2492569557 shutdown_connections 2024-09-19T05:54:28.467 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:27.646+0000 7f430c38a640 1 -- 172.21.15.5:0/2492569557 wait complete. 2024-09-19T05:54:28.467 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mgr service... 2024-09-19T05:54:28.709 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:28 smithi005 ceph-mon[30955]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:54:28.709 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:28 smithi005 ceph-mon[30955]: Saving service mon spec with placement count:5 2024-09-19T05:54:28.709 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:28 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' 2024-09-19T05:54:29.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T05:54:29.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.879+0000 7f7eb82f7640 1 Processor -- start 2024-09-19T05:54:29.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.879+0000 7f7eb82f7640 1 -- start start 2024-09-19T05:54:29.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.879+0000 7f7eb82f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 0x7f7eb01079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:29.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.879+0000 7f7eb82f7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0107f90 con 0x7f7eb01075c0 2024-09-19T05:54:29.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.879+0000 7f7eb6093640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 0x7f7eb01079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:29.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.879+0000 7f7eb6093640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 0x7f7eb01079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39820/0 (socket says 172.21.15.5:39820) 2024-09-19T05:54:29.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.879+0000 7f7eb6093640 1 -- 172.21.15.5:0/599499780 learned_addr learned my addr 172.21.15.5:0/599499780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.880+0000 7f7eb6093640 1 -- 172.21.15.5:0/599499780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb01087c0 con 0x7f7eb01075c0 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.880+0000 7f7eb6093640 1 --2- 172.21.15.5:0/599499780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 0x7f7eb01079c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009820 tx=0x7f7eac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbf3c9a154d11ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.881+0000 7f7eb5091640 1 -- 172.21.15.5:0/599499780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eac037070 con 0x7f7eb01075c0 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.881+0000 7f7eb5091640 1 -- 172.21.15.5:0/599499780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7eac0328d0 con 0x7f7eb01075c0 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.881+0000 7f7eb5091640 1 -- 172.21.15.5:0/599499780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eac0307d0 con 0x7f7eb01075c0 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.881+0000 7f7eb82f7640 1 -- 172.21.15.5:0/599499780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 msgr2=0x7f7eb01079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.881+0000 7f7eb82f7640 1 --2- 172.21.15.5:0/599499780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 0x7f7eb01079c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009820 tx=0x7f7eac02a270 comp rx=0 tx=0).stop 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.881+0000 7f7eb82f7640 1 -- 172.21.15.5:0/599499780 shutdown_connections 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.881+0000 7f7eb82f7640 1 --2- 172.21.15.5:0/599499780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 0x7f7eb01079c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.881+0000 7f7eb82f7640 1 -- 172.21.15.5:0/599499780 >> 172.21.15.5:0/599499780 conn(0x7f7eb0102d50 msgr2=0x7f7eb0105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.881+0000 7f7eb82f7640 1 -- 172.21.15.5:0/599499780 shutdown_connections 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.881+0000 7f7eb82f7640 1 -- 172.21.15.5:0/599499780 wait complete. 2024-09-19T05:54:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.882+0000 7f7eb82f7640 1 Processor -- start 2024-09-19T05:54:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.882+0000 7f7eb82f7640 1 -- start start 2024-09-19T05:54:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.882+0000 7f7eb82f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 0x7f7eb019b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.882+0000 7f7eb82f7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb010bb70 con 0x7f7eb01075c0 2024-09-19T05:54:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.882+0000 7f7eb6093640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 0x7f7eb019b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.883+0000 7f7eb6093640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 0x7f7eb019b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39836/0 (socket says 172.21.15.5:39836) 2024-09-19T05:54:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.883+0000 7f7eb6093640 1 -- 172.21.15.5:0/2434203390 learned_addr learned my addr 172.21.15.5:0/2434203390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.883+0000 7f7eb6093640 1 -- 172.21.15.5:0/2434203390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb019bb00 con 0x7f7eb01075c0 2024-09-19T05:54:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.883+0000 7f7eb6093640 1 --2- 172.21.15.5:0/2434203390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 0x7f7eb019b5c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0097f0 tx=0x7f7eac030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.883+0000 7f7ea77fe640 1 -- 172.21.15.5:0/2434203390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eac037040 con 0x7f7eb01075c0 2024-09-19T05:54:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.883+0000 7f7eb82f7640 1 -- 172.21.15.5:0/2434203390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb019bd90 con 0x7f7eb01075c0 2024-09-19T05:54:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.883+0000 7f7ea77fe640 1 -- 172.21.15.5:0/2434203390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7eac032bb0 con 0x7f7eb01075c0 2024-09-19T05:54:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.883+0000 7f7ea77fe640 1 -- 172.21.15.5:0/2434203390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eac03a390 con 0x7f7eb01075c0 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.883+0000 7f7eb82f7640 1 -- 172.21.15.5:0/2434203390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb019ea80 con 0x7f7eb01075c0 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.884+0000 7f7ea77fe640 1 -- 172.21.15.5:0/2434203390 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f7eac03a5d0 con 0x7f7eb01075c0 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.884+0000 7f7eb82f7640 1 -- 172.21.15.5:0/2434203390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb010bcf0 con 0x7f7eb01075c0 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.884+0000 7f7ea77fe640 1 --2- 172.21.15.5:0/2434203390 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f7e9c0391b0 0x7f7e9c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.885+0000 7f7ea77fe640 1 -- 172.21.15.5:0/2434203390 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7eac06c5e0 con 0x7f7eb01075c0 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.885+0000 7f7eb5892640 1 --2- 172.21.15.5:0/2434203390 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f7e9c0391b0 0x7f7e9c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.885+0000 7f7eb5892640 1 --2- 172.21.15.5:0/2434203390 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f7e9c0391b0 0x7f7e9c03b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00098c0 tx=0x7f7ea0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:28.890+0000 7f7ea77fe640 1 -- 172.21.15.5:0/2434203390 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7eac032dd0 con 0x7f7eb01075c0 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:29.086+0000 7f7eb82f7640 1 -- 172.21.15.5:0/2434203390 --> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7eb0103fe0 con 0x7f7e9c0391b0 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:29.090+0000 7f7ea77fe640 1 -- 172.21.15.5:0/2434203390 <== mgr.14118 v2:172.21.15.5:6800/255461399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7eb0103fe0 con 0x7f7e9c0391b0 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:29.092+0000 7f7eb82f7640 1 -- 172.21.15.5:0/2434203390 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f7e9c0391b0 msgr2=0x7f7e9c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:29.092+0000 7f7eb82f7640 1 --2- 172.21.15.5:0/2434203390 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f7e9c0391b0 0x7f7e9c03b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00098c0 tx=0x7f7ea0006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:29.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:29.092+0000 7f7eb82f7640 1 -- 172.21.15.5:0/2434203390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 msgr2=0x7f7eb019b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:29.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:29.092+0000 7f7eb82f7640 1 --2- 172.21.15.5:0/2434203390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 0x7f7eb019b5c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0097f0 tx=0x7f7eac030d00 comp rx=0 tx=0).stop 2024-09-19T05:54:29.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:29.092+0000 7f7eb82f7640 1 -- 172.21.15.5:0/2434203390 shutdown_connections 2024-09-19T05:54:29.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:29.092+0000 7f7eb82f7640 1 --2- 172.21.15.5:0/2434203390 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f7e9c0391b0 0x7f7e9c03b670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:29.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:29.092+0000 7f7eb82f7640 1 --2- 172.21.15.5:0/2434203390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb01075c0 0x7f7eb019b5c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:29.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:29.092+0000 7f7eb82f7640 1 -- 172.21.15.5:0/2434203390 >> 172.21.15.5:0/2434203390 conn(0x7f7eb0102d50 msgr2=0x7f7eb01038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:29.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:29.092+0000 7f7eb82f7640 1 -- 172.21.15.5:0/2434203390 shutdown_connections 2024-09-19T05:54:29.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:29.092+0000 7f7eb82f7640 1 -- 172.21.15.5:0/2434203390 wait complete. 2024-09-19T05:54:30.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:30 smithi005 ceph-mon[30955]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:54:30.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:30 smithi005 ceph-mon[30955]: Saving service mgr spec with placement count:2 2024-09-19T05:54:30.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:30 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' 2024-09-19T05:54:31.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.268+0000 7f96dab5d640 1 Processor -- start 2024-09-19T05:54:31.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.268+0000 7f96dab5d640 1 -- start start 2024-09-19T05:54:31.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.268+0000 7f96dab5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 0x7f96d407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.268+0000 7f96dab5d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96d407c8a0 con 0x7f96d407de70 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.268+0000 7f96d88f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 0x7f96d407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.268+0000 7f96d88f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 0x7f96d407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39852/0 (socket says 172.21.15.5:39852) 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.268+0000 7f96d88f9640 1 -- 172.21.15.5:0/3256153214 learned_addr learned my addr 172.21.15.5:0/3256153214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.268+0000 7f96d88f9640 1 -- 172.21.15.5:0/3256153214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d407ca20 con 0x7f96d407de70 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.269+0000 7f96d88f9640 1 --2- 172.21.15.5:0/3256153214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 0x7f96d407c2d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009820 tx=0x7f96c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47314474ca7dd94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.269+0000 7f96d37fe640 1 -- 172.21.15.5:0/3256153214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c4037070 con 0x7f96d407de70 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.269+0000 7f96d37fe640 1 -- 172.21.15.5:0/3256153214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f96c40328d0 con 0x7f96d407de70 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.269+0000 7f96d37fe640 1 -- 172.21.15.5:0/3256153214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c40307d0 con 0x7f96d407de70 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.270+0000 7f96dab5d640 1 -- 172.21.15.5:0/3256153214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 msgr2=0x7f96d407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.270+0000 7f96dab5d640 1 --2- 172.21.15.5:0/3256153214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 0x7f96d407c2d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009820 tx=0x7f96c402a270 comp rx=0 tx=0).stop 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.270+0000 7f96dab5d640 1 -- 172.21.15.5:0/3256153214 shutdown_connections 2024-09-19T05:54:31.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.270+0000 7f96dab5d640 1 --2- 172.21.15.5:0/3256153214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 0x7f96d407c2d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.270+0000 7f96dab5d640 1 -- 172.21.15.5:0/3256153214 >> 172.21.15.5:0/3256153214 conn(0x7f96d4102d50 msgr2=0x7f96d4105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.270+0000 7f96dab5d640 1 -- 172.21.15.5:0/3256153214 shutdown_connections 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.270+0000 7f96dab5d640 1 -- 172.21.15.5:0/3256153214 wait complete. 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.271+0000 7f96dab5d640 1 Processor -- start 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.271+0000 7f96dab5d640 1 -- start start 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.271+0000 7f96dab5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 0x7f96d419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.271+0000 7f96dab5d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96d410db90 con 0x7f96d407de70 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.271+0000 7f96d88f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 0x7f96d419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.271+0000 7f96d88f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 0x7f96d419d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39862/0 (socket says 172.21.15.5:39862) 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.271+0000 7f96d88f9640 1 -- 172.21.15.5:0/3398449294 learned_addr learned my addr 172.21.15.5:0/3398449294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.271+0000 7f96d88f9640 1 -- 172.21.15.5:0/3398449294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d419dd00 con 0x7f96d407de70 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.272+0000 7f96d88f9640 1 --2- 172.21.15.5:0/3398449294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 0x7f96d419d7c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f96c40097f0 tx=0x7f96c4030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:31.015 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.272+0000 7f96d1ffb640 1 -- 172.21.15.5:0/3398449294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c4037040 con 0x7f96d407de70 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.272+0000 7f96d1ffb640 1 -- 172.21.15.5:0/3398449294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f96c403b3f0 con 0x7f96d407de70 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.272+0000 7f96dab5d640 1 -- 172.21.15.5:0/3398449294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d419df90 con 0x7f96d407de70 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.272+0000 7f96d1ffb640 1 -- 172.21.15.5:0/3398449294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c403a520 con 0x7f96d407de70 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.272+0000 7f96dab5d640 1 -- 172.21.15.5:0/3398449294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d41a0c80 con 0x7f96d407de70 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.273+0000 7f96d1ffb640 1 -- 172.21.15.5:0/3398449294 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f96c403a6c0 con 0x7f96d407de70 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.273+0000 7f96d1ffb640 1 --2- 172.21.15.5:0/3398449294 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f96bc039200 0x7f96bc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.273+0000 7f96d1ffb640 1 -- 172.21.15.5:0/3398449294 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f96c406c460 con 0x7f96d407de70 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.273+0000 7f96dab5d640 1 -- 172.21.15.5:0/3398449294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d407c350 con 0x7f96d407de70 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.273+0000 7f96d3fff640 1 --2- 172.21.15.5:0/3398449294 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f96bc039200 0x7f96bc03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.274+0000 7f96d3fff640 1 --2- 172.21.15.5:0/3398449294 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f96bc039200 0x7f96bc03b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f96c80098c0 tx=0x7f96c8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.278+0000 7f96d1ffb640 1 -- 172.21.15.5:0/3398449294 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f96c4032820 con 0x7f96d407de70 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.455+0000 7f96dab5d640 1 -- 172.21.15.5:0/3398449294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f96d41a12d0 con 0x7f96d407de70 2024-09-19T05:54:31.016 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.457+0000 7f96d1ffb640 1 -- 172.21.15.5:0/3398449294 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f96c40319b0 con 0x7f96d407de70 2024-09-19T05:54:31.017 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.460+0000 7f96dab5d640 1 -- 172.21.15.5:0/3398449294 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f96bc039200 msgr2=0x7f96bc03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:31.017 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.460+0000 7f96dab5d640 1 --2- 172.21.15.5:0/3398449294 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f96bc039200 0x7f96bc03b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f96c80098c0 tx=0x7f96c8006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.017 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.460+0000 7f96dab5d640 1 -- 172.21.15.5:0/3398449294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 msgr2=0x7f96d419d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:31.017 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.460+0000 7f96dab5d640 1 --2- 172.21.15.5:0/3398449294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 0x7f96d419d7c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f96c40097f0 tx=0x7f96c4030d00 comp rx=0 tx=0).stop 2024-09-19T05:54:31.017 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.460+0000 7f96dab5d640 1 -- 172.21.15.5:0/3398449294 shutdown_connections 2024-09-19T05:54:31.017 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.460+0000 7f96dab5d640 1 --2- 172.21.15.5:0/3398449294 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f96bc039200 0x7f96bc03b6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.017 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.460+0000 7f96dab5d640 1 --2- 172.21.15.5:0/3398449294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d407de70 0x7f96d419d7c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.017 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.460+0000 7f96dab5d640 1 -- 172.21.15.5:0/3398449294 >> 172.21.15.5:0/3398449294 conn(0x7f96d4102d50 msgr2=0x7f96d4103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:31.017 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.460+0000 7f96dab5d640 1 -- 172.21.15.5:0/3398449294 shutdown_connections 2024-09-19T05:54:31.017 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:30.460+0000 7f96dab5d640 1 -- 172.21.15.5:0/3398449294 wait complete. 2024-09-19T05:54:31.588 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:31 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3398449294' entity='client.admin' 2024-09-19T05:54:32.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:32 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' 2024-09-19T05:54:32.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:32 smithi005 ceph-mon[30955]: from='mgr.14118 172.21.15.5:0/1990046460' entity='mgr.x' 2024-09-19T05:54:32.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:32 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3739425097' entity='client.admin' 2024-09-19T05:54:32.966 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.112+0000 7f1e725fd640 1 Processor -- start 2024-09-19T05:54:32.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.112+0000 7f1e725fd640 1 -- start start 2024-09-19T05:54:32.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.112+0000 7f1e725fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 0x7f1e6c10a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:32.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.112+0000 7f1e725fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c0634d0 con 0x7f1e6c108020 2024-09-19T05:54:32.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.113+0000 7f1e6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 0x7f1e6c10a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:32.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.113+0000 7f1e6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 0x7f1e6c10a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39872/0 (socket says 172.21.15.5:39872) 2024-09-19T05:54:32.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.113+0000 7f1e6bfff640 1 -- 172.21.15.5:0/3558926426 learned_addr learned my addr 172.21.15.5:0/3558926426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:32.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.113+0000 7f1e6bfff640 1 -- 172.21.15.5:0/3558926426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c10a950 con 0x7f1e6c108020 2024-09-19T05:54:32.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.114+0000 7f1e6bfff640 1 --2- 172.21.15.5:0/3558926426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 0x7f1e6c10a410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009820 tx=0x7f1e5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2bd58e12f6126e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:32.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.114+0000 7f1e6b7fe640 1 -- 172.21.15.5:0/3558926426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e5c037070 con 0x7f1e6c108020 2024-09-19T05:54:32.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.114+0000 7f1e6b7fe640 1 -- 172.21.15.5:0/3558926426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e5c0328f0 con 0x7f1e6c108020 2024-09-19T05:54:32.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.114+0000 7f1e6b7fe640 1 -- 172.21.15.5:0/3558926426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e5c030740 con 0x7f1e6c108020 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.115+0000 7f1e725fd640 1 -- 172.21.15.5:0/3558926426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 msgr2=0x7f1e6c10a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.115+0000 7f1e725fd640 1 --2- 172.21.15.5:0/3558926426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 0x7f1e6c10a410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009820 tx=0x7f1e5c02a270 comp rx=0 tx=0).stop 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.115+0000 7f1e725fd640 1 -- 172.21.15.5:0/3558926426 shutdown_connections 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.115+0000 7f1e725fd640 1 --2- 172.21.15.5:0/3558926426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 0x7f1e6c10a410 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.115+0000 7f1e725fd640 1 -- 172.21.15.5:0/3558926426 >> 172.21.15.5:0/3558926426 conn(0x7f1e6c101c00 msgr2=0x7f1e6c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.115+0000 7f1e725fd640 1 -- 172.21.15.5:0/3558926426 shutdown_connections 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.115+0000 7f1e725fd640 1 -- 172.21.15.5:0/3558926426 wait complete. 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.115+0000 7f1e725fd640 1 Processor -- start 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.115+0000 7f1e725fd640 1 -- start start 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.116+0000 7f1e725fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 0x7f1e6c19b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.116+0000 7f1e725fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c10b970 con 0x7f1e6c108020 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.116+0000 7f1e6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 0x7f1e6c19b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:32.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.116+0000 7f1e6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 0x7f1e6c19b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39882/0 (socket says 172.21.15.5:39882) 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.116+0000 7f1e6bfff640 1 -- 172.21.15.5:0/3739425097 learned_addr learned my addr 172.21.15.5:0/3739425097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.116+0000 7f1e6bfff640 1 -- 172.21.15.5:0/3739425097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c19ba90 con 0x7f1e6c108020 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.116+0000 7f1e6bfff640 1 --2- 172.21.15.5:0/3739425097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 0x7f1e6c19b550 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c030cc0 tx=0x7f1e5c030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.116+0000 7f1e69ffb640 1 -- 172.21.15.5:0/3739425097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e5c042070 con 0x7f1e6c108020 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.116+0000 7f1e69ffb640 1 -- 172.21.15.5:0/3739425097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e5c030e60 con 0x7f1e6c108020 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.116+0000 7f1e725fd640 1 -- 172.21.15.5:0/3739425097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6c19bd20 con 0x7f1e6c108020 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.116+0000 7f1e725fd640 1 -- 172.21.15.5:0/3739425097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6c19ea10 con 0x7f1e6c108020 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.117+0000 7f1e69ffb640 1 -- 172.21.15.5:0/3739425097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e5c037040 con 0x7f1e6c108020 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.117+0000 7f1e69ffb640 1 -- 172.21.15.5:0/3739425097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f1e5c045070 con 0x7f1e6c108020 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.118+0000 7f1e69ffb640 1 --2- 172.21.15.5:0/3739425097 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f1e580391b0 0x7f1e5803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.118+0000 7f1e69ffb640 1 -- 172.21.15.5:0/3739425097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1e5c06d270 con 0x7f1e6c108020 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.118+0000 7f1e63fff640 1 --2- 172.21.15.5:0/3739425097 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f1e580391b0 0x7f1e5803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:32.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.118+0000 7f1e725fd640 1 -- 172.21.15.5:0/3739425097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4c0052c0 con 0x7f1e6c108020 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.118+0000 7f1e63fff640 1 --2- 172.21.15.5:0/3739425097 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f1e580391b0 0x7f1e5803b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e540098c0 tx=0x7f1e54006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.123+0000 7f1e69ffb640 1 -- 172.21.15.5:0/3739425097 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1e5c045360 con 0x7f1e6c108020 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.319+0000 7f1e725fd640 1 -- 172.21.15.5:0/3739425097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f1e4c0055b0 con 0x7f1e6c108020 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.322+0000 7f1e69ffb640 1 -- 172.21.15.5:0/3739425097 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f1e5c02ed90 con 0x7f1e6c108020 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.324+0000 7f1e725fd640 1 -- 172.21.15.5:0/3739425097 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f1e580391b0 msgr2=0x7f1e5803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.324+0000 7f1e725fd640 1 --2- 172.21.15.5:0/3739425097 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f1e580391b0 0x7f1e5803b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e540098c0 tx=0x7f1e54006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.324+0000 7f1e725fd640 1 -- 172.21.15.5:0/3739425097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 msgr2=0x7f1e6c19b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.324+0000 7f1e725fd640 1 --2- 172.21.15.5:0/3739425097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 0x7f1e6c19b550 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c030cc0 tx=0x7f1e5c030cf0 comp rx=0 tx=0).stop 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.325+0000 7f1e725fd640 1 -- 172.21.15.5:0/3739425097 shutdown_connections 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.325+0000 7f1e725fd640 1 --2- 172.21.15.5:0/3739425097 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f1e580391b0 0x7f1e5803b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.325+0000 7f1e725fd640 1 --2- 172.21.15.5:0/3739425097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c108020 0x7f1e6c19b550 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.325+0000 7f1e725fd640 1 -- 172.21.15.5:0/3739425097 >> 172.21.15.5:0/3739425097 conn(0x7f1e6c101c00 msgr2=0x7f1e6c1027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.325+0000 7f1e725fd640 1 -- 172.21.15.5:0/3739425097 shutdown_connections 2024-09-19T05:54:32.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:32.325+0000 7f1e725fd640 1 -- 172.21.15.5:0/3739425097 wait complete. 2024-09-19T05:54:32.971 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling the dashboard module... 2024-09-19T05:54:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:33 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1644033057' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T05:54:33.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:33 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: ignoring --setuser ceph since I am not root 2024-09-19T05:54:33.880 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:33 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: ignoring --setgroup ceph since I am not root 2024-09-19T05:54:33.880 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:33 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:33.787+0000 7f22bb765040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:54:33.880 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:33 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:33.847+0000 7f22bb765040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:54:34.350 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:34 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:34.003+0000 7f22bb765040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:54:34.351 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:34 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:34.072+0000 7f22bb765040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:54:34.351 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:34 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:34.227+0000 7f22bb765040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:54:34.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.383+0000 7f676e99e640 1 Processor -- start 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.384+0000 7f676e99e640 1 -- start start 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.384+0000 7f676e99e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 0x7f6768107db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.384+0000 7f676e99e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6768105ef0 con 0x7f67681079b0 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.384+0000 7f676d99c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 0x7f6768107db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.384+0000 7f676d99c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 0x7f6768107db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39892/0 (socket says 172.21.15.5:39892) 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.384+0000 7f676d99c640 1 -- 172.21.15.5:0/3888046088 learned_addr learned my addr 172.21.15.5:0/3888046088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.385+0000 7f676d99c640 1 -- 172.21.15.5:0/3888046088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6768106070 con 0x7f67681079b0 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.385+0000 7f676d99c640 1 --2- 172.21.15.5:0/3888046088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 0x7f6768107db0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6764009820 tx=0x7f676402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7c0d0d228c1869 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.385+0000 7f676c99a640 1 -- 172.21.15.5:0/3888046088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6764037070 con 0x7f67681079b0 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.386+0000 7f676c99a640 1 -- 172.21.15.5:0/3888046088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67640328f0 con 0x7f67681079b0 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.386+0000 7f676c99a640 1 -- 172.21.15.5:0/3888046088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6764030740 con 0x7f67681079b0 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.386+0000 7f676e99e640 1 -- 172.21.15.5:0/3888046088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 msgr2=0x7f6768107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:34.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.386+0000 7f676e99e640 1 --2- 172.21.15.5:0/3888046088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 0x7f6768107db0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6764009820 tx=0x7f676402a270 comp rx=0 tx=0).stop 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.386+0000 7f676e99e640 1 -- 172.21.15.5:0/3888046088 shutdown_connections 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.386+0000 7f676e99e640 1 --2- 172.21.15.5:0/3888046088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 0x7f6768107db0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.386+0000 7f676e99e640 1 -- 172.21.15.5:0/3888046088 >> 172.21.15.5:0/3888046088 conn(0x7f6768101a60 msgr2=0x7f6768103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.386+0000 7f676e99e640 1 -- 172.21.15.5:0/3888046088 shutdown_connections 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.386+0000 7f676e99e640 1 -- 172.21.15.5:0/3888046088 wait complete. 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.387+0000 7f676e99e640 1 Processor -- start 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.387+0000 7f676e99e640 1 -- start start 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.387+0000 7f676e99e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 0x7f67681915a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.387+0000 7f676e99e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f676810a880 con 0x7f67681079b0 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.387+0000 7f676d99c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 0x7f67681915a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.387+0000 7f676d99c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 0x7f67681915a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39898/0 (socket says 172.21.15.5:39898) 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.387+0000 7f676d99c640 1 -- 172.21.15.5:0/1644033057 learned_addr learned my addr 172.21.15.5:0/1644033057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.388+0000 7f676d99c640 1 -- 172.21.15.5:0/1644033057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6768191ae0 con 0x7f67681079b0 2024-09-19T05:54:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.388+0000 7f676d99c640 1 --2- 172.21.15.5:0/1644033057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 0x7f67681915a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6764009940 tx=0x7f6764030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:34.360 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.388+0000 7f675effd640 1 -- 172.21.15.5:0/1644033057 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6764042070 con 0x7f67681079b0 2024-09-19T05:54:34.360 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.388+0000 7f675effd640 1 -- 172.21.15.5:0/1644033057 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6764030e30 con 0x7f67681079b0 2024-09-19T05:54:34.360 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.388+0000 7f676e99e640 1 -- 172.21.15.5:0/1644033057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676818fcd0 con 0x7f67681079b0 2024-09-19T05:54:34.360 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.388+0000 7f675effd640 1 -- 172.21.15.5:0/1644033057 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6764037040 con 0x7f67681079b0 2024-09-19T05:54:34.360 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.388+0000 7f676e99e640 1 -- 172.21.15.5:0/1644033057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6768190210 con 0x7f67681079b0 2024-09-19T05:54:34.360 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.389+0000 7f675effd640 1 -- 172.21.15.5:0/1644033057 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6764045070 con 0x7f67681079b0 2024-09-19T05:54:34.360 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.389+0000 7f676e99e640 1 -- 172.21.15.5:0/1644033057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6768107e30 con 0x7f67681079b0 2024-09-19T05:54:34.360 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.389+0000 7f675effd640 1 --2- 172.21.15.5:0/1644033057 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f6754039200 0x7f675403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:34.360 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.389+0000 7f675effd640 1 -- 172.21.15.5:0/1644033057 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f676406c360 con 0x7f67681079b0 2024-09-19T05:54:34.360 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.390+0000 7f676d19b640 1 --2- 172.21.15.5:0/1644033057 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f6754039200 0x7f675403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:34.360 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.390+0000 7f676d19b640 1 --2- 172.21.15.5:0/1644033057 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f6754039200 0x7f675403b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f67580098c0 tx=0x7f6758006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:34.360 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.394+0000 7f675effd640 1 -- 172.21.15.5:0/1644033057 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f676404b170 con 0x7f67681079b0 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.618+0000 7f676e99e640 1 -- 172.21.15.5:0/1644033057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f67681951d0 con 0x7f67681079b0 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.625+0000 7f675effd640 1 -- 172.21.15.5:0/1644033057 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f676406bcb0 con 0x7f67681079b0 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.627+0000 7f675effd640 1 -- 172.21.15.5:0/1644033057 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f676403d950 con 0x7f67681079b0 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.628+0000 7f676e99e640 1 -- 172.21.15.5:0/1644033057 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f6754039200 msgr2=0x7f675403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.628+0000 7f676e99e640 1 --2- 172.21.15.5:0/1644033057 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f6754039200 0x7f675403b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f67580098c0 tx=0x7f6758006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.628+0000 7f676e99e640 1 -- 172.21.15.5:0/1644033057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 msgr2=0x7f67681915a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.628+0000 7f676e99e640 1 --2- 172.21.15.5:0/1644033057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 0x7f67681915a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6764009940 tx=0x7f6764030cc0 comp rx=0 tx=0).stop 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.628+0000 7f676e99e640 1 -- 172.21.15.5:0/1644033057 shutdown_connections 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.628+0000 7f676e99e640 1 --2- 172.21.15.5:0/1644033057 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f6754039200 0x7f675403b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.628+0000 7f676e99e640 1 --2- 172.21.15.5:0/1644033057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67681079b0 0x7f67681915a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.628+0000 7f676e99e640 1 -- 172.21.15.5:0/1644033057 >> 172.21.15.5:0/1644033057 conn(0x7f6768101a60 msgr2=0x7f6768102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.629+0000 7f676e99e640 1 -- 172.21.15.5:0/1644033057 shutdown_connections 2024-09-19T05:54:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:33.629+0000 7f676e99e640 1 -- 172.21.15.5:0/1644033057 wait complete. 2024-09-19T05:54:34.620 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:34 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:34.379+0000 7f22bb765040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:54:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:34 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1644033057' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T05:54:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:34 smithi005 ceph-mon[30955]: mgrmap e9: x(active, since 18s) 2024-09-19T05:54:34.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:34 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:34.714+0000 7f22bb765040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:54:34.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:34 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:34.778+0000 7f22bb765040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:54:35.351 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:34 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:34.977+0000 7f22bb765040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:54:35.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:35 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/467631670' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T05:54:36.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:54:36.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T05:54:36.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T05:54:36.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T05:54:36.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T05:54:36.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:54:36.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.142+0000 7f2351c51640 1 Processor -- start 2024-09-19T05:54:36.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.142+0000 7f2351c51640 1 -- start start 2024-09-19T05:54:36.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.142+0000 7f2351c51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 0x7f234c10a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:36.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.142+0000 7f2351c51640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0634b0 con 0x7f234c108040 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.143+0000 7f234b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 0x7f234c10a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.143+0000 7f234b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 0x7f234c10a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49352/0 (socket says 172.21.15.5:49352) 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.143+0000 7f234b7fe640 1 -- 172.21.15.5:0/1599343062 learned_addr learned my addr 172.21.15.5:0/1599343062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.143+0000 7f234b7fe640 1 -- 172.21.15.5:0/1599343062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c10a970 con 0x7f234c108040 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.144+0000 7f234b7fe640 1 --2- 172.21.15.5:0/1599343062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 0x7f234c10a430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f233c009820 tx=0x7f233c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7db2c422d2a86b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.144+0000 7f234a7fc640 1 -- 172.21.15.5:0/1599343062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f233c037070 con 0x7f234c108040 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.144+0000 7f234a7fc640 1 -- 172.21.15.5:0/1599343062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f233c0328f0 con 0x7f234c108040 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.144+0000 7f234a7fc640 1 -- 172.21.15.5:0/1599343062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f233c030740 con 0x7f234c108040 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.145+0000 7f2351c51640 1 -- 172.21.15.5:0/1599343062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 msgr2=0x7f234c10a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.145+0000 7f2351c51640 1 --2- 172.21.15.5:0/1599343062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 0x7f234c10a430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f233c009820 tx=0x7f233c02a270 comp rx=0 tx=0).stop 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.145+0000 7f2351c51640 1 -- 172.21.15.5:0/1599343062 shutdown_connections 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.145+0000 7f2351c51640 1 --2- 172.21.15.5:0/1599343062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 0x7f234c10a430 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.145+0000 7f2351c51640 1 -- 172.21.15.5:0/1599343062 >> 172.21.15.5:0/1599343062 conn(0x7f234c101c00 msgr2=0x7f234c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:36.043 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.145+0000 7f2351c51640 1 -- 172.21.15.5:0/1599343062 shutdown_connections 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.145+0000 7f2351c51640 1 -- 172.21.15.5:0/1599343062 wait complete. 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.146+0000 7f2351c51640 1 Processor -- start 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.146+0000 7f2351c51640 1 -- start start 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.146+0000 7f2351c51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 0x7f234c19b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.146+0000 7f2351c51640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f234c10b990 con 0x7f234c108040 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.146+0000 7f234b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 0x7f234c19b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.146+0000 7f234b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 0x7f234c19b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49358/0 (socket says 172.21.15.5:49358) 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.146+0000 7f234b7fe640 1 -- 172.21.15.5:0/467631670 learned_addr learned my addr 172.21.15.5:0/467631670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.146+0000 7f234b7fe640 1 -- 172.21.15.5:0/467631670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c19bab0 con 0x7f234c108040 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.147+0000 7f234b7fe640 1 --2- 172.21.15.5:0/467631670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 0x7f234c19b570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f233c0097f0 tx=0x7f233c030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.147+0000 7f2348ff9640 1 -- 172.21.15.5:0/467631670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f233c042070 con 0x7f234c108040 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.147+0000 7f2348ff9640 1 -- 172.21.15.5:0/467631670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f233c030d80 con 0x7f234c108040 2024-09-19T05:54:36.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.147+0000 7f2351c51640 1 -- 172.21.15.5:0/467631670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234c19bd40 con 0x7f234c108040 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.147+0000 7f2348ff9640 1 -- 172.21.15.5:0/467631670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f233c037040 con 0x7f234c108040 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.147+0000 7f2351c51640 1 -- 172.21.15.5:0/467631670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234c19ea30 con 0x7f234c108040 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.148+0000 7f2348ff9640 1 -- 172.21.15.5:0/467631670 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f233c045070 con 0x7f234c108040 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.148+0000 7f2348ff9640 1 --2- 172.21.15.5:0/467631670 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2334039200 0x7f233403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.148+0000 7f2348ff9640 1 -- 172.21.15.5:0/467631670 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f233c06d2c0 con 0x7f234c108040 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.148+0000 7f234affd640 1 -- 172.21.15.5:0/467631670 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2334039200 msgr2=0x7f233403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/255461399 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.148+0000 7f234affd640 1 --2- 172.21.15.5:0/467631670 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2334039200 0x7f233403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.148+0000 7f2351c51640 1 -- 172.21.15.5:0/467631670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f234c1084a0 con 0x7f234c108040 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.153+0000 7f2348ff9640 1 -- 172.21.15.5:0/467631670 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f233c04b170 con 0x7f234c108040 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.349+0000 7f234affd640 1 -- 172.21.15.5:0/467631670 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2334039200 msgr2=0x7f233403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/255461399 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.349+0000 7f234affd640 1 --2- 172.21.15.5:0/467631670 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2334039200 0x7f233403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.378+0000 7f2351c51640 1 -- 172.21.15.5:0/467631670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f234c19eef0 con 0x7f234c108040 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.379+0000 7f2348ff9640 1 -- 172.21.15.5:0/467631670 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f233c045760 con 0x7f234c108040 2024-09-19T05:54:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.381+0000 7f2351c51640 1 -- 172.21.15.5:0/467631670 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2334039200 msgr2=0x7f233403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:54:36.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.381+0000 7f2351c51640 1 --2- 172.21.15.5:0/467631670 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2334039200 0x7f233403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:36.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.381+0000 7f2351c51640 1 -- 172.21.15.5:0/467631670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 msgr2=0x7f234c19b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:36.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.381+0000 7f2351c51640 1 --2- 172.21.15.5:0/467631670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 0x7f234c19b570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f233c0097f0 tx=0x7f233c030c70 comp rx=0 tx=0).stop 2024-09-19T05:54:36.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.381+0000 7f2351c51640 1 -- 172.21.15.5:0/467631670 shutdown_connections 2024-09-19T05:54:36.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.381+0000 7f2351c51640 1 --2- 172.21.15.5:0/467631670 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2334039200 0x7f233403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:36.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.381+0000 7f2351c51640 1 --2- 172.21.15.5:0/467631670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234c108040 0x7f234c19b570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:36.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.381+0000 7f2351c51640 1 -- 172.21.15.5:0/467631670 >> 172.21.15.5:0/467631670 conn(0x7f234c101c00 msgr2=0x7f234c102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:36.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.381+0000 7f2351c51640 1 -- 172.21.15.5:0/467631670 shutdown_connections 2024-09-19T05:54:36.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:35.381+0000 7f2351c51640 1 -- 172.21.15.5:0/467631670 wait complete. 2024-09-19T05:54:36.046 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-09-19T05:54:36.046 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 9... 2024-09-19T05:54:36.251 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:35 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:35.913+0000 7f22bb765040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:54:36.251 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:35 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:35.984+0000 7f22bb765040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:54:36.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:36 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:36.742+0000 7f22bb765040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:54:36.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:36 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:36.862+0000 7f22bb765040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:54:37.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:37.011+0000 7f22bb765040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:54:37.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:37.075+0000 7f22bb765040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:54:37.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:37.231+0000 7f22bb765040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:54:37.304 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:37.301+0000 7f22bb765040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:54:37.598 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:37.364+0000 7f22bb765040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:54:37.599 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:37.529+0000 7f22bb765040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:54:37.599 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:37.597+0000 7f22bb765040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:54:37.877 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:37.666+0000 7f22bb765040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:54:37.877 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:37.752+0000 7f22bb765040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:54:37.877 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:37.814+0000 7f22bb765040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:54:37.878 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:37.875+0000 7f22bb765040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:54:38.953 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:38.701+0000 7f22bb765040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:54:38.953 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: Active manager daemon x restarted 2024-09-19T05:54:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: Activating manager daemon x 2024-09-19T05:54:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T05:54:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: mgrmap e10: x(active, starting, since 0.00311069s) 2024-09-19T05:54:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:54:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T05:54:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T05:54:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T05:54:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T05:54:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: Manager daemon x is now available 2024-09-19T05:54:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:54:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:54:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:54:40.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:54:40.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T05:54:40.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T05:54:40.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:54:40.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.416+0000 7f22744f0640 1 Processor -- start 2024-09-19T05:54:40.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.416+0000 7f22744f0640 1 -- start start 2024-09-19T05:54:40.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.416+0000 7f22744f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 0x7f226c1079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:40.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.416+0000 7f22744f0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f226c107fa0 con 0x7f226c1075d0 2024-09-19T05:54:40.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.416+0000 7f227228c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 0x7f226c1079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:40.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.416+0000 7f227228c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 0x7f226c1079d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49370/0 (socket says 172.21.15.5:49370) 2024-09-19T05:54:40.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.416+0000 7f227228c640 1 -- 172.21.15.5:0/243986107 learned_addr learned my addr 172.21.15.5:0/243986107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.417+0000 7f227228c640 1 -- 172.21.15.5:0/243986107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c1087d0 con 0x7f226c1075d0 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.417+0000 7f227228c640 1 --2- 172.21.15.5:0/243986107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 0x7f226c1079d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2268009820 tx=0x7f226802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9004bd530e429243 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.418+0000 7f227128a640 1 -- 172.21.15.5:0/243986107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2268037070 con 0x7f226c1075d0 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.418+0000 7f227128a640 1 -- 172.21.15.5:0/243986107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22680328f0 con 0x7f226c1075d0 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.418+0000 7f227128a640 1 -- 172.21.15.5:0/243986107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2268030740 con 0x7f226c1075d0 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.418+0000 7f22744f0640 1 -- 172.21.15.5:0/243986107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 msgr2=0x7f226c1079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.418+0000 7f22744f0640 1 --2- 172.21.15.5:0/243986107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 0x7f226c1079d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2268009820 tx=0x7f226802a270 comp rx=0 tx=0).stop 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.418+0000 7f22744f0640 1 -- 172.21.15.5:0/243986107 shutdown_connections 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.418+0000 7f22744f0640 1 --2- 172.21.15.5:0/243986107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 0x7f226c1079d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.418+0000 7f22744f0640 1 -- 172.21.15.5:0/243986107 >> 172.21.15.5:0/243986107 conn(0x7f226c102da0 msgr2=0x7f226c1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.418+0000 7f22744f0640 1 -- 172.21.15.5:0/243986107 shutdown_connections 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.418+0000 7f22744f0640 1 -- 172.21.15.5:0/243986107 wait complete. 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.419+0000 7f22744f0640 1 Processor -- start 2024-09-19T05:54:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.419+0000 7f22744f0640 1 -- start start 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.419+0000 7f22744f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 0x7f226c19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.419+0000 7f22744f0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f226c10bb80 con 0x7f226c1075d0 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.419+0000 7f227228c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 0x7f226c19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.419+0000 7f227228c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 0x7f226c19d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49378/0 (socket says 172.21.15.5:49378) 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.419+0000 7f227228c640 1 -- 172.21.15.5:0/3129345215 learned_addr learned my addr 172.21.15.5:0/3129345215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.420+0000 7f227228c640 1 -- 172.21.15.5:0/3129345215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c19dd00 con 0x7f226c1075d0 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.420+0000 7f227228c640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 0x7f226c19d7c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2268009940 tx=0x7f2268030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.420+0000 7f22637fe640 1 -- 172.21.15.5:0/3129345215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2268042070 con 0x7f226c1075d0 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.420+0000 7f22744f0640 1 -- 172.21.15.5:0/3129345215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226c19df90 con 0x7f226c1075d0 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.420+0000 7f22637fe640 1 -- 172.21.15.5:0/3129345215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2268030e30 con 0x7f226c1075d0 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.420+0000 7f22637fe640 1 -- 172.21.15.5:0/3129345215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2268037040 con 0x7f226c1075d0 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.420+0000 7f22744f0640 1 -- 172.21.15.5:0/3129345215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f226c1a0c80 con 0x7f226c1075d0 2024-09-19T05:54:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.421+0000 7f22637fe640 1 -- 172.21.15.5:0/3129345215 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f2268045070 con 0x7f226c1075d0 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.422+0000 7f22637fe640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2258039200 0x7f225803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.422+0000 7f22637fe640 1 -- 172.21.15.5:0/3129345215 --> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f225803bdd0 con 0x7f2258039200 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.422+0000 7f22637fe640 1 -- 172.21.15.5:0/3129345215 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f226806c340 con 0x7f226c1075d0 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.422+0000 7f2271a8b640 1 -- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2258039200 msgr2=0x7f225803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/255461399 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.422+0000 7f2271a8b640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2258039200 0x7f225803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.623+0000 7f2271a8b640 1 -- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2258039200 msgr2=0x7f225803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/255461399 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:36.623+0000 7f2271a8b640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2258039200 0x7f225803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:37.024+0000 7f2271a8b640 1 -- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2258039200 msgr2=0x7f225803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/255461399 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:37.024+0000 7f2271a8b640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2258039200 0x7f225803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:37.825+0000 7f2271a8b640 1 -- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2258039200 msgr2=0x7f225803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/255461399 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:37.825+0000 7f2271a8b640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2258039200 0x7f225803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:38.704+0000 7f22637fe640 1 -- 172.21.15.5:0/3129345215 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f226803d950 con 0x7f226c1075d0 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:38.704+0000 7f22637fe640 1 -- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2258039200 msgr2=0x7f225803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:54:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:38.704+0000 7f22637fe640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2258039200 0x7f225803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.705+0000 7f22637fe640 1 -- 172.21.15.5:0/3129345215 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f226803d430 con 0x7f226c1075d0 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.706+0000 7f22637fe640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f225803ccc0 0x7f225803f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.706+0000 7f22637fe640 1 -- 172.21.15.5:0/3129345215 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f225803bdd0 con 0x7f225803ccc0 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.706+0000 7f2271a8b640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f225803ccc0 0x7f225803f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.707+0000 7f2271a8b640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f225803ccc0 0x7f225803f0b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f225c003980 tx=0x7f225c006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.708+0000 7f22637fe640 1 -- 172.21.15.5:0/3129345215 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f225803bdd0 con 0x7f225803ccc0 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22744f0640 1 -- 172.21.15.5:0/3129345215 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f226c10bd00 con 0x7f225803ccc0 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22637fe640 1 -- 172.21.15.5:0/3129345215 <== mgr.14150 v2:172.21.15.5:6800/709970884 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f226c10bd00 con 0x7f225803ccc0 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22744f0640 1 -- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f225803ccc0 msgr2=0x7f225803f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22744f0640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f225803ccc0 0x7f225803f0b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f225c003980 tx=0x7f225c006e10 comp rx=0 tx=0).stop 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22744f0640 1 -- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 msgr2=0x7f226c19d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22744f0640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 0x7f226c19d7c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2268009940 tx=0x7f2268030cc0 comp rx=0 tx=0).stop 2024-09-19T05:54:40.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22744f0640 1 -- 172.21.15.5:0/3129345215 shutdown_connections 2024-09-19T05:54:40.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22744f0640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f225803ccc0 0x7f225803f0b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:40.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22744f0640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:6800/255461399,v1:172.21.15.5:6801/255461399] conn(0x7f2258039200 0x7f225803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:40.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22744f0640 1 --2- 172.21.15.5:0/3129345215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c1075d0 0x7f226c19d7c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:40.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22744f0640 1 -- 172.21.15.5:0/3129345215 >> 172.21.15.5:0/3129345215 conn(0x7f226c102da0 msgr2=0x7f226c103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:40.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22744f0640 1 -- 172.21.15.5:0/3129345215 shutdown_connections 2024-09-19T05:54:40.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:39.715+0000 7f22744f0640 1 -- 172.21.15.5:0/3129345215 wait complete. 2024-09-19T05:54:40.624 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 9 is available 2024-09-19T05:54:40.624 INFO:teuthology.orchestra.run.smithi005.stdout:Generating a dashboard self-signed certificate... 2024-09-19T05:54:40.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:40 smithi005 ceph-mon[30955]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T05:54:40.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:40 smithi005 ceph-mon[30955]: mgrmap e11: x(active, since 1.00571s) 2024-09-19T05:54:40.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:40 smithi005 ceph-mon[30955]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T05:54:40.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:40 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:41.274 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:41.128+0000 7f2226a39640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:05:54:41] ENGINE Error in HTTPServer.serve 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: Traceback (most recent call last): 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self._connections.run(self.expiration_interval) 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self._run(expiration_interval) 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: s = self.context.wrap_socket( 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: return self.sslsocket_class._create( 2024-09-19T05:54:41.275 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:54:41.276 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self.do_handshake() 2024-09-19T05:54:41.276 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:54:41.276 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self._sslobj.do_handshake() 2024-09-19T05:54:41.276 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:54:41.276 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 05:54:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T05:54:41.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T05:54:41.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.078+0000 7f0e354a3640 1 Processor -- start 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.078+0000 7f0e354a3640 1 -- start start 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.079+0000 7f0e354a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 0x7f0e30106250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.079+0000 7f0e354a3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e30063490 con 0x7f0e30105e50 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.079+0000 7f0e2effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 0x7f0e30106250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.079+0000 7f0e2effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 0x7f0e30106250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49444/0 (socket says 172.21.15.5:49444) 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.079+0000 7f0e2effd640 1 -- 172.21.15.5:0/888968820 learned_addr learned my addr 172.21.15.5:0/888968820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.079+0000 7f0e2effd640 1 -- 172.21.15.5:0/888968820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e30106790 con 0x7f0e30105e50 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.080+0000 7f0e2effd640 1 --2- 172.21.15.5:0/888968820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 0x7f0e30106250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009820 tx=0x7f0e2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cd2673e7fcef25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.080+0000 7f0e2dffb640 1 -- 172.21.15.5:0/888968820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e20037070 con 0x7f0e30105e50 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.080+0000 7f0e2dffb640 1 -- 172.21.15.5:0/888968820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e200328f0 con 0x7f0e30105e50 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.080+0000 7f0e2dffb640 1 -- 172.21.15.5:0/888968820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e20030740 con 0x7f0e30105e50 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.081+0000 7f0e354a3640 1 -- 172.21.15.5:0/888968820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 msgr2=0x7f0e30106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:41.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.081+0000 7f0e354a3640 1 --2- 172.21.15.5:0/888968820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 0x7f0e30106250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009820 tx=0x7f0e2002a270 comp rx=0 tx=0).stop 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.081+0000 7f0e354a3640 1 -- 172.21.15.5:0/888968820 shutdown_connections 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.081+0000 7f0e354a3640 1 --2- 172.21.15.5:0/888968820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 0x7f0e30106250 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.081+0000 7f0e354a3640 1 -- 172.21.15.5:0/888968820 >> 172.21.15.5:0/888968820 conn(0x7f0e30101c00 msgr2=0x7f0e30104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.081+0000 7f0e354a3640 1 -- 172.21.15.5:0/888968820 shutdown_connections 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.081+0000 7f0e354a3640 1 -- 172.21.15.5:0/888968820 wait complete. 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.081+0000 7f0e354a3640 1 Processor -- start 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.082+0000 7f0e354a3640 1 -- start start 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.082+0000 7f0e354a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 0x7f0e3019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.082+0000 7f0e354a3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e30107870 con 0x7f0e30105e50 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.082+0000 7f0e2effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 0x7f0e3019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.082+0000 7f0e2effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 0x7f0e3019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49450/0 (socket says 172.21.15.5:49450) 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.082+0000 7f0e2effd640 1 -- 172.21.15.5:0/3987856950 learned_addr learned my addr 172.21.15.5:0/3987856950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.082+0000 7f0e2effd640 1 -- 172.21.15.5:0/3987856950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3019dc50 con 0x7f0e30105e50 2024-09-19T05:54:41.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.082+0000 7f0e2effd640 1 --2- 172.21.15.5:0/3987856950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 0x7f0e3019d710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e200097f0 tx=0x7f0e20030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.082+0000 7f0e17fff640 1 -- 172.21.15.5:0/3987856950 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e20042070 con 0x7f0e30105e50 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.082+0000 7f0e17fff640 1 -- 172.21.15.5:0/3987856950 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e20030d80 con 0x7f0e30105e50 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.082+0000 7f0e354a3640 1 -- 172.21.15.5:0/3987856950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3019dee0 con 0x7f0e30105e50 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.083+0000 7f0e17fff640 1 -- 172.21.15.5:0/3987856950 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e20037040 con 0x7f0e30105e50 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.083+0000 7f0e354a3640 1 -- 172.21.15.5:0/3987856950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e301a0bd0 con 0x7f0e30105e50 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.083+0000 7f0e17fff640 1 -- 172.21.15.5:0/3987856950 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f0e20045070 con 0x7f0e30105e50 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.084+0000 7f0e354a3640 1 -- 172.21.15.5:0/3987856950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e30106250 con 0x7f0e30105e50 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.084+0000 7f0e17fff640 1 --2- 172.21.15.5:0/3987856950 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0e18039260 0x7f0e1803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.084+0000 7f0e17fff640 1 -- 172.21.15.5:0/3987856950 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0e2006c420 con 0x7f0e30105e50 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.084+0000 7f0e2e7fc640 1 --2- 172.21.15.5:0/3987856950 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0e18039260 0x7f0e1803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.084+0000 7f0e2e7fc640 1 --2- 172.21.15.5:0/3987856950 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0e18039260 0x7f0e1803b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0e240098c0 tx=0x7f0e24006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.089+0000 7f0e17fff640 1 -- 172.21.15.5:0/3987856950 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0e20045360 con 0x7f0e30105e50 2024-09-19T05:54:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.272+0000 7f0e354a3640 1 -- 172.21.15.5:0/3987856950 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0e30074b50 con 0x7f0e18039260 2024-09-19T05:54:41.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.347+0000 7f0e17fff640 1 -- 172.21.15.5:0/3987856950 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0e30074b50 con 0x7f0e18039260 2024-09-19T05:54:41.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.349+0000 7f0e354a3640 1 -- 172.21.15.5:0/3987856950 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0e18039260 msgr2=0x7f0e1803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:41.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.349+0000 7f0e354a3640 1 --2- 172.21.15.5:0/3987856950 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0e18039260 0x7f0e1803b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0e240098c0 tx=0x7f0e24006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:41.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.349+0000 7f0e354a3640 1 -- 172.21.15.5:0/3987856950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 msgr2=0x7f0e3019d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:41.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.349+0000 7f0e354a3640 1 --2- 172.21.15.5:0/3987856950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 0x7f0e3019d710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e200097f0 tx=0x7f0e20030c70 comp rx=0 tx=0).stop 2024-09-19T05:54:41.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.349+0000 7f0e354a3640 1 -- 172.21.15.5:0/3987856950 shutdown_connections 2024-09-19T05:54:41.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.349+0000 7f0e354a3640 1 --2- 172.21.15.5:0/3987856950 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0e18039260 0x7f0e1803b720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:41.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.349+0000 7f0e354a3640 1 --2- 172.21.15.5:0/3987856950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e30105e50 0x7f0e3019d710 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:41.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.349+0000 7f0e354a3640 1 -- 172.21.15.5:0/3987856950 >> 172.21.15.5:0/3987856950 conn(0x7f0e30101c00 msgr2=0x7f0e30191b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:41.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.349+0000 7f0e354a3640 1 -- 172.21.15.5:0/3987856950 shutdown_connections 2024-09-19T05:54:41.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:41.349+0000 7f0e354a3640 1 -- 172.21.15.5:0/3987856950 wait complete. 2024-09-19T05:54:41.869 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial admin user... 2024-09-19T05:54:42.278 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:42 smithi005 ceph-mon[30955]: [19/Sep/2024:05:54:41] ENGINE Bus STARTING 2024-09-19T05:54:42.278 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:42 smithi005 ceph-mon[30955]: mgrmap e12: x(active, since 2s) 2024-09-19T05:54:42.278 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:42 smithi005 ceph-mon[30955]: [19/Sep/2024:05:54:41] ENGINE Serving on https://172.21.15.5:7150 2024-09-19T05:54:42.278 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:42 smithi005 ceph-mon[30955]: [19/Sep/2024:05:54:41] ENGINE Error in HTTPServer.serve 2024-09-19T05:54:42.278 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-19T05:54:42.278 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:54:42.278 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-19T05:54:42.278 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:54:42.278 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-19T05:54:42.278 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:54:42.278 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:42 smithi005 ceph-mon[30955]: [19/Sep/2024:05:54:41] ENGINE Serving on http://172.21.15.5:8765 2024-09-19T05:54:42.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:42 smithi005 ceph-mon[30955]: [19/Sep/2024:05:54:41] ENGINE Bus STARTED 2024-09-19T05:54:42.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:42 smithi005 ceph-mon[30955]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:54:42.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:42.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:43.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:43 smithi005 ceph-mon[30955]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:54:43.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:43.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$qFWRU8mySDj6NMjEYPDwWObSxWDERzwvGvnn/5r1UWV9cBpo2BEsm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726725282, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T05:54:43.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.288+0000 7f48765ac640 1 Processor -- start 2024-09-19T05:54:43.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.288+0000 7f48765ac640 1 -- start start 2024-09-19T05:54:43.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.288+0000 7f48765ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 0x7f4870109c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:43.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.288+0000 7f48765ac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f487010a1f0 con 0x7f4870109820 2024-09-19T05:54:43.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.289+0000 7f486ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 0x7f4870109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:43.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.289+0000 7f486ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 0x7f4870109c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49458/0 (socket says 172.21.15.5:49458) 2024-09-19T05:54:43.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.289+0000 7f486ffff640 1 -- 172.21.15.5:0/628082926 learned_addr learned my addr 172.21.15.5:0/628082926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:43.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.289+0000 7f486ffff640 1 -- 172.21.15.5:0/628082926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487010aa20 con 0x7f4870109820 2024-09-19T05:54:43.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.290+0000 7f486ffff640 1 --2- 172.21.15.5:0/628082926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 0x7f4870109c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4860009820 tx=0x7f486002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d155bc3565be51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:43.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.290+0000 7f486effd640 1 -- 172.21.15.5:0/628082926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4860037070 con 0x7f4870109820 2024-09-19T05:54:43.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.290+0000 7f486effd640 1 -- 172.21.15.5:0/628082926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48600328f0 con 0x7f4870109820 2024-09-19T05:54:43.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.290+0000 7f486effd640 1 -- 172.21.15.5:0/628082926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4860030740 con 0x7f4870109820 2024-09-19T05:54:43.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.291+0000 7f48765ac640 1 -- 172.21.15.5:0/628082926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 msgr2=0x7f4870109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:43.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.291+0000 7f48765ac640 1 --2- 172.21.15.5:0/628082926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 0x7f4870109c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4860009820 tx=0x7f486002a270 comp rx=0 tx=0).stop 2024-09-19T05:54:43.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.291+0000 7f48765ac640 1 -- 172.21.15.5:0/628082926 shutdown_connections 2024-09-19T05:54:43.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.291+0000 7f48765ac640 1 --2- 172.21.15.5:0/628082926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 0x7f4870109c20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.291+0000 7f48765ac640 1 -- 172.21.15.5:0/628082926 >> 172.21.15.5:0/628082926 conn(0x7f48701053a0 msgr2=0x7f4870107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.291+0000 7f48765ac640 1 -- 172.21.15.5:0/628082926 shutdown_connections 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.291+0000 7f48765ac640 1 -- 172.21.15.5:0/628082926 wait complete. 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.291+0000 7f48765ac640 1 Processor -- start 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.292+0000 7f48765ac640 1 -- start start 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.292+0000 7f48765ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 0x7f487019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.292+0000 7f48765ac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f487010ddd0 con 0x7f4870109820 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.292+0000 7f486ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 0x7f487019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.292+0000 7f486ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 0x7f487019d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49470/0 (socket says 172.21.15.5:49470) 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.292+0000 7f486ffff640 1 -- 172.21.15.5:0/3904135317 learned_addr learned my addr 172.21.15.5:0/3904135317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.292+0000 7f486ffff640 1 -- 172.21.15.5:0/3904135317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487019dcb0 con 0x7f4870109820 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.292+0000 7f486ffff640 1 --2- 172.21.15.5:0/3904135317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 0x7f487019d770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4860009940 tx=0x7f4860030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.293+0000 7f486d7fa640 1 -- 172.21.15.5:0/3904135317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4860042070 con 0x7f4870109820 2024-09-19T05:54:43.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.293+0000 7f486d7fa640 1 -- 172.21.15.5:0/3904135317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4860030e30 con 0x7f4870109820 2024-09-19T05:54:43.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.293+0000 7f48765ac640 1 -- 172.21.15.5:0/3904135317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487019df40 con 0x7f4870109820 2024-09-19T05:54:43.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.293+0000 7f48765ac640 1 -- 172.21.15.5:0/3904135317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48701a0c30 con 0x7f4870109820 2024-09-19T05:54:43.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.293+0000 7f486d7fa640 1 -- 172.21.15.5:0/3904135317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4860037040 con 0x7f4870109820 2024-09-19T05:54:43.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.294+0000 7f486d7fa640 1 -- 172.21.15.5:0/3904135317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f4860045070 con 0x7f4870109820 2024-09-19T05:54:43.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.294+0000 7f48765ac640 1 -- 172.21.15.5:0/3904135317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4870109ca0 con 0x7f4870109820 2024-09-19T05:54:43.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.294+0000 7f486d7fa640 1 --2- 172.21.15.5:0/3904135317 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4858039260 0x7f485803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.294+0000 7f486d7fa640 1 -- 172.21.15.5:0/3904135317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f486006c410 con 0x7f4870109820 2024-09-19T05:54:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.294+0000 7f486f7fe640 1 --2- 172.21.15.5:0/3904135317 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4858039260 0x7f485803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.295+0000 7f486f7fe640 1 --2- 172.21.15.5:0/3904135317 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4858039260 0x7f485803b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f48640098c0 tx=0x7f4864006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.300+0000 7f486d7fa640 1 -- 172.21.15.5:0/3904135317 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4860045520 con 0x7f4870109820 2024-09-19T05:54:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.494+0000 7f48765ac640 1 -- 172.21.15.5:0/3904135317 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f48701a0ec0 con 0x7f4858039260 2024-09-19T05:54:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.743+0000 7f486d7fa640 1 -- 172.21.15.5:0/3904135317 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f48701a0ec0 con 0x7f4858039260 2024-09-19T05:54:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.745+0000 7f48765ac640 1 -- 172.21.15.5:0/3904135317 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4858039260 msgr2=0x7f485803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.745+0000 7f48765ac640 1 --2- 172.21.15.5:0/3904135317 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4858039260 0x7f485803b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f48640098c0 tx=0x7f4864006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.745+0000 7f48765ac640 1 -- 172.21.15.5:0/3904135317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 msgr2=0x7f487019d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.745+0000 7f48765ac640 1 --2- 172.21.15.5:0/3904135317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 0x7f487019d770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4860009940 tx=0x7f4860030cc0 comp rx=0 tx=0).stop 2024-09-19T05:54:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.745+0000 7f48765ac640 1 -- 172.21.15.5:0/3904135317 shutdown_connections 2024-09-19T05:54:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.745+0000 7f48765ac640 1 --2- 172.21.15.5:0/3904135317 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4858039260 0x7f485803b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:43.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.745+0000 7f48765ac640 1 --2- 172.21.15.5:0/3904135317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4870109820 0x7f487019d770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:43.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.745+0000 7f48765ac640 1 -- 172.21.15.5:0/3904135317 >> 172.21.15.5:0/3904135317 conn(0x7f48701053a0 msgr2=0x7f4870106ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:43.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.745+0000 7f48765ac640 1 -- 172.21.15.5:0/3904135317 shutdown_connections 2024-09-19T05:54:43.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:42.745+0000 7f48765ac640 1 -- 172.21.15.5:0/3904135317 wait complete. 2024-09-19T05:54:43.505 INFO:teuthology.orchestra.run.smithi005.stdout:Fetching dashboard port number... 2024-09-19T05:54:44.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:44 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2594581550' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T05:54:45.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T05:54:45.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.360+0000 7fcc16433640 1 Processor -- start 2024-09-19T05:54:45.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.360+0000 7fcc16433640 1 -- start start 2024-09-19T05:54:45.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.361+0000 7fcc16433640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 0x7fcc1010a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:45.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.361+0000 7fcc16433640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc10063510 con 0x7fcc10107fe0 2024-09-19T05:54:45.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.361+0000 7fcc0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 0x7fcc1010a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:45.092 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.361+0000 7fcc0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 0x7fcc1010a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49476/0 (socket says 172.21.15.5:49476) 2024-09-19T05:54:45.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.361+0000 7fcc0ffff640 1 -- 172.21.15.5:0/158744615 learned_addr learned my addr 172.21.15.5:0/158744615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:45.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.361+0000 7fcc0ffff640 1 -- 172.21.15.5:0/158744615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1010a910 con 0x7fcc10107fe0 2024-09-19T05:54:45.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.362+0000 7fcc0ffff640 1 --2- 172.21.15.5:0/158744615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 0x7fcc1010a3d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009820 tx=0x7fcc0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb68d30ff05193c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:45.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.362+0000 7fcc0effd640 1 -- 172.21.15.5:0/158744615 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc00037070 con 0x7fcc10107fe0 2024-09-19T05:54:45.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.362+0000 7fcc0effd640 1 -- 172.21.15.5:0/158744615 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc000328f0 con 0x7fcc10107fe0 2024-09-19T05:54:45.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.362+0000 7fcc0effd640 1 -- 172.21.15.5:0/158744615 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc00030740 con 0x7fcc10107fe0 2024-09-19T05:54:45.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.362+0000 7fcc16433640 1 -- 172.21.15.5:0/158744615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 msgr2=0x7fcc1010a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:45.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.362+0000 7fcc16433640 1 --2- 172.21.15.5:0/158744615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 0x7fcc1010a3d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009820 tx=0x7fcc0002a270 comp rx=0 tx=0).stop 2024-09-19T05:54:45.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.363+0000 7fcc16433640 1 -- 172.21.15.5:0/158744615 shutdown_connections 2024-09-19T05:54:45.093 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.363+0000 7fcc16433640 1 --2- 172.21.15.5:0/158744615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 0x7fcc1010a3d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:45.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.363+0000 7fcc16433640 1 -- 172.21.15.5:0/158744615 >> 172.21.15.5:0/158744615 conn(0x7fcc10101c00 msgr2=0x7fcc10104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:45.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.363+0000 7fcc16433640 1 -- 172.21.15.5:0/158744615 shutdown_connections 2024-09-19T05:54:45.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.363+0000 7fcc16433640 1 -- 172.21.15.5:0/158744615 wait complete. 2024-09-19T05:54:45.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.363+0000 7fcc16433640 1 Processor -- start 2024-09-19T05:54:45.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.363+0000 7fcc16433640 1 -- start start 2024-09-19T05:54:45.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.364+0000 7fcc16433640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 0x7fcc1019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:45.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.364+0000 7fcc16433640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1010b990 con 0x7fcc10107fe0 2024-09-19T05:54:45.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.364+0000 7fcc0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 0x7fcc1019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:45.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.364+0000 7fcc0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 0x7fcc1019d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49490/0 (socket says 172.21.15.5:49490) 2024-09-19T05:54:45.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.364+0000 7fcc0ffff640 1 -- 172.21.15.5:0/2594581550 learned_addr learned my addr 172.21.15.5:0/2594581550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.364+0000 7fcc0ffff640 1 -- 172.21.15.5:0/2594581550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1019dc70 con 0x7fcc10107fe0 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.364+0000 7fcc0ffff640 1 --2- 172.21.15.5:0/2594581550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 0x7fcc1019d730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcc000097f0 tx=0x7fcc00030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.364+0000 7fcc0d7fa640 1 -- 172.21.15.5:0/2594581550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc00042070 con 0x7fcc10107fe0 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.364+0000 7fcc0d7fa640 1 -- 172.21.15.5:0/2594581550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc00030d80 con 0x7fcc10107fe0 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.364+0000 7fcc16433640 1 -- 172.21.15.5:0/2594581550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1019df00 con 0x7fcc10107fe0 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.364+0000 7fcc0d7fa640 1 -- 172.21.15.5:0/2594581550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc00037040 con 0x7fcc10107fe0 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.364+0000 7fcc16433640 1 -- 172.21.15.5:0/2594581550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc101a0bf0 con 0x7fcc10107fe0 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.365+0000 7fcc0d7fa640 1 -- 172.21.15.5:0/2594581550 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fcc00045070 con 0x7fcc10107fe0 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.365+0000 7fcc0d7fa640 1 --2- 172.21.15.5:0/2594581550 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcbf8039200 0x7fcbf803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.365+0000 7fcc0d7fa640 1 -- 172.21.15.5:0/2594581550 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcc0006c480 con 0x7fcc10107fe0 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.366+0000 7fcc0f7fe640 1 --2- 172.21.15.5:0/2594581550 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcbf8039200 0x7fcbf803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.366+0000 7fcc16433640 1 -- 172.21.15.5:0/2594581550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc10108440 con 0x7fcc10107fe0 2024-09-19T05:54:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.366+0000 7fcc0f7fe640 1 --2- 172.21.15.5:0/2594581550 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcbf8039200 0x7fcbf803b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc040098c0 tx=0x7fcc04006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:45.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.371+0000 7fcc0d7fa640 1 -- 172.21.15.5:0/2594581550 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcc0004b170 con 0x7fcc10107fe0 2024-09-19T05:54:45.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.546+0000 7fcc16433640 1 -- 172.21.15.5:0/2594581550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fcc1019e3e0 con 0x7fcc10107fe0 2024-09-19T05:54:45.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.547+0000 7fcc0d7fa640 1 -- 172.21.15.5:0/2594581550 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fcc1019e3e0 con 0x7fcc10107fe0 2024-09-19T05:54:45.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.549+0000 7fcc16433640 1 -- 172.21.15.5:0/2594581550 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcbf8039200 msgr2=0x7fcbf803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:45.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.549+0000 7fcc16433640 1 --2- 172.21.15.5:0/2594581550 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcbf8039200 0x7fcbf803b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc040098c0 tx=0x7fcc04006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:45.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.549+0000 7fcc16433640 1 -- 172.21.15.5:0/2594581550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 msgr2=0x7fcc1019d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:45.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.549+0000 7fcc16433640 1 --2- 172.21.15.5:0/2594581550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 0x7fcc1019d730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcc000097f0 tx=0x7fcc00030c70 comp rx=0 tx=0).stop 2024-09-19T05:54:45.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.549+0000 7fcc16433640 1 -- 172.21.15.5:0/2594581550 shutdown_connections 2024-09-19T05:54:45.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.549+0000 7fcc16433640 1 --2- 172.21.15.5:0/2594581550 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcbf8039200 0x7fcbf803b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:45.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.549+0000 7fcc16433640 1 --2- 172.21.15.5:0/2594581550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc10107fe0 0x7fcc1019d730 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:45.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.549+0000 7fcc16433640 1 -- 172.21.15.5:0/2594581550 >> 172.21.15.5:0/2594581550 conn(0x7fcc10101c00 msgr2=0x7fcc10102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:45.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.549+0000 7fcc16433640 1 -- 172.21.15.5:0/2594581550 shutdown_connections 2024-09-19T05:54:45.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:44.549+0000 7fcc16433640 1 -- 172.21.15.5:0/2594581550 wait complete. 2024-09-19T05:54:45.100 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T05:54:45.100 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-19T05:54:45.108 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T05:54:45.108 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-19T05:54:45.108 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-19T05:54:45.108 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T05:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph Dashboard is now available at: 2024-09-19T05:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: URL: https://smithi005.front.sepia.ceph.com:8443/ 2024-09-19T05:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: User: admin 2024-09-19T05:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: Password: 4uaq4hig5e 2024-09-19T05:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout:Saving cluster configuration to /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config directory 2024-09-19T05:54:46.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.537+0000 7f3f0bcc8640 1 Processor -- start 2024-09-19T05:54:46.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.537+0000 7f3f0bcc8640 1 -- start start 2024-09-19T05:54:46.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.537+0000 7f3f0bcc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 0x7f3f0407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:46.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.537+0000 7f3f0bcc8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0407c8a0 con 0x7f3f0407de70 2024-09-19T05:54:46.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.537+0000 7f3f09a64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 0x7f3f0407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:46.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.537+0000 7f3f09a64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 0x7f3f0407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56778/0 (socket says 172.21.15.5:56778) 2024-09-19T05:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.537+0000 7f3f09a64640 1 -- 172.21.15.5:0/651760504 learned_addr learned my addr 172.21.15.5:0/651760504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.538+0000 7f3f09a64640 1 -- 172.21.15.5:0/651760504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0407ca20 con 0x7f3f0407de70 2024-09-19T05:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.538+0000 7f3f09a64640 1 --2- 172.21.15.5:0/651760504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 0x7f3f0407c2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009820 tx=0x7f3f0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5112b95c0446090 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.539+0000 7f3f08a62640 1 -- 172.21.15.5:0/651760504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f00037070 con 0x7f3f0407de70 2024-09-19T05:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.539+0000 7f3f08a62640 1 -- 172.21.15.5:0/651760504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f000328f0 con 0x7f3f0407de70 2024-09-19T05:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.539+0000 7f3f08a62640 1 -- 172.21.15.5:0/651760504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f00030740 con 0x7f3f0407de70 2024-09-19T05:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.539+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/651760504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 msgr2=0x7f3f0407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.539+0000 7f3f0bcc8640 1 --2- 172.21.15.5:0/651760504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 0x7f3f0407c2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009820 tx=0x7f3f0002a270 comp rx=0 tx=0).stop 2024-09-19T05:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.539+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/651760504 shutdown_connections 2024-09-19T05:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.539+0000 7f3f0bcc8640 1 --2- 172.21.15.5:0/651760504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 0x7f3f0407c2d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.539+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/651760504 >> 172.21.15.5:0/651760504 conn(0x7f3f04102d50 msgr2=0x7f3f04105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.539+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/651760504 shutdown_connections 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.539+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/651760504 wait complete. 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.540+0000 7f3f0bcc8640 1 Processor -- start 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.540+0000 7f3f0bcc8640 1 -- start start 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.540+0000 7f3f0bcc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 0x7f3f0419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.540+0000 7f3f0bcc8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0410dbf0 con 0x7f3f0407de70 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.540+0000 7f3f09a64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 0x7f3f0419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.540+0000 7f3f09a64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 0x7f3f0419d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56792/0 (socket says 172.21.15.5:56792) 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.540+0000 7f3f09a64640 1 -- 172.21.15.5:0/1709500193 learned_addr learned my addr 172.21.15.5:0/1709500193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.541+0000 7f3f09a64640 1 -- 172.21.15.5:0/1709500193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0419dd30 con 0x7f3f0407de70 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.541+0000 7f3f09a64640 1 --2- 172.21.15.5:0/1709500193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 0x7f3f0419d7f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3f000097f0 tx=0x7f3f00030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.541+0000 7f3efaffd640 1 -- 172.21.15.5:0/1709500193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f00042070 con 0x7f3f0407de70 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.541+0000 7f3efaffd640 1 -- 172.21.15.5:0/1709500193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f00030e80 con 0x7f3f0407de70 2024-09-19T05:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.541+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/1709500193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0419dfc0 con 0x7f3f0407de70 2024-09-19T05:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.541+0000 7f3efaffd640 1 -- 172.21.15.5:0/1709500193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f00037040 con 0x7f3f0407de70 2024-09-19T05:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.541+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/1709500193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f041a0cb0 con 0x7f3f0407de70 2024-09-19T05:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.542+0000 7f3efaffd640 1 -- 172.21.15.5:0/1709500193 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f3f00045070 con 0x7f3f0407de70 2024-09-19T05:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.542+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/1709500193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f0407c350 con 0x7f3f0407de70 2024-09-19T05:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.542+0000 7f3efaffd640 1 --2- 172.21.15.5:0/1709500193 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3ef0039260 0x7f3ef003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.542+0000 7f3efaffd640 1 -- 172.21.15.5:0/1709500193 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f0006c440 con 0x7f3f0407de70 2024-09-19T05:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.542+0000 7f3f09263640 1 --2- 172.21.15.5:0/1709500193 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3ef0039260 0x7f3ef003b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.543+0000 7f3f09263640 1 --2- 172.21.15.5:0/1709500193 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3ef0039260 0x7f3ef003b720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40098c0 tx=0x7f3ef4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.547+0000 7f3efaffd640 1 -- 172.21.15.5:0/1709500193 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f0003db70 con 0x7f3f0407de70 2024-09-19T05:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.600+0000 7f3efaffd640 1 -- 172.21.15.5:0/1709500193 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f3f0002fd40 con 0x7f3f0407de70 2024-09-19T05:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.813+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/1709500193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f3f041a12a0 con 0x7f3f0407de70 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.815+0000 7f3efaffd640 1 -- 172.21.15.5:0/1709500193 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7f3f00062710 con 0x7f3f0407de70 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.817+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/1709500193 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3ef0039260 msgr2=0x7f3ef003b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.817+0000 7f3f0bcc8640 1 --2- 172.21.15.5:0/1709500193 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3ef0039260 0x7f3ef003b720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40098c0 tx=0x7f3ef4006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.817+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/1709500193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 msgr2=0x7f3f0419d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.817+0000 7f3f0bcc8640 1 --2- 172.21.15.5:0/1709500193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 0x7f3f0419d7f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3f000097f0 tx=0x7f3f00030c70 comp rx=0 tx=0).stop 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.817+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/1709500193 shutdown_connections 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.817+0000 7f3f0bcc8640 1 --2- 172.21.15.5:0/1709500193 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3ef0039260 0x7f3ef003b720 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.817+0000 7f3f0bcc8640 1 --2- 172.21.15.5:0/1709500193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f0407de70 0x7f3f0419d7f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.817+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/1709500193 >> 172.21.15.5:0/1709500193 conn(0x7f3f04102d50 msgr2=0x7f3f04103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.818+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/1709500193 shutdown_connections 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-19T05:54:45.818+0000 7f3f0bcc8640 1 -- 172.21.15.5:0/1709500193 wait complete. 2024-09-19T05:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: ceph telemetry on 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout:For more information see: 2024-09-19T05:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:54:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T05:54:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:54:46.355 INFO:teuthology.orchestra.run.smithi005.stdout:Bootstrap complete. 2024-09-19T05:54:46.390 INFO:tasks.cephadm:Fetching config... 2024-09-19T05:54:46.390 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:54:46.390 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T05:54:46.408 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T05:54:46.408 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:54:46.408 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T05:54:46.463 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T05:54:46.464 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:54:46.464 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T05:54:46.529 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T05:54:46.529 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:54:46.530 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T05:54:46.586 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T05:54:46.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtqLgIeURvpD/Isv7Ln3I0mfg/xdl2R9z1grX5vfkoUMyw28IaOwPmpHnNRg24I0PvhvJkWh5qXy7KzYedgb4GhztYfVY++4aUZl8maj6uhU5POJohCG9DyxQ+w70S7+Cv3voRA/ZNG5+Klymxk/eN95uLGk8id/W3osznz0IsfvRVO4GMHoE+Ldy1WUQUEjOnujXhs4mkriBRhltYUjlIQn2SuXlGRVQzsDI8U2zV4vfvDKx/2E9qiee83qhn6x8zzkBJ6u/PW1XvghLnqosVlSIM/LWSkPb2TC31Immn/nr34l6bxzxRYAhZAx9qQGYcH+PzZa5Q9+FRBgBPR5IDDkLCatreX26Es41Ru+D1vCF8tJBBtQSby7qceeI2NIphPzvA96S9C3WIBOWifPVQ4WZGtySFcMc3xfx9nDWKZ3oJX2qh0Ixdozmmbe1Ih0hSQtHsz8QGWaOSYm5QOSSSQlX2A7IObzuZLrZjh3uLkqCkng1b5PsxyXYQJwynBR0= ceph-6b5d55e8-764b-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T05:54:46.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:46 smithi005 ceph-mon[30955]: mgrmap e13: x(active, since 6s) 2024-09-19T05:54:46.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:46 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1709500193' entity='client.admin' 2024-09-19T05:54:46.661 INFO:teuthology.orchestra.run.smithi005.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtqLgIeURvpD/Isv7Ln3I0mfg/xdl2R9z1grX5vfkoUMyw28IaOwPmpHnNRg24I0PvhvJkWh5qXy7KzYedgb4GhztYfVY++4aUZl8maj6uhU5POJohCG9DyxQ+w70S7+Cv3voRA/ZNG5+Klymxk/eN95uLGk8id/W3osznz0IsfvRVO4GMHoE+Ldy1WUQUEjOnujXhs4mkriBRhltYUjlIQn2SuXlGRVQzsDI8U2zV4vfvDKx/2E9qiee83qhn6x8zzkBJ6u/PW1XvghLnqosVlSIM/LWSkPb2TC31Immn/nr34l6bxzxRYAhZAx9qQGYcH+PzZa5Q9+FRBgBPR5IDDkLCatreX26Es41Ru+D1vCF8tJBBtQSby7qceeI2NIphPzvA96S9C3WIBOWifPVQ4WZGtySFcMc3xfx9nDWKZ3oJX2qh0Ixdozmmbe1Ih0hSQtHsz8QGWaOSYm5QOSSSQlX2A7IObzuZLrZjh3uLkqCkng1b5PsxyXYQJwynBR0= ceph-6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:54:46.673 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtqLgIeURvpD/Isv7Ln3I0mfg/xdl2R9z1grX5vfkoUMyw28IaOwPmpHnNRg24I0PvhvJkWh5qXy7KzYedgb4GhztYfVY++4aUZl8maj6uhU5POJohCG9DyxQ+w70S7+Cv3voRA/ZNG5+Klymxk/eN95uLGk8id/W3osznz0IsfvRVO4GMHoE+Ldy1WUQUEjOnujXhs4mkriBRhltYUjlIQn2SuXlGRVQzsDI8U2zV4vfvDKx/2E9qiee83qhn6x8zzkBJ6u/PW1XvghLnqosVlSIM/LWSkPb2TC31Immn/nr34l6bxzxRYAhZAx9qQGYcH+PzZa5Q9+FRBgBPR5IDDkLCatreX26Es41Ru+D1vCF8tJBBtQSby7qceeI2NIphPzvA96S9C3WIBOWifPVQ4WZGtySFcMc3xfx9nDWKZ3oJX2qh0Ixdozmmbe1Ih0hSQtHsz8QGWaOSYm5QOSSSQlX2A7IObzuZLrZjh3uLkqCkng1b5PsxyXYQJwynBR0= ceph-6b5d55e8-764b-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T05:54:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtqLgIeURvpD/Isv7Ln3I0mfg/xdl2R9z1grX5vfkoUMyw28IaOwPmpHnNRg24I0PvhvJkWh5qXy7KzYedgb4GhztYfVY++4aUZl8maj6uhU5POJohCG9DyxQ+w70S7+Cv3voRA/ZNG5+Klymxk/eN95uLGk8id/W3osznz0IsfvRVO4GMHoE+Ldy1WUQUEjOnujXhs4mkriBRhltYUjlIQn2SuXlGRVQzsDI8U2zV4vfvDKx/2E9qiee83qhn6x8zzkBJ6u/PW1XvghLnqosVlSIM/LWSkPb2TC31Immn/nr34l6bxzxRYAhZAx9qQGYcH+PzZa5Q9+FRBgBPR5IDDkLCatreX26Es41Ru+D1vCF8tJBBtQSby7qceeI2NIphPzvA96S9C3WIBOWifPVQ4WZGtySFcMc3xfx9nDWKZ3oJX2qh0Ixdozmmbe1Ih0hSQtHsz8QGWaOSYm5QOSSSQlX2A7IObzuZLrZjh3uLkqCkng1b5PsxyXYQJwynBR0= ceph-6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:54:46.742 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtqLgIeURvpD/Isv7Ln3I0mfg/xdl2R9z1grX5vfkoUMyw28IaOwPmpHnNRg24I0PvhvJkWh5qXy7KzYedgb4GhztYfVY++4aUZl8maj6uhU5POJohCG9DyxQ+w70S7+Cv3voRA/ZNG5+Klymxk/eN95uLGk8id/W3osznz0IsfvRVO4GMHoE+Ldy1WUQUEjOnujXhs4mkriBRhltYUjlIQn2SuXlGRVQzsDI8U2zV4vfvDKx/2E9qiee83qhn6x8zzkBJ6u/PW1XvghLnqosVlSIM/LWSkPb2TC31Immn/nr34l6bxzxRYAhZAx9qQGYcH+PzZa5Q9+FRBgBPR5IDDkLCatreX26Es41Ru+D1vCF8tJBBtQSby7qceeI2NIphPzvA96S9C3WIBOWifPVQ4WZGtySFcMc3xfx9nDWKZ3oJX2qh0Ixdozmmbe1Ih0hSQtHsz8QGWaOSYm5QOSSSQlX2A7IObzuZLrZjh3uLkqCkng1b5PsxyXYQJwynBR0= ceph-6b5d55e8-764b-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T05:54:46.795 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtqLgIeURvpD/Isv7Ln3I0mfg/xdl2R9z1grX5vfkoUMyw28IaOwPmpHnNRg24I0PvhvJkWh5qXy7KzYedgb4GhztYfVY++4aUZl8maj6uhU5POJohCG9DyxQ+w70S7+Cv3voRA/ZNG5+Klymxk/eN95uLGk8id/W3osznz0IsfvRVO4GMHoE+Ldy1WUQUEjOnujXhs4mkriBRhltYUjlIQn2SuXlGRVQzsDI8U2zV4vfvDKx/2E9qiee83qhn6x8zzkBJ6u/PW1XvghLnqosVlSIM/LWSkPb2TC31Immn/nr34l6bxzxRYAhZAx9qQGYcH+PzZa5Q9+FRBgBPR5IDDkLCatreX26Es41Ru+D1vCF8tJBBtQSby7qceeI2NIphPzvA96S9C3WIBOWifPVQ4WZGtySFcMc3xfx9nDWKZ3oJX2qh0Ixdozmmbe1Ih0hSQtHsz8QGWaOSYm5QOSSSQlX2A7IObzuZLrZjh3uLkqCkng1b5PsxyXYQJwynBR0= ceph-6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:54:46.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T05:54:47.061 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:47.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.592+0000 7f5f8de5e640 1 -- 172.21.15.5:0/4094418865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88101620 msgr2=0x7f5f88101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:47.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.592+0000 7f5f8de5e640 1 --2- 172.21.15.5:0/4094418865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88101620 0x7f5f88101a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009880 tx=0x7f5f7802a5d0 comp rx=0 tx=0).stop 2024-09-19T05:54:47.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.592+0000 7f5f8de5e640 1 -- 172.21.15.5:0/4094418865 shutdown_connections 2024-09-19T05:54:47.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.592+0000 7f5f8de5e640 1 --2- 172.21.15.5:0/4094418865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88101620 0x7f5f88101a20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:47.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.592+0000 7f5f8de5e640 1 -- 172.21.15.5:0/4094418865 >> 172.21.15.5:0/4094418865 conn(0x7f5f880fcdf0 msgr2=0x7f5f880ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:47.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.592+0000 7f5f8de5e640 1 -- 172.21.15.5:0/4094418865 shutdown_connections 2024-09-19T05:54:47.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.592+0000 7f5f8de5e640 1 -- 172.21.15.5:0/4094418865 wait complete. 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.592+0000 7f5f8de5e640 1 Processor -- start 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.593+0000 7f5f8de5e640 1 -- start start 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.593+0000 7f5f8de5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88101620 0x7f5f88075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.593+0000 7f5f8de5e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f881075e0 con 0x7f5f88101620 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.593+0000 7f5f877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88101620 0x7f5f88075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.593+0000 7f5f877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88101620 0x7f5f88075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56808/0 (socket says 172.21.15.5:56808) 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.593+0000 7f5f877fe640 1 -- 172.21.15.5:0/1336772908 learned_addr learned my addr 172.21.15.5:0/1336772908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.593+0000 7f5f877fe640 1 -- 172.21.15.5:0/1336772908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f880763f0 con 0x7f5f88101620 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.593+0000 7f5f877fe640 1 --2- 172.21.15.5:0/1336772908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88101620 0x7f5f88075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009230 tx=0x7f5f780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.594+0000 7f5f84ff9640 1 -- 172.21.15.5:0/1336772908 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f7803f070 con 0x7f5f88101620 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.594+0000 7f5f84ff9640 1 -- 172.21.15.5:0/1336772908 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f78003aa0 con 0x7f5f88101620 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.594+0000 7f5f8de5e640 1 -- 172.21.15.5:0/1336772908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f880745e0 con 0x7f5f88101620 2024-09-19T05:54:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.594+0000 7f5f84ff9640 1 -- 172.21.15.5:0/1336772908 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f78035030 con 0x7f5f88101620 2024-09-19T05:54:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.594+0000 7f5f8de5e640 1 -- 172.21.15.5:0/1336772908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f88074b20 con 0x7f5f88101620 2024-09-19T05:54:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.595+0000 7f5f84ff9640 1 -- 172.21.15.5:0/1336772908 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f5f78042070 con 0x7f5f88101620 2024-09-19T05:54:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.595+0000 7f5f8de5e640 1 -- 172.21.15.5:0/1336772908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f88107d60 con 0x7f5f88101620 2024-09-19T05:54:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.595+0000 7f5f84ff9640 1 --2- 172.21.15.5:0/1336772908 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5f700392a0 0x7f5f7003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.595+0000 7f5f84ff9640 1 -- 172.21.15.5:0/1336772908 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5f7806d150 con 0x7f5f88101620 2024-09-19T05:54:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.595+0000 7f5f86ffd640 1 --2- 172.21.15.5:0/1336772908 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5f700392a0 0x7f5f7003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.595+0000 7f5f86ffd640 1 --2- 172.21.15.5:0/1336772908 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5f700392a0 0x7f5f7003b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0098c0 tx=0x7f5f7c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.600+0000 7f5f84ff9640 1 -- 172.21.15.5:0/1336772908 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5f7802f300 con 0x7f5f88101620 2024-09-19T05:54:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.779+0000 7f5f8de5e640 1 -- 172.21.15.5:0/1336772908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f5f88190080 con 0x7f5f88101620 2024-09-19T05:54:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.783+0000 7f5f84ff9640 1 -- 172.21.15.5:0/1336772908 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f5f78042c60 con 0x7f5f88101620 2024-09-19T05:54:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.785+0000 7f5f8de5e640 1 -- 172.21.15.5:0/1336772908 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5f700392a0 msgr2=0x7f5f7003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.785+0000 7f5f8de5e640 1 --2- 172.21.15.5:0/1336772908 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5f700392a0 0x7f5f7003b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0098c0 tx=0x7f5f7c006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.785+0000 7f5f8de5e640 1 -- 172.21.15.5:0/1336772908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88101620 msgr2=0x7f5f88075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.785+0000 7f5f8de5e640 1 --2- 172.21.15.5:0/1336772908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88101620 0x7f5f88075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009230 tx=0x7f5f780048f0 comp rx=0 tx=0).stop 2024-09-19T05:54:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.785+0000 7f5f8de5e640 1 -- 172.21.15.5:0/1336772908 shutdown_connections 2024-09-19T05:54:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.785+0000 7f5f8de5e640 1 --2- 172.21.15.5:0/1336772908 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5f700392a0 0x7f5f7003b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.785+0000 7f5f8de5e640 1 --2- 172.21.15.5:0/1336772908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88101620 0x7f5f88075eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.785+0000 7f5f8de5e640 1 -- 172.21.15.5:0/1336772908 >> 172.21.15.5:0/1336772908 conn(0x7f5f880fcdf0 msgr2=0x7f5f880fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.785+0000 7f5f8de5e640 1 -- 172.21.15.5:0/1336772908 shutdown_connections 2024-09-19T05:54:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:47.785+0000 7f5f8de5e640 1 -- 172.21.15.5:0/1336772908 wait complete. 2024-09-19T05:54:48.431 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T05:54:48.432 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T05:54:49.019 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:49.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:48 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1336772908' entity='client.admin' 2024-09-19T05:54:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.444+0000 7f6c36771640 1 -- 172.21.15.5:0/2829574349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c30100510 msgr2=0x7f6c30100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.444+0000 7f6c36771640 1 --2- 172.21.15.5:0/2829574349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c30100510 0x7f6c30100910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009880 tx=0x7f6c2002a5d0 comp rx=0 tx=0).stop 2024-09-19T05:54:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.444+0000 7f6c36771640 1 -- 172.21.15.5:0/2829574349 shutdown_connections 2024-09-19T05:54:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.444+0000 7f6c36771640 1 --2- 172.21.15.5:0/2829574349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c30100510 0x7f6c30100910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.444+0000 7f6c36771640 1 -- 172.21.15.5:0/2829574349 >> 172.21.15.5:0/2829574349 conn(0x7f6c300fbc80 msgr2=0x7f6c300fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.444+0000 7f6c36771640 1 -- 172.21.15.5:0/2829574349 shutdown_connections 2024-09-19T05:54:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.444+0000 7f6c36771640 1 -- 172.21.15.5:0/2829574349 wait complete. 2024-09-19T05:54:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.444+0000 7f6c36771640 1 Processor -- start 2024-09-19T05:54:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.445+0000 7f6c36771640 1 -- start start 2024-09-19T05:54:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.445+0000 7f6c36771640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c30100510 0x7f6c30196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.445+0000 7f6c36771640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c301064d0 con 0x7f6c30100510 2024-09-19T05:54:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.445+0000 7f6c2ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c30100510 0x7f6c30196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.445+0000 7f6c2ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c30100510 0x7f6c30196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56826/0 (socket says 172.21.15.5:56826) 2024-09-19T05:54:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.445+0000 7f6c2ffff640 1 -- 172.21.15.5:0/2876098764 learned_addr learned my addr 172.21.15.5:0/2876098764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.445+0000 7f6c2ffff640 1 -- 172.21.15.5:0/2876098764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c301971f0 con 0x7f6c30100510 2024-09-19T05:54:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.445+0000 7f6c2ffff640 1 --2- 172.21.15.5:0/2876098764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c30100510 0x7f6c30196cb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009230 tx=0x7f6c200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.446+0000 7f6c2d7fa640 1 -- 172.21.15.5:0/2876098764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c2003f070 con 0x7f6c30100510 2024-09-19T05:54:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.446+0000 7f6c36771640 1 -- 172.21.15.5:0/2876098764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c30197480 con 0x7f6c30100510 2024-09-19T05:54:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.446+0000 7f6c2d7fa640 1 -- 172.21.15.5:0/2876098764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c20003aa0 con 0x7f6c30100510 2024-09-19T05:54:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.446+0000 7f6c2d7fa640 1 -- 172.21.15.5:0/2876098764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c20035030 con 0x7f6c30100510 2024-09-19T05:54:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.446+0000 7f6c36771640 1 -- 172.21.15.5:0/2876098764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3019a170 con 0x7f6c30100510 2024-09-19T05:54:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.447+0000 7f6c2d7fa640 1 -- 172.21.15.5:0/2876098764 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f6c20042070 con 0x7f6c30100510 2024-09-19T05:54:49.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.447+0000 7f6c2d7fa640 1 --2- 172.21.15.5:0/2876098764 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6c1c0392a0 0x7f6c1c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:49.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.447+0000 7f6c2d7fa640 1 -- 172.21.15.5:0/2876098764 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6c2006d160 con 0x7f6c30100510 2024-09-19T05:54:49.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.447+0000 7f6c2f7fe640 1 --2- 172.21.15.5:0/2876098764 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6c1c0392a0 0x7f6c1c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:49.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.447+0000 7f6c36771640 1 -- 172.21.15.5:0/2876098764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c30106c50 con 0x7f6c30100510 2024-09-19T05:54:49.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.447+0000 7f6c2f7fe640 1 --2- 172.21.15.5:0/2876098764 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6c1c0392a0 0x7f6c1c03b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c180098c0 tx=0x7f6c18006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:49.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.452+0000 7f6c2d7fa640 1 -- 172.21.15.5:0/2876098764 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6c2002f300 con 0x7f6c30100510 2024-09-19T05:54:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.630+0000 7f6c36771640 1 -- 172.21.15.5:0/2876098764 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f6c30062630 con 0x7f6c1c0392a0 2024-09-19T05:54:49.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.633+0000 7f6c2d7fa640 1 -- 172.21.15.5:0/2876098764 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6c30062630 con 0x7f6c1c0392a0 2024-09-19T05:54:49.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.635+0000 7f6c36771640 1 -- 172.21.15.5:0/2876098764 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6c1c0392a0 msgr2=0x7f6c1c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:49.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.635+0000 7f6c36771640 1 --2- 172.21.15.5:0/2876098764 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6c1c0392a0 0x7f6c1c03b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c180098c0 tx=0x7f6c18006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:49.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.636+0000 7f6c36771640 1 -- 172.21.15.5:0/2876098764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c30100510 msgr2=0x7f6c30196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:49.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.636+0000 7f6c36771640 1 --2- 172.21.15.5:0/2876098764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c30100510 0x7f6c30196cb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009230 tx=0x7f6c200048f0 comp rx=0 tx=0).stop 2024-09-19T05:54:49.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.636+0000 7f6c36771640 1 -- 172.21.15.5:0/2876098764 shutdown_connections 2024-09-19T05:54:49.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.636+0000 7f6c36771640 1 --2- 172.21.15.5:0/2876098764 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6c1c0392a0 0x7f6c1c03b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:49.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.636+0000 7f6c36771640 1 --2- 172.21.15.5:0/2876098764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c30100510 0x7f6c30196cb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:49.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.636+0000 7f6c36771640 1 -- 172.21.15.5:0/2876098764 >> 172.21.15.5:0/2876098764 conn(0x7f6c300fbc80 msgr2=0x7f6c3006a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:49.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.636+0000 7f6c36771640 1 -- 172.21.15.5:0/2876098764 shutdown_connections 2024-09-19T05:54:49.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:49.636+0000 7f6c36771640 1 -- 172.21.15.5:0/2876098764 wait complete. 2024-09-19T05:54:50.339 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:49 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:50.340 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:49 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:50.340 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:49 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi005", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:54:50.340 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:49 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:50.340 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:49 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:54:50.340 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:49 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:50.340 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:49 smithi005 ceph-mon[30955]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:54:50.340 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:49 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:50.340 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:49 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:54:50.341 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:49 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:54:50.341 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:49 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:54:50.341 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:49 smithi005 ceph-mon[30955]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-19T05:54:50.371 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi049 2024-09-19T05:54:50.371 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-19T05:54:50.371 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/etc/ceph/ceph.conf 2024-09-19T05:54:50.391 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-19T05:54:50.391 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:54:50.451 INFO:tasks.cephadm:Adding host smithi049 to orchestrator... 2024-09-19T05:54:50.451 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch host add smithi049 2024-09-19T05:54:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.141+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3184216358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f28102730 msgr2=0x7f8f28102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.141+0000 7f8f2f2f5640 1 --2- 172.21.15.5:0/3184216358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f28102730 0x7f8f28102b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009c10 tx=0x7f8f1802aa10 comp rx=0 tx=0).stop 2024-09-19T05:54:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.141+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3184216358 shutdown_connections 2024-09-19T05:54:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.141+0000 7f8f2f2f5640 1 --2- 172.21.15.5:0/3184216358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f28102730 0x7f8f28102b10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.141+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3184216358 >> 172.21.15.5:0/3184216358 conn(0x7f8f28077310 msgr2=0x7f8f280fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.141+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3184216358 shutdown_connections 2024-09-19T05:54:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.141+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3184216358 wait complete. 2024-09-19T05:54:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.141+0000 7f8f2f2f5640 1 Processor -- start 2024-09-19T05:54:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.141+0000 7f8f2f2f5640 1 -- start start 2024-09-19T05:54:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.142+0000 7f8f2f2f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f28102730 0x7f8f28197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.142+0000 7f8f2f2f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810bb30 con 0x7f8f28102730 2024-09-19T05:54:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.142+0000 7f8f2d091640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f28102730 0x7f8f28197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.142+0000 7f8f2d091640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f28102730 0x7f8f28197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56840/0 (socket says 172.21.15.5:56840) 2024-09-19T05:54:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.142+0000 7f8f2d091640 1 -- 172.21.15.5:0/3991032437 learned_addr learned my addr 172.21.15.5:0/3991032437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.142+0000 7f8f2d091640 1 -- 172.21.15.5:0/3991032437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f28197650 con 0x7f8f28102730 2024-09-19T05:54:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.142+0000 7f8f2d091640 1 --2- 172.21.15.5:0/3991032437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f28102730 0x7f8f28197110 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f18005dd0 tx=0x7f8f1802f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.142+0000 7f8f267fc640 1 -- 172.21.15.5:0/3991032437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f18034070 con 0x7f8f28102730 2024-09-19T05:54:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.142+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3991032437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f281978e0 con 0x7f8f28102730 2024-09-19T05:54:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.142+0000 7f8f267fc640 1 -- 172.21.15.5:0/3991032437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f1802feb0 con 0x7f8f28102730 2024-09-19T05:54:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.142+0000 7f8f267fc640 1 -- 172.21.15.5:0/3991032437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f18004360 con 0x7f8f28102730 2024-09-19T05:54:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.142+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3991032437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f28197ce0 con 0x7f8f28102730 2024-09-19T05:54:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.143+0000 7f8f267fc640 1 -- 172.21.15.5:0/3991032437 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f8f18037020 con 0x7f8f28102730 2024-09-19T05:54:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.143+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3991032437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f28069f90 con 0x7f8f28102730 2024-09-19T05:54:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.144+0000 7f8f267fc640 1 --2- 172.21.15.5:0/3991032437 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f100392a0 0x7f8f1003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.144+0000 7f8f267fc640 1 -- 172.21.15.5:0/3991032437 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8f1806d210 con 0x7f8f28102730 2024-09-19T05:54:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.144+0000 7f8f2c890640 1 --2- 172.21.15.5:0/3991032437 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f100392a0 0x7f8f1003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.144+0000 7f8f2c890640 1 --2- 172.21.15.5:0/3991032437 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f100392a0 0x7f8f1003b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0098c0 tx=0x7f8f1c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.148+0000 7f8f267fc640 1 -- 172.21.15.5:0/3991032437 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8f1803f070 con 0x7f8f28102730 2024-09-19T05:54:51.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:51.331+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3991032437 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi049", "target": ["mon-mgr", ""]}) -- 0x7f8f28198500 con 0x7f8f100392a0 2024-09-19T05:54:51.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:51 smithi005 ceph-mon[30955]: Updating smithi005:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:54:51.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:51 smithi005 ceph-mon[30955]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:54:51.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:51 smithi005 ceph-mon[30955]: Updating smithi005:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:54:51.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:51 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:51.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:51 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:51.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:51 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:52 smithi005 ceph-mon[30955]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi049", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:54:53.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:53 smithi005 ceph-mon[30955]: Deploying cephadm binary to smithi049 2024-09-19T05:54:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:55.230+0000 7f8f267fc640 1 -- 172.21.15.5:0/3991032437 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8f28198500 con 0x7f8f100392a0 2024-09-19T05:54:55.232 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi049' with addr '172.21.15.49' 2024-09-19T05:54:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:55.232+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3991032437 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f100392a0 msgr2=0x7f8f1003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:55.232+0000 7f8f2f2f5640 1 --2- 172.21.15.5:0/3991032437 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f100392a0 0x7f8f1003b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0098c0 tx=0x7f8f1c006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:55.232+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3991032437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f28102730 msgr2=0x7f8f28197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:55.232+0000 7f8f2f2f5640 1 --2- 172.21.15.5:0/3991032437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f28102730 0x7f8f28197110 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f18005dd0 tx=0x7f8f1802f980 comp rx=0 tx=0).stop 2024-09-19T05:54:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:55.232+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3991032437 shutdown_connections 2024-09-19T05:54:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:55.232+0000 7f8f2f2f5640 1 --2- 172.21.15.5:0/3991032437 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f100392a0 0x7f8f1003b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:55.232+0000 7f8f2f2f5640 1 --2- 172.21.15.5:0/3991032437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f28102730 0x7f8f28197110 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:55.232+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3991032437 >> 172.21.15.5:0/3991032437 conn(0x7f8f28077310 msgr2=0x7f8f281054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:55.232+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3991032437 shutdown_connections 2024-09-19T05:54:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:55.232+0000 7f8f2f2f5640 1 -- 172.21.15.5:0/3991032437 wait complete. 2024-09-19T05:54:55.840 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T05:54:56.094 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:56.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:56 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:56.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:56 smithi005 ceph-mon[30955]: Added host smithi049 2024-09-19T05:54:56.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:56 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:54:56.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:56 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:56.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:56 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:56.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.530+0000 7f6db60f5640 1 -- 172.21.15.5:0/1294245226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db0106560 msgr2=0x7f6db0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:56.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.530+0000 7f6db60f5640 1 --2- 172.21.15.5:0/1294245226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db0106560 0x7f6db0106940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009880 tx=0x7f6da002a5d0 comp rx=0 tx=0).stop 2024-09-19T05:54:56.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.530+0000 7f6db60f5640 1 -- 172.21.15.5:0/1294245226 shutdown_connections 2024-09-19T05:54:56.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.530+0000 7f6db60f5640 1 --2- 172.21.15.5:0/1294245226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db0106560 0x7f6db0106940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:56.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.530+0000 7f6db60f5640 1 -- 172.21.15.5:0/1294245226 >> 172.21.15.5:0/1294245226 conn(0x7f6db00fc1b0 msgr2=0x7f6db00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:56.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.530+0000 7f6db60f5640 1 -- 172.21.15.5:0/1294245226 shutdown_connections 2024-09-19T05:54:56.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.530+0000 7f6db60f5640 1 -- 172.21.15.5:0/1294245226 wait complete. 2024-09-19T05:54:56.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.530+0000 7f6db60f5640 1 Processor -- start 2024-09-19T05:54:56.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.531+0000 7f6db60f5640 1 -- start start 2024-09-19T05:54:56.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.531+0000 7f6db60f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db0106560 0x7f6db00fc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:56.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.531+0000 7f6db60f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6db010aa40 con 0x7f6db0106560 2024-09-19T05:54:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.531+0000 7f6daf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db0106560 0x7f6db00fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.531+0000 7f6daf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db0106560 0x7f6db00fc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38348/0 (socket says 172.21.15.5:38348) 2024-09-19T05:54:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.531+0000 7f6daf7fe640 1 -- 172.21.15.5:0/3178067401 learned_addr learned my addr 172.21.15.5:0/3178067401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.531+0000 7f6daf7fe640 1 -- 172.21.15.5:0/3178067401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db00fe550 con 0x7f6db0106560 2024-09-19T05:54:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.531+0000 7f6daf7fe640 1 --2- 172.21.15.5:0/3178067401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db0106560 0x7f6db00fc660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009230 tx=0x7f6da00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.532+0000 7f6dacff9640 1 -- 172.21.15.5:0/3178067401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6da003f070 con 0x7f6db0106560 2024-09-19T05:54:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.532+0000 7f6dacff9640 1 -- 172.21.15.5:0/3178067401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6da0004d10 con 0x7f6db0106560 2024-09-19T05:54:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.532+0000 7f6db60f5640 1 -- 172.21.15.5:0/3178067401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db00fcba0 con 0x7f6db0106560 2024-09-19T05:54:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.532+0000 7f6dacff9640 1 -- 172.21.15.5:0/3178067401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6da0035030 con 0x7f6db0106560 2024-09-19T05:54:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.532+0000 7f6db60f5640 1 -- 172.21.15.5:0/3178067401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db00fd060 con 0x7f6db0106560 2024-09-19T05:54:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.533+0000 7f6db60f5640 1 -- 172.21.15.5:0/3178067401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db0101bd0 con 0x7f6db0106560 2024-09-19T05:54:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.533+0000 7f6dacff9640 1 -- 172.21.15.5:0/3178067401 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f6da0042070 con 0x7f6db0106560 2024-09-19T05:54:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.533+0000 7f6dacff9640 1 --2- 172.21.15.5:0/3178067401 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6d98039300 0x7f6d9803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.533+0000 7f6dacff9640 1 -- 172.21.15.5:0/3178067401 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6da006cf30 con 0x7f6db0106560 2024-09-19T05:54:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.534+0000 7f6daeffd640 1 --2- 172.21.15.5:0/3178067401 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6d98039300 0x7f6d9803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.534+0000 7f6daeffd640 1 --2- 172.21.15.5:0/3178067401 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6d98039300 0x7f6d9803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6da40098c0 tx=0x7f6da4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.538+0000 7f6dacff9640 1 -- 172.21.15.5:0/3178067401 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6da0042360 con 0x7f6db0106560 2024-09-19T05:54:56.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.720+0000 7f6db60f5640 1 -- 172.21.15.5:0/3178067401 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6db00fdc00 con 0x7f6d98039300 2024-09-19T05:54:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.721+0000 7f6dacff9640 1 -- 172.21.15.5:0/3178067401 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7f6db00fdc00 con 0x7f6d98039300 2024-09-19T05:54:56.722 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:54:56.723 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}] 2024-09-19T05:54:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.723+0000 7f6db60f5640 1 -- 172.21.15.5:0/3178067401 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6d98039300 msgr2=0x7f6d9803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.723+0000 7f6db60f5640 1 --2- 172.21.15.5:0/3178067401 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6d98039300 0x7f6d9803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6da40098c0 tx=0x7f6da4006db0 comp rx=0 tx=0).stop 2024-09-19T05:54:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.723+0000 7f6db60f5640 1 -- 172.21.15.5:0/3178067401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db0106560 msgr2=0x7f6db00fc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.723+0000 7f6db60f5640 1 --2- 172.21.15.5:0/3178067401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db0106560 0x7f6db00fc660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009230 tx=0x7f6da00048f0 comp rx=0 tx=0).stop 2024-09-19T05:54:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.723+0000 7f6db60f5640 1 -- 172.21.15.5:0/3178067401 shutdown_connections 2024-09-19T05:54:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.723+0000 7f6db60f5640 1 --2- 172.21.15.5:0/3178067401 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6d98039300 0x7f6d9803b7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.723+0000 7f6db60f5640 1 --2- 172.21.15.5:0/3178067401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db0106560 0x7f6db00fc660 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.723+0000 7f6db60f5640 1 -- 172.21.15.5:0/3178067401 >> 172.21.15.5:0/3178067401 conn(0x7f6db00fc1b0 msgr2=0x7f6db0069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.723+0000 7f6db60f5640 1 -- 172.21.15.5:0/3178067401 shutdown_connections 2024-09-19T05:54:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:56.723+0000 7f6db60f5640 1 -- 172.21.15.5:0/3178067401 wait complete. 2024-09-19T05:54:57.285 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2024-09-19T05:54:57.285 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T05:54:57.285 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2024-09-19T05:54:57.304 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T05:54:57.304 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:54:57.362 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2024-09-19T05:54:57.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch host add smithi157 2024-09-19T05:54:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:57.872 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.046+0000 7f11b0f5a640 1 -- 172.21.15.5:0/4201978694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ac102750 msgr2=0x7f11ac102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.046+0000 7f11b0f5a640 1 --2- 172.21.15.5:0/4201978694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ac102750 0x7f11ac102b30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f119c009880 tx=0x7f119c02a670 comp rx=0 tx=0).stop 2024-09-19T05:54:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.046+0000 7f11b0f5a640 1 -- 172.21.15.5:0/4201978694 shutdown_connections 2024-09-19T05:54:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.046+0000 7f11b0f5a640 1 --2- 172.21.15.5:0/4201978694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ac102750 0x7f11ac102b30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.046+0000 7f11b0f5a640 1 -- 172.21.15.5:0/4201978694 >> 172.21.15.5:0/4201978694 conn(0x7f11ac0fc1b0 msgr2=0x7f11ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.046+0000 7f11b0f5a640 1 -- 172.21.15.5:0/4201978694 shutdown_connections 2024-09-19T05:54:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.046+0000 7f11b0f5a640 1 -- 172.21.15.5:0/4201978694 wait complete. 2024-09-19T05:54:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.047+0000 7f11b0f5a640 1 Processor -- start 2024-09-19T05:54:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.047+0000 7f11b0f5a640 1 -- start start 2024-09-19T05:54:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.047+0000 7f11b0f5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ac102750 0x7f11ac1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.047+0000 7f11b0f5a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac10bb70 con 0x7f11ac102750 2024-09-19T05:54:58.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.047+0000 7f11aa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ac102750 0x7f11ac1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.047+0000 7f11aa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ac102750 0x7f11ac1970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38378/0 (socket says 172.21.15.5:38378) 2024-09-19T05:54:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.047+0000 7f11aa59c640 1 -- 172.21.15.5:0/1865890678 learned_addr learned my addr 172.21.15.5:0/1865890678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:54:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.048+0000 7f11aa59c640 1 -- 172.21.15.5:0/1865890678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac197630 con 0x7f11ac102750 2024-09-19T05:54:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.048+0000 7f11aa59c640 1 --2- 172.21.15.5:0/1865890678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ac102750 0x7f11ac1970f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f119c009230 tx=0x7f119c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.048+0000 7f119b7fe640 1 -- 172.21.15.5:0/1865890678 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f119c03f070 con 0x7f11ac102750 2024-09-19T05:54:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.048+0000 7f11b0f5a640 1 -- 172.21.15.5:0/1865890678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ac1978c0 con 0x7f11ac102750 2024-09-19T05:54:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.048+0000 7f11b0f5a640 1 -- 172.21.15.5:0/1865890678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ac197d20 con 0x7f11ac102750 2024-09-19T05:54:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.048+0000 7f119b7fe640 1 -- 172.21.15.5:0/1865890678 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f119c004d10 con 0x7f11ac102750 2024-09-19T05:54:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.048+0000 7f119b7fe640 1 -- 172.21.15.5:0/1865890678 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f119c035030 con 0x7f11ac102750 2024-09-19T05:54:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.049+0000 7f119b7fe640 1 -- 172.21.15.5:0/1865890678 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f119c042070 con 0x7f11ac102750 2024-09-19T05:54:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.049+0000 7f11b0f5a640 1 -- 172.21.15.5:0/1865890678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ac06a7b0 con 0x7f11ac102750 2024-09-19T05:54:58.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.050+0000 7f119b7fe640 1 --2- 172.21.15.5:0/1865890678 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f11940392a0 0x7f119403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:58.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.050+0000 7f119b7fe640 1 -- 172.21.15.5:0/1865890678 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f119c06d280 con 0x7f11ac102750 2024-09-19T05:54:58.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.050+0000 7f11a9d9b640 1 --2- 172.21.15.5:0/1865890678 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f11940392a0 0x7f119403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:58.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.050+0000 7f11a9d9b640 1 --2- 172.21.15.5:0/1865890678 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f11940392a0 0x7f119403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f11a00098c0 tx=0x7f11a0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:58.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.054+0000 7f119b7fe640 1 -- 172.21.15.5:0/1865890678 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f119c02f300 con 0x7f11ac102750 2024-09-19T05:54:58.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:54:58.238+0000 7f11b0f5a640 1 -- 172.21.15.5:0/1865890678 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}) -- 0x7f11ac198450 con 0x7f11940392a0 2024-09-19T05:54:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:58 smithi005 ceph-mon[30955]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:54:59.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:59 smithi005 ceph-mon[30955]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:54:59.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:59 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:59.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:59 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:59.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:59 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:59.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:59 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:54:59.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:59 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi049", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:54:59.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:59 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:54:59.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:54:59 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:00 smithi005 ceph-mon[30955]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:00 smithi005 ceph-mon[30955]: Updating smithi049:/etc/ceph/ceph.conf 2024-09-19T05:55:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:00 smithi005 ceph-mon[30955]: Deploying cephadm binary to smithi157 2024-09-19T05:55:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:00 smithi005 ceph-mon[30955]: Updating smithi049:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:00.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:00.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:00.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:01.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:01 smithi005 ceph-mon[30955]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:55:01.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:01 smithi005 ceph-mon[30955]: Updating smithi049:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:55:02.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:02.066+0000 7f119b7fe640 1 -- 172.21.15.5:0/1865890678 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f11ac198450 con 0x7f11940392a0 2024-09-19T05:55:02.068 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi157' with addr '172.21.15.157' 2024-09-19T05:55:02.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:02.068+0000 7f11b0f5a640 1 -- 172.21.15.5:0/1865890678 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f11940392a0 msgr2=0x7f119403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:02.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:02.068+0000 7f11b0f5a640 1 --2- 172.21.15.5:0/1865890678 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f11940392a0 0x7f119403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f11a00098c0 tx=0x7f11a0006db0 comp rx=0 tx=0).stop 2024-09-19T05:55:02.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:02.068+0000 7f11b0f5a640 1 -- 172.21.15.5:0/1865890678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ac102750 msgr2=0x7f11ac1970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:02.068+0000 7f11b0f5a640 1 --2- 172.21.15.5:0/1865890678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ac102750 0x7f11ac1970f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f119c009230 tx=0x7f119c0048f0 comp rx=0 tx=0).stop 2024-09-19T05:55:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:02.068+0000 7f11b0f5a640 1 -- 172.21.15.5:0/1865890678 shutdown_connections 2024-09-19T05:55:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:02.068+0000 7f11b0f5a640 1 --2- 172.21.15.5:0/1865890678 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f11940392a0 0x7f119403b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:02.068+0000 7f11b0f5a640 1 --2- 172.21.15.5:0/1865890678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ac102750 0x7f11ac1970f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:02.068+0000 7f11b0f5a640 1 -- 172.21.15.5:0/1865890678 >> 172.21.15.5:0/1865890678 conn(0x7f11ac0fc1b0 msgr2=0x7f11ac105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:02.068+0000 7f11b0f5a640 1 -- 172.21.15.5:0/1865890678 shutdown_connections 2024-09-19T05:55:02.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:02.068+0000 7f11b0f5a640 1 -- 172.21.15.5:0/1865890678 wait complete. 2024-09-19T05:55:02.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T05:55:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:02 smithi005 ceph-mon[30955]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:02.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:02 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:02.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:02 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:02.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:02 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:55:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.401+0000 7f0825fba640 1 -- 172.21.15.5:0/1496243533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0820107660 msgr2=0x7f0820107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.401+0000 7f0825fba640 1 --2- 172.21.15.5:0/1496243533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0820107660 0x7f0820107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0810009c20 tx=0x7f081002aa90 comp rx=0 tx=0).stop 2024-09-19T05:55:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.401+0000 7f0825fba640 1 -- 172.21.15.5:0/1496243533 shutdown_connections 2024-09-19T05:55:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.401+0000 7f0825fba640 1 --2- 172.21.15.5:0/1496243533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0820107660 0x7f0820107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.401+0000 7f0825fba640 1 -- 172.21.15.5:0/1496243533 >> 172.21.15.5:0/1496243533 conn(0x7f08200fd290 msgr2=0x7f08200ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.402+0000 7f0825fba640 1 -- 172.21.15.5:0/1496243533 shutdown_connections 2024-09-19T05:55:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.402+0000 7f0825fba640 1 -- 172.21.15.5:0/1496243533 wait complete. 2024-09-19T05:55:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.402+0000 7f0825fba640 1 Processor -- start 2024-09-19T05:55:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.402+0000 7f0825fba640 1 -- start start 2024-09-19T05:55:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.403+0000 7f0825fba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0820107660 0x7f08200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.403+0000 7f0825fba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f082010bb40 con 0x7f0820107660 2024-09-19T05:55:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.403+0000 7f081f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0820107660 0x7f08200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.403+0000 7f081f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0820107660 0x7f08200744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38398/0 (socket says 172.21.15.5:38398) 2024-09-19T05:55:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.403+0000 7f081f7fe640 1 -- 172.21.15.5:0/1219521327 learned_addr learned my addr 172.21.15.5:0/1219521327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:55:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.403+0000 7f081f7fe640 1 -- 172.21.15.5:0/1219521327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08200763e0 con 0x7f0820107660 2024-09-19T05:55:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.403+0000 7f081f7fe640 1 --2- 172.21.15.5:0/1219521327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0820107660 0x7f08200744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0810005e30 tx=0x7f081002b010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.403+0000 7f081cff9640 1 -- 172.21.15.5:0/1219521327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0810047070 con 0x7f0820107660 2024-09-19T05:55:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.403+0000 7f081cff9640 1 -- 172.21.15.5:0/1219521327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0810003e20 con 0x7f0820107660 2024-09-19T05:55:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.403+0000 7f0825fba640 1 -- 172.21.15.5:0/1219521327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0820074a30 con 0x7f0820107660 2024-09-19T05:55:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.403+0000 7f081cff9640 1 -- 172.21.15.5:0/1219521327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0810033050 con 0x7f0820107660 2024-09-19T05:55:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.403+0000 7f0825fba640 1 -- 172.21.15.5:0/1219521327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0820074e90 con 0x7f0820107660 2024-09-19T05:55:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.404+0000 7f081cff9640 1 -- 172.21.15.5:0/1219521327 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f081004a070 con 0x7f0820107660 2024-09-19T05:55:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.405+0000 7f0825fba640 1 -- 172.21.15.5:0/1219521327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0820102cd0 con 0x7f0820107660 2024-09-19T05:55:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.405+0000 7f081cff9640 1 --2- 172.21.15.5:0/1219521327 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f080c0392a0 0x7f080c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.405+0000 7f081cff9640 1 -- 172.21.15.5:0/1219521327 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f081003e030 con 0x7f0820107660 2024-09-19T05:55:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.405+0000 7f081effd640 1 --2- 172.21.15.5:0/1219521327 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f080c0392a0 0x7f080c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.405+0000 7f081effd640 1 --2- 172.21.15.5:0/1219521327 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f080c0392a0 0x7f080c03b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f08080098c0 tx=0x7f0808006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:03.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.410+0000 7f081cff9640 1 -- 172.21.15.5:0/1219521327 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f081002edf0 con 0x7f0820107660 2024-09-19T05:55:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.594+0000 7f0825fba640 1 -- 172.21.15.5:0/1219521327 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0820075a90 con 0x7f080c0392a0 2024-09-19T05:55:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.595+0000 7f081cff9640 1 -- 172.21.15.5:0/1219521327 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f0820075a90 con 0x7f080c0392a0 2024-09-19T05:55:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:55:03.597 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2024-09-19T05:55:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.597+0000 7f0825fba640 1 -- 172.21.15.5:0/1219521327 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f080c0392a0 msgr2=0x7f080c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.597+0000 7f0825fba640 1 --2- 172.21.15.5:0/1219521327 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f080c0392a0 0x7f080c03b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f08080098c0 tx=0x7f0808006db0 comp rx=0 tx=0).stop 2024-09-19T05:55:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.597+0000 7f0825fba640 1 -- 172.21.15.5:0/1219521327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0820107660 msgr2=0x7f08200744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.597+0000 7f0825fba640 1 --2- 172.21.15.5:0/1219521327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0820107660 0x7f08200744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0810005e30 tx=0x7f081002b010 comp rx=0 tx=0).stop 2024-09-19T05:55:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.597+0000 7f0825fba640 1 -- 172.21.15.5:0/1219521327 shutdown_connections 2024-09-19T05:55:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.597+0000 7f0825fba640 1 --2- 172.21.15.5:0/1219521327 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f080c0392a0 0x7f080c03b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.597+0000 7f0825fba640 1 --2- 172.21.15.5:0/1219521327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0820107660 0x7f08200744f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.598+0000 7f0825fba640 1 -- 172.21.15.5:0/1219521327 >> 172.21.15.5:0/1219521327 conn(0x7f08200fd290 msgr2=0x7f08200fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.598+0000 7f0825fba640 1 -- 172.21.15.5:0/1219521327 shutdown_connections 2024-09-19T05:55:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:03.598+0000 7f0825fba640 1 -- 172.21.15.5:0/1219521327 wait complete. 2024-09-19T05:55:03.653 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:03 smithi005 ceph-mon[30955]: Added host smithi157 2024-09-19T05:55:03.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:03.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:04.349 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T05:55:04.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T05:55:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:55:04.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:04 smithi005 ceph-mon[30955]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:04.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:04 smithi005 ceph-mon[30955]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:55:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.043+0000 7f0df70cf640 1 -- 172.21.15.5:0/3718613515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0df0102730 msgr2=0x7f0df0102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.043+0000 7f0df70cf640 1 --2- 172.21.15.5:0/3718613515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0df0102730 0x7f0df0102b10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009810 tx=0x7f0de402a540 comp rx=0 tx=0).stop 2024-09-19T05:55:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.043+0000 7f0df70cf640 1 -- 172.21.15.5:0/3718613515 shutdown_connections 2024-09-19T05:55:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.043+0000 7f0df70cf640 1 --2- 172.21.15.5:0/3718613515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0df0102730 0x7f0df0102b10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.043+0000 7f0df70cf640 1 -- 172.21.15.5:0/3718613515 >> 172.21.15.5:0/3718613515 conn(0x7f0df0077310 msgr2=0x7f0df00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.043+0000 7f0df70cf640 1 -- 172.21.15.5:0/3718613515 shutdown_connections 2024-09-19T05:55:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.043+0000 7f0df70cf640 1 -- 172.21.15.5:0/3718613515 wait complete. 2024-09-19T05:55:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.043+0000 7f0df70cf640 1 Processor -- start 2024-09-19T05:55:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.043+0000 7f0df70cf640 1 -- start start 2024-09-19T05:55:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.044+0000 7f0df70cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0df0102730 0x7f0df0100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.044+0000 7f0df70cf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0df010bb30 con 0x7f0df0102730 2024-09-19T05:55:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.044+0000 7f0df4e6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0df0102730 0x7f0df0100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.044+0000 7f0df4e6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0df0102730 0x7f0df0100390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54394/0 (socket says 172.21.15.5:54394) 2024-09-19T05:55:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.044+0000 7f0df4e6b640 1 -- 172.21.15.5:0/162215437 learned_addr learned my addr 172.21.15.5:0/162215437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:55:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.044+0000 7f0df4e6b640 1 -- 172.21.15.5:0/162215437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df0102280 con 0x7f0df0102730 2024-09-19T05:55:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.044+0000 7f0df4e6b640 1 --2- 172.21.15.5:0/162215437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0df0102730 0x7f0df0100390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0de4030b30 tx=0x7f0de4030b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.044+0000 7f0dedffb640 1 -- 172.21.15.5:0/162215437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0de4040070 con 0x7f0df0102730 2024-09-19T05:55:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.045+0000 7f0df70cf640 1 -- 172.21.15.5:0/162215437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df01008d0 con 0x7f0df0102730 2024-09-19T05:55:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.045+0000 7f0dedffb640 1 -- 172.21.15.5:0/162215437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0de40318b0 con 0x7f0df0102730 2024-09-19T05:55:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.045+0000 7f0dedffb640 1 -- 172.21.15.5:0/162215437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0de4004950 con 0x7f0df0102730 2024-09-19T05:55:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.045+0000 7f0df70cf640 1 -- 172.21.15.5:0/162215437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df0100d90 con 0x7f0df0102730 2024-09-19T05:55:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.046+0000 7f0dedffb640 1 -- 172.21.15.5:0/162215437 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f0de4039040 con 0x7f0df0102730 2024-09-19T05:55:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.046+0000 7f0df70cf640 1 -- 172.21.15.5:0/162215437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df0069f90 con 0x7f0df0102730 2024-09-19T05:55:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.046+0000 7f0dedffb640 1 --2- 172.21.15.5:0/162215437 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0de00392a0 0x7f0de003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.046+0000 7f0dedffb640 1 -- 172.21.15.5:0/162215437 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0de406d3c0 con 0x7f0df0102730 2024-09-19T05:55:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.046+0000 7f0deffff640 1 --2- 172.21.15.5:0/162215437 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0de00392a0 0x7f0de003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.047+0000 7f0deffff640 1 --2- 172.21.15.5:0/162215437 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0de00392a0 0x7f0de003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0098c0 tx=0x7f0ddc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:05.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.051+0000 7f0dedffb640 1 -- 172.21.15.5:0/162215437 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0de404c350 con 0x7f0df0102730 2024-09-19T05:55:05.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.234+0000 7f0df70cf640 1 -- 172.21.15.5:0/162215437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f0df01018c0 con 0x7f0df0102730 2024-09-19T05:55:05.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.653+0000 7f0dedffb640 1 -- 172.21.15.5:0/162215437 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f0de404b2f0 con 0x7f0df0102730 2024-09-19T05:55:05.656 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-09-19T05:55:05.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.656+0000 7f0df70cf640 1 -- 172.21.15.5:0/162215437 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0de00392a0 msgr2=0x7f0de003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:05.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.656+0000 7f0df70cf640 1 --2- 172.21.15.5:0/162215437 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0de00392a0 0x7f0de003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0098c0 tx=0x7f0ddc006db0 comp rx=0 tx=0).stop 2024-09-19T05:55:05.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.656+0000 7f0df70cf640 1 -- 172.21.15.5:0/162215437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0df0102730 msgr2=0x7f0df0100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:05.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.656+0000 7f0df70cf640 1 --2- 172.21.15.5:0/162215437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0df0102730 0x7f0df0100390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0de4030b30 tx=0x7f0de4030b60 comp rx=0 tx=0).stop 2024-09-19T05:55:05.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.656+0000 7f0df70cf640 1 -- 172.21.15.5:0/162215437 shutdown_connections 2024-09-19T05:55:05.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.656+0000 7f0df70cf640 1 --2- 172.21.15.5:0/162215437 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0de00392a0 0x7f0de003b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:05.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.656+0000 7f0df70cf640 1 --2- 172.21.15.5:0/162215437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0df0102730 0x7f0df0100390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:05.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.656+0000 7f0df70cf640 1 -- 172.21.15.5:0/162215437 >> 172.21.15.5:0/162215437 conn(0x7f0df0077310 msgr2=0x7f0df01034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:05.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.656+0000 7f0df70cf640 1 -- 172.21.15.5:0/162215437 shutdown_connections 2024-09-19T05:55:05.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:05.656+0000 7f0df70cf640 1 -- 172.21.15.5:0/162215437 wait complete. 2024-09-19T05:55:05.946 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:05 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/162215437' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T05:55:06.261 INFO:tasks.cephadm:Adding mon.a on smithi005 2024-09-19T05:55:06.262 INFO:tasks.cephadm:Adding mon.b on smithi049 2024-09-19T05:55:06.262 INFO:tasks.cephadm:Adding mon.c on smithi157 2024-09-19T05:55:06.262 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi157:172.21.15.157=c' 2024-09-19T05:55:06.520 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T05:55:06.565 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T05:55:07.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.010+0000 7f61488a2640 1 -- 172.21.15.157:0/784749915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144100510 msgr2=0x7f6144100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:07.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.010+0000 7f61488a2640 1 --2- 172.21.15.157:0/784749915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144100510 0x7f6144100910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6134009880 tx=0x7f613402a5d0 comp rx=0 tx=0).stop 2024-09-19T05:55:07.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.010+0000 7f61488a2640 1 -- 172.21.15.157:0/784749915 shutdown_connections 2024-09-19T05:55:07.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.010+0000 7f61488a2640 1 --2- 172.21.15.157:0/784749915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144100510 0x7f6144100910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:07.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.010+0000 7f61488a2640 1 -- 172.21.15.157:0/784749915 >> 172.21.15.157:0/784749915 conn(0x7f61440fbca0 msgr2=0x7f61440fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:07.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.010+0000 7f61488a2640 1 -- 172.21.15.157:0/784749915 shutdown_connections 2024-09-19T05:55:07.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.010+0000 7f61488a2640 1 -- 172.21.15.157:0/784749915 wait complete. 2024-09-19T05:55:07.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.010+0000 7f61488a2640 1 Processor -- start 2024-09-19T05:55:07.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.011+0000 7f61488a2640 1 -- start start 2024-09-19T05:55:07.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.011+0000 7f61488a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144100510 0x7f61440fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:07.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.011+0000 7f61488a2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61441064d0 con 0x7f6144100510 2024-09-19T05:55:07.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.011+0000 7f614259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144100510 0x7f61440fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:07.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.011+0000 7f614259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144100510 0x7f61440fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.157:42224/0 (socket says 172.21.15.157:42224) 2024-09-19T05:55:07.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.011+0000 7f614259c640 1 -- 172.21.15.157:0/1691412736 learned_addr learned my addr 172.21.15.157:0/1691412736 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T05:55:07.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.011+0000 7f614259c640 1 -- 172.21.15.157:0/1691412736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61440fe0a0 con 0x7f6144100510 2024-09-19T05:55:07.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.011+0000 7f614259c640 1 --2- 172.21.15.157:0/1691412736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144100510 0x7f61440fdb60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6134009230 tx=0x7f61340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:07.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.012+0000 7f61337fe640 1 -- 172.21.15.157:0/1691412736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f613403f070 con 0x7f6144100510 2024-09-19T05:55:07.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.012+0000 7f61337fe640 1 -- 172.21.15.157:0/1691412736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6134003aa0 con 0x7f6144100510 2024-09-19T05:55:07.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.012+0000 7f61488a2640 1 -- 172.21.15.157:0/1691412736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61440fc290 con 0x7f6144100510 2024-09-19T05:55:07.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.012+0000 7f61488a2640 1 -- 172.21.15.157:0/1691412736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61440fc7d0 con 0x7f6144100510 2024-09-19T05:55:07.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.012+0000 7f61337fe640 1 -- 172.21.15.157:0/1691412736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6134035030 con 0x7f6144100510 2024-09-19T05:55:07.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.013+0000 7f61337fe640 1 -- 172.21.15.157:0/1691412736 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f6134042070 con 0x7f6144100510 2024-09-19T05:55:07.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.013+0000 7f61488a2640 1 -- 172.21.15.157:0/1691412736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6144106c50 con 0x7f6144100510 2024-09-19T05:55:07.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.013+0000 7f61337fe640 1 --2- 172.21.15.157:0/1691412736 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f612c039300 0x7f612c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:07.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.013+0000 7f61337fe640 1 -- 172.21.15.157:0/1691412736 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f613406d220 con 0x7f6144100510 2024-09-19T05:55:07.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.013+0000 7f6141d9b640 1 --2- 172.21.15.157:0/1691412736 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f612c039300 0x7f612c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:07.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.014+0000 7f6141d9b640 1 --2- 172.21.15.157:0/1691412736 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f612c039300 0x7f612c03b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f61380098c0 tx=0x7f6138006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:07.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.018+0000 7f61337fe640 1 -- 172.21.15.157:0/1691412736 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6134039350 con 0x7f6144100510 2024-09-19T05:55:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:06 smithi005 ceph-mon[30955]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:06 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/162215437' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T05:55:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:06 smithi005 ceph-mon[30955]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:55:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:07.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:55:07.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:07.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.204+0000 7f61488a2640 1 -- 172.21.15.157:0/1691412736 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}) -- 0x7f6144190050 con 0x7f612c039300 2024-09-19T05:55:07.210 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.209+0000 7f61337fe640 1 -- 172.21.15.157:0/1691412736 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6144190050 con 0x7f612c039300 2024-09-19T05:55:07.210 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mon update... 2024-09-19T05:55:07.212 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.211+0000 7f61488a2640 1 -- 172.21.15.157:0/1691412736 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f612c039300 msgr2=0x7f612c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:07.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.211+0000 7f61488a2640 1 --2- 172.21.15.157:0/1691412736 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f612c039300 0x7f612c03b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f61380098c0 tx=0x7f6138006db0 comp rx=0 tx=0).stop 2024-09-19T05:55:07.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.211+0000 7f61488a2640 1 -- 172.21.15.157:0/1691412736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144100510 msgr2=0x7f61440fdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:07.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.211+0000 7f61488a2640 1 --2- 172.21.15.157:0/1691412736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144100510 0x7f61440fdb60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6134009230 tx=0x7f61340048f0 comp rx=0 tx=0).stop 2024-09-19T05:55:07.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.211+0000 7f61488a2640 1 -- 172.21.15.157:0/1691412736 shutdown_connections 2024-09-19T05:55:07.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.211+0000 7f61488a2640 1 --2- 172.21.15.157:0/1691412736 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f612c039300 0x7f612c03b7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:07.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.211+0000 7f61488a2640 1 --2- 172.21.15.157:0/1691412736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144100510 0x7f61440fdb60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:07.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.211+0000 7f61488a2640 1 -- 172.21.15.157:0/1691412736 >> 172.21.15.157:0/1691412736 conn(0x7f61440fbca0 msgr2=0x7f614406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:07.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.211+0000 7f61488a2640 1 -- 172.21.15.157:0/1691412736 shutdown_connections 2024-09-19T05:55:07.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:07.211+0000 7f61488a2640 1 -- 172.21.15.157:0/1691412736 wait complete. 2024-09-19T05:55:07.750 DEBUG:teuthology.orchestra.run.smithi049:mon.b> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T05:55:07.753 DEBUG:teuthology.orchestra.run.smithi157:mon.c> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T05:55:07.756 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T05:55:07.756 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T05:55:07.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-19T05:55:07.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:07.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:07.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:07.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:07.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:07.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:07.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:07.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:07.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:07.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:07.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:55:07.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:08.025 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.453+0000 7fe7eec88640 1 -- 172.21.15.157:0/2509934350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8106560 msgr2=0x7fe7e8106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.453+0000 7fe7eec88640 1 --2- 172.21.15.157:0/2509934350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8106560 0x7fe7e8106940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009880 tx=0x7fe7d802a5d0 comp rx=0 tx=0).stop 2024-09-19T05:55:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.453+0000 7fe7eec88640 1 -- 172.21.15.157:0/2509934350 shutdown_connections 2024-09-19T05:55:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.453+0000 7fe7eec88640 1 --2- 172.21.15.157:0/2509934350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8106560 0x7fe7e8106940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.453+0000 7fe7eec88640 1 -- 172.21.15.157:0/2509934350 >> 172.21.15.157:0/2509934350 conn(0x7fe7e80fc1b0 msgr2=0x7fe7e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.453+0000 7fe7eec88640 1 -- 172.21.15.157:0/2509934350 shutdown_connections 2024-09-19T05:55:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.453+0000 7fe7eec88640 1 -- 172.21.15.157:0/2509934350 wait complete. 2024-09-19T05:55:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.454+0000 7fe7eec88640 1 Processor -- start 2024-09-19T05:55:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.454+0000 7fe7eec88640 1 -- start start 2024-09-19T05:55:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.454+0000 7fe7eec88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8106560 0x7fe7e8197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.454+0000 7fe7eec88640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e810aa40 con 0x7fe7e8106560 2024-09-19T05:55:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.455+0000 7fe7eca24640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8106560 0x7fe7e8197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.455+0000 7fe7eca24640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8106560 0x7fe7e8197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.157:42242/0 (socket says 172.21.15.157:42242) 2024-09-19T05:55:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.455+0000 7fe7eca24640 1 -- 172.21.15.157:0/4098414034 learned_addr learned my addr 172.21.15.157:0/4098414034 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T05:55:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.455+0000 7fe7eca24640 1 -- 172.21.15.157:0/4098414034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e8197560 con 0x7fe7e8106560 2024-09-19T05:55:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.455+0000 7fe7eca24640 1 --2- 172.21.15.157:0/4098414034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8106560 0x7fe7e8197020 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009230 tx=0x7fe7d80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.455+0000 7fe7e5ffb640 1 -- 172.21.15.157:0/4098414034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7d803f070 con 0x7fe7e8106560 2024-09-19T05:55:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.455+0000 7fe7eec88640 1 -- 172.21.15.157:0/4098414034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e81977f0 con 0x7fe7e8106560 2024-09-19T05:55:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.455+0000 7fe7e5ffb640 1 -- 172.21.15.157:0/4098414034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7d8003aa0 con 0x7fe7e8106560 2024-09-19T05:55:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.455+0000 7fe7e5ffb640 1 -- 172.21.15.157:0/4098414034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7d8035030 con 0x7fe7e8106560 2024-09-19T05:55:08.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.456+0000 7fe7eec88640 1 -- 172.21.15.157:0/4098414034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e819b8f0 con 0x7fe7e8106560 2024-09-19T05:55:08.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.456+0000 7fe7eec88640 1 -- 172.21.15.157:0/4098414034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e8101bd0 con 0x7fe7e8106560 2024-09-19T05:55:08.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.456+0000 7fe7e5ffb640 1 -- 172.21.15.157:0/4098414034 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fe7d8042070 con 0x7fe7e8106560 2024-09-19T05:55:08.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.457+0000 7fe7e5ffb640 1 --2- 172.21.15.157:0/4098414034 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe7d00392a0 0x7fe7d003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:08.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.457+0000 7fe7e5ffb640 1 -- 172.21.15.157:0/4098414034 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe7d806d070 con 0x7fe7e8106560 2024-09-19T05:55:08.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.457+0000 7fe7e7fff640 1 --2- 172.21.15.157:0/4098414034 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe7d00392a0 0x7fe7d003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:08.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.458+0000 7fe7e7fff640 1 --2- 172.21.15.157:0/4098414034 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe7d00392a0 0x7fe7d003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0098c0 tx=0x7fe7dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.462+0000 7fe7e5ffb640 1 -- 172.21.15.157:0/4098414034 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe7d804a260 con 0x7fe7e8106560 2024-09-19T05:55:08.700 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.698+0000 7fe7eec88640 1 -- 172.21.15.157:0/4098414034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe7e8198380 con 0x7fe7e8106560 2024-09-19T05:55:08.700 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.699+0000 7fe7e5ffb640 1 -- 172.21.15.157:0/4098414034 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7fe7d8031400 con 0x7fe7e8106560 2024-09-19T05:55:08.701 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:55:08.701 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","modified":"2024-09-19T05:53:38.259181Z","created":"2024-09-19T05:53:38.259181Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T05:55:08.701 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-09-19T05:55:08.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.702+0000 7fe7eec88640 1 -- 172.21.15.157:0/4098414034 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe7d00392a0 msgr2=0x7fe7d003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:08.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.702+0000 7fe7eec88640 1 --2- 172.21.15.157:0/4098414034 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe7d00392a0 0x7fe7d003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0098c0 tx=0x7fe7dc006db0 comp rx=0 tx=0).stop 2024-09-19T05:55:08.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.702+0000 7fe7eec88640 1 -- 172.21.15.157:0/4098414034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8106560 msgr2=0x7fe7e8197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:08.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.702+0000 7fe7eec88640 1 --2- 172.21.15.157:0/4098414034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8106560 0x7fe7e8197020 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009230 tx=0x7fe7d80048f0 comp rx=0 tx=0).stop 2024-09-19T05:55:08.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.702+0000 7fe7eec88640 1 -- 172.21.15.157:0/4098414034 shutdown_connections 2024-09-19T05:55:08.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.702+0000 7fe7eec88640 1 --2- 172.21.15.157:0/4098414034 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe7d00392a0 0x7fe7d003b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:08.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.702+0000 7fe7eec88640 1 --2- 172.21.15.157:0/4098414034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8106560 0x7fe7e8197020 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:08.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.702+0000 7fe7eec88640 1 -- 172.21.15.157:0/4098414034 >> 172.21.15.157:0/4098414034 conn(0x7fe7e80fc1b0 msgr2=0x7fe7e8069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:08.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.702+0000 7fe7eec88640 1 -- 172.21.15.157:0/4098414034 shutdown_connections 2024-09-19T05:55:08.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:08.702+0000 7fe7eec88640 1 -- 172.21.15.157:0/4098414034 wait complete. 2024-09-19T05:55:08.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:55:08.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:08.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:55:08.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:55:08.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi157:172.21.15.157=c;count:3 2024-09-19T05:55:08.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T05:55:08.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: Reconfiguring daemon mon.a on smithi005 2024-09-19T05:55:08.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:08.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:08.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:08.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:08.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:08.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:08.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:08.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:10.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:09 smithi005 ceph-mon[30955]: Deploying daemon mon.c on smithi157 2024-09-19T05:55:10.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:09 smithi005 ceph-mon[30955]: from='client.? 172.21.15.157:0/4098414034' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:55:10.327 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T05:55:10.327 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T05:55:11.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:10 smithi005 ceph-mon[30955]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:11.467 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:55:11.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.995+0000 7f39fe249640 1 -- 172.21.15.157:0/2337406448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f81076b0 msgr2=0x7f39f8107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:11.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.995+0000 7f39fe249640 1 --2- 172.21.15.157:0/2337406448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f81076b0 0x7f39f8107a90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009c20 tx=0x7f39e802aa90 comp rx=0 tx=0).stop 2024-09-19T05:55:11.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.995+0000 7f39fe249640 1 -- 172.21.15.157:0/2337406448 shutdown_connections 2024-09-19T05:55:11.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.995+0000 7f39fe249640 1 --2- 172.21.15.157:0/2337406448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f81076b0 0x7f39f8107a90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:11.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.995+0000 7f39fe249640 1 -- 172.21.15.157:0/2337406448 >> 172.21.15.157:0/2337406448 conn(0x7f39f80fd3b0 msgr2=0x7f39f80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:11.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.995+0000 7f39fe249640 1 -- 172.21.15.157:0/2337406448 shutdown_connections 2024-09-19T05:55:11.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.995+0000 7f39fe249640 1 -- 172.21.15.157:0/2337406448 wait complete. 2024-09-19T05:55:11.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.995+0000 7f39fe249640 1 Processor -- start 2024-09-19T05:55:11.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.996+0000 7f39fe249640 1 -- start start 2024-09-19T05:55:11.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.996+0000 7f39fe249640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f81076b0 0x7f39f8199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:12.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.996+0000 7f39fe249640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39f8075ae0 con 0x7f39f81076b0 2024-09-19T05:55:12.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.996+0000 7f39f77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f81076b0 0x7f39f8199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:12.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.996+0000 7f39f77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f81076b0 0x7f39f8199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.157:42258/0 (socket says 172.21.15.157:42258) 2024-09-19T05:55:12.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.996+0000 7f39f77fe640 1 -- 172.21.15.157:0/1949288839 learned_addr learned my addr 172.21.15.157:0/1949288839 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T05:55:12.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.996+0000 7f39f77fe640 1 -- 172.21.15.157:0/1949288839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f8199880 con 0x7f39f81076b0 2024-09-19T05:55:12.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.997+0000 7f39f77fe640 1 --2- 172.21.15.157:0/1949288839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f81076b0 0x7f39f8199340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39e8005e30 tx=0x7f39e802b010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:12.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.997+0000 7f39f4ff9640 1 -- 172.21.15.157:0/1949288839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e8047070 con 0x7f39f81076b0 2024-09-19T05:55:12.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.997+0000 7f39f4ff9640 1 -- 172.21.15.157:0/1949288839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39e8003e20 con 0x7f39f81076b0 2024-09-19T05:55:12.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.997+0000 7f39f4ff9640 1 -- 172.21.15.157:0/1949288839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e8033050 con 0x7f39f81076b0 2024-09-19T05:55:12.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.997+0000 7f39fe249640 1 -- 172.21.15.157:0/1949288839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f8199b10 con 0x7f39f81076b0 2024-09-19T05:55:12.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.997+0000 7f39fe249640 1 -- 172.21.15.157:0/1949288839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f819dc10 con 0x7f39f81076b0 2024-09-19T05:55:12.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.998+0000 7f39f4ff9640 1 -- 172.21.15.157:0/1949288839 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f39e804a070 con 0x7f39f81076b0 2024-09-19T05:55:12.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.998+0000 7f39fe249640 1 -- 172.21.15.157:0/1949288839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f8102df0 con 0x7f39f81076b0 2024-09-19T05:55:12.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.998+0000 7f39f4ff9640 1 --2- 172.21.15.157:0/1949288839 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f39e00392a0 0x7f39e003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:12.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.998+0000 7f39f4ff9640 1 -- 172.21.15.157:0/1949288839 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f39e803e030 con 0x7f39f81076b0 2024-09-19T05:55:12.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.998+0000 7f39f6ffd640 1 --2- 172.21.15.157:0/1949288839 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f39e00392a0 0x7f39e003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:12.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:11.999+0000 7f39f6ffd640 1 --2- 172.21.15.157:0/1949288839 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f39e00392a0 0x7f39e003b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0098c0 tx=0x7f39ec006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:12.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.003+0000 7f39f4ff9640 1 -- 172.21.15.157:0/1949288839 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f39e802edf0 con 0x7f39f81076b0 2024-09-19T05:55:12.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.255+0000 7f39fe249640 1 -- 172.21.15.157:0/1949288839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f39f8107b10 con 0x7f39f81076b0 2024-09-19T05:55:12.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.256+0000 7f39f4ff9640 1 -- 172.21.15.157:0/1949288839 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f39e8051090 con 0x7f39f81076b0 2024-09-19T05:55:12.259 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:55:12.259 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","modified":"2024-09-19T05:53:38.259181Z","created":"2024-09-19T05:53:38.259181Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T05:55:12.259 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-09-19T05:55:12.260 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.258+0000 7f39fe249640 1 -- 172.21.15.157:0/1949288839 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f39e00392a0 msgr2=0x7f39e003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:12.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.258+0000 7f39fe249640 1 --2- 172.21.15.157:0/1949288839 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f39e00392a0 0x7f39e003b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0098c0 tx=0x7f39ec006db0 comp rx=0 tx=0).stop 2024-09-19T05:55:12.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.258+0000 7f39fe249640 1 -- 172.21.15.157:0/1949288839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f81076b0 msgr2=0x7f39f8199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:12.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.258+0000 7f39fe249640 1 --2- 172.21.15.157:0/1949288839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f81076b0 0x7f39f8199340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39e8005e30 tx=0x7f39e802b010 comp rx=0 tx=0).stop 2024-09-19T05:55:12.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.258+0000 7f39fe249640 1 -- 172.21.15.157:0/1949288839 shutdown_connections 2024-09-19T05:55:12.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.258+0000 7f39fe249640 1 --2- 172.21.15.157:0/1949288839 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f39e00392a0 0x7f39e003b760 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:12.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.258+0000 7f39fe249640 1 --2- 172.21.15.157:0/1949288839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f81076b0 0x7f39f8199340 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:12.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.258+0000 7f39fe249640 1 -- 172.21.15.157:0/1949288839 >> 172.21.15.157:0/1949288839 conn(0x7f39f80fd3b0 msgr2=0x7f39f8106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:12.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.258+0000 7f39fe249640 1 -- 172.21.15.157:0/1949288839 shutdown_connections 2024-09-19T05:55:12.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:12.258+0000 7f39fe249640 1 -- 172.21.15.157:0/1949288839 wait complete. 2024-09-19T05:55:12.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:12 smithi157 systemd[1]: Starting Ceph mon.c for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T05:55:13.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:12 smithi157 podman[35327]: 2024-09-19 05:55:12.984030264 +0000 UTC m=+0.112358238 container create 517a293e9503fad54b83dff27f5bc486bf1c147a5466728d5526c900ef64d169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:55:13.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:12 smithi157 podman[35327]: 2024-09-19 05:55:12.88748685 +0000 UTC m=+0.015814797 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:55:13.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:12 smithi005 ceph-mon[30955]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:13.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:12 smithi005 ceph-mon[30955]: from='client.? 172.21.15.157:0/1949288839' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:55:13.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 podman[35327]: 2024-09-19 05:55:13.194804807 +0000 UTC m=+0.323132739 container init 517a293e9503fad54b83dff27f5bc486bf1c147a5466728d5526c900ef64d169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-c, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T05:55:13.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 podman[35327]: 2024-09-19 05:55:13.198023506 +0000 UTC m=+0.326351439 container start 517a293e9503fad54b83dff27f5bc486bf1c147a5466728d5526c900ef64d169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:55:13.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T05:55:13.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T05:55:13.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: pidfile_write: ignore empty --pid-file 2024-09-19T05:55:13.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: load: jerasure load: lrc 2024-09-19T05:55:13.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: RocksDB version: 7.9.2 2024-09-19T05:55:13.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Git sha 0 2024-09-19T05:55:13.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T05:55:13.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: DB SUMMARY 2024-09-19T05:55:13.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: DB Session ID: K8X79CWMS4E4OMG6QXOX 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: CURRENT file: CURRENT 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.error_if_exists: 0 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.create_if_missing: 0 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.paranoid_checks: 1 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.env: 0x56220f5f0940 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T05:55:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.info_log: 0x56221085d6e0 2024-09-19T05:55:13.554 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T05:55:13.555 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.statistics: (nil) 2024-09-19T05:55:13.555 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.use_fsync: 0 2024-09-19T05:55:13.555 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_log_file_size: 0 2024-09-19T05:55:13.555 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T05:55:13.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T05:55:13.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T05:55:13.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T05:55:13.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.allow_fallocate: 1 2024-09-19T05:55:13.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T05:55:13.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T05:55:13.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.use_direct_reads: 0 2024-09-19T05:55:13.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T05:55:13.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T05:55:13.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.db_log_dir: 2024-09-19T05:55:13.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.wal_dir: 2024-09-19T05:55:13.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.write_buffer_manager: 0x56221086cdc0 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T05:55:13.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.unordered_write: 0 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.row_cache: None 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.wal_filter: None 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.two_write_queues: 0 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T05:55:13.558 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.wal_compression: 0 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.atomic_flush: 0 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.log_readahead_size: 0 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_background_jobs: 2 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_background_compactions: -1 2024-09-19T05:55:13.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_subcompactions: 1 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_open_files: -1 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T05:55:13.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_background_flushes: -1 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Compression algorithms supported: 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: kZSTD supported: 0 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: kXpressCompression supported: 0 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: kBZip2Compression supported: 0 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: kLZ4Compression supported: 1 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: kZlibCompression supported: 1 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: kSnappyCompression supported: 1 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-19T05:55:13.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.merge_operator: 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_filter: None 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56221085d320) 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout: cache_index_and_filter_blocks: 1 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout: pin_top_level_index_and_filter: 1 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_type: 0 2024-09-19T05:55:13.562 INFO:journalctl@ceph.mon.c.smithi157.stdout: data_block_index_type: 0 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_shortening: 1 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: checksum: 4 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: no_block_cache: 0 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache: 0x5622108511f0 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_name: BinnedLRUCache 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_options: 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: capacity : 536870912 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: num_shard_bits : 4 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: strict_capacity_limit : 0 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: high_pri_pool_ratio: 0.000 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_compressed: (nil) 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: persistent_cache: (nil) 2024-09-19T05:55:13.563 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_size: 4096 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_size_deviation: 10 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_restart_interval: 16 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_block_restart_interval: 1 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: metadata_block_size: 4096 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: partition_filters: 0 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: use_delta_encoding: 1 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: filter_policy: bloomfilter 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: whole_key_filtering: 1 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: verify_compression: 0 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: read_amp_bytes_per_bit: 0 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: format_version: 5 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: enable_index_compression: 1 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_align: 0 2024-09-19T05:55:13.564 INFO:journalctl@ceph.mon.c.smithi157.stdout: max_auto_readahead_size: 262144 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout: prepopulate_block_cache: 0 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout: initial_auto_readahead_size: 8192 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compression: NoCompression 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.num_levels: 7 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T05:55:13.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T05:55:13.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T05:55:13.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T05:55:13.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.inplace_update_support: 0 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T05:55:13.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.bloom_locality: 0 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.max_successive_merges: 0 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.ttl: 2592000 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.enable_blob_files: false 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.min_blob_size: 0 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T05:55:13.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 464a7772-5d87-43a9-b160-e2ffb0e54277 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726725313225599, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726725313226012, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726725313, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "464a7772-5d87-43a9-b160-e2ffb0e54277", "db_session_id": "K8X79CWMS4E4OMG6QXOX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726725313226106, "job": 1, "event": "recovery_finished"} 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T05:55:13.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56221087ee00 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: DB pointer 0x56221097a000 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** DB Stats ** 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** Compaction Stats [default] ** 2024-09-19T05:55:13.572 INFO:journalctl@ceph.mon.c.smithi157.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T05:55:13.573 INFO:journalctl@ceph.mon.c.smithi157.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T05:55:13.573 INFO:journalctl@ceph.mon.c.smithi157.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T05:55:13.573 INFO:journalctl@ceph.mon.c.smithi157.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T05:55:13.573 INFO:journalctl@ceph.mon.c.smithi157.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T05:55:13.573 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-09-19T05:55:13.573 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** Compaction Stats [default] ** 2024-09-19T05:55:13.573 INFO:journalctl@ceph.mon.c.smithi157.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T05:55:13.573 INFO:journalctl@ceph.mon.c.smithi157.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T05:55:13.573 INFO:journalctl@ceph.mon.c.smithi157.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T05:55:13.573 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-09-19T05:55:13.573 INFO:journalctl@ceph.mon.c.smithi157.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T05:55:13.573 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: Block cache BinnedLRUCache@0x5622108511f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-19T05:55:13.574 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: using public_addr v2:172.21.15.157:0/0 -> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: starting mon.c rank -1 at public addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] at bind addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(???) e0 preinit fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).mds e1 new map 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout: e1 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout: btime 2024-09-19T05:53:51:453912+0000 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout: legacy client fscid: -1 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout: No filesystems configured 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T05:55:13.575 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/162215437' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/162215437' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:55:13.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi157:172.21.15.157=c;count:3 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: Reconfiguring daemon mon.a on smithi005 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: Deploying daemon mon.c on smithi157 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='client.? 172.21.15.157:0/4098414034' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: from='client.? 172.21.15.157:0/1949288839' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 ceph-mon[35346]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 bash[35327]: 517a293e9503fad54b83dff27f5bc486bf1c147a5466728d5526c900ef64d169 2024-09-19T05:55:13.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:13 smithi157 systemd[1]: Started Ceph mon.c for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T05:55:13.828 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T05:55:13.829 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T05:55:14.090 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:55:18.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:55:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: mon.a calling monitor election 2024-09-19T05:55:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: mon.c calling monitor election 2024-09-19T05:55:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: monmap epoch 2 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: last_changed 2024-09-19T05:55:13.249939+0000 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: created 2024-09-19T05:53:38.259181+0000 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: min_mon_release 19 (squid) 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: election_strategy: 1 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T05:55:18.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: fsmap 2024-09-19T05:55:18.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:55:18.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: mgrmap e13: x(active, since 39s) 2024-09-19T05:55:18.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: overall HEALTH_OK 2024-09-19T05:55:18.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:18.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:18.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:18.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:18.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:55:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: mon.a calling monitor election 2024-09-19T05:55:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: mon.c calling monitor election 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: monmap epoch 2 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: last_changed 2024-09-19T05:55:13.249939+0000 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: created 2024-09-19T05:53:38.259181+0000 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: min_mon_release 19 (squid) 2024-09-19T05:55:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: election_strategy: 1 2024-09-19T05:55:18.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-19T05:55:18.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T05:55:18.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: fsmap 2024-09-19T05:55:18.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:55:18.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: mgrmap e13: x(active, since 39s) 2024-09-19T05:55:18.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: overall HEALTH_OK 2024-09-19T05:55:18.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:18.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:18.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:18.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:18.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:18.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.898+0000 7f466f4ba640 1 -- 172.21.15.157:0/3778918295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4654003530 msgr2=0x7f46540059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:18.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.898+0000 7f466f4ba640 1 --2- 172.21.15.157:0/3778918295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4654003530 0x7f46540059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4668069d80 tx=0x7f466402c490 comp rx=0 tx=0).stop 2024-09-19T05:55:18.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.898+0000 7f466f4ba640 1 -- 172.21.15.157:0/3778918295 shutdown_connections 2024-09-19T05:55:18.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.898+0000 7f466f4ba640 1 --2- 172.21.15.157:0/3778918295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4654003530 0x7f46540059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:18.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.898+0000 7f466f4ba640 1 -- 172.21.15.157:0/3778918295 >> 172.21.15.157:0/3778918295 conn(0x7f46680fb9a0 msgr2=0x7f46680fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:18.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.898+0000 7f466f4ba640 1 -- 172.21.15.157:0/3778918295 shutdown_connections 2024-09-19T05:55:18.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.898+0000 7f466f4ba640 1 -- 172.21.15.157:0/3778918295 wait complete. 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466f4ba640 1 Processor -- start 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466f4ba640 1 -- start start 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466f4ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4654003530 0x7f46681993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466f4ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668199920 0x7f466819dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466f4ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4668109640 con 0x7f4654003530 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466f4ba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46681094c0 con 0x7f4668199920 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466ca55640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668199920 0x7f466819dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466d256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4654003530 0x7f46681993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466d256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4654003530 0x7f46681993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.157:46218/0 (socket says 172.21.15.157:46218) 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466ca55640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668199920 0x7f466819dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37632/0 (socket says 172.21.15.157:37632) 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466d256640 1 -- 172.21.15.157:0/2713788208 learned_addr learned my addr 172.21.15.157:0/2713788208 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466ca55640 1 -- 172.21.15.157:0/2713788208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668199920 msgr2=0x7f466819dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T05:55:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466ca55640 1 -- 172.21.15.157:0/2713788208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668199920 msgr2=0x7f466819dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T05:55:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466ca55640 1 --2- 172.21.15.157:0/2713788208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668199920 0x7f466819dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T05:55:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.899+0000 7f466ca55640 1 --2- 172.21.15.157:0/2713788208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668199920 0x7f466819dcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:55:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.900+0000 7f466d256640 1 -- 172.21.15.157:0/2713788208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668199920 msgr2=0x7f466819dcb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:55:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.900+0000 7f466d256640 1 --2- 172.21.15.157:0/2713788208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668199920 0x7f466819dcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.900+0000 7f466d256640 1 -- 172.21.15.157:0/2713788208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4668199f40 con 0x7f4654003530 2024-09-19T05:55:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.900+0000 7f466d256640 1 --2- 172.21.15.157:0/2713788208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4654003530 0x7f46681993e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4664005f30 tx=0x7f46640339a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.900+0000 7f465e7fc640 1 -- 172.21.15.157:0/2713788208 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4664005110 con 0x7f4654003530 2024-09-19T05:55:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.900+0000 7f465e7fc640 1 -- 172.21.15.157:0/2713788208 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4664041070 con 0x7f4654003530 2024-09-19T05:55:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.900+0000 7f466f4ba640 1 -- 172.21.15.157:0/2713788208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466819e1f0 con 0x7f4654003530 2024-09-19T05:55:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.900+0000 7f465e7fc640 1 -- 172.21.15.157:0/2713788208 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4664004980 con 0x7f4654003530 2024-09-19T05:55:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.900+0000 7f466f4ba640 1 -- 172.21.15.157:0/2713788208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466819e730 con 0x7f4654003530 2024-09-19T05:55:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.901+0000 7f465e7fc640 1 -- 172.21.15.157:0/2713788208 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f4664004c30 con 0x7f4654003530 2024-09-19T05:55:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.902+0000 7f466f4ba640 1 -- 172.21.15.157:0/2713788208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4668102e90 con 0x7f4654003530 2024-09-19T05:55:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.902+0000 7f465e7fc640 1 --2- 172.21.15.157:0/2713788208 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f464c039090 0x7f464c03b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.902+0000 7f465e7fc640 1 -- 172.21.15.157:0/2713788208 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f466406e4c0 con 0x7f4654003530 2024-09-19T05:55:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.902+0000 7f466ca55640 1 --2- 172.21.15.157:0/2713788208 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f464c039090 0x7f464c03b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:18.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.903+0000 7f466ca55640 1 --2- 172.21.15.157:0/2713788208 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f464c039090 0x7f464c03b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4658006fd0 tx=0x7f4658008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:18.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:18.907+0000 7f465e7fc640 1 -- 172.21.15.157:0/2713788208 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4664046460 con 0x7f4654003530 2024-09-19T05:55:19.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.137+0000 7f466f4ba640 1 -- 172.21.15.157:0/2713788208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f46681068d0 con 0x7f4654003530 2024-09-19T05:55:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.138+0000 7f465e7fc640 1 -- 172.21.15.157:0/2713788208 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f4664033a50 con 0x7f4654003530 2024-09-19T05:55:19.140 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:55:19.140 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","modified":"2024-09-19T05:55:13.249939Z","created":"2024-09-19T05:53:38.259181Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T05:55:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-09-19T05:55:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.140+0000 7f466f4ba640 1 -- 172.21.15.157:0/2713788208 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f464c039090 msgr2=0x7f464c03b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.140+0000 7f466f4ba640 1 --2- 172.21.15.157:0/2713788208 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f464c039090 0x7f464c03b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4658006fd0 tx=0x7f4658008040 comp rx=0 tx=0).stop 2024-09-19T05:55:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.140+0000 7f466f4ba640 1 -- 172.21.15.157:0/2713788208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4654003530 msgr2=0x7f46681993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.140+0000 7f466f4ba640 1 --2- 172.21.15.157:0/2713788208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4654003530 0x7f46681993e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4664005f30 tx=0x7f46640339a0 comp rx=0 tx=0).stop 2024-09-19T05:55:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.140+0000 7f466f4ba640 1 -- 172.21.15.157:0/2713788208 shutdown_connections 2024-09-19T05:55:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.140+0000 7f466f4ba640 1 --2- 172.21.15.157:0/2713788208 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f464c039090 0x7f464c03b550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.140+0000 7f466f4ba640 1 --2- 172.21.15.157:0/2713788208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668199920 0x7f466819dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.140+0000 7f466f4ba640 1 --2- 172.21.15.157:0/2713788208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4654003530 0x7f46681993e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.140+0000 7f466f4ba640 1 -- 172.21.15.157:0/2713788208 >> 172.21.15.157:0/2713788208 conn(0x7f46680fb9a0 msgr2=0x7f46680fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.140+0000 7f466f4ba640 1 -- 172.21.15.157:0/2713788208 shutdown_connections 2024-09-19T05:55:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:19.140+0000 7f466f4ba640 1 -- 172.21.15.157:0/2713788208 wait complete. 2024-09-19T05:55:19.418 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:19 smithi157 ceph-mon[35346]: Deploying daemon mon.b on smithi049 2024-09-19T05:55:19.418 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:19 smithi157 ceph-mon[35346]: from='client.? 172.21.15.157:0/2713788208' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:55:19.418 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:19 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:19.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:19 smithi005 ceph-mon[30955]: Deploying daemon mon.b on smithi049 2024-09-19T05:55:19.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:19 smithi005 ceph-mon[30955]: from='client.? 172.21.15.157:0/2713788208' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:55:19.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:19 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:20 smithi005 ceph-mon[30955]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:20.671 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T05:55:20.671 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T05:55:20.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:20 smithi157 ceph-mon[35346]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:20.921 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:55:21.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.308+0000 7fa4e7c11640 1 -- 172.21.15.157:0/3062170213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e0102820 msgr2=0x7fa4e0102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:21.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.308+0000 7fa4e7c11640 1 --2- 172.21.15.157:0/3062170213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e0102820 0x7fa4e0102c00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009880 tx=0x7fa4dc02a670 comp rx=0 tx=0).stop 2024-09-19T05:55:21.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.308+0000 7fa4e7c11640 1 -- 172.21.15.157:0/3062170213 shutdown_connections 2024-09-19T05:55:21.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.308+0000 7fa4e7c11640 1 --2- 172.21.15.157:0/3062170213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e0102820 0x7fa4e0102c00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:21.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.308+0000 7fa4e7c11640 1 -- 172.21.15.157:0/3062170213 >> 172.21.15.157:0/3062170213 conn(0x7fa4e00fc220 msgr2=0x7fa4e00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:21.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.308+0000 7fa4e7c11640 1 -- 172.21.15.157:0/3062170213 shutdown_connections 2024-09-19T05:55:21.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.308+0000 7fa4e7c11640 1 -- 172.21.15.157:0/3062170213 wait complete. 2024-09-19T05:55:21.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e7c11640 1 Processor -- start 2024-09-19T05:55:21.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e7c11640 1 -- start start 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e7c11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102820 0x7fa4e0197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e7c11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e0197840 0x7fa4e019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e7c11640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e010bc20 con 0x7fa4e0197840 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e7c11640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e010baa0 con 0x7fa4e0102820 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e59ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102820 0x7fa4e0197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e59ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102820 0x7fa4e0197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37648/0 (socket says 172.21.15.157:37648) 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e59ad640 1 -- 172.21.15.157:0/210968766 learned_addr learned my addr 172.21.15.157:0/210968766 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e59ad640 1 -- 172.21.15.157:0/210968766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102820 msgr2=0x7fa4e0197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e59ad640 1 -- 172.21.15.157:0/210968766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102820 msgr2=0x7fa4e0197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e59ad640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102820 0x7fa4e0197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e59ad640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102820 0x7fa4e0197300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.309+0000 7fa4e51ac640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e0197840 0x7fa4e019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.310+0000 7fa4e51ac640 1 -- 172.21.15.157:0/210968766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102820 msgr2=0x7fa4e0197300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.310+0000 7fa4e51ac640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102820 0x7fa4e0197300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.310+0000 7fa4e51ac640 1 -- 172.21.15.157:0/210968766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e0197e60 con 0x7fa4e0197840 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.310+0000 7fa4e51ac640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e0197840 0x7fa4e019bbd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000b3d0 tx=0x7fa4d000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.310+0000 7fa4d6ffd640 1 -- 172.21.15.157:0/210968766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa4d0011630 con 0x7fa4e0197840 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.310+0000 7fa4d6ffd640 1 -- 172.21.15.157:0/210968766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d0011c80 con 0x7fa4e0197840 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.310+0000 7fa4e7c11640 1 -- 172.21.15.157:0/210968766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e019c110 con 0x7fa4e0197840 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.310+0000 7fa4d6ffd640 1 -- 172.21.15.157:0/210968766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa4d0010d80 con 0x7fa4e0197840 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.310+0000 7fa4e7c11640 1 -- 172.21.15.157:0/210968766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e019c650 con 0x7fa4e0197840 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.311+0000 7fa4d6ffd640 1 -- 172.21.15.157:0/210968766 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fa4d0018020 con 0x7fa4e0197840 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.311+0000 7fa4e7c11640 1 -- 172.21.15.157:0/210968766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e006ad10 con 0x7fa4e0197840 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.311+0000 7fa4d6ffd640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa4cc039030 0x7fa4cc03b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.311+0000 7fa4d6ffd640 1 -- 172.21.15.157:0/210968766 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa4d004bfd0 con 0x7fa4e0197840 2024-09-19T05:55:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.312+0000 7fa4e59ad640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa4cc039030 0x7fa4cc03b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:21.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.312+0000 7fa4e59ad640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa4cc039030 0x7fa4cc03b4f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc02ac60 tx=0x7fa4dc02ab70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:21.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.317+0000 7fa4d6ffd640 1 -- 172.21.15.157:0/210968766 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa4d001f020 con 0x7fa4e0197840 2024-09-19T05:55:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.546+0000 7fa4e7c11640 1 -- 172.21.15.157:0/210968766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa4e019c960 con 0x7fa4e0197840 2024-09-19T05:55:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.547+0000 7fa4d6ffd640 1 -- 172.21.15.157:0/210968766 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fa4d001a920 con 0x7fa4e0197840 2024-09-19T05:55:21.549 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:55:21.549 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","modified":"2024-09-19T05:55:13.249939Z","created":"2024-09-19T05:53:38.259181Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T05:55:21.550 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-09-19T05:55:21.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.549+0000 7fa4e7c11640 1 -- 172.21.15.157:0/210968766 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa4cc039030 msgr2=0x7fa4cc03b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:21.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.549+0000 7fa4e7c11640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa4cc039030 0x7fa4cc03b4f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc02ac60 tx=0x7fa4dc02ab70 comp rx=0 tx=0).stop 2024-09-19T05:55:21.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.549+0000 7fa4e7c11640 1 -- 172.21.15.157:0/210968766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e0197840 msgr2=0x7fa4e019bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.549+0000 7fa4e7c11640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e0197840 0x7fa4e019bbd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000b3d0 tx=0x7fa4d000b890 comp rx=0 tx=0).stop 2024-09-19T05:55:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.549+0000 7fa4e7c11640 1 -- 172.21.15.157:0/210968766 shutdown_connections 2024-09-19T05:55:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.549+0000 7fa4e7c11640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa4cc039030 0x7fa4cc03b4f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.549+0000 7fa4e7c11640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e0197840 0x7fa4e019bbd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.549+0000 7fa4e7c11640 1 --2- 172.21.15.157:0/210968766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102820 0x7fa4e0197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.549+0000 7fa4e7c11640 1 -- 172.21.15.157:0/210968766 >> 172.21.15.157:0/210968766 conn(0x7fa4e00fc220 msgr2=0x7fa4e01055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.549+0000 7fa4e7c11640 1 -- 172.21.15.157:0/210968766 shutdown_connections 2024-09-19T05:55:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:21.549+0000 7fa4e7c11640 1 -- 172.21.15.157:0/210968766 wait complete. 2024-09-19T05:55:22.043 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-19T05:55:22.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: e1 2024-09-19T05:55:22.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: btime 2024-09-19T05:53:51:453912+0000 2024-09-19T05:55:22.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T05:55:22.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T05:55:22.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: legacy client fscid: -1 2024-09-19T05:55:22.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: 2024-09-19T05:55:22.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: No filesystems configured 2024-09-19T05:55:22.044 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T05:55:22.044 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T05:55:22.044 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T05:55:22.044 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T05:55:22.044 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T05:55:22.044 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T05:55:22.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T05:55:22.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:55:22.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:55:22.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:55:22.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:22.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:55:22.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/162215437' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T05:55:22.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/162215437' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:55:22.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi157:172.21.15.157=c;count:3 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: Reconfiguring daemon mon.a on smithi005 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: Deploying daemon mon.c on smithi157 2024-09-19T05:55:22.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='client.? 172.21.15.157:0/4098414034' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='client.? 172.21.15.157:0/1949288839' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.a calling monitor election 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.c calling monitor election 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:22.401 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:22.402 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:22.402 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:22.402 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T05:55:22.402 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: monmap epoch 2 2024-09-19T05:55:22.402 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:22.402 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: last_changed 2024-09-19T05:55:13.249939+0000 2024-09-19T05:55:22.402 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: created 2024-09-19T05:53:38.259181+0000 2024-09-19T05:55:22.402 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: min_mon_release 19 (squid) 2024-09-19T05:55:22.402 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: election_strategy: 1 2024-09-19T05:55:22.402 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: fsmap 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mgrmap e13: x(active, since 39s) 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: overall HEALTH_OK 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: Deploying daemon mon.b on smithi049 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='client.? 172.21.15.157:0/2713788208' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:22.403 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:22.404 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 ceph-mon[34640]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T05:55:22.404 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 bash[34621]: 492ac721e6fc6af54cd66e4454e68f17a52210143ea06b4e65532363c87660cd 2024-09-19T05:55:22.404 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:22 smithi049 systemd[1]: Started Ceph mon.b for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T05:55:23.160 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T05:55:23.160 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T05:55:23.407 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:55:27.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.059+0000 7f43e9158640 1 -- 172.21.15.157:0/2090962393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43d0003530 msgr2=0x7f43d00059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:27.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.059+0000 7f43e9158640 1 --2- 172.21.15.157:0/2090962393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43d0003530 0x7f43d00059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f43e41016a0 tx=0x7f43d402c150 comp rx=0 tx=0).stop 2024-09-19T05:55:27.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.059+0000 7f43e9158640 1 -- 172.21.15.157:0/2090962393 shutdown_connections 2024-09-19T05:55:27.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.059+0000 7f43e9158640 1 --2- 172.21.15.157:0/2090962393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43d0003530 0x7f43d00059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:27.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.059+0000 7f43e9158640 1 -- 172.21.15.157:0/2090962393 >> 172.21.15.157:0/2090962393 conn(0x7f43e40fc220 msgr2=0x7f43e40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:27.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.059+0000 7f43e9158640 1 -- 172.21.15.157:0/2090962393 shutdown_connections 2024-09-19T05:55:27.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.059+0000 7f43e9158640 1 -- 172.21.15.157:0/2090962393 wait complete. 2024-09-19T05:55:27.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.060+0000 7f43e9158640 1 Processor -- start 2024-09-19T05:55:27.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e9158640 1 -- start start 2024-09-19T05:55:27.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e9158640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d0003530 0x7f43e41021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:27.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e9158640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43e4102730 0x7f43e4102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:27.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e9158640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e41a3460 0x7f43e41a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:27.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e9158640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4108520 con 0x7f43e4102730 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e9158640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41083a0 con 0x7f43d0003530 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e9158640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41086a0 con 0x7f43e41a3460 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e41a3460 0x7f43e41a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e41a3460 0x7f43e41a5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44810/0 (socket says 172.21.15.157:44810) 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e359e640 1 -- 172.21.15.157:0/2467999984 learned_addr learned my addr 172.21.15.157:0/2467999984 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e259c640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43e4102730 0x7f43e4102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e2d9d640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d0003530 0x7f43e41021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e359e640 1 -- 172.21.15.157:0/2467999984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e41a3460 msgr2=0x7f43e41a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e359e640 1 -- 172.21.15.157:0/2467999984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e41a3460 msgr2=0x7f43e41a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e359e640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e41a3460 0x7f43e41a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.061+0000 7f43e359e640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e41a3460 0x7f43e41a5850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.062+0000 7f43e2d9d640 1 -- 172.21.15.157:0/2467999984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d0003530 msgr2=0x7f43e41021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.062+0000 7f43e2d9d640 1 -- 172.21.15.157:0/2467999984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d0003530 msgr2=0x7f43e41021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T05:55:27.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.062+0000 7f43e2d9d640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d0003530 0x7f43e41021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T05:55:27.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.062+0000 7f43e2d9d640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d0003530 0x7f43e41021f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:55:27.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.062+0000 7f43e259c640 1 -- 172.21.15.157:0/2467999984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e41a3460 msgr2=0x7f43e41a5850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:55:27.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.062+0000 7f43e259c640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e41a3460 0x7f43e41a5850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:27.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.062+0000 7f43e259c640 1 -- 172.21.15.157:0/2467999984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d0003530 msgr2=0x7f43e41021f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:55:27.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.062+0000 7f43e259c640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d0003530 0x7f43e41021f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:27.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.062+0000 7f43e259c640 1 -- 172.21.15.157:0/2467999984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e41a5e90 con 0x7f43e4102730 2024-09-19T05:55:27.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.062+0000 7f43e259c640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43e4102730 0x7f43e4102b90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00b450 tx=0x7f43cc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:27.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.062+0000 7f43dbfff640 1 -- 172.21.15.157:0/2467999984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc004230 con 0x7f43e4102730 2024-09-19T05:55:27.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.063+0000 7f43dbfff640 1 -- 172.21.15.157:0/2467999984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43cc0043d0 con 0x7f43e4102730 2024-09-19T05:55:27.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.063+0000 7f43e9158640 1 -- 172.21.15.157:0/2467999984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e41a6180 con 0x7f43e4102730 2024-09-19T05:55:27.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.063+0000 7f43dbfff640 1 -- 172.21.15.157:0/2467999984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc003be0 con 0x7f43e4102730 2024-09-19T05:55:27.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.063+0000 7f43e9158640 1 -- 172.21.15.157:0/2467999984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e41a6710 con 0x7f43e4102730 2024-09-19T05:55:27.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.064+0000 7f43dbfff640 1 -- 172.21.15.157:0/2467999984 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f43cc00f050 con 0x7f43e4102730 2024-09-19T05:55:27.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.064+0000 7f43dbfff640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f43c4039730 0x7f43c403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:27.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.064+0000 7f43dbfff640 1 -- 172.21.15.157:0/2467999984 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f43cc04c0f0 con 0x7f43e4102730 2024-09-19T05:55:27.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.064+0000 7f43e2d9d640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f43c4039730 0x7f43c403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:27.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.064+0000 7f43e9158640 1 -- 172.21.15.157:0/2467999984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e41095d0 con 0x7f43e4102730 2024-09-19T05:55:27.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.065+0000 7f43e2d9d640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f43c4039730 0x7f43c403bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f43d4005fd0 tx=0x7f43d4005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:27.072 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.070+0000 7f43dbfff640 1 -- 172.21.15.157:0/2467999984 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f43cc00d040 con 0x7f43e4102730 2024-09-19T05:55:27.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.313+0000 7f43e9158640 1 -- 172.21.15.157:0/2467999984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f43e41a6a30 con 0x7f43e4102730 2024-09-19T05:55:27.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: mon.c calling monitor election 2024-09-19T05:55:27.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:55:27.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: mon.a calling monitor election 2024-09-19T05:55:27.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: monmap epoch 3 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: last_changed 2024-09-19T05:55:22.046974+0000 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: created 2024-09-19T05:53:38.259181+0000 2024-09-19T05:55:27.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: min_mon_release 19 (squid) 2024-09-19T05:55:27.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: election_strategy: 1 2024-09-19T05:55:27.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-19T05:55:27.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T05:55:27.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: 2: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-09-19T05:55:27.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: fsmap 2024-09-19T05:55:27.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:55:27.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: mgrmap e13: x(active, since 48s) 2024-09-19T05:55:27.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: overall HEALTH_OK 2024-09-19T05:55:27.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:27.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:27.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:27.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:27.318 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.314+0000 7f43dbfff640 1 -- 172.21.15.157:0/2467999984 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7f43cc019df0 con 0x7f43e4102730 2024-09-19T05:55:27.319 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:55:27.319 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","modified":"2024-09-19T05:55:22.046974Z","created":"2024-09-19T05:53:38.259181Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T05:55:27.319 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 3 2024-09-19T05:55:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.317+0000 7f43e9158640 1 -- 172.21.15.157:0/2467999984 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f43c4039730 msgr2=0x7f43c403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.317+0000 7f43e9158640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f43c4039730 0x7f43c403bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f43d4005fd0 tx=0x7f43d4005d10 comp rx=0 tx=0).stop 2024-09-19T05:55:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.317+0000 7f43e9158640 1 -- 172.21.15.157:0/2467999984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43e4102730 msgr2=0x7f43e4102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.317+0000 7f43e9158640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43e4102730 0x7f43e4102b90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00b450 tx=0x7f43cc00b910 comp rx=0 tx=0).stop 2024-09-19T05:55:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.317+0000 7f43e9158640 1 -- 172.21.15.157:0/2467999984 shutdown_connections 2024-09-19T05:55:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.317+0000 7f43e9158640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f43c4039730 0x7f43c403bbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.317+0000 7f43e9158640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e41a3460 0x7f43e41a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.317+0000 7f43e9158640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43e4102730 0x7f43e4102b90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.317+0000 7f43e9158640 1 --2- 172.21.15.157:0/2467999984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d0003530 0x7f43e41021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.317+0000 7f43e9158640 1 -- 172.21.15.157:0/2467999984 >> 172.21.15.157:0/2467999984 conn(0x7f43e40fc220 msgr2=0x7f43e41059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.317+0000 7f43e9158640 1 -- 172.21.15.157:0/2467999984 shutdown_connections 2024-09-19T05:55:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:27.317+0000 7f43e9158640 1 -- 172.21.15.157:0/2467999984 wait complete. 2024-09-19T05:55:27.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: mon.c calling monitor election 2024-09-19T05:55:27.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:55:27.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: mon.a calling monitor election 2024-09-19T05:55:27.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:27.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:27.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:27.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:27.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:27.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:27.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:27.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:27.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:27.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: monmap epoch 3 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: last_changed 2024-09-19T05:55:22.046974+0000 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: created 2024-09-19T05:53:38.259181+0000 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: min_mon_release 19 (squid) 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: election_strategy: 1 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: 2: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: fsmap 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: mgrmap e13: x(active, since 48s) 2024-09-19T05:55:27.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: overall HEALTH_OK 2024-09-19T05:55:27.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:27.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:27.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:27.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:27.894 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T05:55:27.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T05:55:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:55:28.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:28 smithi005 ceph-mon[30955]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:28.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:28 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:28.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:28 smithi005 ceph-mon[30955]: from='client.? 172.21.15.157:0/2467999984' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:55:28.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:28 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:28.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:28 smithi157 ceph-mon[35346]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:28.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:28 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:28.381 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:28 smithi157 ceph-mon[35346]: from='client.? 172.21.15.157:0/2467999984' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:55:28.381 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:28 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.637+0000 7f4eeba9b640 1 -- 172.21.15.5:0/3139029092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee4100630 msgr2=0x7f4ee4100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.637+0000 7f4eeba9b640 1 --2- 172.21.15.5:0/3139029092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee4100630 0x7f4ee4100a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009880 tx=0x7f4ee002a670 comp rx=0 tx=0).stop 2024-09-19T05:55:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.637+0000 7f4eeba9b640 1 -- 172.21.15.5:0/3139029092 shutdown_connections 2024-09-19T05:55:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.637+0000 7f4eeba9b640 1 --2- 172.21.15.5:0/3139029092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee4100630 0x7f4ee4100a10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.637+0000 7f4eeba9b640 1 -- 172.21.15.5:0/3139029092 >> 172.21.15.5:0/3139029092 conn(0x7f4ee40fc280 msgr2=0x7f4ee40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.637+0000 7f4eeba9b640 1 -- 172.21.15.5:0/3139029092 shutdown_connections 2024-09-19T05:55:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.637+0000 7f4eeba9b640 1 -- 172.21.15.5:0/3139029092 wait complete. 2024-09-19T05:55:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.638+0000 7f4eeba9b640 1 Processor -- start 2024-09-19T05:55:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.638+0000 7f4eeba9b640 1 -- start start 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.638+0000 7f4eeba9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ee4100630 0x7f4ee41a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.638+0000 7f4eeba9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee41a3c70 0x7f4ee419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.638+0000 7f4eeba9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ee419de30 0x7f4ee419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.638+0000 7f4eeba9b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4112270 con 0x7f4ee41a3c70 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.638+0000 7f4eeba9b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41120f0 con 0x7f4ee419de30 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.638+0000 7f4eeba9b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41123f0 con 0x7f4ee4100630 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.638+0000 7f4ee9036640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee41a3c70 0x7f4ee419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.638+0000 7f4ee9036640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee41a3c70 0x7f4ee419d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54946/0 (socket says 172.21.15.5:54946) 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.638+0000 7f4ee9036640 1 -- 172.21.15.5:0/1898782228 learned_addr learned my addr 172.21.15.5:0/1898782228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.639+0000 7f4ee9036640 1 -- 172.21.15.5:0/1898782228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ee4100630 msgr2=0x7f4ee41a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.639+0000 7f4ee9036640 1 --2- 172.21.15.5:0/1898782228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ee4100630 0x7f4ee41a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.639+0000 7f4ee9036640 1 -- 172.21.15.5:0/1898782228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ee419de30 msgr2=0x7f4ee419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.639+0000 7f4ee9036640 1 --2- 172.21.15.5:0/1898782228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ee419de30 0x7f4ee419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.639+0000 7f4ee9036640 1 -- 172.21.15.5:0/1898782228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee41aa600 con 0x7f4ee41a3c70 2024-09-19T05:55:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.639+0000 7f4ee9036640 1 --2- 172.21.15.5:0/1898782228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee41a3c70 0x7f4ee419d8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400b520 tx=0x7f4ed400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.639+0000 7f4edaffd640 1 -- 172.21.15.5:0/1898782228 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4004230 con 0x7f4ee41a3c70 2024-09-19T05:55:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.639+0000 7f4edaffd640 1 -- 172.21.15.5:0/1898782228 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed40043d0 con 0x7f4ee41a3c70 2024-09-19T05:55:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.639+0000 7f4eeba9b640 1 -- 172.21.15.5:0/1898782228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee41aa830 con 0x7f4ee41a3c70 2024-09-19T05:55:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.639+0000 7f4edaffd640 1 -- 172.21.15.5:0/1898782228 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4003cf0 con 0x7f4ee41a3c70 2024-09-19T05:55:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.639+0000 7f4eeba9b640 1 -- 172.21.15.5:0/1898782228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41aadc0 con 0x7f4ee41a3c70 2024-09-19T05:55:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.640+0000 7f4edaffd640 1 -- 172.21.15.5:0/1898782228 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f4ed400e070 con 0x7f4ee41a3c70 2024-09-19T05:55:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.640+0000 7f4eeba9b640 1 -- 172.21.15.5:0/1898782228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee4100a90 con 0x7f4ee41a3c70 2024-09-19T05:55:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.640+0000 7f4edaffd640 1 --2- 172.21.15.5:0/1898782228 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4ed0039730 0x7f4ed003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.640+0000 7f4edaffd640 1 -- 172.21.15.5:0/1898782228 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4ed404bc80 con 0x7f4ee41a3c70 2024-09-19T05:55:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.640+0000 7f4ee9837640 1 --2- 172.21.15.5:0/1898782228 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4ed0039730 0x7f4ed003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.641+0000 7f4ee9837640 1 --2- 172.21.15.5:0/1898782228 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4ed0039730 0x7f4ed003bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009230 tx=0x7f4ee0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.645+0000 7f4edaffd640 1 -- 172.21.15.5:0/1898782228 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4ed4014020 con 0x7f4ee41a3c70 2024-09-19T05:55:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.905+0000 7f4eeba9b640 1 -- 172.21.15.5:0/1898782228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f4ee41ab2a0 con 0x7f4ee41a3c70 2024-09-19T05:55:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.906+0000 7f4edaffd640 1 -- 172.21.15.5:0/1898782228 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f4ed4020350 con 0x7f4ee41a3c70 2024-09-19T05:55:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:# minimal ceph.conf for 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-09-19T05:55:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: mon_host = [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-09-19T05:55:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.909+0000 7f4eeba9b640 1 -- 172.21.15.5:0/1898782228 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4ed0039730 msgr2=0x7f4ed003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.909+0000 7f4eeba9b640 1 --2- 172.21.15.5:0/1898782228 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4ed0039730 0x7f4ed003bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009230 tx=0x7f4ee0031400 comp rx=0 tx=0).stop 2024-09-19T05:55:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.909+0000 7f4eeba9b640 1 -- 172.21.15.5:0/1898782228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee41a3c70 msgr2=0x7f4ee419d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.909+0000 7f4eeba9b640 1 --2- 172.21.15.5:0/1898782228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee41a3c70 0x7f4ee419d8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400b520 tx=0x7f4ed400b9e0 comp rx=0 tx=0).stop 2024-09-19T05:55:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.909+0000 7f4eeba9b640 1 -- 172.21.15.5:0/1898782228 shutdown_connections 2024-09-19T05:55:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.909+0000 7f4eeba9b640 1 --2- 172.21.15.5:0/1898782228 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4ed0039730 0x7f4ed003bbf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.909+0000 7f4eeba9b640 1 --2- 172.21.15.5:0/1898782228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ee419de30 0x7f4ee419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.909+0000 7f4eeba9b640 1 --2- 172.21.15.5:0/1898782228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee41a3c70 0x7f4ee419d8f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.909+0000 7f4eeba9b640 1 --2- 172.21.15.5:0/1898782228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ee4100630 0x7f4ee41a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.909+0000 7f4eeba9b640 1 -- 172.21.15.5:0/1898782228 >> 172.21.15.5:0/1898782228 conn(0x7f4ee40fc280 msgr2=0x7f4ee40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.909+0000 7f4eeba9b640 1 -- 172.21.15.5:0/1898782228 shutdown_connections 2024-09-19T05:55:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:28.909+0000 7f4eeba9b640 1 -- 172.21.15.5:0/1898782228 wait complete. 2024-09-19T05:55:29.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: mon.c calling monitor election 2024-09-19T05:55:29.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:55:29.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: mon.a calling monitor election 2024-09-19T05:55:29.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:29.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:55:29.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:29.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:29.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:29.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:29.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:29.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:29.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:29.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T05:55:29.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: monmap epoch 3 2024-09-19T05:55:29.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:29.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: last_changed 2024-09-19T05:55:22.046974+0000 2024-09-19T05:55:29.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: created 2024-09-19T05:53:38.259181+0000 2024-09-19T05:55:29.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: min_mon_release 19 (squid) 2024-09-19T05:55:29.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: election_strategy: 1 2024-09-19T05:55:29.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: 2: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: fsmap 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: mgrmap e13: x(active, since 48s) 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: overall HEALTH_OK 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:29.399 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='client.? 172.21.15.157:0/2467999984' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:55:29.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:29.657 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T05:55:29.657 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:55:29.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T05:55:29.688 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:55:29.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:55:29.754 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-19T05:55:29.754 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T05:55:29.782 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-19T05:55:29.782 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:55:29.844 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: mon.b calling monitor election 2024-09-19T05:55:29.845 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:29.845 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:29.845 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: Updating smithi005:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:29.845 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: Updating smithi049:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:29.845 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: mon.b calling monitor election 2024-09-19T05:55:29.845 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: mon.c calling monitor election 2024-09-19T05:55:29.845 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: mon.a calling monitor election 2024-09-19T05:55:29.845 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T05:55:29.845 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: monmap epoch 3 2024-09-19T05:55:29.845 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:29.845 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: last_changed 2024-09-19T05:55:22.046974+0000 2024-09-19T05:55:29.845 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: created 2024-09-19T05:53:38.259181+0000 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: min_mon_release 19 (squid) 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: election_strategy: 1 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: 2: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: fsmap 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: mgrmap e13: x(active, since 50s) 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: overall HEALTH_OK 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.846 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.847 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.847 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.847 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.847 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: Reconfiguring mon.a (monmap changed)... 2024-09-19T05:55:29.847 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:29.847 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:55:29.847 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:29.847 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:29 smithi049 ceph-mon[34640]: Reconfiguring daemon mon.a on smithi005 2024-09-19T05:55:29.848 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T05:55:29.848 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T05:55:29.876 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T05:55:29.876 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:55:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: mon.b calling monitor election 2024-09-19T05:55:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: Updating smithi005:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: Updating smithi049:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: mon.b calling monitor election 2024-09-19T05:55:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: mon.c calling monitor election 2024-09-19T05:55:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: mon.a calling monitor election 2024-09-19T05:55:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T05:55:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: monmap epoch 3 2024-09-19T05:55:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: last_changed 2024-09-19T05:55:22.046974+0000 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: created 2024-09-19T05:53:38.259181+0000 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: min_mon_release 19 (squid) 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: election_strategy: 1 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: 2: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: fsmap 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: mgrmap e13: x(active, since 50s) 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: overall HEALTH_OK 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:29.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: Reconfiguring mon.a (monmap changed)... 2024-09-19T05:55:29.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:29.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:55:29.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:29.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:29 smithi005 ceph-mon[30955]: Reconfiguring daemon mon.a on smithi005 2024-09-19T05:55:29.942 INFO:tasks.cephadm:Adding mgr.x on smithi005 2024-09-19T05:55:29.942 INFO:tasks.cephadm:Adding mgr.y on smithi049 2024-09-19T05:55:29.942 INFO:tasks.cephadm:Adding mgr.z on smithi157 2024-09-19T05:55:29.942 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi005=x;smithi049=y;smithi157=z' 2024-09-19T05:55:30.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: mon.b calling monitor election 2024-09-19T05:55:30.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:30.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:30.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: Updating smithi005:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: Updating smithi049:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: mon.b calling monitor election 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: mon.c calling monitor election 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: mon.a calling monitor election 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: monmap epoch 3 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: last_changed 2024-09-19T05:55:22.046974+0000 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: created 2024-09-19T05:53:38.259181+0000 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: min_mon_release 19 (squid) 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: election_strategy: 1 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: 2: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-09-19T05:55:30.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: fsmap 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: mgrmap e13: x(active, since 50s) 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: overall HEALTH_OK 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: Reconfiguring mon.a (monmap changed)... 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:55:30.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:30.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:29 smithi157 ceph-mon[35346]: Reconfiguring daemon mon.a on smithi005 2024-09-19T05:55:30.219 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:55:30.647 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.642+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1684968407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19e81005f0 msgr2=0x7f19e81009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.642+0000 7f19ee7ba640 1 --2- 172.21.15.157:0/1684968407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19e81005f0 0x7f19e81009f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009880 tx=0x7f19d802a5d0 comp rx=0 tx=0).stop 2024-09-19T05:55:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.642+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1684968407 shutdown_connections 2024-09-19T05:55:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.642+0000 7f19ee7ba640 1 --2- 172.21.15.157:0/1684968407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19e81005f0 0x7f19e81009f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.642+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1684968407 >> 172.21.15.157:0/1684968407 conn(0x7f19e80fbda0 msgr2=0x7f19e80fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.642+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1684968407 shutdown_connections 2024-09-19T05:55:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.642+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1684968407 wait complete. 2024-09-19T05:55:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ee7ba640 1 Processor -- start 2024-09-19T05:55:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ee7ba640 1 -- start start 2024-09-19T05:55:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ee7ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19e81005f0 0x7f19e818d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ee7ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e818bdf0 0x7f19e818c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ee7ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e818c7b0 0x7f19e81a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ee7ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8106590 con 0x7f19e81005f0 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ee7ba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8106410 con 0x7f19e818bdf0 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ee7ba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8106710 con 0x7f19e818c7b0 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19e81005f0 0x7f19e818d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ecd57640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e818c7b0 0x7f19e81a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19e81005f0 0x7f19e818d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.157:48552/0 (socket says 172.21.15.157:48552) 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ecd57640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e818c7b0 0x7f19e81a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44812/0 (socket says 172.21.15.157:44812) 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19e7fff640 1 -- 172.21.15.157:0/1558797954 learned_addr learned my addr 172.21.15.157:0/1558797954 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19dffff640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e818bdf0 0x7f19e818c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ecd57640 1 -- 172.21.15.157:0/1558797954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e818bdf0 msgr2=0x7f19e818c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ecd57640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e818bdf0 0x7f19e818c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ecd57640 1 -- 172.21.15.157:0/1558797954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19e81005f0 msgr2=0x7f19e818d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ecd57640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19e81005f0 0x7f19e818d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.643+0000 7f19ecd57640 1 -- 172.21.15.157:0/1558797954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e81a1530 con 0x7f19e818c7b0 2024-09-19T05:55:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.644+0000 7f19ecd57640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e818c7b0 0x7f19e81a0fa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f19e000bd50 tx=0x7f19e000be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.644+0000 7f19e5ffb640 1 -- 172.21.15.157:0/1558797954 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0019070 con 0x7f19e818c7b0 2024-09-19T05:55:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.644+0000 7f19e5ffb640 1 -- 172.21.15.157:0/1558797954 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19e00148f0 con 0x7f19e818c7b0 2024-09-19T05:55:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.644+0000 7f19e5ffb640 1 -- 172.21.15.157:0/1558797954 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0012910 con 0x7f19e818c7b0 2024-09-19T05:55:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.644+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1558797954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e81a1820 con 0x7f19e818c7b0 2024-09-19T05:55:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.644+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1558797954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e81a1de0 con 0x7f19e818c7b0 2024-09-19T05:55:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.645+0000 7f19e5ffb640 1 -- 172.21.15.157:0/1558797954 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f19e001c070 con 0x7f19e818c7b0 2024-09-19T05:55:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.645+0000 7f19e5ffb640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f19d4039730 0x7f19d403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.645+0000 7f19e5ffb640 1 -- 172.21.15.157:0/1558797954 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f19e004c240 con 0x7f19e818c7b0 2024-09-19T05:55:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.645+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1558797954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e81a2040 con 0x7f19e818c7b0 2024-09-19T05:55:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.645+0000 7f19e7fff640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f19d4039730 0x7f19d403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:30.651 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.646+0000 7f19e7fff640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f19d4039730 0x7f19d403bbf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f19d80096c0 tx=0x7f19d8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:30.652 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.650+0000 7f19e5ffb640 1 -- 172.21.15.157:0/1558797954 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f19e0013c30 con 0x7f19e818c7b0 2024-09-19T05:55:30.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.830+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1558797954 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi049=y;smithi157=z", "target": ["mon-mgr", ""]}) -- 0x7f19e806ab40 con 0x7f19d4039730 2024-09-19T05:55:30.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.836+0000 7f19e5ffb640 1 -- 172.21.15.157:0/1558797954 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f19e806ab40 con 0x7f19d4039730 2024-09-19T05:55:30.839 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mgr update... 2024-09-19T05:55:30.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.838+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1558797954 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f19d4039730 msgr2=0x7f19d403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:30.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.838+0000 7f19ee7ba640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f19d4039730 0x7f19d403bbf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f19d80096c0 tx=0x7f19d8005a20 comp rx=0 tx=0).stop 2024-09-19T05:55:30.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.838+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1558797954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e818c7b0 msgr2=0x7f19e81a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:30.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.838+0000 7f19ee7ba640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e818c7b0 0x7f19e81a0fa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f19e000bd50 tx=0x7f19e000be40 comp rx=0 tx=0).stop 2024-09-19T05:55:30.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.838+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1558797954 shutdown_connections 2024-09-19T05:55:30.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.838+0000 7f19ee7ba640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f19d4039730 0x7f19d403bbf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:30.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.838+0000 7f19ee7ba640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e818c7b0 0x7f19e81a0fa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:30.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.838+0000 7f19ee7ba640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e818bdf0 0x7f19e818c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:30.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.838+0000 7f19ee7ba640 1 --2- 172.21.15.157:0/1558797954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19e81005f0 0x7f19e818d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:30.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.838+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1558797954 >> 172.21.15.157:0/1558797954 conn(0x7f19e80fbda0 msgr2=0x7f19e806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:30.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.839+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1558797954 shutdown_connections 2024-09-19T05:55:30.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:55:30.839+0000 7f19ee7ba640 1 -- 172.21.15.157:0/1558797954 wait complete. 2024-09-19T05:55:30.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:30 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:31.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:30 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:31.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:30 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:55:31.369 DEBUG:teuthology.orchestra.run.smithi049:mgr.y> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T05:55:31.373 DEBUG:teuthology.orchestra.run.smithi157:mgr.z> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T05:55:31.411 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T05:55:31.412 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T05:55:31.412 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T05:55:31.428 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T05:55:31.428 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-09-19T05:55:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T05:55:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:55:31.485 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T05:55:31.485 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:55:31.485 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:55:31.485 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-19 05:54:47.251368067 +0000 2024-09-19T05:55:31.485 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-19 05:50:36.584046505 +0000 2024-09-19T05:55:31.485 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-19 05:50:36.584046505 +0000 2024-09-19T05:55:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-19 05:50:36.584046505 +0000 2024-09-19T05:55:31.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T05:55:31.551 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-19T05:55:31.551 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-19T05:55:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000104063 s, 4.9 MB/s 2024-09-19T05:55:31.553 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T05:55:31.609 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-09-19T05:55:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T05:55:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:55:31.666 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T05:55:31.667 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:55:31.667 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:55:31.667 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-19 05:54:47.277367532 +0000 2024-09-19T05:55:31.667 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-19 05:50:36.901039965 +0000 2024-09-19T05:55:31.667 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-19 05:50:36.901039965 +0000 2024-09-19T05:55:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-19 05:50:36.901039965 +0000 2024-09-19T05:55:31.667 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T05:55:31.731 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-19T05:55:31.731 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-19T05:55:31.731 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000110278 s, 4.6 MB/s 2024-09-19T05:55:31.733 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T05:55:31.789 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-09-19T05:55:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T05:55:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:55:31.846 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-19T05:55:31.846 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:55:31.846 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:55:31.846 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-19 05:54:47.304366976 +0000 2024-09-19T05:55:31.846 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-19 05:50:37.220033384 +0000 2024-09-19T05:55:31.846 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-19 05:50:37.220033384 +0000 2024-09-19T05:55:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-19 05:50:37.220033384 +0000 2024-09-19T05:55:31.846 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T05:55:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-19T05:55:31.911 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-19T05:55:31.911 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.00010894 s, 4.7 MB/s 2024-09-19T05:55:31.912 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T05:55:31.969 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-09-19T05:55:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T05:55:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:55:32.025 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-19T05:55:32.025 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:55:32.025 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:55:32.026 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-19 05:54:47.334366358 +0000 2024-09-19T05:55:32.026 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-19 05:50:37.546026659 +0000 2024-09-19T05:55:32.026 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-19 05:50:37.546026659 +0000 2024-09-19T05:55:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-19 05:50:37.546026659 +0000 2024-09-19T05:55:32.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T05:55:32.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:31 smithi005 ceph-mon[30955]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:32.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:31 smithi005 ceph-mon[30955]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi049=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:55:32.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:31 smithi005 ceph-mon[30955]: Saving service mgr spec with placement smithi005=x;smithi049=y;smithi157=z;count:3 2024-09-19T05:55:32.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:31 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:32.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:31 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:32.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:31 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:32.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:31 smithi005 ceph-mon[30955]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T05:55:32.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:31 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:55:32.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:31 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:55:32.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:31 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:32.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:31 smithi005 ceph-mon[30955]: Reconfiguring daemon mgr.x on smithi005 2024-09-19T05:55:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-19T05:55:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-19T05:55:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000102656 s, 5.0 MB/s 2024-09-19T05:55:32.091 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T05:55:32.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:31 smithi049 ceph-mon[34640]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:32.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:31 smithi049 ceph-mon[34640]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi049=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:55:32.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:31 smithi049 ceph-mon[34640]: Saving service mgr spec with placement smithi005=x;smithi049=y;smithi157=z;count:3 2024-09-19T05:55:32.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:31 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:32.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:31 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:32.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:31 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:32.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:31 smithi049 ceph-mon[34640]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T05:55:32.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:31 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:55:32.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:31 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:55:32.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:31 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:32.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:31 smithi049 ceph-mon[34640]: Reconfiguring daemon mgr.x on smithi005 2024-09-19T05:55:32.148 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-19T05:55:32.148 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T05:55:32.169 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T05:55:32.170 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-09-19T05:55:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T05:55:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:55:32.229 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T05:55:32.229 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:55:32.229 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:55:32.229 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-09-19 05:55:21.478066601 +0000 2024-09-19T05:55:32.229 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-09-19 05:50:36.754767665 +0000 2024-09-19T05:55:32.229 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-09-19 05:50:36.754767665 +0000 2024-09-19T05:55:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-09-19 05:50:36.754767665 +0000 2024-09-19T05:55:32.230 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T05:55:32.299 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-09-19T05:55:32.299 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-09-19T05:55:32.299 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000131486 s, 3.9 MB/s 2024-09-19T05:55:32.301 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T05:55:32.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:31 smithi157 ceph-mon[35346]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:32.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:31 smithi157 ceph-mon[35346]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi049=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:55:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:31 smithi157 ceph-mon[35346]: Saving service mgr spec with placement smithi005=x;smithi049=y;smithi157=z;count:3 2024-09-19T05:55:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:31 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:31 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:31 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:31 smithi157 ceph-mon[35346]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T05:55:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:31 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:55:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:31 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:55:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:31 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:31 smithi157 ceph-mon[35346]: Reconfiguring daemon mgr.x on smithi005 2024-09-19T05:55:32.361 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-09-19T05:55:32.419 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T05:55:32.419 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:55:32.419 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T05:55:32.419 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:55:32.419 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:55:32.419 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-09-19 05:55:21.479066574 +0000 2024-09-19T05:55:32.419 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-09-19 05:50:37.078758917 +0000 2024-09-19T05:55:32.420 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-09-19 05:50:37.078758917 +0000 2024-09-19T05:55:32.420 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-09-19 05:50:37.078758917 +0000 2024-09-19T05:55:32.420 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T05:55:32.488 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-09-19T05:55:32.488 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-09-19T05:55:32.488 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000128267 s, 4.0 MB/s 2024-09-19T05:55:32.489 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T05:55:32.548 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-09-19T05:55:32.605 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T05:55:32.605 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:55:32.605 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-19T05:55:32.605 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:55:32.605 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:55:32.605 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-09-19 05:55:21.479066574 +0000 2024-09-19T05:55:32.605 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-09-19 05:50:37.412749900 +0000 2024-09-19T05:55:32.606 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-09-19 05:50:37.412749900 +0000 2024-09-19T05:55:32.606 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-09-19 05:50:37.412749900 +0000 2024-09-19T05:55:32.606 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T05:55:32.670 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-09-19T05:55:32.670 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-09-19T05:55:32.670 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000110377 s, 4.6 MB/s 2024-09-19T05:55:32.671 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T05:55:32.728 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-09-19T05:55:32.786 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T05:55:32.786 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:55:32.786 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T05:55:32.786 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:55:32.786 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:55:32.786 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-09-19 05:55:21.479066574 +0000 2024-09-19T05:55:32.786 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-09-19 05:50:37.745740909 +0000 2024-09-19T05:55:32.786 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-09-19 05:50:37.745740909 +0000 2024-09-19T05:55:32.786 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-09-19 05:50:37.745740909 +0000 2024-09-19T05:55:32.787 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T05:55:32.851 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-09-19T05:55:32.851 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-09-19T05:55:32.851 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000108087 s, 4.7 MB/s 2024-09-19T05:55:32.852 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T05:55:32.909 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T05:55:32.910 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T05:55:32.929 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T05:55:32.929 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-09-19T05:55:32.987 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T05:55:32.988 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:55:32.988 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T05:55:32.988 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:55:32.988 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:55:32.988 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-19 05:55:12.358945135 +0000 2024-09-19T05:55:32.988 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-19 05:50:32.217026802 +0000 2024-09-19T05:55:32.988 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-19 05:50:32.217026802 +0000 2024-09-19T05:55:32.988 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-19 05:50:32.217026802 +0000 2024-09-19T05:55:32.988 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T05:55:33.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-19T05:55:33.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-19T05:55:33.056 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000129906 s, 3.9 MB/s 2024-09-19T05:55:33.058 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T05:55:33.117 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-09-19T05:55:33.174 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T05:55:33.174 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:55:33.175 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-19T05:55:33.175 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:55:33.175 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:55:33.175 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-19 05:55:12.359945117 +0000 2024-09-19T05:55:33.175 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-19 05:50:32.547021149 +0000 2024-09-19T05:55:33.175 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-19 05:50:32.547021149 +0000 2024-09-19T05:55:33.175 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-19 05:50:32.547021149 +0000 2024-09-19T05:55:33.175 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T05:55:33.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-19T05:55:33.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-19T05:55:33.240 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.00010206 s, 5.0 MB/s 2024-09-19T05:55:33.241 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T05:55:33.299 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-09-19T05:55:33.356 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T05:55:33.357 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:55:33.357 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-19T05:55:33.357 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:55:33.357 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:55:33.357 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-19 05:55:12.359945117 +0000 2024-09-19T05:55:33.357 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-19 05:50:32.858015820 +0000 2024-09-19T05:55:33.357 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-19 05:50:32.858015820 +0000 2024-09-19T05:55:33.357 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-19 05:50:32.858015820 +0000 2024-09-19T05:55:33.358 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T05:55:33.421 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-19T05:55:33.421 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-19T05:55:33.422 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000111474 s, 4.6 MB/s 2024-09-19T05:55:33.423 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T05:55:33.480 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-09-19T05:55:33.536 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T05:55:33.536 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:55:33.537 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-19T05:55:33.537 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:55:33.537 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:55:33.537 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-19 05:55:12.359945117 +0000 2024-09-19T05:55:33.537 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-19 05:50:33.203009909 +0000 2024-09-19T05:55:33.537 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-19 05:50:33.203009909 +0000 2024-09-19T05:55:33.537 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-19 05:50:33.203009909 +0000 2024-09-19T05:55:33.538 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T05:55:33.600 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-19T05:55:33.600 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-19T05:55:33.600 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000114217 s, 4.5 MB/s 2024-09-19T05:55:33.601 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T05:55:33.658 INFO:tasks.cephadm:Deploying osd.0 on smithi005 with /dev/vg_nvme/lv_4... 2024-09-19T05:55:33.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T05:55:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:33 smithi157 ceph-mon[35346]: Reconfiguring mon.b (monmap changed)... 2024-09-19T05:55:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:55:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:33 smithi157 ceph-mon[35346]: Reconfiguring daemon mon.b on smithi049 2024-09-19T05:55:33.842 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:33.842 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:33.842 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:33 smithi049 ceph-mon[34640]: Reconfiguring mon.b (monmap changed)... 2024-09-19T05:55:33.843 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:33.843 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:55:33.843 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:33.843 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:33 smithi049 ceph-mon[34640]: Reconfiguring daemon mon.b on smithi049 2024-09-19T05:55:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:33 smithi005 ceph-mon[30955]: Reconfiguring mon.b (monmap changed)... 2024-09-19T05:55:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:55:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:33 smithi005 ceph-mon[30955]: Reconfiguring daemon mon.b on smithi049 2024-09-19T05:55:33.924 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:55:34.544 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:34 smithi157 ceph-mon[35346]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:34.544 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:34.544 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:34.545 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:34.545 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:55:34.545 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:34 smithi005 ceph-mon[30955]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:55:34.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:34 smithi049 ceph-mon[34640]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:34.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:34.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:55:34.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:55:34.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:35.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:35 smithi005 ceph-mon[30955]: Reconfiguring mon.c (monmap changed)... 2024-09-19T05:55:35.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:35 smithi005 ceph-mon[30955]: Reconfiguring daemon mon.c on smithi157 2024-09-19T05:55:35.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:35.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:35.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:35.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:35.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:35.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:35.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:55:35.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T05:55:35.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:55:35.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:35.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:35 smithi157 ceph-mon[35346]: Reconfiguring mon.c (monmap changed)... 2024-09-19T05:55:35.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:35 smithi157 ceph-mon[35346]: Reconfiguring daemon mon.c on smithi157 2024-09-19T05:55:35.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:35.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:35.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:35.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:35.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:35.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:35.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:55:35.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T05:55:35.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:55:35.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:35.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:35 smithi049 ceph-mon[34640]: Reconfiguring mon.c (monmap changed)... 2024-09-19T05:55:35.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:35 smithi049 ceph-mon[34640]: Reconfiguring daemon mon.c on smithi157 2024-09-19T05:55:35.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:35.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:35.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:35.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:35.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:35.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:35.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:55:35.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T05:55:35.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:55:35.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:36.272 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:55:36.299 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_4 2024-09-19T05:55:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:55:36.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:36 smithi005 ceph-mon[30955]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:36.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:36 smithi005 ceph-mon[30955]: Deploying daemon mgr.z on smithi157 2024-09-19T05:55:36.720 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:36 smithi157 ceph-mon[35346]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:36.720 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:36 smithi157 ceph-mon[35346]: Deploying daemon mgr.z on smithi157 2024-09-19T05:55:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:36 smithi049 ceph-mon[34640]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:36 smithi049 ceph-mon[34640]: Deploying daemon mgr.z on smithi157 2024-09-19T05:55:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.964+0000 7f3559f0d640 1 -- 172.21.15.5:0/866164301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3554101c00 msgr2=0x7f3554102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.964+0000 7f3559f0d640 1 --2- 172.21.15.5:0/866164301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3554101c00 0x7f3554102000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3544009900 tx=0x7f354402a410 comp rx=0 tx=0).stop 2024-09-19T05:55:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.964+0000 7f3559f0d640 1 -- 172.21.15.5:0/866164301 shutdown_connections 2024-09-19T05:55:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.964+0000 7f3559f0d640 1 --2- 172.21.15.5:0/866164301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35541037c0 0x7f355410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.964+0000 7f3559f0d640 1 --2- 172.21.15.5:0/866164301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3554102e00 0x7f3554103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.964+0000 7f3559f0d640 1 --2- 172.21.15.5:0/866164301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3554101c00 0x7f3554102000 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.964+0000 7f3559f0d640 1 -- 172.21.15.5:0/866164301 >> 172.21.15.5:0/866164301 conn(0x7f35540fd390 msgr2=0x7f35540ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.964+0000 7f3559f0d640 1 -- 172.21.15.5:0/866164301 shutdown_connections 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.964+0000 7f3559f0d640 1 -- 172.21.15.5:0/866164301 wait complete. 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f3559f0d640 1 Processor -- start 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f3559f0d640 1 -- start start 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f3559f0d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3554101c00 0x7f3554197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f3559f0d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3554102e00 0x7f35541983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f3559f0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35541037c0 0x7f355419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f3559f0d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f355410d790 con 0x7f35541037c0 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f3559f0d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f355410d610 con 0x7f3554101c00 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f3559f0d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f355410d910 con 0x7f3554102e00 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f3553fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35541037c0 0x7f355419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f3553fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35541037c0 0x7f355419f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60802/0 (socket says 172.21.15.5:60802) 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f3553fff640 1 -- 172.21.15.5:0/53997216 learned_addr learned my addr 172.21.15.5:0/53997216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f354affd640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3554102e00 0x7f35541983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.965+0000 7f35537fe640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3554101c00 0x7f3554197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.966+0000 7f3553fff640 1 -- 172.21.15.5:0/53997216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3554102e00 msgr2=0x7f35541983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.966+0000 7f3553fff640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3554102e00 0x7f35541983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.966+0000 7f3553fff640 1 -- 172.21.15.5:0/53997216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3554101c00 msgr2=0x7f3554197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.966+0000 7f3553fff640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3554101c00 0x7f3554197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.966+0000 7f3553fff640 1 -- 172.21.15.5:0/53997216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355419fae0 con 0x7f35541037c0 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.966+0000 7f35537fe640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3554101c00 0x7f3554197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.966+0000 7f3553fff640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35541037c0 0x7f355419f470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f354c009fd0 tx=0x7f354c00def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.966+0000 7f35517fa640 1 -- 172.21.15.5:0/53997216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c017070 con 0x7f35541037c0 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.966+0000 7f35517fa640 1 -- 172.21.15.5:0/53997216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f354c003e20 con 0x7f35541037c0 2024-09-19T05:55:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.966+0000 7f3559f0d640 1 -- 172.21.15.5:0/53997216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355419fdd0 con 0x7f35541037c0 2024-09-19T05:55:36.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.966+0000 7f35517fa640 1 -- 172.21.15.5:0/53997216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c010460 con 0x7f35541037c0 2024-09-19T05:55:36.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.966+0000 7f3559f0d640 1 -- 172.21.15.5:0/53997216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35541a0280 con 0x7f35541037c0 2024-09-19T05:55:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.967+0000 7f35517fa640 1 -- 172.21.15.5:0/53997216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f354c010600 con 0x7f35541037c0 2024-09-19T05:55:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.967+0000 7f3559f0d640 1 -- 172.21.15.5:0/53997216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35541090a0 con 0x7f35541037c0 2024-09-19T05:55:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.967+0000 7f35517fa640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3540039500 0x7f354003b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:55:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.967+0000 7f35517fa640 1 -- 172.21.15.5:0/53997216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f354c04c4b0 con 0x7f35541037c0 2024-09-19T05:55:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.967+0000 7f35537fe640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3540039500 0x7f354003b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:55:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.968+0000 7f35537fe640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3540039500 0x7f354003b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3544000c00 tx=0x7f3544032d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:55:36.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:36.972+0000 7f35517fa640 1 -- 172.21.15.5:0/53997216 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f354c01a360 con 0x7f35541037c0 2024-09-19T05:55:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:37.159+0000 7f3559f0d640 1 -- 172.21.15.5:0/53997216 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3554062630 con 0x7f3540039500 2024-09-19T05:55:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:55:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:55:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:37.802 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:37.663+0000 7ff0285b5040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:55:37.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:55:37.817 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:55:37.817 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:55:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:55:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:38.166 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:37.810+0000 7ff0285b5040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:55:38.167 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:37.876+0000 7ff0285b5040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:55:38.167 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:38.023+0000 7ff0285b5040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:55:38.167 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:38.163+0000 7ff0285b5040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:55:38.801 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:38.482+0000 7ff0285b5040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:55:38.801 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:38.542+0000 7ff0285b5040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:55:38.801 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:38.727+0000 7ff0285b5040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:55:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-mon[35346]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-mon[35346]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:55:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:55:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T05:55:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:55:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:38 smithi157 ceph-mon[35346]: Deploying daemon mgr.y on smithi049 2024-09-19T05:55:38.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:38 smithi005 ceph-mon[30955]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:38.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:38 smithi005 ceph-mon[30955]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:55:38.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:38.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:38.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:38.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:55:38.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T05:55:38.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:55:38.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:38.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:38 smithi005 ceph-mon[30955]: Deploying daemon mgr.y on smithi049 2024-09-19T05:55:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:38 smithi049 ceph-mon[34640]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:38 smithi049 ceph-mon[34640]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:55:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:38.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:55:38.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T05:55:38.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:55:38.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:38.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:38 smithi049 ceph-mon[34640]: Deploying daemon mgr.y on smithi049 2024-09-19T05:55:39.604 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:39 smithi049 systemd[1]: Starting Ceph mgr.y for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T05:55:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:39 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3897541175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3c10d60-d354-407f-9cdf-099344599844"}]: dispatch 2024-09-19T05:55:39.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:39 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3897541175' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3c10d60-d354-407f-9cdf-099344599844"}]': finished 2024-09-19T05:55:39.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:39 smithi005 ceph-mon[30955]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T05:55:39.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:55:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:39 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3897541175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3c10d60-d354-407f-9cdf-099344599844"}]: dispatch 2024-09-19T05:55:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:39 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3897541175' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3c10d60-d354-407f-9cdf-099344599844"}]': finished 2024-09-19T05:55:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:39 smithi049 ceph-mon[34640]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T05:55:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:55:39.897 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:39 smithi049 podman[35984]: 2024-09-19 05:55:39.505164966 +0000 UTC m=+0.015957973 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:55:39.897 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:39 smithi049 podman[35984]: 2024-09-19 05:55:39.606954524 +0000 UTC m=+0.117747510 container create 9571f2bb9a57f07075d1e9cbdaa260b301bebef1fbed847a9583def821a381b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T05:55:39.897 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:39 smithi049 podman[35984]: 2024-09-19 05:55:39.817522312 +0000 UTC m=+0.328315310 container init 9571f2bb9a57f07075d1e9cbdaa260b301bebef1fbed847a9583def821a381b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:55:39.897 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:39 smithi049 podman[35984]: 2024-09-19 05:55:39.820441674 +0000 UTC m=+0.331234669 container start 9571f2bb9a57f07075d1e9cbdaa260b301bebef1fbed847a9583def821a381b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T05:55:39.897 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:39 smithi049 ceph-mgr[36003]: -- 172.21.15.49:0/409633675 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f613810e0 con 0x556f6135e800 2024-09-19T05:55:40.051 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:39 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:39.603+0000 7ff0285b5040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:55:40.051 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:39 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:39.671+0000 7ff0285b5040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:55:40.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:39 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3897541175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3c10d60-d354-407f-9cdf-099344599844"}]: dispatch 2024-09-19T05:55:40.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:39 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3897541175' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3c10d60-d354-407f-9cdf-099344599844"}]': finished 2024-09-19T05:55:40.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:39 smithi157 ceph-mon[35346]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T05:55:40.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:55:40.171 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:39 smithi049 bash[35984]: 9571f2bb9a57f07075d1e9cbdaa260b301bebef1fbed847a9583def821a381b6 2024-09-19T05:55:40.171 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:39 smithi049 systemd[1]: Started Ceph mgr.y for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T05:55:40.171 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:39 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:39.947+0000 7f9baa375040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:55:40.171 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:40.009+0000 7f9baa375040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:55:40.171 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:40.169+0000 7f9baa375040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:55:40.561 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:40.239+0000 7f9baa375040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:55:40.561 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:40.401+0000 7f9baa375040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:55:40.562 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:40.559+0000 7f9baa375040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:55:40.696 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:40.428+0000 7ff0285b5040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:55:40.696 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:40.550+0000 7ff0285b5040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:55:40.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-mon[35346]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:40.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:40.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:40.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:40.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:40.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:40.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2570039187' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:55:40.854 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-mon[34640]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:40.854 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:40.854 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:40.854 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:40.855 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:40.855 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:40.855 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2570039187' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:55:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:40 smithi005 ceph-mon[30955]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:40 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:40 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:40 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:40 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:40 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:40 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2570039187' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:55:40.988 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:40.697+0000 7ff0285b5040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:55:40.988 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:40.762+0000 7ff0285b5040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:55:40.988 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:40.916+0000 7ff0285b5040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:55:40.988 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:40 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:40.986+0000 7ff0285b5040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:55:41.146 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:40.901+0000 7f9baa375040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:55:41.146 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:40 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:40.965+0000 7f9baa375040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:55:41.286 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:41 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:41.048+0000 7ff0285b5040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:55:41.286 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:41 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:41.216+0000 7ff0285b5040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:55:41.286 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:41 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:41.284+0000 7ff0285b5040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:55:41.551 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:41 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:41.352+0000 7ff0285b5040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:55:41.551 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:41 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:41.433+0000 7ff0285b5040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:55:41.551 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:41 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:41.493+0000 7ff0285b5040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:55:41.646 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:41 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:41.166+0000 7f9baa375040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:55:42.051 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:41 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:41.553+0000 7ff0285b5040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:55:42.396 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:42.097+0000 7f9baa375040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:55:42.396 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:42.168+0000 7f9baa375040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:55:42.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-mon[34640]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.551 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T05:55:42.349+0000 7ff0285b5040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:55:42.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-mon[35346]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:42 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:42 smithi005 ceph-mon[30955]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:42.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:42.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:42.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:42.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:42.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:42.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:42.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:42 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:43.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:43.147+0000 7f35517fa640 1 -- 172.21.15.5:0/53997216 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 14) ==== 92407+0+0 (secure 0 0 0) 0x7f354c018d00 con 0x7f35541037c0 2024-09-19T05:55:43.201 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-mon[34640]: Standby manager daemon z started 2024-09-19T05:55:43.202 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.157:0/2643893628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T05:55:43.202 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.157:0/2643893628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:55:43.202 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.157:0/2643893628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T05:55:43.202 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.157:0/2643893628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:55:43.202 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:42 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:42.928+0000 7f9baa375040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:55:43.202 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:43.049+0000 7f9baa375040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:55:43.493 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:43.199+0000 7f9baa375040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:55:43.493 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:43.264+0000 7f9baa375040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:55:43.493 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:43.420+0000 7f9baa375040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:55:43.493 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:43.491+0000 7f9baa375040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:55:43.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:43 smithi157 ceph-mon[35346]: Standby manager daemon z started 2024-09-19T05:55:43.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:43 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.157:0/2643893628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T05:55:43.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:43 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.157:0/2643893628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:55:43.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:43 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.157:0/2643893628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T05:55:43.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:43 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.157:0/2643893628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:55:43.553 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:43 smithi005 ceph-mon[30955]: Standby manager daemon z started 2024-09-19T05:55:43.553 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:43 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.157:0/2643893628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T05:55:43.553 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:43 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.157:0/2643893628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:55:43.553 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:43 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.157:0/2643893628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T05:55:43.554 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:43 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.157:0/2643893628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:55:43.789 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:43.554+0000 7f9baa375040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:55:43.790 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:43.719+0000 7f9baa375040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:55:43.790 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:43.787+0000 7f9baa375040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:55:44.069 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:43.857+0000 7f9baa375040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:55:44.069 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:43.943+0000 7f9baa375040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:55:44.069 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:44 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:44.005+0000 7f9baa375040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:55:44.069 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:44 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:44.067+0000 7f9baa375040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:55:44.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:44 smithi049 ceph-mon[34640]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:44.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:44 smithi049 ceph-mon[34640]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-19T05:55:44.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:44 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T05:55:44.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:44 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:44.430 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:44 smithi005 ceph-mon[30955]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:44.430 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:44 smithi005 ceph-mon[30955]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-19T05:55:44.430 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:44 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T05:55:44.430 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:44 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:44.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:44 smithi157 ceph-mon[35346]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:44.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:44 smithi157 ceph-mon[35346]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-19T05:55:44.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:44 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T05:55:44.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:44 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:45.155 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 05:55:44 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T05:55:44.897+0000 7f9baa375040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:55:45.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:45.158+0000 7f35517fa640 1 -- 172.21.15.5:0/53997216 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 15) ==== 138298+0+0 (secure 0 0 0) 0x7f354c019c10 con 0x7f35541037c0 2024-09-19T05:55:45.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:45 smithi005 ceph-mon[30955]: Standby manager daemon y started 2024-09-19T05:55:45.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:45 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.49:0/1387769852' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T05:55:45.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:45 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.49:0/1387769852' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:55:45.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:45 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.49:0/1387769852' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T05:55:45.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:45 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.49:0/1387769852' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:55:45.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:45 smithi157 ceph-mon[35346]: Standby manager daemon y started 2024-09-19T05:55:45.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:45 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.49:0/1387769852' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T05:55:45.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:45 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.49:0/1387769852' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:55:45.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:45 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.49:0/1387769852' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T05:55:45.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:45 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.49:0/1387769852' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:55:45.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:45 smithi049 ceph-mon[34640]: Standby manager daemon y started 2024-09-19T05:55:45.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:45 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.49:0/1387769852' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T05:55:45.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:45 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.49:0/1387769852' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:55:45.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:45 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.49:0/1387769852' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T05:55:45.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:45 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.49:0/1387769852' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:55:46.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:46 smithi005 ceph-mon[30955]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:46.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:46 smithi005 ceph-mon[30955]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-19T05:55:46.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:46 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T05:55:46.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:46 smithi157 ceph-mon[35346]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:46.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:46 smithi157 ceph-mon[35346]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-19T05:55:46.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:46 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T05:55:46.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:46 smithi049 ceph-mon[34640]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:46.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:46 smithi049 ceph-mon[34640]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-19T05:55:46.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:46 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T05:55:48.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:48 smithi005 ceph-mon[30955]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:48.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T05:55:48.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:48.449 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:48 smithi049 ceph-mon[34640]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:48.449 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T05:55:48.449 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:48.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:48 smithi157 ceph-mon[35346]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:48.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T05:55:48.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:49.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:49 smithi005 ceph-mon[30955]: Deploying daemon osd.0 on smithi005 2024-09-19T05:55:49.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:49 smithi157 ceph-mon[35346]: Deploying daemon osd.0 on smithi005 2024-09-19T05:55:49.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:49 smithi049 ceph-mon[34640]: Deploying daemon osd.0 on smithi005 2024-09-19T05:55:50.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:50 smithi005 ceph-mon[30955]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:50.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:50 smithi157 ceph-mon[35346]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:50.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:50 smithi049 ceph-mon[34640]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:52.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:52 smithi157 ceph-mon[35346]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:52 smithi005 ceph-mon[30955]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:52.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:52 smithi049 ceph-mon[34640]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:54.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:54 smithi157 ceph-mon[35346]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:54.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:54 smithi005 ceph-mon[30955]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:54.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:54 smithi049 ceph-mon[34640]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:55.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:55 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:55.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:55 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:55.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:55 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:55.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:55 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:55.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:55 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:55.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:55 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:55.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:55 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:55.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:55 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:55.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:55 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:56.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:56 smithi157 ceph-mon[35346]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:56.577 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:56 smithi005 ceph-mon[30955]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:56.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:56 smithi049 ceph-mon[34640]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:58.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:57 smithi005 ceph-mon[30955]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:58.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:58.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:58.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:58.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:58.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:58.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:57 smithi157 ceph-mon[35346]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:58.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:58.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:58.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:58.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:58.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:58.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:57 smithi049 ceph-mon[34640]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:55:58.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:58.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:58.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:58.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:58.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:58.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.291+0000 7f35517fa640 1 -- 172.21.15.5:0/53997216 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3554062630 con 0x7f3540039500 2024-09-19T05:55:58.295 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 0 on host 'smithi005' 2024-09-19T05:55:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.294+0000 7f3559f0d640 1 -- 172.21.15.5:0/53997216 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3540039500 msgr2=0x7f354003b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.294+0000 7f3559f0d640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3540039500 0x7f354003b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3544000c00 tx=0x7f3544032d20 comp rx=0 tx=0).stop 2024-09-19T05:55:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.294+0000 7f3559f0d640 1 -- 172.21.15.5:0/53997216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35541037c0 msgr2=0x7f355419f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:55:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.294+0000 7f3559f0d640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35541037c0 0x7f355419f470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f354c009fd0 tx=0x7f354c00def0 comp rx=0 tx=0).stop 2024-09-19T05:55:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.294+0000 7f3559f0d640 1 -- 172.21.15.5:0/53997216 shutdown_connections 2024-09-19T05:55:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.294+0000 7f3559f0d640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3540039500 0x7f354003b9c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.294+0000 7f3559f0d640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35541037c0 0x7f355419f470 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.294+0000 7f3559f0d640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3554102e00 0x7f35541983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.294+0000 7f3559f0d640 1 --2- 172.21.15.5:0/53997216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3554101c00 0x7f3554197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:55:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.294+0000 7f3559f0d640 1 -- 172.21.15.5:0/53997216 >> 172.21.15.5:0/53997216 conn(0x7f35540fd390 msgr2=0x7f3554106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:55:58.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.294+0000 7f3559f0d640 1 -- 172.21.15.5:0/53997216 shutdown_connections 2024-09-19T05:55:58.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:55:58.294+0000 7f3559f0d640 1 -- 172.21.15.5:0/53997216 wait complete. 2024-09-19T05:55:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:55:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:58.920 DEBUG:teuthology.orchestra.run.smithi005:osd.0> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T05:55:58.923 INFO:tasks.cephadm:Deploying osd.1 on smithi049 with /dev/vg_nvme/lv_4... 2024-09-19T05:55:58.923 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T05:55:59.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:59.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:59.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:55:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:59.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:59.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:59.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:55:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:55:59.170 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:56:00.479 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:00 smithi005 ceph-mon[30955]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:56:00.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:00 smithi157 ceph-mon[35346]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:56:00.571 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:00 smithi049 ceph-mon[34640]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:56:00.880 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 05:56:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0[42192]: 2024-09-19T05:56:00.476+0000 7fd867de3640 -1 osd.0 0 log_to_monitors true 2024-09-19T05:56:01.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:01 smithi005 ceph-mon[30955]: from='osd.0 [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T05:56:01.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:01 smithi005 ceph-mon[30955]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T05:56:01.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:01 smithi157 ceph-mon[35346]: from='osd.0 [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T05:56:01.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:01 smithi157 ceph-mon[35346]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T05:56:01.553 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:01 smithi049 ceph-mon[34640]: from='osd.0 [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T05:56:01.553 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:01 smithi049 ceph-mon[34640]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T05:56:01.578 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:56:01.605 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_4 2024-09-19T05:56:01.867 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:56:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.354+0000 7f4fa218a640 1 -- 172.21.15.49:0/3147453740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f9c104f90 msgr2=0x7f4f9c107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.354+0000 7f4fa218a640 1 --2- 172.21.15.49:0/3147453740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f9c104f90 0x7f4f9c107380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009900 tx=0x7f4f8402a410 comp rx=0 tx=0).stop 2024-09-19T05:56:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.354+0000 7f4fa218a640 1 -- 172.21.15.49:0/3147453740 shutdown_connections 2024-09-19T05:56:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.354+0000 7f4fa218a640 1 --2- 172.21.15.49:0/3147453740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c1078c0 0x7f4f9c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.354+0000 7f4fa218a640 1 --2- 172.21.15.49:0/3147453740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f9c104f90 0x7f4f9c107380 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.354+0000 7f4fa218a640 1 --2- 172.21.15.49:0/3147453740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c102660 0x7f4f9c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.354+0000 7f4fa218a640 1 -- 172.21.15.49:0/3147453740 >> 172.21.15.49:0/3147453740 conn(0x7f4f9c0fc280 msgr2=0x7f4f9c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.354+0000 7f4fa218a640 1 -- 172.21.15.49:0/3147453740 shutdown_connections 2024-09-19T05:56:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.354+0000 7f4fa218a640 1 -- 172.21.15.49:0/3147453740 wait complete. 2024-09-19T05:56:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4fa218a640 1 Processor -- start 2024-09-19T05:56:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4fa218a640 1 -- start start 2024-09-19T05:56:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4fa218a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c102660 0x7f4f9c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4fa218a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c104f90 0x7f4f9c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4fa218a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f9c1078c0 0x7f4f9c19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4fa218a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c10d740 con 0x7f4f9c104f90 2024-09-19T05:56:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4fa218a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c10d5c0 con 0x7f4f9c1078c0 2024-09-19T05:56:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4fa218a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c10d8c0 con 0x7f4f9c102660 2024-09-19T05:56:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4f9bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f9c1078c0 0x7f4f9c19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4f9bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f9c1078c0 0x7f4f9c19f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54262/0 (socket says 172.21.15.49:54262) 2024-09-19T05:56:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4f9bfff640 1 -- 172.21.15.49:0/2088605045 learned_addr learned my addr 172.21.15.49:0/2088605045 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-19T05:56:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4f9affd640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c104f90 0x7f4f9c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.355+0000 7f4f9b7fe640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c102660 0x7f4f9c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.356+0000 7f4f9bfff640 1 -- 172.21.15.49:0/2088605045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c102660 msgr2=0x7f4f9c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.356+0000 7f4f9bfff640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c102660 0x7f4f9c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.356+0000 7f4f9bfff640 1 -- 172.21.15.49:0/2088605045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c104f90 msgr2=0x7f4f9c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.356+0000 7f4f9bfff640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c104f90 0x7f4f9c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.356+0000 7f4f9bfff640 1 -- 172.21.15.49:0/2088605045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9c19fab0 con 0x7f4f9c1078c0 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.356+0000 7f4f9bfff640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f9c1078c0 0x7f4f9c19f440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400eb10 tx=0x7f4f9400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.356+0000 7f4f98ff9640 1 -- 172.21.15.49:0/2088605045 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94004b20 con 0x7f4f9c1078c0 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.356+0000 7f4f98ff9640 1 -- 172.21.15.49:0/2088605045 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f94004cc0 con 0x7f4f9c1078c0 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.356+0000 7f4fa218a640 1 -- 172.21.15.49:0/2088605045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9c19fda0 con 0x7f4f9c1078c0 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.356+0000 7f4f98ff9640 1 -- 172.21.15.49:0/2088605045 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94010690 con 0x7f4f9c1078c0 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.356+0000 7f4fa218a640 1 -- 172.21.15.49:0/2088605045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f9c1a0250 con 0x7f4f9c1078c0 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.357+0000 7f4fa218a640 1 -- 172.21.15.49:0/2088605045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f9c0fdd70 con 0x7f4f9c1078c0 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.359+0000 7f4f98ff9640 1 -- 172.21.15.49:0/2088605045 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) ==== 138298+0+0 (secure 0 0 0) 0x7f4f94005000 con 0x7f4f9c1078c0 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.360+0000 7f4f98ff9640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4f880a3260 0x7f4f880a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.360+0000 7f4f98ff9640 1 -- 172.21.15.49:0/2088605045 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f4f940ce3f0 con 0x7f4f9c1078c0 2024-09-19T05:56:02.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.360+0000 7f4f9b7fe640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4f880a3260 0x7f4f880a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:02.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.360+0000 7f4f9b7fe640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4f880a3260 0x7f4f880a5720 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00bbe0 tx=0x7f4f8c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:02.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.363+0000 7f4f98ff9640 1 -- 172.21.15.49:0/2088605045 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f94054210 con 0x7f4f9c1078c0 2024-09-19T05:56:02.502 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0[42192]: 2024-09-19T05:56:02.304+0000 7fd863543640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T05:56:02.503 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0[42192]: 2024-09-19T05:56:02.307+0000 7fd85fb7d640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:56:02.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:56:02.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T05:56:02.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: from='osd.0 [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:56:02.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T05:56:02.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:56:02.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:56:02.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:02.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:02.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:56:02.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:02.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:02.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:02.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:02 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:02.548+0000 7f4fa218a640 1 -- 172.21.15.49:0/2088605045 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4f9c062630 con 0x7f4f880a3260 2024-09-19T05:56:02.552 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:56:02.552 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T05:56:02.552 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: from='osd.0 [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:56:02.552 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T05:56:02.552 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:56:02.552 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:56:02.552 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:02.552 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:02.552 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:56:02.552 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:02.552 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:02.553 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:02.553 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:02 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:56:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T05:56:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: from='osd.0 [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:56:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T05:56:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:56:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:56:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:56:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:02 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:03.566 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:03 smithi049 ceph-mon[34640]: Detected new or changed devices on smithi005 2024-09-19T05:56:03.567 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:03 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi005 to 17092M 2024-09-19T05:56:03.567 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:03 smithi049 ceph-mon[34640]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-19T05:56:03.567 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:03 smithi049 ceph-mon[34640]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T05:56:03.567 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:56:03.567 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:56:03.567 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:03 smithi049 ceph-mon[34640]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:56:03.567 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:56:03.567 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:56:03.567 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:03.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:03 smithi005 ceph-mon[30955]: Detected new or changed devices on smithi005 2024-09-19T05:56:03.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:03 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi005 to 17092M 2024-09-19T05:56:03.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:03 smithi005 ceph-mon[30955]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-19T05:56:03.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:03 smithi005 ceph-mon[30955]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T05:56:03.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:56:03.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:56:03.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:03 smithi005 ceph-mon[30955]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:56:03.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:56:03.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:56:03.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:03.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:03 smithi157 ceph-mon[35346]: Detected new or changed devices on smithi005 2024-09-19T05:56:03.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:03 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi005 to 17092M 2024-09-19T05:56:03.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:03 smithi157 ceph-mon[35346]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-19T05:56:03.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:03 smithi157 ceph-mon[35346]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T05:56:03.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:56:03.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:56:03.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:03 smithi157 ceph-mon[35346]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:56:03.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:56:03.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:56:03.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:04.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:04 smithi005 ceph-mon[30955]: purged_snaps scrub starts 2024-09-19T05:56:04.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:04 smithi005 ceph-mon[30955]: purged_snaps scrub ok 2024-09-19T05:56:04.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:04 smithi005 ceph-mon[30955]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:56:04.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:04 smithi005 ceph-mon[30955]: osd.0 [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810] boot 2024-09-19T05:56:04.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:04 smithi005 ceph-mon[30955]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T05:56:04.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:04 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:56:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:04 smithi049 ceph-mon[34640]: purged_snaps scrub starts 2024-09-19T05:56:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:04 smithi049 ceph-mon[34640]: purged_snaps scrub ok 2024-09-19T05:56:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:04 smithi049 ceph-mon[34640]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:56:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:04 smithi049 ceph-mon[34640]: osd.0 [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810] boot 2024-09-19T05:56:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:04 smithi049 ceph-mon[34640]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T05:56:04.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:04 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:56:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:04 smithi157 ceph-mon[35346]: purged_snaps scrub starts 2024-09-19T05:56:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:04 smithi157 ceph-mon[35346]: purged_snaps scrub ok 2024-09-19T05:56:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:04 smithi157 ceph-mon[35346]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:56:04.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:04 smithi157 ceph-mon[35346]: osd.0 [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810] boot 2024-09-19T05:56:04.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:04 smithi157 ceph-mon[35346]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T05:56:04.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:04 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:56:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:05 smithi005 ceph-mon[30955]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T05:56:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:05 smithi005 ceph-mon[30955]: from='client.? 172.21.15.49:0/689339040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f40f636-7761-45ac-87f3-910a2fa2feb2"}]: dispatch 2024-09-19T05:56:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:05 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f40f636-7761-45ac-87f3-910a2fa2feb2"}]: dispatch 2024-09-19T05:56:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:05 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f40f636-7761-45ac-87f3-910a2fa2feb2"}]': finished 2024-09-19T05:56:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:05 smithi005 ceph-mon[30955]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T05:56:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:05 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:05.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:05 smithi049 ceph-mon[34640]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T05:56:05.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:05 smithi049 ceph-mon[34640]: from='client.? 172.21.15.49:0/689339040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f40f636-7761-45ac-87f3-910a2fa2feb2"}]: dispatch 2024-09-19T05:56:05.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:05 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f40f636-7761-45ac-87f3-910a2fa2feb2"}]: dispatch 2024-09-19T05:56:05.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:05 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f40f636-7761-45ac-87f3-910a2fa2feb2"}]': finished 2024-09-19T05:56:05.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:05 smithi049 ceph-mon[34640]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T05:56:05.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:05 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:05 smithi157 ceph-mon[35346]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T05:56:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:05 smithi157 ceph-mon[35346]: from='client.? 172.21.15.49:0/689339040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f40f636-7761-45ac-87f3-910a2fa2feb2"}]: dispatch 2024-09-19T05:56:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:05 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f40f636-7761-45ac-87f3-910a2fa2feb2"}]: dispatch 2024-09-19T05:56:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:05 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f40f636-7761-45ac-87f3-910a2fa2feb2"}]': finished 2024-09-19T05:56:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:05 smithi157 ceph-mon[35346]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T05:56:05.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:05 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:06.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:06 smithi005 ceph-mon[30955]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:06.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:06 smithi005 ceph-mon[30955]: from='client.? 172.21.15.49:0/1038830138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:56:06.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:06 smithi049 ceph-mon[34640]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:06.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:06 smithi049 ceph-mon[34640]: from='client.? 172.21.15.49:0/1038830138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:56:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:06 smithi157 ceph-mon[35346]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:06 smithi157 ceph-mon[35346]: from='client.? 172.21.15.49:0/1038830138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:56:08.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:08 smithi005 ceph-mon[30955]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:08.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:08 smithi049 ceph-mon[34640]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:08 smithi157 ceph-mon[35346]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:10.575 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:10 smithi049 ceph-mon[34640]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:10.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:10 smithi005 ceph-mon[30955]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:10 smithi157 ceph-mon[35346]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:12.541 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:12 smithi049 ceph-mon[34640]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:12.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:12 smithi005 ceph-mon[30955]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:12 smithi157 ceph-mon[35346]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:14.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:14 smithi005 ceph-mon[30955]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T05:56:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:14 smithi005 ceph-mon[30955]: Deploying daemon osd.1 on smithi049 2024-09-19T05:56:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:14 smithi049 ceph-mon[34640]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T05:56:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:14 smithi049 ceph-mon[34640]: Deploying daemon osd.1 on smithi049 2024-09-19T05:56:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:14 smithi157 ceph-mon[35346]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T05:56:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:14 smithi157 ceph-mon[35346]: Deploying daemon osd.1 on smithi049 2024-09-19T05:56:16.578 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:16 smithi049 ceph-mon[34640]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:16 smithi005 ceph-mon[30955]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:16 smithi157 ceph-mon[35346]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:18.603 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:18 smithi157 ceph-mon[35346]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:18.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:18 smithi005 ceph-mon[30955]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:18.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:18 smithi049 ceph-mon[34640]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:20.592 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:20 smithi049 ceph-mon[34640]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:20.592 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:20 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:20.593 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:20 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:20.593 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:20 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:20 smithi005 ceph-mon[30955]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:20 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:20 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:20 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:20 smithi157 ceph-mon[35346]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:20 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:20 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:20.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:20 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:22.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:22 smithi049 ceph-mon[34640]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:22.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:22 smithi005 ceph-mon[30955]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:22 smithi157 ceph-mon[35346]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:23.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.448+0000 7f4f98ff9640 1 -- 172.21.15.49:0/2088605045 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4f9c062630 con 0x7f4f880a3260 2024-09-19T05:56:23.450 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 1 on host 'smithi049' 2024-09-19T05:56:23.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.450+0000 7f4fa218a640 1 -- 172.21.15.49:0/2088605045 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4f880a3260 msgr2=0x7f4f880a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:23.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.450+0000 7f4fa218a640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4f880a3260 0x7f4f880a5720 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00bbe0 tx=0x7f4f8c00b390 comp rx=0 tx=0).stop 2024-09-19T05:56:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.450+0000 7f4fa218a640 1 -- 172.21.15.49:0/2088605045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f9c1078c0 msgr2=0x7f4f9c19f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.450+0000 7f4fa218a640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f9c1078c0 0x7f4f9c19f440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400eb10 tx=0x7f4f9400c2d0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.450+0000 7f4fa218a640 1 -- 172.21.15.49:0/2088605045 shutdown_connections 2024-09-19T05:56:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.450+0000 7f4fa218a640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4f880a3260 0x7f4f880a5720 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.450+0000 7f4fa218a640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f9c1078c0 0x7f4f9c19f440 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.450+0000 7f4fa218a640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c104f90 0x7f4f9c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.450+0000 7f4fa218a640 1 --2- 172.21.15.49:0/2088605045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c102660 0x7f4f9c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.451+0000 7f4fa218a640 1 -- 172.21.15.49:0/2088605045 >> 172.21.15.49:0/2088605045 conn(0x7f4f9c0fc280 msgr2=0x7f4f9c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.451+0000 7f4fa218a640 1 -- 172.21.15.49:0/2088605045 shutdown_connections 2024-09-19T05:56:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:56:23.451+0000 7f4fa218a640 1 -- 172.21.15.49:0/2088605045 wait complete. 2024-09-19T05:56:23.754 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:23 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.755 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:23 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.755 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:23 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:23.755 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:23 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:23.755 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:23 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.755 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:23 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:23.755 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:23 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.755 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:23 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:23 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:23 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:23 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:23 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:23 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:23 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:23 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:23 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:23 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:23 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:23 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:23 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:23 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:23 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:23 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:23 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:24.035 DEBUG:teuthology.orchestra.run.smithi049:osd.1> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T05:56:24.037 INFO:tasks.cephadm:Deploying osd.2 on smithi157 with /dev/vg_nvme/lv_4... 2024-09-19T05:56:24.037 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T05:56:24.290 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:56:24.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:24 smithi049 ceph-mon[34640]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:24.727 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:24 smithi157 ceph-mon[35346]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:24 smithi005 ceph-mon[30955]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:26.396 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 05:56:26 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1[39135]: 2024-09-19T05:56:26.172+0000 7f53b6544640 -1 osd.1 0 log_to_monitors true 2024-09-19T05:56:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:26 smithi157 ceph-mon[35346]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:26 smithi157 ceph-mon[35346]: from='osd.1 [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T05:56:26.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:26 smithi157 ceph-mon[35346]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T05:56:26.733 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:56:26.758 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:26 smithi049 ceph-mon[34640]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:26.758 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:26 smithi049 ceph-mon[34640]: from='osd.1 [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T05:56:26.758 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:26 smithi049 ceph-mon[34640]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T05:56:26.759 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_4 2024-09-19T05:56:26.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:26 smithi005 ceph-mon[30955]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:26.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:26 smithi005 ceph-mon[30955]: from='osd.1 [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T05:56:26.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:26 smithi005 ceph-mon[30955]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T05:56:27.008 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:56:27.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.446+0000 7fddfd916640 1 -- 172.21.15.157:0/1769663694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf81026b0 msgr2=0x7fddf8108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:27.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.446+0000 7fddfd916640 1 --2- 172.21.15.157:0/1769663694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf81026b0 0x7fddf8108f40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009ca0 tx=0x7fddf002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:56:27.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.446+0000 7fddfd916640 1 -- 172.21.15.157:0/1769663694 shutdown_connections 2024-09-19T05:56:27.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.446+0000 7fddfd916640 1 --2- 172.21.15.157:0/1769663694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf81026b0 0x7fddf8108f40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:27.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.446+0000 7fddfd916640 1 --2- 172.21.15.157:0/1769663694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddf8101cf0 0x7fddf8102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:27.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.446+0000 7fddfd916640 1 --2- 172.21.15.157:0/1769663694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddf8100af0 0x7fddf8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:27.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.446+0000 7fddfd916640 1 -- 172.21.15.157:0/1769663694 >> 172.21.15.157:0/1769663694 conn(0x7fddf80fc2a0 msgr2=0x7fddf80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:27.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.446+0000 7fddfd916640 1 -- 172.21.15.157:0/1769663694 shutdown_connections 2024-09-19T05:56:27.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.446+0000 7fddfd916640 1 -- 172.21.15.157:0/1769663694 wait complete. 2024-09-19T05:56:27.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddfd916640 1 Processor -- start 2024-09-19T05:56:27.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddfd916640 1 -- start start 2024-09-19T05:56:27.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddfd916640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddf8100af0 0x7fddf8104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:27.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddfd916640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddf8101cf0 0x7fddf81030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:27.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddfd916640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf81026b0 0x7fddf8103610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:27.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddfd916640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fddf810c680 con 0x7fddf8100af0 2024-09-19T05:56:27.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddfd916640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fddf810c500 con 0x7fddf8101cf0 2024-09-19T05:56:27.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddfd916640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fddf810c800 con 0x7fddf81026b0 2024-09-19T05:56:27.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddf77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf81026b0 0x7fddf8103610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:27.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddf77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf81026b0 0x7fddf8103610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53976/0 (socket says 172.21.15.157:53976) 2024-09-19T05:56:27.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddf77fe640 1 -- 172.21.15.157:0/3167581971 learned_addr learned my addr 172.21.15.157:0/3167581971 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T05:56:27.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddf6ffd640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddf8100af0 0x7fddf8104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:27.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.447+0000 7fddeffff640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddf8101cf0 0x7fddf81030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:27.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.448+0000 7fddf77fe640 1 -- 172.21.15.157:0/3167581971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddf8101cf0 msgr2=0x7fddf81030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:27.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.448+0000 7fddf77fe640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddf8101cf0 0x7fddf81030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:27.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.448+0000 7fddf77fe640 1 -- 172.21.15.157:0/3167581971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddf8100af0 msgr2=0x7fddf8104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:27.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.448+0000 7fddf77fe640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddf8100af0 0x7fddf8104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:27.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.448+0000 7fddf77fe640 1 -- 172.21.15.157:0/3167581971 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf8103ed0 con 0x7fddf81026b0 2024-09-19T05:56:27.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.448+0000 7fddeffff640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddf8101cf0 0x7fddf81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:56:27.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.448+0000 7fddf77fe640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf81026b0 0x7fddf8103610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009dc0 tx=0x7fddf0003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:27.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.448+0000 7fddfc914640 1 -- 172.21.15.157:0/3167581971 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0004040 con 0x7fddf81026b0 2024-09-19T05:56:27.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.448+0000 7fddfc914640 1 -- 172.21.15.157:0/3167581971 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddf0004690 con 0x7fddf81026b0 2024-09-19T05:56:27.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.448+0000 7fddfd916640 1 -- 172.21.15.157:0/3167581971 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf81a1fc0 con 0x7fddf81026b0 2024-09-19T05:56:27.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.448+0000 7fddfc914640 1 -- 172.21.15.157:0/3167581971 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf002f4c0 con 0x7fddf81026b0 2024-09-19T05:56:27.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.448+0000 7fddfd916640 1 -- 172.21.15.157:0/3167581971 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf81a24c0 con 0x7fddf81026b0 2024-09-19T05:56:27.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.449+0000 7fddfd916640 1 -- 172.21.15.157:0/3167581971 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf81a26f0 con 0x7fddf81026b0 2024-09-19T05:56:27.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.450+0000 7fddfc914640 1 -- 172.21.15.157:0/3167581971 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) ==== 138298+0+0 (secure 0 0 0) 0x7fddf0030650 con 0x7fddf81026b0 2024-09-19T05:56:27.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.451+0000 7fddfc914640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdde40a3230 0x7fdde40a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:27.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.451+0000 7fddfc914640 1 -- 172.21.15.157:0/3167581971 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7fddf00ef650 con 0x7fddf81026b0 2024-09-19T05:56:27.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.452+0000 7fddf6ffd640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdde40a3230 0x7fdde40a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:27.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.452+0000 7fddf6ffd640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdde40a3230 0x7fdde40a56f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fddf8101b50 tx=0x7fdde800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:27.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.456+0000 7fddfc914640 1 -- 172.21.15.157:0/3167581971 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fddf00bdd60 con 0x7fddf81026b0 2024-09-19T05:56:27.638 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:27.635+0000 7fddfd916640 1 -- 172.21.15.157:0/3167581971 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fddf806aef0 con 0x7fdde40a3230 2024-09-19T05:56:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:27 smithi157 ceph-mon[35346]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T05:56:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:27 smithi157 ceph-mon[35346]: from='osd.1 [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:56:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:27 smithi157 ceph-mon[35346]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T05:56:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:27 smithi157 ceph-mon[35346]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:56:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:56:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:27 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:27 smithi005 ceph-mon[30955]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T05:56:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:27 smithi005 ceph-mon[30955]: from='osd.1 [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:56:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:27 smithi005 ceph-mon[30955]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T05:56:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:27 smithi005 ceph-mon[30955]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:56:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:56:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:27 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:27 smithi049 ceph-mon[34640]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T05:56:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:27 smithi049 ceph-mon[34640]: from='osd.1 [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:56:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:27 smithi049 ceph-mon[34640]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T05:56:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:27 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:27 smithi049 ceph-mon[34640]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:56:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:27 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:27 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:27 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:56:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:27 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:27 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:27 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:27 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:28.497 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1[39135]: 2024-09-19T05:56:28.173+0000 7f53b1ca4640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T05:56:28.497 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1[39135]: 2024-09-19T05:56:28.177+0000 7f53aa2d6640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:56:28.722 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:28 smithi157 ceph-mon[35346]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:28.722 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:28 smithi157 ceph-mon[35346]: Detected new or changed devices on smithi049 2024-09-19T05:56:28.722 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:28 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi049 to 17088M 2024-09-19T05:56:28.722 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:28 smithi157 ceph-mon[35346]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-19T05:56:28.722 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:28 smithi157 ceph-mon[35346]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T05:56:28.722 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:28 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:28.722 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:28 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:28.722 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:28 smithi157 ceph-mon[35346]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:56:28.722 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:28 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:56:28.722 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:28 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:56:28.723 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:28 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:28.723 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:28 smithi157 ceph-mon[35346]: from='osd.1 ' entity='osd.1' 2024-09-19T05:56:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:28 smithi005 ceph-mon[30955]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:28 smithi005 ceph-mon[30955]: Detected new or changed devices on smithi049 2024-09-19T05:56:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:28 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi049 to 17088M 2024-09-19T05:56:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:28 smithi005 ceph-mon[30955]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-19T05:56:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:28 smithi005 ceph-mon[30955]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T05:56:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:28 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:28 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:28 smithi005 ceph-mon[30955]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:56:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:28 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:56:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:28 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:56:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:28 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:28 smithi005 ceph-mon[30955]: from='osd.1 ' entity='osd.1' 2024-09-19T05:56:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-mon[34640]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-mon[34640]: Detected new or changed devices on smithi049 2024-09-19T05:56:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi049 to 17088M 2024-09-19T05:56:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-mon[34640]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-19T05:56:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-mon[34640]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T05:56:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-mon[34640]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:56:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:56:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:56:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:28 smithi049 ceph-mon[34640]: from='osd.1 ' entity='osd.1' 2024-09-19T05:56:29.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:29 smithi049 ceph-mon[34640]: purged_snaps scrub starts 2024-09-19T05:56:29.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:29 smithi049 ceph-mon[34640]: purged_snaps scrub ok 2024-09-19T05:56:29.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:29.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:29 smithi049 ceph-mon[34640]: osd.1 [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014] boot 2024-09-19T05:56:29.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:29 smithi049 ceph-mon[34640]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T05:56:29.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:29.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:29 smithi157 ceph-mon[35346]: purged_snaps scrub starts 2024-09-19T05:56:29.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:29 smithi157 ceph-mon[35346]: purged_snaps scrub ok 2024-09-19T05:56:29.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:29.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:29 smithi157 ceph-mon[35346]: osd.1 [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014] boot 2024-09-19T05:56:29.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:29 smithi157 ceph-mon[35346]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T05:56:29.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:29 smithi005 ceph-mon[30955]: purged_snaps scrub starts 2024-09-19T05:56:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:29 smithi005 ceph-mon[30955]: purged_snaps scrub ok 2024-09-19T05:56:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:29 smithi005 ceph-mon[30955]: osd.1 [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014] boot 2024-09-19T05:56:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:29 smithi005 ceph-mon[30955]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T05:56:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:56:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:30 smithi157 ceph-mon[35346]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:30 smithi157 ceph-mon[35346]: from='client.? 172.21.15.157:0/2509237603' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad00940c-4fdd-4f5f-9eaa-b1f38bcac526"}]: dispatch 2024-09-19T05:56:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:30 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad00940c-4fdd-4f5f-9eaa-b1f38bcac526"}]: dispatch 2024-09-19T05:56:30.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:30 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad00940c-4fdd-4f5f-9eaa-b1f38bcac526"}]': finished 2024-09-19T05:56:30.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:30 smithi157 ceph-mon[35346]: osdmap e14: 3 total, 2 up, 3 in 2024-09-19T05:56:30.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:30 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:30.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:30 smithi157 ceph-mon[35346]: from='client.? 172.21.15.157:0/2023149354' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:56:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:30 smithi005 ceph-mon[30955]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:30 smithi005 ceph-mon[30955]: from='client.? 172.21.15.157:0/2509237603' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad00940c-4fdd-4f5f-9eaa-b1f38bcac526"}]: dispatch 2024-09-19T05:56:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:30 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad00940c-4fdd-4f5f-9eaa-b1f38bcac526"}]: dispatch 2024-09-19T05:56:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:30 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad00940c-4fdd-4f5f-9eaa-b1f38bcac526"}]': finished 2024-09-19T05:56:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:30 smithi005 ceph-mon[30955]: osdmap e14: 3 total, 2 up, 3 in 2024-09-19T05:56:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:30 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:30 smithi005 ceph-mon[30955]: from='client.? 172.21.15.157:0/2023149354' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:56:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:30 smithi049 ceph-mon[34640]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:56:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:30 smithi049 ceph-mon[34640]: from='client.? 172.21.15.157:0/2509237603' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad00940c-4fdd-4f5f-9eaa-b1f38bcac526"}]: dispatch 2024-09-19T05:56:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:30 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad00940c-4fdd-4f5f-9eaa-b1f38bcac526"}]: dispatch 2024-09-19T05:56:30.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:30 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad00940c-4fdd-4f5f-9eaa-b1f38bcac526"}]': finished 2024-09-19T05:56:30.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:30 smithi049 ceph-mon[34640]: osdmap e14: 3 total, 2 up, 3 in 2024-09-19T05:56:30.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:30 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:30.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:30 smithi049 ceph-mon[34640]: from='client.? 172.21.15.157:0/2023149354' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:56:32.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:31 smithi157 ceph-mon[35346]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:32.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:31 smithi157 ceph-mon[35346]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T05:56:32.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:31 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:31 smithi005 ceph-mon[30955]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:31 smithi005 ceph-mon[30955]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T05:56:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:31 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:32.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:31 smithi049 ceph-mon[34640]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:32.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:31 smithi049 ceph-mon[34640]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T05:56:32.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:31 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:33 smithi157 ceph-mon[35346]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:34.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:33 smithi005 ceph-mon[30955]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:34.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:33 smithi049 ceph-mon[34640]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:36.196 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:35 smithi157 ceph-mon[35346]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:36.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:35 smithi005 ceph-mon[30955]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:36.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:35 smithi049 ceph-mon[34640]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:38.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:37 smithi157 ceph-mon[35346]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:38.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:37 smithi005 ceph-mon[30955]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:38.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:37 smithi049 ceph-mon[34640]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:39.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T05:56:39.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:39.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:38 smithi157 ceph-mon[35346]: Deploying daemon osd.2 on smithi157 2024-09-19T05:56:39.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T05:56:39.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:39.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:38 smithi005 ceph-mon[30955]: Deploying daemon osd.2 on smithi157 2024-09-19T05:56:39.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T05:56:39.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:39.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:38 smithi049 ceph-mon[34640]: Deploying daemon osd.2 on smithi157 2024-09-19T05:56:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:39 smithi157 ceph-mon[35346]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:40.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:39 smithi005 ceph-mon[30955]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:40.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:39 smithi049 ceph-mon[34640]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:42.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:41 smithi157 ceph-mon[35346]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:42.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:41 smithi005 ceph-mon[30955]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:42.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:41 smithi049 ceph-mon[34640]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:44.147 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:43 smithi157 ceph-mon[35346]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:44.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:43 smithi005 ceph-mon[30955]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:44.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:43 smithi049 ceph-mon[34640]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:44.978 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:44 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:44.978 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:44 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:44.978 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:44 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:45.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:44 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:45.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:44 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:45.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:44 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:44 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:44 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:44 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:46.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:46 smithi157 ceph-mon[35346]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:46.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:46 smithi005 ceph-mon[30955]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:46.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:46 smithi049 ceph-mon[34640]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:47.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:47.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:47.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:47.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:47.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:47.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:47.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:47.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:47.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:47.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:48.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:48.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:48.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:48.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:48.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:48.383 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 2 on host 'smithi157' 2024-09-19T05:56:48.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.381+0000 7fddfc914640 1 -- 172.21.15.157:0/3167581971 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fddf806aef0 con 0x7fdde40a3230 2024-09-19T05:56:48.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.384+0000 7fddfd916640 1 -- 172.21.15.157:0/3167581971 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdde40a3230 msgr2=0x7fdde40a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:48.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.384+0000 7fddfd916640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdde40a3230 0x7fdde40a56f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fddf8101b50 tx=0x7fdde800a2d0 comp rx=0 tx=0).stop 2024-09-19T05:56:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.384+0000 7fddfd916640 1 -- 172.21.15.157:0/3167581971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf81026b0 msgr2=0x7fddf8103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.384+0000 7fddfd916640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf81026b0 0x7fddf8103610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009dc0 tx=0x7fddf0003ba0 comp rx=0 tx=0).stop 2024-09-19T05:56:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.384+0000 7fddfd916640 1 -- 172.21.15.157:0/3167581971 shutdown_connections 2024-09-19T05:56:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.384+0000 7fddfd916640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdde40a3230 0x7fdde40a56f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.384+0000 7fddfd916640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf81026b0 0x7fddf8103610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:48.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.384+0000 7fddfd916640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddf8101cf0 0x7fddf81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:48.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.384+0000 7fddfd916640 1 --2- 172.21.15.157:0/3167581971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddf8100af0 0x7fddf8104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:48.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.384+0000 7fddfd916640 1 -- 172.21.15.157:0/3167581971 >> 172.21.15.157:0/3167581971 conn(0x7fddf80fc2a0 msgr2=0x7fddf806a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:48.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.384+0000 7fddfd916640 1 -- 172.21.15.157:0/3167581971 shutdown_connections 2024-09-19T05:56:48.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:56:48.384+0000 7fddfd916640 1 -- 172.21.15.157:0/3167581971 wait complete. 2024-09-19T05:56:48.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:48 smithi157 ceph-mon[35346]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:48.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:48.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:48.671 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:48.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:48 smithi005 ceph-mon[30955]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:48.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:48.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:48.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:48 smithi049 ceph-mon[34640]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:48.948 DEBUG:teuthology.orchestra.run.smithi157:osd.2> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T05:56:48.950 INFO:tasks.cephadm:Deploying osd.3 on smithi005 with /dev/vg_nvme/lv_3... 2024-09-19T05:56:48.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T05:56:49.218 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:56:50.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:50 smithi005 ceph-mon[30955]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:50.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:50 smithi005 ceph-mon[30955]: from='osd.2 [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T05:56:50.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:50 smithi005 ceph-mon[30955]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T05:56:50.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:50 smithi157 ceph-mon[35346]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:50.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:50 smithi157 ceph-mon[35346]: from='osd.2 [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T05:56:50.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:50 smithi157 ceph-mon[35346]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T05:56:50.802 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 05:56:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2[40332]: 2024-09-19T05:56:50.560+0000 7f00c26c8640 -1 osd.2 0 log_to_monitors true 2024-09-19T05:56:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:50 smithi049 ceph-mon[34640]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:50 smithi049 ceph-mon[34640]: from='osd.2 [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T05:56:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:50 smithi049 ceph-mon[34640]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T05:56:51.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:51 smithi157 ceph-mon[35346]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T05:56:51.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:51 smithi157 ceph-mon[35346]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T05:56:51.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:51 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:51.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:51 smithi157 ceph-mon[35346]: from='osd.2 [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:56:51.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:51 smithi157 ceph-mon[35346]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:56:51.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:51 smithi005 ceph-mon[30955]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T05:56:51.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:51 smithi005 ceph-mon[30955]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T05:56:51.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:51 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:51.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:51 smithi005 ceph-mon[30955]: from='osd.2 [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:56:51.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:51 smithi005 ceph-mon[30955]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:56:51.829 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:56:51.857 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_3 2024-09-19T05:56:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:51 smithi049 ceph-mon[34640]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T05:56:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:51 smithi049 ceph-mon[34640]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T05:56:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:51 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:51 smithi049 ceph-mon[34640]: from='osd.2 [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:56:51.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:51 smithi049 ceph-mon[34640]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:56:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:56:52.551 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2[40332]: 2024-09-19T05:56:52.229+0000 7f00bde28640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T05:56:52.552 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2[40332]: 2024-09-19T05:56:52.233+0000 7f00b645a640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:56:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.547+0000 7f3eab512640 1 -- 172.21.15.5:0/3542466614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4101cd0 msgr2=0x7f3ea4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.547+0000 7f3eab512640 1 --2- 172.21.15.5:0/3542466614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4101cd0 0x7f3ea4102150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009900 tx=0x7f3e9402a410 comp rx=0 tx=0).stop 2024-09-19T05:56:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.547+0000 7f3eab512640 1 -- 172.21.15.5:0/3542466614 shutdown_connections 2024-09-19T05:56:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.547+0000 7f3eab512640 1 --2- 172.21.15.5:0/3542466614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea4102690 0x7f3ea4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.547+0000 7f3eab512640 1 --2- 172.21.15.5:0/3542466614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4101cd0 0x7f3ea4102150 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.547+0000 7f3eab512640 1 --2- 172.21.15.5:0/3542466614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea4100ad0 0x7f3ea4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.547+0000 7f3eab512640 1 -- 172.21.15.5:0/3542466614 >> 172.21.15.5:0/3542466614 conn(0x7f3ea40fc280 msgr2=0x7f3ea40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.547+0000 7f3eab512640 1 -- 172.21.15.5:0/3542466614 shutdown_connections 2024-09-19T05:56:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.547+0000 7f3eab512640 1 -- 172.21.15.5:0/3542466614 wait complete. 2024-09-19T05:56:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.548+0000 7f3eab512640 1 Processor -- start 2024-09-19T05:56:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.548+0000 7f3eab512640 1 -- start start 2024-09-19T05:56:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.548+0000 7f3eab512640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea4100ad0 0x7f3ea4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.548+0000 7f3eab512640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea4101cd0 0x7f3ea4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.548+0000 7f3eab512640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102690 0x7f3ea419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.548+0000 7f3eab512640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea410c6d0 con 0x7f3ea4102690 2024-09-19T05:56:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.548+0000 7f3eab512640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea410c550 con 0x7f3ea4100ad0 2024-09-19T05:56:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.548+0000 7f3eab512640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea410c850 con 0x7f3ea4101cd0 2024-09-19T05:56:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.548+0000 7f3ea9aaf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102690 0x7f3ea419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.548+0000 7f3ea9aaf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102690 0x7f3ea419f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50192/0 (socket says 172.21.15.5:50192) 2024-09-19T05:56:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.548+0000 7f3ea9aaf640 1 -- 172.21.15.5:0/4181969174 learned_addr learned my addr 172.21.15.5:0/4181969174 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:56:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3ea8aad640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea4101cd0 0x7f3ea4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3ea92ae640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea4100ad0 0x7f3ea4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3ea9aaf640 1 -- 172.21.15.5:0/4181969174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea4101cd0 msgr2=0x7f3ea4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3ea9aaf640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea4101cd0 0x7f3ea4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3ea9aaf640 1 -- 172.21.15.5:0/4181969174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea4100ad0 msgr2=0x7f3ea4197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3ea9aaf640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea4100ad0 0x7f3ea4197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3ea9aaf640 1 -- 172.21.15.5:0/4181969174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea419fb60 con 0x7f3ea4102690 2024-09-19T05:56:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3ea9aaf640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102690 0x7f3ea419f4f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009fd0 tx=0x7f3e9c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3e9a7fc640 1 -- 172.21.15.5:0/4181969174 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c017070 con 0x7f3ea4102690 2024-09-19T05:56:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3e9a7fc640 1 -- 172.21.15.5:0/4181969174 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e9c003e20 con 0x7f3ea4102690 2024-09-19T05:56:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3eab512640 1 -- 172.21.15.5:0/4181969174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea419fe50 con 0x7f3ea4102690 2024-09-19T05:56:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3e9a7fc640 1 -- 172.21.15.5:0/4181969174 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c010460 con 0x7f3ea4102690 2024-09-19T05:56:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.549+0000 7f3eab512640 1 -- 172.21.15.5:0/4181969174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea41a0300 con 0x7f3ea4102690 2024-09-19T05:56:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.550+0000 7f3eab512640 1 -- 172.21.15.5:0/4181969174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea41a0530 con 0x7f3ea4102690 2024-09-19T05:56:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.552+0000 7f3e9a7fc640 1 -- 172.21.15.5:0/4181969174 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 15) ==== 138298+0+0 (secure 0 0 0) 0x7f3e9c003970 con 0x7f3ea4102690 2024-09-19T05:56:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.552+0000 7f3e9a7fc640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3e900a3260 0x7f3e900a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.552+0000 7f3ea92ae640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3e900a3260 0x7f3e900a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.552+0000 7f3e9a7fc640 1 -- 172.21.15.5:0/4181969174 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f3e9c0ce050 con 0x7f3ea4102690 2024-09-19T05:56:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.553+0000 7f3ea92ae640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3e900a3260 0x7f3e900a5720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4101b30 tx=0x7f3ea0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.557+0000 7f3e9a7fc640 1 -- 172.21.15.5:0/4181969174 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3e9c09c4c0 con 0x7f3ea4102690 2024-09-19T05:56:52.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:52.745+0000 7f3eab512640 1 -- 172.21.15.5:0/4181969174 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3ea406a680 con 0x7f3e900a3260 2024-09-19T05:56:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T05:56:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T05:56:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:56:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:52.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:52.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:52 smithi005 ceph-mon[30955]: from='osd.2 ' entity='osd.2' 2024-09-19T05:56:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T05:56:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T05:56:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:56:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:52 smithi049 ceph-mon[34640]: from='osd.2 ' entity='osd.2' 2024-09-19T05:56:53.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:53.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T05:56:53.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T05:56:53.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:53.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:53.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:53.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:53.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:56:53.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:53.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:53.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:53.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:53.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:52 smithi157 ceph-mon[35346]: from='osd.2 ' entity='osd.2' 2024-09-19T05:56:53.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:53 smithi005 ceph-mon[30955]: purged_snaps scrub starts 2024-09-19T05:56:53.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:53 smithi005 ceph-mon[30955]: purged_snaps scrub ok 2024-09-19T05:56:53.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:53 smithi005 ceph-mon[30955]: Detected new or changed devices on smithi157 2024-09-19T05:56:53.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:53 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi157 to 17088M 2024-09-19T05:56:53.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:53.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:56:53.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:56:53.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:53.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:53 smithi005 ceph-mon[30955]: osd.2 [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622] boot 2024-09-19T05:56:53.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:53 smithi005 ceph-mon[30955]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T05:56:53.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:53.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:53 smithi049 ceph-mon[34640]: purged_snaps scrub starts 2024-09-19T05:56:53.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:53 smithi049 ceph-mon[34640]: purged_snaps scrub ok 2024-09-19T05:56:53.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:53 smithi049 ceph-mon[34640]: Detected new or changed devices on smithi157 2024-09-19T05:56:53.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:53 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi157 to 17088M 2024-09-19T05:56:53.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:53.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:56:53.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:56:53.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:53.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:53 smithi049 ceph-mon[34640]: osd.2 [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622] boot 2024-09-19T05:56:53.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:53 smithi049 ceph-mon[34640]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T05:56:53.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:54.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:53 smithi157 ceph-mon[35346]: purged_snaps scrub starts 2024-09-19T05:56:54.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:53 smithi157 ceph-mon[35346]: purged_snaps scrub ok 2024-09-19T05:56:54.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:53 smithi157 ceph-mon[35346]: Detected new or changed devices on smithi157 2024-09-19T05:56:54.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:53 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi157 to 17088M 2024-09-19T05:56:54.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:54.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:56:54.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:56:54.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:54.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:53 smithi157 ceph-mon[35346]: osd.2 [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622] boot 2024-09-19T05:56:54.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:53 smithi157 ceph-mon[35346]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T05:56:54.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:56:54.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:54 smithi005 ceph-mon[30955]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:54.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:54 smithi005 ceph-mon[30955]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:56:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:54 smithi049 ceph-mon[34640]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:54 smithi049 ceph-mon[34640]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:56:55.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:54 smithi157 ceph-mon[35346]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:56:55.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:54 smithi157 ceph-mon[35346]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:56:55.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:55 smithi005 ceph-mon[30955]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T05:56:55.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:55 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:56:55.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:55 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/4237072626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63fec329-1b62-4c5e-b8ca-084a84d4d762"}]: dispatch 2024-09-19T05:56:55.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:55 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T05:56:55.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:55 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/4237072626' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63fec329-1b62-4c5e-b8ca-084a84d4d762"}]': finished 2024-09-19T05:56:55.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:55 smithi005 ceph-mon[30955]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T05:56:55.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:55 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:56:55.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:55 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:56:55.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:55 smithi049 ceph-mon[34640]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T05:56:55.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:55 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:56:55.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:55 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/4237072626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63fec329-1b62-4c5e-b8ca-084a84d4d762"}]: dispatch 2024-09-19T05:56:55.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:55 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T05:56:55.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:55 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/4237072626' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63fec329-1b62-4c5e-b8ca-084a84d4d762"}]': finished 2024-09-19T05:56:55.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:55 smithi049 ceph-mon[34640]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T05:56:55.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:55 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:56:55.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:55 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:56:56.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:55 smithi157 ceph-mon[35346]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T05:56:56.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:55 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:56:56.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:55 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/4237072626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63fec329-1b62-4c5e-b8ca-084a84d4d762"}]: dispatch 2024-09-19T05:56:56.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:55 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T05:56:56.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:55 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/4237072626' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63fec329-1b62-4c5e-b8ca-084a84d4d762"}]': finished 2024-09-19T05:56:56.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:55 smithi157 ceph-mon[35346]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T05:56:56.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:55 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:56:56.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:55 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:56:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:56 smithi005 ceph-mon[30955]: pgmap v77: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:56:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:56 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1085265357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:56:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:56 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:56:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:56 smithi005 ceph-mon[30955]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T05:56:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:56 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:56:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:56 smithi049 ceph-mon[34640]: pgmap v77: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:56:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:56 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1085265357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:56:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:56 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:56:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:56 smithi049 ceph-mon[34640]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T05:56:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:56 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:56:57.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:56 smithi157 ceph-mon[35346]: pgmap v77: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:56:57.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:56 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1085265357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:56:57.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:56 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:56:57.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:56 smithi157 ceph-mon[35346]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T05:56:57.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:56 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:56:57.379 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 05:56:56 smithi005 sudo[46685]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T05:56:57.379 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 05:56:56 smithi005 sudo[46685]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:56:57.379 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 05:56:56 smithi005 sudo[46685]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:56:57.379 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 05:56:56 smithi005 sudo[46685]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:56:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:56:57.619+0000 7f3e9a7fc640 1 -- 172.21.15.5:0/4181969174 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f3e9c098430 con 0x7f3ea4102690 2024-09-19T05:56:57.879 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 05:56:57 smithi005 sudo[47057]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T05:56:57.879 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 05:56:57 smithi005 sudo[47057]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:56:57.879 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 05:56:57 smithi005 sudo[47057]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:56:57.879 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 05:56:57 smithi005 sudo[47057]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:56:58.146 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 05:56:57 smithi049 sudo[43277]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T05:56:58.146 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 05:56:57 smithi049 sudo[43277]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:56:58.146 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 05:56:57 smithi049 sudo[43277]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:56:58.146 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 05:56:57 smithi049 sudo[43277]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:56:58.646 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 05:56:58 smithi049 sudo[43280]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T05:56:58.647 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 05:56:58 smithi049 sudo[43280]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:56:58.647 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 05:56:58 smithi049 sudo[43280]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:56:58.647 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 05:56:58 smithi049 sudo[43280]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:56:58.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:58 smithi049 ceph-mon[34640]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:56:58.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:58 smithi049 ceph-mon[34640]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T05:56:58.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:56:58.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:58.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:58 smithi049 ceph-mon[34640]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T05:56:58.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:56:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:58.801 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 05:56:58 smithi157 sudo[44409]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T05:56:58.801 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 05:56:58 smithi157 sudo[44409]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:56:58.801 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 05:56:58 smithi157 sudo[44409]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:56:58.801 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 05:56:58 smithi157 sudo[44409]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:56:58.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:58 smithi157 ceph-mon[35346]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:56:58.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:58 smithi157 ceph-mon[35346]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T05:56:58.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:56:58.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:58.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:58 smithi157 ceph-mon[35346]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T05:56:58.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:56:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:58 smithi005 ceph-mon[30955]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:56:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:58 smithi005 ceph-mon[30955]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T05:56:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:56:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:58 smithi005 ceph-mon[30955]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T05:56:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:56:59.301 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 05:56:58 smithi157 sudo[44412]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T05:56:59.301 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 05:56:58 smithi157 sudo[44412]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:56:59.301 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 05:56:58 smithi157 sudo[44412]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:56:59.301 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 05:56:58 smithi157 sudo[44412]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:56:59.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:59 smithi005 sudo[47441]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T05:56:59.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:59 smithi005 sudo[47441]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:56:59.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:59 smithi005 sudo[47441]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:56:59.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:59 smithi005 sudo[47441]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:57:00.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:59 smithi005 sudo[47817]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T05:57:00.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:59 smithi005 sudo[47817]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:57:00.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:59 smithi005 sudo[47817]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:57:00.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:56:59 smithi005 sudo[47817]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:57:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 sudo[44415]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T05:57:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 sudo[44415]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:57:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 sudo[44415]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:57:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 sudo[44415]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:57:00.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 sudo[44418]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T05:57:00.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 sudo[44418]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:57:00.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 sudo[44418]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:57:00.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 sudo[44418]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:57:00.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 ceph-mon[35346]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:00.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:00.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 ceph-mon[35346]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:57:00.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 ceph-mon[35346]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:57:00.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:57:00.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:57:00.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:57:00.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:57:00.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:57:00.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:57:00.569 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 ceph-mon[34640]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:00.569 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:00.569 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 ceph-mon[34640]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:57:00.570 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 ceph-mon[34640]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:57:00.570 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:57:00.570 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:57:00.570 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:57:00.570 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:57:00.570 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:57:00.570 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:57:00.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:00 smithi005 ceph-mon[30955]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:00.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:00.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:00 smithi005 ceph-mon[30955]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:57:00.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:00 smithi005 ceph-mon[30955]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:57:00.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:57:00.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:57:00.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:57:00.580 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:57:00.580 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:57:00.580 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:57:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 sudo[43283]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T05:57:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 sudo[43283]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:57:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 sudo[43283]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:57:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:00 smithi049 sudo[43283]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:57:01.603 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:01 smithi005 ceph-mon[30955]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:57:01.603 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:01 smithi005 ceph-mon[30955]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:57:01.603 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:01 smithi005 ceph-mon[30955]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:57:01.603 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:01 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:57:01.603 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:01 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:57:01.603 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:01 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:57:01.603 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:01 smithi005 ceph-mon[30955]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:57:01.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:01 smithi049 sudo[43286]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T05:57:01.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:01 smithi049 sudo[43286]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:57:01.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:01 smithi049 sudo[43286]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:57:01.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:01 smithi049 sudo[43286]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:57:01.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:01 smithi049 ceph-mon[34640]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:57:01.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:01 smithi049 ceph-mon[34640]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:57:01.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:01 smithi049 ceph-mon[34640]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:57:01.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:01 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:57:01.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:01 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:57:01.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:01 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:57:01.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:01 smithi049 ceph-mon[34640]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:57:01.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:01 smithi157 ceph-mon[35346]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:57:01.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:01 smithi157 ceph-mon[35346]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:57:01.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:01 smithi157 ceph-mon[35346]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:57:01.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:01 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:57:01.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:01 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:57:01.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:01 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:57:01.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:01 smithi157 ceph-mon[35346]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:57:02.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:02 smithi005 ceph-mon[30955]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:02.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:02 smithi049 ceph-mon[34640]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:02 smithi157 ceph-mon[35346]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:04.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:04 smithi005 ceph-mon[30955]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:04.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:04 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T05:57:04.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:04 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:04 smithi049 ceph-mon[34640]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:04 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T05:57:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:04 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:04 smithi157 ceph-mon[35346]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:04 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T05:57:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:04 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:05 smithi005 ceph-mon[30955]: Deploying daemon osd.3 on smithi005 2024-09-19T05:57:05.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:05 smithi049 ceph-mon[34640]: Deploying daemon osd.3 on smithi005 2024-09-19T05:57:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:05 smithi157 ceph-mon[35346]: Deploying daemon osd.3 on smithi005 2024-09-19T05:57:06.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:06 smithi005 ceph-mon[30955]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T05:57:06.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:06 smithi049 ceph-mon[34640]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T05:57:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:06 smithi157 ceph-mon[35346]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T05:57:08.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:08 smithi005 ceph-mon[30955]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T05:57:08.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:08 smithi049 ceph-mon[34640]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T05:57:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:08 smithi157 ceph-mon[35346]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T05:57:10.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:10 smithi005 ceph-mon[30955]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T05:57:10.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:10 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:10.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:10 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:10.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:10 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:10 smithi049 ceph-mon[34640]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T05:57:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:10 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:10 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:10 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:10 smithi157 ceph-mon[35346]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T05:57:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:10 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:10 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:10 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:12.590 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:12 smithi005 ceph-mon[30955]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T05:57:12.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:12 smithi049 ceph-mon[34640]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T05:57:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:12 smithi157 ceph-mon[35346]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T05:57:14.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:13 smithi049 ceph-mon[34640]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T05:57:14.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:13 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:14.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:13 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:14.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:13 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:14.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:13 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:14.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:13 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:14.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:13 smithi005 ceph-mon[30955]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T05:57:14.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:13 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:14.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:13 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:14.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:13 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:14.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:13 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:14.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:13 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.267+0000 7f3e9a7fc640 1 -- 172.21.15.5:0/4181969174 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3ea406a680 con 0x7f3e900a3260 2024-09-19T05:57:14.269 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 3 on host 'smithi005' 2024-09-19T05:57:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.269+0000 7f3eab512640 1 -- 172.21.15.5:0/4181969174 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3e900a3260 msgr2=0x7f3e900a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.269+0000 7f3eab512640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3e900a3260 0x7f3e900a5720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4101b30 tx=0x7f3ea0008040 comp rx=0 tx=0).stop 2024-09-19T05:57:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.269+0000 7f3eab512640 1 -- 172.21.15.5:0/4181969174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102690 msgr2=0x7f3ea419f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.269+0000 7f3eab512640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102690 0x7f3ea419f4f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009fd0 tx=0x7f3e9c00b260 comp rx=0 tx=0).stop 2024-09-19T05:57:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.270+0000 7f3eab512640 1 -- 172.21.15.5:0/4181969174 shutdown_connections 2024-09-19T05:57:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.270+0000 7f3eab512640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3e900a3260 0x7f3e900a5720 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.270+0000 7f3eab512640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102690 0x7f3ea419f4f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.270+0000 7f3eab512640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea4101cd0 0x7f3ea4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.270+0000 7f3eab512640 1 --2- 172.21.15.5:0/4181969174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea4100ad0 0x7f3ea4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.270+0000 7f3eab512640 1 -- 172.21.15.5:0/4181969174 >> 172.21.15.5:0/4181969174 conn(0x7f3ea40fc280 msgr2=0x7f3ea4069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:57:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.270+0000 7f3eab512640 1 -- 172.21.15.5:0/4181969174 shutdown_connections 2024-09-19T05:57:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:57:14.270+0000 7f3eab512640 1 -- 172.21.15.5:0/4181969174 wait complete. 2024-09-19T05:57:14.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:13 smithi157 ceph-mon[35346]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T05:57:14.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:14.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:14.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:14.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:14.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:14.886 DEBUG:teuthology.orchestra.run.smithi005:osd.3> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T05:57:14.889 INFO:tasks.cephadm:Deploying osd.4 on smithi049 with /dev/vg_nvme/lv_3... 2024-09-19T05:57:14.889 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T05:57:15.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:15.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:15.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:15.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:15.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:15.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:15.165 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:57:15.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:15.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:15.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:16.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:16 smithi005 ceph-mon[30955]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T05:57:16.513 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:16 smithi049 ceph-mon[34640]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T05:57:16.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:16 smithi157 ceph-mon[35346]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T05:57:16.879 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 05:57:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3[48677]: 2024-09-19T05:57:16.628+0000 7f92ba6cb640 -1 osd.3 0 log_to_monitors true 2024-09-19T05:57:17.497 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:17 smithi005 ceph-mon[30955]: from='osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T05:57:17.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:17 smithi157 ceph-mon[35346]: from='osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T05:57:17.591 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:17 smithi049 ceph-mon[34640]: from='osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T05:57:17.594 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:57:17.620 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_3 2024-09-19T05:57:17.876 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:57:18.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.311+0000 7fb75f7b7640 1 -- 172.21.15.49:0/2667630567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb758100ad0 msgr2=0x7fb758100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:18.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.311+0000 7fb75f7b7640 1 --2- 172.21.15.49:0/2667630567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb758100ad0 0x7fb758100ed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb754009900 tx=0x7fb75402a410 comp rx=0 tx=0).stop 2024-09-19T05:57:18.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.311+0000 7fb75f7b7640 1 -- 172.21.15.49:0/2667630567 shutdown_connections 2024-09-19T05:57:18.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.311+0000 7fb75f7b7640 1 --2- 172.21.15.49:0/2667630567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb758102690 0x7fb758108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:18.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.311+0000 7fb75f7b7640 1 --2- 172.21.15.49:0/2667630567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb758101cd0 0x7fb758102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.311+0000 7fb75f7b7640 1 --2- 172.21.15.49:0/2667630567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb758100ad0 0x7fb758100ed0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.311+0000 7fb75f7b7640 1 -- 172.21.15.49:0/2667630567 >> 172.21.15.49:0/2667630567 conn(0x7fb7580fc280 msgr2=0x7fb7580fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:57:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.311+0000 7fb75f7b7640 1 -- 172.21.15.49:0/2667630567 shutdown_connections 2024-09-19T05:57:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.311+0000 7fb75f7b7640 1 -- 172.21.15.49:0/2667630567 wait complete. 2024-09-19T05:57:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.312+0000 7fb75f7b7640 1 Processor -- start 2024-09-19T05:57:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.312+0000 7fb75f7b7640 1 -- start start 2024-09-19T05:57:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.312+0000 7fb75f7b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb758100ad0 0x7fb75806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.312+0000 7fb75f7b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb758101cd0 0x7fb75819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75f7b7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb758102690 0x7fb75819f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75f7b7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810c660 con 0x7fb758101cd0 2024-09-19T05:57:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75f7b7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810c4e0 con 0x7fb758102690 2024-09-19T05:57:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75f7b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810c7e0 con 0x7fb758100ad0 2024-09-19T05:57:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75dd54640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb758102690 0x7fb75819f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75dd54640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb758102690 0x7fb75819f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33182/0 (socket says 172.21.15.49:33182) 2024-09-19T05:57:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75dd54640 1 -- 172.21.15.49:0/1067040961 learned_addr learned my addr 172.21.15.49:0/1067040961 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-19T05:57:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75cd52640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb758101cd0 0x7fb75819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75d553640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb758100ad0 0x7fb75806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75dd54640 1 -- 172.21.15.49:0/1067040961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb758100ad0 msgr2=0x7fb75806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75dd54640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb758100ad0 0x7fb75806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75dd54640 1 -- 172.21.15.49:0/1067040961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb758101cd0 msgr2=0x7fb75819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75dd54640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb758101cd0 0x7fb75819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75dd54640 1 -- 172.21.15.49:0/1067040961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75819fb10 con 0x7fb758102690 2024-09-19T05:57:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75d553640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb758100ad0 0x7fb75806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:57:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb75dd54640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb758102690 0x7fb75819f200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb75000cad0 tx=0x7fb75000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:57:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.313+0000 7fb74e7fc640 1 -- 172.21.15.49:0/1067040961 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75000ae20 con 0x7fb758102690 2024-09-19T05:57:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.314+0000 7fb74e7fc640 1 -- 172.21.15.49:0/1067040961 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb750005020 con 0x7fb758102690 2024-09-19T05:57:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.314+0000 7fb74e7fc640 1 -- 172.21.15.49:0/1067040961 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75000f6b0 con 0x7fb758102690 2024-09-19T05:57:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.314+0000 7fb75f7b7640 1 -- 172.21.15.49:0/1067040961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75819fda0 con 0x7fb758102690 2024-09-19T05:57:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.314+0000 7fb75f7b7640 1 -- 172.21.15.49:0/1067040961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7581a0260 con 0x7fb758102690 2024-09-19T05:57:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.315+0000 7fb75f7b7640 1 -- 172.21.15.49:0/1067040961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb758107f70 con 0x7fb758102690 2024-09-19T05:57:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.316+0000 7fb74e7fc640 1 -- 172.21.15.49:0/1067040961 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fb7500051d0 con 0x7fb758102690 2024-09-19T05:57:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.317+0000 7fb74e7fc640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb7440a32c0 0x7fb7440a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.317+0000 7fb74e7fc640 1 -- 172.21.15.49:0/1067040961 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3389+0+0 (secure 0 0 0) 0x7fb7500ce830 con 0x7fb758102690 2024-09-19T05:57:18.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.317+0000 7fb75d553640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb7440a32c0 0x7fb7440a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:18.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.318+0000 7fb75d553640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb7440a32c0 0x7fb7440a5780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb754006010 tx=0x7fb754000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:57:18.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.320+0000 7fb74e7fc640 1 -- 172.21.15.49:0/1067040961 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb750014030 con 0x7fb758102690 2024-09-19T05:57:18.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:18.506+0000 7fb75f7b7640 1 -- 172.21.15.49:0/1067040961 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb758062630 con 0x7fb7440a32c0 2024-09-19T05:57:18.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:18.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: from='osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T05:57:18.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T05:57:18.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: from='osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:57:18.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:18.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:18.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:18.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:57:18.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:57:18.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:18.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:18.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:18.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:18.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: from='osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T05:57:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T05:57:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: from='osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:57:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:57:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:57:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:18.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:18.631 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3[48677]: 2024-09-19T05:57:18.280+0000 7f92b6e62640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T05:57:18.631 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 05:57:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3[48677]: 2024-09-19T05:57:18.286+0000 7f92b2465640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:57:18.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:18.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: from='osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T05:57:18.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T05:57:18.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: from='osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:57:18.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:18.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:18.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:18.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:57:18.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:57:18.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:18.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:18.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:18.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:18 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:19.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:19 smithi157 ceph-mon[35346]: Detected new or changed devices on smithi005 2024-09-19T05:57:19.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:19 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi005 to 8546M 2024-09-19T05:57:19.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:19 smithi157 ceph-mon[35346]: from='osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-19T05:57:19.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:19 smithi157 ceph-mon[35346]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T05:57:19.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:19 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:19.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:19 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:19.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:19 smithi157 ceph-mon[35346]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:57:19.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:19 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:57:19.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:19 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:57:19.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:19 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:19.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:19 smithi005 ceph-mon[30955]: Detected new or changed devices on smithi005 2024-09-19T05:57:19.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:19 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi005 to 8546M 2024-09-19T05:57:19.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:19 smithi005 ceph-mon[30955]: from='osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-19T05:57:19.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:19 smithi005 ceph-mon[30955]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T05:57:19.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:19 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:19.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:19 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:19.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:19 smithi005 ceph-mon[30955]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:57:19.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:19 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:57:19.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:19 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:57:19.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:19 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:19.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:19 smithi049 ceph-mon[34640]: Detected new or changed devices on smithi005 2024-09-19T05:57:19.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:19 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi005 to 8546M 2024-09-19T05:57:19.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:19 smithi049 ceph-mon[34640]: from='osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-19T05:57:19.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:19 smithi049 ceph-mon[34640]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T05:57:19.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:19 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:19.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:19 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:19.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:19 smithi049 ceph-mon[34640]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:57:19.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:19 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:57:19.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:19 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:57:19.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:19 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:20.463 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:20 smithi049 ceph-mon[34640]: purged_snaps scrub starts 2024-09-19T05:57:20.463 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:20 smithi049 ceph-mon[34640]: purged_snaps scrub ok 2024-09-19T05:57:20.463 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:20 smithi049 ceph-mon[34640]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:20.463 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:20 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:20.463 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:20 smithi049 ceph-mon[34640]: osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753] boot 2024-09-19T05:57:20.463 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:20 smithi049 ceph-mon[34640]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T05:57:20.463 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:20 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:20.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:20 smithi157 ceph-mon[35346]: purged_snaps scrub starts 2024-09-19T05:57:20.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:20 smithi157 ceph-mon[35346]: purged_snaps scrub ok 2024-09-19T05:57:20.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:20 smithi157 ceph-mon[35346]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:20.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:20 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:20.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:20 smithi157 ceph-mon[35346]: osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753] boot 2024-09-19T05:57:20.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:20 smithi157 ceph-mon[35346]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T05:57:20.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:20 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:20 smithi005 ceph-mon[30955]: purged_snaps scrub starts 2024-09-19T05:57:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:20 smithi005 ceph-mon[30955]: purged_snaps scrub ok 2024-09-19T05:57:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:20 smithi005 ceph-mon[30955]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:57:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:20 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:20 smithi005 ceph-mon[30955]: osd.3 [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753] boot 2024-09-19T05:57:20.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:20 smithi005 ceph-mon[30955]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T05:57:20.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:20 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:57:21.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:21 smithi157 ceph-mon[35346]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T05:57:21.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:21 smithi157 ceph-mon[35346]: from='client.? 172.21.15.49:0/2387368382' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26"}]: dispatch 2024-09-19T05:57:21.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:21 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26"}]: dispatch 2024-09-19T05:57:21.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:21 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26"}]': finished 2024-09-19T05:57:21.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:21 smithi157 ceph-mon[35346]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T05:57:21.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:21.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:21 smithi005 ceph-mon[30955]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T05:57:21.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:21 smithi005 ceph-mon[30955]: from='client.? 172.21.15.49:0/2387368382' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26"}]: dispatch 2024-09-19T05:57:21.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:21 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26"}]: dispatch 2024-09-19T05:57:21.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:21 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26"}]': finished 2024-09-19T05:57:21.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:21 smithi005 ceph-mon[30955]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T05:57:21.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:21.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:21 smithi049 ceph-mon[34640]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T05:57:21.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:21 smithi049 ceph-mon[34640]: from='client.? 172.21.15.49:0/2387368382' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26"}]: dispatch 2024-09-19T05:57:21.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:21 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26"}]: dispatch 2024-09-19T05:57:21.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:21 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26"}]': finished 2024-09-19T05:57:21.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:21 smithi049 ceph-mon[34640]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T05:57:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:22.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:22 smithi157 ceph-mon[35346]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:22.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:22 smithi157 ceph-mon[35346]: from='client.? 172.21.15.49:0/4154856882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:57:22.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:22 smithi157 ceph-mon[35346]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T05:57:22.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:22 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:22.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:22 smithi005 ceph-mon[30955]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:22.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:22 smithi005 ceph-mon[30955]: from='client.? 172.21.15.49:0/4154856882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:57:22.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:22 smithi005 ceph-mon[30955]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T05:57:22.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:22 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:22.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:22 smithi049 ceph-mon[34640]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:22.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:22 smithi049 ceph-mon[34640]: from='client.? 172.21.15.49:0/4154856882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:57:22.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:22 smithi049 ceph-mon[34640]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T05:57:22.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:24.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:24 smithi005 ceph-mon[30955]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:24.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:24 smithi005 ceph-mon[30955]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T05:57:24.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:24 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:24.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:24 smithi049 ceph-mon[34640]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:24.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:24 smithi049 ceph-mon[34640]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T05:57:24.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:24 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:24.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:24 smithi157 ceph-mon[35346]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:24.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:24 smithi157 ceph-mon[35346]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T05:57:24.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:24 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:26 smithi049 ceph-mon[34640]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-09-19T05:57:26.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:26 smithi005 ceph-mon[30955]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-09-19T05:57:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:26 smithi157 ceph-mon[35346]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-09-19T05:57:28.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:28 smithi005 ceph-mon[30955]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T05:57:28.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:28 smithi049 ceph-mon[34640]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T05:57:28.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:28 smithi157 ceph-mon[35346]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T05:57:30.322 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:30 smithi049 ceph-mon[34640]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-19T05:57:30.322 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:30 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T05:57:30.322 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:30 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:30.322 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:30 smithi049 ceph-mon[34640]: Deploying daemon osd.4 on smithi049 2024-09-19T05:57:30.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:30 smithi005 ceph-mon[30955]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-19T05:57:30.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:30 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T05:57:30.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:30 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:30.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:30 smithi005 ceph-mon[30955]: Deploying daemon osd.4 on smithi049 2024-09-19T05:57:30.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:30 smithi157 ceph-mon[35346]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-19T05:57:30.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:30 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T05:57:30.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:30 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:30.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:30 smithi157 ceph-mon[35346]: Deploying daemon osd.4 on smithi049 2024-09-19T05:57:32.370 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:32 smithi049 ceph-mon[34640]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T05:57:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:32 smithi005 ceph-mon[30955]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T05:57:32.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:32 smithi157 ceph-mon[35346]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T05:57:34.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:34 smithi005 ceph-mon[30955]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T05:57:34.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:34 smithi049 ceph-mon[34640]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T05:57:34.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:34 smithi157 ceph-mon[35346]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T05:57:36.376 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:36 smithi049 ceph-mon[34640]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T05:57:36.376 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:36 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:36.376 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:36 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:36.376 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:36 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:36.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:36 smithi005 ceph-mon[30955]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T05:57:36.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:36 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:36.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:36 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:36.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:36 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:36.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:36 smithi157 ceph-mon[35346]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T05:57:36.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:36 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:36.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:36 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:36.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:36 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:38.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:38 smithi005 ceph-mon[30955]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:38.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:38 smithi049 ceph-mon[34640]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:38.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:38 smithi157 ceph-mon[35346]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:39.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.397+0000 7fb74e7fc640 1 -- 172.21.15.49:0/1067040961 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb758062630 con 0x7fb7440a32c0 2024-09-19T05:57:39.399 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 4 on host 'smithi049' 2024-09-19T05:57:39.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.399+0000 7fb75f7b7640 1 -- 172.21.15.49:0/1067040961 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb7440a32c0 msgr2=0x7fb7440a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:39.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.399+0000 7fb75f7b7640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb7440a32c0 0x7fb7440a5780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb754006010 tx=0x7fb754000bc0 comp rx=0 tx=0).stop 2024-09-19T05:57:39.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.399+0000 7fb75f7b7640 1 -- 172.21.15.49:0/1067040961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb758102690 msgr2=0x7fb75819f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:39.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.399+0000 7fb75f7b7640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb758102690 0x7fb75819f200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb75000cad0 tx=0x7fb75000cf90 comp rx=0 tx=0).stop 2024-09-19T05:57:39.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.399+0000 7fb75f7b7640 1 -- 172.21.15.49:0/1067040961 shutdown_connections 2024-09-19T05:57:39.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.399+0000 7fb75f7b7640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb7440a32c0 0x7fb7440a5780 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:39.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.399+0000 7fb75f7b7640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb758102690 0x7fb75819f200 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:39.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.399+0000 7fb75f7b7640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb758101cd0 0x7fb75819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:39.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.399+0000 7fb75f7b7640 1 --2- 172.21.15.49:0/1067040961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb758100ad0 0x7fb75806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:39.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.399+0000 7fb75f7b7640 1 -- 172.21.15.49:0/1067040961 >> 172.21.15.49:0/1067040961 conn(0x7fb7580fc280 msgr2=0x7fb758105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:57:39.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.400+0000 7fb75f7b7640 1 -- 172.21.15.49:0/1067040961 shutdown_connections 2024-09-19T05:57:39.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:57:39.400+0000 7fb75f7b7640 1 -- 172.21.15.49:0/1067040961 wait complete. 2024-09-19T05:57:39.987 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:39.987 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:39.987 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:39.987 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:39.987 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:39.987 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:39.987 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:39.987 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:40.014 DEBUG:teuthology.orchestra.run.smithi049:osd.4> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T05:57:40.016 INFO:tasks.cephadm:Deploying osd.5 on smithi157 with /dev/vg_nvme/lv_3... 2024-09-19T05:57:40.016 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T05:57:40.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:40.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:40.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:40.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:40.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:40.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:40.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:40.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:40.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:40.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:40.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:40.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:40.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:40.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:40.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:40.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:40.270 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:57:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:40 smithi049 ceph-mon[34640]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:41.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:40 smithi157 ceph-mon[35346]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:41.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:40 smithi005 ceph-mon[30955]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:42.282 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 05:57:42 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4[46049]: 2024-09-19T05:57:42.025+0000 7f02436ba640 -1 osd.4 0 log_to_monitors true 2024-09-19T05:57:42.671 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:57:42.696 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_3 2024-09-19T05:57:42.859 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:42 smithi049 ceph-mon[34640]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:42.859 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:42 smithi049 ceph-mon[34640]: from='osd.4 [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:57:42.859 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:42 smithi049 ceph-mon[34640]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:57:42.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:42 smithi005 ceph-mon[30955]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:42.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:42 smithi005 ceph-mon[30955]: from='osd.4 [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:57:42.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:42 smithi005 ceph-mon[30955]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:57:42.984 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:57:43.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:42 smithi157 ceph-mon[35346]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:43.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:42 smithi157 ceph-mon[35346]: from='osd.4 [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:57:43.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:42 smithi157 ceph-mon[35346]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:57:43.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.436+0000 7fdc4cc19640 1 -- 172.21.15.157:0/868067435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48101cd0 msgr2=0x7fdc48102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:43.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.436+0000 7fdc4cc19640 1 --2- 172.21.15.157:0/868067435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48101cd0 0x7fdc48102150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009900 tx=0x7fdc3c02a410 comp rx=0 tx=0).stop 2024-09-19T05:57:43.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.437+0000 7fdc4cc19640 1 -- 172.21.15.157:0/868067435 shutdown_connections 2024-09-19T05:57:43.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.437+0000 7fdc4cc19640 1 --2- 172.21.15.157:0/868067435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc48102690 0x7fdc48108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:43.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.437+0000 7fdc4cc19640 1 --2- 172.21.15.157:0/868067435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48101cd0 0x7fdc48102150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:43.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.437+0000 7fdc4cc19640 1 --2- 172.21.15.157:0/868067435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc48100ad0 0x7fdc48100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:43.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.437+0000 7fdc4cc19640 1 -- 172.21.15.157:0/868067435 >> 172.21.15.157:0/868067435 conn(0x7fdc480fc280 msgr2=0x7fdc480fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:57:43.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.437+0000 7fdc4cc19640 1 -- 172.21.15.157:0/868067435 shutdown_connections 2024-09-19T05:57:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.437+0000 7fdc4cc19640 1 -- 172.21.15.157:0/868067435 wait complete. 2024-09-19T05:57:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.437+0000 7fdc4cc19640 1 Processor -- start 2024-09-19T05:57:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.437+0000 7fdc4cc19640 1 -- start start 2024-09-19T05:57:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.437+0000 7fdc4cc19640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc48100ad0 0x7fdc48196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.437+0000 7fdc4cc19640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc48101cd0 0x7fdc48197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc4cc19640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48102690 0x7fdc4819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc4cc19640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4810c600 con 0x7fdc48100ad0 2024-09-19T05:57:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc4cc19640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4810c480 con 0x7fdc48101cd0 2024-09-19T05:57:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc4cc19640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4810c780 con 0x7fdc48102690 2024-09-19T05:57:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc46d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48102690 0x7fdc4819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc46d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48102690 0x7fdc4819e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:32988/0 (socket says 172.21.15.157:32988) 2024-09-19T05:57:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc46d9d640 1 -- 172.21.15.157:0/1381637501 learned_addr learned my addr 172.21.15.157:0/1381637501 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T05:57:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc4659c640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc48100ad0 0x7fdc48196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc45d9b640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc48101cd0 0x7fdc48197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc46d9d640 1 -- 172.21.15.157:0/1381637501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc48101cd0 msgr2=0x7fdc48197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc46d9d640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc48101cd0 0x7fdc48197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc46d9d640 1 -- 172.21.15.157:0/1381637501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc48100ad0 msgr2=0x7fdc48196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc46d9d640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc48100ad0 0x7fdc48196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc46d9d640 1 -- 172.21.15.157:0/1381637501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4819e920 con 0x7fdc48102690 2024-09-19T05:57:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc45d9b640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc48101cd0 0x7fdc48197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:57:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc46d9d640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48102690 0x7fdc4819e280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000eaa0 tx=0x7fdc4000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:57:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc377fe640 1 -- 172.21.15.157:0/1381637501 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000cdb0 con 0x7fdc48102690 2024-09-19T05:57:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc4cc19640 1 -- 172.21.15.157:0/1381637501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4819ec10 con 0x7fdc48102690 2024-09-19T05:57:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc377fe640 1 -- 172.21.15.157:0/1381637501 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc40005020 con 0x7fdc48102690 2024-09-19T05:57:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc377fe640 1 -- 172.21.15.157:0/1381637501 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc400116d0 con 0x7fdc48102690 2024-09-19T05:57:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.438+0000 7fdc4cc19640 1 -- 172.21.15.157:0/1381637501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc4819f1a0 con 0x7fdc48102690 2024-09-19T05:57:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.440+0000 7fdc4cc19640 1 -- 172.21.15.157:0/1381637501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc48107f90 con 0x7fdc48102690 2024-09-19T05:57:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.440+0000 7fdc377fe640 1 -- 172.21.15.157:0/1381637501 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fdc400051d0 con 0x7fdc48102690 2024-09-19T05:57:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.441+0000 7fdc377fe640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdc300a3240 0x7fdc300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.441+0000 7fdc377fe640 1 -- 172.21.15.157:0/1381637501 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3805+0+0 (secure 0 0 0) 0x7fdc400ce920 con 0x7fdc48102690 2024-09-19T05:57:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.441+0000 7fdc4659c640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdc300a3240 0x7fdc300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.442+0000 7fdc4659c640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdc300a3240 0x7fdc300a5700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc481981e0 tx=0x7fdc3800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:57:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.446+0000 7fdc377fe640 1 -- 172.21.15.157:0/1381637501 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdc40016030 con 0x7fdc48102690 2024-09-19T05:57:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:57:43.624+0000 7fdc4cc19640 1 -- 172.21.15.157:0/1381637501 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdc48062630 con 0x7fdc300a3240 2024-09-19T05:57:43.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T05:57:43.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='osd.4 [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:57:43.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T05:57:43.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:43.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:57:43.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:43.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:43.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:57:43.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:57:43.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:43.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:43.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:43.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:43.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:57:43.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:57:43.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:43.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:43 smithi005 ceph-mon[30955]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-19T05:57:44.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T05:57:44.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='osd.4 [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:57:44.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T05:57:44.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:44.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:57:44.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:44.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:44.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:57:44.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:57:44.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:44.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:44.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:44.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:44.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:57:44.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:57:44.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:44.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:43 smithi157 ceph-mon[35346]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-19T05:57:44.146 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4[46049]: 2024-09-19T05:57:43.704+0000 7f023ee1a640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T05:57:44.146 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4[46049]: 2024-09-19T05:57:43.714+0000 7f023b454640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:57:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T05:57:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='osd.4 [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:57:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T05:57:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:57:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:57:44.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:57:44.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:44.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:44.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:44.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:57:44.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:57:44.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:57:44.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:44.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:43 smithi049 ceph-mon[34640]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-19T05:57:45.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:44 smithi157 ceph-mon[35346]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:45.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:44 smithi157 ceph-mon[35346]: Detected new or changed devices on smithi049 2024-09-19T05:57:45.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:44 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi049 to 8544M 2024-09-19T05:57:45.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:44 smithi157 ceph-mon[35346]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:57:45.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:44 smithi157 ceph-mon[35346]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T05:57:45.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:44 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:45.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:44 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:45.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:44 smithi005 ceph-mon[30955]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:45.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:44 smithi005 ceph-mon[30955]: Detected new or changed devices on smithi049 2024-09-19T05:57:45.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:44 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi049 to 8544M 2024-09-19T05:57:45.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:44 smithi005 ceph-mon[30955]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:57:45.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:44 smithi005 ceph-mon[30955]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T05:57:45.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:44 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:45.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:44 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:45.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:44 smithi049 ceph-mon[34640]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:45.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:44 smithi049 ceph-mon[34640]: Detected new or changed devices on smithi049 2024-09-19T05:57:45.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:44 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi049 to 8544M 2024-09-19T05:57:45.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:44 smithi049 ceph-mon[34640]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:57:45.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:44 smithi049 ceph-mon[34640]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T05:57:45.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:44 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:45.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:44 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:46.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:45 smithi157 ceph-mon[35346]: purged_snaps scrub starts 2024-09-19T05:57:46.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:45 smithi157 ceph-mon[35346]: purged_snaps scrub ok 2024-09-19T05:57:46.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:45 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:46.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:45 smithi157 ceph-mon[35346]: osd.4 [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580] boot 2024-09-19T05:57:46.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:45 smithi157 ceph-mon[35346]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T05:57:46.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:45 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:46.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:45 smithi005 ceph-mon[30955]: purged_snaps scrub starts 2024-09-19T05:57:46.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:45 smithi005 ceph-mon[30955]: purged_snaps scrub ok 2024-09-19T05:57:46.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:45 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:46.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:45 smithi005 ceph-mon[30955]: osd.4 [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580] boot 2024-09-19T05:57:46.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:45 smithi005 ceph-mon[30955]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T05:57:46.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:45 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:46.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:45 smithi049 ceph-mon[34640]: purged_snaps scrub starts 2024-09-19T05:57:46.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:45 smithi049 ceph-mon[34640]: purged_snaps scrub ok 2024-09-19T05:57:46.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:45 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:46.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:45 smithi049 ceph-mon[34640]: osd.4 [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580] boot 2024-09-19T05:57:46.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:45 smithi049 ceph-mon[34640]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T05:57:46.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:45 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:57:46.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:46 smithi157 ceph-mon[35346]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:46.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:46 smithi157 ceph-mon[35346]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T05:57:46.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:46 smithi157 ceph-mon[35346]: from='client.? 172.21.15.157:0/3508815406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a7a7d3-ce42-4688-bfec-714be635eaa2"}]: dispatch 2024-09-19T05:57:46.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:46 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a7a7d3-ce42-4688-bfec-714be635eaa2"}]: dispatch 2024-09-19T05:57:46.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:46 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21a7a7d3-ce42-4688-bfec-714be635eaa2"}]': finished 2024-09-19T05:57:46.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:46 smithi157 ceph-mon[35346]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T05:57:46.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:46 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:57:46.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:46 smithi157 ceph-mon[35346]: from='client.? 172.21.15.157:0/2779967426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:57:47.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:46 smithi005 ceph-mon[30955]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:47.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:46 smithi005 ceph-mon[30955]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T05:57:47.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:46 smithi005 ceph-mon[30955]: from='client.? 172.21.15.157:0/3508815406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a7a7d3-ce42-4688-bfec-714be635eaa2"}]: dispatch 2024-09-19T05:57:47.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:46 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a7a7d3-ce42-4688-bfec-714be635eaa2"}]: dispatch 2024-09-19T05:57:47.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:46 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21a7a7d3-ce42-4688-bfec-714be635eaa2"}]': finished 2024-09-19T05:57:47.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:46 smithi005 ceph-mon[30955]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T05:57:47.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:46 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:57:47.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:46 smithi005 ceph-mon[30955]: from='client.? 172.21.15.157:0/2779967426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:57:47.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:46 smithi049 ceph-mon[34640]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:57:47.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:46 smithi049 ceph-mon[34640]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T05:57:47.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:46 smithi049 ceph-mon[34640]: from='client.? 172.21.15.157:0/3508815406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a7a7d3-ce42-4688-bfec-714be635eaa2"}]: dispatch 2024-09-19T05:57:47.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:46 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a7a7d3-ce42-4688-bfec-714be635eaa2"}]: dispatch 2024-09-19T05:57:47.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:46 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21a7a7d3-ce42-4688-bfec-714be635eaa2"}]': finished 2024-09-19T05:57:47.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:46 smithi049 ceph-mon[34640]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T05:57:47.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:46 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:57:47.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:46 smithi049 ceph-mon[34640]: from='client.? 172.21.15.157:0/2779967426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:57:49.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:48 smithi157 ceph-mon[35346]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:49.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:48 smithi005 ceph-mon[30955]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:49.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:48 smithi049 ceph-mon[34640]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:49 smithi157 ceph-mon[35346]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:49 smithi005 ceph-mon[30955]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:50.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:49 smithi049 ceph-mon[34640]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:52.046 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:51 smithi157 ceph-mon[35346]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:52.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:51 smithi005 ceph-mon[30955]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:52.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:51 smithi049 ceph-mon[34640]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:54.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:53 smithi157 ceph-mon[35346]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:54.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:53 smithi005 ceph-mon[30955]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:54.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:53 smithi049 ceph-mon[34640]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:55.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:54 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T05:57:55.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:54 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:55.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:54 smithi005 ceph-mon[30955]: Deploying daemon osd.5 on smithi157 2024-09-19T05:57:55.141 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:54 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T05:57:55.141 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:54 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:55.141 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:54 smithi157 ceph-mon[35346]: Deploying daemon osd.5 on smithi157 2024-09-19T05:57:55.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:54 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T05:57:55.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:54 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:55.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:54 smithi049 ceph-mon[34640]: Deploying daemon osd.5 on smithi157 2024-09-19T05:57:56.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:55 smithi005 ceph-mon[30955]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:56.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:55 smithi049 ceph-mon[34640]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:56.176 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:55 smithi157 ceph-mon[35346]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:58.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:57 smithi005 ceph-mon[30955]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:58.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:57 smithi049 ceph-mon[34640]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:57:58.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:57 smithi157 ceph-mon[35346]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:00.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:57:59 smithi157 ceph-mon[35346]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:00.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:57:59 smithi005 ceph-mon[30955]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:00.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:57:59 smithi049 ceph-mon[34640]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:01.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:01.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:01.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:01.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:01.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:01.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:02.267 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:01 smithi157 ceph-mon[35346]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:02.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:01 smithi005 ceph-mon[30955]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:02.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:01 smithi049 ceph-mon[34640]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:03.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.141+0000 7fdc377fe640 1 -- 172.21.15.157:0/1381637501 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdc48062630 con 0x7fdc300a3240 2024-09-19T05:58:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 5 on host 'smithi157' 2024-09-19T05:58:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.143+0000 7fdc4cc19640 1 -- 172.21.15.157:0/1381637501 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdc300a3240 msgr2=0x7fdc300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.143+0000 7fdc4cc19640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdc300a3240 0x7fdc300a5700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc481981e0 tx=0x7fdc3800b390 comp rx=0 tx=0).stop 2024-09-19T05:58:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.143+0000 7fdc4cc19640 1 -- 172.21.15.157:0/1381637501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48102690 msgr2=0x7fdc4819e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.143+0000 7fdc4cc19640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48102690 0x7fdc4819e280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000eaa0 tx=0x7fdc4000ef60 comp rx=0 tx=0).stop 2024-09-19T05:58:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.143+0000 7fdc4cc19640 1 -- 172.21.15.157:0/1381637501 shutdown_connections 2024-09-19T05:58:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.143+0000 7fdc4cc19640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdc300a3240 0x7fdc300a5700 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.143+0000 7fdc4cc19640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48102690 0x7fdc4819e280 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.143+0000 7fdc4cc19640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc48101cd0 0x7fdc48197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:03.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.143+0000 7fdc4cc19640 1 --2- 172.21.15.157:0/1381637501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc48100ad0 0x7fdc48196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:03.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.143+0000 7fdc4cc19640 1 -- 172.21.15.157:0/1381637501 >> 172.21.15.157:0/1381637501 conn(0x7fdc480fc280 msgr2=0x7fdc48105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:03.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.144+0000 7fdc4cc19640 1 -- 172.21.15.157:0/1381637501 shutdown_connections 2024-09-19T05:58:03.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:03.144+0000 7fdc4cc19640 1 -- 172.21.15.157:0/1381637501 wait complete. 2024-09-19T05:58:03.754 DEBUG:teuthology.orchestra.run.smithi157:osd.5> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T05:58:03.756 INFO:tasks.cephadm:Deploying osd.6 on smithi005 with /dev/vg_nvme/lv_2... 2024-09-19T05:58:03.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T05:58:03.979 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:03 smithi157 ceph-mon[35346]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:03.979 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:03.979 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:03.979 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:03.979 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:03.979 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:03.979 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:03.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:03.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:03 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:04.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:03 smithi005 ceph-mon[30955]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:04.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:04.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:04.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:04.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:04.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:04.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:04.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:04.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:03 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:04.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:03 smithi049 ceph-mon[34640]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:04.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:04.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:04.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:04.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:04.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:04.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:04.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:04.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:03 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:06.050 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:05 smithi005 ceph-mon[30955]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:06.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:05 smithi049 ceph-mon[34640]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:06.180 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:05 smithi157 ceph-mon[35346]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:06.489 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T05:58:06.291+0000 7fdd2ccb1640 -1 osd.5 0 log_to_monitors true 2024-09-19T05:58:06.772 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:58:06.799 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_2 2024-09-19T05:58:07.069 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:06 smithi005 ceph-mon[30955]: from='osd.5 [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:58:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:06 smithi005 ceph-mon[30955]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:58:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:06 smithi005 ceph-mon[30955]: Detected new or changed devices on smithi157 2024-09-19T05:58:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:07.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:07.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:06 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi157 to 8544M 2024-09-19T05:58:07.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:07.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:07.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:07.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:07.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:06 smithi049 ceph-mon[34640]: from='osd.5 [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:58:07.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:06 smithi049 ceph-mon[34640]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:58:07.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:06 smithi049 ceph-mon[34640]: Detected new or changed devices on smithi157 2024-09-19T05:58:07.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:07.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:07.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:07.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:07.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:06 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi157 to 8544M 2024-09-19T05:58:07.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:07.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:07.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:07.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:07.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-mon[35346]: from='osd.5 [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:58:07.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-mon[35346]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:58:07.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-mon[35346]: Detected new or changed devices on smithi157 2024-09-19T05:58:07.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:07.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:07.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:07.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:07.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi157 to 8544M 2024-09-19T05:58:07.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:07.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:07.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:07.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.506+0000 7fcb5f59e640 1 -- 172.21.15.5:0/2229377484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb601024e0 msgr2=0x7fcb601048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:07.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.506+0000 7fcb5f59e640 1 --2- 172.21.15.5:0/2229377484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb601024e0 0x7fcb601048d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009900 tx=0x7fcb5002a410 comp rx=0 tx=0).stop 2024-09-19T05:58:07.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.506+0000 7fcb5f59e640 1 -- 172.21.15.5:0/2229377484 shutdown_connections 2024-09-19T05:58:07.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.506+0000 7fcb5f59e640 1 --2- 172.21.15.5:0/2229377484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60107740 0x7fcb60109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:07.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.506+0000 7fcb5f59e640 1 --2- 172.21.15.5:0/2229377484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb60104e10 0x7fcb60107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:07.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.506+0000 7fcb5f59e640 1 --2- 172.21.15.5:0/2229377484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb601024e0 0x7fcb601048d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:07.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.506+0000 7fcb5f59e640 1 -- 172.21.15.5:0/2229377484 >> 172.21.15.5:0/2229377484 conn(0x7fcb600fc0e0 msgr2=0x7fcb600fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:07.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.506+0000 7fcb5f59e640 1 -- 172.21.15.5:0/2229377484 shutdown_connections 2024-09-19T05:58:07.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.506+0000 7fcb5f59e640 1 -- 172.21.15.5:0/2229377484 wait complete. 2024-09-19T05:58:07.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.507+0000 7fcb5f59e640 1 Processor -- start 2024-09-19T05:58:07.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.507+0000 7fcb5f59e640 1 -- start start 2024-09-19T05:58:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.507+0000 7fcb5f59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb601024e0 0x7fcb60101e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.507+0000 7fcb5f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60104e10 0x7fcb60100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.507+0000 7fcb5f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60107740 0x7fcb601009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.507+0000 7fcb5f59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6010d610 con 0x7fcb60104e10 2024-09-19T05:58:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.507+0000 7fcb5f59e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6010d490 con 0x7fcb601024e0 2024-09-19T05:58:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.507+0000 7fcb5f59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6010d790 con 0x7fcb60107740 2024-09-19T05:58:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.507+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60104e10 0x7fcb60100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.507+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60104e10 0x7fcb60100490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50620/0 (socket says 172.21.15.5:50620) 2024-09-19T05:58:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.507+0000 7fcb57fff640 1 -- 172.21.15.5:0/3250641766 learned_addr learned my addr 172.21.15.5:0/3250641766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:58:07.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.508+0000 7fcb5ed9d640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60107740 0x7fcb601009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:07.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.508+0000 7fcb57fff640 1 -- 172.21.15.5:0/3250641766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60107740 msgr2=0x7fcb601009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:07.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.508+0000 7fcb57fff640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60107740 0x7fcb601009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:07.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.508+0000 7fcb57fff640 1 -- 172.21.15.5:0/3250641766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb601024e0 msgr2=0x7fcb60101e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:07.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.508+0000 7fcb57fff640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb601024e0 0x7fcb60101e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:07.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.508+0000 7fcb57fff640 1 -- 172.21.15.5:0/3250641766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb60101290 con 0x7fcb60104e10 2024-09-19T05:58:07.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.508+0000 7fcb57fff640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60104e10 0x7fcb60100490 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcb48005fd0 tx=0x7fcb4800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:07.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.508+0000 7fcb577fe640 1 -- 172.21.15.5:0/3250641766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4801c070 con 0x7fcb60104e10 2024-09-19T05:58:07.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.508+0000 7fcb5f59e640 1 -- 172.21.15.5:0/3250641766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb60197980 con 0x7fcb60104e10 2024-09-19T05:58:07.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.508+0000 7fcb577fe640 1 -- 172.21.15.5:0/3250641766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb48009820 con 0x7fcb60104e10 2024-09-19T05:58:07.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.508+0000 7fcb5f59e640 1 -- 172.21.15.5:0/3250641766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb60197f10 con 0x7fcb60104e10 2024-09-19T05:58:07.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.508+0000 7fcb577fe640 1 -- 172.21.15.5:0/3250641766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb480038e0 con 0x7fcb60104e10 2024-09-19T05:58:07.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.510+0000 7fcb5f59e640 1 -- 172.21.15.5:0/3250641766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb600fdbd0 con 0x7fcb60104e10 2024-09-19T05:58:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.511+0000 7fcb577fe640 1 -- 172.21.15.5:0/3250641766 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fcb480099d0 con 0x7fcb60104e10 2024-09-19T05:58:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.511+0000 7fcb577fe640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcb4c0a3240 0x7fcb4c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.512+0000 7fcb577fe640 1 -- 172.21.15.5:0/3250641766 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4237+0+0 (secure 0 0 0) 0x7fcb480ce1e0 con 0x7fcb60104e10 2024-09-19T05:58:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.512+0000 7fcb5e59c640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcb4c0a3240 0x7fcb4c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.512+0000 7fcb5e59c640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcb4c0a3240 0x7fcb4c0a5700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb5002a930 tx=0x7fcb500314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:07.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.515+0000 7fcb577fe640 1 -- 172.21.15.5:0/3250641766 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcb48024020 con 0x7fcb60104e10 2024-09-19T05:58:07.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:07.703+0000 7fcb5f59e640 1 -- 172.21.15.5:0/3250641766 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fcb60062630 con 0x7fcb4c0a3240 2024-09-19T05:58:08.123 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:07 smithi005 ceph-mon[30955]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:08.123 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:07 smithi005 ceph-mon[30955]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T05:58:08.123 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:07 smithi005 ceph-mon[30955]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T05:58:08.123 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:07 smithi005 ceph-mon[30955]: from='osd.5 [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:58:08.123 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:58:08.123 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:07 smithi005 ceph-mon[30955]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:58:08.123 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:07 smithi005 ceph-mon[30955]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:58:08.123 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:58:08.123 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:58:08.123 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:07 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:08.123 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:07 smithi005 ceph-mon[30955]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T05:58:08.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:07 smithi049 ceph-mon[34640]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:08.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:07 smithi049 ceph-mon[34640]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T05:58:08.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:07 smithi049 ceph-mon[34640]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T05:58:08.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:07 smithi049 ceph-mon[34640]: from='osd.5 [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:58:08.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:07 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:58:08.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:07 smithi049 ceph-mon[34640]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:58:08.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:07 smithi049 ceph-mon[34640]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:58:08.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:07 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:58:08.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:07 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:58:08.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:07 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:08.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:07 smithi049 ceph-mon[34640]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T05:58:08.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T05:58:07.819+0000 7fdd28411640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T05:58:08.302 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T05:58:07.828+0000 7fdd24a4b640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:58:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-mon[35346]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-mon[35346]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T05:58:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-mon[35346]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T05:58:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-mon[35346]: from='osd.5 [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:58:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:58:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-mon[35346]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:58:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-mon[35346]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:58:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:58:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:58:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:08.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:07 smithi157 ceph-mon[35346]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T05:58:09.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:08 smithi005 ceph-mon[30955]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T05:58:09.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:58:09.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:58:09.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:08 smithi005 ceph-mon[30955]: osd.5 [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971] boot 2024-09-19T05:58:09.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:08 smithi005 ceph-mon[30955]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T05:58:09.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:58:09.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:08 smithi049 ceph-mon[34640]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T05:58:09.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:58:09.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:58:09.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:08 smithi049 ceph-mon[34640]: osd.5 [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971] boot 2024-09-19T05:58:09.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:08 smithi049 ceph-mon[34640]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T05:58:09.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:58:09.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:08 smithi157 ceph-mon[35346]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T05:58:09.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:58:09.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:58:09.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:08 smithi157 ceph-mon[35346]: osd.5 [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971] boot 2024-09-19T05:58:09.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:08 smithi157 ceph-mon[35346]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T05:58:09.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:58:10.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:10 smithi005 ceph-mon[30955]: purged_snaps scrub starts 2024-09-19T05:58:10.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:10 smithi005 ceph-mon[30955]: purged_snaps scrub ok 2024-09-19T05:58:10.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:10 smithi005 ceph-mon[30955]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:10.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:10 smithi005 ceph-mon[30955]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T05:58:10.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:10 smithi157 ceph-mon[35346]: purged_snaps scrub starts 2024-09-19T05:58:10.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:10 smithi157 ceph-mon[35346]: purged_snaps scrub ok 2024-09-19T05:58:10.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:10 smithi157 ceph-mon[35346]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:10.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:10 smithi157 ceph-mon[35346]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T05:58:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:10 smithi049 ceph-mon[34640]: purged_snaps scrub starts 2024-09-19T05:58:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:10 smithi049 ceph-mon[34640]: purged_snaps scrub ok 2024-09-19T05:58:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:10 smithi049 ceph-mon[34640]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:58:10.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:10 smithi049 ceph-mon[34640]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T05:58:11.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:11 smithi005 ceph-mon[30955]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T05:58:11.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:11 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1050189814' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbef0e08-d6b0-4515-a2ee-2c81b048583b"}]: dispatch 2024-09-19T05:58:11.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:11 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1050189814' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbef0e08-d6b0-4515-a2ee-2c81b048583b"}]': finished 2024-09-19T05:58:11.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:11 smithi005 ceph-mon[30955]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T05:58:11.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:11 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:11.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:11 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/900574623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:58:11.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:11 smithi157 ceph-mon[35346]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T05:58:11.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:11 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1050189814' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbef0e08-d6b0-4515-a2ee-2c81b048583b"}]: dispatch 2024-09-19T05:58:11.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:11 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1050189814' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbef0e08-d6b0-4515-a2ee-2c81b048583b"}]': finished 2024-09-19T05:58:11.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:11 smithi157 ceph-mon[35346]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T05:58:11.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:11 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:11.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:11 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/900574623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:58:11.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:11 smithi049 ceph-mon[34640]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T05:58:11.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:11 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1050189814' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbef0e08-d6b0-4515-a2ee-2c81b048583b"}]: dispatch 2024-09-19T05:58:11.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:11 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1050189814' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbef0e08-d6b0-4515-a2ee-2c81b048583b"}]': finished 2024-09-19T05:58:11.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:11 smithi049 ceph-mon[34640]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T05:58:11.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:11 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:11.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:11 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/900574623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:58:12.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:12 smithi005 ceph-mon[30955]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:12.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:12 smithi005 ceph-mon[30955]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T05:58:12.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:12 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:12.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:12 smithi049 ceph-mon[34640]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:12.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:12 smithi049 ceph-mon[34640]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T05:58:12.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:12 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:12 smithi157 ceph-mon[35346]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:12 smithi157 ceph-mon[35346]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T05:58:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:12 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:14.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:14 smithi005 ceph-mon[30955]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:14 smithi049 ceph-mon[34640]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:14.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:14 smithi157 ceph-mon[35346]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:16.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:16 smithi005 ceph-mon[30955]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-19T05:58:16.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:16 smithi049 ceph-mon[34640]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-19T05:58:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:16 smithi157 ceph-mon[35346]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-19T05:58:18.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:18 smithi005 ceph-mon[30955]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T05:58:18.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:18 smithi049 ceph-mon[34640]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T05:58:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:18 smithi157 ceph-mon[35346]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T05:58:19.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:19 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T05:58:19.469 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:19 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:19.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:19 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T05:58:19.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:19 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:19.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:19 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T05:58:19.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:19 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:20 smithi005 ceph-mon[30955]: Deploying daemon osd.6 on smithi005 2024-09-19T05:58:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:20 smithi005 ceph-mon[30955]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-19T05:58:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:20 smithi049 ceph-mon[34640]: Deploying daemon osd.6 on smithi005 2024-09-19T05:58:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:20 smithi049 ceph-mon[34640]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-19T05:58:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:20 smithi157 ceph-mon[35346]: Deploying daemon osd.6 on smithi005 2024-09-19T05:58:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:20 smithi157 ceph-mon[35346]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-19T05:58:22.551 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:22 smithi005 ceph-mon[30955]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T05:58:22.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:22 smithi049 ceph-mon[34640]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T05:58:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:22 smithi157 ceph-mon[35346]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T05:58:24.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:24 smithi005 ceph-mon[30955]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T05:58:24.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:24 smithi049 ceph-mon[34640]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T05:58:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:24 smithi157 ceph-mon[35346]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T05:58:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:25 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:25 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:25 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:25.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:25 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:25.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:25 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:25.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:25 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:25 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:25 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:25 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:26.439 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:26 smithi005 ceph-mon[30955]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:58:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:26 smithi049 ceph-mon[34640]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:58:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:26 smithi157 ceph-mon[35346]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:58:28.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:28 smithi005 ceph-mon[30955]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:28.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:28 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:28.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:28 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:28.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:28 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:28.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:28 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:28.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:28 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:28.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:28 smithi049 ceph-mon[34640]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:28.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:28 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:28.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:28 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:28.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:28 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:28.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:28 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:28.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:28 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:28 smithi157 ceph-mon[35346]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:28 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:28 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:28 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:28 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:28 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.857+0000 7fcb577fe640 1 -- 172.21.15.5:0/3250641766 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcb60062630 con 0x7fcb4c0a3240 2024-09-19T05:58:28.860 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 6 on host 'smithi005' 2024-09-19T05:58:28.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.860+0000 7fcb5f59e640 1 -- 172.21.15.5:0/3250641766 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcb4c0a3240 msgr2=0x7fcb4c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:28.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.860+0000 7fcb5f59e640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcb4c0a3240 0x7fcb4c0a5700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb5002a930 tx=0x7fcb500314a0 comp rx=0 tx=0).stop 2024-09-19T05:58:28.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.860+0000 7fcb5f59e640 1 -- 172.21.15.5:0/3250641766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60104e10 msgr2=0x7fcb60100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:28.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.860+0000 7fcb5f59e640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60104e10 0x7fcb60100490 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcb48005fd0 tx=0x7fcb4800dd20 comp rx=0 tx=0).stop 2024-09-19T05:58:28.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.860+0000 7fcb5f59e640 1 -- 172.21.15.5:0/3250641766 shutdown_connections 2024-09-19T05:58:28.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.860+0000 7fcb5f59e640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcb4c0a3240 0x7fcb4c0a5700 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:28.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.860+0000 7fcb5f59e640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60107740 0x7fcb601009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:28.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.860+0000 7fcb5f59e640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60104e10 0x7fcb60100490 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:28.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.860+0000 7fcb5f59e640 1 --2- 172.21.15.5:0/3250641766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb601024e0 0x7fcb60101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:28.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.860+0000 7fcb5f59e640 1 -- 172.21.15.5:0/3250641766 >> 172.21.15.5:0/3250641766 conn(0x7fcb600fc0e0 msgr2=0x7fcb60102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:28.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.860+0000 7fcb5f59e640 1 -- 172.21.15.5:0/3250641766 shutdown_connections 2024-09-19T05:58:28.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:58:28.860+0000 7fcb5f59e640 1 -- 172.21.15.5:0/3250641766 wait complete. 2024-09-19T05:58:29.445 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:29.445 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:29.446 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:29 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:29.473 DEBUG:teuthology.orchestra.run.smithi005:osd.6> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T05:58:29.476 INFO:tasks.cephadm:Deploying osd.7 on smithi049 with /dev/vg_nvme/lv_2... 2024-09-19T05:58:29.476 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T05:58:29.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:29.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:29.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:29 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:29.741 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:58:29.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:29.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:29.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:29 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:30.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:30 smithi049 ceph-mon[34640]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:30.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:30 smithi005 ceph-mon[30955]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:30 smithi157 ceph-mon[35346]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:31.351 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 05:58:31 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T05:58:31.159+0000 7fa34998f640 -1 osd.6 0 log_to_monitors true 2024-09-19T05:58:31.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:31 smithi005 ceph-mon[30955]: from='osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T05:58:31.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:31 smithi049 ceph-mon[34640]: from='osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T05:58:31.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:31 smithi157 ceph-mon[35346]: from='osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T05:58:32.604 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:32 smithi049 ceph-mon[34640]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:32.604 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:32 smithi049 ceph-mon[34640]: from='osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T05:58:32.604 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:32 smithi049 ceph-mon[34640]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T05:58:32.604 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:32 smithi049 ceph-mon[34640]: from='osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:58:32.604 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:32 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:32.607 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:58:32.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:32 smithi005 ceph-mon[30955]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:32.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:32 smithi005 ceph-mon[30955]: from='osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T05:58:32.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:32 smithi005 ceph-mon[30955]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T05:58:32.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:32 smithi005 ceph-mon[30955]: from='osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:58:32.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:32 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:32.630 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 05:58:32 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T05:58:32.362+0000 7fa3450ef640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T05:58:32.630 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 05:58:32 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T05:58:32.371+0000 7fa341729640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:58:32.633 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_2 2024-09-19T05:58:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:32 smithi157 ceph-mon[35346]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:32 smithi157 ceph-mon[35346]: from='osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T05:58:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:32 smithi157 ceph-mon[35346]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T05:58:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:32 smithi157 ceph-mon[35346]: from='osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:58:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:32 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:32.906 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:58:33.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.418+0000 7f71d992a640 1 -- 172.21.15.49:0/3825446960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d41026a0 msgr2=0x7f71d4104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.418+0000 7f71d992a640 1 --2- 172.21.15.49:0/3825446960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d41026a0 0x7f71d4104a90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009900 tx=0x7f71c402a410 comp rx=0 tx=0).stop 2024-09-19T05:58:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.418+0000 7f71d992a640 1 -- 172.21.15.49:0/3825446960 shutdown_connections 2024-09-19T05:58:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.418+0000 7f71d992a640 1 --2- 172.21.15.49:0/3825446960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d4107900 0x7f71d4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.418+0000 7f71d992a640 1 --2- 172.21.15.49:0/3825446960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d4104fd0 0x7f71d41073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.418+0000 7f71d992a640 1 --2- 172.21.15.49:0/3825446960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d41026a0 0x7f71d4104a90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.418+0000 7f71d992a640 1 -- 172.21.15.49:0/3825446960 >> 172.21.15.49:0/3825446960 conn(0x7f71d40fc2a0 msgr2=0x7f71d40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.418+0000 7f71d992a640 1 -- 172.21.15.49:0/3825446960 shutdown_connections 2024-09-19T05:58:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.418+0000 7f71d992a640 1 -- 172.21.15.49:0/3825446960 wait complete. 2024-09-19T05:58:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.419+0000 7f71d992a640 1 Processor -- start 2024-09-19T05:58:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.419+0000 7f71d992a640 1 -- start start 2024-09-19T05:58:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.419+0000 7f71d992a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d41026a0 0x7f71d4197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.419+0000 7f71d992a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d4104fd0 0x7f71d4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.419+0000 7f71d992a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d4107900 0x7f71d419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.419+0000 7f71d992a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71d410d780 con 0x7f71d4107900 2024-09-19T05:58:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.419+0000 7f71d992a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f71d410d600 con 0x7f71d4104fd0 2024-09-19T05:58:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.419+0000 7f71d992a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71d410d900 con 0x7f71d41026a0 2024-09-19T05:58:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.419+0000 7f71d27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d4104fd0 0x7f71d4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.419+0000 7f71d27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d4104fd0 0x7f71d4198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33592/0 (socket says 172.21.15.49:33592) 2024-09-19T05:58:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.419+0000 7f71d27fc640 1 -- 172.21.15.49:0/3767700912 learned_addr learned my addr 172.21.15.49:0/3767700912 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-19T05:58:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d2ffd640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d41026a0 0x7f71d4197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d37fe640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d4107900 0x7f71d419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d27fc640 1 -- 172.21.15.49:0/3767700912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d41026a0 msgr2=0x7f71d4197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d27fc640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d41026a0 0x7f71d4197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d27fc640 1 -- 172.21.15.49:0/3767700912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d4107900 msgr2=0x7f71d419f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d27fc640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d4107900 0x7f71d419f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d27fc640 1 -- 172.21.15.49:0/3767700912 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d419faf0 con 0x7f71d4104fd0 2024-09-19T05:58:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d27fc640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d4104fd0 0x7f71d4198400 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e9f0 tx=0x7f71c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d8928640 1 -- 172.21.15.49:0/3767700912 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c800ccc0 con 0x7f71d4104fd0 2024-09-19T05:58:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d8928640 1 -- 172.21.15.49:0/3767700912 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71c80054d0 con 0x7f71d4104fd0 2024-09-19T05:58:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d8928640 1 -- 172.21.15.49:0/3767700912 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8010430 con 0x7f71d4104fd0 2024-09-19T05:58:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d992a640 1 -- 172.21.15.49:0/3767700912 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d419fde0 con 0x7f71d4104fd0 2024-09-19T05:58:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.420+0000 7f71d992a640 1 -- 172.21.15.49:0/3767700912 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d41a0240 con 0x7f71d4104fd0 2024-09-19T05:58:33.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.422+0000 7f71d992a640 1 -- 172.21.15.49:0/3767700912 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d40fdd90 con 0x7f71d4104fd0 2024-09-19T05:58:33.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.422+0000 7f71d8928640 1 -- 172.21.15.49:0/3767700912 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f71c8005020 con 0x7f71d4104fd0 2024-09-19T05:58:33.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.423+0000 7f71d8928640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f71bc0a32f0 0x7f71bc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:33.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.423+0000 7f71d8928640 1 -- 172.21.15.49:0/3767700912 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f71c80ceed0 con 0x7f71d4104fd0 2024-09-19T05:58:33.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.423+0000 7f71d2ffd640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f71bc0a32f0 0x7f71bc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:33.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.424+0000 7f71d2ffd640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f71bc0a32f0 0x7f71bc0a57b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f71c4006010 tx=0x7f71c4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:33.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.427+0000 7f71d8928640 1 -- 172.21.15.49:0/3767700912 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f71c809c900 con 0x7f71d4104fd0 2024-09-19T05:58:33.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:33.613+0000 7f71d992a640 1 -- 172.21.15.49:0/3767700912 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f71d4062630 con 0x7f71bc0a32f0 2024-09-19T05:58:33.615 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: from='osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-19T05:58:33.615 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T05:58:33.615 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:33.615 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:33.615 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: Detected new or changed devices on smithi005 2024-09-19T05:58:33.615 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:33.616 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:33.616 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:33.616 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:33.616 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:33.616 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi005 to 5697M 2024-09-19T05:58:33.616 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:33.616 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:33.616 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:33.616 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:33.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: from='osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-19T05:58:33.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T05:58:33.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:33.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:33.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: Detected new or changed devices on smithi005 2024-09-19T05:58:33.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:33.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:33.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:33.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:33.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:33.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi005 to 5697M 2024-09-19T05:58:33.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:33.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:33.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:33.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: from='osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-19T05:58:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: Detected new or changed devices on smithi005 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi005 to 5697M 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:34 smithi005 ceph-mon[30955]: purged_snaps scrub starts 2024-09-19T05:58:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:34 smithi005 ceph-mon[30955]: purged_snaps scrub ok 2024-09-19T05:58:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:34 smithi005 ceph-mon[30955]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:34.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:34 smithi005 ceph-mon[30955]: osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194] boot 2024-09-19T05:58:34.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:34 smithi005 ceph-mon[30955]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T05:58:34.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:34.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:34 smithi005 ceph-mon[30955]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:58:34.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:58:34.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:58:34.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:34.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:34 smithi049 ceph-mon[34640]: purged_snaps scrub starts 2024-09-19T05:58:34.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:34 smithi049 ceph-mon[34640]: purged_snaps scrub ok 2024-09-19T05:58:34.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:34 smithi049 ceph-mon[34640]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:34.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:34 smithi049 ceph-mon[34640]: osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194] boot 2024-09-19T05:58:34.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:34 smithi049 ceph-mon[34640]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T05:58:34.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:34.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:34 smithi049 ceph-mon[34640]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:58:34.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:58:34.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:58:34.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:34 smithi157 ceph-mon[35346]: purged_snaps scrub starts 2024-09-19T05:58:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:34 smithi157 ceph-mon[35346]: purged_snaps scrub ok 2024-09-19T05:58:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:34 smithi157 ceph-mon[35346]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:58:34.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:34 smithi157 ceph-mon[35346]: osd.6 [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194] boot 2024-09-19T05:58:34.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:34 smithi157 ceph-mon[35346]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T05:58:34.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:58:34.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:34 smithi157 ceph-mon[35346]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:58:34.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:58:34.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:58:34.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:35.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:35 smithi049 ceph-mon[34640]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T05:58:35.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:35 smithi157 ceph-mon[35346]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T05:58:35.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:35 smithi005 ceph-mon[30955]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T05:58:36.616 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:36 smithi049 ceph-mon[34640]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:36.617 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:36 smithi049 ceph-mon[34640]: from='client.? 172.21.15.49:0/3132168284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d261ac55-532d-4aea-9b3e-ed2b4eeb1da6"}]: dispatch 2024-09-19T05:58:36.617 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:36 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d261ac55-532d-4aea-9b3e-ed2b4eeb1da6"}]: dispatch 2024-09-19T05:58:36.617 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:36 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d261ac55-532d-4aea-9b3e-ed2b4eeb1da6"}]': finished 2024-09-19T05:58:36.617 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:36 smithi049 ceph-mon[34640]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T05:58:36.617 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:36 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:36 smithi157 ceph-mon[35346]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:36 smithi157 ceph-mon[35346]: from='client.? 172.21.15.49:0/3132168284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d261ac55-532d-4aea-9b3e-ed2b4eeb1da6"}]: dispatch 2024-09-19T05:58:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:36 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d261ac55-532d-4aea-9b3e-ed2b4eeb1da6"}]: dispatch 2024-09-19T05:58:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:36 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d261ac55-532d-4aea-9b3e-ed2b4eeb1da6"}]': finished 2024-09-19T05:58:36.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:36 smithi157 ceph-mon[35346]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T05:58:36.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:36 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:36 smithi005 ceph-mon[30955]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:36 smithi005 ceph-mon[30955]: from='client.? 172.21.15.49:0/3132168284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d261ac55-532d-4aea-9b3e-ed2b4eeb1da6"}]: dispatch 2024-09-19T05:58:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:36 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d261ac55-532d-4aea-9b3e-ed2b4eeb1da6"}]: dispatch 2024-09-19T05:58:36.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:36 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d261ac55-532d-4aea-9b3e-ed2b4eeb1da6"}]': finished 2024-09-19T05:58:36.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:36 smithi005 ceph-mon[30955]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T05:58:36.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:36 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:37 smithi157 ceph-mon[35346]: from='client.? 172.21.15.49:0/2065889587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:58:37.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:37 smithi005 ceph-mon[30955]: from='client.? 172.21.15.49:0/2065889587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:58:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:37 smithi049 ceph-mon[34640]: from='client.? 172.21.15.49:0/2065889587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:58:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:38 smithi157 ceph-mon[35346]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:38 smithi005 ceph-mon[30955]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:38 smithi049 ceph-mon[34640]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:40 smithi157 ceph-mon[35346]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:40.866 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:40 smithi049 ceph-mon[34640]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:40 smithi005 ceph-mon[30955]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:42.567 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:42 smithi049 ceph-mon[34640]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:42 smithi157 ceph-mon[35346]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:42 smithi005 ceph-mon[30955]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:44.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:44 smithi005 ceph-mon[30955]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:44.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:44 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T05:58:44.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:44 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:44.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:44 smithi049 ceph-mon[34640]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:44.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:44 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T05:58:44.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:44 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:44 smithi157 ceph-mon[35346]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:44 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T05:58:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:44 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:45.692 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:45 smithi049 ceph-mon[34640]: Deploying daemon osd.7 on smithi049 2024-09-19T05:58:45.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:45 smithi157 ceph-mon[35346]: Deploying daemon osd.7 on smithi049 2024-09-19T05:58:45.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:45 smithi005 ceph-mon[30955]: Deploying daemon osd.7 on smithi049 2024-09-19T05:58:46.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:46 smithi049 ceph-mon[34640]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:46 smithi157 ceph-mon[35346]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:46 smithi005 ceph-mon[30955]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:48.705 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:48 smithi049 ceph-mon[34640]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:48.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:48 smithi157 ceph-mon[35346]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:48.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:48 smithi005 ceph-mon[30955]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:50.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:50 smithi049 ceph-mon[34640]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:50.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:50 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:50.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:50 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:50.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:50 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:50 smithi157 ceph-mon[35346]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:50 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:50 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:50 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:50 smithi005 ceph-mon[30955]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:50 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:50 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:50 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:52.641 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:52 smithi049 ceph-mon[34640]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:52 smithi157 ceph-mon[35346]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:52 smithi005 ceph-mon[30955]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:53.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.885+0000 7f71d8928640 1 -- 172.21.15.49:0/3767700912 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f71d4062630 con 0x7f71bc0a32f0 2024-09-19T05:58:53.887 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 7 on host 'smithi049' 2024-09-19T05:58:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.887+0000 7f71d992a640 1 -- 172.21.15.49:0/3767700912 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f71bc0a32f0 msgr2=0x7f71bc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.887+0000 7f71d992a640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f71bc0a32f0 0x7f71bc0a57b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f71c4006010 tx=0x7f71c4000bc0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.887+0000 7f71d992a640 1 -- 172.21.15.49:0/3767700912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d4104fd0 msgr2=0x7f71d4198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.887+0000 7f71d992a640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d4104fd0 0x7f71d4198400 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e9f0 tx=0x7f71c800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.887+0000 7f71d992a640 1 -- 172.21.15.49:0/3767700912 shutdown_connections 2024-09-19T05:58:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.887+0000 7f71d992a640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f71bc0a32f0 0x7f71bc0a57b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.887+0000 7f71d992a640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d4107900 0x7f71d419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.887+0000 7f71d992a640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d4104fd0 0x7f71d4198400 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.887+0000 7f71d992a640 1 --2- 172.21.15.49:0/3767700912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d41026a0 0x7f71d4197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.887+0000 7f71d992a640 1 -- 172.21.15.49:0/3767700912 >> 172.21.15.49:0/3767700912 conn(0x7f71d40fc2a0 msgr2=0x7f71d4103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:53.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.887+0000 7f71d992a640 1 -- 172.21.15.49:0/3767700912 shutdown_connections 2024-09-19T05:58:53.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:58:53.887+0000 7f71d992a640 1 -- 172.21.15.49:0/3767700912 wait complete. 2024-09-19T05:58:54.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:53 smithi049 ceph-mon[34640]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:54.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:54.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:54.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:54.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:53 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:53 smithi157 ceph-mon[35346]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:54.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:54.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:54.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:54.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:53 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:53 smithi005 ceph-mon[30955]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:54.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:54.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:54.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:54.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:53 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:54.542 DEBUG:teuthology.orchestra.run.smithi049:osd.7> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T05:58:54.544 INFO:tasks.cephadm:Deploying osd.8 on smithi157 with /dev/vg_nvme/lv_2... 2024-09-19T05:58:54.545 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T05:58:54.798 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:58:55.965 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:55 smithi049 ceph-mon[34640]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:56.177 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:55 smithi157 ceph-mon[35346]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:56.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:55 smithi005 ceph-mon[30955]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:56.647 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 05:58:56 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T05:58:56.332+0000 7fb38e05b640 -1 osd.7 0 log_to_monitors true 2024-09-19T05:58:57.061 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:56 smithi049 ceph-mon[34640]: from='osd.7 [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:58:57.061 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:56 smithi049 ceph-mon[34640]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:58:57.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:56 smithi157 ceph-mon[35346]: from='osd.7 [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:58:57.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:56 smithi157 ceph-mon[35346]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:58:57.293 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T05:58:57.319 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_2 2024-09-19T05:58:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:56 smithi005 ceph-mon[30955]: from='osd.7 [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:58:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:56 smithi005 ceph-mon[30955]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:58:57.570 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:58:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.005+0000 7f7913f70640 1 -- 172.21.15.157:0/503453891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c101cd0 msgr2=0x7f790c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.005+0000 7f7913f70640 1 --2- 172.21.15.157:0/503453891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c101cd0 0x7f790c102150 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009900 tx=0x7f78fc02a410 comp rx=0 tx=0).stop 2024-09-19T05:58:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.005+0000 7f7913f70640 1 -- 172.21.15.157:0/503453891 shutdown_connections 2024-09-19T05:58:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.005+0000 7f7913f70640 1 --2- 172.21.15.157:0/503453891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f790c102690 0x7f790c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.005+0000 7f7913f70640 1 --2- 172.21.15.157:0/503453891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c101cd0 0x7f790c102150 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.005+0000 7f7913f70640 1 --2- 172.21.15.157:0/503453891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f790c100ad0 0x7f790c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.005+0000 7f7913f70640 1 -- 172.21.15.157:0/503453891 >> 172.21.15.157:0/503453891 conn(0x7f790c0fc280 msgr2=0x7f790c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.005+0000 7f7913f70640 1 -- 172.21.15.157:0/503453891 shutdown_connections 2024-09-19T05:58:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.005+0000 7f7913f70640 1 -- 172.21.15.157:0/503453891 wait complete. 2024-09-19T05:58:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f7913f70640 1 Processor -- start 2024-09-19T05:58:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f7913f70640 1 -- start start 2024-09-19T05:58:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f7913f70640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c100ad0 0x7f790c104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f7913f70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f790c101cd0 0x7f790c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f7913f70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f790c102690 0x7f790c1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f7913f70640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f790c10c600 con 0x7f790c101cd0 2024-09-19T05:58:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f7913f70640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f790c10c480 con 0x7f790c102690 2024-09-19T05:58:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f7913f70640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f790c10c780 con 0x7f790c100ad0 2024-09-19T05:58:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f7911d0c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c100ad0 0x7f790c104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f7911d0c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c100ad0 0x7f790c104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49290/0 (socket says 172.21.15.157:49290) 2024-09-19T05:58:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f7911d0c640 1 -- 172.21.15.157:0/2641750396 learned_addr learned my addr 172.21.15.157:0/2641750396 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T05:58:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f791150b640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f790c101cd0 0x7f790c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.006+0000 7f791250d640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f790c102690 0x7f790c1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.007+0000 7f7911d0c640 1 -- 172.21.15.157:0/2641750396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f790c102690 msgr2=0x7f790c1035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.007+0000 7f7911d0c640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f790c102690 0x7f790c1035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.007+0000 7f7911d0c640 1 -- 172.21.15.157:0/2641750396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f790c101cd0 msgr2=0x7f790c1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.007+0000 7f7911d0c640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f790c101cd0 0x7f790c1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.007+0000 7f7911d0c640 1 -- 172.21.15.157:0/2641750396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c103eb0 con 0x7f790c100ad0 2024-09-19T05:58:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.007+0000 7f7911d0c640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c100ad0 0x7f790c104a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f790800d950 tx=0x7f790800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.007+0000 7f7902ffd640 1 -- 172.21.15.157:0/2641750396 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908017070 con 0x7f790c100ad0 2024-09-19T05:58:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.007+0000 7f7902ffd640 1 -- 172.21.15.157:0/2641750396 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7908009c30 con 0x7f790c100ad0 2024-09-19T05:58:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.007+0000 7f7913f70640 1 -- 172.21.15.157:0/2641750396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790c1a1fd0 con 0x7f790c100ad0 2024-09-19T05:58:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.007+0000 7f7902ffd640 1 -- 172.21.15.157:0/2641750396 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790800f620 con 0x7f790c100ad0 2024-09-19T05:58:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.007+0000 7f7913f70640 1 -- 172.21.15.157:0/2641750396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790c1a2370 con 0x7f790c100ad0 2024-09-19T05:58:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.008+0000 7f7913f70640 1 -- 172.21.15.157:0/2641750396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f790c1a25a0 con 0x7f790c100ad0 2024-09-19T05:58:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.010+0000 7f7902ffd640 1 -- 172.21.15.157:0/2641750396 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f790801a070 con 0x7f790c100ad0 2024-09-19T05:58:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.010+0000 7f7902ffd640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f78f80a3240 0x7f78f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.011+0000 7f7902ffd640 1 -- 172.21.15.157:0/2641750396 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5117+0+0 (secure 0 0 0) 0x7f79080ce620 con 0x7f790c100ad0 2024-09-19T05:58:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.011+0000 7f791150b640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f78f80a3240 0x7f78f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.011+0000 7f791150b640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f78f80a3240 0x7f78f80a5700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f78fc006010 tx=0x7f78fc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.013+0000 7f7902ffd640 1 -- 172.21.15.157:0/2641750396 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7908009de0 con 0x7f790c100ad0 2024-09-19T05:58:58.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:58:58.195+0000 7f7913f70640 1 -- 172.21.15.157:0/2641750396 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f790c06a6d0 con 0x7f78f80a3240 2024-09-19T05:58:58.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:58.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T05:58:58.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='osd.7 [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:58:58.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T05:58:58.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:58.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:58:58.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: Detected new or changed devices on smithi049 2024-09-19T05:58:58.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:58.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:58.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:58.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:58.198 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:58.198 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi049 to 5696M 2024-09-19T05:58:58.198 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:58.198 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:58.198 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:58.198 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:57 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:58.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:58.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T05:58:58.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='osd.7 [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:58:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T05:58:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:58:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: Detected new or changed devices on smithi049 2024-09-19T05:58:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi049 to 5696M 2024-09-19T05:58:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:58.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:58.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:57 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:58.396 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T05:58:57.946+0000 7fb38a7f2640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T05:58:58.396 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T05:58:57.957+0000 7fb385df5640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:58:58.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:58:58.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T05:58:58.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='osd.7 [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:58:58.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T05:58:58.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:58.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T05:58:58.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: Detected new or changed devices on smithi049 2024-09-19T05:58:58.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:58.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:58.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:58.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:58.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:58.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi049 to 5696M 2024-09-19T05:58:58.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:58.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:58.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:58.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:57 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:58:59.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:58 smithi157 ceph-mon[35346]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-19T05:58:59.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:58 smithi157 ceph-mon[35346]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T05:58:59.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:59.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:59.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:58 smithi157 ceph-mon[35346]: osd.7 [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400] boot 2024-09-19T05:58:59.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:58 smithi157 ceph-mon[35346]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T05:58:59.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:59.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:58 smithi157 ceph-mon[35346]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:58:59.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:58:59.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:58:59.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:58:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:59.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:58 smithi005 ceph-mon[30955]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-19T05:58:59.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:58 smithi005 ceph-mon[30955]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T05:58:59.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:59.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:59.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:58 smithi005 ceph-mon[30955]: osd.7 [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400] boot 2024-09-19T05:58:59.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:58 smithi005 ceph-mon[30955]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T05:58:59.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:59.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:58 smithi005 ceph-mon[30955]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:58:59.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:58:59.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:58:59.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:58:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:59.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:58 smithi049 ceph-mon[34640]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-19T05:58:59.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:58 smithi049 ceph-mon[34640]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T05:58:59.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:59.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:59.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:58 smithi049 ceph-mon[34640]: osd.7 [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400] boot 2024-09-19T05:58:59.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:58 smithi049 ceph-mon[34640]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T05:58:59.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:58:59.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:58 smithi049 ceph-mon[34640]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:58:59.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:58:59.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:58:59.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:58:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:00 smithi157 ceph-mon[35346]: purged_snaps scrub starts 2024-09-19T05:59:00.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:00 smithi157 ceph-mon[35346]: purged_snaps scrub ok 2024-09-19T05:59:00.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:00 smithi157 ceph-mon[35346]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:59:00.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:00 smithi157 ceph-mon[35346]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T05:59:00.425 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:00 smithi049 ceph-mon[34640]: purged_snaps scrub starts 2024-09-19T05:59:00.425 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:00 smithi049 ceph-mon[34640]: purged_snaps scrub ok 2024-09-19T05:59:00.425 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:00 smithi049 ceph-mon[34640]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:59:00.425 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:00 smithi049 ceph-mon[34640]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T05:59:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:00 smithi005 ceph-mon[30955]: purged_snaps scrub starts 2024-09-19T05:59:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:00 smithi005 ceph-mon[30955]: purged_snaps scrub ok 2024-09-19T05:59:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:00 smithi005 ceph-mon[30955]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:59:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:00 smithi005 ceph-mon[30955]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T05:59:01.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:01 smithi157 ceph-mon[35346]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T05:59:01.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:01 smithi157 ceph-mon[35346]: from='client.? 172.21.15.157:0/2979994408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15d0eaf7-d604-4546-b393-4d6d39018f42"}]: dispatch 2024-09-19T05:59:01.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:01 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15d0eaf7-d604-4546-b393-4d6d39018f42"}]: dispatch 2024-09-19T05:59:01.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:01 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15d0eaf7-d604-4546-b393-4d6d39018f42"}]': finished 2024-09-19T05:59:01.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:01 smithi157 ceph-mon[35346]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T05:59:01.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:01 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:01.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:01 smithi157 ceph-mon[35346]: from='client.? 172.21.15.157:0/434053972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:59:01.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:01 smithi005 ceph-mon[30955]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T05:59:01.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:01 smithi005 ceph-mon[30955]: from='client.? 172.21.15.157:0/2979994408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15d0eaf7-d604-4546-b393-4d6d39018f42"}]: dispatch 2024-09-19T05:59:01.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:01 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15d0eaf7-d604-4546-b393-4d6d39018f42"}]: dispatch 2024-09-19T05:59:01.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:01 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15d0eaf7-d604-4546-b393-4d6d39018f42"}]': finished 2024-09-19T05:59:01.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:01 smithi005 ceph-mon[30955]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T05:59:01.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:01 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:01.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:01 smithi005 ceph-mon[30955]: from='client.? 172.21.15.157:0/434053972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:59:01.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:01 smithi049 ceph-mon[34640]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T05:59:01.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:01 smithi049 ceph-mon[34640]: from='client.? 172.21.15.157:0/2979994408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15d0eaf7-d604-4546-b393-4d6d39018f42"}]: dispatch 2024-09-19T05:59:01.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:01 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15d0eaf7-d604-4546-b393-4d6d39018f42"}]: dispatch 2024-09-19T05:59:01.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:01 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15d0eaf7-d604-4546-b393-4d6d39018f42"}]': finished 2024-09-19T05:59:01.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:01 smithi049 ceph-mon[34640]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T05:59:01.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:01 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:01.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:01 smithi049 ceph-mon[34640]: from='client.? 172.21.15.157:0/434053972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:59:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:02 smithi157 ceph-mon[35346]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:02 smithi157 ceph-mon[35346]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T05:59:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:02 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:02.818 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:02 smithi005 ceph-mon[30955]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:02.818 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:02 smithi005 ceph-mon[30955]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T05:59:02.818 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:02 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:02 smithi049 ceph-mon[34640]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:02 smithi049 ceph-mon[34640]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T05:59:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:02 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:04.773 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:04 smithi157 ceph-mon[35346]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:04.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:04 smithi005 ceph-mon[30955]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:04.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:04 smithi049 ceph-mon[34640]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:06.774 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:06 smithi157 ceph-mon[35346]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:06 smithi005 ceph-mon[30955]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:06 smithi049 ceph-mon[34640]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:08.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:08 smithi157 ceph-mon[35346]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:08.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T05:59:08.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:08 smithi005 ceph-mon[30955]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T05:59:08.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:08 smithi049 ceph-mon[34640]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T05:59:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:09.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:09 smithi157 ceph-mon[35346]: Deploying daemon osd.8 on smithi157 2024-09-19T05:59:09.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:09 smithi005 ceph-mon[30955]: Deploying daemon osd.8 on smithi157 2024-09-19T05:59:09.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:09 smithi049 ceph-mon[34640]: Deploying daemon osd.8 on smithi157 2024-09-19T05:59:10.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:10 smithi157 ceph-mon[35346]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:10 smithi005 ceph-mon[30955]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:10 smithi049 ceph-mon[34640]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:12.783 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:12 smithi157 ceph-mon[35346]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:12 smithi005 ceph-mon[30955]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:12 smithi049 ceph-mon[34640]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:14.658 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:14 smithi157 ceph-mon[35346]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:14.658 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:59:14.658 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:14.658 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:14 smithi005 ceph-mon[30955]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:59:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:14 smithi049 ceph-mon[34640]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:59:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:16 smithi157 ceph-mon[35346]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:16 smithi005 ceph-mon[30955]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:16 smithi049 ceph-mon[34640]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:17.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.619+0000 7f7902ffd640 1 -- 172.21.15.157:0/2641750396 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f790c06a6d0 con 0x7f78f80a3240 2024-09-19T05:59:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 8 on host 'smithi157' 2024-09-19T05:59:17.623 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.621+0000 7f7913f70640 1 -- 172.21.15.157:0/2641750396 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f78f80a3240 msgr2=0x7f78f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:59:17.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.621+0000 7f7913f70640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f78f80a3240 0x7f78f80a5700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f78fc006010 tx=0x7f78fc000bc0 comp rx=0 tx=0).stop 2024-09-19T05:59:17.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.621+0000 7f7913f70640 1 -- 172.21.15.157:0/2641750396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c100ad0 msgr2=0x7f790c104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:59:17.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.621+0000 7f7913f70640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c100ad0 0x7f790c104a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f790800d950 tx=0x7f790800de10 comp rx=0 tx=0).stop 2024-09-19T05:59:17.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.621+0000 7f7913f70640 1 -- 172.21.15.157:0/2641750396 shutdown_connections 2024-09-19T05:59:17.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.621+0000 7f7913f70640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f78f80a3240 0x7f78f80a5700 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:17.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.621+0000 7f7913f70640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f790c102690 0x7f790c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:17.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.621+0000 7f7913f70640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f790c101cd0 0x7f790c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:17.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.621+0000 7f7913f70640 1 --2- 172.21.15.157:0/2641750396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c100ad0 0x7f790c104a60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:17.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.621+0000 7f7913f70640 1 -- 172.21.15.157:0/2641750396 >> 172.21.15.157:0/2641750396 conn(0x7f790c0fc280 msgr2=0x7f790c069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:59:17.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.621+0000 7f7913f70640 1 -- 172.21.15.157:0/2641750396 shutdown_connections 2024-09-19T05:59:17.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T05:59:17.621+0000 7f7913f70640 1 -- 172.21.15.157:0/2641750396 wait complete. 2024-09-19T05:59:17.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:17.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:17.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:17.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:59:17.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:17.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:59:17.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:17.968 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:18.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:18.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:18.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:18.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:59:18.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:18.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:59:18.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:18.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:18.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:18.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:18.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:18.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:59:18.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:18.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:59:18.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:18.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:18.293 DEBUG:teuthology.orchestra.run.smithi157:osd.8> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T05:59:18.297 INFO:tasks.cephadm:Deploying osd.9 on smithi005 with /dev/vg_nvme/lv_1... 2024-09-19T05:59:18.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T05:59:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:59:18.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:18 smithi005 ceph-mon[30955]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:19.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:18 smithi049 ceph-mon[34640]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:19.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:18 smithi157 ceph-mon[35346]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:20.335 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 05:59:20 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T05:59:20.286+0000 7fb6f4f5e640 -1 osd.8 0 log_to_monitors true 2024-09-19T05:59:21.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:20 smithi157 ceph-mon[35346]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:21.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:20 smithi157 ceph-mon[35346]: from='osd.8 [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:59:21.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:20 smithi157 ceph-mon[35346]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:59:21.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:20 smithi005 ceph-mon[30955]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:21.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:20 smithi005 ceph-mon[30955]: from='osd.8 [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:59:21.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:20 smithi005 ceph-mon[30955]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:59:21.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:20 smithi049 ceph-mon[34640]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:21.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:20 smithi049 ceph-mon[34640]: from='osd.8 [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:59:21.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:20 smithi049 ceph-mon[34640]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:59:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T05:59:21.397 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_1 2024-09-19T05:59:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:59:22.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='osd.8 [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:59:22.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:22.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T05:59:22.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T05:59:22.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:22.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='osd.8 [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:59:22.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:59:22.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:22.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:22.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:22.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:22.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:22.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:22.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:22.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:59:22.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:22.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T05:59:22.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T05:59:22.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:22.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='osd.8 [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:59:22.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T05:59:22.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:22.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:22.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:22.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:22.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:22.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:22.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:22.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:59:22.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.296+0000 7f1001eaa640 1 -- 172.21.15.5:0/701186948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc104f90 msgr2=0x7f0ffc107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:59:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.296+0000 7f1001eaa640 1 --2- 172.21.15.5:0/701186948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc104f90 0x7f0ffc107380 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009900 tx=0x7f0fe402a410 comp rx=0 tx=0).stop 2024-09-19T05:59:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.296+0000 7f1001eaa640 1 -- 172.21.15.5:0/701186948 shutdown_connections 2024-09-19T05:59:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.296+0000 7f1001eaa640 1 --2- 172.21.15.5:0/701186948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc1078c0 0x7f0ffc109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.296+0000 7f1001eaa640 1 --2- 172.21.15.5:0/701186948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc104f90 0x7f0ffc107380 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.296+0000 7f1001eaa640 1 --2- 172.21.15.5:0/701186948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ffc102660 0x7f0ffc104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.296+0000 7f1001eaa640 1 -- 172.21.15.5:0/701186948 >> 172.21.15.5:0/701186948 conn(0x7f0ffc0fc280 msgr2=0x7f0ffc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:59:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.296+0000 7f1001eaa640 1 -- 172.21.15.5:0/701186948 shutdown_connections 2024-09-19T05:59:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.296+0000 7f1001eaa640 1 -- 172.21.15.5:0/701186948 wait complete. 2024-09-19T05:59:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.297+0000 7f1001eaa640 1 Processor -- start 2024-09-19T05:59:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.297+0000 7f1001eaa640 1 -- start start 2024-09-19T05:59:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.297+0000 7f1001eaa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ffc102660 0x7f0ffc19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:59:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.297+0000 7f1001eaa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc104f90 0x7f0ffc19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:59:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.297+0000 7f1001eaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc1078c0 0x7f0ffc1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:59:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.297+0000 7f1001eaa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc10d790 con 0x7f0ffc1078c0 2024-09-19T05:59:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.297+0000 7f1001eaa640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc10d610 con 0x7f0ffc102660 2024-09-19T05:59:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.297+0000 7f1001eaa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc10d910 con 0x7f0ffc104f90 2024-09-19T05:59:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.297+0000 7f0ffbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc1078c0 0x7f0ffc1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:59:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ffbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc1078c0 0x7f0ffc1a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38764/0 (socket says 172.21.15.5:38764) 2024-09-19T05:59:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ffbfff640 1 -- 172.21.15.5:0/395174571 learned_addr learned my addr 172.21.15.5:0/395174571 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T05:59:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ff2ffd640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc104f90 0x7f0ffc19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:59:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ffb7fe640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ffc102660 0x7f0ffc19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:59:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ffbfff640 1 -- 172.21.15.5:0/395174571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc104f90 msgr2=0x7f0ffc19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:59:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ffbfff640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc104f90 0x7f0ffc19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ffbfff640 1 -- 172.21.15.5:0/395174571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ffc102660 msgr2=0x7f0ffc19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:59:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ffbfff640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ffc102660 0x7f0ffc19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ffbfff640 1 -- 172.21.15.5:0/395174571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc1a1cd0 con 0x7f0ffc1078c0 2024-09-19T05:59:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ff2ffd640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc104f90 0x7f0ffc19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:59:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ffbfff640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc1078c0 0x7f0ffc1a1660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400eb10 tx=0x7f0ff400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:59:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ff97fa640 1 -- 172.21.15.5:0/395174571 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4004b20 con 0x7f0ffc1078c0 2024-09-19T05:59:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ff97fa640 1 -- 172.21.15.5:0/395174571 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ff4004cc0 con 0x7f0ffc1078c0 2024-09-19T05:59:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f1001eaa640 1 -- 172.21.15.5:0/395174571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc1a1fc0 con 0x7f0ffc1078c0 2024-09-19T05:59:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f0ff97fa640 1 -- 172.21.15.5:0/395174571 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4010690 con 0x7f0ffc1078c0 2024-09-19T05:59:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.298+0000 7f1001eaa640 1 -- 172.21.15.5:0/395174571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1a24f0 con 0x7f0ffc1078c0 2024-09-19T05:59:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.300+0000 7f1001eaa640 1 -- 172.21.15.5:0/395174571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ffc109410 con 0x7f0ffc1078c0 2024-09-19T05:59:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.301+0000 7f0ff97fa640 1 -- 172.21.15.5:0/395174571 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f0ff4005020 con 0x7f0ffc1078c0 2024-09-19T05:59:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.302+0000 7f0ff97fa640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0fe80a3270 0x7f0fe80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:59:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.302+0000 7f0ffb7fe640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0fe80a3270 0x7f0fe80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:59:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.302+0000 7f0ff97fa640 1 -- 172.21.15.5:0/395174571 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5549+0+0 (secure 0 0 0) 0x7f0ff40cf4a0 con 0x7f0ffc1078c0 2024-09-19T05:59:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.303+0000 7f0ffb7fe640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0fe80a3270 0x7f0fe80a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009890 tx=0x7f0fec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:59:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.306+0000 7f0ff97fa640 1 -- 172.21.15.5:0/395174571 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0ff409cbf0 con 0x7f0ffc1078c0 2024-09-19T05:59:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:22.509+0000 7f1001eaa640 1 -- 172.21.15.5:0/395174571 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0ffc062630 con 0x7f0fe80a3270 2024-09-19T05:59:22.740 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T05:59:22.403+0000 7fb6f06be640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T05:59:22.740 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T05:59:22.414+0000 7fb6e8cf0640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:59:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:22 smithi005 ceph-mon[30955]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:22 smithi005 ceph-mon[30955]: Detected new or changed devices on smithi157 2024-09-19T05:59:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:22 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi157 to 5696M 2024-09-19T05:59:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:22 smithi005 ceph-mon[30955]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T05:59:22.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:22 smithi005 ceph-mon[30955]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T05:59:22.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:22 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:22.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:22 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:22.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:22 smithi005 ceph-mon[30955]: from='osd.8 ' entity='osd.8' 2024-09-19T05:59:22.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:22 smithi005 ceph-mon[30955]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:59:22.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:22 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:59:22.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:22 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:59:22.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:22 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-mon[35346]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-mon[35346]: Detected new or changed devices on smithi157 2024-09-19T05:59:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi157 to 5696M 2024-09-19T05:59:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-mon[35346]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T05:59:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-mon[35346]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T05:59:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-mon[35346]: from='osd.8 ' entity='osd.8' 2024-09-19T05:59:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-mon[35346]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:59:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:59:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:59:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:22 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:23.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:22 smithi049 ceph-mon[34640]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:23.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:22 smithi049 ceph-mon[34640]: Detected new or changed devices on smithi157 2024-09-19T05:59:23.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:22 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi157 to 5696M 2024-09-19T05:59:23.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:22 smithi049 ceph-mon[34640]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T05:59:23.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:22 smithi049 ceph-mon[34640]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T05:59:23.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:23.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:23.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:22 smithi049 ceph-mon[34640]: from='osd.8 ' entity='osd.8' 2024-09-19T05:59:23.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:22 smithi049 ceph-mon[34640]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:59:23.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:59:23.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:59:23.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:24.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:23 smithi157 ceph-mon[35346]: purged_snaps scrub starts 2024-09-19T05:59:24.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:23 smithi157 ceph-mon[35346]: purged_snaps scrub ok 2024-09-19T05:59:24.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:23 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:24.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:23 smithi157 ceph-mon[35346]: osd.8 [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859] boot 2024-09-19T05:59:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:23 smithi157 ceph-mon[35346]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T05:59:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:23 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:23 smithi157 ceph-mon[35346]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:23 smithi157 ceph-mon[35346]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T05:59:24.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:23 smithi005 ceph-mon[30955]: purged_snaps scrub starts 2024-09-19T05:59:24.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:23 smithi005 ceph-mon[30955]: purged_snaps scrub ok 2024-09-19T05:59:24.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:23 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:24.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:23 smithi005 ceph-mon[30955]: osd.8 [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859] boot 2024-09-19T05:59:24.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:23 smithi005 ceph-mon[30955]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T05:59:24.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:23 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:24.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:23 smithi005 ceph-mon[30955]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:24.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:23 smithi005 ceph-mon[30955]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T05:59:24.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:23 smithi049 ceph-mon[34640]: purged_snaps scrub starts 2024-09-19T05:59:24.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:23 smithi049 ceph-mon[34640]: purged_snaps scrub ok 2024-09-19T05:59:24.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:23 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:24.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:23 smithi049 ceph-mon[34640]: osd.8 [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859] boot 2024-09-19T05:59:24.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:23 smithi049 ceph-mon[34640]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T05:59:24.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:23 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:59:24.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:23 smithi049 ceph-mon[34640]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:59:24.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:23 smithi049 ceph-mon[34640]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T05:59:25.459 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:25 smithi005 ceph-mon[30955]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T05:59:25.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:25 smithi157 ceph-mon[35346]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T05:59:25.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:25 smithi049 ceph-mon[34640]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T05:59:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:26 smithi157 ceph-mon[35346]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:26 smithi157 ceph-mon[35346]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T05:59:26.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:26 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3992428677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30c08537-f593-4299-bfa1-fa7ba8a95169"}]: dispatch 2024-09-19T05:59:26.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:26 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3992428677' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30c08537-f593-4299-bfa1-fa7ba8a95169"}]': finished 2024-09-19T05:59:26.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:26 smithi157 ceph-mon[35346]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T05:59:26.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:26 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:26.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:26 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3849581129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:59:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:26 smithi005 ceph-mon[30955]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:26 smithi005 ceph-mon[30955]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T05:59:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:26 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3992428677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30c08537-f593-4299-bfa1-fa7ba8a95169"}]: dispatch 2024-09-19T05:59:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:26 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3992428677' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30c08537-f593-4299-bfa1-fa7ba8a95169"}]': finished 2024-09-19T05:59:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:26 smithi005 ceph-mon[30955]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T05:59:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:26 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:26 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3849581129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:59:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:26 smithi049 ceph-mon[34640]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:26 smithi049 ceph-mon[34640]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T05:59:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:26 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3992428677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30c08537-f593-4299-bfa1-fa7ba8a95169"}]: dispatch 2024-09-19T05:59:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:26 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3992428677' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30c08537-f593-4299-bfa1-fa7ba8a95169"}]': finished 2024-09-19T05:59:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:26 smithi049 ceph-mon[34640]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T05:59:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:26 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:26 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3849581129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:59:28.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:28 smithi157 ceph-mon[35346]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:28.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:28 smithi005 ceph-mon[30955]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:28.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:28 smithi049 ceph-mon[34640]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:30.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:30 smithi005 ceph-mon[30955]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:30.422 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:30 smithi049 ceph-mon[34640]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:30.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:30 smithi157 ceph-mon[35346]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:32.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:32 smithi157 ceph-mon[35346]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T05:59:32.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:32 smithi005 ceph-mon[30955]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T05:59:32.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:32 smithi049 ceph-mon[34640]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T05:59:34.453 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:34 smithi005 ceph-mon[30955]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:59:34.453 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T05:59:34.453 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:34.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:34 smithi157 ceph-mon[35346]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:59:34.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T05:59:34.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:34.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:34 smithi049 ceph-mon[34640]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:59:34.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T05:59:34.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:35.546 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:35 smithi005 ceph-mon[30955]: Deploying daemon osd.9 on smithi005 2024-09-19T05:59:35.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:35 smithi157 ceph-mon[35346]: Deploying daemon osd.9 on smithi005 2024-09-19T05:59:35.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:35 smithi049 ceph-mon[34640]: Deploying daemon osd.9 on smithi005 2024-09-19T05:59:36.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:36 smithi005 ceph-mon[30955]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T05:59:36.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:36 smithi157 ceph-mon[35346]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T05:59:36.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:36 smithi049 ceph-mon[34640]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T05:59:38.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:38 smithi157 ceph-mon[35346]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T05:59:38.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:38 smithi005 ceph-mon[30955]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T05:59:38.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:38 smithi049 ceph-mon[34640]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T05:59:40.450 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:40 smithi005 ceph-mon[30955]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:59:40.450 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:40 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:59:40.450 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:40 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:40.450 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:40 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:40.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:40 smithi157 ceph-mon[35346]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:59:40.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:40 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:59:40.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:40 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:40.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:40 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:40.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:40 smithi049 ceph-mon[34640]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:59:40.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:40 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:59:40.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:40 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:40.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:40 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:42.435 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:42 smithi005 ceph-mon[30955]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:59:42.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:42 smithi157 ceph-mon[35346]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:59:42.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:42 smithi049 ceph-mon[34640]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:59:43.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.508+0000 7f0ff97fa640 1 -- 172.21.15.5:0/395174571 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0ffc062630 con 0x7f0fe80a3270 2024-09-19T05:59:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 9 on host 'smithi005' 2024-09-19T05:59:43.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.511+0000 7f1001eaa640 1 -- 172.21.15.5:0/395174571 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0fe80a3270 msgr2=0x7f0fe80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:59:43.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.511+0000 7f1001eaa640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0fe80a3270 0x7f0fe80a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009890 tx=0x7f0fec008040 comp rx=0 tx=0).stop 2024-09-19T05:59:43.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.511+0000 7f1001eaa640 1 -- 172.21.15.5:0/395174571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc1078c0 msgr2=0x7f0ffc1a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:59:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.511+0000 7f1001eaa640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc1078c0 0x7f0ffc1a1660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400eb10 tx=0x7f0ff400c2d0 comp rx=0 tx=0).stop 2024-09-19T05:59:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.511+0000 7f1001eaa640 1 -- 172.21.15.5:0/395174571 shutdown_connections 2024-09-19T05:59:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.511+0000 7f1001eaa640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0fe80a3270 0x7f0fe80a5730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.511+0000 7f1001eaa640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc1078c0 0x7f0ffc1a1660 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.511+0000 7f1001eaa640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc104f90 0x7f0ffc19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.511+0000 7f1001eaa640 1 --2- 172.21.15.5:0/395174571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ffc102660 0x7f0ffc19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.511+0000 7f1001eaa640 1 -- 172.21.15.5:0/395174571 >> 172.21.15.5:0/395174571 conn(0x7f0ffc0fc280 msgr2=0x7f0ffc101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:59:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.511+0000 7f1001eaa640 1 -- 172.21.15.5:0/395174571 shutdown_connections 2024-09-19T05:59:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T05:59:43.511+0000 7f1001eaa640 1 -- 172.21.15.5:0/395174571 wait complete. 2024-09-19T05:59:43.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:43.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:43.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:43.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:59:43.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:43.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:59:43.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:43.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:43 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:44.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:44.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:44.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:44.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:59:44.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:44.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:59:44.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:44.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:43 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:59:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:59:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:43 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:44.153 DEBUG:teuthology.orchestra.run.smithi005:osd.9> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T05:59:44.156 INFO:tasks.cephadm:Deploying osd.10 on smithi049 with /dev/vg_nvme/lv_1... 2024-09-19T05:59:44.156 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T05:59:44.422 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:59:44.793 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:44 smithi049 ceph-mon[34640]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:45.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:44 smithi157 ceph-mon[35346]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:45.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:44 smithi005 ceph-mon[30955]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:46.380 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 05:59:46 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T05:59:46.160+0000 7f83cf9a5640 -1 osd.9 0 log_to_monitors true 2024-09-19T05:59:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:46 smithi005 ceph-mon[30955]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:46 smithi005 ceph-mon[30955]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:59:47.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:46 smithi157 ceph-mon[35346]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:47.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:46 smithi157 ceph-mon[35346]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:59:47.141 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:46 smithi049 ceph-mon[34640]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:47.141 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:46 smithi049 ceph-mon[34640]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:59:47.144 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-19T05:59:47.170 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_1 2024-09-19T05:59:47.433 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:59:47.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.920+0000 7fa6e084d640 1 -- 172.21.15.49:0/351749351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6dc102e00 msgr2=0x7fa6dc103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:59:47.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.920+0000 7fa6e084d640 1 --2- 172.21.15.49:0/351749351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6dc102e00 0x7fa6dc103280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009900 tx=0x7fa6d002a410 comp rx=0 tx=0).stop 2024-09-19T05:59:47.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.920+0000 7fa6e084d640 1 -- 172.21.15.49:0/351749351 shutdown_connections 2024-09-19T05:59:47.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.920+0000 7fa6e084d640 1 --2- 172.21.15.49:0/351749351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6dc1037c0 0x7fa6dc10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:47.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.920+0000 7fa6e084d640 1 --2- 172.21.15.49:0/351749351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6dc102e00 0x7fa6dc103280 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:47.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.920+0000 7fa6e084d640 1 --2- 172.21.15.49:0/351749351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc101c00 0x7fa6dc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:47.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.920+0000 7fa6e084d640 1 -- 172.21.15.49:0/351749351 >> 172.21.15.49:0/351749351 conn(0x7fa6dc0fd390 msgr2=0x7fa6dc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:59:47.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.920+0000 7fa6e084d640 1 -- 172.21.15.49:0/351749351 shutdown_connections 2024-09-19T05:59:47.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.920+0000 7fa6e084d640 1 -- 172.21.15.49:0/351749351 wait complete. 2024-09-19T05:59:47.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6e084d640 1 Processor -- start 2024-09-19T05:59:47.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6e084d640 1 -- start start 2024-09-19T05:59:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6e084d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc101c00 0x7fa6dc075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:59:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6e084d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6dc102e00 0x7fa6dc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:59:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6e084d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6dc1037c0 0x7fa6dc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:59:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6e084d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc10d730 con 0x7fa6dc101c00 2024-09-19T05:59:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6e084d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc10d5b0 con 0x7fa6dc1037c0 2024-09-19T05:59:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6e084d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc10d8b0 con 0x7fa6dc102e00 2024-09-19T05:59:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc101c00 0x7fa6dc075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:59:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6dad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6dc1037c0 0x7fa6dc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:59:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc101c00 0x7fa6dc075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.49:49600/0 (socket says 172.21.15.49:49600) 2024-09-19T05:59:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6da59c640 1 -- 172.21.15.49:0/1149015682 learned_addr learned my addr 172.21.15.49:0/1149015682 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-19T05:59:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.921+0000 7fa6d9d9b640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6dc102e00 0x7fa6dc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:59:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.922+0000 7fa6dad9d640 1 -- 172.21.15.49:0/1149015682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6dc102e00 msgr2=0x7fa6dc0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:59:47.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.922+0000 7fa6dad9d640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6dc102e00 0x7fa6dc0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:47.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.922+0000 7fa6dad9d640 1 -- 172.21.15.49:0/1149015682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc101c00 msgr2=0x7fa6dc075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:59:47.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.922+0000 7fa6dad9d640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc101c00 0x7fa6dc075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:59:47.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.922+0000 7fa6dad9d640 1 -- 172.21.15.49:0/1149015682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc0753e0 con 0x7fa6dc1037c0 2024-09-19T05:59:47.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.922+0000 7fa6dad9d640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6dc1037c0 0x7fa6dc074b20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400e940 tx=0x7fa6d400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:59:47.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.922+0000 7fa6cb7fe640 1 -- 172.21.15.49:0/1149015682 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4004d20 con 0x7fa6dc1037c0 2024-09-19T05:59:47.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.922+0000 7fa6cb7fe640 1 -- 172.21.15.49:0/1149015682 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6d4005020 con 0x7fa6dc1037c0 2024-09-19T05:59:47.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.922+0000 7fa6e084d640 1 -- 172.21.15.49:0/1149015682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc1a1fa0 con 0x7fa6dc1037c0 2024-09-19T05:59:47.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.922+0000 7fa6cb7fe640 1 -- 172.21.15.49:0/1149015682 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d40116d0 con 0x7fa6dc1037c0 2024-09-19T05:59:47.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.922+0000 7fa6e084d640 1 -- 172.21.15.49:0/1149015682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc1a24a0 con 0x7fa6dc1037c0 2024-09-19T05:59:47.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.923+0000 7fa6e084d640 1 -- 172.21.15.49:0/1149015682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6dc1090a0 con 0x7fa6dc1037c0 2024-09-19T05:59:47.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.924+0000 7fa6cb7fe640 1 -- 172.21.15.49:0/1149015682 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fa6d400cb20 con 0x7fa6dc1037c0 2024-09-19T05:59:47.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.925+0000 7fa6cb7fe640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa6c40a32c0 0x7fa6c40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:59:47.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.926+0000 7fa6da59c640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa6c40a32c0 0x7fa6c40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:59:47.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.926+0000 7fa6cb7fe640 1 -- 172.21.15.49:0/1149015682 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7fa6d40cf310 con 0x7fa6dc1037c0 2024-09-19T05:59:47.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.926+0000 7fa6da59c640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa6c40a32c0 0x7fa6c40a5780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00bc80 tx=0x7fa6cc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:59:47.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:47.930+0000 7fa6cb7fe640 1 -- 172.21.15.49:0/1149015682 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa6d4016030 con 0x7fa6dc1037c0 2024-09-19T05:59:48.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T05:59:48.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T05:59:48.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:59:48.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:48.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:48.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:48.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:48.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:48.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:48.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:48.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:48.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:48.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:59:48.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:48.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:47 smithi157 ceph-mon[35346]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-19T05:59:48.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T05:59:48.115+0000 7fa6e084d640 1 -- 172.21.15.49:0/1149015682 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa6dc062630 con 0x7fa6c40a32c0 2024-09-19T05:59:48.117 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:59:48.118 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:48.119 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:47 smithi049 ceph-mon[34640]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-19T05:59:48.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T05:59:48.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T05:59:48.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-19T05:59:48.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:48.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:48.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:48.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:48.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:48.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:48.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:59:48.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:48.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:48.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:59:48.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T05:59:48.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:47 smithi005 ceph-mon[30955]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-19T05:59:48.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 05:59:48 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T05:59:48.367+0000 7f83cc13c640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T05:59:48.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 05:59:48 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T05:59:48.381+0000 7f83c3737640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:59:48.805 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:48 smithi049 ceph-mon[34640]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:48.805 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:48 smithi049 ceph-mon[34640]: Detected new or changed devices on smithi005 2024-09-19T05:59:48.805 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:48 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-19T05:59:48.805 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:48 smithi049 ceph-mon[34640]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T05:59:48.806 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:48.806 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:48.806 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:59:48.806 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:59:48.806 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:48.806 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:48 smithi049 ceph-mon[34640]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' 2024-09-19T05:59:49.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:48 smithi157 ceph-mon[35346]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:49.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:48 smithi157 ceph-mon[35346]: Detected new or changed devices on smithi005 2024-09-19T05:59:49.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:48 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-19T05:59:49.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:48 smithi157 ceph-mon[35346]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T05:59:49.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:49.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:49.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:59:49.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:59:49.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:49.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:48 smithi157 ceph-mon[35346]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' 2024-09-19T05:59:49.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:48 smithi005 ceph-mon[30955]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:49.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:48 smithi005 ceph-mon[30955]: Detected new or changed devices on smithi005 2024-09-19T05:59:49.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:48 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-19T05:59:49.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:48 smithi005 ceph-mon[30955]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T05:59:49.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:49.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:49.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:59:49.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:59:49.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:59:49.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:48 smithi005 ceph-mon[30955]: from='osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237]' entity='osd.9' 2024-09-19T05:59:49.973 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:49 smithi049 ceph-mon[34640]: purged_snaps scrub starts 2024-09-19T05:59:49.974 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:49 smithi049 ceph-mon[34640]: purged_snaps scrub ok 2024-09-19T05:59:49.974 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:49 smithi049 ceph-mon[34640]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:59:49.974 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:49 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:49.974 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:49 smithi049 ceph-mon[34640]: osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237] boot 2024-09-19T05:59:49.974 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:49 smithi049 ceph-mon[34640]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T05:59:49.974 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:49 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:49 smithi157 ceph-mon[35346]: purged_snaps scrub starts 2024-09-19T05:59:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:49 smithi157 ceph-mon[35346]: purged_snaps scrub ok 2024-09-19T05:59:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:49 smithi157 ceph-mon[35346]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:59:50.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:49 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:50.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:49 smithi157 ceph-mon[35346]: osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237] boot 2024-09-19T05:59:50.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:49 smithi157 ceph-mon[35346]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T05:59:50.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:49 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:49 smithi005 ceph-mon[30955]: purged_snaps scrub starts 2024-09-19T05:59:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:49 smithi005 ceph-mon[30955]: purged_snaps scrub ok 2024-09-19T05:59:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:49 smithi005 ceph-mon[30955]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:59:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:49 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:49 smithi005 ceph-mon[30955]: osd.9 [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237] boot 2024-09-19T05:59:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:49 smithi005 ceph-mon[30955]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T05:59:50.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:49 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:59:50.966 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:50 smithi157 ceph-mon[35346]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:50.966 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:50 smithi157 ceph-mon[35346]: from='client.? 172.21.15.49:0/3286971852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68821e98-bd81-49ec-8cd6-51fb9880b553"}]: dispatch 2024-09-19T05:59:50.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:50 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68821e98-bd81-49ec-8cd6-51fb9880b553"}]: dispatch 2024-09-19T05:59:50.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:50 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68821e98-bd81-49ec-8cd6-51fb9880b553"}]': finished 2024-09-19T05:59:50.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:50 smithi157 ceph-mon[35346]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T05:59:50.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:50 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:59:50.989 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:50 smithi049 ceph-mon[34640]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:50.989 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:50 smithi049 ceph-mon[34640]: from='client.? 172.21.15.49:0/3286971852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68821e98-bd81-49ec-8cd6-51fb9880b553"}]: dispatch 2024-09-19T05:59:50.989 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:50 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68821e98-bd81-49ec-8cd6-51fb9880b553"}]: dispatch 2024-09-19T05:59:50.989 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:50 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68821e98-bd81-49ec-8cd6-51fb9880b553"}]': finished 2024-09-19T05:59:50.989 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:50 smithi049 ceph-mon[34640]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T05:59:50.989 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:50 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:59:51.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:50 smithi005 ceph-mon[30955]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:59:51.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:50 smithi005 ceph-mon[30955]: from='client.? 172.21.15.49:0/3286971852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68821e98-bd81-49ec-8cd6-51fb9880b553"}]: dispatch 2024-09-19T05:59:51.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:50 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68821e98-bd81-49ec-8cd6-51fb9880b553"}]: dispatch 2024-09-19T05:59:51.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:50 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68821e98-bd81-49ec-8cd6-51fb9880b553"}]': finished 2024-09-19T05:59:51.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:50 smithi005 ceph-mon[30955]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T05:59:51.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:50 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:59:52.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:51 smithi157 ceph-mon[35346]: from='client.? 172.21.15.49:0/2746312364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:59:52.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:51 smithi005 ceph-mon[30955]: from='client.? 172.21.15.49:0/2746312364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:59:52.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:51 smithi049 ceph-mon[34640]: from='client.? 172.21.15.49:0/2746312364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:59:53.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:52 smithi157 ceph-mon[35346]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:59:53.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:52 smithi005 ceph-mon[30955]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:59:53.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:52 smithi049 ceph-mon[34640]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:59:55.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:54 smithi157 ceph-mon[35346]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:59:55.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:54 smithi005 ceph-mon[30955]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:59:55.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:54 smithi049 ceph-mon[34640]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:59:57.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:56 smithi157 ceph-mon[35346]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:59:57.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:56 smithi005 ceph-mon[30955]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:59:57.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:56 smithi049 ceph-mon[34640]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:59:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:58 smithi049 ceph-mon[34640]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:59:59.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:58 smithi157 ceph-mon[35346]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:59:59.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:58 smithi005 ceph-mon[30955]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:00.033 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:59 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T06:00:00.034 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 05:59:59 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:00.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:59 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T06:00:00.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 05:59:59 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:00.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:59 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T06:00:00.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 05:59:59 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:00.997 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:00 smithi049 ceph-mon[34640]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:00.997 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:00 smithi049 ceph-mon[34640]: Deploying daemon osd.10 on smithi049 2024-09-19T06:00:00.997 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:00 smithi049 ceph-mon[34640]: overall HEALTH_OK 2024-09-19T06:00:01.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:00 smithi157 ceph-mon[35346]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:01.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:00 smithi157 ceph-mon[35346]: Deploying daemon osd.10 on smithi049 2024-09-19T06:00:01.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:00 smithi157 ceph-mon[35346]: overall HEALTH_OK 2024-09-19T06:00:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:00 smithi005 ceph-mon[30955]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:00 smithi005 ceph-mon[30955]: Deploying daemon osd.10 on smithi049 2024-09-19T06:00:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:00 smithi005 ceph-mon[30955]: overall HEALTH_OK 2024-09-19T06:00:02.997 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:02 smithi049 ceph-mon[34640]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:03.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:02 smithi157 ceph-mon[35346]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:03.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:02 smithi005 ceph-mon[30955]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:04.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:04 smithi049 ceph-mon[34640]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:05.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:04 smithi157 ceph-mon[35346]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:05.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:04 smithi005 ceph-mon[30955]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:05.978 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:05 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:05.978 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:05 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:05.978 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:05 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:06.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:05 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:06.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:05 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:06.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:05 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:06.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:05 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:06.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:05 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:06.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:05 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:06.941 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:06 smithi049 ceph-mon[34640]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:07.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:06 smithi157 ceph-mon[35346]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:06 smithi005 ceph-mon[30955]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:07.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:07 smithi049 ceph-mon[34640]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:08.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:07 smithi157 ceph-mon[35346]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:08.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:07 smithi005 ceph-mon[30955]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:08.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.568+0000 7fa6cb7fe640 1 -- 172.21.15.49:0/1149015682 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa6dc062630 con 0x7fa6c40a32c0 2024-09-19T06:00:08.570 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 10 on host 'smithi049' 2024-09-19T06:00:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.570+0000 7fa6e084d640 1 -- 172.21.15.49:0/1149015682 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa6c40a32c0 msgr2=0x7fa6c40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.570+0000 7fa6e084d640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa6c40a32c0 0x7fa6c40a5780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00bc80 tx=0x7fa6cc00b410 comp rx=0 tx=0).stop 2024-09-19T06:00:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.570+0000 7fa6e084d640 1 -- 172.21.15.49:0/1149015682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6dc1037c0 msgr2=0x7fa6dc074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.570+0000 7fa6e084d640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6dc1037c0 0x7fa6dc074b20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400e940 tx=0x7fa6d400ee00 comp rx=0 tx=0).stop 2024-09-19T06:00:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.570+0000 7fa6e084d640 1 -- 172.21.15.49:0/1149015682 shutdown_connections 2024-09-19T06:00:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.570+0000 7fa6e084d640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa6c40a32c0 0x7fa6c40a5780 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:08.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.570+0000 7fa6e084d640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6dc1037c0 0x7fa6dc074b20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:08.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.570+0000 7fa6e084d640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6dc102e00 0x7fa6dc0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:08.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.570+0000 7fa6e084d640 1 --2- 172.21.15.49:0/1149015682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc101c00 0x7fa6dc075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:08.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.570+0000 7fa6e084d640 1 -- 172.21.15.49:0/1149015682 >> 172.21.15.49:0/1149015682 conn(0x7fa6dc0fd390 msgr2=0x7fa6dc106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:08.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.570+0000 7fa6e084d640 1 -- 172.21.15.49:0/1149015682 shutdown_connections 2024-09-19T06:00:08.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-19T06:00:08.570+0000 7fa6e084d640 1 -- 172.21.15.49:0/1149015682 wait complete. 2024-09-19T06:00:09.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:09.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:09.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:09.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.266 DEBUG:teuthology.orchestra.run.smithi049:osd.10> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T06:00:09.269 INFO:tasks.cephadm:Deploying osd.11 on smithi157 with /dev/vg_nvme/lv_1... 2024-09-19T06:00:09.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T06:00:09.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:09.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:09.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:09.298 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.298 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:09.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:09.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:09.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:09.525 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:00:10.779 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:10 smithi049 ceph-mon[34640]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:10.854 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:10 smithi157 ceph-mon[35346]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:10 smithi005 ceph-mon[30955]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:11.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:11 smithi157 ceph-mon[35346]: from='osd.10 [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:00:11.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:11 smithi157 ceph-mon[35346]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:00:11.812 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:11 smithi049 ceph-mon[34640]: from='osd.10 [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:00:11.812 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:11 smithi049 ceph-mon[34640]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:00:11.812 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:00:11 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:00:11.526+0000 7fbd5e6a4640 -1 osd.10 0 log_to_monitors true 2024-09-19T06:00:11.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:11 smithi005 ceph-mon[30955]: from='osd.10 [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:00:11.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:11 smithi005 ceph-mon[30955]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:00:12.098 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T06:00:12.131 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_1 2024-09-19T06:00:12.425 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:00:12.684 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:12.684 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T06:00:12.684 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='osd.10 [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T06:00:12.684 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T06:00:12.684 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:12.684 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T06:00:12.684 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:12.684 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:12.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:12.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:12.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:12.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:12.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:12.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:12.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:12.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:12 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:12.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T06:00:12.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='osd.10 [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T06:00:12.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T06:00:12.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:12.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T06:00:12.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:12.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:12.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:12.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:12.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:12.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:12.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:12.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:12.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:12.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:12 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:12.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.887+0000 7fcc7de77640 1 -- 172.21.15.157:0/688716828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc78102e00 msgr2=0x7fcc78103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:12.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.887+0000 7fcc7de77640 1 --2- 172.21.15.157:0/688716828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc78102e00 0x7fcc78103280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009900 tx=0x7fcc6c02a410 comp rx=0 tx=0).stop 2024-09-19T06:00:12.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.887+0000 7fcc7de77640 1 -- 172.21.15.157:0/688716828 shutdown_connections 2024-09-19T06:00:12.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.887+0000 7fcc7de77640 1 --2- 172.21.15.157:0/688716828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc781037c0 0x7fcc7810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:12.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.887+0000 7fcc7de77640 1 --2- 172.21.15.157:0/688716828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc78102e00 0x7fcc78103280 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:12.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.887+0000 7fcc7de77640 1 --2- 172.21.15.157:0/688716828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc78101c00 0x7fcc78102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:12.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.887+0000 7fcc7de77640 1 -- 172.21.15.157:0/688716828 >> 172.21.15.157:0/688716828 conn(0x7fcc780fd3b0 msgr2=0x7fcc780ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:12.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.887+0000 7fcc7de77640 1 -- 172.21.15.157:0/688716828 shutdown_connections 2024-09-19T06:00:12.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.887+0000 7fcc7de77640 1 -- 172.21.15.157:0/688716828 wait complete. 2024-09-19T06:00:12.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.887+0000 7fcc7de77640 1 Processor -- start 2024-09-19T06:00:12.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc7de77640 1 -- start start 2024-09-19T06:00:12.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc7de77640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc78101c00 0x7fcc78197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:12.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc7de77640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc78102e00 0x7fcc78198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:12.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc7de77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc781037c0 0x7fcc7819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:12.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc7de77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7810d730 con 0x7fcc781037c0 2024-09-19T06:00:12.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc7de77640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7810d5b0 con 0x7fcc78101c00 2024-09-19T06:00:12.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc7de77640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7810d8b0 con 0x7fcc78102e00 2024-09-19T06:00:12.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc76ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc78102e00 0x7fcc78198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:12.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc78101c00 0x7fcc78197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:12.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc78101c00 0x7fcc78197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.157:40804/0 (socket says 172.21.15.157:40804) 2024-09-19T06:00:12.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc76ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc78102e00 0x7fcc78198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42872/0 (socket says 172.21.15.157:42872) 2024-09-19T06:00:12.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc777fe640 1 -- 172.21.15.157:0/3752293117 learned_addr learned my addr 172.21.15.157:0/3752293117 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T06:00:12.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc77fff640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc781037c0 0x7fcc7819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:12.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc777fe640 1 -- 172.21.15.157:0/3752293117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc78102e00 msgr2=0x7fcc78198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:12.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc777fe640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc78102e00 0x7fcc78198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:12.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc777fe640 1 -- 172.21.15.157:0/3752293117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc781037c0 msgr2=0x7fcc7819f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:12.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc777fe640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc781037c0 0x7fcc7819f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:12.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc777fe640 1 -- 172.21.15.157:0/3752293117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7819fa10 con 0x7fcc78101c00 2024-09-19T06:00:12.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc76ffd640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc78102e00 0x7fcc78198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:00:12.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.888+0000 7fcc777fe640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc78101c00 0x7fcc78197de0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800c970 tx=0x7fcc6800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:12.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.889+0000 7fcc74ff9640 1 -- 172.21.15.157:0/3752293117 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68004220 con 0x7fcc78101c00 2024-09-19T06:00:12.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.889+0000 7fcc7de77640 1 -- 172.21.15.157:0/3752293117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7819fd00 con 0x7fcc78101c00 2024-09-19T06:00:12.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.889+0000 7fcc74ff9640 1 -- 172.21.15.157:0/3752293117 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc680043c0 con 0x7fcc78101c00 2024-09-19T06:00:12.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.889+0000 7fcc74ff9640 1 -- 172.21.15.157:0/3752293117 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800f450 con 0x7fcc78101c00 2024-09-19T06:00:12.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.889+0000 7fcc7de77640 1 -- 172.21.15.157:0/3752293117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc781a01c0 con 0x7fcc78101c00 2024-09-19T06:00:12.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.890+0000 7fcc7de77640 1 -- 172.21.15.157:0/3752293117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc781090a0 con 0x7fcc78101c00 2024-09-19T06:00:12.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.891+0000 7fcc74ff9640 1 -- 172.21.15.157:0/3752293117 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fcc680079b0 con 0x7fcc78101c00 2024-09-19T06:00:12.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.892+0000 7fcc74ff9640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcc600a3270 0x7fcc600a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:12.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.892+0000 7fcc74ff9640 1 -- 172.21.15.157:0/3752293117 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6414+0+0 (secure 0 0 0) 0x7fcc680cf250 con 0x7fcc78101c00 2024-09-19T06:00:12.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.892+0000 7fcc76ffd640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcc600a3270 0x7fcc600a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:12.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.893+0000 7fcc76ffd640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcc600a3270 0x7fcc600a5730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c006010 tx=0x7fcc6c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T06:00:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='osd.10 [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T06:00:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T06:00:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:12.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-09-19T06:00:12.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:12.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:12.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:12.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:12.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:12.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:12.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:12.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:12.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:12.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:12 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:12.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:12.899+0000 7fcc74ff9640 1 -- 172.21.15.157:0/3752293117 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcc68014030 con 0x7fcc78101c00 2024-09-19T06:00:13.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:13.073+0000 7fcc7de77640 1 -- 172.21.15.157:0/3752293117 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcc78062630 con 0x7fcc600a3270 2024-09-19T06:00:13.584 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:00:13.288+0000 7fbd59e04640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T06:00:13.585 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:00:13.300+0000 7fbd52436640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:00:13.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:13 smithi157 ceph-mon[35346]: Detected new or changed devices on smithi049 2024-09-19T06:00:13.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:13 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi049 to 4272M 2024-09-19T06:00:13.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:13 smithi157 ceph-mon[35346]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-19T06:00:13.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:13 smithi157 ceph-mon[35346]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T06:00:13.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:13.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:13.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:00:13.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:00:13.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:13.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:13 smithi157 ceph-mon[35346]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T06:00:13.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:13 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:13.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:13 smithi157 ceph-mon[35346]: from='osd.10 ' entity='osd.10' 2024-09-19T06:00:13.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:13 smithi005 ceph-mon[30955]: Detected new or changed devices on smithi049 2024-09-19T06:00:13.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:13 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi049 to 4272M 2024-09-19T06:00:13.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:13 smithi005 ceph-mon[30955]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-19T06:00:13.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:13 smithi005 ceph-mon[30955]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T06:00:13.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:13 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:13.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:13 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:13.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:13 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:00:13.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:13 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:00:13.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:13 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:13.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:13 smithi005 ceph-mon[30955]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T06:00:13.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:13 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:13.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:13 smithi005 ceph-mon[30955]: from='osd.10 ' entity='osd.10' 2024-09-19T06:00:13.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-mon[34640]: Detected new or changed devices on smithi049 2024-09-19T06:00:13.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi049 to 4272M 2024-09-19T06:00:13.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-mon[34640]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-19T06:00:13.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-mon[34640]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T06:00:13.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:13.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:13.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:00:13.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:00:13.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:13.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-mon[34640]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T06:00:13.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:13.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:13 smithi049 ceph-mon[34640]: from='osd.10 ' entity='osd.10' 2024-09-19T06:00:14.839 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:14 smithi157 ceph-mon[35346]: purged_snaps scrub starts 2024-09-19T06:00:14.839 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:14 smithi157 ceph-mon[35346]: purged_snaps scrub ok 2024-09-19T06:00:14.839 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:14 smithi157 ceph-mon[35346]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:14.839 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:14 smithi157 ceph-mon[35346]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:00:14.839 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:14.839 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:14 smithi157 ceph-mon[35346]: osd.10 [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002] boot 2024-09-19T06:00:14.839 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:14 smithi157 ceph-mon[35346]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T06:00:14.839 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:14 smithi005 ceph-mon[30955]: purged_snaps scrub starts 2024-09-19T06:00:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:14 smithi005 ceph-mon[30955]: purged_snaps scrub ok 2024-09-19T06:00:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:14 smithi005 ceph-mon[30955]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:14 smithi005 ceph-mon[30955]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:00:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:14 smithi005 ceph-mon[30955]: osd.10 [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002] boot 2024-09-19T06:00:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:14 smithi005 ceph-mon[30955]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T06:00:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:14 smithi049 ceph-mon[34640]: purged_snaps scrub starts 2024-09-19T06:00:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:14 smithi049 ceph-mon[34640]: purged_snaps scrub ok 2024-09-19T06:00:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:14 smithi049 ceph-mon[34640]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:00:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:14 smithi049 ceph-mon[34640]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:00:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:14 smithi049 ceph-mon[34640]: osd.10 [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002] boot 2024-09-19T06:00:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:14 smithi049 ceph-mon[34640]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T06:00:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:00:16.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:16 smithi157 ceph-mon[35346]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:16.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:16 smithi157 ceph-mon[35346]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T06:00:16.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:16 smithi157 ceph-mon[35346]: from='client.? 172.21.15.157:0/3438040255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e443b46-2697-4d0b-8452-868463a1ee1b"}]: dispatch 2024-09-19T06:00:16.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:16 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e443b46-2697-4d0b-8452-868463a1ee1b"}]: dispatch 2024-09-19T06:00:16.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:16 smithi157 ceph-mon[35346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e443b46-2697-4d0b-8452-868463a1ee1b"}]': finished 2024-09-19T06:00:16.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:16 smithi157 ceph-mon[35346]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T06:00:16.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:16 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:16 smithi005 ceph-mon[30955]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:16 smithi005 ceph-mon[30955]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T06:00:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:16 smithi005 ceph-mon[30955]: from='client.? 172.21.15.157:0/3438040255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e443b46-2697-4d0b-8452-868463a1ee1b"}]: dispatch 2024-09-19T06:00:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:16 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e443b46-2697-4d0b-8452-868463a1ee1b"}]: dispatch 2024-09-19T06:00:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:16 smithi005 ceph-mon[30955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e443b46-2697-4d0b-8452-868463a1ee1b"}]': finished 2024-09-19T06:00:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:16 smithi005 ceph-mon[30955]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T06:00:16.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:16 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:16.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:16 smithi049 ceph-mon[34640]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:16.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:16 smithi049 ceph-mon[34640]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T06:00:16.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:16 smithi049 ceph-mon[34640]: from='client.? 172.21.15.157:0/3438040255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e443b46-2697-4d0b-8452-868463a1ee1b"}]: dispatch 2024-09-19T06:00:16.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:16 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e443b46-2697-4d0b-8452-868463a1ee1b"}]: dispatch 2024-09-19T06:00:16.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:16 smithi049 ceph-mon[34640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e443b46-2697-4d0b-8452-868463a1ee1b"}]': finished 2024-09-19T06:00:16.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:16 smithi049 ceph-mon[34640]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T06:00:16.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:16 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:17.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:17 smithi157 ceph-mon[35346]: from='client.? 172.21.15.157:0/3130004979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:00:17.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:17 smithi157 ceph-mon[35346]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T06:00:17.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:17.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:17 smithi005 ceph-mon[30955]: from='client.? 172.21.15.157:0/3130004979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:00:17.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:17 smithi005 ceph-mon[30955]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T06:00:17.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:17.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:17 smithi049 ceph-mon[34640]: from='client.? 172.21.15.157:0/3130004979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:00:17.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:17 smithi049 ceph-mon[34640]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T06:00:17.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:18.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:18 smithi005 ceph-mon[30955]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:18.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:18 smithi005 ceph-mon[30955]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T06:00:18.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:18 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:18 smithi049 ceph-mon[34640]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:18 smithi049 ceph-mon[34640]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T06:00:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:18 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:19.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:18 smithi157 ceph-mon[35346]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:19.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:18 smithi157 ceph-mon[35346]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T06:00:19.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:18 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:20.744 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:20 smithi157 ceph-mon[35346]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:20.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:20 smithi005 ceph-mon[30955]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:20 smithi049 ceph-mon[34640]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:22 smithi005 ceph-mon[30955]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-19T06:00:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:22 smithi049 ceph-mon[34640]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-19T06:00:22.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:22 smithi157 ceph-mon[35346]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-19T06:00:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:24 smithi005 ceph-mon[30955]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-19T06:00:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:24 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T06:00:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:24 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:24.882 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:24 smithi157 ceph-mon[35346]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-19T06:00:24.882 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:24 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T06:00:24.883 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:24 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:24 smithi049 ceph-mon[34640]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-19T06:00:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:24 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T06:00:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:24 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:25 smithi157 ceph-mon[35346]: Deploying daemon osd.11 on smithi157 2024-09-19T06:00:25.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:25 smithi049 ceph-mon[34640]: Deploying daemon osd.11 on smithi157 2024-09-19T06:00:26.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:25 smithi005 ceph-mon[30955]: Deploying daemon osd.11 on smithi157 2024-09-19T06:00:26.723 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:26 smithi157 ceph-mon[35346]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-19T06:00:26.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:26 smithi049 ceph-mon[34640]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-19T06:00:27.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:26 smithi005 ceph-mon[30955]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-19T06:00:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:28 smithi049 ceph-mon[34640]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:00:28.964 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:28 smithi157 ceph-mon[35346]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:00:29.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:28 smithi005 ceph-mon[30955]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:00:30.870 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:30 smithi157 ceph-mon[35346]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T06:00:30.870 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:30 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:30.871 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:30 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:30.871 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:30 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:31.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:30 smithi005 ceph-mon[30955]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T06:00:31.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:30 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:31.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:30 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:31.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:30 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:31.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:30 smithi049 ceph-mon[34640]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T06:00:31.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:30 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:31.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:30 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:31.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:30 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:32 smithi157 ceph-mon[35346]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:00:32.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:32 smithi005 ceph-mon[30955]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:00:33.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:32 smithi049 ceph-mon[34640]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:00:33.491 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 11 on host 'smithi157' 2024-09-19T06:00:33.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.490+0000 7fcc74ff9640 1 -- 172.21.15.157:0/3752293117 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fcc78062630 con 0x7fcc600a3270 2024-09-19T06:00:33.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.492+0000 7fcc7de77640 1 -- 172.21.15.157:0/3752293117 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcc600a3270 msgr2=0x7fcc600a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:33.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.492+0000 7fcc7de77640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcc600a3270 0x7fcc600a5730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c006010 tx=0x7fcc6c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:00:33.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.492+0000 7fcc7de77640 1 -- 172.21.15.157:0/3752293117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc78101c00 msgr2=0x7fcc78197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:33.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.492+0000 7fcc7de77640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc78101c00 0x7fcc78197de0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800c970 tx=0x7fcc6800ce30 comp rx=0 tx=0).stop 2024-09-19T06:00:33.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.492+0000 7fcc7de77640 1 -- 172.21.15.157:0/3752293117 shutdown_connections 2024-09-19T06:00:33.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.492+0000 7fcc7de77640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcc600a3270 0x7fcc600a5730 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:33.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.492+0000 7fcc7de77640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc781037c0 0x7fcc7819f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:33.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.492+0000 7fcc7de77640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc78102e00 0x7fcc78198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:33.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.492+0000 7fcc7de77640 1 --2- 172.21.15.157:0/3752293117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc78101c00 0x7fcc78197de0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:33.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.492+0000 7fcc7de77640 1 -- 172.21.15.157:0/3752293117 >> 172.21.15.157:0/3752293117 conn(0x7fcc780fd3b0 msgr2=0x7fcc78106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:33.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.492+0000 7fcc7de77640 1 -- 172.21.15.157:0/3752293117 shutdown_connections 2024-09-19T06:00:33.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:33.492+0000 7fcc7de77640 1 -- 172.21.15.157:0/3752293117 wait complete. 2024-09-19T06:00:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:33 smithi157 ceph-mon[35346]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:34.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:34.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:34.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:33 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:33 smithi005 ceph-mon[30955]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:34.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:34.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:34.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:34.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:33 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:33 smithi049 ceph-mon[34640]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:33 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:34.167 DEBUG:teuthology.orchestra.run.smithi157:osd.11> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T06:00:34.169 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T06:00:34.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T06:00:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:00:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.909+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/3150489324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd28101d10 msgr2=0x7fdd28102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.909+0000 7fdd2ed4a640 1 --2- 172.21.15.5:0/3150489324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd28101d10 0x7fdd28102190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009900 tx=0x7fdd1802a410 comp rx=0 tx=0).stop 2024-09-19T06:00:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.909+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/3150489324 shutdown_connections 2024-09-19T06:00:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.909+0000 7fdd2ed4a640 1 --2- 172.21.15.5:0/3150489324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd281026d0 0x7fdd28108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.909+0000 7fdd2ed4a640 1 --2- 172.21.15.5:0/3150489324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd28101d10 0x7fdd28102190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.909+0000 7fdd2ed4a640 1 --2- 172.21.15.5:0/3150489324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd28100b10 0x7fdd28100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.909+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/3150489324 >> 172.21.15.5:0/3150489324 conn(0x7fdd280fc2a0 msgr2=0x7fdd280fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.909+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/3150489324 shutdown_connections 2024-09-19T06:00:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.909+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/3150489324 wait complete. 2024-09-19T06:00:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd2ed4a640 1 Processor -- start 2024-09-19T06:00:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd2ed4a640 1 -- start start 2024-09-19T06:00:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd2ed4a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd28100b10 0x7fdd28197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd2ed4a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd28101d10 0x7fdd28198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd2ed4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd281026d0 0x7fdd2819f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd2ed4a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2810c6c0 con 0x7fdd281026d0 2024-09-19T06:00:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd2ed4a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2810c540 con 0x7fdd28101d10 2024-09-19T06:00:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd2ed4a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2810c840 con 0x7fdd28100b10 2024-09-19T06:00:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd2d2e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd281026d0 0x7fdd2819f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd2d2e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd281026d0 0x7fdd2819f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52196/0 (socket says 172.21.15.5:52196) 2024-09-19T06:00:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd2d2e7640 1 -- 172.21.15.5:0/953042890 learned_addr learned my addr 172.21.15.5:0/953042890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:00:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd2cae6640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd28100b10 0x7fdd28197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.910+0000 7fdd1ffff640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd28101d10 0x7fdd28198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.911+0000 7fdd2d2e7640 1 -- 172.21.15.5:0/953042890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd28100b10 msgr2=0x7fdd28197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.911+0000 7fdd2d2e7640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd28100b10 0x7fdd28197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.911+0000 7fdd2d2e7640 1 -- 172.21.15.5:0/953042890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd28101d10 msgr2=0x7fdd28198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.911+0000 7fdd2d2e7640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd28101d10 0x7fdd28198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.911+0000 7fdd2d2e7640 1 -- 172.21.15.5:0/953042890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2819fb00 con 0x7fdd281026d0 2024-09-19T06:00:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.911+0000 7fdd2d2e7640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd281026d0 0x7fdd2819f490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000b970 tx=0x7fdd2000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.911+0000 7fdd1dffb640 1 -- 172.21.15.5:0/953042890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20019070 con 0x7fdd281026d0 2024-09-19T06:00:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.911+0000 7fdd1dffb640 1 -- 172.21.15.5:0/953042890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd20014440 con 0x7fdd281026d0 2024-09-19T06:00:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.911+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/953042890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2819fdf0 con 0x7fdd281026d0 2024-09-19T06:00:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.911+0000 7fdd1dffb640 1 -- 172.21.15.5:0/953042890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd200075d0 con 0x7fdd281026d0 2024-09-19T06:00:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.911+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/953042890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd281a0320 con 0x7fdd281026d0 2024-09-19T06:00:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.912+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/953042890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd281a05e0 con 0x7fdd281026d0 2024-09-19T06:00:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.913+0000 7fdd1dffb640 1 -- 172.21.15.5:0/953042890 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fdd2001c070 con 0x7fdd281026d0 2024-09-19T06:00:34.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.914+0000 7fdd1dffb640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdd100a32f0 0x7fdd100a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:34.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.914+0000 7fdd2cae6640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdd100a32f0 0x7fdd100a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:34.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.914+0000 7fdd1dffb640 1 -- 172.21.15.5:0/953042890 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7fdd200ce900 con 0x7fdd281026d0 2024-09-19T06:00:34.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.914+0000 7fdd2cae6640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdd100a32f0 0x7fdd100a57b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdd24005fd0 tx=0x7fdd24005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:34.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:34.918+0000 7fdd1dffb640 1 -- 172.21.15.5:0/953042890 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdd2009bc00 con 0x7fdd281026d0 2024-09-19T06:00:35.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.094+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/953042890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fdd281a0850 con 0x7fdd281026d0 2024-09-19T06:00:35.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.094+0000 7fdd1dffb640 1 -- 172.21.15.5:0/953042890 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7fdd2009b500 con 0x7fdd281026d0 2024-09-19T06:00:35.096 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:00:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.096+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/953042890 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdd100a32f0 msgr2=0x7fdd100a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.096+0000 7fdd2ed4a640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdd100a32f0 0x7fdd100a57b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdd24005fd0 tx=0x7fdd24005850 comp rx=0 tx=0).stop 2024-09-19T06:00:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.096+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/953042890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd281026d0 msgr2=0x7fdd2819f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.096+0000 7fdd2ed4a640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd281026d0 0x7fdd2819f490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000b970 tx=0x7fdd2000be30 comp rx=0 tx=0).stop 2024-09-19T06:00:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.096+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/953042890 shutdown_connections 2024-09-19T06:00:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.096+0000 7fdd2ed4a640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdd100a32f0 0x7fdd100a57b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.096+0000 7fdd2ed4a640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd281026d0 0x7fdd2819f490 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.096+0000 7fdd2ed4a640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd28101d10 0x7fdd28198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.096+0000 7fdd2ed4a640 1 --2- 172.21.15.5:0/953042890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd28100b10 0x7fdd28197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.096+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/953042890 >> 172.21.15.5:0/953042890 conn(0x7fdd280fc2a0 msgr2=0x7fdd2806a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.096+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/953042890 shutdown_connections 2024-09-19T06:00:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:35.097+0000 7fdd2ed4a640 1 -- 172.21.15.5:0/953042890 wait complete. 2024-09-19T06:00:35.982 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726725614,"num_in_osds":12,"osd_in_since":1726725615,"num_remapped_pgs":0} 2024-09-19T06:00:36.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:35 smithi157 ceph-mon[35346]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:36.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:35 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/953042890' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:00:36.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:35 smithi005 ceph-mon[30955]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:36.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:35 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/953042890' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:00:36.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:35 smithi049 ceph-mon[34640]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:36.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:35 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/953042890' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:00:36.552 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:00:36 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:00:36.274+0000 7f7f80fe1640 -1 osd.11 0 log_to_monitors true 2024-09-19T06:00:36.982 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T06:00:37.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:36 smithi157 ceph-mon[35346]: from='osd.11 [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:00:37.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:36 smithi157 ceph-mon[35346]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:00:37.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:36 smithi005 ceph-mon[30955]: from='osd.11 [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:00:37.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:36 smithi005 ceph-mon[30955]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:00:37.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:36 smithi049 ceph-mon[34640]: from='osd.11 [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:00:37.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:36 smithi049 ceph-mon[34640]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:00:37.256 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:00:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.698+0000 7f6f476b9640 1 -- 172.21.15.5:0/449345035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f40102e00 msgr2=0x7f6f40103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.698+0000 7f6f476b9640 1 --2- 172.21.15.5:0/449345035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f40102e00 0x7f6f40103280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009900 tx=0x7f6f3002a410 comp rx=0 tx=0).stop 2024-09-19T06:00:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.698+0000 7f6f476b9640 1 -- 172.21.15.5:0/449345035 shutdown_connections 2024-09-19T06:00:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.698+0000 7f6f476b9640 1 --2- 172.21.15.5:0/449345035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f401037c0 0x7f6f4010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.698+0000 7f6f476b9640 1 --2- 172.21.15.5:0/449345035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f40102e00 0x7f6f40103280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.698+0000 7f6f476b9640 1 --2- 172.21.15.5:0/449345035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f40101c00 0x7f6f40102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.698+0000 7f6f476b9640 1 -- 172.21.15.5:0/449345035 >> 172.21.15.5:0/449345035 conn(0x7f6f400fd390 msgr2=0x7f6f400ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.698+0000 7f6f476b9640 1 -- 172.21.15.5:0/449345035 shutdown_connections 2024-09-19T06:00:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.698+0000 7f6f476b9640 1 -- 172.21.15.5:0/449345035 wait complete. 2024-09-19T06:00:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f476b9640 1 Processor -- start 2024-09-19T06:00:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f476b9640 1 -- start start 2024-09-19T06:00:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f476b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f40101c00 0x7f6f40197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f476b9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f40102e00 0x7f6f40198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f476b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f401037c0 0x7f6f4019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f476b9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4010d730 con 0x7f6f40101c00 2024-09-19T06:00:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f476b9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4010d5b0 con 0x7f6f401037c0 2024-09-19T06:00:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f476b9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4010d8b0 con 0x7f6f40102e00 2024-09-19T06:00:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f45455640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f40101c00 0x7f6f40197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f45455640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f40101c00 0x7f6f40197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52210/0 (socket says 172.21.15.5:52210) 2024-09-19T06:00:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f45455640 1 -- 172.21.15.5:0/1462645780 learned_addr learned my addr 172.21.15.5:0/1462645780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:00:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f44c54640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f40102e00 0x7f6f40198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.699+0000 7f6f45c56640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f401037c0 0x7f6f4019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.700+0000 7f6f45455640 1 -- 172.21.15.5:0/1462645780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f40102e00 msgr2=0x7f6f40198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.700+0000 7f6f45455640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f40102e00 0x7f6f40198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.700+0000 7f6f45455640 1 -- 172.21.15.5:0/1462645780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f401037c0 msgr2=0x7f6f4019f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.700+0000 7f6f45455640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f401037c0 0x7f6f4019f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.700+0000 7f6f45455640 1 -- 172.21.15.5:0/1462645780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4019fa40 con 0x7f6f40101c00 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.700+0000 7f6f45c56640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f401037c0 0x7f6f4019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.700+0000 7f6f45455640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f40101c00 0x7f6f40197e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00c970 tx=0x7f6f3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.700+0000 7f6f367fc640 1 -- 172.21.15.5:0/1462645780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c004220 con 0x7f6f40101c00 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.700+0000 7f6f367fc640 1 -- 172.21.15.5:0/1462645780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f3c0043c0 con 0x7f6f40101c00 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.700+0000 7f6f476b9640 1 -- 172.21.15.5:0/1462645780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4019fd30 con 0x7f6f40101c00 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.700+0000 7f6f367fc640 1 -- 172.21.15.5:0/1462645780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00f450 con 0x7f6f40101c00 2024-09-19T06:00:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.700+0000 7f6f476b9640 1 -- 172.21.15.5:0/1462645780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f401a0190 con 0x7f6f40101c00 2024-09-19T06:00:37.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.701+0000 7f6f476b9640 1 -- 172.21.15.5:0/1462645780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f401090a0 con 0x7f6f40101c00 2024-09-19T06:00:37.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.702+0000 7f6f367fc640 1 -- 172.21.15.5:0/1462645780 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f6f3c0079b0 con 0x7f6f40101c00 2024-09-19T06:00:37.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.703+0000 7f6f367fc640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6f2c0a3270 0x7f6f2c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:37.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.703+0000 7f6f44c54640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6f2c0a3270 0x7f6f2c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:37.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.703+0000 7f6f367fc640 1 -- 172.21.15.5:0/1462645780 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6831+0+0 (secure 0 0 0) 0x7f6f3c0cf450 con 0x7f6f40101c00 2024-09-19T06:00:37.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.703+0000 7f6f44c54640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6f2c0a3270 0x7f6f2c0a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f30006010 tx=0x7f6f30000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:37.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.706+0000 7f6f367fc640 1 -- 172.21.15.5:0/1462645780 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6f3c014030 con 0x7f6f40101c00 2024-09-19T06:00:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.888+0000 7f6f476b9640 1 -- 172.21.15.5:0/1462645780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6f40062ef0 con 0x7f6f40101c00 2024-09-19T06:00:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.888+0000 7f6f367fc640 1 -- 172.21.15.5:0/1462645780 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f6f3c09c740 con 0x7f6f40101c00 2024-09-19T06:00:37.891 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:00:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.891+0000 7f6f476b9640 1 -- 172.21.15.5:0/1462645780 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6f2c0a3270 msgr2=0x7f6f2c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.891+0000 7f6f476b9640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6f2c0a3270 0x7f6f2c0a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f30006010 tx=0x7f6f30000bc0 comp rx=0 tx=0).stop 2024-09-19T06:00:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.891+0000 7f6f476b9640 1 -- 172.21.15.5:0/1462645780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f40101c00 msgr2=0x7f6f40197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.891+0000 7f6f476b9640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f40101c00 0x7f6f40197e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00c970 tx=0x7f6f3c00ce30 comp rx=0 tx=0).stop 2024-09-19T06:00:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.891+0000 7f6f476b9640 1 -- 172.21.15.5:0/1462645780 shutdown_connections 2024-09-19T06:00:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.891+0000 7f6f476b9640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6f2c0a3270 0x7f6f2c0a5730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.891+0000 7f6f476b9640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f401037c0 0x7f6f4019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.891+0000 7f6f476b9640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f40102e00 0x7f6f40198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.891+0000 7f6f476b9640 1 --2- 172.21.15.5:0/1462645780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f40101c00 0x7f6f40197e10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.891+0000 7f6f476b9640 1 -- 172.21.15.5:0/1462645780 >> 172.21.15.5:0/1462645780 conn(0x7f6f400fd390 msgr2=0x7f6f40106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.891+0000 7f6f476b9640 1 -- 172.21.15.5:0/1462645780 shutdown_connections 2024-09-19T06:00:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:37.891+0000 7f6f476b9640 1 -- 172.21.15.5:0/1462645780 wait complete. 2024-09-19T06:00:38.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:38.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='osd.11 [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: Detected new or changed devices on smithi157 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi157 to 4272M 2024-09-19T06:00:38.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:38.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:38.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:38.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:37 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:38.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:38.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='osd.11 [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: Detected new or changed devices on smithi157 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi157 to 4272M 2024-09-19T06:00:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:38.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:38.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:38.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:37 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:38.301 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:00:37.840+0000 7f7f7c741640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T06:00:38.301 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:00:37.857+0000 7f7f78d7b640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:00:38.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:38.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T06:00:38.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='osd.11 [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T06:00:38.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T06:00:38.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:38.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T06:00:38.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: Detected new or changed devices on smithi157 2024-09-19T06:00:38.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:38.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:00:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi157 to 4272M 2024-09-19T06:00:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:37 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:38.757 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726725614,"num_in_osds":12,"osd_in_since":1726725615,"num_remapped_pgs":0} 2024-09-19T06:00:39.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:38 smithi005 ceph-mon[30955]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T06:00:39.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:38 smithi005 ceph-mon[30955]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T06:00:39.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:39.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:39.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:38 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1462645780' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:00:39.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:38 smithi049 ceph-mon[34640]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T06:00:39.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:38 smithi049 ceph-mon[34640]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T06:00:39.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:39.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:39.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:38 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1462645780' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:00:39.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:38 smithi157 ceph-mon[35346]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T06:00:39.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:38 smithi157 ceph-mon[35346]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T06:00:39.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:38 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1462645780' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:00:39.758 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T06:00:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:00:40.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:39 smithi005 ceph-mon[30955]: purged_snaps scrub starts 2024-09-19T06:00:40.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:39 smithi005 ceph-mon[30955]: purged_snaps scrub ok 2024-09-19T06:00:40.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:39 smithi005 ceph-mon[30955]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:40.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:39 smithi005 ceph-mon[30955]: osd.11 [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544] boot 2024-09-19T06:00:40.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:39 smithi005 ceph-mon[30955]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T06:00:40.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:40.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:39 smithi049 ceph-mon[34640]: purged_snaps scrub starts 2024-09-19T06:00:40.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:39 smithi049 ceph-mon[34640]: purged_snaps scrub ok 2024-09-19T06:00:40.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:39 smithi049 ceph-mon[34640]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:40.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:39 smithi049 ceph-mon[34640]: osd.11 [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544] boot 2024-09-19T06:00:40.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:39 smithi049 ceph-mon[34640]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T06:00:40.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:39 smithi157 ceph-mon[35346]: purged_snaps scrub starts 2024-09-19T06:00:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:39 smithi157 ceph-mon[35346]: purged_snaps scrub ok 2024-09-19T06:00:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:39 smithi157 ceph-mon[35346]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:00:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:39 smithi157 ceph-mon[35346]: osd.11 [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544] boot 2024-09-19T06:00:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:39 smithi157 ceph-mon[35346]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T06:00:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:00:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.473+0000 7f5f0929a640 1 -- 172.21.15.5:0/401204439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f041026a0 msgr2=0x7f5f04104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.473+0000 7f5f0929a640 1 --2- 172.21.15.5:0/401204439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f041026a0 0x7f5f04104a90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009900 tx=0x7f5ef402a410 comp rx=0 tx=0).stop 2024-09-19T06:00:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.473+0000 7f5f0929a640 1 -- 172.21.15.5:0/401204439 shutdown_connections 2024-09-19T06:00:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.473+0000 7f5f0929a640 1 --2- 172.21.15.5:0/401204439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f04107900 0x7f5f04109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.473+0000 7f5f0929a640 1 --2- 172.21.15.5:0/401204439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f04104fd0 0x7f5f041073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.473+0000 7f5f0929a640 1 --2- 172.21.15.5:0/401204439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f041026a0 0x7f5f04104a90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.473+0000 7f5f0929a640 1 -- 172.21.15.5:0/401204439 >> 172.21.15.5:0/401204439 conn(0x7f5f040fc2a0 msgr2=0x7f5f040fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.473+0000 7f5f0929a640 1 -- 172.21.15.5:0/401204439 shutdown_connections 2024-09-19T06:00:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.473+0000 7f5f0929a640 1 -- 172.21.15.5:0/401204439 wait complete. 2024-09-19T06:00:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.474+0000 7f5f0929a640 1 Processor -- start 2024-09-19T06:00:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.474+0000 7f5f0929a640 1 -- start start 2024-09-19T06:00:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.474+0000 7f5f0929a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f041026a0 0x7f5f04197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.474+0000 7f5f0929a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f04104fd0 0x7f5f041983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.474+0000 7f5f0929a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f04107900 0x7f5f0419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.474+0000 7f5f0929a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0410d760 con 0x7f5f041026a0 2024-09-19T06:00:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.474+0000 7f5f0929a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0410d5e0 con 0x7f5f04107900 2024-09-19T06:00:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.474+0000 7f5f0929a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0410d8e0 con 0x7f5f04104fd0 2024-09-19T06:00:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.474+0000 7f5f02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f041026a0 0x7f5f04197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.474+0000 7f5f02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f041026a0 0x7f5f04197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52234/0 (socket says 172.21.15.5:52234) 2024-09-19T06:00:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.474+0000 7f5f02ffd640 1 -- 172.21.15.5:0/2801879174 learned_addr learned my addr 172.21.15.5:0/2801879174 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:00:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.474+0000 7f5f027fc640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f04104fd0 0x7f5f041983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5f037fe640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f04107900 0x7f5f0419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5f02ffd640 1 -- 172.21.15.5:0/2801879174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f04104fd0 msgr2=0x7f5f041983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5f02ffd640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f04104fd0 0x7f5f041983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5f02ffd640 1 -- 172.21.15.5:0/2801879174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f04107900 msgr2=0x7f5f0419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5f02ffd640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f04107900 0x7f5f0419f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5f02ffd640 1 -- 172.21.15.5:0/2801879174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0419fad0 con 0x7f5f041026a0 2024-09-19T06:00:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5f037fe640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f04107900 0x7f5f0419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:00:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5f02ffd640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f041026a0 0x7f5f04197ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4004770 tx=0x7f5ef40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5eebfff640 1 -- 172.21.15.5:0/2801879174 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef403f030 con 0x7f5f041026a0 2024-09-19T06:00:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5eebfff640 1 -- 172.21.15.5:0/2801879174 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ef4003cb0 con 0x7f5f041026a0 2024-09-19T06:00:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5f0929a640 1 -- 172.21.15.5:0/2801879174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0419fd60 con 0x7f5f041026a0 2024-09-19T06:00:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5eebfff640 1 -- 172.21.15.5:0/2801879174 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4035070 con 0x7f5f041026a0 2024-09-19T06:00:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.475+0000 7f5f0929a640 1 -- 172.21.15.5:0/2801879174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f041a0190 con 0x7f5f041026a0 2024-09-19T06:00:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.476+0000 7f5f0929a640 1 -- 172.21.15.5:0/2801879174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f040fdd90 con 0x7f5f041026a0 2024-09-19T06:00:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.477+0000 7f5eebfff640 1 -- 172.21.15.5:0/2801879174 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f5ef4042030 con 0x7f5f041026a0 2024-09-19T06:00:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.478+0000 7f5eebfff640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5eec0a3270 0x7f5eec0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.478+0000 7f5f027fc640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5eec0a3270 0x7f5eec0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.478+0000 7f5eebfff640 1 -- 172.21.15.5:0/2801879174 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f5ef40efb50 con 0x7f5f041026a0 2024-09-19T06:00:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.479+0000 7f5f027fc640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5eec0a3270 0x7f5eec0a5730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5f041993c0 tx=0x7f5ef800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.482+0000 7f5eebfff640 1 -- 172.21.15.5:0/2801879174 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5ef40bcc70 con 0x7f5f041026a0 2024-09-19T06:00:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.660+0000 7f5f0929a640 1 -- 172.21.15.5:0/2801879174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f5f04062ec0 con 0x7f5f041026a0 2024-09-19T06:00:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.660+0000 7f5eebfff640 1 -- 172.21.15.5:0/2801879174 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f5ef40dc020 con 0x7f5f041026a0 2024-09-19T06:00:40.662 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:00:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.662+0000 7f5f0929a640 1 -- 172.21.15.5:0/2801879174 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5eec0a3270 msgr2=0x7f5eec0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.662+0000 7f5f0929a640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5eec0a3270 0x7f5eec0a5730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5f041993c0 tx=0x7f5ef800b390 comp rx=0 tx=0).stop 2024-09-19T06:00:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.662+0000 7f5f0929a640 1 -- 172.21.15.5:0/2801879174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f041026a0 msgr2=0x7f5f04197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.662+0000 7f5f0929a640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f041026a0 0x7f5f04197ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4004770 tx=0x7f5ef40042f0 comp rx=0 tx=0).stop 2024-09-19T06:00:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.662+0000 7f5f0929a640 1 -- 172.21.15.5:0/2801879174 shutdown_connections 2024-09-19T06:00:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.662+0000 7f5f0929a640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5eec0a3270 0x7f5eec0a5730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.662+0000 7f5f0929a640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f04107900 0x7f5f0419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.662+0000 7f5f0929a640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f04104fd0 0x7f5f041983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.662+0000 7f5f0929a640 1 --2- 172.21.15.5:0/2801879174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f041026a0 0x7f5f04197ea0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.662+0000 7f5f0929a640 1 -- 172.21.15.5:0/2801879174 >> 172.21.15.5:0/2801879174 conn(0x7f5f040fc2a0 msgr2=0x7f5f04103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.662+0000 7f5f0929a640 1 -- 172.21.15.5:0/2801879174 shutdown_connections 2024-09-19T06:00:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:40.663+0000 7f5f0929a640 1 -- 172.21.15.5:0/2801879174 wait complete. 2024-09-19T06:00:41.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:40 smithi005 ceph-mon[30955]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T06:00:41.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:40 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2801879174' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:00:41.203 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:40 smithi049 ceph-mon[34640]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T06:00:41.203 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:40 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2801879174' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:00:41.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:40 smithi157 ceph-mon[35346]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T06:00:41.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:40 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2801879174' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:00:41.418 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1726725638,"num_in_osds":12,"osd_in_since":1726725615,"num_remapped_pgs":0} 2024-09-19T06:00:41.419 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:00:41.670 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:00:42.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:41 smithi005 ceph-mon[30955]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.105+0000 7f34327f4640 1 -- 172.21.15.5:0/3270341907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f342c102570 msgr2=0x7f342c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.105+0000 7f34327f4640 1 --2- 172.21.15.5:0/3270341907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f342c102570 0x7f342c1029b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3414009900 tx=0x7f341402a480 comp rx=0 tx=0).stop 2024-09-19T06:00:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.105+0000 7f34327f4640 1 -- 172.21.15.5:0/3270341907 shutdown_connections 2024-09-19T06:00:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.105+0000 7f34327f4640 1 --2- 172.21.15.5:0/3270341907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f342c102ef0 0x7f342c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.105+0000 7f34327f4640 1 --2- 172.21.15.5:0/3270341907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f342c102570 0x7f342c1029b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.105+0000 7f34327f4640 1 --2- 172.21.15.5:0/3270341907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f342c100e20 0x7f342c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.105+0000 7f34327f4640 1 -- 172.21.15.5:0/3270341907 >> 172.21.15.5:0/3270341907 conn(0x7f342c0fc7d0 msgr2=0x7f342c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.105+0000 7f34327f4640 1 -- 172.21.15.5:0/3270341907 shutdown_connections 2024-09-19T06:00:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.105+0000 7f34327f4640 1 -- 172.21.15.5:0/3270341907 wait complete. 2024-09-19T06:00:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f34327f4640 1 Processor -- start 2024-09-19T06:00:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f34327f4640 1 -- start start 2024-09-19T06:00:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f34327f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f342c100e20 0x7f342c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f34327f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f342c102570 0x7f342c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f34327f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f342c102ef0 0x7f342c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f34327f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f342c110e50 con 0x7f342c102ef0 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f34327f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f342c110cd0 con 0x7f342c102570 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f34327f4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f342c110fd0 con 0x7f342c100e20 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f3430d91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f342c102ef0 0x7f342c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f3430d91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f342c102ef0 0x7f342c19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52244/0 (socket says 172.21.15.5:52244) 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f3430d91640 1 -- 172.21.15.5:0/3802337058 learned_addr learned my addr 172.21.15.5:0/3802337058 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f342bfff640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f342c100e20 0x7f342c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f3423fff640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f342c102570 0x7f342c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f3430d91640 1 -- 172.21.15.5:0/3802337058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f342c100e20 msgr2=0x7f342c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.106+0000 7f3430d91640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f342c100e20 0x7f342c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.107+0000 7f3430d91640 1 -- 172.21.15.5:0/3802337058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f342c102570 msgr2=0x7f342c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.107+0000 7f3430d91640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f342c102570 0x7f342c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.107+0000 7f3430d91640 1 -- 172.21.15.5:0/3802337058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c19d470 con 0x7f342c102ef0 2024-09-19T06:00:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.107+0000 7f3430d91640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f342c102ef0 0x7f342c19cd90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f342400cad0 tx=0x7f342400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:42.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.107+0000 7f3429ffb640 1 -- 172.21.15.5:0/3802337058 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342400ad60 con 0x7f342c102ef0 2024-09-19T06:00:42.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.107+0000 7f34327f4640 1 -- 172.21.15.5:0/3802337058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c19d760 con 0x7f342c102ef0 2024-09-19T06:00:42.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.107+0000 7f3429ffb640 1 -- 172.21.15.5:0/3802337058 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3424005020 con 0x7f342c102ef0 2024-09-19T06:00:42.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.107+0000 7f3429ffb640 1 -- 172.21.15.5:0/3802337058 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342400f6b0 con 0x7f342c102ef0 2024-09-19T06:00:42.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.107+0000 7f34327f4640 1 -- 172.21.15.5:0/3802337058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1a5040 con 0x7f342c102ef0 2024-09-19T06:00:42.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.108+0000 7f34327f4640 1 -- 172.21.15.5:0/3802337058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c101200 con 0x7f342c102ef0 2024-09-19T06:00:42.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.109+0000 7f3429ffb640 1 -- 172.21.15.5:0/3802337058 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f34240051d0 con 0x7f342c102ef0 2024-09-19T06:00:42.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.110+0000 7f3429ffb640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f34180a3240 0x7f34180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:42.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.110+0000 7f342bfff640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f34180a3240 0x7f34180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:42.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.110+0000 7f3429ffb640 1 -- 172.21.15.5:0/3802337058 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f34240cf920 con 0x7f342c102ef0 2024-09-19T06:00:42.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.110+0000 7f342bfff640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f34180a3240 0x7f34180a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f341c009990 tx=0x7f341c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:42.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.113+0000 7f3429ffb640 1 -- 172.21.15.5:0/3802337058 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3424014030 con 0x7f342c102ef0 2024-09-19T06:00:42.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:41 smithi049 ceph-mon[34640]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:42.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.289+0000 7f34327f4640 1 -- 172.21.15.5:0/3802337058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f342c1a5240 con 0x7f342c102ef0 2024-09-19T06:00:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.290+0000 7f3429ffb640 1 -- 172.21.15.5:0/3802337058 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18395 (secure 0 0 0) 0x7f342409ca40 con 0x7f342c102ef0 2024-09-19T06:00:42.292 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:00:42.292 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":76,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","created":"2024-09-19T05:53:51.454068+0000","modified":"2024-09-19T06:00:39.844339+0000","last_up_change":"2024-09-19T06:00:38.837916+0000","last_in_change":"2024-09-19T06:00:15.609776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:56:55.213931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f3c10d60-d354-407f-9cdf-099344599844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6803","nonce":3442981810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6805","nonce":3442981810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6809","nonce":3442981810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6807","nonce":3442981810}]},"public_addr":"172.21.15.5:6803/3442981810","cluster_addr":"172.21.15.5:6805/3442981810","heartbeat_back_addr":"172.21.15.5:6809/3442981810","heartbeat_front_addr":"172.21.15.5:6807/3442981810","state":["exists","up"]},{"osd":1,"uuid":"5f40f636-7761-45ac-87f3-910a2fa2feb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448265014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6803","nonce":1448265014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6807","nonce":1448265014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6805","nonce":1448265014}]},"public_addr":"172.21.15.49:6801/1448265014","cluster_addr":"172.21.15.49:6803/1448265014","heartbeat_back_addr":"172.21.15.49:6807/1448265014","heartbeat_front_addr":"172.21.15.49:6805/1448265014","state":["exists","up"]},{"osd":2,"uuid":"ad00940c-4fdd-4f5f-9eaa-b1f38bcac526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6801","nonce":2319626622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6803","nonce":2319626622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6807","nonce":2319626622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6805","nonce":2319626622}]},"public_addr":"172.21.15.157:6801/2319626622","cluster_addr":"172.21.15.157:6803/2319626622","heartbeat_back_addr":"172.21.15.157:6807/2319626622","heartbeat_front_addr":"172.21.15.157:6805/2319626622","state":["exists","up"]},{"osd":3,"uuid":"63fec329-1b62-4c5e-b8ca-084a84d4d762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6811","nonce":1452491753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6813","nonce":1452491753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6817","nonce":1452491753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6815","nonce":1452491753}]},"public_addr":"172.21.15.5:6811/1452491753","cluster_addr":"172.21.15.5:6813/1452491753","heartbeat_back_addr":"172.21.15.5:6817/1452491753","heartbeat_front_addr":"172.21.15.5:6815/1452491753","state":["exists","up"]},{"osd":4,"uuid":"ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6809","nonce":3114952580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6811","nonce":3114952580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6815","nonce":3114952580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6813","nonce":3114952580}]},"public_addr":"172.21.15.49:6809/3114952580","cluster_addr":"172.21.15.49:6811/3114952580","heartbeat_back_addr":"172.21.15.49:6815/3114952580","heartbeat_front_addr":"172.21.15.49:6813/3114952580","state":["exists","up"]},{"osd":5,"uuid":"21a7a7d3-ce42-4688-bfec-714be635eaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6809","nonce":3219063971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6811","nonce":3219063971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6815","nonce":3219063971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6813","nonce":3219063971}]},"public_addr":"172.21.15.157:6809/3219063971","cluster_addr":"172.21.15.157:6811/3219063971","heartbeat_back_addr":"172.21.15.157:6815/3219063971","heartbeat_front_addr":"172.21.15.157:6813/3219063971","state":["exists","up"]},{"osd":6,"uuid":"cbef0e08-d6b0-4515-a2ee-2c81b048583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6819","nonce":2957795194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6821","nonce":2957795194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6825","nonce":2957795194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6823","nonce":2957795194}]},"public_addr":"172.21.15.5:6819/2957795194","cluster_addr":"172.21.15.5:6821/2957795194","heartbeat_back_addr":"172.21.15.5:6825/2957795194","heartbeat_front_addr":"172.21.15.5:6823/2957795194","state":["exists","up"]},{"osd":7,"uuid":"d261ac55-532d-4aea-9b3e-ed2b4eeb1da6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6817","nonce":2959731400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6819","nonce":2959731400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6823","nonce":2959731400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6821","nonce":2959731400}]},"public_addr":"172.21.15.49:6817/2959731400","cluster_addr":"172.21.15.49:6819/2959731400","heartbeat_back_addr":"172.21.15.49:6823/2959731400","heartbeat_front_addr":"172.21.15.49:6821/2959731400","state":["exists","up"]},{"osd":8,"uuid":"15d0eaf7-d604-4546-b393-4d6d39018f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6817","nonce":290026859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6819","nonce":290026859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6823","nonce":290026859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6821","nonce":290026859}]},"public_addr":"172.21.15.157:6817/290026859","cluster_addr":"172.21.15.157:6819/290026859","heartbeat_back_addr":"172.21.15.157:6823/290026859","heartbeat_front_addr":"172.21.15.157:6821/290026859","state":["exists","up"]},{"osd":9,"uuid":"30c08537-f593-4299-bfa1-fa7ba8a95169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6827","nonce":4122448237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6829","nonce":4122448237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6833","nonce":4122448237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6831","nonce":4122448237}]},"public_addr":"172.21.15.5:6827/4122448237","cluster_addr":"172.21.15.5:6829/4122448237","heartbeat_back_addr":"172.21.15.5:6833/4122448237","heartbeat_front_addr":"172.21.15.5:6831/4122448237","state":["exists","up"]},{"osd":10,"uuid":"68821e98-bd81-49ec-8cd6-51fb9880b553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6825","nonce":2551748002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6827","nonce":2551748002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6831","nonce":2551748002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6829","nonce":2551748002}]},"public_addr":"172.21.15.49:6825/2551748002","cluster_addr":"172.21.15.49:6827/2551748002","heartbeat_back_addr":"172.21.15.49:6831/2551748002","heartbeat_front_addr":"172.21.15.49:6829/2551748002","state":["exists","up"]},{"osd":11,"uuid":"7e443b46-2697-4d0b-8452-868463a1ee1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6825","nonce":1996582544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6827","nonce":1996582544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6831","nonce":1996582544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6829","nonce":1996582544}]},"public_addr":"172.21.15.157:6825/1996582544","cluster_addr":"172.21.15.157:6827/1996582544","heartbeat_back_addr":"172.21.15.157:6831/1996582544","heartbeat_front_addr":"172.21.15.157:6829/1996582544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:01.453592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:27.194705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:51.525385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:17.676749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:43.075555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:07.258154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:32.112583+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:57.306534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:21.320950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:47.164610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:12.548337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:37.262580+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4102275559":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/2593868621":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/1340147787":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6800/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/850678758":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6800/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2930274429":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/1367598148":"2024-09-20T05:54:15.334457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:00:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.292+0000 7f34327f4640 1 -- 172.21.15.5:0/3802337058 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f34180a3240 msgr2=0x7f34180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.292+0000 7f34327f4640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f34180a3240 0x7f34180a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f341c009990 tx=0x7f341c009110 comp rx=0 tx=0).stop 2024-09-19T06:00:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.292+0000 7f34327f4640 1 -- 172.21.15.5:0/3802337058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f342c102ef0 msgr2=0x7f342c19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.292+0000 7f34327f4640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f342c102ef0 0x7f342c19cd90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f342400cad0 tx=0x7f342400ced0 comp rx=0 tx=0).stop 2024-09-19T06:00:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.292+0000 7f34327f4640 1 -- 172.21.15.5:0/3802337058 shutdown_connections 2024-09-19T06:00:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.292+0000 7f34327f4640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f34180a3240 0x7f34180a5700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.292+0000 7f34327f4640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f342c102ef0 0x7f342c19cd90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.292+0000 7f34327f4640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f342c102570 0x7f342c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.292+0000 7f34327f4640 1 --2- 172.21.15.5:0/3802337058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f342c100e20 0x7f342c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.292+0000 7f34327f4640 1 -- 172.21.15.5:0/3802337058 >> 172.21.15.5:0/3802337058 conn(0x7f342c0fc7d0 msgr2=0x7f342c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.292+0000 7f34327f4640 1 -- 172.21.15.5:0/3802337058 shutdown_connections 2024-09-19T06:00:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:42.292+0000 7f34327f4640 1 -- 172.21.15.5:0/3802337058 wait complete. 2024-09-19T06:00:42.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:41 smithi157 ceph-mon[35346]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:42.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:42 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3802337058' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:00:42.961 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T05:56:55.213931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T06:00:42.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T06:00:43.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:42 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3802337058' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:00:43.218 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:00:43.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:42 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3802337058' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:00:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.691+0000 7ff57ced5640 1 -- 172.21.15.5:0/1017369052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff578106a80 msgr2=0x7ff578106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.691+0000 7ff57ced5640 1 --2- 172.21.15.5:0/1017369052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff578106a80 0x7ff578106e60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff568009930 tx=0x7ff56802a420 comp rx=0 tx=0).stop 2024-09-19T06:00:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.691+0000 7ff57ced5640 1 -- 172.21.15.5:0/1017369052 shutdown_connections 2024-09-19T06:00:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.691+0000 7ff57ced5640 1 --2- 172.21.15.5:0/1017369052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff578101420 0x7ff57810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.691+0000 7ff57ced5640 1 --2- 172.21.15.5:0/1017369052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff578100a80 0x7ff578100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.691+0000 7ff57ced5640 1 --2- 172.21.15.5:0/1017369052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff578106a80 0x7ff578106e60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.691+0000 7ff57ced5640 1 -- 172.21.15.5:0/1017369052 >> 172.21.15.5:0/1017369052 conn(0x7ff5780fc7a0 msgr2=0x7ff5780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.691+0000 7ff57ced5640 1 -- 172.21.15.5:0/1017369052 shutdown_connections 2024-09-19T06:00:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.691+0000 7ff57ced5640 1 -- 172.21.15.5:0/1017369052 wait complete. 2024-09-19T06:00:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.692+0000 7ff57ced5640 1 Processor -- start 2024-09-19T06:00:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.692+0000 7ff57ced5640 1 -- start start 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.692+0000 7ff57ced5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff578100a80 0x7ff5781983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.692+0000 7ff57ced5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff578101420 0x7ff578198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.692+0000 7ff57ced5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff578106a80 0x7ff57819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.692+0000 7ff57ced5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff578110d30 con 0x7ff578106a80 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.692+0000 7ff57ced5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff578110bb0 con 0x7ff578101420 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.692+0000 7ff57ced5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff578110eb0 con 0x7ff578100a80 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff576d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff578106a80 0x7ff57819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff576d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff578106a80 0x7ff57819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52252/0 (socket says 172.21.15.5:52252) 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff575d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff578101420 0x7ff578198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff576d9d640 1 -- 172.21.15.5:0/1228597293 learned_addr learned my addr 172.21.15.5:0/1228597293 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff57659c640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff578100a80 0x7ff5781983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff576d9d640 1 -- 172.21.15.5:0/1228597293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff578100a80 msgr2=0x7ff5781983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff576d9d640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff578100a80 0x7ff5781983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff576d9d640 1 -- 172.21.15.5:0/1228597293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff578101420 msgr2=0x7ff578198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff576d9d640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff578101420 0x7ff578198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff576d9d640 1 -- 172.21.15.5:0/1228597293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57819d390 con 0x7ff578106a80 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff576d9d640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff578106a80 0x7ff57819ccb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff57000cad0 tx=0x7ff57000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff56f7fe640 1 -- 172.21.15.5:0/1228597293 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57000ae20 con 0x7ff578106a80 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff57ced5640 1 -- 172.21.15.5:0/1228597293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57819d680 con 0x7ff578106a80 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff56f7fe640 1 -- 172.21.15.5:0/1228597293 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff570005020 con 0x7ff578106a80 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff56f7fe640 1 -- 172.21.15.5:0/1228597293 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57000f6b0 con 0x7ff578106a80 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.693+0000 7ff57ced5640 1 -- 172.21.15.5:0/1228597293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5781a4f60 con 0x7ff578106a80 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.695+0000 7ff57ced5640 1 -- 172.21.15.5:0/1228597293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff578100ee0 con 0x7ff578106a80 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.695+0000 7ff56f7fe640 1 -- 172.21.15.5:0/1228597293 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7ff5700051d0 con 0x7ff578106a80 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.696+0000 7ff56f7fe640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff5640a3240 0x7ff5640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.696+0000 7ff57659c640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff5640a3240 0x7ff5640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.696+0000 7ff56f7fe640 1 -- 172.21.15.5:0/1228597293 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7ff5700cf3e0 con 0x7ff578106a80 2024-09-19T06:00:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.697+0000 7ff57659c640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff5640a3240 0x7ff5640a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff568006010 tx=0x7ff568000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.700+0000 7ff56f7fe640 1 -- 172.21.15.5:0/1228597293 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff570014030 con 0x7ff578106a80 2024-09-19T06:00:43.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.876+0000 7ff57ced5640 1 -- 172.21.15.5:0/1228597293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff57810a060 con 0x7ff578106a80 2024-09-19T06:00:43.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.877+0000 7ff56f7fe640 1 -- 172.21.15.5:0/1228597293 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7ff57009bf50 con 0x7ff578106a80 2024-09-19T06:00:43.879 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-19T06:00:43.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.879+0000 7ff57ced5640 1 -- 172.21.15.5:0/1228597293 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff5640a3240 msgr2=0x7ff5640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.879+0000 7ff57ced5640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff5640a3240 0x7ff5640a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff568006010 tx=0x7ff568000bc0 comp rx=0 tx=0).stop 2024-09-19T06:00:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.879+0000 7ff57ced5640 1 -- 172.21.15.5:0/1228597293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff578106a80 msgr2=0x7ff57819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.879+0000 7ff57ced5640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff578106a80 0x7ff57819ccb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff57000cad0 tx=0x7ff57000cf90 comp rx=0 tx=0).stop 2024-09-19T06:00:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.879+0000 7ff57ced5640 1 -- 172.21.15.5:0/1228597293 shutdown_connections 2024-09-19T06:00:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.879+0000 7ff57ced5640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff5640a3240 0x7ff5640a5700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.879+0000 7ff57ced5640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff578106a80 0x7ff57819ccb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.879+0000 7ff57ced5640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff578101420 0x7ff578198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.879+0000 7ff57ced5640 1 --2- 172.21.15.5:0/1228597293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff578100a80 0x7ff5781983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.879+0000 7ff57ced5640 1 -- 172.21.15.5:0/1228597293 >> 172.21.15.5:0/1228597293 conn(0x7ff5780fc7a0 msgr2=0x7ff57806ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.879+0000 7ff57ced5640 1 -- 172.21.15.5:0/1228597293 shutdown_connections 2024-09-19T06:00:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:43.879+0000 7ff57ced5640 1 -- 172.21.15.5:0/1228597293 wait complete. 2024-09-19T06:00:44.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:43 smithi005 ceph-mon[30955]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:43 smithi049 ceph-mon[34640]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:44.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:43 smithi157 ceph-mon[35346]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:44.604 INFO:tasks.cephadm:Adding mds.a on smithi005 2024-09-19T06:00:44.604 INFO:tasks.cephadm:Adding mds.d on smithi005 2024-09-19T06:00:44.604 INFO:tasks.cephadm:Adding mds.g on smithi005 2024-09-19T06:00:44.604 INFO:tasks.cephadm:Adding mds.j on smithi005 2024-09-19T06:00:44.604 INFO:tasks.cephadm:Adding mds.b on smithi049 2024-09-19T06:00:44.604 INFO:tasks.cephadm:Adding mds.e on smithi049 2024-09-19T06:00:44.605 INFO:tasks.cephadm:Adding mds.h on smithi049 2024-09-19T06:00:44.605 INFO:tasks.cephadm:Adding mds.k on smithi049 2024-09-19T06:00:44.605 INFO:tasks.cephadm:Adding mds.c on smithi157 2024-09-19T06:00:44.605 INFO:tasks.cephadm:Adding mds.f on smithi157 2024-09-19T06:00:44.605 INFO:tasks.cephadm:Adding mds.i on smithi157 2024-09-19T06:00:44.605 INFO:tasks.cephadm:Adding mds.l on smithi157 2024-09-19T06:00:44.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l' 2024-09-19T06:00:44.864 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:00:44.985 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:44 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1228597293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:00:45.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:44 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1228597293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:00:45.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:44 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1228597293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:00:45.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.289+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/191891412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0101c00 msgr2=0x7fcdf0102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:45.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.289+0000 7fcdf4d2f640 1 --2- 172.21.15.157:0/191891412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0101c00 0x7fcdf0102000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009900 tx=0x7fcde002a410 comp rx=0 tx=0).stop 2024-09-19T06:00:45.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.289+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/191891412 shutdown_connections 2024-09-19T06:00:45.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.289+0000 7fcdf4d2f640 1 --2- 172.21.15.157:0/191891412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdf01037c0 0x7fcdf010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:45.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.289+0000 7fcdf4d2f640 1 --2- 172.21.15.157:0/191891412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf0102e00 0x7fcdf0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:45.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.289+0000 7fcdf4d2f640 1 --2- 172.21.15.157:0/191891412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0101c00 0x7fcdf0102000 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:45.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.289+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/191891412 >> 172.21.15.157:0/191891412 conn(0x7fcdf00fd390 msgr2=0x7fcdf00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:45.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.289+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/191891412 shutdown_connections 2024-09-19T06:00:45.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.289+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/191891412 wait complete. 2024-09-19T06:00:45.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.290+0000 7fcdf4d2f640 1 Processor -- start 2024-09-19T06:00:45.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.290+0000 7fcdf4d2f640 1 -- start start 2024-09-19T06:00:45.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.290+0000 7fcdf4d2f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf0101c00 0x7fcdf0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:45.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.290+0000 7fcdf4d2f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdf0102e00 0x7fcdf00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:45.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.290+0000 7fcdf4d2f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf01037c0 0x7fcdf0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:45.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.290+0000 7fcdf4d2f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf010d7b0 con 0x7fcdf0101c00 2024-09-19T06:00:45.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.290+0000 7fcdf4d2f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf010d630 con 0x7fcdf0102e00 2024-09-19T06:00:45.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.290+0000 7fcdf4d2f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf010d930 con 0x7fcdf01037c0 2024-09-19T06:00:45.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcdeed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf01037c0 0x7fcdf0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:45.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcdeed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf01037c0 0x7fcdf0074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56844/0 (socket says 172.21.15.157:56844) 2024-09-19T06:00:45.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcdeed9d640 1 -- 172.21.15.157:0/3033735664 learned_addr learned my addr 172.21.15.157:0/3033735664 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T06:00:45.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcde7fff640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdf0102e00 0x7fcdf00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcdee59c640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf0101c00 0x7fcdf0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcdeed9d640 1 -- 172.21.15.157:0/3033735664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdf0102e00 msgr2=0x7fcdf00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcdeed9d640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdf0102e00 0x7fcdf00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcdeed9d640 1 -- 172.21.15.157:0/3033735664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf0101c00 msgr2=0x7fcdf0075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcdeed9d640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf0101c00 0x7fcdf0075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcdeed9d640 1 -- 172.21.15.157:0/3033735664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf00753e0 con 0x7fcdf01037c0 2024-09-19T06:00:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcdeed9d640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf01037c0 0x7fcdf0074b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcde800eaa0 tx=0x7fcde800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcde77fe640 1 -- 172.21.15.157:0/3033735664 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8004d20 con 0x7fcdf01037c0 2024-09-19T06:00:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/3033735664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf01a1fe0 con 0x7fcdf01037c0 2024-09-19T06:00:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcde77fe640 1 -- 172.21.15.157:0/3033735664 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcde8005020 con 0x7fcdf01037c0 2024-09-19T06:00:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.291+0000 7fcde77fe640 1 -- 172.21.15.157:0/3033735664 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80116d0 con 0x7fcdf01037c0 2024-09-19T06:00:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.292+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/3033735664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf01a25c0 con 0x7fcdf01037c0 2024-09-19T06:00:45.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.293+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/3033735664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf0109090 con 0x7fcdf01037c0 2024-09-19T06:00:45.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.294+0000 7fcde77fe640 1 -- 172.21.15.157:0/3033735664 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fcde800caf0 con 0x7fcdf01037c0 2024-09-19T06:00:45.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.294+0000 7fcde77fe640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcddc0a32c0 0x7fcddc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:45.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.295+0000 7fcde77fe640 1 -- 172.21.15.157:0/3033735664 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fcde80cf8f0 con 0x7fcdf01037c0 2024-09-19T06:00:45.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.295+0000 7fcdee59c640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcddc0a32c0 0x7fcddc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:45.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.295+0000 7fcdee59c640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcddc0a32c0 0x7fcddc0a5780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcde0000c00 tx=0x7fcde00314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:45.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.298+0000 7fcde77fe640 1 -- 172.21.15.157:0/3033735664 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcde8016030 con 0x7fcdf01037c0 2024-09-19T06:00:45.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.482+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/3033735664 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}) -- 0x7fcdf0062730 con 0x7fcddc0a32c0 2024-09-19T06:00:45.491 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.490+0000 7fcde77fe640 1 -- 172.21.15.157:0/3033735664 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fcdf0062730 con 0x7fcddc0a32c0 2024-09-19T06:00:45.492 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mds.all update... 2024-09-19T06:00:45.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.493+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/3033735664 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcddc0a32c0 msgr2=0x7fcddc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:45.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.493+0000 7fcdf4d2f640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcddc0a32c0 0x7fcddc0a5780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcde0000c00 tx=0x7fcde00314a0 comp rx=0 tx=0).stop 2024-09-19T06:00:45.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.493+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/3033735664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf01037c0 msgr2=0x7fcdf0074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:45.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.493+0000 7fcdf4d2f640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf01037c0 0x7fcdf0074b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcde800eaa0 tx=0x7fcde800eea0 comp rx=0 tx=0).stop 2024-09-19T06:00:45.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.493+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/3033735664 shutdown_connections 2024-09-19T06:00:45.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.493+0000 7fcdf4d2f640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fcddc0a32c0 0x7fcddc0a5780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:45.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.493+0000 7fcdf4d2f640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf01037c0 0x7fcdf0074b20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:45.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.493+0000 7fcdf4d2f640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdf0102e00 0x7fcdf00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:45.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.493+0000 7fcdf4d2f640 1 --2- 172.21.15.157:0/3033735664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf0101c00 0x7fcdf0075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:45.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.493+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/3033735664 >> 172.21.15.157:0/3033735664 conn(0x7fcdf00fd390 msgr2=0x7fcdf0106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:45.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.493+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/3033735664 shutdown_connections 2024-09-19T06:00:45.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T06:00:45.493+0000 7fcdf4d2f640 1 -- 172.21.15.157:0/3033735664 wait complete. 2024-09-19T06:00:46.099 DEBUG:teuthology.orchestra.run.smithi005:mds.a> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T06:00:46.102 DEBUG:teuthology.orchestra.run.smithi005:mds.d> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T06:00:46.147 DEBUG:teuthology.orchestra.run.smithi005:mds.g> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T06:00:46.191 DEBUG:teuthology.orchestra.run.smithi005:mds.j> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T06:00:46.234 DEBUG:teuthology.orchestra.run.smithi049:mds.b> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T06:00:46.237 DEBUG:teuthology.orchestra.run.smithi049:mds.e> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T06:00:46.283 DEBUG:teuthology.orchestra.run.smithi049:mds.h> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T06:00:46.325 DEBUG:teuthology.orchestra.run.smithi049:mds.k> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T06:00:46.369 DEBUG:teuthology.orchestra.run.smithi157:mds.c> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T06:00:46.371 DEBUG:teuthology.orchestra.run.smithi157:mds.f> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T06:00:46.414 DEBUG:teuthology.orchestra.run.smithi157:mds.i> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T06:00:46.457 DEBUG:teuthology.orchestra.run.smithi157:mds.l> sudo journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T06:00:46.500 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T06:00:46.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:00:46.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:46 smithi157 ceph-mon[35346]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:46.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:46 smithi157 ceph-mon[35346]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:00:46.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:46 smithi157 ceph-mon[35346]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-09-19T06:00:46.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:46 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:46.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:46 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:46.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:46 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:46.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:46 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:46.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:46 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:46.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:46 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:46.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:46 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:46.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:46 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:46.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:46 smithi049 ceph-mon[34640]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:46.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:46 smithi049 ceph-mon[34640]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:00:46.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:46 smithi049 ceph-mon[34640]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-09-19T06:00:46.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:46 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:46.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:46 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:46.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:46 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:46.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:46 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:46.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:46 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:46.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:46 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:46.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:46 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:46.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:46 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:00:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:46 smithi005 ceph-mon[30955]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:46 smithi005 ceph-mon[30955]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:00:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:46 smithi005 ceph-mon[30955]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-09-19T06:00:46.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:46 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:46.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:46 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:00:46.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:46 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:46.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:46 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:00:46.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:46 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:46.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:46 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:46.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:46 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:46.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:46 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:47.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.213+0000 7ff5516b6640 1 -- 172.21.15.5:0/2006069085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c103660 msgr2=0x7ff54c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:47.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.213+0000 7ff5516b6640 1 --2- 172.21.15.5:0/2006069085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c103660 0x7ff54c10bb50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff534009860 tx=0x7ff53402a3d0 comp rx=0 tx=0).stop 2024-09-19T06:00:47.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.213+0000 7ff5516b6640 1 -- 172.21.15.5:0/2006069085 shutdown_connections 2024-09-19T06:00:47.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.213+0000 7ff5516b6640 1 --2- 172.21.15.5:0/2006069085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c10c170 0x7ff54c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:47.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.213+0000 7ff5516b6640 1 --2- 172.21.15.5:0/2006069085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c103660 0x7ff54c10bb50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:47.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.213+0000 7ff5516b6640 1 --2- 172.21.15.5:0/2006069085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff54c102d40 0x7ff54c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:47.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.213+0000 7ff5516b6640 1 -- 172.21.15.5:0/2006069085 >> 172.21.15.5:0/2006069085 conn(0x7ff54c0fc7a0 msgr2=0x7ff54c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.213+0000 7ff5516b6640 1 -- 172.21.15.5:0/2006069085 shutdown_connections 2024-09-19T06:00:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.213+0000 7ff5516b6640 1 -- 172.21.15.5:0/2006069085 wait complete. 2024-09-19T06:00:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff5516b6640 1 Processor -- start 2024-09-19T06:00:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff5516b6640 1 -- start start 2024-09-19T06:00:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff5516b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c102d40 0x7ff54c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff5516b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff54c103660 0x7ff54c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff5516b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c10c170 0x7ff54c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff5516b6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c111e20 con 0x7ff54c102d40 2024-09-19T06:00:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff5516b6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c111ca0 con 0x7ff54c103660 2024-09-19T06:00:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff5516b6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c111fa0 con 0x7ff54c10c170 2024-09-19T06:00:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff54affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c102d40 0x7ff54c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff54affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c102d40 0x7ff54c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57488/0 (socket says 172.21.15.5:57488) 2024-09-19T06:00:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff54affd640 1 -- 172.21.15.5:0/312965043 learned_addr learned my addr 172.21.15.5:0/312965043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:00:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff54b7fe640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c10c170 0x7ff54c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff54a7fc640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff54c103660 0x7ff54c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff54affd640 1 -- 172.21.15.5:0/312965043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c10c170 msgr2=0x7ff54c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff54affd640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c10c170 0x7ff54c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff54affd640 1 -- 172.21.15.5:0/312965043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff54c103660 msgr2=0x7ff54c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.214+0000 7ff54affd640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff54c103660 0x7ff54c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.215+0000 7ff54affd640 1 -- 172.21.15.5:0/312965043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54c19d370 con 0x7ff54c102d40 2024-09-19T06:00:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.215+0000 7ff54b7fe640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c10c170 0x7ff54c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:00:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.215+0000 7ff54affd640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c102d40 0x7ff54c1983c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e9c0 tx=0x7ff53c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.215+0000 7ff543fff640 1 -- 172.21.15.5:0/312965043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00ccc0 con 0x7ff54c102d40 2024-09-19T06:00:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.215+0000 7ff5516b6640 1 -- 172.21.15.5:0/312965043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54c19d660 con 0x7ff54c102d40 2024-09-19T06:00:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.215+0000 7ff543fff640 1 -- 172.21.15.5:0/312965043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff53c0054d0 con 0x7ff54c102d40 2024-09-19T06:00:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.215+0000 7ff543fff640 1 -- 172.21.15.5:0/312965043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c010430 con 0x7ff54c102d40 2024-09-19T06:00:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.215+0000 7ff5516b6640 1 -- 172.21.15.5:0/312965043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54c1a4f40 con 0x7ff54c102d40 2024-09-19T06:00:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.216+0000 7ff5516b6640 1 -- 172.21.15.5:0/312965043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff54c103120 con 0x7ff54c102d40 2024-09-19T06:00:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.217+0000 7ff543fff640 1 -- 172.21.15.5:0/312965043 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7ff53c0046c0 con 0x7ff54c102d40 2024-09-19T06:00:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.218+0000 7ff543fff640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff5380a3240 0x7ff5380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.218+0000 7ff54a7fc640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff5380a3240 0x7ff5380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.218+0000 7ff543fff640 1 -- 172.21.15.5:0/312965043 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7ff53c0cfa00 con 0x7ff54c102d40 2024-09-19T06:00:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.218+0000 7ff54a7fc640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff5380a3240 0x7ff5380a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0fee10 tx=0x7ff534009540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.221+0000 7ff543fff640 1 -- 172.21.15.5:0/312965043 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff53c054210 con 0x7ff54c102d40 2024-09-19T06:00:47.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.432+0000 7ff5516b6640 1 -- 172.21.15.5:0/312965043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff54c107210 con 0x7ff54c102d40 2024-09-19T06:00:47.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.433+0000 7ff543fff640 1 -- 172.21.15.5:0/312965043 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7ff53c09cbc0 con 0x7ff54c102d40 2024-09-19T06:00:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:00:47.436 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":1,"btime":"2024-09-19T05:53:51:453912+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T06:00:47.436 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 1 2024-09-19T06:00:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.435+0000 7ff5516b6640 1 -- 172.21.15.5:0/312965043 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff5380a3240 msgr2=0x7ff5380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.435+0000 7ff5516b6640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff5380a3240 0x7ff5380a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0fee10 tx=0x7ff534009540 comp rx=0 tx=0).stop 2024-09-19T06:00:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.435+0000 7ff5516b6640 1 -- 172.21.15.5:0/312965043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c102d40 msgr2=0x7ff54c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.435+0000 7ff5516b6640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c102d40 0x7ff54c1983c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e9c0 tx=0x7ff53c00ee80 comp rx=0 tx=0).stop 2024-09-19T06:00:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.435+0000 7ff5516b6640 1 -- 172.21.15.5:0/312965043 shutdown_connections 2024-09-19T06:00:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.435+0000 7ff5516b6640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff5380a3240 0x7ff5380a5700 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.435+0000 7ff5516b6640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c10c170 0x7ff54c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.435+0000 7ff5516b6640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff54c103660 0x7ff54c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.435+0000 7ff5516b6640 1 --2- 172.21.15.5:0/312965043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c102d40 0x7ff54c1983c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.435+0000 7ff5516b6640 1 -- 172.21.15.5:0/312965043 >> 172.21.15.5:0/312965043 conn(0x7ff54c0fc7a0 msgr2=0x7ff54c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.435+0000 7ff5516b6640 1 -- 172.21.15.5:0/312965043 shutdown_connections 2024-09-19T06:00:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:47.435+0000 7ff5516b6640 1 -- 172.21.15.5:0/312965043 wait complete. 2024-09-19T06:00:47.545 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:47 smithi157 ceph-mon[35346]: Deploying daemon mds.l on smithi157 2024-09-19T06:00:47.545 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:47 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/312965043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:00:47.801 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:00:47 smithi157 systemd[1]: Starting Ceph mds.l for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:00:47.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:47 smithi005 ceph-mon[30955]: Deploying daemon mds.l on smithi157 2024-09-19T06:00:47.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:47 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/312965043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:00:47.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:47 smithi049 ceph-mon[34640]: Deploying daemon mds.l on smithi157 2024-09-19T06:00:47.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:47 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/312965043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:00:48.282 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:00:47 smithi157 podman[64939]: 2024-09-19 06:00:47.871940636 +0000 UTC m=+0.015657824 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:00:48.282 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:00:47 smithi157 podman[64939]: 2024-09-19 06:00:47.977928376 +0000 UTC m=+0.121645555 container create 45250dc07c390cbde5a89a2e3a82ddab7c8f0794b007a6d0103f70f30af08e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-19T06:00:48.282 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:00:48 smithi157 podman[64939]: 2024-09-19 06:00:48.188613986 +0000 UTC m=+0.332331170 container init 45250dc07c390cbde5a89a2e3a82ddab7c8f0794b007a6d0103f70f30af08e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:00:48.282 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:00:48 smithi157 podman[64939]: 2024-09-19 06:00:48.191402741 +0000 UTC m=+0.335119921 container start 45250dc07c390cbde5a89a2e3a82ddab7c8f0794b007a6d0103f70f30af08e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:00:48.282 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:00:48 smithi157 ceph-mds[64959]: -- 172.21.15.157:0/792410795 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3fb761860 con 0x55b3fb73f000 2024-09-19T06:00:48.282 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:00:48 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-l[64955]: starting mds.l at 2024-09-19T06:00:48.282 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:00:48 smithi157 bash[64939]: 45250dc07c390cbde5a89a2e3a82ddab7c8f0794b007a6d0103f70f30af08e2a 2024-09-19T06:00:48.551 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:00:48 smithi157 systemd[1]: Started Ceph mds.l for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:00:48.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:48 smithi157 ceph-mon[35346]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:48.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:48 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.157:6832/1048665474,v1:172.21.15.157:6833/1048665474] up:boot 2024-09-19T06:00:48.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:48 smithi157 ceph-mon[35346]: fsmap 1 up:standby 2024-09-19T06:00:48.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:00:48.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:48.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:48.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:48.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:48.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:48.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:48 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:48.780 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:48 smithi049 ceph-mon[34640]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:48.780 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:48 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.157:6832/1048665474,v1:172.21.15.157:6833/1048665474] up:boot 2024-09-19T06:00:48.780 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:48 smithi049 ceph-mon[34640]: fsmap 1 up:standby 2024-09-19T06:00:48.780 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:00:48.780 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:48.780 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:48.780 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:48.780 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:48.781 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:48.781 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:48 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:48.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:48 smithi005 ceph-mon[30955]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:48.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:48 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.157:6832/1048665474,v1:172.21.15.157:6833/1048665474] up:boot 2024-09-19T06:00:48.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:48 smithi005 ceph-mon[30955]: fsmap 1 up:standby 2024-09-19T06:00:48.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:00:48.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:48.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:48.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:48.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:48.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:48.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:48 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:49.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:49 smithi157 ceph-mon[35346]: Deploying daemon mds.k on smithi049 2024-09-19T06:00:49.868 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:49 smithi049 ceph-mon[34640]: Deploying daemon mds.k on smithi049 2024-09-19T06:00:49.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:49 smithi005 ceph-mon[30955]: Deploying daemon mds.k on smithi049 2024-09-19T06:00:50.134 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:00:50.400 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:00:50.503 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:00:50 smithi049 systemd[1]: Starting Ceph mds.k for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:00:50.774 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:50 smithi005 ceph-mon[30955]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:50 smithi157 ceph-mon[35346]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:50.823 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:50 smithi049 ceph-mon[34640]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:50.824 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:00:50 smithi049 podman[64142]: 2024-09-19 06:00:50.459388061 +0000 UTC m=+0.016341154 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:00:50.824 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:00:50 smithi049 podman[64142]: 2024-09-19 06:00:50.595141882 +0000 UTC m=+0.152094964 container create bc1bd613a9967427c9dfbff367f39bc58e3e7fad9d4251cb6b68586b794856cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-k, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T06:00:50.824 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:00:50 smithi049 podman[64142]: 2024-09-19 06:00:50.822839321 +0000 UTC m=+0.379792403 container init bc1bd613a9967427c9dfbff367f39bc58e3e7fad9d4251cb6b68586b794856cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T06:00:50.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.853+0000 7f4e46119640 1 -- 172.21.15.5:0/2485093052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e40106f10 msgr2=0x7f4e4010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:50.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.853+0000 7f4e46119640 1 --2- 172.21.15.5:0/2485093052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e40106f10 0x7f4e4010ea90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009ca0 tx=0x7f4e3802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:00:50.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.853+0000 7f4e46119640 1 -- 172.21.15.5:0/2485093052 shutdown_connections 2024-09-19T06:00:50.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.853+0000 7f4e46119640 1 --2- 172.21.15.5:0/2485093052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e40106f10 0x7f4e4010ea90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:50.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.853+0000 7f4e46119640 1 --2- 172.21.15.5:0/2485093052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e40102790 0x7f4e401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:50.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.853+0000 7f4e46119640 1 --2- 172.21.15.5:0/2485093052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e40101de0 0x7f4e401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:50.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.853+0000 7f4e46119640 1 -- 172.21.15.5:0/2485093052 >> 172.21.15.5:0/2485093052 conn(0x7f4e400fd8c0 msgr2=0x7f4e400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:50.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.853+0000 7f4e46119640 1 -- 172.21.15.5:0/2485093052 shutdown_connections 2024-09-19T06:00:50.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.854+0000 7f4e46119640 1 -- 172.21.15.5:0/2485093052 wait complete. 2024-09-19T06:00:50.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.854+0000 7f4e46119640 1 Processor -- start 2024-09-19T06:00:50.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.854+0000 7f4e46119640 1 -- start start 2024-09-19T06:00:50.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.854+0000 7f4e46119640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e40101de0 0x7f4e40198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:50.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.854+0000 7f4e46119640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e40102790 0x7f4e40198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:50.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.854+0000 7f4e46119640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e40106f10 0x7f4e4019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.854+0000 7f4e46119640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e40111e20 con 0x7f4e40101de0 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.854+0000 7f4e46119640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e40111ca0 con 0x7f4e40102790 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.854+0000 7f4e46119640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e40111fa0 con 0x7f4e40106f10 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e40101de0 0x7f4e40198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e40101de0 0x7f4e40198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57506/0 (socket says 172.21.15.5:57506) 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3f7fe640 1 -- 172.21.15.5:0/3570457762 learned_addr learned my addr 172.21.15.5:0/3570457762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3effd640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e40102790 0x7f4e40198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3f7fe640 1 -- 172.21.15.5:0/3570457762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e40106f10 msgr2=0x7f4e4019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3ffff640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e40106f10 0x7f4e4019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3f7fe640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e40106f10 0x7f4e4019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3f7fe640 1 -- 172.21.15.5:0/3570457762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e40102790 msgr2=0x7f4e40198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3f7fe640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e40102790 0x7f4e40198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:50.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3f7fe640 1 -- 172.21.15.5:0/3570457762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4019d2d0 con 0x7f4e40101de0 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3ffff640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e40106f10 0x7f4e4019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3f7fe640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e40101de0 0x7f4e40198320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000d770 tx=0x7f4e3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3cff9640 1 -- 172.21.15.5:0/3570457762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30017070 con 0x7f4e40101de0 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3cff9640 1 -- 172.21.15.5:0/3570457762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e300054d0 con 0x7f4e40101de0 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e46119640 1 -- 172.21.15.5:0/3570457762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4019d5c0 con 0x7f4e40101de0 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e3cff9640 1 -- 172.21.15.5:0/3570457762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30004cf0 con 0x7f4e40101de0 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.855+0000 7f4e46119640 1 -- 172.21.15.5:0/3570457762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e401a4ea0 con 0x7f4e40101de0 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.856+0000 7f4e46119640 1 -- 172.21.15.5:0/3570457762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e401021c0 con 0x7f4e40101de0 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.858+0000 7f4e3cff9640 1 -- 172.21.15.5:0/3570457762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f4e30004e90 con 0x7f4e40101de0 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.858+0000 7f4e3cff9640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4e280a3240 0x7f4e280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.858+0000 7f4e3effd640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4e280a3240 0x7f4e280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.858+0000 7f4e3cff9640 1 -- 172.21.15.5:0/3570457762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f4e300cef80 con 0x7f4e40101de0 2024-09-19T06:00:50.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.859+0000 7f4e3effd640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4e280a3240 0x7f4e280a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4e400fdce0 tx=0x7f4e34005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:50.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:50.862+0000 7f4e3cff9640 1 -- 172.21.15.5:0/3570457762 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4e3009c0a0 con 0x7f4e40101de0 2024-09-19T06:00:51.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.075+0000 7f4e46119640 1 -- 172.21.15.5:0/3570457762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e40199680 con 0x7f4e40101de0 2024-09-19T06:00:51.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.076+0000 7f4e3cff9640 1 -- 172.21.15.5:0/3570457762 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f4e3009b9a0 con 0x7f4e40101de0 2024-09-19T06:00:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:00:51.079 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"btime":"2024-09-19T06:00:48:219451+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T06:00:51.079 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2024-09-19T06:00:51.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.078+0000 7f4e46119640 1 -- 172.21.15.5:0/3570457762 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4e280a3240 msgr2=0x7f4e280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:51.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.078+0000 7f4e46119640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4e280a3240 0x7f4e280a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4e400fdce0 tx=0x7f4e34005f90 comp rx=0 tx=0).stop 2024-09-19T06:00:51.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.078+0000 7f4e46119640 1 -- 172.21.15.5:0/3570457762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e40101de0 msgr2=0x7f4e40198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:51.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.078+0000 7f4e46119640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e40101de0 0x7f4e40198320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000d770 tx=0x7f4e3000dc30 comp rx=0 tx=0).stop 2024-09-19T06:00:51.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.078+0000 7f4e46119640 1 -- 172.21.15.5:0/3570457762 shutdown_connections 2024-09-19T06:00:51.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.078+0000 7f4e46119640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4e280a3240 0x7f4e280a5700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:51.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.078+0000 7f4e46119640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e40106f10 0x7f4e4019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:51.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.078+0000 7f4e46119640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e40102790 0x7f4e40198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:51.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.078+0000 7f4e46119640 1 --2- 172.21.15.5:0/3570457762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e40101de0 0x7f4e40198320 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:51.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.078+0000 7f4e46119640 1 -- 172.21.15.5:0/3570457762 >> 172.21.15.5:0/3570457762 conn(0x7f4e400fd8c0 msgr2=0x7f4e400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:51.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.078+0000 7f4e46119640 1 -- 172.21.15.5:0/3570457762 shutdown_connections 2024-09-19T06:00:51.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:51.078+0000 7f4e46119640 1 -- 172.21.15.5:0/3570457762 wait complete. 2024-09-19T06:00:51.146 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:00:50 smithi049 podman[64142]: 2024-09-19 06:00:50.825853341 +0000 UTC m=+0.382806423 container start bc1bd613a9967427c9dfbff367f39bc58e3e7fad9d4251cb6b68586b794856cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-19T06:00:51.146 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:00:50 smithi049 ceph-mds[64164]: -- 172.21.15.49:0/1875244104 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b52b465860 con 0x55b52b442800 2024-09-19T06:00:51.147 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:00:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-k[64160]: starting mds.k at 2024-09-19T06:00:51.147 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:00:51 smithi049 bash[64142]: bc1bd613a9967427c9dfbff367f39bc58e3e7fad9d4251cb6b68586b794856cf 2024-09-19T06:00:51.147 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:00:51 smithi049 systemd[1]: Started Ceph mds.k for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:00:51.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:51 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3570457762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:00:51.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:51 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:51.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:51 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:51.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:51 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:51.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:51 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:51.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:51 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:51.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:51 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:51 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3570457762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:00:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:51 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:51 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:51 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:51 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:51.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:51 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:51.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:51 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:51 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3570457762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:00:51.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:51 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:51.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:51 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:51.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:51 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:51.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:51 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:51.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:51 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:51.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:51 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:52 smithi005 ceph-mon[30955]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:52 smithi005 ceph-mon[30955]: Deploying daemon mds.j on smithi005 2024-09-19T06:00:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:52 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.49:6832/2427602354,v1:172.21.15.49:6833/2427602354] up:boot 2024-09-19T06:00:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:52 smithi005 ceph-mon[30955]: fsmap 2 up:standby 2024-09-19T06:00:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:52 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:00:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:52 smithi157 ceph-mon[35346]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:52 smithi157 ceph-mon[35346]: Deploying daemon mds.j on smithi005 2024-09-19T06:00:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:52 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.49:6832/2427602354,v1:172.21.15.49:6833/2427602354] up:boot 2024-09-19T06:00:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:52 smithi157 ceph-mon[35346]: fsmap 2 up:standby 2024-09-19T06:00:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:52 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:00:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:52 smithi049 ceph-mon[34640]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:52 smithi049 ceph-mon[34640]: Deploying daemon mds.j on smithi005 2024-09-19T06:00:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:52 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.49:6832/2427602354,v1:172.21.15.49:6833/2427602354] up:boot 2024-09-19T06:00:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:52 smithi049 ceph-mon[34640]: fsmap 2 up:standby 2024-09-19T06:00:52.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:52 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:00:53.630 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:00:53 smithi005 systemd[1]: Starting Ceph mds.j for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:00:53.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:00:54.000 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:00:53 smithi005 podman[68103]: 2024-09-19 06:00:53.615354892 +0000 UTC m=+0.016135187 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:00:54.001 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:00:53 smithi005 podman[68103]: 2024-09-19 06:00:53.746351186 +0000 UTC m=+0.147131506 container create 8b209cf01c7a97c1db2eab1dbf31580ab5a14741cac8d7e304315c00b746e449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-j, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T06:00:54.001 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:00:53 smithi005 podman[68103]: 2024-09-19 06:00:53.999124433 +0000 UTC m=+0.399904721 container init 8b209cf01c7a97c1db2eab1dbf31580ab5a14741cac8d7e304315c00b746e449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-j, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:00:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:00:54.300 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:00:54 smithi005 podman[68103]: 2024-09-19 06:00:54.002393205 +0000 UTC m=+0.403173484 container start 8b209cf01c7a97c1db2eab1dbf31580ab5a14741cac8d7e304315c00b746e449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T06:00:54.300 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:00:54 smithi005 ceph-mds[68153]: -- 172.21.15.5:0/1928467104 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9da165860 con 0x55e9da143000 2024-09-19T06:00:54.300 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:00:54 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-j[68136]: starting mds.j at 2024-09-19T06:00:54.300 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:00:54 smithi005 bash[68103]: 8b209cf01c7a97c1db2eab1dbf31580ab5a14741cac8d7e304315c00b746e449 2024-09-19T06:00:54.300 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:00:54 smithi005 systemd[1]: Started Ceph mds.j for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:00:54.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:54 smithi005 ceph-mon[30955]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:54.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:54 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:54.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:54 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:54.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:54 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:54.580 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:54 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:54.580 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:54 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:54.580 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:54 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.660+0000 7fe1936af640 1 -- 172.21.15.5:0/55405371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe18c1008d0 msgr2=0x7fe18c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.660+0000 7fe1936af640 1 --2- 172.21.15.5:0/55405371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe18c1008d0 0x7fe18c100cb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe188009900 tx=0x7fe18802a3f0 comp rx=0 tx=0).stop 2024-09-19T06:00:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.660+0000 7fe1936af640 1 -- 172.21.15.5:0/55405371 shutdown_connections 2024-09-19T06:00:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.660+0000 7fe1936af640 1 --2- 172.21.15.5:0/55405371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe18c10e480 0x7fe18c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.660+0000 7fe1936af640 1 --2- 172.21.15.5:0/55405371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe18c101280 0x7fe18c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.660+0000 7fe1936af640 1 --2- 172.21.15.5:0/55405371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe18c1008d0 0x7fe18c100cb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.660+0000 7fe1936af640 1 -- 172.21.15.5:0/55405371 >> 172.21.15.5:0/55405371 conn(0x7fe18c0fc7a0 msgr2=0x7fe18c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.660+0000 7fe1936af640 1 -- 172.21.15.5:0/55405371 shutdown_connections 2024-09-19T06:00:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.661+0000 7fe1936af640 1 -- 172.21.15.5:0/55405371 wait complete. 2024-09-19T06:00:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.661+0000 7fe1936af640 1 Processor -- start 2024-09-19T06:00:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.661+0000 7fe1936af640 1 -- start start 2024-09-19T06:00:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.661+0000 7fe1936af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe18c1008d0 0x7fe18c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.661+0000 7fe1936af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe18c101280 0x7fe18c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.661+0000 7fe1936af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe18c10e480 0x7fe18c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.661+0000 7fe1936af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c114060 con 0x7fe18c10e480 2024-09-19T06:00:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.661+0000 7fe1936af640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c113ee0 con 0x7fe18c101280 2024-09-19T06:00:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.661+0000 7fe1936af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1141e0 con 0x7fe18c1008d0 2024-09-19T06:00:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.661+0000 7fe191c4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe18c10e480 0x7fe18c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.662+0000 7fe191c4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe18c10e480 0x7fe18c19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51160/0 (socket says 172.21.15.5:51160) 2024-09-19T06:00:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.662+0000 7fe190c4a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe18c101280 0x7fe18c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.662+0000 7fe191c4c640 1 -- 172.21.15.5:0/1395299861 learned_addr learned my addr 172.21.15.5:0/1395299861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:00:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.662+0000 7fe19144b640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe18c1008d0 0x7fe18c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.662+0000 7fe191c4c640 1 -- 172.21.15.5:0/1395299861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe18c1008d0 msgr2=0x7fe18c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.662+0000 7fe191c4c640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe18c1008d0 0x7fe18c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.662+0000 7fe191c4c640 1 -- 172.21.15.5:0/1395299861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe18c101280 msgr2=0x7fe18c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.662+0000 7fe191c4c640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe18c101280 0x7fe18c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.662+0000 7fe191c4c640 1 -- 172.21.15.5:0/1395299861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c19f4f0 con 0x7fe18c10e480 2024-09-19T06:00:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.662+0000 7fe191c4c640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe18c10e480 0x7fe18c19ee10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe184009fd0 tx=0x7fe18400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.662+0000 7fe1827fc640 1 -- 172.21.15.5:0/1395299861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184004d20 con 0x7fe18c10e480 2024-09-19T06:00:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.662+0000 7fe1827fc640 1 -- 172.21.15.5:0/1395299861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe184005020 con 0x7fe18c10e480 2024-09-19T06:00:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.663+0000 7fe1827fc640 1 -- 172.21.15.5:0/1395299861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840116b0 con 0x7fe18c10e480 2024-09-19T06:00:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.663+0000 7fe1936af640 1 -- 172.21.15.5:0/1395299861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c19f7e0 con 0x7fe18c10e480 2024-09-19T06:00:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.663+0000 7fe1936af640 1 -- 172.21.15.5:0/1395299861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18c1a70c0 con 0x7fe18c10e480 2024-09-19T06:00:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.664+0000 7fe1936af640 1 -- 172.21.15.5:0/1395299861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18c100cb0 con 0x7fe18c10e480 2024-09-19T06:00:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.665+0000 7fe1827fc640 1 -- 172.21.15.5:0/1395299861 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fe1840051d0 con 0x7fe18c10e480 2024-09-19T06:00:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.665+0000 7fe1827fc640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe1780a3390 0x7fe1780a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.666+0000 7fe19144b640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe1780a3390 0x7fe1780a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.666+0000 7fe1827fc640 1 -- 172.21.15.5:0/1395299861 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fe1840cf840 con 0x7fe18c10e480 2024-09-19T06:00:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.666+0000 7fe19144b640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe1780a3390 0x7fe1780a5850 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe188031440 tx=0x7fe188005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.670+0000 7fe1827fc640 1 -- 172.21.15.5:0/1395299861 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe184016030 con 0x7fe18c10e480 2024-09-19T06:00:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:54 smithi157 ceph-mon[35346]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:54.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:54 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:54.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:54 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:54.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:54 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:54.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:54 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:54.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:54 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:54.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:54 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.884+0000 7fe1936af640 1 -- 172.21.15.5:0/1395299861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe18c02d7c0 con 0x7fe18c10e480 2024-09-19T06:00:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.885+0000 7fe1827fc640 1 -- 172.21.15.5:0/1395299861 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3082 (secure 0 0 0) 0x7fe18409ca00 con 0x7fe18c10e480 2024-09-19T06:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"btime":"2024-09-19T06:00:54:524388+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T06:00:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 4 2024-09-19T06:00:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.887+0000 7fe1936af640 1 -- 172.21.15.5:0/1395299861 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe1780a3390 msgr2=0x7fe1780a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.887+0000 7fe1936af640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe1780a3390 0x7fe1780a5850 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe188031440 tx=0x7fe188005b50 comp rx=0 tx=0).stop 2024-09-19T06:00:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.887+0000 7fe1936af640 1 -- 172.21.15.5:0/1395299861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe18c10e480 msgr2=0x7fe18c19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.887+0000 7fe1936af640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe18c10e480 0x7fe18c19ee10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe184009fd0 tx=0x7fe18400eda0 comp rx=0 tx=0).stop 2024-09-19T06:00:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.887+0000 7fe1936af640 1 -- 172.21.15.5:0/1395299861 shutdown_connections 2024-09-19T06:00:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.887+0000 7fe1936af640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe1780a3390 0x7fe1780a5850 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.887+0000 7fe1936af640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe18c10e480 0x7fe18c19ee10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.887+0000 7fe1936af640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe18c101280 0x7fe18c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.887+0000 7fe1936af640 1 --2- 172.21.15.5:0/1395299861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe18c1008d0 0x7fe18c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.887+0000 7fe1936af640 1 -- 172.21.15.5:0/1395299861 >> 172.21.15.5:0/1395299861 conn(0x7fe18c0fc7a0 msgr2=0x7fe18c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.887+0000 7fe1936af640 1 -- 172.21.15.5:0/1395299861 shutdown_connections 2024-09-19T06:00:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:54.887+0000 7fe1936af640 1 -- 172.21.15.5:0/1395299861 wait complete. 2024-09-19T06:00:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:54 smithi049 ceph-mon[34640]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:54 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:54 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:54 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:54.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:54 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:54.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:54 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:54.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:54 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:55.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:55 smithi157 ceph-mon[35346]: Deploying daemon mds.d on smithi005 2024-09-19T06:00:55.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:55 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.5:6834/959534083,v1:172.21.15.5:6835/959534083] up:boot 2024-09-19T06:00:55.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:55 smithi157 ceph-mon[35346]: fsmap 3 up:standby 2024-09-19T06:00:55.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:55 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:00:55.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:55 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1395299861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:00:55.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:55 smithi005 ceph-mon[30955]: Deploying daemon mds.d on smithi005 2024-09-19T06:00:55.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:55 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.5:6834/959534083,v1:172.21.15.5:6835/959534083] up:boot 2024-09-19T06:00:55.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:55 smithi005 ceph-mon[30955]: fsmap 3 up:standby 2024-09-19T06:00:55.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:55 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:00:55.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:55 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1395299861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:00:55.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:55 smithi049 ceph-mon[34640]: Deploying daemon mds.d on smithi005 2024-09-19T06:00:55.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:55 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.5:6834/959534083,v1:172.21.15.5:6835/959534083] up:boot 2024-09-19T06:00:55.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:55 smithi049 ceph-mon[34640]: fsmap 3 up:standby 2024-09-19T06:00:55.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:55 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:00:55.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:55 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1395299861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:00:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:56 smithi157 ceph-mon[35346]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:56.851 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:56 smithi005 ceph-mon[30955]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:56 smithi049 ceph-mon[34640]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:57.129 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:00:57 smithi005 systemd[1]: Starting Ceph mds.d for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:00:57.511 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:00:57 smithi005 podman[68517]: 2024-09-19 06:00:57.144350277 +0000 UTC m=+0.016482577 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:00:57.511 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:00:57 smithi005 podman[68517]: 2024-09-19 06:00:57.299583341 +0000 UTC m=+0.171715625 container create 3923c8f34b2c13eb6bcbb3a45f342b99a30d431867cb0dd4d63ed95429f03a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:00:57.511 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:00:57 smithi005 podman[68517]: 2024-09-19 06:00:57.510316917 +0000 UTC m=+0.382449209 container init 3923c8f34b2c13eb6bcbb3a45f342b99a30d431867cb0dd4d63ed95429f03a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-d, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T06:00:57.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:00:57.879 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:00:57 smithi005 podman[68517]: 2024-09-19 06:00:57.513629636 +0000 UTC m=+0.385761928 container start 3923c8f34b2c13eb6bcbb3a45f342b99a30d431867cb0dd4d63ed95429f03a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T06:00:57.879 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:00:57 smithi005 ceph-mds[68536]: -- 172.21.15.5:0/1303043461 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555930293860 con 0x555930271400 2024-09-19T06:00:57.879 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:00:57 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-d[68532]: starting mds.d at 2024-09-19T06:00:57.880 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:00:57 smithi005 bash[68517]: 3923c8f34b2c13eb6bcbb3a45f342b99a30d431867cb0dd4d63ed95429f03a42 2024-09-19T06:00:57.880 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:00:57 smithi005 systemd[1]: Started Ceph mds.d for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:00:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:00:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.463+0000 7f417191d640 1 -- 172.21.15.5:0/1336212598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f416c1036e0 msgr2=0x7f416c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.463+0000 7f417191d640 1 --2- 172.21.15.5:0/1336212598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f416c1036e0 0x7f416c107b90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4160009930 tx=0x7f416002a420 comp rx=0 tx=0).stop 2024-09-19T06:00:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.463+0000 7f417191d640 1 -- 172.21.15.5:0/1336212598 shutdown_connections 2024-09-19T06:00:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.463+0000 7f417191d640 1 --2- 172.21.15.5:0/1336212598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c1082a0 0x7f416c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.463+0000 7f417191d640 1 --2- 172.21.15.5:0/1336212598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f416c1036e0 0x7f416c107b90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.463+0000 7f417191d640 1 --2- 172.21.15.5:0/1336212598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f416c102d30 0x7f416c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.463+0000 7f417191d640 1 -- 172.21.15.5:0/1336212598 >> 172.21.15.5:0/1336212598 conn(0x7f416c0fc7d0 msgr2=0x7f416c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.463+0000 7f417191d640 1 -- 172.21.15.5:0/1336212598 shutdown_connections 2024-09-19T06:00:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.463+0000 7f417191d640 1 -- 172.21.15.5:0/1336212598 wait complete. 2024-09-19T06:00:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.464+0000 7f417191d640 1 Processor -- start 2024-09-19T06:00:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.464+0000 7f417191d640 1 -- start start 2024-09-19T06:00:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.464+0000 7f417191d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f416c102d30 0x7f416c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.464+0000 7f417191d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c1036e0 0x7f416c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.464+0000 7f417191d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f416c1082a0 0x7f416c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.464+0000 7f417191d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f416c111f70 con 0x7f416c102d30 2024-09-19T06:00:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.464+0000 7f417191d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f416c111df0 con 0x7f416c1036e0 2024-09-19T06:00:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.464+0000 7f417191d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1120f0 con 0x7f416c1082a0 2024-09-19T06:00:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f416affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f416c102d30 0x7f416c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f416affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f416c102d30 0x7f416c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51194/0 (socket says 172.21.15.5:51194) 2024-09-19T06:00:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f416affd640 1 -- 172.21.15.5:0/641609231 learned_addr learned my addr 172.21.15.5:0/641609231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:00:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f416a7fc640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c1036e0 0x7f416c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f416b7fe640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f416c1082a0 0x7f416c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f416affd640 1 -- 172.21.15.5:0/641609231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f416c1082a0 msgr2=0x7f416c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f416affd640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f416c1082a0 0x7f416c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f416affd640 1 -- 172.21.15.5:0/641609231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c1036e0 msgr2=0x7f416c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f416affd640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c1036e0 0x7f416c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f416affd640 1 -- 172.21.15.5:0/641609231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1a6a30 con 0x7f416c102d30 2024-09-19T06:00:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f416affd640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f416c102d30 0x7f416c100990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f415c00b4a0 tx=0x7f415c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f417091b640 1 -- 172.21.15.5:0/641609231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c014070 con 0x7f416c102d30 2024-09-19T06:00:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f417091b640 1 -- 172.21.15.5:0/641609231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f415c010ac0 con 0x7f416c102d30 2024-09-19T06:00:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f417091b640 1 -- 172.21.15.5:0/641609231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c017c30 con 0x7f416c102d30 2024-09-19T06:00:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f417191d640 1 -- 172.21.15.5:0/641609231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1a6d20 con 0x7f416c102d30 2024-09-19T06:00:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.465+0000 7f417191d640 1 -- 172.21.15.5:0/641609231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1a7220 con 0x7f416c102d30 2024-09-19T06:00:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.466+0000 7f417191d640 1 -- 172.21.15.5:0/641609231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c06a850 con 0x7f416c102d30 2024-09-19T06:00:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.468+0000 7f417091b640 1 -- 172.21.15.5:0/641609231 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f415c017070 con 0x7f416c102d30 2024-09-19T06:00:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.468+0000 7f417091b640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f41540a32c0 0x7f41540a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:00:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.468+0000 7f416a7fc640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f41540a32c0 0x7f41540a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:00:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.468+0000 7f417091b640 1 -- 172.21.15.5:0/641609231 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f415c0ceb40 con 0x7f416c102d30 2024-09-19T06:00:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.469+0000 7f416a7fc640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f41540a32c0 0x7f41540a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4160031440 tx=0x7f4160000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:00:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.473+0000 7f417091b640 1 -- 172.21.15.5:0/641609231 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f415c09bc60 con 0x7f416c102d30 2024-09-19T06:00:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.686+0000 7f417191d640 1 -- 172.21.15.5:0/641609231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f416c02d7c0 con 0x7f416c102d30 2024-09-19T06:00:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.687+0000 7f417091b640 1 -- 172.21.15.5:0/641609231 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3904 (secure 0 0 0) 0x7f415c09b560 con 0x7f416c102d30 2024-09-19T06:00:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:00:58.689 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"btime":"2024-09-19T06:00:58:536583+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T06:00:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 5 2024-09-19T06:00:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.689+0000 7f417191d640 1 -- 172.21.15.5:0/641609231 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f41540a32c0 msgr2=0x7f41540a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.689+0000 7f417191d640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f41540a32c0 0x7f41540a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4160031440 tx=0x7f4160000bc0 comp rx=0 tx=0).stop 2024-09-19T06:00:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.689+0000 7f417191d640 1 -- 172.21.15.5:0/641609231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f416c102d30 msgr2=0x7f416c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:00:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.689+0000 7f417191d640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f416c102d30 0x7f416c100990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f415c00b4a0 tx=0x7f415c00b960 comp rx=0 tx=0).stop 2024-09-19T06:00:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.689+0000 7f417191d640 1 -- 172.21.15.5:0/641609231 shutdown_connections 2024-09-19T06:00:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.689+0000 7f417191d640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f41540a32c0 0x7f41540a5780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.689+0000 7f417191d640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f416c1082a0 0x7f416c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.689+0000 7f417191d640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c1036e0 0x7f416c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.689+0000 7f417191d640 1 --2- 172.21.15.5:0/641609231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f416c102d30 0x7f416c100990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:00:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.689+0000 7f417191d640 1 -- 172.21.15.5:0/641609231 >> 172.21.15.5:0/641609231 conn(0x7f416c0fc7d0 msgr2=0x7f416c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:00:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.689+0000 7f417191d640 1 -- 172.21.15.5:0/641609231 shutdown_connections 2024-09-19T06:00:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:00:58.689+0000 7f417191d640 1 -- 172.21.15.5:0/641609231 wait complete. 2024-09-19T06:00:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:58 smithi049 ceph-mon[34640]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:58.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:58.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:58.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:58.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:58.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:58.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:58 smithi049 ceph-mon[34640]: Deploying daemon mds.e on smithi049 2024-09-19T06:00:58.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:58 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.5:6836/3191633882,v1:172.21.15.5:6837/3191633882] up:boot 2024-09-19T06:00:58.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:58 smithi049 ceph-mon[34640]: fsmap 4 up:standby 2024-09-19T06:00:58.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:58 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:00:59.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:58 smithi005 ceph-mon[30955]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:59.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:59.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:59.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:59.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:59.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:59.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:59.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:58 smithi005 ceph-mon[30955]: Deploying daemon mds.e on smithi049 2024-09-19T06:00:59.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:58 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.5:6836/3191633882,v1:172.21.15.5:6837/3191633882] up:boot 2024-09-19T06:00:59.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:58 smithi005 ceph-mon[30955]: fsmap 4 up:standby 2024-09-19T06:00:59.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:58 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:00:59.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:58 smithi157 ceph-mon[35346]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:00:59.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:00:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:00:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:00:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:00:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:58 smithi157 ceph-mon[35346]: Deploying daemon mds.e on smithi049 2024-09-19T06:00:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:58 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.5:6836/3191633882,v1:172.21.15.5:6837/3191633882] up:boot 2024-09-19T06:00:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:58 smithi157 ceph-mon[35346]: fsmap 4 up:standby 2024-09-19T06:00:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:58 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:01:00.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:00:59 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/641609231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:00.090 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:00:59 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/641609231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:00.090 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:00:59 smithi049 systemd[1]: Starting Ceph mds.e for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:01:00.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:00:59 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/641609231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:00.397 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:01:00 smithi049 podman[64455]: 2024-09-19 06:00:59.990873518 +0000 UTC m=+0.016295514 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:01:00.397 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:01:00 smithi049 podman[64455]: 2024-09-19 06:01:00.100042237 +0000 UTC m=+0.125464248 container create d8d4fef20850f9954e44fcae320b7d63ad779ad4982066924079c1bd8c9b7d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T06:01:00.397 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:01:00 smithi049 podman[64455]: 2024-09-19 06:01:00.3106103 +0000 UTC m=+0.336032289 container init d8d4fef20850f9954e44fcae320b7d63ad779ad4982066924079c1bd8c9b7d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:01:00.397 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:01:00 smithi049 podman[64455]: 2024-09-19 06:01:00.313528054 +0000 UTC m=+0.338950031 container start d8d4fef20850f9954e44fcae320b7d63ad779ad4982066924079c1bd8c9b7d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T06:01:00.397 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:01:00 smithi049 ceph-mds[64474]: -- 172.21.15.49:0/55593847 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56149e38d860 con 0x56149e36b000 2024-09-19T06:01:00.397 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:01:00 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e[64470]: starting mds.e at 2024-09-19T06:01:00.676 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:01:00 smithi049 bash[64455]: d8d4fef20850f9954e44fcae320b7d63ad779ad4982066924079c1bd8c9b7d12 2024-09-19T06:01:00.676 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:01:00 smithi049 systemd[1]: Started Ceph mds.e for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:01:00.940 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:00 smithi157 ceph-mon[35346]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:00.940 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:00.940 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:00.940 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:00.940 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:00.940 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:00.940 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:00 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:00 smithi005 ceph-mon[30955]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:01.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:01.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:01.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:01.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:00 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:01.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:00 smithi049 ceph-mon[34640]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:01.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:01.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:01.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:01.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:01.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:01.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:00 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:01.397 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:01:01.668 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:01.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:01 smithi157 ceph-mon[35346]: Deploying daemon mds.f on smithi157 2024-09-19T06:01:01.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:01 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.49:6834/2353689058,v1:172.21.15.49:6835/2353689058] up:boot 2024-09-19T06:01:01.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:01 smithi157 ceph-mon[35346]: fsmap 5 up:standby 2024-09-19T06:01:01.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:01 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:01:01.939 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:01 smithi005 ceph-mon[30955]: Deploying daemon mds.f on smithi157 2024-09-19T06:01:01.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:01 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.49:6834/2353689058,v1:172.21.15.49:6835/2353689058] up:boot 2024-09-19T06:01:01.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:01 smithi005 ceph-mon[30955]: fsmap 5 up:standby 2024-09-19T06:01:01.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:01 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:01:02.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.120+0000 7f3f91dd2640 1 -- 172.21.15.5:0/1742563081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f8c103660 msgr2=0x7f3f8c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:02.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.120+0000 7f3f91dd2640 1 --2- 172.21.15.5:0/1742563081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f8c103660 0x7f3f8c10bb50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009900 tx=0x7f3f8002a480 comp rx=0 tx=0).stop 2024-09-19T06:01:02.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.120+0000 7f3f91dd2640 1 -- 172.21.15.5:0/1742563081 shutdown_connections 2024-09-19T06:01:02.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.120+0000 7f3f91dd2640 1 --2- 172.21.15.5:0/1742563081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f8c10c170 0x7f3f8c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:02.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.120+0000 7f3f91dd2640 1 --2- 172.21.15.5:0/1742563081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f8c103660 0x7f3f8c10bb50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:02.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.120+0000 7f3f91dd2640 1 --2- 172.21.15.5:0/1742563081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c102d40 0x7f3f8c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:02.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.120+0000 7f3f91dd2640 1 -- 172.21.15.5:0/1742563081 >> 172.21.15.5:0/1742563081 conn(0x7f3f8c0fc7a0 msgr2=0x7f3f8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:02.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.120+0000 7f3f91dd2640 1 -- 172.21.15.5:0/1742563081 shutdown_connections 2024-09-19T06:01:02.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.120+0000 7f3f91dd2640 1 -- 172.21.15.5:0/1742563081 wait complete. 2024-09-19T06:01:02.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.120+0000 7f3f91dd2640 1 Processor -- start 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.120+0000 7f3f91dd2640 1 -- start start 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f91dd2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c102d40 0x7f3f8c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f91dd2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f8c103660 0x7f3f8c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f91dd2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f8c10c170 0x7f3f8c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f91dd2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c111e70 con 0x7f3f8c10c170 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f91dd2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c111cf0 con 0x7f3f8c102d40 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f91dd2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c111ff0 con 0x7f3f8c103660 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f8c10c170 0x7f3f8c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f8c10c170 0x7f3f8c19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51210/0 (socket says 172.21.15.5:51210) 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f8bfff640 1 -- 172.21.15.5:0/3620013780 learned_addr learned my addr 172.21.15.5:0/3620013780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f8affd640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f8c103660 0x7f3f8c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f8b7fe640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c102d40 0x7f3f8c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:02.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f8bfff640 1 -- 172.21.15.5:0/3620013780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f8c103660 msgr2=0x7f3f8c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f8bfff640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f8c103660 0x7f3f8c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f8bfff640 1 -- 172.21.15.5:0/3620013780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c102d40 msgr2=0x7f3f8c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f8bfff640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c102d40 0x7f3f8c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f8bfff640 1 -- 172.21.15.5:0/3620013780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c19d340 con 0x7f3f8c10c170 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.121+0000 7f3f8bfff640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f8c10c170 0x7f3f8c19cc60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400eaa0 tx=0x7f3f8400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.122+0000 7f3f88ff9640 1 -- 172.21.15.5:0/3620013780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8400cdb0 con 0x7f3f8c10c170 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.122+0000 7f3f91dd2640 1 -- 172.21.15.5:0/3620013780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8c19d630 con 0x7f3f8c10c170 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.122+0000 7f3f88ff9640 1 -- 172.21.15.5:0/3620013780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f84005020 con 0x7f3f8c10c170 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.122+0000 7f3f88ff9640 1 -- 172.21.15.5:0/3620013780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f840116d0 con 0x7f3f8c10c170 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.122+0000 7f3f91dd2640 1 -- 172.21.15.5:0/3620013780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8c1a4f10 con 0x7f3f8c10c170 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.123+0000 7f3f91dd2640 1 -- 172.21.15.5:0/3620013780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8c103120 con 0x7f3f8c10c170 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.124+0000 7f3f88ff9640 1 -- 172.21.15.5:0/3620013780 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f3f840051d0 con 0x7f3f8c10c170 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.124+0000 7f3f88ff9640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3f740a3240 0x7f3f740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:02.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.125+0000 7f3f8b7fe640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3f740a3240 0x7f3f740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:02.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.125+0000 7f3f88ff9640 1 -- 172.21.15.5:0/3620013780 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f3f840cf4d0 con 0x7f3f8c10c170 2024-09-19T06:01:02.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.125+0000 7f3f8b7fe640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3f740a3240 0x7f3f740a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c005d00 tx=0x7f3f7c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:02.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.129+0000 7f3f88ff9640 1 -- 172.21.15.5:0/3620013780 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f84016030 con 0x7f3f8c10c170 2024-09-19T06:01:02.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:01 smithi049 ceph-mon[34640]: Deploying daemon mds.f on smithi157 2024-09-19T06:01:02.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:01 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.49:6834/2353689058,v1:172.21.15.49:6835/2353689058] up:boot 2024-09-19T06:01:02.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:01 smithi049 ceph-mon[34640]: fsmap 5 up:standby 2024-09-19T06:01:02.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:01 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:01:02.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.338+0000 7f3f91dd2640 1 -- 172.21.15.5:0/3620013780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f8c02d7c0 con 0x7f3f8c10c170 2024-09-19T06:01:02.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.339+0000 7f3f88ff9640 1 -- 172.21.15.5:0/3620013780 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4729 (secure 0 0 0) 0x7f3f8409c040 con 0x7f3f8c10c170 2024-09-19T06:01:02.341 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:02.341 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":6,"btime":"2024-09-19T06:01:00:673694+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T06:01:02.341 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 6 2024-09-19T06:01:02.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.342+0000 7f3f91dd2640 1 -- 172.21.15.5:0/3620013780 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3f740a3240 msgr2=0x7f3f740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:02.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.342+0000 7f3f91dd2640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3f740a3240 0x7f3f740a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c005d00 tx=0x7f3f7c00b410 comp rx=0 tx=0).stop 2024-09-19T06:01:02.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.342+0000 7f3f91dd2640 1 -- 172.21.15.5:0/3620013780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f8c10c170 msgr2=0x7f3f8c19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:02.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.342+0000 7f3f91dd2640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f8c10c170 0x7f3f8c19cc60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400eaa0 tx=0x7f3f8400ef60 comp rx=0 tx=0).stop 2024-09-19T06:01:02.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.342+0000 7f3f91dd2640 1 -- 172.21.15.5:0/3620013780 shutdown_connections 2024-09-19T06:01:02.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.342+0000 7f3f91dd2640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3f740a3240 0x7f3f740a5700 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:02.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.342+0000 7f3f91dd2640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f8c10c170 0x7f3f8c19cc60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:02.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.342+0000 7f3f91dd2640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f8c103660 0x7f3f8c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:02.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.342+0000 7f3f91dd2640 1 --2- 172.21.15.5:0/3620013780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c102d40 0x7f3f8c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:02.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.342+0000 7f3f91dd2640 1 -- 172.21.15.5:0/3620013780 >> 172.21.15.5:0/3620013780 conn(0x7f3f8c0fc7a0 msgr2=0x7f3f8c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:02.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.342+0000 7f3f91dd2640 1 -- 172.21.15.5:0/3620013780 shutdown_connections 2024-09-19T06:01:02.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:02.343+0000 7f3f91dd2640 1 -- 172.21.15.5:0/3620013780 wait complete. 2024-09-19T06:01:02.552 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:01:02 smithi157 systemd[1]: Starting Ceph mds.f for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:01:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:02 smithi005 ceph-mon[30955]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:02 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3620013780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:02.900 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:02 smithi157 ceph-mon[35346]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:02.900 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:02 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3620013780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:02.900 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:01:02 smithi157 podman[65255]: 2024-09-19 06:01:02.531449943 +0000 UTC m=+0.015860439 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:01:02.900 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:01:02 smithi157 podman[65255]: 2024-09-19 06:01:02.655120907 +0000 UTC m=+0.139531394 container create 0b945e16d702cc6a73ec2cff6e05752772e54c9d4356a58a0345e161fe2395ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T06:01:03.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:02 smithi049 ceph-mon[34640]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:03.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:02 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3620013780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:03.301 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:01:02 smithi157 podman[65255]: 2024-09-19 06:01:02.899336611 +0000 UTC m=+0.383747103 container init 0b945e16d702cc6a73ec2cff6e05752772e54c9d4356a58a0345e161fe2395ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T06:01:03.301 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:01:02 smithi157 podman[65255]: 2024-09-19 06:01:02.902323911 +0000 UTC m=+0.386734411 container start 0b945e16d702cc6a73ec2cff6e05752772e54c9d4356a58a0345e161fe2395ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T06:01:03.301 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:01:02 smithi157 ceph-mds[65276]: -- 172.21.15.157:0/18477970 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bde0fc7860 con 0x55bde0fa4800 2024-09-19T06:01:03.301 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:01:02 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f[65272]: starting mds.f at 2024-09-19T06:01:03.301 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:01:03 smithi157 bash[65255]: 0b945e16d702cc6a73ec2cff6e05752772e54c9d4356a58a0345e161fe2395ea 2024-09-19T06:01:03.302 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:01:03 smithi157 systemd[1]: Started Ceph mds.f for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:01:04.199 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:04 smithi157 ceph-mon[35346]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:04.199 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:04 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:04.199 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:04 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:04.199 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:04 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:04.200 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:04 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:04.200 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:04 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:04.200 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:04 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:04.200 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:04 smithi157 ceph-mon[35346]: Deploying daemon mds.c on smithi157 2024-09-19T06:01:04.200 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:04 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] up:boot 2024-09-19T06:01:04.200 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:04 smithi157 ceph-mon[35346]: fsmap 6 up:standby 2024-09-19T06:01:04.200 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:04 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:01:04.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:04 smithi005 ceph-mon[30955]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:04.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:04 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:04.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:04 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:04.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:04 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:04.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:04 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:04.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:04 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:04.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:04 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:04.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:04 smithi005 ceph-mon[30955]: Deploying daemon mds.c on smithi157 2024-09-19T06:01:04.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:04 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] up:boot 2024-09-19T06:01:04.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:04 smithi005 ceph-mon[30955]: fsmap 6 up:standby 2024-09-19T06:01:04.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:04 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:01:04.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:04 smithi049 ceph-mon[34640]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:04.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:04 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:04.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:04 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:04.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:04 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:04.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:04 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:04.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:04 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:04.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:04 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:04.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:04 smithi049 ceph-mon[34640]: Deploying daemon mds.c on smithi157 2024-09-19T06:01:04.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:04 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] up:boot 2024-09-19T06:01:04.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:04 smithi049 ceph-mon[34640]: fsmap 6 up:standby 2024-09-19T06:01:04.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:04 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:01:05.051 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:01:04 smithi157 systemd[1]: Starting Ceph mds.c for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:01:05.157 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:01:05.429 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:05.431 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:01:05 smithi157 podman[65561]: 2024-09-19 06:01:05.082896143 +0000 UTC m=+0.015826905 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:01:05.432 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:01:05 smithi157 podman[65561]: 2024-09-19 06:01:05.200679625 +0000 UTC m=+0.133610377 container create 2cdb769b3739ebb7d118655d3d40eeeb285c62b67fba00c7119e6f899ff430a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T06:01:05.432 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:01:05 smithi157 podman[65561]: 2024-09-19 06:01:05.402984122 +0000 UTC m=+0.335914874 container init 2cdb769b3739ebb7d118655d3d40eeeb285c62b67fba00c7119e6f899ff430a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T06:01:05.432 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:01:05 smithi157 podman[65561]: 2024-09-19 06:01:05.405782177 +0000 UTC m=+0.338712941 container start 2cdb769b3739ebb7d118655d3d40eeeb285c62b67fba00c7119e6f899ff430a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T06:01:05.432 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:01:05 smithi157 ceph-mds[65580]: -- 172.21.15.157:0/2325988043 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55616e0c1860 con 0x55616e09f000 2024-09-19T06:01:05.801 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:01:05 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-c[65576]: starting mds.c at 2024-09-19T06:01:05.801 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:01:05 smithi157 bash[65561]: 2cdb769b3739ebb7d118655d3d40eeeb285c62b67fba00c7119e6f899ff430a4 2024-09-19T06:01:05.801 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:01:05 smithi157 systemd[1]: Started Ceph mds.c for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:01:05.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.970+0000 7fedd0700640 1 -- 172.21.15.5:0/1488709735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedc8108300 msgr2=0x7fedc810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:05.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.970+0000 7fedd0700640 1 --2- 172.21.15.5:0/1488709735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedc8108300 0x7fedc810ec70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009cd0 tx=0x7fedc002a8d0 comp rx=0 tx=0).stop 2024-09-19T06:01:05.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.970+0000 7fedd0700640 1 -- 172.21.15.5:0/1488709735 shutdown_connections 2024-09-19T06:01:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.970+0000 7fedd0700640 1 --2- 172.21.15.5:0/1488709735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedc8108300 0x7fedc810ec70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.970+0000 7fedd0700640 1 --2- 172.21.15.5:0/1488709735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc8103780 0x7fedc8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.970+0000 7fedd0700640 1 --2- 172.21.15.5:0/1488709735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedc8102dd0 0x7fedc81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.970+0000 7fedd0700640 1 -- 172.21.15.5:0/1488709735 >> 172.21.15.5:0/1488709735 conn(0x7fedc80fc7d0 msgr2=0x7fedc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.970+0000 7fedd0700640 1 -- 172.21.15.5:0/1488709735 shutdown_connections 2024-09-19T06:01:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.970+0000 7fedd0700640 1 -- 172.21.15.5:0/1488709735 wait complete. 2024-09-19T06:01:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.971+0000 7fedd0700640 1 Processor -- start 2024-09-19T06:01:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.971+0000 7fedd0700640 1 -- start start 2024-09-19T06:01:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.971+0000 7fedd0700640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc8102dd0 0x7fedc8100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.971+0000 7fedd0700640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedc8103780 0x7fedc8100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.971+0000 7fedd0700640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedc8108300 0x7fedc81014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.971+0000 7fedd0700640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedc8111fe0 con 0x7fedc8103780 2024-09-19T06:01:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.971+0000 7fedd0700640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedc8111e60 con 0x7fedc8108300 2024-09-19T06:01:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.971+0000 7fedd0700640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fedc8112160 con 0x7fedc8102dd0 2024-09-19T06:01:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.971+0000 7fedcdc9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedc8103780 0x7fedc8100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.971+0000 7fedcdc9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedc8103780 0x7fedc8100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35576/0 (socket says 172.21.15.5:35576) 2024-09-19T06:01:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.971+0000 7fedcdc9b640 1 -- 172.21.15.5:0/2987328051 learned_addr learned my addr 172.21.15.5:0/2987328051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.971+0000 7fedce49c640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc8102dd0 0x7fedc8100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedcec9d640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedc8108300 0x7fedc81014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedcdc9b640 1 -- 172.21.15.5:0/2987328051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc8102dd0 msgr2=0x7fedc8100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedcdc9b640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc8102dd0 0x7fedc8100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedcdc9b640 1 -- 172.21.15.5:0/2987328051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedc8108300 msgr2=0x7fedc81014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedcdc9b640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedc8108300 0x7fedc81014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedcdc9b640 1 -- 172.21.15.5:0/2987328051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc81a69e0 con 0x7fedc8103780 2024-09-19T06:01:05.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedcec9d640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedc8108300 0x7fedc81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:01:05.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedcdc9b640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedc8103780 0x7fedc8100f70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fedb800c970 tx=0x7fedb800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:05.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedbf7fe640 1 -- 172.21.15.5:0/2987328051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb8004220 con 0x7fedc8103780 2024-09-19T06:01:05.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedbf7fe640 1 -- 172.21.15.5:0/2987328051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedb80043c0 con 0x7fedc8103780 2024-09-19T06:01:05.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedd0700640 1 -- 172.21.15.5:0/2987328051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc81a6cd0 con 0x7fedc8103780 2024-09-19T06:01:05.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedbf7fe640 1 -- 172.21.15.5:0/2987328051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb8004a00 con 0x7fedc8103780 2024-09-19T06:01:05.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.972+0000 7fedd0700640 1 -- 172.21.15.5:0/2987328051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc81a71c0 con 0x7fedc8103780 2024-09-19T06:01:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.973+0000 7fedd0700640 1 -- 172.21.15.5:0/2987328051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc81031b0 con 0x7fedc8103780 2024-09-19T06:01:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.974+0000 7fedbf7fe640 1 -- 172.21.15.5:0/2987328051 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fedb8004ba0 con 0x7fedc8103780 2024-09-19T06:01:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.975+0000 7fedbf7fe640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fedb40a32c0 0x7fedb40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.975+0000 7fedce49c640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fedb40a32c0 0x7fedb40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.975+0000 7fedbf7fe640 1 -- 172.21.15.5:0/2987328051 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fedb80cf530 con 0x7fedc8103780 2024-09-19T06:01:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.976+0000 7fedce49c640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fedb40a32c0 0x7fedb40a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009990 tx=0x7fedc4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:05.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:05.978+0000 7fedbf7fe640 1 -- 172.21.15.5:0/2987328051 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fedb8014030 con 0x7fedc8103780 2024-09-19T06:01:06.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.190+0000 7fedd0700640 1 -- 172.21.15.5:0/2987328051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedc810a290 con 0x7fedc8103780 2024-09-19T06:01:06.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.191+0000 7fedbf7fe640 1 -- 172.21.15.5:0/2987328051 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6385 (secure 0 0 0) 0x7fedb809c770 con 0x7fedc8103780 2024-09-19T06:01:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:06.193 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"btime":"2024-09-19T06:01:06:058848+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T06:01:06.193 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-09-19T06:01:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.193+0000 7fedd0700640 1 -- 172.21.15.5:0/2987328051 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fedb40a32c0 msgr2=0x7fedb40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.193+0000 7fedd0700640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fedb40a32c0 0x7fedb40a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009990 tx=0x7fedc4009110 comp rx=0 tx=0).stop 2024-09-19T06:01:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.193+0000 7fedd0700640 1 -- 172.21.15.5:0/2987328051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedc8103780 msgr2=0x7fedc8100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.193+0000 7fedd0700640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedc8103780 0x7fedc8100f70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fedb800c970 tx=0x7fedb800cd70 comp rx=0 tx=0).stop 2024-09-19T06:01:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.193+0000 7fedd0700640 1 -- 172.21.15.5:0/2987328051 shutdown_connections 2024-09-19T06:01:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.193+0000 7fedd0700640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fedb40a32c0 0x7fedb40a5780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.193+0000 7fedd0700640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedc8108300 0x7fedc81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.193+0000 7fedd0700640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedc8103780 0x7fedc8100f70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.193+0000 7fedd0700640 1 --2- 172.21.15.5:0/2987328051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc8102dd0 0x7fedc8100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.193+0000 7fedd0700640 1 -- 172.21.15.5:0/2987328051 >> 172.21.15.5:0/2987328051 conn(0x7fedc80fc7d0 msgr2=0x7fedc8105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.193+0000 7fedd0700640 1 -- 172.21.15.5:0/2987328051 shutdown_connections 2024-09-19T06:01:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:06.193+0000 7fedd0700640 1 -- 172.21.15.5:0/2987328051 wait complete. 2024-09-19T06:01:06.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:06 smithi049 ceph-mon[34640]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:06.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:06.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:06.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:06.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:06.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:06.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:06.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:06 smithi049 ceph-mon[34640]: Deploying daemon mds.h on smithi049 2024-09-19T06:01:06.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:06 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] up:boot 2024-09-19T06:01:06.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:06 smithi049 ceph-mon[34640]: fsmap 7 up:standby 2024-09-19T06:01:06.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:06 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:01:06.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:06 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2987328051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:06 smithi157 ceph-mon[35346]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:06.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:06.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:06.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:06.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:06.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:06.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:06 smithi157 ceph-mon[35346]: Deploying daemon mds.h on smithi049 2024-09-19T06:01:06.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:06 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] up:boot 2024-09-19T06:01:06.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:06 smithi157 ceph-mon[35346]: fsmap 7 up:standby 2024-09-19T06:01:06.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:06 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:01:06.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:06 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2987328051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:06 smithi005 ceph-mon[30955]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:06.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:06.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:06.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:06 smithi005 ceph-mon[30955]: Deploying daemon mds.h on smithi049 2024-09-19T06:01:06.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:06 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] up:boot 2024-09-19T06:01:06.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:06 smithi005 ceph-mon[30955]: fsmap 7 up:standby 2024-09-19T06:01:06.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:06 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:01:06.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:06 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2987328051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:07.646 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:01:07 smithi049 systemd[1]: Starting Ceph mds.h for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:01:07.991 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:01:07 smithi049 podman[64773]: 2024-09-19 06:01:07.629852817 +0000 UTC m=+0.017238250 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:01:07.991 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:01:07 smithi049 podman[64773]: 2024-09-19 06:01:07.745732251 +0000 UTC m=+0.133117673 container create c5637008da4752bc17648beb2827202928ac0cc7349d0e8e0345be266953c530 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:01:07.991 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:01:07 smithi049 podman[64773]: 2024-09-19 06:01:07.989947772 +0000 UTC m=+0.377333205 container init c5637008da4752bc17648beb2827202928ac0cc7349d0e8e0345be266953c530 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-h, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T06:01:08.396 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:01:07 smithi049 podman[64773]: 2024-09-19 06:01:07.993266416 +0000 UTC m=+0.380651838 container start c5637008da4752bc17648beb2827202928ac0cc7349d0e8e0345be266953c530 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-h, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T06:01:08.397 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:01:08 smithi049 ceph-mds[64792]: -- 172.21.15.49:0/2741616558 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56549fab5860 con 0x56549fa93400 2024-09-19T06:01:08.397 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:01:08 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-h[64788]: starting mds.h at 2024-09-19T06:01:08.397 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:01:08 smithi049 bash[64773]: c5637008da4752bc17648beb2827202928ac0cc7349d0e8e0345be266953c530 2024-09-19T06:01:08.397 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:01:08 smithi049 systemd[1]: Started Ceph mds.h for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:01:08.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:01:08.806 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:08 smithi157 ceph-mon[35346]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:08.807 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:08.807 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:08.807 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:08.807 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:08.807 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:08.807 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:08 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:08.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:08 smithi005 ceph-mon[30955]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:08.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:08.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:08.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:08.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:08.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:08.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:08 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:08 smithi049 ceph-mon[34640]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:08.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:08.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:08.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:08.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:08.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:08.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:08 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:09.052 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.554+0000 7f6b5ef69640 1 -- 172.21.15.5:0/2467784404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b58106a80 msgr2=0x7f6b58106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.554+0000 7f6b5ef69640 1 --2- 172.21.15.5:0/2467784404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b58106a80 0x7f6b58106e60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00d940 tx=0x7f6b4c02c3b0 comp rx=0 tx=0).stop 2024-09-19T06:01:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.554+0000 7f6b5ef69640 1 -- 172.21.15.5:0/2467784404 shutdown_connections 2024-09-19T06:01:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.554+0000 7f6b5ef69640 1 --2- 172.21.15.5:0/2467784404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101420 0x7f6b5810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.554+0000 7f6b5ef69640 1 --2- 172.21.15.5:0/2467784404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b58100a80 0x7f6b58100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.554+0000 7f6b5ef69640 1 --2- 172.21.15.5:0/2467784404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b58106a80 0x7f6b58106e60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.554+0000 7f6b5ef69640 1 -- 172.21.15.5:0/2467784404 >> 172.21.15.5:0/2467784404 conn(0x7f6b580fc7a0 msgr2=0x7f6b580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.554+0000 7f6b5ef69640 1 -- 172.21.15.5:0/2467784404 shutdown_connections 2024-09-19T06:01:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.554+0000 7f6b5ef69640 1 -- 172.21.15.5:0/2467784404 wait complete. 2024-09-19T06:01:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5ef69640 1 Processor -- start 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5ef69640 1 -- start start 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5ef69640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b58100a80 0x7f6b5819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5ef69640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101420 0x7f6b5819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5ef69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b58106a80 0x7f6b5819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5ef69640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5810fc20 con 0x7f6b58106a80 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5ef69640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5810faa0 con 0x7f6b58101420 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5ef69640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5810fda0 con 0x7f6b58100a80 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5cd05640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b58100a80 0x7f6b5819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5d506640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b58106a80 0x7f6b5819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5cd05640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b58100a80 0x7f6b5819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:37710/0 (socket says 172.21.15.5:37710) 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5cd05640 1 -- 172.21.15.5:0/865566459 learned_addr learned my addr 172.21.15.5:0/865566459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.555+0000 7f6b5d506640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b58106a80 0x7f6b5819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35584/0 (socket says 172.21.15.5:35584) 2024-09-19T06:01:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.556+0000 7f6b57fff640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101420 0x7f6b5819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.556+0000 7f6b5d506640 1 -- 172.21.15.5:0/865566459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b58100a80 msgr2=0x7f6b5819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.556+0000 7f6b5d506640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b58100a80 0x7f6b5819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.556+0000 7f6b5d506640 1 -- 172.21.15.5:0/865566459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101420 msgr2=0x7f6b5819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.556+0000 7f6b5d506640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101420 0x7f6b5819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.556+0000 7f6b5d506640 1 -- 172.21.15.5:0/865566459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5819f580 con 0x7f6b58106a80 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.556+0000 7f6b5cd05640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b58100a80 0x7f6b5819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.556+0000 7f6b5d506640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b58106a80 0x7f6b5819eea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000e940 tx=0x7f6b5000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.558+0000 7f6b55ffb640 1 -- 172.21.15.5:0/865566459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b50004d20 con 0x7f6b58106a80 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.558+0000 7f6b55ffb640 1 -- 172.21.15.5:0/865566459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b50005020 con 0x7f6b58106a80 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.558+0000 7f6b5ef69640 1 -- 172.21.15.5:0/865566459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5819f870 con 0x7f6b58106a80 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.558+0000 7f6b55ffb640 1 -- 172.21.15.5:0/865566459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b500116d0 con 0x7f6b58106a80 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.558+0000 7f6b5ef69640 1 -- 172.21.15.5:0/865566459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b581a7150 con 0x7f6b58106a80 2024-09-19T06:01:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.559+0000 7f6b5ef69640 1 -- 172.21.15.5:0/865566459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b58100ee0 con 0x7f6b58106a80 2024-09-19T06:01:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.560+0000 7f6b55ffb640 1 -- 172.21.15.5:0/865566459 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f6b5000cb20 con 0x7f6b58106a80 2024-09-19T06:01:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.561+0000 7f6b55ffb640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6b480a3240 0x7f6b480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.561+0000 7f6b5cd05640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6b480a3240 0x7f6b480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.561+0000 7f6b55ffb640 1 -- 172.21.15.5:0/865566459 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f6b500cf550 con 0x7f6b58106a80 2024-09-19T06:01:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.562+0000 7f6b5cd05640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6b480a3240 0x7f6b480a5700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c005ec0 tx=0x7f6b4c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.565+0000 7f6b55ffb640 1 -- 172.21.15.5:0/865566459 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6b50016030 con 0x7f6b58106a80 2024-09-19T06:01:09.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.776+0000 7f6b5ef69640 1 -- 172.21.15.5:0/865566459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b5802d7c0 con 0x7f6b58106a80 2024-09-19T06:01:09.778 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:09 smithi005 ceph-mon[30955]: Deploying daemon mds.i on smithi157 2024-09-19T06:01:09.778 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:09 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] up:boot 2024-09-19T06:01:09.778 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:09 smithi005 ceph-mon[30955]: fsmap 8 up:standby 2024-09-19T06:01:09.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:09 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:01:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.776+0000 7f6b55ffb640 1 -- 172.21.15.5:0/865566459 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7210 (secure 0 0 0) 0x7f6b5009c020 con 0x7f6b58106a80 2024-09-19T06:01:09.779 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:09.779 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"btime":"2024-09-19T06:01:08:545753+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24437,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T06:01:09.780 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-09-19T06:01:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.779+0000 7f6b5ef69640 1 -- 172.21.15.5:0/865566459 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6b480a3240 msgr2=0x7f6b480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.779+0000 7f6b5ef69640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6b480a3240 0x7f6b480a5700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c005ec0 tx=0x7f6b4c005a20 comp rx=0 tx=0).stop 2024-09-19T06:01:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.779+0000 7f6b5ef69640 1 -- 172.21.15.5:0/865566459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b58106a80 msgr2=0x7f6b5819eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.779+0000 7f6b5ef69640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b58106a80 0x7f6b5819eea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000e940 tx=0x7f6b5000ee00 comp rx=0 tx=0).stop 2024-09-19T06:01:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.779+0000 7f6b5ef69640 1 -- 172.21.15.5:0/865566459 shutdown_connections 2024-09-19T06:01:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.779+0000 7f6b5ef69640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6b480a3240 0x7f6b480a5700 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.779+0000 7f6b5ef69640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b58106a80 0x7f6b5819eea0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.779+0000 7f6b5ef69640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101420 0x7f6b5819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.779+0000 7f6b5ef69640 1 --2- 172.21.15.5:0/865566459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b58100a80 0x7f6b5819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.779+0000 7f6b5ef69640 1 -- 172.21.15.5:0/865566459 >> 172.21.15.5:0/865566459 conn(0x7f6b580fc7a0 msgr2=0x7f6b580fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:09.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.779+0000 7f6b5ef69640 1 -- 172.21.15.5:0/865566459 shutdown_connections 2024-09-19T06:01:09.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:09.779+0000 7f6b5ef69640 1 -- 172.21.15.5:0/865566459 wait complete. 2024-09-19T06:01:09.842 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:09 smithi157 ceph-mon[35346]: Deploying daemon mds.i on smithi157 2024-09-19T06:01:09.842 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:09 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] up:boot 2024-09-19T06:01:09.842 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:09 smithi157 ceph-mon[35346]: fsmap 8 up:standby 2024-09-19T06:01:09.842 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:09 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:01:09.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:09 smithi049 ceph-mon[34640]: Deploying daemon mds.i on smithi157 2024-09-19T06:01:09.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:09 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] up:boot 2024-09-19T06:01:09.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:09 smithi049 ceph-mon[34640]: fsmap 8 up:standby 2024-09-19T06:01:09.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:09 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:01:10.237 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:01:10 smithi157 systemd[1]: Starting Ceph mds.i for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:01:10.506 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:01:10 smithi157 podman[65859]: 2024-09-19 06:01:10.138300882 +0000 UTC m=+0.015804113 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:01:10.507 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:01:10 smithi157 podman[65859]: 2024-09-19 06:01:10.244801821 +0000 UTC m=+0.122305032 container create 052a38933963a477f2c2097209bd64b0ab27158bade7c6aa3184a4760269b2b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:01:10.507 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:01:10 smithi157 podman[65859]: 2024-09-19 06:01:10.505644493 +0000 UTC m=+0.383147718 container init 052a38933963a477f2c2097209bd64b0ab27158bade7c6aa3184a4760269b2b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-09-19T06:01:10.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:10 smithi005 ceph-mon[30955]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:10.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:10 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/865566459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:10 smithi157 ceph-mon[35346]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:10.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:10 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/865566459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:10.802 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:01:10 smithi157 podman[65859]: 2024-09-19 06:01:10.508744545 +0000 UTC m=+0.386247763 container start 052a38933963a477f2c2097209bd64b0ab27158bade7c6aa3184a4760269b2b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-i, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T06:01:10.802 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:01:10 smithi157 ceph-mds[65878]: -- 172.21.15.157:0/2870158300 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cbcf735860 con 0x55cbcf713000 2024-09-19T06:01:10.803 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:01:10 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-i[65874]: starting mds.i at 2024-09-19T06:01:10.803 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:01:10 smithi157 bash[65859]: 052a38933963a477f2c2097209bd64b0ab27158bade7c6aa3184a4760269b2b1 2024-09-19T06:01:10.803 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:01:10 smithi157 systemd[1]: Started Ceph mds.i for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:01:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:10 smithi049 ceph-mon[34640]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:10.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:10 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/865566459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:11.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:11 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] up:boot 2024-09-19T06:01:11.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:11 smithi005 ceph-mon[30955]: fsmap 9 up:standby 2024-09-19T06:01:11.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:11 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:01:11.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:11 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:11.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:11 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:11.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:11 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:11.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:11 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:11.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:11 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:11.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:11 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:11.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:11 smithi005 ceph-mon[30955]: Deploying daemon mds.g on smithi005 2024-09-19T06:01:11.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:11 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] up:boot 2024-09-19T06:01:11.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:11 smithi049 ceph-mon[34640]: fsmap 9 up:standby 2024-09-19T06:01:11.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:11 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:01:11.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:11 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:11.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:11 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:11.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:11 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:11.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:11 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:11.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:11 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:11.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:11 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:11.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:11 smithi049 ceph-mon[34640]: Deploying daemon mds.g on smithi005 2024-09-19T06:01:12.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:11 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] up:boot 2024-09-19T06:01:12.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:11 smithi157 ceph-mon[35346]: fsmap 9 up:standby 2024-09-19T06:01:12.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:11 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:01:12.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:11 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:12.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:11 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:12.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:11 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:12.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:11 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:12.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:11 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:12.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:11 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:12.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:11 smithi157 ceph-mon[35346]: Deploying daemon mds.g on smithi005 2024-09-19T06:01:12.390 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:01:12.629 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:01:12 smithi005 systemd[1]: Starting Ceph mds.g for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:01:12.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:12 smithi005 ceph-mon[30955]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:12.938 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:01:12 smithi005 podman[69314]: 2024-09-19 06:01:12.683058367 +0000 UTC m=+0.017347943 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:01:12.938 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:01:12 smithi005 podman[69314]: 2024-09-19 06:01:12.817341653 +0000 UTC m=+0.151631258 container create 270ffe4313e0dba4d6c9ef0b293794784beccf47e2c8ee491edc493a4c542111 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T06:01:13.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:12 smithi157 ceph-mon[35346]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:13.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:12 smithi049 ceph-mon[34640]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:13.328 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:01:13 smithi005 podman[69314]: 2024-09-19 06:01:13.061795455 +0000 UTC m=+0.396085026 container init 270ffe4313e0dba4d6c9ef0b293794784beccf47e2c8ee491edc493a4c542111 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T06:01:13.328 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:01:13 smithi005 podman[69314]: 2024-09-19 06:01:13.06476087 +0000 UTC m=+0.399050442 container start 270ffe4313e0dba4d6c9ef0b293794784beccf47e2c8ee491edc493a4c542111 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T06:01:13.328 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:01:13 smithi005 ceph-mds[69356]: -- 172.21.15.5:0/3169494241 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ba9b6a7860 con 0x55ba9b684800 2024-09-19T06:01:13.328 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:01:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-g[69340]: starting mds.g at 2024-09-19T06:01:13.328 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:01:13 smithi005 bash[69314]: 270ffe4313e0dba4d6c9ef0b293794784beccf47e2c8ee491edc493a4c542111 2024-09-19T06:01:13.328 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:01:13 smithi005 systemd[1]: Started Ceph mds.g for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:01:13.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.714+0000 7fa5f43fa640 1 -- 172.21.15.5:0/1327886181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec101de0 msgr2=0x7fa5ec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.714+0000 7fa5f43fa640 1 --2- 172.21.15.5:0/1327886181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec101de0 0x7fa5ec1021c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009900 tx=0x7fa5e802a410 comp rx=0 tx=0).stop 2024-09-19T06:01:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.714+0000 7fa5f43fa640 1 -- 172.21.15.5:0/1327886181 shutdown_connections 2024-09-19T06:01:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.714+0000 7fa5f43fa640 1 --2- 172.21.15.5:0/1327886181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5ec106f10 0x7fa5ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.714+0000 7fa5f43fa640 1 --2- 172.21.15.5:0/1327886181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ec102790 0x7fa5ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.714+0000 7fa5f43fa640 1 --2- 172.21.15.5:0/1327886181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec101de0 0x7fa5ec1021c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.714+0000 7fa5f43fa640 1 -- 172.21.15.5:0/1327886181 >> 172.21.15.5:0/1327886181 conn(0x7fa5ec0fd8c0 msgr2=0x7fa5ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.714+0000 7fa5f43fa640 1 -- 172.21.15.5:0/1327886181 shutdown_connections 2024-09-19T06:01:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.714+0000 7fa5f43fa640 1 -- 172.21.15.5:0/1327886181 wait complete. 2024-09-19T06:01:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.715+0000 7fa5f43fa640 1 Processor -- start 2024-09-19T06:01:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.715+0000 7fa5f43fa640 1 -- start start 2024-09-19T06:01:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.715+0000 7fa5f43fa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ec101de0 0x7fa5ec198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.715+0000 7fa5f43fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5ec102790 0x7fa5ec1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.715+0000 7fa5f43fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec106f10 0x7fa5ec19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.715+0000 7fa5f43fa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec111e70 con 0x7fa5ec106f10 2024-09-19T06:01:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.715+0000 7fa5f43fa640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec111cf0 con 0x7fa5ec101de0 2024-09-19T06:01:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.715+0000 7fa5f43fa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec111ff0 con 0x7fa5ec102790 2024-09-19T06:01:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.715+0000 7fa5f2997640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec106f10 0x7fa5ec19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.715+0000 7fa5f2997640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec106f10 0x7fa5ec19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35628/0 (socket says 172.21.15.5:35628) 2024-09-19T06:01:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.715+0000 7fa5f2997640 1 -- 172.21.15.5:0/2837209584 learned_addr learned my addr 172.21.15.5:0/2837209584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5f2196640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ec101de0 0x7fa5ec198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5f1995640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5ec102790 0x7fa5ec1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5f2997640 1 -- 172.21.15.5:0/2837209584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5ec102790 msgr2=0x7fa5ec1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5f2997640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5ec102790 0x7fa5ec1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5f2997640 1 -- 172.21.15.5:0/2837209584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ec101de0 msgr2=0x7fa5ec198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5f2997640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ec101de0 0x7fa5ec198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5f2997640 1 -- 172.21.15.5:0/2837209584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec19d310 con 0x7fa5ec106f10 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5f2196640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ec101de0 0x7fa5ec198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5f2997640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec106f10 0x7fa5ec19cc30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400eaa0 tx=0x7fa5e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5e37fe640 1 -- 172.21.15.5:0/2837209584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400cdb0 con 0x7fa5ec106f10 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5f43fa640 1 -- 172.21.15.5:0/2837209584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ec19d600 con 0x7fa5ec106f10 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5e37fe640 1 -- 172.21.15.5:0/2837209584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5e4005020 con 0x7fa5ec106f10 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5e37fe640 1 -- 172.21.15.5:0/2837209584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40116d0 con 0x7fa5ec106f10 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.716+0000 7fa5f43fa640 1 -- 172.21.15.5:0/2837209584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ec1a4ee0 con 0x7fa5ec106f10 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.717+0000 7fa5f43fa640 1 -- 172.21.15.5:0/2837209584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ec1021c0 con 0x7fa5ec106f10 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.719+0000 7fa5e37fe640 1 -- 172.21.15.5:0/2837209584 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fa5e40051d0 con 0x7fa5ec106f10 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.719+0000 7fa5e37fe640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa5d80a3240 0x7fa5d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.719+0000 7fa5f2196640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa5d80a3240 0x7fa5d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:13.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.719+0000 7fa5e37fe640 1 -- 172.21.15.5:0/2837209584 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fa5e40cfaa0 con 0x7fa5ec106f10 2024-09-19T06:01:13.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.720+0000 7fa5f2196640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa5d80a3240 0x7fa5d80a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8000c00 tx=0x7fa5e8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:13.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.723+0000 7fa5e37fe640 1 -- 172.21.15.5:0/2837209584 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa5e4016030 con 0x7fa5ec106f10 2024-09-19T06:01:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.951+0000 7fa5f43fa640 1 -- 172.21.15.5:0/2837209584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5ec02d7c0 con 0x7fa5ec106f10 2024-09-19T06:01:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.952+0000 7fa5e37fe640 1 -- 172.21.15.5:0/2837209584 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8863 (secure 0 0 0) 0x7fa5e409cbc0 con 0x7fa5ec106f10 2024-09-19T06:01:13.954 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:13.954 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":11,"btime":"2024-09-19T06:01:13:667657+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24437,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T06:01:13.954 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 11 2024-09-19T06:01:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.954+0000 7fa5f43fa640 1 -- 172.21.15.5:0/2837209584 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa5d80a3240 msgr2=0x7fa5d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.954+0000 7fa5f43fa640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa5d80a3240 0x7fa5d80a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8000c00 tx=0x7fa5e8005a20 comp rx=0 tx=0).stop 2024-09-19T06:01:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.954+0000 7fa5f43fa640 1 -- 172.21.15.5:0/2837209584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec106f10 msgr2=0x7fa5ec19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.954+0000 7fa5f43fa640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec106f10 0x7fa5ec19cc30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400eaa0 tx=0x7fa5e400ef60 comp rx=0 tx=0).stop 2024-09-19T06:01:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.954+0000 7fa5f43fa640 1 -- 172.21.15.5:0/2837209584 shutdown_connections 2024-09-19T06:01:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.954+0000 7fa5f43fa640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa5d80a3240 0x7fa5d80a5700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.954+0000 7fa5f43fa640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec106f10 0x7fa5ec19cc30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.954+0000 7fa5f43fa640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5ec102790 0x7fa5ec1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.954+0000 7fa5f43fa640 1 --2- 172.21.15.5:0/2837209584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ec101de0 0x7fa5ec198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.954+0000 7fa5f43fa640 1 -- 172.21.15.5:0/2837209584 >> 172.21.15.5:0/2837209584 conn(0x7fa5ec0fd8c0 msgr2=0x7fa5ec0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.954+0000 7fa5f43fa640 1 -- 172.21.15.5:0/2837209584 shutdown_connections 2024-09-19T06:01:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:13.954+0000 7fa5f43fa640 1 -- 172.21.15.5:0/2837209584 wait complete. 2024-09-19T06:01:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:14 smithi157 ceph-mon[35346]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:14 smithi157 ceph-mon[35346]: Deploying daemon mds.a on smithi005 2024-09-19T06:01:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:14 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.5:6838/1310081153,v1:172.21.15.5:6839/1310081153] up:boot 2024-09-19T06:01:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:14 smithi157 ceph-mon[35346]: fsmap 10 up:standby 2024-09-19T06:01:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:14 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:01:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:14 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2837209584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:14.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:14 smithi005 ceph-mon[30955]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:14 smithi005 ceph-mon[30955]: Deploying daemon mds.a on smithi005 2024-09-19T06:01:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:14 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.5:6838/1310081153,v1:172.21.15.5:6839/1310081153] up:boot 2024-09-19T06:01:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:14 smithi005 ceph-mon[30955]: fsmap 10 up:standby 2024-09-19T06:01:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:14 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:01:14.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:14 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2837209584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:14 smithi049 ceph-mon[34640]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:14.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:14.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:14.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:14.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:14.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:14.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:14.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:14 smithi049 ceph-mon[34640]: Deploying daemon mds.a on smithi005 2024-09-19T06:01:14.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:14 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.5:6838/1310081153,v1:172.21.15.5:6839/1310081153] up:boot 2024-09-19T06:01:14.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:14 smithi049 ceph-mon[34640]: fsmap 10 up:standby 2024-09-19T06:01:14.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:14 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:01:14.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:14 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2837209584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:16.129 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:01:16 smithi005 systemd[1]: Starting Ceph mds.a for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:01:16.467 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:16 smithi005 ceph-mon[30955]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:16.468 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:01:16 smithi005 podman[69706]: 2024-09-19 06:01:16.148380798 +0000 UTC m=+0.016514079 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:01:16.468 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:01:16 smithi005 podman[69706]: 2024-09-19 06:01:16.255983844 +0000 UTC m=+0.124117152 container create d95c9f55a0daee6fa6a485f162eb4ed33e88b44de7898d05899d44830e0f30b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T06:01:16.468 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:01:16 smithi005 podman[69706]: 2024-09-19 06:01:16.466778933 +0000 UTC m=+0.334912216 container init d95c9f55a0daee6fa6a485f162eb4ed33e88b44de7898d05899d44830e0f30b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T06:01:16.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:16 smithi157 ceph-mon[35346]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:16.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:16 smithi049 ceph-mon[34640]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:16.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:01:16.727 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:01:16 smithi005 podman[69706]: 2024-09-19 06:01:16.469921651 +0000 UTC m=+0.338054919 container start d95c9f55a0daee6fa6a485f162eb4ed33e88b44de7898d05899d44830e0f30b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-a, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T06:01:16.727 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:01:16 smithi005 ceph-mds[69725]: -- 172.21.15.5:0/2838889228 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aee66d7860 con 0x55aee66b5000 2024-09-19T06:01:16.727 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:01:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-a[69721]: starting mds.a at 2024-09-19T06:01:16.727 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:01:16 smithi005 bash[69706]: d95c9f55a0daee6fa6a485f162eb4ed33e88b44de7898d05899d44830e0f30b7 2024-09-19T06:01:16.727 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:01:16 smithi005 systemd[1]: Started Ceph mds.a for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:01:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.462+0000 7f885ae6d640 1 -- 172.21.15.5:0/1901925724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8854106a80 msgr2=0x7f8854106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.462+0000 7f885ae6d640 1 --2- 172.21.15.5:0/1901925724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8854106a80 0x7f8854106e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f883c009930 tx=0x7f883c02a420 comp rx=0 tx=0).stop 2024-09-19T06:01:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.462+0000 7f885ae6d640 1 -- 172.21.15.5:0/1901925724 shutdown_connections 2024-09-19T06:01:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.462+0000 7f885ae6d640 1 --2- 172.21.15.5:0/1901925724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8854101420 0x7f885410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.462+0000 7f885ae6d640 1 --2- 172.21.15.5:0/1901925724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8854100a80 0x7f8854100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.462+0000 7f885ae6d640 1 --2- 172.21.15.5:0/1901925724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8854106a80 0x7f8854106e60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.462+0000 7f885ae6d640 1 -- 172.21.15.5:0/1901925724 >> 172.21.15.5:0/1901925724 conn(0x7f88540fc7a0 msgr2=0x7f88540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.462+0000 7f885ae6d640 1 -- 172.21.15.5:0/1901925724 shutdown_connections 2024-09-19T06:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.462+0000 7f885ae6d640 1 -- 172.21.15.5:0/1901925724 wait complete. 2024-09-19T06:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f885ae6d640 1 Processor -- start 2024-09-19T06:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f885ae6d640 1 -- start start 2024-09-19T06:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f885ae6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8854100a80 0x7f88541983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f885ae6d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8854101420 0x7f88541988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f885ae6d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8854106a80 0x7f885419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f885ae6d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8854110d40 con 0x7f8854100a80 2024-09-19T06:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f885ae6d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8854110bc0 con 0x7f8854101420 2024-09-19T06:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f885ae6d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8854110ec0 con 0x7f8854106a80 2024-09-19T06:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f8858c09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8854100a80 0x7f88541983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f8858c09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8854100a80 0x7f88541983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48028/0 (socket says 172.21.15.5:48028) 2024-09-19T06:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f8858c09640 1 -- 172.21.15.5:0/2019514911 learned_addr learned my addr 172.21.15.5:0/2019514911 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f8853fff640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8854101420 0x7f88541988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.463+0000 7f885940a640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8854106a80 0x7f885419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.464+0000 7f8858c09640 1 -- 172.21.15.5:0/2019514911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8854106a80 msgr2=0x7f885419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.464+0000 7f8858c09640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8854106a80 0x7f885419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.464+0000 7f8858c09640 1 -- 172.21.15.5:0/2019514911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8854101420 msgr2=0x7f88541988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.464+0000 7f8858c09640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8854101420 0x7f88541988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.464+0000 7f8858c09640 1 -- 172.21.15.5:0/2019514911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885419d360 con 0x7f8854100a80 2024-09-19T06:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.464+0000 7f8858c09640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8854100a80 0x7f88541983b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f883c005ab0 tx=0x7f883c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.464+0000 7f8851ffb640 1 -- 172.21.15.5:0/2019514911 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c03f070 con 0x7f8854100a80 2024-09-19T06:01:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.464+0000 7f8851ffb640 1 -- 172.21.15.5:0/2019514911 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f883c031b50 con 0x7f8854100a80 2024-09-19T06:01:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.464+0000 7f885ae6d640 1 -- 172.21.15.5:0/2019514911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885419d5f0 con 0x7f8854100a80 2024-09-19T06:01:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.464+0000 7f8851ffb640 1 -- 172.21.15.5:0/2019514911 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c035030 con 0x7f8854100a80 2024-09-19T06:01:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.464+0000 7f885ae6d640 1 -- 172.21.15.5:0/2019514911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88541a4f30 con 0x7f8854100a80 2024-09-19T06:01:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.465+0000 7f885ae6d640 1 -- 172.21.15.5:0/2019514911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88541021c0 con 0x7f8854100a80 2024-09-19T06:01:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.466+0000 7f8851ffb640 1 -- 172.21.15.5:0/2019514911 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f883c042070 con 0x7f8854100a80 2024-09-19T06:01:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.467+0000 7f8851ffb640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f88400a32c0 0x7f88400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.468+0000 7f8853fff640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f88400a32c0 0x7f88400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.468+0000 7f8851ffb640 1 -- 172.21.15.5:0/2019514911 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f883c0efb90 con 0x7f8854100a80 2024-09-19T06:01:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.468+0000 7f8853fff640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f88400a32c0 0x7f88400a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f88540feda0 tx=0x7f8844009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.472+0000 7f8851ffb640 1 -- 172.21.15.5:0/2019514911 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f883c0bccb0 con 0x7f8854100a80 2024-09-19T06:01:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.683+0000 7f885ae6d640 1 -- 172.21.15.5:0/2019514911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88541a5100 con 0x7f8854100a80 2024-09-19T06:01:17.685 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:17.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:17.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:17.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:17.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:17.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:17.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:17 smithi005 ceph-mon[30955]: Deploying daemon mds.b on smithi049 2024-09-19T06:01:17.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:17 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.5:6840/569764382,v1:172.21.15.5:6841/569764382] up:boot 2024-09-19T06:01:17.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:17 smithi005 ceph-mon[30955]: fsmap 11 up:standby 2024-09-19T06:01:17.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:17 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:01:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.684+0000 7f8851ffb640 1 -- 172.21.15.5:0/2019514911 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9683 (secure 0 0 0) 0x7f883c0bc5b0 con 0x7f8854100a80 2024-09-19T06:01:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:17.687 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":12,"btime":"2024-09-19T06:01:17:206994+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24437,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T06:01:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 12 2024-09-19T06:01:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.686+0000 7f885ae6d640 1 -- 172.21.15.5:0/2019514911 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f88400a32c0 msgr2=0x7f88400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.686+0000 7f885ae6d640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f88400a32c0 0x7f88400a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f88540feda0 tx=0x7f8844009760 comp rx=0 tx=0).stop 2024-09-19T06:01:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.686+0000 7f885ae6d640 1 -- 172.21.15.5:0/2019514911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8854100a80 msgr2=0x7f88541983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.686+0000 7f885ae6d640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8854100a80 0x7f88541983b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f883c005ab0 tx=0x7f883c003b90 comp rx=0 tx=0).stop 2024-09-19T06:01:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.686+0000 7f885ae6d640 1 -- 172.21.15.5:0/2019514911 shutdown_connections 2024-09-19T06:01:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.686+0000 7f885ae6d640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f88400a32c0 0x7f88400a5780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.686+0000 7f885ae6d640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8854106a80 0x7f885419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.686+0000 7f885ae6d640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8854101420 0x7f88541988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.686+0000 7f885ae6d640 1 --2- 172.21.15.5:0/2019514911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8854100a80 0x7f88541983b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.686+0000 7f885ae6d640 1 -- 172.21.15.5:0/2019514911 >> 172.21.15.5:0/2019514911 conn(0x7f88540fc7a0 msgr2=0x7f88541048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.686+0000 7f885ae6d640 1 -- 172.21.15.5:0/2019514911 shutdown_connections 2024-09-19T06:01:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:17.686+0000 7f885ae6d640 1 -- 172.21.15.5:0/2019514911 wait complete. 2024-09-19T06:01:17.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:17.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:17.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:17.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:17.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:17.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:17.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:17 smithi049 ceph-mon[34640]: Deploying daemon mds.b on smithi049 2024-09-19T06:01:17.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:17 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.5:6840/569764382,v1:172.21.15.5:6841/569764382] up:boot 2024-09-19T06:01:17.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:17 smithi049 ceph-mon[34640]: fsmap 11 up:standby 2024-09-19T06:01:17.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:17 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:01:18.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:18.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:18.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:18.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:01:18.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:01:18.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:18.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:17 smithi157 ceph-mon[35346]: Deploying daemon mds.b on smithi049 2024-09-19T06:01:18.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:17 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.5:6840/569764382,v1:172.21.15.5:6841/569764382] up:boot 2024-09-19T06:01:18.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:17 smithi157 ceph-mon[35346]: fsmap 11 up:standby 2024-09-19T06:01:18.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:17 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:01:18.631 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:18 smithi049 systemd[1]: Starting Ceph mds.b for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:01:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:18 smithi049 ceph-mon[34640]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:18.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:18 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2019514911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:18.897 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:18 smithi049 podman[65076]: 2024-09-19 06:01:18.642701229 +0000 UTC m=+0.016501612 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:01:18.897 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:18 smithi049 podman[65076]: 2024-09-19 06:01:18.748674912 +0000 UTC m=+0.122475277 container create 855f6af864068389d939cc28e97ae6e139b7d69c18c9a6bf1cf3472977bc9ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T06:01:19.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:18 smithi157 ceph-mon[35346]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:19.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:18 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2019514911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:19.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:18 smithi005 ceph-mon[30955]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:19.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:18 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2019514911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:19.396 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:18 smithi049 podman[65076]: 2024-09-19 06:01:18.976679819 +0000 UTC m=+0.350480194 container init 855f6af864068389d939cc28e97ae6e139b7d69c18c9a6bf1cf3472977bc9ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T06:01:19.397 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:18 smithi049 podman[65076]: 2024-09-19 06:01:18.979665654 +0000 UTC m=+0.353466023 container start 855f6af864068389d939cc28e97ae6e139b7d69c18c9a6bf1cf3472977bc9ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T06:01:19.397 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:19 smithi049 ceph-mds[65095]: -- 172.21.15.49:0/1031331421 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559db2089860 con 0x559db2066800 2024-09-19T06:01:19.397 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: starting mds.b at 2024-09-19T06:01:19.397 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:19 smithi049 bash[65076]: 855f6af864068389d939cc28e97ae6e139b7d69c18c9a6bf1cf3472977bc9ba9 2024-09-19T06:01:19.397 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:19 smithi049 systemd[1]: Started Ceph mds.b for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:01:20.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:20 smithi049 ceph-mon[34640]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:20.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:20 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:20 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:20 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:20 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:20 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:20 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:01:20.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:20 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] up:boot 2024-09-19T06:01:20.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:20 smithi049 ceph-mon[34640]: fsmap 12 up:standby 2024-09-19T06:01:20.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:20 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:01:20.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:20 smithi157 ceph-mon[35346]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:20.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:20 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:20 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:20 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:20 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:20 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:20 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:01:20.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:20 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] up:boot 2024-09-19T06:01:20.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:20 smithi157 ceph-mon[35346]: fsmap 12 up:standby 2024-09-19T06:01:20.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:20 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:01:20.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:01:20.332 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:20 smithi005 ceph-mon[30955]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:20.332 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:20 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.332 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:20 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.333 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:20 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.333 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:20 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.333 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:20 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:20.333 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:20 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:01:20.333 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:20 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] up:boot 2024-09-19T06:01:20.333 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:20 smithi005 ceph-mon[30955]: fsmap 12 up:standby 2024-09-19T06:01:20.333 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:20 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:01:20.601 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:21.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.097+0000 7fd16eba6640 1 -- 172.21.15.5:0/2822716907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd168106f10 msgr2=0x7fd16810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:21.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.097+0000 7fd16eba6640 1 --2- 172.21.15.5:0/2822716907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd168106f10 0x7fd16810ea90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd160009ca0 tx=0x7fd16002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:01:21.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.097+0000 7fd16eba6640 1 -- 172.21.15.5:0/2822716907 shutdown_connections 2024-09-19T06:01:21.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.097+0000 7fd16eba6640 1 --2- 172.21.15.5:0/2822716907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd168106f10 0x7fd16810ea90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:21.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.097+0000 7fd16eba6640 1 --2- 172.21.15.5:0/2822716907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd168102790 0x7fd1681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.097+0000 7fd16eba6640 1 --2- 172.21.15.5:0/2822716907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168101de0 0x7fd1681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.097+0000 7fd16eba6640 1 -- 172.21.15.5:0/2822716907 >> 172.21.15.5:0/2822716907 conn(0x7fd1680fd8c0 msgr2=0x7fd1680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.097+0000 7fd16eba6640 1 -- 172.21.15.5:0/2822716907 shutdown_connections 2024-09-19T06:01:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.097+0000 7fd16eba6640 1 -- 172.21.15.5:0/2822716907 wait complete. 2024-09-19T06:01:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd16eba6640 1 Processor -- start 2024-09-19T06:01:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd16eba6640 1 -- start start 2024-09-19T06:01:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd16eba6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd168101de0 0x7fd1681982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd16eba6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168102790 0x7fd168198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd16eba6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd168106f10 0x7fd16819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd16eba6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd168111e20 con 0x7fd168101de0 2024-09-19T06:01:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd16eba6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd168111ca0 con 0x7fd168102790 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd16eba6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd168111fa0 con 0x7fd168106f10 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd16c942640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd168101de0 0x7fd1681982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd16c942640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd168101de0 0x7fd1681982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48052/0 (socket says 172.21.15.5:48052) 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd16c942640 1 -- 172.21.15.5:0/2460599958 learned_addr learned my addr 172.21.15.5:0/2460599958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd15ffff640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168102790 0x7fd168198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.098+0000 7fd16d143640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd168106f10 0x7fd16819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.099+0000 7fd16c942640 1 -- 172.21.15.5:0/2460599958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd168106f10 msgr2=0x7fd16819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.099+0000 7fd16c942640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd168106f10 0x7fd16819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.099+0000 7fd16c942640 1 -- 172.21.15.5:0/2460599958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168102790 msgr2=0x7fd168198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.099+0000 7fd16c942640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168102790 0x7fd168198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.099+0000 7fd16c942640 1 -- 172.21.15.5:0/2460599958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16819d2a0 con 0x7fd168101de0 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.099+0000 7fd16d143640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd168106f10 0x7fd16819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:01:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.099+0000 7fd16c942640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd168101de0 0x7fd1681982f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd16400e9f0 tx=0x7fd16400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:21.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.099+0000 7fd15dffb640 1 -- 172.21.15.5:0/2460599958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16400cd80 con 0x7fd168101de0 2024-09-19T06:01:21.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.099+0000 7fd15dffb640 1 -- 172.21.15.5:0/2460599958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1640108e0 con 0x7fd168101de0 2024-09-19T06:01:21.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.099+0000 7fd16eba6640 1 -- 172.21.15.5:0/2460599958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16819d590 con 0x7fd168101de0 2024-09-19T06:01:21.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.099+0000 7fd15dffb640 1 -- 172.21.15.5:0/2460599958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164018920 con 0x7fd168101de0 2024-09-19T06:01:21.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.099+0000 7fd16eba6640 1 -- 172.21.15.5:0/2460599958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1681a4e70 con 0x7fd168101de0 2024-09-19T06:01:21.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.100+0000 7fd16eba6640 1 -- 172.21.15.5:0/2460599958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1681021c0 con 0x7fd168101de0 2024-09-19T06:01:21.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.101+0000 7fd15dffb640 1 -- 172.21.15.5:0/2460599958 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fd164010a90 con 0x7fd168101de0 2024-09-19T06:01:21.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.102+0000 7fd15dffb640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd1500a32c0 0x7fd1500a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:21.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.102+0000 7fd15ffff640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd1500a32c0 0x7fd1500a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:21.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.102+0000 7fd15dffb640 1 -- 172.21.15.5:0/2460599958 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fd1640cfc00 con 0x7fd168101de0 2024-09-19T06:01:21.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.103+0000 7fd15ffff640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd1500a32c0 0x7fd1500a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1680fdce0 tx=0x7fd15800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.105+0000 7fd15dffb640 1 -- 172.21.15.5:0/2460599958 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd16409cda0 con 0x7fd168101de0 2024-09-19T06:01:21.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.324+0000 7fd16eba6640 1 -- 172.21.15.5:0/2460599958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd168199650 con 0x7fd168101de0 2024-09-19T06:01:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.325+0000 7fd15dffb640 1 -- 172.21.15.5:0/2460599958 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10509 (secure 0 0 0) 0x7fd16409c6a0 con 0x7fd168101de0 2024-09-19T06:01:21.327 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:21.327 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"btime":"2024-09-19T06:01:19:629487+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24437,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T06:01:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 13 2024-09-19T06:01:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.328+0000 7fd16eba6640 1 -- 172.21.15.5:0/2460599958 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd1500a32c0 msgr2=0x7fd1500a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.328+0000 7fd16eba6640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd1500a32c0 0x7fd1500a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1680fdce0 tx=0x7fd15800b410 comp rx=0 tx=0).stop 2024-09-19T06:01:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.328+0000 7fd16eba6640 1 -- 172.21.15.5:0/2460599958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd168101de0 msgr2=0x7fd1681982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.328+0000 7fd16eba6640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd168101de0 0x7fd1681982f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd16400e9f0 tx=0x7fd16400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:01:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.328+0000 7fd16eba6640 1 -- 172.21.15.5:0/2460599958 shutdown_connections 2024-09-19T06:01:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.328+0000 7fd16eba6640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd1500a32c0 0x7fd1500a5780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.328+0000 7fd16eba6640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd168106f10 0x7fd16819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.328+0000 7fd16eba6640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168102790 0x7fd168198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.328+0000 7fd16eba6640 1 --2- 172.21.15.5:0/2460599958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd168101de0 0x7fd1681982f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.328+0000 7fd16eba6640 1 -- 172.21.15.5:0/2460599958 >> 172.21.15.5:0/2460599958 conn(0x7fd1680fd8c0 msgr2=0x7fd1680fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.328+0000 7fd16eba6640 1 -- 172.21.15.5:0/2460599958 shutdown_connections 2024-09-19T06:01:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:21.328+0000 7fd16eba6640 1 -- 172.21.15.5:0/2460599958 wait complete. 2024-09-19T06:01:21.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:21.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:01:21.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:21 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:01:21.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:21 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2460599958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:21.973 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.973 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.973 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.973 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.973 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.973 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.973 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:21.973 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:01:21.973 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:21.973 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:21 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:01:21.973 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:21 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2460599958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:22.001 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T06:01:22.003 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:22.003 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:22.003 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:22.003 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:22.003 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:22.003 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:22.003 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:22.003 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:01:22.003 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:22.003 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:21 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:01:22.003 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:21 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2460599958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:22 smithi157 ceph-mon[35346]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:22 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:22 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:22 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:23.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:22 smithi005 ceph-mon[30955]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:23.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:22 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:23.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:22 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:23.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:22 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:23.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:22 smithi049 ceph-mon[34640]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:23.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:23.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:23.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:22 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.324+0000 7f37242d7640 1 -- 172.21.15.5:0/4099990564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f371c100a80 msgr2=0x7f371c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.324+0000 7f37242d7640 1 --2- 172.21.15.5:0/4099990564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f371c100a80 0x7f371c100ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f370c009930 tx=0x7f370c02a420 comp rx=0 tx=0).stop 2024-09-19T06:01:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.324+0000 7f37242d7640 1 -- 172.21.15.5:0/4099990564 shutdown_connections 2024-09-19T06:01:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.324+0000 7f37242d7640 1 --2- 172.21.15.5:0/4099990564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c101420 0x7f371c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.324+0000 7f37242d7640 1 --2- 172.21.15.5:0/4099990564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f371c100a80 0x7f371c100ee0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.324+0000 7f37242d7640 1 --2- 172.21.15.5:0/4099990564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f371c106a80 0x7f371c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.324+0000 7f37242d7640 1 -- 172.21.15.5:0/4099990564 >> 172.21.15.5:0/4099990564 conn(0x7f371c0fc7a0 msgr2=0x7f371c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.324+0000 7f37242d7640 1 -- 172.21.15.5:0/4099990564 shutdown_connections 2024-09-19T06:01:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.324+0000 7f37242d7640 1 -- 172.21.15.5:0/4099990564 wait complete. 2024-09-19T06:01:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f37242d7640 1 Processor -- start 2024-09-19T06:01:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f37242d7640 1 -- start start 2024-09-19T06:01:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f37242d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f371c100a80 0x7f371c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f37242d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c101420 0x7f371c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f37242d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f371c106a80 0x7f371c1a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f37242d7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f371c110cf0 con 0x7f371c100a80 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f37242d7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f371c110b70 con 0x7f371c101420 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f37242d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f371c110e70 con 0x7f371c106a80 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f3722073640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f371c100a80 0x7f371c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f3722073640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f371c100a80 0x7f371c1991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48072/0 (socket says 172.21.15.5:48072) 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f3722073640 1 -- 172.21.15.5:0/1661585778 learned_addr learned my addr 172.21.15.5:0/1661585778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f3721872640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c101420 0x7f371c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f3722874640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f371c106a80 0x7f371c1a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f3722073640 1 -- 172.21.15.5:0/1661585778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f371c106a80 msgr2=0x7f371c1a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f3722073640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f371c106a80 0x7f371c1a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f3722073640 1 -- 172.21.15.5:0/1661585778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c101420 msgr2=0x7f371c1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f3722073640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c101420 0x7f371c1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.325+0000 7f3722073640 1 -- 172.21.15.5:0/1661585778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371c1a4760 con 0x7f371c100a80 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.326+0000 7f3722073640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f371c100a80 0x7f371c1991a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f371800c920 tx=0x7f371800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.326+0000 7f37137fe640 1 -- 172.21.15.5:0/1661585778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718007c70 con 0x7f371c100a80 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.326+0000 7f37137fe640 1 -- 172.21.15.5:0/1661585778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3718007e10 con 0x7f371c100a80 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.326+0000 7f37242d7640 1 -- 172.21.15.5:0/1661585778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371c1a4930 con 0x7f371c100a80 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.326+0000 7f37137fe640 1 -- 172.21.15.5:0/1661585778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718018820 con 0x7f371c100a80 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.326+0000 7f37242d7640 1 -- 172.21.15.5:0/1661585778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371c1a4e40 con 0x7f371c100a80 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.327+0000 7f37242d7640 1 -- 172.21.15.5:0/1661585778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f371c100ee0 con 0x7f371c100a80 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.328+0000 7f37137fe640 1 -- 172.21.15.5:0/1661585778 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f37180189c0 con 0x7f371c100a80 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.328+0000 7f37137fe640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f37080a32c0 0x7f37080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.329+0000 7f3721872640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f37080a32c0 0x7f37080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.329+0000 7f37137fe640 1 -- 172.21.15.5:0/1661585778 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f37180cf440 con 0x7f371c100a80 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.329+0000 7f3721872640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f37080a32c0 0x7f37080a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f370c009900 tx=0x7f370c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.333+0000 7f37137fe640 1 -- 172.21.15.5:0/1661585778 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3718014030 con 0x7f371c100a80 2024-09-19T06:01:23.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.522+0000 7f37242d7640 1 -- 172.21.15.5:0/1661585778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f371c109010 con 0x7f371c100a80 2024-09-19T06:01:23.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.523+0000 7f37137fe640 1 -- 172.21.15.5:0/1661585778 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f371809c600 con 0x7f371c100a80 2024-09-19T06:01:23.525 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:23.525 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-09-19T06:01:23.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:01:23.525 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-09-19T06:01:23.525 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-09-19T06:01:23.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:01:23.525 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-09-19T06:01:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.525+0000 7f37242d7640 1 -- 172.21.15.5:0/1661585778 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f37080a32c0 msgr2=0x7f37080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.525+0000 7f37242d7640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f37080a32c0 0x7f37080a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f370c009900 tx=0x7f370c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:01:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.525+0000 7f37242d7640 1 -- 172.21.15.5:0/1661585778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f371c100a80 msgr2=0x7f371c1991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.525+0000 7f37242d7640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f371c100a80 0x7f371c1991a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f371800c920 tx=0x7f371800cde0 comp rx=0 tx=0).stop 2024-09-19T06:01:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.525+0000 7f37242d7640 1 -- 172.21.15.5:0/1661585778 shutdown_connections 2024-09-19T06:01:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.525+0000 7f37242d7640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f37080a32c0 0x7f37080a5780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.525+0000 7f37242d7640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f371c106a80 0x7f371c1a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.525+0000 7f37242d7640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c101420 0x7f371c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.525+0000 7f37242d7640 1 --2- 172.21.15.5:0/1661585778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f371c100a80 0x7f371c1991a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.525+0000 7f37242d7640 1 -- 172.21.15.5:0/1661585778 >> 172.21.15.5:0/1661585778 conn(0x7f371c0fc7a0 msgr2=0x7f371c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:23.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.525+0000 7f37242d7640 1 -- 172.21.15.5:0/1661585778 shutdown_connections 2024-09-19T06:01:23.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:23.525+0000 7f37242d7640 1 -- 172.21.15.5:0/1661585778 wait complete. 2024-09-19T06:01:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:23 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:23 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1661585778' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T06:01:24.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:23 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:24.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:23 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1661585778' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T06:01:24.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:23 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:24.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:23 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1661585778' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T06:01:24.253 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T06:01:24.253 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T06:01:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:24.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:24 smithi005 ceph-mon[30955]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:24.944 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:24 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:24.944 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:24 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:24.944 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:24 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:24.944 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:24 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:24.944 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:24 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:01:24.944 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:24 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:24 smithi157 ceph-mon[35346]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:24 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:24 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:24 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:24 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:24 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:01:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:24 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:25.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.028+0000 7fdafa0b1640 1 -- 172.21.15.5:0/2037184095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf4103660 msgr2=0x7fdaf410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:25.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.028+0000 7fdafa0b1640 1 --2- 172.21.15.5:0/2037184095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf4103660 0x7fdaf410bb50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009900 tx=0x7fdadc02a480 comp rx=0 tx=0).stop 2024-09-19T06:01:25.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.028+0000 7fdafa0b1640 1 -- 172.21.15.5:0/2037184095 shutdown_connections 2024-09-19T06:01:25.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.028+0000 7fdafa0b1640 1 --2- 172.21.15.5:0/2037184095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf410c170 0x7fdaf410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:25.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.028+0000 7fdafa0b1640 1 --2- 172.21.15.5:0/2037184095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf4103660 0x7fdaf410bb50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:25.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.028+0000 7fdafa0b1640 1 --2- 172.21.15.5:0/2037184095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaf4102d40 0x7fdaf4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:25.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.028+0000 7fdafa0b1640 1 -- 172.21.15.5:0/2037184095 >> 172.21.15.5:0/2037184095 conn(0x7fdaf40fc7a0 msgr2=0x7fdaf40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:25.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.028+0000 7fdafa0b1640 1 -- 172.21.15.5:0/2037184095 shutdown_connections 2024-09-19T06:01:25.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.028+0000 7fdafa0b1640 1 -- 172.21.15.5:0/2037184095 wait complete. 2024-09-19T06:01:25.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.029+0000 7fdafa0b1640 1 Processor -- start 2024-09-19T06:01:25.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.029+0000 7fdafa0b1640 1 -- start start 2024-09-19T06:01:25.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.029+0000 7fdafa0b1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf4102d40 0x7fdaf41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:25.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.029+0000 7fdafa0b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaf4103660 0x7fdaf4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:25.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.029+0000 7fdafa0b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf410c170 0x7fdaf419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:25.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.029+0000 7fdafa0b1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4111ee0 con 0x7fdaf410c170 2024-09-19T06:01:25.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.029+0000 7fdafa0b1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4111d60 con 0x7fdaf4102d40 2024-09-19T06:01:25.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.029+0000 7fdafa0b1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4112060 con 0x7fdaf4103660 2024-09-19T06:01:25.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.029+0000 7fdaf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf410c170 0x7fdaf419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:25.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.029+0000 7fdaf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf410c170 0x7fdaf419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54784/0 (socket says 172.21.15.5:54784) 2024-09-19T06:01:25.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.029+0000 7fdaf3fff640 1 -- 172.21.15.5:0/4068617516 learned_addr learned my addr 172.21.15.5:0/4068617516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:25.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.029+0000 7fdaf37fe640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf4102d40 0x7fdaf41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:25.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdaeaffd640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaf4103660 0x7fdaf4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdaf3fff640 1 -- 172.21.15.5:0/4068617516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaf4103660 msgr2=0x7fdaf4198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdaf3fff640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaf4103660 0x7fdaf4198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdaf3fff640 1 -- 172.21.15.5:0/4068617516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf4102d40 msgr2=0x7fdaf41984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdaf3fff640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf4102d40 0x7fdaf41984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdaf3fff640 1 -- 172.21.15.5:0/4068617516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf419d490 con 0x7fdaf410c170 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdaf37fe640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf4102d40 0x7fdaf41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdaf3fff640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf410c170 0x7fdaf419cdb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00b970 tx=0x7fdaec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdaf17fa640 1 -- 172.21.15.5:0/4068617516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec019070 con 0x7fdaf410c170 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdafa0b1640 1 -- 172.21.15.5:0/4068617516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf419d780 con 0x7fdaf410c170 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdaf17fa640 1 -- 172.21.15.5:0/4068617516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaec014440 con 0x7fdaf410c170 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdaf17fa640 1 -- 172.21.15.5:0/4068617516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0075d0 con 0x7fdaf410c170 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.030+0000 7fdafa0b1640 1 -- 172.21.15.5:0/4068617516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf41a5060 con 0x7fdaf410c170 2024-09-19T06:01:25.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.031+0000 7fdafa0b1640 1 -- 172.21.15.5:0/4068617516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf41031a0 con 0x7fdaf410c170 2024-09-19T06:01:25.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.032+0000 7fdaf17fa640 1 -- 172.21.15.5:0/4068617516 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fdaec01c070 con 0x7fdaf410c170 2024-09-19T06:01:25.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.033+0000 7fdaf17fa640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdae00a3240 0x7fdae00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:25.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.033+0000 7fdaf37fe640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdae00a3240 0x7fdae00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:25.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.033+0000 7fdaf17fa640 1 -- 172.21.15.5:0/4068617516 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fdaec0ceaf0 con 0x7fdaf410c170 2024-09-19T06:01:25.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.034+0000 7fdaf37fe640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdae00a3240 0x7fdae00a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009930 tx=0x7fdae4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:25.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.037+0000 7fdaf17fa640 1 -- 172.21.15.5:0/4068617516 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdaec09bcb0 con 0x7fdaf410c170 2024-09-19T06:01:25.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:24 smithi049 ceph-mon[34640]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:24 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:24 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:24 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:24 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:01:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:24 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:01:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:24 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:01:25.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.216+0000 7fdafa0b1640 1 -- 172.21.15.5:0/4068617516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fdaf4107240 con 0x7fdaf410c170 2024-09-19T06:01:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.767+0000 7fdaf17fa640 1 -- 172.21.15.5:0/4068617516 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7fdaec09b5b0 con 0x7fdaf410c170 2024-09-19T06:01:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-09-19T06:01:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.771+0000 7fdafa0b1640 1 -- 172.21.15.5:0/4068617516 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdae00a3240 msgr2=0x7fdae00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.771+0000 7fdafa0b1640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdae00a3240 0x7fdae00a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009930 tx=0x7fdae4008040 comp rx=0 tx=0).stop 2024-09-19T06:01:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.771+0000 7fdafa0b1640 1 -- 172.21.15.5:0/4068617516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf410c170 msgr2=0x7fdaf419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.771+0000 7fdafa0b1640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf410c170 0x7fdaf419cdb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00b970 tx=0x7fdaec00be30 comp rx=0 tx=0).stop 2024-09-19T06:01:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.771+0000 7fdafa0b1640 1 -- 172.21.15.5:0/4068617516 shutdown_connections 2024-09-19T06:01:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.771+0000 7fdafa0b1640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdae00a3240 0x7fdae00a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.771+0000 7fdafa0b1640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf410c170 0x7fdaf419cdb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.771+0000 7fdafa0b1640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaf4103660 0x7fdaf4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.771+0000 7fdafa0b1640 1 --2- 172.21.15.5:0/4068617516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf4102d40 0x7fdaf41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.771+0000 7fdafa0b1640 1 -- 172.21.15.5:0/4068617516 >> 172.21.15.5:0/4068617516 conn(0x7fdaf40fc7a0 msgr2=0x7fdaf40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.771+0000 7fdafa0b1640 1 -- 172.21.15.5:0/4068617516 shutdown_connections 2024-09-19T06:01:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:25.771+0000 7fdafa0b1640 1 -- 172.21.15.5:0/4068617516 wait complete. 2024-09-19T06:01:26.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:25 smithi157 ceph-mon[35346]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:25 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/4068617516' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T06:01:26.080 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:25 smithi005 ceph-mon[30955]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:26.080 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:25 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/4068617516' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T06:01:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:25 smithi049 ceph-mon[34640]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:25 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/4068617516' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T06:01:26.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T06:01:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:27.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:26 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/4068617516' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T06:01:27.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:26 smithi157 ceph-mon[35346]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T06:01:27.052 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:26 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/4068617516' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T06:01:27.052 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:26 smithi005 ceph-mon[30955]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T06:01:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.140+0000 7f3d8b274640 1 -- 172.21.15.5:0/1664025635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84102790 msgr2=0x7f3d841067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.140+0000 7f3d8b274640 1 --2- 172.21.15.5:0/1664025635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84102790 0x7f3d841067e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009900 tx=0x7f3d7402a410 comp rx=0 tx=0).stop 2024-09-19T06:01:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.140+0000 7f3d8b274640 1 -- 172.21.15.5:0/1664025635 shutdown_connections 2024-09-19T06:01:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.140+0000 7f3d8b274640 1 --2- 172.21.15.5:0/1664025635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d84106f10 0x7f3d8410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.140+0000 7f3d8b274640 1 --2- 172.21.15.5:0/1664025635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84102790 0x7f3d841067e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.140+0000 7f3d8b274640 1 --2- 172.21.15.5:0/1664025635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d84101de0 0x7f3d841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.140+0000 7f3d8b274640 1 -- 172.21.15.5:0/1664025635 >> 172.21.15.5:0/1664025635 conn(0x7f3d840fd8c0 msgr2=0x7f3d840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.140+0000 7f3d8b274640 1 -- 172.21.15.5:0/1664025635 shutdown_connections 2024-09-19T06:01:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.140+0000 7f3d8b274640 1 -- 172.21.15.5:0/1664025635 wait complete. 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d8b274640 1 Processor -- start 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d8b274640 1 -- start start 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d8b274640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84101de0 0x7f3d84198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d8b274640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d84102790 0x7f3d841989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d8b274640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d84106f10 0x7f3d8419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d8b274640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84111e60 con 0x7f3d84101de0 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d8b274640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84111ce0 con 0x7f3d84106f10 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d8b274640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84111fe0 con 0x7f3d84102790 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d89010640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84101de0 0x7f3d84198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d89010640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84101de0 0x7f3d84198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54796/0 (socket says 172.21.15.5:54796) 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d89010640 1 -- 172.21.15.5:0/1322864023 learned_addr learned my addr 172.21.15.5:0/1322864023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d8880f640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d84102790 0x7f3d841989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d89811640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d84106f10 0x7f3d8419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d89010640 1 -- 172.21.15.5:0/1322864023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d84102790 msgr2=0x7f3d841989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d89010640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d84102790 0x7f3d841989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d89010640 1 -- 172.21.15.5:0/1322864023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d84106f10 msgr2=0x7f3d8419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d89010640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d84106f10 0x7f3d8419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d89010640 1 -- 172.21.15.5:0/1322864023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8419d410 con 0x7f3d84101de0 2024-09-19T06:01:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.141+0000 7f3d8880f640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d84102790 0x7f3d841989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:01:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.142+0000 7f3d89010640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84101de0 0x7f3d84198460 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000c970 tx=0x7f3d8000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.142+0000 7f3d7a7fc640 1 -- 172.21.15.5:0/1322864023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80007bb0 con 0x7f3d84101de0 2024-09-19T06:01:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.142+0000 7f3d7a7fc640 1 -- 172.21.15.5:0/1322864023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d80007d50 con 0x7f3d84101de0 2024-09-19T06:01:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.142+0000 7f3d8b274640 1 -- 172.21.15.5:0/1322864023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8419d700 con 0x7f3d84101de0 2024-09-19T06:01:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.142+0000 7f3d7a7fc640 1 -- 172.21.15.5:0/1322864023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8000f450 con 0x7f3d84101de0 2024-09-19T06:01:27.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.142+0000 7f3d8b274640 1 -- 172.21.15.5:0/1322864023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d841a4fe0 con 0x7f3d84101de0 2024-09-19T06:01:27.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:26 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/4068617516' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T06:01:27.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:26 smithi049 ceph-mon[34640]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T06:01:27.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.143+0000 7f3d8b274640 1 -- 172.21.15.5:0/1322864023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d841021c0 con 0x7f3d84101de0 2024-09-19T06:01:27.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.144+0000 7f3d7a7fc640 1 -- 172.21.15.5:0/1322864023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f3d80004f00 con 0x7f3d84101de0 2024-09-19T06:01:27.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.145+0000 7f3d7a7fc640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3d700a32c0 0x7f3d700a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:27.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.145+0000 7f3d8880f640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3d700a32c0 0x7f3d700a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:27.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.145+0000 7f3d7a7fc640 1 -- 172.21.15.5:0/1322864023 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7f3d800cf830 con 0x7f3d84101de0 2024-09-19T06:01:27.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.145+0000 7f3d8880f640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3d700a32c0 0x7f3d700a5780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d74031440 tx=0x7f3d740095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:27.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.148+0000 7f3d7a7fc640 1 -- 172.21.15.5:0/1322864023 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3d80014030 con 0x7f3d84101de0 2024-09-19T06:01:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.324+0000 7f3d8b274640 1 -- 172.21.15.5:0/1322864023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3d8402d7c0 con 0x7f3d84101de0 2024-09-19T06:01:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.782+0000 7f3d7a7fc640 1 -- 172.21.15.5:0/1322864023 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7f3d8009c860 con 0x7f3d84101de0 2024-09-19T06:01:27.784 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-09-19T06:01:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.784+0000 7f3d8b274640 1 -- 172.21.15.5:0/1322864023 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3d700a32c0 msgr2=0x7f3d700a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.784+0000 7f3d8b274640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3d700a32c0 0x7f3d700a5780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d74031440 tx=0x7f3d740095f0 comp rx=0 tx=0).stop 2024-09-19T06:01:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.784+0000 7f3d8b274640 1 -- 172.21.15.5:0/1322864023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84101de0 msgr2=0x7f3d84198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.784+0000 7f3d8b274640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84101de0 0x7f3d84198460 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000c970 tx=0x7f3d8000ce30 comp rx=0 tx=0).stop 2024-09-19T06:01:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.784+0000 7f3d8b274640 1 -- 172.21.15.5:0/1322864023 shutdown_connections 2024-09-19T06:01:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.784+0000 7f3d8b274640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3d700a32c0 0x7f3d700a5780 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.784+0000 7f3d8b274640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d84106f10 0x7f3d8419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.784+0000 7f3d8b274640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d84102790 0x7f3d841989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.784+0000 7f3d8b274640 1 --2- 172.21.15.5:0/1322864023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84101de0 0x7f3d84198460 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.784+0000 7f3d8b274640 1 -- 172.21.15.5:0/1322864023 >> 172.21.15.5:0/1322864023 conn(0x7f3d840fd8c0 msgr2=0x7f3d840fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.784+0000 7f3d8b274640 1 -- 172.21.15.5:0/1322864023 shutdown_connections 2024-09-19T06:01:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:27.784+0000 7f3d8b274640 1 -- 172.21.15.5:0/1322864023 wait complete. 2024-09-19T06:01:28.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:27 smithi157 ceph-mon[35346]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T06:01:28.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:27 smithi157 ceph-mon[35346]: pgmap v272: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:28.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:27 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1322864023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T06:01:28.104 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:27 smithi005 ceph-mon[30955]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T06:01:28.104 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:27 smithi005 ceph-mon[30955]: pgmap v272: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:28.105 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:27 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1322864023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T06:01:28.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:27 smithi049 ceph-mon[34640]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T06:01:28.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:27 smithi049 ceph-mon[34640]: pgmap v272: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:28.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:27 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1322864023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T06:01:28.578 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T06:01:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:28 smithi005 ceph-mon[30955]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T06:01:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:28 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1322864023' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T06:01:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:28 smithi005 ceph-mon[30955]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T06:01:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:28 smithi157 ceph-mon[35346]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T06:01:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:28 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1322864023' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T06:01:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:28 smithi157 ceph-mon[35346]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T06:01:29.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:28 smithi049 ceph-mon[34640]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T06:01:29.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:28 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1322864023' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T06:01:29.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:28 smithi049 ceph-mon[34640]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T06:01:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.309+0000 7f59f34b1640 1 -- 172.21.15.5:0/3773529738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ec1002f0 msgr2=0x7f59ec1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.309+0000 7f59f34b1640 1 --2- 172.21.15.5:0/3773529738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ec1002f0 0x7f59ec1006d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009930 tx=0x7f59e802a420 comp rx=0 tx=0).stop 2024-09-19T06:01:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.309+0000 7f59f34b1640 1 -- 172.21.15.5:0/3773529738 shutdown_connections 2024-09-19T06:01:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.309+0000 7f59f34b1640 1 --2- 172.21.15.5:0/3773529738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59ec106f50 0x7f59ec10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.309+0000 7f59f34b1640 1 --2- 172.21.15.5:0/3773529738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59ec100ca0 0x7f59ec106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.309+0000 7f59f34b1640 1 --2- 172.21.15.5:0/3773529738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ec1002f0 0x7f59ec1006d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.309+0000 7f59f34b1640 1 -- 172.21.15.5:0/3773529738 >> 172.21.15.5:0/3773529738 conn(0x7f59ec077210 msgr2=0x7f59ec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.309+0000 7f59f34b1640 1 -- 172.21.15.5:0/3773529738 shutdown_connections 2024-09-19T06:01:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.309+0000 7f59f34b1640 1 -- 172.21.15.5:0/3773529738 wait complete. 2024-09-19T06:01:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.309+0000 7f59f34b1640 1 Processor -- start 2024-09-19T06:01:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.309+0000 7f59f34b1640 1 -- start start 2024-09-19T06:01:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f34b1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59ec1002f0 0x7f59ec101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f34b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59ec100ca0 0x7f59ec1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f34b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ec106f50 0x7f59ec101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f34b1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec111e50 con 0x7f59ec106f50 2024-09-19T06:01:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f34b1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec111cd0 con 0x7f59ec1002f0 2024-09-19T06:01:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f34b1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec111fd0 con 0x7f59ec100ca0 2024-09-19T06:01:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f1a4e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ec106f50 0x7f59ec101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f1a4e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ec106f50 0x7f59ec101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54816/0 (socket says 172.21.15.5:54816) 2024-09-19T06:01:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f1a4e640 1 -- 172.21.15.5:0/1310659300 learned_addr learned my addr 172.21.15.5:0/1310659300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f124d640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59ec1002f0 0x7f59ec101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f1a4e640 1 -- 172.21.15.5:0/1310659300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59ec100ca0 msgr2=0x7f59ec1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f0a4c640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59ec100ca0 0x7f59ec1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f1a4e640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59ec100ca0 0x7f59ec1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f1a4e640 1 -- 172.21.15.5:0/1310659300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59ec1002f0 msgr2=0x7f59ec101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f1a4e640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59ec1002f0 0x7f59ec101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f1a4e640 1 -- 172.21.15.5:0/1310659300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ec1a6850 con 0x7f59ec106f50 2024-09-19T06:01:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f124d640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59ec1002f0 0x7f59ec101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:01:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.310+0000 7f59f1a4e640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ec106f50 0x7f59ec101cf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f59e400eaa0 tx=0x7f59e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.311+0000 7f59e27fc640 1 -- 172.21.15.5:0/1310659300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e400cdb0 con 0x7f59ec106f50 2024-09-19T06:01:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.311+0000 7f59f34b1640 1 -- 172.21.15.5:0/1310659300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ec1a6b40 con 0x7f59ec106f50 2024-09-19T06:01:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.311+0000 7f59e27fc640 1 -- 172.21.15.5:0/1310659300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59e4005020 con 0x7f59ec106f50 2024-09-19T06:01:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.311+0000 7f59e27fc640 1 -- 172.21.15.5:0/1310659300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e40116d0 con 0x7f59ec106f50 2024-09-19T06:01:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.311+0000 7f59f34b1640 1 -- 172.21.15.5:0/1310659300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ec1a7080 con 0x7f59ec106f50 2024-09-19T06:01:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.312+0000 7f59f34b1640 1 -- 172.21.15.5:0/1310659300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ec1006d0 con 0x7f59ec106f50 2024-09-19T06:01:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.314+0000 7f59e27fc640 1 -- 172.21.15.5:0/1310659300 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f59e40051d0 con 0x7f59ec106f50 2024-09-19T06:01:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.314+0000 7f59e27fc640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f59d80a3240 0x7f59d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.315+0000 7f59f124d640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f59d80a3240 0x7f59d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.315+0000 7f59e27fc640 1 -- 172.21.15.5:0/1310659300 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7f59e40cfa30 con 0x7f59ec106f50 2024-09-19T06:01:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.315+0000 7f59f124d640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f59d80a3240 0x7f59d80a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f59e8000c00 tx=0x7f59e8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.318+0000 7f59e27fc640 1 -- 172.21.15.5:0/1310659300 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59e4016030 con 0x7f59ec106f50 2024-09-19T06:01:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.529+0000 7f59f34b1640 1 -- 172.21.15.5:0/1310659300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f59ec02d7c0 con 0x7f59ec106f50 2024-09-19T06:01:29.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.784+0000 7f59e27fc640 1 -- 172.21.15.5:0/1310659300 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f59e409c860 con 0x7f59ec106f50 2024-09-19T06:01:29.786 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T06:01:29.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.786+0000 7f59f34b1640 1 -- 172.21.15.5:0/1310659300 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f59d80a3240 msgr2=0x7f59d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:29.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.786+0000 7f59f34b1640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f59d80a3240 0x7f59d80a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f59e8000c00 tx=0x7f59e8031400 comp rx=0 tx=0).stop 2024-09-19T06:01:29.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.786+0000 7f59f34b1640 1 -- 172.21.15.5:0/1310659300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ec106f50 msgr2=0x7f59ec101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.786+0000 7f59f34b1640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ec106f50 0x7f59ec101cf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f59e400eaa0 tx=0x7f59e400ef60 comp rx=0 tx=0).stop 2024-09-19T06:01:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.787+0000 7f59f34b1640 1 -- 172.21.15.5:0/1310659300 shutdown_connections 2024-09-19T06:01:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.787+0000 7f59f34b1640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f59d80a3240 0x7f59d80a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.787+0000 7f59f34b1640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ec106f50 0x7f59ec101cf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.787+0000 7f59f34b1640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59ec100ca0 0x7f59ec1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.787+0000 7f59f34b1640 1 --2- 172.21.15.5:0/1310659300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59ec1002f0 0x7f59ec101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.787+0000 7f59f34b1640 1 -- 172.21.15.5:0/1310659300 >> 172.21.15.5:0/1310659300 conn(0x7f59ec077210 msgr2=0x7f59ec1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.787+0000 7f59f34b1640 1 -- 172.21.15.5:0/1310659300 shutdown_connections 2024-09-19T06:01:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:29.787+0000 7f59f34b1640 1 -- 172.21.15.5:0/1310659300 wait complete. 2024-09-19T06:01:30.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:29 smithi157 ceph-mon[35346]: pgmap v274: 97 pgs: 8 creating+peering, 79 unknown, 10 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:30.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:29 smithi157 ceph-mon[35346]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T06:01:30.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:29 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1310659300' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T06:01:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:29 smithi157 ceph-mon[35346]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T06:01:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:29 smithi157 ceph-mon[35346]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T06:01:30.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:29 smithi005 ceph-mon[30955]: pgmap v274: 97 pgs: 8 creating+peering, 79 unknown, 10 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:30.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:29 smithi005 ceph-mon[30955]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T06:01:30.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:29 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1310659300' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T06:01:30.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:29 smithi005 ceph-mon[30955]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T06:01:30.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:29 smithi005 ceph-mon[30955]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T06:01:30.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:29 smithi049 ceph-mon[34640]: pgmap v274: 97 pgs: 8 creating+peering, 79 unknown, 10 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:01:30.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:29 smithi049 ceph-mon[34640]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T06:01:30.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:29 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1310659300' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T06:01:30.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:29 smithi049 ceph-mon[34640]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T06:01:30.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:29 smithi049 ceph-mon[34640]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T06:01:30.472 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:01:30.731 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:31.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:30 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1310659300' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T06:01:31.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:30 smithi157 ceph-mon[35346]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T06:01:31.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:30 smithi157 ceph-mon[35346]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T06:01:31.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:30 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1310659300' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T06:01:31.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:30 smithi005 ceph-mon[30955]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T06:01:31.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:30 smithi005 ceph-mon[30955]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T06:01:31.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:30 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1310659300' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T06:01:31.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:30 smithi049 ceph-mon[34640]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T06:01:31.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:30 smithi049 ceph-mon[34640]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T06:01:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.197+0000 7f1e33b56640 1 -- 172.21.15.5:0/2051579924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e2c102790 msgr2=0x7f1e2c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.197+0000 7f1e33b56640 1 --2- 172.21.15.5:0/2051579924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e2c102790 0x7f1e2c1067e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009900 tx=0x7f1e2002a410 comp rx=0 tx=0).stop 2024-09-19T06:01:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.197+0000 7f1e33b56640 1 -- 172.21.15.5:0/2051579924 shutdown_connections 2024-09-19T06:01:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.197+0000 7f1e33b56640 1 --2- 172.21.15.5:0/2051579924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e2c106f10 0x7f1e2c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.197+0000 7f1e33b56640 1 --2- 172.21.15.5:0/2051579924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e2c102790 0x7f1e2c1067e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.197+0000 7f1e33b56640 1 --2- 172.21.15.5:0/2051579924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e2c101de0 0x7f1e2c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.197+0000 7f1e33b56640 1 -- 172.21.15.5:0/2051579924 >> 172.21.15.5:0/2051579924 conn(0x7f1e2c0fd8c0 msgr2=0x7f1e2c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.197+0000 7f1e33b56640 1 -- 172.21.15.5:0/2051579924 shutdown_connections 2024-09-19T06:01:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.197+0000 7f1e33b56640 1 -- 172.21.15.5:0/2051579924 wait complete. 2024-09-19T06:01:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.197+0000 7f1e33b56640 1 Processor -- start 2024-09-19T06:01:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e33b56640 1 -- start start 2024-09-19T06:01:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e33b56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e2c101de0 0x7f1e2c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e33b56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e2c102790 0x7f1e2c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e33b56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e2c106f10 0x7f1e2c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e33b56640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c111e40 con 0x7f1e2c102790 2024-09-19T06:01:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e33b56640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c111cc0 con 0x7f1e2c101de0 2024-09-19T06:01:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e33b56640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c111fc0 con 0x7f1e2c106f10 2024-09-19T06:01:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e310f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e2c102790 0x7f1e2c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e310f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e2c102790 0x7f1e2c1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54824/0 (socket says 172.21.15.5:54824) 2024-09-19T06:01:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e310f1640 1 -- 172.21.15.5:0/1244055486 learned_addr learned my addr 172.21.15.5:0/1244055486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e310f1640 1 -- 172.21.15.5:0/1244055486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e2c106f10 msgr2=0x7f1e2c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e310f1640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e2c106f10 0x7f1e2c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e318f2640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e2c101de0 0x7f1e2c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e310f1640 1 -- 172.21.15.5:0/1244055486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e2c101de0 msgr2=0x7f1e2c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e310f1640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e2c101de0 0x7f1e2c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e310f1640 1 -- 172.21.15.5:0/1244055486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2c19d460 con 0x7f1e2c102790 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.198+0000 7f1e310f1640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e2c102790 0x7f1e2c1989f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e20005940 tx=0x7f1e20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.199+0000 7f1e1effd640 1 -- 172.21.15.5:0/1244055486 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2003f030 con 0x7f1e2c102790 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.199+0000 7f1e1effd640 1 -- 172.21.15.5:0/1244055486 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e20003aa0 con 0x7f1e2c102790 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.199+0000 7f1e33b56640 1 -- 172.21.15.5:0/1244055486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2c19d6f0 con 0x7f1e2c102790 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.199+0000 7f1e1effd640 1 -- 172.21.15.5:0/1244055486 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20035070 con 0x7f1e2c102790 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.199+0000 7f1e33b56640 1 -- 172.21.15.5:0/1244055486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2c1a5030 con 0x7f1e2c102790 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.200+0000 7f1e33b56640 1 -- 172.21.15.5:0/1244055486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2c1021c0 con 0x7f1e2c102790 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.201+0000 7f1e1effd640 1 -- 172.21.15.5:0/1244055486 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f1e20042030 con 0x7f1e2c102790 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.202+0000 7f1e1effd640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f1e180a32c0 0x7f1e180a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.202+0000 7f1e318f2640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f1e180a32c0 0x7f1e180a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.202+0000 7f1e1effd640 1 -- 172.21.15.5:0/1244055486 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f1e200f02c0 con 0x7f1e2c102790 2024-09-19T06:01:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.202+0000 7f1e318f2640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f1e180a32c0 0x7f1e180a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1e28005fd0 tx=0x7f1e28005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.205+0000 7f1e1effd640 1 -- 172.21.15.5:0/1244055486 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1e200bd090 con 0x7f1e2c102790 2024-09-19T06:01:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.384+0000 7f1e33b56640 1 -- 172.21.15.5:0/1244055486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e2c10a0f0 con 0x7f1e2c102790 2024-09-19T06:01:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.385+0000 7f1e1effd640 1 -- 172.21.15.5:0/1244055486 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22206 (secure 0 0 0) 0x7f1e200bc990 con 0x7f1e2c102790 2024-09-19T06:01:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:31.389 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":81,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","created":"2024-09-19T05:53:51.454068+0000","modified":"2024-09-19T06:01:29.530796+0000","last_up_change":"2024-09-19T06:00:38.837916+0000","last_in_change":"2024-09-19T06:00:15.609776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:56:55.213931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:01:25.648588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:01:27.760408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f3c10d60-d354-407f-9cdf-099344599844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6803","nonce":3442981810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6805","nonce":3442981810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6809","nonce":3442981810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6807","nonce":3442981810}]},"public_addr":"172.21.15.5:6803/3442981810","cluster_addr":"172.21.15.5:6805/3442981810","heartbeat_back_addr":"172.21.15.5:6809/3442981810","heartbeat_front_addr":"172.21.15.5:6807/3442981810","state":["exists","up"]},{"osd":1,"uuid":"5f40f636-7761-45ac-87f3-910a2fa2feb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448265014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6803","nonce":1448265014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6807","nonce":1448265014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6805","nonce":1448265014}]},"public_addr":"172.21.15.49:6801/1448265014","cluster_addr":"172.21.15.49:6803/1448265014","heartbeat_back_addr":"172.21.15.49:6807/1448265014","heartbeat_front_addr":"172.21.15.49:6805/1448265014","state":["exists","up"]},{"osd":2,"uuid":"ad00940c-4fdd-4f5f-9eaa-b1f38bcac526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6801","nonce":2319626622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6803","nonce":2319626622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6807","nonce":2319626622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6805","nonce":2319626622}]},"public_addr":"172.21.15.157:6801/2319626622","cluster_addr":"172.21.15.157:6803/2319626622","heartbeat_back_addr":"172.21.15.157:6807/2319626622","heartbeat_front_addr":"172.21.15.157:6805/2319626622","state":["exists","up"]},{"osd":3,"uuid":"63fec329-1b62-4c5e-b8ca-084a84d4d762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6811","nonce":1452491753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6813","nonce":1452491753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6817","nonce":1452491753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6815","nonce":1452491753}]},"public_addr":"172.21.15.5:6811/1452491753","cluster_addr":"172.21.15.5:6813/1452491753","heartbeat_back_addr":"172.21.15.5:6817/1452491753","heartbeat_front_addr":"172.21.15.5:6815/1452491753","state":["exists","up"]},{"osd":4,"uuid":"ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6809","nonce":3114952580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6811","nonce":3114952580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6815","nonce":3114952580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6813","nonce":3114952580}]},"public_addr":"172.21.15.49:6809/3114952580","cluster_addr":"172.21.15.49:6811/3114952580","heartbeat_back_addr":"172.21.15.49:6815/3114952580","heartbeat_front_addr":"172.21.15.49:6813/3114952580","state":["exists","up"]},{"osd":5,"uuid":"21a7a7d3-ce42-4688-bfec-714be635eaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6809","nonce":3219063971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6811","nonce":3219063971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6815","nonce":3219063971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6813","nonce":3219063971}]},"public_addr":"172.21.15.157:6809/3219063971","cluster_addr":"172.21.15.157:6811/3219063971","heartbeat_back_addr":"172.21.15.157:6815/3219063971","heartbeat_front_addr":"172.21.15.157:6813/3219063971","state":["exists","up"]},{"osd":6,"uuid":"cbef0e08-d6b0-4515-a2ee-2c81b048583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6819","nonce":2957795194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6821","nonce":2957795194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6825","nonce":2957795194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6823","nonce":2957795194}]},"public_addr":"172.21.15.5:6819/2957795194","cluster_addr":"172.21.15.5:6821/2957795194","heartbeat_back_addr":"172.21.15.5:6825/2957795194","heartbeat_front_addr":"172.21.15.5:6823/2957795194","state":["exists","up"]},{"osd":7,"uuid":"d261ac55-532d-4aea-9b3e-ed2b4eeb1da6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6817","nonce":2959731400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6819","nonce":2959731400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6823","nonce":2959731400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6821","nonce":2959731400}]},"public_addr":"172.21.15.49:6817/2959731400","cluster_addr":"172.21.15.49:6819/2959731400","heartbeat_back_addr":"172.21.15.49:6823/2959731400","heartbeat_front_addr":"172.21.15.49:6821/2959731400","state":["exists","up"]},{"osd":8,"uuid":"15d0eaf7-d604-4546-b393-4d6d39018f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6817","nonce":290026859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6819","nonce":290026859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6823","nonce":290026859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6821","nonce":290026859}]},"public_addr":"172.21.15.157:6817/290026859","cluster_addr":"172.21.15.157:6819/290026859","heartbeat_back_addr":"172.21.15.157:6823/290026859","heartbeat_front_addr":"172.21.15.157:6821/290026859","state":["exists","up"]},{"osd":9,"uuid":"30c08537-f593-4299-bfa1-fa7ba8a95169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6827","nonce":4122448237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6829","nonce":4122448237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6833","nonce":4122448237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6831","nonce":4122448237}]},"public_addr":"172.21.15.5:6827/4122448237","cluster_addr":"172.21.15.5:6829/4122448237","heartbeat_back_addr":"172.21.15.5:6833/4122448237","heartbeat_front_addr":"172.21.15.5:6831/4122448237","state":["exists","up"]},{"osd":10,"uuid":"68821e98-bd81-49ec-8cd6-51fb9880b553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6825","nonce":2551748002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6827","nonce":2551748002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6831","nonce":2551748002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6829","nonce":2551748002}]},"public_addr":"172.21.15.49:6825/2551748002","cluster_addr":"172.21.15.49:6827/2551748002","heartbeat_back_addr":"172.21.15.49:6831/2551748002","heartbeat_front_addr":"172.21.15.49:6829/2551748002","state":["exists","up"]},{"osd":11,"uuid":"7e443b46-2697-4d0b-8452-868463a1ee1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6825","nonce":1996582544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6827","nonce":1996582544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6831","nonce":1996582544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6829","nonce":1996582544}]},"public_addr":"172.21.15.157:6825/1996582544","cluster_addr":"172.21.15.157:6827/1996582544","heartbeat_back_addr":"172.21.15.157:6831/1996582544","heartbeat_front_addr":"172.21.15.157:6829/1996582544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:01.453592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:27.194705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:51.525385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:17.676749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:43.075555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:07.258154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:32.112583+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:57.306534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:21.320950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:47.164610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:12.548337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:37.262580+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4102275559":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/2593868621":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/1340147787":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6800/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/850678758":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6800/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2930274429":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/1367598148":"2024-09-20T05:54:15.334457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:01:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.387+0000 7f1e33b56640 1 -- 172.21.15.5:0/1244055486 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f1e180a32c0 msgr2=0x7f1e180a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.387+0000 7f1e33b56640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f1e180a32c0 0x7f1e180a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1e28005fd0 tx=0x7f1e28005850 comp rx=0 tx=0).stop 2024-09-19T06:01:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.387+0000 7f1e33b56640 1 -- 172.21.15.5:0/1244055486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e2c102790 msgr2=0x7f1e2c1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.387+0000 7f1e33b56640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e2c102790 0x7f1e2c1989f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e20005940 tx=0x7f1e20004270 comp rx=0 tx=0).stop 2024-09-19T06:01:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.387+0000 7f1e33b56640 1 -- 172.21.15.5:0/1244055486 shutdown_connections 2024-09-19T06:01:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.387+0000 7f1e33b56640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f1e180a32c0 0x7f1e180a5780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.387+0000 7f1e33b56640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e2c106f10 0x7f1e2c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.387+0000 7f1e33b56640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e2c102790 0x7f1e2c1989f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.387+0000 7f1e33b56640 1 --2- 172.21.15.5:0/1244055486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e2c101de0 0x7f1e2c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.387+0000 7f1e33b56640 1 -- 172.21.15.5:0/1244055486 >> 172.21.15.5:0/1244055486 conn(0x7f1e2c0fd8c0 msgr2=0x7f1e2c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.388+0000 7f1e33b56640 1 -- 172.21.15.5:0/1244055486 shutdown_connections 2024-09-19T06:01:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:31.388+0000 7f1e33b56640 1 -- 172.21.15.5:0/1244055486 wait complete. 2024-09-19T06:01:32.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:01:32.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:31 smithi157 ceph-mon[35346]: pgmap v277: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-19T06:01:32.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:31 smithi157 ceph-mon[35346]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T06:01:32.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:31 smithi157 ceph-mon[35346]: Cluster is now healthy 2024-09-19T06:01:32.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:31 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] up:active 2024-09-19T06:01:32.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:31 smithi157 ceph-mon[35346]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:01:32.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:31 smithi157 ceph-mon[35346]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:01:32.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:31 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1244055486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:01:32.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:31 smithi005 ceph-mon[30955]: pgmap v277: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-19T06:01:32.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:31 smithi005 ceph-mon[30955]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T06:01:32.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:31 smithi005 ceph-mon[30955]: Cluster is now healthy 2024-09-19T06:01:32.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:31 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] up:active 2024-09-19T06:01:32.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:31 smithi005 ceph-mon[30955]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:01:32.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:31 smithi005 ceph-mon[30955]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:01:32.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:31 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1244055486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:01:32.114 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:31 smithi049 ceph-mon[34640]: pgmap v277: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-19T06:01:32.114 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:31 smithi049 ceph-mon[34640]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T06:01:32.114 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:31 smithi049 ceph-mon[34640]: Cluster is now healthy 2024-09-19T06:01:32.114 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:31 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] up:active 2024-09-19T06:01:32.115 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:31 smithi049 ceph-mon[34640]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:01:32.115 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:31 smithi049 ceph-mon[34640]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:01:32.115 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:31 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1244055486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:01:32.299 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.763+0000 7f50a2877640 1 -- 172.21.15.5:0/1226298335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509c102ef0 msgr2=0x7f509c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.763+0000 7f50a2877640 1 --2- 172.21.15.5:0/1226298335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509c102ef0 0x7f509c10da40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5094009ca0 tx=0x7f509402a950 comp rx=0 tx=0).stop 2024-09-19T06:01:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.763+0000 7f50a2877640 1 -- 172.21.15.5:0/1226298335 shutdown_connections 2024-09-19T06:01:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.763+0000 7f50a2877640 1 --2- 172.21.15.5:0/1226298335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509c102ef0 0x7f509c10da40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.763+0000 7f50a2877640 1 --2- 172.21.15.5:0/1226298335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509c102570 0x7f509c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.763+0000 7f50a2877640 1 --2- 172.21.15.5:0/1226298335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f509c100e20 0x7f509c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.763+0000 7f50a2877640 1 -- 172.21.15.5:0/1226298335 >> 172.21.15.5:0/1226298335 conn(0x7f509c0fc7d0 msgr2=0x7f509c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.763+0000 7f50a2877640 1 -- 172.21.15.5:0/1226298335 shutdown_connections 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.763+0000 7f50a2877640 1 -- 172.21.15.5:0/1226298335 wait complete. 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.764+0000 7f50a2877640 1 Processor -- start 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.764+0000 7f50a2877640 1 -- start start 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.764+0000 7f50a2877640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509c100e20 0x7f509c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.764+0000 7f50a2877640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f509c102570 0x7f509c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.764+0000 7f50a2877640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509c102ef0 0x7f509c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.764+0000 7f50a2877640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f509c110e30 con 0x7f509c102ef0 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.764+0000 7f50a2877640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f509c110cb0 con 0x7f509c102570 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.764+0000 7f50a2877640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f509c110fb0 con 0x7f509c100e20 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.764+0000 7f50a0e14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509c102ef0 0x7f509c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.764+0000 7f50a0e14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509c102ef0 0x7f509c19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54846/0 (socket says 172.21.15.5:54846) 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.764+0000 7f50a0e14640 1 -- 172.21.15.5:0/2588203183 learned_addr learned my addr 172.21.15.5:0/2588203183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.764+0000 7f509bfff640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509c100e20 0x7f509c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.765+0000 7f5093fff640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f509c102570 0x7f509c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.765+0000 7f50a0e14640 1 -- 172.21.15.5:0/2588203183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509c100e20 msgr2=0x7f509c1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.765+0000 7f50a0e14640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509c100e20 0x7f509c1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.765+0000 7f50a0e14640 1 -- 172.21.15.5:0/2588203183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f509c102570 msgr2=0x7f509c1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.765+0000 7f50a0e14640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f509c102570 0x7f509c1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.765+0000 7f50a0e14640 1 -- 172.21.15.5:0/2588203183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509c19c260 con 0x7f509c102ef0 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.765+0000 7f50a0e14640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509c102ef0 0x7f509c19bb80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f509402ae70 tx=0x7f509402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.765+0000 7f5099ffb640 1 -- 172.21.15.5:0/2588203183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094034040 con 0x7f509c102ef0 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.765+0000 7f5099ffb640 1 -- 172.21.15.5:0/2588203183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50940308f0 con 0x7f509c102ef0 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.765+0000 7f50a2877640 1 -- 172.21.15.5:0/2588203183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509c19c4f0 con 0x7f509c102ef0 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.765+0000 7f5099ffb640 1 -- 172.21.15.5:0/2588203183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094004750 con 0x7f509c102ef0 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.765+0000 7f50a2877640 1 -- 172.21.15.5:0/2588203183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f509c1a3e30 con 0x7f509c102ef0 2024-09-19T06:01:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.766+0000 7f50a2877640 1 -- 172.21.15.5:0/2588203183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f509c101200 con 0x7f509c102ef0 2024-09-19T06:01:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.767+0000 7f5099ffb640 1 -- 172.21.15.5:0/2588203183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f5094037020 con 0x7f509c102ef0 2024-09-19T06:01:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.768+0000 7f5099ffb640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f50880a3240 0x7f50880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.768+0000 7f509bfff640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f50880a3240 0x7f50880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.768+0000 7f5099ffb640 1 -- 172.21.15.5:0/2588203183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f50940f0780 con 0x7f509c102ef0 2024-09-19T06:01:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.768+0000 7f509bfff640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f50880a3240 0x7f50880a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f508c00abe0 tx=0x7f508c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.772+0000 7f5099ffb640 1 -- 172.21.15.5:0/2588203183 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f50940bd5f0 con 0x7f509c102ef0 2024-09-19T06:01:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.953+0000 7f50a2877640 1 -- 172.21.15.5:0/2588203183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f509c1a4000 con 0x7f509c102ef0 2024-09-19T06:01:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.954+0000 7f5099ffb640 1 -- 172.21.15.5:0/2588203183 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22206 (secure 0 0 0) 0x7f50940bcef0 con 0x7f509c102ef0 2024-09-19T06:01:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:32.956 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":81,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","created":"2024-09-19T05:53:51.454068+0000","modified":"2024-09-19T06:01:29.530796+0000","last_up_change":"2024-09-19T06:00:38.837916+0000","last_in_change":"2024-09-19T06:00:15.609776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:56:55.213931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:01:25.648588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:01:27.760408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f3c10d60-d354-407f-9cdf-099344599844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6803","nonce":3442981810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6805","nonce":3442981810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6809","nonce":3442981810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6807","nonce":3442981810}]},"public_addr":"172.21.15.5:6803/3442981810","cluster_addr":"172.21.15.5:6805/3442981810","heartbeat_back_addr":"172.21.15.5:6809/3442981810","heartbeat_front_addr":"172.21.15.5:6807/3442981810","state":["exists","up"]},{"osd":1,"uuid":"5f40f636-7761-45ac-87f3-910a2fa2feb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448265014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6803","nonce":1448265014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6807","nonce":1448265014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6805","nonce":1448265014}]},"public_addr":"172.21.15.49:6801/1448265014","cluster_addr":"172.21.15.49:6803/1448265014","heartbeat_back_addr":"172.21.15.49:6807/1448265014","heartbeat_front_addr":"172.21.15.49:6805/1448265014","state":["exists","up"]},{"osd":2,"uuid":"ad00940c-4fdd-4f5f-9eaa-b1f38bcac526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6801","nonce":2319626622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6803","nonce":2319626622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6807","nonce":2319626622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6805","nonce":2319626622}]},"public_addr":"172.21.15.157:6801/2319626622","cluster_addr":"172.21.15.157:6803/2319626622","heartbeat_back_addr":"172.21.15.157:6807/2319626622","heartbeat_front_addr":"172.21.15.157:6805/2319626622","state":["exists","up"]},{"osd":3,"uuid":"63fec329-1b62-4c5e-b8ca-084a84d4d762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6811","nonce":1452491753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6813","nonce":1452491753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6817","nonce":1452491753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6815","nonce":1452491753}]},"public_addr":"172.21.15.5:6811/1452491753","cluster_addr":"172.21.15.5:6813/1452491753","heartbeat_back_addr":"172.21.15.5:6817/1452491753","heartbeat_front_addr":"172.21.15.5:6815/1452491753","state":["exists","up"]},{"osd":4,"uuid":"ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6809","nonce":3114952580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6811","nonce":3114952580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6815","nonce":3114952580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6813","nonce":3114952580}]},"public_addr":"172.21.15.49:6809/3114952580","cluster_addr":"172.21.15.49:6811/3114952580","heartbeat_back_addr":"172.21.15.49:6815/3114952580","heartbeat_front_addr":"172.21.15.49:6813/3114952580","state":["exists","up"]},{"osd":5,"uuid":"21a7a7d3-ce42-4688-bfec-714be635eaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6809","nonce":3219063971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6811","nonce":3219063971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6815","nonce":3219063971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6813","nonce":3219063971}]},"public_addr":"172.21.15.157:6809/3219063971","cluster_addr":"172.21.15.157:6811/3219063971","heartbeat_back_addr":"172.21.15.157:6815/3219063971","heartbeat_front_addr":"172.21.15.157:6813/3219063971","state":["exists","up"]},{"osd":6,"uuid":"cbef0e08-d6b0-4515-a2ee-2c81b048583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6819","nonce":2957795194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6821","nonce":2957795194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6825","nonce":2957795194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6823","nonce":2957795194}]},"public_addr":"172.21.15.5:6819/2957795194","cluster_addr":"172.21.15.5:6821/2957795194","heartbeat_back_addr":"172.21.15.5:6825/2957795194","heartbeat_front_addr":"172.21.15.5:6823/2957795194","state":["exists","up"]},{"osd":7,"uuid":"d261ac55-532d-4aea-9b3e-ed2b4eeb1da6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6817","nonce":2959731400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6819","nonce":2959731400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6823","nonce":2959731400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6821","nonce":2959731400}]},"public_addr":"172.21.15.49:6817/2959731400","cluster_addr":"172.21.15.49:6819/2959731400","heartbeat_back_addr":"172.21.15.49:6823/2959731400","heartbeat_front_addr":"172.21.15.49:6821/2959731400","state":["exists","up"]},{"osd":8,"uuid":"15d0eaf7-d604-4546-b393-4d6d39018f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6817","nonce":290026859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6819","nonce":290026859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6823","nonce":290026859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6821","nonce":290026859}]},"public_addr":"172.21.15.157:6817/290026859","cluster_addr":"172.21.15.157:6819/290026859","heartbeat_back_addr":"172.21.15.157:6823/290026859","heartbeat_front_addr":"172.21.15.157:6821/290026859","state":["exists","up"]},{"osd":9,"uuid":"30c08537-f593-4299-bfa1-fa7ba8a95169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6827","nonce":4122448237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6829","nonce":4122448237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6833","nonce":4122448237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6831","nonce":4122448237}]},"public_addr":"172.21.15.5:6827/4122448237","cluster_addr":"172.21.15.5:6829/4122448237","heartbeat_back_addr":"172.21.15.5:6833/4122448237","heartbeat_front_addr":"172.21.15.5:6831/4122448237","state":["exists","up"]},{"osd":10,"uuid":"68821e98-bd81-49ec-8cd6-51fb9880b553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6825","nonce":2551748002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6827","nonce":2551748002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6831","nonce":2551748002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6829","nonce":2551748002}]},"public_addr":"172.21.15.49:6825/2551748002","cluster_addr":"172.21.15.49:6827/2551748002","heartbeat_back_addr":"172.21.15.49:6831/2551748002","heartbeat_front_addr":"172.21.15.49:6829/2551748002","state":["exists","up"]},{"osd":11,"uuid":"7e443b46-2697-4d0b-8452-868463a1ee1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6825","nonce":1996582544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6827","nonce":1996582544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6831","nonce":1996582544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6829","nonce":1996582544}]},"public_addr":"172.21.15.157:6825/1996582544","cluster_addr":"172.21.15.157:6827/1996582544","heartbeat_back_addr":"172.21.15.157:6831/1996582544","heartbeat_front_addr":"172.21.15.157:6829/1996582544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:01.453592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:27.194705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:51.525385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:17.676749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:43.075555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:07.258154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:32.112583+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:57.306534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:21.320950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:47.164610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:12.548337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:37.262580+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4102275559":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/2593868621":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/1340147787":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6800/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/850678758":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6800/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2930274429":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/1367598148":"2024-09-20T05:54:15.334457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:01:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.956+0000 7f50a2877640 1 -- 172.21.15.5:0/2588203183 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f50880a3240 msgr2=0x7f50880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.956+0000 7f50a2877640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f50880a3240 0x7f50880a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f508c00abe0 tx=0x7f508c00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:01:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.956+0000 7f50a2877640 1 -- 172.21.15.5:0/2588203183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509c102ef0 msgr2=0x7f509c19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.956+0000 7f50a2877640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509c102ef0 0x7f509c19bb80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f509402ae70 tx=0x7f509402f5d0 comp rx=0 tx=0).stop 2024-09-19T06:01:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.957+0000 7f50a2877640 1 -- 172.21.15.5:0/2588203183 shutdown_connections 2024-09-19T06:01:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.957+0000 7f50a2877640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f50880a3240 0x7f50880a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.957+0000 7f50a2877640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509c102ef0 0x7f509c19bb80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.957+0000 7f50a2877640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f509c102570 0x7f509c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.957+0000 7f50a2877640 1 --2- 172.21.15.5:0/2588203183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509c100e20 0x7f509c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.957+0000 7f50a2877640 1 -- 172.21.15.5:0/2588203183 >> 172.21.15.5:0/2588203183 conn(0x7f509c0fc7d0 msgr2=0x7f509c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.957+0000 7f50a2877640 1 -- 172.21.15.5:0/2588203183 shutdown_connections 2024-09-19T06:01:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:32.957+0000 7f50a2877640 1 -- 172.21.15.5:0/2588203183 wait complete. 2024-09-19T06:01:33.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T06:01:33.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:33 smithi005 ceph-mon[30955]: pgmap v278: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-19T06:01:33.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:33 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2588203183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:01:33.975 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:33 smithi157 ceph-mon[35346]: pgmap v278: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-19T06:01:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:33 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2588203183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:01:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:33 smithi049 ceph-mon[34640]: pgmap v278: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-19T06:01:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:33 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2588203183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:01:34.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.461+0000 7fd545361640 1 -- 172.21.15.5:0/77868734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5401036e0 msgr2=0x7fd540107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:34.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.461+0000 7fd545361640 1 --2- 172.21.15.5:0/77868734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5401036e0 0x7fd540107b90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd528009930 tx=0x7fd52802a420 comp rx=0 tx=0).stop 2024-09-19T06:01:34.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.461+0000 7fd545361640 1 -- 172.21.15.5:0/77868734 shutdown_connections 2024-09-19T06:01:34.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.461+0000 7fd545361640 1 --2- 172.21.15.5:0/77868734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5401082a0 0x7fd54010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:34.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.461+0000 7fd545361640 1 --2- 172.21.15.5:0/77868734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5401036e0 0x7fd540107b90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:34.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.461+0000 7fd545361640 1 --2- 172.21.15.5:0/77868734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd540102d30 0x7fd540103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:34.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.461+0000 7fd545361640 1 -- 172.21.15.5:0/77868734 >> 172.21.15.5:0/77868734 conn(0x7fd5400fc7d0 msgr2=0x7fd5400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:34.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.461+0000 7fd545361640 1 -- 172.21.15.5:0/77868734 shutdown_connections 2024-09-19T06:01:34.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.461+0000 7fd545361640 1 -- 172.21.15.5:0/77868734 wait complete. 2024-09-19T06:01:34.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.462+0000 7fd545361640 1 Processor -- start 2024-09-19T06:01:34.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.462+0000 7fd545361640 1 -- start start 2024-09-19T06:01:34.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.462+0000 7fd545361640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd540102d30 0x7fd540100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:34.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.462+0000 7fd545361640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5401036e0 0x7fd540100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:34.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.462+0000 7fd545361640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5401082a0 0x7fd540101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:34.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.462+0000 7fd545361640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd540111f40 con 0x7fd5401036e0 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.462+0000 7fd545361640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd540111dc0 con 0x7fd540102d30 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.462+0000 7fd545361640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401120c0 con 0x7fd5401082a0 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd53e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5401036e0 0x7fd540100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd53e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5401036e0 0x7fd540100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52312/0 (socket says 172.21.15.5:52312) 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd53e7fc640 1 -- 172.21.15.5:0/616190631 learned_addr learned my addr 172.21.15.5:0/616190631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd53effd640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd540102d30 0x7fd540100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd53f7fe640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5401082a0 0x7fd540101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd53e7fc640 1 -- 172.21.15.5:0/616190631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5401082a0 msgr2=0x7fd540101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd53e7fc640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5401082a0 0x7fd540101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd53e7fc640 1 -- 172.21.15.5:0/616190631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd540102d30 msgr2=0x7fd540100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd53e7fc640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd540102d30 0x7fd540100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd53e7fc640 1 -- 172.21.15.5:0/616190631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5401a6960 con 0x7fd5401036e0 2024-09-19T06:01:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd53f7fe640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5401082a0 0x7fd540101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:01:34.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd53e7fc640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5401036e0 0x7fd540100ed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd528031440 tx=0x7fd528003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:34.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd537fff640 1 -- 172.21.15.5:0/616190631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52803f070 con 0x7fd5401036e0 2024-09-19T06:01:34.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd545361640 1 -- 172.21.15.5:0/616190631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5401a6bf0 con 0x7fd5401036e0 2024-09-19T06:01:34.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd537fff640 1 -- 172.21.15.5:0/616190631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd528031d60 con 0x7fd5401036e0 2024-09-19T06:01:34.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.463+0000 7fd537fff640 1 -- 172.21.15.5:0/616190631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528035030 con 0x7fd5401036e0 2024-09-19T06:01:34.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.464+0000 7fd545361640 1 -- 172.21.15.5:0/616190631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5401a7120 con 0x7fd5401036e0 2024-09-19T06:01:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.465+0000 7fd545361640 1 -- 172.21.15.5:0/616190631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd540103110 con 0x7fd5401036e0 2024-09-19T06:01:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.466+0000 7fd537fff640 1 -- 172.21.15.5:0/616190631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fd528042070 con 0x7fd5401036e0 2024-09-19T06:01:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.467+0000 7fd537fff640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd52c0a3240 0x7fd52c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.467+0000 7fd53effd640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd52c0a3240 0x7fd52c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.467+0000 7fd537fff640 1 -- 172.21.15.5:0/616190631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fd5280f03d0 con 0x7fd5401036e0 2024-09-19T06:01:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.467+0000 7fd53effd640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd52c0a3240 0x7fd52c0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd538005c40 tx=0x7fd53800a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.470+0000 7fd537fff640 1 -- 172.21.15.5:0/616190631 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd5280bd240 con 0x7fd5401036e0 2024-09-19T06:01:34.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.682+0000 7fd545361640 1 -- 172.21.15.5:0/616190631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd54010a2b0 con 0x7fd5401036e0 2024-09-19T06:01:34.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.801+0000 7fd537fff640 1 -- 172.21.15.5:0/616190631 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fd5280bcb40 con 0x7fd5401036e0 2024-09-19T06:01:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.803+0000 7fd545361640 1 -- 172.21.15.5:0/616190631 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd52c0a3240 msgr2=0x7fd52c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.803+0000 7fd545361640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd52c0a3240 0x7fd52c0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd538005c40 tx=0x7fd53800a590 comp rx=0 tx=0).stop 2024-09-19T06:01:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.803+0000 7fd545361640 1 -- 172.21.15.5:0/616190631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5401036e0 msgr2=0x7fd540100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.803+0000 7fd545361640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5401036e0 0x7fd540100ed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd528031440 tx=0x7fd528003d50 comp rx=0 tx=0).stop 2024-09-19T06:01:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.803+0000 7fd545361640 1 -- 172.21.15.5:0/616190631 shutdown_connections 2024-09-19T06:01:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.803+0000 7fd545361640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd52c0a3240 0x7fd52c0a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.803+0000 7fd545361640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5401082a0 0x7fd540101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.803+0000 7fd545361640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5401036e0 0x7fd540100ed0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.803+0000 7fd545361640 1 --2- 172.21.15.5:0/616190631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd540102d30 0x7fd540100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.803+0000 7fd545361640 1 -- 172.21.15.5:0/616190631 >> 172.21.15.5:0/616190631 conn(0x7fd5400fc7d0 msgr2=0x7fd540105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.803+0000 7fd545361640 1 -- 172.21.15.5:0/616190631 shutdown_connections 2024-09-19T06:01:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:34.803+0000 7fd545361640 1 -- 172.21.15.5:0/616190631 wait complete. 2024-09-19T06:01:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:34 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/616190631' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T06:01:34.897 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:34 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:01:34.639+0000 7f9e1bdf3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T06:01:34.992 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:34 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/616190631' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T06:01:35.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:34 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/616190631' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T06:01:35.490 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-19T06:01:35.490 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-19T06:01:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:36.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:35 smithi005 ceph-mon[30955]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T06:01:36.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:35 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/616190631' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T06:01:36.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:35 smithi005 ceph-mon[30955]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:01:36.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:35 smithi049 ceph-mon[34640]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T06:01:36.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:35 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/616190631' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T06:01:36.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:35 smithi049 ceph-mon[34640]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:01:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.222+0000 7f2e764a1640 1 -- 172.21.15.5:0/850885955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e70100e20 msgr2=0x7f2e70101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.222+0000 7f2e764a1640 1 --2- 172.21.15.5:0/850885955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e70100e20 0x7f2e70101200 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009900 tx=0x7f2e6002a480 comp rx=0 tx=0).stop 2024-09-19T06:01:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.222+0000 7f2e764a1640 1 -- 172.21.15.5:0/850885955 shutdown_connections 2024-09-19T06:01:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.222+0000 7f2e764a1640 1 --2- 172.21.15.5:0/850885955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e70102ef0 0x7f2e7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.222+0000 7f2e764a1640 1 --2- 172.21.15.5:0/850885955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e70102570 0x7f2e701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.222+0000 7f2e764a1640 1 --2- 172.21.15.5:0/850885955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e70100e20 0x7f2e70101200 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.222+0000 7f2e764a1640 1 -- 172.21.15.5:0/850885955 >> 172.21.15.5:0/850885955 conn(0x7f2e700fc7d0 msgr2=0x7f2e700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.222+0000 7f2e764a1640 1 -- 172.21.15.5:0/850885955 shutdown_connections 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.222+0000 7f2e764a1640 1 -- 172.21.15.5:0/850885955 wait complete. 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e764a1640 1 Processor -- start 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e764a1640 1 -- start start 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e764a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e70100e20 0x7f2e700fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e764a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e70102570 0x7f2e700fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e764a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e70102ef0 0x7f2e700fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e764a1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70110e30 con 0x7f2e70100e20 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e764a1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70110cb0 con 0x7f2e70102ef0 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e764a1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70110fb0 con 0x7f2e70102570 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e70100e20 0x7f2e700fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e70100e20 0x7f2e700fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52344/0 (socket says 172.21.15.5:52344) 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e6ffff640 1 -- 172.21.15.5:0/3533408631 learned_addr learned my addr 172.21.15.5:0/3533408631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e67fff640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e70102570 0x7f2e700fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.223+0000 7f2e74a3e640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e70102ef0 0x7f2e700fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.224+0000 7f2e6ffff640 1 -- 172.21.15.5:0/3533408631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e70102570 msgr2=0x7f2e700fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.224+0000 7f2e6ffff640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e70102570 0x7f2e700fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.224+0000 7f2e6ffff640 1 -- 172.21.15.5:0/3533408631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e70102ef0 msgr2=0x7f2e700fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.224+0000 7f2e6ffff640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e70102ef0 0x7f2e700fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.224+0000 7f2e6ffff640 1 -- 172.21.15.5:0/3533408631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e701a69c0 con 0x7f2e70100e20 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.224+0000 7f2e74a3e640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e70102ef0 0x7f2e700fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.224+0000 7f2e6ffff640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e70100e20 0x7f2e700fcc80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2e60006010 tx=0x7f2e600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.224+0000 7f2e6dffb640 1 -- 172.21.15.5:0/3533408631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6003f030 con 0x7f2e70100e20 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.224+0000 7f2e6dffb640 1 -- 172.21.15.5:0/3533408631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e60003cb0 con 0x7f2e70100e20 2024-09-19T06:01:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.224+0000 7f2e764a1640 1 -- 172.21.15.5:0/3533408631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e701a6c50 con 0x7f2e70100e20 2024-09-19T06:01:36.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.224+0000 7f2e6dffb640 1 -- 172.21.15.5:0/3533408631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60035070 con 0x7f2e70100e20 2024-09-19T06:01:36.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.224+0000 7f2e764a1640 1 -- 172.21.15.5:0/3533408631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e701a7180 con 0x7f2e70100e20 2024-09-19T06:01:36.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.225+0000 7f2e764a1640 1 -- 172.21.15.5:0/3533408631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e70101200 con 0x7f2e70100e20 2024-09-19T06:01:36.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.226+0000 7f2e6dffb640 1 -- 172.21.15.5:0/3533408631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f2e60042030 con 0x7f2e70100e20 2024-09-19T06:01:36.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.227+0000 7f2e6dffb640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f2e5c0a32c0 0x7f2e5c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:36.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.227+0000 7f2e67fff640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f2e5c0a32c0 0x7f2e5c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:36.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.227+0000 7f2e6dffb640 1 -- 172.21.15.5:0/3533408631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f2e600eff30 con 0x7f2e70100e20 2024-09-19T06:01:36.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.228+0000 7f2e67fff640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f2e5c0a32c0 0x7f2e5c0a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2e700fe710 tx=0x7f2e5800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:36.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.230+0000 7f2e6dffb640 1 -- 172.21.15.5:0/3533408631 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2e600bcd00 con 0x7f2e70100e20 2024-09-19T06:01:36.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:35 smithi157 ceph-mon[35346]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T06:01:36.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:35 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/616190631' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T06:01:36.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:35 smithi157 ceph-mon[35346]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:01:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.445+0000 7f2e764a1640 1 -- 172.21.15.5:0/3533408631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f2e7010a0a0 con 0x7f2e70100e20 2024-09-19T06:01:36.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.807+0000 7f2e6dffb640 1 -- 172.21.15.5:0/3533408631 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f2e600bc600 con 0x7f2e70100e20 2024-09-19T06:01:36.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.810+0000 7f2e764a1640 1 -- 172.21.15.5:0/3533408631 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f2e5c0a32c0 msgr2=0x7f2e5c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:36.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.810+0000 7f2e764a1640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f2e5c0a32c0 0x7f2e5c0a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2e700fe710 tx=0x7f2e5800b390 comp rx=0 tx=0).stop 2024-09-19T06:01:36.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.810+0000 7f2e764a1640 1 -- 172.21.15.5:0/3533408631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e70100e20 msgr2=0x7f2e700fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:36.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.810+0000 7f2e764a1640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e70100e20 0x7f2e700fcc80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2e60006010 tx=0x7f2e600042f0 comp rx=0 tx=0).stop 2024-09-19T06:01:36.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.810+0000 7f2e764a1640 1 -- 172.21.15.5:0/3533408631 shutdown_connections 2024-09-19T06:01:36.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.810+0000 7f2e764a1640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f2e5c0a32c0 0x7f2e5c0a5780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:36.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.810+0000 7f2e764a1640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e70102ef0 0x7f2e700fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:36.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.810+0000 7f2e764a1640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e70102570 0x7f2e700fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:36.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.810+0000 7f2e764a1640 1 --2- 172.21.15.5:0/3533408631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e70100e20 0x7f2e700fcc80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:36.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.810+0000 7f2e764a1640 1 -- 172.21.15.5:0/3533408631 >> 172.21.15.5:0/3533408631 conn(0x7f2e700fc7d0 msgr2=0x7f2e7006a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:36.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.811+0000 7f2e764a1640 1 -- 172.21.15.5:0/3533408631 shutdown_connections 2024-09-19T06:01:36.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:36.811+0000 7f2e764a1640 1 -- 172.21.15.5:0/3533408631 wait complete. 2024-09-19T06:01:37.119 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:36 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3533408631' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T06:01:37.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:36 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3533408631' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T06:01:37.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:36 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3533408631' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T06:01:37.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T06:01:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:38.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:37 smithi005 ceph-mon[30955]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T06:01:38.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:37 smithi005 ceph-mon[30955]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T06:01:38.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:37 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3533408631' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T06:01:38.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:37 smithi005 ceph-mon[30955]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T06:01:38.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:37 smithi005 ceph-mon[30955]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:01:38.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:37 smithi005 ceph-mon[30955]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T06:01:38.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:37 smithi005 ceph-mon[30955]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T06:01:38.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:37 smithi049 ceph-mon[34640]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T06:01:38.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:37 smithi049 ceph-mon[34640]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T06:01:38.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:37 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3533408631' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T06:01:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:37 smithi049 ceph-mon[34640]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T06:01:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:37 smithi049 ceph-mon[34640]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:01:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:37 smithi049 ceph-mon[34640]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T06:01:38.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:37 smithi049 ceph-mon[34640]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T06:01:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.164+0000 7fb88ffcf640 1 -- 172.21.15.5:0/170999752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb888100480 msgr2=0x7fb888100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.164+0000 7fb88ffcf640 1 --2- 172.21.15.5:0/170999752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb888100480 0x7fb888100860 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb88400aa10 tx=0x7fb88402b720 comp rx=0 tx=0).stop 2024-09-19T06:01:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.164+0000 7fb88ffcf640 1 -- 172.21.15.5:0/170999752 shutdown_connections 2024-09-19T06:01:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.164+0000 7fb88ffcf640 1 --2- 172.21.15.5:0/170999752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8881055b0 0x7fb88810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.164+0000 7fb88ffcf640 1 --2- 172.21.15.5:0/170999752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb888100e30 0x7fb888104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.164+0000 7fb88ffcf640 1 --2- 172.21.15.5:0/170999752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb888100480 0x7fb888100860 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.164+0000 7fb88ffcf640 1 -- 172.21.15.5:0/170999752 >> 172.21.15.5:0/170999752 conn(0x7fb888077210 msgr2=0x7fb8880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.165+0000 7fb88ffcf640 1 -- 172.21.15.5:0/170999752 shutdown_connections 2024-09-19T06:01:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.165+0000 7fb88ffcf640 1 -- 172.21.15.5:0/170999752 wait complete. 2024-09-19T06:01:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.165+0000 7fb88ffcf640 1 Processor -- start 2024-09-19T06:01:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.165+0000 7fb88ffcf640 1 -- start start 2024-09-19T06:01:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.165+0000 7fb88ffcf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb888100480 0x7fb888198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.165+0000 7fb88ffcf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb888100e30 0x7fb8881988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.165+0000 7fb88ffcf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8881055b0 0x7fb88819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.165+0000 7fb88ffcf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb88810fc40 con 0x7fb8881055b0 2024-09-19T06:01:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.165+0000 7fb88ffcf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb88810fac0 con 0x7fb888100e30 2024-09-19T06:01:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.165+0000 7fb88ffcf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb88810fdc0 con 0x7fb888100480 2024-09-19T06:01:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88e56c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8881055b0 0x7fb88819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88e56c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8881055b0 0x7fb88819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52366/0 (socket says 172.21.15.5:52366) 2024-09-19T06:01:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88e56c640 1 -- 172.21.15.5:0/1894857085 learned_addr learned my addr 172.21.15.5:0/1894857085 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88dd6b640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb888100480 0x7fb888198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88d56a640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb888100e30 0x7fb8881988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88e56c640 1 -- 172.21.15.5:0/1894857085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb888100480 msgr2=0x7fb888198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88e56c640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb888100480 0x7fb888198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88e56c640 1 -- 172.21.15.5:0/1894857085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb888100e30 msgr2=0x7fb8881988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88e56c640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb888100e30 0x7fb8881988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88e56c640 1 -- 172.21.15.5:0/1894857085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88819d320 con 0x7fb8881055b0 2024-09-19T06:01:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88e56c640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8881055b0 0x7fb88819cc40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb88000eaa0 tx=0x7fb88000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb87effd640 1 -- 172.21.15.5:0/1894857085 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88000cdb0 con 0x7fb8881055b0 2024-09-19T06:01:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88ffcf640 1 -- 172.21.15.5:0/1894857085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88819d610 con 0x7fb8881055b0 2024-09-19T06:01:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb87effd640 1 -- 172.21.15.5:0/1894857085 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb880005020 con 0x7fb8881055b0 2024-09-19T06:01:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb87effd640 1 -- 172.21.15.5:0/1894857085 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800116d0 con 0x7fb8881055b0 2024-09-19T06:01:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.166+0000 7fb88ffcf640 1 -- 172.21.15.5:0/1894857085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8881a4ef0 con 0x7fb8881055b0 2024-09-19T06:01:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.167+0000 7fb88ffcf640 1 -- 172.21.15.5:0/1894857085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb888100860 con 0x7fb8881055b0 2024-09-19T06:01:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.169+0000 7fb87effd640 1 -- 172.21.15.5:0/1894857085 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fb8800051d0 con 0x7fb8881055b0 2024-09-19T06:01:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.169+0000 7fb87effd640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb8740a3240 0x7fb8740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.169+0000 7fb88dd6b640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb8740a3240 0x7fb8740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.169+0000 7fb87effd640 1 -- 172.21.15.5:0/1894857085 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb8800cfd30 con 0x7fb8881055b0 2024-09-19T06:01:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.170+0000 7fb88dd6b640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb8740a3240 0x7fb8740a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb884030550 tx=0x7fb88400a700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.173+0000 7fb87effd640 1 -- 172.21.15.5:0/1894857085 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb880016030 con 0x7fb8881055b0 2024-09-19T06:01:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:37 smithi157 ceph-mon[35346]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T06:01:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:37 smithi157 ceph-mon[35346]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T06:01:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:37 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3533408631' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T06:01:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:37 smithi157 ceph-mon[35346]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T06:01:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:37 smithi157 ceph-mon[35346]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:01:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:37 smithi157 ceph-mon[35346]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T06:01:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:37 smithi157 ceph-mon[35346]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T06:01:38.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.385+0000 7fb88ffcf640 1 -- 172.21.15.5:0/1894857085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fb8881087f0 con 0x7fb8881055b0 2024-09-19T06:01:38.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.820+0000 7fb87effd640 1 -- 172.21.15.5:0/1894857085 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fb88009cba0 con 0x7fb8881055b0 2024-09-19T06:01:38.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.822+0000 7fb88ffcf640 1 -- 172.21.15.5:0/1894857085 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb8740a3240 msgr2=0x7fb8740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.822+0000 7fb88ffcf640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb8740a3240 0x7fb8740a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb884030550 tx=0x7fb88400a700 comp rx=0 tx=0).stop 2024-09-19T06:01:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.822+0000 7fb88ffcf640 1 -- 172.21.15.5:0/1894857085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8881055b0 msgr2=0x7fb88819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.822+0000 7fb88ffcf640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8881055b0 0x7fb88819cc40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb88000eaa0 tx=0x7fb88000ef60 comp rx=0 tx=0).stop 2024-09-19T06:01:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.822+0000 7fb88ffcf640 1 -- 172.21.15.5:0/1894857085 shutdown_connections 2024-09-19T06:01:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.822+0000 7fb88ffcf640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb8740a3240 0x7fb8740a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.822+0000 7fb88ffcf640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8881055b0 0x7fb88819cc40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.822+0000 7fb88ffcf640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb888100e30 0x7fb8881988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.822+0000 7fb88ffcf640 1 --2- 172.21.15.5:0/1894857085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb888100480 0x7fb888198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.822+0000 7fb88ffcf640 1 -- 172.21.15.5:0/1894857085 >> 172.21.15.5:0/1894857085 conn(0x7fb888077210 msgr2=0x7fb8880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.822+0000 7fb88ffcf640 1 -- 172.21.15.5:0/1894857085 shutdown_connections 2024-09-19T06:01:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:38.823+0000 7fb88ffcf640 1 -- 172.21.15.5:0/1894857085 wait complete. 2024-09-19T06:01:39.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] up:active 2024-09-19T06:01:39.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T06:01:39.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T06:01:39.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T06:01:39.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T06:01:39.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1894857085' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T06:01:39.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-19T06:01:39.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-09-19T06:01:39.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-19T06:01:39.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-19T06:01:39.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-19T06:01:39.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6, 3, 8]}]: dispatch 2024-09-19T06:01:39.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-19T06:01:39.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-09-19T06:01:39.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:38 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-09-19T06:01:39.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] up:active 2024-09-19T06:01:39.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T06:01:39.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T06:01:39.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T06:01:39.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T06:01:39.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1894857085' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T06:01:39.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-19T06:01:39.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-09-19T06:01:39.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-19T06:01:39.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-19T06:01:39.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-19T06:01:39.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6, 3, 8]}]: dispatch 2024-09-19T06:01:39.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-19T06:01:39.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-09-19T06:01:39.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:38 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-09-19T06:01:39.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] up:active 2024-09-19T06:01:39.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1894857085' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6, 3, 8]}]: dispatch 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-09-19T06:01:39.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:38 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-09-19T06:01:39.523 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T06:01:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:39.896 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:01:39.640+0000 7f9e1bdf3640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T06:01:39.896 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:01:39.640+0000 7f9e1bdf3640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T06:01:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: pgmap v281: 97 pgs: 97 active+clean; 580 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-09-19T06:01:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1894857085' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T06:01:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] up:active 2024-09-19T06:01:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T06:01:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T06:01:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-19T06:01:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-09-19T06:01:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-19T06:01:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-19T06:01:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-19T06:01:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6, 3, 8]}]': finished 2024-09-19T06:01:39.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-19T06:01:39.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-09-19T06:01:39.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-09-19T06:01:39.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T06:01:39.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:39 smithi049 ceph-mon[34640]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-19T06:01:40.080 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: pgmap v281: 97 pgs: 97 active+clean; 580 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-09-19T06:01:40.080 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1894857085' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T06:01:40.080 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] up:active 2024-09-19T06:01:40.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T06:01:40.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T06:01:40.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-19T06:01:40.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-09-19T06:01:40.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-19T06:01:40.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-19T06:01:40.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-19T06:01:40.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6, 3, 8]}]': finished 2024-09-19T06:01:40.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-19T06:01:40.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-09-19T06:01:40.083 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-09-19T06:01:40.083 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T06:01:40.083 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:39 smithi005 ceph-mon[30955]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-19T06:01:40.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.279+0000 7f90283fd640 1 -- 172.21.15.5:0/2650300748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9020100e30 msgr2=0x7f9020104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:40.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.279+0000 7f90283fd640 1 --2- 172.21.15.5:0/2650300748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9020100e30 0x7f9020104e80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9010009900 tx=0x7f901002a410 comp rx=0 tx=0).stop 2024-09-19T06:01:40.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.279+0000 7f90283fd640 1 -- 172.21.15.5:0/2650300748 shutdown_connections 2024-09-19T06:01:40.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.279+0000 7f90283fd640 1 --2- 172.21.15.5:0/2650300748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90201055b0 0x7f902010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:40.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.279+0000 7f90283fd640 1 --2- 172.21.15.5:0/2650300748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9020100e30 0x7f9020104e80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:40.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.279+0000 7f90283fd640 1 --2- 172.21.15.5:0/2650300748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9020100480 0x7f9020100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:40.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.279+0000 7f90283fd640 1 -- 172.21.15.5:0/2650300748 >> 172.21.15.5:0/2650300748 conn(0x7f9020077210 msgr2=0x7f90200fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:40.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.279+0000 7f90283fd640 1 -- 172.21.15.5:0/2650300748 shutdown_connections 2024-09-19T06:01:40.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.279+0000 7f90283fd640 1 -- 172.21.15.5:0/2650300748 wait complete. 2024-09-19T06:01:40.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f90283fd640 1 Processor -- start 2024-09-19T06:01:40.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f90283fd640 1 -- start start 2024-09-19T06:01:40.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f90283fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9020100480 0x7f9020198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:40.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f90283fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9020100e30 0x7f9020198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:40.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f90283fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90201055b0 0x7f902019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:40.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f90283fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9020111ee0 con 0x7f90201055b0 2024-09-19T06:01:40.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f90283fd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9020111d60 con 0x7f9020100e30 2024-09-19T06:01:40.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f90283fd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9020112060 con 0x7f9020100480 2024-09-19T06:01:40.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f902699a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90201055b0 0x7f902019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f902699a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90201055b0 0x7f902019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52382/0 (socket says 172.21.15.5:52382) 2024-09-19T06:01:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f902699a640 1 -- 172.21.15.5:0/1064493257 learned_addr learned my addr 172.21.15.5:0/1064493257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f9026199640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9020100480 0x7f9020198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.280+0000 7f9025998640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9020100e30 0x7f9020198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.281+0000 7f902699a640 1 -- 172.21.15.5:0/1064493257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9020100480 msgr2=0x7f9020198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.281+0000 7f902699a640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9020100480 0x7f9020198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.281+0000 7f902699a640 1 -- 172.21.15.5:0/1064493257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9020100e30 msgr2=0x7f9020198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.281+0000 7f902699a640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9020100e30 0x7f9020198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.281+0000 7f902699a640 1 -- 172.21.15.5:0/1064493257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902019d4f0 con 0x7f90201055b0 2024-09-19T06:01:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.281+0000 7f902699a640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90201055b0 0x7f902019ce10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9018009fd0 tx=0x7f901800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.281+0000 7f90177fe640 1 -- 172.21.15.5:0/1064493257 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018011830 con 0x7f90201055b0 2024-09-19T06:01:40.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.281+0000 7f90177fe640 1 -- 172.21.15.5:0/1064493257 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9018016070 con 0x7f90201055b0 2024-09-19T06:01:40.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.281+0000 7f90177fe640 1 -- 172.21.15.5:0/1064493257 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018010de0 con 0x7f90201055b0 2024-09-19T06:01:40.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.281+0000 7f90283fd640 1 -- 172.21.15.5:0/1064493257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902019d7e0 con 0x7f90201055b0 2024-09-19T06:01:40.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.281+0000 7f90283fd640 1 -- 172.21.15.5:0/1064493257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90201a50c0 con 0x7f90201055b0 2024-09-19T06:01:40.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.282+0000 7f90283fd640 1 -- 172.21.15.5:0/1064493257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90201008e0 con 0x7f90201055b0 2024-09-19T06:01:40.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.283+0000 7f90177fe640 1 -- 172.21.15.5:0/1064493257 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f90180119d0 con 0x7f90201055b0 2024-09-19T06:01:40.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.284+0000 7f90177fe640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f900c0a3240 0x7f900c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:40.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.284+0000 7f9026199640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f900c0a3240 0x7f900c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:40.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.284+0000 7f90177fe640 1 -- 172.21.15.5:0/1064493257 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f90180cf090 con 0x7f90201055b0 2024-09-19T06:01:40.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.285+0000 7f9026199640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f900c0a3240 0x7f900c0a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f901c00a400 tx=0x7f901c00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:40.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.289+0000 7f90177fe640 1 -- 172.21.15.5:0/1064493257 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f901809bd40 con 0x7f90201055b0 2024-09-19T06:01:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: pgmap v281: 97 pgs: 97 active+clean; 580 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-09-19T06:01:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1894857085' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T06:01:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] up:active 2024-09-19T06:01:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T06:01:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T06:01:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-19T06:01:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-09-19T06:01:40.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-19T06:01:40.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-19T06:01:40.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-19T06:01:40.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6, 3, 8]}]': finished 2024-09-19T06:01:40.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-19T06:01:40.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-09-19T06:01:40.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-09-19T06:01:40.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T06:01:40.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:39 smithi157 ceph-mon[35346]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-19T06:01:40.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.501+0000 7f90283fd640 1 -- 172.21.15.5:0/1064493257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f9020108790 con 0x7f90201055b0 2024-09-19T06:01:40.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.831+0000 7f90177fe640 1 -- 172.21.15.5:0/1064493257 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f9018097030 con 0x7f90201055b0 2024-09-19T06:01:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.834+0000 7f90283fd640 1 -- 172.21.15.5:0/1064493257 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f900c0a3240 msgr2=0x7f900c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.834+0000 7f90283fd640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f900c0a3240 0x7f900c0a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f901c00a400 tx=0x7f901c00d040 comp rx=0 tx=0).stop 2024-09-19T06:01:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.834+0000 7f90283fd640 1 -- 172.21.15.5:0/1064493257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90201055b0 msgr2=0x7f902019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.834+0000 7f90283fd640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90201055b0 0x7f902019ce10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9018009fd0 tx=0x7f901800b8e0 comp rx=0 tx=0).stop 2024-09-19T06:01:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.834+0000 7f90283fd640 1 -- 172.21.15.5:0/1064493257 shutdown_connections 2024-09-19T06:01:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.834+0000 7f90283fd640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f900c0a3240 0x7f900c0a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.834+0000 7f90283fd640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90201055b0 0x7f902019ce10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.834+0000 7f90283fd640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9020100e30 0x7f9020198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.834+0000 7f90283fd640 1 --2- 172.21.15.5:0/1064493257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9020100480 0x7f9020198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.834+0000 7f90283fd640 1 -- 172.21.15.5:0/1064493257 >> 172.21.15.5:0/1064493257 conn(0x7f9020077210 msgr2=0x7f90201039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.834+0000 7f90283fd640 1 -- 172.21.15.5:0/1064493257 shutdown_connections 2024-09-19T06:01:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:40.834+0000 7f90283fd640 1 -- 172.21.15.5:0/1064493257 wait complete. 2024-09-19T06:01:41.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:40 smithi005 ceph-mon[30955]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T06:01:41.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:40 smithi005 ceph-mon[30955]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-19T06:01:41.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:40 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1064493257' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T06:01:41.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:40 smithi049 ceph-mon[34640]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T06:01:41.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:40 smithi049 ceph-mon[34640]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-19T06:01:41.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:40 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1064493257' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T06:01:41.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:40 smithi157 ceph-mon[35346]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T06:01:41.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:40 smithi157 ceph-mon[35346]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-19T06:01:41.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:40 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1064493257' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T06:01:41.454 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-19T06:01:41.454 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T06:01:41.718 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:42.088 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:41 smithi005 ceph-mon[30955]: pgmap v284: 97 pgs: 5 peering, 1 active+recovering, 91 active+clean; 582 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T06:01:42.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:41 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1064493257' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T06:01:42.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:41 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] up:active 2024-09-19T06:01:42.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:41 smithi005 ceph-mon[30955]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T06:01:42.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:41 smithi005 ceph-mon[30955]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T06:01:42.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:41 smithi005 ceph-mon[30955]: Cluster is now healthy 2024-09-19T06:01:42.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:41 smithi005 ceph-mon[30955]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-19T06:01:42.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:41 smithi005 ceph-mon[30955]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T06:01:42.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:41 smithi005 ceph-mon[30955]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-19T06:01:42.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:41 smithi049 ceph-mon[34640]: pgmap v284: 97 pgs: 5 peering, 1 active+recovering, 91 active+clean; 582 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T06:01:42.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:41 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1064493257' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T06:01:42.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:41 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] up:active 2024-09-19T06:01:42.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:41 smithi049 ceph-mon[34640]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T06:01:42.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:41 smithi049 ceph-mon[34640]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T06:01:42.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:41 smithi049 ceph-mon[34640]: Cluster is now healthy 2024-09-19T06:01:42.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:41 smithi049 ceph-mon[34640]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-19T06:01:42.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:41 smithi049 ceph-mon[34640]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T06:01:42.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:41 smithi049 ceph-mon[34640]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-19T06:01:42.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.171+0000 7f3208b27640 1 -- 172.21.15.5:0/294119830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3204106f10 msgr2=0x7f320410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:42.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.171+0000 7f3208b27640 1 --2- 172.21.15.5:0/294119830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3204106f10 0x7f320410ea90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009ca0 tx=0x7f31fc02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:01:42.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.171+0000 7f3208b27640 1 -- 172.21.15.5:0/294119830 shutdown_connections 2024-09-19T06:01:42.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.171+0000 7f3208b27640 1 --2- 172.21.15.5:0/294119830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3204106f10 0x7f320410ea90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:42.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.171+0000 7f3208b27640 1 --2- 172.21.15.5:0/294119830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3204102790 0x7f32041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:42.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.171+0000 7f3208b27640 1 --2- 172.21.15.5:0/294119830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3204101de0 0x7f32041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:42.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.171+0000 7f3208b27640 1 -- 172.21.15.5:0/294119830 >> 172.21.15.5:0/294119830 conn(0x7f32040fd8c0 msgr2=0x7f32040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:42.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.171+0000 7f3208b27640 1 -- 172.21.15.5:0/294119830 shutdown_connections 2024-09-19T06:01:42.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.171+0000 7f3208b27640 1 -- 172.21.15.5:0/294119830 wait complete. 2024-09-19T06:01:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.172+0000 7f3208b27640 1 Processor -- start 2024-09-19T06:01:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.172+0000 7f3208b27640 1 -- start start 2024-09-19T06:01:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.172+0000 7f3208b27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3204101de0 0x7f320419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.172+0000 7f3208b27640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3204102790 0x7f320419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.172+0000 7f3208b27640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3204106f10 0x7f320419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.172+0000 7f3208b27640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3204111e20 con 0x7f3204101de0 2024-09-19T06:01:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.172+0000 7f3208b27640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3204111ca0 con 0x7f3204106f10 2024-09-19T06:01:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.172+0000 7f3208b27640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3204111fa0 con 0x7f3204102790 2024-09-19T06:01:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.172+0000 7f320259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3204101de0 0x7f320419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.172+0000 7f320259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3204101de0 0x7f320419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52392/0 (socket says 172.21.15.5:52392) 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.172+0000 7f320259c640 1 -- 172.21.15.5:0/2481402502 learned_addr learned my addr 172.21.15.5:0/2481402502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.172+0000 7f3201d9b640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3204102790 0x7f320419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f3202d9d640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3204106f10 0x7f320419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f320259c640 1 -- 172.21.15.5:0/2481402502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3204102790 msgr2=0x7f320419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f320259c640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3204102790 0x7f320419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f320259c640 1 -- 172.21.15.5:0/2481402502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3204106f10 msgr2=0x7f320419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f320259c640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3204106f10 0x7f320419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f320259c640 1 -- 172.21.15.5:0/2481402502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320419f4c0 con 0x7f3204101de0 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f3202d9d640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3204106f10 0x7f320419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f320259c640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3204101de0 0x7f320419a510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f31f400d950 tx=0x7f31f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f31fb7fe640 1 -- 172.21.15.5:0/2481402502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4017070 con 0x7f3204101de0 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f31fb7fe640 1 -- 172.21.15.5:0/2481402502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31f4009c30 con 0x7f3204101de0 2024-09-19T06:01:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f3208b27640 1 -- 172.21.15.5:0/2481402502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320419f7b0 con 0x7f3204101de0 2024-09-19T06:01:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f31fb7fe640 1 -- 172.21.15.5:0/2481402502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f400f620 con 0x7f3204101de0 2024-09-19T06:01:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.173+0000 7f3208b27640 1 -- 172.21.15.5:0/2481402502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32041a7090 con 0x7f3204101de0 2024-09-19T06:01:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.174+0000 7f3208b27640 1 -- 172.21.15.5:0/2481402502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32041021c0 con 0x7f3204101de0 2024-09-19T06:01:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.175+0000 7f31fb7fe640 1 -- 172.21.15.5:0/2481402502 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f31f40109a0 con 0x7f3204101de0 2024-09-19T06:01:42.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.176+0000 7f31fb7fe640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f31f00a32c0 0x7f31f00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:42.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.176+0000 7f3201d9b640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f31f00a32c0 0x7f31f00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:42.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.176+0000 7f31fb7fe640 1 -- 172.21.15.5:0/2481402502 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f31f40cf540 con 0x7f3204101de0 2024-09-19T06:01:42.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.177+0000 7f3201d9b640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f31f00a32c0 0x7f31f00a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f31ec005fd0 tx=0x7f31ec00b510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:42.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.179+0000 7f31fb7fe640 1 -- 172.21.15.5:0/2481402502 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f31f4009de0 con 0x7f3204101de0 2024-09-19T06:01:42.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:41 smithi157 ceph-mon[35346]: pgmap v284: 97 pgs: 5 peering, 1 active+recovering, 91 active+clean; 582 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T06:01:42.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:41 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1064493257' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T06:01:42.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:41 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] up:active 2024-09-19T06:01:42.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:41 smithi157 ceph-mon[35346]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T06:01:42.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:41 smithi157 ceph-mon[35346]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T06:01:42.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:41 smithi157 ceph-mon[35346]: Cluster is now healthy 2024-09-19T06:01:42.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:41 smithi157 ceph-mon[35346]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-19T06:01:42.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:41 smithi157 ceph-mon[35346]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T06:01:42.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:41 smithi157 ceph-mon[35346]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-19T06:01:42.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.379+0000 7f3208b27640 1 -- 172.21.15.5:0/2481402502 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3204075430 con 0x7f31f00a32c0 2024-09-19T06:01:42.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.413+0000 7f31fb7fe640 1 -- 172.21.15.5:0/2481402502 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3204075430 con 0x7f31f00a32c0 2024-09-19T06:01:42.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.416+0000 7f3208b27640 1 -- 172.21.15.5:0/2481402502 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f31f00a32c0 msgr2=0x7f31f00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.416+0000 7f3208b27640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f31f00a32c0 0x7f31f00a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f31ec005fd0 tx=0x7f31ec00b510 comp rx=0 tx=0).stop 2024-09-19T06:01:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.416+0000 7f3208b27640 1 -- 172.21.15.5:0/2481402502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3204101de0 msgr2=0x7f320419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.416+0000 7f3208b27640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3204101de0 0x7f320419a510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f31f400d950 tx=0x7f31f400de10 comp rx=0 tx=0).stop 2024-09-19T06:01:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.416+0000 7f3208b27640 1 -- 172.21.15.5:0/2481402502 shutdown_connections 2024-09-19T06:01:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.416+0000 7f3208b27640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f31f00a32c0 0x7f31f00a5780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.416+0000 7f3208b27640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3204106f10 0x7f320419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.416+0000 7f3208b27640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3204102790 0x7f320419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.416+0000 7f3208b27640 1 --2- 172.21.15.5:0/2481402502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3204101de0 0x7f320419a510 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.416+0000 7f3208b27640 1 -- 172.21.15.5:0/2481402502 >> 172.21.15.5:0/2481402502 conn(0x7f32040fd8c0 msgr2=0x7f32040febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.416+0000 7f3208b27640 1 -- 172.21.15.5:0/2481402502 shutdown_connections 2024-09-19T06:01:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:42.416+0000 7f3208b27640 1 -- 172.21.15.5:0/2481402502 wait complete. 2024-09-19T06:01:42.841 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:01:42 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:01:42.384+0000 7f22272fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:01:42.841 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:01:42 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:01:42.384+0000 7f22272fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:01:42.841 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:01:42 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:01:42.384+0000 7f22272fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:01:42.841 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:01:42 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:01:42.384+0000 7f22272fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:01:42.841 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:01:42 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:01:42.384+0000 7f22272fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:01:43.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:42 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] up:active 2024-09-19T06:01:43.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:42 smithi005 ceph-mon[30955]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T06:01:43.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:42 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:01:43.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:42 smithi005 ceph-mon[30955]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:43.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T06:01:43.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:42 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] up:active 2024-09-19T06:01:43.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:42 smithi049 ceph-mon[34640]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T06:01:43.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:42 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:01:43.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:42 smithi049 ceph-mon[34640]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:43.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:42 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] up:active 2024-09-19T06:01:43.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:42 smithi157 ceph-mon[35346]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T06:01:43.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:42 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:01:43.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:42 smithi157 ceph-mon[35346]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:43.388 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.824+0000 7f64c8623640 1 -- 172.21.15.5:0/2969964545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64c010c170 msgr2=0x7f64c010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.824+0000 7f64c8623640 1 --2- 172.21.15.5:0/2969964545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64c010c170 0x7f64c010e610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009ca0 tx=0x7f64b802a950 comp rx=0 tx=0).stop 2024-09-19T06:01:43.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.824+0000 7f64c8623640 1 -- 172.21.15.5:0/2969964545 shutdown_connections 2024-09-19T06:01:43.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.824+0000 7f64c8623640 1 --2- 172.21.15.5:0/2969964545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64c010c170 0x7f64c010e610 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:43.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.824+0000 7f64c8623640 1 --2- 172.21.15.5:0/2969964545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64c0103660 0x7f64c010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:43.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.824+0000 7f64c8623640 1 --2- 172.21.15.5:0/2969964545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64c0102d40 0x7f64c0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:43.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.824+0000 7f64c8623640 1 -- 172.21.15.5:0/2969964545 >> 172.21.15.5:0/2969964545 conn(0x7f64c00fc7a0 msgr2=0x7f64c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:43.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.824+0000 7f64c8623640 1 -- 172.21.15.5:0/2969964545 shutdown_connections 2024-09-19T06:01:43.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.824+0000 7f64c8623640 1 -- 172.21.15.5:0/2969964545 wait complete. 2024-09-19T06:01:43.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.825+0000 7f64c8623640 1 Processor -- start 2024-09-19T06:01:43.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.825+0000 7f64c8623640 1 -- start start 2024-09-19T06:01:43.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.825+0000 7f64c8623640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64c0102d40 0x7f64c01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:43.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.825+0000 7f64c8623640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64c0103660 0x7f64c0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:43.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.825+0000 7f64c8623640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64c010c170 0x7f64c0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.825+0000 7f64c8623640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64c0111e00 con 0x7f64c0103660 2024-09-19T06:01:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.825+0000 7f64c8623640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f64c0111c80 con 0x7f64c010c170 2024-09-19T06:01:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.825+0000 7f64c8623640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f64c0111f80 con 0x7f64c0102d40 2024-09-19T06:01:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.825+0000 7f64c5bbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64c0103660 0x7f64c0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.825+0000 7f64c5bbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64c0103660 0x7f64c0100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52426/0 (socket says 172.21.15.5:52426) 2024-09-19T06:01:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.825+0000 7f64c5bbe640 1 -- 172.21.15.5:0/3577137159 learned_addr learned my addr 172.21.15.5:0/3577137159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.825+0000 7f64c63bf640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64c0102d40 0x7f64c01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.826+0000 7f64c6bc0640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64c010c170 0x7f64c0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.826+0000 7f64c5bbe640 1 -- 172.21.15.5:0/3577137159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64c0102d40 msgr2=0x7f64c01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.826+0000 7f64c5bbe640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64c0102d40 0x7f64c01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.826+0000 7f64c5bbe640 1 -- 172.21.15.5:0/3577137159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64c010c170 msgr2=0x7f64c0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.826+0000 7f64c5bbe640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64c010c170 0x7f64c0101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.826+0000 7f64c5bbe640 1 -- 172.21.15.5:0/3577137159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c01a67d0 con 0x7f64c0103660 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.826+0000 7f64c5bbe640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64c0103660 0x7f64c0100ee0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f64b000d750 tx=0x7f64b000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.826+0000 7f64b77fe640 1 -- 172.21.15.5:0/3577137159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0017070 con 0x7f64c0103660 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.826+0000 7f64c8623640 1 -- 172.21.15.5:0/3577137159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c01a6ac0 con 0x7f64c0103660 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.826+0000 7f64b77fe640 1 -- 172.21.15.5:0/3577137159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64b0009820 con 0x7f64c0103660 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.826+0000 7f64b77fe640 1 -- 172.21.15.5:0/3577137159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0004b20 con 0x7f64c0103660 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.826+0000 7f64c8623640 1 -- 172.21.15.5:0/3577137159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c01a7000 con 0x7f64c0103660 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.827+0000 7f64c8623640 1 -- 172.21.15.5:0/3577137159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c0103120 con 0x7f64c0103660 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.828+0000 7f64b77fe640 1 -- 172.21.15.5:0/3577137159 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f64b0004cc0 con 0x7f64c0103660 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.829+0000 7f64b77fe640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f64ac0a3240 0x7f64ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.829+0000 7f64c63bf640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f64ac0a3240 0x7f64ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.829+0000 7f64b77fe640 1 -- 172.21.15.5:0/3577137159 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f64b00cf310 con 0x7f64c0103660 2024-09-19T06:01:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.830+0000 7f64c63bf640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f64ac0a3240 0x7f64ac0a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00abe0 tx=0x7f64bc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.833+0000 7f64b77fe640 1 -- 172.21.15.5:0/3577137159 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f64b009c020 con 0x7f64c0103660 2024-09-19T06:01:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:43.852+0000 7f64b77fe640 1 -- 172.21.15.5:0/3577137159 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f64b0097f90 con 0x7f64c0103660 2024-09-19T06:01:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.041+0000 7f64c8623640 1 -- 172.21.15.5:0/3577137159 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f64c0107250 con 0x7f64ac0a3240 2024-09-19T06:01:44.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.066+0000 7f64b77fe640 1 -- 172.21.15.5:0/3577137159 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f64bc00fc20 con 0x7f64ac0a3240 2024-09-19T06:01:44.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.068+0000 7f64c8623640 1 -- 172.21.15.5:0/3577137159 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f64ac0a3240 msgr2=0x7f64ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:44.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.068+0000 7f64c8623640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f64ac0a3240 0x7f64ac0a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00abe0 tx=0x7f64bc00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:01:44.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.068+0000 7f64c8623640 1 -- 172.21.15.5:0/3577137159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64c0103660 msgr2=0x7f64c0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:44.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.068+0000 7f64c8623640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64c0103660 0x7f64c0100ee0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f64b000d750 tx=0x7f64b000db50 comp rx=0 tx=0).stop 2024-09-19T06:01:44.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.068+0000 7f64c8623640 1 -- 172.21.15.5:0/3577137159 shutdown_connections 2024-09-19T06:01:44.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.068+0000 7f64c8623640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f64ac0a3240 0x7f64ac0a5700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.068+0000 7f64c8623640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64c010c170 0x7f64c0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.068+0000 7f64c8623640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64c0103660 0x7f64c0100ee0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.068+0000 7f64c8623640 1 --2- 172.21.15.5:0/3577137159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64c0102d40 0x7f64c01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.068+0000 7f64c8623640 1 -- 172.21.15.5:0/3577137159 >> 172.21.15.5:0/3577137159 conn(0x7f64c00fc7a0 msgr2=0x7f64c00fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.068+0000 7f64c8623640 1 -- 172.21.15.5:0/3577137159 shutdown_connections 2024-09-19T06:01:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:44.069+0000 7f64c8623640 1 -- 172.21.15.5:0/3577137159 wait complete. 2024-09-19T06:01:44.079 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:43 smithi005 ceph-mon[30955]: pgmap v285: 97 pgs: 5 peering, 1 active+recovering, 91 active+clean; 582 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T06:01:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:43 smithi049 ceph-mon[34640]: pgmap v285: 97 pgs: 5 peering, 1 active+recovering, 91 active+clean; 582 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T06:01:44.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:43 smithi157 ceph-mon[35346]: pgmap v285: 97 pgs: 5 peering, 1 active+recovering, 91 active+clean; 582 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T06:01:44.726 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T06:01:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:44 smithi049 ceph-mon[34640]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T06:01:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:44 smithi049 ceph-mon[34640]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:44.897 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:44 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:01:44.640+0000 7f9e1bdf3640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-19T06:01:44.897 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:01:44 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:01:44.640+0000 7f9e1bdf3640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-19T06:01:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:45.112 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:44 smithi005 ceph-mon[30955]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T06:01:45.113 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:44 smithi005 ceph-mon[30955]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:45.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:44 smithi157 ceph-mon[35346]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T06:01:45.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:44 smithi157 ceph-mon[35346]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.473+0000 7fb951ba7640 1 -- 172.21.15.5:0/976745857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c106a80 msgr2=0x7fb94c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.473+0000 7fb951ba7640 1 --2- 172.21.15.5:0/976745857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c106a80 0x7fb94c106e60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009930 tx=0x7fb93c02a420 comp rx=0 tx=0).stop 2024-09-19T06:01:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.473+0000 7fb951ba7640 1 -- 172.21.15.5:0/976745857 shutdown_connections 2024-09-19T06:01:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.473+0000 7fb951ba7640 1 --2- 172.21.15.5:0/976745857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb94c101420 0x7fb94c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.473+0000 7fb951ba7640 1 --2- 172.21.15.5:0/976745857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb94c100a80 0x7fb94c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.473+0000 7fb951ba7640 1 --2- 172.21.15.5:0/976745857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c106a80 0x7fb94c106e60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.473+0000 7fb951ba7640 1 -- 172.21.15.5:0/976745857 >> 172.21.15.5:0/976745857 conn(0x7fb94c0fc7a0 msgr2=0x7fb94c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.473+0000 7fb951ba7640 1 -- 172.21.15.5:0/976745857 shutdown_connections 2024-09-19T06:01:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.473+0000 7fb951ba7640 1 -- 172.21.15.5:0/976745857 wait complete. 2024-09-19T06:01:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.474+0000 7fb951ba7640 1 Processor -- start 2024-09-19T06:01:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.474+0000 7fb951ba7640 1 -- start start 2024-09-19T06:01:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.474+0000 7fb951ba7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb94c100a80 0x7fb94c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.474+0000 7fb951ba7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c101420 0x7fb94c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.474+0000 7fb951ba7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb94c106a80 0x7fb94c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.474+0000 7fb951ba7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c110cf0 con 0x7fb94c101420 2024-09-19T06:01:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.474+0000 7fb951ba7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c110b70 con 0x7fb94c106a80 2024-09-19T06:01:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.474+0000 7fb951ba7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c110e70 con 0x7fb94c100a80 2024-09-19T06:01:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb942ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c101420 0x7fb94c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb942ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c101420 0x7fb94c197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48184/0 (socket says 172.21.15.5:48184) 2024-09-19T06:01:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb942ffd640 1 -- 172.21.15.5:0/2631834689 learned_addr learned my addr 172.21.15.5:0/2631834689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb94b7fe640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb94c100a80 0x7fb94c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb942ffd640 1 -- 172.21.15.5:0/2631834689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb94c100a80 msgr2=0x7fb94c197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb94bfff640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb94c106a80 0x7fb94c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb942ffd640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb94c100a80 0x7fb94c197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb942ffd640 1 -- 172.21.15.5:0/2631834689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb94c106a80 msgr2=0x7fb94c19bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb942ffd640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb94c106a80 0x7fb94c19bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb942ffd640 1 -- 172.21.15.5:0/2631834689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c19c1c0 con 0x7fb94c101420 2024-09-19T06:01:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb94b7fe640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb94c100a80 0x7fb94c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:01:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb94bfff640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb94c106a80 0x7fb94c19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:01:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb942ffd640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c101420 0x7fb94c197750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb93400d950 tx=0x7fb93400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb9497fa640 1 -- 172.21.15.5:0/2631834689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934017030 con 0x7fb94c101420 2024-09-19T06:01:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb9497fa640 1 -- 172.21.15.5:0/2631834689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb934004020 con 0x7fb94c101420 2024-09-19T06:01:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb951ba7640 1 -- 172.21.15.5:0/2631834689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94c19c4b0 con 0x7fb94c101420 2024-09-19T06:01:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb9497fa640 1 -- 172.21.15.5:0/2631834689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9340047a0 con 0x7fb94c101420 2024-09-19T06:01:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.475+0000 7fb951ba7640 1 -- 172.21.15.5:0/2631834689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94c1a3d90 con 0x7fb94c101420 2024-09-19T06:01:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.476+0000 7fb951ba7640 1 -- 172.21.15.5:0/2631834689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94c100ee0 con 0x7fb94c101420 2024-09-19T06:01:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.477+0000 7fb9497fa640 1 -- 172.21.15.5:0/2631834689 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fb9340041d0 con 0x7fb94c101420 2024-09-19T06:01:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.478+0000 7fb9497fa640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb9380a3350 0x7fb9380a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.479+0000 7fb94b7fe640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb9380a3350 0x7fb9380a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.479+0000 7fb9497fa640 1 -- 172.21.15.5:0/2631834689 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb9340cf550 con 0x7fb94c101420 2024-09-19T06:01:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.479+0000 7fb94b7fe640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb9380a3350 0x7fb9380a5810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb93c000c00 tx=0x7fb93c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.483+0000 7fb9497fa640 1 -- 172.21.15.5:0/2631834689 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb93409c100 con 0x7fb94c101420 2024-09-19T06:01:45.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.685+0000 7fb951ba7640 1 -- 172.21.15.5:0/2631834689 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fb94c062630 con 0x7fb9380a3350 2024-09-19T06:01:45.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.689+0000 7fb9497fa640 1 -- 172.21.15.5:0/2631834689 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb94c062630 con 0x7fb9380a3350 2024-09-19T06:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9 2024-09-19T06:01:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.691+0000 7fb951ba7640 1 -- 172.21.15.5:0/2631834689 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb9380a3350 msgr2=0x7fb9380a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.691+0000 7fb951ba7640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb9380a3350 0x7fb9380a5810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb93c000c00 tx=0x7fb93c031400 comp rx=0 tx=0).stop 2024-09-19T06:01:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.691+0000 7fb951ba7640 1 -- 172.21.15.5:0/2631834689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c101420 msgr2=0x7fb94c197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.691+0000 7fb951ba7640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c101420 0x7fb94c197750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb93400d950 tx=0x7fb93400de10 comp rx=0 tx=0).stop 2024-09-19T06:01:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.691+0000 7fb951ba7640 1 -- 172.21.15.5:0/2631834689 shutdown_connections 2024-09-19T06:01:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.691+0000 7fb951ba7640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb9380a3350 0x7fb9380a5810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.691+0000 7fb951ba7640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb94c106a80 0x7fb94c19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.692+0000 7fb951ba7640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c101420 0x7fb94c197750 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.692+0000 7fb951ba7640 1 --2- 172.21.15.5:0/2631834689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb94c100a80 0x7fb94c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.692+0000 7fb951ba7640 1 -- 172.21.15.5:0/2631834689 >> 172.21.15.5:0/2631834689 conn(0x7fb94c0fc7a0 msgr2=0x7fb94c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.692+0000 7fb951ba7640 1 -- 172.21.15.5:0/2631834689 shutdown_connections 2024-09-19T06:01:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:45.692+0000 7fb951ba7640 1 -- 172.21.15.5:0/2631834689 wait complete. 2024-09-19T06:01:46.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:45 smithi005 ceph-mon[30955]: pgmap v286: 97 pgs: 1 active+recovering, 96 active+clean; 605 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9.0 KiB/s wr, 26 op/s; 150 B/s, 0 objects/s recovering 2024-09-19T06:01:46.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:45 smithi005 ceph-mon[30955]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:46.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:45 smithi049 ceph-mon[34640]: pgmap v286: 97 pgs: 1 active+recovering, 96 active+clean; 605 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9.0 KiB/s wr, 26 op/s; 150 B/s, 0 objects/s recovering 2024-09-19T06:01:46.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:45 smithi049 ceph-mon[34640]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:46.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:45 smithi157 ceph-mon[35346]: pgmap v286: 97 pgs: 1 active+recovering, 96 active+clean; 605 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9.0 KiB/s wr, 26 op/s; 150 B/s, 0 objects/s recovering 2024-09-19T06:01:46.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:45 smithi157 ceph-mon[35346]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:46.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T06:01:46.662 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.105+0000 7f986fc83640 1 -- 172.21.15.5:0/3535655417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868102ef0 msgr2=0x7f986810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.105+0000 7f986fc83640 1 --2- 172.21.15.5:0/3535655417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868102ef0 0x7f986810da40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9860009ca0 tx=0x7f986002a950 comp rx=0 tx=0).stop 2024-09-19T06:01:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.105+0000 7f986fc83640 1 -- 172.21.15.5:0/3535655417 shutdown_connections 2024-09-19T06:01:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.105+0000 7f986fc83640 1 --2- 172.21.15.5:0/3535655417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868102ef0 0x7f986810da40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.105+0000 7f986fc83640 1 --2- 172.21.15.5:0/3535655417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9868102570 0x7f98681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.105+0000 7f986fc83640 1 --2- 172.21.15.5:0/3535655417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868100e20 0x7f9868101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.105+0000 7f986fc83640 1 -- 172.21.15.5:0/3535655417 >> 172.21.15.5:0/3535655417 conn(0x7f98680fc7d0 msgr2=0x7f98680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.105+0000 7f986fc83640 1 -- 172.21.15.5:0/3535655417 shutdown_connections 2024-09-19T06:01:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.105+0000 7f986fc83640 1 -- 172.21.15.5:0/3535655417 wait complete. 2024-09-19T06:01:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.106+0000 7f986fc83640 1 Processor -- start 2024-09-19T06:01:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.106+0000 7f986fc83640 1 -- start start 2024-09-19T06:01:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986fc83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868100e20 0x7f9868199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986fc83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868102570 0x7f9868199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986fc83640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9868102ef0 0x7f98681a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986fc83640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9868110e30 con 0x7f9868100e20 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986fc83640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9868110cb0 con 0x7f9868102ef0 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986fc83640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9868110fb0 con 0x7f9868102570 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986da1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868100e20 0x7f9868199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986da1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868100e20 0x7f9868199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48194/0 (socket says 172.21.15.5:48194) 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986da1f640 1 -- 172.21.15.5:0/46576605 learned_addr learned my addr 172.21.15.5:0/46576605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986e220640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9868102ef0 0x7f98681a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986d21e640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868102570 0x7f9868199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986da1f640 1 -- 172.21.15.5:0/46576605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868102570 msgr2=0x7f9868199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986da1f640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868102570 0x7f9868199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986da1f640 1 -- 172.21.15.5:0/46576605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9868102ef0 msgr2=0x7f98681a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986da1f640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9868102ef0 0x7f98681a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986da1f640 1 -- 172.21.15.5:0/46576605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98681a47d0 con 0x7f9868100e20 2024-09-19T06:01:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986d21e640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868102570 0x7f9868199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:01:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986da1f640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868100e20 0x7f9868199210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f986400d770 tx=0x7f986400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f985effd640 1 -- 172.21.15.5:0/46576605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9864017070 con 0x7f9868100e20 2024-09-19T06:01:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f985effd640 1 -- 172.21.15.5:0/46576605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98640054d0 con 0x7f9868100e20 2024-09-19T06:01:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f986fc83640 1 -- 172.21.15.5:0/46576605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98681a49a0 con 0x7f9868100e20 2024-09-19T06:01:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.107+0000 7f985effd640 1 -- 172.21.15.5:0/46576605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9864004cf0 con 0x7f9868100e20 2024-09-19T06:01:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.108+0000 7f986fc83640 1 -- 172.21.15.5:0/46576605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98681a4e60 con 0x7f9868100e20 2024-09-19T06:01:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.109+0000 7f986fc83640 1 -- 172.21.15.5:0/46576605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9868101200 con 0x7f9868100e20 2024-09-19T06:01:47.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.110+0000 7f985effd640 1 -- 172.21.15.5:0/46576605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f9864004e90 con 0x7f9868100e20 2024-09-19T06:01:47.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.110+0000 7f985effd640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f98540a32d0 0x7f98540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:47.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.111+0000 7f985effd640 1 -- 172.21.15.5:0/46576605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f98640cf520 con 0x7f9868100e20 2024-09-19T06:01:47.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.111+0000 7f986d21e640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f98540a32d0 0x7f98540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:47.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.111+0000 7f986d21e640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f98540a32d0 0x7f98540a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9858005fd0 tx=0x7f98580094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:47.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.115+0000 7f985effd640 1 -- 172.21.15.5:0/46576605 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f986409c0d0 con 0x7f9868100e20 2024-09-19T06:01:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.315+0000 7f986fc83640 1 -- 172.21.15.5:0/46576605 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9868062630 con 0x7f98540a32d0 2024-09-19T06:01:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.345+0000 7f985effd640 1 -- 172.21.15.5:0/46576605 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9868062630 con 0x7f98540a32d0 2024-09-19T06:01:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.346+0000 7f986fc83640 1 -- 172.21.15.5:0/46576605 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f98540a32d0 msgr2=0x7f98540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.346+0000 7f986fc83640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f98540a32d0 0x7f98540a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9858005fd0 tx=0x7f98580094d0 comp rx=0 tx=0).stop 2024-09-19T06:01:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.346+0000 7f986fc83640 1 -- 172.21.15.5:0/46576605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868100e20 msgr2=0x7f9868199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.346+0000 7f986fc83640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868100e20 0x7f9868199210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f986400d770 tx=0x7f986400dc30 comp rx=0 tx=0).stop 2024-09-19T06:01:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.347+0000 7f986fc83640 1 -- 172.21.15.5:0/46576605 shutdown_connections 2024-09-19T06:01:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.347+0000 7f986fc83640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f98540a32d0 0x7f98540a5790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.347+0000 7f986fc83640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9868102ef0 0x7f98681a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.347+0000 7f986fc83640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868102570 0x7f9868199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.347+0000 7f986fc83640 1 --2- 172.21.15.5:0/46576605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868100e20 0x7f9868199210 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.347+0000 7f986fc83640 1 -- 172.21.15.5:0/46576605 >> 172.21.15.5:0/46576605 conn(0x7f98680fc7d0 msgr2=0x7f98680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.347+0000 7f986fc83640 1 -- 172.21.15.5:0/46576605 shutdown_connections 2024-09-19T06:01:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:47.347+0000 7f986fc83640 1 -- 172.21.15.5:0/46576605 wait complete. 2024-09-19T06:01:48.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:47 smithi005 ceph-mon[30955]: pgmap v287: 97 pgs: 97 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.2 KiB/s wr, 26 op/s; 153 B/s, 0 objects/s recovering 2024-09-19T06:01:48.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:47 smithi005 ceph-mon[30955]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:48.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T06:01:48.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:47 smithi049 ceph-mon[34640]: pgmap v287: 97 pgs: 97 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.2 KiB/s wr, 26 op/s; 153 B/s, 0 objects/s recovering 2024-09-19T06:01:48.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:47 smithi049 ceph-mon[34640]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:48.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:47 smithi157 ceph-mon[35346]: pgmap v287: 97 pgs: 97 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.2 KiB/s wr, 26 op/s; 153 B/s, 0 objects/s recovering 2024-09-19T06:01:48.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:47 smithi157 ceph-mon[35346]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:48.310 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.814+0000 7f0ce6254640 1 -- 172.21.15.5:0/2143171566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ce0102790 msgr2=0x7f0ce01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.814+0000 7f0ce6254640 1 --2- 172.21.15.5:0/2143171566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ce0102790 0x7f0ce01067e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009900 tx=0x7f0cc802a3f0 comp rx=0 tx=0).stop 2024-09-19T06:01:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.814+0000 7f0ce6254640 1 -- 172.21.15.5:0/2143171566 shutdown_connections 2024-09-19T06:01:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.814+0000 7f0ce6254640 1 --2- 172.21.15.5:0/2143171566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ce0106f10 0x7f0ce010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.814+0000 7f0ce6254640 1 --2- 172.21.15.5:0/2143171566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ce0102790 0x7f0ce01067e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.814+0000 7f0ce6254640 1 --2- 172.21.15.5:0/2143171566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce0101de0 0x7f0ce01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.814+0000 7f0ce6254640 1 -- 172.21.15.5:0/2143171566 >> 172.21.15.5:0/2143171566 conn(0x7f0ce00fd8c0 msgr2=0x7f0ce00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.814+0000 7f0ce6254640 1 -- 172.21.15.5:0/2143171566 shutdown_connections 2024-09-19T06:01:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.814+0000 7f0ce6254640 1 -- 172.21.15.5:0/2143171566 wait complete. 2024-09-19T06:01:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.815+0000 7f0ce6254640 1 Processor -- start 2024-09-19T06:01:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.815+0000 7f0ce6254640 1 -- start start 2024-09-19T06:01:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.815+0000 7f0ce6254640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ce0101de0 0x7f0ce0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.815+0000 7f0ce6254640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ce0102790 0x7f0ce01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.815+0000 7f0ce6254640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce0106f10 0x7f0ce019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.815+0000 7f0ce6254640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce0111e70 con 0x7f0ce0102790 2024-09-19T06:01:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.815+0000 7f0ce6254640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce0111cf0 con 0x7f0ce0106f10 2024-09-19T06:01:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.815+0000 7f0ce6254640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce0111ff0 con 0x7f0ce0101de0 2024-09-19T06:01:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.815+0000 7f0cd7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ce0102790 0x7f0ce01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.815+0000 7f0cd7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ce0102790 0x7f0ce01987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48216/0 (socket says 172.21.15.5:48216) 2024-09-19T06:01:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.815+0000 7f0cd7fff640 1 -- 172.21.15.5:0/690478428 learned_addr learned my addr 172.21.15.5:0/690478428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.815+0000 7f0cdf7fe640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ce0101de0 0x7f0ce0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.816+0000 7f0cdffff640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce0106f10 0x7f0ce019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.816+0000 7f0cd7fff640 1 -- 172.21.15.5:0/690478428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ce0101de0 msgr2=0x7f0ce0198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.816+0000 7f0cd7fff640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ce0101de0 0x7f0ce0198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.816+0000 7f0cd7fff640 1 -- 172.21.15.5:0/690478428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce0106f10 msgr2=0x7f0ce019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.816+0000 7f0cd7fff640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce0106f10 0x7f0ce019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.816+0000 7f0cd7fff640 1 -- 172.21.15.5:0/690478428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce019d230 con 0x7f0ce0102790 2024-09-19T06:01:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.816+0000 7f0cd7fff640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ce0102790 0x7f0ce01987c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8005ab0 tx=0x7f0cc8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.816+0000 7f0cdd7fa640 1 -- 172.21.15.5:0/690478428 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc803f070 con 0x7f0ce0102790 2024-09-19T06:01:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.816+0000 7f0cdd7fa640 1 -- 172.21.15.5:0/690478428 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cc8031d60 con 0x7f0ce0102790 2024-09-19T06:01:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.816+0000 7f0ce6254640 1 -- 172.21.15.5:0/690478428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce019d4c0 con 0x7f0ce0102790 2024-09-19T06:01:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.816+0000 7f0cdd7fa640 1 -- 172.21.15.5:0/690478428 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8035030 con 0x7f0ce0102790 2024-09-19T06:01:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.816+0000 7f0ce6254640 1 -- 172.21.15.5:0/690478428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce01a4e00 con 0x7f0ce0102790 2024-09-19T06:01:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.817+0000 7f0ce6254640 1 -- 172.21.15.5:0/690478428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce01032c0 con 0x7f0ce0102790 2024-09-19T06:01:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.818+0000 7f0cdd7fa640 1 -- 172.21.15.5:0/690478428 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f0cc8042070 con 0x7f0ce0102790 2024-09-19T06:01:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.819+0000 7f0cdd7fa640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0ccc0a32d0 0x7f0ccc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.819+0000 7f0cdf7fe640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0ccc0a32d0 0x7f0ccc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.819+0000 7f0cdd7fa640 1 -- 172.21.15.5:0/690478428 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0cc80f05c0 con 0x7f0ce0102790 2024-09-19T06:01:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.819+0000 7f0cdf7fe640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0ccc0a32d0 0x7f0ccc0a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0005e10 tx=0x7f0cd0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:48.823+0000 7f0cdd7fa640 1 -- 172.21.15.5:0/690478428 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0cc80bd110 con 0x7f0ce0102790 2024-09-19T06:01:49.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.027+0000 7f0ce6254640 1 -- 172.21.15.5:0/690478428 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f0ce0062630 con 0x7f0ccc0a32d0 2024-09-19T06:01:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.032+0000 7f0cdd7fa640 1 -- 172.21.15.5:0/690478428 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0ce0062630 con 0x7f0ccc0a32d0 2024-09-19T06:01:49.034 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_1/465d3653-ef74-435d-94d6-29eed9e79638 2024-09-19T06:01:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.034+0000 7f0ce6254640 1 -- 172.21.15.5:0/690478428 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0ccc0a32d0 msgr2=0x7f0ccc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.034+0000 7f0ce6254640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0ccc0a32d0 0x7f0ccc0a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0005e10 tx=0x7f0cd0005da0 comp rx=0 tx=0).stop 2024-09-19T06:01:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.034+0000 7f0ce6254640 1 -- 172.21.15.5:0/690478428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ce0102790 msgr2=0x7f0ce01987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.034+0000 7f0ce6254640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ce0102790 0x7f0ce01987c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8005ab0 tx=0x7f0cc8003d50 comp rx=0 tx=0).stop 2024-09-19T06:01:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.034+0000 7f0ce6254640 1 -- 172.21.15.5:0/690478428 shutdown_connections 2024-09-19T06:01:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.034+0000 7f0ce6254640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0ccc0a32d0 0x7f0ccc0a5790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.034+0000 7f0ce6254640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce0106f10 0x7f0ce019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.034+0000 7f0ce6254640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ce0102790 0x7f0ce01987c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.034+0000 7f0ce6254640 1 --2- 172.21.15.5:0/690478428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ce0101de0 0x7f0ce0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.034+0000 7f0ce6254640 1 -- 172.21.15.5:0/690478428 >> 172.21.15.5:0/690478428 conn(0x7f0ce00fd8c0 msgr2=0x7f0ce00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.034+0000 7f0ce6254640 1 -- 172.21.15.5:0/690478428 shutdown_connections 2024-09-19T06:01:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:49.034+0000 7f0ce6254640 1 -- 172.21.15.5:0/690478428 wait complete. 2024-09-19T06:01:49.802 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:01:50.068 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:49 smithi005 ceph-mon[30955]: pgmap v288: 97 pgs: 97 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.6 KiB/s wr, 21 op/s; 123 B/s, 0 objects/s recovering 2024-09-19T06:01:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:49 smithi005 ceph-mon[30955]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:50.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:49 smithi049 ceph-mon[34640]: pgmap v288: 97 pgs: 97 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.6 KiB/s wr, 21 op/s; 123 B/s, 0 objects/s recovering 2024-09-19T06:01:50.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:49 smithi049 ceph-mon[34640]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:50.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:49 smithi157 ceph-mon[35346]: pgmap v288: 97 pgs: 97 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.6 KiB/s wr, 21 op/s; 123 B/s, 0 objects/s recovering 2024-09-19T06:01:50.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:49 smithi157 ceph-mon[35346]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:01:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.543+0000 7f8e3de62640 1 -- 172.21.15.5:0/3762281199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e38112640 msgr2=0x7f8e38112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.543+0000 7f8e3de62640 1 --2- 172.21.15.5:0/3762281199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e38112640 0x7f8e38112a20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009900 tx=0x7f8e2802a410 comp rx=0 tx=0).stop 2024-09-19T06:01:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.543+0000 7f8e3de62640 1 -- 172.21.15.5:0/3762281199 shutdown_connections 2024-09-19T06:01:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.543+0000 7f8e3de62640 1 --2- 172.21.15.5:0/3762281199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e3810cf30 0x7f8e38119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.543+0000 7f8e3de62640 1 --2- 172.21.15.5:0/3762281199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3810c590 0x7f8e3810c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.543+0000 7f8e3de62640 1 --2- 172.21.15.5:0/3762281199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e38112640 0x7f8e38112a20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.543+0000 7f8e3de62640 1 -- 172.21.15.5:0/3762281199 >> 172.21.15.5:0/3762281199 conn(0x7f8e3806cdc0 msgr2=0x7f8e3806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.543+0000 7f8e3de62640 1 -- 172.21.15.5:0/3762281199 shutdown_connections 2024-09-19T06:01:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.543+0000 7f8e3de62640 1 -- 172.21.15.5:0/3762281199 wait complete. 2024-09-19T06:01:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.544+0000 7f8e3de62640 1 Processor -- start 2024-09-19T06:01:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.544+0000 7f8e3de62640 1 -- start start 2024-09-19T06:01:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.544+0000 7f8e3de62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3810c590 0x7f8e381a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.544+0000 7f8e3de62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e3810cf30 0x7f8e381a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.544+0000 7f8e3de62640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e38112640 0x7f8e381a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.544+0000 7f8e3de62640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3811c8c0 con 0x7f8e38112640 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.544+0000 7f8e3de62640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3811c740 con 0x7f8e3810c590 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.544+0000 7f8e3de62640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3811ca40 con 0x7f8e3810cf30 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e37fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e38112640 0x7f8e381a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e37fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e38112640 0x7f8e381a7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48238/0 (socket says 172.21.15.5:48238) 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e37fff640 1 -- 172.21.15.5:0/482391591 learned_addr learned my addr 172.21.15.5:0/482391591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e377fe640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3810c590 0x7f8e381a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e2effd640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e3810cf30 0x7f8e381a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e37fff640 1 -- 172.21.15.5:0/482391591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e3810cf30 msgr2=0x7f8e381a32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e37fff640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e3810cf30 0x7f8e381a32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e37fff640 1 -- 172.21.15.5:0/482391591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3810c590 msgr2=0x7f8e381a2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e37fff640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3810c590 0x7f8e381a2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e37fff640 1 -- 172.21.15.5:0/482391591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e381a7d10 con 0x7f8e38112640 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e37fff640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e38112640 0x7f8e381a7630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000eaa0 tx=0x7f8e3000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e357fa640 1 -- 172.21.15.5:0/482391591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30004d20 con 0x7f8e38112640 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e357fa640 1 -- 172.21.15.5:0/482391591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e30005020 con 0x7f8e38112640 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e3de62640 1 -- 172.21.15.5:0/482391591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e381a8000 con 0x7f8e38112640 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e357fa640 1 -- 172.21.15.5:0/482391591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e300116b0 con 0x7f8e38112640 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.545+0000 7f8e3de62640 1 -- 172.21.15.5:0/482391591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e381af8e0 con 0x7f8e38112640 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.546+0000 7f8e3de62640 1 -- 172.21.15.5:0/482391591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3810c9f0 con 0x7f8e38112640 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.547+0000 7f8e357fa640 1 -- 172.21.15.5:0/482391591 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f8e300051d0 con 0x7f8e38112640 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.548+0000 7f8e357fa640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8e240a3350 0x7f8e240a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.548+0000 7f8e357fa640 1 -- 172.21.15.5:0/482391591 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8e300cfbb0 con 0x7f8e38112640 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.548+0000 7f8e377fe640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8e240a3350 0x7f8e240a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.549+0000 7f8e377fe640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8e240a3350 0x7f8e240a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8e28000c00 tx=0x7f8e28005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.552+0000 7f8e357fa640 1 -- 172.21.15.5:0/482391591 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8e30016030 con 0x7f8e38112640 2024-09-19T06:01:50.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.765+0000 7f8e3de62640 1 -- 172.21.15.5:0/482391591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e3802d7c0 con 0x7f8e38112640 2024-09-19T06:01:50.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.765+0000 7f8e357fa640 1 -- 172.21.15.5:0/482391591 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11959 (secure 0 0 0) 0x7f8e3009c760 con 0x7f8e38112640 2024-09-19T06:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-19T06:01:41:840933+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:01:41.840914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:01:50.769 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-19T06:01:50.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.768+0000 7f8e3de62640 1 -- 172.21.15.5:0/482391591 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8e240a3350 msgr2=0x7f8e240a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:50.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.768+0000 7f8e3de62640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8e240a3350 0x7f8e240a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8e28000c00 tx=0x7f8e28005e80 comp rx=0 tx=0).stop 2024-09-19T06:01:50.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.768+0000 7f8e3de62640 1 -- 172.21.15.5:0/482391591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e38112640 msgr2=0x7f8e381a7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:50.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.768+0000 7f8e3de62640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e38112640 0x7f8e381a7630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000eaa0 tx=0x7f8e3000eea0 comp rx=0 tx=0).stop 2024-09-19T06:01:50.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.768+0000 7f8e3de62640 1 -- 172.21.15.5:0/482391591 shutdown_connections 2024-09-19T06:01:50.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.768+0000 7f8e3de62640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8e240a3350 0x7f8e240a5810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:50.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.768+0000 7f8e3de62640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e38112640 0x7f8e381a7630 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:50.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.768+0000 7f8e3de62640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e3810cf30 0x7f8e381a32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:50.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.768+0000 7f8e3de62640 1 --2- 172.21.15.5:0/482391591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3810c590 0x7f8e381a2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:50.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.768+0000 7f8e3de62640 1 -- 172.21.15.5:0/482391591 >> 172.21.15.5:0/482391591 conn(0x7f8e3806cdc0 msgr2=0x7f8e3810af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:50.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.768+0000 7f8e3de62640 1 -- 172.21.15.5:0/482391591 shutdown_connections 2024-09-19T06:01:50.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:50.768+0000 7f8e3de62640 1 -- 172.21.15.5:0/482391591 wait complete. 2024-09-19T06:01:51.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:50 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/482391591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:51.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:50 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/482391591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:51.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:50 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/482391591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:01:51.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:01:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:52.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:51 smithi005 ceph-mon[30955]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 20 op/s; 111 B/s, 0 objects/s recovering 2024-09-19T06:01:52.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:51 smithi049 ceph-mon[34640]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 20 op/s; 111 B/s, 0 objects/s recovering 2024-09-19T06:01:52.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:51 smithi157 ceph-mon[35346]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 20 op/s; 111 B/s, 0 objects/s recovering 2024-09-19T06:01:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.275+0000 7f12826ec640 1 -- 172.21.15.5:0/3750498324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127c100a80 msgr2=0x7f127c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.275+0000 7f12826ec640 1 --2- 172.21.15.5:0/3750498324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127c100a80 0x7f127c100ee0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1270009930 tx=0x7f127002a420 comp rx=0 tx=0).stop 2024-09-19T06:01:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.275+0000 7f12826ec640 1 -- 172.21.15.5:0/3750498324 shutdown_connections 2024-09-19T06:01:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.275+0000 7f12826ec640 1 --2- 172.21.15.5:0/3750498324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f127c101420 0x7f127c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.275+0000 7f12826ec640 1 --2- 172.21.15.5:0/3750498324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127c100a80 0x7f127c100ee0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.275+0000 7f12826ec640 1 --2- 172.21.15.5:0/3750498324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127c106a80 0x7f127c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.275+0000 7f12826ec640 1 -- 172.21.15.5:0/3750498324 >> 172.21.15.5:0/3750498324 conn(0x7f127c0fc7a0 msgr2=0x7f127c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.275+0000 7f12826ec640 1 -- 172.21.15.5:0/3750498324 shutdown_connections 2024-09-19T06:01:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.275+0000 7f12826ec640 1 -- 172.21.15.5:0/3750498324 wait complete. 2024-09-19T06:01:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.276+0000 7f12826ec640 1 Processor -- start 2024-09-19T06:01:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.276+0000 7f12826ec640 1 -- start start 2024-09-19T06:01:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.276+0000 7f12826ec640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127c100a80 0x7f127c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.276+0000 7f12826ec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127c101420 0x7f127c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.276+0000 7f12826ec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f127c106a80 0x7f127c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.276+0000 7f12826ec640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f127c110dd0 con 0x7f127c101420 2024-09-19T06:01:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.276+0000 7f12826ec640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f127c110c50 con 0x7f127c100a80 2024-09-19T06:01:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.276+0000 7f12826ec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f127c110f50 con 0x7f127c106a80 2024-09-19T06:01:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.276+0000 7f127b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127c101420 0x7f127c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.276+0000 7f127b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127c101420 0x7f127c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48260/0 (socket says 172.21.15.5:48260) 2024-09-19T06:01:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.276+0000 7f127b7fe640 1 -- 172.21.15.5:0/924895996 learned_addr learned my addr 172.21.15.5:0/924895996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f1280c89640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f127c106a80 0x7f127c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f127bfff640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127c100a80 0x7f127c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f127b7fe640 1 -- 172.21.15.5:0/924895996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f127c106a80 msgr2=0x7f127c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f127b7fe640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f127c106a80 0x7f127c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f127b7fe640 1 -- 172.21.15.5:0/924895996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127c100a80 msgr2=0x7f127c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f127b7fe640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127c100a80 0x7f127c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f127b7fe640 1 -- 172.21.15.5:0/924895996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127c19d510 con 0x7f127c101420 2024-09-19T06:01:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f127b7fe640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127c101420 0x7f127c198aa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1270005940 tx=0x7f12700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f12797fa640 1 -- 172.21.15.5:0/924895996 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127003f030 con 0x7f127c101420 2024-09-19T06:01:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f12826ec640 1 -- 172.21.15.5:0/924895996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127c19d7a0 con 0x7f127c101420 2024-09-19T06:01:52.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f12797fa640 1 -- 172.21.15.5:0/924895996 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1270003cb0 con 0x7f127c101420 2024-09-19T06:01:52.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f12797fa640 1 -- 172.21.15.5:0/924895996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270035070 con 0x7f127c101420 2024-09-19T06:01:52.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.277+0000 7f12826ec640 1 -- 172.21.15.5:0/924895996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f127c1a50e0 con 0x7f127c101420 2024-09-19T06:01:52.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.278+0000 7f12826ec640 1 -- 172.21.15.5:0/924895996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f127c1021c0 con 0x7f127c101420 2024-09-19T06:01:52.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.280+0000 7f12797fa640 1 -- 172.21.15.5:0/924895996 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f1270042030 con 0x7f127c101420 2024-09-19T06:01:52.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.280+0000 7f12797fa640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f12640a32d0 0x7f12640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:52.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.280+0000 7f12797fa640 1 -- 172.21.15.5:0/924895996 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f12700f01b0 con 0x7f127c101420 2024-09-19T06:01:52.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.280+0000 7f127bfff640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f12640a32d0 0x7f12640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:52.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.281+0000 7f127bfff640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f12640a32d0 0x7f12640a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f126c009990 tx=0x7f126c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:52.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.283+0000 7f12797fa640 1 -- 172.21.15.5:0/924895996 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f12700bcd60 con 0x7f127c101420 2024-09-19T06:01:52.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.461+0000 7f12826ec640 1 -- 172.21.15.5:0/924895996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f127c10a050 con 0x7f127c101420 2024-09-19T06:01:52.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.462+0000 7f12797fa640 1 -- 172.21.15.5:0/924895996 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22702 (secure 0 0 0) 0x7f12700bc660 con 0x7f127c101420 2024-09-19T06:01:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:52.465 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","created":"2024-09-19T05:53:51.454068+0000","modified":"2024-09-19T06:01:39.826379+0000","last_up_change":"2024-09-19T06:00:38.837916+0000","last_in_change":"2024-09-19T06:00:15.609776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:56:55.213931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:01:25.648588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:01:27.760408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f3c10d60-d354-407f-9cdf-099344599844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6803","nonce":3442981810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6805","nonce":3442981810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6809","nonce":3442981810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6807","nonce":3442981810}]},"public_addr":"172.21.15.5:6803/3442981810","cluster_addr":"172.21.15.5:6805/3442981810","heartbeat_back_addr":"172.21.15.5:6809/3442981810","heartbeat_front_addr":"172.21.15.5:6807/3442981810","state":["exists","up"]},{"osd":1,"uuid":"5f40f636-7761-45ac-87f3-910a2fa2feb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448265014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6803","nonce":1448265014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6807","nonce":1448265014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6805","nonce":1448265014}]},"public_addr":"172.21.15.49:6801/1448265014","cluster_addr":"172.21.15.49:6803/1448265014","heartbeat_back_addr":"172.21.15.49:6807/1448265014","heartbeat_front_addr":"172.21.15.49:6805/1448265014","state":["exists","up"]},{"osd":2,"uuid":"ad00940c-4fdd-4f5f-9eaa-b1f38bcac526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6801","nonce":2319626622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6803","nonce":2319626622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6807","nonce":2319626622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6805","nonce":2319626622}]},"public_addr":"172.21.15.157:6801/2319626622","cluster_addr":"172.21.15.157:6803/2319626622","heartbeat_back_addr":"172.21.15.157:6807/2319626622","heartbeat_front_addr":"172.21.15.157:6805/2319626622","state":["exists","up"]},{"osd":3,"uuid":"63fec329-1b62-4c5e-b8ca-084a84d4d762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6811","nonce":1452491753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6813","nonce":1452491753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6817","nonce":1452491753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6815","nonce":1452491753}]},"public_addr":"172.21.15.5:6811/1452491753","cluster_addr":"172.21.15.5:6813/1452491753","heartbeat_back_addr":"172.21.15.5:6817/1452491753","heartbeat_front_addr":"172.21.15.5:6815/1452491753","state":["exists","up"]},{"osd":4,"uuid":"ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6809","nonce":3114952580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6811","nonce":3114952580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6815","nonce":3114952580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6813","nonce":3114952580}]},"public_addr":"172.21.15.49:6809/3114952580","cluster_addr":"172.21.15.49:6811/3114952580","heartbeat_back_addr":"172.21.15.49:6815/3114952580","heartbeat_front_addr":"172.21.15.49:6813/3114952580","state":["exists","up"]},{"osd":5,"uuid":"21a7a7d3-ce42-4688-bfec-714be635eaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6809","nonce":3219063971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6811","nonce":3219063971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6815","nonce":3219063971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6813","nonce":3219063971}]},"public_addr":"172.21.15.157:6809/3219063971","cluster_addr":"172.21.15.157:6811/3219063971","heartbeat_back_addr":"172.21.15.157:6815/3219063971","heartbeat_front_addr":"172.21.15.157:6813/3219063971","state":["exists","up"]},{"osd":6,"uuid":"cbef0e08-d6b0-4515-a2ee-2c81b048583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6819","nonce":2957795194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6821","nonce":2957795194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6825","nonce":2957795194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6823","nonce":2957795194}]},"public_addr":"172.21.15.5:6819/2957795194","cluster_addr":"172.21.15.5:6821/2957795194","heartbeat_back_addr":"172.21.15.5:6825/2957795194","heartbeat_front_addr":"172.21.15.5:6823/2957795194","state":["exists","up"]},{"osd":7,"uuid":"d261ac55-532d-4aea-9b3e-ed2b4eeb1da6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6817","nonce":2959731400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6819","nonce":2959731400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6823","nonce":2959731400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6821","nonce":2959731400}]},"public_addr":"172.21.15.49:6817/2959731400","cluster_addr":"172.21.15.49:6819/2959731400","heartbeat_back_addr":"172.21.15.49:6823/2959731400","heartbeat_front_addr":"172.21.15.49:6821/2959731400","state":["exists","up"]},{"osd":8,"uuid":"15d0eaf7-d604-4546-b393-4d6d39018f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6817","nonce":290026859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6819","nonce":290026859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6823","nonce":290026859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6821","nonce":290026859}]},"public_addr":"172.21.15.157:6817/290026859","cluster_addr":"172.21.15.157:6819/290026859","heartbeat_back_addr":"172.21.15.157:6823/290026859","heartbeat_front_addr":"172.21.15.157:6821/290026859","state":["exists","up"]},{"osd":9,"uuid":"30c08537-f593-4299-bfa1-fa7ba8a95169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6827","nonce":4122448237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6829","nonce":4122448237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6833","nonce":4122448237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6831","nonce":4122448237}]},"public_addr":"172.21.15.5:6827/4122448237","cluster_addr":"172.21.15.5:6829/4122448237","heartbeat_back_addr":"172.21.15.5:6833/4122448237","heartbeat_front_addr":"172.21.15.5:6831/4122448237","state":["exists","up"]},{"osd":10,"uuid":"68821e98-bd81-49ec-8cd6-51fb9880b553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6825","nonce":2551748002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6827","nonce":2551748002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6831","nonce":2551748002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6829","nonce":2551748002}]},"public_addr":"172.21.15.49:6825/2551748002","cluster_addr":"172.21.15.49:6827/2551748002","heartbeat_back_addr":"172.21.15.49:6831/2551748002","heartbeat_front_addr":"172.21.15.49:6829/2551748002","state":["exists","up"]},{"osd":11,"uuid":"7e443b46-2697-4d0b-8452-868463a1ee1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6825","nonce":1996582544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6827","nonce":1996582544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6831","nonce":1996582544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6829","nonce":1996582544}]},"public_addr":"172.21.15.157:6825/1996582544","cluster_addr":"172.21.15.157:6827/1996582544","heartbeat_back_addr":"172.21.15.157:6831/1996582544","heartbeat_front_addr":"172.21.15.157:6829/1996582544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:01.453592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:27.194705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:51.525385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:17.676749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:43.075555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:07.258154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:32.112583+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:57.306534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:21.320950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:47.164610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:12.548337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:37.262580+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4102275559":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/2593868621":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/1340147787":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6800/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/850678758":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6800/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2930274429":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/1367598148":"2024-09-20T05:54:15.334457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:01:52.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.465+0000 7f12826ec640 1 -- 172.21.15.5:0/924895996 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f12640a32d0 msgr2=0x7f12640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.465+0000 7f12826ec640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f12640a32d0 0x7f12640a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f126c009990 tx=0x7f126c009110 comp rx=0 tx=0).stop 2024-09-19T06:01:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.465+0000 7f12826ec640 1 -- 172.21.15.5:0/924895996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127c101420 msgr2=0x7f127c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.465+0000 7f12826ec640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127c101420 0x7f127c198aa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1270005940 tx=0x7f12700042f0 comp rx=0 tx=0).stop 2024-09-19T06:01:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.465+0000 7f12826ec640 1 -- 172.21.15.5:0/924895996 shutdown_connections 2024-09-19T06:01:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.465+0000 7f12826ec640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f12640a32d0 0x7f12640a5790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:52.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.465+0000 7f12826ec640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f127c106a80 0x7f127c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:52.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.465+0000 7f12826ec640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127c101420 0x7f127c198aa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:52.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.465+0000 7f12826ec640 1 --2- 172.21.15.5:0/924895996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127c100a80 0x7f127c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:52.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.465+0000 7f12826ec640 1 -- 172.21.15.5:0/924895996 >> 172.21.15.5:0/924895996 conn(0x7f127c0fc7a0 msgr2=0x7f127c06ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:52.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.465+0000 7f12826ec640 1 -- 172.21.15.5:0/924895996 shutdown_connections 2024-09-19T06:01:52.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:52.465+0000 7f12826ec640 1 -- 172.21.15.5:0/924895996 wait complete. 2024-09-19T06:01:53.068 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:52 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/924895996' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:01:53.096 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T06:01:53.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:01:53.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:52 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/924895996' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:01:53.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:52 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/924895996' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:01:53.354 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:53.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.816+0000 7f08a17db640 1 -- 172.21.15.5:0/120910943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f089c106f10 msgr2=0x7f089c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.816+0000 7f08a17db640 1 --2- 172.21.15.5:0/120910943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f089c106f10 0x7f089c10ea90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0894009900 tx=0x7f089402a410 comp rx=0 tx=0).stop 2024-09-19T06:01:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.816+0000 7f08a17db640 1 -- 172.21.15.5:0/120910943 shutdown_connections 2024-09-19T06:01:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.816+0000 7f08a17db640 1 --2- 172.21.15.5:0/120910943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f089c106f10 0x7f089c10ea90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.816+0000 7f08a17db640 1 --2- 172.21.15.5:0/120910943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f089c102790 0x7f089c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.816+0000 7f08a17db640 1 --2- 172.21.15.5:0/120910943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f089c101de0 0x7f089c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.816+0000 7f08a17db640 1 -- 172.21.15.5:0/120910943 >> 172.21.15.5:0/120910943 conn(0x7f089c0fd8c0 msgr2=0x7f089c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.816+0000 7f08a17db640 1 -- 172.21.15.5:0/120910943 shutdown_connections 2024-09-19T06:01:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.816+0000 7f08a17db640 1 -- 172.21.15.5:0/120910943 wait complete. 2024-09-19T06:01:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.817+0000 7f08a17db640 1 Processor -- start 2024-09-19T06:01:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.817+0000 7f08a17db640 1 -- start start 2024-09-19T06:01:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.817+0000 7f08a17db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f089c101de0 0x7f089c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.817+0000 7f08a17db640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f089c102790 0x7f089c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.817+0000 7f08a17db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f089c106f10 0x7f089c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.817+0000 7f08a17db640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f089c111e20 con 0x7f089c106f10 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.817+0000 7f08a17db640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f089c111ca0 con 0x7f089c101de0 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.817+0000 7f08a17db640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f089c111fa0 con 0x7f089c102790 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f089b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f089c106f10 0x7f089c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f089b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f089c106f10 0x7f089c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48278/0 (socket says 172.21.15.5:48278) 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f089b7fe640 1 -- 172.21.15.5:0/519631778 learned_addr learned my addr 172.21.15.5:0/519631778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f089affd640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f089c101de0 0x7f089c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f089a7fc640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f089c102790 0x7f089c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f089b7fe640 1 -- 172.21.15.5:0/519631778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f089c102790 msgr2=0x7f089c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f089b7fe640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f089c102790 0x7f089c198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f089b7fe640 1 -- 172.21.15.5:0/519631778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f089c101de0 msgr2=0x7f089c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f089b7fe640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f089c101de0 0x7f089c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f089b7fe640 1 -- 172.21.15.5:0/519631778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c19d2f0 con 0x7f089c106f10 2024-09-19T06:01:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f089b7fe640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f089c106f10 0x7f089c19cc10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0894004770 tx=0x7f0894004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f0883fff640 1 -- 172.21.15.5:0/519631778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089403f030 con 0x7f089c106f10 2024-09-19T06:01:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f08a17db640 1 -- 172.21.15.5:0/519631778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c19d580 con 0x7f089c106f10 2024-09-19T06:01:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f0883fff640 1 -- 172.21.15.5:0/519631778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0894004440 con 0x7f089c106f10 2024-09-19T06:01:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f0883fff640 1 -- 172.21.15.5:0/519631778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894035070 con 0x7f089c106f10 2024-09-19T06:01:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.818+0000 7f08a17db640 1 -- 172.21.15.5:0/519631778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c1a4ec0 con 0x7f089c106f10 2024-09-19T06:01:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.819+0000 7f08a17db640 1 -- 172.21.15.5:0/519631778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f089c1032c0 con 0x7f089c106f10 2024-09-19T06:01:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.820+0000 7f0883fff640 1 -- 172.21.15.5:0/519631778 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f0894042030 con 0x7f089c106f10 2024-09-19T06:01:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.821+0000 7f0883fff640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f08840a32d0 0x7f08840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.821+0000 7f089affd640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f08840a32d0 0x7f08840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.821+0000 7f0883fff640 1 -- 172.21.15.5:0/519631778 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f08940efeb0 con 0x7f089c106f10 2024-09-19T06:01:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.822+0000 7f089affd640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f08840a32d0 0x7f08840a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f088c005fd0 tx=0x7f088c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.826+0000 7f0883fff640 1 -- 172.21.15.5:0/519631778 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f08940bca60 con 0x7f089c106f10 2024-09-19T06:01:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:53.999+0000 7f08a17db640 1 -- 172.21.15.5:0/519631778 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f089c10a160 con 0x7f08840a32d0 2024-09-19T06:01:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.008+0000 7f0883fff640 1 -- 172.21.15.5:0/519631778 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258758 (secure 0 0 0) 0x7f088c00e460 con 0x7f08840a32d0 2024-09-19T06:01:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:54.015 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-19T06:01:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.011+0000 7f08a17db640 1 -- 172.21.15.5:0/519631778 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f08840a32d0 msgr2=0x7f08840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.011+0000 7f08a17db640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f08840a32d0 0x7f08840a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f088c005fd0 tx=0x7f088c009500 comp rx=0 tx=0).stop 2024-09-19T06:01:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.011+0000 7f08a17db640 1 -- 172.21.15.5:0/519631778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f089c106f10 msgr2=0x7f089c19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.011+0000 7f08a17db640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f089c106f10 0x7f089c19cc10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0894004770 tx=0x7f0894004040 comp rx=0 tx=0).stop 2024-09-19T06:01:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.011+0000 7f08a17db640 1 -- 172.21.15.5:0/519631778 shutdown_connections 2024-09-19T06:01:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.011+0000 7f08a17db640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f08840a32d0 0x7f08840a5790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.011+0000 7f08a17db640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f089c106f10 0x7f089c19cc10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.011+0000 7f08a17db640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f089c102790 0x7f089c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.011+0000 7f08a17db640 1 --2- 172.21.15.5:0/519631778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f089c101de0 0x7f089c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.011+0000 7f08a17db640 1 -- 172.21.15.5:0/519631778 >> 172.21.15.5:0/519631778 conn(0x7f089c0fd8c0 msgr2=0x7f089c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.011+0000 7f08a17db640 1 -- 172.21.15.5:0/519631778 shutdown_connections 2024-09-19T06:01:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:54.011+0000 7f08a17db640 1 -- 172.21.15.5:0/519631778 wait complete. 2024-09-19T06:01:54.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:54 smithi005 ceph-mon[30955]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 KiB/s wr, 16 op/s; 102 B/s, 0 objects/s recovering 2024-09-19T06:01:54.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:54 smithi157 ceph-mon[35346]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 KiB/s wr, 16 op/s; 102 B/s, 0 objects/s recovering 2024-09-19T06:01:54.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:54 smithi049 ceph-mon[34640]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 KiB/s wr, 16 op/s; 102 B/s, 0 objects/s recovering 2024-09-19T06:01:54.780 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-19T06:01:52.788890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645398,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":255,"num_read_kb":235,"num_write":412,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387040,"kb_used_data":12816,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554784,"statfs":{"total":1151940427776,"available":1151544098816,"internally_reserved":0,"allocated":13123584,"data_stored":7162935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49696,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":-2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":119,"num_read_kb":116,"num_write":78,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002927"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833240+0000","last_change":"2024-09-19T06:01:28.812722+0000","last_active":"2024-09-19T06:01:39.833240+0000","last_peered":"2024-09-19T06:01:39.833240+0000","last_clean":"2024-09-19T06:01:39.833240+0000","last_became_active":"2024-09-19T06:01:28.812283+0000","last_became_peered":"2024-09-19T06:01:28.812283+0000","last_unstale":"2024-09-19T06:01:39.833240+0000","last_undegraded":"2024-09-19T06:01:39.833240+0000","last_fullsized":"2024-09-19T06:01:39.833240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:59.146636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831968+0000","last_change":"2024-09-19T06:01:28.805210+0000","last_active":"2024-09-19T06:01:39.831968+0000","last_peered":"2024-09-19T06:01:39.831968+0000","last_clean":"2024-09-19T06:01:39.831968+0000","last_became_active":"2024-09-19T06:01:28.804906+0000","last_became_peered":"2024-09-19T06:01:28.804906+0000","last_unstale":"2024-09-19T06:01:39.831968+0000","last_undegraded":"2024-09-19T06:01:39.831968+0000","last_fullsized":"2024-09-19T06:01:39.831968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:00.227101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832025+0000","last_change":"2024-09-19T06:01:28.811909+0000","last_active":"2024-09-19T06:01:39.832025+0000","last_peered":"2024-09-19T06:01:39.832025+0000","last_clean":"2024-09-19T06:01:39.832025+0000","last_became_active":"2024-09-19T06:01:28.811547+0000","last_became_peered":"2024-09-19T06:01:28.811547+0000","last_unstale":"2024-09-19T06:01:39.832025+0000","last_undegraded":"2024-09-19T06:01:39.832025+0000","last_fullsized":"2024-09-19T06:01:39.832025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:50.022439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830196+0000","last_change":"2024-09-19T06:01:28.806782+0000","last_active":"2024-09-19T06:01:39.830196+0000","last_peered":"2024-09-19T06:01:39.830196+0000","last_clean":"2024-09-19T06:01:39.830196+0000","last_became_active":"2024-09-19T06:01:28.805520+0000","last_became_peered":"2024-09-19T06:01:28.805520+0000","last_unstale":"2024-09-19T06:01:39.830196+0000","last_undegraded":"2024-09-19T06:01:39.830196+0000","last_fullsized":"2024-09-19T06:01:39.830196+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:01.652669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837308+0000","last_change":"2024-09-19T06:01:28.809971+0000","last_active":"2024-09-19T06:01:39.837308+0000","last_peered":"2024-09-19T06:01:39.837308+0000","last_clean":"2024-09-19T06:01:39.837308+0000","last_became_active":"2024-09-19T06:01:28.809593+0000","last_became_peered":"2024-09-19T06:01:28.809593+0000","last_unstale":"2024-09-19T06:01:39.837308+0000","last_undegraded":"2024-09-19T06:01:39.837308+0000","last_fullsized":"2024-09-19T06:01:39.837308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.546526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323650+0000","last_change":"2024-09-19T06:01:28.801395+0000","last_active":"2024-09-19T06:01:40.323650+0000","last_peered":"2024-09-19T06:01:40.323650+0000","last_clean":"2024-09-19T06:01:40.323650+0000","last_became_active":"2024-09-19T06:01:28.800987+0000","last_became_peered":"2024-09-19T06:01:28.800987+0000","last_unstale":"2024-09-19T06:01:40.323650+0000","last_undegraded":"2024-09-19T06:01:40.323650+0000","last_fullsized":"2024-09-19T06:01:40.323650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:18.986489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837262+0000","last_change":"2024-09-19T06:01:28.806554+0000","last_active":"2024-09-19T06:01:39.837262+0000","last_peered":"2024-09-19T06:01:39.837262+0000","last_clean":"2024-09-19T06:01:39.837262+0000","last_became_active":"2024-09-19T06:01:28.805946+0000","last_became_peered":"2024-09-19T06:01:28.805946+0000","last_unstale":"2024-09-19T06:01:39.837262+0000","last_undegraded":"2024-09-19T06:01:39.837262+0000","last_fullsized":"2024-09-19T06:01:39.837262+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:56:20.130414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.979824+0000","last_change":"2024-09-19T06:01:28.804789+0000","last_active":"2024-09-19T06:01:39.979824+0000","last_peered":"2024-09-19T06:01:39.979824+0000","last_clean":"2024-09-19T06:01:39.979824+0000","last_became_active":"2024-09-19T06:01:28.804484+0000","last_became_peered":"2024-09-19T06:01:28.804484+0000","last_unstale":"2024-09-19T06:01:39.979824+0000","last_undegraded":"2024-09-19T06:01:39.979824+0000","last_fullsized":"2024-09-19T06:01:39.979824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:19.737928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832749+0000","last_change":"2024-09-19T06:01:28.814504+0000","last_active":"2024-09-19T06:01:39.832749+0000","last_peered":"2024-09-19T06:01:39.832749+0000","last_clean":"2024-09-19T06:01:39.832749+0000","last_became_active":"2024-09-19T06:01:28.814066+0000","last_became_peered":"2024-09-19T06:01:28.814066+0000","last_unstale":"2024-09-19T06:01:39.832749+0000","last_undegraded":"2024-09-19T06:01:39.832749+0000","last_fullsized":"2024-09-19T06:01:39.832749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:50.904164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.324053+0000","last_change":"2024-09-19T06:01:28.807303+0000","last_active":"2024-09-19T06:01:40.324053+0000","last_peered":"2024-09-19T06:01:40.324053+0000","last_clean":"2024-09-19T06:01:40.324053+0000","last_became_active":"2024-09-19T06:01:28.806978+0000","last_became_peered":"2024-09-19T06:01:28.806978+0000","last_unstale":"2024-09-19T06:01:40.324053+0000","last_undegraded":"2024-09-19T06:01:40.324053+0000","last_fullsized":"2024-09-19T06:01:40.324053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:33.259261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837723+0000","last_change":"2024-09-19T06:01:28.814844+0000","last_active":"2024-09-19T06:01:39.837723+0000","last_peered":"2024-09-19T06:01:39.837723+0000","last_clean":"2024-09-19T06:01:39.837723+0000","last_became_active":"2024-09-19T06:01:28.814531+0000","last_became_peered":"2024-09-19T06:01:28.814531+0000","last_unstale":"2024-09-19T06:01:39.837723+0000","last_undegraded":"2024-09-19T06:01:39.837723+0000","last_fullsized":"2024-09-19T06:01:39.837723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:17.209890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830253+0000","last_change":"2024-09-19T06:01:28.806378+0000","last_active":"2024-09-19T06:01:39.830253+0000","last_peered":"2024-09-19T06:01:39.830253+0000","last_clean":"2024-09-19T06:01:39.830253+0000","last_became_active":"2024-09-19T06:01:28.805053+0000","last_became_peered":"2024-09-19T06:01:28.805053+0000","last_unstale":"2024-09-19T06:01:39.830253+0000","last_undegraded":"2024-09-19T06:01:39.830253+0000","last_fullsized":"2024-09-19T06:01:39.830253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:15.080548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829093+0000","last_change":"2024-09-19T06:01:28.802941+0000","last_active":"2024-09-19T06:01:39.829093+0000","last_peered":"2024-09-19T06:01:39.829093+0000","last_clean":"2024-09-19T06:01:39.829093+0000","last_became_active":"2024-09-19T06:01:28.802590+0000","last_became_peered":"2024-09-19T06:01:28.802590+0000","last_unstale":"2024-09-19T06:01:39.829093+0000","last_undegraded":"2024-09-19T06:01:39.829093+0000","last_fullsized":"2024-09-19T06:01:39.829093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:25.581854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831644+0000","last_change":"2024-09-19T06:01:28.800960+0000","last_active":"2024-09-19T06:01:39.831644+0000","last_peered":"2024-09-19T06:01:39.831644+0000","last_clean":"2024-09-19T06:01:39.831644+0000","last_became_active":"2024-09-19T06:01:28.800484+0000","last_became_peered":"2024-09-19T06:01:28.800484+0000","last_unstale":"2024-09-19T06:01:39.831644+0000","last_undegraded":"2024-09-19T06:01:39.831644+0000","last_fullsized":"2024-09-19T06:01:39.831644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:08.506105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831592+0000","last_change":"2024-09-19T06:01:28.800771+0000","last_active":"2024-09-19T06:01:39.831592+0000","last_peered":"2024-09-19T06:01:39.831592+0000","last_clean":"2024-09-19T06:01:39.831592+0000","last_became_active":"2024-09-19T06:01:28.800427+0000","last_became_peered":"2024-09-19T06:01:28.800427+0000","last_unstale":"2024-09-19T06:01:39.831592+0000","last_undegraded":"2024-09-19T06:01:39.831592+0000","last_fullsized":"2024-09-19T06:01:39.831592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:19.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831899+0000","last_change":"2024-09-19T06:01:28.815528+0000","last_active":"2024-09-19T06:01:39.831899+0000","last_peered":"2024-09-19T06:01:39.831899+0000","last_clean":"2024-09-19T06:01:39.831899+0000","last_became_active":"2024-09-19T06:01:28.815129+0000","last_became_peered":"2024-09-19T06:01:28.815129+0000","last_unstale":"2024-09-19T06:01:39.831899+0000","last_undegraded":"2024-09-19T06:01:39.831899+0000","last_fullsized":"2024-09-19T06:01:39.831899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:01.936711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833355+0000","last_change":"2024-09-19T06:01:28.816120+0000","last_active":"2024-09-19T06:01:39.833355+0000","last_peered":"2024-09-19T06:01:39.833355+0000","last_clean":"2024-09-19T06:01:39.833355+0000","last_became_active":"2024-09-19T06:01:28.815822+0000","last_became_peered":"2024-09-19T06:01:28.815822+0000","last_unstale":"2024-09-19T06:01:39.833355+0000","last_undegraded":"2024-09-19T06:01:39.833355+0000","last_fullsized":"2024-09-19T06:01:39.833355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:14:58.670452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.836100+0000","last_change":"2024-09-19T06:01:28.809825+0000","last_active":"2024-09-19T06:01:39.836100+0000","last_peered":"2024-09-19T06:01:39.836100+0000","last_clean":"2024-09-19T06:01:39.836100+0000","last_became_active":"2024-09-19T06:01:28.809530+0000","last_became_peered":"2024-09-19T06:01:28.809530+0000","last_unstale":"2024-09-19T06:01:39.836100+0000","last_undegraded":"2024-09-19T06:01:39.836100+0000","last_fullsized":"2024-09-19T06:01:39.836100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:49:23.442436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830644+0000","last_change":"2024-09-19T06:01:28.801297+0000","last_active":"2024-09-19T06:01:39.830644+0000","last_peered":"2024-09-19T06:01:39.830644+0000","last_clean":"2024-09-19T06:01:39.830644+0000","last_became_active":"2024-09-19T06:01:28.800984+0000","last_became_peered":"2024-09-19T06:01:28.800984+0000","last_unstale":"2024-09-19T06:01:39.830644+0000","last_undegraded":"2024-09-19T06:01:39.830644+0000","last_fullsized":"2024-09-19T06:01:39.830644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:01.485951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832407+0000","last_change":"2024-09-19T06:01:28.804567+0000","last_active":"2024-09-19T06:01:39.832407+0000","last_peered":"2024-09-19T06:01:39.832407+0000","last_clean":"2024-09-19T06:01:39.832407+0000","last_became_active":"2024-09-19T06:01:28.804213+0000","last_became_peered":"2024-09-19T06:01:28.804213+0000","last_unstale":"2024-09-19T06:01:39.832407+0000","last_undegraded":"2024-09-19T06:01:39.832407+0000","last_fullsized":"2024-09-19T06:01:39.832407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.186460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829142+0000","last_change":"2024-09-19T06:01:28.801961+0000","last_active":"2024-09-19T06:01:39.829142+0000","last_peered":"2024-09-19T06:01:39.829142+0000","last_clean":"2024-09-19T06:01:39.829142+0000","last_became_active":"2024-09-19T06:01:28.801631+0000","last_became_peered":"2024-09-19T06:01:28.801631+0000","last_unstale":"2024-09-19T06:01:39.829142+0000","last_undegraded":"2024-09-19T06:01:39.829142+0000","last_fullsized":"2024-09-19T06:01:39.829142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:15.628908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832042+0000","last_change":"2024-09-19T06:01:28.815160+0000","last_active":"2024-09-19T06:01:39.832042+0000","last_peered":"2024-09-19T06:01:39.832042+0000","last_clean":"2024-09-19T06:01:39.832042+0000","last_became_active":"2024-09-19T06:01:28.813329+0000","last_became_peered":"2024-09-19T06:01:28.813329+0000","last_unstale":"2024-09-19T06:01:39.832042+0000","last_undegraded":"2024-09-19T06:01:39.832042+0000","last_fullsized":"2024-09-19T06:01:39.832042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:17.027393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.979874+0000","last_change":"2024-09-19T06:01:28.799205+0000","last_active":"2024-09-19T06:01:39.979874+0000","last_peered":"2024-09-19T06:01:39.979874+0000","last_clean":"2024-09-19T06:01:39.979874+0000","last_became_active":"2024-09-19T06:01:28.798891+0000","last_became_peered":"2024-09-19T06:01:28.798891+0000","last_unstale":"2024-09-19T06:01:39.979874+0000","last_undegraded":"2024-09-19T06:01:39.979874+0000","last_fullsized":"2024-09-19T06:01:39.979874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:41.227721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832707+0000","last_change":"2024-09-19T06:01:28.795823+0000","last_active":"2024-09-19T06:01:39.832707+0000","last_peered":"2024-09-19T06:01:39.832707+0000","last_clean":"2024-09-19T06:01:39.832707+0000","last_became_active":"2024-09-19T06:01:28.795510+0000","last_became_peered":"2024-09-19T06:01:28.795510+0000","last_unstale":"2024-09-19T06:01:39.832707+0000","last_undegraded":"2024-09-19T06:01:39.832707+0000","last_fullsized":"2024-09-19T06:01:39.832707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:54.831610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980449+0000","last_change":"2024-09-19T06:01:28.799696+0000","last_active":"2024-09-19T06:01:39.980449+0000","last_peered":"2024-09-19T06:01:39.980449+0000","last_clean":"2024-09-19T06:01:39.980449+0000","last_became_active":"2024-09-19T06:01:28.799417+0000","last_became_peered":"2024-09-19T06:01:28.799417+0000","last_unstale":"2024-09-19T06:01:39.980449+0000","last_undegraded":"2024-09-19T06:01:39.980449+0000","last_fullsized":"2024-09-19T06:01:39.980449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:02.330808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830714+0000","last_change":"2024-09-19T06:01:28.807146+0000","last_active":"2024-09-19T06:01:39.830714+0000","last_peered":"2024-09-19T06:01:39.830714+0000","last_clean":"2024-09-19T06:01:39.830714+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:01:39.830714+0000","last_undegraded":"2024-09-19T06:01:39.830714+0000","last_fullsized":"2024-09-19T06:01:39.830714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:12.308421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832819+0000","last_change":"2024-09-19T06:01:28.803555+0000","last_active":"2024-09-19T06:01:39.832819+0000","last_peered":"2024-09-19T06:01:39.832819+0000","last_clean":"2024-09-19T06:01:39.832819+0000","last_became_active":"2024-09-19T06:01:28.803242+0000","last_became_peered":"2024-09-19T06:01:28.803242+0000","last_unstale":"2024-09-19T06:01:39.832819+0000","last_undegraded":"2024-09-19T06:01:39.832819+0000","last_fullsized":"2024-09-19T06:01:39.832819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:02.281152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829217+0000","last_change":"2024-09-19T06:01:28.804378+0000","last_active":"2024-09-19T06:01:39.829217+0000","last_peered":"2024-09-19T06:01:39.829217+0000","last_clean":"2024-09-19T06:01:39.829217+0000","last_became_active":"2024-09-19T06:01:28.804099+0000","last_became_peered":"2024-09-19T06:01:28.804099+0000","last_unstale":"2024-09-19T06:01:39.829217+0000","last_undegraded":"2024-09-19T06:01:39.829217+0000","last_fullsized":"2024-09-19T06:01:39.829217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:11.558384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831784+0000","last_change":"2024-09-19T06:01:28.799999+0000","last_active":"2024-09-19T06:01:39.831784+0000","last_peered":"2024-09-19T06:01:39.831784+0000","last_clean":"2024-09-19T06:01:39.831784+0000","last_became_active":"2024-09-19T06:01:28.799645+0000","last_became_peered":"2024-09-19T06:01:28.799645+0000","last_unstale":"2024-09-19T06:01:39.831784+0000","last_undegraded":"2024-09-19T06:01:39.831784+0000","last_fullsized":"2024-09-19T06:01:39.831784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:20.442119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830148+0000","last_change":"2024-09-19T06:01:28.800143+0000","last_active":"2024-09-19T06:01:39.830148+0000","last_peered":"2024-09-19T06:01:39.830148+0000","last_clean":"2024-09-19T06:01:39.830148+0000","last_became_active":"2024-09-19T06:01:28.799830+0000","last_became_peered":"2024-09-19T06:01:28.799830+0000","last_unstale":"2024-09-19T06:01:39.830148+0000","last_undegraded":"2024-09-19T06:01:39.830148+0000","last_fullsized":"2024-09-19T06:01:39.830148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:43.824740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830661+0000","last_change":"2024-09-19T06:01:28.806130+0000","last_active":"2024-09-19T06:01:39.830661+0000","last_peered":"2024-09-19T06:01:39.830661+0000","last_clean":"2024-09-19T06:01:39.830661+0000","last_became_active":"2024-09-19T06:01:28.805772+0000","last_became_peered":"2024-09-19T06:01:28.805772+0000","last_unstale":"2024-09-19T06:01:39.830661+0000","last_undegraded":"2024-09-19T06:01:39.830661+0000","last_fullsized":"2024-09-19T06:01:39.830661+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:00.316311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837412+0000","last_change":"2024-09-19T06:01:39.837412+0000","last_active":"2024-09-19T06:01:39.837412+0000","last_peered":"2024-09-19T06:01:39.837412+0000","last_clean":"2024-09-19T06:01:39.837412+0000","last_became_active":"2024-09-19T06:01:39.837035+0000","last_became_peered":"2024-09-19T06:01:39.837035+0000","last_unstale":"2024-09-19T06:01:39.837412+0000","last_undegraded":"2024-09-19T06:01:39.837412+0000","last_fullsized":"2024-09-19T06:01:39.837412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:00.073884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837634+0000","last_change":"2024-09-19T06:01:28.800324+0000","last_active":"2024-09-19T06:01:39.837634+0000","last_peered":"2024-09-19T06:01:39.837634+0000","last_clean":"2024-09-19T06:01:39.837634+0000","last_became_active":"2024-09-19T06:01:28.799986+0000","last_became_peered":"2024-09-19T06:01:28.799986+0000","last_unstale":"2024-09-19T06:01:39.837634+0000","last_undegraded":"2024-09-19T06:01:39.837634+0000","last_fullsized":"2024-09-19T06:01:39.837634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:41.963346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:46.858367+0000","last_change":"2024-09-19T06:01:39.844897+0000","last_active":"2024-09-19T06:01:46.858367+0000","last_peered":"2024-09-19T06:01:46.858367+0000","last_clean":"2024-09-19T06:01:46.858367+0000","last_became_active":"2024-09-19T06:01:39.838347+0000","last_became_peered":"2024-09-19T06:01:39.838347+0000","last_unstale":"2024-09-19T06:01:46.858367+0000","last_undegraded":"2024-09-19T06:01:46.858367+0000","last_fullsized":"2024-09-19T06:01:46.858367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:26.235227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.834624+0000","last_change":"2024-09-19T06:01:28.802644+0000","last_active":"2024-09-19T06:01:39.834624+0000","last_peered":"2024-09-19T06:01:39.834624+0000","last_clean":"2024-09-19T06:01:39.834624+0000","last_became_active":"2024-09-19T06:01:28.802288+0000","last_became_peered":"2024-09-19T06:01:28.802288+0000","last_unstale":"2024-09-19T06:01:39.834624+0000","last_undegraded":"2024-09-19T06:01:39.834624+0000","last_fullsized":"2024-09-19T06:01:39.834624+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:32.031860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323748+0000","last_change":"2024-09-19T06:01:26.781104+0000","last_active":"2024-09-19T06:01:40.323748+0000","last_peered":"2024-09-19T06:01:40.323748+0000","last_clean":"2024-09-19T06:01:40.323748+0000","last_became_active":"2024-09-19T06:01:26.780733+0000","last_became_peered":"2024-09-19T06:01:26.780733+0000","last_unstale":"2024-09-19T06:01:40.323748+0000","last_undegraded":"2024-09-19T06:01:40.323748+0000","last_fullsized":"2024-09-19T06:01:40.323748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:54.490867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.828967+0000","last_change":"2024-09-19T06:01:26.782901+0000","last_active":"2024-09-19T06:01:39.828967+0000","last_peered":"2024-09-19T06:01:39.828967+0000","last_clean":"2024-09-19T06:01:39.828967+0000","last_became_active":"2024-09-19T06:01:26.782450+0000","last_became_peered":"2024-09-19T06:01:26.782450+0000","last_unstale":"2024-09-19T06:01:39.828967+0000","last_undegraded":"2024-09-19T06:01:39.828967+0000","last_fullsized":"2024-09-19T06:01:39.828967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:07.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.979905+0000","last_change":"2024-09-19T06:01:28.804105+0000","last_active":"2024-09-19T06:01:39.979905+0000","last_peered":"2024-09-19T06:01:39.979905+0000","last_clean":"2024-09-19T06:01:39.979905+0000","last_became_active":"2024-09-19T06:01:28.797980+0000","last_became_peered":"2024-09-19T06:01:28.797980+0000","last_unstale":"2024-09-19T06:01:39.979905+0000","last_undegraded":"2024-09-19T06:01:39.979905+0000","last_fullsized":"2024-09-19T06:01:39.979905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:50.130102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.848777+0000","last_change":"2024-09-19T06:01:26.786923+0000","last_active":"2024-09-19T06:01:40.848777+0000","last_peered":"2024-09-19T06:01:40.848777+0000","last_clean":"2024-09-19T06:01:40.848777+0000","last_became_active":"2024-09-19T06:01:26.786542+0000","last_became_peered":"2024-09-19T06:01:26.786542+0000","last_unstale":"2024-09-19T06:01:40.848777+0000","last_undegraded":"2024-09-19T06:01:40.848777+0000","last_fullsized":"2024-09-19T06:01:40.848777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:30.032992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.835994+0000","last_change":"2024-09-19T06:01:28.802172+0000","last_active":"2024-09-19T06:01:39.835994+0000","last_peered":"2024-09-19T06:01:39.835994+0000","last_clean":"2024-09-19T06:01:39.835994+0000","last_became_active":"2024-09-19T06:01:28.801815+0000","last_became_peered":"2024-09-19T06:01:28.801815+0000","last_unstale":"2024-09-19T06:01:39.835994+0000","last_undegraded":"2024-09-19T06:01:39.835994+0000","last_fullsized":"2024-09-19T06:01:39.835994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:09.587638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:48.859729+0000","last_change":"2024-09-19T06:01:26.777870+0000","last_active":"2024-09-19T06:01:48.859729+0000","last_peered":"2024-09-19T06:01:48.859729+0000","last_clean":"2024-09-19T06:01:48.859729+0000","last_became_active":"2024-09-19T06:01:26.777509+0000","last_became_peered":"2024-09-19T06:01:26.777509+0000","last_unstale":"2024-09-19T06:01:48.859729+0000","last_undegraded":"2024-09-19T06:01:48.859729+0000","last_fullsized":"2024-09-19T06:01:48.859729+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:15.885911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832446+0000","last_change":"2024-09-19T06:01:28.812201+0000","last_active":"2024-09-19T06:01:39.832446+0000","last_peered":"2024-09-19T06:01:39.832446+0000","last_clean":"2024-09-19T06:01:39.832446+0000","last_became_active":"2024-09-19T06:01:28.811801+0000","last_became_peered":"2024-09-19T06:01:28.811801+0000","last_unstale":"2024-09-19T06:01:39.832446+0000","last_undegraded":"2024-09-19T06:01:39.832446+0000","last_fullsized":"2024-09-19T06:01:39.832446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:32.904787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.848667+0000","last_change":"2024-09-19T06:01:26.787845+0000","last_active":"2024-09-19T06:01:40.848667+0000","last_peered":"2024-09-19T06:01:40.848667+0000","last_clean":"2024-09-19T06:01:40.848667+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:01:40.848667+0000","last_undegraded":"2024-09-19T06:01:40.848667+0000","last_fullsized":"2024-09-19T06:01:40.848667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.114865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832096+0000","last_change":"2024-09-19T06:01:28.801523+0000","last_active":"2024-09-19T06:01:39.832096+0000","last_peered":"2024-09-19T06:01:39.832096+0000","last_clean":"2024-09-19T06:01:39.832096+0000","last_became_active":"2024-09-19T06:01:28.801205+0000","last_became_peered":"2024-09-19T06:01:28.801205+0000","last_unstale":"2024-09-19T06:01:39.832096+0000","last_undegraded":"2024-09-19T06:01:39.832096+0000","last_fullsized":"2024-09-19T06:01:39.832096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:10.314085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833424+0000","last_change":"2024-09-19T06:01:27.022172+0000","last_active":"2024-09-19T06:01:39.833424+0000","last_peered":"2024-09-19T06:01:39.833424+0000","last_clean":"2024-09-19T06:01:39.833424+0000","last_became_active":"2024-09-19T06:01:27.021568+0000","last_became_peered":"2024-09-19T06:01:27.021568+0000","last_unstale":"2024-09-19T06:01:39.833424+0000","last_undegraded":"2024-09-19T06:01:39.833424+0000","last_fullsized":"2024-09-19T06:01:39.833424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:18.031690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.839061+0000","last_change":"2024-09-19T06:01:39.839061+0000","last_active":"2024-09-19T06:01:39.839061+0000","last_peered":"2024-09-19T06:01:39.839061+0000","last_clean":"2024-09-19T06:01:39.839061+0000","last_became_active":"2024-09-19T06:01:39.838644+0000","last_became_peered":"2024-09-19T06:01:39.838644+0000","last_unstale":"2024-09-19T06:01:39.839061+0000","last_undegraded":"2024-09-19T06:01:39.839061+0000","last_fullsized":"2024-09-19T06:01:39.839061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:02.136222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'24","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:49.011063+0000","last_change":"2024-09-19T06:01:26.784781+0000","last_active":"2024-09-19T06:01:49.011063+0000","last_peered":"2024-09-19T06:01:49.011063+0000","last_clean":"2024-09-19T06:01:49.011063+0000","last_became_active":"2024-09-19T06:01:26.784284+0000","last_became_peered":"2024-09-19T06:01:26.784284+0000","last_unstale":"2024-09-19T06:01:49.011063+0000","last_undegraded":"2024-09-19T06:01:49.011063+0000","last_fullsized":"2024-09-19T06:01:49.011063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:13.063299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":50,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.839724+0000","last_change":"2024-09-19T06:01:28.802604+0000","last_active":"2024-09-19T06:01:39.839724+0000","last_peered":"2024-09-19T06:01:39.839724+0000","last_clean":"2024-09-19T06:01:39.839724+0000","last_became_active":"2024-09-19T06:01:28.802285+0000","last_became_peered":"2024-09-19T06:01:28.802285+0000","last_unstale":"2024-09-19T06:01:39.839724+0000","last_undegraded":"2024-09-19T06:01:39.839724+0000","last_fullsized":"2024-09-19T06:01:39.839724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:57.177791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:41.846861+0000","last_change":"2024-09-19T06:01:26.783678+0000","last_active":"2024-09-19T06:01:41.846861+0000","last_peered":"2024-09-19T06:01:41.846861+0000","last_clean":"2024-09-19T06:01:41.846861+0000","last_became_active":"2024-09-19T06:01:26.783165+0000","last_became_peered":"2024-09-19T06:01:26.783165+0000","last_unstale":"2024-09-19T06:01:41.846861+0000","last_undegraded":"2024-09-19T06:01:41.846861+0000","last_fullsized":"2024-09-19T06:01:41.846861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:50.535733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.828953+0000","last_change":"2024-09-19T06:01:28.808553+0000","last_active":"2024-09-19T06:01:39.828953+0000","last_peered":"2024-09-19T06:01:39.828953+0000","last_clean":"2024-09-19T06:01:39.828953+0000","last_became_active":"2024-09-19T06:01:28.808215+0000","last_became_peered":"2024-09-19T06:01:28.808215+0000","last_unstale":"2024-09-19T06:01:39.828953+0000","last_undegraded":"2024-09-19T06:01:39.828953+0000","last_fullsized":"2024-09-19T06:01:39.828953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:55.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.848791+0000","last_change":"2024-09-19T06:01:26.787137+0000","last_active":"2024-09-19T06:01:40.848791+0000","last_peered":"2024-09-19T06:01:40.848791+0000","last_clean":"2024-09-19T06:01:40.848791+0000","last_became_active":"2024-09-19T06:01:26.786676+0000","last_became_peered":"2024-09-19T06:01:26.786676+0000","last_unstale":"2024-09-19T06:01:40.848791+0000","last_undegraded":"2024-09-19T06:01:40.848791+0000","last_fullsized":"2024-09-19T06:01:40.848791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.609969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837656+0000","last_change":"2024-09-19T06:01:28.812106+0000","last_active":"2024-09-19T06:01:39.837656+0000","last_peered":"2024-09-19T06:01:39.837656+0000","last_clean":"2024-09-19T06:01:39.837656+0000","last_became_active":"2024-09-19T06:01:28.811771+0000","last_became_peered":"2024-09-19T06:01:28.811771+0000","last_unstale":"2024-09-19T06:01:39.837656+0000","last_undegraded":"2024-09-19T06:01:39.837656+0000","last_fullsized":"2024-09-19T06:01:39.837656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:07:06.832685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:48.860453+0000","last_change":"2024-09-19T06:01:26.786746+0000","last_active":"2024-09-19T06:01:48.860453+0000","last_peered":"2024-09-19T06:01:48.860453+0000","last_clean":"2024-09-19T06:01:48.860453+0000","last_became_active":"2024-09-19T06:01:26.786414+0000","last_became_peered":"2024-09-19T06:01:26.786414+0000","last_unstale":"2024-09-19T06:01:48.860453+0000","last_undegraded":"2024-09-19T06:01:48.860453+0000","last_fullsized":"2024-09-19T06:01:48.860453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:25.042796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837014+0000","last_change":"2024-09-19T06:01:28.807183+0000","last_active":"2024-09-19T06:01:39.837014+0000","last_peered":"2024-09-19T06:01:39.837014+0000","last_clean":"2024-09-19T06:01:39.837014+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:01:39.837014+0000","last_undegraded":"2024-09-19T06:01:39.837014+0000","last_fullsized":"2024-09-19T06:01:39.837014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:23.030312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847960+0000","last_change":"2024-09-19T06:01:27.022409+0000","last_active":"2024-09-19T06:01:40.847960+0000","last_peered":"2024-09-19T06:01:40.847960+0000","last_clean":"2024-09-19T06:01:40.847960+0000","last_became_active":"2024-09-19T06:01:27.022034+0000","last_became_peered":"2024-09-19T06:01:27.022034+0000","last_unstale":"2024-09-19T06:01:40.847960+0000","last_undegraded":"2024-09-19T06:01:40.847960+0000","last_fullsized":"2024-09-19T06:01:40.847960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:08.663575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832674+0000","last_change":"2024-09-19T06:01:28.812079+0000","last_active":"2024-09-19T06:01:39.832674+0000","last_peered":"2024-09-19T06:01:39.832674+0000","last_clean":"2024-09-19T06:01:39.832674+0000","last_became_active":"2024-09-19T06:01:28.811606+0000","last_became_peered":"2024-09-19T06:01:28.811606+0000","last_unstale":"2024-09-19T06:01:39.832674+0000","last_undegraded":"2024-09-19T06:01:39.832674+0000","last_fullsized":"2024-09-19T06:01:39.832674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:49.434235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.838741+0000","last_change":"2024-09-19T06:01:26.783823+0000","last_active":"2024-09-19T06:01:39.838741+0000","last_peered":"2024-09-19T06:01:39.838741+0000","last_clean":"2024-09-19T06:01:39.838741+0000","last_became_active":"2024-09-19T06:01:26.783209+0000","last_became_peered":"2024-09-19T06:01:26.783209+0000","last_unstale":"2024-09-19T06:01:39.838741+0000","last_undegraded":"2024-09-19T06:01:39.838741+0000","last_fullsized":"2024-09-19T06:01:39.838741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:38.833417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837462+0000","last_change":"2024-09-19T06:01:39.837462+0000","last_active":"2024-09-19T06:01:39.837462+0000","last_peered":"2024-09-19T06:01:39.837462+0000","last_clean":"2024-09-19T06:01:39.837462+0000","last_became_active":"2024-09-19T06:01:39.837082+0000","last_became_peered":"2024-09-19T06:01:39.837082+0000","last_unstale":"2024-09-19T06:01:39.837462+0000","last_undegraded":"2024-09-19T06:01:39.837462+0000","last_fullsized":"2024-09-19T06:01:39.837462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:47.421017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.845150+0000","last_change":"2024-09-19T06:01:26.779992+0000","last_active":"2024-09-19T06:01:40.845150+0000","last_peered":"2024-09-19T06:01:40.845150+0000","last_clean":"2024-09-19T06:01:40.845150+0000","last_became_active":"2024-09-19T06:01:26.779609+0000","last_became_peered":"2024-09-19T06:01:26.779609+0000","last_unstale":"2024-09-19T06:01:40.845150+0000","last_undegraded":"2024-09-19T06:01:40.845150+0000","last_fullsized":"2024-09-19T06:01:40.845150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:48.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837947+0000","last_change":"2024-09-19T06:01:28.798827+0000","last_active":"2024-09-19T06:01:39.837947+0000","last_peered":"2024-09-19T06:01:39.837947+0000","last_clean":"2024-09-19T06:01:39.837947+0000","last_became_active":"2024-09-19T06:01:28.798457+0000","last_became_peered":"2024-09-19T06:01:28.798457+0000","last_unstale":"2024-09-19T06:01:39.837947+0000","last_undegraded":"2024-09-19T06:01:39.837947+0000","last_fullsized":"2024-09-19T06:01:39.837947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:41.111051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.846954+0000","last_change":"2024-09-19T06:01:26.775691+0000","last_active":"2024-09-19T06:01:40.846954+0000","last_peered":"2024-09-19T06:01:40.846954+0000","last_clean":"2024-09-19T06:01:40.846954+0000","last_became_active":"2024-09-19T06:01:26.775183+0000","last_became_peered":"2024-09-19T06:01:26.775183+0000","last_unstale":"2024-09-19T06:01:40.846954+0000","last_undegraded":"2024-09-19T06:01:40.846954+0000","last_fullsized":"2024-09-19T06:01:40.846954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:34.712968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837674+0000","last_change":"2024-09-19T06:01:28.809459+0000","last_active":"2024-09-19T06:01:39.837674+0000","last_peered":"2024-09-19T06:01:39.837674+0000","last_clean":"2024-09-19T06:01:39.837674+0000","last_became_active":"2024-09-19T06:01:28.809046+0000","last_became_peered":"2024-09-19T06:01:28.809046+0000","last_unstale":"2024-09-19T06:01:39.837674+0000","last_undegraded":"2024-09-19T06:01:39.837674+0000","last_fullsized":"2024-09-19T06:01:39.837674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:56.996190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:47.858815+0000","last_change":"2024-09-19T06:01:39.843932+0000","last_active":"2024-09-19T06:01:47.858815+0000","last_peered":"2024-09-19T06:01:47.858815+0000","last_clean":"2024-09-19T06:01:47.858815+0000","last_became_active":"2024-09-19T06:01:39.836008+0000","last_became_peered":"2024-09-19T06:01:39.836008+0000","last_unstale":"2024-09-19T06:01:47.858815+0000","last_undegraded":"2024-09-19T06:01:47.858815+0000","last_fullsized":"2024-09-19T06:01:47.858815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:58.798859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837726+0000","last_change":"2024-09-19T06:01:28.814181+0000","last_active":"2024-09-19T06:01:39.837726+0000","last_peered":"2024-09-19T06:01:39.837726+0000","last_clean":"2024-09-19T06:01:39.837726+0000","last_became_active":"2024-09-19T06:01:28.813817+0000","last_became_peered":"2024-09-19T06:01:28.813817+0000","last_unstale":"2024-09-19T06:01:39.837726+0000","last_undegraded":"2024-09-19T06:01:39.837726+0000","last_fullsized":"2024-09-19T06:01:39.837726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:40.057863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837197+0000","last_change":"2024-09-19T06:00:17.624509+0000","last_active":"2024-09-19T06:01:39.837197+0000","last_peered":"2024-09-19T06:01:39.837197+0000","last_clean":"2024-09-19T06:01:39.837197+0000","last_became_active":"2024-09-19T06:00:17.623943+0000","last_became_peered":"2024-09-19T06:00:17.623943+0000","last_unstale":"2024-09-19T06:01:39.837197+0000","last_undegraded":"2024-09-19T06:01:39.837197+0000","last_fullsized":"2024-09-19T06:01:39.837197+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_clean_scrub_stamp":"2024-09-19T05:56:55.414488+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:39.116345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:41.847767+0000","last_change":"2024-09-19T06:01:26.783896+0000","last_active":"2024-09-19T06:01:41.847767+0000","last_peered":"2024-09-19T06:01:41.847767+0000","last_clean":"2024-09-19T06:01:41.847767+0000","last_became_active":"2024-09-19T06:01:26.783332+0000","last_became_peered":"2024-09-19T06:01:26.783332+0000","last_unstale":"2024-09-19T06:01:41.847767+0000","last_undegraded":"2024-09-19T06:01:41.847767+0000","last_fullsized":"2024-09-19T06:01:41.847767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:33.072763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980113+0000","last_change":"2024-09-19T06:01:28.807756+0000","last_active":"2024-09-19T06:01:39.980113+0000","last_peered":"2024-09-19T06:01:39.980113+0000","last_clean":"2024-09-19T06:01:39.980113+0000","last_became_active":"2024-09-19T06:01:28.807077+0000","last_became_peered":"2024-09-19T06:01:28.807077+0000","last_unstale":"2024-09-19T06:01:39.980113+0000","last_undegraded":"2024-09-19T06:01:39.980113+0000","last_fullsized":"2024-09-19T06:01:39.980113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:20.784199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837194+0000","last_change":"2024-09-19T06:01:26.784830+0000","last_active":"2024-09-19T06:01:39.837194+0000","last_peered":"2024-09-19T06:01:39.837194+0000","last_clean":"2024-09-19T06:01:39.837194+0000","last_became_active":"2024-09-19T06:01:26.783887+0000","last_became_peered":"2024-09-19T06:01:26.783887+0000","last_unstale":"2024-09-19T06:01:39.837194+0000","last_undegraded":"2024-09-19T06:01:39.837194+0000","last_fullsized":"2024-09-19T06:01:39.837194+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:33:38.255169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837354+0000","last_change":"2024-09-19T06:01:39.837354+0000","last_active":"2024-09-19T06:01:39.837354+0000","last_peered":"2024-09-19T06:01:39.837354+0000","last_clean":"2024-09-19T06:01:39.837354+0000","last_became_active":"2024-09-19T06:01:39.836930+0000","last_became_peered":"2024-09-19T06:01:39.836930+0000","last_unstale":"2024-09-19T06:01:39.837354+0000","last_undegraded":"2024-09-19T06:01:39.837354+0000","last_fullsized":"2024-09-19T06:01:39.837354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:17.571834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:47.858757+0000","last_change":"2024-09-19T06:01:27.021753+0000","last_active":"2024-09-19T06:01:47.858757+0000","last_peered":"2024-09-19T06:01:47.858757+0000","last_clean":"2024-09-19T06:01:47.858757+0000","last_became_active":"2024-09-19T06:01:27.021346+0000","last_became_peered":"2024-09-19T06:01:27.021346+0000","last_unstale":"2024-09-19T06:01:47.858757+0000","last_undegraded":"2024-09-19T06:01:47.858757+0000","last_fullsized":"2024-09-19T06:01:47.858757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:07.291283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832814+0000","last_change":"2024-09-19T06:01:28.803103+0000","last_active":"2024-09-19T06:01:39.832814+0000","last_peered":"2024-09-19T06:01:39.832814+0000","last_clean":"2024-09-19T06:01:39.832814+0000","last_became_active":"2024-09-19T06:01:28.802806+0000","last_became_peered":"2024-09-19T06:01:28.802806+0000","last_unstale":"2024-09-19T06:01:39.832814+0000","last_undegraded":"2024-09-19T06:01:39.832814+0000","last_fullsized":"2024-09-19T06:01:39.832814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:33.329231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:47.858456+0000","last_change":"2024-09-19T06:01:26.786605+0000","last_active":"2024-09-19T06:01:47.858456+0000","last_peered":"2024-09-19T06:01:47.858456+0000","last_clean":"2024-09-19T06:01:47.858456+0000","last_became_active":"2024-09-19T06:01:26.786278+0000","last_became_peered":"2024-09-19T06:01:26.786278+0000","last_unstale":"2024-09-19T06:01:47.858456+0000","last_undegraded":"2024-09-19T06:01:47.858456+0000","last_fullsized":"2024-09-19T06:01:47.858456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.581502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323778+0000","last_change":"2024-09-19T06:01:28.808152+0000","last_active":"2024-09-19T06:01:40.323778+0000","last_peered":"2024-09-19T06:01:40.323778+0000","last_clean":"2024-09-19T06:01:40.323778+0000","last_became_active":"2024-09-19T06:01:28.807788+0000","last_became_peered":"2024-09-19T06:01:28.807788+0000","last_unstale":"2024-09-19T06:01:40.323778+0000","last_undegraded":"2024-09-19T06:01:40.323778+0000","last_fullsized":"2024-09-19T06:01:40.323778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:42.877539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.836049+0000","last_change":"2024-09-19T06:01:27.022719+0000","last_active":"2024-09-19T06:01:39.836049+0000","last_peered":"2024-09-19T06:01:39.836049+0000","last_clean":"2024-09-19T06:01:39.836049+0000","last_became_active":"2024-09-19T06:01:27.022263+0000","last_became_peered":"2024-09-19T06:01:27.022263+0000","last_unstale":"2024-09-19T06:01:39.836049+0000","last_undegraded":"2024-09-19T06:01:39.836049+0000","last_fullsized":"2024-09-19T06:01:39.836049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:45.158229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323926+0000","last_change":"2024-09-19T06:01:28.809869+0000","last_active":"2024-09-19T06:01:40.323926+0000","last_peered":"2024-09-19T06:01:40.323926+0000","last_clean":"2024-09-19T06:01:40.323926+0000","last_became_active":"2024-09-19T06:01:28.809548+0000","last_became_peered":"2024-09-19T06:01:28.809548+0000","last_unstale":"2024-09-19T06:01:40.323926+0000","last_undegraded":"2024-09-19T06:01:40.323926+0000","last_fullsized":"2024-09-19T06:01:40.323926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:41.587846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847043+0000","last_change":"2024-09-19T06:01:39.846740+0000","last_active":"2024-09-19T06:01:40.847043+0000","last_peered":"2024-09-19T06:01:40.847043+0000","last_clean":"2024-09-19T06:01:40.847043+0000","last_became_active":"2024-09-19T06:01:39.836378+0000","last_became_peered":"2024-09-19T06:01:39.836378+0000","last_unstale":"2024-09-19T06:01:40.847043+0000","last_undegraded":"2024-09-19T06:01:40.847043+0000","last_fullsized":"2024-09-19T06:01:40.847043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.064918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829305+0000","last_change":"2024-09-19T06:01:28.807796+0000","last_active":"2024-09-19T06:01:39.829305+0000","last_peered":"2024-09-19T06:01:39.829305+0000","last_clean":"2024-09-19T06:01:39.829305+0000","last_became_active":"2024-09-19T06:01:28.807503+0000","last_became_peered":"2024-09-19T06:01:28.807503+0000","last_unstale":"2024-09-19T06:01:39.829305+0000","last_undegraded":"2024-09-19T06:01:39.829305+0000","last_fullsized":"2024-09-19T06:01:39.829305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:43.472647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'10","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:44.793783+0000","last_change":"2024-09-19T06:01:26.783517+0000","last_active":"2024-09-19T06:01:44.793783+0000","last_peered":"2024-09-19T06:01:44.793783+0000","last_clean":"2024-09-19T06:01:44.793783+0000","last_became_active":"2024-09-19T06:01:26.783164+0000","last_became_peered":"2024-09-19T06:01:26.783164+0000","last_unstale":"2024-09-19T06:01:44.793783+0000","last_undegraded":"2024-09-19T06:01:44.793783+0000","last_fullsized":"2024-09-19T06:01:44.793783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:57.609192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980454+0000","last_change":"2024-09-19T06:01:28.806548+0000","last_active":"2024-09-19T06:01:39.980454+0000","last_peered":"2024-09-19T06:01:39.980454+0000","last_clean":"2024-09-19T06:01:39.980454+0000","last_became_active":"2024-09-19T06:01:28.805724+0000","last_became_peered":"2024-09-19T06:01:28.805724+0000","last_unstale":"2024-09-19T06:01:39.980454+0000","last_undegraded":"2024-09-19T06:01:39.980454+0000","last_fullsized":"2024-09-19T06:01:39.980454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:14.360928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847547+0000","last_change":"2024-09-19T06:01:27.128213+0000","last_active":"2024-09-19T06:01:40.847547+0000","last_peered":"2024-09-19T06:01:40.847547+0000","last_clean":"2024-09-19T06:01:40.847547+0000","last_became_active":"2024-09-19T06:01:27.127737+0000","last_became_peered":"2024-09-19T06:01:27.127737+0000","last_unstale":"2024-09-19T06:01:40.847547+0000","last_undegraded":"2024-09-19T06:01:40.847547+0000","last_fullsized":"2024-09-19T06:01:40.847547+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:33:35.107152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.839027+0000","last_change":"2024-09-19T06:01:39.839027+0000","last_active":"2024-09-19T06:01:39.839027+0000","last_peered":"2024-09-19T06:01:39.839027+0000","last_clean":"2024-09-19T06:01:39.839027+0000","last_became_active":"2024-09-19T06:01:39.838601+0000","last_became_peered":"2024-09-19T06:01:39.838601+0000","last_unstale":"2024-09-19T06:01:39.839027+0000","last_undegraded":"2024-09-19T06:01:39.839027+0000","last_fullsized":"2024-09-19T06:01:39.839027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:09.864997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832051+0000","last_change":"2024-09-19T06:01:27.022438+0000","last_active":"2024-09-19T06:01:39.832051+0000","last_peered":"2024-09-19T06:01:39.832051+0000","last_clean":"2024-09-19T06:01:39.832051+0000","last_became_active":"2024-09-19T06:01:27.021953+0000","last_became_peered":"2024-09-19T06:01:27.021953+0000","last_unstale":"2024-09-19T06:01:39.832051+0000","last_undegraded":"2024-09-19T06:01:39.832051+0000","last_fullsized":"2024-09-19T06:01:39.832051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:24.383178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837088+0000","last_change":"2024-09-19T06:01:28.803807+0000","last_active":"2024-09-19T06:01:39.837088+0000","last_peered":"2024-09-19T06:01:39.837088+0000","last_clean":"2024-09-19T06:01:39.837088+0000","last_became_active":"2024-09-19T06:01:28.803486+0000","last_became_peered":"2024-09-19T06:01:28.803486+0000","last_unstale":"2024-09-19T06:01:39.837088+0000","last_undegraded":"2024-09-19T06:01:39.837088+0000","last_fullsized":"2024-09-19T06:01:39.837088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:26:17.920089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.845539+0000","last_change":"2024-09-19T06:01:26.777171+0000","last_active":"2024-09-19T06:01:40.845539+0000","last_peered":"2024-09-19T06:01:40.845539+0000","last_clean":"2024-09-19T06:01:40.845539+0000","last_became_active":"2024-09-19T06:01:26.776818+0000","last_became_peered":"2024-09-19T06:01:26.776818+0000","last_unstale":"2024-09-19T06:01:40.845539+0000","last_undegraded":"2024-09-19T06:01:40.845539+0000","last_fullsized":"2024-09-19T06:01:40.845539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832532+0000","last_change":"2024-09-19T06:01:28.799266+0000","last_active":"2024-09-19T06:01:39.832532+0000","last_peered":"2024-09-19T06:01:39.832532+0000","last_clean":"2024-09-19T06:01:39.832532+0000","last_became_active":"2024-09-19T06:01:28.798969+0000","last_became_peered":"2024-09-19T06:01:28.798969+0000","last_unstale":"2024-09-19T06:01:39.832532+0000","last_undegraded":"2024-09-19T06:01:39.832532+0000","last_fullsized":"2024-09-19T06:01:39.832532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:34.064795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830288+0000","last_change":"2024-09-19T06:01:26.781680+0000","last_active":"2024-09-19T06:01:39.830288+0000","last_peered":"2024-09-19T06:01:39.830288+0000","last_clean":"2024-09-19T06:01:39.830288+0000","last_became_active":"2024-09-19T06:01:26.781035+0000","last_became_peered":"2024-09-19T06:01:26.781035+0000","last_unstale":"2024-09-19T06:01:39.830288+0000","last_undegraded":"2024-09-19T06:01:39.830288+0000","last_fullsized":"2024-09-19T06:01:39.830288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:38.287018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980152+0000","last_change":"2024-09-19T06:01:28.802970+0000","last_active":"2024-09-19T06:01:39.980152+0000","last_peered":"2024-09-19T06:01:39.980152+0000","last_clean":"2024-09-19T06:01:39.980152+0000","last_became_active":"2024-09-19T06:01:28.802587+0000","last_became_peered":"2024-09-19T06:01:28.802587+0000","last_unstale":"2024-09-19T06:01:39.980152+0000","last_undegraded":"2024-09-19T06:01:39.980152+0000","last_fullsized":"2024-09-19T06:01:39.980152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837492+0000","last_change":"2024-09-19T06:01:26.788140+0000","last_active":"2024-09-19T06:01:39.837492+0000","last_peered":"2024-09-19T06:01:39.837492+0000","last_clean":"2024-09-19T06:01:39.837492+0000","last_became_active":"2024-09-19T06:01:26.787603+0000","last_became_peered":"2024-09-19T06:01:26.787603+0000","last_unstale":"2024-09-19T06:01:39.837492+0000","last_undegraded":"2024-09-19T06:01:39.837492+0000","last_fullsized":"2024-09-19T06:01:39.837492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:13.880656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837369+0000","last_change":"2024-09-19T06:01:28.813144+0000","last_active":"2024-09-19T06:01:39.837369+0000","last_peered":"2024-09-19T06:01:39.837369+0000","last_clean":"2024-09-19T06:01:39.837369+0000","last_became_active":"2024-09-19T06:01:28.812792+0000","last_became_peered":"2024-09-19T06:01:28.812792+0000","last_unstale":"2024-09-19T06:01:39.837369+0000","last_undegraded":"2024-09-19T06:01:39.837369+0000","last_fullsized":"2024-09-19T06:01:39.837369+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:35.896240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833256+0000","last_change":"2024-09-19T06:01:28.812086+0000","last_active":"2024-09-19T06:01:39.833256+0000","last_peered":"2024-09-19T06:01:39.833256+0000","last_clean":"2024-09-19T06:01:39.833256+0000","last_became_active":"2024-09-19T06:01:28.811696+0000","last_became_peered":"2024-09-19T06:01:28.811696+0000","last_unstale":"2024-09-19T06:01:39.833256+0000","last_undegraded":"2024-09-19T06:01:39.833256+0000","last_fullsized":"2024-09-19T06:01:39.833256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:32.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847847+0000","last_change":"2024-09-19T06:01:39.842250+0000","last_active":"2024-09-19T06:01:40.847847+0000","last_peered":"2024-09-19T06:01:40.847847+0000","last_clean":"2024-09-19T06:01:40.847847+0000","last_became_active":"2024-09-19T06:01:39.835761+0000","last_became_peered":"2024-09-19T06:01:39.835761+0000","last_unstale":"2024-09-19T06:01:40.847847+0000","last_undegraded":"2024-09-19T06:01:40.847847+0000","last_fullsized":"2024-09-19T06:01:40.847847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:51.026349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832601+0000","last_change":"2024-09-19T06:01:28.812983+0000","last_active":"2024-09-19T06:01:39.832601+0000","last_peered":"2024-09-19T06:01:39.832601+0000","last_clean":"2024-09-19T06:01:39.832601+0000","last_became_active":"2024-09-19T06:01:28.812257+0000","last_became_peered":"2024-09-19T06:01:28.812257+0000","last_unstale":"2024-09-19T06:01:39.832601+0000","last_undegraded":"2024-09-19T06:01:39.832601+0000","last_fullsized":"2024-09-19T06:01:39.832601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:41:42.059736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:41.842632+0000","last_change":"2024-09-19T06:01:26.787869+0000","last_active":"2024-09-19T06:01:41.842632+0000","last_peered":"2024-09-19T06:01:41.842632+0000","last_clean":"2024-09-19T06:01:41.842632+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:01:41.842632+0000","last_undegraded":"2024-09-19T06:01:41.842632+0000","last_fullsized":"2024-09-19T06:01:41.842632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:02.869878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831993+0000","last_change":"2024-09-19T06:01:28.811986+0000","last_active":"2024-09-19T06:01:39.831993+0000","last_peered":"2024-09-19T06:01:39.831993+0000","last_clean":"2024-09-19T06:01:39.831993+0000","last_became_active":"2024-09-19T06:01:28.811565+0000","last_became_peered":"2024-09-19T06:01:28.811565+0000","last_unstale":"2024-09-19T06:01:39.831993+0000","last_undegraded":"2024-09-19T06:01:39.831993+0000","last_fullsized":"2024-09-19T06:01:39.831993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:01.324985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980467+0000","last_change":"2024-09-19T06:01:27.021683+0000","last_active":"2024-09-19T06:01:39.980467+0000","last_peered":"2024-09-19T06:01:39.980467+0000","last_clean":"2024-09-19T06:01:39.980467+0000","last_became_active":"2024-09-19T06:01:27.021358+0000","last_became_peered":"2024-09-19T06:01:27.021358+0000","last_unstale":"2024-09-19T06:01:39.980467+0000","last_undegraded":"2024-09-19T06:01:39.980467+0000","last_fullsized":"2024-09-19T06:01:39.980467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:29.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980328+0000","last_change":"2024-09-19T06:01:28.798963+0000","last_active":"2024-09-19T06:01:39.980328+0000","last_peered":"2024-09-19T06:01:39.980328+0000","last_clean":"2024-09-19T06:01:39.980328+0000","last_became_active":"2024-09-19T06:01:28.797571+0000","last_became_peered":"2024-09-19T06:01:28.797571+0000","last_unstale":"2024-09-19T06:01:39.980328+0000","last_undegraded":"2024-09-19T06:01:39.980328+0000","last_fullsized":"2024-09-19T06:01:39.980328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:43.867609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:47.857925+0000","last_change":"2024-09-19T06:01:26.786427+0000","last_active":"2024-09-19T06:01:47.857925+0000","last_peered":"2024-09-19T06:01:47.857925+0000","last_clean":"2024-09-19T06:01:47.857925+0000","last_became_active":"2024-09-19T06:01:26.786092+0000","last_became_peered":"2024-09-19T06:01:26.786092+0000","last_unstale":"2024-09-19T06:01:47.857925+0000","last_undegraded":"2024-09-19T06:01:47.857925+0000","last_fullsized":"2024-09-19T06:01:47.857925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:49.144366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54796,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":119,"num_read_kb":116,"num_write":229,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":158651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547216,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":439275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1593344,"data_stored":1070675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939675,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":438118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":437802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397540,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1031802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561064,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789998,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1030694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953524,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":439104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":437660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40448,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704704,"statfs":{"total":95995035648,"available":95953616896,"internally_reserved":0,"allocated":966656,"data_stored":478241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40452,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704700,"statfs":{"total":95995035648,"available":95953612800,"internally_reserved":0,"allocated":970752,"data_stored":481769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":439567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":50510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":41245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:01:54.791 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:01:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:55.428 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:55 smithi005 ceph-mon[30955]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:01:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.509+0000 7ff267347640 1 -- 172.21.15.5:0/83661152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260103720 msgr2=0x7ff26010bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.509+0000 7ff267347640 1 --2- 172.21.15.5:0/83661152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260103720 0x7ff26010bc10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff250009900 tx=0x7ff25002a480 comp rx=0 tx=0).stop 2024-09-19T06:01:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.509+0000 7ff267347640 1 -- 172.21.15.5:0/83661152 shutdown_connections 2024-09-19T06:01:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.509+0000 7ff267347640 1 --2- 172.21.15.5:0/83661152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26010c1a0 0x7ff26010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.509+0000 7ff267347640 1 --2- 172.21.15.5:0/83661152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260103720 0x7ff26010bc10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.509+0000 7ff267347640 1 --2- 172.21.15.5:0/83661152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff260102e00 0x7ff2601031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.509+0000 7ff267347640 1 -- 172.21.15.5:0/83661152 >> 172.21.15.5:0/83661152 conn(0x7ff2600fc7a0 msgr2=0x7ff2600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.509+0000 7ff267347640 1 -- 172.21.15.5:0/83661152 shutdown_connections 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.509+0000 7ff267347640 1 -- 172.21.15.5:0/83661152 wait complete. 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff267347640 1 Processor -- start 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff267347640 1 -- start start 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff267347640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260102e00 0x7ff260198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff267347640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff260103720 0x7ff260198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff267347640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26010c1a0 0x7ff26019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff267347640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff260111e60 con 0x7ff260102e00 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff267347640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff260111ce0 con 0x7ff26010c1a0 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff267347640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff260111fe0 con 0x7ff260103720 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff2650e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260102e00 0x7ff260198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff2650e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260102e00 0x7ff260198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54914/0 (socket says 172.21.15.5:54914) 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff2650e3640 1 -- 172.21.15.5:0/4105386902 learned_addr learned my addr 172.21.15.5:0/4105386902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff2658e4640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26010c1a0 0x7ff26019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff2648e2640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff260103720 0x7ff260198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff2650e3640 1 -- 172.21.15.5:0/4105386902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff260103720 msgr2=0x7ff260198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff2650e3640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff260103720 0x7ff260198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff2650e3640 1 -- 172.21.15.5:0/4105386902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26010c1a0 msgr2=0x7ff26019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff2650e3640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26010c1a0 0x7ff26019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff2650e3640 1 -- 172.21.15.5:0/4105386902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26019d3d0 con 0x7ff260102e00 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.510+0000 7ff2658e4640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26010c1a0 0x7ff26019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.511+0000 7ff2650e3640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260102e00 0x7ff260198420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00b4a0 tx=0x7ff25c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.511+0000 7ff2567fc640 1 -- 172.21.15.5:0/4105386902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c014070 con 0x7ff260102e00 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.511+0000 7ff2567fc640 1 -- 172.21.15.5:0/4105386902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff25c010ac0 con 0x7ff260102e00 2024-09-19T06:01:55.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.511+0000 7ff2567fc640 1 -- 172.21.15.5:0/4105386902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c017c30 con 0x7ff260102e00 2024-09-19T06:01:55.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.511+0000 7ff267347640 1 -- 172.21.15.5:0/4105386902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26019d6c0 con 0x7ff260102e00 2024-09-19T06:01:55.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.511+0000 7ff267347640 1 -- 172.21.15.5:0/4105386902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2601a4fa0 con 0x7ff260102e00 2024-09-19T06:01:55.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.512+0000 7ff267347640 1 -- 172.21.15.5:0/4105386902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2601031e0 con 0x7ff260102e00 2024-09-19T06:01:55.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.513+0000 7ff2567fc640 1 -- 172.21.15.5:0/4105386902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7ff25c017070 con 0x7ff260102e00 2024-09-19T06:01:55.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.513+0000 7ff2567fc640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff24c0a32d0 0x7ff24c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:55.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.514+0000 7ff2567fc640 1 -- 172.21.15.5:0/4105386902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff25c0cf120 con 0x7ff260102e00 2024-09-19T06:01:55.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.514+0000 7ff2648e2640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff24c0a32d0 0x7ff24c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:55.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.514+0000 7ff2648e2640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff24c0a32d0 0x7ff24c0a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff250006010 tx=0x7ff250000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:55.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.518+0000 7ff2567fc640 1 -- 172.21.15.5:0/4105386902 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff25c09bde0 con 0x7ff260102e00 2024-09-19T06:01:55.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:55 smithi157 ceph-mon[35346]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:01:55.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:55 smithi049 ceph-mon[34640]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:01:55.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.690+0000 7ff267347640 1 -- 172.21.15.5:0/4105386902 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff260062630 con 0x7ff24c0a32d0 2024-09-19T06:01:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.697+0000 7ff2567fc640 1 -- 172.21.15.5:0/4105386902 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258760 (secure 0 0 0) 0x7ff260062630 con 0x7ff24c0a32d0 2024-09-19T06:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-19T06:01:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.700+0000 7ff267347640 1 -- 172.21.15.5:0/4105386902 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff24c0a32d0 msgr2=0x7ff24c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.700+0000 7ff267347640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff24c0a32d0 0x7ff24c0a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff250006010 tx=0x7ff250000bc0 comp rx=0 tx=0).stop 2024-09-19T06:01:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.700+0000 7ff267347640 1 -- 172.21.15.5:0/4105386902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260102e00 msgr2=0x7ff260198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.700+0000 7ff267347640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260102e00 0x7ff260198420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00b4a0 tx=0x7ff25c00b960 comp rx=0 tx=0).stop 2024-09-19T06:01:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.700+0000 7ff267347640 1 -- 172.21.15.5:0/4105386902 shutdown_connections 2024-09-19T06:01:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.700+0000 7ff267347640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff24c0a32d0 0x7ff24c0a5790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.700+0000 7ff267347640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26010c1a0 0x7ff26019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.700+0000 7ff267347640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff260103720 0x7ff260198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.700+0000 7ff267347640 1 --2- 172.21.15.5:0/4105386902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260102e00 0x7ff260198420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.700+0000 7ff267347640 1 -- 172.21.15.5:0/4105386902 >> 172.21.15.5:0/4105386902 conn(0x7ff2600fc7a0 msgr2=0x7ff2600fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.700+0000 7ff267347640 1 -- 172.21.15.5:0/4105386902 shutdown_connections 2024-09-19T06:01:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:55.700+0000 7ff267347640 1 -- 172.21.15.5:0/4105386902 wait complete. 2024-09-19T06:01:56.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:56 smithi005 ceph-mon[30955]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 21 op/s; 102 B/s, 0 objects/s recovering 2024-09-19T06:01:56.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:56 smithi005 ceph-mon[30955]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:01:56.512 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-19T06:01:54.789524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653041,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":301,"num_read_kb":266,"num_write":422,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387064,"kb_used_data":12840,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554760,"statfs":{"total":1151940427776,"available":1151544074240,"internally_reserved":0,"allocated":13148160,"data_stored":7180146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":57339,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":-2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":165,"num_read_kb":147,"num_write":88,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003113"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833240+0000","last_change":"2024-09-19T06:01:28.812722+0000","last_active":"2024-09-19T06:01:39.833240+0000","last_peered":"2024-09-19T06:01:39.833240+0000","last_clean":"2024-09-19T06:01:39.833240+0000","last_became_active":"2024-09-19T06:01:28.812283+0000","last_became_peered":"2024-09-19T06:01:28.812283+0000","last_unstale":"2024-09-19T06:01:39.833240+0000","last_undegraded":"2024-09-19T06:01:39.833240+0000","last_fullsized":"2024-09-19T06:01:39.833240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:59.146636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831968+0000","last_change":"2024-09-19T06:01:28.805210+0000","last_active":"2024-09-19T06:01:39.831968+0000","last_peered":"2024-09-19T06:01:39.831968+0000","last_clean":"2024-09-19T06:01:39.831968+0000","last_became_active":"2024-09-19T06:01:28.804906+0000","last_became_peered":"2024-09-19T06:01:28.804906+0000","last_unstale":"2024-09-19T06:01:39.831968+0000","last_undegraded":"2024-09-19T06:01:39.831968+0000","last_fullsized":"2024-09-19T06:01:39.831968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:00.227101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832025+0000","last_change":"2024-09-19T06:01:28.811909+0000","last_active":"2024-09-19T06:01:39.832025+0000","last_peered":"2024-09-19T06:01:39.832025+0000","last_clean":"2024-09-19T06:01:39.832025+0000","last_became_active":"2024-09-19T06:01:28.811547+0000","last_became_peered":"2024-09-19T06:01:28.811547+0000","last_unstale":"2024-09-19T06:01:39.832025+0000","last_undegraded":"2024-09-19T06:01:39.832025+0000","last_fullsized":"2024-09-19T06:01:39.832025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:50.022439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830196+0000","last_change":"2024-09-19T06:01:28.806782+0000","last_active":"2024-09-19T06:01:39.830196+0000","last_peered":"2024-09-19T06:01:39.830196+0000","last_clean":"2024-09-19T06:01:39.830196+0000","last_became_active":"2024-09-19T06:01:28.805520+0000","last_became_peered":"2024-09-19T06:01:28.805520+0000","last_unstale":"2024-09-19T06:01:39.830196+0000","last_undegraded":"2024-09-19T06:01:39.830196+0000","last_fullsized":"2024-09-19T06:01:39.830196+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:01.652669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837308+0000","last_change":"2024-09-19T06:01:28.809971+0000","last_active":"2024-09-19T06:01:39.837308+0000","last_peered":"2024-09-19T06:01:39.837308+0000","last_clean":"2024-09-19T06:01:39.837308+0000","last_became_active":"2024-09-19T06:01:28.809593+0000","last_became_peered":"2024-09-19T06:01:28.809593+0000","last_unstale":"2024-09-19T06:01:39.837308+0000","last_undegraded":"2024-09-19T06:01:39.837308+0000","last_fullsized":"2024-09-19T06:01:39.837308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.546526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323650+0000","last_change":"2024-09-19T06:01:28.801395+0000","last_active":"2024-09-19T06:01:40.323650+0000","last_peered":"2024-09-19T06:01:40.323650+0000","last_clean":"2024-09-19T06:01:40.323650+0000","last_became_active":"2024-09-19T06:01:28.800987+0000","last_became_peered":"2024-09-19T06:01:28.800987+0000","last_unstale":"2024-09-19T06:01:40.323650+0000","last_undegraded":"2024-09-19T06:01:40.323650+0000","last_fullsized":"2024-09-19T06:01:40.323650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:18.986489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837262+0000","last_change":"2024-09-19T06:01:28.806554+0000","last_active":"2024-09-19T06:01:39.837262+0000","last_peered":"2024-09-19T06:01:39.837262+0000","last_clean":"2024-09-19T06:01:39.837262+0000","last_became_active":"2024-09-19T06:01:28.805946+0000","last_became_peered":"2024-09-19T06:01:28.805946+0000","last_unstale":"2024-09-19T06:01:39.837262+0000","last_undegraded":"2024-09-19T06:01:39.837262+0000","last_fullsized":"2024-09-19T06:01:39.837262+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:56:20.130414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.979824+0000","last_change":"2024-09-19T06:01:28.804789+0000","last_active":"2024-09-19T06:01:39.979824+0000","last_peered":"2024-09-19T06:01:39.979824+0000","last_clean":"2024-09-19T06:01:39.979824+0000","last_became_active":"2024-09-19T06:01:28.804484+0000","last_became_peered":"2024-09-19T06:01:28.804484+0000","last_unstale":"2024-09-19T06:01:39.979824+0000","last_undegraded":"2024-09-19T06:01:39.979824+0000","last_fullsized":"2024-09-19T06:01:39.979824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:19.737928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832749+0000","last_change":"2024-09-19T06:01:28.814504+0000","last_active":"2024-09-19T06:01:39.832749+0000","last_peered":"2024-09-19T06:01:39.832749+0000","last_clean":"2024-09-19T06:01:39.832749+0000","last_became_active":"2024-09-19T06:01:28.814066+0000","last_became_peered":"2024-09-19T06:01:28.814066+0000","last_unstale":"2024-09-19T06:01:39.832749+0000","last_undegraded":"2024-09-19T06:01:39.832749+0000","last_fullsized":"2024-09-19T06:01:39.832749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:50.904164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.324053+0000","last_change":"2024-09-19T06:01:28.807303+0000","last_active":"2024-09-19T06:01:40.324053+0000","last_peered":"2024-09-19T06:01:40.324053+0000","last_clean":"2024-09-19T06:01:40.324053+0000","last_became_active":"2024-09-19T06:01:28.806978+0000","last_became_peered":"2024-09-19T06:01:28.806978+0000","last_unstale":"2024-09-19T06:01:40.324053+0000","last_undegraded":"2024-09-19T06:01:40.324053+0000","last_fullsized":"2024-09-19T06:01:40.324053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:33.259261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837723+0000","last_change":"2024-09-19T06:01:28.814844+0000","last_active":"2024-09-19T06:01:39.837723+0000","last_peered":"2024-09-19T06:01:39.837723+0000","last_clean":"2024-09-19T06:01:39.837723+0000","last_became_active":"2024-09-19T06:01:28.814531+0000","last_became_peered":"2024-09-19T06:01:28.814531+0000","last_unstale":"2024-09-19T06:01:39.837723+0000","last_undegraded":"2024-09-19T06:01:39.837723+0000","last_fullsized":"2024-09-19T06:01:39.837723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:17.209890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830253+0000","last_change":"2024-09-19T06:01:28.806378+0000","last_active":"2024-09-19T06:01:39.830253+0000","last_peered":"2024-09-19T06:01:39.830253+0000","last_clean":"2024-09-19T06:01:39.830253+0000","last_became_active":"2024-09-19T06:01:28.805053+0000","last_became_peered":"2024-09-19T06:01:28.805053+0000","last_unstale":"2024-09-19T06:01:39.830253+0000","last_undegraded":"2024-09-19T06:01:39.830253+0000","last_fullsized":"2024-09-19T06:01:39.830253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:15.080548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829093+0000","last_change":"2024-09-19T06:01:28.802941+0000","last_active":"2024-09-19T06:01:39.829093+0000","last_peered":"2024-09-19T06:01:39.829093+0000","last_clean":"2024-09-19T06:01:39.829093+0000","last_became_active":"2024-09-19T06:01:28.802590+0000","last_became_peered":"2024-09-19T06:01:28.802590+0000","last_unstale":"2024-09-19T06:01:39.829093+0000","last_undegraded":"2024-09-19T06:01:39.829093+0000","last_fullsized":"2024-09-19T06:01:39.829093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:25.581854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831644+0000","last_change":"2024-09-19T06:01:28.800960+0000","last_active":"2024-09-19T06:01:39.831644+0000","last_peered":"2024-09-19T06:01:39.831644+0000","last_clean":"2024-09-19T06:01:39.831644+0000","last_became_active":"2024-09-19T06:01:28.800484+0000","last_became_peered":"2024-09-19T06:01:28.800484+0000","last_unstale":"2024-09-19T06:01:39.831644+0000","last_undegraded":"2024-09-19T06:01:39.831644+0000","last_fullsized":"2024-09-19T06:01:39.831644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:08.506105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831592+0000","last_change":"2024-09-19T06:01:28.800771+0000","last_active":"2024-09-19T06:01:39.831592+0000","last_peered":"2024-09-19T06:01:39.831592+0000","last_clean":"2024-09-19T06:01:39.831592+0000","last_became_active":"2024-09-19T06:01:28.800427+0000","last_became_peered":"2024-09-19T06:01:28.800427+0000","last_unstale":"2024-09-19T06:01:39.831592+0000","last_undegraded":"2024-09-19T06:01:39.831592+0000","last_fullsized":"2024-09-19T06:01:39.831592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:19.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831899+0000","last_change":"2024-09-19T06:01:28.815528+0000","last_active":"2024-09-19T06:01:39.831899+0000","last_peered":"2024-09-19T06:01:39.831899+0000","last_clean":"2024-09-19T06:01:39.831899+0000","last_became_active":"2024-09-19T06:01:28.815129+0000","last_became_peered":"2024-09-19T06:01:28.815129+0000","last_unstale":"2024-09-19T06:01:39.831899+0000","last_undegraded":"2024-09-19T06:01:39.831899+0000","last_fullsized":"2024-09-19T06:01:39.831899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:01.936711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833355+0000","last_change":"2024-09-19T06:01:28.816120+0000","last_active":"2024-09-19T06:01:39.833355+0000","last_peered":"2024-09-19T06:01:39.833355+0000","last_clean":"2024-09-19T06:01:39.833355+0000","last_became_active":"2024-09-19T06:01:28.815822+0000","last_became_peered":"2024-09-19T06:01:28.815822+0000","last_unstale":"2024-09-19T06:01:39.833355+0000","last_undegraded":"2024-09-19T06:01:39.833355+0000","last_fullsized":"2024-09-19T06:01:39.833355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:14:58.670452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.836100+0000","last_change":"2024-09-19T06:01:28.809825+0000","last_active":"2024-09-19T06:01:39.836100+0000","last_peered":"2024-09-19T06:01:39.836100+0000","last_clean":"2024-09-19T06:01:39.836100+0000","last_became_active":"2024-09-19T06:01:28.809530+0000","last_became_peered":"2024-09-19T06:01:28.809530+0000","last_unstale":"2024-09-19T06:01:39.836100+0000","last_undegraded":"2024-09-19T06:01:39.836100+0000","last_fullsized":"2024-09-19T06:01:39.836100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:49:23.442436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830644+0000","last_change":"2024-09-19T06:01:28.801297+0000","last_active":"2024-09-19T06:01:39.830644+0000","last_peered":"2024-09-19T06:01:39.830644+0000","last_clean":"2024-09-19T06:01:39.830644+0000","last_became_active":"2024-09-19T06:01:28.800984+0000","last_became_peered":"2024-09-19T06:01:28.800984+0000","last_unstale":"2024-09-19T06:01:39.830644+0000","last_undegraded":"2024-09-19T06:01:39.830644+0000","last_fullsized":"2024-09-19T06:01:39.830644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:01.485951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832407+0000","last_change":"2024-09-19T06:01:28.804567+0000","last_active":"2024-09-19T06:01:39.832407+0000","last_peered":"2024-09-19T06:01:39.832407+0000","last_clean":"2024-09-19T06:01:39.832407+0000","last_became_active":"2024-09-19T06:01:28.804213+0000","last_became_peered":"2024-09-19T06:01:28.804213+0000","last_unstale":"2024-09-19T06:01:39.832407+0000","last_undegraded":"2024-09-19T06:01:39.832407+0000","last_fullsized":"2024-09-19T06:01:39.832407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.186460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829142+0000","last_change":"2024-09-19T06:01:28.801961+0000","last_active":"2024-09-19T06:01:39.829142+0000","last_peered":"2024-09-19T06:01:39.829142+0000","last_clean":"2024-09-19T06:01:39.829142+0000","last_became_active":"2024-09-19T06:01:28.801631+0000","last_became_peered":"2024-09-19T06:01:28.801631+0000","last_unstale":"2024-09-19T06:01:39.829142+0000","last_undegraded":"2024-09-19T06:01:39.829142+0000","last_fullsized":"2024-09-19T06:01:39.829142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:15.628908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832042+0000","last_change":"2024-09-19T06:01:28.815160+0000","last_active":"2024-09-19T06:01:39.832042+0000","last_peered":"2024-09-19T06:01:39.832042+0000","last_clean":"2024-09-19T06:01:39.832042+0000","last_became_active":"2024-09-19T06:01:28.813329+0000","last_became_peered":"2024-09-19T06:01:28.813329+0000","last_unstale":"2024-09-19T06:01:39.832042+0000","last_undegraded":"2024-09-19T06:01:39.832042+0000","last_fullsized":"2024-09-19T06:01:39.832042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:17.027393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.979874+0000","last_change":"2024-09-19T06:01:28.799205+0000","last_active":"2024-09-19T06:01:39.979874+0000","last_peered":"2024-09-19T06:01:39.979874+0000","last_clean":"2024-09-19T06:01:39.979874+0000","last_became_active":"2024-09-19T06:01:28.798891+0000","last_became_peered":"2024-09-19T06:01:28.798891+0000","last_unstale":"2024-09-19T06:01:39.979874+0000","last_undegraded":"2024-09-19T06:01:39.979874+0000","last_fullsized":"2024-09-19T06:01:39.979874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:41.227721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832707+0000","last_change":"2024-09-19T06:01:28.795823+0000","last_active":"2024-09-19T06:01:39.832707+0000","last_peered":"2024-09-19T06:01:39.832707+0000","last_clean":"2024-09-19T06:01:39.832707+0000","last_became_active":"2024-09-19T06:01:28.795510+0000","last_became_peered":"2024-09-19T06:01:28.795510+0000","last_unstale":"2024-09-19T06:01:39.832707+0000","last_undegraded":"2024-09-19T06:01:39.832707+0000","last_fullsized":"2024-09-19T06:01:39.832707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:54.831610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980449+0000","last_change":"2024-09-19T06:01:28.799696+0000","last_active":"2024-09-19T06:01:39.980449+0000","last_peered":"2024-09-19T06:01:39.980449+0000","last_clean":"2024-09-19T06:01:39.980449+0000","last_became_active":"2024-09-19T06:01:28.799417+0000","last_became_peered":"2024-09-19T06:01:28.799417+0000","last_unstale":"2024-09-19T06:01:39.980449+0000","last_undegraded":"2024-09-19T06:01:39.980449+0000","last_fullsized":"2024-09-19T06:01:39.980449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:02.330808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830714+0000","last_change":"2024-09-19T06:01:28.807146+0000","last_active":"2024-09-19T06:01:39.830714+0000","last_peered":"2024-09-19T06:01:39.830714+0000","last_clean":"2024-09-19T06:01:39.830714+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:01:39.830714+0000","last_undegraded":"2024-09-19T06:01:39.830714+0000","last_fullsized":"2024-09-19T06:01:39.830714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:12.308421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832819+0000","last_change":"2024-09-19T06:01:28.803555+0000","last_active":"2024-09-19T06:01:39.832819+0000","last_peered":"2024-09-19T06:01:39.832819+0000","last_clean":"2024-09-19T06:01:39.832819+0000","last_became_active":"2024-09-19T06:01:28.803242+0000","last_became_peered":"2024-09-19T06:01:28.803242+0000","last_unstale":"2024-09-19T06:01:39.832819+0000","last_undegraded":"2024-09-19T06:01:39.832819+0000","last_fullsized":"2024-09-19T06:01:39.832819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:02.281152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829217+0000","last_change":"2024-09-19T06:01:28.804378+0000","last_active":"2024-09-19T06:01:39.829217+0000","last_peered":"2024-09-19T06:01:39.829217+0000","last_clean":"2024-09-19T06:01:39.829217+0000","last_became_active":"2024-09-19T06:01:28.804099+0000","last_became_peered":"2024-09-19T06:01:28.804099+0000","last_unstale":"2024-09-19T06:01:39.829217+0000","last_undegraded":"2024-09-19T06:01:39.829217+0000","last_fullsized":"2024-09-19T06:01:39.829217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:11.558384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831784+0000","last_change":"2024-09-19T06:01:28.799999+0000","last_active":"2024-09-19T06:01:39.831784+0000","last_peered":"2024-09-19T06:01:39.831784+0000","last_clean":"2024-09-19T06:01:39.831784+0000","last_became_active":"2024-09-19T06:01:28.799645+0000","last_became_peered":"2024-09-19T06:01:28.799645+0000","last_unstale":"2024-09-19T06:01:39.831784+0000","last_undegraded":"2024-09-19T06:01:39.831784+0000","last_fullsized":"2024-09-19T06:01:39.831784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:20.442119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830148+0000","last_change":"2024-09-19T06:01:28.800143+0000","last_active":"2024-09-19T06:01:39.830148+0000","last_peered":"2024-09-19T06:01:39.830148+0000","last_clean":"2024-09-19T06:01:39.830148+0000","last_became_active":"2024-09-19T06:01:28.799830+0000","last_became_peered":"2024-09-19T06:01:28.799830+0000","last_unstale":"2024-09-19T06:01:39.830148+0000","last_undegraded":"2024-09-19T06:01:39.830148+0000","last_fullsized":"2024-09-19T06:01:39.830148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:43.824740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830661+0000","last_change":"2024-09-19T06:01:28.806130+0000","last_active":"2024-09-19T06:01:39.830661+0000","last_peered":"2024-09-19T06:01:39.830661+0000","last_clean":"2024-09-19T06:01:39.830661+0000","last_became_active":"2024-09-19T06:01:28.805772+0000","last_became_peered":"2024-09-19T06:01:28.805772+0000","last_unstale":"2024-09-19T06:01:39.830661+0000","last_undegraded":"2024-09-19T06:01:39.830661+0000","last_fullsized":"2024-09-19T06:01:39.830661+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:00.316311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837412+0000","last_change":"2024-09-19T06:01:39.837412+0000","last_active":"2024-09-19T06:01:39.837412+0000","last_peered":"2024-09-19T06:01:39.837412+0000","last_clean":"2024-09-19T06:01:39.837412+0000","last_became_active":"2024-09-19T06:01:39.837035+0000","last_became_peered":"2024-09-19T06:01:39.837035+0000","last_unstale":"2024-09-19T06:01:39.837412+0000","last_undegraded":"2024-09-19T06:01:39.837412+0000","last_fullsized":"2024-09-19T06:01:39.837412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:00.073884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837634+0000","last_change":"2024-09-19T06:01:28.800324+0000","last_active":"2024-09-19T06:01:39.837634+0000","last_peered":"2024-09-19T06:01:39.837634+0000","last_clean":"2024-09-19T06:01:39.837634+0000","last_became_active":"2024-09-19T06:01:28.799986+0000","last_became_peered":"2024-09-19T06:01:28.799986+0000","last_unstale":"2024-09-19T06:01:39.837634+0000","last_undegraded":"2024-09-19T06:01:39.837634+0000","last_fullsized":"2024-09-19T06:01:39.837634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:41.963346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":55,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:51.865265+0000","last_change":"2024-09-19T06:01:39.844897+0000","last_active":"2024-09-19T06:01:51.865265+0000","last_peered":"2024-09-19T06:01:51.865265+0000","last_clean":"2024-09-19T06:01:51.865265+0000","last_became_active":"2024-09-19T06:01:39.838347+0000","last_became_peered":"2024-09-19T06:01:39.838347+0000","last_unstale":"2024-09-19T06:01:51.865265+0000","last_undegraded":"2024-09-19T06:01:51.865265+0000","last_fullsized":"2024-09-19T06:01:51.865265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:26.235227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.834624+0000","last_change":"2024-09-19T06:01:28.802644+0000","last_active":"2024-09-19T06:01:39.834624+0000","last_peered":"2024-09-19T06:01:39.834624+0000","last_clean":"2024-09-19T06:01:39.834624+0000","last_became_active":"2024-09-19T06:01:28.802288+0000","last_became_peered":"2024-09-19T06:01:28.802288+0000","last_unstale":"2024-09-19T06:01:39.834624+0000","last_undegraded":"2024-09-19T06:01:39.834624+0000","last_fullsized":"2024-09-19T06:01:39.834624+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:32.031860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323748+0000","last_change":"2024-09-19T06:01:26.781104+0000","last_active":"2024-09-19T06:01:40.323748+0000","last_peered":"2024-09-19T06:01:40.323748+0000","last_clean":"2024-09-19T06:01:40.323748+0000","last_became_active":"2024-09-19T06:01:26.780733+0000","last_became_peered":"2024-09-19T06:01:26.780733+0000","last_unstale":"2024-09-19T06:01:40.323748+0000","last_undegraded":"2024-09-19T06:01:40.323748+0000","last_fullsized":"2024-09-19T06:01:40.323748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:54.490867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.828967+0000","last_change":"2024-09-19T06:01:26.782901+0000","last_active":"2024-09-19T06:01:39.828967+0000","last_peered":"2024-09-19T06:01:39.828967+0000","last_clean":"2024-09-19T06:01:39.828967+0000","last_became_active":"2024-09-19T06:01:26.782450+0000","last_became_peered":"2024-09-19T06:01:26.782450+0000","last_unstale":"2024-09-19T06:01:39.828967+0000","last_undegraded":"2024-09-19T06:01:39.828967+0000","last_fullsized":"2024-09-19T06:01:39.828967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:07.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.979905+0000","last_change":"2024-09-19T06:01:28.804105+0000","last_active":"2024-09-19T06:01:39.979905+0000","last_peered":"2024-09-19T06:01:39.979905+0000","last_clean":"2024-09-19T06:01:39.979905+0000","last_became_active":"2024-09-19T06:01:28.797980+0000","last_became_peered":"2024-09-19T06:01:28.797980+0000","last_unstale":"2024-09-19T06:01:39.979905+0000","last_undegraded":"2024-09-19T06:01:39.979905+0000","last_fullsized":"2024-09-19T06:01:39.979905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:50.130102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.848777+0000","last_change":"2024-09-19T06:01:26.786923+0000","last_active":"2024-09-19T06:01:40.848777+0000","last_peered":"2024-09-19T06:01:40.848777+0000","last_clean":"2024-09-19T06:01:40.848777+0000","last_became_active":"2024-09-19T06:01:26.786542+0000","last_became_peered":"2024-09-19T06:01:26.786542+0000","last_unstale":"2024-09-19T06:01:40.848777+0000","last_undegraded":"2024-09-19T06:01:40.848777+0000","last_fullsized":"2024-09-19T06:01:40.848777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:30.032992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.835994+0000","last_change":"2024-09-19T06:01:28.802172+0000","last_active":"2024-09-19T06:01:39.835994+0000","last_peered":"2024-09-19T06:01:39.835994+0000","last_clean":"2024-09-19T06:01:39.835994+0000","last_became_active":"2024-09-19T06:01:28.801815+0000","last_became_peered":"2024-09-19T06:01:28.801815+0000","last_unstale":"2024-09-19T06:01:39.835994+0000","last_undegraded":"2024-09-19T06:01:39.835994+0000","last_fullsized":"2024-09-19T06:01:39.835994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:09.587638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:53.866134+0000","last_change":"2024-09-19T06:01:26.777870+0000","last_active":"2024-09-19T06:01:53.866134+0000","last_peered":"2024-09-19T06:01:53.866134+0000","last_clean":"2024-09-19T06:01:53.866134+0000","last_became_active":"2024-09-19T06:01:26.777509+0000","last_became_peered":"2024-09-19T06:01:26.777509+0000","last_unstale":"2024-09-19T06:01:53.866134+0000","last_undegraded":"2024-09-19T06:01:53.866134+0000","last_fullsized":"2024-09-19T06:01:53.866134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:15.885911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832446+0000","last_change":"2024-09-19T06:01:28.812201+0000","last_active":"2024-09-19T06:01:39.832446+0000","last_peered":"2024-09-19T06:01:39.832446+0000","last_clean":"2024-09-19T06:01:39.832446+0000","last_became_active":"2024-09-19T06:01:28.811801+0000","last_became_peered":"2024-09-19T06:01:28.811801+0000","last_unstale":"2024-09-19T06:01:39.832446+0000","last_undegraded":"2024-09-19T06:01:39.832446+0000","last_fullsized":"2024-09-19T06:01:39.832446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:32.904787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.848667+0000","last_change":"2024-09-19T06:01:26.787845+0000","last_active":"2024-09-19T06:01:40.848667+0000","last_peered":"2024-09-19T06:01:40.848667+0000","last_clean":"2024-09-19T06:01:40.848667+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:01:40.848667+0000","last_undegraded":"2024-09-19T06:01:40.848667+0000","last_fullsized":"2024-09-19T06:01:40.848667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.114865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832096+0000","last_change":"2024-09-19T06:01:28.801523+0000","last_active":"2024-09-19T06:01:39.832096+0000","last_peered":"2024-09-19T06:01:39.832096+0000","last_clean":"2024-09-19T06:01:39.832096+0000","last_became_active":"2024-09-19T06:01:28.801205+0000","last_became_peered":"2024-09-19T06:01:28.801205+0000","last_unstale":"2024-09-19T06:01:39.832096+0000","last_undegraded":"2024-09-19T06:01:39.832096+0000","last_fullsized":"2024-09-19T06:01:39.832096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:10.314085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833424+0000","last_change":"2024-09-19T06:01:27.022172+0000","last_active":"2024-09-19T06:01:39.833424+0000","last_peered":"2024-09-19T06:01:39.833424+0000","last_clean":"2024-09-19T06:01:39.833424+0000","last_became_active":"2024-09-19T06:01:27.021568+0000","last_became_peered":"2024-09-19T06:01:27.021568+0000","last_unstale":"2024-09-19T06:01:39.833424+0000","last_undegraded":"2024-09-19T06:01:39.833424+0000","last_fullsized":"2024-09-19T06:01:39.833424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:18.031690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.839061+0000","last_change":"2024-09-19T06:01:39.839061+0000","last_active":"2024-09-19T06:01:39.839061+0000","last_peered":"2024-09-19T06:01:39.839061+0000","last_clean":"2024-09-19T06:01:39.839061+0000","last_became_active":"2024-09-19T06:01:39.838644+0000","last_became_peered":"2024-09-19T06:01:39.838644+0000","last_unstale":"2024-09-19T06:01:39.839061+0000","last_undegraded":"2024-09-19T06:01:39.839061+0000","last_fullsized":"2024-09-19T06:01:39.839061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:02.136222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'27","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:54.011332+0000","last_change":"2024-09-19T06:01:26.784781+0000","last_active":"2024-09-19T06:01:54.011332+0000","last_peered":"2024-09-19T06:01:54.011332+0000","last_clean":"2024-09-19T06:01:54.011332+0000","last_became_active":"2024-09-19T06:01:26.784284+0000","last_became_peered":"2024-09-19T06:01:26.784284+0000","last_unstale":"2024-09-19T06:01:54.011332+0000","last_undegraded":"2024-09-19T06:01:54.011332+0000","last_fullsized":"2024-09-19T06:01:54.011332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:13.063299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54445,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":61,"num_write":29,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.839724+0000","last_change":"2024-09-19T06:01:28.802604+0000","last_active":"2024-09-19T06:01:39.839724+0000","last_peered":"2024-09-19T06:01:39.839724+0000","last_clean":"2024-09-19T06:01:39.839724+0000","last_became_active":"2024-09-19T06:01:28.802285+0000","last_became_peered":"2024-09-19T06:01:28.802285+0000","last_unstale":"2024-09-19T06:01:39.839724+0000","last_undegraded":"2024-09-19T06:01:39.839724+0000","last_fullsized":"2024-09-19T06:01:39.839724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:57.177791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:41.846861+0000","last_change":"2024-09-19T06:01:26.783678+0000","last_active":"2024-09-19T06:01:41.846861+0000","last_peered":"2024-09-19T06:01:41.846861+0000","last_clean":"2024-09-19T06:01:41.846861+0000","last_became_active":"2024-09-19T06:01:26.783165+0000","last_became_peered":"2024-09-19T06:01:26.783165+0000","last_unstale":"2024-09-19T06:01:41.846861+0000","last_undegraded":"2024-09-19T06:01:41.846861+0000","last_fullsized":"2024-09-19T06:01:41.846861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:50.535733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.828953+0000","last_change":"2024-09-19T06:01:28.808553+0000","last_active":"2024-09-19T06:01:39.828953+0000","last_peered":"2024-09-19T06:01:39.828953+0000","last_clean":"2024-09-19T06:01:39.828953+0000","last_became_active":"2024-09-19T06:01:28.808215+0000","last_became_peered":"2024-09-19T06:01:28.808215+0000","last_unstale":"2024-09-19T06:01:39.828953+0000","last_undegraded":"2024-09-19T06:01:39.828953+0000","last_fullsized":"2024-09-19T06:01:39.828953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:55.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.848791+0000","last_change":"2024-09-19T06:01:26.787137+0000","last_active":"2024-09-19T06:01:40.848791+0000","last_peered":"2024-09-19T06:01:40.848791+0000","last_clean":"2024-09-19T06:01:40.848791+0000","last_became_active":"2024-09-19T06:01:26.786676+0000","last_became_peered":"2024-09-19T06:01:26.786676+0000","last_unstale":"2024-09-19T06:01:40.848791+0000","last_undegraded":"2024-09-19T06:01:40.848791+0000","last_fullsized":"2024-09-19T06:01:40.848791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.609969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837656+0000","last_change":"2024-09-19T06:01:28.812106+0000","last_active":"2024-09-19T06:01:39.837656+0000","last_peered":"2024-09-19T06:01:39.837656+0000","last_clean":"2024-09-19T06:01:39.837656+0000","last_became_active":"2024-09-19T06:01:28.811771+0000","last_became_peered":"2024-09-19T06:01:28.811771+0000","last_unstale":"2024-09-19T06:01:39.837656+0000","last_undegraded":"2024-09-19T06:01:39.837656+0000","last_fullsized":"2024-09-19T06:01:39.837656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:07:06.832685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:48.860453+0000","last_change":"2024-09-19T06:01:26.786746+0000","last_active":"2024-09-19T06:01:48.860453+0000","last_peered":"2024-09-19T06:01:48.860453+0000","last_clean":"2024-09-19T06:01:48.860453+0000","last_became_active":"2024-09-19T06:01:26.786414+0000","last_became_peered":"2024-09-19T06:01:26.786414+0000","last_unstale":"2024-09-19T06:01:48.860453+0000","last_undegraded":"2024-09-19T06:01:48.860453+0000","last_fullsized":"2024-09-19T06:01:48.860453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:25.042796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837014+0000","last_change":"2024-09-19T06:01:28.807183+0000","last_active":"2024-09-19T06:01:39.837014+0000","last_peered":"2024-09-19T06:01:39.837014+0000","last_clean":"2024-09-19T06:01:39.837014+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:01:39.837014+0000","last_undegraded":"2024-09-19T06:01:39.837014+0000","last_fullsized":"2024-09-19T06:01:39.837014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:23.030312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847960+0000","last_change":"2024-09-19T06:01:27.022409+0000","last_active":"2024-09-19T06:01:40.847960+0000","last_peered":"2024-09-19T06:01:40.847960+0000","last_clean":"2024-09-19T06:01:40.847960+0000","last_became_active":"2024-09-19T06:01:27.022034+0000","last_became_peered":"2024-09-19T06:01:27.022034+0000","last_unstale":"2024-09-19T06:01:40.847960+0000","last_undegraded":"2024-09-19T06:01:40.847960+0000","last_fullsized":"2024-09-19T06:01:40.847960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:08.663575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832674+0000","last_change":"2024-09-19T06:01:28.812079+0000","last_active":"2024-09-19T06:01:39.832674+0000","last_peered":"2024-09-19T06:01:39.832674+0000","last_clean":"2024-09-19T06:01:39.832674+0000","last_became_active":"2024-09-19T06:01:28.811606+0000","last_became_peered":"2024-09-19T06:01:28.811606+0000","last_unstale":"2024-09-19T06:01:39.832674+0000","last_undegraded":"2024-09-19T06:01:39.832674+0000","last_fullsized":"2024-09-19T06:01:39.832674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:49.434235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.838741+0000","last_change":"2024-09-19T06:01:26.783823+0000","last_active":"2024-09-19T06:01:39.838741+0000","last_peered":"2024-09-19T06:01:39.838741+0000","last_clean":"2024-09-19T06:01:39.838741+0000","last_became_active":"2024-09-19T06:01:26.783209+0000","last_became_peered":"2024-09-19T06:01:26.783209+0000","last_unstale":"2024-09-19T06:01:39.838741+0000","last_undegraded":"2024-09-19T06:01:39.838741+0000","last_fullsized":"2024-09-19T06:01:39.838741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:38.833417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837462+0000","last_change":"2024-09-19T06:01:39.837462+0000","last_active":"2024-09-19T06:01:39.837462+0000","last_peered":"2024-09-19T06:01:39.837462+0000","last_clean":"2024-09-19T06:01:39.837462+0000","last_became_active":"2024-09-19T06:01:39.837082+0000","last_became_peered":"2024-09-19T06:01:39.837082+0000","last_unstale":"2024-09-19T06:01:39.837462+0000","last_undegraded":"2024-09-19T06:01:39.837462+0000","last_fullsized":"2024-09-19T06:01:39.837462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:47.421017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.845150+0000","last_change":"2024-09-19T06:01:26.779992+0000","last_active":"2024-09-19T06:01:40.845150+0000","last_peered":"2024-09-19T06:01:40.845150+0000","last_clean":"2024-09-19T06:01:40.845150+0000","last_became_active":"2024-09-19T06:01:26.779609+0000","last_became_peered":"2024-09-19T06:01:26.779609+0000","last_unstale":"2024-09-19T06:01:40.845150+0000","last_undegraded":"2024-09-19T06:01:40.845150+0000","last_fullsized":"2024-09-19T06:01:40.845150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:48.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837947+0000","last_change":"2024-09-19T06:01:28.798827+0000","last_active":"2024-09-19T06:01:39.837947+0000","last_peered":"2024-09-19T06:01:39.837947+0000","last_clean":"2024-09-19T06:01:39.837947+0000","last_became_active":"2024-09-19T06:01:28.798457+0000","last_became_peered":"2024-09-19T06:01:28.798457+0000","last_unstale":"2024-09-19T06:01:39.837947+0000","last_undegraded":"2024-09-19T06:01:39.837947+0000","last_fullsized":"2024-09-19T06:01:39.837947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:41.111051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.846954+0000","last_change":"2024-09-19T06:01:26.775691+0000","last_active":"2024-09-19T06:01:40.846954+0000","last_peered":"2024-09-19T06:01:40.846954+0000","last_clean":"2024-09-19T06:01:40.846954+0000","last_became_active":"2024-09-19T06:01:26.775183+0000","last_became_peered":"2024-09-19T06:01:26.775183+0000","last_unstale":"2024-09-19T06:01:40.846954+0000","last_undegraded":"2024-09-19T06:01:40.846954+0000","last_fullsized":"2024-09-19T06:01:40.846954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:34.712968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837674+0000","last_change":"2024-09-19T06:01:28.809459+0000","last_active":"2024-09-19T06:01:39.837674+0000","last_peered":"2024-09-19T06:01:39.837674+0000","last_clean":"2024-09-19T06:01:39.837674+0000","last_became_active":"2024-09-19T06:01:28.809046+0000","last_became_peered":"2024-09-19T06:01:28.809046+0000","last_unstale":"2024-09-19T06:01:39.837674+0000","last_undegraded":"2024-09-19T06:01:39.837674+0000","last_fullsized":"2024-09-19T06:01:39.837674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:56.996190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:52.865441+0000","last_change":"2024-09-19T06:01:39.843932+0000","last_active":"2024-09-19T06:01:52.865441+0000","last_peered":"2024-09-19T06:01:52.865441+0000","last_clean":"2024-09-19T06:01:52.865441+0000","last_became_active":"2024-09-19T06:01:39.836008+0000","last_became_peered":"2024-09-19T06:01:39.836008+0000","last_unstale":"2024-09-19T06:01:52.865441+0000","last_undegraded":"2024-09-19T06:01:52.865441+0000","last_fullsized":"2024-09-19T06:01:52.865441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:58.798859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837726+0000","last_change":"2024-09-19T06:01:28.814181+0000","last_active":"2024-09-19T06:01:39.837726+0000","last_peered":"2024-09-19T06:01:39.837726+0000","last_clean":"2024-09-19T06:01:39.837726+0000","last_became_active":"2024-09-19T06:01:28.813817+0000","last_became_peered":"2024-09-19T06:01:28.813817+0000","last_unstale":"2024-09-19T06:01:39.837726+0000","last_undegraded":"2024-09-19T06:01:39.837726+0000","last_fullsized":"2024-09-19T06:01:39.837726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:40.057863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837197+0000","last_change":"2024-09-19T06:00:17.624509+0000","last_active":"2024-09-19T06:01:39.837197+0000","last_peered":"2024-09-19T06:01:39.837197+0000","last_clean":"2024-09-19T06:01:39.837197+0000","last_became_active":"2024-09-19T06:00:17.623943+0000","last_became_peered":"2024-09-19T06:00:17.623943+0000","last_unstale":"2024-09-19T06:01:39.837197+0000","last_undegraded":"2024-09-19T06:01:39.837197+0000","last_fullsized":"2024-09-19T06:01:39.837197+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_clean_scrub_stamp":"2024-09-19T05:56:55.414488+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:39.116345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:41.847767+0000","last_change":"2024-09-19T06:01:26.783896+0000","last_active":"2024-09-19T06:01:41.847767+0000","last_peered":"2024-09-19T06:01:41.847767+0000","last_clean":"2024-09-19T06:01:41.847767+0000","last_became_active":"2024-09-19T06:01:26.783332+0000","last_became_peered":"2024-09-19T06:01:26.783332+0000","last_unstale":"2024-09-19T06:01:41.847767+0000","last_undegraded":"2024-09-19T06:01:41.847767+0000","last_fullsized":"2024-09-19T06:01:41.847767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:33.072763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980113+0000","last_change":"2024-09-19T06:01:28.807756+0000","last_active":"2024-09-19T06:01:39.980113+0000","last_peered":"2024-09-19T06:01:39.980113+0000","last_clean":"2024-09-19T06:01:39.980113+0000","last_became_active":"2024-09-19T06:01:28.807077+0000","last_became_peered":"2024-09-19T06:01:28.807077+0000","last_unstale":"2024-09-19T06:01:39.980113+0000","last_undegraded":"2024-09-19T06:01:39.980113+0000","last_fullsized":"2024-09-19T06:01:39.980113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:20.784199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837194+0000","last_change":"2024-09-19T06:01:26.784830+0000","last_active":"2024-09-19T06:01:39.837194+0000","last_peered":"2024-09-19T06:01:39.837194+0000","last_clean":"2024-09-19T06:01:39.837194+0000","last_became_active":"2024-09-19T06:01:26.783887+0000","last_became_peered":"2024-09-19T06:01:26.783887+0000","last_unstale":"2024-09-19T06:01:39.837194+0000","last_undegraded":"2024-09-19T06:01:39.837194+0000","last_fullsized":"2024-09-19T06:01:39.837194+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:33:38.255169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837354+0000","last_change":"2024-09-19T06:01:39.837354+0000","last_active":"2024-09-19T06:01:39.837354+0000","last_peered":"2024-09-19T06:01:39.837354+0000","last_clean":"2024-09-19T06:01:39.837354+0000","last_became_active":"2024-09-19T06:01:39.836930+0000","last_became_peered":"2024-09-19T06:01:39.836930+0000","last_unstale":"2024-09-19T06:01:39.837354+0000","last_undegraded":"2024-09-19T06:01:39.837354+0000","last_fullsized":"2024-09-19T06:01:39.837354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:17.571834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:52.864443+0000","last_change":"2024-09-19T06:01:27.021753+0000","last_active":"2024-09-19T06:01:52.864443+0000","last_peered":"2024-09-19T06:01:52.864443+0000","last_clean":"2024-09-19T06:01:52.864443+0000","last_became_active":"2024-09-19T06:01:27.021346+0000","last_became_peered":"2024-09-19T06:01:27.021346+0000","last_unstale":"2024-09-19T06:01:52.864443+0000","last_undegraded":"2024-09-19T06:01:52.864443+0000","last_fullsized":"2024-09-19T06:01:52.864443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:07.291283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832814+0000","last_change":"2024-09-19T06:01:28.803103+0000","last_active":"2024-09-19T06:01:39.832814+0000","last_peered":"2024-09-19T06:01:39.832814+0000","last_clean":"2024-09-19T06:01:39.832814+0000","last_became_active":"2024-09-19T06:01:28.802806+0000","last_became_peered":"2024-09-19T06:01:28.802806+0000","last_unstale":"2024-09-19T06:01:39.832814+0000","last_undegraded":"2024-09-19T06:01:39.832814+0000","last_fullsized":"2024-09-19T06:01:39.832814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:33.329231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:52.864856+0000","last_change":"2024-09-19T06:01:26.786605+0000","last_active":"2024-09-19T06:01:52.864856+0000","last_peered":"2024-09-19T06:01:52.864856+0000","last_clean":"2024-09-19T06:01:52.864856+0000","last_became_active":"2024-09-19T06:01:26.786278+0000","last_became_peered":"2024-09-19T06:01:26.786278+0000","last_unstale":"2024-09-19T06:01:52.864856+0000","last_undegraded":"2024-09-19T06:01:52.864856+0000","last_fullsized":"2024-09-19T06:01:52.864856+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.581502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323778+0000","last_change":"2024-09-19T06:01:28.808152+0000","last_active":"2024-09-19T06:01:40.323778+0000","last_peered":"2024-09-19T06:01:40.323778+0000","last_clean":"2024-09-19T06:01:40.323778+0000","last_became_active":"2024-09-19T06:01:28.807788+0000","last_became_peered":"2024-09-19T06:01:28.807788+0000","last_unstale":"2024-09-19T06:01:40.323778+0000","last_undegraded":"2024-09-19T06:01:40.323778+0000","last_fullsized":"2024-09-19T06:01:40.323778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:42.877539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.836049+0000","last_change":"2024-09-19T06:01:27.022719+0000","last_active":"2024-09-19T06:01:39.836049+0000","last_peered":"2024-09-19T06:01:39.836049+0000","last_clean":"2024-09-19T06:01:39.836049+0000","last_became_active":"2024-09-19T06:01:27.022263+0000","last_became_peered":"2024-09-19T06:01:27.022263+0000","last_unstale":"2024-09-19T06:01:39.836049+0000","last_undegraded":"2024-09-19T06:01:39.836049+0000","last_fullsized":"2024-09-19T06:01:39.836049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:45.158229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323926+0000","last_change":"2024-09-19T06:01:28.809869+0000","last_active":"2024-09-19T06:01:40.323926+0000","last_peered":"2024-09-19T06:01:40.323926+0000","last_clean":"2024-09-19T06:01:40.323926+0000","last_became_active":"2024-09-19T06:01:28.809548+0000","last_became_peered":"2024-09-19T06:01:28.809548+0000","last_unstale":"2024-09-19T06:01:40.323926+0000","last_undegraded":"2024-09-19T06:01:40.323926+0000","last_fullsized":"2024-09-19T06:01:40.323926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:41.587846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847043+0000","last_change":"2024-09-19T06:01:39.846740+0000","last_active":"2024-09-19T06:01:40.847043+0000","last_peered":"2024-09-19T06:01:40.847043+0000","last_clean":"2024-09-19T06:01:40.847043+0000","last_became_active":"2024-09-19T06:01:39.836378+0000","last_became_peered":"2024-09-19T06:01:39.836378+0000","last_unstale":"2024-09-19T06:01:40.847043+0000","last_undegraded":"2024-09-19T06:01:40.847043+0000","last_fullsized":"2024-09-19T06:01:40.847043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.064918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829305+0000","last_change":"2024-09-19T06:01:28.807796+0000","last_active":"2024-09-19T06:01:39.829305+0000","last_peered":"2024-09-19T06:01:39.829305+0000","last_clean":"2024-09-19T06:01:39.829305+0000","last_became_active":"2024-09-19T06:01:28.807503+0000","last_became_peered":"2024-09-19T06:01:28.807503+0000","last_unstale":"2024-09-19T06:01:39.829305+0000","last_undegraded":"2024-09-19T06:01:39.829305+0000","last_fullsized":"2024-09-19T06:01:39.829305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:43.472647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'13","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:51.795547+0000","last_change":"2024-09-19T06:01:26.783517+0000","last_active":"2024-09-19T06:01:51.795547+0000","last_peered":"2024-09-19T06:01:51.795547+0000","last_clean":"2024-09-19T06:01:51.795547+0000","last_became_active":"2024-09-19T06:01:26.783164+0000","last_became_peered":"2024-09-19T06:01:26.783164+0000","last_unstale":"2024-09-19T06:01:51.795547+0000","last_undegraded":"2024-09-19T06:01:51.795547+0000","last_fullsized":"2024-09-19T06:01:51.795547+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:57.609192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980454+0000","last_change":"2024-09-19T06:01:28.806548+0000","last_active":"2024-09-19T06:01:39.980454+0000","last_peered":"2024-09-19T06:01:39.980454+0000","last_clean":"2024-09-19T06:01:39.980454+0000","last_became_active":"2024-09-19T06:01:28.805724+0000","last_became_peered":"2024-09-19T06:01:28.805724+0000","last_unstale":"2024-09-19T06:01:39.980454+0000","last_undegraded":"2024-09-19T06:01:39.980454+0000","last_fullsized":"2024-09-19T06:01:39.980454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:14.360928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847547+0000","last_change":"2024-09-19T06:01:27.128213+0000","last_active":"2024-09-19T06:01:40.847547+0000","last_peered":"2024-09-19T06:01:40.847547+0000","last_clean":"2024-09-19T06:01:40.847547+0000","last_became_active":"2024-09-19T06:01:27.127737+0000","last_became_peered":"2024-09-19T06:01:27.127737+0000","last_unstale":"2024-09-19T06:01:40.847547+0000","last_undegraded":"2024-09-19T06:01:40.847547+0000","last_fullsized":"2024-09-19T06:01:40.847547+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:33:35.107152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.839027+0000","last_change":"2024-09-19T06:01:39.839027+0000","last_active":"2024-09-19T06:01:39.839027+0000","last_peered":"2024-09-19T06:01:39.839027+0000","last_clean":"2024-09-19T06:01:39.839027+0000","last_became_active":"2024-09-19T06:01:39.838601+0000","last_became_peered":"2024-09-19T06:01:39.838601+0000","last_unstale":"2024-09-19T06:01:39.839027+0000","last_undegraded":"2024-09-19T06:01:39.839027+0000","last_fullsized":"2024-09-19T06:01:39.839027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:09.864997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832051+0000","last_change":"2024-09-19T06:01:27.022438+0000","last_active":"2024-09-19T06:01:39.832051+0000","last_peered":"2024-09-19T06:01:39.832051+0000","last_clean":"2024-09-19T06:01:39.832051+0000","last_became_active":"2024-09-19T06:01:27.021953+0000","last_became_peered":"2024-09-19T06:01:27.021953+0000","last_unstale":"2024-09-19T06:01:39.832051+0000","last_undegraded":"2024-09-19T06:01:39.832051+0000","last_fullsized":"2024-09-19T06:01:39.832051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:24.383178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837088+0000","last_change":"2024-09-19T06:01:28.803807+0000","last_active":"2024-09-19T06:01:39.837088+0000","last_peered":"2024-09-19T06:01:39.837088+0000","last_clean":"2024-09-19T06:01:39.837088+0000","last_became_active":"2024-09-19T06:01:28.803486+0000","last_became_peered":"2024-09-19T06:01:28.803486+0000","last_unstale":"2024-09-19T06:01:39.837088+0000","last_undegraded":"2024-09-19T06:01:39.837088+0000","last_fullsized":"2024-09-19T06:01:39.837088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:26:17.920089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.845539+0000","last_change":"2024-09-19T06:01:26.777171+0000","last_active":"2024-09-19T06:01:40.845539+0000","last_peered":"2024-09-19T06:01:40.845539+0000","last_clean":"2024-09-19T06:01:40.845539+0000","last_became_active":"2024-09-19T06:01:26.776818+0000","last_became_peered":"2024-09-19T06:01:26.776818+0000","last_unstale":"2024-09-19T06:01:40.845539+0000","last_undegraded":"2024-09-19T06:01:40.845539+0000","last_fullsized":"2024-09-19T06:01:40.845539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832532+0000","last_change":"2024-09-19T06:01:28.799266+0000","last_active":"2024-09-19T06:01:39.832532+0000","last_peered":"2024-09-19T06:01:39.832532+0000","last_clean":"2024-09-19T06:01:39.832532+0000","last_became_active":"2024-09-19T06:01:28.798969+0000","last_became_peered":"2024-09-19T06:01:28.798969+0000","last_unstale":"2024-09-19T06:01:39.832532+0000","last_undegraded":"2024-09-19T06:01:39.832532+0000","last_fullsized":"2024-09-19T06:01:39.832532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:34.064795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830288+0000","last_change":"2024-09-19T06:01:26.781680+0000","last_active":"2024-09-19T06:01:39.830288+0000","last_peered":"2024-09-19T06:01:39.830288+0000","last_clean":"2024-09-19T06:01:39.830288+0000","last_became_active":"2024-09-19T06:01:26.781035+0000","last_became_peered":"2024-09-19T06:01:26.781035+0000","last_unstale":"2024-09-19T06:01:39.830288+0000","last_undegraded":"2024-09-19T06:01:39.830288+0000","last_fullsized":"2024-09-19T06:01:39.830288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:38.287018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980152+0000","last_change":"2024-09-19T06:01:28.802970+0000","last_active":"2024-09-19T06:01:39.980152+0000","last_peered":"2024-09-19T06:01:39.980152+0000","last_clean":"2024-09-19T06:01:39.980152+0000","last_became_active":"2024-09-19T06:01:28.802587+0000","last_became_peered":"2024-09-19T06:01:28.802587+0000","last_unstale":"2024-09-19T06:01:39.980152+0000","last_undegraded":"2024-09-19T06:01:39.980152+0000","last_fullsized":"2024-09-19T06:01:39.980152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837492+0000","last_change":"2024-09-19T06:01:26.788140+0000","last_active":"2024-09-19T06:01:39.837492+0000","last_peered":"2024-09-19T06:01:39.837492+0000","last_clean":"2024-09-19T06:01:39.837492+0000","last_became_active":"2024-09-19T06:01:26.787603+0000","last_became_peered":"2024-09-19T06:01:26.787603+0000","last_unstale":"2024-09-19T06:01:39.837492+0000","last_undegraded":"2024-09-19T06:01:39.837492+0000","last_fullsized":"2024-09-19T06:01:39.837492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:13.880656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837369+0000","last_change":"2024-09-19T06:01:28.813144+0000","last_active":"2024-09-19T06:01:39.837369+0000","last_peered":"2024-09-19T06:01:39.837369+0000","last_clean":"2024-09-19T06:01:39.837369+0000","last_became_active":"2024-09-19T06:01:28.812792+0000","last_became_peered":"2024-09-19T06:01:28.812792+0000","last_unstale":"2024-09-19T06:01:39.837369+0000","last_undegraded":"2024-09-19T06:01:39.837369+0000","last_fullsized":"2024-09-19T06:01:39.837369+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:35.896240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833256+0000","last_change":"2024-09-19T06:01:28.812086+0000","last_active":"2024-09-19T06:01:39.833256+0000","last_peered":"2024-09-19T06:01:39.833256+0000","last_clean":"2024-09-19T06:01:39.833256+0000","last_became_active":"2024-09-19T06:01:28.811696+0000","last_became_peered":"2024-09-19T06:01:28.811696+0000","last_unstale":"2024-09-19T06:01:39.833256+0000","last_undegraded":"2024-09-19T06:01:39.833256+0000","last_fullsized":"2024-09-19T06:01:39.833256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:32.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847847+0000","last_change":"2024-09-19T06:01:39.842250+0000","last_active":"2024-09-19T06:01:40.847847+0000","last_peered":"2024-09-19T06:01:40.847847+0000","last_clean":"2024-09-19T06:01:40.847847+0000","last_became_active":"2024-09-19T06:01:39.835761+0000","last_became_peered":"2024-09-19T06:01:39.835761+0000","last_unstale":"2024-09-19T06:01:40.847847+0000","last_undegraded":"2024-09-19T06:01:40.847847+0000","last_fullsized":"2024-09-19T06:01:40.847847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:51.026349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832601+0000","last_change":"2024-09-19T06:01:28.812983+0000","last_active":"2024-09-19T06:01:39.832601+0000","last_peered":"2024-09-19T06:01:39.832601+0000","last_clean":"2024-09-19T06:01:39.832601+0000","last_became_active":"2024-09-19T06:01:28.812257+0000","last_became_peered":"2024-09-19T06:01:28.812257+0000","last_unstale":"2024-09-19T06:01:39.832601+0000","last_undegraded":"2024-09-19T06:01:39.832601+0000","last_fullsized":"2024-09-19T06:01:39.832601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:41:42.059736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:41.842632+0000","last_change":"2024-09-19T06:01:26.787869+0000","last_active":"2024-09-19T06:01:41.842632+0000","last_peered":"2024-09-19T06:01:41.842632+0000","last_clean":"2024-09-19T06:01:41.842632+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:01:41.842632+0000","last_undegraded":"2024-09-19T06:01:41.842632+0000","last_fullsized":"2024-09-19T06:01:41.842632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:02.869878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831993+0000","last_change":"2024-09-19T06:01:28.811986+0000","last_active":"2024-09-19T06:01:39.831993+0000","last_peered":"2024-09-19T06:01:39.831993+0000","last_clean":"2024-09-19T06:01:39.831993+0000","last_became_active":"2024-09-19T06:01:28.811565+0000","last_became_peered":"2024-09-19T06:01:28.811565+0000","last_unstale":"2024-09-19T06:01:39.831993+0000","last_undegraded":"2024-09-19T06:01:39.831993+0000","last_fullsized":"2024-09-19T06:01:39.831993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:01.324985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980467+0000","last_change":"2024-09-19T06:01:27.021683+0000","last_active":"2024-09-19T06:01:39.980467+0000","last_peered":"2024-09-19T06:01:39.980467+0000","last_clean":"2024-09-19T06:01:39.980467+0000","last_became_active":"2024-09-19T06:01:27.021358+0000","last_became_peered":"2024-09-19T06:01:27.021358+0000","last_unstale":"2024-09-19T06:01:39.980467+0000","last_undegraded":"2024-09-19T06:01:39.980467+0000","last_fullsized":"2024-09-19T06:01:39.980467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:29.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980328+0000","last_change":"2024-09-19T06:01:28.798963+0000","last_active":"2024-09-19T06:01:39.980328+0000","last_peered":"2024-09-19T06:01:39.980328+0000","last_clean":"2024-09-19T06:01:39.980328+0000","last_became_active":"2024-09-19T06:01:28.797571+0000","last_became_peered":"2024-09-19T06:01:28.797571+0000","last_unstale":"2024-09-19T06:01:39.980328+0000","last_undegraded":"2024-09-19T06:01:39.980328+0000","last_fullsized":"2024-09-19T06:01:39.980328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:43.867609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:52.864302+0000","last_change":"2024-09-19T06:01:26.786427+0000","last_active":"2024-09-19T06:01:52.864302+0000","last_peered":"2024-09-19T06:01:52.864302+0000","last_clean":"2024-09-19T06:01:52.864302+0000","last_became_active":"2024-09-19T06:01:26.786092+0000","last_became_peered":"2024-09-19T06:01:26.786092+0000","last_unstale":"2024-09-19T06:01:52.864302+0000","last_undegraded":"2024-09-19T06:01:52.864302+0000","last_fullsized":"2024-09-19T06:01:52.864302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:49.144366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62439,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":165,"num_read_kb":147,"num_write":239,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":179674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547216,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":439275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36908,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708244,"statfs":{"total":95995035648,"available":95957241856,"internally_reserved":0,"allocated":1601536,"data_stored":1076412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":438118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168609,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":437802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397541,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1031802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561065,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789999,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1030694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953524,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":439104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":437660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40456,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704696,"statfs":{"total":95995035648,"available":95953608704,"internally_reserved":0,"allocated":974848,"data_stored":483978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40460,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704692,"statfs":{"total":95995035648,"available":95953604608,"internally_reserved":0,"allocated":978944,"data_stored":487506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":439567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":58153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":46982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:01:56.517 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T06:01:56.517 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T06:01:56.518 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T06:01:56.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:56 smithi157 ceph-mon[35346]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 21 op/s; 102 B/s, 0 objects/s recovering 2024-09-19T06:01:56.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:56 smithi157 ceph-mon[35346]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:01:56.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:56 smithi049 ceph-mon[34640]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 21 op/s; 102 B/s, 0 objects/s recovering 2024-09-19T06:01:56.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:56 smithi049 ceph-mon[34640]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:01:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.277+0000 7f24660c7640 1 -- 172.21.15.5:0/538690455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2460101e50 msgr2=0x7f24601086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.277+0000 7f24660c7640 1 --2- 172.21.15.5:0/538690455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2460101e50 0x7f24601086e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f245800adb0 tx=0x7f245802b7a0 comp rx=0 tx=0).stop 2024-09-19T06:01:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.277+0000 7f24660c7640 1 -- 172.21.15.5:0/538690455 shutdown_connections 2024-09-19T06:01:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.277+0000 7f24660c7640 1 --2- 172.21.15.5:0/538690455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2460101e50 0x7f24601086e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.277+0000 7f24660c7640 1 --2- 172.21.15.5:0/538690455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2460101490 0x7f2460101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.278+0000 7f24660c7640 1 --2- 172.21.15.5:0/538690455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2460100290 0x7f2460100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.278+0000 7f24660c7640 1 -- 172.21.15.5:0/538690455 >> 172.21.15.5:0/538690455 conn(0x7f24600fba40 msgr2=0x7f24600fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.278+0000 7f24660c7640 1 -- 172.21.15.5:0/538690455 shutdown_connections 2024-09-19T06:01:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.278+0000 7f24660c7640 1 -- 172.21.15.5:0/538690455 wait complete. 2024-09-19T06:01:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.278+0000 7f24660c7640 1 Processor -- start 2024-09-19T06:01:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.278+0000 7f24660c7640 1 -- start start 2024-09-19T06:01:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.278+0000 7f24660c7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2460100290 0x7f2460195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f24660c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2460101490 0x7f24601960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f24660c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2460101e50 0x7f246019d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f24660c7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f246010b520 con 0x7f2460101e50 2024-09-19T06:01:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f24660c7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f246010b3a0 con 0x7f2460100290 2024-09-19T06:01:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f24660c7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f246010b6a0 con 0x7f2460101490 2024-09-19T06:01:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2460101e50 0x7f246019d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2460101e50 0x7f246019d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54932/0 (socket says 172.21.15.5:54932) 2024-09-19T06:01:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245ffff640 1 -- 172.21.15.5:0/124680169 learned_addr learned my addr 172.21.15.5:0/124680169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245f7fe640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2460100290 0x7f2460195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245effd640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2460101490 0x7f24601960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245ffff640 1 -- 172.21.15.5:0/124680169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2460101490 msgr2=0x7f24601960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245ffff640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2460101490 0x7f24601960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245ffff640 1 -- 172.21.15.5:0/124680169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2460100290 msgr2=0x7f2460195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245ffff640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2460100290 0x7f2460195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245ffff640 1 -- 172.21.15.5:0/124680169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246019d7d0 con 0x7f2460101e50 2024-09-19T06:01:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245effd640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2460101490 0x7f24601960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:01:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245ffff640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2460101e50 0x7f246019d160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2458031040 tx=0x7f24580329a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245cff9640 1 -- 172.21.15.5:0/124680169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458041070 con 0x7f2460101e50 2024-09-19T06:01:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245cff9640 1 -- 172.21.15.5:0/124680169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24580309c0 con 0x7f2460101e50 2024-09-19T06:01:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f24660c7640 1 -- 172.21.15.5:0/124680169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246019da60 con 0x7f2460101e50 2024-09-19T06:01:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f245cff9640 1 -- 172.21.15.5:0/124680169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458031610 con 0x7f2460101e50 2024-09-19T06:01:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.279+0000 7f24660c7640 1 -- 172.21.15.5:0/124680169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246019df40 con 0x7f2460101e50 2024-09-19T06:01:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.280+0000 7f24660c7640 1 -- 172.21.15.5:0/124680169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2460107660 con 0x7f2460101e50 2024-09-19T06:01:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.282+0000 7f245cff9640 1 -- 172.21.15.5:0/124680169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f245803a040 con 0x7f2460101e50 2024-09-19T06:01:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.282+0000 7f245cff9640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f24480a3300 0x7f24480a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.282+0000 7f245cff9640 1 -- 172.21.15.5:0/124680169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f24580f1f40 con 0x7f2460101e50 2024-09-19T06:01:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.282+0000 7f245f7fe640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f24480a3300 0x7f24480a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.283+0000 7f245f7fe640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f24480a3300 0x7f24480a57c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f24601012f0 tx=0x7f2450005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.286+0000 7f245cff9640 1 -- 172.21.15.5:0/124680169 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f245800a9c0 con 0x7f2460101e50 2024-09-19T06:01:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.528+0000 7f24660c7640 1 -- 172.21.15.5:0/124680169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2460062df0 con 0x7f2460101e50 2024-09-19T06:01:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.530+0000 7f245cff9640 1 -- 172.21.15.5:0/124680169 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f24580bebb0 con 0x7f2460101e50 2024-09-19T06:01:57.532 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-09-19T06:01:57.532 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBVvutmqeOTHxAAq3n6C5i46AFn/EVqjBf+6g== 2024-09-19T06:01:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.533+0000 7f24660c7640 1 -- 172.21.15.5:0/124680169 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f24480a3300 msgr2=0x7f24480a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.533+0000 7f24660c7640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f24480a3300 0x7f24480a57c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f24601012f0 tx=0x7f2450005f90 comp rx=0 tx=0).stop 2024-09-19T06:01:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.533+0000 7f24660c7640 1 -- 172.21.15.5:0/124680169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2460101e50 msgr2=0x7f246019d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.533+0000 7f24660c7640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2460101e50 0x7f246019d160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2458031040 tx=0x7f24580329a0 comp rx=0 tx=0).stop 2024-09-19T06:01:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.533+0000 7f24660c7640 1 -- 172.21.15.5:0/124680169 shutdown_connections 2024-09-19T06:01:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.533+0000 7f24660c7640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f24480a3300 0x7f24480a57c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.533+0000 7f24660c7640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2460101e50 0x7f246019d160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.533+0000 7f24660c7640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2460101490 0x7f24601960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.533+0000 7f24660c7640 1 --2- 172.21.15.5:0/124680169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2460100290 0x7f2460195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.533+0000 7f24660c7640 1 -- 172.21.15.5:0/124680169 >> 172.21.15.5:0/124680169 conn(0x7f24600fba40 msgr2=0x7f2460104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.533+0000 7f24660c7640 1 -- 172.21.15.5:0/124680169 shutdown_connections 2024-09-19T06:01:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:57.533+0000 7f24660c7640 1 -- 172.21.15.5:0/124680169 wait complete. 2024-09-19T06:01:58.167 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T06:01:58.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T06:01:58.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T06:01:58.207 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T06:01:58.207 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T06:01:58.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T06:01:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:01:58.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:58 smithi157 ceph-mon[35346]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.7 KiB/s wr, 12 op/s; 1 B/s, 0 objects/s recovering 2024-09-19T06:01:58.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:58 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/124680169' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:01:58.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:01:58 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/124680169' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:01:58.623 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:58 smithi005 ceph-mon[30955]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.7 KiB/s wr, 12 op/s; 1 B/s, 0 objects/s recovering 2024-09-19T06:01:58.624 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:58 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/124680169' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:01:58.624 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:01:58 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/124680169' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:01:58.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:58 smithi049 ceph-mon[34640]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.7 KiB/s wr, 12 op/s; 1 B/s, 0 objects/s recovering 2024-09-19T06:01:58.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:58 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/124680169' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:01:58.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:01:58 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/124680169' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:01:59.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.025+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/3979029499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a41036e0 msgr2=0x7fe8a4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:59.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.025+0000 7fe8ac4e5640 1 --2- 172.21.15.5:0/3979029499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a41036e0 0x7fe8a4107b90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe894009930 tx=0x7fe89402a420 comp rx=0 tx=0).stop 2024-09-19T06:01:59.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/3979029499 shutdown_connections 2024-09-19T06:01:59.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 --2- 172.21.15.5:0/3979029499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a41082a0 0x7fe8a410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 --2- 172.21.15.5:0/3979029499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a41036e0 0x7fe8a4107b90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 --2- 172.21.15.5:0/3979029499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8a4102d30 0x7fe8a4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/3979029499 >> 172.21.15.5:0/3979029499 conn(0x7fe8a40fc7d0 msgr2=0x7fe8a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/3979029499 shutdown_connections 2024-09-19T06:01:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/3979029499 wait complete. 2024-09-19T06:01:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 Processor -- start 2024-09-19T06:01:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 -- start start 2024-09-19T06:01:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a4102d30 0x7fe8a419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a41036e0 0x7fe8a419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8a41082a0 0x7fe8a419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.026+0000 7fe8ac4e5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4111f50 con 0x7fe8a4102d30 2024-09-19T06:01:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8ac4e5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4111dd0 con 0x7fe8a41082a0 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8ac4e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a41120d0 con 0x7fe8a41036e0 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8aa281640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a4102d30 0x7fe8a419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8aa281640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a4102d30 0x7fe8a419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54960/0 (socket says 172.21.15.5:54960) 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8aa281640 1 -- 172.21.15.5:0/2371307275 learned_addr learned my addr 172.21.15.5:0/2371307275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8a9a80640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a41036e0 0x7fe8a419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8aaa82640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8a41082a0 0x7fe8a419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8aa281640 1 -- 172.21.15.5:0/2371307275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a41036e0 msgr2=0x7fe8a419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8aa281640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a41036e0 0x7fe8a419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8aa281640 1 -- 172.21.15.5:0/2371307275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8a41082a0 msgr2=0x7fe8a419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8aa281640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8a41082a0 0x7fe8a419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8aa281640 1 -- 172.21.15.5:0/2371307275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a419f6c0 con 0x7fe8a4102d30 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8a9a80640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a41036e0 0x7fe8a419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:01:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8aa281640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a4102d30 0x7fe8a419a710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000d950 tx=0x7fe8a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:59.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe89b7fe640 1 -- 172.21.15.5:0/2371307275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0017070 con 0x7fe8a4102d30 2024-09-19T06:01:59.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe89b7fe640 1 -- 172.21.15.5:0/2371307275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8a0009c30 con 0x7fe8a4102d30 2024-09-19T06:01:59.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/2371307275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a419f9b0 con 0x7fe8a4102d30 2024-09-19T06:01:59.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe89b7fe640 1 -- 172.21.15.5:0/2371307275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000f620 con 0x7fe8a4102d30 2024-09-19T06:01:59.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.027+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/2371307275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a41a7290 con 0x7fe8a4102d30 2024-09-19T06:01:59.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.029+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/2371307275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a4103110 con 0x7fe8a4102d30 2024-09-19T06:01:59.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.030+0000 7fe89b7fe640 1 -- 172.21.15.5:0/2371307275 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fe8a00109a0 con 0x7fe8a4102d30 2024-09-19T06:01:59.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.031+0000 7fe89b7fe640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe8900a3350 0x7fe8900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:01:59.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.031+0000 7fe8a9a80640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe8900a3350 0x7fe8900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:01:59.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.031+0000 7fe89b7fe640 1 -- 172.21.15.5:0/2371307275 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe8a00cf690 con 0x7fe8a4102d30 2024-09-19T06:01:59.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.031+0000 7fe8a9a80640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe8900a3350 0x7fe8900a5810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe894009900 tx=0x7fe894032c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:01:59.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.035+0000 7fe89b7fe640 1 -- 172.21.15.5:0/2371307275 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe8a0009de0 con 0x7fe8a4102d30 2024-09-19T06:01:59.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.258+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/2371307275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe8a410c2c0 con 0x7fe8a4102d30 2024-09-19T06:01:59.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.263+0000 7fe89b7fe640 1 -- 172.21.15.5:0/2371307275 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264038 (secure 0 0 0) 0x7fe8a009c240 con 0x7fe8a4102d30 2024-09-19T06:01:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:01:59.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.266+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/2371307275 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe8900a3350 msgr2=0x7fe8900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:59.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.266+0000 7fe8ac4e5640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe8900a3350 0x7fe8900a5810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe894009900 tx=0x7fe894032c20 comp rx=0 tx=0).stop 2024-09-19T06:01:59.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.266+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/2371307275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a4102d30 msgr2=0x7fe8a419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:01:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.266+0000 7fe8ac4e5640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a4102d30 0x7fe8a419a710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000d950 tx=0x7fe8a000de10 comp rx=0 tx=0).stop 2024-09-19T06:01:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.266+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/2371307275 shutdown_connections 2024-09-19T06:01:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.266+0000 7fe8ac4e5640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe8900a3350 0x7fe8900a5810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.266+0000 7fe8ac4e5640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8a41082a0 0x7fe8a419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.266+0000 7fe8ac4e5640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a41036e0 0x7fe8a419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.266+0000 7fe8ac4e5640 1 --2- 172.21.15.5:0/2371307275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a4102d30 0x7fe8a419a710 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:01:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.266+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/2371307275 >> 172.21.15.5:0/2371307275 conn(0x7fe8a40fc7d0 msgr2=0x7fe8a4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:01:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.266+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/2371307275 shutdown_connections 2024-09-19T06:01:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:01:59.266+0000 7fe8ac4e5640 1 -- 172.21.15.5:0/2371307275 wait complete. 2024-09-19T06:02:00.050 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":709970884},{"type":"v1","addr":"172.21.15.5:6801","nonce":709970884}]},"active_addr":"172.21.15.5:6801/709970884","active_change":"2024-09-19T05:54:38.703303+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.5:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1335130331}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1639646508}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3703679969}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2197596976}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1761468825}]}]} 2024-09-19T06:02:00.057 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T06:02:00.057 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T06:02:00.057 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:02:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:00.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:00 smithi005 ceph-mon[30955]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-19T06:02:00.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:00 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2371307275' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T06:02:00.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:00 smithi157 ceph-mon[35346]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-19T06:02:00.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:00 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2371307275' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T06:02:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:00 smithi049 ceph-mon[34640]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-19T06:02:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:00 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2371307275' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T06:02:00.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.823+0000 7f9ab9200640 1 -- 172.21.15.5:0/31085984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ab4107b00 msgr2=0x7f9ab4107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:00.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.823+0000 7f9ab9200640 1 --2- 172.21.15.5:0/31085984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ab4107b00 0x7f9ab4107ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009930 tx=0x7f9aa402a420 comp rx=0 tx=0).stop 2024-09-19T06:02:00.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.823+0000 7f9ab9200640 1 -- 172.21.15.5:0/31085984 shutdown_connections 2024-09-19T06:02:00.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.823+0000 7f9ab9200640 1 --2- 172.21.15.5:0/31085984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ab4102390 0x7f9ab410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:00.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.823+0000 7f9ab9200640 1 --2- 172.21.15.5:0/31085984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ab41019f0 0x7f9ab4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:00.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.823+0000 7f9ab9200640 1 --2- 172.21.15.5:0/31085984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ab4107b00 0x7f9ab4107ee0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:00.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.823+0000 7f9ab9200640 1 -- 172.21.15.5:0/31085984 >> 172.21.15.5:0/31085984 conn(0x7f9ab40fd710 msgr2=0x7f9ab40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:00.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.823+0000 7f9ab9200640 1 -- 172.21.15.5:0/31085984 shutdown_connections 2024-09-19T06:02:00.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.823+0000 7f9ab9200640 1 -- 172.21.15.5:0/31085984 wait complete. 2024-09-19T06:02:00.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.824+0000 7f9ab9200640 1 Processor -- start 2024-09-19T06:02:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.824+0000 7f9ab9200640 1 -- start start 2024-09-19T06:02:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.824+0000 7f9ab9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ab41019f0 0x7f9ab41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.824+0000 7f9ab9200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ab4102390 0x7f9ab4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.824+0000 7f9ab9200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ab4107b00 0x7f9ab419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.824+0000 7f9ab9200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4111db0 con 0x7f9ab41019f0 2024-09-19T06:02:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.824+0000 7f9ab9200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4111c30 con 0x7f9ab4107b00 2024-09-19T06:02:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.824+0000 7f9ab9200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4111f30 con 0x7f9ab4102390 2024-09-19T06:02:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.824+0000 7f9ab2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ab41019f0 0x7f9ab41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.824+0000 7f9ab2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ab41019f0 0x7f9ab41983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54980/0 (socket says 172.21.15.5:54980) 2024-09-19T06:02:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.824+0000 7f9ab2d9d640 1 -- 172.21.15.5:0/3054661742 learned_addr learned my addr 172.21.15.5:0/3054661742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.824+0000 7f9ab259c640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ab4102390 0x7f9ab4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9ab359e640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ab4107b00 0x7f9ab419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9ab2d9d640 1 -- 172.21.15.5:0/3054661742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ab4102390 msgr2=0x7f9ab4198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9ab2d9d640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ab4102390 0x7f9ab4198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9ab2d9d640 1 -- 172.21.15.5:0/3054661742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ab4107b00 msgr2=0x7f9ab419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:00.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9ab2d9d640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ab4107b00 0x7f9ab419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:00.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9ab2d9d640 1 -- 172.21.15.5:0/3054661742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab419d380 con 0x7f9ab41019f0 2024-09-19T06:02:00.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9ab359e640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ab4107b00 0x7f9ab419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:00.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9ab2d9d640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ab41019f0 0x7f9ab41983d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4004770 tx=0x7f9aa40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:00.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9a9bfff640 1 -- 172.21.15.5:0/3054661742 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa403f070 con 0x7f9ab41019f0 2024-09-19T06:02:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9a9bfff640 1 -- 172.21.15.5:0/3054661742 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9aa403b460 con 0x7f9ab41019f0 2024-09-19T06:02:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9ab9200640 1 -- 172.21.15.5:0/3054661742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab419d610 con 0x7f9ab41019f0 2024-09-19T06:02:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9a9bfff640 1 -- 172.21.15.5:0/3054661742 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4035070 con 0x7f9ab41019f0 2024-09-19T06:02:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.825+0000 7f9ab9200640 1 -- 172.21.15.5:0/3054661742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab41a4f50 con 0x7f9ab41019f0 2024-09-19T06:02:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.826+0000 7f9ab9200640 1 -- 172.21.15.5:0/3054661742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab4101e50 con 0x7f9ab41019f0 2024-09-19T06:02:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.827+0000 7f9a9bfff640 1 -- 172.21.15.5:0/3054661742 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f9aa403b610 con 0x7f9ab41019f0 2024-09-19T06:02:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.828+0000 7f9a9bfff640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f9a9c0a3350 0x7f9a9c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.828+0000 7f9ab259c640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f9a9c0a3350 0x7f9a9c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.829+0000 7f9a9bfff640 1 -- 172.21.15.5:0/3054661742 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9aa40efda0 con 0x7f9ab41019f0 2024-09-19T06:02:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.829+0000 7f9ab259c640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f9a9c0a3350 0x7f9a9c0a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4100d30 tx=0x7f9aa800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:00.833+0000 7f9a9bfff640 1 -- 172.21.15.5:0/3054661742 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9aa40bc950 con 0x7f9ab41019f0 2024-09-19T06:02:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.008+0000 7f9ab9200640 1 -- 172.21.15.5:0/3054661742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ab4103130 con 0x7f9ab41019f0 2024-09-19T06:02:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.010+0000 7f9a9bfff640 1 -- 172.21.15.5:0/3054661742 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22702 (secure 0 0 0) 0x7f9aa40bc250 con 0x7f9ab41019f0 2024-09-19T06:02:01.012 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:01.012 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","created":"2024-09-19T05:53:51.454068+0000","modified":"2024-09-19T06:01:39.826379+0000","last_up_change":"2024-09-19T06:00:38.837916+0000","last_in_change":"2024-09-19T06:00:15.609776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:56:55.213931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:01:25.648588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:01:27.760408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f3c10d60-d354-407f-9cdf-099344599844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6803","nonce":3442981810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6805","nonce":3442981810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6809","nonce":3442981810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6807","nonce":3442981810}]},"public_addr":"172.21.15.5:6803/3442981810","cluster_addr":"172.21.15.5:6805/3442981810","heartbeat_back_addr":"172.21.15.5:6809/3442981810","heartbeat_front_addr":"172.21.15.5:6807/3442981810","state":["exists","up"]},{"osd":1,"uuid":"5f40f636-7761-45ac-87f3-910a2fa2feb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448265014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6803","nonce":1448265014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6807","nonce":1448265014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6805","nonce":1448265014}]},"public_addr":"172.21.15.49:6801/1448265014","cluster_addr":"172.21.15.49:6803/1448265014","heartbeat_back_addr":"172.21.15.49:6807/1448265014","heartbeat_front_addr":"172.21.15.49:6805/1448265014","state":["exists","up"]},{"osd":2,"uuid":"ad00940c-4fdd-4f5f-9eaa-b1f38bcac526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6801","nonce":2319626622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6803","nonce":2319626622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6807","nonce":2319626622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6805","nonce":2319626622}]},"public_addr":"172.21.15.157:6801/2319626622","cluster_addr":"172.21.15.157:6803/2319626622","heartbeat_back_addr":"172.21.15.157:6807/2319626622","heartbeat_front_addr":"172.21.15.157:6805/2319626622","state":["exists","up"]},{"osd":3,"uuid":"63fec329-1b62-4c5e-b8ca-084a84d4d762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6811","nonce":1452491753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6813","nonce":1452491753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6817","nonce":1452491753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6815","nonce":1452491753}]},"public_addr":"172.21.15.5:6811/1452491753","cluster_addr":"172.21.15.5:6813/1452491753","heartbeat_back_addr":"172.21.15.5:6817/1452491753","heartbeat_front_addr":"172.21.15.5:6815/1452491753","state":["exists","up"]},{"osd":4,"uuid":"ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6809","nonce":3114952580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6811","nonce":3114952580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6815","nonce":3114952580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6813","nonce":3114952580}]},"public_addr":"172.21.15.49:6809/3114952580","cluster_addr":"172.21.15.49:6811/3114952580","heartbeat_back_addr":"172.21.15.49:6815/3114952580","heartbeat_front_addr":"172.21.15.49:6813/3114952580","state":["exists","up"]},{"osd":5,"uuid":"21a7a7d3-ce42-4688-bfec-714be635eaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6809","nonce":3219063971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6811","nonce":3219063971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6815","nonce":3219063971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6813","nonce":3219063971}]},"public_addr":"172.21.15.157:6809/3219063971","cluster_addr":"172.21.15.157:6811/3219063971","heartbeat_back_addr":"172.21.15.157:6815/3219063971","heartbeat_front_addr":"172.21.15.157:6813/3219063971","state":["exists","up"]},{"osd":6,"uuid":"cbef0e08-d6b0-4515-a2ee-2c81b048583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6819","nonce":2957795194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6821","nonce":2957795194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6825","nonce":2957795194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6823","nonce":2957795194}]},"public_addr":"172.21.15.5:6819/2957795194","cluster_addr":"172.21.15.5:6821/2957795194","heartbeat_back_addr":"172.21.15.5:6825/2957795194","heartbeat_front_addr":"172.21.15.5:6823/2957795194","state":["exists","up"]},{"osd":7,"uuid":"d261ac55-532d-4aea-9b3e-ed2b4eeb1da6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6817","nonce":2959731400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6819","nonce":2959731400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6823","nonce":2959731400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6821","nonce":2959731400}]},"public_addr":"172.21.15.49:6817/2959731400","cluster_addr":"172.21.15.49:6819/2959731400","heartbeat_back_addr":"172.21.15.49:6823/2959731400","heartbeat_front_addr":"172.21.15.49:6821/2959731400","state":["exists","up"]},{"osd":8,"uuid":"15d0eaf7-d604-4546-b393-4d6d39018f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6817","nonce":290026859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6819","nonce":290026859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6823","nonce":290026859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6821","nonce":290026859}]},"public_addr":"172.21.15.157:6817/290026859","cluster_addr":"172.21.15.157:6819/290026859","heartbeat_back_addr":"172.21.15.157:6823/290026859","heartbeat_front_addr":"172.21.15.157:6821/290026859","state":["exists","up"]},{"osd":9,"uuid":"30c08537-f593-4299-bfa1-fa7ba8a95169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6827","nonce":4122448237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6829","nonce":4122448237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6833","nonce":4122448237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6831","nonce":4122448237}]},"public_addr":"172.21.15.5:6827/4122448237","cluster_addr":"172.21.15.5:6829/4122448237","heartbeat_back_addr":"172.21.15.5:6833/4122448237","heartbeat_front_addr":"172.21.15.5:6831/4122448237","state":["exists","up"]},{"osd":10,"uuid":"68821e98-bd81-49ec-8cd6-51fb9880b553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6825","nonce":2551748002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6827","nonce":2551748002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6831","nonce":2551748002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6829","nonce":2551748002}]},"public_addr":"172.21.15.49:6825/2551748002","cluster_addr":"172.21.15.49:6827/2551748002","heartbeat_back_addr":"172.21.15.49:6831/2551748002","heartbeat_front_addr":"172.21.15.49:6829/2551748002","state":["exists","up"]},{"osd":11,"uuid":"7e443b46-2697-4d0b-8452-868463a1ee1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6825","nonce":1996582544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6827","nonce":1996582544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6831","nonce":1996582544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6829","nonce":1996582544}]},"public_addr":"172.21.15.157:6825/1996582544","cluster_addr":"172.21.15.157:6827/1996582544","heartbeat_back_addr":"172.21.15.157:6831/1996582544","heartbeat_front_addr":"172.21.15.157:6829/1996582544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:01.453592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:27.194705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:51.525385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:17.676749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:43.075555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:07.258154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:32.112583+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:57.306534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:21.320950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:47.164610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:12.548337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:37.262580+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4102275559":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/2593868621":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/1340147787":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6800/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/850678758":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6800/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2930274429":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/1367598148":"2024-09-20T05:54:15.334457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:02:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.012+0000 7f9ab9200640 1 -- 172.21.15.5:0/3054661742 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f9a9c0a3350 msgr2=0x7f9a9c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.012+0000 7f9ab9200640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f9a9c0a3350 0x7f9a9c0a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4100d30 tx=0x7f9aa800b390 comp rx=0 tx=0).stop 2024-09-19T06:02:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.012+0000 7f9ab9200640 1 -- 172.21.15.5:0/3054661742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ab41019f0 msgr2=0x7f9ab41983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.012+0000 7f9ab9200640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ab41019f0 0x7f9ab41983d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4004770 tx=0x7f9aa40042f0 comp rx=0 tx=0).stop 2024-09-19T06:02:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.012+0000 7f9ab9200640 1 -- 172.21.15.5:0/3054661742 shutdown_connections 2024-09-19T06:02:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.012+0000 7f9ab9200640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f9a9c0a3350 0x7f9a9c0a5810 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.012+0000 7f9ab9200640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ab4107b00 0x7f9ab419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.012+0000 7f9ab9200640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ab4102390 0x7f9ab4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.012+0000 7f9ab9200640 1 --2- 172.21.15.5:0/3054661742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ab41019f0 0x7f9ab41983d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.012+0000 7f9ab9200640 1 -- 172.21.15.5:0/3054661742 >> 172.21.15.5:0/3054661742 conn(0x7f9ab40fd710 msgr2=0x7f9ab40ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.012+0000 7f9ab9200640 1 -- 172.21.15.5:0/3054661742 shutdown_connections 2024-09-19T06:02:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:01.012+0000 7f9ab9200640 1 -- 172.21.15.5:0/3054661742 wait complete. 2024-09-19T06:02:01.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:01 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3054661742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:02:01.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:01 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3054661742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:02:01.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:01 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3054661742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:02:01.770 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T06:02:01.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:02:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:02.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:02 smithi005 ceph-mon[30955]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-09-19T06:02:02.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.471+0000 7f52d00bc640 1 -- 172.21.15.5:0/3784657528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c8106f10 msgr2=0x7f52c810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:02.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.471+0000 7f52d00bc640 1 --2- 172.21.15.5:0/3784657528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c8106f10 0x7f52c810ea90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009ca0 tx=0x7f52c002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:02:02.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.471+0000 7f52d00bc640 1 -- 172.21.15.5:0/3784657528 shutdown_connections 2024-09-19T06:02:02.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.471+0000 7f52d00bc640 1 --2- 172.21.15.5:0/3784657528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c8106f10 0x7f52c810ea90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:02.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.471+0000 7f52d00bc640 1 --2- 172.21.15.5:0/3784657528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8102790 0x7f52c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:02.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.471+0000 7f52d00bc640 1 --2- 172.21.15.5:0/3784657528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8101de0 0x7f52c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:02.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.471+0000 7f52d00bc640 1 -- 172.21.15.5:0/3784657528 >> 172.21.15.5:0/3784657528 conn(0x7f52c80fd8c0 msgr2=0x7f52c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:02.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.471+0000 7f52d00bc640 1 -- 172.21.15.5:0/3784657528 shutdown_connections 2024-09-19T06:02:02.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.471+0000 7f52d00bc640 1 -- 172.21.15.5:0/3784657528 wait complete. 2024-09-19T06:02:02.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.471+0000 7f52d00bc640 1 Processor -- start 2024-09-19T06:02:02.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.471+0000 7f52d00bc640 1 -- start start 2024-09-19T06:02:02.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52d00bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c8101de0 0x7f52c8198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:02.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52d00bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8102790 0x7f52c8198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:02.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52d00bc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8106f10 0x7f52c819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:02.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52d00bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8111e70 con 0x7f52c8101de0 2024-09-19T06:02:02.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52d00bc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8111cf0 con 0x7f52c8106f10 2024-09-19T06:02:02.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52d00bc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8111ff0 con 0x7f52c8102790 2024-09-19T06:02:02.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52cde58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c8101de0 0x7f52c8198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:02.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52cde58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c8101de0 0x7f52c8198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54988/0 (socket says 172.21.15.5:54988) 2024-09-19T06:02:02.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52cde58640 1 -- 172.21.15.5:0/3383321880 learned_addr learned my addr 172.21.15.5:0/3383321880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:02.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52ce659640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8106f10 0x7f52c819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:02.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52cd657640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8102790 0x7f52c8198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:02.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52cde58640 1 -- 172.21.15.5:0/3383321880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8102790 msgr2=0x7f52c8198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:02.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52cde58640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8102790 0x7f52c8198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:02.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52cde58640 1 -- 172.21.15.5:0/3383321880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8106f10 msgr2=0x7f52c819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:02.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52cde58640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8106f10 0x7f52c819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52cde58640 1 -- 172.21.15.5:0/3383321880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c819d1f0 con 0x7f52c8101de0 2024-09-19T06:02:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52cd657640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8102790 0x7f52c8198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52cde58640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c8101de0 0x7f52c8198240 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f52c400d950 tx=0x7f52c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52beffd640 1 -- 172.21.15.5:0/3383321880 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4017070 con 0x7f52c8101de0 2024-09-19T06:02:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52d00bc640 1 -- 172.21.15.5:0/3383321880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c819d4e0 con 0x7f52c8101de0 2024-09-19T06:02:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52beffd640 1 -- 172.21.15.5:0/3383321880 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52c4009c30 con 0x7f52c8101de0 2024-09-19T06:02:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.472+0000 7f52beffd640 1 -- 172.21.15.5:0/3383321880 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c400f620 con 0x7f52c8101de0 2024-09-19T06:02:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.473+0000 7f52d00bc640 1 -- 172.21.15.5:0/3383321880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c81a4dc0 con 0x7f52c8101de0 2024-09-19T06:02:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.474+0000 7f52d00bc640 1 -- 172.21.15.5:0/3383321880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c81021c0 con 0x7f52c8101de0 2024-09-19T06:02:02.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.475+0000 7f52beffd640 1 -- 172.21.15.5:0/3383321880 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f52c40109a0 con 0x7f52c8101de0 2024-09-19T06:02:02.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.476+0000 7f52beffd640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f52b40a32d0 0x7f52b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:02.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.476+0000 7f52cd657640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f52b40a32d0 0x7f52b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:02.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.476+0000 7f52beffd640 1 -- 172.21.15.5:0/3383321880 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f52c40cf7b0 con 0x7f52c8101de0 2024-09-19T06:02:02.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.476+0000 7f52cd657640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f52b40a32d0 0x7f52b40a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f52c8108870 tx=0x7f52b8005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:02.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.480+0000 7f52beffd640 1 -- 172.21.15.5:0/3383321880 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f52c4009de0 con 0x7f52c8101de0 2024-09-19T06:02:02.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:02 smithi157 ceph-mon[35346]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-09-19T06:02:02.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:02 smithi049 ceph-mon[34640]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-09-19T06:02:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.656+0000 7f52d00bc640 1 -- 172.21.15.5:0/3383321880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52c81995a0 con 0x7f52c8101de0 2024-09-19T06:02:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.657+0000 7f52beffd640 1 -- 172.21.15.5:0/3383321880 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22702 (secure 0 0 0) 0x7f52c409c360 con 0x7f52c8101de0 2024-09-19T06:02:02.659 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:02.659 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","created":"2024-09-19T05:53:51.454068+0000","modified":"2024-09-19T06:01:39.826379+0000","last_up_change":"2024-09-19T06:00:38.837916+0000","last_in_change":"2024-09-19T06:00:15.609776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:56:55.213931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:01:25.648588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:01:27.760408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f3c10d60-d354-407f-9cdf-099344599844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6803","nonce":3442981810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6805","nonce":3442981810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6809","nonce":3442981810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6807","nonce":3442981810}]},"public_addr":"172.21.15.5:6803/3442981810","cluster_addr":"172.21.15.5:6805/3442981810","heartbeat_back_addr":"172.21.15.5:6809/3442981810","heartbeat_front_addr":"172.21.15.5:6807/3442981810","state":["exists","up"]},{"osd":1,"uuid":"5f40f636-7761-45ac-87f3-910a2fa2feb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448265014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6803","nonce":1448265014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6807","nonce":1448265014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6805","nonce":1448265014}]},"public_addr":"172.21.15.49:6801/1448265014","cluster_addr":"172.21.15.49:6803/1448265014","heartbeat_back_addr":"172.21.15.49:6807/1448265014","heartbeat_front_addr":"172.21.15.49:6805/1448265014","state":["exists","up"]},{"osd":2,"uuid":"ad00940c-4fdd-4f5f-9eaa-b1f38bcac526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6801","nonce":2319626622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6803","nonce":2319626622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6807","nonce":2319626622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6805","nonce":2319626622}]},"public_addr":"172.21.15.157:6801/2319626622","cluster_addr":"172.21.15.157:6803/2319626622","heartbeat_back_addr":"172.21.15.157:6807/2319626622","heartbeat_front_addr":"172.21.15.157:6805/2319626622","state":["exists","up"]},{"osd":3,"uuid":"63fec329-1b62-4c5e-b8ca-084a84d4d762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6811","nonce":1452491753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6813","nonce":1452491753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6817","nonce":1452491753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6815","nonce":1452491753}]},"public_addr":"172.21.15.5:6811/1452491753","cluster_addr":"172.21.15.5:6813/1452491753","heartbeat_back_addr":"172.21.15.5:6817/1452491753","heartbeat_front_addr":"172.21.15.5:6815/1452491753","state":["exists","up"]},{"osd":4,"uuid":"ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6809","nonce":3114952580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6811","nonce":3114952580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6815","nonce":3114952580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6813","nonce":3114952580}]},"public_addr":"172.21.15.49:6809/3114952580","cluster_addr":"172.21.15.49:6811/3114952580","heartbeat_back_addr":"172.21.15.49:6815/3114952580","heartbeat_front_addr":"172.21.15.49:6813/3114952580","state":["exists","up"]},{"osd":5,"uuid":"21a7a7d3-ce42-4688-bfec-714be635eaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6809","nonce":3219063971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6811","nonce":3219063971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6815","nonce":3219063971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6813","nonce":3219063971}]},"public_addr":"172.21.15.157:6809/3219063971","cluster_addr":"172.21.15.157:6811/3219063971","heartbeat_back_addr":"172.21.15.157:6815/3219063971","heartbeat_front_addr":"172.21.15.157:6813/3219063971","state":["exists","up"]},{"osd":6,"uuid":"cbef0e08-d6b0-4515-a2ee-2c81b048583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6819","nonce":2957795194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6821","nonce":2957795194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6825","nonce":2957795194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6823","nonce":2957795194}]},"public_addr":"172.21.15.5:6819/2957795194","cluster_addr":"172.21.15.5:6821/2957795194","heartbeat_back_addr":"172.21.15.5:6825/2957795194","heartbeat_front_addr":"172.21.15.5:6823/2957795194","state":["exists","up"]},{"osd":7,"uuid":"d261ac55-532d-4aea-9b3e-ed2b4eeb1da6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6817","nonce":2959731400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6819","nonce":2959731400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6823","nonce":2959731400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6821","nonce":2959731400}]},"public_addr":"172.21.15.49:6817/2959731400","cluster_addr":"172.21.15.49:6819/2959731400","heartbeat_back_addr":"172.21.15.49:6823/2959731400","heartbeat_front_addr":"172.21.15.49:6821/2959731400","state":["exists","up"]},{"osd":8,"uuid":"15d0eaf7-d604-4546-b393-4d6d39018f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6817","nonce":290026859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6819","nonce":290026859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6823","nonce":290026859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6821","nonce":290026859}]},"public_addr":"172.21.15.157:6817/290026859","cluster_addr":"172.21.15.157:6819/290026859","heartbeat_back_addr":"172.21.15.157:6823/290026859","heartbeat_front_addr":"172.21.15.157:6821/290026859","state":["exists","up"]},{"osd":9,"uuid":"30c08537-f593-4299-bfa1-fa7ba8a95169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6827","nonce":4122448237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6829","nonce":4122448237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6833","nonce":4122448237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6831","nonce":4122448237}]},"public_addr":"172.21.15.5:6827/4122448237","cluster_addr":"172.21.15.5:6829/4122448237","heartbeat_back_addr":"172.21.15.5:6833/4122448237","heartbeat_front_addr":"172.21.15.5:6831/4122448237","state":["exists","up"]},{"osd":10,"uuid":"68821e98-bd81-49ec-8cd6-51fb9880b553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6825","nonce":2551748002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6827","nonce":2551748002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6831","nonce":2551748002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6829","nonce":2551748002}]},"public_addr":"172.21.15.49:6825/2551748002","cluster_addr":"172.21.15.49:6827/2551748002","heartbeat_back_addr":"172.21.15.49:6831/2551748002","heartbeat_front_addr":"172.21.15.49:6829/2551748002","state":["exists","up"]},{"osd":11,"uuid":"7e443b46-2697-4d0b-8452-868463a1ee1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6825","nonce":1996582544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6827","nonce":1996582544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6831","nonce":1996582544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6829","nonce":1996582544}]},"public_addr":"172.21.15.157:6825/1996582544","cluster_addr":"172.21.15.157:6827/1996582544","heartbeat_back_addr":"172.21.15.157:6831/1996582544","heartbeat_front_addr":"172.21.15.157:6829/1996582544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:01.453592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:27.194705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:51.525385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:17.676749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:43.075555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:07.258154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:32.112583+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:57.306534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:21.320950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:47.164610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:12.548337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:37.262580+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4102275559":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/2593868621":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/1340147787":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6800/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/850678758":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6800/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2930274429":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/1367598148":"2024-09-20T05:54:15.334457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:02:02.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.660+0000 7f52d00bc640 1 -- 172.21.15.5:0/3383321880 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f52b40a32d0 msgr2=0x7f52b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:02.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.660+0000 7f52d00bc640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f52b40a32d0 0x7f52b40a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f52c8108870 tx=0x7f52b8005d20 comp rx=0 tx=0).stop 2024-09-19T06:02:02.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.660+0000 7f52d00bc640 1 -- 172.21.15.5:0/3383321880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c8101de0 msgr2=0x7f52c8198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:02.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.660+0000 7f52d00bc640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c8101de0 0x7f52c8198240 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f52c400d950 tx=0x7f52c400de10 comp rx=0 tx=0).stop 2024-09-19T06:02:02.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.660+0000 7f52d00bc640 1 -- 172.21.15.5:0/3383321880 shutdown_connections 2024-09-19T06:02:02.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.660+0000 7f52d00bc640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f52b40a32d0 0x7f52b40a5790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:02.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.660+0000 7f52d00bc640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8106f10 0x7f52c819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:02.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.660+0000 7f52d00bc640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8102790 0x7f52c8198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:02.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.660+0000 7f52d00bc640 1 --2- 172.21.15.5:0/3383321880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c8101de0 0x7f52c8198240 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:02.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.660+0000 7f52d00bc640 1 -- 172.21.15.5:0/3383321880 >> 172.21.15.5:0/3383321880 conn(0x7f52c80fd8c0 msgr2=0x7f52c80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:02.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.661+0000 7f52d00bc640 1 -- 172.21.15.5:0/3383321880 shutdown_connections 2024-09-19T06:02:02.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:02.661+0000 7f52d00bc640 1 -- 172.21.15.5:0/3383321880 wait complete. 2024-09-19T06:02:03.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T06:02:03.401 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T06:02:03.401 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T06:02:03.402 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T06:02:03.402 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T06:02:03.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T06:02:03.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T06:02:03.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T06:02:03.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T06:02:03.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T06:02:03.405 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T06:02:03.405 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T06:02:03.448 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:03 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3383321880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:02:03.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:03 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3383321880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:02:03.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:03 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3383321880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:02:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:04.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:04 smithi005 ceph-mon[30955]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T06:02:04.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:04 smithi157 ceph-mon[35346]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T06:02:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:04 smithi049 ceph-mon[34640]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T06:02:04.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.706+0000 7f5abf816640 1 -- 172.21.15.5:0/556219776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab8100e20 msgr2=0x7f5ab8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:04.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.706+0000 7f5abf816640 1 --2- 172.21.15.5:0/556219776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab8100e20 0x7f5ab8101200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009900 tx=0x7f5ab402a480 comp rx=0 tx=0).stop 2024-09-19T06:02:04.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.707+0000 7f5abf816640 1 -- 172.21.15.5:0/556219776 shutdown_connections 2024-09-19T06:02:04.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.707+0000 7f5abf816640 1 --2- 172.21.15.5:0/556219776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab8102ef0 0x7f5ab810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:04.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.707+0000 7f5abf816640 1 --2- 172.21.15.5:0/556219776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ab8102570 0x7f5ab81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:04.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.707+0000 7f5abf816640 1 --2- 172.21.15.5:0/556219776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab8100e20 0x7f5ab8101200 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:04.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.707+0000 7f5abf816640 1 -- 172.21.15.5:0/556219776 >> 172.21.15.5:0/556219776 conn(0x7f5ab80fc7d0 msgr2=0x7f5ab80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:04.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.707+0000 7f5abf816640 1 -- 172.21.15.5:0/556219776 shutdown_connections 2024-09-19T06:02:04.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.707+0000 7f5abf816640 1 -- 172.21.15.5:0/556219776 wait complete. 2024-09-19T06:02:04.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.707+0000 7f5abf816640 1 Processor -- start 2024-09-19T06:02:04.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.707+0000 7f5abf816640 1 -- start start 2024-09-19T06:02:04.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abf816640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ab8100e20 0x7f5ab8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:04.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abf816640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab8102570 0x7f5ab81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:04.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abf816640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab8102ef0 0x7f5ab819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:04.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abf816640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8110e00 con 0x7f5ab8102ef0 2024-09-19T06:02:04.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abf816640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8110c80 con 0x7f5ab8100e20 2024-09-19T06:02:04.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abf816640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8110f80 con 0x7f5ab8102570 2024-09-19T06:02:04.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abddb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab8102ef0 0x7f5ab819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:04.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abddb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab8102ef0 0x7f5ab819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42674/0 (socket says 172.21.15.5:42674) 2024-09-19T06:02:04.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abddb3640 1 -- 172.21.15.5:0/3316828369 learned_addr learned my addr 172.21.15.5:0/3316828369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:04.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abd5b2640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ab8100e20 0x7f5ab8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:04.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abcdb1640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab8102570 0x7f5ab81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abddb3640 1 -- 172.21.15.5:0/3316828369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab8102570 msgr2=0x7f5ab81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abddb3640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab8102570 0x7f5ab81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abddb3640 1 -- 172.21.15.5:0/3316828369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ab8100e20 msgr2=0x7f5ab8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abddb3640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ab8100e20 0x7f5ab8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abddb3640 1 -- 172.21.15.5:0/3316828369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab819d440 con 0x7f5ab8102ef0 2024-09-19T06:02:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abd5b2640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ab8100e20 0x7f5ab8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.708+0000 7f5abddb3640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab8102ef0 0x7f5ab819cd60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000eaa0 tx=0x7f5ab000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.709+0000 7f5aae7fc640 1 -- 172.21.15.5:0/3316828369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000cdb0 con 0x7f5ab8102ef0 2024-09-19T06:02:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.709+0000 7f5aae7fc640 1 -- 172.21.15.5:0/3316828369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ab0005020 con 0x7f5ab8102ef0 2024-09-19T06:02:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.709+0000 7f5abf816640 1 -- 172.21.15.5:0/3316828369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab819d730 con 0x7f5ab8102ef0 2024-09-19T06:02:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.709+0000 7f5aae7fc640 1 -- 172.21.15.5:0/3316828369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00116d0 con 0x7f5ab8102ef0 2024-09-19T06:02:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.709+0000 7f5abf816640 1 -- 172.21.15.5:0/3316828369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab81a5010 con 0x7f5ab8102ef0 2024-09-19T06:02:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.710+0000 7f5abf816640 1 -- 172.21.15.5:0/3316828369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ab8062980 con 0x7f5ab8102ef0 2024-09-19T06:02:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.711+0000 7f5aae7fc640 1 -- 172.21.15.5:0/3316828369 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f5ab00051d0 con 0x7f5ab8102ef0 2024-09-19T06:02:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.711+0000 7f5aae7fc640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5aa40a32d0 0x7f5aa40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.712+0000 7f5abd5b2640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5aa40a32d0 0x7f5aa40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.712+0000 7f5aae7fc640 1 -- 172.21.15.5:0/3316828369 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5ab00cfcc0 con 0x7f5ab8102ef0 2024-09-19T06:02:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.712+0000 7f5aae7fc640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580] conn(0x7f5aa40acf00 0x7f5aa40af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.712+0000 7f5aae7fc640 1 -- 172.21.15.5:0/3316828369 --> [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ab0018ec0 con 0x7f5aa40acf00 2024-09-19T06:02:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.712+0000 7f5aae7fc640 1 -- 172.21.15.5:0/3316828369 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f5ab00d22e0 con 0x7f5ab8102ef0 2024-09-19T06:02:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.712+0000 7f5abd5b2640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5aa40a32d0 0x7f5aa40a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4005ab0 tx=0x7f5ab4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.712+0000 7f5abcdb1640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580] conn(0x7f5aa40acf00 0x7f5aa40af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.713+0000 7f5abcdb1640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580] conn(0x7f5aa40acf00 0x7f5aa40af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.713+0000 7f5aae7fc640 1 -- 172.21.15.5:0/3316828369 <== osd.4 v2:172.21.15.49:6808/3114952580 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ab0018ec0 con 0x7f5aa40acf00 2024-09-19T06:02:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.741+0000 7f5abf816640 1 -- 172.21.15.5:0/3316828369 --> [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ab8062e00 con 0x7f5aa40acf00 2024-09-19T06:02:04.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.741+0000 7f5aae7fc640 1 -- 172.21.15.5:0/3316828369 <== osd.4 v2:172.21.15.49:6808/3114952580 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ab8062e00 con 0x7f5aa40acf00 2024-09-19T06:02:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.742+0000 7f5abf816640 1 -- 172.21.15.5:0/3316828369 >> [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580] conn(0x7f5aa40acf00 msgr2=0x7f5aa40af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.742+0000 7f5abf816640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580] conn(0x7f5aa40acf00 0x7f5aa40af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.742+0000 7f5abf816640 1 -- 172.21.15.5:0/3316828369 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5aa40a32d0 msgr2=0x7f5aa40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.742+0000 7f5abf816640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5aa40a32d0 0x7f5aa40a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4005ab0 tx=0x7f5ab4005a20 comp rx=0 tx=0).stop 2024-09-19T06:02:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.742+0000 7f5abf816640 1 -- 172.21.15.5:0/3316828369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab8102ef0 msgr2=0x7f5ab819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.742+0000 7f5abf816640 1 --2- 172.21.15.5:0/3316828369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab8102ef0 0x7f5ab819cd60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000eaa0 tx=0x7f5ab000ef60 comp rx=0 tx=0).stop 2024-09-19T06:02:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.742+0000 7f5abddb3640 1 -- 172.21.15.5:0/3316828369 reap_dead start 2024-09-19T06:02:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.742+0000 7f5abf816640 1 -- 172.21.15.5:0/3316828369 shutdown_connections 2024-09-19T06:02:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.742+0000 7f5abf816640 1 -- 172.21.15.5:0/3316828369 >> 172.21.15.5:0/3316828369 conn(0x7f5ab80fc7d0 msgr2=0x7f5ab806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.742+0000 7f5abf816640 1 -- 172.21.15.5:0/3316828369 shutdown_connections 2024-09-19T06:02:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:04.742+0000 7f5abf816640 1 -- 172.21.15.5:0/3316828369 wait complete. 2024-09-19T06:02:05.526 INFO:teuthology.orchestra.run.smithi005.stdout:137438953527 2024-09-19T06:02:05.526 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T06:02:05.601 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:05.602 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:05.612 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:05.613 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:05.615 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:06.222 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:06.224 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:06.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:06 smithi005 ceph-mon[30955]: pgmap v296: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-19T06:02:06.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:06 smithi157 ceph-mon[35346]: pgmap v296: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-19T06:02:06.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:06 smithi049 ceph-mon[34640]: pgmap v296: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-19T06:02:06.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.553+0000 7fa5c3d92640 1 -- 172.21.15.5:0/398459367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc106f10 msgr2=0x7fa5bc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:06.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.553+0000 7fa5c3d92640 1 --2- 172.21.15.5:0/398459367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc106f10 0x7fa5bc10ea90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009ca0 tx=0x7fa5b402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.553+0000 7fa5c3d92640 1 -- 172.21.15.5:0/398459367 shutdown_connections 2024-09-19T06:02:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.553+0000 7fa5c3d92640 1 --2- 172.21.15.5:0/398459367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc106f10 0x7fa5bc10ea90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.553+0000 7fa5c3d92640 1 --2- 172.21.15.5:0/398459367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5bc102790 0x7fa5bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.553+0000 7fa5c3d92640 1 --2- 172.21.15.5:0/398459367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5bc101de0 0x7fa5bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.553+0000 7fa5c3d92640 1 -- 172.21.15.5:0/398459367 >> 172.21.15.5:0/398459367 conn(0x7fa5bc0fd8c0 msgr2=0x7fa5bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.553+0000 7fa5c3d92640 1 -- 172.21.15.5:0/398459367 shutdown_connections 2024-09-19T06:02:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.553+0000 7fa5c3d92640 1 -- 172.21.15.5:0/398459367 wait complete. 2024-09-19T06:02:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c3d92640 1 Processor -- start 2024-09-19T06:02:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c3d92640 1 -- start start 2024-09-19T06:02:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c3d92640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5bc101de0 0x7fa5bc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c3d92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc102790 0x7fa5bc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c3d92640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5bc106f10 0x7fa5bc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c3d92640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc111e20 con 0x7fa5bc102790 2024-09-19T06:02:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c3d92640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc111ca0 con 0x7fa5bc101de0 2024-09-19T06:02:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c3d92640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc111fa0 con 0x7fa5bc106f10 2024-09-19T06:02:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c132d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc102790 0x7fa5bc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c132d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc102790 0x7fa5bc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42688/0 (socket says 172.21.15.5:42688) 2024-09-19T06:02:06.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c132d640 1 -- 172.21.15.5:0/4254139864 learned_addr learned my addr 172.21.15.5:0/4254139864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:06.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c232f640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5bc106f10 0x7fa5bc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:06.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c1b2e640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5bc101de0 0x7fa5bc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:06.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c132d640 1 -- 172.21.15.5:0/4254139864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5bc106f10 msgr2=0x7fa5bc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c132d640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5bc106f10 0x7fa5bc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c132d640 1 -- 172.21.15.5:0/4254139864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5bc101de0 msgr2=0x7fa5bc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c132d640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5bc101de0 0x7fa5bc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c132d640 1 -- 172.21.15.5:0/4254139864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5bc1a6800 con 0x7fa5bc102790 2024-09-19T06:02:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.554+0000 7fa5c1b2e640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5bc101de0 0x7fa5bc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.555+0000 7fa5c132d640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc102790 0x7fa5bc074a30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00c890 tx=0x7fa5ac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.555+0000 7fa5b2ffd640 1 -- 172.21.15.5:0/4254139864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac004220 con 0x7fa5bc102790 2024-09-19T06:02:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.555+0000 7fa5b2ffd640 1 -- 172.21.15.5:0/4254139864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5ac0043c0 con 0x7fa5bc102790 2024-09-19T06:02:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.555+0000 7fa5c3d92640 1 -- 172.21.15.5:0/4254139864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5bc1a6af0 con 0x7fa5bc102790 2024-09-19T06:02:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.555+0000 7fa5b2ffd640 1 -- 172.21.15.5:0/4254139864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac004a00 con 0x7fa5bc102790 2024-09-19T06:02:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.555+0000 7fa5c3d92640 1 -- 172.21.15.5:0/4254139864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5bc1a7030 con 0x7fa5bc102790 2024-09-19T06:02:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.556+0000 7fa5c3d92640 1 -- 172.21.15.5:0/4254139864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5bc0628c0 con 0x7fa5bc102790 2024-09-19T06:02:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.557+0000 7fa5b2ffd640 1 -- 172.21.15.5:0/4254139864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fa5ac004ba0 con 0x7fa5bc102790 2024-09-19T06:02:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.558+0000 7fa5b2ffd640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa5a80a32d0 0x7fa5a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.558+0000 7fa5c1b2e640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa5a80a32d0 0x7fa5a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.558+0000 7fa5b2ffd640 1 -- 172.21.15.5:0/4254139864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa5ac0cfbe0 con 0x7fa5bc102790 2024-09-19T06:02:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.558+0000 7fa5b2ffd640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753] conn(0x7fa5a80acf00 0x7fa5a80af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.558+0000 7fa5b2ffd640 1 -- 172.21.15.5:0/4254139864 --> [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5ac010e50 con 0x7fa5a80acf00 2024-09-19T06:02:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.558+0000 7fa5b2ffd640 1 -- 172.21.15.5:0/4254139864 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fa5ac0d0030 con 0x7fa5bc102790 2024-09-19T06:02:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.558+0000 7fa5c232f640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753] conn(0x7fa5a80acf00 0x7fa5a80af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.558+0000 7fa5c1b2e640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa5a80a32d0 0x7fa5a80a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8005d00 tx=0x7fa5b8009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.559+0000 7fa5c232f640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753] conn(0x7fa5a80acf00 0x7fa5a80af3a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.559+0000 7fa5b2ffd640 1 -- 172.21.15.5:0/4254139864 <== osd.3 v2:172.21.15.5:6810/1452491753 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa5ac010e50 con 0x7fa5a80acf00 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.574+0000 7fa5c3d92640 1 -- 172.21.15.5:0/4254139864 --> [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5bc062d40 con 0x7fa5a80acf00 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.575+0000 7fa5b2ffd640 1 -- 172.21.15.5:0/4254139864 <== osd.3 v2:172.21.15.5:6810/1452491753 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa5bc062d40 con 0x7fa5a80acf00 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.575+0000 7fa5c3d92640 1 -- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753] conn(0x7fa5a80acf00 msgr2=0x7fa5a80af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.575+0000 7fa5c3d92640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753] conn(0x7fa5a80acf00 0x7fa5a80af3a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.575+0000 7fa5c3d92640 1 -- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa5a80a32d0 msgr2=0x7fa5a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.575+0000 7fa5c3d92640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa5a80a32d0 0x7fa5a80a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8005d00 tx=0x7fa5b8009550 comp rx=0 tx=0).stop 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.575+0000 7fa5c3d92640 1 -- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc102790 msgr2=0x7fa5bc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.575+0000 7fa5c3d92640 1 --2- 172.21.15.5:0/4254139864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc102790 0x7fa5bc074a30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00c890 tx=0x7fa5ac00cd50 comp rx=0 tx=0).stop 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.575+0000 7fa5c232f640 1 -- 172.21.15.5:0/4254139864 reap_dead start 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.575+0000 7fa5c3d92640 1 -- 172.21.15.5:0/4254139864 shutdown_connections 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.575+0000 7fa5c3d92640 1 -- 172.21.15.5:0/4254139864 >> 172.21.15.5:0/4254139864 conn(0x7fa5bc0fd8c0 msgr2=0x7fa5bc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.575+0000 7fa5c3d92640 1 -- 172.21.15.5:0/4254139864 shutdown_connections 2024-09-19T06:02:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.575+0000 7fa5c3d92640 1 -- 172.21.15.5:0/4254139864 wait complete. 2024-09-19T06:02:06.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.965+0000 7f7b50e54640 1 -- 172.21.15.5:0/1488038693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b4c1062b0 msgr2=0x7f7b4c106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:06.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.965+0000 7f7b50e54640 1 --2- 172.21.15.5:0/1488038693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b4c1062b0 0x7f7b4c106690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00ade0 tx=0x7f7b3c02b7e0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.966+0000 7f7b50e54640 1 -- 172.21.15.5:0/1488038693 shutdown_connections 2024-09-19T06:02:06.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.966+0000 7f7b50e54640 1 --2- 172.21.15.5:0/1488038693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c100c00 0x7f7b4c10d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.966+0000 7f7b50e54640 1 --2- 172.21.15.5:0/1488038693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b4c1002a0 0x7f7b4c1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.966+0000 7f7b50e54640 1 --2- 172.21.15.5:0/1488038693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b4c1062b0 0x7f7b4c106690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.966+0000 7f7b50e54640 1 -- 172.21.15.5:0/1488038693 >> 172.21.15.5:0/1488038693 conn(0x7f7b4c0fc5c0 msgr2=0x7f7b4c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:06.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.966+0000 7f7b50e54640 1 -- 172.21.15.5:0/1488038693 shutdown_connections 2024-09-19T06:02:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.966+0000 7f7b50e54640 1 -- 172.21.15.5:0/1488038693 wait complete. 2024-09-19T06:02:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.966+0000 7f7b50e54640 1 Processor -- start 2024-09-19T06:02:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.966+0000 7f7b50e54640 1 -- start start 2024-09-19T06:02:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b50e54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b4c1002a0 0x7f7b4c101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b50e54640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c100c00 0x7f7b4c102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b50e54640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b4c1062b0 0x7f7b4c1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b50e54640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c10fc90 con 0x7f7b4c1002a0 2024-09-19T06:02:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b50e54640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c10fb10 con 0x7f7b4c100c00 2024-09-19T06:02:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b50e54640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c10fe10 con 0x7f7b4c1062b0 2024-09-19T06:02:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b4c1002a0 0x7f7b4c101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b4c1002a0 0x7f7b4c101f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42696/0 (socket says 172.21.15.5:42696) 2024-09-19T06:02:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b4a59c640 1 -- 172.21.15.5:0/3332315859 learned_addr learned my addr 172.21.15.5:0/3332315859 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b49d9b640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c100c00 0x7f7b4c102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b4ad9d640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b4c1062b0 0x7f7b4c1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b4a59c640 1 -- 172.21.15.5:0/3332315859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b4c1062b0 msgr2=0x7f7b4c1029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b4a59c640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b4c1062b0 0x7f7b4c1029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b4a59c640 1 -- 172.21.15.5:0/3332315859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c100c00 msgr2=0x7f7b4c102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b4a59c640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c100c00 0x7f7b4c102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b4a59c640 1 -- 172.21.15.5:0/3332315859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c1a68b0 con 0x7f7b4c1002a0 2024-09-19T06:02:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.967+0000 7f7b4a59c640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b4c1002a0 0x7f7b4c101f20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00adb0 tx=0x7f7b3c0326b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.968+0000 7f7b3b7fe640 1 -- 172.21.15.5:0/3332315859 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c040070 con 0x7f7b4c1002a0 2024-09-19T06:02:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.968+0000 7f7b50e54640 1 -- 172.21.15.5:0/3332315859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c1a6b40 con 0x7f7b4c1002a0 2024-09-19T06:02:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.968+0000 7f7b3b7fe640 1 -- 172.21.15.5:0/3332315859 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b3c004040 con 0x7f7b4c1002a0 2024-09-19T06:02:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.968+0000 7f7b3b7fe640 1 -- 172.21.15.5:0/3332315859 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c030bc0 con 0x7f7b4c1002a0 2024-09-19T06:02:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.968+0000 7f7b50e54640 1 -- 172.21.15.5:0/3332315859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c1a7070 con 0x7f7b4c1002a0 2024-09-19T06:02:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.969+0000 7f7b50e54640 1 -- 172.21.15.5:0/3332315859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b4c0628c0 con 0x7f7b4c1002a0 2024-09-19T06:02:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.970+0000 7f7b3b7fe640 1 -- 172.21.15.5:0/3332315859 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f7b3c004580 con 0x7f7b4c1002a0 2024-09-19T06:02:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.970+0000 7f7b3b7fe640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f7b340a32d0 0x7f7b340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.971+0000 7f7b49d9b640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f7b340a32d0 0x7f7b340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.971+0000 7f7b3b7fe640 1 -- 172.21.15.5:0/3332315859 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7b3c0f1600 con 0x7f7b4c1002a0 2024-09-19T06:02:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.971+0000 7f7b3b7fe640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810] conn(0x7f7b340acf00 0x7f7b340af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.971+0000 7f7b3b7fe640 1 -- 172.21.15.5:0/3332315859 --> [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b3c030d60 con 0x7f7b340acf00 2024-09-19T06:02:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.971+0000 7f7b3b7fe640 1 -- 172.21.15.5:0/3332315859 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f7b3c0f1a50 con 0x7f7b4c1002a0 2024-09-19T06:02:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.971+0000 7f7b4ad9d640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810] conn(0x7f7b340acf00 0x7f7b340af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.971+0000 7f7b49d9b640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f7b340a32d0 0x7f7b340a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c1039b0 tx=0x7f7b4000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.971+0000 7f7b4ad9d640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810] conn(0x7f7b340acf00 0x7f7b340af3a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.972+0000 7f7b3b7fe640 1 -- 172.21.15.5:0/3332315859 <== osd.0 v2:172.21.15.5:6802/3442981810 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b3c030d60 con 0x7f7b340acf00 2024-09-19T06:02:06.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.985+0000 7f7b50e54640 1 -- 172.21.15.5:0/3332315859 --> [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b4c062cc0 con 0x7f7b340acf00 2024-09-19T06:02:06.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.986+0000 7f7b3b7fe640 1 -- 172.21.15.5:0/3332315859 <== osd.0 v2:172.21.15.5:6802/3442981810 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7b4c062cc0 con 0x7f7b340acf00 2024-09-19T06:02:06.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.986+0000 7f7b50e54640 1 -- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810] conn(0x7f7b340acf00 msgr2=0x7f7b340af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:06.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.986+0000 7f7b50e54640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810] conn(0x7f7b340acf00 0x7f7b340af3a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.986+0000 7f7b50e54640 1 -- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f7b340a32d0 msgr2=0x7f7b340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:06.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.986+0000 7f7b50e54640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f7b340a32d0 0x7f7b340a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c1039b0 tx=0x7f7b4000a250 comp rx=0 tx=0).stop 2024-09-19T06:02:06.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.986+0000 7f7b50e54640 1 -- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b4c1002a0 msgr2=0x7f7b4c101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:06.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.986+0000 7f7b50e54640 1 --2- 172.21.15.5:0/3332315859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b4c1002a0 0x7f7b4c101f20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00adb0 tx=0x7f7b3c0326b0 comp rx=0 tx=0).stop 2024-09-19T06:02:06.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.986+0000 7f7b4ad9d640 1 -- 172.21.15.5:0/3332315859 reap_dead start 2024-09-19T06:02:06.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.986+0000 7f7b50e54640 1 -- 172.21.15.5:0/3332315859 shutdown_connections 2024-09-19T06:02:06.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.986+0000 7f7b50e54640 1 -- 172.21.15.5:0/3332315859 >> 172.21.15.5:0/3332315859 conn(0x7f7b4c0fc5c0 msgr2=0x7f7b4c0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:06.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.986+0000 7f7b50e54640 1 -- 172.21.15.5:0/3332315859 shutdown_connections 2024-09-19T06:02:06.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:06.986+0000 7f7b50e54640 1 -- 172.21.15.5:0/3332315859 wait complete. 2024-09-19T06:02:07.642 INFO:teuthology.orchestra.run.smithi005.stdout:34359738444 2024-09-19T06:02:07.643 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T06:02:08.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:08 smithi157 ceph-mon[35346]: pgmap v297: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T06:02:08.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:08 smithi005 ceph-mon[30955]: pgmap v297: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T06:02:08.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:08 smithi049 ceph-mon[34640]: pgmap v297: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T06:02:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.238+0000 7f20d5928640 1 -- 172.21.15.5:0/2840192513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d0101420 msgr2=0x7f20d010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.238+0000 7f20d5928640 1 --2- 172.21.15.5:0/2840192513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d0101420 0x7f20d010d950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009cd0 tx=0x7f20c802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:02:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.238+0000 7f20d5928640 1 -- 172.21.15.5:0/2840192513 shutdown_connections 2024-09-19T06:02:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.238+0000 7f20d5928640 1 --2- 172.21.15.5:0/2840192513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d0101420 0x7f20d010d950 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.238+0000 7f20d5928640 1 --2- 172.21.15.5:0/2840192513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d0100a80 0x7f20d0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.238+0000 7f20d5928640 1 --2- 172.21.15.5:0/2840192513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d0106a80 0x7f20d0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.238+0000 7f20d5928640 1 -- 172.21.15.5:0/2840192513 >> 172.21.15.5:0/2840192513 conn(0x7f20d00fc7a0 msgr2=0x7f20d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.238+0000 7f20d5928640 1 -- 172.21.15.5:0/2840192513 shutdown_connections 2024-09-19T06:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.238+0000 7f20d5928640 1 -- 172.21.15.5:0/2840192513 wait complete. 2024-09-19T06:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.239+0000 7f20d5928640 1 Processor -- start 2024-09-19T06:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.239+0000 7f20d5928640 1 -- start start 2024-09-19T06:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.239+0000 7f20d5928640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d0100a80 0x7f20d0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.239+0000 7f20d5928640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d0101420 0x7f20d0198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.239+0000 7f20d5928640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d0106a80 0x7f20d019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.239+0000 7f20d5928640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20d0110d30 con 0x7f20d0100a80 2024-09-19T06:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.239+0000 7f20d5928640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f20d0110bb0 con 0x7f20d0106a80 2024-09-19T06:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.239+0000 7f20d5928640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20d0110eb0 con 0x7f20d0101420 2024-09-19T06:02:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.239+0000 7f20ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d0100a80 0x7f20d0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.239+0000 7f20ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d0100a80 0x7f20d0198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42718/0 (socket says 172.21.15.5:42718) 2024-09-19T06:02:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.239+0000 7f20ceffd640 1 -- 172.21.15.5:0/1271415213 learned_addr learned my addr 172.21.15.5:0/1271415213 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.240+0000 7f20ceffd640 1 -- 172.21.15.5:0/1271415213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d0101420 msgr2=0x7f20d0198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.240+0000 7f20ceffd640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d0101420 0x7f20d0198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.240+0000 7f20ceffd640 1 -- 172.21.15.5:0/1271415213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d0106a80 msgr2=0x7f20d019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.240+0000 7f20cf7fe640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d0106a80 0x7f20d019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.240+0000 7f20ceffd640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d0106a80 0x7f20d019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.240+0000 7f20ceffd640 1 -- 172.21.15.5:0/1271415213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d019d300 con 0x7f20d0100a80 2024-09-19T06:02:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.240+0000 7f20ceffd640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d0100a80 0x7f20d0198350 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20c000c890 tx=0x7f20c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.240+0000 7f20d4926640 1 -- 172.21.15.5:0/1271415213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0004220 con 0x7f20d0100a80 2024-09-19T06:02:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.240+0000 7f20d4926640 1 -- 172.21.15.5:0/1271415213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20c00043c0 con 0x7f20d0100a80 2024-09-19T06:02:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.240+0000 7f20d5928640 1 -- 172.21.15.5:0/1271415213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d019d5f0 con 0x7f20d0100a80 2024-09-19T06:02:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.240+0000 7f20d4926640 1 -- 172.21.15.5:0/1271415213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0004a00 con 0x7f20d0100a80 2024-09-19T06:02:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.240+0000 7f20d5928640 1 -- 172.21.15.5:0/1271415213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d01a4ed0 con 0x7f20d0100a80 2024-09-19T06:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.241+0000 7f20d5928640 1 -- 172.21.15.5:0/1271415213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20d0062940 con 0x7f20d0100a80 2024-09-19T06:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.242+0000 7f20d4926640 1 -- 172.21.15.5:0/1271415213 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f20c0004ba0 con 0x7f20d0100a80 2024-09-19T06:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.243+0000 7f20d4926640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f20b80a32d0 0x7f20b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.243+0000 7f20d4926640 1 -- 172.21.15.5:0/1271415213 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f20c00cfc10 con 0x7f20d0100a80 2024-09-19T06:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.243+0000 7f20ce7fc640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f20b80a32d0 0x7f20b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.243+0000 7f20d4926640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002] conn(0x7f20b80acf00 0x7f20b80af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.243+0000 7f20d4926640 1 -- 172.21.15.5:0/1271415213 --> [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20b80afa70 con 0x7f20b80acf00 2024-09-19T06:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.243+0000 7f20d4926640 1 -- 172.21.15.5:0/1271415213 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f20c00cffc0 con 0x7f20d0100a80 2024-09-19T06:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.243+0000 7f20cf7fe640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002] conn(0x7f20b80acf00 0x7f20b80af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.243+0000 7f20ce7fc640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f20b80a32d0 0x7f20b80a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f20d00ff270 tx=0x7f20c4005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.244+0000 7f20cf7fe640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002] conn(0x7f20b80acf00 0x7f20b80af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.245+0000 7f20d4926640 1 -- 172.21.15.5:0/1271415213 <== osd.10 v2:172.21.15.49:6824/2551748002 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20b80afa70 con 0x7f20b80acf00 2024-09-19T06:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.258+0000 7f20d5928640 1 -- 172.21.15.5:0/1271415213 --> [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20d0062dc0 con 0x7f20b80acf00 2024-09-19T06:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.259+0000 7f20d4926640 1 -- 172.21.15.5:0/1271415213 <== osd.10 v2:172.21.15.49:6824/2551748002 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f20d0062dc0 con 0x7f20b80acf00 2024-09-19T06:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.259+0000 7f20d5928640 1 -- 172.21.15.5:0/1271415213 >> [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002] conn(0x7f20b80acf00 msgr2=0x7f20b80af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.259+0000 7f20d5928640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002] conn(0x7f20b80acf00 0x7f20b80af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.259+0000 7f20d5928640 1 -- 172.21.15.5:0/1271415213 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f20b80a32d0 msgr2=0x7f20b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.259+0000 7f20d5928640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f20b80a32d0 0x7f20b80a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f20d00ff270 tx=0x7f20c4005bf0 comp rx=0 tx=0).stop 2024-09-19T06:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.259+0000 7f20d5928640 1 -- 172.21.15.5:0/1271415213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d0100a80 msgr2=0x7f20d0198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.259+0000 7f20d5928640 1 --2- 172.21.15.5:0/1271415213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d0100a80 0x7f20d0198350 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20c000c890 tx=0x7f20c000cd50 comp rx=0 tx=0).stop 2024-09-19T06:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.259+0000 7f20cf7fe640 1 -- 172.21.15.5:0/1271415213 reap_dead start 2024-09-19T06:02:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.259+0000 7f20d5928640 1 -- 172.21.15.5:0/1271415213 shutdown_connections 2024-09-19T06:02:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.259+0000 7f20d5928640 1 -- 172.21.15.5:0/1271415213 >> 172.21.15.5:0/1271415213 conn(0x7f20d00fc7a0 msgr2=0x7f20d006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.259+0000 7f20d5928640 1 -- 172.21.15.5:0/1271415213 shutdown_connections 2024-09-19T06:02:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:08.259+0000 7f20d5928640 1 -- 172.21.15.5:0/1271415213 wait complete. 2024-09-19T06:02:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.277+0000 7fe0712f9640 1 -- 172.21.15.5:0/2050346817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe06c106f10 msgr2=0x7fe06c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.277+0000 7fe0712f9640 1 --2- 172.21.15.5:0/2050346817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe06c106f10 0x7fe06c10ea90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe064009ca0 tx=0x7fe06402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:02:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.277+0000 7fe0712f9640 1 -- 172.21.15.5:0/2050346817 shutdown_connections 2024-09-19T06:02:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.277+0000 7fe0712f9640 1 --2- 172.21.15.5:0/2050346817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe06c106f10 0x7fe06c10ea90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.277+0000 7fe0712f9640 1 --2- 172.21.15.5:0/2050346817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06c102790 0x7fe06c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.277+0000 7fe0712f9640 1 --2- 172.21.15.5:0/2050346817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe06c101de0 0x7fe06c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.277+0000 7fe0712f9640 1 -- 172.21.15.5:0/2050346817 >> 172.21.15.5:0/2050346817 conn(0x7fe06c0fd8c0 msgr2=0x7fe06c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.277+0000 7fe0712f9640 1 -- 172.21.15.5:0/2050346817 shutdown_connections 2024-09-19T06:02:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.277+0000 7fe0712f9640 1 -- 172.21.15.5:0/2050346817 wait complete. 2024-09-19T06:02:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.278+0000 7fe0712f9640 1 Processor -- start 2024-09-19T06:02:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.278+0000 7fe0712f9640 1 -- start start 2024-09-19T06:02:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.278+0000 7fe0712f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe06c101de0 0x7fe06c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.278+0000 7fe0712f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe06c102790 0x7fe06c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.278+0000 7fe0712f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06c106f10 0x7fe06c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.278+0000 7fe0712f9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c111f10 con 0x7fe06c101de0 2024-09-19T06:02:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.278+0000 7fe0712f9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c111d90 con 0x7fe06c102790 2024-09-19T06:02:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.278+0000 7fe0712f9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c112090 con 0x7fe06c106f10 2024-09-19T06:02:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.278+0000 7fe06affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe06c101de0 0x7fe06c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.278+0000 7fe06affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe06c101de0 0x7fe06c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42738/0 (socket says 172.21.15.5:42738) 2024-09-19T06:02:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.278+0000 7fe06affd640 1 -- 172.21.15.5:0/2648536798 learned_addr learned my addr 172.21.15.5:0/2648536798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe06a7fc640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe06c102790 0x7fe06c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe06b7fe640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06c106f10 0x7fe06c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe06affd640 1 -- 172.21.15.5:0/2648536798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06c106f10 msgr2=0x7fe06c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe06affd640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06c106f10 0x7fe06c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe06affd640 1 -- 172.21.15.5:0/2648536798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe06c102790 msgr2=0x7fe06c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe06affd640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe06c102790 0x7fe06c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe06affd640 1 -- 172.21.15.5:0/2648536798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06c19d3c0 con 0x7fe06c101de0 2024-09-19T06:02:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe06b7fe640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06c106f10 0x7fe06c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe06a7fc640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe06c102790 0x7fe06c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe06affd640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe06c101de0 0x7fe06c198410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00b4a0 tx=0x7fe05c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe053fff640 1 -- 172.21.15.5:0/2648536798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c014070 con 0x7fe06c101de0 2024-09-19T06:02:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe0712f9640 1 -- 172.21.15.5:0/2648536798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06c19d6b0 con 0x7fe06c101de0 2024-09-19T06:02:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe053fff640 1 -- 172.21.15.5:0/2648536798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe05c010ac0 con 0x7fe06c101de0 2024-09-19T06:02:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe053fff640 1 -- 172.21.15.5:0/2648536798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c017c30 con 0x7fe06c101de0 2024-09-19T06:02:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.279+0000 7fe0712f9640 1 -- 172.21.15.5:0/2648536798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06c1a4f90 con 0x7fe06c101de0 2024-09-19T06:02:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.280+0000 7fe0712f9640 1 -- 172.21.15.5:0/2648536798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe06c062900 con 0x7fe06c101de0 2024-09-19T06:02:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.282+0000 7fe053fff640 1 -- 172.21.15.5:0/2648536798 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fe05c017070 con 0x7fe06c101de0 2024-09-19T06:02:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.282+0000 7fe053fff640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe0540a32d0 0x7fe0540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:09.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.282+0000 7fe06a7fc640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe0540a32d0 0x7fe0540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:09.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.282+0000 7fe053fff640 1 -- 172.21.15.5:0/2648536798 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe05c0cf240 con 0x7fe06c101de0 2024-09-19T06:02:09.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.282+0000 7fe053fff640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622] conn(0x7fe0540acf00 0x7fe0540af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:09.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.283+0000 7fe053fff640 1 -- 172.21.15.5:0/2648536798 --> [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe05c00ee50 con 0x7fe0540acf00 2024-09-19T06:02:09.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.283+0000 7fe053fff640 1 -- 172.21.15.5:0/2648536798 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fe05c0cf690 con 0x7fe06c101de0 2024-09-19T06:02:09.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.283+0000 7fe06a7fc640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe0540a32d0 0x7fe0540a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0fdce0 tx=0x7fe060008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:09.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.283+0000 7fe06b7fe640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622] conn(0x7fe0540acf00 0x7fe0540af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:09.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.283+0000 7fe06b7fe640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622] conn(0x7fe0540acf00 0x7fe0540af3a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:09.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.284+0000 7fe053fff640 1 -- 172.21.15.5:0/2648536798 <== osd.2 v2:172.21.15.157:6800/2319626622 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe05c00ee50 con 0x7fe0540acf00 2024-09-19T06:02:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.298+0000 7fe0712f9640 1 -- 172.21.15.5:0/2648536798 --> [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe06c062d80 con 0x7fe0540acf00 2024-09-19T06:02:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.299+0000 7fe053fff640 1 -- 172.21.15.5:0/2648536798 <== osd.2 v2:172.21.15.157:6800/2319626622 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe06c062d80 con 0x7fe0540acf00 2024-09-19T06:02:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.299+0000 7fe0712f9640 1 -- 172.21.15.5:0/2648536798 >> [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622] conn(0x7fe0540acf00 msgr2=0x7fe0540af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.299+0000 7fe0712f9640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622] conn(0x7fe0540acf00 0x7fe0540af3a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.299+0000 7fe0712f9640 1 -- 172.21.15.5:0/2648536798 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe0540a32d0 msgr2=0x7fe0540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.299+0000 7fe0712f9640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe0540a32d0 0x7fe0540a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0fdce0 tx=0x7fe060008040 comp rx=0 tx=0).stop 2024-09-19T06:02:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.299+0000 7fe0712f9640 1 -- 172.21.15.5:0/2648536798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe06c101de0 msgr2=0x7fe06c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.299+0000 7fe0712f9640 1 --2- 172.21.15.5:0/2648536798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe06c101de0 0x7fe06c198410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00b4a0 tx=0x7fe05c00b960 comp rx=0 tx=0).stop 2024-09-19T06:02:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.299+0000 7fe06b7fe640 1 -- 172.21.15.5:0/2648536798 reap_dead start 2024-09-19T06:02:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.299+0000 7fe0712f9640 1 -- 172.21.15.5:0/2648536798 shutdown_connections 2024-09-19T06:02:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.299+0000 7fe0712f9640 1 -- 172.21.15.5:0/2648536798 >> 172.21.15.5:0/2648536798 conn(0x7fe06c0fd8c0 msgr2=0x7fe06c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.299+0000 7fe0712f9640 1 -- 172.21.15.5:0/2648536798 shutdown_connections 2024-09-19T06:02:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:09.299+0000 7fe0712f9640 1 -- 172.21.15.5:0/2648536798 wait complete. 2024-09-19T06:02:10.353 INFO:teuthology.orchestra.run.smithi005.stdout:292057776154 2024-09-19T06:02:10.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T06:02:10.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:10 smithi005 ceph-mon[30955]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T06:02:10.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:10 smithi157 ceph-mon[35346]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T06:02:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:10 smithi049 ceph-mon[34640]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T06:02:10.945 INFO:teuthology.orchestra.run.smithi005.stdout:107374182460 2024-09-19T06:02:10.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T06:02:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.237+0000 7f46c5e0e640 1 -- 172.21.15.5:0/4167014172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46c0102790 msgr2=0x7f46c01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.237+0000 7f46c5e0e640 1 --2- 172.21.15.5:0/4167014172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46c0102790 0x7f46c01067e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009900 tx=0x7f46b402a410 comp rx=0 tx=0).stop 2024-09-19T06:02:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.238+0000 7f46c5e0e640 1 -- 172.21.15.5:0/4167014172 shutdown_connections 2024-09-19T06:02:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.238+0000 7f46c5e0e640 1 --2- 172.21.15.5:0/4167014172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c0106f10 0x7f46c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.238+0000 7f46c5e0e640 1 --2- 172.21.15.5:0/4167014172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46c0102790 0x7f46c01067e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.238+0000 7f46c5e0e640 1 --2- 172.21.15.5:0/4167014172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46c0101de0 0x7f46c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.238+0000 7f46c5e0e640 1 -- 172.21.15.5:0/4167014172 >> 172.21.15.5:0/4167014172 conn(0x7f46c00fd8c0 msgr2=0x7f46c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.238+0000 7f46c5e0e640 1 -- 172.21.15.5:0/4167014172 shutdown_connections 2024-09-19T06:02:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.238+0000 7f46c5e0e640 1 -- 172.21.15.5:0/4167014172 wait complete. 2024-09-19T06:02:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.238+0000 7f46c5e0e640 1 Processor -- start 2024-09-19T06:02:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.238+0000 7f46c5e0e640 1 -- start start 2024-09-19T06:02:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46c5e0e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46c0101de0 0x7f46c0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46c5e0e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c0102790 0x7f46c0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46c5e0e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46c0106f10 0x7f46c019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46c5e0e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0111e20 con 0x7f46c0106f10 2024-09-19T06:02:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46c5e0e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0111ca0 con 0x7f46c0101de0 2024-09-19T06:02:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46c5e0e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0111fa0 con 0x7f46c0102790 2024-09-19T06:02:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46c0106f10 0x7f46c019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46c0106f10 0x7f46c019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42778/0 (socket says 172.21.15.5:42778) 2024-09-19T06:02:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46bffff640 1 -- 172.21.15.5:0/2028141724 learned_addr learned my addr 172.21.15.5:0/2028141724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46bf7fe640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46c0101de0 0x7f46c0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46beffd640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c0102790 0x7f46c0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46bffff640 1 -- 172.21.15.5:0/2028141724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c0102790 msgr2=0x7f46c0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46bffff640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c0102790 0x7f46c0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46bffff640 1 -- 172.21.15.5:0/2028141724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46c0101de0 msgr2=0x7f46c0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46bffff640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46c0101de0 0x7f46c0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46bffff640 1 -- 172.21.15.5:0/2028141724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c019d2d0 con 0x7f46c0106f10 2024-09-19T06:02:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46beffd640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c0102790 0x7f46c0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46bf7fe640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46c0101de0 0x7f46c0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:02:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.239+0000 7f46bffff640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46c0106f10 0x7f46c019cbf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f46b800cce0 tx=0x7f46b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.240+0000 7f46bcff9640 1 -- 172.21.15.5:0/2028141724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b800f450 con 0x7f46c0106f10 2024-09-19T06:02:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.240+0000 7f46c5e0e640 1 -- 172.21.15.5:0/2028141724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c019d5c0 con 0x7f46c0106f10 2024-09-19T06:02:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.240+0000 7f46bcff9640 1 -- 172.21.15.5:0/2028141724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46b800faa0 con 0x7f46c0106f10 2024-09-19T06:02:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.240+0000 7f46bcff9640 1 -- 172.21.15.5:0/2028141724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8018b70 con 0x7f46c0106f10 2024-09-19T06:02:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.240+0000 7f46c5e0e640 1 -- 172.21.15.5:0/2028141724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c01a4ea0 con 0x7f46c0106f10 2024-09-19T06:02:11.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.240+0000 7f46c5e0e640 1 -- 172.21.15.5:0/2028141724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46c0062980 con 0x7f46c0106f10 2024-09-19T06:02:11.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.242+0000 7f46bcff9640 1 -- 172.21.15.5:0/2028141724 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f46b8018d10 con 0x7f46c0106f10 2024-09-19T06:02:11.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.243+0000 7f46bcff9640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f46a80a32d0 0x7f46a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:11.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.243+0000 7f46bcff9640 1 -- 172.21.15.5:0/2028141724 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f46b80d0130 con 0x7f46c0106f10 2024-09-19T06:02:11.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.243+0000 7f46bf7fe640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f46a80a32d0 0x7f46a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:11.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.243+0000 7f46bcff9640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014] conn(0x7f46a80acf00 0x7f46a80af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:11.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.243+0000 7f46bcff9640 1 -- 172.21.15.5:0/2028141724 --> [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46b8015e70 con 0x7f46a80acf00 2024-09-19T06:02:11.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.243+0000 7f46bcff9640 1 -- 172.21.15.5:0/2028141724 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f46b80d0580 con 0x7f46c0106f10 2024-09-19T06:02:11.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.244+0000 7f46beffd640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014] conn(0x7f46a80acf00 0x7f46a80af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:11.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.244+0000 7f46bf7fe640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f46a80a32d0 0x7f46a80a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f46b000bfd0 tx=0x7f46b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:11.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.244+0000 7f46beffd640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014] conn(0x7f46a80acf00 0x7f46a80af3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:11.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.245+0000 7f46bcff9640 1 -- 172.21.15.5:0/2028141724 <== osd.1 v2:172.21.15.49:6800/1448265014 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46b8015e70 con 0x7f46a80acf00 2024-09-19T06:02:11.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.258+0000 7f46c5e0e640 1 -- 172.21.15.5:0/2028141724 --> [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46c0062e00 con 0x7f46a80acf00 2024-09-19T06:02:11.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.259+0000 7f46bcff9640 1 -- 172.21.15.5:0/2028141724 <== osd.1 v2:172.21.15.49:6800/1448265014 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f46c0062e00 con 0x7f46a80acf00 2024-09-19T06:02:11.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.259+0000 7f46c5e0e640 1 -- 172.21.15.5:0/2028141724 >> [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014] conn(0x7f46a80acf00 msgr2=0x7f46a80af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:11.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.259+0000 7f46c5e0e640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014] conn(0x7f46a80acf00 0x7f46a80af3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:11.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.259+0000 7f46c5e0e640 1 -- 172.21.15.5:0/2028141724 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f46a80a32d0 msgr2=0x7f46a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:11.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.259+0000 7f46c5e0e640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f46a80a32d0 0x7f46a80a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f46b000bfd0 tx=0x7f46b000b390 comp rx=0 tx=0).stop 2024-09-19T06:02:11.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.259+0000 7f46c5e0e640 1 -- 172.21.15.5:0/2028141724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46c0106f10 msgr2=0x7f46c019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:11.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.259+0000 7f46c5e0e640 1 --2- 172.21.15.5:0/2028141724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46c0106f10 0x7f46c019cbf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f46b800cce0 tx=0x7f46b800a620 comp rx=0 tx=0).stop 2024-09-19T06:02:11.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.259+0000 7f46bffff640 1 -- 172.21.15.5:0/2028141724 reap_dead start 2024-09-19T06:02:11.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.259+0000 7f46c5e0e640 1 -- 172.21.15.5:0/2028141724 shutdown_connections 2024-09-19T06:02:11.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.259+0000 7f46c5e0e640 1 -- 172.21.15.5:0/2028141724 >> 172.21.15.5:0/2028141724 conn(0x7f46c00fd8c0 msgr2=0x7f46c00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:11.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.259+0000 7f46c5e0e640 1 -- 172.21.15.5:0/2028141724 shutdown_connections 2024-09-19T06:02:11.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.259+0000 7f46c5e0e640 1 -- 172.21.15.5:0/2028141724 wait complete. 2024-09-19T06:02:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.991+0000 7fd553cf5640 1 -- 172.21.15.5:0/1935397113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd54c102790 msgr2=0x7fd54c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.991+0000 7fd553cf5640 1 --2- 172.21.15.5:0/1935397113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd54c102790 0x7fd54c1067e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009900 tx=0x7fd53c02a410 comp rx=0 tx=0).stop 2024-09-19T06:02:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.991+0000 7fd553cf5640 1 -- 172.21.15.5:0/1935397113 shutdown_connections 2024-09-19T06:02:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.991+0000 7fd553cf5640 1 --2- 172.21.15.5:0/1935397113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c106f10 0x7fd54c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.991+0000 7fd553cf5640 1 --2- 172.21.15.5:0/1935397113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd54c102790 0x7fd54c1067e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.991+0000 7fd553cf5640 1 --2- 172.21.15.5:0/1935397113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd54c101de0 0x7fd54c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.991+0000 7fd553cf5640 1 -- 172.21.15.5:0/1935397113 >> 172.21.15.5:0/1935397113 conn(0x7fd54c0fd8c0 msgr2=0x7fd54c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.991+0000 7fd553cf5640 1 -- 172.21.15.5:0/1935397113 shutdown_connections 2024-09-19T06:02:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.991+0000 7fd553cf5640 1 -- 172.21.15.5:0/1935397113 wait complete. 2024-09-19T06:02:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd553cf5640 1 Processor -- start 2024-09-19T06:02:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd553cf5640 1 -- start start 2024-09-19T06:02:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd553cf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd54c101de0 0x7fd54c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd553cf5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd54c102790 0x7fd54c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd553cf5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c106f10 0x7fd54c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd553cf5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c111ec0 con 0x7fd54c101de0 2024-09-19T06:02:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd553cf5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c111d40 con 0x7fd54c106f10 2024-09-19T06:02:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd553cf5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c112040 con 0x7fd54c102790 2024-09-19T06:02:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd551a91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd54c101de0 0x7fd54c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd551a91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd54c101de0 0x7fd54c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42802/0 (socket says 172.21.15.5:42802) 2024-09-19T06:02:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd551a91640 1 -- 172.21.15.5:0/3457599436 learned_addr learned my addr 172.21.15.5:0/3457599436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd551290640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd54c102790 0x7fd54c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd552292640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c106f10 0x7fd54c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd551a91640 1 -- 172.21.15.5:0/3457599436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd54c102790 msgr2=0x7fd54c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd551a91640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd54c102790 0x7fd54c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd551a91640 1 -- 172.21.15.5:0/3457599436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c106f10 msgr2=0x7fd54c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd551a91640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c106f10 0x7fd54c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd551a91640 1 -- 172.21.15.5:0/3457599436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c1a69f0 con 0x7fd54c101de0 2024-09-19T06:02:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd551290640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd54c102790 0x7fd54c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.992+0000 7fd552292640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c106f10 0x7fd54c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.993+0000 7fd551a91640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd54c101de0 0x7fd54c0744f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b4a0 tx=0x7fd54800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.993+0000 7fd542ffd640 1 -- 172.21.15.5:0/3457599436 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548014070 con 0x7fd54c101de0 2024-09-19T06:02:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.993+0000 7fd553cf5640 1 -- 172.21.15.5:0/3457599436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c1a6ce0 con 0x7fd54c101de0 2024-09-19T06:02:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.993+0000 7fd542ffd640 1 -- 172.21.15.5:0/3457599436 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd548010ac0 con 0x7fd54c101de0 2024-09-19T06:02:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.993+0000 7fd542ffd640 1 -- 172.21.15.5:0/3457599436 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548017c30 con 0x7fd54c101de0 2024-09-19T06:02:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.993+0000 7fd553cf5640 1 -- 172.21.15.5:0/3457599436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c1a71a0 con 0x7fd54c101de0 2024-09-19T06:02:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.994+0000 7fd553cf5640 1 -- 172.21.15.5:0/3457599436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd54c062880 con 0x7fd54c101de0 2024-09-19T06:02:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.995+0000 7fd542ffd640 1 -- 172.21.15.5:0/3457599436 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fd548017070 con 0x7fd54c101de0 2024-09-19T06:02:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.996+0000 7fd542ffd640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd5380a32d0 0x7fd5380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.996+0000 7fd551290640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd5380a32d0 0x7fd5380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.996+0000 7fd542ffd640 1 -- 172.21.15.5:0/3457599436 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd5480cf200 con 0x7fd54c101de0 2024-09-19T06:02:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.996+0000 7fd542ffd640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237] conn(0x7fd5380acf00 0x7fd5380af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.996+0000 7fd542ffd640 1 -- 172.21.15.5:0/3457599436 --> [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd54801be50 con 0x7fd5380acf00 2024-09-19T06:02:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.996+0000 7fd542ffd640 1 -- 172.21.15.5:0/3457599436 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fd5480cf650 con 0x7fd54c101de0 2024-09-19T06:02:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.996+0000 7fd552292640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237] conn(0x7fd5380acf00 0x7fd5380af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.996+0000 7fd551290640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd5380a32d0 0x7fd5380a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd53c031440 tx=0x7fd53c0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.997+0000 7fd552292640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237] conn(0x7fd5380acf00 0x7fd5380af3a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.997+0000 7fd542ffd640 1 -- 172.21.15.5:0/3457599436 <== osd.9 v2:172.21.15.5:6826/4122448237 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd54801be50 con 0x7fd5380acf00 2024-09-19T06:02:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.011+0000 7fd553cf5640 1 -- 172.21.15.5:0/3457599436 --> [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd54c062c80 con 0x7fd5380acf00 2024-09-19T06:02:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.011+0000 7fd542ffd640 1 -- 172.21.15.5:0/3457599436 <== osd.9 v2:172.21.15.5:6826/4122448237 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd54c062c80 con 0x7fd5380acf00 2024-09-19T06:02:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.012+0000 7fd553cf5640 1 -- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237] conn(0x7fd5380acf00 msgr2=0x7fd5380af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.012+0000 7fd553cf5640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237] conn(0x7fd5380acf00 0x7fd5380af3a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.012+0000 7fd553cf5640 1 -- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd5380a32d0 msgr2=0x7fd5380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.012+0000 7fd553cf5640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd5380a32d0 0x7fd5380a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd53c031440 tx=0x7fd53c0095f0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.012+0000 7fd553cf5640 1 -- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd54c101de0 msgr2=0x7fd54c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.012+0000 7fd553cf5640 1 --2- 172.21.15.5:0/3457599436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd54c101de0 0x7fd54c0744f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b4a0 tx=0x7fd54800b960 comp rx=0 tx=0).stop 2024-09-19T06:02:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.012+0000 7fd552292640 1 -- 172.21.15.5:0/3457599436 reap_dead start 2024-09-19T06:02:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.012+0000 7fd553cf5640 1 -- 172.21.15.5:0/3457599436 shutdown_connections 2024-09-19T06:02:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.012+0000 7fd553cf5640 1 -- 172.21.15.5:0/3457599436 >> 172.21.15.5:0/3457599436 conn(0x7fd54c0fd8c0 msgr2=0x7fd54c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.012+0000 7fd553cf5640 1 -- 172.21.15.5:0/3457599436 shutdown_connections 2024-09-19T06:02:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.012+0000 7fd553cf5640 1 -- 172.21.15.5:0/3457599436 wait complete. 2024-09-19T06:02:12.481 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:12 smithi005 ceph-mon[30955]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-19T06:02:12.509 INFO:teuthology.orchestra.run.smithi005.stdout:77309411395 2024-09-19T06:02:12.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T06:02:12.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:12 smithi157 ceph-mon[35346]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-19T06:02:12.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.133+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/687449192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cbc1036e0 msgr2=0x7f8cbc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.133+0000 7f8cc1b6d640 1 --2- 172.21.15.5:0/687449192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cbc1036e0 0x7f8cbc107b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009930 tx=0x7f8cac02a420 comp rx=0 tx=0).stop 2024-09-19T06:02:12.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.133+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/687449192 shutdown_connections 2024-09-19T06:02:12.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.133+0000 7f8cc1b6d640 1 --2- 172.21.15.5:0/687449192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cbc1082a0 0x7f8cbc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.133+0000 7f8cc1b6d640 1 --2- 172.21.15.5:0/687449192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cbc1036e0 0x7f8cbc107b90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.133+0000 7f8cc1b6d640 1 --2- 172.21.15.5:0/687449192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cbc102d30 0x7f8cbc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.133+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/687449192 >> 172.21.15.5:0/687449192 conn(0x7f8cbc0fc7d0 msgr2=0x7f8cbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:12.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.133+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/687449192 shutdown_connections 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.133+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/687449192 wait complete. 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.134+0000 7f8cc1b6d640 1 Processor -- start 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.134+0000 7f8cc1b6d640 1 -- start start 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.134+0000 7f8cc1b6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cbc102d30 0x7f8cbc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.134+0000 7f8cc1b6d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cbc1082a0 0x7f8cbc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.134+0000 7f8cc1b6d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cbc101410 0x7f8cbc1a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.134+0000 7f8cc1b6d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc112040 con 0x7f8cbc102d30 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.134+0000 7f8cc1b6d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc111ec0 con 0x7f8cbc1082a0 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.134+0000 7f8cc1b6d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1121c0 con 0x7f8cbc101410 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.134+0000 7f8cbb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cbc102d30 0x7f8cbc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.134+0000 7f8cbbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cbc101410 0x7f8cbc1a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.134+0000 7f8cbbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cbc101410 0x7f8cbc1a6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:58964/0 (socket says 172.21.15.5:58964) 2024-09-19T06:02:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.134+0000 7f8cbbfff640 1 -- 172.21.15.5:0/1460811424 learned_addr learned my addr 172.21.15.5:0/1460811424 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cbaffd640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cbc1082a0 0x7f8cbc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cbb7fe640 1 -- 172.21.15.5:0/1460811424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cbc101410 msgr2=0x7f8cbc1a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cbb7fe640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cbc101410 0x7f8cbc1a6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cbb7fe640 1 -- 172.21.15.5:0/1460811424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cbc1082a0 msgr2=0x7f8cbc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cbb7fe640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cbc1082a0 0x7f8cbc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cbb7fe640 1 -- 172.21.15.5:0/1460811424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc1a7110 con 0x7f8cbc102d30 2024-09-19T06:02:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cbaffd640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cbc1082a0 0x7f8cbc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cbbfff640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cbc101410 0x7f8cbc1a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:02:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cbb7fe640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cbc102d30 0x7f8cbc100990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000b3f0 tx=0x7f8cb000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cb8ff9640 1 -- 172.21.15.5:0/1460811424 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0015020 con 0x7f8cbc102d30 2024-09-19T06:02:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cb8ff9640 1 -- 172.21.15.5:0/1460811424 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cb0011ce0 con 0x7f8cbc102d30 2024-09-19T06:02:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/1460811424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cbc1a7400 con 0x7f8cbc102d30 2024-09-19T06:02:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/1460811424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cbc1a7910 con 0x7f8cbc102d30 2024-09-19T06:02:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.135+0000 7f8cb8ff9640 1 -- 172.21.15.5:0/1460811424 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0010d70 con 0x7f8cbc102d30 2024-09-19T06:02:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.136+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/1460811424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8cbc062900 con 0x7f8cbc102d30 2024-09-19T06:02:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.138+0000 7f8cb8ff9640 1 -- 172.21.15.5:0/1460811424 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f8cb0018020 con 0x7f8cbc102d30 2024-09-19T06:02:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.138+0000 7f8cb8ff9640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8ca40a32d0 0x7f8ca40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.139+0000 7f8cb8ff9640 1 -- 172.21.15.5:0/1460811424 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8cb00cf910 con 0x7f8cbc102d30 2024-09-19T06:02:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.139+0000 7f8cbaffd640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8ca40a32d0 0x7f8ca40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.139+0000 7f8cb8ff9640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194] conn(0x7f8ca40acf00 0x7f8ca40af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.139+0000 7f8cb8ff9640 1 -- 172.21.15.5:0/1460811424 --> [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8cb0018ec0 con 0x7f8ca40acf00 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.139+0000 7f8cb8ff9640 1 -- 172.21.15.5:0/1460811424 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f8cb00cfcc0 con 0x7f8cbc102d30 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.139+0000 7f8cbbfff640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194] conn(0x7f8ca40acf00 0x7f8ca40af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.139+0000 7f8cbaffd640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8ca40a32d0 0x7f8ca40a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8cac008000 tx=0x7f8cac031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.139+0000 7f8cbbfff640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194] conn(0x7f8ca40acf00 0x7f8ca40af3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.140+0000 7f8cb8ff9640 1 -- 172.21.15.5:0/1460811424 <== osd.6 v2:172.21.15.5:6818/2957795194 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8cb0018ec0 con 0x7f8ca40acf00 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.154+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/1460811424 --> [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8cbc062db0 con 0x7f8ca40acf00 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.154+0000 7f8cb8ff9640 1 -- 172.21.15.5:0/1460811424 <== osd.6 v2:172.21.15.5:6818/2957795194 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8cbc062db0 con 0x7f8ca40acf00 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.155+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194] conn(0x7f8ca40acf00 msgr2=0x7f8ca40af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.155+0000 7f8cc1b6d640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194] conn(0x7f8ca40acf00 0x7f8ca40af3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.155+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8ca40a32d0 msgr2=0x7f8ca40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.155+0000 7f8cc1b6d640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8ca40a32d0 0x7f8ca40a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8cac008000 tx=0x7f8cac031400 comp rx=0 tx=0).stop 2024-09-19T06:02:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.155+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cbc102d30 msgr2=0x7f8cbc100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.155+0000 7f8cc1b6d640 1 --2- 172.21.15.5:0/1460811424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cbc102d30 0x7f8cbc100990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000b3f0 tx=0x7f8cb000b8b0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.155+0000 7f8cbbfff640 1 -- 172.21.15.5:0/1460811424 reap_dead start 2024-09-19T06:02:12.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.155+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/1460811424 shutdown_connections 2024-09-19T06:02:12.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.155+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/1460811424 >> 172.21.15.5:0/1460811424 conn(0x7f8cbc0fc7d0 msgr2=0x7f8cbc103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:12.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.155+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/1460811424 shutdown_connections 2024-09-19T06:02:12.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:11.155+0000 7f8cc1b6d640 1 -- 172.21.15.5:0/1460811424 wait complete. 2024-09-19T06:02:12.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:12 smithi049 ceph-mon[34640]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-19T06:02:12.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.906+0000 7fdcc38b0640 1 -- 172.21.15.5:0/1503886998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcbc101de0 msgr2=0x7fdcbc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.906+0000 7fdcc38b0640 1 --2- 172.21.15.5:0/1503886998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcbc101de0 0x7fdcbc1021c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009900 tx=0x7fdcb802a410 comp rx=0 tx=0).stop 2024-09-19T06:02:12.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.906+0000 7fdcc38b0640 1 -- 172.21.15.5:0/1503886998 shutdown_connections 2024-09-19T06:02:12.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.906+0000 7fdcc38b0640 1 --2- 172.21.15.5:0/1503886998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc106f10 0x7fdcbc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.906+0000 7fdcc38b0640 1 --2- 172.21.15.5:0/1503886998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcbc102790 0x7fdcbc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.906+0000 7fdcc38b0640 1 --2- 172.21.15.5:0/1503886998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcbc101de0 0x7fdcbc1021c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.906+0000 7fdcc38b0640 1 -- 172.21.15.5:0/1503886998 >> 172.21.15.5:0/1503886998 conn(0x7fdcbc0fd8c0 msgr2=0x7fdcbc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:12.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.906+0000 7fdcc38b0640 1 -- 172.21.15.5:0/1503886998 shutdown_connections 2024-09-19T06:02:12.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.906+0000 7fdcc38b0640 1 -- 172.21.15.5:0/1503886998 wait complete. 2024-09-19T06:02:12.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.906+0000 7fdcc38b0640 1 Processor -- start 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.906+0000 7fdcc38b0640 1 -- start start 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc38b0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcbc101de0 0x7fdcbc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc38b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcbc102790 0x7fdcbc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc38b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc106f10 0x7fdcbc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc38b0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc111e40 con 0x7fdcbc102790 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc38b0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc111cc0 con 0x7fdcbc101de0 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc38b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc111fc0 con 0x7fdcbc106f10 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc0e4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcbc102790 0x7fdcbc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc0e4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcbc102790 0x7fdcbc198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42820/0 (socket says 172.21.15.5:42820) 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc0e4b640 1 -- 172.21.15.5:0/718615588 learned_addr learned my addr 172.21.15.5:0/718615588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc164c640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcbc101de0 0x7fdcbc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc0e4b640 1 -- 172.21.15.5:0/718615588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc106f10 msgr2=0x7fdcbc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc1e4d640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc106f10 0x7fdcbc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc0e4b640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc106f10 0x7fdcbc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc0e4b640 1 -- 172.21.15.5:0/718615588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcbc101de0 msgr2=0x7fdcbc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc0e4b640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcbc101de0 0x7fdcbc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc0e4b640 1 -- 172.21.15.5:0/718615588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc19d3b0 con 0x7fdcbc102790 2024-09-19T06:02:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc1e4d640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc106f10 0x7fdcbc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcc0e4b640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcbc102790 0x7fdcbc198940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00d950 tx=0x7fdcac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcb27fc640 1 -- 172.21.15.5:0/718615588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac017070 con 0x7fdcbc102790 2024-09-19T06:02:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.907+0000 7fdcb27fc640 1 -- 172.21.15.5:0/718615588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcac009c30 con 0x7fdcbc102790 2024-09-19T06:02:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.908+0000 7fdcc38b0640 1 -- 172.21.15.5:0/718615588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcbc19d6a0 con 0x7fdcbc102790 2024-09-19T06:02:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.908+0000 7fdcb27fc640 1 -- 172.21.15.5:0/718615588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00f620 con 0x7fdcbc102790 2024-09-19T06:02:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.908+0000 7fdcc38b0640 1 -- 172.21.15.5:0/718615588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcbc1a4f80 con 0x7fdcbc102790 2024-09-19T06:02:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.908+0000 7fdcc38b0640 1 -- 172.21.15.5:0/718615588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdcbc062940 con 0x7fdcbc102790 2024-09-19T06:02:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.910+0000 7fdcb27fc640 1 -- 172.21.15.5:0/718615588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fdcac0109a0 con 0x7fdcbc102790 2024-09-19T06:02:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.910+0000 7fdcb27fc640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdca80a32d0 0x7fdca80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.911+0000 7fdcc164c640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdca80a32d0 0x7fdca80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.911+0000 7fdcb27fc640 1 -- 172.21.15.5:0/718615588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdcac0cfae0 con 0x7fdcbc102790 2024-09-19T06:02:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.911+0000 7fdcb27fc640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544] conn(0x7fdca80acf00 0x7fdca80af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.911+0000 7fdcb27fc640 1 -- 172.21.15.5:0/718615588 --> [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdca80afa70 con 0x7fdca80acf00 2024-09-19T06:02:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.911+0000 7fdcb27fc640 1 -- 172.21.15.5:0/718615588 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fdcac0cfe90 con 0x7fdcbc102790 2024-09-19T06:02:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.911+0000 7fdcc164c640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdca80a32d0 0x7fdca80a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8031440 tx=0x7fdcb8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.911+0000 7fdcc1e4d640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544] conn(0x7fdca80acf00 0x7fdca80af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.912+0000 7fdcc1e4d640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544] conn(0x7fdca80acf00 0x7fdca80af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.912+0000 7fdcb27fc640 1 -- 172.21.15.5:0/718615588 <== osd.11 v2:172.21.15.157:6824/1996582544 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdca80afa70 con 0x7fdca80acf00 2024-09-19T06:02:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.926+0000 7fdcc38b0640 1 -- 172.21.15.5:0/718615588 --> [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdcbc062dc0 con 0x7fdca80acf00 2024-09-19T06:02:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.926+0000 7fdcb27fc640 1 -- 172.21.15.5:0/718615588 <== osd.11 v2:172.21.15.157:6824/1996582544 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdcbc062dc0 con 0x7fdca80acf00 2024-09-19T06:02:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.927+0000 7fdcc38b0640 1 -- 172.21.15.5:0/718615588 >> [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544] conn(0x7fdca80acf00 msgr2=0x7fdca80af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.927+0000 7fdcc38b0640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544] conn(0x7fdca80acf00 0x7fdca80af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.927+0000 7fdcc38b0640 1 -- 172.21.15.5:0/718615588 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdca80a32d0 msgr2=0x7fdca80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.927+0000 7fdcc38b0640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fdca80a32d0 0x7fdca80a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8031440 tx=0x7fdcb8005b50 comp rx=0 tx=0).stop 2024-09-19T06:02:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.927+0000 7fdcc38b0640 1 -- 172.21.15.5:0/718615588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcbc102790 msgr2=0x7fdcbc198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.927+0000 7fdcc38b0640 1 --2- 172.21.15.5:0/718615588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcbc102790 0x7fdcbc198940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00d950 tx=0x7fdcac00de10 comp rx=0 tx=0).stop 2024-09-19T06:02:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.927+0000 7fdcc1e4d640 1 -- 172.21.15.5:0/718615588 reap_dead start 2024-09-19T06:02:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.927+0000 7fdcc38b0640 1 -- 172.21.15.5:0/718615588 shutdown_connections 2024-09-19T06:02:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.927+0000 7fdcc38b0640 1 -- 172.21.15.5:0/718615588 >> 172.21.15.5:0/718615588 conn(0x7fdcbc0fd8c0 msgr2=0x7fdcbc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.927+0000 7fdcc38b0640 1 -- 172.21.15.5:0/718615588 shutdown_connections 2024-09-19T06:02:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:12.927+0000 7fdcc38b0640 1 -- 172.21.15.5:0/718615588 wait complete. 2024-09-19T06:02:13.621 INFO:teuthology.orchestra.run.smithi005.stdout:55834574920 2024-09-19T06:02:13.622 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T06:02:14.471 INFO:teuthology.orchestra.run.smithi005.stdout:188978561069 2024-09-19T06:02:14.471 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T06:02:14.495 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:14 smithi005 ceph-mon[30955]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T06:02:14.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:14 smithi157 ceph-mon[35346]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T06:02:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:14 smithi049 ceph-mon[34640]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T06:02:15.081 INFO:teuthology.orchestra.run.smithi005.stdout:322122547221 2024-09-19T06:02:15.081 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T06:02:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.536+0000 7f920c391640 1 -- 172.21.15.5:0/3439136339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9204102790 msgr2=0x7f92041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.536+0000 7f920c391640 1 --2- 172.21.15.5:0/3439136339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9204102790 0x7f92041067e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9200009900 tx=0x7f920002a410 comp rx=0 tx=0).stop 2024-09-19T06:02:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.536+0000 7f920c391640 1 -- 172.21.15.5:0/3439136339 shutdown_connections 2024-09-19T06:02:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.536+0000 7f920c391640 1 --2- 172.21.15.5:0/3439136339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9204106f10 0x7f920410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.536+0000 7f920c391640 1 --2- 172.21.15.5:0/3439136339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9204102790 0x7f92041067e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.536+0000 7f920c391640 1 --2- 172.21.15.5:0/3439136339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9204101de0 0x7f92041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.536+0000 7f920c391640 1 -- 172.21.15.5:0/3439136339 >> 172.21.15.5:0/3439136339 conn(0x7f92040fd8c0 msgr2=0x7f92040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.536+0000 7f920c391640 1 -- 172.21.15.5:0/3439136339 shutdown_connections 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.536+0000 7f920c391640 1 -- 172.21.15.5:0/3439136339 wait complete. 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.537+0000 7f920c391640 1 Processor -- start 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.537+0000 7f920c391640 1 -- start start 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.537+0000 7f920c391640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9204101de0 0x7f9204198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920c391640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9204102790 0x7f9204198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920c391640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9204106f10 0x7f920419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920c391640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9204111e20 con 0x7f9204102790 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920c391640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9204111ca0 con 0x7f9204101de0 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920c391640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9204111fa0 con 0x7f9204106f10 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920992c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9204102790 0x7f9204198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920992c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9204102790 0x7f9204198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42760/0 (socket says 172.21.15.5:42760) 2024-09-19T06:02:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920a12d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9204101de0 0x7f9204198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920992c640 1 -- 172.21.15.5:0/2752564095 learned_addr learned my addr 172.21.15.5:0/2752564095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920a92e640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9204106f10 0x7f920419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920992c640 1 -- 172.21.15.5:0/2752564095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9204106f10 msgr2=0x7f920419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920992c640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9204106f10 0x7f920419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920992c640 1 -- 172.21.15.5:0/2752564095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9204101de0 msgr2=0x7f9204198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920992c640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9204101de0 0x7f9204198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920992c640 1 -- 172.21.15.5:0/2752564095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920419d400 con 0x7f9204102790 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.538+0000 7f920a12d640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9204101de0 0x7f9204198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.539+0000 7f920992c640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9204102790 0x7f9204198990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9200005940 tx=0x7f92000040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.539+0000 7f91f77fe640 1 -- 172.21.15.5:0/2752564095 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920003f030 con 0x7f9204102790 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.539+0000 7f91f77fe640 1 -- 172.21.15.5:0/2752564095 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92000044c0 con 0x7f9204102790 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.539+0000 7f920c391640 1 -- 172.21.15.5:0/2752564095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920419d690 con 0x7f9204102790 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.539+0000 7f91f77fe640 1 -- 172.21.15.5:0/2752564095 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200035070 con 0x7f9204102790 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.539+0000 7f920c391640 1 -- 172.21.15.5:0/2752564095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92041a4fd0 con 0x7f9204102790 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.540+0000 7f920c391640 1 -- 172.21.15.5:0/2752564095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9204062900 con 0x7f9204102790 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.541+0000 7f91f77fe640 1 -- 172.21.15.5:0/2752564095 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f9200042030 con 0x7f9204102790 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.542+0000 7f91f77fe640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f91f00a3350 0x7f91f00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.542+0000 7f920a12d640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f91f00a3350 0x7f91f00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.542+0000 7f91f77fe640 1 -- 172.21.15.5:0/2752564095 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f92000efa10 con 0x7f9204102790 2024-09-19T06:02:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.542+0000 7f91f77fe640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859] conn(0x7f91f00acf80 0x7f91f00af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.542+0000 7f91f77fe640 1 -- 172.21.15.5:0/2752564095 --> [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9200001d30 con 0x7f91f00acf80 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.542+0000 7f91f77fe640 1 -- 172.21.15.5:0/2752564095 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f92000f42c0 con 0x7f9204102790 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.543+0000 7f920a12d640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f91f00a3350 0x7f91f00a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009fc0 tx=0x7f91f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.543+0000 7f920a92e640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859] conn(0x7f91f00acf80 0x7f91f00af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.543+0000 7f920a92e640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859] conn(0x7f91f00acf80 0x7f91f00af420 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.544+0000 7f91f77fe640 1 -- 172.21.15.5:0/2752564095 <== osd.8 v2:172.21.15.157:6816/290026859 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9200001d30 con 0x7f91f00acf80 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.559+0000 7f920c391640 1 -- 172.21.15.5:0/2752564095 --> [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9204062d00 con 0x7f91f00acf80 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.559+0000 7f91f77fe640 1 -- 172.21.15.5:0/2752564095 <== osd.8 v2:172.21.15.157:6816/290026859 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9204062d00 con 0x7f91f00acf80 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.560+0000 7f920c391640 1 -- 172.21.15.5:0/2752564095 >> [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859] conn(0x7f91f00acf80 msgr2=0x7f91f00af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.560+0000 7f920c391640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859] conn(0x7f91f00acf80 0x7f91f00af420 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.560+0000 7f920c391640 1 -- 172.21.15.5:0/2752564095 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f91f00a3350 msgr2=0x7f91f00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.560+0000 7f920c391640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f91f00a3350 0x7f91f00a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009fc0 tx=0x7f91f8005f90 comp rx=0 tx=0).stop 2024-09-19T06:02:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.560+0000 7f920c391640 1 -- 172.21.15.5:0/2752564095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9204102790 msgr2=0x7f9204198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.560+0000 7f920c391640 1 --2- 172.21.15.5:0/2752564095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9204102790 0x7f9204198990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9200005940 tx=0x7f92000040c0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.560+0000 7f920a92e640 1 -- 172.21.15.5:0/2752564095 reap_dead start 2024-09-19T06:02:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.560+0000 7f920c391640 1 -- 172.21.15.5:0/2752564095 shutdown_connections 2024-09-19T06:02:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.560+0000 7f920c391640 1 -- 172.21.15.5:0/2752564095 >> 172.21.15.5:0/2752564095 conn(0x7f92040fd8c0 msgr2=0x7f92040fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.560+0000 7f920c391640 1 -- 172.21.15.5:0/2752564095 shutdown_connections 2024-09-19T06:02:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:10.560+0000 7f920c391640 1 -- 172.21.15.5:0/2752564095 wait complete. 2024-09-19T06:02:15.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.937+0000 7f28d1c42640 1 -- 172.21.15.5:0/220259593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc102d40 msgr2=0x7f28cc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:15.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.937+0000 7f28d1c42640 1 --2- 172.21.15.5:0/220259593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc102d40 0x7f28cc103120 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009900 tx=0x7f28bc02a480 comp rx=0 tx=0).stop 2024-09-19T06:02:15.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.937+0000 7f28d1c42640 1 -- 172.21.15.5:0/220259593 shutdown_connections 2024-09-19T06:02:15.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.937+0000 7f28d1c42640 1 --2- 172.21.15.5:0/220259593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc10c170 0x7f28cc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.937+0000 7f28d1c42640 1 --2- 172.21.15.5:0/220259593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28cc103660 0x7f28cc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.937+0000 7f28d1c42640 1 --2- 172.21.15.5:0/220259593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc102d40 0x7f28cc103120 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.937+0000 7f28d1c42640 1 -- 172.21.15.5:0/220259593 >> 172.21.15.5:0/220259593 conn(0x7f28cc0fc7a0 msgr2=0x7f28cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:15.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.937+0000 7f28d1c42640 1 -- 172.21.15.5:0/220259593 shutdown_connections 2024-09-19T06:02:15.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.937+0000 7f28d1c42640 1 -- 172.21.15.5:0/220259593 wait complete. 2024-09-19T06:02:15.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.937+0000 7f28d1c42640 1 Processor -- start 2024-09-19T06:02:15.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28d1c42640 1 -- start start 2024-09-19T06:02:15.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28d1c42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc102d40 0x7f28cc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:15.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28d1c42640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc103660 0x7f28cc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:15.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28d1c42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28cc10c170 0x7f28cc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:15.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28d1c42640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc111e20 con 0x7f28cc102d40 2024-09-19T06:02:15.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28d1c42640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc111ca0 con 0x7f28cc103660 2024-09-19T06:02:15.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28d1c42640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc111fa0 con 0x7f28cc10c170 2024-09-19T06:02:15.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc102d40 0x7f28cc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:15.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc102d40 0x7f28cc1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44360/0 (socket says 172.21.15.5:44360) 2024-09-19T06:02:15.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28cb7fe640 1 -- 172.21.15.5:0/3606774687 learned_addr learned my addr 172.21.15.5:0/3606774687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:15.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28caffd640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc103660 0x7f28cc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28cbfff640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28cc10c170 0x7f28cc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28cb7fe640 1 -- 172.21.15.5:0/3606774687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28cc10c170 msgr2=0x7f28cc19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28cb7fe640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28cc10c170 0x7f28cc19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28cb7fe640 1 -- 172.21.15.5:0/3606774687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc103660 msgr2=0x7f28cc1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28cb7fe640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc103660 0x7f28cc1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28cb7fe640 1 -- 172.21.15.5:0/3606774687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc19d350 con 0x7f28cc102d40 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28cbfff640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28cc10c170 0x7f28cc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.938+0000 7f28cb7fe640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc102d40 0x7f28cc1983a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f28bc005940 tx=0x7f28bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.939+0000 7f28c8ff9640 1 -- 172.21.15.5:0/3606774687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc03f030 con 0x7f28cc102d40 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.939+0000 7f28c8ff9640 1 -- 172.21.15.5:0/3606774687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28bc003aa0 con 0x7f28cc102d40 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.939+0000 7f28d1c42640 1 -- 172.21.15.5:0/3606774687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc19d5e0 con 0x7f28cc102d40 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.939+0000 7f28c8ff9640 1 -- 172.21.15.5:0/3606774687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc035070 con 0x7f28cc102d40 2024-09-19T06:02:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.939+0000 7f28d1c42640 1 -- 172.21.15.5:0/3606774687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc1a4f20 con 0x7f28cc102d40 2024-09-19T06:02:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.940+0000 7f28d1c42640 1 -- 172.21.15.5:0/3606774687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28cc062980 con 0x7f28cc102d40 2024-09-19T06:02:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.941+0000 7f28c8ff9640 1 -- 172.21.15.5:0/3606774687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f28bc042030 con 0x7f28cc102d40 2024-09-19T06:02:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.942+0000 7f28c8ff9640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f28b40a32d0 0x7f28b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.942+0000 7f28caffd640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f28b40a32d0 0x7f28b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.942+0000 7f28c8ff9640 1 -- 172.21.15.5:0/3606774687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f28bc0f0990 con 0x7f28cc102d40 2024-09-19T06:02:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.942+0000 7f28c8ff9640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400] conn(0x7f28b40acf00 0x7f28b40af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.942+0000 7f28c8ff9640 1 -- 172.21.15.5:0/3606774687 --> [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28bc001d30 con 0x7f28b40acf00 2024-09-19T06:02:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.942+0000 7f28c8ff9640 1 -- 172.21.15.5:0/3606774687 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f28bc0f0d40 con 0x7f28cc102d40 2024-09-19T06:02:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.942+0000 7f28caffd640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f28b40a32d0 0x7f28b40a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0fee10 tx=0x7f28c000b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.942+0000 7f28cbfff640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400] conn(0x7f28b40acf00 0x7f28b40af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.943+0000 7f28cbfff640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400] conn(0x7f28b40acf00 0x7f28b40af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.944+0000 7f28c8ff9640 1 -- 172.21.15.5:0/3606774687 <== osd.7 v2:172.21.15.49:6816/2959731400 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28bc001d30 con 0x7f28b40acf00 2024-09-19T06:02:15.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.957+0000 7f28d1c42640 1 -- 172.21.15.5:0/3606774687 --> [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f28cc062e00 con 0x7f28b40acf00 2024-09-19T06:02:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.958+0000 7f28c8ff9640 1 -- 172.21.15.5:0/3606774687 <== osd.7 v2:172.21.15.49:6816/2959731400 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f28cc062e00 con 0x7f28b40acf00 2024-09-19T06:02:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.958+0000 7f28d1c42640 1 -- 172.21.15.5:0/3606774687 >> [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400] conn(0x7f28b40acf00 msgr2=0x7f28b40af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.958+0000 7f28d1c42640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400] conn(0x7f28b40acf00 0x7f28b40af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.959+0000 7f28d1c42640 1 -- 172.21.15.5:0/3606774687 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f28b40a32d0 msgr2=0x7f28b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.959+0000 7f28d1c42640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f28b40a32d0 0x7f28b40a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0fee10 tx=0x7f28c000b2e0 comp rx=0 tx=0).stop 2024-09-19T06:02:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.959+0000 7f28d1c42640 1 -- 172.21.15.5:0/3606774687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc102d40 msgr2=0x7f28cc1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.959+0000 7f28d1c42640 1 --2- 172.21.15.5:0/3606774687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc102d40 0x7f28cc1983a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f28bc005940 tx=0x7f28bc004270 comp rx=0 tx=0).stop 2024-09-19T06:02:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.959+0000 7f28cbfff640 1 -- 172.21.15.5:0/3606774687 reap_dead start 2024-09-19T06:02:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.959+0000 7f28d1c42640 1 -- 172.21.15.5:0/3606774687 shutdown_connections 2024-09-19T06:02:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.959+0000 7f28d1c42640 1 -- 172.21.15.5:0/3606774687 >> 172.21.15.5:0/3606774687 conn(0x7f28cc0fc7a0 msgr2=0x7f28cc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.959+0000 7f28d1c42640 1 -- 172.21.15.5:0/3606774687 shutdown_connections 2024-09-19T06:02:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:15.959+0000 7f28d1c42640 1 -- 172.21.15.5:0/3606774687 wait complete. 2024-09-19T06:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout:240518168613 2024-09-19T06:02:16.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T06:02:16.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:16 smithi157 ceph-mon[35346]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-19T06:02:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:16 smithi005 ceph-mon[30955]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-19T06:02:16.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:16 smithi049 ceph-mon[34640]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-19T06:02:16.673 INFO:teuthology.orchestra.run.smithi005.stdout:210453397546 2024-09-19T06:02:16.673 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T06:02:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:17.013 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:17.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.569+0000 7f09ee740640 1 -- 172.21.15.5:0/1213159371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e8100a80 msgr2=0x7f09e8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:17.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.569+0000 7f09ee740640 1 --2- 172.21.15.5:0/1213159371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e8100a80 0x7f09e8100ee0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009930 tx=0x7f09dc02a420 comp rx=0 tx=0).stop 2024-09-19T06:02:17.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.570+0000 7f09ee740640 1 -- 172.21.15.5:0/1213159371 shutdown_connections 2024-09-19T06:02:17.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.570+0000 7f09ee740640 1 --2- 172.21.15.5:0/1213159371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09e8101420 0x7f09e810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:17.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.570+0000 7f09ee740640 1 --2- 172.21.15.5:0/1213159371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e8100a80 0x7f09e8100ee0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:17.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.570+0000 7f09ee740640 1 --2- 172.21.15.5:0/1213159371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09e8106a80 0x7f09e8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:17.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.570+0000 7f09ee740640 1 -- 172.21.15.5:0/1213159371 >> 172.21.15.5:0/1213159371 conn(0x7f09e80fc7a0 msgr2=0x7f09e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:17.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.570+0000 7f09ee740640 1 -- 172.21.15.5:0/1213159371 shutdown_connections 2024-09-19T06:02:17.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.570+0000 7f09ee740640 1 -- 172.21.15.5:0/1213159371 wait complete. 2024-09-19T06:02:17.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.570+0000 7f09ee740640 1 Processor -- start 2024-09-19T06:02:17.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.570+0000 7f09ee740640 1 -- start start 2024-09-19T06:02:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09ee740640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e8100a80 0x7f09e8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09ee740640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09e8101420 0x7f09e8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09ee740640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09e8106a80 0x7f09e819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09ee740640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8110dd0 con 0x7f09e8100a80 2024-09-19T06:02:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09ee740640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8110c50 con 0x7f09e8106a80 2024-09-19T06:02:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09ee740640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8110f50 con 0x7f09e8101420 2024-09-19T06:02:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e8100a80 0x7f09e8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e8100a80 0x7f09e8198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44380/0 (socket says 172.21.15.5:44380) 2024-09-19T06:02:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09e7fff640 1 -- 172.21.15.5:0/1808880512 learned_addr learned my addr 172.21.15.5:0/1808880512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09e77fe640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09e8101420 0x7f09e8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09eccdd640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09e8106a80 0x7f09e819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09e7fff640 1 -- 172.21.15.5:0/1808880512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09e8101420 msgr2=0x7f09e8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09e7fff640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09e8101420 0x7f09e8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09e7fff640 1 -- 172.21.15.5:0/1808880512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09e8106a80 msgr2=0x7f09e819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09e7fff640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09e8106a80 0x7f09e819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09e7fff640 1 -- 172.21.15.5:0/1808880512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e819d4d0 con 0x7f09e8100a80 2024-09-19T06:02:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09e7fff640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e8100a80 0x7f09e8198520 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f09d800b4a0 tx=0x7f09d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09e57fa640 1 -- 172.21.15.5:0/1808880512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8014070 con 0x7f09e8100a80 2024-09-19T06:02:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09e57fa640 1 -- 172.21.15.5:0/1808880512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09d8010ac0 con 0x7f09e8100a80 2024-09-19T06:02:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.571+0000 7f09ee740640 1 -- 172.21.15.5:0/1808880512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e819d7c0 con 0x7f09e8100a80 2024-09-19T06:02:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.572+0000 7f09e57fa640 1 -- 172.21.15.5:0/1808880512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8017c30 con 0x7f09e8100a80 2024-09-19T06:02:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.572+0000 7f09ee740640 1 -- 172.21.15.5:0/1808880512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e81a50a0 con 0x7f09e8100a80 2024-09-19T06:02:17.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.572+0000 7f09ee740640 1 -- 172.21.15.5:0/1808880512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e8100f60 con 0x7f09e8100a80 2024-09-19T06:02:17.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.574+0000 7f09e57fa640 1 -- 172.21.15.5:0/1808880512 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f09d8017070 con 0x7f09e8100a80 2024-09-19T06:02:17.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.574+0000 7f09e57fa640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f09d00a32d0 0x7f09d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:17.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.574+0000 7f09e77fe640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f09d00a32d0 0x7f09d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:17.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.574+0000 7f09e57fa640 1 -- 172.21.15.5:0/1808880512 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f09d80cefd0 con 0x7f09e8100a80 2024-09-19T06:02:17.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.575+0000 7f09e77fe640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f09d00a32d0 0x7f09d00a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f09dc006010 tx=0x7f09dc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:17.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.579+0000 7f09e57fa640 1 -- 172.21.15.5:0/1808880512 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f09d809bd30 con 0x7f09e8100a80 2024-09-19T06:02:17.591 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:17.592 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.779+0000 7f09ee740640 1 -- 172.21.15.5:0/1808880512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f09e810a020 con 0x7f09e8100a80 2024-09-19T06:02:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.779+0000 7f09e57fa640 1 -- 172.21.15.5:0/1808880512 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f09d809b630 con 0x7f09e8100a80 2024-09-19T06:02:17.781 INFO:teuthology.orchestra.run.smithi005.stdout:137438953530 2024-09-19T06:02:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.782+0000 7f09ee740640 1 -- 172.21.15.5:0/1808880512 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f09d00a32d0 msgr2=0x7f09d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.782+0000 7f09ee740640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f09d00a32d0 0x7f09d00a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f09dc006010 tx=0x7f09dc000bc0 comp rx=0 tx=0).stop 2024-09-19T06:02:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.782+0000 7f09ee740640 1 -- 172.21.15.5:0/1808880512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e8100a80 msgr2=0x7f09e8198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.782+0000 7f09ee740640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e8100a80 0x7f09e8198520 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f09d800b4a0 tx=0x7f09d800b960 comp rx=0 tx=0).stop 2024-09-19T06:02:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.782+0000 7f09ee740640 1 -- 172.21.15.5:0/1808880512 shutdown_connections 2024-09-19T06:02:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.782+0000 7f09ee740640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f09d00a32d0 0x7f09d00a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.782+0000 7f09ee740640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09e8106a80 0x7f09e819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.782+0000 7f09ee740640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09e8101420 0x7f09e8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.782+0000 7f09ee740640 1 --2- 172.21.15.5:0/1808880512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e8100a80 0x7f09e8198520 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.782+0000 7f09ee740640 1 -- 172.21.15.5:0/1808880512 >> 172.21.15.5:0/1808880512 conn(0x7f09e80fc7a0 msgr2=0x7f09e806abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.782+0000 7f09ee740640 1 -- 172.21.15.5:0/1808880512 shutdown_connections 2024-09-19T06:02:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:17.782+0000 7f09ee740640 1 -- 172.21.15.5:0/1808880512 wait complete. 2024-09-19T06:02:18.530 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:18 smithi005 ceph-mon[30955]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-19T06:02:18.530 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:18 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1808880512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T06:02:18.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:18 smithi157 ceph-mon[35346]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-19T06:02:18.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:18 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1808880512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T06:02:18.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:18 smithi049 ceph-mon[34640]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-19T06:02:18.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:18 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1808880512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T06:02:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.704+0000 7fb899a62640 1 -- 172.21.15.5:0/1348098226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89410c590 msgr2=0x7fb89410c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.704+0000 7fb899a62640 1 --2- 172.21.15.5:0/1348098226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89410c590 0x7fb89410c9f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb888009900 tx=0x7fb88802a410 comp rx=0 tx=0).stop 2024-09-19T06:02:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.704+0000 7fb899a62640 1 -- 172.21.15.5:0/1348098226 shutdown_connections 2024-09-19T06:02:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.704+0000 7fb899a62640 1 --2- 172.21.15.5:0/1348098226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb89410cf30 0x7fb894119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.704+0000 7fb899a62640 1 --2- 172.21.15.5:0/1348098226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89410c590 0x7fb89410c9f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.704+0000 7fb899a62640 1 --2- 172.21.15.5:0/1348098226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb894112640 0x7fb894112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.704+0000 7fb899a62640 1 -- 172.21.15.5:0/1348098226 >> 172.21.15.5:0/1348098226 conn(0x7fb89406cdc0 msgr2=0x7fb89406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.704+0000 7fb899a62640 1 -- 172.21.15.5:0/1348098226 shutdown_connections 2024-09-19T06:02:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.704+0000 7fb899a62640 1 -- 172.21.15.5:0/1348098226 wait complete. 2024-09-19T06:02:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.705+0000 7fb899a62640 1 Processor -- start 2024-09-19T06:02:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.705+0000 7fb899a62640 1 -- start start 2024-09-19T06:02:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.705+0000 7fb899a62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb89410c590 0x7fb8941a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.705+0000 7fb899a62640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89410cf30 0x7fb8941a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.705+0000 7fb899a62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894112640 0x7fb8941a7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.705+0000 7fb899a62640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb89411c8d0 con 0x7fb89410cf30 2024-09-19T06:02:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.705+0000 7fb899a62640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb89411c750 con 0x7fb894112640 2024-09-19T06:02:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.705+0000 7fb899a62640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb89411ca50 con 0x7fb89410c590 2024-09-19T06:02:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.705+0000 7fb892ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89410cf30 0x7fb8941a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.705+0000 7fb892ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89410cf30 0x7fb8941a32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44398/0 (socket says 172.21.15.5:44398) 2024-09-19T06:02:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.705+0000 7fb892ffd640 1 -- 172.21.15.5:0/1876150779 learned_addr learned my addr 172.21.15.5:0/1876150779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.705+0000 7fb8937fe640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb89410c590 0x7fb8941a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb893fff640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894112640 0x7fb8941a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb892ffd640 1 -- 172.21.15.5:0/1876150779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb89410c590 msgr2=0x7fb8941a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb892ffd640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb89410c590 0x7fb8941a2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb892ffd640 1 -- 172.21.15.5:0/1876150779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894112640 msgr2=0x7fb8941a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb892ffd640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894112640 0x7fb8941a7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb892ffd640 1 -- 172.21.15.5:0/1876150779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8941a7d40 con 0x7fb89410cf30 2024-09-19T06:02:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb893fff640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894112640 0x7fb8941a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb892ffd640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89410cf30 0x7fb8941a32d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb888006010 tx=0x7fb8880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb890ff9640 1 -- 172.21.15.5:0/1876150779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88803f070 con 0x7fb89410cf30 2024-09-19T06:02:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb899a62640 1 -- 172.21.15.5:0/1876150779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8941a7fd0 con 0x7fb89410cf30 2024-09-19T06:02:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb890ff9640 1 -- 172.21.15.5:0/1876150779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb888003aa0 con 0x7fb89410cf30 2024-09-19T06:02:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb890ff9640 1 -- 172.21.15.5:0/1876150779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888035030 con 0x7fb89410cf30 2024-09-19T06:02:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.706+0000 7fb899a62640 1 -- 172.21.15.5:0/1876150779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8941af910 con 0x7fb89410cf30 2024-09-19T06:02:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.707+0000 7fb899a62640 1 -- 172.21.15.5:0/1876150779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89410dcb0 con 0x7fb89410cf30 2024-09-19T06:02:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.708+0000 7fb890ff9640 1 -- 172.21.15.5:0/1876150779 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fb888042070 con 0x7fb89410cf30 2024-09-19T06:02:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.709+0000 7fb890ff9640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb87c0a32d0 0x7fb87c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.709+0000 7fb890ff9640 1 -- 172.21.15.5:0/1876150779 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb8880f04c0 con 0x7fb89410cf30 2024-09-19T06:02:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.709+0000 7fb8937fe640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb87c0a32d0 0x7fb87c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.709+0000 7fb8937fe640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb87c0a32d0 0x7fb87c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb884009990 tx=0x7fb884009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.713+0000 7fb890ff9640 1 -- 172.21.15.5:0/1876150779 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb8880bd010 con 0x7fb89410cf30 2024-09-19T06:02:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.892+0000 7fb899a62640 1 -- 172.21.15.5:0/1876150779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb8941144b0 con 0x7fb89410cf30 2024-09-19T06:02:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.892+0000 7fb890ff9640 1 -- 172.21.15.5:0/1876150779 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb8880a70e0 con 0x7fb89410cf30 2024-09-19T06:02:18.894 INFO:teuthology.orchestra.run.smithi005.stdout:55834574921 2024-09-19T06:02:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.894+0000 7fb899a62640 1 -- 172.21.15.5:0/1876150779 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb87c0a32d0 msgr2=0x7fb87c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.894+0000 7fb899a62640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb87c0a32d0 0x7fb87c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb884009990 tx=0x7fb884009110 comp rx=0 tx=0).stop 2024-09-19T06:02:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.894+0000 7fb899a62640 1 -- 172.21.15.5:0/1876150779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89410cf30 msgr2=0x7fb8941a32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.894+0000 7fb899a62640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89410cf30 0x7fb8941a32d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb888006010 tx=0x7fb8880042f0 comp rx=0 tx=0).stop 2024-09-19T06:02:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.894+0000 7fb899a62640 1 -- 172.21.15.5:0/1876150779 shutdown_connections 2024-09-19T06:02:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.894+0000 7fb899a62640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb87c0a32d0 0x7fb87c0a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.894+0000 7fb899a62640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894112640 0x7fb8941a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.894+0000 7fb899a62640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89410cf30 0x7fb8941a32d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.894+0000 7fb899a62640 1 --2- 172.21.15.5:0/1876150779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb89410c590 0x7fb8941a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.894+0000 7fb899a62640 1 -- 172.21.15.5:0/1876150779 >> 172.21.15.5:0/1876150779 conn(0x7fb89406cdc0 msgr2=0x7fb89410af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:18.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.894+0000 7fb899a62640 1 -- 172.21.15.5:0/1876150779 shutdown_connections 2024-09-19T06:02:18.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:18.894+0000 7fb899a62640 1 -- 172.21.15.5:0/1876150779 wait complete. 2024-09-19T06:02:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.030+0000 7f6e564c8640 1 -- 172.21.15.5:0/47018847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e50100e20 msgr2=0x7f6e50101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.030+0000 7f6e564c8640 1 --2- 172.21.15.5:0/47018847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e50100e20 0x7f6e50101200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009900 tx=0x7f6e4002a480 comp rx=0 tx=0).stop 2024-09-19T06:02:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.030+0000 7f6e564c8640 1 -- 172.21.15.5:0/47018847 shutdown_connections 2024-09-19T06:02:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.030+0000 7f6e564c8640 1 --2- 172.21.15.5:0/47018847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e50102ef0 0x7f6e5010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.030+0000 7f6e564c8640 1 --2- 172.21.15.5:0/47018847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e50102570 0x7f6e501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.030+0000 7f6e564c8640 1 --2- 172.21.15.5:0/47018847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e50100e20 0x7f6e50101200 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.030+0000 7f6e564c8640 1 -- 172.21.15.5:0/47018847 >> 172.21.15.5:0/47018847 conn(0x7f6e500fc7d0 msgr2=0x7f6e500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.030+0000 7f6e564c8640 1 -- 172.21.15.5:0/47018847 shutdown_connections 2024-09-19T06:02:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.030+0000 7f6e564c8640 1 -- 172.21.15.5:0/47018847 wait complete. 2024-09-19T06:02:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.030+0000 7f6e564c8640 1 Processor -- start 2024-09-19T06:02:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.030+0000 7f6e564c8640 1 -- start start 2024-09-19T06:02:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e564c8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e50100e20 0x7f6e500fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e564c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e50102570 0x7f6e500fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e564c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e50102ef0 0x7f6e500fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e564c8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6e50110e50 con 0x7f6e50102ef0 2024-09-19T06:02:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e564c8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6e50110cd0 con 0x7f6e50100e20 2024-09-19T06:02:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e564c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6e50110fd0 con 0x7f6e50102570 2024-09-19T06:02:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e54a65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e50102ef0 0x7f6e500fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e54a65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e50102ef0 0x7f6e500fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44426/0 (socket says 172.21.15.5:44426) 2024-09-19T06:02:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e54a65640 1 -- 172.21.15.5:0/3131459504 learned_addr learned my addr 172.21.15.5:0/3131459504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e4ffff640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e50100e20 0x7f6e500fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e4f7fe640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e50102570 0x7f6e500fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e54a65640 1 -- 172.21.15.5:0/3131459504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e50102570 msgr2=0x7f6e500fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e54a65640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e50102570 0x7f6e500fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e54a65640 1 -- 172.21.15.5:0/3131459504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e50100e20 msgr2=0x7f6e500fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e54a65640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e50100e20 0x7f6e500fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e54a65640 1 -- 172.21.15.5:0/3131459504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e501a69f0 con 0x7f6e50102ef0 2024-09-19T06:02:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.031+0000 7f6e54a65640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e50102ef0 0x7f6e500fd700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800eaa0 tx=0x7f6e4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.032+0000 7f6e4d7fa640 1 -- 172.21.15.5:0/3131459504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800cdb0 con 0x7f6e50102ef0 2024-09-19T06:02:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.032+0000 7f6e4d7fa640 1 -- 172.21.15.5:0/3131459504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e48005020 con 0x7f6e50102ef0 2024-09-19T06:02:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.032+0000 7f6e564c8640 1 -- 172.21.15.5:0/3131459504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e501a6ce0 con 0x7f6e50102ef0 2024-09-19T06:02:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.032+0000 7f6e4d7fa640 1 -- 172.21.15.5:0/3131459504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e480116d0 con 0x7f6e50102ef0 2024-09-19T06:02:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.032+0000 7f6e564c8640 1 -- 172.21.15.5:0/3131459504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e501a7220 con 0x7f6e50102ef0 2024-09-19T06:02:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.033+0000 7f6e564c8640 1 -- 172.21.15.5:0/3131459504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e50101200 con 0x7f6e50102ef0 2024-09-19T06:02:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.034+0000 7f6e4d7fa640 1 -- 172.21.15.5:0/3131459504 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f6e480051d0 con 0x7f6e50102ef0 2024-09-19T06:02:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.035+0000 7f6e4d7fa640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6e380a3350 0x7f6e380a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.035+0000 7f6e4ffff640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6e380a3350 0x7f6e380a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.035+0000 7f6e4d7fa640 1 -- 172.21.15.5:0/3131459504 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f6e480cfc50 con 0x7f6e50102ef0 2024-09-19T06:02:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.035+0000 7f6e4ffff640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6e380a3350 0x7f6e380a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009230 tx=0x7f6e400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.039+0000 7f6e4d7fa640 1 -- 172.21.15.5:0/3131459504 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6e48016030 con 0x7f6e50102ef0 2024-09-19T06:02:19.090 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953527 got 137438953530 for osd.4 2024-09-19T06:02:19.090 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:19.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.221+0000 7f6e564c8640 1 -- 172.21.15.5:0/3131459504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6e5010a0a0 con 0x7f6e50102ef0 2024-09-19T06:02:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.222+0000 7f6e4d7fa640 1 -- 172.21.15.5:0/3131459504 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6e4809c800 con 0x7f6e50102ef0 2024-09-19T06:02:19.224 INFO:teuthology.orchestra.run.smithi005.stdout:34359738446 2024-09-19T06:02:19.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.224+0000 7f6e564c8640 1 -- 172.21.15.5:0/3131459504 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6e380a3350 msgr2=0x7f6e380a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:19.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.224+0000 7f6e564c8640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6e380a3350 0x7f6e380a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009230 tx=0x7f6e400058d0 comp rx=0 tx=0).stop 2024-09-19T06:02:19.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.224+0000 7f6e564c8640 1 -- 172.21.15.5:0/3131459504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e50102ef0 msgr2=0x7f6e500fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:19.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.224+0000 7f6e564c8640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e50102ef0 0x7f6e500fd700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800eaa0 tx=0x7f6e4800ef60 comp rx=0 tx=0).stop 2024-09-19T06:02:19.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.224+0000 7f6e564c8640 1 -- 172.21.15.5:0/3131459504 shutdown_connections 2024-09-19T06:02:19.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.224+0000 7f6e564c8640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f6e380a3350 0x7f6e380a5810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:19.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.224+0000 7f6e564c8640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e50102ef0 0x7f6e500fd700 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:19.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.224+0000 7f6e564c8640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e50102570 0x7f6e500fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.224+0000 7f6e564c8640 1 --2- 172.21.15.5:0/3131459504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e50100e20 0x7f6e500fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.224+0000 7f6e564c8640 1 -- 172.21.15.5:0/3131459504 >> 172.21.15.5:0/3131459504 conn(0x7f6e500fc7d0 msgr2=0x7f6e5006a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.224+0000 7f6e564c8640 1 -- 172.21.15.5:0/3131459504 shutdown_connections 2024-09-19T06:02:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:19.224+0000 7f6e564c8640 1 -- 172.21.15.5:0/3131459504 wait complete. 2024-09-19T06:02:19.487 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:19 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1876150779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T06:02:19.487 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:19 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3131459504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T06:02:19.514 INFO:teuthology.orchestra.run.smithi005.stdout:270582939680 2024-09-19T06:02:19.514 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T06:02:19.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:19 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1876150779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T06:02:19.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:19 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3131459504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T06:02:19.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:19 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1876150779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T06:02:19.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:19 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3131459504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T06:02:20.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:20 smithi005 ceph-mon[30955]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-19T06:02:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:20.397 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:20.399 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:20.433 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738446 for osd.0 2024-09-19T06:02:20.434 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:20.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:20 smithi157 ceph-mon[35346]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-19T06:02:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:20 smithi049 ceph-mon[34640]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-19T06:02:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.813+0000 7f371da6f640 1 -- 172.21.15.5:0/3860822773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718101420 msgr2=0x7f371810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.813+0000 7f371da6f640 1 --2- 172.21.15.5:0/3860822773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718101420 0x7f371810d950 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3710009cd0 tx=0x7f371002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:02:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.813+0000 7f371da6f640 1 -- 172.21.15.5:0/3860822773 shutdown_connections 2024-09-19T06:02:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.813+0000 7f371da6f640 1 --2- 172.21.15.5:0/3860822773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718101420 0x7f371810d950 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.813+0000 7f371da6f640 1 --2- 172.21.15.5:0/3860822773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3718100a80 0x7f3718100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.813+0000 7f371da6f640 1 --2- 172.21.15.5:0/3860822773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3718106a80 0x7f3718106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.813+0000 7f371da6f640 1 -- 172.21.15.5:0/3860822773 >> 172.21.15.5:0/3860822773 conn(0x7f37180fc7a0 msgr2=0x7f37180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.813+0000 7f371da6f640 1 -- 172.21.15.5:0/3860822773 shutdown_connections 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.813+0000 7f371da6f640 1 -- 172.21.15.5:0/3860822773 wait complete. 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.813+0000 7f371da6f640 1 Processor -- start 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f371da6f640 1 -- start start 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f371da6f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718100a80 0x7f3718198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f371da6f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3718101420 0x7f37181988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f371da6f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3718106a80 0x7f371819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f371da6f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3718110d30 con 0x7f3718100a80 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f371da6f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3718110bb0 con 0x7f3718101420 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f371da6f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3718110eb0 con 0x7f3718106a80 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f37177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718100a80 0x7f3718198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f37177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718100a80 0x7f3718198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44436/0 (socket says 172.21.15.5:44436) 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f37177fe640 1 -- 172.21.15.5:0/565481345 learned_addr learned my addr 172.21.15.5:0/565481345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f3717fff640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3718106a80 0x7f371819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f3716ffd640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3718101420 0x7f37181988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f37177fe640 1 -- 172.21.15.5:0/565481345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3718106a80 msgr2=0x7f371819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f37177fe640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3718106a80 0x7f371819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f37177fe640 1 -- 172.21.15.5:0/565481345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3718101420 msgr2=0x7f37181988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f37177fe640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3718101420 0x7f37181988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.814+0000 7f37177fe640 1 -- 172.21.15.5:0/565481345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371819d310 con 0x7f3718100a80 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.815+0000 7f37177fe640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718100a80 0x7f3718198360 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f370800d950 tx=0x7f370800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.815+0000 7f3714ff9640 1 -- 172.21.15.5:0/565481345 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708017070 con 0x7f3718100a80 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.815+0000 7f3714ff9640 1 -- 172.21.15.5:0/565481345 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3708009c30 con 0x7f3718100a80 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.815+0000 7f371da6f640 1 -- 172.21.15.5:0/565481345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371819d600 con 0x7f3718100a80 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.815+0000 7f3714ff9640 1 -- 172.21.15.5:0/565481345 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370800f620 con 0x7f3718100a80 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.815+0000 7f371da6f640 1 -- 172.21.15.5:0/565481345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37181a4ee0 con 0x7f3718100a80 2024-09-19T06:02:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.816+0000 7f371da6f640 1 -- 172.21.15.5:0/565481345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3718062980 con 0x7f3718100a80 2024-09-19T06:02:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.817+0000 7f3714ff9640 1 -- 172.21.15.5:0/565481345 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f37080109a0 con 0x7f3718100a80 2024-09-19T06:02:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.818+0000 7f3714ff9640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f37040a3350 0x7f37040a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.818+0000 7f3716ffd640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f37040a3350 0x7f37040a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.818+0000 7f3714ff9640 1 -- 172.21.15.5:0/565481345 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f37080cf940 con 0x7f3718100a80 2024-09-19T06:02:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.818+0000 7f3714ff9640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971] conn(0x7f37040acf80 0x7f37040af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.818+0000 7f3714ff9640 1 -- 172.21.15.5:0/565481345 --> [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f370801de50 con 0x7f37040acf80 2024-09-19T06:02:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.818+0000 7f3714ff9640 1 -- 172.21.15.5:0/565481345 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f37080cfd90 con 0x7f3718100a80 2024-09-19T06:02:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.818+0000 7f3716ffd640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f37040a3350 0x7f37040a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f37181051c0 tx=0x7f370000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.818+0000 7f3717fff640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971] conn(0x7f37040acf80 0x7f37040af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.819+0000 7f3717fff640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971] conn(0x7f37040acf80 0x7f37040af420 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.820+0000 7f3714ff9640 1 -- 172.21.15.5:0/565481345 <== osd.5 v2:172.21.15.157:6808/3219063971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f370801de50 con 0x7f37040acf80 2024-09-19T06:02:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.833+0000 7f371da6f640 1 -- 172.21.15.5:0/565481345 --> [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3718062d80 con 0x7f37040acf80 2024-09-19T06:02:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.834+0000 7f3714ff9640 1 -- 172.21.15.5:0/565481345 <== osd.5 v2:172.21.15.157:6808/3219063971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3718062d80 con 0x7f37040acf80 2024-09-19T06:02:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.834+0000 7f371da6f640 1 -- 172.21.15.5:0/565481345 >> [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971] conn(0x7f37040acf80 msgr2=0x7f37040af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.834+0000 7f371da6f640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971] conn(0x7f37040acf80 0x7f37040af420 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.834+0000 7f371da6f640 1 -- 172.21.15.5:0/565481345 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f37040a3350 msgr2=0x7f37040a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.834+0000 7f371da6f640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f37040a3350 0x7f37040a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f37181051c0 tx=0x7f370000b410 comp rx=0 tx=0).stop 2024-09-19T06:02:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.834+0000 7f371da6f640 1 -- 172.21.15.5:0/565481345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718100a80 msgr2=0x7f3718198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.834+0000 7f371da6f640 1 --2- 172.21.15.5:0/565481345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718100a80 0x7f3718198360 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f370800d950 tx=0x7f370800de10 comp rx=0 tx=0).stop 2024-09-19T06:02:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.834+0000 7f3717fff640 1 -- 172.21.15.5:0/565481345 reap_dead start 2024-09-19T06:02:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.834+0000 7f371da6f640 1 -- 172.21.15.5:0/565481345 shutdown_connections 2024-09-19T06:02:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.834+0000 7f371da6f640 1 -- 172.21.15.5:0/565481345 >> 172.21.15.5:0/565481345 conn(0x7f37180fc7a0 msgr2=0x7f371806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.834+0000 7f371da6f640 1 -- 172.21.15.5:0/565481345 shutdown_connections 2024-09-19T06:02:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:20.834+0000 7f371da6f640 1 -- 172.21.15.5:0/565481345 wait complete. 2024-09-19T06:02:21.465 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574921 for osd.1 2024-09-19T06:02:21.465 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.744+0000 7f7d24def640 1 -- 172.21.15.5:0/2498246695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d20102790 msgr2=0x7f7d201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.744+0000 7f7d24def640 1 --2- 172.21.15.5:0/2498246695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d20102790 0x7f7d201067e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009900 tx=0x7f7d0802a410 comp rx=0 tx=0).stop 2024-09-19T06:02:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.744+0000 7f7d24def640 1 -- 172.21.15.5:0/2498246695 shutdown_connections 2024-09-19T06:02:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.744+0000 7f7d24def640 1 --2- 172.21.15.5:0/2498246695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d20106f10 0x7f7d2010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.744+0000 7f7d24def640 1 --2- 172.21.15.5:0/2498246695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d20102790 0x7f7d201067e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.744+0000 7f7d24def640 1 --2- 172.21.15.5:0/2498246695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d20101de0 0x7f7d201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.744+0000 7f7d24def640 1 -- 172.21.15.5:0/2498246695 >> 172.21.15.5:0/2498246695 conn(0x7f7d200fd8c0 msgr2=0x7f7d200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.744+0000 7f7d24def640 1 -- 172.21.15.5:0/2498246695 shutdown_connections 2024-09-19T06:02:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.744+0000 7f7d24def640 1 -- 172.21.15.5:0/2498246695 wait complete. 2024-09-19T06:02:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.745+0000 7f7d24def640 1 Processor -- start 2024-09-19T06:02:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.745+0000 7f7d24def640 1 -- start start 2024-09-19T06:02:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.745+0000 7f7d24def640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d20101de0 0x7f7d200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.745+0000 7f7d24def640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d20102790 0x7f7d20074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.745+0000 7f7d24def640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d20106f10 0x7f7d20074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.745+0000 7f7d24def640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20111e60 con 0x7f7d20102790 2024-09-19T06:02:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.745+0000 7f7d24def640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20111ce0 con 0x7f7d20101de0 2024-09-19T06:02:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.745+0000 7f7d24def640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20111fe0 con 0x7f7d20106f10 2024-09-19T06:02:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.745+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d20102790 0x7f7d20074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d20102790 0x7f7d20074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44446/0 (socket says 172.21.15.5:44446) 2024-09-19T06:02:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d17fff640 1 -- 172.21.15.5:0/1211335553 learned_addr learned my addr 172.21.15.5:0/1211335553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d1ed9d640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d20106f10 0x7f7d20074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d1e59c640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d20101de0 0x7f7d200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d17fff640 1 -- 172.21.15.5:0/1211335553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d20106f10 msgr2=0x7f7d20074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d17fff640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d20106f10 0x7f7d20074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d17fff640 1 -- 172.21.15.5:0/1211335553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d20101de0 msgr2=0x7f7d200744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d17fff640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d20101de0 0x7f7d200744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d17fff640 1 -- 172.21.15.5:0/1211335553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d201a6920 con 0x7f7d20102790 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d17fff640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d20102790 0x7f7d20074a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d08005940 tx=0x7f7d08004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d177fe640 1 -- 172.21.15.5:0/1211335553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0803f030 con 0x7f7d20102790 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d24def640 1 -- 172.21.15.5:0/1211335553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d201a6bb0 con 0x7f7d20102790 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d177fe640 1 -- 172.21.15.5:0/1211335553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d08003aa0 con 0x7f7d20102790 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d177fe640 1 -- 172.21.15.5:0/1211335553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08035070 con 0x7f7d20102790 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.746+0000 7f7d24def640 1 -- 172.21.15.5:0/1211335553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d201a70e0 con 0x7f7d20102790 2024-09-19T06:02:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.747+0000 7f7d24def640 1 -- 172.21.15.5:0/1211335553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d201021c0 con 0x7f7d20102790 2024-09-19T06:02:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.749+0000 7f7d177fe640 1 -- 172.21.15.5:0/1211335553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f7d08042030 con 0x7f7d20102790 2024-09-19T06:02:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.749+0000 7f7d177fe640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f7d0c0a32d0 0x7f7d0c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.749+0000 7f7d1e59c640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f7d0c0a32d0 0x7f7d0c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.749+0000 7f7d177fe640 1 -- 172.21.15.5:0/1211335553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7d080f0390 con 0x7f7d20102790 2024-09-19T06:02:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.750+0000 7f7d1e59c640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f7d0c0a32d0 0x7f7d0c0a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7d10005fd0 tx=0x7f7d1000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.752+0000 7f7d177fe640 1 -- 172.21.15.5:0/1211335553 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7d080bc8f0 con 0x7f7d20102790 2024-09-19T06:02:21.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.931+0000 7f7d24def640 1 -- 172.21.15.5:0/1211335553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f7d2010a150 con 0x7f7d20102790 2024-09-19T06:02:21.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.931+0000 7f7d177fe640 1 -- 172.21.15.5:0/1211335553 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7d080c0bf0 con 0x7f7d20102790 2024-09-19T06:02:21.933 INFO:teuthology.orchestra.run.smithi005.stdout:322122547223 2024-09-19T06:02:21.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.933+0000 7f7d24def640 1 -- 172.21.15.5:0/1211335553 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f7d0c0a32d0 msgr2=0x7f7d0c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.933+0000 7f7d24def640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f7d0c0a32d0 0x7f7d0c0a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7d10005fd0 tx=0x7f7d1000b390 comp rx=0 tx=0).stop 2024-09-19T06:02:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.933+0000 7f7d24def640 1 -- 172.21.15.5:0/1211335553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d20102790 msgr2=0x7f7d20074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.933+0000 7f7d24def640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d20102790 0x7f7d20074a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d08005940 tx=0x7f7d08004270 comp rx=0 tx=0).stop 2024-09-19T06:02:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.934+0000 7f7d24def640 1 -- 172.21.15.5:0/1211335553 shutdown_connections 2024-09-19T06:02:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.934+0000 7f7d24def640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f7d0c0a32d0 0x7f7d0c0a5790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.934+0000 7f7d24def640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d20106f10 0x7f7d20074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.934+0000 7f7d24def640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d20102790 0x7f7d20074a30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.934+0000 7f7d24def640 1 --2- 172.21.15.5:0/1211335553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d20101de0 0x7f7d200744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.934+0000 7f7d24def640 1 -- 172.21.15.5:0/1211335553 >> 172.21.15.5:0/1211335553 conn(0x7f7d200fd8c0 msgr2=0x7f7d200fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.934+0000 7f7d24def640 1 -- 172.21.15.5:0/1211335553 shutdown_connections 2024-09-19T06:02:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:21.934+0000 7f7d24def640 1 -- 172.21.15.5:0/1211335553 wait complete. 2024-09-19T06:02:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:22.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:22 smithi005 ceph-mon[30955]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-09-19T06:02:22.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:22 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1211335553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T06:02:22.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:22 smithi157 ceph-mon[35346]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-09-19T06:02:22.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:22 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1211335553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T06:02:22.605 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547221 got 322122547223 for osd.11 2024-09-19T06:02:22.605 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:22.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:22 smithi049 ceph-mon[34640]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-09-19T06:02:22.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:22 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1211335553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T06:02:23.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.094+0000 7fd3675e3640 1 -- 172.21.15.5:0/1106505368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd360106a80 msgr2=0x7fd360106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:23.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.094+0000 7fd3675e3640 1 --2- 172.21.15.5:0/1106505368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd360106a80 0x7fd360106e60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009930 tx=0x7fd35c02a420 comp rx=0 tx=0).stop 2024-09-19T06:02:23.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.094+0000 7fd3675e3640 1 -- 172.21.15.5:0/1106505368 shutdown_connections 2024-09-19T06:02:23.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.094+0000 7fd3675e3640 1 --2- 172.21.15.5:0/1106505368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd360101420 0x7fd36010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:23.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.094+0000 7fd3675e3640 1 --2- 172.21.15.5:0/1106505368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd360100a80 0x7fd360100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:23.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.094+0000 7fd3675e3640 1 --2- 172.21.15.5:0/1106505368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd360106a80 0x7fd360106e60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:23.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.094+0000 7fd3675e3640 1 -- 172.21.15.5:0/1106505368 >> 172.21.15.5:0/1106505368 conn(0x7fd3600fc7a0 msgr2=0x7fd3600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:23.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.094+0000 7fd3675e3640 1 -- 172.21.15.5:0/1106505368 shutdown_connections 2024-09-19T06:02:23.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.094+0000 7fd3675e3640 1 -- 172.21.15.5:0/1106505368 wait complete. 2024-09-19T06:02:23.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd3675e3640 1 Processor -- start 2024-09-19T06:02:23.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd3675e3640 1 -- start start 2024-09-19T06:02:23.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd3675e3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd360100a80 0x7fd360199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:23.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd3675e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd360101420 0x7fd3601997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd3675e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd360106a80 0x7fd3601a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd3675e3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd360110d70 con 0x7fd360101420 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd3675e3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd360110bf0 con 0x7fd360100a80 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd3675e3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd360110ef0 con 0x7fd360106a80 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd364b7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd360101420 0x7fd3601997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd364b7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd360101420 0x7fd3601997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44462/0 (socket says 172.21.15.5:44462) 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd364b7e640 1 -- 172.21.15.5:0/1689028671 learned_addr learned my addr 172.21.15.5:0/1689028671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd36537f640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd360100a80 0x7fd360199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.095+0000 7fd365b80640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd360106a80 0x7fd3601a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.096+0000 7fd364b7e640 1 -- 172.21.15.5:0/1689028671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd360106a80 msgr2=0x7fd3601a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.096+0000 7fd364b7e640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd360106a80 0x7fd3601a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.096+0000 7fd364b7e640 1 -- 172.21.15.5:0/1689028671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd360100a80 msgr2=0x7fd360199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.096+0000 7fd364b7e640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd360100a80 0x7fd360199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.096+0000 7fd364b7e640 1 -- 172.21.15.5:0/1689028671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3601a4840 con 0x7fd360101420 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.096+0000 7fd364b7e640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd360101420 0x7fd3601997c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd35000d770 tx=0x7fd35000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.096+0000 7fd3567fc640 1 -- 172.21.15.5:0/1689028671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350017070 con 0x7fd360101420 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.096+0000 7fd3567fc640 1 -- 172.21.15.5:0/1689028671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3500054d0 con 0x7fd360101420 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.096+0000 7fd3567fc640 1 -- 172.21.15.5:0/1689028671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350004cf0 con 0x7fd360101420 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.096+0000 7fd3675e3640 1 -- 172.21.15.5:0/1689028671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3601a4a10 con 0x7fd360101420 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.096+0000 7fd3675e3640 1 -- 172.21.15.5:0/1689028671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3601a4f20 con 0x7fd360101420 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.097+0000 7fd3675e3640 1 -- 172.21.15.5:0/1689028671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd360100ee0 con 0x7fd360101420 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.098+0000 7fd3567fc640 1 -- 172.21.15.5:0/1689028671 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fd350004e90 con 0x7fd360101420 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.099+0000 7fd3567fc640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd34c0a3350 0x7fd34c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.099+0000 7fd3567fc640 1 -- 172.21.15.5:0/1689028671 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd3500cf830 con 0x7fd360101420 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.099+0000 7fd36537f640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd34c0a3350 0x7fd34c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.100+0000 7fd36537f640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd34c0a3350 0x7fd34c0a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009900 tx=0x7fd35c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.102+0000 7fd3567fc640 1 -- 172.21.15.5:0/1689028671 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd35009be10 con 0x7fd360101420 2024-09-19T06:02:23.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.297+0000 7fd3675e3640 1 -- 172.21.15.5:0/1689028671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd360108ff0 con 0x7fd360101420 2024-09-19T06:02:23.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.297+0000 7fd3567fc640 1 -- 172.21.15.5:0/1689028671 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd3500a0110 con 0x7fd360101420 2024-09-19T06:02:23.301 INFO:teuthology.orchestra.run.smithi005.stdout:292057776157 2024-09-19T06:02:23.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.300+0000 7fd3675e3640 1 -- 172.21.15.5:0/1689028671 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd34c0a3350 msgr2=0x7fd34c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:23.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.300+0000 7fd3675e3640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd34c0a3350 0x7fd34c0a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009900 tx=0x7fd35c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:02:23.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.300+0000 7fd3675e3640 1 -- 172.21.15.5:0/1689028671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd360101420 msgr2=0x7fd3601997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:23.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.300+0000 7fd3675e3640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd360101420 0x7fd3601997c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd35000d770 tx=0x7fd35000dc30 comp rx=0 tx=0).stop 2024-09-19T06:02:23.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.300+0000 7fd3675e3640 1 -- 172.21.15.5:0/1689028671 shutdown_connections 2024-09-19T06:02:23.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.300+0000 7fd3675e3640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd34c0a3350 0x7fd34c0a5810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:23.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.300+0000 7fd3675e3640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd360106a80 0x7fd3601a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:23.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.300+0000 7fd3675e3640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd360101420 0x7fd3601997c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:23.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.300+0000 7fd3675e3640 1 --2- 172.21.15.5:0/1689028671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd360100a80 0x7fd360199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:23.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.300+0000 7fd3675e3640 1 -- 172.21.15.5:0/1689028671 >> 172.21.15.5:0/1689028671 conn(0x7fd3600fc7a0 msgr2=0x7fd3600fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:23.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.300+0000 7fd3675e3640 1 -- 172.21.15.5:0/1689028671 shutdown_connections 2024-09-19T06:02:23.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:23.300+0000 7fd3675e3640 1 -- 172.21.15.5:0/1689028671 wait complete. 2024-09-19T06:02:23.689 INFO:teuthology.orchestra.run.smithi005.stdout:158913790005 2024-09-19T06:02:23.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T06:02:24.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.297+0000 7fef35ce6640 1 -- 172.21.15.5:0/253466069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef3010c170 msgr2=0x7fef3010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:24.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.297+0000 7fef35ce6640 1 --2- 172.21.15.5:0/253466069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef3010c170 0x7fef3010e610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fef28009ca0 tx=0x7fef2802a950 comp rx=0 tx=0).stop 2024-09-19T06:02:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.297+0000 7fef35ce6640 1 -- 172.21.15.5:0/253466069 shutdown_connections 2024-09-19T06:02:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.297+0000 7fef35ce6640 1 --2- 172.21.15.5:0/253466069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef3010c170 0x7fef3010e610 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.297+0000 7fef35ce6640 1 --2- 172.21.15.5:0/253466069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef30103660 0x7fef3010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.297+0000 7fef35ce6640 1 --2- 172.21.15.5:0/253466069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef30102d40 0x7fef30103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.297+0000 7fef35ce6640 1 -- 172.21.15.5:0/253466069 >> 172.21.15.5:0/253466069 conn(0x7fef300fc7a0 msgr2=0x7fef300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.297+0000 7fef35ce6640 1 -- 172.21.15.5:0/253466069 shutdown_connections 2024-09-19T06:02:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.297+0000 7fef35ce6640 1 -- 172.21.15.5:0/253466069 wait complete. 2024-09-19T06:02:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.298+0000 7fef35ce6640 1 Processor -- start 2024-09-19T06:02:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.298+0000 7fef35ce6640 1 -- start start 2024-09-19T06:02:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.298+0000 7fef35ce6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef30102d40 0x7fef30198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.298+0000 7fef35ce6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef30103660 0x7fef30198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:24.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.298+0000 7fef35ce6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef3010c170 0x7fef3019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:24.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.298+0000 7fef35ce6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef30111e70 con 0x7fef30103660 2024-09-19T06:02:24.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.298+0000 7fef35ce6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef30111cf0 con 0x7fef30102d40 2024-09-19T06:02:24.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.298+0000 7fef35ce6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef30111ff0 con 0x7fef3010c170 2024-09-19T06:02:24.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.298+0000 7fef2effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef30103660 0x7fef30198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:24.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.298+0000 7fef2effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef30103660 0x7fef30198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44480/0 (socket says 172.21.15.5:44480) 2024-09-19T06:02:24.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.298+0000 7fef2effd640 1 -- 172.21.15.5:0/629895172 learned_addr learned my addr 172.21.15.5:0/629895172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:24.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef2ffff640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef3010c170 0x7fef3019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:24.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef2f7fe640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef30102d40 0x7fef30198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:24.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef2effd640 1 -- 172.21.15.5:0/629895172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef3010c170 msgr2=0x7fef3019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:24.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef2effd640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef3010c170 0x7fef3019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:24.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef2effd640 1 -- 172.21.15.5:0/629895172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef30102d40 msgr2=0x7fef30198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:24.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef2effd640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef30102d40 0x7fef30198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:24.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef2effd640 1 -- 172.21.15.5:0/629895172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3019d2c0 con 0x7fef30103660 2024-09-19T06:02:24.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef2f7fe640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef30102d40 0x7fef30198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:24.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef2effd640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef30103660 0x7fef30198850 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fef1800d950 tx=0x7fef1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:24.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef2cff9640 1 -- 172.21.15.5:0/629895172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18017030 con 0x7fef30103660 2024-09-19T06:02:24.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef35ce6640 1 -- 172.21.15.5:0/629895172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3019d5b0 con 0x7fef30103660 2024-09-19T06:02:24.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef2cff9640 1 -- 172.21.15.5:0/629895172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef18004020 con 0x7fef30103660 2024-09-19T06:02:24.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef2cff9640 1 -- 172.21.15.5:0/629895172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180055e0 con 0x7fef30103660 2024-09-19T06:02:24.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.299+0000 7fef35ce6640 1 -- 172.21.15.5:0/629895172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef301a4e90 con 0x7fef30103660 2024-09-19T06:02:24.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.300+0000 7fef35ce6640 1 -- 172.21.15.5:0/629895172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef30103120 con 0x7fef30103660 2024-09-19T06:02:24.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.301+0000 7fef2cff9640 1 -- 172.21.15.5:0/629895172 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fef180041d0 con 0x7fef30103660 2024-09-19T06:02:24.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.303+0000 7fef2cff9640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fef1c0a32d0 0x7fef1c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:24.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.303+0000 7fef2f7fe640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fef1c0a32d0 0x7fef1c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:24.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.303+0000 7fef2cff9640 1 -- 172.21.15.5:0/629895172 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fef180cfa50 con 0x7fef30103660 2024-09-19T06:02:24.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.303+0000 7fef2f7fe640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fef1c0a32d0 0x7fef1c0a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fef20009c20 tx=0x7fef20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:24.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.306+0000 7fef2cff9640 1 -- 172.21.15.5:0/629895172 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fef1809c5a0 con 0x7fef30103660 2024-09-19T06:02:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.486+0000 7fef35ce6640 1 -- 172.21.15.5:0/629895172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fef3002d7c0 con 0x7fef30103660 2024-09-19T06:02:24.489 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:24 smithi005 ceph-mon[30955]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:24.489 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:24 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1689028671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T06:02:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.486+0000 7fef2cff9640 1 -- 172.21.15.5:0/629895172 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fef1809bea0 con 0x7fef30103660 2024-09-19T06:02:24.490 INFO:teuthology.orchestra.run.smithi005.stdout:210453397547 2024-09-19T06:02:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.489+0000 7fef35ce6640 1 -- 172.21.15.5:0/629895172 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fef1c0a32d0 msgr2=0x7fef1c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.489+0000 7fef35ce6640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fef1c0a32d0 0x7fef1c0a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fef20009c20 tx=0x7fef20005f90 comp rx=0 tx=0).stop 2024-09-19T06:02:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.489+0000 7fef35ce6640 1 -- 172.21.15.5:0/629895172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef30103660 msgr2=0x7fef30198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.489+0000 7fef35ce6640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef30103660 0x7fef30198850 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fef1800d950 tx=0x7fef1800de10 comp rx=0 tx=0).stop 2024-09-19T06:02:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.489+0000 7fef35ce6640 1 -- 172.21.15.5:0/629895172 shutdown_connections 2024-09-19T06:02:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.489+0000 7fef35ce6640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fef1c0a32d0 0x7fef1c0a5790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.489+0000 7fef35ce6640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef3010c170 0x7fef3019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.489+0000 7fef35ce6640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef30103660 0x7fef30198850 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.489+0000 7fef35ce6640 1 --2- 172.21.15.5:0/629895172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef30102d40 0x7fef30198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.489+0000 7fef35ce6640 1 -- 172.21.15.5:0/629895172 >> 172.21.15.5:0/629895172 conn(0x7fef300fc7a0 msgr2=0x7fef300fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.489+0000 7fef35ce6640 1 -- 172.21.15.5:0/629895172 shutdown_connections 2024-09-19T06:02:24.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.489+0000 7fef35ce6640 1 -- 172.21.15.5:0/629895172 wait complete. 2024-09-19T06:02:24.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:24 smithi157 ceph-mon[35346]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:24.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:24 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1689028671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T06:02:24.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:24 smithi049 ceph-mon[34640]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:24.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:24 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1689028671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T06:02:25.330 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:25 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:02:25.330 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:25 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/629895172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T06:02:25.330 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:25 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3980411375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T06:02:25.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:25 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:02:25.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:25 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/629895172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T06:02:25.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:25 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3980411375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T06:02:25.493 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:25 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:02:25.493 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:25 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/629895172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T06:02:25.493 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:25 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3980411375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T06:02:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.489+0000 7f3bc54eb640 1 -- 172.21.15.5:0/2605224597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc01036e0 msgr2=0x7f3bc0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.489+0000 7f3bc54eb640 1 --2- 172.21.15.5:0/2605224597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc01036e0 0x7f3bc0107b90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009930 tx=0x7f3ba802a420 comp rx=0 tx=0).stop 2024-09-19T06:02:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.489+0000 7f3bc54eb640 1 -- 172.21.15.5:0/2605224597 shutdown_connections 2024-09-19T06:02:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.489+0000 7f3bc54eb640 1 --2- 172.21.15.5:0/2605224597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bc01082a0 0x7f3bc010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.489+0000 7f3bc54eb640 1 --2- 172.21.15.5:0/2605224597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc01036e0 0x7f3bc0107b90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.489+0000 7f3bc54eb640 1 --2- 172.21.15.5:0/2605224597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc0102d30 0x7f3bc0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.489+0000 7f3bc54eb640 1 -- 172.21.15.5:0/2605224597 >> 172.21.15.5:0/2605224597 conn(0x7f3bc00fc7d0 msgr2=0x7f3bc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.489+0000 7f3bc54eb640 1 -- 172.21.15.5:0/2605224597 shutdown_connections 2024-09-19T06:02:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.489+0000 7f3bc54eb640 1 -- 172.21.15.5:0/2605224597 wait complete. 2024-09-19T06:02:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.490+0000 7f3bc54eb640 1 Processor -- start 2024-09-19T06:02:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.490+0000 7f3bc54eb640 1 -- start start 2024-09-19T06:02:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bc54eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bc0102d30 0x7f3bc0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bc54eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc01036e0 0x7f3bc0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bc54eb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc01082a0 0x7f3bc019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bc54eb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc0111f50 con 0x7f3bc01036e0 2024-09-19T06:02:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bc54eb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc0111dd0 con 0x7f3bc01082a0 2024-09-19T06:02:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bc54eb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc01120d0 con 0x7f3bc0102d30 2024-09-19T06:02:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bbe7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc01036e0 0x7f3bc0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bbe7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc01036e0 0x7f3bc0198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37242/0 (socket says 172.21.15.5:37242) 2024-09-19T06:02:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bbe7fc640 1 -- 172.21.15.5:0/1492398167 learned_addr learned my addr 172.21.15.5:0/1492398167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bbf7fe640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc01082a0 0x7f3bc019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bbeffd640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bc0102d30 0x7f3bc0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bbe7fc640 1 -- 172.21.15.5:0/1492398167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bc0102d30 msgr2=0x7f3bc0198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bbe7fc640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bc0102d30 0x7f3bc0198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bbe7fc640 1 -- 172.21.15.5:0/1492398167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc01082a0 msgr2=0x7f3bc019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bbe7fc640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc01082a0 0x7f3bc019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bbe7fc640 1 -- 172.21.15.5:0/1492398167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc019d4c0 con 0x7f3bc01036e0 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bbeffd640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bc0102d30 0x7f3bc0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bbe7fc640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc01036e0 0x7f3bc0198a50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8031440 tx=0x7f3ba8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.491+0000 7f3bb7fff640 1 -- 172.21.15.5:0/1492398167 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba803f030 con 0x7f3bc01036e0 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.492+0000 7f3bc54eb640 1 -- 172.21.15.5:0/1492398167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc019d750 con 0x7f3bc01036e0 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.492+0000 7f3bb7fff640 1 -- 172.21.15.5:0/1492398167 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ba8003aa0 con 0x7f3bc01036e0 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.492+0000 7f3bb7fff640 1 -- 172.21.15.5:0/1492398167 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8035070 con 0x7f3bc01036e0 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.492+0000 7f3bc54eb640 1 -- 172.21.15.5:0/1492398167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc01a5090 con 0x7f3bc01036e0 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.493+0000 7f3bc54eb640 1 -- 172.21.15.5:0/1492398167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc0103110 con 0x7f3bc01036e0 2024-09-19T06:02:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.494+0000 7f3bb7fff640 1 -- 172.21.15.5:0/1492398167 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f3ba8042030 con 0x7f3bc01036e0 2024-09-19T06:02:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.495+0000 7f3bb7fff640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3bac0a32d0 0x7f3bac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.495+0000 7f3bbeffd640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3bac0a32d0 0x7f3bac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.495+0000 7f3bb7fff640 1 -- 172.21.15.5:0/1492398167 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f3ba80f0660 con 0x7f3bc01036e0 2024-09-19T06:02:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.495+0000 7f3bbeffd640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3bac0a32d0 0x7f3bac0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80058c0 tx=0x7f3bb8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.498+0000 7f3bb7fff640 1 -- 172.21.15.5:0/1492398167 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3ba80bd1b0 con 0x7f3bc01036e0 2024-09-19T06:02:25.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.681+0000 7f3bc54eb640 1 -- 172.21.15.5:0/1492398167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3bc010a2b0 con 0x7f3bc01036e0 2024-09-19T06:02:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.682+0000 7f3bb7fff640 1 -- 172.21.15.5:0/1492398167 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3ba80bcab0 con 0x7f3bc01036e0 2024-09-19T06:02:25.685 INFO:teuthology.orchestra.run.smithi005.stdout:107374182463 2024-09-19T06:02:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.684+0000 7f3bc54eb640 1 -- 172.21.15.5:0/1492398167 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3bac0a32d0 msgr2=0x7f3bac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.684+0000 7f3bc54eb640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3bac0a32d0 0x7f3bac0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80058c0 tx=0x7f3bb8005850 comp rx=0 tx=0).stop 2024-09-19T06:02:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.684+0000 7f3bc54eb640 1 -- 172.21.15.5:0/1492398167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc01036e0 msgr2=0x7f3bc0198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.684+0000 7f3bc54eb640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc01036e0 0x7f3bc0198a50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8031440 tx=0x7f3ba8004270 comp rx=0 tx=0).stop 2024-09-19T06:02:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.684+0000 7f3bc54eb640 1 -- 172.21.15.5:0/1492398167 shutdown_connections 2024-09-19T06:02:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.684+0000 7f3bc54eb640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f3bac0a32d0 0x7f3bac0a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.684+0000 7f3bc54eb640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc01082a0 0x7f3bc019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.684+0000 7f3bc54eb640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc01036e0 0x7f3bc0198a50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.684+0000 7f3bc54eb640 1 --2- 172.21.15.5:0/1492398167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bc0102d30 0x7f3bc0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:25.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.684+0000 7f3bc54eb640 1 -- 172.21.15.5:0/1492398167 >> 172.21.15.5:0/1492398167 conn(0x7f3bc00fc7d0 msgr2=0x7f3bc0105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:25.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.684+0000 7f3bc54eb640 1 -- 172.21.15.5:0/1492398167 shutdown_connections 2024-09-19T06:02:25.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:25.684+0000 7f3bc54eb640 1 -- 172.21.15.5:0/1492398167 wait complete. 2024-09-19T06:02:25.993 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397546 got 210453397547 for osd.7 2024-09-19T06:02:25.993 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:26.272 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776154 got 292057776157 for osd.10 2024-09-19T06:02:26.273 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:26 smithi157 ceph-mon[35346]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-19T06:02:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:26 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1492398167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T06:02:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:26 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:26.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:26 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:26.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:26 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:26.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:26 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:26.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:26 smithi005 ceph-mon[30955]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-19T06:02:26.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:26 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1492398167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T06:02:26.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:26 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:26.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:26 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:26.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:26 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:26.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:26 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:26 smithi049 ceph-mon[34640]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-19T06:02:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:26 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1492398167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T06:02:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:26 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:26 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:26 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:26 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.700+0000 7fa2942ea640 1 -- 172.21.15.5:0/345396372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c101420 msgr2=0x7fa28c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.700+0000 7fa2942ea640 1 --2- 172.21.15.5:0/345396372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c101420 0x7fa28c10d950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa284009cd0 tx=0x7fa28402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.701+0000 7fa2942ea640 1 -- 172.21.15.5:0/345396372 shutdown_connections 2024-09-19T06:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.701+0000 7fa2942ea640 1 --2- 172.21.15.5:0/345396372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c101420 0x7fa28c10d950 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.701+0000 7fa2942ea640 1 --2- 172.21.15.5:0/345396372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c100a80 0x7fa28c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.701+0000 7fa2942ea640 1 --2- 172.21.15.5:0/345396372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c106a80 0x7fa28c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.701+0000 7fa2942ea640 1 -- 172.21.15.5:0/345396372 >> 172.21.15.5:0/345396372 conn(0x7fa28c0fc7a0 msgr2=0x7fa28c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.701+0000 7fa2942ea640 1 -- 172.21.15.5:0/345396372 shutdown_connections 2024-09-19T06:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.701+0000 7fa2942ea640 1 -- 172.21.15.5:0/345396372 wait complete. 2024-09-19T06:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.701+0000 7fa2942ea640 1 Processor -- start 2024-09-19T06:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.701+0000 7fa2942ea640 1 -- start start 2024-09-19T06:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa2942ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c100a80 0x7fa28c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa2942ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c101420 0x7fa28c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa2942ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c106a80 0x7fa28c1a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa2942ea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c110d30 con 0x7fa28c101420 2024-09-19T06:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa2942ea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c110bb0 con 0x7fa28c100a80 2024-09-19T06:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa2942ea640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c110eb0 con 0x7fa28c106a80 2024-09-19T06:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa291885640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c101420 0x7fa28c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa291885640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c101420 0x7fa28c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37248/0 (socket says 172.21.15.5:37248) 2024-09-19T06:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa292086640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c100a80 0x7fa28c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa291885640 1 -- 172.21.15.5:0/942412267 learned_addr learned my addr 172.21.15.5:0/942412267 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa292887640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c106a80 0x7fa28c1a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa291885640 1 -- 172.21.15.5:0/942412267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c106a80 msgr2=0x7fa28c1a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa291885640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c106a80 0x7fa28c1a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa291885640 1 -- 172.21.15.5:0/942412267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c100a80 msgr2=0x7fa28c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa291885640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c100a80 0x7fa28c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa291885640 1 -- 172.21.15.5:0/942412267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c1a4780 con 0x7fa28c101420 2024-09-19T06:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa292887640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c106a80 0x7fa28c1a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa291885640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c101420 0x7fa28c199700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00d950 tx=0x7fa27c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa2837fe640 1 -- 172.21.15.5:0/942412267 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c017070 con 0x7fa28c101420 2024-09-19T06:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa2837fe640 1 -- 172.21.15.5:0/942412267 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa27c009c30 con 0x7fa28c101420 2024-09-19T06:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa2942ea640 1 -- 172.21.15.5:0/942412267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28c1a4950 con 0x7fa28c101420 2024-09-19T06:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.702+0000 7fa2837fe640 1 -- 172.21.15.5:0/942412267 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00f620 con 0x7fa28c101420 2024-09-19T06:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.703+0000 7fa2942ea640 1 -- 172.21.15.5:0/942412267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28c1a4e60 con 0x7fa28c101420 2024-09-19T06:02:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.703+0000 7fa2942ea640 1 -- 172.21.15.5:0/942412267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28c100ee0 con 0x7fa28c101420 2024-09-19T06:02:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.705+0000 7fa2837fe640 1 -- 172.21.15.5:0/942412267 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fa27c0109a0 con 0x7fa28c101420 2024-09-19T06:02:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.705+0000 7fa2837fe640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa2780a32d0 0x7fa2780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.706+0000 7fa292086640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa2780a32d0 0x7fa2780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.706+0000 7fa2837fe640 1 -- 172.21.15.5:0/942412267 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa27c0cf6c0 con 0x7fa28c101420 2024-09-19T06:02:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.706+0000 7fa292086640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa2780a32d0 0x7fa2780a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa288005fd0 tx=0x7fa28800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.709+0000 7fa2837fe640 1 -- 172.21.15.5:0/942412267 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa27c009de0 con 0x7fa28c101420 2024-09-19T06:02:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.885+0000 7fa2942ea640 1 -- 172.21.15.5:0/942412267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa28c108ff0 con 0x7fa28c101420 2024-09-19T06:02:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.885+0000 7fa2837fe640 1 -- 172.21.15.5:0/942412267 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa27c09c270 con 0x7fa28c101420 2024-09-19T06:02:26.888 INFO:teuthology.orchestra.run.smithi005.stdout:77309411398 2024-09-19T06:02:26.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.887+0000 7fa2942ea640 1 -- 172.21.15.5:0/942412267 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa2780a32d0 msgr2=0x7fa2780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:26.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.887+0000 7fa2942ea640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa2780a32d0 0x7fa2780a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa288005fd0 tx=0x7fa28800b390 comp rx=0 tx=0).stop 2024-09-19T06:02:26.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.887+0000 7fa2942ea640 1 -- 172.21.15.5:0/942412267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c101420 msgr2=0x7fa28c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:26.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.887+0000 7fa2942ea640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c101420 0x7fa28c199700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00d950 tx=0x7fa27c00de10 comp rx=0 tx=0).stop 2024-09-19T06:02:26.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.887+0000 7fa2942ea640 1 -- 172.21.15.5:0/942412267 shutdown_connections 2024-09-19T06:02:26.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.887+0000 7fa2942ea640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa2780a32d0 0x7fa2780a5790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:26.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.887+0000 7fa2942ea640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c106a80 0x7fa28c1a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:26.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.887+0000 7fa2942ea640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c101420 0x7fa28c199700 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:26.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.887+0000 7fa2942ea640 1 --2- 172.21.15.5:0/942412267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c100a80 0x7fa28c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:26.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.887+0000 7fa2942ea640 1 -- 172.21.15.5:0/942412267 >> 172.21.15.5:0/942412267 conn(0x7fa28c0fc7a0 msgr2=0x7fa28c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:26.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.887+0000 7fa2942ea640 1 -- 172.21.15.5:0/942412267 shutdown_connections 2024-09-19T06:02:26.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:26.887+0000 7fa2942ea640 1 -- 172.21.15.5:0/942412267 wait complete. 2024-09-19T06:02:27.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:27 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/942412267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T06:02:27.516 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182460 got 107374182463 for osd.3 2024-09-19T06:02:27.516 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:27.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:27 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/942412267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T06:02:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.549+0000 7fb619453640 1 -- 172.21.15.5:0/4079626893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb614100310 msgr2=0x7fb6141006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.549+0000 7fb619453640 1 --2- 172.21.15.5:0/4079626893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb614100310 0x7fb6141006f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb604009900 tx=0x7fb60402a410 comp rx=0 tx=0).stop 2024-09-19T06:02:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.549+0000 7fb619453640 1 -- 172.21.15.5:0/4079626893 shutdown_connections 2024-09-19T06:02:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.549+0000 7fb619453640 1 --2- 172.21.15.5:0/4079626893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614106f30 0x7fb61410eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.549+0000 7fb619453640 1 --2- 172.21.15.5:0/4079626893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb614100cc0 0x7fb614106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.549+0000 7fb619453640 1 --2- 172.21.15.5:0/4079626893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb614100310 0x7fb6141006f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.549+0000 7fb619453640 1 -- 172.21.15.5:0/4079626893 >> 172.21.15.5:0/4079626893 conn(0x7fb614077210 msgr2=0x7fb6140fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.550+0000 7fb619453640 1 -- 172.21.15.5:0/4079626893 shutdown_connections 2024-09-19T06:02:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.550+0000 7fb619453640 1 -- 172.21.15.5:0/4079626893 wait complete. 2024-09-19T06:02:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.550+0000 7fb619453640 1 Processor -- start 2024-09-19T06:02:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.550+0000 7fb619453640 1 -- start start 2024-09-19T06:02:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.550+0000 7fb619453640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb614100310 0x7fb614101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.550+0000 7fb619453640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614100cc0 0x7fb6141017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb619453640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb614106f30 0x7fb614101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb619453640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb614111e40 con 0x7fb614106f30 2024-09-19T06:02:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb619453640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb614111cc0 con 0x7fb614100310 2024-09-19T06:02:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb619453640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb614111fc0 con 0x7fb614100cc0 2024-09-19T06:02:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb6137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb614106f30 0x7fb614101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb6137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb614106f30 0x7fb614101d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37236/0 (socket says 172.21.15.5:37236) 2024-09-19T06:02:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb612ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb614100310 0x7fb614101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb6137fe640 1 -- 172.21.15.5:0/3980411375 learned_addr learned my addr 172.21.15.5:0/3980411375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb6127fc640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614100cc0 0x7fb6141017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb6137fe640 1 -- 172.21.15.5:0/3980411375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614100cc0 msgr2=0x7fb6141017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb6137fe640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614100cc0 0x7fb6141017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb6137fe640 1 -- 172.21.15.5:0/3980411375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb614100310 msgr2=0x7fb614101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb6137fe640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb614100310 0x7fb614101290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb6137fe640 1 -- 172.21.15.5:0/3980411375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6141a6860 con 0x7fb614106f30 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb612ffd640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb614100310 0x7fb614101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb6137fe640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb614106f30 0x7fb614101d10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00eb10 tx=0x7fb60c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb5fbfff640 1 -- 172.21.15.5:0/3980411375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c004b20 con 0x7fb614106f30 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb619453640 1 -- 172.21.15.5:0/3980411375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6141a6b50 con 0x7fb614106f30 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb5fbfff640 1 -- 172.21.15.5:0/3980411375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb60c004cc0 con 0x7fb614106f30 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.551+0000 7fb5fbfff640 1 -- 172.21.15.5:0/3980411375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c010690 con 0x7fb614106f30 2024-09-19T06:02:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.552+0000 7fb619453640 1 -- 172.21.15.5:0/3980411375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6141a7040 con 0x7fb614106f30 2024-09-19T06:02:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.553+0000 7fb619453640 1 -- 172.21.15.5:0/3980411375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6141006f0 con 0x7fb614106f30 2024-09-19T06:02:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.554+0000 7fb5fbfff640 1 -- 172.21.15.5:0/3980411375 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fb60c005020 con 0x7fb614106f30 2024-09-19T06:02:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.554+0000 7fb5fbfff640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb5fc0a32d0 0x7fb5fc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.555+0000 7fb612ffd640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb5fc0a32d0 0x7fb5fc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.555+0000 7fb5fbfff640 1 -- 172.21.15.5:0/3980411375 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb60c0d0020 con 0x7fb614106f30 2024-09-19T06:02:27.596 INFO:teuthology.orchestra.run.smithi005.stdout:240518168615 2024-09-19T06:02:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.555+0000 7fb612ffd640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb5fc0a32d0 0x7fb5fc0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb604031440 tx=0x7fb604005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.559+0000 7fb5fbfff640 1 -- 172.21.15.5:0/3980411375 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb60c09cb70 con 0x7fb614106f30 2024-09-19T06:02:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.757+0000 7fb619453640 1 -- 172.21.15.5:0/3980411375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb61410a110 con 0x7fb614106f30 2024-09-19T06:02:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.757+0000 7fb5fbfff640 1 -- 172.21.15.5:0/3980411375 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb60c09c470 con 0x7fb614106f30 2024-09-19T06:02:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.759+0000 7fb619453640 1 -- 172.21.15.5:0/3980411375 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb5fc0a32d0 msgr2=0x7fb5fc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.759+0000 7fb619453640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb5fc0a32d0 0x7fb5fc0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb604031440 tx=0x7fb604005b50 comp rx=0 tx=0).stop 2024-09-19T06:02:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.759+0000 7fb619453640 1 -- 172.21.15.5:0/3980411375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb614106f30 msgr2=0x7fb614101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.759+0000 7fb619453640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb614106f30 0x7fb614101d10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00eb10 tx=0x7fb60c00c2d0 comp rx=0 tx=0).stop 2024-09-19T06:02:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.760+0000 7fb619453640 1 -- 172.21.15.5:0/3980411375 shutdown_connections 2024-09-19T06:02:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.760+0000 7fb619453640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb5fc0a32d0 0x7fb5fc0a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.760+0000 7fb619453640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb614106f30 0x7fb614101d10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.760+0000 7fb619453640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614100cc0 0x7fb6141017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.760+0000 7fb619453640 1 --2- 172.21.15.5:0/3980411375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb614100310 0x7fb614101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:27.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.760+0000 7fb619453640 1 -- 172.21.15.5:0/3980411375 >> 172.21.15.5:0/3980411375 conn(0x7fb614077210 msgr2=0x7fb6140fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:27.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.760+0000 7fb619453640 1 -- 172.21.15.5:0/3980411375 shutdown_connections 2024-09-19T06:02:27.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:24.760+0000 7fb619453640 1 -- 172.21.15.5:0/3980411375 wait complete. 2024-09-19T06:02:27.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:27 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/942412267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T06:02:28.183 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168613 got 240518168615 for osd.8 2024-09-19T06:02:28.183 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:28.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:28 smithi157 ceph-mon[35346]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:28.592 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:28 smithi005 ceph-mon[30955]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:28.627 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411398 for osd.2 2024-09-19T06:02:28.627 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:28.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:28 smithi049 ceph-mon[34640]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.900+0000 7fbbe35e4640 1 -- 172.21.15.5:0/807998964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbdc102d40 msgr2=0x7fbbdc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.900+0000 7fbbe35e4640 1 --2- 172.21.15.5:0/807998964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbdc102d40 0x7fbbdc103120 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009ca0 tx=0x7fbbd802a950 comp rx=0 tx=0).stop 2024-09-19T06:02:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.900+0000 7fbbe35e4640 1 -- 172.21.15.5:0/807998964 shutdown_connections 2024-09-19T06:02:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.900+0000 7fbbe35e4640 1 --2- 172.21.15.5:0/807998964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbdc10c170 0x7fbbdc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.900+0000 7fbbe35e4640 1 --2- 172.21.15.5:0/807998964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbdc103660 0x7fbbdc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.900+0000 7fbbe35e4640 1 --2- 172.21.15.5:0/807998964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbdc102d40 0x7fbbdc103120 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.900+0000 7fbbe35e4640 1 -- 172.21.15.5:0/807998964 >> 172.21.15.5:0/807998964 conn(0x7fbbdc0fc7a0 msgr2=0x7fbbdc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.900+0000 7fbbe35e4640 1 -- 172.21.15.5:0/807998964 shutdown_connections 2024-09-19T06:02:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.900+0000 7fbbe35e4640 1 -- 172.21.15.5:0/807998964 wait complete. 2024-09-19T06:02:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.900+0000 7fbbe35e4640 1 Processor -- start 2024-09-19T06:02:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.900+0000 7fbbe35e4640 1 -- start start 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe35e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbdc102d40 0x7fbbdc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe35e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbdc103660 0x7fbbdc198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe35e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbdc10c170 0x7fbbdc19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe35e4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc111e70 con 0x7fbbdc103660 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe35e4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc111cf0 con 0x7fbbdc102d40 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe35e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc111ff0 con 0x7fbbdc10c170 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe0b7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbdc103660 0x7fbbdc198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe0b7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbdc103660 0x7fbbdc198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37266/0 (socket says 172.21.15.5:37266) 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe0b7f640 1 -- 172.21.15.5:0/1607276510 learned_addr learned my addr 172.21.15.5:0/1607276510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe1380640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbdc102d40 0x7fbbdc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe1b81640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbdc10c170 0x7fbbdc19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe0b7f640 1 -- 172.21.15.5:0/1607276510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbdc10c170 msgr2=0x7fbbdc19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe0b7f640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbdc10c170 0x7fbbdc19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe0b7f640 1 -- 172.21.15.5:0/1607276510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbdc102d40 msgr2=0x7fbbdc198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe0b7f640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbdc102d40 0x7fbbdc198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe0b7f640 1 -- 172.21.15.5:0/1607276510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc19d300 con 0x7fbbdc103660 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe1380640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbdc102d40 0x7fbbdc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.901+0000 7fbbe0b7f640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbdc103660 0x7fbbdc198890 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00d950 tx=0x7fbbcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.902+0000 7fbbd27fc640 1 -- 172.21.15.5:0/1607276510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc017030 con 0x7fbbdc103660 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.902+0000 7fbbd27fc640 1 -- 172.21.15.5:0/1607276510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbcc004020 con 0x7fbbdc103660 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.902+0000 7fbbe35e4640 1 -- 172.21.15.5:0/1607276510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbdc19d5f0 con 0x7fbbdc103660 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.902+0000 7fbbd27fc640 1 -- 172.21.15.5:0/1607276510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc0055e0 con 0x7fbbdc103660 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.902+0000 7fbbe35e4640 1 -- 172.21.15.5:0/1607276510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbdc1a4ed0 con 0x7fbbdc103660 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.903+0000 7fbbe35e4640 1 -- 172.21.15.5:0/1607276510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbdc103120 con 0x7fbbdc103660 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.904+0000 7fbbd27fc640 1 -- 172.21.15.5:0/1607276510 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fbbcc0041d0 con 0x7fbbdc103660 2024-09-19T06:02:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.905+0000 7fbbd27fc640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbbc80a3350 0x7fbbc80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.905+0000 7fbbe1380640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbbc80a3350 0x7fbbc80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.905+0000 7fbbd27fc640 1 -- 172.21.15.5:0/1607276510 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fbbcc0cfa50 con 0x7fbbdc103660 2024-09-19T06:02:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.905+0000 7fbbe1380640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbbc80a3350 0x7fbbc80a5810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8005e30 tx=0x7fbbd8005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:28.908+0000 7fbbd27fc640 1 -- 172.21.15.5:0/1607276510 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbbcc09c620 con 0x7fbbdc103660 2024-09-19T06:02:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.095+0000 7fbbe35e4640 1 -- 172.21.15.5:0/1607276510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbbdc1996b0 con 0x7fbbdc103660 2024-09-19T06:02:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.095+0000 7fbbd27fc640 1 -- 172.21.15.5:0/1607276510 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbbdc1996b0 con 0x7fbbdc103660 2024-09-19T06:02:29.097 INFO:teuthology.orchestra.run.smithi005.stdout:188978561072 2024-09-19T06:02:29.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.097+0000 7fbbe35e4640 1 -- 172.21.15.5:0/1607276510 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbbc80a3350 msgr2=0x7fbbc80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:29.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.097+0000 7fbbe35e4640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbbc80a3350 0x7fbbc80a5810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8005e30 tx=0x7fbbd8005d80 comp rx=0 tx=0).stop 2024-09-19T06:02:29.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.097+0000 7fbbe35e4640 1 -- 172.21.15.5:0/1607276510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbdc103660 msgr2=0x7fbbdc198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:29.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.097+0000 7fbbe35e4640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbdc103660 0x7fbbdc198890 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00d950 tx=0x7fbbcc00de10 comp rx=0 tx=0).stop 2024-09-19T06:02:29.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.097+0000 7fbbe35e4640 1 -- 172.21.15.5:0/1607276510 shutdown_connections 2024-09-19T06:02:29.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.097+0000 7fbbe35e4640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbbc80a3350 0x7fbbc80a5810 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:29.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.097+0000 7fbbe35e4640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbdc10c170 0x7fbbdc19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:29.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.097+0000 7fbbe35e4640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbdc103660 0x7fbbdc198890 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:29.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.097+0000 7fbbe35e4640 1 --2- 172.21.15.5:0/1607276510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbdc102d40 0x7fbbdc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:29.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.097+0000 7fbbe35e4640 1 -- 172.21.15.5:0/1607276510 >> 172.21.15.5:0/1607276510 conn(0x7fbbdc0fc7a0 msgr2=0x7fbbdc0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:29.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.097+0000 7fbbe35e4640 1 -- 172.21.15.5:0/1607276510 shutdown_connections 2024-09-19T06:02:29.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:29.097+0000 7fbbe35e4640 1 -- 172.21.15.5:0/1607276510 wait complete. 2024-09-19T06:02:29.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:29 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1607276510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T06:02:29.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:29 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1607276510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T06:02:29.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:29 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1607276510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T06:02:29.883 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561069 got 188978561072 for osd.6 2024-09-19T06:02:29.883 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.063+0000 7f0658e67640 1 -- 172.21.15.5:0/1499923074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0654101de0 msgr2=0x7f06541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.063+0000 7f0658e67640 1 --2- 172.21.15.5:0/1499923074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0654101de0 0x7f06541021c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0644009900 tx=0x7f064402a410 comp rx=0 tx=0).stop 2024-09-19T06:02:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.063+0000 7f0658e67640 1 -- 172.21.15.5:0/1499923074 shutdown_connections 2024-09-19T06:02:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.063+0000 7f0658e67640 1 --2- 172.21.15.5:0/1499923074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0654106f10 0x7f065410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.063+0000 7f0658e67640 1 --2- 172.21.15.5:0/1499923074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0654102790 0x7f06541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.063+0000 7f0658e67640 1 --2- 172.21.15.5:0/1499923074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0654101de0 0x7f06541021c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.063+0000 7f0658e67640 1 -- 172.21.15.5:0/1499923074 >> 172.21.15.5:0/1499923074 conn(0x7f06540fd8c0 msgr2=0x7f06540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.064+0000 7f0658e67640 1 -- 172.21.15.5:0/1499923074 shutdown_connections 2024-09-19T06:02:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.064+0000 7f0658e67640 1 -- 172.21.15.5:0/1499923074 wait complete. 2024-09-19T06:02:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.064+0000 7f0658e67640 1 Processor -- start 2024-09-19T06:02:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.064+0000 7f0658e67640 1 -- start start 2024-09-19T06:02:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f0658e67640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0654101de0 0x7f065419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f0658e67640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0654102790 0x7f065419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f0658e67640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0654106f10 0x7f065419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f0658e67640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0654111de0 con 0x7f0654101de0 2024-09-19T06:02:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f0658e67640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0654111c60 con 0x7f0654106f10 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f0658e67640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0654111f60 con 0x7f0654102790 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f065259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0654101de0 0x7f065419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f065259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0654101de0 0x7f065419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37284/0 (socket says 172.21.15.5:37284) 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f065259c640 1 -- 172.21.15.5:0/3006937074 learned_addr learned my addr 172.21.15.5:0/3006937074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f0652d9d640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0654106f10 0x7f065419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f0651d9b640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0654102790 0x7f065419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f065259c640 1 -- 172.21.15.5:0/3006937074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0654102790 msgr2=0x7f065419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f065259c640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0654102790 0x7f065419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f065259c640 1 -- 172.21.15.5:0/3006937074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0654106f10 msgr2=0x7f065419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f065259c640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0654106f10 0x7f065419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f065259c640 1 -- 172.21.15.5:0/3006937074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065419f590 con 0x7f0654101de0 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f0651d9b640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0654102790 0x7f065419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f065259c640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0654101de0 0x7f065419a5e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0644005940 tx=0x7f0644004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.065+0000 7f06437fe640 1 -- 172.21.15.5:0/3006937074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064403f030 con 0x7f0654101de0 2024-09-19T06:02:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.066+0000 7f06437fe640 1 -- 172.21.15.5:0/3006937074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0644003aa0 con 0x7f0654101de0 2024-09-19T06:02:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.066+0000 7f0658e67640 1 -- 172.21.15.5:0/3006937074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065419f820 con 0x7f0654101de0 2024-09-19T06:02:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.066+0000 7f06437fe640 1 -- 172.21.15.5:0/3006937074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644035070 con 0x7f0654101de0 2024-09-19T06:02:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.066+0000 7f0658e67640 1 -- 172.21.15.5:0/3006937074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06541a7160 con 0x7f0654101de0 2024-09-19T06:02:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.067+0000 7f0658e67640 1 -- 172.21.15.5:0/3006937074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06541032c0 con 0x7f0654101de0 2024-09-19T06:02:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.068+0000 7f06437fe640 1 -- 172.21.15.5:0/3006937074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f0644042030 con 0x7f0654101de0 2024-09-19T06:02:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.069+0000 7f06437fe640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f063c0a32d0 0x7f063c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.069+0000 7f0651d9b640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f063c0a32d0 0x7f063c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.069+0000 7f06437fe640 1 -- 172.21.15.5:0/3006937074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f06440f0520 con 0x7f0654101de0 2024-09-19T06:02:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.069+0000 7f0651d9b640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f063c0a32d0 0x7f063c0a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0654100370 tx=0x7f0648005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.072+0000 7f06437fe640 1 -- 172.21.15.5:0/3006937074 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f06440bd070 con 0x7f0654101de0 2024-09-19T06:02:30.135 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.260+0000 7f0658e67640 1 -- 172.21.15.5:0/3006937074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f06541a7330 con 0x7f0654101de0 2024-09-19T06:02:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.260+0000 7f06437fe640 1 -- 172.21.15.5:0/3006937074 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f06440bc970 con 0x7f0654101de0 2024-09-19T06:02:30.262 INFO:teuthology.orchestra.run.smithi005.stdout:270582939683 2024-09-19T06:02:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.262+0000 7f0658e67640 1 -- 172.21.15.5:0/3006937074 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f063c0a32d0 msgr2=0x7f063c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.262+0000 7f0658e67640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f063c0a32d0 0x7f063c0a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0654100370 tx=0x7f0648005e50 comp rx=0 tx=0).stop 2024-09-19T06:02:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.262+0000 7f0658e67640 1 -- 172.21.15.5:0/3006937074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0654101de0 msgr2=0x7f065419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.262+0000 7f0658e67640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0654101de0 0x7f065419a5e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0644005940 tx=0x7f0644004270 comp rx=0 tx=0).stop 2024-09-19T06:02:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.262+0000 7f0658e67640 1 -- 172.21.15.5:0/3006937074 shutdown_connections 2024-09-19T06:02:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.262+0000 7f0658e67640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f063c0a32d0 0x7f063c0a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.262+0000 7f0658e67640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0654106f10 0x7f065419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.262+0000 7f0658e67640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0654102790 0x7f065419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.262+0000 7f0658e67640 1 --2- 172.21.15.5:0/3006937074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0654101de0 0x7f065419a5e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.262+0000 7f0658e67640 1 -- 172.21.15.5:0/3006937074 >> 172.21.15.5:0/3006937074 conn(0x7f06540fd8c0 msgr2=0x7f06540fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.262+0000 7f0658e67640 1 -- 172.21.15.5:0/3006937074 shutdown_connections 2024-09-19T06:02:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:30.263+0000 7f0658e67640 1 -- 172.21.15.5:0/3006937074 wait complete. 2024-09-19T06:02:30.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:30 smithi157 ceph-mon[35346]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:30.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:30 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3006937074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T06:02:30.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:30 smithi005 ceph-mon[30955]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:30.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:30 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3006937074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T06:02:30.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:30 smithi049 ceph-mon[34640]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:30.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:30 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3006937074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T06:02:31.048 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939680 got 270582939683 for osd.9 2024-09-19T06:02:31.049 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.271+0000 7fb35e765640 1 -- 172.21.15.5:0/2640278839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3581082a0 msgr2=0x7fb35810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.271+0000 7fb35e765640 1 --2- 172.21.15.5:0/2640278839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3581082a0 0x7fb35810ec40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb350009cd0 tx=0x7fb35002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:02:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.271+0000 7fb35e765640 1 -- 172.21.15.5:0/2640278839 shutdown_connections 2024-09-19T06:02:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.271+0000 7fb35e765640 1 --2- 172.21.15.5:0/2640278839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3581082a0 0x7fb35810ec40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.271+0000 7fb35e765640 1 --2- 172.21.15.5:0/2640278839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3581036e0 0x7fb358107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.271+0000 7fb35e765640 1 --2- 172.21.15.5:0/2640278839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb358102d30 0x7fb358103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.271+0000 7fb35e765640 1 -- 172.21.15.5:0/2640278839 >> 172.21.15.5:0/2640278839 conn(0x7fb3580fc7d0 msgr2=0x7fb3580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.271+0000 7fb35e765640 1 -- 172.21.15.5:0/2640278839 shutdown_connections 2024-09-19T06:02:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.271+0000 7fb35e765640 1 -- 172.21.15.5:0/2640278839 wait complete. 2024-09-19T06:02:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.271+0000 7fb35e765640 1 Processor -- start 2024-09-19T06:02:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.271+0000 7fb35e765640 1 -- start start 2024-09-19T06:02:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb35e765640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb358102d30 0x7fb3581983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb35e765640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3581036e0 0x7fb358198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb35e765640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3581082a0 0x7fb35819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb35e765640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb358111f50 con 0x7fb3581036e0 2024-09-19T06:02:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb35e765640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb358111dd0 con 0x7fb358102d30 2024-09-19T06:02:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb35e765640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581120d0 con 0x7fb3581082a0 2024-09-19T06:02:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb3577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3581036e0 0x7fb358198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb3577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3581036e0 0x7fb358198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37304/0 (socket says 172.21.15.5:37304) 2024-09-19T06:02:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb3577fe640 1 -- 172.21.15.5:0/1203680257 learned_addr learned my addr 172.21.15.5:0/1203680257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb35cd02640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3581082a0 0x7fb35819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb357fff640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb358102d30 0x7fb3581983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb3577fe640 1 -- 172.21.15.5:0/1203680257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3581082a0 msgr2=0x7fb35819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb3577fe640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3581082a0 0x7fb35819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb3577fe640 1 -- 172.21.15.5:0/1203680257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb358102d30 msgr2=0x7fb3581983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.272+0000 7fb3577fe640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb358102d30 0x7fb3581983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.273+0000 7fb3577fe640 1 -- 172.21.15.5:0/1203680257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35819d370 con 0x7fb3581036e0 2024-09-19T06:02:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.273+0000 7fb357fff640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb358102d30 0x7fb3581983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.273+0000 7fb3577fe640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3581036e0 0x7fb358198900 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00d950 tx=0x7fb34c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.273+0000 7fb3557fa640 1 -- 172.21.15.5:0/1203680257 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c017070 con 0x7fb3581036e0 2024-09-19T06:02:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.273+0000 7fb35e765640 1 -- 172.21.15.5:0/1203680257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35819d660 con 0x7fb3581036e0 2024-09-19T06:02:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.273+0000 7fb3557fa640 1 -- 172.21.15.5:0/1203680257 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb34c009c30 con 0x7fb3581036e0 2024-09-19T06:02:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.273+0000 7fb3557fa640 1 -- 172.21.15.5:0/1203680257 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0047a0 con 0x7fb3581036e0 2024-09-19T06:02:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.273+0000 7fb35e765640 1 -- 172.21.15.5:0/1203680257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3581a4f40 con 0x7fb3581036e0 2024-09-19T06:02:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.274+0000 7fb35e765640 1 -- 172.21.15.5:0/1203680257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb358103110 con 0x7fb3581036e0 2024-09-19T06:02:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.275+0000 7fb3557fa640 1 -- 172.21.15.5:0/1203680257 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fb34c004020 con 0x7fb3581036e0 2024-09-19T06:02:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.276+0000 7fb3557fa640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb3400a3350 0x7fb3400a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.276+0000 7fb357fff640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb3400a3350 0x7fb3400a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.276+0000 7fb3557fa640 1 -- 172.21.15.5:0/1203680257 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb34c0cf6a0 con 0x7fb3581036e0 2024-09-19T06:02:31.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.277+0000 7fb357fff640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb3400a3350 0x7fb3400a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb34800abe0 tx=0x7fb34800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.280+0000 7fb3557fa640 1 -- 172.21.15.5:0/1203680257 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb34c020020 con 0x7fb3581036e0 2024-09-19T06:02:31.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.461+0000 7fb35e765640 1 -- 172.21.15.5:0/1203680257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb35810a2b0 con 0x7fb3581036e0 2024-09-19T06:02:31.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.462+0000 7fb3557fa640 1 -- 172.21.15.5:0/1203680257 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb34c09c1f0 con 0x7fb3581036e0 2024-09-19T06:02:31.464 INFO:teuthology.orchestra.run.smithi005.stdout:158913790007 2024-09-19T06:02:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.464+0000 7fb35e765640 1 -- 172.21.15.5:0/1203680257 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb3400a3350 msgr2=0x7fb3400a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.464+0000 7fb35e765640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb3400a3350 0x7fb3400a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb34800abe0 tx=0x7fb34800a5c0 comp rx=0 tx=0).stop 2024-09-19T06:02:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.464+0000 7fb35e765640 1 -- 172.21.15.5:0/1203680257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3581036e0 msgr2=0x7fb358198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.464+0000 7fb35e765640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3581036e0 0x7fb358198900 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00d950 tx=0x7fb34c00de10 comp rx=0 tx=0).stop 2024-09-19T06:02:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.464+0000 7fb35e765640 1 -- 172.21.15.5:0/1203680257 shutdown_connections 2024-09-19T06:02:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.464+0000 7fb35e765640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb3400a3350 0x7fb3400a5810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.464+0000 7fb35e765640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3581082a0 0x7fb35819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.464+0000 7fb35e765640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3581036e0 0x7fb358198900 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.464+0000 7fb35e765640 1 --2- 172.21.15.5:0/1203680257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb358102d30 0x7fb3581983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.464+0000 7fb35e765640 1 -- 172.21.15.5:0/1203680257 >> 172.21.15.5:0/1203680257 conn(0x7fb3580fc7d0 msgr2=0x7fb358105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:31.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.464+0000 7fb35e765640 1 -- 172.21.15.5:0/1203680257 shutdown_connections 2024-09-19T06:02:31.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:31.464+0000 7fb35e765640 1 -- 172.21.15.5:0/1203680257 wait complete. 2024-09-19T06:02:32.197 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790005 got 158913790007 for osd.5 2024-09-19T06:02:32.197 DEBUG:teuthology.parallel:result is None 2024-09-19T06:02:32.197 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T06:02:32.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:02:32.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:32 smithi005 ceph-mon[30955]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:32.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:32 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1203680257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T06:02:32.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:32 smithi157 ceph-mon[35346]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:32.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:32 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1203680257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T06:02:32.644 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:32.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:32 smithi049 ceph-mon[34640]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:32.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:32 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1203680257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T06:02:33.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.084+0000 7ff61d09c640 1 -- 172.21.15.5:0/3375918468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff618100e20 msgr2=0x7ff618101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:33.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.084+0000 7ff61d09c640 1 --2- 172.21.15.5:0/3375918468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff618100e20 0x7ff618101200 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff608009ca0 tx=0x7ff60802a950 comp rx=0 tx=0).stop 2024-09-19T06:02:33.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.085+0000 7ff61d09c640 1 -- 172.21.15.5:0/3375918468 shutdown_connections 2024-09-19T06:02:33.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.085+0000 7ff61d09c640 1 --2- 172.21.15.5:0/3375918468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff618102ef0 0x7ff61810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:33.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.085+0000 7ff61d09c640 1 --2- 172.21.15.5:0/3375918468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff618102570 0x7ff6181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:33.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.085+0000 7ff61d09c640 1 --2- 172.21.15.5:0/3375918468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff618100e20 0x7ff618101200 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.085+0000 7ff61d09c640 1 -- 172.21.15.5:0/3375918468 >> 172.21.15.5:0/3375918468 conn(0x7ff6180fc7d0 msgr2=0x7ff6180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.085+0000 7ff61d09c640 1 -- 172.21.15.5:0/3375918468 shutdown_connections 2024-09-19T06:02:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.085+0000 7ff61d09c640 1 -- 172.21.15.5:0/3375918468 wait complete. 2024-09-19T06:02:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff61d09c640 1 Processor -- start 2024-09-19T06:02:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff61d09c640 1 -- start start 2024-09-19T06:02:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff61d09c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff618100e20 0x7ff618197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff61d09c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff618102570 0x7ff618197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff61d09c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff618102ef0 0x7ff61819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff61d09c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff618110e50 con 0x7ff618102570 2024-09-19T06:02:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff61d09c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff618110cd0 con 0x7ff618100e20 2024-09-19T06:02:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff61d09c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff618110fd0 con 0x7ff618102ef0 2024-09-19T06:02:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff61659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff618102570 0x7ff618197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff61659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff618102570 0x7ff618197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37314/0 (socket says 172.21.15.5:37314) 2024-09-19T06:02:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff61659c640 1 -- 172.21.15.5:0/3591581349 learned_addr learned my addr 172.21.15.5:0/3591581349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff61759e640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff618102ef0 0x7ff61819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.086+0000 7ff616d9d640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff618100e20 0x7ff618197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.087+0000 7ff61659c640 1 -- 172.21.15.5:0/3591581349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff618102ef0 msgr2=0x7ff61819bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.087+0000 7ff61659c640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff618102ef0 0x7ff61819bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.087+0000 7ff61659c640 1 -- 172.21.15.5:0/3591581349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff618100e20 msgr2=0x7ff618197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.087+0000 7ff61659c640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff618100e20 0x7ff618197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.087+0000 7ff61659c640 1 -- 172.21.15.5:0/3591581349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61819c2e0 con 0x7ff618102570 2024-09-19T06:02:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.087+0000 7ff61659c640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff618102570 0x7ff618197870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c890 tx=0x7ff60c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.087+0000 7ff5fffff640 1 -- 172.21.15.5:0/3591581349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c004220 con 0x7ff618102570 2024-09-19T06:02:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.087+0000 7ff61d09c640 1 -- 172.21.15.5:0/3591581349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61819c5d0 con 0x7ff618102570 2024-09-19T06:02:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.087+0000 7ff5fffff640 1 -- 172.21.15.5:0/3591581349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff60c0043c0 con 0x7ff618102570 2024-09-19T06:02:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.087+0000 7ff5fffff640 1 -- 172.21.15.5:0/3591581349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c004a00 con 0x7ff618102570 2024-09-19T06:02:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.087+0000 7ff61d09c640 1 -- 172.21.15.5:0/3591581349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6181a3eb0 con 0x7ff618102570 2024-09-19T06:02:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.088+0000 7ff61d09c640 1 -- 172.21.15.5:0/3591581349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff618101200 con 0x7ff618102570 2024-09-19T06:02:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.089+0000 7ff5fffff640 1 -- 172.21.15.5:0/3591581349 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7ff60c004ba0 con 0x7ff618102570 2024-09-19T06:02:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.090+0000 7ff5fffff640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff6000a32d0 0x7ff6000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.090+0000 7ff616d9d640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff6000a32d0 0x7ff6000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.090+0000 7ff5fffff640 1 -- 172.21.15.5:0/3591581349 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff60c0cf770 con 0x7ff618102570 2024-09-19T06:02:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.090+0000 7ff616d9d640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff6000a32d0 0x7ff6000a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff608009a60 tx=0x7ff608009890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:33.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.094+0000 7ff5fffff640 1 -- 172.21.15.5:0/3591581349 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff60c014030 con 0x7ff618102570 2024-09-19T06:02:33.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.279+0000 7ff61d09c640 1 -- 172.21.15.5:0/3591581349 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff618062630 con 0x7ff6000a32d0 2024-09-19T06:02:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.288+0000 7ff5fffff640 1 -- 172.21.15.5:0/3591581349 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258749 (secure 0 0 0) 0x7ff618062630 con 0x7ff6000a32d0 2024-09-19T06:02:33.293 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:33.294 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-19T06:02:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.292+0000 7ff61d09c640 1 -- 172.21.15.5:0/3591581349 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff6000a32d0 msgr2=0x7ff6000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.292+0000 7ff61d09c640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff6000a32d0 0x7ff6000a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff608009a60 tx=0x7ff608009890 comp rx=0 tx=0).stop 2024-09-19T06:02:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.292+0000 7ff61d09c640 1 -- 172.21.15.5:0/3591581349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff618102570 msgr2=0x7ff618197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.292+0000 7ff61d09c640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff618102570 0x7ff618197870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c890 tx=0x7ff60c00cd50 comp rx=0 tx=0).stop 2024-09-19T06:02:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.292+0000 7ff61d09c640 1 -- 172.21.15.5:0/3591581349 shutdown_connections 2024-09-19T06:02:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.292+0000 7ff61d09c640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff6000a32d0 0x7ff6000a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.292+0000 7ff61d09c640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff618102ef0 0x7ff61819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.292+0000 7ff61d09c640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff618102570 0x7ff618197870 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.292+0000 7ff61d09c640 1 --2- 172.21.15.5:0/3591581349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff618100e20 0x7ff618197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.292+0000 7ff61d09c640 1 -- 172.21.15.5:0/3591581349 >> 172.21.15.5:0/3591581349 conn(0x7ff6180fc7d0 msgr2=0x7ff6180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.292+0000 7ff61d09c640 1 -- 172.21.15.5:0/3591581349 shutdown_connections 2024-09-19T06:02:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:33.293+0000 7ff61d09c640 1 -- 172.21.15.5:0/3591581349 wait complete. 2024-09-19T06:02:34.105 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-19T06:02:32.798969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659768,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":659,"num_read_kb":456,"num_write":428,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387100,"kb_used_data":12876,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554724,"statfs":{"total":1151940427776,"available":1151544037376,"internally_reserved":0,"allocated":13185024,"data_stored":7223256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003116"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833240+0000","last_change":"2024-09-19T06:01:28.812722+0000","last_active":"2024-09-19T06:01:39.833240+0000","last_peered":"2024-09-19T06:01:39.833240+0000","last_clean":"2024-09-19T06:01:39.833240+0000","last_became_active":"2024-09-19T06:01:28.812283+0000","last_became_peered":"2024-09-19T06:01:28.812283+0000","last_unstale":"2024-09-19T06:01:39.833240+0000","last_undegraded":"2024-09-19T06:01:39.833240+0000","last_fullsized":"2024-09-19T06:01:39.833240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:59.146636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831968+0000","last_change":"2024-09-19T06:01:28.805210+0000","last_active":"2024-09-19T06:01:39.831968+0000","last_peered":"2024-09-19T06:01:39.831968+0000","last_clean":"2024-09-19T06:01:39.831968+0000","last_became_active":"2024-09-19T06:01:28.804906+0000","last_became_peered":"2024-09-19T06:01:28.804906+0000","last_unstale":"2024-09-19T06:01:39.831968+0000","last_undegraded":"2024-09-19T06:01:39.831968+0000","last_fullsized":"2024-09-19T06:01:39.831968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:00.227101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832025+0000","last_change":"2024-09-19T06:01:28.811909+0000","last_active":"2024-09-19T06:01:39.832025+0000","last_peered":"2024-09-19T06:01:39.832025+0000","last_clean":"2024-09-19T06:01:39.832025+0000","last_became_active":"2024-09-19T06:01:28.811547+0000","last_became_peered":"2024-09-19T06:01:28.811547+0000","last_unstale":"2024-09-19T06:01:39.832025+0000","last_undegraded":"2024-09-19T06:01:39.832025+0000","last_fullsized":"2024-09-19T06:01:39.832025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:50.022439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830196+0000","last_change":"2024-09-19T06:01:28.806782+0000","last_active":"2024-09-19T06:01:39.830196+0000","last_peered":"2024-09-19T06:01:39.830196+0000","last_clean":"2024-09-19T06:01:39.830196+0000","last_became_active":"2024-09-19T06:01:28.805520+0000","last_became_peered":"2024-09-19T06:01:28.805520+0000","last_unstale":"2024-09-19T06:01:39.830196+0000","last_undegraded":"2024-09-19T06:01:39.830196+0000","last_fullsized":"2024-09-19T06:01:39.830196+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:01.652669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837308+0000","last_change":"2024-09-19T06:01:28.809971+0000","last_active":"2024-09-19T06:01:39.837308+0000","last_peered":"2024-09-19T06:01:39.837308+0000","last_clean":"2024-09-19T06:01:39.837308+0000","last_became_active":"2024-09-19T06:01:28.809593+0000","last_became_peered":"2024-09-19T06:01:28.809593+0000","last_unstale":"2024-09-19T06:01:39.837308+0000","last_undegraded":"2024-09-19T06:01:39.837308+0000","last_fullsized":"2024-09-19T06:01:39.837308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.546526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323650+0000","last_change":"2024-09-19T06:01:28.801395+0000","last_active":"2024-09-19T06:01:40.323650+0000","last_peered":"2024-09-19T06:01:40.323650+0000","last_clean":"2024-09-19T06:01:40.323650+0000","last_became_active":"2024-09-19T06:01:28.800987+0000","last_became_peered":"2024-09-19T06:01:28.800987+0000","last_unstale":"2024-09-19T06:01:40.323650+0000","last_undegraded":"2024-09-19T06:01:40.323650+0000","last_fullsized":"2024-09-19T06:01:40.323650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:18.986489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837262+0000","last_change":"2024-09-19T06:01:28.806554+0000","last_active":"2024-09-19T06:01:39.837262+0000","last_peered":"2024-09-19T06:01:39.837262+0000","last_clean":"2024-09-19T06:01:39.837262+0000","last_became_active":"2024-09-19T06:01:28.805946+0000","last_became_peered":"2024-09-19T06:01:28.805946+0000","last_unstale":"2024-09-19T06:01:39.837262+0000","last_undegraded":"2024-09-19T06:01:39.837262+0000","last_fullsized":"2024-09-19T06:01:39.837262+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:56:20.130414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.979824+0000","last_change":"2024-09-19T06:01:28.804789+0000","last_active":"2024-09-19T06:01:39.979824+0000","last_peered":"2024-09-19T06:01:39.979824+0000","last_clean":"2024-09-19T06:01:39.979824+0000","last_became_active":"2024-09-19T06:01:28.804484+0000","last_became_peered":"2024-09-19T06:01:28.804484+0000","last_unstale":"2024-09-19T06:01:39.979824+0000","last_undegraded":"2024-09-19T06:01:39.979824+0000","last_fullsized":"2024-09-19T06:01:39.979824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:19.737928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832749+0000","last_change":"2024-09-19T06:01:28.814504+0000","last_active":"2024-09-19T06:01:39.832749+0000","last_peered":"2024-09-19T06:01:39.832749+0000","last_clean":"2024-09-19T06:01:39.832749+0000","last_became_active":"2024-09-19T06:01:28.814066+0000","last_became_peered":"2024-09-19T06:01:28.814066+0000","last_unstale":"2024-09-19T06:01:39.832749+0000","last_undegraded":"2024-09-19T06:01:39.832749+0000","last_fullsized":"2024-09-19T06:01:39.832749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:50.904164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.324053+0000","last_change":"2024-09-19T06:01:28.807303+0000","last_active":"2024-09-19T06:01:40.324053+0000","last_peered":"2024-09-19T06:01:40.324053+0000","last_clean":"2024-09-19T06:01:40.324053+0000","last_became_active":"2024-09-19T06:01:28.806978+0000","last_became_peered":"2024-09-19T06:01:28.806978+0000","last_unstale":"2024-09-19T06:01:40.324053+0000","last_undegraded":"2024-09-19T06:01:40.324053+0000","last_fullsized":"2024-09-19T06:01:40.324053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:33.259261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837723+0000","last_change":"2024-09-19T06:01:28.814844+0000","last_active":"2024-09-19T06:01:39.837723+0000","last_peered":"2024-09-19T06:01:39.837723+0000","last_clean":"2024-09-19T06:01:39.837723+0000","last_became_active":"2024-09-19T06:01:28.814531+0000","last_became_peered":"2024-09-19T06:01:28.814531+0000","last_unstale":"2024-09-19T06:01:39.837723+0000","last_undegraded":"2024-09-19T06:01:39.837723+0000","last_fullsized":"2024-09-19T06:01:39.837723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:17.209890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830253+0000","last_change":"2024-09-19T06:01:28.806378+0000","last_active":"2024-09-19T06:01:39.830253+0000","last_peered":"2024-09-19T06:01:39.830253+0000","last_clean":"2024-09-19T06:01:39.830253+0000","last_became_active":"2024-09-19T06:01:28.805053+0000","last_became_peered":"2024-09-19T06:01:28.805053+0000","last_unstale":"2024-09-19T06:01:39.830253+0000","last_undegraded":"2024-09-19T06:01:39.830253+0000","last_fullsized":"2024-09-19T06:01:39.830253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:15.080548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829093+0000","last_change":"2024-09-19T06:01:28.802941+0000","last_active":"2024-09-19T06:01:39.829093+0000","last_peered":"2024-09-19T06:01:39.829093+0000","last_clean":"2024-09-19T06:01:39.829093+0000","last_became_active":"2024-09-19T06:01:28.802590+0000","last_became_peered":"2024-09-19T06:01:28.802590+0000","last_unstale":"2024-09-19T06:01:39.829093+0000","last_undegraded":"2024-09-19T06:01:39.829093+0000","last_fullsized":"2024-09-19T06:01:39.829093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:25.581854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831644+0000","last_change":"2024-09-19T06:01:28.800960+0000","last_active":"2024-09-19T06:01:39.831644+0000","last_peered":"2024-09-19T06:01:39.831644+0000","last_clean":"2024-09-19T06:01:39.831644+0000","last_became_active":"2024-09-19T06:01:28.800484+0000","last_became_peered":"2024-09-19T06:01:28.800484+0000","last_unstale":"2024-09-19T06:01:39.831644+0000","last_undegraded":"2024-09-19T06:01:39.831644+0000","last_fullsized":"2024-09-19T06:01:39.831644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:08.506105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831592+0000","last_change":"2024-09-19T06:01:28.800771+0000","last_active":"2024-09-19T06:01:39.831592+0000","last_peered":"2024-09-19T06:01:39.831592+0000","last_clean":"2024-09-19T06:01:39.831592+0000","last_became_active":"2024-09-19T06:01:28.800427+0000","last_became_peered":"2024-09-19T06:01:28.800427+0000","last_unstale":"2024-09-19T06:01:39.831592+0000","last_undegraded":"2024-09-19T06:01:39.831592+0000","last_fullsized":"2024-09-19T06:01:39.831592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:19.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831899+0000","last_change":"2024-09-19T06:01:28.815528+0000","last_active":"2024-09-19T06:01:39.831899+0000","last_peered":"2024-09-19T06:01:39.831899+0000","last_clean":"2024-09-19T06:01:39.831899+0000","last_became_active":"2024-09-19T06:01:28.815129+0000","last_became_peered":"2024-09-19T06:01:28.815129+0000","last_unstale":"2024-09-19T06:01:39.831899+0000","last_undegraded":"2024-09-19T06:01:39.831899+0000","last_fullsized":"2024-09-19T06:01:39.831899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:01.936711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833355+0000","last_change":"2024-09-19T06:01:28.816120+0000","last_active":"2024-09-19T06:01:39.833355+0000","last_peered":"2024-09-19T06:01:39.833355+0000","last_clean":"2024-09-19T06:01:39.833355+0000","last_became_active":"2024-09-19T06:01:28.815822+0000","last_became_peered":"2024-09-19T06:01:28.815822+0000","last_unstale":"2024-09-19T06:01:39.833355+0000","last_undegraded":"2024-09-19T06:01:39.833355+0000","last_fullsized":"2024-09-19T06:01:39.833355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:14:58.670452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.836100+0000","last_change":"2024-09-19T06:01:28.809825+0000","last_active":"2024-09-19T06:01:39.836100+0000","last_peered":"2024-09-19T06:01:39.836100+0000","last_clean":"2024-09-19T06:01:39.836100+0000","last_became_active":"2024-09-19T06:01:28.809530+0000","last_became_peered":"2024-09-19T06:01:28.809530+0000","last_unstale":"2024-09-19T06:01:39.836100+0000","last_undegraded":"2024-09-19T06:01:39.836100+0000","last_fullsized":"2024-09-19T06:01:39.836100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:49:23.442436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830644+0000","last_change":"2024-09-19T06:01:28.801297+0000","last_active":"2024-09-19T06:01:39.830644+0000","last_peered":"2024-09-19T06:01:39.830644+0000","last_clean":"2024-09-19T06:01:39.830644+0000","last_became_active":"2024-09-19T06:01:28.800984+0000","last_became_peered":"2024-09-19T06:01:28.800984+0000","last_unstale":"2024-09-19T06:01:39.830644+0000","last_undegraded":"2024-09-19T06:01:39.830644+0000","last_fullsized":"2024-09-19T06:01:39.830644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:01.485951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832407+0000","last_change":"2024-09-19T06:01:28.804567+0000","last_active":"2024-09-19T06:01:39.832407+0000","last_peered":"2024-09-19T06:01:39.832407+0000","last_clean":"2024-09-19T06:01:39.832407+0000","last_became_active":"2024-09-19T06:01:28.804213+0000","last_became_peered":"2024-09-19T06:01:28.804213+0000","last_unstale":"2024-09-19T06:01:39.832407+0000","last_undegraded":"2024-09-19T06:01:39.832407+0000","last_fullsized":"2024-09-19T06:01:39.832407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.186460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829142+0000","last_change":"2024-09-19T06:01:28.801961+0000","last_active":"2024-09-19T06:01:39.829142+0000","last_peered":"2024-09-19T06:01:39.829142+0000","last_clean":"2024-09-19T06:01:39.829142+0000","last_became_active":"2024-09-19T06:01:28.801631+0000","last_became_peered":"2024-09-19T06:01:28.801631+0000","last_unstale":"2024-09-19T06:01:39.829142+0000","last_undegraded":"2024-09-19T06:01:39.829142+0000","last_fullsized":"2024-09-19T06:01:39.829142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:15.628908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832042+0000","last_change":"2024-09-19T06:01:28.815160+0000","last_active":"2024-09-19T06:01:39.832042+0000","last_peered":"2024-09-19T06:01:39.832042+0000","last_clean":"2024-09-19T06:01:39.832042+0000","last_became_active":"2024-09-19T06:01:28.813329+0000","last_became_peered":"2024-09-19T06:01:28.813329+0000","last_unstale":"2024-09-19T06:01:39.832042+0000","last_undegraded":"2024-09-19T06:01:39.832042+0000","last_fullsized":"2024-09-19T06:01:39.832042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:17.027393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.979874+0000","last_change":"2024-09-19T06:01:28.799205+0000","last_active":"2024-09-19T06:01:39.979874+0000","last_peered":"2024-09-19T06:01:39.979874+0000","last_clean":"2024-09-19T06:01:39.979874+0000","last_became_active":"2024-09-19T06:01:28.798891+0000","last_became_peered":"2024-09-19T06:01:28.798891+0000","last_unstale":"2024-09-19T06:01:39.979874+0000","last_undegraded":"2024-09-19T06:01:39.979874+0000","last_fullsized":"2024-09-19T06:01:39.979874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:41.227721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832707+0000","last_change":"2024-09-19T06:01:28.795823+0000","last_active":"2024-09-19T06:01:39.832707+0000","last_peered":"2024-09-19T06:01:39.832707+0000","last_clean":"2024-09-19T06:01:39.832707+0000","last_became_active":"2024-09-19T06:01:28.795510+0000","last_became_peered":"2024-09-19T06:01:28.795510+0000","last_unstale":"2024-09-19T06:01:39.832707+0000","last_undegraded":"2024-09-19T06:01:39.832707+0000","last_fullsized":"2024-09-19T06:01:39.832707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:54.831610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980449+0000","last_change":"2024-09-19T06:01:28.799696+0000","last_active":"2024-09-19T06:01:39.980449+0000","last_peered":"2024-09-19T06:01:39.980449+0000","last_clean":"2024-09-19T06:01:39.980449+0000","last_became_active":"2024-09-19T06:01:28.799417+0000","last_became_peered":"2024-09-19T06:01:28.799417+0000","last_unstale":"2024-09-19T06:01:39.980449+0000","last_undegraded":"2024-09-19T06:01:39.980449+0000","last_fullsized":"2024-09-19T06:01:39.980449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:02.330808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830714+0000","last_change":"2024-09-19T06:01:28.807146+0000","last_active":"2024-09-19T06:01:39.830714+0000","last_peered":"2024-09-19T06:01:39.830714+0000","last_clean":"2024-09-19T06:01:39.830714+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:01:39.830714+0000","last_undegraded":"2024-09-19T06:01:39.830714+0000","last_fullsized":"2024-09-19T06:01:39.830714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:12.308421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832819+0000","last_change":"2024-09-19T06:01:28.803555+0000","last_active":"2024-09-19T06:01:39.832819+0000","last_peered":"2024-09-19T06:01:39.832819+0000","last_clean":"2024-09-19T06:01:39.832819+0000","last_became_active":"2024-09-19T06:01:28.803242+0000","last_became_peered":"2024-09-19T06:01:28.803242+0000","last_unstale":"2024-09-19T06:01:39.832819+0000","last_undegraded":"2024-09-19T06:01:39.832819+0000","last_fullsized":"2024-09-19T06:01:39.832819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:02.281152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829217+0000","last_change":"2024-09-19T06:01:28.804378+0000","last_active":"2024-09-19T06:01:39.829217+0000","last_peered":"2024-09-19T06:01:39.829217+0000","last_clean":"2024-09-19T06:01:39.829217+0000","last_became_active":"2024-09-19T06:01:28.804099+0000","last_became_peered":"2024-09-19T06:01:28.804099+0000","last_unstale":"2024-09-19T06:01:39.829217+0000","last_undegraded":"2024-09-19T06:01:39.829217+0000","last_fullsized":"2024-09-19T06:01:39.829217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:11.558384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831784+0000","last_change":"2024-09-19T06:01:28.799999+0000","last_active":"2024-09-19T06:01:39.831784+0000","last_peered":"2024-09-19T06:01:39.831784+0000","last_clean":"2024-09-19T06:01:39.831784+0000","last_became_active":"2024-09-19T06:01:28.799645+0000","last_became_peered":"2024-09-19T06:01:28.799645+0000","last_unstale":"2024-09-19T06:01:39.831784+0000","last_undegraded":"2024-09-19T06:01:39.831784+0000","last_fullsized":"2024-09-19T06:01:39.831784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:20.442119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830148+0000","last_change":"2024-09-19T06:01:28.800143+0000","last_active":"2024-09-19T06:01:39.830148+0000","last_peered":"2024-09-19T06:01:39.830148+0000","last_clean":"2024-09-19T06:01:39.830148+0000","last_became_active":"2024-09-19T06:01:28.799830+0000","last_became_peered":"2024-09-19T06:01:28.799830+0000","last_unstale":"2024-09-19T06:01:39.830148+0000","last_undegraded":"2024-09-19T06:01:39.830148+0000","last_fullsized":"2024-09-19T06:01:39.830148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:43.824740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830661+0000","last_change":"2024-09-19T06:01:28.806130+0000","last_active":"2024-09-19T06:01:39.830661+0000","last_peered":"2024-09-19T06:01:39.830661+0000","last_clean":"2024-09-19T06:01:39.830661+0000","last_became_active":"2024-09-19T06:01:28.805772+0000","last_became_peered":"2024-09-19T06:01:28.805772+0000","last_unstale":"2024-09-19T06:01:39.830661+0000","last_undegraded":"2024-09-19T06:01:39.830661+0000","last_fullsized":"2024-09-19T06:01:39.830661+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:00.316311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837412+0000","last_change":"2024-09-19T06:01:39.837412+0000","last_active":"2024-09-19T06:01:39.837412+0000","last_peered":"2024-09-19T06:01:39.837412+0000","last_clean":"2024-09-19T06:01:39.837412+0000","last_became_active":"2024-09-19T06:01:39.837035+0000","last_became_peered":"2024-09-19T06:01:39.837035+0000","last_unstale":"2024-09-19T06:01:39.837412+0000","last_undegraded":"2024-09-19T06:01:39.837412+0000","last_fullsized":"2024-09-19T06:01:39.837412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:00.073884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837634+0000","last_change":"2024-09-19T06:01:28.800324+0000","last_active":"2024-09-19T06:01:39.837634+0000","last_peered":"2024-09-19T06:01:39.837634+0000","last_clean":"2024-09-19T06:01:39.837634+0000","last_became_active":"2024-09-19T06:01:28.799986+0000","last_became_peered":"2024-09-19T06:01:28.799986+0000","last_unstale":"2024-09-19T06:01:39.837634+0000","last_undegraded":"2024-09-19T06:01:39.837634+0000","last_fullsized":"2024-09-19T06:01:39.837634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:41.963346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":90,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:26.911880+0000","last_change":"2024-09-19T06:01:39.844897+0000","last_active":"2024-09-19T06:02:26.911880+0000","last_peered":"2024-09-19T06:02:26.911880+0000","last_clean":"2024-09-19T06:02:26.911880+0000","last_became_active":"2024-09-19T06:01:39.838347+0000","last_became_peered":"2024-09-19T06:01:39.838347+0000","last_unstale":"2024-09-19T06:02:26.911880+0000","last_undegraded":"2024-09-19T06:02:26.911880+0000","last_fullsized":"2024-09-19T06:02:26.911880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:26.235227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.834624+0000","last_change":"2024-09-19T06:01:28.802644+0000","last_active":"2024-09-19T06:01:39.834624+0000","last_peered":"2024-09-19T06:01:39.834624+0000","last_clean":"2024-09-19T06:01:39.834624+0000","last_became_active":"2024-09-19T06:01:28.802288+0000","last_became_peered":"2024-09-19T06:01:28.802288+0000","last_unstale":"2024-09-19T06:01:39.834624+0000","last_undegraded":"2024-09-19T06:01:39.834624+0000","last_fullsized":"2024-09-19T06:01:39.834624+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:32.031860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323748+0000","last_change":"2024-09-19T06:01:26.781104+0000","last_active":"2024-09-19T06:01:40.323748+0000","last_peered":"2024-09-19T06:01:40.323748+0000","last_clean":"2024-09-19T06:01:40.323748+0000","last_became_active":"2024-09-19T06:01:26.780733+0000","last_became_peered":"2024-09-19T06:01:26.780733+0000","last_unstale":"2024-09-19T06:01:40.323748+0000","last_undegraded":"2024-09-19T06:01:40.323748+0000","last_fullsized":"2024-09-19T06:01:40.323748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:54.490867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.828967+0000","last_change":"2024-09-19T06:01:26.782901+0000","last_active":"2024-09-19T06:01:39.828967+0000","last_peered":"2024-09-19T06:01:39.828967+0000","last_clean":"2024-09-19T06:01:39.828967+0000","last_became_active":"2024-09-19T06:01:26.782450+0000","last_became_peered":"2024-09-19T06:01:26.782450+0000","last_unstale":"2024-09-19T06:01:39.828967+0000","last_undegraded":"2024-09-19T06:01:39.828967+0000","last_fullsized":"2024-09-19T06:01:39.828967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:07.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.979905+0000","last_change":"2024-09-19T06:01:28.804105+0000","last_active":"2024-09-19T06:01:39.979905+0000","last_peered":"2024-09-19T06:01:39.979905+0000","last_clean":"2024-09-19T06:01:39.979905+0000","last_became_active":"2024-09-19T06:01:28.797980+0000","last_became_peered":"2024-09-19T06:01:28.797980+0000","last_unstale":"2024-09-19T06:01:39.979905+0000","last_undegraded":"2024-09-19T06:01:39.979905+0000","last_fullsized":"2024-09-19T06:01:39.979905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:50.130102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.848777+0000","last_change":"2024-09-19T06:01:26.786923+0000","last_active":"2024-09-19T06:01:40.848777+0000","last_peered":"2024-09-19T06:01:40.848777+0000","last_clean":"2024-09-19T06:01:40.848777+0000","last_became_active":"2024-09-19T06:01:26.786542+0000","last_became_peered":"2024-09-19T06:01:26.786542+0000","last_unstale":"2024-09-19T06:01:40.848777+0000","last_undegraded":"2024-09-19T06:01:40.848777+0000","last_fullsized":"2024-09-19T06:01:40.848777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:30.032992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.835994+0000","last_change":"2024-09-19T06:01:28.802172+0000","last_active":"2024-09-19T06:01:39.835994+0000","last_peered":"2024-09-19T06:01:39.835994+0000","last_clean":"2024-09-19T06:01:39.835994+0000","last_became_active":"2024-09-19T06:01:28.801815+0000","last_became_peered":"2024-09-19T06:01:28.801815+0000","last_unstale":"2024-09-19T06:01:39.835994+0000","last_undegraded":"2024-09-19T06:01:39.835994+0000","last_fullsized":"2024-09-19T06:01:39.835994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:09.587638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:28.910552+0000","last_change":"2024-09-19T06:01:26.777870+0000","last_active":"2024-09-19T06:02:28.910552+0000","last_peered":"2024-09-19T06:02:28.910552+0000","last_clean":"2024-09-19T06:02:28.910552+0000","last_became_active":"2024-09-19T06:01:26.777509+0000","last_became_peered":"2024-09-19T06:01:26.777509+0000","last_unstale":"2024-09-19T06:02:28.910552+0000","last_undegraded":"2024-09-19T06:02:28.910552+0000","last_fullsized":"2024-09-19T06:02:28.910552+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:15.885911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832446+0000","last_change":"2024-09-19T06:01:28.812201+0000","last_active":"2024-09-19T06:01:39.832446+0000","last_peered":"2024-09-19T06:01:39.832446+0000","last_clean":"2024-09-19T06:01:39.832446+0000","last_became_active":"2024-09-19T06:01:28.811801+0000","last_became_peered":"2024-09-19T06:01:28.811801+0000","last_unstale":"2024-09-19T06:01:39.832446+0000","last_undegraded":"2024-09-19T06:01:39.832446+0000","last_fullsized":"2024-09-19T06:01:39.832446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:32.904787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.848667+0000","last_change":"2024-09-19T06:01:26.787845+0000","last_active":"2024-09-19T06:01:40.848667+0000","last_peered":"2024-09-19T06:01:40.848667+0000","last_clean":"2024-09-19T06:01:40.848667+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:01:40.848667+0000","last_undegraded":"2024-09-19T06:01:40.848667+0000","last_fullsized":"2024-09-19T06:01:40.848667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.114865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832096+0000","last_change":"2024-09-19T06:01:28.801523+0000","last_active":"2024-09-19T06:01:39.832096+0000","last_peered":"2024-09-19T06:01:39.832096+0000","last_clean":"2024-09-19T06:01:39.832096+0000","last_became_active":"2024-09-19T06:01:28.801205+0000","last_became_peered":"2024-09-19T06:01:28.801205+0000","last_unstale":"2024-09-19T06:01:39.832096+0000","last_undegraded":"2024-09-19T06:01:39.832096+0000","last_fullsized":"2024-09-19T06:01:39.832096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:10.314085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833424+0000","last_change":"2024-09-19T06:01:27.022172+0000","last_active":"2024-09-19T06:01:39.833424+0000","last_peered":"2024-09-19T06:01:39.833424+0000","last_clean":"2024-09-19T06:01:39.833424+0000","last_became_active":"2024-09-19T06:01:27.021568+0000","last_became_peered":"2024-09-19T06:01:27.021568+0000","last_unstale":"2024-09-19T06:01:39.833424+0000","last_undegraded":"2024-09-19T06:01:39.833424+0000","last_fullsized":"2024-09-19T06:01:39.833424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:18.031690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.839061+0000","last_change":"2024-09-19T06:01:39.839061+0000","last_active":"2024-09-19T06:01:39.839061+0000","last_peered":"2024-09-19T06:01:39.839061+0000","last_clean":"2024-09-19T06:01:39.839061+0000","last_became_active":"2024-09-19T06:01:39.838644+0000","last_became_peered":"2024-09-19T06:01:39.838644+0000","last_unstale":"2024-09-19T06:01:39.839061+0000","last_undegraded":"2024-09-19T06:01:39.839061+0000","last_fullsized":"2024-09-19T06:01:39.839061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:02.136222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:28.913494+0000","last_change":"2024-09-19T06:01:26.784781+0000","last_active":"2024-09-19T06:02:28.913494+0000","last_peered":"2024-09-19T06:02:28.913494+0000","last_clean":"2024-09-19T06:02:28.913494+0000","last_became_active":"2024-09-19T06:01:26.784284+0000","last_became_peered":"2024-09-19T06:01:26.784284+0000","last_unstale":"2024-09-19T06:02:28.913494+0000","last_undegraded":"2024-09-19T06:02:28.913494+0000","last_fullsized":"2024-09-19T06:02:28.913494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:13.063299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61172,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":111,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.839724+0000","last_change":"2024-09-19T06:01:28.802604+0000","last_active":"2024-09-19T06:01:39.839724+0000","last_peered":"2024-09-19T06:01:39.839724+0000","last_clean":"2024-09-19T06:01:39.839724+0000","last_became_active":"2024-09-19T06:01:28.802285+0000","last_became_peered":"2024-09-19T06:01:28.802285+0000","last_unstale":"2024-09-19T06:01:39.839724+0000","last_undegraded":"2024-09-19T06:01:39.839724+0000","last_fullsized":"2024-09-19T06:01:39.839724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:57.177791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:41.846861+0000","last_change":"2024-09-19T06:01:26.783678+0000","last_active":"2024-09-19T06:01:41.846861+0000","last_peered":"2024-09-19T06:01:41.846861+0000","last_clean":"2024-09-19T06:01:41.846861+0000","last_became_active":"2024-09-19T06:01:26.783165+0000","last_became_peered":"2024-09-19T06:01:26.783165+0000","last_unstale":"2024-09-19T06:01:41.846861+0000","last_undegraded":"2024-09-19T06:01:41.846861+0000","last_fullsized":"2024-09-19T06:01:41.846861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:50.535733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.828953+0000","last_change":"2024-09-19T06:01:28.808553+0000","last_active":"2024-09-19T06:01:39.828953+0000","last_peered":"2024-09-19T06:01:39.828953+0000","last_clean":"2024-09-19T06:01:39.828953+0000","last_became_active":"2024-09-19T06:01:28.808215+0000","last_became_peered":"2024-09-19T06:01:28.808215+0000","last_unstale":"2024-09-19T06:01:39.828953+0000","last_undegraded":"2024-09-19T06:01:39.828953+0000","last_fullsized":"2024-09-19T06:01:39.828953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:55.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.848791+0000","last_change":"2024-09-19T06:01:26.787137+0000","last_active":"2024-09-19T06:01:40.848791+0000","last_peered":"2024-09-19T06:01:40.848791+0000","last_clean":"2024-09-19T06:01:40.848791+0000","last_became_active":"2024-09-19T06:01:26.786676+0000","last_became_peered":"2024-09-19T06:01:26.786676+0000","last_unstale":"2024-09-19T06:01:40.848791+0000","last_undegraded":"2024-09-19T06:01:40.848791+0000","last_fullsized":"2024-09-19T06:01:40.848791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.609969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837656+0000","last_change":"2024-09-19T06:01:28.812106+0000","last_active":"2024-09-19T06:01:39.837656+0000","last_peered":"2024-09-19T06:01:39.837656+0000","last_clean":"2024-09-19T06:01:39.837656+0000","last_became_active":"2024-09-19T06:01:28.811771+0000","last_became_peered":"2024-09-19T06:01:28.811771+0000","last_unstale":"2024-09-19T06:01:39.837656+0000","last_undegraded":"2024-09-19T06:01:39.837656+0000","last_fullsized":"2024-09-19T06:01:39.837656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:07:06.832685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":80,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:28.911036+0000","last_change":"2024-09-19T06:01:26.786746+0000","last_active":"2024-09-19T06:02:28.911036+0000","last_peered":"2024-09-19T06:02:28.911036+0000","last_clean":"2024-09-19T06:02:28.911036+0000","last_became_active":"2024-09-19T06:01:26.786414+0000","last_became_peered":"2024-09-19T06:01:26.786414+0000","last_unstale":"2024-09-19T06:02:28.911036+0000","last_undegraded":"2024-09-19T06:02:28.911036+0000","last_fullsized":"2024-09-19T06:02:28.911036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:25.042796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837014+0000","last_change":"2024-09-19T06:01:28.807183+0000","last_active":"2024-09-19T06:01:39.837014+0000","last_peered":"2024-09-19T06:01:39.837014+0000","last_clean":"2024-09-19T06:01:39.837014+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:01:39.837014+0000","last_undegraded":"2024-09-19T06:01:39.837014+0000","last_fullsized":"2024-09-19T06:01:39.837014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:23.030312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847960+0000","last_change":"2024-09-19T06:01:27.022409+0000","last_active":"2024-09-19T06:01:40.847960+0000","last_peered":"2024-09-19T06:01:40.847960+0000","last_clean":"2024-09-19T06:01:40.847960+0000","last_became_active":"2024-09-19T06:01:27.022034+0000","last_became_peered":"2024-09-19T06:01:27.022034+0000","last_unstale":"2024-09-19T06:01:40.847960+0000","last_undegraded":"2024-09-19T06:01:40.847960+0000","last_fullsized":"2024-09-19T06:01:40.847960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:08.663575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832674+0000","last_change":"2024-09-19T06:01:28.812079+0000","last_active":"2024-09-19T06:01:39.832674+0000","last_peered":"2024-09-19T06:01:39.832674+0000","last_clean":"2024-09-19T06:01:39.832674+0000","last_became_active":"2024-09-19T06:01:28.811606+0000","last_became_peered":"2024-09-19T06:01:28.811606+0000","last_unstale":"2024-09-19T06:01:39.832674+0000","last_undegraded":"2024-09-19T06:01:39.832674+0000","last_fullsized":"2024-09-19T06:01:39.832674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:49.434235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.838741+0000","last_change":"2024-09-19T06:01:26.783823+0000","last_active":"2024-09-19T06:01:39.838741+0000","last_peered":"2024-09-19T06:01:39.838741+0000","last_clean":"2024-09-19T06:01:39.838741+0000","last_became_active":"2024-09-19T06:01:26.783209+0000","last_became_peered":"2024-09-19T06:01:26.783209+0000","last_unstale":"2024-09-19T06:01:39.838741+0000","last_undegraded":"2024-09-19T06:01:39.838741+0000","last_fullsized":"2024-09-19T06:01:39.838741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:38.833417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837462+0000","last_change":"2024-09-19T06:01:39.837462+0000","last_active":"2024-09-19T06:01:39.837462+0000","last_peered":"2024-09-19T06:01:39.837462+0000","last_clean":"2024-09-19T06:01:39.837462+0000","last_became_active":"2024-09-19T06:01:39.837082+0000","last_became_peered":"2024-09-19T06:01:39.837082+0000","last_unstale":"2024-09-19T06:01:39.837462+0000","last_undegraded":"2024-09-19T06:01:39.837462+0000","last_fullsized":"2024-09-19T06:01:39.837462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:47.421017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.845150+0000","last_change":"2024-09-19T06:01:26.779992+0000","last_active":"2024-09-19T06:01:40.845150+0000","last_peered":"2024-09-19T06:01:40.845150+0000","last_clean":"2024-09-19T06:01:40.845150+0000","last_became_active":"2024-09-19T06:01:26.779609+0000","last_became_peered":"2024-09-19T06:01:26.779609+0000","last_unstale":"2024-09-19T06:01:40.845150+0000","last_undegraded":"2024-09-19T06:01:40.845150+0000","last_fullsized":"2024-09-19T06:01:40.845150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:48.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837947+0000","last_change":"2024-09-19T06:01:28.798827+0000","last_active":"2024-09-19T06:01:39.837947+0000","last_peered":"2024-09-19T06:01:39.837947+0000","last_clean":"2024-09-19T06:01:39.837947+0000","last_became_active":"2024-09-19T06:01:28.798457+0000","last_became_peered":"2024-09-19T06:01:28.798457+0000","last_unstale":"2024-09-19T06:01:39.837947+0000","last_undegraded":"2024-09-19T06:01:39.837947+0000","last_fullsized":"2024-09-19T06:01:39.837947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:41.111051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.846954+0000","last_change":"2024-09-19T06:01:26.775691+0000","last_active":"2024-09-19T06:01:40.846954+0000","last_peered":"2024-09-19T06:01:40.846954+0000","last_clean":"2024-09-19T06:01:40.846954+0000","last_became_active":"2024-09-19T06:01:26.775183+0000","last_became_peered":"2024-09-19T06:01:26.775183+0000","last_unstale":"2024-09-19T06:01:40.846954+0000","last_undegraded":"2024-09-19T06:01:40.846954+0000","last_fullsized":"2024-09-19T06:01:40.846954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:34.712968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837674+0000","last_change":"2024-09-19T06:01:28.809459+0000","last_active":"2024-09-19T06:01:39.837674+0000","last_peered":"2024-09-19T06:01:39.837674+0000","last_clean":"2024-09-19T06:01:39.837674+0000","last_became_active":"2024-09-19T06:01:28.809046+0000","last_became_peered":"2024-09-19T06:01:28.809046+0000","last_unstale":"2024-09-19T06:01:39.837674+0000","last_undegraded":"2024-09-19T06:01:39.837674+0000","last_fullsized":"2024-09-19T06:01:39.837674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:56.996190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:27.909859+0000","last_change":"2024-09-19T06:01:39.843932+0000","last_active":"2024-09-19T06:02:27.909859+0000","last_peered":"2024-09-19T06:02:27.909859+0000","last_clean":"2024-09-19T06:02:27.909859+0000","last_became_active":"2024-09-19T06:01:39.836008+0000","last_became_peered":"2024-09-19T06:01:39.836008+0000","last_unstale":"2024-09-19T06:02:27.909859+0000","last_undegraded":"2024-09-19T06:02:27.909859+0000","last_fullsized":"2024-09-19T06:02:27.909859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:58.798859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837726+0000","last_change":"2024-09-19T06:01:28.814181+0000","last_active":"2024-09-19T06:01:39.837726+0000","last_peered":"2024-09-19T06:01:39.837726+0000","last_clean":"2024-09-19T06:01:39.837726+0000","last_became_active":"2024-09-19T06:01:28.813817+0000","last_became_peered":"2024-09-19T06:01:28.813817+0000","last_unstale":"2024-09-19T06:01:39.837726+0000","last_undegraded":"2024-09-19T06:01:39.837726+0000","last_fullsized":"2024-09-19T06:01:39.837726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:40.057863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837197+0000","last_change":"2024-09-19T06:00:17.624509+0000","last_active":"2024-09-19T06:01:39.837197+0000","last_peered":"2024-09-19T06:01:39.837197+0000","last_clean":"2024-09-19T06:01:39.837197+0000","last_became_active":"2024-09-19T06:00:17.623943+0000","last_became_peered":"2024-09-19T06:00:17.623943+0000","last_unstale":"2024-09-19T06:01:39.837197+0000","last_undegraded":"2024-09-19T06:01:39.837197+0000","last_fullsized":"2024-09-19T06:01:39.837197+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_clean_scrub_stamp":"2024-09-19T05:56:55.414488+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:39.116345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:41.847767+0000","last_change":"2024-09-19T06:01:26.783896+0000","last_active":"2024-09-19T06:01:41.847767+0000","last_peered":"2024-09-19T06:01:41.847767+0000","last_clean":"2024-09-19T06:01:41.847767+0000","last_became_active":"2024-09-19T06:01:26.783332+0000","last_became_peered":"2024-09-19T06:01:26.783332+0000","last_unstale":"2024-09-19T06:01:41.847767+0000","last_undegraded":"2024-09-19T06:01:41.847767+0000","last_fullsized":"2024-09-19T06:01:41.847767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:33.072763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980113+0000","last_change":"2024-09-19T06:01:28.807756+0000","last_active":"2024-09-19T06:01:39.980113+0000","last_peered":"2024-09-19T06:01:39.980113+0000","last_clean":"2024-09-19T06:01:39.980113+0000","last_became_active":"2024-09-19T06:01:28.807077+0000","last_became_peered":"2024-09-19T06:01:28.807077+0000","last_unstale":"2024-09-19T06:01:39.980113+0000","last_undegraded":"2024-09-19T06:01:39.980113+0000","last_fullsized":"2024-09-19T06:01:39.980113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:20.784199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837194+0000","last_change":"2024-09-19T06:01:26.784830+0000","last_active":"2024-09-19T06:01:39.837194+0000","last_peered":"2024-09-19T06:01:39.837194+0000","last_clean":"2024-09-19T06:01:39.837194+0000","last_became_active":"2024-09-19T06:01:26.783887+0000","last_became_peered":"2024-09-19T06:01:26.783887+0000","last_unstale":"2024-09-19T06:01:39.837194+0000","last_undegraded":"2024-09-19T06:01:39.837194+0000","last_fullsized":"2024-09-19T06:01:39.837194+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:33:38.255169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837354+0000","last_change":"2024-09-19T06:01:39.837354+0000","last_active":"2024-09-19T06:01:39.837354+0000","last_peered":"2024-09-19T06:01:39.837354+0000","last_clean":"2024-09-19T06:01:39.837354+0000","last_became_active":"2024-09-19T06:01:39.836930+0000","last_became_peered":"2024-09-19T06:01:39.836930+0000","last_unstale":"2024-09-19T06:01:39.837354+0000","last_undegraded":"2024-09-19T06:01:39.837354+0000","last_fullsized":"2024-09-19T06:01:39.837354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:17.571834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:27.909164+0000","last_change":"2024-09-19T06:01:27.021753+0000","last_active":"2024-09-19T06:02:27.909164+0000","last_peered":"2024-09-19T06:02:27.909164+0000","last_clean":"2024-09-19T06:02:27.909164+0000","last_became_active":"2024-09-19T06:01:27.021346+0000","last_became_peered":"2024-09-19T06:01:27.021346+0000","last_unstale":"2024-09-19T06:02:27.909164+0000","last_undegraded":"2024-09-19T06:02:27.909164+0000","last_fullsized":"2024-09-19T06:02:27.909164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:07.291283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832814+0000","last_change":"2024-09-19T06:01:28.803103+0000","last_active":"2024-09-19T06:01:39.832814+0000","last_peered":"2024-09-19T06:01:39.832814+0000","last_clean":"2024-09-19T06:01:39.832814+0000","last_became_active":"2024-09-19T06:01:28.802806+0000","last_became_peered":"2024-09-19T06:01:28.802806+0000","last_unstale":"2024-09-19T06:01:39.832814+0000","last_undegraded":"2024-09-19T06:01:39.832814+0000","last_fullsized":"2024-09-19T06:01:39.832814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:33.329231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:27.910769+0000","last_change":"2024-09-19T06:01:26.786605+0000","last_active":"2024-09-19T06:02:27.910769+0000","last_peered":"2024-09-19T06:02:27.910769+0000","last_clean":"2024-09-19T06:02:27.910769+0000","last_became_active":"2024-09-19T06:01:26.786278+0000","last_became_peered":"2024-09-19T06:01:26.786278+0000","last_unstale":"2024-09-19T06:02:27.910769+0000","last_undegraded":"2024-09-19T06:02:27.910769+0000","last_fullsized":"2024-09-19T06:02:27.910769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.581502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323778+0000","last_change":"2024-09-19T06:01:28.808152+0000","last_active":"2024-09-19T06:01:40.323778+0000","last_peered":"2024-09-19T06:01:40.323778+0000","last_clean":"2024-09-19T06:01:40.323778+0000","last_became_active":"2024-09-19T06:01:28.807788+0000","last_became_peered":"2024-09-19T06:01:28.807788+0000","last_unstale":"2024-09-19T06:01:40.323778+0000","last_undegraded":"2024-09-19T06:01:40.323778+0000","last_fullsized":"2024-09-19T06:01:40.323778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:42.877539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.836049+0000","last_change":"2024-09-19T06:01:27.022719+0000","last_active":"2024-09-19T06:01:39.836049+0000","last_peered":"2024-09-19T06:01:39.836049+0000","last_clean":"2024-09-19T06:01:39.836049+0000","last_became_active":"2024-09-19T06:01:27.022263+0000","last_became_peered":"2024-09-19T06:01:27.022263+0000","last_unstale":"2024-09-19T06:01:39.836049+0000","last_undegraded":"2024-09-19T06:01:39.836049+0000","last_fullsized":"2024-09-19T06:01:39.836049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:45.158229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323926+0000","last_change":"2024-09-19T06:01:28.809869+0000","last_active":"2024-09-19T06:01:40.323926+0000","last_peered":"2024-09-19T06:01:40.323926+0000","last_clean":"2024-09-19T06:01:40.323926+0000","last_became_active":"2024-09-19T06:01:28.809548+0000","last_became_peered":"2024-09-19T06:01:28.809548+0000","last_unstale":"2024-09-19T06:01:40.323926+0000","last_undegraded":"2024-09-19T06:01:40.323926+0000","last_fullsized":"2024-09-19T06:01:40.323926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:41.587846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847043+0000","last_change":"2024-09-19T06:01:39.846740+0000","last_active":"2024-09-19T06:01:40.847043+0000","last_peered":"2024-09-19T06:01:40.847043+0000","last_clean":"2024-09-19T06:01:40.847043+0000","last_became_active":"2024-09-19T06:01:39.836378+0000","last_became_peered":"2024-09-19T06:01:39.836378+0000","last_unstale":"2024-09-19T06:01:40.847043+0000","last_undegraded":"2024-09-19T06:01:40.847043+0000","last_fullsized":"2024-09-19T06:01:40.847043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.064918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829305+0000","last_change":"2024-09-19T06:01:28.807796+0000","last_active":"2024-09-19T06:01:39.829305+0000","last_peered":"2024-09-19T06:01:39.829305+0000","last_clean":"2024-09-19T06:01:39.829305+0000","last_became_active":"2024-09-19T06:01:28.807503+0000","last_became_peered":"2024-09-19T06:01:28.807503+0000","last_unstale":"2024-09-19T06:01:39.829305+0000","last_undegraded":"2024-09-19T06:01:39.829305+0000","last_fullsized":"2024-09-19T06:01:39.829305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:43.472647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:52.795819+0000","last_change":"2024-09-19T06:01:26.783517+0000","last_active":"2024-09-19T06:01:52.795819+0000","last_peered":"2024-09-19T06:01:52.795819+0000","last_clean":"2024-09-19T06:01:52.795819+0000","last_became_active":"2024-09-19T06:01:26.783164+0000","last_became_peered":"2024-09-19T06:01:26.783164+0000","last_unstale":"2024-09-19T06:01:52.795819+0000","last_undegraded":"2024-09-19T06:01:52.795819+0000","last_fullsized":"2024-09-19T06:01:52.795819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:57.609192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980454+0000","last_change":"2024-09-19T06:01:28.806548+0000","last_active":"2024-09-19T06:01:39.980454+0000","last_peered":"2024-09-19T06:01:39.980454+0000","last_clean":"2024-09-19T06:01:39.980454+0000","last_became_active":"2024-09-19T06:01:28.805724+0000","last_became_peered":"2024-09-19T06:01:28.805724+0000","last_unstale":"2024-09-19T06:01:39.980454+0000","last_undegraded":"2024-09-19T06:01:39.980454+0000","last_fullsized":"2024-09-19T06:01:39.980454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:14.360928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847547+0000","last_change":"2024-09-19T06:01:27.128213+0000","last_active":"2024-09-19T06:01:40.847547+0000","last_peered":"2024-09-19T06:01:40.847547+0000","last_clean":"2024-09-19T06:01:40.847547+0000","last_became_active":"2024-09-19T06:01:27.127737+0000","last_became_peered":"2024-09-19T06:01:27.127737+0000","last_unstale":"2024-09-19T06:01:40.847547+0000","last_undegraded":"2024-09-19T06:01:40.847547+0000","last_fullsized":"2024-09-19T06:01:40.847547+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:33:35.107152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.839027+0000","last_change":"2024-09-19T06:01:39.839027+0000","last_active":"2024-09-19T06:01:39.839027+0000","last_peered":"2024-09-19T06:01:39.839027+0000","last_clean":"2024-09-19T06:01:39.839027+0000","last_became_active":"2024-09-19T06:01:39.838601+0000","last_became_peered":"2024-09-19T06:01:39.838601+0000","last_unstale":"2024-09-19T06:01:39.839027+0000","last_undegraded":"2024-09-19T06:01:39.839027+0000","last_fullsized":"2024-09-19T06:01:39.839027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:09.864997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832051+0000","last_change":"2024-09-19T06:01:27.022438+0000","last_active":"2024-09-19T06:01:39.832051+0000","last_peered":"2024-09-19T06:01:39.832051+0000","last_clean":"2024-09-19T06:01:39.832051+0000","last_became_active":"2024-09-19T06:01:27.021953+0000","last_became_peered":"2024-09-19T06:01:27.021953+0000","last_unstale":"2024-09-19T06:01:39.832051+0000","last_undegraded":"2024-09-19T06:01:39.832051+0000","last_fullsized":"2024-09-19T06:01:39.832051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:24.383178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837088+0000","last_change":"2024-09-19T06:01:28.803807+0000","last_active":"2024-09-19T06:01:39.837088+0000","last_peered":"2024-09-19T06:01:39.837088+0000","last_clean":"2024-09-19T06:01:39.837088+0000","last_became_active":"2024-09-19T06:01:28.803486+0000","last_became_peered":"2024-09-19T06:01:28.803486+0000","last_unstale":"2024-09-19T06:01:39.837088+0000","last_undegraded":"2024-09-19T06:01:39.837088+0000","last_fullsized":"2024-09-19T06:01:39.837088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:26:17.920089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.845539+0000","last_change":"2024-09-19T06:01:26.777171+0000","last_active":"2024-09-19T06:01:40.845539+0000","last_peered":"2024-09-19T06:01:40.845539+0000","last_clean":"2024-09-19T06:01:40.845539+0000","last_became_active":"2024-09-19T06:01:26.776818+0000","last_became_peered":"2024-09-19T06:01:26.776818+0000","last_unstale":"2024-09-19T06:01:40.845539+0000","last_undegraded":"2024-09-19T06:01:40.845539+0000","last_fullsized":"2024-09-19T06:01:40.845539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832532+0000","last_change":"2024-09-19T06:01:28.799266+0000","last_active":"2024-09-19T06:01:39.832532+0000","last_peered":"2024-09-19T06:01:39.832532+0000","last_clean":"2024-09-19T06:01:39.832532+0000","last_became_active":"2024-09-19T06:01:28.798969+0000","last_became_peered":"2024-09-19T06:01:28.798969+0000","last_unstale":"2024-09-19T06:01:39.832532+0000","last_undegraded":"2024-09-19T06:01:39.832532+0000","last_fullsized":"2024-09-19T06:01:39.832532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:34.064795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830288+0000","last_change":"2024-09-19T06:01:26.781680+0000","last_active":"2024-09-19T06:01:39.830288+0000","last_peered":"2024-09-19T06:01:39.830288+0000","last_clean":"2024-09-19T06:01:39.830288+0000","last_became_active":"2024-09-19T06:01:26.781035+0000","last_became_peered":"2024-09-19T06:01:26.781035+0000","last_unstale":"2024-09-19T06:01:39.830288+0000","last_undegraded":"2024-09-19T06:01:39.830288+0000","last_fullsized":"2024-09-19T06:01:39.830288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:38.287018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980152+0000","last_change":"2024-09-19T06:01:28.802970+0000","last_active":"2024-09-19T06:01:39.980152+0000","last_peered":"2024-09-19T06:01:39.980152+0000","last_clean":"2024-09-19T06:01:39.980152+0000","last_became_active":"2024-09-19T06:01:28.802587+0000","last_became_peered":"2024-09-19T06:01:28.802587+0000","last_unstale":"2024-09-19T06:01:39.980152+0000","last_undegraded":"2024-09-19T06:01:39.980152+0000","last_fullsized":"2024-09-19T06:01:39.980152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837492+0000","last_change":"2024-09-19T06:01:26.788140+0000","last_active":"2024-09-19T06:01:39.837492+0000","last_peered":"2024-09-19T06:01:39.837492+0000","last_clean":"2024-09-19T06:01:39.837492+0000","last_became_active":"2024-09-19T06:01:26.787603+0000","last_became_peered":"2024-09-19T06:01:26.787603+0000","last_unstale":"2024-09-19T06:01:39.837492+0000","last_undegraded":"2024-09-19T06:01:39.837492+0000","last_fullsized":"2024-09-19T06:01:39.837492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:13.880656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837369+0000","last_change":"2024-09-19T06:01:28.813144+0000","last_active":"2024-09-19T06:01:39.837369+0000","last_peered":"2024-09-19T06:01:39.837369+0000","last_clean":"2024-09-19T06:01:39.837369+0000","last_became_active":"2024-09-19T06:01:28.812792+0000","last_became_peered":"2024-09-19T06:01:28.812792+0000","last_unstale":"2024-09-19T06:01:39.837369+0000","last_undegraded":"2024-09-19T06:01:39.837369+0000","last_fullsized":"2024-09-19T06:01:39.837369+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:35.896240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833256+0000","last_change":"2024-09-19T06:01:28.812086+0000","last_active":"2024-09-19T06:01:39.833256+0000","last_peered":"2024-09-19T06:01:39.833256+0000","last_clean":"2024-09-19T06:01:39.833256+0000","last_became_active":"2024-09-19T06:01:28.811696+0000","last_became_peered":"2024-09-19T06:01:28.811696+0000","last_unstale":"2024-09-19T06:01:39.833256+0000","last_undegraded":"2024-09-19T06:01:39.833256+0000","last_fullsized":"2024-09-19T06:01:39.833256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:32.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847847+0000","last_change":"2024-09-19T06:01:39.842250+0000","last_active":"2024-09-19T06:01:40.847847+0000","last_peered":"2024-09-19T06:01:40.847847+0000","last_clean":"2024-09-19T06:01:40.847847+0000","last_became_active":"2024-09-19T06:01:39.835761+0000","last_became_peered":"2024-09-19T06:01:39.835761+0000","last_unstale":"2024-09-19T06:01:40.847847+0000","last_undegraded":"2024-09-19T06:01:40.847847+0000","last_fullsized":"2024-09-19T06:01:40.847847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:51.026349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832601+0000","last_change":"2024-09-19T06:01:28.812983+0000","last_active":"2024-09-19T06:01:39.832601+0000","last_peered":"2024-09-19T06:01:39.832601+0000","last_clean":"2024-09-19T06:01:39.832601+0000","last_became_active":"2024-09-19T06:01:28.812257+0000","last_became_peered":"2024-09-19T06:01:28.812257+0000","last_unstale":"2024-09-19T06:01:39.832601+0000","last_undegraded":"2024-09-19T06:01:39.832601+0000","last_fullsized":"2024-09-19T06:01:39.832601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:41:42.059736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:41.842632+0000","last_change":"2024-09-19T06:01:26.787869+0000","last_active":"2024-09-19T06:01:41.842632+0000","last_peered":"2024-09-19T06:01:41.842632+0000","last_clean":"2024-09-19T06:01:41.842632+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:01:41.842632+0000","last_undegraded":"2024-09-19T06:01:41.842632+0000","last_fullsized":"2024-09-19T06:01:41.842632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:02.869878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831993+0000","last_change":"2024-09-19T06:01:28.811986+0000","last_active":"2024-09-19T06:01:39.831993+0000","last_peered":"2024-09-19T06:01:39.831993+0000","last_clean":"2024-09-19T06:01:39.831993+0000","last_became_active":"2024-09-19T06:01:28.811565+0000","last_became_peered":"2024-09-19T06:01:28.811565+0000","last_unstale":"2024-09-19T06:01:39.831993+0000","last_undegraded":"2024-09-19T06:01:39.831993+0000","last_fullsized":"2024-09-19T06:01:39.831993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:01.324985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980467+0000","last_change":"2024-09-19T06:01:27.021683+0000","last_active":"2024-09-19T06:01:39.980467+0000","last_peered":"2024-09-19T06:01:39.980467+0000","last_clean":"2024-09-19T06:01:39.980467+0000","last_became_active":"2024-09-19T06:01:27.021358+0000","last_became_peered":"2024-09-19T06:01:27.021358+0000","last_unstale":"2024-09-19T06:01:39.980467+0000","last_undegraded":"2024-09-19T06:01:39.980467+0000","last_fullsized":"2024-09-19T06:01:39.980467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:29.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980328+0000","last_change":"2024-09-19T06:01:28.798963+0000","last_active":"2024-09-19T06:01:39.980328+0000","last_peered":"2024-09-19T06:01:39.980328+0000","last_clean":"2024-09-19T06:01:39.980328+0000","last_became_active":"2024-09-19T06:01:28.797571+0000","last_became_peered":"2024-09-19T06:01:28.797571+0000","last_unstale":"2024-09-19T06:01:39.980328+0000","last_undegraded":"2024-09-19T06:01:39.980328+0000","last_fullsized":"2024-09-19T06:01:39.980328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:43.867609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":84,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:27.910348+0000","last_change":"2024-09-19T06:01:26.786427+0000","last_active":"2024-09-19T06:02:27.910348+0000","last_peered":"2024-09-19T06:02:27.910348+0000","last_clean":"2024-09-19T06:02:27.910348+0000","last_became_active":"2024-09-19T06:01:26.786092+0000","last_became_peered":"2024-09-19T06:01:26.786092+0000","last_unstale":"2024-09-19T06:02:27.910348+0000","last_undegraded":"2024-09-19T06:02:27.910348+0000","last_fullsized":"2024-09-19T06:02:27.910348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:49.144366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69166,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":523,"num_read_kb":337,"num_write":245,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":439275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36920,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708232,"statfs":{"total":95995035648,"available":95957229568,"internally_reserved":0,"allocated":1613824,"data_stored":1090782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939684,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":438118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":437802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397549,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1031802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790007,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1030694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":439104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":437660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40468,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704684,"statfs":{"total":95995035648,"available":95953596416,"internally_reserved":0,"allocated":987136,"data_stored":498348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40472,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704680,"statfs":{"total":95995035648,"available":95953592320,"internally_reserved":0,"allocated":991232,"data_stored":501876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":439567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:02:34.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:02:34.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:34 smithi005 ceph-mon[30955]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:34.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:34 smithi005 ceph-mon[30955]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:02:34.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:34.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:34 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:34.394 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:34.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:34 smithi157 ceph-mon[35346]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:34.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:34 smithi157 ceph-mon[35346]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:02:34.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:34.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:34 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:34.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:34 smithi049 ceph-mon[34640]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:34.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:34 smithi049 ceph-mon[34640]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:02:34.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:34.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:34 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:34.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.856+0000 7f1b20466640 1 -- 172.21.15.5:0/471020933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b18102ef0 msgr2=0x7f1b1810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.856+0000 7f1b20466640 1 --2- 172.21.15.5:0/471020933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b18102ef0 0x7f1b1810da40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009ca0 tx=0x7f1b1002a950 comp rx=0 tx=0).stop 2024-09-19T06:02:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.856+0000 7f1b20466640 1 -- 172.21.15.5:0/471020933 shutdown_connections 2024-09-19T06:02:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.856+0000 7f1b20466640 1 --2- 172.21.15.5:0/471020933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b18102ef0 0x7f1b1810da40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.856+0000 7f1b20466640 1 --2- 172.21.15.5:0/471020933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b18102570 0x7f1b181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.856+0000 7f1b20466640 1 --2- 172.21.15.5:0/471020933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b18100e20 0x7f1b18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.856+0000 7f1b20466640 1 -- 172.21.15.5:0/471020933 >> 172.21.15.5:0/471020933 conn(0x7f1b180fc7d0 msgr2=0x7f1b180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.856+0000 7f1b20466640 1 -- 172.21.15.5:0/471020933 shutdown_connections 2024-09-19T06:02:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.856+0000 7f1b20466640 1 -- 172.21.15.5:0/471020933 wait complete. 2024-09-19T06:02:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.857+0000 7f1b20466640 1 Processor -- start 2024-09-19T06:02:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.857+0000 7f1b20466640 1 -- start start 2024-09-19T06:02:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.857+0000 7f1b20466640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b18100e20 0x7f1b181983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.857+0000 7f1b20466640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b18102570 0x7f1b18198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.857+0000 7f1b20466640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b18102ef0 0x7f1b1819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.857+0000 7f1b20466640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b18110e30 con 0x7f1b18100e20 2024-09-19T06:02:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.857+0000 7f1b20466640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b18110cb0 con 0x7f1b18102ef0 2024-09-19T06:02:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.857+0000 7f1b20466640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1b18110fb0 con 0x7f1b18102570 2024-09-19T06:02:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1da01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b18102570 0x7f1b18198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1e202640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b18100e20 0x7f1b181983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1da01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b18102570 0x7f1b18198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59284/0 (socket says 172.21.15.5:59284) 2024-09-19T06:02:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1e202640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b18100e20 0x7f1b181983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49912/0 (socket says 172.21.15.5:49912) 2024-09-19T06:02:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1da01640 1 -- 172.21.15.5:0/2757405390 learned_addr learned my addr 172.21.15.5:0/2757405390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1ea03640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b18102ef0 0x7f1b1819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1e202640 1 -- 172.21.15.5:0/2757405390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b18102570 msgr2=0x7f1b18198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1e202640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b18102570 0x7f1b18198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1e202640 1 -- 172.21.15.5:0/2757405390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b18102ef0 msgr2=0x7f1b1819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1e202640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b18102ef0 0x7f1b1819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1e202640 1 -- 172.21.15.5:0/2757405390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1819d370 con 0x7f1b18100e20 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1da01640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b18102570 0x7f1b18198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.858+0000 7f1b1e202640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b18100e20 0x7f1b181983c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400e9f0 tx=0x7f1b1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.859+0000 7f1b0f7fe640 1 -- 172.21.15.5:0/2757405390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1400ccc0 con 0x7f1b18100e20 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.859+0000 7f1b0f7fe640 1 -- 172.21.15.5:0/2757405390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b140108e0 con 0x7f1b18100e20 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.859+0000 7f1b20466640 1 -- 172.21.15.5:0/2757405390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1819d660 con 0x7f1b18100e20 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.859+0000 7f1b0f7fe640 1 -- 172.21.15.5:0/2757405390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14018880 con 0x7f1b18100e20 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.859+0000 7f1b20466640 1 -- 172.21.15.5:0/2757405390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b181a4f40 con 0x7f1b18100e20 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.860+0000 7f1b20466640 1 -- 172.21.15.5:0/2757405390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b18101200 con 0x7f1b18100e20 2024-09-19T06:02:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.861+0000 7f1b0f7fe640 1 -- 172.21.15.5:0/2757405390 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f1b14010a90 con 0x7f1b18100e20 2024-09-19T06:02:34.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.862+0000 7f1b0f7fe640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f1b040a3350 0x7f1b040a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:34.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.862+0000 7f1b1da01640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f1b040a3350 0x7f1b040a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:34.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.862+0000 7f1b0f7fe640 1 -- 172.21.15.5:0/2757405390 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1b140cfd60 con 0x7f1b18100e20 2024-09-19T06:02:34.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.862+0000 7f1b1da01640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f1b040a3350 0x7f1b040a5810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1b18100050 tx=0x7f1b08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:34.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:34.866+0000 7f1b0f7fe640 1 -- 172.21.15.5:0/2757405390 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1b1409c2c0 con 0x7f1b18100e20 2024-09-19T06:02:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.043+0000 7f1b20466640 1 -- 172.21.15.5:0/2757405390 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1b18062630 con 0x7f1b040a3350 2024-09-19T06:02:35.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.050+0000 7f1b0f7fe640 1 -- 172.21.15.5:0/2757405390 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258751 (secure 0 0 0) 0x7f1b18062630 con 0x7f1b040a3350 2024-09-19T06:02:35.054 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-19T06:02:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.052+0000 7f1b20466640 1 -- 172.21.15.5:0/2757405390 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f1b040a3350 msgr2=0x7f1b040a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.052+0000 7f1b20466640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f1b040a3350 0x7f1b040a5810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1b18100050 tx=0x7f1b08005f90 comp rx=0 tx=0).stop 2024-09-19T06:02:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.052+0000 7f1b20466640 1 -- 172.21.15.5:0/2757405390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b18100e20 msgr2=0x7f1b181983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.052+0000 7f1b20466640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b18100e20 0x7f1b181983c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400e9f0 tx=0x7f1b1400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:02:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.052+0000 7f1b20466640 1 -- 172.21.15.5:0/2757405390 shutdown_connections 2024-09-19T06:02:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.052+0000 7f1b20466640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f1b040a3350 0x7f1b040a5810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.052+0000 7f1b20466640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b18102ef0 0x7f1b1819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.052+0000 7f1b20466640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b18102570 0x7f1b18198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.052+0000 7f1b20466640 1 --2- 172.21.15.5:0/2757405390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b18100e20 0x7f1b181983c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.052+0000 7f1b20466640 1 -- 172.21.15.5:0/2757405390 >> 172.21.15.5:0/2757405390 conn(0x7f1b180fc7d0 msgr2=0x7f1b1806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.052+0000 7f1b20466640 1 -- 172.21.15.5:0/2757405390 shutdown_connections 2024-09-19T06:02:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:35.052+0000 7f1b20466640 1 -- 172.21.15.5:0/2757405390 wait complete. 2024-09-19T06:02:35.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:02:35.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:02:35.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:35 smithi005 ceph-mon[30955]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:35.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:02:35.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:02:35.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:35 smithi157 ceph-mon[35346]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:35.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:02:35.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:02:35.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:35 smithi049 ceph-mon[34640]: from='mgr.14150 172.21.15.5:0/1938358312' entity='mgr.x' 2024-09-19T06:02:35.705 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-19T06:02:34.799314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659768,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":704,"num_read_kb":481,"num_write":428,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387100,"kb_used_data":12876,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554724,"statfs":{"total":1151940427776,"available":1151544037376,"internally_reserved":0,"allocated":13185024,"data_stored":7223256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002911"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833240+0000","last_change":"2024-09-19T06:01:28.812722+0000","last_active":"2024-09-19T06:01:39.833240+0000","last_peered":"2024-09-19T06:01:39.833240+0000","last_clean":"2024-09-19T06:01:39.833240+0000","last_became_active":"2024-09-19T06:01:28.812283+0000","last_became_peered":"2024-09-19T06:01:28.812283+0000","last_unstale":"2024-09-19T06:01:39.833240+0000","last_undegraded":"2024-09-19T06:01:39.833240+0000","last_fullsized":"2024-09-19T06:01:39.833240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:59.146636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831968+0000","last_change":"2024-09-19T06:01:28.805210+0000","last_active":"2024-09-19T06:01:39.831968+0000","last_peered":"2024-09-19T06:01:39.831968+0000","last_clean":"2024-09-19T06:01:39.831968+0000","last_became_active":"2024-09-19T06:01:28.804906+0000","last_became_peered":"2024-09-19T06:01:28.804906+0000","last_unstale":"2024-09-19T06:01:39.831968+0000","last_undegraded":"2024-09-19T06:01:39.831968+0000","last_fullsized":"2024-09-19T06:01:39.831968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:00.227101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832025+0000","last_change":"2024-09-19T06:01:28.811909+0000","last_active":"2024-09-19T06:01:39.832025+0000","last_peered":"2024-09-19T06:01:39.832025+0000","last_clean":"2024-09-19T06:01:39.832025+0000","last_became_active":"2024-09-19T06:01:28.811547+0000","last_became_peered":"2024-09-19T06:01:28.811547+0000","last_unstale":"2024-09-19T06:01:39.832025+0000","last_undegraded":"2024-09-19T06:01:39.832025+0000","last_fullsized":"2024-09-19T06:01:39.832025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:50.022439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830196+0000","last_change":"2024-09-19T06:01:28.806782+0000","last_active":"2024-09-19T06:01:39.830196+0000","last_peered":"2024-09-19T06:01:39.830196+0000","last_clean":"2024-09-19T06:01:39.830196+0000","last_became_active":"2024-09-19T06:01:28.805520+0000","last_became_peered":"2024-09-19T06:01:28.805520+0000","last_unstale":"2024-09-19T06:01:39.830196+0000","last_undegraded":"2024-09-19T06:01:39.830196+0000","last_fullsized":"2024-09-19T06:01:39.830196+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:01.652669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837308+0000","last_change":"2024-09-19T06:01:28.809971+0000","last_active":"2024-09-19T06:01:39.837308+0000","last_peered":"2024-09-19T06:01:39.837308+0000","last_clean":"2024-09-19T06:01:39.837308+0000","last_became_active":"2024-09-19T06:01:28.809593+0000","last_became_peered":"2024-09-19T06:01:28.809593+0000","last_unstale":"2024-09-19T06:01:39.837308+0000","last_undegraded":"2024-09-19T06:01:39.837308+0000","last_fullsized":"2024-09-19T06:01:39.837308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.546526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323650+0000","last_change":"2024-09-19T06:01:28.801395+0000","last_active":"2024-09-19T06:01:40.323650+0000","last_peered":"2024-09-19T06:01:40.323650+0000","last_clean":"2024-09-19T06:01:40.323650+0000","last_became_active":"2024-09-19T06:01:28.800987+0000","last_became_peered":"2024-09-19T06:01:28.800987+0000","last_unstale":"2024-09-19T06:01:40.323650+0000","last_undegraded":"2024-09-19T06:01:40.323650+0000","last_fullsized":"2024-09-19T06:01:40.323650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:18.986489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837262+0000","last_change":"2024-09-19T06:01:28.806554+0000","last_active":"2024-09-19T06:01:39.837262+0000","last_peered":"2024-09-19T06:01:39.837262+0000","last_clean":"2024-09-19T06:01:39.837262+0000","last_became_active":"2024-09-19T06:01:28.805946+0000","last_became_peered":"2024-09-19T06:01:28.805946+0000","last_unstale":"2024-09-19T06:01:39.837262+0000","last_undegraded":"2024-09-19T06:01:39.837262+0000","last_fullsized":"2024-09-19T06:01:39.837262+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:56:20.130414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.979824+0000","last_change":"2024-09-19T06:01:28.804789+0000","last_active":"2024-09-19T06:01:39.979824+0000","last_peered":"2024-09-19T06:01:39.979824+0000","last_clean":"2024-09-19T06:01:39.979824+0000","last_became_active":"2024-09-19T06:01:28.804484+0000","last_became_peered":"2024-09-19T06:01:28.804484+0000","last_unstale":"2024-09-19T06:01:39.979824+0000","last_undegraded":"2024-09-19T06:01:39.979824+0000","last_fullsized":"2024-09-19T06:01:39.979824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:19.737928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832749+0000","last_change":"2024-09-19T06:01:28.814504+0000","last_active":"2024-09-19T06:01:39.832749+0000","last_peered":"2024-09-19T06:01:39.832749+0000","last_clean":"2024-09-19T06:01:39.832749+0000","last_became_active":"2024-09-19T06:01:28.814066+0000","last_became_peered":"2024-09-19T06:01:28.814066+0000","last_unstale":"2024-09-19T06:01:39.832749+0000","last_undegraded":"2024-09-19T06:01:39.832749+0000","last_fullsized":"2024-09-19T06:01:39.832749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:50.904164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.324053+0000","last_change":"2024-09-19T06:01:28.807303+0000","last_active":"2024-09-19T06:01:40.324053+0000","last_peered":"2024-09-19T06:01:40.324053+0000","last_clean":"2024-09-19T06:01:40.324053+0000","last_became_active":"2024-09-19T06:01:28.806978+0000","last_became_peered":"2024-09-19T06:01:28.806978+0000","last_unstale":"2024-09-19T06:01:40.324053+0000","last_undegraded":"2024-09-19T06:01:40.324053+0000","last_fullsized":"2024-09-19T06:01:40.324053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:33.259261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837723+0000","last_change":"2024-09-19T06:01:28.814844+0000","last_active":"2024-09-19T06:01:39.837723+0000","last_peered":"2024-09-19T06:01:39.837723+0000","last_clean":"2024-09-19T06:01:39.837723+0000","last_became_active":"2024-09-19T06:01:28.814531+0000","last_became_peered":"2024-09-19T06:01:28.814531+0000","last_unstale":"2024-09-19T06:01:39.837723+0000","last_undegraded":"2024-09-19T06:01:39.837723+0000","last_fullsized":"2024-09-19T06:01:39.837723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:17.209890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830253+0000","last_change":"2024-09-19T06:01:28.806378+0000","last_active":"2024-09-19T06:01:39.830253+0000","last_peered":"2024-09-19T06:01:39.830253+0000","last_clean":"2024-09-19T06:01:39.830253+0000","last_became_active":"2024-09-19T06:01:28.805053+0000","last_became_peered":"2024-09-19T06:01:28.805053+0000","last_unstale":"2024-09-19T06:01:39.830253+0000","last_undegraded":"2024-09-19T06:01:39.830253+0000","last_fullsized":"2024-09-19T06:01:39.830253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:15.080548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829093+0000","last_change":"2024-09-19T06:01:28.802941+0000","last_active":"2024-09-19T06:01:39.829093+0000","last_peered":"2024-09-19T06:01:39.829093+0000","last_clean":"2024-09-19T06:01:39.829093+0000","last_became_active":"2024-09-19T06:01:28.802590+0000","last_became_peered":"2024-09-19T06:01:28.802590+0000","last_unstale":"2024-09-19T06:01:39.829093+0000","last_undegraded":"2024-09-19T06:01:39.829093+0000","last_fullsized":"2024-09-19T06:01:39.829093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:25.581854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831644+0000","last_change":"2024-09-19T06:01:28.800960+0000","last_active":"2024-09-19T06:01:39.831644+0000","last_peered":"2024-09-19T06:01:39.831644+0000","last_clean":"2024-09-19T06:01:39.831644+0000","last_became_active":"2024-09-19T06:01:28.800484+0000","last_became_peered":"2024-09-19T06:01:28.800484+0000","last_unstale":"2024-09-19T06:01:39.831644+0000","last_undegraded":"2024-09-19T06:01:39.831644+0000","last_fullsized":"2024-09-19T06:01:39.831644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:08.506105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831592+0000","last_change":"2024-09-19T06:01:28.800771+0000","last_active":"2024-09-19T06:01:39.831592+0000","last_peered":"2024-09-19T06:01:39.831592+0000","last_clean":"2024-09-19T06:01:39.831592+0000","last_became_active":"2024-09-19T06:01:28.800427+0000","last_became_peered":"2024-09-19T06:01:28.800427+0000","last_unstale":"2024-09-19T06:01:39.831592+0000","last_undegraded":"2024-09-19T06:01:39.831592+0000","last_fullsized":"2024-09-19T06:01:39.831592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:19.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831899+0000","last_change":"2024-09-19T06:01:28.815528+0000","last_active":"2024-09-19T06:01:39.831899+0000","last_peered":"2024-09-19T06:01:39.831899+0000","last_clean":"2024-09-19T06:01:39.831899+0000","last_became_active":"2024-09-19T06:01:28.815129+0000","last_became_peered":"2024-09-19T06:01:28.815129+0000","last_unstale":"2024-09-19T06:01:39.831899+0000","last_undegraded":"2024-09-19T06:01:39.831899+0000","last_fullsized":"2024-09-19T06:01:39.831899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:01.936711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833355+0000","last_change":"2024-09-19T06:01:28.816120+0000","last_active":"2024-09-19T06:01:39.833355+0000","last_peered":"2024-09-19T06:01:39.833355+0000","last_clean":"2024-09-19T06:01:39.833355+0000","last_became_active":"2024-09-19T06:01:28.815822+0000","last_became_peered":"2024-09-19T06:01:28.815822+0000","last_unstale":"2024-09-19T06:01:39.833355+0000","last_undegraded":"2024-09-19T06:01:39.833355+0000","last_fullsized":"2024-09-19T06:01:39.833355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:14:58.670452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.836100+0000","last_change":"2024-09-19T06:01:28.809825+0000","last_active":"2024-09-19T06:01:39.836100+0000","last_peered":"2024-09-19T06:01:39.836100+0000","last_clean":"2024-09-19T06:01:39.836100+0000","last_became_active":"2024-09-19T06:01:28.809530+0000","last_became_peered":"2024-09-19T06:01:28.809530+0000","last_unstale":"2024-09-19T06:01:39.836100+0000","last_undegraded":"2024-09-19T06:01:39.836100+0000","last_fullsized":"2024-09-19T06:01:39.836100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:49:23.442436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830644+0000","last_change":"2024-09-19T06:01:28.801297+0000","last_active":"2024-09-19T06:01:39.830644+0000","last_peered":"2024-09-19T06:01:39.830644+0000","last_clean":"2024-09-19T06:01:39.830644+0000","last_became_active":"2024-09-19T06:01:28.800984+0000","last_became_peered":"2024-09-19T06:01:28.800984+0000","last_unstale":"2024-09-19T06:01:39.830644+0000","last_undegraded":"2024-09-19T06:01:39.830644+0000","last_fullsized":"2024-09-19T06:01:39.830644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:01.485951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832407+0000","last_change":"2024-09-19T06:01:28.804567+0000","last_active":"2024-09-19T06:01:39.832407+0000","last_peered":"2024-09-19T06:01:39.832407+0000","last_clean":"2024-09-19T06:01:39.832407+0000","last_became_active":"2024-09-19T06:01:28.804213+0000","last_became_peered":"2024-09-19T06:01:28.804213+0000","last_unstale":"2024-09-19T06:01:39.832407+0000","last_undegraded":"2024-09-19T06:01:39.832407+0000","last_fullsized":"2024-09-19T06:01:39.832407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.186460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829142+0000","last_change":"2024-09-19T06:01:28.801961+0000","last_active":"2024-09-19T06:01:39.829142+0000","last_peered":"2024-09-19T06:01:39.829142+0000","last_clean":"2024-09-19T06:01:39.829142+0000","last_became_active":"2024-09-19T06:01:28.801631+0000","last_became_peered":"2024-09-19T06:01:28.801631+0000","last_unstale":"2024-09-19T06:01:39.829142+0000","last_undegraded":"2024-09-19T06:01:39.829142+0000","last_fullsized":"2024-09-19T06:01:39.829142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:15.628908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832042+0000","last_change":"2024-09-19T06:01:28.815160+0000","last_active":"2024-09-19T06:01:39.832042+0000","last_peered":"2024-09-19T06:01:39.832042+0000","last_clean":"2024-09-19T06:01:39.832042+0000","last_became_active":"2024-09-19T06:01:28.813329+0000","last_became_peered":"2024-09-19T06:01:28.813329+0000","last_unstale":"2024-09-19T06:01:39.832042+0000","last_undegraded":"2024-09-19T06:01:39.832042+0000","last_fullsized":"2024-09-19T06:01:39.832042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:17.027393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.979874+0000","last_change":"2024-09-19T06:01:28.799205+0000","last_active":"2024-09-19T06:01:39.979874+0000","last_peered":"2024-09-19T06:01:39.979874+0000","last_clean":"2024-09-19T06:01:39.979874+0000","last_became_active":"2024-09-19T06:01:28.798891+0000","last_became_peered":"2024-09-19T06:01:28.798891+0000","last_unstale":"2024-09-19T06:01:39.979874+0000","last_undegraded":"2024-09-19T06:01:39.979874+0000","last_fullsized":"2024-09-19T06:01:39.979874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:41.227721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832707+0000","last_change":"2024-09-19T06:01:28.795823+0000","last_active":"2024-09-19T06:01:39.832707+0000","last_peered":"2024-09-19T06:01:39.832707+0000","last_clean":"2024-09-19T06:01:39.832707+0000","last_became_active":"2024-09-19T06:01:28.795510+0000","last_became_peered":"2024-09-19T06:01:28.795510+0000","last_unstale":"2024-09-19T06:01:39.832707+0000","last_undegraded":"2024-09-19T06:01:39.832707+0000","last_fullsized":"2024-09-19T06:01:39.832707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:54.831610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980449+0000","last_change":"2024-09-19T06:01:28.799696+0000","last_active":"2024-09-19T06:01:39.980449+0000","last_peered":"2024-09-19T06:01:39.980449+0000","last_clean":"2024-09-19T06:01:39.980449+0000","last_became_active":"2024-09-19T06:01:28.799417+0000","last_became_peered":"2024-09-19T06:01:28.799417+0000","last_unstale":"2024-09-19T06:01:39.980449+0000","last_undegraded":"2024-09-19T06:01:39.980449+0000","last_fullsized":"2024-09-19T06:01:39.980449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:02.330808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830714+0000","last_change":"2024-09-19T06:01:28.807146+0000","last_active":"2024-09-19T06:01:39.830714+0000","last_peered":"2024-09-19T06:01:39.830714+0000","last_clean":"2024-09-19T06:01:39.830714+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:01:39.830714+0000","last_undegraded":"2024-09-19T06:01:39.830714+0000","last_fullsized":"2024-09-19T06:01:39.830714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:12.308421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832819+0000","last_change":"2024-09-19T06:01:28.803555+0000","last_active":"2024-09-19T06:01:39.832819+0000","last_peered":"2024-09-19T06:01:39.832819+0000","last_clean":"2024-09-19T06:01:39.832819+0000","last_became_active":"2024-09-19T06:01:28.803242+0000","last_became_peered":"2024-09-19T06:01:28.803242+0000","last_unstale":"2024-09-19T06:01:39.832819+0000","last_undegraded":"2024-09-19T06:01:39.832819+0000","last_fullsized":"2024-09-19T06:01:39.832819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:02.281152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829217+0000","last_change":"2024-09-19T06:01:28.804378+0000","last_active":"2024-09-19T06:01:39.829217+0000","last_peered":"2024-09-19T06:01:39.829217+0000","last_clean":"2024-09-19T06:01:39.829217+0000","last_became_active":"2024-09-19T06:01:28.804099+0000","last_became_peered":"2024-09-19T06:01:28.804099+0000","last_unstale":"2024-09-19T06:01:39.829217+0000","last_undegraded":"2024-09-19T06:01:39.829217+0000","last_fullsized":"2024-09-19T06:01:39.829217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:11.558384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831784+0000","last_change":"2024-09-19T06:01:28.799999+0000","last_active":"2024-09-19T06:01:39.831784+0000","last_peered":"2024-09-19T06:01:39.831784+0000","last_clean":"2024-09-19T06:01:39.831784+0000","last_became_active":"2024-09-19T06:01:28.799645+0000","last_became_peered":"2024-09-19T06:01:28.799645+0000","last_unstale":"2024-09-19T06:01:39.831784+0000","last_undegraded":"2024-09-19T06:01:39.831784+0000","last_fullsized":"2024-09-19T06:01:39.831784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:20.442119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830148+0000","last_change":"2024-09-19T06:01:28.800143+0000","last_active":"2024-09-19T06:01:39.830148+0000","last_peered":"2024-09-19T06:01:39.830148+0000","last_clean":"2024-09-19T06:01:39.830148+0000","last_became_active":"2024-09-19T06:01:28.799830+0000","last_became_peered":"2024-09-19T06:01:28.799830+0000","last_unstale":"2024-09-19T06:01:39.830148+0000","last_undegraded":"2024-09-19T06:01:39.830148+0000","last_fullsized":"2024-09-19T06:01:39.830148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:43.824740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830661+0000","last_change":"2024-09-19T06:01:28.806130+0000","last_active":"2024-09-19T06:01:39.830661+0000","last_peered":"2024-09-19T06:01:39.830661+0000","last_clean":"2024-09-19T06:01:39.830661+0000","last_became_active":"2024-09-19T06:01:28.805772+0000","last_became_peered":"2024-09-19T06:01:28.805772+0000","last_unstale":"2024-09-19T06:01:39.830661+0000","last_undegraded":"2024-09-19T06:01:39.830661+0000","last_fullsized":"2024-09-19T06:01:39.830661+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:00.316311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837412+0000","last_change":"2024-09-19T06:01:39.837412+0000","last_active":"2024-09-19T06:01:39.837412+0000","last_peered":"2024-09-19T06:01:39.837412+0000","last_clean":"2024-09-19T06:01:39.837412+0000","last_became_active":"2024-09-19T06:01:39.837035+0000","last_became_peered":"2024-09-19T06:01:39.837035+0000","last_unstale":"2024-09-19T06:01:39.837412+0000","last_undegraded":"2024-09-19T06:01:39.837412+0000","last_fullsized":"2024-09-19T06:01:39.837412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:00.073884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837634+0000","last_change":"2024-09-19T06:01:28.800324+0000","last_active":"2024-09-19T06:01:39.837634+0000","last_peered":"2024-09-19T06:01:39.837634+0000","last_clean":"2024-09-19T06:01:39.837634+0000","last_became_active":"2024-09-19T06:01:28.799986+0000","last_became_peered":"2024-09-19T06:01:28.799986+0000","last_unstale":"2024-09-19T06:01:39.837634+0000","last_undegraded":"2024-09-19T06:01:39.837634+0000","last_fullsized":"2024-09-19T06:01:39.837634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:41.963346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":95,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:31.918272+0000","last_change":"2024-09-19T06:01:39.844897+0000","last_active":"2024-09-19T06:02:31.918272+0000","last_peered":"2024-09-19T06:02:31.918272+0000","last_clean":"2024-09-19T06:02:31.918272+0000","last_became_active":"2024-09-19T06:01:39.838347+0000","last_became_peered":"2024-09-19T06:01:39.838347+0000","last_unstale":"2024-09-19T06:02:31.918272+0000","last_undegraded":"2024-09-19T06:02:31.918272+0000","last_fullsized":"2024-09-19T06:02:31.918272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:26.235227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.834624+0000","last_change":"2024-09-19T06:01:28.802644+0000","last_active":"2024-09-19T06:01:39.834624+0000","last_peered":"2024-09-19T06:01:39.834624+0000","last_clean":"2024-09-19T06:01:39.834624+0000","last_became_active":"2024-09-19T06:01:28.802288+0000","last_became_peered":"2024-09-19T06:01:28.802288+0000","last_unstale":"2024-09-19T06:01:39.834624+0000","last_undegraded":"2024-09-19T06:01:39.834624+0000","last_fullsized":"2024-09-19T06:01:39.834624+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:32.031860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323748+0000","last_change":"2024-09-19T06:01:26.781104+0000","last_active":"2024-09-19T06:01:40.323748+0000","last_peered":"2024-09-19T06:01:40.323748+0000","last_clean":"2024-09-19T06:01:40.323748+0000","last_became_active":"2024-09-19T06:01:26.780733+0000","last_became_peered":"2024-09-19T06:01:26.780733+0000","last_unstale":"2024-09-19T06:01:40.323748+0000","last_undegraded":"2024-09-19T06:01:40.323748+0000","last_fullsized":"2024-09-19T06:01:40.323748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:54.490867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.828967+0000","last_change":"2024-09-19T06:01:26.782901+0000","last_active":"2024-09-19T06:01:39.828967+0000","last_peered":"2024-09-19T06:01:39.828967+0000","last_clean":"2024-09-19T06:01:39.828967+0000","last_became_active":"2024-09-19T06:01:26.782450+0000","last_became_peered":"2024-09-19T06:01:26.782450+0000","last_unstale":"2024-09-19T06:01:39.828967+0000","last_undegraded":"2024-09-19T06:01:39.828967+0000","last_fullsized":"2024-09-19T06:01:39.828967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:07.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.979905+0000","last_change":"2024-09-19T06:01:28.804105+0000","last_active":"2024-09-19T06:01:39.979905+0000","last_peered":"2024-09-19T06:01:39.979905+0000","last_clean":"2024-09-19T06:01:39.979905+0000","last_became_active":"2024-09-19T06:01:28.797980+0000","last_became_peered":"2024-09-19T06:01:28.797980+0000","last_unstale":"2024-09-19T06:01:39.979905+0000","last_undegraded":"2024-09-19T06:01:39.979905+0000","last_fullsized":"2024-09-19T06:01:39.979905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:50.130102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.848777+0000","last_change":"2024-09-19T06:01:26.786923+0000","last_active":"2024-09-19T06:01:40.848777+0000","last_peered":"2024-09-19T06:01:40.848777+0000","last_clean":"2024-09-19T06:01:40.848777+0000","last_became_active":"2024-09-19T06:01:26.786542+0000","last_became_peered":"2024-09-19T06:01:26.786542+0000","last_unstale":"2024-09-19T06:01:40.848777+0000","last_undegraded":"2024-09-19T06:01:40.848777+0000","last_fullsized":"2024-09-19T06:01:40.848777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:30.032992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.835994+0000","last_change":"2024-09-19T06:01:28.802172+0000","last_active":"2024-09-19T06:01:39.835994+0000","last_peered":"2024-09-19T06:01:39.835994+0000","last_clean":"2024-09-19T06:01:39.835994+0000","last_became_active":"2024-09-19T06:01:28.801815+0000","last_became_peered":"2024-09-19T06:01:28.801815+0000","last_unstale":"2024-09-19T06:01:39.835994+0000","last_undegraded":"2024-09-19T06:01:39.835994+0000","last_fullsized":"2024-09-19T06:01:39.835994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:09.587638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:33.917138+0000","last_change":"2024-09-19T06:01:26.777870+0000","last_active":"2024-09-19T06:02:33.917138+0000","last_peered":"2024-09-19T06:02:33.917138+0000","last_clean":"2024-09-19T06:02:33.917138+0000","last_became_active":"2024-09-19T06:01:26.777509+0000","last_became_peered":"2024-09-19T06:01:26.777509+0000","last_unstale":"2024-09-19T06:02:33.917138+0000","last_undegraded":"2024-09-19T06:02:33.917138+0000","last_fullsized":"2024-09-19T06:02:33.917138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:15.885911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832446+0000","last_change":"2024-09-19T06:01:28.812201+0000","last_active":"2024-09-19T06:01:39.832446+0000","last_peered":"2024-09-19T06:01:39.832446+0000","last_clean":"2024-09-19T06:01:39.832446+0000","last_became_active":"2024-09-19T06:01:28.811801+0000","last_became_peered":"2024-09-19T06:01:28.811801+0000","last_unstale":"2024-09-19T06:01:39.832446+0000","last_undegraded":"2024-09-19T06:01:39.832446+0000","last_fullsized":"2024-09-19T06:01:39.832446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:32.904787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.848667+0000","last_change":"2024-09-19T06:01:26.787845+0000","last_active":"2024-09-19T06:01:40.848667+0000","last_peered":"2024-09-19T06:01:40.848667+0000","last_clean":"2024-09-19T06:01:40.848667+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:01:40.848667+0000","last_undegraded":"2024-09-19T06:01:40.848667+0000","last_fullsized":"2024-09-19T06:01:40.848667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.114865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832096+0000","last_change":"2024-09-19T06:01:28.801523+0000","last_active":"2024-09-19T06:01:39.832096+0000","last_peered":"2024-09-19T06:01:39.832096+0000","last_clean":"2024-09-19T06:01:39.832096+0000","last_became_active":"2024-09-19T06:01:28.801205+0000","last_became_peered":"2024-09-19T06:01:28.801205+0000","last_unstale":"2024-09-19T06:01:39.832096+0000","last_undegraded":"2024-09-19T06:01:39.832096+0000","last_fullsized":"2024-09-19T06:01:39.832096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:10.314085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833424+0000","last_change":"2024-09-19T06:01:27.022172+0000","last_active":"2024-09-19T06:01:39.833424+0000","last_peered":"2024-09-19T06:01:39.833424+0000","last_clean":"2024-09-19T06:01:39.833424+0000","last_became_active":"2024-09-19T06:01:27.021568+0000","last_became_peered":"2024-09-19T06:01:27.021568+0000","last_unstale":"2024-09-19T06:01:39.833424+0000","last_undegraded":"2024-09-19T06:01:39.833424+0000","last_fullsized":"2024-09-19T06:01:39.833424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:18.031690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.839061+0000","last_change":"2024-09-19T06:01:39.839061+0000","last_active":"2024-09-19T06:01:39.839061+0000","last_peered":"2024-09-19T06:01:39.839061+0000","last_clean":"2024-09-19T06:01:39.839061+0000","last_became_active":"2024-09-19T06:01:39.838644+0000","last_became_peered":"2024-09-19T06:01:39.838644+0000","last_unstale":"2024-09-19T06:01:39.839061+0000","last_undegraded":"2024-09-19T06:01:39.839061+0000","last_fullsized":"2024-09-19T06:01:39.839061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:02.136222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:33.919803+0000","last_change":"2024-09-19T06:01:26.784781+0000","last_active":"2024-09-19T06:02:33.919803+0000","last_peered":"2024-09-19T06:02:33.919803+0000","last_clean":"2024-09-19T06:02:33.919803+0000","last_became_active":"2024-09-19T06:01:26.784284+0000","last_became_peered":"2024-09-19T06:01:26.784284+0000","last_unstale":"2024-09-19T06:02:33.919803+0000","last_undegraded":"2024-09-19T06:02:33.919803+0000","last_fullsized":"2024-09-19T06:02:33.919803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:13.063299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61172,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":116,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.839724+0000","last_change":"2024-09-19T06:01:28.802604+0000","last_active":"2024-09-19T06:01:39.839724+0000","last_peered":"2024-09-19T06:01:39.839724+0000","last_clean":"2024-09-19T06:01:39.839724+0000","last_became_active":"2024-09-19T06:01:28.802285+0000","last_became_peered":"2024-09-19T06:01:28.802285+0000","last_unstale":"2024-09-19T06:01:39.839724+0000","last_undegraded":"2024-09-19T06:01:39.839724+0000","last_fullsized":"2024-09-19T06:01:39.839724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:57.177791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:41.846861+0000","last_change":"2024-09-19T06:01:26.783678+0000","last_active":"2024-09-19T06:01:41.846861+0000","last_peered":"2024-09-19T06:01:41.846861+0000","last_clean":"2024-09-19T06:01:41.846861+0000","last_became_active":"2024-09-19T06:01:26.783165+0000","last_became_peered":"2024-09-19T06:01:26.783165+0000","last_unstale":"2024-09-19T06:01:41.846861+0000","last_undegraded":"2024-09-19T06:01:41.846861+0000","last_fullsized":"2024-09-19T06:01:41.846861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:50.535733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.828953+0000","last_change":"2024-09-19T06:01:28.808553+0000","last_active":"2024-09-19T06:01:39.828953+0000","last_peered":"2024-09-19T06:01:39.828953+0000","last_clean":"2024-09-19T06:01:39.828953+0000","last_became_active":"2024-09-19T06:01:28.808215+0000","last_became_peered":"2024-09-19T06:01:28.808215+0000","last_unstale":"2024-09-19T06:01:39.828953+0000","last_undegraded":"2024-09-19T06:01:39.828953+0000","last_fullsized":"2024-09-19T06:01:39.828953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:55.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.848791+0000","last_change":"2024-09-19T06:01:26.787137+0000","last_active":"2024-09-19T06:01:40.848791+0000","last_peered":"2024-09-19T06:01:40.848791+0000","last_clean":"2024-09-19T06:01:40.848791+0000","last_became_active":"2024-09-19T06:01:26.786676+0000","last_became_peered":"2024-09-19T06:01:26.786676+0000","last_unstale":"2024-09-19T06:01:40.848791+0000","last_undegraded":"2024-09-19T06:01:40.848791+0000","last_fullsized":"2024-09-19T06:01:40.848791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.609969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837656+0000","last_change":"2024-09-19T06:01:28.812106+0000","last_active":"2024-09-19T06:01:39.837656+0000","last_peered":"2024-09-19T06:01:39.837656+0000","last_clean":"2024-09-19T06:01:39.837656+0000","last_became_active":"2024-09-19T06:01:28.811771+0000","last_became_peered":"2024-09-19T06:01:28.811771+0000","last_unstale":"2024-09-19T06:01:39.837656+0000","last_undegraded":"2024-09-19T06:01:39.837656+0000","last_fullsized":"2024-09-19T06:01:39.837656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:07:06.832685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":80,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:28.911036+0000","last_change":"2024-09-19T06:01:26.786746+0000","last_active":"2024-09-19T06:02:28.911036+0000","last_peered":"2024-09-19T06:02:28.911036+0000","last_clean":"2024-09-19T06:02:28.911036+0000","last_became_active":"2024-09-19T06:01:26.786414+0000","last_became_peered":"2024-09-19T06:01:26.786414+0000","last_unstale":"2024-09-19T06:02:28.911036+0000","last_undegraded":"2024-09-19T06:02:28.911036+0000","last_fullsized":"2024-09-19T06:02:28.911036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:25.042796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837014+0000","last_change":"2024-09-19T06:01:28.807183+0000","last_active":"2024-09-19T06:01:39.837014+0000","last_peered":"2024-09-19T06:01:39.837014+0000","last_clean":"2024-09-19T06:01:39.837014+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:01:39.837014+0000","last_undegraded":"2024-09-19T06:01:39.837014+0000","last_fullsized":"2024-09-19T06:01:39.837014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:23.030312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847960+0000","last_change":"2024-09-19T06:01:27.022409+0000","last_active":"2024-09-19T06:01:40.847960+0000","last_peered":"2024-09-19T06:01:40.847960+0000","last_clean":"2024-09-19T06:01:40.847960+0000","last_became_active":"2024-09-19T06:01:27.022034+0000","last_became_peered":"2024-09-19T06:01:27.022034+0000","last_unstale":"2024-09-19T06:01:40.847960+0000","last_undegraded":"2024-09-19T06:01:40.847960+0000","last_fullsized":"2024-09-19T06:01:40.847960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:08.663575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832674+0000","last_change":"2024-09-19T06:01:28.812079+0000","last_active":"2024-09-19T06:01:39.832674+0000","last_peered":"2024-09-19T06:01:39.832674+0000","last_clean":"2024-09-19T06:01:39.832674+0000","last_became_active":"2024-09-19T06:01:28.811606+0000","last_became_peered":"2024-09-19T06:01:28.811606+0000","last_unstale":"2024-09-19T06:01:39.832674+0000","last_undegraded":"2024-09-19T06:01:39.832674+0000","last_fullsized":"2024-09-19T06:01:39.832674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:49.434235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.838741+0000","last_change":"2024-09-19T06:01:26.783823+0000","last_active":"2024-09-19T06:01:39.838741+0000","last_peered":"2024-09-19T06:01:39.838741+0000","last_clean":"2024-09-19T06:01:39.838741+0000","last_became_active":"2024-09-19T06:01:26.783209+0000","last_became_peered":"2024-09-19T06:01:26.783209+0000","last_unstale":"2024-09-19T06:01:39.838741+0000","last_undegraded":"2024-09-19T06:01:39.838741+0000","last_fullsized":"2024-09-19T06:01:39.838741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:38.833417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837462+0000","last_change":"2024-09-19T06:01:39.837462+0000","last_active":"2024-09-19T06:01:39.837462+0000","last_peered":"2024-09-19T06:01:39.837462+0000","last_clean":"2024-09-19T06:01:39.837462+0000","last_became_active":"2024-09-19T06:01:39.837082+0000","last_became_peered":"2024-09-19T06:01:39.837082+0000","last_unstale":"2024-09-19T06:01:39.837462+0000","last_undegraded":"2024-09-19T06:01:39.837462+0000","last_fullsized":"2024-09-19T06:01:39.837462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:47.421017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.845150+0000","last_change":"2024-09-19T06:01:26.779992+0000","last_active":"2024-09-19T06:01:40.845150+0000","last_peered":"2024-09-19T06:01:40.845150+0000","last_clean":"2024-09-19T06:01:40.845150+0000","last_became_active":"2024-09-19T06:01:26.779609+0000","last_became_peered":"2024-09-19T06:01:26.779609+0000","last_unstale":"2024-09-19T06:01:40.845150+0000","last_undegraded":"2024-09-19T06:01:40.845150+0000","last_fullsized":"2024-09-19T06:01:40.845150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:48.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837947+0000","last_change":"2024-09-19T06:01:28.798827+0000","last_active":"2024-09-19T06:01:39.837947+0000","last_peered":"2024-09-19T06:01:39.837947+0000","last_clean":"2024-09-19T06:01:39.837947+0000","last_became_active":"2024-09-19T06:01:28.798457+0000","last_became_peered":"2024-09-19T06:01:28.798457+0000","last_unstale":"2024-09-19T06:01:39.837947+0000","last_undegraded":"2024-09-19T06:01:39.837947+0000","last_fullsized":"2024-09-19T06:01:39.837947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:41.111051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.846954+0000","last_change":"2024-09-19T06:01:26.775691+0000","last_active":"2024-09-19T06:01:40.846954+0000","last_peered":"2024-09-19T06:01:40.846954+0000","last_clean":"2024-09-19T06:01:40.846954+0000","last_became_active":"2024-09-19T06:01:26.775183+0000","last_became_peered":"2024-09-19T06:01:26.775183+0000","last_unstale":"2024-09-19T06:01:40.846954+0000","last_undegraded":"2024-09-19T06:01:40.846954+0000","last_fullsized":"2024-09-19T06:01:40.846954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:34.712968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837674+0000","last_change":"2024-09-19T06:01:28.809459+0000","last_active":"2024-09-19T06:01:39.837674+0000","last_peered":"2024-09-19T06:01:39.837674+0000","last_clean":"2024-09-19T06:01:39.837674+0000","last_became_active":"2024-09-19T06:01:28.809046+0000","last_became_peered":"2024-09-19T06:01:28.809046+0000","last_unstale":"2024-09-19T06:01:39.837674+0000","last_undegraded":"2024-09-19T06:01:39.837674+0000","last_fullsized":"2024-09-19T06:01:39.837674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:56.996190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:32.916301+0000","last_change":"2024-09-19T06:01:39.843932+0000","last_active":"2024-09-19T06:02:32.916301+0000","last_peered":"2024-09-19T06:02:32.916301+0000","last_clean":"2024-09-19T06:02:32.916301+0000","last_became_active":"2024-09-19T06:01:39.836008+0000","last_became_peered":"2024-09-19T06:01:39.836008+0000","last_unstale":"2024-09-19T06:02:32.916301+0000","last_undegraded":"2024-09-19T06:02:32.916301+0000","last_fullsized":"2024-09-19T06:02:32.916301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:58.798859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837726+0000","last_change":"2024-09-19T06:01:28.814181+0000","last_active":"2024-09-19T06:01:39.837726+0000","last_peered":"2024-09-19T06:01:39.837726+0000","last_clean":"2024-09-19T06:01:39.837726+0000","last_became_active":"2024-09-19T06:01:28.813817+0000","last_became_peered":"2024-09-19T06:01:28.813817+0000","last_unstale":"2024-09-19T06:01:39.837726+0000","last_undegraded":"2024-09-19T06:01:39.837726+0000","last_fullsized":"2024-09-19T06:01:39.837726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:40.057863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837197+0000","last_change":"2024-09-19T06:00:17.624509+0000","last_active":"2024-09-19T06:01:39.837197+0000","last_peered":"2024-09-19T06:01:39.837197+0000","last_clean":"2024-09-19T06:01:39.837197+0000","last_became_active":"2024-09-19T06:00:17.623943+0000","last_became_peered":"2024-09-19T06:00:17.623943+0000","last_unstale":"2024-09-19T06:01:39.837197+0000","last_undegraded":"2024-09-19T06:01:39.837197+0000","last_fullsized":"2024-09-19T06:01:39.837197+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_clean_scrub_stamp":"2024-09-19T05:56:55.414488+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:39.116345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:41.847767+0000","last_change":"2024-09-19T06:01:26.783896+0000","last_active":"2024-09-19T06:01:41.847767+0000","last_peered":"2024-09-19T06:01:41.847767+0000","last_clean":"2024-09-19T06:01:41.847767+0000","last_became_active":"2024-09-19T06:01:26.783332+0000","last_became_peered":"2024-09-19T06:01:26.783332+0000","last_unstale":"2024-09-19T06:01:41.847767+0000","last_undegraded":"2024-09-19T06:01:41.847767+0000","last_fullsized":"2024-09-19T06:01:41.847767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:33.072763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980113+0000","last_change":"2024-09-19T06:01:28.807756+0000","last_active":"2024-09-19T06:01:39.980113+0000","last_peered":"2024-09-19T06:01:39.980113+0000","last_clean":"2024-09-19T06:01:39.980113+0000","last_became_active":"2024-09-19T06:01:28.807077+0000","last_became_peered":"2024-09-19T06:01:28.807077+0000","last_unstale":"2024-09-19T06:01:39.980113+0000","last_undegraded":"2024-09-19T06:01:39.980113+0000","last_fullsized":"2024-09-19T06:01:39.980113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:20.784199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837194+0000","last_change":"2024-09-19T06:01:26.784830+0000","last_active":"2024-09-19T06:01:39.837194+0000","last_peered":"2024-09-19T06:01:39.837194+0000","last_clean":"2024-09-19T06:01:39.837194+0000","last_became_active":"2024-09-19T06:01:26.783887+0000","last_became_peered":"2024-09-19T06:01:26.783887+0000","last_unstale":"2024-09-19T06:01:39.837194+0000","last_undegraded":"2024-09-19T06:01:39.837194+0000","last_fullsized":"2024-09-19T06:01:39.837194+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:33:38.255169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837354+0000","last_change":"2024-09-19T06:01:39.837354+0000","last_active":"2024-09-19T06:01:39.837354+0000","last_peered":"2024-09-19T06:01:39.837354+0000","last_clean":"2024-09-19T06:01:39.837354+0000","last_became_active":"2024-09-19T06:01:39.836930+0000","last_became_peered":"2024-09-19T06:01:39.836930+0000","last_unstale":"2024-09-19T06:01:39.837354+0000","last_undegraded":"2024-09-19T06:01:39.837354+0000","last_fullsized":"2024-09-19T06:01:39.837354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:17.571834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:32.915892+0000","last_change":"2024-09-19T06:01:27.021753+0000","last_active":"2024-09-19T06:02:32.915892+0000","last_peered":"2024-09-19T06:02:32.915892+0000","last_clean":"2024-09-19T06:02:32.915892+0000","last_became_active":"2024-09-19T06:01:27.021346+0000","last_became_peered":"2024-09-19T06:01:27.021346+0000","last_unstale":"2024-09-19T06:02:32.915892+0000","last_undegraded":"2024-09-19T06:02:32.915892+0000","last_fullsized":"2024-09-19T06:02:32.915892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:07.291283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832814+0000","last_change":"2024-09-19T06:01:28.803103+0000","last_active":"2024-09-19T06:01:39.832814+0000","last_peered":"2024-09-19T06:01:39.832814+0000","last_clean":"2024-09-19T06:01:39.832814+0000","last_became_active":"2024-09-19T06:01:28.802806+0000","last_became_peered":"2024-09-19T06:01:28.802806+0000","last_unstale":"2024-09-19T06:01:39.832814+0000","last_undegraded":"2024-09-19T06:01:39.832814+0000","last_fullsized":"2024-09-19T06:01:39.832814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:33.329231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":83,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:32.916178+0000","last_change":"2024-09-19T06:01:26.786605+0000","last_active":"2024-09-19T06:02:32.916178+0000","last_peered":"2024-09-19T06:02:32.916178+0000","last_clean":"2024-09-19T06:02:32.916178+0000","last_became_active":"2024-09-19T06:01:26.786278+0000","last_became_peered":"2024-09-19T06:01:26.786278+0000","last_unstale":"2024-09-19T06:02:32.916178+0000","last_undegraded":"2024-09-19T06:02:32.916178+0000","last_fullsized":"2024-09-19T06:02:32.916178+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.581502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323778+0000","last_change":"2024-09-19T06:01:28.808152+0000","last_active":"2024-09-19T06:01:40.323778+0000","last_peered":"2024-09-19T06:01:40.323778+0000","last_clean":"2024-09-19T06:01:40.323778+0000","last_became_active":"2024-09-19T06:01:28.807788+0000","last_became_peered":"2024-09-19T06:01:28.807788+0000","last_unstale":"2024-09-19T06:01:40.323778+0000","last_undegraded":"2024-09-19T06:01:40.323778+0000","last_fullsized":"2024-09-19T06:01:40.323778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:42.877539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.836049+0000","last_change":"2024-09-19T06:01:27.022719+0000","last_active":"2024-09-19T06:01:39.836049+0000","last_peered":"2024-09-19T06:01:39.836049+0000","last_clean":"2024-09-19T06:01:39.836049+0000","last_became_active":"2024-09-19T06:01:27.022263+0000","last_became_peered":"2024-09-19T06:01:27.022263+0000","last_unstale":"2024-09-19T06:01:39.836049+0000","last_undegraded":"2024-09-19T06:01:39.836049+0000","last_fullsized":"2024-09-19T06:01:39.836049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:45.158229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.323926+0000","last_change":"2024-09-19T06:01:28.809869+0000","last_active":"2024-09-19T06:01:40.323926+0000","last_peered":"2024-09-19T06:01:40.323926+0000","last_clean":"2024-09-19T06:01:40.323926+0000","last_became_active":"2024-09-19T06:01:28.809548+0000","last_became_peered":"2024-09-19T06:01:28.809548+0000","last_unstale":"2024-09-19T06:01:40.323926+0000","last_undegraded":"2024-09-19T06:01:40.323926+0000","last_fullsized":"2024-09-19T06:01:40.323926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:41.587846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847043+0000","last_change":"2024-09-19T06:01:39.846740+0000","last_active":"2024-09-19T06:01:40.847043+0000","last_peered":"2024-09-19T06:01:40.847043+0000","last_clean":"2024-09-19T06:01:40.847043+0000","last_became_active":"2024-09-19T06:01:39.836378+0000","last_became_peered":"2024-09-19T06:01:39.836378+0000","last_unstale":"2024-09-19T06:01:40.847043+0000","last_undegraded":"2024-09-19T06:01:40.847043+0000","last_fullsized":"2024-09-19T06:01:40.847043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.064918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.829305+0000","last_change":"2024-09-19T06:01:28.807796+0000","last_active":"2024-09-19T06:01:39.829305+0000","last_peered":"2024-09-19T06:01:39.829305+0000","last_clean":"2024-09-19T06:01:39.829305+0000","last_became_active":"2024-09-19T06:01:28.807503+0000","last_became_peered":"2024-09-19T06:01:28.807503+0000","last_unstale":"2024-09-19T06:01:39.829305+0000","last_undegraded":"2024-09-19T06:01:39.829305+0000","last_fullsized":"2024-09-19T06:01:39.829305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:43.472647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:52.795819+0000","last_change":"2024-09-19T06:01:26.783517+0000","last_active":"2024-09-19T06:01:52.795819+0000","last_peered":"2024-09-19T06:01:52.795819+0000","last_clean":"2024-09-19T06:01:52.795819+0000","last_became_active":"2024-09-19T06:01:26.783164+0000","last_became_peered":"2024-09-19T06:01:26.783164+0000","last_unstale":"2024-09-19T06:01:52.795819+0000","last_undegraded":"2024-09-19T06:01:52.795819+0000","last_fullsized":"2024-09-19T06:01:52.795819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:57.609192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980454+0000","last_change":"2024-09-19T06:01:28.806548+0000","last_active":"2024-09-19T06:01:39.980454+0000","last_peered":"2024-09-19T06:01:39.980454+0000","last_clean":"2024-09-19T06:01:39.980454+0000","last_became_active":"2024-09-19T06:01:28.805724+0000","last_became_peered":"2024-09-19T06:01:28.805724+0000","last_unstale":"2024-09-19T06:01:39.980454+0000","last_undegraded":"2024-09-19T06:01:39.980454+0000","last_fullsized":"2024-09-19T06:01:39.980454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:14.360928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847547+0000","last_change":"2024-09-19T06:01:27.128213+0000","last_active":"2024-09-19T06:01:40.847547+0000","last_peered":"2024-09-19T06:01:40.847547+0000","last_clean":"2024-09-19T06:01:40.847547+0000","last_became_active":"2024-09-19T06:01:27.127737+0000","last_became_peered":"2024-09-19T06:01:27.127737+0000","last_unstale":"2024-09-19T06:01:40.847547+0000","last_undegraded":"2024-09-19T06:01:40.847547+0000","last_fullsized":"2024-09-19T06:01:40.847547+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:33:35.107152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.839027+0000","last_change":"2024-09-19T06:01:39.839027+0000","last_active":"2024-09-19T06:01:39.839027+0000","last_peered":"2024-09-19T06:01:39.839027+0000","last_clean":"2024-09-19T06:01:39.839027+0000","last_became_active":"2024-09-19T06:01:39.838601+0000","last_became_peered":"2024-09-19T06:01:39.838601+0000","last_unstale":"2024-09-19T06:01:39.839027+0000","last_undegraded":"2024-09-19T06:01:39.839027+0000","last_fullsized":"2024-09-19T06:01:39.839027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:09.864997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832051+0000","last_change":"2024-09-19T06:01:27.022438+0000","last_active":"2024-09-19T06:01:39.832051+0000","last_peered":"2024-09-19T06:01:39.832051+0000","last_clean":"2024-09-19T06:01:39.832051+0000","last_became_active":"2024-09-19T06:01:27.021953+0000","last_became_peered":"2024-09-19T06:01:27.021953+0000","last_unstale":"2024-09-19T06:01:39.832051+0000","last_undegraded":"2024-09-19T06:01:39.832051+0000","last_fullsized":"2024-09-19T06:01:39.832051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:24.383178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837088+0000","last_change":"2024-09-19T06:01:28.803807+0000","last_active":"2024-09-19T06:01:39.837088+0000","last_peered":"2024-09-19T06:01:39.837088+0000","last_clean":"2024-09-19T06:01:39.837088+0000","last_became_active":"2024-09-19T06:01:28.803486+0000","last_became_peered":"2024-09-19T06:01:28.803486+0000","last_unstale":"2024-09-19T06:01:39.837088+0000","last_undegraded":"2024-09-19T06:01:39.837088+0000","last_fullsized":"2024-09-19T06:01:39.837088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:26:17.920089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.845539+0000","last_change":"2024-09-19T06:01:26.777171+0000","last_active":"2024-09-19T06:01:40.845539+0000","last_peered":"2024-09-19T06:01:40.845539+0000","last_clean":"2024-09-19T06:01:40.845539+0000","last_became_active":"2024-09-19T06:01:26.776818+0000","last_became_peered":"2024-09-19T06:01:26.776818+0000","last_unstale":"2024-09-19T06:01:40.845539+0000","last_undegraded":"2024-09-19T06:01:40.845539+0000","last_fullsized":"2024-09-19T06:01:40.845539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832532+0000","last_change":"2024-09-19T06:01:28.799266+0000","last_active":"2024-09-19T06:01:39.832532+0000","last_peered":"2024-09-19T06:01:39.832532+0000","last_clean":"2024-09-19T06:01:39.832532+0000","last_became_active":"2024-09-19T06:01:28.798969+0000","last_became_peered":"2024-09-19T06:01:28.798969+0000","last_unstale":"2024-09-19T06:01:39.832532+0000","last_undegraded":"2024-09-19T06:01:39.832532+0000","last_fullsized":"2024-09-19T06:01:39.832532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:34.064795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.830288+0000","last_change":"2024-09-19T06:01:26.781680+0000","last_active":"2024-09-19T06:01:39.830288+0000","last_peered":"2024-09-19T06:01:39.830288+0000","last_clean":"2024-09-19T06:01:39.830288+0000","last_became_active":"2024-09-19T06:01:26.781035+0000","last_became_peered":"2024-09-19T06:01:26.781035+0000","last_unstale":"2024-09-19T06:01:39.830288+0000","last_undegraded":"2024-09-19T06:01:39.830288+0000","last_fullsized":"2024-09-19T06:01:39.830288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:38.287018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980152+0000","last_change":"2024-09-19T06:01:28.802970+0000","last_active":"2024-09-19T06:01:39.980152+0000","last_peered":"2024-09-19T06:01:39.980152+0000","last_clean":"2024-09-19T06:01:39.980152+0000","last_became_active":"2024-09-19T06:01:28.802587+0000","last_became_peered":"2024-09-19T06:01:28.802587+0000","last_unstale":"2024-09-19T06:01:39.980152+0000","last_undegraded":"2024-09-19T06:01:39.980152+0000","last_fullsized":"2024-09-19T06:01:39.980152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837492+0000","last_change":"2024-09-19T06:01:26.788140+0000","last_active":"2024-09-19T06:01:39.837492+0000","last_peered":"2024-09-19T06:01:39.837492+0000","last_clean":"2024-09-19T06:01:39.837492+0000","last_became_active":"2024-09-19T06:01:26.787603+0000","last_became_peered":"2024-09-19T06:01:26.787603+0000","last_unstale":"2024-09-19T06:01:39.837492+0000","last_undegraded":"2024-09-19T06:01:39.837492+0000","last_fullsized":"2024-09-19T06:01:39.837492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:13.880656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.837369+0000","last_change":"2024-09-19T06:01:28.813144+0000","last_active":"2024-09-19T06:01:39.837369+0000","last_peered":"2024-09-19T06:01:39.837369+0000","last_clean":"2024-09-19T06:01:39.837369+0000","last_became_active":"2024-09-19T06:01:28.812792+0000","last_became_peered":"2024-09-19T06:01:28.812792+0000","last_unstale":"2024-09-19T06:01:39.837369+0000","last_undegraded":"2024-09-19T06:01:39.837369+0000","last_fullsized":"2024-09-19T06:01:39.837369+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:35.896240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.833256+0000","last_change":"2024-09-19T06:01:28.812086+0000","last_active":"2024-09-19T06:01:39.833256+0000","last_peered":"2024-09-19T06:01:39.833256+0000","last_clean":"2024-09-19T06:01:39.833256+0000","last_became_active":"2024-09-19T06:01:28.811696+0000","last_became_peered":"2024-09-19T06:01:28.811696+0000","last_unstale":"2024-09-19T06:01:39.833256+0000","last_undegraded":"2024-09-19T06:01:39.833256+0000","last_fullsized":"2024-09-19T06:01:39.833256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:32.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:40.847847+0000","last_change":"2024-09-19T06:01:39.842250+0000","last_active":"2024-09-19T06:01:40.847847+0000","last_peered":"2024-09-19T06:01:40.847847+0000","last_clean":"2024-09-19T06:01:40.847847+0000","last_became_active":"2024-09-19T06:01:39.835761+0000","last_became_peered":"2024-09-19T06:01:39.835761+0000","last_unstale":"2024-09-19T06:01:40.847847+0000","last_undegraded":"2024-09-19T06:01:40.847847+0000","last_fullsized":"2024-09-19T06:01:40.847847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:51.026349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.832601+0000","last_change":"2024-09-19T06:01:28.812983+0000","last_active":"2024-09-19T06:01:39.832601+0000","last_peered":"2024-09-19T06:01:39.832601+0000","last_clean":"2024-09-19T06:01:39.832601+0000","last_became_active":"2024-09-19T06:01:28.812257+0000","last_became_peered":"2024-09-19T06:01:28.812257+0000","last_unstale":"2024-09-19T06:01:39.832601+0000","last_undegraded":"2024-09-19T06:01:39.832601+0000","last_fullsized":"2024-09-19T06:01:39.832601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:41:42.059736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:41.842632+0000","last_change":"2024-09-19T06:01:26.787869+0000","last_active":"2024-09-19T06:01:41.842632+0000","last_peered":"2024-09-19T06:01:41.842632+0000","last_clean":"2024-09-19T06:01:41.842632+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:01:41.842632+0000","last_undegraded":"2024-09-19T06:01:41.842632+0000","last_fullsized":"2024-09-19T06:01:41.842632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:02.869878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.831993+0000","last_change":"2024-09-19T06:01:28.811986+0000","last_active":"2024-09-19T06:01:39.831993+0000","last_peered":"2024-09-19T06:01:39.831993+0000","last_clean":"2024-09-19T06:01:39.831993+0000","last_became_active":"2024-09-19T06:01:28.811565+0000","last_became_peered":"2024-09-19T06:01:28.811565+0000","last_unstale":"2024-09-19T06:01:39.831993+0000","last_undegraded":"2024-09-19T06:01:39.831993+0000","last_fullsized":"2024-09-19T06:01:39.831993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:01.324985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980467+0000","last_change":"2024-09-19T06:01:27.021683+0000","last_active":"2024-09-19T06:01:39.980467+0000","last_peered":"2024-09-19T06:01:39.980467+0000","last_clean":"2024-09-19T06:01:39.980467+0000","last_became_active":"2024-09-19T06:01:27.021358+0000","last_became_peered":"2024-09-19T06:01:27.021358+0000","last_unstale":"2024-09-19T06:01:39.980467+0000","last_undegraded":"2024-09-19T06:01:39.980467+0000","last_fullsized":"2024-09-19T06:01:39.980467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:29.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:01:39.980328+0000","last_change":"2024-09-19T06:01:28.798963+0000","last_active":"2024-09-19T06:01:39.980328+0000","last_peered":"2024-09-19T06:01:39.980328+0000","last_clean":"2024-09-19T06:01:39.980328+0000","last_became_active":"2024-09-19T06:01:28.797571+0000","last_became_peered":"2024-09-19T06:01:28.797571+0000","last_unstale":"2024-09-19T06:01:39.980328+0000","last_undegraded":"2024-09-19T06:01:39.980328+0000","last_fullsized":"2024-09-19T06:01:39.980328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:43.867609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":89,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:02:32.915647+0000","last_change":"2024-09-19T06:01:26.786427+0000","last_active":"2024-09-19T06:02:32.915647+0000","last_peered":"2024-09-19T06:02:32.915647+0000","last_clean":"2024-09-19T06:02:32.915647+0000","last_became_active":"2024-09-19T06:01:26.786092+0000","last_became_peered":"2024-09-19T06:01:26.786092+0000","last_unstale":"2024-09-19T06:02:32.915647+0000","last_undegraded":"2024-09-19T06:02:32.915647+0000","last_fullsized":"2024-09-19T06:02:32.915647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:49.144366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69166,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":568,"num_read_kb":362,"num_write":245,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":439275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776160,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36920,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708232,"statfs":{"total":95995035648,"available":95957229568,"internally_reserved":0,"allocated":1613824,"data_stored":1090782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":438118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168618,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":437802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1031802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1030694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":439104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":437660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40468,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704684,"statfs":{"total":95995035648,"available":95953596416,"internally_reserved":0,"allocated":987136,"data_stored":498348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40472,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704680,"statfs":{"total":95995035648,"available":95953592320,"internally_reserved":0,"allocated":991232,"data_stored":501876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":439567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:02:35.709 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T06:02:35.710 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T06:02:35.710 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T06:02:35.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T06:02:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:36.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:36 smithi005 ceph-mon[30955]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:02:36.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:36 smithi005 ceph-mon[30955]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:02:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.441+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/135752380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f2410c170 msgr2=0x7f2f2410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.441+0000 7f2f2a7f0640 1 --2- 172.21.15.5:0/135752380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f2410c170 0x7f2f2410e610 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009ca0 tx=0x7f2f1c02a950 comp rx=0 tx=0).stop 2024-09-19T06:02:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.441+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/135752380 shutdown_connections 2024-09-19T06:02:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.441+0000 7f2f2a7f0640 1 --2- 172.21.15.5:0/135752380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f2410c170 0x7f2f2410e610 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.441+0000 7f2f2a7f0640 1 --2- 172.21.15.5:0/135752380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f24103660 0x7f2f2410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.441+0000 7f2f2a7f0640 1 --2- 172.21.15.5:0/135752380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f24102d40 0x7f2f24103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.441+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/135752380 >> 172.21.15.5:0/135752380 conn(0x7f2f240fc7a0 msgr2=0x7f2f240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.441+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/135752380 shutdown_connections 2024-09-19T06:02:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.441+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/135752380 wait complete. 2024-09-19T06:02:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f2a7f0640 1 Processor -- start 2024-09-19T06:02:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f2a7f0640 1 -- start start 2024-09-19T06:02:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f2a7f0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f24102d40 0x7f2f24198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f2a7f0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f24103660 0x7f2f241988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f2a7f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f2410c170 0x7f2f2419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f2a7f0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f24111e60 con 0x7f2f2410c170 2024-09-19T06:02:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f2a7f0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2f24111ce0 con 0x7f2f24103660 2024-09-19T06:02:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f2a7f0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f24111fe0 con 0x7f2f24102d40 2024-09-19T06:02:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f28d8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f2410c170 0x7f2f2419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f28d8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f2410c170 0x7f2f2419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49932/0 (socket says 172.21.15.5:49932) 2024-09-19T06:02:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f28d8d640 1 -- 172.21.15.5:0/156846329 learned_addr learned my addr 172.21.15.5:0/156846329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f23fff640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f24102d40 0x7f2f24198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.442+0000 7f2f237fe640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f24103660 0x7f2f241988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.443+0000 7f2f28d8d640 1 -- 172.21.15.5:0/156846329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f24102d40 msgr2=0x7f2f24198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.443+0000 7f2f28d8d640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f24102d40 0x7f2f24198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.443+0000 7f2f28d8d640 1 -- 172.21.15.5:0/156846329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f24103660 msgr2=0x7f2f241988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.443+0000 7f2f28d8d640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f24103660 0x7f2f241988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.443+0000 7f2f28d8d640 1 -- 172.21.15.5:0/156846329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2419d330 con 0x7f2f2410c170 2024-09-19T06:02:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.443+0000 7f2f28d8d640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f2410c170 0x7f2f2419cc50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c02ae70 tx=0x7f2f1c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.443+0000 7f2f217fa640 1 -- 172.21.15.5:0/156846329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c034040 con 0x7f2f2410c170 2024-09-19T06:02:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.443+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/156846329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2419d5c0 con 0x7f2f2410c170 2024-09-19T06:02:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.443+0000 7f2f217fa640 1 -- 172.21.15.5:0/156846329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f1c0308f0 con 0x7f2f2410c170 2024-09-19T06:02:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.443+0000 7f2f217fa640 1 -- 172.21.15.5:0/156846329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c0047f0 con 0x7f2f2410c170 2024-09-19T06:02:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.443+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/156846329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f241a4f00 con 0x7f2f2410c170 2024-09-19T06:02:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.444+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/156846329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f24103120 con 0x7f2f2410c170 2024-09-19T06:02:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.446+0000 7f2f217fa640 1 -- 172.21.15.5:0/156846329 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f2f1c037020 con 0x7f2f2410c170 2024-09-19T06:02:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.447+0000 7f2f217fa640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f2f0c0a32d0 0x7f2f0c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.447+0000 7f2f23fff640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f2f0c0a32d0 0x7f2f0c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.447+0000 7f2f217fa640 1 -- 172.21.15.5:0/156846329 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f2f1c0f0a90 con 0x7f2f2410c170 2024-09-19T06:02:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.447+0000 7f2f23fff640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f2f0c0a32d0 0x7f2f0c0a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400bbe0 tx=0x7f2f1400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.450+0000 7f2f217fa640 1 -- 172.21.15.5:0/156846329 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2f1c081210 con 0x7f2f2410c170 2024-09-19T06:02:36.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:36 smithi157 ceph-mon[35346]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:02:36.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:36 smithi157 ceph-mon[35346]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:02:36.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:36 smithi049 ceph-mon[34640]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:02:36.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:36 smithi049 ceph-mon[34640]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:02:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.667+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/156846329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2f241071e0 con 0x7f2f2410c170 2024-09-19T06:02:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.668+0000 7f2f217fa640 1 -- 172.21.15.5:0/156846329 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2f1c0bd750 con 0x7f2f2410c170 2024-09-19T06:02:36.670 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:36.670 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T06:02:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.670+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/156846329 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f2f0c0a32d0 msgr2=0x7f2f0c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.670+0000 7f2f2a7f0640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f2f0c0a32d0 0x7f2f0c0a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400bbe0 tx=0x7f2f1400b390 comp rx=0 tx=0).stop 2024-09-19T06:02:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.670+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/156846329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f2410c170 msgr2=0x7f2f2419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.670+0000 7f2f2a7f0640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f2410c170 0x7f2f2419cc50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c02ae70 tx=0x7f2f1c02f5d0 comp rx=0 tx=0).stop 2024-09-19T06:02:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.670+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/156846329 shutdown_connections 2024-09-19T06:02:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.670+0000 7f2f2a7f0640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f2f0c0a32d0 0x7f2f0c0a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.670+0000 7f2f2a7f0640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f2410c170 0x7f2f2419cc50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.670+0000 7f2f2a7f0640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f24103660 0x7f2f241988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.670+0000 7f2f2a7f0640 1 --2- 172.21.15.5:0/156846329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f24102d40 0x7f2f24198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.670+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/156846329 >> 172.21.15.5:0/156846329 conn(0x7f2f240fc7a0 msgr2=0x7f2f240fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.670+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/156846329 shutdown_connections 2024-09-19T06:02:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:36.670+0000 7f2f2a7f0640 1 -- 172.21.15.5:0/156846329 wait complete. 2024-09-19T06:02:37.363 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T06:02:37.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:02:37.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:37 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/156846329' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:02:37.621 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:37.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:37 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/156846329' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:02:37.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:37 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/156846329' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:02:38.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.044+0000 7f0c9b525640 1 -- 172.21.15.5:0/222810074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c94102790 msgr2=0x7f0c941067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:38.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.044+0000 7f0c9b525640 1 --2- 172.21.15.5:0/222810074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c94102790 0x7f0c941067e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0c84009900 tx=0x7f0c8402a410 comp rx=0 tx=0).stop 2024-09-19T06:02:38.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.044+0000 7f0c9b525640 1 -- 172.21.15.5:0/222810074 shutdown_connections 2024-09-19T06:02:38.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.044+0000 7f0c9b525640 1 --2- 172.21.15.5:0/222810074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c94106f10 0x7f0c9410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:38.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.044+0000 7f0c9b525640 1 --2- 172.21.15.5:0/222810074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c94102790 0x7f0c941067e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:38.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.044+0000 7f0c9b525640 1 --2- 172.21.15.5:0/222810074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c94101de0 0x7f0c941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:38.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.044+0000 7f0c9b525640 1 -- 172.21.15.5:0/222810074 >> 172.21.15.5:0/222810074 conn(0x7f0c940fd8c0 msgr2=0x7f0c940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:38.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.045+0000 7f0c9b525640 1 -- 172.21.15.5:0/222810074 shutdown_connections 2024-09-19T06:02:38.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.045+0000 7f0c9b525640 1 -- 172.21.15.5:0/222810074 wait complete. 2024-09-19T06:02:38.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.045+0000 7f0c9b525640 1 Processor -- start 2024-09-19T06:02:38.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.045+0000 7f0c9b525640 1 -- start start 2024-09-19T06:02:38.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.045+0000 7f0c9b525640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c94101de0 0x7f0c940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.045+0000 7f0c9b525640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c94102790 0x7f0c94074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.045+0000 7f0c9b525640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c94106f10 0x7f0c94074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.045+0000 7f0c9b525640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c94111e70 con 0x7f0c94106f10 2024-09-19T06:02:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.045+0000 7f0c9b525640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c94111cf0 con 0x7f0c94102790 2024-09-19T06:02:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.045+0000 7f0c9b525640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c94111ff0 con 0x7f0c94101de0 2024-09-19T06:02:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c99ac2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c94106f10 0x7f0c94074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c99ac2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c94106f10 0x7f0c94074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49950/0 (socket says 172.21.15.5:49950) 2024-09-19T06:02:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c99ac2640 1 -- 172.21.15.5:0/850355366 learned_addr learned my addr 172.21.15.5:0/850355366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c98ac0640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c94102790 0x7f0c94074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c992c1640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c94101de0 0x7f0c940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c99ac2640 1 -- 172.21.15.5:0/850355366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c94101de0 msgr2=0x7f0c940744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c99ac2640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c94101de0 0x7f0c940744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c99ac2640 1 -- 172.21.15.5:0/850355366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c94102790 msgr2=0x7f0c94074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c99ac2640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c94102790 0x7f0c94074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c99ac2640 1 -- 172.21.15.5:0/850355366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c941a68f0 con 0x7f0c94106f10 2024-09-19T06:02:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c99ac2640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c94106f10 0x7f0c94074f70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00cad0 tx=0x7f0c8c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c8a7fc640 1 -- 172.21.15.5:0/850355366 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00ae20 con 0x7f0c94106f10 2024-09-19T06:02:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c9b525640 1 -- 172.21.15.5:0/850355366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c941a6be0 con 0x7f0c94106f10 2024-09-19T06:02:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c8a7fc640 1 -- 172.21.15.5:0/850355366 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c8c005020 con 0x7f0c94106f10 2024-09-19T06:02:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c8a7fc640 1 -- 172.21.15.5:0/850355366 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00f6b0 con 0x7f0c94106f10 2024-09-19T06:02:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.046+0000 7f0c9b525640 1 -- 172.21.15.5:0/850355366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c941a7140 con 0x7f0c94106f10 2024-09-19T06:02:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.048+0000 7f0c9b525640 1 -- 172.21.15.5:0/850355366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c941021c0 con 0x7f0c94106f10 2024-09-19T06:02:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.049+0000 7f0c8a7fc640 1 -- 172.21.15.5:0/850355366 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f0c8c0051d0 con 0x7f0c94106f10 2024-09-19T06:02:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.049+0000 7f0c8a7fc640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0c800a32d0 0x7f0c800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.049+0000 7f0c992c1640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0c800a32d0 0x7f0c800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.050+0000 7f0c8a7fc640 1 -- 172.21.15.5:0/850355366 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0c8c0cf620 con 0x7f0c94106f10 2024-09-19T06:02:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.050+0000 7f0c992c1640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0c800a32d0 0x7f0c800a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0c900058c0 tx=0x7f0c9000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.053+0000 7f0c8a7fc640 1 -- 172.21.15.5:0/850355366 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0c8c014030 con 0x7f0c94106f10 2024-09-19T06:02:38.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.264+0000 7f0c9b525640 1 -- 172.21.15.5:0/850355366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c9410a0f0 con 0x7f0c94106f10 2024-09-19T06:02:38.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.265+0000 7f0c8a7fc640 1 -- 172.21.15.5:0/850355366 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11959 (secure 0 0 0) 0x7f0c8c09bb80 con 0x7f0c94106f10 2024-09-19T06:02:38.267 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:38.267 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-19T06:01:41:840933+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:01:41.840914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:02:38.268 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-19T06:02:38.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.267+0000 7f0c9b525640 1 -- 172.21.15.5:0/850355366 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0c800a32d0 msgr2=0x7f0c800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:38.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.267+0000 7f0c9b525640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0c800a32d0 0x7f0c800a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0c900058c0 tx=0x7f0c9000a250 comp rx=0 tx=0).stop 2024-09-19T06:02:38.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.267+0000 7f0c9b525640 1 -- 172.21.15.5:0/850355366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c94106f10 msgr2=0x7f0c94074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:38.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.267+0000 7f0c9b525640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c94106f10 0x7f0c94074f70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00cad0 tx=0x7f0c8c00cf90 comp rx=0 tx=0).stop 2024-09-19T06:02:38.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.267+0000 7f0c9b525640 1 -- 172.21.15.5:0/850355366 shutdown_connections 2024-09-19T06:02:38.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.267+0000 7f0c9b525640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f0c800a32d0 0x7f0c800a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:38.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.267+0000 7f0c9b525640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c94106f10 0x7f0c94074f70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:38.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.267+0000 7f0c9b525640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c94102790 0x7f0c94074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:38.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.267+0000 7f0c9b525640 1 --2- 172.21.15.5:0/850355366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c94101de0 0x7f0c940744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:38.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.267+0000 7f0c9b525640 1 -- 172.21.15.5:0/850355366 >> 172.21.15.5:0/850355366 conn(0x7f0c940fd8c0 msgr2=0x7f0c940fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:38.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.267+0000 7f0c9b525640 1 -- 172.21.15.5:0/850355366 shutdown_connections 2024-09-19T06:02:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:38.267+0000 7f0c9b525640 1 -- 172.21.15.5:0/850355366 wait complete. 2024-09-19T06:02:38.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:38 smithi157 ceph-mon[35346]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:38.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:38 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/850355366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:02:38.626 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:38 smithi005 ceph-mon[30955]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:38.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:38 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/850355366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:02:38.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:38 smithi049 ceph-mon[34640]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:38.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:38 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/850355366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:02:38.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:02:39.187 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.626+0000 7ffb07da3640 1 -- 172.21.15.5:0/1059733939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb00102d40 msgr2=0x7ffb00103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.626+0000 7ffb07da3640 1 --2- 172.21.15.5:0/1059733939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb00102d40 0x7ffb00103120 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009ca0 tx=0x7ffafc02a950 comp rx=0 tx=0).stop 2024-09-19T06:02:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.626+0000 7ffb07da3640 1 -- 172.21.15.5:0/1059733939 shutdown_connections 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.626+0000 7ffb07da3640 1 --2- 172.21.15.5:0/1059733939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb0010c170 0x7ffb0010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.626+0000 7ffb07da3640 1 --2- 172.21.15.5:0/1059733939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb00103660 0x7ffb0010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.626+0000 7ffb07da3640 1 --2- 172.21.15.5:0/1059733939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb00102d40 0x7ffb00103120 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.626+0000 7ffb07da3640 1 -- 172.21.15.5:0/1059733939 >> 172.21.15.5:0/1059733939 conn(0x7ffb000fc7a0 msgr2=0x7ffb000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.626+0000 7ffb07da3640 1 -- 172.21.15.5:0/1059733939 shutdown_connections 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.626+0000 7ffb07da3640 1 -- 172.21.15.5:0/1059733939 wait complete. 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb07da3640 1 Processor -- start 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb07da3640 1 -- start start 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb07da3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb00102d40 0x7ffb001009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb07da3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb00103660 0x7ffb00100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb07da3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb0010c170 0x7ffb00101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb07da3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00111ee0 con 0x7ffb00103660 2024-09-19T06:02:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb07da3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00111d60 con 0x7ffb00102d40 2024-09-19T06:02:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb07da3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00112060 con 0x7ffb0010c170 2024-09-19T06:02:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb0533e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb00103660 0x7ffb00100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb0533e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb00103660 0x7ffb00100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49982/0 (socket says 172.21.15.5:49982) 2024-09-19T06:02:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb0533e640 1 -- 172.21.15.5:0/2378226789 learned_addr learned my addr 172.21.15.5:0/2378226789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb05b3f640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb00102d40 0x7ffb001009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb06340640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb0010c170 0x7ffb00101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb0533e640 1 -- 172.21.15.5:0/2378226789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb0010c170 msgr2=0x7ffb00101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb0533e640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb0010c170 0x7ffb00101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb0533e640 1 -- 172.21.15.5:0/2378226789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb00102d40 msgr2=0x7ffb001009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.627+0000 7ffb0533e640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb00102d40 0x7ffb001009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.628+0000 7ffb0533e640 1 -- 172.21.15.5:0/2378226789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb001a69c0 con 0x7ffb00103660 2024-09-19T06:02:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.628+0000 7ffb05b3f640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb00102d40 0x7ffb001009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:39.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.628+0000 7ffb0533e640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb00103660 0x7ffb00100ee0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000b3f0 tx=0x7ffaf000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:39.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.628+0000 7ffaf6ffd640 1 -- 172.21.15.5:0/2378226789 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0015020 con 0x7ffb00103660 2024-09-19T06:02:39.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.628+0000 7ffaf6ffd640 1 -- 172.21.15.5:0/2378226789 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffaf0011ce0 con 0x7ffb00103660 2024-09-19T06:02:39.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.628+0000 7ffb07da3640 1 -- 172.21.15.5:0/2378226789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb001a6cb0 con 0x7ffb00103660 2024-09-19T06:02:39.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.628+0000 7ffaf6ffd640 1 -- 172.21.15.5:0/2378226789 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0010d80 con 0x7ffb00103660 2024-09-19T06:02:39.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.628+0000 7ffb07da3640 1 -- 172.21.15.5:0/2378226789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb001a71a0 con 0x7ffb00103660 2024-09-19T06:02:39.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.629+0000 7ffb07da3640 1 -- 172.21.15.5:0/2378226789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb001031a0 con 0x7ffb00103660 2024-09-19T06:02:39.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.631+0000 7ffaf6ffd640 1 -- 172.21.15.5:0/2378226789 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7ffaf0018020 con 0x7ffb00103660 2024-09-19T06:02:39.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.631+0000 7ffaf6ffd640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ffaec0a32d0 0x7ffaec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:39.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.631+0000 7ffb05b3f640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ffaec0a32d0 0x7ffaec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:39.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.631+0000 7ffaf6ffd640 1 -- 172.21.15.5:0/2378226789 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ffaf00cf5c0 con 0x7ffb00103660 2024-09-19T06:02:39.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.632+0000 7ffb05b3f640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ffaec0a32d0 0x7ffaec0a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffafc006010 tx=0x7ffafc005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:39.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.635+0000 7ffaf6ffd640 1 -- 172.21.15.5:0/2378226789 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffaf001f030 con 0x7ffb00103660 2024-09-19T06:02:39.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.851+0000 7ffb07da3640 1 -- 172.21.15.5:0/2378226789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb00101ed0 con 0x7ffb00103660 2024-09-19T06:02:39.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.852+0000 7ffaf6ffd640 1 -- 172.21.15.5:0/2378226789 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11959 (secure 0 0 0) 0x7ffb00101ed0 con 0x7ffb00103660 2024-09-19T06:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-19T06:01:41:840933+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:01:41.840914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:02:39.854 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-19T06:02:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.855+0000 7ffb07da3640 1 -- 172.21.15.5:0/2378226789 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ffaec0a32d0 msgr2=0x7ffaec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.855+0000 7ffb07da3640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ffaec0a32d0 0x7ffaec0a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffafc006010 tx=0x7ffafc005f80 comp rx=0 tx=0).stop 2024-09-19T06:02:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.855+0000 7ffb07da3640 1 -- 172.21.15.5:0/2378226789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb00103660 msgr2=0x7ffb00100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.855+0000 7ffb07da3640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb00103660 0x7ffb00100ee0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000b3f0 tx=0x7ffaf000b8b0 comp rx=0 tx=0).stop 2024-09-19T06:02:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.856+0000 7ffb07da3640 1 -- 172.21.15.5:0/2378226789 shutdown_connections 2024-09-19T06:02:39.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.856+0000 7ffb07da3640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ffaec0a32d0 0x7ffaec0a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:39.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.856+0000 7ffb07da3640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb0010c170 0x7ffb00101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:39.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.856+0000 7ffb07da3640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb00103660 0x7ffb00100ee0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:39.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.856+0000 7ffb07da3640 1 --2- 172.21.15.5:0/2378226789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb00102d40 0x7ffb001009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:39.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.856+0000 7ffb07da3640 1 -- 172.21.15.5:0/2378226789 >> 172.21.15.5:0/2378226789 conn(0x7ffb000fc7a0 msgr2=0x7ffb000fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:39.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.856+0000 7ffb07da3640 1 -- 172.21.15.5:0/2378226789 shutdown_connections 2024-09-19T06:02:39.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:39.856+0000 7ffb07da3640 1 -- 172.21.15.5:0/2378226789 wait complete. 2024-09-19T06:02:40.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:02:40.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:40 smithi005 ceph-mon[30955]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:40.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:40 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2378226789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:02:40.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:40 smithi049 ceph-mon[34640]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:40.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:40 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2378226789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:02:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:40 smithi157 ceph-mon[35346]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:40 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2378226789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:02:40.808 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.258+0000 7f4e22ad6640 1 -- 172.21.15.5:0/1152043313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c106f50 msgr2=0x7f4e1c10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.258+0000 7f4e22ad6640 1 --2- 172.21.15.5:0/1152043313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c106f50 0x7f4e1c10eaf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009cd0 tx=0x7f4e0c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:02:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.258+0000 7f4e22ad6640 1 -- 172.21.15.5:0/1152043313 shutdown_connections 2024-09-19T06:02:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.258+0000 7f4e22ad6640 1 --2- 172.21.15.5:0/1152043313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c106f50 0x7f4e1c10eaf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.258+0000 7f4e22ad6640 1 --2- 172.21.15.5:0/1152043313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c100ca0 0x7f4e1c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.258+0000 7f4e22ad6640 1 --2- 172.21.15.5:0/1152043313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e1c1002f0 0x7f4e1c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.258+0000 7f4e22ad6640 1 -- 172.21.15.5:0/1152043313 >> 172.21.15.5:0/1152043313 conn(0x7f4e1c077210 msgr2=0x7f4e1c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.258+0000 7f4e22ad6640 1 -- 172.21.15.5:0/1152043313 shutdown_connections 2024-09-19T06:02:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.258+0000 7f4e22ad6640 1 -- 172.21.15.5:0/1152043313 wait complete. 2024-09-19T06:02:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e22ad6640 1 Processor -- start 2024-09-19T06:02:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e22ad6640 1 -- start start 2024-09-19T06:02:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e22ad6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e1c1002f0 0x7f4e1c101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e22ad6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c100ca0 0x7f4e1c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e22ad6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c106f50 0x7f4e1c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e22ad6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c111f30 con 0x7f4e1c100ca0 2024-09-19T06:02:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e22ad6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c111db0 con 0x7f4e1c106f50 2024-09-19T06:02:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e22ad6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c1120b0 con 0x7f4e1c1002f0 2024-09-19T06:02:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e1bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c100ca0 0x7f4e1c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e1bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c100ca0 0x7f4e1c1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50004/0 (socket says 172.21.15.5:50004) 2024-09-19T06:02:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e1bfff640 1 -- 172.21.15.5:0/3082031052 learned_addr learned my addr 172.21.15.5:0/3082031052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e20872640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e1c1002f0 0x7f4e1c101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.259+0000 7f4e21073640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c106f50 0x7f4e1c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.260+0000 7f4e1bfff640 1 -- 172.21.15.5:0/3082031052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e1c1002f0 msgr2=0x7f4e1c101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.260+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e1c1002f0 0x7f4e1c101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.260+0000 7f4e1bfff640 1 -- 172.21.15.5:0/3082031052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c106f50 msgr2=0x7f4e1c101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.260+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c106f50 0x7f4e1c101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.260+0000 7f4e1bfff640 1 -- 172.21.15.5:0/3082031052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1c1a6a50 con 0x7f4e1c100ca0 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.260+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c100ca0 0x7f4e1c1017b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400b450 tx=0x7f4e0400b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.260+0000 7f4e19ffb640 1 -- 172.21.15.5:0/3082031052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04015020 con 0x7f4e1c100ca0 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.260+0000 7f4e19ffb640 1 -- 172.21.15.5:0/3082031052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e04011ce0 con 0x7f4e1c100ca0 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.260+0000 7f4e22ad6640 1 -- 172.21.15.5:0/3082031052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1c1a6d40 con 0x7f4e1c100ca0 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.260+0000 7f4e19ffb640 1 -- 172.21.15.5:0/3082031052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04010dc0 con 0x7f4e1c100ca0 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.260+0000 7f4e22ad6640 1 -- 172.21.15.5:0/3082031052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1c1a7270 con 0x7f4e1c100ca0 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.261+0000 7f4e22ad6640 1 -- 172.21.15.5:0/3082031052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e1c100750 con 0x7f4e1c100ca0 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.262+0000 7f4e19ffb640 1 -- 172.21.15.5:0/3082031052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f4e04018020 con 0x7f4e1c100ca0 2024-09-19T06:02:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.263+0000 7f4e19ffb640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4e080a3350 0x7f4e080a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.263+0000 7f4e19ffb640 1 -- 172.21.15.5:0/3082031052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f4e040cf2d0 con 0x7f4e1c100ca0 2024-09-19T06:02:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.263+0000 7f4e20872640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4e080a3350 0x7f4e080a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.264+0000 7f4e20872640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4e080a3350 0x7f4e080a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009990 tx=0x7f4e14009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.267+0000 7f4e19ffb640 1 -- 172.21.15.5:0/3082031052 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4e0401f020 con 0x7f4e1c100ca0 2024-09-19T06:02:41.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.444+0000 7f4e22ad6640 1 -- 172.21.15.5:0/3082031052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e1c02d7c0 con 0x7f4e1c100ca0 2024-09-19T06:02:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.445+0000 7f4e19ffb640 1 -- 172.21.15.5:0/3082031052 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22702 (secure 0 0 0) 0x7f4e0409bf30 con 0x7f4e1c100ca0 2024-09-19T06:02:41.448 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:41.448 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","created":"2024-09-19T05:53:51.454068+0000","modified":"2024-09-19T06:01:39.826379+0000","last_up_change":"2024-09-19T06:00:38.837916+0000","last_in_change":"2024-09-19T06:00:15.609776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:56:55.213931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:01:25.648588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:01:27.760408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f3c10d60-d354-407f-9cdf-099344599844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6803","nonce":3442981810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6805","nonce":3442981810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6809","nonce":3442981810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6807","nonce":3442981810}]},"public_addr":"172.21.15.5:6803/3442981810","cluster_addr":"172.21.15.5:6805/3442981810","heartbeat_back_addr":"172.21.15.5:6809/3442981810","heartbeat_front_addr":"172.21.15.5:6807/3442981810","state":["exists","up"]},{"osd":1,"uuid":"5f40f636-7761-45ac-87f3-910a2fa2feb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448265014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6803","nonce":1448265014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6807","nonce":1448265014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6805","nonce":1448265014}]},"public_addr":"172.21.15.49:6801/1448265014","cluster_addr":"172.21.15.49:6803/1448265014","heartbeat_back_addr":"172.21.15.49:6807/1448265014","heartbeat_front_addr":"172.21.15.49:6805/1448265014","state":["exists","up"]},{"osd":2,"uuid":"ad00940c-4fdd-4f5f-9eaa-b1f38bcac526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6801","nonce":2319626622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6803","nonce":2319626622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6807","nonce":2319626622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6805","nonce":2319626622}]},"public_addr":"172.21.15.157:6801/2319626622","cluster_addr":"172.21.15.157:6803/2319626622","heartbeat_back_addr":"172.21.15.157:6807/2319626622","heartbeat_front_addr":"172.21.15.157:6805/2319626622","state":["exists","up"]},{"osd":3,"uuid":"63fec329-1b62-4c5e-b8ca-084a84d4d762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6811","nonce":1452491753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6813","nonce":1452491753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6817","nonce":1452491753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6815","nonce":1452491753}]},"public_addr":"172.21.15.5:6811/1452491753","cluster_addr":"172.21.15.5:6813/1452491753","heartbeat_back_addr":"172.21.15.5:6817/1452491753","heartbeat_front_addr":"172.21.15.5:6815/1452491753","state":["exists","up"]},{"osd":4,"uuid":"ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6809","nonce":3114952580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6811","nonce":3114952580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6815","nonce":3114952580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6813","nonce":3114952580}]},"public_addr":"172.21.15.49:6809/3114952580","cluster_addr":"172.21.15.49:6811/3114952580","heartbeat_back_addr":"172.21.15.49:6815/3114952580","heartbeat_front_addr":"172.21.15.49:6813/3114952580","state":["exists","up"]},{"osd":5,"uuid":"21a7a7d3-ce42-4688-bfec-714be635eaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6809","nonce":3219063971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6811","nonce":3219063971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6815","nonce":3219063971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6813","nonce":3219063971}]},"public_addr":"172.21.15.157:6809/3219063971","cluster_addr":"172.21.15.157:6811/3219063971","heartbeat_back_addr":"172.21.15.157:6815/3219063971","heartbeat_front_addr":"172.21.15.157:6813/3219063971","state":["exists","up"]},{"osd":6,"uuid":"cbef0e08-d6b0-4515-a2ee-2c81b048583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6819","nonce":2957795194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6821","nonce":2957795194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6825","nonce":2957795194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6823","nonce":2957795194}]},"public_addr":"172.21.15.5:6819/2957795194","cluster_addr":"172.21.15.5:6821/2957795194","heartbeat_back_addr":"172.21.15.5:6825/2957795194","heartbeat_front_addr":"172.21.15.5:6823/2957795194","state":["exists","up"]},{"osd":7,"uuid":"d261ac55-532d-4aea-9b3e-ed2b4eeb1da6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6817","nonce":2959731400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6819","nonce":2959731400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6823","nonce":2959731400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6821","nonce":2959731400}]},"public_addr":"172.21.15.49:6817/2959731400","cluster_addr":"172.21.15.49:6819/2959731400","heartbeat_back_addr":"172.21.15.49:6823/2959731400","heartbeat_front_addr":"172.21.15.49:6821/2959731400","state":["exists","up"]},{"osd":8,"uuid":"15d0eaf7-d604-4546-b393-4d6d39018f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6817","nonce":290026859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6819","nonce":290026859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6823","nonce":290026859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6821","nonce":290026859}]},"public_addr":"172.21.15.157:6817/290026859","cluster_addr":"172.21.15.157:6819/290026859","heartbeat_back_addr":"172.21.15.157:6823/290026859","heartbeat_front_addr":"172.21.15.157:6821/290026859","state":["exists","up"]},{"osd":9,"uuid":"30c08537-f593-4299-bfa1-fa7ba8a95169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6827","nonce":4122448237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6829","nonce":4122448237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6833","nonce":4122448237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6831","nonce":4122448237}]},"public_addr":"172.21.15.5:6827/4122448237","cluster_addr":"172.21.15.5:6829/4122448237","heartbeat_back_addr":"172.21.15.5:6833/4122448237","heartbeat_front_addr":"172.21.15.5:6831/4122448237","state":["exists","up"]},{"osd":10,"uuid":"68821e98-bd81-49ec-8cd6-51fb9880b553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6825","nonce":2551748002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6827","nonce":2551748002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6831","nonce":2551748002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6829","nonce":2551748002}]},"public_addr":"172.21.15.49:6825/2551748002","cluster_addr":"172.21.15.49:6827/2551748002","heartbeat_back_addr":"172.21.15.49:6831/2551748002","heartbeat_front_addr":"172.21.15.49:6829/2551748002","state":["exists","up"]},{"osd":11,"uuid":"7e443b46-2697-4d0b-8452-868463a1ee1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6825","nonce":1996582544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6827","nonce":1996582544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6831","nonce":1996582544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6829","nonce":1996582544}]},"public_addr":"172.21.15.157:6825/1996582544","cluster_addr":"172.21.15.157:6827/1996582544","heartbeat_back_addr":"172.21.15.157:6831/1996582544","heartbeat_front_addr":"172.21.15.157:6829/1996582544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:01.453592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:27.194705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:51.525385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:17.676749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:43.075555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:07.258154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:32.112583+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:57.306534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:21.320950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:47.164610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:12.548337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:37.262580+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4102275559":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/2593868621":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/1340147787":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6800/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/850678758":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6800/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2930274429":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/1367598148":"2024-09-20T05:54:15.334457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:02:41.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.448+0000 7f4e22ad6640 1 -- 172.21.15.5:0/3082031052 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4e080a3350 msgr2=0x7f4e080a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:41.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.448+0000 7f4e22ad6640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4e080a3350 0x7f4e080a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009990 tx=0x7f4e14009110 comp rx=0 tx=0).stop 2024-09-19T06:02:41.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.448+0000 7f4e22ad6640 1 -- 172.21.15.5:0/3082031052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c100ca0 msgr2=0x7f4e1c1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:41.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.448+0000 7f4e22ad6640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c100ca0 0x7f4e1c1017b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400b450 tx=0x7f4e0400b850 comp rx=0 tx=0).stop 2024-09-19T06:02:41.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.448+0000 7f4e22ad6640 1 -- 172.21.15.5:0/3082031052 shutdown_connections 2024-09-19T06:02:41.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.448+0000 7f4e22ad6640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f4e080a3350 0x7f4e080a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:41.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.448+0000 7f4e22ad6640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c106f50 0x7f4e1c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:41.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.448+0000 7f4e22ad6640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c100ca0 0x7f4e1c1017b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:41.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.448+0000 7f4e22ad6640 1 --2- 172.21.15.5:0/3082031052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e1c1002f0 0x7f4e1c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:41.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.448+0000 7f4e22ad6640 1 -- 172.21.15.5:0/3082031052 >> 172.21.15.5:0/3082031052 conn(0x7f4e1c077210 msgr2=0x7f4e1c105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:41.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.449+0000 7f4e22ad6640 1 -- 172.21.15.5:0/3082031052 shutdown_connections 2024-09-19T06:02:41.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:41.449+0000 7f4e22ad6640 1 -- 172.21.15.5:0/3082031052 wait complete. 2024-09-19T06:02:42.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:02:42.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:42 smithi005 ceph-mon[30955]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:42.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:42 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3082031052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:02:42.433 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:42.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:42 smithi049 ceph-mon[34640]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:42.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:42 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3082031052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:02:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:42 smithi157 ceph-mon[35346]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:42 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3082031052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:02:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.866+0000 7f501ca39640 1 -- 172.21.15.5:0/694210143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5018106a80 msgr2=0x7f5018106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.866+0000 7f501ca39640 1 --2- 172.21.15.5:0/694210143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5018106a80 0x7f5018106e60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5008009930 tx=0x7f500802a420 comp rx=0 tx=0).stop 2024-09-19T06:02:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.866+0000 7f501ca39640 1 -- 172.21.15.5:0/694210143 shutdown_connections 2024-09-19T06:02:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.866+0000 7f501ca39640 1 --2- 172.21.15.5:0/694210143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018101420 0x7f501810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.866+0000 7f501ca39640 1 --2- 172.21.15.5:0/694210143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5018100a80 0x7f5018100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.866+0000 7f501ca39640 1 --2- 172.21.15.5:0/694210143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5018106a80 0x7f5018106e60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.866+0000 7f501ca39640 1 -- 172.21.15.5:0/694210143 >> 172.21.15.5:0/694210143 conn(0x7f50180fc7a0 msgr2=0x7f50180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.866+0000 7f501ca39640 1 -- 172.21.15.5:0/694210143 shutdown_connections 2024-09-19T06:02:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.866+0000 7f501ca39640 1 -- 172.21.15.5:0/694210143 wait complete. 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.867+0000 7f501ca39640 1 Processor -- start 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.867+0000 7f501ca39640 1 -- start start 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.867+0000 7f501ca39640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018100a80 0x7f5018199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.867+0000 7f501ca39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5018101420 0x7f50181996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f501ca39640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5018106a80 0x7f50181a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f501ca39640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5018110d30 con 0x7f5018101420 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f501ca39640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5018110bb0 con 0x7f5018100a80 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f501ca39640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5018110eb0 con 0x7f5018106a80 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f5015d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5018101420 0x7f50181996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f5015d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5018101420 0x7f50181996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50022/0 (socket says 172.21.15.5:50022) 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f5015d9b640 1 -- 172.21.15.5:0/2660610184 learned_addr learned my addr 172.21.15.5:0/2660610184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f501659c640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018100a80 0x7f5018199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f5016d9d640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5018106a80 0x7f50181a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f5015d9b640 1 -- 172.21.15.5:0/2660610184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5018106a80 msgr2=0x7f50181a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f5015d9b640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5018106a80 0x7f50181a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f5015d9b640 1 -- 172.21.15.5:0/2660610184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018100a80 msgr2=0x7f5018199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f5015d9b640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018100a80 0x7f5018199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f5015d9b640 1 -- 172.21.15.5:0/2660610184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50181a4740 con 0x7f5018101420 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f5016d9d640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5018106a80 0x7f50181a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.868+0000 7f5015d9b640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5018101420 0x7f50181996c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f500c00d770 tx=0x7f500c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.869+0000 7f50077fe640 1 -- 172.21.15.5:0/2660610184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c017070 con 0x7f5018101420 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.869+0000 7f501ca39640 1 -- 172.21.15.5:0/2660610184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50181a4910 con 0x7f5018101420 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.869+0000 7f50077fe640 1 -- 172.21.15.5:0/2660610184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f500c0054d0 con 0x7f5018101420 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.869+0000 7f50077fe640 1 -- 172.21.15.5:0/2660610184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c004cf0 con 0x7f5018101420 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.869+0000 7f501ca39640 1 -- 172.21.15.5:0/2660610184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50181a4e00 con 0x7f5018101420 2024-09-19T06:02:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.870+0000 7f501ca39640 1 -- 172.21.15.5:0/2660610184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5018100ee0 con 0x7f5018101420 2024-09-19T06:02:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.871+0000 7f50077fe640 1 -- 172.21.15.5:0/2660610184 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f500c004e90 con 0x7f5018101420 2024-09-19T06:02:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.871+0000 7f50077fe640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f50000a32d0 0x7f50000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.872+0000 7f501659c640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f50000a32d0 0x7f50000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.872+0000 7f50077fe640 1 -- 172.21.15.5:0/2660610184 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f500c0cf520 con 0x7f5018101420 2024-09-19T06:02:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.872+0000 7f501659c640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f50000a32d0 0x7f50000a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5008009900 tx=0x7f5008000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:42.876+0000 7f50077fe640 1 -- 172.21.15.5:0/2660610184 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f500c09c0d0 con 0x7f5018101420 2024-09-19T06:02:43.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.093+0000 7f501ca39640 1 -- 172.21.15.5:0/2660610184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5018108ff0 con 0x7f5018101420 2024-09-19T06:02:43.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.093+0000 7f50077fe640 1 -- 172.21.15.5:0/2660610184 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11959 (secure 0 0 0) 0x7f500c09b9d0 con 0x7f5018101420 2024-09-19T06:02:43.095 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:43.096 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-19T06:01:41:840933+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:01:41.840914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:02:43.096 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-19T06:02:43.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.096+0000 7f501ca39640 1 -- 172.21.15.5:0/2660610184 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f50000a32d0 msgr2=0x7f50000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:43.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.096+0000 7f501ca39640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f50000a32d0 0x7f50000a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5008009900 tx=0x7f5008000bc0 comp rx=0 tx=0).stop 2024-09-19T06:02:43.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.096+0000 7f501ca39640 1 -- 172.21.15.5:0/2660610184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5018101420 msgr2=0x7f50181996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:43.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.096+0000 7f501ca39640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5018101420 0x7f50181996c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f500c00d770 tx=0x7f500c00dc30 comp rx=0 tx=0).stop 2024-09-19T06:02:43.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.096+0000 7f501ca39640 1 -- 172.21.15.5:0/2660610184 shutdown_connections 2024-09-19T06:02:43.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.096+0000 7f501ca39640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f50000a32d0 0x7f50000a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:43.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.096+0000 7f501ca39640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5018106a80 0x7f50181a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:43.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.096+0000 7f501ca39640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5018101420 0x7f50181996c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:43.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.096+0000 7f501ca39640 1 --2- 172.21.15.5:0/2660610184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018100a80 0x7f5018199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:43.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.096+0000 7f501ca39640 1 -- 172.21.15.5:0/2660610184 >> 172.21.15.5:0/2660610184 conn(0x7f50180fc7a0 msgr2=0x7f50180fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:43.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.096+0000 7f501ca39640 1 -- 172.21.15.5:0/2660610184 shutdown_connections 2024-09-19T06:02:43.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:43.096+0000 7f501ca39640 1 -- 172.21.15.5:0/2660610184 wait complete. 2024-09-19T06:02:43.482 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:43 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2660610184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:02:43.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:43 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2660610184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:02:43.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:43 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2660610184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:02:43.828 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T06:01:29.531007+0000', 'modified': '2024-09-19T06:01:41.840914+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24473, 'mds_1': 24463, 'mds_2': 24448, 'mds_3': 24442, 'mds_4': 24437}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14625': {'gid': 14625, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/1310081153', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 1310081153}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 1310081153}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14637': {'gid': 14637, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/569764382', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 569764382}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 569764382}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6833/2427602354', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6832', 'nonce': 2427602354}, {'type': 'v1', 'addr': '172.21.15.49:6833', 'nonce': 2427602354}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6833/1048665474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 1048665474}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 1048665474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6835/2353689058', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 2353689058}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 2353689058}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.49:6837/2858664643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6836', 'nonce': 2858664643}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 2858664643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.157:6835/2070926870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 2070926870}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 2070926870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.157:6837/3264499329', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 3264499329}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 3264499329}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.157:6839/4294596962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 4294596962}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 4294596962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.49:6839/2359039211', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6838', 'nonce': 2359039211}, {'type': 'v1', 'addr': '172.21.15.49:6839', 'nonce': 2359039211}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24473, 'qdb_cluster': [24473, 24448, 24442, 24463, 24437]} 2024-09-19T06:02:43.828 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T06:02:43.828 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T06:02:44.091 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:44.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:44 smithi005 ceph-mon[30955]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.558+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3168333992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4106f10 msgr2=0x7f15d410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.558+0000 7f15dc3f9640 1 --2- 172.21.15.5:0/3168333992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4106f10 0x7f15d410ea90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009ca0 tx=0x7f15cc02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.558+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3168333992 shutdown_connections 2024-09-19T06:02:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.558+0000 7f15dc3f9640 1 --2- 172.21.15.5:0/3168333992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4106f10 0x7f15d410ea90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.558+0000 7f15dc3f9640 1 --2- 172.21.15.5:0/3168333992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d4102790 0x7f15d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.558+0000 7f15dc3f9640 1 --2- 172.21.15.5:0/3168333992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4101de0 0x7f15d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.558+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3168333992 >> 172.21.15.5:0/3168333992 conn(0x7f15d40fd8c0 msgr2=0x7f15d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3168333992 shutdown_connections 2024-09-19T06:02:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3168333992 wait complete. 2024-09-19T06:02:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15dc3f9640 1 Processor -- start 2024-09-19T06:02:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15dc3f9640 1 -- start start 2024-09-19T06:02:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15dc3f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4101de0 0x7f15d41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15dc3f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4102790 0x7f15d41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15dc3f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d4106f10 0x7f15d419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15dc3f9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4111e20 con 0x7f15d4101de0 2024-09-19T06:02:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15dc3f9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4111ca0 con 0x7f15d4106f10 2024-09-19T06:02:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15dc3f9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4111fa0 con 0x7f15d4102790 2024-09-19T06:02:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15da195640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4101de0 0x7f15d41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15da195640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4101de0 0x7f15d41983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51046/0 (socket says 172.21.15.5:51046) 2024-09-19T06:02:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.559+0000 7f15da195640 1 -- 172.21.15.5:0/3821172200 learned_addr learned my addr 172.21.15.5:0/3821172200 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15da996640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d4106f10 0x7f15d419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15d9994640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4102790 0x7f15d41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15da195640 1 -- 172.21.15.5:0/3821172200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4102790 msgr2=0x7f15d41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15da195640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4102790 0x7f15d41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15da195640 1 -- 172.21.15.5:0/3821172200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d4106f10 msgr2=0x7f15d419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15da195640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d4106f10 0x7f15d419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15da195640 1 -- 172.21.15.5:0/3821172200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d419d350 con 0x7f15d4101de0 2024-09-19T06:02:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15da996640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d4106f10 0x7f15d419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15da195640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4101de0 0x7f15d41983a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f15d000e9f0 tx=0x7f15d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15cb7fe640 1 -- 172.21.15.5:0/3821172200 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d000cd80 con 0x7f15d4101de0 2024-09-19T06:02:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15cb7fe640 1 -- 172.21.15.5:0/3821172200 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15d00054d0 con 0x7f15d4101de0 2024-09-19T06:02:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3821172200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d419d640 con 0x7f15d4101de0 2024-09-19T06:02:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15cb7fe640 1 -- 172.21.15.5:0/3821172200 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0010490 con 0x7f15d4101de0 2024-09-19T06:02:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.560+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3821172200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d4074420 con 0x7f15d4101de0 2024-09-19T06:02:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.563+0000 7f15de7d2740 1 Processor -- start 2024-09-19T06:02:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.563+0000 7f15cb7fe640 1 -- 172.21.15.5:0/3821172200 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f15d0005020 con 0x7f15d4101de0 2024-09-19T06:02:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.563+0000 7f15de7d2740 1 -- start start 2024-09-19T06:02:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15de7d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d729d0 0x55c8a8d72db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15de7d2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d732f0 0x55c8a8d7d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15de7d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d7ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15de7d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c8a8cd9090 con 0x55c8a8d729d0 2024-09-19T06:02:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15de7d2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c8a8c80a70 con 0x55c8a8d732f0 2024-09-19T06:02:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15de7d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c8a8cd5050 con 0x55c8a8d7dbb0 2024-09-19T06:02:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15da996640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d729d0 0x55c8a8d72db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15da996640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d729d0 0x55c8a8d72db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51048/0 (socket says 172.21.15.5:51048) 2024-09-19T06:02:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15da996640 1 -- 172.21.15.5:0/3817322880 learned_addr learned my addr 172.21.15.5:0/3817322880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15da195640 1 --2- 172.21.15.5:0/3817322880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d732f0 0x55c8a8d7d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15d9994640 1 --2- 172.21.15.5:0/3817322880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d7ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15da996640 1 -- 172.21.15.5:0/3817322880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d7dbb0 msgr2=0x55c8a8d7ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15da996640 1 --2- 172.21.15.5:0/3817322880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d7ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15da996640 1 -- 172.21.15.5:0/3817322880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d732f0 msgr2=0x55c8a8d7d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15da996640 1 --2- 172.21.15.5:0/3817322880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d732f0 0x55c8a8d7d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15da996640 1 -- 172.21.15.5:0/3817322880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8a8c8d130 con 0x55c8a8d729d0 2024-09-19T06:02:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15cb7fe640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f15c00a3350 0x7f15c00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15da996640 1 --2- 172.21.15.5:0/3817322880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d729d0 0x55c8a8d72db0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f15cc005ef0 tx=0x7f15cc005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2b4fe9d86dad77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.564+0000 7f15da195640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f15c00a3350 0x7f15c00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.565+0000 7f15cb7fe640 1 -- 172.21.15.5:0/3821172200 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f15d00cfef0 con 0x7f15d4101de0 2024-09-19T06:02:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.565+0000 7f15c8dea640 1 -- 172.21.15.5:0/3817322880 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc031cc0 con 0x55c8a8d729d0 2024-09-19T06:02:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.565+0000 7f15c8dea640 1 -- 172.21.15.5:0/3817322880 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15cc040070 con 0x55c8a8d729d0 2024-09-19T06:02:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.565+0000 7f15c8dea640 1 -- 172.21.15.5:0/3817322880 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc004330 con 0x55c8a8d729d0 2024-09-19T06:02:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.565+0000 7f15da195640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f15c00a3350 0x7f15c00a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55c8a8d6fdc0 tx=0x7f15d00d3a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.565+0000 7f15de7d2740 1 -- 172.21.15.5:0/3817322880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d729d0 msgr2=0x55c8a8d72db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.565+0000 7f15de7d2740 1 --2- 172.21.15.5:0/3817322880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d729d0 0x55c8a8d72db0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f15cc005ef0 tx=0x7f15cc005f20 comp rx=0 tx=0).stop 2024-09-19T06:02:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 -- 172.21.15.5:0/3817322880 shutdown_connections 2024-09-19T06:02:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 --2- 172.21.15.5:0/3817322880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d7ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 --2- 172.21.15.5:0/3817322880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d732f0 0x55c8a8d7d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 --2- 172.21.15.5:0/3817322880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d729d0 0x55c8a8d72db0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 -- 172.21.15.5:0/3817322880 >> 172.21.15.5:0/3817322880 conn(0x55c8a8d6ef60 msgr2=0x55c8a8d6f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 -- 172.21.15.5:0/3817322880 shutdown_connections 2024-09-19T06:02:44.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 -- 172.21.15.5:0/3817322880 wait complete. 2024-09-19T06:02:44.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 -- start start 2024-09-19T06:02:44.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 Processor -- start 2024-09-19T06:02:44.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d729d0 0x55c8a8d7d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 0x55c8a8d7a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d7acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c8a8d80780 con 0x55c8a8d732f0 2024-09-19T06:02:44.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c8a8c93f20 con 0x55c8a8d7dbb0 2024-09-19T06:02:44.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.566+0000 7f15de7d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c8a8d80900 con 0x55c8a8d729d0 2024-09-19T06:02:44.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15d9994640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 0x55c8a8d7a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15de7d2740 10 client.0 _ll_register_callbacks cb 0x55c8a8c5d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:02:44.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15d9994640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 0x55c8a8d7a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51056/0 (socket says 172.21.15.5:51056) 2024-09-19T06:02:44.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15d9994640 1 -- 172.21.15.5:0/3430837697 learned_addr learned my addr 172.21.15.5:0/3430837697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:44.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15da996640 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d729d0 0x55c8a8d7d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15da996640 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d7acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15d9994640 1 -- 172.21.15.5:0/3430837697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d729d0 msgr2=0x55c8a8d7d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:44.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15d9994640 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d729d0 0x55c8a8d7d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15d9994640 1 -- 172.21.15.5:0/3430837697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d7dbb0 msgr2=0x55c8a8d7acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:44.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15d9994640 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d7acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15d9994640 1 -- 172.21.15.5:0/3430837697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8a8a4a2b0 con 0x55c8a8d732f0 2024-09-19T06:02:44.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15da996640 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d7acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:44.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15d9994640 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 0x55c8a8d7a7b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f15c40103f0 tx=0x7f15c40108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d95bd858d718ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:44.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:02:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15bdffb640 1 -- 172.21.15.5:0/3430837697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4012920 con 0x55c8a8d732f0 2024-09-19T06:02:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15bdffb640 1 -- 172.21.15.5:0/3430837697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f15c4012ac0 con 0x55c8a8d732f0 2024-09-19T06:02:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15de7d2740 1 -- 172.21.15.5:0/3430837697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15cc040070 con 0x55c8a8d732f0 2024-09-19T06:02:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15bdffb640 1 -- 172.21.15.5:0/3430837697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f15c4012cc0 con 0x55c8a8d732f0 2024-09-19T06:02:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.567+0000 7f15de7d2740 10 client.14991 fetch_fsmap learned FSMap version 27 2024-09-19T06:02:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.568+0000 7f15de7d2740 1 -- 172.21.15.5:0/3430837697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8a8d8c100 con 0x55c8a8d732f0 2024-09-19T06:02:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.568+0000 7f15bdffb640 1 -- 172.21.15.5:0/3430837697 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15cc040070 con 0x55c8a8d732f0 2024-09-19T06:02:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.568+0000 7f15bdffb640 1 -- 172.21.15.5:0/3430837697 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f15c400f040 con 0x55c8a8d732f0 2024-09-19T06:02:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.568+0000 7f15de7d2740 10 client.14991 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:02:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.568+0000 7f15de7d2740 10 client.14991 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:02:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.568+0000 7f15de7d2740 20 client.14991 populate_metadata read hostname 'smithi005' 2024-09-19T06:02:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.568+0000 7f15de7d2740 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c8a8d8e770 0x55c8a8d8fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.568+0000 7f15de7d2740 4 client.14991 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:02:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.568+0000 7f15de7d2740 1 -- 172.21.15.5:0/3430837697 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8a8ce6600 con 0x55c8a8d8e770 2024-09-19T06:02:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.568+0000 7f15da996640 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c8a8d8e770 0x55c8a8d8fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.569+0000 7f15da996640 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c8a8d8e770 0x55c8a8d8fbe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66ebf1c0fceb569b server_cookie=89f6a96899ffcac4 in_seq=0 out_seq=0 2024-09-19T06:02:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.569+0000 7f15bdffb640 10 client.14991 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:02:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.569+0000 7f15bdffb640 1 -- 172.21.15.5:0/3430837697 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8a8ce6600 con 0x55c8a8d8e770 2024-09-19T06:02:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.569+0000 7f15bdffb640 10 client.14991 handle_command_reply: tid=0 2024-09-19T06:02:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.569+0000 7f15de7d2740 1 client.14991 shutdown 2024-09-19T06:02:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.570+0000 7f15de7d2740 1 -- 172.21.15.5:0/3430837697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 msgr2=0x55c8a8d7a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.570+0000 7f15de7d2740 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 0x55c8a8d7a7b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f15c40103f0 tx=0x7f15c40108b0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.570+0000 7f15de7d2740 1 -- 172.21.15.5:0/3430837697 shutdown_connections 2024-09-19T06:02:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.570+0000 7f15de7d2740 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c8a8d8e770 0x55c8a8d8fbe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.570+0000 7f15de7d2740 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d7acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.570+0000 7f15de7d2740 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 0x55c8a8d7a7b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.570+0000 7f15de7d2740 1 --2- 172.21.15.5:0/3430837697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d729d0 0x55c8a8d7d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.570+0000 7f15de7d2740 1 -- 172.21.15.5:0/3430837697 >> 172.21.15.5:0/3430837697 conn(0x55c8a8d6ef60 msgr2=0x55c8a8d74c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.570+0000 7f15bdffb640 0 client.14991 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:02:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.570+0000 7f15de7d2740 1 -- 172.21.15.5:0/3430837697 shutdown_connections 2024-09-19T06:02:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.570+0000 7f15de7d2740 1 -- 172.21.15.5:0/3430837697 wait complete. 2024-09-19T06:02:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.570+0000 7f15de7d2740 20 client.14991 trim_cache size 0 max 16384 2024-09-19T06:02:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15de7d2740 1 Processor -- start 2024-09-19T06:02:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15de7d2740 1 -- start start 2024-09-19T06:02:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15de7d2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d729d0 0x55c8a8d8fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15de7d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 0x55c8a8d6f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15de7d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d97ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15de7d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c8a8d8df20 con 0x55c8a8d732f0 2024-09-19T06:02:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15de7d2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c8a8d8e5f0 con 0x55c8a8d729d0 2024-09-19T06:02:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15de7d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c8a8d8ecc0 con 0x55c8a8d7dbb0 2024-09-19T06:02:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15d9994640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 0x55c8a8d6f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15d9994640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 0x55c8a8d6f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51068/0 (socket says 172.21.15.5:51068) 2024-09-19T06:02:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15d9994640 1 -- 172.21.15.5:0/276793988 learned_addr learned my addr 172.21.15.5:0/276793988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15da996640 1 --2- 172.21.15.5:0/276793988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d729d0 0x55c8a8d8fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15d9994640 1 -- 172.21.15.5:0/276793988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d7dbb0 msgr2=0x55c8a8d97ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15d9994640 1 --2- 172.21.15.5:0/276793988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d97ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15d9994640 1 -- 172.21.15.5:0/276793988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d729d0 msgr2=0x55c8a8d8fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15d9994640 1 --2- 172.21.15.5:0/276793988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d729d0 0x55c8a8d8fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15d9994640 1 -- 172.21.15.5:0/276793988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8a8d72800 con 0x55c8a8d732f0 2024-09-19T06:02:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15da996640 1 --2- 172.21.15.5:0/276793988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d729d0 0x55c8a8d8fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.588+0000 7f15d9994640 1 --2- 172.21.15.5:0/276793988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 0x55c8a8d6f340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f15c4026580 tx=0x7f15c4005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1330e1ac0f9c950e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.589+0000 7f15bffff640 1 -- 172.21.15.5:0/276793988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4012e60 con 0x55c8a8d732f0 2024-09-19T06:02:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.589+0000 7f15bffff640 1 -- 172.21.15.5:0/276793988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15c40111c0 con 0x55c8a8d732f0 2024-09-19T06:02:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.589+0000 7f15bffff640 1 -- 172.21.15.5:0/276793988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4026990 con 0x55c8a8d732f0 2024-09-19T06:02:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.589+0000 7f15de7d2740 1 -- 172.21.15.5:0/276793988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 msgr2=0x55c8a8d6f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.589+0000 7f15de7d2740 1 --2- 172.21.15.5:0/276793988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 0x55c8a8d6f340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f15c4026580 tx=0x7f15c4005690 comp rx=0 tx=0).stop 2024-09-19T06:02:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.589+0000 7f15de7d2740 1 -- 172.21.15.5:0/276793988 shutdown_connections 2024-09-19T06:02:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.589+0000 7f15de7d2740 1 --2- 172.21.15.5:0/276793988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d97ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.589+0000 7f15de7d2740 1 --2- 172.21.15.5:0/276793988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d732f0 0x55c8a8d6f340 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.589+0000 7f15de7d2740 1 --2- 172.21.15.5:0/276793988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d729d0 0x55c8a8d8fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.589+0000 7f15de7d2740 1 -- 172.21.15.5:0/276793988 >> 172.21.15.5:0/276793988 conn(0x55c8a8d6ef60 msgr2=0x55c8a8d798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.589+0000 7f15de7d2740 1 -- 172.21.15.5:0/276793988 shutdown_connections 2024-09-19T06:02:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.589+0000 7f15de7d2740 1 -- 172.21.15.5:0/276793988 wait complete. 2024-09-19T06:02:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15de7d2740 1 -- start start 2024-09-19T06:02:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15de7d2740 1 Processor -- start 2024-09-19T06:02:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15de7d2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d729d0 0x55c8a8d98410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15de7d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d732f0 0x55c8a8d93790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15de7d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d93cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15de7d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c8a8d8fa60 con 0x55c8a8d7dbb0 2024-09-19T06:02:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15de7d2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c8a8d8f390 con 0x55c8a8d729d0 2024-09-19T06:02:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15de7d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c8a8d90130 con 0x55c8a8d732f0 2024-09-19T06:02:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15de7d2740 10 client.0 _ll_register_callbacks cb 0x7f15c4005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:02:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15da996640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d93cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15da996640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d93cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51080/0 (socket says 172.21.15.5:51080) 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15da996640 1 -- 172.21.15.5:0/322784682 learned_addr learned my addr 172.21.15.5:0/322784682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15da996640 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d729d0 0x55c8a8d98410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15d9994640 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d732f0 0x55c8a8d93790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15da996640 1 -- 172.21.15.5:0/322784682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d732f0 msgr2=0x55c8a8d93790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15da996640 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d732f0 0x55c8a8d93790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15da996640 1 -- 172.21.15.5:0/322784682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d729d0 msgr2=0x55c8a8d98410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.590+0000 7f15da996640 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d729d0 0x55c8a8d98410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15da996640 1 -- 172.21.15.5:0/322784682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8a8d94310 con 0x55c8a8d7dbb0 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15da996640 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d93cd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009a60 tx=0x7f15cc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2059f00f5fd7b5d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15bdffb640 1 -- 172.21.15.5:0/322784682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc031450 con 0x55c8a8d7dbb0 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15bdffb640 1 -- 172.21.15.5:0/322784682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f15cc0315f0 con 0x55c8a8d7dbb0 2024-09-19T06:02:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15de7d2740 1 -- 172.21.15.5:0/322784682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15c40111c0 con 0x55c8a8d7dbb0 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15bdffb640 1 -- 172.21.15.5:0/322784682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f15cc0317f0 con 0x55c8a8d7dbb0 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15de7d2740 10 client.14997 fetch_fsmap learned FSMap version 27 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15de7d2740 1 -- 172.21.15.5:0/322784682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8a8df8c00 con 0x55c8a8d7dbb0 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15bdffb640 1 -- 172.21.15.5:0/322784682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15c40111c0 con 0x55c8a8d7dbb0 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15bdffb640 1 -- 172.21.15.5:0/322784682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f15cc0305c0 con 0x55c8a8d7dbb0 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15de7d2740 10 client.14997 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15de7d2740 10 client.14997 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.591+0000 7f15de7d2740 20 client.14997 populate_metadata read hostname 'smithi005' 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.592+0000 7f15de7d2740 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c8a8dfd300 0x55c8a8dfc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.592+0000 7f15de7d2740 4 client.14997 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.592+0000 7f15de7d2740 1 -- 172.21.15.5:0/322784682 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "status"}) -- 0x55c8a8ce0b10 con 0x55c8a8dfd300 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.592+0000 7f15d9994640 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c8a8dfd300 0x55c8a8dfc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.592+0000 7f15d9994640 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c8a8dfd300 0x55c8a8dfc010 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6cdee443873e3dee server_cookie=c98add0b5143e566 in_seq=0 out_seq=0 2024-09-19T06:02:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.592+0000 7f15bdffb640 10 client.14997 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.592+0000 7f15bdffb640 1 -- 172.21.15.5:0/322784682 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c8a8ce0b10 con 0x55c8a8dfd300 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.592+0000 7f15bdffb640 10 client.14997 handle_command_reply: tid=0 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.592+0000 7f15de7d2740 1 client.14997 shutdown 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15de7d2740 1 -- 172.21.15.5:0/322784682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d7dbb0 msgr2=0x55c8a8d93cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15de7d2740 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d93cd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009a60 tx=0x7f15cc02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15de7d2740 1 -- 172.21.15.5:0/322784682 shutdown_connections 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15de7d2740 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c8a8dfd300 0x55c8a8dfc010 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15de7d2740 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8a8d7dbb0 0x55c8a8d93cd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15de7d2740 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c8a8d732f0 0x55c8a8d93790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15de7d2740 1 --2- 172.21.15.5:0/322784682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8a8d729d0 0x55c8a8d98410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15de7d2740 1 -- 172.21.15.5:0/322784682 >> 172.21.15.5:0/322784682 conn(0x55c8a8d6ef60 msgr2=0x55c8a8d6d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15bdffb640 0 client.14997 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:02:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15de7d2740 1 -- 172.21.15.5:0/322784682 shutdown_connections 2024-09-19T06:02:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15de7d2740 1 -- 172.21.15.5:0/322784682 wait complete. 2024-09-19T06:02:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15de7d2740 20 client.14997 trim_cache size 0 max 16384 2024-09-19T06:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T06:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-09-19T06:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24437, 2024-09-19T06:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-19T06:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-19T06:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-19T06:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 63.755271796999999, 2024-09-19T06:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-19T06:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-19T06:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 96.572937961999997 2024-09-19T06:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:02:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3821172200 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f15c00a3350 msgr2=0x7f15c00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15dc3f9640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f15c00a3350 0x7f15c00a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55c8a8d6fdc0 tx=0x7f15d00d3a40 comp rx=0 tx=0).stop 2024-09-19T06:02:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3821172200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4101de0 msgr2=0x7f15d41983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15dc3f9640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4101de0 0x7f15d41983a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f15d000e9f0 tx=0x7f15d000edf0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3821172200 shutdown_connections 2024-09-19T06:02:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15dc3f9640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f15c00a3350 0x7f15c00a5810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15dc3f9640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d4106f10 0x7f15d419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15dc3f9640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4102790 0x7f15d41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15dc3f9640 1 --2- 172.21.15.5:0/3821172200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4101de0 0x7f15d41983a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.593+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3821172200 >> 172.21.15.5:0/3821172200 conn(0x7f15d40fd8c0 msgr2=0x7f15d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.594+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3821172200 shutdown_connections 2024-09-19T06:02:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:44.594+0000 7f15dc3f9640 1 -- 172.21.15.5:0/3821172200 wait complete. 2024-09-19T06:02:44.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:44 smithi049 ceph-mon[34640]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:44 smithi157 ceph-mon[35346]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:45.394 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T06:02:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.108+0000 7f003f913640 1 -- 172.21.15.5:0/437280721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0038106f10 msgr2=0x7f003810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.108+0000 7f003f913640 1 --2- 172.21.15.5:0/437280721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0038106f10 0x7f003810ea90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0030009ca0 tx=0x7f003002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.108+0000 7f003f913640 1 -- 172.21.15.5:0/437280721 shutdown_connections 2024-09-19T06:02:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.108+0000 7f003f913640 1 --2- 172.21.15.5:0/437280721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0038106f10 0x7f003810ea90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.108+0000 7f003f913640 1 --2- 172.21.15.5:0/437280721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038102790 0x7f00381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.108+0000 7f003f913640 1 --2- 172.21.15.5:0/437280721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0038101de0 0x7f00381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.108+0000 7f003f913640 1 -- 172.21.15.5:0/437280721 >> 172.21.15.5:0/437280721 conn(0x7f00380fd8c0 msgr2=0x7f00380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.108+0000 7f003f913640 1 -- 172.21.15.5:0/437280721 shutdown_connections 2024-09-19T06:02:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.108+0000 7f003f913640 1 -- 172.21.15.5:0/437280721 wait complete. 2024-09-19T06:02:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.108+0000 7f003f913640 1 Processor -- start 2024-09-19T06:02:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003f913640 1 -- start start 2024-09-19T06:02:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003f913640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0038101de0 0x7f00380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003f913640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038102790 0x7f0038074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003f913640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0038106f10 0x7f0038074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003f913640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0038111e40 con 0x7f0038101de0 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003f913640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0038111cc0 con 0x7f0038102790 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003f913640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0038111fc0 con 0x7f0038106f10 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003d6af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0038101de0 0x7f00380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003d6af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0038101de0 0x7f00380744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51088/0 (socket says 172.21.15.5:51088) 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003d6af640 1 -- 172.21.15.5:0/3847526583 learned_addr learned my addr 172.21.15.5:0/3847526583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003ceae640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038102790 0x7f0038074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003deb0640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0038106f10 0x7f0038074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003d6af640 1 -- 172.21.15.5:0/3847526583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0038106f10 msgr2=0x7f0038074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003d6af640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0038106f10 0x7f0038074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003d6af640 1 -- 172.21.15.5:0/3847526583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038102790 msgr2=0x7f0038074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003d6af640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038102790 0x7f0038074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003d6af640 1 -- 172.21.15.5:0/3847526583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00381a6840 con 0x7f0038101de0 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003deb0640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0038106f10 0x7f0038074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.109+0000 7f003d6af640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0038101de0 0x7f00380744f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f003400b4a0 tx=0x7f003400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.110+0000 7f002e7fc640 1 -- 172.21.15.5:0/3847526583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034014070 con 0x7f0038101de0 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.110+0000 7f002e7fc640 1 -- 172.21.15.5:0/3847526583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0034010ac0 con 0x7f0038101de0 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.110+0000 7f003f913640 1 -- 172.21.15.5:0/3847526583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00381a6b30 con 0x7f0038101de0 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.110+0000 7f002e7fc640 1 -- 172.21.15.5:0/3847526583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034017c30 con 0x7f0038101de0 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.110+0000 7f003f913640 1 -- 172.21.15.5:0/3847526583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00381a7070 con 0x7f0038101de0 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.112+0000 7f002e7fc640 1 -- 172.21.15.5:0/3847526583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f0034017070 con 0x7f0038101de0 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.112+0000 7f0041cec740 1 Processor -- start 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.112+0000 7f0041cec740 1 -- start start 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f0041cec740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 0x561d5d357db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f0041cec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3582f0 0x561d5d362670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f0041cec740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 0x561d5d364fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f0041cec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561d5d2be090 con 0x561d5d3582f0 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f0041cec740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561d5d265a70 con 0x561d5d3579d0 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f0041cec740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561d5d2ba050 con 0x561d5d362bb0 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f003deb0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 0x561d5d357db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f003deb0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 0x561d5d357db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57262/0 (socket says 172.21.15.5:57262) 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f003deb0640 1 -- 172.21.15.5:0/2151074825 learned_addr learned my addr 172.21.15.5:0/2151074825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f003d6af640 1 --2- 172.21.15.5:0/2151074825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3582f0 0x561d5d362670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f003ceae640 1 --2- 172.21.15.5:0/2151074825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 0x561d5d364fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f002e7fc640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f00240a32d0 0x7f00240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f003deb0640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f00240a32d0 0x7f00240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f002e7fc640 1 -- 172.21.15.5:0/3847526583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f00340cefd0 con 0x7f0038101de0 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f003deb0640 1 -- 172.21.15.5:0/2151074825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 msgr2=0x561d5d364fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f003deb0640 1 --2- 172.21.15.5:0/2151074825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 0x561d5d364fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f003deb0640 1 -- 172.21.15.5:0/2151074825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3582f0 msgr2=0x561d5d362670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f003deb0640 1 --2- 172.21.15.5:0/2151074825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3582f0 0x561d5d362670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f003deb0640 1 -- 172.21.15.5:0/2151074825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d5d272130 con 0x561d5d3579d0 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.113+0000 7f003ceae640 1 --2- 172.21.15.5:0/2151074825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 0x561d5d364fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.114+0000 7f003deb0640 1 --2- 172.21.15.5:0/2151074825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 0x561d5d357db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0030031530 tx=0x7f0030031560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6e63c10cf4d2ac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.114+0000 7f003deb0640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f00240a32d0 0x7f00240a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0030046000 tx=0x7f0030048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.114+0000 7f00237fe640 1 -- 172.21.15.5:0/2151074825 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030004040 con 0x561d5d3579d0 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.114+0000 7f00237fe640 1 -- 172.21.15.5:0/2151074825 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0030036070 con 0x561d5d3579d0 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.114+0000 7f00237fe640 1 -- 172.21.15.5:0/2151074825 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030030c90 con 0x561d5d3579d0 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.115+0000 7f0041cec740 1 -- 172.21.15.5:0/2151074825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 msgr2=0x561d5d357db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.115+0000 7f0041cec740 1 --2- 172.21.15.5:0/2151074825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 0x561d5d357db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0030031530 tx=0x7f0030031560 comp rx=0 tx=0).stop 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.115+0000 7f0041cec740 1 -- 172.21.15.5:0/2151074825 shutdown_connections 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.115+0000 7f0041cec740 1 --2- 172.21.15.5:0/2151074825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 0x561d5d364fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.115+0000 7f0041cec740 1 --2- 172.21.15.5:0/2151074825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3582f0 0x561d5d362670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.115+0000 7f0041cec740 1 --2- 172.21.15.5:0/2151074825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 0x561d5d357db0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.115+0000 7f0041cec740 1 -- 172.21.15.5:0/2151074825 >> 172.21.15.5:0/2151074825 conn(0x561d5d353f60 msgr2=0x561d5d354340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.115+0000 7f0041cec740 1 -- 172.21.15.5:0/2151074825 shutdown_connections 2024-09-19T06:02:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.115+0000 7f0041cec740 1 -- 172.21.15.5:0/2151074825 wait complete. 2024-09-19T06:02:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.115+0000 7f0041cec740 1 -- start start 2024-09-19T06:02:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.115+0000 7f0041cec740 1 Processor -- start 2024-09-19T06:02:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f0041cec740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d3579d0 0x561d5d362480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f0041cec740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3582f0 0x561d5d35f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f0041cec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d362bb0 0x561d5d35fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f0041cec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561d5d365720 con 0x561d5d362bb0 2024-09-19T06:02:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f0041cec740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561d5d278f20 con 0x561d5d3582f0 2024-09-19T06:02:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f0041cec740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561d5d3658a0 con 0x561d5d3579d0 2024-09-19T06:02:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f0041cec740 10 client.0 _ll_register_callbacks cb 0x561d5d242030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:02:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003deb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d362bb0 0x561d5d35fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003deb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d362bb0 0x561d5d35fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51106/0 (socket says 172.21.15.5:51106) 2024-09-19T06:02:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003deb0640 1 -- 172.21.15.5:0/4115826110 learned_addr learned my addr 172.21.15.5:0/4115826110 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003ceae640 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3582f0 0x561d5d35f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003d6af640 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d3579d0 0x561d5d362480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003deb0640 1 -- 172.21.15.5:0/4115826110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d3579d0 msgr2=0x561d5d362480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003deb0640 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d3579d0 0x561d5d362480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003deb0640 1 -- 172.21.15.5:0/4115826110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3582f0 msgr2=0x561d5d35f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003deb0640 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3582f0 0x561d5d35f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003deb0640 1 -- 172.21.15.5:0/4115826110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5d02f2b0 con 0x561d5d362bb0 2024-09-19T06:02:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003d6af640 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d3579d0 0x561d5d362480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003ceae640 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3582f0 0x561d5d35f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:02:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f003deb0640 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d362bb0 0x561d5d35fd10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0030030d60 tx=0x7f00300095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25f51fbb68418f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f0020ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:02:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f0020ff9640 1 -- 172.21.15.5:0/4115826110 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030059070 con 0x561d5d362bb0 2024-09-19T06:02:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f0020ff9640 1 -- 172.21.15.5:0/4115826110 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0030004240 con 0x561d5d362bb0 2024-09-19T06:02:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.116+0000 7f0041cec740 1 -- 172.21.15.5:0/4115826110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0030036070 con 0x561d5d362bb0 2024-09-19T06:02:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.117+0000 7f0020ff9640 1 -- 172.21.15.5:0/4115826110 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0030065050 con 0x561d5d362bb0 2024-09-19T06:02:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.117+0000 7f0041cec740 10 client.15009 fetch_fsmap learned FSMap version 27 2024-09-19T06:02:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.117+0000 7f0041cec740 1 -- 172.21.15.5:0/4115826110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561d5d365720 con 0x561d5d362bb0 2024-09-19T06:02:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.117+0000 7f0020ff9640 1 -- 172.21.15.5:0/4115826110 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0030036070 con 0x561d5d362bb0 2024-09-19T06:02:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.117+0000 7f0020ff9640 1 -- 172.21.15.5:0/4115826110 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f003002f970 con 0x561d5d362bb0 2024-09-19T06:02:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.117+0000 7f0041cec740 10 client.15009 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:02:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.117+0000 7f0041cec740 10 client.15009 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T06:02:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.117+0000 7f0041cec740 20 client.15009 populate_metadata read hostname 'smithi005' 2024-09-19T06:02:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.117+0000 7f0041cec740 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561d5d373240 0x561d5d3746b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.117+0000 7f0041cec740 4 client.15009 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.117+0000 7f0041cec740 1 -- 172.21.15.5:0/4115826110 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d5d2cb600 con 0x561d5d373240 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.117+0000 7f003d6af640 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561d5d373240 0x561d5d3746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.118+0000 7f003d6af640 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561d5d373240 0x561d5d3746b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=479e6a23c41c28d2 server_cookie=990c6991179e1f72 in_seq=0 out_seq=0 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.118+0000 7f0020ff9640 10 client.15009 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0020ff9640 1 -- 172.21.15.5:0/4115826110 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d5d2cb600 con 0x561d5d373240 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0020ff9640 10 client.15009 handle_command_reply: tid=0 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0041cec740 1 client.15009 shutdown 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0041cec740 1 -- 172.21.15.5:0/4115826110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d362bb0 msgr2=0x561d5d35fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0041cec740 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d362bb0 0x561d5d35fd10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0030030d60 tx=0x7f00300095c0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0041cec740 1 -- 172.21.15.5:0/4115826110 shutdown_connections 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0041cec740 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561d5d373240 0x561d5d3746b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0041cec740 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d362bb0 0x561d5d35fd10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0041cec740 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3582f0 0x561d5d35f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0020ff9640 0 client.15009 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:02:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0041cec740 1 --2- 172.21.15.5:0/4115826110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d3579d0 0x561d5d362480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0041cec740 1 -- 172.21.15.5:0/4115826110 >> 172.21.15.5:0/4115826110 conn(0x561d5d353f60 msgr2=0x561d5d359c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0041cec740 1 -- 172.21.15.5:0/4115826110 shutdown_connections 2024-09-19T06:02:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0041cec740 1 -- 172.21.15.5:0/4115826110 wait complete. 2024-09-19T06:02:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.119+0000 7f0041cec740 20 client.15009 trim_cache size 0 max 16384 2024-09-19T06:02:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f0041cec740 1 Processor -- start 2024-09-19T06:02:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f0041cec740 1 -- start start 2024-09-19T06:02:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f0041cec740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 0x561d5d373fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f0041cec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3582f0 0x561d5d37c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f0041cec740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 0x561d5d37ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f0041cec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561d5d3729f0 con 0x561d5d3582f0 2024-09-19T06:02:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f0041cec740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561d5d3730c0 con 0x561d5d3579d0 2024-09-19T06:02:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f0041cec740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561d5d373790 con 0x561d5d362bb0 2024-09-19T06:02:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f003ceae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3582f0 0x561d5d37c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f003ceae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3582f0 0x561d5d37c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51116/0 (socket says 172.21.15.5:51116) 2024-09-19T06:02:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f003ceae640 1 -- 172.21.15.5:0/1456991000 learned_addr learned my addr 172.21.15.5:0/1456991000 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f003ceae640 1 -- 172.21.15.5:0/1456991000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 msgr2=0x561d5d37ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f003d6af640 1 --2- 172.21.15.5:0/1456991000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 0x561d5d373fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f003ceae640 1 --2- 172.21.15.5:0/1456991000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 0x561d5d37ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.137+0000 7f003ceae640 1 -- 172.21.15.5:0/1456991000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 msgr2=0x561d5d373fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.138+0000 7f003ceae640 1 --2- 172.21.15.5:0/1456991000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 0x561d5d373fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.138+0000 7f003ceae640 1 -- 172.21.15.5:0/1456991000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d5d3620a0 con 0x561d5d3582f0 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.138+0000 7f003d6af640 1 --2- 172.21.15.5:0/1456991000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 0x561d5d373fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.138+0000 7f003ceae640 1 --2- 172.21.15.5:0/1456991000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3582f0 0x561d5d37c990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f00280058c0 tx=0x7f002800a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748f1ac0ec103e98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.138+0000 7f0022ffd640 1 -- 172.21.15.5:0/1456991000 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002801e030 con 0x561d5d3582f0 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.138+0000 7f0022ffd640 1 -- 172.21.15.5:0/1456991000 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00280054d0 con 0x561d5d3582f0 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.138+0000 7f0022ffd640 1 -- 172.21.15.5:0/1456991000 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002801a870 con 0x561d5d3582f0 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.138+0000 7f0041cec740 1 -- 172.21.15.5:0/1456991000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3582f0 msgr2=0x561d5d37c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.138+0000 7f0041cec740 1 --2- 172.21.15.5:0/1456991000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3582f0 0x561d5d37c990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f00280058c0 tx=0x7f002800a2d0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.139+0000 7f0041cec740 1 -- 172.21.15.5:0/1456991000 shutdown_connections 2024-09-19T06:02:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.139+0000 7f0041cec740 1 --2- 172.21.15.5:0/1456991000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 0x561d5d37ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.139+0000 7f0041cec740 1 --2- 172.21.15.5:0/1456991000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3582f0 0x561d5d37c990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.139+0000 7f0041cec740 1 --2- 172.21.15.5:0/1456991000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3579d0 0x561d5d373fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.139+0000 7f0041cec740 1 -- 172.21.15.5:0/1456991000 >> 172.21.15.5:0/1456991000 conn(0x561d5d353f60 msgr2=0x561d5d3721f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.139+0000 7f0041cec740 1 -- 172.21.15.5:0/1456991000 shutdown_connections 2024-09-19T06:02:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.139+0000 7f0041cec740 1 -- 172.21.15.5:0/1456991000 wait complete. 2024-09-19T06:02:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.139+0000 7f0041cec740 1 -- start start 2024-09-19T06:02:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.139+0000 7f0041cec740 1 Processor -- start 2024-09-19T06:02:46.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.139+0000 7f0041cec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3579d0 0x561d5d37ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.139+0000 7f0041cec740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3582f0 0x561d5d378140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.139+0000 7f0041cec740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 0x561d5d378680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f0041cec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561d5d374530 con 0x561d5d3579d0 2024-09-19T06:02:46.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f0041cec740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561d5d373e60 con 0x561d5d3582f0 2024-09-19T06:02:46.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f0041cec740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561d5d374c00 con 0x561d5d362bb0 2024-09-19T06:02:46.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f0041cec740 10 client.0 _ll_register_callbacks cb 0x7f002800a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f003d6af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3579d0 0x561d5d37ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f003d6af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3579d0 0x561d5d37ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51132/0 (socket says 172.21.15.5:51132) 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f003d6af640 1 -- 172.21.15.5:0/980609164 learned_addr learned my addr 172.21.15.5:0/980609164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f003deb0640 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 0x561d5d378680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f003ceae640 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3582f0 0x561d5d378140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f003d6af640 1 -- 172.21.15.5:0/980609164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 msgr2=0x561d5d378680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f003d6af640 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 0x561d5d378680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f003d6af640 1 -- 172.21.15.5:0/980609164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3582f0 msgr2=0x561d5d378140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f003d6af640 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3582f0 0x561d5d378140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f003d6af640 1 -- 172.21.15.5:0/980609164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5d378bc0 con 0x561d5d3579d0 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f003d6af640 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3579d0 0x561d5d37ce10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f003409b330 tx=0x7f00340047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6595a846f660b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f0020ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:02:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f0020ff9640 1 -- 172.21.15.5:0/980609164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003401e040 con 0x561d5d3579d0 2024-09-19T06:02:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f0020ff9640 1 -- 172.21.15.5:0/980609164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f00340ccc60 con 0x561d5d3579d0 2024-09-19T06:02:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f0041cec740 1 -- 172.21.15.5:0/980609164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00280054d0 con 0x561d5d3579d0 2024-09-19T06:02:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f0020ff9640 1 -- 172.21.15.5:0/980609164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f003409f9e0 con 0x561d5d3579d0 2024-09-19T06:02:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f0041cec740 10 client.15015 fetch_fsmap learned FSMap version 27 2024-09-19T06:02:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.140+0000 7f0041cec740 1 -- 172.21.15.5:0/980609164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561d5d3dd610 con 0x561d5d3579d0 2024-09-19T06:02:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.141+0000 7f0020ff9640 1 -- 172.21.15.5:0/980609164 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00280054d0 con 0x561d5d3579d0 2024-09-19T06:02:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.141+0000 7f0020ff9640 1 -- 172.21.15.5:0/980609164 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f003409b890 con 0x561d5d3579d0 2024-09-19T06:02:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.141+0000 7f0041cec740 10 client.15015 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:02:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.141+0000 7f0041cec740 10 client.15015 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T06:02:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.141+0000 7f0041cec740 20 client.15015 populate_metadata read hostname 'smithi005' 2024-09-19T06:02:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.141+0000 7f0041cec740 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561d5d3e21f0 0x561d5d3e0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.141+0000 7f0041cec740 4 client.15015 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.141+0000 7f0041cec740 1 -- 172.21.15.5:0/980609164 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "status"}) -- 0x561d5d2c5b10 con 0x561d5d3e21f0 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.141+0000 7f003ceae640 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561d5d3e21f0 0x561d5d3e0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f003ceae640 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561d5d3e21f0 0x561d5d3e0f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d1c1fc50e65082b server_cookie=2f41b27a4752faaf in_seq=0 out_seq=0 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0020ff9640 10 client.15015 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0020ff9640 1 -- 172.21.15.5:0/980609164 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561d5d2c5b10 con 0x561d5d3e21f0 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0020ff9640 10 client.15015 handle_command_reply: tid=0 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0041cec740 1 client.15015 shutdown 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0041cec740 1 -- 172.21.15.5:0/980609164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3579d0 msgr2=0x561d5d37ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0041cec740 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3579d0 0x561d5d37ce10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f003409b330 tx=0x7f00340047d0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0041cec740 1 -- 172.21.15.5:0/980609164 shutdown_connections 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0041cec740 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561d5d3e21f0 0x561d5d3e0f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0041cec740 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561d5d362bb0 0x561d5d378680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0041cec740 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d5d3582f0 0x561d5d378140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0041cec740 1 --2- 172.21.15.5:0/980609164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5d3579d0 0x561d5d37ce10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0041cec740 1 -- 172.21.15.5:0/980609164 >> 172.21.15.5:0/980609164 conn(0x561d5d353f60 msgr2=0x561d5d3527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0020ff9640 0 client.15015 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:02:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0041cec740 1 -- 172.21.15.5:0/980609164 shutdown_connections 2024-09-19T06:02:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0041cec740 1 -- 172.21.15.5:0/980609164 wait complete. 2024-09-19T06:02:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.142+0000 7f0041cec740 20 client.15015 trim_cache size 0 max 16384 2024-09-19T06:02:46.150 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:02:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T06:02:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-09-19T06:02:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24442, 2024-09-19T06:02:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-19T06:02:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-19T06:02:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-19T06:02:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 67.315649159000003, 2024-09-19T06:02:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-19T06:02:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-19T06:02:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:02:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 103.21495516500001 2024-09-19T06:02:46.151 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:02:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.143+0000 7f003f913640 1 -- 172.21.15.5:0/3847526583 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f00240a32d0 msgr2=0x7f00240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.143+0000 7f003f913640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f00240a32d0 0x7f00240a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0030046000 tx=0x7f0030048000 comp rx=0 tx=0).stop 2024-09-19T06:02:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.143+0000 7f003f913640 1 -- 172.21.15.5:0/3847526583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0038101de0 msgr2=0x7f00380744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.143+0000 7f003f913640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0038101de0 0x7f00380744f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f003400b4a0 tx=0x7f003400b960 comp rx=0 tx=0).stop 2024-09-19T06:02:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.143+0000 7f003f913640 1 -- 172.21.15.5:0/3847526583 shutdown_connections 2024-09-19T06:02:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.143+0000 7f003f913640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f00240a32d0 0x7f00240a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.143+0000 7f003f913640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0038106f10 0x7f0038074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.143+0000 7f003f913640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038102790 0x7f0038074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.143+0000 7f003f913640 1 --2- 172.21.15.5:0/3847526583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0038101de0 0x7f00380744f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:46.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.143+0000 7f003f913640 1 -- 172.21.15.5:0/3847526583 >> 172.21.15.5:0/3847526583 conn(0x7f00380fd8c0 msgr2=0x7f00380feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:46.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.143+0000 7f003f913640 1 -- 172.21.15.5:0/3847526583 shutdown_connections 2024-09-19T06:02:46.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:46.143+0000 7f003f913640 1 -- 172.21.15.5:0/3847526583 wait complete. 2024-09-19T06:02:46.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:46 smithi005 ceph-mon[30955]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:46.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:46 smithi049 ceph-mon[34640]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:46 smithi157 ceph-mon[35346]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:46.835 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T06:02:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.548+0000 7fe839f80640 1 -- 172.21.15.5:0/1531582935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe83410c170 msgr2=0x7fe83410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.548+0000 7fe839f80640 1 --2- 172.21.15.5:0/1531582935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe83410c170 0x7fe83410e610 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009ca0 tx=0x7fe82c02a950 comp rx=0 tx=0).stop 2024-09-19T06:02:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.548+0000 7fe839f80640 1 -- 172.21.15.5:0/1531582935 shutdown_connections 2024-09-19T06:02:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.548+0000 7fe839f80640 1 --2- 172.21.15.5:0/1531582935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe83410c170 0x7fe83410e610 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.548+0000 7fe839f80640 1 --2- 172.21.15.5:0/1531582935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe834103660 0x7fe83410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.548+0000 7fe839f80640 1 --2- 172.21.15.5:0/1531582935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe834102d40 0x7fe834103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.548+0000 7fe839f80640 1 -- 172.21.15.5:0/1531582935 >> 172.21.15.5:0/1531582935 conn(0x7fe8340fc7a0 msgr2=0x7fe8340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.548+0000 7fe839f80640 1 -- 172.21.15.5:0/1531582935 shutdown_connections 2024-09-19T06:02:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.548+0000 7fe839f80640 1 -- 172.21.15.5:0/1531582935 wait complete. 2024-09-19T06:02:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.549+0000 7fe839f80640 1 Processor -- start 2024-09-19T06:02:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.549+0000 7fe839f80640 1 -- start start 2024-09-19T06:02:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.549+0000 7fe839f80640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834102d40 0x7fe8341009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.549+0000 7fe839f80640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe834103660 0x7fe834100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.549+0000 7fe839f80640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83410c170 0x7fe834101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.549+0000 7fe839f80640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe834111e20 con 0x7fe834102d40 2024-09-19T06:02:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.549+0000 7fe839f80640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe834111ca0 con 0x7fe834103660 2024-09-19T06:02:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.549+0000 7fe839f80640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe834111fa0 con 0x7fe83410c170 2024-09-19T06:02:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe8337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834102d40 0x7fe8341009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe8337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834102d40 0x7fe8341009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51156/0 (socket says 172.21.15.5:51156) 2024-09-19T06:02:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe8337fe640 1 -- 172.21.15.5:0/3822203881 learned_addr learned my addr 172.21.15.5:0/3822203881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe82bfff640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe834103660 0x7fe834100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe8337fe640 1 -- 172.21.15.5:0/3822203881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83410c170 msgr2=0x7fe834101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe8337fe640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83410c170 0x7fe834101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe8337fe640 1 -- 172.21.15.5:0/3822203881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe834103660 msgr2=0x7fe834100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe8337fe640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe834103660 0x7fe834100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe8337fe640 1 -- 172.21.15.5:0/3822203881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8341980e0 con 0x7fe834102d40 2024-09-19T06:02:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe8337fe640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834102d40 0x7fe8341009a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe82400d950 tx=0x7fe82400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe8317fa640 1 -- 172.21.15.5:0/3822203881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824017070 con 0x7fe834102d40 2024-09-19T06:02:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe8317fa640 1 -- 172.21.15.5:0/3822203881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe824009c30 con 0x7fe834102d40 2024-09-19T06:02:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe839f80640 1 -- 172.21.15.5:0/3822203881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8341983d0 con 0x7fe834102d40 2024-09-19T06:02:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe8317fa640 1 -- 172.21.15.5:0/3822203881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82400f620 con 0x7fe834102d40 2024-09-19T06:02:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.550+0000 7fe839f80640 1 -- 172.21.15.5:0/3822203881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe834198910 con 0x7fe834102d40 2024-09-19T06:02:47.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.553+0000 7fe8317fa640 1 -- 172.21.15.5:0/3822203881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fe8240109a0 con 0x7fe834102d40 2024-09-19T06:02:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe83c359740 1 Processor -- start 2024-09-19T06:02:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe83c359740 1 -- start start 2024-09-19T06:02:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe83c359740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abd9d0 0x55f0a4abddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe83c359740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ac8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe83c359740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4acafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe83c359740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4a24090 con 0x55f0a4abd9d0 2024-09-19T06:02:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe83c359740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f0a49cba70 con 0x55f0a4ac8bb0 2024-09-19T06:02:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe83c359740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4a20050 con 0x55f0a4abe2f0 2024-09-19T06:02:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe833fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abd9d0 0x55f0a4abddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe8317fa640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe8200a32d0 0x7fe8200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe833fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abd9d0 0x55f0a4abddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51162/0 (socket says 172.21.15.5:51162) 2024-09-19T06:02:47.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe833fff640 1 -- 172.21.15.5:0/1887898155 learned_addr learned my addr 172.21.15.5:0/1887898155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:47.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe8337fe640 1 --2- 172.21.15.5:0/1887898155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ac8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe833fff640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe8200a32d0 0x7fe8200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe82bfff640 1 --2- 172.21.15.5:0/1887898155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4acafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.554+0000 7fe8317fa640 1 -- 172.21.15.5:0/3822203881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe8240cf6f0 con 0x7fe834102d40 2024-09-19T06:02:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.555+0000 7fe833fff640 1 -- 172.21.15.5:0/1887898155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4abe2f0 msgr2=0x55f0a4ac8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.555+0000 7fe833fff640 1 --2- 172.21.15.5:0/1887898155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ac8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.555+0000 7fe833fff640 1 -- 172.21.15.5:0/1887898155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4ac8bb0 msgr2=0x55f0a4acafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.555+0000 7fe833fff640 1 --2- 172.21.15.5:0/1887898155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4acafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.555+0000 7fe833fff640 1 -- 172.21.15.5:0/1887898155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0a49d8130 con 0x55f0a4abd9d0 2024-09-19T06:02:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.555+0000 7fe833fff640 1 --2- 172.21.15.5:0/1887898155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abd9d0 0x55f0a4abddb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe82c005e30 tx=0x7fe82c02fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7297758a20ef55ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.555+0000 7fe829ffb640 1 -- 172.21.15.5:0/1887898155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c034040 con 0x55f0a4abd9d0 2024-09-19T06:02:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.555+0000 7fe829ffb640 1 -- 172.21.15.5:0/1887898155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe82c04c070 con 0x55f0a4abd9d0 2024-09-19T06:02:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.555+0000 7fe829ffb640 1 -- 172.21.15.5:0/1887898155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c03b470 con 0x55f0a4abd9d0 2024-09-19T06:02:47.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.556+0000 7fe83c359740 1 -- 172.21.15.5:0/1887898155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abd9d0 msgr2=0x55f0a4abddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:47.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.556+0000 7fe83c359740 1 --2- 172.21.15.5:0/1887898155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abd9d0 0x55f0a4abddb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe82c005e30 tx=0x7fe82c02fbb0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.556+0000 7fe833fff640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe8200a32d0 0x7fe8200a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009a60 tx=0x7fe82c04c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:47.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.556+0000 7fe83c359740 1 -- 172.21.15.5:0/1887898155 shutdown_connections 2024-09-19T06:02:47.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.556+0000 7fe83c359740 1 --2- 172.21.15.5:0/1887898155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4acafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.556+0000 7fe83c359740 1 --2- 172.21.15.5:0/1887898155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ac8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.556+0000 7fe83c359740 1 --2- 172.21.15.5:0/1887898155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abd9d0 0x55f0a4abddb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.556+0000 7fe83c359740 1 -- 172.21.15.5:0/1887898155 >> 172.21.15.5:0/1887898155 conn(0x55f0a4ab9f60 msgr2=0x55f0a4aba340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.556+0000 7fe83c359740 1 -- 172.21.15.5:0/1887898155 shutdown_connections 2024-09-19T06:02:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.556+0000 7fe83c359740 1 -- 172.21.15.5:0/1887898155 wait complete. 2024-09-19T06:02:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.556+0000 7fe83c359740 1 -- start start 2024-09-19T06:02:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe83c359740 1 Processor -- start 2024-09-19T06:02:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe83c359740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4abd9d0 0x55f0a4ac8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe83c359740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ac57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe83c359740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4ac5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe83c359740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4acb780 con 0x55f0a4abe2f0 2024-09-19T06:02:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe83c359740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f0a49def20 con 0x55f0a4abd9d0 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe83c359740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4acb900 con 0x55f0a4ac8bb0 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe83c359740 10 client.0 _ll_register_callbacks cb 0x55f0a49a8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe82bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ac57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe82bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ac57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51170/0 (socket says 172.21.15.5:51170) 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe8337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4abd9d0 0x55f0a4ac8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe82bfff640 1 -- 172.21.15.5:0/218418997 learned_addr learned my addr 172.21.15.5:0/218418997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe833fff640 1 --2- 172.21.15.5:0/218418997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4ac5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe82bfff640 1 -- 172.21.15.5:0/218418997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4ac8bb0 msgr2=0x55f0a4ac5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe82bfff640 1 --2- 172.21.15.5:0/218418997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4ac5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe82bfff640 1 -- 172.21.15.5:0/218418997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4abd9d0 msgr2=0x55f0a4ac8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe82bfff640 1 --2- 172.21.15.5:0/218418997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4abd9d0 0x55f0a4ac8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe82bfff640 1 -- 172.21.15.5:0/218418997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0a47952b0 con 0x55f0a4abe2f0 2024-09-19T06:02:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe82bfff640 1 --2- 172.21.15.5:0/218418997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ac57c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe81c010800 tx=0x7fe81c010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98572286a018b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.557+0000 7fe812ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe812ffd640 1 -- 172.21.15.5:0/218418997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c01e060 con 0x55f0a4abe2f0 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe812ffd640 1 -- 172.21.15.5:0/218418997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe81c00fc50 con 0x55f0a4abe2f0 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe83c359740 1 -- 172.21.15.5:0/218418997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe82c04c070 con 0x55f0a4abe2f0 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe812ffd640 1 -- 172.21.15.5:0/218418997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe81c00fe50 con 0x55f0a4abe2f0 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe83c359740 10 client.15027 fetch_fsmap learned FSMap version 27 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe83c359740 1 -- 172.21.15.5:0/218418997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f0a4ad70a0 con 0x55f0a4abe2f0 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe812ffd640 1 -- 172.21.15.5:0/218418997 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe82c04c070 con 0x55f0a4abe2f0 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe812ffd640 1 -- 172.21.15.5:0/218418997 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe81c0245f0 con 0x55f0a4abe2f0 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe83c359740 10 client.15027 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe83c359740 10 client.15027 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe83c359740 20 client.15027 populate_metadata read hostname 'smithi005' 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe83c359740 1 --2- 172.21.15.5:0/218418997 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f0a4ad9760 0x55f0a4adabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe83c359740 4 client.15027 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.558+0000 7fe83c359740 1 -- 172.21.15.5:0/218418997 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0a4a31600 con 0x55f0a4ad9760 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.559+0000 7fe8337fe640 1 --2- 172.21.15.5:0/218418997 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f0a4ad9760 0x55f0a4adabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.559+0000 7fe8337fe640 1 --2- 172.21.15.5:0/218418997 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f0a4ad9760 0x55f0a4adabd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=feefe928f5ad387d server_cookie=65ec12b8c3d3839f in_seq=0 out_seq=0 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.559+0000 7fe812ffd640 10 client.15027 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe812ffd640 1 -- 172.21.15.5:0/218418997 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0a4a31600 con 0x55f0a4ad9760 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe812ffd640 10 client.15027 handle_command_reply: tid=0 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe83c359740 1 client.15027 shutdown 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe83c359740 1 -- 172.21.15.5:0/218418997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 msgr2=0x55f0a4ac57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe83c359740 1 --2- 172.21.15.5:0/218418997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ac57c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe81c010800 tx=0x7fe81c010cc0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe83c359740 1 -- 172.21.15.5:0/218418997 shutdown_connections 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe83c359740 1 --2- 172.21.15.5:0/218418997 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f0a4ad9760 0x55f0a4adabd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe83c359740 1 --2- 172.21.15.5:0/218418997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4ac5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe83c359740 1 --2- 172.21.15.5:0/218418997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ac57c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe83c359740 1 --2- 172.21.15.5:0/218418997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4abd9d0 0x55f0a4ac8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe812ffd640 0 client.15027 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:02:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe83c359740 1 -- 172.21.15.5:0/218418997 >> 172.21.15.5:0/218418997 conn(0x55f0a4ab9f60 msgr2=0x55f0a4abfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe83c359740 1 -- 172.21.15.5:0/218418997 shutdown_connections 2024-09-19T06:02:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.560+0000 7fe83c359740 1 -- 172.21.15.5:0/218418997 wait complete. 2024-09-19T06:02:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.561+0000 7fe83c359740 20 client.15027 trim_cache size 0 max 16384 2024-09-19T06:02:47.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.578+0000 7fe83c359740 1 Processor -- start 2024-09-19T06:02:47.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.578+0000 7fe83c359740 1 -- start start 2024-09-19T06:02:47.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.578+0000 7fe83c359740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4abd9d0 0x55f0a4ad50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.578+0000 7fe83c359740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ae2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.578+0000 7fe83c359740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4ae33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.578+0000 7fe83c359740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4ad8f10 con 0x55f0a4abe2f0 2024-09-19T06:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.578+0000 7fe83c359740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4ad95e0 con 0x55f0a4abd9d0 2024-09-19T06:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.578+0000 7fe83c359740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4ad9cb0 con 0x55f0a4ac8bb0 2024-09-19T06:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.578+0000 7fe82bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ae2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.579+0000 7fe82bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ae2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51172/0 (socket says 172.21.15.5:51172) 2024-09-19T06:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.579+0000 7fe82bfff640 1 -- 172.21.15.5:0/1768827279 learned_addr learned my addr 172.21.15.5:0/1768827279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.579+0000 7fe82bfff640 1 -- 172.21.15.5:0/1768827279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4ac8bb0 msgr2=0x55f0a4ae33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.579+0000 7fe82bfff640 1 --2- 172.21.15.5:0/1768827279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4ae33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.579+0000 7fe82bfff640 1 -- 172.21.15.5:0/1768827279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4abd9d0 msgr2=0x55f0a4ad50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.579+0000 7fe82bfff640 1 --2- 172.21.15.5:0/1768827279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4abd9d0 0x55f0a4ad50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.579+0000 7fe82bfff640 1 -- 172.21.15.5:0/1768827279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0a4ac3d10 con 0x55f0a4abe2f0 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.579+0000 7fe82bfff640 1 --2- 172.21.15.5:0/1768827279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ae2e80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe81c010010 tx=0x7fe81c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff32eda154562d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.579+0000 7fe8297fa640 1 -- 172.21.15.5:0/1768827279 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c000e60 con 0x55f0a4abe2f0 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.579+0000 7fe8297fa640 1 -- 172.21.15.5:0/1768827279 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe81c005030 con 0x55f0a4abe2f0 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.579+0000 7fe8297fa640 1 -- 172.21.15.5:0/1768827279 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c004950 con 0x55f0a4abe2f0 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.580+0000 7fe83c359740 1 -- 172.21.15.5:0/1768827279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 msgr2=0x55f0a4ae2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.580+0000 7fe83c359740 1 --2- 172.21.15.5:0/1768827279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ae2e80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe81c010010 tx=0x7fe81c0058b0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.580+0000 7fe83c359740 1 -- 172.21.15.5:0/1768827279 shutdown_connections 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.580+0000 7fe83c359740 1 --2- 172.21.15.5:0/1768827279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4ae33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.580+0000 7fe83c359740 1 --2- 172.21.15.5:0/1768827279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ae2e80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.580+0000 7fe83c359740 1 --2- 172.21.15.5:0/1768827279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4abd9d0 0x55f0a4ad50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.580+0000 7fe83c359740 1 -- 172.21.15.5:0/1768827279 >> 172.21.15.5:0/1768827279 conn(0x55f0a4ab9f60 msgr2=0x55f0a4ac4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.580+0000 7fe83c359740 1 -- 172.21.15.5:0/1768827279 shutdown_connections 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.580+0000 7fe83c359740 1 -- 172.21.15.5:0/1768827279 wait complete. 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.580+0000 7fe83c359740 1 -- start start 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.580+0000 7fe83c359740 1 Processor -- start 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe83c359740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4abd9d0 0x55f0a4ae3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe83c359740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ae36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe83c359740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4ade570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe83c359740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4adaa50 con 0x55f0a4abe2f0 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe83c359740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4ada380 con 0x55f0a4ac8bb0 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe83c359740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4adb120 con 0x55f0a4abd9d0 2024-09-19T06:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe83c359740 10 client.0 _ll_register_callbacks cb 0x7fe81c00a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe82bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ae36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe82bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ae36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51180/0 (socket says 172.21.15.5:51180) 2024-09-19T06:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe82bfff640 1 -- 172.21.15.5:0/349489277 learned_addr learned my addr 172.21.15.5:0/349489277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe833fff640 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4ade570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe8337fe640 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4abd9d0 0x55f0a4ae3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe82bfff640 1 -- 172.21.15.5:0/349489277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4abd9d0 msgr2=0x55f0a4ae3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe82bfff640 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4abd9d0 0x55f0a4ae3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe82bfff640 1 -- 172.21.15.5:0/349489277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4ac8bb0 msgr2=0x55f0a4ade570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe82bfff640 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4ade570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe82bfff640 1 -- 172.21.15.5:0/349489277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0a4aded70 con 0x55f0a4abe2f0 2024-09-19T06:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe82bfff640 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ae36d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe81c005940 tx=0x7fe81c0104d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf4af052a4c818c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe812ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.581+0000 7fe812ffd640 1 -- 172.21.15.5:0/349489277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c0051e0 con 0x55f0a4abe2f0 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe812ffd640 1 -- 172.21.15.5:0/349489277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe81c005380 con 0x55f0a4abe2f0 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe83c359740 1 -- 172.21.15.5:0/349489277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe81c005030 con 0x55f0a4abe2f0 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe812ffd640 1 -- 172.21.15.5:0/349489277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe81c0055c0 con 0x55f0a4abe2f0 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe83c359740 10 client.15033 fetch_fsmap learned FSMap version 27 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe83c359740 1 -- 172.21.15.5:0/349489277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f0a4b43a10 con 0x55f0a4abe2f0 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe812ffd640 1 -- 172.21.15.5:0/349489277 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe81c005030 con 0x55f0a4abe2f0 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe812ffd640 1 -- 172.21.15.5:0/349489277 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe81c033020 con 0x55f0a4abe2f0 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe83c359740 10 client.15033 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe83c359740 10 client.15033 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe83c359740 20 client.15033 populate_metadata read hostname 'smithi005' 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe83c359740 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f0a4b48390 0x55f0a4b470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe83c359740 4 client.15033 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe83c359740 1 -- 172.21.15.5:0/349489277 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "status"}) -- 0x55f0a4a2bb10 con 0x55f0a4b48390 2024-09-19T06:02:47.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.582+0000 7fe8337fe640 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f0a4b48390 0x55f0a4b470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe812ffd640 10 client.15033 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe8337fe640 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f0a4b48390 0x55f0a4b470a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ae7288db093d74d server_cookie=5261d3b1df9ca28e in_seq=0 out_seq=0 2024-09-19T06:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe812ffd640 1 -- 172.21.15.5:0/349489277 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f0a4a2bb10 con 0x55f0a4b48390 2024-09-19T06:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe812ffd640 10 client.15033 handle_command_reply: tid=0 2024-09-19T06:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe83c359740 1 client.15033 shutdown 2024-09-19T06:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe83c359740 1 -- 172.21.15.5:0/349489277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 msgr2=0x55f0a4ae36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe83c359740 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ae36d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe81c005940 tx=0x7fe81c0104d0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe83c359740 1 -- 172.21.15.5:0/349489277 shutdown_connections 2024-09-19T06:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe83c359740 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f0a4b48390 0x55f0a4b470a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe83c359740 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0a4ac8bb0 0x55f0a4ade570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe83c359740 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0a4abe2f0 0x55f0a4ae36d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe83c359740 1 --2- 172.21.15.5:0/349489277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0a4abd9d0 0x55f0a4ae3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe83c359740 1 -- 172.21.15.5:0/349489277 >> 172.21.15.5:0/349489277 conn(0x55f0a4ab9f60 msgr2=0x55f0a4ab87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:47.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe812ffd640 0 client.15033 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:02:47.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe83c359740 1 -- 172.21.15.5:0/349489277 shutdown_connections 2024-09-19T06:02:47.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.583+0000 7fe83c359740 1 -- 172.21.15.5:0/349489277 wait complete. 2024-09-19T06:02:47.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe83c359740 20 client.15033 trim_cache size 0 max 16384 2024-09-19T06:02:47.593 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:02:47.593 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T06:02:47.593 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-19T06:02:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24448, 2024-09-19T06:02:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-19T06:02:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-19T06:02:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-19T06:02:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 69.763656322000003, 2024-09-19T06:02:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-19T06:02:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-19T06:02:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:02:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 102.15217970499999 2024-09-19T06:02:47.594 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:02:47.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe839f80640 1 -- 172.21.15.5:0/3822203881 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe8200a32d0 msgr2=0x7fe8200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:47.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe839f80640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe8200a32d0 0x7fe8200a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009a60 tx=0x7fe82c04c000 comp rx=0 tx=0).stop 2024-09-19T06:02:47.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe839f80640 1 -- 172.21.15.5:0/3822203881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834102d40 msgr2=0x7fe8341009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:47.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe839f80640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834102d40 0x7fe8341009a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe82400d950 tx=0x7fe82400de10 comp rx=0 tx=0).stop 2024-09-19T06:02:47.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe839f80640 1 -- 172.21.15.5:0/3822203881 shutdown_connections 2024-09-19T06:02:47.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe839f80640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fe8200a32d0 0x7fe8200a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe839f80640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83410c170 0x7fe834101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe839f80640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe834103660 0x7fe834100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe839f80640 1 --2- 172.21.15.5:0/3822203881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834102d40 0x7fe8341009a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:47.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe839f80640 1 -- 172.21.15.5:0/3822203881 >> 172.21.15.5:0/3822203881 conn(0x7fe8340fc7a0 msgr2=0x7fe8340fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:47.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe839f80640 1 -- 172.21.15.5:0/3822203881 shutdown_connections 2024-09-19T06:02:47.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:47.584+0000 7fe839f80640 1 -- 172.21.15.5:0/3822203881 wait complete. 2024-09-19T06:02:48.284 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T06:02:48.541 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:48.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:48 smithi005 ceph-mon[30955]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:48.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:48 smithi049 ceph-mon[34640]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:48.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:48 smithi157 ceph-mon[35346]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.016+0000 7ff0d4e93640 1 -- 172.21.15.5:0/95992257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d0106a90 msgr2=0x7ff0d0106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.016+0000 7ff0d4e93640 1 --2- 172.21.15.5:0/95992257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d0106a90 0x7ff0d0106e70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009930 tx=0x7ff0c002a420 comp rx=0 tx=0).stop 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.016+0000 7ff0d4e93640 1 -- 172.21.15.5:0/95992257 shutdown_connections 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.016+0000 7ff0d4e93640 1 --2- 172.21.15.5:0/95992257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0101430 0x7ff0d010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.016+0000 7ff0d4e93640 1 --2- 172.21.15.5:0/95992257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d0100a90 0x7ff0d0100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.016+0000 7ff0d4e93640 1 --2- 172.21.15.5:0/95992257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d0106a90 0x7ff0d0106e70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.016+0000 7ff0d4e93640 1 -- 172.21.15.5:0/95992257 >> 172.21.15.5:0/95992257 conn(0x7ff0d00fc7d0 msgr2=0x7ff0d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.016+0000 7ff0d4e93640 1 -- 172.21.15.5:0/95992257 shutdown_connections 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.016+0000 7ff0d4e93640 1 -- 172.21.15.5:0/95992257 wait complete. 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.017+0000 7ff0d4e93640 1 Processor -- start 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.017+0000 7ff0d4e93640 1 -- start start 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.017+0000 7ff0d4e93640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0100a90 0x7ff0d01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0d4e93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d0101430 0x7ff0d0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0d4e93640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d0106a90 0x7ff0d019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0d4e93640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0111ef0 con 0x7ff0d0101430 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0d4e93640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0111d70 con 0x7ff0d0100a90 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0d4e93640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0112070 con 0x7ff0d0106a90 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0cdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d0101430 0x7ff0d0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0ce59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0100a90 0x7ff0d01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0cdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d0101430 0x7ff0d0198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51200/0 (socket says 172.21.15.5:51200) 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0ce59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0100a90 0x7ff0d01984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57364/0 (socket says 172.21.15.5:57364) 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0cdd9b640 1 -- 172.21.15.5:0/3699384641 learned_addr learned my addr 172.21.15.5:0/3699384641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d0106a90 0x7ff0d019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0cdd9b640 1 -- 172.21.15.5:0/3699384641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d0106a90 msgr2=0x7ff0d019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0cdd9b640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d0106a90 0x7ff0d019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0cdd9b640 1 -- 172.21.15.5:0/3699384641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0100a90 msgr2=0x7ff0d01984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0cdd9b640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0100a90 0x7ff0d01984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0cdd9b640 1 -- 172.21.15.5:0/3699384641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d019d4a0 con 0x7ff0d0101430 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0cdd9b640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d0101430 0x7ff0d0198a30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400b4a0 tx=0x7ff0c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0bf7fe640 1 -- 172.21.15.5:0/3699384641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4014070 con 0x7ff0d0101430 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0bf7fe640 1 -- 172.21.15.5:0/3699384641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0c4010ac0 con 0x7ff0d0101430 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0d4e93640 1 -- 172.21.15.5:0/3699384641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d019d790 con 0x7ff0d0101430 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.018+0000 7ff0bf7fe640 1 -- 172.21.15.5:0/3699384641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4017c30 con 0x7ff0d0101430 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.019+0000 7ff0d4e93640 1 -- 172.21.15.5:0/3699384641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d01a5070 con 0x7ff0d0101430 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.021+0000 7ff0bf7fe640 1 -- 172.21.15.5:0/3699384641 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7ff0c4017070 con 0x7ff0d0101430 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.021+0000 7ff0d726c740 1 Processor -- start 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.021+0000 7ff0d726c740 1 -- start start 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0bf7fe640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff0b80a32d0 0x7ff0b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0d726c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad459d0 0x55614ad45db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0d726c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 0x55614ad50670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0d726c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad50bb0 0x55614ad52fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0d726c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55614acac090 con 0x55614ad459d0 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0d726c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55614ac53a70 con 0x55614ad50bb0 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0d726c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55614aca8050 con 0x55614ad462f0 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff0b80a32d0 0x7ff0b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0ce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad459d0 0x55614ad45db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0ce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad459d0 0x55614ad45db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51208/0 (socket says 172.21.15.5:51208) 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0ce59c640 1 -- 172.21.15.5:0/853963112 learned_addr learned my addr 172.21.15.5:0/853963112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0bf7fe640 1 -- 172.21.15.5:0/3699384641 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff0c40cece0 con 0x7ff0d0101430 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/853963112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad50bb0 0x55614ad52fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0cdd9b640 1 --2- 172.21.15.5:0/853963112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 0x55614ad50670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0ce59c640 1 -- 172.21.15.5:0/853963112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 msgr2=0x55614ad50670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0ce59c640 1 --2- 172.21.15.5:0/853963112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 0x55614ad50670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0ce59c640 1 -- 172.21.15.5:0/853963112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad50bb0 msgr2=0x55614ad52fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0ce59c640 1 --2- 172.21.15.5:0/853963112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad50bb0 0x55614ad52fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0ce59c640 1 -- 172.21.15.5:0/853963112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614ac60130 con 0x55614ad459d0 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff0b80a32d0 0x7ff0b80a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009900 tx=0x7ff0c8009ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.022+0000 7ff0ce59c640 1 --2- 172.21.15.5:0/853963112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad459d0 0x55614ad45db0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009930 tx=0x7ff0c0032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83c691f8b4541f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.023+0000 7ff0bcff9640 1 -- 172.21.15.5:0/853963112 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0035040 con 0x55614ad459d0 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.023+0000 7ff0bcff9640 1 -- 172.21.15.5:0/853963112 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0c00316a0 con 0x55614ad459d0 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.023+0000 7ff0bcff9640 1 -- 172.21.15.5:0/853963112 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00047a0 con 0x55614ad459d0 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.023+0000 7ff0d726c740 1 -- 172.21.15.5:0/853963112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad459d0 msgr2=0x55614ad45db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.023+0000 7ff0d726c740 1 --2- 172.21.15.5:0/853963112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad459d0 0x55614ad45db0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009930 tx=0x7ff0c0032d60 comp rx=0 tx=0).stop 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.023+0000 7ff0d726c740 1 -- 172.21.15.5:0/853963112 shutdown_connections 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.023+0000 7ff0d726c740 1 --2- 172.21.15.5:0/853963112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad50bb0 0x55614ad52fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.023+0000 7ff0d726c740 1 --2- 172.21.15.5:0/853963112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 0x55614ad50670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.023+0000 7ff0d726c740 1 --2- 172.21.15.5:0/853963112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad459d0 0x55614ad45db0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.023+0000 7ff0d726c740 1 -- 172.21.15.5:0/853963112 >> 172.21.15.5:0/853963112 conn(0x55614ad41f60 msgr2=0x55614ad42340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0d726c740 1 -- 172.21.15.5:0/853963112 shutdown_connections 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0d726c740 1 -- 172.21.15.5:0/853963112 wait complete. 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0d726c740 1 -- start start 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0d726c740 1 Processor -- start 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0d726c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad459d0 0x55614ad50380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0d726c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad462f0 0x55614ad4d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0d726c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614ad4dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0d726c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55614ad53780 con 0x55614ad50bb0 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0d726c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55614ac66f20 con 0x55614ad462f0 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0d726c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55614ad53900 con 0x55614ad459d0 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0d726c740 10 client.0 _ll_register_callbacks cb 0x55614ac30030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614ad4dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614ad4dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51220/0 (socket says 172.21.15.5:51220) 2024-09-19T06:02:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.024+0000 7ff0ced9d640 1 -- 172.21.15.5:0/1781080044 learned_addr learned my addr 172.21.15.5:0/1781080044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0cdd9b640 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad462f0 0x55614ad4d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0ce59c640 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad459d0 0x55614ad50380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0ced9d640 1 -- 172.21.15.5:0/1781080044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad459d0 msgr2=0x55614ad50380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad459d0 0x55614ad50380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0ced9d640 1 -- 172.21.15.5:0/1781080044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad462f0 msgr2=0x55614ad4d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad462f0 0x55614ad4d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0ced9d640 1 -- 172.21.15.5:0/1781080044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614aa1d2b0 con 0x55614ad50bb0 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614ad4dc10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800a7d0 tx=0x7ff0c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e228d06b4276ba0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0b5ffb640 1 -- 172.21.15.5:0/1781080044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8014040 con 0x55614ad50bb0 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0b5ffb640 1 -- 172.21.15.5:0/1781080044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff0c8019760 con 0x55614ad50bb0 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0d726c740 1 -- 172.21.15.5:0/1781080044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c00316a0 con 0x55614ad50bb0 2024-09-19T06:02:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0b5ffb640 1 -- 172.21.15.5:0/1781080044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0c802e050 con 0x55614ad50bb0 2024-09-19T06:02:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0d726c740 10 client.15045 fetch_fsmap learned FSMap version 27 2024-09-19T06:02:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.025+0000 7ff0d726c740 1 -- 172.21.15.5:0/1781080044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55614ad5efb0 con 0x55614ad50bb0 2024-09-19T06:02:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.026+0000 7ff0b5ffb640 1 -- 172.21.15.5:0/1781080044 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0c00316a0 con 0x55614ad50bb0 2024-09-19T06:02:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.026+0000 7ff0b5ffb640 1 -- 172.21.15.5:0/1781080044 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff0c80268a0 con 0x55614ad50bb0 2024-09-19T06:02:49.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.026+0000 7ff0d726c740 10 client.15045 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:02:49.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.026+0000 7ff0d726c740 10 client.15045 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T06:02:49.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.026+0000 7ff0d726c740 20 client.15045 populate_metadata read hostname 'smithi005' 2024-09-19T06:02:49.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.026+0000 7ff0d726c740 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55614ad61610 0x55614ad62a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.026+0000 7ff0d726c740 4 client.15045 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:02:49.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.026+0000 7ff0d726c740 1 -- 172.21.15.5:0/1781080044 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55614acb9600 con 0x55614ad61610 2024-09-19T06:02:49.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.026+0000 7ff0ce59c640 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55614ad61610 0x55614ad62a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.026+0000 7ff0b5ffb640 10 client.15045 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:02:49.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.026+0000 7ff0ce59c640 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55614ad61610 0x55614ad62a80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aac78b2df9e981aa server_cookie=248e5e786b7f5f7b in_seq=0 out_seq=0 2024-09-19T06:02:49.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.027+0000 7ff0b5ffb640 1 -- 172.21.15.5:0/1781080044 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55614acb9600 con 0x55614ad61610 2024-09-19T06:02:49.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.027+0000 7ff0b5ffb640 10 client.15045 handle_command_reply: tid=0 2024-09-19T06:02:49.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.027+0000 7ff0d726c740 1 client.15045 shutdown 2024-09-19T06:02:49.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.027+0000 7ff0d726c740 1 -- 172.21.15.5:0/1781080044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 msgr2=0x55614ad4dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:49.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.027+0000 7ff0d726c740 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614ad4dc10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800a7d0 tx=0x7ff0c8004a40 comp rx=0 tx=0).stop 2024-09-19T06:02:49.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.027+0000 7ff0d726c740 1 -- 172.21.15.5:0/1781080044 shutdown_connections 2024-09-19T06:02:49.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.027+0000 7ff0d726c740 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55614ad61610 0x55614ad62a80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.027+0000 7ff0d726c740 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614ad4dc10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.027+0000 7ff0d726c740 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad462f0 0x55614ad4d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.027+0000 7ff0b5ffb640 0 client.15045 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:02:49.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.027+0000 7ff0d726c740 1 --2- 172.21.15.5:0/1781080044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad459d0 0x55614ad50380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.027+0000 7ff0d726c740 1 -- 172.21.15.5:0/1781080044 >> 172.21.15.5:0/1781080044 conn(0x55614ad41f60 msgr2=0x55614ad47c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:49.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.028+0000 7ff0d726c740 1 -- 172.21.15.5:0/1781080044 shutdown_connections 2024-09-19T06:02:49.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.028+0000 7ff0d726c740 1 -- 172.21.15.5:0/1781080044 wait complete. 2024-09-19T06:02:49.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.028+0000 7ff0d726c740 20 client.15045 trim_cache size 0 max 16384 2024-09-19T06:02:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.046+0000 7ff0d726c740 1 Processor -- start 2024-09-19T06:02:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.046+0000 7ff0d726c740 1 -- start start 2024-09-19T06:02:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.046+0000 7ff0d726c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad459d0 0x55614ad4cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.046+0000 7ff0d726c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 0x55614ad4bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.046+0000 7ff0d726c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614ad4c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.046+0000 7ff0d726c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55614ad60dc0 con 0x55614ad50bb0 2024-09-19T06:02:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.046+0000 7ff0d726c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55614ad61490 con 0x55614ad459d0 2024-09-19T06:02:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.046+0000 7ff0d726c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55614ad61b60 con 0x55614ad462f0 2024-09-19T06:02:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.046+0000 7ff0ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614ad4c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.046+0000 7ff0ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614ad4c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51234/0 (socket says 172.21.15.5:51234) 2024-09-19T06:02:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.046+0000 7ff0ced9d640 1 -- 172.21.15.5:0/4226531343 learned_addr learned my addr 172.21.15.5:0/4226531343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.047+0000 7ff0ced9d640 1 -- 172.21.15.5:0/4226531343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 msgr2=0x55614ad4bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.047+0000 7ff0ce59c640 1 --2- 172.21.15.5:0/4226531343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad459d0 0x55614ad4cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.047+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/4226531343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 0x55614ad4bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.047+0000 7ff0ced9d640 1 -- 172.21.15.5:0/4226531343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad459d0 msgr2=0x55614ad4cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.047+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/4226531343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad459d0 0x55614ad4cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.047+0000 7ff0ced9d640 1 -- 172.21.15.5:0/4226531343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614ad41d90 con 0x55614ad50bb0 2024-09-19T06:02:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.047+0000 7ff0ce59c640 1 --2- 172.21.15.5:0/4226531343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad459d0 0x55614ad4cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.047+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/4226531343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614ad4c4e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8026c70 tx=0x7ff0c80096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138aaef51d9e4784 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.047+0000 7ff0b7fff640 1 -- 172.21.15.5:0/4226531343 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8036070 con 0x55614ad50bb0 2024-09-19T06:02:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.047+0000 7ff0b7fff640 1 -- 172.21.15.5:0/4226531343 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0c8026860 con 0x55614ad50bb0 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.047+0000 7ff0b7fff640 1 -- 172.21.15.5:0/4226531343 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8019b60 con 0x55614ad50bb0 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.048+0000 7ff0d726c740 1 -- 172.21.15.5:0/4226531343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 msgr2=0x55614ad4c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.048+0000 7ff0d726c740 1 --2- 172.21.15.5:0/4226531343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614ad4c4e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8026c70 tx=0x7ff0c80096f0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.048+0000 7ff0d726c740 1 -- 172.21.15.5:0/4226531343 shutdown_connections 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.048+0000 7ff0d726c740 1 --2- 172.21.15.5:0/4226531343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614ad4c4e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.048+0000 7ff0d726c740 1 --2- 172.21.15.5:0/4226531343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 0x55614ad4bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.048+0000 7ff0d726c740 1 --2- 172.21.15.5:0/4226531343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad459d0 0x55614ad4cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.048+0000 7ff0d726c740 1 -- 172.21.15.5:0/4226531343 >> 172.21.15.5:0/4226531343 conn(0x55614ad41f60 msgr2=0x55614ad4d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.048+0000 7ff0d726c740 1 -- 172.21.15.5:0/4226531343 shutdown_connections 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.048+0000 7ff0d726c740 1 -- 172.21.15.5:0/4226531343 wait complete. 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.048+0000 7ff0d726c740 1 -- start start 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.048+0000 7ff0d726c740 1 Processor -- start 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0d726c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad459d0 0x55614ad66480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0d726c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 0x55614ad669c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0d726c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614adc3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0d726c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55614ad62900 con 0x55614ad50bb0 2024-09-19T06:02:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0d726c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55614ad62230 con 0x55614ad459d0 2024-09-19T06:02:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0d726c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55614ad62fd0 con 0x55614ad462f0 2024-09-19T06:02:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0d726c740 10 client.0 _ll_register_callbacks cb 0x7ff0c8009390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:02:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614adc3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0ce59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad459d0 0x55614ad66480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614adc3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51238/0 (socket says 172.21.15.5:51238) 2024-09-19T06:02:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0ced9d640 1 -- 172.21.15.5:0/4092425754 learned_addr learned my addr 172.21.15.5:0/4092425754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0cdd9b640 1 --2- 172.21.15.5:0/4092425754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 0x55614ad669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0ced9d640 1 -- 172.21.15.5:0/4092425754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 msgr2=0x55614ad669c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/4092425754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 0x55614ad669c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0ced9d640 1 -- 172.21.15.5:0/4092425754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad459d0 msgr2=0x55614ad66480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/4092425754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad459d0 0x55614ad66480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0ced9d640 1 -- 172.21.15.5:0/4092425754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614adc4320 con 0x55614ad50bb0 2024-09-19T06:02:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0ced9d640 1 --2- 172.21.15.5:0/4092425754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614adc3ce0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009690 tx=0x7ff0c8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555e20f5d814c746 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:02:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.049+0000 7ff0b5ffb640 1 -- 172.21.15.5:0/4092425754 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8036040 con 0x55614ad50bb0 2024-09-19T06:02:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0b5ffb640 1 -- 172.21.15.5:0/4092425754 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff0c80265b0 con 0x55614ad50bb0 2024-09-19T06:02:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0d726c740 1 -- 172.21.15.5:0/4092425754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c8026860 con 0x55614ad50bb0 2024-09-19T06:02:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0b5ffb640 1 -- 172.21.15.5:0/4092425754 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0c804b050 con 0x55614ad50bb0 2024-09-19T06:02:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0d726c740 10 client.15051 fetch_fsmap learned FSMap version 27 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0d726c740 1 -- 172.21.15.5:0/4092425754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55614adcb5e0 con 0x55614ad50bb0 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0b5ffb640 1 -- 172.21.15.5:0/4092425754 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0c8026860 con 0x55614ad50bb0 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0b5ffb640 1 -- 172.21.15.5:0/4092425754 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff0c803b780 con 0x55614ad50bb0 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0d726c740 10 client.15051 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0d726c740 10 client.15051 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0d726c740 20 client.15051 populate_metadata read hostname 'smithi005' 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0d726c740 1 --2- 172.21.15.5:0/4092425754 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55614adcfef0 0x55614adcec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0d726c740 4 client.15051 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0d726c740 1 -- 172.21.15.5:0/4092425754 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "status"}) -- 0x55614acb3b10 con 0x55614adcfef0 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.050+0000 7ff0ce59c640 1 --2- 172.21.15.5:0/4092425754 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55614adcfef0 0x55614adcec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0ce59c640 1 --2- 172.21.15.5:0/4092425754 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55614adcfef0 0x55614adcec00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8fe1ebf03336b28 server_cookie=e2774d84032aae97 in_seq=0 out_seq=0 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0b5ffb640 10 client.15051 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:02:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0b5ffb640 1 -- 172.21.15.5:0/4092425754 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55614acb3b10 con 0x55614adcfef0 2024-09-19T06:02:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0b5ffb640 10 client.15051 handle_command_reply: tid=0 2024-09-19T06:02:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0d726c740 1 client.15051 shutdown 2024-09-19T06:02:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0d726c740 1 -- 172.21.15.5:0/4092425754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 msgr2=0x55614adc3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0d726c740 1 --2- 172.21.15.5:0/4092425754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614adc3ce0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009690 tx=0x7ff0c8009a20 comp rx=0 tx=0).stop 2024-09-19T06:02:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0d726c740 1 -- 172.21.15.5:0/4092425754 shutdown_connections 2024-09-19T06:02:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0d726c740 1 --2- 172.21.15.5:0/4092425754 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55614adcfef0 0x55614adcec00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0d726c740 1 --2- 172.21.15.5:0/4092425754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55614ad50bb0 0x55614adc3ce0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0d726c740 1 --2- 172.21.15.5:0/4092425754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55614ad462f0 0x55614ad669c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0d726c740 1 --2- 172.21.15.5:0/4092425754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55614ad459d0 0x55614ad66480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0b5ffb640 0 client.15051 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:02:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0d726c740 1 -- 172.21.15.5:0/4092425754 >> 172.21.15.5:0/4092425754 conn(0x55614ad41f60 msgr2=0x55614ad4bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0d726c740 1 -- 172.21.15.5:0/4092425754 shutdown_connections 2024-09-19T06:02:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.051+0000 7ff0d726c740 1 -- 172.21.15.5:0/4092425754 wait complete. 2024-09-19T06:02:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d726c740 20 client.15051 trim_cache size 0 max 16384 2024-09-19T06:02:49.064 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:02:49.064 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T06:02:49.064 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-19T06:02:49.064 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24463, 2024-09-19T06:02:49.065 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-19T06:02:49.065 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-19T06:02:49.065 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-19T06:02:49.065 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 72.236444176000006, 2024-09-19T06:02:49.065 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-19T06:02:49.065 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-19T06:02:49.065 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:02:49.065 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 98.517457754999995 2024-09-19T06:02:49.065 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:02:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d4e93640 1 -- 172.21.15.5:0/3699384641 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff0b80a32d0 msgr2=0x7ff0b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d4e93640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff0b80a32d0 0x7ff0b80a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009900 tx=0x7ff0c8009ea0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d4e93640 1 -- 172.21.15.5:0/3699384641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d0101430 msgr2=0x7ff0d0198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d4e93640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d0101430 0x7ff0d0198a30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400b4a0 tx=0x7ff0c400b960 comp rx=0 tx=0).stop 2024-09-19T06:02:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d4e93640 1 -- 172.21.15.5:0/3699384641 shutdown_connections 2024-09-19T06:02:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d4e93640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7ff0b80a32d0 0x7ff0b80a5790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d4e93640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d0106a90 0x7ff0d019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d4e93640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d0101430 0x7ff0d0198a30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d4e93640 1 --2- 172.21.15.5:0/3699384641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0100a90 0x7ff0d01984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d4e93640 1 -- 172.21.15.5:0/3699384641 >> 172.21.15.5:0/3699384641 conn(0x7ff0d00fc7d0 msgr2=0x7ff0d00fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d4e93640 1 -- 172.21.15.5:0/3699384641 shutdown_connections 2024-09-19T06:02:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:49.052+0000 7ff0d4e93640 1 -- 172.21.15.5:0/3699384641 wait complete. 2024-09-19T06:02:49.776 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T06:02:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:50.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:50 smithi005 ceph-mon[30955]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.490+0000 7fd6ab36d640 1 -- 172.21.15.5:0/4010324694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a4074f40 msgr2=0x7fd6a410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.490+0000 7fd6ab36d640 1 --2- 172.21.15.5:0/4010324694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a4074f40 0x7fd6a410de80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd694009900 tx=0x7fd69402a480 comp rx=0 tx=0).stop 2024-09-19T06:02:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.490+0000 7fd6ab36d640 1 -- 172.21.15.5:0/4010324694 shutdown_connections 2024-09-19T06:02:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.490+0000 7fd6ab36d640 1 --2- 172.21.15.5:0/4010324694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a410e3c0 0x7fd6a4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.490+0000 7fd6ab36d640 1 --2- 172.21.15.5:0/4010324694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a4074f40 0x7fd6a410de80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.490+0000 7fd6ab36d640 1 --2- 172.21.15.5:0/4010324694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6a4074620 0x7fd6a4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.490+0000 7fd6ab36d640 1 -- 172.21.15.5:0/4010324694 >> 172.21.15.5:0/4010324694 conn(0x7fd6a40fd8c0 msgr2=0x7fd6a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.490+0000 7fd6ab36d640 1 -- 172.21.15.5:0/4010324694 shutdown_connections 2024-09-19T06:02:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.490+0000 7fd6ab36d640 1 -- 172.21.15.5:0/4010324694 wait complete. 2024-09-19T06:02:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.491+0000 7fd6ab36d640 1 Processor -- start 2024-09-19T06:02:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.491+0000 7fd6ab36d640 1 -- start start 2024-09-19T06:02:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.491+0000 7fd6ab36d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6a4074620 0x7fd6a419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.491+0000 7fd6ab36d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a4074f40 0x7fd6a419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.491+0000 7fd6ab36d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a410e3c0 0x7fd6a419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.491+0000 7fd6ab36d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a41140c0 con 0x7fd6a4074f40 2024-09-19T06:02:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.491+0000 7fd6ab36d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a4113f40 con 0x7fd6a4074620 2024-09-19T06:02:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.491+0000 7fd6ab36d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a4114240 con 0x7fd6a410e3c0 2024-09-19T06:02:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a8908640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a4074f40 0x7fd6a419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a8908640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a4074f40 0x7fd6a419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51270/0 (socket says 172.21.15.5:51270) 2024-09-19T06:02:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a8908640 1 -- 172.21.15.5:0/1061172067 learned_addr learned my addr 172.21.15.5:0/1061172067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a990a640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a410e3c0 0x7fd6a419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a9109640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6a4074620 0x7fd6a419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a8908640 1 -- 172.21.15.5:0/1061172067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a410e3c0 msgr2=0x7fd6a419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a8908640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a410e3c0 0x7fd6a419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a8908640 1 -- 172.21.15.5:0/1061172067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6a4074620 msgr2=0x7fd6a419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a8908640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6a4074620 0x7fd6a419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a8908640 1 -- 172.21.15.5:0/1061172067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a419f5b0 con 0x7fd6a4074f40 2024-09-19T06:02:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a990a640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a410e3c0 0x7fd6a419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a9109640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6a4074620 0x7fd6a419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6a8908640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a4074f40 0x7fd6a419ab40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd694005ab0 tx=0x7fd694003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd69a7fc640 1 -- 172.21.15.5:0/1061172067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69403f070 con 0x7fd6a4074f40 2024-09-19T06:02:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd69a7fc640 1 -- 172.21.15.5:0/1061172067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd694031d60 con 0x7fd6a4074f40 2024-09-19T06:02:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6ab36d640 1 -- 172.21.15.5:0/1061172067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a419f840 con 0x7fd6a4074f40 2024-09-19T06:02:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd69a7fc640 1 -- 172.21.15.5:0/1061172067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694035030 con 0x7fd6a4074f40 2024-09-19T06:02:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.492+0000 7fd6ab36d640 1 -- 172.21.15.5:0/1061172067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a41a7180 con 0x7fd6a4074f40 2024-09-19T06:02:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.495+0000 7fd69a7fc640 1 -- 172.21.15.5:0/1061172067 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fd694042070 con 0x7fd6a4074f40 2024-09-19T06:02:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.495+0000 7fd6ad746740 1 Processor -- start 2024-09-19T06:02:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.495+0000 7fd6ad746740 1 -- start start 2024-09-19T06:02:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.495+0000 7fd6ad746740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b39569c9d0 0x55b39569cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.495+0000 7fd6ad746740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 0x55b3956a7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.495+0000 7fd6ad746740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3956a7bb0 0x55b3956a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.495+0000 7fd6ad746740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b395603090 con 0x55b39569d2f0 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.495+0000 7fd6ad746740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b3955aaa70 con 0x55b39569c9d0 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.495+0000 7fd6ad746740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b3955ff050 con 0x55b3956a7bb0 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.495+0000 7fd6a9109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 0x55b3956a7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.495+0000 7fd6a9109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 0x55b3956a7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51278/0 (socket says 172.21.15.5:51278) 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.495+0000 7fd6a9109640 1 -- 172.21.15.5:0/17625117 learned_addr learned my addr 172.21.15.5:0/17625117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd6a990a640 1 --2- 172.21.15.5:0/17625117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b39569c9d0 0x55b39569cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd69a7fc640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd6900a3350 0x7fd6900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd6a9109640 1 -- 172.21.15.5:0/17625117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3956a7bb0 msgr2=0x55b3956a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd6a990a640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd6900a3350 0x7fd6900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd69a7fc640 1 -- 172.21.15.5:0/1061172067 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd6940f0500 con 0x7fd6a4074f40 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd6a8908640 1 --2- 172.21.15.5:0/17625117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3956a7bb0 0x55b3956a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd6a9109640 1 --2- 172.21.15.5:0/17625117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3956a7bb0 0x55b3956a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd6a9109640 1 -- 172.21.15.5:0/17625117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b39569c9d0 msgr2=0x55b39569cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd6a9109640 1 --2- 172.21.15.5:0/17625117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b39569c9d0 0x55b39569cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd6a9109640 1 -- 172.21.15.5:0/17625117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3955b7130 con 0x55b39569d2f0 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd6a8908640 1 --2- 172.21.15.5:0/17625117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3956a7bb0 0x55b3956a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd6a990a640 1 --2- 172.21.15.5:0/17625117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b39569c9d0 0x55b39569cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd6a9109640 1 --2- 172.21.15.5:0/17625117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 0x55b3956a7670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0007850 tx=0x7fd6a0007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce724c8eaa3a70f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd68f7fe640 1 -- 172.21.15.5:0/17625117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a000d950 con 0x55b39569d2f0 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.496+0000 7fd6a990a640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd6900a3350 0x7fd6900a5810 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00afd0 tx=0x7fd69c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.497+0000 7fd68f7fe640 1 -- 172.21.15.5:0/17625117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6a0018070 con 0x55b39569d2f0 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.497+0000 7fd68f7fe640 1 -- 172.21.15.5:0/17625117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0013e30 con 0x55b39569d2f0 2024-09-19T06:02:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.497+0000 7fd6ad746740 1 -- 172.21.15.5:0/17625117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 msgr2=0x55b3956a7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.497+0000 7fd6ad746740 1 --2- 172.21.15.5:0/17625117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 0x55b3956a7670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0007850 tx=0x7fd6a0007d10 comp rx=0 tx=0).stop 2024-09-19T06:02:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.497+0000 7fd6ad746740 1 -- 172.21.15.5:0/17625117 shutdown_connections 2024-09-19T06:02:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.497+0000 7fd6ad746740 1 --2- 172.21.15.5:0/17625117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3956a7bb0 0x55b3956a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.497+0000 7fd6ad746740 1 --2- 172.21.15.5:0/17625117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 0x55b3956a7670 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.497+0000 7fd6ad746740 1 --2- 172.21.15.5:0/17625117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b39569c9d0 0x55b39569cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.497+0000 7fd6ad746740 1 -- 172.21.15.5:0/17625117 >> 172.21.15.5:0/17625117 conn(0x55b395698f60 msgr2=0x55b395699340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.497+0000 7fd6ad746740 1 -- 172.21.15.5:0/17625117 shutdown_connections 2024-09-19T06:02:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.497+0000 7fd6ad746740 1 -- 172.21.15.5:0/17625117 wait complete. 2024-09-19T06:02:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.498+0000 7fd6ad746740 1 -- start start 2024-09-19T06:02:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.498+0000 7fd6ad746740 1 Processor -- start 2024-09-19T06:02:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.498+0000 7fd6ad746740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 0x55b3956a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.498+0000 7fd6ad746740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b39569d2f0 0x55b3956a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.498+0000 7fd6ad746740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3956a7bb0 0x55b3956a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.498+0000 7fd6ad746740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b3956aa780 con 0x55b39569c9d0 2024-09-19T06:02:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.498+0000 7fd6ad746740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b3955bdf20 con 0x55b39569d2f0 2024-09-19T06:02:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.498+0000 7fd6ad746740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b3956aa900 con 0x55b3956a7bb0 2024-09-19T06:02:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.498+0000 7fd6ad746740 10 client.0 _ll_register_callbacks cb 0x55b395587030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:02:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.498+0000 7fd6a9109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 0x55b3956a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.498+0000 7fd6a9109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 0x55b3956a73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51284/0 (socket says 172.21.15.5:51284) 2024-09-19T06:02:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.498+0000 7fd6a9109640 1 -- 172.21.15.5:0/3924787912 learned_addr learned my addr 172.21.15.5:0/3924787912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd6a990a640 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3956a7bb0 0x55b3956a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd6a9109640 1 -- 172.21.15.5:0/3924787912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3956a7bb0 msgr2=0x55b3956a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd6a8908640 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b39569d2f0 0x55b3956a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd6a9109640 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3956a7bb0 0x55b3956a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd6a9109640 1 -- 172.21.15.5:0/3924787912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b39569d2f0 msgr2=0x55b3956a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd6a9109640 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b39569d2f0 0x55b3956a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd6a9109640 1 -- 172.21.15.5:0/3924787912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3953742b0 con 0x55b39569c9d0 2024-09-19T06:02:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd6a9109640 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 0x55b3956a73b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0000c00 tx=0x7fd6a0013e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792a7d31c45a8afc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd68cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd68cff9640 1 -- 172.21.15.5:0/3924787912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a000d950 con 0x55b39569c9d0 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd68cff9640 1 -- 172.21.15.5:0/3924787912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd6a000daf0 con 0x55b39569c9d0 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd6ad746740 1 -- 172.21.15.5:0/3924787912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a0018070 con 0x55b39569c9d0 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd68cff9640 1 -- 172.21.15.5:0/3924787912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6a000dcf0 con 0x55b39569c9d0 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd6ad746740 10 client.15063 fetch_fsmap learned FSMap version 27 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd6ad746740 1 -- 172.21.15.5:0/3924787912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3956b5f80 con 0x55b39569c9d0 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.499+0000 7fd68cff9640 1 -- 172.21.15.5:0/3924787912 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6a0018070 con 0x55b39569c9d0 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.500+0000 7fd68cff9640 1 -- 172.21.15.5:0/3924787912 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd6a001d400 con 0x55b39569c9d0 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.500+0000 7fd6ad746740 10 client.15063 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.500+0000 7fd6ad746740 10 client.15063 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.500+0000 7fd6ad746740 20 client.15063 populate_metadata read hostname 'smithi005' 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.500+0000 7fd6ad746740 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b3956b8210 0x55b3956b9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.500+0000 7fd6ad746740 4 client.15063 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:02:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.500+0000 7fd6ad746740 1 -- 172.21.15.5:0/3924787912 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b395610600 con 0x55b3956b8210 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.500+0000 7fd6a8908640 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b3956b8210 0x55b3956b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.501+0000 7fd6a8908640 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b3956b8210 0x55b3956b9680 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d857f44f8b74bbe2 server_cookie=193655881779acd3 in_seq=0 out_seq=0 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.501+0000 7fd68cff9640 10 client.15063 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.501+0000 7fd68cff9640 1 -- 172.21.15.5:0/3924787912 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b395610600 con 0x55b3956b8210 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.501+0000 7fd68cff9640 10 client.15063 handle_command_reply: tid=0 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.501+0000 7fd6ad746740 1 client.15063 shutdown 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.502+0000 7fd6ad746740 1 -- 172.21.15.5:0/3924787912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 msgr2=0x55b3956a73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.502+0000 7fd6ad746740 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 0x55b3956a73b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0000c00 tx=0x7fd6a0013e60 comp rx=0 tx=0).stop 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.502+0000 7fd6ad746740 1 -- 172.21.15.5:0/3924787912 shutdown_connections 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.502+0000 7fd6ad746740 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b3956b8210 0x55b3956b9680 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.502+0000 7fd6ad746740 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3956a7bb0 0x55b3956a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.502+0000 7fd6ad746740 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b39569d2f0 0x55b3956a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.502+0000 7fd6ad746740 1 --2- 172.21.15.5:0/3924787912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 0x55b3956a73b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.502+0000 7fd6ad746740 1 -- 172.21.15.5:0/3924787912 >> 172.21.15.5:0/3924787912 conn(0x55b395698f60 msgr2=0x55b39569ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.502+0000 7fd68cff9640 0 client.15063 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:02:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.502+0000 7fd6ad746740 1 -- 172.21.15.5:0/3924787912 shutdown_connections 2024-09-19T06:02:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.502+0000 7fd6ad746740 1 -- 172.21.15.5:0/3924787912 wait complete. 2024-09-19T06:02:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.502+0000 7fd6ad746740 20 client.15063 trim_cache size 0 max 16384 2024-09-19T06:02:50.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6ad746740 1 Processor -- start 2024-09-19T06:02:50.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6ad746740 1 -- start start 2024-09-19T06:02:50.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6ad746740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b39569c9d0 0x55b3956b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6ad746740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 0x55b3956b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6ad746740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b3956a7bb0 0x55b395697ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6ad746740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b3956b79c0 con 0x55b39569d2f0 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6ad746740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b3956b8090 con 0x55b3956a7bb0 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6ad746740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b3956b8760 con 0x55b39569c9d0 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6a8908640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 0x55b3956b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6a8908640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 0x55b3956b8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51288/0 (socket says 172.21.15.5:51288) 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6a8908640 1 -- 172.21.15.5:0/1676411894 learned_addr learned my addr 172.21.15.5:0/1676411894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6a990a640 1 --2- 172.21.15.5:0/1676411894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b3956a7bb0 0x55b395697ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6a9109640 1 --2- 172.21.15.5:0/1676411894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b39569c9d0 0x55b3956b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6a8908640 1 -- 172.21.15.5:0/1676411894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b39569c9d0 msgr2=0x55b3956b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6a8908640 1 --2- 172.21.15.5:0/1676411894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b39569c9d0 0x55b3956b7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6a8908640 1 -- 172.21.15.5:0/1676411894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b3956a7bb0 msgr2=0x55b395697ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6a8908640 1 --2- 172.21.15.5:0/1676411894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b3956a7bb0 0x55b395697ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.520+0000 7fd6a8908640 1 -- 172.21.15.5:0/1676411894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b395698a00 con 0x55b39569d2f0 2024-09-19T06:02:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd6a8908640 1 --2- 172.21.15.5:0/1676411894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 0x55b3956b8fb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd694005ec0 tx=0x7fd694031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84501abb3cd7be51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd68effd640 1 -- 172.21.15.5:0/1676411894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694049040 con 0x55b39569d2f0 2024-09-19T06:02:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd68effd640 1 -- 172.21.15.5:0/1676411894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd69403be50 con 0x55b39569d2f0 2024-09-19T06:02:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd68effd640 1 -- 172.21.15.5:0/1676411894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6940b9020 con 0x55b39569d2f0 2024-09-19T06:02:50.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd6ad746740 1 -- 172.21.15.5:0/1676411894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 msgr2=0x55b3956b8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:50.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd6ad746740 1 --2- 172.21.15.5:0/1676411894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 0x55b3956b8fb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd694005ec0 tx=0x7fd694031f10 comp rx=0 tx=0).stop 2024-09-19T06:02:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd6ad746740 1 -- 172.21.15.5:0/1676411894 shutdown_connections 2024-09-19T06:02:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd6ad746740 1 --2- 172.21.15.5:0/1676411894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b3956a7bb0 0x55b395697ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd6ad746740 1 --2- 172.21.15.5:0/1676411894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569d2f0 0x55b3956b8fb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd6ad746740 1 --2- 172.21.15.5:0/1676411894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b39569c9d0 0x55b3956b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd6ad746740 1 -- 172.21.15.5:0/1676411894 >> 172.21.15.5:0/1676411894 conn(0x55b395698f60 msgr2=0x55b3956977e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd6ad746740 1 -- 172.21.15.5:0/1676411894 shutdown_connections 2024-09-19T06:02:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.521+0000 7fd6ad746740 1 -- 172.21.15.5:0/1676411894 wait complete. 2024-09-19T06:02:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6ad746740 1 -- start start 2024-09-19T06:02:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6ad746740 1 Processor -- start 2024-09-19T06:02:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6ad746740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 0x55b3956bc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6ad746740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b39569d2f0 0x55b3956bcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6ad746740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b3956a7bb0 0x55b3956bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6ad746740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b3956b9500 con 0x55b39569c9d0 2024-09-19T06:02:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6ad746740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b3956b8e30 con 0x55b3956a7bb0 2024-09-19T06:02:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6ad746740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b3956b9bd0 con 0x55b39569d2f0 2024-09-19T06:02:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6a9109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 0x55b3956bc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6ad746740 10 client.0 _ll_register_callbacks cb 0x7fd69c00a690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:02:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6a9109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 0x55b3956bc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51304/0 (socket says 172.21.15.5:51304) 2024-09-19T06:02:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6a9109640 1 -- 172.21.15.5:0/1688448809 learned_addr learned my addr 172.21.15.5:0/1688448809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6a8908640 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b39569d2f0 0x55b3956bcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.522+0000 7fd6a990a640 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b3956a7bb0 0x55b3956bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd6a9109640 1 -- 172.21.15.5:0/1688448809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b39569d2f0 msgr2=0x55b3956bcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd6a9109640 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b39569d2f0 0x55b3956bcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd6a9109640 1 -- 172.21.15.5:0/1688448809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b3956a7bb0 msgr2=0x55b3956bd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd6a9109640 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b3956a7bb0 0x55b3956bd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd6a9109640 1 -- 172.21.15.5:0/1688448809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b395720990 con 0x55b39569c9d0 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd6a9109640 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 0x55b3956bc740 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0006de0 tx=0x7fd6a001d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f934e7c01bfce20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd68cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd68cff9640 1 -- 172.21.15.5:0/1688448809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0013750 con 0x55b39569c9d0 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd68cff9640 1 -- 172.21.15.5:0/1688448809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd6a00138f0 con 0x55b39569c9d0 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd6ad746740 1 -- 172.21.15.5:0/1688448809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd69403be50 con 0x55b39569c9d0 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd68cff9640 1 -- 172.21.15.5:0/1688448809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6a0041050 con 0x55b39569c9d0 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd6ad746740 10 client.15069 fetch_fsmap learned FSMap version 27 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd6ad746740 1 -- 172.21.15.5:0/1688448809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b395721cf0 con 0x55b39569c9d0 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.523+0000 7fd68cff9640 1 -- 172.21.15.5:0/1688448809 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd69403be50 con 0x55b39569c9d0 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd68cff9640 1 -- 172.21.15.5:0/1688448809 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd6a0027410 con 0x55b39569c9d0 2024-09-19T06:02:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd6ad746740 10 client.15069 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:02:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd6ad746740 10 client.15069 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:02:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd6ad746740 20 client.15069 populate_metadata read hostname 'smithi005' 2024-09-19T06:02:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd6ad746740 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b395726970 0x55b395725680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd6ad746740 4 client.15069 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:02:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd6ad746740 1 -- 172.21.15.5:0/1688448809 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "status"}) -- 0x55b39560ab10 con 0x55b395726970 2024-09-19T06:02:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd6a8908640 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b395726970 0x55b395725680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd68cff9640 10 client.15069 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:02:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd6a8908640 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b395726970 0x55b395725680 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8cab98f07919142 server_cookie=4826a492a163da74 in_seq=0 out_seq=0 2024-09-19T06:02:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd68cff9640 1 -- 172.21.15.5:0/1688448809 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b39560ab10 con 0x55b395726970 2024-09-19T06:02:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd68cff9640 10 client.15069 handle_command_reply: tid=0 2024-09-19T06:02:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.524+0000 7fd6ad746740 1 client.15069 shutdown 2024-09-19T06:02:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ad746740 1 -- 172.21.15.5:0/1688448809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 msgr2=0x55b3956bc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ad746740 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 0x55b3956bc740 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0006de0 tx=0x7fd6a001d6b0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ad746740 1 -- 172.21.15.5:0/1688448809 shutdown_connections 2024-09-19T06:02:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ad746740 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b395726970 0x55b395725680 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ad746740 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b3956a7bb0 0x55b3956bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ad746740 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b39569d2f0 0x55b3956bcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ad746740 1 --2- 172.21.15.5:0/1688448809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39569c9d0 0x55b3956bc740 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ad746740 1 -- 172.21.15.5:0/1688448809 >> 172.21.15.5:0/1688448809 conn(0x55b395698f60 msgr2=0x55b3956977e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd68cff9640 0 client.15069 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:02:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ad746740 1 -- 172.21.15.5:0/1688448809 shutdown_connections 2024-09-19T06:02:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ad746740 1 -- 172.21.15.5:0/1688448809 wait complete. 2024-09-19T06:02:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ad746740 20 client.15069 trim_cache size 0 max 16384 2024-09-19T06:02:50.533 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:02:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T06:02:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-19T06:02:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24473, 2024-09-19T06:02:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-19T06:02:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-19T06:02:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-19T06:02:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 80.738814466999997, 2024-09-19T06:02:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-19T06:02:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-19T06:02:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:02:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 91.519190499000004 2024-09-19T06:02:50.534 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:02:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ab36d640 1 -- 172.21.15.5:0/1061172067 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd6900a3350 msgr2=0x7fd6900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ab36d640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd6900a3350 0x7fd6900a5810 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00afd0 tx=0x7fd69c004ab0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ab36d640 1 -- 172.21.15.5:0/1061172067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a4074f40 msgr2=0x7fd6a419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ab36d640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a4074f40 0x7fd6a419ab40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd694005ab0 tx=0x7fd694003d50 comp rx=0 tx=0).stop 2024-09-19T06:02:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ab36d640 1 -- 172.21.15.5:0/1061172067 shutdown_connections 2024-09-19T06:02:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ab36d640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd6900a3350 0x7fd6900a5810 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ab36d640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a410e3c0 0x7fd6a419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ab36d640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a4074f40 0x7fd6a419ab40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ab36d640 1 --2- 172.21.15.5:0/1061172067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6a4074620 0x7fd6a419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.525+0000 7fd6ab36d640 1 -- 172.21.15.5:0/1061172067 >> 172.21.15.5:0/1061172067 conn(0x7fd6a40fd8c0 msgr2=0x7fd6a40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.526+0000 7fd6ab36d640 1 -- 172.21.15.5:0/1061172067 shutdown_connections 2024-09-19T06:02:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:50.526+0000 7fd6ab36d640 1 -- 172.21.15.5:0/1061172067 wait complete. 2024-09-19T06:02:50.603 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:50 smithi157 ceph-mon[35346]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:50.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:50 smithi049 ceph-mon[34640]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:51.257 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T06:02:51.257 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T06:02:51.267 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-19T06:02:51.267 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T06:02:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:51.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.963+0000 7f5d40f9e640 1 -- 172.21.15.5:0/2866162917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c101c00 msgr2=0x7f5d3c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.963+0000 7f5d40f9e640 1 --2- 172.21.15.5:0/2866162917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c101c00 0x7f5d3c102000 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009900 tx=0x7f5d2c02a410 comp rx=0 tx=0).stop 2024-09-19T06:02:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.963+0000 7f5d40f9e640 1 -- 172.21.15.5:0/2866162917 shutdown_connections 2024-09-19T06:02:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.963+0000 7f5d40f9e640 1 --2- 172.21.15.5:0/2866162917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d3c1037c0 0x7f5d3c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.963+0000 7f5d40f9e640 1 --2- 172.21.15.5:0/2866162917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d3c102e00 0x7f5d3c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.963+0000 7f5d40f9e640 1 --2- 172.21.15.5:0/2866162917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c101c00 0x7f5d3c102000 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.963+0000 7f5d40f9e640 1 -- 172.21.15.5:0/2866162917 >> 172.21.15.5:0/2866162917 conn(0x7f5d3c0fd390 msgr2=0x7f5d3c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.963+0000 7f5d40f9e640 1 -- 172.21.15.5:0/2866162917 shutdown_connections 2024-09-19T06:02:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.963+0000 7f5d40f9e640 1 -- 172.21.15.5:0/2866162917 wait complete. 2024-09-19T06:02:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.964+0000 7f5d40f9e640 1 Processor -- start 2024-09-19T06:02:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.964+0000 7f5d40f9e640 1 -- start start 2024-09-19T06:02:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.964+0000 7f5d40f9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d3c101c00 0x7f5d3c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.964+0000 7f5d40f9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d3c102e00 0x7f5d3c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.964+0000 7f5d40f9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c1037c0 0x7f5d3c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.964+0000 7f5d40f9e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10d740 con 0x7f5d3c1037c0 2024-09-19T06:02:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.964+0000 7f5d40f9e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10d5c0 con 0x7f5d3c101c00 2024-09-19T06:02:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.964+0000 7f5d40f9e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10d8c0 con 0x7f5d3c102e00 2024-09-19T06:02:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.964+0000 7f5d3ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c1037c0 0x7f5d3c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.964+0000 7f5d3ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c1037c0 0x7f5d3c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51322/0 (socket says 172.21.15.5:51322) 2024-09-19T06:02:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.964+0000 7f5d3ad9d640 1 -- 172.21.15.5:0/207071784 learned_addr learned my addr 172.21.15.5:0/207071784 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.964+0000 7f5d3a59c640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d3c101c00 0x7f5d3c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.965+0000 7f5d39d9b640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d3c102e00 0x7f5d3c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.965+0000 7f5d3ad9d640 1 -- 172.21.15.5:0/207071784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d3c102e00 msgr2=0x7f5d3c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.965+0000 7f5d3ad9d640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d3c102e00 0x7f5d3c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.965+0000 7f5d3ad9d640 1 -- 172.21.15.5:0/207071784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d3c101c00 msgr2=0x7f5d3c075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.965+0000 7f5d3ad9d640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d3c101c00 0x7f5d3c075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.965+0000 7f5d3ad9d640 1 -- 172.21.15.5:0/207071784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c0753e0 con 0x7f5d3c1037c0 2024-09-19T06:02:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.965+0000 7f5d3ad9d640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c1037c0 0x7f5d3c074b20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400cad0 tx=0x7f5d3400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.965+0000 7f5d2b7fe640 1 -- 172.21.15.5:0/207071784 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400ae20 con 0x7f5d3c1037c0 2024-09-19T06:02:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.965+0000 7f5d2b7fe640 1 -- 172.21.15.5:0/207071784 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d34005020 con 0x7f5d3c1037c0 2024-09-19T06:02:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.965+0000 7f5d40f9e640 1 -- 172.21.15.5:0/207071784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c1a1fd0 con 0x7f5d3c1037c0 2024-09-19T06:02:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.965+0000 7f5d2b7fe640 1 -- 172.21.15.5:0/207071784 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400f6b0 con 0x7f5d3c1037c0 2024-09-19T06:02:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.965+0000 7f5d40f9e640 1 -- 172.21.15.5:0/207071784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a24d0 con 0x7f5d3c1037c0 2024-09-19T06:02:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.966+0000 7f5d40f9e640 1 -- 172.21.15.5:0/207071784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c109090 con 0x7f5d3c1037c0 2024-09-19T06:02:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.968+0000 7f5d2b7fe640 1 -- 172.21.15.5:0/207071784 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f5d340051d0 con 0x7f5d3c1037c0 2024-09-19T06:02:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.968+0000 7f5d2b7fe640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5d240a3350 0x7f5d240a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.968+0000 7f5d3a59c640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5d240a3350 0x7f5d240a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.968+0000 7f5d2b7fe640 1 -- 172.21.15.5:0/207071784 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5d340cf9d0 con 0x7f5d3c1037c0 2024-09-19T06:02:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.969+0000 7f5d3a59c640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5d240a3350 0x7f5d240a5810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c004770 tx=0x7f5d2c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:51.973+0000 7f5d2b7fe640 1 -- 172.21.15.5:0/207071784 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d34014030 con 0x7f5d3c1037c0 2024-09-19T06:02:52.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.162+0000 7f5d40f9e640 1 -- 172.21.15.5:0/207071784 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f5d3c062630 con 0x7f5d240a3350 2024-09-19T06:02:52.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.163+0000 7f5d2b7fe640 1 -- 172.21.15.5:0/207071784 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f5d3c062630 con 0x7f5d240a3350 2024-09-19T06:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout:Backend: cephadm 2024-09-19T06:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout:Available: Yes 2024-09-19T06:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout:Paused: No 2024-09-19T06:02:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.165+0000 7f5d40f9e640 1 -- 172.21.15.5:0/207071784 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5d240a3350 msgr2=0x7f5d240a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.165+0000 7f5d40f9e640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5d240a3350 0x7f5d240a5810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c004770 tx=0x7f5d2c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:02:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.165+0000 7f5d40f9e640 1 -- 172.21.15.5:0/207071784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c1037c0 msgr2=0x7f5d3c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.165+0000 7f5d40f9e640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c1037c0 0x7f5d3c074b20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400cad0 tx=0x7f5d3400cf90 comp rx=0 tx=0).stop 2024-09-19T06:02:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.165+0000 7f5d40f9e640 1 -- 172.21.15.5:0/207071784 shutdown_connections 2024-09-19T06:02:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.165+0000 7f5d40f9e640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5d240a3350 0x7f5d240a5810 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.165+0000 7f5d40f9e640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c1037c0 0x7f5d3c074b20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.165+0000 7f5d40f9e640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d3c102e00 0x7f5d3c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.165+0000 7f5d40f9e640 1 --2- 172.21.15.5:0/207071784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d3c101c00 0x7f5d3c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.165+0000 7f5d40f9e640 1 -- 172.21.15.5:0/207071784 >> 172.21.15.5:0/207071784 conn(0x7f5d3c0fd390 msgr2=0x7f5d3c106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.165+0000 7f5d40f9e640 1 -- 172.21.15.5:0/207071784 shutdown_connections 2024-09-19T06:02:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:52.165+0000 7f5d40f9e640 1 -- 172.21.15.5:0/207071784 wait complete. 2024-09-19T06:02:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:52 smithi005 ceph-mon[30955]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:52.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:52 smithi049 ceph-mon[34640]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:52 smithi157 ceph-mon[35346]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:52.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T06:02:53.063 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:53.449 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:53 smithi005 ceph-mon[30955]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:53.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.530+0000 7fd21e642640 1 -- 172.21.15.5:0/1809473680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd218107980 msgr2=0x7fd218109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:53.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.530+0000 7fd21e642640 1 --2- 172.21.15.5:0/1809473680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd218107980 0x7fd218109d70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd210009ca0 tx=0x7fd21002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:02:53.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.530+0000 7fd21e642640 1 -- 172.21.15.5:0/1809473680 shutdown_connections 2024-09-19T06:02:53.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.530+0000 7fd21e642640 1 --2- 172.21.15.5:0/1809473680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd218107980 0x7fd218109d70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.530+0000 7fd21e642640 1 --2- 172.21.15.5:0/1809473680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd218105050 0x7fd218107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.530+0000 7fd21e642640 1 --2- 172.21.15.5:0/1809473680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd218102720 0x7fd218104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.530+0000 7fd21e642640 1 -- 172.21.15.5:0/1809473680 >> 172.21.15.5:0/1809473680 conn(0x7fd2180fc280 msgr2=0x7fd2180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.530+0000 7fd21e642640 1 -- 172.21.15.5:0/1809473680 shutdown_connections 2024-09-19T06:02:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.530+0000 7fd21e642640 1 -- 172.21.15.5:0/1809473680 wait complete. 2024-09-19T06:02:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.530+0000 7fd21e642640 1 Processor -- start 2024-09-19T06:02:53.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.530+0000 7fd21e642640 1 -- start start 2024-09-19T06:02:53.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21e642640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd218102720 0x7fd218197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:53.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21e642640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd218105050 0x7fd218198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:53.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21e642640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd218107980 0x7fd21819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:53.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21e642640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd21810d720 con 0x7fd218107980 2024-09-19T06:02:53.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21e642640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd21810d5a0 con 0x7fd218105050 2024-09-19T06:02:53.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21e642640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd21810d8a0 con 0x7fd218102720 2024-09-19T06:02:53.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21cbdf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd218107980 0x7fd21819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:53.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21cbdf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd218107980 0x7fd21819f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51346/0 (socket says 172.21.15.5:51346) 2024-09-19T06:02:53.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21cbdf640 1 -- 172.21.15.5:0/3678674903 learned_addr learned my addr 172.21.15.5:0/3678674903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:53.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd2177fe640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd218105050 0x7fd218198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:53.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd217fff640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd218102720 0x7fd218197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:53.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21cbdf640 1 -- 172.21.15.5:0/3678674903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd218102720 msgr2=0x7fd218197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:53.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21cbdf640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd218102720 0x7fd218197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:53.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21cbdf640 1 -- 172.21.15.5:0/3678674903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd218105050 msgr2=0x7fd218198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:53.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21cbdf640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd218105050 0x7fd218198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:53.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21cbdf640 1 -- 172.21.15.5:0/3678674903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21819fa10 con 0x7fd218107980 2024-09-19T06:02:53.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd2177fe640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd218105050 0x7fd218198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:02:53.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.531+0000 7fd21cbdf640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd218107980 0x7fd21819f3a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd210031970 tx=0x7fd210003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:53.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.532+0000 7fd2157fa640 1 -- 172.21.15.5:0/3678674903 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210004040 con 0x7fd218107980 2024-09-19T06:02:53.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.532+0000 7fd2157fa640 1 -- 172.21.15.5:0/3678674903 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd210004690 con 0x7fd218107980 2024-09-19T06:02:53.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.532+0000 7fd21e642640 1 -- 172.21.15.5:0/3678674903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21819fca0 con 0x7fd218107980 2024-09-19T06:02:53.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.532+0000 7fd2157fa640 1 -- 172.21.15.5:0/3678674903 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210030c60 con 0x7fd218107980 2024-09-19T06:02:53.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.532+0000 7fd21e642640 1 -- 172.21.15.5:0/3678674903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2181a0190 con 0x7fd218107980 2024-09-19T06:02:53.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.533+0000 7fd21e642640 1 -- 172.21.15.5:0/3678674903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2180fdd70 con 0x7fd218107980 2024-09-19T06:02:53.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.534+0000 7fd2157fa640 1 -- 172.21.15.5:0/3678674903 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fd2100041e0 con 0x7fd218107980 2024-09-19T06:02:53.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.535+0000 7fd2157fa640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd2000a3300 0x7fd2000a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:53.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.535+0000 7fd2157fa640 1 -- 172.21.15.5:0/3678674903 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd2100f0db0 con 0x7fd218107980 2024-09-19T06:02:53.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.535+0000 7fd217fff640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd2000a3300 0x7fd2000a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:53.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.535+0000 7fd217fff640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd2000a3300 0x7fd2000a57c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd208009bd0 tx=0x7fd208005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:53.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.539+0000 7fd2157fa640 1 -- 172.21.15.5:0/3678674903 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd2100bda70 con 0x7fd218107980 2024-09-19T06:02:53.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:53 smithi049 ceph-mon[34640]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.725+0000 7fd21e642640 1 -- 172.21.15.5:0/3678674903 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd218062630 con 0x7fd2000a3300 2024-09-19T06:02:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.737+0000 7fd2157fa640 1 -- 172.21.15.5:0/3678674903 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fd218062630 con 0x7fd2000a3300 2024-09-19T06:02:53.739 INFO:teuthology.orchestra.run.smithi005.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T06:02:53.739 INFO:teuthology.orchestra.run.smithi005.stdout:mds.a smithi005 running (97s) 19s ago 97s 16.3M - 19.1.1-361-g2f599a46 d2baa855c067 d95c9f55a0da 2024-09-19T06:02:53.739 INFO:teuthology.orchestra.run.smithi005.stdout:mds.b smithi049 running (95s) 27s ago 94s 26.1M - 19.1.1-361-g2f599a46 d2baa855c067 855f6af86406 2024-09-19T06:02:53.739 INFO:teuthology.orchestra.run.smithi005.stdout:mds.c smithi157 running (108s) 27s ago 108s 19.3M - 19.1.1-361-g2f599a46 d2baa855c067 2cdb769b3739 2024-09-19T06:02:53.740 INFO:teuthology.orchestra.run.smithi005.stdout:mds.d smithi005 running (116s) 19s ago 116s 15.9M - 19.1.1-361-g2f599a46 d2baa855c067 3923c8f34b2c 2024-09-19T06:02:53.740 INFO:teuthology.orchestra.run.smithi005.stdout:mds.e smithi049 running (113s) 27s ago 113s 18.0M - 19.1.1-361-g2f599a46 d2baa855c067 d8d4fef20850 2024-09-19T06:02:53.740 INFO:teuthology.orchestra.run.smithi005.stdout:mds.f smithi157 running (111s) 27s ago 110s 20.0M - 19.1.1-361-g2f599a46 d2baa855c067 0b945e16d702 2024-09-19T06:02:53.740 INFO:teuthology.orchestra.run.smithi005.stdout:mds.g smithi005 running (101s) 19s ago 100s 18.5M - 19.1.1-361-g2f599a46 d2baa855c067 270ffe4313e0 2024-09-19T06:02:53.740 INFO:teuthology.orchestra.run.smithi005.stdout:mds.h smithi049 running (106s) 27s ago 105s 19.4M - 19.1.1-361-g2f599a46 d2baa855c067 c5637008da47 2024-09-19T06:02:53.740 INFO:teuthology.orchestra.run.smithi005.stdout:mds.i smithi157 running (103s) 27s ago 103s 18.0M - 19.1.1-361-g2f599a46 d2baa855c067 052a38933963 2024-09-19T06:02:53.740 INFO:teuthology.orchestra.run.smithi005.stdout:mds.j smithi005 running (2m) 19s ago 119s 14.4M - 19.1.1-361-g2f599a46 d2baa855c067 8b209cf01c7a 2024-09-19T06:02:53.740 INFO:teuthology.orchestra.run.smithi005.stdout:mds.k smithi049 running (2m) 27s ago 2m 16.6M - 19.1.1-361-g2f599a46 d2baa855c067 bc1bd613a996 2024-09-19T06:02:53.740 INFO:teuthology.orchestra.run.smithi005.stdout:mds.l smithi157 running (2m) 27s ago 2m 16.8M - 19.1.1-361-g2f599a46 d2baa855c067 45250dc07c39 2024-09-19T06:02:53.740 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.x smithi005 *:9283,8765 running (8m) 19s ago 8m 508M - 19.1.1-361-g2f599a46 d2baa855c067 67f13a9e350e 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.y smithi049 *:8443,8765 running (7m) 27s ago 7m 442M - 19.1.1-361-g2f599a46 d2baa855c067 9571f2bb9a57 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.z smithi157 *:8443,8765 running (7m) 27s ago 7m 441M - 19.1.1-361-g2f599a46 d2baa855c067 48f555e37563 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a smithi005 running (8m) 19s ago 9m 49.8M 2048M 19.1.1-361-g2f599a46 d2baa855c067 550e96d2d684 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b smithi049 running (7m) 27s ago 7m 40.5M 2048M 19.1.1-361-g2f599a46 d2baa855c067 492ac721e6fc 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c smithi157 running (7m) 27s ago 7m 39.9M 2048M 19.1.1-361-g2f599a46 d2baa855c067 517a293e9503 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 smithi005 running (6m) 19s ago 6m 47.8M 4273M 19.1.1-361-g2f599a46 d2baa855c067 989bb7ea8e56 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 smithi049 running (6m) 27s ago 6m 55.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 644ad51420e3 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 smithi157 running (6m) 27s ago 6m 57.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 7400c3cdf67f 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 smithi005 running (5m) 19s ago 5m 47.0M 4273M 19.1.1-361-g2f599a46 d2baa855c067 957a72744101 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 smithi049 running (5m) 27s ago 5m 44.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 6cef94a9081d 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 smithi157 running (4m) 27s ago 4m 47.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 821df6f682be 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 smithi005 running (4m) 19s ago 4m 45.4M 4273M 19.1.1-361-g2f599a46 d2baa855c067 e87b3dbe7a4f 2024-09-19T06:02:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 smithi049 running (4m) 27s ago 4m 47.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 9ad847267e0c 2024-09-19T06:02:53.742 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 smithi157 running (3m) 27s ago 3m 55.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 145bede32c23 2024-09-19T06:02:53.742 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 smithi005 running (3m) 19s ago 3m 51.9M 4273M 19.1.1-361-g2f599a46 d2baa855c067 1e3a2822cc1f 2024-09-19T06:02:53.742 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 smithi049 running (2m) 27s ago 2m 55.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 b304c37a2bd7 2024-09-19T06:02:53.742 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 smithi157 running (2m) 27s ago 2m 46.1M 4272M 19.1.1-361-g2f599a46 d2baa855c067 3efabb13c3be 2024-09-19T06:02:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.739+0000 7fd21e642640 1 -- 172.21.15.5:0/3678674903 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd2000a3300 msgr2=0x7fd2000a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.739+0000 7fd21e642640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd2000a3300 0x7fd2000a57c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd208009bd0 tx=0x7fd208005f90 comp rx=0 tx=0).stop 2024-09-19T06:02:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.739+0000 7fd21e642640 1 -- 172.21.15.5:0/3678674903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd218107980 msgr2=0x7fd21819f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.739+0000 7fd21e642640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd218107980 0x7fd21819f3a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd210031970 tx=0x7fd210003ba0 comp rx=0 tx=0).stop 2024-09-19T06:02:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.740+0000 7fd21e642640 1 -- 172.21.15.5:0/3678674903 shutdown_connections 2024-09-19T06:02:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.740+0000 7fd21e642640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd2000a3300 0x7fd2000a57c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.740+0000 7fd21e642640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd218107980 0x7fd21819f3a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.740+0000 7fd21e642640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd218105050 0x7fd218198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.740+0000 7fd21e642640 1 --2- 172.21.15.5:0/3678674903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd218102720 0x7fd218197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.740+0000 7fd21e642640 1 -- 172.21.15.5:0/3678674903 >> 172.21.15.5:0/3678674903 conn(0x7fd2180fc280 msgr2=0x7fd218103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.740+0000 7fd21e642640 1 -- 172.21.15.5:0/3678674903 shutdown_connections 2024-09-19T06:02:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:53.740+0000 7fd21e642640 1 -- 172.21.15.5:0/3678674903 wait complete. 2024-09-19T06:02:53.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:53 smithi157 ceph-mon[35346]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:54.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T06:02:54.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:54 smithi005 ceph-mon[30955]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:54.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:54 smithi005 ceph-mon[30955]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:54.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:54 smithi049 ceph-mon[34640]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:54.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:54 smithi049 ceph-mon[34640]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:54 smithi157 ceph-mon[35346]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:54 smithi157 ceph-mon[35346]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:55.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.114+0000 7f5d73658640 1 -- 172.21.15.5:0/205639364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d6c103210 msgr2=0x7f5d6c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.114+0000 7f5d73658640 1 --2- 172.21.15.5:0/205639364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d6c103210 0x7f5d6c103610 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009900 tx=0x7f5d6802a410 comp rx=0 tx=0).stop 2024-09-19T06:02:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.114+0000 7f5d73658640 1 -- 172.21.15.5:0/205639364 shutdown_connections 2024-09-19T06:02:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.114+0000 7f5d73658640 1 --2- 172.21.15.5:0/205639364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6c100350 0x7f5d6c10a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.114+0000 7f5d73658640 1 --2- 172.21.15.5:0/205639364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d6c103be0 0x7f5d6c0ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.114+0000 7f5d73658640 1 --2- 172.21.15.5:0/205639364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d6c103210 0x7f5d6c103610 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.114+0000 7f5d73658640 1 -- 172.21.15.5:0/205639364 >> 172.21.15.5:0/205639364 conn(0x7f5d6c0fba40 msgr2=0x7f5d6c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.114+0000 7f5d73658640 1 -- 172.21.15.5:0/205639364 shutdown_connections 2024-09-19T06:02:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.114+0000 7f5d73658640 1 -- 172.21.15.5:0/205639364 wait complete. 2024-09-19T06:02:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d73658640 1 Processor -- start 2024-09-19T06:02:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d73658640 1 -- start start 2024-09-19T06:02:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d73658640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d6c100350 0x7f5d6c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d73658640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6c103210 0x7f5d6c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d73658640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d6c103be0 0x7f5d6c19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d73658640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10d7d0 con 0x7f5d6c100350 2024-09-19T06:02:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d73658640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10d650 con 0x7f5d6c103210 2024-09-19T06:02:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d73658640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10d950 con 0x7f5d6c103be0 2024-09-19T06:02:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d713f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d6c100350 0x7f5d6c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d713f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d6c100350 0x7f5d6c197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42270/0 (socket says 172.21.15.5:42270) 2024-09-19T06:02:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d713f4640 1 -- 172.21.15.5:0/2390525339 learned_addr learned my addr 172.21.15.5:0/2390525339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d70bf3640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6c103210 0x7f5d6c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.115+0000 7f5d71bf5640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d6c103be0 0x7f5d6c19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.116+0000 7f5d713f4640 1 -- 172.21.15.5:0/2390525339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d6c103be0 msgr2=0x7f5d6c19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.116+0000 7f5d713f4640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d6c103be0 0x7f5d6c19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.116+0000 7f5d713f4640 1 -- 172.21.15.5:0/2390525339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6c103210 msgr2=0x7f5d6c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.116+0000 7f5d713f4640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6c103210 0x7f5d6c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.116+0000 7f5d713f4640 1 -- 172.21.15.5:0/2390525339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c19fb10 con 0x7f5d6c100350 2024-09-19T06:02:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.116+0000 7f5d71bf5640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d6c103be0 0x7f5d6c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.116+0000 7f5d713f4640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d6c100350 0x7f5d6c197ee0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5d68004340 tx=0x7f5d68004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.116+0000 7f5d627fc640 1 -- 172.21.15.5:0/2390525339 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6803f030 con 0x7f5d6c100350 2024-09-19T06:02:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.116+0000 7f5d627fc640 1 -- 172.21.15.5:0/2390525339 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d68003cb0 con 0x7f5d6c100350 2024-09-19T06:02:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.116+0000 7f5d73658640 1 -- 172.21.15.5:0/2390525339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6c19fda0 con 0x7f5d6c100350 2024-09-19T06:02:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.116+0000 7f5d627fc640 1 -- 172.21.15.5:0/2390525339 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68035070 con 0x7f5d6c100350 2024-09-19T06:02:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.116+0000 7f5d73658640 1 -- 172.21.15.5:0/2390525339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6c1a0240 con 0x7f5d6c100350 2024-09-19T06:02:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.117+0000 7f5d73658640 1 -- 172.21.15.5:0/2390525339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d6c1092a0 con 0x7f5d6c100350 2024-09-19T06:02:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.118+0000 7f5d627fc640 1 -- 172.21.15.5:0/2390525339 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f5d68042030 con 0x7f5d6c100350 2024-09-19T06:02:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.119+0000 7f5d627fc640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5d580a3300 0x7f5d580a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.119+0000 7f5d70bf3640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5d580a3300 0x7f5d580a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.119+0000 7f5d627fc640 1 -- 172.21.15.5:0/2390525339 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5d680f0530 con 0x7f5d6c100350 2024-09-19T06:02:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.119+0000 7f5d70bf3640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5d580a3300 0x7f5d580a57c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c199400 tx=0x7f5d5c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.122+0000 7f5d627fc640 1 -- 172.21.15.5:0/2390525339 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d680bd080 con 0x7f5d6c100350 2024-09-19T06:02:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.306+0000 7f5d73658640 1 -- 172.21.15.5:0/2390525339 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f5d6c062630 con 0x7f5d580a3300 2024-09-19T06:02:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.311+0000 7f5d627fc640 1 -- 172.21.15.5:0/2390525339 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f5d6c062630 con 0x7f5d580a3300 2024-09-19T06:02:55.313 INFO:teuthology.orchestra.run.smithi005.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T06:02:55.313 INFO:teuthology.orchestra.run.smithi005.stdout:mds.all 12/12 29s ago 2m smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-09-19T06:02:55.313 INFO:teuthology.orchestra.run.smithi005.stdout:mgr 3/3 29s ago 7m smithi005=x;smithi049=y;smithi157=z;count:3 2024-09-19T06:02:55.313 INFO:teuthology.orchestra.run.smithi005.stdout:mon 3/3 29s ago 7m smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi157:172.21.15.157=c;count:3 2024-09-19T06:02:55.314 INFO:teuthology.orchestra.run.smithi005.stdout:osd 12 29s ago - 2024-09-19T06:02:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.314+0000 7f5d73658640 1 -- 172.21.15.5:0/2390525339 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5d580a3300 msgr2=0x7f5d580a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.314+0000 7f5d73658640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5d580a3300 0x7f5d580a57c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c199400 tx=0x7f5d5c009110 comp rx=0 tx=0).stop 2024-09-19T06:02:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.314+0000 7f5d73658640 1 -- 172.21.15.5:0/2390525339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d6c100350 msgr2=0x7f5d6c197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.314+0000 7f5d73658640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d6c100350 0x7f5d6c197ee0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5d68004340 tx=0x7f5d68004370 comp rx=0 tx=0).stop 2024-09-19T06:02:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.314+0000 7f5d73658640 1 -- 172.21.15.5:0/2390525339 shutdown_connections 2024-09-19T06:02:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.314+0000 7f5d73658640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f5d580a3300 0x7f5d580a57c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.314+0000 7f5d73658640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d6c103be0 0x7f5d6c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.314+0000 7f5d73658640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6c103210 0x7f5d6c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.314+0000 7f5d73658640 1 --2- 172.21.15.5:0/2390525339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d6c100350 0x7f5d6c197ee0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.314+0000 7f5d73658640 1 -- 172.21.15.5:0/2390525339 >> 172.21.15.5:0/2390525339 conn(0x7f5d6c0fba40 msgr2=0x7f5d6c106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.314+0000 7f5d73658640 1 -- 172.21.15.5:0/2390525339 shutdown_connections 2024-09-19T06:02:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:55.314+0000 7f5d73658640 1 -- 172.21.15.5:0/2390525339 wait complete. 2024-09-19T06:02:55.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T06:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:56.582 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:56 smithi005 ceph-mon[30955]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:56.582 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:56 smithi005 ceph-mon[30955]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:56.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:56 smithi049 ceph-mon[34640]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:56.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:56 smithi049 ceph-mon[34640]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.664+0000 7fb761b94640 1 -- 172.21.15.5:0/726813509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb75c1026d0 msgr2=0x7fb75c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.664+0000 7fb761b94640 1 --2- 172.21.15.5:0/726813509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb75c1026d0 0x7fb75c108f60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb754009ca0 tx=0x7fb75402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:02:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.664+0000 7fb761b94640 1 -- 172.21.15.5:0/726813509 shutdown_connections 2024-09-19T06:02:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.664+0000 7fb761b94640 1 --2- 172.21.15.5:0/726813509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb75c1026d0 0x7fb75c108f60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.664+0000 7fb761b94640 1 --2- 172.21.15.5:0/726813509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb75c101d10 0x7fb75c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.664+0000 7fb761b94640 1 --2- 172.21.15.5:0/726813509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb75c100b10 0x7fb75c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.664+0000 7fb761b94640 1 -- 172.21.15.5:0/726813509 >> 172.21.15.5:0/726813509 conn(0x7fb75c0fc2a0 msgr2=0x7fb75c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.664+0000 7fb761b94640 1 -- 172.21.15.5:0/726813509 shutdown_connections 2024-09-19T06:02:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.664+0000 7fb761b94640 1 -- 172.21.15.5:0/726813509 wait complete. 2024-09-19T06:02:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb761b94640 1 Processor -- start 2024-09-19T06:02:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb761b94640 1 -- start start 2024-09-19T06:02:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb761b94640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb75c100b10 0x7fb75c104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb761b94640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb75c101d10 0x7fb75c1030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb761b94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb75c1026d0 0x7fb75c103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb761b94640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c10c670 con 0x7fb75c1026d0 2024-09-19T06:02:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb761b94640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c10c4f0 con 0x7fb75c101d10 2024-09-19T06:02:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb761b94640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c10c7f0 con 0x7fb75c100b10 2024-09-19T06:02:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb75bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb75c1026d0 0x7fb75c103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb75bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb75c1026d0 0x7fb75c103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42286/0 (socket says 172.21.15.5:42286) 2024-09-19T06:02:56.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb75bfff640 1 -- 172.21.15.5:0/188107672 learned_addr learned my addr 172.21.15.5:0/188107672 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:56.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb75b7fe640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb75c100b10 0x7fb75c104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:56.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb75affd640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb75c101d10 0x7fb75c1030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:56.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb75bfff640 1 -- 172.21.15.5:0/188107672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb75c100b10 msgr2=0x7fb75c104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:56.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb75bfff640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb75c100b10 0x7fb75c104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:56.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb75bfff640 1 -- 172.21.15.5:0/188107672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb75c101d10 msgr2=0x7fb75c1030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:56.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb75bfff640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb75c101d10 0x7fb75c1030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:56.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb75bfff640 1 -- 172.21.15.5:0/188107672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75c103ef0 con 0x7fb75c1026d0 2024-09-19T06:02:56.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.665+0000 7fb75affd640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb75c101d10 0x7fb75c1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:02:56.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.666+0000 7fb75bfff640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb75c1026d0 0x7fb75c103630 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb754031040 tx=0x7fb754003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:56.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.666+0000 7fb758ff9640 1 -- 172.21.15.5:0/188107672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754032ec0 con 0x7fb75c1026d0 2024-09-19T06:02:56.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.666+0000 7fb758ff9640 1 -- 172.21.15.5:0/188107672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7540044f0 con 0x7fb75c1026d0 2024-09-19T06:02:56.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.666+0000 7fb761b94640 1 -- 172.21.15.5:0/188107672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75c1a1f20 con 0x7fb75c1026d0 2024-09-19T06:02:56.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.666+0000 7fb758ff9640 1 -- 172.21.15.5:0/188107672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754030d10 con 0x7fb75c1026d0 2024-09-19T06:02:56.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.666+0000 7fb761b94640 1 -- 172.21.15.5:0/188107672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75c1a23d0 con 0x7fb75c1026d0 2024-09-19T06:02:56.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.667+0000 7fb761b94640 1 -- 172.21.15.5:0/188107672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb75c1a2600 con 0x7fb75c1026d0 2024-09-19T06:02:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.668+0000 7fb758ff9640 1 -- 172.21.15.5:0/188107672 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fb754004040 con 0x7fb75c1026d0 2024-09-19T06:02:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.669+0000 7fb758ff9640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb7440a3470 0x7fb7440a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.669+0000 7fb75b7fe640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb7440a3470 0x7fb7440a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.669+0000 7fb758ff9640 1 -- 172.21.15.5:0/188107672 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb7540f1160 con 0x7fb75c1026d0 2024-09-19T06:02:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.669+0000 7fb75b7fe640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb7440a3470 0x7fb7440a5930 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb75c101b70 tx=0x7fb74c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:56.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.672+0000 7fb758ff9640 1 -- 172.21.15.5:0/188107672 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb7540bdeb0 con 0x7fb75c1026d0 2024-09-19T06:02:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:56 smithi157 ceph-mon[35346]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:02:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:56 smithi157 ceph-mon[35346]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.857+0000 7fb761b94640 1 -- 172.21.15.5:0/188107672 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fb75c1a27d0 con 0x7fb7440a3470 2024-09-19T06:02:56.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.859+0000 7fb758ff9640 1 -- 172.21.15.5:0/188107672 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fb75c1a27d0 con 0x7fb7440a3470 2024-09-19T06:02:56.861 INFO:teuthology.orchestra.run.smithi005.stdout:HOST ADDR LABELS STATUS 2024-09-19T06:02:56.861 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 172.21.15.5 2024-09-19T06:02:56.861 INFO:teuthology.orchestra.run.smithi005.stdout:smithi049 172.21.15.49 2024-09-19T06:02:56.861 INFO:teuthology.orchestra.run.smithi005.stdout:smithi157 172.21.15.157 2024-09-19T06:02:56.861 INFO:teuthology.orchestra.run.smithi005.stdout:3 hosts in cluster 2024-09-19T06:02:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.861+0000 7fb761b94640 1 -- 172.21.15.5:0/188107672 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb7440a3470 msgr2=0x7fb7440a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.861+0000 7fb761b94640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb7440a3470 0x7fb7440a5930 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb75c101b70 tx=0x7fb74c00b390 comp rx=0 tx=0).stop 2024-09-19T06:02:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.861+0000 7fb761b94640 1 -- 172.21.15.5:0/188107672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb75c1026d0 msgr2=0x7fb75c103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.861+0000 7fb761b94640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb75c1026d0 0x7fb75c103630 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb754031040 tx=0x7fb754003ba0 comp rx=0 tx=0).stop 2024-09-19T06:02:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.861+0000 7fb761b94640 1 -- 172.21.15.5:0/188107672 shutdown_connections 2024-09-19T06:02:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.861+0000 7fb761b94640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb7440a3470 0x7fb7440a5930 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.861+0000 7fb761b94640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb75c1026d0 0x7fb75c103630 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.861+0000 7fb761b94640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb75c101d10 0x7fb75c1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.861+0000 7fb761b94640 1 --2- 172.21.15.5:0/188107672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb75c100b10 0x7fb75c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.861+0000 7fb761b94640 1 -- 172.21.15.5:0/188107672 >> 172.21.15.5:0/188107672 conn(0x7fb75c0fc2a0 msgr2=0x7fb75c06a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.861+0000 7fb761b94640 1 -- 172.21.15.5:0/188107672 shutdown_connections 2024-09-19T06:02:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:56.861+0000 7fb761b94640 1 -- 172.21.15.5:0/188107672 wait complete. 2024-09-19T06:02:57.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T06:02:57.695 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.222+0000 7fa476a90640 1 -- 172.21.15.5:0/1632036698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4701037c0 msgr2=0x7fa47010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.222+0000 7fa476a90640 1 --2- 172.21.15.5:0/1632036698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4701037c0 0x7fa47010a050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa468009ca0 tx=0x7fa46802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:02:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.222+0000 7fa476a90640 1 -- 172.21.15.5:0/1632036698 shutdown_connections 2024-09-19T06:02:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.222+0000 7fa476a90640 1 --2- 172.21.15.5:0/1632036698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4701037c0 0x7fa47010a050 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.222+0000 7fa476a90640 1 --2- 172.21.15.5:0/1632036698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa470102e00 0x7fa470103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.222+0000 7fa476a90640 1 --2- 172.21.15.5:0/1632036698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa470101c00 0x7fa470102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.222+0000 7fa476a90640 1 -- 172.21.15.5:0/1632036698 >> 172.21.15.5:0/1632036698 conn(0x7fa4700fd390 msgr2=0x7fa4700ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.222+0000 7fa476a90640 1 -- 172.21.15.5:0/1632036698 shutdown_connections 2024-09-19T06:02:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.222+0000 7fa476a90640 1 -- 172.21.15.5:0/1632036698 wait complete. 2024-09-19T06:02:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.222+0000 7fa476a90640 1 Processor -- start 2024-09-19T06:02:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa476a90640 1 -- start start 2024-09-19T06:02:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa476a90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa470101c00 0x7fa470075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa476a90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa470102e00 0x7fa4700745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa476a90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4701037c0 0x7fa470074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa476a90640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa47010d880 con 0x7fa4701037c0 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa476a90640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa47010d700 con 0x7fa470102e00 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa476a90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa47010da00 con 0x7fa470101c00 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa47502d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4701037c0 0x7fa470074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa47502d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4701037c0 0x7fa470074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42296/0 (socket says 172.21.15.5:42296) 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa47502d640 1 -- 172.21.15.5:0/3109413797 learned_addr learned my addr 172.21.15.5:0/3109413797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa46ffff640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa470102e00 0x7fa4700745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa47482c640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa470101c00 0x7fa470075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa47502d640 1 -- 172.21.15.5:0/3109413797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa470101c00 msgr2=0x7fa470075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa47502d640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa470101c00 0x7fa470075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa47502d640 1 -- 172.21.15.5:0/3109413797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa470102e00 msgr2=0x7fa4700745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa47502d640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa470102e00 0x7fa4700745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:58.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa47502d640 1 -- 172.21.15.5:0/3109413797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4700753e0 con 0x7fa4701037c0 2024-09-19T06:02:58.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.223+0000 7fa47502d640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4701037c0 0x7fa470074b20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa468031040 tx=0x7fa468003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:58.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.224+0000 7fa46dffb640 1 -- 172.21.15.5:0/3109413797 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468004040 con 0x7fa4701037c0 2024-09-19T06:02:58.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.224+0000 7fa46dffb640 1 -- 172.21.15.5:0/3109413797 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa468004690 con 0x7fa4701037c0 2024-09-19T06:02:58.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.224+0000 7fa46dffb640 1 -- 172.21.15.5:0/3109413797 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46803b410 con 0x7fa4701037c0 2024-09-19T06:02:58.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.224+0000 7fa476a90640 1 -- 172.21.15.5:0/3109413797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4701a2000 con 0x7fa4701037c0 2024-09-19T06:02:58.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.224+0000 7fa476a90640 1 -- 172.21.15.5:0/3109413797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4701a25e0 con 0x7fa4701037c0 2024-09-19T06:02:58.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.225+0000 7fa476a90640 1 -- 172.21.15.5:0/3109413797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa470109000 con 0x7fa4701037c0 2024-09-19T06:02:58.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.226+0000 7fa46dffb640 1 -- 172.21.15.5:0/3109413797 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fa4680305c0 con 0x7fa4701037c0 2024-09-19T06:02:58.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.227+0000 7fa46dffb640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa4600a3350 0x7fa4600a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:58.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.227+0000 7fa47482c640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa4600a3350 0x7fa4600a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:58.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.227+0000 7fa46dffb640 1 -- 172.21.15.5:0/3109413797 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa4680f12e0 con 0x7fa4701037c0 2024-09-19T06:02:58.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.227+0000 7fa47482c640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa4600a3350 0x7fa4600a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa470102c60 tx=0x7fa464008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:58.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.231+0000 7fa46dffb640 1 -- 172.21.15.5:0/3109413797 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa4680be060 con 0x7fa4701037c0 2024-09-19T06:02:58.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.414+0000 7fa476a90640 1 -- 172.21.15.5:0/3109413797 --> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa470062630 con 0x7fa4600a3350 2024-09-19T06:02:58.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:58 smithi005 ceph-mon[30955]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:58.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:58 smithi005 ceph-mon[30955]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.416+0000 7fa46dffb640 1 -- 172.21.15.5:0/3109413797 <== mgr.14150 v2:172.21.15.5:6800/709970884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa470062630 con 0x7fa4600a3350 2024-09-19T06:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T06:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EW480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-19T06:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout:smithi049 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747500D2480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-19T06:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout:smithi157 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002V480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-19T06:02:58.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.418+0000 7fa476a90640 1 -- 172.21.15.5:0/3109413797 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa4600a3350 msgr2=0x7fa4600a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:58.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.418+0000 7fa476a90640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa4600a3350 0x7fa4600a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa470102c60 tx=0x7fa464008040 comp rx=0 tx=0).stop 2024-09-19T06:02:58.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.419+0000 7fa476a90640 1 -- 172.21.15.5:0/3109413797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4701037c0 msgr2=0x7fa470074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:58.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.419+0000 7fa476a90640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4701037c0 0x7fa470074b20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa468031040 tx=0x7fa468003c10 comp rx=0 tx=0).stop 2024-09-19T06:02:58.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.419+0000 7fa476a90640 1 -- 172.21.15.5:0/3109413797 shutdown_connections 2024-09-19T06:02:58.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.419+0000 7fa476a90640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa4600a3350 0x7fa4600a5810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:58.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.419+0000 7fa476a90640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4701037c0 0x7fa470074b20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:58.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.419+0000 7fa476a90640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa470102e00 0x7fa4700745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:58.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.419+0000 7fa476a90640 1 --2- 172.21.15.5:0/3109413797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa470101c00 0x7fa470075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:58.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.419+0000 7fa476a90640 1 -- 172.21.15.5:0/3109413797 >> 172.21.15.5:0/3109413797 conn(0x7fa4700fd390 msgr2=0x7fa4701061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:58.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.419+0000 7fa476a90640 1 -- 172.21.15.5:0/3109413797 shutdown_connections 2024-09-19T06:02:58.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:58.419+0000 7fa476a90640 1 -- 172.21.15.5:0/3109413797 wait complete. 2024-09-19T06:02:58.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:58 smithi049 ceph-mon[34640]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:58.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:58 smithi049 ceph-mon[34640]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:58 smithi157 ceph-mon[35346]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:02:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:58 smithi157 ceph-mon[35346]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:58.997 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T06:02:59.009 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-19T06:02:59.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T06:02:59.267 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:02:59.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:02:59 smithi005 ceph-mon[30955]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:59.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:02:59 smithi049 ceph-mon[34640]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:59.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.718+0000 7fbeada73640 1 -- 172.21.15.5:0/620693116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea81026a0 msgr2=0x7fbea8104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:59.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.718+0000 7fbeada73640 1 --2- 172.21.15.5:0/620693116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea81026a0 0x7fbea8104a90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009900 tx=0x7fbe9802a410 comp rx=0 tx=0).stop 2024-09-19T06:02:59.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.718+0000 7fbeada73640 1 -- 172.21.15.5:0/620693116 shutdown_connections 2024-09-19T06:02:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.718+0000 7fbeada73640 1 --2- 172.21.15.5:0/620693116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbea8107900 0x7fbea8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.718+0000 7fbeada73640 1 --2- 172.21.15.5:0/620693116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbea8104fd0 0x7fbea81073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.718+0000 7fbeada73640 1 --2- 172.21.15.5:0/620693116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea81026a0 0x7fbea8104a90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.718+0000 7fbeada73640 1 -- 172.21.15.5:0/620693116 >> 172.21.15.5:0/620693116 conn(0x7fbea80fc2a0 msgr2=0x7fbea80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.718+0000 7fbeada73640 1 -- 172.21.15.5:0/620693116 shutdown_connections 2024-09-19T06:02:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.718+0000 7fbeada73640 1 -- 172.21.15.5:0/620693116 wait complete. 2024-09-19T06:02:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.719+0000 7fbeada73640 1 Processor -- start 2024-09-19T06:02:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.719+0000 7fbeada73640 1 -- start start 2024-09-19T06:02:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.719+0000 7fbeada73640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbea81026a0 0x7fbea8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.719+0000 7fbeada73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8104fd0 0x7fbea81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.719+0000 7fbeada73640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbea8107900 0x7fbea819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.719+0000 7fbeada73640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbea810d840 con 0x7fbea8104fd0 2024-09-19T06:02:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.719+0000 7fbeada73640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbea810d6c0 con 0x7fbea81026a0 2024-09-19T06:02:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.719+0000 7fbeada73640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbea810d9c0 con 0x7fbea8107900 2024-09-19T06:02:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.719+0000 7fbe9effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8104fd0 0x7fbea81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.719+0000 7fbe9effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8104fd0 0x7fbea81984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42304/0 (socket says 172.21.15.5:42304) 2024-09-19T06:02:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbe9effd640 1 -- 172.21.15.5:0/1214056908 learned_addr learned my addr 172.21.15.5:0/1214056908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:02:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbea7fff640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbea8107900 0x7fbea819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbea77fe640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbea81026a0 0x7fbea8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbe9effd640 1 -- 172.21.15.5:0/1214056908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbea8107900 msgr2=0x7fbea819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbe9effd640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbea8107900 0x7fbea819f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbe9effd640 1 -- 172.21.15.5:0/1214056908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbea81026a0 msgr2=0x7fbea8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbe9effd640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbea81026a0 0x7fbea8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbe9effd640 1 -- 172.21.15.5:0/1214056908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea819fbe0 con 0x7fbea8104fd0 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbea77fe640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbea81026a0 0x7fbea8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbe9effd640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8104fd0 0x7fbea81984f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000b450 tx=0x7fbe9000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbea57fa640 1 -- 172.21.15.5:0/1214056908 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90015020 con 0x7fbea8104fd0 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbea57fa640 1 -- 172.21.15.5:0/1214056908 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe90011d40 con 0x7fbea8104fd0 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbeada73640 1 -- 172.21.15.5:0/1214056908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea819fed0 con 0x7fbea8104fd0 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbea57fa640 1 -- 172.21.15.5:0/1214056908 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000f340 con 0x7fbea8104fd0 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.720+0000 7fbeada73640 1 -- 172.21.15.5:0/1214056908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea81a0400 con 0x7fbea8104fd0 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.721+0000 7fbeada73640 1 -- 172.21.15.5:0/1214056908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea80fdd90 con 0x7fbea8104fd0 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.722+0000 7fbea57fa640 1 -- 172.21.15.5:0/1214056908 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fbe90018020 con 0x7fbea8104fd0 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.723+0000 7fbea57fa640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbe940a3300 0x7fbe940a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.723+0000 7fbea77fe640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbe940a3300 0x7fbe940a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:02:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.723+0000 7fbea57fa640 1 -- 172.21.15.5:0/1214056908 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fbe900cf400 con 0x7fbea8104fd0 2024-09-19T06:02:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.724+0000 7fbea77fe640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbe940a3300 0x7fbe940a57c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbe980314e0 tx=0x7fbe98031d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:02:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.727+0000 7fbea57fa640 1 -- 172.21.15.5:0/1214056908 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbe9001f020 con 0x7fbea8104fd0 2024-09-19T06:02:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:02:59 smithi157 ceph-mon[35346]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:02:59.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.939+0000 7fbeada73640 1 -- 172.21.15.5:0/1214056908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fbea802d7c0 con 0x7fbea8104fd0 2024-09-19T06:02:59.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.940+0000 7fbea57fa640 1 -- 172.21.15.5:0/1214056908 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3119 (secure 0 0 0) 0x7fbe9000d040 con 0x7fbea8104fd0 2024-09-19T06:02:59.942 INFO:teuthology.orchestra.run.smithi005.stdout:e27 2024-09-19T06:02:59.942 INFO:teuthology.orchestra.run.smithi005.stdout:btime 2024-09-19T06:01:41:840933+0000 2024-09-19T06:02:59.942 INFO:teuthology.orchestra.run.smithi005.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T06:02:59.942 INFO:teuthology.orchestra.run.smithi005.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T06:02:59.942 INFO:teuthology.orchestra.run.smithi005.stdout:legacy client fscid: 1 2024-09-19T06:02:59.942 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:59.942 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem 'cephfs' (1) 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:fs_name cephfs 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 27 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-09-19T06:01:29.531007+0000 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-09-19T06:01:41.840914+0000 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:tableserver 0 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:root 0 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:session_timeout 300 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:session_autoclose 300 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:max_file_size 1099511627776 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:max_xattr_size 65536 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:required_client_features {} 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure 0 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure_osd_epoch 0 2024-09-19T06:02:59.943 INFO:teuthology.orchestra.run.smithi005.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:max_mds 5 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:in 0,1,2,3,4 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:up {0=24473,1=24463,2=24448,3=24442,4=24437} 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:failed 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:damaged 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:stopped 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:data_pools [3] 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:metadata_pool 2 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:inline_data disabled 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:balancer 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:bal_rank_mask -1 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:standby_count_wanted 0 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:qdb_cluster leader: 24473 members: 24473,24448,24442,24463,24437 2024-09-19T06:02:59.944 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b{0:24473} state up:active seq 4 addr [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.e{0:24413} state up:standby-replay seq 1 addr [v2:172.21.15.49:6834/2353689058,v1:172.21.15.49:6835/2353689058] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.i{1:24463} state up:active seq 8 addr [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.l{1:24409} state up:standby-replay seq 1 addr [v2:172.21.15.157:6832/1048665474,v1:172.21.15.157:6833/1048665474] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c{2:24448} state up:active seq 10 addr [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.k{2:24389} state up:standby-replay seq 1 addr [v2:172.21.15.49:6832/2427602354,v1:172.21.15.49:6833/2427602354] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.f{3:24442} state up:active seq 11 addr [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a{3:14637} state up:standby-replay seq 1 addr [v2:172.21.15.5:6840/569764382,v1:172.21.15.5:6841/569764382] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.h{4:24437} state up:active seq 10 addr [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.g{4:14625} state up:standby-replay seq 1 addr [v2:172.21.15.5:6838/1310081153,v1:172.21.15.5:6839/1310081153] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout:Standby daemons: 2024-09-19T06:02:59.945 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:02:59.946 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.j{-1:14565} state up:standby seq 1 addr [v2:172.21.15.5:6834/959534083,v1:172.21.15.5:6835/959534083] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:02:59.946 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d{-1:14577} state up:standby seq 1 addr [v2:172.21.15.5:6836/3191633882,v1:172.21.15.5:6837/3191633882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:02:59.946 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-19T06:02:59.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.942+0000 7fbeada73640 1 -- 172.21.15.5:0/1214056908 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbe940a3300 msgr2=0x7fbe940a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:59.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.942+0000 7fbeada73640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbe940a3300 0x7fbe940a57c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbe980314e0 tx=0x7fbe98031d30 comp rx=0 tx=0).stop 2024-09-19T06:02:59.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.942+0000 7fbeada73640 1 -- 172.21.15.5:0/1214056908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8104fd0 msgr2=0x7fbea81984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:02:59.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.942+0000 7fbeada73640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8104fd0 0x7fbea81984f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000b450 tx=0x7fbe9000b910 comp rx=0 tx=0).stop 2024-09-19T06:02:59.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.942+0000 7fbeada73640 1 -- 172.21.15.5:0/1214056908 shutdown_connections 2024-09-19T06:02:59.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.942+0000 7fbeada73640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbe940a3300 0x7fbe940a57c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:59.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.942+0000 7fbeada73640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbea8107900 0x7fbea819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:59.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.942+0000 7fbeada73640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8104fd0 0x7fbea81984f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:59.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.942+0000 7fbeada73640 1 --2- 172.21.15.5:0/1214056908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbea81026a0 0x7fbea8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:02:59.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.942+0000 7fbeada73640 1 -- 172.21.15.5:0/1214056908 >> 172.21.15.5:0/1214056908 conn(0x7fbea80fc2a0 msgr2=0x7fbea8103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:02:59.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.942+0000 7fbeada73640 1 -- 172.21.15.5:0/1214056908 shutdown_connections 2024-09-19T06:02:59.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:02:59.942+0000 7fbeada73640 1 -- 172.21.15.5:0/1214056908 wait complete. 2024-09-19T06:03:00.509 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:00 smithi005 ceph-mon[30955]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:03:00.509 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:00 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1214056908' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T06:03:00.537 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T06:03:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:00 smithi049 ceph-mon[34640]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:03:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:00 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1214056908' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T06:03:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:00 smithi157 ceph-mon[35346]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:03:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:00 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1214056908' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T06:03:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.259+0000 7f929918a640 1 -- 172.21.15.5:0/2727879962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92941026a0 msgr2=0x7f9294104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.259+0000 7f929918a640 1 --2- 172.21.15.5:0/2727879962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92941026a0 0x7f9294104a90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f927c009900 tx=0x7f927c02a410 comp rx=0 tx=0).stop 2024-09-19T06:03:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.259+0000 7f929918a640 1 -- 172.21.15.5:0/2727879962 shutdown_connections 2024-09-19T06:03:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.259+0000 7f929918a640 1 --2- 172.21.15.5:0/2727879962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9294107900 0x7f9294109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.259+0000 7f929918a640 1 --2- 172.21.15.5:0/2727879962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9294104fd0 0x7f92941073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.259+0000 7f929918a640 1 --2- 172.21.15.5:0/2727879962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92941026a0 0x7f9294104a90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.259+0000 7f929918a640 1 -- 172.21.15.5:0/2727879962 >> 172.21.15.5:0/2727879962 conn(0x7f92940fc2a0 msgr2=0x7f92940fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.259+0000 7f929918a640 1 -- 172.21.15.5:0/2727879962 shutdown_connections 2024-09-19T06:03:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.259+0000 7f929918a640 1 -- 172.21.15.5:0/2727879962 wait complete. 2024-09-19T06:03:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.260+0000 7f929918a640 1 Processor -- start 2024-09-19T06:03:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.260+0000 7f929918a640 1 -- start start 2024-09-19T06:03:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.260+0000 7f929918a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92941026a0 0x7f9294102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.260+0000 7f929918a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9294104fd0 0x7f9294100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.260+0000 7f929918a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294107900 0x7f9294100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.260+0000 7f929918a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f929410d760 con 0x7f9294107900 2024-09-19T06:03:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.260+0000 7f929918a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f929410d5e0 con 0x7f9294104fd0 2024-09-19T06:03:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.260+0000 7f929918a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f929410d8e0 con 0x7f92941026a0 2024-09-19T06:03:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294107900 0x7f9294100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294107900 0x7f9294100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42330/0 (socket says 172.21.15.5:42330) 2024-09-19T06:03:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929359e640 1 -- 172.21.15.5:0/611445406 learned_addr learned my addr 172.21.15.5:0/611445406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f9292d9d640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92941026a0 0x7f9294102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929259c640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9294104fd0 0x7f9294100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929359e640 1 -- 172.21.15.5:0/611445406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92941026a0 msgr2=0x7f9294102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929359e640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92941026a0 0x7f9294102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929359e640 1 -- 172.21.15.5:0/611445406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9294104fd0 msgr2=0x7f9294100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929359e640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9294104fd0 0x7f9294100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929359e640 1 -- 172.21.15.5:0/611445406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9294101450 con 0x7f9294107900 2024-09-19T06:03:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929259c640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9294104fd0 0x7f9294100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929359e640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294107900 0x7f9294100b90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f928c00cce0 tx=0x7f928c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f928bfff640 1 -- 172.21.15.5:0/611445406 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c00f450 con 0x7f9294107900 2024-09-19T06:03:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f928bfff640 1 -- 172.21.15.5:0/611445406 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f928c00faa0 con 0x7f9294107900 2024-09-19T06:03:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929918a640 1 -- 172.21.15.5:0/611445406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92941a1f10 con 0x7f9294107900 2024-09-19T06:03:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f928bfff640 1 -- 172.21.15.5:0/611445406 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c018b70 con 0x7f9294107900 2024-09-19T06:03:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.261+0000 7f929918a640 1 -- 172.21.15.5:0/611445406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92941a2370 con 0x7f9294107900 2024-09-19T06:03:01.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.263+0000 7f929918a640 1 -- 172.21.15.5:0/611445406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92940fdd90 con 0x7f9294107900 2024-09-19T06:03:01.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.264+0000 7f928bfff640 1 -- 172.21.15.5:0/611445406 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f928c018d10 con 0x7f9294107900 2024-09-19T06:03:01.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.264+0000 7f928bfff640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f92800a32d0 0x7f92800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:01.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.265+0000 7f9292d9d640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f92800a32d0 0x7f92800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:01.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.265+0000 7f928bfff640 1 -- 172.21.15.5:0/611445406 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f928c0cff00 con 0x7f9294107900 2024-09-19T06:03:01.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.265+0000 7f9292d9d640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f92800a32d0 0x7f92800a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f927c000c00 tx=0x7f927c005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:01.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.268+0000 7f928bfff640 1 -- 172.21.15.5:0/611445406 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f928c014030 con 0x7f9294107900 2024-09-19T06:03:01.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.447+0000 7f929918a640 1 -- 172.21.15.5:0/611445406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f929402d970 con 0x7f9294107900 2024-09-19T06:03:01.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.448+0000 7f928bfff640 1 -- 172.21.15.5:0/611445406 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5274 (secure 0 0 0) 0x7f928c09cab0 con 0x7f9294107900 2024-09-19T06:03:01.450 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 83 2024-09-19T06:03:01.450 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T06:03:01.450 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-09-19T05:53:51.454068+0000 2024-09-19T06:03:01.450 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-09-19T06:01:39.826379+0000 2024-09-19T06:03:01.450 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 26 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client luminous 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release squid 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 12 2024-09-19T06:03:01.451 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6802/3442981810,v1:172.21.15.5:6803/3442981810] [v2:172.21.15.5:6804/3442981810,v1:172.21.15.5:6805/3442981810] exists,up f3c10d60-d354-407f-9cdf-099344599844 2024-09-19T06:03:01.452 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6800/1448265014,v1:172.21.15.49:6801/1448265014] [v2:172.21.15.49:6802/1448265014,v1:172.21.15.49:6803/1448265014] exists,up 5f40f636-7761-45ac-87f3-910a2fa2feb2 2024-09-19T06:03:01.452 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 18 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6800/2319626622,v1:172.21.15.157:6801/2319626622] [v2:172.21.15.157:6802/2319626622,v1:172.21.15.157:6803/2319626622] exists,up ad00940c-4fdd-4f5f-9eaa-b1f38bcac526 2024-09-19T06:03:01.452 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 25 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6810/1452491753,v1:172.21.15.5:6811/1452491753] [v2:172.21.15.5:6812/1452491753,v1:172.21.15.5:6813/1452491753] exists,up 63fec329-1b62-4c5e-b8ca-084a84d4d762 2024-09-19T06:03:01.452 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 32 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6808/3114952580,v1:172.21.15.49:6809/3114952580] [v2:172.21.15.49:6810/3114952580,v1:172.21.15.49:6811/3114952580] exists,up ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26 2024-09-19T06:03:01.452 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 37 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6808/3219063971,v1:172.21.15.157:6809/3219063971] [v2:172.21.15.157:6810/3219063971,v1:172.21.15.157:6811/3219063971] exists,up 21a7a7d3-ce42-4688-bfec-714be635eaa2 2024-09-19T06:03:01.452 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 up in weight 1 up_from 44 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6818/2957795194,v1:172.21.15.5:6819/2957795194] [v2:172.21.15.5:6820/2957795194,v1:172.21.15.5:6821/2957795194] exists,up cbef0e08-d6b0-4515-a2ee-2c81b048583b 2024-09-19T06:03:01.452 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 up in weight 1 up_from 49 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6816/2959731400,v1:172.21.15.49:6817/2959731400] [v2:172.21.15.49:6818/2959731400,v1:172.21.15.49:6819/2959731400] exists,up d261ac55-532d-4aea-9b3e-ed2b4eeb1da6 2024-09-19T06:03:01.452 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 up in weight 1 up_from 56 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6816/290026859,v1:172.21.15.157:6817/290026859] [v2:172.21.15.157:6818/290026859,v1:172.21.15.157:6819/290026859] exists,up 15d0eaf7-d604-4546-b393-4d6d39018f42 2024-09-19T06:03:01.452 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 up in weight 1 up_from 63 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6826/4122448237,v1:172.21.15.5:6827/4122448237] [v2:172.21.15.5:6828/4122448237,v1:172.21.15.5:6829/4122448237] exists,up 30c08537-f593-4299-bfa1-fa7ba8a95169 2024-09-19T06:03:01.452 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 up in weight 1 up_from 68 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6824/2551748002,v1:172.21.15.49:6825/2551748002] [v2:172.21.15.49:6826/2551748002,v1:172.21.15.49:6827/2551748002] exists,up 68821e98-bd81-49ec-8cd6-51fb9880b553 2024-09-19T06:03:01.452 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 up in weight 1 up_from 75 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6824/1996582544,v1:172.21.15.157:6825/1996582544] [v2:172.21.15.157:6826/1996582544,v1:172.21.15.157:6827/1996582544] exists,up 7e443b46-2697-4d0b-8452-868463a1ee1b 2024-09-19T06:03:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.5 [8,11] 2024-09-19T06:03:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.c [2,1] 2024-09-19T06:03:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1b [2,4] 2024-09-19T06:03:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1c [8,11] 2024-09-19T06:03:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.1 [4,9] 2024-09-19T06:03:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.6 [4,6,3,8] 2024-09-19T06:03:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.e [5,9] 2024-09-19T06:03:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.14 [4,7] 2024-09-19T06:03:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.20 [5,9] 2024-09-19T06:03:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/4102275559 expires 2024-09-20T05:54:38.703198+0000 2024-09-19T06:03:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2593868621 expires 2024-09-20T05:54:38.703198+0000 2024-09-19T06:03:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1340147787 expires 2024-09-20T05:54:38.703198+0000 2024-09-19T06:03:01.454 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/255461399 expires 2024-09-20T05:54:38.703198+0000 2024-09-19T06:03:01.454 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/255461399 expires 2024-09-20T05:54:38.703198+0000 2024-09-19T06:03:01.454 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/330367622 expires 2024-09-20T05:54:15.334457+0000 2024-09-19T06:03:01.454 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/850678758 expires 2024-09-20T05:54:15.334457+0000 2024-09-19T06:03:01.454 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/330367622 expires 2024-09-20T05:54:15.334457+0000 2024-09-19T06:03:01.454 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2930274429 expires 2024-09-20T05:54:15.334457+0000 2024-09-19T06:03:01.454 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1367598148 expires 2024-09-20T05:54:15.334457+0000 2024-09-19T06:03:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.451+0000 7f929918a640 1 -- 172.21.15.5:0/611445406 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f92800a32d0 msgr2=0x7f92800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.451+0000 7f929918a640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f92800a32d0 0x7f92800a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f927c000c00 tx=0x7f927c005e40 comp rx=0 tx=0).stop 2024-09-19T06:03:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.451+0000 7f929918a640 1 -- 172.21.15.5:0/611445406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294107900 msgr2=0x7f9294100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.451+0000 7f929918a640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294107900 0x7f9294100b90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f928c00cce0 tx=0x7f928c00a620 comp rx=0 tx=0).stop 2024-09-19T06:03:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.451+0000 7f929918a640 1 -- 172.21.15.5:0/611445406 shutdown_connections 2024-09-19T06:03:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.451+0000 7f929918a640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f92800a32d0 0x7f92800a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.451+0000 7f929918a640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294107900 0x7f9294100b90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.451+0000 7f929918a640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9294104fd0 0x7f9294100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.451+0000 7f929918a640 1 --2- 172.21.15.5:0/611445406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92941026a0 0x7f9294102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.451+0000 7f929918a640 1 -- 172.21.15.5:0/611445406 >> 172.21.15.5:0/611445406 conn(0x7f92940fc2a0 msgr2=0x7f9294102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.451+0000 7f929918a640 1 -- 172.21.15.5:0/611445406 shutdown_connections 2024-09-19T06:03:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:01.451+0000 7f929918a640 1 -- 172.21.15.5:0/611445406 wait complete. 2024-09-19T06:03:02.039 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T06:03:02.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:03:02.306 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:02.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:02 smithi005 ceph-mon[30955]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:03:02.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:02 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/611445406' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T06:03:02.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:02 smithi049 ceph-mon[34640]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:03:02.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:02 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/611445406' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T06:03:02.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.772+0000 7f7448c4b640 1 -- 172.21.15.5:0/3280931995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7444103700 msgr2=0x7f744410bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:02.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.772+0000 7f7448c4b640 1 --2- 172.21.15.5:0/3280931995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7444103700 0x7f744410bbf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f742c009900 tx=0x7f742c02a480 comp rx=0 tx=0).stop 2024-09-19T06:03:02.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.772+0000 7f7448c4b640 1 -- 172.21.15.5:0/3280931995 shutdown_connections 2024-09-19T06:03:02.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.772+0000 7f7448c4b640 1 --2- 172.21.15.5:0/3280931995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f744410c180 0x7f744410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:02.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.772+0000 7f7448c4b640 1 --2- 172.21.15.5:0/3280931995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7444103700 0x7f744410bbf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:02.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.772+0000 7f7448c4b640 1 --2- 172.21.15.5:0/3280931995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7444102de0 0x7f74441031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:02.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.772+0000 7f7448c4b640 1 -- 172.21.15.5:0/3280931995 >> 172.21.15.5:0/3280931995 conn(0x7f74440fc7a0 msgr2=0x7f74440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:02.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.772+0000 7f7448c4b640 1 -- 172.21.15.5:0/3280931995 shutdown_connections 2024-09-19T06:03:02.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.772+0000 7f7448c4b640 1 -- 172.21.15.5:0/3280931995 wait complete. 2024-09-19T06:03:02.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.773+0000 7f7448c4b640 1 Processor -- start 2024-09-19T06:03:02.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.773+0000 7f7448c4b640 1 -- start start 2024-09-19T06:03:02.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.773+0000 7f7448c4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7444102de0 0x7f744419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:02.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.773+0000 7f7448c4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7444103700 0x7f744419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:02.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.773+0000 7f7448c4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f744410c180 0x7f744419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:02.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.773+0000 7f7448c4b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7444111e20 con 0x7f7444103700 2024-09-19T06:03:02.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.773+0000 7f7448c4b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7444111ca0 con 0x7f7444102de0 2024-09-19T06:03:02.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.773+0000 7f7448c4b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7444111fa0 con 0x7f744410c180 2024-09-19T06:03:02.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.773+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7444103700 0x7f744419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:02.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.773+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7444103700 0x7f744419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42340/0 (socket says 172.21.15.5:42340) 2024-09-19T06:03:02.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.773+0000 7f7439d9b640 1 -- 172.21.15.5:0/1868975638 learned_addr learned my addr 172.21.15.5:0/1868975638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:02.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f7442d9d640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f744410c180 0x7f744419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:02.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f744259c640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7444102de0 0x7f744419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:02.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f7439d9b640 1 -- 172.21.15.5:0/1868975638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f744410c180 msgr2=0x7f744419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:02.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f7439d9b640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f744410c180 0x7f744419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:02.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f7439d9b640 1 -- 172.21.15.5:0/1868975638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7444102de0 msgr2=0x7f744419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:02.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f7439d9b640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7444102de0 0x7f744419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:02.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f7439d9b640 1 -- 172.21.15.5:0/1868975638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744419f510 con 0x7f7444103700 2024-09-19T06:03:02.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f7439d9b640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7444103700 0x7f744419aaa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f742c005940 tx=0x7f742c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:02.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f743bfff640 1 -- 172.21.15.5:0/1868975638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c03f030 con 0x7f7444103700 2024-09-19T06:03:02.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f7448c4b640 1 -- 172.21.15.5:0/1868975638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744419f7a0 con 0x7f7444103700 2024-09-19T06:03:02.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f743bfff640 1 -- 172.21.15.5:0/1868975638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f742c003aa0 con 0x7f7444103700 2024-09-19T06:03:02.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f743bfff640 1 -- 172.21.15.5:0/1868975638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c035070 con 0x7f7444103700 2024-09-19T06:03:02.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.774+0000 7f7448c4b640 1 -- 172.21.15.5:0/1868975638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74441a70e0 con 0x7f7444103700 2024-09-19T06:03:02.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.775+0000 7f7448c4b640 1 -- 172.21.15.5:0/1868975638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f744406afd0 con 0x7f7444103700 2024-09-19T06:03:02.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.777+0000 7f743bfff640 1 -- 172.21.15.5:0/1868975638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f742c042030 con 0x7f7444103700 2024-09-19T06:03:02.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.777+0000 7f743bfff640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f74300a32d0 0x7f74300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:02.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.777+0000 7f744259c640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f74300a32d0 0x7f74300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:02.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.778+0000 7f743bfff640 1 -- 172.21.15.5:0/1868975638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f742c0f0410 con 0x7f7444103700 2024-09-19T06:03:02.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.778+0000 7f744259c640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f74300a32d0 0x7f74300a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f743400bbe0 tx=0x7f743400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:02.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.781+0000 7f743bfff640 1 -- 172.21.15.5:0/1868975638 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f742c0bc970 con 0x7f7444103700 2024-09-19T06:03:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:02 smithi157 ceph-mon[35346]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:03:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:02 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/611445406' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T06:03:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.996+0000 7f7448c4b640 1 -- 172.21.15.5:0/1868975638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f744419b8c0 con 0x7f7444103700 2024-09-19T06:03:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.996+0000 7f743bfff640 1 -- 172.21.15.5:0/1868975638 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11959 (secure 0 0 0) 0x7f742c0c0c70 con 0x7f7444103700 2024-09-19T06:03:02.998 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:03:02.999 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-19T06:01:41:840933+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:01:41.840914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:03:02.999 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-19T06:03:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.998+0000 7f7448c4b640 1 -- 172.21.15.5:0/1868975638 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f74300a32d0 msgr2=0x7f74300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.998+0000 7f7448c4b640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f74300a32d0 0x7f74300a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f743400bbe0 tx=0x7f743400b390 comp rx=0 tx=0).stop 2024-09-19T06:03:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.998+0000 7f7448c4b640 1 -- 172.21.15.5:0/1868975638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7444103700 msgr2=0x7f744419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.998+0000 7f7448c4b640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7444103700 0x7f744419aaa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f742c005940 tx=0x7f742c004270 comp rx=0 tx=0).stop 2024-09-19T06:03:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.998+0000 7f7448c4b640 1 -- 172.21.15.5:0/1868975638 shutdown_connections 2024-09-19T06:03:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.998+0000 7f7448c4b640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f74300a32d0 0x7f74300a5790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.998+0000 7f7448c4b640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f744410c180 0x7f744419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.999+0000 7f7448c4b640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7444103700 0x7f744419aaa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.999+0000 7f7448c4b640 1 --2- 172.21.15.5:0/1868975638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7444102de0 0x7f744419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.999+0000 7f7448c4b640 1 -- 172.21.15.5:0/1868975638 >> 172.21.15.5:0/1868975638 conn(0x7f74440fc7a0 msgr2=0x7f7444101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.999+0000 7f7448c4b640 1 -- 172.21.15.5:0/1868975638 shutdown_connections 2024-09-19T06:03:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:02.999+0000 7f7448c4b640 1 -- 172.21.15.5:0/1868975638 wait complete. 2024-09-19T06:03:03.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:03 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1868975638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:03:03.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:03 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1868975638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:03:03.661 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:03:03.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:03 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1868975638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:03:03.924 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.418+0000 7fa30c517640 1 -- 172.21.15.5:0/3001214679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304103660 msgr2=0x7fa30410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.418+0000 7fa30c517640 1 --2- 172.21.15.5:0/3001214679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304103660 0x7fa30410bb50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009900 tx=0x7fa2f402a480 comp rx=0 tx=0).stop 2024-09-19T06:03:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 -- 172.21.15.5:0/3001214679 shutdown_connections 2024-09-19T06:03:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 --2- 172.21.15.5:0/3001214679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa30410c170 0x7fa30410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 --2- 172.21.15.5:0/3001214679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304103660 0x7fa30410bb50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 --2- 172.21.15.5:0/3001214679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa304102d40 0x7fa304103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 -- 172.21.15.5:0/3001214679 >> 172.21.15.5:0/3001214679 conn(0x7fa3040fc7a0 msgr2=0x7fa3040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 -- 172.21.15.5:0/3001214679 shutdown_connections 2024-09-19T06:03:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 -- 172.21.15.5:0/3001214679 wait complete. 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 Processor -- start 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 -- start start 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa304102d40 0x7fa304198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa304103660 0x7fa304198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30410c170 0x7fa30419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa304111e20 con 0x7fa30410c170 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa304111ca0 con 0x7fa304102d40 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.419+0000 7fa30c517640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa304111fa0 con 0x7fa304103660 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa30aab4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30410c170 0x7fa30419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa30aab4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30410c170 0x7fa30419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42364/0 (socket says 172.21.15.5:42364) 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa30aab4640 1 -- 172.21.15.5:0/2604942692 learned_addr learned my addr 172.21.15.5:0/2604942692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa30a2b3640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa304102d40 0x7fa304198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa309ab2640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa304103660 0x7fa304198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa30aab4640 1 -- 172.21.15.5:0/2604942692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa304103660 msgr2=0x7fa304198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa30aab4640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa304103660 0x7fa304198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa30aab4640 1 -- 172.21.15.5:0/2604942692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa304102d40 msgr2=0x7fa304198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa30aab4640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa304102d40 0x7fa304198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa30aab4640 1 -- 172.21.15.5:0/2604942692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30419d300 con 0x7fa30410c170 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa309ab2640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa304103660 0x7fa304198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa30aab4640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30410c170 0x7fa30419cc20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00eb10 tx=0x7fa2fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa2fb7fe640 1 -- 172.21.15.5:0/2604942692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc004b20 con 0x7fa30410c170 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa30c517640 1 -- 172.21.15.5:0/2604942692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30419d5f0 con 0x7fa30410c170 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa2fb7fe640 1 -- 172.21.15.5:0/2604942692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2fc004cc0 con 0x7fa30410c170 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa2fb7fe640 1 -- 172.21.15.5:0/2604942692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc010690 con 0x7fa30410c170 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.420+0000 7fa30c517640 1 -- 172.21.15.5:0/2604942692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3041a4ed0 con 0x7fa30410c170 2024-09-19T06:03:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.421+0000 7fa30c517640 1 -- 172.21.15.5:0/2604942692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa304103120 con 0x7fa30410c170 2024-09-19T06:03:04.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.423+0000 7fa2fb7fe640 1 -- 172.21.15.5:0/2604942692 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fa2fc005000 con 0x7fa30410c170 2024-09-19T06:03:04.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.423+0000 7fa2fb7fe640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa2f00a32d0 0x7fa2f00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:04.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.423+0000 7fa30a2b3640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa2f00a32d0 0x7fa2f00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:04.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.423+0000 7fa2fb7fe640 1 -- 172.21.15.5:0/2604942692 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa2fc0d0310 con 0x7fa30410c170 2024-09-19T06:03:04.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.424+0000 7fa30a2b3640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa2f00a32d0 0x7fa2f00a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa300009b60 tx=0x7fa300005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:04.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.427+0000 7fa2fb7fe640 1 -- 172.21.15.5:0/2604942692 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa2fc054210 con 0x7fa30410c170 2024-09-19T06:03:04.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:04 smithi005 ceph-mon[30955]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:03:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.639+0000 7fa30c517640 1 -- 172.21.15.5:0/2604942692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa304107240 con 0x7fa30410c170 2024-09-19T06:03:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.640+0000 7fa2fb7fe640 1 -- 172.21.15.5:0/2604942692 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11959 (secure 0 0 0) 0x7fa2fc09cec0 con 0x7fa30410c170 2024-09-19T06:03:04.642 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:03:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-19T06:03:04.643 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-19T06:01:41:840933+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:01:41.840914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:03:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.642+0000 7fa30c517640 1 -- 172.21.15.5:0/2604942692 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa2f00a32d0 msgr2=0x7fa2f00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.642+0000 7fa30c517640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa2f00a32d0 0x7fa2f00a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa300009b60 tx=0x7fa300005f90 comp rx=0 tx=0).stop 2024-09-19T06:03:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.642+0000 7fa30c517640 1 -- 172.21.15.5:0/2604942692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30410c170 msgr2=0x7fa30419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.643+0000 7fa30c517640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30410c170 0x7fa30419cc20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00eb10 tx=0x7fa2fc00c2d0 comp rx=0 tx=0).stop 2024-09-19T06:03:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.643+0000 7fa30c517640 1 -- 172.21.15.5:0/2604942692 shutdown_connections 2024-09-19T06:03:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.643+0000 7fa30c517640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fa2f00a32d0 0x7fa2f00a5790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.643+0000 7fa30c517640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30410c170 0x7fa30419cc20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.643+0000 7fa30c517640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa304103660 0x7fa304198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.643+0000 7fa30c517640 1 --2- 172.21.15.5:0/2604942692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa304102d40 0x7fa304198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.643+0000 7fa30c517640 1 -- 172.21.15.5:0/2604942692 >> 172.21.15.5:0/2604942692 conn(0x7fa3040fc7a0 msgr2=0x7fa3040fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.643+0000 7fa30c517640 1 -- 172.21.15.5:0/2604942692 shutdown_connections 2024-09-19T06:03:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:04.643+0000 7fa30c517640 1 -- 172.21.15.5:0/2604942692 wait complete. 2024-09-19T06:03:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:04 smithi049 ceph-mon[34640]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:03:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:04 smithi157 ceph-mon[35346]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:03:05.529 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:03:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:05 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2604942692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:03:05.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:05 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2604942692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:03:05.797 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:05 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2604942692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:03:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.260+0000 7fb2e151d640 1 -- 172.21.15.5:0/4008566689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc101de0 msgr2=0x7fb2dc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.260+0000 7fb2e151d640 1 --2- 172.21.15.5:0/4008566689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc101de0 0x7fb2dc1021c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009900 tx=0x7fb2cc02a410 comp rx=0 tx=0).stop 2024-09-19T06:03:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.260+0000 7fb2e151d640 1 -- 172.21.15.5:0/4008566689 shutdown_connections 2024-09-19T06:03:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.260+0000 7fb2e151d640 1 --2- 172.21.15.5:0/4008566689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2dc106f10 0x7fb2dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.260+0000 7fb2e151d640 1 --2- 172.21.15.5:0/4008566689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2dc102790 0x7fb2dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.260+0000 7fb2e151d640 1 --2- 172.21.15.5:0/4008566689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc101de0 0x7fb2dc1021c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.260+0000 7fb2e151d640 1 -- 172.21.15.5:0/4008566689 >> 172.21.15.5:0/4008566689 conn(0x7fb2dc0fd8c0 msgr2=0x7fb2dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.260+0000 7fb2e151d640 1 -- 172.21.15.5:0/4008566689 shutdown_connections 2024-09-19T06:03:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.260+0000 7fb2e151d640 1 -- 172.21.15.5:0/4008566689 wait complete. 2024-09-19T06:03:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.260+0000 7fb2e151d640 1 Processor -- start 2024-09-19T06:03:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.260+0000 7fb2e151d640 1 -- start start 2024-09-19T06:03:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2e151d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2dc101de0 0x7fb2dc19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2e151d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc102790 0x7fb2dc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2e151d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2dc106f10 0x7fb2dc19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2e151d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc111e40 con 0x7fb2dc102790 2024-09-19T06:03:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2e151d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc111cc0 con 0x7fb2dc106f10 2024-09-19T06:03:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2e151d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc111fc0 con 0x7fb2dc101de0 2024-09-19T06:03:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2da7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc102790 0x7fb2dc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2da7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc102790 0x7fb2dc19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51908/0 (socket says 172.21.15.5:51908) 2024-09-19T06:03:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2da7fc640 1 -- 172.21.15.5:0/2473053851 learned_addr learned my addr 172.21.15.5:0/2473053851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2db7fe640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2dc106f10 0x7fb2dc19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2daffd640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2dc101de0 0x7fb2dc19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2da7fc640 1 -- 172.21.15.5:0/2473053851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2dc101de0 msgr2=0x7fb2dc19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2da7fc640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2dc101de0 0x7fb2dc19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2da7fc640 1 -- 172.21.15.5:0/2473053851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2dc106f10 msgr2=0x7fb2dc19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2da7fc640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2dc106f10 0x7fb2dc19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2da7fc640 1 -- 172.21.15.5:0/2473053851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc19f550 con 0x7fb2dc102790 2024-09-19T06:03:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.261+0000 7fb2da7fc640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc102790 0x7fb2dc19aae0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000b4a0 tx=0x7fb2d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.262+0000 7fb2c3fff640 1 -- 172.21.15.5:0/2473053851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0014070 con 0x7fb2dc102790 2024-09-19T06:03:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.262+0000 7fb2c3fff640 1 -- 172.21.15.5:0/2473053851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2d0010ac0 con 0x7fb2dc102790 2024-09-19T06:03:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.262+0000 7fb2e151d640 1 -- 172.21.15.5:0/2473053851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc19f840 con 0x7fb2dc102790 2024-09-19T06:03:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.262+0000 7fb2c3fff640 1 -- 172.21.15.5:0/2473053851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0017c30 con 0x7fb2dc102790 2024-09-19T06:03:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.262+0000 7fb2e151d640 1 -- 172.21.15.5:0/2473053851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc1a7120 con 0x7fb2dc102790 2024-09-19T06:03:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.263+0000 7fb2e151d640 1 -- 172.21.15.5:0/2473053851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2dc1021c0 con 0x7fb2dc102790 2024-09-19T06:03:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.264+0000 7fb2c3fff640 1 -- 172.21.15.5:0/2473053851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fb2d0017070 con 0x7fb2dc102790 2024-09-19T06:03:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.265+0000 7fb2c3fff640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb2c40a32d0 0x7fb2c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.265+0000 7fb2daffd640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb2c40a32d0 0x7fb2c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.265+0000 7fb2c3fff640 1 -- 172.21.15.5:0/2473053851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb2d00cf0f0 con 0x7fb2dc102790 2024-09-19T06:03:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.265+0000 7fb2daffd640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb2c40a32d0 0x7fb2c40a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc031440 tx=0x7fb2cc0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.268+0000 7fb2c3fff640 1 -- 172.21.15.5:0/2473053851 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb2d009bdb0 con 0x7fb2dc102790 2024-09-19T06:03:06.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.446+0000 7fb2e151d640 1 -- 172.21.15.5:0/2473053851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2dc19b900 con 0x7fb2dc102790 2024-09-19T06:03:06.448 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:06 smithi005 ceph-mon[30955]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:03:06.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.448+0000 7fb2c3fff640 1 -- 172.21.15.5:0/2473053851 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22702 (secure 0 0 0) 0x7fb2dc19b900 con 0x7fb2dc102790 2024-09-19T06:03:06.450 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:03:06.450 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","created":"2024-09-19T05:53:51.454068+0000","modified":"2024-09-19T06:01:39.826379+0000","last_up_change":"2024-09-19T06:00:38.837916+0000","last_in_change":"2024-09-19T06:00:15.609776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:56:55.213931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:01:25.648588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:01:27.760408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f3c10d60-d354-407f-9cdf-099344599844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6803","nonce":3442981810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6805","nonce":3442981810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6809","nonce":3442981810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6807","nonce":3442981810}]},"public_addr":"172.21.15.5:6803/3442981810","cluster_addr":"172.21.15.5:6805/3442981810","heartbeat_back_addr":"172.21.15.5:6809/3442981810","heartbeat_front_addr":"172.21.15.5:6807/3442981810","state":["exists","up"]},{"osd":1,"uuid":"5f40f636-7761-45ac-87f3-910a2fa2feb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448265014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6803","nonce":1448265014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6807","nonce":1448265014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6805","nonce":1448265014}]},"public_addr":"172.21.15.49:6801/1448265014","cluster_addr":"172.21.15.49:6803/1448265014","heartbeat_back_addr":"172.21.15.49:6807/1448265014","heartbeat_front_addr":"172.21.15.49:6805/1448265014","state":["exists","up"]},{"osd":2,"uuid":"ad00940c-4fdd-4f5f-9eaa-b1f38bcac526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6801","nonce":2319626622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6803","nonce":2319626622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6807","nonce":2319626622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6805","nonce":2319626622}]},"public_addr":"172.21.15.157:6801/2319626622","cluster_addr":"172.21.15.157:6803/2319626622","heartbeat_back_addr":"172.21.15.157:6807/2319626622","heartbeat_front_addr":"172.21.15.157:6805/2319626622","state":["exists","up"]},{"osd":3,"uuid":"63fec329-1b62-4c5e-b8ca-084a84d4d762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6811","nonce":1452491753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6813","nonce":1452491753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6817","nonce":1452491753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6815","nonce":1452491753}]},"public_addr":"172.21.15.5:6811/1452491753","cluster_addr":"172.21.15.5:6813/1452491753","heartbeat_back_addr":"172.21.15.5:6817/1452491753","heartbeat_front_addr":"172.21.15.5:6815/1452491753","state":["exists","up"]},{"osd":4,"uuid":"ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6809","nonce":3114952580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6811","nonce":3114952580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6815","nonce":3114952580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6813","nonce":3114952580}]},"public_addr":"172.21.15.49:6809/3114952580","cluster_addr":"172.21.15.49:6811/3114952580","heartbeat_back_addr":"172.21.15.49:6815/3114952580","heartbeat_front_addr":"172.21.15.49:6813/3114952580","state":["exists","up"]},{"osd":5,"uuid":"21a7a7d3-ce42-4688-bfec-714be635eaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6809","nonce":3219063971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6811","nonce":3219063971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6815","nonce":3219063971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6813","nonce":3219063971}]},"public_addr":"172.21.15.157:6809/3219063971","cluster_addr":"172.21.15.157:6811/3219063971","heartbeat_back_addr":"172.21.15.157:6815/3219063971","heartbeat_front_addr":"172.21.15.157:6813/3219063971","state":["exists","up"]},{"osd":6,"uuid":"cbef0e08-d6b0-4515-a2ee-2c81b048583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6819","nonce":2957795194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6821","nonce":2957795194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6825","nonce":2957795194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6823","nonce":2957795194}]},"public_addr":"172.21.15.5:6819/2957795194","cluster_addr":"172.21.15.5:6821/2957795194","heartbeat_back_addr":"172.21.15.5:6825/2957795194","heartbeat_front_addr":"172.21.15.5:6823/2957795194","state":["exists","up"]},{"osd":7,"uuid":"d261ac55-532d-4aea-9b3e-ed2b4eeb1da6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6817","nonce":2959731400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6819","nonce":2959731400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6823","nonce":2959731400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6821","nonce":2959731400}]},"public_addr":"172.21.15.49:6817/2959731400","cluster_addr":"172.21.15.49:6819/2959731400","heartbeat_back_addr":"172.21.15.49:6823/2959731400","heartbeat_front_addr":"172.21.15.49:6821/2959731400","state":["exists","up"]},{"osd":8,"uuid":"15d0eaf7-d604-4546-b393-4d6d39018f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6817","nonce":290026859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6819","nonce":290026859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6823","nonce":290026859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6821","nonce":290026859}]},"public_addr":"172.21.15.157:6817/290026859","cluster_addr":"172.21.15.157:6819/290026859","heartbeat_back_addr":"172.21.15.157:6823/290026859","heartbeat_front_addr":"172.21.15.157:6821/290026859","state":["exists","up"]},{"osd":9,"uuid":"30c08537-f593-4299-bfa1-fa7ba8a95169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6827","nonce":4122448237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6829","nonce":4122448237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6833","nonce":4122448237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6831","nonce":4122448237}]},"public_addr":"172.21.15.5:6827/4122448237","cluster_addr":"172.21.15.5:6829/4122448237","heartbeat_back_addr":"172.21.15.5:6833/4122448237","heartbeat_front_addr":"172.21.15.5:6831/4122448237","state":["exists","up"]},{"osd":10,"uuid":"68821e98-bd81-49ec-8cd6-51fb9880b553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6825","nonce":2551748002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6827","nonce":2551748002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6831","nonce":2551748002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6829","nonce":2551748002}]},"public_addr":"172.21.15.49:6825/2551748002","cluster_addr":"172.21.15.49:6827/2551748002","heartbeat_back_addr":"172.21.15.49:6831/2551748002","heartbeat_front_addr":"172.21.15.49:6829/2551748002","state":["exists","up"]},{"osd":11,"uuid":"7e443b46-2697-4d0b-8452-868463a1ee1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6825","nonce":1996582544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6827","nonce":1996582544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6831","nonce":1996582544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6829","nonce":1996582544}]},"public_addr":"172.21.15.157:6825/1996582544","cluster_addr":"172.21.15.157:6827/1996582544","heartbeat_back_addr":"172.21.15.157:6831/1996582544","heartbeat_front_addr":"172.21.15.157:6829/1996582544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:01.453592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:27.194705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:51.525385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:17.676749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:43.075555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:07.258154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:32.112583+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:57.306534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:21.320950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:47.164610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:12.548337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:37.262580+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4102275559":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/2593868621":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/1340147787":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6800/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6801/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/850678758":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6800/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2930274429":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/1367598148":"2024-09-20T05:54:15.334457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:03:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.450+0000 7fb2e151d640 1 -- 172.21.15.5:0/2473053851 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb2c40a32d0 msgr2=0x7fb2c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.450+0000 7fb2e151d640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb2c40a32d0 0x7fb2c40a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc031440 tx=0x7fb2cc0095f0 comp rx=0 tx=0).stop 2024-09-19T06:03:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.450+0000 7fb2e151d640 1 -- 172.21.15.5:0/2473053851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc102790 msgr2=0x7fb2dc19aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.450+0000 7fb2e151d640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc102790 0x7fb2dc19aae0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000b4a0 tx=0x7fb2d000b960 comp rx=0 tx=0).stop 2024-09-19T06:03:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.450+0000 7fb2e151d640 1 -- 172.21.15.5:0/2473053851 shutdown_connections 2024-09-19T06:03:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.450+0000 7fb2e151d640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fb2c40a32d0 0x7fb2c40a5790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.450+0000 7fb2e151d640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2dc106f10 0x7fb2dc19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.450+0000 7fb2e151d640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc102790 0x7fb2dc19aae0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.450+0000 7fb2e151d640 1 --2- 172.21.15.5:0/2473053851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2dc101de0 0x7fb2dc19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.450+0000 7fb2e151d640 1 -- 172.21.15.5:0/2473053851 >> 172.21.15.5:0/2473053851 conn(0x7fb2dc0fd8c0 msgr2=0x7fb2dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.450+0000 7fb2e151d640 1 -- 172.21.15.5:0/2473053851 shutdown_connections 2024-09-19T06:03:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:06.450+0000 7fb2e151d640 1 -- 172.21.15.5:0/2473053851 wait complete. 2024-09-19T06:03:06.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:06 smithi049 ceph-mon[34640]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:03:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:06 smithi157 ceph-mon[35346]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T06:03:07.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T06:01:29.531007+0000', 'modified': '2024-09-19T06:01:41.840914+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24473, 'mds_1': 24463, 'mds_2': 24448, 'mds_3': 24442, 'mds_4': 24437}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14625': {'gid': 14625, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/1310081153', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 1310081153}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 1310081153}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14637': {'gid': 14637, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/569764382', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 569764382}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 569764382}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6833/2427602354', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6832', 'nonce': 2427602354}, {'type': 'v1', 'addr': '172.21.15.49:6833', 'nonce': 2427602354}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6833/1048665474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 1048665474}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 1048665474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6835/2353689058', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 2353689058}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 2353689058}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.49:6837/2858664643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6836', 'nonce': 2858664643}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 2858664643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.157:6835/2070926870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 2070926870}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 2070926870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.157:6837/3264499329', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 3264499329}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 3264499329}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.157:6839/4294596962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 4294596962}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 4294596962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.49:6839/2359039211', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6838', 'nonce': 2359039211}, {'type': 'v1', 'addr': '172.21.15.49:6839', 'nonce': 2359039211}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24473, 'qdb_cluster': [24473, 24448, 24442, 24463, 24437]} 2024-09-19T06:03:07.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T06:03:07.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T06:03:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:07.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:07 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2473053851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:03:07.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:07 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2473053851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:03:07.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:07 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2473053851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:03:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.903+0000 7fee14f42640 1 -- 172.21.15.5:0/449487495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee10100a80 msgr2=0x7fee10100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.903+0000 7fee14f42640 1 --2- 172.21.15.5:0/449487495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee10100a80 0x7fee10100ee0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009930 tx=0x7fedf802a420 comp rx=0 tx=0).stop 2024-09-19T06:03:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.903+0000 7fee14f42640 1 -- 172.21.15.5:0/449487495 shutdown_connections 2024-09-19T06:03:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.903+0000 7fee14f42640 1 --2- 172.21.15.5:0/449487495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee10101420 0x7fee1010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.903+0000 7fee14f42640 1 --2- 172.21.15.5:0/449487495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee10100a80 0x7fee10100ee0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.903+0000 7fee14f42640 1 --2- 172.21.15.5:0/449487495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee10106a80 0x7fee10106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.903+0000 7fee14f42640 1 -- 172.21.15.5:0/449487495 >> 172.21.15.5:0/449487495 conn(0x7fee100fc7a0 msgr2=0x7fee100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.903+0000 7fee14f42640 1 -- 172.21.15.5:0/449487495 shutdown_connections 2024-09-19T06:03:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.903+0000 7fee14f42640 1 -- 172.21.15.5:0/449487495 wait complete. 2024-09-19T06:03:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.904+0000 7fee14f42640 1 Processor -- start 2024-09-19T06:03:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.904+0000 7fee14f42640 1 -- start start 2024-09-19T06:03:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.904+0000 7fee14f42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee10100a80 0x7fee101991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.904+0000 7fee14f42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee10101420 0x7fee10199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.904+0000 7fee14f42640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee10106a80 0x7fee101a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.904+0000 7fee14f42640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee10110d30 con 0x7fee10100a80 2024-09-19T06:03:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.904+0000 7fee14f42640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee10110bb0 con 0x7fee10106a80 2024-09-19T06:03:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.904+0000 7fee14f42640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee10110eb0 con 0x7fee10101420 2024-09-19T06:03:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.904+0000 7fee0e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee10100a80 0x7fee101991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.904+0000 7fee0e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee10100a80 0x7fee101991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51926/0 (socket says 172.21.15.5:51926) 2024-09-19T06:03:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.904+0000 7fee0e59c640 1 -- 172.21.15.5:0/1415716145 learned_addr learned my addr 172.21.15.5:0/1415716145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.904+0000 7fee0ed9d640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee10106a80 0x7fee101a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee07fff640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee10101420 0x7fee10199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee0e59c640 1 -- 172.21.15.5:0/1415716145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee10101420 msgr2=0x7fee10199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee0e59c640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee10101420 0x7fee10199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee0e59c640 1 -- 172.21.15.5:0/1415716145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee10106a80 msgr2=0x7fee101a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee0e59c640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee10106a80 0x7fee101a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee0e59c640 1 -- 172.21.15.5:0/1415716145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee101a4780 con 0x7fee10100a80 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee07fff640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee10101420 0x7fee10199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee0e59c640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee10100a80 0x7fee101991c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fee0000d950 tx=0x7fee0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee077fe640 1 -- 172.21.15.5:0/1415716145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00017070 con 0x7fee10100a80 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee077fe640 1 -- 172.21.15.5:0/1415716145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee00009c30 con 0x7fee10100a80 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee077fe640 1 -- 172.21.15.5:0/1415716145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0000f620 con 0x7fee10100a80 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee14f42640 1 -- 172.21.15.5:0/1415716145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee101a4950 con 0x7fee10100a80 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.905+0000 7fee14f42640 1 -- 172.21.15.5:0/1415716145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee101a4e60 con 0x7fee10100a80 2024-09-19T06:03:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee077fe640 1 -- 172.21.15.5:0/1415716145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fee000109a0 con 0x7fee10100a80 2024-09-19T06:03:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee1731b740 1 Processor -- start 2024-09-19T06:03:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee1731b740 1 -- start start 2024-09-19T06:03:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee1731b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39af5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee1731b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39b00670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee1731b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39b02fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee1731b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d39a5c090 con 0x557d39b00bb0 2024-09-19T06:03:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee1731b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557d39a03a70 con 0x557d39af59d0 2024-09-19T06:03:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee1731b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557d39a58050 con 0x557d39af62f0 2024-09-19T06:03:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39b02fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39b02fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51938/0 (socket says 172.21.15.5:51938) 2024-09-19T06:03:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee07fff640 1 -- 172.21.15.5:0/4037437939 learned_addr learned my addr 172.21.15.5:0/4037437939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee0ed9d640 1 --2- 172.21.15.5:0/4037437939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39af5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.908+0000 7fee077fe640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fedfc0a3350 0x7fedfc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee0e59c640 1 --2- 172.21.15.5:0/4037437939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39b00670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee0ed9d640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fedfc0a3350 0x7fedfc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee07fff640 1 -- 172.21.15.5:0/4037437939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 msgr2=0x557d39b00670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee077fe640 1 -- 172.21.15.5:0/1415716145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fee000cf640 con 0x7fee10100a80 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee07fff640 1 --2- 172.21.15.5:0/4037437939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39b00670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee07fff640 1 -- 172.21.15.5:0/4037437939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 msgr2=0x557d39af5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee07fff640 1 --2- 172.21.15.5:0/4037437939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39af5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee07fff640 1 -- 172.21.15.5:0/4037437939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d39a10130 con 0x557d39b00bb0 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee0ed9d640 1 --2- 172.21.15.5:0/4037437939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39af5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee07fff640 1 --2- 172.21.15.5:0/4037437939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39b02fa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fedf8031440 tx=0x7fedf8032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506ab52b598efa29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee04ff9640 1 -- 172.21.15.5:0/4037437939 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8035050 con 0x557d39b00bb0 2024-09-19T06:03:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee0ed9d640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fedfc0a3350 0x7fedfc0a5810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fee08007310 tx=0x7fee08004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee04ff9640 1 -- 172.21.15.5:0/4037437939 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedf8004030 con 0x557d39b00bb0 2024-09-19T06:03:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.909+0000 7fee04ff9640 1 -- 172.21.15.5:0/4037437939 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf80053d0 con 0x557d39b00bb0 2024-09-19T06:03:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.910+0000 7fee1731b740 1 -- 172.21.15.5:0/4037437939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 msgr2=0x557d39b02fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.910+0000 7fee1731b740 1 --2- 172.21.15.5:0/4037437939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39b02fa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fedf8031440 tx=0x7fedf8032df0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.910+0000 7fee1731b740 1 -- 172.21.15.5:0/4037437939 shutdown_connections 2024-09-19T06:03:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.910+0000 7fee1731b740 1 --2- 172.21.15.5:0/4037437939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39b02fa0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.910+0000 7fee1731b740 1 --2- 172.21.15.5:0/4037437939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39b00670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.910+0000 7fee1731b740 1 --2- 172.21.15.5:0/4037437939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39af5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.910+0000 7fee1731b740 1 -- 172.21.15.5:0/4037437939 >> 172.21.15.5:0/4037437939 conn(0x557d39af1f60 msgr2=0x557d39af2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.910+0000 7fee1731b740 1 -- 172.21.15.5:0/4037437939 shutdown_connections 2024-09-19T06:03:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.910+0000 7fee1731b740 1 -- 172.21.15.5:0/4037437939 wait complete. 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.910+0000 7fee1731b740 1 -- start start 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee1731b740 1 Processor -- start 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee1731b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39b003e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee1731b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39afd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee1731b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39afdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee1731b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d39b03720 con 0x557d39b00bb0 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee1731b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557d39a16f20 con 0x557d39af59d0 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee1731b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557d39b038a0 con 0x557d39af62f0 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee0ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39afdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee0ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39afdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51950/0 (socket says 172.21.15.5:51950) 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee0ed9d640 1 -- 172.21.15.5:0/3689644529 learned_addr learned my addr 172.21.15.5:0/3689644529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee0e59c640 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39b003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee07fff640 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39afd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee1731b740 10 client.0 _ll_register_callbacks cb 0x557d399e0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee0ed9d640 1 -- 172.21.15.5:0/3689644529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 msgr2=0x557d39afd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee0ed9d640 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39afd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee0ed9d640 1 -- 172.21.15.5:0/3689644529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 msgr2=0x557d39b003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee0ed9d640 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39b003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee0ed9d640 1 -- 172.21.15.5:0/3689644529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d397cd2b0 con 0x557d39b00bb0 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.911+0000 7fee07fff640 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39afd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.912+0000 7fee0ed9d640 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39afdc70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fee0800f6c0 tx=0x7fee08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b6c6b3b6844a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.912+0000 7fedf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.912+0000 7fedf5ffb640 1 -- 172.21.15.5:0/3689644529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08019070 con 0x557d39b00bb0 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.912+0000 7fedf5ffb640 1 -- 172.21.15.5:0/3689644529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fee080208d0 con 0x557d39b00bb0 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.913+0000 7fee1731b740 1 -- 172.21.15.5:0/3689644529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedf8004030 con 0x557d39b00bb0 2024-09-19T06:03:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.913+0000 7fedf5ffb640 1 -- 172.21.15.5:0/3689644529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee08020ad0 con 0x557d39b00bb0 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.913+0000 7fee1731b740 10 client.15141 fetch_fsmap learned FSMap version 27 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.913+0000 7fee1731b740 1 -- 172.21.15.5:0/3689644529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d39b0f040 con 0x557d39b00bb0 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.913+0000 7fedf5ffb640 1 -- 172.21.15.5:0/3689644529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedf8004030 con 0x557d39b00bb0 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.913+0000 7fedf5ffb640 1 -- 172.21.15.5:0/3689644529 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fee08028c90 con 0x557d39b00bb0 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.913+0000 7fee1731b740 10 client.15141 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.913+0000 7fee1731b740 10 client.15141 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.913+0000 7fee1731b740 20 client.15141 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.913+0000 7fee1731b740 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557d39b116c0 0x557d39b12b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.913+0000 7fee1731b740 4 client.15141 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.913+0000 7fee1731b740 1 -- 172.21.15.5:0/3689644529 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d39a69600 con 0x557d39b116c0 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.914+0000 7fee0e59c640 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557d39b116c0 0x557d39b12b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.914+0000 7fedf5ffb640 10 client.15141 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:03:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.914+0000 7fee0e59c640 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557d39b116c0 0x557d39b12b30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b254207098edb621 server_cookie=41f57d0b2ff4106f in_seq=0 out_seq=0 2024-09-19T06:03:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fedf5ffb640 1 -- 172.21.15.5:0/3689644529 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d39a69600 con 0x557d39b116c0 2024-09-19T06:03:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fedf5ffb640 10 client.15141 handle_command_reply: tid=0 2024-09-19T06:03:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fee1731b740 1 client.15141 shutdown 2024-09-19T06:03:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fee1731b740 1 -- 172.21.15.5:0/3689644529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 msgr2=0x557d39afdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fee1731b740 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39afdc70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fee0800f6c0 tx=0x7fee08004a40 comp rx=0 tx=0).stop 2024-09-19T06:03:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fee1731b740 1 -- 172.21.15.5:0/3689644529 shutdown_connections 2024-09-19T06:03:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fee1731b740 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557d39b116c0 0x557d39b12b30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fedf5ffb640 0 client.15141 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:03:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fee1731b740 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39afdc70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fee1731b740 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39afd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fee1731b740 1 --2- 172.21.15.5:0/3689644529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39b003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fee1731b740 1 -- 172.21.15.5:0/3689644529 >> 172.21.15.5:0/3689644529 conn(0x557d39af1f60 msgr2=0x557d39af7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fee1731b740 1 -- 172.21.15.5:0/3689644529 shutdown_connections 2024-09-19T06:03:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fee1731b740 1 -- 172.21.15.5:0/3689644529 wait complete. 2024-09-19T06:03:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.915+0000 7fee1731b740 20 client.15141 trim_cache size 0 max 16384 2024-09-19T06:03:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.933+0000 7fee1731b740 1 Processor -- start 2024-09-19T06:03:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.933+0000 7fee1731b740 1 -- start start 2024-09-19T06:03:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.933+0000 7fee1731b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39af59d0 0x557d39b12b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.933+0000 7fee1731b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39afc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.933+0000 7fee1731b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39b00bb0 0x557d39afc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee1731b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d39b10e70 con 0x557d39af59d0 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee1731b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557d39b11540 con 0x557d39b00bb0 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee1731b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557d39b11c10 con 0x557d39af62f0 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee0e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39af59d0 0x557d39b12b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee0e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39af59d0 0x557d39b12b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51956/0 (socket says 172.21.15.5:51956) 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee0e59c640 1 -- 172.21.15.5:0/2357433106 learned_addr learned my addr 172.21.15.5:0/2357433106 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee07fff640 1 --2- 172.21.15.5:0/2357433106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39afc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee0ed9d640 1 --2- 172.21.15.5:0/2357433106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39b00bb0 0x557d39afc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee0e59c640 1 -- 172.21.15.5:0/2357433106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 msgr2=0x557d39afc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee0e59c640 1 --2- 172.21.15.5:0/2357433106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39afc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee0e59c640 1 -- 172.21.15.5:0/2357433106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39b00bb0 msgr2=0x557d39afc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee0e59c640 1 --2- 172.21.15.5:0/2357433106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39b00bb0 0x557d39afc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee0e59c640 1 -- 172.21.15.5:0/2357433106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d39b00000 con 0x557d39af59d0 2024-09-19T06:03:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee07fff640 1 --2- 172.21.15.5:0/2357433106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39afc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fee0e59c640 1 --2- 172.21.15.5:0/2357433106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39af59d0 0x557d39b12b30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fee000cd010 tx=0x7fee0001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e26a536ddbc167 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fedf7fff640 1 -- 172.21.15.5:0/2357433106 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00020040 con 0x557d39af59d0 2024-09-19T06:03:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fedf7fff640 1 -- 172.21.15.5:0/2357433106 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee000a4760 con 0x557d39af59d0 2024-09-19T06:03:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.934+0000 7fedf7fff640 1 -- 172.21.15.5:0/2357433106 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee000a42d0 con 0x557d39af59d0 2024-09-19T06:03:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.935+0000 7fee1731b740 1 -- 172.21.15.5:0/2357433106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39af59d0 msgr2=0x557d39b12b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.935+0000 7fee1731b740 1 --2- 172.21.15.5:0/2357433106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39af59d0 0x557d39b12b30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fee000cd010 tx=0x7fee0001d040 comp rx=0 tx=0).stop 2024-09-19T06:03:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.935+0000 7fee1731b740 1 -- 172.21.15.5:0/2357433106 shutdown_connections 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.935+0000 7fee1731b740 1 --2- 172.21.15.5:0/2357433106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39b00bb0 0x557d39afc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.935+0000 7fee1731b740 1 --2- 172.21.15.5:0/2357433106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39afc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.935+0000 7fee1731b740 1 --2- 172.21.15.5:0/2357433106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39af59d0 0x557d39b12b30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.935+0000 7fee1731b740 1 -- 172.21.15.5:0/2357433106 >> 172.21.15.5:0/2357433106 conn(0x557d39af1f60 msgr2=0x557d39afd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.935+0000 7fee1731b740 1 -- 172.21.15.5:0/2357433106 shutdown_connections 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.935+0000 7fee1731b740 1 -- 172.21.15.5:0/2357433106 wait complete. 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.935+0000 7fee1731b740 1 -- start start 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.935+0000 7fee1731b740 1 Processor -- start 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee1731b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39b1b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee1731b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39b14570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee1731b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39b14ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee1731b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d39b129b0 con 0x557d39b00bb0 2024-09-19T06:03:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee1731b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557d39b122e0 con 0x557d39af59d0 2024-09-19T06:03:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee1731b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557d39b13080 con 0x557d39af62f0 2024-09-19T06:03:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee1731b740 10 client.0 _ll_register_callbacks cb 0x7fee08011f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee0ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39b14ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee0ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39b14ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51970/0 (socket says 172.21.15.5:51970) 2024-09-19T06:03:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee0ed9d640 1 -- 172.21.15.5:0/72785540 learned_addr learned my addr 172.21.15.5:0/72785540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee0e59c640 1 --2- 172.21.15.5:0/72785540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39b1b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee0ed9d640 1 -- 172.21.15.5:0/72785540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 msgr2=0x557d39b14570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee0ed9d640 1 --2- 172.21.15.5:0/72785540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39b14570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee0ed9d640 1 -- 172.21.15.5:0/72785540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 msgr2=0x557d39b1b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee0ed9d640 1 --2- 172.21.15.5:0/72785540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39b1b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee0ed9d640 1 -- 172.21.15.5:0/72785540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d39b15020 con 0x557d39b00bb0 2024-09-19T06:03:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fee0ed9d640 1 --2- 172.21.15.5:0/72785540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39b14ab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fee08011010 tx=0x7fee08020a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce2720ef9d67ade server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fedf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.936+0000 7fedf5ffb640 1 -- 172.21.15.5:0/72785540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08019040 con 0x557d39b00bb0 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fedf5ffb640 1 -- 172.21.15.5:0/72785540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fee08031700 con 0x557d39b00bb0 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fee1731b740 1 -- 172.21.15.5:0/72785540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee000a4760 con 0x557d39b00bb0 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fedf5ffb640 1 -- 172.21.15.5:0/72785540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee08031920 con 0x557d39b00bb0 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fee1731b740 10 client.15147 fetch_fsmap learned FSMap version 27 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fee1731b740 1 -- 172.21.15.5:0/72785540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d39b19b30 con 0x557d39b00bb0 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fedf5ffb640 1 -- 172.21.15.5:0/72785540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee000a4760 con 0x557d39b00bb0 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fedf5ffb640 1 -- 172.21.15.5:0/72785540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fee08032770 con 0x557d39b00bb0 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fee1731b740 10 client.15147 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fee1731b740 10 client.15147 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fee1731b740 20 client.15147 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fee1731b740 1 --2- 172.21.15.5:0/72785540 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557d39b116c0 0x557d39b7f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fee1731b740 4 client.15147 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.937+0000 7fee1731b740 1 -- 172.21.15.5:0/72785540 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "status"}) -- 0x557d39a63b10 con 0x557d39b116c0 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fee0e59c640 1 --2- 172.21.15.5:0/72785540 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557d39b116c0 0x557d39b7f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fedf5ffb640 10 client.15147 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fee0e59c640 1 --2- 172.21.15.5:0/72785540 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557d39b116c0 0x557d39b7f4c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=301f20a4d6b4935b server_cookie=4f3d7fd6e0335107 in_seq=0 out_seq=0 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fedf5ffb640 1 -- 172.21.15.5:0/72785540 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557d39a63b10 con 0x557d39b116c0 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fedf5ffb640 10 client.15147 handle_command_reply: tid=0 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fee1731b740 1 client.15147 shutdown 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fee1731b740 1 -- 172.21.15.5:0/72785540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 msgr2=0x557d39b14ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fee1731b740 1 --2- 172.21.15.5:0/72785540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39b14ab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fee08011010 tx=0x7fee08020a00 comp rx=0 tx=0).stop 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fee1731b740 1 -- 172.21.15.5:0/72785540 shutdown_connections 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fee1731b740 1 --2- 172.21.15.5:0/72785540 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557d39b116c0 0x557d39b7f4c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fedf5ffb640 0 client.15147 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:03:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fee1731b740 1 --2- 172.21.15.5:0/72785540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d39b00bb0 0x557d39b14ab0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fee1731b740 1 --2- 172.21.15.5:0/72785540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d39af62f0 0x557d39b14570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fee1731b740 1 --2- 172.21.15.5:0/72785540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d39af59d0 0x557d39b1b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.938+0000 7fee1731b740 1 -- 172.21.15.5:0/72785540 >> 172.21.15.5:0/72785540 conn(0x557d39af1f60 msgr2=0x557d39af8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee1731b740 1 -- 172.21.15.5:0/72785540 shutdown_connections 2024-09-19T06:03:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee1731b740 1 -- 172.21.15.5:0/72785540 wait complete. 2024-09-19T06:03:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee1731b740 20 client.15147 trim_cache size 0 max 16384 2024-09-19T06:03:07.946 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:03:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T06:03:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-09-19T06:03:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24437, 2024-09-19T06:03:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-19T06:03:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-19T06:03:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-19T06:03:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 87.101032490999998, 2024-09-19T06:03:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-19T06:03:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-19T06:03:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:03:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 119.91869821900001 2024-09-19T06:03:07.947 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:03:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee14f42640 1 -- 172.21.15.5:0/1415716145 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fedfc0a3350 msgr2=0x7fedfc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee14f42640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fedfc0a3350 0x7fedfc0a5810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fee08007310 tx=0x7fee08004ab0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee14f42640 1 -- 172.21.15.5:0/1415716145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee10100a80 msgr2=0x7fee101991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee14f42640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee10100a80 0x7fee101991c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fee0000d950 tx=0x7fee0000de10 comp rx=0 tx=0).stop 2024-09-19T06:03:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee14f42640 1 -- 172.21.15.5:0/1415716145 shutdown_connections 2024-09-19T06:03:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee14f42640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fedfc0a3350 0x7fedfc0a5810 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee14f42640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee10106a80 0x7fee101a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee14f42640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee10101420 0x7fee10199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee14f42640 1 --2- 172.21.15.5:0/1415716145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee10100a80 0x7fee101991c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee14f42640 1 -- 172.21.15.5:0/1415716145 >> 172.21.15.5:0/1415716145 conn(0x7fee100fc7a0 msgr2=0x7fee100fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:07.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee14f42640 1 -- 172.21.15.5:0/1415716145 shutdown_connections 2024-09-19T06:03:07.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:07.939+0000 7fee14f42640 1 -- 172.21.15.5:0/1415716145 wait complete. 2024-09-19T06:03:08.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:08 smithi005 ceph-mon[30955]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:03:08.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:08 smithi049 ceph-mon[34640]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:03:08.672 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T06:03:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:08 smithi157 ceph-mon[35346]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:03:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:09.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.414+0000 7f37488d3640 1 -- 172.21.15.5:0/2484295242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3744101de0 msgr2=0x7f37441021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:09.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.414+0000 7f37488d3640 1 --2- 172.21.15.5:0/2484295242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3744101de0 0x7f37441021c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3734009900 tx=0x7f373402a410 comp rx=0 tx=0).stop 2024-09-19T06:03:09.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.414+0000 7f37488d3640 1 -- 172.21.15.5:0/2484295242 shutdown_connections 2024-09-19T06:03:09.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.414+0000 7f37488d3640 1 --2- 172.21.15.5:0/2484295242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3744106f10 0x7f374410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.414+0000 7f37488d3640 1 --2- 172.21.15.5:0/2484295242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3744102790 0x7f37441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.414+0000 7f37488d3640 1 --2- 172.21.15.5:0/2484295242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3744101de0 0x7f37441021c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.414+0000 7f37488d3640 1 -- 172.21.15.5:0/2484295242 >> 172.21.15.5:0/2484295242 conn(0x7f37440fd8c0 msgr2=0x7f37440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:09.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.414+0000 7f37488d3640 1 -- 172.21.15.5:0/2484295242 shutdown_connections 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.414+0000 7f37488d3640 1 -- 172.21.15.5:0/2484295242 wait complete. 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.415+0000 7f37488d3640 1 Processor -- start 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.415+0000 7f37488d3640 1 -- start start 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.415+0000 7f37488d3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3744101de0 0x7f3744198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f37488d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3744102790 0x7f3744198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f37488d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3744106f10 0x7f374419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f37488d3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3744111de0 con 0x7f3744106f10 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f37488d3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3744111c60 con 0x7f3744101de0 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f37488d3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3744111f60 con 0x7f3744102790 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f3742d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3744106f10 0x7f374419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f3742d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3744106f10 0x7f374419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51992/0 (socket says 172.21.15.5:51992) 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f3742d9d640 1 -- 172.21.15.5:0/2679603042 learned_addr learned my addr 172.21.15.5:0/2679603042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:09.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f374259c640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3744101de0 0x7f3744198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f3742d9d640 1 -- 172.21.15.5:0/2679603042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3744102790 msgr2=0x7f3744198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f3742d9d640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3744102790 0x7f3744198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f3742d9d640 1 -- 172.21.15.5:0/2679603042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3744101de0 msgr2=0x7f3744198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f3742d9d640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3744101de0 0x7f3744198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f3742d9d640 1 -- 172.21.15.5:0/2679603042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374419d2f0 con 0x7f3744106f10 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f3742d9d640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3744106f10 0x7f374419cc10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f373c00eaa0 tx=0x7f373c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f37337fe640 1 -- 172.21.15.5:0/2679603042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c00cdb0 con 0x7f3744106f10 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f37337fe640 1 -- 172.21.15.5:0/2679603042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f373c005020 con 0x7f3744106f10 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f37337fe640 1 -- 172.21.15.5:0/2679603042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c0116d0 con 0x7f3744106f10 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.416+0000 7f37488d3640 1 -- 172.21.15.5:0/2679603042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374419d5e0 con 0x7f3744106f10 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.417+0000 7f37488d3640 1 -- 172.21.15.5:0/2679603042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37441a4ec0 con 0x7f3744106f10 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.419+0000 7f37337fe640 1 -- 172.21.15.5:0/2679603042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f373c0051d0 con 0x7f3744106f10 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.419+0000 7f374acac740 1 Processor -- start 2024-09-19T06:03:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.419+0000 7f374acac740 1 -- start start 2024-09-19T06:03:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.419+0000 7f374acac740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0cb8b0 0x563d8a0cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.419+0000 7f374acac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0daa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.419+0000 7f374acac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0dafc0 0x563d8a0dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.419+0000 7f374acac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563d8a032090 con 0x563d8a0dafc0 2024-09-19T06:03:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.419+0000 7f374acac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563d89fd9a70 con 0x563d8a0cc1d0 2024-09-19T06:03:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.419+0000 7f374acac740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563d8a02e050 con 0x563d8a0cb8b0 2024-09-19T06:03:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0dafc0 0x563d8a0dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f37337fe640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f372c0a32c0 0x7f372c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0dafc0 0x563d8a0dd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51996/0 (socket says 172.21.15.5:51996) 2024-09-19T06:03:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3741d9b640 1 -- 172.21.15.5:0/1913761871 learned_addr learned my addr 172.21.15.5:0/1913761871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3741d9b640 1 --2- 172.21.15.5:0/1913761871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0cb8b0 0x563d8a0cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f374259c640 1 --2- 172.21.15.5:0/1913761871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0daa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3742d9d640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f372c0a32c0 0x7f372c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3741d9b640 1 -- 172.21.15.5:0/1913761871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0cb8b0 msgr2=0x563d8a0cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3741d9b640 1 --2- 172.21.15.5:0/1913761871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0cb8b0 0x563d8a0cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3741d9b640 1 -- 172.21.15.5:0/1913761871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 msgr2=0x563d8a0daa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3741d9b640 1 --2- 172.21.15.5:0/1913761871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0daa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f37337fe640 1 -- 172.21.15.5:0/2679603042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f373c0cfa60 con 0x7f3744106f10 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3741d9b640 1 -- 172.21.15.5:0/1913761871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d89fe6130 con 0x563d8a0dafc0 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f374259c640 1 --2- 172.21.15.5:0/1913761871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0daa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3741d9b640 1 --2- 172.21.15.5:0/1913761871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0dafc0 0x563d8a0dd3b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f37380127b0 tx=0x7f373800d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c84ca8e120a8c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3742d9d640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f372c0a32c0 0x7f372c0a5780 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f373c0a47b0 tx=0x7f373c06eef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3730ff9640 1 -- 172.21.15.5:0/1913761871 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373800a6b0 con 0x563d8a0dafc0 2024-09-19T06:03:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3730ff9640 1 -- 172.21.15.5:0/1913761871 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f373800ad00 con 0x563d8a0dafc0 2024-09-19T06:03:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.420+0000 7f3730ff9640 1 -- 172.21.15.5:0/1913761871 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373800fc50 con 0x563d8a0dafc0 2024-09-19T06:03:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.421+0000 7f374acac740 1 -- 172.21.15.5:0/1913761871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0dafc0 msgr2=0x563d8a0dd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.421+0000 7f374acac740 1 --2- 172.21.15.5:0/1913761871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0dafc0 0x563d8a0dd3b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f37380127b0 tx=0x7f373800d3c0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.421+0000 7f374acac740 1 -- 172.21.15.5:0/1913761871 shutdown_connections 2024-09-19T06:03:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.421+0000 7f374acac740 1 --2- 172.21.15.5:0/1913761871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0dafc0 0x563d8a0dd3b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.421+0000 7f374acac740 1 --2- 172.21.15.5:0/1913761871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0daa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.421+0000 7f374acac740 1 --2- 172.21.15.5:0/1913761871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0cb8b0 0x563d8a0cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.421+0000 7f374acac740 1 -- 172.21.15.5:0/1913761871 >> 172.21.15.5:0/1913761871 conn(0x563d8a0c7f60 msgr2=0x563d8a0c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.421+0000 7f374acac740 1 -- 172.21.15.5:0/1913761871 shutdown_connections 2024-09-19T06:03:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.421+0000 7f374acac740 1 -- 172.21.15.5:0/1913761871 wait complete. 2024-09-19T06:03:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f374acac740 1 -- start start 2024-09-19T06:03:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f374acac740 1 Processor -- start 2024-09-19T06:03:09.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f374acac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 0x563d8a0da880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f374acac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0d7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f374acac740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 0x563d8a0d8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f374acac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563d8a0ddb90 con 0x563d8a0cb8b0 2024-09-19T06:03:09.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f374acac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563d89fecf20 con 0x563d8a0cc1d0 2024-09-19T06:03:09.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f374acac740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563d8a0ddd10 con 0x563d8a0dafc0 2024-09-19T06:03:09.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f374acac740 10 client.0 _ll_register_callbacks cb 0x563d89fb6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:09.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 0x563d8a0da880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 0x563d8a0da880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52004/0 (socket says 172.21.15.5:52004) 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f3741d9b640 1 -- 172.21.15.5:0/3757541896 learned_addr learned my addr 172.21.15.5:0/3757541896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f3741d9b640 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 0x563d8a0d8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f374259c640 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0d7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f3741d9b640 1 -- 172.21.15.5:0/3757541896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 msgr2=0x563d8a0d8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f3741d9b640 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 0x563d8a0d8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f3741d9b640 1 -- 172.21.15.5:0/3757541896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 msgr2=0x563d8a0d7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f3741d9b640 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0d7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.422+0000 7f3741d9b640 1 -- 172.21.15.5:0/3757541896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d89da32b0 con 0x563d8a0cb8b0 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f3741d9b640 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 0x563d8a0da880 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3738005d00 tx=0x7f373800f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c5e0565702894c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f3729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f3729ffb640 1 -- 172.21.15.5:0/3757541896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373800a6b0 con 0x563d8a0cb8b0 2024-09-19T06:03:09.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f3729ffb640 1 -- 172.21.15.5:0/3757541896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f373800a850 con 0x563d8a0cb8b0 2024-09-19T06:03:09.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f374acac740 1 -- 172.21.15.5:0/3757541896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f373800ad00 con 0x563d8a0cb8b0 2024-09-19T06:03:09.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f3729ffb640 1 -- 172.21.15.5:0/3757541896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3738033050 con 0x563d8a0cb8b0 2024-09-19T06:03:09.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f374acac740 10 client.15159 fetch_fsmap learned FSMap version 27 2024-09-19T06:03:09.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f374acac740 1 -- 172.21.15.5:0/3757541896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d8a0e9520 con 0x563d8a0cb8b0 2024-09-19T06:03:09.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f3729ffb640 1 -- 172.21.15.5:0/3757541896 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f373800ad00 con 0x563d8a0cb8b0 2024-09-19T06:03:09.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f3729ffb640 1 -- 172.21.15.5:0/3757541896 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3738005320 con 0x563d8a0cb8b0 2024-09-19T06:03:09.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f374acac740 10 client.15159 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:03:09.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f374acac740 10 client.15159 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T06:03:09.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f374acac740 20 client.15159 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:09.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f374acac740 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563d8a0eb820 0x563d8a0ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f374acac740 4 client.15159 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:03:09.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.423+0000 7f374acac740 1 -- 172.21.15.5:0/3757541896 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d8a03f600 con 0x563d8a0eb820 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.424+0000 7f374259c640 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563d8a0eb820 0x563d8a0ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.424+0000 7f374259c640 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563d8a0eb820 0x563d8a0ecc90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e6a64f76c6c05a7a server_cookie=c1d069d72c6af471 in_seq=0 out_seq=0 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.424+0000 7f3729ffb640 10 client.15159 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f3729ffb640 1 -- 172.21.15.5:0/3757541896 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d8a03f600 con 0x563d8a0eb820 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f3729ffb640 10 client.15159 handle_command_reply: tid=0 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f374acac740 1 client.15159 shutdown 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f374acac740 1 -- 172.21.15.5:0/3757541896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 msgr2=0x563d8a0da880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f374acac740 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 0x563d8a0da880 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3738005d00 tx=0x7f373800f830 comp rx=0 tx=0).stop 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f374acac740 1 -- 172.21.15.5:0/3757541896 shutdown_connections 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f374acac740 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563d8a0eb820 0x563d8a0ecc90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f374acac740 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 0x563d8a0d8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f374acac740 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0d7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f374acac740 1 --2- 172.21.15.5:0/3757541896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 0x563d8a0da880 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f374acac740 1 -- 172.21.15.5:0/3757541896 >> 172.21.15.5:0/3757541896 conn(0x563d8a0c7f60 msgr2=0x563d8a0cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:09.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f3729ffb640 0 client.15159 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:03:09.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f374acac740 1 -- 172.21.15.5:0/3757541896 shutdown_connections 2024-09-19T06:03:09.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f374acac740 1 -- 172.21.15.5:0/3757541896 wait complete. 2024-09-19T06:03:09.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.425+0000 7f374acac740 20 client.15159 trim_cache size 0 max 16384 2024-09-19T06:03:09.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.443+0000 7f374acac740 1 Processor -- start 2024-09-19T06:03:09.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.443+0000 7f374acac740 1 -- start start 2024-09-19T06:03:09.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.443+0000 7f374acac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cb8b0 0x563d8a0eb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.443+0000 7f374acac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cc1d0 0x563d8a0ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.443+0000 7f374acac740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 0x563d8a0eb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.443+0000 7f374acac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563d8a0eafd0 con 0x563d8a0cc1d0 2024-09-19T06:03:09.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.443+0000 7f374acac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563d8a0eb6a0 con 0x563d8a0cb8b0 2024-09-19T06:03:09.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.443+0000 7f374acac740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563d8a0ebd70 con 0x563d8a0dafc0 2024-09-19T06:03:09.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.443+0000 7f374259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cc1d0 0x563d8a0ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.443+0000 7f374259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cc1d0 0x563d8a0ec5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52018/0 (socket says 172.21.15.5:52018) 2024-09-19T06:03:09.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.443+0000 7f374259c640 1 -- 172.21.15.5:0/4125642868 learned_addr learned my addr 172.21.15.5:0/4125642868 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:09.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.444+0000 7f3741d9b640 1 --2- 172.21.15.5:0/4125642868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cb8b0 0x563d8a0eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.444+0000 7f3741d9b640 1 --2- 172.21.15.5:0/4125642868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 0x563d8a0eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.444+0000 7f374259c640 1 -- 172.21.15.5:0/4125642868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 msgr2=0x563d8a0eb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.444+0000 7f374259c640 1 --2- 172.21.15.5:0/4125642868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 0x563d8a0eb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.444+0000 7f374259c640 1 -- 172.21.15.5:0/4125642868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cb8b0 msgr2=0x563d8a0eb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.444+0000 7f374259c640 1 --2- 172.21.15.5:0/4125642868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cb8b0 0x563d8a0eb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.444+0000 7f374259c640 1 -- 172.21.15.5:0/4125642868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d8a0d6120 con 0x563d8a0cc1d0 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.444+0000 7f3741d9b640 1 --2- 172.21.15.5:0/4125642868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cb8b0 0x563d8a0eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.444+0000 7f374259c640 1 --2- 172.21.15.5:0/4125642868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cc1d0 0x563d8a0ec5c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3734031440 tx=0x7f3734009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1066c4575459f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.444+0000 7f372bfff640 1 -- 172.21.15.5:0/4125642868 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734049070 con 0x563d8a0cc1d0 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.444+0000 7f372bfff640 1 -- 172.21.15.5:0/4125642868 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37340054f0 con 0x563d8a0cc1d0 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.444+0000 7f372bfff640 1 -- 172.21.15.5:0/4125642868 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734035040 con 0x563d8a0cc1d0 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 -- 172.21.15.5:0/4125642868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cc1d0 msgr2=0x563d8a0ec5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 --2- 172.21.15.5:0/4125642868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cc1d0 0x563d8a0ec5c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3734031440 tx=0x7f3734009580 comp rx=0 tx=0).stop 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 -- 172.21.15.5:0/4125642868 shutdown_connections 2024-09-19T06:03:09.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 --2- 172.21.15.5:0/4125642868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 0x563d8a0eb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 --2- 172.21.15.5:0/4125642868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cc1d0 0x563d8a0ec5c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 --2- 172.21.15.5:0/4125642868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cb8b0 0x563d8a0eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 -- 172.21.15.5:0/4125642868 >> 172.21.15.5:0/4125642868 conn(0x563d8a0c7f60 msgr2=0x563d8a0c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 -- 172.21.15.5:0/4125642868 shutdown_connections 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 -- 172.21.15.5:0/4125642868 wait complete. 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 -- start start 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 Processor -- start 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 0x563d8a0ee070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0ee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 0x563d8a153f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563d8a0ecb10 con 0x563d8a0cb8b0 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563d8a0ec440 con 0x563d8a0cc1d0 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563d8a0ed1e0 con 0x563d8a0dafc0 2024-09-19T06:03:09.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f374acac740 10 client.0 _ll_register_callbacks cb 0x7f373800dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:09.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 0x563d8a0ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 0x563d8a0ee070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52020/0 (socket says 172.21.15.5:52020) 2024-09-19T06:03:09.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.445+0000 7f3741d9b640 1 -- 172.21.15.5:0/3252062591 learned_addr learned my addr 172.21.15.5:0/3252062591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:09.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f374259c640 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0ee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f3741d9b640 1 -- 172.21.15.5:0/3252062591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 msgr2=0x563d8a153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:09.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f3741d9b640 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 0x563d8a153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f3741d9b640 1 -- 172.21.15.5:0/3252062591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 msgr2=0x563d8a0ee5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:09.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f3741d9b640 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0ee5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f3741d9b640 1 -- 172.21.15.5:0/3252062591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d8a0eee60 con 0x563d8a0cb8b0 2024-09-19T06:03:09.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f374259c640 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0ee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:09.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f3741d9b640 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 0x563d8a0ee070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f373800dbc0 tx=0x7f3738012770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde93ce67f9b512 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:09.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f3729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f3729ffb640 1 -- 172.21.15.5:0/3252062591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373803b070 con 0x563d8a0cb8b0 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f3729ffb640 1 -- 172.21.15.5:0/3252062591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f373800faa0 con 0x563d8a0cb8b0 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f374acac740 1 -- 172.21.15.5:0/3252062591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37340054f0 con 0x563d8a0cb8b0 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f3729ffb640 1 -- 172.21.15.5:0/3252062591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3738047050 con 0x563d8a0cb8b0 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f374acac740 10 client.15165 fetch_fsmap learned FSMap version 27 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f374acac740 1 -- 172.21.15.5:0/3252062591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d8a155540 con 0x563d8a0cb8b0 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.446+0000 7f3729ffb640 1 -- 172.21.15.5:0/3252062591 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37340054f0 con 0x563d8a0cb8b0 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.447+0000 7f3729ffb640 1 -- 172.21.15.5:0/3252062591 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f373802c430 con 0x563d8a0cb8b0 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.447+0000 7f374acac740 10 client.15165 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.447+0000 7f374acac740 10 client.15165 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.447+0000 7f374acac740 20 client.15165 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.447+0000 7f374acac740 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563d8a15a000 0x563d8a158d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:09.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.447+0000 7f374acac740 4 client.15165 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.447+0000 7f374acac740 1 -- 172.21.15.5:0/3252062591 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "status"}) -- 0x563d8a039b10 con 0x563d8a15a000 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.447+0000 7f374259c640 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563d8a15a000 0x563d8a158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.447+0000 7f3729ffb640 10 client.15165 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.447+0000 7f374259c640 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563d8a15a000 0x563d8a158d10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5d8083be4a91f713 server_cookie=f7ff8917730a01c in_seq=0 out_seq=0 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f3729ffb640 1 -- 172.21.15.5:0/3252062591 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563d8a039b10 con 0x563d8a15a000 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f3729ffb640 10 client.15165 handle_command_reply: tid=0 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f374acac740 1 client.15165 shutdown 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f374acac740 1 -- 172.21.15.5:0/3252062591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 msgr2=0x563d8a0ee070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f374acac740 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 0x563d8a0ee070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f373800dbc0 tx=0x7f3738012770 comp rx=0 tx=0).stop 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f374acac740 1 -- 172.21.15.5:0/3252062591 shutdown_connections 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f374acac740 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563d8a15a000 0x563d8a158d10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f374acac740 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d8a0dafc0 0x563d8a153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f374acac740 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563d8a0cc1d0 0x563d8a0ee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f374acac740 1 --2- 172.21.15.5:0/3252062591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d8a0cb8b0 0x563d8a0ee070 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f374acac740 1 -- 172.21.15.5:0/3252062591 >> 172.21.15.5:0/3252062591 conn(0x563d8a0c7f60 msgr2=0x563d8a0d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f3729ffb640 0 client.15165 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:03:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f374acac740 1 -- 172.21.15.5:0/3252062591 shutdown_connections 2024-09-19T06:03:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f374acac740 1 -- 172.21.15.5:0/3252062591 wait complete. 2024-09-19T06:03:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.448+0000 7f374acac740 20 client.15165 trim_cache size 0 max 16384 2024-09-19T06:03:09.460 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:03:09.460 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T06:03:09.460 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-09-19T06:03:09.460 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24442, 2024-09-19T06:03:09.460 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-19T06:03:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-19T06:03:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-19T06:03:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 90.621343289999999, 2024-09-19T06:03:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-19T06:03:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-19T06:03:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:03:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 126.520649643 2024-09-19T06:03:09.461 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:03:09.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.449+0000 7f37488d3640 1 -- 172.21.15.5:0/2679603042 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f372c0a32c0 msgr2=0x7f372c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:09.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.449+0000 7f37488d3640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f372c0a32c0 0x7f372c0a5780 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f373c0a47b0 tx=0x7f373c06eef0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.449+0000 7f37488d3640 1 -- 172.21.15.5:0/2679603042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3744106f10 msgr2=0x7f374419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:09.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.449+0000 7f37488d3640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3744106f10 0x7f374419cc10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f373c00eaa0 tx=0x7f373c00ef60 comp rx=0 tx=0).stop 2024-09-19T06:03:09.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.449+0000 7f37488d3640 1 -- 172.21.15.5:0/2679603042 shutdown_connections 2024-09-19T06:03:09.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.449+0000 7f37488d3640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f372c0a32c0 0x7f372c0a5780 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.449+0000 7f37488d3640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3744106f10 0x7f374419cc10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.449+0000 7f37488d3640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3744102790 0x7f3744198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.449+0000 7f37488d3640 1 --2- 172.21.15.5:0/2679603042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3744101de0 0x7f3744198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:09.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.449+0000 7f37488d3640 1 -- 172.21.15.5:0/2679603042 >> 172.21.15.5:0/2679603042 conn(0x7f37440fd8c0 msgr2=0x7f37440fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:09.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.449+0000 7f37488d3640 1 -- 172.21.15.5:0/2679603042 shutdown_connections 2024-09-19T06:03:09.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:09.449+0000 7f37488d3640 1 -- 172.21.15.5:0/2679603042 wait complete. 2024-09-19T06:03:10.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T06:03:10.347 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:10.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:10 smithi005 ceph-mon[30955]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:03:10.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:10 smithi005 ceph-mon[30955]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T06:03:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:10 smithi049 ceph-mon[34640]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:03:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:10 smithi049 ceph-mon[34640]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T06:03:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:10 smithi157 ceph-mon[35346]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:03:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:10 smithi157 ceph-mon[35346]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T06:03:10.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.823+0000 7fd328de9640 1 -- 172.21.15.5:0/358986862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd324101de0 msgr2=0x7fd3241021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:10.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.823+0000 7fd328de9640 1 --2- 172.21.15.5:0/358986862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd324101de0 0x7fd3241021c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd314009900 tx=0x7fd31402a410 comp rx=0 tx=0).stop 2024-09-19T06:03:10.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.823+0000 7fd328de9640 1 -- 172.21.15.5:0/358986862 shutdown_connections 2024-09-19T06:03:10.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.823+0000 7fd328de9640 1 --2- 172.21.15.5:0/358986862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd324106f10 0x7fd32410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.823+0000 7fd328de9640 1 --2- 172.21.15.5:0/358986862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd324102790 0x7fd3241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.823+0000 7fd328de9640 1 --2- 172.21.15.5:0/358986862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd324101de0 0x7fd3241021c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.823+0000 7fd328de9640 1 -- 172.21.15.5:0/358986862 >> 172.21.15.5:0/358986862 conn(0x7fd3240fd8c0 msgr2=0x7fd3240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.823+0000 7fd328de9640 1 -- 172.21.15.5:0/358986862 shutdown_connections 2024-09-19T06:03:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.823+0000 7fd328de9640 1 -- 172.21.15.5:0/358986862 wait complete. 2024-09-19T06:03:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.824+0000 7fd328de9640 1 Processor -- start 2024-09-19T06:03:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.824+0000 7fd328de9640 1 -- start start 2024-09-19T06:03:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.824+0000 7fd328de9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd324101de0 0x7fd324198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.824+0000 7fd328de9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd324102790 0x7fd324198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.824+0000 7fd328de9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd324106f10 0x7fd32419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.824+0000 7fd328de9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd324111de0 con 0x7fd324102790 2024-09-19T06:03:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.824+0000 7fd328de9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd324111c60 con 0x7fd324101de0 2024-09-19T06:03:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.824+0000 7fd328de9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd324111f60 con 0x7fd324106f10 2024-09-19T06:03:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.824+0000 7fd321d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd324102790 0x7fd324198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.824+0000 7fd321d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd324102790 0x7fd324198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52036/0 (socket says 172.21.15.5:52036) 2024-09-19T06:03:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.824+0000 7fd321d9b640 1 -- 172.21.15.5:0/3800377350 learned_addr learned my addr 172.21.15.5:0/3800377350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd322d9d640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd324106f10 0x7fd32419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd32259c640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd324101de0 0x7fd324198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd321d9b640 1 -- 172.21.15.5:0/3800377350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd324106f10 msgr2=0x7fd32419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd321d9b640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd324106f10 0x7fd32419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd321d9b640 1 -- 172.21.15.5:0/3800377350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd324101de0 msgr2=0x7fd324198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd321d9b640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd324101de0 0x7fd324198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd321d9b640 1 -- 172.21.15.5:0/3800377350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32419d2f0 con 0x7fd324102790 2024-09-19T06:03:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd321d9b640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd324102790 0x7fd324198880 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd31800d950 tx=0x7fd31800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd3137fe640 1 -- 172.21.15.5:0/3800377350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318017070 con 0x7fd324102790 2024-09-19T06:03:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd3137fe640 1 -- 172.21.15.5:0/3800377350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd318009c30 con 0x7fd324102790 2024-09-19T06:03:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd328de9640 1 -- 172.21.15.5:0/3800377350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32419d5e0 con 0x7fd324102790 2024-09-19T06:03:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd3137fe640 1 -- 172.21.15.5:0/3800377350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31800f620 con 0x7fd324102790 2024-09-19T06:03:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.825+0000 7fd328de9640 1 -- 172.21.15.5:0/3800377350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3241a4ec0 con 0x7fd324102790 2024-09-19T06:03:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.828+0000 7fd3137fe640 1 -- 172.21.15.5:0/3800377350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd3180109a0 con 0x7fd324102790 2024-09-19T06:03:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.828+0000 7fd32b1c2740 1 Processor -- start 2024-09-19T06:03:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.828+0000 7fd32b1c2740 1 -- start start 2024-09-19T06:03:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32b1c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb39d0 0x564e46bb3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32b1c2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb42f0 0x564e46bbe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32b1c2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bc0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32b1c2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e46b1a090 con 0x564e46bb42f0 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32b1c2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564e46ac1a70 con 0x564e46bbebb0 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32b1c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564e46b16050 con 0x564e46bb39d0 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb42f0 0x564e46bbe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb42f0 0x564e46bbe670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52048/0 (socket says 172.21.15.5:52048) 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32259c640 1 -- 172.21.15.5:0/2104273055 learned_addr learned my addr 172.21.15.5:0/2104273055 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd322d9d640 1 --2- 172.21.15.5:0/2104273055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb39d0 0x564e46bb3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd321d9b640 1 --2- 172.21.15.5:0/2104273055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bc0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32259c640 1 -- 172.21.15.5:0/2104273055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb39d0 msgr2=0x564e46bb3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd3137fe640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd30c0a32c0 0x7fd30c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32259c640 1 --2- 172.21.15.5:0/2104273055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb39d0 0x564e46bb3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32259c640 1 -- 172.21.15.5:0/2104273055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 msgr2=0x564e46bc0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32259c640 1 --2- 172.21.15.5:0/2104273055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bc0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32259c640 1 -- 172.21.15.5:0/2104273055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e46ace130 con 0x564e46bb42f0 2024-09-19T06:03:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd321d9b640 1 --2- 172.21.15.5:0/2104273055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bc0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd322d9d640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd30c0a32c0 0x7fd30c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd32259c640 1 --2- 172.21.15.5:0/2104273055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb42f0 0x564e46bbe670 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd314009900 tx=0x7fd314032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769de1de1439a616 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.829+0000 7fd3137fe640 1 -- 172.21.15.5:0/3800377350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd3180cf600 con 0x7fd324102790 2024-09-19T06:03:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.830+0000 7fd310ff9640 1 -- 172.21.15.5:0/2104273055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31403f070 con 0x564e46bb42f0 2024-09-19T06:03:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.830+0000 7fd310ff9640 1 -- 172.21.15.5:0/2104273055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3140316a0 con 0x564e46bb42f0 2024-09-19T06:03:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.830+0000 7fd310ff9640 1 -- 172.21.15.5:0/2104273055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314035070 con 0x564e46bb42f0 2024-09-19T06:03:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.830+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2104273055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb42f0 msgr2=0x564e46bbe670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.830+0000 7fd322d9d640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd30c0a32c0 0x7fd30c0a5780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd31c005fd0 tx=0x7fd31c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.830+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/2104273055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb42f0 0x564e46bbe670 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd314009900 tx=0x7fd314032c20 comp rx=0 tx=0).stop 2024-09-19T06:03:10.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.830+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2104273055 shutdown_connections 2024-09-19T06:03:10.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.830+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/2104273055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bc0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.830+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/2104273055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb42f0 0x564e46bbe670 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.830+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/2104273055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb39d0 0x564e46bb3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2104273055 >> 172.21.15.5:0/2104273055 conn(0x564e46baff60 msgr2=0x564e46bb0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:10.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2104273055 shutdown_connections 2024-09-19T06:03:10.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2104273055 wait complete. 2024-09-19T06:03:10.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32b1c2740 1 -- start start 2024-09-19T06:03:10.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32b1c2740 1 Processor -- start 2024-09-19T06:03:10.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32b1c2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 0x564e46bbe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32b1c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb42f0 0x564e46bbb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32b1c2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bbbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32b1c2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e46bc1780 con 0x564e46bb39d0 2024-09-19T06:03:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32b1c2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564e46ad4f20 con 0x564e46bbebb0 2024-09-19T06:03:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32b1c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564e46bc1900 con 0x564e46bb42f0 2024-09-19T06:03:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 0x564e46bbe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32b1c2740 10 client.0 _ll_register_callbacks cb 0x564e46a9e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 0x564e46bbe440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52064/0 (socket says 172.21.15.5:52064) 2024-09-19T06:03:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.831+0000 7fd32259c640 1 -- 172.21.15.5:0/2525376225 learned_addr learned my addr 172.21.15.5:0/2525376225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd322d9d640 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bbbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd321d9b640 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb42f0 0x564e46bbb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd32259c640 1 -- 172.21.15.5:0/2525376225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb42f0 msgr2=0x564e46bbb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd32259c640 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb42f0 0x564e46bbb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd32259c640 1 -- 172.21.15.5:0/2525376225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 msgr2=0x564e46bbbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd32259c640 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bbbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd32259c640 1 -- 172.21.15.5:0/2525376225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e4688b2b0 con 0x564e46bb39d0 2024-09-19T06:03:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd322d9d640 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bbbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd321d9b640 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb42f0 0x564e46bbb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:03:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd32259c640 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 0x564e46bbe440 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd31402ab30 tx=0x7fd31402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86efd4b6bb2f3635 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd309ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd309ffb640 1 -- 172.21.15.5:0/2525376225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31404a020 con 0x564e46bb39d0 2024-09-19T06:03:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd309ffb640 1 -- 172.21.15.5:0/2525376225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd314004450 con 0x564e46bb39d0 2024-09-19T06:03:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2525376225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3140316a0 con 0x564e46bb39d0 2024-09-19T06:03:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd309ffb640 1 -- 172.21.15.5:0/2525376225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd314055050 con 0x564e46bb39d0 2024-09-19T06:03:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd32b1c2740 10 client.15177 fetch_fsmap learned FSMap version 27 2024-09-19T06:03:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2525376225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e46bcd090 con 0x564e46bb39d0 2024-09-19T06:03:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.832+0000 7fd309ffb640 1 -- 172.21.15.5:0/2525376225 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3140316a0 con 0x564e46bb39d0 2024-09-19T06:03:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.833+0000 7fd309ffb640 1 -- 172.21.15.5:0/2525376225 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd3140049c0 con 0x564e46bb39d0 2024-09-19T06:03:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.833+0000 7fd32b1c2740 10 client.15177 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:03:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.833+0000 7fd32b1c2740 10 client.15177 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T06:03:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.833+0000 7fd32b1c2740 20 client.15177 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.833+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x564e46bcf2b0 0x564e46bd0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.833+0000 7fd32b1c2740 4 client.15177 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.833+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2525376225 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e46b27600 con 0x564e46bcf2b0 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.833+0000 7fd321d9b640 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x564e46bcf2b0 0x564e46bd0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.833+0000 7fd309ffb640 10 client.15177 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.833+0000 7fd321d9b640 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x564e46bcf2b0 0x564e46bd0720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74490d5cfef63368 server_cookie=b3d1481692651fa4 in_seq=0 out_seq=0 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd309ffb640 1 -- 172.21.15.5:0/2525376225 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e46b27600 con 0x564e46bcf2b0 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd309ffb640 10 client.15177 handle_command_reply: tid=0 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd32b1c2740 1 client.15177 shutdown 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2525376225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 msgr2=0x564e46bbe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 0x564e46bbe440 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd31402ab30 tx=0x7fd31402a410 comp rx=0 tx=0).stop 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2525376225 shutdown_connections 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x564e46bcf2b0 0x564e46bd0720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bbbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb42f0 0x564e46bbb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd309ffb640 0 client.15177 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:03:10.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/2525376225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 0x564e46bbe440 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2525376225 >> 172.21.15.5:0/2525376225 conn(0x564e46baff60 msgr2=0x564e46bb5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:10.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2525376225 shutdown_connections 2024-09-19T06:03:10.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd32b1c2740 1 -- 172.21.15.5:0/2525376225 wait complete. 2024-09-19T06:03:10.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.834+0000 7fd32b1c2740 20 client.15177 trim_cache size 0 max 16384 2024-09-19T06:03:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.852+0000 7fd32b1c2740 1 Processor -- start 2024-09-19T06:03:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32b1c2740 1 -- start start 2024-09-19T06:03:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32b1c2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 0x564e46bd0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32b1c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb42f0 0x564e46baae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32b1c2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bbd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32b1c2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e46bcea60 con 0x564e46bb39d0 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32b1c2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564e46bcf130 con 0x564e46bbebb0 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32b1c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564e46bcf800 con 0x564e46bb42f0 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 0x564e46bd0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 0x564e46bd0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52078/0 (socket says 172.21.15.5:52078) 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32259c640 1 -- 172.21.15.5:0/3481435620 learned_addr learned my addr 172.21.15.5:0/3481435620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32259c640 1 -- 172.21.15.5:0/3481435620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb42f0 msgr2=0x564e46baae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd321d9b640 1 --2- 172.21.15.5:0/3481435620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb42f0 0x564e46baae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32259c640 1 --2- 172.21.15.5:0/3481435620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb42f0 0x564e46baae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32259c640 1 -- 172.21.15.5:0/3481435620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 msgr2=0x564e46bbd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd322d9d640 1 --2- 172.21.15.5:0/3481435620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bbd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32259c640 1 --2- 172.21.15.5:0/3481435620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bbd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32259c640 1 -- 172.21.15.5:0/3481435620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e46bc1300 con 0x564e46bb39d0 2024-09-19T06:03:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd32259c640 1 --2- 172.21.15.5:0/3481435620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 0x564e46bd0720 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd31402ab60 tx=0x7fd31403b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986c45b576fa6edb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.853+0000 7fd30bfff640 1 -- 172.21.15.5:0/3481435620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31404a070 con 0x564e46bb39d0 2024-09-19T06:03:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.854+0000 7fd30bfff640 1 -- 172.21.15.5:0/3481435620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd314004250 con 0x564e46bb39d0 2024-09-19T06:03:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.854+0000 7fd30bfff640 1 -- 172.21.15.5:0/3481435620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31403f070 con 0x564e46bb39d0 2024-09-19T06:03:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.854+0000 7fd32b1c2740 1 -- 172.21.15.5:0/3481435620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 msgr2=0x564e46bd0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.854+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/3481435620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 0x564e46bd0720 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd31402ab60 tx=0x7fd31403b430 comp rx=0 tx=0).stop 2024-09-19T06:03:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.854+0000 7fd32b1c2740 1 -- 172.21.15.5:0/3481435620 shutdown_connections 2024-09-19T06:03:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.854+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/3481435620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bbebb0 0x564e46bbd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.854+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/3481435620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb42f0 0x564e46baae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.854+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/3481435620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bb39d0 0x564e46bd0720 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.854+0000 7fd32b1c2740 1 -- 172.21.15.5:0/3481435620 >> 172.21.15.5:0/3481435620 conn(0x564e46baff60 msgr2=0x564e46bb0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.854+0000 7fd32b1c2740 1 -- 172.21.15.5:0/3481435620 shutdown_connections 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.854+0000 7fd32b1c2740 1 -- 172.21.15.5:0/3481435620 wait complete. 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd32b1c2740 1 -- start start 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd32b1c2740 1 Processor -- start 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd32b1c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb39d0 0x564e46bd3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd32b1c2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bb42f0 0x564e46bd44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd32b1c2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bbebb0 0x564e46c31980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd32b1c2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e46bd05a0 con 0x564e46bbebb0 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd32b1c2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564e46bcfed0 con 0x564e46bb42f0 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd32b1c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564e46bd0c70 con 0x564e46bb39d0 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd32b1c2740 10 client.0 _ll_register_callbacks cb 0x7fd31c009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd322d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bbebb0 0x564e46c31980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd322d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bbebb0 0x564e46c31980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52088/0 (socket says 172.21.15.5:52088) 2024-09-19T06:03:10.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd322d9d640 1 -- 172.21.15.5:0/181568244 learned_addr learned my addr 172.21.15.5:0/181568244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:10.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd321d9b640 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bb42f0 0x564e46bd44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd32259c640 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb39d0 0x564e46bd3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd322d9d640 1 -- 172.21.15.5:0/181568244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb39d0 msgr2=0x564e46bd3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:10.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.855+0000 7fd322d9d640 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb39d0 0x564e46bd3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd322d9d640 1 -- 172.21.15.5:0/181568244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bb42f0 msgr2=0x564e46bd44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:10.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd322d9d640 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bb42f0 0x564e46bd44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd322d9d640 1 -- 172.21.15.5:0/181568244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e46c31fc0 con 0x564e46bbebb0 2024-09-19T06:03:10.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd321d9b640 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bb42f0 0x564e46bd44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:10.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd322d9d640 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bbebb0 0x564e46c31980 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd31c012a30 tx=0x7fd31c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa50ac7b2398e119 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd309ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd309ffb640 1 -- 172.21.15.5:0/181568244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c011d70 con 0x564e46bbebb0 2024-09-19T06:03:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd309ffb640 1 -- 172.21.15.5:0/181568244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd31c021440 con 0x564e46bbebb0 2024-09-19T06:03:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd32b1c2740 1 -- 172.21.15.5:0/181568244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd314004250 con 0x564e46bbebb0 2024-09-19T06:03:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd309ffb640 1 -- 172.21.15.5:0/181568244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd31c021680 con 0x564e46bbebb0 2024-09-19T06:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd32b1c2740 10 client.15183 fetch_fsmap learned FSMap version 27 2024-09-19T06:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd32b1c2740 1 -- 172.21.15.5:0/181568244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e46c39370 con 0x564e46bbebb0 2024-09-19T06:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.856+0000 7fd309ffb640 1 -- 172.21.15.5:0/181568244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd314004250 con 0x564e46bbebb0 2024-09-19T06:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.857+0000 7fd309ffb640 1 -- 172.21.15.5:0/181568244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd31c00f040 con 0x564e46bbebb0 2024-09-19T06:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.857+0000 7fd32b1c2740 10 client.15183 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.857+0000 7fd32b1c2740 10 client.15183 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T06:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.857+0000 7fd32b1c2740 20 client.15183 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.857+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x564e46c3e040 0x564e46c3cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.857+0000 7fd32b1c2740 4 client.15183 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.857+0000 7fd32b1c2740 1 -- 172.21.15.5:0/181568244 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "status"}) -- 0x564e46b21b10 con 0x564e46c3e040 2024-09-19T06:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.857+0000 7fd32259c640 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x564e46c3e040 0x564e46c3cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32259c640 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x564e46c3e040 0x564e46c3cd50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87abdb5e28f0d9e1 server_cookie=a1b115285b06489b in_seq=0 out_seq=0 2024-09-19T06:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd309ffb640 10 client.15183 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd309ffb640 1 -- 172.21.15.5:0/181568244 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564e46b21b10 con 0x564e46c3e040 2024-09-19T06:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd309ffb640 10 client.15183 handle_command_reply: tid=0 2024-09-19T06:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32b1c2740 1 client.15183 shutdown 2024-09-19T06:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32b1c2740 1 -- 172.21.15.5:0/181568244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bbebb0 msgr2=0x564e46c31980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bbebb0 0x564e46c31980 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd31c012a30 tx=0x7fd31c004ab0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32b1c2740 1 -- 172.21.15.5:0/181568244 shutdown_connections 2024-09-19T06:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x564e46c3e040 0x564e46c3cd50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e46bbebb0 0x564e46c31980 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e46bb42f0 0x564e46bd44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32b1c2740 1 --2- 172.21.15.5:0/181568244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e46bb39d0 0x564e46bd3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32b1c2740 1 -- 172.21.15.5:0/181568244 >> 172.21.15.5:0/181568244 conn(0x564e46baff60 msgr2=0x564e46bbb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd309ffb640 0 client.15183 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:03:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32b1c2740 1 -- 172.21.15.5:0/181568244 shutdown_connections 2024-09-19T06:03:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32b1c2740 1 -- 172.21.15.5:0/181568244 wait complete. 2024-09-19T06:03:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.858+0000 7fd32b1c2740 20 client.15183 trim_cache size 0 max 16384 2024-09-19T06:03:10.874 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:03:10.874 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T06:03:10.874 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-19T06:03:10.874 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24448, 2024-09-19T06:03:10.874 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-19T06:03:10.874 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-19T06:03:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-19T06:03:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 93.039353145999996, 2024-09-19T06:03:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-19T06:03:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-19T06:03:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:03:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 125.42787604599999 2024-09-19T06:03:10.876 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.859+0000 7fd328de9640 1 -- 172.21.15.5:0/3800377350 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd30c0a32c0 msgr2=0x7fd30c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.859+0000 7fd328de9640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd30c0a32c0 0x7fd30c0a5780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd31c005fd0 tx=0x7fd31c004a40 comp rx=0 tx=0).stop 2024-09-19T06:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.859+0000 7fd328de9640 1 -- 172.21.15.5:0/3800377350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd324102790 msgr2=0x7fd324198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.859+0000 7fd328de9640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd324102790 0x7fd324198880 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd31800d950 tx=0x7fd31800de10 comp rx=0 tx=0).stop 2024-09-19T06:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.859+0000 7fd328de9640 1 -- 172.21.15.5:0/3800377350 shutdown_connections 2024-09-19T06:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.859+0000 7fd328de9640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd30c0a32c0 0x7fd30c0a5780 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.859+0000 7fd328de9640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd324106f10 0x7fd32419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.859+0000 7fd328de9640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd324102790 0x7fd324198880 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.859+0000 7fd328de9640 1 --2- 172.21.15.5:0/3800377350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd324101de0 0x7fd324198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.859+0000 7fd328de9640 1 -- 172.21.15.5:0/3800377350 >> 172.21.15.5:0/3800377350 conn(0x7fd3240fd8c0 msgr2=0x7fd3240fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.859+0000 7fd328de9640 1 -- 172.21.15.5:0/3800377350 shutdown_connections 2024-09-19T06:03:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:10.859+0000 7fd328de9640 1 -- 172.21.15.5:0/3800377350 wait complete. 2024-09-19T06:03:11.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T06:03:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.255+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3231927209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc106f20 msgr2=0x7fbadc10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.255+0000 7fbae3f3a640 1 --2- 172.21.15.5:0/3231927209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc106f20 0x7fbadc10eaa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009ca0 tx=0x7fbad402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.255+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3231927209 shutdown_connections 2024-09-19T06:03:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.255+0000 7fbae3f3a640 1 --2- 172.21.15.5:0/3231927209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc106f20 0x7fbadc10eaa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.255+0000 7fbae3f3a640 1 --2- 172.21.15.5:0/3231927209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbadc1027a0 0x7fbadc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.255+0000 7fbae3f3a640 1 --2- 172.21.15.5:0/3231927209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbadc101df0 0x7fbadc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.255+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3231927209 >> 172.21.15.5:0/3231927209 conn(0x7fbadc0fd8f0 msgr2=0x7fbadc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.255+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3231927209 shutdown_connections 2024-09-19T06:03:12.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.255+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3231927209 wait complete. 2024-09-19T06:03:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae3f3a640 1 Processor -- start 2024-09-19T06:03:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae3f3a640 1 -- start start 2024-09-19T06:03:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae3f3a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc101df0 0x7fbadc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae3f3a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbadc1027a0 0x7fbadc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae3f3a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbadc106f20 0x7fbadc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae3f3a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc111e30 con 0x7fbadc101df0 2024-09-19T06:03:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae3f3a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc111cb0 con 0x7fbadc1027a0 2024-09-19T06:03:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae3f3a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc111fb0 con 0x7fbadc106f20 2024-09-19T06:03:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae1cd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc101df0 0x7fbadc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae1cd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc101df0 0x7fbadc1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52112/0 (socket says 172.21.15.5:52112) 2024-09-19T06:03:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae1cd6640 1 -- 172.21.15.5:0/3866594841 learned_addr learned my addr 172.21.15.5:0/3866594841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae14d5640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbadc1027a0 0x7fbadc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae24d7640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbadc106f20 0x7fbadc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae1cd6640 1 -- 172.21.15.5:0/3866594841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbadc106f20 msgr2=0x7fbadc19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbadc106f20 0x7fbadc19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae1cd6640 1 -- 172.21.15.5:0/3866594841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbadc1027a0 msgr2=0x7fbadc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbadc1027a0 0x7fbadc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.256+0000 7fbae1cd6640 1 -- 172.21.15.5:0/3866594841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc19d3a0 con 0x7fbadc101df0 2024-09-19T06:03:12.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.257+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc101df0 0x7fbadc1983f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbad800d770 tx=0x7fbad800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:12.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.257+0000 7fbad2ffd640 1 -- 172.21.15.5:0/3866594841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8017070 con 0x7fbadc101df0 2024-09-19T06:03:12.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.257+0000 7fbad2ffd640 1 -- 172.21.15.5:0/3866594841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbad80054d0 con 0x7fbadc101df0 2024-09-19T06:03:12.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.257+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3866594841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbadc19d690 con 0x7fbadc101df0 2024-09-19T06:03:12.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.257+0000 7fbad2ffd640 1 -- 172.21.15.5:0/3866594841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8004cf0 con 0x7fbadc101df0 2024-09-19T06:03:12.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.257+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3866594841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbadc1a4f70 con 0x7fbadc101df0 2024-09-19T06:03:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.259+0000 7fbad2ffd640 1 -- 172.21.15.5:0/3866594841 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fbad8004e90 con 0x7fbadc101df0 2024-09-19T06:03:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.259+0000 7fbae6313740 1 Processor -- start 2024-09-19T06:03:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae6313740 1 -- start start 2024-09-19T06:03:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae6313740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468adde0 0x5561468ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae6313740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468ae700 0x5561468b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae6313740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x5561468bb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae6313740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556146810090 con 0x5561468ae700 2024-09-19T06:03:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae6313740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5561467b7a70 con 0x5561468adde0 2024-09-19T06:03:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae6313740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55614680c050 con 0x5561468b8fc0 2024-09-19T06:03:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae1cd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468ae700 0x5561468b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae1cd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468ae700 0x5561468b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52128/0 (socket says 172.21.15.5:52128) 2024-09-19T06:03:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbad2ffd640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbac80a3240 0x7fbac80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae1cd6640 1 -- 172.21.15.5:0/805889835 learned_addr learned my addr 172.21.15.5:0/805889835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae24d7640 1 --2- 172.21.15.5:0/805889835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468adde0 0x5561468ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae14d5640 1 --2- 172.21.15.5:0/805889835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x5561468bb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae24d7640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbac80a3240 0x7fbac80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae1cd6640 1 -- 172.21.15.5:0/805889835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 msgr2=0x5561468bb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/805889835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x5561468bb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae1cd6640 1 -- 172.21.15.5:0/805889835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468adde0 msgr2=0x5561468ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/805889835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468adde0 0x5561468ae1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.260+0000 7fbae1cd6640 1 -- 172.21.15.5:0/805889835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561467c4130 con 0x5561468ae700 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.261+0000 7fbad2ffd640 1 -- 172.21.15.5:0/3866594841 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fbad80cf390 con 0x7fbadc101df0 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.261+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/805889835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468ae700 0x5561468b8a80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbad809b6a0 tx=0x7fbad8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25aa74ae9cfbafe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.261+0000 7fbac7fff640 1 -- 172.21.15.5:0/805889835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8005180 con 0x5561468ae700 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.261+0000 7fbae24d7640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbac80a3240 0x7fbac80a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbad4005e30 tx=0x7fbad40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.261+0000 7fbac7fff640 1 -- 172.21.15.5:0/805889835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbad8009c30 con 0x5561468ae700 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.261+0000 7fbac7fff640 1 -- 172.21.15.5:0/805889835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad80a4610 con 0x5561468ae700 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.262+0000 7fbae6313740 1 -- 172.21.15.5:0/805889835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468ae700 msgr2=0x5561468b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.262+0000 7fbae6313740 1 --2- 172.21.15.5:0/805889835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468ae700 0x5561468b8a80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbad809b6a0 tx=0x7fbad8022000 comp rx=0 tx=0).stop 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.262+0000 7fbae6313740 1 -- 172.21.15.5:0/805889835 shutdown_connections 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.262+0000 7fbae6313740 1 --2- 172.21.15.5:0/805889835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x5561468bb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.262+0000 7fbae6313740 1 --2- 172.21.15.5:0/805889835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468ae700 0x5561468b8a80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.262+0000 7fbae6313740 1 --2- 172.21.15.5:0/805889835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468adde0 0x5561468ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.262+0000 7fbae6313740 1 -- 172.21.15.5:0/805889835 >> 172.21.15.5:0/805889835 conn(0x5561468aa370 msgr2=0x5561468aa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.262+0000 7fbae6313740 1 -- 172.21.15.5:0/805889835 shutdown_connections 2024-09-19T06:03:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.262+0000 7fbae6313740 1 -- 172.21.15.5:0/805889835 wait complete. 2024-09-19T06:03:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.262+0000 7fbae6313740 1 -- start start 2024-09-19T06:03:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.262+0000 7fbae6313740 1 Processor -- start 2024-09-19T06:03:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae6313740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae6313740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 0x5561468b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae6313740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x5561468b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae6313740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5561468bbb30 con 0x5561468adde0 2024-09-19T06:03:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae6313740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5561467caf20 con 0x5561468ae700 2024-09-19T06:03:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae6313740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5561468bbcb0 con 0x5561468b8fc0 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae6313740 10 client.0 _ll_register_callbacks cb 0x556146794030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae1cd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae1cd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468b8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52134/0 (socket says 172.21.15.5:52134) 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae1cd6640 1 -- 172.21.15.5:0/1301392231 learned_addr learned my addr 172.21.15.5:0/1301392231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae14d5640 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 0x5561468b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae24d7640 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x5561468b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae1cd6640 1 -- 172.21.15.5:0/1301392231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 msgr2=0x5561468b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x5561468b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae1cd6640 1 -- 172.21.15.5:0/1301392231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 msgr2=0x5561468b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 0x5561468b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae1cd6640 1 -- 172.21.15.5:0/1301392231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561465812b0 con 0x5561468adde0 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468b8800 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbad80cced0 tx=0x7fbad800de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4ea00c9ea3278 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.263+0000 7fbac57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbac57fa640 1 -- 172.21.15.5:0/1301392231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad809b9a0 con 0x5561468adde0 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbac57fa640 1 -- 172.21.15.5:0/1301392231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fbad8098010 con 0x5561468adde0 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbae6313740 1 -- 172.21.15.5:0/1301392231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55614681d600 con 0x5561468adde0 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbac57fa640 1 -- 172.21.15.5:0/1301392231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbad809c0a0 con 0x5561468adde0 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbae6313740 10 client.15195 fetch_fsmap learned FSMap version 27 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbae6313740 1 -- 172.21.15.5:0/1301392231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561468c73f0 con 0x5561468adde0 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbac57fa640 1 -- 172.21.15.5:0/1301392231 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55614681d600 con 0x5561468adde0 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbac57fa640 1 -- 172.21.15.5:0/1301392231 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbad80ba0f0 con 0x5561468adde0 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbae6313740 10 client.15195 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbae6313740 10 client.15195 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbae6313740 20 client.15195 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbae6313740 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5561468c9510 0x5561468ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbae6313740 4 client.15195 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:03:12.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.264+0000 7fbae6313740 1 -- 172.21.15.5:0/1301392231 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556146817b10 con 0x5561468c9510 2024-09-19T06:03:12.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.265+0000 7fbae14d5640 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5561468c9510 0x5561468ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.265+0000 7fbae14d5640 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5561468c9510 0x5561468ca980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3de3ff358fb2ea4 server_cookie=e21b0e0d340670d2 in_seq=0 out_seq=0 2024-09-19T06:03:12.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.265+0000 7fbac57fa640 10 client.15195 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:03:12.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbac57fa640 1 -- 172.21.15.5:0/1301392231 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556146817b10 con 0x5561468c9510 2024-09-19T06:03:12.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbac57fa640 10 client.15195 handle_command_reply: tid=0 2024-09-19T06:03:12.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbae6313740 1 client.15195 shutdown 2024-09-19T06:03:12.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbae6313740 1 -- 172.21.15.5:0/1301392231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 msgr2=0x5561468b8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:12.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbae6313740 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468b8800 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbad80cced0 tx=0x7fbad800de80 comp rx=0 tx=0).stop 2024-09-19T06:03:12.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbae6313740 1 -- 172.21.15.5:0/1301392231 shutdown_connections 2024-09-19T06:03:12.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbae6313740 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5561468c9510 0x5561468ca980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbae6313740 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x5561468b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbac57fa640 0 client.15195 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:03:12.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbae6313740 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 0x5561468b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbae6313740 1 --2- 172.21.15.5:0/1301392231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468b8800 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbae6313740 1 -- 172.21.15.5:0/1301392231 >> 172.21.15.5:0/1301392231 conn(0x5561468aa370 msgr2=0x5561468b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:12.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbae6313740 1 -- 172.21.15.5:0/1301392231 shutdown_connections 2024-09-19T06:03:12.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbae6313740 1 -- 172.21.15.5:0/1301392231 wait complete. 2024-09-19T06:03:12.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.266+0000 7fbae6313740 20 client.15195 trim_cache size 0 max 16384 2024-09-19T06:03:12.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.284+0000 7fbae6313740 1 Processor -- start 2024-09-19T06:03:12.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.284+0000 7fbae6313740 1 -- start start 2024-09-19T06:03:12.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.284+0000 7fbae6313740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.284+0000 7fbae6313740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 0x5561468a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.284+0000 7fbae6313740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x5561468b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.284+0000 7fbae6313740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5561468c8cc0 con 0x5561468adde0 2024-09-19T06:03:12.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.284+0000 7fbae6313740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5561468c9390 con 0x5561468ae700 2024-09-19T06:03:12.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.284+0000 7fbae6313740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5561468c9a60 con 0x5561468b8fc0 2024-09-19T06:03:12.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.284+0000 7fbae1cd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.284+0000 7fbae1cd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52148/0 (socket says 172.21.15.5:52148) 2024-09-19T06:03:12.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.284+0000 7fbae1cd6640 1 -- 172.21.15.5:0/3640434826 learned_addr learned my addr 172.21.15.5:0/3640434826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:12.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.285+0000 7fbae14d5640 1 --2- 172.21.15.5:0/3640434826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 0x5561468a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.285+0000 7fbae1cd6640 1 -- 172.21.15.5:0/3640434826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 msgr2=0x5561468b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:12.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.285+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/3640434826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x5561468b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.285+0000 7fbae1cd6640 1 -- 172.21.15.5:0/3640434826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 msgr2=0x5561468a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:12.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.285+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/3640434826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 0x5561468a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.285+0000 7fbae1cd6640 1 -- 172.21.15.5:0/3640434826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561468ae530 con 0x5561468adde0 2024-09-19T06:03:12.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.285+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/3640434826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468ca980 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbad809b930 tx=0x7fbad80862e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4819a8bdfd3c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:12.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.285+0000 7fbac77fe640 1 -- 172.21.15.5:0/3640434826 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8020050 con 0x5561468adde0 2024-09-19T06:03:12.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.285+0000 7fbac77fe640 1 -- 172.21.15.5:0/3640434826 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbad80b4ba0 con 0x5561468adde0 2024-09-19T06:03:12.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.285+0000 7fbac77fe640 1 -- 172.21.15.5:0/3640434826 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad80ba0f0 con 0x5561468adde0 2024-09-19T06:03:12.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.286+0000 7fbae6313740 1 -- 172.21.15.5:0/3640434826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 msgr2=0x5561468ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:12.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.286+0000 7fbae6313740 1 --2- 172.21.15.5:0/3640434826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468ca980 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbad809b930 tx=0x7fbad80862e0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.286+0000 7fbae6313740 1 -- 172.21.15.5:0/3640434826 shutdown_connections 2024-09-19T06:03:12.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.286+0000 7fbae6313740 1 --2- 172.21.15.5:0/3640434826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x5561468b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.286+0000 7fbae6313740 1 --2- 172.21.15.5:0/3640434826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 0x5561468a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.286+0000 7fbae6313740 1 --2- 172.21.15.5:0/3640434826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468ca980 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.286+0000 7fbae6313740 1 -- 172.21.15.5:0/3640434826 >> 172.21.15.5:0/3640434826 conn(0x5561468aa370 msgr2=0x5561468a47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:12.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.286+0000 7fbae6313740 1 -- 172.21.15.5:0/3640434826 shutdown_connections 2024-09-19T06:03:12.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.286+0000 7fbae6313740 1 -- 172.21.15.5:0/3640434826 wait complete. 2024-09-19T06:03:12.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.286+0000 7fbae6313740 1 -- start start 2024-09-19T06:03:12.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.286+0000 7fbae6313740 1 Processor -- start 2024-09-19T06:03:12.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae6313740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae6313740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 0x5561468ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae6313740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x556146931c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae6313740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5561468ca800 con 0x5561468adde0 2024-09-19T06:03:12.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae6313740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5561468ca130 con 0x5561468ae700 2024-09-19T06:03:12.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae6313740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5561468caed0 con 0x5561468b8fc0 2024-09-19T06:03:12.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae6313740 10 client.0 _ll_register_callbacks cb 0x7fbad809b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:12.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae1cd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae1cd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52164/0 (socket says 172.21.15.5:52164) 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae1cd6640 1 -- 172.21.15.5:0/1256078543 learned_addr learned my addr 172.21.15.5:0/1256078543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae14d5640 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 0x5561468ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae24d7640 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x556146931c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae1cd6640 1 -- 172.21.15.5:0/1256078543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 msgr2=0x556146931c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x556146931c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae1cd6640 1 -- 172.21.15.5:0/1256078543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 msgr2=0x5561468ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 0x5561468ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae1cd6640 1 -- 172.21.15.5:0/1256078543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556146932200 con 0x5561468adde0 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae1cd6640 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468ce190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbad80120c0 tx=0x7fbad809fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f11db0d3e41ac56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbac57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbac57fa640 1 -- 172.21.15.5:0/1256078543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8020030 con 0x5561468adde0 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbac57fa640 1 -- 172.21.15.5:0/1256078543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fbad80ba0f0 con 0x5561468adde0 2024-09-19T06:03:12.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.287+0000 7fbae6313740 1 -- 172.21.15.5:0/1256078543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5561467940e0 con 0x5561468adde0 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.288+0000 7fbac57fa640 1 -- 172.21.15.5:0/1256078543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbad80ba2f0 con 0x5561468adde0 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.288+0000 7fbae6313740 10 client.15201 fetch_fsmap learned FSMap version 27 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.288+0000 7fbae6313740 1 -- 172.21.15.5:0/1256078543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556146932200 con 0x5561468adde0 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.288+0000 7fbac57fa640 1 -- 172.21.15.5:0/1256078543 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5561467940e0 con 0x5561468adde0 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.288+0000 7fbac57fa640 1 -- 172.21.15.5:0/1256078543 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbad809a9d0 con 0x5561468adde0 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.288+0000 7fbae6313740 10 client.15201 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.288+0000 7fbae6313740 10 client.15201 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.288+0000 7fbae6313740 20 client.15201 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.288+0000 7fbae6313740 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5561469379f0 0x556146936700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.288+0000 7fbae6313740 4 client.15201 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.288+0000 7fbae6313740 1 -- 172.21.15.5:0/1256078543 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "status"}) -- 0x5561467e1750 con 0x5561469379f0 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.288+0000 7fbae14d5640 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5561469379f0 0x556146936700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae14d5640 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5561469379f0 0x556146936700 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=845ee36cd1d0962f server_cookie=759a3f4fed1555 in_seq=0 out_seq=0 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbac57fa640 10 client.15201 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbac57fa640 1 -- 172.21.15.5:0/1256078543 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5561467e1750 con 0x5561469379f0 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbac57fa640 10 client.15201 handle_command_reply: tid=0 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae6313740 1 client.15201 shutdown 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae6313740 1 -- 172.21.15.5:0/1256078543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 msgr2=0x5561468ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae6313740 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468ce190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbad80120c0 tx=0x7fbad809fde0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae6313740 1 -- 172.21.15.5:0/1256078543 shutdown_connections 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae6313740 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5561469379f0 0x556146936700 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae6313740 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561468b8fc0 0x556146931c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae6313740 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561468ae700 0x5561468ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbac57fa640 0 client.15201 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae6313740 1 --2- 172.21.15.5:0/1256078543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561468adde0 0x5561468ce190 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae6313740 1 -- 172.21.15.5:0/1256078543 >> 172.21.15.5:0/1256078543 conn(0x5561468aa370 msgr2=0x5561468a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:12.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae6313740 1 -- 172.21.15.5:0/1256078543 shutdown_connections 2024-09-19T06:03:12.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae6313740 1 -- 172.21.15.5:0/1256078543 wait complete. 2024-09-19T06:03:12.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.289+0000 7fbae6313740 20 client.15201 trim_cache size 0 max 16384 2024-09-19T06:03:12.305 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:03:12.305 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T06:03:12.305 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-19T06:03:12.305 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24463, 2024-09-19T06:03:12.305 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-19T06:03:12.305 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-19T06:03:12.305 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-19T06:03:12.306 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 95.475237054000004, 2024-09-19T06:03:12.306 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-19T06:03:12.306 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-19T06:03:12.306 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:03:12.306 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 121.756252901 2024-09-19T06:03:12.306 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:03:12.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.290+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3866594841 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbac80a3240 msgr2=0x7fbac80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:12.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.290+0000 7fbae3f3a640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbac80a3240 0x7fbac80a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbad4005e30 tx=0x7fbad40095c0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.290+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3866594841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc101df0 msgr2=0x7fbadc1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:12.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.290+0000 7fbae3f3a640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc101df0 0x7fbadc1983f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbad800d770 tx=0x7fbad800dc30 comp rx=0 tx=0).stop 2024-09-19T06:03:12.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.290+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3866594841 shutdown_connections 2024-09-19T06:03:12.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.290+0000 7fbae3f3a640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fbac80a3240 0x7fbac80a5700 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.290+0000 7fbae3f3a640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbadc106f20 0x7fbadc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.290+0000 7fbae3f3a640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbadc1027a0 0x7fbadc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.290+0000 7fbae3f3a640 1 --2- 172.21.15.5:0/3866594841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc101df0 0x7fbadc1983f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:12.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.290+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3866594841 >> 172.21.15.5:0/3866594841 conn(0x7fbadc0fd8f0 msgr2=0x7fbadc0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:12.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.290+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3866594841 shutdown_connections 2024-09-19T06:03:12.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:12.290+0000 7fbae3f3a640 1 -- 172.21.15.5:0/3866594841 wait complete. 2024-09-19T06:03:12.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:12 smithi005 ceph-mon[30955]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T06:03:12.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:12 smithi049 ceph-mon[34640]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T06:03:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:12 smithi157 ceph-mon[35346]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T06:03:13.014 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T06:03:13.271 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.719+0000 7fd69858c640 1 -- 172.21.15.5:0/429644509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690102790 msgr2=0x7fd6901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.719+0000 7fd69858c640 1 --2- 172.21.15.5:0/429644509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690102790 0x7fd6901067e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd680009900 tx=0x7fd68002a410 comp rx=0 tx=0).stop 2024-09-19T06:03:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.719+0000 7fd69858c640 1 -- 172.21.15.5:0/429644509 shutdown_connections 2024-09-19T06:03:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.719+0000 7fd69858c640 1 --2- 172.21.15.5:0/429644509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690106f10 0x7fd69010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.719+0000 7fd69858c640 1 --2- 172.21.15.5:0/429644509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690102790 0x7fd6901067e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.719+0000 7fd69858c640 1 --2- 172.21.15.5:0/429644509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd690101de0 0x7fd6901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.719+0000 7fd69858c640 1 -- 172.21.15.5:0/429644509 >> 172.21.15.5:0/429644509 conn(0x7fd6900fd8c0 msgr2=0x7fd6900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.720+0000 7fd69858c640 1 -- 172.21.15.5:0/429644509 shutdown_connections 2024-09-19T06:03:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.720+0000 7fd69858c640 1 -- 172.21.15.5:0/429644509 wait complete. 2024-09-19T06:03:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.720+0000 7fd69858c640 1 Processor -- start 2024-09-19T06:03:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.720+0000 7fd69858c640 1 -- start start 2024-09-19T06:03:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.720+0000 7fd69858c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690101de0 0x7fd690198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.720+0000 7fd69858c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690102790 0x7fd690198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.720+0000 7fd69858c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd690106f10 0x7fd69019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.720+0000 7fd69858c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd690111e70 con 0x7fd690102790 2024-09-19T06:03:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.720+0000 7fd69858c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd690111cf0 con 0x7fd690106f10 2024-09-19T06:03:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.720+0000 7fd69858c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd690111ff0 con 0x7fd690101de0 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.720+0000 7fd695b27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690102790 0x7fd690198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd695b27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690102790 0x7fd690198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52180/0 (socket says 172.21.15.5:52180) 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd695b27640 1 -- 172.21.15.5:0/136802024 learned_addr learned my addr 172.21.15.5:0/136802024 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd696b29640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd690106f10 0x7fd69019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd696328640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690101de0 0x7fd690198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd695b27640 1 -- 172.21.15.5:0/136802024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690101de0 msgr2=0x7fd690198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd695b27640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690101de0 0x7fd690198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd695b27640 1 -- 172.21.15.5:0/136802024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd690106f10 msgr2=0x7fd69019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd695b27640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd690106f10 0x7fd69019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd695b27640 1 -- 172.21.15.5:0/136802024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69019d2c0 con 0x7fd690102790 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd696b29640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd690106f10 0x7fd69019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd696328640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690101de0 0x7fd690198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:03:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd695b27640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690102790 0x7fd690198850 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd680005ab0 tx=0x7fd680003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd6877fe640 1 -- 172.21.15.5:0/136802024 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68003f070 con 0x7fd690102790 2024-09-19T06:03:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd69858c640 1 -- 172.21.15.5:0/136802024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69019d550 con 0x7fd690102790 2024-09-19T06:03:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd6877fe640 1 -- 172.21.15.5:0/136802024 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd680031d60 con 0x7fd690102790 2024-09-19T06:03:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd69858c640 1 -- 172.21.15.5:0/136802024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6901a4e90 con 0x7fd690102790 2024-09-19T06:03:13.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.721+0000 7fd6877fe640 1 -- 172.21.15.5:0/136802024 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680035030 con 0x7fd690102790 2024-09-19T06:03:13.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.724+0000 7fd6877fe640 1 -- 172.21.15.5:0/136802024 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd680042070 con 0x7fd690102790 2024-09-19T06:03:13.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.724+0000 7fd69a965740 1 Processor -- start 2024-09-19T06:03:13.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.724+0000 7fd69a965740 1 -- start start 2024-09-19T06:03:13.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.724+0000 7fd69a965740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63df9d0 0x5610b63dfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.724+0000 7fd69a965740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63e02f0 0x5610b63ea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.724+0000 7fd69a965740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 0x5610b63ecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.724+0000 7fd69a965740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5610b6346090 con 0x5610b63e02f0 2024-09-19T06:03:13.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.724+0000 7fd69a965740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5610b62eda70 con 0x5610b63df9d0 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.724+0000 7fd69a965740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5610b6342050 con 0x5610b63eabb0 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696328640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63e02f0 0x5610b63ea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696328640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63e02f0 0x5610b63ea670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52190/0 (socket says 172.21.15.5:52190) 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696328640 1 -- 172.21.15.5:0/169871408 learned_addr learned my addr 172.21.15.5:0/169871408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd6877fe640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd67c0a32c0 0x7fd67c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd695b27640 1 --2- 172.21.15.5:0/169871408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 0x5610b63ecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696b29640 1 --2- 172.21.15.5:0/169871408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63df9d0 0x5610b63dfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696328640 1 -- 172.21.15.5:0/169871408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 msgr2=0x5610b63ecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696b29640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd67c0a32c0 0x7fd67c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696328640 1 --2- 172.21.15.5:0/169871408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 0x5610b63ecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696328640 1 -- 172.21.15.5:0/169871408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63df9d0 msgr2=0x5610b63dfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696328640 1 --2- 172.21.15.5:0/169871408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63df9d0 0x5610b63dfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696328640 1 -- 172.21.15.5:0/169871408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b62fa130 con 0x5610b63e02f0 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd6877fe640 1 -- 172.21.15.5:0/136802024 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd6800f03a0 con 0x7fd690102790 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd695b27640 1 --2- 172.21.15.5:0/169871408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 0x5610b63ecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696b29640 1 --2- 172.21.15.5:0/169871408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63df9d0 0x5610b63dfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696328640 1 --2- 172.21.15.5:0/169871408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63e02f0 0x5610b63ea670 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00bc20 tx=0x7fd68c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01f9095501fb1b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd696b29640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd67c0a32c0 0x7fd67c0a5780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd688013600 tx=0x7fd688005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd684ff9640 1 -- 172.21.15.5:0/169871408 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c017620 con 0x5610b63e02f0 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd684ff9640 1 -- 172.21.15.5:0/169871408 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd68c017c70 con 0x5610b63e02f0 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.725+0000 7fd684ff9640 1 -- 172.21.15.5:0/169871408 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00fc50 con 0x5610b63e02f0 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.726+0000 7fd69a965740 1 -- 172.21.15.5:0/169871408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63e02f0 msgr2=0x5610b63ea670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.726+0000 7fd69a965740 1 --2- 172.21.15.5:0/169871408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63e02f0 0x5610b63ea670 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00bc20 tx=0x7fd68c00d710 comp rx=0 tx=0).stop 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.726+0000 7fd69a965740 1 -- 172.21.15.5:0/169871408 shutdown_connections 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.726+0000 7fd69a965740 1 --2- 172.21.15.5:0/169871408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 0x5610b63ecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.726+0000 7fd69a965740 1 --2- 172.21.15.5:0/169871408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63e02f0 0x5610b63ea670 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.726+0000 7fd69a965740 1 --2- 172.21.15.5:0/169871408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63df9d0 0x5610b63dfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.726+0000 7fd69a965740 1 -- 172.21.15.5:0/169871408 >> 172.21.15.5:0/169871408 conn(0x5610b63dbf60 msgr2=0x5610b63dc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:13.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.726+0000 7fd69a965740 1 -- 172.21.15.5:0/169871408 shutdown_connections 2024-09-19T06:03:13.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.726+0000 7fd69a965740 1 -- 172.21.15.5:0/169871408 wait complete. 2024-09-19T06:03:13.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd69a965740 1 -- start start 2024-09-19T06:03:13.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd69a965740 1 Processor -- start 2024-09-19T06:03:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd69a965740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63df9d0 0x5610b63ea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd69a965740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63e02f0 0x5610b63e7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd69a965740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63eabb0 0x5610b63e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd69a965740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5610b63ed780 con 0x5610b63eabb0 2024-09-19T06:03:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd69a965740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5610b6300f20 con 0x5610b63df9d0 2024-09-19T06:03:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd69a965740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5610b63ed900 con 0x5610b63e02f0 2024-09-19T06:03:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd69a965740 10 client.0 _ll_register_callbacks cb 0x5610b62ca030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd696b29640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63eabb0 0x5610b63e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd696b29640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63eabb0 0x5610b63e7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52204/0 (socket says 172.21.15.5:52204) 2024-09-19T06:03:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd696b29640 1 -- 172.21.15.5:0/2471098186 learned_addr learned my addr 172.21.15.5:0/2471098186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd695b27640 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63e02f0 0x5610b63e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd696328640 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63df9d0 0x5610b63ea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd696b29640 1 -- 172.21.15.5:0/2471098186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63e02f0 msgr2=0x5610b63e7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd696b29640 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63e02f0 0x5610b63e7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd696b29640 1 -- 172.21.15.5:0/2471098186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63df9d0 msgr2=0x5610b63ea3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd696b29640 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63df9d0 0x5610b63ea3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.727+0000 7fd696b29640 1 -- 172.21.15.5:0/2471098186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b60b72b0 con 0x5610b63eabb0 2024-09-19T06:03:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.728+0000 7fd695b27640 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63e02f0 0x5610b63e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.728+0000 7fd696b29640 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63eabb0 0x5610b63e7c50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd688010f60 tx=0x7fd688004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d20efdee645250d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.728+0000 7fd679ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.728+0000 7fd679ffb640 1 -- 172.21.15.5:0/2471098186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688024020 con 0x5610b63eabb0 2024-09-19T06:03:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.728+0000 7fd69a965740 1 -- 172.21.15.5:0/2471098186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68c017c70 con 0x5610b63eabb0 2024-09-19T06:03:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.728+0000 7fd679ffb640 1 -- 172.21.15.5:0/2471098186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd688020410 con 0x5610b63eabb0 2024-09-19T06:03:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.728+0000 7fd679ffb640 1 -- 172.21.15.5:0/2471098186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd68802f050 con 0x5610b63eabb0 2024-09-19T06:03:13.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.728+0000 7fd69a965740 10 client.15213 fetch_fsmap learned FSMap version 27 2024-09-19T06:03:13.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.728+0000 7fd69a965740 1 -- 172.21.15.5:0/2471098186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5610b63f8fe0 con 0x5610b63eabb0 2024-09-19T06:03:13.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.728+0000 7fd679ffb640 1 -- 172.21.15.5:0/2471098186 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd68c017c70 con 0x5610b63eabb0 2024-09-19T06:03:13.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.728+0000 7fd679ffb640 1 -- 172.21.15.5:0/2471098186 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd68800d4c0 con 0x5610b63eabb0 2024-09-19T06:03:13.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.729+0000 7fd69a965740 10 client.15213 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:03:13.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.729+0000 7fd69a965740 10 client.15213 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:03:13.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.729+0000 7fd69a965740 20 client.15213 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:13.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.729+0000 7fd69a965740 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5610b63fb6c0 0x5610b63fcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.729+0000 7fd69a965740 4 client.15213 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:03:13.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.729+0000 7fd69a965740 1 -- 172.21.15.5:0/2471098186 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610b6353600 con 0x5610b63fb6c0 2024-09-19T06:03:13.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.729+0000 7fd696328640 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5610b63fb6c0 0x5610b63fcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.729+0000 7fd679ffb640 10 client.15213 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:13.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.729+0000 7fd696328640 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5610b63fb6c0 0x5610b63fcb30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a6b99a712f24e1 server_cookie=50a1ffb834b06c7e in_seq=0 out_seq=0 2024-09-19T06:03:13.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd679ffb640 1 -- 172.21.15.5:0/2471098186 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610b6353600 con 0x5610b63fb6c0 2024-09-19T06:03:13.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd679ffb640 10 client.15213 handle_command_reply: tid=0 2024-09-19T06:03:13.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd69a965740 1 client.15213 shutdown 2024-09-19T06:03:13.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd69a965740 1 -- 172.21.15.5:0/2471098186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63eabb0 msgr2=0x5610b63e7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:13.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd69a965740 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63eabb0 0x5610b63e7c50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd688010f60 tx=0x7fd688004a40 comp rx=0 tx=0).stop 2024-09-19T06:03:13.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd69a965740 1 -- 172.21.15.5:0/2471098186 shutdown_connections 2024-09-19T06:03:13.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd69a965740 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5610b63fb6c0 0x5610b63fcb30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd69a965740 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63eabb0 0x5610b63e7c50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd69a965740 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63e02f0 0x5610b63e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd679ffb640 0 client.15213 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd69a965740 1 --2- 172.21.15.5:0/2471098186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63df9d0 0x5610b63ea3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd69a965740 1 -- 172.21.15.5:0/2471098186 >> 172.21.15.5:0/2471098186 conn(0x5610b63dbf60 msgr2=0x5610b63e1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd69a965740 1 -- 172.21.15.5:0/2471098186 shutdown_connections 2024-09-19T06:03:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.730+0000 7fd69a965740 1 -- 172.21.15.5:0/2471098186 wait complete. 2024-09-19T06:03:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.731+0000 7fd69a965740 20 client.15213 trim_cache size 0 max 16384 2024-09-19T06:03:13.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.748+0000 7fd69a965740 1 Processor -- start 2024-09-19T06:03:13.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.748+0000 7fd69a965740 1 -- start start 2024-09-19T06:03:13.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.748+0000 7fd69a965740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 0x5610b63e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.748+0000 7fd69a965740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63e02f0 0x5610b63e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.748+0000 7fd69a965740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 0x5610b63e6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.748+0000 7fd69a965740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5610b63fae70 con 0x5610b63df9d0 2024-09-19T06:03:13.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.748+0000 7fd69a965740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5610b63fb540 con 0x5610b63e02f0 2024-09-19T06:03:13.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.748+0000 7fd69a965740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5610b63fbc10 con 0x5610b63eabb0 2024-09-19T06:03:13.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd696328640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 0x5610b63e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd696328640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 0x5610b63e76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52216/0 (socket says 172.21.15.5:52216) 2024-09-19T06:03:13.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd696328640 1 -- 172.21.15.5:0/3070080495 learned_addr learned my addr 172.21.15.5:0/3070080495 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:13.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd695b27640 1 --2- 172.21.15.5:0/3070080495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63e02f0 0x5610b63e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd696328640 1 -- 172.21.15.5:0/3070080495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 msgr2=0x5610b63e6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd696328640 1 --2- 172.21.15.5:0/3070080495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 0x5610b63e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd696328640 1 -- 172.21.15.5:0/3070080495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63e02f0 msgr2=0x5610b63e7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd696328640 1 --2- 172.21.15.5:0/3070080495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63e02f0 0x5610b63e7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd696328640 1 -- 172.21.15.5:0/3070080495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b63dbd90 con 0x5610b63df9d0 2024-09-19T06:03:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd695b27640 1 --2- 172.21.15.5:0/3070080495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63e02f0 0x5610b63e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:03:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd696328640 1 --2- 172.21.15.5:0/3070080495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 0x5610b63e76d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00dc10 tx=0x7fd68c00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b3f7a4bdc21e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd67bfff640 1 -- 172.21.15.5:0/3070080495 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00f940 con 0x5610b63df9d0 2024-09-19T06:03:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd67bfff640 1 -- 172.21.15.5:0/3070080495 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd68c005040 con 0x5610b63df9d0 2024-09-19T06:03:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.749+0000 7fd67bfff640 1 -- 172.21.15.5:0/3070080495 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0048c0 con 0x5610b63df9d0 2024-09-19T06:03:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 -- 172.21.15.5:0/3070080495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 msgr2=0x5610b63e76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 --2- 172.21.15.5:0/3070080495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 0x5610b63e76d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00dc10 tx=0x7fd68c00f810 comp rx=0 tx=0).stop 2024-09-19T06:03:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 -- 172.21.15.5:0/3070080495 shutdown_connections 2024-09-19T06:03:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 --2- 172.21.15.5:0/3070080495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 0x5610b63e6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 --2- 172.21.15.5:0/3070080495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63e02f0 0x5610b63e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 --2- 172.21.15.5:0/3070080495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 0x5610b63e76d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 -- 172.21.15.5:0/3070080495 >> 172.21.15.5:0/3070080495 conn(0x5610b63dbf60 msgr2=0x5610b63fa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 -- 172.21.15.5:0/3070080495 shutdown_connections 2024-09-19T06:03:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 -- 172.21.15.5:0/3070080495 wait complete. 2024-09-19T06:03:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 -- start start 2024-09-19T06:03:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 Processor -- start 2024-09-19T06:03:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 0x5610b63e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63e02f0 0x5610b63e4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 0x5610b6403df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5610b63fc9b0 con 0x5610b63df9d0 2024-09-19T06:03:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5610b63fc2e0 con 0x5610b63e02f0 2024-09-19T06:03:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.750+0000 7fd69a965740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5610b63fd080 con 0x5610b63eabb0 2024-09-19T06:03:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd69a965740 10 client.0 _ll_register_callbacks cb 0x7fd68800aee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd696328640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 0x5610b63e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd696328640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 0x5610b63e4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52230/0 (socket says 172.21.15.5:52230) 2024-09-19T06:03:13.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd696328640 1 -- 172.21.15.5:0/2314883127 learned_addr learned my addr 172.21.15.5:0/2314883127 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:13.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd695b27640 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63e02f0 0x5610b63e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd696328640 1 -- 172.21.15.5:0/2314883127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 msgr2=0x5610b6403df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:13.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd696328640 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 0x5610b6403df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd696328640 1 -- 172.21.15.5:0/2314883127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63e02f0 msgr2=0x5610b63e4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd696328640 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63e02f0 0x5610b63e4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd696328640 1 -- 172.21.15.5:0/2314883127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b64043b0 con 0x5610b63df9d0 2024-09-19T06:03:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd695b27640 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63e02f0 0x5610b63e4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd696328640 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 0x5610b63e4120 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd68c000c00 tx=0x7fd68c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c2c7ef149fe2cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd679ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd679ffb640 1 -- 172.21.15.5:0/2314883127 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c039020 con 0x5610b63df9d0 2024-09-19T06:03:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd679ffb640 1 -- 172.21.15.5:0/2314883127 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd68c00f880 con 0x5610b63df9d0 2024-09-19T06:03:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd69a965740 1 -- 172.21.15.5:0/2314883127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68c005040 con 0x5610b63df9d0 2024-09-19T06:03:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd679ffb640 1 -- 172.21.15.5:0/2314883127 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd68c00fa80 con 0x5610b63df9d0 2024-09-19T06:03:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd69a965740 10 client.15219 fetch_fsmap learned FSMap version 27 2024-09-19T06:03:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.751+0000 7fd69a965740 1 -- 172.21.15.5:0/2314883127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5610b6405600 con 0x5610b63df9d0 2024-09-19T06:03:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.752+0000 7fd679ffb640 1 -- 172.21.15.5:0/2314883127 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd68c005040 con 0x5610b63df9d0 2024-09-19T06:03:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.752+0000 7fd679ffb640 1 -- 172.21.15.5:0/2314883127 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd68c02b430 con 0x5610b63df9d0 2024-09-19T06:03:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.752+0000 7fd69a965740 10 client.15219 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:03:13.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.752+0000 7fd69a965740 10 client.15219 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:03:13.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.752+0000 7fd69a965740 20 client.15219 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.752+0000 7fd69a965740 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5610b63fb6c0 0x5610b6468a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.752+0000 7fd69a965740 4 client.15219 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:03:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.752+0000 7fd69a965740 1 -- 172.21.15.5:0/2314883127 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "status"}) -- 0x5610b634db10 con 0x5610b63fb6c0 2024-09-19T06:03:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.752+0000 7fd695b27640 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5610b63fb6c0 0x5610b6468a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd695b27640 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5610b63fb6c0 0x5610b6468a40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=898a53d88ae65eb5 server_cookie=3c05f0f264047e77 in_seq=0 out_seq=0 2024-09-19T06:03:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd679ffb640 10 client.15219 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd679ffb640 1 -- 172.21.15.5:0/2314883127 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5610b634db10 con 0x5610b63fb6c0 2024-09-19T06:03:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd679ffb640 10 client.15219 handle_command_reply: tid=0 2024-09-19T06:03:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd69a965740 1 client.15219 shutdown 2024-09-19T06:03:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd69a965740 1 -- 172.21.15.5:0/2314883127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 msgr2=0x5610b63e4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd69a965740 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 0x5610b63e4120 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd68c000c00 tx=0x7fd68c005e50 comp rx=0 tx=0).stop 2024-09-19T06:03:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd69a965740 1 -- 172.21.15.5:0/2314883127 shutdown_connections 2024-09-19T06:03:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd69a965740 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5610b63fb6c0 0x5610b6468a40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd69a965740 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610b63eabb0 0x5610b6403df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd69a965740 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5610b63e02f0 0x5610b63e4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd69a965740 1 --2- 172.21.15.5:0/2314883127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610b63df9d0 0x5610b63e4120 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd679ffb640 0 client.15219 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd69a965740 1 -- 172.21.15.5:0/2314883127 >> 172.21.15.5:0/2314883127 conn(0x5610b63dbf60 msgr2=0x5610b63e76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd69a965740 1 -- 172.21.15.5:0/2314883127 shutdown_connections 2024-09-19T06:03:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd69a965740 1 -- 172.21.15.5:0/2314883127 wait complete. 2024-09-19T06:03:13.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.753+0000 7fd69a965740 20 client.15219 trim_cache size 0 max 16384 2024-09-19T06:03:13.772 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:03:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "6b5d55e8-764b-11ef-bceb-c7b262605968", 2024-09-19T06:03:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-19T06:03:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24473, 2024-09-19T06:03:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-19T06:03:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-19T06:03:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-19T06:03:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 103.968143, 2024-09-19T06:03:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-19T06:03:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-19T06:03:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:03:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 114.74851918500001 2024-09-19T06:03:13.773 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:03:13.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.754+0000 7fd69858c640 1 -- 172.21.15.5:0/136802024 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd67c0a32c0 msgr2=0x7fd67c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.754+0000 7fd69858c640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd67c0a32c0 0x7fd67c0a5780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd688013600 tx=0x7fd688005c60 comp rx=0 tx=0).stop 2024-09-19T06:03:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.754+0000 7fd69858c640 1 -- 172.21.15.5:0/136802024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690102790 msgr2=0x7fd690198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.754+0000 7fd69858c640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690102790 0x7fd690198850 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd680005ab0 tx=0x7fd680003d50 comp rx=0 tx=0).stop 2024-09-19T06:03:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.754+0000 7fd69858c640 1 -- 172.21.15.5:0/136802024 shutdown_connections 2024-09-19T06:03:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.754+0000 7fd69858c640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7fd67c0a32c0 0x7fd67c0a5780 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.754+0000 7fd69858c640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd690106f10 0x7fd69019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.754+0000 7fd69858c640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690102790 0x7fd690198850 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.754+0000 7fd69858c640 1 --2- 172.21.15.5:0/136802024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690101de0 0x7fd690198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.754+0000 7fd69858c640 1 -- 172.21.15.5:0/136802024 >> 172.21.15.5:0/136802024 conn(0x7fd6900fd8c0 msgr2=0x7fd6900fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.754+0000 7fd69858c640 1 -- 172.21.15.5:0/136802024 shutdown_connections 2024-09-19T06:03:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:13.754+0000 7fd69858c640 1 -- 172.21.15.5:0/136802024 wait complete. 2024-09-19T06:03:14.500 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T06:03:14.510 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T06:03:14.510 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T06:03:14.510 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T06:03:14.510 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-19T06:03:14.511 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T06:03:14.511 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-19T06:03:14.511 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-19T06:03:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.615+0000 7fafb6b6f640 1 -- 172.21.15.5:0/582806513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb01078b0 msgr2=0x7fafb0109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.615+0000 7fafb6b6f640 1 --2- 172.21.15.5:0/582806513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb01078b0 0x7fafb0109cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009ca0 tx=0x7fafa802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:03:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.615+0000 7fafb6b6f640 1 -- 172.21.15.5:0/582806513 shutdown_connections 2024-09-19T06:03:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.615+0000 7fafb6b6f640 1 --2- 172.21.15.5:0/582806513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb01078b0 0x7fafb0109cc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.615+0000 7fafb6b6f640 1 --2- 172.21.15.5:0/582806513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafb0104f80 0x7fafb0107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.615+0000 7fafb6b6f640 1 --2- 172.21.15.5:0/582806513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb0102650 0x7fafb0104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.615+0000 7fafb6b6f640 1 -- 172.21.15.5:0/582806513 >> 172.21.15.5:0/582806513 conn(0x7fafb00fc1f0 msgr2=0x7fafb00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.615+0000 7fafb6b6f640 1 -- 172.21.15.5:0/582806513 shutdown_connections 2024-09-19T06:03:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.615+0000 7fafb6b6f640 1 -- 172.21.15.5:0/582806513 wait complete. 2024-09-19T06:03:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb6b6f640 1 Processor -- start 2024-09-19T06:03:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb6b6f640 1 -- start start 2024-09-19T06:03:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb6b6f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafb0102650 0x7fafb0197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb6b6f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb0104f80 0x7fafb0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb6b6f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb01078b0 0x7fafb019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb6b6f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010d6c0 con 0x7fafb01078b0 2024-09-19T06:03:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb6b6f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010d540 con 0x7fafb0104f80 2024-09-19T06:03:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb6b6f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010d840 con 0x7fafb0102650 2024-09-19T06:03:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb510c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb01078b0 0x7fafb019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb510c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb01078b0 0x7fafb019f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56120/0 (socket says 172.21.15.5:56120) 2024-09-19T06:03:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb510c640 1 -- 172.21.15.5:0/3695970283 learned_addr learned my addr 172.21.15.5:0/3695970283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb490b640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafb0102650 0x7fafb0197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafaffff640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb0104f80 0x7fafb0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb510c640 1 -- 172.21.15.5:0/3695970283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafb0102650 msgr2=0x7fafb0197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb510c640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafb0102650 0x7fafb0197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.616+0000 7fafb510c640 1 -- 172.21.15.5:0/3695970283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb0104f80 msgr2=0x7fafb0198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.617+0000 7fafb510c640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb0104f80 0x7fafb0198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.617+0000 7fafb510c640 1 -- 172.21.15.5:0/3695970283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb019fa50 con 0x7fafb01078b0 2024-09-19T06:03:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.617+0000 7fafaffff640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb0104f80 0x7fafb0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:03:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.617+0000 7fafb510c640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb01078b0 0x7fafb019f3e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fafa8003ba0 tx=0x7fafa8003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.617+0000 7fafadffb640 1 -- 172.21.15.5:0/3695970283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa80317e0 con 0x7fafb01078b0 2024-09-19T06:03:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.617+0000 7fafadffb640 1 -- 172.21.15.5:0/3695970283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafa80044f0 con 0x7fafb01078b0 2024-09-19T06:03:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.617+0000 7fafadffb640 1 -- 172.21.15.5:0/3695970283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8030c40 con 0x7fafb01078b0 2024-09-19T06:03:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.617+0000 7fafb6b6f640 1 -- 172.21.15.5:0/3695970283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb019fce0 con 0x7fafb01078b0 2024-09-19T06:03:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.617+0000 7fafb6b6f640 1 -- 172.21.15.5:0/3695970283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb01a01d0 con 0x7fafb01078b0 2024-09-19T06:03:14.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.618+0000 7fafb6b6f640 1 -- 172.21.15.5:0/3695970283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb00fdce0 con 0x7fafb01078b0 2024-09-19T06:03:14.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.619+0000 7fafadffb640 1 -- 172.21.15.5:0/3695970283 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fafa8004040 con 0x7fafb01078b0 2024-09-19T06:03:14.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.620+0000 7fafadffb640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7faf9c0a3240 0x7faf9c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:14.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.620+0000 7fafadffb640 1 -- 172.21.15.5:0/3695970283 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fafa80f0f30 con 0x7fafb01078b0 2024-09-19T06:03:14.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.620+0000 7fafb490b640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7faf9c0a3240 0x7faf9c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:14.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.620+0000 7fafb490b640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7faf9c0a3240 0x7faf9c0a5700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fafa0005d00 tx=0x7fafa000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:14.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.624+0000 7fafadffb640 1 -- 172.21.15.5:0/3695970283 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fafa80bdbe0 con 0x7fafb01078b0 2024-09-19T06:03:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:14 smithi049 ceph-mon[34640]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T06:03:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:14 smithi157 ceph-mon[35346]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T06:03:14.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:14.848+0000 7fafb6b6f640 1 -- 172.21.15.5:0/3695970283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fafb0062e90 con 0x7fafb01078b0 2024-09-19T06:03:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:14 smithi005 ceph-mon[30955]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T06:03:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.394+0000 7fafadffb640 1 -- 172.21.15.5:0/3695970283 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fafa80bd4e0 con 0x7fafb01078b0 2024-09-19T06:03:15.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.397+0000 7fafb6b6f640 1 -- 172.21.15.5:0/3695970283 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7faf9c0a3240 msgr2=0x7faf9c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.397+0000 7fafb6b6f640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7faf9c0a3240 0x7faf9c0a5700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fafa0005d00 tx=0x7fafa000b410 comp rx=0 tx=0).stop 2024-09-19T06:03:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.397+0000 7fafb6b6f640 1 -- 172.21.15.5:0/3695970283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb01078b0 msgr2=0x7fafb019f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.397+0000 7fafb6b6f640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb01078b0 0x7fafb019f3e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fafa8003ba0 tx=0x7fafa8003bd0 comp rx=0 tx=0).stop 2024-09-19T06:03:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.399+0000 7fafb6b6f640 1 -- 172.21.15.5:0/3695970283 shutdown_connections 2024-09-19T06:03:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.399+0000 7fafb6b6f640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7faf9c0a3240 0x7faf9c0a5700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.399+0000 7fafb6b6f640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb01078b0 0x7fafb019f3e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.399+0000 7fafb6b6f640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb0104f80 0x7fafb0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.399+0000 7fafb6b6f640 1 --2- 172.21.15.5:0/3695970283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafb0102650 0x7fafb0197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.399+0000 7fafb6b6f640 1 -- 172.21.15.5:0/3695970283 >> 172.21.15.5:0/3695970283 conn(0x7fafb00fc1f0 msgr2=0x7fafb0103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.400+0000 7fafb6b6f640 1 -- 172.21.15.5:0/3695970283 shutdown_connections 2024-09-19T06:03:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:15.400+0000 7fafb6b6f640 1 -- 172.21.15.5:0/3695970283 wait complete. 2024-09-19T06:03:15.418 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-19T06:03:15.447 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-19T06:03:15.447 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-19T06:03:15.447 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-19T06:03:15.447 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T06:03:15.447 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T06:03:15.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:15.448 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-19T06:03:15.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:15.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-09-19T06:03:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-09-19T06:03:15.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:03:15.538 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-19T06:03:15.538 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T06:03:15.538 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T06:03:15.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:15 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3695970283' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T06:03:15.646 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: ignoring --setuser ceph since I am not root 2024-09-19T06:03:15.647 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: ignoring --setgroup ceph since I am not root 2024-09-19T06:03:15.647 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:15 smithi049 ceph-mgr[36003]: -- 172.21.15.49:0/2338976969 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac5eb790e0 con 0x55ac5eb56800 2024-09-19T06:03:15.647 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:15.551+0000 7f5590626040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:03:15.647 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:15.611+0000 7f5590626040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:03:15.755 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:15 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3695970283' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T06:03:15.755 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: ignoring --setuser ceph since I am not root 2024-09-19T06:03:15.756 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: ignoring --setgroup ceph since I am not root 2024-09-19T06:03:15.756 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:15 smithi157 ceph-mgr[37149]: -- 172.21.15.157:0/1359192671 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b0dffbb0e0 con 0x55b0dff99000 2024-09-19T06:03:15.756 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:15.548+0000 7fbdb8631040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:03:15.756 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:15.605+0000 7fbdb8631040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:03:15.756 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:15.752+0000 7fbdb8631040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:03:15.774 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:15 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3695970283' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T06:03:15.774 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: ignoring --setuser ceph since I am not root 2024-09-19T06:03:15.775 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: ignoring --setgroup ceph since I am not root 2024-09-19T06:03:15.775 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:15.551+0000 7f3fd61f6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:03:15.775 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:15.612+0000 7f3fd61f6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:03:15.775 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:15.771+0000 7f3fd61f6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:03:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:16.051 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:15.820+0000 7fbdb8631040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:03:16.051 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:15.970+0000 7fbdb8631040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:03:16.091 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:15.854+0000 7f3fd61f6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:03:16.091 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:16.018+0000 7f3fd61f6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:03:16.141 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:15.767+0000 7f5590626040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:03:16.141 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:15.836+0000 7f5590626040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:03:16.141 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:15.991+0000 7f5590626040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:03:16.142 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:16 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:16.139+0000 7f5590626040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:03:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.278+0000 7f8f4fa79640 1 -- 172.21.15.5:0/1283832679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f481082a0 msgr2=0x7f8f4810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.278+0000 7f8f4fa79640 1 --2- 172.21.15.5:0/1283832679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f481082a0 0x7f8f4810ec40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009cd0 tx=0x7f8f4002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:03:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.278+0000 7f8f4fa79640 1 -- 172.21.15.5:0/1283832679 shutdown_connections 2024-09-19T06:03:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.278+0000 7f8f4fa79640 1 --2- 172.21.15.5:0/1283832679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f481082a0 0x7f8f4810ec40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.278+0000 7f8f4fa79640 1 --2- 172.21.15.5:0/1283832679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f481036e0 0x7f8f48107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.278+0000 7f8f4fa79640 1 --2- 172.21.15.5:0/1283832679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f48102d30 0x7f8f48103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.278+0000 7f8f4fa79640 1 -- 172.21.15.5:0/1283832679 >> 172.21.15.5:0/1283832679 conn(0x7f8f480fc7d0 msgr2=0x7f8f480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.279+0000 7f8f4fa79640 1 -- 172.21.15.5:0/1283832679 shutdown_connections 2024-09-19T06:03:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.279+0000 7f8f4fa79640 1 -- 172.21.15.5:0/1283832679 wait complete. 2024-09-19T06:03:16.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.279+0000 7f8f4fa79640 1 Processor -- start 2024-09-19T06:03:16.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.279+0000 7f8f4fa79640 1 -- start start 2024-09-19T06:03:16.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4fa79640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f48102d30 0x7f8f48198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:16.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4fa79640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f481036e0 0x7f8f481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:16.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4fa79640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f481082a0 0x7f8f4819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:16.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4fa79640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48111f50 con 0x7f8f481082a0 2024-09-19T06:03:16.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4fa79640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48111dd0 con 0x7f8f481036e0 2024-09-19T06:03:16.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4fa79640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481120d0 con 0x7f8f48102d30 2024-09-19T06:03:16.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4d014640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f481036e0 0x7f8f481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:16.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4e016640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f481082a0 0x7f8f4819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:16.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4d815640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f48102d30 0x7f8f48198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:16.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4d014640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f481036e0 0x7f8f481989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58406/0 (socket says 172.21.15.5:58406) 2024-09-19T06:03:16.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4e016640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f481082a0 0x7f8f4819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56140/0 (socket says 172.21.15.5:56140) 2024-09-19T06:03:16.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4d014640 1 -- 172.21.15.5:0/2345036099 learned_addr learned my addr 172.21.15.5:0/2345036099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:16.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4d014640 1 -- 172.21.15.5:0/2345036099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f48102d30 msgr2=0x7f8f48198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:16.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4d014640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f48102d30 0x7f8f48198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:16.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4d014640 1 -- 172.21.15.5:0/2345036099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f481082a0 msgr2=0x7f8f4819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:16.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4d014640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f481082a0 0x7f8f4819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:16.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4d014640 1 -- 172.21.15.5:0/2345036099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4819d420 con 0x7f8f481036e0 2024-09-19T06:03:16.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4d815640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f48102d30 0x7f8f48198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:16.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.280+0000 7f8f4d014640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f481036e0 0x7f8f481989b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f38010fd0 tx=0x7f8f3800b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:16.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.281+0000 7f8f3effd640 1 -- 172.21.15.5:0/2345036099 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38004020 con 0x7f8f481036e0 2024-09-19T06:03:16.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.281+0000 7f8f3effd640 1 -- 172.21.15.5:0/2345036099 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f380041c0 con 0x7f8f481036e0 2024-09-19T06:03:16.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.281+0000 7f8f3effd640 1 -- 172.21.15.5:0/2345036099 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38019670 con 0x7f8f481036e0 2024-09-19T06:03:16.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.281+0000 7f8f4fa79640 1 -- 172.21.15.5:0/2345036099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4819d710 con 0x7f8f481036e0 2024-09-19T06:03:16.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.281+0000 7f8f4fa79640 1 -- 172.21.15.5:0/2345036099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f481008c0 con 0x7f8f481036e0 2024-09-19T06:03:16.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.282+0000 7f8f4fa79640 1 -- 172.21.15.5:0/2345036099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f48103110 con 0x7f8f481036e0 2024-09-19T06:03:16.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.283+0000 7f8f3effd640 1 -- 172.21.15.5:0/2345036099 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) ==== 138371+0+0 (secure 0 0 0) 0x7f8f380035f0 con 0x7f8f481036e0 2024-09-19T06:03:16.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.284+0000 7f8f3effd640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f340a3290 0x7f8f340a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:16.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.284+0000 7f8f4d815640 1 -- 172.21.15.5:0/2345036099 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f340a3290 msgr2=0x7f8f340a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/709970884 2024-09-19T06:03:16.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.284+0000 7f8f3effd640 1 -- 172.21.15.5:0/2345036099 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8f380d3310 con 0x7f8f481036e0 2024-09-19T06:03:16.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.284+0000 7f8f4d815640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f340a3290 0x7f8f340a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:03:16.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.288+0000 7f8f3effd640 1 -- 172.21.15.5:0/2345036099 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8f3809f9b0 con 0x7f8f481036e0 2024-09-19T06:03:16.379 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:16.170+0000 7f3fd61f6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:03:16.397 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:16 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:16.114+0000 7fbdb8631040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:03:16.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.485+0000 7f8f4d815640 1 -- 172.21.15.5:0/2345036099 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f340a3290 msgr2=0x7f8f340a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/709970884 2024-09-19T06:03:16.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.485+0000 7f8f4d815640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f340a3290 0x7f8f340a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:03:16.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.514+0000 7f8f4fa79640 1 -- 172.21.15.5:0/2345036099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8f4810a2b0 con 0x7f8f481036e0 2024-09-19T06:03:16.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.514+0000 7f8f3effd640 1 -- 172.21.15.5:0/2345036099 <== mon.2 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f8f380a3cb0 con 0x7f8f481036e0 2024-09-19T06:03:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T06:03:16.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.516+0000 7f8f4fa79640 1 -- 172.21.15.5:0/2345036099 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f340a3290 msgr2=0x7f8f340a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:03:16.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.517+0000 7f8f4fa79640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f340a3290 0x7f8f340a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:16.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.517+0000 7f8f4fa79640 1 -- 172.21.15.5:0/2345036099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f481036e0 msgr2=0x7f8f481989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:16.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.517+0000 7f8f4fa79640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f481036e0 0x7f8f481989b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f38010fd0 tx=0x7f8f3800b690 comp rx=0 tx=0).stop 2024-09-19T06:03:16.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.517+0000 7f8f4fa79640 1 -- 172.21.15.5:0/2345036099 shutdown_connections 2024-09-19T06:03:16.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.517+0000 7f8f4fa79640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f8f340a3290 0x7f8f340a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:16.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.517+0000 7f8f4fa79640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f481082a0 0x7f8f4819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:16.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.517+0000 7f8f4fa79640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f481036e0 0x7f8f481989b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:16.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.517+0000 7f8f4fa79640 1 --2- 172.21.15.5:0/2345036099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f48102d30 0x7f8f48198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:16.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.517+0000 7f8f4fa79640 1 -- 172.21.15.5:0/2345036099 >> 172.21.15.5:0/2345036099 conn(0x7f8f480fc7d0 msgr2=0x7f8f48105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:16.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.517+0000 7f8f4fa79640 1 -- 172.21.15.5:0/2345036099 shutdown_connections 2024-09-19T06:03:16.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:16.517+0000 7f8f4fa79640 1 -- 172.21.15.5:0/2345036099 wait complete. 2024-09-19T06:03:16.677 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:16 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3695970283' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T06:03:16.677 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:16 smithi157 ceph-mon[35346]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T06:03:16.677 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:16 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:16.431+0000 7fbdb8631040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:03:16.678 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:16 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:16.491+0000 7fbdb8631040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:03:16.727 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:16 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3695970283' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T06:03:16.728 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:16 smithi049 ceph-mon[34640]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T06:03:16.728 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:16 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:16.468+0000 7f5590626040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:03:16.728 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:16 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:16.532+0000 7f5590626040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:03:16.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:16 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3695970283' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T06:03:16.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:16 smithi005 ceph-mon[30955]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T06:03:16.809 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:16.545+0000 7f3fd61f6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:03:16.809 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:16.611+0000 7f3fd61f6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:03:17.129 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:16.806+0000 7f3fd61f6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:03:17.146 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:16 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:16.725+0000 7f5590626040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:03:17.301 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T06:03:17.301 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T06:03:17.301 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-09-19T06:03:17.301 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T06:03:17.301 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T06:03:17.301 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T06:03:17.302 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T06:03:17.302 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T06:03:17.302 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T06:03:17.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:17.302 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-09-19T06:03:17.319 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T06:03:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T06:03:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T06:03:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:03:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-09-19T06:03:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:03:17.319 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:03:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-09-19T06:03:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-09-19T06:03:17.320 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:03:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-09-19T06:03:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-09-19T06:03:17.320 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T06:03:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-09-19T06:03:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-09-19T06:03:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T06:03:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42154sec preferred_lft 42154sec 2024-09-19T06:03:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-09-19T06:03:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:03:17.321 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:03:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-09-19T06:03:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-09-19T06:03:17.321 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T06:03:17.321 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:03:17.321 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-19T06:03:17.322 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-09-19T06:03:17.322 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-09-19T06:03:17.322 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-09-19T06:03:17.322 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T06:03:17.322 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-19T06:03:17.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:03:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:03:17.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:17.521 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T06:03:17.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:17 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2345036099' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:03:17.548 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-09-19T06:03:17.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:17.549 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-09-19T06:03:17.628 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-09-19T06:03:17.628 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T06:03:17.629 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T06:03:17.629 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T06:03:17.629 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T06:03:17.629 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:03:17.630 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-19T06:03:17.630 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T06:03:17.630 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T06:03:17.630 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T06:03:17.630 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-19T06:03:17.655 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:17 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2345036099' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:03:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:03:17.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:17 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2345036099' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:03:17.801 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:16 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:16.674+0000 7fbdb8631040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:03:17.804 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:17 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:17.556+0000 7fbdb8631040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:03:17.804 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:17 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:17.625+0000 7fbdb8631040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:03:17.830 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:17 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:17.756+0000 7f3fd61f6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:03:17.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:03:17.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:17.953 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-19T06:03:17.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:17.970 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-09-19T06:03:18.026 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:03:18.026 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-19T06:03:18.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:03:18.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T06:03:18.026 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-19T06:03:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:03:18.126 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:17 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:17.829+0000 7f3fd61f6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:03:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:03:18.146 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:17 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:17.654+0000 7f5590626040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:03:18.146 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:17 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:17.725+0000 7f5590626040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:03:18.150 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T06:03:18.150 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:03:18.150 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-19T06:03:18.150 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T06:03:18.150 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T06:03:18.150 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T06:03:18.150 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T06:03:18.150 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T06:03:18.150 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-19T06:03:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:03:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:03:18.288 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:03:18.289 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-19T06:03:18.289 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-09-19T06:03:18.289 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T06:03:18.289 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-19T06:03:18.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:03:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:03:18.369 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-19T06:03:18.369 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T06:03:18.369 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T06:03:18.386 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T06:03:18.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T06:03:18.386 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T06:03:18.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-09-19T06:03:18.509 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T06:03:18.566 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-09-19T06:03:18.566 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-09-19T06:03:18.566 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-09-19T06:03:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/386cfa8e47d4ca686bcd06e7a9c5b57017436b8f1787919e4f7219fb56720f80/merged 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/ccec1fe4ed9c69349e7aafb4247ea9bb9ef25f13a137111c3df397b6e452a284/merged 2024-09-19T06:03:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/0 2024-09-19T06:03:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:03:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/b39139dbe249b6b9404158c822985d60c816511f470f7beb741e97d1a2c727de/merged 2024-09-19T06:03:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/3a32a92d2262537784006d327754ba04ef72400559e3a34243afb923c54d5551/merged 2024-09-19T06:03:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/cd1fb13b3f745a597aba93b9e7ff8183182704a26167371891993139a9ea36ec/merged 2024-09-19T06:03:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/ba99b210ba3ef39300df59d85180c75862d859b97125df5883efb5e7b18ac4c3/merged 2024-09-19T06:03:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/4c19f043b8605929de5736d2a4fb9f7dc9e37a3ba8ac756dc13ceac1560e92bd/merged 2024-09-19T06:03:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/f093e6a87941114c654851a315e6d19533eaed5dee42d39dca0902fef6a15937/merged 2024-09-19T06:03:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/e8d24dee5aa6f551769d26eb64c02c5034f7b2faf49ed20653b66e70b2a406ad/merged 2024-09-19T06:03:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/6cbb09a240ac90a529bfc4bf5ab963b6bc1642f893105ee8f5aa644a1b241663/merged 2024-09-19T06:03:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-09-19T06:03:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:03:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:03:18.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:18.570 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-09-19T06:03:18.606 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:18 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:18.342+0000 7fbdb8631040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:03:18.606 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:18 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:18.460+0000 7fbdb8631040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:03:18.622 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T06:03:18.623 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9 --client_fs=cephfs) 2024-09-19T06:03:18.665 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-09-19T06:03:18.691 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T06:03:18.748 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-09-19T06:03:18.748 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-09-19T06:03:18.748 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-09-19T06:03:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-09-19T06:03:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-09-19T06:03:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-09-19T06:03:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-09-19T06:03:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-09-19T06:03:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-09-19T06:03:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-09-19T06:03:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-09-19T06:03:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-09-19T06:03:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:03:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-09-19T06:03:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-09-19T06:03:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-09-19T06:03:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-09-19T06:03:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-09-19T06:03:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-09-19T06:03:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T06:03:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T06:03:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T06:03:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T06:03:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-09-19T06:03:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-09-19T06:03:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay 2024-09-19T06:03:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/386cfa8e47d4ca686bcd06e7a9c5b57017436b8f1787919e4f7219fb56720f80/merged 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/ccec1fe4ed9c69349e7aafb4247ea9bb9ef25f13a137111c3df397b6e452a284/merged 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/0 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/b39139dbe249b6b9404158c822985d60c816511f470f7beb741e97d1a2c727de/merged 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/3a32a92d2262537784006d327754ba04ef72400559e3a34243afb923c54d5551/merged 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/cd1fb13b3f745a597aba93b9e7ff8183182704a26167371891993139a9ea36ec/merged 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/ba99b210ba3ef39300df59d85180c75862d859b97125df5883efb5e7b18ac4c3/merged 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/4c19f043b8605929de5736d2a4fb9f7dc9e37a3ba8ac756dc13ceac1560e92bd/merged 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/f093e6a87941114c654851a315e6d19533eaed5dee42d39dca0902fef6a15937/merged 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/e8d24dee5aa6f551769d26eb64c02c5034f7b2faf49ed20653b66e70b2a406ad/merged 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/6cbb09a240ac90a529bfc4bf5ab963b6bc1642f893105ee8f5aa644a1b241663/merged 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-09-19T06:03:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:03:18.752 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:03:18.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:18.752 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-09-19T06:03:18.752 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:18 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:18.478+0000 7f5590626040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:03:18.753 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:18 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:18.599+0000 7f5590626040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:03:18.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:2024-09-19T06:03:18.798+0000 7f9ad0845480 -1 init, newargv = 0x5637edf9eac0 newargc=15 2024-09-19T06:03:18.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[80522]: starting ceph client 2024-09-19T06:03:18.821 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[80522]: starting fuse 2024-09-19T06:03:18.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:18.606+0000 7f3fd61f6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:03:18.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:18.730+0000 7f3fd61f6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:03:18.891 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:18 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:18.603+0000 7fbdb8631040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:03:18.891 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:18 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:18.664+0000 7fbdb8631040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:03:18.891 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:18 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:18.817+0000 7fbdb8631040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:03:18.891 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:18 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:18.888+0000 7fbdb8631040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:03:19.048 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:18 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:18.748+0000 7f5590626040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:03:19.048 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:18 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:18.815+0000 7f5590626040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:03:19.048 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:18 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:18.973+0000 7f5590626040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:03:19.048 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:19.046+0000 7f5590626040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:03:19.177 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:18 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:18.949+0000 7fbdb8631040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:03:19.177 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:19.111+0000 7fbdb8631040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:03:19.177 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:19.176+0000 7fbdb8631040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:03:19.184 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:18.889+0000 7f3fd61f6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:03:19.184 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:18.953+0000 7f3fd61f6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:03:19.185 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:19.111+0000 7f3fd61f6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:03:19.185 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:19.182+0000 7f3fd61f6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:03:19.345 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:19.109+0000 7f5590626040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:03:19.345 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:19.275+0000 7f5590626040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:03:19.445 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:19.242+0000 7fbdb8631040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:03:19.445 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:19.324+0000 7fbdb8631040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:03:19.445 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:19.384+0000 7fbdb8631040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:03:19.445 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:19.444+0000 7fbdb8631040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:03:19.482 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:19.246+0000 7f3fd61f6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:03:19.482 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:19.412+0000 7f3fd61f6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:03:19.482 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:19.480+0000 7f3fd61f6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:03:19.626 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:19.343+0000 7f5590626040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:03:19.626 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:19.413+0000 7f5590626040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:03:19.626 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:19.499+0000 7f5590626040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:03:19.626 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:19.562+0000 7f5590626040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:03:19.626 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:19.624+0000 7f5590626040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:03:19.763 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:19.550+0000 7f3fd61f6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:03:19.764 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:19.637+0000 7f3fd61f6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:03:19.764 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:19.699+0000 7f3fd61f6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:03:19.764 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:19.761+0000 7f3fd61f6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:03:19.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-09-19T06:03:19.934 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T06:03:19.991 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-09-19T06:03:19.991 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-09-19T06:03:19.992 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-09-19T06:03:19.992 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-09-19T06:03:19.992 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-09-19T06:03:19.992 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-09-19T06:03:19.992 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-09-19T06:03:19.992 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-09-19T06:03:19.992 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-09-19T06:03:19.992 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-09-19T06:03:19.992 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-09-19T06:03:19.992 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-09-19T06:03:19.992 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:03:19.992 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay 2024-09-19T06:03:19.993 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/386cfa8e47d4ca686bcd06e7a9c5b57017436b8f1787919e4f7219fb56720f80/merged 2024-09-19T06:03:19.994 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/ccec1fe4ed9c69349e7aafb4247ea9bb9ef25f13a137111c3df397b6e452a284/merged 2024-09-19T06:03:19.994 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/0 2024-09-19T06:03:19.994 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T06:03:19.994 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/b39139dbe249b6b9404158c822985d60c816511f470f7beb741e97d1a2c727de/merged 2024-09-19T06:03:19.994 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/3a32a92d2262537784006d327754ba04ef72400559e3a34243afb923c54d5551/merged 2024-09-19T06:03:19.994 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/cd1fb13b3f745a597aba93b9e7ff8183182704a26167371891993139a9ea36ec/merged 2024-09-19T06:03:19.994 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/ba99b210ba3ef39300df59d85180c75862d859b97125df5883efb5e7b18ac4c3/merged 2024-09-19T06:03:19.994 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/4c19f043b8605929de5736d2a4fb9f7dc9e37a3ba8ac756dc13ceac1560e92bd/merged 2024-09-19T06:03:19.994 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/f093e6a87941114c654851a315e6d19533eaed5dee42d39dca0902fef6a15937/merged 2024-09-19T06:03:19.994 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/e8d24dee5aa6f551769d26eb64c02c5034f7b2faf49ed20653b66e70b2a406ad/merged 2024-09-19T06:03:19.995 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/6cbb09a240ac90a529bfc4bf5ab963b6bc1642f893105ee8f5aa644a1b241663/merged 2024-09-19T06:03:19.995 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-09-19T06:03:19.995 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:03:19.995 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:03:19.995 INFO:teuthology.orchestra.run.smithi005.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T06:03:19.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:19.995 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-09-19T06:03:20.049 INFO:teuthology.orchestra.run.smithi005.stdout:120 2024-09-19T06:03:20.049 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-19T06:03:20.049 DEBUG:teuthology.orchestra.run.smithi005:> sudo stdin-killer -- python3 -c ' 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> import re 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2024-09-19T06:03:20.050 DEBUG:teuthology.orchestra.run.smithi005:> 2024-09-19T06:03:20.051 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2024-09-19T06:03:20.051 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T06:03:20.052 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T06:03:20.052 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T06:03:20.052 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2024-09-19T06:03:20.052 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2024-09-19T06:03:20.052 DEBUG:teuthology.orchestra.run.smithi005:> return f 2024-09-19T06:03:20.052 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T06:03:20.052 DEBUG:teuthology.orchestra.run.smithi005:> 2024-09-19T06:03:20.052 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.0")) 2024-09-19T06:03:20.052 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-09-19T06:03:20.170 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.0.80522.asok 2024-09-19T06:03:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:03:20.181 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.80522.asok 2024-09-19T06:03:20.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:20.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.80522.asok status 2024-09-19T06:03:20.328 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:03:20.328 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2024-09-19T06:03:20.328 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T06:03:20.329 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T06:03:20.329 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "0", 2024-09-19T06:03:20.329 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2024-09-19T06:03:20.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T06:03:20.329 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "80522", 2024-09-19T06:03:20.329 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9" 2024-09-19T06:03:20.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:03:20.330 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2024-09-19T06:03:20.330 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2024-09-19T06:03:20.330 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 15255, 2024-09-19T06:03:20.330 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2024-09-19T06:03:20.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2024-09-19T06:03:20.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2024-09-19T06:03:20.330 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 15255 2024-09-19T06:03:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:03:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-09-19T06:03:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-19T06:03:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-09-19T06:03:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2875039701 2024-09-19T06:03:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:03:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:03:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-09-19T06:03:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-19T06:03:20.332 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-09-19T06:03:20.332 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2875039701 2024-09-19T06:03:20.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:03:20.332 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.15255 192.168.0.1:0/2875039701", 2024-09-19T06:03:20.332 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.1:0/2875039701", 2024-09-19T06:03:20.332 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 5, 2024-09-19T06:03:20.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 27, 2024-09-19T06:03:20.332 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 83, 2024-09-19T06:03:20.332 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2024-09-19T06:03:20.332 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2024-09-19T06:03:20.332 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2024-09-19T06:03:20.332 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:03:20.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:20.337 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2024-09-19T06:03:20.338 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T06:03:20.354 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2024-09-19T06:03:20.354 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T06:03:20.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:03:20.354 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T06:03:20.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-mon[30955]: Standby manager daemon z restarted 2024-09-19T06:03:20.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-mon[30955]: Standby manager daemon z started 2024-09-19T06:03:20.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.157:0/3832021913' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:03:20.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.157:0/3832021913' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:03:20.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.157:0/3832021913' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:03:20.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.157:0/3832021913' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:03:20.407 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T06:03:20.416 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T06:03:20.416 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-19T06:03:20.416 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-19T06:03:20.551 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:03:20 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z[37145]: 2024-09-19T06:03:20.235+0000 7fbdb8631040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:03:20.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:20 smithi157 ceph-mon[35346]: Standby manager daemon z restarted 2024-09-19T06:03:20.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:20 smithi157 ceph-mon[35346]: Standby manager daemon z started 2024-09-19T06:03:20.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:20 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.157:0/3832021913' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:03:20.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:20 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.157:0/3832021913' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:03:20.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:20 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.157:0/3832021913' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:03:20.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:20 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.157:0/3832021913' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:03:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.551+0000 7f3931f11640 1 -- 172.21.15.5:0/3029386452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c1025f0 msgr2=0x7f392c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.551+0000 7f3931f11640 1 --2- 172.21.15.5:0/3029386452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c1025f0 0x7f392c108e80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3924009ca0 tx=0x7f392402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:03:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.551+0000 7f3931f11640 1 -- 172.21.15.5:0/3029386452 shutdown_connections 2024-09-19T06:03:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.551+0000 7f3931f11640 1 --2- 172.21.15.5:0/3029386452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c1025f0 0x7f392c108e80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.551+0000 7f3931f11640 1 --2- 172.21.15.5:0/3029386452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c101c30 0x7f392c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.551+0000 7f3931f11640 1 --2- 172.21.15.5:0/3029386452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f392c100a40 0x7f392c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.551+0000 7f3931f11640 1 -- 172.21.15.5:0/3029386452 >> 172.21.15.5:0/3029386452 conn(0x7f392c0fc1c0 msgr2=0x7f392c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.552+0000 7f3931f11640 1 -- 172.21.15.5:0/3029386452 shutdown_connections 2024-09-19T06:03:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.552+0000 7f3931f11640 1 -- 172.21.15.5:0/3029386452 wait complete. 2024-09-19T06:03:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.552+0000 7f3931f11640 1 Processor -- start 2024-09-19T06:03:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.552+0000 7f3931f11640 1 -- start start 2024-09-19T06:03:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.552+0000 7f3931f11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f392c100a40 0x7f392c197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.552+0000 7f3931f11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c101c30 0x7f392c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f3931f11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c1025f0 0x7f392c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f3931f11640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f392c10c5a0 con 0x7f392c101c30 2024-09-19T06:03:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f3931f11640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f392c10c420 con 0x7f392c1025f0 2024-09-19T06:03:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f3931f11640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f392c10c720 con 0x7f392c100a40 2024-09-19T06:03:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f392affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c101c30 0x7f392c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f392affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c101c30 0x7f392c198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56158/0 (socket says 172.21.15.5:56158) 2024-09-19T06:03:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f392affd640 1 -- 172.21.15.5:0/3303375432 learned_addr learned my addr 172.21.15.5:0/3303375432 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f392b7fe640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f392c100a40 0x7f392c197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f392bfff640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c1025f0 0x7f392c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f392affd640 1 -- 172.21.15.5:0/3303375432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f392c100a40 msgr2=0x7f392c197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f392affd640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f392c100a40 0x7f392c197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f392affd640 1 -- 172.21.15.5:0/3303375432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c1025f0 msgr2=0x7f392c19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f392affd640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c1025f0 0x7f392c19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f392affd640 1 -- 172.21.15.5:0/3303375432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392c19f900 con 0x7f392c101c30 2024-09-19T06:03:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f392affd640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c101c30 0x7f392c198200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f392000c920 tx=0x7f392000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f3928ff9640 1 -- 172.21.15.5:0/3303375432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920007c70 con 0x7f392c101c30 2024-09-19T06:03:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f3931f11640 1 -- 172.21.15.5:0/3303375432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392c19fbf0 con 0x7f392c101c30 2024-09-19T06:03:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f3928ff9640 1 -- 172.21.15.5:0/3303375432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3920007e10 con 0x7f392c101c30 2024-09-19T06:03:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f3928ff9640 1 -- 172.21.15.5:0/3303375432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920018820 con 0x7f392c101c30 2024-09-19T06:03:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.553+0000 7f3931f11640 1 -- 172.21.15.5:0/3303375432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f392c1a0190 con 0x7f392c101c30 2024-09-19T06:03:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.554+0000 7f3931f11640 1 -- 172.21.15.5:0/3303375432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f392c1a03c0 con 0x7f392c101c30 2024-09-19T06:03:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.556+0000 7f3928ff9640 1 -- 172.21.15.5:0/3303375432 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 138371+0+0 (secure 0 0 0) 0x7f39200189c0 con 0x7f392c101c30 2024-09-19T06:03:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.556+0000 7f3928ff9640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f39140a3290 0x7f39140a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.556+0000 7f392b7fe640 1 -- 172.21.15.5:0/3303375432 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f39140a3290 msgr2=0x7f39140a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/709970884 2024-09-19T06:03:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.556+0000 7f3928ff9640 1 -- 172.21.15.5:0/3303375432 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f39200cfa00 con 0x7f392c101c30 2024-09-19T06:03:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.556+0000 7f392b7fe640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f39140a3290 0x7f39140a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:03:20.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.561+0000 7f3928ff9640 1 -- 172.21.15.5:0/3303375432 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3920014030 con 0x7f392c101c30 2024-09-19T06:03:20.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.633+0000 7f3928ff9640 1 -- 172.21.15.5:0/3303375432 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7f3920098680 con 0x7f392c101c30 2024-09-19T06:03:20.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.634+0000 7f3928ff9640 1 -- 172.21.15.5:0/3303375432 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f39140a3290 msgr2=0x7f39140a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:03:20.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.634+0000 7f3928ff9640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f39140a3290 0x7f39140a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:20.646 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:03:20 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y[35999]: 2024-09-19T06:03:20.460+0000 7f5590626040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:03:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:20 smithi049 ceph-mon[34640]: Standby manager daemon z restarted 2024-09-19T06:03:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:20 smithi049 ceph-mon[34640]: Standby manager daemon z started 2024-09-19T06:03:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:20 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.157:0/3832021913' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:03:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:20 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.157:0/3832021913' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:03:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:20 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.157:0/3832021913' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:03:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:20 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.157:0/3832021913' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:03:20.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:20.861+0000 7f3931f11640 1 -- 172.21.15.5:0/3303375432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f392c1a07b0 con 0x7f392c101c30 2024-09-19T06:03:20.879 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:20.620+0000 7f3fd61f6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:03:21.209 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:20.877+0000 7f3f40b4e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:03:21.209 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:20.877+0000 7f3f40b4e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:03:21.209 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:20.877+0000 7f3f40b4e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:03:21.209 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:20.877+0000 7f3f40b4e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:03:21.209 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:20.877+0000 7f3f40b4e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:03:21.209 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:20.877+0000 7f3f3db48640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:03:21.209 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:20.877+0000 7f3f3db48640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:03:21.209 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:20.877+0000 7f3f3db48640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:03:21.209 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:20.877+0000 7f3f3db48640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:03:21.209 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:20.877+0000 7f3f3db48640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:03:21.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.626+0000 7f3928ff9640 1 -- 172.21.15.5:0/3303375432 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f392009c710 con 0x7f392c101c30 2024-09-19T06:03:21.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-09-19T06:03:21.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: Standby manager daemon y restarted 2024-09-19T06:03:21.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: Standby manager daemon y started 2024-09-19T06:03:21.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.49:0/3724542622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:03:21.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.49:0/3724542622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:03:21.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.49:0/3724542622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:03:21.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.? 172.21.15.49:0/3724542622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:03:21.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: Active manager daemon x restarted 2024-09-19T06:03:21.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: Activating manager daemon x 2024-09-19T06:03:21.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T06:03:21.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: mgrmap e21: x(active, starting, since 0.0124673s), standbys: z, y 2024-09-19T06:03:21.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:03:21.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:03:21.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:03:21.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:03:21.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:03:21.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:03:21.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:03:21.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:03:21.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:03:21.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:03:21.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:03:21.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:03:21.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:03:21.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:03:21.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:03:21.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:03:21.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:03:21.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:03:21.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:03:21.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:03:21.634 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:03:21.634 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: Manager daemon x is now available 2024-09-19T06:03:21.634 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:21.634 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:03:21.634 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:03:21.634 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:21 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3303375432' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T06:03:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.629+0000 7f3931f11640 1 -- 172.21.15.5:0/3303375432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c101c30 msgr2=0x7f392c198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.629+0000 7f3931f11640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c101c30 0x7f392c198200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f392000c920 tx=0x7f392000cde0 comp rx=0 tx=0).stop 2024-09-19T06:03:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.629+0000 7f3931f11640 1 -- 172.21.15.5:0/3303375432 shutdown_connections 2024-09-19T06:03:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.629+0000 7f3931f11640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.5:6800/709970884,v1:172.21.15.5:6801/709970884] conn(0x7f39140a3290 0x7f39140a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.629+0000 7f3931f11640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c1025f0 0x7f392c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.629+0000 7f3931f11640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c101c30 0x7f392c198200 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.629+0000 7f3931f11640 1 --2- 172.21.15.5:0/3303375432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f392c100a40 0x7f392c197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.629+0000 7f3931f11640 1 -- 172.21.15.5:0/3303375432 >> 172.21.15.5:0/3303375432 conn(0x7f392c0fc1c0 msgr2=0x7f392c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.629+0000 7f3931f11640 1 -- 172.21.15.5:0/3303375432 shutdown_connections 2024-09-19T06:03:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.629+0000 7f3931f11640 1 -- 172.21.15.5:0/3303375432 wait complete. 2024-09-19T06:03:21.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: Standby manager daemon y restarted 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: Standby manager daemon y started 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.49:0/3724542622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.49:0/3724542622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.49:0/3724542622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.? 172.21.15.49:0/3724542622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: Active manager daemon x restarted 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: Activating manager daemon x 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: mgrmap e21: x(active, starting, since 0.0124673s), standbys: z, y 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:03:21.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:03:21.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:03:21.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:03:21.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:03:21.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:03:21.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:03:21.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:03:21.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:03:21.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:03:21.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:03:21.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:03:21.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:03:21.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:03:21.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:03:21.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:03:21.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:03:21.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:03:21.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:03:21.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:03:21.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:03:21.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:03:21.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:03:21.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:03:21.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:03:21.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:03:21.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:03:21.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:03:21.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:03:21.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:03:21.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:03:21.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: Manager daemon x is now available 2024-09-19T06:03:21.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:21.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:03:21.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:03:21.651 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:21 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3303375432' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T06:03:21.653 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T06:03:21.664 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T06:03:21.740 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T06:03:21.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-09-19T06:03:21.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: Standby manager daemon y restarted 2024-09-19T06:03:21.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: Standby manager daemon y started 2024-09-19T06:03:21.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.49:0/3724542622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:03:21.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.49:0/3724542622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:03:21.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.49:0/3724542622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:03:21.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.? 172.21.15.49:0/3724542622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:03:21.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: Active manager daemon x restarted 2024-09-19T06:03:21.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: Activating manager daemon x 2024-09-19T06:03:21.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T06:03:21.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: mgrmap e21: x(active, starting, since 0.0124673s), standbys: z, y 2024-09-19T06:03:21.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:03:21.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:03:21.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:03:21.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:03:21.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:03:21.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:03:21.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:03:21.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:03:21.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:03:21.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:03:21.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:03:21.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:03:21.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:03:21.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:03:21.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:03:21.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:03:21.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:03:21.763 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:03:21.763 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:03:21.763 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:03:21.763 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: Manager daemon x is now available 2024-09-19T06:03:21.763 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:21.763 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:03:21.763 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:03:21.763 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:21 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3303375432' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T06:03:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.846+0000 7f369214b640 1 -- 172.21.15.5:0/3677423535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c102630 msgr2=0x7f368c104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.846+0000 7f369214b640 1 --2- 172.21.15.5:0/3677423535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c102630 0x7f368c104a20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f367c009860 tx=0x7f367c02a340 comp rx=0 tx=0).stop 2024-09-19T06:03:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.846+0000 7f369214b640 1 -- 172.21.15.5:0/3677423535 shutdown_connections 2024-09-19T06:03:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.846+0000 7f369214b640 1 --2- 172.21.15.5:0/3677423535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f368c107890 0x7f368c109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.846+0000 7f369214b640 1 --2- 172.21.15.5:0/3677423535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c104f60 0x7f368c107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.846+0000 7f369214b640 1 --2- 172.21.15.5:0/3677423535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c102630 0x7f368c104a20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.846+0000 7f369214b640 1 -- 172.21.15.5:0/3677423535 >> 172.21.15.5:0/3677423535 conn(0x7f368c0fc1f0 msgr2=0x7f368c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.846+0000 7f369214b640 1 -- 172.21.15.5:0/3677423535 shutdown_connections 2024-09-19T06:03:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.846+0000 7f369214b640 1 -- 172.21.15.5:0/3677423535 wait complete. 2024-09-19T06:03:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.846+0000 7f369214b640 1 Processor -- start 2024-09-19T06:03:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f369214b640 1 -- start start 2024-09-19T06:03:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f369214b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c102630 0x7f368c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f369214b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c104f60 0x7f368c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f369214b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f368c107890 0x7f368c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f369214b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f368c10d6e0 con 0x7f368c102630 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f369214b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f368c10d560 con 0x7f368c104f60 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f369214b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f368c10d860 con 0x7f368c107890 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f368b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c102630 0x7f368c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f368affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c104f60 0x7f368c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f368b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c102630 0x7f368c197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56260/0 (socket says 172.21.15.5:56260) 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f368affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c104f60 0x7f368c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58542/0 (socket says 172.21.15.5:58542) 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f368b7fe640 1 -- 172.21.15.5:0/1502410725 learned_addr learned my addr 172.21.15.5:0/1502410725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f368bfff640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f368c107890 0x7f368c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f368affd640 1 -- 172.21.15.5:0/1502410725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f368c107890 msgr2=0x7f368c19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f368affd640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f368c107890 0x7f368c19f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f368affd640 1 -- 172.21.15.5:0/1502410725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c102630 msgr2=0x7f368c197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f368affd640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c102630 0x7f368c197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.847+0000 7f368affd640 1 -- 172.21.15.5:0/1502410725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c19fa20 con 0x7f368c104f60 2024-09-19T06:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.848+0000 7f368affd640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c104f60 0x7f368c198340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f368c069ff0 tx=0x7f368000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.848+0000 7f3688ff9640 1 -- 172.21.15.5:0/1502410725 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368000ccd0 con 0x7f368c104f60 2024-09-19T06:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.848+0000 7f3688ff9640 1 -- 172.21.15.5:0/1502410725 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36800035f0 con 0x7f368c104f60 2024-09-19T06:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.848+0000 7f369214b640 1 -- 172.21.15.5:0/1502410725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368c19fd10 con 0x7f368c104f60 2024-09-19T06:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.848+0000 7f3688ff9640 1 -- 172.21.15.5:0/1502410725 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680010690 con 0x7f368c104f60 2024-09-19T06:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.848+0000 7f369214b640 1 -- 172.21.15.5:0/1502410725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368c1a0250 con 0x7f368c104f60 2024-09-19T06:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.849+0000 7f369214b640 1 -- 172.21.15.5:0/1502410725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f368c0fdce0 con 0x7f368c104f60 2024-09-19T06:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.850+0000 7f3688ff9640 1 -- 172.21.15.5:0/1502410725 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f36800037a0 con 0x7f368c104f60 2024-09-19T06:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.851+0000 7f3688ff9640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36740a30d0 0x7f36740a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.851+0000 7f3688ff9640 1 -- 172.21.15.5:0/1502410725 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f36800cfb40 con 0x7f368c104f60 2024-09-19T06:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.851+0000 7f368b7fe640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36740a30d0 0x7f36740a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.852+0000 7f368b7fe640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36740a30d0 0x7f36740a5590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f367c02a930 tx=0x7f367c0058b0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:21.854+0000 7f3688ff9640 1 -- 172.21.15.5:0/1502410725 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f368009ca40 con 0x7f368c104f60 2024-09-19T06:03:22.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.035+0000 7f369214b640 1 -- 172.21.15.5:0/1502410725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f368c02d990 con 0x7f368c104f60 2024-09-19T06:03:22.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.037+0000 7f3688ff9640 1 -- 172.21.15.5:0/1502410725 <== mon.2 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+23072 (secure 0 0 0) 0x7f368009c340 con 0x7f368c104f60 2024-09-19T06:03:22.039 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:03:22.039 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":84,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","created":"2024-09-19T05:53:51.454068+0000","modified":"2024-09-19T06:03:20.622077+0000","last_up_change":"2024-09-19T06:00:38.837916+0000","last_in_change":"2024-09-19T06:00:15.609776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:56:55.213931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:01:25.648588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:01:27.760408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f3c10d60-d354-407f-9cdf-099344599844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6803","nonce":3442981810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6805","nonce":3442981810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6809","nonce":3442981810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6807","nonce":3442981810}]},"public_addr":"172.21.15.5:6803/3442981810","cluster_addr":"172.21.15.5:6805/3442981810","heartbeat_back_addr":"172.21.15.5:6809/3442981810","heartbeat_front_addr":"172.21.15.5:6807/3442981810","state":["exists","up"]},{"osd":1,"uuid":"5f40f636-7761-45ac-87f3-910a2fa2feb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448265014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6803","nonce":1448265014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6807","nonce":1448265014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6805","nonce":1448265014}]},"public_addr":"172.21.15.49:6801/1448265014","cluster_addr":"172.21.15.49:6803/1448265014","heartbeat_back_addr":"172.21.15.49:6807/1448265014","heartbeat_front_addr":"172.21.15.49:6805/1448265014","state":["exists","up"]},{"osd":2,"uuid":"ad00940c-4fdd-4f5f-9eaa-b1f38bcac526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6801","nonce":2319626622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6803","nonce":2319626622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6807","nonce":2319626622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6805","nonce":2319626622}]},"public_addr":"172.21.15.157:6801/2319626622","cluster_addr":"172.21.15.157:6803/2319626622","heartbeat_back_addr":"172.21.15.157:6807/2319626622","heartbeat_front_addr":"172.21.15.157:6805/2319626622","state":["exists","up"]},{"osd":3,"uuid":"63fec329-1b62-4c5e-b8ca-084a84d4d762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6811","nonce":1452491753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6813","nonce":1452491753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6817","nonce":1452491753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6815","nonce":1452491753}]},"public_addr":"172.21.15.5:6811/1452491753","cluster_addr":"172.21.15.5:6813/1452491753","heartbeat_back_addr":"172.21.15.5:6817/1452491753","heartbeat_front_addr":"172.21.15.5:6815/1452491753","state":["exists","up"]},{"osd":4,"uuid":"ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6809","nonce":3114952580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6811","nonce":3114952580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6815","nonce":3114952580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6813","nonce":3114952580}]},"public_addr":"172.21.15.49:6809/3114952580","cluster_addr":"172.21.15.49:6811/3114952580","heartbeat_back_addr":"172.21.15.49:6815/3114952580","heartbeat_front_addr":"172.21.15.49:6813/3114952580","state":["exists","up"]},{"osd":5,"uuid":"21a7a7d3-ce42-4688-bfec-714be635eaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6809","nonce":3219063971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6811","nonce":3219063971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6815","nonce":3219063971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6813","nonce":3219063971}]},"public_addr":"172.21.15.157:6809/3219063971","cluster_addr":"172.21.15.157:6811/3219063971","heartbeat_back_addr":"172.21.15.157:6815/3219063971","heartbeat_front_addr":"172.21.15.157:6813/3219063971","state":["exists","up"]},{"osd":6,"uuid":"cbef0e08-d6b0-4515-a2ee-2c81b048583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6819","nonce":2957795194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6821","nonce":2957795194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6825","nonce":2957795194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6823","nonce":2957795194}]},"public_addr":"172.21.15.5:6819/2957795194","cluster_addr":"172.21.15.5:6821/2957795194","heartbeat_back_addr":"172.21.15.5:6825/2957795194","heartbeat_front_addr":"172.21.15.5:6823/2957795194","state":["exists","up"]},{"osd":7,"uuid":"d261ac55-532d-4aea-9b3e-ed2b4eeb1da6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6817","nonce":2959731400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6819","nonce":2959731400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6823","nonce":2959731400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6821","nonce":2959731400}]},"public_addr":"172.21.15.49:6817/2959731400","cluster_addr":"172.21.15.49:6819/2959731400","heartbeat_back_addr":"172.21.15.49:6823/2959731400","heartbeat_front_addr":"172.21.15.49:6821/2959731400","state":["exists","up"]},{"osd":8,"uuid":"15d0eaf7-d604-4546-b393-4d6d39018f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6817","nonce":290026859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6819","nonce":290026859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6823","nonce":290026859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6821","nonce":290026859}]},"public_addr":"172.21.15.157:6817/290026859","cluster_addr":"172.21.15.157:6819/290026859","heartbeat_back_addr":"172.21.15.157:6823/290026859","heartbeat_front_addr":"172.21.15.157:6821/290026859","state":["exists","up"]},{"osd":9,"uuid":"30c08537-f593-4299-bfa1-fa7ba8a95169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6827","nonce":4122448237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6829","nonce":4122448237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6833","nonce":4122448237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6831","nonce":4122448237}]},"public_addr":"172.21.15.5:6827/4122448237","cluster_addr":"172.21.15.5:6829/4122448237","heartbeat_back_addr":"172.21.15.5:6833/4122448237","heartbeat_front_addr":"172.21.15.5:6831/4122448237","state":["exists","up"]},{"osd":10,"uuid":"68821e98-bd81-49ec-8cd6-51fb9880b553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6825","nonce":2551748002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6827","nonce":2551748002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6831","nonce":2551748002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6829","nonce":2551748002}]},"public_addr":"172.21.15.49:6825/2551748002","cluster_addr":"172.21.15.49:6827/2551748002","heartbeat_back_addr":"172.21.15.49:6831/2551748002","heartbeat_front_addr":"172.21.15.49:6829/2551748002","state":["exists","up"]},{"osd":11,"uuid":"7e443b46-2697-4d0b-8452-868463a1ee1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6825","nonce":1996582544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6827","nonce":1996582544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6831","nonce":1996582544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6829","nonce":1996582544}]},"public_addr":"172.21.15.157:6825/1996582544","cluster_addr":"172.21.15.157:6827/1996582544","heartbeat_back_addr":"172.21.15.157:6831/1996582544","heartbeat_front_addr":"172.21.15.157:6829/1996582544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:01.453592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:27.194705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:51.525385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:17.676749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:43.075555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:07.258154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:32.112583+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:57.306534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:21.320950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:47.164610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:12.548337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:37.262580+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3703679969":"2024-09-20T06:03:20.622034+0000","172.21.15.5:6801/709970884":"2024-09-20T06:03:20.622034+0000","172.21.15.5:0/1367598148":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2930274429":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6800/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2197596976":"2024-09-20T06:03:20.622034+0000","172.21.15.5:6801/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/1335130331":"2024-09-20T06:03:20.622034+0000","172.21.15.5:6800/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/1639646508":"2024-09-20T06:03:20.622034+0000","172.21.15.5:0/1340147787":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/850678758":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6801/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/2593868621":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/4102275559":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6800/709970884":"2024-09-20T06:03:20.622034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:03:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.039+0000 7f369214b640 1 -- 172.21.15.5:0/1502410725 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36740a30d0 msgr2=0x7f36740a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.039+0000 7f369214b640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36740a30d0 0x7f36740a5590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f367c02a930 tx=0x7f367c0058b0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.039+0000 7f369214b640 1 -- 172.21.15.5:0/1502410725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c104f60 msgr2=0x7f368c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.039+0000 7f369214b640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c104f60 0x7f368c198340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f368c069ff0 tx=0x7f368000ee50 comp rx=0 tx=0).stop 2024-09-19T06:03:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.039+0000 7f369214b640 1 -- 172.21.15.5:0/1502410725 shutdown_connections 2024-09-19T06:03:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.039+0000 7f369214b640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36740a30d0 0x7f36740a5590 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.039+0000 7f369214b640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f368c107890 0x7f368c19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.039+0000 7f369214b640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c104f60 0x7f368c198340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.039+0000 7f369214b640 1 --2- 172.21.15.5:0/1502410725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c102630 0x7f368c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.039+0000 7f369214b640 1 -- 172.21.15.5:0/1502410725 >> 172.21.15.5:0/1502410725 conn(0x7f368c0fc1f0 msgr2=0x7f368c1030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.039+0000 7f369214b640 1 -- 172.21.15.5:0/1502410725 shutdown_connections 2024-09-19T06:03:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.039+0000 7f369214b640 1 -- 172.21.15.5:0/1502410725 wait complete. 2024-09-19T06:03:22.052 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T05:56:55.213931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T06:01:25.648588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T06:01:27.760408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T06:03:22.052 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T06:03:22.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.159+0000 7fc83f677640 1 -- 172.21.15.5:0/702214379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8381025e0 msgr2=0x7fc838108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.159+0000 7fc83f677640 1 --2- 172.21.15.5:0/702214379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8381025e0 0x7fc838108e70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc830009ca0 tx=0x7fc83002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.159+0000 7fc83f677640 1 -- 172.21.15.5:0/702214379 shutdown_connections 2024-09-19T06:03:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.159+0000 7fc83f677640 1 --2- 172.21.15.5:0/702214379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8381025e0 0x7fc838108e70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.159+0000 7fc83f677640 1 --2- 172.21.15.5:0/702214379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc838101c20 0x7fc8381020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.159+0000 7fc83f677640 1 --2- 172.21.15.5:0/702214379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc838100a30 0x7fc838100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.159+0000 7fc83f677640 1 -- 172.21.15.5:0/702214379 >> 172.21.15.5:0/702214379 conn(0x7fc8380fc190 msgr2=0x7fc8380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.159+0000 7fc83f677640 1 -- 172.21.15.5:0/702214379 shutdown_connections 2024-09-19T06:03:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.159+0000 7fc83f677640 1 -- 172.21.15.5:0/702214379 wait complete. 2024-09-19T06:03:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.159+0000 7fc83f677640 1 Processor -- start 2024-09-19T06:03:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.159+0000 7fc83f677640 1 -- start start 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83f677640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc838100a30 0x7fc8381049c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83f677640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc838101c20 0x7fc838103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83f677640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8381025e0 0x7fc838103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83f677640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc83810c560 con 0x7fc838101c20 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83f677640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc83810c3e0 con 0x7fc838100a30 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83f677640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc83810c6e0 con 0x7fc8381025e0 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83cc12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc838101c20 0x7fc838103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83d413640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc838100a30 0x7fc8381049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83cc12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc838101c20 0x7fc838103000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56264/0 (socket says 172.21.15.5:56264) 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83cc12640 1 -- 172.21.15.5:0/1155833585 learned_addr learned my addr 172.21.15.5:0/1155833585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83dc14640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8381025e0 0x7fc838103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83cc12640 1 -- 172.21.15.5:0/1155833585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8381025e0 msgr2=0x7fc838103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83cc12640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8381025e0 0x7fc838103540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83cc12640 1 -- 172.21.15.5:0/1155833585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc838100a30 msgr2=0x7fc8381049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83cc12640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc838100a30 0x7fc8381049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83cc12640 1 -- 172.21.15.5:0/1155833585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc838103dd0 con 0x7fc838101c20 2024-09-19T06:03:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc83cc12640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc838101c20 0x7fc838103000 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc82800d950 tx=0x7fc82800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.160+0000 7fc82e7fc640 1 -- 172.21.15.5:0/1155833585 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828017070 con 0x7fc838101c20 2024-09-19T06:03:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.161+0000 7fc83f677640 1 -- 172.21.15.5:0/1155833585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8381a1f40 con 0x7fc838101c20 2024-09-19T06:03:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.161+0000 7fc82e7fc640 1 -- 172.21.15.5:0/1155833585 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc828009c30 con 0x7fc838101c20 2024-09-19T06:03:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.161+0000 7fc82e7fc640 1 -- 172.21.15.5:0/1155833585 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82800f620 con 0x7fc838101c20 2024-09-19T06:03:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.161+0000 7fc83f677640 1 -- 172.21.15.5:0/1155833585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8381a23b0 con 0x7fc838101c20 2024-09-19T06:03:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.162+0000 7fc83f677640 1 -- 172.21.15.5:0/1155833585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8381a26d0 con 0x7fc838101c20 2024-09-19T06:03:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.163+0000 7fc82e7fc640 1 -- 172.21.15.5:0/1155833585 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fc8280109a0 con 0x7fc838101c20 2024-09-19T06:03:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.164+0000 7fc82e7fc640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc8240a3050 0x7fc8240a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.164+0000 7fc82e7fc640 1 -- 172.21.15.5:0/1155833585 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fc8280cf140 con 0x7fc838101c20 2024-09-19T06:03:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.164+0000 7fc83d413640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc8240a3050 0x7fc8240a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.164+0000 7fc83d413640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc8240a3050 0x7fc8240a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc834005fd0 tx=0x7fc83400b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.172+0000 7fc82e7fc640 1 -- 172.21.15.5:0/1155833585 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc828009de0 con 0x7fc838101c20 2024-09-19T06:03:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.357+0000 7fc83f677640 1 -- 172.21.15.5:0/1155833585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8381a2a60 con 0x7fc838101c20 2024-09-19T06:03:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.357+0000 7fc82e7fc640 1 -- 172.21.15.5:0/1155833585 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fc82809b9f0 con 0x7fc838101c20 2024-09-19T06:03:22.359 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-19T06:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.360+0000 7fc83f677640 1 -- 172.21.15.5:0/1155833585 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc8240a3050 msgr2=0x7fc8240a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.360+0000 7fc83f677640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc8240a3050 0x7fc8240a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc834005fd0 tx=0x7fc83400b390 comp rx=0 tx=0).stop 2024-09-19T06:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.360+0000 7fc83f677640 1 -- 172.21.15.5:0/1155833585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc838101c20 msgr2=0x7fc838103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.360+0000 7fc83f677640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc838101c20 0x7fc838103000 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc82800d950 tx=0x7fc82800de10 comp rx=0 tx=0).stop 2024-09-19T06:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.360+0000 7fc83f677640 1 -- 172.21.15.5:0/1155833585 shutdown_connections 2024-09-19T06:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.360+0000 7fc83f677640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc8240a3050 0x7fc8240a5510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.360+0000 7fc83f677640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8381025e0 0x7fc838103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.360+0000 7fc83f677640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc838101c20 0x7fc838103000 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.360+0000 7fc83f677640 1 --2- 172.21.15.5:0/1155833585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc838100a30 0x7fc8381049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.360+0000 7fc83f677640 1 -- 172.21.15.5:0/1155833585 >> 172.21.15.5:0/1155833585 conn(0x7fc8380fc190 msgr2=0x7fc838069580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.360+0000 7fc83f677640 1 -- 172.21.15.5:0/1155833585 shutdown_connections 2024-09-19T06:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.360+0000 7fc83f677640 1 -- 172.21.15.5:0/1155833585 wait complete. 2024-09-19T06:03:22.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T06:03:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.503+0000 7fd70cd78640 1 -- 172.21.15.5:0/175037489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd708104ee0 msgr2=0x7fd7081072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.503+0000 7fd70cd78640 1 --2- 172.21.15.5:0/175037489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd708104ee0 0x7fd7081072d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc009900 tx=0x7fd6fc02a410 comp rx=0 tx=0).stop 2024-09-19T06:03:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.504+0000 7fd70cd78640 1 -- 172.21.15.5:0/175037489 shutdown_connections 2024-09-19T06:03:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.504+0000 7fd70cd78640 1 --2- 172.21.15.5:0/175037489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd708107810 0x7fd708109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.504+0000 7fd70cd78640 1 --2- 172.21.15.5:0/175037489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd708104ee0 0x7fd7081072d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.504+0000 7fd70cd78640 1 --2- 172.21.15.5:0/175037489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7081025b0 0x7fd7081049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.504+0000 7fd70cd78640 1 -- 172.21.15.5:0/175037489 >> 172.21.15.5:0/175037489 conn(0x7fd7080fc190 msgr2=0x7fd7080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.504+0000 7fd70cd78640 1 -- 172.21.15.5:0/175037489 shutdown_connections 2024-09-19T06:03:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.504+0000 7fd70cd78640 1 -- 172.21.15.5:0/175037489 wait complete. 2024-09-19T06:03:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70cd78640 1 Processor -- start 2024-09-19T06:03:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70cd78640 1 -- start start 2024-09-19T06:03:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70cd78640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7081025b0 0x7fd708197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70cd78640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd708104ee0 0x7fd708198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70cd78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd708107810 0x7fd70819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70cd78640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd70810d690 con 0x7fd7081025b0 2024-09-19T06:03:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70cd78640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd70810d510 con 0x7fd708104ee0 2024-09-19T06:03:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70cd78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd70810d810 con 0x7fd708107810 2024-09-19T06:03:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7081025b0 0x7fd708197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7081025b0 0x7fd708197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56284/0 (socket says 172.21.15.5:56284) 2024-09-19T06:03:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70659c640 1 -- 172.21.15.5:0/1633899933 learned_addr learned my addr 172.21.15.5:0/1633899933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd705d9b640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd708104ee0 0x7fd708198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd706d9d640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd708107810 0x7fd70819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70659c640 1 -- 172.21.15.5:0/1633899933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd708107810 msgr2=0x7fd70819f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70659c640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd708107810 0x7fd70819f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70659c640 1 -- 172.21.15.5:0/1633899933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd708104ee0 msgr2=0x7fd708198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70659c640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd708104ee0 0x7fd708198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd70659c640 1 -- 172.21.15.5:0/1633899933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70819fa70 con 0x7fd7081025b0 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.505+0000 7fd705d9b640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd708104ee0 0x7fd708198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.506+0000 7fd70659c640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7081025b0 0x7fd708197e50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800e9f0 tx=0x7fd6f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.506+0000 7fd6f77fe640 1 -- 172.21.15.5:0/1633899933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f800cd80 con 0x7fd7081025b0 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.506+0000 7fd70cd78640 1 -- 172.21.15.5:0/1633899933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70819fd60 con 0x7fd7081025b0 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.506+0000 7fd6f77fe640 1 -- 172.21.15.5:0/1633899933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6f80054d0 con 0x7fd7081025b0 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.506+0000 7fd6f77fe640 1 -- 172.21.15.5:0/1633899933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8010490 con 0x7fd7081025b0 2024-09-19T06:03:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.506+0000 7fd70cd78640 1 -- 172.21.15.5:0/1633899933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7081a01d0 con 0x7fd7081025b0 2024-09-19T06:03:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.507+0000 7fd70cd78640 1 -- 172.21.15.5:0/1633899933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7080fdc80 con 0x7fd7081025b0 2024-09-19T06:03:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.509+0000 7fd6f77fe640 1 -- 172.21.15.5:0/1633899933 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fd6f8005000 con 0x7fd7081025b0 2024-09-19T06:03:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.510+0000 7fd6f77fe640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd6f00a3050 0x7fd6f00a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.510+0000 7fd705d9b640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd6f00a3050 0x7fd6f00a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.511+0000 7fd6f77fe640 1 -- 172.21.15.5:0/1633899933 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fd6f80cfba0 con 0x7fd7081025b0 2024-09-19T06:03:22.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.511+0000 7fd705d9b640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd6f00a3050 0x7fd6f00a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc004770 tx=0x7fd6fc000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.513+0000 7fd6f77fe640 1 -- 172.21.15.5:0/1633899933 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd6f8054210 con 0x7fd7081025b0 2024-09-19T06:03:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.734+0000 7fd70cd78640 1 -- 172.21.15.5:0/1633899933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd70802d7c0 con 0x7fd7081025b0 2024-09-19T06:03:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.735+0000 7fd6f77fe640 1 -- 172.21.15.5:0/1633899933 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7fd6f80052f0 con 0x7fd7081025b0 2024-09-19T06:03:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-19T06:03:22.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.737+0000 7fd70cd78640 1 -- 172.21.15.5:0/1633899933 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd6f00a3050 msgr2=0x7fd6f00a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.737+0000 7fd70cd78640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd6f00a3050 0x7fd6f00a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc004770 tx=0x7fd6fc000bc0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.737+0000 7fd70cd78640 1 -- 172.21.15.5:0/1633899933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7081025b0 msgr2=0x7fd708197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.737+0000 7fd70cd78640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7081025b0 0x7fd708197e50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800e9f0 tx=0x7fd6f800edf0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.737+0000 7fd70cd78640 1 -- 172.21.15.5:0/1633899933 shutdown_connections 2024-09-19T06:03:22.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.737+0000 7fd70cd78640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd6f00a3050 0x7fd6f00a5510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.737+0000 7fd70cd78640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd708107810 0x7fd70819f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.737+0000 7fd70cd78640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd708104ee0 0x7fd708198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.737+0000 7fd70cd78640 1 --2- 172.21.15.5:0/1633899933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7081025b0 0x7fd708197e50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.737+0000 7fd70cd78640 1 -- 172.21.15.5:0/1633899933 >> 172.21.15.5:0/1633899933 conn(0x7fd7080fc190 msgr2=0x7fd708102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:22.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.737+0000 7fd70cd78640 1 -- 172.21.15.5:0/1633899933 shutdown_connections 2024-09-19T06:03:22.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.737+0000 7fd70cd78640 1 -- 172.21.15.5:0/1633899933 wait complete. 2024-09-19T06:03:22.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T06:03:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.862+0000 7fcba070a640 1 -- 172.21.15.5:0/1741026305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98100420 msgr2=0x7fcb98100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.862+0000 7fcba070a640 1 --2- 172.21.15.5:0/1741026305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98100420 0x7fcb98100820 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009900 tx=0x7fcb9402a480 comp rx=0 tx=0).stop 2024-09-19T06:03:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.862+0000 7fcba070a640 1 -- 172.21.15.5:0/1741026305 shutdown_connections 2024-09-19T06:03:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.862+0000 7fcba070a640 1 --2- 172.21.15.5:0/1741026305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb98101700 0x7fcb98101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.862+0000 7fcba070a640 1 --2- 172.21.15.5:0/1741026305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb98100d60 0x7fcb981011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.862+0000 7fcba070a640 1 --2- 172.21.15.5:0/1741026305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98100420 0x7fcb98100820 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.862+0000 7fcba070a640 1 -- 172.21.15.5:0/1741026305 >> 172.21.15.5:0/1741026305 conn(0x7fcb980fc190 msgr2=0x7fcb980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.862+0000 7fcba070a640 1 -- 172.21.15.5:0/1741026305 shutdown_connections 2024-09-19T06:03:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.862+0000 7fcba070a640 1 -- 172.21.15.5:0/1741026305 wait complete. 2024-09-19T06:03:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcba070a640 1 Processor -- start 2024-09-19T06:03:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcba070a640 1 -- start start 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcba070a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98100420 0x7fcb9819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcba070a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb98100d60 0x7fcb9819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcba070a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb98101700 0x7fcb981a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcba070a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9810f8b0 con 0x7fcb98100420 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcba070a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9810f730 con 0x7fcb98100d60 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcba070a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9810fa30 con 0x7fcb98101700 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcb9e4a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98100420 0x7fcb9819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcb9e4a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98100420 0x7fcb9819a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56314/0 (socket says 172.21.15.5:56314) 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcb9e4a6640 1 -- 172.21.15.5:0/3352390944 learned_addr learned my addr 172.21.15.5:0/3352390944 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcb9dca5640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb98100d60 0x7fcb9819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcb9eca7640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb98101700 0x7fcb981a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcb9dca5640 1 -- 172.21.15.5:0/3352390944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb98101700 msgr2=0x7fcb981a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcb9dca5640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb98101700 0x7fcb981a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcb9dca5640 1 -- 172.21.15.5:0/3352390944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98100420 msgr2=0x7fcb9819a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcb9dca5640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98100420 0x7fcb9819a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcb9dca5640 1 -- 172.21.15.5:0/3352390944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb981a1c30 con 0x7fcb98100d60 2024-09-19T06:03:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.863+0000 7fcb9e4a6640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98100420 0x7fcb9819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.864+0000 7fcb9dca5640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb98100d60 0x7fcb9819a550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800e9f0 tx=0x7fcb8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.864+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/3352390944 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8800ccc0 con 0x7fcb98100d60 2024-09-19T06:03:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.864+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/3352390944 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb880108e0 con 0x7fcb98100d60 2024-09-19T06:03:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.864+0000 7fcba070a640 1 -- 172.21.15.5:0/3352390944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb981a1f20 con 0x7fcb98100d60 2024-09-19T06:03:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.864+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/3352390944 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88018880 con 0x7fcb98100d60 2024-09-19T06:03:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.864+0000 7fcba070a640 1 -- 172.21.15.5:0/3352390944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb981a23e0 con 0x7fcb98100d60 2024-09-19T06:03:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.865+0000 7fcba070a640 1 -- 172.21.15.5:0/3352390944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb9810b170 con 0x7fcb98100d60 2024-09-19T06:03:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.867+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/3352390944 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fcb88010a90 con 0x7fcb98100d60 2024-09-19T06:03:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.868+0000 7fcb8f7fe640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcb840a3050 0x7fcb840a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.868+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/3352390944 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fcb880cfe30 con 0x7fcb98100d60 2024-09-19T06:03:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.868+0000 7fcb9e4a6640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcb840a3050 0x7fcb840a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.868+0000 7fcb9e4a6640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcb840a3050 0x7fcb840a5510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb940096c0 tx=0x7fcb94005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:22.870+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/3352390944 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcb8809c6a0 con 0x7fcb98100d60 2024-09-19T06:03:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:22 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3303375432' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T06:03:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:22 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:03:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:22 smithi049 ceph-mon[34640]: mgrmap e22: x(active, since 1.02895s), standbys: z, y 2024-09-19T06:03:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:22 smithi049 ceph-mon[34640]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:03:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:22 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1502410725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:03:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:22 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1155833585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:03:22.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:22 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:22.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:22 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:22.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:22 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:22.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:22 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:22 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3303375432' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T06:03:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:22 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:03:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:22 smithi157 ceph-mon[35346]: mgrmap e22: x(active, since 1.02895s), standbys: z, y 2024-09-19T06:03:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:22 smithi157 ceph-mon[35346]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:03:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:22 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1502410725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:03:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:22 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1155833585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:03:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:22 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:22 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:22 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:22 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.066+0000 7fcba070a640 1 -- 172.21.15.5:0/3352390944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb9802da10 con 0x7fcb98100d60 2024-09-19T06:03:23.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.067+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/3352390944 <== mon.2 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7fcb880a09a0 con 0x7fcb98100d60 2024-09-19T06:03:23.069 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-09-19T06:03:23.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.070+0000 7fcba070a640 1 -- 172.21.15.5:0/3352390944 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcb840a3050 msgr2=0x7fcb840a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:23.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.070+0000 7fcba070a640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcb840a3050 0x7fcb840a5510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb940096c0 tx=0x7fcb94005a20 comp rx=0 tx=0).stop 2024-09-19T06:03:23.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.070+0000 7fcba070a640 1 -- 172.21.15.5:0/3352390944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb98100d60 msgr2=0x7fcb9819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:23.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.070+0000 7fcba070a640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb98100d60 0x7fcb9819a550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800e9f0 tx=0x7fcb8800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:03:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.070+0000 7fcba070a640 1 -- 172.21.15.5:0/3352390944 shutdown_connections 2024-09-19T06:03:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.070+0000 7fcba070a640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcb840a3050 0x7fcb840a5510 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.070+0000 7fcba070a640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb98101700 0x7fcb981a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.070+0000 7fcba070a640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb98100d60 0x7fcb9819a550 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.070+0000 7fcba070a640 1 --2- 172.21.15.5:0/3352390944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98100420 0x7fcb9819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.070+0000 7fcba070a640 1 -- 172.21.15.5:0/3352390944 >> 172.21.15.5:0/3352390944 conn(0x7fcb980fc190 msgr2=0x7fcb980fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:23.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.070+0000 7fcba070a640 1 -- 172.21.15.5:0/3352390944 shutdown_connections 2024-09-19T06:03:23.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.070+0000 7fcba070a640 1 -- 172.21.15.5:0/3352390944 wait complete. 2024-09-19T06:03:23.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:22 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3303375432' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T06:03:23.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:22 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:03:23.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:22 smithi005 ceph-mon[30955]: mgrmap e22: x(active, since 1.02895s), standbys: z, y 2024-09-19T06:03:23.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:22 smithi005 ceph-mon[30955]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:03:23.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:22 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1502410725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:03:23.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:22 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1155833585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:03:23.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:22 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:22 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:22 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:22 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.083 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T06:03:23.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:03:23.352 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:23.614 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:23.360+0000 7f3f25998640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:06:03:23] ENGINE Error in HTTPServer.serve 2024-09-19T06:03:23.615 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: Traceback (most recent call last): 2024-09-19T06:03:23.615 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:03:23.615 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self._connections.run(self.expiration_interval) 2024-09-19T06:03:23.615 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:03:23.615 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self._run(expiration_interval) 2024-09-19T06:03:23.615 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:03:23.615 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:03:23.615 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:03:23.615 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:03:23.616 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:03:23.616 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: s = self.context.wrap_socket( 2024-09-19T06:03:23.616 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:03:23.616 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: return self.sslsocket_class._create( 2024-09-19T06:03:23.616 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:03:23.616 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self.do_handshake() 2024-09-19T06:03:23.616 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:03:23.616 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self._sslobj.do_handshake() 2024-09-19T06:03:23.616 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:03:23.616 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:03:23.875 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1633899933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:03:23.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3352390944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:03:23.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: [19/Sep/2024:06:03:23] ENGINE Bus STARTING 2024-09-19T06:03:23.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T06:03:23.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: Unable to set osd_memory_target on smithi157 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-19T06:03:23.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi049 to 176.0M 2024-09-19T06:03:23.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: Unable to set osd_memory_target on smithi049 to 184553062: error parsing value: Value '184553062' is below minimum 939524096 2024-09-19T06:03:23.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: [19/Sep/2024:06:03:23] ENGINE Serving on https://172.21.15.5:7150 2024-09-19T06:03:23.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: [19/Sep/2024:06:03:23] ENGINE Error in HTTPServer.serve 2024-09-19T06:03:23.877 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-19T06:03:23.878 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:03:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-19T06:03:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:03:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-19T06:03:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:03:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:23 smithi005 ceph-mon[30955]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T06:03:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.872+0000 7f02d4857640 1 -- 172.21.15.5:0/3386101076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d0102d30 msgr2=0x7f02d0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.872+0000 7f02d4857640 1 --2- 172.21.15.5:0/3386101076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d0102d30 0x7f02d0103110 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009930 tx=0x7f02c002a420 comp rx=0 tx=0).stop 2024-09-19T06:03:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.872+0000 7f02d4857640 1 -- 172.21.15.5:0/3386101076 shutdown_connections 2024-09-19T06:03:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.872+0000 7f02d4857640 1 --2- 172.21.15.5:0/3386101076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d01082a0 0x7f02d010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.872+0000 7f02d4857640 1 --2- 172.21.15.5:0/3386101076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02d01036e0 0x7f02d0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.872+0000 7f02d4857640 1 --2- 172.21.15.5:0/3386101076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d0102d30 0x7f02d0103110 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.872+0000 7f02d4857640 1 -- 172.21.15.5:0/3386101076 >> 172.21.15.5:0/3386101076 conn(0x7f02d00fc7d0 msgr2=0x7f02d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.872+0000 7f02d4857640 1 -- 172.21.15.5:0/3386101076 shutdown_connections 2024-09-19T06:03:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.872+0000 7f02d4857640 1 -- 172.21.15.5:0/3386101076 wait complete. 2024-09-19T06:03:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02d4857640 1 Processor -- start 2024-09-19T06:03:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02d4857640 1 -- start start 2024-09-19T06:03:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02d4857640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02d0102d30 0x7f02d019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02d4857640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d01036e0 0x7f02d019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02d4857640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d01082a0 0x7f02d019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02d4857640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f02d0111f50 con 0x7f02d01082a0 2024-09-19T06:03:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02d4857640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f02d0111dd0 con 0x7f02d0102d30 2024-09-19T06:03:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02d4857640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01120d0 con 0x7f02d01036e0 2024-09-19T06:03:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d01082a0 0x7f02d019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d01082a0 0x7f02d019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56332/0 (socket says 172.21.15.5:56332) 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02ced9d640 1 -- 172.21.15.5:0/2866070020 learned_addr learned my addr 172.21.15.5:0/2866070020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02ce59c640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02d0102d30 0x7f02d019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.873+0000 7f02cdd9b640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d01036e0 0x7f02d019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.874+0000 7f02ced9d640 1 -- 172.21.15.5:0/2866070020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d01036e0 msgr2=0x7f02d019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.874+0000 7f02ced9d640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d01036e0 0x7f02d019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.874+0000 7f02ced9d640 1 -- 172.21.15.5:0/2866070020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02d0102d30 msgr2=0x7f02d019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.874+0000 7f02ced9d640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02d0102d30 0x7f02d019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.874+0000 7f02ced9d640 1 -- 172.21.15.5:0/2866070020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d019f660 con 0x7f02d01082a0 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.874+0000 7f02ce59c640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02d0102d30 0x7f02d019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.874+0000 7f02ced9d640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d01082a0 0x7f02d019ef80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f02c800eb10 tx=0x7f02c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.874+0000 7f02bf7fe640 1 -- 172.21.15.5:0/2866070020 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8004b20 con 0x7f02d01082a0 2024-09-19T06:03:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.874+0000 7f02d4857640 1 -- 172.21.15.5:0/2866070020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d019f950 con 0x7f02d01082a0 2024-09-19T06:03:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.874+0000 7f02bf7fe640 1 -- 172.21.15.5:0/2866070020 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02c8004cc0 con 0x7f02d01082a0 2024-09-19T06:03:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.874+0000 7f02bf7fe640 1 -- 172.21.15.5:0/2866070020 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8010690 con 0x7f02d01082a0 2024-09-19T06:03:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.874+0000 7f02d4857640 1 -- 172.21.15.5:0/2866070020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d01a7230 con 0x7f02d01082a0 2024-09-19T06:03:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.875+0000 7f02d4857640 1 -- 172.21.15.5:0/2866070020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d006a850 con 0x7f02d01082a0 2024-09-19T06:03:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.877+0000 7f02bf7fe640 1 -- 172.21.15.5:0/2866070020 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138474+0+0 (secure 0 0 0) 0x7f02c8005020 con 0x7f02d01082a0 2024-09-19T06:03:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.878+0000 7f02bf7fe640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f02b80a3430 0x7f02b80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.878+0000 7f02ce59c640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f02b80a3430 0x7f02b80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.878+0000 7f02bf7fe640 1 -- 172.21.15.5:0/2866070020 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f02c80d0150 con 0x7f02d01082a0 2024-09-19T06:03:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.878+0000 7f02ce59c640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f02b80a3430 0x7f02b80a58f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009900 tx=0x7f02c0000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:23.882+0000 7f02bf7fe640 1 -- 172.21.15.5:0/2866070020 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f02c809cb80 con 0x7f02d01082a0 2024-09-19T06:03:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1633899933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:03:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3352390944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:03:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: [19/Sep/2024:06:03:23] ENGINE Bus STARTING 2024-09-19T06:03:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: Unable to set osd_memory_target on smithi157 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi049 to 176.0M 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: Unable to set osd_memory_target on smithi049 to 184553062: error parsing value: Value '184553062' is below minimum 939524096 2024-09-19T06:03:23.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: [19/Sep/2024:06:03:23] ENGINE Serving on https://172.21.15.5:7150 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: [19/Sep/2024:06:03:23] ENGINE Error in HTTPServer.serve 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: Traceback (most recent call last): 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: self._run(expiration_interval) 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: s = self.context.wrap_socket( 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:03:23.899 INFO:journalctl@ceph.mon.b.smithi049.stdout: return self.sslsocket_class._create( 2024-09-19T06:03:23.900 INFO:journalctl@ceph.mon.b.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:03:23.900 INFO:journalctl@ceph.mon.b.smithi049.stdout: self.do_handshake() 2024-09-19T06:03:23.900 INFO:journalctl@ceph.mon.b.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:03:23.900 INFO:journalctl@ceph.mon.b.smithi049.stdout: self._sslobj.do_handshake() 2024-09-19T06:03:23.900 INFO:journalctl@ceph.mon.b.smithi049.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:03:23.900 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:23 smithi049 ceph-mon[34640]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T06:03:24.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1633899933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:03:24.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3352390944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:03:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: [19/Sep/2024:06:03:23] ENGINE Bus STARTING 2024-09-19T06:03:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T06:03:24.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: Unable to set osd_memory_target on smithi157 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi049 to 176.0M 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: Unable to set osd_memory_target on smithi049 to 184553062: error parsing value: Value '184553062' is below minimum 939524096 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: [19/Sep/2024:06:03:23] ENGINE Serving on https://172.21.15.5:7150 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: [19/Sep/2024:06:03:23] ENGINE Error in HTTPServer.serve 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout: Traceback (most recent call last): 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:03:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:03:24.054 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:03:24.054 INFO:journalctl@ceph.mon.c.smithi157.stdout: self._run(expiration_interval) 2024-09-19T06:03:24.054 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:03:24.054 INFO:journalctl@ceph.mon.c.smithi157.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:03:24.054 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:03:24.054 INFO:journalctl@ceph.mon.c.smithi157.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:03:24.054 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:03:24.054 INFO:journalctl@ceph.mon.c.smithi157.stdout: s = self.context.wrap_socket( 2024-09-19T06:03:24.054 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:03:24.054 INFO:journalctl@ceph.mon.c.smithi157.stdout: return self.sslsocket_class._create( 2024-09-19T06:03:24.054 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:03:24.054 INFO:journalctl@ceph.mon.c.smithi157.stdout: self.do_handshake() 2024-09-19T06:03:24.055 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:03:24.055 INFO:journalctl@ceph.mon.c.smithi157.stdout: self._sslobj.do_handshake() 2024-09-19T06:03:24.055 INFO:journalctl@ceph.mon.c.smithi157.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:03:24.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:23 smithi157 ceph-mon[35346]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T06:03:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.113+0000 7f02d4857640 1 -- 172.21.15.5:0/2866070020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02d010c2c0 con 0x7f02d01082a0 2024-09-19T06:03:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.114+0000 7f02bf7fe640 1 -- 172.21.15.5:0/2866070020 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11959 (secure 0 0 0) 0x7f02c809c480 con 0x7f02d01082a0 2024-09-19T06:03:24.116 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:03:24.116 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-19T06:03:21:625357+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:03:20.863188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:03:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-19T06:03:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.116+0000 7f02d4857640 1 -- 172.21.15.5:0/2866070020 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f02b80a3430 msgr2=0x7f02b80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.116+0000 7f02d4857640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f02b80a3430 0x7f02b80a58f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009900 tx=0x7f02c0000bc0 comp rx=0 tx=0).stop 2024-09-19T06:03:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.116+0000 7f02d4857640 1 -- 172.21.15.5:0/2866070020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d01082a0 msgr2=0x7f02d019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.116+0000 7f02d4857640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d01082a0 0x7f02d019ef80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f02c800eb10 tx=0x7f02c800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:03:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.116+0000 7f02d4857640 1 -- 172.21.15.5:0/2866070020 shutdown_connections 2024-09-19T06:03:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.116+0000 7f02d4857640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d01082a0 0x7f02d019ef80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.116+0000 7f02d4857640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f02b80a3430 0x7f02b80a58f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.116+0000 7f02d4857640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d01036e0 0x7f02d019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.116+0000 7f02d4857640 1 --2- 172.21.15.5:0/2866070020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02d0102d30 0x7f02d019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.116+0000 7f02d4857640 1 -- 172.21.15.5:0/2866070020 >> 172.21.15.5:0/2866070020 conn(0x7f02d00fc7d0 msgr2=0x7f02d0100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.116+0000 7f02d4857640 1 -- 172.21.15.5:0/2866070020 shutdown_connections 2024-09-19T06:03:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.117+0000 7f02d4857640 1 -- 172.21.15.5:0/2866070020 wait complete. 2024-09-19T06:03:24.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: [19/Sep/2024:06:03:23] ENGINE Serving on http://172.21.15.5:8765 2024-09-19T06:03:24.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: [19/Sep/2024:06:03:23] ENGINE Bus STARTED 2024-09-19T06:03:24.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:24.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:24.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:24.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:24.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:24.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:24.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-19T06:03:24.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T06:03:24.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:24.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:24.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-19T06:03:24.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Updating smithi049:/etc/ceph/ceph.conf 2024-09-19T06:03:24.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-19T06:03:24.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Updating smithi005:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:03:24.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Updating smithi049:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:03:24.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:03:24.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2866070020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:03:24.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:03:24.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:03:24.837 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:03:24.837 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Updating smithi005:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:03:24.837 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:03:24.837 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:24 smithi005 ceph-mon[30955]: Updating smithi049:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:03:24.862 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T06:03:24.862 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T06:03:24.862 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T06:03:24.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.968+0000 7fd2f70fa640 1 -- 172.21.15.5:0/465716963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f0102d00 msgr2=0x7fd2f01030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.968+0000 7fd2f70fa640 1 --2- 172.21.15.5:0/465716963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f0102d00 0x7fd2f01030e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009900 tx=0x7fd2e402a480 comp rx=0 tx=0).stop 2024-09-19T06:03:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.968+0000 7fd2f70fa640 1 -- 172.21.15.5:0/465716963 shutdown_connections 2024-09-19T06:03:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.968+0000 7fd2f70fa640 1 --2- 172.21.15.5:0/465716963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2f010c0d0 0x7fd2f010e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.968+0000 7fd2f70fa640 1 --2- 172.21.15.5:0/465716963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2f0103620 0x7fd2f010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.968+0000 7fd2f70fa640 1 --2- 172.21.15.5:0/465716963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f0102d00 0x7fd2f01030e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.968+0000 7fd2f70fa640 1 -- 172.21.15.5:0/465716963 >> 172.21.15.5:0/465716963 conn(0x7fd2f00fc6b0 msgr2=0x7fd2f00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.968+0000 7fd2f70fa640 1 -- 172.21.15.5:0/465716963 shutdown_connections 2024-09-19T06:03:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.968+0000 7fd2f70fa640 1 -- 172.21.15.5:0/465716963 wait complete. 2024-09-19T06:03:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2f70fa640 1 Processor -- start 2024-09-19T06:03:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2f70fa640 1 -- start start 2024-09-19T06:03:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2f70fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2f0102d00 0x7fd2f0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2f70fa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2f0103620 0x7fd2f0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2f70fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f010c0d0 0x7fd2f019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2f70fa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f0111dd0 con 0x7fd2f010c0d0 2024-09-19T06:03:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2f70fa640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f0111c50 con 0x7fd2f0103620 2024-09-19T06:03:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2f70fa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f0111f50 con 0x7fd2f0102d00 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2f5697640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f010c0d0 0x7fd2f019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2f5697640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f010c0d0 0x7fd2f019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37926/0 (socket says 172.21.15.5:37926) 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2f5697640 1 -- 172.21.15.5:0/1267165920 learned_addr learned my addr 172.21.15.5:0/1267165920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2f4e96640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2f0102d00 0x7fd2f0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.969+0000 7fd2effff640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2f0103620 0x7fd2f0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.970+0000 7fd2f5697640 1 -- 172.21.15.5:0/1267165920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2f0102d00 msgr2=0x7fd2f0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.970+0000 7fd2f5697640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2f0102d00 0x7fd2f0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.970+0000 7fd2f5697640 1 -- 172.21.15.5:0/1267165920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2f0103620 msgr2=0x7fd2f0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.970+0000 7fd2f5697640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2f0103620 0x7fd2f0198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.970+0000 7fd2f5697640 1 -- 172.21.15.5:0/1267165920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f019d330 con 0x7fd2f010c0d0 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.970+0000 7fd2f4e96640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2f0102d00 0x7fd2f0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.970+0000 7fd2f5697640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f010c0d0 0x7fd2f019cc00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800eb10 tx=0x7fd2e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.970+0000 7fd2edffb640 1 -- 172.21.15.5:0/1267165920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8004b20 con 0x7fd2f010c0d0 2024-09-19T06:03:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.970+0000 7fd2edffb640 1 -- 172.21.15.5:0/1267165920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2e8004cc0 con 0x7fd2f010c0d0 2024-09-19T06:03:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.970+0000 7fd2f70fa640 1 -- 172.21.15.5:0/1267165920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f019d620 con 0x7fd2f010c0d0 2024-09-19T06:03:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.970+0000 7fd2edffb640 1 -- 172.21.15.5:0/1267165920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8010690 con 0x7fd2f010c0d0 2024-09-19T06:03:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.970+0000 7fd2f70fa640 1 -- 172.21.15.5:0/1267165920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f01a4eb0 con 0x7fd2f010c0d0 2024-09-19T06:03:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.971+0000 7fd2f70fa640 1 -- 172.21.15.5:0/1267165920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f01030e0 con 0x7fd2f010c0d0 2024-09-19T06:03:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.972+0000 7fd2edffb640 1 -- 172.21.15.5:0/1267165920 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7fd2e8005020 con 0x7fd2f010c0d0 2024-09-19T06:03:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.973+0000 7fd2edffb640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd2e00a3450 0x7fd2e00a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.973+0000 7fd2edffb640 1 -- 172.21.15.5:0/1267165920 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fd2e80d0560 con 0x7fd2f010c0d0 2024-09-19T06:03:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.973+0000 7fd2f4e96640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd2e00a3450 0x7fd2e00a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.973+0000 7fd2f4e96640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd2e00a3450 0x7fd2e00a5910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4005ab0 tx=0x7fd2e4005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:24.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:24.977+0000 7fd2edffb640 1 -- 172.21.15.5:0/1267165920 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd2e809cd80 con 0x7fd2f010c0d0 2024-09-19T06:03:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: [19/Sep/2024:06:03:23] ENGINE Serving on http://172.21.15.5:8765 2024-09-19T06:03:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: [19/Sep/2024:06:03:23] ENGINE Bus STARTED 2024-09-19T06:03:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-19T06:03:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T06:03:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-19T06:03:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Updating smithi049:/etc/ceph/ceph.conf 2024-09-19T06:03:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-19T06:03:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Updating smithi005:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:03:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Updating smithi049:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:03:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:03:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2866070020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:03:25.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:03:25.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:03:25.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:03:25.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Updating smithi005:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:03:25.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:03:25.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:24 smithi157 ceph-mon[35346]: Updating smithi049:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:03:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: [19/Sep/2024:06:03:23] ENGINE Serving on http://172.21.15.5:8765 2024-09-19T06:03:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: [19/Sep/2024:06:03:23] ENGINE Bus STARTED 2024-09-19T06:03:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:03:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-19T06:03:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T06:03:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Updating smithi049:/etc/ceph/ceph.conf 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Updating smithi005:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Updating smithi049:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2866070020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Updating smithi005:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Updating smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:03:25.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:24 smithi049 ceph-mon[34640]: Updating smithi049:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:03:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.149+0000 7fd2f70fa640 1 -- 172.21.15.5:0/1267165920 --> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2f0062630 con 0x7fd2e00a3450 2024-09-19T06:03:25.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.156+0000 7fd2edffb640 1 -- 172.21.15.5:0/1267165920 <== mgr.15240 v2:172.21.15.5:6800/2837801016 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257786 (secure 0 0 0) 0x7fd2f0062630 con 0x7fd2e00a3450 2024-09-19T06:03:25.160 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:03:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-19T06:03:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.159+0000 7fd2f70fa640 1 -- 172.21.15.5:0/1267165920 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd2e00a3450 msgr2=0x7fd2e00a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.159+0000 7fd2f70fa640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd2e00a3450 0x7fd2e00a5910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4005ab0 tx=0x7fd2e4005a20 comp rx=0 tx=0).stop 2024-09-19T06:03:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.159+0000 7fd2f70fa640 1 -- 172.21.15.5:0/1267165920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f010c0d0 msgr2=0x7fd2f019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.159+0000 7fd2f70fa640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f010c0d0 0x7fd2f019cc00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800eb10 tx=0x7fd2e800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.159+0000 7fd2f70fa640 1 -- 172.21.15.5:0/1267165920 shutdown_connections 2024-09-19T06:03:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.159+0000 7fd2f70fa640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f010c0d0 0x7fd2f019cc00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.159+0000 7fd2f70fa640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2f0103620 0x7fd2f0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.159+0000 7fd2f70fa640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd2e00a3450 0x7fd2e00a5910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.159+0000 7fd2f70fa640 1 --2- 172.21.15.5:0/1267165920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2f0102d00 0x7fd2f0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.159+0000 7fd2f70fa640 1 -- 172.21.15.5:0/1267165920 >> 172.21.15.5:0/1267165920 conn(0x7fd2f00fc6b0 msgr2=0x7fd2f00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.159+0000 7fd2f70fa640 1 -- 172.21.15.5:0/1267165920 shutdown_connections 2024-09-19T06:03:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.159+0000 7fd2f70fa640 1 -- 172.21.15.5:0/1267165920 wait complete. 2024-09-19T06:03:25.172 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T06:03:24.640412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194544,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1285,"num_read_kb":813,"num_write":570,"num_write_kb":4414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":950,"ondisk_log_size":950,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392800,"kb_used_data":14224,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549024,"statfs":{"total":1151940427776,"available":1151538200576,"internally_reserved":0,"allocated":14565376,"data_stored":8523222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.982108"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887438+0000","last_change":"2024-09-19T06:01:28.812722+0000","last_active":"2024-09-19T06:03:20.887438+0000","last_peered":"2024-09-19T06:03:20.887438+0000","last_clean":"2024-09-19T06:03:20.887438+0000","last_became_active":"2024-09-19T06:01:28.812283+0000","last_became_peered":"2024-09-19T06:01:28.812283+0000","last_unstale":"2024-09-19T06:03:20.887438+0000","last_undegraded":"2024-09-19T06:03:20.887438+0000","last_fullsized":"2024-09-19T06:03:20.887438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:59.146636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.888342+0000","last_change":"2024-09-19T06:01:28.805210+0000","last_active":"2024-09-19T06:03:20.888342+0000","last_peered":"2024-09-19T06:03:20.888342+0000","last_clean":"2024-09-19T06:03:20.888342+0000","last_became_active":"2024-09-19T06:01:28.804906+0000","last_became_peered":"2024-09-19T06:01:28.804906+0000","last_unstale":"2024-09-19T06:03:20.888342+0000","last_undegraded":"2024-09-19T06:03:20.888342+0000","last_fullsized":"2024-09-19T06:03:20.888342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:00.227101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.889300+0000","last_change":"2024-09-19T06:01:28.811909+0000","last_active":"2024-09-19T06:03:20.889300+0000","last_peered":"2024-09-19T06:03:20.889300+0000","last_clean":"2024-09-19T06:03:20.889300+0000","last_became_active":"2024-09-19T06:01:28.811547+0000","last_became_peered":"2024-09-19T06:01:28.811547+0000","last_unstale":"2024-09-19T06:03:20.889300+0000","last_undegraded":"2024-09-19T06:03:20.889300+0000","last_fullsized":"2024-09-19T06:03:20.889300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:50.022439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.893221+0000","last_change":"2024-09-19T06:01:28.806782+0000","last_active":"2024-09-19T06:03:20.893221+0000","last_peered":"2024-09-19T06:03:20.893221+0000","last_clean":"2024-09-19T06:03:20.893221+0000","last_became_active":"2024-09-19T06:01:28.805520+0000","last_became_peered":"2024-09-19T06:01:28.805520+0000","last_unstale":"2024-09-19T06:03:20.893221+0000","last_undegraded":"2024-09-19T06:03:20.893221+0000","last_fullsized":"2024-09-19T06:03:20.893221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:01.652669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722796+0000","last_change":"2024-09-19T06:01:28.809971+0000","last_active":"2024-09-19T06:03:20.722796+0000","last_peered":"2024-09-19T06:03:20.722796+0000","last_clean":"2024-09-19T06:03:20.722796+0000","last_became_active":"2024-09-19T06:01:28.809593+0000","last_became_peered":"2024-09-19T06:01:28.809593+0000","last_unstale":"2024-09-19T06:03:20.722796+0000","last_undegraded":"2024-09-19T06:03:20.722796+0000","last_fullsized":"2024-09-19T06:03:20.722796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.546526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732082+0000","last_change":"2024-09-19T06:01:28.801395+0000","last_active":"2024-09-19T06:03:20.732082+0000","last_peered":"2024-09-19T06:03:20.732082+0000","last_clean":"2024-09-19T06:03:20.732082+0000","last_became_active":"2024-09-19T06:01:28.800987+0000","last_became_peered":"2024-09-19T06:01:28.800987+0000","last_unstale":"2024-09-19T06:03:20.732082+0000","last_undegraded":"2024-09-19T06:03:20.732082+0000","last_fullsized":"2024-09-19T06:03:20.732082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:18.986489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628078+0000","last_change":"2024-09-19T06:01:28.806554+0000","last_active":"2024-09-19T06:03:20.628078+0000","last_peered":"2024-09-19T06:03:20.628078+0000","last_clean":"2024-09-19T06:03:20.628078+0000","last_became_active":"2024-09-19T06:01:28.805946+0000","last_became_peered":"2024-09-19T06:01:28.805946+0000","last_unstale":"2024-09-19T06:03:20.628078+0000","last_undegraded":"2024-09-19T06:03:20.628078+0000","last_fullsized":"2024-09-19T06:03:20.628078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:56:20.130414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722784+0000","last_change":"2024-09-19T06:01:28.804789+0000","last_active":"2024-09-19T06:03:20.722784+0000","last_peered":"2024-09-19T06:03:20.722784+0000","last_clean":"2024-09-19T06:03:20.722784+0000","last_became_active":"2024-09-19T06:01:28.804484+0000","last_became_peered":"2024-09-19T06:01:28.804484+0000","last_unstale":"2024-09-19T06:03:20.722784+0000","last_undegraded":"2024-09-19T06:03:20.722784+0000","last_fullsized":"2024-09-19T06:03:20.722784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:19.737928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884972+0000","last_change":"2024-09-19T06:01:28.814504+0000","last_active":"2024-09-19T06:03:20.884972+0000","last_peered":"2024-09-19T06:03:20.884972+0000","last_clean":"2024-09-19T06:03:20.884972+0000","last_became_active":"2024-09-19T06:01:28.814066+0000","last_became_peered":"2024-09-19T06:01:28.814066+0000","last_unstale":"2024-09-19T06:03:20.884972+0000","last_undegraded":"2024-09-19T06:03:20.884972+0000","last_fullsized":"2024-09-19T06:03:20.884972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:50.904164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732391+0000","last_change":"2024-09-19T06:01:28.807303+0000","last_active":"2024-09-19T06:03:20.732391+0000","last_peered":"2024-09-19T06:03:20.732391+0000","last_clean":"2024-09-19T06:03:20.732391+0000","last_became_active":"2024-09-19T06:01:28.806978+0000","last_became_peered":"2024-09-19T06:01:28.806978+0000","last_unstale":"2024-09-19T06:03:20.732391+0000","last_undegraded":"2024-09-19T06:03:20.732391+0000","last_fullsized":"2024-09-19T06:03:20.732391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:33.259261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628504+0000","last_change":"2024-09-19T06:01:28.814844+0000","last_active":"2024-09-19T06:03:20.628504+0000","last_peered":"2024-09-19T06:03:20.628504+0000","last_clean":"2024-09-19T06:03:20.628504+0000","last_became_active":"2024-09-19T06:01:28.814531+0000","last_became_peered":"2024-09-19T06:01:28.814531+0000","last_unstale":"2024-09-19T06:03:20.628504+0000","last_undegraded":"2024-09-19T06:03:20.628504+0000","last_fullsized":"2024-09-19T06:03:20.628504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:17.209890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.893037+0000","last_change":"2024-09-19T06:01:28.806378+0000","last_active":"2024-09-19T06:03:20.893037+0000","last_peered":"2024-09-19T06:03:20.893037+0000","last_clean":"2024-09-19T06:03:20.893037+0000","last_became_active":"2024-09-19T06:01:28.805053+0000","last_became_peered":"2024-09-19T06:01:28.805053+0000","last_unstale":"2024-09-19T06:03:20.893037+0000","last_undegraded":"2024-09-19T06:03:20.893037+0000","last_fullsized":"2024-09-19T06:03:20.893037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:15.080548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627148+0000","last_change":"2024-09-19T06:01:28.802941+0000","last_active":"2024-09-19T06:03:20.627148+0000","last_peered":"2024-09-19T06:03:20.627148+0000","last_clean":"2024-09-19T06:03:20.627148+0000","last_became_active":"2024-09-19T06:01:28.802590+0000","last_became_peered":"2024-09-19T06:01:28.802590+0000","last_unstale":"2024-09-19T06:03:20.627148+0000","last_undegraded":"2024-09-19T06:03:20.627148+0000","last_fullsized":"2024-09-19T06:03:20.627148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:25.581854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887443+0000","last_change":"2024-09-19T06:01:28.800960+0000","last_active":"2024-09-19T06:03:20.887443+0000","last_peered":"2024-09-19T06:03:20.887443+0000","last_clean":"2024-09-19T06:03:20.887443+0000","last_became_active":"2024-09-19T06:01:28.800484+0000","last_became_peered":"2024-09-19T06:01:28.800484+0000","last_unstale":"2024-09-19T06:03:20.887443+0000","last_undegraded":"2024-09-19T06:03:20.887443+0000","last_fullsized":"2024-09-19T06:03:20.887443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:08.506105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.888999+0000","last_change":"2024-09-19T06:01:28.800771+0000","last_active":"2024-09-19T06:03:20.888999+0000","last_peered":"2024-09-19T06:03:20.888999+0000","last_clean":"2024-09-19T06:03:20.888999+0000","last_became_active":"2024-09-19T06:01:28.800427+0000","last_became_peered":"2024-09-19T06:01:28.800427+0000","last_unstale":"2024-09-19T06:03:20.888999+0000","last_undegraded":"2024-09-19T06:03:20.888999+0000","last_fullsized":"2024-09-19T06:03:20.888999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:19.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884875+0000","last_change":"2024-09-19T06:01:28.815528+0000","last_active":"2024-09-19T06:03:20.884875+0000","last_peered":"2024-09-19T06:03:20.884875+0000","last_clean":"2024-09-19T06:03:20.884875+0000","last_became_active":"2024-09-19T06:01:28.815129+0000","last_became_peered":"2024-09-19T06:01:28.815129+0000","last_unstale":"2024-09-19T06:03:20.884875+0000","last_undegraded":"2024-09-19T06:03:20.884875+0000","last_fullsized":"2024-09-19T06:03:20.884875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:01.936711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887589+0000","last_change":"2024-09-19T06:01:28.816120+0000","last_active":"2024-09-19T06:03:20.887589+0000","last_peered":"2024-09-19T06:03:20.887589+0000","last_clean":"2024-09-19T06:03:20.887589+0000","last_became_active":"2024-09-19T06:01:28.815822+0000","last_became_peered":"2024-09-19T06:01:28.815822+0000","last_unstale":"2024-09-19T06:03:20.887589+0000","last_undegraded":"2024-09-19T06:03:20.887589+0000","last_fullsized":"2024-09-19T06:03:20.887589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:14:58.670452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.721021+0000","last_change":"2024-09-19T06:01:28.809825+0000","last_active":"2024-09-19T06:03:20.721021+0000","last_peered":"2024-09-19T06:03:20.721021+0000","last_clean":"2024-09-19T06:03:20.721021+0000","last_became_active":"2024-09-19T06:01:28.809530+0000","last_became_peered":"2024-09-19T06:01:28.809530+0000","last_unstale":"2024-09-19T06:03:20.721021+0000","last_undegraded":"2024-09-19T06:03:20.721021+0000","last_fullsized":"2024-09-19T06:03:20.721021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:49:23.442436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.895924+0000","last_change":"2024-09-19T06:01:28.801297+0000","last_active":"2024-09-19T06:03:20.895924+0000","last_peered":"2024-09-19T06:03:20.895924+0000","last_clean":"2024-09-19T06:03:20.895924+0000","last_became_active":"2024-09-19T06:01:28.800984+0000","last_became_peered":"2024-09-19T06:01:28.800984+0000","last_unstale":"2024-09-19T06:03:20.895924+0000","last_undegraded":"2024-09-19T06:03:20.895924+0000","last_fullsized":"2024-09-19T06:03:20.895924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:01.485951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887070+0000","last_change":"2024-09-19T06:01:28.804567+0000","last_active":"2024-09-19T06:03:20.887070+0000","last_peered":"2024-09-19T06:03:20.887070+0000","last_clean":"2024-09-19T06:03:20.887070+0000","last_became_active":"2024-09-19T06:01:28.804213+0000","last_became_peered":"2024-09-19T06:01:28.804213+0000","last_unstale":"2024-09-19T06:03:20.887070+0000","last_undegraded":"2024-09-19T06:03:20.887070+0000","last_fullsized":"2024-09-19T06:03:20.887070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.186460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627307+0000","last_change":"2024-09-19T06:01:28.801961+0000","last_active":"2024-09-19T06:03:20.627307+0000","last_peered":"2024-09-19T06:03:20.627307+0000","last_clean":"2024-09-19T06:03:20.627307+0000","last_became_active":"2024-09-19T06:01:28.801631+0000","last_became_peered":"2024-09-19T06:01:28.801631+0000","last_unstale":"2024-09-19T06:03:20.627307+0000","last_undegraded":"2024-09-19T06:03:20.627307+0000","last_fullsized":"2024-09-19T06:03:20.627307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:15.628908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884896+0000","last_change":"2024-09-19T06:01:28.815160+0000","last_active":"2024-09-19T06:03:20.884896+0000","last_peered":"2024-09-19T06:03:20.884896+0000","last_clean":"2024-09-19T06:03:20.884896+0000","last_became_active":"2024-09-19T06:01:28.813329+0000","last_became_peered":"2024-09-19T06:01:28.813329+0000","last_unstale":"2024-09-19T06:03:20.884896+0000","last_undegraded":"2024-09-19T06:03:20.884896+0000","last_fullsized":"2024-09-19T06:03:20.884896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:17.027393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722875+0000","last_change":"2024-09-19T06:01:28.799205+0000","last_active":"2024-09-19T06:03:20.722875+0000","last_peered":"2024-09-19T06:03:20.722875+0000","last_clean":"2024-09-19T06:03:20.722875+0000","last_became_active":"2024-09-19T06:01:28.798891+0000","last_became_peered":"2024-09-19T06:01:28.798891+0000","last_unstale":"2024-09-19T06:03:20.722875+0000","last_undegraded":"2024-09-19T06:03:20.722875+0000","last_fullsized":"2024-09-19T06:03:20.722875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:41.227721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627907+0000","last_change":"2024-09-19T06:01:28.795823+0000","last_active":"2024-09-19T06:03:20.627907+0000","last_peered":"2024-09-19T06:03:20.627907+0000","last_clean":"2024-09-19T06:03:20.627907+0000","last_became_active":"2024-09-19T06:01:28.795510+0000","last_became_peered":"2024-09-19T06:01:28.795510+0000","last_unstale":"2024-09-19T06:03:20.627907+0000","last_undegraded":"2024-09-19T06:03:20.627907+0000","last_fullsized":"2024-09-19T06:03:20.627907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:54.831610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723807+0000","last_change":"2024-09-19T06:01:28.799696+0000","last_active":"2024-09-19T06:03:20.723807+0000","last_peered":"2024-09-19T06:03:20.723807+0000","last_clean":"2024-09-19T06:03:20.723807+0000","last_became_active":"2024-09-19T06:01:28.799417+0000","last_became_peered":"2024-09-19T06:01:28.799417+0000","last_unstale":"2024-09-19T06:03:20.723807+0000","last_undegraded":"2024-09-19T06:03:20.723807+0000","last_fullsized":"2024-09-19T06:03:20.723807+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:02.330808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.896104+0000","last_change":"2024-09-19T06:01:28.807146+0000","last_active":"2024-09-19T06:03:20.896104+0000","last_peered":"2024-09-19T06:03:20.896104+0000","last_clean":"2024-09-19T06:03:20.896104+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:03:20.896104+0000","last_undegraded":"2024-09-19T06:03:20.896104+0000","last_fullsized":"2024-09-19T06:03:20.896104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:12.308421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887102+0000","last_change":"2024-09-19T06:01:28.803555+0000","last_active":"2024-09-19T06:03:20.887102+0000","last_peered":"2024-09-19T06:03:20.887102+0000","last_clean":"2024-09-19T06:03:20.887102+0000","last_became_active":"2024-09-19T06:01:28.803242+0000","last_became_peered":"2024-09-19T06:01:28.803242+0000","last_unstale":"2024-09-19T06:03:20.887102+0000","last_undegraded":"2024-09-19T06:03:20.887102+0000","last_fullsized":"2024-09-19T06:03:20.887102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:02.281152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626997+0000","last_change":"2024-09-19T06:01:28.804378+0000","last_active":"2024-09-19T06:03:20.626997+0000","last_peered":"2024-09-19T06:03:20.626997+0000","last_clean":"2024-09-19T06:03:20.626997+0000","last_became_active":"2024-09-19T06:01:28.804099+0000","last_became_peered":"2024-09-19T06:01:28.804099+0000","last_unstale":"2024-09-19T06:03:20.626997+0000","last_undegraded":"2024-09-19T06:03:20.626997+0000","last_fullsized":"2024-09-19T06:03:20.626997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:11.558384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887754+0000","last_change":"2024-09-19T06:01:28.799999+0000","last_active":"2024-09-19T06:03:20.887754+0000","last_peered":"2024-09-19T06:03:20.887754+0000","last_clean":"2024-09-19T06:03:20.887754+0000","last_became_active":"2024-09-19T06:01:28.799645+0000","last_became_peered":"2024-09-19T06:01:28.799645+0000","last_unstale":"2024-09-19T06:03:20.887754+0000","last_undegraded":"2024-09-19T06:03:20.887754+0000","last_fullsized":"2024-09-19T06:03:20.887754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:20.442119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.895777+0000","last_change":"2024-09-19T06:01:28.800143+0000","last_active":"2024-09-19T06:03:20.895777+0000","last_peered":"2024-09-19T06:03:20.895777+0000","last_clean":"2024-09-19T06:03:20.895777+0000","last_became_active":"2024-09-19T06:01:28.799830+0000","last_became_peered":"2024-09-19T06:01:28.799830+0000","last_unstale":"2024-09-19T06:03:20.895777+0000","last_undegraded":"2024-09-19T06:03:20.895777+0000","last_fullsized":"2024-09-19T06:03:20.895777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:43.824740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.895772+0000","last_change":"2024-09-19T06:01:28.806130+0000","last_active":"2024-09-19T06:03:20.895772+0000","last_peered":"2024-09-19T06:03:20.895772+0000","last_clean":"2024-09-19T06:03:20.895772+0000","last_became_active":"2024-09-19T06:01:28.805772+0000","last_became_peered":"2024-09-19T06:01:28.805772+0000","last_unstale":"2024-09-19T06:03:20.895772+0000","last_undegraded":"2024-09-19T06:03:20.895772+0000","last_fullsized":"2024-09-19T06:03:20.895772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:00.316311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887402+0000","last_change":"2024-09-19T06:01:39.837412+0000","last_active":"2024-09-19T06:03:20.887402+0000","last_peered":"2024-09-19T06:03:20.887402+0000","last_clean":"2024-09-19T06:03:20.887402+0000","last_became_active":"2024-09-19T06:01:39.837035+0000","last_became_peered":"2024-09-19T06:01:39.837035+0000","last_unstale":"2024-09-19T06:03:20.887402+0000","last_undegraded":"2024-09-19T06:03:20.887402+0000","last_fullsized":"2024-09-19T06:03:20.887402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:00.073884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722768+0000","last_change":"2024-09-19T06:01:28.800324+0000","last_active":"2024-09-19T06:03:20.722768+0000","last_peered":"2024-09-19T06:03:20.722768+0000","last_clean":"2024-09-19T06:03:20.722768+0000","last_became_active":"2024-09-19T06:01:28.799986+0000","last_became_peered":"2024-09-19T06:01:28.799986+0000","last_unstale":"2024-09-19T06:03:20.722768+0000","last_undegraded":"2024-09-19T06:03:20.722768+0000","last_fullsized":"2024-09-19T06:03:20.722768+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:41.963346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":151,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.978373+0000","last_change":"2024-09-19T06:01:39.844897+0000","last_active":"2024-09-19T06:03:20.978373+0000","last_peered":"2024-09-19T06:03:20.978373+0000","last_clean":"2024-09-19T06:03:20.978373+0000","last_became_active":"2024-09-19T06:01:39.838347+0000","last_became_peered":"2024-09-19T06:01:39.838347+0000","last_unstale":"2024-09-19T06:03:20.978373+0000","last_undegraded":"2024-09-19T06:03:20.978373+0000","last_fullsized":"2024-09-19T06:03:20.978373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:26.235227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2044,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628020+0000","last_change":"2024-09-19T06:01:28.802644+0000","last_active":"2024-09-19T06:03:20.628020+0000","last_peered":"2024-09-19T06:03:20.628020+0000","last_clean":"2024-09-19T06:03:20.628020+0000","last_became_active":"2024-09-19T06:01:28.802288+0000","last_became_peered":"2024-09-19T06:01:28.802288+0000","last_unstale":"2024-09-19T06:03:20.628020+0000","last_undegraded":"2024-09-19T06:03:20.628020+0000","last_fullsized":"2024-09-19T06:03:20.628020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:32.031860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732271+0000","last_change":"2024-09-19T06:01:26.781104+0000","last_active":"2024-09-19T06:03:20.732271+0000","last_peered":"2024-09-19T06:03:20.732271+0000","last_clean":"2024-09-19T06:03:20.732271+0000","last_became_active":"2024-09-19T06:01:26.780733+0000","last_became_peered":"2024-09-19T06:01:26.780733+0000","last_unstale":"2024-09-19T06:03:20.732271+0000","last_undegraded":"2024-09-19T06:03:20.732271+0000","last_fullsized":"2024-09-19T06:03:20.732271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:54.490867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626771+0000","last_change":"2024-09-19T06:01:26.782901+0000","last_active":"2024-09-19T06:03:20.626771+0000","last_peered":"2024-09-19T06:03:20.626771+0000","last_clean":"2024-09-19T06:03:20.626771+0000","last_became_active":"2024-09-19T06:01:26.782450+0000","last_became_peered":"2024-09-19T06:01:26.782450+0000","last_unstale":"2024-09-19T06:03:20.626771+0000","last_undegraded":"2024-09-19T06:03:20.626771+0000","last_fullsized":"2024-09-19T06:03:20.626771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:07.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722809+0000","last_change":"2024-09-19T06:01:28.804105+0000","last_active":"2024-09-19T06:03:20.722809+0000","last_peered":"2024-09-19T06:03:20.722809+0000","last_clean":"2024-09-19T06:03:20.722809+0000","last_became_active":"2024-09-19T06:01:28.797980+0000","last_became_peered":"2024-09-19T06:01:28.797980+0000","last_unstale":"2024-09-19T06:03:20.722809+0000","last_undegraded":"2024-09-19T06:03:20.722809+0000","last_fullsized":"2024-09-19T06:03:20.722809+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:50.130102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628139+0000","last_change":"2024-09-19T06:01:26.786923+0000","last_active":"2024-09-19T06:03:20.628139+0000","last_peered":"2024-09-19T06:03:20.628139+0000","last_clean":"2024-09-19T06:03:20.628139+0000","last_became_active":"2024-09-19T06:01:26.786542+0000","last_became_peered":"2024-09-19T06:01:26.786542+0000","last_unstale":"2024-09-19T06:03:20.628139+0000","last_undegraded":"2024-09-19T06:03:20.628139+0000","last_fullsized":"2024-09-19T06:03:20.628139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:30.032992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.720767+0000","last_change":"2024-09-19T06:01:28.802172+0000","last_active":"2024-09-19T06:03:20.720767+0000","last_peered":"2024-09-19T06:03:20.720767+0000","last_clean":"2024-09-19T06:03:20.720767+0000","last_became_active":"2024-09-19T06:01:28.801815+0000","last_became_peered":"2024-09-19T06:01:28.801815+0000","last_unstale":"2024-09-19T06:03:20.720767+0000","last_undegraded":"2024-09-19T06:03:20.720767+0000","last_fullsized":"2024-09-19T06:03:20.720767+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:09.587638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":230,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.977949+0000","last_change":"2024-09-19T06:01:26.777870+0000","last_active":"2024-09-19T06:03:20.977949+0000","last_peered":"2024-09-19T06:03:20.977949+0000","last_clean":"2024-09-19T06:03:20.977949+0000","last_became_active":"2024-09-19T06:01:26.777509+0000","last_became_peered":"2024-09-19T06:01:26.777509+0000","last_unstale":"2024-09-19T06:03:20.977949+0000","last_undegraded":"2024-09-19T06:03:20.977949+0000","last_fullsized":"2024-09-19T06:03:20.977949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:15.885911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884945+0000","last_change":"2024-09-19T06:01:28.812201+0000","last_active":"2024-09-19T06:03:20.884945+0000","last_peered":"2024-09-19T06:03:20.884945+0000","last_clean":"2024-09-19T06:03:20.884945+0000","last_became_active":"2024-09-19T06:01:28.811801+0000","last_became_peered":"2024-09-19T06:01:28.811801+0000","last_unstale":"2024-09-19T06:03:20.884945+0000","last_undegraded":"2024-09-19T06:03:20.884945+0000","last_fullsized":"2024-09-19T06:03:20.884945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:32.904787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628546+0000","last_change":"2024-09-19T06:01:26.787845+0000","last_active":"2024-09-19T06:03:20.628546+0000","last_peered":"2024-09-19T06:03:20.628546+0000","last_clean":"2024-09-19T06:03:20.628546+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:03:20.628546+0000","last_undegraded":"2024-09-19T06:03:20.628546+0000","last_fullsized":"2024-09-19T06:03:20.628546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.114865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.889521+0000","last_change":"2024-09-19T06:01:28.801523+0000","last_active":"2024-09-19T06:03:20.889521+0000","last_peered":"2024-09-19T06:03:20.889521+0000","last_clean":"2024-09-19T06:03:20.889521+0000","last_became_active":"2024-09-19T06:01:28.801205+0000","last_became_peered":"2024-09-19T06:01:28.801205+0000","last_unstale":"2024-09-19T06:03:20.889521+0000","last_undegraded":"2024-09-19T06:03:20.889521+0000","last_fullsized":"2024-09-19T06:03:20.889521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:10.314085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628080+0000","last_change":"2024-09-19T06:01:27.022172+0000","last_active":"2024-09-19T06:03:20.628080+0000","last_peered":"2024-09-19T06:03:20.628080+0000","last_clean":"2024-09-19T06:03:20.628080+0000","last_became_active":"2024-09-19T06:01:27.021568+0000","last_became_peered":"2024-09-19T06:01:27.021568+0000","last_unstale":"2024-09-19T06:03:20.628080+0000","last_undegraded":"2024-09-19T06:03:20.628080+0000","last_fullsized":"2024-09-19T06:03:20.628080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:18.031690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.885201+0000","last_change":"2024-09-19T06:01:39.839061+0000","last_active":"2024-09-19T06:03:20.885201+0000","last_peered":"2024-09-19T06:03:20.885201+0000","last_clean":"2024-09-19T06:03:20.885201+0000","last_became_active":"2024-09-19T06:01:39.838644+0000","last_became_peered":"2024-09-19T06:01:39.838644+0000","last_unstale":"2024-09-19T06:03:20.885201+0000","last_undegraded":"2024-09-19T06:03:20.885201+0000","last_fullsized":"2024-09-19T06:03:20.885201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:02.136222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'37","reported_seq":273,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.977666+0000","last_change":"2024-09-19T06:01:26.784781+0000","last_active":"2024-09-19T06:03:20.977666+0000","last_peered":"2024-09-19T06:03:20.977666+0000","last_clean":"2024-09-19T06:03:20.977666+0000","last_became_active":"2024-09-19T06:01:26.784284+0000","last_became_peered":"2024-09-19T06:01:26.784284+0000","last_unstale":"2024-09-19T06:03:20.977666+0000","last_undegraded":"2024-09-19T06:03:20.977666+0000","last_fullsized":"2024-09-19T06:03:20.977666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:13.063299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67016,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":212,"num_read_kb":170,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722979+0000","last_change":"2024-09-19T06:01:28.802604+0000","last_active":"2024-09-19T06:03:20.722979+0000","last_peered":"2024-09-19T06:03:20.722979+0000","last_clean":"2024-09-19T06:03:20.722979+0000","last_became_active":"2024-09-19T06:01:28.802285+0000","last_became_peered":"2024-09-19T06:01:28.802285+0000","last_unstale":"2024-09-19T06:03:20.722979+0000","last_undegraded":"2024-09-19T06:03:20.722979+0000","last_fullsized":"2024-09-19T06:03:20.722979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:57.177791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.895972+0000","last_change":"2024-09-19T06:01:26.783678+0000","last_active":"2024-09-19T06:03:20.895972+0000","last_peered":"2024-09-19T06:03:20.895972+0000","last_clean":"2024-09-19T06:03:20.895972+0000","last_became_active":"2024-09-19T06:01:26.783165+0000","last_became_peered":"2024-09-19T06:01:26.783165+0000","last_unstale":"2024-09-19T06:03:20.895972+0000","last_undegraded":"2024-09-19T06:03:20.895972+0000","last_fullsized":"2024-09-19T06:03:20.895972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:50.535733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626891+0000","last_change":"2024-09-19T06:01:28.808553+0000","last_active":"2024-09-19T06:03:20.626891+0000","last_peered":"2024-09-19T06:03:20.626891+0000","last_clean":"2024-09-19T06:03:20.626891+0000","last_became_active":"2024-09-19T06:01:28.808215+0000","last_became_peered":"2024-09-19T06:01:28.808215+0000","last_unstale":"2024-09-19T06:03:20.626891+0000","last_undegraded":"2024-09-19T06:03:20.626891+0000","last_fullsized":"2024-09-19T06:03:20.626891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:55.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627942+0000","last_change":"2024-09-19T06:01:26.787137+0000","last_active":"2024-09-19T06:03:20.627942+0000","last_peered":"2024-09-19T06:03:20.627942+0000","last_clean":"2024-09-19T06:03:20.627942+0000","last_became_active":"2024-09-19T06:01:26.786676+0000","last_became_peered":"2024-09-19T06:01:26.786676+0000","last_unstale":"2024-09-19T06:03:20.627942+0000","last_undegraded":"2024-09-19T06:03:20.627942+0000","last_fullsized":"2024-09-19T06:03:20.627942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.609969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628199+0000","last_change":"2024-09-19T06:01:28.812106+0000","last_active":"2024-09-19T06:03:20.628199+0000","last_peered":"2024-09-19T06:03:20.628199+0000","last_clean":"2024-09-19T06:03:20.628199+0000","last_became_active":"2024-09-19T06:01:28.811771+0000","last_became_peered":"2024-09-19T06:01:28.811771+0000","last_unstale":"2024-09-19T06:03:20.628199+0000","last_undegraded":"2024-09-19T06:03:20.628199+0000","last_fullsized":"2024-09-19T06:03:20.628199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:07:06.832685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'9","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.978122+0000","last_change":"2024-09-19T06:01:26.786746+0000","last_active":"2024-09-19T06:03:20.978122+0000","last_peered":"2024-09-19T06:03:20.978122+0000","last_clean":"2024-09-19T06:03:20.978122+0000","last_became_active":"2024-09-19T06:01:26.786414+0000","last_became_peered":"2024-09-19T06:01:26.786414+0000","last_unstale":"2024-09-19T06:03:20.978122+0000","last_undegraded":"2024-09-19T06:03:20.978122+0000","last_fullsized":"2024-09-19T06:03:20.978122+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:25.042796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627937+0000","last_change":"2024-09-19T06:01:28.807183+0000","last_active":"2024-09-19T06:03:20.627937+0000","last_peered":"2024-09-19T06:03:20.627937+0000","last_clean":"2024-09-19T06:03:20.627937+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:03:20.627937+0000","last_undegraded":"2024-09-19T06:03:20.627937+0000","last_fullsized":"2024-09-19T06:03:20.627937+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:23.030312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722771+0000","last_change":"2024-09-19T06:01:27.022409+0000","last_active":"2024-09-19T06:03:20.722771+0000","last_peered":"2024-09-19T06:03:20.722771+0000","last_clean":"2024-09-19T06:03:20.722771+0000","last_became_active":"2024-09-19T06:01:27.022034+0000","last_became_peered":"2024-09-19T06:01:27.022034+0000","last_unstale":"2024-09-19T06:03:20.722771+0000","last_undegraded":"2024-09-19T06:03:20.722771+0000","last_fullsized":"2024-09-19T06:03:20.722771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:08.663575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627932+0000","last_change":"2024-09-19T06:01:28.812079+0000","last_active":"2024-09-19T06:03:20.627932+0000","last_peered":"2024-09-19T06:03:20.627932+0000","last_clean":"2024-09-19T06:03:20.627932+0000","last_became_active":"2024-09-19T06:01:28.811606+0000","last_became_peered":"2024-09-19T06:01:28.811606+0000","last_unstale":"2024-09-19T06:03:20.627932+0000","last_undegraded":"2024-09-19T06:03:20.627932+0000","last_fullsized":"2024-09-19T06:03:20.627932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:49.434235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723153+0000","last_change":"2024-09-19T06:01:26.783823+0000","last_active":"2024-09-19T06:03:20.723153+0000","last_peered":"2024-09-19T06:03:20.723153+0000","last_clean":"2024-09-19T06:03:20.723153+0000","last_became_active":"2024-09-19T06:01:26.783209+0000","last_became_peered":"2024-09-19T06:01:26.783209+0000","last_unstale":"2024-09-19T06:03:20.723153+0000","last_undegraded":"2024-09-19T06:03:20.723153+0000","last_fullsized":"2024-09-19T06:03:20.723153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:38.833417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887395+0000","last_change":"2024-09-19T06:01:39.837462+0000","last_active":"2024-09-19T06:03:20.887395+0000","last_peered":"2024-09-19T06:03:20.887395+0000","last_clean":"2024-09-19T06:03:20.887395+0000","last_became_active":"2024-09-19T06:01:39.837082+0000","last_became_peered":"2024-09-19T06:01:39.837082+0000","last_unstale":"2024-09-19T06:03:20.887395+0000","last_undegraded":"2024-09-19T06:03:20.887395+0000","last_fullsized":"2024-09-19T06:03:20.887395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:47.421017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627424+0000","last_change":"2024-09-19T06:01:26.779992+0000","last_active":"2024-09-19T06:03:20.627424+0000","last_peered":"2024-09-19T06:03:20.627424+0000","last_clean":"2024-09-19T06:03:20.627424+0000","last_became_active":"2024-09-19T06:01:26.779609+0000","last_became_peered":"2024-09-19T06:01:26.779609+0000","last_unstale":"2024-09-19T06:03:20.627424+0000","last_undegraded":"2024-09-19T06:03:20.627424+0000","last_fullsized":"2024-09-19T06:03:20.627424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:48.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628684+0000","last_change":"2024-09-19T06:01:28.798827+0000","last_active":"2024-09-19T06:03:20.628684+0000","last_peered":"2024-09-19T06:03:20.628684+0000","last_clean":"2024-09-19T06:03:20.628684+0000","last_became_active":"2024-09-19T06:01:28.798457+0000","last_became_peered":"2024-09-19T06:01:28.798457+0000","last_unstale":"2024-09-19T06:03:20.628684+0000","last_undegraded":"2024-09-19T06:03:20.628684+0000","last_fullsized":"2024-09-19T06:03:20.628684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:41.111051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'41","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.850980+0000","last_change":"2024-09-19T06:01:26.775691+0000","last_active":"2024-09-19T06:03:20.850980+0000","last_peered":"2024-09-19T06:03:20.850980+0000","last_clean":"2024-09-19T06:03:20.850980+0000","last_became_active":"2024-09-19T06:01:26.775183+0000","last_became_peered":"2024-09-19T06:01:26.775183+0000","last_unstale":"2024-09-19T06:03:20.850980+0000","last_undegraded":"2024-09-19T06:03:20.850980+0000","last_fullsized":"2024-09-19T06:03:20.850980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:34.712968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628457+0000","last_change":"2024-09-19T06:01:28.809459+0000","last_active":"2024-09-19T06:03:20.628457+0000","last_peered":"2024-09-19T06:03:20.628457+0000","last_clean":"2024-09-19T06:03:20.628457+0000","last_became_active":"2024-09-19T06:01:28.809046+0000","last_became_peered":"2024-09-19T06:01:28.809046+0000","last_unstale":"2024-09-19T06:03:20.628457+0000","last_undegraded":"2024-09-19T06:03:20.628457+0000","last_fullsized":"2024-09-19T06:03:20.628457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:56.996190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'10","reported_seq":132,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.984256+0000","last_change":"2024-09-19T06:01:39.843932+0000","last_active":"2024-09-19T06:03:20.984256+0000","last_peered":"2024-09-19T06:03:20.984256+0000","last_clean":"2024-09-19T06:03:20.984256+0000","last_became_active":"2024-09-19T06:01:39.836008+0000","last_became_peered":"2024-09-19T06:01:39.836008+0000","last_unstale":"2024-09-19T06:03:20.984256+0000","last_undegraded":"2024-09-19T06:03:20.984256+0000","last_fullsized":"2024-09-19T06:03:20.984256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:58.798859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2985,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628540+0000","last_change":"2024-09-19T06:01:28.814181+0000","last_active":"2024-09-19T06:03:20.628540+0000","last_peered":"2024-09-19T06:03:20.628540+0000","last_clean":"2024-09-19T06:03:20.628540+0000","last_became_active":"2024-09-19T06:01:28.813817+0000","last_became_peered":"2024-09-19T06:01:28.813817+0000","last_unstale":"2024-09-19T06:03:20.628540+0000","last_undegraded":"2024-09-19T06:03:20.628540+0000","last_fullsized":"2024-09-19T06:03:20.628540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:40.057863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628100+0000","last_change":"2024-09-19T06:00:17.624509+0000","last_active":"2024-09-19T06:03:20.628100+0000","last_peered":"2024-09-19T06:03:20.628100+0000","last_clean":"2024-09-19T06:03:20.628100+0000","last_became_active":"2024-09-19T06:00:17.623943+0000","last_became_peered":"2024-09-19T06:00:17.623943+0000","last_unstale":"2024-09-19T06:03:20.628100+0000","last_undegraded":"2024-09-19T06:03:20.628100+0000","last_fullsized":"2024-09-19T06:03:20.628100+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_clean_scrub_stamp":"2024-09-19T05:56:55.414488+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:39.116345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.893738+0000","last_change":"2024-09-19T06:01:26.783896+0000","last_active":"2024-09-19T06:03:20.893738+0000","last_peered":"2024-09-19T06:03:20.893738+0000","last_clean":"2024-09-19T06:03:20.893738+0000","last_became_active":"2024-09-19T06:01:26.783332+0000","last_became_peered":"2024-09-19T06:01:26.783332+0000","last_unstale":"2024-09-19T06:03:20.893738+0000","last_undegraded":"2024-09-19T06:03:20.893738+0000","last_fullsized":"2024-09-19T06:03:20.893738+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:33.072763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723257+0000","last_change":"2024-09-19T06:01:28.807756+0000","last_active":"2024-09-19T06:03:20.723257+0000","last_peered":"2024-09-19T06:03:20.723257+0000","last_clean":"2024-09-19T06:03:20.723257+0000","last_became_active":"2024-09-19T06:01:28.807077+0000","last_became_peered":"2024-09-19T06:01:28.807077+0000","last_unstale":"2024-09-19T06:03:20.723257+0000","last_undegraded":"2024-09-19T06:03:20.723257+0000","last_fullsized":"2024-09-19T06:03:20.723257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:20.784199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628079+0000","last_change":"2024-09-19T06:01:26.784830+0000","last_active":"2024-09-19T06:03:20.628079+0000","last_peered":"2024-09-19T06:03:20.628079+0000","last_clean":"2024-09-19T06:03:20.628079+0000","last_became_active":"2024-09-19T06:01:26.783887+0000","last_became_peered":"2024-09-19T06:01:26.783887+0000","last_unstale":"2024-09-19T06:03:20.628079+0000","last_undegraded":"2024-09-19T06:03:20.628079+0000","last_fullsized":"2024-09-19T06:03:20.628079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:33:38.255169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626764+0000","last_change":"2024-09-19T06:01:39.837354+0000","last_active":"2024-09-19T06:03:20.626764+0000","last_peered":"2024-09-19T06:03:20.626764+0000","last_clean":"2024-09-19T06:03:20.626764+0000","last_became_active":"2024-09-19T06:01:39.836930+0000","last_became_peered":"2024-09-19T06:01:39.836930+0000","last_unstale":"2024-09-19T06:03:20.626764+0000","last_undegraded":"2024-09-19T06:03:20.626764+0000","last_fullsized":"2024-09-19T06:03:20.626764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:17.571834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":131,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626969+0000","last_change":"2024-09-19T06:01:27.021753+0000","last_active":"2024-09-19T06:03:20.626969+0000","last_peered":"2024-09-19T06:03:20.626969+0000","last_clean":"2024-09-19T06:03:20.626969+0000","last_became_active":"2024-09-19T06:01:27.021346+0000","last_became_peered":"2024-09-19T06:01:27.021346+0000","last_unstale":"2024-09-19T06:03:20.626969+0000","last_undegraded":"2024-09-19T06:03:20.626969+0000","last_fullsized":"2024-09-19T06:03:20.626969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:07.291283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627909+0000","last_change":"2024-09-19T06:01:28.803103+0000","last_active":"2024-09-19T06:03:20.627909+0000","last_peered":"2024-09-19T06:03:20.627909+0000","last_clean":"2024-09-19T06:03:20.627909+0000","last_became_active":"2024-09-19T06:01:28.802806+0000","last_became_peered":"2024-09-19T06:01:28.802806+0000","last_unstale":"2024-09-19T06:03:20.627909+0000","last_undegraded":"2024-09-19T06:03:20.627909+0000","last_fullsized":"2024-09-19T06:03:20.627909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:33.329231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'8","reported_seq":136,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628270+0000","last_change":"2024-09-19T06:01:26.786605+0000","last_active":"2024-09-19T06:03:20.628270+0000","last_peered":"2024-09-19T06:03:20.628270+0000","last_clean":"2024-09-19T06:03:20.628270+0000","last_became_active":"2024-09-19T06:01:26.786278+0000","last_became_peered":"2024-09-19T06:01:26.786278+0000","last_unstale":"2024-09-19T06:03:20.628270+0000","last_undegraded":"2024-09-19T06:03:20.628270+0000","last_fullsized":"2024-09-19T06:03:20.628270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.581502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732180+0000","last_change":"2024-09-19T06:01:28.808152+0000","last_active":"2024-09-19T06:03:20.732180+0000","last_peered":"2024-09-19T06:03:20.732180+0000","last_clean":"2024-09-19T06:03:20.732180+0000","last_became_active":"2024-09-19T06:01:28.807788+0000","last_became_peered":"2024-09-19T06:01:28.807788+0000","last_unstale":"2024-09-19T06:03:20.732180+0000","last_undegraded":"2024-09-19T06:03:20.732180+0000","last_fullsized":"2024-09-19T06:03:20.732180+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:42.877539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.720863+0000","last_change":"2024-09-19T06:01:27.022719+0000","last_active":"2024-09-19T06:03:20.720863+0000","last_peered":"2024-09-19T06:03:20.720863+0000","last_clean":"2024-09-19T06:03:20.720863+0000","last_became_active":"2024-09-19T06:01:27.022263+0000","last_became_peered":"2024-09-19T06:01:27.022263+0000","last_unstale":"2024-09-19T06:03:20.720863+0000","last_undegraded":"2024-09-19T06:03:20.720863+0000","last_fullsized":"2024-09-19T06:03:20.720863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:45.158229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732175+0000","last_change":"2024-09-19T06:01:28.809869+0000","last_active":"2024-09-19T06:03:20.732175+0000","last_peered":"2024-09-19T06:03:20.732175+0000","last_clean":"2024-09-19T06:03:20.732175+0000","last_became_active":"2024-09-19T06:01:28.809548+0000","last_became_peered":"2024-09-19T06:01:28.809548+0000","last_unstale":"2024-09-19T06:03:20.732175+0000","last_undegraded":"2024-09-19T06:03:20.732175+0000","last_fullsized":"2024-09-19T06:03:20.732175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:41.587846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887443+0000","last_change":"2024-09-19T06:01:39.846740+0000","last_active":"2024-09-19T06:03:20.887443+0000","last_peered":"2024-09-19T06:03:20.887443+0000","last_clean":"2024-09-19T06:03:20.887443+0000","last_became_active":"2024-09-19T06:01:39.836378+0000","last_became_peered":"2024-09-19T06:01:39.836378+0000","last_unstale":"2024-09-19T06:03:20.887443+0000","last_undegraded":"2024-09-19T06:03:20.887443+0000","last_fullsized":"2024-09-19T06:03:20.887443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.064918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627124+0000","last_change":"2024-09-19T06:01:28.807796+0000","last_active":"2024-09-19T06:03:20.627124+0000","last_peered":"2024-09-19T06:03:20.627124+0000","last_clean":"2024-09-19T06:03:20.627124+0000","last_became_active":"2024-09-19T06:01:28.807503+0000","last_became_peered":"2024-09-19T06:01:28.807503+0000","last_unstale":"2024-09-19T06:03:20.627124+0000","last_undegraded":"2024-09-19T06:03:20.627124+0000","last_fullsized":"2024-09-19T06:03:20.627124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:43.472647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'51","reported_seq":98,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.853902+0000","last_change":"2024-09-19T06:01:26.783517+0000","last_active":"2024-09-19T06:03:20.853902+0000","last_peered":"2024-09-19T06:03:20.853902+0000","last_clean":"2024-09-19T06:03:20.853902+0000","last_became_active":"2024-09-19T06:01:26.783164+0000","last_became_peered":"2024-09-19T06:01:26.783164+0000","last_unstale":"2024-09-19T06:03:20.853902+0000","last_undegraded":"2024-09-19T06:03:20.853902+0000","last_fullsized":"2024-09-19T06:03:20.853902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:57.609192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.724088+0000","last_change":"2024-09-19T06:01:28.806548+0000","last_active":"2024-09-19T06:03:20.724088+0000","last_peered":"2024-09-19T06:03:20.724088+0000","last_clean":"2024-09-19T06:03:20.724088+0000","last_became_active":"2024-09-19T06:01:28.805724+0000","last_became_peered":"2024-09-19T06:01:28.805724+0000","last_unstale":"2024-09-19T06:03:20.724088+0000","last_undegraded":"2024-09-19T06:03:20.724088+0000","last_fullsized":"2024-09-19T06:03:20.724088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:14.360928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.893742+0000","last_change":"2024-09-19T06:01:27.128213+0000","last_active":"2024-09-19T06:03:20.893742+0000","last_peered":"2024-09-19T06:03:20.893742+0000","last_clean":"2024-09-19T06:03:20.893742+0000","last_became_active":"2024-09-19T06:01:27.127737+0000","last_became_peered":"2024-09-19T06:01:27.127737+0000","last_unstale":"2024-09-19T06:03:20.893742+0000","last_undegraded":"2024-09-19T06:03:20.893742+0000","last_fullsized":"2024-09-19T06:03:20.893742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:33:35.107152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.888504+0000","last_change":"2024-09-19T06:01:39.839027+0000","last_active":"2024-09-19T06:03:20.888504+0000","last_peered":"2024-09-19T06:03:20.888504+0000","last_clean":"2024-09-19T06:03:20.888504+0000","last_became_active":"2024-09-19T06:01:39.838601+0000","last_became_peered":"2024-09-19T06:01:39.838601+0000","last_unstale":"2024-09-19T06:03:20.888504+0000","last_undegraded":"2024-09-19T06:03:20.888504+0000","last_fullsized":"2024-09-19T06:03:20.888504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:09.864997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.885207+0000","last_change":"2024-09-19T06:01:27.022438+0000","last_active":"2024-09-19T06:03:20.885207+0000","last_peered":"2024-09-19T06:03:20.885207+0000","last_clean":"2024-09-19T06:03:20.885207+0000","last_became_active":"2024-09-19T06:01:27.021953+0000","last_became_peered":"2024-09-19T06:01:27.021953+0000","last_unstale":"2024-09-19T06:03:20.885207+0000","last_undegraded":"2024-09-19T06:03:20.885207+0000","last_fullsized":"2024-09-19T06:03:20.885207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:24.383178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722756+0000","last_change":"2024-09-19T06:01:28.803807+0000","last_active":"2024-09-19T06:03:20.722756+0000","last_peered":"2024-09-19T06:03:20.722756+0000","last_clean":"2024-09-19T06:03:20.722756+0000","last_became_active":"2024-09-19T06:01:28.803486+0000","last_became_peered":"2024-09-19T06:01:28.803486+0000","last_unstale":"2024-09-19T06:03:20.722756+0000","last_undegraded":"2024-09-19T06:03:20.722756+0000","last_fullsized":"2024-09-19T06:03:20.722756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:26:17.920089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723040+0000","last_change":"2024-09-19T06:01:26.777171+0000","last_active":"2024-09-19T06:03:20.723040+0000","last_peered":"2024-09-19T06:03:20.723040+0000","last_clean":"2024-09-19T06:03:20.723040+0000","last_became_active":"2024-09-19T06:01:26.776818+0000","last_became_peered":"2024-09-19T06:01:26.776818+0000","last_unstale":"2024-09-19T06:03:20.723040+0000","last_undegraded":"2024-09-19T06:03:20.723040+0000","last_fullsized":"2024-09-19T06:03:20.723040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887551+0000","last_change":"2024-09-19T06:01:28.799266+0000","last_active":"2024-09-19T06:03:20.887551+0000","last_peered":"2024-09-19T06:03:20.887551+0000","last_clean":"2024-09-19T06:03:20.887551+0000","last_became_active":"2024-09-19T06:01:28.798969+0000","last_became_peered":"2024-09-19T06:01:28.798969+0000","last_unstale":"2024-09-19T06:03:20.887551+0000","last_undegraded":"2024-09-19T06:03:20.887551+0000","last_fullsized":"2024-09-19T06:03:20.887551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:34.064795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.896088+0000","last_change":"2024-09-19T06:01:26.781680+0000","last_active":"2024-09-19T06:03:20.896088+0000","last_peered":"2024-09-19T06:03:20.896088+0000","last_clean":"2024-09-19T06:03:20.896088+0000","last_became_active":"2024-09-19T06:01:26.781035+0000","last_became_peered":"2024-09-19T06:01:26.781035+0000","last_unstale":"2024-09-19T06:03:20.896088+0000","last_undegraded":"2024-09-19T06:03:20.896088+0000","last_fullsized":"2024-09-19T06:03:20.896088+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:38.287018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723555+0000","last_change":"2024-09-19T06:01:28.802970+0000","last_active":"2024-09-19T06:03:20.723555+0000","last_peered":"2024-09-19T06:03:20.723555+0000","last_clean":"2024-09-19T06:03:20.723555+0000","last_became_active":"2024-09-19T06:01:28.802587+0000","last_became_peered":"2024-09-19T06:01:28.802587+0000","last_unstale":"2024-09-19T06:03:20.723555+0000","last_undegraded":"2024-09-19T06:03:20.723555+0000","last_fullsized":"2024-09-19T06:03:20.723555+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722866+0000","last_change":"2024-09-19T06:01:26.788140+0000","last_active":"2024-09-19T06:03:20.722866+0000","last_peered":"2024-09-19T06:03:20.722866+0000","last_clean":"2024-09-19T06:03:20.722866+0000","last_became_active":"2024-09-19T06:01:26.787603+0000","last_became_peered":"2024-09-19T06:01:26.787603+0000","last_unstale":"2024-09-19T06:03:20.722866+0000","last_undegraded":"2024-09-19T06:03:20.722866+0000","last_fullsized":"2024-09-19T06:03:20.722866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:13.880656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628239+0000","last_change":"2024-09-19T06:01:28.813144+0000","last_active":"2024-09-19T06:03:20.628239+0000","last_peered":"2024-09-19T06:03:20.628239+0000","last_clean":"2024-09-19T06:03:20.628239+0000","last_became_active":"2024-09-19T06:01:28.812792+0000","last_became_peered":"2024-09-19T06:01:28.812792+0000","last_unstale":"2024-09-19T06:03:20.628239+0000","last_undegraded":"2024-09-19T06:03:20.628239+0000","last_fullsized":"2024-09-19T06:03:20.628239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:35.896240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628078+0000","last_change":"2024-09-19T06:01:28.812086+0000","last_active":"2024-09-19T06:03:20.628078+0000","last_peered":"2024-09-19T06:03:20.628078+0000","last_clean":"2024-09-19T06:03:20.628078+0000","last_became_active":"2024-09-19T06:01:28.811696+0000","last_became_peered":"2024-09-19T06:01:28.811696+0000","last_unstale":"2024-09-19T06:03:20.628078+0000","last_undegraded":"2024-09-19T06:03:20.628078+0000","last_fullsized":"2024-09-19T06:03:20.628078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:32.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887254+0000","last_change":"2024-09-19T06:01:39.842250+0000","last_active":"2024-09-19T06:03:20.887254+0000","last_peered":"2024-09-19T06:03:20.887254+0000","last_clean":"2024-09-19T06:03:20.887254+0000","last_became_active":"2024-09-19T06:01:39.835761+0000","last_became_peered":"2024-09-19T06:01:39.835761+0000","last_unstale":"2024-09-19T06:03:20.887254+0000","last_undegraded":"2024-09-19T06:03:20.887254+0000","last_fullsized":"2024-09-19T06:03:20.887254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:51.026349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884914+0000","last_change":"2024-09-19T06:01:28.812983+0000","last_active":"2024-09-19T06:03:20.884914+0000","last_peered":"2024-09-19T06:03:20.884914+0000","last_clean":"2024-09-19T06:03:20.884914+0000","last_became_active":"2024-09-19T06:01:28.812257+0000","last_became_peered":"2024-09-19T06:01:28.812257+0000","last_unstale":"2024-09-19T06:03:20.884914+0000","last_undegraded":"2024-09-19T06:03:20.884914+0000","last_fullsized":"2024-09-19T06:03:20.884914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:41:42.059736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723024+0000","last_change":"2024-09-19T06:01:26.787869+0000","last_active":"2024-09-19T06:03:20.723024+0000","last_peered":"2024-09-19T06:03:20.723024+0000","last_clean":"2024-09-19T06:03:20.723024+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:03:20.723024+0000","last_undegraded":"2024-09-19T06:03:20.723024+0000","last_fullsized":"2024-09-19T06:03:20.723024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:02.869878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.889703+0000","last_change":"2024-09-19T06:01:28.811986+0000","last_active":"2024-09-19T06:03:20.889703+0000","last_peered":"2024-09-19T06:03:20.889703+0000","last_clean":"2024-09-19T06:03:20.889703+0000","last_became_active":"2024-09-19T06:01:28.811565+0000","last_became_peered":"2024-09-19T06:01:28.811565+0000","last_unstale":"2024-09-19T06:03:20.889703+0000","last_undegraded":"2024-09-19T06:03:20.889703+0000","last_fullsized":"2024-09-19T06:03:20.889703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:01.324985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723946+0000","last_change":"2024-09-19T06:01:27.021683+0000","last_active":"2024-09-19T06:03:20.723946+0000","last_peered":"2024-09-19T06:03:20.723946+0000","last_clean":"2024-09-19T06:03:20.723946+0000","last_became_active":"2024-09-19T06:01:27.021358+0000","last_became_peered":"2024-09-19T06:01:27.021358+0000","last_unstale":"2024-09-19T06:03:20.723946+0000","last_undegraded":"2024-09-19T06:03:20.723946+0000","last_fullsized":"2024-09-19T06:03:20.723946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:29.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723619+0000","last_change":"2024-09-19T06:01:28.798963+0000","last_active":"2024-09-19T06:03:20.723619+0000","last_peered":"2024-09-19T06:03:20.723619+0000","last_clean":"2024-09-19T06:03:20.723619+0000","last_became_active":"2024-09-19T06:01:28.797571+0000","last_became_peered":"2024-09-19T06:01:28.797571+0000","last_unstale":"2024-09-19T06:03:20.723619+0000","last_undegraded":"2024-09-19T06:03:20.723619+0000","last_fullsized":"2024-09-19T06:03:20.723619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:43.867609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.721283+0000","last_change":"2024-09-19T06:01:26.786427+0000","last_active":"2024-09-19T06:03:20.721283+0000","last_peered":"2024-09-19T06:03:20.721283+0000","last_clean":"2024-09-19T06:03:20.721283+0000","last_became_active":"2024-09-19T06:01:26.786092+0000","last_became_peered":"2024-09-19T06:01:26.786092+0000","last_unstale":"2024-09-19T06:03:20.721283+0000","last_undegraded":"2024-09-19T06:03:20.721283+0000","last_fullsized":"2024-09-19T06:03:20.721283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:49.144366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603942,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1149,"num_read_kb":694,"num_write":387,"num_write_kb":1474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1811826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":449661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41236,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703916,"statfs":{"total":95995035648,"available":95952809984,"internally_reserved":0,"allocated":1773568,"data_stored":1238219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939694,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":438118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36648,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708504,"statfs":{"total":95995035648,"available":95957508096,"internally_reserved":0,"allocated":1331200,"data_stored":839982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397559,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1548288,"data_stored":1043460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":447828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1695744,"data_stored":1172287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":450126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1482752,"data_stored":971440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40496,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704656,"statfs":{"total":95995035648,"available":95953567744,"internally_reserved":0,"allocated":1011712,"data_stored":513156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40500,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704652,"statfs":{"total":95995035648,"available":95953563648,"internally_reserved":0,"allocated":1015808,"data_stored":518742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":440203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":201774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:03:25.176 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T06:03:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.314+0000 7fc7e461e640 1 -- 172.21.15.5:0/1234734107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc102cf0 msgr2=0x7fc7dc1030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.314+0000 7fc7e461e640 1 --2- 172.21.15.5:0/1234734107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc102cf0 0x7fc7dc1030d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009ca0 tx=0x7fc7d802a950 comp rx=0 tx=0).stop 2024-09-19T06:03:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.314+0000 7fc7e461e640 1 -- 172.21.15.5:0/1234734107 shutdown_connections 2024-09-19T06:03:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.314+0000 7fc7e461e640 1 --2- 172.21.15.5:0/1234734107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7dc10c0c0 0x7fc7dc10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.314+0000 7fc7e461e640 1 --2- 172.21.15.5:0/1234734107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7dc103610 0x7fc7dc10baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.314+0000 7fc7e461e640 1 --2- 172.21.15.5:0/1234734107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc102cf0 0x7fc7dc1030d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.314+0000 7fc7e461e640 1 -- 172.21.15.5:0/1234734107 >> 172.21.15.5:0/1234734107 conn(0x7fc7dc077a10 msgr2=0x7fc7dc0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.314+0000 7fc7e461e640 1 -- 172.21.15.5:0/1234734107 shutdown_connections 2024-09-19T06:03:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.314+0000 7fc7e461e640 1 -- 172.21.15.5:0/1234734107 wait complete. 2024-09-19T06:03:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.314+0000 7fc7e461e640 1 Processor -- start 2024-09-19T06:03:25.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.314+0000 7fc7e461e640 1 -- start start 2024-09-19T06:03:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e461e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7dc102cf0 0x7fc7dc1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e461e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc103610 0x7fc7dc198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e461e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7dc10c0c0 0x7fc7dc19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e461e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc111d70 con 0x7fc7dc103610 2024-09-19T06:03:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e461e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc111bf0 con 0x7fc7dc102cf0 2024-09-19T06:03:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e461e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc111ef0 con 0x7fc7dc10c0c0 2024-09-19T06:03:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e1bb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc103610 0x7fc7dc198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e1bb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc103610 0x7fc7dc198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37938/0 (socket says 172.21.15.5:37938) 2024-09-19T06:03:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e1bb9640 1 -- 172.21.15.5:0/3077930529 learned_addr learned my addr 172.21.15.5:0/3077930529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e23ba640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7dc102cf0 0x7fc7dc1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e2bbb640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7dc10c0c0 0x7fc7dc19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e1bb9640 1 -- 172.21.15.5:0/3077930529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7dc10c0c0 msgr2=0x7fc7dc19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e1bb9640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7dc10c0c0 0x7fc7dc19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e1bb9640 1 -- 172.21.15.5:0/3077930529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7dc102cf0 msgr2=0x7fc7dc1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e1bb9640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7dc102cf0 0x7fc7dc1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e1bb9640 1 -- 172.21.15.5:0/3077930529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7dc19d2d0 con 0x7fc7dc103610 2024-09-19T06:03:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e23ba640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7dc102cf0 0x7fc7dc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e1bb9640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc103610 0x7fc7dc198820 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00e9f0 tx=0x7fc7cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7d37fe640 1 -- 172.21.15.5:0/3077930529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc00cd80 con 0x7fc7dc103610 2024-09-19T06:03:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7d37fe640 1 -- 172.21.15.5:0/3077930529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7cc0108e0 con 0x7fc7dc103610 2024-09-19T06:03:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e461e640 1 -- 172.21.15.5:0/3077930529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7dc19d5c0 con 0x7fc7dc103610 2024-09-19T06:03:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7d37fe640 1 -- 172.21.15.5:0/3077930529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc018920 con 0x7fc7dc103610 2024-09-19T06:03:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.315+0000 7fc7e461e640 1 -- 172.21.15.5:0/3077930529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7dc1a4e50 con 0x7fc7dc103610 2024-09-19T06:03:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.316+0000 7fc7e461e640 1 -- 172.21.15.5:0/3077930529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7dc1030d0 con 0x7fc7dc103610 2024-09-19T06:03:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.317+0000 7fc7d37fe640 1 -- 172.21.15.5:0/3077930529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7fc7cc010a90 con 0x7fc7dc103610 2024-09-19T06:03:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.318+0000 7fc7d37fe640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc7c80a3450 0x7fc7c80a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.318+0000 7fc7e23ba640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc7c80a3450 0x7fc7c80a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.318+0000 7fc7d37fe640 1 -- 172.21.15.5:0/3077930529 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fc7cc0d01e0 con 0x7fc7dc103610 2024-09-19T06:03:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.319+0000 7fc7e23ba640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc7c80a3450 0x7fc7c80a5910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8000f80 tx=0x7fc7d802fc20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.322+0000 7fc7d37fe640 1 -- 172.21.15.5:0/3077930529 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc7cc09c3b0 con 0x7fc7dc103610 2024-09-19T06:03:25.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.495+0000 7fc7e461e640 1 -- 172.21.15.5:0/3077930529 --> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7dc062630 con 0x7fc7c80a3450 2024-09-19T06:03:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.501+0000 7fc7d37fe640 1 -- 172.21.15.5:0/3077930529 <== mgr.15240 v2:172.21.15.5:6800/2837801016 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257786 (secure 0 0 0) 0x7fc7dc062630 con 0x7fc7c80a3450 2024-09-19T06:03:25.507 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:03:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-19T06:03:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.504+0000 7fc7e461e640 1 -- 172.21.15.5:0/3077930529 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc7c80a3450 msgr2=0x7fc7c80a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.504+0000 7fc7e461e640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc7c80a3450 0x7fc7c80a5910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8000f80 tx=0x7fc7d802fc20 comp rx=0 tx=0).stop 2024-09-19T06:03:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.504+0000 7fc7e461e640 1 -- 172.21.15.5:0/3077930529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc103610 msgr2=0x7fc7dc198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.504+0000 7fc7e461e640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc103610 0x7fc7dc198820 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00e9f0 tx=0x7fc7cc00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.504+0000 7fc7e461e640 1 -- 172.21.15.5:0/3077930529 shutdown_connections 2024-09-19T06:03:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.504+0000 7fc7e461e640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc7c80a3450 0x7fc7c80a5910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.504+0000 7fc7e461e640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7dc10c0c0 0x7fc7dc19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.504+0000 7fc7e461e640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc103610 0x7fc7dc198820 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.504+0000 7fc7e461e640 1 --2- 172.21.15.5:0/3077930529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7dc102cf0 0x7fc7dc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.504+0000 7fc7e461e640 1 -- 172.21.15.5:0/3077930529 >> 172.21.15.5:0/3077930529 conn(0x7fc7dc077a10 msgr2=0x7fc7dc0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.504+0000 7fc7e461e640 1 -- 172.21.15.5:0/3077930529 shutdown_connections 2024-09-19T06:03:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.505+0000 7fc7e461e640 1 -- 172.21.15.5:0/3077930529 wait complete. 2024-09-19T06:03:25.517 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T06:03:24.640412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194544,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1285,"num_read_kb":813,"num_write":570,"num_write_kb":4414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":950,"ondisk_log_size":950,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392800,"kb_used_data":14224,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549024,"statfs":{"total":1151940427776,"available":1151538200576,"internally_reserved":0,"allocated":14565376,"data_stored":8523222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.982108"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887438+0000","last_change":"2024-09-19T06:01:28.812722+0000","last_active":"2024-09-19T06:03:20.887438+0000","last_peered":"2024-09-19T06:03:20.887438+0000","last_clean":"2024-09-19T06:03:20.887438+0000","last_became_active":"2024-09-19T06:01:28.812283+0000","last_became_peered":"2024-09-19T06:01:28.812283+0000","last_unstale":"2024-09-19T06:03:20.887438+0000","last_undegraded":"2024-09-19T06:03:20.887438+0000","last_fullsized":"2024-09-19T06:03:20.887438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:59.146636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.888342+0000","last_change":"2024-09-19T06:01:28.805210+0000","last_active":"2024-09-19T06:03:20.888342+0000","last_peered":"2024-09-19T06:03:20.888342+0000","last_clean":"2024-09-19T06:03:20.888342+0000","last_became_active":"2024-09-19T06:01:28.804906+0000","last_became_peered":"2024-09-19T06:01:28.804906+0000","last_unstale":"2024-09-19T06:03:20.888342+0000","last_undegraded":"2024-09-19T06:03:20.888342+0000","last_fullsized":"2024-09-19T06:03:20.888342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:00.227101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.889300+0000","last_change":"2024-09-19T06:01:28.811909+0000","last_active":"2024-09-19T06:03:20.889300+0000","last_peered":"2024-09-19T06:03:20.889300+0000","last_clean":"2024-09-19T06:03:20.889300+0000","last_became_active":"2024-09-19T06:01:28.811547+0000","last_became_peered":"2024-09-19T06:01:28.811547+0000","last_unstale":"2024-09-19T06:03:20.889300+0000","last_undegraded":"2024-09-19T06:03:20.889300+0000","last_fullsized":"2024-09-19T06:03:20.889300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:50.022439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.893221+0000","last_change":"2024-09-19T06:01:28.806782+0000","last_active":"2024-09-19T06:03:20.893221+0000","last_peered":"2024-09-19T06:03:20.893221+0000","last_clean":"2024-09-19T06:03:20.893221+0000","last_became_active":"2024-09-19T06:01:28.805520+0000","last_became_peered":"2024-09-19T06:01:28.805520+0000","last_unstale":"2024-09-19T06:03:20.893221+0000","last_undegraded":"2024-09-19T06:03:20.893221+0000","last_fullsized":"2024-09-19T06:03:20.893221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:01.652669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722796+0000","last_change":"2024-09-19T06:01:28.809971+0000","last_active":"2024-09-19T06:03:20.722796+0000","last_peered":"2024-09-19T06:03:20.722796+0000","last_clean":"2024-09-19T06:03:20.722796+0000","last_became_active":"2024-09-19T06:01:28.809593+0000","last_became_peered":"2024-09-19T06:01:28.809593+0000","last_unstale":"2024-09-19T06:03:20.722796+0000","last_undegraded":"2024-09-19T06:03:20.722796+0000","last_fullsized":"2024-09-19T06:03:20.722796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.546526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732082+0000","last_change":"2024-09-19T06:01:28.801395+0000","last_active":"2024-09-19T06:03:20.732082+0000","last_peered":"2024-09-19T06:03:20.732082+0000","last_clean":"2024-09-19T06:03:20.732082+0000","last_became_active":"2024-09-19T06:01:28.800987+0000","last_became_peered":"2024-09-19T06:01:28.800987+0000","last_unstale":"2024-09-19T06:03:20.732082+0000","last_undegraded":"2024-09-19T06:03:20.732082+0000","last_fullsized":"2024-09-19T06:03:20.732082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:18.986489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628078+0000","last_change":"2024-09-19T06:01:28.806554+0000","last_active":"2024-09-19T06:03:20.628078+0000","last_peered":"2024-09-19T06:03:20.628078+0000","last_clean":"2024-09-19T06:03:20.628078+0000","last_became_active":"2024-09-19T06:01:28.805946+0000","last_became_peered":"2024-09-19T06:01:28.805946+0000","last_unstale":"2024-09-19T06:03:20.628078+0000","last_undegraded":"2024-09-19T06:03:20.628078+0000","last_fullsized":"2024-09-19T06:03:20.628078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:56:20.130414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722784+0000","last_change":"2024-09-19T06:01:28.804789+0000","last_active":"2024-09-19T06:03:20.722784+0000","last_peered":"2024-09-19T06:03:20.722784+0000","last_clean":"2024-09-19T06:03:20.722784+0000","last_became_active":"2024-09-19T06:01:28.804484+0000","last_became_peered":"2024-09-19T06:01:28.804484+0000","last_unstale":"2024-09-19T06:03:20.722784+0000","last_undegraded":"2024-09-19T06:03:20.722784+0000","last_fullsized":"2024-09-19T06:03:20.722784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:19.737928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884972+0000","last_change":"2024-09-19T06:01:28.814504+0000","last_active":"2024-09-19T06:03:20.884972+0000","last_peered":"2024-09-19T06:03:20.884972+0000","last_clean":"2024-09-19T06:03:20.884972+0000","last_became_active":"2024-09-19T06:01:28.814066+0000","last_became_peered":"2024-09-19T06:01:28.814066+0000","last_unstale":"2024-09-19T06:03:20.884972+0000","last_undegraded":"2024-09-19T06:03:20.884972+0000","last_fullsized":"2024-09-19T06:03:20.884972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:50.904164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732391+0000","last_change":"2024-09-19T06:01:28.807303+0000","last_active":"2024-09-19T06:03:20.732391+0000","last_peered":"2024-09-19T06:03:20.732391+0000","last_clean":"2024-09-19T06:03:20.732391+0000","last_became_active":"2024-09-19T06:01:28.806978+0000","last_became_peered":"2024-09-19T06:01:28.806978+0000","last_unstale":"2024-09-19T06:03:20.732391+0000","last_undegraded":"2024-09-19T06:03:20.732391+0000","last_fullsized":"2024-09-19T06:03:20.732391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:33.259261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628504+0000","last_change":"2024-09-19T06:01:28.814844+0000","last_active":"2024-09-19T06:03:20.628504+0000","last_peered":"2024-09-19T06:03:20.628504+0000","last_clean":"2024-09-19T06:03:20.628504+0000","last_became_active":"2024-09-19T06:01:28.814531+0000","last_became_peered":"2024-09-19T06:01:28.814531+0000","last_unstale":"2024-09-19T06:03:20.628504+0000","last_undegraded":"2024-09-19T06:03:20.628504+0000","last_fullsized":"2024-09-19T06:03:20.628504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:17.209890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.893037+0000","last_change":"2024-09-19T06:01:28.806378+0000","last_active":"2024-09-19T06:03:20.893037+0000","last_peered":"2024-09-19T06:03:20.893037+0000","last_clean":"2024-09-19T06:03:20.893037+0000","last_became_active":"2024-09-19T06:01:28.805053+0000","last_became_peered":"2024-09-19T06:01:28.805053+0000","last_unstale":"2024-09-19T06:03:20.893037+0000","last_undegraded":"2024-09-19T06:03:20.893037+0000","last_fullsized":"2024-09-19T06:03:20.893037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:15.080548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627148+0000","last_change":"2024-09-19T06:01:28.802941+0000","last_active":"2024-09-19T06:03:20.627148+0000","last_peered":"2024-09-19T06:03:20.627148+0000","last_clean":"2024-09-19T06:03:20.627148+0000","last_became_active":"2024-09-19T06:01:28.802590+0000","last_became_peered":"2024-09-19T06:01:28.802590+0000","last_unstale":"2024-09-19T06:03:20.627148+0000","last_undegraded":"2024-09-19T06:03:20.627148+0000","last_fullsized":"2024-09-19T06:03:20.627148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:25.581854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887443+0000","last_change":"2024-09-19T06:01:28.800960+0000","last_active":"2024-09-19T06:03:20.887443+0000","last_peered":"2024-09-19T06:03:20.887443+0000","last_clean":"2024-09-19T06:03:20.887443+0000","last_became_active":"2024-09-19T06:01:28.800484+0000","last_became_peered":"2024-09-19T06:01:28.800484+0000","last_unstale":"2024-09-19T06:03:20.887443+0000","last_undegraded":"2024-09-19T06:03:20.887443+0000","last_fullsized":"2024-09-19T06:03:20.887443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:08.506105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.888999+0000","last_change":"2024-09-19T06:01:28.800771+0000","last_active":"2024-09-19T06:03:20.888999+0000","last_peered":"2024-09-19T06:03:20.888999+0000","last_clean":"2024-09-19T06:03:20.888999+0000","last_became_active":"2024-09-19T06:01:28.800427+0000","last_became_peered":"2024-09-19T06:01:28.800427+0000","last_unstale":"2024-09-19T06:03:20.888999+0000","last_undegraded":"2024-09-19T06:03:20.888999+0000","last_fullsized":"2024-09-19T06:03:20.888999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:19.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884875+0000","last_change":"2024-09-19T06:01:28.815528+0000","last_active":"2024-09-19T06:03:20.884875+0000","last_peered":"2024-09-19T06:03:20.884875+0000","last_clean":"2024-09-19T06:03:20.884875+0000","last_became_active":"2024-09-19T06:01:28.815129+0000","last_became_peered":"2024-09-19T06:01:28.815129+0000","last_unstale":"2024-09-19T06:03:20.884875+0000","last_undegraded":"2024-09-19T06:03:20.884875+0000","last_fullsized":"2024-09-19T06:03:20.884875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:01.936711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887589+0000","last_change":"2024-09-19T06:01:28.816120+0000","last_active":"2024-09-19T06:03:20.887589+0000","last_peered":"2024-09-19T06:03:20.887589+0000","last_clean":"2024-09-19T06:03:20.887589+0000","last_became_active":"2024-09-19T06:01:28.815822+0000","last_became_peered":"2024-09-19T06:01:28.815822+0000","last_unstale":"2024-09-19T06:03:20.887589+0000","last_undegraded":"2024-09-19T06:03:20.887589+0000","last_fullsized":"2024-09-19T06:03:20.887589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:14:58.670452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.721021+0000","last_change":"2024-09-19T06:01:28.809825+0000","last_active":"2024-09-19T06:03:20.721021+0000","last_peered":"2024-09-19T06:03:20.721021+0000","last_clean":"2024-09-19T06:03:20.721021+0000","last_became_active":"2024-09-19T06:01:28.809530+0000","last_became_peered":"2024-09-19T06:01:28.809530+0000","last_unstale":"2024-09-19T06:03:20.721021+0000","last_undegraded":"2024-09-19T06:03:20.721021+0000","last_fullsized":"2024-09-19T06:03:20.721021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:49:23.442436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.895924+0000","last_change":"2024-09-19T06:01:28.801297+0000","last_active":"2024-09-19T06:03:20.895924+0000","last_peered":"2024-09-19T06:03:20.895924+0000","last_clean":"2024-09-19T06:03:20.895924+0000","last_became_active":"2024-09-19T06:01:28.800984+0000","last_became_peered":"2024-09-19T06:01:28.800984+0000","last_unstale":"2024-09-19T06:03:20.895924+0000","last_undegraded":"2024-09-19T06:03:20.895924+0000","last_fullsized":"2024-09-19T06:03:20.895924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:01.485951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887070+0000","last_change":"2024-09-19T06:01:28.804567+0000","last_active":"2024-09-19T06:03:20.887070+0000","last_peered":"2024-09-19T06:03:20.887070+0000","last_clean":"2024-09-19T06:03:20.887070+0000","last_became_active":"2024-09-19T06:01:28.804213+0000","last_became_peered":"2024-09-19T06:01:28.804213+0000","last_unstale":"2024-09-19T06:03:20.887070+0000","last_undegraded":"2024-09-19T06:03:20.887070+0000","last_fullsized":"2024-09-19T06:03:20.887070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.186460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627307+0000","last_change":"2024-09-19T06:01:28.801961+0000","last_active":"2024-09-19T06:03:20.627307+0000","last_peered":"2024-09-19T06:03:20.627307+0000","last_clean":"2024-09-19T06:03:20.627307+0000","last_became_active":"2024-09-19T06:01:28.801631+0000","last_became_peered":"2024-09-19T06:01:28.801631+0000","last_unstale":"2024-09-19T06:03:20.627307+0000","last_undegraded":"2024-09-19T06:03:20.627307+0000","last_fullsized":"2024-09-19T06:03:20.627307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:15.628908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884896+0000","last_change":"2024-09-19T06:01:28.815160+0000","last_active":"2024-09-19T06:03:20.884896+0000","last_peered":"2024-09-19T06:03:20.884896+0000","last_clean":"2024-09-19T06:03:20.884896+0000","last_became_active":"2024-09-19T06:01:28.813329+0000","last_became_peered":"2024-09-19T06:01:28.813329+0000","last_unstale":"2024-09-19T06:03:20.884896+0000","last_undegraded":"2024-09-19T06:03:20.884896+0000","last_fullsized":"2024-09-19T06:03:20.884896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:17.027393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722875+0000","last_change":"2024-09-19T06:01:28.799205+0000","last_active":"2024-09-19T06:03:20.722875+0000","last_peered":"2024-09-19T06:03:20.722875+0000","last_clean":"2024-09-19T06:03:20.722875+0000","last_became_active":"2024-09-19T06:01:28.798891+0000","last_became_peered":"2024-09-19T06:01:28.798891+0000","last_unstale":"2024-09-19T06:03:20.722875+0000","last_undegraded":"2024-09-19T06:03:20.722875+0000","last_fullsized":"2024-09-19T06:03:20.722875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:41.227721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627907+0000","last_change":"2024-09-19T06:01:28.795823+0000","last_active":"2024-09-19T06:03:20.627907+0000","last_peered":"2024-09-19T06:03:20.627907+0000","last_clean":"2024-09-19T06:03:20.627907+0000","last_became_active":"2024-09-19T06:01:28.795510+0000","last_became_peered":"2024-09-19T06:01:28.795510+0000","last_unstale":"2024-09-19T06:03:20.627907+0000","last_undegraded":"2024-09-19T06:03:20.627907+0000","last_fullsized":"2024-09-19T06:03:20.627907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:54.831610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723807+0000","last_change":"2024-09-19T06:01:28.799696+0000","last_active":"2024-09-19T06:03:20.723807+0000","last_peered":"2024-09-19T06:03:20.723807+0000","last_clean":"2024-09-19T06:03:20.723807+0000","last_became_active":"2024-09-19T06:01:28.799417+0000","last_became_peered":"2024-09-19T06:01:28.799417+0000","last_unstale":"2024-09-19T06:03:20.723807+0000","last_undegraded":"2024-09-19T06:03:20.723807+0000","last_fullsized":"2024-09-19T06:03:20.723807+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:02.330808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.896104+0000","last_change":"2024-09-19T06:01:28.807146+0000","last_active":"2024-09-19T06:03:20.896104+0000","last_peered":"2024-09-19T06:03:20.896104+0000","last_clean":"2024-09-19T06:03:20.896104+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:03:20.896104+0000","last_undegraded":"2024-09-19T06:03:20.896104+0000","last_fullsized":"2024-09-19T06:03:20.896104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:12.308421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887102+0000","last_change":"2024-09-19T06:01:28.803555+0000","last_active":"2024-09-19T06:03:20.887102+0000","last_peered":"2024-09-19T06:03:20.887102+0000","last_clean":"2024-09-19T06:03:20.887102+0000","last_became_active":"2024-09-19T06:01:28.803242+0000","last_became_peered":"2024-09-19T06:01:28.803242+0000","last_unstale":"2024-09-19T06:03:20.887102+0000","last_undegraded":"2024-09-19T06:03:20.887102+0000","last_fullsized":"2024-09-19T06:03:20.887102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:02.281152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626997+0000","last_change":"2024-09-19T06:01:28.804378+0000","last_active":"2024-09-19T06:03:20.626997+0000","last_peered":"2024-09-19T06:03:20.626997+0000","last_clean":"2024-09-19T06:03:20.626997+0000","last_became_active":"2024-09-19T06:01:28.804099+0000","last_became_peered":"2024-09-19T06:01:28.804099+0000","last_unstale":"2024-09-19T06:03:20.626997+0000","last_undegraded":"2024-09-19T06:03:20.626997+0000","last_fullsized":"2024-09-19T06:03:20.626997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:11.558384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887754+0000","last_change":"2024-09-19T06:01:28.799999+0000","last_active":"2024-09-19T06:03:20.887754+0000","last_peered":"2024-09-19T06:03:20.887754+0000","last_clean":"2024-09-19T06:03:20.887754+0000","last_became_active":"2024-09-19T06:01:28.799645+0000","last_became_peered":"2024-09-19T06:01:28.799645+0000","last_unstale":"2024-09-19T06:03:20.887754+0000","last_undegraded":"2024-09-19T06:03:20.887754+0000","last_fullsized":"2024-09-19T06:03:20.887754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:20.442119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.895777+0000","last_change":"2024-09-19T06:01:28.800143+0000","last_active":"2024-09-19T06:03:20.895777+0000","last_peered":"2024-09-19T06:03:20.895777+0000","last_clean":"2024-09-19T06:03:20.895777+0000","last_became_active":"2024-09-19T06:01:28.799830+0000","last_became_peered":"2024-09-19T06:01:28.799830+0000","last_unstale":"2024-09-19T06:03:20.895777+0000","last_undegraded":"2024-09-19T06:03:20.895777+0000","last_fullsized":"2024-09-19T06:03:20.895777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:43.824740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.895772+0000","last_change":"2024-09-19T06:01:28.806130+0000","last_active":"2024-09-19T06:03:20.895772+0000","last_peered":"2024-09-19T06:03:20.895772+0000","last_clean":"2024-09-19T06:03:20.895772+0000","last_became_active":"2024-09-19T06:01:28.805772+0000","last_became_peered":"2024-09-19T06:01:28.805772+0000","last_unstale":"2024-09-19T06:03:20.895772+0000","last_undegraded":"2024-09-19T06:03:20.895772+0000","last_fullsized":"2024-09-19T06:03:20.895772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:00.316311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887402+0000","last_change":"2024-09-19T06:01:39.837412+0000","last_active":"2024-09-19T06:03:20.887402+0000","last_peered":"2024-09-19T06:03:20.887402+0000","last_clean":"2024-09-19T06:03:20.887402+0000","last_became_active":"2024-09-19T06:01:39.837035+0000","last_became_peered":"2024-09-19T06:01:39.837035+0000","last_unstale":"2024-09-19T06:03:20.887402+0000","last_undegraded":"2024-09-19T06:03:20.887402+0000","last_fullsized":"2024-09-19T06:03:20.887402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:00.073884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722768+0000","last_change":"2024-09-19T06:01:28.800324+0000","last_active":"2024-09-19T06:03:20.722768+0000","last_peered":"2024-09-19T06:03:20.722768+0000","last_clean":"2024-09-19T06:03:20.722768+0000","last_became_active":"2024-09-19T06:01:28.799986+0000","last_became_peered":"2024-09-19T06:01:28.799986+0000","last_unstale":"2024-09-19T06:03:20.722768+0000","last_undegraded":"2024-09-19T06:03:20.722768+0000","last_fullsized":"2024-09-19T06:03:20.722768+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:41.963346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":151,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.978373+0000","last_change":"2024-09-19T06:01:39.844897+0000","last_active":"2024-09-19T06:03:20.978373+0000","last_peered":"2024-09-19T06:03:20.978373+0000","last_clean":"2024-09-19T06:03:20.978373+0000","last_became_active":"2024-09-19T06:01:39.838347+0000","last_became_peered":"2024-09-19T06:01:39.838347+0000","last_unstale":"2024-09-19T06:03:20.978373+0000","last_undegraded":"2024-09-19T06:03:20.978373+0000","last_fullsized":"2024-09-19T06:03:20.978373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:26.235227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2044,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628020+0000","last_change":"2024-09-19T06:01:28.802644+0000","last_active":"2024-09-19T06:03:20.628020+0000","last_peered":"2024-09-19T06:03:20.628020+0000","last_clean":"2024-09-19T06:03:20.628020+0000","last_became_active":"2024-09-19T06:01:28.802288+0000","last_became_peered":"2024-09-19T06:01:28.802288+0000","last_unstale":"2024-09-19T06:03:20.628020+0000","last_undegraded":"2024-09-19T06:03:20.628020+0000","last_fullsized":"2024-09-19T06:03:20.628020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:32.031860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732271+0000","last_change":"2024-09-19T06:01:26.781104+0000","last_active":"2024-09-19T06:03:20.732271+0000","last_peered":"2024-09-19T06:03:20.732271+0000","last_clean":"2024-09-19T06:03:20.732271+0000","last_became_active":"2024-09-19T06:01:26.780733+0000","last_became_peered":"2024-09-19T06:01:26.780733+0000","last_unstale":"2024-09-19T06:03:20.732271+0000","last_undegraded":"2024-09-19T06:03:20.732271+0000","last_fullsized":"2024-09-19T06:03:20.732271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:54.490867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626771+0000","last_change":"2024-09-19T06:01:26.782901+0000","last_active":"2024-09-19T06:03:20.626771+0000","last_peered":"2024-09-19T06:03:20.626771+0000","last_clean":"2024-09-19T06:03:20.626771+0000","last_became_active":"2024-09-19T06:01:26.782450+0000","last_became_peered":"2024-09-19T06:01:26.782450+0000","last_unstale":"2024-09-19T06:03:20.626771+0000","last_undegraded":"2024-09-19T06:03:20.626771+0000","last_fullsized":"2024-09-19T06:03:20.626771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:07.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722809+0000","last_change":"2024-09-19T06:01:28.804105+0000","last_active":"2024-09-19T06:03:20.722809+0000","last_peered":"2024-09-19T06:03:20.722809+0000","last_clean":"2024-09-19T06:03:20.722809+0000","last_became_active":"2024-09-19T06:01:28.797980+0000","last_became_peered":"2024-09-19T06:01:28.797980+0000","last_unstale":"2024-09-19T06:03:20.722809+0000","last_undegraded":"2024-09-19T06:03:20.722809+0000","last_fullsized":"2024-09-19T06:03:20.722809+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:50.130102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628139+0000","last_change":"2024-09-19T06:01:26.786923+0000","last_active":"2024-09-19T06:03:20.628139+0000","last_peered":"2024-09-19T06:03:20.628139+0000","last_clean":"2024-09-19T06:03:20.628139+0000","last_became_active":"2024-09-19T06:01:26.786542+0000","last_became_peered":"2024-09-19T06:01:26.786542+0000","last_unstale":"2024-09-19T06:03:20.628139+0000","last_undegraded":"2024-09-19T06:03:20.628139+0000","last_fullsized":"2024-09-19T06:03:20.628139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:30.032992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.720767+0000","last_change":"2024-09-19T06:01:28.802172+0000","last_active":"2024-09-19T06:03:20.720767+0000","last_peered":"2024-09-19T06:03:20.720767+0000","last_clean":"2024-09-19T06:03:20.720767+0000","last_became_active":"2024-09-19T06:01:28.801815+0000","last_became_peered":"2024-09-19T06:01:28.801815+0000","last_unstale":"2024-09-19T06:03:20.720767+0000","last_undegraded":"2024-09-19T06:03:20.720767+0000","last_fullsized":"2024-09-19T06:03:20.720767+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:09.587638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":230,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.977949+0000","last_change":"2024-09-19T06:01:26.777870+0000","last_active":"2024-09-19T06:03:20.977949+0000","last_peered":"2024-09-19T06:03:20.977949+0000","last_clean":"2024-09-19T06:03:20.977949+0000","last_became_active":"2024-09-19T06:01:26.777509+0000","last_became_peered":"2024-09-19T06:01:26.777509+0000","last_unstale":"2024-09-19T06:03:20.977949+0000","last_undegraded":"2024-09-19T06:03:20.977949+0000","last_fullsized":"2024-09-19T06:03:20.977949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:15.885911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884945+0000","last_change":"2024-09-19T06:01:28.812201+0000","last_active":"2024-09-19T06:03:20.884945+0000","last_peered":"2024-09-19T06:03:20.884945+0000","last_clean":"2024-09-19T06:03:20.884945+0000","last_became_active":"2024-09-19T06:01:28.811801+0000","last_became_peered":"2024-09-19T06:01:28.811801+0000","last_unstale":"2024-09-19T06:03:20.884945+0000","last_undegraded":"2024-09-19T06:03:20.884945+0000","last_fullsized":"2024-09-19T06:03:20.884945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:32.904787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628546+0000","last_change":"2024-09-19T06:01:26.787845+0000","last_active":"2024-09-19T06:03:20.628546+0000","last_peered":"2024-09-19T06:03:20.628546+0000","last_clean":"2024-09-19T06:03:20.628546+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:03:20.628546+0000","last_undegraded":"2024-09-19T06:03:20.628546+0000","last_fullsized":"2024-09-19T06:03:20.628546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.114865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.889521+0000","last_change":"2024-09-19T06:01:28.801523+0000","last_active":"2024-09-19T06:03:20.889521+0000","last_peered":"2024-09-19T06:03:20.889521+0000","last_clean":"2024-09-19T06:03:20.889521+0000","last_became_active":"2024-09-19T06:01:28.801205+0000","last_became_peered":"2024-09-19T06:01:28.801205+0000","last_unstale":"2024-09-19T06:03:20.889521+0000","last_undegraded":"2024-09-19T06:03:20.889521+0000","last_fullsized":"2024-09-19T06:03:20.889521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:10.314085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628080+0000","last_change":"2024-09-19T06:01:27.022172+0000","last_active":"2024-09-19T06:03:20.628080+0000","last_peered":"2024-09-19T06:03:20.628080+0000","last_clean":"2024-09-19T06:03:20.628080+0000","last_became_active":"2024-09-19T06:01:27.021568+0000","last_became_peered":"2024-09-19T06:01:27.021568+0000","last_unstale":"2024-09-19T06:03:20.628080+0000","last_undegraded":"2024-09-19T06:03:20.628080+0000","last_fullsized":"2024-09-19T06:03:20.628080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:18.031690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.885201+0000","last_change":"2024-09-19T06:01:39.839061+0000","last_active":"2024-09-19T06:03:20.885201+0000","last_peered":"2024-09-19T06:03:20.885201+0000","last_clean":"2024-09-19T06:03:20.885201+0000","last_became_active":"2024-09-19T06:01:39.838644+0000","last_became_peered":"2024-09-19T06:01:39.838644+0000","last_unstale":"2024-09-19T06:03:20.885201+0000","last_undegraded":"2024-09-19T06:03:20.885201+0000","last_fullsized":"2024-09-19T06:03:20.885201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:02.136222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'37","reported_seq":273,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.977666+0000","last_change":"2024-09-19T06:01:26.784781+0000","last_active":"2024-09-19T06:03:20.977666+0000","last_peered":"2024-09-19T06:03:20.977666+0000","last_clean":"2024-09-19T06:03:20.977666+0000","last_became_active":"2024-09-19T06:01:26.784284+0000","last_became_peered":"2024-09-19T06:01:26.784284+0000","last_unstale":"2024-09-19T06:03:20.977666+0000","last_undegraded":"2024-09-19T06:03:20.977666+0000","last_fullsized":"2024-09-19T06:03:20.977666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:13.063299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67016,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":212,"num_read_kb":170,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722979+0000","last_change":"2024-09-19T06:01:28.802604+0000","last_active":"2024-09-19T06:03:20.722979+0000","last_peered":"2024-09-19T06:03:20.722979+0000","last_clean":"2024-09-19T06:03:20.722979+0000","last_became_active":"2024-09-19T06:01:28.802285+0000","last_became_peered":"2024-09-19T06:01:28.802285+0000","last_unstale":"2024-09-19T06:03:20.722979+0000","last_undegraded":"2024-09-19T06:03:20.722979+0000","last_fullsized":"2024-09-19T06:03:20.722979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:57.177791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.895972+0000","last_change":"2024-09-19T06:01:26.783678+0000","last_active":"2024-09-19T06:03:20.895972+0000","last_peered":"2024-09-19T06:03:20.895972+0000","last_clean":"2024-09-19T06:03:20.895972+0000","last_became_active":"2024-09-19T06:01:26.783165+0000","last_became_peered":"2024-09-19T06:01:26.783165+0000","last_unstale":"2024-09-19T06:03:20.895972+0000","last_undegraded":"2024-09-19T06:03:20.895972+0000","last_fullsized":"2024-09-19T06:03:20.895972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:50.535733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626891+0000","last_change":"2024-09-19T06:01:28.808553+0000","last_active":"2024-09-19T06:03:20.626891+0000","last_peered":"2024-09-19T06:03:20.626891+0000","last_clean":"2024-09-19T06:03:20.626891+0000","last_became_active":"2024-09-19T06:01:28.808215+0000","last_became_peered":"2024-09-19T06:01:28.808215+0000","last_unstale":"2024-09-19T06:03:20.626891+0000","last_undegraded":"2024-09-19T06:03:20.626891+0000","last_fullsized":"2024-09-19T06:03:20.626891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:55.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627942+0000","last_change":"2024-09-19T06:01:26.787137+0000","last_active":"2024-09-19T06:03:20.627942+0000","last_peered":"2024-09-19T06:03:20.627942+0000","last_clean":"2024-09-19T06:03:20.627942+0000","last_became_active":"2024-09-19T06:01:26.786676+0000","last_became_peered":"2024-09-19T06:01:26.786676+0000","last_unstale":"2024-09-19T06:03:20.627942+0000","last_undegraded":"2024-09-19T06:03:20.627942+0000","last_fullsized":"2024-09-19T06:03:20.627942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.609969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628199+0000","last_change":"2024-09-19T06:01:28.812106+0000","last_active":"2024-09-19T06:03:20.628199+0000","last_peered":"2024-09-19T06:03:20.628199+0000","last_clean":"2024-09-19T06:03:20.628199+0000","last_became_active":"2024-09-19T06:01:28.811771+0000","last_became_peered":"2024-09-19T06:01:28.811771+0000","last_unstale":"2024-09-19T06:03:20.628199+0000","last_undegraded":"2024-09-19T06:03:20.628199+0000","last_fullsized":"2024-09-19T06:03:20.628199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:07:06.832685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'9","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.978122+0000","last_change":"2024-09-19T06:01:26.786746+0000","last_active":"2024-09-19T06:03:20.978122+0000","last_peered":"2024-09-19T06:03:20.978122+0000","last_clean":"2024-09-19T06:03:20.978122+0000","last_became_active":"2024-09-19T06:01:26.786414+0000","last_became_peered":"2024-09-19T06:01:26.786414+0000","last_unstale":"2024-09-19T06:03:20.978122+0000","last_undegraded":"2024-09-19T06:03:20.978122+0000","last_fullsized":"2024-09-19T06:03:20.978122+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:25.042796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627937+0000","last_change":"2024-09-19T06:01:28.807183+0000","last_active":"2024-09-19T06:03:20.627937+0000","last_peered":"2024-09-19T06:03:20.627937+0000","last_clean":"2024-09-19T06:03:20.627937+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:03:20.627937+0000","last_undegraded":"2024-09-19T06:03:20.627937+0000","last_fullsized":"2024-09-19T06:03:20.627937+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:23.030312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722771+0000","last_change":"2024-09-19T06:01:27.022409+0000","last_active":"2024-09-19T06:03:20.722771+0000","last_peered":"2024-09-19T06:03:20.722771+0000","last_clean":"2024-09-19T06:03:20.722771+0000","last_became_active":"2024-09-19T06:01:27.022034+0000","last_became_peered":"2024-09-19T06:01:27.022034+0000","last_unstale":"2024-09-19T06:03:20.722771+0000","last_undegraded":"2024-09-19T06:03:20.722771+0000","last_fullsized":"2024-09-19T06:03:20.722771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:08.663575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627932+0000","last_change":"2024-09-19T06:01:28.812079+0000","last_active":"2024-09-19T06:03:20.627932+0000","last_peered":"2024-09-19T06:03:20.627932+0000","last_clean":"2024-09-19T06:03:20.627932+0000","last_became_active":"2024-09-19T06:01:28.811606+0000","last_became_peered":"2024-09-19T06:01:28.811606+0000","last_unstale":"2024-09-19T06:03:20.627932+0000","last_undegraded":"2024-09-19T06:03:20.627932+0000","last_fullsized":"2024-09-19T06:03:20.627932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:49.434235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723153+0000","last_change":"2024-09-19T06:01:26.783823+0000","last_active":"2024-09-19T06:03:20.723153+0000","last_peered":"2024-09-19T06:03:20.723153+0000","last_clean":"2024-09-19T06:03:20.723153+0000","last_became_active":"2024-09-19T06:01:26.783209+0000","last_became_peered":"2024-09-19T06:01:26.783209+0000","last_unstale":"2024-09-19T06:03:20.723153+0000","last_undegraded":"2024-09-19T06:03:20.723153+0000","last_fullsized":"2024-09-19T06:03:20.723153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:38.833417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887395+0000","last_change":"2024-09-19T06:01:39.837462+0000","last_active":"2024-09-19T06:03:20.887395+0000","last_peered":"2024-09-19T06:03:20.887395+0000","last_clean":"2024-09-19T06:03:20.887395+0000","last_became_active":"2024-09-19T06:01:39.837082+0000","last_became_peered":"2024-09-19T06:01:39.837082+0000","last_unstale":"2024-09-19T06:03:20.887395+0000","last_undegraded":"2024-09-19T06:03:20.887395+0000","last_fullsized":"2024-09-19T06:03:20.887395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:47.421017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627424+0000","last_change":"2024-09-19T06:01:26.779992+0000","last_active":"2024-09-19T06:03:20.627424+0000","last_peered":"2024-09-19T06:03:20.627424+0000","last_clean":"2024-09-19T06:03:20.627424+0000","last_became_active":"2024-09-19T06:01:26.779609+0000","last_became_peered":"2024-09-19T06:01:26.779609+0000","last_unstale":"2024-09-19T06:03:20.627424+0000","last_undegraded":"2024-09-19T06:03:20.627424+0000","last_fullsized":"2024-09-19T06:03:20.627424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:48.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628684+0000","last_change":"2024-09-19T06:01:28.798827+0000","last_active":"2024-09-19T06:03:20.628684+0000","last_peered":"2024-09-19T06:03:20.628684+0000","last_clean":"2024-09-19T06:03:20.628684+0000","last_became_active":"2024-09-19T06:01:28.798457+0000","last_became_peered":"2024-09-19T06:01:28.798457+0000","last_unstale":"2024-09-19T06:03:20.628684+0000","last_undegraded":"2024-09-19T06:03:20.628684+0000","last_fullsized":"2024-09-19T06:03:20.628684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:41.111051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'41","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.850980+0000","last_change":"2024-09-19T06:01:26.775691+0000","last_active":"2024-09-19T06:03:20.850980+0000","last_peered":"2024-09-19T06:03:20.850980+0000","last_clean":"2024-09-19T06:03:20.850980+0000","last_became_active":"2024-09-19T06:01:26.775183+0000","last_became_peered":"2024-09-19T06:01:26.775183+0000","last_unstale":"2024-09-19T06:03:20.850980+0000","last_undegraded":"2024-09-19T06:03:20.850980+0000","last_fullsized":"2024-09-19T06:03:20.850980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:34.712968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628457+0000","last_change":"2024-09-19T06:01:28.809459+0000","last_active":"2024-09-19T06:03:20.628457+0000","last_peered":"2024-09-19T06:03:20.628457+0000","last_clean":"2024-09-19T06:03:20.628457+0000","last_became_active":"2024-09-19T06:01:28.809046+0000","last_became_peered":"2024-09-19T06:01:28.809046+0000","last_unstale":"2024-09-19T06:03:20.628457+0000","last_undegraded":"2024-09-19T06:03:20.628457+0000","last_fullsized":"2024-09-19T06:03:20.628457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:56.996190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'10","reported_seq":132,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.984256+0000","last_change":"2024-09-19T06:01:39.843932+0000","last_active":"2024-09-19T06:03:20.984256+0000","last_peered":"2024-09-19T06:03:20.984256+0000","last_clean":"2024-09-19T06:03:20.984256+0000","last_became_active":"2024-09-19T06:01:39.836008+0000","last_became_peered":"2024-09-19T06:01:39.836008+0000","last_unstale":"2024-09-19T06:03:20.984256+0000","last_undegraded":"2024-09-19T06:03:20.984256+0000","last_fullsized":"2024-09-19T06:03:20.984256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:58.798859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2985,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628540+0000","last_change":"2024-09-19T06:01:28.814181+0000","last_active":"2024-09-19T06:03:20.628540+0000","last_peered":"2024-09-19T06:03:20.628540+0000","last_clean":"2024-09-19T06:03:20.628540+0000","last_became_active":"2024-09-19T06:01:28.813817+0000","last_became_peered":"2024-09-19T06:01:28.813817+0000","last_unstale":"2024-09-19T06:03:20.628540+0000","last_undegraded":"2024-09-19T06:03:20.628540+0000","last_fullsized":"2024-09-19T06:03:20.628540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:40.057863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628100+0000","last_change":"2024-09-19T06:00:17.624509+0000","last_active":"2024-09-19T06:03:20.628100+0000","last_peered":"2024-09-19T06:03:20.628100+0000","last_clean":"2024-09-19T06:03:20.628100+0000","last_became_active":"2024-09-19T06:00:17.623943+0000","last_became_peered":"2024-09-19T06:00:17.623943+0000","last_unstale":"2024-09-19T06:03:20.628100+0000","last_undegraded":"2024-09-19T06:03:20.628100+0000","last_fullsized":"2024-09-19T06:03:20.628100+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_clean_scrub_stamp":"2024-09-19T05:56:55.414488+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:39.116345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.893738+0000","last_change":"2024-09-19T06:01:26.783896+0000","last_active":"2024-09-19T06:03:20.893738+0000","last_peered":"2024-09-19T06:03:20.893738+0000","last_clean":"2024-09-19T06:03:20.893738+0000","last_became_active":"2024-09-19T06:01:26.783332+0000","last_became_peered":"2024-09-19T06:01:26.783332+0000","last_unstale":"2024-09-19T06:03:20.893738+0000","last_undegraded":"2024-09-19T06:03:20.893738+0000","last_fullsized":"2024-09-19T06:03:20.893738+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:33.072763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723257+0000","last_change":"2024-09-19T06:01:28.807756+0000","last_active":"2024-09-19T06:03:20.723257+0000","last_peered":"2024-09-19T06:03:20.723257+0000","last_clean":"2024-09-19T06:03:20.723257+0000","last_became_active":"2024-09-19T06:01:28.807077+0000","last_became_peered":"2024-09-19T06:01:28.807077+0000","last_unstale":"2024-09-19T06:03:20.723257+0000","last_undegraded":"2024-09-19T06:03:20.723257+0000","last_fullsized":"2024-09-19T06:03:20.723257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:20.784199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628079+0000","last_change":"2024-09-19T06:01:26.784830+0000","last_active":"2024-09-19T06:03:20.628079+0000","last_peered":"2024-09-19T06:03:20.628079+0000","last_clean":"2024-09-19T06:03:20.628079+0000","last_became_active":"2024-09-19T06:01:26.783887+0000","last_became_peered":"2024-09-19T06:01:26.783887+0000","last_unstale":"2024-09-19T06:03:20.628079+0000","last_undegraded":"2024-09-19T06:03:20.628079+0000","last_fullsized":"2024-09-19T06:03:20.628079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:33:38.255169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626764+0000","last_change":"2024-09-19T06:01:39.837354+0000","last_active":"2024-09-19T06:03:20.626764+0000","last_peered":"2024-09-19T06:03:20.626764+0000","last_clean":"2024-09-19T06:03:20.626764+0000","last_became_active":"2024-09-19T06:01:39.836930+0000","last_became_peered":"2024-09-19T06:01:39.836930+0000","last_unstale":"2024-09-19T06:03:20.626764+0000","last_undegraded":"2024-09-19T06:03:20.626764+0000","last_fullsized":"2024-09-19T06:03:20.626764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:17.571834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":131,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626969+0000","last_change":"2024-09-19T06:01:27.021753+0000","last_active":"2024-09-19T06:03:20.626969+0000","last_peered":"2024-09-19T06:03:20.626969+0000","last_clean":"2024-09-19T06:03:20.626969+0000","last_became_active":"2024-09-19T06:01:27.021346+0000","last_became_peered":"2024-09-19T06:01:27.021346+0000","last_unstale":"2024-09-19T06:03:20.626969+0000","last_undegraded":"2024-09-19T06:03:20.626969+0000","last_fullsized":"2024-09-19T06:03:20.626969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:07.291283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627909+0000","last_change":"2024-09-19T06:01:28.803103+0000","last_active":"2024-09-19T06:03:20.627909+0000","last_peered":"2024-09-19T06:03:20.627909+0000","last_clean":"2024-09-19T06:03:20.627909+0000","last_became_active":"2024-09-19T06:01:28.802806+0000","last_became_peered":"2024-09-19T06:01:28.802806+0000","last_unstale":"2024-09-19T06:03:20.627909+0000","last_undegraded":"2024-09-19T06:03:20.627909+0000","last_fullsized":"2024-09-19T06:03:20.627909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:33.329231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'8","reported_seq":136,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628270+0000","last_change":"2024-09-19T06:01:26.786605+0000","last_active":"2024-09-19T06:03:20.628270+0000","last_peered":"2024-09-19T06:03:20.628270+0000","last_clean":"2024-09-19T06:03:20.628270+0000","last_became_active":"2024-09-19T06:01:26.786278+0000","last_became_peered":"2024-09-19T06:01:26.786278+0000","last_unstale":"2024-09-19T06:03:20.628270+0000","last_undegraded":"2024-09-19T06:03:20.628270+0000","last_fullsized":"2024-09-19T06:03:20.628270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.581502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732180+0000","last_change":"2024-09-19T06:01:28.808152+0000","last_active":"2024-09-19T06:03:20.732180+0000","last_peered":"2024-09-19T06:03:20.732180+0000","last_clean":"2024-09-19T06:03:20.732180+0000","last_became_active":"2024-09-19T06:01:28.807788+0000","last_became_peered":"2024-09-19T06:01:28.807788+0000","last_unstale":"2024-09-19T06:03:20.732180+0000","last_undegraded":"2024-09-19T06:03:20.732180+0000","last_fullsized":"2024-09-19T06:03:20.732180+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:42.877539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.720863+0000","last_change":"2024-09-19T06:01:27.022719+0000","last_active":"2024-09-19T06:03:20.720863+0000","last_peered":"2024-09-19T06:03:20.720863+0000","last_clean":"2024-09-19T06:03:20.720863+0000","last_became_active":"2024-09-19T06:01:27.022263+0000","last_became_peered":"2024-09-19T06:01:27.022263+0000","last_unstale":"2024-09-19T06:03:20.720863+0000","last_undegraded":"2024-09-19T06:03:20.720863+0000","last_fullsized":"2024-09-19T06:03:20.720863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:45.158229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732175+0000","last_change":"2024-09-19T06:01:28.809869+0000","last_active":"2024-09-19T06:03:20.732175+0000","last_peered":"2024-09-19T06:03:20.732175+0000","last_clean":"2024-09-19T06:03:20.732175+0000","last_became_active":"2024-09-19T06:01:28.809548+0000","last_became_peered":"2024-09-19T06:01:28.809548+0000","last_unstale":"2024-09-19T06:03:20.732175+0000","last_undegraded":"2024-09-19T06:03:20.732175+0000","last_fullsized":"2024-09-19T06:03:20.732175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:41.587846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887443+0000","last_change":"2024-09-19T06:01:39.846740+0000","last_active":"2024-09-19T06:03:20.887443+0000","last_peered":"2024-09-19T06:03:20.887443+0000","last_clean":"2024-09-19T06:03:20.887443+0000","last_became_active":"2024-09-19T06:01:39.836378+0000","last_became_peered":"2024-09-19T06:01:39.836378+0000","last_unstale":"2024-09-19T06:03:20.887443+0000","last_undegraded":"2024-09-19T06:03:20.887443+0000","last_fullsized":"2024-09-19T06:03:20.887443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.064918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627124+0000","last_change":"2024-09-19T06:01:28.807796+0000","last_active":"2024-09-19T06:03:20.627124+0000","last_peered":"2024-09-19T06:03:20.627124+0000","last_clean":"2024-09-19T06:03:20.627124+0000","last_became_active":"2024-09-19T06:01:28.807503+0000","last_became_peered":"2024-09-19T06:01:28.807503+0000","last_unstale":"2024-09-19T06:03:20.627124+0000","last_undegraded":"2024-09-19T06:03:20.627124+0000","last_fullsized":"2024-09-19T06:03:20.627124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:43.472647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'51","reported_seq":98,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.853902+0000","last_change":"2024-09-19T06:01:26.783517+0000","last_active":"2024-09-19T06:03:20.853902+0000","last_peered":"2024-09-19T06:03:20.853902+0000","last_clean":"2024-09-19T06:03:20.853902+0000","last_became_active":"2024-09-19T06:01:26.783164+0000","last_became_peered":"2024-09-19T06:01:26.783164+0000","last_unstale":"2024-09-19T06:03:20.853902+0000","last_undegraded":"2024-09-19T06:03:20.853902+0000","last_fullsized":"2024-09-19T06:03:20.853902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:57.609192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.724088+0000","last_change":"2024-09-19T06:01:28.806548+0000","last_active":"2024-09-19T06:03:20.724088+0000","last_peered":"2024-09-19T06:03:20.724088+0000","last_clean":"2024-09-19T06:03:20.724088+0000","last_became_active":"2024-09-19T06:01:28.805724+0000","last_became_peered":"2024-09-19T06:01:28.805724+0000","last_unstale":"2024-09-19T06:03:20.724088+0000","last_undegraded":"2024-09-19T06:03:20.724088+0000","last_fullsized":"2024-09-19T06:03:20.724088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:14.360928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.893742+0000","last_change":"2024-09-19T06:01:27.128213+0000","last_active":"2024-09-19T06:03:20.893742+0000","last_peered":"2024-09-19T06:03:20.893742+0000","last_clean":"2024-09-19T06:03:20.893742+0000","last_became_active":"2024-09-19T06:01:27.127737+0000","last_became_peered":"2024-09-19T06:01:27.127737+0000","last_unstale":"2024-09-19T06:03:20.893742+0000","last_undegraded":"2024-09-19T06:03:20.893742+0000","last_fullsized":"2024-09-19T06:03:20.893742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:33:35.107152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.888504+0000","last_change":"2024-09-19T06:01:39.839027+0000","last_active":"2024-09-19T06:03:20.888504+0000","last_peered":"2024-09-19T06:03:20.888504+0000","last_clean":"2024-09-19T06:03:20.888504+0000","last_became_active":"2024-09-19T06:01:39.838601+0000","last_became_peered":"2024-09-19T06:01:39.838601+0000","last_unstale":"2024-09-19T06:03:20.888504+0000","last_undegraded":"2024-09-19T06:03:20.888504+0000","last_fullsized":"2024-09-19T06:03:20.888504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:09.864997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.885207+0000","last_change":"2024-09-19T06:01:27.022438+0000","last_active":"2024-09-19T06:03:20.885207+0000","last_peered":"2024-09-19T06:03:20.885207+0000","last_clean":"2024-09-19T06:03:20.885207+0000","last_became_active":"2024-09-19T06:01:27.021953+0000","last_became_peered":"2024-09-19T06:01:27.021953+0000","last_unstale":"2024-09-19T06:03:20.885207+0000","last_undegraded":"2024-09-19T06:03:20.885207+0000","last_fullsized":"2024-09-19T06:03:20.885207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:24.383178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722756+0000","last_change":"2024-09-19T06:01:28.803807+0000","last_active":"2024-09-19T06:03:20.722756+0000","last_peered":"2024-09-19T06:03:20.722756+0000","last_clean":"2024-09-19T06:03:20.722756+0000","last_became_active":"2024-09-19T06:01:28.803486+0000","last_became_peered":"2024-09-19T06:01:28.803486+0000","last_unstale":"2024-09-19T06:03:20.722756+0000","last_undegraded":"2024-09-19T06:03:20.722756+0000","last_fullsized":"2024-09-19T06:03:20.722756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:26:17.920089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723040+0000","last_change":"2024-09-19T06:01:26.777171+0000","last_active":"2024-09-19T06:03:20.723040+0000","last_peered":"2024-09-19T06:03:20.723040+0000","last_clean":"2024-09-19T06:03:20.723040+0000","last_became_active":"2024-09-19T06:01:26.776818+0000","last_became_peered":"2024-09-19T06:01:26.776818+0000","last_unstale":"2024-09-19T06:03:20.723040+0000","last_undegraded":"2024-09-19T06:03:20.723040+0000","last_fullsized":"2024-09-19T06:03:20.723040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887551+0000","last_change":"2024-09-19T06:01:28.799266+0000","last_active":"2024-09-19T06:03:20.887551+0000","last_peered":"2024-09-19T06:03:20.887551+0000","last_clean":"2024-09-19T06:03:20.887551+0000","last_became_active":"2024-09-19T06:01:28.798969+0000","last_became_peered":"2024-09-19T06:01:28.798969+0000","last_unstale":"2024-09-19T06:03:20.887551+0000","last_undegraded":"2024-09-19T06:03:20.887551+0000","last_fullsized":"2024-09-19T06:03:20.887551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:34.064795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.896088+0000","last_change":"2024-09-19T06:01:26.781680+0000","last_active":"2024-09-19T06:03:20.896088+0000","last_peered":"2024-09-19T06:03:20.896088+0000","last_clean":"2024-09-19T06:03:20.896088+0000","last_became_active":"2024-09-19T06:01:26.781035+0000","last_became_peered":"2024-09-19T06:01:26.781035+0000","last_unstale":"2024-09-19T06:03:20.896088+0000","last_undegraded":"2024-09-19T06:03:20.896088+0000","last_fullsized":"2024-09-19T06:03:20.896088+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:38.287018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723555+0000","last_change":"2024-09-19T06:01:28.802970+0000","last_active":"2024-09-19T06:03:20.723555+0000","last_peered":"2024-09-19T06:03:20.723555+0000","last_clean":"2024-09-19T06:03:20.723555+0000","last_became_active":"2024-09-19T06:01:28.802587+0000","last_became_peered":"2024-09-19T06:01:28.802587+0000","last_unstale":"2024-09-19T06:03:20.723555+0000","last_undegraded":"2024-09-19T06:03:20.723555+0000","last_fullsized":"2024-09-19T06:03:20.723555+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722866+0000","last_change":"2024-09-19T06:01:26.788140+0000","last_active":"2024-09-19T06:03:20.722866+0000","last_peered":"2024-09-19T06:03:20.722866+0000","last_clean":"2024-09-19T06:03:20.722866+0000","last_became_active":"2024-09-19T06:01:26.787603+0000","last_became_peered":"2024-09-19T06:01:26.787603+0000","last_unstale":"2024-09-19T06:03:20.722866+0000","last_undegraded":"2024-09-19T06:03:20.722866+0000","last_fullsized":"2024-09-19T06:03:20.722866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:13.880656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628239+0000","last_change":"2024-09-19T06:01:28.813144+0000","last_active":"2024-09-19T06:03:20.628239+0000","last_peered":"2024-09-19T06:03:20.628239+0000","last_clean":"2024-09-19T06:03:20.628239+0000","last_became_active":"2024-09-19T06:01:28.812792+0000","last_became_peered":"2024-09-19T06:01:28.812792+0000","last_unstale":"2024-09-19T06:03:20.628239+0000","last_undegraded":"2024-09-19T06:03:20.628239+0000","last_fullsized":"2024-09-19T06:03:20.628239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:35.896240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628078+0000","last_change":"2024-09-19T06:01:28.812086+0000","last_active":"2024-09-19T06:03:20.628078+0000","last_peered":"2024-09-19T06:03:20.628078+0000","last_clean":"2024-09-19T06:03:20.628078+0000","last_became_active":"2024-09-19T06:01:28.811696+0000","last_became_peered":"2024-09-19T06:01:28.811696+0000","last_unstale":"2024-09-19T06:03:20.628078+0000","last_undegraded":"2024-09-19T06:03:20.628078+0000","last_fullsized":"2024-09-19T06:03:20.628078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:32.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887254+0000","last_change":"2024-09-19T06:01:39.842250+0000","last_active":"2024-09-19T06:03:20.887254+0000","last_peered":"2024-09-19T06:03:20.887254+0000","last_clean":"2024-09-19T06:03:20.887254+0000","last_became_active":"2024-09-19T06:01:39.835761+0000","last_became_peered":"2024-09-19T06:01:39.835761+0000","last_unstale":"2024-09-19T06:03:20.887254+0000","last_undegraded":"2024-09-19T06:03:20.887254+0000","last_fullsized":"2024-09-19T06:03:20.887254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:51.026349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884914+0000","last_change":"2024-09-19T06:01:28.812983+0000","last_active":"2024-09-19T06:03:20.884914+0000","last_peered":"2024-09-19T06:03:20.884914+0000","last_clean":"2024-09-19T06:03:20.884914+0000","last_became_active":"2024-09-19T06:01:28.812257+0000","last_became_peered":"2024-09-19T06:01:28.812257+0000","last_unstale":"2024-09-19T06:03:20.884914+0000","last_undegraded":"2024-09-19T06:03:20.884914+0000","last_fullsized":"2024-09-19T06:03:20.884914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:41:42.059736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723024+0000","last_change":"2024-09-19T06:01:26.787869+0000","last_active":"2024-09-19T06:03:20.723024+0000","last_peered":"2024-09-19T06:03:20.723024+0000","last_clean":"2024-09-19T06:03:20.723024+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:03:20.723024+0000","last_undegraded":"2024-09-19T06:03:20.723024+0000","last_fullsized":"2024-09-19T06:03:20.723024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:02.869878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.889703+0000","last_change":"2024-09-19T06:01:28.811986+0000","last_active":"2024-09-19T06:03:20.889703+0000","last_peered":"2024-09-19T06:03:20.889703+0000","last_clean":"2024-09-19T06:03:20.889703+0000","last_became_active":"2024-09-19T06:01:28.811565+0000","last_became_peered":"2024-09-19T06:01:28.811565+0000","last_unstale":"2024-09-19T06:03:20.889703+0000","last_undegraded":"2024-09-19T06:03:20.889703+0000","last_fullsized":"2024-09-19T06:03:20.889703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:01.324985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723946+0000","last_change":"2024-09-19T06:01:27.021683+0000","last_active":"2024-09-19T06:03:20.723946+0000","last_peered":"2024-09-19T06:03:20.723946+0000","last_clean":"2024-09-19T06:03:20.723946+0000","last_became_active":"2024-09-19T06:01:27.021358+0000","last_became_peered":"2024-09-19T06:01:27.021358+0000","last_unstale":"2024-09-19T06:03:20.723946+0000","last_undegraded":"2024-09-19T06:03:20.723946+0000","last_fullsized":"2024-09-19T06:03:20.723946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:29.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723619+0000","last_change":"2024-09-19T06:01:28.798963+0000","last_active":"2024-09-19T06:03:20.723619+0000","last_peered":"2024-09-19T06:03:20.723619+0000","last_clean":"2024-09-19T06:03:20.723619+0000","last_became_active":"2024-09-19T06:01:28.797571+0000","last_became_peered":"2024-09-19T06:01:28.797571+0000","last_unstale":"2024-09-19T06:03:20.723619+0000","last_undegraded":"2024-09-19T06:03:20.723619+0000","last_fullsized":"2024-09-19T06:03:20.723619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:43.867609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.721283+0000","last_change":"2024-09-19T06:01:26.786427+0000","last_active":"2024-09-19T06:03:20.721283+0000","last_peered":"2024-09-19T06:03:20.721283+0000","last_clean":"2024-09-19T06:03:20.721283+0000","last_became_active":"2024-09-19T06:01:26.786092+0000","last_became_peered":"2024-09-19T06:01:26.786092+0000","last_unstale":"2024-09-19T06:03:20.721283+0000","last_undegraded":"2024-09-19T06:03:20.721283+0000","last_fullsized":"2024-09-19T06:03:20.721283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:49.144366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603942,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1149,"num_read_kb":694,"num_write":387,"num_write_kb":1474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1811826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":449661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41236,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703916,"statfs":{"total":95995035648,"available":95952809984,"internally_reserved":0,"allocated":1773568,"data_stored":1238219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939694,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":438118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36648,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708504,"statfs":{"total":95995035648,"available":95957508096,"internally_reserved":0,"allocated":1331200,"data_stored":839982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397559,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1548288,"data_stored":1043460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":447828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1695744,"data_stored":1172287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":450126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1482752,"data_stored":971440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40496,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704656,"statfs":{"total":95995035648,"available":95953567744,"internally_reserved":0,"allocated":1011712,"data_stored":513156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40500,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704652,"statfs":{"total":95995035648,"available":95953563648,"internally_reserved":0,"allocated":1015808,"data_stored":518742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":440203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":201774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:03:25.521 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T06:03:25.521 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T06:03:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.659+0000 7f7bec1d8640 1 -- 172.21.15.5:0/749662672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be410c0b0 msgr2=0x7f7be410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.659+0000 7f7bec1d8640 1 --2- 172.21.15.5:0/749662672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be410c0b0 0x7f7be410e550 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc009ca0 tx=0x7f7bdc02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.659+0000 7f7bec1d8640 1 -- 172.21.15.5:0/749662672 shutdown_connections 2024-09-19T06:03:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.659+0000 7f7bec1d8640 1 --2- 172.21.15.5:0/749662672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be410c0b0 0x7f7be410e550 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.659+0000 7f7bec1d8640 1 --2- 172.21.15.5:0/749662672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7be4103600 0x7f7be410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.659+0000 7f7bec1d8640 1 --2- 172.21.15.5:0/749662672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7be4102ce0 0x7f7be41030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.659+0000 7f7bec1d8640 1 -- 172.21.15.5:0/749662672 >> 172.21.15.5:0/749662672 conn(0x7f7be40fc6d0 msgr2=0x7f7be40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.659+0000 7f7bec1d8640 1 -- 172.21.15.5:0/749662672 shutdown_connections 2024-09-19T06:03:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.659+0000 7f7bec1d8640 1 -- 172.21.15.5:0/749662672 wait complete. 2024-09-19T06:03:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7bec1d8640 1 Processor -- start 2024-09-19T06:03:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7bec1d8640 1 -- start start 2024-09-19T06:03:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7bec1d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7be4102ce0 0x7f7be4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7bec1d8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7be4103600 0x7f7be41987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7bec1d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be410c0b0 0x7f7be419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7bec1d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7be4111d60 con 0x7f7be410c0b0 2024-09-19T06:03:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7bec1d8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7be4111be0 con 0x7f7be4102ce0 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7bec1d8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7be4111ee0 con 0x7f7be4103600 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7bea775640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be410c0b0 0x7f7be419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7bea775640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be410c0b0 0x7f7be419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37946/0 (socket says 172.21.15.5:37946) 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7bea775640 1 -- 172.21.15.5:0/1030941791 learned_addr learned my addr 172.21.15.5:0/1030941791 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7be9773640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7be4103600 0x7f7be41987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.660+0000 7f7be9f74640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7be4102ce0 0x7f7be4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.661+0000 7f7bea775640 1 -- 172.21.15.5:0/1030941791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7be4103600 msgr2=0x7f7be41987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.661+0000 7f7bea775640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7be4103600 0x7f7be41987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.661+0000 7f7bea775640 1 -- 172.21.15.5:0/1030941791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7be4102ce0 msgr2=0x7f7be4198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.661+0000 7f7bea775640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7be4102ce0 0x7f7be4198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.661+0000 7f7bea775640 1 -- 172.21.15.5:0/1030941791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be419d270 con 0x7f7be410c0b0 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.661+0000 7f7bea775640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be410c0b0 0x7f7be419cb40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc006010 tx=0x7f7bdc032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.661+0000 7f7bdaffd640 1 -- 172.21.15.5:0/1030941791 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc003cf0 con 0x7f7be410c0b0 2024-09-19T06:03:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.661+0000 7f7bdaffd640 1 -- 172.21.15.5:0/1030941791 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bdc0044f0 con 0x7f7be410c0b0 2024-09-19T06:03:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.661+0000 7f7bec1d8640 1 -- 172.21.15.5:0/1030941791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be419d500 con 0x7f7be410c0b0 2024-09-19T06:03:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.661+0000 7f7bdaffd640 1 -- 172.21.15.5:0/1030941791 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc030c00 con 0x7f7be410c0b0 2024-09-19T06:03:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.661+0000 7f7bec1d8640 1 -- 172.21.15.5:0/1030941791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be41a4df0 con 0x7f7be410c0b0 2024-09-19T06:03:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.662+0000 7f7bec1d8640 1 -- 172.21.15.5:0/1030941791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be41030c0 con 0x7f7be410c0b0 2024-09-19T06:03:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.663+0000 7f7bdaffd640 1 -- 172.21.15.5:0/1030941791 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7f7bdc004040 con 0x7f7be410c0b0 2024-09-19T06:03:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.664+0000 7f7bdaffd640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7bd00a34d0 0x7f7bd00a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.664+0000 7f7be9f74640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7bd00a34d0 0x7f7bd00a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.664+0000 7f7bdaffd640 1 -- 172.21.15.5:0/1030941791 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f7bdc0f12c0 con 0x7f7be410c0b0 2024-09-19T06:03:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.665+0000 7f7be9f74640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7bd00a34d0 0x7f7bd00a5990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7be0005fd0 tx=0x7f7be000b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.668+0000 7f7bdaffd640 1 -- 172.21.15.5:0/1030941791 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7bdc0bdd10 con 0x7f7be410c0b0 2024-09-19T06:03:25.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.840+0000 7f7bec1d8640 1 -- 172.21.15.5:0/1030941791 --> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7be4062630 con 0x7f7bd00a34d0 2024-09-19T06:03:25.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.847+0000 7f7bdaffd640 1 -- 172.21.15.5:0/1030941791 <== mgr.15240 v2:172.21.15.5:6800/2837801016 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257786 (secure 0 0 0) 0x7f7be4062630 con 0x7f7bd00a34d0 2024-09-19T06:03:25.851 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:03:25.852 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-19T06:03:25.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.850+0000 7f7bec1d8640 1 -- 172.21.15.5:0/1030941791 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7bd00a34d0 msgr2=0x7f7bd00a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.850+0000 7f7bec1d8640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7bd00a34d0 0x7f7bd00a5990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7be0005fd0 tx=0x7f7be000b390 comp rx=0 tx=0).stop 2024-09-19T06:03:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.850+0000 7f7bec1d8640 1 -- 172.21.15.5:0/1030941791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be410c0b0 msgr2=0x7f7be419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.850+0000 7f7bec1d8640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be410c0b0 0x7f7be419cb40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc006010 tx=0x7f7bdc032ff0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.850+0000 7f7bec1d8640 1 -- 172.21.15.5:0/1030941791 shutdown_connections 2024-09-19T06:03:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.850+0000 7f7bec1d8640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be410c0b0 0x7f7be419cb40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.850+0000 7f7bec1d8640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7bd00a34d0 0x7f7bd00a5990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.850+0000 7f7bec1d8640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7be4103600 0x7f7be41987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.850+0000 7f7bec1d8640 1 --2- 172.21.15.5:0/1030941791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7be4102ce0 0x7f7be4198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.850+0000 7f7bec1d8640 1 -- 172.21.15.5:0/1030941791 >> 172.21.15.5:0/1030941791 conn(0x7f7be40fc6d0 msgr2=0x7f7be40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.851+0000 7f7bec1d8640 1 -- 172.21.15.5:0/1030941791 shutdown_connections 2024-09-19T06:03:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:25.851+0000 7f7bec1d8640 1 -- 172.21.15.5:0/1030941791 wait complete. 2024-09-19T06:03:25.862 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:03:25.862 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:25.862 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:25.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:25.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T06:03:25.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:25.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:25.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:25.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:25.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:25.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:25.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:25.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:25.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:25 smithi005 ceph-mon[30955]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:03:25.864 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T06:03:24.640412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194544,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1285,"num_read_kb":813,"num_write":570,"num_write_kb":4414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":950,"ondisk_log_size":950,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392800,"kb_used_data":14224,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549024,"statfs":{"total":1151940427776,"available":1151538200576,"internally_reserved":0,"allocated":14565376,"data_stored":8523222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.982108"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887438+0000","last_change":"2024-09-19T06:01:28.812722+0000","last_active":"2024-09-19T06:03:20.887438+0000","last_peered":"2024-09-19T06:03:20.887438+0000","last_clean":"2024-09-19T06:03:20.887438+0000","last_became_active":"2024-09-19T06:01:28.812283+0000","last_became_peered":"2024-09-19T06:01:28.812283+0000","last_unstale":"2024-09-19T06:03:20.887438+0000","last_undegraded":"2024-09-19T06:03:20.887438+0000","last_fullsized":"2024-09-19T06:03:20.887438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:59.146636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.888342+0000","last_change":"2024-09-19T06:01:28.805210+0000","last_active":"2024-09-19T06:03:20.888342+0000","last_peered":"2024-09-19T06:03:20.888342+0000","last_clean":"2024-09-19T06:03:20.888342+0000","last_became_active":"2024-09-19T06:01:28.804906+0000","last_became_peered":"2024-09-19T06:01:28.804906+0000","last_unstale":"2024-09-19T06:03:20.888342+0000","last_undegraded":"2024-09-19T06:03:20.888342+0000","last_fullsized":"2024-09-19T06:03:20.888342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:00.227101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.889300+0000","last_change":"2024-09-19T06:01:28.811909+0000","last_active":"2024-09-19T06:03:20.889300+0000","last_peered":"2024-09-19T06:03:20.889300+0000","last_clean":"2024-09-19T06:03:20.889300+0000","last_became_active":"2024-09-19T06:01:28.811547+0000","last_became_peered":"2024-09-19T06:01:28.811547+0000","last_unstale":"2024-09-19T06:03:20.889300+0000","last_undegraded":"2024-09-19T06:03:20.889300+0000","last_fullsized":"2024-09-19T06:03:20.889300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:50.022439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.893221+0000","last_change":"2024-09-19T06:01:28.806782+0000","last_active":"2024-09-19T06:03:20.893221+0000","last_peered":"2024-09-19T06:03:20.893221+0000","last_clean":"2024-09-19T06:03:20.893221+0000","last_became_active":"2024-09-19T06:01:28.805520+0000","last_became_peered":"2024-09-19T06:01:28.805520+0000","last_unstale":"2024-09-19T06:03:20.893221+0000","last_undegraded":"2024-09-19T06:03:20.893221+0000","last_fullsized":"2024-09-19T06:03:20.893221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:01.652669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722796+0000","last_change":"2024-09-19T06:01:28.809971+0000","last_active":"2024-09-19T06:03:20.722796+0000","last_peered":"2024-09-19T06:03:20.722796+0000","last_clean":"2024-09-19T06:03:20.722796+0000","last_became_active":"2024-09-19T06:01:28.809593+0000","last_became_peered":"2024-09-19T06:01:28.809593+0000","last_unstale":"2024-09-19T06:03:20.722796+0000","last_undegraded":"2024-09-19T06:03:20.722796+0000","last_fullsized":"2024-09-19T06:03:20.722796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.546526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732082+0000","last_change":"2024-09-19T06:01:28.801395+0000","last_active":"2024-09-19T06:03:20.732082+0000","last_peered":"2024-09-19T06:03:20.732082+0000","last_clean":"2024-09-19T06:03:20.732082+0000","last_became_active":"2024-09-19T06:01:28.800987+0000","last_became_peered":"2024-09-19T06:01:28.800987+0000","last_unstale":"2024-09-19T06:03:20.732082+0000","last_undegraded":"2024-09-19T06:03:20.732082+0000","last_fullsized":"2024-09-19T06:03:20.732082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:18.986489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628078+0000","last_change":"2024-09-19T06:01:28.806554+0000","last_active":"2024-09-19T06:03:20.628078+0000","last_peered":"2024-09-19T06:03:20.628078+0000","last_clean":"2024-09-19T06:03:20.628078+0000","last_became_active":"2024-09-19T06:01:28.805946+0000","last_became_peered":"2024-09-19T06:01:28.805946+0000","last_unstale":"2024-09-19T06:03:20.628078+0000","last_undegraded":"2024-09-19T06:03:20.628078+0000","last_fullsized":"2024-09-19T06:03:20.628078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:56:20.130414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722784+0000","last_change":"2024-09-19T06:01:28.804789+0000","last_active":"2024-09-19T06:03:20.722784+0000","last_peered":"2024-09-19T06:03:20.722784+0000","last_clean":"2024-09-19T06:03:20.722784+0000","last_became_active":"2024-09-19T06:01:28.804484+0000","last_became_peered":"2024-09-19T06:01:28.804484+0000","last_unstale":"2024-09-19T06:03:20.722784+0000","last_undegraded":"2024-09-19T06:03:20.722784+0000","last_fullsized":"2024-09-19T06:03:20.722784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:19.737928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884972+0000","last_change":"2024-09-19T06:01:28.814504+0000","last_active":"2024-09-19T06:03:20.884972+0000","last_peered":"2024-09-19T06:03:20.884972+0000","last_clean":"2024-09-19T06:03:20.884972+0000","last_became_active":"2024-09-19T06:01:28.814066+0000","last_became_peered":"2024-09-19T06:01:28.814066+0000","last_unstale":"2024-09-19T06:03:20.884972+0000","last_undegraded":"2024-09-19T06:03:20.884972+0000","last_fullsized":"2024-09-19T06:03:20.884972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:50.904164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732391+0000","last_change":"2024-09-19T06:01:28.807303+0000","last_active":"2024-09-19T06:03:20.732391+0000","last_peered":"2024-09-19T06:03:20.732391+0000","last_clean":"2024-09-19T06:03:20.732391+0000","last_became_active":"2024-09-19T06:01:28.806978+0000","last_became_peered":"2024-09-19T06:01:28.806978+0000","last_unstale":"2024-09-19T06:03:20.732391+0000","last_undegraded":"2024-09-19T06:03:20.732391+0000","last_fullsized":"2024-09-19T06:03:20.732391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:33.259261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628504+0000","last_change":"2024-09-19T06:01:28.814844+0000","last_active":"2024-09-19T06:03:20.628504+0000","last_peered":"2024-09-19T06:03:20.628504+0000","last_clean":"2024-09-19T06:03:20.628504+0000","last_became_active":"2024-09-19T06:01:28.814531+0000","last_became_peered":"2024-09-19T06:01:28.814531+0000","last_unstale":"2024-09-19T06:03:20.628504+0000","last_undegraded":"2024-09-19T06:03:20.628504+0000","last_fullsized":"2024-09-19T06:03:20.628504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:17.209890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.893037+0000","last_change":"2024-09-19T06:01:28.806378+0000","last_active":"2024-09-19T06:03:20.893037+0000","last_peered":"2024-09-19T06:03:20.893037+0000","last_clean":"2024-09-19T06:03:20.893037+0000","last_became_active":"2024-09-19T06:01:28.805053+0000","last_became_peered":"2024-09-19T06:01:28.805053+0000","last_unstale":"2024-09-19T06:03:20.893037+0000","last_undegraded":"2024-09-19T06:03:20.893037+0000","last_fullsized":"2024-09-19T06:03:20.893037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:15.080548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627148+0000","last_change":"2024-09-19T06:01:28.802941+0000","last_active":"2024-09-19T06:03:20.627148+0000","last_peered":"2024-09-19T06:03:20.627148+0000","last_clean":"2024-09-19T06:03:20.627148+0000","last_became_active":"2024-09-19T06:01:28.802590+0000","last_became_peered":"2024-09-19T06:01:28.802590+0000","last_unstale":"2024-09-19T06:03:20.627148+0000","last_undegraded":"2024-09-19T06:03:20.627148+0000","last_fullsized":"2024-09-19T06:03:20.627148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:25.581854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887443+0000","last_change":"2024-09-19T06:01:28.800960+0000","last_active":"2024-09-19T06:03:20.887443+0000","last_peered":"2024-09-19T06:03:20.887443+0000","last_clean":"2024-09-19T06:03:20.887443+0000","last_became_active":"2024-09-19T06:01:28.800484+0000","last_became_peered":"2024-09-19T06:01:28.800484+0000","last_unstale":"2024-09-19T06:03:20.887443+0000","last_undegraded":"2024-09-19T06:03:20.887443+0000","last_fullsized":"2024-09-19T06:03:20.887443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:08.506105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.888999+0000","last_change":"2024-09-19T06:01:28.800771+0000","last_active":"2024-09-19T06:03:20.888999+0000","last_peered":"2024-09-19T06:03:20.888999+0000","last_clean":"2024-09-19T06:03:20.888999+0000","last_became_active":"2024-09-19T06:01:28.800427+0000","last_became_peered":"2024-09-19T06:01:28.800427+0000","last_unstale":"2024-09-19T06:03:20.888999+0000","last_undegraded":"2024-09-19T06:03:20.888999+0000","last_fullsized":"2024-09-19T06:03:20.888999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:19.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884875+0000","last_change":"2024-09-19T06:01:28.815528+0000","last_active":"2024-09-19T06:03:20.884875+0000","last_peered":"2024-09-19T06:03:20.884875+0000","last_clean":"2024-09-19T06:03:20.884875+0000","last_became_active":"2024-09-19T06:01:28.815129+0000","last_became_peered":"2024-09-19T06:01:28.815129+0000","last_unstale":"2024-09-19T06:03:20.884875+0000","last_undegraded":"2024-09-19T06:03:20.884875+0000","last_fullsized":"2024-09-19T06:03:20.884875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:01.936711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887589+0000","last_change":"2024-09-19T06:01:28.816120+0000","last_active":"2024-09-19T06:03:20.887589+0000","last_peered":"2024-09-19T06:03:20.887589+0000","last_clean":"2024-09-19T06:03:20.887589+0000","last_became_active":"2024-09-19T06:01:28.815822+0000","last_became_peered":"2024-09-19T06:01:28.815822+0000","last_unstale":"2024-09-19T06:03:20.887589+0000","last_undegraded":"2024-09-19T06:03:20.887589+0000","last_fullsized":"2024-09-19T06:03:20.887589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:14:58.670452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.721021+0000","last_change":"2024-09-19T06:01:28.809825+0000","last_active":"2024-09-19T06:03:20.721021+0000","last_peered":"2024-09-19T06:03:20.721021+0000","last_clean":"2024-09-19T06:03:20.721021+0000","last_became_active":"2024-09-19T06:01:28.809530+0000","last_became_peered":"2024-09-19T06:01:28.809530+0000","last_unstale":"2024-09-19T06:03:20.721021+0000","last_undegraded":"2024-09-19T06:03:20.721021+0000","last_fullsized":"2024-09-19T06:03:20.721021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:49:23.442436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.895924+0000","last_change":"2024-09-19T06:01:28.801297+0000","last_active":"2024-09-19T06:03:20.895924+0000","last_peered":"2024-09-19T06:03:20.895924+0000","last_clean":"2024-09-19T06:03:20.895924+0000","last_became_active":"2024-09-19T06:01:28.800984+0000","last_became_peered":"2024-09-19T06:01:28.800984+0000","last_unstale":"2024-09-19T06:03:20.895924+0000","last_undegraded":"2024-09-19T06:03:20.895924+0000","last_fullsized":"2024-09-19T06:03:20.895924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:01.485951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887070+0000","last_change":"2024-09-19T06:01:28.804567+0000","last_active":"2024-09-19T06:03:20.887070+0000","last_peered":"2024-09-19T06:03:20.887070+0000","last_clean":"2024-09-19T06:03:20.887070+0000","last_became_active":"2024-09-19T06:01:28.804213+0000","last_became_peered":"2024-09-19T06:01:28.804213+0000","last_unstale":"2024-09-19T06:03:20.887070+0000","last_undegraded":"2024-09-19T06:03:20.887070+0000","last_fullsized":"2024-09-19T06:03:20.887070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.186460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627307+0000","last_change":"2024-09-19T06:01:28.801961+0000","last_active":"2024-09-19T06:03:20.627307+0000","last_peered":"2024-09-19T06:03:20.627307+0000","last_clean":"2024-09-19T06:03:20.627307+0000","last_became_active":"2024-09-19T06:01:28.801631+0000","last_became_peered":"2024-09-19T06:01:28.801631+0000","last_unstale":"2024-09-19T06:03:20.627307+0000","last_undegraded":"2024-09-19T06:03:20.627307+0000","last_fullsized":"2024-09-19T06:03:20.627307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:15.628908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884896+0000","last_change":"2024-09-19T06:01:28.815160+0000","last_active":"2024-09-19T06:03:20.884896+0000","last_peered":"2024-09-19T06:03:20.884896+0000","last_clean":"2024-09-19T06:03:20.884896+0000","last_became_active":"2024-09-19T06:01:28.813329+0000","last_became_peered":"2024-09-19T06:01:28.813329+0000","last_unstale":"2024-09-19T06:03:20.884896+0000","last_undegraded":"2024-09-19T06:03:20.884896+0000","last_fullsized":"2024-09-19T06:03:20.884896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:17.027393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722875+0000","last_change":"2024-09-19T06:01:28.799205+0000","last_active":"2024-09-19T06:03:20.722875+0000","last_peered":"2024-09-19T06:03:20.722875+0000","last_clean":"2024-09-19T06:03:20.722875+0000","last_became_active":"2024-09-19T06:01:28.798891+0000","last_became_peered":"2024-09-19T06:01:28.798891+0000","last_unstale":"2024-09-19T06:03:20.722875+0000","last_undegraded":"2024-09-19T06:03:20.722875+0000","last_fullsized":"2024-09-19T06:03:20.722875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:41.227721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627907+0000","last_change":"2024-09-19T06:01:28.795823+0000","last_active":"2024-09-19T06:03:20.627907+0000","last_peered":"2024-09-19T06:03:20.627907+0000","last_clean":"2024-09-19T06:03:20.627907+0000","last_became_active":"2024-09-19T06:01:28.795510+0000","last_became_peered":"2024-09-19T06:01:28.795510+0000","last_unstale":"2024-09-19T06:03:20.627907+0000","last_undegraded":"2024-09-19T06:03:20.627907+0000","last_fullsized":"2024-09-19T06:03:20.627907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:54.831610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723807+0000","last_change":"2024-09-19T06:01:28.799696+0000","last_active":"2024-09-19T06:03:20.723807+0000","last_peered":"2024-09-19T06:03:20.723807+0000","last_clean":"2024-09-19T06:03:20.723807+0000","last_became_active":"2024-09-19T06:01:28.799417+0000","last_became_peered":"2024-09-19T06:01:28.799417+0000","last_unstale":"2024-09-19T06:03:20.723807+0000","last_undegraded":"2024-09-19T06:03:20.723807+0000","last_fullsized":"2024-09-19T06:03:20.723807+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:02.330808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.896104+0000","last_change":"2024-09-19T06:01:28.807146+0000","last_active":"2024-09-19T06:03:20.896104+0000","last_peered":"2024-09-19T06:03:20.896104+0000","last_clean":"2024-09-19T06:03:20.896104+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:03:20.896104+0000","last_undegraded":"2024-09-19T06:03:20.896104+0000","last_fullsized":"2024-09-19T06:03:20.896104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:12.308421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887102+0000","last_change":"2024-09-19T06:01:28.803555+0000","last_active":"2024-09-19T06:03:20.887102+0000","last_peered":"2024-09-19T06:03:20.887102+0000","last_clean":"2024-09-19T06:03:20.887102+0000","last_became_active":"2024-09-19T06:01:28.803242+0000","last_became_peered":"2024-09-19T06:01:28.803242+0000","last_unstale":"2024-09-19T06:03:20.887102+0000","last_undegraded":"2024-09-19T06:03:20.887102+0000","last_fullsized":"2024-09-19T06:03:20.887102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:02.281152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626997+0000","last_change":"2024-09-19T06:01:28.804378+0000","last_active":"2024-09-19T06:03:20.626997+0000","last_peered":"2024-09-19T06:03:20.626997+0000","last_clean":"2024-09-19T06:03:20.626997+0000","last_became_active":"2024-09-19T06:01:28.804099+0000","last_became_peered":"2024-09-19T06:01:28.804099+0000","last_unstale":"2024-09-19T06:03:20.626997+0000","last_undegraded":"2024-09-19T06:03:20.626997+0000","last_fullsized":"2024-09-19T06:03:20.626997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:11.558384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887754+0000","last_change":"2024-09-19T06:01:28.799999+0000","last_active":"2024-09-19T06:03:20.887754+0000","last_peered":"2024-09-19T06:03:20.887754+0000","last_clean":"2024-09-19T06:03:20.887754+0000","last_became_active":"2024-09-19T06:01:28.799645+0000","last_became_peered":"2024-09-19T06:01:28.799645+0000","last_unstale":"2024-09-19T06:03:20.887754+0000","last_undegraded":"2024-09-19T06:03:20.887754+0000","last_fullsized":"2024-09-19T06:03:20.887754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:20.442119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.895777+0000","last_change":"2024-09-19T06:01:28.800143+0000","last_active":"2024-09-19T06:03:20.895777+0000","last_peered":"2024-09-19T06:03:20.895777+0000","last_clean":"2024-09-19T06:03:20.895777+0000","last_became_active":"2024-09-19T06:01:28.799830+0000","last_became_peered":"2024-09-19T06:01:28.799830+0000","last_unstale":"2024-09-19T06:03:20.895777+0000","last_undegraded":"2024-09-19T06:03:20.895777+0000","last_fullsized":"2024-09-19T06:03:20.895777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:43.824740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.895772+0000","last_change":"2024-09-19T06:01:28.806130+0000","last_active":"2024-09-19T06:03:20.895772+0000","last_peered":"2024-09-19T06:03:20.895772+0000","last_clean":"2024-09-19T06:03:20.895772+0000","last_became_active":"2024-09-19T06:01:28.805772+0000","last_became_peered":"2024-09-19T06:01:28.805772+0000","last_unstale":"2024-09-19T06:03:20.895772+0000","last_undegraded":"2024-09-19T06:03:20.895772+0000","last_fullsized":"2024-09-19T06:03:20.895772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:00.316311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887402+0000","last_change":"2024-09-19T06:01:39.837412+0000","last_active":"2024-09-19T06:03:20.887402+0000","last_peered":"2024-09-19T06:03:20.887402+0000","last_clean":"2024-09-19T06:03:20.887402+0000","last_became_active":"2024-09-19T06:01:39.837035+0000","last_became_peered":"2024-09-19T06:01:39.837035+0000","last_unstale":"2024-09-19T06:03:20.887402+0000","last_undegraded":"2024-09-19T06:03:20.887402+0000","last_fullsized":"2024-09-19T06:03:20.887402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:00.073884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722768+0000","last_change":"2024-09-19T06:01:28.800324+0000","last_active":"2024-09-19T06:03:20.722768+0000","last_peered":"2024-09-19T06:03:20.722768+0000","last_clean":"2024-09-19T06:03:20.722768+0000","last_became_active":"2024-09-19T06:01:28.799986+0000","last_became_peered":"2024-09-19T06:01:28.799986+0000","last_unstale":"2024-09-19T06:03:20.722768+0000","last_undegraded":"2024-09-19T06:03:20.722768+0000","last_fullsized":"2024-09-19T06:03:20.722768+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:41.963346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":151,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.978373+0000","last_change":"2024-09-19T06:01:39.844897+0000","last_active":"2024-09-19T06:03:20.978373+0000","last_peered":"2024-09-19T06:03:20.978373+0000","last_clean":"2024-09-19T06:03:20.978373+0000","last_became_active":"2024-09-19T06:01:39.838347+0000","last_became_peered":"2024-09-19T06:01:39.838347+0000","last_unstale":"2024-09-19T06:03:20.978373+0000","last_undegraded":"2024-09-19T06:03:20.978373+0000","last_fullsized":"2024-09-19T06:03:20.978373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:26.235227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2044,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628020+0000","last_change":"2024-09-19T06:01:28.802644+0000","last_active":"2024-09-19T06:03:20.628020+0000","last_peered":"2024-09-19T06:03:20.628020+0000","last_clean":"2024-09-19T06:03:20.628020+0000","last_became_active":"2024-09-19T06:01:28.802288+0000","last_became_peered":"2024-09-19T06:01:28.802288+0000","last_unstale":"2024-09-19T06:03:20.628020+0000","last_undegraded":"2024-09-19T06:03:20.628020+0000","last_fullsized":"2024-09-19T06:03:20.628020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:32.031860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732271+0000","last_change":"2024-09-19T06:01:26.781104+0000","last_active":"2024-09-19T06:03:20.732271+0000","last_peered":"2024-09-19T06:03:20.732271+0000","last_clean":"2024-09-19T06:03:20.732271+0000","last_became_active":"2024-09-19T06:01:26.780733+0000","last_became_peered":"2024-09-19T06:01:26.780733+0000","last_unstale":"2024-09-19T06:03:20.732271+0000","last_undegraded":"2024-09-19T06:03:20.732271+0000","last_fullsized":"2024-09-19T06:03:20.732271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:54.490867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626771+0000","last_change":"2024-09-19T06:01:26.782901+0000","last_active":"2024-09-19T06:03:20.626771+0000","last_peered":"2024-09-19T06:03:20.626771+0000","last_clean":"2024-09-19T06:03:20.626771+0000","last_became_active":"2024-09-19T06:01:26.782450+0000","last_became_peered":"2024-09-19T06:01:26.782450+0000","last_unstale":"2024-09-19T06:03:20.626771+0000","last_undegraded":"2024-09-19T06:03:20.626771+0000","last_fullsized":"2024-09-19T06:03:20.626771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:07.948680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722809+0000","last_change":"2024-09-19T06:01:28.804105+0000","last_active":"2024-09-19T06:03:20.722809+0000","last_peered":"2024-09-19T06:03:20.722809+0000","last_clean":"2024-09-19T06:03:20.722809+0000","last_became_active":"2024-09-19T06:01:28.797980+0000","last_became_peered":"2024-09-19T06:01:28.797980+0000","last_unstale":"2024-09-19T06:03:20.722809+0000","last_undegraded":"2024-09-19T06:03:20.722809+0000","last_fullsized":"2024-09-19T06:03:20.722809+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:50.130102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628139+0000","last_change":"2024-09-19T06:01:26.786923+0000","last_active":"2024-09-19T06:03:20.628139+0000","last_peered":"2024-09-19T06:03:20.628139+0000","last_clean":"2024-09-19T06:03:20.628139+0000","last_became_active":"2024-09-19T06:01:26.786542+0000","last_became_peered":"2024-09-19T06:01:26.786542+0000","last_unstale":"2024-09-19T06:03:20.628139+0000","last_undegraded":"2024-09-19T06:03:20.628139+0000","last_fullsized":"2024-09-19T06:03:20.628139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:30.032992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.720767+0000","last_change":"2024-09-19T06:01:28.802172+0000","last_active":"2024-09-19T06:03:20.720767+0000","last_peered":"2024-09-19T06:03:20.720767+0000","last_clean":"2024-09-19T06:03:20.720767+0000","last_became_active":"2024-09-19T06:01:28.801815+0000","last_became_peered":"2024-09-19T06:01:28.801815+0000","last_unstale":"2024-09-19T06:03:20.720767+0000","last_undegraded":"2024-09-19T06:03:20.720767+0000","last_fullsized":"2024-09-19T06:03:20.720767+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:09.587638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":230,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.977949+0000","last_change":"2024-09-19T06:01:26.777870+0000","last_active":"2024-09-19T06:03:20.977949+0000","last_peered":"2024-09-19T06:03:20.977949+0000","last_clean":"2024-09-19T06:03:20.977949+0000","last_became_active":"2024-09-19T06:01:26.777509+0000","last_became_peered":"2024-09-19T06:01:26.777509+0000","last_unstale":"2024-09-19T06:03:20.977949+0000","last_undegraded":"2024-09-19T06:03:20.977949+0000","last_fullsized":"2024-09-19T06:03:20.977949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:15.885911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884945+0000","last_change":"2024-09-19T06:01:28.812201+0000","last_active":"2024-09-19T06:03:20.884945+0000","last_peered":"2024-09-19T06:03:20.884945+0000","last_clean":"2024-09-19T06:03:20.884945+0000","last_became_active":"2024-09-19T06:01:28.811801+0000","last_became_peered":"2024-09-19T06:01:28.811801+0000","last_unstale":"2024-09-19T06:03:20.884945+0000","last_undegraded":"2024-09-19T06:03:20.884945+0000","last_fullsized":"2024-09-19T06:03:20.884945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:32.904787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628546+0000","last_change":"2024-09-19T06:01:26.787845+0000","last_active":"2024-09-19T06:03:20.628546+0000","last_peered":"2024-09-19T06:03:20.628546+0000","last_clean":"2024-09-19T06:03:20.628546+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:03:20.628546+0000","last_undegraded":"2024-09-19T06:03:20.628546+0000","last_fullsized":"2024-09-19T06:03:20.628546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.114865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.889521+0000","last_change":"2024-09-19T06:01:28.801523+0000","last_active":"2024-09-19T06:03:20.889521+0000","last_peered":"2024-09-19T06:03:20.889521+0000","last_clean":"2024-09-19T06:03:20.889521+0000","last_became_active":"2024-09-19T06:01:28.801205+0000","last_became_peered":"2024-09-19T06:01:28.801205+0000","last_unstale":"2024-09-19T06:03:20.889521+0000","last_undegraded":"2024-09-19T06:03:20.889521+0000","last_fullsized":"2024-09-19T06:03:20.889521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:10.314085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628080+0000","last_change":"2024-09-19T06:01:27.022172+0000","last_active":"2024-09-19T06:03:20.628080+0000","last_peered":"2024-09-19T06:03:20.628080+0000","last_clean":"2024-09-19T06:03:20.628080+0000","last_became_active":"2024-09-19T06:01:27.021568+0000","last_became_peered":"2024-09-19T06:01:27.021568+0000","last_unstale":"2024-09-19T06:03:20.628080+0000","last_undegraded":"2024-09-19T06:03:20.628080+0000","last_fullsized":"2024-09-19T06:03:20.628080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:18.031690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.885201+0000","last_change":"2024-09-19T06:01:39.839061+0000","last_active":"2024-09-19T06:03:20.885201+0000","last_peered":"2024-09-19T06:03:20.885201+0000","last_clean":"2024-09-19T06:03:20.885201+0000","last_became_active":"2024-09-19T06:01:39.838644+0000","last_became_peered":"2024-09-19T06:01:39.838644+0000","last_unstale":"2024-09-19T06:03:20.885201+0000","last_undegraded":"2024-09-19T06:03:20.885201+0000","last_fullsized":"2024-09-19T06:03:20.885201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:02.136222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'37","reported_seq":273,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.977666+0000","last_change":"2024-09-19T06:01:26.784781+0000","last_active":"2024-09-19T06:03:20.977666+0000","last_peered":"2024-09-19T06:03:20.977666+0000","last_clean":"2024-09-19T06:03:20.977666+0000","last_became_active":"2024-09-19T06:01:26.784284+0000","last_became_peered":"2024-09-19T06:01:26.784284+0000","last_unstale":"2024-09-19T06:03:20.977666+0000","last_undegraded":"2024-09-19T06:03:20.977666+0000","last_fullsized":"2024-09-19T06:03:20.977666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:13.063299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67016,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":212,"num_read_kb":170,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722979+0000","last_change":"2024-09-19T06:01:28.802604+0000","last_active":"2024-09-19T06:03:20.722979+0000","last_peered":"2024-09-19T06:03:20.722979+0000","last_clean":"2024-09-19T06:03:20.722979+0000","last_became_active":"2024-09-19T06:01:28.802285+0000","last_became_peered":"2024-09-19T06:01:28.802285+0000","last_unstale":"2024-09-19T06:03:20.722979+0000","last_undegraded":"2024-09-19T06:03:20.722979+0000","last_fullsized":"2024-09-19T06:03:20.722979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:57.177791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.895972+0000","last_change":"2024-09-19T06:01:26.783678+0000","last_active":"2024-09-19T06:03:20.895972+0000","last_peered":"2024-09-19T06:03:20.895972+0000","last_clean":"2024-09-19T06:03:20.895972+0000","last_became_active":"2024-09-19T06:01:26.783165+0000","last_became_peered":"2024-09-19T06:01:26.783165+0000","last_unstale":"2024-09-19T06:03:20.895972+0000","last_undegraded":"2024-09-19T06:03:20.895972+0000","last_fullsized":"2024-09-19T06:03:20.895972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:50.535733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626891+0000","last_change":"2024-09-19T06:01:28.808553+0000","last_active":"2024-09-19T06:03:20.626891+0000","last_peered":"2024-09-19T06:03:20.626891+0000","last_clean":"2024-09-19T06:03:20.626891+0000","last_became_active":"2024-09-19T06:01:28.808215+0000","last_became_peered":"2024-09-19T06:01:28.808215+0000","last_unstale":"2024-09-19T06:03:20.626891+0000","last_undegraded":"2024-09-19T06:03:20.626891+0000","last_fullsized":"2024-09-19T06:03:20.626891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:55.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627942+0000","last_change":"2024-09-19T06:01:26.787137+0000","last_active":"2024-09-19T06:03:20.627942+0000","last_peered":"2024-09-19T06:03:20.627942+0000","last_clean":"2024-09-19T06:03:20.627942+0000","last_became_active":"2024-09-19T06:01:26.786676+0000","last_became_peered":"2024-09-19T06:01:26.786676+0000","last_unstale":"2024-09-19T06:03:20.627942+0000","last_undegraded":"2024-09-19T06:03:20.627942+0000","last_fullsized":"2024-09-19T06:03:20.627942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.609969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628199+0000","last_change":"2024-09-19T06:01:28.812106+0000","last_active":"2024-09-19T06:03:20.628199+0000","last_peered":"2024-09-19T06:03:20.628199+0000","last_clean":"2024-09-19T06:03:20.628199+0000","last_became_active":"2024-09-19T06:01:28.811771+0000","last_became_peered":"2024-09-19T06:01:28.811771+0000","last_unstale":"2024-09-19T06:03:20.628199+0000","last_undegraded":"2024-09-19T06:03:20.628199+0000","last_fullsized":"2024-09-19T06:03:20.628199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:07:06.832685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'9","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.978122+0000","last_change":"2024-09-19T06:01:26.786746+0000","last_active":"2024-09-19T06:03:20.978122+0000","last_peered":"2024-09-19T06:03:20.978122+0000","last_clean":"2024-09-19T06:03:20.978122+0000","last_became_active":"2024-09-19T06:01:26.786414+0000","last_became_peered":"2024-09-19T06:01:26.786414+0000","last_unstale":"2024-09-19T06:03:20.978122+0000","last_undegraded":"2024-09-19T06:03:20.978122+0000","last_fullsized":"2024-09-19T06:03:20.978122+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:25.042796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627937+0000","last_change":"2024-09-19T06:01:28.807183+0000","last_active":"2024-09-19T06:03:20.627937+0000","last_peered":"2024-09-19T06:03:20.627937+0000","last_clean":"2024-09-19T06:03:20.627937+0000","last_became_active":"2024-09-19T06:01:28.806837+0000","last_became_peered":"2024-09-19T06:01:28.806837+0000","last_unstale":"2024-09-19T06:03:20.627937+0000","last_undegraded":"2024-09-19T06:03:20.627937+0000","last_fullsized":"2024-09-19T06:03:20.627937+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:23.030312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722771+0000","last_change":"2024-09-19T06:01:27.022409+0000","last_active":"2024-09-19T06:03:20.722771+0000","last_peered":"2024-09-19T06:03:20.722771+0000","last_clean":"2024-09-19T06:03:20.722771+0000","last_became_active":"2024-09-19T06:01:27.022034+0000","last_became_peered":"2024-09-19T06:01:27.022034+0000","last_unstale":"2024-09-19T06:03:20.722771+0000","last_undegraded":"2024-09-19T06:03:20.722771+0000","last_fullsized":"2024-09-19T06:03:20.722771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:08.663575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627932+0000","last_change":"2024-09-19T06:01:28.812079+0000","last_active":"2024-09-19T06:03:20.627932+0000","last_peered":"2024-09-19T06:03:20.627932+0000","last_clean":"2024-09-19T06:03:20.627932+0000","last_became_active":"2024-09-19T06:01:28.811606+0000","last_became_peered":"2024-09-19T06:01:28.811606+0000","last_unstale":"2024-09-19T06:03:20.627932+0000","last_undegraded":"2024-09-19T06:03:20.627932+0000","last_fullsized":"2024-09-19T06:03:20.627932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:49.434235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723153+0000","last_change":"2024-09-19T06:01:26.783823+0000","last_active":"2024-09-19T06:03:20.723153+0000","last_peered":"2024-09-19T06:03:20.723153+0000","last_clean":"2024-09-19T06:03:20.723153+0000","last_became_active":"2024-09-19T06:01:26.783209+0000","last_became_peered":"2024-09-19T06:01:26.783209+0000","last_unstale":"2024-09-19T06:03:20.723153+0000","last_undegraded":"2024-09-19T06:03:20.723153+0000","last_fullsized":"2024-09-19T06:03:20.723153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:38.833417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887395+0000","last_change":"2024-09-19T06:01:39.837462+0000","last_active":"2024-09-19T06:03:20.887395+0000","last_peered":"2024-09-19T06:03:20.887395+0000","last_clean":"2024-09-19T06:03:20.887395+0000","last_became_active":"2024-09-19T06:01:39.837082+0000","last_became_peered":"2024-09-19T06:01:39.837082+0000","last_unstale":"2024-09-19T06:03:20.887395+0000","last_undegraded":"2024-09-19T06:03:20.887395+0000","last_fullsized":"2024-09-19T06:03:20.887395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:47.421017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627424+0000","last_change":"2024-09-19T06:01:26.779992+0000","last_active":"2024-09-19T06:03:20.627424+0000","last_peered":"2024-09-19T06:03:20.627424+0000","last_clean":"2024-09-19T06:03:20.627424+0000","last_became_active":"2024-09-19T06:01:26.779609+0000","last_became_peered":"2024-09-19T06:01:26.779609+0000","last_unstale":"2024-09-19T06:03:20.627424+0000","last_undegraded":"2024-09-19T06:03:20.627424+0000","last_fullsized":"2024-09-19T06:03:20.627424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:48.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628684+0000","last_change":"2024-09-19T06:01:28.798827+0000","last_active":"2024-09-19T06:03:20.628684+0000","last_peered":"2024-09-19T06:03:20.628684+0000","last_clean":"2024-09-19T06:03:20.628684+0000","last_became_active":"2024-09-19T06:01:28.798457+0000","last_became_peered":"2024-09-19T06:01:28.798457+0000","last_unstale":"2024-09-19T06:03:20.628684+0000","last_undegraded":"2024-09-19T06:03:20.628684+0000","last_fullsized":"2024-09-19T06:03:20.628684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:41.111051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'41","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.850980+0000","last_change":"2024-09-19T06:01:26.775691+0000","last_active":"2024-09-19T06:03:20.850980+0000","last_peered":"2024-09-19T06:03:20.850980+0000","last_clean":"2024-09-19T06:03:20.850980+0000","last_became_active":"2024-09-19T06:01:26.775183+0000","last_became_peered":"2024-09-19T06:01:26.775183+0000","last_unstale":"2024-09-19T06:03:20.850980+0000","last_undegraded":"2024-09-19T06:03:20.850980+0000","last_fullsized":"2024-09-19T06:03:20.850980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:34.712968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628457+0000","last_change":"2024-09-19T06:01:28.809459+0000","last_active":"2024-09-19T06:03:20.628457+0000","last_peered":"2024-09-19T06:03:20.628457+0000","last_clean":"2024-09-19T06:03:20.628457+0000","last_became_active":"2024-09-19T06:01:28.809046+0000","last_became_peered":"2024-09-19T06:01:28.809046+0000","last_unstale":"2024-09-19T06:03:20.628457+0000","last_undegraded":"2024-09-19T06:03:20.628457+0000","last_fullsized":"2024-09-19T06:03:20.628457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:56.996190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'10","reported_seq":132,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.984256+0000","last_change":"2024-09-19T06:01:39.843932+0000","last_active":"2024-09-19T06:03:20.984256+0000","last_peered":"2024-09-19T06:03:20.984256+0000","last_clean":"2024-09-19T06:03:20.984256+0000","last_became_active":"2024-09-19T06:01:39.836008+0000","last_became_peered":"2024-09-19T06:01:39.836008+0000","last_unstale":"2024-09-19T06:03:20.984256+0000","last_undegraded":"2024-09-19T06:03:20.984256+0000","last_fullsized":"2024-09-19T06:03:20.984256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:58.798859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2985,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628540+0000","last_change":"2024-09-19T06:01:28.814181+0000","last_active":"2024-09-19T06:03:20.628540+0000","last_peered":"2024-09-19T06:03:20.628540+0000","last_clean":"2024-09-19T06:03:20.628540+0000","last_became_active":"2024-09-19T06:01:28.813817+0000","last_became_peered":"2024-09-19T06:01:28.813817+0000","last_unstale":"2024-09-19T06:03:20.628540+0000","last_undegraded":"2024-09-19T06:03:20.628540+0000","last_fullsized":"2024-09-19T06:03:20.628540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:40.057863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628100+0000","last_change":"2024-09-19T06:00:17.624509+0000","last_active":"2024-09-19T06:03:20.628100+0000","last_peered":"2024-09-19T06:03:20.628100+0000","last_clean":"2024-09-19T06:03:20.628100+0000","last_became_active":"2024-09-19T06:00:17.623943+0000","last_became_peered":"2024-09-19T06:00:17.623943+0000","last_unstale":"2024-09-19T06:03:20.628100+0000","last_undegraded":"2024-09-19T06:03:20.628100+0000","last_fullsized":"2024-09-19T06:03:20.628100+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:56:55.414488+0000","last_clean_scrub_stamp":"2024-09-19T05:56:55.414488+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:39.116345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.893738+0000","last_change":"2024-09-19T06:01:26.783896+0000","last_active":"2024-09-19T06:03:20.893738+0000","last_peered":"2024-09-19T06:03:20.893738+0000","last_clean":"2024-09-19T06:03:20.893738+0000","last_became_active":"2024-09-19T06:01:26.783332+0000","last_became_peered":"2024-09-19T06:01:26.783332+0000","last_unstale":"2024-09-19T06:03:20.893738+0000","last_undegraded":"2024-09-19T06:03:20.893738+0000","last_fullsized":"2024-09-19T06:03:20.893738+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:33.072763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723257+0000","last_change":"2024-09-19T06:01:28.807756+0000","last_active":"2024-09-19T06:03:20.723257+0000","last_peered":"2024-09-19T06:03:20.723257+0000","last_clean":"2024-09-19T06:03:20.723257+0000","last_became_active":"2024-09-19T06:01:28.807077+0000","last_became_peered":"2024-09-19T06:01:28.807077+0000","last_unstale":"2024-09-19T06:03:20.723257+0000","last_undegraded":"2024-09-19T06:03:20.723257+0000","last_fullsized":"2024-09-19T06:03:20.723257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:20.784199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628079+0000","last_change":"2024-09-19T06:01:26.784830+0000","last_active":"2024-09-19T06:03:20.628079+0000","last_peered":"2024-09-19T06:03:20.628079+0000","last_clean":"2024-09-19T06:03:20.628079+0000","last_became_active":"2024-09-19T06:01:26.783887+0000","last_became_peered":"2024-09-19T06:01:26.783887+0000","last_unstale":"2024-09-19T06:03:20.628079+0000","last_undegraded":"2024-09-19T06:03:20.628079+0000","last_fullsized":"2024-09-19T06:03:20.628079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:33:38.255169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626764+0000","last_change":"2024-09-19T06:01:39.837354+0000","last_active":"2024-09-19T06:03:20.626764+0000","last_peered":"2024-09-19T06:03:20.626764+0000","last_clean":"2024-09-19T06:03:20.626764+0000","last_became_active":"2024-09-19T06:01:39.836930+0000","last_became_peered":"2024-09-19T06:01:39.836930+0000","last_unstale":"2024-09-19T06:03:20.626764+0000","last_undegraded":"2024-09-19T06:03:20.626764+0000","last_fullsized":"2024-09-19T06:03:20.626764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:17.571834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":131,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.626969+0000","last_change":"2024-09-19T06:01:27.021753+0000","last_active":"2024-09-19T06:03:20.626969+0000","last_peered":"2024-09-19T06:03:20.626969+0000","last_clean":"2024-09-19T06:03:20.626969+0000","last_became_active":"2024-09-19T06:01:27.021346+0000","last_became_peered":"2024-09-19T06:01:27.021346+0000","last_unstale":"2024-09-19T06:03:20.626969+0000","last_undegraded":"2024-09-19T06:03:20.626969+0000","last_fullsized":"2024-09-19T06:03:20.626969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:07.291283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627909+0000","last_change":"2024-09-19T06:01:28.803103+0000","last_active":"2024-09-19T06:03:20.627909+0000","last_peered":"2024-09-19T06:03:20.627909+0000","last_clean":"2024-09-19T06:03:20.627909+0000","last_became_active":"2024-09-19T06:01:28.802806+0000","last_became_peered":"2024-09-19T06:01:28.802806+0000","last_unstale":"2024-09-19T06:03:20.627909+0000","last_undegraded":"2024-09-19T06:03:20.627909+0000","last_fullsized":"2024-09-19T06:03:20.627909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:33.329231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'8","reported_seq":136,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628270+0000","last_change":"2024-09-19T06:01:26.786605+0000","last_active":"2024-09-19T06:03:20.628270+0000","last_peered":"2024-09-19T06:03:20.628270+0000","last_clean":"2024-09-19T06:03:20.628270+0000","last_became_active":"2024-09-19T06:01:26.786278+0000","last_became_peered":"2024-09-19T06:01:26.786278+0000","last_unstale":"2024-09-19T06:03:20.628270+0000","last_undegraded":"2024-09-19T06:03:20.628270+0000","last_fullsized":"2024-09-19T06:03:20.628270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.581502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732180+0000","last_change":"2024-09-19T06:01:28.808152+0000","last_active":"2024-09-19T06:03:20.732180+0000","last_peered":"2024-09-19T06:03:20.732180+0000","last_clean":"2024-09-19T06:03:20.732180+0000","last_became_active":"2024-09-19T06:01:28.807788+0000","last_became_peered":"2024-09-19T06:01:28.807788+0000","last_unstale":"2024-09-19T06:03:20.732180+0000","last_undegraded":"2024-09-19T06:03:20.732180+0000","last_fullsized":"2024-09-19T06:03:20.732180+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:42.877539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.720863+0000","last_change":"2024-09-19T06:01:27.022719+0000","last_active":"2024-09-19T06:03:20.720863+0000","last_peered":"2024-09-19T06:03:20.720863+0000","last_clean":"2024-09-19T06:03:20.720863+0000","last_became_active":"2024-09-19T06:01:27.022263+0000","last_became_peered":"2024-09-19T06:01:27.022263+0000","last_unstale":"2024-09-19T06:03:20.720863+0000","last_undegraded":"2024-09-19T06:03:20.720863+0000","last_fullsized":"2024-09-19T06:03:20.720863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:45.158229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.732175+0000","last_change":"2024-09-19T06:01:28.809869+0000","last_active":"2024-09-19T06:03:20.732175+0000","last_peered":"2024-09-19T06:03:20.732175+0000","last_clean":"2024-09-19T06:03:20.732175+0000","last_became_active":"2024-09-19T06:01:28.809548+0000","last_became_peered":"2024-09-19T06:01:28.809548+0000","last_unstale":"2024-09-19T06:03:20.732175+0000","last_undegraded":"2024-09-19T06:03:20.732175+0000","last_fullsized":"2024-09-19T06:03:20.732175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:41.587846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887443+0000","last_change":"2024-09-19T06:01:39.846740+0000","last_active":"2024-09-19T06:03:20.887443+0000","last_peered":"2024-09-19T06:03:20.887443+0000","last_clean":"2024-09-19T06:03:20.887443+0000","last_became_active":"2024-09-19T06:01:39.836378+0000","last_became_peered":"2024-09-19T06:01:39.836378+0000","last_unstale":"2024-09-19T06:03:20.887443+0000","last_undegraded":"2024-09-19T06:03:20.887443+0000","last_fullsized":"2024-09-19T06:03:20.887443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.064918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.627124+0000","last_change":"2024-09-19T06:01:28.807796+0000","last_active":"2024-09-19T06:03:20.627124+0000","last_peered":"2024-09-19T06:03:20.627124+0000","last_clean":"2024-09-19T06:03:20.627124+0000","last_became_active":"2024-09-19T06:01:28.807503+0000","last_became_peered":"2024-09-19T06:01:28.807503+0000","last_unstale":"2024-09-19T06:03:20.627124+0000","last_undegraded":"2024-09-19T06:03:20.627124+0000","last_fullsized":"2024-09-19T06:03:20.627124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:43.472647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'51","reported_seq":98,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.853902+0000","last_change":"2024-09-19T06:01:26.783517+0000","last_active":"2024-09-19T06:03:20.853902+0000","last_peered":"2024-09-19T06:03:20.853902+0000","last_clean":"2024-09-19T06:03:20.853902+0000","last_became_active":"2024-09-19T06:01:26.783164+0000","last_became_peered":"2024-09-19T06:01:26.783164+0000","last_unstale":"2024-09-19T06:03:20.853902+0000","last_undegraded":"2024-09-19T06:03:20.853902+0000","last_fullsized":"2024-09-19T06:03:20.853902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:57.609192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.724088+0000","last_change":"2024-09-19T06:01:28.806548+0000","last_active":"2024-09-19T06:03:20.724088+0000","last_peered":"2024-09-19T06:03:20.724088+0000","last_clean":"2024-09-19T06:03:20.724088+0000","last_became_active":"2024-09-19T06:01:28.805724+0000","last_became_peered":"2024-09-19T06:01:28.805724+0000","last_unstale":"2024-09-19T06:03:20.724088+0000","last_undegraded":"2024-09-19T06:03:20.724088+0000","last_fullsized":"2024-09-19T06:03:20.724088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:14.360928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.893742+0000","last_change":"2024-09-19T06:01:27.128213+0000","last_active":"2024-09-19T06:03:20.893742+0000","last_peered":"2024-09-19T06:03:20.893742+0000","last_clean":"2024-09-19T06:03:20.893742+0000","last_became_active":"2024-09-19T06:01:27.127737+0000","last_became_peered":"2024-09-19T06:01:27.127737+0000","last_unstale":"2024-09-19T06:03:20.893742+0000","last_undegraded":"2024-09-19T06:03:20.893742+0000","last_fullsized":"2024-09-19T06:03:20.893742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:33:35.107152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.888504+0000","last_change":"2024-09-19T06:01:39.839027+0000","last_active":"2024-09-19T06:03:20.888504+0000","last_peered":"2024-09-19T06:03:20.888504+0000","last_clean":"2024-09-19T06:03:20.888504+0000","last_became_active":"2024-09-19T06:01:39.838601+0000","last_became_peered":"2024-09-19T06:01:39.838601+0000","last_unstale":"2024-09-19T06:03:20.888504+0000","last_undegraded":"2024-09-19T06:03:20.888504+0000","last_fullsized":"2024-09-19T06:03:20.888504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:09.864997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.885207+0000","last_change":"2024-09-19T06:01:27.022438+0000","last_active":"2024-09-19T06:03:20.885207+0000","last_peered":"2024-09-19T06:03:20.885207+0000","last_clean":"2024-09-19T06:03:20.885207+0000","last_became_active":"2024-09-19T06:01:27.021953+0000","last_became_peered":"2024-09-19T06:01:27.021953+0000","last_unstale":"2024-09-19T06:03:20.885207+0000","last_undegraded":"2024-09-19T06:03:20.885207+0000","last_fullsized":"2024-09-19T06:03:20.885207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:24.383178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722756+0000","last_change":"2024-09-19T06:01:28.803807+0000","last_active":"2024-09-19T06:03:20.722756+0000","last_peered":"2024-09-19T06:03:20.722756+0000","last_clean":"2024-09-19T06:03:20.722756+0000","last_became_active":"2024-09-19T06:01:28.803486+0000","last_became_peered":"2024-09-19T06:01:28.803486+0000","last_unstale":"2024-09-19T06:03:20.722756+0000","last_undegraded":"2024-09-19T06:03:20.722756+0000","last_fullsized":"2024-09-19T06:03:20.722756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:26:17.920089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723040+0000","last_change":"2024-09-19T06:01:26.777171+0000","last_active":"2024-09-19T06:03:20.723040+0000","last_peered":"2024-09-19T06:03:20.723040+0000","last_clean":"2024-09-19T06:03:20.723040+0000","last_became_active":"2024-09-19T06:01:26.776818+0000","last_became_peered":"2024-09-19T06:01:26.776818+0000","last_unstale":"2024-09-19T06:03:20.723040+0000","last_undegraded":"2024-09-19T06:03:20.723040+0000","last_fullsized":"2024-09-19T06:03:20.723040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887551+0000","last_change":"2024-09-19T06:01:28.799266+0000","last_active":"2024-09-19T06:03:20.887551+0000","last_peered":"2024-09-19T06:03:20.887551+0000","last_clean":"2024-09-19T06:03:20.887551+0000","last_became_active":"2024-09-19T06:01:28.798969+0000","last_became_peered":"2024-09-19T06:01:28.798969+0000","last_unstale":"2024-09-19T06:03:20.887551+0000","last_undegraded":"2024-09-19T06:03:20.887551+0000","last_fullsized":"2024-09-19T06:03:20.887551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:34.064795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.896088+0000","last_change":"2024-09-19T06:01:26.781680+0000","last_active":"2024-09-19T06:03:20.896088+0000","last_peered":"2024-09-19T06:03:20.896088+0000","last_clean":"2024-09-19T06:03:20.896088+0000","last_became_active":"2024-09-19T06:01:26.781035+0000","last_became_peered":"2024-09-19T06:01:26.781035+0000","last_unstale":"2024-09-19T06:03:20.896088+0000","last_undegraded":"2024-09-19T06:03:20.896088+0000","last_fullsized":"2024-09-19T06:03:20.896088+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:38.287018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723555+0000","last_change":"2024-09-19T06:01:28.802970+0000","last_active":"2024-09-19T06:03:20.723555+0000","last_peered":"2024-09-19T06:03:20.723555+0000","last_clean":"2024-09-19T06:03:20.723555+0000","last_became_active":"2024-09-19T06:01:28.802587+0000","last_became_peered":"2024-09-19T06:01:28.802587+0000","last_unstale":"2024-09-19T06:03:20.723555+0000","last_undegraded":"2024-09-19T06:03:20.723555+0000","last_fullsized":"2024-09-19T06:03:20.723555+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.722866+0000","last_change":"2024-09-19T06:01:26.788140+0000","last_active":"2024-09-19T06:03:20.722866+0000","last_peered":"2024-09-19T06:03:20.722866+0000","last_clean":"2024-09-19T06:03:20.722866+0000","last_became_active":"2024-09-19T06:01:26.787603+0000","last_became_peered":"2024-09-19T06:01:26.787603+0000","last_unstale":"2024-09-19T06:03:20.722866+0000","last_undegraded":"2024-09-19T06:03:20.722866+0000","last_fullsized":"2024-09-19T06:03:20.722866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:13.880656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628239+0000","last_change":"2024-09-19T06:01:28.813144+0000","last_active":"2024-09-19T06:03:20.628239+0000","last_peered":"2024-09-19T06:03:20.628239+0000","last_clean":"2024-09-19T06:03:20.628239+0000","last_became_active":"2024-09-19T06:01:28.812792+0000","last_became_peered":"2024-09-19T06:01:28.812792+0000","last_unstale":"2024-09-19T06:03:20.628239+0000","last_undegraded":"2024-09-19T06:03:20.628239+0000","last_fullsized":"2024-09-19T06:03:20.628239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:35.896240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.628078+0000","last_change":"2024-09-19T06:01:28.812086+0000","last_active":"2024-09-19T06:03:20.628078+0000","last_peered":"2024-09-19T06:03:20.628078+0000","last_clean":"2024-09-19T06:03:20.628078+0000","last_became_active":"2024-09-19T06:01:28.811696+0000","last_became_peered":"2024-09-19T06:01:28.811696+0000","last_unstale":"2024-09-19T06:03:20.628078+0000","last_undegraded":"2024-09-19T06:03:20.628078+0000","last_fullsized":"2024-09-19T06:03:20.628078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:32.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.887254+0000","last_change":"2024-09-19T06:01:39.842250+0000","last_active":"2024-09-19T06:03:20.887254+0000","last_peered":"2024-09-19T06:03:20.887254+0000","last_clean":"2024-09-19T06:03:20.887254+0000","last_became_active":"2024-09-19T06:01:39.835761+0000","last_became_peered":"2024-09-19T06:01:39.835761+0000","last_unstale":"2024-09-19T06:03:20.887254+0000","last_undegraded":"2024-09-19T06:03:20.887254+0000","last_fullsized":"2024-09-19T06:03:20.887254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:51.026349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.884914+0000","last_change":"2024-09-19T06:01:28.812983+0000","last_active":"2024-09-19T06:03:20.884914+0000","last_peered":"2024-09-19T06:03:20.884914+0000","last_clean":"2024-09-19T06:03:20.884914+0000","last_became_active":"2024-09-19T06:01:28.812257+0000","last_became_peered":"2024-09-19T06:01:28.812257+0000","last_unstale":"2024-09-19T06:03:20.884914+0000","last_undegraded":"2024-09-19T06:03:20.884914+0000","last_fullsized":"2024-09-19T06:03:20.884914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:41:42.059736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723024+0000","last_change":"2024-09-19T06:01:26.787869+0000","last_active":"2024-09-19T06:03:20.723024+0000","last_peered":"2024-09-19T06:03:20.723024+0000","last_clean":"2024-09-19T06:03:20.723024+0000","last_became_active":"2024-09-19T06:01:26.787538+0000","last_became_peered":"2024-09-19T06:01:26.787538+0000","last_unstale":"2024-09-19T06:03:20.723024+0000","last_undegraded":"2024-09-19T06:03:20.723024+0000","last_fullsized":"2024-09-19T06:03:20.723024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:02.869878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.889703+0000","last_change":"2024-09-19T06:01:28.811986+0000","last_active":"2024-09-19T06:03:20.889703+0000","last_peered":"2024-09-19T06:03:20.889703+0000","last_clean":"2024-09-19T06:03:20.889703+0000","last_became_active":"2024-09-19T06:01:28.811565+0000","last_became_peered":"2024-09-19T06:01:28.811565+0000","last_unstale":"2024-09-19T06:03:20.889703+0000","last_undegraded":"2024-09-19T06:03:20.889703+0000","last_fullsized":"2024-09-19T06:03:20.889703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:01.324985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723946+0000","last_change":"2024-09-19T06:01:27.021683+0000","last_active":"2024-09-19T06:03:20.723946+0000","last_peered":"2024-09-19T06:03:20.723946+0000","last_clean":"2024-09-19T06:03:20.723946+0000","last_became_active":"2024-09-19T06:01:27.021358+0000","last_became_peered":"2024-09-19T06:01:27.021358+0000","last_unstale":"2024-09-19T06:03:20.723946+0000","last_undegraded":"2024-09-19T06:03:20.723946+0000","last_fullsized":"2024-09-19T06:03:20.723946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:29.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.723619+0000","last_change":"2024-09-19T06:01:28.798963+0000","last_active":"2024-09-19T06:03:20.723619+0000","last_peered":"2024-09-19T06:03:20.723619+0000","last_clean":"2024-09-19T06:03:20.723619+0000","last_became_active":"2024-09-19T06:01:28.797571+0000","last_became_peered":"2024-09-19T06:01:28.797571+0000","last_unstale":"2024-09-19T06:03:20.723619+0000","last_undegraded":"2024-09-19T06:03:20.723619+0000","last_fullsized":"2024-09-19T06:03:20.723619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:27.777396+0000","last_clean_scrub_stamp":"2024-09-19T06:01:27.777396+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:43.867609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:03:20.721283+0000","last_change":"2024-09-19T06:01:26.786427+0000","last_active":"2024-09-19T06:03:20.721283+0000","last_peered":"2024-09-19T06:03:20.721283+0000","last_clean":"2024-09-19T06:03:20.721283+0000","last_became_active":"2024-09-19T06:01:26.786092+0000","last_became_peered":"2024-09-19T06:01:26.786092+0000","last_unstale":"2024-09-19T06:03:20.721283+0000","last_undegraded":"2024-09-19T06:03:20.721283+0000","last_fullsized":"2024-09-19T06:03:20.721283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:01:25.763217+0000","last_clean_scrub_stamp":"2024-09-19T06:01:25.763217+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:49.144366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603942,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1149,"num_read_kb":694,"num_write":387,"num_write_kb":1474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1811826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":449661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41236,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703916,"statfs":{"total":95995035648,"available":95952809984,"internally_reserved":0,"allocated":1773568,"data_stored":1238219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939694,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":438118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36648,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708504,"statfs":{"total":95995035648,"available":95957508096,"internally_reserved":0,"allocated":1331200,"data_stored":839982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397559,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1548288,"data_stored":1043460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":447828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1695744,"data_stored":1172287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":450126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1482752,"data_stored":971440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40496,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704656,"statfs":{"total":95995035648,"available":95953567744,"internally_reserved":0,"allocated":1011712,"data_stored":513156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40500,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704652,"statfs":{"total":95995035648,"available":95953563648,"internally_reserved":0,"allocated":1015808,"data_stored":518742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":440203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":201774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:03:25.869 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:03:26.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:03:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T06:03:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:26.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:25 smithi157 ceph-mon[35346]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:03:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:03:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T06:03:26.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:26.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:26.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:26.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:26.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:25 smithi049 ceph-mon[34640]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:03:26.157 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.628+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3775577093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe81055b0 msgr2=0x7f1fe810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.628+0000 7f1fefbd3640 1 --2- 172.21.15.5:0/3775577093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe81055b0 0x7f1fe810d130 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009ca0 tx=0x7f1fe002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:03:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.628+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3775577093 shutdown_connections 2024-09-19T06:03:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.628+0000 7f1fefbd3640 1 --2- 172.21.15.5:0/3775577093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe81055b0 0x7f1fe810d130 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.628+0000 7f1fefbd3640 1 --2- 172.21.15.5:0/3775577093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe8100e30 0x7f1fe8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.628+0000 7f1fefbd3640 1 --2- 172.21.15.5:0/3775577093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe8100480 0x7f1fe8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.628+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3775577093 >> 172.21.15.5:0/3775577093 conn(0x7f1fe8077210 msgr2=0x7f1fe80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.628+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3775577093 shutdown_connections 2024-09-19T06:03:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.628+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3775577093 wait complete. 2024-09-19T06:03:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fefbd3640 1 Processor -- start 2024-09-19T06:03:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fefbd3640 1 -- start start 2024-09-19T06:03:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fefbd3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe8100480 0x7f1fe819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fefbd3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe8100e30 0x7f1fe819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fefbd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe81055b0 0x7f1fe819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fefbd3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8110490 con 0x7f1fe81055b0 2024-09-19T06:03:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fefbd3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8110310 con 0x7f1fe8100480 2024-09-19T06:03:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fefbd3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8110610 con 0x7f1fe8100e30 2024-09-19T06:03:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fee170640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe81055b0 0x7f1fe819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fee170640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe81055b0 0x7f1fe819ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37960/0 (socket says 172.21.15.5:37960) 2024-09-19T06:03:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fee170640 1 -- 172.21.15.5:0/3209139655 learned_addr learned my addr 172.21.15.5:0/3209139655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fed16e640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe8100e30 0x7f1fe819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.629+0000 7f1fed96f640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe8100480 0x7f1fe819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.630+0000 7f1fee170640 1 -- 172.21.15.5:0/3209139655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe8100e30 msgr2=0x7f1fe819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.630+0000 7f1fee170640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe8100e30 0x7f1fe819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.630+0000 7f1fee170640 1 -- 172.21.15.5:0/3209139655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe8100480 msgr2=0x7f1fe819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.630+0000 7f1fee170640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe8100480 0x7f1fe819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.630+0000 7f1fee170640 1 -- 172.21.15.5:0/3209139655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe819f530 con 0x7f1fe81055b0 2024-09-19T06:03:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.630+0000 7f1fed16e640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe8100e30 0x7f1fe819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.630+0000 7f1fee170640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe81055b0 0x7f1fe819ee50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009dc0 tx=0x7f1fe0032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.630+0000 7f1fdeffd640 1 -- 172.21.15.5:0/3209139655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0003c30 con 0x7f1fe81055b0 2024-09-19T06:03:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.630+0000 7f1fdeffd640 1 -- 172.21.15.5:0/3209139655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fe0003dd0 con 0x7f1fe81055b0 2024-09-19T06:03:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.630+0000 7f1fdeffd640 1 -- 172.21.15.5:0/3209139655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0030aa0 con 0x7f1fe81055b0 2024-09-19T06:03:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.630+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3209139655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe819f7c0 con 0x7f1fe81055b0 2024-09-19T06:03:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.630+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3209139655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe81a7100 con 0x7f1fe81055b0 2024-09-19T06:03:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.631+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3209139655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe8100860 con 0x7f1fe81055b0 2024-09-19T06:03:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.632+0000 7f1fdeffd640 1 -- 172.21.15.5:0/3209139655 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7f1fe0030ca0 con 0x7f1fe81055b0 2024-09-19T06:03:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.633+0000 7f1fdeffd640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1fd40a34c0 0x7f1fd40a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.633+0000 7f1fdeffd640 1 -- 172.21.15.5:0/3209139655 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f1fe00f1340 con 0x7f1fe81055b0 2024-09-19T06:03:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.633+0000 7f1fed96f640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1fd40a34c0 0x7f1fd40a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.634+0000 7f1fed96f640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1fd40a34c0 0x7f1fd40a5980 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009bd0 tx=0x7f1fe4005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.636+0000 7f1fdeffd640 1 -- 172.21.15.5:0/3209139655 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1fe00bdd00 con 0x7f1fe81055b0 2024-09-19T06:03:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.851+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3209139655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fe819b8e0 con 0x7f1fe81055b0 2024-09-19T06:03:26.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:26 smithi005 ceph-mon[30955]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:03:26.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:26 smithi005 ceph-mon[30955]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:03:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.851+0000 7f1fdeffd640 1 -- 172.21.15.5:0/3209139655 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11959 (secure 0 0 0) 0x7f1fe00dd020 con 0x7f1fe81055b0 2024-09-19T06:03:26.853 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:03:26.854 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-19T06:03:21:625357+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:03:20.863188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:03:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-19T06:03:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.853+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3209139655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1fd40a34c0 msgr2=0x7f1fd40a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.853+0000 7f1fefbd3640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1fd40a34c0 0x7f1fd40a5980 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009bd0 tx=0x7f1fe4005f90 comp rx=0 tx=0).stop 2024-09-19T06:03:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.853+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3209139655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe81055b0 msgr2=0x7f1fe819ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.853+0000 7f1fefbd3640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe81055b0 0x7f1fe819ee50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009dc0 tx=0x7f1fe0032ff0 comp rx=0 tx=0).stop 2024-09-19T06:03:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.853+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3209139655 shutdown_connections 2024-09-19T06:03:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.853+0000 7f1fefbd3640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe81055b0 0x7f1fe819ee50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.853+0000 7f1fefbd3640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1fd40a34c0 0x7f1fd40a5980 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.853+0000 7f1fefbd3640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe8100e30 0x7f1fe819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.853+0000 7f1fefbd3640 1 --2- 172.21.15.5:0/3209139655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe8100480 0x7f1fe819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.853+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3209139655 >> 172.21.15.5:0/3209139655 conn(0x7f1fe8077210 msgr2=0x7f1fe810bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.854+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3209139655 shutdown_connections 2024-09-19T06:03:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:26.854+0000 7f1fefbd3640 1 -- 172.21.15.5:0/3209139655 wait complete. 2024-09-19T06:03:27.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:26 smithi157 ceph-mon[35346]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:03:27.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:26 smithi157 ceph-mon[35346]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:03:27.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:26 smithi049 ceph-mon[34640]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:03:27.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:26 smithi049 ceph-mon[34640]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:03:27.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:03:27.859 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:27 smithi005 ceph-mon[30955]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:03:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:27 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3209139655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:03:28.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:27 smithi157 ceph-mon[35346]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:03:28.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:27 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3209139655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:03:28.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:27 smithi049 ceph-mon[34640]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:03:28.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:27 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3209139655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:03:28.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.318+0000 7f729d32a640 1 -- 172.21.15.5:0/1571147660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298100a80 msgr2=0x7f7298100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.318+0000 7f729d32a640 1 --2- 172.21.15.5:0/1571147660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298100a80 0x7f7298100ee0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7280009930 tx=0x7f728002a420 comp rx=0 tx=0).stop 2024-09-19T06:03:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.319+0000 7f729d32a640 1 -- 172.21.15.5:0/1571147660 shutdown_connections 2024-09-19T06:03:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.319+0000 7f729d32a640 1 --2- 172.21.15.5:0/1571147660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7298101420 0x7f729810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.319+0000 7f729d32a640 1 --2- 172.21.15.5:0/1571147660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298100a80 0x7f7298100ee0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.319+0000 7f729d32a640 1 --2- 172.21.15.5:0/1571147660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7298106a80 0x7f7298106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.319+0000 7f729d32a640 1 -- 172.21.15.5:0/1571147660 >> 172.21.15.5:0/1571147660 conn(0x7f72980fc7a0 msgr2=0x7f72980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.319+0000 7f729d32a640 1 -- 172.21.15.5:0/1571147660 shutdown_connections 2024-09-19T06:03:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.319+0000 7f729d32a640 1 -- 172.21.15.5:0/1571147660 wait complete. 2024-09-19T06:03:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.319+0000 7f729d32a640 1 Processor -- start 2024-09-19T06:03:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.319+0000 7f729d32a640 1 -- start start 2024-09-19T06:03:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f729d32a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7298100a80 0x7f72981983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f729d32a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7298101420 0x7f7298198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f729d32a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298106a80 0x7f729819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f729d32a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7298110d30 con 0x7f7298106a80 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f729d32a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7298110bb0 con 0x7f7298100a80 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f729d32a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7298110eb0 con 0x7f7298101420 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f72977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298106a80 0x7f729819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f72977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298106a80 0x7f729819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37972/0 (socket says 172.21.15.5:37972) 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f72977fe640 1 -- 172.21.15.5:0/3255999008 learned_addr learned my addr 172.21.15.5:0/3255999008 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f7296ffd640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7298100a80 0x7f72981983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f72967fc640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7298101420 0x7f7298198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f72977fe640 1 -- 172.21.15.5:0/3255999008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7298101420 msgr2=0x7f7298198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f72977fe640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7298101420 0x7f7298198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f72977fe640 1 -- 172.21.15.5:0/3255999008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7298100a80 msgr2=0x7f72981983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f72977fe640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7298100a80 0x7f72981983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f72977fe640 1 -- 172.21.15.5:0/3255999008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729819d380 con 0x7f7298106a80 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f72967fc640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7298101420 0x7f7298198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f72977fe640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298106a80 0x7f729819cca0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f729000eb10 tx=0x7f729000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.320+0000 7f728ffff640 1 -- 172.21.15.5:0/3255999008 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290004b20 con 0x7f7298106a80 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.321+0000 7f728ffff640 1 -- 172.21.15.5:0/3255999008 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7290004cc0 con 0x7f7298106a80 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.321+0000 7f729d32a640 1 -- 172.21.15.5:0/3255999008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729819d670 con 0x7f7298106a80 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.321+0000 7f728ffff640 1 -- 172.21.15.5:0/3255999008 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290010690 con 0x7f7298106a80 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.321+0000 7f729d32a640 1 -- 172.21.15.5:0/3255999008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72981a4f50 con 0x7f7298106a80 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.322+0000 7f729d32a640 1 -- 172.21.15.5:0/3255999008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7298100ee0 con 0x7f7298106a80 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.323+0000 7f728ffff640 1 -- 172.21.15.5:0/3255999008 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7f7290005020 con 0x7f7298106a80 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.324+0000 7f728ffff640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f72840a3440 0x7f72840a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.324+0000 7f7296ffd640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f72840a3440 0x7f72840a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.324+0000 7f728ffff640 1 -- 172.21.15.5:0/3255999008 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f72900d04d0 con 0x7f7298106a80 2024-09-19T06:03:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.324+0000 7f7296ffd640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f72840a3440 0x7f72840a5900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f72880058c0 tx=0x7f728800a250 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.328+0000 7f728ffff640 1 -- 172.21.15.5:0/3255999008 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f729009cd50 con 0x7f7298106a80 2024-09-19T06:03:28.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.505+0000 7f729d32a640 1 -- 172.21.15.5:0/3255999008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7298109a70 con 0x7f7298106a80 2024-09-19T06:03:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.508+0000 7f728ffff640 1 -- 172.21.15.5:0/3255999008 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+23072 (secure 0 0 0) 0x7f729009c650 con 0x7f7298106a80 2024-09-19T06:03:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:03:28.510 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":84,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","created":"2024-09-19T05:53:51.454068+0000","modified":"2024-09-19T06:03:20.622077+0000","last_up_change":"2024-09-19T06:00:38.837916+0000","last_in_change":"2024-09-19T06:00:15.609776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:56:55.213931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:01:25.648588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:01:27.760408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f3c10d60-d354-407f-9cdf-099344599844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6803","nonce":3442981810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6805","nonce":3442981810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6809","nonce":3442981810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6807","nonce":3442981810}]},"public_addr":"172.21.15.5:6803/3442981810","cluster_addr":"172.21.15.5:6805/3442981810","heartbeat_back_addr":"172.21.15.5:6809/3442981810","heartbeat_front_addr":"172.21.15.5:6807/3442981810","state":["exists","up"]},{"osd":1,"uuid":"5f40f636-7761-45ac-87f3-910a2fa2feb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448265014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6803","nonce":1448265014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6807","nonce":1448265014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6805","nonce":1448265014}]},"public_addr":"172.21.15.49:6801/1448265014","cluster_addr":"172.21.15.49:6803/1448265014","heartbeat_back_addr":"172.21.15.49:6807/1448265014","heartbeat_front_addr":"172.21.15.49:6805/1448265014","state":["exists","up"]},{"osd":2,"uuid":"ad00940c-4fdd-4f5f-9eaa-b1f38bcac526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6801","nonce":2319626622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6803","nonce":2319626622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6807","nonce":2319626622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6805","nonce":2319626622}]},"public_addr":"172.21.15.157:6801/2319626622","cluster_addr":"172.21.15.157:6803/2319626622","heartbeat_back_addr":"172.21.15.157:6807/2319626622","heartbeat_front_addr":"172.21.15.157:6805/2319626622","state":["exists","up"]},{"osd":3,"uuid":"63fec329-1b62-4c5e-b8ca-084a84d4d762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6811","nonce":1452491753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6813","nonce":1452491753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6817","nonce":1452491753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6815","nonce":1452491753}]},"public_addr":"172.21.15.5:6811/1452491753","cluster_addr":"172.21.15.5:6813/1452491753","heartbeat_back_addr":"172.21.15.5:6817/1452491753","heartbeat_front_addr":"172.21.15.5:6815/1452491753","state":["exists","up"]},{"osd":4,"uuid":"ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6809","nonce":3114952580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6811","nonce":3114952580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6815","nonce":3114952580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6813","nonce":3114952580}]},"public_addr":"172.21.15.49:6809/3114952580","cluster_addr":"172.21.15.49:6811/3114952580","heartbeat_back_addr":"172.21.15.49:6815/3114952580","heartbeat_front_addr":"172.21.15.49:6813/3114952580","state":["exists","up"]},{"osd":5,"uuid":"21a7a7d3-ce42-4688-bfec-714be635eaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6809","nonce":3219063971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6811","nonce":3219063971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6815","nonce":3219063971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6813","nonce":3219063971}]},"public_addr":"172.21.15.157:6809/3219063971","cluster_addr":"172.21.15.157:6811/3219063971","heartbeat_back_addr":"172.21.15.157:6815/3219063971","heartbeat_front_addr":"172.21.15.157:6813/3219063971","state":["exists","up"]},{"osd":6,"uuid":"cbef0e08-d6b0-4515-a2ee-2c81b048583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6819","nonce":2957795194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6821","nonce":2957795194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6825","nonce":2957795194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6823","nonce":2957795194}]},"public_addr":"172.21.15.5:6819/2957795194","cluster_addr":"172.21.15.5:6821/2957795194","heartbeat_back_addr":"172.21.15.5:6825/2957795194","heartbeat_front_addr":"172.21.15.5:6823/2957795194","state":["exists","up"]},{"osd":7,"uuid":"d261ac55-532d-4aea-9b3e-ed2b4eeb1da6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6817","nonce":2959731400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6819","nonce":2959731400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6823","nonce":2959731400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6821","nonce":2959731400}]},"public_addr":"172.21.15.49:6817/2959731400","cluster_addr":"172.21.15.49:6819/2959731400","heartbeat_back_addr":"172.21.15.49:6823/2959731400","heartbeat_front_addr":"172.21.15.49:6821/2959731400","state":["exists","up"]},{"osd":8,"uuid":"15d0eaf7-d604-4546-b393-4d6d39018f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6817","nonce":290026859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6819","nonce":290026859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6823","nonce":290026859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6821","nonce":290026859}]},"public_addr":"172.21.15.157:6817/290026859","cluster_addr":"172.21.15.157:6819/290026859","heartbeat_back_addr":"172.21.15.157:6823/290026859","heartbeat_front_addr":"172.21.15.157:6821/290026859","state":["exists","up"]},{"osd":9,"uuid":"30c08537-f593-4299-bfa1-fa7ba8a95169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6827","nonce":4122448237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6829","nonce":4122448237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6833","nonce":4122448237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6831","nonce":4122448237}]},"public_addr":"172.21.15.5:6827/4122448237","cluster_addr":"172.21.15.5:6829/4122448237","heartbeat_back_addr":"172.21.15.5:6833/4122448237","heartbeat_front_addr":"172.21.15.5:6831/4122448237","state":["exists","up"]},{"osd":10,"uuid":"68821e98-bd81-49ec-8cd6-51fb9880b553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6825","nonce":2551748002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6827","nonce":2551748002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6831","nonce":2551748002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6829","nonce":2551748002}]},"public_addr":"172.21.15.49:6825/2551748002","cluster_addr":"172.21.15.49:6827/2551748002","heartbeat_back_addr":"172.21.15.49:6831/2551748002","heartbeat_front_addr":"172.21.15.49:6829/2551748002","state":["exists","up"]},{"osd":11,"uuid":"7e443b46-2697-4d0b-8452-868463a1ee1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6825","nonce":1996582544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6827","nonce":1996582544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6831","nonce":1996582544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6829","nonce":1996582544}]},"public_addr":"172.21.15.157:6825/1996582544","cluster_addr":"172.21.15.157:6827/1996582544","heartbeat_back_addr":"172.21.15.157:6831/1996582544","heartbeat_front_addr":"172.21.15.157:6829/1996582544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:01.453592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:27.194705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:51.525385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:17.676749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:43.075555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:07.258154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:32.112583+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:57.306534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:21.320950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:47.164610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:12.548337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:37.262580+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3703679969":"2024-09-20T06:03:20.622034+0000","172.21.15.5:6801/709970884":"2024-09-20T06:03:20.622034+0000","172.21.15.5:0/1367598148":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2930274429":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6800/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2197596976":"2024-09-20T06:03:20.622034+0000","172.21.15.5:6801/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/1335130331":"2024-09-20T06:03:20.622034+0000","172.21.15.5:6800/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/1639646508":"2024-09-20T06:03:20.622034+0000","172.21.15.5:0/1340147787":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/850678758":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6801/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/2593868621":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/4102275559":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6800/709970884":"2024-09-20T06:03:20.622034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:03:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.510+0000 7f729d32a640 1 -- 172.21.15.5:0/3255999008 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f72840a3440 msgr2=0x7f72840a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.510+0000 7f729d32a640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f72840a3440 0x7f72840a5900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f72880058c0 tx=0x7f728800a250 comp rx=0 tx=0).stop 2024-09-19T06:03:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.510+0000 7f729d32a640 1 -- 172.21.15.5:0/3255999008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298106a80 msgr2=0x7f729819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.510+0000 7f729d32a640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298106a80 0x7f729819cca0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f729000eb10 tx=0x7f729000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:03:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.510+0000 7f729d32a640 1 -- 172.21.15.5:0/3255999008 shutdown_connections 2024-09-19T06:03:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.510+0000 7f729d32a640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298106a80 0x7f729819cca0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.510+0000 7f729d32a640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f72840a3440 0x7f72840a5900 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.510+0000 7f729d32a640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7298101420 0x7f7298198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.510+0000 7f729d32a640 1 --2- 172.21.15.5:0/3255999008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7298100a80 0x7f72981983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.510+0000 7f729d32a640 1 -- 172.21.15.5:0/3255999008 >> 172.21.15.5:0/3255999008 conn(0x7f72980fc7a0 msgr2=0x7f729806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.510+0000 7f729d32a640 1 -- 172.21.15.5:0/3255999008 shutdown_connections 2024-09-19T06:03:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:28.510+0000 7f729d32a640 1 -- 172.21.15.5:0/3255999008 wait complete. 2024-09-19T06:03:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:28 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3255999008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:03:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:28 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3255999008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:03:29.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:28 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3255999008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:03:29.269 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T06:03:29.269 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T06:03:29.280 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T06:03:29.280 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-19T06:03:29.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-19T06:03:29.281 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T06:03:29.281 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:03:29.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:03:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.394+0000 7f424101e640 1 -- 172.21.15.5:0/2343989034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f423c1069b0 msgr2=0x7f423c106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.394+0000 7f424101e640 1 --2- 172.21.15.5:0/2343989034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f423c1069b0 0x7f423c106d90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f422c009930 tx=0x7f422c02a420 comp rx=0 tx=0).stop 2024-09-19T06:03:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.394+0000 7f424101e640 1 -- 172.21.15.5:0/2343989034 shutdown_connections 2024-09-19T06:03:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.394+0000 7f424101e640 1 --2- 172.21.15.5:0/2343989034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c101340 0x7f423c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.394+0000 7f424101e640 1 --2- 172.21.15.5:0/2343989034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f423c1009a0 0x7f423c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.394+0000 7f424101e640 1 --2- 172.21.15.5:0/2343989034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f423c1069b0 0x7f423c106d90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.394+0000 7f424101e640 1 -- 172.21.15.5:0/2343989034 >> 172.21.15.5:0/2343989034 conn(0x7f423c0779a0 msgr2=0x7f423c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.394+0000 7f424101e640 1 -- 172.21.15.5:0/2343989034 shutdown_connections 2024-09-19T06:03:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.394+0000 7f424101e640 1 -- 172.21.15.5:0/2343989034 wait complete. 2024-09-19T06:03:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f424101e640 1 Processor -- start 2024-09-19T06:03:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f424101e640 1 -- start start 2024-09-19T06:03:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f424101e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f423c1009a0 0x7f423c102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f424101e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f423c101340 0x7f423c102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f424101e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c1069b0 0x7f423c102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f424101e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f423c110cf0 con 0x7f423c101340 2024-09-19T06:03:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f424101e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f423c110b70 con 0x7f423c1069b0 2024-09-19T06:03:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f424101e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f423c110e70 con 0x7f423c1009a0 2024-09-19T06:03:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f423a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f423c101340 0x7f423c102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f423a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f423c101340 0x7f423c102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37996/0 (socket says 172.21.15.5:37996) 2024-09-19T06:03:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f423a59c640 1 -- 172.21.15.5:0/4178007070 learned_addr learned my addr 172.21.15.5:0/4178007070 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f423ad9d640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f423c1009a0 0x7f423c102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.395+0000 7f423b59e640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c1069b0 0x7f423c102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.396+0000 7f423a59c640 1 -- 172.21.15.5:0/4178007070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f423c1009a0 msgr2=0x7f423c102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.396+0000 7f423a59c640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f423c1009a0 0x7f423c102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.396+0000 7f423a59c640 1 -- 172.21.15.5:0/4178007070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c1069b0 msgr2=0x7f423c102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.396+0000 7f423a59c640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c1069b0 0x7f423c102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.396+0000 7f423a59c640 1 -- 172.21.15.5:0/4178007070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c1a6800 con 0x7f423c101340 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.396+0000 7f423ad9d640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f423c1009a0 0x7f423c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.396+0000 7f423a59c640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f423c101340 0x7f423c102ab0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f422400d950 tx=0x7f422400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.396+0000 7f4233fff640 1 -- 172.21.15.5:0/4178007070 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224017070 con 0x7f423c101340 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.396+0000 7f424101e640 1 -- 172.21.15.5:0/4178007070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c1a6af0 con 0x7f423c101340 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.396+0000 7f4233fff640 1 -- 172.21.15.5:0/4178007070 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4224009c30 con 0x7f423c101340 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.396+0000 7f4233fff640 1 -- 172.21.15.5:0/4178007070 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422400f620 con 0x7f423c101340 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.396+0000 7f424101e640 1 -- 172.21.15.5:0/4178007070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c1a7090 con 0x7f423c101340 2024-09-19T06:03:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.397+0000 7f424101e640 1 -- 172.21.15.5:0/4178007070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f423c1020e0 con 0x7f423c101340 2024-09-19T06:03:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.398+0000 7f4233fff640 1 -- 172.21.15.5:0/4178007070 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7f42240109a0 con 0x7f423c101340 2024-09-19T06:03:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.399+0000 7f4233fff640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42280a3440 0x7f42280a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.399+0000 7f423ad9d640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42280a3440 0x7f42280a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.399+0000 7f4233fff640 1 -- 172.21.15.5:0/4178007070 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f42240cfb90 con 0x7f423c101340 2024-09-19T06:03:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.400+0000 7f423ad9d640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42280a3440 0x7f42280a5900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f422c006010 tx=0x7f422c000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.407+0000 7f4233fff640 1 -- 172.21.15.5:0/4178007070 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4224009de0 con 0x7f423c101340 2024-09-19T06:03:29.560 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:29.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.601+0000 7f424101e640 1 -- 172.21.15.5:0/4178007070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f423c1a7300 con 0x7f423c101340 2024-09-19T06:03:29.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.604+0000 7f4233fff640 1 -- 172.21.15.5:0/4178007070 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7f422409c410 con 0x7f423c101340 2024-09-19T06:03:29.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.606+0000 7f424101e640 1 -- 172.21.15.5:0/4178007070 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42280a3440 msgr2=0x7f42280a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:29.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.606+0000 7f424101e640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42280a3440 0x7f42280a5900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f422c006010 tx=0x7f422c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.606+0000 7f424101e640 1 -- 172.21.15.5:0/4178007070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f423c101340 msgr2=0x7f423c102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:29.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.606+0000 7f424101e640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f423c101340 0x7f423c102ab0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f422400d950 tx=0x7f422400de10 comp rx=0 tx=0).stop 2024-09-19T06:03:29.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.607+0000 7f424101e640 1 -- 172.21.15.5:0/4178007070 shutdown_connections 2024-09-19T06:03:29.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.607+0000 7f424101e640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c1069b0 0x7f423c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.607+0000 7f424101e640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f423c101340 0x7f423c102ab0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.607+0000 7f424101e640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42280a3440 0x7f42280a5900 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.607+0000 7f424101e640 1 --2- 172.21.15.5:0/4178007070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f423c1009a0 0x7f423c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.607+0000 7f424101e640 1 -- 172.21.15.5:0/4178007070 >> 172.21.15.5:0/4178007070 conn(0x7f423c0779a0 msgr2=0x7f423c069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:29.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.607+0000 7f424101e640 1 -- 172.21.15.5:0/4178007070 shutdown_connections 2024-09-19T06:03:29.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.607+0000 7f424101e640 1 -- 172.21.15.5:0/4178007070 wait complete. 2024-09-19T06:03:29.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-19T06:03:29.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.728+0000 7f90f85fd640 1 -- 172.21.15.5:0/945033723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f01069b0 msgr2=0x7f90f0106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:29.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.728+0000 7f90f85fd640 1 --2- 172.21.15.5:0/945033723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f01069b0 0x7f90f0106d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009cd0 tx=0x7f90ec02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.728+0000 7f90f85fd640 1 -- 172.21.15.5:0/945033723 shutdown_connections 2024-09-19T06:03:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.728+0000 7f90f85fd640 1 --2- 172.21.15.5:0/945033723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90f0101340 0x7f90f010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.728+0000 7f90f85fd640 1 --2- 172.21.15.5:0/945033723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90f01009a0 0x7f90f0100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.728+0000 7f90f85fd640 1 --2- 172.21.15.5:0/945033723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f01069b0 0x7f90f0106d90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.728+0000 7f90f85fd640 1 -- 172.21.15.5:0/945033723 >> 172.21.15.5:0/945033723 conn(0x7f90f00779a0 msgr2=0x7f90f00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.728+0000 7f90f85fd640 1 -- 172.21.15.5:0/945033723 shutdown_connections 2024-09-19T06:03:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.728+0000 7f90f85fd640 1 -- 172.21.15.5:0/945033723 wait complete. 2024-09-19T06:03:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.728+0000 7f90f85fd640 1 Processor -- start 2024-09-19T06:03:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f85fd640 1 -- start start 2024-09-19T06:03:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f85fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f01009a0 0x7f90f0199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f85fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90f0101340 0x7f90f0199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f85fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90f01069b0 0x7f90f01a4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f85fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0110ca0 con 0x7f90f01009a0 2024-09-19T06:03:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f85fd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0110b20 con 0x7f90f01069b0 2024-09-19T06:03:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f85fd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0110e20 con 0x7f90f0101340 2024-09-19T06:03:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f6399640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f01009a0 0x7f90f0199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f6399640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f01009a0 0x7f90f0199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38020/0 (socket says 172.21.15.5:38020) 2024-09-19T06:03:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f6399640 1 -- 172.21.15.5:0/2521386918 learned_addr learned my addr 172.21.15.5:0/2521386918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f5b98640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90f0101340 0x7f90f0199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f6b9a640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90f01069b0 0x7f90f01a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f6399640 1 -- 172.21.15.5:0/2521386918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90f0101340 msgr2=0x7f90f0199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f6399640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90f0101340 0x7f90f0199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f6399640 1 -- 172.21.15.5:0/2521386918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90f01069b0 msgr2=0x7f90f01a4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f6399640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90f01069b0 0x7f90f01a4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f6399640 1 -- 172.21.15.5:0/2521386918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f01a46d0 con 0x7f90f01009a0 2024-09-19T06:03:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.729+0000 7f90f6399640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f01009a0 0x7f90f0199110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f90ec02ae00 tx=0x7f90ec003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.730+0000 7f90e77fe640 1 -- 172.21.15.5:0/2521386918 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec004240 con 0x7f90f01009a0 2024-09-19T06:03:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.730+0000 7f90e77fe640 1 -- 172.21.15.5:0/2521386918 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90ec0305a0 con 0x7f90f01009a0 2024-09-19T06:03:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.730+0000 7f90e77fe640 1 -- 172.21.15.5:0/2521386918 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec03b640 con 0x7f90f01009a0 2024-09-19T06:03:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.730+0000 7f90f85fd640 1 -- 172.21.15.5:0/2521386918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f01a48a0 con 0x7f90f01009a0 2024-09-19T06:03:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.730+0000 7f90f85fd640 1 -- 172.21.15.5:0/2521386918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f01a4d60 con 0x7f90f01009a0 2024-09-19T06:03:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.731+0000 7f90f85fd640 1 -- 172.21.15.5:0/2521386918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f01020e0 con 0x7f90f01009a0 2024-09-19T06:03:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.732+0000 7f90e77fe640 1 -- 172.21.15.5:0/2521386918 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7f90ec030ac0 con 0x7f90f01009a0 2024-09-19T06:03:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.732+0000 7f90e77fe640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90dc0a3440 0x7f90dc0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.733+0000 7f90e77fe640 1 -- 172.21.15.5:0/2521386918 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f90ec02e030 con 0x7f90f01009a0 2024-09-19T06:03:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.733+0000 7f90f5b98640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90dc0a3440 0x7f90dc0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.733+0000 7f90f5b98640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90dc0a3440 0x7f90dc0a5900 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f90f006a560 tx=0x7f90e000b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.738+0000 7f90e77fe640 1 -- 172.21.15.5:0/2521386918 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f90ec0bd850 con 0x7f90f01009a0 2024-09-19T06:03:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:29 smithi005 ceph-mon[30955]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 293 B/s wr, 25 op/s 2024-09-19T06:03:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:29 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/4178007070' entity='client.admin' 2024-09-19T06:03:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:29 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:29 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:29 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:29 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:29.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.944+0000 7f90f85fd640 1 -- 172.21.15.5:0/2521386918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f90f0108f10 con 0x7f90f01009a0 2024-09-19T06:03:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.948+0000 7f90e77fe640 1 -- 172.21.15.5:0/2521386918 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f90ec0bd150 con 0x7f90f01009a0 2024-09-19T06:03:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.951+0000 7f90f85fd640 1 -- 172.21.15.5:0/2521386918 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90dc0a3440 msgr2=0x7f90dc0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.951+0000 7f90f85fd640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90dc0a3440 0x7f90dc0a5900 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f90f006a560 tx=0x7f90e000b390 comp rx=0 tx=0).stop 2024-09-19T06:03:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.951+0000 7f90f85fd640 1 -- 172.21.15.5:0/2521386918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f01009a0 msgr2=0x7f90f0199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.951+0000 7f90f85fd640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f01009a0 0x7f90f0199110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f90ec02ae00 tx=0x7f90ec003ba0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.951+0000 7f90f85fd640 1 -- 172.21.15.5:0/2521386918 shutdown_connections 2024-09-19T06:03:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.951+0000 7f90f85fd640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90f01069b0 0x7f90f01a4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.951+0000 7f90f85fd640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90dc0a3440 0x7f90dc0a5900 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.951+0000 7f90f85fd640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90f0101340 0x7f90f0199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.951+0000 7f90f85fd640 1 --2- 172.21.15.5:0/2521386918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f01009a0 0x7f90f0199110 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.951+0000 7f90f85fd640 1 -- 172.21.15.5:0/2521386918 >> 172.21.15.5:0/2521386918 conn(0x7f90f00779a0 msgr2=0x7f90f0103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:29.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.951+0000 7f90f85fd640 1 -- 172.21.15.5:0/2521386918 shutdown_connections 2024-09-19T06:03:29.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:29.951+0000 7f90f85fd640 1 -- 172.21.15.5:0/2521386918 wait complete. 2024-09-19T06:03:29.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-19T06:03:30.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:29 smithi157 ceph-mon[35346]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 293 B/s wr, 25 op/s 2024-09-19T06:03:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:29 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/4178007070' entity='client.admin' 2024-09-19T06:03:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:29 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:29 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:29 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:29 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.073+0000 7f2377cc9640 1 -- 172.21.15.5:0/322504514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23701009a0 msgr2=0x7f2370100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.073+0000 7f2377cc9640 1 --2- 172.21.15.5:0/322504514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23701009a0 0x7f2370100e00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2364009930 tx=0x7f236402a420 comp rx=0 tx=0).stop 2024-09-19T06:03:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.073+0000 7f2377cc9640 1 -- 172.21.15.5:0/322504514 shutdown_connections 2024-09-19T06:03:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.073+0000 7f2377cc9640 1 --2- 172.21.15.5:0/322504514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2370101340 0x7f237010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.073+0000 7f2377cc9640 1 --2- 172.21.15.5:0/322504514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23701009a0 0x7f2370100e00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.073+0000 7f2377cc9640 1 --2- 172.21.15.5:0/322504514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23701069b0 0x7f2370106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.073+0000 7f2377cc9640 1 -- 172.21.15.5:0/322504514 >> 172.21.15.5:0/322504514 conn(0x7f23700779a0 msgr2=0x7f23700fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.073+0000 7f2377cc9640 1 -- 172.21.15.5:0/322504514 shutdown_connections 2024-09-19T06:03:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.073+0000 7f2377cc9640 1 -- 172.21.15.5:0/322504514 wait complete. 2024-09-19T06:03:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2377cc9640 1 Processor -- start 2024-09-19T06:03:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2377cc9640 1 -- start start 2024-09-19T06:03:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2377cc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23701009a0 0x7f23701991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2377cc9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2370101340 0x7f23701996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2377cc9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23701069b0 0x7f23701a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2377cc9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2370110cf0 con 0x7f2370101340 2024-09-19T06:03:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2377cc9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2370110b70 con 0x7f23701069b0 2024-09-19T06:03:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2377cc9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2370110e70 con 0x7f23701009a0 2024-09-19T06:03:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2375264640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2370101340 0x7f23701996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2375264640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2370101340 0x7f23701996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38032/0 (socket says 172.21.15.5:38032) 2024-09-19T06:03:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2375264640 1 -- 172.21.15.5:0/976497934 learned_addr learned my addr 172.21.15.5:0/976497934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2375a65640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23701009a0 0x7f23701991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2376266640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23701069b0 0x7f23701a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2375264640 1 -- 172.21.15.5:0/976497934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23701009a0 msgr2=0x7f23701991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2375264640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23701009a0 0x7f23701991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2375264640 1 -- 172.21.15.5:0/976497934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23701069b0 msgr2=0x7f23701a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2375264640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23701069b0 0x7f23701a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.074+0000 7f2375264640 1 -- 172.21.15.5:0/976497934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23701a4760 con 0x7f2370101340 2024-09-19T06:03:30.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.075+0000 7f2375264640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2370101340 0x7f23701996e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2364005ab0 tx=0x7f2364003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:30.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.075+0000 7f236effd640 1 -- 172.21.15.5:0/976497934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236403f070 con 0x7f2370101340 2024-09-19T06:03:30.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.075+0000 7f2377cc9640 1 -- 172.21.15.5:0/976497934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23701a4930 con 0x7f2370101340 2024-09-19T06:03:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.075+0000 7f236effd640 1 -- 172.21.15.5:0/976497934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2364031b50 con 0x7f2370101340 2024-09-19T06:03:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.075+0000 7f236effd640 1 -- 172.21.15.5:0/976497934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364035030 con 0x7f2370101340 2024-09-19T06:03:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.075+0000 7f2377cc9640 1 -- 172.21.15.5:0/976497934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23701a4da0 con 0x7f2370101340 2024-09-19T06:03:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.076+0000 7f2377cc9640 1 -- 172.21.15.5:0/976497934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23701020e0 con 0x7f2370101340 2024-09-19T06:03:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.078+0000 7f236effd640 1 -- 172.21.15.5:0/976497934 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7f2364042070 con 0x7f2370101340 2024-09-19T06:03:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.078+0000 7f236effd640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f23580a3440 0x7f23580a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.079+0000 7f2375a65640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f23580a3440 0x7f23580a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.079+0000 7f236effd640 1 -- 172.21.15.5:0/976497934 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f23640f0460 con 0x7f2370101340 2024-09-19T06:03:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.079+0000 7f2375a65640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f23580a3440 0x7f23580a5900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2368009b50 tx=0x7f2368005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.083+0000 7f236effd640 1 -- 172.21.15.5:0/976497934 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f23640bc690 con 0x7f2370101340 2024-09-19T06:03:30.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.078+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3038230573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d24106a70 msgr2=0x7f2d24106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.078+0000 7f2d2a2f3640 1 --2- 172.21.15.5:0/3038230573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d24106a70 0x7f2d24106e50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009930 tx=0x7f2d1402a420 comp rx=0 tx=0).stop 2024-09-19T06:03:30.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.078+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3038230573 shutdown_connections 2024-09-19T06:03:30.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.078+0000 7f2d2a2f3640 1 --2- 172.21.15.5:0/3038230573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d24101410 0x7f2d2410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.079+0000 7f2d2a2f3640 1 --2- 172.21.15.5:0/3038230573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d24100a70 0x7f2d24100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.079+0000 7f2d2a2f3640 1 --2- 172.21.15.5:0/3038230573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d24106a70 0x7f2d24106e50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.079+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3038230573 >> 172.21.15.5:0/3038230573 conn(0x7f2d240fc790 msgr2=0x7f2d240febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.079+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3038230573 shutdown_connections 2024-09-19T06:03:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.079+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3038230573 wait complete. 2024-09-19T06:03:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.079+0000 7f2d2a2f3640 1 Processor -- start 2024-09-19T06:03:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.079+0000 7f2d2a2f3640 1 -- start start 2024-09-19T06:03:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d2a2f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d24100a70 0x7f2d241983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d2a2f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d24101410 0x7f2d241988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d2a2f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d24106a70 0x7f2d2419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d2a2f3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d24110d20 con 0x7f2d24106a70 2024-09-19T06:03:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d2a2f3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d24110ba0 con 0x7f2d24100a70 2024-09-19T06:03:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d2a2f3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2d24110ea0 con 0x7f2d24101410 2024-09-19T06:03:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d28890640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d24106a70 0x7f2d2419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d28890640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d24106a70 0x7f2d2419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38046/0 (socket says 172.21.15.5:38046) 2024-09-19T06:03:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d28890640 1 -- 172.21.15.5:0/3656749934 learned_addr learned my addr 172.21.15.5:0/3656749934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d237fe640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d24101410 0x7f2d241988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d23fff640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d24100a70 0x7f2d241983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d28890640 1 -- 172.21.15.5:0/3656749934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d24101410 msgr2=0x7f2d241988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d28890640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d24101410 0x7f2d241988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d28890640 1 -- 172.21.15.5:0/3656749934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d24100a70 msgr2=0x7f2d241983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d28890640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d24100a70 0x7f2d241983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d28890640 1 -- 172.21.15.5:0/3656749934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2419d360 con 0x7f2d24106a70 2024-09-19T06:03:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.080+0000 7f2d237fe640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d24101410 0x7f2d241988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.081+0000 7f2d28890640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d24106a70 0x7f2d2419cc80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00eb10 tx=0x7f2d1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.083+0000 7f2d217fa640 1 -- 172.21.15.5:0/3656749934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c004b20 con 0x7f2d24106a70 2024-09-19T06:03:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.083+0000 7f2d217fa640 1 -- 172.21.15.5:0/3656749934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d1c004cc0 con 0x7f2d24106a70 2024-09-19T06:03:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.083+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3656749934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2419d650 con 0x7f2d24106a70 2024-09-19T06:03:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.083+0000 7f2d217fa640 1 -- 172.21.15.5:0/3656749934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c010690 con 0x7f2d24106a70 2024-09-19T06:03:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.083+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3656749934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d241a4f30 con 0x7f2d24106a70 2024-09-19T06:03:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.086+0000 7f2d2c6cc740 1 Processor -- start 2024-09-19T06:03:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.086+0000 7f2d217fa640 1 -- 172.21.15.5:0/3656749934 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7f2d1c005020 con 0x7f2d24106a70 2024-09-19T06:03:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d2c6cc740 1 -- start start 2024-09-19T06:03:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d2c6cc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266150f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d2c6cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x55726615b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d2c6cc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 0x55726615e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d2c6cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5572660b6280 con 0x557266151480 2024-09-19T06:03:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d2c6cc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55726605db10 con 0x557266150b60 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d2c6cc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5572660b2240 con 0x55726615bd40 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x55726615b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60722/0 (socket says 172.21.15.5:60722) 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d237fe640 1 -- 172.21.15.5:0/3300834028 learned_addr learned my addr 172.21.15.5:0/3300834028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d217fa640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d0c0a34c0 0x7f2d0c0a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d237fe640 1 --2- 172.21.15.5:0/3300834028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 0x55726615e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.087+0000 7f2d217fa640 1 -- 172.21.15.5:0/3656749934 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f2d1c0d0340 con 0x7f2d24106a70 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.088+0000 7f2d23fff640 1 -- 172.21.15.5:0/3300834028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 msgr2=0x55726615e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.088+0000 7f2d23fff640 1 --2- 172.21.15.5:0/3300834028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 0x55726615e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.088+0000 7f2d23fff640 1 -- 172.21.15.5:0/3300834028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 msgr2=0x557266150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.088+0000 7f2d23fff640 1 --2- 172.21.15.5:0/3300834028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.088+0000 7f2d23fff640 1 -- 172.21.15.5:0/3300834028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726606a1f0 con 0x557266151480 2024-09-19T06:03:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.088+0000 7f2d237fe640 1 --2- 172.21.15.5:0/3300834028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.088+0000 7f2d28890640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d0c0a34c0 0x7f2d0c0a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.088+0000 7f2d23fff640 1 --2- 172.21.15.5:0/3300834028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x55726615b800 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2d14031440 tx=0x7f2d14005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb891f19e096825 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.088+0000 7f2d0a7fc640 1 -- 172.21.15.5:0/3300834028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14035040 con 0x557266151480 2024-09-19T06:03:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.088+0000 7f2d0a7fc640 1 -- 172.21.15.5:0/3300834028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d14031db0 con 0x557266151480 2024-09-19T06:03:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.088+0000 7f2d0a7fc640 1 -- 172.21.15.5:0/3300834028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14005280 con 0x557266151480 2024-09-19T06:03:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.089+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3300834028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 msgr2=0x55726615b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.089+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/3300834028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x55726615b800 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2d14031440 tx=0x7f2d14005be0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.089+0000 7f2d28890640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d0c0a34c0 0x7f2d0c0a5980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0cda40 tx=0x7f2d1c005b40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.089+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3300834028 shutdown_connections 2024-09-19T06:03:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.089+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/3300834028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 0x55726615e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.089+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/3300834028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x55726615b800 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.089+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/3300834028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.089+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3300834028 >> 172.21.15.5:0/3300834028 conn(0x55726614d0f0 msgr2=0x55726614d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.089+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3300834028 shutdown_connections 2024-09-19T06:03:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.089+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3300834028 wait complete. 2024-09-19T06:03:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d2c6cc740 1 -- start start 2024-09-19T06:03:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d2c6cc740 1 Processor -- start 2024-09-19T06:03:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d2c6cc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266154e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d2c6cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x557266155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d2c6cc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 0x5572661558c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d2c6cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55726615e900 con 0x557266151480 2024-09-19T06:03:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d2c6cc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557266070f10 con 0x557266150b60 2024-09-19T06:03:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d2c6cc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55726615ea80 con 0x55726615bd40 2024-09-19T06:03:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d2c6cc740 10 client.0 _ll_register_callbacks cb 0x557266039a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x557266155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x557266155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38072/0 (socket says 172.21.15.5:38072) 2024-09-19T06:03:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d23fff640 1 -- 172.21.15.5:0/321229581 learned_addr learned my addr 172.21.15.5:0/321229581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.090+0000 7f2d237fe640 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 0x5572661558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d237fe640 1 -- 172.21.15.5:0/321229581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 msgr2=0x5572661558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d237fe640 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 0x5572661558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d237fe640 1 -- 172.21.15.5:0/321229581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 msgr2=0x557266155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d237fe640 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x557266155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d237fe640 1 -- 172.21.15.5:0/321229581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557265e276a0 con 0x557266150b60 2024-09-19T06:03:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d23fff640 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x557266155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d237fe640 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266154e40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55726614e470 tx=0x7f2d18004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db9e68a98b5c6f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d037fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:03:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d037fe640 1 -- 172.21.15.5:0/321229581 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1801a040 con 0x557266150b60 2024-09-19T06:03:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/321229581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d14031db0 con 0x557266150b60 2024-09-19T06:03:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d037fe640 1 -- 172.21.15.5:0/321229581 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f2d1800e7d0 con 0x557266150b60 2024-09-19T06:03:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d037fe640 1 -- 172.21.15.5:0/321229581 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d1802e050 con 0x557266150b60 2024-09-19T06:03:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d2c6cc740 10 client.25031 fetch_fsmap learned FSMap version 28 2024-09-19T06:03:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.091+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/321229581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55726616a5b0 con 0x557266150b60 2024-09-19T06:03:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.092+0000 7f2d037fe640 1 -- 172.21.15.5:0/321229581 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d14031db0 con 0x557266150b60 2024-09-19T06:03:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.092+0000 7f2d037fe640 1 -- 172.21.15.5:0/321229581 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2d180208b0 con 0x557266150b60 2024-09-19T06:03:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.092+0000 7f2d2c6cc740 10 client.25031 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:03:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.092+0000 7f2d2c6cc740 10 client.25031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:03:30.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.092+0000 7f2d2c6cc740 20 client.25031 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:30.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.092+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55726616b9c0 0x55726616de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.092+0000 7f2d2c6cc740 4 client.25031 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:03:30.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.092+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/321229581 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572660c37f0 con 0x55726616b9c0 2024-09-19T06:03:30.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.092+0000 7f2d23fff640 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55726616b9c0 0x55726616de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.093+0000 7f2d037fe640 10 client.25031 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:30.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.093+0000 7f2d23fff640 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55726616b9c0 0x55726616de60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0b799d248cc94ee server_cookie=6c014bfa6242dcf1 in_seq=0 out_seq=0 2024-09-19T06:03:30.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.093+0000 7f2d037fe640 1 -- 172.21.15.5:0/321229581 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572660c37f0 con 0x55726616b9c0 2024-09-19T06:03:30.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.093+0000 7f2d037fe640 10 client.25031 handle_command_reply: tid=0 2024-09-19T06:03:30.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.093+0000 7f2d2c6cc740 1 client.25031 shutdown 2024-09-19T06:03:30.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.094+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/321229581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 msgr2=0x557266154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.094+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266154e40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55726614e470 tx=0x7f2d18004730 comp rx=0 tx=0).stop 2024-09-19T06:03:30.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.094+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/321229581 shutdown_connections 2024-09-19T06:03:30.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.094+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55726616b9c0 0x55726616de60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.094+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 0x5572661558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.094+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x557266155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.094+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/321229581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266154e40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.094+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/321229581 >> 172.21.15.5:0/321229581 conn(0x55726614d0f0 msgr2=0x55726614c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:30.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.094+0000 7f2d037fe640 0 client.25031 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:30.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.094+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/321229581 shutdown_connections 2024-09-19T06:03:30.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.094+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/321229581 wait complete. 2024-09-19T06:03:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.094+0000 7f2d2c6cc740 20 client.25031 trim_cache size 0 max 16384 2024-09-19T06:03:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d2c6cc740 1 Processor -- start 2024-09-19T06:03:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d2c6cc740 1 -- start start 2024-09-19T06:03:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d2c6cc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266155710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d2c6cc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557266151480 0x557266173730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d2c6cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55726615bd40 0x557266173c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d2c6cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241ab650 con 0x55726615bd40 2024-09-19T06:03:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d2c6cc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241ad190 con 0x557266150b60 2024-09-19T06:03:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d2c6cc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241abd20 con 0x557266151480 2024-09-19T06:03:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55726615bd40 0x557266173c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60742/0 (socket says 172.21.15.5:60742) 2024-09-19T06:03:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d237fe640 1 -- 172.21.15.5:0/1012766204 learned_addr learned my addr 172.21.15.5:0/1012766204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.111+0000 7f2d23fff640 1 --2- 172.21.15.5:0/1012766204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557266151480 0x557266173730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.112+0000 7f2d237fe640 1 -- 172.21.15.5:0/1012766204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557266151480 msgr2=0x557266173730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.112+0000 7f2d237fe640 1 --2- 172.21.15.5:0/1012766204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557266151480 0x557266173730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.112+0000 7f2d237fe640 1 -- 172.21.15.5:0/1012766204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55726615bd40 msgr2=0x557266173c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.112+0000 7f2d237fe640 1 --2- 172.21.15.5:0/1012766204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55726615bd40 0x557266173c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.112+0000 7f2d237fe640 1 -- 172.21.15.5:0/1012766204 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726616e4e0 con 0x557266150b60 2024-09-19T06:03:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.112+0000 7f2d237fe640 1 --2- 172.21.15.5:0/1012766204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266155710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009fd0 tx=0x7f2d18000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61809d507faf5707 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.112+0000 7f2d09ffb640 1 -- 172.21.15.5:0/1012766204 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18011a40 con 0x557266150b60 2024-09-19T06:03:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.112+0000 7f2d09ffb640 1 -- 172.21.15.5:0/1012766204 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d18020e50 con 0x557266150b60 2024-09-19T06:03:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.112+0000 7f2d09ffb640 1 -- 172.21.15.5:0/1012766204 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18011e60 con 0x557266150b60 2024-09-19T06:03:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.113+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/1012766204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 msgr2=0x557266155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.113+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/1012766204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266155710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009fd0 tx=0x7f2d18000bc0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.113+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/1012766204 shutdown_connections 2024-09-19T06:03:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.113+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/1012766204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55726615bd40 0x557266173c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.113+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/1012766204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557266151480 0x557266173730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.113+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/1012766204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x557266155710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.113+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/1012766204 >> 172.21.15.5:0/1012766204 conn(0x55726614d0f0 msgr2=0x557266153ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.113+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/1012766204 shutdown_connections 2024-09-19T06:03:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.113+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/1012766204 wait complete. 2024-09-19T06:03:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.113+0000 7f2d2c6cc740 1 -- start start 2024-09-19T06:03:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.113+0000 7f2d2c6cc740 1 Processor -- start 2024-09-19T06:03:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d2c6cc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x5572661680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d2c6cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x557266163410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d2c6cc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 0x557266163950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d2c6cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241acac0 con 0x557266151480 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d2c6cc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241ac3f0 con 0x557266150b60 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d2c6cc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241a9650 con 0x55726615bd40 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d2c6cc740 10 client.0 _ll_register_callbacks cb 0x7f2d180129e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x557266163410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x5572661680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x557266163410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38098/0 (socket says 172.21.15.5:38098) 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d23fff640 1 -- 172.21.15.5:0/3467852766 learned_addr learned my addr 172.21.15.5:0/3467852766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d237fe640 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 0x557266163950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d237fe640 1 -- 172.21.15.5:0/3467852766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 msgr2=0x557266163950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d237fe640 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 0x557266163950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d237fe640 1 -- 172.21.15.5:0/3467852766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 msgr2=0x557266163410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d237fe640 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x557266163410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d237fe640 1 -- 172.21.15.5:0/3467852766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557266163e90 con 0x557266150b60 2024-09-19T06:03:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d23fff640 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x557266163410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d237fe640 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x5572661680e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d18011540 tx=0x7f2d18012d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b90119330a5fa40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.114+0000 7f2d037fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:03:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d037fe640 1 -- 172.21.15.5:0/3467852766 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18011de0 con 0x557266150b60 2024-09-19T06:03:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3467852766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d18020e50 con 0x557266150b60 2024-09-19T06:03:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d037fe640 1 -- 172.21.15.5:0/3467852766 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f2d18004260 con 0x557266150b60 2024-09-19T06:03:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d037fe640 1 -- 172.21.15.5:0/3467852766 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d180044a0 con 0x557266150b60 2024-09-19T06:03:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d2c6cc740 10 client.25037 fetch_fsmap learned FSMap version 28 2024-09-19T06:03:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3467852766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5572661d6410 con 0x557266150b60 2024-09-19T06:03:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d037fe640 1 -- 172.21.15.5:0/3467852766 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d18020e50 con 0x557266150b60 2024-09-19T06:03:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d037fe640 1 -- 172.21.15.5:0/3467852766 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2d1803e040 con 0x557266150b60 2024-09-19T06:03:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d2c6cc740 10 client.25037 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:03:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d2c6cc740 10 client.25037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:03:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d2c6cc740 20 client.25037 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5572661d6ad0 0x5572661d8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d2c6cc740 4 client.25037 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:03:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3467852766 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5572660bdd00 con 0x5572661d6ad0 2024-09-19T06:03:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.115+0000 7f2d23fff640 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5572661d6ad0 0x5572661d8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.116+0000 7f2d037fe640 10 client.25037 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.116+0000 7f2d23fff640 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5572661d6ad0 0x5572661d8ec0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fa238de8800aab1 server_cookie=af9e5a214be7f384 in_seq=0 out_seq=0 2024-09-19T06:03:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d037fe640 1 -- 172.21.15.5:0/3467852766 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5572660bdd00 con 0x5572661d6ad0 2024-09-19T06:03:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d037fe640 10 client.25037 handle_command_reply: tid=0 2024-09-19T06:03:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d2c6cc740 1 client.25037 shutdown 2024-09-19T06:03:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3467852766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 msgr2=0x5572661680e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x5572661680e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d18011540 tx=0x7f2d18012d40 comp rx=0 tx=0).stop 2024-09-19T06:03:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3467852766 shutdown_connections 2024-09-19T06:03:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5572661d6ad0 0x5572661d8ec0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55726615bd40 0x557266163950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557266151480 0x557266163410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d2c6cc740 1 --2- 172.21.15.5:0/3467852766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557266150b60 0x5572661680e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d037fe640 0 client.25037 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3467852766 >> 172.21.15.5:0/3467852766 conn(0x55726614d0f0 msgr2=0x557266158530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3467852766 shutdown_connections 2024-09-19T06:03:30.157 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:03:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:03:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e9b0dcdf-5878-455d-8a55-dd76f905ecfd", 2024-09-19T06:03:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:03:30.157 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:03:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d2c6cc740 1 -- 172.21.15.5:0/3467852766 wait complete. 2024-09-19T06:03:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.117+0000 7f2d2c6cc740 20 client.25037 trim_cache size 0 max 16384 2024-09-19T06:03:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.118+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3656749934 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d0c0a34c0 msgr2=0x7f2d0c0a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.118+0000 7f2d2a2f3640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d0c0a34c0 0x7f2d0c0a5980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0cda40 tx=0x7f2d1c005b40 comp rx=0 tx=0).stop 2024-09-19T06:03:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.118+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3656749934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d24106a70 msgr2=0x7f2d2419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.118+0000 7f2d2a2f3640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d24106a70 0x7f2d2419cc80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00eb10 tx=0x7f2d1c00c2d0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.118+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3656749934 shutdown_connections 2024-09-19T06:03:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.118+0000 7f2d2a2f3640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d24106a70 0x7f2d2419cc80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.118+0000 7f2d2a2f3640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d0c0a34c0 0x7f2d0c0a5980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.118+0000 7f2d2a2f3640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d24101410 0x7f2d241988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.118+0000 7f2d2a2f3640 1 --2- 172.21.15.5:0/3656749934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d24100a70 0x7f2d241983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.118+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3656749934 >> 172.21.15.5:0/3656749934 conn(0x7f2d240fc790 msgr2=0x7f2d240fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.118+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3656749934 shutdown_connections 2024-09-19T06:03:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.118+0000 7f2d2a2f3640 1 -- 172.21.15.5:0/3656749934 wait complete. 2024-09-19T06:03:30.159 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:29 smithi049 ceph-mon[34640]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 293 B/s wr, 25 op/s 2024-09-19T06:03:30.159 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:29 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/4178007070' entity='client.admin' 2024-09-19T06:03:30.159 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:29 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:30.159 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:29 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:30.159 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:29 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:30.159 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:29 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.283+0000 7f2377cc9640 1 -- 172.21.15.5:0/976497934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f23701089a0 con 0x7f2370101340 2024-09-19T06:03:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.287+0000 7f236effd640 1 -- 172.21.15.5:0/976497934 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f23640c0990 con 0x7f2370101340 2024-09-19T06:03:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.289+0000 7f2377cc9640 1 -- 172.21.15.5:0/976497934 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f23580a3440 msgr2=0x7f23580a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.289+0000 7f2377cc9640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f23580a3440 0x7f23580a5900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2368009b50 tx=0x7f2368005f90 comp rx=0 tx=0).stop 2024-09-19T06:03:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.289+0000 7f2377cc9640 1 -- 172.21.15.5:0/976497934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2370101340 msgr2=0x7f23701996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.289+0000 7f2377cc9640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2370101340 0x7f23701996e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2364005ab0 tx=0x7f2364003b90 comp rx=0 tx=0).stop 2024-09-19T06:03:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.289+0000 7f2377cc9640 1 -- 172.21.15.5:0/976497934 shutdown_connections 2024-09-19T06:03:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.289+0000 7f2377cc9640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23701069b0 0x7f23701a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.289+0000 7f2377cc9640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2370101340 0x7f23701996e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.289+0000 7f2377cc9640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f23580a3440 0x7f23580a5900 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.289+0000 7f2377cc9640 1 --2- 172.21.15.5:0/976497934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23701009a0 0x7f23701991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.289+0000 7f2377cc9640 1 -- 172.21.15.5:0/976497934 >> 172.21.15.5:0/976497934 conn(0x7f23700779a0 msgr2=0x7f2370103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.289+0000 7f2377cc9640 1 -- 172.21.15.5:0/976497934 shutdown_connections 2024-09-19T06:03:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.289+0000 7f2377cc9640 1 -- 172.21.15.5:0/976497934 wait complete. 2024-09-19T06:03:30.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-19T06:03:30.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.411+0000 7f1583fcf640 1 -- 172.21.15.5:0/2879584353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c1069e0 msgr2=0x7f157c106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.411+0000 7f1583fcf640 1 --2- 172.21.15.5:0/2879584353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c1069e0 0x7f157c106dc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1578009930 tx=0x7f157802a420 comp rx=0 tx=0).stop 2024-09-19T06:03:30.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.412+0000 7f1583fcf640 1 -- 172.21.15.5:0/2879584353 shutdown_connections 2024-09-19T06:03:30.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.412+0000 7f1583fcf640 1 --2- 172.21.15.5:0/2879584353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c101370 0x7f157c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.412+0000 7f1583fcf640 1 --2- 172.21.15.5:0/2879584353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f157c1009d0 0x7f157c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.412+0000 7f1583fcf640 1 --2- 172.21.15.5:0/2879584353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c1069e0 0x7f157c106dc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.412+0000 7f1583fcf640 1 -- 172.21.15.5:0/2879584353 >> 172.21.15.5:0/2879584353 conn(0x7f157c0fc6b0 msgr2=0x7f157c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:30.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.412+0000 7f1583fcf640 1 -- 172.21.15.5:0/2879584353 shutdown_connections 2024-09-19T06:03:30.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.412+0000 7f1583fcf640 1 -- 172.21.15.5:0/2879584353 wait complete. 2024-09-19T06:03:30.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.412+0000 7f1583fcf640 1 Processor -- start 2024-09-19T06:03:30.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.412+0000 7f1583fcf640 1 -- start start 2024-09-19T06:03:30.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.412+0000 7f1583fcf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c1009d0 0x7f157c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1583fcf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c101370 0x7f157c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1583fcf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f157c1069e0 0x7f157c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1583fcf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f157c110ce0 con 0x7f157c1009d0 2024-09-19T06:03:30.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1583fcf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f157c110b60 con 0x7f157c1069e0 2024-09-19T06:03:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1583fcf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f157c110e60 con 0x7f157c101370 2024-09-19T06:03:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1581d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c1009d0 0x7f157c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1581d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c1009d0 0x7f157c197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38124/0 (socket says 172.21.15.5:38124) 2024-09-19T06:03:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1581d6b640 1 -- 172.21.15.5:0/3799064864 learned_addr learned my addr 172.21.15.5:0/3799064864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f158256c640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f157c1069e0 0x7f157c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f158156a640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c101370 0x7f157c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1581d6b640 1 -- 172.21.15.5:0/3799064864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c101370 msgr2=0x7f157c1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1581d6b640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c101370 0x7f157c1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1581d6b640 1 -- 172.21.15.5:0/3799064864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f157c1069e0 msgr2=0x7f157c19bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1581d6b640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f157c1069e0 0x7f157c19bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.413+0000 7f1581d6b640 1 -- 172.21.15.5:0/3799064864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c19c260 con 0x7f157c1009d0 2024-09-19T06:03:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.414+0000 7f1581d6b640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c1009d0 0x7f157c197270 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1578005ab0 tx=0x7f1578003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.414+0000 7f1572ffd640 1 -- 172.21.15.5:0/3799064864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157803f070 con 0x7f157c1009d0 2024-09-19T06:03:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.414+0000 7f1572ffd640 1 -- 172.21.15.5:0/3799064864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1578031d60 con 0x7f157c1009d0 2024-09-19T06:03:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.414+0000 7f1572ffd640 1 -- 172.21.15.5:0/3799064864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578035030 con 0x7f157c1009d0 2024-09-19T06:03:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.414+0000 7f1583fcf640 1 -- 172.21.15.5:0/3799064864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157c19c4f0 con 0x7f157c1009d0 2024-09-19T06:03:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.414+0000 7f1583fcf640 1 -- 172.21.15.5:0/3799064864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f157c1a3de0 con 0x7f157c1009d0 2024-09-19T06:03:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.416+0000 7f1583fcf640 1 -- 172.21.15.5:0/3799064864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157c100e30 con 0x7f157c1009d0 2024-09-19T06:03:30.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.417+0000 7f1572ffd640 1 -- 172.21.15.5:0/3799064864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7f1578042070 con 0x7f157c1009d0 2024-09-19T06:03:30.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.418+0000 7f1572ffd640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15680a34c0 0x7f15680a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.419+0000 7f158156a640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15680a34c0 0x7f15680a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.419+0000 7f1572ffd640 1 -- 172.21.15.5:0/3799064864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f15780f01e0 con 0x7f157c1009d0 2024-09-19T06:03:30.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.419+0000 7f158156a640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15680a34c0 0x7f15680a5980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f157c0fed30 tx=0x7f156c00b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:30.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.423+0000 7f1572ffd640 1 -- 172.21.15.5:0/3799064864 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f15780bc410 con 0x7f157c1009d0 2024-09-19T06:03:30.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.622+0000 7f1583fcf640 1 -- 172.21.15.5:0/3799064864 --> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f157c062630 con 0x7f15680a34c0 2024-09-19T06:03:30.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.691+0000 7f1572ffd640 1 -- 172.21.15.5:0/3799064864 <== mgr.15240 v2:172.21.15.5:6800/2837801016 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f157c062630 con 0x7f15680a34c0 2024-09-19T06:03:30.693 INFO:teuthology.orchestra.run.smithi005.stdout:Schedule set for path / 2024-09-19T06:03:30.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.694+0000 7f1583fcf640 1 -- 172.21.15.5:0/3799064864 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15680a34c0 msgr2=0x7f15680a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.694+0000 7f1583fcf640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15680a34c0 0x7f15680a5980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f157c0fed30 tx=0x7f156c00b390 comp rx=0 tx=0).stop 2024-09-19T06:03:30.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.694+0000 7f1583fcf640 1 -- 172.21.15.5:0/3799064864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c1009d0 msgr2=0x7f157c197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.694+0000 7f1583fcf640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c1009d0 0x7f157c197270 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1578005ab0 tx=0x7f1578003cd0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.694+0000 7f1583fcf640 1 -- 172.21.15.5:0/3799064864 shutdown_connections 2024-09-19T06:03:30.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.694+0000 7f1583fcf640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f157c1069e0 0x7f157c19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.694+0000 7f1583fcf640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15680a34c0 0x7f15680a5980 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.694+0000 7f1583fcf640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c101370 0x7f157c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.694+0000 7f1583fcf640 1 --2- 172.21.15.5:0/3799064864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c1009d0 0x7f157c197270 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.694+0000 7f1583fcf640 1 -- 172.21.15.5:0/3799064864 >> 172.21.15.5:0/3799064864 conn(0x7f157c0fc6b0 msgr2=0x7f157c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:30.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.694+0000 7f1583fcf640 1 -- 172.21.15.5:0/3799064864 shutdown_connections 2024-09-19T06:03:30.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.694+0000 7f1583fcf640 1 -- 172.21.15.5:0/3799064864 wait complete. 2024-09-19T06:03:30.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-19T06:03:30.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.812+0000 7effc9349640 1 -- 172.21.15.5:0/1475797645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc410c110 msgr2=0x7effc410e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.812+0000 7effc9349640 1 --2- 172.21.15.5:0/1475797645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc410c110 0x7effc410e580 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7effbc009ca0 tx=0x7effbc02a950 comp rx=0 tx=0).stop 2024-09-19T06:03:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.812+0000 7effc9349640 1 -- 172.21.15.5:0/1475797645 shutdown_connections 2024-09-19T06:03:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.812+0000 7effc9349640 1 --2- 172.21.15.5:0/1475797645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc410c110 0x7effc410e580 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.812+0000 7effc9349640 1 --2- 172.21.15.5:0/1475797645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc4103660 0x7effc410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.812+0000 7effc9349640 1 --2- 172.21.15.5:0/1475797645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effc4102d40 0x7effc4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.812+0000 7effc9349640 1 -- 172.21.15.5:0/1475797645 >> 172.21.15.5:0/1475797645 conn(0x7effc40fc6b0 msgr2=0x7effc40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.812+0000 7effc9349640 1 -- 172.21.15.5:0/1475797645 shutdown_connections 2024-09-19T06:03:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.812+0000 7effc9349640 1 -- 172.21.15.5:0/1475797645 wait complete. 2024-09-19T06:03:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc9349640 1 Processor -- start 2024-09-19T06:03:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc9349640 1 -- start start 2024-09-19T06:03:30.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc9349640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effc4102d40 0x7effc4198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc9349640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc4103660 0x7effc4198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc9349640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc410c110 0x7effc419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc9349640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7effc4111d70 con 0x7effc4103660 2024-09-19T06:03:30.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc9349640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7effc4111bf0 con 0x7effc4102d40 2024-09-19T06:03:30.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc9349640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7effc4111ef0 con 0x7effc410c110 2024-09-19T06:03:30.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc4103660 0x7effc4198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc4103660 0x7effc4198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38148/0 (socket says 172.21.15.5:38148) 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc27fc640 1 -- 172.21.15.5:0/816175093 learned_addr learned my addr 172.21.15.5:0/816175093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc37fe640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc410c110 0x7effc419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.813+0000 7effc2ffd640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effc4102d40 0x7effc4198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.814+0000 7effc27fc640 1 -- 172.21.15.5:0/816175093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc410c110 msgr2=0x7effc419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.814+0000 7effc27fc640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc410c110 0x7effc419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.814+0000 7effc27fc640 1 -- 172.21.15.5:0/816175093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effc4102d40 msgr2=0x7effc4198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.814+0000 7effc27fc640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effc4102d40 0x7effc4198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.814+0000 7effc27fc640 1 -- 172.21.15.5:0/816175093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc419d210 con 0x7effc4103660 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.814+0000 7effc27fc640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc4103660 0x7effc4198760 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7effb800d770 tx=0x7effb800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.814+0000 7effabfff640 1 -- 172.21.15.5:0/816175093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8017070 con 0x7effc4103660 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.814+0000 7effabfff640 1 -- 172.21.15.5:0/816175093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effb80054d0 con 0x7effc4103660 2024-09-19T06:03:30.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.814+0000 7effabfff640 1 -- 172.21.15.5:0/816175093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8004cf0 con 0x7effc4103660 2024-09-19T06:03:30.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.814+0000 7effc9349640 1 -- 172.21.15.5:0/816175093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc419d500 con 0x7effc4103660 2024-09-19T06:03:30.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.814+0000 7effc9349640 1 -- 172.21.15.5:0/816175093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc41a4d90 con 0x7effc4103660 2024-09-19T06:03:30.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.815+0000 7effc9349640 1 -- 172.21.15.5:0/816175093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc4103120 con 0x7effc4103660 2024-09-19T06:03:30.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.816+0000 7effabfff640 1 -- 172.21.15.5:0/816175093 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7effb8004e90 con 0x7effc4103660 2024-09-19T06:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.817+0000 7effabfff640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7effac0a3440 0x7effac0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.817+0000 7effc2ffd640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7effac0a3440 0x7effac0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.817+0000 7effabfff640 1 -- 172.21.15.5:0/816175093 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7effb80cf930 con 0x7effc4103660 2024-09-19T06:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.817+0000 7effc2ffd640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7effac0a3440 0x7effac0a5900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effb4005e10 tx=0x7effb4005da0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:30.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:30.821+0000 7effabfff640 1 -- 172.21.15.5:0/816175093 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7effb809c1b0 con 0x7effc4103660 2024-09-19T06:03:30.936 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:03:30.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:30 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2521386918' entity='client.admin' 2024-09-19T06:03:30.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:30 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:30.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:30 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:30.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:30 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:30.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:30 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:30.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:30 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/976497934' entity='client.admin' 2024-09-19T06:03:30.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:30 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:30.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:30 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:30.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:30 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:30.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:30 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:31.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.023+0000 7effc9349640 1 -- 172.21.15.5:0/816175093 --> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7effc4062630 con 0x7effac0a3440 2024-09-19T06:03:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.087+0000 7effabfff640 1 -- 172.21.15.5:0/816175093 <== mgr.15240 v2:172.21.15.5:6800/2837801016 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7effc4062630 con 0x7effac0a3440 2024-09-19T06:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout:Retention added to path / 2024-09-19T06:03:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.089+0000 7effc9349640 1 -- 172.21.15.5:0/816175093 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7effac0a3440 msgr2=0x7effac0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.089+0000 7effc9349640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7effac0a3440 0x7effac0a5900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effb4005e10 tx=0x7effb4005da0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.089+0000 7effc9349640 1 -- 172.21.15.5:0/816175093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc4103660 msgr2=0x7effc4198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.089+0000 7effc9349640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc4103660 0x7effc4198760 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7effb800d770 tx=0x7effb800dc30 comp rx=0 tx=0).stop 2024-09-19T06:03:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.089+0000 7effc9349640 1 -- 172.21.15.5:0/816175093 shutdown_connections 2024-09-19T06:03:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.089+0000 7effc9349640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7effac0a3440 0x7effac0a5900 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.089+0000 7effc9349640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc410c110 0x7effc419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.089+0000 7effc9349640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc4103660 0x7effc4198760 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.089+0000 7effc9349640 1 --2- 172.21.15.5:0/816175093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effc4102d40 0x7effc4198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.089+0000 7effc9349640 1 -- 172.21.15.5:0/816175093 >> 172.21.15.5:0/816175093 conn(0x7effc40fc6b0 msgr2=0x7effc40fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.090+0000 7effc9349640 1 -- 172.21.15.5:0/816175093 shutdown_connections 2024-09-19T06:03:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.090+0000 7effc9349640 1 -- 172.21.15.5:0/816175093 wait complete. 2024-09-19T06:03:31.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T06:03:31.210 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:03:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.265+0000 7f4a2af0a640 1 -- 172.21.15.5:0/1041679910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a241009a0 msgr2=0x7f4a24100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.265+0000 7f4a2af0a640 1 --2- 172.21.15.5:0/1041679910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a241009a0 0x7f4a24100e00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009930 tx=0x7f4a1802a420 comp rx=0 tx=0).stop 2024-09-19T06:03:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.265+0000 7f4a2af0a640 1 -- 172.21.15.5:0/1041679910 shutdown_connections 2024-09-19T06:03:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.265+0000 7f4a2af0a640 1 --2- 172.21.15.5:0/1041679910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a24101340 0x7f4a2410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.265+0000 7f4a2af0a640 1 --2- 172.21.15.5:0/1041679910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a241009a0 0x7f4a24100e00 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.265+0000 7f4a2af0a640 1 --2- 172.21.15.5:0/1041679910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a241069b0 0x7f4a24106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.265+0000 7f4a2af0a640 1 -- 172.21.15.5:0/1041679910 >> 172.21.15.5:0/1041679910 conn(0x7f4a240779a0 msgr2=0x7f4a240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.265+0000 7f4a2af0a640 1 -- 172.21.15.5:0/1041679910 shutdown_connections 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.265+0000 7f4a2af0a640 1 -- 172.21.15.5:0/1041679910 wait complete. 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.265+0000 7f4a2af0a640 1 Processor -- start 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a2af0a640 1 -- start start 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a2af0a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a241009a0 0x7f4a24198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a2af0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a24101340 0x7f4a24198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a2af0a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a241069b0 0x7f4a2419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a2af0a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a24110cf0 con 0x7f4a24101340 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a2af0a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4a24110b70 con 0x7f4a241069b0 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a2af0a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a24110e70 con 0x7f4a241009a0 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a24101340 0x7f4a24198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a24101340 0x7f4a24198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38166/0 (socket says 172.21.15.5:38166) 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a23fff640 1 -- 172.21.15.5:0/567017065 learned_addr learned my addr 172.21.15.5:0/567017065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a28ca6640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a241009a0 0x7f4a24198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a294a7640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a241069b0 0x7f4a2419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a23fff640 1 -- 172.21.15.5:0/567017065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a241009a0 msgr2=0x7f4a24198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a23fff640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a241009a0 0x7f4a24198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a23fff640 1 -- 172.21.15.5:0/567017065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a241069b0 msgr2=0x7f4a2419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a23fff640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a241069b0 0x7f4a2419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a23fff640 1 -- 172.21.15.5:0/567017065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2419d300 con 0x7f4a24101340 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a294a7640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a241069b0 0x7f4a2419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.266+0000 7f4a23fff640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a24101340 0x7f4a24198850 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4a18005ab0 tx=0x7f4a18003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.267+0000 7f4a21ffb640 1 -- 172.21.15.5:0/567017065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1803f070 con 0x7f4a24101340 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.267+0000 7f4a21ffb640 1 -- 172.21.15.5:0/567017065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a18031d60 con 0x7f4a24101340 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.267+0000 7f4a21ffb640 1 -- 172.21.15.5:0/567017065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18035030 con 0x7f4a24101340 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.267+0000 7f4a2af0a640 1 -- 172.21.15.5:0/567017065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a2419d590 con 0x7f4a24101340 2024-09-19T06:03:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.267+0000 7f4a2af0a640 1 -- 172.21.15.5:0/567017065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a241a4e80 con 0x7f4a24101340 2024-09-19T06:03:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.268+0000 7f4a2af0a640 1 -- 172.21.15.5:0/567017065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a241020e0 con 0x7f4a24101340 2024-09-19T06:03:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.269+0000 7f4a21ffb640 1 -- 172.21.15.5:0/567017065 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7f4a18042070 con 0x7f4a24101340 2024-09-19T06:03:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.270+0000 7f4a21ffb640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a0c0a34c0 0x7f4a0c0a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.270+0000 7f4a21ffb640 1 -- 172.21.15.5:0/567017065 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f4a180f0290 con 0x7f4a24101340 2024-09-19T06:03:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.270+0000 7f4a28ca6640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a0c0a34c0 0x7f4a0c0a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.270+0000 7f4a28ca6640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a0c0a34c0 0x7f4a0c0a5980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400bb60 tx=0x7f4a1400b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.274+0000 7f4a21ffb640 1 -- 172.21.15.5:0/567017065 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4a180bcb10 con 0x7f4a24101340 2024-09-19T06:03:31.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:30 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2521386918' entity='client.admin' 2024-09-19T06:03:31.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:30 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:31.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:30 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:31.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:30 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:31.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:30 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:31.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:30 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/976497934' entity='client.admin' 2024-09-19T06:03:31.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:30 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:31.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:30 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:31.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:30 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:31.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:30 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:31.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:30 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2521386918' entity='client.admin' 2024-09-19T06:03:31.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:30 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:31.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:30 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:31.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:30 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:31.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:30 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:31.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:30 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/976497934' entity='client.admin' 2024-09-19T06:03:31.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:30 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:03:31.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:30 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:03:31.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:30 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:03:31.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:30 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:03:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.471+0000 7f4a2af0a640 1 -- 172.21.15.5:0/567017065 --> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f4a24109f50 con 0x7f4a0c0a34c0 2024-09-19T06:03:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.479+0000 7f4a21ffb640 1 -- 172.21.15.5:0/567017065 <== mgr.15240 v2:172.21.15.5:6800/2837801016 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f4a24109f50 con 0x7f4a0c0a34c0 2024-09-19T06:03:31.481 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T06:03:30", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-19T06:03:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.481+0000 7f4a2af0a640 1 -- 172.21.15.5:0/567017065 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a0c0a34c0 msgr2=0x7f4a0c0a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.481+0000 7f4a2af0a640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a0c0a34c0 0x7f4a0c0a5980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400bb60 tx=0x7f4a1400b390 comp rx=0 tx=0).stop 2024-09-19T06:03:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.481+0000 7f4a2af0a640 1 -- 172.21.15.5:0/567017065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a24101340 msgr2=0x7f4a24198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.481+0000 7f4a2af0a640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a24101340 0x7f4a24198850 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4a18005ab0 tx=0x7f4a18003cd0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.481+0000 7f4a2af0a640 1 -- 172.21.15.5:0/567017065 shutdown_connections 2024-09-19T06:03:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.481+0000 7f4a2af0a640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a241069b0 0x7f4a2419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.481+0000 7f4a2af0a640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a24101340 0x7f4a24198850 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.481+0000 7f4a2af0a640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a0c0a34c0 0x7f4a0c0a5980 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.481+0000 7f4a2af0a640 1 --2- 172.21.15.5:0/567017065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a241009a0 0x7f4a24198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.481+0000 7f4a2af0a640 1 -- 172.21.15.5:0/567017065 >> 172.21.15.5:0/567017065 conn(0x7f4a240779a0 msgr2=0x7f4a24069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.481+0000 7f4a2af0a640 1 -- 172.21.15.5:0/567017065 shutdown_connections 2024-09-19T06:03:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.481+0000 7f4a2af0a640 1 -- 172.21.15.5:0/567017065 wait complete. 2024-09-19T06:03:31.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T06:03:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.599+0000 7efe7488b640 1 -- 172.21.15.5:0/3588763671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7010c130 msgr2=0x7efe7010e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.599+0000 7efe7488b640 1 --2- 172.21.15.5:0/3588763671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7010c130 0x7efe7010e580 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efe68009900 tx=0x7efe6802a410 comp rx=0 tx=0).stop 2024-09-19T06:03:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.599+0000 7efe7488b640 1 -- 172.21.15.5:0/3588763671 shutdown_connections 2024-09-19T06:03:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.599+0000 7efe7488b640 1 --2- 172.21.15.5:0/3588763671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7010c130 0x7efe7010e580 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.599+0000 7efe7488b640 1 --2- 172.21.15.5:0/3588763671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe701035e0 0x7efe7010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.599+0000 7efe7488b640 1 --2- 172.21.15.5:0/3588763671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe70102c30 0x7efe70103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.599+0000 7efe7488b640 1 -- 172.21.15.5:0/3588763671 >> 172.21.15.5:0/3588763671 conn(0x7efe700fc6f0 msgr2=0x7efe700feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.599+0000 7efe7488b640 1 -- 172.21.15.5:0/3588763671 shutdown_connections 2024-09-19T06:03:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.599+0000 7efe7488b640 1 -- 172.21.15.5:0/3588763671 wait complete. 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe7488b640 1 Processor -- start 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe7488b640 1 -- start start 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe7488b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe70102c30 0x7efe701982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe7488b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe701035e0 0x7efe70198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe7488b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7010c130 0x7efe7019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe7488b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe70111de0 con 0x7efe7010c130 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe7488b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efe70111c60 con 0x7efe70102c30 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe7488b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efe70111f60 con 0x7efe701035e0 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe6ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7010c130 0x7efe7019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe6ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7010c130 0x7efe7019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38182/0 (socket says 172.21.15.5:38182) 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe6ed9d640 1 -- 172.21.15.5:0/1805005868 learned_addr learned my addr 172.21.15.5:0/1805005868 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe6dd9b640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe701035e0 0x7efe70198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe6e59c640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe70102c30 0x7efe701982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe6ed9d640 1 -- 172.21.15.5:0/1805005868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe701035e0 msgr2=0x7efe70198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe6ed9d640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe701035e0 0x7efe70198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe6ed9d640 1 -- 172.21.15.5:0/1805005868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe70102c30 msgr2=0x7efe701982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe6ed9d640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe70102c30 0x7efe701982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.600+0000 7efe6ed9d640 1 -- 172.21.15.5:0/1805005868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7019d2c0 con 0x7efe7010c130 2024-09-19T06:03:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.601+0000 7efe6ed9d640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7010c130 0x7efe7019cb90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efe6802a930 tx=0x7efe68003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.601+0000 7efe5f7fe640 1 -- 172.21.15.5:0/1805005868 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6803f070 con 0x7efe7010c130 2024-09-19T06:03:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.601+0000 7efe5f7fe640 1 -- 172.21.15.5:0/1805005868 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe68003e20 con 0x7efe7010c130 2024-09-19T06:03:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.601+0000 7efe7488b640 1 -- 172.21.15.5:0/1805005868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7019d550 con 0x7efe7010c130 2024-09-19T06:03:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.601+0000 7efe5f7fe640 1 -- 172.21.15.5:0/1805005868 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68035030 con 0x7efe7010c130 2024-09-19T06:03:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.601+0000 7efe7488b640 1 -- 172.21.15.5:0/1805005868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe701a4e40 con 0x7efe7010c130 2024-09-19T06:03:31.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.602+0000 7efe7488b640 1 -- 172.21.15.5:0/1805005868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe70103010 con 0x7efe7010c130 2024-09-19T06:03:31.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.603+0000 7efe5f7fe640 1 -- 172.21.15.5:0/1805005868 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7efe68042070 con 0x7efe7010c130 2024-09-19T06:03:31.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.603+0000 7efe5f7fe640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efe580a3440 0x7efe580a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.604+0000 7efe6e59c640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efe580a3440 0x7efe580a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.604+0000 7efe5f7fe640 1 -- 172.21.15.5:0/1805005868 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7efe680f05e0 con 0x7efe7010c130 2024-09-19T06:03:31.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.604+0000 7efe6e59c640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efe580a3440 0x7efe580a5900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efe60005c40 tx=0x7efe6000a570 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:31.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.608+0000 7efe5f7fe640 1 -- 172.21.15.5:0/1805005868 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efe680bc7b0 con 0x7efe7010c130 2024-09-19T06:03:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.733+0000 7f4a2d164640 1 -- 172.21.15.5:0/3681122842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a281082a0 msgr2=0x7f4a2810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.733+0000 7f4a2d164640 1 --2- 172.21.15.5:0/3681122842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a281082a0 0x7f4a2810ec40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009cd0 tx=0x7f4a2002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.733+0000 7f4a2d164640 1 -- 172.21.15.5:0/3681122842 shutdown_connections 2024-09-19T06:03:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.733+0000 7f4a2d164640 1 --2- 172.21.15.5:0/3681122842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a281082a0 0x7f4a2810ec40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.733+0000 7f4a2d164640 1 --2- 172.21.15.5:0/3681122842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a281036e0 0x7f4a28107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.733+0000 7f4a2d164640 1 --2- 172.21.15.5:0/3681122842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a28102d30 0x7f4a28103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.733+0000 7f4a2d164640 1 -- 172.21.15.5:0/3681122842 >> 172.21.15.5:0/3681122842 conn(0x7f4a280fc7d0 msgr2=0x7f4a280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.734+0000 7f4a2d164640 1 -- 172.21.15.5:0/3681122842 shutdown_connections 2024-09-19T06:03:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.734+0000 7f4a2d164640 1 -- 172.21.15.5:0/3681122842 wait complete. 2024-09-19T06:03:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.734+0000 7f4a2d164640 1 Processor -- start 2024-09-19T06:03:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2d164640 1 -- start start 2024-09-19T06:03:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2d164640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a28102d30 0x7f4a281984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2d164640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a281036e0 0x7f4a28198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2d164640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a281082a0 0x7f4a2819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2d164640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a28111f80 con 0x7f4a281082a0 2024-09-19T06:03:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2d164640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4a28111e00 con 0x7f4a281036e0 2024-09-19T06:03:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2d164640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a28112100 con 0x7f4a28102d30 2024-09-19T06:03:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a281082a0 0x7f4a2819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a281082a0 0x7f4a2819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38204/0 (socket says 172.21.15.5:38204) 2024-09-19T06:03:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2759e640 1 -- 172.21.15.5:0/1674326734 learned_addr learned my addr 172.21.15.5:0/1674326734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a26d9d640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a28102d30 0x7f4a281984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2659c640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a281036e0 0x7f4a28198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2759e640 1 -- 172.21.15.5:0/1674326734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a28102d30 msgr2=0x7f4a281984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2759e640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a28102d30 0x7f4a281984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2759e640 1 -- 172.21.15.5:0/1674326734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a281036e0 msgr2=0x7f4a28198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2759e640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a281036e0 0x7f4a28198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.735+0000 7f4a2759e640 1 -- 172.21.15.5:0/1674326734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2819d470 con 0x7f4a281082a0 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.736+0000 7f4a2659c640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a281036e0 0x7f4a28198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.736+0000 7f4a2759e640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a281082a0 0x7f4a2819cd90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4a20005e30 tx=0x7f4a20003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.736+0000 7f4a0ffff640 1 -- 172.21.15.5:0/1674326734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20004240 con 0x7f4a281082a0 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.736+0000 7f4a2d164640 1 -- 172.21.15.5:0/1674326734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a2819d700 con 0x7f4a281082a0 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.736+0000 7f4a0ffff640 1 -- 172.21.15.5:0/1674326734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a20031ce0 con 0x7f4a281082a0 2024-09-19T06:03:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.736+0000 7f4a0ffff640 1 -- 172.21.15.5:0/1674326734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20030ec0 con 0x7f4a281082a0 2024-09-19T06:03:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.736+0000 7f4a2d164640 1 -- 172.21.15.5:0/1674326734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a281a5040 con 0x7f4a281082a0 2024-09-19T06:03:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.738+0000 7f4a2f53d740 1 Processor -- start 2024-09-19T06:03:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.738+0000 7f4a0ffff640 1 -- 172.21.15.5:0/1674326734 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7f4a200043e0 con 0x7f4a281082a0 2024-09-19T06:03:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.739+0000 7f4a2f53d740 1 -- start start 2024-09-19T06:03:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.739+0000 7f4a2f53d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505aa5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.739+0000 7f4a2f53d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 0x55e505ab4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.739+0000 7f4a2f53d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 0x55e505ab6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.739+0000 7f4a2f53d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e505a0b2e0 con 0x55e505aa53a0 2024-09-19T06:03:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.739+0000 7f4a2f53d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e5059b2ce0 con 0x55e505ab4a90 2024-09-19T06:03:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.739+0000 7f4a2f53d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e505a072a0 con 0x55e505aa5cc0 2024-09-19T06:03:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.739+0000 7f4a2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505aa5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.739+0000 7f4a2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505aa5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38208/0 (socket says 172.21.15.5:38208) 2024-09-19T06:03:31.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.739+0000 7f4a2659c640 1 -- 172.21.15.5:0/1570188054 learned_addr learned my addr 172.21.15.5:0/1570188054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:31.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a0ffff640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a100a35e0 0x7f4a100a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a2659c640 1 --2- 172.21.15.5:0/1570188054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 0x55e505ab6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a2659c640 1 -- 172.21.15.5:0/1570188054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 msgr2=0x55e505ab4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:31.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a2659c640 1 --2- 172.21.15.5:0/1570188054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 0x55e505ab4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a2659c640 1 -- 172.21.15.5:0/1570188054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 msgr2=0x55e505ab6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:31.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a2659c640 1 --2- 172.21.15.5:0/1570188054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 0x55e505ab6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a2659c640 1 -- 172.21.15.5:0/1570188054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e505997440 con 0x55e505aa53a0 2024-09-19T06:03:31.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a2759e640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a100a35e0 0x7f4a100a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a0ffff640 1 -- 172.21.15.5:0/1674326734 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f4a2002e030 con 0x7f4a281082a0 2024-09-19T06:03:31.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a2659c640 1 --2- 172.21.15.5:0/1570188054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505aa5780 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00bfc0 tx=0x7f4a1c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55f717d44acce03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:31.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a0d7fa640 1 -- 172.21.15.5:0/1570188054 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c019030 con 0x55e505aa53a0 2024-09-19T06:03:31.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a0d7fa640 1 -- 172.21.15.5:0/1570188054 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a1c0035f0 con 0x55e505aa53a0 2024-09-19T06:03:31.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.740+0000 7f4a0d7fa640 1 -- 172.21.15.5:0/1570188054 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c0116e0 con 0x55e505aa53a0 2024-09-19T06:03:31.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.741+0000 7f4a2f53d740 1 -- 172.21.15.5:0/1570188054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 msgr2=0x55e505aa5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.741+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/1570188054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505aa5780 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00bfc0 tx=0x7f4a1c00b670 comp rx=0 tx=0).stop 2024-09-19T06:03:31.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.741+0000 7f4a2759e640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a100a35e0 0x7f4a100a5aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a200eed40 tx=0x7f4a200a8040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:31.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.741+0000 7f4a2f53d740 1 -- 172.21.15.5:0/1570188054 shutdown_connections 2024-09-19T06:03:31.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.741+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/1570188054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 0x55e505ab6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.741+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/1570188054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 0x55e505ab4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.741+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/1570188054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505aa5780 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.741+0000 7f4a2f53d740 1 -- 172.21.15.5:0/1570188054 >> 172.21.15.5:0/1570188054 conn(0x55e505aa1a30 msgr2=0x55e505aa1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.741+0000 7f4a2f53d740 1 -- 172.21.15.5:0/1570188054 shutdown_connections 2024-09-19T06:03:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.741+0000 7f4a2f53d740 1 -- 172.21.15.5:0/1570188054 wait complete. 2024-09-19T06:03:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.741+0000 7f4a2f53d740 1 -- start start 2024-09-19T06:03:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2f53d740 1 Processor -- start 2024-09-19T06:03:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2f53d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505ab4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2f53d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 0x55e505ab16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2f53d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 0x55e505ab1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2f53d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e505ab76c0 con 0x55e505aa53a0 2024-09-19T06:03:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2f53d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e5059c6050 con 0x55e505ab4a90 2024-09-19T06:03:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2f53d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e505ab7840 con 0x55e505aa5cc0 2024-09-19T06:03:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2f53d740 10 client.0 _ll_register_callbacks cb 0x55e505a06160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505ab4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a26d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 0x55e505ab16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505ab4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38214/0 (socket says 172.21.15.5:38214) 2024-09-19T06:03:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a26d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 0x55e505ab16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40296/0 (socket says 172.21.15.5:40296) 2024-09-19T06:03:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2659c640 1 -- 172.21.15.5:0/2014879950 learned_addr learned my addr 172.21.15.5:0/2014879950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2659c640 1 -- 172.21.15.5:0/2014879950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 msgr2=0x55e505ab16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2659c640 1 --2- 172.21.15.5:0/2014879950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 0x55e505ab16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.742+0000 7f4a2659c640 1 -- 172.21.15.5:0/2014879950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 msgr2=0x55e505ab1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.743+0000 7f4a2659c640 1 --2- 172.21.15.5:0/2014879950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 0x55e505ab1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.743+0000 7f4a2659c640 1 -- 172.21.15.5:0/2014879950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e50577c3c0 con 0x55e505aa53a0 2024-09-19T06:03:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.743+0000 7f4a2659c640 1 --2- 172.21.15.5:0/2014879950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505ab4420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00b970 tx=0x7f4a1c0037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f88b783538b989 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.743+0000 7f4a0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.743+0000 7f4a0aff5640 1 -- 172.21.15.5:0/2014879950 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c019070 con 0x55e505aa53a0 2024-09-19T06:03:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.743+0000 7f4a0aff5640 1 -- 172.21.15.5:0/2014879950 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f4a1c004200 con 0x55e505aa53a0 2024-09-19T06:03:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.743+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2014879950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a1c0035f0 con 0x55e505aa53a0 2024-09-19T06:03:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.743+0000 7f4a0aff5640 1 -- 172.21.15.5:0/2014879950 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a1c003ba0 con 0x55e505aa53a0 2024-09-19T06:03:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.743+0000 7f4a2f53d740 10 client.15435 fetch_fsmap learned FSMap version 28 2024-09-19T06:03:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.743+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2014879950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e505ac30f0 con 0x55e505aa53a0 2024-09-19T06:03:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.743+0000 7f4a0aff5640 1 -- 172.21.15.5:0/2014879950 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a1c0035f0 con 0x55e505aa53a0 2024-09-19T06:03:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.744+0000 7f4a0aff5640 1 -- 172.21.15.5:0/2014879950 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a1c026070 con 0x55e505aa53a0 2024-09-19T06:03:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.744+0000 7f4a2f53d740 10 client.15435 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:03:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.744+0000 7f4a2f53d740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:03:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.744+0000 7f4a2f53d740 20 client.15435 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.744+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/2014879950 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e505ac4440 0x55e505ac6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.744+0000 7f4a2f53d740 4 client.15435 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.744+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2014879950 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e505a18850 con 0x55e505ac4440 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.744+0000 7f4a26d9d640 1 --2- 172.21.15.5:0/2014879950 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e505ac4440 0x55e505ac6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.745+0000 7f4a26d9d640 1 --2- 172.21.15.5:0/2014879950 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e505ac4440 0x55e505ac6830 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ea9299e0d33183 server_cookie=35986bd37d3d1fe9 in_seq=0 out_seq=0 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.745+0000 7f4a0aff5640 10 client.15435 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.745+0000 7f4a0aff5640 1 -- 172.21.15.5:0/2014879950 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e505a18850 con 0x55e505ac4440 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.745+0000 7f4a0aff5640 10 client.15435 handle_command_reply: tid=0 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.745+0000 7f4a2f53d740 1 client.15435 shutdown 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.745+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2014879950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 msgr2=0x55e505ab4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.745+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/2014879950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505ab4420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00b970 tx=0x7f4a1c0037a0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.746+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2014879950 shutdown_connections 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.746+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/2014879950 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e505ac4440 0x55e505ac6830 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.746+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/2014879950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 0x55e505ab1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.746+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/2014879950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 0x55e505ab16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.746+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/2014879950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505ab4420 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.746+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2014879950 >> 172.21.15.5:0/2014879950 conn(0x55e505aa1a30 msgr2=0x55e505aa7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:31.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.746+0000 7f4a0aff5640 0 client.15435 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:31.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.746+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2014879950 shutdown_connections 2024-09-19T06:03:31.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.746+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2014879950 wait complete. 2024-09-19T06:03:31.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.746+0000 7f4a2f53d740 20 client.15435 trim_cache size 0 max 16384 2024-09-19T06:03:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.763+0000 7f4a2f53d740 1 Processor -- start 2024-09-19T06:03:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.763+0000 7f4a2f53d740 1 -- start start 2024-09-19T06:03:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.763+0000 7f4a2f53d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505aa53a0 0x55e505ac0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.763+0000 7f4a2f53d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa5cc0 0x55e505acbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.763+0000 7f4a2f53d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505ab4a90 0x55e505acc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.763+0000 7f4a2f53d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281ab590 con 0x55e505aa5cc0 2024-09-19T06:03:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.763+0000 7f4a2f53d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281ad0d0 con 0x55e505aa53a0 2024-09-19T06:03:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.763+0000 7f4a2f53d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281abc60 con 0x55e505ab4a90 2024-09-19T06:03:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.763+0000 7f4a26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa5cc0 0x55e505acbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.763+0000 7f4a26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa5cc0 0x55e505acbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38228/0 (socket says 172.21.15.5:38228) 2024-09-19T06:03:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.763+0000 7f4a26d9d640 1 -- 172.21.15.5:0/2680079389 learned_addr learned my addr 172.21.15.5:0/2680079389 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.763+0000 7f4a2659c640 1 --2- 172.21.15.5:0/2680079389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505aa53a0 0x55e505ac0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.764+0000 7f4a2659c640 1 --2- 172.21.15.5:0/2680079389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505ab4a90 0x55e505acc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.764+0000 7f4a26d9d640 1 -- 172.21.15.5:0/2680079389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505ab4a90 msgr2=0x55e505acc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.764+0000 7f4a26d9d640 1 --2- 172.21.15.5:0/2680079389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505ab4a90 0x55e505acc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.764+0000 7f4a26d9d640 1 -- 172.21.15.5:0/2680079389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505aa53a0 msgr2=0x55e505ac0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.764+0000 7f4a26d9d640 1 --2- 172.21.15.5:0/2680079389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505aa53a0 0x55e505ac0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.764+0000 7f4a26d9d640 1 -- 172.21.15.5:0/2680079389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e505ab4790 con 0x55e505aa5cc0 2024-09-19T06:03:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.764+0000 7f4a2659c640 1 --2- 172.21.15.5:0/2680079389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505ab4a90 0x55e505acc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:03:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.764+0000 7f4a26d9d640 1 --2- 172.21.15.5:0/2680079389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa5cc0 0x55e505acbee0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55e505ab0f60 tx=0x7f4a18012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90592457d75fe90c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.764+0000 7f4a0cff9640 1 -- 172.21.15.5:0/2680079389 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18012110 con 0x55e505aa5cc0 2024-09-19T06:03:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.764+0000 7f4a0cff9640 1 -- 172.21.15.5:0/2680079389 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a180054d0 con 0x55e505aa5cc0 2024-09-19T06:03:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.764+0000 7f4a0cff9640 1 -- 172.21.15.5:0/2680079389 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18004d90 con 0x55e505aa5cc0 2024-09-19T06:03:31.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.765+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2680079389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa5cc0 msgr2=0x55e505acbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.765+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/2680079389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa5cc0 0x55e505acbee0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55e505ab0f60 tx=0x7f4a18012820 comp rx=0 tx=0).stop 2024-09-19T06:03:31.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.765+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2680079389 shutdown_connections 2024-09-19T06:03:31.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.765+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/2680079389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505ab4a90 0x55e505acc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.765+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/2680079389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa5cc0 0x55e505acbee0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.765+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/2680079389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505aa53a0 0x55e505ac0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.765+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2680079389 >> 172.21.15.5:0/2680079389 conn(0x55e505aa1a30 msgr2=0x55e505aaffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:31.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.765+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2680079389 shutdown_connections 2024-09-19T06:03:31.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.765+0000 7f4a2f53d740 1 -- 172.21.15.5:0/2680079389 wait complete. 2024-09-19T06:03:31.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.765+0000 7f4a2f53d740 1 -- start start 2024-09-19T06:03:31.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.766+0000 7f4a2f53d740 1 Processor -- start 2024-09-19T06:03:31.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.766+0000 7f4a2f53d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505abcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.766+0000 7f4a2f53d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 0x55e505ab82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.766+0000 7f4a2f53d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 0x55e505ab87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.766+0000 7f4a2f53d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281aca00 con 0x55e505aa53a0 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.766+0000 7f4a2f53d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281ac330 con 0x55e505ab4a90 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.766+0000 7f4a2f53d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281a9570 con 0x55e505aa5cc0 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.766+0000 7f4a2f53d740 10 client.0 _ll_register_callbacks cb 0x7f4a1c013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.766+0000 7f4a2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505abcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.766+0000 7f4a2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505abcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38242/0 (socket says 172.21.15.5:38242) 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.766+0000 7f4a2659c640 1 -- 172.21.15.5:0/453530306 learned_addr learned my addr 172.21.15.5:0/453530306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.766+0000 7f4a26d9d640 1 --2- 172.21.15.5:0/453530306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 0x55e505ab82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a2659c640 1 -- 172.21.15.5:0/453530306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 msgr2=0x55e505ab82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a2659c640 1 --2- 172.21.15.5:0/453530306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 0x55e505ab82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a2659c640 1 -- 172.21.15.5:0/453530306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 msgr2=0x55e505ab87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a2659c640 1 --2- 172.21.15.5:0/453530306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 0x55e505ab87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a2659c640 1 -- 172.21.15.5:0/453530306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e505ab8d30 con 0x55e505aa53a0 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a2659c640 1 --2- 172.21.15.5:0/453530306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505abcf30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00b9a0 tx=0x7f4a1c00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33a4a7362f6090b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a0aff5640 1 -- 172.21.15.5:0/453530306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c019070 con 0x55e505aa53a0 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a0aff5640 1 -- 172.21.15.5:0/453530306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f4a1c01ebd0 con 0x55e505aa53a0 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a2f53d740 1 -- 172.21.15.5:0/453530306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a180054d0 con 0x55e505aa53a0 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a0aff5640 1 -- 172.21.15.5:0/453530306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a1c040050 con 0x55e505aa53a0 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a2f53d740 10 client.15441 fetch_fsmap learned FSMap version 28 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a2f53d740 1 -- 172.21.15.5:0/453530306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e505b2eb40 con 0x55e505aa53a0 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.767+0000 7f4a0aff5640 1 -- 172.21.15.5:0/453530306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a180054d0 con 0x55e505aa53a0 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.768+0000 7f4a0aff5640 1 -- 172.21.15.5:0/453530306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a1c026040 con 0x55e505aa53a0 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.768+0000 7f4a2f53d740 10 client.15441 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.768+0000 7f4a2f53d740 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:03:31.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.768+0000 7f4a2f53d740 20 client.15441 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:31.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.768+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/453530306 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e505b2f290 0x55e505b316b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:31.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.768+0000 7f4a2f53d740 4 client.15441 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:03:31.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.768+0000 7f4a2f53d740 1 -- 172.21.15.5:0/453530306 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e505a12d60 con 0x55e505b2f290 2024-09-19T06:03:31.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.768+0000 7f4a26d9d640 1 --2- 172.21.15.5:0/453530306 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e505b2f290 0x55e505b316b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:31.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.768+0000 7f4a0aff5640 10 client.15441 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:31.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.768+0000 7f4a26d9d640 1 --2- 172.21.15.5:0/453530306 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e505b2f290 0x55e505b316b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a5a8d5414808b9e server_cookie=c40ce643c5927399 in_seq=0 out_seq=0 2024-09-19T06:03:31.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a0aff5640 1 -- 172.21.15.5:0/453530306 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e505a12d60 con 0x55e505b2f290 2024-09-19T06:03:31.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a0aff5640 10 client.15441 handle_command_reply: tid=0 2024-09-19T06:03:31.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a2f53d740 1 client.15441 shutdown 2024-09-19T06:03:31.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a2f53d740 1 -- 172.21.15.5:0/453530306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 msgr2=0x55e505abcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/453530306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505abcf30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00b9a0 tx=0x7f4a1c00bf80 comp rx=0 tx=0).stop 2024-09-19T06:03:31.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a2f53d740 1 -- 172.21.15.5:0/453530306 shutdown_connections 2024-09-19T06:03:31.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/453530306 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e505b2f290 0x55e505b316b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/453530306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e505ab4a90 0x55e505ab87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/453530306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e505aa5cc0 0x55e505ab82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a2f53d740 1 --2- 172.21.15.5:0/453530306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e505aa53a0 0x55e505abcf30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a2f53d740 1 -- 172.21.15.5:0/453530306 >> 172.21.15.5:0/453530306 conn(0x55e505aa1a30 msgr2=0x55e505ab0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:31.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a0aff5640 0 client.15441 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:31.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a2f53d740 1 -- 172.21.15.5:0/453530306 shutdown_connections 2024-09-19T06:03:31.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a2f53d740 1 -- 172.21.15.5:0/453530306 wait complete. 2024-09-19T06:03:31.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.769+0000 7f4a2f53d740 20 client.15441 trim_cache size 0 max 16384 2024-09-19T06:03:31.777 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:03:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:03:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:03:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "e9b0dcdf-5878-455d-8a55-dd76f905ecfd": { 2024-09-19T06:03:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:03:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e9b0dcdf-5878-455d-8a55-dd76f905ecfd", 2024-09-19T06:03:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:03:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:03:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:03:31.777 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:03:31.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.770+0000 7f4a2d164640 1 -- 172.21.15.5:0/1674326734 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a100a35e0 msgr2=0x7f4a100a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.770+0000 7f4a2d164640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a100a35e0 0x7f4a100a5aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a200eed40 tx=0x7f4a200a8040 comp rx=0 tx=0).stop 2024-09-19T06:03:31.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.770+0000 7f4a2d164640 1 -- 172.21.15.5:0/1674326734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a281082a0 msgr2=0x7f4a2819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.770+0000 7f4a2d164640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a281082a0 0x7f4a2819cd90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4a20005e30 tx=0x7f4a20003ba0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.770+0000 7f4a2d164640 1 -- 172.21.15.5:0/1674326734 shutdown_connections 2024-09-19T06:03:31.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.770+0000 7f4a2d164640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a281082a0 0x7f4a2819cd90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.770+0000 7f4a2d164640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a281036e0 0x7f4a28198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.770+0000 7f4a2d164640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a100a35e0 0x7f4a100a5aa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.770+0000 7f4a2d164640 1 --2- 172.21.15.5:0/1674326734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a28102d30 0x7f4a281984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.770+0000 7f4a2d164640 1 -- 172.21.15.5:0/1674326734 >> 172.21.15.5:0/1674326734 conn(0x7f4a280fc7d0 msgr2=0x7f4a280fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:31.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.770+0000 7f4a2d164640 1 -- 172.21.15.5:0/1674326734 shutdown_connections 2024-09-19T06:03:31.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.770+0000 7f4a2d164640 1 -- 172.21.15.5:0/1674326734 wait complete. 2024-09-19T06:03:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.819+0000 7efe7488b640 1 -- 172.21.15.5:0/1805005868 --> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7efe701071d0 con 0x7efe580a3440 2024-09-19T06:03:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.826+0000 7efe5f7fe640 1 -- 172.21.15.5:0/1805005868 <== mgr.15240 v2:172.21.15.5:6800/2837801016 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7efe6000aad0 con 0x7efe580a3440 2024-09-19T06:03:31.828 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2024-09-19T06:03:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.828+0000 7efe7488b640 1 -- 172.21.15.5:0/1805005868 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efe580a3440 msgr2=0x7efe580a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.828+0000 7efe7488b640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efe580a3440 0x7efe580a5900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efe60005c40 tx=0x7efe6000a570 comp rx=0 tx=0).stop 2024-09-19T06:03:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.828+0000 7efe7488b640 1 -- 172.21.15.5:0/1805005868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7010c130 msgr2=0x7efe7019cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.828+0000 7efe7488b640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7010c130 0x7efe7019cb90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efe6802a930 tx=0x7efe68003c00 comp rx=0 tx=0).stop 2024-09-19T06:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.828+0000 7efe7488b640 1 -- 172.21.15.5:0/1805005868 shutdown_connections 2024-09-19T06:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.828+0000 7efe7488b640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7010c130 0x7efe7019cb90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.828+0000 7efe7488b640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efe580a3440 0x7efe580a5900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.828+0000 7efe7488b640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe701035e0 0x7efe70198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.828+0000 7efe7488b640 1 --2- 172.21.15.5:0/1805005868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe70102c30 0x7efe701982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.828+0000 7efe7488b640 1 -- 172.21.15.5:0/1805005868 >> 172.21.15.5:0/1805005868 conn(0x7efe700fc6f0 msgr2=0x7efe700fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.828+0000 7efe7488b640 1 -- 172.21.15.5:0/1805005868 shutdown_connections 2024-09-19T06:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:03:31.828+0000 7efe7488b640 1 -- 172.21.15.5:0/1805005868 wait complete. 2024-09-19T06:03:31.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-19T06:03:31.869 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-19T06:03:31.879 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-19T06:03:31.947 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-19T06:03:31.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-19T06:03:31.951 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T06:03:31.982 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T06:03:31.982 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T06:03:31.982 INFO:tasks.workunit:timeout=3h 2024-09-19T06:03:31.982 INFO:tasks.workunit:cleanup=True 2024-09-19T06:03:31.983 DEBUG:teuthology.orchestra.run.smithi005:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T06:03:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T06:03:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-19T06:03:31.999 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-19T06:03:31.999 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:03:31.999 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-19T06:03:32.000 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-19 06:01:44.047145913 +0000 2024-09-19T06:03:32.000 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-19 06:01:44.047145913 +0000 2024-09-19T06:03:32.000 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-19 06:03:20.403383151 +0000 2024-09-19T06:03:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-09-19T06:03:32.000 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T06:03:32.000 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T06:03:32.075 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T06:03:32.131 INFO:tasks.workunit.client.0.smithi005.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T06:03:32.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:31 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:03:32.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:31 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:03:32.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:31 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:03:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:31 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:03:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:31 smithi157 ceph-mon[35346]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:03:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:31 smithi157 ceph-mon[35346]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 227 B/s wr, 19 op/s 2024-09-19T06:03:32.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:31 smithi157 ceph-mon[35346]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:03:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:31 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:03:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:31 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:03:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:31 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:03:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:31 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:03:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:31 smithi005 ceph-mon[30955]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:03:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:31 smithi005 ceph-mon[30955]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 227 B/s wr, 19 op/s 2024-09-19T06:03:32.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:31 smithi005 ceph-mon[30955]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:03:32.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:31 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:03:32.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:31 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:03:32.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:31 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:03:32.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:31 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:03:32.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:31 smithi049 ceph-mon[34640]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:03:32.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:31 smithi049 ceph-mon[34640]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 227 B/s wr, 19 op/s 2024-09-19T06:03:32.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:31 smithi049 ceph-mon[34640]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:03:32.608 INFO:tasks.cephfs.filesystem:scrub status for tag:e9b0dcdf-5878-455d-8a55-dd76f905ecfd - {'path': '/', 'tag': 'e9b0dcdf-5878-455d-8a55-dd76f905ecfd', 'options': 'recursive,force'} 2024-09-19T06:03:33.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:32 smithi157 ceph-mon[35346]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:03:33.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:32 smithi157 ceph-mon[35346]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:03:33.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:32 smithi005 ceph-mon[30955]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:03:33.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:32 smithi005 ceph-mon[30955]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:03:33.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:32 smithi049 ceph-mon[34640]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:03:33.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:32 smithi049 ceph-mon[34640]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:03:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:33 smithi157 ceph-mon[35346]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 54 op/s 2024-09-19T06:03:34.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:33 smithi005 ceph-mon[30955]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 54 op/s 2024-09-19T06:03:34.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:33 smithi049 ceph-mon[34640]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 54 op/s 2024-09-19T06:03:36.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:35 smithi157 ceph-mon[35346]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T06:03:36.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:35 smithi005 ceph-mon[30955]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T06:03:36.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:35 smithi049 ceph-mon[34640]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T06:03:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:37 smithi157 ceph-mon[35346]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T06:03:38.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:37 smithi005 ceph-mon[30955]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T06:03:38.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:37 smithi049 ceph-mon[34640]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T06:03:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:39 smithi157 ceph-mon[35346]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-19T06:03:40.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:39 smithi005 ceph-mon[30955]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-19T06:03:40.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:39 smithi049 ceph-mon[34640]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-19T06:03:42.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:41 smithi157 ceph-mon[35346]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T06:03:42.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:41 smithi005 ceph-mon[30955]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T06:03:42.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:41 smithi049 ceph-mon[34640]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T06:03:44.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:44 smithi157 ceph-mon[35346]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-09-19T06:03:44.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:03:44.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:44 smithi005 ceph-mon[30955]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-09-19T06:03:44.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:03:44.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:44 smithi049 ceph-mon[34640]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-09-19T06:03:44.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:03:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.086+0000 7fcef014b640 1 -- 172.21.15.5:0/3177510644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcee8102ca0 msgr2=0x7fcee8103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.086+0000 7fcef014b640 1 --2- 172.21.15.5:0/3177510644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcee8102ca0 0x7fcee8103080 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009900 tx=0x7fcee402a480 comp rx=0 tx=0).stop 2024-09-19T06:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.086+0000 7fcef014b640 1 -- 172.21.15.5:0/3177510644 shutdown_connections 2024-09-19T06:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.086+0000 7fcef014b640 1 --2- 172.21.15.5:0/3177510644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee810c0f0 0x7fcee810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.086+0000 7fcef014b640 1 --2- 172.21.15.5:0/3177510644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee81035c0 0x7fcee810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.086+0000 7fcef014b640 1 --2- 172.21.15.5:0/3177510644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcee8102ca0 0x7fcee8103080 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.086+0000 7fcef014b640 1 -- 172.21.15.5:0/3177510644 >> 172.21.15.5:0/3177510644 conn(0x7fcee80fc6b0 msgr2=0x7fcee80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.086+0000 7fcef014b640 1 -- 172.21.15.5:0/3177510644 shutdown_connections 2024-09-19T06:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.086+0000 7fcef014b640 1 -- 172.21.15.5:0/3177510644 wait complete. 2024-09-19T06:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fcef014b640 1 Processor -- start 2024-09-19T06:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fcef014b640 1 -- start start 2024-09-19T06:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fcef014b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8102ca0 0x7fcee8198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fcef014b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee81035c0 0x7fcee81987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fcef014b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcee810c0f0 0x7fcee819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fcef014b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcee8111d80 con 0x7fcee810c0f0 2024-09-19T06:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fcef014b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcee8111c00 con 0x7fcee8102ca0 2024-09-19T06:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fcef014b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcee8111f00 con 0x7fcee81035c0 2024-09-19T06:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fceee6e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcee810c0f0 0x7fcee819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fceee6e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcee810c0f0 0x7fcee819cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44266/0 (socket says 172.21.15.5:44266) 2024-09-19T06:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fceee6e8640 1 -- 172.21.15.5:0/3588341147 learned_addr learned my addr 172.21.15.5:0/3588341147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fceed6e6640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee81035c0 0x7fcee81987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.087+0000 7fceedee7640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8102ca0 0x7fcee8198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.088+0000 7fceee6e8640 1 -- 172.21.15.5:0/3588341147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee81035c0 msgr2=0x7fcee81987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.088+0000 7fceee6e8640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee81035c0 0x7fcee81987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.088+0000 7fceee6e8640 1 -- 172.21.15.5:0/3588341147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8102ca0 msgr2=0x7fcee8198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.088+0000 7fceee6e8640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8102ca0 0x7fcee8198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.088+0000 7fceee6e8640 1 -- 172.21.15.5:0/3588341147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee819d250 con 0x7fcee810c0f0 2024-09-19T06:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.088+0000 7fceedee7640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8102ca0 0x7fcee8198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.088+0000 7fceee6e8640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcee810c0f0 0x7fcee819cb20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcee000eaa0 tx=0x7fcee000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.088+0000 7fcedeffd640 1 -- 172.21.15.5:0/3588341147 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee000cdb0 con 0x7fcee810c0f0 2024-09-19T06:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.088+0000 7fcedeffd640 1 -- 172.21.15.5:0/3588341147 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcee0005020 con 0x7fcee810c0f0 2024-09-19T06:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.088+0000 7fcef014b640 1 -- 172.21.15.5:0/3588341147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee819d540 con 0x7fcee810c0f0 2024-09-19T06:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.088+0000 7fcedeffd640 1 -- 172.21.15.5:0/3588341147 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee00116d0 con 0x7fcee810c0f0 2024-09-19T06:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.088+0000 7fcef014b640 1 -- 172.21.15.5:0/3588341147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee81a4dd0 con 0x7fcee810c0f0 2024-09-19T06:03:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fcedeffd640 1 -- 172.21.15.5:0/3588341147 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7fcee00051d0 con 0x7fcee810c0f0 2024-09-19T06:03:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fcef2fc6740 1 Processor -- start 2024-09-19T06:03:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fcef2fc6740 1 -- start start 2024-09-19T06:03:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fcef2fc6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 0x55ed64884ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fcef2fc6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64885410 0x55ed6488f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fcef2fc6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed6488fcb0 0x55ed648920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fcef2fc6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ed647e92f0 con 0x55ed64884af0 2024-09-19T06:03:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fcef2fc6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ed64791270 con 0x55ed64885410 2024-09-19T06:03:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fcef2fc6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ed647e52b0 con 0x55ed6488fcb0 2024-09-19T06:03:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fceed6e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 0x55ed64884ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fceed6e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 0x55ed64884ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44274/0 (socket says 172.21.15.5:44274) 2024-09-19T06:03:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fceed6e6640 1 -- 172.21.15.5:0/1596186889 learned_addr learned my addr 172.21.15.5:0/1596186889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.091+0000 7fceedee7640 1 --2- 172.21.15.5:0/1596186889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64885410 0x55ed6488f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fceed6e6640 1 --2- 172.21.15.5:0/1596186889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed6488fcb0 0x55ed648920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fcedeffd640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fced40a34c0 0x7fced40a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fceed6e6640 1 -- 172.21.15.5:0/1596186889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed6488fcb0 msgr2=0x55ed648920a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fceed6e6640 1 --2- 172.21.15.5:0/1596186889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed6488fcb0 0x55ed648920a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fceed6e6640 1 -- 172.21.15.5:0/1596186889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64885410 msgr2=0x55ed6488f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fceed6e6640 1 --2- 172.21.15.5:0/1596186889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64885410 0x55ed6488f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fceed6e6640 1 -- 172.21.15.5:0/1596186889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed64790340 con 0x55ed64884af0 2024-09-19T06:03:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fceedee7640 1 --2- 172.21.15.5:0/1596186889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64885410 0x55ed6488f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fceee6e8640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fced40a34c0 0x7fced40a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fcedeffd640 1 -- 172.21.15.5:0/3588341147 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fcee00cffa0 con 0x7fcee810c0f0 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fceed6e6640 1 --2- 172.21.15.5:0/1596186889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 0x55ed64884ed0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fced800afc0 tx=0x7fced800a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499c2cdb321a5a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fced3fff640 1 -- 172.21.15.5:0/1596186889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8019070 con 0x55ed64884af0 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fced3fff640 1 -- 172.21.15.5:0/1596186889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fced8011de0 con 0x55ed64884af0 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.092+0000 7fced3fff640 1 -- 172.21.15.5:0/1596186889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced801ed40 con 0x55ed64884af0 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.093+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1596186889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 msgr2=0x55ed64884ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.093+0000 7fceee6e8640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fced40a34c0 0x7fced40a5980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcee00cd6a0 tx=0x7fcee00bc040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.093+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/1596186889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 0x55ed64884ed0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fced800afc0 tx=0x7fced800a800 comp rx=0 tx=0).stop 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.093+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1596186889 shutdown_connections 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.093+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/1596186889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed6488fcb0 0x55ed648920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.093+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/1596186889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64885410 0x55ed6488f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.093+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/1596186889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 0x55ed64884ed0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.093+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1596186889 >> 172.21.15.5:0/1596186889 conn(0x55ed64881060 msgr2=0x55ed64881440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.093+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1596186889 shutdown_connections 2024-09-19T06:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.093+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1596186889 wait complete. 2024-09-19T06:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.093+0000 7fcef2fc6740 1 -- start start 2024-09-19T06:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fcef2fc6740 1 Processor -- start 2024-09-19T06:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fcef2fc6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64884af0 0x55ed6488cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fcef2fc6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed64885410 0x55ed6488d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fcef2fc6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 0x55ed6488d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fcef2fc6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ed64892880 con 0x55ed6488fcb0 2024-09-19T06:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fcef2fc6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ed64772290 con 0x55ed64884af0 2024-09-19T06:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fcef2fc6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ed64892a00 con 0x55ed64885410 2024-09-19T06:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fcef2fc6740 10 client.0 _ll_register_callbacks cb 0x55ed647a3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fceed6e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 0x55ed6488d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fceed6e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 0x55ed6488d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44278/0 (socket says 172.21.15.5:44278) 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fceed6e6640 1 -- 172.21.15.5:0/1601775496 learned_addr learned my addr 172.21.15.5:0/1601775496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fceedee7640 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed64885410 0x55ed6488d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fceed6e6640 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64884af0 0x55ed6488cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fceed6e6640 1 -- 172.21.15.5:0/1601775496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed64885410 msgr2=0x55ed6488d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fceed6e6640 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed64885410 0x55ed6488d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fceed6e6640 1 -- 172.21.15.5:0/1601775496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64884af0 msgr2=0x55ed6488cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fceed6e6640 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64884af0 0x55ed6488cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.094+0000 7fceed6e6640 1 -- 172.21.15.5:0/1601775496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed644fc210 con 0x55ed6488fcb0 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fceed6e6640 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 0x55ed6488d610 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fced800a2d0 tx=0x7fced8022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5831ed8f8b007b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fced17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fced17fa640 1 -- 172.21.15.5:0/1601775496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8019040 con 0x55ed6488fcb0 2024-09-19T06:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fced17fa640 1 -- 172.21.15.5:0/1601775496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fced8011730 con 0x55ed6488fcb0 2024-09-19T06:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1601775496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fced8011de0 con 0x55ed6488fcb0 2024-09-19T06:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fced17fa640 1 -- 172.21.15.5:0/1601775496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fced8011930 con 0x55ed6488fcb0 2024-09-19T06:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fcef2fc6740 10 client.15453 fetch_fsmap learned FSMap version 28 2024-09-19T06:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1601775496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed645c8620 con 0x55ed6488fcb0 2024-09-19T06:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fced17fa640 1 -- 172.21.15.5:0/1601775496 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fced8011de0 con 0x55ed6488fcb0 2024-09-19T06:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fced17fa640 1 -- 172.21.15.5:0/1601775496 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fced8004880 con 0x55ed6488fcb0 2024-09-19T06:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fcef2fc6740 10 client.15453 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fcef2fc6740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fcef2fc6740 20 client.15453 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed6489fb40 0x55ed648a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fcef2fc6740 4 client.15453 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.095+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1601775496 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed647f6a20 con 0x55ed6489fb40 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.096+0000 7fceedee7640 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed6489fb40 0x55ed648a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.096+0000 7fceedee7640 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed6489fb40 0x55ed648a1f30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd6915b626bac6f server_cookie=bb9e69554bb400db in_seq=0 out_seq=0 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.096+0000 7fced17fa640 10 client.15453 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fced17fa640 1 -- 172.21.15.5:0/1601775496 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed647f6a20 con 0x55ed6489fb40 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fced17fa640 10 client.15453 handle_command_reply: tid=0 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fcef2fc6740 1 client.15453 shutdown 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1601775496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 msgr2=0x55ed6488d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 0x55ed6488d610 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fced800a2d0 tx=0x7fced8022040 comp rx=0 tx=0).stop 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1601775496 shutdown_connections 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed6489fb40 0x55ed648a1f30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 0x55ed6488d610 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed64885410 0x55ed6488d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fced17fa640 0 client.15453 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/1601775496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64884af0 0x55ed6488cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1601775496 >> 172.21.15.5:0/1601775496 conn(0x55ed64881060 msgr2=0x55ed64886dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1601775496 shutdown_connections 2024-09-19T06:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fcef2fc6740 1 -- 172.21.15.5:0/1601775496 wait complete. 2024-09-19T06:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.097+0000 7fcef2fc6740 20 client.15453 trim_cache size 0 max 16384 2024-09-19T06:03:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.111+0000 7fcef2fc6740 1 Processor -- start 2024-09-19T06:03:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fcef2fc6740 1 -- start start 2024-09-19T06:03:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fcef2fc6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64884af0 0x55ed6488d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fcef2fc6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed64885410 0x55ed64886c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fcef2fc6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 0x55ed648a7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fcef2fc6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcee81ad790 con 0x55ed6488fcb0 2024-09-19T06:03:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fcef2fc6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcee81af2d0 con 0x55ed64884af0 2024-09-19T06:03:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fcef2fc6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcee81ade60 con 0x55ed64885410 2024-09-19T06:03:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fceed6e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 0x55ed648a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fceed6e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 0x55ed648a7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44290/0 (socket says 172.21.15.5:44290) 2024-09-19T06:03:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fceed6e6640 1 -- 172.21.15.5:0/3161897358 learned_addr learned my addr 172.21.15.5:0/3161897358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fceed6e6640 1 --2- 172.21.15.5:0/3161897358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64884af0 0x55ed6488d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fceedee7640 1 --2- 172.21.15.5:0/3161897358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed64885410 0x55ed64886c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fceed6e6640 1 -- 172.21.15.5:0/3161897358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed64885410 msgr2=0x55ed64886c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fceed6e6640 1 --2- 172.21.15.5:0/3161897358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed64885410 0x55ed64886c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fceed6e6640 1 -- 172.21.15.5:0/3161897358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64884af0 msgr2=0x55ed6488d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fceed6e6640 1 --2- 172.21.15.5:0/3161897358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64884af0 0x55ed6488d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fceed6e6640 1 -- 172.21.15.5:0/3161897358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed64885240 con 0x55ed6488fcb0 2024-09-19T06:03:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fceed6e6640 1 --2- 172.21.15.5:0/3161897358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 0x55ed648a7620 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fced8013550 tx=0x7fced801e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db47e8772ee56fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fced37fe640 1 -- 172.21.15.5:0/3161897358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8019070 con 0x55ed6488fcb0 2024-09-19T06:03:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fced37fe640 1 -- 172.21.15.5:0/3161897358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fced800ae50 con 0x55ed6488fcb0 2024-09-19T06:03:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.112+0000 7fced37fe640 1 -- 172.21.15.5:0/3161897358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8036040 con 0x55ed6488fcb0 2024-09-19T06:03:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.113+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3161897358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 msgr2=0x55ed648a7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.113+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/3161897358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 0x55ed648a7620 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fced8013550 tx=0x7fced801e9a0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.113+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3161897358 shutdown_connections 2024-09-19T06:03:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.113+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/3161897358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed6488fcb0 0x55ed648a7620 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.113+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/3161897358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed64885410 0x55ed64886c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.113+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/3161897358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64884af0 0x55ed6488d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.113+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3161897358 >> 172.21.15.5:0/3161897358 conn(0x55ed64881060 msgr2=0x55ed64886550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.113+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3161897358 shutdown_connections 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.113+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3161897358 wait complete. 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.113+0000 7fcef2fc6740 1 -- start start 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.113+0000 7fcef2fc6740 1 Processor -- start 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fcef2fc6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 0x55ed64897f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fcef2fc6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64885410 0x55ed648984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fcef2fc6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed6488fcb0 0x55ed64893360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fcef2fc6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcee81aec00 con 0x55ed64884af0 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fcef2fc6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcee81ae530 con 0x55ed64885410 2024-09-19T06:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fcef2fc6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcee81ab770 con 0x55ed6488fcb0 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fcef2fc6740 10 client.0 _ll_register_callbacks cb 0x7fcee4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fceed6e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 0x55ed64897f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fceed6e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 0x55ed64897f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44306/0 (socket says 172.21.15.5:44306) 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fceed6e6640 1 -- 172.21.15.5:0/3495149428 learned_addr learned my addr 172.21.15.5:0/3495149428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fceedee7640 1 --2- 172.21.15.5:0/3495149428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64885410 0x55ed648984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fceed6e6640 1 -- 172.21.15.5:0/3495149428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed6488fcb0 msgr2=0x55ed64893360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fceed6e6640 1 --2- 172.21.15.5:0/3495149428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed6488fcb0 0x55ed64893360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fceed6e6640 1 -- 172.21.15.5:0/3495149428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64885410 msgr2=0x55ed648984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fceed6e6640 1 --2- 172.21.15.5:0/3495149428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64885410 0x55ed648984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fceed6e6640 1 -- 172.21.15.5:0/3495149428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed64893b60 con 0x55ed64884af0 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fceed6e6640 1 --2- 172.21.15.5:0/3495149428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 0x55ed64897f90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fced800a2d0 tx=0x7fced8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768f284706ffbb61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.114+0000 7fced17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fced17fa640 1 -- 172.21.15.5:0/3495149428 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8023070 con 0x55ed64884af0 2024-09-19T06:03:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fced17fa640 1 -- 172.21.15.5:0/3495149428 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fced8011730 con 0x55ed64884af0 2024-09-19T06:03:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3495149428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fced800ae50 con 0x55ed64884af0 2024-09-19T06:03:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fced17fa640 1 -- 172.21.15.5:0/3495149428 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fced8049050 con 0x55ed64884af0 2024-09-19T06:03:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fcef2fc6740 10 client.15459 fetch_fsmap learned FSMap version 28 2024-09-19T06:03:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3495149428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed6490a1a0 con 0x55ed64884af0 2024-09-19T06:03:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fced17fa640 1 -- 172.21.15.5:0/3495149428 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fced800ae50 con 0x55ed64884af0 2024-09-19T06:03:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fced17fa640 1 -- 172.21.15.5:0/3495149428 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fced802a680 con 0x55ed64884af0 2024-09-19T06:03:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fcef2fc6740 10 client.15459 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:03:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fcef2fc6740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:03:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fcef2fc6740 20 client.15459 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/3495149428 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed6490a750 0x55ed6490cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fcef2fc6740 4 client.15459 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:03:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3495149428 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed647f0d70 con 0x55ed6490a750 2024-09-19T06:03:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.115+0000 7fceedee7640 1 --2- 172.21.15.5:0/3495149428 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed6490a750 0x55ed6490cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.116+0000 7fceedee7640 1 --2- 172.21.15.5:0/3495149428 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed6490a750 0x55ed6490cb40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb10a7c718ae633e server_cookie=24cdf7b6094c65a9 in_seq=0 out_seq=0 2024-09-19T06:03:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.116+0000 7fced17fa640 10 client.15459 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.127+0000 7fced17fa640 1 -- 172.21.15.5:0/3495149428 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed647f0d70 con 0x55ed6490a750 2024-09-19T06:03:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.127+0000 7fced17fa640 10 client.15459 handle_command_reply: tid=0 2024-09-19T06:03:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.127+0000 7fcef2fc6740 1 client.15459 shutdown 2024-09-19T06:03:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3495149428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 msgr2=0x55ed64897f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/3495149428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 0x55ed64897f90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fced800a2d0 tx=0x7fced8005850 comp rx=0 tx=0).stop 2024-09-19T06:03:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3495149428 shutdown_connections 2024-09-19T06:03:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/3495149428 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed6490a750 0x55ed6490cb40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/3495149428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed6488fcb0 0x55ed64893360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/3495149428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed64885410 0x55ed648984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fced17fa640 0 client.15459 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef2fc6740 1 --2- 172.21.15.5:0/3495149428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed64884af0 0x55ed64897f90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3495149428 >> 172.21.15.5:0/3495149428 conn(0x55ed64881060 msgr2=0x55ed6488e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3495149428 shutdown_connections 2024-09-19T06:03:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef2fc6740 1 -- 172.21.15.5:0/3495149428 wait complete. 2024-09-19T06:03:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef2fc6740 20 client.15459 trim_cache size 0 max 16384 2024-09-19T06:03:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:03:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:03:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:03:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:03:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef014b640 1 -- 172.21.15.5:0/3588341147 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fced40a34c0 msgr2=0x7fced40a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef014b640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fced40a34c0 0x7fced40a5980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcee00cd6a0 tx=0x7fcee00bc040 comp rx=0 tx=0).stop 2024-09-19T06:03:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef014b640 1 -- 172.21.15.5:0/3588341147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcee810c0f0 msgr2=0x7fcee819cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.128+0000 7fcef014b640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcee810c0f0 0x7fcee819cb20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcee000eaa0 tx=0x7fcee000ef60 comp rx=0 tx=0).stop 2024-09-19T06:03:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.129+0000 7fcef014b640 1 -- 172.21.15.5:0/3588341147 shutdown_connections 2024-09-19T06:03:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.129+0000 7fcef014b640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcee810c0f0 0x7fcee819cb20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.129+0000 7fcef014b640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fced40a34c0 0x7fced40a5980 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.129+0000 7fcef014b640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee81035c0 0x7fcee81987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.129+0000 7fcef014b640 1 --2- 172.21.15.5:0/3588341147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8102ca0 0x7fcee8198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.129+0000 7fcef014b640 1 -- 172.21.15.5:0/3588341147 >> 172.21.15.5:0/3588341147 conn(0x7fcee80fc6b0 msgr2=0x7fcee80fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.129+0000 7fcef014b640 1 -- 172.21.15.5:0/3588341147 shutdown_connections 2024-09-19T06:03:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:45.129+0000 7fcef014b640 1 -- 172.21.15.5:0/3588341147 wait complete. 2024-09-19T06:03:46.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:46 smithi157 ceph-mon[35346]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T06:03:46.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:46 smithi005 ceph-mon[30955]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T06:03:46.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:46 smithi049 ceph-mon[34640]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T06:03:48.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:48 smithi157 ceph-mon[35346]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T06:03:48.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:48 smithi005 ceph-mon[30955]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T06:03:48.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:48 smithi049 ceph-mon[34640]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T06:03:50.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:50 smithi157 ceph-mon[35346]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.7 KiB/s wr, 18 op/s 2024-09-19T06:03:50.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:50 smithi005 ceph-mon[30955]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.7 KiB/s wr, 18 op/s 2024-09-19T06:03:50.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:50 smithi049 ceph-mon[34640]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.7 KiB/s wr, 18 op/s 2024-09-19T06:03:52.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:52 smithi157 ceph-mon[35346]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-19T06:03:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:52 smithi005 ceph-mon[30955]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-19T06:03:52.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:52 smithi049 ceph-mon[34640]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-19T06:03:54.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:54 smithi157 ceph-mon[35346]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 17 op/s 2024-09-19T06:03:54.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:54 smithi005 ceph-mon[30955]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 17 op/s 2024-09-19T06:03:54.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:54 smithi049 ceph-mon[34640]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 17 op/s 2024-09-19T06:03:56.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:56 smithi157 ceph-mon[35346]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T06:03:56.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:56 smithi005 ceph-mon[30955]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T06:03:56.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:56 smithi049 ceph-mon[34640]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T06:03:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.221+0000 7f795bd6d640 1 -- 172.21.15.5:0/631932711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795410c0a0 msgr2=0x7f795410e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.221+0000 7f795bd6d640 1 --2- 172.21.15.5:0/631932711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795410c0a0 0x7f795410e540 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f794c009ca0 tx=0x7f794c02a950 comp rx=0 tx=0).stop 2024-09-19T06:03:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.221+0000 7f795bd6d640 1 -- 172.21.15.5:0/631932711 shutdown_connections 2024-09-19T06:03:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.221+0000 7f795bd6d640 1 --2- 172.21.15.5:0/631932711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795410c0a0 0x7f795410e540 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.221+0000 7f795bd6d640 1 --2- 172.21.15.5:0/631932711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79541035a0 0x7f795410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.221+0000 7f795bd6d640 1 --2- 172.21.15.5:0/631932711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7954102c80 0x7f7954103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.221+0000 7f795bd6d640 1 -- 172.21.15.5:0/631932711 >> 172.21.15.5:0/631932711 conn(0x7f79540779a0 msgr2=0x7f79540fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.221+0000 7f795bd6d640 1 -- 172.21.15.5:0/631932711 shutdown_connections 2024-09-19T06:03:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.221+0000 7f795bd6d640 1 -- 172.21.15.5:0/631932711 wait complete. 2024-09-19T06:03:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.221+0000 7f795bd6d640 1 Processor -- start 2024-09-19T06:03:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f795bd6d640 1 -- start start 2024-09-19T06:03:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f795bd6d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7954102c80 0x7f79541008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f795bd6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79541035a0 0x7f7954100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f795bd6d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f795410c0a0 0x7f7954101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f795bd6d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7954111da0 con 0x7f79541035a0 2024-09-19T06:03:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f795bd6d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7954111c20 con 0x7f7954102c80 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f795bd6d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7954111f20 con 0x7f795410c0a0 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f7959308640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79541035a0 0x7f7954100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f7959308640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79541035a0 0x7f7954100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50678/0 (socket says 172.21.15.5:50678) 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f7959308640 1 -- 172.21.15.5:0/1885204238 learned_addr learned my addr 172.21.15.5:0/1885204238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f795a30a640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f795410c0a0 0x7f7954101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f7959b09640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7954102c80 0x7f79541008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f7959308640 1 -- 172.21.15.5:0/1885204238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f795410c0a0 msgr2=0x7f7954101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f7959308640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f795410c0a0 0x7f7954101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f7959308640 1 -- 172.21.15.5:0/1885204238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7954102c80 msgr2=0x7f79541008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f7959308640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7954102c80 0x7f79541008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f7959308640 1 -- 172.21.15.5:0/1885204238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79541a66d0 con 0x7f79541035a0 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.222+0000 7f7959308640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79541035a0 0x7f7954100e10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f794400c920 tx=0x7f794400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.223+0000 7f794affd640 1 -- 172.21.15.5:0/1885204238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944007c70 con 0x7f79541035a0 2024-09-19T06:03:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.223+0000 7f794affd640 1 -- 172.21.15.5:0/1885204238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7944007e10 con 0x7f79541035a0 2024-09-19T06:03:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.223+0000 7f794affd640 1 -- 172.21.15.5:0/1885204238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944018820 con 0x7f79541035a0 2024-09-19T06:03:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.223+0000 7f795bd6d640 1 -- 172.21.15.5:0/1885204238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79541a69c0 con 0x7f79541035a0 2024-09-19T06:03:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.223+0000 7f795bd6d640 1 -- 172.21.15.5:0/1885204238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79541a6f60 con 0x7f79541035a0 2024-09-19T06:03:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.225+0000 7f794affd640 1 -- 172.21.15.5:0/1885204238 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138530+0+0 (secure 0 0 0) 0x7f79440189c0 con 0x7f79541035a0 2024-09-19T06:03:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.225+0000 7f795ebea740 1 Processor -- start 2024-09-19T06:03:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795ebea740 1 -- start start 2024-09-19T06:03:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795ebea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b4af0 0x558be93b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795ebea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b5410 0x558be93bf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795ebea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93bfcb0 0x558be93c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795ebea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558be93192f0 con 0x558be93b4af0 2024-09-19T06:03:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795ebea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558be92c1270 con 0x558be93b5410 2024-09-19T06:03:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795ebea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558be93152b0 con 0x558be93bfcb0 2024-09-19T06:03:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795a30a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b4af0 0x558be93b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795a30a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b4af0 0x558be93b4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50686/0 (socket says 172.21.15.5:50686) 2024-09-19T06:03:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795a30a640 1 -- 172.21.15.5:0/4260431433 learned_addr learned my addr 172.21.15.5:0/4260431433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f7959b09640 1 --2- 172.21.15.5:0/4260431433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b5410 0x558be93bf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f7959308640 1 --2- 172.21.15.5:0/4260431433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93bfcb0 0x558be93c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f794affd640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f79400a34c0 0x7f79400a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795a30a640 1 -- 172.21.15.5:0/4260431433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93bfcb0 msgr2=0x558be93c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795a30a640 1 --2- 172.21.15.5:0/4260431433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93bfcb0 0x558be93c20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795a30a640 1 -- 172.21.15.5:0/4260431433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b5410 msgr2=0x558be93bf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795a30a640 1 --2- 172.21.15.5:0/4260431433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b5410 0x558be93bf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.226+0000 7f795a30a640 1 -- 172.21.15.5:0/4260431433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be92c0340 con 0x558be93b4af0 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.227+0000 7f794affd640 1 -- 172.21.15.5:0/1885204238 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f79440cff30 con 0x7f79541035a0 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.227+0000 7f795a30a640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f79400a34c0 0x7f79400a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.227+0000 7f795a30a640 1 --2- 172.21.15.5:0/4260431433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b4af0 0x558be93b4ed0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f794c009ca0 tx=0x7f794c02f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b67a3a074b0cba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.227+0000 7f793ffff640 1 -- 172.21.15.5:0/4260431433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c034040 con 0x558be93b4af0 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.227+0000 7f793ffff640 1 -- 172.21.15.5:0/4260431433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f794c02fec0 con 0x558be93b4af0 2024-09-19T06:03:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.227+0000 7f793ffff640 1 -- 172.21.15.5:0/4260431433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c0467b0 con 0x558be93b4af0 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.228+0000 7f795ebea740 1 -- 172.21.15.5:0/4260431433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b4af0 msgr2=0x558be93b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.228+0000 7f795a30a640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f79400a34c0 0x7f79400a5980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f794c04d000 tx=0x7f794c04a000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.228+0000 7f795ebea740 1 --2- 172.21.15.5:0/4260431433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b4af0 0x558be93b4ed0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f794c009ca0 tx=0x7f794c02f720 comp rx=0 tx=0).stop 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.228+0000 7f795ebea740 1 -- 172.21.15.5:0/4260431433 shutdown_connections 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.228+0000 7f795ebea740 1 --2- 172.21.15.5:0/4260431433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93bfcb0 0x558be93c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.228+0000 7f795ebea740 1 --2- 172.21.15.5:0/4260431433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b5410 0x558be93bf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.228+0000 7f795ebea740 1 --2- 172.21.15.5:0/4260431433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b4af0 0x558be93b4ed0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.228+0000 7f795ebea740 1 -- 172.21.15.5:0/4260431433 >> 172.21.15.5:0/4260431433 conn(0x558be93b1060 msgr2=0x558be93b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.228+0000 7f795ebea740 1 -- 172.21.15.5:0/4260431433 shutdown_connections 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.228+0000 7f795ebea740 1 -- 172.21.15.5:0/4260431433 wait complete. 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.228+0000 7f795ebea740 1 -- start start 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.228+0000 7f795ebea740 1 Processor -- start 2024-09-19T06:03:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795ebea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b4af0 0x558be93bcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795ebea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93b5410 0x558be93bd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795ebea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93bfcb0 0x558be93bd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795ebea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558be93c2880 con 0x558be93bfcb0 2024-09-19T06:03:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795ebea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558be92a2290 con 0x558be93b4af0 2024-09-19T06:03:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795ebea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558be93c2a00 con 0x558be93b5410 2024-09-19T06:03:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795ebea740 10 client.0 _ll_register_callbacks cb 0x558be92d3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795a30a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93bfcb0 0x558be93bd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795a30a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93bfcb0 0x558be93bd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50688/0 (socket says 172.21.15.5:50688) 2024-09-19T06:03:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795a30a640 1 -- 172.21.15.5:0/234253464 learned_addr learned my addr 172.21.15.5:0/234253464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f7959b09640 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b4af0 0x558be93bcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f7959308640 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93b5410 0x558be93bd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795a30a640 1 -- 172.21.15.5:0/234253464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93b5410 msgr2=0x558be93bd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795a30a640 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93b5410 0x558be93bd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795a30a640 1 -- 172.21.15.5:0/234253464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b4af0 msgr2=0x558be93bcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795a30a640 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b4af0 0x558be93bcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795a30a640 1 -- 172.21.15.5:0/234253464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be902c210 con 0x558be93bfcb0 2024-09-19T06:03:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f7959b09640 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b4af0 0x558be93bcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:03:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f795a30a640 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93bfcb0 0x558be93bd5e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f794c005dc0 tx=0x7f794c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a215134122f4873c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.229+0000 7f793d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f793d7fa640 1 -- 172.21.15.5:0/234253464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c004040 con 0x558be93bfcb0 2024-09-19T06:03:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f793d7fa640 1 -- 172.21.15.5:0/234253464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f794c0041e0 con 0x558be93bfcb0 2024-09-19T06:03:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f795ebea740 1 -- 172.21.15.5:0/234253464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f794c02fec0 con 0x558be93bfcb0 2024-09-19T06:03:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f793d7fa640 1 -- 172.21.15.5:0/234253464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f794c063050 con 0x558be93bfcb0 2024-09-19T06:03:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f795ebea740 10 client.15471 fetch_fsmap learned FSMap version 28 2024-09-19T06:03:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f795ebea740 1 -- 172.21.15.5:0/234253464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558be90f8620 con 0x558be93bfcb0 2024-09-19T06:03:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f793d7fa640 1 -- 172.21.15.5:0/234253464 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f794c02fec0 con 0x558be93bfcb0 2024-09-19T06:03:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f793d7fa640 1 -- 172.21.15.5:0/234253464 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f794c051a90 con 0x558be93bfcb0 2024-09-19T06:03:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f795ebea740 10 client.15471 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:03:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f795ebea740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:03:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f795ebea740 20 client.15471 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f795ebea740 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558be93cfaa0 0x558be93d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f795ebea740 4 client.15471 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:03:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.230+0000 7f795ebea740 1 -- 172.21.15.5:0/234253464 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558be9326a20 con 0x558be93cfaa0 2024-09-19T06:03:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.231+0000 7f7959b09640 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558be93cfaa0 0x558be93d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.231+0000 7f7959b09640 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558be93cfaa0 0x558be93d1ed0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43c09b882544d968 server_cookie=64a63810a6be55c1 in_seq=0 out_seq=0 2024-09-19T06:03:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.231+0000 7f793d7fa640 10 client.15471 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f793d7fa640 1 -- 172.21.15.5:0/234253464 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558be9326a20 con 0x558be93cfaa0 2024-09-19T06:03:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f793d7fa640 10 client.15471 handle_command_reply: tid=0 2024-09-19T06:03:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f795ebea740 1 client.15471 shutdown 2024-09-19T06:03:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f795ebea740 1 -- 172.21.15.5:0/234253464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93bfcb0 msgr2=0x558be93bd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f795ebea740 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93bfcb0 0x558be93bd5e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f794c005dc0 tx=0x7f794c009800 comp rx=0 tx=0).stop 2024-09-19T06:03:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f795ebea740 1 -- 172.21.15.5:0/234253464 shutdown_connections 2024-09-19T06:03:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f795ebea740 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558be93cfaa0 0x558be93d1ed0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f795ebea740 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93bfcb0 0x558be93bd5e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f795ebea740 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93b5410 0x558be93bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f795ebea740 1 --2- 172.21.15.5:0/234253464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b4af0 0x558be93bcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f793d7fa640 0 client.15471 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f795ebea740 1 -- 172.21.15.5:0/234253464 >> 172.21.15.5:0/234253464 conn(0x558be93b1060 msgr2=0x558be93b6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f795ebea740 1 -- 172.21.15.5:0/234253464 shutdown_connections 2024-09-19T06:03:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f795ebea740 1 -- 172.21.15.5:0/234253464 wait complete. 2024-09-19T06:03:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.232+0000 7f795ebea740 20 client.15471 trim_cache size 0 max 16384 2024-09-19T06:03:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.251+0000 7f795ebea740 1 Processor -- start 2024-09-19T06:03:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.251+0000 7f795ebea740 1 -- start start 2024-09-19T06:03:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.251+0000 7f795ebea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93b4af0 0x558be93b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.251+0000 7f795ebea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 0x558be93bd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.251+0000 7f795ebea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93bfcb0 0x558be93d77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.251+0000 7f795ebea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79541af7a0 con 0x558be93b5410 2024-09-19T06:03:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.251+0000 7f795ebea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b12e0 con 0x558be93bfcb0 2024-09-19T06:03:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.251+0000 7f795ebea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f79541afe70 con 0x558be93b4af0 2024-09-19T06:03:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.251+0000 7f7959308640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 0x558be93bd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.251+0000 7f7959b09640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93b4af0 0x558be93b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.251+0000 7f7959308640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 0x558be93bd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50692/0 (socket says 172.21.15.5:50692) 2024-09-19T06:03:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.251+0000 7f7959308640 1 -- 172.21.15.5:0/3976678793 learned_addr learned my addr 172.21.15.5:0/3976678793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.252+0000 7f795a30a640 1 --2- 172.21.15.5:0/3976678793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93bfcb0 0x558be93d77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.252+0000 7f7959308640 1 -- 172.21.15.5:0/3976678793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93b4af0 msgr2=0x558be93b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.252+0000 7f7959308640 1 --2- 172.21.15.5:0/3976678793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93b4af0 0x558be93b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.252+0000 7f7959308640 1 -- 172.21.15.5:0/3976678793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93bfcb0 msgr2=0x558be93d77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.252+0000 7f7959308640 1 --2- 172.21.15.5:0/3976678793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93bfcb0 0x558be93d77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.252+0000 7f7959308640 1 -- 172.21.15.5:0/3976678793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be93b2110 con 0x558be93b5410 2024-09-19T06:03:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.252+0000 7f795a30a640 1 --2- 172.21.15.5:0/3976678793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93bfcb0 0x558be93d77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:03:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.252+0000 7f7959b09640 1 --2- 172.21.15.5:0/3976678793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93b4af0 0x558be93b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:03:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.252+0000 7f7959308640 1 --2- 172.21.15.5:0/3976678793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 0x558be93bd3e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f794409bdb0 tx=0x7f7944067040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d617631cda7de97a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.252+0000 7f793f7fe640 1 -- 172.21.15.5:0/3976678793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944018cb0 con 0x558be93b5410 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.252+0000 7f793f7fe640 1 -- 172.21.15.5:0/3976678793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f794400fb00 con 0x558be93b5410 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.252+0000 7f793f7fe640 1 -- 172.21.15.5:0/3976678793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794409a440 con 0x558be93b5410 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.253+0000 7f795ebea740 1 -- 172.21.15.5:0/3976678793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 msgr2=0x558be93bd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.253+0000 7f795ebea740 1 --2- 172.21.15.5:0/3976678793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 0x558be93bd3e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f794409bdb0 tx=0x7f7944067040 comp rx=0 tx=0).stop 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.253+0000 7f795ebea740 1 -- 172.21.15.5:0/3976678793 shutdown_connections 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.253+0000 7f795ebea740 1 --2- 172.21.15.5:0/3976678793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93bfcb0 0x558be93d77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.253+0000 7f795ebea740 1 --2- 172.21.15.5:0/3976678793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 0x558be93bd3e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.253+0000 7f795ebea740 1 --2- 172.21.15.5:0/3976678793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93b4af0 0x558be93b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.253+0000 7f795ebea740 1 -- 172.21.15.5:0/3976678793 >> 172.21.15.5:0/3976678793 conn(0x558be93b1060 msgr2=0x558be93bebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.253+0000 7f795ebea740 1 -- 172.21.15.5:0/3976678793 shutdown_connections 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.253+0000 7f795ebea740 1 -- 172.21.15.5:0/3976678793 wait complete. 2024-09-19T06:03:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.253+0000 7f795ebea740 1 -- start start 2024-09-19T06:03:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.254+0000 7f795ebea740 1 Processor -- start 2024-09-19T06:03:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.254+0000 7f795ebea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b4af0 0x558be93c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.254+0000 7f795ebea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 0x558be93c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.254+0000 7f795ebea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93bfcb0 0x558be93c3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.254+0000 7f795ebea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b0c10 con 0x558be93b5410 2024-09-19T06:03:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.254+0000 7f795ebea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b0540 con 0x558be93b4af0 2024-09-19T06:03:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.254+0000 7f795ebea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f79541ad780 con 0x558be93bfcb0 2024-09-19T06:03:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.254+0000 7f795ebea740 10 client.0 _ll_register_callbacks cb 0x7f795000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:03:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.254+0000 7f7959308640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 0x558be93c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.254+0000 7f7959308640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 0x558be93c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50704/0 (socket says 172.21.15.5:50704) 2024-09-19T06:03:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.254+0000 7f7959308640 1 -- 172.21.15.5:0/728560256 learned_addr learned my addr 172.21.15.5:0/728560256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:03:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.254+0000 7f795a30a640 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93bfcb0 0x558be93c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f7959b09640 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b4af0 0x558be93c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f7959308640 1 -- 172.21.15.5:0/728560256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93bfcb0 msgr2=0x558be93c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f7959308640 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93bfcb0 0x558be93c3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f7959308640 1 -- 172.21.15.5:0/728560256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b4af0 msgr2=0x558be93c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:03:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f7959308640 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b4af0 0x558be93c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f7959308640 1 -- 172.21.15.5:0/728560256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be93b8720 con 0x558be93b5410 2024-09-19T06:03:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f7959308640 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 0x558be93c84f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f79440a47f0 tx=0x7f7944018cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65cfb41ab75dd45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:03:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f793d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:03:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f793d7fa640 1 -- 172.21.15.5:0/728560256 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794409c7b0 con 0x558be93b5410 2024-09-19T06:03:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f793d7fa640 1 -- 172.21.15.5:0/728560256 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f7944098720 con 0x558be93b5410 2024-09-19T06:03:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f795ebea740 1 -- 172.21.15.5:0/728560256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558be9320d70 con 0x558be93b5410 2024-09-19T06:03:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f793d7fa640 1 -- 172.21.15.5:0/728560256 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f79440a4d20 con 0x558be93b5410 2024-09-19T06:03:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f795ebea740 10 client.15477 fetch_fsmap learned FSMap version 28 2024-09-19T06:03:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f795ebea740 1 -- 172.21.15.5:0/728560256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558be93ac2d0 con 0x558be93b5410 2024-09-19T06:03:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.255+0000 7f793d7fa640 1 -- 172.21.15.5:0/728560256 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x558be9320d70 con 0x558be93b5410 2024-09-19T06:03:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.256+0000 7f793d7fa640 1 -- 172.21.15.5:0/728560256 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f79440a4890 con 0x558be93b5410 2024-09-19T06:03:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.256+0000 7f795ebea740 10 client.15477 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:03:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.256+0000 7f795ebea740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:03:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.256+0000 7f795ebea740 20 client.15477 populate_metadata read hostname 'smithi005' 2024-09-19T06:03:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.256+0000 7f795ebea740 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558be943a6b0 0x558be943cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:03:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.256+0000 7f795ebea740 4 client.15477 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:03:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.256+0000 7f795ebea740 1 -- 172.21.15.5:0/728560256 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558be929d900 con 0x558be943a6b0 2024-09-19T06:03:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.256+0000 7f7959b09640 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558be943a6b0 0x558be943cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:03:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.256+0000 7f7959b09640 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558be943a6b0 0x558be943cad0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fefecaa3975c571b server_cookie=3463b7de2bfb67 in_seq=0 out_seq=0 2024-09-19T06:03:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.256+0000 7f793d7fa640 10 client.15477 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f793d7fa640 1 -- 172.21.15.5:0/728560256 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558be929d900 con 0x558be943a6b0 2024-09-19T06:03:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f793d7fa640 10 client.15477 handle_command_reply: tid=0 2024-09-19T06:03:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f795ebea740 1 client.15477 shutdown 2024-09-19T06:03:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f795ebea740 1 -- 172.21.15.5:0/728560256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 msgr2=0x558be93c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f795ebea740 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 0x558be93c84f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f79440a47f0 tx=0x7f7944018cb0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f795ebea740 1 -- 172.21.15.5:0/728560256 shutdown_connections 2024-09-19T06:03:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f795ebea740 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558be943a6b0 0x558be943cad0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f795ebea740 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558be93bfcb0 0x558be93c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f795ebea740 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558be93b5410 0x558be93c84f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f793d7fa640 0 client.15477 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:03:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f795ebea740 1 --2- 172.21.15.5:0/728560256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be93b4af0 0x558be93c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f795ebea740 1 -- 172.21.15.5:0/728560256 >> 172.21.15.5:0/728560256 conn(0x558be93b1060 msgr2=0x558be93af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f795ebea740 1 -- 172.21.15.5:0/728560256 shutdown_connections 2024-09-19T06:03:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.260+0000 7f795ebea740 1 -- 172.21.15.5:0/728560256 wait complete. 2024-09-19T06:03:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795ebea740 20 client.15477 trim_cache size 0 max 16384 2024-09-19T06:03:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:03:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:03:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:03:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:03:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795bd6d640 1 -- 172.21.15.5:0/1885204238 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f79400a34c0 msgr2=0x7f79400a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795bd6d640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f79400a34c0 0x7f79400a5980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f794c04d000 tx=0x7f794c04a000 comp rx=0 tx=0).stop 2024-09-19T06:03:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795bd6d640 1 -- 172.21.15.5:0/1885204238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79541035a0 msgr2=0x7f7954100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:03:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795bd6d640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79541035a0 0x7f7954100e10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f794400c920 tx=0x7f794400cde0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795bd6d640 1 -- 172.21.15.5:0/1885204238 shutdown_connections 2024-09-19T06:03:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795bd6d640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f79400a34c0 0x7f79400a5980 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795bd6d640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f795410c0a0 0x7f7954101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795bd6d640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79541035a0 0x7f7954100e10 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795bd6d640 1 --2- 172.21.15.5:0/1885204238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7954102c80 0x7f79541008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:03:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795bd6d640 1 -- 172.21.15.5:0/1885204238 >> 172.21.15.5:0/1885204238 conn(0x7f79540779a0 msgr2=0x7f79540fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:03:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795bd6d640 1 -- 172.21.15.5:0/1885204238 shutdown_connections 2024-09-19T06:03:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:03:58.261+0000 7f795bd6d640 1 -- 172.21.15.5:0/1885204238 wait complete. 2024-09-19T06:03:58.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:03:58 smithi157 ceph-mon[35346]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T06:03:58.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:03:58 smithi005 ceph-mon[30955]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T06:03:58.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:03:58 smithi049 ceph-mon[34640]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T06:04:00.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:00 smithi005 ceph-mon[30955]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-09-19T06:04:00.379 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:04:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:04:00.095+0000 7f3f1a0c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:04:00.379 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:04:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:04:00.095+0000 7f3f1a0c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:04:00.379 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:04:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:04:00.095+0000 7f3f1a0c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:04:00.379 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:04:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:04:00.095+0000 7f3f1a0c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:04:00.380 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:04:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:04:00.095+0000 7f3f1a0c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:04:00.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:00 smithi157 ceph-mon[35346]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-09-19T06:04:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:00 smithi049 ceph-mon[34640]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-09-19T06:04:02.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:02 smithi157 ceph-mon[35346]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:04:02.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:02 smithi157 ceph-mon[35346]: mgrmap e25: x(active, since 40s), standbys: z, y 2024-09-19T06:04:02.610 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:04:02.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:02 smithi005 ceph-mon[30955]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:04:02.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:02 smithi005 ceph-mon[30955]: mgrmap e25: x(active, since 40s), standbys: z, y 2024-09-19T06:04:02.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:02 smithi049 ceph-mon[34640]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:04:02.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:02 smithi049 ceph-mon[34640]: mgrmap e25: x(active, since 40s), standbys: z, y 2024-09-19T06:04:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:03.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.504+0000 7f30e05ac640 1 -- 172.21.15.5:0/1513902157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d8103740 msgr2=0x7f30d8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:03.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.504+0000 7f30e05ac640 1 --2- 172.21.15.5:0/1513902157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d8103740 0x7f30d8107b80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009930 tx=0x7f30c802a420 comp rx=0 tx=0).stop 2024-09-19T06:04:03.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.504+0000 7f30e05ac640 1 -- 172.21.15.5:0/1513902157 shutdown_connections 2024-09-19T06:04:03.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.504+0000 7f30e05ac640 1 --2- 172.21.15.5:0/1513902157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30d8108290 0x7f30d810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.504+0000 7f30e05ac640 1 --2- 172.21.15.5:0/1513902157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d8103740 0x7f30d8107b80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.504+0000 7f30e05ac640 1 --2- 172.21.15.5:0/1513902157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d8102d90 0x7f30d8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.504+0000 7f30e05ac640 1 -- 172.21.15.5:0/1513902157 >> 172.21.15.5:0/1513902157 conn(0x7f30d80fc7d0 msgr2=0x7f30d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:03.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.504+0000 7f30e05ac640 1 -- 172.21.15.5:0/1513902157 shutdown_connections 2024-09-19T06:04:03.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.504+0000 7f30e05ac640 1 -- 172.21.15.5:0/1513902157 wait complete. 2024-09-19T06:04:03.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.504+0000 7f30e05ac640 1 Processor -- start 2024-09-19T06:04:03.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30e05ac640 1 -- start start 2024-09-19T06:04:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30e05ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30d8102d90 0x7f30d8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30e05ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d8103740 0x7f30d8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30e05ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d8108290 0x7f30d819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30e05ac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8111f40 con 0x7f30d8108290 2024-09-19T06:04:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30e05ac640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8111dc0 con 0x7f30d8102d90 2024-09-19T06:04:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30e05ac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81120c0 con 0x7f30d8103740 2024-09-19T06:04:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30deb49640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d8108290 0x7f30d819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30de348640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30d8102d90 0x7f30d8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30de348640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30d8102d90 0x7f30d8198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55012/0 (socket says 172.21.15.5:55012) 2024-09-19T06:04:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30deb49640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d8108290 0x7f30d819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50728/0 (socket says 172.21.15.5:50728) 2024-09-19T06:04:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30de348640 1 -- 172.21.15.5:0/1241479299 learned_addr learned my addr 172.21.15.5:0/1241479299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.505+0000 7f30ddb47640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d8103740 0x7f30d8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.506+0000 7f30deb49640 1 -- 172.21.15.5:0/1241479299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d8103740 msgr2=0x7f30d8198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.506+0000 7f30deb49640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d8103740 0x7f30d8198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.506+0000 7f30deb49640 1 -- 172.21.15.5:0/1241479299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30d8102d90 msgr2=0x7f30d8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.506+0000 7f30deb49640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30d8102d90 0x7f30d8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.506+0000 7f30deb49640 1 -- 172.21.15.5:0/1241479299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d819d4b0 con 0x7f30d8108290 2024-09-19T06:04:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.506+0000 7f30de348640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30d8102d90 0x7f30d8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.506+0000 7f30deb49640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d8108290 0x7f30d819cdd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f30d000da10 tx=0x7f30d000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.506+0000 7f30cf7fe640 1 -- 172.21.15.5:0/1241479299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d000bd70 con 0x7f30d8108290 2024-09-19T06:04:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.506+0000 7f30cf7fe640 1 -- 172.21.15.5:0/1241479299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30d0003e20 con 0x7f30d8108290 2024-09-19T06:04:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.506+0000 7f30e05ac640 1 -- 172.21.15.5:0/1241479299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d819d7a0 con 0x7f30d8108290 2024-09-19T06:04:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.506+0000 7f30e05ac640 1 -- 172.21.15.5:0/1241479299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d8100920 con 0x7f30d8108290 2024-09-19T06:04:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.507+0000 7f30cf7fe640 1 -- 172.21.15.5:0/1241479299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d00106c0 con 0x7f30d8108290 2024-09-19T06:04:03.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.509+0000 7f30cf7fe640 1 -- 172.21.15.5:0/1241479299 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7f30d0003970 con 0x7f30d8108290 2024-09-19T06:04:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.509+0000 7f30cf7fe640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f30c40a35d0 0x7f30c40a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.510+0000 7f30cf7fe640 1 -- 172.21.15.5:0/1241479299 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f30d00d02f0 con 0x7f30d8108290 2024-09-19T06:04:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.510+0000 7f30de348640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f30c40a35d0 0x7f30c40a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.510+0000 7f30e2985740 1 Processor -- start 2024-09-19T06:04:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.511+0000 7f30e2985740 1 -- start start 2024-09-19T06:04:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.511+0000 7f30e2985740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e41cb0 0x55c140e42090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.511+0000 7f30e2985740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e425d0 0x55c140e4c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.511+0000 7f30e2985740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e4ce90 0x55c140e4f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.511+0000 7f30e2985740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c140d9f2e0 con 0x55c140e425d0 2024-09-19T06:04:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.511+0000 7f30e2985740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c140d46ce0 con 0x55c140e41cb0 2024-09-19T06:04:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.511+0000 7f30e2985740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c140d9b2a0 con 0x55c140e4ce90 2024-09-19T06:04:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.511+0000 7f30de348640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f30c40a35d0 0x7f30c40a5a90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f30d40104e0 tx=0x7f30d400b410 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.511+0000 7f30deb49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e4ce90 0x55c140e4f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.511+0000 7f30deb49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e4ce90 0x55c140e4f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38816/0 (socket says 172.21.15.5:38816) 2024-09-19T06:04:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.511+0000 7f30deb49640 1 -- 172.21.15.5:0/2396959306 learned_addr learned my addr 172.21.15.5:0/2396959306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.511+0000 7f30de348640 1 --2- 172.21.15.5:0/2396959306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e41cb0 0x55c140e42090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.512+0000 7f30deb49640 1 -- 172.21.15.5:0/2396959306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e41cb0 msgr2=0x55c140e42090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:03.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.512+0000 7f30deb49640 1 --2- 172.21.15.5:0/2396959306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e41cb0 0x55c140e42090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.512+0000 7f30deb49640 1 -- 172.21.15.5:0/2396959306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e425d0 msgr2=0x55c140e4c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:03.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.512+0000 7f30ddb47640 1 --2- 172.21.15.5:0/2396959306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e425d0 0x55c140e4c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.512+0000 7f30deb49640 1 --2- 172.21.15.5:0/2396959306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e425d0 0x55c140e4c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.512+0000 7f30deb49640 1 -- 172.21.15.5:0/2396959306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c140d2b440 con 0x55c140e4ce90 2024-09-19T06:04:03.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.512+0000 7f30ddb47640 1 --2- 172.21.15.5:0/2396959306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e425d0 0x55c140e4c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:04:03.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.512+0000 7f30deb49640 1 --2- 172.21.15.5:0/2396959306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e4ce90 0x55c140e4f280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30d00a4a70 tx=0x7f30d009bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1977e3abb6625f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:03.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.512+0000 7f30ccff9640 1 -- 172.21.15.5:0/2396959306 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d009c330 con 0x55c140e4ce90 2024-09-19T06:04:03.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.513+0000 7f30ccff9640 1 -- 172.21.15.5:0/2396959306 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30d00a06e0 con 0x55c140e4ce90 2024-09-19T06:04:03.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.513+0000 7f30ccff9640 1 -- 172.21.15.5:0/2396959306 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0004100 con 0x55c140e4ce90 2024-09-19T06:04:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.514+0000 7f30e2985740 1 -- 172.21.15.5:0/2396959306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e4ce90 msgr2=0x55c140e4f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.514+0000 7f30e2985740 1 --2- 172.21.15.5:0/2396959306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e4ce90 0x55c140e4f280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30d00a4a70 tx=0x7f30d009bf30 comp rx=0 tx=0).stop 2024-09-19T06:04:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.514+0000 7f30e2985740 1 -- 172.21.15.5:0/2396959306 shutdown_connections 2024-09-19T06:04:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.514+0000 7f30e2985740 1 --2- 172.21.15.5:0/2396959306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e4ce90 0x55c140e4f280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.514+0000 7f30e2985740 1 --2- 172.21.15.5:0/2396959306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e425d0 0x55c140e4c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.514+0000 7f30e2985740 1 --2- 172.21.15.5:0/2396959306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e41cb0 0x55c140e42090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.514+0000 7f30e2985740 1 -- 172.21.15.5:0/2396959306 >> 172.21.15.5:0/2396959306 conn(0x55c140e3e240 msgr2=0x55c140e3e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.514+0000 7f30e2985740 1 -- 172.21.15.5:0/2396959306 shutdown_connections 2024-09-19T06:04:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.514+0000 7f30e2985740 1 -- 172.21.15.5:0/2396959306 wait complete. 2024-09-19T06:04:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.514+0000 7f30e2985740 1 -- start start 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30e2985740 1 Processor -- start 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30e2985740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e41cb0 0x55c140e4c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30e2985740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e425d0 0x55c140e49a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30e2985740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e4ce90 0x55c140e49fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30e2985740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c140e4fa60 con 0x55c140e4ce90 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30e2985740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c140d5a050 con 0x55c140e41cb0 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30e2985740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c140e4fbe0 con 0x55c140e425d0 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30e2985740 10 client.0 _ll_register_callbacks cb 0x55c140d9a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30de348640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e41cb0 0x55c140e4c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30de348640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e41cb0 0x55c140e4c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55028/0 (socket says 172.21.15.5:55028) 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30de348640 1 -- 172.21.15.5:0/3074546486 learned_addr learned my addr 172.21.15.5:0/3074546486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30ddb47640 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e425d0 0x55c140e49a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.515+0000 7f30deb49640 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e4ce90 0x55c140e49fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.516+0000 7f30de348640 1 -- 172.21.15.5:0/3074546486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e425d0 msgr2=0x55c140e49a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.516+0000 7f30de348640 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e425d0 0x55c140e49a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.516+0000 7f30de348640 1 -- 172.21.15.5:0/3074546486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e4ce90 msgr2=0x55c140e49fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.516+0000 7f30de348640 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e4ce90 0x55c140e49fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.516+0000 7f30de348640 1 -- 172.21.15.5:0/3074546486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c140b103c0 con 0x55c140e41cb0 2024-09-19T06:04:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.516+0000 7f30de348640 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e41cb0 0x55c140e4c7b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f30d4011c60 tx=0x7f30d4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67551d636ca61b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.516+0000 7f30c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:04:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.516+0000 7f30c1ffb640 1 -- 172.21.15.5:0/3074546486 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d401ee90 con 0x55c140e41cb0 2024-09-19T06:04:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.516+0000 7f30c1ffb640 1 -- 172.21.15.5:0/3074546486 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f30d401f430 con 0x55c140e41cb0 2024-09-19T06:04:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.516+0000 7f30e2985740 1 -- 172.21.15.5:0/3074546486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30d00a06e0 con 0x55c140e41cb0 2024-09-19T06:04:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.517+0000 7f30c1ffb640 1 -- 172.21.15.5:0/3074546486 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30d401f630 con 0x55c140e41cb0 2024-09-19T06:04:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.517+0000 7f30e2985740 10 client.25112 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.517+0000 7f30e2985740 1 -- 172.21.15.5:0/3074546486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c140e5b3b0 con 0x55c140e41cb0 2024-09-19T06:04:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.517+0000 7f30c1ffb640 1 -- 172.21.15.5:0/3074546486 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30d00a06e0 con 0x55c140e41cb0 2024-09-19T06:04:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.517+0000 7f30c1ffb640 1 -- 172.21.15.5:0/3074546486 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f30d4032800 con 0x55c140e41cb0 2024-09-19T06:04:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.517+0000 7f30e2985740 10 client.25112 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.517+0000 7f30e2985740 10 client.25112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.517+0000 7f30e2985740 20 client.25112 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.517+0000 7f30e2985740 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c140e5c7c0 0x55c140e5ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.517+0000 7f30e2985740 4 client.25112 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.517+0000 7f30e2985740 1 -- 172.21.15.5:0/3074546486 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c140dac850 con 0x55c140e5c7c0 2024-09-19T06:04:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.517+0000 7f30ddb47640 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c140e5c7c0 0x55c140e5ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.518+0000 7f30c1ffb640 10 client.25112 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.518+0000 7f30ddb47640 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c140e5c7c0 0x55c140e5ebb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5531b9b19266dfcb server_cookie=6e91651fdcc854e9 in_seq=0 out_seq=0 2024-09-19T06:04:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30c1ffb640 1 -- 172.21.15.5:0/3074546486 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c140dac850 con 0x55c140e5c7c0 2024-09-19T06:04:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30c1ffb640 10 client.25112 handle_command_reply: tid=0 2024-09-19T06:04:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30e2985740 1 client.25112 shutdown 2024-09-19T06:04:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30e2985740 1 -- 172.21.15.5:0/3074546486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e41cb0 msgr2=0x55c140e4c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30e2985740 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e41cb0 0x55c140e4c7b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f30d4011c60 tx=0x7f30d4004730 comp rx=0 tx=0).stop 2024-09-19T06:04:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30e2985740 1 -- 172.21.15.5:0/3074546486 shutdown_connections 2024-09-19T06:04:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30e2985740 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c140e5c7c0 0x55c140e5ebb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30e2985740 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e4ce90 0x55c140e49fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30e2985740 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e425d0 0x55c140e49a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30e2985740 1 --2- 172.21.15.5:0/3074546486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e41cb0 0x55c140e4c7b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30e2985740 1 -- 172.21.15.5:0/3074546486 >> 172.21.15.5:0/3074546486 conn(0x55c140e3e240 msgr2=0x55c140e43f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30c1ffb640 0 client.25112 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30e2985740 1 -- 172.21.15.5:0/3074546486 shutdown_connections 2024-09-19T06:04:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.519+0000 7f30e2985740 1 -- 172.21.15.5:0/3074546486 wait complete. 2024-09-19T06:04:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.520+0000 7f30e2985740 20 client.25112 trim_cache size 0 max 16384 2024-09-19T06:04:03.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.555+0000 7f30e2985740 1 Processor -- start 2024-09-19T06:04:03.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.555+0000 7f30e2985740 1 -- start start 2024-09-19T06:04:03.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.555+0000 7f30e2985740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e3e240 0x55c140e58cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.555+0000 7f30e2985740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e41cb0 0x55c140e44fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.555+0000 7f30de348640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e3e240 0x55c140e58cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.555+0000 7f30de348640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e3e240 0x55c140e58cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50756/0 (socket says 172.21.15.5:50756) 2024-09-19T06:04:03.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.555+0000 7f30de348640 1 -- 172.21.15.5:0/1388190492 learned_addr learned my addr 172.21.15.5:0/1388190492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:03.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.555+0000 7f30e2985740 1 --2- 172.21.15.5:0/1388190492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e425d0 0x55c140e64460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.555+0000 7f30e2985740 1 -- 172.21.15.5:0/1388190492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81ab6a0 con 0x55c140e3e240 2024-09-19T06:04:03.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.555+0000 7f30e2985740 1 -- 172.21.15.5:0/1388190492 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81ad1e0 con 0x55c140e425d0 2024-09-19T06:04:03.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.555+0000 7f30e2985740 1 -- 172.21.15.5:0/1388190492 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81abd70 con 0x55c140e41cb0 2024-09-19T06:04:03.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.556+0000 7f30ddb47640 1 --2- 172.21.15.5:0/1388190492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e41cb0 0x55c140e44fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.556+0000 7f30deb49640 1 --2- 172.21.15.5:0/1388190492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e425d0 0x55c140e64460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.556+0000 7f30ddb47640 1 -- 172.21.15.5:0/1388190492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e425d0 msgr2=0x55c140e64460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:03.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.556+0000 7f30ddb47640 1 --2- 172.21.15.5:0/1388190492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e425d0 0x55c140e64460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.556+0000 7f30ddb47640 1 -- 172.21.15.5:0/1388190492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e3e240 msgr2=0x55c140e58cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:03.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.556+0000 7f30ddb47640 1 --2- 172.21.15.5:0/1388190492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e3e240 0x55c140e58cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.556+0000 7f30ddb47640 1 -- 172.21.15.5:0/1388190492 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c140e3e070 con 0x55c140e41cb0 2024-09-19T06:04:03.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.556+0000 7f30de348640 1 --2- 172.21.15.5:0/1388190492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e3e240 0x55c140e58cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:03.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.556+0000 7f30ddb47640 1 --2- 172.21.15.5:0/1388190492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e41cb0 0x55c140e44fb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009230 tx=0x7f30c8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b44e9f7c5a6fe51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.557+0000 7f30c3fff640 1 -- 172.21.15.5:0/1388190492 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8049070 con 0x55c140e41cb0 2024-09-19T06:04:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.557+0000 7f30c3fff640 1 -- 172.21.15.5:0/1388190492 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30c8031950 con 0x55c140e41cb0 2024-09-19T06:04:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.557+0000 7f30c3fff640 1 -- 172.21.15.5:0/1388190492 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8035040 con 0x55c140e41cb0 2024-09-19T06:04:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.557+0000 7f30e2985740 1 -- 172.21.15.5:0/1388190492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e41cb0 msgr2=0x55c140e44fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.557+0000 7f30e2985740 1 --2- 172.21.15.5:0/1388190492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e41cb0 0x55c140e44fb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009230 tx=0x7f30c8000bc0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.558+0000 7f30e2985740 1 -- 172.21.15.5:0/1388190492 shutdown_connections 2024-09-19T06:04:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.558+0000 7f30e2985740 1 --2- 172.21.15.5:0/1388190492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e425d0 0x55c140e64460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.558+0000 7f30e2985740 1 --2- 172.21.15.5:0/1388190492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e41cb0 0x55c140e44fb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.558+0000 7f30e2985740 1 --2- 172.21.15.5:0/1388190492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e3e240 0x55c140e58cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.558+0000 7f30e2985740 1 -- 172.21.15.5:0/1388190492 >> 172.21.15.5:0/1388190492 conn(0x55c140e4a100 msgr2=0x55c140e448a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.558+0000 7f30e2985740 1 -- 172.21.15.5:0/1388190492 shutdown_connections 2024-09-19T06:04:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.558+0000 7f30e2985740 1 -- 172.21.15.5:0/1388190492 wait complete. 2024-09-19T06:04:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.558+0000 7f30e2985740 1 -- start start 2024-09-19T06:04:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30e2985740 1 Processor -- start 2024-09-19T06:04:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30e2985740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e3e240 0x55c140e55430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30e2985740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e41cb0 0x55c140e50760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30ddb47640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e41cb0 0x55c140e50760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30ddb47640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e41cb0 0x55c140e50760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50760/0 (socket says 172.21.15.5:50760) 2024-09-19T06:04:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30ddb47640 1 -- 172.21.15.5:0/997572841 learned_addr learned my addr 172.21.15.5:0/997572841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30e2985740 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e425d0 0x55c140e50ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30e2985740 1 -- 172.21.15.5:0/997572841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81acb10 con 0x55c140e41cb0 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30e2985740 1 -- 172.21.15.5:0/997572841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81ac440 con 0x55c140e425d0 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30e2985740 1 -- 172.21.15.5:0/997572841 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81a9680 con 0x55c140e3e240 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30e2985740 10 client.0 _ll_register_callbacks cb 0x7f30d40115a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30de348640 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e3e240 0x55c140e55430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.559+0000 7f30deb49640 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e425d0 0x55c140e50ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.560+0000 7f30ddb47640 1 -- 172.21.15.5:0/997572841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e3e240 msgr2=0x55c140e55430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.560+0000 7f30ddb47640 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e3e240 0x55c140e55430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.560+0000 7f30ddb47640 1 -- 172.21.15.5:0/997572841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e425d0 msgr2=0x55c140e50ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.560+0000 7f30ddb47640 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e425d0 0x55c140e50ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.560+0000 7f30ddb47640 1 -- 172.21.15.5:0/997572841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c140e3edf0 con 0x55c140e41cb0 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.560+0000 7f30de348640 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e3e240 0x55c140e55430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.560+0000 7f30ddb47640 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e41cb0 0x55c140e50760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f30c8047040 tx=0x7f30c8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86e810dbbfdf05c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.560+0000 7f30c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.560+0000 7f30c1ffb640 1 -- 172.21.15.5:0/997572841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8035050 con 0x55c140e41cb0 2024-09-19T06:04:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.560+0000 7f30c1ffb640 1 -- 172.21.15.5:0/997572841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f30c80043c0 con 0x55c140e41cb0 2024-09-19T06:04:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.560+0000 7f30e2985740 1 -- 172.21.15.5:0/997572841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c8031950 con 0x55c140e41cb0 2024-09-19T06:04:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.561+0000 7f30c1ffb640 1 -- 172.21.15.5:0/997572841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30c8049040 con 0x55c140e41cb0 2024-09-19T06:04:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.561+0000 7f30e2985740 10 client.15498 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.561+0000 7f30e2985740 1 -- 172.21.15.5:0/997572841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c140ec7110 con 0x55c140e41cb0 2024-09-19T06:04:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.561+0000 7f30c1ffb640 1 -- 172.21.15.5:0/997572841 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30c8052070 con 0x55c140e41cb0 2024-09-19T06:04:03.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.561+0000 7f30c1ffb640 1 -- 172.21.15.5:0/997572841 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f30c8004900 con 0x55c140e41cb0 2024-09-19T06:04:03.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.561+0000 7f30e2985740 10 client.15498 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:03.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.561+0000 7f30e2985740 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:03.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.561+0000 7f30e2985740 20 client.15498 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:03.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.561+0000 7f30e2985740 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c140e4ce90 0x55c140ec9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:03.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.561+0000 7f30e2985740 4 client.15498 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.561+0000 7f30e2985740 1 -- 172.21.15.5:0/997572841 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f30c8031950 con 0x55c140e4ce90 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.562+0000 7f30de348640 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c140e4ce90 0x55c140ec9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.562+0000 7f30c1ffb640 10 client.15498 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.562+0000 7f30de348640 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c140e4ce90 0x55c140ec9840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dafed2b1b3a3e5e server_cookie=38671d4511c98219 in_seq=0 out_seq=0 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30c1ffb640 1 -- 172.21.15.5:0/997572841 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f30c8031950 con 0x55c140e4ce90 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30c1ffb640 10 client.15498 handle_command_reply: tid=0 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30e2985740 1 client.15498 shutdown 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30e2985740 1 -- 172.21.15.5:0/997572841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e41cb0 msgr2=0x55c140e50760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30e2985740 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e41cb0 0x55c140e50760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f30c8047040 tx=0x7f30c8005fd0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30e2985740 1 -- 172.21.15.5:0/997572841 shutdown_connections 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30e2985740 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c140e4ce90 0x55c140ec9840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30e2985740 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c140e425d0 0x55c140e50ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30e2985740 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c140e41cb0 0x55c140e50760 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30e2985740 1 --2- 172.21.15.5:0/997572841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c140e3e240 0x55c140e55430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30e2985740 1 -- 172.21.15.5:0/997572841 >> 172.21.15.5:0/997572841 conn(0x55c140e4a100 msgr2=0x55c140e450c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30c1ffb640 0 client.15498 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30e2985740 1 -- 172.21.15.5:0/997572841 shutdown_connections 2024-09-19T06:04:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.563+0000 7f30e2985740 1 -- 172.21.15.5:0/997572841 wait complete. 2024-09-19T06:04:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.564+0000 7f30e2985740 20 client.15498 trim_cache size 0 max 16384 2024-09-19T06:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:04:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.564+0000 7f30e05ac640 1 -- 172.21.15.5:0/1241479299 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f30c40a35d0 msgr2=0x7f30c40a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.564+0000 7f30e05ac640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f30c40a35d0 0x7f30c40a5a90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f30d40104e0 tx=0x7f30d400b410 comp rx=0 tx=0).stop 2024-09-19T06:04:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.564+0000 7f30e05ac640 1 -- 172.21.15.5:0/1241479299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d8108290 msgr2=0x7f30d819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.564+0000 7f30e05ac640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d8108290 0x7f30d819cdd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f30d000da10 tx=0x7f30d000ded0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.565+0000 7f30e05ac640 1 -- 172.21.15.5:0/1241479299 shutdown_connections 2024-09-19T06:04:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.565+0000 7f30e05ac640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d8108290 0x7f30d819cdd0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.565+0000 7f30e05ac640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f30c40a35d0 0x7f30c40a5a90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.565+0000 7f30e05ac640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d8103740 0x7f30d8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.565+0000 7f30e05ac640 1 --2- 172.21.15.5:0/1241479299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30d8102d90 0x7f30d8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.565+0000 7f30e05ac640 1 -- 172.21.15.5:0/1241479299 >> 172.21.15.5:0/1241479299 conn(0x7f30d80fc7d0 msgr2=0x7f30d8105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.565+0000 7f30e05ac640 1 -- 172.21.15.5:0/1241479299 shutdown_connections 2024-09-19T06:04:03.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:03.565+0000 7f30e05ac640 1 -- 172.21.15.5:0/1241479299 wait complete. 2024-09-19T06:04:04.317 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:04:04.317 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:04:04.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:04 smithi157 ceph-mon[35346]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T06:04:04.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:04 smithi005 ceph-mon[30955]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T06:04:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:04 smithi049 ceph-mon[34640]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T06:04:04.717 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:05.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.297+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1542687848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d04100a80 msgr2=0x7f5d04100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.297+0000 7f5d0ab62640 1 --2- 172.21.15.5:0/1542687848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d04100a80 0x7f5d04100ee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009930 tx=0x7f5cf002a420 comp rx=0 tx=0).stop 2024-09-19T06:04:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.297+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1542687848 shutdown_connections 2024-09-19T06:04:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.297+0000 7f5d0ab62640 1 --2- 172.21.15.5:0/1542687848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d04101420 0x7f5d0410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.297+0000 7f5d0ab62640 1 --2- 172.21.15.5:0/1542687848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d04100a80 0x7f5d04100ee0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.297+0000 7f5d0ab62640 1 --2- 172.21.15.5:0/1542687848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d04106a80 0x7f5d04106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.297+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1542687848 >> 172.21.15.5:0/1542687848 conn(0x7f5d040fc7a0 msgr2=0x7f5d040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.297+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1542687848 shutdown_connections 2024-09-19T06:04:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.297+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1542687848 wait complete. 2024-09-19T06:04:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.298+0000 7f5d0ab62640 1 Processor -- start 2024-09-19T06:04:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.298+0000 7f5d0ab62640 1 -- start start 2024-09-19T06:04:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.298+0000 7f5d0ab62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d04100a80 0x7f5d0419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.298+0000 7f5d0ab62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d04101420 0x7f5d0419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.298+0000 7f5d0ab62640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d04106a80 0x7f5d0419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.298+0000 7f5d0ab62640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d04110d80 con 0x7f5d04106a80 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.298+0000 7f5d0ab62640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d04110c00 con 0x7f5d04101420 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.298+0000 7f5d0ab62640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5d04110f00 con 0x7f5d04100a80 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.299+0000 7f5d090ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d04106a80 0x7f5d0419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.299+0000 7f5d090ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d04106a80 0x7f5d0419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41496/0 (socket says 172.21.15.5:41496) 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.299+0000 7f5d090ff640 1 -- 172.21.15.5:0/1838091246 learned_addr learned my addr 172.21.15.5:0/1838091246 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.299+0000 7f5d088fe640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d04100a80 0x7f5d0419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.299+0000 7f5d03fff640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d04101420 0x7f5d0419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.299+0000 7f5d090ff640 1 -- 172.21.15.5:0/1838091246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d04100a80 msgr2=0x7f5d0419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.299+0000 7f5d090ff640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d04100a80 0x7f5d0419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.299+0000 7f5d090ff640 1 -- 172.21.15.5:0/1838091246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d04101420 msgr2=0x7f5d0419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.299+0000 7f5d090ff640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d04101420 0x7f5d0419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.299+0000 7f5d090ff640 1 -- 172.21.15.5:0/1838091246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0419f520 con 0x7f5d04106a80 2024-09-19T06:04:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.299+0000 7f5d03fff640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d04101420 0x7f5d0419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.299+0000 7f5d090ff640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d04106a80 0x7f5d0419ee40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00cce0 tx=0x7f5cfc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.300+0000 7f5d01ffb640 1 -- 172.21.15.5:0/1838091246 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc00f450 con 0x7f5d04106a80 2024-09-19T06:04:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.300+0000 7f5d01ffb640 1 -- 172.21.15.5:0/1838091246 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cfc00faa0 con 0x7f5d04106a80 2024-09-19T06:04:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.300+0000 7f5d01ffb640 1 -- 172.21.15.5:0/1838091246 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc018be0 con 0x7f5d04106a80 2024-09-19T06:04:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.300+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1838091246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0419f810 con 0x7f5d04106a80 2024-09-19T06:04:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.300+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1838091246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0419fc70 con 0x7f5d04106a80 2024-09-19T06:04:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.302+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1838091246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d04100ee0 con 0x7f5d04106a80 2024-09-19T06:04:05.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.303+0000 7f5d01ffb640 1 -- 172.21.15.5:0/1838091246 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7f5cfc00fc50 con 0x7f5d04106a80 2024-09-19T06:04:05.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.303+0000 7f5d01ffb640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5cf40a36a0 0x7f5cf40a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:05.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.304+0000 7f5d01ffb640 1 -- 172.21.15.5:0/1838091246 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f5cfc0d0420 con 0x7f5d04106a80 2024-09-19T06:04:05.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.304+0000 7f5d088fe640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5cf40a36a0 0x7f5cf40a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:05.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.305+0000 7f5d088fe640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5cf40a36a0 0x7f5cf40a5b60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8005d80 tx=0x7f5cf8005c90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:05.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.308+0000 7f5d01ffb640 1 -- 172.21.15.5:0/1838091246 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d0419fc70 con 0x7f5d04106a80 2024-09-19T06:04:05.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.631+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1838091246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d041a7240 con 0x7f5d04106a80 2024-09-19T06:04:05.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.632+0000 7f5d01ffb640 1 -- 172.21.15.5:0/1838091246 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11959 (secure 0 0 0) 0x7f5cfc014030 con 0x7f5d04106a80 2024-09-19T06:04:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:04:05.636 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-19T06:03:21:625357+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:03:20.863188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:04:05.636 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-19T06:04:05.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.634+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1838091246 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5cf40a36a0 msgr2=0x7f5cf40a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:05.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.634+0000 7f5d0ab62640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5cf40a36a0 0x7f5cf40a5b60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8005d80 tx=0x7f5cf8005c90 comp rx=0 tx=0).stop 2024-09-19T06:04:05.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.634+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1838091246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d04106a80 msgr2=0x7f5d0419ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:05.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.634+0000 7f5d0ab62640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d04106a80 0x7f5d0419ee40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00cce0 tx=0x7f5cfc00a620 comp rx=0 tx=0).stop 2024-09-19T06:04:05.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.634+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1838091246 shutdown_connections 2024-09-19T06:04:05.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.634+0000 7f5d0ab62640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d04106a80 0x7f5d0419ee40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:05.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.634+0000 7f5d0ab62640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d04101420 0x7f5d0419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:05.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.634+0000 7f5d0ab62640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5cf40a36a0 0x7f5cf40a5b60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:05.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.634+0000 7f5d0ab62640 1 --2- 172.21.15.5:0/1838091246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d04100a80 0x7f5d0419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:05.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.634+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1838091246 >> 172.21.15.5:0/1838091246 conn(0x7f5d040fc7a0 msgr2=0x7f5d0406a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:05.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.634+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1838091246 shutdown_connections 2024-09-19T06:04:05.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:05.634+0000 7f5d0ab62640 1 -- 172.21.15.5:0/1838091246 wait complete. 2024-09-19T06:04:06.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:04:06.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:06 smithi157 ceph-mon[35346]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T06:04:06.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:06 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1838091246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:04:06.607 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:06.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:06 smithi005 ceph-mon[30955]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T06:04:06.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:06 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1838091246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:04:06.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:06 smithi049 ceph-mon[34640]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T06:04:06.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:06 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1838091246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:04:07.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.083+0000 7f8d7c27e640 1 -- 172.21.15.5:0/4036966033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7410e4d0 msgr2=0x7f8d741108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:07.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.083+0000 7f8d7c27e640 1 --2- 172.21.15.5:0/4036966033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7410e4d0 0x7f8d741108e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009ca0 tx=0x7f8d6c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.084+0000 7f8d7c27e640 1 -- 172.21.15.5:0/4036966033 shutdown_connections 2024-09-19T06:04:07.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.084+0000 7f8d7c27e640 1 --2- 172.21.15.5:0/4036966033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7410e4d0 0x7f8d741108e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.084+0000 7f8d7c27e640 1 --2- 172.21.15.5:0/4036966033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d74101360 0x7f8d7410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.084+0000 7f8d7c27e640 1 --2- 172.21.15.5:0/4036966033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d741009b0 0x7f8d74100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.084+0000 7f8d7c27e640 1 -- 172.21.15.5:0/4036966033 >> 172.21.15.5:0/4036966033 conn(0x7f8d740fc7d0 msgr2=0x7f8d740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.084+0000 7f8d7c27e640 1 -- 172.21.15.5:0/4036966033 shutdown_connections 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.084+0000 7f8d7c27e640 1 -- 172.21.15.5:0/4036966033 wait complete. 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.084+0000 7f8d7c27e640 1 Processor -- start 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.085+0000 7f8d7c27e640 1 -- start start 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.085+0000 7f8d7c27e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d741009b0 0x7f8d7419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.085+0000 7f8d7c27e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d74101360 0x7f8d7419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.085+0000 7f8d7c27e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7410e4d0 0x7f8d7419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.085+0000 7f8d7c27e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d74114120 con 0x7f8d7410e4d0 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.085+0000 7f8d7c27e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8d74113fa0 con 0x7f8d74101360 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.085+0000 7f8d7c27e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8d741142a0 con 0x7f8d741009b0 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.085+0000 7f8d7a81b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7410e4d0 0x7f8d7419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.085+0000 7f8d7a81b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7410e4d0 0x7f8d7419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41502/0 (socket says 172.21.15.5:41502) 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.085+0000 7f8d7a81b640 1 -- 172.21.15.5:0/3840890849 learned_addr learned my addr 172.21.15.5:0/3840890849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.085+0000 7f8d79819640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d74101360 0x7f8d7419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.085+0000 7f8d7a01a640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d741009b0 0x7f8d7419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.086+0000 7f8d79819640 1 -- 172.21.15.5:0/3840890849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d741009b0 msgr2=0x7f8d7419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.086+0000 7f8d79819640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d741009b0 0x7f8d7419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.086+0000 7f8d79819640 1 -- 172.21.15.5:0/3840890849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7410e4d0 msgr2=0x7f8d7419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.086+0000 7f8d79819640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7410e4d0 0x7f8d7419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.086+0000 7f8d79819640 1 -- 172.21.15.5:0/3840890849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7419f670 con 0x7f8d74101360 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.086+0000 7f8d7a81b640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7410e4d0 0x7f8d7419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.086+0000 7f8d79819640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d74101360 0x7f8d7419ac00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400e9f0 tx=0x7f8d6400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.086+0000 7f8d6b7fe640 1 -- 172.21.15.5:0/3840890849 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400ccc0 con 0x7f8d74101360 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.086+0000 7f8d6b7fe640 1 -- 172.21.15.5:0/3840890849 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d64005020 con 0x7f8d74101360 2024-09-19T06:04:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.086+0000 7f8d7c27e640 1 -- 172.21.15.5:0/3840890849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7419f960 con 0x7f8d74101360 2024-09-19T06:04:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.086+0000 7f8d7c27e640 1 -- 172.21.15.5:0/3840890849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d741a7240 con 0x7f8d74101360 2024-09-19T06:04:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.087+0000 7f8d6b7fe640 1 -- 172.21.15.5:0/3840890849 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64004a00 con 0x7f8d74101360 2024-09-19T06:04:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.089+0000 7f8d7e657740 1 Processor -- start 2024-09-19T06:04:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.089+0000 7f8d7e657740 1 -- start start 2024-09-19T06:04:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7e657740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e0d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7e657740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e0d5d0 0x55ccd4e0da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7e657740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e18370 0x55ccd4e1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7e657740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ccd4d732f0 con 0x55ccd4e18370 2024-09-19T06:04:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7e657740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ccd4d1acf0 con 0x55ccd4e0eb40 2024-09-19T06:04:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7e657740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ccd4d6f2b0 con 0x55ccd4e0d5d0 2024-09-19T06:04:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7a01a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e0d5d0 0x55ccd4e0da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7a01a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e0d5d0 0x55ccd4e0da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52258/0 (socket says 172.21.15.5:52258) 2024-09-19T06:04:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7a81b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e0d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7a01a640 1 -- 172.21.15.5:0/872979412 learned_addr learned my addr 172.21.15.5:0/872979412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d6b7fe640 1 -- 172.21.15.5:0/3840890849 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7f8d64004ba0 con 0x7f8d74101360 2024-09-19T06:04:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7a01a640 1 -- 172.21.15.5:0/872979412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e0eb40 msgr2=0x55ccd4e0d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7a01a640 1 --2- 172.21.15.5:0/872979412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e0d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.090+0000 7f8d7a01a640 1 -- 172.21.15.5:0/872979412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e18370 msgr2=0x55ccd4e1a760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:04:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.091+0000 7f8d79819640 1 --2- 172.21.15.5:0/872979412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e18370 0x55ccd4e1a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.091+0000 7f8d7a01a640 1 --2- 172.21.15.5:0/872979412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e18370 0x55ccd4e1a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.091+0000 7f8d7a01a640 1 -- 172.21.15.5:0/872979412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccd4cff3a0 con 0x55ccd4e0d5d0 2024-09-19T06:04:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.091+0000 7f8d79819640 1 --2- 172.21.15.5:0/872979412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e18370 0x55ccd4e1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:04:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.091+0000 7f8d7a01a640 1 --2- 172.21.15.5:0/872979412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e0d5d0 0x55ccd4e0da10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009cf0 tx=0x7f8d70011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2920e55eb8e2564d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.091+0000 7f8d68dea640 1 -- 172.21.15.5:0/872979412 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70004230 con 0x55ccd4e0d5d0 2024-09-19T06:04:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.092+0000 7f8d68dea640 1 -- 172.21.15.5:0/872979412 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d700043d0 con 0x55ccd4e0d5d0 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.092+0000 7f8d68dea640 1 -- 172.21.15.5:0/872979412 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d700197e0 con 0x55ccd4e0d5d0 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.092+0000 7f8d6b7fe640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8d600a3550 0x7f8d600a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.092+0000 7f8d7a81b640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8d600a3550 0x7f8d600a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.092+0000 7f8d7e657740 1 -- 172.21.15.5:0/872979412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e0d5d0 msgr2=0x55ccd4e0da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.092+0000 7f8d7e657740 1 --2- 172.21.15.5:0/872979412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e0d5d0 0x55ccd4e0da10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009cf0 tx=0x7f8d70011810 comp rx=0 tx=0).stop 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.092+0000 7f8d6b7fe640 1 -- 172.21.15.5:0/3840890849 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f8d640cfe50 con 0x7f8d74101360 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.092+0000 7f8d7e657740 1 -- 172.21.15.5:0/872979412 shutdown_connections 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.092+0000 7f8d7e657740 1 --2- 172.21.15.5:0/872979412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e18370 0x55ccd4e1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.092+0000 7f8d7e657740 1 --2- 172.21.15.5:0/872979412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e0d5d0 0x55ccd4e0da10 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009cf0 tx=0x7f8d70011810 comp rx=0 tx=0).stop 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.092+0000 7f8d7e657740 1 --2- 172.21.15.5:0/872979412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e0d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.092+0000 7f8d7e657740 1 -- 172.21.15.5:0/872979412 >> 172.21.15.5:0/872979412 conn(0x55ccd4dd6310 msgr2=0x55ccd4e09c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.093+0000 7f8d7e657740 1 -- 172.21.15.5:0/872979412 shutdown_connections 2024-09-19T06:04:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.093+0000 7f8d7e657740 1 -- 172.21.15.5:0/872979412 wait complete. 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.093+0000 7f8d7a81b640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8d600a3550 0x7f8d600a5a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0319e0 tx=0x7f8d6c009360 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.093+0000 7f8d7e657740 1 -- start start 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.094+0000 7f8d7e657740 1 Processor -- start 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.094+0000 7f8d7e657740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e15210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.094+0000 7f8d7e657740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e18370 0x55ccd4e15750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.094+0000 7f8d7e657740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e15c90 0x55ccd4e24520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.094+0000 7f8d7e657740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ccd4e1b070 con 0x55ccd4e0eb40 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.094+0000 7f8d7e657740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ccd4d2e060 con 0x55ccd4e18370 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.094+0000 7f8d7e657740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ccd4e1b1f0 con 0x55ccd4e15c90 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.094+0000 7f8d7e657740 10 client.0 _ll_register_callbacks cb 0x55ccd4a818c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.094+0000 7f8d7a01a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e15210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.094+0000 7f8d7a01a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e15210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41512/0 (socket says 172.21.15.5:41512) 2024-09-19T06:04:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.094+0000 7f8d7a01a640 1 -- 172.21.15.5:0/2401499469 learned_addr learned my addr 172.21.15.5:0/2401499469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.094+0000 7f8d7a81b640 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e15c90 0x55ccd4e24520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.095+0000 7f8d79819640 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e18370 0x55ccd4e15750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.095+0000 7f8d7a81b640 1 -- 172.21.15.5:0/2401499469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e18370 msgr2=0x55ccd4e15750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.095+0000 7f8d7a81b640 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e18370 0x55ccd4e15750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.095+0000 7f8d7a81b640 1 -- 172.21.15.5:0/2401499469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e0eb40 msgr2=0x55ccd4e15210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.095+0000 7f8d7a81b640 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e15210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.095+0000 7f8d7a81b640 1 -- 172.21.15.5:0/2401499469 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccd4ae43c0 con 0x55ccd4e15c90 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.095+0000 7f8d7a81b640 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e15c90 0x55ccd4e24520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c006010 tx=0x7f8d6c02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc6839a7cba67e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.095+0000 7f8d5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.096+0000 7f8d5dffb640 1 -- 172.21.15.5:0/2401499469 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c004250 con 0x55ccd4e15c90 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.096+0000 7f8d7e657740 1 -- 172.21.15.5:0/2401499469 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d700043d0 con 0x55ccd4e15c90 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.096+0000 7f8d5dffb640 1 -- 172.21.15.5:0/2401499469 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f8d6c0043f0 con 0x55ccd4e15c90 2024-09-19T06:04:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.096+0000 7f8d5dffb640 1 -- 172.21.15.5:0/2401499469 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d6c0045f0 con 0x55ccd4e15c90 2024-09-19T06:04:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.096+0000 7f8d7e657740 10 client.25087 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.096+0000 7f8d7e657740 1 -- 172.21.15.5:0/2401499469 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccd4e27050 con 0x55ccd4e15c90 2024-09-19T06:04:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.096+0000 7f8d5dffb640 1 -- 172.21.15.5:0/2401499469 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d700043d0 con 0x55ccd4e15c90 2024-09-19T06:04:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.096+0000 7f8d5dffb640 1 -- 172.21.15.5:0/2401499469 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8d6c051070 con 0x55ccd4e15c90 2024-09-19T06:04:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.096+0000 7f8d7e657740 10 client.25087 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.096+0000 7f8d7e657740 10 client.25087 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:04:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.097+0000 7f8d7e657740 20 client.25087 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.097+0000 7f8d7e657740 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55ccd4e284b0 0x55ccd4e2a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.097+0000 7f8d7e657740 4 client.25087 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.097+0000 7f8d7e657740 1 -- 172.21.15.5:0/2401499469 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccd4d80860 con 0x55ccd4e284b0 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.097+0000 7f8d7a01a640 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55ccd4e284b0 0x55ccd4e2a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.097+0000 7f8d5dffb640 10 client.25087 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.097+0000 7f8d7a01a640 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55ccd4e284b0 0x55ccd4e2a900 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=44eb9fd87efc88 server_cookie=9ebcedb11040f82e in_seq=0 out_seq=0 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.098+0000 7f8d5dffb640 1 -- 172.21.15.5:0/2401499469 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ccd4d80860 con 0x55ccd4e284b0 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.098+0000 7f8d5dffb640 10 client.25087 handle_command_reply: tid=0 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.098+0000 7f8d7e657740 1 client.25087 shutdown 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.098+0000 7f8d7e657740 1 -- 172.21.15.5:0/2401499469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e15c90 msgr2=0x55ccd4e24520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.098+0000 7f8d7e657740 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e15c90 0x55ccd4e24520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c006010 tx=0x7f8d6c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.098+0000 7f8d7e657740 1 -- 172.21.15.5:0/2401499469 shutdown_connections 2024-09-19T06:04:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.098+0000 7f8d7e657740 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55ccd4e284b0 0x55ccd4e2a900 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.098+0000 7f8d7e657740 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e15c90 0x55ccd4e24520 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.098+0000 7f8d7e657740 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e18370 0x55ccd4e15750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.098+0000 7f8d5dffb640 0 client.25087 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:04:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.098+0000 7f8d7e657740 1 --2- 172.21.15.5:0/2401499469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e15210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.098+0000 7f8d7e657740 1 -- 172.21.15.5:0/2401499469 >> 172.21.15.5:0/2401499469 conn(0x55ccd4dd6310 msgr2=0x55ccd4e0f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.099+0000 7f8d7e657740 1 -- 172.21.15.5:0/2401499469 shutdown_connections 2024-09-19T06:04:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.099+0000 7f8d7e657740 1 -- 172.21.15.5:0/2401499469 wait complete. 2024-09-19T06:04:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.099+0000 7f8d7e657740 20 client.25087 trim_cache size 0 max 16384 2024-09-19T06:04:07.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.124+0000 7f8d7e657740 1 Processor -- start 2024-09-19T06:04:07.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.124+0000 7f8d7e657740 1 -- start start 2024-09-19T06:04:07.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d7e657740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e13ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d7e657740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e15c90 0x55ccd4e10120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d7e657740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e18370 0x55ccd4e10660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d7e657740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d741af980 con 0x55ccd4e15c90 2024-09-19T06:04:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d7e657740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8d741b14c0 con 0x55ccd4e0eb40 2024-09-19T06:04:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d7e657740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8d741b0050 con 0x55ccd4e18370 2024-09-19T06:04:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d79819640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e15c90 0x55ccd4e10120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d79819640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e15c90 0x55ccd4e10120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41528/0 (socket says 172.21.15.5:41528) 2024-09-19T06:04:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d79819640 1 -- 172.21.15.5:0/3608603268 learned_addr learned my addr 172.21.15.5:0/3608603268 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d7a81b640 1 --2- 172.21.15.5:0/3608603268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e18370 0x55ccd4e10660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d7a01a640 1 --2- 172.21.15.5:0/3608603268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e13ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d79819640 1 -- 172.21.15.5:0/3608603268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e18370 msgr2=0x55ccd4e10660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d79819640 1 --2- 172.21.15.5:0/3608603268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e18370 0x55ccd4e10660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d79819640 1 -- 172.21.15.5:0/3608603268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e0eb40 msgr2=0x55ccd4e13ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:07.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d79819640 1 --2- 172.21.15.5:0/3608603268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e13ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.125+0000 7f8d79819640 1 -- 172.21.15.5:0/3608603268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccd4e094f0 con 0x55ccd4e15c90 2024-09-19T06:04:07.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.126+0000 7f8d79819640 1 --2- 172.21.15.5:0/3608603268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e15c90 0x55ccd4e10120 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400c6a0 tx=0x7f8d6409bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70407fdef415ced9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:07.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.126+0000 7f8d5ffff640 1 -- 172.21.15.5:0/3608603268 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d640a0280 con 0x55ccd4e15c90 2024-09-19T06:04:07.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.126+0000 7f8d5ffff640 1 -- 172.21.15.5:0/3608603268 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d640a4b40 con 0x55ccd4e15c90 2024-09-19T06:04:07.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.126+0000 7f8d5ffff640 1 -- 172.21.15.5:0/3608603268 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d640a46b0 con 0x55ccd4e15c90 2024-09-19T06:04:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.127+0000 7f8d7e657740 1 -- 172.21.15.5:0/3608603268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e15c90 msgr2=0x55ccd4e10120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.127+0000 7f8d7e657740 1 --2- 172.21.15.5:0/3608603268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e15c90 0x55ccd4e10120 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400c6a0 tx=0x7f8d6409bad0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.127+0000 7f8d7e657740 1 -- 172.21.15.5:0/3608603268 shutdown_connections 2024-09-19T06:04:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.127+0000 7f8d7e657740 1 --2- 172.21.15.5:0/3608603268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e18370 0x55ccd4e10660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.127+0000 7f8d7e657740 1 --2- 172.21.15.5:0/3608603268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e15c90 0x55ccd4e10120 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.127+0000 7f8d7e657740 1 --2- 172.21.15.5:0/3608603268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e13ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.127+0000 7f8d7e657740 1 -- 172.21.15.5:0/3608603268 >> 172.21.15.5:0/3608603268 conn(0x55ccd4dd6310 msgr2=0x55ccd4e17290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.127+0000 7f8d7e657740 1 -- 172.21.15.5:0/3608603268 shutdown_connections 2024-09-19T06:04:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.127+0000 7f8d7e657740 1 -- 172.21.15.5:0/3608603268 wait complete. 2024-09-19T06:04:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.127+0000 7f8d7e657740 1 -- start start 2024-09-19T06:04:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.127+0000 7f8d7e657740 1 Processor -- start 2024-09-19T06:04:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.128+0000 7f8d7e657740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e20480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.128+0000 7f8d7e657740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e15c90 0x55ccd4e209c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.128+0000 7f8d7e657740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e18370 0x55ccd4e11480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.128+0000 7f8d7e657740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d741b0df0 con 0x55ccd4e0eb40 2024-09-19T06:04:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.128+0000 7f8d7e657740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8d741b0720 con 0x55ccd4e18370 2024-09-19T06:04:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.128+0000 7f8d7e657740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8d741ad960 con 0x55ccd4e15c90 2024-09-19T06:04:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.128+0000 7f8d7e657740 10 client.0 _ll_register_callbacks cb 0x7f8d6c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.128+0000 7f8d7a81b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e18370 0x55ccd4e11480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.128+0000 7f8d79819640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e15c90 0x55ccd4e209c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.128+0000 7f8d7a81b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e18370 0x55ccd4e11480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55990/0 (socket says 172.21.15.5:55990) 2024-09-19T06:04:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.128+0000 7f8d7a81b640 1 -- 172.21.15.5:0/3703064695 learned_addr learned my addr 172.21.15.5:0/3703064695 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.128+0000 7f8d7a01a640 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e20480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d7a81b640 1 -- 172.21.15.5:0/3703064695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e15c90 msgr2=0x55ccd4e209c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d7a81b640 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e15c90 0x55ccd4e209c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d7a81b640 1 -- 172.21.15.5:0/3703064695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e0eb40 msgr2=0x55ccd4e20480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d7a81b640 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e20480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d7a81b640 1 -- 172.21.15.5:0/3703064695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccd4e1a2e0 con 0x55ccd4e18370 2024-09-19T06:04:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d79819640 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e15c90 0x55ccd4e209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d7a81b640 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e18370 0x55ccd4e11480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005e30 tx=0x7f8d6c009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5edd4fd4ac3671 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:04:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d5dffb640 1 -- 172.21.15.5:0/3703064695 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c053700 con 0x55ccd4e18370 2024-09-19T06:04:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d7e657740 1 -- 172.21.15.5:0/3703064695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d640a4b40 con 0x55ccd4e18370 2024-09-19T06:04:07.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d5dffb640 1 -- 172.21.15.5:0/3703064695 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f8d6c0538a0 con 0x55ccd4e18370 2024-09-19T06:04:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d5dffb640 1 -- 172.21.15.5:0/3703064695 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d6c068050 con 0x55ccd4e18370 2024-09-19T06:04:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d7e657740 10 client.25139 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.129+0000 7f8d7e657740 1 -- 172.21.15.5:0/3703064695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccd4e129c0 con 0x55ccd4e18370 2024-09-19T06:04:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.130+0000 7f8d5dffb640 1 -- 172.21.15.5:0/3703064695 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d640a4b40 con 0x55ccd4e18370 2024-09-19T06:04:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.130+0000 7f8d5dffb640 1 -- 172.21.15.5:0/3703064695 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8d6c051040 con 0x55ccd4e18370 2024-09-19T06:04:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.130+0000 7f8d7e657740 10 client.25139 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.130+0000 7f8d7e657740 10 client.25139 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:04:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.130+0000 7f8d7e657740 20 client.25139 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.130+0000 7f8d7e657740 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55ccd4e941b0 0x55ccd4e965a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.130+0000 7f8d7e657740 4 client.25139 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:04:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.130+0000 7f8d7e657740 1 -- 172.21.15.5:0/3703064695 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccd4d7ad70 con 0x55ccd4e941b0 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.130+0000 7f8d7a01a640 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55ccd4e941b0 0x55ccd4e965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7a01a640 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55ccd4e941b0 0x55ccd4e965a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c4d6c3a6cc3e927 server_cookie=422f9529fb16e6a1 in_seq=0 out_seq=0 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d5dffb640 10 client.25139 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d5dffb640 1 -- 172.21.15.5:0/3703064695 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccd4d7ad70 con 0x55ccd4e941b0 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d5dffb640 10 client.25139 handle_command_reply: tid=0 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7e657740 1 client.25139 shutdown 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7e657740 1 -- 172.21.15.5:0/3703064695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e18370 msgr2=0x55ccd4e11480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7e657740 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e18370 0x55ccd4e11480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005e30 tx=0x7f8d6c009960 comp rx=0 tx=0).stop 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7e657740 1 -- 172.21.15.5:0/3703064695 shutdown_connections 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7e657740 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55ccd4e941b0 0x55ccd4e965a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7e657740 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccd4e18370 0x55ccd4e11480 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7e657740 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd4e15c90 0x55ccd4e209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d5dffb640 0 client.25139 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:04:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7e657740 1 --2- 172.21.15.5:0/3703064695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd4e0eb40 0x55ccd4e20480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7e657740 1 -- 172.21.15.5:0/3703064695 >> 172.21.15.5:0/3703064695 conn(0x55ccd4dd6310 msgr2=0x55ccd4e084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7e657740 1 -- 172.21.15.5:0/3703064695 shutdown_connections 2024-09-19T06:04:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7e657740 1 -- 172.21.15.5:0/3703064695 wait complete. 2024-09-19T06:04:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.131+0000 7f8d7e657740 20 client.25139 trim_cache size 0 max 16384 2024-09-19T06:04:07.137 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:04:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.132+0000 7f8d7c27e640 1 -- 172.21.15.5:0/3840890849 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8d600a3550 msgr2=0x7f8d600a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.132+0000 7f8d7c27e640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8d600a3550 0x7f8d600a5a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0319e0 tx=0x7f8d6c009360 comp rx=0 tx=0).stop 2024-09-19T06:04:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.132+0000 7f8d7c27e640 1 -- 172.21.15.5:0/3840890849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d74101360 msgr2=0x7f8d7419ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.132+0000 7f8d7c27e640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d74101360 0x7f8d7419ac00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400e9f0 tx=0x7f8d6400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.132+0000 7f8d7c27e640 1 -- 172.21.15.5:0/3840890849 shutdown_connections 2024-09-19T06:04:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.132+0000 7f8d7c27e640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7410e4d0 0x7f8d7419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.132+0000 7f8d7c27e640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d74101360 0x7f8d7419ac00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.132+0000 7f8d7c27e640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8d600a3550 0x7f8d600a5a10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.132+0000 7f8d7c27e640 1 --2- 172.21.15.5:0/3840890849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d741009b0 0x7f8d7419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.132+0000 7f8d7c27e640 1 -- 172.21.15.5:0/3840890849 >> 172.21.15.5:0/3840890849 conn(0x7f8d740fc7d0 msgr2=0x7f8d740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.132+0000 7f8d7c27e640 1 -- 172.21.15.5:0/3840890849 shutdown_connections 2024-09-19T06:04:07.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:07.133+0000 7f8d7c27e640 1 -- 172.21.15.5:0/3840890849 wait complete. 2024-09-19T06:04:07.804 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:04:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:08.489 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:08 smithi005 ceph-mon[30955]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T06:04:08.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:08 smithi157 ceph-mon[35346]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T06:04:08.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.588+0000 7f261c41e640 1 -- 172.21.15.5:0/3392756989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2614102570 msgr2=0x7f26141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:08.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.588+0000 7f261c41e640 1 --2- 172.21.15.5:0/3392756989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2614102570 0x7f26141029b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2604009900 tx=0x7f260402a480 comp rx=0 tx=0).stop 2024-09-19T06:04:08.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.588+0000 7f261c41e640 1 -- 172.21.15.5:0/3392756989 shutdown_connections 2024-09-19T06:04:08.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.588+0000 7f261c41e640 1 --2- 172.21.15.5:0/3392756989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2614102ef0 0x7f261410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.588+0000 7f261c41e640 1 --2- 172.21.15.5:0/3392756989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2614102570 0x7f26141029b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.588+0000 7f261c41e640 1 --2- 172.21.15.5:0/3392756989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2614100e20 0x7f2614101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.588+0000 7f261c41e640 1 -- 172.21.15.5:0/3392756989 >> 172.21.15.5:0/3392756989 conn(0x7f26140fc7d0 msgr2=0x7f26140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.588+0000 7f261c41e640 1 -- 172.21.15.5:0/3392756989 shutdown_connections 2024-09-19T06:04:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.588+0000 7f261c41e640 1 -- 172.21.15.5:0/3392756989 wait complete. 2024-09-19T06:04:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f261c41e640 1 Processor -- start 2024-09-19T06:04:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f261c41e640 1 -- start start 2024-09-19T06:04:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f261c41e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2614100e20 0x7f2614197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f261c41e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2614102570 0x7f26141975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f261c41e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2614102ef0 0x7f2614197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f261c41e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2614110e00 con 0x7f2614102570 2024-09-19T06:04:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f261c41e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2614110c80 con 0x7f2614100e20 2024-09-19T06:04:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f261c41e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2614110f80 con 0x7f2614102ef0 2024-09-19T06:04:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f26199b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2614102570 0x7f26141975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f26199b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2614102570 0x7f26141975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41552/0 (socket says 172.21.15.5:41552) 2024-09-19T06:04:08.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f26199b9640 1 -- 172.21.15.5:0/1579542811 learned_addr learned my addr 172.21.15.5:0/1579542811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:08.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f261a9bb640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2614102ef0 0x7f2614197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.589+0000 7f261a1ba640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2614100e20 0x7f2614197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.590+0000 7f26199b9640 1 -- 172.21.15.5:0/1579542811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2614102ef0 msgr2=0x7f2614197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.590+0000 7f26199b9640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2614102ef0 0x7f2614197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.590+0000 7f26199b9640 1 -- 172.21.15.5:0/1579542811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2614100e20 msgr2=0x7f2614197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.590+0000 7f26199b9640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2614100e20 0x7f2614197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.590+0000 7f26199b9640 1 -- 172.21.15.5:0/1579542811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26141a68d0 con 0x7f2614102570 2024-09-19T06:04:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.590+0000 7f261a1ba640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2614100e20 0x7f2614197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.590+0000 7f26199b9640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2614102570 0x7f26141975a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2604006010 tx=0x7f26040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.590+0000 7f260b7fe640 1 -- 172.21.15.5:0/1579542811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260403f030 con 0x7f2614102570 2024-09-19T06:04:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.590+0000 7f260b7fe640 1 -- 172.21.15.5:0/1579542811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2604003cb0 con 0x7f2614102570 2024-09-19T06:04:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.590+0000 7f261c41e640 1 -- 172.21.15.5:0/1579542811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26141a6b60 con 0x7f2614102570 2024-09-19T06:04:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.590+0000 7f260b7fe640 1 -- 172.21.15.5:0/1579542811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604035070 con 0x7f2614102570 2024-09-19T06:04:08.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.591+0000 7f261c41e640 1 -- 172.21.15.5:0/1579542811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26141a6fd0 con 0x7f2614102570 2024-09-19T06:04:08.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.593+0000 7f260b7fe640 1 -- 172.21.15.5:0/1579542811 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7f2604042030 con 0x7f2614102570 2024-09-19T06:04:08.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.593+0000 7f260b7fe640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26000a35d0 0x7f26000a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.594+0000 7f261a1ba640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26000a35d0 0x7f26000a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.594+0000 7f260b7fe640 1 -- 172.21.15.5:0/1579542811 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f26040f0130 con 0x7f2614102570 2024-09-19T06:04:08.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.594+0000 7f261e7f7740 1 Processor -- start 2024-09-19T06:04:08.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.594+0000 7f261e7f7740 1 -- start start 2024-09-19T06:04:08.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.594+0000 7f261a1ba640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26000a35d0 0x7f26000a5a90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2610005d00 tx=0x7f2610005c70 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:08.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f261e7f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 0x558e1c089090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f261e7f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c089a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f261e7f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c096760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f261e7f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e1bfef2f0 con 0x558e1c0895d0 2024-09-19T06:04:08.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f261e7f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558e1bf96cf0 con 0x558e1c08ab40 2024-09-19T06:04:08.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f261e7f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558e1bfeb2b0 con 0x558e1c094370 2024-09-19T06:04:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f26199b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c089a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f26199b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c089a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41568/0 (socket says 172.21.15.5:41568) 2024-09-19T06:04:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f26199b9640 1 -- 172.21.15.5:0/214054639 learned_addr learned my addr 172.21.15.5:0/214054639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f261a1ba640 1 --2- 172.21.15.5:0/214054639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 0x558e1c089090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f26199b9640 1 -- 172.21.15.5:0/214054639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 msgr2=0x558e1c096760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:04:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f26199b9640 1 --2- 172.21.15.5:0/214054639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c096760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f26199b9640 1 -- 172.21.15.5:0/214054639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 msgr2=0x558e1c089090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f26199b9640 1 --2- 172.21.15.5:0/214054639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 0x558e1c089090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f26199b9640 1 -- 172.21.15.5:0/214054639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e1bf7b3a0 con 0x558e1c0895d0 2024-09-19T06:04:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f26199b9640 1 --2- 172.21.15.5:0/214054639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c089a10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2604005ab0 tx=0x7f26040bbba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d840ec153b96ad07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.595+0000 7f2608dea640 1 -- 172.21.15.5:0/214054639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604049040 con 0x558e1c0895d0 2024-09-19T06:04:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.596+0000 7f2608dea640 1 -- 172.21.15.5:0/214054639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26040bc240 con 0x558e1c0895d0 2024-09-19T06:04:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.596+0000 7f2608dea640 1 -- 172.21.15.5:0/214054639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26040bc4a0 con 0x558e1c0895d0 2024-09-19T06:04:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.597+0000 7f261e7f7740 1 -- 172.21.15.5:0/214054639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 msgr2=0x558e1c089a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.597+0000 7f261e7f7740 1 --2- 172.21.15.5:0/214054639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c089a10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2604005ab0 tx=0x7f26040bbba0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.597+0000 7f261e7f7740 1 -- 172.21.15.5:0/214054639 shutdown_connections 2024-09-19T06:04:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.597+0000 7f261e7f7740 1 --2- 172.21.15.5:0/214054639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c096760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.597+0000 7f261e7f7740 1 --2- 172.21.15.5:0/214054639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c089a10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.597+0000 7f261e7f7740 1 --2- 172.21.15.5:0/214054639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 0x558e1c089090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.597+0000 7f261e7f7740 1 -- 172.21.15.5:0/214054639 >> 172.21.15.5:0/214054639 conn(0x558e1c052310 msgr2=0x558e1c085c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.597+0000 7f261e7f7740 1 -- 172.21.15.5:0/214054639 shutdown_connections 2024-09-19T06:04:08.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.597+0000 7f261e7f7740 1 -- 172.21.15.5:0/214054639 wait complete. 2024-09-19T06:04:08.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.598+0000 7f261e7f7740 1 -- start start 2024-09-19T06:04:08.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.598+0000 7f261e7f7740 1 Processor -- start 2024-09-19T06:04:08.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.598+0000 7f261e7f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c093e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.598+0000 7f261e7f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c08ab40 0x558e1c0910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.598+0000 7f261e7f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c094370 0x558e1c091870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.598+0000 7f261e7f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e1c096e30 con 0x558e1c0895d0 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.598+0000 7f261e7f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558e1bfaa060 con 0x558e1c094370 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.598+0000 7f261e7f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558e1c096fb0 con 0x558e1c08ab40 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.598+0000 7f261a1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c093e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.598+0000 7f261e7f7740 10 client.0 _ll_register_callbacks cb 0x558e1bcfd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.598+0000 7f261a1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c093e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41576/0 (socket says 172.21.15.5:41576) 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.598+0000 7f261a1ba640 1 -- 172.21.15.5:0/2008551318 learned_addr learned my addr 172.21.15.5:0/2008551318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.599+0000 7f26199b9640 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c08ab40 0x558e1c0910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.599+0000 7f261a9bb640 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c094370 0x558e1c091870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.599+0000 7f261a1ba640 1 -- 172.21.15.5:0/2008551318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c08ab40 msgr2=0x558e1c0910e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.599+0000 7f261a1ba640 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c08ab40 0x558e1c0910e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.599+0000 7f261a1ba640 1 -- 172.21.15.5:0/2008551318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c094370 msgr2=0x558e1c091870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.599+0000 7f261a1ba640 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c094370 0x558e1c091870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.599+0000 7f261a1ba640 1 -- 172.21.15.5:0/2008551318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e1bd603c0 con 0x558e1c0895d0 2024-09-19T06:04:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.599+0000 7f261a1ba640 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c093e30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f261000b9c0 tx=0x7f26100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0131e3f8e03ee3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.599+0000 7f25fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.599+0000 7f25fdffb640 1 -- 172.21.15.5:0/2008551318 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610024070 con 0x558e1c0895d0 2024-09-19T06:04:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f261e7f7740 1 -- 172.21.15.5:0/2008551318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26040bc240 con 0x558e1c0895d0 2024-09-19T06:04:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f25fdffb640 1 -- 172.21.15.5:0/2008551318 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f2610011640 con 0x558e1c0895d0 2024-09-19T06:04:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f25fdffb640 1 -- 172.21.15.5:0/2008551318 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2610030050 con 0x558e1c0895d0 2024-09-19T06:04:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f261e7f7740 10 client.15531 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f261e7f7740 1 -- 172.21.15.5:0/2008551318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e1c0a2920 con 0x558e1c0895d0 2024-09-19T06:04:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f25fdffb640 1 -- 172.21.15.5:0/2008551318 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26040bc240 con 0x558e1c0895d0 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f25fdffb640 1 -- 172.21.15.5:0/2008551318 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f26100209b0 con 0x558e1c0895d0 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f261e7f7740 10 client.15531 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f261e7f7740 10 client.15531 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f261e7f7740 20 client.15531 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f261e7f7740 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x558e1c0a4160 0x558e1c0a6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f261e7f7740 4 client.15531 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.600+0000 7f261e7f7740 1 -- 172.21.15.5:0/2008551318 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e1bffc860 con 0x558e1c0a4160 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.601+0000 7f26199b9640 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x558e1c0a4160 0x558e1c0a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.601+0000 7f25fdffb640 10 client.15531 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.601+0000 7f26199b9640 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x558e1c0a4160 0x558e1c0a6600 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d9bfcdff17de788e server_cookie=dd37cb417f26b8d6 in_seq=0 out_seq=0 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.602+0000 7f25fdffb640 1 -- 172.21.15.5:0/2008551318 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e1bffc860 con 0x558e1c0a4160 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.602+0000 7f25fdffb640 10 client.15531 handle_command_reply: tid=0 2024-09-19T06:04:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.602+0000 7f261e7f7740 1 client.15531 shutdown 2024-09-19T06:04:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.602+0000 7f261e7f7740 1 -- 172.21.15.5:0/2008551318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 msgr2=0x558e1c093e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.602+0000 7f261e7f7740 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c093e30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f261000b9c0 tx=0x7f26100046c0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.602+0000 7f261e7f7740 1 -- 172.21.15.5:0/2008551318 shutdown_connections 2024-09-19T06:04:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.602+0000 7f261e7f7740 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x558e1c0a4160 0x558e1c0a6600 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.602+0000 7f261e7f7740 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c094370 0x558e1c091870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.602+0000 7f261e7f7740 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c08ab40 0x558e1c0910e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.602+0000 7f261e7f7740 1 --2- 172.21.15.5:0/2008551318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c093e30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.602+0000 7f261e7f7740 1 -- 172.21.15.5:0/2008551318 >> 172.21.15.5:0/2008551318 conn(0x558e1c052310 msgr2=0x558e1c08b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.602+0000 7f25fdffb640 0 client.15531 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:04:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.603+0000 7f261e7f7740 1 -- 172.21.15.5:0/2008551318 shutdown_connections 2024-09-19T06:04:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.603+0000 7f261e7f7740 1 -- 172.21.15.5:0/2008551318 wait complete. 2024-09-19T06:04:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.603+0000 7f261e7f7740 20 client.15531 trim_cache size 0 max 16384 2024-09-19T06:04:08.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.620+0000 7f261e7f7740 1 Processor -- start 2024-09-19T06:04:08.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261e7f7740 1 -- start start 2024-09-19T06:04:08.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261e7f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c09fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261e7f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 0x558e1c0abed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261e7f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c0ac410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261e7f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26141af950 con 0x558e1c0895d0 2024-09-19T06:04:08.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261e7f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f26141b1490 con 0x558e1c08ab40 2024-09-19T06:04:08.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261e7f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26141b0020 con 0x558e1c094370 2024-09-19T06:04:08.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261a1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c09fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261a1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c09fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41586/0 (socket says 172.21.15.5:41586) 2024-09-19T06:04:08.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261a1ba640 1 -- 172.21.15.5:0/582908674 learned_addr learned my addr 172.21.15.5:0/582908674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:08.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f26199b9640 1 --2- 172.21.15.5:0/582908674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 0x558e1c0abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261a9bb640 1 --2- 172.21.15.5:0/582908674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c0ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261a1ba640 1 -- 172.21.15.5:0/582908674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 msgr2=0x558e1c0ac410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:08.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261a1ba640 1 --2- 172.21.15.5:0/582908674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c0ac410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261a1ba640 1 -- 172.21.15.5:0/582908674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 msgr2=0x558e1c0abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:08.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261a1ba640 1 --2- 172.21.15.5:0/582908674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 0x558e1c0abed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261a1ba640 1 -- 172.21.15.5:0/582908674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e1c091be0 con 0x558e1c0895d0 2024-09-19T06:04:08.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.621+0000 7f261a9bb640 1 --2- 172.21.15.5:0/582908674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c0ac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:04:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.622+0000 7f261a1ba640 1 --2- 172.21.15.5:0/582908674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c09fbc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2610020e00 tx=0x7f2610004030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1156e0da782d2219 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.622+0000 7f25fffff640 1 -- 172.21.15.5:0/582908674 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610038070 con 0x558e1c0895d0 2024-09-19T06:04:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.622+0000 7f25fffff640 1 -- 172.21.15.5:0/582908674 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2610010cf0 con 0x558e1c0895d0 2024-09-19T06:04:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.622+0000 7f25fffff640 1 -- 172.21.15.5:0/582908674 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610024040 con 0x558e1c0895d0 2024-09-19T06:04:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.622+0000 7f261e7f7740 1 -- 172.21.15.5:0/582908674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 msgr2=0x558e1c09fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.622+0000 7f261e7f7740 1 --2- 172.21.15.5:0/582908674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c09fbc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2610020e00 tx=0x7f2610004030 comp rx=0 tx=0).stop 2024-09-19T06:04:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.623+0000 7f261e7f7740 1 -- 172.21.15.5:0/582908674 shutdown_connections 2024-09-19T06:04:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.623+0000 7f261e7f7740 1 --2- 172.21.15.5:0/582908674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c0ac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.623+0000 7f261e7f7740 1 --2- 172.21.15.5:0/582908674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 0x558e1c0abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.623+0000 7f261e7f7740 1 --2- 172.21.15.5:0/582908674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c09fbc0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.623+0000 7f261e7f7740 1 -- 172.21.15.5:0/582908674 >> 172.21.15.5:0/582908674 conn(0x558e1c052310 msgr2=0x558e1c08f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.623+0000 7f261e7f7740 1 -- 172.21.15.5:0/582908674 shutdown_connections 2024-09-19T06:04:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.623+0000 7f261e7f7740 1 -- 172.21.15.5:0/582908674 wait complete. 2024-09-19T06:04:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.623+0000 7f261e7f7740 1 -- start start 2024-09-19T06:04:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.623+0000 7f261e7f7740 1 Processor -- start 2024-09-19T06:04:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.623+0000 7f261e7f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c097470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.623+0000 7f261e7f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 0x558e1c0979b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261e7f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c10f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261a1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c097470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261e7f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26141b0dc0 con 0x558e1c0895d0 2024-09-19T06:04:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261e7f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f26141b06f0 con 0x558e1c08ab40 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261a1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c097470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41598/0 (socket says 172.21.15.5:41598) 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261e7f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26141ad930 con 0x558e1c094370 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261a1ba640 1 -- 172.21.15.5:0/1824448588 learned_addr learned my addr 172.21.15.5:0/1824448588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261e7f7740 10 client.0 _ll_register_callbacks cb 0x7f261000b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261a9bb640 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c10f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261a1ba640 1 -- 172.21.15.5:0/1824448588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 msgr2=0x558e1c10f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261a1ba640 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c10f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261a1ba640 1 -- 172.21.15.5:0/1824448588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 msgr2=0x558e1c0979b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261a1ba640 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 0x558e1c0979b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261a1ba640 1 -- 172.21.15.5:0/1824448588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e1c10fce0 con 0x558e1c0895d0 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261a9bb640 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c10f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f261a1ba640 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c097470 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f261000fc30 tx=0x7f2610010750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44817f7211dac39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.624+0000 7f25fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.625+0000 7f25fdffb640 1 -- 172.21.15.5:0/1824448588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610041020 con 0x558e1c0895d0 2024-09-19T06:04:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.625+0000 7f25fdffb640 1 -- 172.21.15.5:0/1824448588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f2610010a40 con 0x558e1c0895d0 2024-09-19T06:04:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.625+0000 7f261e7f7740 1 -- 172.21.15.5:0/1824448588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2610010cf0 con 0x558e1c0895d0 2024-09-19T06:04:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.625+0000 7f25fdffb640 1 -- 172.21.15.5:0/1824448588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2610038070 con 0x558e1c0895d0 2024-09-19T06:04:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.625+0000 7f261e7f7740 10 client.15537 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.625+0000 7f261e7f7740 1 -- 172.21.15.5:0/1824448588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e1c116fb0 con 0x558e1c0895d0 2024-09-19T06:04:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.625+0000 7f25fdffb640 1 -- 172.21.15.5:0/1824448588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2610010cf0 con 0x558e1c0895d0 2024-09-19T06:04:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.625+0000 7f25fdffb640 1 -- 172.21.15.5:0/1824448588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f261003d950 con 0x558e1c0895d0 2024-09-19T06:04:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.626+0000 7f261e7f7740 10 client.15537 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.626+0000 7f261e7f7740 10 client.15537 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:04:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.626+0000 7f261e7f7740 20 client.15537 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.626+0000 7f261e7f7740 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x558e1c117600 0x558e1c1199f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.626+0000 7f261e7f7740 4 client.15537 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.626+0000 7f261e7f7740 1 -- 172.21.15.5:0/1824448588 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e1bff6d70 con 0x558e1c117600 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.626+0000 7f26199b9640 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x558e1c117600 0x558e1c1199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.626+0000 7f25fdffb640 10 client.15537 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.626+0000 7f26199b9640 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x558e1c117600 0x558e1c1199f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b64a22aea133f2e server_cookie=676b75b4624cd45b in_seq=0 out_seq=0 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f25fdffb640 1 -- 172.21.15.5:0/1824448588 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e1bff6d70 con 0x558e1c117600 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f25fdffb640 10 client.15537 handle_command_reply: tid=0 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f261e7f7740 1 client.15537 shutdown 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f261e7f7740 1 -- 172.21.15.5:0/1824448588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 msgr2=0x558e1c097470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f261e7f7740 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c097470 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f261000fc30 tx=0x7f2610010750 comp rx=0 tx=0).stop 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f261e7f7740 1 -- 172.21.15.5:0/1824448588 shutdown_connections 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f261e7f7740 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x558e1c117600 0x558e1c1199f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f25fdffb640 0 client.15537 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:04:08.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f261e7f7740 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e1c094370 0x558e1c10f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f261e7f7740 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e1c08ab40 0x558e1c0979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f261e7f7740 1 --2- 172.21.15.5:0/1824448588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e1c0895d0 0x558e1c097470 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f261e7f7740 1 -- 172.21.15.5:0/1824448588 >> 172.21.15.5:0/1824448588 conn(0x558e1c052310 msgr2=0x558e1c08b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f261e7f7740 1 -- 172.21.15.5:0/1824448588 shutdown_connections 2024-09-19T06:04:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f261e7f7740 1 -- 172.21.15.5:0/1824448588 wait complete. 2024-09-19T06:04:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.627+0000 7f261e7f7740 20 client.15537 trim_cache size 0 max 16384 2024-09-19T06:04:08.635 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:04:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.628+0000 7f261c41e640 1 -- 172.21.15.5:0/1579542811 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26000a35d0 msgr2=0x7f26000a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.628+0000 7f261c41e640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26000a35d0 0x7f26000a5a90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2610005d00 tx=0x7f2610005c70 comp rx=0 tx=0).stop 2024-09-19T06:04:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.628+0000 7f261c41e640 1 -- 172.21.15.5:0/1579542811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2614102570 msgr2=0x7f26141975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.628+0000 7f261c41e640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2614102570 0x7f26141975a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2604006010 tx=0x7f26040042f0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.628+0000 7f261c41e640 1 -- 172.21.15.5:0/1579542811 shutdown_connections 2024-09-19T06:04:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.628+0000 7f261c41e640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26000a35d0 0x7f26000a5a90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.628+0000 7f261c41e640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2614102ef0 0x7f2614197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.628+0000 7f261c41e640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2614102570 0x7f26141975a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.628+0000 7f261c41e640 1 --2- 172.21.15.5:0/1579542811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2614100e20 0x7f2614197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.628+0000 7f261c41e640 1 -- 172.21.15.5:0/1579542811 >> 172.21.15.5:0/1579542811 conn(0x7f26140fc7d0 msgr2=0x7f26140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:08.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.628+0000 7f261c41e640 1 -- 172.21.15.5:0/1579542811 shutdown_connections 2024-09-19T06:04:08.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:08.628+0000 7f261c41e640 1 -- 172.21.15.5:0/1579542811 wait complete. 2024-09-19T06:04:08.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:08 smithi049 ceph-mon[34640]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T06:04:09.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:04:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:10.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.208+0000 7ffa57cd7640 1 -- 172.21.15.5:0/3353921292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa50102d30 msgr2=0x7ffa50103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:10.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.208+0000 7ffa57cd7640 1 --2- 172.21.15.5:0/3353921292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa50102d30 0x7ffa50103110 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009930 tx=0x7ffa4c02a420 comp rx=0 tx=0).stop 2024-09-19T06:04:10.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.208+0000 7ffa57cd7640 1 -- 172.21.15.5:0/3353921292 shutdown_connections 2024-09-19T06:04:10.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.208+0000 7ffa57cd7640 1 --2- 172.21.15.5:0/3353921292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa501082a0 0x7ffa5010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.208+0000 7ffa57cd7640 1 --2- 172.21.15.5:0/3353921292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa501036e0 0x7ffa50107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.208+0000 7ffa57cd7640 1 --2- 172.21.15.5:0/3353921292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa50102d30 0x7ffa50103110 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.208+0000 7ffa57cd7640 1 -- 172.21.15.5:0/3353921292 >> 172.21.15.5:0/3353921292 conn(0x7ffa500fc7d0 msgr2=0x7ffa500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:10.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.209+0000 7ffa57cd7640 1 -- 172.21.15.5:0/3353921292 shutdown_connections 2024-09-19T06:04:10.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.209+0000 7ffa57cd7640 1 -- 172.21.15.5:0/3353921292 wait complete. 2024-09-19T06:04:10.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.209+0000 7ffa57cd7640 1 Processor -- start 2024-09-19T06:04:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.209+0000 7ffa57cd7640 1 -- start start 2024-09-19T06:04:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.210+0000 7ffa57cd7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa50102d30 0x7ffa50198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.210+0000 7ffa57cd7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa501036e0 0x7ffa50198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.210+0000 7ffa57cd7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa501082a0 0x7ffa5019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.210+0000 7ffa57cd7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50111f50 con 0x7ffa501036e0 2024-09-19T06:04:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.210+0000 7ffa55a73640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa50102d30 0x7ffa50198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.210+0000 7ffa55a73640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa50102d30 0x7ffa50198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52360/0 (socket says 172.21.15.5:52360) 2024-09-19T06:04:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.210+0000 7ffa55a73640 1 -- 172.21.15.5:0/1811373428 learned_addr learned my addr 172.21.15.5:0/1811373428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.210+0000 7ffa55272640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa501036e0 0x7ffa50198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.210+0000 7ffa57cd7640 1 -- 172.21.15.5:0/1811373428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50111dd0 con 0x7ffa501082a0 2024-09-19T06:04:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.210+0000 7ffa57cd7640 1 -- 172.21.15.5:0/1811373428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501120d0 con 0x7ffa50102d30 2024-09-19T06:04:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.210+0000 7ffa56274640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa501082a0 0x7ffa5019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.211+0000 7ffa55a73640 1 -- 172.21.15.5:0/1811373428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa501082a0 msgr2=0x7ffa5019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.211+0000 7ffa55a73640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa501082a0 0x7ffa5019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.211+0000 7ffa55a73640 1 -- 172.21.15.5:0/1811373428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa501036e0 msgr2=0x7ffa50198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.211+0000 7ffa55a73640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa501036e0 0x7ffa50198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.211+0000 7ffa55a73640 1 -- 172.21.15.5:0/1811373428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5019d4c0 con 0x7ffa50102d30 2024-09-19T06:04:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.211+0000 7ffa55272640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa501036e0 0x7ffa50198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.211+0000 7ffa55a73640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa50102d30 0x7ffa50198510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c005940 tx=0x7ffa4c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.211+0000 7ffa46ffd640 1 -- 172.21.15.5:0/1811373428 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c03f030 con 0x7ffa50102d30 2024-09-19T06:04:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.211+0000 7ffa46ffd640 1 -- 172.21.15.5:0/1811373428 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c0318a0 con 0x7ffa50102d30 2024-09-19T06:04:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.211+0000 7ffa57cd7640 1 -- 172.21.15.5:0/1811373428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5019d750 con 0x7ffa50102d30 2024-09-19T06:04:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.211+0000 7ffa46ffd640 1 -- 172.21.15.5:0/1811373428 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c035070 con 0x7ffa50102d30 2024-09-19T06:04:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.212+0000 7ffa57cd7640 1 -- 172.21.15.5:0/1811373428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5019dbe0 con 0x7ffa50102d30 2024-09-19T06:04:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.214+0000 7ffa46ffd640 1 -- 172.21.15.5:0/1811373428 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7ffa4c042030 con 0x7ffa50102d30 2024-09-19T06:04:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.215+0000 7ffa46ffd640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ffa3c0a34d0 0x7ffa3c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.215+0000 7ffa46ffd640 1 -- 172.21.15.5:0/1811373428 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7ffa4c0f01b0 con 0x7ffa50102d30 2024-09-19T06:04:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.215+0000 7ffa55272640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ffa3c0a34d0 0x7ffa3c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.216+0000 7ffa5a0b0740 1 Processor -- start 2024-09-19T06:04:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.216+0000 7ffa55272640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ffa3c0a34d0 0x7ffa3c0a5990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa500fef20 tx=0x7ffa40005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.216+0000 7ffa5a0b0740 1 -- start start 2024-09-19T06:04:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.216+0000 7ffa5a0b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 0x55d0f0ffd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.216+0000 7ffa5a0b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffd5d0 0x55d0f0ffda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.216+0000 7ffa5a0b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f1008370 0x55d0f100a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.216+0000 7ffa5a0b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0f632f0 con 0x55d0f0ffd5d0 2024-09-19T06:04:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.216+0000 7ffa5a0b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0f0acf0 con 0x55d0f0ffeb40 2024-09-19T06:04:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.216+0000 7ffa5a0b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0f5f2b0 con 0x55d0f1008370 2024-09-19T06:04:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.217+0000 7ffa55a73640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 0x55d0f0ffd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.217+0000 7ffa55a73640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 0x55d0f0ffd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56062/0 (socket says 172.21.15.5:56062) 2024-09-19T06:04:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.217+0000 7ffa55a73640 1 -- 172.21.15.5:0/1528971227 learned_addr learned my addr 172.21.15.5:0/1528971227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.217+0000 7ffa56274640 1 --2- 172.21.15.5:0/1528971227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f1008370 0x55d0f100a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.217+0000 7ffa55a73640 1 -- 172.21.15.5:0/1528971227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f1008370 msgr2=0x55d0f100a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.217+0000 7ffa55a73640 1 --2- 172.21.15.5:0/1528971227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f1008370 0x55d0f100a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.217+0000 7ffa55a73640 1 -- 172.21.15.5:0/1528971227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffd5d0 msgr2=0x55d0f0ffda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.217+0000 7ffa55272640 1 --2- 172.21.15.5:0/1528971227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffd5d0 0x55d0f0ffda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.217+0000 7ffa55a73640 1 --2- 172.21.15.5:0/1528971227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffd5d0 0x55d0f0ffda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.217+0000 7ffa55a73640 1 -- 172.21.15.5:0/1528971227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f0eef3a0 con 0x55d0f0ffeb40 2024-09-19T06:04:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.217+0000 7ffa55272640 1 --2- 172.21.15.5:0/1528971227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffd5d0 0x55d0f0ffda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:04:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.218+0000 7ffa55a73640 1 --2- 172.21.15.5:0/1528971227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 0x55d0f0ffd090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0bc300 tx=0x7ffa4c0bbd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5c247c2f50fe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.218+0000 7ffa3bfff640 1 -- 172.21.15.5:0/1528971227 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c049030 con 0x55d0f0ffeb40 2024-09-19T06:04:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.218+0000 7ffa3bfff640 1 -- 172.21.15.5:0/1528971227 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c0c0670 con 0x55d0f0ffeb40 2024-09-19T06:04:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.218+0000 7ffa3bfff640 1 -- 172.21.15.5:0/1528971227 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0eda10 con 0x55d0f0ffeb40 2024-09-19T06:04:10.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.219+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1528971227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 msgr2=0x55d0f0ffd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:10.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.219+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/1528971227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 0x55d0f0ffd090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0bc300 tx=0x7ffa4c0bbd90 comp rx=0 tx=0).stop 2024-09-19T06:04:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.219+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1528971227 shutdown_connections 2024-09-19T06:04:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.219+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/1528971227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f1008370 0x55d0f100a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.219+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/1528971227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffd5d0 0x55d0f0ffda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.219+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/1528971227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 0x55d0f0ffd090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.219+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1528971227 >> 172.21.15.5:0/1528971227 conn(0x55d0f0fc6310 msgr2=0x55d0f0ff9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.220+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1528971227 shutdown_connections 2024-09-19T06:04:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.220+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1528971227 wait complete. 2024-09-19T06:04:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.220+0000 7ffa5a0b0740 1 -- start start 2024-09-19T06:04:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.220+0000 7ffa5a0b0740 1 Processor -- start 2024-09-19T06:04:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa5a0b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffd5d0 0x55d0f1007d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa5a0b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffeb40 0x55d0f1005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa5a0b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f1008370 0x55d0f10056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa5a0b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0f0acf0 con 0x55d0f0ffeb40 2024-09-19T06:04:10.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa5a0b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0f1e060 con 0x55d0f1008370 2024-09-19T06:04:10.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa5a0b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d0f100ae30 con 0x55d0f0ffd5d0 2024-09-19T06:04:10.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa55272640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffeb40 0x55d0f1005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa55272640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffeb40 0x55d0f1005040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41630/0 (socket says 172.21.15.5:41630) 2024-09-19T06:04:10.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa55272640 1 -- 172.21.15.5:0/2872161288 learned_addr learned my addr 172.21.15.5:0/2872161288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:10.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa56274640 1 --2- 172.21.15.5:0/2872161288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f1008370 0x55d0f10056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa5a0b0740 10 client.0 _ll_register_callbacks cb 0x55d0f0c718c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:10.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa55272640 1 -- 172.21.15.5:0/2872161288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffd5d0 msgr2=0x55d0f1007d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:04:10.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa55272640 1 --2- 172.21.15.5:0/2872161288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffd5d0 0x55d0f1007d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa55272640 1 -- 172.21.15.5:0/2872161288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f1008370 msgr2=0x55d0f10056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:10.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa55272640 1 --2- 172.21.15.5:0/2872161288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f1008370 0x55d0f10056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.221+0000 7ffa55272640 1 -- 172.21.15.5:0/2872161288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f0cd43c0 con 0x55d0f0ffeb40 2024-09-19T06:04:10.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.222+0000 7ffa55272640 1 --2- 172.21.15.5:0/2872161288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffeb40 0x55d0f1005040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffa40003870 tx=0x7ffa40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0c8ecd5cb532d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:10.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.222+0000 7ffa397fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:10.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.222+0000 7ffa397fa640 1 -- 172.21.15.5:0/2872161288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4000fe10 con 0x55d0f0ffeb40 2024-09-19T06:04:10.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.222+0000 7ffa397fa640 1 -- 172.21.15.5:0/2872161288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7ffa40027470 con 0x55d0f0ffeb40 2024-09-19T06:04:10.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.222+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/2872161288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4c0c0670 con 0x55d0f0ffeb40 2024-09-19T06:04:10.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.222+0000 7ffa397fa640 1 -- 172.21.15.5:0/2872161288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa40027670 con 0x55d0f0ffeb40 2024-09-19T06:04:10.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.222+0000 7ffa5a0b0740 10 client.15546 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:10.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.222+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/2872161288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0f1016710 con 0x55d0f0ffeb40 2024-09-19T06:04:10.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.223+0000 7ffa397fa640 1 -- 172.21.15.5:0/2872161288 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c0c0670 con 0x55d0f0ffeb40 2024-09-19T06:04:10.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.223+0000 7ffa397fa640 1 -- 172.21.15.5:0/2872161288 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffa4002e050 con 0x55d0f0ffeb40 2024-09-19T06:04:10.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.223+0000 7ffa5a0b0740 10 client.15546 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:10.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.223+0000 7ffa5a0b0740 10 client.15546 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:04:10.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.223+0000 7ffa5a0b0740 20 client.15546 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:10.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.223+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/2872161288 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d0f1017b70 0x55d0f101a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.223+0000 7ffa5a0b0740 4 client.15546 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:10.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.223+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/2872161288 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0f0f70860 con 0x55d0f1017b70 2024-09-19T06:04:10.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.223+0000 7ffa55a73640 1 --2- 172.21.15.5:0/2872161288 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d0f1017b70 0x55d0f101a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.224+0000 7ffa397fa640 10 client.15546 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:04:10.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.224+0000 7ffa55a73640 1 --2- 172.21.15.5:0/2872161288 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d0f1017b70 0x55d0f101a010 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c32f7c70b86ae2b8 server_cookie=10cf67ac7e019042 in_seq=0 out_seq=0 2024-09-19T06:04:10.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.225+0000 7ffa397fa640 1 -- 172.21.15.5:0/2872161288 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0f0f70860 con 0x55d0f1017b70 2024-09-19T06:04:10.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.225+0000 7ffa397fa640 10 client.15546 handle_command_reply: tid=0 2024-09-19T06:04:10.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.225+0000 7ffa5a0b0740 1 client.15546 shutdown 2024-09-19T06:04:10.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.225+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/2872161288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffeb40 msgr2=0x55d0f1005040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:10.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.225+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/2872161288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffeb40 0x55d0f1005040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffa40003870 tx=0x7ffa40004730 comp rx=0 tx=0).stop 2024-09-19T06:04:10.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.226+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/2872161288 shutdown_connections 2024-09-19T06:04:10.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.226+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/2872161288 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d0f1017b70 0x55d0f101a010 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.226+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/2872161288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f1008370 0x55d0f10056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.226+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/2872161288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffeb40 0x55d0f1005040 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.226+0000 7ffa397fa640 0 client.15546 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:04:10.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.226+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/2872161288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffd5d0 0x55d0f1007d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.226+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/2872161288 >> 172.21.15.5:0/2872161288 conn(0x55d0f0fc6310 msgr2=0x55d0f0fff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:10.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.226+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/2872161288 shutdown_connections 2024-09-19T06:04:10.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.226+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/2872161288 wait complete. 2024-09-19T06:04:10.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.226+0000 7ffa5a0b0740 20 client.15546 trim_cache size 0 max 16384 2024-09-19T06:04:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.252+0000 7ffa5a0b0740 1 Processor -- start 2024-09-19T06:04:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.252+0000 7ffa5a0b0740 1 -- start start 2024-09-19T06:04:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.252+0000 7ffa5a0b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffd5d0 0x55d0f10001b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.252+0000 7ffa5a0b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 0x55d0f10006f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.252+0000 7ffa5a0b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f1008370 0x55d0f101f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.252+0000 7ffa5a0b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501ad920 con 0x55d0f1008370 2024-09-19T06:04:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.252+0000 7ffa5a0b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501af460 con 0x55d0f0ffeb40 2024-09-19T06:04:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.252+0000 7ffa5a0b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501adff0 con 0x55d0f0ffd5d0 2024-09-19T06:04:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.252+0000 7ffa55272640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 0x55d0f10006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.252+0000 7ffa55272640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 0x55d0f10006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56080/0 (socket says 172.21.15.5:56080) 2024-09-19T06:04:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.252+0000 7ffa55272640 1 -- 172.21.15.5:0/173016475 learned_addr learned my addr 172.21.15.5:0/173016475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.253+0000 7ffa55a73640 1 --2- 172.21.15.5:0/173016475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffd5d0 0x55d0f10001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.253+0000 7ffa56274640 1 --2- 172.21.15.5:0/173016475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f1008370 0x55d0f101f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.253+0000 7ffa55272640 1 -- 172.21.15.5:0/173016475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffd5d0 msgr2=0x55d0f10001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.253+0000 7ffa55272640 1 --2- 172.21.15.5:0/173016475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffd5d0 0x55d0f10001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.253+0000 7ffa55272640 1 -- 172.21.15.5:0/173016475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f1008370 msgr2=0x55d0f101f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.253+0000 7ffa55272640 1 --2- 172.21.15.5:0/173016475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f1008370 0x55d0f101f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.253+0000 7ffa55272640 1 -- 172.21.15.5:0/173016475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f101a690 con 0x55d0f0ffeb40 2024-09-19T06:04:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.253+0000 7ffa55272640 1 --2- 172.21.15.5:0/173016475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 0x55d0f10006f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa4002f960 tx=0x7ffa400096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d87c334c23b281 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.256+0000 7ffa3b7fe640 1 -- 172.21.15.5:0/173016475 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40004480 con 0x55d0f0ffeb40 2024-09-19T06:04:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.256+0000 7ffa3b7fe640 1 -- 172.21.15.5:0/173016475 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa40022070 con 0x55d0f0ffeb40 2024-09-19T06:04:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.256+0000 7ffa3b7fe640 1 -- 172.21.15.5:0/173016475 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40027810 con 0x55d0f0ffeb40 2024-09-19T06:04:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.257+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/173016475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 msgr2=0x55d0f10006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.257+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/173016475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 0x55d0f10006f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa4002f960 tx=0x7ffa400096c0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.257+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/173016475 shutdown_connections 2024-09-19T06:04:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.257+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/173016475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f1008370 0x55d0f101f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.257+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/173016475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f0ffeb40 0x55d0f10006f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.257+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/173016475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffd5d0 0x55d0f10001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.257+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/173016475 >> 172.21.15.5:0/173016475 conn(0x55d0f0fc6310 msgr2=0x55d0f10070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.257+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/173016475 shutdown_connections 2024-09-19T06:04:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.258+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/173016475 wait complete. 2024-09-19T06:04:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.258+0000 7ffa5a0b0740 1 -- start start 2024-09-19T06:04:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.258+0000 7ffa5a0b0740 1 Processor -- start 2024-09-19T06:04:10.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa5a0b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffd5d0 0x55d0f10100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa5a0b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffeb40 0x55d0f1010600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa5a0b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f1008370 0x55d0f107c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa5a0b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501aed90 con 0x55d0f0ffd5d0 2024-09-19T06:04:10.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa5a0b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501ae6c0 con 0x55d0f1008370 2024-09-19T06:04:10.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa5a0b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501ab900 con 0x55d0f0ffeb40 2024-09-19T06:04:10.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa5a0b0740 10 client.0 _ll_register_callbacks cb 0x7ffa40009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:10.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa55a73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffd5d0 0x55d0f10100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa55a73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffd5d0 0x55d0f10100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41660/0 (socket says 172.21.15.5:41660) 2024-09-19T06:04:10.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa55a73640 1 -- 172.21.15.5:0/1774279490 learned_addr learned my addr 172.21.15.5:0/1774279490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:10.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa56274640 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f1008370 0x55d0f107c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa55272640 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffeb40 0x55d0f1010600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa56274640 1 -- 172.21.15.5:0/1774279490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffeb40 msgr2=0x55d0f1010600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:10.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa56274640 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffeb40 0x55d0f1010600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.259+0000 7ffa56274640 1 -- 172.21.15.5:0/1774279490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffd5d0 msgr2=0x55d0f10100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:10.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.260+0000 7ffa56274640 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffd5d0 0x55d0f10100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.260+0000 7ffa56274640 1 -- 172.21.15.5:0/1774279490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f1010d40 con 0x55d0f1008370 2024-09-19T06:04:10.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.260+0000 7ffa56274640 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f1008370 0x55d0f107c880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55d0f1003820 tx=0x7ffa48012e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=726e74879990c34a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:10.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.260+0000 7ffa397fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:04:10.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.260+0000 7ffa397fa640 1 -- 172.21.15.5:0/1774279490 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48022060 con 0x55d0f1008370 2024-09-19T06:04:10.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.260+0000 7ffa397fa640 1 -- 172.21.15.5:0/1774279490 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7ffa48011950 con 0x55d0f1008370 2024-09-19T06:04:10.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.260+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1774279490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa40022070 con 0x55d0f1008370 2024-09-19T06:04:10.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.260+0000 7ffa397fa640 1 -- 172.21.15.5:0/1774279490 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa48003e40 con 0x55d0f1008370 2024-09-19T06:04:10.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.260+0000 7ffa5a0b0740 10 client.25172 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:10.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.260+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1774279490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0f108bb80 con 0x55d0f1008370 2024-09-19T06:04:10.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.261+0000 7ffa397fa640 1 -- 172.21.15.5:0/1774279490 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa48004050 con 0x55d0f1008370 2024-09-19T06:04:10.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.261+0000 7ffa397fa640 1 -- 172.21.15.5:0/1774279490 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffa4801c030 con 0x55d0f1008370 2024-09-19T06:04:10.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.261+0000 7ffa5a0b0740 10 client.25172 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:10.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.261+0000 7ffa5a0b0740 10 client.25172 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:04:10.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.261+0000 7ffa5a0b0740 20 client.25172 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:10.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.261+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d0f108c330 0x55d0f108e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:10.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.261+0000 7ffa5a0b0740 4 client.25172 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:04:10.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.261+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1774279490 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ffa40022070 con 0x55d0f108c330 2024-09-19T06:04:10.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.262+0000 7ffa55a73640 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d0f108c330 0x55d0f108e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:10.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.262+0000 7ffa55a73640 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d0f108c330 0x55d0f108e750 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d499131649077291 server_cookie=62d5ca9cf87c9602 in_seq=0 out_seq=0 2024-09-19T06:04:10.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.262+0000 7ffa397fa640 10 client.25172 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:04:10.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa397fa640 1 -- 172.21.15.5:0/1774279490 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ffa40022070 con 0x55d0f108c330 2024-09-19T06:04:10.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa397fa640 10 client.25172 handle_command_reply: tid=0 2024-09-19T06:04:10.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa5a0b0740 1 client.25172 shutdown 2024-09-19T06:04:10.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1774279490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f1008370 msgr2=0x55d0f107c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:10.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f1008370 0x55d0f107c880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55d0f1003820 tx=0x7ffa48012e70 comp rx=0 tx=0).stop 2024-09-19T06:04:10.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1774279490 shutdown_connections 2024-09-19T06:04:10.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d0f108c330 0x55d0f108e750 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0f1008370 0x55d0f107c880 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa397fa640 0 client.25172 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:04:10.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f0ffeb40 0x55d0f1010600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.269 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:04:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa5a0b0740 1 --2- 172.21.15.5:0/1774279490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0ffd5d0 0x55d0f10100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1774279490 >> 172.21.15.5:0/1774279490 conn(0x55d0f0fc6310 msgr2=0x55d0f0ff84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1774279490 shutdown_connections 2024-09-19T06:04:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa5a0b0740 1 -- 172.21.15.5:0/1774279490 wait complete. 2024-09-19T06:04:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.263+0000 7ffa5a0b0740 20 client.25172 trim_cache size 0 max 16384 2024-09-19T06:04:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.264+0000 7ffa57cd7640 1 -- 172.21.15.5:0/1811373428 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ffa3c0a34d0 msgr2=0x7ffa3c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.264+0000 7ffa57cd7640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ffa3c0a34d0 0x7ffa3c0a5990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa500fef20 tx=0x7ffa40005f90 comp rx=0 tx=0).stop 2024-09-19T06:04:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.264+0000 7ffa57cd7640 1 -- 172.21.15.5:0/1811373428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa50102d30 msgr2=0x7ffa50198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.264+0000 7ffa57cd7640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa50102d30 0x7ffa50198510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c005940 tx=0x7ffa4c0042f0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.264+0000 7ffa57cd7640 1 -- 172.21.15.5:0/1811373428 shutdown_connections 2024-09-19T06:04:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.264+0000 7ffa57cd7640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa501082a0 0x7ffa5019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.264+0000 7ffa57cd7640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa501036e0 0x7ffa50198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.264+0000 7ffa57cd7640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ffa3c0a34d0 0x7ffa3c0a5990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.264+0000 7ffa57cd7640 1 --2- 172.21.15.5:0/1811373428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa50102d30 0x7ffa50198510 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.264+0000 7ffa57cd7640 1 -- 172.21.15.5:0/1811373428 >> 172.21.15.5:0/1811373428 conn(0x7ffa500fc7d0 msgr2=0x7ffa50105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.264+0000 7ffa57cd7640 1 -- 172.21.15.5:0/1811373428 shutdown_connections 2024-09-19T06:04:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:10.264+0000 7ffa57cd7640 1 -- 172.21.15.5:0/1811373428 wait complete. 2024-09-19T06:04:10.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:10 smithi005 ceph-mon[30955]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T06:04:10.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:10 smithi157 ceph-mon[35346]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T06:04:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:10 smithi049 ceph-mon[34640]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T06:04:10.982 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:04:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.396+0000 7f91ebfff640 1 -- 172.21.15.5:0/2288104657 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0036020 con 0x7f91f4100810 2024-09-19T06:04:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.397+0000 7f91fb4f1640 1 -- 172.21.15.5:0/2288104657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91f4100810 msgr2=0x7f91f4100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.397+0000 7f91fb4f1640 1 --2- 172.21.15.5:0/2288104657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91f4100810 0x7f91f4100bf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f91f000aa10 tx=0x7f91f002b720 comp rx=0 tx=0).stop 2024-09-19T06:04:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.397+0000 7f91fb4f1640 1 -- 172.21.15.5:0/2288104657 shutdown_connections 2024-09-19T06:04:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.397+0000 7f91fb4f1640 1 --2- 172.21.15.5:0/2288104657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91f4105500 0x7f91f410d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.397+0000 7f91fb4f1640 1 --2- 172.21.15.5:0/2288104657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f41011c0 0x7f91f4104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.397+0000 7f91fb4f1640 1 --2- 172.21.15.5:0/2288104657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91f4100810 0x7f91f4100bf0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.397+0000 7f91fb4f1640 1 -- 172.21.15.5:0/2288104657 >> 172.21.15.5:0/2288104657 conn(0x7f91f4075eb0 msgr2=0x7f91f40762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.397+0000 7f91fb4f1640 1 -- 172.21.15.5:0/2288104657 shutdown_connections 2024-09-19T06:04:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.397+0000 7f91fb4f1640 1 -- 172.21.15.5:0/2288104657 wait complete. 2024-09-19T06:04:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91fb4f1640 1 Processor -- start 2024-09-19T06:04:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91fb4f1640 1 -- start start 2024-09-19T06:04:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91fb4f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91f4100810 0x7f91f4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91fb4f1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91f4105500 0x7f91f41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91fb4f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f419cc30 0x7f91f419d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91fb4f1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91f410fbe0 con 0x7f91f4100810 2024-09-19T06:04:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91fb4f1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f91f410fa60 con 0x7f91f4105500 2024-09-19T06:04:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91fb4f1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91f410fd60 con 0x7f91f419cc30 2024-09-19T06:04:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91f928d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91f4100810 0x7f91f4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91f928d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91f4100810 0x7f91f4198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41666/0 (socket says 172.21.15.5:41666) 2024-09-19T06:04:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91f928d640 1 -- 172.21.15.5:0/551436108 learned_addr learned my addr 172.21.15.5:0/551436108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91f4105500 0x7f91f41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.398+0000 7f91f9a8e640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f419cc30 0x7f91f419d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.399+0000 7f91f928d640 1 -- 172.21.15.5:0/551436108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f419cc30 msgr2=0x7f91f419d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.399+0000 7f91f928d640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f419cc30 0x7f91f419d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.399+0000 7f91f928d640 1 -- 172.21.15.5:0/551436108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91f4105500 msgr2=0x7f91f41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.399+0000 7f91f928d640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91f4105500 0x7f91f41988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.399+0000 7f91f928d640 1 -- 172.21.15.5:0/551436108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f419d720 con 0x7f91f4100810 2024-09-19T06:04:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.399+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91f4105500 0x7f91f41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.399+0000 7f91f928d640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91f4100810 0x7f91f4198370 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f91f0000c00 tx=0x7f91f0030b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.399+0000 7f91ea7fc640 1 -- 172.21.15.5:0/551436108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f003f070 con 0x7f91f4100810 2024-09-19T06:04:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.399+0000 7f91fb4f1640 1 -- 172.21.15.5:0/551436108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f419d9b0 con 0x7f91f4100810 2024-09-19T06:04:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.399+0000 7f91ea7fc640 1 -- 172.21.15.5:0/551436108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91f0003ca0 con 0x7f91f4100810 2024-09-19T06:04:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.399+0000 7f91ea7fc640 1 -- 172.21.15.5:0/551436108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0036040 con 0x7f91f4100810 2024-09-19T06:04:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.399+0000 7f91fb4f1640 1 -- 172.21.15.5:0/551436108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f41a51d0 con 0x7f91f4100810 2024-09-19T06:04:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.402+0000 7f91ea7fc640 1 -- 172.21.15.5:0/551436108 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7f91f0042070 con 0x7f91f4100810 2024-09-19T06:04:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91fe28b740 1 Processor -- start 2024-09-19T06:04:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91fe28b740 1 -- start start 2024-09-19T06:04:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91ea7fc640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91e00a3550 0x7f91e00a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91fe28b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9749b0 0x555bae974d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91fe28b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae983b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91fe28b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9840c0 0x555bae9864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91fe28b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555bae8d92f0 con 0x555bae9752d0 2024-09-19T06:04:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91fe28b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555bae881270 con 0x555bae9840c0 2024-09-19T06:04:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91fe28b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555bae8d52b0 con 0x555bae9749b0 2024-09-19T06:04:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91ea7fc640 1 -- 172.21.15.5:0/551436108 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f91f00f0510 con 0x7f91f4100810 2024-09-19T06:04:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91f9a8e640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91e00a3550 0x7f91e00a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91f8a8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae983b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91f8a8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae983b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41678/0 (socket says 172.21.15.5:41678) 2024-09-19T06:04:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91f8a8c640 1 -- 172.21.15.5:0/1828180365 learned_addr learned my addr 172.21.15.5:0/1828180365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91f9a8e640 1 --2- 172.21.15.5:0/1828180365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9840c0 0x555bae9864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.403+0000 7f91f928d640 1 --2- 172.21.15.5:0/1828180365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9749b0 0x555bae974d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.404+0000 7f91f8a8c640 1 -- 172.21.15.5:0/1828180365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9749b0 msgr2=0x555bae974d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.404+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/1828180365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9749b0 0x555bae974d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.404+0000 7f91f8a8c640 1 -- 172.21.15.5:0/1828180365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9840c0 msgr2=0x555bae9864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.404+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/1828180365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9840c0 0x555bae9864b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.404+0000 7f91f8a8c640 1 -- 172.21.15.5:0/1828180365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bae880340 con 0x555bae9752d0 2024-09-19T06:04:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.404+0000 7f91f9a8e640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91e00a3550 0x7f91e00a5a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f91ec005c40 tx=0x7f91ec009af0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.404+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/1828180365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae983b80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009c70 tx=0x7f91e40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e78dba8a76b19c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.404+0000 7f91df7fe640 1 -- 172.21.15.5:0/1828180365 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4012450 con 0x555bae9752d0 2024-09-19T06:04:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.404+0000 7f91df7fe640 1 -- 172.21.15.5:0/1828180365 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91e4012aa0 con 0x555bae9752d0 2024-09-19T06:04:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.404+0000 7f91df7fe640 1 -- 172.21.15.5:0/1828180365 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e400fb10 con 0x555bae9752d0 2024-09-19T06:04:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.405+0000 7f91fe28b740 1 -- 172.21.15.5:0/1828180365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 msgr2=0x555bae983b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.405+0000 7f91fe28b740 1 --2- 172.21.15.5:0/1828180365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae983b80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009c70 tx=0x7f91e40117b0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.405+0000 7f91fe28b740 1 -- 172.21.15.5:0/1828180365 shutdown_connections 2024-09-19T06:04:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.405+0000 7f91fe28b740 1 --2- 172.21.15.5:0/1828180365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9840c0 0x555bae9864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.405+0000 7f91fe28b740 1 --2- 172.21.15.5:0/1828180365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae983b80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.405+0000 7f91fe28b740 1 --2- 172.21.15.5:0/1828180365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9749b0 0x555bae974d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.405+0000 7f91fe28b740 1 -- 172.21.15.5:0/1828180365 >> 172.21.15.5:0/1828180365 conn(0x555bae971060 msgr2=0x555bae971440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.405+0000 7f91fe28b740 1 -- 172.21.15.5:0/1828180365 shutdown_connections 2024-09-19T06:04:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.405+0000 7f91fe28b740 1 -- 172.21.15.5:0/1828180365 wait complete. 2024-09-19T06:04:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.406+0000 7f91fe28b740 1 -- start start 2024-09-19T06:04:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.406+0000 7f91fe28b740 1 Processor -- start 2024-09-19T06:04:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.406+0000 7f91fe28b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae981060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.406+0000 7f91fe28b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9752d0 0x555bae9815a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.406+0000 7f91fe28b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9840c0 0x555bae981ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.406+0000 7f91fe28b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555bae986c90 con 0x555bae9840c0 2024-09-19T06:04:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.406+0000 7f91fe28b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555bae862290 con 0x555bae9749b0 2024-09-19T06:04:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.406+0000 7f91fe28b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555bae986e10 con 0x555bae9752d0 2024-09-19T06:04:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f928d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae981060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91fe28b740 10 client.0 _ll_register_callbacks cb 0x555bae893140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f928d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae981060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56112/0 (socket says 172.21.15.5:56112) 2024-09-19T06:04:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f928d640 1 -- 172.21.15.5:0/842626666 learned_addr learned my addr 172.21.15.5:0/842626666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f9a8e640 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9840c0 0x555bae981ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9752d0 0x555bae9815a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f928d640 1 -- 172.21.15.5:0/842626666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9752d0 msgr2=0x555bae9815a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f928d640 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9752d0 0x555bae9815a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f928d640 1 -- 172.21.15.5:0/842626666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9840c0 msgr2=0x555bae981ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f928d640 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9840c0 0x555bae981ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f928d640 1 -- 172.21.15.5:0/842626666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bae5ec210 con 0x555bae9749b0 2024-09-19T06:04:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9752d0 0x555bae9815a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:04:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f9a8e640 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9840c0 0x555bae981ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91f928d640 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae981060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91f0005ec0 tx=0x7f91f00b8a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e461f7702605449 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.407+0000 7f91dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:04:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91dcff9640 1 -- 172.21.15.5:0/842626666 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0049040 con 0x555bae9749b0 2024-09-19T06:04:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91fe28b740 1 -- 172.21.15.5:0/842626666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91e4012aa0 con 0x555bae9749b0 2024-09-19T06:04:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91dcff9640 1 -- 172.21.15.5:0/842626666 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f91f00766d0 con 0x555bae9749b0 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91dcff9640 1 -- 172.21.15.5:0/842626666 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91f00bcd80 con 0x555bae9749b0 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91fe28b740 10 client.25184 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91fe28b740 1 -- 172.21.15.5:0/842626666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555bae6b8620 con 0x555bae9749b0 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91dcff9640 1 -- 172.21.15.5:0/842626666 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91e4012aa0 con 0x555bae9749b0 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91dcff9640 1 -- 172.21.15.5:0/842626666 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f91f000b040 con 0x555bae9749b0 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91fe28b740 10 client.25184 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91fe28b740 10 client.25184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91fe28b740 20 client.25184 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91fe28b740 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555bae994150 0x555bae996560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91fe28b740 4 client.25184 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.408+0000 7f91fe28b740 1 -- 172.21.15.5:0/842626666 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bae8e6a20 con 0x555bae994150 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.409+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555bae994150 0x555bae996560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.409+0000 7f91dcff9640 10 client.25184 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.409+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555bae994150 0x555bae996560 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=640d871ed80f5930 server_cookie=6ebb1dddc260b45a in_seq=0 out_seq=0 2024-09-19T06:04:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.410+0000 7f91dcff9640 1 -- 172.21.15.5:0/842626666 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bae8e6a20 con 0x555bae994150 2024-09-19T06:04:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.410+0000 7f91dcff9640 10 client.25184 handle_command_reply: tid=0 2024-09-19T06:04:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.410+0000 7f91fe28b740 1 client.25184 shutdown 2024-09-19T06:04:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.410+0000 7f91fe28b740 1 -- 172.21.15.5:0/842626666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 msgr2=0x555bae981060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.410+0000 7f91fe28b740 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae981060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91f0005ec0 tx=0x7f91f00b8a40 comp rx=0 tx=0).stop 2024-09-19T06:04:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.410+0000 7f91fe28b740 1 -- 172.21.15.5:0/842626666 shutdown_connections 2024-09-19T06:04:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.410+0000 7f91fe28b740 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555bae994150 0x555bae996560 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.410+0000 7f91fe28b740 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9840c0 0x555bae981ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.410+0000 7f91fe28b740 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9752d0 0x555bae9815a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.410+0000 7f91dcff9640 0 client.25184 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.410+0000 7f91fe28b740 1 --2- 172.21.15.5:0/842626666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae981060 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.410+0000 7f91fe28b740 1 -- 172.21.15.5:0/842626666 >> 172.21.15.5:0/842626666 conn(0x555bae971060 msgr2=0x555bae976dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.411+0000 7f91fe28b740 1 -- 172.21.15.5:0/842626666 shutdown_connections 2024-09-19T06:04:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.411+0000 7f91fe28b740 1 -- 172.21.15.5:0/842626666 wait complete. 2024-09-19T06:04:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.411+0000 7f91fe28b740 20 client.25184 trim_cache size 0 max 16384 2024-09-19T06:04:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.438+0000 7f91fe28b740 1 Processor -- start 2024-09-19T06:04:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.438+0000 7f91fe28b740 1 -- start start 2024-09-19T06:04:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91fe28b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae982fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91fe28b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae99be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91fe28b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9840c0 0x555bae99c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91fe28b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41ab570 con 0x555bae9752d0 2024-09-19T06:04:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91fe28b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41ad0b0 con 0x555bae9749b0 2024-09-19T06:04:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91fe28b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41abc40 con 0x555bae9840c0 2024-09-19T06:04:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91f8a8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae99be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91f8a8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae99be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41700/0 (socket says 172.21.15.5:41700) 2024-09-19T06:04:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91f8a8c640 1 -- 172.21.15.5:0/4073783847 learned_addr learned my addr 172.21.15.5:0/4073783847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91f928d640 1 --2- 172.21.15.5:0/4073783847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae982fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91f8a8c640 1 -- 172.21.15.5:0/4073783847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9840c0 msgr2=0x555bae99c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/4073783847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9840c0 0x555bae99c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91f8a8c640 1 -- 172.21.15.5:0/4073783847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 msgr2=0x555bae982fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/4073783847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae982fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91f8a8c640 1 -- 172.21.15.5:0/4073783847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bae970e90 con 0x555bae9752d0 2024-09-19T06:04:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.439+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/4073783847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae99be70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f91e4011cb0 tx=0x7f91e4005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c51fcfa8428a4cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.440+0000 7f91deffd640 1 -- 172.21.15.5:0/4073783847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4038070 con 0x555bae9752d0 2024-09-19T06:04:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.440+0000 7f91deffd640 1 -- 172.21.15.5:0/4073783847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91e4012450 con 0x555bae9752d0 2024-09-19T06:04:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.440+0000 7f91deffd640 1 -- 172.21.15.5:0/4073783847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e400fa80 con 0x555bae9752d0 2024-09-19T06:04:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.441+0000 7f91fe28b740 1 -- 172.21.15.5:0/4073783847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 msgr2=0x555bae99be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.441+0000 7f91fe28b740 1 --2- 172.21.15.5:0/4073783847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae99be70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f91e4011cb0 tx=0x7f91e4005890 comp rx=0 tx=0).stop 2024-09-19T06:04:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.441+0000 7f91fe28b740 1 -- 172.21.15.5:0/4073783847 shutdown_connections 2024-09-19T06:04:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.441+0000 7f91fe28b740 1 --2- 172.21.15.5:0/4073783847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9840c0 0x555bae99c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.441+0000 7f91fe28b740 1 --2- 172.21.15.5:0/4073783847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae99be70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.441+0000 7f91fe28b740 1 --2- 172.21.15.5:0/4073783847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae982fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.441+0000 7f91fe28b740 1 -- 172.21.15.5:0/4073783847 >> 172.21.15.5:0/4073783847 conn(0x555bae971060 msgr2=0x555bae9828c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.441+0000 7f91fe28b740 1 -- 172.21.15.5:0/4073783847 shutdown_connections 2024-09-19T06:04:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.441+0000 7f91fe28b740 1 -- 172.21.15.5:0/4073783847 wait complete. 2024-09-19T06:04:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.441+0000 7f91fe28b740 1 -- start start 2024-09-19T06:04:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.442+0000 7f91fe28b740 1 Processor -- start 2024-09-19T06:04:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.442+0000 7f91fe28b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae97b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.442+0000 7f91fe28b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae98c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.442+0000 7f91fe28b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9840c0 0x555bae98c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.442+0000 7f91fe28b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41ac9e0 con 0x555bae9752d0 2024-09-19T06:04:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.442+0000 7f91fe28b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41ac310 con 0x555bae9749b0 2024-09-19T06:04:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.442+0000 7f91fe28b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41a9570 con 0x555bae9840c0 2024-09-19T06:04:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.442+0000 7f91fe28b740 10 client.0 _ll_register_callbacks cb 0x7f91f00bc450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.442+0000 7f91f8a8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae98c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91f8a8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae98c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41708/0 (socket says 172.21.15.5:41708) 2024-09-19T06:04:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91f8a8c640 1 -- 172.21.15.5:0/4243684729 learned_addr learned my addr 172.21.15.5:0/4243684729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91f928d640 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae97b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91f9a8e640 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9840c0 0x555bae98c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91f8a8c640 1 -- 172.21.15.5:0/4243684729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9840c0 msgr2=0x555bae98c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9840c0 0x555bae98c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91f8a8c640 1 -- 172.21.15.5:0/4243684729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 msgr2=0x555bae97b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae97b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91f8a8c640 1 -- 172.21.15.5:0/4243684729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bae98ccd0 con 0x555bae9752d0 2024-09-19T06:04:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91f8a8c640 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae98c0a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f91e4005fd0 tx=0x7f91e4034c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5350ebb07e3c0b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91dcff9640 1 -- 172.21.15.5:0/4243684729 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4044020 con 0x555bae9752d0 2024-09-19T06:04:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91dcff9640 1 -- 172.21.15.5:0/4243684729 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f91e4012a10 con 0x555bae9752d0 2024-09-19T06:04:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.443+0000 7f91fe28b740 1 -- 172.21.15.5:0/4243684729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91e4012450 con 0x555bae9752d0 2024-09-19T06:04:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.444+0000 7f91dcff9640 1 -- 172.21.15.5:0/4243684729 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91e404f050 con 0x555bae9752d0 2024-09-19T06:04:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.444+0000 7f91fe28b740 10 client.15570 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.444+0000 7f91fe28b740 1 -- 172.21.15.5:0/4243684729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555bae9faee0 con 0x555bae9752d0 2024-09-19T06:04:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.444+0000 7f91dcff9640 1 -- 172.21.15.5:0/4243684729 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91e4012450 con 0x555bae9752d0 2024-09-19T06:04:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.444+0000 7f91dcff9640 1 -- 172.21.15.5:0/4243684729 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f91e400fae0 con 0x555bae9752d0 2024-09-19T06:04:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.444+0000 7f91fe28b740 10 client.15570 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.444+0000 7f91fe28b740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.444+0000 7f91fe28b740 20 client.15570 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.444+0000 7f91fe28b740 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555bae9fb350 0x555bae9fd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.444+0000 7f91fe28b740 4 client.15570 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:04:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.444+0000 7f91fe28b740 1 -- 172.21.15.5:0/4243684729 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bae8e0d70 con 0x555bae9fb350 2024-09-19T06:04:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.444+0000 7f91f928d640 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555bae9fb350 0x555bae9fd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.445+0000 7f91dcff9640 10 client.15570 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.445+0000 7f91f928d640 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555bae9fb350 0x555bae9fd740 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3f82232ff3dc821 server_cookie=c37a0e4df69d658c in_seq=0 out_seq=0 2024-09-19T06:04:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.451+0000 7f91dcff9640 1 -- 172.21.15.5:0/4243684729 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555bae8e0d70 con 0x555bae9fb350 2024-09-19T06:04:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.451+0000 7f91dcff9640 10 client.15570 handle_command_reply: tid=0 2024-09-19T06:04:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.451+0000 7f91fe28b740 1 client.15570 shutdown 2024-09-19T06:04:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.451+0000 7f91fe28b740 1 -- 172.21.15.5:0/4243684729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 msgr2=0x555bae98c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.451+0000 7f91fe28b740 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae98c0a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f91e4005fd0 tx=0x7f91e4034c90 comp rx=0 tx=0).stop 2024-09-19T06:04:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.452+0000 7f91fe28b740 1 -- 172.21.15.5:0/4243684729 shutdown_connections 2024-09-19T06:04:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.452+0000 7f91fe28b740 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555bae9fb350 0x555bae9fd740 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.452+0000 7f91dcff9640 0 client.15570 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.452+0000 7f91fe28b740 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bae9840c0 0x555bae98c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.452+0000 7f91fe28b740 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bae9752d0 0x555bae98c0a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.452+0000 7f91fe28b740 1 --2- 172.21.15.5:0/4243684729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555bae9749b0 0x555bae97b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.452+0000 7f91fe28b740 1 -- 172.21.15.5:0/4243684729 >> 172.21.15.5:0/4243684729 conn(0x555bae971060 msgr2=0x555bae9830e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.452+0000 7f91fe28b740 1 -- 172.21.15.5:0/4243684729 shutdown_connections 2024-09-19T06:04:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.452+0000 7f91fe28b740 1 -- 172.21.15.5:0/4243684729 wait complete. 2024-09-19T06:04:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.452+0000 7f91fe28b740 20 client.15570 trim_cache size 0 max 16384 2024-09-19T06:04:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:04:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:04:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:04:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:04:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.452+0000 7f91fb4f1640 1 -- 172.21.15.5:0/551436108 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91e00a3550 msgr2=0x7f91e00a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.452+0000 7f91fb4f1640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91e00a3550 0x7f91e00a5a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f91ec005c40 tx=0x7f91ec009af0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.453+0000 7f91fb4f1640 1 -- 172.21.15.5:0/551436108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91f4100810 msgr2=0x7f91f4198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.453+0000 7f91fb4f1640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91f4100810 0x7f91f4198370 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f91f0000c00 tx=0x7f91f0030b00 comp rx=0 tx=0).stop 2024-09-19T06:04:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.453+0000 7f91fb4f1640 1 -- 172.21.15.5:0/551436108 shutdown_connections 2024-09-19T06:04:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.453+0000 7f91fb4f1640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91e00a3550 0x7f91e00a5a10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.453+0000 7f91fb4f1640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f419cc30 0x7f91f419d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.453+0000 7f91fb4f1640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91f4105500 0x7f91f41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.453+0000 7f91fb4f1640 1 --2- 172.21.15.5:0/551436108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91f4100810 0x7f91f4198370 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.453+0000 7f91fb4f1640 1 -- 172.21.15.5:0/551436108 >> 172.21.15.5:0/551436108 conn(0x7f91f4075eb0 msgr2=0x7f91f40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.453+0000 7f91fb4f1640 1 -- 172.21.15.5:0/551436108 shutdown_connections 2024-09-19T06:04:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:11.453+0000 7f91fb4f1640 1 -- 172.21.15.5:0/551436108 wait complete. 2024-09-19T06:04:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.774+0000 7fb57cb6a640 1 -- 172.21.15.5:0/901741707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5781036e0 msgr2=0x7fb578107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.774+0000 7fb57cb6a640 1 --2- 172.21.15.5:0/901741707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5781036e0 0x7fb578107b90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009930 tx=0x7fb56c02a420 comp rx=0 tx=0).stop 2024-09-19T06:04:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.774+0000 7fb57cb6a640 1 -- 172.21.15.5:0/901741707 shutdown_connections 2024-09-19T06:04:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.774+0000 7fb57cb6a640 1 --2- 172.21.15.5:0/901741707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5781082a0 0x7fb57810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.774+0000 7fb57cb6a640 1 --2- 172.21.15.5:0/901741707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5781036e0 0x7fb578107b90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.774+0000 7fb57cb6a640 1 --2- 172.21.15.5:0/901741707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb578102d30 0x7fb578103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.774+0000 7fb57cb6a640 1 -- 172.21.15.5:0/901741707 >> 172.21.15.5:0/901741707 conn(0x7fb5780fc7d0 msgr2=0x7fb5780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.774+0000 7fb57cb6a640 1 -- 172.21.15.5:0/901741707 shutdown_connections 2024-09-19T06:04:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.774+0000 7fb57cb6a640 1 -- 172.21.15.5:0/901741707 wait complete. 2024-09-19T06:04:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.775+0000 7fb57cb6a640 1 Processor -- start 2024-09-19T06:04:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.775+0000 7fb57cb6a640 1 -- start start 2024-09-19T06:04:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57cb6a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb578102d30 0x7fb578198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57cb6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5781036e0 0x7fb578198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57cb6a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5781082a0 0x7fb57819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57cb6a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb578111f60 con 0x7fb5781036e0 2024-09-19T06:04:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57cb6a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb578111de0 con 0x7fb578102d30 2024-09-19T06:04:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57cb6a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5781120e0 con 0x7fb5781082a0 2024-09-19T06:04:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb578102d30 0x7fb578198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb578102d30 0x7fb578198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56150/0 (socket says 172.21.15.5:56150) 2024-09-19T06:04:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57659c640 1 -- 172.21.15.5:0/1166563810 learned_addr learned my addr 172.21.15.5:0/1166563810 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb576d9d640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5781082a0 0x7fb57819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb575d9b640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5781036e0 0x7fb578198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57659c640 1 -- 172.21.15.5:0/1166563810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5781082a0 msgr2=0x7fb57819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57659c640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5781082a0 0x7fb57819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57659c640 1 -- 172.21.15.5:0/1166563810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5781036e0 msgr2=0x7fb578198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57659c640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5781036e0 0x7fb578198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb57659c640 1 -- 172.21.15.5:0/1166563810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57819d4d0 con 0x7fb578102d30 2024-09-19T06:04:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.776+0000 7fb575d9b640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5781036e0 0x7fb578198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:04:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.777+0000 7fb57659c640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb578102d30 0x7fb578198520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb56800d770 tx=0x7fb56800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.777+0000 7fb5677fe640 1 -- 172.21.15.5:0/1166563810 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568017070 con 0x7fb578102d30 2024-09-19T06:04:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.777+0000 7fb57cb6a640 1 -- 172.21.15.5:0/1166563810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57819d7c0 con 0x7fb578102d30 2024-09-19T06:04:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.777+0000 7fb5677fe640 1 -- 172.21.15.5:0/1166563810 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5680054d0 con 0x7fb578102d30 2024-09-19T06:04:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.777+0000 7fb5677fe640 1 -- 172.21.15.5:0/1166563810 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568004cf0 con 0x7fb578102d30 2024-09-19T06:04:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.777+0000 7fb57cb6a640 1 -- 172.21.15.5:0/1166563810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5781a50a0 con 0x7fb578102d30 2024-09-19T06:04:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.780+0000 7fb5677fe640 1 -- 172.21.15.5:0/1166563810 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7fb568004e90 con 0x7fb578102d30 2024-09-19T06:04:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.781+0000 7fb5677fe640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5600a34d0 0x7fb5600a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.781+0000 7fb5677fe640 1 -- 172.21.15.5:0/1166563810 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fb5680cfa90 con 0x7fb578102d30 2024-09-19T06:04:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.781+0000 7fb575d9b640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5600a34d0 0x7fb5600a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.782+0000 7fb575d9b640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5600a34d0 0x7fb5600a5990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb56c02a940 tx=0x7fb56c005fd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.782+0000 7fb57ef43740 1 Processor -- start 2024-09-19T06:04:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.782+0000 7fb57ef43740 1 -- start start 2024-09-19T06:04:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.783+0000 7fb57ef43740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 0x5575717a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.783+0000 7fb57ef43740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 0x5575717a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.783+0000 7fb57ef43740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717b2370 0x5575717b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.783+0000 7fb57ef43740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55757170d2f0 con 0x5575717a75d0 2024-09-19T06:04:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.783+0000 7fb57ef43740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5575716b4cf0 con 0x5575717a8b40 2024-09-19T06:04:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.783+0000 7fb57ef43740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5575717092b0 con 0x5575717b2370 2024-09-19T06:04:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.783+0000 7fb57659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 0x5575717a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.783+0000 7fb575d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 0x5575717a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.783+0000 7fb575d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 0x5575717a7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41726/0 (socket says 172.21.15.5:41726) 2024-09-19T06:04:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.783+0000 7fb575d9b640 1 -- 172.21.15.5:0/1116333193 learned_addr learned my addr 172.21.15.5:0/1116333193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.784+0000 7fb576d9d640 1 --2- 172.21.15.5:0/1116333193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717b2370 0x5575717b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.784+0000 7fb575d9b640 1 -- 172.21.15.5:0/1116333193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717b2370 msgr2=0x5575717b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.784+0000 7fb575d9b640 1 --2- 172.21.15.5:0/1116333193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717b2370 0x5575717b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.784+0000 7fb575d9b640 1 -- 172.21.15.5:0/1116333193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 msgr2=0x5575717a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.784+0000 7fb575d9b640 1 --2- 172.21.15.5:0/1116333193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 0x5575717a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.784+0000 7fb575d9b640 1 -- 172.21.15.5:0/1116333193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575716993a0 con 0x5575717a75d0 2024-09-19T06:04:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.784+0000 7fb575d9b640 1 --2- 172.21.15.5:0/1116333193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 0x5575717a7a10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009930 tx=0x7fb56c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5c52ecef1ff363 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.785+0000 7fb564ff9640 1 -- 172.21.15.5:0/1116333193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c035050 con 0x5575717a75d0 2024-09-19T06:04:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.785+0000 7fb564ff9640 1 -- 172.21.15.5:0/1116333193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb56c04c070 con 0x5575717a75d0 2024-09-19T06:04:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.785+0000 7fb564ff9640 1 -- 172.21.15.5:0/1116333193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0324b0 con 0x5575717a75d0 2024-09-19T06:04:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.786+0000 7fb57ef43740 1 -- 172.21.15.5:0/1116333193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 msgr2=0x5575717a7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.786+0000 7fb57ef43740 1 --2- 172.21.15.5:0/1116333193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 0x5575717a7a10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009930 tx=0x7fb56c02a420 comp rx=0 tx=0).stop 2024-09-19T06:04:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.786+0000 7fb57ef43740 1 -- 172.21.15.5:0/1116333193 shutdown_connections 2024-09-19T06:04:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.786+0000 7fb57ef43740 1 --2- 172.21.15.5:0/1116333193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717b2370 0x5575717b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.786+0000 7fb57ef43740 1 --2- 172.21.15.5:0/1116333193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 0x5575717a7a10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.786+0000 7fb57ef43740 1 --2- 172.21.15.5:0/1116333193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 0x5575717a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.786+0000 7fb57ef43740 1 -- 172.21.15.5:0/1116333193 >> 172.21.15.5:0/1116333193 conn(0x557571770310 msgr2=0x5575717a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.786+0000 7fb57ef43740 1 -- 172.21.15.5:0/1116333193 shutdown_connections 2024-09-19T06:04:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.786+0000 7fb57ef43740 1 -- 172.21.15.5:0/1116333193 wait complete. 2024-09-19T06:04:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.787+0000 7fb57ef43740 1 -- start start 2024-09-19T06:04:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.787+0000 7fb57ef43740 1 Processor -- start 2024-09-19T06:04:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.788+0000 7fb57ef43740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717a75d0 0x5575717b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.788+0000 7fb57ef43740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 0x5575717af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.788+0000 7fb57ef43740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717b2370 0x5575717af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.788+0000 7fb57ef43740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5575717b5010 con 0x5575717b2370 2024-09-19T06:04:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.788+0000 7fb57ef43740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5575716c8060 con 0x5575717a8b40 2024-09-19T06:04:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.788+0000 7fb57ef43740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5575717b5190 con 0x5575717a75d0 2024-09-19T06:04:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.788+0000 7fb57ef43740 10 client.0 _ll_register_callbacks cb 0x55757141b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.788+0000 7fb576d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717b2370 0x5575717af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.788+0000 7fb57659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717a75d0 0x5575717b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.788+0000 7fb57659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717a75d0 0x5575717b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52484/0 (socket says 172.21.15.5:52484) 2024-09-19T06:04:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.788+0000 7fb57659c640 1 -- 172.21.15.5:0/420366927 learned_addr learned my addr 172.21.15.5:0/420366927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.788+0000 7fb575d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 0x5575717af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.789+0000 7fb576d9d640 1 -- 172.21.15.5:0/420366927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717a75d0 msgr2=0x5575717b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.789+0000 7fb576d9d640 1 --2- 172.21.15.5:0/420366927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717a75d0 0x5575717b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.789+0000 7fb576d9d640 1 -- 172.21.15.5:0/420366927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 msgr2=0x5575717af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.789+0000 7fb576d9d640 1 --2- 172.21.15.5:0/420366927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 0x5575717af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.789+0000 7fb576d9d640 1 -- 172.21.15.5:0/420366927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757147e3c0 con 0x5575717b2370 2024-09-19T06:04:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.789+0000 7fb576d9d640 1 --2- 172.21.15.5:0/420366927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717b2370 0x5575717af7d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb570005fd0 tx=0x7fb570013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e68d77c70936055 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.789+0000 7fb55dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.790+0000 7fb55dffb640 1 -- 172.21.15.5:0/420366927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570011e90 con 0x5575717b2370 2024-09-19T06:04:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.790+0000 7fb55dffb640 1 -- 172.21.15.5:0/420366927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fb570003970 con 0x5575717b2370 2024-09-19T06:04:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.790+0000 7fb57ef43740 1 -- 172.21.15.5:0/420366927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb56c04c070 con 0x5575717b2370 2024-09-19T06:04:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.790+0000 7fb55dffb640 1 -- 172.21.15.5:0/420366927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb570003b70 con 0x5575717b2370 2024-09-19T06:04:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.790+0000 7fb57ef43740 10 client.15579 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.790+0000 7fb57ef43740 1 -- 172.21.15.5:0/420366927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5575717c0bf0 con 0x5575717b2370 2024-09-19T06:04:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.790+0000 7fb55dffb640 1 -- 172.21.15.5:0/420366927 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb56c04c070 con 0x5575717b2370 2024-09-19T06:04:11.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.791+0000 7fb55dffb640 1 -- 172.21.15.5:0/420366927 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb57000f040 con 0x5575717b2370 2024-09-19T06:04:11.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.791+0000 7fb57ef43740 10 client.15579 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:11.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.791+0000 7fb57ef43740 10 client.15579 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:04:11.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.791+0000 7fb57ef43740 20 client.15579 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:11.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.791+0000 7fb57ef43740 1 --2- 172.21.15.5:0/420366927 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5575717c20b0 0x5575717c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.791+0000 7fb57ef43740 4 client.15579 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:11.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.791+0000 7fb57ef43740 1 -- 172.21.15.5:0/420366927 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55757171a860 con 0x5575717c20b0 2024-09-19T06:04:11.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.791+0000 7fb57659c640 1 --2- 172.21.15.5:0/420366927 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5575717c20b0 0x5575717c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.791+0000 7fb57659c640 1 --2- 172.21.15.5:0/420366927 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5575717c20b0 0x5575717c44d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=751e72c36f88f764 server_cookie=1943ecfe0fe8d6fd in_seq=0 out_seq=0 2024-09-19T06:04:11.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.791+0000 7fb55dffb640 10 client.15579 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.792+0000 7fb55dffb640 1 -- 172.21.15.5:0/420366927 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55757171a860 con 0x5575717c20b0 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.792+0000 7fb55dffb640 10 client.15579 handle_command_reply: tid=0 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.792+0000 7fb57ef43740 1 client.15579 shutdown 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.792+0000 7fb57ef43740 1 -- 172.21.15.5:0/420366927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717b2370 msgr2=0x5575717af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.792+0000 7fb57ef43740 1 --2- 172.21.15.5:0/420366927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717b2370 0x5575717af7d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb570005fd0 tx=0x7fb570013dd0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.793+0000 7fb57ef43740 1 -- 172.21.15.5:0/420366927 shutdown_connections 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.793+0000 7fb57ef43740 1 --2- 172.21.15.5:0/420366927 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5575717c20b0 0x5575717c44d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.793+0000 7fb57ef43740 1 --2- 172.21.15.5:0/420366927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717b2370 0x5575717af7d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.793+0000 7fb57ef43740 1 --2- 172.21.15.5:0/420366927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 0x5575717af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.793+0000 7fb57ef43740 1 --2- 172.21.15.5:0/420366927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717a75d0 0x5575717b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.793+0000 7fb57ef43740 1 -- 172.21.15.5:0/420366927 >> 172.21.15.5:0/420366927 conn(0x557571770310 msgr2=0x5575717a9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.793+0000 7fb55dffb640 0 client.15579 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:04:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.793+0000 7fb57ef43740 1 -- 172.21.15.5:0/420366927 shutdown_connections 2024-09-19T06:04:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.793+0000 7fb57ef43740 1 -- 172.21.15.5:0/420366927 wait complete. 2024-09-19T06:04:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.793+0000 7fb57ef43740 20 client.15579 trim_cache size 0 max 16384 2024-09-19T06:04:11.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.818+0000 7fb57ef43740 1 Processor -- start 2024-09-19T06:04:11.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.818+0000 7fb57ef43740 1 -- start start 2024-09-19T06:04:11.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.818+0000 7fb57ef43740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717a75d0 0x5575717af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.818+0000 7fb57ef43740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a8b40 0x5575717be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.818+0000 7fb57ef43740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717b2370 0x5575717c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.818+0000 7fb57ef43740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5781ab5d0 con 0x5575717a8b40 2024-09-19T06:04:11.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.818+0000 7fb57ef43740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5781ad110 con 0x5575717b2370 2024-09-19T06:04:11.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.818+0000 7fb57ef43740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5781abca0 con 0x5575717a75d0 2024-09-19T06:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.819+0000 7fb575d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a8b40 0x5575717be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.819+0000 7fb575d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a8b40 0x5575717be300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41748/0 (socket says 172.21.15.5:41748) 2024-09-19T06:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.819+0000 7fb575d9b640 1 -- 172.21.15.5:0/3254561112 learned_addr learned my addr 172.21.15.5:0/3254561112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.819+0000 7fb57659c640 1 --2- 172.21.15.5:0/3254561112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717a75d0 0x5575717af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.819+0000 7fb576d9d640 1 --2- 172.21.15.5:0/3254561112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717b2370 0x5575717c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.819+0000 7fb576d9d640 1 -- 172.21.15.5:0/3254561112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717a75d0 msgr2=0x5575717af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.819+0000 7fb576d9d640 1 --2- 172.21.15.5:0/3254561112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717a75d0 0x5575717af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.819+0000 7fb576d9d640 1 -- 172.21.15.5:0/3254561112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a8b40 msgr2=0x5575717be300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.819+0000 7fb576d9d640 1 --2- 172.21.15.5:0/3254561112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a8b40 0x5575717be300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.819+0000 7fb576d9d640 1 -- 172.21.15.5:0/3254561112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575717a34f0 con 0x5575717b2370 2024-09-19T06:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.819+0000 7fb575d9b640 1 --2- 172.21.15.5:0/3254561112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a8b40 0x5575717be300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.819+0000 7fb576d9d640 1 --2- 172.21.15.5:0/3254561112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717b2370 0x5575717c9da0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb570004970 tx=0x7fb570005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ff5ba945bb5883 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:11.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.820+0000 7fb55ffff640 1 -- 172.21.15.5:0/3254561112 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570003970 con 0x5575717b2370 2024-09-19T06:04:11.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.820+0000 7fb55ffff640 1 -- 172.21.15.5:0/3254561112 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb57001c070 con 0x5575717b2370 2024-09-19T06:04:11.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.820+0000 7fb55ffff640 1 -- 172.21.15.5:0/3254561112 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5700042b0 con 0x5575717b2370 2024-09-19T06:04:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.820+0000 7fb57ef43740 1 -- 172.21.15.5:0/3254561112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717b2370 msgr2=0x5575717c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.820+0000 7fb57ef43740 1 --2- 172.21.15.5:0/3254561112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717b2370 0x5575717c9da0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb570004970 tx=0x7fb570005c70 comp rx=0 tx=0).stop 2024-09-19T06:04:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.820+0000 7fb57ef43740 1 -- 172.21.15.5:0/3254561112 shutdown_connections 2024-09-19T06:04:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.820+0000 7fb57ef43740 1 --2- 172.21.15.5:0/3254561112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717b2370 0x5575717c9da0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.820+0000 7fb57ef43740 1 --2- 172.21.15.5:0/3254561112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a8b40 0x5575717be300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.820+0000 7fb57ef43740 1 --2- 172.21.15.5:0/3254561112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717a75d0 0x5575717af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.820+0000 7fb57ef43740 1 -- 172.21.15.5:0/3254561112 >> 172.21.15.5:0/3254561112 conn(0x557571770310 msgr2=0x5575717ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57ef43740 1 -- 172.21.15.5:0/3254561112 shutdown_connections 2024-09-19T06:04:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57ef43740 1 -- 172.21.15.5:0/3254561112 wait complete. 2024-09-19T06:04:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57ef43740 1 -- start start 2024-09-19T06:04:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57ef43740 1 Processor -- start 2024-09-19T06:04:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57ef43740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 0x5575717ba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57ef43740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 0x5575717baa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57ef43740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717b2370 0x5575717b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57ef43740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5781aca40 con 0x5575717a75d0 2024-09-19T06:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57ef43740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5781ac370 con 0x5575717a8b40 2024-09-19T06:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57ef43740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5781a95b0 con 0x5575717b2370 2024-09-19T06:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57ef43740 10 client.0 _ll_register_callbacks cb 0x7fb570009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 0x5575717ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 0x5575717ba540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41758/0 (socket says 172.21.15.5:41758) 2024-09-19T06:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.821+0000 7fb57659c640 1 -- 172.21.15.5:0/696916603 learned_addr learned my addr 172.21.15.5:0/696916603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.822+0000 7fb575d9b640 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 0x5575717baa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.822+0000 7fb576d9d640 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717b2370 0x5575717b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.822+0000 7fb57659c640 1 -- 172.21.15.5:0/696916603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717b2370 msgr2=0x5575717b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.822+0000 7fb57659c640 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717b2370 0x5575717b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.822+0000 7fb57659c640 1 -- 172.21.15.5:0/696916603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 msgr2=0x5575717baa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.822+0000 7fb57659c640 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 0x5575717baa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.822+0000 7fb57659c640 1 -- 172.21.15.5:0/696916603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575717b6200 con 0x5575717a75d0 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.822+0000 7fb57659c640 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 0x5575717ba540 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb56809b7d0 tx=0x7fb5680cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b5bfbd65a8e7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.822+0000 7fb55dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.822+0000 7fb55dffb640 1 -- 172.21.15.5:0/696916603 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568020040 con 0x5575717a75d0 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.822+0000 7fb55dffb640 1 -- 172.21.15.5:0/696916603 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fb568098140 con 0x5575717a75d0 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.822+0000 7fb57ef43740 1 -- 172.21.15.5:0/696916603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb57001c070 con 0x5575717a75d0 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.823+0000 7fb55dffb640 1 -- 172.21.15.5:0/696916603 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb56809fe80 con 0x5575717a75d0 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.823+0000 7fb57ef43740 10 client.15585 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.823+0000 7fb57ef43740 1 -- 172.21.15.5:0/696916603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55757182c8b0 con 0x5575717a75d0 2024-09-19T06:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.823+0000 7fb55dffb640 1 -- 172.21.15.5:0/696916603 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5680a4740 con 0x5575717a75d0 2024-09-19T06:04:11.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.823+0000 7fb55dffb640 1 -- 172.21.15.5:0/696916603 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb568081050 con 0x5575717a75d0 2024-09-19T06:04:11.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.823+0000 7fb57ef43740 10 client.15585 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:11.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.823+0000 7fb57ef43740 10 client.15585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:04:11.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.823+0000 7fb57ef43740 20 client.15585 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:11.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.823+0000 7fb57ef43740 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55757182cf80 0x55757182f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:11.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.823+0000 7fb57ef43740 4 client.15585 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:04:11.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.823+0000 7fb57ef43740 1 -- 172.21.15.5:0/696916603 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb57001c070 con 0x55757182cf80 2024-09-19T06:04:11.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.823+0000 7fb575d9b640 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55757182cf80 0x55757182f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:11.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.824+0000 7fb55dffb640 10 client.15585 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:04:11.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.824+0000 7fb575d9b640 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55757182cf80 0x55757182f3a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea54412cd9bb72b server_cookie=6714acdae06f78a3 in_seq=0 out_seq=0 2024-09-19T06:04:11.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.824+0000 7fb55dffb640 1 -- 172.21.15.5:0/696916603 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb57001c070 con 0x55757182cf80 2024-09-19T06:04:11.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.824+0000 7fb55dffb640 10 client.15585 handle_command_reply: tid=0 2024-09-19T06:04:11.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.824+0000 7fb57ef43740 1 client.15585 shutdown 2024-09-19T06:04:11.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.824+0000 7fb57ef43740 1 -- 172.21.15.5:0/696916603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 msgr2=0x5575717ba540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.824+0000 7fb57ef43740 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 0x5575717ba540 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb56809b7d0 tx=0x7fb5680cf040 comp rx=0 tx=0).stop 2024-09-19T06:04:11.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.824+0000 7fb57ef43740 1 -- 172.21.15.5:0/696916603 shutdown_connections 2024-09-19T06:04:11.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.824+0000 7fb57ef43740 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55757182cf80 0x55757182f3a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.825+0000 7fb57ef43740 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575717b2370 0x5575717b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.825+0000 7fb55dffb640 0 client.15585 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:04:11.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.825+0000 7fb57ef43740 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575717a8b40 0x5575717baa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.825+0000 7fb57ef43740 1 --2- 172.21.15.5:0/696916603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575717a75d0 0x5575717ba540 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.825+0000 7fb57ef43740 1 -- 172.21.15.5:0/696916603 >> 172.21.15.5:0/696916603 conn(0x557571770310 msgr2=0x5575717a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:11.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.825+0000 7fb57ef43740 1 -- 172.21.15.5:0/696916603 shutdown_connections 2024-09-19T06:04:11.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.825+0000 7fb57ef43740 1 -- 172.21.15.5:0/696916603 wait complete. 2024-09-19T06:04:11.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.825+0000 7fb57ef43740 20 client.15585 trim_cache size 0 max 16384 2024-09-19T06:04:11.831 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:04:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.825+0000 7fb57cb6a640 1 -- 172.21.15.5:0/1166563810 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5600a34d0 msgr2=0x7fb5600a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.825+0000 7fb57cb6a640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5600a34d0 0x7fb5600a5990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb56c02a940 tx=0x7fb56c005fd0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.825+0000 7fb57cb6a640 1 -- 172.21.15.5:0/1166563810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb578102d30 msgr2=0x7fb578198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.825+0000 7fb57cb6a640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb578102d30 0x7fb578198520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb56800d770 tx=0x7fb56800dc30 comp rx=0 tx=0).stop 2024-09-19T06:04:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.826+0000 7fb57cb6a640 1 -- 172.21.15.5:0/1166563810 shutdown_connections 2024-09-19T06:04:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.826+0000 7fb57cb6a640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5600a34d0 0x7fb5600a5990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.826+0000 7fb57cb6a640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5781082a0 0x7fb57819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.826+0000 7fb57cb6a640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5781036e0 0x7fb578198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.826+0000 7fb57cb6a640 1 --2- 172.21.15.5:0/1166563810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb578102d30 0x7fb578198520 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.826+0000 7fb57cb6a640 1 -- 172.21.15.5:0/1166563810 >> 172.21.15.5:0/1166563810 conn(0x7fb5780fc7d0 msgr2=0x7fb578105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.826+0000 7fb57cb6a640 1 -- 172.21.15.5:0/1166563810 shutdown_connections 2024-09-19T06:04:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:11.826+0000 7fb57cb6a640 1 -- 172.21.15.5:0/1166563810 wait complete. 2024-09-19T06:04:12.467 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:12 smithi005 ceph-mon[30955]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T06:04:12.496 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:04:12.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:12 smithi157 ceph-mon[35346]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T06:04:12.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:12 smithi049 ceph-mon[34640]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T06:04:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.318+0000 7f7c09a4f640 1 -- 172.21.15.5:0/3778511426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c041082a0 msgr2=0x7f7c0410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.318+0000 7f7c09a4f640 1 --2- 172.21.15.5:0/3778511426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c041082a0 0x7f7c0410ec40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009cd0 tx=0x7f7bfc02a910 comp rx=0 tx=0).stop 2024-09-19T06:04:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.318+0000 7f7c09a4f640 1 -- 172.21.15.5:0/3778511426 shutdown_connections 2024-09-19T06:04:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.318+0000 7f7c09a4f640 1 --2- 172.21.15.5:0/3778511426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c041082a0 0x7f7c0410ec40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.318+0000 7f7c09a4f640 1 --2- 172.21.15.5:0/3778511426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c041036e0 0x7f7c04107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.318+0000 7f7c09a4f640 1 --2- 172.21.15.5:0/3778511426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c04102d30 0x7f7c04103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.318+0000 7f7c09a4f640 1 -- 172.21.15.5:0/3778511426 >> 172.21.15.5:0/3778511426 conn(0x7f7c040fc7d0 msgr2=0x7f7c040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.318+0000 7f7c09a4f640 1 -- 172.21.15.5:0/3778511426 shutdown_connections 2024-09-19T06:04:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.318+0000 7f7c09a4f640 1 -- 172.21.15.5:0/3778511426 wait complete. 2024-09-19T06:04:13.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.319+0000 7f7c09a4f640 1 Processor -- start 2024-09-19T06:04:13.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c09a4f640 1 -- start start 2024-09-19T06:04:13.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c09a4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c04102d30 0x7f7c041984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c09a4f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c041036e0 0x7f7c04198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c09a4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c041082a0 0x7f7c0419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c09a4f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04111f00 con 0x7f7c04102d30 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c09a4f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04111d80 con 0x7f7c041082a0 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c09a4f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04112080 con 0x7f7c041036e0 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c04102d30 0x7f7c041984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c04102d30 0x7f7c041984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41772/0 (socket says 172.21.15.5:41772) 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c02ffd640 1 -- 172.21.15.5:0/2514933551 learned_addr learned my addr 172.21.15.5:0/2514933551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c037fe640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c041082a0 0x7f7c0419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c02ffd640 1 -- 172.21.15.5:0/2514933551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c041036e0 msgr2=0x7f7c04198a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c041036e0 0x7f7c04198a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c02ffd640 1 -- 172.21.15.5:0/2514933551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c041082a0 msgr2=0x7f7c0419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:13.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c041082a0 0x7f7c0419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.320+0000 7f7c02ffd640 1 -- 172.21.15.5:0/2514933551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0419d470 con 0x7f7c04102d30 2024-09-19T06:04:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.321+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c04102d30 0x7f7c041984c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400e9f0 tx=0x7f7bf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.321+0000 7f7c08a4d640 1 -- 172.21.15.5:0/2514933551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400cd80 con 0x7f7c04102d30 2024-09-19T06:04:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.321+0000 7f7c08a4d640 1 -- 172.21.15.5:0/2514933551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bf40108e0 con 0x7f7c04102d30 2024-09-19T06:04:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.321+0000 7f7c08a4d640 1 -- 172.21.15.5:0/2514933551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4018920 con 0x7f7c04102d30 2024-09-19T06:04:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.322+0000 7f7c09a4f640 1 -- 172.21.15.5:0/2514933551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0419d760 con 0x7f7c04102d30 2024-09-19T06:04:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.323+0000 7f7c09a4f640 1 -- 172.21.15.5:0/2514933551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c041a5040 con 0x7f7c04102d30 2024-09-19T06:04:13.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.326+0000 7f7c08a4d640 1 -- 172.21.15.5:0/2514933551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7f7bf401f020 con 0x7f7c04102d30 2024-09-19T06:04:13.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.326+0000 7f7c0be28740 1 Processor -- start 2024-09-19T06:04:13.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.326+0000 7f7c0be28740 1 -- start start 2024-09-19T06:04:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.326+0000 7f7c0be28740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d369b40 0x56105d368090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.326+0000 7f7c0be28740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d3685d0 0x56105d368a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.326+0000 7f7c0be28740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d373370 0x56105d375760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.326+0000 7f7c0be28740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56105d2ce2f0 con 0x56105d369b40 2024-09-19T06:04:13.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.326+0000 7f7c0be28740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56105d275cf0 con 0x56105d3685d0 2024-09-19T06:04:13.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c0be28740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56105d2ca2b0 con 0x56105d373370 2024-09-19T06:04:13.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c02ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d3685d0 0x56105d368a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c027fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d373370 0x56105d375760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c02ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d3685d0 0x56105d368a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56190/0 (socket says 172.21.15.5:56190) 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c02ffd640 1 -- 172.21.15.5:0/3985443158 learned_addr learned my addr 172.21.15.5:0/3985443158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c08a4d640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7bec0a34d0 0x7f7bec0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c037fe640 1 --2- 172.21.15.5:0/3985443158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d369b40 0x56105d368090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c08a4d640 1 -- 172.21.15.5:0/2514933551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f7bf40d0100 con 0x7f7c04102d30 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c02ffd640 1 -- 172.21.15.5:0/3985443158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d373370 msgr2=0x56105d375760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/3985443158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d373370 0x56105d375760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c02ffd640 1 -- 172.21.15.5:0/3985443158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d369b40 msgr2=0x56105d368090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/3985443158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d369b40 0x56105d368090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c02ffd640 1 -- 172.21.15.5:0/3985443158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105d25a3a0 con 0x56105d3685d0 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.327+0000 7f7c037fe640 1 --2- 172.21.15.5:0/3985443158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d369b40 0x56105d368090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.328+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/3985443158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d3685d0 0x56105d368a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40cd810 tx=0x7f7bf4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d7bfebe593a18f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.328+0000 7f7c037fe640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7bec0a34d0 0x7f7bec0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.328+0000 7f7be9ffb640 1 -- 172.21.15.5:0/3985443158 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf409c940 con 0x56105d3685d0 2024-09-19T06:04:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.328+0000 7f7be9ffb640 1 -- 172.21.15.5:0/3985443158 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bf409c240 con 0x56105d3685d0 2024-09-19T06:04:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.328+0000 7f7be9ffb640 1 -- 172.21.15.5:0/3985443158 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40bc0d0 con 0x56105d3685d0 2024-09-19T06:04:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.329+0000 7f7c0be28740 1 -- 172.21.15.5:0/3985443158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d3685d0 msgr2=0x56105d368a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.329+0000 7f7c037fe640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7bec0a34d0 0x7f7bec0a5990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc006010 tx=0x7f7bfc03f040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.329+0000 7f7c0be28740 1 --2- 172.21.15.5:0/3985443158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d3685d0 0x56105d368a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40cd810 tx=0x7f7bf4000bc0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.329+0000 7f7c0be28740 1 -- 172.21.15.5:0/3985443158 shutdown_connections 2024-09-19T06:04:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.329+0000 7f7c0be28740 1 --2- 172.21.15.5:0/3985443158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d373370 0x56105d375760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.329+0000 7f7c0be28740 1 --2- 172.21.15.5:0/3985443158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d3685d0 0x56105d368a10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.329+0000 7f7c0be28740 1 --2- 172.21.15.5:0/3985443158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d369b40 0x56105d368090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.329+0000 7f7c0be28740 1 -- 172.21.15.5:0/3985443158 >> 172.21.15.5:0/3985443158 conn(0x56105d331310 msgr2=0x56105d364c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.329+0000 7f7c0be28740 1 -- 172.21.15.5:0/3985443158 shutdown_connections 2024-09-19T06:04:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.329+0000 7f7c0be28740 1 -- 172.21.15.5:0/3985443158 wait complete. 2024-09-19T06:04:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.329+0000 7f7c0be28740 1 -- start start 2024-09-19T06:04:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.330+0000 7f7c0be28740 1 Processor -- start 2024-09-19T06:04:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.330+0000 7f7c0be28740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d372e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.330+0000 7f7c0be28740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 0x56105d3700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d372e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d372e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41790/0 (socket says 172.21.15.5:41790) 2024-09-19T06:04:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c02ffd640 1 -- 172.21.15.5:0/3152848366 learned_addr learned my addr 172.21.15.5:0/3152848366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c0be28740 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d370840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c0be28740 1 -- 172.21.15.5:0/3152848366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56105d376010 con 0x56105d3685d0 2024-09-19T06:04:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c0be28740 1 -- 172.21.15.5:0/3152848366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56105d289060 con 0x56105d373370 2024-09-19T06:04:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c0be28740 1 -- 172.21.15.5:0/3152848366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56105d376190 con 0x56105d369b40 2024-09-19T06:04:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c037fe640 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d370840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c027fc640 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 0x56105d3700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c0be28740 10 client.0 _ll_register_callbacks cb 0x56105cfdc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c037fe640 1 -- 172.21.15.5:0/3152848366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 msgr2=0x56105d3700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c037fe640 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 0x56105d3700b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c037fe640 1 -- 172.21.15.5:0/3152848366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 msgr2=0x56105d372e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c037fe640 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d372e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.331+0000 7f7c037fe640 1 -- 172.21.15.5:0/3152848366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105d03f3c0 con 0x56105d373370 2024-09-19T06:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.332+0000 7f7c037fe640 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d370840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009ca0 tx=0x7f7bfc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=540ff5f177790922 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.332+0000 7f7be2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.332+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d372e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.332+0000 7f7be2ffd640 1 -- 172.21.15.5:0/3152848366 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc030eb0 con 0x56105d373370 2024-09-19T06:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.332+0000 7f7be2ffd640 1 -- 172.21.15.5:0/3152848366 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f7bfc046480 con 0x56105d373370 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.332+0000 7f7c0be28740 1 -- 172.21.15.5:0/3152848366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bf409c240 con 0x56105d373370 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.333+0000 7f7be2ffd640 1 -- 172.21.15.5:0/3152848366 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7bfc046680 con 0x56105d373370 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.333+0000 7f7c0be28740 10 client.25220 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.333+0000 7f7c0be28740 1 -- 172.21.15.5:0/3152848366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56105d381bb0 con 0x56105d373370 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.333+0000 7f7be2ffd640 1 -- 172.21.15.5:0/3152848366 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bf409c240 con 0x56105d373370 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.333+0000 7f7be2ffd640 1 -- 172.21.15.5:0/3152848366 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7bfc051880 con 0x56105d373370 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.333+0000 7f7c0be28740 10 client.25220 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.333+0000 7f7c0be28740 10 client.25220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.333+0000 7f7c0be28740 20 client.25220 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.333+0000 7f7c0be28740 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56105d383070 0x56105d3854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.334+0000 7f7c0be28740 4 client.25220 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.334+0000 7f7c0be28740 1 -- 172.21.15.5:0/3152848366 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56105d2db860 con 0x56105d383070 2024-09-19T06:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.334+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56105d383070 0x56105d3854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.334+0000 7f7be2ffd640 10 client.25220 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.335+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56105d383070 0x56105d3854d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b949ff426d2942 server_cookie=e51ef38b11964456 in_seq=0 out_seq=0 2024-09-19T06:04:13.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.335+0000 7f7be2ffd640 1 -- 172.21.15.5:0/3152848366 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56105d2db860 con 0x56105d383070 2024-09-19T06:04:13.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.335+0000 7f7be2ffd640 10 client.25220 handle_command_reply: tid=0 2024-09-19T06:04:13.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.335+0000 7f7c0be28740 1 client.25220 shutdown 2024-09-19T06:04:13.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.336+0000 7f7c0be28740 1 -- 172.21.15.5:0/3152848366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 msgr2=0x56105d370840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.336+0000 7f7c0be28740 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d370840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009ca0 tx=0x7f7bfc004a40 comp rx=0 tx=0).stop 2024-09-19T06:04:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.336+0000 7f7c0be28740 1 -- 172.21.15.5:0/3152848366 shutdown_connections 2024-09-19T06:04:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.336+0000 7f7c0be28740 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56105d383070 0x56105d3854d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.336+0000 7f7be2ffd640 0 client.25220 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.336+0000 7f7c0be28740 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d370840 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.336+0000 7f7c0be28740 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 0x56105d3700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.336+0000 7f7c0be28740 1 --2- 172.21.15.5:0/3152848366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d372e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.336+0000 7f7c0be28740 1 -- 172.21.15.5:0/3152848366 >> 172.21.15.5:0/3152848366 conn(0x56105d331310 msgr2=0x56105d36a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.336+0000 7f7c0be28740 1 -- 172.21.15.5:0/3152848366 shutdown_connections 2024-09-19T06:04:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.336+0000 7f7c0be28740 1 -- 172.21.15.5:0/3152848366 wait complete. 2024-09-19T06:04:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.336+0000 7f7c0be28740 20 client.25220 trim_cache size 0 max 16384 2024-09-19T06:04:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.355+0000 7f7c0be28740 1 Processor -- start 2024-09-19T06:04:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.355+0000 7f7c0be28740 1 -- start start 2024-09-19T06:04:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c0be28740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d36f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c0be28740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 0x56105d36f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c0be28740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d38ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c0be28740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04074280 con 0x56105d3685d0 2024-09-19T06:04:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c0be28740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041a6bb0 con 0x56105d373370 2024-09-19T06:04:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c0be28740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041a5740 con 0x56105d369b40 2024-09-19T06:04:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d36f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d36f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41794/0 (socket says 172.21.15.5:41794) 2024-09-19T06:04:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c02ffd640 1 -- 172.21.15.5:0/2483990219 learned_addr learned my addr 172.21.15.5:0/2483990219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c037fe640 1 --2- 172.21.15.5:0/2483990219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d38ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c027fc640 1 --2- 172.21.15.5:0/2483990219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 0x56105d36f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c02ffd640 1 -- 172.21.15.5:0/2483990219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 msgr2=0x56105d36f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/2483990219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 0x56105d36f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c02ffd640 1 -- 172.21.15.5:0/2483990219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 msgr2=0x56105d38ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/2483990219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d38ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c02ffd640 1 -- 172.21.15.5:0/2483990219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105d369300 con 0x56105d3685d0 2024-09-19T06:04:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.356+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/2483990219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d36f970 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7bf409a4f0 tx=0x7f7bf40d04a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93b46f1e07429bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.357+0000 7f7be97fa640 1 -- 172.21.15.5:0/2483990219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf409c240 con 0x56105d3685d0 2024-09-19T06:04:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.357+0000 7f7be97fa640 1 -- 172.21.15.5:0/2483990219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bf40988b0 con 0x56105d3685d0 2024-09-19T06:04:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.357+0000 7f7be97fa640 1 -- 172.21.15.5:0/2483990219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf409c700 con 0x56105d3685d0 2024-09-19T06:04:13.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.357+0000 7f7c0be28740 1 -- 172.21.15.5:0/2483990219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 msgr2=0x56105d36f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:13.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.357+0000 7f7c0be28740 1 --2- 172.21.15.5:0/2483990219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d36f970 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7bf409a4f0 tx=0x7f7bf40d04a0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.358+0000 7f7c0be28740 1 -- 172.21.15.5:0/2483990219 shutdown_connections 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.358+0000 7f7c0be28740 1 --2- 172.21.15.5:0/2483990219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d38ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.358+0000 7f7c0be28740 1 --2- 172.21.15.5:0/2483990219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 0x56105d36f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.358+0000 7f7c0be28740 1 --2- 172.21.15.5:0/2483990219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d36f970 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.358+0000 7f7c0be28740 1 -- 172.21.15.5:0/2483990219 >> 172.21.15.5:0/2483990219 conn(0x56105d331310 msgr2=0x56105d36e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.358+0000 7f7c0be28740 1 -- 172.21.15.5:0/2483990219 shutdown_connections 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.358+0000 7f7c0be28740 1 -- 172.21.15.5:0/2483990219 wait complete. 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.358+0000 7f7c0be28740 1 -- start start 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.358+0000 7f7c0be28740 1 Processor -- start 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c0be28740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d36d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c0be28740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 0x56105d36c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c0be28740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d36c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d36d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d36d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41804/0 (socket says 172.21.15.5:41804) 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c02ffd640 1 -- 172.21.15.5:0/2530447930 learned_addr learned my addr 172.21.15.5:0/2530447930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c0be28740 1 -- 172.21.15.5:0/2530447930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041a64e0 con 0x56105d3685d0 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c0be28740 1 -- 172.21.15.5:0/2530447930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041a5e10 con 0x56105d373370 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c0be28740 1 -- 172.21.15.5:0/2530447930 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04072260 con 0x56105d369b40 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c0be28740 10 client.0 _ll_register_callbacks cb 0x7f7bf8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c027fc640 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 0x56105d36c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c037fe640 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d36c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c02ffd640 1 -- 172.21.15.5:0/2530447930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 msgr2=0x56105d36c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 0x56105d36c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c02ffd640 1 -- 172.21.15.5:0/2530447930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 msgr2=0x56105d36c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d36c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c02ffd640 1 -- 172.21.15.5:0/2530447930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105d36ce80 con 0x56105d3685d0 2024-09-19T06:04:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c037fe640 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d36c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:04:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7c02ffd640 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d36d900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400cfd0 tx=0x7f7bf40a0460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e6683aeff41210 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.359+0000 7f7be2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7be2ffd640 1 -- 172.21.15.5:0/2530447930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40b5060 con 0x56105d3685d0 2024-09-19T06:04:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7c0be28740 1 -- 172.21.15.5:0/2530447930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bf40988b0 con 0x56105d3685d0 2024-09-19T06:04:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7be2ffd640 1 -- 172.21.15.5:0/2530447930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f7bf409c6b0 con 0x56105d3685d0 2024-09-19T06:04:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7be2ffd640 1 -- 172.21.15.5:0/2530447930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7bf409c940 con 0x56105d3685d0 2024-09-19T06:04:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7c0be28740 10 client.15603 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7c0be28740 1 -- 172.21.15.5:0/2530447930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56105d3f1ab0 con 0x56105d3685d0 2024-09-19T06:04:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7be2ffd640 1 -- 172.21.15.5:0/2530447930 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bf40988b0 con 0x56105d3685d0 2024-09-19T06:04:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7be2ffd640 1 -- 172.21.15.5:0/2530447930 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7bf40cdaf0 con 0x56105d3685d0 2024-09-19T06:04:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7c0be28740 10 client.15603 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7c0be28740 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7c0be28740 20 client.15603 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7c0be28740 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56105d3f1ce0 0x56105d3f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7c0be28740 4 client.15603 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.360+0000 7f7c0be28740 1 -- 172.21.15.5:0/2530447930 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56105d2d5d70 con 0x56105d3f1ce0 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.361+0000 7f7c027fc640 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56105d3f1ce0 0x56105d3f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.361+0000 7f7be2ffd640 10 client.15603 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.361+0000 7f7c027fc640 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56105d3f1ce0 0x56105d3f40d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de996222d5b7aff server_cookie=b48dea955013161c in_seq=0 out_seq=0 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.361+0000 7f7be2ffd640 1 -- 172.21.15.5:0/2530447930 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56105d2d5d70 con 0x56105d3f1ce0 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.361+0000 7f7be2ffd640 10 client.15603 handle_command_reply: tid=0 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7c0be28740 1 client.15603 shutdown 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7c0be28740 1 -- 172.21.15.5:0/2530447930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 msgr2=0x56105d36d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7c0be28740 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d36d900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400cfd0 tx=0x7f7bf40a0460 comp rx=0 tx=0).stop 2024-09-19T06:04:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7c0be28740 1 -- 172.21.15.5:0/2530447930 shutdown_connections 2024-09-19T06:04:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7c0be28740 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56105d3f1ce0 0x56105d3f40d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7be2ffd640 0 client.15603 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7c0be28740 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105d373370 0x56105d36c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7c0be28740 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105d369b40 0x56105d36c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7c0be28740 1 --2- 172.21.15.5:0/2530447930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105d3685d0 0x56105d36d900 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7c0be28740 1 -- 172.21.15.5:0/2530447930 >> 172.21.15.5:0/2530447930 conn(0x56105d331310 msgr2=0x56105d36f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7c0be28740 1 -- 172.21.15.5:0/2530447930 shutdown_connections 2024-09-19T06:04:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7c0be28740 1 -- 172.21.15.5:0/2530447930 wait complete. 2024-09-19T06:04:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.362+0000 7f7c0be28740 20 client.15603 trim_cache size 0 max 16384 2024-09-19T06:04:13.370 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:04:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.363+0000 7f7c09a4f640 1 -- 172.21.15.5:0/2514933551 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7bec0a34d0 msgr2=0x7f7bec0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.363+0000 7f7c09a4f640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7bec0a34d0 0x7f7bec0a5990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc006010 tx=0x7f7bfc03f040 comp rx=0 tx=0).stop 2024-09-19T06:04:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.363+0000 7f7c09a4f640 1 -- 172.21.15.5:0/2514933551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c04102d30 msgr2=0x7f7c041984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.363+0000 7f7c09a4f640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c04102d30 0x7f7c041984c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400e9f0 tx=0x7f7bf400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.363+0000 7f7c09a4f640 1 -- 172.21.15.5:0/2514933551 shutdown_connections 2024-09-19T06:04:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.363+0000 7f7c09a4f640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c041082a0 0x7f7c0419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.363+0000 7f7c09a4f640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7bec0a34d0 0x7f7bec0a5990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.363+0000 7f7c09a4f640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c041036e0 0x7f7c04198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.363+0000 7f7c09a4f640 1 --2- 172.21.15.5:0/2514933551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c04102d30 0x7f7c041984c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.363+0000 7f7c09a4f640 1 -- 172.21.15.5:0/2514933551 >> 172.21.15.5:0/2514933551 conn(0x7f7c040fc7d0 msgr2=0x7f7c04105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.363+0000 7f7c09a4f640 1 -- 172.21.15.5:0/2514933551 shutdown_connections 2024-09-19T06:04:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:13.363+0000 7f7c09a4f640 1 -- 172.21.15.5:0/2514933551 wait complete. 2024-09-19T06:04:14.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:14 smithi005 ceph-mon[30955]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-19T06:04:14.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:14 smithi157 ceph-mon[35346]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-19T06:04:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:14 smithi049 ceph-mon[34640]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-19T06:04:15.172 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:04:15.172 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:04:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:16.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.103+0000 7f53fd11c640 1 -- 172.21.15.5:0/160088266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53f8102570 msgr2=0x7f53f81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:16.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.103+0000 7f53fd11c640 1 --2- 172.21.15.5:0/160088266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53f8102570 0x7f53f81029b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009900 tx=0x7f53e002a480 comp rx=0 tx=0).stop 2024-09-19T06:04:16.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.103+0000 7f53fd11c640 1 -- 172.21.15.5:0/160088266 shutdown_connections 2024-09-19T06:04:16.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.103+0000 7f53fd11c640 1 --2- 172.21.15.5:0/160088266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8102ef0 0x7f53f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.103+0000 7f53fd11c640 1 --2- 172.21.15.5:0/160088266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53f8102570 0x7f53f81029b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.103+0000 7f53fd11c640 1 --2- 172.21.15.5:0/160088266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f8100e20 0x7f53f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.103+0000 7f53fd11c640 1 -- 172.21.15.5:0/160088266 >> 172.21.15.5:0/160088266 conn(0x7f53f80fc7d0 msgr2=0x7f53f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:16.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.104+0000 7f53fd11c640 1 -- 172.21.15.5:0/160088266 shutdown_connections 2024-09-19T06:04:16.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.104+0000 7f53fd11c640 1 -- 172.21.15.5:0/160088266 wait complete. 2024-09-19T06:04:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.105+0000 7f53fd11c640 1 Processor -- start 2024-09-19T06:04:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.105+0000 7f53fd11c640 1 -- start start 2024-09-19T06:04:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.105+0000 7f53fd11c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8100e20 0x7f53f810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.105+0000 7f53fd11c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53f8102570 0x7f53f810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.105+0000 7f53fd11c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f8102ef0 0x7f53f810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.105+0000 7f53fd11c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8111f80 con 0x7f53f8102570 2024-09-19T06:04:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.105+0000 7f53fd11c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8111e00 con 0x7f53f8100e20 2024-09-19T06:04:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.105+0000 7f53fd11c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8112100 con 0x7f53f8102ef0 2024-09-19T06:04:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.105+0000 7f53effff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53f8102570 0x7f53f810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.105+0000 7f53effff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53f8102570 0x7f53f810ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41136/0 (socket says 172.21.15.5:41136) 2024-09-19T06:04:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.106+0000 7f53effff640 1 -- 172.21.15.5:0/2265528384 learned_addr learned my addr 172.21.15.5:0/2265528384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.106+0000 7f53f759e640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f8102ef0 0x7f53f810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.106+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8100e20 0x7f53f810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.106+0000 7f53effff640 1 -- 172.21.15.5:0/2265528384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f8102ef0 msgr2=0x7f53f810ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.106+0000 7f53effff640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f8102ef0 0x7f53f810ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.106+0000 7f53effff640 1 -- 172.21.15.5:0/2265528384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8100e20 msgr2=0x7f53f810e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.106+0000 7f53effff640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8100e20 0x7f53f810e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.106+0000 7f53effff640 1 -- 172.21.15.5:0/2265528384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f81a68a0 con 0x7f53f8102570 2024-09-19T06:04:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.106+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8100e20 0x7f53f810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.107+0000 7f53effff640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53f8102570 0x7f53f810ea10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f53e002aa70 tx=0x7f53e0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.107+0000 7f53f4d99640 1 -- 172.21.15.5:0/2265528384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e003f070 con 0x7f53f8102570 2024-09-19T06:04:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.107+0000 7f53fd11c640 1 -- 172.21.15.5:0/2265528384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f81a6b30 con 0x7f53f8102570 2024-09-19T06:04:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.107+0000 7f53fd11c640 1 -- 172.21.15.5:0/2265528384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f81a7060 con 0x7f53f8102570 2024-09-19T06:04:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.107+0000 7f53f4d99640 1 -- 172.21.15.5:0/2265528384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e003bb20 con 0x7f53f8102570 2024-09-19T06:04:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.108+0000 7f53f4d99640 1 -- 172.21.15.5:0/2265528384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0035030 con 0x7f53f8102570 2024-09-19T06:04:16.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.110+0000 7f53ff4f5740 1 Processor -- start 2024-09-19T06:04:16.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.111+0000 7f53ff4f5740 1 -- start start 2024-09-19T06:04:16.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.111+0000 7f53ff4f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bb5b60 0x558af6bb5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.111+0000 7f53ff4f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb6480 0x558af6bc0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.111+0000 7f53ff4f5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bc0d40 0x558af6bc3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.111+0000 7f53ff4f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558af6b1b280 con 0x558af6bb6480 2024-09-19T06:04:16.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.111+0000 7f53ff4f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558af6ac2b10 con 0x558af6bb5b60 2024-09-19T06:04:16.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.111+0000 7f53ff4f5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558af6b17240 con 0x558af6bc0d40 2024-09-19T06:04:16.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.112+0000 7f53f6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb6480 0x558af6bc0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.112+0000 7f53f6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb6480 0x558af6bc0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41144/0 (socket says 172.21.15.5:41144) 2024-09-19T06:04:16.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.112+0000 7f53f6d9d640 1 -- 172.21.15.5:0/3327442384 learned_addr learned my addr 172.21.15.5:0/3327442384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:16.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.112+0000 7f53f759e640 1 --2- 172.21.15.5:0/3327442384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bb5b60 0x558af6bb5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.112+0000 7f53f4d99640 1 -- 172.21.15.5:0/2265528384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7f53e0042070 con 0x7f53f8102570 2024-09-19T06:04:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.112+0000 7f53f6d9d640 1 -- 172.21.15.5:0/3327442384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bc0d40 msgr2=0x558af6bc3130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:04:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.112+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/3327442384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bc0d40 0x558af6bc3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.112+0000 7f53f6d9d640 1 -- 172.21.15.5:0/3327442384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bb5b60 msgr2=0x558af6bb5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.112+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/3327442384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bb5b60 0x558af6bb5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.112+0000 7f53f6d9d640 1 -- 172.21.15.5:0/3327442384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af6acf1f0 con 0x558af6bb6480 2024-09-19T06:04:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.112+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/3327442384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb6480 0x558af6bc0800 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009d40 tx=0x7f53e80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4144edeaa3ad0ed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.113+0000 7f53f759e640 1 --2- 172.21.15.5:0/3327442384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bb5b60 0x558af6bb5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.113+0000 7f53ed7fa640 1 -- 172.21.15.5:0/3327442384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8012650 con 0x558af6bb6480 2024-09-19T06:04:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.113+0000 7f53ed7fa640 1 -- 172.21.15.5:0/3327442384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e8009e30 con 0x558af6bb6480 2024-09-19T06:04:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.113+0000 7f53ed7fa640 1 -- 172.21.15.5:0/3327442384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e800fc00 con 0x558af6bb6480 2024-09-19T06:04:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.114+0000 7f53f4d99640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53e40a3550 0x7f53e40a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.114+0000 7f53ff4f5740 1 -- 172.21.15.5:0/3327442384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb6480 msgr2=0x558af6bc0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.114+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/3327442384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb6480 0x558af6bc0800 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009d40 tx=0x7f53e80117b0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.114+0000 7f53f759e640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53e40a3550 0x7f53e40a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.114+0000 7f53ff4f5740 1 -- 172.21.15.5:0/3327442384 shutdown_connections 2024-09-19T06:04:16.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.114+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/3327442384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bc0d40 0x558af6bc3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.114+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/3327442384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb6480 0x558af6bc0800 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.115+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/3327442384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bb5b60 0x558af6bb5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.115+0000 7f53ff4f5740 1 -- 172.21.15.5:0/3327442384 >> 172.21.15.5:0/3327442384 conn(0x558af6bb20f0 msgr2=0x558af6bb24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:16.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.115+0000 7f53f4d99640 1 -- 172.21.15.5:0/2265528384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f53e00f04e0 con 0x7f53f8102570 2024-09-19T06:04:16.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.115+0000 7f53f759e640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53e40a3550 0x7f53e40a5a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f53f0005fd0 tx=0x7f53f0009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:16.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.115+0000 7f53ff4f5740 1 -- 172.21.15.5:0/3327442384 shutdown_connections 2024-09-19T06:04:16.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.115+0000 7f53ff4f5740 1 -- 172.21.15.5:0/3327442384 wait complete. 2024-09-19T06:04:16.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.116+0000 7f53ff4f5740 1 -- start start 2024-09-19T06:04:16.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.116+0000 7f53ff4f5740 1 Processor -- start 2024-09-19T06:04:16.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.116+0000 7f53ff4f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 0x558af6bbdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.116+0000 7f53ff4f5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bb6480 0x558af6bbe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.116+0000 7f53ff4f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bbe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.116+0000 7f53ff4f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558af6bc3910 con 0x558af6bb5b60 2024-09-19T06:04:16.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.116+0000 7f53ff4f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558af6ad5f10 con 0x558af6bc0d40 2024-09-19T06:04:16.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.116+0000 7f53ff4f5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558af6bc3a90 con 0x558af6bb6480 2024-09-19T06:04:16.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.116+0000 7f53ff4f5740 10 client.0 _ll_register_callbacks cb 0x558af6a9ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:16.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.117+0000 7f53f759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bbe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.117+0000 7f53f759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bbe610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59220/0 (socket says 172.21.15.5:59220) 2024-09-19T06:04:16.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.117+0000 7f53effff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bb6480 0x558af6bbe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.117+0000 7f53f759e640 1 -- 172.21.15.5:0/2869190481 learned_addr learned my addr 172.21.15.5:0/2869190481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:16.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.117+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 0x558af6bbdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.117+0000 7f53effff640 1 -- 172.21.15.5:0/2869190481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 msgr2=0x558af6bbe610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:16.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.117+0000 7f53effff640 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bbe610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.117+0000 7f53effff640 1 -- 172.21.15.5:0/2869190481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 msgr2=0x558af6bbdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:16.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.118+0000 7f53effff640 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 0x558af6bbdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.118+0000 7f53effff640 1 -- 172.21.15.5:0/2869190481 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af688c6a0 con 0x558af6bb6480 2024-09-19T06:04:16.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.118+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 0x558af6bbdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:16.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.118+0000 7f53effff640 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bb6480 0x558af6bbe0d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f53e00eda60 tx=0x7f53e00bab00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=589db333d04aca87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:16.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.118+0000 7f53de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:04:16.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.118+0000 7f53de7fc640 1 -- 172.21.15.5:0/2869190481 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00bbf50 con 0x558af6bb6480 2024-09-19T06:04:16.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.118+0000 7f53ff4f5740 1 -- 172.21.15.5:0/2869190481 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e8009e30 con 0x558af6bb6480 2024-09-19T06:04:16.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.118+0000 7f53de7fc640 1 -- 172.21.15.5:0/2869190481 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f53e00b8c60 con 0x558af6bb6480 2024-09-19T06:04:16.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.118+0000 7f53de7fc640 1 -- 172.21.15.5:0/2869190481 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f53e00bccf0 con 0x558af6bb6480 2024-09-19T06:04:16.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.118+0000 7f53ff4f5740 10 client.25165 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:16.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.118+0000 7f53ff4f5740 1 -- 172.21.15.5:0/2869190481 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558af6bc39c0 con 0x558af6bb6480 2024-09-19T06:04:16.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.119+0000 7f53de7fc640 1 -- 172.21.15.5:0/2869190481 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e8009e30 con 0x558af6bb6480 2024-09-19T06:04:16.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.119+0000 7f53de7fc640 1 -- 172.21.15.5:0/2869190481 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f53e00c4dd0 con 0x558af6bb6480 2024-09-19T06:04:16.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.119+0000 7f53ff4f5740 10 client.25165 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:16.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.119+0000 7f53ff4f5740 10 client.25165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:16.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.120+0000 7f53ff4f5740 20 client.25165 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:16.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.120+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558af6bd0c30 0x558af6bd3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.120+0000 7f53ff4f5740 4 client.25165 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:16.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.120+0000 7f53ff4f5740 1 -- 172.21.15.5:0/2869190481 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558af6b287f0 con 0x558af6bd0c30 2024-09-19T06:04:16.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.120+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558af6bd0c30 0x558af6bd3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.120+0000 7f53de7fc640 10 client.25165 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:16.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.120+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558af6bd0c30 0x558af6bd3050 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f04ad892c2799c1c server_cookie=f21304c0b4f1ab96 in_seq=0 out_seq=0 2024-09-19T06:04:16.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.121+0000 7f53de7fc640 1 -- 172.21.15.5:0/2869190481 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558af6b287f0 con 0x558af6bd0c30 2024-09-19T06:04:16.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.121+0000 7f53de7fc640 10 client.25165 handle_command_reply: tid=0 2024-09-19T06:04:16.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.121+0000 7f53ff4f5740 1 client.25165 shutdown 2024-09-19T06:04:16.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.121+0000 7f53ff4f5740 1 -- 172.21.15.5:0/2869190481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bb6480 msgr2=0x558af6bbe0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:16.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.121+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bb6480 0x558af6bbe0d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f53e00eda60 tx=0x7f53e00bab00 comp rx=0 tx=0).stop 2024-09-19T06:04:16.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.121+0000 7f53ff4f5740 1 -- 172.21.15.5:0/2869190481 shutdown_connections 2024-09-19T06:04:16.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.121+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558af6bd0c30 0x558af6bd3050 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.121+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bbe610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.121+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bb6480 0x558af6bbe0d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.121+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/2869190481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 0x558af6bbdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.121+0000 7f53ff4f5740 1 -- 172.21.15.5:0/2869190481 >> 172.21.15.5:0/2869190481 conn(0x558af6bb20f0 msgr2=0x558af6bb7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:16.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.122+0000 7f53de7fc640 0 client.25165 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:16.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.122+0000 7f53ff4f5740 1 -- 172.21.15.5:0/2869190481 shutdown_connections 2024-09-19T06:04:16.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.122+0000 7f53ff4f5740 1 -- 172.21.15.5:0/2869190481 wait complete. 2024-09-19T06:04:16.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.122+0000 7f53ff4f5740 20 client.25165 trim_cache size 0 max 16384 2024-09-19T06:04:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.157+0000 7f53ff4f5740 1 Processor -- start 2024-09-19T06:04:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.157+0000 7f53ff4f5740 1 -- start start 2024-09-19T06:04:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.157+0000 7f53ff4f5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bb5b60 0x558af6bbf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.157+0000 7f53ff4f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bd8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.158+0000 7f53ff4f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bd8e80 0x558af6bbae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.158+0000 7f53ff4f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53d800e4d0 con 0x558af6bd8e80 2024-09-19T06:04:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.158+0000 7f53ff4f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53d8010010 con 0x558af6bc0d40 2024-09-19T06:04:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.158+0000 7f53ff4f5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53d800eba0 con 0x558af6bb5b60 2024-09-19T06:04:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.158+0000 7f53effff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bd8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.158+0000 7f53f759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bd8e80 0x558af6bbae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.158+0000 7f53f6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bb5b60 0x558af6bbf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.158+0000 7f53f759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bd8e80 0x558af6bbae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41158/0 (socket says 172.21.15.5:41158) 2024-09-19T06:04:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.158+0000 7f53f759e640 1 -- 172.21.15.5:0/1171860589 learned_addr learned my addr 172.21.15.5:0/1171860589 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.159+0000 7f53f6d9d640 1 -- 172.21.15.5:0/1171860589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 msgr2=0x558af6bd8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.159+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/1171860589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bd8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.159+0000 7f53f6d9d640 1 -- 172.21.15.5:0/1171860589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bd8e80 msgr2=0x558af6bbae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.159+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/1171860589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bd8e80 0x558af6bbae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.159+0000 7f53f6d9d640 1 -- 172.21.15.5:0/1171860589 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af6bad830 con 0x558af6bb5b60 2024-09-19T06:04:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.159+0000 7f53f759e640 1 --2- 172.21.15.5:0/1171860589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bd8e80 0x558af6bbae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.159+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/1171860589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bb5b60 0x558af6bbf9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f53e801c040 tx=0x7f53e8014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa6b05fb0701e78d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.159+0000 7f53ecff9640 1 -- 172.21.15.5:0/1171860589 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8031070 con 0x558af6bb5b60 2024-09-19T06:04:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.159+0000 7f53ecff9640 1 -- 172.21.15.5:0/1171860589 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e8009680 con 0x558af6bb5b60 2024-09-19T06:04:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.159+0000 7f53ecff9640 1 -- 172.21.15.5:0/1171860589 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8004660 con 0x558af6bb5b60 2024-09-19T06:04:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.160+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1171860589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bb5b60 msgr2=0x558af6bbf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.160+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/1171860589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bb5b60 0x558af6bbf9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f53e801c040 tx=0x7f53e8014040 comp rx=0 tx=0).stop 2024-09-19T06:04:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.161+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1171860589 shutdown_connections 2024-09-19T06:04:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.161+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/1171860589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bd8e80 0x558af6bbae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.161+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/1171860589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bd8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.161+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/1171860589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bb5b60 0x558af6bbf9f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.161+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1171860589 >> 172.21.15.5:0/1171860589 conn(0x558af6bb20f0 msgr2=0x558af6bbf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.161+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1171860589 shutdown_connections 2024-09-19T06:04:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.161+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1171860589 wait complete. 2024-09-19T06:04:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.162+0000 7f53ff4f5740 1 -- start start 2024-09-19T06:04:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.162+0000 7f53ff4f5740 1 Processor -- start 2024-09-19T06:04:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.163+0000 7f53ff4f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 0x558af6bbab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.163+0000 7f53ff4f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bbb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.163+0000 7f53ff4f5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bd8e80 0x558af6bbb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.163+0000 7f53f6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 0x558af6bbab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.163+0000 7f53f6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 0x558af6bbab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41174/0 (socket says 172.21.15.5:41174) 2024-09-19T06:04:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.163+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1304269613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53d800f940 con 0x558af6bb5b60 2024-09-19T06:04:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.163+0000 7f53f6d9d640 1 -- 172.21.15.5:0/1304269613 learned_addr learned my addr 172.21.15.5:0/1304269613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.163+0000 7f53effff640 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bbb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.163+0000 7f53f759e640 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bd8e80 0x558af6bbb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.163+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1304269613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53d800f270 con 0x558af6bc0d40 2024-09-19T06:04:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.163+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1304269613 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53d800c560 con 0x558af6bd8e80 2024-09-19T06:04:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.164+0000 7f53ff4f5740 10 client.0 _ll_register_callbacks cb 0x7f53e00edaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.164+0000 7f53effff640 1 -- 172.21.15.5:0/1304269613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bd8e80 msgr2=0x558af6bbb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.164+0000 7f53effff640 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bd8e80 0x558af6bbb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.164+0000 7f53effff640 1 -- 172.21.15.5:0/1304269613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 msgr2=0x558af6bbab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.164+0000 7f53effff640 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 0x558af6bbab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.164+0000 7f53effff640 1 -- 172.21.15.5:0/1304269613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af6c3a910 con 0x558af6bc0d40 2024-09-19T06:04:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.164+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 0x558af6bbab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.164+0000 7f53effff640 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bbb070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x558af6bbec80 tx=0x7f53e003bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1a1219e2ccb4b3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.164+0000 7f53de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:04:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.164+0000 7f53de7fc640 1 -- 172.21.15.5:0/1304269613 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00bbf50 con 0x558af6bc0d40 2024-09-19T06:04:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.164+0000 7f53de7fc640 1 -- 172.21.15.5:0/1304269613 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f53e00edf90 con 0x558af6bc0d40 2024-09-19T06:04:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.164+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1304269613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e8009680 con 0x558af6bc0d40 2024-09-19T06:04:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.165+0000 7f53de7fc640 1 -- 172.21.15.5:0/1304269613 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f53e00bc3c0 con 0x558af6bc0d40 2024-09-19T06:04:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.165+0000 7f53ff4f5740 10 client.25241 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.165+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1304269613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558af6c3baf0 con 0x558af6bc0d40 2024-09-19T06:04:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.165+0000 7f53de7fc640 1 -- 172.21.15.5:0/1304269613 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e8009680 con 0x558af6bc0d40 2024-09-19T06:04:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.165+0000 7f53de7fc640 1 -- 172.21.15.5:0/1304269613 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f53e0087260 con 0x558af6bc0d40 2024-09-19T06:04:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.165+0000 7f53ff4f5740 10 client.25241 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.165+0000 7f53ff4f5740 10 client.25241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.165+0000 7f53ff4f5740 20 client.25241 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.165+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558af6c3c0b0 0x558af6c3e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.166+0000 7f53ff4f5740 4 client.25241 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:04:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.166+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1304269613 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558af6b22d00 con 0x558af6c3c0b0 2024-09-19T06:04:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.166+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558af6c3c0b0 0x558af6c3e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.166+0000 7f53f6d9d640 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558af6c3c0b0 0x558af6c3e4d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1431c2cfcd13167e server_cookie=d9c7b579a59b5325 in_seq=0 out_seq=0 2024-09-19T06:04:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.166+0000 7f53de7fc640 10 client.25241 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.167+0000 7f53de7fc640 1 -- 172.21.15.5:0/1304269613 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558af6b22d00 con 0x558af6c3c0b0 2024-09-19T06:04:16.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.167+0000 7f53de7fc640 10 client.25241 handle_command_reply: tid=0 2024-09-19T06:04:16.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.167+0000 7f53ff4f5740 1 client.25241 shutdown 2024-09-19T06:04:16.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.167+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1304269613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 msgr2=0x558af6bbb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:16.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.168+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bbb070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x558af6bbec80 tx=0x7f53e003bf00 comp rx=0 tx=0).stop 2024-09-19T06:04:16.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.168+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1304269613 shutdown_connections 2024-09-19T06:04:16.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.168+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558af6c3c0b0 0x558af6c3e4d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.168+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558af6bd8e80 0x558af6bbb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.168+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af6bc0d40 0x558af6bbb070 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.168+0000 7f53ff4f5740 1 --2- 172.21.15.5:0/1304269613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af6bb5b60 0x558af6bbab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.168+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1304269613 >> 172.21.15.5:0/1304269613 conn(0x558af6bb20f0 msgr2=0x558af6bb7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:16.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.168+0000 7f53de7fc640 0 client.25241 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:16.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.168+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1304269613 shutdown_connections 2024-09-19T06:04:16.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.168+0000 7f53ff4f5740 1 -- 172.21.15.5:0/1304269613 wait complete. 2024-09-19T06:04:16.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.168+0000 7f53ff4f5740 20 client.25241 trim_cache size 0 max 16384 2024-09-19T06:04:16.173 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:04:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:04:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "95acf04e-a2f2-43e6-a872-ac0b569aa536", 2024-09-19T06:04:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:04:16.173 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:04:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.169+0000 7f53fd11c640 1 -- 172.21.15.5:0/2265528384 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53e40a3550 msgr2=0x7f53e40a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.169+0000 7f53fd11c640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53e40a3550 0x7f53e40a5a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f53f0005fd0 tx=0x7f53f0009500 comp rx=0 tx=0).stop 2024-09-19T06:04:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.169+0000 7f53fd11c640 1 -- 172.21.15.5:0/2265528384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53f8102570 msgr2=0x7f53f810ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.169+0000 7f53fd11c640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53f8102570 0x7f53f810ea10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f53e002aa70 tx=0x7f53e0003c10 comp rx=0 tx=0).stop 2024-09-19T06:04:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.169+0000 7f53fd11c640 1 -- 172.21.15.5:0/2265528384 shutdown_connections 2024-09-19T06:04:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.170+0000 7f53fd11c640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53e40a3550 0x7f53e40a5a10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.170+0000 7f53fd11c640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f8102ef0 0x7f53f810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.170+0000 7f53fd11c640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53f8102570 0x7f53f810ea10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.170+0000 7f53fd11c640 1 --2- 172.21.15.5:0/2265528384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8100e20 0x7f53f810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:16.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.170+0000 7f53fd11c640 1 -- 172.21.15.5:0/2265528384 >> 172.21.15.5:0/2265528384 conn(0x7f53f80fc7d0 msgr2=0x7f53f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:16.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.170+0000 7f53fd11c640 1 -- 172.21.15.5:0/2265528384 shutdown_connections 2024-09-19T06:04:16.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:16.170+0000 7f53fd11c640 1 -- 172.21.15.5:0/2265528384 wait complete. 2024-09-19T06:04:16.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:16 smithi157 ceph-mon[35346]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T06:04:16.612 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:16 smithi005 ceph-mon[30955]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T06:04:16.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:16 smithi049 ceph-mon[34640]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T06:04:16.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:04:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:17.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:17 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:04:17.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:17 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:04:17.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:17 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:04:17.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:17 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:04:17.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:17 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:04:17.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:17 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:04:17.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:17 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:04:17.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:17 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:04:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.632+0000 7f132b4ae640 1 -- 172.21.15.5:0/1856266032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13241036e0 msgr2=0x7f1324107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.632+0000 7f132b4ae640 1 --2- 172.21.15.5:0/1856266032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13241036e0 0x7f1324107b90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1314009930 tx=0x7f131402a440 comp rx=0 tx=0).stop 2024-09-19T06:04:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.632+0000 7f132b4ae640 1 -- 172.21.15.5:0/1856266032 shutdown_connections 2024-09-19T06:04:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.632+0000 7f132b4ae640 1 --2- 172.21.15.5:0/1856266032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13241082a0 0x7f132410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.632+0000 7f132b4ae640 1 --2- 172.21.15.5:0/1856266032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13241036e0 0x7f1324107b90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.632+0000 7f132b4ae640 1 --2- 172.21.15.5:0/1856266032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1324102d30 0x7f1324103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.632+0000 7f132b4ae640 1 -- 172.21.15.5:0/1856266032 >> 172.21.15.5:0/1856266032 conn(0x7f13240fc7d0 msgr2=0x7f13240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.632+0000 7f132b4ae640 1 -- 172.21.15.5:0/1856266032 shutdown_connections 2024-09-19T06:04:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.632+0000 7f132b4ae640 1 -- 172.21.15.5:0/1856266032 wait complete. 2024-09-19T06:04:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.633+0000 7f132b4ae640 1 Processor -- start 2024-09-19T06:04:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.633+0000 7f132b4ae640 1 -- start start 2024-09-19T06:04:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.633+0000 7f132b4ae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1324102d30 0x7f1324198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.633+0000 7f132b4ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13241036e0 0x7f1324198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.633+0000 7f132b4ae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13241082a0 0x7f132419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.633+0000 7f132924a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1324102d30 0x7f1324198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.633+0000 7f132924a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1324102d30 0x7f1324198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41188/0 (socket says 172.21.15.5:41188) 2024-09-19T06:04:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.634+0000 7f1328a49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13241036e0 0x7f1324198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.634+0000 7f1328a49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13241036e0 0x7f1324198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56462/0 (socket says 172.21.15.5:56462) 2024-09-19T06:04:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.634+0000 7f132b4ae640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1324111fc0 con 0x7f1324102d30 2024-09-19T06:04:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.634+0000 7f132924a640 1 -- 172.21.15.5:0/1359362825 learned_addr learned my addr 172.21.15.5:0/1359362825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.634+0000 7f132b4ae640 1 -- 172.21.15.5:0/1359362825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1324111e40 con 0x7f13241082a0 2024-09-19T06:04:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.634+0000 7f132b4ae640 1 -- 172.21.15.5:0/1359362825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1324112140 con 0x7f13241036e0 2024-09-19T06:04:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.634+0000 7f1329a4b640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13241082a0 0x7f132419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.634+0000 7f132924a640 1 -- 172.21.15.5:0/1359362825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13241036e0 msgr2=0x7f1324198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.634+0000 7f132924a640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13241036e0 0x7f1324198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.634+0000 7f132924a640 1 -- 172.21.15.5:0/1359362825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13241082a0 msgr2=0x7f132419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.634+0000 7f132924a640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13241082a0 0x7f132419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.634+0000 7f132924a640 1 -- 172.21.15.5:0/1359362825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132419d530 con 0x7f1324102d30 2024-09-19T06:04:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.635+0000 7f132924a640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1324102d30 0x7f1324198580 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f132000b4a0 tx=0x7f132000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.635+0000 7f131a7fc640 1 -- 172.21.15.5:0/1359362825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320014070 con 0x7f1324102d30 2024-09-19T06:04:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.635+0000 7f131a7fc640 1 -- 172.21.15.5:0/1359362825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1320010ac0 con 0x7f1324102d30 2024-09-19T06:04:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.635+0000 7f132b4ae640 1 -- 172.21.15.5:0/1359362825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132419d820 con 0x7f1324102d30 2024-09-19T06:04:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.635+0000 7f131a7fc640 1 -- 172.21.15.5:0/1359362825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320017c30 con 0x7f1324102d30 2024-09-19T06:04:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.636+0000 7f132b4ae640 1 -- 172.21.15.5:0/1359362825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13241008c0 con 0x7f1324102d30 2024-09-19T06:04:17.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f132d887740 1 Processor -- start 2024-09-19T06:04:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f132d887740 1 -- start start 2024-09-19T06:04:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f132d887740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626784204a0 0x562678420880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f132d887740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562678420dc0 0x56267842b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f132d887740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56267842b680 0x56267842da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f132d887740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5626783862e0 con 0x562678420dc0 2024-09-19T06:04:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f132d887740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56267832dce0 con 0x5626784204a0 2024-09-19T06:04:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f132d887740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5626783822a0 con 0x56267842b680 2024-09-19T06:04:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f1328a49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56267842b680 0x56267842da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f1329a4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626784204a0 0x562678420880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f1328a49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56267842b680 0x56267842da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56476/0 (socket says 172.21.15.5:56476) 2024-09-19T06:04:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f1328a49640 1 -- 172.21.15.5:0/2550630637 learned_addr learned my addr 172.21.15.5:0/2550630637 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.639+0000 7f1329a4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626784204a0 0x562678420880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59258/0 (socket says 172.21.15.5:59258) 2024-09-19T06:04:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.640+0000 7f1328a49640 1 -- 172.21.15.5:0/2550630637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626784204a0 msgr2=0x562678420880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.640+0000 7f1328a49640 1 --2- 172.21.15.5:0/2550630637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626784204a0 0x562678420880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.640+0000 7f1328a49640 1 -- 172.21.15.5:0/2550630637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562678420dc0 msgr2=0x56267842b140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:04:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.640+0000 7f1328a49640 1 --2- 172.21.15.5:0/2550630637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562678420dc0 0x56267842b140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.640+0000 7f1328a49640 1 -- 172.21.15.5:0/2550630637 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562678312440 con 0x56267842b680 2024-09-19T06:04:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.640+0000 7f1329a4b640 1 --2- 172.21.15.5:0/2550630637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626784204a0 0x562678420880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.640+0000 7f1328a49640 1 --2- 172.21.15.5:0/2550630637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56267842b680 0x56267842da70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1314009900 tx=0x7f1314004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6034e174dee67c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:17.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.640+0000 7f130f7fe640 1 -- 172.21.15.5:0/2550630637 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314040070 con 0x56267842b680 2024-09-19T06:04:17.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.640+0000 7f130f7fe640 1 -- 172.21.15.5:0/2550630637 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13140044b0 con 0x56267842b680 2024-09-19T06:04:17.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.640+0000 7f130f7fe640 1 -- 172.21.15.5:0/2550630637 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314035040 con 0x56267842b680 2024-09-19T06:04:17.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.641+0000 7f131a7fc640 1 -- 172.21.15.5:0/1359362825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7f1320017070 con 0x7f1324102d30 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.641+0000 7f132d887740 1 -- 172.21.15.5:0/2550630637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56267842b680 msgr2=0x56267842da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.641+0000 7f132d887740 1 --2- 172.21.15.5:0/2550630637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56267842b680 0x56267842da70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1314009900 tx=0x7f1314004200 comp rx=0 tx=0).stop 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.641+0000 7f132d887740 1 -- 172.21.15.5:0/2550630637 shutdown_connections 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.641+0000 7f132d887740 1 --2- 172.21.15.5:0/2550630637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56267842b680 0x56267842da70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.641+0000 7f132d887740 1 --2- 172.21.15.5:0/2550630637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562678420dc0 0x56267842b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.641+0000 7f132d887740 1 --2- 172.21.15.5:0/2550630637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626784204a0 0x562678420880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.641+0000 7f132d887740 1 -- 172.21.15.5:0/2550630637 >> 172.21.15.5:0/2550630637 conn(0x56267841ca30 msgr2=0x56267841ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.642+0000 7f132d887740 1 -- 172.21.15.5:0/2550630637 shutdown_connections 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.642+0000 7f132d887740 1 -- 172.21.15.5:0/2550630637 wait complete. 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.642+0000 7f131a7fc640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f13100a3470 0x7f13100a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.642+0000 7f1328a49640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f13100a3470 0x7f13100a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.642+0000 7f131a7fc640 1 -- 172.21.15.5:0/1359362825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f13200cf430 con 0x7f1324102d30 2024-09-19T06:04:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.642+0000 7f132d887740 1 -- start start 2024-09-19T06:04:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.643+0000 7f132d887740 1 Processor -- start 2024-09-19T06:04:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.643+0000 7f1328a49640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f13100a3470 0x7f13100a5930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f131402aa30 tx=0x7f131402a440 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.643+0000 7f132d887740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626784204a0 0x56267842b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.643+0000 7f132d887740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562678420dc0 0x5626784282d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.643+0000 7f132d887740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 0x562678428870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.643+0000 7f132d887740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56267842e250 con 0x56267842b680 2024-09-19T06:04:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.643+0000 7f132d887740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562678341050 con 0x5626784204a0 2024-09-19T06:04:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.643+0000 7f132d887740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56267842e3d0 con 0x562678420dc0 2024-09-19T06:04:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.643+0000 7f132d887740 10 client.0 _ll_register_callbacks cb 0x562678381160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.644+0000 7f1328a49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562678420dc0 0x5626784282d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:17 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:04:17.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:17 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:04:17.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:17 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:04:17.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:17 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:04:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.644+0000 7f1329a4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 0x562678428870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.644+0000 7f1329a4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 0x562678428870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41198/0 (socket says 172.21.15.5:41198) 2024-09-19T06:04:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.644+0000 7f1329a4b640 1 -- 172.21.15.5:0/2059972180 learned_addr learned my addr 172.21.15.5:0/2059972180 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.644+0000 7f132924a640 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626784204a0 0x56267842b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.644+0000 7f1329a4b640 1 -- 172.21.15.5:0/2059972180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562678420dc0 msgr2=0x5626784282d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.644+0000 7f1329a4b640 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562678420dc0 0x5626784282d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.644+0000 7f1329a4b640 1 -- 172.21.15.5:0/2059972180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626784204a0 msgr2=0x56267842b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.644+0000 7f1329a4b640 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626784204a0 0x56267842b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.644+0000 7f1329a4b640 1 -- 172.21.15.5:0/2059972180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626780f73c0 con 0x56267842b680 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.644+0000 7f1328a49640 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562678420dc0 0x5626784282d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.645+0000 7f1329a4b640 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 0x562678428870 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f131c00ee10 tx=0x7f131c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c476be61902e9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.645+0000 7f130cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.645+0000 7f130cff9640 1 -- 172.21.15.5:0/2059972180 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c01f060 con 0x56267842b680 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.645+0000 7f132d887740 1 -- 172.21.15.5:0/2059972180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13140044b0 con 0x56267842b680 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.645+0000 7f130cff9640 1 -- 172.21.15.5:0/2059972180 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f131c003970 con 0x56267842b680 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.645+0000 7f130cff9640 1 -- 172.21.15.5:0/2059972180 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f131c003b70 con 0x56267842b680 2024-09-19T06:04:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.645+0000 7f132d887740 10 client.15630 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.645+0000 7f132d887740 1 -- 172.21.15.5:0/2059972180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562678442520 con 0x56267842b680 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.646+0000 7f130cff9640 1 -- 172.21.15.5:0/2059972180 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13140044b0 con 0x56267842b680 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.646+0000 7f130cff9640 1 -- 172.21.15.5:0/2059972180 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f131c019070 con 0x56267842b680 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.646+0000 7f132d887740 10 client.15630 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.646+0000 7f132d887740 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.646+0000 7f132d887740 20 client.15630 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.646+0000 7f132d887740 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562678443930 0x562678445d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.646+0000 7f132d887740 4 client.15630 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.646+0000 7f132d887740 1 -- 172.21.15.5:0/2059972180 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562678393850 con 0x562678443930 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.646+0000 7f132924a640 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562678443930 0x562678445d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.647+0000 7f132924a640 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562678443930 0x562678445d20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b2e6d647d9440c server_cookie=a917661fda0aad2 in_seq=0 out_seq=0 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.647+0000 7f130cff9640 10 client.15630 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f130cff9640 1 -- 172.21.15.5:0/2059972180 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562678393850 con 0x562678443930 2024-09-19T06:04:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f130cff9640 10 client.15630 handle_command_reply: tid=0 2024-09-19T06:04:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f132d887740 1 client.15630 shutdown 2024-09-19T06:04:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f132d887740 1 -- 172.21.15.5:0/2059972180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 msgr2=0x562678428870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f132d887740 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 0x562678428870 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f131c00ee10 tx=0x7f131c00c6a0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f132d887740 1 -- 172.21.15.5:0/2059972180 shutdown_connections 2024-09-19T06:04:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f132d887740 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562678443930 0x562678445d20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f132d887740 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 0x562678428870 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f132d887740 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562678420dc0 0x5626784282d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f132d887740 1 --2- 172.21.15.5:0/2059972180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626784204a0 0x56267842b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f130cff9640 0 client.15630 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f132d887740 1 -- 172.21.15.5:0/2059972180 >> 172.21.15.5:0/2059972180 conn(0x56267841ca30 msgr2=0x562678422760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f132d887740 1 -- 172.21.15.5:0/2059972180 shutdown_connections 2024-09-19T06:04:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.648+0000 7f132d887740 1 -- 172.21.15.5:0/2059972180 wait complete. 2024-09-19T06:04:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.649+0000 7f132d887740 20 client.15630 trim_cache size 0 max 16384 2024-09-19T06:04:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.683+0000 7f132d887740 1 Processor -- start 2024-09-19T06:04:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.683+0000 7f132d887740 1 -- start start 2024-09-19T06:04:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.683+0000 7f132d887740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626784204a0 0x56267842a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.683+0000 7f132d887740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 0x56267844b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.683+0000 7f132d887740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56267842b680 0x56267844bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.683+0000 7f132d887740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13241ab6c0 con 0x5626784204a0 2024-09-19T06:04:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.683+0000 7f132d887740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f13241ad200 con 0x562678420dc0 2024-09-19T06:04:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.683+0000 7f132d887740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13241abd90 con 0x56267842b680 2024-09-19T06:04:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.683+0000 7f1328a49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 0x56267844b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.683+0000 7f1328a49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 0x56267844b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59286/0 (socket says 172.21.15.5:59286) 2024-09-19T06:04:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.683+0000 7f1328a49640 1 -- 172.21.15.5:0/1057566399 learned_addr learned my addr 172.21.15.5:0/1057566399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.683+0000 7f132924a640 1 --2- 172.21.15.5:0/1057566399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626784204a0 0x56267842a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.684+0000 7f1328a49640 1 -- 172.21.15.5:0/1057566399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56267842b680 msgr2=0x56267844bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.684+0000 7f1328a49640 1 --2- 172.21.15.5:0/1057566399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56267842b680 0x56267844bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.684+0000 7f1328a49640 1 -- 172.21.15.5:0/1057566399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626784204a0 msgr2=0x56267842a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.684+0000 7f1328a49640 1 --2- 172.21.15.5:0/1057566399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626784204a0 0x56267842a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.684+0000 7f1328a49640 1 -- 172.21.15.5:0/1057566399 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56267841d6c0 con 0x562678420dc0 2024-09-19T06:04:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.684+0000 7f1328a49640 1 --2- 172.21.15.5:0/1057566399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 0x56267844b5f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1314009930 tx=0x7f1314040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3443609895e8ae7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.684+0000 7f130effd640 1 -- 172.21.15.5:0/1057566399 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314040070 con 0x562678420dc0 2024-09-19T06:04:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.685+0000 7f130effd640 1 -- 172.21.15.5:0/1057566399 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1314003e80 con 0x562678420dc0 2024-09-19T06:04:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.685+0000 7f130effd640 1 -- 172.21.15.5:0/1057566399 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314035040 con 0x562678420dc0 2024-09-19T06:04:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.686+0000 7f132d887740 1 -- 172.21.15.5:0/1057566399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 msgr2=0x56267844b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.686+0000 7f132d887740 1 --2- 172.21.15.5:0/1057566399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 0x56267844b5f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1314009930 tx=0x7f1314040000 comp rx=0 tx=0).stop 2024-09-19T06:04:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.686+0000 7f132d887740 1 -- 172.21.15.5:0/1057566399 shutdown_connections 2024-09-19T06:04:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.686+0000 7f132d887740 1 --2- 172.21.15.5:0/1057566399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56267842b680 0x56267844bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.686+0000 7f132d887740 1 --2- 172.21.15.5:0/1057566399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 0x56267844b5f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.686+0000 7f132d887740 1 --2- 172.21.15.5:0/1057566399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626784204a0 0x56267842a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.686+0000 7f132d887740 1 -- 172.21.15.5:0/1057566399 >> 172.21.15.5:0/1057566399 conn(0x56267841ca30 msgr2=0x562678429b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.686+0000 7f132d887740 1 -- 172.21.15.5:0/1057566399 shutdown_connections 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.686+0000 7f132d887740 1 -- 172.21.15.5:0/1057566399 wait complete. 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.686+0000 7f132d887740 1 -- start start 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.686+0000 7f132d887740 1 Processor -- start 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.687+0000 7f132d887740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626784204a0 0x56267843c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.687+0000 7f132d887740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 0x562678439a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.687+0000 7f132d887740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 0x562678439f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.687+0000 7f1328a49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 0x562678439a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.687+0000 7f132924a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626784204a0 0x56267843c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.687+0000 7f1328a49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 0x562678439a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59292/0 (socket says 172.21.15.5:59292) 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.687+0000 7f132924a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626784204a0 0x56267843c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56506/0 (socket says 172.21.15.5:56506) 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f1329a4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 0x562678439f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f1329a4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 0x562678439f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41204/0 (socket says 172.21.15.5:41204) 2024-09-19T06:04:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f1328a49640 1 -- 172.21.15.5:0/3991287705 learned_addr learned my addr 172.21.15.5:0/3991287705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f132d887740 1 -- 172.21.15.5:0/3991287705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13241acb30 con 0x56267842b680 2024-09-19T06:04:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f132d887740 1 -- 172.21.15.5:0/3991287705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f13241ac460 con 0x562678420dc0 2024-09-19T06:04:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f132d887740 1 -- 172.21.15.5:0/3991287705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13241a96a0 con 0x5626784204a0 2024-09-19T06:04:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f132d887740 10 client.0 _ll_register_callbacks cb 0x7f131c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f132924a640 1 -- 172.21.15.5:0/3991287705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 msgr2=0x562678439a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f132924a640 1 --2- 172.21.15.5:0/3991287705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 0x562678439a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f132924a640 1 -- 172.21.15.5:0/3991287705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 msgr2=0x562678439f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f132924a640 1 --2- 172.21.15.5:0/3991287705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 0x562678439f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f132924a640 1 -- 172.21.15.5:0/3991287705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56267843a7c0 con 0x5626784204a0 2024-09-19T06:04:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.688+0000 7f1328a49640 1 --2- 172.21.15.5:0/3991287705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 0x562678439a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.689+0000 7f132924a640 1 --2- 172.21.15.5:0/3991287705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626784204a0 0x56267843c660 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f13200ccb20 tx=0x7f1320097320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31de30459cee3b0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.689+0000 7f130cff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:04:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.689+0000 7f130cff9640 1 -- 172.21.15.5:0/3991287705 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13200b3070 con 0x5626784204a0 2024-09-19T06:04:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.689+0000 7f130cff9640 1 -- 172.21.15.5:0/3991287705 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f1320097d00 con 0x5626784204a0 2024-09-19T06:04:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.689+0000 7f132d887740 1 -- 172.21.15.5:0/3991287705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56267838dd60 con 0x5626784204a0 2024-09-19T06:04:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.689+0000 7f130cff9640 1 -- 172.21.15.5:0/3991287705 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f132001e040 con 0x5626784204a0 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.689+0000 7f132d887740 10 client.25183 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.689+0000 7f132d887740 1 -- 172.21.15.5:0/3991287705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5626784ae660 con 0x5626784204a0 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.690+0000 7f130cff9640 1 -- 172.21.15.5:0/3991287705 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56267838dd60 con 0x5626784204a0 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.690+0000 7f130cff9640 1 -- 172.21.15.5:0/3991287705 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f132007c020 con 0x5626784204a0 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.690+0000 7f132d887740 10 client.25183 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.690+0000 7f132d887740 10 client.25183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.690+0000 7f132d887740 20 client.25183 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.690+0000 7f132d887740 1 --2- 172.21.15.5:0/3991287705 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5626784aedc0 0x5626784b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.690+0000 7f132d887740 4 client.25183 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.690+0000 7f132d887740 1 -- 172.21.15.5:0/3991287705 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56267830a210 con 0x5626784aedc0 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f1328a49640 1 --2- 172.21.15.5:0/3991287705 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5626784aedc0 0x5626784b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f1328a49640 1 --2- 172.21.15.5:0/3991287705 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5626784aedc0 0x5626784b11b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4df0b888360445db server_cookie=93d4a8488d9a2ee in_seq=0 out_seq=0 2024-09-19T06:04:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f130cff9640 10 client.25183 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f130cff9640 1 -- 172.21.15.5:0/3991287705 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56267830a210 con 0x5626784aedc0 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f130cff9640 10 client.25183 handle_command_reply: tid=0 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f132d887740 1 client.25183 shutdown 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f132d887740 1 -- 172.21.15.5:0/3991287705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626784204a0 msgr2=0x56267843c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f132d887740 1 --2- 172.21.15.5:0/3991287705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626784204a0 0x56267843c660 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f13200ccb20 tx=0x7f1320097320 comp rx=0 tx=0).stop 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f132d887740 1 -- 172.21.15.5:0/3991287705 shutdown_connections 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f132d887740 1 --2- 172.21.15.5:0/3991287705 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5626784aedc0 0x5626784b11b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f130cff9640 0 client.25183 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f132d887740 1 --2- 172.21.15.5:0/3991287705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56267842b680 0x562678439f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.691+0000 7f132d887740 1 --2- 172.21.15.5:0/3991287705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562678420dc0 0x562678439a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.692+0000 7f132d887740 1 --2- 172.21.15.5:0/3991287705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626784204a0 0x56267843c660 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.692+0000 7f132d887740 1 -- 172.21.15.5:0/3991287705 >> 172.21.15.5:0/3991287705 conn(0x56267841ca30 msgr2=0x56267842a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.692+0000 7f132d887740 1 -- 172.21.15.5:0/3991287705 shutdown_connections 2024-09-19T06:04:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.692+0000 7f132d887740 1 -- 172.21.15.5:0/3991287705 wait complete. 2024-09-19T06:04:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.692+0000 7f132d887740 20 client.25183 trim_cache size 0 max 16384 2024-09-19T06:04:17.697 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:04:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:04:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:04:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "95acf04e-a2f2-43e6-a872-ac0b569aa536": { 2024-09-19T06:04:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:04:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "95acf04e-a2f2-43e6-a872-ac0b569aa536", 2024-09-19T06:04:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:04:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:04:17.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:04:17.698 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:04:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.693+0000 7f132b4ae640 1 -- 172.21.15.5:0/1359362825 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f13100a3470 msgr2=0x7f13100a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.693+0000 7f132b4ae640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f13100a3470 0x7f13100a5930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f131402aa30 tx=0x7f131402a440 comp rx=0 tx=0).stop 2024-09-19T06:04:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.693+0000 7f132b4ae640 1 -- 172.21.15.5:0/1359362825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1324102d30 msgr2=0x7f1324198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.693+0000 7f132b4ae640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1324102d30 0x7f1324198580 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f132000b4a0 tx=0x7f132000b960 comp rx=0 tx=0).stop 2024-09-19T06:04:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.693+0000 7f132b4ae640 1 -- 172.21.15.5:0/1359362825 shutdown_connections 2024-09-19T06:04:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.693+0000 7f132b4ae640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13241082a0 0x7f132419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.693+0000 7f132b4ae640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f13100a3470 0x7f13100a5930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.693+0000 7f132b4ae640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13241036e0 0x7f1324198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.693+0000 7f132b4ae640 1 --2- 172.21.15.5:0/1359362825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1324102d30 0x7f1324198580 secure :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f132000b4a0 tx=0x7f132000b960 comp rx=0 tx=0).stop 2024-09-19T06:04:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.693+0000 7f132b4ae640 1 -- 172.21.15.5:0/1359362825 >> 172.21.15.5:0/1359362825 conn(0x7f13240fc7d0 msgr2=0x7f1324105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.693+0000 7f132b4ae640 1 -- 172.21.15.5:0/1359362825 shutdown_connections 2024-09-19T06:04:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:17.693+0000 7f132b4ae640 1 -- 172.21.15.5:0/1359362825 wait complete. 2024-09-19T06:04:18.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:18 smithi005 ceph-mon[30955]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T06:04:18.336 INFO:tasks.cephfs.filesystem:scrub status for tag:95acf04e-a2f2-43e6-a872-ac0b569aa536 - {'path': '/', 'tag': '95acf04e-a2f2-43e6-a872-ac0b569aa536', 'options': 'recursive,force'} 2024-09-19T06:04:18.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:18 smithi157 ceph-mon[35346]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T06:04:18.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:18 smithi049 ceph-mon[34640]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T06:04:19.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:19 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:04:19.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:19 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:04:19.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:19 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:04:20.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:20 smithi157 ceph-mon[35346]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 682 B/s wr, 14 op/s 2024-09-19T06:04:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:20 smithi005 ceph-mon[30955]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 682 B/s wr, 14 op/s 2024-09-19T06:04:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:20 smithi049 ceph-mon[34640]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 682 B/s wr, 14 op/s 2024-09-19T06:04:22.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:22 smithi157 ceph-mon[35346]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T06:04:22.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:22 smithi005 ceph-mon[30955]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T06:04:22.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:22 smithi049 ceph-mon[34640]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T06:04:24.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:24 smithi157 ceph-mon[35346]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 597 B/s wr, 14 op/s 2024-09-19T06:04:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.588+0000 7f1e56077640 1 -- 172.21.15.5:0/1730027168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e501069e0 msgr2=0x7f1e50106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.588+0000 7f1e56077640 1 --2- 172.21.15.5:0/1730027168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e501069e0 0x7f1e50106dc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009930 tx=0x7f1e4002a420 comp rx=0 tx=0).stop 2024-09-19T06:04:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.588+0000 7f1e56077640 1 -- 172.21.15.5:0/1730027168 shutdown_connections 2024-09-19T06:04:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.588+0000 7f1e56077640 1 --2- 172.21.15.5:0/1730027168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e50101370 0x7f1e5010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.588+0000 7f1e56077640 1 --2- 172.21.15.5:0/1730027168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e501009d0 0x7f1e50100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.588+0000 7f1e56077640 1 --2- 172.21.15.5:0/1730027168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e501069e0 0x7f1e50106dc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.588+0000 7f1e56077640 1 -- 172.21.15.5:0/1730027168 >> 172.21.15.5:0/1730027168 conn(0x7f1e500fc6b0 msgr2=0x7f1e500fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.588+0000 7f1e56077640 1 -- 172.21.15.5:0/1730027168 shutdown_connections 2024-09-19T06:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.588+0000 7f1e56077640 1 -- 172.21.15.5:0/1730027168 wait complete. 2024-09-19T06:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.589+0000 7f1e56077640 1 Processor -- start 2024-09-19T06:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.589+0000 7f1e56077640 1 -- start start 2024-09-19T06:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.589+0000 7f1e56077640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e501009d0 0x7f1e500fce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.589+0000 7f1e56077640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e50101370 0x7f1e500fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.589+0000 7f1e56077640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e501069e0 0x7f1e500fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.589+0000 7f1e56077640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50110cd0 con 0x7f1e50101370 2024-09-19T06:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.589+0000 7f1e56077640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50110b50 con 0x7f1e501069e0 2024-09-19T06:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.589+0000 7f1e56077640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50110e50 con 0x7f1e501009d0 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.589+0000 7f1e4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e50101370 0x7f1e500fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.589+0000 7f1e4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e50101370 0x7f1e500fd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54098/0 (socket says 172.21.15.5:54098) 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.589+0000 7f1e4effd640 1 -- 172.21.15.5:0/2896356403 learned_addr learned my addr 172.21.15.5:0/2896356403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.589+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e501009d0 0x7f1e500fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e4ffff640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e501069e0 0x7f1e500fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e4effd640 1 -- 172.21.15.5:0/2896356403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e501009d0 msgr2=0x7f1e500fce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e4effd640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e501009d0 0x7f1e500fce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e4effd640 1 -- 172.21.15.5:0/2896356403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e501069e0 msgr2=0x7f1e500fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e4effd640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e501069e0 0x7f1e500fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e4effd640 1 -- 172.21.15.5:0/2896356403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e501a6860 con 0x7f1e50101370 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e501009d0 0x7f1e500fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e4effd640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e50101370 0x7f1e500fd3c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400e9f0 tx=0x7f1e4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e4cff9640 1 -- 172.21.15.5:0/2896356403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400cd80 con 0x7f1e50101370 2024-09-19T06:04:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e4cff9640 1 -- 172.21.15.5:0/2896356403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e440054d0 con 0x7f1e50101370 2024-09-19T06:04:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e56077640 1 -- 172.21.15.5:0/2896356403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e501a6b50 con 0x7f1e50101370 2024-09-19T06:04:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e4cff9640 1 -- 172.21.15.5:0/2896356403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44010490 con 0x7f1e50101370 2024-09-19T06:04:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.590+0000 7f1e56077640 1 -- 172.21.15.5:0/2896356403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e501a7050 con 0x7f1e50101370 2024-09-19T06:04:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.593+0000 7f1e58e11740 1 Processor -- start 2024-09-19T06:04:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.593+0000 7f1e58e11740 1 -- start start 2024-09-19T06:04:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e58e11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 0x55f6120886e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e58e11740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088c20 0x55f612092f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e58e11740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6120934c0 0x55f6120958b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e58e11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f611fe42f0 con 0x55f612088300 2024-09-19T06:04:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e58e11740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f611f8c270 con 0x55f6120934c0 2024-09-19T06:04:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e58e11740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f611fe02b0 con 0x55f612088c20 2024-09-19T06:04:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 0x55f6120886e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 0x55f6120886e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54108/0 (socket says 172.21.15.5:54108) 2024-09-19T06:04:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e4ffff640 1 -- 172.21.15.5:0/3202110363 learned_addr learned my addr 172.21.15.5:0/3202110363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/3202110363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088c20 0x55f612092f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e4ffff640 1 -- 172.21.15.5:0/3202110363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088c20 msgr2=0x55f612092f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e4ffff640 1 --2- 172.21.15.5:0/3202110363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088c20 0x55f612092f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e4ffff640 1 -- 172.21.15.5:0/3202110363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6120934c0 msgr2=0x55f6120958b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:04:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e4ffff640 1 --2- 172.21.15.5:0/3202110363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6120934c0 0x55f6120958b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e4ffff640 1 -- 172.21.15.5:0/3202110363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f611f8b340 con 0x55f612088300 2024-09-19T06:04:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/3202110363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088c20 0x55f612092f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e4cff9640 1 -- 172.21.15.5:0/2896356403 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7f1e44010690 con 0x7f1e50101370 2024-09-19T06:04:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.594+0000 7f1e4ffff640 1 --2- 172.21.15.5:0/3202110363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 0x55f6120886e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009f50 tx=0x7f1e480118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1ef0b1401fdb1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.595+0000 7f1e4cff9640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1e380a3550 0x7f1e380a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.595+0000 7f1e35ffb640 1 -- 172.21.15.5:0/3202110363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48004d20 con 0x55f612088300 2024-09-19T06:04:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.595+0000 7f1e35ffb640 1 -- 172.21.15.5:0/3202110363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e48004050 con 0x55f612088300 2024-09-19T06:04:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.595+0000 7f1e35ffb640 1 -- 172.21.15.5:0/3202110363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48005280 con 0x55f612088300 2024-09-19T06:04:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.595+0000 7f1e4cff9640 1 -- 172.21.15.5:0/2896356403 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f1e440d0380 con 0x7f1e50101370 2024-09-19T06:04:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.596+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1e380a3550 0x7f1e380a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.596+0000 7f1e58e11740 1 -- 172.21.15.5:0/3202110363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 msgr2=0x55f6120886e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.596+0000 7f1e58e11740 1 --2- 172.21.15.5:0/3202110363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 0x55f6120886e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009f50 tx=0x7f1e480118d0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.596+0000 7f1e58e11740 1 -- 172.21.15.5:0/3202110363 shutdown_connections 2024-09-19T06:04:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.596+0000 7f1e58e11740 1 --2- 172.21.15.5:0/3202110363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6120934c0 0x55f6120958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.596+0000 7f1e58e11740 1 --2- 172.21.15.5:0/3202110363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088c20 0x55f612092f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.596+0000 7f1e58e11740 1 --2- 172.21.15.5:0/3202110363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 0x55f6120886e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.596+0000 7f1e58e11740 1 -- 172.21.15.5:0/3202110363 >> 172.21.15.5:0/3202110363 conn(0x55f612084870 msgr2=0x55f612084c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.596+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1e380a3550 0x7f1e380a5a10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1e40000c00 tx=0x7f1e400058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.596+0000 7f1e58e11740 1 -- 172.21.15.5:0/3202110363 shutdown_connections 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.596+0000 7f1e58e11740 1 -- 172.21.15.5:0/3202110363 wait complete. 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.597+0000 7f1e58e11740 1 -- start start 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.597+0000 7f1e58e11740 1 Processor -- start 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.597+0000 7f1e58e11740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f612088300 0x55f612090460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.597+0000 7f1e58e11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088c20 0x55f6120909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.597+0000 7f1e58e11740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6120934c0 0x55f612090ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e58e11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f612096090 con 0x55f612088c20 2024-09-19T06:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e58e11740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f611f6d290 con 0x55f612088300 2024-09-19T06:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e58e11740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f612096210 con 0x55f6120934c0 2024-09-19T06:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088c20 0x55f6120909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088c20 0x55f6120909a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54124/0 (socket says 172.21.15.5:54124) 2024-09-19T06:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e4effd640 1 -- 172.21.15.5:0/3507523038 learned_addr learned my addr 172.21.15.5:0/3507523038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e58e11740 10 client.0 _ll_register_callbacks cb 0x55f611f9e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/3507523038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f612088300 0x55f612090460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e4effd640 1 -- 172.21.15.5:0/3507523038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6120934c0 msgr2=0x55f612090ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e4effd640 1 --2- 172.21.15.5:0/3507523038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6120934c0 0x55f612090ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e4effd640 1 -- 172.21.15.5:0/3507523038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f612088300 msgr2=0x55f612090460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e4effd640 1 --2- 172.21.15.5:0/3507523038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f612088300 0x55f612090460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e4effd640 1 -- 172.21.15.5:0/3507523038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f611cf7210 con 0x55f612088c20 2024-09-19T06:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e4effd640 1 --2- 172.21.15.5:0/3507523038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088c20 0x55f6120909a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1e440cda70 tx=0x7f1e4409be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d5982a3a9d7ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e2effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e2effd640 1 -- 172.21.15.5:0/3507523038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e440b4020 con 0x55f612088c20 2024-09-19T06:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.598+0000 7f1e2effd640 1 -- 172.21.15.5:0/3507523038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f1e440a4c00 con 0x55f612088c20 2024-09-19T06:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.599+0000 7f1e58e11740 1 -- 172.21.15.5:0/3507523038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e48004050 con 0x55f612088c20 2024-09-19T06:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.599+0000 7f1e2effd640 1 -- 172.21.15.5:0/3507523038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e440ba050 con 0x55f612088c20 2024-09-19T06:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.599+0000 7f1e58e11740 10 client.15648 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.599+0000 7f1e58e11740 1 -- 172.21.15.5:0/3507523038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f611dc3620 con 0x55f612088c20 2024-09-19T06:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.599+0000 7f1e2effd640 1 -- 172.21.15.5:0/3507523038 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e48004050 con 0x55f612088c20 2024-09-19T06:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.599+0000 7f1e2effd640 1 -- 172.21.15.5:0/3507523038 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1e440cdc40 con 0x55f612088c20 2024-09-19T06:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.599+0000 7f1e58e11740 10 client.15648 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.599+0000 7f1e58e11740 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.599+0000 7f1e58e11740 20 client.15648 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.600+0000 7f1e58e11740 1 --2- 172.21.15.5:0/3507523038 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f6120a3500 0x55f6120a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.600+0000 7f1e58e11740 4 client.15648 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.600+0000 7f1e58e11740 1 -- 172.21.15.5:0/3507523038 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f611ff1a20 con 0x55f6120a3500 2024-09-19T06:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.600+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/3507523038 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f6120a3500 0x55f6120a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.600+0000 7f1e2effd640 10 client.15648 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.600+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/3507523038 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f6120a3500 0x55f6120a58f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fd0bde30382b08d server_cookie=ef13c0b437559766 in_seq=0 out_seq=0 2024-09-19T06:04:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.601+0000 7f1e2effd640 1 -- 172.21.15.5:0/3507523038 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f611ff1a20 con 0x55f6120a3500 2024-09-19T06:04:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.601+0000 7f1e2effd640 10 client.15648 handle_command_reply: tid=0 2024-09-19T06:04:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.601+0000 7f1e58e11740 1 client.15648 shutdown 2024-09-19T06:04:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.601+0000 7f1e58e11740 1 -- 172.21.15.5:0/3507523038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088c20 msgr2=0x55f6120909a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.601+0000 7f1e58e11740 1 --2- 172.21.15.5:0/3507523038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088c20 0x55f6120909a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1e440cda70 tx=0x7f1e4409be40 comp rx=0 tx=0).stop 2024-09-19T06:04:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.601+0000 7f1e58e11740 1 -- 172.21.15.5:0/3507523038 shutdown_connections 2024-09-19T06:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.601+0000 7f1e58e11740 1 --2- 172.21.15.5:0/3507523038 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f6120a3500 0x55f6120a58f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.601+0000 7f1e2effd640 0 client.15648 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.601+0000 7f1e58e11740 1 --2- 172.21.15.5:0/3507523038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6120934c0 0x55f612090ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.601+0000 7f1e58e11740 1 --2- 172.21.15.5:0/3507523038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088c20 0x55f6120909a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.601+0000 7f1e58e11740 1 --2- 172.21.15.5:0/3507523038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f612088300 0x55f612090460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.601+0000 7f1e58e11740 1 -- 172.21.15.5:0/3507523038 >> 172.21.15.5:0/3507523038 conn(0x55f612084870 msgr2=0x55f61208a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.602+0000 7f1e58e11740 1 -- 172.21.15.5:0/3507523038 shutdown_connections 2024-09-19T06:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.602+0000 7f1e58e11740 1 -- 172.21.15.5:0/3507523038 wait complete. 2024-09-19T06:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.602+0000 7f1e58e11740 20 client.15648 trim_cache size 0 max 16384 2024-09-19T06:04:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.616+0000 7f1e58e11740 1 Processor -- start 2024-09-19T06:04:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.616+0000 7f1e58e11740 1 -- start start 2024-09-19T06:04:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.616+0000 7f1e58e11740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088300 0x55f612090b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.616+0000 7f1e58e11740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f612088c20 0x55f6120ab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.616+0000 7f1e58e11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6120934c0 0x55f6120ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e58e11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501ab580 con 0x55f6120934c0 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e58e11740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501ad0c0 con 0x55f612088c20 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e58e11740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501abc50 con 0x55f612088300 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6120934c0 0x55f6120ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6120934c0 0x55f6120ab6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54132/0 (socket says 172.21.15.5:54132) 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e4ffff640 1 -- 172.21.15.5:0/1109169582 learned_addr learned my addr 172.21.15.5:0/1109169582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e4effd640 1 --2- 172.21.15.5:0/1109169582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f612088c20 0x55f6120ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/1109169582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088300 0x55f612090b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e4ffff640 1 -- 172.21.15.5:0/1109169582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088300 msgr2=0x55f612090b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e4ffff640 1 --2- 172.21.15.5:0/1109169582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088300 0x55f612090b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e4ffff640 1 -- 172.21.15.5:0/1109169582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f612088c20 msgr2=0x55f6120ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e4ffff640 1 --2- 172.21.15.5:0/1109169582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f612088c20 0x55f6120ab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e4ffff640 1 -- 172.21.15.5:0/1109169582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6120762f0 con 0x55f6120934c0 2024-09-19T06:04:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e4effd640 1 --2- 172.21.15.5:0/1109169582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f612088c20 0x55f6120ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e4ffff640 1 --2- 172.21.15.5:0/1109169582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6120934c0 0x55f6120ab6d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1e48012fd0 tx=0x7f1e48003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d9139a71cb7603 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.617+0000 7f1e357fa640 1 -- 172.21.15.5:0/1109169582 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48028070 con 0x55f6120934c0 2024-09-19T06:04:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.618+0000 7f1e357fa640 1 -- 172.21.15.5:0/1109169582 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e480054f0 con 0x55f6120934c0 2024-09-19T06:04:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.618+0000 7f1e357fa640 1 -- 172.21.15.5:0/1109169582 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e480249b0 con 0x55f6120934c0 2024-09-19T06:04:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.618+0000 7f1e58e11740 1 -- 172.21.15.5:0/1109169582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6120934c0 msgr2=0x55f6120ab6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.618+0000 7f1e58e11740 1 --2- 172.21.15.5:0/1109169582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6120934c0 0x55f6120ab6d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1e48012fd0 tx=0x7f1e48003bc0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.618+0000 7f1e58e11740 1 -- 172.21.15.5:0/1109169582 shutdown_connections 2024-09-19T06:04:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.618+0000 7f1e58e11740 1 --2- 172.21.15.5:0/1109169582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6120934c0 0x55f6120ab6d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.618+0000 7f1e58e11740 1 --2- 172.21.15.5:0/1109169582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f612088c20 0x55f6120ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.618+0000 7f1e58e11740 1 --2- 172.21.15.5:0/1109169582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088300 0x55f612090b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.618+0000 7f1e58e11740 1 -- 172.21.15.5:0/1109169582 >> 172.21.15.5:0/1109169582 conn(0x55f612084870 msgr2=0x55f612090420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.619+0000 7f1e58e11740 1 -- 172.21.15.5:0/1109169582 shutdown_connections 2024-09-19T06:04:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.619+0000 7f1e58e11740 1 -- 172.21.15.5:0/1109169582 wait complete. 2024-09-19T06:04:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.619+0000 7f1e58e11740 1 -- start start 2024-09-19T06:04:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.619+0000 7f1e58e11740 1 Processor -- start 2024-09-19T06:04:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.620+0000 7f1e58e11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 0x55f61209bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.620+0000 7f1e58e11740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088c20 0x55f612096fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.620+0000 7f1e58e11740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6120934c0 0x55f612097530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.620+0000 7f1e4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 0x55f61209bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.620+0000 7f1e4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 0x55f61209bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54142/0 (socket says 172.21.15.5:54142) 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.620+0000 7f1e4f7fe640 1 -- 172.21.15.5:0/2285742484 learned_addr learned my addr 172.21.15.5:0/2285742484 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.620+0000 7f1e58e11740 1 -- 172.21.15.5:0/2285742484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501ac9f0 con 0x55f612088300 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.620+0000 7f1e4effd640 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088c20 0x55f612096fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.620+0000 7f1e58e11740 1 -- 172.21.15.5:0/2285742484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501ac320 con 0x55f6120934c0 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.620+0000 7f1e58e11740 1 -- 172.21.15.5:0/2285742484 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501a9560 con 0x55f612088c20 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.620+0000 7f1e4ffff640 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6120934c0 0x55f612097530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e4f7fe640 1 -- 172.21.15.5:0/2285742484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088c20 msgr2=0x55f612096fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088c20 0x55f612096fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e4f7fe640 1 -- 172.21.15.5:0/2285742484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6120934c0 msgr2=0x55f612097530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6120934c0 0x55f612097530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e58e11740 10 client.0 _ll_register_callbacks cb 0x7f1e4400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e4f7fe640 1 -- 172.21.15.5:0/2285742484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f612097d60 con 0x55f612088300 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e58e11740 1 -- 172.21.15.5:0/2285742484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e480054f0 con 0x55f612088300 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e4f7fe640 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 0x55f61209bc90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1e400096c0 tx=0x7f1e40009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c5e03464ef0727 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e2effd640 10 client.15654 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e2effd640 1 -- 172.21.15.5:0/2285742484 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40035050 con 0x55f612088300 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e2effd640 1 -- 172.21.15.5:0/2285742484 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f1e40005250 con 0x55f612088300 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e2effd640 1 -- 172.21.15.5:0/2285742484 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e40005470 con 0x55f612088300 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e58e11740 10 client.15654 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.621+0000 7f1e58e11740 1 -- 172.21.15.5:0/2285742484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55f61210e160 con 0x55f612088300 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.622+0000 7f1e2effd640 1 -- 172.21.15.5:0/2285742484 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1e400046c0 con 0x55f612088300 2024-09-19T06:04:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.622+0000 7f1e58e11740 10 client.15654 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.622+0000 7f1e58e11740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.622+0000 7f1e58e11740 20 client.15654 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.622+0000 7f1e58e11740 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f61210e7d0 0x55f612110c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.622+0000 7f1e58e11740 4 client.15654 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:04:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.622+0000 7f1e58e11740 1 -- 172.21.15.5:0/2285742484 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f611febd70 con 0x55f61210e7d0 2024-09-19T06:04:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.622+0000 7f1e4effd640 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f61210e7d0 0x55f612110c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.623+0000 7f1e2effd640 10 client.15654 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.623+0000 7f1e4effd640 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f61210e7d0 0x55f612110c20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=419f6e270d75162f server_cookie=b67b60d9e23c5563 in_seq=0 out_seq=0 2024-09-19T06:04:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.626+0000 7f1e2effd640 1 -- 172.21.15.5:0/2285742484 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f611febd70 con 0x55f61210e7d0 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.626+0000 7f1e2effd640 10 client.15654 handle_command_reply: tid=0 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.626+0000 7f1e58e11740 1 client.15654 shutdown 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.626+0000 7f1e58e11740 1 -- 172.21.15.5:0/2285742484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 msgr2=0x55f61209bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.626+0000 7f1e58e11740 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 0x55f61209bc90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1e400096c0 tx=0x7f1e40009260 comp rx=0 tx=0).stop 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.626+0000 7f1e58e11740 1 -- 172.21.15.5:0/2285742484 shutdown_connections 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.626+0000 7f1e58e11740 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f61210e7d0 0x55f612110c20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.627+0000 7f1e2effd640 0 client.15654 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.627+0000 7f1e58e11740 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6120934c0 0x55f612097530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.627+0000 7f1e58e11740 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f612088c20 0x55f612096fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.627+0000 7f1e58e11740 1 --2- 172.21.15.5:0/2285742484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f612088300 0x55f61209bc90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.627+0000 7f1e58e11740 1 -- 172.21.15.5:0/2285742484 >> 172.21.15.5:0/2285742484 conn(0x55f612084870 msgr2=0x55f61208a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.627+0000 7f1e58e11740 1 -- 172.21.15.5:0/2285742484 shutdown_connections 2024-09-19T06:04:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.627+0000 7f1e58e11740 1 -- 172.21.15.5:0/2285742484 wait complete. 2024-09-19T06:04:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.627+0000 7f1e58e11740 20 client.15654 trim_cache size 0 max 16384 2024-09-19T06:04:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:04:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:04:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:04:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:04:24.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:24 smithi005 ceph-mon[30955]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 597 B/s wr, 14 op/s 2024-09-19T06:04:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.627+0000 7f1e56077640 1 -- 172.21.15.5:0/2896356403 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1e380a3550 msgr2=0x7f1e380a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.628+0000 7f1e56077640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1e380a3550 0x7f1e380a5a10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1e40000c00 tx=0x7f1e400058d0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.628+0000 7f1e56077640 1 -- 172.21.15.5:0/2896356403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e50101370 msgr2=0x7f1e500fd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.628+0000 7f1e56077640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e50101370 0x7f1e500fd3c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400e9f0 tx=0x7f1e4400edf0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.628+0000 7f1e56077640 1 -- 172.21.15.5:0/2896356403 shutdown_connections 2024-09-19T06:04:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.628+0000 7f1e56077640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e501069e0 0x7f1e500fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.628+0000 7f1e56077640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e50101370 0x7f1e500fd3c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.628+0000 7f1e56077640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1e380a3550 0x7f1e380a5a10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.628+0000 7f1e56077640 1 --2- 172.21.15.5:0/2896356403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e501009d0 0x7f1e500fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.628+0000 7f1e56077640 1 -- 172.21.15.5:0/2896356403 >> 172.21.15.5:0/2896356403 conn(0x7f1e500fc6b0 msgr2=0x7f1e5006a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.628+0000 7f1e56077640 1 -- 172.21.15.5:0/2896356403 shutdown_connections 2024-09-19T06:04:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:24.628+0000 7f1e56077640 1 -- 172.21.15.5:0/2896356403 wait complete. 2024-09-19T06:04:24.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:24 smithi049 ceph-mon[34640]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 597 B/s wr, 14 op/s 2024-09-19T06:04:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:26 smithi157 ceph-mon[35346]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-19T06:04:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:26 smithi005 ceph-mon[30955]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-19T06:04:26.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:26 smithi049 ceph-mon[34640]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-19T06:04:28.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:28 smithi157 ceph-mon[35346]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-19T06:04:28.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:28 smithi005 ceph-mon[30955]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-19T06:04:28.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:28 smithi049 ceph-mon[34640]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-19T06:04:30.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:30 smithi157 ceph-mon[35346]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T06:04:30.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:30 smithi005 ceph-mon[30955]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T06:04:30.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:30 smithi049 ceph-mon[34640]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T06:04:31.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:31 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:04:31.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:31 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:04:31.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:31 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:04:31.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:31 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:04:31.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:31 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:04:31.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:31 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:04:31.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:31 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:04:31.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:31 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:04:31.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:31 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:04:31.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:31 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:04:31.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:31 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:04:31.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:31 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:04:32.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:32 smithi157 ceph-mon[35346]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:04:32.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:32 smithi005 ceph-mon[30955]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:04:32.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:32 smithi049 ceph-mon[34640]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:04:32.751 INFO:tasks.workunit.client.0.smithi005.stderr:Updating files: 94% (12224/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T06:04:33.651 INFO:tasks.workunit.client.0.smithi005.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T06:04:33.651 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-19T06:04:33.651 INFO:tasks.workunit.client.0.smithi005.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T06:04:33.651 INFO:tasks.workunit.client.0.smithi005.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T06:04:33.651 INFO:tasks.workunit.client.0.smithi005.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr: git switch -c 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr:Or undo this operation with: 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr: git switch - 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-19T06:04:33.652 INFO:tasks.workunit.client.0.smithi005.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T06:04:33.657 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T06:04:33.687 INFO:tasks.workunit.client.0.smithi005.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T06:04:33.689 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T06:04:33.689 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T06:04:34.439 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T06:04:34.523 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T06:04:34.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:34 smithi157 ceph-mon[35346]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T06:04:34.566 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T06:04:34.567 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T06:04:34.568 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T06:04:34.608 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T06:04:34.611 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T06:04:34.612 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T06:04:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:34 smithi005 ceph-mon[30955]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T06:04:34.645 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-19T06:04:34.646 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-19T06:04:34.647 DEBUG:teuthology.orchestra.run.smithi005:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-19T06:04:34.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:34 smithi049 ceph-mon[34640]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T06:04:34.711 INFO:tasks.workunit.client.0.smithi005.stderr:+ dbench 1 2024-09-19T06:04:34.716 INFO:tasks.workunit.client.0.smithi005.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-19T06:04:34.716 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-19T06:04:34.716 INFO:tasks.workunit.client.0.smithi005.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-19T06:04:34.721 INFO:tasks.workunit.client.0.smithi005.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-19T06:04:34.821 INFO:tasks.workunit.client.0.smithi005.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-19T06:04:34.821 INFO:tasks.workunit.client.0.smithi005.stdout:releasing clients 2024-09-19T06:04:35.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 942 21.78 MB/sec warmup 1 sec latency 14.643 ms 2024-09-19T06:04:36.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:36 smithi157 ceph-mon[35346]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:04:36.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:36 smithi005 ceph-mon[30955]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:04:36.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:36 smithi049 ceph-mon[34640]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:04:36.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1707 13.11 MB/sec warmup 2 sec latency 16.984 ms 2024-09-19T06:04:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.726+0000 7fb6ea791640 1 -- 172.21.15.5:0/514122951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e41069e0 msgr2=0x7fb6e4106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.726+0000 7fb6ea791640 1 --2- 172.21.15.5:0/514122951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e41069e0 0x7fb6e4106dc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009930 tx=0x7fb6d402a420 comp rx=0 tx=0).stop 2024-09-19T06:04:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.726+0000 7fb6ea791640 1 -- 172.21.15.5:0/514122951 shutdown_connections 2024-09-19T06:04:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.726+0000 7fb6ea791640 1 --2- 172.21.15.5:0/514122951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e4101370 0x7fb6e410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.726+0000 7fb6ea791640 1 --2- 172.21.15.5:0/514122951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e41009d0 0x7fb6e4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.726+0000 7fb6ea791640 1 --2- 172.21.15.5:0/514122951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e41069e0 0x7fb6e4106dc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.726+0000 7fb6ea791640 1 -- 172.21.15.5:0/514122951 >> 172.21.15.5:0/514122951 conn(0x7fb6e40fc6b0 msgr2=0x7fb6e40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.726+0000 7fb6ea791640 1 -- 172.21.15.5:0/514122951 shutdown_connections 2024-09-19T06:04:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.726+0000 7fb6ea791640 1 -- 172.21.15.5:0/514122951 wait complete. 2024-09-19T06:04:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.727+0000 7fb6ea791640 1 Processor -- start 2024-09-19T06:04:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.727+0000 7fb6ea791640 1 -- start start 2024-09-19T06:04:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.727+0000 7fb6ea791640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e41009d0 0x7fb6e4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.727+0000 7fb6ea791640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e4101370 0x7fb6e41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.727+0000 7fb6ea791640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e41069e0 0x7fb6e419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.727+0000 7fb6ea791640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4110cd0 con 0x7fb6e41009d0 2024-09-19T06:04:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.727+0000 7fb6ea791640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4110b50 con 0x7fb6e41069e0 2024-09-19T06:04:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.727+0000 7fb6ea791640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4110e50 con 0x7fb6e4101370 2024-09-19T06:04:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.727+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e41009d0 0x7fb6e4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e41009d0 0x7fb6e4198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44082/0 (socket says 172.21.15.5:44082) 2024-09-19T06:04:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6e3fff640 1 -- 172.21.15.5:0/2938805511 learned_addr learned my addr 172.21.15.5:0/2938805511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6e8d2e640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e41069e0 0x7fb6e419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e4101370 0x7fb6e41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6e3fff640 1 -- 172.21.15.5:0/2938805511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e4101370 msgr2=0x7fb6e41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6e3fff640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e4101370 0x7fb6e41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6e3fff640 1 -- 172.21.15.5:0/2938805511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e41069e0 msgr2=0x7fb6e419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6e3fff640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e41069e0 0x7fb6e419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6e3fff640 1 -- 172.21.15.5:0/2938805511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e419d350 con 0x7fb6e41009d0 2024-09-19T06:04:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6e3fff640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e41009d0 0x7fb6e4198360 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4005940 tx=0x7fb6d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6e1ffb640 1 -- 172.21.15.5:0/2938805511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d403f030 con 0x7fb6e41009d0 2024-09-19T06:04:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6e1ffb640 1 -- 172.21.15.5:0/2938805511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6d40318a0 con 0x7fb6e41009d0 2024-09-19T06:04:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6ea791640 1 -- 172.21.15.5:0/2938805511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e419d5e0 con 0x7fb6e41009d0 2024-09-19T06:04:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6e1ffb640 1 -- 172.21.15.5:0/2938805511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4035070 con 0x7fb6e41009d0 2024-09-19T06:04:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.728+0000 7fb6ea791640 1 -- 172.21.15.5:0/2938805511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e41a4ed0 con 0x7fb6e41009d0 2024-09-19T06:04:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.731+0000 7fb6e1ffb640 1 -- 172.21.15.5:0/2938805511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7fb6d4042030 con 0x7fb6e41009d0 2024-09-19T06:04:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.731+0000 7fb6ed609740 1 Processor -- start 2024-09-19T06:04:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.731+0000 7fb6ed609740 1 -- start start 2024-09-19T06:04:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.731+0000 7fb6ed609740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 0x55d9df2cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.731+0000 7fb6ed609740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.731+0000 7fb6ed609740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.731+0000 7fb6ed609740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d9df2342f0 con 0x55d9df2cf9d0 2024-09-19T06:04:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.731+0000 7fb6ed609740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d9df1dc270 con 0x55d9df2df0c0 2024-09-19T06:04:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.731+0000 7fb6ed609740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d9df2302b0 con 0x55d9df2d02f0 2024-09-19T06:04:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6e8d2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 0x55d9df2cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6e8d2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 0x55d9df2cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44092/0 (socket says 172.21.15.5:44092) 2024-09-19T06:04:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6e8d2e640 1 -- 172.21.15.5:0/1750023955 learned_addr learned my addr 172.21.15.5:0/1750023955 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/1750023955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6e3fff640 1 --2- 172.21.15.5:0/1750023955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6e1ffb640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb6d00a3550 0x7fb6d00a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6dbfff640 1 -- 172.21.15.5:0/1750023955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 msgr2=0x55d9df2deb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/1750023955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2deb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6dbfff640 1 -- 172.21.15.5:0/1750023955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 msgr2=0x55d9df2cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/1750023955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 0x55d9df2cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6dbfff640 1 -- 172.21.15.5:0/1750023955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9df1db340 con 0x55d9df2df0c0 2024-09-19T06:04:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6e3fff640 1 --2- 172.21.15.5:0/1750023955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2deb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6e8d2e640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb6d00a3550 0x7fb6d00a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6e1ffb640 1 -- 172.21.15.5:0/2938805511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fb6d40f0560 con 0x7fb6e41009d0 2024-09-19T06:04:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/1750023955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2e14b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc004630 tx=0x7fb6cc00bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a346dc50753b9571 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.732+0000 7fb6da7fc640 1 -- 172.21.15.5:0/1750023955 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc003bd0 con 0x55d9df2df0c0 2024-09-19T06:04:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.733+0000 7fb6da7fc640 1 -- 172.21.15.5:0/1750023955 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6cc015070 con 0x55d9df2df0c0 2024-09-19T06:04:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.733+0000 7fb6da7fc640 1 -- 172.21.15.5:0/1750023955 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc004c40 con 0x55d9df2df0c0 2024-09-19T06:04:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.733+0000 7fb6e8d2e640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb6d00a3550 0x7fb6d00a5a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc010610 tx=0x7fb6dc004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.733+0000 7fb6ed609740 1 -- 172.21.15.5:0/1750023955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 msgr2=0x55d9df2e14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.733+0000 7fb6ed609740 1 --2- 172.21.15.5:0/1750023955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2e14b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc004630 tx=0x7fb6cc00bb20 comp rx=0 tx=0).stop 2024-09-19T06:04:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.733+0000 7fb6ed609740 1 -- 172.21.15.5:0/1750023955 shutdown_connections 2024-09-19T06:04:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.733+0000 7fb6ed609740 1 --2- 172.21.15.5:0/1750023955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2e14b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.733+0000 7fb6ed609740 1 --2- 172.21.15.5:0/1750023955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2deb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.733+0000 7fb6ed609740 1 --2- 172.21.15.5:0/1750023955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 0x55d9df2cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.733+0000 7fb6ed609740 1 -- 172.21.15.5:0/1750023955 >> 172.21.15.5:0/1750023955 conn(0x55d9df2cc060 msgr2=0x55d9df2cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.733+0000 7fb6ed609740 1 -- 172.21.15.5:0/1750023955 shutdown_connections 2024-09-19T06:04:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.733+0000 7fb6ed609740 1 -- 172.21.15.5:0/1750023955 wait complete. 2024-09-19T06:04:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6ed609740 1 -- start start 2024-09-19T06:04:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6ed609740 1 Processor -- start 2024-09-19T06:04:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6ed609740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2cf9d0 0x55d9df2dc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6ed609740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2dc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6ed609740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2df0c0 0x55d9df2dca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6ed609740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d9df2e1cf0 con 0x55d9df2df0c0 2024-09-19T06:04:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6ed609740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d9df1bd290 con 0x55d9df2cf9d0 2024-09-19T06:04:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6ed609740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d9df2e1e70 con 0x55d9df2d02f0 2024-09-19T06:04:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6ed609740 10 client.0 _ll_register_callbacks cb 0x55d9df1ee140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6e8d2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2df0c0 0x55d9df2dca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2cf9d0 0x55d9df2dc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6e8d2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2df0c0 0x55d9df2dca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44100/0 (socket says 172.21.15.5:44100) 2024-09-19T06:04:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6e8d2e640 1 -- 172.21.15.5:0/3888551965 learned_addr learned my addr 172.21.15.5:0/3888551965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.734+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2dc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6e8d2e640 1 -- 172.21.15.5:0/3888551965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 msgr2=0x55d9df2dc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6e8d2e640 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2dc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6e8d2e640 1 -- 172.21.15.5:0/3888551965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2cf9d0 msgr2=0x55d9df2dc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6e8d2e640 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2cf9d0 0x55d9df2dc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6e8d2e640 1 -- 172.21.15.5:0/3888551965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9def47210 con 0x55d9df2df0c0 2024-09-19T06:04:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2dc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6e8d2e640 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2df0c0 0x55d9df2dca80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00fac0 tx=0x7fb6dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3ebcdbb0c9955b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6c37fe640 1 -- 172.21.15.5:0/3888551965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc01e9c0 con 0x55d9df2df0c0 2024-09-19T06:04:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6c37fe640 1 -- 172.21.15.5:0/3888551965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fb6dc01eb60 con 0x55d9df2df0c0 2024-09-19T06:04:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6ed609740 1 -- 172.21.15.5:0/3888551965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6cc015070 con 0x55d9df2df0c0 2024-09-19T06:04:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6c37fe640 1 -- 172.21.15.5:0/3888551965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb6dc01ed60 con 0x55d9df2df0c0 2024-09-19T06:04:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6ed609740 10 client.15666 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6ed609740 1 -- 172.21.15.5:0/3888551965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9df013620 con 0x55d9df2df0c0 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.735+0000 7fb6c37fe640 1 -- 172.21.15.5:0/3888551965 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6cc015070 con 0x55d9df2df0c0 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.736+0000 7fb6c37fe640 1 -- 172.21.15.5:0/3888551965 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb6dc028420 con 0x55d9df2df0c0 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.736+0000 7fb6ed609740 10 client.15666 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.736+0000 7fb6ed609740 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.736+0000 7fb6ed609740 20 client.15666 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.736+0000 7fb6ed609740 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9df2ef120 0x55d9df2f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.736+0000 7fb6ed609740 4 client.15666 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.736+0000 7fb6ed609740 1 -- 172.21.15.5:0/3888551965 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9df241a20 con 0x55d9df2ef120 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.736+0000 7fb6e3fff640 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9df2ef120 0x55d9df2f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.736+0000 7fb6c37fe640 10 client.15666 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.736+0000 7fb6e3fff640 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9df2ef120 0x55d9df2f15b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1ae07ec4a5ec140 server_cookie=dc3ea79e39e601b9 in_seq=0 out_seq=0 2024-09-19T06:04:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6c37fe640 1 -- 172.21.15.5:0/3888551965 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9df241a20 con 0x55d9df2ef120 2024-09-19T06:04:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6c37fe640 10 client.15666 handle_command_reply: tid=0 2024-09-19T06:04:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6ed609740 1 client.15666 shutdown 2024-09-19T06:04:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6ed609740 1 -- 172.21.15.5:0/3888551965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2df0c0 msgr2=0x55d9df2dca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6ed609740 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2df0c0 0x55d9df2dca80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00fac0 tx=0x7fb6dc004ab0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6ed609740 1 -- 172.21.15.5:0/3888551965 shutdown_connections 2024-09-19T06:04:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6ed609740 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9df2ef120 0x55d9df2f15b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6ed609740 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2df0c0 0x55d9df2dca80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6c37fe640 0 client.15666 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6ed609740 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2dc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6ed609740 1 --2- 172.21.15.5:0/3888551965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2cf9d0 0x55d9df2dc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6ed609740 1 -- 172.21.15.5:0/3888551965 >> 172.21.15.5:0/3888551965 conn(0x55d9df2cc060 msgr2=0x55d9df2cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6ed609740 1 -- 172.21.15.5:0/3888551965 shutdown_connections 2024-09-19T06:04:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6ed609740 1 -- 172.21.15.5:0/3888551965 wait complete. 2024-09-19T06:04:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.737+0000 7fb6ed609740 20 client.15666 trim_cache size 0 max 16384 2024-09-19T06:04:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.753+0000 7fb6ed609740 1 Processor -- start 2024-09-19T06:04:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.753+0000 7fb6ed609740 1 -- start start 2024-09-19T06:04:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.753+0000 7fb6ed609740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2cf9d0 0x55d9df2ddf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.753+0000 7fb6ed609740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2d02f0 0x55d9df2f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.753+0000 7fb6ed609740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2f7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.753+0000 7fb6ed609740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41ab580 con 0x55d9df2d02f0 2024-09-19T06:04:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.753+0000 7fb6ed609740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41ad0c0 con 0x55d9df2df0c0 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.753+0000 7fb6ed609740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41abc50 con 0x55d9df2cf9d0 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6dbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2d02f0 0x55d9df2f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6dbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2d02f0 0x55d9df2f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44108/0 (socket says 172.21.15.5:44108) 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6dbfff640 1 -- 172.21.15.5:0/2258883809 learned_addr learned my addr 172.21.15.5:0/2258883809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6e8d2e640 1 --2- 172.21.15.5:0/2258883809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6e3fff640 1 --2- 172.21.15.5:0/2258883809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2cf9d0 0x55d9df2ddf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6dbfff640 1 -- 172.21.15.5:0/2258883809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2cf9d0 msgr2=0x55d9df2ddf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/2258883809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2cf9d0 0x55d9df2ddf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6dbfff640 1 -- 172.21.15.5:0/2258883809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 msgr2=0x55d9df2f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/2258883809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2f7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6dbfff640 1 -- 172.21.15.5:0/2258883809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9df2d00e0 con 0x55d9df2d02f0 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/2258883809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2d02f0 0x55d9df2f6ec0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00b620 tx=0x7fb6cc005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b7559b79ce08c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6d9ffb640 1 -- 172.21.15.5:0/2258883809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc029070 con 0x55d9df2d02f0 2024-09-19T06:04:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6d9ffb640 1 -- 172.21.15.5:0/2258883809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6cc015070 con 0x55d9df2d02f0 2024-09-19T06:04:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.754+0000 7fb6d9ffb640 1 -- 172.21.15.5:0/2258883809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc011440 con 0x55d9df2d02f0 2024-09-19T06:04:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.755+0000 7fb6ed609740 1 -- 172.21.15.5:0/2258883809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2d02f0 msgr2=0x55d9df2f6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.755+0000 7fb6ed609740 1 --2- 172.21.15.5:0/2258883809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2d02f0 0x55d9df2f6ec0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00b620 tx=0x7fb6cc005d10 comp rx=0 tx=0).stop 2024-09-19T06:04:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.755+0000 7fb6ed609740 1 -- 172.21.15.5:0/2258883809 shutdown_connections 2024-09-19T06:04:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.755+0000 7fb6ed609740 1 --2- 172.21.15.5:0/2258883809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.755+0000 7fb6ed609740 1 --2- 172.21.15.5:0/2258883809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2d02f0 0x55d9df2f6ec0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.755+0000 7fb6ed609740 1 --2- 172.21.15.5:0/2258883809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2cf9d0 0x55d9df2ddf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.755+0000 7fb6ed609740 1 -- 172.21.15.5:0/2258883809 >> 172.21.15.5:0/2258883809 conn(0x55d9df2cc060 msgr2=0x55d9df2dd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.755+0000 7fb6ed609740 1 -- 172.21.15.5:0/2258883809 shutdown_connections 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.755+0000 7fb6ed609740 1 -- 172.21.15.5:0/2258883809 wait complete. 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.755+0000 7fb6ed609740 1 -- start start 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6ed609740 1 Processor -- start 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6ed609740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 0x55d9df2e2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6ed609740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2e3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6ed609740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2e3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6ed609740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41ac9f0 con 0x55d9df2cf9d0 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6ed609740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41ac320 con 0x55d9df2df0c0 2024-09-19T06:04:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6ed609740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41a9560 con 0x55d9df2d02f0 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6ed609740 10 client.0 _ll_register_callbacks cb 0x7fb6dc009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 0x55d9df2e2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 0x55d9df2e2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44120/0 (socket says 172.21.15.5:44120) 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6e3fff640 1 -- 172.21.15.5:0/4223046072 learned_addr learned my addr 172.21.15.5:0/4223046072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6e8d2e640 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2e3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2e3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6e3fff640 1 -- 172.21.15.5:0/4223046072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 msgr2=0x55d9df2e3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6e3fff640 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2e3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6e3fff640 1 -- 172.21.15.5:0/4223046072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 msgr2=0x55d9df2e3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6e3fff640 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2e3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6e3fff640 1 -- 172.21.15.5:0/4223046072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9df358f10 con 0x55d9df2cf9d0 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6e8d2e640 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2e3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6e3fff640 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 0x55d9df2e2ed0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40edc50 tx=0x7fb6d40bc610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa53d34c2194116 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.756+0000 7fb6c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6c37fe640 1 -- 172.21.15.5:0/4223046072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4049040 con 0x55d9df2cf9d0 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6c37fe640 1 -- 172.21.15.5:0/4223046072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fb6d40bc9e0 con 0x55d9df2cf9d0 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6ed609740 1 -- 172.21.15.5:0/4223046072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6cc015070 con 0x55d9df2cf9d0 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6c37fe640 1 -- 172.21.15.5:0/4223046072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb6d40f3bc0 con 0x55d9df2cf9d0 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6ed609740 10 client.15672 fetch_fsmap learned FSMap version 28 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6ed609740 1 -- 172.21.15.5:0/4223046072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9df35a260 con 0x55d9df2cf9d0 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6c37fe640 1 -- 172.21.15.5:0/4223046072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6cc015070 con 0x55d9df2cf9d0 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6c37fe640 1 -- 172.21.15.5:0/4223046072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb6d40a5030 con 0x55d9df2cf9d0 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6ed609740 10 client.15672 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6ed609740 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6ed609740 20 client.15672 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6ed609740 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9df35a980 0x55d9df35cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6ed609740 4 client.15672 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:04:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6ed609740 1 -- 172.21.15.5:0/4223046072 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9df23bd70 con 0x55d9df35a980 2024-09-19T06:04:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.757+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9df35a980 0x55d9df35cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.758+0000 7fb6dbfff640 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9df35a980 0x55d9df35cd70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f41e7e6fab876745 server_cookie=16097c4fb95b3762 in_seq=0 out_seq=0 2024-09-19T06:04:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.758+0000 7fb6c37fe640 10 client.15672 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.792+0000 7fb6c37fe640 1 -- 172.21.15.5:0/4223046072 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9df23bd70 con 0x55d9df35a980 2024-09-19T06:04:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.792+0000 7fb6c37fe640 10 client.15672 handle_command_reply: tid=0 2024-09-19T06:04:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6ed609740 1 client.15672 shutdown 2024-09-19T06:04:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6ed609740 1 -- 172.21.15.5:0/4223046072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 msgr2=0x55d9df2e2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6ed609740 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 0x55d9df2e2ed0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40edc50 tx=0x7fb6d40bc610 comp rx=0 tx=0).stop 2024-09-19T06:04:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6ed609740 1 -- 172.21.15.5:0/4223046072 shutdown_connections 2024-09-19T06:04:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6ed609740 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9df35a980 0x55d9df35cd70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6ed609740 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9df2df0c0 0x55d9df2e3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6c37fe640 0 client.15672 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6ed609740 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9df2d02f0 0x55d9df2e3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6ed609740 1 --2- 172.21.15.5:0/4223046072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9df2cf9d0 0x55d9df2e2ed0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6ed609740 1 -- 172.21.15.5:0/4223046072 >> 172.21.15.5:0/4223046072 conn(0x55d9df2cc060 msgr2=0x55d9df2d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6ed609740 1 -- 172.21.15.5:0/4223046072 shutdown_connections 2024-09-19T06:04:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6ed609740 1 -- 172.21.15.5:0/4223046072 wait complete. 2024-09-19T06:04:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.793+0000 7fb6ed609740 20 client.15672 trim_cache size 0 max 16384 2024-09-19T06:04:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:04:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:04:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:04:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:04:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.794+0000 7fb6ea791640 1 -- 172.21.15.5:0/2938805511 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb6d00a3550 msgr2=0x7fb6d00a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.794+0000 7fb6ea791640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb6d00a3550 0x7fb6d00a5a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc010610 tx=0x7fb6dc004a40 comp rx=0 tx=0).stop 2024-09-19T06:04:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.794+0000 7fb6ea791640 1 -- 172.21.15.5:0/2938805511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e41009d0 msgr2=0x7fb6e4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.794+0000 7fb6ea791640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e41009d0 0x7fb6e4198360 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4005940 tx=0x7fb6d40042f0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.794+0000 7fb6ea791640 1 -- 172.21.15.5:0/2938805511 shutdown_connections 2024-09-19T06:04:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.794+0000 7fb6ea791640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e41069e0 0x7fb6e419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.794+0000 7fb6ea791640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb6d00a3550 0x7fb6d00a5a10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.794+0000 7fb6ea791640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e4101370 0x7fb6e41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.794+0000 7fb6ea791640 1 --2- 172.21.15.5:0/2938805511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e41009d0 0x7fb6e4198360 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.794+0000 7fb6ea791640 1 -- 172.21.15.5:0/2938805511 >> 172.21.15.5:0/2938805511 conn(0x7fb6e40fc6b0 msgr2=0x7fb6e406a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.794+0000 7fb6ea791640 1 -- 172.21.15.5:0/2938805511 shutdown_connections 2024-09-19T06:04:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:37.794+0000 7fb6ea791640 1 -- 172.21.15.5:0/2938805511 wait complete. 2024-09-19T06:04:37.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 2441 9.37 MB/sec warmup 3 sec latency 34.860 ms 2024-09-19T06:04:38.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:38 smithi157 ceph-mon[35346]: pgmap v41: 97 pgs: 97 active+clean; 4.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 231 KiB/s wr, 10 op/s 2024-09-19T06:04:38.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:38 smithi005 ceph-mon[30955]: pgmap v41: 97 pgs: 97 active+clean; 4.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 231 KiB/s wr, 10 op/s 2024-09-19T06:04:38.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:38 smithi049 ceph-mon[34640]: pgmap v41: 97 pgs: 97 active+clean; 4.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 231 KiB/s wr, 10 op/s 2024-09-19T06:04:38.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 3313 8.84 MB/sec warmup 4 sec latency 14.354 ms 2024-09-19T06:04:39.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 3902 8.01 MB/sec warmup 5 sec latency 14.977 ms 2024-09-19T06:04:40.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:40 smithi005 ceph-mon[30955]: pgmap v42: 97 pgs: 97 active+clean; 22 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 32 op/s 2024-09-19T06:04:40.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:40 smithi049 ceph-mon[34640]: pgmap v42: 97 pgs: 97 active+clean; 22 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 32 op/s 2024-09-19T06:04:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:40 smithi157 ceph-mon[35346]: pgmap v42: 97 pgs: 97 active+clean; 22 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 32 op/s 2024-09-19T06:04:40.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 4467 6.80 MB/sec warmup 6 sec latency 13.986 ms 2024-09-19T06:04:41.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 5197 6.47 MB/sec warmup 7 sec latency 15.869 ms 2024-09-19T06:04:42.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:42 smithi005 ceph-mon[30955]: pgmap v43: 97 pgs: 97 active+clean; 22 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 27 op/s 2024-09-19T06:04:42.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:42 smithi049 ceph-mon[34640]: pgmap v43: 97 pgs: 97 active+clean; 22 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 27 op/s 2024-09-19T06:04:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:42 smithi157 ceph-mon[35346]: pgmap v43: 97 pgs: 97 active+clean; 22 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 27 op/s 2024-09-19T06:04:42.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 5949 5.89 MB/sec warmup 8 sec latency 13.553 ms 2024-09-19T06:04:43.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 6848 6.05 MB/sec warmup 9 sec latency 10.585 ms 2024-09-19T06:04:44.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:44 smithi005 ceph-mon[30955]: pgmap v44: 97 pgs: 97 active+clean; 27 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.6 MiB/s wr, 90 op/s 2024-09-19T06:04:44.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:44 smithi049 ceph-mon[34640]: pgmap v44: 97 pgs: 97 active+clean; 27 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.6 MiB/s wr, 90 op/s 2024-09-19T06:04:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:44 smithi157 ceph-mon[35346]: pgmap v44: 97 pgs: 97 active+clean; 27 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.6 MiB/s wr, 90 op/s 2024-09-19T06:04:44.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 7391 5.89 MB/sec warmup 10 sec latency 20.308 ms 2024-09-19T06:04:45.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:45 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:04:45.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:45 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:04:45.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:45 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:04:45.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 7882 5.42 MB/sec warmup 11 sec latency 10.765 ms 2024-09-19T06:04:46.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:46 smithi005 ceph-mon[30955]: pgmap v45: 97 pgs: 97 active+clean; 27 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.6 MiB/s wr, 86 op/s 2024-09-19T06:04:46.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:46 smithi049 ceph-mon[34640]: pgmap v45: 97 pgs: 97 active+clean; 27 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.6 MiB/s wr, 86 op/s 2024-09-19T06:04:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:46 smithi157 ceph-mon[35346]: pgmap v45: 97 pgs: 97 active+clean; 27 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.6 MiB/s wr, 86 op/s 2024-09-19T06:04:46.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 8575 5.36 MB/sec warmup 12 sec latency 13.896 ms 2024-09-19T06:04:47.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 9279 5.00 MB/sec warmup 13 sec latency 12.485 ms 2024-09-19T06:04:48.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:04:48.610 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:48.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:48 smithi005 ceph-mon[30955]: pgmap v46: 97 pgs: 97 active+clean; 27 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.6 MiB/s wr, 86 op/s 2024-09-19T06:04:48.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:48 smithi049 ceph-mon[34640]: pgmap v46: 97 pgs: 97 active+clean; 27 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.6 MiB/s wr, 86 op/s 2024-09-19T06:04:48.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:48 smithi157 ceph-mon[35346]: pgmap v46: 97 pgs: 97 active+clean; 27 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.6 MiB/s wr, 86 op/s 2024-09-19T06:04:48.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 9973 5.03 MB/sec warmup 14 sec latency 14.859 ms 2024-09-19T06:04:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.134+0000 7fedb6526640 1 -- 172.21.15.5:0/254953320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb0102db0 msgr2=0x7fedb0103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.134+0000 7fedb6526640 1 --2- 172.21.15.5:0/254953320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb0102db0 0x7fedb0103190 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7feda0009930 tx=0x7feda002a420 comp rx=0 tx=0).stop 2024-09-19T06:04:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.134+0000 7fedb6526640 1 -- 172.21.15.5:0/254953320 shutdown_connections 2024-09-19T06:04:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.134+0000 7fedb6526640 1 --2- 172.21.15.5:0/254953320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedb01082e0 0x7fedb010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.134+0000 7fedb6526640 1 --2- 172.21.15.5:0/254953320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedb0103760 0x7fedb0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.134+0000 7fedb6526640 1 --2- 172.21.15.5:0/254953320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb0102db0 0x7fedb0103190 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.134+0000 7fedb6526640 1 -- 172.21.15.5:0/254953320 >> 172.21.15.5:0/254953320 conn(0x7fedb00fc7d0 msgr2=0x7fedb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.134+0000 7fedb6526640 1 -- 172.21.15.5:0/254953320 shutdown_connections 2024-09-19T06:04:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.134+0000 7fedb6526640 1 -- 172.21.15.5:0/254953320 wait complete. 2024-09-19T06:04:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.134+0000 7fedb6526640 1 Processor -- start 2024-09-19T06:04:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.134+0000 7fedb6526640 1 -- start start 2024-09-19T06:04:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb6526640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedb0102db0 0x7fedb0198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb6526640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedb0103760 0x7fedb0198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb6526640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb01082e0 0x7fedb019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb6526640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedb0111f40 con 0x7fedb01082e0 2024-09-19T06:04:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb6526640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedb0111dc0 con 0x7fedb0103760 2024-09-19T06:04:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb6526640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01120c0 con 0x7fedb0102db0 2024-09-19T06:04:49.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb4ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb01082e0 0x7fedb019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb4ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb01082e0 0x7fedb019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37166/0 (socket says 172.21.15.5:37166) 2024-09-19T06:04:49.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedb0103760 0x7fedb0198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb4ac3640 1 -- 172.21.15.5:0/4277965176 learned_addr learned my addr 172.21.15.5:0/4277965176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:49.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedaffff640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedb0102db0 0x7fedb0198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb4ac3640 1 -- 172.21.15.5:0/4277965176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedb0102db0 msgr2=0x7fedb0198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:49.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb4ac3640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedb0102db0 0x7fedb0198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb4ac3640 1 -- 172.21.15.5:0/4277965176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedb0103760 msgr2=0x7fedb0198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:49.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb4ac3640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedb0103760 0x7fedb0198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedb4ac3640 1 -- 172.21.15.5:0/4277965176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb019d4d0 con 0x7fedb01082e0 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.135+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedb0103760 0x7fedb0198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.136+0000 7fedb4ac3640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb01082e0 0x7fedb019cdf0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7feda8009fd0 tx=0x7feda800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.136+0000 7fedad7fa640 1 -- 172.21.15.5:0/4277965176 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8017070 con 0x7fedb01082e0 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.136+0000 7fedad7fa640 1 -- 172.21.15.5:0/4277965176 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feda8003e20 con 0x7fedb01082e0 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.136+0000 7fedb6526640 1 -- 172.21.15.5:0/4277965176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb019d7c0 con 0x7fedb01082e0 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.136+0000 7fedad7fa640 1 -- 172.21.15.5:0/4277965176 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8010460 con 0x7fedb01082e0 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.136+0000 7fedb6526640 1 -- 172.21.15.5:0/4277965176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb0100940 con 0x7fedb01082e0 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.138+0000 7fedad7fa640 1 -- 172.21.15.5:0/4277965176 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7feda8003970 con 0x7fedb01082e0 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.138+0000 7fedb88ff740 1 Processor -- start 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.138+0000 7fedb88ff740 1 -- start start 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedb88ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edc6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedb88ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edc6dc0 0x555c8edd1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedb88ff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edd1680 0x555c8edd3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedb88ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c8ed2c2e0 con 0x555c8edc64a0 2024-09-19T06:04:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedb88ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555c8ecd3ce0 con 0x555c8edc6dc0 2024-09-19T06:04:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedb88ff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555c8ed282a0 con 0x555c8edd1680 2024-09-19T06:04:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edc6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedaffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edc6dc0 0x555c8edd1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedad7fa640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fed980a34d0 0x7fed980a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedaffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edc6dc0 0x555c8edd1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33928/0 (socket says 172.21.15.5:33928) 2024-09-19T06:04:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedaffff640 1 -- 172.21.15.5:0/942293179 learned_addr learned my addr 172.21.15.5:0/942293179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edc6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37172/0 (socket says 172.21.15.5:37172) 2024-09-19T06:04:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedb4ac3640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fed980a34d0 0x7fed980a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedad7fa640 1 -- 172.21.15.5:0/4277965176 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7feda80cfd10 con 0x7fedb01082e0 2024-09-19T06:04:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedaffff640 1 -- 172.21.15.5:0/942293179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edd1680 msgr2=0x555c8edd3a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:04:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedaffff640 1 --2- 172.21.15.5:0/942293179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edd1680 0x555c8edd3a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedaffff640 1 -- 172.21.15.5:0/942293179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 msgr2=0x555c8edc6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedaffff640 1 --2- 172.21.15.5:0/942293179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edc6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.139+0000 7fedaffff640 1 -- 172.21.15.5:0/942293179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8ecb8440 con 0x555c8edc6dc0 2024-09-19T06:04:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.140+0000 7fedaffff640 1 --2- 172.21.15.5:0/942293179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edc6dc0 0x555c8edd1140 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feda0005ec0 tx=0x7feda0005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f970b513c74165c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.140+0000 7fedb4ac3640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fed980a34d0 0x7fed980a5990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feda80cd350 tx=0x7feda80d0010 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.140+0000 7fed967fc640 1 -- 172.21.15.5:0/942293179 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0040070 con 0x555c8edc6dc0 2024-09-19T06:04:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.140+0000 7fed967fc640 1 -- 172.21.15.5:0/942293179 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feda0031e10 con 0x555c8edc6dc0 2024-09-19T06:04:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.140+0000 7fed967fc640 1 -- 172.21.15.5:0/942293179 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0035040 con 0x555c8edc6dc0 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.140+0000 7fedb88ff740 1 -- 172.21.15.5:0/942293179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edc6dc0 msgr2=0x555c8edd1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.140+0000 7fedb88ff740 1 --2- 172.21.15.5:0/942293179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edc6dc0 0x555c8edd1140 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feda0005ec0 tx=0x7feda0005c00 comp rx=0 tx=0).stop 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.140+0000 7fedb88ff740 1 -- 172.21.15.5:0/942293179 shutdown_connections 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.141+0000 7fedb88ff740 1 --2- 172.21.15.5:0/942293179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edd1680 0x555c8edd3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.141+0000 7fedb88ff740 1 --2- 172.21.15.5:0/942293179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edc6dc0 0x555c8edd1140 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.141+0000 7fedb88ff740 1 --2- 172.21.15.5:0/942293179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edc6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.141+0000 7fedb88ff740 1 -- 172.21.15.5:0/942293179 >> 172.21.15.5:0/942293179 conn(0x555c8edc2a30 msgr2=0x555c8edc2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.141+0000 7fedb88ff740 1 -- 172.21.15.5:0/942293179 shutdown_connections 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.141+0000 7fedb88ff740 1 -- 172.21.15.5:0/942293179 wait complete. 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.141+0000 7fedb88ff740 1 -- start start 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.141+0000 7fedb88ff740 1 Processor -- start 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedb88ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edd0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedb88ff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8edce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedb88ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8edce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedb88ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c8edd4250 con 0x555c8edc64a0 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedb88ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555c8ece7050 con 0x555c8edd1680 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedb88ff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555c8edd43d0 con 0x555c8edc6dc0 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedb88ff740 10 client.0 _ll_register_callbacks cb 0x555c8ed27160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edd0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedaffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8edce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edd0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37176/0 (socket says 172.21.15.5:37176) 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedaf7fe640 1 -- 172.21.15.5:0/3455873438 learned_addr learned my addr 172.21.15.5:0/3455873438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8edce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedaffff640 1 -- 172.21.15.5:0/3455873438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 msgr2=0x555c8edce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedaffff640 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8edce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedaffff640 1 -- 172.21.15.5:0/3455873438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 msgr2=0x555c8edd0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedaffff640 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edd0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedaffff640 1 -- 172.21.15.5:0/3455873438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c8ea9d3c0 con 0x555c8edc6dc0 2024-09-19T06:04:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8edce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fedaffff640 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8edce230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feda0006010 tx=0x7feda0005560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91b747cfe8bc9323 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.142+0000 7fed8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:04:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fed8f7fe640 1 -- 172.21.15.5:0/3455873438 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda004b030 con 0x555c8edc6dc0 2024-09-19T06:04:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fedb88ff740 1 -- 172.21.15.5:0/3455873438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda0031e10 con 0x555c8edc6dc0 2024-09-19T06:04:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fed8f7fe640 1 -- 172.21.15.5:0/3455873438 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7feda0031640 con 0x555c8edc6dc0 2024-09-19T06:04:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fed8f7fe640 1 -- 172.21.15.5:0/3455873438 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feda0055050 con 0x555c8edc6dc0 2024-09-19T06:04:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fedb88ff740 10 client.25225 fetch_fsmap learned FSMap version 29 2024-09-19T06:04:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fedb88ff740 1 -- 172.21.15.5:0/3455873438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555c8edd4250 con 0x555c8edc6dc0 2024-09-19T06:04:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fed8f7fe640 1 -- 172.21.15.5:0/3455873438 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feda0031e10 con 0x555c8edc6dc0 2024-09-19T06:04:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fed8f7fe640 1 -- 172.21.15.5:0/3455873438 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7feda0004bc0 con 0x555c8edc6dc0 2024-09-19T06:04:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fedb88ff740 10 client.25225 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:04:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fedb88ff740 10 client.25225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fedb88ff740 20 client.25225 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fedb88ff740 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555c8ede0d90 0x555c8ede3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fedb88ff740 4 client.25225 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fedb88ff740 1 -- 172.21.15.5:0/3455873438 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c8ed39850 con 0x555c8ede0d90 2024-09-19T06:04:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.143+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555c8ede0d90 0x555c8ede3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.144+0000 7fed8f7fe640 10 client.25225 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.144+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555c8ede0d90 0x555c8ede3180 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84bee6fe0f651321 server_cookie=abc2b2c5c7c57af6 in_seq=0 out_seq=0 2024-09-19T06:04:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.144+0000 7fed8f7fe640 1 -- 172.21.15.5:0/3455873438 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c8ed39850 con 0x555c8ede0d90 2024-09-19T06:04:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.144+0000 7fed8f7fe640 10 client.25225 handle_command_reply: tid=0 2024-09-19T06:04:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.144+0000 7fedb88ff740 1 client.25225 shutdown 2024-09-19T06:04:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.145+0000 7fedb88ff740 1 -- 172.21.15.5:0/3455873438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 msgr2=0x555c8edce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.145+0000 7fedb88ff740 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8edce230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feda0006010 tx=0x7feda0005560 comp rx=0 tx=0).stop 2024-09-19T06:04:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.145+0000 7fedb88ff740 1 -- 172.21.15.5:0/3455873438 shutdown_connections 2024-09-19T06:04:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.145+0000 7fedb88ff740 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555c8ede0d90 0x555c8ede3180 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.145+0000 7fedb88ff740 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8edce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.145+0000 7fedb88ff740 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8edce230 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.145+0000 7fedb88ff740 1 --2- 172.21.15.5:0/3455873438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edd0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.145+0000 7fed8f7fe640 0 client.25225 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.145+0000 7fedb88ff740 1 -- 172.21.15.5:0/3455873438 >> 172.21.15.5:0/3455873438 conn(0x555c8edc2a30 msgr2=0x555c8edc8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.145+0000 7fedb88ff740 1 -- 172.21.15.5:0/3455873438 shutdown_connections 2024-09-19T06:04:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.145+0000 7fedb88ff740 1 -- 172.21.15.5:0/3455873438 wait complete. 2024-09-19T06:04:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.145+0000 7fedb88ff740 20 client.25225 trim_cache size 0 max 16384 2024-09-19T06:04:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedb88ff740 1 Processor -- start 2024-09-19T06:04:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedb88ff740 1 -- start start 2024-09-19T06:04:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedb88ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edcd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedb88ff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8ede8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedb88ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8ede8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedb88ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01ab5a0 con 0x555c8edc64a0 2024-09-19T06:04:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedb88ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01ad110 con 0x555c8edd1680 2024-09-19T06:04:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedb88ff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01abc70 con 0x555c8edc6dc0 2024-09-19T06:04:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edcd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edcd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37186/0 (socket says 172.21.15.5:37186) 2024-09-19T06:04:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedaf7fe640 1 -- 172.21.15.5:0/2831309486 learned_addr learned my addr 172.21.15.5:0/2831309486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/2831309486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8ede8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedaffff640 1 --2- 172.21.15.5:0/2831309486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8ede8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedaf7fe640 1 -- 172.21.15.5:0/2831309486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 msgr2=0x555c8ede8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/2831309486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8ede8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedaf7fe640 1 -- 172.21.15.5:0/2831309486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 msgr2=0x555c8ede8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/2831309486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8ede8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.164+0000 7fedaf7fe640 1 -- 172.21.15.5:0/2831309486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8edceaa0 con 0x555c8edc64a0 2024-09-19T06:04:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.165+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/2831309486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edcd4b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feda4005fd0 tx=0x7feda400f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7e8d0a88c3f419 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.165+0000 7fed95ffb640 1 -- 172.21.15.5:0/2831309486 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4004220 con 0x555c8edc64a0 2024-09-19T06:04:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.165+0000 7fed95ffb640 1 -- 172.21.15.5:0/2831309486 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feda40043c0 con 0x555c8edc64a0 2024-09-19T06:04:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.165+0000 7fed95ffb640 1 -- 172.21.15.5:0/2831309486 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4004cf0 con 0x555c8edc64a0 2024-09-19T06:04:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.166+0000 7fedb88ff740 1 -- 172.21.15.5:0/2831309486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 msgr2=0x555c8edcd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.166+0000 7fedb88ff740 1 --2- 172.21.15.5:0/2831309486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edcd4b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feda4005fd0 tx=0x7feda400f820 comp rx=0 tx=0).stop 2024-09-19T06:04:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.166+0000 7fedb88ff740 1 -- 172.21.15.5:0/2831309486 shutdown_connections 2024-09-19T06:04:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.166+0000 7fedb88ff740 1 --2- 172.21.15.5:0/2831309486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8ede8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.166+0000 7fedb88ff740 1 --2- 172.21.15.5:0/2831309486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8ede8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.166+0000 7fedb88ff740 1 --2- 172.21.15.5:0/2831309486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edcd4b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.166+0000 7fedb88ff740 1 -- 172.21.15.5:0/2831309486 >> 172.21.15.5:0/2831309486 conn(0x555c8edc2a30 msgr2=0x555c8edd02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.166+0000 7fedb88ff740 1 -- 172.21.15.5:0/2831309486 shutdown_connections 2024-09-19T06:04:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.166+0000 7fedb88ff740 1 -- 172.21.15.5:0/2831309486 wait complete. 2024-09-19T06:04:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.166+0000 7fedb88ff740 1 -- start start 2024-09-19T06:04:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedb88ff740 1 Processor -- start 2024-09-19T06:04:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedb88ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edd99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedb88ff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8edd9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedb88ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8edd6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedb88ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01aca10 con 0x555c8edc64a0 2024-09-19T06:04:49.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedb88ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01ac340 con 0x555c8edd1680 2024-09-19T06:04:49.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedb88ff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01a9580 con 0x555c8edc6dc0 2024-09-19T06:04:49.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedb88ff740 10 client.0 _ll_register_callbacks cb 0x7feda0031aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:49.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edd99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edd99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37190/0 (socket says 172.21.15.5:37190) 2024-09-19T06:04:49.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedaf7fe640 1 -- 172.21.15.5:0/2964520926 learned_addr learned my addr 172.21.15.5:0/2964520926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:49.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedaffff640 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8edd9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8edd6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedaf7fe640 1 -- 172.21.15.5:0/2964520926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 msgr2=0x555c8edd9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:49.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8edd9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedaf7fe640 1 -- 172.21.15.5:0/2964520926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 msgr2=0x555c8edd6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8edd6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedaf7fe640 1 -- 172.21.15.5:0/2964520926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c8edd7740 con 0x555c8edc64a0 2024-09-19T06:04:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.167+0000 7fedaffff640 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8edd9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.168+0000 7fedaf7fe640 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edd99c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7feda4004610 tx=0x7feda400f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23db3f258eb1196b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.168+0000 7fed8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.168+0000 7fed8f7fe640 1 -- 172.21.15.5:0/2964520926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4032030 con 0x555c8edc64a0 2024-09-19T06:04:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.168+0000 7fed8f7fe640 1 -- 172.21.15.5:0/2964520926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7feda4021800 con 0x555c8edc64a0 2024-09-19T06:04:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.168+0000 7fedb88ff740 1 -- 172.21.15.5:0/2964520926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda40043c0 con 0x555c8edc64a0 2024-09-19T06:04:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.168+0000 7fed8f7fe640 1 -- 172.21.15.5:0/2964520926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feda4044050 con 0x555c8edc64a0 2024-09-19T06:04:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.168+0000 7fedb88ff740 10 client.15690 fetch_fsmap learned FSMap version 29 2024-09-19T06:04:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.168+0000 7fedb88ff740 1 -- 172.21.15.5:0/2964520926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555c8ee4b5e0 con 0x555c8edc64a0 2024-09-19T06:04:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.168+0000 7fed8f7fe640 1 -- 172.21.15.5:0/2964520926 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feda40043c0 con 0x555c8edc64a0 2024-09-19T06:04:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.168+0000 7fed8f7fe640 1 -- 172.21.15.5:0/2964520926 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7feda4028040 con 0x555c8edc64a0 2024-09-19T06:04:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.169+0000 7fedb88ff740 10 client.15690 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:04:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.169+0000 7fedb88ff740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.169+0000 7fedb88ff740 20 client.15690 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.169+0000 7fedb88ff740 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555c8ee4bbf0 0x555c8ee4e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.169+0000 7fedb88ff740 4 client.15690 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:04:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.169+0000 7fedb88ff740 1 -- 172.21.15.5:0/2964520926 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c8ed33d60 con 0x555c8ee4bbf0 2024-09-19T06:04:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.169+0000 7fedaffff640 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555c8ee4bbf0 0x555c8ee4e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.169+0000 7fed8f7fe640 10 client.15690 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.169+0000 7fedaffff640 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555c8ee4bbf0 0x555c8ee4e010 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd710518384a2428 server_cookie=c7fbed2bb9332eb2 in_seq=0 out_seq=0 2024-09-19T06:04:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.169+0000 7fed8f7fe640 1 -- 172.21.15.5:0/2964520926 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555c8ed33d60 con 0x555c8ee4bbf0 2024-09-19T06:04:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.169+0000 7fed8f7fe640 10 client.15690 handle_command_reply: tid=0 2024-09-19T06:04:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fedb88ff740 1 client.15690 shutdown 2024-09-19T06:04:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fedb88ff740 1 -- 172.21.15.5:0/2964520926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 msgr2=0x555c8edd99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fedb88ff740 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edd99c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7feda4004610 tx=0x7feda400f440 comp rx=0 tx=0).stop 2024-09-19T06:04:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fedb88ff740 1 -- 172.21.15.5:0/2964520926 shutdown_connections 2024-09-19T06:04:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fedb88ff740 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555c8ee4bbf0 0x555c8ee4e010 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fedb88ff740 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c8edd1680 0x555c8edd6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fedb88ff740 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c8edc6dc0 0x555c8edd9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fedb88ff740 1 --2- 172.21.15.5:0/2964520926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c8edc64a0 0x555c8edd99c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fedb88ff740 1 -- 172.21.15.5:0/2964520926 >> 172.21.15.5:0/2964520926 conn(0x555c8edc2a30 msgr2=0x555c8edc1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fed8f7fe640 0 client.15690 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fedb88ff740 1 -- 172.21.15.5:0/2964520926 shutdown_connections 2024-09-19T06:04:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fedb88ff740 1 -- 172.21.15.5:0/2964520926 wait complete. 2024-09-19T06:04:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.170+0000 7fedb88ff740 20 client.15690 trim_cache size 0 max 16384 2024-09-19T06:04:49.187 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:04:49.187 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:04:49.188 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:04:49.188 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:04:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.171+0000 7fedb6526640 1 -- 172.21.15.5:0/4277965176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fed980a34d0 msgr2=0x7fed980a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.171+0000 7fedb6526640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fed980a34d0 0x7fed980a5990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feda80cd350 tx=0x7feda80d0010 comp rx=0 tx=0).stop 2024-09-19T06:04:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.171+0000 7fedb6526640 1 -- 172.21.15.5:0/4277965176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb01082e0 msgr2=0x7fedb019cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.171+0000 7fedb6526640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb01082e0 0x7fedb019cdf0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7feda8009fd0 tx=0x7feda800b260 comp rx=0 tx=0).stop 2024-09-19T06:04:49.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.171+0000 7fedb6526640 1 -- 172.21.15.5:0/4277965176 shutdown_connections 2024-09-19T06:04:49.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.171+0000 7fedb6526640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb01082e0 0x7fedb019cdf0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.171+0000 7fedb6526640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedb0103760 0x7fedb0198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.171+0000 7fedb6526640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fed980a34d0 0x7fed980a5990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.171+0000 7fedb6526640 1 --2- 172.21.15.5:0/4277965176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedb0102db0 0x7fedb0198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:49.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.171+0000 7fedb6526640 1 -- 172.21.15.5:0/4277965176 >> 172.21.15.5:0/4277965176 conn(0x7fedb00fc7d0 msgr2=0x7fedb0105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:49.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.171+0000 7fedb6526640 1 -- 172.21.15.5:0/4277965176 shutdown_connections 2024-09-19T06:04:49.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:49.171+0000 7fedb6526640 1 -- 172.21.15.5:0/4277965176 wait complete. 2024-09-19T06:04:49.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 10448 4.91 MB/sec warmup 15 sec latency 28.191 ms 2024-09-19T06:04:49.881 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:04:49.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:04:50.173 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:50.598 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:50 smithi005 ceph-mon[30955]: pgmap v47: 97 pgs: 97 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 3.1 MiB/s wr, 129 op/s 2024-09-19T06:04:50.598 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:50 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:04:50.603 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:50 smithi157 ceph-mon[35346]: pgmap v47: 97 pgs: 97 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 3.1 MiB/s wr, 129 op/s 2024-09-19T06:04:50.603 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:50 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:04:50.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:50 smithi049 ceph-mon[34640]: pgmap v47: 97 pgs: 97 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 3.1 MiB/s wr, 129 op/s 2024-09-19T06:04:50.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:50 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:04:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.690+0000 7ff8abdd3640 1 -- 172.21.15.5:0/242788330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4102ef0 msgr2=0x7ff8a410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.690+0000 7ff8abdd3640 1 --2- 172.21.15.5:0/242788330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4102ef0 0x7ff8a410da40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009ca0 tx=0x7ff89c02a950 comp rx=0 tx=0).stop 2024-09-19T06:04:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.690+0000 7ff8abdd3640 1 -- 172.21.15.5:0/242788330 shutdown_connections 2024-09-19T06:04:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.690+0000 7ff8abdd3640 1 --2- 172.21.15.5:0/242788330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4102ef0 0x7ff8a410da40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.690+0000 7ff8abdd3640 1 --2- 172.21.15.5:0/242788330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a4102570 0x7ff8a41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.690+0000 7ff8abdd3640 1 --2- 172.21.15.5:0/242788330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a4100e20 0x7ff8a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.690+0000 7ff8abdd3640 1 -- 172.21.15.5:0/242788330 >> 172.21.15.5:0/242788330 conn(0x7ff8a40fc7d0 msgr2=0x7ff8a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.690+0000 7ff8abdd3640 1 -- 172.21.15.5:0/242788330 shutdown_connections 2024-09-19T06:04:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.690+0000 7ff8abdd3640 1 -- 172.21.15.5:0/242788330 wait complete. 2024-09-19T06:04:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8abdd3640 1 Processor -- start 2024-09-19T06:04:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8abdd3640 1 -- start start 2024-09-19T06:04:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8abdd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4100e20 0x7ff8a41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8abdd3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a4102570 0x7ff8a41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8abdd3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a4102ef0 0x7ff8a419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8abdd3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4110e30 con 0x7ff8a4100e20 2024-09-19T06:04:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8abdd3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4110cb0 con 0x7ff8a4102570 2024-09-19T06:04:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8abdd3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4110fb0 con 0x7ff8a4102ef0 2024-09-19T06:04:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8a9b6f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4100e20 0x7ff8a41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8a9b6f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4100e20 0x7ff8a41983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37210/0 (socket says 172.21.15.5:37210) 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8a9b6f640 1 -- 172.21.15.5:0/1629490912 learned_addr learned my addr 172.21.15.5:0/1629490912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8aa370640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a4102ef0 0x7ff8a419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.691+0000 7ff8a936e640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a4102570 0x7ff8a41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.692+0000 7ff8a9b6f640 1 -- 172.21.15.5:0/1629490912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a4102ef0 msgr2=0x7ff8a419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.692+0000 7ff8a9b6f640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a4102ef0 0x7ff8a419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.692+0000 7ff8a9b6f640 1 -- 172.21.15.5:0/1629490912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a4102570 msgr2=0x7ff8a41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.692+0000 7ff8a9b6f640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a4102570 0x7ff8a41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.692+0000 7ff8a9b6f640 1 -- 172.21.15.5:0/1629490912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a419d360 con 0x7ff8a4100e20 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.692+0000 7ff8a9b6f640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4100e20 0x7ff8a41983b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000e9f0 tx=0x7ff8a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.692+0000 7ff89affd640 1 -- 172.21.15.5:0/1629490912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a000cd80 con 0x7ff8a4100e20 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.692+0000 7ff89affd640 1 -- 172.21.15.5:0/1629490912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8a00108e0 con 0x7ff8a4100e20 2024-09-19T06:04:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.692+0000 7ff8abdd3640 1 -- 172.21.15.5:0/1629490912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a419d650 con 0x7ff8a4100e20 2024-09-19T06:04:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.692+0000 7ff89affd640 1 -- 172.21.15.5:0/1629490912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0018920 con 0x7ff8a4100e20 2024-09-19T06:04:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.692+0000 7ff8abdd3640 1 -- 172.21.15.5:0/1629490912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a41a4f30 con 0x7ff8a4100e20 2024-09-19T06:04:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.693+0000 7ff8abdd3640 1 -- 172.21.15.5:0/1629490912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a4101200 con 0x7ff8a4100e20 2024-09-19T06:04:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.694+0000 7ff89affd640 1 -- 172.21.15.5:0/1629490912 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7ff8a0010a90 con 0x7ff8a4100e20 2024-09-19T06:04:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.695+0000 7ff89affd640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff8900a3550 0x7ff8900a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.695+0000 7ff8a936e640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff8900a3550 0x7ff8900a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.695+0000 7ff89affd640 1 -- 172.21.15.5:0/1629490912 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7ff8a00d01f0 con 0x7ff8a4100e20 2024-09-19T06:04:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.696+0000 7ff8a936e640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff8900a3550 0x7ff8900a5a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4100030 tx=0x7ff894005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.700+0000 7ff89affd640 1 -- 172.21.15.5:0/1629490912 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff8a009c340 con 0x7ff8a4100e20 2024-09-19T06:04:50.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 10928 4.88 MB/sec warmup 16 sec latency 30.364 ms 2024-09-19T06:04:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.890+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/201505859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8ec10e430 msgr2=0x7fd8ec110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.890+0000 7fd8f1ef1640 1 --2- 172.21.15.5:0/201505859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8ec10e430 0x7fd8ec110820 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009ca0 tx=0x7fd8e402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.890+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/201505859 shutdown_connections 2024-09-19T06:04:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.890+0000 7fd8f1ef1640 1 --2- 172.21.15.5:0/201505859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8ec10e430 0x7fd8ec110820 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.890+0000 7fd8f1ef1640 1 --2- 172.21.15.5:0/201505859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8ec101230 0x7fd8ec10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.890+0000 7fd8f1ef1640 1 --2- 172.21.15.5:0/201505859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8ec100880 0x7fd8ec100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.890+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/201505859 >> 172.21.15.5:0/201505859 conn(0x7fd8ec0779a0 msgr2=0x7fd8ec0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.890+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/201505859 shutdown_connections 2024-09-19T06:04:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.890+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/201505859 wait complete. 2024-09-19T06:04:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.891+0000 7fd8f1ef1640 1 Processor -- start 2024-09-19T06:04:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.891+0000 7fd8f1ef1640 1 -- start start 2024-09-19T06:04:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.891+0000 7fd8f1ef1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8ec100880 0x7fd8ec19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.891+0000 7fd8f1ef1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8ec101230 0x7fd8ec19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.891+0000 7fd8f1ef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8ec10e430 0x7fd8ec19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.891+0000 7fd8f1ef1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec113f40 con 0x7fd8ec101230 2024-09-19T06:04:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.891+0000 7fd8f1ef1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec113dc0 con 0x7fd8ec10e430 2024-09-19T06:04:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.891+0000 7fd8f1ef1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1140c0 con 0x7fd8ec100880 2024-09-19T06:04:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.891+0000 7fd8eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8ec101230 0x7fd8ec19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8eb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8ec100880 0x7fd8ec19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8ebfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8ec10e430 0x7fd8ec19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8ebfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8ec10e430 0x7fd8ec19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33996/0 (socket says 172.21.15.5:33996) 2024-09-19T06:04:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8eb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8ec100880 0x7fd8ec19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33882/0 (socket says 172.21.15.5:33882) 2024-09-19T06:04:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8ebfff640 1 -- 172.21.15.5:0/1923763130 learned_addr learned my addr 172.21.15.5:0/1923763130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8ec101230 0x7fd8ec19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37226/0 (socket says 172.21.15.5:37226) 2024-09-19T06:04:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8eaffd640 1 -- 172.21.15.5:0/1923763130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8ec100880 msgr2=0x7fd8ec19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8eaffd640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8ec100880 0x7fd8ec19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8eaffd640 1 -- 172.21.15.5:0/1923763130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8ec10e430 msgr2=0x7fd8ec19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8eaffd640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8ec10e430 0x7fd8ec19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8eaffd640 1 -- 172.21.15.5:0/1923763130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec19f5d0 con 0x7fd8ec101230 2024-09-19T06:04:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8ebfff640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8ec10e430 0x7fd8ec19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8eaffd640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8ec101230 0x7fd8ec19ab20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000e9f0 tx=0x7fd8e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8e8ff9640 1 -- 172.21.15.5:0/1923763130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000ccc0 con 0x7fd8ec101230 2024-09-19T06:04:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/1923763130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec19f8c0 con 0x7fd8ec101230 2024-09-19T06:04:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8e8ff9640 1 -- 172.21.15.5:0/1923763130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8e0005020 con 0x7fd8ec101230 2024-09-19T06:04:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8e8ff9640 1 -- 172.21.15.5:0/1923763130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0004a00 con 0x7fd8ec101230 2024-09-19T06:04:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.892+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/1923763130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec1a7150 con 0x7fd8ec101230 2024-09-19T06:04:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.895+0000 7fd8f4c8c740 1 Processor -- start 2024-09-19T06:04:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.895+0000 7fd8f4c8c740 1 -- start start 2024-09-19T06:04:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.895+0000 7fd8f4c8c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.895+0000 7fd8f4c8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c7410 0x55ea4f1d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.895+0000 7fd8f4c8c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.895+0000 7fd8f4c8c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea4f12b2f0 con 0x55ea4f1d1cb0 2024-09-19T06:04:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.895+0000 7fd8f4c8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ea4f0d3270 con 0x55ea4f1c7410 2024-09-19T06:04:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.895+0000 7fd8f4c8c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ea4f1272b0 con 0x55ea4f1c6af0 2024-09-19T06:04:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8eb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c7410 0x55ea4f1d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8eb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c7410 0x55ea4f1d1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:34004/0 (socket says 172.21.15.5:34004) 2024-09-19T06:04:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/468681546 learned_addr learned my addr 172.21.15.5:0/468681546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8ebfff640 1 --2- 172.21.15.5:0/468681546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/468681546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1c6af0 msgr2=0x55ea4f1c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8eb7fe640 1 --2- 172.21.15.5:0/468681546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1c6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/468681546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1d1cb0 msgr2=0x55ea4f1d40a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:04:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8eb7fe640 1 --2- 172.21.15.5:0/468681546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1d40a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/468681546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea4f0d2340 con 0x55ea4f1c7410 2024-09-19T06:04:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8eb7fe640 1 --2- 172.21.15.5:0/468681546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c7410 0x55ea4f1d1770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009cf0 tx=0x7fd8dc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=521c80eb52372166 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8e8ff9640 1 -- 172.21.15.5:0/1923763130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7fd8e0004ba0 con 0x7fd8ec101230 2024-09-19T06:04:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8d1ffb640 1 -- 172.21.15.5:0/468681546 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc004230 con 0x55ea4f1c7410 2024-09-19T06:04:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8d1ffb640 1 -- 172.21.15.5:0/468681546 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8dc0043d0 con 0x55ea4f1c7410 2024-09-19T06:04:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.896+0000 7fd8d1ffb640 1 -- 172.21.15.5:0/468681546 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc019850 con 0x55ea4f1c7410 2024-09-19T06:04:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.897+0000 7fd8e8ff9640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8d40a34d0 0x7fd8d40a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.897+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/468681546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c7410 msgr2=0x55ea4f1d1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.897+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/468681546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c7410 0x55ea4f1d1770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009cf0 tx=0x7fd8dc011810 comp rx=0 tx=0).stop 2024-09-19T06:04:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.897+0000 7fd8e8ff9640 1 -- 172.21.15.5:0/1923763130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fd8e00cfeb0 con 0x7fd8ec101230 2024-09-19T06:04:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.897+0000 7fd8ebfff640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8d40a34d0 0x7fd8d40a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.897+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/468681546 shutdown_connections 2024-09-19T06:04:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.897+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/468681546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1d40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.897+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/468681546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c7410 0x55ea4f1d1770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.897+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/468681546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.897+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/468681546 >> 172.21.15.5:0/468681546 conn(0x55ea4f1c3060 msgr2=0x55ea4f1c3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.897+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/468681546 shutdown_connections 2024-09-19T06:04:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.897+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/468681546 wait complete. 2024-09-19T06:04:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.898+0000 7fd8f4c8c740 1 -- start start 2024-09-19T06:04:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.898+0000 7fd8ebfff640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8d40a34d0 0x7fd8d40a5990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4004af0 tx=0x7fd8e4004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.898+0000 7fd8f4c8c740 1 Processor -- start 2024-09-19T06:04:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.898+0000 7fd8f4c8c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8f4c8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c7410 0x55ea4f1cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8f4c8c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8f4c8c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea4f1d4880 con 0x55ea4f1c6af0 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8f4c8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ea4f0b4290 con 0x55ea4f1c7410 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8f4c8c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ea4f1d4a00 con 0x55ea4f1d1cb0 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8f4c8c740 10 client.0 _ll_register_callbacks cb 0x55ea4f0e5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37228/0 (socket says 172.21.15.5:37228) 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/378202641 learned_addr learned my addr 172.21.15.5:0/378202641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8ebfff640 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8eaffd640 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c7410 0x55ea4f1cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/378202641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1d1cb0 msgr2=0x55ea4f1cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8eb7fe640 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/378202641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c7410 msgr2=0x55ea4f1cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8eb7fe640 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c7410 0x55ea4f1cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/378202641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea4ee3e210 con 0x55ea4f1c6af0 2024-09-19T06:04:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8eb7fe640 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1ceba0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc005270 tx=0x7fd8dc0052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9a9a6d6bb76ea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8caffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.899+0000 7fd8caffd640 1 -- 172.21.15.5:0/378202641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc027070 con 0x55ea4f1c6af0 2024-09-19T06:04:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8caffd640 1 -- 172.21.15.5:0/378202641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fd8dc0037f0 con 0x55ea4f1c6af0 2024-09-19T06:04:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/378202641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8dc0043d0 con 0x55ea4f1c6af0 2024-09-19T06:04:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8caffd640 1 -- 172.21.15.5:0/378202641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8dc033050 con 0x55ea4f1c6af0 2024-09-19T06:04:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8f4c8c740 10 client.15705 fetch_fsmap learned FSMap version 30 2024-09-19T06:04:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/378202641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ea4ef0a620 con 0x55ea4f1c6af0 2024-09-19T06:04:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8caffd640 1 -- 172.21.15.5:0/378202641 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8dc0043d0 con 0x55ea4f1c6af0 2024-09-19T06:04:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8caffd640 1 -- 172.21.15.5:0/378202641 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd8dc004b70 con 0x55ea4f1c6af0 2024-09-19T06:04:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8f4c8c740 10 client.15705 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:04:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8f4c8c740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8f4c8c740 20 client.15705 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea4f1e1b30 0x55ea4f1e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8f4c8c740 4 client.15705 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/378202641 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea4f138a20 con 0x55ea4f1e1b30 2024-09-19T06:04:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.900+0000 7fd8eaffd640 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea4f1e1b30 0x55ea4f1e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.901+0000 7fd8eaffd640 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea4f1e1b30 0x55ea4f1e3f20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a3314a2567e1d1 server_cookie=1cb00e3be22d306c in_seq=0 out_seq=0 2024-09-19T06:04:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.901+0000 7fd8caffd640 10 client.15705 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.901+0000 7fd8caffd640 1 -- 172.21.15.5:0/378202641 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea4f138a20 con 0x55ea4f1e1b30 2024-09-19T06:04:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.901+0000 7fd8caffd640 10 client.15705 handle_command_reply: tid=0 2024-09-19T06:04:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8f4c8c740 1 client.15705 shutdown 2024-09-19T06:04:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/378202641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c6af0 msgr2=0x55ea4f1ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1ceba0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc005270 tx=0x7fd8dc0052a0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/378202641 shutdown_connections 2024-09-19T06:04:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea4f1e1b30 0x55ea4f1e3f20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c7410 0x55ea4f1cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/378202641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1ceba0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/378202641 >> 172.21.15.5:0/378202641 conn(0x55ea4f1c3060 msgr2=0x55ea4f1c8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8caffd640 0 client.15705 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/378202641 shutdown_connections 2024-09-19T06:04:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/378202641 wait complete. 2024-09-19T06:04:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.902+0000 7fd8f4c8c740 20 client.15705 trim_cache size 0 max 16384 2024-09-19T06:04:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.922+0000 7fd8f4c8c740 1 Processor -- start 2024-09-19T06:04:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.922+0000 7fd8f4c8c740 1 -- start start 2024-09-19T06:04:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8f4c8c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8f4c8c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c7410 0x55ea4f1c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8f4c8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1e9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8f4c8c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1afa00 con 0x55ea4f1c7410 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8f4c8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1b15a0 con 0x55ea4f1d1cb0 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8f4c8c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1b00d0 con 0x55ea4f1c6af0 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8eb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8eb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33918/0 (socket says 172.21.15.5:33918) 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/1925448531 learned_addr learned my addr 172.21.15.5:0/1925448531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c7410 0x55ea4f1c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8ebfff640 1 --2- 172.21.15.5:0/1925448531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/1925448531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1d1cb0 msgr2=0x55ea4f1e9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8eb7fe640 1 --2- 172.21.15.5:0/1925448531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1e9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/1925448531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c7410 msgr2=0x55ea4f1c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8eb7fe640 1 --2- 172.21.15.5:0/1925448531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c7410 0x55ea4f1c20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/1925448531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea4f1c2e90 con 0x55ea4f1c6af0 2024-09-19T06:04:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.923+0000 7fd8eb7fe640 1 --2- 172.21.15.5:0/1925448531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1cf450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc004fd0 tx=0x7fd8dc0056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a5edda48ad891ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.924+0000 7fd8d17fa640 1 -- 172.21.15.5:0/1925448531 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc030070 con 0x55ea4f1c6af0 2024-09-19T06:04:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.924+0000 7fd8d17fa640 1 -- 172.21.15.5:0/1925448531 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8dc005310 con 0x55ea4f1c6af0 2024-09-19T06:04:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.924+0000 7fd8d17fa640 1 -- 172.21.15.5:0/1925448531 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc027030 con 0x55ea4f1c6af0 2024-09-19T06:04:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.924+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/1925448531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1c6af0 msgr2=0x55ea4f1cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.924+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/1925448531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1cf450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc004fd0 tx=0x7fd8dc0056d0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.924+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/1925448531 shutdown_connections 2024-09-19T06:04:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.924+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/1925448531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1e9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.924+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/1925448531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c7410 0x55ea4f1c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.924+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/1925448531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1cf450 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.924+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/1925448531 >> 172.21.15.5:0/1925448531 conn(0x55ea4f1c3060 msgr2=0x55ea4f1c19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.924+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/1925448531 shutdown_connections 2024-09-19T06:04:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.925+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/1925448531 wait complete. 2024-09-19T06:04:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.925+0000 7fd8f4c8c740 1 -- start start 2024-09-19T06:04:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.925+0000 7fd8f4c8c740 1 Processor -- start 2024-09-19T06:04:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.925+0000 7fd8f4c8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1d9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.925+0000 7fd8f4c8c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c7410 0x55ea4f1da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.925+0000 7fd8f4c8c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.925+0000 7fd8f4c8c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1b0e70 con 0x55ea4f1c7410 2024-09-19T06:04:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.925+0000 7fd8f4c8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1b07a0 con 0x55ea4f1c6af0 2024-09-19T06:04:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.925+0000 7fd8f4c8c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1ad9e0 con 0x55ea4f1d1cb0 2024-09-19T06:04:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.925+0000 7fd8f4c8c740 10 client.0 _ll_register_callbacks cb 0x7fd8e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8eb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1d9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c7410 0x55ea4f1da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8eb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1d9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:34042/0 (socket says 172.21.15.5:34042) 2024-09-19T06:04:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8eb7fe640 1 -- 172.21.15.5:0/220281345 learned_addr learned my addr 172.21.15.5:0/220281345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8ebfff640 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8eaffd640 1 -- 172.21.15.5:0/220281345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1d1cb0 msgr2=0x55ea4f1cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8eaffd640 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8eaffd640 1 -- 172.21.15.5:0/220281345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c6af0 msgr2=0x55ea4f1d9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8eaffd640 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1d9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8eaffd640 1 -- 172.21.15.5:0/220281345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea4f1be2d0 con 0x55ea4f1c7410 2024-09-19T06:04:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8ebfff640 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8eaffd640 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c7410 0x55ea4f1da3c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00cd490 tx=0x7fd8e00054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3126424ec8fe4d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8caffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8caffd640 1 -- 172.21.15.5:0/220281345 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e001c040 con 0x55ea4f1c7410 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8caffd640 1 -- 172.21.15.5:0/220281345 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fd8e00051d0 con 0x55ea4f1c7410 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/220281345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8dc005310 con 0x55ea4f1c7410 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.926+0000 7fd8caffd640 1 -- 172.21.15.5:0/220281345 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8e00a0300 con 0x55ea4f1c7410 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.927+0000 7fd8f4c8c740 10 client.15711 fetch_fsmap learned FSMap version 30 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.927+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/220281345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ea4f24bf70 con 0x55ea4f1c7410 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.927+0000 7fd8caffd640 1 -- 172.21.15.5:0/220281345 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8dc005310 con 0x55ea4f1c7410 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.927+0000 7fd8caffd640 1 -- 172.21.15.5:0/220281345 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd8e00b9020 con 0x55ea4f1c7410 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.927+0000 7fd8f4c8c740 10 client.15711 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.927+0000 7fd8f4c8c740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.927+0000 7fd8f4c8c740 20 client.15711 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.927+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea4f24c720 0x55ea4f24eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.927+0000 7fd8f4c8c740 4 client.15711 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.927+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/220281345 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea4f132d70 con 0x55ea4f24c720 2024-09-19T06:04:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.927+0000 7fd8eb7fe640 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea4f24c720 0x55ea4f24eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.928+0000 7fd8caffd640 10 client.15711 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.928+0000 7fd8eb7fe640 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea4f24c720 0x55ea4f24eb40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3acc32af7d2c4a2 server_cookie=4db96c64da125a85 in_seq=0 out_seq=0 2024-09-19T06:04:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.947+0000 7fd8caffd640 1 -- 172.21.15.5:0/220281345 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ea4f132d70 con 0x55ea4f24c720 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.947+0000 7fd8caffd640 10 client.15711 handle_command_reply: tid=0 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.947+0000 7fd8f4c8c740 1 client.15711 shutdown 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.947+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/220281345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c7410 msgr2=0x55ea4f1da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.947+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c7410 0x55ea4f1da3c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00cd490 tx=0x7fd8e00054d0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.947+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/220281345 shutdown_connections 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.947+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea4f24c720 0x55ea4f24eb40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.947+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4f1d1cb0 0x55ea4f1cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.947+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea4f1c7410 0x55ea4f1da3c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.947+0000 7fd8caffd640 0 client.15711 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.947+0000 7fd8f4c8c740 1 --2- 172.21.15.5:0/220281345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea4f1c6af0 0x55ea4f1d9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.947+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/220281345 >> 172.21.15.5:0/220281345 conn(0x55ea4f1c3060 msgr2=0x55ea4f1d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/220281345 shutdown_connections 2024-09-19T06:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f4c8c740 1 -- 172.21.15.5:0/220281345 wait complete. 2024-09-19T06:04:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.948+0000 7ff8abdd3640 1 -- 172.21.15.5:0/1629490912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8a410a0a0 con 0x7ff8a4100e20 2024-09-19T06:04:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f4c8c740 20 client.15711 trim_cache size 0 max 16384 2024-09-19T06:04:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:04:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/1923763130 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8d40a34d0 msgr2=0x7fd8d40a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f1ef1640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8d40a34d0 0x7fd8d40a5990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4004af0 tx=0x7fd8e4004a40 comp rx=0 tx=0).stop 2024-09-19T06:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/1923763130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8ec101230 msgr2=0x7fd8ec19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f1ef1640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8ec101230 0x7fd8ec19ab20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000e9f0 tx=0x7fd8e000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/1923763130 shutdown_connections 2024-09-19T06:04:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f1ef1640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8ec10e430 0x7fd8ec19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f1ef1640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8ec101230 0x7fd8ec19ab20 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f1ef1640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8d40a34d0 0x7fd8d40a5990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f1ef1640 1 --2- 172.21.15.5:0/1923763130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8ec100880 0x7fd8ec19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.948+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/1923763130 >> 172.21.15.5:0/1923763130 conn(0x7fd8ec0779a0 msgr2=0x7fd8ec0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.949+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/1923763130 shutdown_connections 2024-09-19T06:04:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:04:50.949+0000 7fd8f1ef1640 1 -- 172.21.15.5:0/1923763130 wait complete. 2024-09-19T06:04:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.949+0000 7ff89affd640 1 -- 172.21.15.5:0/1629490912 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11962 (secure 0 0 0) 0x7ff8a00a0640 con 0x7ff8a4100e20 2024-09-19T06:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-19T06:04:49:315680+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:04:49.014116+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:04:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-19T06:04:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.951+0000 7ff8abdd3640 1 -- 172.21.15.5:0/1629490912 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff8900a3550 msgr2=0x7ff8900a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.951+0000 7ff8abdd3640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff8900a3550 0x7ff8900a5a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4100030 tx=0x7ff894005f90 comp rx=0 tx=0).stop 2024-09-19T06:04:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.951+0000 7ff8abdd3640 1 -- 172.21.15.5:0/1629490912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4100e20 msgr2=0x7ff8a41983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.951+0000 7ff8abdd3640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4100e20 0x7ff8a41983b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000e9f0 tx=0x7ff8a000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.951+0000 7ff8abdd3640 1 -- 172.21.15.5:0/1629490912 shutdown_connections 2024-09-19T06:04:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.951+0000 7ff8abdd3640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff8900a3550 0x7ff8900a5a10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.951+0000 7ff8abdd3640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a4102ef0 0x7ff8a419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.951+0000 7ff8abdd3640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a4102570 0x7ff8a41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.951+0000 7ff8abdd3640 1 --2- 172.21.15.5:0/1629490912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4100e20 0x7ff8a41983b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:50.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.951+0000 7ff8abdd3640 1 -- 172.21.15.5:0/1629490912 >> 172.21.15.5:0/1629490912 conn(0x7ff8a40fc7d0 msgr2=0x7ff8a406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:50.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.951+0000 7ff8abdd3640 1 -- 172.21.15.5:0/1629490912 shutdown_connections 2024-09-19T06:04:50.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:50.951+0000 7ff8abdd3640 1 -- 172.21.15.5:0/1629490912 wait complete. 2024-09-19T06:04:51.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:51 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1629490912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:04:51.587 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:04:51.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:51 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1629490912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:04:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:51 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1629490912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:04:51.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 11217 4.61 MB/sec warmup 17 sec latency 11.715 ms 2024-09-19T06:04:51.869 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.369+0000 7f29a6eea640 1 -- 172.21.15.5:0/2542719063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a0101de0 msgr2=0x7f29a01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.369+0000 7f29a6eea640 1 --2- 172.21.15.5:0/2542719063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a0101de0 0x7f29a01021c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2990009ca0 tx=0x7f299002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 -- 172.21.15.5:0/2542719063 shutdown_connections 2024-09-19T06:04:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 --2- 172.21.15.5:0/2542719063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29a0106f10 0x7f29a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 --2- 172.21.15.5:0/2542719063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a0102790 0x7f29a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 --2- 172.21.15.5:0/2542719063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a0101de0 0x7f29a01021c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 -- 172.21.15.5:0/2542719063 >> 172.21.15.5:0/2542719063 conn(0x7f29a00fd8c0 msgr2=0x7f29a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 -- 172.21.15.5:0/2542719063 shutdown_connections 2024-09-19T06:04:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 -- 172.21.15.5:0/2542719063 wait complete. 2024-09-19T06:04:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 Processor -- start 2024-09-19T06:04:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 -- start start 2024-09-19T06:04:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a0101de0 0x7f29a01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29a0102790 0x7f29a0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a0106f10 0x7f29a019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0111e20 con 0x7f29a0101de0 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0111ca0 con 0x7f29a0102790 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.370+0000 7f29a6eea640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0111fa0 con 0x7f29a0106f10 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f29a4c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a0101de0 0x7f29a01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f29a4c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a0101de0 0x7f29a01982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37254/0 (socket says 172.21.15.5:37254) 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f29a4c86640 1 -- 172.21.15.5:0/1203301106 learned_addr learned my addr 172.21.15.5:0/1203301106 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f299ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29a0102790 0x7f29a0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f29a5487640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a0106f10 0x7f29a019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f29a4c86640 1 -- 172.21.15.5:0/1203301106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a0106f10 msgr2=0x7f29a019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f29a4c86640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a0106f10 0x7f29a019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f29a4c86640 1 -- 172.21.15.5:0/1203301106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29a0102790 msgr2=0x7f29a0198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f29a4c86640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29a0102790 0x7f29a0198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f29a4c86640 1 -- 172.21.15.5:0/1203301106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a019d290 con 0x7f29a0101de0 2024-09-19T06:04:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f299ffff640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29a0102790 0x7f29a0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f29a5487640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a0106f10 0x7f29a019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f29a4c86640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a0101de0 0x7f29a01982e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f299002ade0 tx=0x7f2990032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f299dffb640 1 -- 172.21.15.5:0/1203301106 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990003c70 con 0x7f29a0101de0 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f29a6eea640 1 -- 172.21.15.5:0/1203301106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a019d520 con 0x7f29a0101de0 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f299dffb640 1 -- 172.21.15.5:0/1203301106 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2990003e10 con 0x7f29a0101de0 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.371+0000 7f299dffb640 1 -- 172.21.15.5:0/1203301106 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990030c30 con 0x7f29a0101de0 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.372+0000 7f29a6eea640 1 -- 172.21.15.5:0/1203301106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a01a4e60 con 0x7f29a0101de0 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.374+0000 7f299dffb640 1 -- 172.21.15.5:0/1203301106 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138587+0+0 (secure 0 0 0) 0x7f2990004040 con 0x7f29a0101de0 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.374+0000 7f29a92c3740 1 Processor -- start 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.374+0000 7f29a92c3740 1 -- start start 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f29a92c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d0cf50 0x55fb88d0b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f29a92c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d0be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f29a92c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 0x55fb88d18b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f29a92c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fb88c6d2f0 con 0x55fb88d16780 2024-09-19T06:04:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f29a92c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fb88c14cf0 con 0x55fb88d0cf50 2024-09-19T06:04:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f29a92c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fb88c692b0 con 0x55fb88d0b9e0 2024-09-19T06:04:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f299ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 0x55fb88d18b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f299ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 0x55fb88d18b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37260/0 (socket says 172.21.15.5:37260) 2024-09-19T06:04:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f299ffff640 1 -- 172.21.15.5:0/1565006171 learned_addr learned my addr 172.21.15.5:0/1565006171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f299dffb640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f298c0a34d0 0x7f298c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f29a5487640 1 --2- 172.21.15.5:0/1565006171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d0cf50 0x55fb88d0b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f29a4c86640 1 --2- 172.21.15.5:0/1565006171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d0be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f29a5487640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f298c0a34d0 0x7f298c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f299ffff640 1 -- 172.21.15.5:0/1565006171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 msgr2=0x55fb88d0be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f299ffff640 1 --2- 172.21.15.5:0/1565006171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d0be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f299ffff640 1 -- 172.21.15.5:0/1565006171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d0cf50 msgr2=0x55fb88d0b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f299ffff640 1 --2- 172.21.15.5:0/1565006171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d0cf50 0x55fb88d0b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f299dffb640 1 -- 172.21.15.5:0/1203301106 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f29900f1370 con 0x7f29a0101de0 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f299ffff640 1 -- 172.21.15.5:0/1565006171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb88bf93a0 con 0x55fb88d16780 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f29a5487640 1 --2- 172.21.15.5:0/1565006171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d0cf50 0x55fb88d0b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.375+0000 7f299ffff640 1 --2- 172.21.15.5:0/1565006171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 0x55fb88d18b70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f29880114f0 tx=0x7f29880119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43113fb336c3f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f2996ffd640 1 -- 172.21.15.5:0/1565006171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988004020 con 0x55fb88d16780 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f29a5487640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f298c0a34d0 0x7f298c0a5990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2998005dd0 tx=0x7f2998005d60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f2996ffd640 1 -- 172.21.15.5:0/1565006171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29880041c0 con 0x55fb88d16780 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f2996ffd640 1 -- 172.21.15.5:0/1565006171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298800f800 con 0x55fb88d16780 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f29a92c3740 1 -- 172.21.15.5:0/1565006171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 msgr2=0x55fb88d18b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f29a92c3740 1 --2- 172.21.15.5:0/1565006171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 0x55fb88d18b70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f29880114f0 tx=0x7f29880119b0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f29a92c3740 1 -- 172.21.15.5:0/1565006171 shutdown_connections 2024-09-19T06:04:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f29a92c3740 1 --2- 172.21.15.5:0/1565006171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 0x55fb88d18b70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f29a92c3740 1 --2- 172.21.15.5:0/1565006171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d0be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f29a92c3740 1 --2- 172.21.15.5:0/1565006171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d0cf50 0x55fb88d0b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f29a92c3740 1 -- 172.21.15.5:0/1565006171 >> 172.21.15.5:0/1565006171 conn(0x55fb88cd0310 msgr2=0x55fb88d08070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f29a92c3740 1 -- 172.21.15.5:0/1565006171 shutdown_connections 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.376+0000 7f29a92c3740 1 -- 172.21.15.5:0/1565006171 wait complete. 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f29a92c3740 1 -- start start 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f29a92c3740 1 Processor -- start 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f29a92c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d15d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f29a92c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0cf50 0x55fb88d0f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f29a92c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d16780 0x55fb88d0f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f29a92c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fb88d19480 con 0x55fb88d0cf50 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f29a92c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fb88c28060 con 0x55fb88d16780 2024-09-19T06:04:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f29a92c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fb88d19600 con 0x55fb88d0b9e0 2024-09-19T06:04:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f29a92c3740 10 client.0 _ll_register_callbacks cb 0x55fb8897b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f299ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0cf50 0x55fb88d0f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f299ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0cf50 0x55fb88d0f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37270/0 (socket says 172.21.15.5:37270) 2024-09-19T06:04:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f29a4c86640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d15d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.377+0000 7f299ffff640 1 -- 172.21.15.5:0/3263352767 learned_addr learned my addr 172.21.15.5:0/3263352767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f29a5487640 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d16780 0x55fb88d0f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f29a4c86640 1 -- 172.21.15.5:0/3263352767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d16780 msgr2=0x55fb88d0f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f29a4c86640 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d16780 0x55fb88d0f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f29a4c86640 1 -- 172.21.15.5:0/3263352767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0cf50 msgr2=0x55fb88d0f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f29a4c86640 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0cf50 0x55fb88d0f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f29a4c86640 1 -- 172.21.15.5:0/3263352767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb889de3c0 con 0x55fb88d0b9e0 2024-09-19T06:04:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f299ffff640 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0cf50 0x55fb88d0f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f29a4c86640 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d15d70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29900eea60 tx=0x7f2990088390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b491923eaff601b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f297ffff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:04:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f297ffff640 1 -- 172.21.15.5:0/3263352767 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900305c0 con 0x55fb88d0b9e0 2024-09-19T06:04:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f29a92c3740 1 -- 172.21.15.5:0/3263352767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29880041c0 con 0x55fb88d0b9e0 2024-09-19T06:04:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f297ffff640 1 -- 172.21.15.5:0/3263352767 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f29900b9c20 con 0x55fb88d0b9e0 2024-09-19T06:04:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f297ffff640 1 -- 172.21.15.5:0/3263352767 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f299008c0d0 con 0x55fb88d0b9e0 2024-09-19T06:04:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f29a92c3740 10 client.25267 fetch_fsmap learned FSMap version 30 2024-09-19T06:04:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.378+0000 7f29a92c3740 1 -- 172.21.15.5:0/3263352767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb88d19480 con 0x55fb88d0b9e0 2024-09-19T06:04:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.379+0000 7f297ffff640 1 -- 172.21.15.5:0/3263352767 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29880041c0 con 0x55fb88d0b9e0 2024-09-19T06:04:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.379+0000 7f297ffff640 1 -- 172.21.15.5:0/3263352767 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2990087f60 con 0x55fb88d0b9e0 2024-09-19T06:04:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.379+0000 7f29a92c3740 10 client.25267 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:04:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.379+0000 7f29a92c3740 10 client.25267 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:04:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.379+0000 7f29a92c3740 20 client.25267 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.379+0000 7f29a92c3740 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55fb88d2a700 0x55fb88d2cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.379+0000 7f29a92c3740 4 client.25267 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.379+0000 7f29a92c3740 1 -- 172.21.15.5:0/3263352767 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb88c7a860 con 0x55fb88d2a700 2024-09-19T06:04:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.379+0000 7f299ffff640 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55fb88d2a700 0x55fb88d2cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.379+0000 7f297ffff640 10 client.25267 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.379+0000 7f299ffff640 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55fb88d2a700 0x55fb88d2cb50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e2d0953f4cae1c00 server_cookie=6705ab91ae70be7a in_seq=0 out_seq=0 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f297ffff640 1 -- 172.21.15.5:0/3263352767 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb88c7a860 con 0x55fb88d2a700 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f297ffff640 10 client.25267 handle_command_reply: tid=0 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f29a92c3740 1 client.25267 shutdown 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f29a92c3740 1 -- 172.21.15.5:0/3263352767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 msgr2=0x55fb88d15d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f29a92c3740 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d15d70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29900eea60 tx=0x7f2990088390 comp rx=0 tx=0).stop 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f29a92c3740 1 -- 172.21.15.5:0/3263352767 shutdown_connections 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f29a92c3740 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55fb88d2a700 0x55fb88d2cb50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f29a92c3740 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d16780 0x55fb88d0f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f29a92c3740 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0cf50 0x55fb88d0f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f29a92c3740 1 --2- 172.21.15.5:0/3263352767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d15d70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f297ffff640 0 client.25267 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:04:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f29a92c3740 1 -- 172.21.15.5:0/3263352767 >> 172.21.15.5:0/3263352767 conn(0x55fb88cd0310 msgr2=0x55fb88d0dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:52.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f29a92c3740 1 -- 172.21.15.5:0/3263352767 shutdown_connections 2024-09-19T06:04:52.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.380+0000 7f29a92c3740 1 -- 172.21.15.5:0/3263352767 wait complete. 2024-09-19T06:04:52.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.381+0000 7f29a92c3740 20 client.25267 trim_cache size 0 max 16384 2024-09-19T06:04:52.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.393+0000 7f29a92c3740 1 Processor -- start 2024-09-19T06:04:52.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.393+0000 7f29a92c3740 1 -- start start 2024-09-19T06:04:52.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a92c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d08220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a92c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d0cf50 0x55fb88d32430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a92c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 0x55fb88d32970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a92c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01ab4b0 con 0x55fb88d16780 2024-09-19T06:04:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a92c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01ad050 con 0x55fb88d0cf50 2024-09-19T06:04:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a92c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01abb80 con 0x55fb88d0b9e0 2024-09-19T06:04:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a5487640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 0x55fb88d32970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a5487640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 0x55fb88d32970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37284/0 (socket says 172.21.15.5:37284) 2024-09-19T06:04:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a5487640 1 -- 172.21.15.5:0/1312130474 learned_addr learned my addr 172.21.15.5:0/1312130474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f299ffff640 1 --2- 172.21.15.5:0/1312130474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d0cf50 0x55fb88d32430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a4c86640 1 --2- 172.21.15.5:0/1312130474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d08220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a5487640 1 -- 172.21.15.5:0/1312130474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 msgr2=0x55fb88d08220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a5487640 1 --2- 172.21.15.5:0/1312130474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d08220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a5487640 1 -- 172.21.15.5:0/1312130474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d0cf50 msgr2=0x55fb88d32430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a5487640 1 --2- 172.21.15.5:0/1312130474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d0cf50 0x55fb88d32430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a5487640 1 -- 172.21.15.5:0/1312130474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb88d158b0 con 0x55fb88d16780 2024-09-19T06:04:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29a5487640 1 --2- 172.21.15.5:0/1312130474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 0x55fb88d32970 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f299801e750 tx=0x7f299800e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d7d759119a93e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.394+0000 7f29967fc640 1 -- 172.21.15.5:0/1312130474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29980108b0 con 0x55fb88d16780 2024-09-19T06:04:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.395+0000 7f29967fc640 1 -- 172.21.15.5:0/1312130474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29980283f0 con 0x55fb88d16780 2024-09-19T06:04:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.395+0000 7f29967fc640 1 -- 172.21.15.5:0/1312130474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299800f620 con 0x55fb88d16780 2024-09-19T06:04:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.395+0000 7f29a92c3740 1 -- 172.21.15.5:0/1312130474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 msgr2=0x55fb88d32970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.395+0000 7f29a92c3740 1 --2- 172.21.15.5:0/1312130474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 0x55fb88d32970 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f299801e750 tx=0x7f299800e430 comp rx=0 tx=0).stop 2024-09-19T06:04:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.395+0000 7f29a92c3740 1 -- 172.21.15.5:0/1312130474 shutdown_connections 2024-09-19T06:04:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.395+0000 7f29a92c3740 1 --2- 172.21.15.5:0/1312130474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d16780 0x55fb88d32970 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.395+0000 7f29a92c3740 1 --2- 172.21.15.5:0/1312130474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d0cf50 0x55fb88d32430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.395+0000 7f29a92c3740 1 --2- 172.21.15.5:0/1312130474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0b9e0 0x55fb88d08220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.395+0000 7f29a92c3740 1 -- 172.21.15.5:0/1312130474 >> 172.21.15.5:0/1312130474 conn(0x55fb88cd0310 msgr2=0x55fb88d07b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:52.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.395+0000 7f29a92c3740 1 -- 172.21.15.5:0/1312130474 shutdown_connections 2024-09-19T06:04:52.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.395+0000 7f29a92c3740 1 -- 172.21.15.5:0/1312130474 wait complete. 2024-09-19T06:04:52.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a92c3740 1 -- start start 2024-09-19T06:04:52.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a92c3740 1 Processor -- start 2024-09-19T06:04:52.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a92c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0b9e0 0x55fb88d22ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a92c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0cf50 0x55fb88d23020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a92c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d16780 0x55fb88d27050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a92c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01ac920 con 0x55fb88d0b9e0 2024-09-19T06:04:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a92c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01ac250 con 0x55fb88d16780 2024-09-19T06:04:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a92c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01a9490 con 0x55fb88d0cf50 2024-09-19T06:04:52.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a92c3740 10 client.0 _ll_register_callbacks cb 0x7f2990009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:52.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a4c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0b9e0 0x55fb88d22ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a4c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0b9e0 0x55fb88d22ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37296/0 (socket says 172.21.15.5:37296) 2024-09-19T06:04:52.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a4c86640 1 -- 172.21.15.5:0/600123788 learned_addr learned my addr 172.21.15.5:0/600123788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:52.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f299ffff640 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0cf50 0x55fb88d23020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.396+0000 7f29a5487640 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d16780 0x55fb88d27050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f29a4c86640 1 -- 172.21.15.5:0/600123788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0cf50 msgr2=0x55fb88d23020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:52.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f29a4c86640 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0cf50 0x55fb88d23020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f29a4c86640 1 -- 172.21.15.5:0/600123788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d16780 msgr2=0x55fb88d27050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:52.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f29a4c86640 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d16780 0x55fb88d27050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f29a4c86640 1 -- 172.21.15.5:0/600123788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb88d27690 con 0x55fb88d0b9e0 2024-09-19T06:04:52.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f299ffff640 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0cf50 0x55fb88d23020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:52.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f29a5487640 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d16780 0x55fb88d27050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:52.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f29a4c86640 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0b9e0 0x55fb88d22ae0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f29900bd2b0 tx=0x7f29900bd0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde4d0bc9af82fac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:52.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f297ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:52.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f297ffff640 1 -- 172.21.15.5:0/600123788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900305c0 con 0x55fb88d0b9e0 2024-09-19T06:04:52.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f297ffff640 1 -- 172.21.15.5:0/600123788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f29900eec30 con 0x55fb88d0b9e0 2024-09-19T06:04:52.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f29a92c3740 1 -- 172.21.15.5:0/600123788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29980283f0 con 0x55fb88d0b9e0 2024-09-19T06:04:52.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f297ffff640 1 -- 172.21.15.5:0/600123788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2990088400 con 0x55fb88d0b9e0 2024-09-19T06:04:52.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f29a92c3740 10 client.15729 fetch_fsmap learned FSMap version 30 2024-09-19T06:04:52.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f29a92c3740 1 -- 172.21.15.5:0/600123788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb88d94eb0 con 0x55fb88d0b9e0 2024-09-19T06:04:52.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.397+0000 7f297ffff640 1 -- 172.21.15.5:0/600123788 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29980283f0 con 0x55fb88d0b9e0 2024-09-19T06:04:52.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f297ffff640 1 -- 172.21.15.5:0/600123788 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2990004330 con 0x55fb88d0b9e0 2024-09-19T06:04:52.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f29a92c3740 10 client.15729 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:04:52.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f29a92c3740 10 client.15729 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:04:52.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f29a92c3740 20 client.15729 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f29a92c3740 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55fb88d954a0 0x55fb88d978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f29a92c3740 4 client.15729 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:04:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f29a92c3740 1 -- 172.21.15.5:0/600123788 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb88c74d70 con 0x55fb88d954a0 2024-09-19T06:04:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f299ffff640 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55fb88d954a0 0x55fb88d978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f299ffff640 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55fb88d954a0 0x55fb88d978c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aec7abfb32bbe806 server_cookie=17b9f65c84d2e63a in_seq=0 out_seq=0 2024-09-19T06:04:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f297ffff640 10 client.15729 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:04:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f297ffff640 1 -- 172.21.15.5:0/600123788 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb88c74d70 con 0x55fb88d954a0 2024-09-19T06:04:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f297ffff640 10 client.15729 handle_command_reply: tid=0 2024-09-19T06:04:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.398+0000 7f29a92c3740 1 client.15729 shutdown 2024-09-19T06:04:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.399+0000 7f29a92c3740 1 -- 172.21.15.5:0/600123788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0b9e0 msgr2=0x55fb88d22ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.399+0000 7f29a92c3740 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0b9e0 0x55fb88d22ae0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f29900bd2b0 tx=0x7f29900bd0f0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.399+0000 7f29a92c3740 1 -- 172.21.15.5:0/600123788 shutdown_connections 2024-09-19T06:04:52.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.399+0000 7f29a92c3740 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55fb88d954a0 0x55fb88d978c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.399+0000 7f29a92c3740 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb88d16780 0x55fb88d27050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.399+0000 7f297ffff640 0 client.15729 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:04:52.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.399+0000 7f29a92c3740 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb88d0cf50 0x55fb88d23020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.399+0000 7f29a92c3740 1 --2- 172.21.15.5:0/600123788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb88d0b9e0 0x55fb88d22ae0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.399+0000 7f29a92c3740 1 -- 172.21.15.5:0/600123788 >> 172.21.15.5:0/600123788 conn(0x55fb88cd0310 msgr2=0x55fb88d12870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:52.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.399+0000 7f29a92c3740 1 -- 172.21.15.5:0/600123788 shutdown_connections 2024-09-19T06:04:52.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.399+0000 7f29a92c3740 1 -- 172.21.15.5:0/600123788 wait complete. 2024-09-19T06:04:52.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.399+0000 7f29a92c3740 20 client.15729 trim_cache size 0 max 16384 2024-09-19T06:04:52.415 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:04:52.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.400+0000 7f29a6eea640 1 -- 172.21.15.5:0/1203301106 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f298c0a34d0 msgr2=0x7f298c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:52.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.400+0000 7f29a6eea640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f298c0a34d0 0x7f298c0a5990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2998005dd0 tx=0x7f2998005d60 comp rx=0 tx=0).stop 2024-09-19T06:04:52.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.400+0000 7f29a6eea640 1 -- 172.21.15.5:0/1203301106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a0101de0 msgr2=0x7f29a01982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:52.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.400+0000 7f29a6eea640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a0101de0 0x7f29a01982e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f299002ade0 tx=0x7f2990032ff0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.400+0000 7f29a6eea640 1 -- 172.21.15.5:0/1203301106 shutdown_connections 2024-09-19T06:04:52.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.400+0000 7f29a6eea640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f298c0a34d0 0x7f298c0a5990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.400+0000 7f29a6eea640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a0106f10 0x7f29a019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.400+0000 7f29a6eea640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29a0102790 0x7f29a0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.400+0000 7f29a6eea640 1 --2- 172.21.15.5:0/1203301106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a0101de0 0x7f29a01982e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:52.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.400+0000 7f29a6eea640 1 -- 172.21.15.5:0/1203301106 >> 172.21.15.5:0/1203301106 conn(0x7f29a00fd8c0 msgr2=0x7f29a00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:52.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.400+0000 7f29a6eea640 1 -- 172.21.15.5:0/1203301106 shutdown_connections 2024-09-19T06:04:52.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:52.400+0000 7f29a6eea640 1 -- 172.21.15.5:0/1203301106 wait complete. 2024-09-19T06:04:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:52 smithi005 ceph-mon[30955]: pgmap v48: 97 pgs: 97 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-09-19T06:04:52.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:52 smithi049 ceph-mon[34640]: pgmap v48: 97 pgs: 97 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-09-19T06:04:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:52 smithi157 ceph-mon[35346]: pgmap v48: 97 pgs: 97 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-09-19T06:04:52.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 11400 4.37 MB/sec warmup 18 sec latency 536.953 ms 2024-09-19T06:04:53.078 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:04:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:53.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 11400 4.14 MB/sec warmup 19 sec latency 1536.987 ms 2024-09-19T06:04:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.844+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/4255036510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a4101de0 msgr2=0x7fd0a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.844+0000 7fd0ab5cf640 1 --2- 172.21.15.5:0/4255036510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a4101de0 0x7fd0a41021c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009900 tx=0x7fd0a002a410 comp rx=0 tx=0).stop 2024-09-19T06:04:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.844+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/4255036510 shutdown_connections 2024-09-19T06:04:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.844+0000 7fd0ab5cf640 1 --2- 172.21.15.5:0/4255036510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0a4106f10 0x7fd0a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.844+0000 7fd0ab5cf640 1 --2- 172.21.15.5:0/4255036510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a4102790 0x7fd0a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.844+0000 7fd0ab5cf640 1 --2- 172.21.15.5:0/4255036510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a4101de0 0x7fd0a41021c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.844+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/4255036510 >> 172.21.15.5:0/4255036510 conn(0x7fd0a40fd8c0 msgr2=0x7fd0a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.844+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/4255036510 shutdown_connections 2024-09-19T06:04:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.844+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/4255036510 wait complete. 2024-09-19T06:04:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.845+0000 7fd0ab5cf640 1 Processor -- start 2024-09-19T06:04:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.845+0000 7fd0ab5cf640 1 -- start start 2024-09-19T06:04:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.845+0000 7fd0ab5cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a4101de0 0x7fd0a4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.845+0000 7fd0ab5cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a4102790 0x7fd0a4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.845+0000 7fd0ab5cf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0a4106f10 0x7fd0a419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.845+0000 7fd0ab5cf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4111e40 con 0x7fd0a4102790 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.845+0000 7fd0ab5cf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4111cc0 con 0x7fd0a4106f10 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.845+0000 7fd0ab5cf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4111fc0 con 0x7fd0a4101de0 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.845+0000 7fd0a8b6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a4102790 0x7fd0a4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.845+0000 7fd0a8b6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a4102790 0x7fd0a4198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37310/0 (socket says 172.21.15.5:37310) 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd0a8b6a640 1 -- 172.21.15.5:0/3771195810 learned_addr learned my addr 172.21.15.5:0/3771195810 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd0a936b640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a4101de0 0x7fd0a4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd0a9b6c640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0a4106f10 0x7fd0a419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd0a8b6a640 1 -- 172.21.15.5:0/3771195810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a4101de0 msgr2=0x7fd0a4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd0a8b6a640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a4101de0 0x7fd0a4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd0a8b6a640 1 -- 172.21.15.5:0/3771195810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0a4106f10 msgr2=0x7fd0a419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd0a8b6a640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0a4106f10 0x7fd0a419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd0a8b6a640 1 -- 172.21.15.5:0/3771195810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a419d3b0 con 0x7fd0a4102790 2024-09-19T06:04:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd0a8b6a640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a4102790 0x7fd0a4198940 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd09400b4a0 tx=0x7fd09400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd09a7fc640 1 -- 172.21.15.5:0/3771195810 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094014070 con 0x7fd0a4102790 2024-09-19T06:04:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/3771195810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a419d6a0 con 0x7fd0a4102790 2024-09-19T06:04:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd09a7fc640 1 -- 172.21.15.5:0/3771195810 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd094010ac0 con 0x7fd0a4102790 2024-09-19T06:04:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd09a7fc640 1 -- 172.21.15.5:0/3771195810 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094017c30 con 0x7fd0a4102790 2024-09-19T06:04:53.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.846+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/3771195810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a41a4f80 con 0x7fd0a4102790 2024-09-19T06:04:53.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.849+0000 7fd09a7fc640 1 -- 172.21.15.5:0/3771195810 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd094017070 con 0x7fd0a4102790 2024-09-19T06:04:53.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.849+0000 7fd0ad9a8740 1 Processor -- start 2024-09-19T06:04:53.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.849+0000 7fd0ad9a8740 1 -- start start 2024-09-19T06:04:53.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0ad9a8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d02b40 0x555be2d01090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0ad9a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d015d0 0x555be2d01a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0ad9a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d0c370 0x555be2d0e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0ad9a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555be2c672f0 con 0x555be2d015d0 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0ad9a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555be2c0ecf0 con 0x555be2d0c370 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0ad9a8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555be2c632b0 con 0x555be2d02b40 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd09a7fc640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0900a33b0 0x7fd0900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0a936b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d015d0 0x555be2d01a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0a936b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d015d0 0x555be2d01a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37324/0 (socket says 172.21.15.5:37324) 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0a936b640 1 -- 172.21.15.5:0/2716498265 learned_addr learned my addr 172.21.15.5:0/2716498265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0a9b6c640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0900a33b0 0x7fd0900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd09a7fc640 1 -- 172.21.15.5:0/3771195810 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fd0940cf330 con 0x7fd0a4102790 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0a9b6c640 1 --2- 172.21.15.5:0/2716498265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d02b40 0x555be2d01090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0a8b6a640 1 --2- 172.21.15.5:0/2716498265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d0c370 0x555be2d0e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0a936b640 1 -- 172.21.15.5:0/2716498265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d02b40 msgr2=0x555be2d01090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0a936b640 1 --2- 172.21.15.5:0/2716498265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d02b40 0x555be2d01090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.850+0000 7fd0a936b640 1 -- 172.21.15.5:0/2716498265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d0c370 msgr2=0x555be2d0e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:53.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.851+0000 7fd0a936b640 1 --2- 172.21.15.5:0/2716498265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d0c370 0x555be2d0e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.851+0000 7fd0a936b640 1 -- 172.21.15.5:0/2716498265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be2bf33a0 con 0x555be2d015d0 2024-09-19T06:04:53.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.851+0000 7fd0a8b6a640 1 --2- 172.21.15.5:0/2716498265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d0c370 0x555be2d0e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:53.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.851+0000 7fd0a9b6c640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0900a33b0 0x7fd0900a5870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd09c005c40 tx=0x7fd09c00a3c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:53.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.851+0000 7fd0a936b640 1 --2- 172.21.15.5:0/2716498265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d015d0 0x555be2d01a10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009900 tx=0x7fd0a0031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2435c42ffb1b869 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:53.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.851+0000 7fd08f7fe640 1 -- 172.21.15.5:0/2716498265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0040070 con 0x555be2d015d0 2024-09-19T06:04:53.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.851+0000 7fd08f7fe640 1 -- 172.21.15.5:0/2716498265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a00316a0 con 0x555be2d015d0 2024-09-19T06:04:53.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.851+0000 7fd08f7fe640 1 -- 172.21.15.5:0/2716498265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0035040 con 0x555be2d015d0 2024-09-19T06:04:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.852+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2716498265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d015d0 msgr2=0x555be2d01a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.852+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2716498265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d015d0 0x555be2d01a10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009900 tx=0x7fd0a0031dc0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.852+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2716498265 shutdown_connections 2024-09-19T06:04:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.852+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2716498265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d0c370 0x555be2d0e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.852+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2716498265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d015d0 0x555be2d01a10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.852+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2716498265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d02b40 0x555be2d01090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.852+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2716498265 >> 172.21.15.5:0/2716498265 conn(0x555be2cca310 msgr2=0x555be2cfdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.852+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2716498265 shutdown_connections 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.852+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2716498265 wait complete. 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.852+0000 7fd0ad9a8740 1 -- start start 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.852+0000 7fd0ad9a8740 1 Processor -- start 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0ad9a8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d015d0 0x555be2d0bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0ad9a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d02b40 0x555be2d08fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0ad9a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 0x555be2d095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0ad9a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555be2d0f010 con 0x555be2d0c370 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0ad9a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555be2c22060 con 0x555be2d02b40 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0ad9a8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555be2d0f190 con 0x555be2d015d0 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0ad9a8740 10 client.0 _ll_register_callbacks cb 0x555be29758c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0a9b6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 0x555be2d095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0a8b6a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d02b40 0x555be2d08fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0a936b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d015d0 0x555be2d0bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0a9b6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 0x555be2d095d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37332/0 (socket says 172.21.15.5:37332) 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0a9b6c640 1 -- 172.21.15.5:0/1027219809 learned_addr learned my addr 172.21.15.5:0/1027219809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0a9b6c640 1 -- 172.21.15.5:0/1027219809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d015d0 msgr2=0x555be2d0bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0a9b6c640 1 --2- 172.21.15.5:0/1027219809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d015d0 0x555be2d0bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0a9b6c640 1 -- 172.21.15.5:0/1027219809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d02b40 msgr2=0x555be2d08fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0a9b6c640 1 --2- 172.21.15.5:0/1027219809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d02b40 0x555be2d08fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0a9b6c640 1 -- 172.21.15.5:0/1027219809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be29d83c0 con 0x555be2d0c370 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd0a9b6c640 1 --2- 172.21.15.5:0/1027219809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 0x555be2d095d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00ea70 tx=0x7fd09c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264ece92667d16cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.853+0000 7fd08cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd08cff9640 1 -- 172.21.15.5:0/1027219809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c028480 con 0x555be2d0c370 2024-09-19T06:04:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd08cff9640 1 -- 172.21.15.5:0/1027219809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fd09c010430 con 0x555be2d0c370 2024-09-19T06:04:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/1027219809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a00316a0 con 0x555be2d0c370 2024-09-19T06:04:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd08cff9640 1 -- 172.21.15.5:0/1027219809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd09c010630 con 0x555be2d0c370 2024-09-19T06:04:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd0ad9a8740 10 client.15741 fetch_fsmap learned FSMap version 30 2024-09-19T06:04:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/1027219809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555be2d1aad0 con 0x555be2d0c370 2024-09-19T06:04:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd08cff9640 1 -- 172.21.15.5:0/1027219809 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a00316a0 con 0x555be2d0c370 2024-09-19T06:04:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd08cff9640 1 -- 172.21.15.5:0/1027219809 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd09c0289f0 con 0x555be2d0c370 2024-09-19T06:04:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd0ad9a8740 10 client.15741 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:04:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd0ad9a8740 10 client.15741 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:04:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd0ad9a8740 20 client.15741 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/1027219809 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x555be2d1bee0 0x555be2d1e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd0ad9a8740 4 client.15741 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.854+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/1027219809 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555be2c74860 con 0x555be2d1bee0 2024-09-19T06:04:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.855+0000 7fd0a936b640 1 --2- 172.21.15.5:0/1027219809 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x555be2d1bee0 0x555be2d1e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.855+0000 7fd0a936b640 1 --2- 172.21.15.5:0/1027219809 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x555be2d1bee0 0x555be2d1e2f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b9b49df71b11713 server_cookie=40c7e543841b190f in_seq=0 out_seq=0 2024-09-19T06:04:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.855+0000 7fd08cff9640 10 client.15741 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:04:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd08cff9640 1 -- 172.21.15.5:0/1027219809 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555be2c74860 con 0x555be2d1bee0 2024-09-19T06:04:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd08cff9640 10 client.15741 handle_command_reply: tid=0 2024-09-19T06:04:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd0ad9a8740 1 client.15741 shutdown 2024-09-19T06:04:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/1027219809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 msgr2=0x555be2d095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/1027219809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 0x555be2d095d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00ea70 tx=0x7fd09c004ab0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/1027219809 shutdown_connections 2024-09-19T06:04:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/1027219809 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x555be2d1bee0 0x555be2d1e2f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/1027219809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 0x555be2d095d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/1027219809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d02b40 0x555be2d08fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/1027219809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d015d0 0x555be2d0bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd08cff9640 0 client.15741 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:04:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/1027219809 >> 172.21.15.5:0/1027219809 conn(0x555be2cca310 msgr2=0x555be2d03960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/1027219809 shutdown_connections 2024-09-19T06:04:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/1027219809 wait complete. 2024-09-19T06:04:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.856+0000 7fd0ad9a8740 20 client.15741 trim_cache size 0 max 16384 2024-09-19T06:04:53.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.869+0000 7fd0ad9a8740 1 Processor -- start 2024-09-19T06:04:53.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.869+0000 7fd0ad9a8740 1 -- start start 2024-09-19T06:04:53.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0ad9a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d015d0 0x555be2d182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0ad9a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d02b40 0x555be2d0d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0ad9a8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d0c370 0x555be2d0dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0ad9a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41ad8c0 con 0x555be2d02b40 2024-09-19T06:04:53.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0ad9a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41af460 con 0x555be2d015d0 2024-09-19T06:04:53.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0ad9a8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41adf90 con 0x555be2d0c370 2024-09-19T06:04:53.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0a8b6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d02b40 0x555be2d0d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0a8b6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d02b40 0x555be2d0d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37348/0 (socket says 172.21.15.5:37348) 2024-09-19T06:04:53.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0a8b6a640 1 -- 172.21.15.5:0/2648609382 learned_addr learned my addr 172.21.15.5:0/2648609382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:53.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0a936b640 1 --2- 172.21.15.5:0/2648609382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d015d0 0x555be2d182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0a9b6c640 1 --2- 172.21.15.5:0/2648609382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d0c370 0x555be2d0dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0a8b6a640 1 -- 172.21.15.5:0/2648609382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d0c370 msgr2=0x555be2d0dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:53.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0a8b6a640 1 --2- 172.21.15.5:0/2648609382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d0c370 0x555be2d0dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0a8b6a640 1 -- 172.21.15.5:0/2648609382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d015d0 msgr2=0x555be2d182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:53.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0a8b6a640 1 --2- 172.21.15.5:0/2648609382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d015d0 0x555be2d182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0a8b6a640 1 -- 172.21.15.5:0/2648609382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be2d07530 con 0x555be2d02b40 2024-09-19T06:04:53.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd0a8b6a640 1 --2- 172.21.15.5:0/2648609382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d02b40 0x555be2d0d760 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd094000c00 tx=0x7fd09401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a83a8a1a95a1217 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd08effd640 1 -- 172.21.15.5:0/2648609382 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09409fa80 con 0x555be2d02b40 2024-09-19T06:04:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd08effd640 1 -- 172.21.15.5:0/2648609382 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0940ce050 con 0x555be2d02b40 2024-09-19T06:04:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.870+0000 7fd08effd640 1 -- 172.21.15.5:0/2648609382 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09400ce20 con 0x555be2d02b40 2024-09-19T06:04:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.871+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2648609382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d02b40 msgr2=0x555be2d0d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.871+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2648609382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d02b40 0x555be2d0d760 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd094000c00 tx=0x7fd09401d360 comp rx=0 tx=0).stop 2024-09-19T06:04:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.871+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2648609382 shutdown_connections 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.871+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2648609382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d0c370 0x555be2d0dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.871+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2648609382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d02b40 0x555be2d0d760 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.871+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2648609382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d015d0 0x555be2d182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.871+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2648609382 >> 172.21.15.5:0/2648609382 conn(0x555be2cca310 msgr2=0x555be2d08fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.871+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2648609382 shutdown_connections 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.871+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2648609382 wait complete. 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.871+0000 7fd0ad9a8740 1 -- start start 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.871+0000 7fd0ad9a8740 1 Processor -- start 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0ad9a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d015d0 0x555be2d14610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0ad9a8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d02b40 0x555be2d14b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0ad9a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 0x555be2d0f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0ad9a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41aed30 con 0x555be2d0c370 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0ad9a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41ae660 con 0x555be2d015d0 2024-09-19T06:04:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0ad9a8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41ab8a0 con 0x555be2d02b40 2024-09-19T06:04:53.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0ad9a8740 10 client.0 _ll_register_callbacks cb 0x7fd09c00aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:53.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0a9b6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 0x555be2d0f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0a9b6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 0x555be2d0f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37356/0 (socket says 172.21.15.5:37356) 2024-09-19T06:04:53.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0a9b6c640 1 -- 172.21.15.5:0/2406938354 learned_addr learned my addr 172.21.15.5:0/2406938354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:53.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0a936b640 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d015d0 0x555be2d14610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0a8b6a640 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d02b40 0x555be2d14b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0a9b6c640 1 -- 172.21.15.5:0/2406938354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d02b40 msgr2=0x555be2d14b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:53.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0a9b6c640 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d02b40 0x555be2d14b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0a9b6c640 1 -- 172.21.15.5:0/2406938354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d015d0 msgr2=0x555be2d14610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:53.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0a9b6c640 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d015d0 0x555be2d14610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0a9b6c640 1 -- 172.21.15.5:0/2406938354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be2d0f9f0 con 0x555be2d0c370 2024-09-19T06:04:53.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0a8b6a640 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d02b40 0x555be2d14b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:04:53.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd0a9b6c640 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 0x555be2d0f3e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00ab50 tx=0x7fd09c00a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf49f9f6c65118c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:53.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.872+0000 7fd08cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:53.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd08cff9640 1 -- 172.21.15.5:0/2406938354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0107d0 con 0x555be2d0c370 2024-09-19T06:04:53.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd08cff9640 1 -- 172.21.15.5:0/2406938354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fd09c010970 con 0x555be2d0c370 2024-09-19T06:04:53.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2406938354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0940ce050 con 0x555be2d0c370 2024-09-19T06:04:53.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd08cff9640 1 -- 172.21.15.5:0/2406938354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd09c010b70 con 0x555be2d0c370 2024-09-19T06:04:53.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd0ad9a8740 10 client.15747 fetch_fsmap learned FSMap version 30 2024-09-19T06:04:53.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2406938354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555be2d87490 con 0x555be2d0c370 2024-09-19T06:04:53.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd08cff9640 1 -- 172.21.15.5:0/2406938354 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0940ce050 con 0x555be2d0c370 2024-09-19T06:04:53.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd08cff9640 1 -- 172.21.15.5:0/2406938354 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd09c041030 con 0x555be2d0c370 2024-09-19T06:04:53.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd0ad9a8740 10 client.15747 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:04:53.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd0ad9a8740 10 client.15747 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:04:53.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd0ad9a8740 20 client.15747 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:53.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x555be2d87c00 0x555be2d89ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:53.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd0ad9a8740 4 client.15747 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:04:53.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.873+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2406938354 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555be2c6ed70 con 0x555be2d87c00 2024-09-19T06:04:53.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.874+0000 7fd0a936b640 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x555be2d87c00 0x555be2d89ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:53.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd08cff9640 10 client.15747 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:04:53.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0a936b640 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x555be2d87c00 0x555be2d89ff0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e040338c2cf342a1 server_cookie=5d1d6b1f78537bb7 in_seq=0 out_seq=0 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd08cff9640 1 -- 172.21.15.5:0/2406938354 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555be2c6ed70 con 0x555be2d87c00 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd08cff9640 10 client.15747 handle_command_reply: tid=0 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0ad9a8740 1 client.15747 shutdown 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2406938354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 msgr2=0x555be2d0f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 0x555be2d0f3e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00ab50 tx=0x7fd09c00a700 comp rx=0 tx=0).stop 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2406938354 shutdown_connections 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x555be2d87c00 0x555be2d89ff0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd08cff9640 0 client.15747 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be2d0c370 0x555be2d0f3e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555be2d02b40 0x555be2d14b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0ad9a8740 1 --2- 172.21.15.5:0/2406938354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555be2d015d0 0x555be2d14610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2406938354 >> 172.21.15.5:0/2406938354 conn(0x555be2cca310 msgr2=0x555be2cfc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:53.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2406938354 shutdown_connections 2024-09-19T06:04:53.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0ad9a8740 1 -- 172.21.15.5:0/2406938354 wait complete. 2024-09-19T06:04:53.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.875+0000 7fd0ad9a8740 20 client.15747 trim_cache size 0 max 16384 2024-09-19T06:04:53.884 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:04:53.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.876+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/3771195810 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0900a33b0 msgr2=0x7fd0900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:53.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.876+0000 7fd0ab5cf640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0900a33b0 0x7fd0900a5870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd09c005c40 tx=0x7fd09c00a3c0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.876+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/3771195810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a4102790 msgr2=0x7fd0a4198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:53.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.876+0000 7fd0ab5cf640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a4102790 0x7fd0a4198940 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd09400b4a0 tx=0x7fd09400b960 comp rx=0 tx=0).stop 2024-09-19T06:04:53.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.876+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/3771195810 shutdown_connections 2024-09-19T06:04:53.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.876+0000 7fd0ab5cf640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0a4106f10 0x7fd0a419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.876+0000 7fd0ab5cf640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a4102790 0x7fd0a4198940 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.876+0000 7fd0ab5cf640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0900a33b0 0x7fd0900a5870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.876+0000 7fd0ab5cf640 1 --2- 172.21.15.5:0/3771195810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a4101de0 0x7fd0a4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:53.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.876+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/3771195810 >> 172.21.15.5:0/3771195810 conn(0x7fd0a40fd8c0 msgr2=0x7fd0a40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:53.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.876+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/3771195810 shutdown_connections 2024-09-19T06:04:53.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:53.876+0000 7fd0ab5cf640 1 -- 172.21.15.5:0/3771195810 wait complete. 2024-09-19T06:04:54.556 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:04:54.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:54 smithi005 ceph-mon[30955]: pgmap v49: 97 pgs: 97 active+clean; 29 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-09-19T06:04:54.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:54 smithi005 ceph-mon[30955]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T06:04:54.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:54 smithi049 ceph-mon[34640]: pgmap v49: 97 pgs: 97 active+clean; 29 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-09-19T06:04:54.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:54 smithi049 ceph-mon[34640]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T06:04:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:54 smithi157 ceph-mon[35346]: pgmap v49: 97 pgs: 97 active+clean; 29 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-09-19T06:04:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:54 smithi157 ceph-mon[35346]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T06:04:54.815 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:54.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 11400 3.93 MB/sec warmup 20 sec latency 2537.032 ms 2024-09-19T06:04:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.277+0000 7f716d486640 1 -- 172.21.15.5:0/277068187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7168102790 msgr2=0x7f71681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.277+0000 7f716d486640 1 --2- 172.21.15.5:0/277068187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7168102790 0x7f71681067e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f715c009900 tx=0x7f715c02a3f0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.277+0000 7f716d486640 1 -- 172.21.15.5:0/277068187 shutdown_connections 2024-09-19T06:04:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.277+0000 7f716d486640 1 --2- 172.21.15.5:0/277068187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7168106f10 0x7f716810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.277+0000 7f716d486640 1 --2- 172.21.15.5:0/277068187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7168102790 0x7f71681067e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.277+0000 7f716d486640 1 --2- 172.21.15.5:0/277068187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7168101de0 0x7f71681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.277+0000 7f716d486640 1 -- 172.21.15.5:0/277068187 >> 172.21.15.5:0/277068187 conn(0x7f71680fd8c0 msgr2=0x7f71680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.277+0000 7f716d486640 1 -- 172.21.15.5:0/277068187 shutdown_connections 2024-09-19T06:04:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.277+0000 7f716d486640 1 -- 172.21.15.5:0/277068187 wait complete. 2024-09-19T06:04:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f716d486640 1 Processor -- start 2024-09-19T06:04:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f716d486640 1 -- start start 2024-09-19T06:04:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f716d486640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7168101de0 0x7f71681983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f716d486640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7168102790 0x7f7168198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f716d486640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7168106f10 0x7f716819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f716d486640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7168111e20 con 0x7f7168106f10 2024-09-19T06:04:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f716d486640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7168111ca0 con 0x7f7168102790 2024-09-19T06:04:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f716d486640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7168111fa0 con 0x7f7168101de0 2024-09-19T06:04:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f71677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7168106f10 0x7f716819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f71677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7168106f10 0x7f716819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42144/0 (socket says 172.21.15.5:42144) 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f71677fe640 1 -- 172.21.15.5:0/2522212650 learned_addr learned my addr 172.21.15.5:0/2522212650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f7166ffd640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7168101de0 0x7f71681983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.278+0000 7f71667fc640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7168102790 0x7f7168198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.279+0000 7f71677fe640 1 -- 172.21.15.5:0/2522212650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7168101de0 msgr2=0x7f71681983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.279+0000 7f71677fe640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7168101de0 0x7f71681983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.279+0000 7f71677fe640 1 -- 172.21.15.5:0/2522212650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7168102790 msgr2=0x7f7168198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.279+0000 7f71677fe640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7168102790 0x7f7168198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.279+0000 7f71677fe640 1 -- 172.21.15.5:0/2522212650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716819d370 con 0x7f7168106f10 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.279+0000 7f71677fe640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7168106f10 0x7f716819cc90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f716000c970 tx=0x7f716000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.279+0000 7f714ffff640 1 -- 172.21.15.5:0/2522212650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160007bb0 con 0x7f7168106f10 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.279+0000 7f714ffff640 1 -- 172.21.15.5:0/2522212650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7160007d50 con 0x7f7168106f10 2024-09-19T06:04:55.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.279+0000 7f716d486640 1 -- 172.21.15.5:0/2522212650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716819d660 con 0x7f7168106f10 2024-09-19T06:04:55.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.279+0000 7f714ffff640 1 -- 172.21.15.5:0/2522212650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160018700 con 0x7f7168106f10 2024-09-19T06:04:55.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.279+0000 7f716d486640 1 -- 172.21.15.5:0/2522212650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71681a4f40 con 0x7f7168106f10 2024-09-19T06:04:55.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.282+0000 7f714ffff640 1 -- 172.21.15.5:0/2522212650 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f716000fa20 con 0x7f7168106f10 2024-09-19T06:04:55.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.282+0000 7f716f85f740 1 Processor -- start 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.282+0000 7f716f85f740 1 -- start start 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f716f85f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c76220b40 0x557c7621f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f716f85f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c7621f5d0 0x557c7621fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f716f85f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c76230b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f716f85f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557c761852f0 con 0x557c7622e780 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f716f85f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557c7612ccf0 con 0x557c7621f5d0 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f716f85f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557c761812b0 con 0x557c76220b40 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f71667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c76230b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f71667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c76230b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42154/0 (socket says 172.21.15.5:42154) 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f71667fc640 1 -- 172.21.15.5:0/3389594594 learned_addr learned my addr 172.21.15.5:0/3389594594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f7166ffd640 1 --2- 172.21.15.5:0/3389594594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c7621f5d0 0x557c7621fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f71667fc640 1 --2- 172.21.15.5:0/3389594594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c76220b40 0x557c7621f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f714ffff640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f71500a33b0 0x7f71500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f71677fe640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f71500a33b0 0x7f71500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f71667fc640 1 -- 172.21.15.5:0/3389594594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c76220b40 msgr2=0x557c7621f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f71667fc640 1 --2- 172.21.15.5:0/3389594594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c76220b40 0x557c7621f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f71667fc640 1 -- 172.21.15.5:0/3389594594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c7621f5d0 msgr2=0x557c7621fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f71667fc640 1 --2- 172.21.15.5:0/3389594594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c7621f5d0 0x557c7621fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f71667fc640 1 -- 172.21.15.5:0/3389594594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c761113a0 con 0x557c7622e780 2024-09-19T06:04:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f7166ffd640 1 --2- 172.21.15.5:0/3389594594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c7621f5d0 0x557c7621fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.283+0000 7f714ffff640 1 -- 172.21.15.5:0/2522212650 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f71600cfc30 con 0x7f7168106f10 2024-09-19T06:04:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.284+0000 7f71667fc640 1 --2- 172.21.15.5:0/3389594594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c76230b70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f715c02a910 tx=0x7f715c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87363edcd7db5779 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.284+0000 7f71677fe640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f71500a33b0 0x7f71500a5870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7160098890 tx=0x7f7160000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.284+0000 7f714d7fa640 1 -- 172.21.15.5:0/3389594594 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c035070 con 0x557c7622e780 2024-09-19T06:04:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.284+0000 7f714d7fa640 1 -- 172.21.15.5:0/3389594594 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f715c0035f0 con 0x557c7622e780 2024-09-19T06:04:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.284+0000 7f714d7fa640 1 -- 172.21.15.5:0/3389594594 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c0044e0 con 0x557c7622e780 2024-09-19T06:04:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.285+0000 7f716f85f740 1 -- 172.21.15.5:0/3389594594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 msgr2=0x557c76230b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.285+0000 7f716f85f740 1 --2- 172.21.15.5:0/3389594594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c76230b70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f715c02a910 tx=0x7f715c005db0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.285+0000 7f716f85f740 1 -- 172.21.15.5:0/3389594594 shutdown_connections 2024-09-19T06:04:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.285+0000 7f716f85f740 1 --2- 172.21.15.5:0/3389594594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c76230b70 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.285+0000 7f716f85f740 1 --2- 172.21.15.5:0/3389594594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c7621f5d0 0x557c7621fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.285+0000 7f716f85f740 1 --2- 172.21.15.5:0/3389594594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c76220b40 0x557c7621f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.285+0000 7f716f85f740 1 -- 172.21.15.5:0/3389594594 >> 172.21.15.5:0/3389594594 conn(0x557c761e8310 msgr2=0x557c7621bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.285+0000 7f716f85f740 1 -- 172.21.15.5:0/3389594594 shutdown_connections 2024-09-19T06:04:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.285+0000 7f716f85f740 1 -- 172.21.15.5:0/3389594594 wait complete. 2024-09-19T06:04:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.285+0000 7f716f85f740 1 -- start start 2024-09-19T06:04:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.285+0000 7f716f85f740 1 Processor -- start 2024-09-19T06:04:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f716f85f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7621f5d0 0x557c7622e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f716f85f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c76220b40 0x557c7622b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f716f85f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c7622ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f716f85f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557c76231420 con 0x557c7622e780 2024-09-19T06:04:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f716f85f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557c76140060 con 0x557c76220b40 2024-09-19T06:04:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f716f85f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557c762315a0 con 0x557c7621f5d0 2024-09-19T06:04:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f716f85f740 10 client.0 _ll_register_callbacks cb 0x557c75e938c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f71667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c7622ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f71667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c7622ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42170/0 (socket says 172.21.15.5:42170) 2024-09-19T06:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f71667fc640 1 -- 172.21.15.5:0/2344531765 learned_addr learned my addr 172.21.15.5:0/2344531765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f7166ffd640 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c76220b40 0x557c7622b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f71667fc640 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7621f5d0 0x557c7622e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f71667fc640 1 -- 172.21.15.5:0/2344531765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7621f5d0 msgr2=0x557c7622e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f71667fc640 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7621f5d0 0x557c7622e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f71667fc640 1 -- 172.21.15.5:0/2344531765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c76220b40 msgr2=0x557c7622b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f71667fc640 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c76220b40 0x557c7622b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f71667fc640 1 -- 172.21.15.5:0/2344531765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c75ef63c0 con 0x557c7622e780 2024-09-19T06:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f71667fc640 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c7622ba20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f715c0096c0 tx=0x7f715c041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea32a6129da33b36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f714aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f714aff5640 1 -- 172.21.15.5:0/2344531765 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c035070 con 0x557c7622e780 2024-09-19T06:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f714aff5640 1 -- 172.21.15.5:0/2344531765 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f715c003a70 con 0x557c7622e780 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.286+0000 7f716f85f740 1 -- 172.21.15.5:0/2344531765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f715c0035f0 con 0x557c7622e780 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.287+0000 7f714aff5640 1 -- 172.21.15.5:0/2344531765 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f715c05a050 con 0x557c7622e780 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.287+0000 7f716f85f740 10 client.15759 fetch_fsmap learned FSMap version 30 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.287+0000 7f716f85f740 1 -- 172.21.15.5:0/2344531765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557c7623cec0 con 0x557c7622e780 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.287+0000 7f714aff5640 1 -- 172.21.15.5:0/2344531765 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f715c0035f0 con 0x557c7622e780 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.287+0000 7f714aff5640 1 -- 172.21.15.5:0/2344531765 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f715c004750 con 0x557c7622e780 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.287+0000 7f716f85f740 10 client.15759 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.287+0000 7f716f85f740 10 client.15759 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.287+0000 7f716f85f740 20 client.15759 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.287+0000 7f716f85f740 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557c7623e210 0x557c76240600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.287+0000 7f716f85f740 4 client.15759 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.287+0000 7f716f85f740 1 -- 172.21.15.5:0/2344531765 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c76192860 con 0x557c7623e210 2024-09-19T06:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.287+0000 7f7166ffd640 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557c7623e210 0x557c76240600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.288+0000 7f7166ffd640 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557c7623e210 0x557c76240600 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=352fdf125efdacb4 server_cookie=1f912068638cd0e in_seq=0 out_seq=0 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.288+0000 7f714aff5640 10 client.15759 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f714aff5640 1 -- 172.21.15.5:0/2344531765 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c76192860 con 0x557c7623e210 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f714aff5640 10 client.15759 handle_command_reply: tid=0 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f716f85f740 1 client.15759 shutdown 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f716f85f740 1 -- 172.21.15.5:0/2344531765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 msgr2=0x557c7622ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f716f85f740 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c7622ba20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f715c0096c0 tx=0x7f715c041040 comp rx=0 tx=0).stop 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f716f85f740 1 -- 172.21.15.5:0/2344531765 shutdown_connections 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f716f85f740 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557c7623e210 0x557c76240600 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f716f85f740 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c7622ba20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f716f85f740 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c76220b40 0x557c7622b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f716f85f740 1 --2- 172.21.15.5:0/2344531765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7621f5d0 0x557c7622e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f714aff5640 0 client.15759 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:04:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f716f85f740 1 -- 172.21.15.5:0/2344531765 >> 172.21.15.5:0/2344531765 conn(0x557c761e8310 msgr2=0x557c76221960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f716f85f740 1 -- 172.21.15.5:0/2344531765 shutdown_connections 2024-09-19T06:04:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f716f85f740 1 -- 172.21.15.5:0/2344531765 wait complete. 2024-09-19T06:04:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.289+0000 7f716f85f740 20 client.15759 trim_cache size 0 max 16384 2024-09-19T06:04:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.302+0000 7f716f85f740 1 Processor -- start 2024-09-19T06:04:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.302+0000 7f716f85f740 1 -- start start 2024-09-19T06:04:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.302+0000 7f716f85f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7621f5d0 0x557c7622a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.302+0000 7f716f85f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c76220b40 0x557c76243620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.302+0000 7f716f85f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c7622e780 0x557c76243b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.302+0000 7f716f85f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71681ab4f0 con 0x557c7621f5d0 2024-09-19T06:04:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.302+0000 7f716f85f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f71681ad090 con 0x557c7622e780 2024-09-19T06:04:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.302+0000 7f716f85f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71681abbc0 con 0x557c76220b40 2024-09-19T06:04:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.302+0000 7f71667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7621f5d0 0x557c7622a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.303+0000 7f71667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7621f5d0 0x557c7622a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42174/0 (socket says 172.21.15.5:42174) 2024-09-19T06:04:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.303+0000 7f71667fc640 1 -- 172.21.15.5:0/11565943 learned_addr learned my addr 172.21.15.5:0/11565943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.303+0000 7f7166ffd640 1 --2- 172.21.15.5:0/11565943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c76220b40 0x557c76243620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.303+0000 7f71667fc640 1 -- 172.21.15.5:0/11565943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c76220b40 msgr2=0x557c76243620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.303+0000 7f71667fc640 1 --2- 172.21.15.5:0/11565943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c76220b40 0x557c76243620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.303+0000 7f71667fc640 1 -- 172.21.15.5:0/11565943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c7622e780 msgr2=0x557c76243b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.303+0000 7f71667fc640 1 --2- 172.21.15.5:0/11565943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c7622e780 0x557c76243b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.303+0000 7f71667fc640 1 -- 172.21.15.5:0/11565943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c7621f2d0 con 0x557c7621f5d0 2024-09-19T06:04:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.303+0000 7f71667fc640 1 --2- 172.21.15.5:0/11565943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7621f5d0 0x557c7622a430 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f715c009900 tx=0x7f715c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d9d1c0d54cb7ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.303+0000 7f714cff9640 1 -- 172.21.15.5:0/11565943 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c063070 con 0x557c7621f5d0 2024-09-19T06:04:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.303+0000 7f714cff9640 1 -- 172.21.15.5:0/11565943 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f715c04e020 con 0x557c7621f5d0 2024-09-19T06:04:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.303+0000 7f714cff9640 1 -- 172.21.15.5:0/11565943 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c035040 con 0x557c7621f5d0 2024-09-19T06:04:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.304+0000 7f716f85f740 1 -- 172.21.15.5:0/11565943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7621f5d0 msgr2=0x557c7622a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.304+0000 7f716f85f740 1 --2- 172.21.15.5:0/11565943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7621f5d0 0x557c7622a430 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f715c009900 tx=0x7f715c005fd0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.304+0000 7f716f85f740 1 -- 172.21.15.5:0/11565943 shutdown_connections 2024-09-19T06:04:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.304+0000 7f716f85f740 1 --2- 172.21.15.5:0/11565943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c7622e780 0x557c76243b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.304+0000 7f716f85f740 1 --2- 172.21.15.5:0/11565943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c76220b40 0x557c76243620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.304+0000 7f716f85f740 1 --2- 172.21.15.5:0/11565943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7621f5d0 0x557c7622a430 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.304+0000 7f716f85f740 1 -- 172.21.15.5:0/11565943 >> 172.21.15.5:0/11565943 conn(0x557c761e8310 msgr2=0x557c76229d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.304+0000 7f716f85f740 1 -- 172.21.15.5:0/11565943 shutdown_connections 2024-09-19T06:04:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.304+0000 7f716f85f740 1 -- 172.21.15.5:0/11565943 wait complete. 2024-09-19T06:04:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.304+0000 7f716f85f740 1 -- start start 2024-09-19T06:04:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.304+0000 7f716f85f740 1 Processor -- start 2024-09-19T06:04:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f716f85f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7621f5d0 0x557c76243d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f716f85f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c76220b40 0x557c762410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f716f85f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c762415f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f716f85f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71681ac960 con 0x557c7622e780 2024-09-19T06:04:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f716f85f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f71681ac290 con 0x557c76220b40 2024-09-19T06:04:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f716f85f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71681a94d0 con 0x557c7621f5d0 2024-09-19T06:04:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f716f85f740 10 client.0 _ll_register_callbacks cb 0x7f7158009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f71667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c762415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f71667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c762415f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42186/0 (socket says 172.21.15.5:42186) 2024-09-19T06:04:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f71667fc640 1 -- 172.21.15.5:0/2077495833 learned_addr learned my addr 172.21.15.5:0/2077495833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f71667fc640 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7621f5d0 0x557c76243d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f7166ffd640 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c76220b40 0x557c762410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f71667fc640 1 -- 172.21.15.5:0/2077495833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7621f5d0 msgr2=0x557c76243d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f71667fc640 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7621f5d0 0x557c76243d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f71667fc640 1 -- 172.21.15.5:0/2077495833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c76220b40 msgr2=0x557c762410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f71667fc640 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c76220b40 0x557c762410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f71667fc640 1 -- 172.21.15.5:0/2077495833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c76241e20 con 0x557c7622e780 2024-09-19T06:04:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f71667fc640 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c762415f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f715c0096c0 tx=0x7f715c003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da348ccff75487e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f714aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f714aff5640 1 -- 172.21.15.5:0/2077495833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c063070 con 0x557c7622e780 2024-09-19T06:04:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f714aff5640 1 -- 172.21.15.5:0/2077495833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f715c052030 con 0x557c7622e780 2024-09-19T06:04:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.305+0000 7f716f85f740 1 -- 172.21.15.5:0/2077495833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f715c04e020 con 0x557c7622e780 2024-09-19T06:04:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.306+0000 7f714aff5640 1 -- 172.21.15.5:0/2077495833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f715c071050 con 0x557c7622e780 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.306+0000 7f716f85f740 10 client.15765 fetch_fsmap learned FSMap version 30 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.306+0000 7f716f85f740 1 -- 172.21.15.5:0/2077495833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557c762a8b40 con 0x557c7622e780 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.306+0000 7f714aff5640 1 -- 172.21.15.5:0/2077495833 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f715c04e020 con 0x557c7622e780 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.306+0000 7f714aff5640 1 -- 172.21.15.5:0/2077495833 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f715c031760 con 0x557c7622e780 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.306+0000 7f716f85f740 10 client.15765 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.306+0000 7f716f85f740 10 client.15765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.306+0000 7f716f85f740 20 client.15765 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.306+0000 7f716f85f740 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557c762a90f0 0x557c762ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.306+0000 7f716f85f740 4 client.15765 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.306+0000 7f716f85f740 1 -- 172.21.15.5:0/2077495833 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c7618cd70 con 0x557c762a90f0 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.306+0000 7f7166ffd640 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557c762a90f0 0x557c762ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f714aff5640 10 client.15765 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:04:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f7166ffd640 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557c762a90f0 0x557c762ab4e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc72be9430b4e983 server_cookie=8bb9e3588316693 in_seq=0 out_seq=0 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f714aff5640 1 -- 172.21.15.5:0/2077495833 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c7618cd70 con 0x557c762a90f0 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f714aff5640 10 client.15765 handle_command_reply: tid=0 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f716f85f740 1 client.15765 shutdown 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f716f85f740 1 -- 172.21.15.5:0/2077495833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 msgr2=0x557c762415f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f716f85f740 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c762415f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f715c0096c0 tx=0x7f715c003c60 comp rx=0 tx=0).stop 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f716f85f740 1 -- 172.21.15.5:0/2077495833 shutdown_connections 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f716f85f740 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557c762a90f0 0x557c762ab4e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f716f85f740 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7622e780 0x557c762415f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f714aff5640 0 client.15765 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f716f85f740 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c76220b40 0x557c762410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f716f85f740 1 --2- 172.21.15.5:0/2077495833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7621f5d0 0x557c76243d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f716f85f740 1 -- 172.21.15.5:0/2077495833 >> 172.21.15.5:0/2077495833 conn(0x557c761e8310 msgr2=0x557c7622a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f716f85f740 1 -- 172.21.15.5:0/2077495833 shutdown_connections 2024-09-19T06:04:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f716f85f740 1 -- 172.21.15.5:0/2077495833 wait complete. 2024-09-19T06:04:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.307+0000 7f716f85f740 20 client.15765 trim_cache size 0 max 16384 2024-09-19T06:04:55.316 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:04:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.308+0000 7f716d486640 1 -- 172.21.15.5:0/2522212650 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f71500a33b0 msgr2=0x7f71500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.308+0000 7f716d486640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f71500a33b0 0x7f71500a5870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7160098890 tx=0x7f7160000bc0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.308+0000 7f716d486640 1 -- 172.21.15.5:0/2522212650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7168106f10 msgr2=0x7f716819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.308+0000 7f716d486640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7168106f10 0x7f716819cc90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f716000c970 tx=0x7f716000ce30 comp rx=0 tx=0).stop 2024-09-19T06:04:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.308+0000 7f716d486640 1 -- 172.21.15.5:0/2522212650 shutdown_connections 2024-09-19T06:04:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.308+0000 7f716d486640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7168106f10 0x7f716819cc90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.308+0000 7f716d486640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7168102790 0x7f7168198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.308+0000 7f716d486640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f71500a33b0 0x7f71500a5870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.308+0000 7f716d486640 1 --2- 172.21.15.5:0/2522212650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7168101de0 0x7f71681983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.308+0000 7f716d486640 1 -- 172.21.15.5:0/2522212650 >> 172.21.15.5:0/2522212650 conn(0x7f71680fd8c0 msgr2=0x7f71680fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.308+0000 7f716d486640 1 -- 172.21.15.5:0/2522212650 shutdown_connections 2024-09-19T06:04:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:55.308+0000 7f716d486640 1 -- 172.21.15.5:0/2522212650 wait complete. 2024-09-19T06:04:55.821 INFO:tasks.workunit.client.0.smithi005.stdout: 1 11589 3.77 MB/sec warmup 21 sec latency 3162.577 ms 2024-09-19T06:04:56.033 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:04:56.306 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:56.591 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:56 smithi005 ceph-mon[30955]: pgmap v50: 97 pgs: 97 active+clean; 29 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-09-19T06:04:56.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:56 smithi049 ceph-mon[34640]: pgmap v50: 97 pgs: 97 active+clean; 29 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-09-19T06:04:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:56 smithi157 ceph-mon[35346]: pgmap v50: 97 pgs: 97 active+clean; 29 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-09-19T06:04:56.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 12179 3.79 MB/sec warmup 22 sec latency 11.029 ms 2024-09-19T06:04:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.821+0000 7f5df9c17640 1 -- 172.21.15.5:0/2205891766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df4101360 msgr2=0x7f5df410df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.821+0000 7f5df9c17640 1 --2- 172.21.15.5:0/2205891766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df4101360 0x7f5df410df90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009900 tx=0x7f5de802a410 comp rx=0 tx=0).stop 2024-09-19T06:04:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.821+0000 7f5df9c17640 1 -- 172.21.15.5:0/2205891766 shutdown_connections 2024-09-19T06:04:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.821+0000 7f5df9c17640 1 --2- 172.21.15.5:0/2205891766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df410e4d0 0x7f5df41108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.821+0000 7f5df9c17640 1 --2- 172.21.15.5:0/2205891766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df4101360 0x7f5df410df90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.821+0000 7f5df9c17640 1 --2- 172.21.15.5:0/2205891766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5df41009b0 0x7f5df4100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.821+0000 7f5df9c17640 1 -- 172.21.15.5:0/2205891766 >> 172.21.15.5:0/2205891766 conn(0x7f5df40fc7d0 msgr2=0x7f5df40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.821+0000 7f5df9c17640 1 -- 172.21.15.5:0/2205891766 shutdown_connections 2024-09-19T06:04:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.821+0000 7f5df9c17640 1 -- 172.21.15.5:0/2205891766 wait complete. 2024-09-19T06:04:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.821+0000 7f5df9c17640 1 Processor -- start 2024-09-19T06:04:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df9c17640 1 -- start start 2024-09-19T06:04:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df9c17640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5df41009b0 0x7f5df419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df9c17640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df4101360 0x7f5df419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df9c17640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df410e4d0 0x7f5df419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df9c17640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41140d0 con 0x7f5df4101360 2024-09-19T06:04:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df9c17640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4113f50 con 0x7f5df41009b0 2024-09-19T06:04:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df9c17640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4114250 con 0x7f5df410e4d0 2024-09-19T06:04:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df410e4d0 0x7f5df419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df4101360 0x7f5df419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df410e4d0 0x7f5df419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40068/0 (socket says 172.21.15.5:40068) 2024-09-19T06:04:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df3fff640 1 -- 172.21.15.5:0/2189891353 learned_addr learned my addr 172.21.15.5:0/2189891353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df37fe640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5df41009b0 0x7f5df419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df3fff640 1 -- 172.21.15.5:0/2189891353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5df41009b0 msgr2=0x7f5df419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df3fff640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5df41009b0 0x7f5df419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df3fff640 1 -- 172.21.15.5:0/2189891353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df4101360 msgr2=0x7f5df419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df3fff640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df4101360 0x7f5df419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df3fff640 1 -- 172.21.15.5:0/2189891353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df419f640 con 0x7f5df410e4d0 2024-09-19T06:04:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.822+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df4101360 0x7f5df419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.823+0000 7f5df3fff640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df410e4d0 0x7f5df419ef60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00eb10 tx=0x7f5dec00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.823+0000 7f5df0ff9640 1 -- 172.21.15.5:0/2189891353 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec004b20 con 0x7f5df410e4d0 2024-09-19T06:04:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.823+0000 7f5df9c17640 1 -- 172.21.15.5:0/2189891353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df419f930 con 0x7f5df410e4d0 2024-09-19T06:04:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.823+0000 7f5df0ff9640 1 -- 172.21.15.5:0/2189891353 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dec004cc0 con 0x7f5df410e4d0 2024-09-19T06:04:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.823+0000 7f5df0ff9640 1 -- 172.21.15.5:0/2189891353 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec010690 con 0x7f5df410e4d0 2024-09-19T06:04:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.823+0000 7f5df9c17640 1 -- 172.21.15.5:0/2189891353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df41a7210 con 0x7f5df410e4d0 2024-09-19T06:04:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5df0ff9640 1 -- 172.21.15.5:0/2189891353 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5dec005020 con 0x7f5df410e4d0 2024-09-19T06:04:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5dfbff0740 1 Processor -- start 2024-09-19T06:04:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5dfbff0740 1 -- start start 2024-09-19T06:04:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5dfbff0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 0x55b69403b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5dfbff0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403b5d0 0x55b69403ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5dfbff0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b694046370 0x55b694048760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5dfbff0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b693fa12f0 con 0x55b69403cb40 2024-09-19T06:04:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5dfbff0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b693f48cf0 con 0x55b694046370 2024-09-19T06:04:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5dfbff0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b693f9d2b0 con 0x55b69403b5d0 2024-09-19T06:04:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5df2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 0x55b69403b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5df0ff9640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5ddc0a33b0 0x7f5ddc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5df2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 0x55b69403b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42212/0 (socket says 172.21.15.5:42212) 2024-09-19T06:04:56.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.826+0000 7f5df2ffd640 1 -- 172.21.15.5:0/1316667214 learned_addr learned my addr 172.21.15.5:0/1316667214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:56.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5df37fe640 1 --2- 172.21.15.5:0/1316667214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403b5d0 0x55b69403ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5df3fff640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5ddc0a33b0 0x7f5ddc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/1316667214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b694046370 0x55b694048760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5df0ff9640 1 -- 172.21.15.5:0/2189891353 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f5dec0d0390 con 0x7f5df410e4d0 2024-09-19T06:04:56.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5df2ffd640 1 -- 172.21.15.5:0/1316667214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403b5d0 msgr2=0x55b69403ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:56.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/1316667214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403b5d0 0x55b69403ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5df2ffd640 1 -- 172.21.15.5:0/1316667214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b694046370 msgr2=0x55b694048760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:56.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/1316667214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b694046370 0x55b694048760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5df2ffd640 1 -- 172.21.15.5:0/1316667214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b693f2d3a0 con 0x55b69403cb40 2024-09-19T06:04:56.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/1316667214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 0x55b69403b090 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55b69403c070 tx=0x7f5de8032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262ece5efd22eeed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:56.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5df3fff640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5ddc0a33b0 0x7f5ddc0a5870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0ce0f0 tx=0x7f5dec091b10 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:56.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5dd9ffb640 1 -- 172.21.15.5:0/1316667214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8035040 con 0x55b69403cb40 2024-09-19T06:04:56.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5dd9ffb640 1 -- 172.21.15.5:0/1316667214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5de80318b0 con 0x55b69403cb40 2024-09-19T06:04:56.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.827+0000 7f5dd9ffb640 1 -- 172.21.15.5:0/1316667214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de80049d0 con 0x55b69403cb40 2024-09-19T06:04:56.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.828+0000 7f5dfbff0740 1 -- 172.21.15.5:0/1316667214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 msgr2=0x55b69403b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:56.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.828+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/1316667214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 0x55b69403b090 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55b69403c070 tx=0x7f5de8032f10 comp rx=0 tx=0).stop 2024-09-19T06:04:56.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.828+0000 7f5dfbff0740 1 -- 172.21.15.5:0/1316667214 shutdown_connections 2024-09-19T06:04:56.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.828+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/1316667214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b694046370 0x55b694048760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.828+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/1316667214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403b5d0 0x55b69403ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.828+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/1316667214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 0x55b69403b090 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.829+0000 7f5dfbff0740 1 -- 172.21.15.5:0/1316667214 >> 172.21.15.5:0/1316667214 conn(0x55b694004310 msgr2=0x55b694037c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:56.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.829+0000 7f5dfbff0740 1 -- 172.21.15.5:0/1316667214 shutdown_connections 2024-09-19T06:04:56.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.829+0000 7f5dfbff0740 1 -- 172.21.15.5:0/1316667214 wait complete. 2024-09-19T06:04:56.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.829+0000 7f5dfbff0740 1 -- start start 2024-09-19T06:04:56.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.829+0000 7f5dfbff0740 1 Processor -- start 2024-09-19T06:04:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.829+0000 7f5dfbff0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403b5d0 0x55b694045d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5dfbff0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 0x55b694042fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5dfbff0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b694046370 0x55b6940435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5dfbff0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b694049010 con 0x55b69403cb40 2024-09-19T06:04:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5dfbff0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b693f5c060 con 0x55b694046370 2024-09-19T06:04:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5dfbff0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b694049190 con 0x55b69403b5d0 2024-09-19T06:04:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5df2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403b5d0 0x55b694045d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5dfbff0740 10 client.0 _ll_register_callbacks cb 0x55b693caf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5df2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403b5d0 0x55b694045d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40082/0 (socket says 172.21.15.5:40082) 2024-09-19T06:04:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5df37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 0x55b694042fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5df2ffd640 1 -- 172.21.15.5:0/2558277509 learned_addr learned my addr 172.21.15.5:0/2558277509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b694046370 0x55b6940435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5df37fe640 1 -- 172.21.15.5:0/2558277509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403b5d0 msgr2=0x55b694045d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5df37fe640 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403b5d0 0x55b694045d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5df37fe640 1 -- 172.21.15.5:0/2558277509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b694046370 msgr2=0x55b6940435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5df37fe640 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b694046370 0x55b6940435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5df37fe640 1 -- 172.21.15.5:0/2558277509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b693d123c0 con 0x55b69403cb40 2024-09-19T06:04:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b694046370 0x55b6940435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5df37fe640 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 0x55b694042fe0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5de40126d0 tx=0x7f5de4012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc04e256593e7e8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.830+0000 7f5dd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dd2ffd640 1 -- 172.21.15.5:0/2558277509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de400f8c0 con 0x55b69403cb40 2024-09-19T06:04:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dd2ffd640 1 -- 172.21.15.5:0/2558277509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f5de400fa60 con 0x55b69403cb40 2024-09-19T06:04:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2558277509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de80318b0 con 0x55b69403cb40 2024-09-19T06:04:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dd2ffd640 1 -- 172.21.15.5:0/2558277509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5de400fc60 con 0x55b69403cb40 2024-09-19T06:04:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dfbff0740 10 client.15777 fetch_fsmap learned FSMap version 30 2024-09-19T06:04:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2558277509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b694054ab0 con 0x55b69403cb40 2024-09-19T06:04:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dd2ffd640 1 -- 172.21.15.5:0/2558277509 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5de80318b0 con 0x55b69403cb40 2024-09-19T06:04:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dd2ffd640 1 -- 172.21.15.5:0/2558277509 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5de401c420 con 0x55b69403cb40 2024-09-19T06:04:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dfbff0740 10 client.15777 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:04:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dfbff0740 10 client.15777 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:04:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dfbff0740 20 client.15777 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b694056010 0x55b694058430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dfbff0740 4 client.15777 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.831+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2558277509 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b693fae860 con 0x55b694056010 2024-09-19T06:04:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.832+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b694056010 0x55b694058430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.832+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b694056010 0x55b694058430 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ceeb4393b39967df server_cookie=65fb2385fdc78796 in_seq=0 out_seq=0 2024-09-19T06:04:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.832+0000 7f5dd2ffd640 10 client.15777 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:04:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dd2ffd640 1 -- 172.21.15.5:0/2558277509 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b693fae860 con 0x55b694056010 2024-09-19T06:04:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dd2ffd640 10 client.15777 handle_command_reply: tid=0 2024-09-19T06:04:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dfbff0740 1 client.15777 shutdown 2024-09-19T06:04:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2558277509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 msgr2=0x55b694042fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 0x55b694042fe0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5de40126d0 tx=0x7f5de4012b90 comp rx=0 tx=0).stop 2024-09-19T06:04:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2558277509 shutdown_connections 2024-09-19T06:04:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b694056010 0x55b694058430 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dd2ffd640 0 client.15777 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:04:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b694046370 0x55b6940435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403cb40 0x55b694042fe0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2558277509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403b5d0 0x55b694045d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2558277509 >> 172.21.15.5:0/2558277509 conn(0x55b694004310 msgr2=0x55b69403d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2558277509 shutdown_connections 2024-09-19T06:04:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2558277509 wait complete. 2024-09-19T06:04:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.833+0000 7f5dfbff0740 20 client.15777 trim_cache size 0 max 16384 2024-09-19T06:04:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.848+0000 7f5dfbff0740 1 Processor -- start 2024-09-19T06:04:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.848+0000 7f5dfbff0740 1 -- start start 2024-09-19T06:04:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.848+0000 7f5dfbff0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b69403b5d0 0x55b6940522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.848+0000 7f5dfbff0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403cb40 0x55b6940428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5dfbff0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b694046370 0x55b69405dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5dfbff0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41afa00 con 0x55b694046370 2024-09-19T06:04:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5dfbff0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41b15a0 con 0x55b69403b5d0 2024-09-19T06:04:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5dfbff0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41b00d0 con 0x55b69403cb40 2024-09-19T06:04:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5df37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403cb40 0x55b6940428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5df37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403cb40 0x55b6940428a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40094/0 (socket says 172.21.15.5:40094) 2024-09-19T06:04:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5df37fe640 1 -- 172.21.15.5:0/2545704546 learned_addr learned my addr 172.21.15.5:0/2545704546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/2545704546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b69403b5d0 0x55b6940522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/2545704546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b694046370 0x55b69405dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5df37fe640 1 -- 172.21.15.5:0/2545704546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b69403b5d0 msgr2=0x55b6940522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5df37fe640 1 --2- 172.21.15.5:0/2545704546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b69403b5d0 0x55b6940522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5df37fe640 1 -- 172.21.15.5:0/2545704546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b694046370 msgr2=0x55b69405dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5df37fe640 1 --2- 172.21.15.5:0/2545704546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b694046370 0x55b69405dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5df37fe640 1 -- 172.21.15.5:0/2545704546 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b694038960 con 0x55b69403cb40 2024-09-19T06:04:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/2545704546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b694046370 0x55b69405dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:04:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5df37fe640 1 --2- 172.21.15.5:0/2545704546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403cb40 0x55b6940428a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5de401c910 tx=0x7f5de400ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da65fbc45153c4d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.849+0000 7f5dd97fa640 1 -- 172.21.15.5:0/2545704546 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de401cb00 con 0x55b69403cb40 2024-09-19T06:04:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.850+0000 7f5dd97fa640 1 -- 172.21.15.5:0/2545704546 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5de401cca0 con 0x55b69403cb40 2024-09-19T06:04:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.850+0000 7f5dd97fa640 1 -- 172.21.15.5:0/2545704546 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40035f0 con 0x55b69403cb40 2024-09-19T06:04:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.850+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2545704546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403cb40 msgr2=0x55b6940428a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.850+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2545704546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403cb40 0x55b6940428a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5de401c910 tx=0x7f5de400ff90 comp rx=0 tx=0).stop 2024-09-19T06:04:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.850+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2545704546 shutdown_connections 2024-09-19T06:04:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.850+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2545704546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b694046370 0x55b69405dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.850+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2545704546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69403cb40 0x55b6940428a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.850+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2545704546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b69403b5d0 0x55b6940522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.850+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2545704546 >> 172.21.15.5:0/2545704546 conn(0x55b694004310 msgr2=0x55b6940418b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.850+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2545704546 shutdown_connections 2024-09-19T06:04:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.851+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2545704546 wait complete. 2024-09-19T06:04:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.851+0000 7f5dfbff0740 1 -- start start 2024-09-19T06:04:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.851+0000 7f5dfbff0740 1 Processor -- start 2024-09-19T06:04:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.851+0000 7f5dfbff0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403b5d0 0x55b69404ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.851+0000 7f5dfbff0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b69403cb40 0x55b694049dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.851+0000 7f5dfbff0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b694046370 0x55b69404a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.851+0000 7f5dfbff0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41b0e70 con 0x55b69403b5d0 2024-09-19T06:04:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.851+0000 7f5dfbff0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41b07a0 con 0x55b69403cb40 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.851+0000 7f5dfbff0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41ad9e0 con 0x55b694046370 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5dfbff0740 10 client.0 _ll_register_callbacks cb 0x7f5de4009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5df2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403b5d0 0x55b69404ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5df2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403b5d0 0x55b69404ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42236/0 (socket says 172.21.15.5:42236) 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5df2ffd640 1 -- 172.21.15.5:0/2394908719 learned_addr learned my addr 172.21.15.5:0/2394908719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5df37fe640 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b69403cb40 0x55b694049dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b694046370 0x55b69404a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5df2ffd640 1 -- 172.21.15.5:0/2394908719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b694046370 msgr2=0x55b69404a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b694046370 0x55b69404a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5df2ffd640 1 -- 172.21.15.5:0/2394908719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b69403cb40 msgr2=0x55b694049dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b69403cb40 0x55b694049dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5df2ffd640 1 -- 172.21.15.5:0/2394908719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b69404ab30 con 0x55b69403b5d0 2024-09-19T06:04:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5df2ffd640 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403b5d0 0x55b69404ea90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5de8004650 tx=0x7f5de4009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac84584b97991c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5dd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.852+0000 7f5dd2ffd640 1 -- 172.21.15.5:0/2394908719 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8035040 con 0x55b69403b5d0 2024-09-19T06:04:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dd2ffd640 1 -- 172.21.15.5:0/2394908719 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f5de8031b80 con 0x55b69403b5d0 2024-09-19T06:04:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2394908719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de401cca0 con 0x55b69403b5d0 2024-09-19T06:04:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dd2ffd640 1 -- 172.21.15.5:0/2394908719 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5de8031d80 con 0x55b69403b5d0 2024-09-19T06:04:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dfbff0740 10 client.15783 fetch_fsmap learned FSMap version 30 2024-09-19T06:04:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2394908719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b6940c93d0 con 0x55b69403b5d0 2024-09-19T06:04:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dd2ffd640 1 -- 172.21.15.5:0/2394908719 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5de401cca0 con 0x55b69403b5d0 2024-09-19T06:04:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dd2ffd640 1 -- 172.21.15.5:0/2394908719 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5de803c470 con 0x55b69403b5d0 2024-09-19T06:04:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dfbff0740 10 client.15783 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:04:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dfbff0740 10 client.15783 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:04:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dfbff0740 20 client.15783 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b6940c9710 0x55b6940cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dfbff0740 4 client.15783 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.853+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2394908719 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b693fa8d70 con 0x55b6940c9710 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.854+0000 7f5df37fe640 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b6940c9710 0x55b6940cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.854+0000 7f5df37fe640 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b6940c9710 0x55b6940cbb00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78cb81a30e2cee9 server_cookie=d163061832c17234 in_seq=0 out_seq=0 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.854+0000 7f5dd2ffd640 10 client.15783 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.854+0000 7f5dd2ffd640 1 -- 172.21.15.5:0/2394908719 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b693fa8d70 con 0x55b6940c9710 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.854+0000 7f5dd2ffd640 10 client.15783 handle_command_reply: tid=0 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.854+0000 7f5dfbff0740 1 client.15783 shutdown 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.854+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2394908719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403b5d0 msgr2=0x55b69404ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.854+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403b5d0 0x55b69404ea90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5de8004650 tx=0x7f5de4009dd0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.855+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2394908719 shutdown_connections 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.855+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b6940c9710 0x55b6940cbb00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.855+0000 7f5dd2ffd640 0 client.15783 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:04:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.855+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b694046370 0x55b69404a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.855+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b69403cb40 0x55b694049dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.855+0000 7f5dfbff0740 1 --2- 172.21.15.5:0/2394908719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b69403b5d0 0x55b69404ea90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.855+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2394908719 >> 172.21.15.5:0/2394908719 conn(0x55b694004310 msgr2=0x55b6940414b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.855+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2394908719 shutdown_connections 2024-09-19T06:04:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.855+0000 7f5dfbff0740 1 -- 172.21.15.5:0/2394908719 wait complete. 2024-09-19T06:04:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.855+0000 7f5dfbff0740 20 client.15783 trim_cache size 0 max 16384 2024-09-19T06:04:56.873 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:04:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.856+0000 7f5df9c17640 1 -- 172.21.15.5:0/2189891353 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5ddc0a33b0 msgr2=0x7f5ddc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.856+0000 7f5df9c17640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5ddc0a33b0 0x7f5ddc0a5870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0ce0f0 tx=0x7f5dec091b10 comp rx=0 tx=0).stop 2024-09-19T06:04:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.856+0000 7f5df9c17640 1 -- 172.21.15.5:0/2189891353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df410e4d0 msgr2=0x7f5df419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.856+0000 7f5df9c17640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df410e4d0 0x7f5df419ef60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00eb10 tx=0x7f5dec00c2d0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.856+0000 7f5df9c17640 1 -- 172.21.15.5:0/2189891353 shutdown_connections 2024-09-19T06:04:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.856+0000 7f5df9c17640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5ddc0a33b0 0x7f5ddc0a5870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.856+0000 7f5df9c17640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df410e4d0 0x7f5df419ef60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.856+0000 7f5df9c17640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df4101360 0x7f5df419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.856+0000 7f5df9c17640 1 --2- 172.21.15.5:0/2189891353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5df41009b0 0x7f5df419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.856+0000 7f5df9c17640 1 -- 172.21.15.5:0/2189891353 >> 172.21.15.5:0/2189891353 conn(0x7f5df40fc7d0 msgr2=0x7f5df40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.856+0000 7f5df9c17640 1 -- 172.21.15.5:0/2189891353 shutdown_connections 2024-09-19T06:04:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:56.856+0000 7f5df9c17640 1 -- 172.21.15.5:0/2189891353 wait complete. 2024-09-19T06:04:57.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:04:57.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 12875 3.66 MB/sec warmup 23 sec latency 12.609 ms 2024-09-19T06:04:57.959 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:04:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.450+0000 7fa3660b7640 1 -- 172.21.15.5:0/4162398800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360100e20 msgr2=0x7fa360101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.450+0000 7fa3660b7640 1 --2- 172.21.15.5:0/4162398800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360100e20 0x7fa360101200 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa350009900 tx=0x7fa35002a480 comp rx=0 tx=0).stop 2024-09-19T06:04:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.450+0000 7fa3660b7640 1 -- 172.21.15.5:0/4162398800 shutdown_connections 2024-09-19T06:04:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.450+0000 7fa3660b7640 1 --2- 172.21.15.5:0/4162398800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa360102ef0 0x7fa36010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.450+0000 7fa3660b7640 1 --2- 172.21.15.5:0/4162398800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa360102570 0x7fa3601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.450+0000 7fa3660b7640 1 --2- 172.21.15.5:0/4162398800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360100e20 0x7fa360101200 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.450+0000 7fa3660b7640 1 -- 172.21.15.5:0/4162398800 >> 172.21.15.5:0/4162398800 conn(0x7fa3600fc7d0 msgr2=0x7fa3600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.450+0000 7fa3660b7640 1 -- 172.21.15.5:0/4162398800 shutdown_connections 2024-09-19T06:04:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.450+0000 7fa3660b7640 1 -- 172.21.15.5:0/4162398800 wait complete. 2024-09-19T06:04:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.450+0000 7fa3660b7640 1 Processor -- start 2024-09-19T06:04:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa3660b7640 1 -- start start 2024-09-19T06:04:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa3660b7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa360100e20 0x7fa36019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa3660b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360102570 0x7fa36019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa3660b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa360102ef0 0x7fa36019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa3660b7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa360110e30 con 0x7fa360102570 2024-09-19T06:04:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa3660b7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa360110cb0 con 0x7fa360100e20 2024-09-19T06:04:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa3660b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa360110fb0 con 0x7fa360102ef0 2024-09-19T06:04:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa35effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360102570 0x7fa36019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa35effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360102570 0x7fa36019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42250/0 (socket says 172.21.15.5:42250) 2024-09-19T06:04:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa35effd640 1 -- 172.21.15.5:0/2290422592 learned_addr learned my addr 172.21.15.5:0/2290422592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa35f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa360100e20 0x7fa36019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa35ffff640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa360102ef0 0x7fa36019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa35effd640 1 -- 172.21.15.5:0/2290422592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa360102ef0 msgr2=0x7fa36019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa35effd640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa360102ef0 0x7fa36019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa35effd640 1 -- 172.21.15.5:0/2290422592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa360100e20 msgr2=0x7fa36019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa35effd640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa360100e20 0x7fa36019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.451+0000 7fa35effd640 1 -- 172.21.15.5:0/2290422592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36019f610 con 0x7fa360102570 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.452+0000 7fa35effd640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360102570 0x7fa36019aba0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa35400d770 tx=0x7fa35400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.452+0000 7fa35cff9640 1 -- 172.21.15.5:0/2290422592 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354017070 con 0x7fa360102570 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.452+0000 7fa35cff9640 1 -- 172.21.15.5:0/2290422592 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3540054d0 con 0x7fa360102570 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.452+0000 7fa3660b7640 1 -- 172.21.15.5:0/2290422592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36019f900 con 0x7fa360102570 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.452+0000 7fa35cff9640 1 -- 172.21.15.5:0/2290422592 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354004cf0 con 0x7fa360102570 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.453+0000 7fa3660b7640 1 -- 172.21.15.5:0/2290422592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3601a71e0 con 0x7fa360102570 2024-09-19T06:04:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.454+0000 7fa35cff9640 1 -- 172.21.15.5:0/2290422592 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa354004e90 con 0x7fa360102570 2024-09-19T06:04:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.455+0000 7fa35cff9640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa3480a33b0 0x7fa3480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.455+0000 7fa368490740 1 Processor -- start 2024-09-19T06:04:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35f7fe640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa3480a33b0 0x7fa3480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa368490740 1 -- start start 2024-09-19T06:04:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35cff9640 1 -- 172.21.15.5:0/2290422592 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fa3540cf380 con 0x7fa360102570 2024-09-19T06:04:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa368490740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa368490740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa368490740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2ee370 0x55fe5c2f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa368490740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fe5c2492f0 con 0x55fe5c2e4b40 2024-09-19T06:04:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa368490740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fe5c1f0cf0 con 0x55fe5c2e35d0 2024-09-19T06:04:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa368490740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fe5c2452b0 con 0x55fe5c2ee370 2024-09-19T06:04:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2e3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54350/0 (socket says 172.21.15.5:54350) 2024-09-19T06:04:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2ee370 0x55fe5c2f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35effd640 1 -- 172.21.15.5:0/2703377731 learned_addr learned my addr 172.21.15.5:0/2703377731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35f7fe640 1 --2- 172.21.15.5:0/2703377731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35f7fe640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa3480a33b0 0x7fa3480a5870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa350009230 tx=0x7fa350031aa0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35ffff640 1 -- 172.21.15.5:0/2703377731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 msgr2=0x55fe5c2e3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35ffff640 1 --2- 172.21.15.5:0/2703377731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2e3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35ffff640 1 -- 172.21.15.5:0/2703377731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2e4b40 msgr2=0x55fe5c2e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35ffff640 1 --2- 172.21.15.5:0/2703377731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.456+0000 7fa35ffff640 1 -- 172.21.15.5:0/2703377731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe5c1d53a0 con 0x55fe5c2ee370 2024-09-19T06:04:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.457+0000 7fa35effd640 1 --2- 172.21.15.5:0/2703377731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.457+0000 7fa35ffff640 1 --2- 172.21.15.5:0/2703377731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2ee370 0x55fe5c2f0760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa358009f00 tx=0x7fa358011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e83684ed85cbb57b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.457+0000 7fa345ffb640 1 -- 172.21.15.5:0/2703377731 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358004d20 con 0x55fe5c2ee370 2024-09-19T06:04:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.457+0000 7fa345ffb640 1 -- 172.21.15.5:0/2703377731 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa358004050 con 0x55fe5c2ee370 2024-09-19T06:04:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.457+0000 7fa345ffb640 1 -- 172.21.15.5:0/2703377731 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358005280 con 0x55fe5c2ee370 2024-09-19T06:04:58.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.458+0000 7fa368490740 1 -- 172.21.15.5:0/2703377731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2ee370 msgr2=0x55fe5c2f0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:58.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.458+0000 7fa368490740 1 --2- 172.21.15.5:0/2703377731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2ee370 0x55fe5c2f0760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa358009f00 tx=0x7fa358011810 comp rx=0 tx=0).stop 2024-09-19T06:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.458+0000 7fa368490740 1 -- 172.21.15.5:0/2703377731 shutdown_connections 2024-09-19T06:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.458+0000 7fa368490740 1 --2- 172.21.15.5:0/2703377731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2ee370 0x55fe5c2f0760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.458+0000 7fa368490740 1 --2- 172.21.15.5:0/2703377731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.458+0000 7fa368490740 1 --2- 172.21.15.5:0/2703377731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.458+0000 7fa368490740 1 -- 172.21.15.5:0/2703377731 >> 172.21.15.5:0/2703377731 conn(0x55fe5c2ac310 msgr2=0x55fe5c2dfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.458+0000 7fa368490740 1 -- 172.21.15.5:0/2703377731 shutdown_connections 2024-09-19T06:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.458+0000 7fa368490740 1 -- 172.21.15.5:0/2703377731 wait complete. 2024-09-19T06:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.458+0000 7fa368490740 1 -- start start 2024-09-19T06:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa368490740 1 Processor -- start 2024-09-19T06:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa368490740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2edd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa368490740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2eafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa368490740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 0x55fe5c2eb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa368490740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fe5c2f1070 con 0x55fe5c2ee370 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa368490740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fe5c204060 con 0x55fe5c2e35d0 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa368490740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fe5c2f11f0 con 0x55fe5c2e4b40 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa35ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 0x55fe5c2eb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa35ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 0x55fe5c2eb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42264/0 (socket says 172.21.15.5:42264) 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa368490740 10 client.0 _ll_register_callbacks cb 0x55fe5bf578c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa35ffff640 1 -- 172.21.15.5:0/242779548 learned_addr learned my addr 172.21.15.5:0/242779548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa35f7fe640 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2edd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa35effd640 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2eafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa35ffff640 1 -- 172.21.15.5:0/242779548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2e4b40 msgr2=0x55fe5c2eafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa35ffff640 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2eafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa35ffff640 1 -- 172.21.15.5:0/242779548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 msgr2=0x55fe5c2edd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa35ffff640 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2edd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.459+0000 7fa35ffff640 1 -- 172.21.15.5:0/242779548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe5bfba3c0 con 0x55fe5c2ee370 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.460+0000 7fa35ffff640 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 0x55fe5c2eb5d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa358000f80 tx=0x7fa358004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc59285d91709390 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.460+0000 7fa33effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.460+0000 7fa33effd640 1 -- 172.21.15.5:0/242779548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358028020 con 0x55fe5c2ee370 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.460+0000 7fa33effd640 1 -- 172.21.15.5:0/242779548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fa358004c50 con 0x55fe5c2ee370 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.460+0000 7fa368490740 1 -- 172.21.15.5:0/242779548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa358004050 con 0x55fe5c2ee370 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.460+0000 7fa33effd640 1 -- 172.21.15.5:0/242779548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa358033050 con 0x55fe5c2ee370 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.460+0000 7fa368490740 10 client.15795 fetch_fsmap learned FSMap version 30 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.460+0000 7fa368490740 1 -- 172.21.15.5:0/242779548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fe5c2fcad0 con 0x55fe5c2ee370 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.461+0000 7fa33effd640 1 -- 172.21.15.5:0/242779548 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa358004050 con 0x55fe5c2ee370 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.461+0000 7fa33effd640 1 -- 172.21.15.5:0/242779548 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa358023450 con 0x55fe5c2ee370 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.461+0000 7fa368490740 10 client.15795 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:04:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.461+0000 7fa368490740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.461+0000 7fa368490740 20 client.15795 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.461+0000 7fa368490740 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fe5c2fe3a0 0x55fe5c300800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.461+0000 7fa368490740 4 client.15795 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:04:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.461+0000 7fa368490740 1 -- 172.21.15.5:0/242779548 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe5c256860 con 0x55fe5c2fe3a0 2024-09-19T06:04:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.461+0000 7fa35f7fe640 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fe5c2fe3a0 0x55fe5c300800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.462+0000 7fa33effd640 10 client.15795 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.462+0000 7fa35f7fe640 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fe5c2fe3a0 0x55fe5c300800 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5823f5e7e1663c7 server_cookie=f5db0a9e27c608e in_seq=0 out_seq=0 2024-09-19T06:04:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.462+0000 7fa33effd640 1 -- 172.21.15.5:0/242779548 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe5c256860 con 0x55fe5c2fe3a0 2024-09-19T06:04:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.462+0000 7fa33effd640 10 client.15795 handle_command_reply: tid=0 2024-09-19T06:04:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.462+0000 7fa368490740 1 client.15795 shutdown 2024-09-19T06:04:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.463+0000 7fa368490740 1 -- 172.21.15.5:0/242779548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 msgr2=0x55fe5c2eb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.463+0000 7fa368490740 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 0x55fe5c2eb5d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa358000f80 tx=0x7fa358004430 comp rx=0 tx=0).stop 2024-09-19T06:04:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.463+0000 7fa368490740 1 -- 172.21.15.5:0/242779548 shutdown_connections 2024-09-19T06:04:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.463+0000 7fa368490740 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fe5c2fe3a0 0x55fe5c300800 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.463+0000 7fa368490740 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 0x55fe5c2eb5d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.463+0000 7fa368490740 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2eafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.463+0000 7fa368490740 1 --2- 172.21.15.5:0/242779548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2edd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.463+0000 7fa368490740 1 -- 172.21.15.5:0/242779548 >> 172.21.15.5:0/242779548 conn(0x55fe5c2ac310 msgr2=0x55fe5c2e5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.463+0000 7fa33effd640 0 client.15795 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.463+0000 7fa368490740 1 -- 172.21.15.5:0/242779548 shutdown_connections 2024-09-19T06:04:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.463+0000 7fa368490740 1 -- 172.21.15.5:0/242779548 wait complete. 2024-09-19T06:04:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.463+0000 7fa368490740 20 client.15795 trim_cache size 0 max 16384 2024-09-19T06:04:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa368490740 1 Processor -- start 2024-09-19T06:04:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa368490740 1 -- start start 2024-09-19T06:04:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa368490740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2fa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa368490740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2daad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa368490740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2ee370 0x55fe5c2e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa368490740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601af990 con 0x55fe5c2e4b40 2024-09-19T06:04:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa368490740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601b1530 con 0x55fe5c2e35d0 2024-09-19T06:04:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa368490740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601b0060 con 0x55fe5c2ee370 2024-09-19T06:04:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa35effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2daad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa35f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2fa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa35effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2daad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42272/0 (socket says 172.21.15.5:42272) 2024-09-19T06:04:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa35effd640 1 -- 172.21.15.5:0/2218031307 learned_addr learned my addr 172.21.15.5:0/2218031307 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa35ffff640 1 --2- 172.21.15.5:0/2218031307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2ee370 0x55fe5c2e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa35f7fe640 1 -- 172.21.15.5:0/2218031307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2ee370 msgr2=0x55fe5c2e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa35f7fe640 1 --2- 172.21.15.5:0/2218031307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2ee370 0x55fe5c2e98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa35f7fe640 1 -- 172.21.15.5:0/2218031307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2e4b40 msgr2=0x55fe5c2daad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa35f7fe640 1 --2- 172.21.15.5:0/2218031307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2daad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa35f7fe640 1 -- 172.21.15.5:0/2218031307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe5c2ee0a0 con 0x55fe5c2e35d0 2024-09-19T06:04:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.477+0000 7fa35effd640 1 --2- 172.21.15.5:0/2218031307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2daad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.478+0000 7fa35f7fe640 1 --2- 172.21.15.5:0/2218031307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2fa310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa350006010 tx=0x7fa35003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e47c0d07868ce7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.478+0000 7fa3457fa640 1 -- 172.21.15.5:0/2218031307 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350035040 con 0x55fe5c2e35d0 2024-09-19T06:04:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.478+0000 7fa3457fa640 1 -- 172.21.15.5:0/2218031307 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa350004490 con 0x55fe5c2e35d0 2024-09-19T06:04:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.478+0000 7fa3457fa640 1 -- 172.21.15.5:0/2218031307 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3500538f0 con 0x55fe5c2e35d0 2024-09-19T06:04:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.478+0000 7fa368490740 1 -- 172.21.15.5:0/2218031307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 msgr2=0x55fe5c2fa310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.478+0000 7fa368490740 1 --2- 172.21.15.5:0/2218031307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2fa310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa350006010 tx=0x7fa35003f000 comp rx=0 tx=0).stop 2024-09-19T06:04:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 -- 172.21.15.5:0/2218031307 shutdown_connections 2024-09-19T06:04:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 --2- 172.21.15.5:0/2218031307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2ee370 0x55fe5c2e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 --2- 172.21.15.5:0/2218031307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2daad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 --2- 172.21.15.5:0/2218031307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2fa310 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 -- 172.21.15.5:0/2218031307 >> 172.21.15.5:0/2218031307 conn(0x55fe5c2ac310 msgr2=0x55fe5c2eafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 -- 172.21.15.5:0/2218031307 shutdown_connections 2024-09-19T06:04:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 -- 172.21.15.5:0/2218031307 wait complete. 2024-09-19T06:04:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 -- start start 2024-09-19T06:04:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 Processor -- start 2024-09-19T06:04:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2f1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2f1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 0x55fe5c2f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601b0e00 con 0x55fe5c2ee370 2024-09-19T06:04:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601b0730 con 0x55fe5c2e35d0 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601ad970 con 0x55fe5c2e4b40 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.479+0000 7fa368490740 10 client.0 _ll_register_callbacks cb 0x7fa350009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 0x55fe5c2f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2f1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40138/0 (socket says 172.21.15.5:40138) 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54378/0 (socket says 172.21.15.5:54378) 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 0x55fe5c2f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42280/0 (socket says 172.21.15.5:42280) 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35effd640 1 -- 172.21.15.5:0/2357055919 learned_addr learned my addr 172.21.15.5:0/2357055919 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35ffff640 1 -- 172.21.15.5:0/2357055919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2e4b40 msgr2=0x55fe5c2f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35ffff640 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35ffff640 1 -- 172.21.15.5:0/2357055919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 msgr2=0x55fe5c2f1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:04:58.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35ffff640 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2f1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35ffff640 1 -- 172.21.15.5:0/2357055919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe5c3670d0 con 0x55fe5c2ee370 2024-09-19T06:04:58.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35f7fe640 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:58.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35effd640 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:04:58.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa35ffff640 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 0x55fe5c2f1ec0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa358023a10 tx=0x7fa358023fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b5c3174161dcb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:04:58.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa33effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:04:58.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa33effd640 1 -- 172.21.15.5:0/2357055919 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358028040 con 0x55fe5c2ee370 2024-09-19T06:04:58.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa368490740 1 -- 172.21.15.5:0/2357055919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa350004490 con 0x55fe5c2ee370 2024-09-19T06:04:58.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.480+0000 7fa33effd640 1 -- 172.21.15.5:0/2357055919 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fa3580044a0 con 0x55fe5c2ee370 2024-09-19T06:04:58.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.481+0000 7fa33effd640 1 -- 172.21.15.5:0/2357055919 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa358045050 con 0x55fe5c2ee370 2024-09-19T06:04:58.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.481+0000 7fa368490740 10 client.15801 fetch_fsmap learned FSMap version 30 2024-09-19T06:04:58.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.481+0000 7fa368490740 1 -- 172.21.15.5:0/2357055919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fe5c3682e0 con 0x55fe5c2ee370 2024-09-19T06:04:58.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.481+0000 7fa33effd640 1 -- 172.21.15.5:0/2357055919 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa350004490 con 0x55fe5c2ee370 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.481+0000 7fa33effd640 1 -- 172.21.15.5:0/2357055919 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa3580378e0 con 0x55fe5c2ee370 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.481+0000 7fa368490740 10 client.15801 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.481+0000 7fa368490740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.481+0000 7fa368490740 20 client.15801 populate_metadata read hostname 'smithi005' 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.481+0000 7fa368490740 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fe5c3689c0 0x55fe5c36ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.481+0000 7fa368490740 4 client.15801 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.481+0000 7fa368490740 1 -- 172.21.15.5:0/2357055919 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe5c250d70 con 0x55fe5c3689c0 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.481+0000 7fa35f7fe640 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fe5c3689c0 0x55fe5c36ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa33effd640 10 client.15801 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa35f7fe640 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fe5c3689c0 0x55fe5c36ade0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c7ee8ce2e0ddb2 server_cookie=3e9b1fb77a12d50f in_seq=0 out_seq=0 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa33effd640 1 -- 172.21.15.5:0/2357055919 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe5c250d70 con 0x55fe5c3689c0 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa33effd640 10 client.15801 handle_command_reply: tid=0 2024-09-19T06:04:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa368490740 1 client.15801 shutdown 2024-09-19T06:04:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa368490740 1 -- 172.21.15.5:0/2357055919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 msgr2=0x55fe5c2f1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa368490740 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 0x55fe5c2f1ec0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa358023a10 tx=0x7fa358023fa0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa368490740 1 -- 172.21.15.5:0/2357055919 shutdown_connections 2024-09-19T06:04:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa368490740 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fe5c3689c0 0x55fe5c36ade0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa368490740 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe5c2ee370 0x55fe5c2f1ec0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa33effd640 0 client.15801 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:04:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa368490740 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe5c2e4b40 0x55fe5c2f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa368490740 1 --2- 172.21.15.5:0/2357055919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fe5c2e35d0 0x55fe5c2f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa368490740 1 -- 172.21.15.5:0/2357055919 >> 172.21.15.5:0/2357055919 conn(0x55fe5c2ac310 msgr2=0x55fe5c2e9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa368490740 1 -- 172.21.15.5:0/2357055919 shutdown_connections 2024-09-19T06:04:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa368490740 1 -- 172.21.15.5:0/2357055919 wait complete. 2024-09-19T06:04:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.482+0000 7fa368490740 20 client.15801 trim_cache size 0 max 16384 2024-09-19T06:04:58.490 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:04:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.483+0000 7fa3660b7640 1 -- 172.21.15.5:0/2290422592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa3480a33b0 msgr2=0x7fa3480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.483+0000 7fa3660b7640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa3480a33b0 0x7fa3480a5870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa350009230 tx=0x7fa350031aa0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.483+0000 7fa3660b7640 1 -- 172.21.15.5:0/2290422592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360102570 msgr2=0x7fa36019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:04:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.483+0000 7fa3660b7640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360102570 0x7fa36019aba0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa35400d770 tx=0x7fa35400dc30 comp rx=0 tx=0).stop 2024-09-19T06:04:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.483+0000 7fa3660b7640 1 -- 172.21.15.5:0/2290422592 shutdown_connections 2024-09-19T06:04:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.483+0000 7fa3660b7640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa3480a33b0 0x7fa3480a5870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.483+0000 7fa3660b7640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa360102ef0 0x7fa36019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.483+0000 7fa3660b7640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360102570 0x7fa36019aba0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.483+0000 7fa3660b7640 1 --2- 172.21.15.5:0/2290422592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa360100e20 0x7fa36019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:04:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.483+0000 7fa3660b7640 1 -- 172.21.15.5:0/2290422592 >> 172.21.15.5:0/2290422592 conn(0x7fa3600fc7d0 msgr2=0x7fa36006a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:04:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.483+0000 7fa3660b7640 1 -- 172.21.15.5:0/2290422592 shutdown_connections 2024-09-19T06:04:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:04:58.483+0000 7fa3660b7640 1 -- 172.21.15.5:0/2290422592 wait complete. 2024-09-19T06:04:58.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:04:58 smithi005 ceph-mon[30955]: pgmap v51: 97 pgs: 97 active+clean; 29 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-09-19T06:04:58.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:04:58 smithi049 ceph-mon[34640]: pgmap v51: 97 pgs: 97 active+clean; 29 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-09-19T06:04:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:04:58 smithi157 ceph-mon[35346]: pgmap v51: 97 pgs: 97 active+clean; 29 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-09-19T06:04:58.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 13419 3.72 MB/sec warmup 24 sec latency 14.955 ms 2024-09-19T06:04:59.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 13992 3.71 MB/sec warmup 25 sec latency 16.740 ms 2024-09-19T06:05:00.195 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:05:00.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:05:00.484 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:05:00.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:00 smithi005 ceph-mon[30955]: pgmap v52: 97 pgs: 97 active+clean; 27 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-19T06:05:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:00 smithi049 ceph-mon[34640]: pgmap v52: 97 pgs: 97 active+clean; 27 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-19T06:05:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:00 smithi157 ceph-mon[35346]: pgmap v52: 97 pgs: 97 active+clean; 27 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-19T06:05:00.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14476 3.74 MB/sec warmup 26 sec latency 18.408 ms 2024-09-19T06:05:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.062+0000 7f63e00f3640 1 -- 172.21.15.5:0/3696621819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63d8101df0 msgr2=0x7f63d81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.062+0000 7f63e00f3640 1 --2- 172.21.15.5:0/3696621819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63d8101df0 0x7f63d81021d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009900 tx=0x7f63d402a410 comp rx=0 tx=0).stop 2024-09-19T06:05:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.062+0000 7f63e00f3640 1 -- 172.21.15.5:0/3696621819 shutdown_connections 2024-09-19T06:05:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.062+0000 7f63e00f3640 1 --2- 172.21.15.5:0/3696621819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d8106f20 0x7f63d810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.062+0000 7f63e00f3640 1 --2- 172.21.15.5:0/3696621819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63d81027a0 0x7f63d81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.062+0000 7f63e00f3640 1 --2- 172.21.15.5:0/3696621819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63d8101df0 0x7f63d81021d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.062+0000 7f63e00f3640 1 -- 172.21.15.5:0/3696621819 >> 172.21.15.5:0/3696621819 conn(0x7f63d80fd8f0 msgr2=0x7f63d80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.062+0000 7f63e00f3640 1 -- 172.21.15.5:0/3696621819 shutdown_connections 2024-09-19T06:05:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.062+0000 7f63e00f3640 1 -- 172.21.15.5:0/3696621819 wait complete. 2024-09-19T06:05:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.063+0000 7f63e00f3640 1 Processor -- start 2024-09-19T06:05:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.063+0000 7f63e00f3640 1 -- start start 2024-09-19T06:05:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.063+0000 7f63e00f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63d8101df0 0x7f63d8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.063+0000 7f63e00f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d81027a0 0x7f63d81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.063+0000 7f63e00f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63d8106f20 0x7f63d819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.063+0000 7f63e00f3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63d8111df0 con 0x7f63d8106f20 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.063+0000 7f63e00f3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63d8111c70 con 0x7f63d81027a0 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.063+0000 7f63e00f3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f63d8111f70 con 0x7f63d8101df0 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.063+0000 7f63de690640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63d8106f20 0x7f63d819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.063+0000 7f63de690640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63d8106f20 0x7f63d819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42296/0 (socket says 172.21.15.5:42296) 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.063+0000 7f63de690640 1 -- 172.21.15.5:0/1098268101 learned_addr learned my addr 172.21.15.5:0/1098268101 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63dde8f640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63d8101df0 0x7f63d8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63dd68e640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d81027a0 0x7f63d81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63de690640 1 -- 172.21.15.5:0/1098268101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63d8101df0 msgr2=0x7f63d8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63de690640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63d8101df0 0x7f63d8198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63de690640 1 -- 172.21.15.5:0/1098268101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d81027a0 msgr2=0x7f63d81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63de690640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d81027a0 0x7f63d81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63de690640 1 -- 172.21.15.5:0/1098268101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d819d340 con 0x7f63d8106f20 2024-09-19T06:05:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63de690640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63d8106f20 0x7f63d819cc60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f63d000eaa0 tx=0x7f63d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63ceffd640 1 -- 172.21.15.5:0/1098268101 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d000cdb0 con 0x7f63d8106f20 2024-09-19T06:05:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63e00f3640 1 -- 172.21.15.5:0/1098268101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d819d630 con 0x7f63d8106f20 2024-09-19T06:05:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63ceffd640 1 -- 172.21.15.5:0/1098268101 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63d0005020 con 0x7f63d8106f20 2024-09-19T06:05:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63ceffd640 1 -- 172.21.15.5:0/1098268101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d00116d0 con 0x7f63d8106f20 2024-09-19T06:05:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.064+0000 7f63e00f3640 1 -- 172.21.15.5:0/1098268101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d81a4f10 con 0x7f63d8106f20 2024-09-19T06:05:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.067+0000 7f63ceffd640 1 -- 172.21.15.5:0/1098268101 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f63d00051d0 con 0x7f63d8106f20 2024-09-19T06:05:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63e24cc740 1 Processor -- start 2024-09-19T06:05:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63e24cc740 1 -- start start 2024-09-19T06:05:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63e24cc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800abb60 0x5580800abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63e24cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800ac480 0x5580800b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63e24cc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800b6d40 0x5580800b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63e24cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558080011280 con 0x5580800ac480 2024-09-19T06:05:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63e24cc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55807ffb8b10 con 0x5580800abb60 2024-09-19T06:05:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63e24cc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55808000d240 con 0x5580800b6d40 2024-09-19T06:05:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63dde8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800ac480 0x5580800b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63ceffd640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63c40a33b0 0x7f63c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63dde8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800ac480 0x5580800b6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42302/0 (socket says 172.21.15.5:42302) 2024-09-19T06:05:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63dde8f640 1 -- 172.21.15.5:0/2155139721 learned_addr learned my addr 172.21.15.5:0/2155139721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63dd68e640 1 --2- 172.21.15.5:0/2155139721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800abb60 0x5580800abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.068+0000 7f63dd68e640 1 --2- 172.21.15.5:0/2155139721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800b6d40 0x5580800b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.069+0000 7f63de690640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63c40a33b0 0x7f63c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.069+0000 7f63ceffd640 1 -- 172.21.15.5:0/1098268101 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f63d00cfda0 con 0x7f63d8106f20 2024-09-19T06:05:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.069+0000 7f63dd68e640 1 -- 172.21.15.5:0/2155139721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800b6d40 msgr2=0x5580800b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.069+0000 7f63dd68e640 1 --2- 172.21.15.5:0/2155139721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800b6d40 0x5580800b9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.069+0000 7f63dd68e640 1 -- 172.21.15.5:0/2155139721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800ac480 msgr2=0x5580800b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.069+0000 7f63dd68e640 1 --2- 172.21.15.5:0/2155139721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800ac480 0x5580800b6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.069+0000 7f63dd68e640 1 -- 172.21.15.5:0/2155139721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55807ffc51f0 con 0x5580800abb60 2024-09-19T06:05:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.069+0000 7f63dd68e640 1 --2- 172.21.15.5:0/2155139721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800abb60 0x5580800abf40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x5580800a9470 tx=0x7f63c8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69903c365282be22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.069+0000 7f63de690640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63c40a33b0 0x7f63c40a5870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f63d0098b10 tx=0x7f63d00c6e60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.069+0000 7f63c3fff640 1 -- 172.21.15.5:0/2155139721 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c8012ca0 con 0x5580800abb60 2024-09-19T06:05:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.070+0000 7f63c3fff640 1 -- 172.21.15.5:0/2155139721 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63c8012e40 con 0x5580800abb60 2024-09-19T06:05:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.070+0000 7f63c3fff640 1 -- 172.21.15.5:0/2155139721 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c8004a00 con 0x5580800abb60 2024-09-19T06:05:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.070+0000 7f63e24cc740 1 -- 172.21.15.5:0/2155139721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800abb60 msgr2=0x5580800abf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.070+0000 7f63e24cc740 1 --2- 172.21.15.5:0/2155139721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800abb60 0x5580800abf40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x5580800a9470 tx=0x7f63c8004730 comp rx=0 tx=0).stop 2024-09-19T06:05:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.070+0000 7f63e24cc740 1 -- 172.21.15.5:0/2155139721 shutdown_connections 2024-09-19T06:05:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.070+0000 7f63e24cc740 1 --2- 172.21.15.5:0/2155139721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800b6d40 0x5580800b9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.070+0000 7f63e24cc740 1 --2- 172.21.15.5:0/2155139721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800ac480 0x5580800b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.070+0000 7f63e24cc740 1 --2- 172.21.15.5:0/2155139721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800abb60 0x5580800abf40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.070+0000 7f63e24cc740 1 -- 172.21.15.5:0/2155139721 >> 172.21.15.5:0/2155139721 conn(0x5580800a80f0 msgr2=0x5580800a84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.071+0000 7f63e24cc740 1 -- 172.21.15.5:0/2155139721 shutdown_connections 2024-09-19T06:05:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.071+0000 7f63e24cc740 1 -- 172.21.15.5:0/2155139721 wait complete. 2024-09-19T06:05:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.071+0000 7f63e24cc740 1 -- start start 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.071+0000 7f63e24cc740 1 Processor -- start 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.071+0000 7f63e24cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.071+0000 7f63e24cc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 0x5580800b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.071+0000 7f63e24cc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 0x5580800b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.071+0000 7f63e24cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5580800b9910 con 0x5580800abb60 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.071+0000 7f63e24cc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55807ffcbf10 con 0x5580800b6d40 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.071+0000 7f63e24cc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5580800b9a90 con 0x5580800ac480 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63dd68e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63e24cc740 10 client.0 _ll_register_callbacks cb 0x55807ff94a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63dd68e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42308/0 (socket says 172.21.15.5:42308) 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63dd68e640 1 -- 172.21.15.5:0/3614310557 learned_addr learned my addr 172.21.15.5:0/3614310557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63dde8f640 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 0x5580800b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63dd68e640 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 0x5580800b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63dd68e640 1 -- 172.21.15.5:0/3614310557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 msgr2=0x5580800b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63dd68e640 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 0x5580800b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63dd68e640 1 -- 172.21.15.5:0/3614310557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 msgr2=0x5580800b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63dd68e640 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 0x5580800b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63dd68e640 1 -- 172.21.15.5:0/3614310557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55807fd826a0 con 0x5580800abb60 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63dd68e640 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800b3c80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f63c8015040 tx=0x7f63c800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706e094b4e920de5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.072+0000 7f63c17fa640 1 -- 172.21.15.5:0/3614310557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c8010940 con 0x5580800abb60 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.073+0000 7f63e24cc740 1 -- 172.21.15.5:0/3614310557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55808001e7f0 con 0x5580800abb60 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.073+0000 7f63c17fa640 1 -- 172.21.15.5:0/3614310557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f63c8010ae0 con 0x5580800abb60 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.073+0000 7f63c17fa640 1 -- 172.21.15.5:0/3614310557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f63c8030050 con 0x5580800abb60 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.073+0000 7f63e24cc740 10 client.15813 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.073+0000 7f63e24cc740 1 -- 172.21.15.5:0/3614310557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5580800c5570 con 0x5580800abb60 2024-09-19T06:05:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.073+0000 7f63c17fa640 1 -- 172.21.15.5:0/3614310557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55808001e7f0 con 0x5580800abb60 2024-09-19T06:05:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.073+0000 7f63c17fa640 1 -- 172.21.15.5:0/3614310557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f63c801f470 con 0x5580800abb60 2024-09-19T06:05:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.073+0000 7f63e24cc740 10 client.15813 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.073+0000 7f63e24cc740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.073+0000 7f63e24cc740 20 client.15813 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.073+0000 7f63e24cc740 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5580800c68c0 0x5580800c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.074+0000 7f63e24cc740 4 client.15813 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.074+0000 7f63e24cc740 1 -- 172.21.15.5:0/3614310557 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558080018d00 con 0x5580800c68c0 2024-09-19T06:05:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.074+0000 7f63dde8f640 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5580800c68c0 0x5580800c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.074+0000 7f63dde8f640 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5580800c68c0 0x5580800c8cb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=301ec8b22999ce1c server_cookie=d6ff877a683445ef in_seq=0 out_seq=0 2024-09-19T06:05:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.074+0000 7f63c17fa640 10 client.15813 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.075+0000 7f63c17fa640 1 -- 172.21.15.5:0/3614310557 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558080018d00 con 0x5580800c68c0 2024-09-19T06:05:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.075+0000 7f63c17fa640 10 client.15813 handle_command_reply: tid=0 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.075+0000 7f63e24cc740 1 client.15813 shutdown 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.075+0000 7f63e24cc740 1 -- 172.21.15.5:0/3614310557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 msgr2=0x5580800b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.075+0000 7f63e24cc740 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800b3c80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f63c8015040 tx=0x7f63c800b410 comp rx=0 tx=0).stop 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.075+0000 7f63e24cc740 1 -- 172.21.15.5:0/3614310557 shutdown_connections 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.075+0000 7f63e24cc740 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5580800c68c0 0x5580800c8cb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.075+0000 7f63e24cc740 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 0x5580800b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.075+0000 7f63e24cc740 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 0x5580800b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.075+0000 7f63e24cc740 1 --2- 172.21.15.5:0/3614310557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800b3c80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.075+0000 7f63c17fa640 0 client.15813 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.075+0000 7f63e24cc740 1 -- 172.21.15.5:0/3614310557 >> 172.21.15.5:0/3614310557 conn(0x5580800a80f0 msgr2=0x5580800ade20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.076+0000 7f63e24cc740 1 -- 172.21.15.5:0/3614310557 shutdown_connections 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.076+0000 7f63e24cc740 1 -- 172.21.15.5:0/3614310557 wait complete. 2024-09-19T06:05:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.076+0000 7f63e24cc740 20 client.15813 trim_cache size 0 max 16384 2024-09-19T06:05:01.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63e24cc740 1 Processor -- start 2024-09-19T06:05:01.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63e24cc740 1 -- start start 2024-09-19T06:05:01.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63e24cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63e24cc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 0x5580800ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63e24cc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 0x5580800b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63e24cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63d81ad8f0 con 0x5580800abb60 2024-09-19T06:05:01.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63e24cc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63d81af490 con 0x5580800b6d40 2024-09-19T06:05:01.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63e24cc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f63d81adfc0 con 0x5580800ac480 2024-09-19T06:05:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63dd68e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63dd68e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42312/0 (socket says 172.21.15.5:42312) 2024-09-19T06:05:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63dd68e640 1 -- 172.21.15.5:0/1434688821 learned_addr learned my addr 172.21.15.5:0/1434688821 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63dd68e640 1 --2- 172.21.15.5:0/1434688821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 0x5580800b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63dde8f640 1 --2- 172.21.15.5:0/1434688821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 0x5580800ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63dd68e640 1 -- 172.21.15.5:0/1434688821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 msgr2=0x5580800ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63dd68e640 1 --2- 172.21.15.5:0/1434688821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 0x5580800ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63dd68e640 1 -- 172.21.15.5:0/1434688821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 msgr2=0x5580800b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63dd68e640 1 --2- 172.21.15.5:0/1434688821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 0x5580800b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63dd68e640 1 -- 172.21.15.5:0/1434688821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580800b6300 con 0x5580800abb60 2024-09-19T06:05:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.095+0000 7f63dde8f640 1 --2- 172.21.15.5:0/1434688821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 0x5580800ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.096+0000 7f63dd68e640 1 --2- 172.21.15.5:0/1434688821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800ce420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f63c80045e0 tx=0x7f63c8005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcadceca7b133b5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.096+0000 7f63c37fe640 1 -- 172.21.15.5:0/1434688821 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c8005790 con 0x5580800abb60 2024-09-19T06:05:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.096+0000 7f63c37fe640 1 -- 172.21.15.5:0/1434688821 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63c8016040 con 0x5580800abb60 2024-09-19T06:05:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.096+0000 7f63c37fe640 1 -- 172.21.15.5:0/1434688821 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c80035f0 con 0x5580800abb60 2024-09-19T06:05:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.097+0000 7f63e24cc740 1 -- 172.21.15.5:0/1434688821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 msgr2=0x5580800ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.097+0000 7f63e24cc740 1 --2- 172.21.15.5:0/1434688821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800ce420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f63c80045e0 tx=0x7f63c8005fb0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.097+0000 7f63e24cc740 1 -- 172.21.15.5:0/1434688821 shutdown_connections 2024-09-19T06:05:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.097+0000 7f63e24cc740 1 --2- 172.21.15.5:0/1434688821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 0x5580800b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.097+0000 7f63e24cc740 1 --2- 172.21.15.5:0/1434688821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 0x5580800ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.097+0000 7f63e24cc740 1 --2- 172.21.15.5:0/1434688821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800ce420 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.097+0000 7f63e24cc740 1 -- 172.21.15.5:0/1434688821 >> 172.21.15.5:0/1434688821 conn(0x5580800a80f0 msgr2=0x5580800b5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.097+0000 7f63e24cc740 1 -- 172.21.15.5:0/1434688821 shutdown_connections 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.097+0000 7f63e24cc740 1 -- 172.21.15.5:0/1434688821 wait complete. 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.097+0000 7f63e24cc740 1 -- start start 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.097+0000 7f63e24cc740 1 Processor -- start 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63e24cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63e24cc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 0x5580800bb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63e24cc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 0x5580800bb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63e24cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63d81aed60 con 0x5580800abb60 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63e24cc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63d81ae690 con 0x5580800b6d40 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63e24cc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f63d81ab8d0 con 0x5580800ac480 2024-09-19T06:05:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63e24cc740 10 client.0 _ll_register_callbacks cb 0x7f63d4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63dd68e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63dd68e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800bad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42322/0 (socket says 172.21.15.5:42322) 2024-09-19T06:05:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63dd68e640 1 -- 172.21.15.5:0/1965778653 learned_addr learned my addr 172.21.15.5:0/1965778653 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63dd68e640 1 -- 172.21.15.5:0/1965778653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 msgr2=0x5580800bb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63dd68e640 1 --2- 172.21.15.5:0/1965778653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 0x5580800bb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63dd68e640 1 -- 172.21.15.5:0/1965778653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 msgr2=0x5580800bb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63dd68e640 1 --2- 172.21.15.5:0/1965778653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 0x5580800bb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63dd68e640 1 -- 172.21.15.5:0/1965778653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580801306e0 con 0x5580800abb60 2024-09-19T06:05:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63dd68e640 1 --2- 172.21.15.5:0/1965778653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800bad40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f63c8005d00 tx=0x7f63c8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1a3fbeba38fce4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.098+0000 7f63c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:05:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63c17fa640 1 -- 172.21.15.5:0/1965778653 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c801fac0 con 0x5580800abb60 2024-09-19T06:05:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63e24cc740 1 -- 172.21.15.5:0/1965778653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63c8016040 con 0x5580800abb60 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63c17fa640 1 -- 172.21.15.5:0/1965778653 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f63c801f470 con 0x5580800abb60 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63c17fa640 1 -- 172.21.15.5:0/1965778653 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f63c8049050 con 0x5580800abb60 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63e24cc740 10 client.15819 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63e24cc740 1 -- 172.21.15.5:0/1965778653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558080131990 con 0x5580800abb60 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63c17fa640 1 -- 172.21.15.5:0/1965778653 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63c8016040 con 0x5580800abb60 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63c17fa640 1 -- 172.21.15.5:0/1965778653 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f63c8004660 con 0x5580800abb60 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63e24cc740 10 client.15819 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63e24cc740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63e24cc740 20 client.15819 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63e24cc740 1 --2- 172.21.15.5:0/1965778653 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5580801320a0 0x5580801344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63e24cc740 4 client.15819 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.099+0000 7f63e24cc740 1 -- 172.21.15.5:0/1965778653 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55807ff95110 con 0x5580801320a0 2024-09-19T06:05:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.100+0000 7f63dde8f640 1 --2- 172.21.15.5:0/1965778653 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5580801320a0 0x5580801344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.100+0000 7f63dde8f640 1 --2- 172.21.15.5:0/1965778653 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5580801320a0 0x5580801344b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b0896f135c2cf1 server_cookie=bc3c2e56351e76c7 in_seq=0 out_seq=0 2024-09-19T06:05:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.100+0000 7f63c17fa640 10 client.15819 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63c17fa640 1 -- 172.21.15.5:0/1965778653 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55807ff95110 con 0x5580801320a0 2024-09-19T06:05:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63c17fa640 10 client.15819 handle_command_reply: tid=0 2024-09-19T06:05:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63e24cc740 1 client.15819 shutdown 2024-09-19T06:05:01.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63e24cc740 1 -- 172.21.15.5:0/1965778653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 msgr2=0x5580800bad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:01.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63e24cc740 1 --2- 172.21.15.5:0/1965778653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800bad40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f63c8005d00 tx=0x7f63c8005800 comp rx=0 tx=0).stop 2024-09-19T06:05:01.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63e24cc740 1 -- 172.21.15.5:0/1965778653 shutdown_connections 2024-09-19T06:05:01.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63e24cc740 1 --2- 172.21.15.5:0/1965778653 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5580801320a0 0x5580801344b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63e24cc740 1 --2- 172.21.15.5:0/1965778653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5580800b6d40 0x5580800bb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63e24cc740 1 --2- 172.21.15.5:0/1965778653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580800ac480 0x5580800bb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63e24cc740 1 --2- 172.21.15.5:0/1965778653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580800abb60 0x5580800bad40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63e24cc740 1 -- 172.21.15.5:0/1965778653 >> 172.21.15.5:0/1965778653 conn(0x5580800a80f0 msgr2=0x5580800b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:01.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63c17fa640 0 client.15819 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:01.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63e24cc740 1 -- 172.21.15.5:0/1965778653 shutdown_connections 2024-09-19T06:05:01.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63e24cc740 1 -- 172.21.15.5:0/1965778653 wait complete. 2024-09-19T06:05:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.101+0000 7f63e24cc740 20 client.15819 trim_cache size 0 max 16384 2024-09-19T06:05:01.111 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:05:01.111 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:05:01.111 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "7f045632-17ff-4224-8123-47e7f8902444", 2024-09-19T06:05:01.111 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:05:01.111 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:05:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.102+0000 7f63e00f3640 1 -- 172.21.15.5:0/1098268101 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63c40a33b0 msgr2=0x7f63c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.102+0000 7f63e00f3640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63c40a33b0 0x7f63c40a5870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f63d0098b10 tx=0x7f63d00c6e60 comp rx=0 tx=0).stop 2024-09-19T06:05:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.102+0000 7f63e00f3640 1 -- 172.21.15.5:0/1098268101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63d8106f20 msgr2=0x7f63d819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.102+0000 7f63e00f3640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63d8106f20 0x7f63d819cc60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f63d000eaa0 tx=0x7f63d000ef60 comp rx=0 tx=0).stop 2024-09-19T06:05:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.102+0000 7f63e00f3640 1 -- 172.21.15.5:0/1098268101 shutdown_connections 2024-09-19T06:05:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.102+0000 7f63e00f3640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63d8106f20 0x7f63d819cc60 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.102+0000 7f63e00f3640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d81027a0 0x7f63d81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.102+0000 7f63e00f3640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63c40a33b0 0x7f63c40a5870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.102+0000 7f63e00f3640 1 --2- 172.21.15.5:0/1098268101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63d8101df0 0x7f63d8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.102+0000 7f63e00f3640 1 -- 172.21.15.5:0/1098268101 >> 172.21.15.5:0/1098268101 conn(0x7f63d80fd8f0 msgr2=0x7f63d80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.102+0000 7f63e00f3640 1 -- 172.21.15.5:0/1098268101 shutdown_connections 2024-09-19T06:05:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:01.102+0000 7f63e00f3640 1 -- 172.21.15.5:0/1098268101 wait complete. 2024-09-19T06:05:01.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14751 3.61 MB/sec warmup 27 sec latency 38.900 ms 2024-09-19T06:05:01.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:05:02.131 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:05:02.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: pgmap v53: 97 pgs: 97 active+clean; 27 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 825 KiB/s wr, 98 op/s 2024-09-19T06:05:02.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:05:02.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:02.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:02.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:02.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:02.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:02.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:02.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:02.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:02 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.632+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1037848296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f4106f10 msgr2=0x7ff4f410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:02.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.632+0000 7ff4f9fa3640 1 --2- 172.21.15.5:0/1037848296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f4106f10 0x7ff4f410ea90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009ca0 tx=0x7ff4ec02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.632+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1037848296 shutdown_connections 2024-09-19T06:05:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.632+0000 7ff4f9fa3640 1 --2- 172.21.15.5:0/1037848296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f4106f10 0x7ff4f410ea90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.632+0000 7ff4f9fa3640 1 --2- 172.21.15.5:0/1037848296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f4102790 0x7ff4f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.632+0000 7ff4f9fa3640 1 --2- 172.21.15.5:0/1037848296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4101de0 0x7ff4f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.632+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1037848296 >> 172.21.15.5:0/1037848296 conn(0x7ff4f40fd8c0 msgr2=0x7ff4f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.632+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1037848296 shutdown_connections 2024-09-19T06:05:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.632+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1037848296 wait complete. 2024-09-19T06:05:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.633+0000 7ff4f9fa3640 1 Processor -- start 2024-09-19T06:05:02.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.633+0000 7ff4f9fa3640 1 -- start start 2024-09-19T06:05:02.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f9fa3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f4101de0 0x7ff4f4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f9fa3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4102790 0x7ff4f41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f9fa3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f4106f10 0x7ff4f419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f9fa3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4111e20 con 0x7ff4f4106f10 2024-09-19T06:05:02.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f9fa3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4111ca0 con 0x7ff4f4101de0 2024-09-19T06:05:02.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f9fa3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4111fa0 con 0x7ff4f4102790 2024-09-19T06:05:02.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f4101de0 0x7ff4f4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f4106f10 0x7ff4f419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f4101de0 0x7ff4f4198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54464/0 (socket says 172.21.15.5:54464) 2024-09-19T06:05:02.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f37fe640 1 -- 172.21.15.5:0/1143734748 learned_addr learned my addr 172.21.15.5:0/1143734748 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:02.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f2ffd640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4102790 0x7ff4f41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f37fe640 1 -- 172.21.15.5:0/1143734748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4102790 msgr2=0x7ff4f41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4102790 0x7ff4f41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f37fe640 1 -- 172.21.15.5:0/1143734748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f4106f10 msgr2=0x7ff4f419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f4106f10 0x7ff4f419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f37fe640 1 -- 172.21.15.5:0/1143734748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f419d310 con 0x7ff4f4101de0 2024-09-19T06:05:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f3fff640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f4106f10 0x7ff4f419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.634+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f4101de0 0x7ff4f4198360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400e9f0 tx=0x7ff4e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.635+0000 7ff4f0ff9640 1 -- 172.21.15.5:0/1143734748 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400cd80 con 0x7ff4f4101de0 2024-09-19T06:05:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.635+0000 7ff4f0ff9640 1 -- 172.21.15.5:0/1143734748 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4e40108e0 con 0x7ff4f4101de0 2024-09-19T06:05:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.635+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1143734748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f419d600 con 0x7ff4f4101de0 2024-09-19T06:05:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.635+0000 7ff4f0ff9640 1 -- 172.21.15.5:0/1143734748 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4018920 con 0x7ff4f4101de0 2024-09-19T06:05:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.635+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1143734748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f41a4ee0 con 0x7ff4f4101de0 2024-09-19T06:05:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.638+0000 7ff4f0ff9640 1 -- 172.21.15.5:0/1143734748 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff4e4018ac0 con 0x7ff4f4101de0 2024-09-19T06:05:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.638+0000 7ff4fc37c740 1 Processor -- start 2024-09-19T06:05:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.638+0000 7ff4fc37c740 1 -- start start 2024-09-19T06:05:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.638+0000 7ff4fc37c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055340 0x559bf2055720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.638+0000 7ff4fc37c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055c60 0x559bf2064550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.638+0000 7ff4fc37c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2066e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.638+0000 7ff4fc37c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559bf1fbb2e0 con 0x559bf2064a90 2024-09-19T06:05:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.638+0000 7ff4fc37c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559bf1f62ce0 con 0x559bf2055340 2024-09-19T06:05:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.638+0000 7ff4fc37c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559bf1fb72a0 con 0x559bf2055c60 2024-09-19T06:05:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2066e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2066e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42352/0 (socket says 172.21.15.5:42352) 2024-09-19T06:05:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f2ffd640 1 -- 172.21.15.5:0/2300900409 learned_addr learned my addr 172.21.15.5:0/2300900409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f3fff640 1 --2- 172.21.15.5:0/2300900409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055340 0x559bf2055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/2300900409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055c60 0x559bf2064550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f2ffd640 1 -- 172.21.15.5:0/2300900409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055c60 msgr2=0x559bf2064550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f2ffd640 1 --2- 172.21.15.5:0/2300900409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055c60 0x559bf2064550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f2ffd640 1 -- 172.21.15.5:0/2300900409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055340 msgr2=0x559bf2055720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f2ffd640 1 --2- 172.21.15.5:0/2300900409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055340 0x559bf2055720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f2ffd640 1 -- 172.21.15.5:0/2300900409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf1f47440 con 0x559bf2064a90 2024-09-19T06:05:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/2300900409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055c60 0x559bf2064550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:05:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f0ff9640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4dc0a3430 0x7ff4dc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f3fff640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4dc0a3430 0x7ff4dc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f0ff9640 1 -- 172.21.15.5:0/1143734748 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7ff4e40cffc0 con 0x7ff4f4101de0 2024-09-19T06:05:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.639+0000 7ff4f2ffd640 1 --2- 172.21.15.5:0/2300900409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2066e80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009c10 tx=0x7ff4e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69924f2662cd492a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.640+0000 7ff4d9ffb640 1 -- 172.21.15.5:0/2300900409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8004230 con 0x559bf2064a90 2024-09-19T06:05:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.640+0000 7ff4d9ffb640 1 -- 172.21.15.5:0/2300900409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4e80043d0 con 0x559bf2064a90 2024-09-19T06:05:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.640+0000 7ff4d9ffb640 1 -- 172.21.15.5:0/2300900409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8004d30 con 0x559bf2064a90 2024-09-19T06:05:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.640+0000 7ff4f3fff640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4dc0a3430 0x7ff4dc0a58f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec004af0 tx=0x7ff4ec004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.640+0000 7ff4fc37c740 1 -- 172.21.15.5:0/2300900409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 msgr2=0x559bf2066e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.640+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/2300900409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2066e80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009c10 tx=0x7ff4e8011810 comp rx=0 tx=0).stop 2024-09-19T06:05:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.641+0000 7ff4fc37c740 1 -- 172.21.15.5:0/2300900409 shutdown_connections 2024-09-19T06:05:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.641+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/2300900409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2066e80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.641+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/2300900409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055c60 0x559bf2064550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.641+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/2300900409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055340 0x559bf2055720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.641+0000 7ff4fc37c740 1 -- 172.21.15.5:0/2300900409 >> 172.21.15.5:0/2300900409 conn(0x559bf2051a30 msgr2=0x559bf2051e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.641+0000 7ff4fc37c740 1 -- 172.21.15.5:0/2300900409 shutdown_connections 2024-09-19T06:05:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.641+0000 7ff4fc37c740 1 -- 172.21.15.5:0/2300900409 wait complete. 2024-09-19T06:05:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.641+0000 7ff4fc37c740 1 -- start start 2024-09-19T06:05:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.641+0000 7ff4fc37c740 1 Processor -- start 2024-09-19T06:05:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4fc37c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 0x559bf2064360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4fc37c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055c60 0x559bf2061660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4fc37c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2061ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4fc37c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559bf2067660 con 0x559bf2064a90 2024-09-19T06:05:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4fc37c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559bf1f76050 con 0x559bf2055c60 2024-09-19T06:05:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4fc37c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559bf20677e0 con 0x559bf2055340 2024-09-19T06:05:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4fc37c740 10 client.0 _ll_register_callbacks cb 0x559bf1fb6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4f37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 0x559bf2064360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2061ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4f37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 0x559bf2064360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40268/0 (socket says 172.21.15.5:40268) 2024-09-19T06:05:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4f37fe640 1 -- 172.21.15.5:0/3322262215 learned_addr learned my addr 172.21.15.5:0/3322262215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4f2ffd640 1 --2- 172.21.15.5:0/3322262215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055c60 0x559bf2061660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: pgmap v53: 97 pgs: 97 active+clean; 27 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 825 KiB/s wr, 98 op/s 2024-09-19T06:05:02.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:05:02.649 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:02.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:02.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:02.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:02.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:02.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.650 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.651 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:02.651 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.651 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:02.651 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.651 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:02 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4f37fe640 1 -- 172.21.15.5:0/3322262215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055c60 msgr2=0x559bf2061660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/3322262215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055c60 0x559bf2061660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4f37fe640 1 -- 172.21.15.5:0/3322262215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 msgr2=0x559bf2061ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/3322262215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2061ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4f37fe640 1 -- 172.21.15.5:0/3322262215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf1d2c3c0 con 0x559bf2055340 2024-09-19T06:05:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/3322262215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 0x559bf2064360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400e9c0 tx=0x7ff4e409bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=583fd9502eaa63e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.642+0000 7ff4d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:05:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.643+0000 7ff4d2ffd640 1 -- 172.21.15.5:0/3322262215 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e409c360 con 0x559bf2055340 2024-09-19T06:05:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.643+0000 7ff4d2ffd640 1 -- 172.21.15.5:0/3322262215 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7ff4e40d20d0 con 0x559bf2055340 2024-09-19T06:05:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.643+0000 7ff4fc37c740 1 -- 172.21.15.5:0/3322262215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4e80043d0 con 0x559bf2055340 2024-09-19T06:05:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.643+0000 7ff4d2ffd640 1 -- 172.21.15.5:0/3322262215 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4e40d22d0 con 0x559bf2055340 2024-09-19T06:05:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.643+0000 7ff4fc37c740 10 client.25366 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.643+0000 7ff4fc37c740 1 -- 172.21.15.5:0/3322262215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559bf2073020 con 0x559bf2055340 2024-09-19T06:05:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.643+0000 7ff4d2ffd640 1 -- 172.21.15.5:0/3322262215 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4e80043d0 con 0x559bf2055340 2024-09-19T06:05:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.643+0000 7ff4d2ffd640 1 -- 172.21.15.5:0/3322262215 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff4e409b7b0 con 0x559bf2055340 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.644+0000 7ff4fc37c740 10 client.25366 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.644+0000 7ff4fc37c740 10 client.25366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.644+0000 7ff4fc37c740 20 client.25366 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.644+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/3322262215 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559bf2074480 0x559bf20768f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.644+0000 7ff4fc37c740 4 client.25366 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.644+0000 7ff4fc37c740 1 -- 172.21.15.5:0/3322262215 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bf1fc8850 con 0x559bf2074480 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.644+0000 7ff4f2ffd640 1 --2- 172.21.15.5:0/3322262215 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559bf2074480 0x559bf20768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.644+0000 7ff4f2ffd640 1 --2- 172.21.15.5:0/3322262215 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559bf2074480 0x559bf20768f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b30dd59bd79dc server_cookie=5f14d7b2a6685adf in_seq=0 out_seq=0 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.644+0000 7ff4d2ffd640 10 client.25366 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.645+0000 7ff4d2ffd640 1 -- 172.21.15.5:0/3322262215 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559bf1fc8850 con 0x559bf2074480 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.645+0000 7ff4d2ffd640 10 client.25366 handle_command_reply: tid=0 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.645+0000 7ff4fc37c740 1 client.25366 shutdown 2024-09-19T06:05:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.646+0000 7ff4fc37c740 1 -- 172.21.15.5:0/3322262215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 msgr2=0x559bf2064360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.646+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/3322262215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 0x559bf2064360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400e9c0 tx=0x7ff4e409bea0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.646+0000 7ff4fc37c740 1 -- 172.21.15.5:0/3322262215 shutdown_connections 2024-09-19T06:05:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.646+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/3322262215 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559bf2074480 0x559bf20768f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.646+0000 7ff4d2ffd640 0 client.25366 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.646+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/3322262215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2061ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.646+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/3322262215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055c60 0x559bf2061660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.646+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/3322262215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 0x559bf2064360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.646+0000 7ff4fc37c740 1 -- 172.21.15.5:0/3322262215 >> 172.21.15.5:0/3322262215 conn(0x559bf2051a30 msgr2=0x559bf2057790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.646+0000 7ff4fc37c740 1 -- 172.21.15.5:0/3322262215 shutdown_connections 2024-09-19T06:05:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.646+0000 7ff4fc37c740 1 -- 172.21.15.5:0/3322262215 wait complete. 2024-09-19T06:05:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.646+0000 7ff4fc37c740 20 client.25366 trim_cache size 0 max 16384 2024-09-19T06:05:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.670+0000 7ff4fc37c740 1 Processor -- start 2024-09-19T06:05:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.670+0000 7ff4fc37c740 1 -- start start 2024-09-19T06:05:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4fc37c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055340 0x559bf2070900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4fc37c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055c60 0x559bf207c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4fc37c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf207c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4fc37c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41ab690 con 0x559bf2064a90 2024-09-19T06:05:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4fc37c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41ad230 con 0x559bf2055340 2024-09-19T06:05:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4fc37c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41abd60 con 0x559bf2055c60 2024-09-19T06:05:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf207c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055340 0x559bf2070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf207c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42370/0 (socket says 172.21.15.5:42370) 2024-09-19T06:05:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055340 0x559bf2070900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54474/0 (socket says 172.21.15.5:54474) 2024-09-19T06:05:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4f3fff640 1 -- 172.21.15.5:0/2345529960 learned_addr learned my addr 172.21.15.5:0/2345529960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4f2ffd640 1 --2- 172.21.15.5:0/2345529960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055c60 0x559bf207c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4f37fe640 1 -- 172.21.15.5:0/2345529960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055c60 msgr2=0x559bf207c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/2345529960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055c60 0x559bf207c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4f37fe640 1 -- 172.21.15.5:0/2345529960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 msgr2=0x559bf207c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/2345529960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf207c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4f37fe640 1 -- 172.21.15.5:0/2345529960 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf20671c0 con 0x559bf2055340 2024-09-19T06:05:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.671+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/2345529960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055340 0x559bf2070900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40cdd20 tx=0x7ff4e409bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c233bacc9b8dfc4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.672+0000 7ff4d97fa640 1 -- 172.21.15.5:0/2345529960 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e409c360 con 0x559bf2055340 2024-09-19T06:05:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.672+0000 7ff4d97fa640 1 -- 172.21.15.5:0/2345529960 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4e40a0660 con 0x559bf2055340 2024-09-19T06:05:02.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.672+0000 7ff4d97fa640 1 -- 172.21.15.5:0/2345529960 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e409bcc0 con 0x559bf2055340 2024-09-19T06:05:02.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.672+0000 7ff4fc37c740 1 -- 172.21.15.5:0/2345529960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055340 msgr2=0x559bf2070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:02.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.672+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/2345529960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055340 0x559bf2070900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40cdd20 tx=0x7ff4e409bfd0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.673+0000 7ff4fc37c740 1 -- 172.21.15.5:0/2345529960 shutdown_connections 2024-09-19T06:05:02.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.673+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/2345529960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf207c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.673+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/2345529960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055c60 0x559bf207c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.673+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/2345529960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055340 0x559bf2070900 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.673+0000 7ff4fc37c740 1 -- 172.21.15.5:0/2345529960 >> 172.21.15.5:0/2345529960 conn(0x559bf2051a30 msgr2=0x559bf205cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.673+0000 7ff4fc37c740 1 -- 172.21.15.5:0/2345529960 shutdown_connections 2024-09-19T06:05:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.673+0000 7ff4fc37c740 1 -- 172.21.15.5:0/2345529960 wait complete. 2024-09-19T06:05:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.673+0000 7ff4fc37c740 1 -- start start 2024-09-19T06:05:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.673+0000 7ff4fc37c740 1 Processor -- start 2024-09-19T06:05:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4fc37c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 0x559bf20689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4fc37c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055c60 0x559bf2068ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4fc37c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2069430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4fc37c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41acb00 con 0x559bf2064a90 2024-09-19T06:05:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4fc37c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41ac430 con 0x559bf2055c60 2024-09-19T06:05:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4fc37c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41a9670 con 0x559bf2055340 2024-09-19T06:05:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4fc37c740 10 client.0 _ll_register_callbacks cb 0x7ff4e400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4f37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 0x559bf20689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4f37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 0x559bf20689b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40282/0 (socket says 172.21.15.5:40282) 2024-09-19T06:05:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4f37fe640 1 -- 172.21.15.5:0/1787232218 learned_addr learned my addr 172.21.15.5:0/1787232218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4f2ffd640 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055c60 0x559bf2068ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4f3fff640 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4f37fe640 1 -- 172.21.15.5:0/1787232218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055c60 msgr2=0x559bf2068ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055c60 0x559bf2068ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4f37fe640 1 -- 172.21.15.5:0/1787232218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 msgr2=0x559bf2069430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2069430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4f37fe640 1 -- 172.21.15.5:0/1787232218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf20de390 con 0x559bf2055340 2024-09-19T06:05:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4f37fe640 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 0x559bf20689b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4098d30 tx=0x7ff4e409bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faef11dedf441afe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.674+0000 7ff4d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:05:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4d2ffd640 1 -- 172.21.15.5:0/1787232218 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e409bcc0 con 0x559bf2055340 2024-09-19T06:05:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4fc37c740 1 -- 172.21.15.5:0/1787232218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4e40a0660 con 0x559bf2055340 2024-09-19T06:05:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4d2ffd640 1 -- 172.21.15.5:0/1787232218 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7ff4e40bc2f0 con 0x559bf2055340 2024-09-19T06:05:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4d2ffd640 1 -- 172.21.15.5:0/1787232218 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4e409c360 con 0x559bf2055340 2024-09-19T06:05:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4fc37c740 10 client.25372 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4fc37c740 1 -- 172.21.15.5:0/1787232218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559bf20df660 con 0x559bf2055340 2024-09-19T06:05:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4d2ffd640 1 -- 172.21.15.5:0/1787232218 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4e40a0660 con 0x559bf2055340 2024-09-19T06:05:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4d2ffd640 1 -- 172.21.15.5:0/1787232218 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff4e40980b0 con 0x559bf2055340 2024-09-19T06:05:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4fc37c740 10 client.25372 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4fc37c740 10 client.25372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4fc37c740 20 client.25372 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559bf2074480 0x559bf20e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4fc37c740 4 client.25372 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:05:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4fc37c740 1 -- 172.21.15.5:0/1787232218 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559bf2061f30 con 0x559bf2074480 2024-09-19T06:05:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.675+0000 7ff4f2ffd640 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559bf2074480 0x559bf20e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4d2ffd640 10 client.25372 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4f2ffd640 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559bf2074480 0x559bf20e1b50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70c7ee82e8d9b6cc server_cookie=f3d3865f38eb8259 in_seq=0 out_seq=0 2024-09-19T06:05:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4d2ffd640 1 -- 172.21.15.5:0/1787232218 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559bf2061f30 con 0x559bf2074480 2024-09-19T06:05:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4d2ffd640 10 client.25372 handle_command_reply: tid=0 2024-09-19T06:05:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4fc37c740 1 client.25372 shutdown 2024-09-19T06:05:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4fc37c740 1 -- 172.21.15.5:0/1787232218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 msgr2=0x559bf20689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 0x559bf20689b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4098d30 tx=0x7ff4e409bb20 comp rx=0 tx=0).stop 2024-09-19T06:05:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4fc37c740 1 -- 172.21.15.5:0/1787232218 shutdown_connections 2024-09-19T06:05:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559bf2074480 0x559bf20e1b50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bf2064a90 0x559bf2069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559bf2055c60 0x559bf2068ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4d2ffd640 0 client.25372 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4fc37c740 1 --2- 172.21.15.5:0/1787232218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bf2055340 0x559bf20689b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4fc37c740 1 -- 172.21.15.5:0/1787232218 >> 172.21.15.5:0/1787232218 conn(0x559bf2051a30 msgr2=0x559bf2050350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4fc37c740 1 -- 172.21.15.5:0/1787232218 shutdown_connections 2024-09-19T06:05:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.676+0000 7ff4fc37c740 1 -- 172.21.15.5:0/1787232218 wait complete. 2024-09-19T06:05:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4fc37c740 20 client.25372 trim_cache size 0 max 16384 2024-09-19T06:05:02.686 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:05:02.686 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:05:02.686 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:05:02.686 INFO:teuthology.orchestra.run.smithi005.stdout: "7f045632-17ff-4224-8123-47e7f8902444": { 2024-09-19T06:05:02.686 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:05:02.686 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "7f045632-17ff-4224-8123-47e7f8902444", 2024-09-19T06:05:02.686 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:05:02.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:05:02.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:05:02.687 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:05:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1143734748 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4dc0a3430 msgr2=0x7ff4dc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4f9fa3640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4dc0a3430 0x7ff4dc0a58f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec004af0 tx=0x7ff4ec004a40 comp rx=0 tx=0).stop 2024-09-19T06:05:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1143734748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f4101de0 msgr2=0x7ff4f4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4f9fa3640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f4101de0 0x7ff4f4198360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400e9f0 tx=0x7ff4e400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1143734748 shutdown_connections 2024-09-19T06:05:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4f9fa3640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f4106f10 0x7ff4f419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4f9fa3640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4dc0a3430 0x7ff4dc0a58f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4f9fa3640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4102790 0x7ff4f41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4f9fa3640 1 --2- 172.21.15.5:0/1143734748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f4101de0 0x7ff4f4198360 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:02.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1143734748 >> 172.21.15.5:0/1143734748 conn(0x7ff4f40fd8c0 msgr2=0x7ff4f40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:02.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1143734748 shutdown_connections 2024-09-19T06:05:02.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:02.677+0000 7ff4f9fa3640 1 -- 172.21.15.5:0/1143734748 wait complete. 2024-09-19T06:05:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: pgmap v53: 97 pgs: 97 active+clean; 27 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 825 KiB/s wr, 98 op/s 2024-09-19T06:05:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:05:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:02 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:02.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 15252 3.55 MB/sec warmup 28 sec latency 13.389 ms 2024-09-19T06:05:03.451 INFO:tasks.cephfs.filesystem:scrub status for tag:7f045632-17ff-4224-8123-47e7f8902444 - {'path': '/', 'tag': '7f045632-17ff-4224-8123-47e7f8902444', 'options': 'recursive,force'} 2024-09-19T06:05:03.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 15820 3.54 MB/sec warmup 29 sec latency 15.387 ms 2024-09-19T06:05:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.050+0000 7f7187835640 1 -- 172.21.15.5:0/706575439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71801069b0 msgr2=0x7f7180106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.050+0000 7f7187835640 1 --2- 172.21.15.5:0/706575439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71801069b0 0x7f7180106d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f717c009930 tx=0x7f717c02a420 comp rx=0 tx=0).stop 2024-09-19T06:05:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.050+0000 7f7187835640 1 -- 172.21.15.5:0/706575439 shutdown_connections 2024-09-19T06:05:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.050+0000 7f7187835640 1 --2- 172.21.15.5:0/706575439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7180101340 0x7f718010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.050+0000 7f7187835640 1 --2- 172.21.15.5:0/706575439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71801009a0 0x7f7180100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.050+0000 7f7187835640 1 --2- 172.21.15.5:0/706575439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71801069b0 0x7f7180106d90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.050+0000 7f7187835640 1 -- 172.21.15.5:0/706575439 >> 172.21.15.5:0/706575439 conn(0x7f71800779a0 msgr2=0x7f71800fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.050+0000 7f7187835640 1 -- 172.21.15.5:0/706575439 shutdown_connections 2024-09-19T06:05:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.050+0000 7f7187835640 1 -- 172.21.15.5:0/706575439 wait complete. 2024-09-19T06:05:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.051+0000 7f7187835640 1 Processor -- start 2024-09-19T06:05:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.051+0000 7f7187835640 1 -- start start 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.051+0000 7f7187835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71801009a0 0x7f71800fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.051+0000 7f7187835640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7180101340 0x7f71800fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.051+0000 7f7187835640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71801069b0 0x7f71800fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.051+0000 7f7187835640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7180110c70 con 0x7f71801009a0 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.051+0000 7f7187835640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7180110af0 con 0x7f71801069b0 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.051+0000 7f7187835640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7180110df0 con 0x7f7180101340 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.051+0000 7f71855d1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71801009a0 0x7f71800fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.051+0000 7f71855d1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71801009a0 0x7f71800fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42416/0 (socket says 172.21.15.5:42416) 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.051+0000 7f71855d1640 1 -- 172.21.15.5:0/2706361589 learned_addr learned my addr 172.21.15.5:0/2706361589 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.051+0000 7f7184dd0640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7180101340 0x7f71800fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f7185dd2640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71801069b0 0x7f71800fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f71855d1640 1 -- 172.21.15.5:0/2706361589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7180101340 msgr2=0x7f71800fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f71855d1640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7180101340 0x7f71800fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f71855d1640 1 -- 172.21.15.5:0/2706361589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71801069b0 msgr2=0x7f71800fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f71855d1640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71801069b0 0x7f71800fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f71855d1640 1 -- 172.21.15.5:0/2706361589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71801a6810 con 0x7f71801009a0 2024-09-19T06:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f7184dd0640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7180101340 0x7f71800fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f71855d1640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71801009a0 0x7f71800fcde0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f717c005940 tx=0x7f717c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f71767fc640 1 -- 172.21.15.5:0/2706361589 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c03f030 con 0x7f71801009a0 2024-09-19T06:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f7187835640 1 -- 172.21.15.5:0/2706361589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71801a6aa0 con 0x7f71801009a0 2024-09-19T06:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f71767fc640 1 -- 172.21.15.5:0/2706361589 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f717c0318a0 con 0x7f71801009a0 2024-09-19T06:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f71767fc640 1 -- 172.21.15.5:0/2706361589 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c035070 con 0x7f71801009a0 2024-09-19T06:05:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.052+0000 7f7187835640 1 -- 172.21.15.5:0/2706361589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71801a6f90 con 0x7f71801009a0 2024-09-19T06:05:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.055+0000 7f71767fc640 1 -- 172.21.15.5:0/2706361589 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f717c042030 con 0x7f71801009a0 2024-09-19T06:05:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.055+0000 7f718a5d0740 1 Processor -- start 2024-09-19T06:05:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f718a5d0740 1 -- start start 2024-09-19T06:05:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f718a5d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x555779544ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f718a5d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555779545410 0x55577954f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f718a5d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55577954fcb0 0x5557795520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f718a5d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5557794a92f0 con 0x55577954fcb0 2024-09-19T06:05:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f718a5d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555779451270 con 0x555779545410 2024-09-19T06:05:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f718a5d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5557794a52b0 con 0x555779544af0 2024-09-19T06:05:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f7184dd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55577954fcb0 0x5557795520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f7184dd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55577954fcb0 0x5557795520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42424/0 (socket says 172.21.15.5:42424) 2024-09-19T06:05:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f7184dd0640 1 -- 172.21.15.5:0/781440323 learned_addr learned my addr 172.21.15.5:0/781440323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f7185dd2640 1 --2- 172.21.15.5:0/781440323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x555779544ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f71855d1640 1 --2- 172.21.15.5:0/781440323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555779545410 0x55577954f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f7184dd0640 1 -- 172.21.15.5:0/781440323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 msgr2=0x555779544ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f7184dd0640 1 --2- 172.21.15.5:0/781440323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x555779544ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f7184dd0640 1 -- 172.21.15.5:0/781440323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555779545410 msgr2=0x55577954f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f7184dd0640 1 --2- 172.21.15.5:0/781440323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555779545410 0x55577954f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.056+0000 7f7184dd0640 1 -- 172.21.15.5:0/781440323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555779450340 con 0x55577954fcb0 2024-09-19T06:05:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.057+0000 7f71767fc640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f716c0a33b0 0x7f716c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.057+0000 7f7184dd0640 1 --2- 172.21.15.5:0/781440323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55577954fcb0 0x5557795520a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f717000bfd0 tx=0x7f71700108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eeec1187dd4b225 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.057+0000 7f7185dd2640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f716c0a33b0 0x7f716c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.057+0000 7f71767fc640 1 -- 172.21.15.5:0/2706361589 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f717c0effc0 con 0x7f71801009a0 2024-09-19T06:05:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.057+0000 7f716b7fe640 1 -- 172.21.15.5:0/781440323 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170005220 con 0x55577954fcb0 2024-09-19T06:05:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.057+0000 7f716b7fe640 1 -- 172.21.15.5:0/781440323 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7170004c50 con 0x55577954fcb0 2024-09-19T06:05:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.057+0000 7f716b7fe640 1 -- 172.21.15.5:0/781440323 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170011b00 con 0x55577954fcb0 2024-09-19T06:05:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.058+0000 7f718a5d0740 1 -- 172.21.15.5:0/781440323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55577954fcb0 msgr2=0x5557795520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.058+0000 7f7185dd2640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f716c0a33b0 0x7f716c0a5870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7178005fd0 tx=0x7f7178009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.058+0000 7f718a5d0740 1 --2- 172.21.15.5:0/781440323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55577954fcb0 0x5557795520a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f717000bfd0 tx=0x7f71700108e0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.058+0000 7f718a5d0740 1 -- 172.21.15.5:0/781440323 shutdown_connections 2024-09-19T06:05:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.058+0000 7f718a5d0740 1 --2- 172.21.15.5:0/781440323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55577954fcb0 0x5557795520a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.058+0000 7f718a5d0740 1 --2- 172.21.15.5:0/781440323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555779545410 0x55577954f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.058+0000 7f718a5d0740 1 --2- 172.21.15.5:0/781440323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x555779544ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.058+0000 7f718a5d0740 1 -- 172.21.15.5:0/781440323 >> 172.21.15.5:0/781440323 conn(0x555779541060 msgr2=0x555779541440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.058+0000 7f718a5d0740 1 -- 172.21.15.5:0/781440323 shutdown_connections 2024-09-19T06:05:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.058+0000 7f718a5d0740 1 -- 172.21.15.5:0/781440323 wait complete. 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.058+0000 7f718a5d0740 1 -- start start 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.059+0000 7f718a5d0740 1 Processor -- start 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.059+0000 7f718a5d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x55577954cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.059+0000 7f718a5d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779545410 0x55577954d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.059+0000 7f718a5d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 0x55577954d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.059+0000 7f718a5d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555779552880 con 0x555779545410 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.059+0000 7f718a5d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555779432290 con 0x55577954fcb0 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.059+0000 7f718a5d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555779552a00 con 0x555779544af0 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.059+0000 7f718a5d0740 10 client.0 _ll_register_callbacks cb 0x555779463140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.059+0000 7f7184dd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779545410 0x55577954d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.059+0000 7f7184dd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779545410 0x55577954d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42430/0 (socket says 172.21.15.5:42430) 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.059+0000 7f7184dd0640 1 -- 172.21.15.5:0/2606479600 learned_addr learned my addr 172.21.15.5:0/2606479600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.059+0000 7f71855d1640 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x55577954cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f7185dd2640 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 0x55577954d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f7184dd0640 1 -- 172.21.15.5:0/2606479600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 msgr2=0x55577954cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f7184dd0640 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x55577954cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f7184dd0640 1 -- 172.21.15.5:0/2606479600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 msgr2=0x55577954d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f7184dd0640 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 0x55577954d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f7184dd0640 1 -- 172.21.15.5:0/2606479600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557791bc210 con 0x555779545410 2024-09-19T06:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f7185dd2640 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 0x55577954d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f71855d1640 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x55577954cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f7184dd0640 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779545410 0x55577954d100 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7170011010 tx=0x7f71700048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dee2057e19f781 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f7168ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f7168ff9640 1 -- 172.21.15.5:0/2606479600 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170004e00 con 0x555779545410 2024-09-19T06:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f718a5d0740 1 -- 172.21.15.5:0/2606479600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7170004c50 con 0x555779545410 2024-09-19T06:05:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.060+0000 7f7168ff9640 1 -- 172.21.15.5:0/2606479600 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f7170005210 con 0x555779545410 2024-09-19T06:05:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.061+0000 7f7168ff9640 1 -- 172.21.15.5:0/2606479600 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7170004a60 con 0x555779545410 2024-09-19T06:05:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.061+0000 7f718a5d0740 10 client.15849 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.061+0000 7f718a5d0740 1 -- 172.21.15.5:0/2606479600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555779288620 con 0x555779545410 2024-09-19T06:05:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.061+0000 7f7168ff9640 1 -- 172.21.15.5:0/2606479600 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7170004c50 con 0x555779545410 2024-09-19T06:05:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.061+0000 7f7168ff9640 1 -- 172.21.15.5:0/2606479600 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f717001b690 con 0x555779545410 2024-09-19T06:05:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.061+0000 7f718a5d0740 10 client.15849 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.061+0000 7f718a5d0740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.061+0000 7f718a5d0740 20 client.15849 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.061+0000 7f718a5d0740 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55577955fc80 0x555779562070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.061+0000 7f718a5d0740 4 client.15849 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.061+0000 7f718a5d0740 1 -- 172.21.15.5:0/2606479600 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557794b6a20 con 0x55577955fc80 2024-09-19T06:05:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.061+0000 7f71855d1640 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55577955fc80 0x555779562070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.062+0000 7f7168ff9640 10 client.15849 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.062+0000 7f71855d1640 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55577955fc80 0x555779562070 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87117e11fe93fd4b server_cookie=15b886393406910e in_seq=0 out_seq=0 2024-09-19T06:05:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.062+0000 7f7168ff9640 1 -- 172.21.15.5:0/2606479600 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557794b6a20 con 0x55577955fc80 2024-09-19T06:05:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.062+0000 7f7168ff9640 10 client.15849 handle_command_reply: tid=0 2024-09-19T06:05:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.062+0000 7f718a5d0740 1 client.15849 shutdown 2024-09-19T06:05:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.063+0000 7f718a5d0740 1 -- 172.21.15.5:0/2606479600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779545410 msgr2=0x55577954d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.063+0000 7f718a5d0740 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779545410 0x55577954d100 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7170011010 tx=0x7f71700048f0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.063+0000 7f718a5d0740 1 -- 172.21.15.5:0/2606479600 shutdown_connections 2024-09-19T06:05:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.063+0000 7f718a5d0740 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55577955fc80 0x555779562070 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.063+0000 7f718a5d0740 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 0x55577954d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.063+0000 7f718a5d0740 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779545410 0x55577954d100 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.063+0000 7f7168ff9640 0 client.15849 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.063+0000 7f718a5d0740 1 --2- 172.21.15.5:0/2606479600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x55577954cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.063+0000 7f718a5d0740 1 -- 172.21.15.5:0/2606479600 >> 172.21.15.5:0/2606479600 conn(0x555779541060 msgr2=0x555779546d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.063+0000 7f718a5d0740 1 -- 172.21.15.5:0/2606479600 shutdown_connections 2024-09-19T06:05:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.063+0000 7f718a5d0740 1 -- 172.21.15.5:0/2606479600 wait complete. 2024-09-19T06:05:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.063+0000 7f718a5d0740 20 client.15849 trim_cache size 0 max 16384 2024-09-19T06:05:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.079+0000 7f718a5d0740 1 Processor -- start 2024-09-19T06:05:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.079+0000 7f718a5d0740 1 -- start start 2024-09-19T06:05:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f718a5d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x55577954d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f718a5d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779545410 0x555779567890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f718a5d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 0x555779567dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f718a5d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71801af890 con 0x555779545410 2024-09-19T06:05:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f718a5d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f71801b1430 con 0x55577954fcb0 2024-09-19T06:05:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f718a5d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71801aff60 con 0x555779544af0 2024-09-19T06:05:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f7184dd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779545410 0x555779567890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f7185dd2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 0x555779567dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f7185dd2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 0x555779567dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54532/0 (socket says 172.21.15.5:54532) 2024-09-19T06:05:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f7185dd2640 1 -- 172.21.15.5:0/3095749472 learned_addr learned my addr 172.21.15.5:0/3095749472 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f71855d1640 1 --2- 172.21.15.5:0/3095749472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x55577954d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f7185dd2640 1 -- 172.21.15.5:0/3095749472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 msgr2=0x55577954d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f7185dd2640 1 --2- 172.21.15.5:0/3095749472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x55577954d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f7185dd2640 1 -- 172.21.15.5:0/3095749472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779545410 msgr2=0x555779567890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f7185dd2640 1 --2- 172.21.15.5:0/3095749472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779545410 0x555779567890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f7185dd2640 1 -- 172.21.15.5:0/3095749472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555779545240 con 0x55577954fcb0 2024-09-19T06:05:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f71855d1640 1 --2- 172.21.15.5:0/3095749472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x55577954d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.080+0000 7f7185dd2640 1 --2- 172.21.15.5:0/3095749472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 0x555779567dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7178003e20 tx=0x7f7178016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b8a6f300cea743f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.081+0000 7f716affd640 1 -- 172.21.15.5:0/3095749472 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71780045c0 con 0x55577954fcb0 2024-09-19T06:05:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.081+0000 7f716affd640 1 -- 172.21.15.5:0/3095749472 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7178004760 con 0x55577954fcb0 2024-09-19T06:05:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.081+0000 7f716affd640 1 -- 172.21.15.5:0/3095749472 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717801e5a0 con 0x55577954fcb0 2024-09-19T06:05:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.081+0000 7f718a5d0740 1 -- 172.21.15.5:0/3095749472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 msgr2=0x555779567dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.082+0000 7f718a5d0740 1 --2- 172.21.15.5:0/3095749472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 0x555779567dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7178003e20 tx=0x7f7178016000 comp rx=0 tx=0).stop 2024-09-19T06:05:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.082+0000 7f718a5d0740 1 -- 172.21.15.5:0/3095749472 shutdown_connections 2024-09-19T06:05:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.082+0000 7f718a5d0740 1 --2- 172.21.15.5:0/3095749472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55577954fcb0 0x555779567dd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.082+0000 7f718a5d0740 1 --2- 172.21.15.5:0/3095749472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779545410 0x555779567890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.082+0000 7f718a5d0740 1 --2- 172.21.15.5:0/3095749472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555779544af0 0x55577954d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.082+0000 7f718a5d0740 1 -- 172.21.15.5:0/3095749472 >> 172.21.15.5:0/3095749472 conn(0x555779541060 msgr2=0x55577954ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.082+0000 7f718a5d0740 1 -- 172.21.15.5:0/3095749472 shutdown_connections 2024-09-19T06:05:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.082+0000 7f718a5d0740 1 -- 172.21.15.5:0/3095749472 wait complete. 2024-09-19T06:05:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.082+0000 7f718a5d0740 1 -- start start 2024-09-19T06:05:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.082+0000 7f718a5d0740 1 Processor -- start 2024-09-19T06:05:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f718a5d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779544af0 0x555779553420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f718a5d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555779545410 0x555779553960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f718a5d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55577954fcb0 0x5557795c2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f718a5d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71801b0d00 con 0x555779544af0 2024-09-19T06:05:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f718a5d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f71801b0630 con 0x555779545410 2024-09-19T06:05:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f718a5d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71801ad870 con 0x55577954fcb0 2024-09-19T06:05:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f718a5d0740 10 client.0 _ll_register_callbacks cb 0x7f7178009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f71855d1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779544af0 0x555779553420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f7184dd0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555779545410 0x555779553960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f7184dd0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555779545410 0x555779553960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54544/0 (socket says 172.21.15.5:54544) 2024-09-19T06:05:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f7184dd0640 1 -- 172.21.15.5:0/4206594109 learned_addr learned my addr 172.21.15.5:0/4206594109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f71855d1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779544af0 0x555779553420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42448/0 (socket says 172.21.15.5:42448) 2024-09-19T06:05:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f7185dd2640 1 --2- 172.21.15.5:0/4206594109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55577954fcb0 0x5557795c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f7184dd0640 1 -- 172.21.15.5:0/4206594109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55577954fcb0 msgr2=0x5557795c2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f7184dd0640 1 --2- 172.21.15.5:0/4206594109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55577954fcb0 0x5557795c2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f7184dd0640 1 -- 172.21.15.5:0/4206594109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779544af0 msgr2=0x555779553420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f7184dd0640 1 --2- 172.21.15.5:0/4206594109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779544af0 0x555779553420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f7184dd0640 1 -- 172.21.15.5:0/4206594109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557795c2ec0 con 0x555779545410 2024-09-19T06:05:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f7184dd0640 1 --2- 172.21.15.5:0/4206594109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555779545410 0x555779553960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f717001bab0 tx=0x7f7170004960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7954c0c3712587b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.083+0000 7f7168ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:05:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f7168ff9640 1 -- 172.21.15.5:0/4206594109 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717001b640 con 0x555779545410 2024-09-19T06:05:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f7168ff9640 1 -- 172.21.15.5:0/4206594109 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f717001b7e0 con 0x555779545410 2024-09-19T06:05:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f718a5d0740 1 -- 172.21.15.5:0/4206594109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7178004760 con 0x555779545410 2024-09-19T06:05:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f7168ff9640 1 -- 172.21.15.5:0/4206594109 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7170005210 con 0x555779545410 2024-09-19T06:05:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f718a5d0740 10 client.25460 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f718a5d0740 1 -- 172.21.15.5:0/4206594109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557795ca2e0 con 0x555779545410 2024-09-19T06:05:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f7168ff9640 1 -- 172.21.15.5:0/4206594109 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7178004760 con 0x555779545410 2024-09-19T06:05:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f7168ff9640 1 -- 172.21.15.5:0/4206594109 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7170011890 con 0x555779545410 2024-09-19T06:05:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f718a5d0740 10 client.25460 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f718a5d0740 10 client.25460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f718a5d0740 20 client.25460 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f718a5d0740 1 --2- 172.21.15.5:0/4206594109 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5557795caa50 0x5557795cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f718a5d0740 4 client.25460 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:05:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.084+0000 7f718a5d0740 1 -- 172.21.15.5:0/4206594109 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557794b0d70 con 0x5557795caa50 2024-09-19T06:05:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.085+0000 7f71855d1640 1 --2- 172.21.15.5:0/4206594109 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5557795caa50 0x5557795cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.085+0000 7f7168ff9640 10 client.25460 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.085+0000 7f71855d1640 1 --2- 172.21.15.5:0/4206594109 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5557795caa50 0x5557795cce70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc652b07ba475d58 server_cookie=67784ef6f930eac7 in_seq=0 out_seq=0 2024-09-19T06:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.100+0000 7f7168ff9640 1 -- 172.21.15.5:0/4206594109 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557794b0d70 con 0x5557795caa50 2024-09-19T06:05:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.100+0000 7f7168ff9640 10 client.25460 handle_command_reply: tid=0 2024-09-19T06:05:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.100+0000 7f718a5d0740 1 client.25460 shutdown 2024-09-19T06:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.100+0000 7f718a5d0740 1 -- 172.21.15.5:0/4206594109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555779545410 msgr2=0x555779553960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.100+0000 7f718a5d0740 1 --2- 172.21.15.5:0/4206594109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555779545410 0x555779553960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f717001bab0 tx=0x7f7170004960 comp rx=0 tx=0).stop 2024-09-19T06:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.100+0000 7f718a5d0740 1 -- 172.21.15.5:0/4206594109 shutdown_connections 2024-09-19T06:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.100+0000 7f718a5d0740 1 --2- 172.21.15.5:0/4206594109 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5557795caa50 0x5557795cce70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.100+0000 7f718a5d0740 1 --2- 172.21.15.5:0/4206594109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55577954fcb0 0x5557795c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.100+0000 7f718a5d0740 1 --2- 172.21.15.5:0/4206594109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555779545410 0x555779553960 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.100+0000 7f7168ff9640 0 client.25460 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.100+0000 7f718a5d0740 1 --2- 172.21.15.5:0/4206594109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555779544af0 0x555779553420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.100+0000 7f718a5d0740 1 -- 172.21.15.5:0/4206594109 >> 172.21.15.5:0/4206594109 conn(0x555779541060 msgr2=0x55577954ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f718a5d0740 1 -- 172.21.15.5:0/4206594109 shutdown_connections 2024-09-19T06:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f718a5d0740 1 -- 172.21.15.5:0/4206594109 wait complete. 2024-09-19T06:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f718a5d0740 20 client.25460 trim_cache size 0 max 16384 2024-09-19T06:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:05:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f7187835640 1 -- 172.21.15.5:0/2706361589 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f716c0a33b0 msgr2=0x7f716c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f7187835640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f716c0a33b0 0x7f716c0a5870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7178005fd0 tx=0x7f7178009500 comp rx=0 tx=0).stop 2024-09-19T06:05:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f7187835640 1 -- 172.21.15.5:0/2706361589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71801009a0 msgr2=0x7f71800fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f7187835640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71801009a0 0x7f71800fcde0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f717c005940 tx=0x7f717c0042f0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f7187835640 1 -- 172.21.15.5:0/2706361589 shutdown_connections 2024-09-19T06:05:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f7187835640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71801069b0 0x7f71800fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f7187835640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f716c0a33b0 0x7f716c0a5870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f7187835640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7180101340 0x7f71800fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f7187835640 1 --2- 172.21.15.5:0/2706361589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71801009a0 0x7f71800fcde0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f7187835640 1 -- 172.21.15.5:0/2706361589 >> 172.21.15.5:0/2706361589 conn(0x7f71800779a0 msgr2=0x7f7180069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.101+0000 7f7187835640 1 -- 172.21.15.5:0/2706361589 shutdown_connections 2024-09-19T06:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:04.102+0000 7f7187835640 1 -- 172.21.15.5:0/2706361589 wait complete. 2024-09-19T06:05:04.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:04 smithi005 ceph-mon[30955]: pgmap v54: 97 pgs: 97 active+clean; 32 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-09-19T06:05:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:04 smithi049 ceph-mon[34640]: pgmap v54: 97 pgs: 97 active+clean; 32 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-09-19T06:05:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:04 smithi157 ceph-mon[35346]: pgmap v54: 97 pgs: 97 active+clean; 32 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-09-19T06:05:04.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 16520 3.48 MB/sec warmup 30 sec latency 13.545 ms 2024-09-19T06:05:05.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 17089 3.53 MB/sec warmup 31 sec latency 15.551 ms 2024-09-19T06:05:06.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:06 smithi005 ceph-mon[30955]: pgmap v55: 97 pgs: 97 active+clean; 32 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 840 KiB/s wr, 117 op/s 2024-09-19T06:05:06.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:06 smithi049 ceph-mon[34640]: pgmap v55: 97 pgs: 97 active+clean; 32 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 840 KiB/s wr, 117 op/s 2024-09-19T06:05:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:06 smithi157 ceph-mon[35346]: pgmap v55: 97 pgs: 97 active+clean; 32 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 840 KiB/s wr, 117 op/s 2024-09-19T06:05:06.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 17579 3.50 MB/sec warmup 32 sec latency 10.767 ms 2024-09-19T06:05:07.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18053 3.52 MB/sec warmup 33 sec latency 15.174 ms 2024-09-19T06:05:08.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:08 smithi005 ceph-mon[30955]: pgmap v56: 97 pgs: 97 active+clean; 34 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-09-19T06:05:08.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:08 smithi049 ceph-mon[34640]: pgmap v56: 97 pgs: 97 active+clean; 34 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-09-19T06:05:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:08 smithi157 ceph-mon[35346]: pgmap v56: 97 pgs: 97 active+clean; 34 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-09-19T06:05:08.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18343 3.43 MB/sec warmup 34 sec latency 14.449 ms 2024-09-19T06:05:09.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:09 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:05:09.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:09 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:05:09.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:09 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:05:09.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18869 3.39 MB/sec warmup 35 sec latency 15.596 ms 2024-09-19T06:05:10.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:10 smithi005 ceph-mon[30955]: pgmap v57: 97 pgs: 97 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-09-19T06:05:10.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:10 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:05:10.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:10 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:05:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:10 smithi049 ceph-mon[34640]: pgmap v57: 97 pgs: 97 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-09-19T06:05:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:10 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:05:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:10 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:05:10.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:10 smithi157 ceph-mon[35346]: pgmap v57: 97 pgs: 97 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-09-19T06:05:10.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:10 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:05:10.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:10 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:05:10.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 19457 3.38 MB/sec warmup 36 sec latency 15.064 ms 2024-09-19T06:05:11.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 20152 3.34 MB/sec warmup 37 sec latency 15.626 ms 2024-09-19T06:05:12.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:12 smithi005 ceph-mon[30955]: pgmap v58: 97 pgs: 97 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-09-19T06:05:12.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:12 smithi049 ceph-mon[34640]: pgmap v58: 97 pgs: 97 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-09-19T06:05:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:12 smithi157 ceph-mon[35346]: pgmap v58: 97 pgs: 97 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-09-19T06:05:12.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 20769 3.40 MB/sec warmup 38 sec latency 15.177 ms 2024-09-19T06:05:13.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 21324 3.44 MB/sec warmup 39 sec latency 15.044 ms 2024-09-19T06:05:14.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:14 smithi005 ceph-mon[30955]: pgmap v59: 97 pgs: 97 active+clean; 31 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.7 MiB/s wr, 217 op/s 2024-09-19T06:05:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:14 smithi049 ceph-mon[34640]: pgmap v59: 97 pgs: 97 active+clean; 31 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.7 MiB/s wr, 217 op/s 2024-09-19T06:05:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:14 smithi157 ceph-mon[35346]: pgmap v59: 97 pgs: 97 active+clean; 31 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.7 MiB/s wr, 217 op/s 2024-09-19T06:05:14.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 21696 3.39 MB/sec warmup 40 sec latency 13.714 ms 2024-09-19T06:05:15.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 21998 3.32 MB/sec warmup 41 sec latency 12.869 ms 2024-09-19T06:05:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:16 smithi005 ceph-mon[30955]: pgmap v60: 97 pgs: 97 active+clean; 31 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-19T06:05:16.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:16 smithi049 ceph-mon[34640]: pgmap v60: 97 pgs: 97 active+clean; 31 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-19T06:05:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:16 smithi157 ceph-mon[35346]: pgmap v60: 97 pgs: 97 active+clean; 31 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-19T06:05:16.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 22531 3.28 MB/sec warmup 42 sec latency 15.736 ms 2024-09-19T06:05:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.196+0000 7fdb1663c640 1 -- 172.21.15.5:0/2088346371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb101069b0 msgr2=0x7fdb10106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.196+0000 7fdb1663c640 1 --2- 172.21.15.5:0/2088346371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb101069b0 0x7fdb10106d90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009930 tx=0x7fdb0002a420 comp rx=0 tx=0).stop 2024-09-19T06:05:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.196+0000 7fdb1663c640 1 -- 172.21.15.5:0/2088346371 shutdown_connections 2024-09-19T06:05:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.196+0000 7fdb1663c640 1 --2- 172.21.15.5:0/2088346371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb10101340 0x7fdb1010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.196+0000 7fdb1663c640 1 --2- 172.21.15.5:0/2088346371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb101009a0 0x7fdb10100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.196+0000 7fdb1663c640 1 --2- 172.21.15.5:0/2088346371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb101069b0 0x7fdb10106d90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.196+0000 7fdb1663c640 1 -- 172.21.15.5:0/2088346371 >> 172.21.15.5:0/2088346371 conn(0x7fdb100779a0 msgr2=0x7fdb100fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.196+0000 7fdb1663c640 1 -- 172.21.15.5:0/2088346371 shutdown_connections 2024-09-19T06:05:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.196+0000 7fdb1663c640 1 -- 172.21.15.5:0/2088346371 wait complete. 2024-09-19T06:05:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb1663c640 1 Processor -- start 2024-09-19T06:05:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb1663c640 1 -- start start 2024-09-19T06:05:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb1663c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb101009a0 0x7fdb101982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb1663c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb10101340 0x7fdb10198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb1663c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb101069b0 0x7fdb1019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb1663c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb10110cf0 con 0x7fdb10101340 2024-09-19T06:05:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb1663c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb10110b70 con 0x7fdb101009a0 2024-09-19T06:05:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb1663c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdb10110e70 con 0x7fdb101069b0 2024-09-19T06:05:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb0f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb10101340 0x7fdb10198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb0f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb10101340 0x7fdb10198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55376/0 (socket says 172.21.15.5:55376) 2024-09-19T06:05:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/2154082334 learned_addr learned my addr 172.21.15.5:0/2154082334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb101009a0 0x7fdb101982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb14bd9640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb101069b0 0x7fdb1019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/2154082334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb101069b0 msgr2=0x7fdb1019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb101069b0 0x7fdb1019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/2154082334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb101009a0 msgr2=0x7fdb101982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb101009a0 0x7fdb101982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.197+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/2154082334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1019d2d0 con 0x7fdb10101340 2024-09-19T06:05:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.198+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb10101340 0x7fdb10198820 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400e9f0 tx=0x7fdb0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.198+0000 7fdb0d7fa640 1 -- 172.21.15.5:0/2154082334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0400ccc0 con 0x7fdb10101340 2024-09-19T06:05:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.198+0000 7fdb0d7fa640 1 -- 172.21.15.5:0/2154082334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb040108e0 con 0x7fdb10101340 2024-09-19T06:05:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.198+0000 7fdb1663c640 1 -- 172.21.15.5:0/2154082334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1019d5c0 con 0x7fdb10101340 2024-09-19T06:05:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.198+0000 7fdb0d7fa640 1 -- 172.21.15.5:0/2154082334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb040187c0 con 0x7fdb10101340 2024-09-19T06:05:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.198+0000 7fdb1663c640 1 -- 172.21.15.5:0/2154082334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb101a4e50 con 0x7fdb10101340 2024-09-19T06:05:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.200+0000 7fdb0d7fa640 1 -- 172.21.15.5:0/2154082334 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fdb04010a90 con 0x7fdb10101340 2024-09-19T06:05:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.200+0000 7fdb193d8740 1 Processor -- start 2024-09-19T06:05:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb193d8740 1 -- start start 2024-09-19T06:05:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb193d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 0x55ccb5450db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb193d8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54512f0 0x55ccb545fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb193d8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54600c0 0x55ccb54624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb193d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ccb53b52f0 con 0x55ccb54509d0 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb193d8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ccb535d270 con 0x55ccb54600c0 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb193d8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ccb53b12b0 con 0x55ccb54512f0 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb14bd9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 0x55ccb5450db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb0f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54600c0 0x55ccb54624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb14bd9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 0x55ccb5450db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55392/0 (socket says 172.21.15.5:55392) 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb14bd9640 1 -- 172.21.15.5:0/3320464396 learned_addr learned my addr 172.21.15.5:0/3320464396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/3320464396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54512f0 0x55ccb545fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb14bd9640 1 -- 172.21.15.5:0/3320464396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54512f0 msgr2=0x55ccb545fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb14bd9640 1 --2- 172.21.15.5:0/3320464396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54512f0 0x55ccb545fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb14bd9640 1 -- 172.21.15.5:0/3320464396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54600c0 msgr2=0x55ccb54624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb14bd9640 1 --2- 172.21.15.5:0/3320464396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54600c0 0x55ccb54624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb14bd9640 1 -- 172.21.15.5:0/3320464396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb535c340 con 0x55ccb54509d0 2024-09-19T06:05:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb0d7fa640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdaf80a33b0 0x7fdaf80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.201+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdaf80a33b0 0x7fdaf80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.202+0000 7fdb0d7fa640 1 -- 172.21.15.5:0/2154082334 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fdb040d0130 con 0x7fdb10101340 2024-09-19T06:05:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.202+0000 7fdb14bd9640 1 --2- 172.21.15.5:0/3320464396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 0x55ccb5450db0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009f90 tx=0x7fdb080118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511203bba71f6074 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.202+0000 7fdaf67fc640 1 -- 172.21.15.5:0/3320464396 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0801e070 con 0x55ccb54509d0 2024-09-19T06:05:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.202+0000 7fdaf67fc640 1 -- 172.21.15.5:0/3320464396 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb08003c30 con 0x55ccb54509d0 2024-09-19T06:05:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.202+0000 7fdaf67fc640 1 -- 172.21.15.5:0/3320464396 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0800f860 con 0x55ccb54509d0 2024-09-19T06:05:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.202+0000 7fdb193d8740 1 -- 172.21.15.5:0/3320464396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 msgr2=0x55ccb5450db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.202+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3320464396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 0x55ccb5450db0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009f90 tx=0x7fdb080118d0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.202+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdaf80a33b0 0x7fdaf80a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb00031440 tx=0x7fdb00005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.203+0000 7fdb193d8740 1 -- 172.21.15.5:0/3320464396 shutdown_connections 2024-09-19T06:05:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.203+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3320464396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54600c0 0x55ccb54624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.203+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3320464396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54512f0 0x55ccb545fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.203+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3320464396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 0x55ccb5450db0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.203+0000 7fdb193d8740 1 -- 172.21.15.5:0/3320464396 >> 172.21.15.5:0/3320464396 conn(0x55ccb544d060 msgr2=0x55ccb544d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.203+0000 7fdb193d8740 1 -- 172.21.15.5:0/3320464396 shutdown_connections 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.203+0000 7fdb193d8740 1 -- 172.21.15.5:0/3320464396 wait complete. 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.203+0000 7fdb193d8740 1 -- start start 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.203+0000 7fdb193d8740 1 Processor -- start 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.203+0000 7fdb193d8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54509d0 0x55ccb545d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.203+0000 7fdb193d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 0x55ccb545d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.203+0000 7fdb193d8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54600c0 0x55ccb545db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb193d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ccb5462cf0 con 0x55ccb54512f0 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb193d8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ccb533e290 con 0x55ccb54600c0 2024-09-19T06:05:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb193d8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ccb5462e70 con 0x55ccb54509d0 2024-09-19T06:05:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb193d8740 10 client.0 _ll_register_callbacks cb 0x55ccb536f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb0f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 0x55ccb545d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb0f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 0x55ccb545d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55406/0 (socket says 172.21.15.5:55406) 2024-09-19T06:05:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/3671021366 learned_addr learned my addr 172.21.15.5:0/3671021366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb14bd9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54600c0 0x55ccb545db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54509d0 0x55ccb545d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/3671021366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54509d0 msgr2=0x55ccb545d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54509d0 0x55ccb545d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/3671021366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54600c0 msgr2=0x55ccb545db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54600c0 0x55ccb545db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/3671021366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb50c8210 con 0x55ccb54512f0 2024-09-19T06:05:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb14bd9640 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54600c0 0x55ccb545db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 0x55ccb545d5e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdb0409c0c0 tx=0x7fdb04009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8870752efe6d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdaef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdaef7fe640 1 -- 172.21.15.5:0/3671021366 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb040cdcb0 con 0x55ccb54512f0 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdaef7fe640 1 -- 172.21.15.5:0/3671021366 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fdb040d20d0 con 0x55ccb54512f0 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdb193d8740 1 -- 172.21.15.5:0/3671021366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb08003c30 con 0x55ccb54512f0 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.204+0000 7fdaef7fe640 1 -- 172.21.15.5:0/3671021366 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb04010e00 con 0x55ccb54512f0 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.205+0000 7fdb193d8740 10 client.15867 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.205+0000 7fdb193d8740 1 -- 172.21.15.5:0/3671021366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ccb5194620 con 0x55ccb54512f0 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.205+0000 7fdaef7fe640 1 -- 172.21.15.5:0/3671021366 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb08003c30 con 0x55ccb54512f0 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.205+0000 7fdaef7fe640 1 -- 172.21.15.5:0/3671021366 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdb0409b810 con 0x55ccb54512f0 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.205+0000 7fdb193d8740 10 client.15867 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.205+0000 7fdb193d8740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.205+0000 7fdb193d8740 20 client.15867 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.205+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ccb54701b0 0x55ccb54725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.205+0000 7fdb193d8740 4 client.15867 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.205+0000 7fdb193d8740 1 -- 172.21.15.5:0/3671021366 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccb53c2a20 con 0x55ccb54701b0 2024-09-19T06:05:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.205+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ccb54701b0 0x55ccb54725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ccb54701b0 0x55ccb54725a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c59cffb279e248 server_cookie=5c93c870624ae794 in_seq=0 out_seq=0 2024-09-19T06:05:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdaef7fe640 10 client.15867 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdaef7fe640 1 -- 172.21.15.5:0/3671021366 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ccb53c2a20 con 0x55ccb54701b0 2024-09-19T06:05:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdaef7fe640 10 client.15867 handle_command_reply: tid=0 2024-09-19T06:05:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdb193d8740 1 client.15867 shutdown 2024-09-19T06:05:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdb193d8740 1 -- 172.21.15.5:0/3671021366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 msgr2=0x55ccb545d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 0x55ccb545d5e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdb0409c0c0 tx=0x7fdb04009700 comp rx=0 tx=0).stop 2024-09-19T06:05:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdb193d8740 1 -- 172.21.15.5:0/3671021366 shutdown_connections 2024-09-19T06:05:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ccb54701b0 0x55ccb54725a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54600c0 0x55ccb545db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 0x55ccb545d5e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3671021366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54509d0 0x55ccb545d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.206+0000 7fdb193d8740 1 -- 172.21.15.5:0/3671021366 >> 172.21.15.5:0/3671021366 conn(0x55ccb544d060 msgr2=0x55ccb5452dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.207+0000 7fdaef7fe640 0 client.15867 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.207+0000 7fdb193d8740 1 -- 172.21.15.5:0/3671021366 shutdown_connections 2024-09-19T06:05:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.207+0000 7fdb193d8740 1 -- 172.21.15.5:0/3671021366 wait complete. 2024-09-19T06:05:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.207+0000 7fdb193d8740 20 client.15867 trim_cache size 0 max 16384 2024-09-19T06:05:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb193d8740 1 Processor -- start 2024-09-19T06:05:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb193d8740 1 -- start start 2024-09-19T06:05:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb193d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 0x55ccb5477e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb193d8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54512f0 0x55ccb5478370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb193d8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54600c0 0x55ccb545a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb193d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb101ab530 con 0x55ccb54509d0 2024-09-19T06:05:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb193d8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb101ad0d0 con 0x55ccb54512f0 2024-09-19T06:05:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb193d8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdb101abc00 con 0x55ccb54600c0 2024-09-19T06:05:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 0x55ccb5477e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 0x55ccb5477e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55420/0 (socket says 172.21.15.5:55420) 2024-09-19T06:05:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb0ffff640 1 -- 172.21.15.5:0/3762485289 learned_addr learned my addr 172.21.15.5:0/3762485289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb14bd9640 1 --2- 172.21.15.5:0/3762485289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54600c0 0x55ccb545a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/3762485289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54512f0 0x55ccb5478370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb0ffff640 1 -- 172.21.15.5:0/3762485289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54600c0 msgr2=0x55ccb545a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/3762485289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54600c0 0x55ccb545a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb0ffff640 1 -- 172.21.15.5:0/3762485289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54512f0 msgr2=0x55ccb5478370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.222+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/3762485289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54512f0 0x55ccb5478370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.223+0000 7fdb0ffff640 1 -- 172.21.15.5:0/3762485289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb545b5f0 con 0x55ccb54509d0 2024-09-19T06:05:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.223+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/3762485289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54512f0 0x55ccb5478370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:05:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.223+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/3762485289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 0x55ccb5477e30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009930 tx=0x7fdb0003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5766ad13ce40a408 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.223+0000 7fdaf5ffb640 1 -- 172.21.15.5:0/3762485289 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00058070 con 0x55ccb54509d0 2024-09-19T06:05:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.223+0000 7fdaf5ffb640 1 -- 172.21.15.5:0/3762485289 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb00004d80 con 0x55ccb54509d0 2024-09-19T06:05:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.223+0000 7fdaf5ffb640 1 -- 172.21.15.5:0/3762485289 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00035040 con 0x55ccb54509d0 2024-09-19T06:05:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.224+0000 7fdb193d8740 1 -- 172.21.15.5:0/3762485289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 msgr2=0x55ccb5477e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.224+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3762485289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 0x55ccb5477e30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009930 tx=0x7fdb0003f000 comp rx=0 tx=0).stop 2024-09-19T06:05:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.224+0000 7fdb193d8740 1 -- 172.21.15.5:0/3762485289 shutdown_connections 2024-09-19T06:05:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.224+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3762485289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54600c0 0x55ccb545a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.224+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3762485289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54512f0 0x55ccb5478370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.224+0000 7fdb193d8740 1 --2- 172.21.15.5:0/3762485289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54509d0 0x55ccb5477e30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.224+0000 7fdb193d8740 1 -- 172.21.15.5:0/3762485289 >> 172.21.15.5:0/3762485289 conn(0x55ccb544d060 msgr2=0x55ccb545b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.224+0000 7fdb193d8740 1 -- 172.21.15.5:0/3762485289 shutdown_connections 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.224+0000 7fdb193d8740 1 -- 172.21.15.5:0/3762485289 wait complete. 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.224+0000 7fdb193d8740 1 -- start start 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.224+0000 7fdb193d8740 1 Processor -- start 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb193d8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54509d0 0x55ccb54680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb193d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 0x55ccb5468600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb193d8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54600c0 0x55ccb5459410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb193d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb101ac9a0 con 0x55ccb54512f0 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb193d8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb101ac2d0 con 0x55ccb54509d0 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb193d8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdb101a9510 con 0x55ccb54600c0 2024-09-19T06:05:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb193d8740 10 client.0 _ll_register_callbacks cb 0x7fdb04098660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb0f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 0x55ccb5468600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb0f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 0x55ccb5468600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55436/0 (socket says 172.21.15.5:55436) 2024-09-19T06:05:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/1631401347 learned_addr learned my addr 172.21.15.5:0/1631401347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54509d0 0x55ccb54680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb14bd9640 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54600c0 0x55ccb5459410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/1631401347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54600c0 msgr2=0x55ccb5459410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54600c0 0x55ccb5459410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/1631401347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54509d0 msgr2=0x55ccb54680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54509d0 0x55ccb54680c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/1631401347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb5459950 con 0x55ccb54512f0 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54509d0 0x55ccb54680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 0x55ccb5468600 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400e9c0 tx=0x7fdb0400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573faeac4ecc2375 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdaef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdaef7fe640 1 -- 172.21.15.5:0/1631401347 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0409bf00 con 0x55ccb54512f0 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdaef7fe640 1 -- 172.21.15.5:0/1631401347 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fdb04010430 con 0x55ccb54512f0 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.225+0000 7fdb193d8740 1 -- 172.21.15.5:0/1631401347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb00004d80 con 0x55ccb54512f0 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.226+0000 7fdaef7fe640 1 -- 172.21.15.5:0/1631401347 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb040cdcb0 con 0x55ccb54512f0 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.226+0000 7fdb193d8740 10 client.15873 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.226+0000 7fdb193d8740 1 -- 172.21.15.5:0/1631401347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ccb54da3b0 con 0x55ccb54512f0 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.226+0000 7fdaef7fe640 1 -- 172.21.15.5:0/1631401347 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb00004d80 con 0x55ccb54512f0 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.226+0000 7fdaef7fe640 1 -- 172.21.15.5:0/1631401347 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdb040b5140 con 0x55ccb54512f0 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.226+0000 7fdb193d8740 10 client.15873 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.226+0000 7fdb193d8740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.226+0000 7fdb193d8740 20 client.15873 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.226+0000 7fdb193d8740 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ccb54dab30 0x55ccb54dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.226+0000 7fdb193d8740 4 client.15873 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:05:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.226+0000 7fdb193d8740 1 -- 172.21.15.5:0/1631401347 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ccb53bcd70 con 0x55ccb54dab30 2024-09-19T06:05:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.226+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ccb54dab30 0x55ccb54dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.227+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ccb54dab30 0x55ccb54dcf50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c143193a33514f7e server_cookie=c3bf05aad5213c0f in_seq=0 out_seq=0 2024-09-19T06:05:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.227+0000 7fdaef7fe640 10 client.15873 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.244+0000 7fdaef7fe640 1 -- 172.21.15.5:0/1631401347 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ccb53bcd70 con 0x55ccb54dab30 2024-09-19T06:05:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.244+0000 7fdaef7fe640 10 client.15873 handle_command_reply: tid=0 2024-09-19T06:05:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.244+0000 7fdb193d8740 1 client.15873 shutdown 2024-09-19T06:05:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb193d8740 1 -- 172.21.15.5:0/1631401347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 msgr2=0x55ccb5468600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb193d8740 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 0x55ccb5468600 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400e9c0 tx=0x7fdb0400ce60 comp rx=0 tx=0).stop 2024-09-19T06:05:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb193d8740 1 -- 172.21.15.5:0/1631401347 shutdown_connections 2024-09-19T06:05:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb193d8740 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ccb54dab30 0x55ccb54dcf50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb193d8740 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccb54600c0 0x55ccb5459410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb193d8740 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccb54512f0 0x55ccb5468600 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdaef7fe640 0 client.15873 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb193d8740 1 --2- 172.21.15.5:0/1631401347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ccb54509d0 0x55ccb54680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb193d8740 1 -- 172.21.15.5:0/1631401347 >> 172.21.15.5:0/1631401347 conn(0x55ccb544d060 msgr2=0x55ccb544b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb193d8740 1 -- 172.21.15.5:0/1631401347 shutdown_connections 2024-09-19T06:05:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb193d8740 1 -- 172.21.15.5:0/1631401347 wait complete. 2024-09-19T06:05:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb193d8740 20 client.15873 trim_cache size 0 max 16384 2024-09-19T06:05:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:05:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:05:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:05:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:05:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb1663c640 1 -- 172.21.15.5:0/2154082334 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdaf80a33b0 msgr2=0x7fdaf80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb1663c640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdaf80a33b0 0x7fdaf80a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb00031440 tx=0x7fdb00005a20 comp rx=0 tx=0).stop 2024-09-19T06:05:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb1663c640 1 -- 172.21.15.5:0/2154082334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb10101340 msgr2=0x7fdb10198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.245+0000 7fdb1663c640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb10101340 0x7fdb10198820 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400e9f0 tx=0x7fdb0400edf0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.246+0000 7fdb1663c640 1 -- 172.21.15.5:0/2154082334 shutdown_connections 2024-09-19T06:05:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.246+0000 7fdb1663c640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdaf80a33b0 0x7fdaf80a5870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.246+0000 7fdb1663c640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb101069b0 0x7fdb1019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.246+0000 7fdb1663c640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb10101340 0x7fdb10198820 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.246+0000 7fdb1663c640 1 --2- 172.21.15.5:0/2154082334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb101009a0 0x7fdb101982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.246+0000 7fdb1663c640 1 -- 172.21.15.5:0/2154082334 >> 172.21.15.5:0/2154082334 conn(0x7fdb100779a0 msgr2=0x7fdb10069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.246+0000 7fdb1663c640 1 -- 172.21.15.5:0/2154082334 shutdown_connections 2024-09-19T06:05:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:17.246+0000 7fdb1663c640 1 -- 172.21.15.5:0/2154082334 wait complete. 2024-09-19T06:05:17.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 23136 3.28 MB/sec warmup 43 sec latency 16.230 ms 2024-09-19T06:05:18.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:18 smithi005 ceph-mon[30955]: pgmap v61: 97 pgs: 97 active+clean; 34 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-09-19T06:05:18.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:18 smithi049 ceph-mon[34640]: pgmap v61: 97 pgs: 97 active+clean; 34 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-09-19T06:05:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:18 smithi157 ceph-mon[35346]: pgmap v61: 97 pgs: 97 active+clean; 34 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-09-19T06:05:18.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 23732 3.25 MB/sec warmup 44 sec latency 15.102 ms 2024-09-19T06:05:19.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 24500 3.31 MB/sec warmup 45 sec latency 15.636 ms 2024-09-19T06:05:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:20 smithi005 ceph-mon[30955]: pgmap v62: 97 pgs: 97 active+clean; 36 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-09-19T06:05:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:20 smithi049 ceph-mon[34640]: pgmap v62: 97 pgs: 97 active+clean; 36 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-09-19T06:05:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:20 smithi157 ceph-mon[35346]: pgmap v62: 97 pgs: 97 active+clean; 36 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-09-19T06:05:20.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 24955 3.33 MB/sec warmup 46 sec latency 15.212 ms 2024-09-19T06:05:21.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 25267 3.29 MB/sec warmup 47 sec latency 14.882 ms 2024-09-19T06:05:22.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:22 smithi005 ceph-mon[30955]: pgmap v63: 97 pgs: 97 active+clean; 36 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-09-19T06:05:22.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:22 smithi049 ceph-mon[34640]: pgmap v63: 97 pgs: 97 active+clean; 36 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-09-19T06:05:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:22 smithi157 ceph-mon[35346]: pgmap v63: 97 pgs: 97 active+clean; 36 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-09-19T06:05:22.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 25624 3.23 MB/sec warmup 48 sec latency 15.541 ms 2024-09-19T06:05:23.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 26159 3.20 MB/sec warmup 49 sec latency 15.146 ms 2024-09-19T06:05:24.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:24 smithi005 ceph-mon[30955]: pgmap v64: 97 pgs: 97 active+clean; 37 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-19T06:05:24.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:24 smithi049 ceph-mon[34640]: pgmap v64: 97 pgs: 97 active+clean; 37 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-19T06:05:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:24 smithi157 ceph-mon[35346]: pgmap v64: 97 pgs: 97 active+clean; 37 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-19T06:05:24.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 26843 3.21 MB/sec warmup 50 sec latency 14.796 ms 2024-09-19T06:05:25.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 27370 3.19 MB/sec warmup 51 sec latency 61.154 ms 2024-09-19T06:05:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:26 smithi005 ceph-mon[30955]: pgmap v65: 97 pgs: 97 active+clean; 37 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-09-19T06:05:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:26 smithi049 ceph-mon[34640]: pgmap v65: 97 pgs: 97 active+clean; 37 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-09-19T06:05:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:26 smithi157 ceph-mon[35346]: pgmap v65: 97 pgs: 97 active+clean; 37 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-09-19T06:05:26.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 28163 3.25 MB/sec warmup 52 sec latency 14.736 ms 2024-09-19T06:05:27.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 28627 3.28 MB/sec warmup 53 sec latency 14.875 ms 2024-09-19T06:05:28.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:28 smithi005 ceph-mon[30955]: pgmap v66: 97 pgs: 97 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-09-19T06:05:28.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:28 smithi049 ceph-mon[34640]: pgmap v66: 97 pgs: 97 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-09-19T06:05:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:28 smithi157 ceph-mon[35346]: pgmap v66: 97 pgs: 97 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-09-19T06:05:28.822 INFO:tasks.workunit.client.0.smithi005.stdout: 1 28893 3.22 MB/sec warmup 54 sec latency 13.152 ms 2024-09-19T06:05:29.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 29387 3.18 MB/sec warmup 55 sec latency 10.837 ms 2024-09-19T06:05:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.340+0000 7f61e4b6d640 1 -- 172.21.15.5:0/4233763350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e0103620 msgr2=0x7f61e010bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.340+0000 7f61e4b6d640 1 --2- 172.21.15.5:0/4233763350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e0103620 0x7f61e010bb00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009900 tx=0x7f61d402a480 comp rx=0 tx=0).stop 2024-09-19T06:05:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.340+0000 7f61e4b6d640 1 -- 172.21.15.5:0/4233763350 shutdown_connections 2024-09-19T06:05:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.340+0000 7f61e4b6d640 1 --2- 172.21.15.5:0/4233763350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e010c0d0 0x7f61e010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.340+0000 7f61e4b6d640 1 --2- 172.21.15.5:0/4233763350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e0103620 0x7f61e010bb00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.340+0000 7f61e4b6d640 1 --2- 172.21.15.5:0/4233763350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0102d00 0x7f61e01030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.340+0000 7f61e4b6d640 1 -- 172.21.15.5:0/4233763350 >> 172.21.15.5:0/4233763350 conn(0x7f61e00779a0 msgr2=0x7f61e00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.340+0000 7f61e4b6d640 1 -- 172.21.15.5:0/4233763350 shutdown_connections 2024-09-19T06:05:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.340+0000 7f61e4b6d640 1 -- 172.21.15.5:0/4233763350 wait complete. 2024-09-19T06:05:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61e4b6d640 1 Processor -- start 2024-09-19T06:05:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61e4b6d640 1 -- start start 2024-09-19T06:05:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61e4b6d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0102d00 0x7f61e01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61e4b6d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0103620 0x7f61e0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61e4b6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e010c0d0 0x7f61e019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61e4b6d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0111d50 con 0x7f61e010c0d0 2024-09-19T06:05:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61e4b6d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0111bd0 con 0x7f61e0103620 2024-09-19T06:05:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61e4b6d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0111ed0 con 0x7f61e0102d00 2024-09-19T06:05:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61de59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0102d00 0x7f61e01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61ddd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0103620 0x7f61e0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61de59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0102d00 0x7f61e01982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44118/0 (socket says 172.21.15.5:44118) 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61de59c640 1 -- 172.21.15.5:0/326099565 learned_addr learned my addr 172.21.15.5:0/326099565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61ded9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e010c0d0 0x7f61e019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61de59c640 1 -- 172.21.15.5:0/326099565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0103620 msgr2=0x7f61e0198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61de59c640 1 --2- 172.21.15.5:0/326099565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0103620 0x7f61e0198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61de59c640 1 -- 172.21.15.5:0/326099565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e010c0d0 msgr2=0x7f61e019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61de59c640 1 --2- 172.21.15.5:0/326099565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e010c0d0 0x7f61e019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.341+0000 7f61de59c640 1 -- 172.21.15.5:0/326099565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e019d2d0 con 0x7f61e0102d00 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.342+0000 7f61de59c640 1 --2- 172.21.15.5:0/326099565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0102d00 0x7f61e01982e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61d000d950 tx=0x7f61d000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.342+0000 7f61cf7fe640 1 -- 172.21.15.5:0/326099565 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0017070 con 0x7f61e0102d00 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.342+0000 7f61e4b6d640 1 -- 172.21.15.5:0/326099565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e019d5c0 con 0x7f61e0102d00 2024-09-19T06:05:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.342+0000 7f61cf7fe640 1 -- 172.21.15.5:0/326099565 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61d0009c30 con 0x7f61e0102d00 2024-09-19T06:05:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.342+0000 7f61cf7fe640 1 -- 172.21.15.5:0/326099565 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d000f620 con 0x7f61e0102d00 2024-09-19T06:05:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.342+0000 7f61e4b6d640 1 -- 172.21.15.5:0/326099565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e01a4e50 con 0x7f61e0102d00 2024-09-19T06:05:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.344+0000 7f61cf7fe640 1 -- 172.21.15.5:0/326099565 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f61d00109a0 con 0x7f61e0102d00 2024-09-19T06:05:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61e79ec740 1 Processor -- start 2024-09-19T06:05:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61e79ec740 1 -- start start 2024-09-19T06:05:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61e79ec740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424daded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61e79ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424dae410 0x55a424db8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61e79ec740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424db8cb0 0x55a424dbb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61e79ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a424d122f0 con 0x55a424dae410 2024-09-19T06:05:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61e79ec740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a424cba270 con 0x55a424dadaf0 2024-09-19T06:05:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61e79ec740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a424d0e2b0 con 0x55a424db8cb0 2024-09-19T06:05:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61ded9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424daded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61de59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424dae410 0x55a424db8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61ded9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424daded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:47442/0 (socket says 172.21.15.5:47442) 2024-09-19T06:05:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61ded9d640 1 -- 172.21.15.5:0/2477159623 learned_addr learned my addr 172.21.15.5:0/2477159623 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.345+0000 7f61ddd9b640 1 --2- 172.21.15.5:0/2477159623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424db8cb0 0x55a424dbb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61cf7fe640 1 --2- 172.21.15.5:0/326099565 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61c80a33b0 0x7f61c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61ded9d640 1 -- 172.21.15.5:0/2477159623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424db8cb0 msgr2=0x55a424dbb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61ded9d640 1 --2- 172.21.15.5:0/2477159623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424db8cb0 0x55a424dbb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61ded9d640 1 -- 172.21.15.5:0/2477159623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424dae410 msgr2=0x55a424db8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61ded9d640 1 --2- 172.21.15.5:0/2477159623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424dae410 0x55a424db8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61ded9d640 1 -- 172.21.15.5:0/2477159623 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a424cb9340 con 0x55a424dadaf0 2024-09-19T06:05:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61de59c640 1 --2- 172.21.15.5:0/2477159623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424dae410 0x55a424db8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61cf7fe640 1 -- 172.21.15.5:0/326099565 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f61d00cf8f0 con 0x7f61e0102d00 2024-09-19T06:05:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61ded9d640 1 --2- 172.21.15.5:0/326099565 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61c80a33b0 0x7f61c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61ded9d640 1 --2- 172.21.15.5:0/2477159623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424daded0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009f50 tx=0x7f61d80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f40cb18b0a8e1f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61ccff9640 1 -- 172.21.15.5:0/2477159623 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8004250 con 0x55a424dadaf0 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61ccff9640 1 -- 172.21.15.5:0/2477159623 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61d800fe50 con 0x55a424dadaf0 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.346+0000 7f61ccff9640 1 -- 172.21.15.5:0/2477159623 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d801bb20 con 0x55a424dadaf0 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.347+0000 7f61e79ec740 1 -- 172.21.15.5:0/2477159623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 msgr2=0x55a424daded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.347+0000 7f61e79ec740 1 --2- 172.21.15.5:0/2477159623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424daded0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009f50 tx=0x7f61d80118d0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.347+0000 7f61ded9d640 1 --2- 172.21.15.5:0/326099565 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61c80a33b0 0x7f61c80a5870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f61d8012fd0 tx=0x7f61d8021000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.347+0000 7f61e79ec740 1 -- 172.21.15.5:0/2477159623 shutdown_connections 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.347+0000 7f61e79ec740 1 --2- 172.21.15.5:0/2477159623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424db8cb0 0x55a424dbb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.347+0000 7f61e79ec740 1 --2- 172.21.15.5:0/2477159623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424dae410 0x55a424db8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.347+0000 7f61e79ec740 1 --2- 172.21.15.5:0/2477159623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424daded0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.347+0000 7f61e79ec740 1 -- 172.21.15.5:0/2477159623 >> 172.21.15.5:0/2477159623 conn(0x55a424daa060 msgr2=0x55a424daa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.347+0000 7f61e79ec740 1 -- 172.21.15.5:0/2477159623 shutdown_connections 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.347+0000 7f61e79ec740 1 -- 172.21.15.5:0/2477159623 wait complete. 2024-09-19T06:05:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61e79ec740 1 -- start start 2024-09-19T06:05:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61e79ec740 1 Processor -- start 2024-09-19T06:05:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61e79ec740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424db5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61e79ec740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424dae410 0x55a424db60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61e79ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424db8cb0 0x55a424db65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61e79ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a424dbb880 con 0x55a424db8cb0 2024-09-19T06:05:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61e79ec740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a424c9b290 con 0x55a424dadaf0 2024-09-19T06:05:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61e79ec740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a424dbba00 con 0x55a424dae410 2024-09-19T06:05:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61e79ec740 10 client.0 _ll_register_callbacks cb 0x55a424ccc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61de59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424db5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61de59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424db5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:47448/0 (socket says 172.21.15.5:47448) 2024-09-19T06:05:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61de59c640 1 -- 172.21.15.5:0/1618737736 learned_addr learned my addr 172.21.15.5:0/1618737736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61ddd9b640 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424dae410 0x55a424db60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.348+0000 7f61ded9d640 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424db8cb0 0x55a424db65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61de59c640 1 -- 172.21.15.5:0/1618737736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424dae410 msgr2=0x55a424db60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61de59c640 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424dae410 0x55a424db60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61de59c640 1 -- 172.21.15.5:0/1618737736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424db8cb0 msgr2=0x55a424db65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61de59c640 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424db8cb0 0x55a424db65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61de59c640 1 -- 172.21.15.5:0/1618737736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a424a25210 con 0x55a424dadaf0 2024-09-19T06:05:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61de59c640 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424db5b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f61d00cd050 tx=0x7f61d00d3080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6af5cca232c5c2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:05:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61c5ffb640 1 -- 172.21.15.5:0/1618737736 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0020030 con 0x55a424dadaf0 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61c5ffb640 1 -- 172.21.15.5:0/1618737736 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f61d009b080 con 0x55a424dadaf0 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61e79ec740 1 -- 172.21.15.5:0/1618737736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d800fe50 con 0x55a424dadaf0 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61c5ffb640 1 -- 172.21.15.5:0/1618737736 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61d00cd470 con 0x55a424dadaf0 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61e79ec740 10 client.25487 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.349+0000 7f61e79ec740 1 -- 172.21.15.5:0/1618737736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a424af1620 con 0x55a424dadaf0 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.350+0000 7f61c5ffb640 1 -- 172.21.15.5:0/1618737736 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61d800fe50 con 0x55a424dadaf0 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.350+0000 7f61c5ffb640 1 -- 172.21.15.5:0/1618737736 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f61d00cfce0 con 0x55a424dadaf0 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.350+0000 7f61e79ec740 10 client.25487 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.350+0000 7f61e79ec740 10 client.25487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.350+0000 7f61e79ec740 20 client.25487 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.350+0000 7f61e79ec740 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a424dc8c60 0x55a424dcb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.350+0000 7f61e79ec740 4 client.25487 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.350+0000 7f61e79ec740 1 -- 172.21.15.5:0/1618737736 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a424d1fa20 con 0x55a424dc8c60 2024-09-19T06:05:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.350+0000 7f61ddd9b640 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a424dc8c60 0x55a424dcb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61ddd9b640 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a424dc8c60 0x55a424dcb050 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59193733a88af306 server_cookie=b91a637182869e5e in_seq=0 out_seq=0 2024-09-19T06:05:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61c5ffb640 10 client.25487 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61c5ffb640 1 -- 172.21.15.5:0/1618737736 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a424d1fa20 con 0x55a424dc8c60 2024-09-19T06:05:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61c5ffb640 10 client.25487 handle_command_reply: tid=0 2024-09-19T06:05:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61e79ec740 1 client.25487 shutdown 2024-09-19T06:05:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61e79ec740 1 -- 172.21.15.5:0/1618737736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 msgr2=0x55a424db5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61e79ec740 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424db5b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f61d00cd050 tx=0x7f61d00d3080 comp rx=0 tx=0).stop 2024-09-19T06:05:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61e79ec740 1 -- 172.21.15.5:0/1618737736 shutdown_connections 2024-09-19T06:05:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61e79ec740 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a424dc8c60 0x55a424dcb050 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61e79ec740 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424db8cb0 0x55a424db65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61e79ec740 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424dae410 0x55a424db60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61e79ec740 1 --2- 172.21.15.5:0/1618737736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424db5b60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61e79ec740 1 -- 172.21.15.5:0/1618737736 >> 172.21.15.5:0/1618737736 conn(0x55a424daa060 msgr2=0x55a424dafdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.351+0000 7f61c5ffb640 0 client.25487 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.352+0000 7f61e79ec740 1 -- 172.21.15.5:0/1618737736 shutdown_connections 2024-09-19T06:05:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.352+0000 7f61e79ec740 1 -- 172.21.15.5:0/1618737736 wait complete. 2024-09-19T06:05:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.352+0000 7f61e79ec740 20 client.25487 trim_cache size 0 max 16384 2024-09-19T06:05:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61e79ec740 1 Processor -- start 2024-09-19T06:05:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61e79ec740 1 -- start start 2024-09-19T06:05:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61e79ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424dadaf0 0x55a424db05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61e79ec740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dae410 0x55a424db6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61e79ec740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424db8cb0 0x55a424db65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61e79ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ad700 con 0x55a424dadaf0 2024-09-19T06:05:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61e79ec740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01af2a0 con 0x55a424dae410 2024-09-19T06:05:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61e79ec740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01addd0 con 0x55a424db8cb0 2024-09-19T06:05:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61ded9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424db8cb0 0x55a424db65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61ded9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424db8cb0 0x55a424db65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44142/0 (socket says 172.21.15.5:44142) 2024-09-19T06:05:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61ded9d640 1 -- 172.21.15.5:0/2251267230 learned_addr learned my addr 172.21.15.5:0/2251267230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61de59c640 1 --2- 172.21.15.5:0/2251267230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424dadaf0 0x55a424db05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61ddd9b640 1 --2- 172.21.15.5:0/2251267230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dae410 0x55a424db6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61ded9d640 1 -- 172.21.15.5:0/2251267230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dae410 msgr2=0x55a424db6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61ded9d640 1 --2- 172.21.15.5:0/2251267230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dae410 0x55a424db6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61ded9d640 1 -- 172.21.15.5:0/2251267230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424dadaf0 msgr2=0x55a424db05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61ded9d640 1 --2- 172.21.15.5:0/2251267230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424dadaf0 0x55a424db05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61ded9d640 1 -- 172.21.15.5:0/2251267230 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a424dae240 con 0x55a424db8cb0 2024-09-19T06:05:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.369+0000 7f61de59c640 1 --2- 172.21.15.5:0/2251267230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424dadaf0 0x55a424db05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.370+0000 7f61ded9d640 1 --2- 172.21.15.5:0/2251267230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424db8cb0 0x55a424db65d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009f50 tx=0x7f61d8026000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56f2bebb59c6747c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.370+0000 7f61c7fff640 1 -- 172.21.15.5:0/2251267230 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8034070 con 0x55a424db8cb0 2024-09-19T06:05:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.370+0000 7f61c7fff640 1 -- 172.21.15.5:0/2251267230 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61d801b430 con 0x55a424db8cb0 2024-09-19T06:05:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.370+0000 7f61c7fff640 1 -- 172.21.15.5:0/2251267230 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d801be60 con 0x55a424db8cb0 2024-09-19T06:05:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 -- 172.21.15.5:0/2251267230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424db8cb0 msgr2=0x55a424db65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 --2- 172.21.15.5:0/2251267230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424db8cb0 0x55a424db65d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009f50 tx=0x7f61d8026000 comp rx=0 tx=0).stop 2024-09-19T06:05:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 -- 172.21.15.5:0/2251267230 shutdown_connections 2024-09-19T06:05:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 --2- 172.21.15.5:0/2251267230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424db8cb0 0x55a424db65d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 --2- 172.21.15.5:0/2251267230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dae410 0x55a424db6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 --2- 172.21.15.5:0/2251267230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424dadaf0 0x55a424db05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 -- 172.21.15.5:0/2251267230 >> 172.21.15.5:0/2251267230 conn(0x55a424daa060 msgr2=0x55a424db7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 -- 172.21.15.5:0/2251267230 shutdown_connections 2024-09-19T06:05:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 -- 172.21.15.5:0/2251267230 wait complete. 2024-09-19T06:05:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 -- start start 2024-09-19T06:05:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 Processor -- start 2024-09-19T06:05:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424dc0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424dae410 0x55a424dc11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424db8cb0 0x55a424dbc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01aeb70 con 0x55a424db8cb0 2024-09-19T06:05:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ae4a0 con 0x55a424dadaf0 2024-09-19T06:05:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ab6e0 con 0x55a424dae410 2024-09-19T06:05:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61e79ec740 10 client.0 _ll_register_callbacks cb 0x7f61d000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61de59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424dc0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61ddd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424dae410 0x55a424dc11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.371+0000 7f61ded9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424db8cb0 0x55a424dbc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61de59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424dc0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:47468/0 (socket says 172.21.15.5:47468) 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61ddd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424dae410 0x55a424dc11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44148/0 (socket says 172.21.15.5:44148) 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61de59c640 1 -- 172.21.15.5:0/3169630464 learned_addr learned my addr 172.21.15.5:0/3169630464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61ddd9b640 1 -- 172.21.15.5:0/3169630464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 msgr2=0x55a424dc0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61ddd9b640 1 --2- 172.21.15.5:0/3169630464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424dc0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61ddd9b640 1 -- 172.21.15.5:0/3169630464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424db8cb0 msgr2=0x55a424dbc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61ddd9b640 1 --2- 172.21.15.5:0/3169630464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424db8cb0 0x55a424dbc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61ddd9b640 1 -- 172.21.15.5:0/3169630464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a424dbca30 con 0x55a424dae410 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61ded9d640 1 --2- 172.21.15.5:0/3169630464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424db8cb0 0x55a424dbc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61ddd9b640 1 --2- 172.21.15.5:0/3169630464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424dae410 0x55a424dc11b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009230 tx=0x7f61d4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b7d67b79e558e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:05:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61c5ffb640 1 -- 172.21.15.5:0/3169630464 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4035040 con 0x55a424dae410 2024-09-19T06:05:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61e79ec740 1 -- 172.21.15.5:0/3169630464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d801b430 con 0x55a424dae410 2024-09-19T06:05:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61c5ffb640 1 -- 172.21.15.5:0/3169630464 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f61d4005040 con 0x55a424dae410 2024-09-19T06:05:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.372+0000 7f61c5ffb640 1 -- 172.21.15.5:0/3169630464 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61d4054050 con 0x55a424dae410 2024-09-19T06:05:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.373+0000 7f61e79ec740 10 client.25426 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.373+0000 7f61e79ec740 1 -- 172.21.15.5:0/3169630464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a424e33410 con 0x55a424dae410 2024-09-19T06:05:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.373+0000 7f61c5ffb640 1 -- 172.21.15.5:0/3169630464 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61d801b430 con 0x55a424dae410 2024-09-19T06:05:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.373+0000 7f61c5ffb640 1 -- 172.21.15.5:0/3169630464 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f61d40035f0 con 0x55a424dae410 2024-09-19T06:05:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.373+0000 7f61e79ec740 10 client.25426 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.373+0000 7f61e79ec740 10 client.25426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.373+0000 7f61e79ec740 20 client.25426 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.373+0000 7f61e79ec740 1 --2- 172.21.15.5:0/3169630464 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a424e33c20 0x55a424e36040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.373+0000 7f61e79ec740 4 client.25426 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:05:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.373+0000 7f61e79ec740 1 -- 172.21.15.5:0/3169630464 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a424d19d70 con 0x55a424e33c20 2024-09-19T06:05:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.373+0000 7f61de59c640 1 --2- 172.21.15.5:0/3169630464 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a424e33c20 0x55a424e36040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.374+0000 7f61de59c640 1 --2- 172.21.15.5:0/3169630464 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a424e33c20 0x55a424e36040 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab49e284526ebda server_cookie=570d28df65b98eff in_seq=0 out_seq=0 2024-09-19T06:05:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.374+0000 7f61c5ffb640 10 client.25426 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.387+0000 7f61c5ffb640 1 -- 172.21.15.5:0/3169630464 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a424d19d70 con 0x55a424e33c20 2024-09-19T06:05:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.387+0000 7f61c5ffb640 10 client.25426 handle_command_reply: tid=0 2024-09-19T06:05:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.387+0000 7f61e79ec740 1 client.25426 shutdown 2024-09-19T06:05:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.388+0000 7f61e79ec740 1 -- 172.21.15.5:0/3169630464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424dae410 msgr2=0x55a424dc11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.388+0000 7f61e79ec740 1 --2- 172.21.15.5:0/3169630464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424dae410 0x55a424dc11b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009230 tx=0x7f61d4004040 comp rx=0 tx=0).stop 2024-09-19T06:05:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.388+0000 7f61e79ec740 1 -- 172.21.15.5:0/3169630464 shutdown_connections 2024-09-19T06:05:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.388+0000 7f61e79ec740 1 --2- 172.21.15.5:0/3169630464 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a424e33c20 0x55a424e36040 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.388+0000 7f61e79ec740 1 --2- 172.21.15.5:0/3169630464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a424db8cb0 0x55a424dbc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.388+0000 7f61c5ffb640 0 client.25426 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.388+0000 7f61e79ec740 1 --2- 172.21.15.5:0/3169630464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a424dae410 0x55a424dc11b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.388+0000 7f61e79ec740 1 --2- 172.21.15.5:0/3169630464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a424dadaf0 0x55a424dc0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.388+0000 7f61e79ec740 1 -- 172.21.15.5:0/3169630464 >> 172.21.15.5:0/3169630464 conn(0x55a424daa060 msgr2=0x55a424da89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.388+0000 7f61e79ec740 1 -- 172.21.15.5:0/3169630464 shutdown_connections 2024-09-19T06:05:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.388+0000 7f61e79ec740 1 -- 172.21.15.5:0/3169630464 wait complete. 2024-09-19T06:05:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.388+0000 7f61e79ec740 20 client.25426 trim_cache size 0 max 16384 2024-09-19T06:05:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:05:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:05:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:05:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:05:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.389+0000 7f61e4b6d640 1 -- 172.21.15.5:0/326099565 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61c80a33b0 msgr2=0x7f61c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.389+0000 7f61e4b6d640 1 --2- 172.21.15.5:0/326099565 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61c80a33b0 0x7f61c80a5870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f61d8012fd0 tx=0x7f61d8021000 comp rx=0 tx=0).stop 2024-09-19T06:05:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.389+0000 7f61e4b6d640 1 -- 172.21.15.5:0/326099565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0102d00 msgr2=0x7f61e01982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.389+0000 7f61e4b6d640 1 --2- 172.21.15.5:0/326099565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0102d00 0x7f61e01982e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61d000d950 tx=0x7f61d000de10 comp rx=0 tx=0).stop 2024-09-19T06:05:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.389+0000 7f61e4b6d640 1 -- 172.21.15.5:0/326099565 shutdown_connections 2024-09-19T06:05:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.389+0000 7f61e4b6d640 1 --2- 172.21.15.5:0/326099565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e010c0d0 0x7f61e019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.389+0000 7f61e4b6d640 1 --2- 172.21.15.5:0/326099565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0103620 0x7f61e0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.389+0000 7f61e4b6d640 1 --2- 172.21.15.5:0/326099565 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61c80a33b0 0x7f61c80a5870 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.389+0000 7f61e4b6d640 1 --2- 172.21.15.5:0/326099565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0102d00 0x7f61e01982e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.389+0000 7f61e4b6d640 1 -- 172.21.15.5:0/326099565 >> 172.21.15.5:0/326099565 conn(0x7f61e00779a0 msgr2=0x7f61e00fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.389+0000 7f61e4b6d640 1 -- 172.21.15.5:0/326099565 shutdown_connections 2024-09-19T06:05:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:30.389+0000 7f61e4b6d640 1 -- 172.21.15.5:0/326099565 wait complete. 2024-09-19T06:05:30.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:30 smithi005 ceph-mon[30955]: pgmap v67: 97 pgs: 97 active+clean; 38 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-09-19T06:05:30.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:30 smithi049 ceph-mon[34640]: pgmap v67: 97 pgs: 97 active+clean; 38 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-09-19T06:05:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:30 smithi157 ceph-mon[35346]: pgmap v67: 97 pgs: 97 active+clean; 38 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-09-19T06:05:30.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 29999 3.20 MB/sec warmup 56 sec latency 15.975 ms 2024-09-19T06:05:31.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:31 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:05:31.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:31 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:31 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:31 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:31 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:31 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:31 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:31 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:05:31.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:31 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:31 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:31 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:31 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:31 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:31 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 30636 3.15 MB/sec warmup 57 sec latency 14.920 ms 2024-09-19T06:05:31.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:31 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:05:31.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:31 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:31 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:31 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:31 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:31 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:31.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:31 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:32.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:32 smithi049 ceph-mon[34640]: pgmap v68: 97 pgs: 97 active+clean; 38 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:05:32.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:32 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:05:32.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:32 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:05:32.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:32 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:32.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:32 smithi005 ceph-mon[30955]: pgmap v68: 97 pgs: 97 active+clean; 38 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:05:32.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:32 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:05:32.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:32 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:05:32.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:32 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:32 smithi157 ceph-mon[35346]: pgmap v68: 97 pgs: 97 active+clean; 38 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:05:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:32 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:05:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:32 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:05:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:32 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:05:32.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 31294 3.19 MB/sec warmup 58 sec latency 15.261 ms 2024-09-19T06:05:33.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:05:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:05:33.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 31747 3.19 MB/sec warmup 59 sec latency 76.232 ms 2024-09-19T06:05:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.258+0000 7fa071a27640 1 -- 172.21.15.5:0/2397544142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa06c102ef0 msgr2=0x7fa06c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.258+0000 7fa071a27640 1 --2- 172.21.15.5:0/2397544142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa06c102ef0 0x7fa06c10da40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa064009ca0 tx=0x7fa06402a950 comp rx=0 tx=0).stop 2024-09-19T06:05:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.258+0000 7fa071a27640 1 -- 172.21.15.5:0/2397544142 shutdown_connections 2024-09-19T06:05:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.258+0000 7fa071a27640 1 --2- 172.21.15.5:0/2397544142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa06c102ef0 0x7fa06c10da40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.258+0000 7fa071a27640 1 --2- 172.21.15.5:0/2397544142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa06c102570 0x7fa06c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.258+0000 7fa071a27640 1 --2- 172.21.15.5:0/2397544142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa06c100e20 0x7fa06c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.258+0000 7fa071a27640 1 -- 172.21.15.5:0/2397544142 >> 172.21.15.5:0/2397544142 conn(0x7fa06c0fc7d0 msgr2=0x7fa06c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.258+0000 7fa071a27640 1 -- 172.21.15.5:0/2397544142 shutdown_connections 2024-09-19T06:05:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.259+0000 7fa071a27640 1 -- 172.21.15.5:0/2397544142 wait complete. 2024-09-19T06:05:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.259+0000 7fa071a27640 1 Processor -- start 2024-09-19T06:05:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.259+0000 7fa071a27640 1 -- start start 2024-09-19T06:05:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa071a27640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa06c100e20 0x7fa06c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa071a27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa06c102570 0x7fa06c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa071a27640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa06c102ef0 0x7fa06c1a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa071a27640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c110e30 con 0x7fa06c102570 2024-09-19T06:05:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa071a27640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c110cb0 con 0x7fa06c100e20 2024-09-19T06:05:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa071a27640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c110fb0 con 0x7fa06c102ef0 2024-09-19T06:05:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa06affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa06c100e20 0x7fa06c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa06affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa06c100e20 0x7fa06c1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:47496/0 (socket says 172.21.15.5:47496) 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa06affd640 1 -- 172.21.15.5:0/4016445558 learned_addr learned my addr 172.21.15.5:0/4016445558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa06c102ef0 0x7fa06c1a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa0627fc640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa06c102570 0x7fa06c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa06affd640 1 -- 172.21.15.5:0/4016445558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa06c102ef0 msgr2=0x7fa06c1a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa06affd640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa06c102ef0 0x7fa06c1a4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa06affd640 1 -- 172.21.15.5:0/4016445558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa06c102570 msgr2=0x7fa06c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa06affd640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa06c102570 0x7fa06c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa06affd640 1 -- 172.21.15.5:0/4016445558 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c1a4880 con 0x7fa06c100e20 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.260+0000 7fa0627fc640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa06c102570 0x7fa06c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.261+0000 7fa06affd640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa06c100e20 0x7fa06c1992c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00e9f0 tx=0x7fa05c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.261+0000 7fa070a25640 1 -- 172.21.15.5:0/4016445558 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00cd80 con 0x7fa06c100e20 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.261+0000 7fa071a27640 1 -- 172.21.15.5:0/4016445558 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06c1a4a50 con 0x7fa06c100e20 2024-09-19T06:05:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.261+0000 7fa070a25640 1 -- 172.21.15.5:0/4016445558 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa05c0108e0 con 0x7fa06c100e20 2024-09-19T06:05:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.261+0000 7fa070a25640 1 -- 172.21.15.5:0/4016445558 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c018920 con 0x7fa06c100e20 2024-09-19T06:05:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.261+0000 7fa071a27640 1 -- 172.21.15.5:0/4016445558 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06c1a4f90 con 0x7fa06c100e20 2024-09-19T06:05:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.264+0000 7fa073e00740 1 Processor -- start 2024-09-19T06:05:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.264+0000 7fa070a25640 1 -- 172.21.15.5:0/4016445558 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa05c010a90 con 0x7fa06c100e20 2024-09-19T06:05:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.264+0000 7fa073e00740 1 -- start start 2024-09-19T06:05:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.264+0000 7fa073e00740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3980f380 0x557f3980f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.264+0000 7fa073e00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980fca0 0x557f3981e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.264+0000 7fa073e00740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3981ea90 0x557f39820e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.264+0000 7fa073e00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f397752e0 con 0x557f3980fca0 2024-09-19T06:05:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.264+0000 7fa073e00740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557f3971cce0 con 0x557f3980f380 2024-09-19T06:05:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.264+0000 7fa073e00740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557f397712a0 con 0x557f3981ea90 2024-09-19T06:05:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.265+0000 7fa06affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980fca0 0x557f3981e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.265+0000 7fa06b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3980f380 0x557f3980f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.265+0000 7fa06b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3980f380 0x557f3980f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:47500/0 (socket says 172.21.15.5:47500) 2024-09-19T06:05:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.265+0000 7fa06affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980fca0 0x557f3981e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45406/0 (socket says 172.21.15.5:45406) 2024-09-19T06:05:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.265+0000 7fa06b7fe640 1 -- 172.21.15.5:0/1754955775 learned_addr learned my addr 172.21.15.5:0/1754955775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.265+0000 7fa0627fc640 1 --2- 172.21.15.5:0/1754955775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3981ea90 0x557f39820e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.265+0000 7fa06b7fe640 1 -- 172.21.15.5:0/1754955775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3981ea90 msgr2=0x557f39820e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.265+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/1754955775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3981ea90 0x557f39820e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.265+0000 7fa06b7fe640 1 -- 172.21.15.5:0/1754955775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980fca0 msgr2=0x557f3981e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.265+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/1754955775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980fca0 0x557f3981e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.265+0000 7fa06b7fe640 1 -- 172.21.15.5:0/1754955775 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f39701440 con 0x557f3980f380 2024-09-19T06:05:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.266+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/1754955775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3980f380 0x557f3980f760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa064035000 tx=0x7fa0640093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c687ab40cffef99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.266+0000 7fa070a25640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa0580a3430 0x7fa0580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.266+0000 7fa06affd640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa0580a3430 0x7fa0580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.266+0000 7fa061ffb640 1 -- 172.21.15.5:0/1754955775 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064048070 con 0x557f3980f380 2024-09-19T06:05:34.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.266+0000 7fa070a25640 1 -- 172.21.15.5:0/4016445558 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fa05c0d0040 con 0x7fa06c100e20 2024-09-19T06:05:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.266+0000 7fa061ffb640 1 -- 172.21.15.5:0/1754955775 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa06402f8c0 con 0x557f3980f380 2024-09-19T06:05:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.266+0000 7fa061ffb640 1 -- 172.21.15.5:0/1754955775 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064034040 con 0x557f3980f380 2024-09-19T06:05:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.267+0000 7fa073e00740 1 -- 172.21.15.5:0/1754955775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3980f380 msgr2=0x557f3980f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.267+0000 7fa06affd640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa0580a3430 0x7fa0580a58f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0cd850 tx=0x7fa05c0d3b90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.267+0000 7fa073e00740 1 --2- 172.21.15.5:0/1754955775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3980f380 0x557f3980f760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa064035000 tx=0x7fa0640093e0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.267+0000 7fa073e00740 1 -- 172.21.15.5:0/1754955775 shutdown_connections 2024-09-19T06:05:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.267+0000 7fa073e00740 1 --2- 172.21.15.5:0/1754955775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3981ea90 0x557f39820e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.267+0000 7fa073e00740 1 --2- 172.21.15.5:0/1754955775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980fca0 0x557f3981e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.267+0000 7fa073e00740 1 --2- 172.21.15.5:0/1754955775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3980f380 0x557f3980f760 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.267+0000 7fa073e00740 1 -- 172.21.15.5:0/1754955775 >> 172.21.15.5:0/1754955775 conn(0x557f3980ba30 msgr2=0x557f3980be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.267+0000 7fa073e00740 1 -- 172.21.15.5:0/1754955775 shutdown_connections 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.267+0000 7fa073e00740 1 -- 172.21.15.5:0/1754955775 wait complete. 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.267+0000 7fa073e00740 1 -- start start 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.268+0000 7fa073e00740 1 Processor -- start 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.268+0000 7fa073e00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980f380 0x557f3981e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.268+0000 7fa073e00740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980fca0 0x557f3981b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.268+0000 7fa073e00740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3981ea90 0x557f3981bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.268+0000 7fa073e00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f39821660 con 0x557f3980f380 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.268+0000 7fa073e00740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557f39730050 con 0x557f3981ea90 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.268+0000 7fa073e00740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557f398217e0 con 0x557f3980fca0 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.268+0000 7fa073e00740 10 client.0 _ll_register_callbacks cb 0x557f39770160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.269+0000 7fa0627fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980fca0 0x557f3981b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.269+0000 7fa0627fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980fca0 0x557f3981b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44182/0 (socket says 172.21.15.5:44182) 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.269+0000 7fa0627fc640 1 -- 172.21.15.5:0/921229083 learned_addr learned my addr 172.21.15.5:0/921229083 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.269+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/921229083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980f380 0x557f3981e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.269+0000 7fa0627fc640 1 -- 172.21.15.5:0/921229083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3981ea90 msgr2=0x557f3981bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.269+0000 7fa0627fc640 1 --2- 172.21.15.5:0/921229083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3981ea90 0x557f3981bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.269+0000 7fa0627fc640 1 -- 172.21.15.5:0/921229083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980f380 msgr2=0x557f3981e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.269+0000 7fa0627fc640 1 --2- 172.21.15.5:0/921229083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980f380 0x557f3981e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.269+0000 7fa0627fc640 1 -- 172.21.15.5:0/921229083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f394e63c0 con 0x557f3980fca0 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.269+0000 7fa0627fc640 1 --2- 172.21.15.5:0/921229083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980fca0 0x557f3981b7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa0540136e0 tx=0x7fa054013ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea9e3ade49e060f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.269+0000 7fa04affd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.269+0000 7fa04affd640 1 -- 172.21.15.5:0/921229083 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054011dd0 con 0x557f3980fca0 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa073e00740 1 -- 172.21.15.5:0/921229083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa06402f8c0 con 0x557f3980fca0 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa04affd640 1 -- 172.21.15.5:0/921229083 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fa054004030 con 0x557f3980fca0 2024-09-19T06:05:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa04affd640 1 -- 172.21.15.5:0/921229083 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa054004230 con 0x557f3980fca0 2024-09-19T06:05:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa073e00740 10 client.25438 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa073e00740 1 -- 172.21.15.5:0/921229083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f398315f0 con 0x557f3980fca0 2024-09-19T06:05:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa04affd640 1 -- 172.21.15.5:0/921229083 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0540043d0 con 0x557f3980fca0 2024-09-19T06:05:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa04affd640 1 -- 172.21.15.5:0/921229083 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa05400f040 con 0x557f3980fca0 2024-09-19T06:05:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa073e00740 10 client.25438 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa073e00740 10 client.25438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa073e00740 20 client.25438 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa073e00740 1 --2- 172.21.15.5:0/921229083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557f39832aa0 0x557f39834e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa073e00740 4 client.25438 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.270+0000 7fa073e00740 1 -- 172.21.15.5:0/921229083 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa06402f8c0 con 0x557f39832aa0 2024-09-19T06:05:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.271+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/921229083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557f39832aa0 0x557f39834e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.271+0000 7fa04affd640 10 client.25438 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.272+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/921229083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557f39832aa0 0x557f39834e90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867125e24319eec6 server_cookie=4724291c117f8c5 in_seq=0 out_seq=0 2024-09-19T06:05:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.272+0000 7fa04affd640 1 -- 172.21.15.5:0/921229083 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa06402f8c0 con 0x557f39832aa0 2024-09-19T06:05:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.272+0000 7fa04affd640 10 client.25438 handle_command_reply: tid=0 2024-09-19T06:05:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.272+0000 7fa073e00740 1 client.25438 shutdown 2024-09-19T06:05:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.273+0000 7fa073e00740 1 -- 172.21.15.5:0/921229083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980fca0 msgr2=0x557f3981b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.273+0000 7fa073e00740 1 --2- 172.21.15.5:0/921229083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980fca0 0x557f3981b7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa0540136e0 tx=0x7fa054013ba0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.273+0000 7fa073e00740 1 -- 172.21.15.5:0/921229083 shutdown_connections 2024-09-19T06:05:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.273+0000 7fa073e00740 1 --2- 172.21.15.5:0/921229083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557f39832aa0 0x557f39834e90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.273+0000 7fa073e00740 1 --2- 172.21.15.5:0/921229083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3981ea90 0x557f3981bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.273+0000 7fa073e00740 1 --2- 172.21.15.5:0/921229083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980fca0 0x557f3981b7a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.273+0000 7fa073e00740 1 --2- 172.21.15.5:0/921229083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980f380 0x557f3981e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.273+0000 7fa04affd640 0 client.25438 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.273+0000 7fa073e00740 1 -- 172.21.15.5:0/921229083 >> 172.21.15.5:0/921229083 conn(0x557f3980ba30 msgr2=0x557f39811790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.273+0000 7fa073e00740 1 -- 172.21.15.5:0/921229083 shutdown_connections 2024-09-19T06:05:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.273+0000 7fa073e00740 1 -- 172.21.15.5:0/921229083 wait complete. 2024-09-19T06:05:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.273+0000 7fa073e00740 20 client.25438 trim_cache size 0 max 16384 2024-09-19T06:05:34.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.307+0000 7fa073e00740 1 Processor -- start 2024-09-19T06:05:34.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.307+0000 7fa073e00740 1 -- start start 2024-09-19T06:05:34.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.308+0000 7fa073e00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980f380 0x557f3981d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.308+0000 7fa073e00740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980fca0 0x557f3983a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.308+0000 7fa073e00740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3981ea90 0x557f3983aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.308+0000 7fa06b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980f380 0x557f3981d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.308+0000 7fa0627fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980fca0 0x557f3983a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.308+0000 7fa06b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980f380 0x557f3981d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45422/0 (socket says 172.21.15.5:45422) 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.308+0000 7fa06b7fe640 1 -- 172.21.15.5:0/2738475683 learned_addr learned my addr 172.21.15.5:0/2738475683 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.308+0000 7fa073e00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1ab630 con 0x557f3980f380 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.308+0000 7fa073e00740 1 -- 172.21.15.5:0/2738475683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1ad1d0 con 0x557f3981ea90 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.308+0000 7fa073e00740 1 -- 172.21.15.5:0/2738475683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1abd00 con 0x557f3980fca0 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.309+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/2738475683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3981ea90 0x557f3983aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.309+0000 7fa06b7fe640 1 -- 172.21.15.5:0/2738475683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980fca0 msgr2=0x557f3983a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.309+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/2738475683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980fca0 0x557f3983a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.309+0000 7fa06b7fe640 1 -- 172.21.15.5:0/2738475683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3981ea90 msgr2=0x557f3983aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.309+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/2738475683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3981ea90 0x557f3983aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.309+0000 7fa06b7fe640 1 -- 172.21.15.5:0/2738475683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f3981deb0 con 0x557f3980f380 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.309+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/2738475683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980f380 0x557f3981d000 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa064009ca0 tx=0x7fa06402ffe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7bfda938a7df0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.309+0000 7fa0617fa640 1 -- 172.21.15.5:0/2738475683 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064054030 con 0x557f3980f380 2024-09-19T06:05:34.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.309+0000 7fa0617fa640 1 -- 172.21.15.5:0/2738475683 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa06402fb00 con 0x557f3980f380 2024-09-19T06:05:34.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.309+0000 7fa0617fa640 1 -- 172.21.15.5:0/2738475683 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064048040 con 0x557f3980f380 2024-09-19T06:05:34.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.310+0000 7fa073e00740 1 -- 172.21.15.5:0/2738475683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980f380 msgr2=0x557f3981d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:34.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.310+0000 7fa073e00740 1 --2- 172.21.15.5:0/2738475683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980f380 0x557f3981d000 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa064009ca0 tx=0x7fa06402ffe0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.310+0000 7fa073e00740 1 -- 172.21.15.5:0/2738475683 shutdown_connections 2024-09-19T06:05:34.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.310+0000 7fa073e00740 1 --2- 172.21.15.5:0/2738475683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3981ea90 0x557f3983aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.310+0000 7fa073e00740 1 --2- 172.21.15.5:0/2738475683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980fca0 0x557f3983a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.310+0000 7fa073e00740 1 --2- 172.21.15.5:0/2738475683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3980f380 0x557f3981d000 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.310+0000 7fa073e00740 1 -- 172.21.15.5:0/2738475683 >> 172.21.15.5:0/2738475683 conn(0x557f3980ba30 msgr2=0x557f3980ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:34.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.310+0000 7fa073e00740 1 -- 172.21.15.5:0/2738475683 shutdown_connections 2024-09-19T06:05:34.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.310+0000 7fa073e00740 1 -- 172.21.15.5:0/2738475683 wait complete. 2024-09-19T06:05:34.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.311+0000 7fa073e00740 1 -- start start 2024-09-19T06:05:34.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.311+0000 7fa073e00740 1 Processor -- start 2024-09-19T06:05:34.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.312+0000 7fa073e00740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980f380 0x557f3982d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.312+0000 7fa073e00740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3980fca0 0x557f3982dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.312+0000 7fa06b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980f380 0x557f3982d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.312+0000 7fa06b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980f380 0x557f3982d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44200/0 (socket says 172.21.15.5:44200) 2024-09-19T06:05:34.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.312+0000 7fa06b7fe640 1 -- 172.21.15.5:0/3145677286 learned_addr learned my addr 172.21.15.5:0/3145677286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:34.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.312+0000 7fa0627fc640 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3980fca0 0x557f3982dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.312+0000 7fa073e00740 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3981ea90 0x557f3989c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.312+0000 7fa073e00740 1 -- 172.21.15.5:0/3145677286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1acaa0 con 0x557f3981ea90 2024-09-19T06:05:34.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.312+0000 7fa073e00740 1 -- 172.21.15.5:0/3145677286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1ac3d0 con 0x557f3980fca0 2024-09-19T06:05:34.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.312+0000 7fa073e00740 1 -- 172.21.15.5:0/3145677286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1a9610 con 0x557f3980f380 2024-09-19T06:05:34.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.312+0000 7fa073e00740 10 client.0 _ll_register_callbacks cb 0x7fa054005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.313+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3981ea90 0x557f3989c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.313+0000 7fa06b7fe640 1 -- 172.21.15.5:0/3145677286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3980fca0 msgr2=0x557f3982dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.313+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3980fca0 0x557f3982dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.313+0000 7fa06b7fe640 1 -- 172.21.15.5:0/3145677286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3981ea90 msgr2=0x557f3989c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.313+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3981ea90 0x557f3989c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.313+0000 7fa06b7fe640 1 -- 172.21.15.5:0/3145677286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f3989cc80 con 0x557f3980f380 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.313+0000 7fa06b7fe640 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980f380 0x557f3982d8a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa06402ffb0 tx=0x7fa06402a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c904163cb292c1ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.313+0000 7fa04affd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.313+0000 7fa04affd640 1 -- 172.21.15.5:0/3145677286 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06405f020 con 0x557f3980f380 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.313+0000 7fa04affd640 1 -- 172.21.15.5:0/3145677286 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fa064003bf0 con 0x557f3980f380 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.314+0000 7fa073e00740 1 -- 172.21.15.5:0/3145677286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa06402fb00 con 0x557f3980f380 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.314+0000 7fa04affd640 1 -- 172.21.15.5:0/3145677286 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa064048070 con 0x557f3980f380 2024-09-19T06:05:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.314+0000 7fa073e00740 10 client.25444 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.314+0000 7fa073e00740 1 -- 172.21.15.5:0/3145677286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f3989dea0 con 0x557f3980f380 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.314+0000 7fa04affd640 1 -- 172.21.15.5:0/3145677286 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa06402fb00 con 0x557f3980f380 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.314+0000 7fa04affd640 1 -- 172.21.15.5:0/3145677286 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa064062020 con 0x557f3980f380 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.314+0000 7fa073e00740 10 client.25444 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.314+0000 7fa073e00740 10 client.25444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.314+0000 7fa073e00740 20 client.25444 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.314+0000 7fa073e00740 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557f3989e0c0 0x557f398a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.315+0000 7fa073e00740 4 client.25444 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.315+0000 7fa073e00740 1 -- 172.21.15.5:0/3145677286 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f39782850 con 0x557f3989e0c0 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.315+0000 7fa0627fc640 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557f3989e0c0 0x557f398a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa0627fc640 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557f3989e0c0 0x557f398a04b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=668bbc9e25eb111a server_cookie=634e9908ed094410 in_seq=0 out_seq=0 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa04affd640 10 client.25444 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa04affd640 1 -- 172.21.15.5:0/3145677286 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557f39782850 con 0x557f3989e0c0 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa04affd640 10 client.25444 handle_command_reply: tid=0 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa073e00740 1 client.25444 shutdown 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa073e00740 1 -- 172.21.15.5:0/3145677286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980f380 msgr2=0x557f3982d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa073e00740 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980f380 0x557f3982d8a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa06402ffb0 tx=0x7fa06402a950 comp rx=0 tx=0).stop 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa073e00740 1 -- 172.21.15.5:0/3145677286 shutdown_connections 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa073e00740 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557f3989e0c0 0x557f398a04b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa073e00740 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f3981ea90 0x557f3989c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa073e00740 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f3980fca0 0x557f3982dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa073e00740 1 --2- 172.21.15.5:0/3145677286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f3980f380 0x557f3982d8a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa073e00740 1 -- 172.21.15.5:0/3145677286 >> 172.21.15.5:0/3145677286 conn(0x557f3980ba30 msgr2=0x557f3980a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa04affd640 0 client.25444 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa073e00740 1 -- 172.21.15.5:0/3145677286 shutdown_connections 2024-09-19T06:05:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa073e00740 1 -- 172.21.15.5:0/3145677286 wait complete. 2024-09-19T06:05:34.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.316+0000 7fa073e00740 20 client.25444 trim_cache size 0 max 16384 2024-09-19T06:05:34.327 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:05:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:05:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:05:34.327 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:05:34.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.317+0000 7fa071a27640 1 -- 172.21.15.5:0/4016445558 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa0580a3430 msgr2=0x7fa0580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:34.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.317+0000 7fa071a27640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa0580a3430 0x7fa0580a58f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0cd850 tx=0x7fa05c0d3b90 comp rx=0 tx=0).stop 2024-09-19T06:05:34.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.317+0000 7fa071a27640 1 -- 172.21.15.5:0/4016445558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa06c100e20 msgr2=0x7fa06c1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:34.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.317+0000 7fa071a27640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa06c100e20 0x7fa06c1992c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00e9f0 tx=0x7fa05c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.317+0000 7fa071a27640 1 -- 172.21.15.5:0/4016445558 shutdown_connections 2024-09-19T06:05:34.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.317+0000 7fa071a27640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa0580a3430 0x7fa0580a58f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.317+0000 7fa071a27640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa06c102ef0 0x7fa06c1a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.317+0000 7fa071a27640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa06c102570 0x7fa06c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.317+0000 7fa071a27640 1 --2- 172.21.15.5:0/4016445558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa06c100e20 0x7fa06c1992c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:34.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.317+0000 7fa071a27640 1 -- 172.21.15.5:0/4016445558 >> 172.21.15.5:0/4016445558 conn(0x7fa06c0fc7d0 msgr2=0x7fa06c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:34.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.318+0000 7fa071a27640 1 -- 172.21.15.5:0/4016445558 shutdown_connections 2024-09-19T06:05:34.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:34.318+0000 7fa071a27640 1 -- 172.21.15.5:0/4016445558 wait complete. 2024-09-19T06:05:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:34 smithi005 ceph-mon[30955]: pgmap v69: 97 pgs: 97 active+clean; 33 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-19T06:05:34.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:34 smithi049 ceph-mon[34640]: pgmap v69: 97 pgs: 97 active+clean; 33 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-19T06:05:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:34 smithi157 ceph-mon[35346]: pgmap v69: 97 pgs: 97 active+clean; 33 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-19T06:05:34.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 32232 3.21 MB/sec warmup 60 sec latency 16.963 ms 2024-09-19T06:05:35.047 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:05:35.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:05:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:05:35.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.808+0000 7f89b40a9640 1 -- 172.21.15.5:0/2616162572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac102d30 msgr2=0x7f89ac103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:35.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.808+0000 7f89b40a9640 1 --2- 172.21.15.5:0/2616162572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac102d30 0x7f89ac103110 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009930 tx=0x7f89a802a420 comp rx=0 tx=0).stop 2024-09-19T06:05:35.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.808+0000 7f89b40a9640 1 -- 172.21.15.5:0/2616162572 shutdown_connections 2024-09-19T06:05:35.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.808+0000 7f89b40a9640 1 --2- 172.21.15.5:0/2616162572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89ac1082a0 0x7f89ac10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.808+0000 7f89b40a9640 1 --2- 172.21.15.5:0/2616162572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac1036e0 0x7f89ac107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.808+0000 7f89b40a9640 1 --2- 172.21.15.5:0/2616162572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac102d30 0x7f89ac103110 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.808+0000 7f89b40a9640 1 -- 172.21.15.5:0/2616162572 >> 172.21.15.5:0/2616162572 conn(0x7f89ac0fc7d0 msgr2=0x7f89ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.808+0000 7f89b40a9640 1 -- 172.21.15.5:0/2616162572 shutdown_connections 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.808+0000 7f89b40a9640 1 -- 172.21.15.5:0/2616162572 wait complete. 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b40a9640 1 Processor -- start 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b40a9640 1 -- start start 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b40a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89ac102d30 0x7f89ac1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b40a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac1036e0 0x7f89ac198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b40a9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac1082a0 0x7f89ac19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b40a9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac111f50 con 0x7f89ac1036e0 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b40a9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac111dd0 con 0x7f89ac1082a0 2024-09-19T06:05:35.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b40a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac1120d0 con 0x7f89ac102d30 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b1644640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac1036e0 0x7f89ac198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b1644640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac1036e0 0x7f89ac198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56014/0 (socket says 172.21.15.5:56014) 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b1e45640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89ac102d30 0x7f89ac1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b1644640 1 -- 172.21.15.5:0/3906247574 learned_addr learned my addr 172.21.15.5:0/3906247574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b2646640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac1082a0 0x7f89ac19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b1644640 1 -- 172.21.15.5:0/3906247574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89ac102d30 msgr2=0x7f89ac1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b1644640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89ac102d30 0x7f89ac1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b1644640 1 -- 172.21.15.5:0/3906247574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac1082a0 msgr2=0x7f89ac19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b1644640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac1082a0 0x7f89ac19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.809+0000 7f89b1644640 1 -- 172.21.15.5:0/3906247574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ac19d4a0 con 0x7f89ac1036e0 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.810+0000 7f89b1644640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac1036e0 0x7f89ac198a30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f899c00e9f0 tx=0x7f899c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.810+0000 7f89a2ffd640 1 -- 172.21.15.5:0/3906247574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c00ccc0 con 0x7f89ac1036e0 2024-09-19T06:05:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.810+0000 7f89a2ffd640 1 -- 172.21.15.5:0/3906247574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f899c0108e0 con 0x7f89ac1036e0 2024-09-19T06:05:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.810+0000 7f89b40a9640 1 -- 172.21.15.5:0/3906247574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ac19d790 con 0x7f89ac1036e0 2024-09-19T06:05:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.810+0000 7f89a2ffd640 1 -- 172.21.15.5:0/3906247574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c0187e0 con 0x7f89ac1036e0 2024-09-19T06:05:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.810+0000 7f89b40a9640 1 -- 172.21.15.5:0/3906247574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ac1a5070 con 0x7f89ac1036e0 2024-09-19T06:05:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.811+0000 7f89b40a9640 1 -- 172.21.15.5:0/3906247574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ac06a850 con 0x7f89ac1036e0 2024-09-19T06:05:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.812+0000 7f89a2ffd640 1 -- 172.21.15.5:0/3906247574 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f899c010a90 con 0x7f89ac1036e0 2024-09-19T06:05:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.813+0000 7f89a2ffd640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f89980a33b0 0x7f89980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.813+0000 7f89b1e45640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f89980a33b0 0x7f89980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.813+0000 7f89a2ffd640 1 -- 172.21.15.5:0/3906247574 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f899c0d0510 con 0x7f89ac1036e0 2024-09-19T06:05:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.814+0000 7f89b1e45640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f89980a33b0 0x7f89980a5870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009900 tx=0x7f89a8000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:35.817+0000 7f89a2ffd640 1 -- 172.21.15.5:0/3906247574 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f899c09c8b0 con 0x7f89ac1036e0 2024-09-19T06:05:35.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 32518 3.17 MB/sec warmup 61 sec latency 11.775 ms 2024-09-19T06:05:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.056+0000 7f89b40a9640 1 -- 172.21.15.5:0/3906247574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89ac10a2b0 con 0x7f89ac1036e0 2024-09-19T06:05:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.056+0000 7f89a2ffd640 1 -- 172.21.15.5:0/3906247574 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11962 (secure 0 0 0) 0x7f899c0a0bb0 con 0x7f89ac1036e0 2024-09-19T06:05:36.059 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:05:36.060 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-19T06:04:49:315680+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:04:49.014116+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/2353689058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2353689058},{"type":"v1","addr":"172.21.15.49:6835","nonce":2353689058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:05:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-19T06:05:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.058+0000 7f89b40a9640 1 -- 172.21.15.5:0/3906247574 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f89980a33b0 msgr2=0x7f89980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.059+0000 7f89b40a9640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f89980a33b0 0x7f89980a5870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009900 tx=0x7f89a8000bc0 comp rx=0 tx=0).stop 2024-09-19T06:05:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.059+0000 7f89b40a9640 1 -- 172.21.15.5:0/3906247574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac1036e0 msgr2=0x7f89ac198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.059+0000 7f89b40a9640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac1036e0 0x7f89ac198a30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f899c00e9f0 tx=0x7f899c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:05:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.059+0000 7f89b40a9640 1 -- 172.21.15.5:0/3906247574 shutdown_connections 2024-09-19T06:05:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.059+0000 7f89b40a9640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac1082a0 0x7f89ac19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.059+0000 7f89b40a9640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac1036e0 0x7f89ac198a30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.059+0000 7f89b40a9640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f89980a33b0 0x7f89980a5870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.059+0000 7f89b40a9640 1 --2- 172.21.15.5:0/3906247574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89ac102d30 0x7f89ac1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.059+0000 7f89b40a9640 1 -- 172.21.15.5:0/3906247574 >> 172.21.15.5:0/3906247574 conn(0x7f89ac0fc7d0 msgr2=0x7f89ac105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.059+0000 7f89b40a9640 1 -- 172.21.15.5:0/3906247574 shutdown_connections 2024-09-19T06:05:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:36.059+0000 7f89b40a9640 1 -- 172.21.15.5:0/3906247574 wait complete. 2024-09-19T06:05:36.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:36 smithi005 ceph-mon[30955]: pgmap v70: 97 pgs: 97 active+clean; 33 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-19T06:05:36.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:36 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3906247574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:05:36.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:36 smithi049 ceph-mon[34640]: pgmap v70: 97 pgs: 97 active+clean; 33 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-19T06:05:36.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:36 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3906247574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:05:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:36 smithi157 ceph-mon[35346]: pgmap v70: 97 pgs: 97 active+clean; 33 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-19T06:05:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:36 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3906247574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:05:36.809 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:05:36.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 33021 3.15 MB/sec warmup 62 sec latency 13.487 ms 2024-09-19T06:05:37.096 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:05:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.699+0000 7f51c8529640 1 -- 172.21.15.5:0/3969659825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c0101360 msgr2=0x7f51c010df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.699+0000 7f51c8529640 1 --2- 172.21.15.5:0/3969659825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c0101360 0x7f51c010df90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009900 tx=0x7f51b002a410 comp rx=0 tx=0).stop 2024-09-19T06:05:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.699+0000 7f51c8529640 1 -- 172.21.15.5:0/3969659825 shutdown_connections 2024-09-19T06:05:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.699+0000 7f51c8529640 1 --2- 172.21.15.5:0/3969659825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c010e4d0 0x7f51c01108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.699+0000 7f51c8529640 1 --2- 172.21.15.5:0/3969659825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c0101360 0x7f51c010df90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.699+0000 7f51c8529640 1 --2- 172.21.15.5:0/3969659825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c01009b0 0x7f51c0100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.699+0000 7f51c8529640 1 -- 172.21.15.5:0/3969659825 >> 172.21.15.5:0/3969659825 conn(0x7f51c00fc7d0 msgr2=0x7f51c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.699+0000 7f51c8529640 1 -- 172.21.15.5:0/3969659825 shutdown_connections 2024-09-19T06:05:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.699+0000 7f51c8529640 1 -- 172.21.15.5:0/3969659825 wait complete. 2024-09-19T06:05:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.700+0000 7f51c8529640 1 Processor -- start 2024-09-19T06:05:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.700+0000 7f51c8529640 1 -- start start 2024-09-19T06:05:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.700+0000 7f51c8529640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01009b0 0x7f51c019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.700+0000 7f51c8529640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c0101360 0x7f51c019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.700+0000 7f51c8529640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c010e4d0 0x7f51c019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.700+0000 7f51c8529640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01140b0 con 0x7f51c01009b0 2024-09-19T06:05:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.700+0000 7f51c8529640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f51c0113f30 con 0x7f51c0101360 2024-09-19T06:05:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.700+0000 7f51c8529640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51c0114230 con 0x7f51c010e4d0 2024-09-19T06:05:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.700+0000 7f51c62c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01009b0 0x7f51c019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.700+0000 7f51c62c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01009b0 0x7f51c019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56028/0 (socket says 172.21.15.5:56028) 2024-09-19T06:05:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.700+0000 7f51c62c5640 1 -- 172.21.15.5:0/1636695378 learned_addr learned my addr 172.21.15.5:0/1636695378 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.700+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c010e4d0 0x7f51c019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.701+0000 7f51c5ac4640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c0101360 0x7f51c019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.701+0000 7f51c62c5640 1 -- 172.21.15.5:0/1636695378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c010e4d0 msgr2=0x7f51c019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.701+0000 7f51c62c5640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c010e4d0 0x7f51c019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.701+0000 7f51c62c5640 1 -- 172.21.15.5:0/1636695378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c0101360 msgr2=0x7f51c019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.701+0000 7f51c62c5640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c0101360 0x7f51c019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.701+0000 7f51c62c5640 1 -- 172.21.15.5:0/1636695378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c019e6d0 con 0x7f51c01009b0 2024-09-19T06:05:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.701+0000 7f51c62c5640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01009b0 0x7f51c019a780 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00d950 tx=0x7f51bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.701+0000 7f51b77fe640 1 -- 172.21.15.5:0/1636695378 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc017070 con 0x7f51c01009b0 2024-09-19T06:05:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.701+0000 7f51b77fe640 1 -- 172.21.15.5:0/1636695378 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51bc009c30 con 0x7f51c01009b0 2024-09-19T06:05:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.701+0000 7f51c8529640 1 -- 172.21.15.5:0/1636695378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c019e9c0 con 0x7f51c01009b0 2024-09-19T06:05:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.701+0000 7f51b77fe640 1 -- 172.21.15.5:0/1636695378 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc00f620 con 0x7f51c01009b0 2024-09-19T06:05:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.701+0000 7f51c8529640 1 -- 172.21.15.5:0/1636695378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c01a7330 con 0x7f51c01009b0 2024-09-19T06:05:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.704+0000 7f51b77fe640 1 -- 172.21.15.5:0/1636695378 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f51bc0109a0 con 0x7f51c01009b0 2024-09-19T06:05:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.705+0000 7f51ca902740 1 Processor -- start 2024-09-19T06:05:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.705+0000 7f51ca902740 1 -- start start 2024-09-19T06:05:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.705+0000 7f51ca902740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 0x55f9bf3fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.705+0000 7f51ca902740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf3fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.705+0000 7f51ca902740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf408370 0x55f9bf40a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.705+0000 7f51ca902740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9bf3632f0 con 0x55f9bf3fd5d0 2024-09-19T06:05:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.705+0000 7f51ca902740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f9bf30acf0 con 0x55f9bf3feb40 2024-09-19T06:05:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.705+0000 7f51ca902740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f9bf35f2b0 con 0x55f9bf408370 2024-09-19T06:05:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.705+0000 7f51c6ac6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 0x55f9bf3fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.705+0000 7f51c6ac6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 0x55f9bf3fd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58094/0 (socket says 172.21.15.5:58094) 2024-09-19T06:05:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.705+0000 7f51c62c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf3fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.705+0000 7f51c6ac6640 1 -- 172.21.15.5:0/2351759732 learned_addr learned my addr 172.21.15.5:0/2351759732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51c5ac4640 1 --2- 172.21.15.5:0/2351759732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf408370 0x55f9bf40a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51c6ac6640 1 -- 172.21.15.5:0/2351759732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf408370 msgr2=0x55f9bf40a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/2351759732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf408370 0x55f9bf40a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51c6ac6640 1 -- 172.21.15.5:0/2351759732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3fd5d0 msgr2=0x55f9bf3fda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/2351759732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf3fda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51c6ac6640 1 -- 172.21.15.5:0/2351759732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bf2ef3a0 con 0x55f9bf3feb40 2024-09-19T06:05:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51b77fe640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f51ac0a33b0 0x7f51ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/2351759732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 0x55f9bf3fd090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009f70 tx=0x7f51b8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1db9ff58c5a4537 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51c62c5640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f51ac0a33b0 0x7f51ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51b4ff9640 1 -- 172.21.15.5:0/2351759732 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8016070 con 0x55f9bf3feb40 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51b77fe640 1 -- 172.21.15.5:0/1636695378 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f51bc0cf900 con 0x7f51c01009b0 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51b4ff9640 1 -- 172.21.15.5:0/2351759732 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51b8004d20 con 0x55f9bf3feb40 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.706+0000 7f51b4ff9640 1 -- 172.21.15.5:0/2351759732 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b80127d0 con 0x55f9bf3feb40 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.707+0000 7f51ca902740 1 -- 172.21.15.5:0/2351759732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 msgr2=0x55f9bf3fd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.707+0000 7f51ca902740 1 --2- 172.21.15.5:0/2351759732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 0x55f9bf3fd090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009f70 tx=0x7f51b8010800 comp rx=0 tx=0).stop 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.707+0000 7f51c62c5640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f51ac0a33b0 0x7f51ac0a5870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55f9bf3faac0 tx=0x7f51bc0d5530 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.707+0000 7f51ca902740 1 -- 172.21.15.5:0/2351759732 shutdown_connections 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.707+0000 7f51ca902740 1 --2- 172.21.15.5:0/2351759732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf408370 0x55f9bf40a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.707+0000 7f51ca902740 1 --2- 172.21.15.5:0/2351759732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf3fda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.707+0000 7f51ca902740 1 --2- 172.21.15.5:0/2351759732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 0x55f9bf3fd090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.707+0000 7f51ca902740 1 -- 172.21.15.5:0/2351759732 >> 172.21.15.5:0/2351759732 conn(0x55f9bf3c6310 msgr2=0x55f9bf3f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.707+0000 7f51ca902740 1 -- 172.21.15.5:0/2351759732 shutdown_connections 2024-09-19T06:05:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.707+0000 7f51ca902740 1 -- 172.21.15.5:0/2351759732 wait complete. 2024-09-19T06:05:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51ca902740 1 -- start start 2024-09-19T06:05:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51ca902740 1 Processor -- start 2024-09-19T06:05:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51ca902740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf407d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51ca902740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 0x55f9bf405040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51ca902740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf408370 0x55f9bf405650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51ca902740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9bf40b070 con 0x55f9bf408370 2024-09-19T06:05:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51ca902740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f9bf31e060 con 0x55f9bf3feb40 2024-09-19T06:05:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51ca902740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f9bf40b1f0 con 0x55f9bf3fd5d0 2024-09-19T06:05:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51ca902740 10 client.0 _ll_register_callbacks cb 0x55f9bf0718c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:37.781 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:05:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51c6ac6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf407d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51c5ac4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 0x55f9bf405040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51c6ac6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf407d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42448/0 (socket says 172.21.15.5:42448) 2024-09-19T06:05:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51c6ac6640 1 -- 172.21.15.5:0/1576097824 learned_addr learned my addr 172.21.15.5:0/1576097824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.708+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf408370 0x55f9bf405650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51c5ac4640 1 -- 172.21.15.5:0/1576097824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3fd5d0 msgr2=0x55f9bf407d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51c5ac4640 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf407d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51c5ac4640 1 -- 172.21.15.5:0/1576097824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf408370 msgr2=0x55f9bf405650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51c5ac4640 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf408370 0x55f9bf405650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51c5ac4640 1 -- 172.21.15.5:0/1576097824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bf0d43c0 con 0x55f9bf3feb40 2024-09-19T06:05:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf408370 0x55f9bf405650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51c5ac4640 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 0x55f9bf405040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009230 tx=0x7f51b0032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cbe7d3049226dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:05:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51a9ffb640 1 -- 172.21.15.5:0/1576097824 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0035040 con 0x55f9bf3feb40 2024-09-19T06:05:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51a9ffb640 1 -- 172.21.15.5:0/1576097824 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f51b0031860 con 0x55f9bf3feb40 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51ca902740 1 -- 172.21.15.5:0/1576097824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51b8004d20 con 0x55f9bf3feb40 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51a9ffb640 1 -- 172.21.15.5:0/1576097824 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f51b0004040 con 0x55f9bf3feb40 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51ca902740 10 client.25526 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.709+0000 7f51ca902740 1 -- 172.21.15.5:0/1576097824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f9bf416b40 con 0x55f9bf3feb40 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.710+0000 7f51a9ffb640 1 -- 172.21.15.5:0/1576097824 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51b8004d20 con 0x55f9bf3feb40 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.710+0000 7f51a9ffb640 1 -- 172.21.15.5:0/1576097824 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f51b00047e0 con 0x55f9bf3feb40 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.710+0000 7f51ca902740 10 client.25526 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.710+0000 7f51ca902740 10 client.25526 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.710+0000 7f51ca902740 20 client.25526 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.710+0000 7f51ca902740 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55f9bf417f50 0x55f9bf41a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.710+0000 7f51ca902740 4 client.25526 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.710+0000 7f51ca902740 1 -- 172.21.15.5:0/1576097824 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9bf370860 con 0x55f9bf417f50 2024-09-19T06:05:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.710+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55f9bf417f50 0x55f9bf41a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55f9bf417f50 0x55f9bf41a340 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f2f9ffac2a1bf23 server_cookie=e3fc9848a8f4da73 in_seq=0 out_seq=0 2024-09-19T06:05:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51a9ffb640 10 client.25526 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:05:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51a9ffb640 1 -- 172.21.15.5:0/1576097824 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9bf370860 con 0x55f9bf417f50 2024-09-19T06:05:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51a9ffb640 10 client.25526 handle_command_reply: tid=0 2024-09-19T06:05:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51ca902740 1 client.25526 shutdown 2024-09-19T06:05:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51ca902740 1 -- 172.21.15.5:0/1576097824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 msgr2=0x55f9bf405040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51ca902740 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 0x55f9bf405040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009230 tx=0x7f51b0032f90 comp rx=0 tx=0).stop 2024-09-19T06:05:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51ca902740 1 -- 172.21.15.5:0/1576097824 shutdown_connections 2024-09-19T06:05:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51ca902740 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55f9bf417f50 0x55f9bf41a340 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51ca902740 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf408370 0x55f9bf405650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51a9ffb640 0 client.25526 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:05:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51ca902740 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf3feb40 0x55f9bf405040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51ca902740 1 --2- 172.21.15.5:0/1576097824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf407d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.711+0000 7f51ca902740 1 -- 172.21.15.5:0/1576097824 >> 172.21.15.5:0/1576097824 conn(0x55f9bf3c6310 msgr2=0x55f9bf3ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.712+0000 7f51ca902740 1 -- 172.21.15.5:0/1576097824 shutdown_connections 2024-09-19T06:05:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.712+0000 7f51ca902740 1 -- 172.21.15.5:0/1576097824 wait complete. 2024-09-19T06:05:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.712+0000 7f51ca902740 20 client.25526 trim_cache size 0 max 16384 2024-09-19T06:05:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.726+0000 7f51ca902740 1 Processor -- start 2024-09-19T06:05:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.726+0000 7f51ca902740 1 -- start start 2024-09-19T06:05:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51ca902740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf414310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51ca902740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3feb40 0x55f9bf404020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51ca902740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf408370 0x55f9bf41fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51ca902740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01afae0 con 0x55f9bf3feb40 2024-09-19T06:05:37.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51ca902740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01b1680 con 0x55f9bf408370 2024-09-19T06:05:37.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51ca902740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01b01b0 con 0x55f9bf3fd5d0 2024-09-19T06:05:37.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51c5ac4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3feb40 0x55f9bf404020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51c5ac4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3feb40 0x55f9bf404020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56060/0 (socket says 172.21.15.5:56060) 2024-09-19T06:05:37.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51c5ac4640 1 -- 172.21.15.5:0/1772046641 learned_addr learned my addr 172.21.15.5:0/1772046641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/1772046641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf408370 0x55f9bf41fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/1772046641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf414310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51c5ac4640 1 -- 172.21.15.5:0/1772046641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3fd5d0 msgr2=0x55f9bf414310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51c5ac4640 1 --2- 172.21.15.5:0/1772046641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf414310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51c5ac4640 1 -- 172.21.15.5:0/1772046641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf408370 msgr2=0x55f9bf41fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51c5ac4640 1 --2- 172.21.15.5:0/1772046641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf408370 0x55f9bf41fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51c5ac4640 1 -- 172.21.15.5:0/1772046641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bf405fe0 con 0x55f9bf3feb40 2024-09-19T06:05:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51c5ac4640 1 --2- 172.21.15.5:0/1772046641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3feb40 0x55f9bf404020 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f51b00096c0 tx=0x7f51b0031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f321d3989733d8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51abfff640 1 -- 172.21.15.5:0/1772046641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0035070 con 0x55f9bf3feb40 2024-09-19T06:05:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51abfff640 1 -- 172.21.15.5:0/1772046641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51b00048c0 con 0x55f9bf3feb40 2024-09-19T06:05:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.727+0000 7f51abfff640 1 -- 172.21.15.5:0/1772046641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b00037b0 con 0x55f9bf3feb40 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.728+0000 7f51ca902740 1 -- 172.21.15.5:0/1772046641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3feb40 msgr2=0x55f9bf404020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.728+0000 7f51ca902740 1 --2- 172.21.15.5:0/1772046641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3feb40 0x55f9bf404020 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f51b00096c0 tx=0x7f51b0031890 comp rx=0 tx=0).stop 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.728+0000 7f51ca902740 1 -- 172.21.15.5:0/1772046641 shutdown_connections 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.728+0000 7f51ca902740 1 --2- 172.21.15.5:0/1772046641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf408370 0x55f9bf41fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.728+0000 7f51ca902740 1 --2- 172.21.15.5:0/1772046641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3feb40 0x55f9bf404020 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.728+0000 7f51ca902740 1 --2- 172.21.15.5:0/1772046641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf414310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.728+0000 7f51ca902740 1 -- 172.21.15.5:0/1772046641 >> 172.21.15.5:0/1772046641 conn(0x55f9bf3c6310 msgr2=0x55f9bf403910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.728+0000 7f51ca902740 1 -- 172.21.15.5:0/1772046641 shutdown_connections 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.728+0000 7f51ca902740 1 -- 172.21.15.5:0/1772046641 wait complete. 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51ca902740 1 -- start start 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51ca902740 1 Processor -- start 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51ca902740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf40c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51ca902740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3feb40 0x55f9bf40c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51ca902740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf408370 0x55f9bf40cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51ca902740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01b0f50 con 0x55f9bf3fd5d0 2024-09-19T06:05:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51ca902740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01b0880 con 0x55f9bf408370 2024-09-19T06:05:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51ca902740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01adac0 con 0x55f9bf3feb40 2024-09-19T06:05:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51ca902740 10 client.0 _ll_register_callbacks cb 0x7f51b0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51c6ac6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf40c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51c6ac6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf40c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56070/0 (socket says 172.21.15.5:56070) 2024-09-19T06:05:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51c6ac6640 1 -- 172.21.15.5:0/3401183408 learned_addr learned my addr 172.21.15.5:0/3401183408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51c5ac4640 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3feb40 0x55f9bf40c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf408370 0x55f9bf40cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51c6ac6640 1 -- 172.21.15.5:0/3401183408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3feb40 msgr2=0x55f9bf40c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3feb40 0x55f9bf40c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51c6ac6640 1 -- 172.21.15.5:0/3401183408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf408370 msgr2=0x55f9bf40cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.729+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf408370 0x55f9bf40cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.730+0000 7f51c6ac6640 1 -- 172.21.15.5:0/3401183408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bf3f9510 con 0x55f9bf3fd5d0 2024-09-19T06:05:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.730+0000 7f51c6ac6640 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf40c140 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f51b8010d00 tx=0x7f51b8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10515bda5985af0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.730+0000 7f51a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:05:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.730+0000 7f51a9ffb640 1 -- 172.21.15.5:0/3401183408 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8016040 con 0x55f9bf3fd5d0 2024-09-19T06:05:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.730+0000 7f51a9ffb640 1 -- 172.21.15.5:0/3401183408 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f51b8003b70 con 0x55f9bf3fd5d0 2024-09-19T06:05:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.730+0000 7f51ca902740 1 -- 172.21.15.5:0/3401183408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51b00048c0 con 0x55f9bf3fd5d0 2024-09-19T06:05:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.730+0000 7f51a9ffb640 1 -- 172.21.15.5:0/3401183408 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f51b8023040 con 0x55f9bf3fd5d0 2024-09-19T06:05:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.730+0000 7f51ca902740 10 client.15930 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.730+0000 7f51ca902740 1 -- 172.21.15.5:0/3401183408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f9bf482c50 con 0x55f9bf3fd5d0 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.730+0000 7f51a9ffb640 1 -- 172.21.15.5:0/3401183408 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51b00048c0 con 0x55f9bf3fd5d0 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.730+0000 7f51a9ffb640 1 -- 172.21.15.5:0/3401183408 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f51b800f040 con 0x55f9bf3fd5d0 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51ca902740 10 client.15930 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51ca902740 10 client.15930 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51ca902740 20 client.15930 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51ca902740 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55f9bf483230 0x55f9bf485620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51ca902740 4 client.15930 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51ca902740 1 -- 172.21.15.5:0/3401183408 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9bf36ad70 con 0x55f9bf483230 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51c5ac4640 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55f9bf483230 0x55f9bf485620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51a9ffb640 10 client.15930 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51c5ac4640 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55f9bf483230 0x55f9bf485620 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d870dfca534ef43 server_cookie=c3781b52f6b6b7fc in_seq=0 out_seq=0 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51a9ffb640 1 -- 172.21.15.5:0/3401183408 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9bf36ad70 con 0x55f9bf483230 2024-09-19T06:05:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51a9ffb640 10 client.15930 handle_command_reply: tid=0 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51ca902740 1 client.15930 shutdown 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51ca902740 1 -- 172.21.15.5:0/3401183408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3fd5d0 msgr2=0x55f9bf40c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51ca902740 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf40c140 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f51b8010d00 tx=0x7f51b8010800 comp rx=0 tx=0).stop 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.731+0000 7f51ca902740 1 -- 172.21.15.5:0/3401183408 shutdown_connections 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51ca902740 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55f9bf483230 0x55f9bf485620 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51ca902740 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9bf408370 0x55f9bf40cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51a9ffb640 0 client.15930 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51ca902740 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9bf3feb40 0x55f9bf40c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51ca902740 1 --2- 172.21.15.5:0/3401183408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9bf3fd5d0 0x55f9bf40c140 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51ca902740 1 -- 172.21.15.5:0/3401183408 >> 172.21.15.5:0/3401183408 conn(0x55f9bf3c6310 msgr2=0x55f9bf3f84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51ca902740 1 -- 172.21.15.5:0/3401183408 shutdown_connections 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51ca902740 1 -- 172.21.15.5:0/3401183408 wait complete. 2024-09-19T06:05:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51ca902740 20 client.15930 trim_cache size 0 max 16384 2024-09-19T06:05:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51c8529640 1 -- 172.21.15.5:0/1636695378 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f51ac0a33b0 msgr2=0x7f51ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51c8529640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f51ac0a33b0 0x7f51ac0a5870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55f9bf3faac0 tx=0x7f51bc0d5530 comp rx=0 tx=0).stop 2024-09-19T06:05:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51c8529640 1 -- 172.21.15.5:0/1636695378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01009b0 msgr2=0x7f51c019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51c8529640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01009b0 0x7f51c019a780 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00d950 tx=0x7f51bc00de10 comp rx=0 tx=0).stop 2024-09-19T06:05:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51c8529640 1 -- 172.21.15.5:0/1636695378 shutdown_connections 2024-09-19T06:05:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51c8529640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f51ac0a33b0 0x7f51ac0a5870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51c8529640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c010e4d0 0x7f51c019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51c8529640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c0101360 0x7f51c019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51c8529640 1 --2- 172.21.15.5:0/1636695378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01009b0 0x7f51c019a780 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.732+0000 7f51c8529640 1 -- 172.21.15.5:0/1636695378 >> 172.21.15.5:0/1636695378 conn(0x7f51c00fc7d0 msgr2=0x7f51c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.733+0000 7f51c8529640 1 -- 172.21.15.5:0/1636695378 shutdown_connections 2024-09-19T06:05:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:37.733+0000 7f51c8529640 1 -- 172.21.15.5:0/1636695378 wait complete. 2024-09-19T06:05:37.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 33604 3.15 MB/sec warmup 63 sec latency 13.965 ms 2024-09-19T06:05:38.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:38 smithi005 ceph-mon[30955]: pgmap v71: 97 pgs: 97 active+clean; 38 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-09-19T06:05:38.643 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:05:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:38 smithi157 ceph-mon[35346]: pgmap v71: 97 pgs: 97 active+clean; 38 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-09-19T06:05:38.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 34306 3.12 MB/sec warmup 64 sec latency 15.370 ms 2024-09-19T06:05:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:38 smithi049 ceph-mon[34640]: pgmap v71: 97 pgs: 97 active+clean; 38 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-09-19T06:05:38.972 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:05:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.494+0000 7f562726c640 1 -- 172.21.15.5:0/3011822561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620102d40 msgr2=0x7f5620103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.494+0000 7f562726c640 1 --2- 172.21.15.5:0/3011822561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620102d40 0x7f5620103120 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f561c009900 tx=0x7f561c02a480 comp rx=0 tx=0).stop 2024-09-19T06:05:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.494+0000 7f562726c640 1 -- 172.21.15.5:0/3011822561 shutdown_connections 2024-09-19T06:05:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.494+0000 7f562726c640 1 --2- 172.21.15.5:0/3011822561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f562010c170 0x7f562010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.494+0000 7f562726c640 1 --2- 172.21.15.5:0/3011822561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5620103660 0x7f562010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.494+0000 7f562726c640 1 --2- 172.21.15.5:0/3011822561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620102d40 0x7f5620103120 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.494+0000 7f562726c640 1 -- 172.21.15.5:0/3011822561 >> 172.21.15.5:0/3011822561 conn(0x7f56200fc7a0 msgr2=0x7f56200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.494+0000 7f562726c640 1 -- 172.21.15.5:0/3011822561 shutdown_connections 2024-09-19T06:05:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.494+0000 7f562726c640 1 -- 172.21.15.5:0/3011822561 wait complete. 2024-09-19T06:05:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f562726c640 1 Processor -- start 2024-09-19T06:05:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f562726c640 1 -- start start 2024-09-19T06:05:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f562726c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5620102d40 0x7f56201009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f562726c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5620103660 0x7f5620100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f562726c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562010c170 0x7f5620101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f562726c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5620111e20 con 0x7f562010c170 2024-09-19T06:05:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f562726c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5620111ca0 con 0x7f5620102d40 2024-09-19T06:05:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f562726c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5620111fa0 con 0x7f5620103660 2024-09-19T06:05:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f5625809640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562010c170 0x7f5620101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f5625008640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5620102d40 0x7f56201009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f5625809640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562010c170 0x7f5620101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56096/0 (socket says 172.21.15.5:56096) 2024-09-19T06:05:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f5625809640 1 -- 172.21.15.5:0/2287162820 learned_addr learned my addr 172.21.15.5:0/2287162820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.495+0000 7f5624807640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5620103660 0x7f5620100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.496+0000 7f5625809640 1 -- 172.21.15.5:0/2287162820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5620103660 msgr2=0x7f5620100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.496+0000 7f5625809640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5620103660 0x7f5620100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.496+0000 7f5625809640 1 -- 172.21.15.5:0/2287162820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5620102d40 msgr2=0x7f56201009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.496+0000 7f5625809640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5620102d40 0x7f56201009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.496+0000 7f5625809640 1 -- 172.21.15.5:0/2287162820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56201a6800 con 0x7f562010c170 2024-09-19T06:05:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.496+0000 7f5625809640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562010c170 0x7f5620101420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f561800eb10 tx=0x7f561800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.496+0000 7f56167fc640 1 -- 172.21.15.5:0/2287162820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5618004b20 con 0x7f562010c170 2024-09-19T06:05:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.496+0000 7f56167fc640 1 -- 172.21.15.5:0/2287162820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5618004cc0 con 0x7f562010c170 2024-09-19T06:05:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.496+0000 7f562726c640 1 -- 172.21.15.5:0/2287162820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56201a6af0 con 0x7f562010c170 2024-09-19T06:05:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.496+0000 7f56167fc640 1 -- 172.21.15.5:0/2287162820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5618010690 con 0x7f562010c170 2024-09-19T06:05:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.496+0000 7f562726c640 1 -- 172.21.15.5:0/2287162820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56201a7030 con 0x7f562010c170 2024-09-19T06:05:39.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.500+0000 7f5629645740 1 Processor -- start 2024-09-19T06:05:39.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.500+0000 7f56167fc640 1 -- 172.21.15.5:0/2287162820 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5618005020 con 0x7f562010c170 2024-09-19T06:05:39.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.500+0000 7f5629645740 1 -- start start 2024-09-19T06:05:39.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.500+0000 7f5629645740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912235090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.500+0000 7f5629645740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x559912235a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.500+0000 7f5629645740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x559912242760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.500+0000 7f5629645740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55991219b2f0 con 0x559912236b40 2024-09-19T06:05:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.500+0000 7f5629645740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559912142cf0 con 0x559912240370 2024-09-19T06:05:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.500+0000 7f5629645740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5599121972b0 con 0x5599122355d0 2024-09-19T06:05:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f5625008640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x559912235a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f5624807640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912235090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f5625008640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x559912235a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42480/0 (socket says 172.21.15.5:42480) 2024-09-19T06:05:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f5625008640 1 -- 172.21.15.5:0/3275111080 learned_addr learned my addr 172.21.15.5:0/3275111080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f5624807640 1 --2- 172.21.15.5:0/3275111080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x559912242760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f56167fc640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f560c0a33b0 0x7f560c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f5625809640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f560c0a33b0 0x7f560c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f5624807640 1 -- 172.21.15.5:0/3275111080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 msgr2=0x559912235a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f5624807640 1 --2- 172.21.15.5:0/3275111080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x559912235a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f5624807640 1 -- 172.21.15.5:0/3275111080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 msgr2=0x559912242760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f5624807640 1 --2- 172.21.15.5:0/3275111080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x559912242760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f5624807640 1 -- 172.21.15.5:0/3275111080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599121273a0 con 0x559912236b40 2024-09-19T06:05:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.501+0000 7f56167fc640 1 -- 172.21.15.5:0/2287162820 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f56180d0360 con 0x7f562010c170 2024-09-19T06:05:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.502+0000 7f5624807640 1 --2- 172.21.15.5:0/3275111080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912235090 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5610005d00 tx=0x7f5610005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1693603770d68f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.502+0000 7f5625809640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f560c0a33b0 0x7f560c0a5870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f56180ce0c0 tx=0x7f561800efd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.502+0000 7f560b7fe640 1 -- 172.21.15.5:0/3275111080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610004030 con 0x559912236b40 2024-09-19T06:05:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.502+0000 7f560b7fe640 1 -- 172.21.15.5:0/3275111080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56100035f0 con 0x559912236b40 2024-09-19T06:05:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.502+0000 7f560b7fe640 1 -- 172.21.15.5:0/3275111080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610010670 con 0x559912236b40 2024-09-19T06:05:39.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.503+0000 7f5629645740 1 -- 172.21.15.5:0/3275111080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 msgr2=0x559912235090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:39.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.503+0000 7f5629645740 1 --2- 172.21.15.5:0/3275111080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912235090 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5610005d00 tx=0x7f5610005c90 comp rx=0 tx=0).stop 2024-09-19T06:05:39.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.503+0000 7f5629645740 1 -- 172.21.15.5:0/3275111080 shutdown_connections 2024-09-19T06:05:39.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.503+0000 7f5629645740 1 --2- 172.21.15.5:0/3275111080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x559912242760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.503+0000 7f5629645740 1 --2- 172.21.15.5:0/3275111080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x559912235a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.503+0000 7f5629645740 1 --2- 172.21.15.5:0/3275111080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912235090 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.503+0000 7f5629645740 1 -- 172.21.15.5:0/3275111080 >> 172.21.15.5:0/3275111080 conn(0x5599121fe310 msgr2=0x559912231c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:39.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.503+0000 7f5629645740 1 -- 172.21.15.5:0/3275111080 shutdown_connections 2024-09-19T06:05:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.503+0000 7f5629645740 1 -- 172.21.15.5:0/3275111080 wait complete. 2024-09-19T06:05:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.503+0000 7f5629645740 1 -- start start 2024-09-19T06:05:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.503+0000 7f5629645740 1 Processor -- start 2024-09-19T06:05:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5629645740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x55991223fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5629645740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x55991223d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5629645740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x55991223d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5629645740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559912243070 con 0x559912236b40 2024-09-19T06:05:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5629645740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559912156060 con 0x559912240370 2024-09-19T06:05:39.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5629645740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5599122431f0 con 0x5599122355d0 2024-09-19T06:05:39.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5629645740 10 client.0 _ll_register_callbacks cb 0x559911ea98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:39.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5625008640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x55991223d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5625008640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x55991223d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56116/0 (socket says 172.21.15.5:56116) 2024-09-19T06:05:39.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5625008640 1 -- 172.21.15.5:0/3208004306 learned_addr learned my addr 172.21.15.5:0/3208004306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:39.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5624807640 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x55991223d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5624807640 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x55991223fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5625008640 1 -- 172.21.15.5:0/3208004306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 msgr2=0x55991223fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:39.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5625008640 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x55991223fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5625008640 1 -- 172.21.15.5:0/3208004306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 msgr2=0x55991223d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:39.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5625008640 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x55991223d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.504+0000 7f5625008640 1 -- 172.21.15.5:0/3208004306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559911f0c3c0 con 0x559912236b40 2024-09-19T06:05:39.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.505+0000 7f5625008640 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x55991223d080 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f561c032f40 tx=0x7f561c032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca520b107764044a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:39.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.505+0000 7f5608ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:05:39.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.505+0000 7f5608ff9640 1 -- 172.21.15.5:0/3208004306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c035040 con 0x559912236b40 2024-09-19T06:05:39.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.505+0000 7f5608ff9640 1 -- 172.21.15.5:0/3208004306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f561c031860 con 0x559912236b40 2024-09-19T06:05:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.505+0000 7f5629645740 1 -- 172.21.15.5:0/3208004306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56100035f0 con 0x559912236b40 2024-09-19T06:05:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.505+0000 7f5608ff9640 1 -- 172.21.15.5:0/3208004306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f561c004040 con 0x559912236b40 2024-09-19T06:05:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.505+0000 7f5629645740 10 client.15942 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.505+0000 7f5629645740 1 -- 172.21.15.5:0/3208004306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55991224ebc0 con 0x559912236b40 2024-09-19T06:05:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.506+0000 7f5608ff9640 1 -- 172.21.15.5:0/3208004306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56100035f0 con 0x559912236b40 2024-09-19T06:05:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.506+0000 7f5608ff9640 1 -- 172.21.15.5:0/3208004306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f561c004850 con 0x559912236b40 2024-09-19T06:05:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.506+0000 7f5629645740 10 client.15942 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.506+0000 7f5629645740 10 client.15942 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:05:39.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.506+0000 7f5629645740 20 client.15942 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:39.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.506+0000 7f5629645740 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x559912250080 0x5599122524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.506+0000 7f5629645740 4 client.15942 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:39.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.506+0000 7f5629645740 1 -- 172.21.15.5:0/3208004306 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599121a8860 con 0x559912250080 2024-09-19T06:05:39.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.506+0000 7f5624807640 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x559912250080 0x5599122524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5624807640 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x559912250080 0x5599122524a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9544e44d66f0c4d server_cookie=1d387f8adb995ece in_seq=0 out_seq=0 2024-09-19T06:05:39.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5608ff9640 10 client.15942 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:05:39.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5608ff9640 1 -- 172.21.15.5:0/3208004306 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599121a8860 con 0x559912250080 2024-09-19T06:05:39.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5608ff9640 10 client.15942 handle_command_reply: tid=0 2024-09-19T06:05:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5629645740 1 client.15942 shutdown 2024-09-19T06:05:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5629645740 1 -- 172.21.15.5:0/3208004306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 msgr2=0x55991223d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5629645740 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x55991223d080 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f561c032f40 tx=0x7f561c032f70 comp rx=0 tx=0).stop 2024-09-19T06:05:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5629645740 1 -- 172.21.15.5:0/3208004306 shutdown_connections 2024-09-19T06:05:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5629645740 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x559912250080 0x5599122524a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5629645740 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x55991223d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5629645740 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x55991223d080 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5629645740 1 --2- 172.21.15.5:0/3208004306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x55991223fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5629645740 1 -- 172.21.15.5:0/3208004306 >> 172.21.15.5:0/3208004306 conn(0x5599121fe310 msgr2=0x559912237930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.507+0000 7f5608ff9640 0 client.15942 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:05:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.508+0000 7f5629645740 1 -- 172.21.15.5:0/3208004306 shutdown_connections 2024-09-19T06:05:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.508+0000 7f5629645740 1 -- 172.21.15.5:0/3208004306 wait complete. 2024-09-19T06:05:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.508+0000 7f5629645740 20 client.15942 trim_cache size 0 max 16384 2024-09-19T06:05:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5629645740 1 Processor -- start 2024-09-19T06:05:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5629645740 1 -- start start 2024-09-19T06:05:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5629645740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x55991223d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5629645740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912257d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5629645740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x5599122582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5629645740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56201afa40 con 0x559912236b40 2024-09-19T06:05:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5629645740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56201b15e0 con 0x559912240370 2024-09-19T06:05:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5629645740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56201b0110 con 0x5599122355d0 2024-09-19T06:05:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5625008640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912257d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5625008640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912257d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56118/0 (socket says 172.21.15.5:56118) 2024-09-19T06:05:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5625008640 1 -- 172.21.15.5:0/1989698191 learned_addr learned my addr 172.21.15.5:0/1989698191 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5624807640 1 --2- 172.21.15.5:0/1989698191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x55991223d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.523+0000 7f5624807640 1 --2- 172.21.15.5:0/1989698191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x5599122582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.524+0000 7f5625008640 1 -- 172.21.15.5:0/1989698191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 msgr2=0x55991223d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.524+0000 7f5625008640 1 --2- 172.21.15.5:0/1989698191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x55991223d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.524+0000 7f5625008640 1 -- 172.21.15.5:0/1989698191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 msgr2=0x5599122582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.524+0000 7f5625008640 1 --2- 172.21.15.5:0/1989698191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x5599122582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.524+0000 7f5625008640 1 -- 172.21.15.5:0/1989698191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559912238120 con 0x559912236b40 2024-09-19T06:05:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.524+0000 7f5624807640 1 --2- 172.21.15.5:0/1989698191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x55991223d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.524+0000 7f5625008640 1 --2- 172.21.15.5:0/1989698191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912257d70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f561c004de0 tx=0x7f561c0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b656bfdc1943d1d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.524+0000 7f560affd640 1 -- 172.21.15.5:0/1989698191 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c035030 con 0x559912236b40 2024-09-19T06:05:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.524+0000 7f560affd640 1 -- 172.21.15.5:0/1989698191 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f561c005850 con 0x559912236b40 2024-09-19T06:05:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.524+0000 7f560affd640 1 -- 172.21.15.5:0/1989698191 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c053070 con 0x559912236b40 2024-09-19T06:05:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.525+0000 7f5629645740 1 -- 172.21.15.5:0/1989698191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 msgr2=0x559912257d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.525+0000 7f5629645740 1 --2- 172.21.15.5:0/1989698191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912257d70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f561c004de0 tx=0x7f561c0314a0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.525+0000 7f5629645740 1 -- 172.21.15.5:0/1989698191 shutdown_connections 2024-09-19T06:05:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.525+0000 7f5629645740 1 --2- 172.21.15.5:0/1989698191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x5599122582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.525+0000 7f5629645740 1 --2- 172.21.15.5:0/1989698191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912257d70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.525+0000 7f5629645740 1 --2- 172.21.15.5:0/1989698191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x55991223d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.525+0000 7f5629645740 1 -- 172.21.15.5:0/1989698191 >> 172.21.15.5:0/1989698191 conn(0x5599121fe310 msgr2=0x55991223c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:39.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.525+0000 7f5629645740 1 -- 172.21.15.5:0/1989698191 shutdown_connections 2024-09-19T06:05:39.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.525+0000 7f5629645740 1 -- 172.21.15.5:0/1989698191 wait complete. 2024-09-19T06:05:39.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.525+0000 7f5629645740 1 -- start start 2024-09-19T06:05:39.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.525+0000 7f5629645740 1 Processor -- start 2024-09-19T06:05:39.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.526+0000 7f5629645740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x559912248690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.526+0000 7f5629645740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912248bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.526+0000 7f5629645740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x559912239810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.526+0000 7f5629645740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56201b0eb0 con 0x559912236b40 2024-09-19T06:05:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.526+0000 7f5629645740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56201b07e0 con 0x559912240370 2024-09-19T06:05:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.526+0000 7f5625008640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912248bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.526+0000 7f5625008640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912248bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56122/0 (socket says 172.21.15.5:56122) 2024-09-19T06:05:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.526+0000 7f5625008640 1 -- 172.21.15.5:0/365542773 learned_addr learned my addr 172.21.15.5:0/365542773 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.527+0000 7f5624807640 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x559912248690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.527+0000 7f5629645740 1 -- 172.21.15.5:0/365542773 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56201ada20 con 0x5599122355d0 2024-09-19T06:05:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.527+0000 7f5629645740 10 client.0 _ll_register_callbacks cb 0x7f561c005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.527+0000 7f5624807640 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x559912239810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.527+0000 7f5625008640 1 -- 172.21.15.5:0/365542773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 msgr2=0x559912248690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.527+0000 7f5625008640 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x559912248690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.527+0000 7f5625008640 1 -- 172.21.15.5:0/365542773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 msgr2=0x559912239810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.527+0000 7f5625008640 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x559912239810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.527+0000 7f5625008640 1 -- 172.21.15.5:0/365542773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559912239e80 con 0x559912236b40 2024-09-19T06:05:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.527+0000 7f5624807640 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x559912239810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:05:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.528+0000 7f5625008640 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912248bd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f561c009900 tx=0x7f561c032fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29058208ad229d55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.528+0000 7f5608ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:05:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.528+0000 7f5608ff9640 1 -- 172.21.15.5:0/365542773 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c053070 con 0x559912236b40 2024-09-19T06:05:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.528+0000 7f5608ff9640 1 -- 172.21.15.5:0/365542773 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f561c0046c0 con 0x559912236b40 2024-09-19T06:05:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.528+0000 7f5629645740 1 -- 172.21.15.5:0/365542773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f561c005850 con 0x559912236b40 2024-09-19T06:05:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.528+0000 7f5608ff9640 1 -- 172.21.15.5:0/365542773 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f561c068050 con 0x559912236b40 2024-09-19T06:05:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.528+0000 7f5629645740 10 client.15948 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.528+0000 7f5629645740 1 -- 172.21.15.5:0/365542773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5599122ba680 con 0x559912236b40 2024-09-19T06:05:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.528+0000 7f5608ff9640 1 -- 172.21.15.5:0/365542773 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f561c005850 con 0x559912236b40 2024-09-19T06:05:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.529+0000 7f5608ff9640 1 -- 172.21.15.5:0/365542773 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f561c031660 con 0x559912236b40 2024-09-19T06:05:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.529+0000 7f5629645740 10 client.15948 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.529+0000 7f5629645740 10 client.15948 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:05:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.529+0000 7f5629645740 20 client.15948 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.529+0000 7f5629645740 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5599122bac60 0x5599122bd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.529+0000 7f5629645740 4 client.15948 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:05:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.529+0000 7f5629645740 1 -- 172.21.15.5:0/365542773 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599121a2d70 con 0x5599122bac60 2024-09-19T06:05:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.529+0000 7f5624807640 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5599122bac60 0x5599122bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5624807640 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5599122bac60 0x5599122bd080 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4f5dd8ce7713af33 server_cookie=cd62a51d59053631 in_seq=0 out_seq=0 2024-09-19T06:05:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5608ff9640 10 client.15948 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:05:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5608ff9640 1 -- 172.21.15.5:0/365542773 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599121a2d70 con 0x5599122bac60 2024-09-19T06:05:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5608ff9640 10 client.15948 handle_command_reply: tid=0 2024-09-19T06:05:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5629645740 1 client.15948 shutdown 2024-09-19T06:05:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5629645740 1 -- 172.21.15.5:0/365542773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 msgr2=0x559912248bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5629645740 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912248bd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f561c009900 tx=0x7f561c032fa0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5629645740 1 -- 172.21.15.5:0/365542773 shutdown_connections 2024-09-19T06:05:39.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5629645740 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5599122bac60 0x5599122bd080 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5629645740 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559912240370 0x559912239810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5629645740 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559912236b40 0x559912248bd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5629645740 1 --2- 172.21.15.5:0/365542773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599122355d0 0x559912248690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5608ff9640 0 client.15948 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:05:39.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5629645740 1 -- 172.21.15.5:0/365542773 >> 172.21.15.5:0/365542773 conn(0x5599121fe310 msgr2=0x55991223f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:39.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5629645740 1 -- 172.21.15.5:0/365542773 shutdown_connections 2024-09-19T06:05:39.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5629645740 1 -- 172.21.15.5:0/365542773 wait complete. 2024-09-19T06:05:39.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.530+0000 7f5629645740 20 client.15948 trim_cache size 0 max 16384 2024-09-19T06:05:39.705 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:05:39.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.531+0000 7f562726c640 1 -- 172.21.15.5:0/2287162820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f560c0a33b0 msgr2=0x7f560c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:39.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.531+0000 7f562726c640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f560c0a33b0 0x7f560c0a5870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f56180ce0c0 tx=0x7f561800efd0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.531+0000 7f562726c640 1 -- 172.21.15.5:0/2287162820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562010c170 msgr2=0x7f5620101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:39.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.531+0000 7f562726c640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562010c170 0x7f5620101420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f561800eb10 tx=0x7f561800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.531+0000 7f562726c640 1 -- 172.21.15.5:0/2287162820 shutdown_connections 2024-09-19T06:05:39.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.531+0000 7f562726c640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562010c170 0x7f5620101420 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.531+0000 7f562726c640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f560c0a33b0 0x7f560c0a5870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.531+0000 7f562726c640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5620103660 0x7f5620100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.531+0000 7f562726c640 1 --2- 172.21.15.5:0/2287162820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5620102d40 0x7f56201009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.531+0000 7f562726c640 1 -- 172.21.15.5:0/2287162820 >> 172.21.15.5:0/2287162820 conn(0x7f56200fc7a0 msgr2=0x7f56200fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.531+0000 7f562726c640 1 -- 172.21.15.5:0/2287162820 shutdown_connections 2024-09-19T06:05:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:39.531+0000 7f562726c640 1 -- 172.21.15.5:0/2287162820 wait complete. 2024-09-19T06:05:39.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 34865 3.15 MB/sec warmup 65 sec latency 53.496 ms 2024-09-19T06:05:40.279 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:05:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:05:40.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:40 smithi005 ceph-mon[30955]: pgmap v72: 97 pgs: 97 active+clean; 38 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-19T06:05:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:40 smithi157 ceph-mon[35346]: pgmap v72: 97 pgs: 97 active+clean; 38 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-19T06:05:40.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 35333 3.15 MB/sec warmup 66 sec latency 15.287 ms 2024-09-19T06:05:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:40 smithi049 ceph-mon[34640]: pgmap v72: 97 pgs: 97 active+clean; 38 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-19T06:05:41.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 35807 3.16 MB/sec warmup 67 sec latency 15.926 ms 2024-09-19T06:05:42.770 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:42 smithi005 ceph-mon[30955]: pgmap v73: 97 pgs: 97 active+clean; 38 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-19T06:05:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:42 smithi157 ceph-mon[35346]: pgmap v73: 97 pgs: 97 active+clean; 38 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-19T06:05:42.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 36065 3.12 MB/sec warmup 68 sec latency 85.494 ms 2024-09-19T06:05:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:42 smithi049 ceph-mon[34640]: pgmap v73: 97 pgs: 97 active+clean; 38 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-19T06:05:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.490+0000 7f22513b7640 1 -- 172.21.15.5:0/1369333579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c101340 msgr2=0x7f224c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.490+0000 7f22513b7640 1 --2- 172.21.15.5:0/1369333579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c101340 0x7f224c10d870 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2244009cd0 tx=0x7f224402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.490+0000 7f22513b7640 1 -- 172.21.15.5:0/1369333579 shutdown_connections 2024-09-19T06:05:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.490+0000 7f22513b7640 1 --2- 172.21.15.5:0/1369333579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c101340 0x7f224c10d870 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.490+0000 7f22513b7640 1 --2- 172.21.15.5:0/1369333579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f224c1009a0 0x7f224c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.490+0000 7f22513b7640 1 --2- 172.21.15.5:0/1369333579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f224c1069b0 0x7f224c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.490+0000 7f22513b7640 1 -- 172.21.15.5:0/1369333579 >> 172.21.15.5:0/1369333579 conn(0x7f224c0779a0 msgr2=0x7f224c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.490+0000 7f22513b7640 1 -- 172.21.15.5:0/1369333579 shutdown_connections 2024-09-19T06:05:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.490+0000 7f22513b7640 1 -- 172.21.15.5:0/1369333579 wait complete. 2024-09-19T06:05:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f22513b7640 1 Processor -- start 2024-09-19T06:05:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f22513b7640 1 -- start start 2024-09-19T06:05:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f22513b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c1009a0 0x7f224c0fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f22513b7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f224c101340 0x7f224c0fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f22513b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f224c1069b0 0x7f224c0fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f22513b7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f224c110ca0 con 0x7f224c1009a0 2024-09-19T06:05:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f22513b7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f224c110b20 con 0x7f224c101340 2024-09-19T06:05:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f22513b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f224c110e20 con 0x7f224c1069b0 2024-09-19T06:05:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f224affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c1009a0 0x7f224c0fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f224affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c1009a0 0x7f224c0fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56146/0 (socket says 172.21.15.5:56146) 2024-09-19T06:05:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f224affd640 1 -- 172.21.15.5:0/1233296592 learned_addr learned my addr 172.21.15.5:0/1233296592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f2243fff640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f224c101340 0x7f224c0fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f224b7fe640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f224c1069b0 0x7f224c0fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f224affd640 1 -- 172.21.15.5:0/1233296592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f224c1069b0 msgr2=0x7f224c0fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f224affd640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f224c1069b0 0x7f224c0fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f224affd640 1 -- 172.21.15.5:0/1233296592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f224c101340 msgr2=0x7f224c0fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f224affd640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f224c101340 0x7f224c0fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f224affd640 1 -- 172.21.15.5:0/1233296592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c1a67e0 con 0x7f224c1009a0 2024-09-19T06:05:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.491+0000 7f224affd640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c1009a0 0x7f224c0fcde0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f223c00e9f0 tx=0x7f223c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.492+0000 7f2248ff9640 1 -- 172.21.15.5:0/1233296592 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c00cd80 con 0x7f224c1009a0 2024-09-19T06:05:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.492+0000 7f2248ff9640 1 -- 172.21.15.5:0/1233296592 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f223c0108e0 con 0x7f224c1009a0 2024-09-19T06:05:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.492+0000 7f2248ff9640 1 -- 172.21.15.5:0/1233296592 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c018920 con 0x7f224c1009a0 2024-09-19T06:05:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.492+0000 7f22513b7640 1 -- 172.21.15.5:0/1233296592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c1a6ad0 con 0x7f224c1009a0 2024-09-19T06:05:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.492+0000 7f22513b7640 1 -- 172.21.15.5:0/1233296592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c1a7020 con 0x7f224c1009a0 2024-09-19T06:05:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.494+0000 7f2248ff9640 1 -- 172.21.15.5:0/1233296592 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f223c010a90 con 0x7f224c1009a0 2024-09-19T06:05:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.494+0000 7f225422c740 1 Processor -- start 2024-09-19T06:05:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f225422c740 1 -- start start 2024-09-19T06:05:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f225422c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eaded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f225422c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19eb8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f225422c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 0x559b19ebb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f225422c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559b19e122f0 con 0x559b19eb8cb0 2024-09-19T06:05:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f225422c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559b19dba270 con 0x559b19eae410 2024-09-19T06:05:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f225422c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559b19e0e2b0 con 0x559b19eadaf0 2024-09-19T06:05:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f224affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19eb8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f2243fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 0x559b19ebb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f224affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19eb8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58190/0 (socket says 172.21.15.5:58190) 2024-09-19T06:05:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f224affd640 1 -- 172.21.15.5:0/675348786 learned_addr learned my addr 172.21.15.5:0/675348786 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f2243fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 0x559b19ebb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56156/0 (socket says 172.21.15.5:56156) 2024-09-19T06:05:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f2248ff9640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f22380a33b0 0x7f22380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.495+0000 7f224b7fe640 1 --2- 172.21.15.5:0/675348786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eaded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.496+0000 7f224b7fe640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f22380a33b0 0x7f22380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.496+0000 7f2243fff640 1 -- 172.21.15.5:0/675348786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 msgr2=0x559b19eaded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.496+0000 7f2243fff640 1 --2- 172.21.15.5:0/675348786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eaded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.496+0000 7f2243fff640 1 -- 172.21.15.5:0/675348786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 msgr2=0x559b19eb8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.496+0000 7f2243fff640 1 --2- 172.21.15.5:0/675348786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19eb8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.496+0000 7f2243fff640 1 -- 172.21.15.5:0/675348786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b19db9340 con 0x559b19eb8cb0 2024-09-19T06:05:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.496+0000 7f2248ff9640 1 -- 172.21.15.5:0/1233296592 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f223c0cffa0 con 0x7f224c1009a0 2024-09-19T06:05:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.496+0000 7f2243fff640 1 --2- 172.21.15.5:0/675348786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 0x559b19ebb0a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2234009bf0 tx=0x7f2234011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc49d5eb7608e6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.496+0000 7f224b7fe640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f22380a33b0 0x7f22380a5870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2244004af0 tx=0x7f224402a8e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.496+0000 7f22417fa640 1 -- 172.21.15.5:0/675348786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234004050 con 0x559b19eb8cb0 2024-09-19T06:05:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.496+0000 7f22417fa640 1 -- 172.21.15.5:0/675348786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22340054d0 con 0x559b19eb8cb0 2024-09-19T06:05:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.496+0000 7f22417fa640 1 -- 172.21.15.5:0/675348786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234004d40 con 0x559b19eb8cb0 2024-09-19T06:05:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.497+0000 7f225422c740 1 -- 172.21.15.5:0/675348786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 msgr2=0x559b19ebb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.497+0000 7f225422c740 1 --2- 172.21.15.5:0/675348786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 0x559b19ebb0a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2234009bf0 tx=0x7f2234011810 comp rx=0 tx=0).stop 2024-09-19T06:05:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.497+0000 7f225422c740 1 -- 172.21.15.5:0/675348786 shutdown_connections 2024-09-19T06:05:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.497+0000 7f225422c740 1 --2- 172.21.15.5:0/675348786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 0x559b19ebb0a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.497+0000 7f225422c740 1 --2- 172.21.15.5:0/675348786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19eb8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.497+0000 7f225422c740 1 --2- 172.21.15.5:0/675348786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eaded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.497+0000 7f225422c740 1 -- 172.21.15.5:0/675348786 >> 172.21.15.5:0/675348786 conn(0x559b19eaa060 msgr2=0x559b19eaa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.497+0000 7f225422c740 1 -- 172.21.15.5:0/675348786 shutdown_connections 2024-09-19T06:05:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.497+0000 7f225422c740 1 -- 172.21.15.5:0/675348786 wait complete. 2024-09-19T06:05:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.497+0000 7f225422c740 1 -- start start 2024-09-19T06:05:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f225422c740 1 Processor -- start 2024-09-19T06:05:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f225422c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eb5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f225422c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19eb6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f225422c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 0x559b19eb65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f225422c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559b19ebb880 con 0x559b19eb8cb0 2024-09-19T06:05:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f225422c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559b19d9b290 con 0x559b19eae410 2024-09-19T06:05:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f225422c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559b19ebba00 con 0x559b19eadaf0 2024-09-19T06:05:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f225422c740 10 client.0 _ll_register_callbacks cb 0x559b19dcc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f224affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eb5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f224affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eb5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42542/0 (socket says 172.21.15.5:42542) 2024-09-19T06:05:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f224affd640 1 -- 172.21.15.5:0/2757259968 learned_addr learned my addr 172.21.15.5:0/2757259968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f224b7fe640 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 0x559b19eb65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f2243fff640 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19eb6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f224affd640 1 -- 172.21.15.5:0/2757259968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 msgr2=0x559b19eb6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:43.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f224affd640 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19eb6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.498+0000 7f224affd640 1 -- 172.21.15.5:0/2757259968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 msgr2=0x559b19eb65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:43.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.499+0000 7f224affd640 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 0x559b19eb65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.499+0000 7f224affd640 1 -- 172.21.15.5:0/2757259968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b19b25210 con 0x559b19eadaf0 2024-09-19T06:05:43.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.499+0000 7f224b7fe640 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 0x559b19eb65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:43.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.499+0000 7f224affd640 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eb5b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f223c09c2d0 tx=0x7f223c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8ed2f14b3aada7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:43.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.499+0000 7f22327fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:05:43.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.499+0000 7f22327fc640 1 -- 172.21.15.5:0/2757259968 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c09bca0 con 0x559b19eadaf0 2024-09-19T06:05:43.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.499+0000 7f22327fc640 1 -- 172.21.15.5:0/2757259968 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f223c0d20e0 con 0x559b19eadaf0 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.499+0000 7f225422c740 1 -- 172.21.15.5:0/2757259968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22340054d0 con 0x559b19eadaf0 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.500+0000 7f22327fc640 1 -- 172.21.15.5:0/2757259968 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f223c010de0 con 0x559b19eadaf0 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.500+0000 7f225422c740 10 client.25498 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.500+0000 7f225422c740 1 -- 172.21.15.5:0/2757259968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b19bf1620 con 0x559b19eadaf0 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.500+0000 7f22327fc640 1 -- 172.21.15.5:0/2757259968 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22340054d0 con 0x559b19eadaf0 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.500+0000 7f22327fc640 1 -- 172.21.15.5:0/2757259968 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f223c0bc0b0 con 0x559b19eadaf0 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.500+0000 7f225422c740 10 client.25498 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.500+0000 7f225422c740 10 client.25498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.500+0000 7f225422c740 20 client.25498 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.500+0000 7f225422c740 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559b19ec8bb0 0x559b19ecafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.500+0000 7f225422c740 4 client.25498 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.500+0000 7f225422c740 1 -- 172.21.15.5:0/2757259968 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b19e1fa20 con 0x559b19ec8bb0 2024-09-19T06:05:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.500+0000 7f2243fff640 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559b19ec8bb0 0x559b19ecafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.501+0000 7f2243fff640 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559b19ec8bb0 0x559b19ecafe0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f764a5fe0416425 server_cookie=8953e855a227419f in_seq=0 out_seq=0 2024-09-19T06:05:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.501+0000 7f22327fc640 10 client.25498 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.501+0000 7f22327fc640 1 -- 172.21.15.5:0/2757259968 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b19e1fa20 con 0x559b19ec8bb0 2024-09-19T06:05:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.501+0000 7f22327fc640 10 client.25498 handle_command_reply: tid=0 2024-09-19T06:05:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.501+0000 7f225422c740 1 client.25498 shutdown 2024-09-19T06:05:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.502+0000 7f225422c740 1 -- 172.21.15.5:0/2757259968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 msgr2=0x559b19eb5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.502+0000 7f225422c740 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eb5b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f223c09c2d0 tx=0x7f223c009310 comp rx=0 tx=0).stop 2024-09-19T06:05:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.502+0000 7f225422c740 1 -- 172.21.15.5:0/2757259968 shutdown_connections 2024-09-19T06:05:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.502+0000 7f225422c740 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559b19ec8bb0 0x559b19ecafe0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.502+0000 7f225422c740 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eb8cb0 0x559b19eb65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.502+0000 7f225422c740 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19eb6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.502+0000 7f22327fc640 0 client.25498 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.502+0000 7f225422c740 1 --2- 172.21.15.5:0/2757259968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eb5b20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.502+0000 7f225422c740 1 -- 172.21.15.5:0/2757259968 >> 172.21.15.5:0/2757259968 conn(0x559b19eaa060 msgr2=0x559b19eafd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.502+0000 7f225422c740 1 -- 172.21.15.5:0/2757259968 shutdown_connections 2024-09-19T06:05:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.502+0000 7f225422c740 1 -- 172.21.15.5:0/2757259968 wait complete. 2024-09-19T06:05:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.502+0000 7f225422c740 20 client.25498 trim_cache size 0 max 16384 2024-09-19T06:05:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.519+0000 7f225422c740 1 Processor -- start 2024-09-19T06:05:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.519+0000 7f225422c740 1 -- start start 2024-09-19T06:05:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.519+0000 7f225422c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eaa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.519+0000 7f225422c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eae410 0x559b19eb63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.519+0000 7f225422c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eb8cb0 0x559b19ed08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.519+0000 7f225422c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1af910 con 0x559b19eae410 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.519+0000 7f225422c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1b14b0 con 0x559b19eb8cb0 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.519+0000 7f225422c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1affe0 con 0x559b19eadaf0 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f2243fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eae410 0x559b19eb63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f224b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eb8cb0 0x559b19ed08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f2243fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eae410 0x559b19eb63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56182/0 (socket says 172.21.15.5:56182) 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f2243fff640 1 -- 172.21.15.5:0/3116566638 learned_addr learned my addr 172.21.15.5:0/3116566638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f224affd640 1 --2- 172.21.15.5:0/3116566638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eaa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f2243fff640 1 -- 172.21.15.5:0/3116566638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 msgr2=0x559b19eaa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f2243fff640 1 --2- 172.21.15.5:0/3116566638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eaa8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f2243fff640 1 -- 172.21.15.5:0/3116566638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eb8cb0 msgr2=0x559b19ed08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f2243fff640 1 --2- 172.21.15.5:0/3116566638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eb8cb0 0x559b19ed08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f2243fff640 1 -- 172.21.15.5:0/3116566638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b19ec7780 con 0x559b19eae410 2024-09-19T06:05:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f224b7fe640 1 --2- 172.21.15.5:0/3116566638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eb8cb0 0x559b19ed08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f2243fff640 1 --2- 172.21.15.5:0/3116566638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eae410 0x559b19eb63a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f22340049b0 tx=0x7f2234004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ebfc09ec33c955 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f2240ff9640 1 -- 172.21.15.5:0/3116566638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22340045a0 con 0x559b19eae410 2024-09-19T06:05:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f2240ff9640 1 -- 172.21.15.5:0/3116566638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22340035f0 con 0x559b19eae410 2024-09-19T06:05:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.520+0000 7f2240ff9640 1 -- 172.21.15.5:0/3116566638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223402b2d0 con 0x559b19eae410 2024-09-19T06:05:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.521+0000 7f225422c740 1 -- 172.21.15.5:0/3116566638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eae410 msgr2=0x559b19eb63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.521+0000 7f225422c740 1 --2- 172.21.15.5:0/3116566638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eae410 0x559b19eb63a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f22340049b0 tx=0x7f2234004530 comp rx=0 tx=0).stop 2024-09-19T06:05:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.521+0000 7f225422c740 1 -- 172.21.15.5:0/3116566638 shutdown_connections 2024-09-19T06:05:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.521+0000 7f225422c740 1 --2- 172.21.15.5:0/3116566638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eb8cb0 0x559b19ed08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.521+0000 7f225422c740 1 --2- 172.21.15.5:0/3116566638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eae410 0x559b19eb63a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.521+0000 7f225422c740 1 --2- 172.21.15.5:0/3116566638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eadaf0 0x559b19eaa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.521+0000 7f225422c740 1 -- 172.21.15.5:0/3116566638 >> 172.21.15.5:0/3116566638 conn(0x559b19eaa060 msgr2=0x559b19eb7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.521+0000 7f225422c740 1 -- 172.21.15.5:0/3116566638 shutdown_connections 2024-09-19T06:05:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.521+0000 7f225422c740 1 -- 172.21.15.5:0/3116566638 wait complete. 2024-09-19T06:05:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f225422c740 1 -- start start 2024-09-19T06:05:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f225422c740 1 Processor -- start 2024-09-19T06:05:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f225422c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eadaf0 0x559b19ec1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f225422c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19ebc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f225422c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eb8cb0 0x559b19ebc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f225422c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1b0d80 con 0x559b19eadaf0 2024-09-19T06:05:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f225422c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1b06b0 con 0x559b19eae410 2024-09-19T06:05:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f225422c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1ad8f0 con 0x559b19eb8cb0 2024-09-19T06:05:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f225422c740 10 client.0 _ll_register_callbacks cb 0x7f223c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f224affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eadaf0 0x559b19ec1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f2243fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19ebc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f224affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eadaf0 0x559b19ec1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56196/0 (socket says 172.21.15.5:56196) 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f224affd640 1 -- 172.21.15.5:0/166972091 learned_addr learned my addr 172.21.15.5:0/166972091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f224b7fe640 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eb8cb0 0x559b19ebc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f2243fff640 1 -- 172.21.15.5:0/166972091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eb8cb0 msgr2=0x559b19ebc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f2243fff640 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eb8cb0 0x559b19ebc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f2243fff640 1 -- 172.21.15.5:0/166972091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eadaf0 msgr2=0x559b19ec1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f2243fff640 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eadaf0 0x559b19ec1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f2243fff640 1 -- 172.21.15.5:0/166972091 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b19ebcea0 con 0x559b19eae410 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.522+0000 7f224affd640 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eadaf0 0x559b19ec1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.523+0000 7f2243fff640 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19ebc420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x559b19eaa030 tx=0x7f2234005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df2c4fc12fa287d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.523+0000 7f22327fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.523+0000 7f22327fc640 1 -- 172.21.15.5:0/166972091 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223403b060 con 0x559b19eae410 2024-09-19T06:05:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.523+0000 7f225422c740 1 -- 172.21.15.5:0/166972091 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22340035f0 con 0x559b19eae410 2024-09-19T06:05:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.523+0000 7f22327fc640 1 -- 172.21.15.5:0/166972091 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f2234031040 con 0x559b19eae410 2024-09-19T06:05:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.523+0000 7f22327fc640 1 -- 172.21.15.5:0/166972091 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22340045a0 con 0x559b19eae410 2024-09-19T06:05:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.523+0000 7f225422c740 10 client.25562 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.523+0000 7f225422c740 1 -- 172.21.15.5:0/166972091 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b19f335c0 con 0x559b19eae410 2024-09-19T06:05:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.523+0000 7f22327fc640 1 -- 172.21.15.5:0/166972091 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22340035f0 con 0x559b19eae410 2024-09-19T06:05:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.524+0000 7f22327fc640 1 -- 172.21.15.5:0/166972091 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2234035070 con 0x559b19eae410 2024-09-19T06:05:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.524+0000 7f225422c740 10 client.25562 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.524+0000 7f225422c740 10 client.25562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.524+0000 7f225422c740 20 client.25562 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.524+0000 7f225422c740 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559b19f33af0 0x559b19f35f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.524+0000 7f225422c740 4 client.25562 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:05:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.524+0000 7f225422c740 1 -- 172.21.15.5:0/166972091 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b19e19d70 con 0x559b19f33af0 2024-09-19T06:05:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.524+0000 7f224affd640 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559b19f33af0 0x559b19f35f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.524+0000 7f22327fc640 10 client.25562 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.524+0000 7f224affd640 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559b19f33af0 0x559b19f35f10 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805b51e6d550ef55 server_cookie=e79ebd152ea826ca in_seq=0 out_seq=0 2024-09-19T06:05:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f22327fc640 1 -- 172.21.15.5:0/166972091 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b19e19d70 con 0x559b19f33af0 2024-09-19T06:05:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f22327fc640 10 client.25562 handle_command_reply: tid=0 2024-09-19T06:05:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f225422c740 1 client.25562 shutdown 2024-09-19T06:05:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f225422c740 1 -- 172.21.15.5:0/166972091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 msgr2=0x559b19ebc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f225422c740 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19ebc420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x559b19eaa030 tx=0x7f2234005f10 comp rx=0 tx=0).stop 2024-09-19T06:05:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f225422c740 1 -- 172.21.15.5:0/166972091 shutdown_connections 2024-09-19T06:05:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f225422c740 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559b19f33af0 0x559b19f35f10 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f225422c740 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b19eb8cb0 0x559b19ebc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f225422c740 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b19eae410 0x559b19ebc420 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f22327fc640 0 client.25562 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f225422c740 1 --2- 172.21.15.5:0/166972091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b19eadaf0 0x559b19ec1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f225422c740 1 -- 172.21.15.5:0/166972091 >> 172.21.15.5:0/166972091 conn(0x559b19eaa060 msgr2=0x559b19eaf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f225422c740 1 -- 172.21.15.5:0/166972091 shutdown_connections 2024-09-19T06:05:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.542+0000 7f225422c740 1 -- 172.21.15.5:0/166972091 wait complete. 2024-09-19T06:05:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f225422c740 20 client.25562 trim_cache size 0 max 16384 2024-09-19T06:05:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:05:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:05:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:05:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:05:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f22513b7640 1 -- 172.21.15.5:0/1233296592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f22380a33b0 msgr2=0x7f22380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f22513b7640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f22380a33b0 0x7f22380a5870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2244004af0 tx=0x7f224402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f22513b7640 1 -- 172.21.15.5:0/1233296592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c1009a0 msgr2=0x7f224c0fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f22513b7640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c1009a0 0x7f224c0fcde0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f223c00e9f0 tx=0x7f223c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f22513b7640 1 -- 172.21.15.5:0/1233296592 shutdown_connections 2024-09-19T06:05:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f22513b7640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f22380a33b0 0x7f22380a5870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f22513b7640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f224c1069b0 0x7f224c0fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f22513b7640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f224c101340 0x7f224c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f22513b7640 1 --2- 172.21.15.5:0/1233296592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c1009a0 0x7f224c0fcde0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f22513b7640 1 -- 172.21.15.5:0/1233296592 >> 172.21.15.5:0/1233296592 conn(0x7f224c0779a0 msgr2=0x7f224c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f22513b7640 1 -- 172.21.15.5:0/1233296592 shutdown_connections 2024-09-19T06:05:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:43.543+0000 7f22513b7640 1 -- 172.21.15.5:0/1233296592 wait complete. 2024-09-19T06:05:43.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 36489 3.09 MB/sec warmup 69 sec latency 55.610 ms 2024-09-19T06:05:44.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.730+0000 7fe0bb938640 1 -- 172.21.15.5:0/1765078872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4100a90 msgr2=0x7fe0b4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:44.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.730+0000 7fe0bb938640 1 --2- 172.21.15.5:0/1765078872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4100a90 0x7fe0b4100ef0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009930 tx=0x7fe0a402a420 comp rx=0 tx=0).stop 2024-09-19T06:05:44.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.730+0000 7fe0bb938640 1 -- 172.21.15.5:0/1765078872 shutdown_connections 2024-09-19T06:05:44.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.730+0000 7fe0bb938640 1 --2- 172.21.15.5:0/1765078872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b4101430 0x7fe0b410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.730+0000 7fe0bb938640 1 --2- 172.21.15.5:0/1765078872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4100a90 0x7fe0b4100ef0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.730+0000 7fe0bb938640 1 --2- 172.21.15.5:0/1765078872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0b4106a90 0x7fe0b4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.730+0000 7fe0bb938640 1 -- 172.21.15.5:0/1765078872 >> 172.21.15.5:0/1765078872 conn(0x7fe0b40fc7d0 msgr2=0x7fe0b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:44.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.730+0000 7fe0bb938640 1 -- 172.21.15.5:0/1765078872 shutdown_connections 2024-09-19T06:05:44.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.730+0000 7fe0bb938640 1 -- 172.21.15.5:0/1765078872 wait complete. 2024-09-19T06:05:44.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.730+0000 7fe0bb938640 1 Processor -- start 2024-09-19T06:05:44.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.730+0000 7fe0bb938640 1 -- start start 2024-09-19T06:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0bb938640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0b4100a90 0x7fe0b4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0bb938640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4101430 0x7fe0b4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0bb938640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b4106a90 0x7fe0b41a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0bb938640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b4110d90 con 0x7fe0b4101430 2024-09-19T06:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0bb938640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b4110c10 con 0x7fe0b4106a90 2024-09-19T06:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0bb938640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b4110f10 con 0x7fe0b4100a90 2024-09-19T06:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0b8ed3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4101430 0x7fe0b4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0b8ed3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4101430 0x7fe0b4199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56210/0 (socket says 172.21.15.5:56210) 2024-09-19T06:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0b8ed3640 1 -- 172.21.15.5:0/2574255875 learned_addr learned my addr 172.21.15.5:0/2574255875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0b9ed5640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b4106a90 0x7fe0b41a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0b96d4640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0b4100a90 0x7fe0b4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0b8ed3640 1 -- 172.21.15.5:0/2574255875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0b4100a90 msgr2=0x7fe0b4199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0b4100a90 0x7fe0b4199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0b8ed3640 1 -- 172.21.15.5:0/2574255875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b4106a90 msgr2=0x7fe0b41a42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b4106a90 0x7fe0b41a42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0b8ed3640 1 -- 172.21.15.5:0/2574255875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b41a47e0 con 0x7fe0b4101430 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.731+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4101430 0x7fe0b4199760 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4005ab0 tx=0x7fe0a4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.732+0000 7fe0aa7fc640 1 -- 172.21.15.5:0/2574255875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a403f070 con 0x7fe0b4101430 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.732+0000 7fe0bb938640 1 -- 172.21.15.5:0/2574255875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b41a49b0 con 0x7fe0b4101430 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.732+0000 7fe0aa7fc640 1 -- 172.21.15.5:0/2574255875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0a4031b50 con 0x7fe0b4101430 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.732+0000 7fe0aa7fc640 1 -- 172.21.15.5:0/2574255875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4035030 con 0x7fe0b4101430 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.732+0000 7fe0bb938640 1 -- 172.21.15.5:0/2574255875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b41a4e00 con 0x7fe0b4101430 2024-09-19T06:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.734+0000 7fe0aa7fc640 1 -- 172.21.15.5:0/2574255875 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe0a4042070 con 0x7fe0b4101430 2024-09-19T06:05:44.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0bdd11740 1 Processor -- start 2024-09-19T06:05:44.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0bdd11740 1 -- start start 2024-09-19T06:05:44.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0bdd11740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 0x5574fb594090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0bdd11740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb5945d0 0x5574fb594a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0bdd11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb59f370 0x5574fb5a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0bdd11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574fb4fa2f0 con 0x5574fb59f370 2024-09-19T06:05:44.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0bdd11740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5574fb4a1cf0 con 0x5574fb595b40 2024-09-19T06:05:44.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0bdd11740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5574fb4f62b0 con 0x5574fb5945d0 2024-09-19T06:05:44.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0b9ed5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 0x5574fb594090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0b96d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb5945d0 0x5574fb594a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0aa7fc640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe0a00a33b0 0x7fe0a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0b9ed5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 0x5574fb594090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58234/0 (socket says 172.21.15.5:58234) 2024-09-19T06:05:44.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0b9ed5640 1 -- 172.21.15.5:0/2898980399 learned_addr learned my addr 172.21.15.5:0/2898980399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:44.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.735+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/2898980399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb59f370 0x5574fb5a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.736+0000 7fe0b9ed5640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe0a00a33b0 0x7fe0a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.736+0000 7fe0aa7fc640 1 -- 172.21.15.5:0/2574255875 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fe0a40f0370 con 0x7fe0b4101430 2024-09-19T06:05:44.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.736+0000 7fe0b96d4640 1 -- 172.21.15.5:0/2898980399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 msgr2=0x5574fb594090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:44.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.736+0000 7fe0b96d4640 1 --2- 172.21.15.5:0/2898980399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 0x5574fb594090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.736+0000 7fe0b96d4640 1 -- 172.21.15.5:0/2898980399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb59f370 msgr2=0x5574fb5a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:44.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.736+0000 7fe0b96d4640 1 --2- 172.21.15.5:0/2898980399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb59f370 0x5574fb5a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.736+0000 7fe0b96d4640 1 -- 172.21.15.5:0/2898980399 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574fb4863a0 con 0x5574fb5945d0 2024-09-19T06:05:44.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.736+0000 7fe0b9ed5640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe0a00a33b0 0x7fe0a00a5870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009fd0 tx=0x7fe0ac004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:44.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.736+0000 7fe0b96d4640 1 --2- 172.21.15.5:0/2898980399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb5945d0 0x5574fb594a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000aac0 tx=0x7fe0b000af80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50924ce527a4c270 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:44.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.736+0000 7fe09f7fe640 1 -- 172.21.15.5:0/2898980399 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0004020 con 0x5574fb5945d0 2024-09-19T06:05:44.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.736+0000 7fe09f7fe640 1 -- 172.21.15.5:0/2898980399 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0b00041c0 con 0x5574fb5945d0 2024-09-19T06:05:44.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.737+0000 7fe09f7fe640 1 -- 172.21.15.5:0/2898980399 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0018660 con 0x5574fb5945d0 2024-09-19T06:05:44.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.737+0000 7fe0bdd11740 1 -- 172.21.15.5:0/2898980399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb5945d0 msgr2=0x5574fb594a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:44.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.737+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/2898980399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb5945d0 0x5574fb594a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000aac0 tx=0x7fe0b000af80 comp rx=0 tx=0).stop 2024-09-19T06:05:44.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.737+0000 7fe0bdd11740 1 -- 172.21.15.5:0/2898980399 shutdown_connections 2024-09-19T06:05:44.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.737+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/2898980399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb59f370 0x5574fb5a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.737+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/2898980399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb5945d0 0x5574fb594a10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.737+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/2898980399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 0x5574fb594090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.737+0000 7fe0bdd11740 1 -- 172.21.15.5:0/2898980399 >> 172.21.15.5:0/2898980399 conn(0x5574fb55d310 msgr2=0x5574fb590c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.737+0000 7fe0bdd11740 1 -- 172.21.15.5:0/2898980399 shutdown_connections 2024-09-19T06:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.737+0000 7fe0bdd11740 1 -- 172.21.15.5:0/2898980399 wait complete. 2024-09-19T06:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.738+0000 7fe0bdd11740 1 -- start start 2024-09-19T06:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.738+0000 7fe0bdd11740 1 Processor -- start 2024-09-19T06:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.738+0000 7fe0bdd11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb5945d0 0x5574fb59ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.738+0000 7fe0bdd11740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb595b40 0x5574fb59bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.738+0000 7fe0bdd11740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb59f370 0x5574fb59c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0bdd11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574fb5a2070 con 0x5574fb5945d0 2024-09-19T06:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0bdd11740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5574fb4b5060 con 0x5574fb59f370 2024-09-19T06:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0bdd11740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5574fb5a21f0 con 0x5574fb595b40 2024-09-19T06:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0bdd11740 10 client.0 _ll_register_callbacks cb 0x5574fb2088c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b96d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb5945d0 0x5574fb59ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b96d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb5945d0 0x5574fb59ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56232/0 (socket says 172.21.15.5:56232) 2024-09-19T06:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b96d4640 1 -- 172.21.15.5:0/861269139 learned_addr learned my addr 172.21.15.5:0/861269139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b9ed5640 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb59f370 0x5574fb59c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb595b40 0x5574fb59bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b96d4640 1 -- 172.21.15.5:0/861269139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb595b40 msgr2=0x5574fb59bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b96d4640 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb595b40 0x5574fb59bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b96d4640 1 -- 172.21.15.5:0/861269139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb59f370 msgr2=0x5574fb59c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b96d4640 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb59f370 0x5574fb59c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b96d4640 1 -- 172.21.15.5:0/861269139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574fb26b3c0 con 0x5574fb5945d0 2024-09-19T06:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b9ed5640 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb59f370 0x5574fb59c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb595b40 0x5574fb59bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe0b96d4640 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb5945d0 0x5574fb59ecc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0003f60 tx=0x7fe0b0003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7131e3e1c00a41af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe09cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.739+0000 7fe09cff9640 1 -- 172.21.15.5:0/861269139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0027020 con 0x5574fb5945d0 2024-09-19T06:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe09cff9640 1 -- 172.21.15.5:0/861269139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fe0b0005040 con 0x5574fb5945d0 2024-09-19T06:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe0bdd11740 1 -- 172.21.15.5:0/861269139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0b00041c0 con 0x5574fb5945d0 2024-09-19T06:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe09cff9640 1 -- 172.21.15.5:0/861269139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0b0032050 con 0x5574fb5945d0 2024-09-19T06:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe0bdd11740 10 client.15978 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe0bdd11740 1 -- 172.21.15.5:0/861269139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574fb5b1e90 con 0x5574fb5945d0 2024-09-19T06:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe09cff9640 1 -- 172.21.15.5:0/861269139 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0b00041c0 con 0x5574fb5945d0 2024-09-19T06:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe09cff9640 1 -- 172.21.15.5:0/861269139 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe0b0004a30 con 0x5574fb5945d0 2024-09-19T06:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe0bdd11740 10 client.15978 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe0bdd11740 10 client.15978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe0bdd11740 20 client.15978 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5574fb5b3350 0x5574fb5b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe0bdd11740 4 client.15978 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.740+0000 7fe0bdd11740 1 -- 172.21.15.5:0/861269139 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574fb507860 con 0x5574fb5b3350 2024-09-19T06:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.741+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5574fb5b3350 0x5574fb5b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.741+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5574fb5b3350 0x5574fb5b5740 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5da7f9b387566f0d server_cookie=9488a270daa12b91 in_seq=0 out_seq=0 2024-09-19T06:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.741+0000 7fe09cff9640 10 client.15978 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe09cff9640 1 -- 172.21.15.5:0/861269139 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574fb507860 con 0x5574fb5b3350 2024-09-19T06:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe09cff9640 10 client.15978 handle_command_reply: tid=0 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe0bdd11740 1 client.15978 shutdown 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe0bdd11740 1 -- 172.21.15.5:0/861269139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb5945d0 msgr2=0x5574fb59ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb5945d0 0x5574fb59ecc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0003f60 tx=0x7fe0b0003a20 comp rx=0 tx=0).stop 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe0bdd11740 1 -- 172.21.15.5:0/861269139 shutdown_connections 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5574fb5b3350 0x5574fb5b5740 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe09cff9640 0 client.15978 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb59f370 0x5574fb59c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb595b40 0x5574fb59bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/861269139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb5945d0 0x5574fb59ecc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe0bdd11740 1 -- 172.21.15.5:0/861269139 >> 172.21.15.5:0/861269139 conn(0x5574fb55d310 msgr2=0x5574fb596960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.742+0000 7fe0bdd11740 1 -- 172.21.15.5:0/861269139 shutdown_connections 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.743+0000 7fe0bdd11740 1 -- 172.21.15.5:0/861269139 wait complete. 2024-09-19T06:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.743+0000 7fe0bdd11740 20 client.15978 trim_cache size 0 max 16384 2024-09-19T06:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.759+0000 7fe0bdd11740 1 Processor -- start 2024-09-19T06:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.759+0000 7fe0bdd11740 1 -- start start 2024-09-19T06:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.759+0000 7fe0bdd11740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb5945d0 0x5574fb59ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.759+0000 7fe0bdd11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb595b40 0x5574fb5a0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.759+0000 7fe0bdd11740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb59f370 0x5574fb5a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.759+0000 7fe0bdd11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41ab510 con 0x5574fb595b40 2024-09-19T06:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.759+0000 7fe0bdd11740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41ad0b0 con 0x5574fb5945d0 2024-09-19T06:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.759+0000 7fe0bdd11740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41abbe0 con 0x5574fb59f370 2024-09-19T06:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe0b8ed3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb595b40 0x5574fb5a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe0b96d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb5945d0 0x5574fb59ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe0b8ed3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb595b40 0x5574fb5a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56244/0 (socket says 172.21.15.5:56244) 2024-09-19T06:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe0b8ed3640 1 -- 172.21.15.5:0/3157415059 learned_addr learned my addr 172.21.15.5:0/3157415059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe0b9ed5640 1 --2- 172.21.15.5:0/3157415059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb59f370 0x5574fb5a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe0b8ed3640 1 -- 172.21.15.5:0/3157415059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb59f370 msgr2=0x5574fb5a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/3157415059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb59f370 0x5574fb5a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe0b8ed3640 1 -- 172.21.15.5:0/3157415059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb5945d0 msgr2=0x5574fb59ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/3157415059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb5945d0 0x5574fb59ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe0b8ed3640 1 -- 172.21.15.5:0/3157415059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574fb591960 con 0x5574fb595b40 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/3157415059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb595b40 0x5574fb5a0760 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40edb80 tx=0x7fe0a4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea2675c35d80061 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe09effd640 1 -- 172.21.15.5:0/3157415059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4049040 con 0x5574fb595b40 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe09effd640 1 -- 172.21.15.5:0/3157415059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0a4031d00 con 0x5574fb595b40 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.760+0000 7fe09effd640 1 -- 172.21.15.5:0/3157415059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a40c4e40 con 0x5574fb595b40 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.761+0000 7fe0bdd11740 1 -- 172.21.15.5:0/3157415059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb595b40 msgr2=0x5574fb5a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.761+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/3157415059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb595b40 0x5574fb5a0760 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40edb80 tx=0x7fe0a4009260 comp rx=0 tx=0).stop 2024-09-19T06:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.761+0000 7fe0bdd11740 1 -- 172.21.15.5:0/3157415059 shutdown_connections 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.761+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/3157415059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb59f370 0x5574fb5a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.761+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/3157415059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb595b40 0x5574fb5a0760 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.761+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/3157415059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb5945d0 0x5574fb59ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.761+0000 7fe0bdd11740 1 -- 172.21.15.5:0/3157415059 >> 172.21.15.5:0/3157415059 conn(0x5574fb55d310 msgr2=0x5574fb59a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.761+0000 7fe0bdd11740 1 -- 172.21.15.5:0/3157415059 shutdown_connections 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.761+0000 7fe0bdd11740 1 -- 172.21.15.5:0/3157415059 wait complete. 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.761+0000 7fe0bdd11740 1 -- start start 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0bdd11740 1 Processor -- start 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0bdd11740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb5945d0 0x5574fb5a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0bdd11740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 0x5574fb599360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0bdd11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb59f370 0x5574fb5998a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0bdd11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41ac980 con 0x5574fb59f370 2024-09-19T06:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0bdd11740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41ac2b0 con 0x5574fb595b40 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0bdd11740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41a94f0 con 0x5574fb5945d0 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0bdd11740 10 client.0 _ll_register_callbacks cb 0x7fe0a4005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0b9ed5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb59f370 0x5574fb5998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0b8ed3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 0x5574fb599360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0b8ed3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 0x5574fb599360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58260/0 (socket says 172.21.15.5:58260) 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0b8ed3640 1 -- 172.21.15.5:0/1963449185 learned_addr learned my addr 172.21.15.5:0/1963449185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.762+0000 7fe0b96d4640 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb5945d0 0x5574fb5a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe0b8ed3640 1 -- 172.21.15.5:0/1963449185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb5945d0 msgr2=0x5574fb5a7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb5945d0 0x5574fb5a7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe0b8ed3640 1 -- 172.21.15.5:0/1963449185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb59f370 msgr2=0x5574fb5998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb59f370 0x5574fb5998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe0b8ed3640 1 -- 172.21.15.5:0/1963449185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574fb599e60 con 0x5574fb595b40 2024-09-19T06:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe0b9ed5640 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb59f370 0x5574fb5998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe0b8ed3640 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 0x5574fb599360 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40bc6a0 tx=0x7fe0a4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6dbf6dd4aec31d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe09cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe09cff9640 1 -- 172.21.15.5:0/1963449185 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4049040 con 0x5574fb595b40 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe09cff9640 1 -- 172.21.15.5:0/1963449185 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fe0a403b460 con 0x5574fb595b40 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe0bdd11740 1 -- 172.21.15.5:0/1963449185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a4031d00 con 0x5574fb595b40 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe09cff9640 1 -- 172.21.15.5:0/1963449185 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0a40c4e40 con 0x5574fb595b40 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe0bdd11740 10 client.25580 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.763+0000 7fe0bdd11740 1 -- 172.21.15.5:0/1963449185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574fb61dda0 con 0x5574fb595b40 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.764+0000 7fe09cff9640 1 -- 172.21.15.5:0/1963449185 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0a4031d00 con 0x5574fb595b40 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.764+0000 7fe09cff9640 1 -- 172.21.15.5:0/1963449185 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe0a409c050 con 0x5574fb595b40 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.764+0000 7fe0bdd11740 10 client.25580 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.764+0000 7fe0bdd11740 10 client.25580 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:05:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.764+0000 7fe0bdd11740 20 client.25580 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.764+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5574fb5b3350 0x5574fb620150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.764+0000 7fe0bdd11740 4 client.25580 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:05:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.764+0000 7fe0bdd11740 1 -- 172.21.15.5:0/1963449185 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574fb501d70 con 0x5574fb5b3350 2024-09-19T06:05:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.764+0000 7fe0b96d4640 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5574fb5b3350 0x5574fb620150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe09cff9640 10 client.25580 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:05:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe0b96d4640 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5574fb5b3350 0x5574fb620150 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e60d152f31bb475 server_cookie=f67b187cbd20da19 in_seq=0 out_seq=0 2024-09-19T06:05:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe09cff9640 1 -- 172.21.15.5:0/1963449185 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574fb501d70 con 0x5574fb5b3350 2024-09-19T06:05:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe09cff9640 10 client.25580 handle_command_reply: tid=0 2024-09-19T06:05:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe0bdd11740 1 client.25580 shutdown 2024-09-19T06:05:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe0bdd11740 1 -- 172.21.15.5:0/1963449185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 msgr2=0x5574fb599360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 0x5574fb599360 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40bc6a0 tx=0x7fe0a4000bc0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe0bdd11740 1 -- 172.21.15.5:0/1963449185 shutdown_connections 2024-09-19T06:05:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5574fb5b3350 0x5574fb620150 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574fb59f370 0x5574fb5998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574fb595b40 0x5574fb599360 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe0bdd11740 1 --2- 172.21.15.5:0/1963449185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574fb5945d0 0x5574fb5a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.578 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:05:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe0bdd11740 1 -- 172.21.15.5:0/1963449185 >> 172.21.15.5:0/1963449185 conn(0x5574fb55d310 msgr2=0x5574fb59af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.765+0000 7fe09cff9640 0 client.25580 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:05:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bdd11740 1 -- 172.21.15.5:0/1963449185 shutdown_connections 2024-09-19T06:05:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bdd11740 1 -- 172.21.15.5:0/1963449185 wait complete. 2024-09-19T06:05:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bdd11740 20 client.25580 trim_cache size 0 max 16384 2024-09-19T06:05:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bb938640 1 -- 172.21.15.5:0/2574255875 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe0a00a33b0 msgr2=0x7fe0a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bb938640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe0a00a33b0 0x7fe0a00a5870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009fd0 tx=0x7fe0ac004ab0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bb938640 1 -- 172.21.15.5:0/2574255875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4101430 msgr2=0x7fe0b4199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bb938640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4101430 0x7fe0b4199760 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4005ab0 tx=0x7fe0a4003b90 comp rx=0 tx=0).stop 2024-09-19T06:05:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bb938640 1 -- 172.21.15.5:0/2574255875 shutdown_connections 2024-09-19T06:05:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bb938640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b4106a90 0x7fe0b41a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bb938640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4101430 0x7fe0b4199760 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bb938640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe0a00a33b0 0x7fe0a00a5870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bb938640 1 --2- 172.21.15.5:0/2574255875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0b4100a90 0x7fe0b4199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bb938640 1 -- 172.21.15.5:0/2574255875 >> 172.21.15.5:0/2574255875 conn(0x7fe0b40fc7d0 msgr2=0x7fe0b40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.766+0000 7fe0bb938640 1 -- 172.21.15.5:0/2574255875 shutdown_connections 2024-09-19T06:05:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:43.767+0000 7fe0bb938640 1 -- 172.21.15.5:0/2574255875 wait complete. 2024-09-19T06:05:44.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:44 smithi005 ceph-mon[30955]: pgmap v74: 97 pgs: 97 active+clean; 38 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.8 MiB/s wr, 193 op/s 2024-09-19T06:05:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:44 smithi157 ceph-mon[35346]: pgmap v74: 97 pgs: 97 active+clean; 38 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.8 MiB/s wr, 193 op/s 2024-09-19T06:05:44.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 37004 3.10 MB/sec warmup 70 sec latency 90.927 ms 2024-09-19T06:05:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:44 smithi049 ceph-mon[34640]: pgmap v74: 97 pgs: 97 active+clean; 38 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.8 MiB/s wr, 193 op/s 2024-09-19T06:05:45.434 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:05:45.755 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:05:45.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 37651 3.07 MB/sec warmup 71 sec latency 14.731 ms 2024-09-19T06:05:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.265+0000 7fc333830640 1 -- 172.21.15.5:0/2019502264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc32c102ef0 msgr2=0x7fc32c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.265+0000 7fc333830640 1 --2- 172.21.15.5:0/2019502264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc32c102ef0 0x7fc32c10da40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc324009ca0 tx=0x7fc32402a950 comp rx=0 tx=0).stop 2024-09-19T06:05:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.265+0000 7fc333830640 1 -- 172.21.15.5:0/2019502264 shutdown_connections 2024-09-19T06:05:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.265+0000 7fc333830640 1 --2- 172.21.15.5:0/2019502264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc32c102ef0 0x7fc32c10da40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.265+0000 7fc333830640 1 --2- 172.21.15.5:0/2019502264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc32c102570 0x7fc32c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.265+0000 7fc333830640 1 --2- 172.21.15.5:0/2019502264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc32c100e20 0x7fc32c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.265+0000 7fc333830640 1 -- 172.21.15.5:0/2019502264 >> 172.21.15.5:0/2019502264 conn(0x7fc32c0fc7d0 msgr2=0x7fc32c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.265+0000 7fc333830640 1 -- 172.21.15.5:0/2019502264 shutdown_connections 2024-09-19T06:05:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.265+0000 7fc333830640 1 -- 172.21.15.5:0/2019502264 wait complete. 2024-09-19T06:05:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc333830640 1 Processor -- start 2024-09-19T06:05:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc333830640 1 -- start start 2024-09-19T06:05:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc333830640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc32c100e20 0x7fc32c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc333830640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc32c102570 0x7fc32c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc333830640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc32c102ef0 0x7fc32c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc333830640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c110e30 con 0x7fc32c102570 2024-09-19T06:05:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc333830640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c110cb0 con 0x7fc32c102ef0 2024-09-19T06:05:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc333830640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c110fb0 con 0x7fc32c100e20 2024-09-19T06:05:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc330dcb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc32c102570 0x7fc32c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc3315cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc32c100e20 0x7fc32c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc330dcb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc32c102570 0x7fc32c198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46168/0 (socket says 172.21.15.5:46168) 2024-09-19T06:05:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc330dcb640 1 -- 172.21.15.5:0/1232831271 learned_addr learned my addr 172.21.15.5:0/1232831271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:46.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc3315cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc32c100e20 0x7fc32c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59424/0 (socket says 172.21.15.5:59424) 2024-09-19T06:05:46.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc331dcd640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc32c102ef0 0x7fc32c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.266+0000 7fc330dcb640 1 -- 172.21.15.5:0/1232831271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc32c100e20 msgr2=0x7fc32c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:46.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.267+0000 7fc330dcb640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc32c100e20 0x7fc32c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.267+0000 7fc330dcb640 1 -- 172.21.15.5:0/1232831271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc32c102ef0 msgr2=0x7fc32c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:46.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.267+0000 7fc330dcb640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc32c102ef0 0x7fc32c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.267+0000 7fc330dcb640 1 -- 172.21.15.5:0/1232831271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c19d2b0 con 0x7fc32c102570 2024-09-19T06:05:46.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.267+0000 7fc3315cc640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc32c100e20 0x7fc32c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:46.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.267+0000 7fc331dcd640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc32c102ef0 0x7fc32c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.267+0000 7fc330dcb640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc32c102570 0x7fc32c198840 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00d950 tx=0x7fc31c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.267+0000 7fc3227fc640 1 -- 172.21.15.5:0/1232831271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c017030 con 0x7fc32c102570 2024-09-19T06:05:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.267+0000 7fc3227fc640 1 -- 172.21.15.5:0/1232831271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc31c004020 con 0x7fc32c102570 2024-09-19T06:05:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.267+0000 7fc3227fc640 1 -- 172.21.15.5:0/1232831271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c005570 con 0x7fc32c102570 2024-09-19T06:05:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.267+0000 7fc333830640 1 -- 172.21.15.5:0/1232831271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c19d5a0 con 0x7fc32c102570 2024-09-19T06:05:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.267+0000 7fc333830640 1 -- 172.21.15.5:0/1232831271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c1a4e80 con 0x7fc32c102570 2024-09-19T06:05:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc335c09740 1 Processor -- start 2024-09-19T06:05:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc3227fc640 1 -- 172.21.15.5:0/1232831271 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc31c0041d0 con 0x7fc32c102570 2024-09-19T06:05:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc335c09740 1 -- start start 2024-09-19T06:05:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc335c09740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9acb40 0x5606db9ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc335c09740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 0x5606db9aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc335c09740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9b6370 0x5606db9b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc335c09740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5606db9112f0 con 0x5606db9b6370 2024-09-19T06:05:46.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc335c09740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5606db8b8cf0 con 0x5606db9ab5d0 2024-09-19T06:05:46.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc335c09740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5606db90d2b0 con 0x5606db9acb40 2024-09-19T06:05:46.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc330dcb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9b6370 0x5606db9b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc3315cc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 0x5606db9aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc3315cc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 0x5606db9aba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:38760/0 (socket says 172.21.15.5:38760) 2024-09-19T06:05:46.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc3315cc640 1 -- 172.21.15.5:0/3273593577 learned_addr learned my addr 172.21.15.5:0/3273593577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:46.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.270+0000 7fc331dcd640 1 --2- 172.21.15.5:0/3273593577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9acb40 0x5606db9ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.271+0000 7fc3315cc640 1 -- 172.21.15.5:0/3273593577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9acb40 msgr2=0x5606db9ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:46.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.271+0000 7fc3315cc640 1 --2- 172.21.15.5:0/3273593577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9acb40 0x5606db9ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.271+0000 7fc3315cc640 1 -- 172.21.15.5:0/3273593577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9b6370 msgr2=0x5606db9b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.271+0000 7fc3315cc640 1 --2- 172.21.15.5:0/3273593577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9b6370 0x5606db9b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.271+0000 7fc3315cc640 1 -- 172.21.15.5:0/3273593577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606db89d3a0 con 0x5606db9ab5d0 2024-09-19T06:05:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.271+0000 7fc330dcb640 1 --2- 172.21.15.5:0/3273593577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9b6370 0x5606db9b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.271+0000 7fc3227fc640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc3180a3430 0x7fc3180a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.271+0000 7fc331dcd640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc3180a3430 0x7fc3180a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.271+0000 7fc3227fc640 1 -- 172.21.15.5:0/1232831271 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fc31c0cf550 con 0x7fc32c102570 2024-09-19T06:05:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.271+0000 7fc3315cc640 1 --2- 172.21.15.5:0/3273593577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 0x5606db9aba10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc32800f4c0 tx=0x7fc32800f980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=308b49fc4f204d95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.271+0000 7fc3177fe640 1 -- 172.21.15.5:0/3273593577 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328005220 con 0x5606db9ab5d0 2024-09-19T06:05:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.272+0000 7fc331dcd640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc3180a3430 0x7fc3180a58f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc324009430 tx=0x7fc324005c50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.272+0000 7fc3177fe640 1 -- 172.21.15.5:0/3273593577 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3280047a0 con 0x5606db9ab5d0 2024-09-19T06:05:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.272+0000 7fc3177fe640 1 -- 172.21.15.5:0/3273593577 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328010b60 con 0x5606db9ab5d0 2024-09-19T06:05:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.272+0000 7fc335c09740 1 -- 172.21.15.5:0/3273593577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 msgr2=0x5606db9aba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.272+0000 7fc335c09740 1 --2- 172.21.15.5:0/3273593577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 0x5606db9aba10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc32800f4c0 tx=0x7fc32800f980 comp rx=0 tx=0).stop 2024-09-19T06:05:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.272+0000 7fc335c09740 1 -- 172.21.15.5:0/3273593577 shutdown_connections 2024-09-19T06:05:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.272+0000 7fc335c09740 1 --2- 172.21.15.5:0/3273593577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9b6370 0x5606db9b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.272+0000 7fc335c09740 1 --2- 172.21.15.5:0/3273593577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 0x5606db9aba10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.272+0000 7fc335c09740 1 --2- 172.21.15.5:0/3273593577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9acb40 0x5606db9ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.272+0000 7fc335c09740 1 -- 172.21.15.5:0/3273593577 >> 172.21.15.5:0/3273593577 conn(0x5606db974310 msgr2=0x5606db9a7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:46.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.273+0000 7fc335c09740 1 -- 172.21.15.5:0/3273593577 shutdown_connections 2024-09-19T06:05:46.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.273+0000 7fc335c09740 1 -- 172.21.15.5:0/3273593577 wait complete. 2024-09-19T06:05:46.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc335c09740 1 -- start start 2024-09-19T06:05:46.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc335c09740 1 Processor -- start 2024-09-19T06:05:46.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc335c09740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 0x5606db9b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc335c09740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 0x5606db9b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc335c09740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9b6370 0x5606db9b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc335c09740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5606db9b9070 con 0x5606db9acb40 2024-09-19T06:05:46.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc335c09740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5606db8cc060 con 0x5606db9ab5d0 2024-09-19T06:05:46.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc335c09740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5606db9b91f0 con 0x5606db9b6370 2024-09-19T06:05:46.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc335c09740 10 client.0 _ll_register_callbacks cb 0x5606db61f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:46.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc330dcb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 0x5606db9b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc3315cc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 0x5606db9b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc330dcb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 0x5606db9b2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46190/0 (socket says 172.21.15.5:46190) 2024-09-19T06:05:46.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.274+0000 7fc330dcb640 1 -- 172.21.15.5:0/2519036489 learned_addr learned my addr 172.21.15.5:0/2519036489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:46.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.275+0000 7fc331dcd640 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9b6370 0x5606db9b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.275+0000 7fc3315cc640 1 -- 172.21.15.5:0/2519036489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9b6370 msgr2=0x5606db9b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:46.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.275+0000 7fc3315cc640 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9b6370 0x5606db9b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.275+0000 7fc3315cc640 1 -- 172.21.15.5:0/2519036489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 msgr2=0x5606db9b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:46.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.275+0000 7fc3315cc640 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 0x5606db9b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.275+0000 7fc3315cc640 1 -- 172.21.15.5:0/2519036489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606db6823c0 con 0x5606db9ab5d0 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.275+0000 7fc330dcb640 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 0x5606db9b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.275+0000 7fc3315cc640 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 0x5606db9b5d20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc32800fe80 tx=0x7fc328010b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=898545b3733358fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.275+0000 7fc314ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.275+0000 7fc314ff9640 1 -- 172.21.15.5:0/2519036489 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328026070 con 0x5606db9ab5d0 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.275+0000 7fc314ff9640 1 -- 172.21.15.5:0/2519036489 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fc3280051e0 con 0x5606db9ab5d0 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.275+0000 7fc335c09740 1 -- 172.21.15.5:0/2519036489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3280047a0 con 0x5606db9ab5d0 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.276+0000 7fc314ff9640 1 -- 172.21.15.5:0/2519036489 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc328032050 con 0x5606db9ab5d0 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.276+0000 7fc335c09740 10 client.25592 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.276+0000 7fc335c09740 1 -- 172.21.15.5:0/2519036489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5606db9c4ac0 con 0x5606db9ab5d0 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.276+0000 7fc314ff9640 1 -- 172.21.15.5:0/2519036489 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3280047a0 con 0x5606db9ab5d0 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.276+0000 7fc314ff9640 1 -- 172.21.15.5:0/2519036489 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc328018600 con 0x5606db9ab5d0 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.276+0000 7fc335c09740 10 client.25592 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.276+0000 7fc335c09740 10 client.25592 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.276+0000 7fc335c09740 20 client.25592 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.276+0000 7fc335c09740 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5606db9c5e70 0x5606db9c8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.276+0000 7fc335c09740 4 client.25592 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.276+0000 7fc335c09740 1 -- 172.21.15.5:0/2519036489 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606db91e860 con 0x5606db9c5e70 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.276+0000 7fc330dcb640 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5606db9c5e70 0x5606db9c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.277+0000 7fc314ff9640 10 client.25592 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.277+0000 7fc330dcb640 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5606db9c5e70 0x5606db9c8260 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb429557b3ae00b6 server_cookie=61fd657b7e2f34f5 in_seq=0 out_seq=0 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc314ff9640 1 -- 172.21.15.5:0/2519036489 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606db91e860 con 0x5606db9c5e70 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc314ff9640 10 client.25592 handle_command_reply: tid=0 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc335c09740 1 client.25592 shutdown 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc335c09740 1 -- 172.21.15.5:0/2519036489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 msgr2=0x5606db9b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc335c09740 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 0x5606db9b5d20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc32800fe80 tx=0x7fc328010b10 comp rx=0 tx=0).stop 2024-09-19T06:05:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc335c09740 1 -- 172.21.15.5:0/2519036489 shutdown_connections 2024-09-19T06:05:46.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc335c09740 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5606db9c5e70 0x5606db9c8260 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc335c09740 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9b6370 0x5606db9b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc335c09740 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 0x5606db9b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc335c09740 1 --2- 172.21.15.5:0/2519036489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9ab5d0 0x5606db9b5d20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc335c09740 1 -- 172.21.15.5:0/2519036489 >> 172.21.15.5:0/2519036489 conn(0x5606db974310 msgr2=0x5606db9ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:46.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc314ff9640 0 client.25592 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:05:46.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc335c09740 1 -- 172.21.15.5:0/2519036489 shutdown_connections 2024-09-19T06:05:46.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.278+0000 7fc335c09740 1 -- 172.21.15.5:0/2519036489 wait complete. 2024-09-19T06:05:46.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.279+0000 7fc335c09740 20 client.25592 trim_cache size 0 max 16384 2024-09-19T06:05:46.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.293+0000 7fc335c09740 1 Processor -- start 2024-09-19T06:05:46.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc335c09740 1 -- start start 2024-09-19T06:05:46.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc335c09740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9ab5d0 0x5606db9c2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc335c09740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9acb40 0x5606db9ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc335c09740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9b6370 0x5606db9ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc335c09740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c1ad710 con 0x5606db9ab5d0 2024-09-19T06:05:46.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc335c09740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c1af2b0 con 0x5606db9acb40 2024-09-19T06:05:46.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc335c09740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c1adde0 con 0x5606db9b6370 2024-09-19T06:05:46.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc331dcd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9b6370 0x5606db9ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc3315cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9ab5d0 0x5606db9c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc330dcb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9acb40 0x5606db9ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc331dcd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9b6370 0x5606db9ae660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59450/0 (socket says 172.21.15.5:59450) 2024-09-19T06:05:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc331dcd640 1 -- 172.21.15.5:0/1246038166 learned_addr learned my addr 172.21.15.5:0/1246038166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc330dcb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9acb40 0x5606db9ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:38782/0 (socket says 172.21.15.5:38782) 2024-09-19T06:05:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc330dcb640 1 -- 172.21.15.5:0/1246038166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9b6370 msgr2=0x5606db9ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc330dcb640 1 --2- 172.21.15.5:0/1246038166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9b6370 0x5606db9ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc330dcb640 1 -- 172.21.15.5:0/1246038166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9ab5d0 msgr2=0x5606db9c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc330dcb640 1 --2- 172.21.15.5:0/1246038166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9ab5d0 0x5606db9c2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.294+0000 7fc330dcb640 1 -- 172.21.15.5:0/1246038166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606db9a8960 con 0x5606db9acb40 2024-09-19T06:05:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.295+0000 7fc330dcb640 1 --2- 172.21.15.5:0/1246038166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9acb40 0x5606db9ae120 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0ccd10 tx=0x7fc31c086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b85d9fb7b3ce1d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.295+0000 7fc316ffd640 1 -- 172.21.15.5:0/1246038166 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c09b610 con 0x5606db9acb40 2024-09-19T06:05:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.295+0000 7fc316ffd640 1 -- 172.21.15.5:0/1246038166 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc31c09bf40 con 0x5606db9acb40 2024-09-19T06:05:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.295+0000 7fc316ffd640 1 -- 172.21.15.5:0/1246038166 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c09b1a0 con 0x5606db9acb40 2024-09-19T06:05:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.295+0000 7fc335c09740 1 -- 172.21.15.5:0/1246038166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9acb40 msgr2=0x5606db9ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.295+0000 7fc335c09740 1 --2- 172.21.15.5:0/1246038166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9acb40 0x5606db9ae120 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0ccd10 tx=0x7fc31c086040 comp rx=0 tx=0).stop 2024-09-19T06:05:46.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.296+0000 7fc335c09740 1 -- 172.21.15.5:0/1246038166 shutdown_connections 2024-09-19T06:05:46.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.296+0000 7fc335c09740 1 --2- 172.21.15.5:0/1246038166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9b6370 0x5606db9ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.296+0000 7fc335c09740 1 --2- 172.21.15.5:0/1246038166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9acb40 0x5606db9ae120 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.296+0000 7fc335c09740 1 --2- 172.21.15.5:0/1246038166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9ab5d0 0x5606db9c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.296+0000 7fc335c09740 1 -- 172.21.15.5:0/1246038166 >> 172.21.15.5:0/1246038166 conn(0x5606db974310 msgr2=0x5606db9b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:46.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.296+0000 7fc335c09740 1 -- 172.21.15.5:0/1246038166 shutdown_connections 2024-09-19T06:05:46.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.296+0000 7fc335c09740 1 -- 172.21.15.5:0/1246038166 wait complete. 2024-09-19T06:05:46.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.296+0000 7fc335c09740 1 -- start start 2024-09-19T06:05:46.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.296+0000 7fc335c09740 1 Processor -- start 2024-09-19T06:05:46.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc335c09740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9ab5d0 0x5606db9be840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc335c09740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 0x5606db9bed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc335c09740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9b6370 0x5606db9afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc335c09740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c1aeb80 con 0x5606db9acb40 2024-09-19T06:05:46.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc335c09740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c1ae4b0 con 0x5606db9b6370 2024-09-19T06:05:46.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc335c09740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c1ab6f0 con 0x5606db9ab5d0 2024-09-19T06:05:46.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc335c09740 10 client.0 _ll_register_callbacks cb 0x7fc31c0d5060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:46.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc330dcb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 0x5606db9bed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc3315cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9ab5d0 0x5606db9be840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc331dcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9b6370 0x5606db9afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc3315cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9ab5d0 0x5606db9be840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59452/0 (socket says 172.21.15.5:59452) 2024-09-19T06:05:46.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc3315cc640 1 -- 172.21.15.5:0/1188972507 learned_addr learned my addr 172.21.15.5:0/1188972507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:46.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc330dcb640 1 -- 172.21.15.5:0/1188972507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9ab5d0 msgr2=0x5606db9be840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:46.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc330dcb640 1 --2- 172.21.15.5:0/1188972507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9ab5d0 0x5606db9be840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc330dcb640 1 -- 172.21.15.5:0/1188972507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9b6370 msgr2=0x5606db9afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc330dcb640 1 --2- 172.21.15.5:0/1188972507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9b6370 0x5606db9afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc330dcb640 1 -- 172.21.15.5:0/1188972507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606db9b03f0 con 0x5606db9acb40 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc330dcb640 1 --2- 172.21.15.5:0/1188972507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 0x5606db9bed80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0cf980 tx=0x7fc31c00b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f13d4cb21df1c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.297+0000 7fc314ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc314ff9640 1 -- 172.21.15.5:0/1188972507 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0b4060 con 0x5606db9acb40 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc314ff9640 1 -- 172.21.15.5:0/1188972507 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fc31c0d2cd0 con 0x5606db9acb40 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc335c09740 1 -- 172.21.15.5:0/1188972507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc31c09bf40 con 0x5606db9acb40 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc314ff9640 1 -- 172.21.15.5:0/1188972507 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc31c09b380 con 0x5606db9acb40 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc335c09740 10 client.16002 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc335c09740 1 -- 172.21.15.5:0/1188972507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5606dba30460 con 0x5606db9acb40 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc314ff9640 1 -- 172.21.15.5:0/1188972507 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc31c09bf40 con 0x5606db9acb40 2024-09-19T06:05:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc314ff9640 1 -- 172.21.15.5:0/1188972507 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc31c097eb0 con 0x5606db9acb40 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc335c09740 10 client.16002 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc335c09740 10 client.16002 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc335c09740 20 client.16002 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc335c09740 1 --2- 172.21.15.5:0/1188972507 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5606dba30a80 0x5606dba32ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc335c09740 4 client.16002 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.298+0000 7fc335c09740 1 -- 172.21.15.5:0/1188972507 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606db918d70 con 0x5606dba30a80 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc3315cc640 1 --2- 172.21.15.5:0/1188972507 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5606dba30a80 0x5606dba32ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc3315cc640 1 --2- 172.21.15.5:0/1188972507 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5606dba30a80 0x5606dba32ea0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79cda88272af4268 server_cookie=e718587fc2569289 in_seq=0 out_seq=0 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc314ff9640 10 client.16002 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc314ff9640 1 -- 172.21.15.5:0/1188972507 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606db918d70 con 0x5606dba30a80 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc314ff9640 10 client.16002 handle_command_reply: tid=0 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc335c09740 1 client.16002 shutdown 2024-09-19T06:05:46.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc335c09740 1 -- 172.21.15.5:0/1188972507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 msgr2=0x5606db9bed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:46.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc335c09740 1 --2- 172.21.15.5:0/1188972507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 0x5606db9bed80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0cf980 tx=0x7fc31c00b510 comp rx=0 tx=0).stop 2024-09-19T06:05:46.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc335c09740 1 -- 172.21.15.5:0/1188972507 shutdown_connections 2024-09-19T06:05:46.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc335c09740 1 --2- 172.21.15.5:0/1188972507 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5606dba30a80 0x5606dba32ea0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc335c09740 1 --2- 172.21.15.5:0/1188972507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5606db9b6370 0x5606db9afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc335c09740 1 --2- 172.21.15.5:0/1188972507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606db9acb40 0x5606db9bed80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc335c09740 1 --2- 172.21.15.5:0/1188972507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5606db9ab5d0 0x5606db9be840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc335c09740 1 -- 172.21.15.5:0/1188972507 >> 172.21.15.5:0/1188972507 conn(0x5606db974310 msgr2=0x5606db9a64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:46.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.299+0000 7fc314ff9640 0 client.16002 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:05:46.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.300+0000 7fc335c09740 1 -- 172.21.15.5:0/1188972507 shutdown_connections 2024-09-19T06:05:46.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.300+0000 7fc335c09740 1 -- 172.21.15.5:0/1188972507 wait complete. 2024-09-19T06:05:46.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.300+0000 7fc335c09740 20 client.16002 trim_cache size 0 max 16384 2024-09-19T06:05:46.309 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:05:46.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.300+0000 7fc333830640 1 -- 172.21.15.5:0/1232831271 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc3180a3430 msgr2=0x7fc3180a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:46.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.300+0000 7fc333830640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc3180a3430 0x7fc3180a58f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc324009430 tx=0x7fc324005c50 comp rx=0 tx=0).stop 2024-09-19T06:05:46.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.300+0000 7fc333830640 1 -- 172.21.15.5:0/1232831271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc32c102570 msgr2=0x7fc32c198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:46.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.300+0000 7fc333830640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc32c102570 0x7fc32c198840 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00d950 tx=0x7fc31c00dd50 comp rx=0 tx=0).stop 2024-09-19T06:05:46.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.300+0000 7fc333830640 1 -- 172.21.15.5:0/1232831271 shutdown_connections 2024-09-19T06:05:46.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.300+0000 7fc333830640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc32c102ef0 0x7fc32c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.301+0000 7fc333830640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc32c102570 0x7fc32c198840 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.301+0000 7fc333830640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc3180a3430 0x7fc3180a58f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.301+0000 7fc333830640 1 --2- 172.21.15.5:0/1232831271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc32c100e20 0x7fc32c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:46.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.301+0000 7fc333830640 1 -- 172.21.15.5:0/1232831271 >> 172.21.15.5:0/1232831271 conn(0x7fc32c0fc7d0 msgr2=0x7fc32c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:46.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.301+0000 7fc333830640 1 -- 172.21.15.5:0/1232831271 shutdown_connections 2024-09-19T06:05:46.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:46.301+0000 7fc333830640 1 -- 172.21.15.5:0/1232831271 wait complete. 2024-09-19T06:05:46.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:46 smithi005 ceph-mon[30955]: pgmap v75: 97 pgs: 97 active+clean; 38 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-09-19T06:05:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:46 smithi157 ceph-mon[35346]: pgmap v75: 97 pgs: 97 active+clean; 38 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-09-19T06:05:46.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 38111 3.06 MB/sec warmup 72 sec latency 46.803 ms 2024-09-19T06:05:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:46 smithi049 ceph-mon[34640]: pgmap v75: 97 pgs: 97 active+clean; 38 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-09-19T06:05:47.095 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:05:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:05:47.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 38822 3.10 MB/sec warmup 73 sec latency 19.941 ms 2024-09-19T06:05:47.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.931+0000 7fa7de9cd640 1 -- 172.21.15.5:0/4290961060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d81036e0 msgr2=0x7fa7d8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:47.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.931+0000 7fa7de9cd640 1 --2- 172.21.15.5:0/4290961060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d81036e0 0x7fa7d8107b90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009930 tx=0x7fa7cc02a420 comp rx=0 tx=0).stop 2024-09-19T06:05:47.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.931+0000 7fa7de9cd640 1 -- 172.21.15.5:0/4290961060 shutdown_connections 2024-09-19T06:05:47.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.931+0000 7fa7de9cd640 1 --2- 172.21.15.5:0/4290961060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7d81082a0 0x7fa7d810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.931+0000 7fa7de9cd640 1 --2- 172.21.15.5:0/4290961060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d81036e0 0x7fa7d8107b90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.931+0000 7fa7de9cd640 1 --2- 172.21.15.5:0/4290961060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7d8102d30 0x7fa7d8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.931+0000 7fa7de9cd640 1 -- 172.21.15.5:0/4290961060 >> 172.21.15.5:0/4290961060 conn(0x7fa7d80fc7d0 msgr2=0x7fa7d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:47.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.932+0000 7fa7de9cd640 1 -- 172.21.15.5:0/4290961060 shutdown_connections 2024-09-19T06:05:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.932+0000 7fa7de9cd640 1 -- 172.21.15.5:0/4290961060 wait complete. 2024-09-19T06:05:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.932+0000 7fa7de9cd640 1 Processor -- start 2024-09-19T06:05:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.932+0000 7fa7de9cd640 1 -- start start 2024-09-19T06:05:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7de9cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7d8102d30 0x7fa7d8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7de9cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d81036e0 0x7fa7d8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7de9cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7d81082a0 0x7fa7d8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7de9cd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d8111f40 con 0x7fa7d81036e0 2024-09-19T06:05:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7de9cd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d8111dc0 con 0x7fa7d81082a0 2024-09-19T06:05:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7de9cd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81120c0 con 0x7fa7d8102d30 2024-09-19T06:05:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d81036e0 0x7fa7d8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7d7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7d8102d30 0x7fa7d8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7d7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7d8102d30 0x7fa7d8100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59466/0 (socket says 172.21.15.5:59466) 2024-09-19T06:05:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7d7fff640 1 -- 172.21.15.5:0/1772411 learned_addr learned my addr 172.21.15.5:0/1772411 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d81036e0 0x7fa7d8100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46230/0 (socket says 172.21.15.5:46230) 2024-09-19T06:05:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7d81082a0 0x7fa7d8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7d7fff640 1 -- 172.21.15.5:0/1772411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7d81082a0 msgr2=0x7fa7d8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7d7fff640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7d81082a0 0x7fa7d8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7d7fff640 1 -- 172.21.15.5:0/1772411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d81036e0 msgr2=0x7fa7d8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7d7fff640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d81036e0 0x7fa7d8100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7d7fff640 1 -- 172.21.15.5:0/1772411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d81a6a50 con 0x7fa7d8102d30 2024-09-19T06:05:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.933+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d81036e0 0x7fa7d8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:47.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.934+0000 7fa7d7fff640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7d8102d30 0x7fa7d8100990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800d950 tx=0x7fa7c800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.935+0000 7fa7d57fa640 1 -- 172.21.15.5:0/1772411 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8017070 con 0x7fa7d8102d30 2024-09-19T06:05:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.935+0000 7fa7d57fa640 1 -- 172.21.15.5:0/1772411 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7c8009c30 con 0x7fa7d8102d30 2024-09-19T06:05:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.935+0000 7fa7d57fa640 1 -- 172.21.15.5:0/1772411 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c800f620 con 0x7fa7d8102d30 2024-09-19T06:05:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.935+0000 7fa7de9cd640 1 -- 172.21.15.5:0/1772411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d81a6d40 con 0x7fa7d8102d30 2024-09-19T06:05:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.935+0000 7fa7de9cd640 1 -- 172.21.15.5:0/1772411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d81a7240 con 0x7fa7d8102d30 2024-09-19T06:05:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.937+0000 7fa7d57fa640 1 -- 172.21.15.5:0/1772411 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa7c80109a0 con 0x7fa7d8102d30 2024-09-19T06:05:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.937+0000 7fa7e0da6740 1 Processor -- start 2024-09-19T06:05:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7e0da6740 1 -- start start 2024-09-19T06:05:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7e0da6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d32c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7e0da6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d32ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7d57fa640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa7c00a3430 0x7fa7c00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7e0da6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d339760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7e0da6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a78d2922f0 con 0x55a78d32db40 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7e0da6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a78d239cf0 con 0x55a78d32c5d0 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7e0da6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a78d28e2b0 con 0x55a78d337370 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7d57fa640 1 -- 172.21.15.5:0/1772411 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fa7c80cf880 con 0x7fa7d8102d30 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7dcf6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d32c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d32ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d32ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:38814/0 (socket says 172.21.15.5:38814) 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7d77fe640 1 -- 172.21.15.5:0/611039618 learned_addr learned my addr 172.21.15.5:0/611039618 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/611039618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d339760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.938+0000 7fa7d7fff640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa7c00a3430 0x7fa7c00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.939+0000 7fa7d77fe640 1 -- 172.21.15.5:0/611039618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 msgr2=0x55a78d339760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.939+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/611039618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d339760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.939+0000 7fa7d77fe640 1 -- 172.21.15.5:0/611039618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 msgr2=0x55a78d32c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.939+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/611039618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d32c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.939+0000 7fa7d77fe640 1 -- 172.21.15.5:0/611039618 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a78d21e3a0 con 0x55a78d32c5d0 2024-09-19T06:05:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.939+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/611039618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d32c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:05:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.939+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/611039618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d32ca10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc006010 tx=0x7fa7cc004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3565bb2ffe3265f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.939+0000 7fa7d7fff640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa7c00a3430 0x7fa7c00a58f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80a4250 tx=0x7fa7c8005f70 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.939+0000 7fa7be7fc640 1 -- 172.21.15.5:0/611039618 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc035040 con 0x55a78d32c5d0 2024-09-19T06:05:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.939+0000 7fa7be7fc640 1 -- 172.21.15.5:0/611039618 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7cc004c20 con 0x55a78d32c5d0 2024-09-19T06:05:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.940+0000 7fa7be7fc640 1 -- 172.21.15.5:0/611039618 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0039e0 con 0x55a78d32c5d0 2024-09-19T06:05:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.940+0000 7fa7e0da6740 1 -- 172.21.15.5:0/611039618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 msgr2=0x55a78d32ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.940+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/611039618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d32ca10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc006010 tx=0x7fa7cc004530 comp rx=0 tx=0).stop 2024-09-19T06:05:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.940+0000 7fa7e0da6740 1 -- 172.21.15.5:0/611039618 shutdown_connections 2024-09-19T06:05:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.940+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/611039618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d339760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.940+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/611039618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d32ca10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.940+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/611039618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d32c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.940+0000 7fa7e0da6740 1 -- 172.21.15.5:0/611039618 >> 172.21.15.5:0/611039618 conn(0x55a78d2f5310 msgr2=0x55a78d328c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.940+0000 7fa7e0da6740 1 -- 172.21.15.5:0/611039618 shutdown_connections 2024-09-19T06:05:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.940+0000 7fa7e0da6740 1 -- 172.21.15.5:0/611039618 wait complete. 2024-09-19T06:05:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.941+0000 7fa7e0da6740 1 -- start start 2024-09-19T06:05:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.941+0000 7fa7e0da6740 1 Processor -- start 2024-09-19T06:05:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.941+0000 7fa7e0da6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d336d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.941+0000 7fa7e0da6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d333fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.941+0000 7fa7e0da6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d334570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.941+0000 7fa7e0da6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a78d33a010 con 0x55a78d32db40 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.941+0000 7fa7e0da6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a78d24d060 con 0x55a78d32c5d0 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.941+0000 7fa7e0da6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a78d33a190 con 0x55a78d337370 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.942+0000 7fa7e0da6740 10 client.0 _ll_register_callbacks cb 0x55a78cfa08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.942+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d333fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.942+0000 7fa7dcf6a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d334570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.942+0000 7fa7dcf6a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d336d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.942+0000 7fa7dcf6a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d336d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:38826/0 (socket says 172.21.15.5:38826) 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.942+0000 7fa7dcf6a640 1 -- 172.21.15.5:0/1979141083 learned_addr learned my addr 172.21.15.5:0/1979141083 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.942+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d333fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46244/0 (socket says 172.21.15.5:46244) 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.942+0000 7fa7dcf6a640 1 -- 172.21.15.5:0/1979141083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 msgr2=0x55a78d334570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.942+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/1979141083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d334570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.942+0000 7fa7dcf6a640 1 -- 172.21.15.5:0/1979141083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 msgr2=0x55a78d333fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.942+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/1979141083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d333fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.942+0000 7fa7dcf6a640 1 -- 172.21.15.5:0/1979141083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a78d0033c0 con 0x55a78d32c5d0 2024-09-19T06:05:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.943+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/1979141083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d333fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.943+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/1979141083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d336d20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009fd0 tx=0x7fa7d0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f46a050ee759536 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.943+0000 7fa7b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:05:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.943+0000 7fa7b77fe640 1 -- 172.21.15.5:0/1979141083 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0016070 con 0x55a78d32c5d0 2024-09-19T06:05:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.943+0000 7fa7b77fe640 1 -- 172.21.15.5:0/1979141083 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fa7d0010c60 con 0x55a78d32c5d0 2024-09-19T06:05:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.943+0000 7fa7e0da6740 1 -- 172.21.15.5:0/1979141083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7cc004c20 con 0x55a78d32c5d0 2024-09-19T06:05:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.943+0000 7fa7b77fe640 1 -- 172.21.15.5:0/1979141083 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7d0010e60 con 0x55a78d32c5d0 2024-09-19T06:05:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.943+0000 7fa7e0da6740 10 client.25610 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.943+0000 7fa7e0da6740 1 -- 172.21.15.5:0/1979141083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a78d345b10 con 0x55a78d32c5d0 2024-09-19T06:05:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.944+0000 7fa7b77fe640 1 -- 172.21.15.5:0/1979141083 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7cc004c20 con 0x55a78d32c5d0 2024-09-19T06:05:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.944+0000 7fa7b77fe640 1 -- 172.21.15.5:0/1979141083 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa7d0005810 con 0x55a78d32c5d0 2024-09-19T06:05:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.944+0000 7fa7e0da6740 10 client.25610 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.944+0000 7fa7e0da6740 10 client.25610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.944+0000 7fa7e0da6740 20 client.25610 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.944+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/1979141083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a78d346f20 0x55a78d349380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.944+0000 7fa7e0da6740 4 client.25610 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.944+0000 7fa7e0da6740 1 -- 172.21.15.5:0/1979141083 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a78d29f860 con 0x55a78d346f20 2024-09-19T06:05:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.944+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/1979141083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a78d346f20 0x55a78d349380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.944+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/1979141083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a78d346f20 0x55a78d349380 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b915430962164f server_cookie=899104ff797c8902 in_seq=0 out_seq=0 2024-09-19T06:05:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.944+0000 7fa7b77fe640 10 client.25610 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.945+0000 7fa7b77fe640 1 -- 172.21.15.5:0/1979141083 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a78d29f860 con 0x55a78d346f20 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.945+0000 7fa7b77fe640 10 client.25610 handle_command_reply: tid=0 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.945+0000 7fa7e0da6740 1 client.25610 shutdown 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.945+0000 7fa7e0da6740 1 -- 172.21.15.5:0/1979141083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 msgr2=0x55a78d336d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.945+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/1979141083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d336d20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009fd0 tx=0x7fa7d0009920 comp rx=0 tx=0).stop 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.945+0000 7fa7e0da6740 1 -- 172.21.15.5:0/1979141083 shutdown_connections 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.945+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/1979141083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a78d346f20 0x55a78d349380 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.945+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/1979141083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d334570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.945+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/1979141083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d333fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.945+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/1979141083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d336d20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.945+0000 7fa7e0da6740 1 -- 172.21.15.5:0/1979141083 >> 172.21.15.5:0/1979141083 conn(0x55a78d2f5310 msgr2=0x55a78d32e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.945+0000 7fa7b77fe640 0 client.25610 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.946+0000 7fa7e0da6740 1 -- 172.21.15.5:0/1979141083 shutdown_connections 2024-09-19T06:05:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.946+0000 7fa7e0da6740 1 -- 172.21.15.5:0/1979141083 wait complete. 2024-09-19T06:05:47.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.946+0000 7fa7e0da6740 20 client.25610 trim_cache size 0 max 16384 2024-09-19T06:05:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.961+0000 7fa7e0da6740 1 Processor -- start 2024-09-19T06:05:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.961+0000 7fa7e0da6740 1 -- start start 2024-09-19T06:05:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.961+0000 7fa7e0da6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d3432b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.961+0000 7fa7e0da6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d32db40 0x55a78d328700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.961+0000 7fa7e0da6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d337370 0x55a78d3328a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.961+0000 7fa7e0da6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81af980 con 0x55a78d337370 2024-09-19T06:05:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.961+0000 7fa7e0da6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81b1520 con 0x55a78d32c5d0 2024-09-19T06:05:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.961+0000 7fa7e0da6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81b0050 con 0x55a78d32db40 2024-09-19T06:05:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7dcf6a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d3432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7dcf6a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d3432b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:38842/0 (socket says 172.21.15.5:38842) 2024-09-19T06:05:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7dcf6a640 1 -- 172.21.15.5:0/2434363372 learned_addr learned my addr 172.21.15.5:0/2434363372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/2434363372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d337370 0x55a78d3328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/2434363372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d32db40 0x55a78d328700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7dcf6a640 1 -- 172.21.15.5:0/2434363372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d32db40 msgr2=0x55a78d328700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/2434363372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d32db40 0x55a78d328700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7dcf6a640 1 -- 172.21.15.5:0/2434363372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 msgr2=0x55a78d3432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/2434363372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d3432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7dcf6a640 1 -- 172.21.15.5:0/2434363372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a78d332520 con 0x55a78d337370 2024-09-19T06:05:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/2434363372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d32db40 0x55a78d328700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/2434363372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d337370 0x55a78d3328a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009c90 tx=0x7fa7d0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a30d2358dc594e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.962+0000 7fa7bdffb640 1 -- 172.21.15.5:0/2434363372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0016040 con 0x55a78d337370 2024-09-19T06:05:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.963+0000 7fa7bdffb640 1 -- 172.21.15.5:0/2434363372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7d00096a0 con 0x55a78d337370 2024-09-19T06:05:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.963+0000 7fa7bdffb640 1 -- 172.21.15.5:0/2434363372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0012250 con 0x55a78d337370 2024-09-19T06:05:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.963+0000 7fa7e0da6740 1 -- 172.21.15.5:0/2434363372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d337370 msgr2=0x55a78d3328a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.963+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/2434363372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d337370 0x55a78d3328a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009c90 tx=0x7fa7d0000f40 comp rx=0 tx=0).stop 2024-09-19T06:05:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.963+0000 7fa7e0da6740 1 -- 172.21.15.5:0/2434363372 shutdown_connections 2024-09-19T06:05:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.963+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/2434363372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d337370 0x55a78d3328a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.963+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/2434363372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d32db40 0x55a78d328700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.963+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/2434363372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d3432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.963+0000 7fa7e0da6740 1 -- 172.21.15.5:0/2434363372 >> 172.21.15.5:0/2434363372 conn(0x55a78d2f5310 msgr2=0x55a78d32f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.964+0000 7fa7e0da6740 1 -- 172.21.15.5:0/2434363372 shutdown_connections 2024-09-19T06:05:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.964+0000 7fa7e0da6740 1 -- 172.21.15.5:0/2434363372 wait complete. 2024-09-19T06:05:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.964+0000 7fa7e0da6740 1 -- start start 2024-09-19T06:05:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.964+0000 7fa7e0da6740 1 Processor -- start 2024-09-19T06:05:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.964+0000 7fa7e0da6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d33f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.964+0000 7fa7e0da6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d33fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.964+0000 7fa7e0da6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d33e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.964+0000 7fa7e0da6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81b0df0 con 0x55a78d32db40 2024-09-19T06:05:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.964+0000 7fa7e0da6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81b0720 con 0x55a78d32c5d0 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.964+0000 7fa7e0da6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81ad960 con 0x55a78d337370 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.964+0000 7fa7e0da6740 10 client.0 _ll_register_callbacks cb 0x7fa7d0009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d33fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d33fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46258/0 (socket says 172.21.15.5:46258) 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7d77fe640 1 -- 172.21.15.5:0/4236091169 learned_addr learned my addr 172.21.15.5:0/4236091169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d33f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7d77fe640 1 -- 172.21.15.5:0/4236091169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 msgr2=0x55a78d33e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d33e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d33e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7d77fe640 1 -- 172.21.15.5:0/4236091169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 msgr2=0x55a78d33f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d33f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7d77fe640 1 -- 172.21.15.5:0/4236091169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a78d33ec70 con 0x55a78d32db40 2024-09-19T06:05:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d33e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:05:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7d77fe640 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d33fc20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc02a460 tx=0x7fa7cc004fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f428459daf47849b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:05:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7b77fe640 1 -- 172.21.15.5:0/4236091169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc054020 con 0x55a78d32db40 2024-09-19T06:05:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7b77fe640 1 -- 172.21.15.5:0/4236091169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fa7cc004770 con 0x55a78d32db40 2024-09-19T06:05:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.965+0000 7fa7e0da6740 1 -- 172.21.15.5:0/4236091169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d00096a0 con 0x55a78d32db40 2024-09-19T06:05:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.966+0000 7fa7b77fe640 1 -- 172.21.15.5:0/4236091169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7cc05f050 con 0x55a78d32db40 2024-09-19T06:05:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.966+0000 7fa7e0da6740 10 client.16020 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.966+0000 7fa7e0da6740 1 -- 172.21.15.5:0/4236091169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a78d34eda0 con 0x55a78d32db40 2024-09-19T06:05:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.966+0000 7fa7b77fe640 1 -- 172.21.15.5:0/4236091169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7d00096a0 con 0x55a78d32db40 2024-09-19T06:05:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.966+0000 7fa7b77fe640 1 -- 172.21.15.5:0/4236091169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa7cc003bc0 con 0x55a78d32db40 2024-09-19T06:05:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.966+0000 7fa7e0da6740 10 client.16020 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.966+0000 7fa7e0da6740 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.966+0000 7fa7e0da6740 20 client.16020 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.966+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a78d3b1c20 0x55a78d3b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.966+0000 7fa7e0da6740 4 client.16020 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:05:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.966+0000 7fa7e0da6740 1 -- 172.21.15.5:0/4236091169 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a78d299d70 con 0x55a78d3b1c20 2024-09-19T06:05:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.966+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a78d3b1c20 0x55a78d3b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7dcf6a640 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a78d3b1c20 0x55a78d3b4010 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=431c03903bed030f server_cookie=68626e640af2c839 in_seq=0 out_seq=0 2024-09-19T06:05:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7b77fe640 10 client.16020 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7b77fe640 1 -- 172.21.15.5:0/4236091169 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a78d299d70 con 0x55a78d3b1c20 2024-09-19T06:05:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7b77fe640 10 client.16020 handle_command_reply: tid=0 2024-09-19T06:05:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7e0da6740 1 client.16020 shutdown 2024-09-19T06:05:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7e0da6740 1 -- 172.21.15.5:0/4236091169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 msgr2=0x55a78d33fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d33fc20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc02a460 tx=0x7fa7cc004fd0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7e0da6740 1 -- 172.21.15.5:0/4236091169 shutdown_connections 2024-09-19T06:05:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a78d3b1c20 0x55a78d3b4010 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a78d337370 0x55a78d33e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a78d32db40 0x55a78d33fc20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7e0da6740 1 --2- 172.21.15.5:0/4236091169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78d32c5d0 0x55a78d33f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7e0da6740 1 -- 172.21.15.5:0/4236091169 >> 172.21.15.5:0/4236091169 conn(0x55a78d2f5310 msgr2=0x55a78d32f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7b77fe640 0 client.16020 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7e0da6740 1 -- 172.21.15.5:0/4236091169 shutdown_connections 2024-09-19T06:05:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.967+0000 7fa7e0da6740 1 -- 172.21.15.5:0/4236091169 wait complete. 2024-09-19T06:05:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7e0da6740 20 client.16020 trim_cache size 0 max 16384 2024-09-19T06:05:47.975 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:05:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7de9cd640 1 -- 172.21.15.5:0/1772411 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa7c00a3430 msgr2=0x7fa7c00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7de9cd640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa7c00a3430 0x7fa7c00a58f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80a4250 tx=0x7fa7c8005f70 comp rx=0 tx=0).stop 2024-09-19T06:05:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7de9cd640 1 -- 172.21.15.5:0/1772411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7d8102d30 msgr2=0x7fa7d8100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7de9cd640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7d8102d30 0x7fa7d8100990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800d950 tx=0x7fa7c800de10 comp rx=0 tx=0).stop 2024-09-19T06:05:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7de9cd640 1 -- 172.21.15.5:0/1772411 shutdown_connections 2024-09-19T06:05:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7de9cd640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7d81082a0 0x7fa7d8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7de9cd640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d81036e0 0x7fa7d8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7de9cd640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa7c00a3430 0x7fa7c00a58f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7de9cd640 1 --2- 172.21.15.5:0/1772411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7d8102d30 0x7fa7d8100990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7de9cd640 1 -- 172.21.15.5:0/1772411 >> 172.21.15.5:0/1772411 conn(0x7fa7d80fc7d0 msgr2=0x7fa7d8105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7de9cd640 1 -- 172.21.15.5:0/1772411 shutdown_connections 2024-09-19T06:05:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:47.968+0000 7fa7de9cd640 1 -- 172.21.15.5:0/1772411 wait complete. 2024-09-19T06:05:48.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:48 smithi005 ceph-mon[30955]: pgmap v76: 97 pgs: 97 active+clean; 38 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-09-19T06:05:48.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:48 smithi157 ceph-mon[35346]: pgmap v76: 97 pgs: 97 active+clean; 38 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-09-19T06:05:48.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 39287 3.12 MB/sec warmup 74 sec latency 15.275 ms 2024-09-19T06:05:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:48 smithi049 ceph-mon[34640]: pgmap v76: 97 pgs: 97 active+clean; 38 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-09-19T06:05:49.755 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:05:49.755 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:05:49.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 39533 3.08 MB/sec warmup 75 sec latency 14.976 ms 2024-09-19T06:05:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:05:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.560+0000 7f08e1f07640 1 -- 172.21.15.5:0/3586108998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08dc102780 msgr2=0x7f08dc1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.560+0000 7f08e1f07640 1 --2- 172.21.15.5:0/3586108998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08dc102780 0x7f08dc1067d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009900 tx=0x7f08d002a410 comp rx=0 tx=0).stop 2024-09-19T06:05:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.560+0000 7f08e1f07640 1 -- 172.21.15.5:0/3586108998 shutdown_connections 2024-09-19T06:05:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.560+0000 7f08e1f07640 1 --2- 172.21.15.5:0/3586108998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08dc106f00 0x7f08dc10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.560+0000 7f08e1f07640 1 --2- 172.21.15.5:0/3586108998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08dc102780 0x7f08dc1067d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.560+0000 7f08e1f07640 1 --2- 172.21.15.5:0/3586108998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08dc101dd0 0x7f08dc1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.560+0000 7f08e1f07640 1 -- 172.21.15.5:0/3586108998 >> 172.21.15.5:0/3586108998 conn(0x7f08dc0fd8b0 msgr2=0x7f08dc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.560+0000 7f08e1f07640 1 -- 172.21.15.5:0/3586108998 shutdown_connections 2024-09-19T06:05:50.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.560+0000 7f08e1f07640 1 -- 172.21.15.5:0/3586108998 wait complete. 2024-09-19T06:05:50.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.561+0000 7f08e1f07640 1 Processor -- start 2024-09-19T06:05:50.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.561+0000 7f08e1f07640 1 -- start start 2024-09-19T06:05:50.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.561+0000 7f08e1f07640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08dc101dd0 0x7f08dc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.561+0000 7f08e1f07640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08dc102780 0x7f08dc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.561+0000 7f08e1f07640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08dc106f00 0x7f08dc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.561+0000 7f08e1f07640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc111dd0 con 0x7f08dc106f00 2024-09-19T06:05:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.561+0000 7f08e1f07640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc111c50 con 0x7f08dc101dd0 2024-09-19T06:05:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.561+0000 7f08e1f07640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc111f50 con 0x7f08dc102780 2024-09-19T06:05:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.561+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08dc106f00 0x7f08dc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.561+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08dc106f00 0x7f08dc19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46278/0 (socket says 172.21.15.5:46278) 2024-09-19T06:05:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.561+0000 7f08dbfff640 1 -- 172.21.15.5:0/1654877988 learned_addr learned my addr 172.21.15.5:0/1654877988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.561+0000 7f08daffd640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08dc102780 0x7f08dc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.562+0000 7f08db7fe640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08dc101dd0 0x7f08dc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.562+0000 7f08daffd640 1 -- 172.21.15.5:0/1654877988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08dc101dd0 msgr2=0x7f08dc19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.562+0000 7f08daffd640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08dc101dd0 0x7f08dc19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.562+0000 7f08daffd640 1 -- 172.21.15.5:0/1654877988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08dc106f00 msgr2=0x7f08dc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.562+0000 7f08daffd640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08dc106f00 0x7f08dc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.562+0000 7f08daffd640 1 -- 172.21.15.5:0/1654877988 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc19f620 con 0x7f08dc102780 2024-09-19T06:05:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.562+0000 7f08dbfff640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08dc106f00 0x7f08dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.562+0000 7f08daffd640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08dc102780 0x7f08dc19abb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f08d002a930 tx=0x7f08d0031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.562+0000 7f08d8ff9640 1 -- 172.21.15.5:0/1654877988 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d003f030 con 0x7f08dc102780 2024-09-19T06:05:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.562+0000 7f08d8ff9640 1 -- 172.21.15.5:0/1654877988 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08d0004490 con 0x7f08dc102780 2024-09-19T06:05:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.562+0000 7f08d8ff9640 1 -- 172.21.15.5:0/1654877988 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0035070 con 0x7f08dc102780 2024-09-19T06:05:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.562+0000 7f08e1f07640 1 -- 172.21.15.5:0/1654877988 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc19f8b0 con 0x7f08dc102780 2024-09-19T06:05:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.563+0000 7f08e1f07640 1 -- 172.21.15.5:0/1654877988 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc19fd40 con 0x7f08dc102780 2024-09-19T06:05:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.565+0000 7f08d8ff9640 1 -- 172.21.15.5:0/1654877988 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f08d0042030 con 0x7f08dc102780 2024-09-19T06:05:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.565+0000 7f08e42e0740 1 Processor -- start 2024-09-19T06:05:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.565+0000 7f08e42e0740 1 -- start start 2024-09-19T06:05:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.565+0000 7f08e42e0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6eeeb60 0x564ed6eeef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08e42e0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 0x564ed6ef9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08e42e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6ef9d40 0x564ed6efc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08e42e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564ed6e54280 con 0x564ed6ef9d40 2024-09-19T06:05:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08e42e0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564ed6dfbb10 con 0x564ed6eef480 2024-09-19T06:05:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08e42e0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564ed6e50240 con 0x564ed6eeeb60 2024-09-19T06:05:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08db7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 0x564ed6ef9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08daffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6ef9d40 0x564ed6efc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08db7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 0x564ed6ef9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:38884/0 (socket says 172.21.15.5:38884) 2024-09-19T06:05:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6eeeb60 0x564ed6eeef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08db7fe640 1 -- 172.21.15.5:0/2331604147 learned_addr learned my addr 172.21.15.5:0/2331604147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08d8ff9640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f08c40a33b0 0x7f08c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08dbfff640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f08c40a33b0 0x7f08c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08db7fe640 1 -- 172.21.15.5:0/2331604147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6eeeb60 msgr2=0x564ed6eeef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08db7fe640 1 --2- 172.21.15.5:0/2331604147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6eeeb60 0x564ed6eeef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08db7fe640 1 -- 172.21.15.5:0/2331604147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6ef9d40 msgr2=0x564ed6efc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08d8ff9640 1 -- 172.21.15.5:0/1654877988 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f08d00effe0 con 0x7f08dc102780 2024-09-19T06:05:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08db7fe640 1 --2- 172.21.15.5:0/2331604147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6ef9d40 0x564ed6efc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08db7fe640 1 -- 172.21.15.5:0/2331604147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed6e081f0 con 0x564ed6eef480 2024-09-19T06:05:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08dbfff640 1 --2- 172.21.15.5:0/2331604147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6eeeb60 0x564ed6eeef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.566+0000 7f08daffd640 1 --2- 172.21.15.5:0/2331604147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6ef9d40 0x564ed6efc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.567+0000 7f08db7fe640 1 --2- 172.21.15.5:0/2331604147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 0x564ed6ef9800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009c50 tx=0x7f08cc010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c444a1c75aa2f255 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.567+0000 7f08dbfff640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f08c40a33b0 0x7f08c40a5870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f08d40094d0 tx=0x7f08d4004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.567+0000 7f08c1ffb640 1 -- 172.21.15.5:0/2331604147 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc004020 con 0x564ed6eef480 2024-09-19T06:05:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.567+0000 7f08c1ffb640 1 -- 172.21.15.5:0/2331604147 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08cc0041c0 con 0x564ed6eef480 2024-09-19T06:05:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.567+0000 7f08c1ffb640 1 -- 172.21.15.5:0/2331604147 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc003e60 con 0x564ed6eef480 2024-09-19T06:05:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.568+0000 7f08e42e0740 1 -- 172.21.15.5:0/2331604147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 msgr2=0x564ed6ef9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.568+0000 7f08e42e0740 1 --2- 172.21.15.5:0/2331604147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 0x564ed6ef9800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009c50 tx=0x7f08cc010740 comp rx=0 tx=0).stop 2024-09-19T06:05:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.568+0000 7f08e42e0740 1 -- 172.21.15.5:0/2331604147 shutdown_connections 2024-09-19T06:05:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.568+0000 7f08e42e0740 1 --2- 172.21.15.5:0/2331604147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6ef9d40 0x564ed6efc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.568+0000 7f08e42e0740 1 --2- 172.21.15.5:0/2331604147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 0x564ed6ef9800 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.568+0000 7f08e42e0740 1 --2- 172.21.15.5:0/2331604147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6eeeb60 0x564ed6eeef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.568+0000 7f08e42e0740 1 -- 172.21.15.5:0/2331604147 >> 172.21.15.5:0/2331604147 conn(0x564ed6eeb0f0 msgr2=0x564ed6eeb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.568+0000 7f08e42e0740 1 -- 172.21.15.5:0/2331604147 shutdown_connections 2024-09-19T06:05:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.568+0000 7f08e42e0740 1 -- 172.21.15.5:0/2331604147 wait complete. 2024-09-19T06:05:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08e42e0740 1 -- start start 2024-09-19T06:05:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08e42e0740 1 Processor -- start 2024-09-19T06:05:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08e42e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 0x564ed6ef6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08e42e0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 0x564ed6ef7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08e42e0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 0x564ed6ef7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08e42e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564ed6efc910 con 0x564ed6eeeb60 2024-09-19T06:05:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08e42e0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564ed6e0ef10 con 0x564ed6eef480 2024-09-19T06:05:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08e42e0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564ed6efca90 con 0x564ed6ef9d40 2024-09-19T06:05:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08e42e0740 10 client.0 _ll_register_callbacks cb 0x564ed6dd7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08db7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 0x564ed6ef6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 0x564ed6ef7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 0x564ed6ef7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59554/0 (socket says 172.21.15.5:59554) 2024-09-19T06:05:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08db7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 0x564ed6ef6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46292/0 (socket says 172.21.15.5:46292) 2024-09-19T06:05:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.569+0000 7f08dbfff640 1 -- 172.21.15.5:0/363262745 learned_addr learned my addr 172.21.15.5:0/363262745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.570+0000 7f08daffd640 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 0x564ed6ef7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.570+0000 7f08dbfff640 1 -- 172.21.15.5:0/363262745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 msgr2=0x564ed6ef7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.570+0000 7f08dbfff640 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 0x564ed6ef7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.570+0000 7f08dbfff640 1 -- 172.21.15.5:0/363262745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 msgr2=0x564ed6ef6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.570+0000 7f08dbfff640 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 0x564ed6ef6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.570+0000 7f08dbfff640 1 -- 172.21.15.5:0/363262745 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed6bc56a0 con 0x564ed6ef9d40 2024-09-19T06:05:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.570+0000 7f08daffd640 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 0x564ed6ef7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.570+0000 7f08db7fe640 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 0x564ed6ef6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.570+0000 7f08dbfff640 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 0x564ed6ef7670 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f08d4013fd0 tx=0x7f08d4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de7be8ba37ec2a1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.570+0000 7f08baffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:05:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.570+0000 7f08baffd640 1 -- 172.21.15.5:0/363262745 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4022860 con 0x564ed6ef9d40 2024-09-19T06:05:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08baffd640 1 -- 172.21.15.5:0/363262745 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f08d4022a00 con 0x564ed6ef9d40 2024-09-19T06:05:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08e42e0740 1 -- 172.21.15.5:0/363262745 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08cc0041c0 con 0x564ed6ef9d40 2024-09-19T06:05:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08baffd640 1 -- 172.21.15.5:0/363262745 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f08d4022c00 con 0x564ed6ef9d40 2024-09-19T06:05:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08e42e0740 10 client.25561 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08e42e0740 1 -- 172.21.15.5:0/363262745 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564ed6f08540 con 0x564ed6ef9d40 2024-09-19T06:05:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08baffd640 1 -- 172.21.15.5:0/363262745 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08cc0041c0 con 0x564ed6ef9d40 2024-09-19T06:05:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08baffd640 1 -- 172.21.15.5:0/363262745 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f08d401f040 con 0x564ed6ef9d40 2024-09-19T06:05:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08e42e0740 10 client.25561 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08e42e0740 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08e42e0740 20 client.25561 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08e42e0740 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564ed6f09950 0x564ed6f0bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08e42e0740 4 client.25561 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.571+0000 7f08e42e0740 1 -- 172.21.15.5:0/363262745 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ed6e617f0 con 0x564ed6f09950 2024-09-19T06:05:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.572+0000 7f08db7fe640 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564ed6f09950 0x564ed6f0bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.572+0000 7f08baffd640 10 client.25561 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.572+0000 7f08db7fe640 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564ed6f09950 0x564ed6f0bdf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d27a57397203d08 server_cookie=47c18f5cca511670 in_seq=0 out_seq=0 2024-09-19T06:05:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08baffd640 1 -- 172.21.15.5:0/363262745 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ed6e617f0 con 0x564ed6f09950 2024-09-19T06:05:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08baffd640 10 client.25561 handle_command_reply: tid=0 2024-09-19T06:05:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08e42e0740 1 client.25561 shutdown 2024-09-19T06:05:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08e42e0740 1 -- 172.21.15.5:0/363262745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 msgr2=0x564ed6ef7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08e42e0740 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 0x564ed6ef7670 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f08d4013fd0 tx=0x7f08d4004a40 comp rx=0 tx=0).stop 2024-09-19T06:05:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08e42e0740 1 -- 172.21.15.5:0/363262745 shutdown_connections 2024-09-19T06:05:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08e42e0740 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564ed6f09950 0x564ed6f0bdf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08e42e0740 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 0x564ed6ef7670 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08e42e0740 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eef480 0x564ed6ef7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08baffd640 0 client.25561 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:50.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08e42e0740 1 --2- 172.21.15.5:0/363262745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 0x564ed6ef6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08e42e0740 1 -- 172.21.15.5:0/363262745 >> 172.21.15.5:0/363262745 conn(0x564ed6eeb0f0 msgr2=0x564ed6ef0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:50.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08e42e0740 1 -- 172.21.15.5:0/363262745 shutdown_connections 2024-09-19T06:05:50.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08e42e0740 1 -- 172.21.15.5:0/363262745 wait complete. 2024-09-19T06:05:50.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.573+0000 7f08e42e0740 20 client.25561 trim_cache size 0 max 16384 2024-09-19T06:05:50.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.594+0000 7f08e42e0740 1 Processor -- start 2024-09-19T06:05:50.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.594+0000 7f08e42e0740 1 -- start start 2024-09-19T06:05:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08e42e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 0x564ed6ef15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08e42e0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6eef480 0x564ed6ef8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08e42e0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6ef9d40 0x564ed6f116c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08e42e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1afa00 con 0x564ed6eeeb60 2024-09-19T06:05:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08e42e0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b15a0 con 0x564ed6ef9d40 2024-09-19T06:05:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08e42e0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b00d0 con 0x564ed6eef480 2024-09-19T06:05:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08db7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 0x564ed6ef15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08db7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 0x564ed6ef15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46308/0 (socket says 172.21.15.5:46308) 2024-09-19T06:05:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08db7fe640 1 -- 172.21.15.5:0/3600748750 learned_addr learned my addr 172.21.15.5:0/3600748750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08daffd640 1 --2- 172.21.15.5:0/3600748750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6eef480 0x564ed6ef8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08dbfff640 1 --2- 172.21.15.5:0/3600748750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6ef9d40 0x564ed6f116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08daffd640 1 -- 172.21.15.5:0/3600748750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6ef9d40 msgr2=0x564ed6f116c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:50.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08daffd640 1 --2- 172.21.15.5:0/3600748750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6ef9d40 0x564ed6f116c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08daffd640 1 -- 172.21.15.5:0/3600748750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 msgr2=0x564ed6ef15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:50.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08daffd640 1 --2- 172.21.15.5:0/3600748750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 0x564ed6ef15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08daffd640 1 -- 172.21.15.5:0/3600748750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed6eef2b0 con 0x564ed6eef480 2024-09-19T06:05:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.595+0000 7f08db7fe640 1 --2- 172.21.15.5:0/3600748750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 0x564ed6ef15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.596+0000 7f08daffd640 1 --2- 172.21.15.5:0/3600748750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6eef480 0x564ed6ef8a50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f08d00ed740 tx=0x7f08d00bc1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40943fe2878ca7d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.596+0000 7f08c17fa640 1 -- 172.21.15.5:0/3600748750 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0049040 con 0x564ed6eef480 2024-09-19T06:05:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.596+0000 7f08c17fa640 1 -- 172.21.15.5:0/3600748750 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08d00bc9d0 con 0x564ed6eef480 2024-09-19T06:05:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.596+0000 7f08c17fa640 1 -- 172.21.15.5:0/3600748750 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d00b8940 con 0x564ed6eef480 2024-09-19T06:05:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.597+0000 7f08e42e0740 1 -- 172.21.15.5:0/3600748750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6eef480 msgr2=0x564ed6ef8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.597+0000 7f08e42e0740 1 --2- 172.21.15.5:0/3600748750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6eef480 0x564ed6ef8a50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f08d00ed740 tx=0x7f08d00bc1d0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.597+0000 7f08e42e0740 1 -- 172.21.15.5:0/3600748750 shutdown_connections 2024-09-19T06:05:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.597+0000 7f08e42e0740 1 --2- 172.21.15.5:0/3600748750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6ef9d40 0x564ed6f116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.597+0000 7f08e42e0740 1 --2- 172.21.15.5:0/3600748750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6eef480 0x564ed6ef8a50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.597+0000 7f08e42e0740 1 --2- 172.21.15.5:0/3600748750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eeeb60 0x564ed6ef15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.597+0000 7f08e42e0740 1 -- 172.21.15.5:0/3600748750 >> 172.21.15.5:0/3600748750 conn(0x564ed6eeb0f0 msgr2=0x564ed6ef8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.597+0000 7f08e42e0740 1 -- 172.21.15.5:0/3600748750 shutdown_connections 2024-09-19T06:05:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.597+0000 7f08e42e0740 1 -- 172.21.15.5:0/3600748750 wait complete. 2024-09-19T06:05:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.597+0000 7f08e42e0740 1 -- start start 2024-09-19T06:05:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.598+0000 7f08e42e0740 1 Processor -- start 2024-09-19T06:05:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.598+0000 7f08e42e0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eeeb60 0x564ed6efda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.598+0000 7f08e42e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eef480 0x564ed6efdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.598+0000 7f08e42e0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 0x564ed6efe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.598+0000 7f08e42e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b0e70 con 0x564ed6eef480 2024-09-19T06:05:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.598+0000 7f08e42e0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b07a0 con 0x564ed6eeeb60 2024-09-19T06:05:50.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.598+0000 7f08e42e0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1ad9e0 con 0x564ed6ef9d40 2024-09-19T06:05:50.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.598+0000 7f08e42e0740 10 client.0 _ll_register_callbacks cb 0x7f08d40137c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:50.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.598+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 0x564ed6efe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.598+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 0x564ed6efe510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59564/0 (socket says 172.21.15.5:59564) 2024-09-19T06:05:50.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.598+0000 7f08dbfff640 1 -- 172.21.15.5:0/210186890 learned_addr learned my addr 172.21.15.5:0/210186890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:50.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.598+0000 7f08daffd640 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eef480 0x564ed6efdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.599+0000 7f08db7fe640 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eeeb60 0x564ed6efda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.599+0000 7f08dbfff640 1 -- 172.21.15.5:0/210186890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eeeb60 msgr2=0x564ed6efda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:50.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.599+0000 7f08dbfff640 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eeeb60 0x564ed6efda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.599+0000 7f08dbfff640 1 -- 172.21.15.5:0/210186890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eef480 msgr2=0x564ed6efdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:50.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.599+0000 7f08dbfff640 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eef480 0x564ed6efdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.599+0000 7f08dbfff640 1 -- 172.21.15.5:0/210186890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed6f736e0 con 0x564ed6ef9d40 2024-09-19T06:05:50.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.599+0000 7f08daffd640 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eef480 0x564ed6efdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:50.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.599+0000 7f08db7fe640 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eeeb60 0x564ed6efda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:50.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.599+0000 7f08dbfff640 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 0x564ed6efe510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f08d4000f80 tx=0x7f08d4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec34ed945ce36eda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:50.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.599+0000 7f08baffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:05:50.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.600+0000 7f08baffd640 1 -- 172.21.15.5:0/210186890 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4022870 con 0x564ed6ef9d40 2024-09-19T06:05:50.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.600+0000 7f08baffd640 1 -- 172.21.15.5:0/210186890 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f08d40335b0 con 0x564ed6ef9d40 2024-09-19T06:05:50.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.600+0000 7f08e42e0740 1 -- 172.21.15.5:0/210186890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d00bc9d0 con 0x564ed6ef9d40 2024-09-19T06:05:50.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.600+0000 7f08baffd640 1 -- 172.21.15.5:0/210186890 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f08d4022a10 con 0x564ed6ef9d40 2024-09-19T06:05:50.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.600+0000 7f08e42e0740 10 client.25567 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:50.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.600+0000 7f08e42e0740 1 -- 172.21.15.5:0/210186890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564ed6f74a00 con 0x564ed6ef9d40 2024-09-19T06:05:50.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.600+0000 7f08baffd640 1 -- 172.21.15.5:0/210186890 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08d00bc9d0 con 0x564ed6ef9d40 2024-09-19T06:05:50.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.600+0000 7f08baffd640 1 -- 172.21.15.5:0/210186890 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f08d4004d80 con 0x564ed6ef9d40 2024-09-19T06:05:50.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.600+0000 7f08e42e0740 10 client.25567 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:50.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.601+0000 7f08e42e0740 10 client.25567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:50.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.601+0000 7f08e42e0740 20 client.25567 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:50.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.601+0000 7f08e42e0740 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564ed6f75200 0x564ed6f77620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:50.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.601+0000 7f08e42e0740 4 client.25567 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:05:50.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.601+0000 7f08e42e0740 1 -- 172.21.15.5:0/210186890 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564ed6e5bd00 con 0x564ed6f75200 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.601+0000 7f08db7fe640 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564ed6f75200 0x564ed6f77620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.601+0000 7f08db7fe640 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564ed6f75200 0x564ed6f77620 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca9cb54cc2380a server_cookie=c22ba83bf7b8ad01 in_seq=0 out_seq=0 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.601+0000 7f08baffd640 10 client.25567 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.602+0000 7f08baffd640 1 -- 172.21.15.5:0/210186890 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564ed6e5bd00 con 0x564ed6f75200 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.602+0000 7f08baffd640 10 client.25567 handle_command_reply: tid=0 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.602+0000 7f08e42e0740 1 client.25567 shutdown 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.603+0000 7f08e42e0740 1 -- 172.21.15.5:0/210186890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 msgr2=0x564ed6efe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.603+0000 7f08e42e0740 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 0x564ed6efe510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f08d4000f80 tx=0x7f08d4009580 comp rx=0 tx=0).stop 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.603+0000 7f08e42e0740 1 -- 172.21.15.5:0/210186890 shutdown_connections 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.603+0000 7f08e42e0740 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564ed6f75200 0x564ed6f77620 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.603+0000 7f08e42e0740 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564ed6ef9d40 0x564ed6efe510 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.603+0000 7f08baffd640 0 client.25567 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:50.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.603+0000 7f08e42e0740 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ed6eef480 0x564ed6efdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.603+0000 7f08e42e0740 1 --2- 172.21.15.5:0/210186890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564ed6eeeb60 0x564ed6efda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.603+0000 7f08e42e0740 1 -- 172.21.15.5:0/210186890 >> 172.21.15.5:0/210186890 conn(0x564ed6eeb0f0 msgr2=0x564ed6eea270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:50.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.603+0000 7f08e42e0740 1 -- 172.21.15.5:0/210186890 shutdown_connections 2024-09-19T06:05:50.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.603+0000 7f08e42e0740 1 -- 172.21.15.5:0/210186890 wait complete. 2024-09-19T06:05:50.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.604+0000 7f08e42e0740 20 client.25567 trim_cache size 0 max 16384 2024-09-19T06:05:50.610 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:05:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:05:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "94089cf2-3b97-42ce-a701-63304bd762c8", 2024-09-19T06:05:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:05:50.611 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:05:50.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.604+0000 7f08e1f07640 1 -- 172.21.15.5:0/1654877988 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f08c40a33b0 msgr2=0x7f08c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:50.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.604+0000 7f08e1f07640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f08c40a33b0 0x7f08c40a5870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f08d40094d0 tx=0x7f08d4004ab0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.604+0000 7f08e1f07640 1 -- 172.21.15.5:0/1654877988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08dc102780 msgr2=0x7f08dc19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:50.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.604+0000 7f08e1f07640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08dc102780 0x7f08dc19abb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f08d002a930 tx=0x7f08d0031a20 comp rx=0 tx=0).stop 2024-09-19T06:05:50.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.604+0000 7f08e1f07640 1 -- 172.21.15.5:0/1654877988 shutdown_connections 2024-09-19T06:05:50.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.604+0000 7f08e1f07640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08dc106f00 0x7f08dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.604+0000 7f08e1f07640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f08c40a33b0 0x7f08c40a5870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.604+0000 7f08e1f07640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08dc102780 0x7f08dc19abb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.605+0000 7f08e1f07640 1 --2- 172.21.15.5:0/1654877988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08dc101dd0 0x7f08dc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:50.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.605+0000 7f08e1f07640 1 -- 172.21.15.5:0/1654877988 >> 172.21.15.5:0/1654877988 conn(0x7f08dc0fd8b0 msgr2=0x7f08dc0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:50.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.605+0000 7f08e1f07640 1 -- 172.21.15.5:0/1654877988 shutdown_connections 2024-09-19T06:05:50.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:50.605+0000 7f08e1f07640 1 -- 172.21.15.5:0/1654877988 wait complete. 2024-09-19T06:05:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:50 smithi157 ceph-mon[35346]: pgmap v77: 97 pgs: 97 active+clean; 38 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-19T06:05:50.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 39976 3.05 MB/sec warmup 76 sec latency 14.709 ms 2024-09-19T06:05:50.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:50 smithi005 ceph-mon[30955]: pgmap v77: 97 pgs: 97 active+clean; 38 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-19T06:05:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:50 smithi049 ceph-mon[34640]: pgmap v77: 97 pgs: 97 active+clean; 38 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-19T06:05:51.454 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:51 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:05:51.454 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:51 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:51.454 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:51 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:51.454 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:51 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:51.484 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:05:51.764 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:05:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:51 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:05:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:51 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:51 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:51 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:51.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 40526 3.07 MB/sec warmup 77 sec latency 15.973 ms 2024-09-19T06:05:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:51 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:05:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:51 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:51 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:51 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.245+0000 7fab85cdc640 1 -- 172.21.15.5:0/261791239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab80102d90 msgr2=0x7fab80103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.245+0000 7fab85cdc640 1 --2- 172.21.15.5:0/261791239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab80102d90 0x7fab80103170 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fab70009930 tx=0x7fab7002a420 comp rx=0 tx=0).stop 2024-09-19T06:05:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.245+0000 7fab85cdc640 1 -- 172.21.15.5:0/261791239 shutdown_connections 2024-09-19T06:05:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.245+0000 7fab85cdc640 1 --2- 172.21.15.5:0/261791239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab80108290 0x7fab8010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.245+0000 7fab85cdc640 1 --2- 172.21.15.5:0/261791239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80103740 0x7fab80107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.245+0000 7fab85cdc640 1 --2- 172.21.15.5:0/261791239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab80102d90 0x7fab80103170 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.245+0000 7fab85cdc640 1 -- 172.21.15.5:0/261791239 >> 172.21.15.5:0/261791239 conn(0x7fab800fc7d0 msgr2=0x7fab800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.245+0000 7fab85cdc640 1 -- 172.21.15.5:0/261791239 shutdown_connections 2024-09-19T06:05:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.245+0000 7fab85cdc640 1 -- 172.21.15.5:0/261791239 wait complete. 2024-09-19T06:05:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.246+0000 7fab85cdc640 1 Processor -- start 2024-09-19T06:05:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.246+0000 7fab85cdc640 1 -- start start 2024-09-19T06:05:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.246+0000 7fab85cdc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80102d90 0x7fab80198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.246+0000 7fab85cdc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab80103740 0x7fab80198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.246+0000 7fab85cdc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab80108290 0x7fab8019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.246+0000 7fab85cdc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab80111f40 con 0x7fab80103740 2024-09-19T06:05:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.246+0000 7fab85cdc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab80111dc0 con 0x7fab80108290 2024-09-19T06:05:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.246+0000 7fab85cdc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab801120c0 con 0x7fab80102d90 2024-09-19T06:05:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.246+0000 7fab7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab80103740 0x7fab80198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.246+0000 7fab7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab80103740 0x7fab80198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46330/0 (socket says 172.21.15.5:46330) 2024-09-19T06:05:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.246+0000 7fab7effd640 1 -- 172.21.15.5:0/3979430150 learned_addr learned my addr 172.21.15.5:0/3979430150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.246+0000 7fab7ffff640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab80108290 0x7fab8019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.247+0000 7fab7effd640 1 -- 172.21.15.5:0/3979430150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80102d90 msgr2=0x7fab80198500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.247+0000 7fab7effd640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80102d90 0x7fab80198500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.247+0000 7fab7effd640 1 -- 172.21.15.5:0/3979430150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab80108290 msgr2=0x7fab8019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.247+0000 7fab7effd640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab80108290 0x7fab8019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.247+0000 7fab7effd640 1 -- 172.21.15.5:0/3979430150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8019d4b0 con 0x7fab80103740 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.247+0000 7fab7ffff640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab80108290 0x7fab8019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.247+0000 7fab7effd640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab80103740 0x7fab80198a40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fab7400e9f0 tx=0x7fab7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.247+0000 7fab7cff9640 1 -- 172.21.15.5:0/3979430150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7400ccc0 con 0x7fab80103740 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.247+0000 7fab7cff9640 1 -- 172.21.15.5:0/3979430150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab740054d0 con 0x7fab80103740 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.247+0000 7fab7cff9640 1 -- 172.21.15.5:0/3979430150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74010430 con 0x7fab80103740 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.247+0000 7fab85cdc640 1 -- 172.21.15.5:0/3979430150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8019d7a0 con 0x7fab80103740 2024-09-19T06:05:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.247+0000 7fab85cdc640 1 -- 172.21.15.5:0/3979430150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab801a5080 con 0x7fab80103740 2024-09-19T06:05:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.249+0000 7fab7cff9640 1 -- 172.21.15.5:0/3979430150 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fab74005020 con 0x7fab80103740 2024-09-19T06:05:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab880b5740 1 Processor -- start 2024-09-19T06:05:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab880b5740 1 -- start start 2024-09-19T06:05:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab880b5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfeaf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab880b5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeafca0 0x5627bfebe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab880b5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bfec0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab880b5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5627bfe152e0 con 0x5627bfebea90 2024-09-19T06:05:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab880b5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5627bfdbcce0 con 0x5627bfeafca0 2024-09-19T06:05:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab880b5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5627bfe112a0 con 0x5627bfeaf380 2024-09-19T06:05:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab7ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfeaf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab7ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfeaf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59590/0 (socket says 172.21.15.5:59590) 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab7ffff640 1 -- 172.21.15.5:0/1642564103 learned_addr learned my addr 172.21.15.5:0/1642564103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab7f7fe640 1 --2- 172.21.15.5:0/1642564103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeafca0 0x5627bfebe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab7effd640 1 --2- 172.21.15.5:0/1642564103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bfec0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab7cff9640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fab680a3430 0x7fab680a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab7ffff640 1 -- 172.21.15.5:0/1642564103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeafca0 msgr2=0x5627bfebe550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab7ffff640 1 --2- 172.21.15.5:0/1642564103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeafca0 0x5627bfebe550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.250+0000 7fab7ffff640 1 -- 172.21.15.5:0/1642564103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 msgr2=0x5627bfec0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.251+0000 7fab7ffff640 1 --2- 172.21.15.5:0/1642564103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bfec0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.251+0000 7fab7ffff640 1 -- 172.21.15.5:0/1642564103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627bfda1440 con 0x5627bfeaf380 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.251+0000 7fab7effd640 1 --2- 172.21.15.5:0/1642564103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bfec0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.251+0000 7fab7cff9640 1 -- 172.21.15.5:0/3979430150 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fab740d00e0 con 0x7fab80103740 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.251+0000 7fab7ffff640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fab680a3430 0x7fab680a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.251+0000 7fab7ffff640 1 --2- 172.21.15.5:0/1642564103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfeaf760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab78011790 tx=0x7fab78011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c95273b2e30d69bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.251+0000 7fab65ffb640 1 -- 172.21.15.5:0/1642564103 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7800fba0 con 0x5627bfeaf380 2024-09-19T06:05:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.251+0000 7fab65ffb640 1 -- 172.21.15.5:0/1642564103 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab78019930 con 0x5627bfeaf380 2024-09-19T06:05:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.251+0000 7fab65ffb640 1 -- 172.21.15.5:0/1642564103 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7800eb50 con 0x5627bfeaf380 2024-09-19T06:05:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.252+0000 7fab880b5740 1 -- 172.21.15.5:0/1642564103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 msgr2=0x5627bfeaf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.252+0000 7fab7ffff640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fab680a3430 0x7fab680a58f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab7801d000 tx=0x7fab7800edb0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.252+0000 7fab880b5740 1 --2- 172.21.15.5:0/1642564103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfeaf760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab78011790 tx=0x7fab78011c50 comp rx=0 tx=0).stop 2024-09-19T06:05:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.252+0000 7fab880b5740 1 -- 172.21.15.5:0/1642564103 shutdown_connections 2024-09-19T06:05:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.252+0000 7fab880b5740 1 --2- 172.21.15.5:0/1642564103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bfec0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.252+0000 7fab880b5740 1 --2- 172.21.15.5:0/1642564103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeafca0 0x5627bfebe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.252+0000 7fab880b5740 1 --2- 172.21.15.5:0/1642564103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfeaf760 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.252+0000 7fab880b5740 1 -- 172.21.15.5:0/1642564103 >> 172.21.15.5:0/1642564103 conn(0x5627bfeaba30 msgr2=0x5627bfeabe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.252+0000 7fab880b5740 1 -- 172.21.15.5:0/1642564103 shutdown_connections 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.252+0000 7fab880b5740 1 -- 172.21.15.5:0/1642564103 wait complete. 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.252+0000 7fab880b5740 1 -- start start 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab880b5740 1 Processor -- start 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab880b5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfebe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab880b5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfeafca0 0x5627bfebb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab880b5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfebea90 0x5627bfebbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab880b5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5627bfec1660 con 0x5627bfeafca0 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab880b5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5627bfdd0050 con 0x5627bfebea90 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab880b5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5627bfec17e0 con 0x5627bfeaf380 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfeafca0 0x5627bfebb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab7ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfebea90 0x5627bfebbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab7ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfebea90 0x5627bfebbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:38956/0 (socket says 172.21.15.5:38956) 2024-09-19T06:05:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab880b5740 10 client.0 _ll_register_callbacks cb 0x5627bfe10160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfeafca0 0x5627bfebb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46356/0 (socket says 172.21.15.5:46356) 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab7ffff640 1 -- 172.21.15.5:0/3735931807 learned_addr learned my addr 172.21.15.5:0/3735931807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab7f7fe640 1 --2- 172.21.15.5:0/3735931807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfebe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab7ffff640 1 -- 172.21.15.5:0/3735931807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 msgr2=0x5627bfebe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab7ffff640 1 --2- 172.21.15.5:0/3735931807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfebe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab7ffff640 1 -- 172.21.15.5:0/3735931807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfeafca0 msgr2=0x5627bfebb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab7ffff640 1 --2- 172.21.15.5:0/3735931807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfeafca0 0x5627bfebb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.253+0000 7fab7ffff640 1 -- 172.21.15.5:0/3735931807 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627bfb863c0 con 0x5627bfebea90 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.254+0000 7fab7ffff640 1 --2- 172.21.15.5:0/3735931807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfebea90 0x5627bfebbc00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fab7800fc90 tx=0x7fab78017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8159e904e63cf906 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.254+0000 7fab5effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.254+0000 7fab5effd640 1 -- 172.21.15.5:0/3735931807 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78019610 con 0x5627bfebea90 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.254+0000 7fab880b5740 1 -- 172.21.15.5:0/3735931807 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab78019930 con 0x5627bfebea90 2024-09-19T06:05:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.254+0000 7fab5effd640 1 -- 172.21.15.5:0/3735931807 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fab7802b040 con 0x5627bfebea90 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.254+0000 7fab5effd640 1 -- 172.21.15.5:0/3735931807 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fab7803e050 con 0x5627bfebea90 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.254+0000 7fab880b5740 10 client.25643 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.254+0000 7fab880b5740 1 -- 172.21.15.5:0/3735931807 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627bfecd0c0 con 0x5627bfebea90 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.254+0000 7fab5effd640 1 -- 172.21.15.5:0/3735931807 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab78019930 con 0x5627bfebea90 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.255+0000 7fab5effd640 1 -- 172.21.15.5:0/3735931807 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fab780111d0 con 0x5627bfebea90 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.255+0000 7fab880b5740 10 client.25643 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.255+0000 7fab880b5740 10 client.25643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.255+0000 7fab880b5740 20 client.25643 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.255+0000 7fab880b5740 1 --2- 172.21.15.5:0/3735931807 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5627bfece4d0 0x5627bfed08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.255+0000 7fab880b5740 4 client.25643 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.255+0000 7fab880b5740 1 -- 172.21.15.5:0/3735931807 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627bfe22850 con 0x5627bfece4d0 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.255+0000 7fab7f7fe640 1 --2- 172.21.15.5:0/3735931807 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5627bfece4d0 0x5627bfed08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab5effd640 10 client.25643 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab7f7fe640 1 --2- 172.21.15.5:0/3735931807 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5627bfece4d0 0x5627bfed08c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed327fbd680da4c server_cookie=ef558473f1836d2a in_seq=0 out_seq=0 2024-09-19T06:05:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab5effd640 1 -- 172.21.15.5:0/3735931807 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627bfe22850 con 0x5627bfece4d0 2024-09-19T06:05:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab5effd640 10 client.25643 handle_command_reply: tid=0 2024-09-19T06:05:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab880b5740 1 client.25643 shutdown 2024-09-19T06:05:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab880b5740 1 -- 172.21.15.5:0/3735931807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfebea90 msgr2=0x5627bfebbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab880b5740 1 --2- 172.21.15.5:0/3735931807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfebea90 0x5627bfebbc00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fab7800fc90 tx=0x7fab78017000 comp rx=0 tx=0).stop 2024-09-19T06:05:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab880b5740 1 -- 172.21.15.5:0/3735931807 shutdown_connections 2024-09-19T06:05:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab880b5740 1 --2- 172.21.15.5:0/3735931807 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5627bfece4d0 0x5627bfed08c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab880b5740 1 --2- 172.21.15.5:0/3735931807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfebea90 0x5627bfebbc00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab880b5740 1 --2- 172.21.15.5:0/3735931807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfeafca0 0x5627bfebb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab880b5740 1 --2- 172.21.15.5:0/3735931807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfebe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab880b5740 1 -- 172.21.15.5:0/3735931807 >> 172.21.15.5:0/3735931807 conn(0x5627bfeaba30 msgr2=0x5627bfeb1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.256+0000 7fab5effd640 0 client.25643 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:52.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.257+0000 7fab880b5740 1 -- 172.21.15.5:0/3735931807 shutdown_connections 2024-09-19T06:05:52.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.257+0000 7fab880b5740 1 -- 172.21.15.5:0/3735931807 wait complete. 2024-09-19T06:05:52.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.257+0000 7fab880b5740 20 client.25643 trim_cache size 0 max 16384 2024-09-19T06:05:52.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.277+0000 7fab880b5740 1 Processor -- start 2024-09-19T06:05:52.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.277+0000 7fab880b5740 1 -- start start 2024-09-19T06:05:52.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.277+0000 7fab880b5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeaf380 0x5627bfeca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.277+0000 7fab880b5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeafca0 0x5627bfed6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.277+0000 7fab880b5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bfed6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.277+0000 7fab880b5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab801ab6d0 con 0x5627bfebea90 2024-09-19T06:05:52.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.277+0000 7fab880b5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab801ad270 con 0x5627bfeaf380 2024-09-19T06:05:52.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.277+0000 7fab880b5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab801abda0 con 0x5627bfeafca0 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.277+0000 7fab7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bfed6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bfed6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46366/0 (socket says 172.21.15.5:46366) 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab7f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeaf380 0x5627bfeca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab7ffff640 1 -- 172.21.15.5:0/2514390840 learned_addr learned my addr 172.21.15.5:0/2514390840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab7effd640 1 --2- 172.21.15.5:0/2514390840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeafca0 0x5627bfed6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab7ffff640 1 -- 172.21.15.5:0/2514390840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeafca0 msgr2=0x5627bfed6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab7ffff640 1 --2- 172.21.15.5:0/2514390840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeafca0 0x5627bfed6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab7ffff640 1 -- 172.21.15.5:0/2514390840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeaf380 msgr2=0x5627bfeca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab7ffff640 1 --2- 172.21.15.5:0/2514390840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeaf380 0x5627bfeca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab7ffff640 1 -- 172.21.15.5:0/2514390840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627bfeab4d0 con 0x5627bfebea90 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab7ffff640 1 --2- 172.21.15.5:0/2514390840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bfed6660 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fab78002230 tx=0x7fab780094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5e5b743c13ce66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab657fa640 1 -- 172.21.15.5:0/2514390840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7800f790 con 0x5627bfebea90 2024-09-19T06:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab657fa640 1 -- 172.21.15.5:0/2514390840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab78035070 con 0x5627bfebea90 2024-09-19T06:05:52.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.278+0000 7fab657fa640 1 -- 172.21.15.5:0/2514390840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7800e3b0 con 0x5627bfebea90 2024-09-19T06:05:52.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.279+0000 7fab880b5740 1 -- 172.21.15.5:0/2514390840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 msgr2=0x5627bfed6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:52.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.279+0000 7fab880b5740 1 --2- 172.21.15.5:0/2514390840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bfed6660 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fab78002230 tx=0x7fab780094e0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.279+0000 7fab880b5740 1 -- 172.21.15.5:0/2514390840 shutdown_connections 2024-09-19T06:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.279+0000 7fab880b5740 1 --2- 172.21.15.5:0/2514390840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bfed6660 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.279+0000 7fab880b5740 1 --2- 172.21.15.5:0/2514390840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeafca0 0x5627bfed6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.279+0000 7fab880b5740 1 --2- 172.21.15.5:0/2514390840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeaf380 0x5627bfeca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.279+0000 7fab880b5740 1 -- 172.21.15.5:0/2514390840 >> 172.21.15.5:0/2514390840 conn(0x5627bfeaba30 msgr2=0x5627bfeb6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.279+0000 7fab880b5740 1 -- 172.21.15.5:0/2514390840 shutdown_connections 2024-09-19T06:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.279+0000 7fab880b5740 1 -- 172.21.15.5:0/2514390840 wait complete. 2024-09-19T06:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.279+0000 7fab880b5740 1 -- start start 2024-09-19T06:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab880b5740 1 Processor -- start 2024-09-19T06:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab880b5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfec1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab880b5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeafca0 0x5627bfec2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab880b5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bff370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab880b5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab801acb40 con 0x5627bfebea90 2024-09-19T06:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab880b5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab801ac470 con 0x5627bfeafca0 2024-09-19T06:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab880b5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab801a96b0 con 0x5627bfeaf380 2024-09-19T06:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab880b5740 10 client.0 _ll_register_callbacks cb 0x7fab78028070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab7effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeafca0 0x5627bfec2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab7f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfec1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bff370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab7effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeafca0 0x5627bfec2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:38974/0 (socket says 172.21.15.5:38974) 2024-09-19T06:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab7f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfec1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59626/0 (socket says 172.21.15.5:59626) 2024-09-19T06:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.280+0000 7fab7effd640 1 -- 172.21.15.5:0/1309438969 learned_addr learned my addr 172.21.15.5:0/1309438969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab7effd640 1 -- 172.21.15.5:0/1309438969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 msgr2=0x5627bfec1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab7effd640 1 --2- 172.21.15.5:0/1309438969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfec1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab7effd640 1 -- 172.21.15.5:0/1309438969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 msgr2=0x5627bff370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab7effd640 1 --2- 172.21.15.5:0/1309438969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bff370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab7effd640 1 -- 172.21.15.5:0/1309438969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627bff37620 con 0x5627bfeafca0 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab7f7fe640 1 --2- 172.21.15.5:0/1309438969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfec1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab7effd640 1 --2- 172.21.15.5:0/1309438969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeafca0 0x5627bfec2330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab74004ab0 tx=0x7fab7409bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459da053c89da8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab5effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab5effd640 1 -- 172.21.15.5:0/1309438969 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab740046c0 con 0x5627bfeafca0 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab5effd640 1 -- 172.21.15.5:0/1309438969 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fab740a4bb0 con 0x5627bfeafca0 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab880b5740 1 -- 172.21.15.5:0/1309438969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab78035070 con 0x5627bfeafca0 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab5effd640 1 -- 172.21.15.5:0/1309438969 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fab74005310 con 0x5627bfeafca0 2024-09-19T06:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab880b5740 10 client.25649 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.281+0000 7fab880b5740 1 -- 172.21.15.5:0/1309438969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627bff387a0 con 0x5627bfeafca0 2024-09-19T06:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.282+0000 7fab5effd640 1 -- 172.21.15.5:0/1309438969 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab78035070 con 0x5627bfeafca0 2024-09-19T06:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.282+0000 7fab5effd640 1 -- 172.21.15.5:0/1309438969 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fab740b5250 con 0x5627bfeafca0 2024-09-19T06:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.282+0000 7fab880b5740 10 client.25649 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.282+0000 7fab880b5740 10 client.25649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.282+0000 7fab880b5740 20 client.25649 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.282+0000 7fab880b5740 1 --2- 172.21.15.5:0/1309438969 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5627bff38f00 0x5627bff3b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.282+0000 7fab880b5740 4 client.25649 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.282+0000 7fab880b5740 1 -- 172.21.15.5:0/1309438969 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5627bfe1cd60 con 0x5627bff38f00 2024-09-19T06:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.282+0000 7fab7f7fe640 1 --2- 172.21.15.5:0/1309438969 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5627bff38f00 0x5627bff3b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.282+0000 7fab7f7fe640 1 --2- 172.21.15.5:0/1309438969 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5627bff38f00 0x5627bff3b320 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e540ba4ae40db653 server_cookie=8c30ffc999106817 in_seq=0 out_seq=0 2024-09-19T06:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.282+0000 7fab5effd640 10 client.25649 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab5effd640 1 -- 172.21.15.5:0/1309438969 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5627bfe1cd60 con 0x5627bff38f00 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab5effd640 10 client.25649 handle_command_reply: tid=0 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab880b5740 1 client.25649 shutdown 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab880b5740 1 -- 172.21.15.5:0/1309438969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeafca0 msgr2=0x5627bfec2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab880b5740 1 --2- 172.21.15.5:0/1309438969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeafca0 0x5627bfec2330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab74004ab0 tx=0x7fab7409bd30 comp rx=0 tx=0).stop 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab880b5740 1 -- 172.21.15.5:0/1309438969 shutdown_connections 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab880b5740 1 --2- 172.21.15.5:0/1309438969 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5627bff38f00 0x5627bff3b320 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab880b5740 1 --2- 172.21.15.5:0/1309438969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627bfebea90 0x5627bff370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab880b5740 1 --2- 172.21.15.5:0/1309438969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627bfeafca0 0x5627bfec2330 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab880b5740 1 --2- 172.21.15.5:0/1309438969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627bfeaf380 0x5627bfec1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab880b5740 1 -- 172.21.15.5:0/1309438969 >> 172.21.15.5:0/1309438969 conn(0x5627bfeaba30 msgr2=0x5627bfeb9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab5effd640 0 client.25649 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab880b5740 1 -- 172.21.15.5:0/1309438969 shutdown_connections 2024-09-19T06:05:52.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab880b5740 1 -- 172.21.15.5:0/1309438969 wait complete. 2024-09-19T06:05:52.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.283+0000 7fab880b5740 20 client.25649 trim_cache size 0 max 16384 2024-09-19T06:05:52.291 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:05:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:05:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:05:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "94089cf2-3b97-42ce-a701-63304bd762c8": { 2024-09-19T06:05:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:05:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "94089cf2-3b97-42ce-a701-63304bd762c8", 2024-09-19T06:05:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:05:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:05:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:05:52.292 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:05:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.284+0000 7fab85cdc640 1 -- 172.21.15.5:0/3979430150 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fab680a3430 msgr2=0x7fab680a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.284+0000 7fab85cdc640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fab680a3430 0x7fab680a58f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab7801d000 tx=0x7fab7800edb0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.284+0000 7fab85cdc640 1 -- 172.21.15.5:0/3979430150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab80103740 msgr2=0x7fab80198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.284+0000 7fab85cdc640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab80103740 0x7fab80198a40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fab7400e9f0 tx=0x7fab7400edf0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.284+0000 7fab85cdc640 1 -- 172.21.15.5:0/3979430150 shutdown_connections 2024-09-19T06:05:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.284+0000 7fab85cdc640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab80108290 0x7fab8019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.284+0000 7fab85cdc640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab80103740 0x7fab80198a40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.284+0000 7fab85cdc640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fab680a3430 0x7fab680a58f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.284+0000 7fab85cdc640 1 --2- 172.21.15.5:0/3979430150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80102d90 0x7fab80198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.284+0000 7fab85cdc640 1 -- 172.21.15.5:0/3979430150 >> 172.21.15.5:0/3979430150 conn(0x7fab800fc7d0 msgr2=0x7fab80105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.284+0000 7fab85cdc640 1 -- 172.21.15.5:0/3979430150 shutdown_connections 2024-09-19T06:05:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:05:52.284+0000 7fab85cdc640 1 -- 172.21.15.5:0/3979430150 wait complete. 2024-09-19T06:05:52.797 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:52.798 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.798 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:52.798 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:52.798 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.798 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.798 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:52.798 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.798 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:52.798 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.798 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:05:52.798 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.798 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:52 smithi049 ceph-mon[34640]: pgmap v78: 97 pgs: 97 active+clean; 38 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-09-19T06:05:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:05:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:52 smithi157 ceph-mon[35346]: pgmap v78: 97 pgs: 97 active+clean; 38 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-09-19T06:05:52.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 41120 3.04 MB/sec warmup 78 sec latency 28.532 ms 2024-09-19T06:05:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:05:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:05:52.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:52 smithi005 ceph-mon[30955]: pgmap v78: 97 pgs: 97 active+clean; 38 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-09-19T06:05:53.069 INFO:tasks.cephfs.filesystem:scrub status for tag:94089cf2-3b97-42ce-a701-63304bd762c8 - {'path': '/', 'tag': '94089cf2-3b97-42ce-a701-63304bd762c8', 'options': 'recursive,force'} 2024-09-19T06:05:53.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 41653 3.03 MB/sec warmup 79 sec latency 15.404 ms 2024-09-19T06:05:54.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:54 smithi049 ceph-mon[34640]: pgmap v79: 97 pgs: 97 active+clean; 42 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-09-19T06:05:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:54 smithi157 ceph-mon[35346]: pgmap v79: 97 pgs: 97 active+clean; 42 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-09-19T06:05:54.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 42370 3.07 MB/sec warmup 80 sec latency 14.860 ms 2024-09-19T06:05:54.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:54 smithi005 ceph-mon[30955]: pgmap v79: 97 pgs: 97 active+clean; 42 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-09-19T06:05:55.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 42843 3.09 MB/sec warmup 81 sec latency 15.771 ms 2024-09-19T06:05:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.636+0000 7fc2a70af640 1 -- 172.21.15.5:0/1233505675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2a0101ad0 msgr2=0x7fc2a0101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.636+0000 7fc2a70af640 1 --2- 172.21.15.5:0/1233505675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2a0101ad0 0x7fc2a0101f30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc288009930 tx=0x7fc28802a420 comp rx=0 tx=0).stop 2024-09-19T06:05:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.636+0000 7fc2a70af640 1 -- 172.21.15.5:0/1233505675 shutdown_connections 2024-09-19T06:05:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.636+0000 7fc2a70af640 1 --2- 172.21.15.5:0/1233505675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a0102470 0x7fc2a010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.636+0000 7fc2a70af640 1 --2- 172.21.15.5:0/1233505675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2a0101ad0 0x7fc2a0101f30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.636+0000 7fc2a70af640 1 --2- 172.21.15.5:0/1233505675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a0107b70 0x7fc2a0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.636+0000 7fc2a70af640 1 -- 172.21.15.5:0/1233505675 >> 172.21.15.5:0/1233505675 conn(0x7fc2a00fd7d0 msgr2=0x7fc2a00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.636+0000 7fc2a70af640 1 -- 172.21.15.5:0/1233505675 shutdown_connections 2024-09-19T06:05:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.636+0000 7fc2a70af640 1 -- 172.21.15.5:0/1233505675 wait complete. 2024-09-19T06:05:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.637+0000 7fc2a70af640 1 Processor -- start 2024-09-19T06:05:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.637+0000 7fc2a70af640 1 -- start start 2024-09-19T06:05:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.637+0000 7fc2a70af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2a0101ad0 0x7fc2a01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.637+0000 7fc2a70af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a0102470 0x7fc2a01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.637+0000 7fc2a70af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a0107b70 0x7fc2a019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.637+0000 7fc2a70af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a0111e20 con 0x7fc2a0101ad0 2024-09-19T06:05:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.637+0000 7fc2a70af640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a0111ca0 con 0x7fc2a0102470 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.637+0000 7fc2a70af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a0111fa0 con 0x7fc2a0107b70 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.637+0000 7fc2a4e4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2a0101ad0 0x7fc2a01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc29ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a0102470 0x7fc2a01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.637+0000 7fc2a4e4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2a0101ad0 0x7fc2a01983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45848/0 (socket says 172.21.15.5:45848) 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/3599436858 learned_addr learned my addr 172.21.15.5:0/3599436858 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc2a564c640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a0107b70 0x7fc2a019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/3599436858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a0107b70 msgr2=0x7fc2a019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc2a4e4b640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a0107b70 0x7fc2a019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/3599436858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a0102470 msgr2=0x7fc2a01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc2a4e4b640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a0102470 0x7fc2a01988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/3599436858 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a019d3a0 con 0x7fc2a0101ad0 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc2a564c640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a0107b70 0x7fc2a019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:05:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc2a4e4b640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2a0101ad0 0x7fc2a01983b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc29000e9f0 tx=0x7fc29000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc29dffb640 1 -- 172.21.15.5:0/3599436858 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29000cd80 con 0x7fc2a0101ad0 2024-09-19T06:05:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc29dffb640 1 -- 172.21.15.5:0/3599436858 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2900054d0 con 0x7fc2a0101ad0 2024-09-19T06:05:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc2a70af640 1 -- 172.21.15.5:0/3599436858 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a019d690 con 0x7fc2a0101ad0 2024-09-19T06:05:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc29dffb640 1 -- 172.21.15.5:0/3599436858 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290010490 con 0x7fc2a0101ad0 2024-09-19T06:05:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.638+0000 7fc2a70af640 1 -- 172.21.15.5:0/3599436858 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a01a4f20 con 0x7fc2a0101ad0 2024-09-19T06:05:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.641+0000 7fc29dffb640 1 -- 172.21.15.5:0/3599436858 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc290005020 con 0x7fc2a0101ad0 2024-09-19T06:05:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc2a9e42740 1 Processor -- start 2024-09-19T06:05:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc2a9e42740 1 -- start start 2024-09-19T06:05:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc2a9e42740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec158aaf0 0x556ec158aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc2a9e42740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 0x556ec1595770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc2a9e42740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec1595cb0 0x556ec15980a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc2a9e42740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ec14ef2f0 con 0x556ec158b410 2024-09-19T06:05:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc2a9e42740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556ec1497270 con 0x556ec158aaf0 2024-09-19T06:05:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc2a9e42740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556ec14eb2b0 con 0x556ec1595cb0 2024-09-19T06:05:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc2a4e4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 0x556ec1595770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc29ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec1595cb0 0x556ec15980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc2a4e4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 0x556ec1595770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45856/0 (socket says 172.21.15.5:45856) 2024-09-19T06:05:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/2103736525 learned_addr learned my addr 172.21.15.5:0/2103736525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc2a564c640 1 --2- 172.21.15.5:0/2103736525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec158aaf0 0x556ec158aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.642+0000 7fc29dffb640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc2800a33b0 0x7fc2800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.643+0000 7fc2a564c640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc2800a33b0 0x7fc2800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.643+0000 7fc29ffff640 1 -- 172.21.15.5:0/2103736525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec158aaf0 msgr2=0x556ec158aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.643+0000 7fc29ffff640 1 --2- 172.21.15.5:0/2103736525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec158aaf0 0x556ec158aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.643+0000 7fc29ffff640 1 -- 172.21.15.5:0/2103736525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 msgr2=0x556ec1595770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.643+0000 7fc29dffb640 1 -- 172.21.15.5:0/3599436858 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fc2900d0080 con 0x7fc2a0101ad0 2024-09-19T06:05:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.643+0000 7fc29ffff640 1 --2- 172.21.15.5:0/2103736525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 0x556ec1595770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.643+0000 7fc29ffff640 1 -- 172.21.15.5:0/2103736525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec1496340 con 0x556ec1595cb0 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.643+0000 7fc29ffff640 1 --2- 172.21.15.5:0/2103736525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec1595cb0 0x556ec15980a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc28802a940 tx=0x7fc2880040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee49c2181fa8297f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.643+0000 7fc2a564c640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc2800a33b0 0x7fc2800a5870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc298009fd0 tx=0x7fc298004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.643+0000 7fc296ffd640 1 -- 172.21.15.5:0/2103736525 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288035040 con 0x556ec1595cb0 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.643+0000 7fc296ffd640 1 -- 172.21.15.5:0/2103736525 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2880054f0 con 0x556ec1595cb0 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.643+0000 7fc296ffd640 1 -- 172.21.15.5:0/2103736525 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2880035f0 con 0x556ec1595cb0 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.644+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2103736525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec1595cb0 msgr2=0x556ec15980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.644+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/2103736525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec1595cb0 0x556ec15980a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc28802a940 tx=0x7fc2880040c0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.644+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2103736525 shutdown_connections 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.644+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/2103736525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec1595cb0 0x556ec15980a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.644+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/2103736525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 0x556ec1595770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.644+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/2103736525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec158aaf0 0x556ec158aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.644+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2103736525 >> 172.21.15.5:0/2103736525 conn(0x556ec1587060 msgr2=0x556ec1587440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.644+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2103736525 shutdown_connections 2024-09-19T06:05:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.644+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2103736525 wait complete. 2024-09-19T06:05:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.644+0000 7fc2a9e42740 1 -- start start 2024-09-19T06:05:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a9e42740 1 Processor -- start 2024-09-19T06:05:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a9e42740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158aaf0 0x556ec1592b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a9e42740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 0x556ec1593070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a9e42740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec1595cb0 0x556ec15935b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a9e42740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ec1598880 con 0x556ec158b410 2024-09-19T06:05:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a9e42740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556ec1478290 con 0x556ec1595cb0 2024-09-19T06:05:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a9e42740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556ec1598a00 con 0x556ec158aaf0 2024-09-19T06:05:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a9e42740 10 client.0 _ll_register_callbacks cb 0x556ec14a9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a4e4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158aaf0 0x556ec1592b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc29ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 0x556ec1593070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a564c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec1595cb0 0x556ec15935b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a4e4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158aaf0 0x556ec1592b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:37276/0 (socket says 172.21.15.5:37276) 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc29ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 0x556ec1593070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45866/0 (socket says 172.21.15.5:45866) 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/2737770917 learned_addr learned my addr 172.21.15.5:0/2737770917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/2737770917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec1595cb0 msgr2=0x556ec15935b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a4e4b640 1 --2- 172.21.15.5:0/2737770917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec1595cb0 0x556ec15935b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/2737770917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 msgr2=0x556ec1593070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a4e4b640 1 --2- 172.21.15.5:0/2737770917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 0x556ec1593070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.645+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/2737770917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec1202210 con 0x556ec158aaf0 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.646+0000 7fc29ffff640 1 --2- 172.21.15.5:0/2737770917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 0x556ec1593070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.646+0000 7fc2a4e4b640 1 --2- 172.21.15.5:0/2737770917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158aaf0 0x556ec1592b30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc29000c6a0 tx=0x7fc2900d37c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f45864c1dd424f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.646+0000 7fc28ffff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.646+0000 7fc28ffff640 1 -- 172.21.15.5:0/2737770917 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29009bcd0 con 0x556ec158aaf0 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.646+0000 7fc28ffff640 1 -- 172.21.15.5:0/2737770917 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fc2900a4b50 con 0x556ec158aaf0 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.646+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2737770917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2880054f0 con 0x556ec158aaf0 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.646+0000 7fc28ffff640 1 -- 172.21.15.5:0/2737770917 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc290005310 con 0x556ec158aaf0 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.646+0000 7fc2a9e42740 10 client.25594 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.646+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2737770917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ec12ce620 con 0x556ec158aaf0 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.646+0000 7fc28ffff640 1 -- 172.21.15.5:0/2737770917 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2880054f0 con 0x556ec158aaf0 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.646+0000 7fc28ffff640 1 -- 172.21.15.5:0/2737770917 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc2900d20f0 con 0x556ec158aaf0 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.646+0000 7fc2a9e42740 10 client.25594 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.647+0000 7fc2a9e42740 10 client.25594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.647+0000 7fc2a9e42740 20 client.25594 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.647+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/2737770917 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556ec15a5c30 0x556ec15a8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.647+0000 7fc2a9e42740 4 client.25594 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:05:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.647+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2737770917 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ec14fca20 con 0x556ec15a5c30 2024-09-19T06:05:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.647+0000 7fc29ffff640 1 --2- 172.21.15.5:0/2737770917 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556ec15a5c30 0x556ec15a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.647+0000 7fc28ffff640 10 client.25594 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.647+0000 7fc29ffff640 1 --2- 172.21.15.5:0/2737770917 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556ec15a5c30 0x556ec15a8020 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18db902e0a3f9272 server_cookie=26df7bfb990c345f in_seq=0 out_seq=0 2024-09-19T06:05:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc28ffff640 1 -- 172.21.15.5:0/2737770917 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ec14fca20 con 0x556ec15a5c30 2024-09-19T06:05:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc28ffff640 10 client.25594 handle_command_reply: tid=0 2024-09-19T06:05:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc2a9e42740 1 client.25594 shutdown 2024-09-19T06:05:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2737770917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158aaf0 msgr2=0x556ec1592b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/2737770917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158aaf0 0x556ec1592b30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc29000c6a0 tx=0x7fc2900d37c0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2737770917 shutdown_connections 2024-09-19T06:05:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/2737770917 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556ec15a5c30 0x556ec15a8020 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/2737770917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec1595cb0 0x556ec15935b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc28ffff640 0 client.25594 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/2737770917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158b410 0x556ec1593070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/2737770917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158aaf0 0x556ec1592b30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2737770917 >> 172.21.15.5:0/2737770917 conn(0x556ec1587060 msgr2=0x556ec158cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2737770917 shutdown_connections 2024-09-19T06:05:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc2a9e42740 1 -- 172.21.15.5:0/2737770917 wait complete. 2024-09-19T06:05:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.648+0000 7fc2a9e42740 20 client.25594 trim_cache size 0 max 16384 2024-09-19T06:05:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a9e42740 1 Processor -- start 2024-09-19T06:05:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a9e42740 1 -- start start 2024-09-19T06:05:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a9e42740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec158aaf0 0x556ec15933e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a9e42740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158b410 0x556ec15ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a9e42740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec1595cb0 0x556ec15ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a9e42740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a01ad760 con 0x556ec1595cb0 2024-09-19T06:05:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a9e42740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a01af300 con 0x556ec158aaf0 2024-09-19T06:05:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a9e42740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a01ade30 con 0x556ec158b410 2024-09-19T06:05:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a564c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec1595cb0 0x556ec15ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a4e4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec158aaf0 0x556ec15933e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a564c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec1595cb0 0x556ec15ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45878/0 (socket says 172.21.15.5:45878) 2024-09-19T06:05:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a564c640 1 -- 172.21.15.5:0/697429527 learned_addr learned my addr 172.21.15.5:0/697429527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc29ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158b410 0x556ec15ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/697429527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158b410 msgr2=0x556ec15ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a4e4b640 1 --2- 172.21.15.5:0/697429527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158b410 0x556ec15ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/697429527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec1595cb0 msgr2=0x556ec15ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a4e4b640 1 --2- 172.21.15.5:0/697429527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec1595cb0 0x556ec15ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/697429527 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec158b240 con 0x556ec158aaf0 2024-09-19T06:05:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.664+0000 7fc2a564c640 1 --2- 172.21.15.5:0/697429527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec1595cb0 0x556ec15ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.665+0000 7fc2a4e4b640 1 --2- 172.21.15.5:0/697429527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec158aaf0 0x556ec15933e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2900a4ab0 tx=0x7fc29009a880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ef14307a7400fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.665+0000 7fc2967fc640 1 -- 172.21.15.5:0/697429527 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29009bcd0 con 0x556ec158aaf0 2024-09-19T06:05:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.665+0000 7fc2967fc640 1 -- 172.21.15.5:0/697429527 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc290005680 con 0x556ec158aaf0 2024-09-19T06:05:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.665+0000 7fc2967fc640 1 -- 172.21.15.5:0/697429527 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2900a0720 con 0x556ec158aaf0 2024-09-19T06:05:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.666+0000 7fc2a9e42740 1 -- 172.21.15.5:0/697429527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec158aaf0 msgr2=0x556ec15933e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.666+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/697429527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec158aaf0 0x556ec15933e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2900a4ab0 tx=0x7fc29009a880 comp rx=0 tx=0).stop 2024-09-19T06:05:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.666+0000 7fc2a9e42740 1 -- 172.21.15.5:0/697429527 shutdown_connections 2024-09-19T06:05:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.666+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/697429527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec1595cb0 0x556ec15ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.666+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/697429527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158b410 0x556ec15ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.666+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/697429527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec158aaf0 0x556ec15933e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.666+0000 7fc2a9e42740 1 -- 172.21.15.5:0/697429527 >> 172.21.15.5:0/697429527 conn(0x556ec1587060 msgr2=0x556ec1594bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.666+0000 7fc2a9e42740 1 -- 172.21.15.5:0/697429527 shutdown_connections 2024-09-19T06:05:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.666+0000 7fc2a9e42740 1 -- 172.21.15.5:0/697429527 wait complete. 2024-09-19T06:05:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.666+0000 7fc2a9e42740 1 -- start start 2024-09-19T06:05:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.666+0000 7fc2a9e42740 1 Processor -- start 2024-09-19T06:05:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a9e42740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158aaf0 0x556ec159e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a9e42740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158b410 0x556ec1596ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a9e42740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec1595cb0 0x556ec1597010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a9e42740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a01aebd0 con 0x556ec158aaf0 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a9e42740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a01ae500 con 0x556ec1595cb0 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a9e42740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a01ab740 con 0x556ec158b410 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a9e42740 10 client.0 _ll_register_callbacks cb 0x7fc29000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a4e4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158aaf0 0x556ec159e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a4e4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158aaf0 0x556ec159e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45890/0 (socket says 172.21.15.5:45890) 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/4217484866 learned_addr learned my addr 172.21.15.5:0/4217484866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc29ffff640 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158b410 0x556ec1596ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a564c640 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec1595cb0 0x556ec1597010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/4217484866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158b410 msgr2=0x556ec1596ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a4e4b640 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158b410 0x556ec1596ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/4217484866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec1595cb0 msgr2=0x556ec1597010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:05:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a4e4b640 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec1595cb0 0x556ec1597010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a4e4b640 1 -- 172.21.15.5:0/4217484866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec1597860 con 0x556ec158aaf0 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc29ffff640 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158b410 0x556ec1596ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc2a4e4b640 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158aaf0 0x556ec159e3f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc29009a620 tx=0x7fc29000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0caa92aaf2857e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.667+0000 7fc28ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.668+0000 7fc28ffff640 1 -- 172.21.15.5:0/4217484866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29009bcd0 con 0x556ec158aaf0 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.668+0000 7fc2a9e42740 1 -- 172.21.15.5:0/4217484866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc290005680 con 0x556ec158aaf0 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.668+0000 7fc28ffff640 1 -- 172.21.15.5:0/4217484866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fc29009b7c0 con 0x556ec158aaf0 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.668+0000 7fc28ffff640 1 -- 172.21.15.5:0/4217484866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc2900a4d50 con 0x556ec158aaf0 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.668+0000 7fc2a9e42740 10 client.16074 fetch_fsmap learned FSMap version 30 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.668+0000 7fc2a9e42740 1 -- 172.21.15.5:0/4217484866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ec16108c0 con 0x556ec158aaf0 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.669+0000 7fc28ffff640 1 -- 172.21.15.5:0/4217484866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc290005680 con 0x556ec158aaf0 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.669+0000 7fc28ffff640 1 -- 172.21.15.5:0/4217484866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc29009b320 con 0x556ec158aaf0 2024-09-19T06:05:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.669+0000 7fc2a9e42740 10 client.16074 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:05:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.669+0000 7fc2a9e42740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:05:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.669+0000 7fc2a9e42740 20 client.16074 populate_metadata read hostname 'smithi005' 2024-09-19T06:05:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.669+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556ec1610ca0 0x556ec16130b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:05:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.669+0000 7fc2a9e42740 4 client.16074 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:05:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.669+0000 7fc2a9e42740 1 -- 172.21.15.5:0/4217484866 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ec14f6d70 con 0x556ec1610ca0 2024-09-19T06:05:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.669+0000 7fc29ffff640 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556ec1610ca0 0x556ec16130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:05:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.669+0000 7fc29ffff640 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556ec1610ca0 0x556ec16130b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cf5848b81cc122e server_cookie=c99ca400ef374a95 in_seq=0 out_seq=0 2024-09-19T06:05:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.669+0000 7fc28ffff640 10 client.16074 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc28ffff640 1 -- 172.21.15.5:0/4217484866 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ec14f6d70 con 0x556ec1610ca0 2024-09-19T06:05:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc28ffff640 10 client.16074 handle_command_reply: tid=0 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc2a9e42740 1 client.16074 shutdown 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc2a9e42740 1 -- 172.21.15.5:0/4217484866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158aaf0 msgr2=0x556ec159e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158aaf0 0x556ec159e3f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc29009a620 tx=0x7fc29000c840 comp rx=0 tx=0).stop 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc2a9e42740 1 -- 172.21.15.5:0/4217484866 shutdown_connections 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556ec1610ca0 0x556ec16130b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ec1595cb0 0x556ec1597010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ec158b410 0x556ec1596ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc28ffff640 0 client.16074 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc2a9e42740 1 --2- 172.21.15.5:0/4217484866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ec158aaf0 0x556ec159e3f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc2a9e42740 1 -- 172.21.15.5:0/4217484866 >> 172.21.15.5:0/4217484866 conn(0x556ec1587060 msgr2=0x556ec15859a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc2a9e42740 1 -- 172.21.15.5:0/4217484866 shutdown_connections 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc2a9e42740 1 -- 172.21.15.5:0/4217484866 wait complete. 2024-09-19T06:05:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.684+0000 7fc2a9e42740 20 client.16074 trim_cache size 0 max 16384 2024-09-19T06:05:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:05:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:05:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:05:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:05:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.685+0000 7fc2a70af640 1 -- 172.21.15.5:0/3599436858 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc2800a33b0 msgr2=0x7fc2800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.685+0000 7fc2a70af640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc2800a33b0 0x7fc2800a5870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc298009fd0 tx=0x7fc298004ab0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.685+0000 7fc2a70af640 1 -- 172.21.15.5:0/3599436858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2a0101ad0 msgr2=0x7fc2a01983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:05:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.685+0000 7fc2a70af640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2a0101ad0 0x7fc2a01983b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc29000e9f0 tx=0x7fc29000edf0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.685+0000 7fc2a70af640 1 -- 172.21.15.5:0/3599436858 shutdown_connections 2024-09-19T06:05:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.685+0000 7fc2a70af640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc2800a33b0 0x7fc2800a5870 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.685+0000 7fc2a70af640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a0107b70 0x7fc2a019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.685+0000 7fc2a70af640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a0102470 0x7fc2a01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.685+0000 7fc2a70af640 1 --2- 172.21.15.5:0/3599436858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2a0101ad0 0x7fc2a01983b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:05:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.685+0000 7fc2a70af640 1 -- 172.21.15.5:0/3599436858 >> 172.21.15.5:0/3599436858 conn(0x7fc2a00fd7d0 msgr2=0x7fc2a00fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:05:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.685+0000 7fc2a70af640 1 -- 172.21.15.5:0/3599436858 shutdown_connections 2024-09-19T06:05:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:05:56.685+0000 7fc2a70af640 1 -- 172.21.15.5:0/3599436858 wait complete. 2024-09-19T06:05:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:56 smithi157 ceph-mon[35346]: pgmap v80: 97 pgs: 97 active+clean; 42 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-19T06:05:56.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 43112 3.05 MB/sec warmup 82 sec latency 14.759 ms 2024-09-19T06:05:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:56 smithi005 ceph-mon[30955]: pgmap v80: 97 pgs: 97 active+clean; 42 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-19T06:05:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:56 smithi049 ceph-mon[34640]: pgmap v80: 97 pgs: 97 active+clean; 42 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-19T06:05:57.823 INFO:tasks.workunit.client.0.smithi005.stdout: 1 43577 3.03 MB/sec warmup 83 sec latency 15.207 ms 2024-09-19T06:05:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:58 smithi157 ceph-mon[35346]: pgmap v81: 97 pgs: 97 active+clean; 43 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-09-19T06:05:58.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 44191 3.04 MB/sec warmup 84 sec latency 14.653 ms 2024-09-19T06:05:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:58 smithi005 ceph-mon[30955]: pgmap v81: 97 pgs: 97 active+clean; 43 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-09-19T06:05:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:58 smithi049 ceph-mon[34640]: pgmap v81: 97 pgs: 97 active+clean; 43 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-09-19T06:05:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:05:59 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:05:59.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 44881 3.02 MB/sec warmup 85 sec latency 32.997 ms 2024-09-19T06:05:59.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:05:59 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:05:59.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:05:59 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:06:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:00 smithi157 ceph-mon[35346]: pgmap v82: 97 pgs: 97 active+clean; 43 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:06:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:00 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:06:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:00 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:06:00.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 45473 3.04 MB/sec warmup 86 sec latency 15.230 ms 2024-09-19T06:06:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:00 smithi005 ceph-mon[30955]: pgmap v82: 97 pgs: 97 active+clean; 43 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:06:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:00 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:06:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:00 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:06:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:00 smithi049 ceph-mon[34640]: pgmap v82: 97 pgs: 97 active+clean; 43 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:06:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:00 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:06:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:00 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:06:01.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 45962 3.05 MB/sec warmup 87 sec latency 16.184 ms 2024-09-19T06:06:02.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:02 smithi005 ceph-mon[30955]: pgmap v83: 97 pgs: 97 active+clean; 43 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:06:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:02 smithi157 ceph-mon[35346]: pgmap v83: 97 pgs: 97 active+clean; 43 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:06:02.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 46448 3.06 MB/sec warmup 88 sec latency 44.375 ms 2024-09-19T06:06:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:02 smithi049 ceph-mon[34640]: pgmap v83: 97 pgs: 97 active+clean; 43 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:06:03.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 46722 3.03 MB/sec warmup 89 sec latency 13.761 ms 2024-09-19T06:06:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:04 smithi049 ceph-mon[34640]: pgmap v84: 97 pgs: 97 active+clean; 44 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-09-19T06:06:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:04 smithi157 ceph-mon[35346]: pgmap v84: 97 pgs: 97 active+clean; 44 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-09-19T06:06:04.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 47219 3.02 MB/sec warmup 90 sec latency 14.864 ms 2024-09-19T06:06:04.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:04 smithi005 ceph-mon[30955]: pgmap v84: 97 pgs: 97 active+clean; 44 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-09-19T06:06:05.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 47794 3.02 MB/sec warmup 91 sec latency 15.684 ms 2024-09-19T06:06:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:06 smithi157 ceph-mon[35346]: pgmap v85: 97 pgs: 97 active+clean; 44 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-09-19T06:06:06.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 48483 3.00 MB/sec warmup 92 sec latency 22.497 ms 2024-09-19T06:06:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:06 smithi005 ceph-mon[30955]: pgmap v85: 97 pgs: 97 active+clean; 44 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-09-19T06:06:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:06 smithi049 ceph-mon[34640]: pgmap v85: 97 pgs: 97 active+clean; 44 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-09-19T06:06:07.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 49070 3.03 MB/sec warmup 93 sec latency 15.044 ms 2024-09-19T06:06:08.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:08 smithi157 ceph-mon[35346]: pgmap v86: 97 pgs: 97 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.7 MiB/s wr, 209 op/s 2024-09-19T06:06:08.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 49536 3.02 MB/sec warmup 94 sec latency 15.742 ms 2024-09-19T06:06:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:08 smithi005 ceph-mon[30955]: pgmap v86: 97 pgs: 97 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.7 MiB/s wr, 209 op/s 2024-09-19T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:08 smithi049 ceph-mon[34640]: pgmap v86: 97 pgs: 97 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.7 MiB/s wr, 209 op/s 2024-09-19T06:06:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.783+0000 7f40178f1640 1 -- 172.21.15.5:0/1371634866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4010101340 msgr2=0x7f401010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.783+0000 7f40178f1640 1 --2- 172.21.15.5:0/1371634866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4010101340 0x7f401010d870 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4008009cd0 tx=0x7f400802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.783+0000 7f40178f1640 1 -- 172.21.15.5:0/1371634866 shutdown_connections 2024-09-19T06:06:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.783+0000 7f40178f1640 1 --2- 172.21.15.5:0/1371634866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4010101340 0x7f401010d870 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.783+0000 7f40178f1640 1 --2- 172.21.15.5:0/1371634866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40101009a0 0x7f4010100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.783+0000 7f40178f1640 1 --2- 172.21.15.5:0/1371634866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40101069b0 0x7f4010106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.784+0000 7f40178f1640 1 -- 172.21.15.5:0/1371634866 >> 172.21.15.5:0/1371634866 conn(0x7f40100779a0 msgr2=0x7f40100fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.784+0000 7f40178f1640 1 -- 172.21.15.5:0/1371634866 shutdown_connections 2024-09-19T06:06:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.784+0000 7f40178f1640 1 -- 172.21.15.5:0/1371634866 wait complete. 2024-09-19T06:06:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.784+0000 7f40178f1640 1 Processor -- start 2024-09-19T06:06:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.784+0000 7f40178f1640 1 -- start start 2024-09-19T06:06:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f40178f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40101009a0 0x7f4010197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f40178f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4010101340 0x7f40101976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f40178f1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40101069b0 0x7f401019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f40178f1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4010110ca0 con 0x7f4010101340 2024-09-19T06:06:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f40178f1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4010110b20 con 0x7f40101069b0 2024-09-19T06:06:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f40178f1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4010110e20 con 0x7f40101009a0 2024-09-19T06:06:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f4014e8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4010101340 0x7f40101976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f4014e8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4010101340 0x7f40101976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41146/0 (socket says 172.21.15.5:41146) 2024-09-19T06:06:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f4014e8c640 1 -- 172.21.15.5:0/2891931580 learned_addr learned my addr 172.21.15.5:0/2891931580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f401568d640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40101009a0 0x7f4010197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f4015e8e640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40101069b0 0x7f401019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f4014e8c640 1 -- 172.21.15.5:0/2891931580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40101009a0 msgr2=0x7f4010197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f4014e8c640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40101009a0 0x7f4010197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f4014e8c640 1 -- 172.21.15.5:0/2891931580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40101069b0 msgr2=0x7f401019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f4014e8c640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40101069b0 0x7f401019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f4014e8c640 1 -- 172.21.15.5:0/2891931580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401019c160 con 0x7f4010101340 2024-09-19T06:06:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f4014e8c640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4010101340 0x7f40101976b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f400000e9f0 tx=0x7f400000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.785+0000 7f40067fc640 1 -- 172.21.15.5:0/2891931580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400000cd80 con 0x7f4010101340 2024-09-19T06:06:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.786+0000 7f40067fc640 1 -- 172.21.15.5:0/2891931580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40000108e0 con 0x7f4010101340 2024-09-19T06:06:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.786+0000 7f40067fc640 1 -- 172.21.15.5:0/2891931580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000018920 con 0x7f4010101340 2024-09-19T06:06:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.786+0000 7f40178f1640 1 -- 172.21.15.5:0/2891931580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401019c450 con 0x7f4010101340 2024-09-19T06:06:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.786+0000 7f40178f1640 1 -- 172.21.15.5:0/2891931580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4010069a10 con 0x7f4010101340 2024-09-19T06:06:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.788+0000 7f40067fc640 1 -- 172.21.15.5:0/2891931580 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4000010a90 con 0x7f4010101340 2024-09-19T06:06:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f401a693740 1 Processor -- start 2024-09-19T06:06:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f401a693740 1 -- start start 2024-09-19T06:06:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f401a693740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e94119ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f401a693740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e94124770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f401a693740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e941270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f401a693740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560e9407e2f0 con 0x560e94124cb0 2024-09-19T06:06:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f401a693740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560e94026270 con 0x560e9411a410 2024-09-19T06:06:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f401a693740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560e9407a2b0 con 0x560e94119af0 2024-09-19T06:06:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f4014e8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e941270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f4015e8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e94119ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f4015e8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e94119ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47764/0 (socket says 172.21.15.5:47764) 2024-09-19T06:06:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f4015e8e640 1 -- 172.21.15.5:0/1902458682 learned_addr learned my addr 172.21.15.5:0/1902458682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f401568d640 1 --2- 172.21.15.5:0/1902458682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e94124770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f40067fc640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3ffc0a3430 0x7f3ffc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f4015e8e640 1 -- 172.21.15.5:0/1902458682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 msgr2=0x560e94124770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f4015e8e640 1 --2- 172.21.15.5:0/1902458682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e94124770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f4015e8e640 1 -- 172.21.15.5:0/1902458682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 msgr2=0x560e941270a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f4015e8e640 1 --2- 172.21.15.5:0/1902458682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e941270a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.789+0000 7f4015e8e640 1 -- 172.21.15.5:0/1902458682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e94025340 con 0x560e94119af0 2024-09-19T06:06:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.790+0000 7f40067fc640 1 -- 172.21.15.5:0/2891931580 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f40000cff80 con 0x7f4010101340 2024-09-19T06:06:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.790+0000 7f4015e8e640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3ffc0a3430 0x7f3ffc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.790+0000 7f4015e8e640 1 --2- 172.21.15.5:0/1902458682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e94119ed0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4008035000 tx=0x7f4008009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e7fc45a126c346 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.790+0000 7f3ffb7fe640 1 -- 172.21.15.5:0/1902458682 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008004d40 con 0x560e94119af0 2024-09-19T06:06:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.790+0000 7f3ffb7fe640 1 -- 172.21.15.5:0/1902458682 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4008036040 con 0x560e94119af0 2024-09-19T06:06:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.790+0000 7f3ffb7fe640 1 -- 172.21.15.5:0/1902458682 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080045b0 con 0x560e94119af0 2024-09-19T06:06:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.791+0000 7f401a693740 1 -- 172.21.15.5:0/1902458682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 msgr2=0x560e94119ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.791+0000 7f401a693740 1 --2- 172.21.15.5:0/1902458682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e94119ed0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4008035000 tx=0x7f4008009a20 comp rx=0 tx=0).stop 2024-09-19T06:06:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.791+0000 7f4015e8e640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3ffc0a3430 0x7f3ffc0a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f400804d000 tx=0x7f4008050000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.791+0000 7f401a693740 1 -- 172.21.15.5:0/1902458682 shutdown_connections 2024-09-19T06:06:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.791+0000 7f401a693740 1 --2- 172.21.15.5:0/1902458682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e941270a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.791+0000 7f401a693740 1 --2- 172.21.15.5:0/1902458682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e94124770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.791+0000 7f401a693740 1 --2- 172.21.15.5:0/1902458682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e94119ed0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.791+0000 7f401a693740 1 -- 172.21.15.5:0/1902458682 >> 172.21.15.5:0/1902458682 conn(0x560e94116060 msgr2=0x560e94116440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.791+0000 7f401a693740 1 -- 172.21.15.5:0/1902458682 shutdown_connections 2024-09-19T06:06:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.791+0000 7f401a693740 1 -- 172.21.15.5:0/1902458682 wait complete. 2024-09-19T06:06:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.791+0000 7f401a693740 1 -- start start 2024-09-19T06:06:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.791+0000 7f401a693740 1 Processor -- start 2024-09-19T06:06:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401a693740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e94121c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401a693740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9411a410 0x560e94122190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401a693740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e94124cb0 0x560e941226d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401a693740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560e94127880 con 0x560e9411a410 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401a693740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560e94007290 con 0x560e94124cb0 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401a693740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560e94127a00 con 0x560e94119af0 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401a693740 10 client.0 _ll_register_callbacks cb 0x560e94038140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f4014e8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9411a410 0x560e94122190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401568d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e94121c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401568d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e94121c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47766/0 (socket says 172.21.15.5:47766) 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f4014e8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9411a410 0x560e94122190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41164/0 (socket says 172.21.15.5:41164) 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401568d640 1 -- 172.21.15.5:0/221906194 learned_addr learned my addr 172.21.15.5:0/221906194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f4015e8e640 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e94124cb0 0x560e941226d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401568d640 1 -- 172.21.15.5:0/221906194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e94124cb0 msgr2=0x560e941226d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401568d640 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e94124cb0 0x560e941226d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401568d640 1 -- 172.21.15.5:0/221906194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9411a410 msgr2=0x560e94122190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401568d640 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9411a410 0x560e94122190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f401568d640 1 -- 172.21.15.5:0/221906194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e93d91210 con 0x560e94119af0 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.792+0000 7f4014e8c640 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9411a410 0x560e94122190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.793+0000 7f401568d640 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e94121c50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f400c0126e0 tx=0x7f400c012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f5c150d8f655667 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.793+0000 7f3ff8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.793+0000 7f3ff8ff9640 1 -- 172.21.15.5:0/221906194 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0120d0 con 0x560e94119af0 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.793+0000 7f401a693740 1 -- 172.21.15.5:0/221906194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4008036040 con 0x560e94119af0 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.793+0000 7f3ff8ff9640 1 -- 172.21.15.5:0/221906194 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f400c00f920 con 0x560e94119af0 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.793+0000 7f3ff8ff9640 1 -- 172.21.15.5:0/221906194 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f400c012270 con 0x560e94119af0 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.793+0000 7f401a693740 10 client.25612 fetch_fsmap learned FSMap version 30 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.793+0000 7f401a693740 1 -- 172.21.15.5:0/221906194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e93e5d620 con 0x560e94119af0 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.793+0000 7f3ff8ff9640 1 -- 172.21.15.5:0/221906194 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4008036040 con 0x560e94119af0 2024-09-19T06:06:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.794+0000 7f3ff8ff9640 1 -- 172.21.15.5:0/221906194 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f400c010c90 con 0x560e94119af0 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.794+0000 7f401a693740 10 client.25612 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.794+0000 7f401a693740 10 client.25612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.794+0000 7f401a693740 20 client.25612 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.794+0000 7f401a693740 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e94134d60 0x560e941371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.794+0000 7f401a693740 4 client.25612 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.794+0000 7f401a693740 1 -- 172.21.15.5:0/221906194 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e9408ba20 con 0x560e94134d60 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.794+0000 7f4014e8c640 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e94134d60 0x560e941371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.794+0000 7f3ff8ff9640 10 client.25612 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.794+0000 7f4014e8c640 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e94134d60 0x560e941371b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a000f5f4be4157e server_cookie=bc9bda48c5c342c2 in_seq=0 out_seq=0 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f3ff8ff9640 1 -- 172.21.15.5:0/221906194 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e9408ba20 con 0x560e94134d60 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f3ff8ff9640 10 client.25612 handle_command_reply: tid=0 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f401a693740 1 client.25612 shutdown 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f401a693740 1 -- 172.21.15.5:0/221906194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 msgr2=0x560e94121c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f401a693740 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e94121c50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f400c0126e0 tx=0x7f400c012ba0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f401a693740 1 -- 172.21.15.5:0/221906194 shutdown_connections 2024-09-19T06:06:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f401a693740 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e94134d60 0x560e941371b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f401a693740 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e94124cb0 0x560e941226d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f401a693740 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9411a410 0x560e94122190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f3ff8ff9640 0 client.25612 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f401a693740 1 --2- 172.21.15.5:0/221906194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e94121c50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f401a693740 1 -- 172.21.15.5:0/221906194 >> 172.21.15.5:0/221906194 conn(0x560e94116060 msgr2=0x560e9411bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f401a693740 1 -- 172.21.15.5:0/221906194 shutdown_connections 2024-09-19T06:06:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f401a693740 1 -- 172.21.15.5:0/221906194 wait complete. 2024-09-19T06:06:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.795+0000 7f401a693740 20 client.25612 trim_cache size 0 max 16384 2024-09-19T06:06:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f401a693740 1 Processor -- start 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f401a693740 1 -- start start 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f401a693740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e941224f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f401a693740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e94123bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f401a693740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e9413cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f401a693740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ab640 con 0x560e94124cb0 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f401a693740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ad1e0 con 0x560e9411a410 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f401a693740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f40101abd10 con 0x560e94119af0 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f4015e8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e9413cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f4015e8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e9413cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41166/0 (socket says 172.21.15.5:41166) 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f4015e8e640 1 -- 172.21.15.5:0/937198247 learned_addr learned my addr 172.21.15.5:0/937198247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f4014e8c640 1 --2- 172.21.15.5:0/937198247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e94123bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.810+0000 7f401568d640 1 --2- 172.21.15.5:0/937198247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e941224f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.811+0000 7f401568d640 1 -- 172.21.15.5:0/937198247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 msgr2=0x560e94123bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.811+0000 7f401568d640 1 --2- 172.21.15.5:0/937198247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e94123bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.811+0000 7f401568d640 1 -- 172.21.15.5:0/937198247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 msgr2=0x560e9413cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.811+0000 7f401568d640 1 --2- 172.21.15.5:0/937198247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e9413cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.811+0000 7f401568d640 1 -- 172.21.15.5:0/937198247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e94119920 con 0x560e94119af0 2024-09-19T06:06:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.811+0000 7f4014e8c640 1 --2- 172.21.15.5:0/937198247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e94123bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.811+0000 7f401568d640 1 --2- 172.21.15.5:0/937198247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e941224f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f400c01c690 tx=0x7f400c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb9d0ba5f85f13cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.811+0000 7f3ffaffd640 1 -- 172.21.15.5:0/937198247 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c01c710 con 0x560e94119af0 2024-09-19T06:06:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.811+0000 7f3ffaffd640 1 -- 172.21.15.5:0/937198247 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f400c01c8b0 con 0x560e94119af0 2024-09-19T06:06:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.811+0000 7f3ffaffd640 1 -- 172.21.15.5:0/937198247 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c005290 con 0x560e94119af0 2024-09-19T06:06:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.812+0000 7f401a693740 1 -- 172.21.15.5:0/937198247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 msgr2=0x560e941224f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.812+0000 7f401a693740 1 --2- 172.21.15.5:0/937198247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e941224f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f400c01c690 tx=0x7f400c00f790 comp rx=0 tx=0).stop 2024-09-19T06:06:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.812+0000 7f401a693740 1 -- 172.21.15.5:0/937198247 shutdown_connections 2024-09-19T06:06:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.812+0000 7f401a693740 1 --2- 172.21.15.5:0/937198247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e9413cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.812+0000 7f401a693740 1 --2- 172.21.15.5:0/937198247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e94123bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.812+0000 7f401a693740 1 --2- 172.21.15.5:0/937198247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e941224f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.812+0000 7f401a693740 1 -- 172.21.15.5:0/937198247 >> 172.21.15.5:0/937198247 conn(0x560e94116060 msgr2=0x560e941234b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.812+0000 7f401a693740 1 -- 172.21.15.5:0/937198247 shutdown_connections 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.812+0000 7f401a693740 1 -- 172.21.15.5:0/937198247 wait complete. 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.812+0000 7f401a693740 1 -- start start 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.812+0000 7f401a693740 1 Processor -- start 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f401a693740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e9412ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f401a693740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e9412d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f401a693740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e9411e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f401a693740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f40101acab0 con 0x560e94124cb0 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f401a693740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ac3e0 con 0x560e9411a410 2024-09-19T06:06:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f401a693740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f40101a9620 con 0x560e94119af0 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f401a693740 10 client.0 _ll_register_callbacks cb 0x7f400c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f4015e8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e9411e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f401568d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e9412ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f4014e8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e9412d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f4015e8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e9411e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41178/0 (socket says 172.21.15.5:41178) 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f4015e8e640 1 -- 172.21.15.5:0/3402918565 learned_addr learned my addr 172.21.15.5:0/3402918565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f4014e8c640 1 -- 172.21.15.5:0/3402918565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 msgr2=0x560e9412ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f4014e8c640 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e9412ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f4014e8c640 1 -- 172.21.15.5:0/3402918565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 msgr2=0x560e9411e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f4014e8c640 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e9411e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f4014e8c640 1 -- 172.21.15.5:0/3402918565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e9411eb30 con 0x560e9411a410 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f401568d640 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e9412ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f4015e8e640 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e9411e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f4014e8c640 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e9412d3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f40000cd6e0 tx=0x7f40000bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c60e184076983fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f3ff8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f3ff8ff9640 1 -- 172.21.15.5:0/3402918565 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400009c320 con 0x560e9411a410 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f3ff8ff9640 1 -- 172.21.15.5:0/3402918565 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f40000d20b0 con 0x560e9411a410 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.813+0000 7f401a693740 1 -- 172.21.15.5:0/3402918565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f400c01c8b0 con 0x560e9411a410 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.814+0000 7f3ff8ff9640 1 -- 172.21.15.5:0/3402918565 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f40000d22b0 con 0x560e9411a410 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.814+0000 7f401a693740 10 client.25685 fetch_fsmap learned FSMap version 30 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.814+0000 7f401a693740 1 -- 172.21.15.5:0/3402918565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e9419f400 con 0x560e9411a410 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.814+0000 7f3ff8ff9640 1 -- 172.21.15.5:0/3402918565 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f400c01c8b0 con 0x560e9411a410 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.814+0000 7f3ff8ff9640 1 -- 172.21.15.5:0/3402918565 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f40000a0160 con 0x560e9411a410 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.814+0000 7f401a693740 10 client.25685 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.814+0000 7f401a693740 10 client.25685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.814+0000 7f401a693740 20 client.25685 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.814+0000 7f401a693740 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e94134d60 0x560e941a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.814+0000 7f401a693740 4 client.25685 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:06:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.814+0000 7f401a693740 1 -- 172.21.15.5:0/3402918565 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e94085d70 con 0x560e94134d60 2024-09-19T06:06:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.814+0000 7f401568d640 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e94134d60 0x560e941a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.815+0000 7f3ff8ff9640 10 client.25685 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.815+0000 7f401568d640 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e94134d60 0x560e941a1b50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61031a8f217e8617 server_cookie=12565082782661af in_seq=0 out_seq=0 2024-09-19T06:06:09.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 50005 3.04 MB/sec warmup 95 sec latency 15.007 ms 2024-09-19T06:06:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.829+0000 7f3ff8ff9640 1 -- 172.21.15.5:0/3402918565 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e94085d70 con 0x560e94134d60 2024-09-19T06:06:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.829+0000 7f3ff8ff9640 10 client.25685 handle_command_reply: tid=0 2024-09-19T06:06:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.829+0000 7f401a693740 1 client.25685 shutdown 2024-09-19T06:06:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.830+0000 7f401a693740 1 -- 172.21.15.5:0/3402918565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 msgr2=0x560e9412d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.830+0000 7f401a693740 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e9412d3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f40000cd6e0 tx=0x7f40000bc040 comp rx=0 tx=0).stop 2024-09-19T06:06:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.830+0000 7f401a693740 1 -- 172.21.15.5:0/3402918565 shutdown_connections 2024-09-19T06:06:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.830+0000 7f401a693740 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e94134d60 0x560e941a1b50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.830+0000 7f401a693740 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e94124cb0 0x560e9411e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.830+0000 7f3ff8ff9640 0 client.25685 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.830+0000 7f401a693740 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e9411a410 0x560e9412d3c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.830+0000 7f401a693740 1 --2- 172.21.15.5:0/3402918565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e94119af0 0x560e9412ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.830+0000 7f401a693740 1 -- 172.21.15.5:0/3402918565 >> 172.21.15.5:0/3402918565 conn(0x560e94116060 msgr2=0x560e9411bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.830+0000 7f401a693740 1 -- 172.21.15.5:0/3402918565 shutdown_connections 2024-09-19T06:06:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.830+0000 7f401a693740 1 -- 172.21.15.5:0/3402918565 wait complete. 2024-09-19T06:06:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.830+0000 7f401a693740 20 client.25685 trim_cache size 0 max 16384 2024-09-19T06:06:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:06:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:06:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:06:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:06:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.831+0000 7f40178f1640 1 -- 172.21.15.5:0/2891931580 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3ffc0a3430 msgr2=0x7f3ffc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.831+0000 7f40178f1640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3ffc0a3430 0x7f3ffc0a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f400804d000 tx=0x7f4008050000 comp rx=0 tx=0).stop 2024-09-19T06:06:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.831+0000 7f40178f1640 1 -- 172.21.15.5:0/2891931580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4010101340 msgr2=0x7f40101976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.831+0000 7f40178f1640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4010101340 0x7f40101976b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f400000e9f0 tx=0x7f400000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.831+0000 7f40178f1640 1 -- 172.21.15.5:0/2891931580 shutdown_connections 2024-09-19T06:06:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.831+0000 7f40178f1640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40101069b0 0x7f401019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.831+0000 7f40178f1640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4010101340 0x7f40101976b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.831+0000 7f40178f1640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3ffc0a3430 0x7f3ffc0a58f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.831+0000 7f40178f1640 1 --2- 172.21.15.5:0/2891931580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40101009a0 0x7f4010197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.831+0000 7f40178f1640 1 -- 172.21.15.5:0/2891931580 >> 172.21.15.5:0/2891931580 conn(0x7f40100779a0 msgr2=0x7f40100ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.831+0000 7f40178f1640 1 -- 172.21.15.5:0/2891931580 shutdown_connections 2024-09-19T06:06:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:09.831+0000 7f40178f1640 1 -- 172.21.15.5:0/2891931580 wait complete. 2024-09-19T06:06:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:10 smithi157 ceph-mon[35346]: pgmap v87: 97 pgs: 97 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-09-19T06:06:10.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 50298 3.01 MB/sec warmup 96 sec latency 14.951 ms 2024-09-19T06:06:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:10 smithi005 ceph-mon[30955]: pgmap v87: 97 pgs: 97 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-09-19T06:06:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:10 smithi049 ceph-mon[34640]: pgmap v87: 97 pgs: 97 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-09-19T06:06:11.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 50832 3.00 MB/sec warmup 97 sec latency 15.760 ms 2024-09-19T06:06:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:12 smithi157 ceph-mon[35346]: pgmap v88: 97 pgs: 97 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-09-19T06:06:12.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 51466 3.00 MB/sec warmup 98 sec latency 15.599 ms 2024-09-19T06:06:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:12 smithi005 ceph-mon[30955]: pgmap v88: 97 pgs: 97 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-09-19T06:06:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:12 smithi049 ceph-mon[34640]: pgmap v88: 97 pgs: 97 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-09-19T06:06:13.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 52059 2.99 MB/sec warmup 99 sec latency 15.544 ms 2024-09-19T06:06:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:14 smithi157 ceph-mon[35346]: pgmap v89: 97 pgs: 97 active+clean; 46 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-09-19T06:06:14.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 52681 3.01 MB/sec warmup 100 sec latency 15.667 ms 2024-09-19T06:06:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:14 smithi005 ceph-mon[30955]: pgmap v89: 97 pgs: 97 active+clean; 46 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-09-19T06:06:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:14 smithi049 ceph-mon[34640]: pgmap v89: 97 pgs: 97 active+clean; 46 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-09-19T06:06:15.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 53234 3.03 MB/sec warmup 101 sec latency 14.657 ms 2024-09-19T06:06:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:16 smithi157 ceph-mon[35346]: pgmap v90: 97 pgs: 97 active+clean; 46 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-19T06:06:16.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 53603 3.02 MB/sec warmup 102 sec latency 15.272 ms 2024-09-19T06:06:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:16 smithi005 ceph-mon[30955]: pgmap v90: 97 pgs: 97 active+clean; 46 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-19T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:16 smithi049 ceph-mon[34640]: pgmap v90: 97 pgs: 97 active+clean; 46 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-19T06:06:17.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 53915 2.99 MB/sec warmup 103 sec latency 11.415 ms 2024-09-19T06:06:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:18 smithi157 ceph-mon[35346]: pgmap v91: 97 pgs: 97 active+clean; 47 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-19T06:06:18.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 54379 2.98 MB/sec warmup 104 sec latency 99.892 ms 2024-09-19T06:06:18.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:18 smithi005 ceph-mon[30955]: pgmap v91: 97 pgs: 97 active+clean; 47 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-19T06:06:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:18 smithi049 ceph-mon[34640]: pgmap v91: 97 pgs: 97 active+clean; 47 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-19T06:06:19.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 54971 2.98 MB/sec warmup 105 sec latency 15.006 ms 2024-09-19T06:06:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:20 smithi157 ceph-mon[35346]: pgmap v92: 97 pgs: 97 active+clean; 47 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-19T06:06:20.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 55629 2.97 MB/sec warmup 106 sec latency 15.645 ms 2024-09-19T06:06:20.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:20 smithi005 ceph-mon[30955]: pgmap v92: 97 pgs: 97 active+clean; 47 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-19T06:06:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:20 smithi049 ceph-mon[34640]: pgmap v92: 97 pgs: 97 active+clean; 47 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-19T06:06:21.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 56231 2.99 MB/sec warmup 107 sec latency 15.144 ms 2024-09-19T06:06:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:22 smithi157 ceph-mon[35346]: pgmap v93: 97 pgs: 97 active+clean; 47 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-19T06:06:22.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 56790 3.01 MB/sec warmup 108 sec latency 14.684 ms 2024-09-19T06:06:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:22 smithi005 ceph-mon[30955]: pgmap v93: 97 pgs: 97 active+clean; 47 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-19T06:06:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:22 smithi049 ceph-mon[34640]: pgmap v93: 97 pgs: 97 active+clean; 47 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-19T06:06:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.929+0000 7fa264c8d640 1 -- 172.21.15.5:0/777334084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2601069b0 msgr2=0x7fa260106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.929+0000 7fa264c8d640 1 --2- 172.21.15.5:0/777334084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2601069b0 0x7fa260106d90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa250009930 tx=0x7fa25002a420 comp rx=0 tx=0).stop 2024-09-19T06:06:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.929+0000 7fa264c8d640 1 -- 172.21.15.5:0/777334084 shutdown_connections 2024-09-19T06:06:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.929+0000 7fa264c8d640 1 --2- 172.21.15.5:0/777334084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa260101340 0x7fa26010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.929+0000 7fa264c8d640 1 --2- 172.21.15.5:0/777334084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2601009a0 0x7fa260100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.929+0000 7fa264c8d640 1 --2- 172.21.15.5:0/777334084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2601069b0 0x7fa260106d90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.929+0000 7fa264c8d640 1 -- 172.21.15.5:0/777334084 >> 172.21.15.5:0/777334084 conn(0x7fa2600779a0 msgr2=0x7fa2600fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.929+0000 7fa264c8d640 1 -- 172.21.15.5:0/777334084 shutdown_connections 2024-09-19T06:06:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.929+0000 7fa264c8d640 1 -- 172.21.15.5:0/777334084 wait complete. 2024-09-19T06:06:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa264c8d640 1 Processor -- start 2024-09-19T06:06:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa264c8d640 1 -- start start 2024-09-19T06:06:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa264c8d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2601009a0 0x7fa260199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa264c8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa260101340 0x7fa260199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa264c8d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2601069b0 0x7fa2601a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa264c8d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa260110cf0 con 0x7fa260101340 2024-09-19T06:06:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa264c8d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa260110b70 con 0x7fa2601009a0 2024-09-19T06:06:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa264c8d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa260110e70 con 0x7fa2601069b0 2024-09-19T06:06:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa25e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2601009a0 0x7fa260199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa25e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2601009a0 0x7fa260199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39104/0 (socket says 172.21.15.5:39104) 2024-09-19T06:06:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa25e59c640 1 -- 172.21.15.5:0/3087969402 learned_addr learned my addr 172.21.15.5:0/3087969402 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa25ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2601069b0 0x7fa2601a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.930+0000 7fa25dd9b640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa260101340 0x7fa260199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa25e59c640 1 -- 172.21.15.5:0/3087969402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2601069b0 msgr2=0x7fa2601a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa25e59c640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2601069b0 0x7fa2601a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa25e59c640 1 -- 172.21.15.5:0/3087969402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa260101340 msgr2=0x7fa260199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa25e59c640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa260101340 0x7fa260199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa25e59c640 1 -- 172.21.15.5:0/3087969402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2601a47f0 con 0x7fa2601009a0 2024-09-19T06:06:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2601069b0 0x7fa2601a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa25dd9b640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa260101340 0x7fa260199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa25e59c640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2601009a0 0x7fa260199230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa250005ab0 tx=0x7fa250003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa24f7fe640 1 -- 172.21.15.5:0/3087969402 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25003f070 con 0x7fa2601009a0 2024-09-19T06:06:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa24f7fe640 1 -- 172.21.15.5:0/3087969402 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa250031b50 con 0x7fa2601009a0 2024-09-19T06:06:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa264c8d640 1 -- 172.21.15.5:0/3087969402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2601a49c0 con 0x7fa2601009a0 2024-09-19T06:06:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa24f7fe640 1 -- 172.21.15.5:0/3087969402 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250035030 con 0x7fa2601009a0 2024-09-19T06:06:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.931+0000 7fa264c8d640 1 -- 172.21.15.5:0/3087969402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa260197020 con 0x7fa2601009a0 2024-09-19T06:06:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.934+0000 7fa24f7fe640 1 -- 172.21.15.5:0/3087969402 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa250042070 con 0x7fa2601009a0 2024-09-19T06:06:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.934+0000 7fa267a27740 1 Processor -- start 2024-09-19T06:06:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.934+0000 7fa267a27740 1 -- start start 2024-09-19T06:06:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa267a27740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 0x563dc296ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa267a27740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc296d410 0x563dc2977770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa267a27740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc2977cb0 0x563dc297a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa267a27740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563dc28d12f0 con 0x563dc296caf0 2024-09-19T06:06:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa267a27740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563dc2879270 con 0x563dc2977cb0 2024-09-19T06:06:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa267a27740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563dc28cd2b0 con 0x563dc296d410 2024-09-19T06:06:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa24f7fe640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2480a77c0 0x7fa2480a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa25e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc296d410 0x563dc2977770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa25ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 0x563dc296ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa25ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 0x563dc296ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40036/0 (socket says 172.21.15.5:40036) 2024-09-19T06:06:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa25e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc296d410 0x563dc2977770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60472/0 (socket says 172.21.15.5:60472) 2024-09-19T06:06:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa25dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc2977cb0 0x563dc297a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa25ed9d640 1 -- 172.21.15.5:0/1669022977 learned_addr learned my addr 172.21.15.5:0/1669022977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2480a77c0 0x7fa2480a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa25ed9d640 1 -- 172.21.15.5:0/1669022977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc296d410 msgr2=0x563dc2977770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/1669022977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc296d410 0x563dc2977770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa25ed9d640 1 -- 172.21.15.5:0/1669022977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc2977cb0 msgr2=0x563dc297a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/1669022977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc2977cb0 0x563dc297a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa25ed9d640 1 -- 172.21.15.5:0/1669022977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dc2878340 con 0x563dc296caf0 2024-09-19T06:06:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.935+0000 7fa24f7fe640 1 -- 172.21.15.5:0/3087969402 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fa2500f02a0 con 0x7fa2601009a0 2024-09-19T06:06:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.936+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/1669022977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 0x563dc296ced0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa25800f5c0 tx=0x7fa25800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3420b9f2d09dd203 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.936+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2480a77c0 0x7fa2480a9c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa2580099e0 tx=0x7fa25801e000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.936+0000 7fa24cff9640 1 -- 172.21.15.5:0/1669022977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25802b070 con 0x563dc296caf0 2024-09-19T06:06:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.936+0000 7fa24cff9640 1 -- 172.21.15.5:0/1669022977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2580268c0 con 0x563dc296caf0 2024-09-19T06:06:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.936+0000 7fa24cff9640 1 -- 172.21.15.5:0/1669022977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258022070 con 0x563dc296caf0 2024-09-19T06:06:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.936+0000 7fa267a27740 1 -- 172.21.15.5:0/1669022977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 msgr2=0x563dc296ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.936+0000 7fa267a27740 1 --2- 172.21.15.5:0/1669022977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 0x563dc296ced0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa25800f5c0 tx=0x7fa25800fa80 comp rx=0 tx=0).stop 2024-09-19T06:06:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 -- 172.21.15.5:0/1669022977 shutdown_connections 2024-09-19T06:06:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 --2- 172.21.15.5:0/1669022977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc2977cb0 0x563dc297a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 --2- 172.21.15.5:0/1669022977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc296d410 0x563dc2977770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 --2- 172.21.15.5:0/1669022977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 0x563dc296ced0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 -- 172.21.15.5:0/1669022977 >> 172.21.15.5:0/1669022977 conn(0x563dc2969060 msgr2=0x563dc2969440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 -- 172.21.15.5:0/1669022977 shutdown_connections 2024-09-19T06:06:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 -- 172.21.15.5:0/1669022977 wait complete. 2024-09-19T06:06:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 -- start start 2024-09-19T06:06:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 Processor -- start 2024-09-19T06:06:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc296caf0 0x563dc2974b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 0x563dc29750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc2977cb0 0x563dc29755e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563dc297a880 con 0x563dc296d410 2024-09-19T06:06:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563dc285a290 con 0x563dc296caf0 2024-09-19T06:06:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.937+0000 7fa267a27740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563dc297aa00 con 0x563dc2977cb0 2024-09-19T06:06:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa25dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 0x563dc29750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa267a27740 10 client.0 _ll_register_callbacks cb 0x563dc288b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa25e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc296caf0 0x563dc2974b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa25dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 0x563dc29750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40050/0 (socket says 172.21.15.5:40050) 2024-09-19T06:06:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa25dd9b640 1 -- 172.21.15.5:0/3813475460 learned_addr learned my addr 172.21.15.5:0/3813475460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/3813475460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc2977cb0 0x563dc29755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa25dd9b640 1 -- 172.21.15.5:0/3813475460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc2977cb0 msgr2=0x563dc29755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa25dd9b640 1 --2- 172.21.15.5:0/3813475460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc2977cb0 0x563dc29755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa25dd9b640 1 -- 172.21.15.5:0/3813475460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc296caf0 msgr2=0x563dc2974b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa25dd9b640 1 --2- 172.21.15.5:0/3813475460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc296caf0 0x563dc2974b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa25dd9b640 1 -- 172.21.15.5:0/3813475460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dc25e4210 con 0x563dc296d410 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa25dd9b640 1 --2- 172.21.15.5:0/3813475460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 0x563dc29750a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa254012d50 tx=0x7fa25400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73fbbd7b58eefa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa245ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa245ffb640 1 -- 172.21.15.5:0/3813475460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254012110 con 0x563dc296d410 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa245ffb640 1 -- 172.21.15.5:0/3813475460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fa254004050 con 0x563dc296d410 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa267a27740 1 -- 172.21.15.5:0/3813475460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2580268c0 con 0x563dc296d410 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa245ffb640 1 -- 172.21.15.5:0/3813475460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa2540122b0 con 0x563dc296d410 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa267a27740 10 client.16104 fetch_fsmap learned FSMap version 30 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.938+0000 7fa267a27740 1 -- 172.21.15.5:0/3813475460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563dc26b0620 con 0x563dc296d410 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.939+0000 7fa245ffb640 1 -- 172.21.15.5:0/3813475460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2580268c0 con 0x563dc296d410 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.939+0000 7fa245ffb640 1 -- 172.21.15.5:0/3813475460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa254010430 con 0x563dc296d410 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.939+0000 7fa267a27740 10 client.16104 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.939+0000 7fa267a27740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.939+0000 7fa267a27740 20 client.16104 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.939+0000 7fa267a27740 1 --2- 172.21.15.5:0/3813475460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563dc2987c70 0x563dc298a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.939+0000 7fa267a27740 4 client.16104 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:06:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.939+0000 7fa267a27740 1 -- 172.21.15.5:0/3813475460 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dc28dea20 con 0x563dc2987c70 2024-09-19T06:06:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.939+0000 7fa25e59c640 1 --2- 172.21.15.5:0/3813475460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563dc2987c70 0x563dc298a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.939+0000 7fa25e59c640 1 --2- 172.21.15.5:0/3813475460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563dc2987c70 0x563dc298a0c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10a6030fc38e655b server_cookie=44c24f954ee58d32 in_seq=0 out_seq=0 2024-09-19T06:06:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.939+0000 7fa245ffb640 10 client.16104 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa245ffb640 1 -- 172.21.15.5:0/3813475460 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563dc28dea20 con 0x563dc2987c70 2024-09-19T06:06:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa245ffb640 10 client.16104 handle_command_reply: tid=0 2024-09-19T06:06:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa267a27740 1 client.16104 shutdown 2024-09-19T06:06:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa267a27740 1 -- 172.21.15.5:0/3813475460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 msgr2=0x563dc29750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa267a27740 1 --2- 172.21.15.5:0/3813475460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 0x563dc29750a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa254012d50 tx=0x7fa25400f790 comp rx=0 tx=0).stop 2024-09-19T06:06:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa267a27740 1 -- 172.21.15.5:0/3813475460 shutdown_connections 2024-09-19T06:06:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa267a27740 1 --2- 172.21.15.5:0/3813475460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563dc2987c70 0x563dc298a0c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa267a27740 1 --2- 172.21.15.5:0/3813475460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc2977cb0 0x563dc29755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa267a27740 1 --2- 172.21.15.5:0/3813475460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 0x563dc29750a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa245ffb640 0 client.16104 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa267a27740 1 --2- 172.21.15.5:0/3813475460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc296caf0 0x563dc2974b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa267a27740 1 -- 172.21.15.5:0/3813475460 >> 172.21.15.5:0/3813475460 conn(0x563dc2969060 msgr2=0x563dc296ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa267a27740 1 -- 172.21.15.5:0/3813475460 shutdown_connections 2024-09-19T06:06:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.940+0000 7fa267a27740 1 -- 172.21.15.5:0/3813475460 wait complete. 2024-09-19T06:06:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.941+0000 7fa267a27740 20 client.16104 trim_cache size 0 max 16384 2024-09-19T06:06:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa267a27740 1 Processor -- start 2024-09-19T06:06:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa267a27740 1 -- start start 2024-09-19T06:06:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa267a27740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 0x563dc296f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa267a27740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc296d410 0x563dc2975410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa267a27740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc2977cb0 0x563dc298f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa267a27740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2601ab4f0 con 0x563dc296caf0 2024-09-19T06:06:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa267a27740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa2601ad090 con 0x563dc296d410 2024-09-19T06:06:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa267a27740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2601abbc0 con 0x563dc2977cb0 2024-09-19T06:06:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa25e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 0x563dc296f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa25ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc2977cb0 0x563dc298f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa25e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 0x563dc296f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40056/0 (socket says 172.21.15.5:40056) 2024-09-19T06:06:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa25e59c640 1 -- 172.21.15.5:0/2942255280 learned_addr learned my addr 172.21.15.5:0/2942255280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa25dd9b640 1 --2- 172.21.15.5:0/2942255280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc296d410 0x563dc2975410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa25ed9d640 1 -- 172.21.15.5:0/2942255280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc296d410 msgr2=0x563dc2975410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/2942255280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc296d410 0x563dc2975410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa25ed9d640 1 -- 172.21.15.5:0/2942255280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 msgr2=0x563dc296f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/2942255280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 0x563dc296f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa25ed9d640 1 -- 172.21.15.5:0/2942255280 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dc29642d0 con 0x563dc2977cb0 2024-09-19T06:06:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.957+0000 7fa25e59c640 1 --2- 172.21.15.5:0/2942255280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 0x563dc296f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.958+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/2942255280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc2977cb0 0x563dc298f990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa258009f20 tx=0x7fa258011e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc8f6f292762bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.958+0000 7fa247fff640 1 -- 172.21.15.5:0/2942255280 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258022070 con 0x563dc2977cb0 2024-09-19T06:06:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.958+0000 7fa247fff640 1 -- 172.21.15.5:0/2942255280 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa258003e20 con 0x563dc2977cb0 2024-09-19T06:06:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.958+0000 7fa247fff640 1 -- 172.21.15.5:0/2942255280 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25802b070 con 0x563dc2977cb0 2024-09-19T06:06:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.959+0000 7fa267a27740 1 -- 172.21.15.5:0/2942255280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc2977cb0 msgr2=0x563dc298f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.959+0000 7fa267a27740 1 --2- 172.21.15.5:0/2942255280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc2977cb0 0x563dc298f990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa258009f20 tx=0x7fa258011e10 comp rx=0 tx=0).stop 2024-09-19T06:06:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.959+0000 7fa267a27740 1 -- 172.21.15.5:0/2942255280 shutdown_connections 2024-09-19T06:06:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.959+0000 7fa267a27740 1 --2- 172.21.15.5:0/2942255280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc2977cb0 0x563dc298f990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.959+0000 7fa267a27740 1 --2- 172.21.15.5:0/2942255280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc296d410 0x563dc2975410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.959+0000 7fa267a27740 1 --2- 172.21.15.5:0/2942255280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296caf0 0x563dc296f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.959+0000 7fa267a27740 1 -- 172.21.15.5:0/2942255280 >> 172.21.15.5:0/2942255280 conn(0x563dc2969060 msgr2=0x563dc2976be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.959+0000 7fa267a27740 1 -- 172.21.15.5:0/2942255280 shutdown_connections 2024-09-19T06:06:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.959+0000 7fa267a27740 1 -- 172.21.15.5:0/2942255280 wait complete. 2024-09-19T06:06:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.959+0000 7fa267a27740 1 -- start start 2024-09-19T06:06:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.959+0000 7fa267a27740 1 Processor -- start 2024-09-19T06:06:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.960+0000 7fa267a27740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc296caf0 0x563dc29803f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.960+0000 7fa267a27740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 0x563dc297b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.960+0000 7fa267a27740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc2977cb0 0x563dc297bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.960+0000 7fa267a27740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2601ac960 con 0x563dc296d410 2024-09-19T06:06:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.960+0000 7fa267a27740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa2601ac290 con 0x563dc2977cb0 2024-09-19T06:06:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.960+0000 7fa267a27740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2601a94d0 con 0x563dc296caf0 2024-09-19T06:06:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.960+0000 7fa267a27740 10 client.0 _ll_register_callbacks cb 0x7fa254009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.960+0000 7fa25ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc2977cb0 0x563dc297bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.960+0000 7fa25ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc2977cb0 0x563dc297bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39128/0 (socket says 172.21.15.5:39128) 2024-09-19T06:06:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.960+0000 7fa25ed9d640 1 -- 172.21.15.5:0/2771984285 learned_addr learned my addr 172.21.15.5:0/2771984285 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.960+0000 7fa25dd9b640 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 0x563dc297b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.960+0000 7fa25e59c640 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc296caf0 0x563dc29803f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa25ed9d640 1 -- 172.21.15.5:0/2771984285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc296caf0 msgr2=0x563dc29803f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc296caf0 0x563dc29803f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa25ed9d640 1 -- 172.21.15.5:0/2771984285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 msgr2=0x563dc297b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 0x563dc297b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa25ed9d640 1 -- 172.21.15.5:0/2771984285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dc297c430 con 0x563dc2977cb0 2024-09-19T06:06:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa25dd9b640 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 0x563dc297b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa25ed9d640 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc2977cb0 0x563dc297bc50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa25800fac0 tx=0x7fa258005cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91fa081d246f90a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa245ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:06:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa245ffb640 1 -- 172.21.15.5:0/2771984285 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25802b070 con 0x563dc2977cb0 2024-09-19T06:06:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa267a27740 1 -- 172.21.15.5:0/2771984285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa258003e20 con 0x563dc2977cb0 2024-09-19T06:06:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa245ffb640 1 -- 172.21.15.5:0/2771984285 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7fa258011230 con 0x563dc2977cb0 2024-09-19T06:06:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa245ffb640 1 -- 172.21.15.5:0/2771984285 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa258022040 con 0x563dc2977cb0 2024-09-19T06:06:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa267a27740 10 client.25703 fetch_fsmap learned FSMap version 30 2024-09-19T06:06:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.961+0000 7fa267a27740 1 -- 172.21.15.5:0/2771984285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563dc29f2980 con 0x563dc2977cb0 2024-09-19T06:06:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.962+0000 7fa245ffb640 1 -- 172.21.15.5:0/2771984285 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa258003e20 con 0x563dc2977cb0 2024-09-19T06:06:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.962+0000 7fa245ffb640 1 -- 172.21.15.5:0/2771984285 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa258010570 con 0x563dc2977cb0 2024-09-19T06:06:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.962+0000 7fa267a27740 10 client.25703 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:06:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.962+0000 7fa267a27740 10 client.25703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.962+0000 7fa267a27740 20 client.25703 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.962+0000 7fa267a27740 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563dc29f2f50 0x563dc29f5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.962+0000 7fa267a27740 4 client.25703 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:06:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.962+0000 7fa267a27740 1 -- 172.21.15.5:0/2771984285 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563dc28d8d70 con 0x563dc29f2f50 2024-09-19T06:06:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.962+0000 7fa25e59c640 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563dc29f2f50 0x563dc29f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.962+0000 7fa245ffb640 10 client.25703 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.962+0000 7fa25e59c640 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563dc29f2f50 0x563dc29f5370 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4fb8029434bb374 server_cookie=f145157c17f9ae9 in_seq=0 out_seq=0 2024-09-19T06:06:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa245ffb640 1 -- 172.21.15.5:0/2771984285 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563dc28d8d70 con 0x563dc29f2f50 2024-09-19T06:06:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa245ffb640 10 client.25703 handle_command_reply: tid=0 2024-09-19T06:06:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa267a27740 1 client.25703 shutdown 2024-09-19T06:06:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa267a27740 1 -- 172.21.15.5:0/2771984285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc2977cb0 msgr2=0x563dc297bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa267a27740 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc2977cb0 0x563dc297bc50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa25800fac0 tx=0x7fa258005cf0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa267a27740 1 -- 172.21.15.5:0/2771984285 shutdown_connections 2024-09-19T06:06:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa267a27740 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563dc29f2f50 0x563dc29f5370 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa267a27740 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc2977cb0 0x563dc297bc50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa267a27740 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dc296d410 0x563dc297b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa267a27740 1 --2- 172.21.15.5:0/2771984285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dc296caf0 0x563dc29803f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa245ffb640 0 client.25703 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa267a27740 1 -- 172.21.15.5:0/2771984285 >> 172.21.15.5:0/2771984285 conn(0x563dc2969060 msgr2=0x563dc29763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa267a27740 1 -- 172.21.15.5:0/2771984285 shutdown_connections 2024-09-19T06:06:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.979+0000 7fa267a27740 1 -- 172.21.15.5:0/2771984285 wait complete. 2024-09-19T06:06:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa267a27740 20 client.25703 trim_cache size 0 max 16384 2024-09-19T06:06:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:06:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:06:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:06:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:06:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa264c8d640 1 -- 172.21.15.5:0/3087969402 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2480a77c0 msgr2=0x7fa2480a9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa264c8d640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2480a77c0 0x7fa2480a9c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa2580099e0 tx=0x7fa25801e000 comp rx=0 tx=0).stop 2024-09-19T06:06:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa264c8d640 1 -- 172.21.15.5:0/3087969402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2601009a0 msgr2=0x7fa260199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa264c8d640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2601009a0 0x7fa260199230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa250005ab0 tx=0x7fa250003b90 comp rx=0 tx=0).stop 2024-09-19T06:06:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa264c8d640 1 -- 172.21.15.5:0/3087969402 shutdown_connections 2024-09-19T06:06:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa264c8d640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2480a77c0 0x7fa2480a9c80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa264c8d640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2601069b0 0x7fa2601a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa264c8d640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa260101340 0x7fa260199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa264c8d640 1 --2- 172.21.15.5:0/3087969402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2601009a0 0x7fa260199230 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa264c8d640 1 -- 172.21.15.5:0/3087969402 >> 172.21.15.5:0/3087969402 conn(0x7fa2600779a0 msgr2=0x7fa2600ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa264c8d640 1 -- 172.21.15.5:0/3087969402 shutdown_connections 2024-09-19T06:06:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:22.980+0000 7fa264c8d640 1 -- 172.21.15.5:0/3087969402 wait complete. 2024-09-19T06:06:23.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:06:23.351 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:06:23.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 57158 3.00 MB/sec warmup 109 sec latency 30.619 ms 2024-09-19T06:06:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.889+0000 7f33cfe56640 1 -- 172.21.15.5:0/372846263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c8102570 msgr2=0x7f33c81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.889+0000 7f33cfe56640 1 --2- 172.21.15.5:0/372846263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c8102570 0x7f33c81029b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009900 tx=0x7f33bc02a480 comp rx=0 tx=0).stop 2024-09-19T06:06:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.889+0000 7f33cfe56640 1 -- 172.21.15.5:0/372846263 shutdown_connections 2024-09-19T06:06:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.889+0000 7f33cfe56640 1 --2- 172.21.15.5:0/372846263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33c8102ef0 0x7f33c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.889+0000 7f33cfe56640 1 --2- 172.21.15.5:0/372846263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c8102570 0x7f33c81029b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.889+0000 7f33cfe56640 1 --2- 172.21.15.5:0/372846263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33c8100e20 0x7f33c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.889+0000 7f33cfe56640 1 -- 172.21.15.5:0/372846263 >> 172.21.15.5:0/372846263 conn(0x7f33c80fc7d0 msgr2=0x7f33c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.889+0000 7f33cfe56640 1 -- 172.21.15.5:0/372846263 shutdown_connections 2024-09-19T06:06:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.889+0000 7f33cfe56640 1 -- 172.21.15.5:0/372846263 wait complete. 2024-09-19T06:06:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.890+0000 7f33cfe56640 1 Processor -- start 2024-09-19T06:06:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.890+0000 7f33cfe56640 1 -- start start 2024-09-19T06:06:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.890+0000 7f33cfe56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c8100e20 0x7f33c81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.890+0000 7f33cfe56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33c8102570 0x7f33c8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.890+0000 7f33cfe56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33c8102ef0 0x7f33c819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.890+0000 7f33cfe56640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8111f10 con 0x7f33c8100e20 2024-09-19T06:06:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.890+0000 7f33cfe56640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8111d90 con 0x7f33c8102ef0 2024-09-19T06:06:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.890+0000 7f33cfe56640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8112090 con 0x7f33c8102570 2024-09-19T06:06:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.890+0000 7f33cdbf2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c8100e20 0x7f33c81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.890+0000 7f33cdbf2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c8100e20 0x7f33c81983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40098/0 (socket says 172.21.15.5:40098) 2024-09-19T06:06:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.890+0000 7f33cdbf2640 1 -- 172.21.15.5:0/225173517 learned_addr learned my addr 172.21.15.5:0/225173517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33cd3f1640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33c8102570 0x7f33c8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33ce3f3640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33c8102ef0 0x7f33c819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33cdbf2640 1 -- 172.21.15.5:0/225173517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33c8102570 msgr2=0x7f33c8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33cdbf2640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33c8102570 0x7f33c8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33cdbf2640 1 -- 172.21.15.5:0/225173517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33c8102ef0 msgr2=0x7f33c819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33cdbf2640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33c8102ef0 0x7f33c819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33cdbf2640 1 -- 172.21.15.5:0/225173517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c819d3a0 con 0x7f33c8100e20 2024-09-19T06:06:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33ce3f3640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33c8102ef0 0x7f33c819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33cdbf2640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c8100e20 0x7f33c81983f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f33c400d950 tx=0x7f33c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33baffd640 1 -- 172.21.15.5:0/225173517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4017030 con 0x7f33c8100e20 2024-09-19T06:06:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33cfe56640 1 -- 172.21.15.5:0/225173517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c819d690 con 0x7f33c8100e20 2024-09-19T06:06:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33baffd640 1 -- 172.21.15.5:0/225173517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33c4004020 con 0x7f33c8100e20 2024-09-19T06:06:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.891+0000 7f33baffd640 1 -- 172.21.15.5:0/225173517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c40047a0 con 0x7f33c8100e20 2024-09-19T06:06:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.892+0000 7f33cfe56640 1 -- 172.21.15.5:0/225173517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c81a4f70 con 0x7f33c8100e20 2024-09-19T06:06:23.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.895+0000 7f33baffd640 1 -- 172.21.15.5:0/225173517 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f33c40041d0 con 0x7f33c8100e20 2024-09-19T06:06:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.895+0000 7f33d222f740 1 Processor -- start 2024-09-19T06:06:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.895+0000 7f33d222f740 1 -- start start 2024-09-19T06:06:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33d222f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f554a0 0x55e021f55880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33d222f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f55dc0 0x55e021f60140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33d222f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f60680 0x55e021f62a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33d222f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e021ebb2e0 con 0x55e021f60680 2024-09-19T06:06:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33d222f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e021e62ce0 con 0x55e021f554a0 2024-09-19T06:06:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33d222f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e021eb72a0 con 0x55e021f55dc0 2024-09-19T06:06:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33ce3f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f554a0 0x55e021f55880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33ce3f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f554a0 0x55e021f55880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39156/0 (socket says 172.21.15.5:39156) 2024-09-19T06:06:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33baffd640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f33b40a33b0 0x7f33b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33ce3f3640 1 -- 172.21.15.5:0/2699246676 learned_addr learned my addr 172.21.15.5:0/2699246676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33cd3f1640 1 --2- 172.21.15.5:0/2699246676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f60680 0x55e021f62a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33cdbf2640 1 --2- 172.21.15.5:0/2699246676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f55dc0 0x55e021f60140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33ce3f3640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f33b40a33b0 0x7f33b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33baffd640 1 -- 172.21.15.5:0/225173517 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f33c40cf7f0 con 0x7f33c8100e20 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33ce3f3640 1 -- 172.21.15.5:0/2699246676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f55dc0 msgr2=0x55e021f60140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33ce3f3640 1 --2- 172.21.15.5:0/2699246676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f55dc0 0x55e021f60140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33ce3f3640 1 -- 172.21.15.5:0/2699246676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f60680 msgr2=0x55e021f62a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33ce3f3640 1 --2- 172.21.15.5:0/2699246676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f60680 0x55e021f62a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33ce3f3640 1 -- 172.21.15.5:0/2699246676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e021e47440 con 0x55e021f554a0 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.896+0000 7f33cd3f1640 1 --2- 172.21.15.5:0/2699246676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f60680 0x55e021f62a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.897+0000 7f33ce3f3640 1 --2- 172.21.15.5:0/2699246676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f554a0 0x55e021f55880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009fd0 tx=0x7f33c000f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9687906ce219ec75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.897+0000 7f33b3fff640 1 -- 172.21.15.5:0/2699246676 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0023020 con 0x55e021f554a0 2024-09-19T06:06:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.897+0000 7f33ce3f3640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f33b40a33b0 0x7f33b40a5870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f33c0024000 tx=0x7f33c0022000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:23.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.897+0000 7f33b3fff640 1 -- 172.21.15.5:0/2699246676 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33c00138a0 con 0x55e021f554a0 2024-09-19T06:06:23.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.897+0000 7f33b3fff640 1 -- 172.21.15.5:0/2699246676 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0010720 con 0x55e021f554a0 2024-09-19T06:06:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.898+0000 7f33d222f740 1 -- 172.21.15.5:0/2699246676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f554a0 msgr2=0x55e021f55880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.898+0000 7f33d222f740 1 --2- 172.21.15.5:0/2699246676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f554a0 0x55e021f55880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009fd0 tx=0x7f33c000f7d0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.898+0000 7f33d222f740 1 -- 172.21.15.5:0/2699246676 shutdown_connections 2024-09-19T06:06:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.898+0000 7f33d222f740 1 --2- 172.21.15.5:0/2699246676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f60680 0x55e021f62a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.898+0000 7f33d222f740 1 --2- 172.21.15.5:0/2699246676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f55dc0 0x55e021f60140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.898+0000 7f33d222f740 1 --2- 172.21.15.5:0/2699246676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f554a0 0x55e021f55880 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.898+0000 7f33d222f740 1 -- 172.21.15.5:0/2699246676 >> 172.21.15.5:0/2699246676 conn(0x55e021f51a30 msgr2=0x55e021f51e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.898+0000 7f33d222f740 1 -- 172.21.15.5:0/2699246676 shutdown_connections 2024-09-19T06:06:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.898+0000 7f33d222f740 1 -- 172.21.15.5:0/2699246676 wait complete. 2024-09-19T06:06:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.898+0000 7f33d222f740 1 -- start start 2024-09-19T06:06:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33d222f740 1 Processor -- start 2024-09-19T06:06:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33d222f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f554a0 0x55e021f5fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33d222f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f55dc0 0x55e021f5d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33d222f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f60680 0x55e021f5d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33d222f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e021f67660 con 0x55e021f60680 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33d222f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e021e76050 con 0x55e021f55dc0 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33d222f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e021f677e0 con 0x55e021f554a0 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33d222f740 10 client.0 _ll_register_callbacks cb 0x55e021eb6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33cdbf2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f554a0 0x55e021f5fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33cd3f1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f55dc0 0x55e021f5d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33cdbf2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f554a0 0x55e021f5fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60546/0 (socket says 172.21.15.5:60546) 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33cdbf2640 1 -- 172.21.15.5:0/77387798 learned_addr learned my addr 172.21.15.5:0/77387798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.899+0000 7f33ce3f3640 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f60680 0x55e021f5d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.900+0000 7f33cd3f1640 1 -- 172.21.15.5:0/77387798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f554a0 msgr2=0x55e021f5fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.900+0000 7f33cd3f1640 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f554a0 0x55e021f5fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.900+0000 7f33cd3f1640 1 -- 172.21.15.5:0/77387798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f60680 msgr2=0x55e021f5d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.900+0000 7f33cd3f1640 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f60680 0x55e021f5d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.900+0000 7f33cd3f1640 1 -- 172.21.15.5:0/77387798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e021c2c3c0 con 0x55e021f55dc0 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.900+0000 7f33ce3f3640 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f60680 0x55e021f5d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.900+0000 7f33cd3f1640 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f55dc0 0x55e021f5d2a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009230 tx=0x7f33bc0317a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34df18e9f23dab2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.900+0000 7f33b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.900+0000 7f33b17fa640 1 -- 172.21.15.5:0/77387798 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc035040 con 0x55e021f55dc0 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.900+0000 7f33b17fa640 1 -- 172.21.15.5:0/77387798 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f33bc004240 con 0x55e021f55dc0 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.900+0000 7f33d222f740 1 -- 172.21.15.5:0/77387798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e021ec8850 con 0x55e021f55dc0 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.901+0000 7f33b17fa640 1 -- 172.21.15.5:0/77387798 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f33bc004480 con 0x55e021f55dc0 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.901+0000 7f33d222f740 10 client.25721 fetch_fsmap learned FSMap version 30 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.901+0000 7f33d222f740 1 -- 172.21.15.5:0/77387798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e021f77490 con 0x55e021f55dc0 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.901+0000 7f33b17fa640 1 -- 172.21.15.5:0/77387798 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55e021ec8850 con 0x55e021f55dc0 2024-09-19T06:06:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.901+0000 7f33b17fa640 1 -- 172.21.15.5:0/77387798 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f33bc004ca0 con 0x55e021f55dc0 2024-09-19T06:06:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.901+0000 7f33d222f740 10 client.25721 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:06:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.901+0000 7f33d222f740 10 client.25721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.901+0000 7f33d222f740 20 client.25721 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.901+0000 7f33d222f740 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e021f788f0 0x55e021f7ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.901+0000 7f33d222f740 4 client.25721 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:06:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.901+0000 7f33d222f740 1 -- 172.21.15.5:0/77387798 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e021ec2d60 con 0x55e021f788f0 2024-09-19T06:06:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.901+0000 7f33cdbf2640 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e021f788f0 0x55e021f7ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.902+0000 7f33b17fa640 10 client.25721 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.902+0000 7f33cdbf2640 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e021f788f0 0x55e021f7ad10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89259aa063012c1e server_cookie=602cf4004ddc32e5 in_seq=0 out_seq=0 2024-09-19T06:06:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.902+0000 7f33b17fa640 1 -- 172.21.15.5:0/77387798 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e021ec2d60 con 0x55e021f788f0 2024-09-19T06:06:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.902+0000 7f33b17fa640 10 client.25721 handle_command_reply: tid=0 2024-09-19T06:06:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.902+0000 7f33d222f740 1 client.25721 shutdown 2024-09-19T06:06:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.903+0000 7f33d222f740 1 -- 172.21.15.5:0/77387798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f55dc0 msgr2=0x55e021f5d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.903+0000 7f33d222f740 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f55dc0 0x55e021f5d2a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009230 tx=0x7f33bc0317a0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.903+0000 7f33d222f740 1 -- 172.21.15.5:0/77387798 shutdown_connections 2024-09-19T06:06:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.903+0000 7f33d222f740 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e021f788f0 0x55e021f7ad10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.903+0000 7f33d222f740 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f60680 0x55e021f5d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.903+0000 7f33d222f740 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f55dc0 0x55e021f5d2a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.903+0000 7f33b17fa640 0 client.25721 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.903+0000 7f33d222f740 1 --2- 172.21.15.5:0/77387798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f554a0 0x55e021f5fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.903+0000 7f33d222f740 1 -- 172.21.15.5:0/77387798 >> 172.21.15.5:0/77387798 conn(0x55e021f51a30 msgr2=0x55e021f57710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.903+0000 7f33d222f740 1 -- 172.21.15.5:0/77387798 shutdown_connections 2024-09-19T06:06:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.903+0000 7f33d222f740 1 -- 172.21.15.5:0/77387798 wait complete. 2024-09-19T06:06:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.903+0000 7f33d222f740 20 client.25721 trim_cache size 0 max 16384 2024-09-19T06:06:23.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.922+0000 7f33d222f740 1 Processor -- start 2024-09-19T06:06:23.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.922+0000 7f33d222f740 1 -- start start 2024-09-19T06:06:23.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.922+0000 7f33d222f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f554a0 0x55e021f5f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.922+0000 7f33d222f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f55dc0 0x55e021f5db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.922+0000 7f33d222f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f60680 0x55e021f805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.922+0000 7f33d222f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33c81ab4c0 con 0x55e021f554a0 2024-09-19T06:06:23.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.922+0000 7f33d222f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33c81ad060 con 0x55e021f55dc0 2024-09-19T06:06:23.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.922+0000 7f33d222f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33c81abb90 con 0x55e021f60680 2024-09-19T06:06:23.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33cdbf2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f554a0 0x55e021f5f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33cd3f1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f55dc0 0x55e021f5db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33cdbf2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f554a0 0x55e021f5f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40118/0 (socket says 172.21.15.5:40118) 2024-09-19T06:06:23.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33cdbf2640 1 -- 172.21.15.5:0/470051872 learned_addr learned my addr 172.21.15.5:0/470051872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:23.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33ce3f3640 1 --2- 172.21.15.5:0/470051872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f60680 0x55e021f805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33cdbf2640 1 -- 172.21.15.5:0/470051872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f60680 msgr2=0x55e021f805c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:23.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33cdbf2640 1 --2- 172.21.15.5:0/470051872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f60680 0x55e021f805c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33cdbf2640 1 -- 172.21.15.5:0/470051872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f55dc0 msgr2=0x55e021f5db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:23.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33cdbf2640 1 --2- 172.21.15.5:0/470051872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f55dc0 0x55e021f5db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33cdbf2640 1 -- 172.21.15.5:0/470051872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e021f55bf0 con 0x55e021f554a0 2024-09-19T06:06:23.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33ce3f3640 1 --2- 172.21.15.5:0/470051872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f60680 0x55e021f805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:23.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33cd3f1640 1 --2- 172.21.15.5:0/470051872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f55dc0 0x55e021f5db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:23.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.923+0000 7f33cdbf2640 1 --2- 172.21.15.5:0/470051872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f554a0 0x55e021f5f5d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f33c40cd550 tx=0x7f33c4017ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fac5e239c08188 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:23.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.924+0000 7f33b37fe640 1 -- 172.21.15.5:0/470051872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c409c1e0 con 0x55e021f554a0 2024-09-19T06:06:23.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.924+0000 7f33b37fe640 1 -- 172.21.15.5:0/470051872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33c409bae0 con 0x55e021f554a0 2024-09-19T06:06:23.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.924+0000 7f33b37fe640 1 -- 172.21.15.5:0/470051872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c40cd340 con 0x55e021f554a0 2024-09-19T06:06:23.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.925+0000 7f33d222f740 1 -- 172.21.15.5:0/470051872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f554a0 msgr2=0x55e021f5f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:23.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.925+0000 7f33d222f740 1 --2- 172.21.15.5:0/470051872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f554a0 0x55e021f5f5d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f33c40cd550 tx=0x7f33c4017ba0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.925+0000 7f33d222f740 1 -- 172.21.15.5:0/470051872 shutdown_connections 2024-09-19T06:06:23.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.925+0000 7f33d222f740 1 --2- 172.21.15.5:0/470051872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f60680 0x55e021f805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.925+0000 7f33d222f740 1 --2- 172.21.15.5:0/470051872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f55dc0 0x55e021f5db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.925+0000 7f33d222f740 1 --2- 172.21.15.5:0/470051872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f554a0 0x55e021f5f5d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.925+0000 7f33d222f740 1 -- 172.21.15.5:0/470051872 >> 172.21.15.5:0/470051872 conn(0x55e021f51a30 msgr2=0x55e021f50b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:23.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.925+0000 7f33d222f740 1 -- 172.21.15.5:0/470051872 shutdown_connections 2024-09-19T06:06:23.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.925+0000 7f33d222f740 1 -- 172.21.15.5:0/470051872 wait complete. 2024-09-19T06:06:23.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.925+0000 7f33d222f740 1 -- start start 2024-09-19T06:06:23.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.925+0000 7f33d222f740 1 Processor -- start 2024-09-19T06:06:23.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33d222f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f554a0 0x55e021f71140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33d222f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f55dc0 0x55e021f71680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33d222f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f60680 0x55e021f71bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33d222f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33c81ac930 con 0x55e021f55dc0 2024-09-19T06:06:23.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33d222f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33c81ac260 con 0x55e021f60680 2024-09-19T06:06:23.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33d222f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33c81a94a0 con 0x55e021f554a0 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33d222f740 10 client.0 _ll_register_callbacks cb 0x55e021f605d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33cd3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f55dc0 0x55e021f71680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33cd3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f55dc0 0x55e021f71680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40134/0 (socket says 172.21.15.5:40134) 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33cd3f1640 1 -- 172.21.15.5:0/934870920 learned_addr learned my addr 172.21.15.5:0/934870920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33cdbf2640 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f554a0 0x55e021f71140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33ce3f3640 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f60680 0x55e021f71bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33cd3f1640 1 -- 172.21.15.5:0/934870920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f554a0 msgr2=0x55e021f71140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33cd3f1640 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f554a0 0x55e021f71140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33cd3f1640 1 -- 172.21.15.5:0/934870920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f60680 msgr2=0x55e021f71bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33cd3f1640 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f60680 0x55e021f71bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.926+0000 7f33cd3f1640 1 -- 172.21.15.5:0/934870920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e021fe2590 con 0x55e021f55dc0 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.927+0000 7f33cd3f1640 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f55dc0 0x55e021f71680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f33bc02a9a0 tx=0x7f33bc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f202476d229dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:23.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.927+0000 7f33b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.927+0000 7f33b17fa640 1 -- 172.21.15.5:0/934870920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc035040 con 0x55e021f55dc0 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.927+0000 7f33b17fa640 1 -- 172.21.15.5:0/934870920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8506+0+0 (secure 0 0 0) 0x7f33bc003cd0 con 0x55e021f55dc0 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.927+0000 7f33d222f740 1 -- 172.21.15.5:0/934870920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33c409bae0 con 0x55e021f55dc0 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.927+0000 7f33b17fa640 1 -- 172.21.15.5:0/934870920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f33bc032e50 con 0x55e021f55dc0 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.927+0000 7f33d222f740 10 client.16131 fetch_fsmap learned FSMap version 30 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.927+0000 7f33d222f740 1 -- 172.21.15.5:0/934870920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e021fe3850 con 0x55e021f55dc0 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.928+0000 7f33b17fa640 1 -- 172.21.15.5:0/934870920 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33c409bae0 con 0x55e021f55dc0 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.928+0000 7f33b17fa640 1 -- 172.21.15.5:0/934870920 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f33bc0049c0 con 0x55e021f55dc0 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.928+0000 7f33d222f740 10 client.16131 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.928+0000 7f33d222f740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.928+0000 7f33d222f740 20 client.16131 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.928+0000 7f33d222f740 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e021fe4060 0x55e021fe6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:23.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.928+0000 7f33d222f740 4 client.16131 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:06:23.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.928+0000 7f33d222f740 1 -- 172.21.15.5:0/934870920 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e021e3f210 con 0x55e021fe4060 2024-09-19T06:06:23.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.928+0000 7f33cdbf2640 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e021fe4060 0x55e021fe6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:23.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33b17fa640 10 client.16131 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:23.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33cdbf2640 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e021fe4060 0x55e021fe6480 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a0afe12e73a5fc server_cookie=e081bd3e495b2658 in_seq=0 out_seq=0 2024-09-19T06:06:23.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33b17fa640 1 -- 172.21.15.5:0/934870920 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e021e3f210 con 0x55e021fe4060 2024-09-19T06:06:23.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33b17fa640 10 client.16131 handle_command_reply: tid=0 2024-09-19T06:06:23.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33d222f740 1 client.16131 shutdown 2024-09-19T06:06:23.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33d222f740 1 -- 172.21.15.5:0/934870920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f55dc0 msgr2=0x55e021f71680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:23.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33d222f740 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f55dc0 0x55e021f71680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f33bc02a9a0 tx=0x7f33bc005270 comp rx=0 tx=0).stop 2024-09-19T06:06:23.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33d222f740 1 -- 172.21.15.5:0/934870920 shutdown_connections 2024-09-19T06:06:23.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33d222f740 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e021fe4060 0x55e021fe6480 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33d222f740 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e021f60680 0x55e021f71bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33d222f740 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e021f55dc0 0x55e021f71680 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33b17fa640 0 client.16131 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:23.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33d222f740 1 --2- 172.21.15.5:0/934870920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e021f554a0 0x55e021f71140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.929+0000 7f33d222f740 1 -- 172.21.15.5:0/934870920 >> 172.21.15.5:0/934870920 conn(0x55e021f51a30 msgr2=0x55e021f56ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:23.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.930+0000 7f33d222f740 1 -- 172.21.15.5:0/934870920 shutdown_connections 2024-09-19T06:06:23.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.930+0000 7f33d222f740 1 -- 172.21.15.5:0/934870920 wait complete. 2024-09-19T06:06:23.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.930+0000 7f33d222f740 20 client.16131 trim_cache size 0 max 16384 2024-09-19T06:06:23.937 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:06:23.937 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:06:23.937 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:06:23.937 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:06:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.930+0000 7f33cfe56640 1 -- 172.21.15.5:0/225173517 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f33b40a33b0 msgr2=0x7f33b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.930+0000 7f33cfe56640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f33b40a33b0 0x7f33b40a5870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f33c0024000 tx=0x7f33c0022000 comp rx=0 tx=0).stop 2024-09-19T06:06:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.930+0000 7f33cfe56640 1 -- 172.21.15.5:0/225173517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c8100e20 msgr2=0x7f33c81983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.930+0000 7f33cfe56640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c8100e20 0x7f33c81983f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f33c400d950 tx=0x7f33c400de10 comp rx=0 tx=0).stop 2024-09-19T06:06:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.930+0000 7f33cfe56640 1 -- 172.21.15.5:0/225173517 shutdown_connections 2024-09-19T06:06:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.931+0000 7f33cfe56640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33c8102ef0 0x7f33c819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.931+0000 7f33cfe56640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f33b40a33b0 0x7f33b40a5870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.931+0000 7f33cfe56640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33c8102570 0x7f33c8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.931+0000 7f33cfe56640 1 --2- 172.21.15.5:0/225173517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c8100e20 0x7f33c81983f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.931+0000 7f33cfe56640 1 -- 172.21.15.5:0/225173517 >> 172.21.15.5:0/225173517 conn(0x7f33c80fc7d0 msgr2=0x7f33c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.931+0000 7f33cfe56640 1 -- 172.21.15.5:0/225173517 shutdown_connections 2024-09-19T06:06:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:23.931+0000 7f33cfe56640 1 -- 172.21.15.5:0/225173517 wait complete. 2024-09-19T06:06:24.146 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:06:23 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e[64470]: ignoring --setuser ceph since I am not root 2024-09-19T06:06:24.146 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:06:23 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e[64470]: ignoring --setgroup ceph since I am not root 2024-09-19T06:06:24.146 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:06:23 smithi049 ceph-mds[64474]: -- 172.21.15.49:0/596474203 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556de38d9860 con 0x556de38b7000 2024-09-19T06:06:24.146 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:06:23 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e[64470]: starting mds.e at 2024-09-19T06:06:24.720 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:24 smithi005 ceph-mon[30955]: pgmap v94: 97 pgs: 97 active+clean; 48 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.9 MiB/s wr, 194 op/s 2024-09-19T06:06:24.720 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:24 smithi005 ceph-mon[30955]: daemon mds.e restarted 2024-09-19T06:06:24.722 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:06:24.722 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:06:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:24 smithi157 ceph-mon[35346]: pgmap v94: 97 pgs: 97 active+clean; 48 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.9 MiB/s wr, 194 op/s 2024-09-19T06:06:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:24 smithi157 ceph-mon[35346]: daemon mds.e restarted 2024-09-19T06:06:24.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 57446 2.97 MB/sec warmup 110 sec latency 12.650 ms 2024-09-19T06:06:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:24 smithi049 ceph-mon[34640]: pgmap v94: 97 pgs: 97 active+clean; 48 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.9 MiB/s wr, 194 op/s 2024-09-19T06:06:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:24 smithi049 ceph-mon[34640]: daemon mds.e restarted 2024-09-19T06:06:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:06:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.575+0000 7fc178478640 1 -- 172.21.15.5:0/196628176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1701027a0 msgr2=0x7fc1701067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.575+0000 7fc178478640 1 --2- 172.21.15.5:0/196628176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1701027a0 0x7fc1701067f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc160009900 tx=0x7fc16002a410 comp rx=0 tx=0).stop 2024-09-19T06:06:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.575+0000 7fc178478640 1 -- 172.21.15.5:0/196628176 shutdown_connections 2024-09-19T06:06:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.575+0000 7fc178478640 1 --2- 172.21.15.5:0/196628176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc170106f20 0x7fc17010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.575+0000 7fc178478640 1 --2- 172.21.15.5:0/196628176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1701027a0 0x7fc1701067f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.575+0000 7fc178478640 1 --2- 172.21.15.5:0/196628176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc170101df0 0x7fc1701021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.575+0000 7fc178478640 1 -- 172.21.15.5:0/196628176 >> 172.21.15.5:0/196628176 conn(0x7fc1700fd8f0 msgr2=0x7fc1700ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.575+0000 7fc178478640 1 -- 172.21.15.5:0/196628176 shutdown_connections 2024-09-19T06:06:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.575+0000 7fc178478640 1 -- 172.21.15.5:0/196628176 wait complete. 2024-09-19T06:06:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.575+0000 7fc178478640 1 Processor -- start 2024-09-19T06:06:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.576+0000 7fc178478640 1 -- start start 2024-09-19T06:06:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.576+0000 7fc178478640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc170101df0 0x7fc1701983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.576+0000 7fc178478640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1701027a0 0x7fc170198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.576+0000 7fc178478640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc170106f20 0x7fc17019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.576+0000 7fc178478640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc170111e80 con 0x7fc170106f20 2024-09-19T06:06:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.576+0000 7fc178478640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc170111d00 con 0x7fc1701027a0 2024-09-19T06:06:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.576+0000 7fc178478640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc170112000 con 0x7fc170101df0 2024-09-19T06:06:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.576+0000 7fc176214640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc170101df0 0x7fc1701983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.576+0000 7fc175a13640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1701027a0 0x7fc170198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.576+0000 7fc176214640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc170101df0 0x7fc1701983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40380/0 (socket says 172.21.15.5:40380) 2024-09-19T06:06:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.576+0000 7fc176214640 1 -- 172.21.15.5:0/3271683565 learned_addr learned my addr 172.21.15.5:0/3271683565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.576+0000 7fc176a15640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc170106f20 0x7fc17019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.577+0000 7fc175a13640 1 -- 172.21.15.5:0/3271683565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc170101df0 msgr2=0x7fc1701983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.577+0000 7fc175a13640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc170101df0 0x7fc1701983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.577+0000 7fc175a13640 1 -- 172.21.15.5:0/3271683565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc170106f20 msgr2=0x7fc17019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.577+0000 7fc175a13640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc170106f20 0x7fc17019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.577+0000 7fc175a13640 1 -- 172.21.15.5:0/3271683565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17019d370 con 0x7fc1701027a0 2024-09-19T06:06:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.577+0000 7fc176214640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc170101df0 0x7fc1701983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.577+0000 7fc175a13640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1701027a0 0x7fc170198900 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc16002aa00 tx=0x7fc160003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.577+0000 7fc1677fe640 1 -- 172.21.15.5:0/3271683565 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16003f070 con 0x7fc1701027a0 2024-09-19T06:06:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.577+0000 7fc1677fe640 1 -- 172.21.15.5:0/3271683565 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc160031b50 con 0x7fc1701027a0 2024-09-19T06:06:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.577+0000 7fc178478640 1 -- 172.21.15.5:0/3271683565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17019d600 con 0x7fc1701027a0 2024-09-19T06:06:25.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.577+0000 7fc1677fe640 1 -- 172.21.15.5:0/3271683565 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160035030 con 0x7fc1701027a0 2024-09-19T06:06:25.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.577+0000 7fc178478640 1 -- 172.21.15.5:0/3271683565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc17019dab0 con 0x7fc1701027a0 2024-09-19T06:06:25.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.578+0000 7fc178478640 1 -- 172.21.15.5:0/3271683565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1701021d0 con 0x7fc1701027a0 2024-09-19T06:06:25.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.580+0000 7fc1677fe640 1 -- 172.21.15.5:0/3271683565 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc160042070 con 0x7fc1701027a0 2024-09-19T06:06:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.581+0000 7fc1677fe640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc15c0a33b0 0x7fc15c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.581+0000 7fc176214640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc15c0a33b0 0x7fc15c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.581+0000 7fc1677fe640 1 -- 172.21.15.5:0/3271683565 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fc1600f00b0 con 0x7fc1701027a0 2024-09-19T06:06:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.582+0000 7fc176214640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc15c0a33b0 0x7fc15c0a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc16c011500 tx=0x7fc16c005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.585+0000 7fc1677fe640 1 -- 172.21.15.5:0/3271683565 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc1600bc450 con 0x7fc1701027a0 2024-09-19T06:06:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:25 smithi157 ceph-mon[35346]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T06:06:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:25 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.49:6834/1729172575,v1:172.21.15.49:6835/1729172575] up:boot 2024-09-19T06:06:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:25 smithi157 ceph-mon[35346]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T06:06:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:25 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:06:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:25 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:06:25.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 57963 2.96 MB/sec warmup 111 sec latency 17.078 ms 2024-09-19T06:06:25.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.823+0000 7fc178478640 1 -- 172.21.15.5:0/3271683565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc17010a160 con 0x7fc1701027a0 2024-09-19T06:06:25.825 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:25 smithi005 ceph-mon[30955]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T06:06:25.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:25 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.49:6834/1729172575,v1:172.21.15.49:6835/1729172575] up:boot 2024-09-19T06:06:25.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:25 smithi005 ceph-mon[30955]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T06:06:25.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:25 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:06:25.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:25 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:06:25.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.823+0000 7fc1677fe640 1 -- 172.21.15.5:0/3271683565 <== mon.2 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11962 (secure 0 0 0) 0x7fc1600c0750 con 0x7fc1701027a0 2024-09-19T06:06:25.827 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:06:25.827 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"btime":"2024-09-19T06:06:24:465242+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:06:24.465223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:06:25.827 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-09-19T06:06:25.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.826+0000 7fc178478640 1 -- 172.21.15.5:0/3271683565 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc15c0a33b0 msgr2=0x7fc15c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:25.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.826+0000 7fc178478640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc15c0a33b0 0x7fc15c0a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc16c011500 tx=0x7fc16c005f90 comp rx=0 tx=0).stop 2024-09-19T06:06:25.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.826+0000 7fc178478640 1 -- 172.21.15.5:0/3271683565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1701027a0 msgr2=0x7fc170198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:25.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.826+0000 7fc178478640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1701027a0 0x7fc170198900 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc16002aa00 tx=0x7fc160003b90 comp rx=0 tx=0).stop 2024-09-19T06:06:25.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.826+0000 7fc178478640 1 -- 172.21.15.5:0/3271683565 shutdown_connections 2024-09-19T06:06:25.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.826+0000 7fc178478640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc170106f20 0x7fc17019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:25.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.826+0000 7fc178478640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1701027a0 0x7fc170198900 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:25.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.826+0000 7fc178478640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc15c0a33b0 0x7fc15c0a5870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:25.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.826+0000 7fc178478640 1 --2- 172.21.15.5:0/3271683565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc170101df0 0x7fc1701983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:25.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.826+0000 7fc178478640 1 -- 172.21.15.5:0/3271683565 >> 172.21.15.5:0/3271683565 conn(0x7fc1700fd8f0 msgr2=0x7fc1700fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:25.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.826+0000 7fc178478640 1 -- 172.21.15.5:0/3271683565 shutdown_connections 2024-09-19T06:06:25.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:25.826+0000 7fc178478640 1 -- 172.21.15.5:0/3271683565 wait complete. 2024-09-19T06:06:25.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:25 smithi049 ceph-mon[34640]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T06:06:25.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:25 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.49:6834/1729172575,v1:172.21.15.49:6835/1729172575] up:boot 2024-09-19T06:06:25.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:25 smithi049 ceph-mon[34640]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T06:06:25.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:25 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:06:25.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:25 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:06:26.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:06:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:26 smithi157 ceph-mon[35346]: pgmap v96: 97 pgs: 97 active+clean; 48 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-19T06:06:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:26 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3271683565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:06:26.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 58608 2.97 MB/sec warmup 112 sec latency 14.372 ms 2024-09-19T06:06:26.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:26 smithi005 ceph-mon[30955]: pgmap v96: 97 pgs: 97 active+clean; 48 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-19T06:06:26.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:26 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3271683565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:06:26.887 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:06:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:26 smithi049 ceph-mon[34640]: pgmap v96: 97 pgs: 97 active+clean; 48 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-19T06:06:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:26 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3271683565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:06:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.394+0000 7efee6d18640 1 -- 172.21.15.5:0/2474305515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0103760 msgr2=0x7efee0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.394+0000 7efee6d18640 1 --2- 172.21.15.5:0/2474305515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0103760 0x7efee0107ba0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efecc009930 tx=0x7efecc02a420 comp rx=0 tx=0).stop 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.394+0000 7efee6d18640 1 -- 172.21.15.5:0/2474305515 shutdown_connections 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.394+0000 7efee6d18640 1 --2- 172.21.15.5:0/2474305515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efee01082e0 0x7efee010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.394+0000 7efee6d18640 1 --2- 172.21.15.5:0/2474305515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0103760 0x7efee0107ba0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.394+0000 7efee6d18640 1 --2- 172.21.15.5:0/2474305515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efee0102db0 0x7efee0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.394+0000 7efee6d18640 1 -- 172.21.15.5:0/2474305515 >> 172.21.15.5:0/2474305515 conn(0x7efee00fc7d0 msgr2=0x7efee00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.394+0000 7efee6d18640 1 -- 172.21.15.5:0/2474305515 shutdown_connections 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.394+0000 7efee6d18640 1 -- 172.21.15.5:0/2474305515 wait complete. 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efee6d18640 1 Processor -- start 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efee6d18640 1 -- start start 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efee6d18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0102db0 0x7efee0100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efee6d18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efee0103760 0x7efee0100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efee6d18640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efee01082e0 0x7efee0101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efee6d18640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efee0111f20 con 0x7efee0102db0 2024-09-19T06:06:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efee6d18640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efee0111da0 con 0x7efee0103760 2024-09-19T06:06:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efee6d18640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efee01120a0 con 0x7efee01082e0 2024-09-19T06:06:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efee4ab4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0102db0 0x7efee0100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efee4ab4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0102db0 0x7efee0100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34242/0 (socket says 172.21.15.5:34242) 2024-09-19T06:06:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efee4ab4640 1 -- 172.21.15.5:0/1200151470 learned_addr learned my addr 172.21.15.5:0/1200151470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efee52b5640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efee01082e0 0x7efee0101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.395+0000 7efedffff640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efee0103760 0x7efee0100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.396+0000 7efee4ab4640 1 -- 172.21.15.5:0/1200151470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efee01082e0 msgr2=0x7efee0101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.396+0000 7efee4ab4640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efee01082e0 0x7efee0101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.396+0000 7efee4ab4640 1 -- 172.21.15.5:0/1200151470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efee0103760 msgr2=0x7efee0100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.396+0000 7efee4ab4640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efee0103760 0x7efee0100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.396+0000 7efee4ab4640 1 -- 172.21.15.5:0/1200151470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee01a6a50 con 0x7efee0102db0 2024-09-19T06:06:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.396+0000 7efee52b5640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efee01082e0 0x7efee0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.396+0000 7efee4ab4640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0102db0 0x7efee0100a10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efed400e9f0 tx=0x7efed400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.396+0000 7efeddffb640 1 -- 172.21.15.5:0/1200151470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed400cd80 con 0x7efee0102db0 2024-09-19T06:06:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.396+0000 7efeddffb640 1 -- 172.21.15.5:0/1200151470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed40108e0 con 0x7efee0102db0 2024-09-19T06:06:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.396+0000 7efee6d18640 1 -- 172.21.15.5:0/1200151470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee01a6d40 con 0x7efee0102db0 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.396+0000 7efeddffb640 1 -- 172.21.15.5:0/1200151470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4018920 con 0x7efee0102db0 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.396+0000 7efee6d18640 1 -- 172.21.15.5:0/1200151470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01a71f0 con 0x7efee0102db0 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efeddffb640 1 -- 172.21.15.5:0/1200151470 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7efed4010a90 con 0x7efee0102db0 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efee90f1740 1 Processor -- start 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efee90f1740 1 -- start start 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efee90f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 0x55c13926f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efee90f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c13926fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efee90f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13927a370 0x55c13927c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efee90f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c1391d52f0 con 0x55c139270b40 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efee90f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c13917ccf0 con 0x55c13927a370 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efee90f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c1391d12b0 con 0x55c13926f5d0 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efee52b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 0x55c13926f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efee52b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 0x55c13926f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34254/0 (socket says 172.21.15.5:34254) 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efee52b5640 1 -- 172.21.15.5:0/1376045727 learned_addr learned my addr 172.21.15.5:0/1376045727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efedffff640 1 --2- 172.21.15.5:0/1376045727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13927a370 0x55c13927c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.400+0000 7efee4ab4640 1 --2- 172.21.15.5:0/1376045727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c13926fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efee52b5640 1 -- 172.21.15.5:0/1376045727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 msgr2=0x55c13926fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efee52b5640 1 --2- 172.21.15.5:0/1376045727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c13926fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efee52b5640 1 -- 172.21.15.5:0/1376045727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13927a370 msgr2=0x55c13927c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efee52b5640 1 --2- 172.21.15.5:0/1376045727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13927a370 0x55c13927c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efee52b5640 1 -- 172.21.15.5:0/1376045727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1391613a0 con 0x55c139270b40 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efedffff640 1 --2- 172.21.15.5:0/1376045727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13927a370 0x55c13927c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efee4ab4640 1 --2- 172.21.15.5:0/1376045727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c13926fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efee52b5640 1 --2- 172.21.15.5:0/1376045727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 0x55c13926f090 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efed8011790 tx=0x7efed8011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ddf8ae686e99ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efeddffb640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efed00a33b0 0x7efed00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efee4ab4640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efed00a33b0 0x7efed00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efecaffd640 1 -- 172.21.15.5:0/1376045727 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8003c30 con 0x55c139270b40 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efecaffd640 1 -- 172.21.15.5:0/1376045727 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed800fe50 con 0x55c139270b40 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efeddffb640 1 -- 172.21.15.5:0/1200151470 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7efed40cffe0 con 0x7efee0102db0 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.401+0000 7efecaffd640 1 -- 172.21.15.5:0/1376045727 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed80193e0 con 0x55c139270b40 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.402+0000 7efee90f1740 1 -- 172.21.15.5:0/1376045727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 msgr2=0x55c13926f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.402+0000 7efee90f1740 1 --2- 172.21.15.5:0/1376045727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 0x55c13926f090 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efed8011790 tx=0x7efed8011c50 comp rx=0 tx=0).stop 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.402+0000 7efee4ab4640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efed00a33b0 0x7efed00a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efed40cd7f0 tx=0x7efed4000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.402+0000 7efee90f1740 1 -- 172.21.15.5:0/1376045727 shutdown_connections 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.402+0000 7efee90f1740 1 --2- 172.21.15.5:0/1376045727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13927a370 0x55c13927c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.402+0000 7efee90f1740 1 --2- 172.21.15.5:0/1376045727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c13926fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.402+0000 7efee90f1740 1 --2- 172.21.15.5:0/1376045727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 0x55c13926f090 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.402+0000 7efee90f1740 1 -- 172.21.15.5:0/1376045727 >> 172.21.15.5:0/1376045727 conn(0x55c139238310 msgr2=0x55c13926bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.402+0000 7efee90f1740 1 -- 172.21.15.5:0/1376045727 shutdown_connections 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.402+0000 7efee90f1740 1 -- 172.21.15.5:0/1376045727 wait complete. 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.402+0000 7efee90f1740 1 -- start start 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efee90f1740 1 Processor -- start 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efee90f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c139279ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efee90f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 0x55c139276f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efee90f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13927a370 0x55c1392774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efee90f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c13927d110 con 0x55c139270b40 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efee90f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c139190060 con 0x55c13927a370 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efee90f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c13927d290 con 0x55c13926f5d0 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efee90f1740 10 client.0 _ll_register_callbacks cb 0x55c138ee38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efedffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 0x55c139276f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efee52b5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13927a370 0x55c1392774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efedffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 0x55c139276f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34266/0 (socket says 172.21.15.5:34266) 2024-09-19T06:06:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efedffff640 1 -- 172.21.15.5:0/797364553 learned_addr learned my addr 172.21.15.5:0/797364553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.403+0000 7efee52b5640 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c139279ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efedffff640 1 -- 172.21.15.5:0/797364553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 msgr2=0x55c139279ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efedffff640 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c139279ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efedffff640 1 -- 172.21.15.5:0/797364553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13927a370 msgr2=0x55c1392774f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efedffff640 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13927a370 0x55c1392774f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efedffff640 1 -- 172.21.15.5:0/797364553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c138f463c0 con 0x55c139270b40 2024-09-19T06:06:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efee52b5640 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13927a370 0x55c1392774f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efedffff640 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 0x55c139276f90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efecc000c00 tx=0x7efecc032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfb15df46c935b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efebffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:06:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efebffff640 1 -- 172.21.15.5:0/797364553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc035040 con 0x55c139270b40 2024-09-19T06:06:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efebffff640 1 -- 172.21.15.5:0/797364553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7efecc0316f0 con 0x55c139270b40 2024-09-19T06:06:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efee90f1740 1 -- 172.21.15.5:0/797364553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed800fe50 con 0x55c139270b40 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efebffff640 1 -- 172.21.15.5:0/797364553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efecc004040 con 0x55c139270b40 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efee90f1740 10 client.16149 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efee90f1740 1 -- 172.21.15.5:0/797364553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c139288b20 con 0x55c139270b40 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.404+0000 7efebffff640 1 -- 172.21.15.5:0/797364553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed800fe50 con 0x55c139270b40 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.405+0000 7efebffff640 1 -- 172.21.15.5:0/797364553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7efecc004770 con 0x55c139270b40 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.405+0000 7efee90f1740 10 client.16149 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.405+0000 7efee90f1740 10 client.16149 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.405+0000 7efee90f1740 20 client.16149 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.405+0000 7efee90f1740 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c139289fe0 0x55c13928c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.405+0000 7efee90f1740 4 client.16149 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.405+0000 7efee90f1740 1 -- 172.21.15.5:0/797364553 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1391e2860 con 0x55c139289fe0 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.405+0000 7efee52b5640 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c139289fe0 0x55c13928c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.405+0000 7efee52b5640 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c139289fe0 0x55c13928c400 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a6b00ee846a05d8 server_cookie=c2c8dfc09893876f in_seq=0 out_seq=0 2024-09-19T06:06:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.405+0000 7efebffff640 10 client.16149 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efebffff640 1 -- 172.21.15.5:0/797364553 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1391e2860 con 0x55c139289fe0 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efebffff640 10 client.16149 handle_command_reply: tid=0 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efee90f1740 1 client.16149 shutdown 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efee90f1740 1 -- 172.21.15.5:0/797364553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 msgr2=0x55c139276f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efee90f1740 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 0x55c139276f90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efecc000c00 tx=0x7efecc032d70 comp rx=0 tx=0).stop 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efee90f1740 1 -- 172.21.15.5:0/797364553 shutdown_connections 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efee90f1740 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c139289fe0 0x55c13928c400 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efee90f1740 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13927a370 0x55c1392774f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efee90f1740 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c139270b40 0x55c139276f90 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efee90f1740 1 --2- 172.21.15.5:0/797364553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c139279ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efebffff640 0 client.16149 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efee90f1740 1 -- 172.21.15.5:0/797364553 >> 172.21.15.5:0/797364553 conn(0x55c139238310 msgr2=0x55c139271930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efee90f1740 1 -- 172.21.15.5:0/797364553 shutdown_connections 2024-09-19T06:06:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efee90f1740 1 -- 172.21.15.5:0/797364553 wait complete. 2024-09-19T06:06:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.406+0000 7efee90f1740 20 client.16149 trim_cache size 0 max 16384 2024-09-19T06:06:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee90f1740 1 Processor -- start 2024-09-19T06:06:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee90f1740 1 -- start start 2024-09-19T06:06:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee90f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c139276f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee90f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c139270b40 0x55c1392863b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee90f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 0x55c139291cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee90f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efee01afaa0 con 0x55c13927a370 2024-09-19T06:06:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee90f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b1640 con 0x55c139270b40 2024-09-19T06:06:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee90f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b0170 con 0x55c13926f5d0 2024-09-19T06:06:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee52b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 0x55c139291cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee52b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 0x55c139291cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34282/0 (socket says 172.21.15.5:34282) 2024-09-19T06:06:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee52b5640 1 -- 172.21.15.5:0/3050308172 learned_addr learned my addr 172.21.15.5:0/3050308172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee52b5640 1 --2- 172.21.15.5:0/3050308172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c139276f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efedffff640 1 --2- 172.21.15.5:0/3050308172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c139270b40 0x55c1392863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee52b5640 1 -- 172.21.15.5:0/3050308172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 msgr2=0x55c139276f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee52b5640 1 --2- 172.21.15.5:0/3050308172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c139276f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee52b5640 1 -- 172.21.15.5:0/3050308172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c139270b40 msgr2=0x55c1392863b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee52b5640 1 --2- 172.21.15.5:0/3050308172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c139270b40 0x55c1392863b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.422+0000 7efee52b5640 1 -- 172.21.15.5:0/3050308172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c13926bed0 con 0x55c13927a370 2024-09-19T06:06:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.423+0000 7efee52b5640 1 --2- 172.21.15.5:0/3050308172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 0x55c139291cd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efed8004620 tx=0x7efed8011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23a911d5d338392 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.423+0000 7efeca7fc640 1 -- 172.21.15.5:0/3050308172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8003a30 con 0x55c13927a370 2024-09-19T06:06:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.423+0000 7efeca7fc640 1 -- 172.21.15.5:0/3050308172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed800fe50 con 0x55c13927a370 2024-09-19T06:06:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.423+0000 7efeca7fc640 1 -- 172.21.15.5:0/3050308172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed801a910 con 0x55c13927a370 2024-09-19T06:06:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.424+0000 7efee90f1740 1 -- 172.21.15.5:0/3050308172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 msgr2=0x55c139291cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.424+0000 7efee90f1740 1 --2- 172.21.15.5:0/3050308172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 0x55c139291cd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efed8004620 tx=0x7efed8011c50 comp rx=0 tx=0).stop 2024-09-19T06:06:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.424+0000 7efee90f1740 1 -- 172.21.15.5:0/3050308172 shutdown_connections 2024-09-19T06:06:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.424+0000 7efee90f1740 1 --2- 172.21.15.5:0/3050308172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 0x55c139291cd0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.424+0000 7efee90f1740 1 --2- 172.21.15.5:0/3050308172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c139270b40 0x55c1392863b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.424+0000 7efee90f1740 1 --2- 172.21.15.5:0/3050308172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c13926f5d0 0x55c139276f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.424+0000 7efee90f1740 1 -- 172.21.15.5:0/3050308172 >> 172.21.15.5:0/3050308172 conn(0x55c139238310 msgr2=0x55c139276850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.424+0000 7efee90f1740 1 -- 172.21.15.5:0/3050308172 shutdown_connections 2024-09-19T06:06:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.424+0000 7efee90f1740 1 -- 172.21.15.5:0/3050308172 wait complete. 2024-09-19T06:06:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.424+0000 7efee90f1740 1 -- start start 2024-09-19T06:06:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.424+0000 7efee90f1740 1 Processor -- start 2024-09-19T06:06:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee90f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13926f5d0 0x55c139282520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee90f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c139270b40 0x55c139282a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee90f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 0x55c139273200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee90f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b0f70 con 0x55c13927a370 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee90f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b0840 con 0x55c13926f5d0 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee90f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efee01ada80 con 0x55c139270b40 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee90f1740 10 client.0 _ll_register_callbacks cb 0x7efecc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee52b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 0x55c139273200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee52b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 0x55c139273200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34286/0 (socket says 172.21.15.5:34286) 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee52b5640 1 -- 172.21.15.5:0/3311741073 learned_addr learned my addr 172.21.15.5:0/3311741073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efedffff640 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c139270b40 0x55c139282a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee52b5640 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13926f5d0 0x55c139282520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee52b5640 1 -- 172.21.15.5:0/3311741073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c139270b40 msgr2=0x55c139282a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee52b5640 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c139270b40 0x55c139282a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee52b5640 1 -- 172.21.15.5:0/3311741073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13926f5d0 msgr2=0x55c139282520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee52b5640 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13926f5d0 0x55c139282520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.425+0000 7efee52b5640 1 -- 172.21.15.5:0/3311741073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c139274da0 con 0x55c13927a370 2024-09-19T06:06:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efee52b5640 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 0x55c139273200 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efed800f8d0 tx=0x7efed80114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb0b1182bb68ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efebffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:06:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efebffff640 1 -- 172.21.15.5:0/3311741073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8003cf0 con 0x55c13927a370 2024-09-19T06:06:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efee90f1740 1 -- 172.21.15.5:0/3311741073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed800fe50 con 0x55c13927a370 2024-09-19T06:06:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efebffff640 1 -- 172.21.15.5:0/3311741073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7efed8004840 con 0x55c13927a370 2024-09-19T06:06:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efebffff640 1 -- 172.21.15.5:0/3311741073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efed801a440 con 0x55c13927a370 2024-09-19T06:06:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efee90f1740 10 client.16155 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efee90f1740 1 -- 172.21.15.5:0/3311741073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c1392f4080 con 0x55c13927a370 2024-09-19T06:06:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efebffff640 1 -- 172.21.15.5:0/3311741073 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed800fe50 con 0x55c13927a370 2024-09-19T06:06:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efebffff640 1 -- 172.21.15.5:0/3311741073 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7efed8042050 con 0x55c13927a370 2024-09-19T06:06:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efee90f1740 10 client.16155 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efee90f1740 10 client.16155 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:06:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efee90f1740 20 client.16155 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efee90f1740 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c1392f45a0 0x55c1392f6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efee90f1740 4 client.16155 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:06:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.426+0000 7efee90f1740 1 -- 172.21.15.5:0/3311741073 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1391dcd70 con 0x55c1392f45a0 2024-09-19T06:06:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.427+0000 7efedffff640 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c1392f45a0 0x55c1392f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.427+0000 7efedffff640 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c1392f45a0 0x55c1392f6990 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=acd9c9b1692ce04d server_cookie=abfb0a71acebd6e8 in_seq=0 out_seq=0 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.427+0000 7efebffff640 10 client.16155 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.427+0000 7efebffff640 1 -- 172.21.15.5:0/3311741073 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1391dcd70 con 0x55c1392f45a0 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.427+0000 7efebffff640 10 client.16155 handle_command_reply: tid=0 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.427+0000 7efee90f1740 1 client.16155 shutdown 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.427+0000 7efee90f1740 1 -- 172.21.15.5:0/3311741073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 msgr2=0x55c139273200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.427+0000 7efee90f1740 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 0x55c139273200 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efed800f8d0 tx=0x7efed80114c0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.427+0000 7efee90f1740 1 -- 172.21.15.5:0/3311741073 shutdown_connections 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.427+0000 7efee90f1740 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55c1392f45a0 0x55c1392f6990 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.427+0000 7efee90f1740 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c13927a370 0x55c139273200 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.427+0000 7efebffff640 0 client.16155 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.428+0000 7efee90f1740 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c139270b40 0x55c139282a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.428+0000 7efee90f1740 1 --2- 172.21.15.5:0/3311741073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c13926f5d0 0x55c139282520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.428+0000 7efee90f1740 1 -- 172.21.15.5:0/3311741073 >> 172.21.15.5:0/3311741073 conn(0x55c139238310 msgr2=0x55c139276080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.428+0000 7efee90f1740 1 -- 172.21.15.5:0/3311741073 shutdown_connections 2024-09-19T06:06:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.428+0000 7efee90f1740 1 -- 172.21.15.5:0/3311741073 wait complete. 2024-09-19T06:06:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.428+0000 7efee90f1740 20 client.16155 trim_cache size 0 max 16384 2024-09-19T06:06:27.437 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:06:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.429+0000 7efee6d18640 1 -- 172.21.15.5:0/1200151470 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efed00a33b0 msgr2=0x7efed00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.429+0000 7efee6d18640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efed00a33b0 0x7efed00a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efed40cd7f0 tx=0x7efed4000bc0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.429+0000 7efee6d18640 1 -- 172.21.15.5:0/1200151470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0102db0 msgr2=0x7efee0100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.429+0000 7efee6d18640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0102db0 0x7efee0100a10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efed400e9f0 tx=0x7efed400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.429+0000 7efee6d18640 1 -- 172.21.15.5:0/1200151470 shutdown_connections 2024-09-19T06:06:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.429+0000 7efee6d18640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efed00a33b0 0x7efed00a5870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.429+0000 7efee6d18640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efee01082e0 0x7efee0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.429+0000 7efee6d18640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efee0103760 0x7efee0100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.429+0000 7efee6d18640 1 --2- 172.21.15.5:0/1200151470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0102db0 0x7efee0100a10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.429+0000 7efee6d18640 1 -- 172.21.15.5:0/1200151470 >> 172.21.15.5:0/1200151470 conn(0x7efee00fc7d0 msgr2=0x7efee0105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.429+0000 7efee6d18640 1 -- 172.21.15.5:0/1200151470 shutdown_connections 2024-09-19T06:06:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:27.429+0000 7efee6d18640 1 -- 172.21.15.5:0/1200151470 wait complete. 2024-09-19T06:06:27.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 59191 2.96 MB/sec warmup 113 sec latency 15.189 ms 2024-09-19T06:06:28.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:06:28.477 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:06:28.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:28 smithi005 ceph-mon[30955]: pgmap v97: 97 pgs: 97 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-19T06:06:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:28 smithi157 ceph-mon[35346]: pgmap v97: 97 pgs: 97 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-19T06:06:28.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 59943 2.98 MB/sec warmup 114 sec latency 18.501 ms 2024-09-19T06:06:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:28 smithi049 ceph-mon[34640]: pgmap v97: 97 pgs: 97 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-19T06:06:29.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.018+0000 7f7f9e239640 1 -- 172.21.15.5:0/1080496758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106f10 msgr2=0x7f7f9810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.018+0000 7f7f9e239640 1 --2- 172.21.15.5:0/1080496758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106f10 0x7f7f9810ea90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009ca0 tx=0x7f7f9002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.018+0000 7f7f9e239640 1 -- 172.21.15.5:0/1080496758 shutdown_connections 2024-09-19T06:06:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.018+0000 7f7f9e239640 1 --2- 172.21.15.5:0/1080496758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106f10 0x7f7f9810ea90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.018+0000 7f7f9e239640 1 --2- 172.21.15.5:0/1080496758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f98102790 0x7f7f981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.018+0000 7f7f9e239640 1 --2- 172.21.15.5:0/1080496758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98101de0 0x7f7f981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.018+0000 7f7f9e239640 1 -- 172.21.15.5:0/1080496758 >> 172.21.15.5:0/1080496758 conn(0x7f7f980fd880 msgr2=0x7f7f980ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.018+0000 7f7f9e239640 1 -- 172.21.15.5:0/1080496758 shutdown_connections 2024-09-19T06:06:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.018+0000 7f7f9e239640 1 -- 172.21.15.5:0/1080496758 wait complete. 2024-09-19T06:06:29.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.019+0000 7f7f9e239640 1 Processor -- start 2024-09-19T06:06:29.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.019+0000 7f7f9e239640 1 -- start start 2024-09-19T06:06:29.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f9e239640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98101de0 0x7f7f9819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f9e239640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f98102790 0x7f7f9819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f9e239640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106f10 0x7f7f9819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f9e239640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98111e20 con 0x7f7f98106f10 2024-09-19T06:06:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f9e239640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98111ca0 con 0x7f7f98101de0 2024-09-19T06:06:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f9e239640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98111fa0 con 0x7f7f98102790 2024-09-19T06:06:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98101de0 0x7f7f9819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f96ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f98102790 0x7f7f9819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106f10 0x7f7f9819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98101de0 0x7f7f9819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48014/0 (socket says 172.21.15.5:48014) 2024-09-19T06:06:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f977fe640 1 -- 172.21.15.5:0/865920439 learned_addr learned my addr 172.21.15.5:0/865920439 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f977fe640 1 -- 172.21.15.5:0/865920439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f98102790 msgr2=0x7f7f9819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f977fe640 1 --2- 172.21.15.5:0/865920439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f98102790 0x7f7f9819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f977fe640 1 -- 172.21.15.5:0/865920439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106f10 msgr2=0x7f7f9819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f977fe640 1 --2- 172.21.15.5:0/865920439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106f10 0x7f7f9819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.020+0000 7f7f977fe640 1 -- 172.21.15.5:0/865920439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9819f480 con 0x7f7f98101de0 2024-09-19T06:06:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.021+0000 7f7f977fe640 1 --2- 172.21.15.5:0/865920439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98101de0 0x7f7f9819a4d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800e9f0 tx=0x7f7f8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.021+0000 7f7f94ff9640 1 -- 172.21.15.5:0/865920439 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8800cd80 con 0x7f7f98101de0 2024-09-19T06:06:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.021+0000 7f7f9e239640 1 -- 172.21.15.5:0/865920439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f9819f770 con 0x7f7f98101de0 2024-09-19T06:06:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.021+0000 7f7f94ff9640 1 -- 172.21.15.5:0/865920439 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f880108e0 con 0x7f7f98101de0 2024-09-19T06:06:29.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.021+0000 7f7f94ff9640 1 -- 172.21.15.5:0/865920439 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88018920 con 0x7f7f98101de0 2024-09-19T06:06:29.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.021+0000 7f7f9e239640 1 -- 172.21.15.5:0/865920439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f981a7050 con 0x7f7f98101de0 2024-09-19T06:06:29.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7fa0612740 1 Processor -- start 2024-09-19T06:06:29.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7fa0612740 1 -- start start 2024-09-19T06:06:29.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7f94ff9640 1 -- 172.21.15.5:0/865920439 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7f88010a90 con 0x7f7f98101de0 2024-09-19T06:06:29.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7fa0612740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2db40 0x56068cd2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7fa0612740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 0x56068cd2ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7fa0612740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd37370 0x56068cd39760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7fa0612740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56068cc922f0 con 0x56068cd2c5d0 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7fa0612740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56068cc39cf0 con 0x56068cd2db40 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7fa0612740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56068cc8e2b0 con 0x56068cd37370 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2db40 0x56068cd2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2db40 0x56068cd2c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48028/0 (socket says 172.21.15.5:48028) 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7f97fff640 1 -- 172.21.15.5:0/190512939 learned_addr learned my addr 172.21.15.5:0/190512939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7f96ffd640 1 --2- 172.21.15.5:0/190512939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd37370 0x56068cd39760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7f977fe640 1 --2- 172.21.15.5:0/190512939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 0x56068cd2ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7f97fff640 1 -- 172.21.15.5:0/190512939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd37370 msgr2=0x56068cd39760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7f97fff640 1 --2- 172.21.15.5:0/190512939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd37370 0x56068cd39760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7f97fff640 1 -- 172.21.15.5:0/190512939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 msgr2=0x56068cd2ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7f97fff640 1 --2- 172.21.15.5:0/190512939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 0x56068cd2ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7f97fff640 1 -- 172.21.15.5:0/190512939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56068cc1e3a0 con 0x56068cd2db40 2024-09-19T06:06:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.024+0000 7f7f977fe640 1 --2- 172.21.15.5:0/190512939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 0x56068cd2ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:06:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.025+0000 7f7f94ff9640 1 --2- 172.21.15.5:0/865920439 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7f800a33b0 0x7f7f800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.025+0000 7f7f97fff640 1 --2- 172.21.15.5:0/190512939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2db40 0x56068cd2c090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009ca0 tx=0x7f7f90009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34c51b129dd50446 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.025+0000 7f7f977fe640 1 --2- 172.21.15.5:0/865920439 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7f800a33b0 0x7f7f800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.025+0000 7f7f94ff9640 1 -- 172.21.15.5:0/865920439 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f7f880cffe0 con 0x7f7f98101de0 2024-09-19T06:06:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.025+0000 7f7f7dffb640 1 -- 172.21.15.5:0/190512939 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90031d00 con 0x56068cd2db40 2024-09-19T06:06:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.025+0000 7f7f7dffb640 1 -- 172.21.15.5:0/190512939 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f90036030 con 0x56068cd2db40 2024-09-19T06:06:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.025+0000 7f7f7dffb640 1 -- 172.21.15.5:0/190512939 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f900042a0 con 0x56068cd2db40 2024-09-19T06:06:29.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.026+0000 7f7fa0612740 1 -- 172.21.15.5:0/190512939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2db40 msgr2=0x56068cd2c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:29.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.026+0000 7f7f977fe640 1 --2- 172.21.15.5:0/865920439 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7f800a33b0 0x7f7f800a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7f880cd7f0 tx=0x7f7f88000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:29.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.026+0000 7f7fa0612740 1 --2- 172.21.15.5:0/190512939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2db40 0x56068cd2c090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009ca0 tx=0x7f7f90009a20 comp rx=0 tx=0).stop 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.026+0000 7f7fa0612740 1 -- 172.21.15.5:0/190512939 shutdown_connections 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.026+0000 7f7fa0612740 1 --2- 172.21.15.5:0/190512939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd37370 0x56068cd39760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.026+0000 7f7fa0612740 1 --2- 172.21.15.5:0/190512939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 0x56068cd2ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.026+0000 7f7fa0612740 1 --2- 172.21.15.5:0/190512939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2db40 0x56068cd2c090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.026+0000 7f7fa0612740 1 -- 172.21.15.5:0/190512939 >> 172.21.15.5:0/190512939 conn(0x56068ccf5310 msgr2=0x56068cd28c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.026+0000 7f7fa0612740 1 -- 172.21.15.5:0/190512939 shutdown_connections 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.026+0000 7f7fa0612740 1 -- 172.21.15.5:0/190512939 wait complete. 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7fa0612740 1 -- start start 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7fa0612740 1 Processor -- start 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7fa0612740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2c5d0 0x56068cd36cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7fa0612740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2db40 0x56068cd33ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7fa0612740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd37370 0x56068cd34600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7fa0612740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56068cd39e90 con 0x56068cd2db40 2024-09-19T06:06:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7fa0612740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56068cc4d060 con 0x56068cd2c5d0 2024-09-19T06:06:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7fa0612740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56068cd3a010 con 0x56068cd37370 2024-09-19T06:06:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7fa0612740 10 client.0 _ll_register_callbacks cb 0x56068c9a08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7f96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2db40 0x56068cd33ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2c5d0 0x56068cd36cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7f96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2db40 0x56068cd33ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34324/0 (socket says 172.21.15.5:34324) 2024-09-19T06:06:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7f96ffd640 1 -- 172.21.15.5:0/1836033695 learned_addr learned my addr 172.21.15.5:0/1836033695 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.027+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1836033695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd37370 0x56068cd34600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7f96ffd640 1 -- 172.21.15.5:0/1836033695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd37370 msgr2=0x56068cd34600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7f96ffd640 1 --2- 172.21.15.5:0/1836033695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd37370 0x56068cd34600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7f96ffd640 1 -- 172.21.15.5:0/1836033695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2c5d0 msgr2=0x56068cd36cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7f96ffd640 1 --2- 172.21.15.5:0/1836033695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2c5d0 0x56068cd36cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7f96ffd640 1 -- 172.21.15.5:0/1836033695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56068ca033c0 con 0x56068cd2db40 2024-09-19T06:06:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7f96ffd640 1 --2- 172.21.15.5:0/1836033695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2db40 0x56068cd33ff0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c012700 tx=0x7f7f8c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f9bec212d0cd68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7f76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:06:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7f76ffd640 1 -- 172.21.15.5:0/1836033695 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00f8c0 con 0x56068cd2db40 2024-09-19T06:06:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7f76ffd640 1 -- 172.21.15.5:0/1836033695 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f7f8c00fa60 con 0x56068cd2db40 2024-09-19T06:06:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7fa0612740 1 -- 172.21.15.5:0/1836033695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f90036030 con 0x56068cd2db40 2024-09-19T06:06:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7f76ffd640 1 -- 172.21.15.5:0/1836033695 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7f8c00fc60 con 0x56068cd2db40 2024-09-19T06:06:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7fa0612740 10 client.16164 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.028+0000 7f7fa0612740 1 -- 172.21.15.5:0/1836033695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56068cd45920 con 0x56068cd2db40 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.029+0000 7f7f76ffd640 1 -- 172.21.15.5:0/1836033695 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f90036030 con 0x56068cd2db40 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.029+0000 7f7f76ffd640 1 -- 172.21.15.5:0/1836033695 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f7f8c01c470 con 0x56068cd2db40 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.029+0000 7f7fa0612740 10 client.16164 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.029+0000 7f7fa0612740 10 client.16164 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.029+0000 7f7fa0612740 20 client.16164 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.029+0000 7f7fa0612740 1 --2- 172.21.15.5:0/1836033695 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x56068cd46d30 0x56068cd49140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.029+0000 7f7fa0612740 4 client.16164 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.029+0000 7f7fa0612740 1 -- 172.21.15.5:0/1836033695 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56068cc9f860 con 0x56068cd46d30 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.029+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1836033695 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x56068cd46d30 0x56068cd49140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.030+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1836033695 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x56068cd46d30 0x56068cd49140 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=40856cb8ed351f9c server_cookie=366b5c78ab5b6119 in_seq=0 out_seq=0 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.030+0000 7f7f76ffd640 10 client.16164 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:06:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.030+0000 7f7f76ffd640 1 -- 172.21.15.5:0/1836033695 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56068cc9f860 con 0x56068cd46d30 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.030+0000 7f7f76ffd640 10 client.16164 handle_command_reply: tid=0 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.030+0000 7f7fa0612740 1 client.16164 shutdown 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.030+0000 7f7fa0612740 1 -- 172.21.15.5:0/1836033695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2db40 msgr2=0x56068cd33ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.030+0000 7f7fa0612740 1 --2- 172.21.15.5:0/1836033695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2db40 0x56068cd33ff0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c012700 tx=0x7f7f8c012bc0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.031+0000 7f7fa0612740 1 -- 172.21.15.5:0/1836033695 shutdown_connections 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.031+0000 7f7fa0612740 1 --2- 172.21.15.5:0/1836033695 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x56068cd46d30 0x56068cd49140 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.031+0000 7f7f76ffd640 0 client.16164 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.031+0000 7f7fa0612740 1 --2- 172.21.15.5:0/1836033695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd37370 0x56068cd34600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.031+0000 7f7fa0612740 1 --2- 172.21.15.5:0/1836033695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2db40 0x56068cd33ff0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.031+0000 7f7fa0612740 1 --2- 172.21.15.5:0/1836033695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2c5d0 0x56068cd36cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.031+0000 7f7fa0612740 1 -- 172.21.15.5:0/1836033695 >> 172.21.15.5:0/1836033695 conn(0x56068ccf5310 msgr2=0x56068cd2e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.031+0000 7f7fa0612740 1 -- 172.21.15.5:0/1836033695 shutdown_connections 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.031+0000 7f7fa0612740 1 -- 172.21.15.5:0/1836033695 wait complete. 2024-09-19T06:06:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.031+0000 7f7fa0612740 20 client.16164 trim_cache size 0 max 16384 2024-09-19T06:06:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.046+0000 7f7fa0612740 1 Processor -- start 2024-09-19T06:06:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.046+0000 7f7fa0612740 1 -- start start 2024-09-19T06:06:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.046+0000 7f7fa0612740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 0x56068cd43130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.046+0000 7f7fa0612740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd37370 0x56068cd27bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.046+0000 7f7fa0612740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd32d60 0x56068cd28130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.046+0000 7f7fa0612740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981af8e0 con 0x56068cd2c5d0 2024-09-19T06:06:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.046+0000 7f7fa0612740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981b1480 con 0x56068cd37370 2024-09-19T06:06:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.046+0000 7f7fa0612740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981affb0 con 0x56068cd32d60 2024-09-19T06:06:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.047+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 0x56068cd43130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.047+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 0x56068cd43130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34340/0 (socket says 172.21.15.5:34340) 2024-09-19T06:06:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.047+0000 7f7f97fff640 1 -- 172.21.15.5:0/295881242 learned_addr learned my addr 172.21.15.5:0/295881242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.047+0000 7f7f96ffd640 1 --2- 172.21.15.5:0/295881242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd37370 0x56068cd27bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.047+0000 7f7f97fff640 1 --2- 172.21.15.5:0/295881242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd32d60 0x56068cd28130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.047+0000 7f7f97fff640 1 -- 172.21.15.5:0/295881242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd32d60 msgr2=0x56068cd28130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.047+0000 7f7f97fff640 1 --2- 172.21.15.5:0/295881242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd32d60 0x56068cd28130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.047+0000 7f7f97fff640 1 -- 172.21.15.5:0/295881242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd37370 msgr2=0x56068cd27bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.047+0000 7f7f97fff640 1 --2- 172.21.15.5:0/295881242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd37370 0x56068cd27bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.047+0000 7f7f97fff640 1 -- 172.21.15.5:0/295881242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56068cd497c0 con 0x56068cd2c5d0 2024-09-19T06:06:29.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.047+0000 7f7f97fff640 1 --2- 172.21.15.5:0/295881242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 0x56068cd43130 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7f90004af0 tx=0x7f7f90004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87536701cbd3e149 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:29.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.047+0000 7f7f7d7fa640 1 -- 172.21.15.5:0/295881242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9005f070 con 0x56068cd2c5d0 2024-09-19T06:06:29.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.048+0000 7f7f7d7fa640 1 -- 172.21.15.5:0/295881242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f90036040 con 0x56068cd2c5d0 2024-09-19T06:06:29.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.048+0000 7f7f7d7fa640 1 -- 172.21.15.5:0/295881242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f900305c0 con 0x56068cd2c5d0 2024-09-19T06:06:29.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.048+0000 7f7fa0612740 1 -- 172.21.15.5:0/295881242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 msgr2=0x56068cd43130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:29.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.048+0000 7f7fa0612740 1 --2- 172.21.15.5:0/295881242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 0x56068cd43130 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7f90004af0 tx=0x7f7f90004a60 comp rx=0 tx=0).stop 2024-09-19T06:06:29.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.048+0000 7f7fa0612740 1 -- 172.21.15.5:0/295881242 shutdown_connections 2024-09-19T06:06:29.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.048+0000 7f7fa0612740 1 --2- 172.21.15.5:0/295881242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd32d60 0x56068cd28130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.048+0000 7f7fa0612740 1 --2- 172.21.15.5:0/295881242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd37370 0x56068cd27bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.048+0000 7f7fa0612740 1 --2- 172.21.15.5:0/295881242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd2c5d0 0x56068cd43130 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.048+0000 7f7fa0612740 1 -- 172.21.15.5:0/295881242 >> 172.21.15.5:0/295881242 conn(0x56068ccf5310 msgr2=0x56068cd274e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:29.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.049+0000 7f7fa0612740 1 -- 172.21.15.5:0/295881242 shutdown_connections 2024-09-19T06:06:29.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.049+0000 7f7fa0612740 1 -- 172.21.15.5:0/295881242 wait complete. 2024-09-19T06:06:29.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.049+0000 7f7fa0612740 1 -- start start 2024-09-19T06:06:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.049+0000 7f7fa0612740 1 Processor -- start 2024-09-19T06:06:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.049+0000 7f7fa0612740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2c5d0 0x56068cd3f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.049+0000 7f7fa0612740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd32d60 0x56068cd3c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.049+0000 7f7fa0612740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd37370 0x56068cd3c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7fa0612740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981b0db0 con 0x56068cd37370 2024-09-19T06:06:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7fa0612740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981b0680 con 0x56068cd2c5d0 2024-09-19T06:06:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7fa0612740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981ad8c0 con 0x56068cd32d60 2024-09-19T06:06:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7fa0612740 10 client.0 _ll_register_callbacks cb 0x7f7f8c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2c5d0 0x56068cd3f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2c5d0 0x56068cd3f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48062/0 (socket says 172.21.15.5:48062) 2024-09-19T06:06:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7f97fff640 1 -- 172.21.15.5:0/3466902089 learned_addr learned my addr 172.21.15.5:0/3466902089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7f97fff640 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd37370 0x56068cd3c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7f96ffd640 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd32d60 0x56068cd3c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7f97fff640 1 -- 172.21.15.5:0/3466902089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd32d60 msgr2=0x56068cd3c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7f97fff640 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd32d60 0x56068cd3c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7f97fff640 1 -- 172.21.15.5:0/3466902089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd37370 msgr2=0x56068cd3c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7f97fff640 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd37370 0x56068cd3c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7f97fff640 1 -- 172.21.15.5:0/3466902089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56068cd3cea0 con 0x56068cd2c5d0 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7f97fff640 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2c5d0 0x56068cd3f870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7f90006010 tx=0x7f7f90004480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e6a04cf6b1df01b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.050+0000 7f7f76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7f76ffd640 1 -- 172.21.15.5:0/3466902089 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9005f040 con 0x56068cd2c5d0 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7fa0612740 1 -- 172.21.15.5:0/3466902089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f90036040 con 0x56068cd2c5d0 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7f76ffd640 1 -- 172.21.15.5:0/3466902089 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f7f90004520 con 0x56068cd2c5d0 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7f76ffd640 1 -- 172.21.15.5:0/3466902089 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7f90004720 con 0x56068cd2c5d0 2024-09-19T06:06:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7fa0612740 10 client.25769 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7fa0612740 1 -- 172.21.15.5:0/3466902089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56068cdb28f0 con 0x56068cd2c5d0 2024-09-19T06:06:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7f76ffd640 1 -- 172.21.15.5:0/3466902089 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f90036040 con 0x56068cd2c5d0 2024-09-19T06:06:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7f76ffd640 1 -- 172.21.15.5:0/3466902089 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f7f90065040 con 0x56068cd2c5d0 2024-09-19T06:06:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7fa0612740 10 client.25769 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7fa0612740 10 client.25769 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:06:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7fa0612740 20 client.25769 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7fa0612740 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x56068cdb2c30 0x56068cdb5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7fa0612740 4 client.25769 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:06:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.051+0000 7f7fa0612740 1 -- 172.21.15.5:0/3466902089 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56068cc99d70 con 0x56068cdb2c30 2024-09-19T06:06:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.052+0000 7f7f96ffd640 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x56068cdb2c30 0x56068cdb5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.052+0000 7f7f76ffd640 10 client.25769 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:06:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.052+0000 7f7f96ffd640 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x56068cdb2c30 0x56068cdb5020 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bb421e38f6983eb server_cookie=777f7c00c7526d0 in_seq=0 out_seq=0 2024-09-19T06:06:29.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7f76ffd640 1 -- 172.21.15.5:0/3466902089 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56068cc99d70 con 0x56068cdb2c30 2024-09-19T06:06:29.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7f76ffd640 10 client.25769 handle_command_reply: tid=0 2024-09-19T06:06:29.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7fa0612740 1 client.25769 shutdown 2024-09-19T06:06:29.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7fa0612740 1 -- 172.21.15.5:0/3466902089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2c5d0 msgr2=0x56068cd3f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:29.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7fa0612740 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2c5d0 0x56068cd3f870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7f90006010 tx=0x7f7f90004480 comp rx=0 tx=0).stop 2024-09-19T06:06:29.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7fa0612740 1 -- 172.21.15.5:0/3466902089 shutdown_connections 2024-09-19T06:06:29.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7fa0612740 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x56068cdb2c30 0x56068cdb5020 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7fa0612740 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56068cd37370 0x56068cd3c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7fa0612740 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56068cd32d60 0x56068cd3c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7fa0612740 1 --2- 172.21.15.5:0/3466902089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56068cd2c5d0 0x56068cd3f870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7fa0612740 1 -- 172.21.15.5:0/3466902089 >> 172.21.15.5:0/3466902089 conn(0x56068ccf5310 msgr2=0x56068cd27d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:29.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7f76ffd640 0 client.25769 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:06:29.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7fa0612740 1 -- 172.21.15.5:0/3466902089 shutdown_connections 2024-09-19T06:06:29.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7fa0612740 1 -- 172.21.15.5:0/3466902089 wait complete. 2024-09-19T06:06:29.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.053+0000 7f7fa0612740 20 client.25769 trim_cache size 0 max 16384 2024-09-19T06:06:29.061 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:06:29.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.054+0000 7f7f9e239640 1 -- 172.21.15.5:0/865920439 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7f800a33b0 msgr2=0x7f7f800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:29.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.054+0000 7f7f9e239640 1 --2- 172.21.15.5:0/865920439 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7f800a33b0 0x7f7f800a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7f880cd7f0 tx=0x7f7f88000bc0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.054+0000 7f7f9e239640 1 -- 172.21.15.5:0/865920439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98101de0 msgr2=0x7f7f9819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:29.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.054+0000 7f7f9e239640 1 --2- 172.21.15.5:0/865920439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98101de0 0x7f7f9819a4d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800e9f0 tx=0x7f7f8800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.054+0000 7f7f9e239640 1 -- 172.21.15.5:0/865920439 shutdown_connections 2024-09-19T06:06:29.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.054+0000 7f7f9e239640 1 --2- 172.21.15.5:0/865920439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106f10 0x7f7f9819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.054+0000 7f7f9e239640 1 --2- 172.21.15.5:0/865920439 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7f800a33b0 0x7f7f800a5870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.054+0000 7f7f9e239640 1 --2- 172.21.15.5:0/865920439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f98102790 0x7f7f9819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.054+0000 7f7f9e239640 1 --2- 172.21.15.5:0/865920439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98101de0 0x7f7f9819a4d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:29.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.054+0000 7f7f9e239640 1 -- 172.21.15.5:0/865920439 >> 172.21.15.5:0/865920439 conn(0x7f7f980fd880 msgr2=0x7f7f980feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:29.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.054+0000 7f7f9e239640 1 -- 172.21.15.5:0/865920439 shutdown_connections 2024-09-19T06:06:29.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:29.054+0000 7f7f9e239640 1 -- 172.21.15.5:0/865920439 wait complete. 2024-09-19T06:06:29.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:06:29.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 60342 2.99 MB/sec warmup 115 sec latency 55.214 ms 2024-09-19T06:06:30.080 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:06:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.639+0000 7f9e90647640 1 -- 172.21.15.5:0/1720205624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88112b90 msgr2=0x7f9e88114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.639+0000 7f9e90647640 1 --2- 172.21.15.5:0/1720205624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88112b90 0x7f9e88114f80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009ca0 tx=0x7f9e8002a950 comp rx=0 tx=0).stop 2024-09-19T06:06:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.639+0000 7f9e90647640 1 -- 172.21.15.5:0/1720205624 shutdown_connections 2024-09-19T06:06:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.639+0000 7f9e90647640 1 --2- 172.21.15.5:0/1720205624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88112b90 0x7f9e88114f80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.639+0000 7f9e90647640 1 --2- 172.21.15.5:0/1720205624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e88074eb0 0x7f9e88112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.639+0000 7f9e90647640 1 --2- 172.21.15.5:0/1720205624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88074590 0x7f9e88074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.639+0000 7f9e90647640 1 -- 172.21.15.5:0/1720205624 >> 172.21.15.5:0/1720205624 conn(0x7f9e880fd8e0 msgr2=0x7f9e880ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.639+0000 7f9e90647640 1 -- 172.21.15.5:0/1720205624 shutdown_connections 2024-09-19T06:06:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.639+0000 7f9e90647640 1 -- 172.21.15.5:0/1720205624 wait complete. 2024-09-19T06:06:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e90647640 1 Processor -- start 2024-09-19T06:06:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e90647640 1 -- start start 2024-09-19T06:06:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e90647640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88074590 0x7f9e881a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e90647640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e88074eb0 0x7f9e881a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e90647640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88112b90 0x7f9e8819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e90647640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88118410 con 0x7f9e88112b90 2024-09-19T06:06:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e90647640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88118290 con 0x7f9e88074eb0 2024-09-19T06:06:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e90647640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88118590 con 0x7f9e88074590 2024-09-19T06:06:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e8ebe4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88112b90 0x7f9e8819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e8dbe2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e88074eb0 0x7f9e881a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e8e3e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88074590 0x7f9e881a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e8ebe4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88112b90 0x7f9e8819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34364/0 (socket says 172.21.15.5:34364) 2024-09-19T06:06:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e8dbe2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e88074eb0 0x7f9e881a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48088/0 (socket says 172.21.15.5:48088) 2024-09-19T06:06:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.640+0000 7f9e8ebe4640 1 -- 172.21.15.5:0/3167518902 learned_addr learned my addr 172.21.15.5:0/3167518902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:30.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.641+0000 7f9e8ebe4640 1 -- 172.21.15.5:0/3167518902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88074590 msgr2=0x7f9e881a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:30.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.641+0000 7f9e8ebe4640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88074590 0x7f9e881a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.641+0000 7f9e8ebe4640 1 -- 172.21.15.5:0/3167518902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e88074eb0 msgr2=0x7f9e881a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.641+0000 7f9e8ebe4640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e88074eb0 0x7f9e881a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.641+0000 7f9e8ebe4640 1 -- 172.21.15.5:0/3167518902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8819ef60 con 0x7f9e88112b90 2024-09-19T06:06:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.641+0000 7f9e8e3e3640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88074590 0x7f9e881a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.641+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e88074eb0 0x7f9e881a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.641+0000 7f9e8ebe4640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88112b90 0x7f9e8819e7c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9e80004af0 tx=0x7f9e80031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.641+0000 7f9e7f7fe640 1 -- 172.21.15.5:0/3167518902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80034040 con 0x7f9e88112b90 2024-09-19T06:06:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.641+0000 7f9e7f7fe640 1 -- 172.21.15.5:0/3167518902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e800308f0 con 0x7f9e88112b90 2024-09-19T06:06:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.642+0000 7f9e7f7fe640 1 -- 172.21.15.5:0/3167518902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e800045d0 con 0x7f9e88112b90 2024-09-19T06:06:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.642+0000 7f9e90647640 1 -- 172.21.15.5:0/3167518902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8819f1f0 con 0x7f9e88112b90 2024-09-19T06:06:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.642+0000 7f9e90647640 1 -- 172.21.15.5:0/3167518902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e881ab4a0 con 0x7f9e88112b90 2024-09-19T06:06:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.645+0000 7f9e92a20740 1 Processor -- start 2024-09-19T06:06:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.645+0000 7f9e7f7fe640 1 -- 172.21.15.5:0/3167518902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9e80037020 con 0x7f9e88112b90 2024-09-19T06:06:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.645+0000 7f9e92a20740 1 -- start start 2024-09-19T06:06:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.645+0000 7f9e92a20740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.645+0000 7f9e92a20740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 0x5614487e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.645+0000 7f9e92a20740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487f2370 0x5614487f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.645+0000 7f9e92a20740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56144874d2f0 con 0x5614487f2370 2024-09-19T06:06:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.645+0000 7f9e92a20740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5614486f4cf0 con 0x5614487e75d0 2024-09-19T06:06:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.645+0000 7f9e92a20740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5614487492b0 con 0x5614487e8b40 2024-09-19T06:06:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8e3e3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 0x5614487e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8dbe2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487f2370 0x5614487f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8e3e3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 0x5614487e7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48092/0 (socket says 172.21.15.5:48092) 2024-09-19T06:06:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8e3e3640 1 -- 172.21.15.5:0/560561456 learned_addr learned my addr 172.21.15.5:0/560561456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8e3e3640 1 -- 172.21.15.5:0/560561456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 msgr2=0x5614487e7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/560561456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8e3e3640 1 --2- 172.21.15.5:0/560561456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487e7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8e3e3640 1 -- 172.21.15.5:0/560561456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487f2370 msgr2=0x5614487f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8e3e3640 1 --2- 172.21.15.5:0/560561456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487f2370 0x5614487f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8e3e3640 1 -- 172.21.15.5:0/560561456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614486d93a0 con 0x5614487e75d0 2024-09-19T06:06:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/560561456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487e7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:06:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e7f7fe640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9e740a3430 0x7f9e740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8e3e3640 1 --2- 172.21.15.5:0/560561456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 0x5614487e7a10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e840115c0 tx=0x7f9e84011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3173e8161b450e62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e7cff9640 1 -- 172.21.15.5:0/560561456 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8400f640 con 0x5614487e75d0 2024-09-19T06:06:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9e740a3430 0x7f9e740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e7f7fe640 1 -- 172.21.15.5:0/3167518902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f9e800f07f0 con 0x7f9e88112b90 2024-09-19T06:06:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e7cff9640 1 -- 172.21.15.5:0/560561456 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e8400fc90 con 0x5614487e75d0 2024-09-19T06:06:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.646+0000 7f9e7cff9640 1 -- 172.21.15.5:0/560561456 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84019ce0 con 0x5614487e75d0 2024-09-19T06:06:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.647+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9e740a3430 0x7f9e740a58f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5614487e8070 tx=0x7f9e78012040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.647+0000 7f9e92a20740 1 -- 172.21.15.5:0/560561456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 msgr2=0x5614487e7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.647+0000 7f9e92a20740 1 --2- 172.21.15.5:0/560561456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 0x5614487e7a10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e840115c0 tx=0x7f9e84011a80 comp rx=0 tx=0).stop 2024-09-19T06:06:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.647+0000 7f9e92a20740 1 -- 172.21.15.5:0/560561456 shutdown_connections 2024-09-19T06:06:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.647+0000 7f9e92a20740 1 --2- 172.21.15.5:0/560561456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487f2370 0x5614487f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.647+0000 7f9e92a20740 1 --2- 172.21.15.5:0/560561456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 0x5614487e7a10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.647+0000 7f9e92a20740 1 --2- 172.21.15.5:0/560561456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487e7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.647+0000 7f9e92a20740 1 -- 172.21.15.5:0/560561456 >> 172.21.15.5:0/560561456 conn(0x5614487b0310 msgr2=0x5614487e3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.647+0000 7f9e92a20740 1 -- 172.21.15.5:0/560561456 shutdown_connections 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e92a20740 1 -- 172.21.15.5:0/560561456 wait complete. 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e92a20740 1 -- start start 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e92a20740 1 Processor -- start 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e92a20740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 0x5614487f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e92a20740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e92a20740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487f2370 0x5614487ef530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e92a20740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5614487f50c0 con 0x5614487e75d0 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e92a20740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561448708060 con 0x5614487f2370 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e92a20740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5614487f5240 con 0x5614487e8b40 2024-09-19T06:06:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e8e3e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 0x5614487f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e92a20740 10 client.0 _ll_register_callbacks cb 0x56144845b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e8e3e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 0x5614487f1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34386/0 (socket says 172.21.15.5:34386) 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e8e3e3640 1 -- 172.21.15.5:0/2678618155 learned_addr learned my addr 172.21.15.5:0/2678618155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.648+0000 7f9e8ebe4640 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487f2370 0x5614487ef530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e8e3e3640 1 -- 172.21.15.5:0/2678618155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 msgr2=0x5614487eefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e8e3e3640 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487eefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e8e3e3640 1 -- 172.21.15.5:0/2678618155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487f2370 msgr2=0x5614487ef530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e8e3e3640 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487f2370 0x5614487ef530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e8e3e3640 1 -- 172.21.15.5:0/2678618155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614484be3c0 con 0x5614487e75d0 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e8e3e3640 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 0x5614487f1d00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9e84005e00 tx=0x7f9e84019f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958ee8addc62b140 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e71ffb640 1 -- 172.21.15.5:0/2678618155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84026020 con 0x5614487e75d0 2024-09-19T06:06:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e92a20740 1 -- 172.21.15.5:0/2678618155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e8400fc90 con 0x5614487e75d0 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e71ffb640 1 -- 172.21.15.5:0/2678618155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f9e8400fe40 con 0x5614487e75d0 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e71ffb640 1 -- 172.21.15.5:0/2678618155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9e84031050 con 0x5614487e75d0 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.649+0000 7f9e92a20740 10 client.16182 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.650+0000 7f9e92a20740 1 -- 172.21.15.5:0/2678618155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561448800a60 con 0x5614487e75d0 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.650+0000 7f9e71ffb640 1 -- 172.21.15.5:0/2678618155 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e8400fc90 con 0x5614487e75d0 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.650+0000 7f9e71ffb640 1 -- 172.21.15.5:0/2678618155 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f9e840035f0 con 0x5614487e75d0 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.650+0000 7f9e92a20740 10 client.16182 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.650+0000 7f9e92a20740 10 client.16182 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.650+0000 7f9e92a20740 20 client.16182 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.650+0000 7f9e92a20740 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x561448801e10 0x561448804200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.650+0000 7f9e92a20740 4 client.16182 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.650+0000 7f9e92a20740 1 -- 172.21.15.5:0/2678618155 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56144875a860 con 0x561448801e10 2024-09-19T06:06:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.650+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x561448801e10 0x561448804200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.651+0000 7f9e71ffb640 10 client.16182 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:06:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.651+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x561448801e10 0x561448804200 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1954401bef2937ba server_cookie=d256f7cf4df638ba in_seq=0 out_seq=0 2024-09-19T06:06:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.651+0000 7f9e71ffb640 1 -- 172.21.15.5:0/2678618155 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56144875a860 con 0x561448801e10 2024-09-19T06:06:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.651+0000 7f9e71ffb640 10 client.16182 handle_command_reply: tid=0 2024-09-19T06:06:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.651+0000 7f9e92a20740 1 client.16182 shutdown 2024-09-19T06:06:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.652+0000 7f9e92a20740 1 -- 172.21.15.5:0/2678618155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 msgr2=0x5614487f1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.652+0000 7f9e92a20740 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 0x5614487f1d00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9e84005e00 tx=0x7f9e84019f00 comp rx=0 tx=0).stop 2024-09-19T06:06:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.652+0000 7f9e92a20740 1 -- 172.21.15.5:0/2678618155 shutdown_connections 2024-09-19T06:06:30.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.652+0000 7f9e92a20740 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x561448801e10 0x561448804200 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.652+0000 7f9e92a20740 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487f2370 0x5614487ef530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.652+0000 7f9e92a20740 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.652+0000 7f9e71ffb640 0 client.16182 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:06:30.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.652+0000 7f9e92a20740 1 --2- 172.21.15.5:0/2678618155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 0x5614487f1d00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.652+0000 7f9e92a20740 1 -- 172.21.15.5:0/2678618155 >> 172.21.15.5:0/2678618155 conn(0x5614487b0310 msgr2=0x5614487e9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:30.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.652+0000 7f9e92a20740 1 -- 172.21.15.5:0/2678618155 shutdown_connections 2024-09-19T06:06:30.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.652+0000 7f9e92a20740 1 -- 172.21.15.5:0/2678618155 wait complete. 2024-09-19T06:06:30.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.652+0000 7f9e92a20740 20 client.16182 trim_cache size 0 max 16384 2024-09-19T06:06:30.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e92a20740 1 Processor -- start 2024-09-19T06:06:30.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e92a20740 1 -- start start 2024-09-19T06:06:30.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e92a20740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 0x5614487fe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e92a20740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e8b40 0x5614487ea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e92a20740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487f2370 0x5614487ea660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e92a20740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b3e70 con 0x5614487e8b40 2024-09-19T06:06:30.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e92a20740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b5a10 con 0x5614487e75d0 2024-09-19T06:06:30.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e92a20740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b4540 con 0x5614487f2370 2024-09-19T06:06:30.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e8dbe2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e8b40 0x5614487ea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e8dbe2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e8b40 0x5614487ea120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34388/0 (socket says 172.21.15.5:34388) 2024-09-19T06:06:30.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e8dbe2640 1 -- 172.21.15.5:0/1625652003 learned_addr learned my addr 172.21.15.5:0/1625652003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e8e3e3640 1 --2- 172.21.15.5:0/1625652003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 0x5614487fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.667+0000 7f9e8ebe4640 1 --2- 172.21.15.5:0/1625652003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487f2370 0x5614487ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.668+0000 7f9e8dbe2640 1 -- 172.21.15.5:0/1625652003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487f2370 msgr2=0x5614487ea660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.668+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/1625652003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487f2370 0x5614487ea660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.668+0000 7f9e8dbe2640 1 -- 172.21.15.5:0/1625652003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 msgr2=0x5614487fe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.668+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/1625652003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 0x5614487fe360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.668+0000 7f9e8dbe2640 1 -- 172.21.15.5:0/1625652003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614487f1920 con 0x5614487e8b40 2024-09-19T06:06:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.668+0000 7f9e8e3e3640 1 --2- 172.21.15.5:0/1625652003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 0x5614487fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.668+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/1625652003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e8b40 0x5614487ea120 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009730 tx=0x7f9e780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15890a8e5a04506f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.668+0000 7f9e73fff640 1 -- 172.21.15.5:0/1625652003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7801f7b0 con 0x5614487e8b40 2024-09-19T06:06:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.668+0000 7f9e73fff640 1 -- 172.21.15.5:0/1625652003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e7801fe00 con 0x5614487e8b40 2024-09-19T06:06:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.668+0000 7f9e73fff640 1 -- 172.21.15.5:0/1625652003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7801d410 con 0x5614487e8b40 2024-09-19T06:06:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.669+0000 7f9e92a20740 1 -- 172.21.15.5:0/1625652003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e8b40 msgr2=0x5614487ea120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.669+0000 7f9e92a20740 1 --2- 172.21.15.5:0/1625652003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e8b40 0x5614487ea120 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009730 tx=0x7f9e780046c0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.669+0000 7f9e92a20740 1 -- 172.21.15.5:0/1625652003 shutdown_connections 2024-09-19T06:06:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.669+0000 7f9e92a20740 1 --2- 172.21.15.5:0/1625652003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487f2370 0x5614487ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.669+0000 7f9e92a20740 1 --2- 172.21.15.5:0/1625652003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e8b40 0x5614487ea120 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.669+0000 7f9e92a20740 1 --2- 172.21.15.5:0/1625652003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487e75d0 0x5614487fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.669+0000 7f9e92a20740 1 -- 172.21.15.5:0/1625652003 >> 172.21.15.5:0/1625652003 conn(0x5614487b0310 msgr2=0x5614487ee0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.669+0000 7f9e92a20740 1 -- 172.21.15.5:0/1625652003 shutdown_connections 2024-09-19T06:06:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.669+0000 7f9e92a20740 1 -- 172.21.15.5:0/1625652003 wait complete. 2024-09-19T06:06:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.669+0000 7f9e92a20740 1 -- start start 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.669+0000 7f9e92a20740 1 Processor -- start 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e92a20740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 0x5614487fa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e92a20740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487fadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e92a20740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487f2370 0x5614487ebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e92a20740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b5340 con 0x5614487e75d0 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e92a20740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b4c10 con 0x5614487f2370 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e92a20740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b1e50 con 0x5614487e8b40 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e92a20740 10 client.0 _ll_register_callbacks cb 0x7f9e78009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8e3e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 0x5614487fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8e3e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 0x5614487fa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34390/0 (socket says 172.21.15.5:34390) 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8e3e3640 1 -- 172.21.15.5:0/3522648152 learned_addr learned my addr 172.21.15.5:0/3522648152 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8ebe4640 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487f2370 0x5614487ebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487fadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8e3e3640 1 -- 172.21.15.5:0/3522648152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 msgr2=0x5614487fadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8e3e3640 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487fadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8e3e3640 1 -- 172.21.15.5:0/3522648152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487f2370 msgr2=0x5614487ebc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8e3e3640 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487f2370 0x5614487ebc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8e3e3640 1 -- 172.21.15.5:0/3522648152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614487ec440 con 0x5614487e75d0 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8ebe4640 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487f2370 0x5614487ebc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487fadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e8e3e3640 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 0x5614487fa880 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9e84011ac0 tx=0x7f9e840038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483425d08e6df06e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.670+0000 7f9e71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e71ffb640 1 -- 172.21.15.5:0/3522648152 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84026040 con 0x5614487e75d0 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e71ffb640 1 -- 172.21.15.5:0/3522648152 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f9e840050b0 con 0x5614487e75d0 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e92a20740 1 -- 172.21.15.5:0/3522648152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e7801fe00 con 0x5614487e75d0 2024-09-19T06:06:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e71ffb640 1 -- 172.21.15.5:0/3522648152 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9e84019d30 con 0x5614487e75d0 2024-09-19T06:06:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e92a20740 10 client.16188 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e92a20740 1 -- 172.21.15.5:0/3522648152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56144886c6b0 con 0x5614487e75d0 2024-09-19T06:06:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e71ffb640 1 -- 172.21.15.5:0/3522648152 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e7801fe00 con 0x5614487e75d0 2024-09-19T06:06:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e71ffb640 1 -- 172.21.15.5:0/3522648152 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f9e8400fc10 con 0x5614487e75d0 2024-09-19T06:06:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e92a20740 10 client.16188 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e92a20740 10 client.16188 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:06:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e92a20740 20 client.16188 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e92a20740 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56144886ce20 0x56144886f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e92a20740 4 client.16188 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:06:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.671+0000 7f9e92a20740 1 -- 172.21.15.5:0/3522648152 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561448754d70 con 0x56144886ce20 2024-09-19T06:06:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56144886ce20 0x56144886f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:30.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e8dbe2640 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56144886ce20 0x56144886f240 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50c0b62174e9239d server_cookie=3211675f60ee76c3 in_seq=0 out_seq=0 2024-09-19T06:06:30.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e71ffb640 10 client.16188 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:06:30.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e71ffb640 1 -- 172.21.15.5:0/3522648152 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561448754d70 con 0x56144886ce20 2024-09-19T06:06:30.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e71ffb640 10 client.16188 handle_command_reply: tid=0 2024-09-19T06:06:30.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e92a20740 1 client.16188 shutdown 2024-09-19T06:06:30.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e92a20740 1 -- 172.21.15.5:0/3522648152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 msgr2=0x5614487fa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:30.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e92a20740 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 0x5614487fa880 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9e84011ac0 tx=0x7f9e840038a0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e92a20740 1 -- 172.21.15.5:0/3522648152 shutdown_connections 2024-09-19T06:06:30.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e92a20740 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56144886ce20 0x56144886f240 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e92a20740 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614487f2370 0x5614487ebc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e92a20740 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614487e8b40 0x5614487fadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e92a20740 1 --2- 172.21.15.5:0/3522648152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614487e75d0 0x5614487fa880 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e92a20740 1 -- 172.21.15.5:0/3522648152 >> 172.21.15.5:0/3522648152 conn(0x5614487b0310 msgr2=0x5614487f0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e71ffb640 0 client.16188 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:06:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e92a20740 1 -- 172.21.15.5:0/3522648152 shutdown_connections 2024-09-19T06:06:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.672+0000 7f9e92a20740 1 -- 172.21.15.5:0/3522648152 wait complete. 2024-09-19T06:06:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e92a20740 20 client.16188 trim_cache size 0 max 16384 2024-09-19T06:06:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:06:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e90647640 1 -- 172.21.15.5:0/3167518902 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9e740a3430 msgr2=0x7f9e740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e90647640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9e740a3430 0x7f9e740a58f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5614487e8070 tx=0x7f9e78012040 comp rx=0 tx=0).stop 2024-09-19T06:06:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e90647640 1 -- 172.21.15.5:0/3167518902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88112b90 msgr2=0x7f9e8819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e90647640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88112b90 0x7f9e8819e7c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9e80004af0 tx=0x7f9e80031f50 comp rx=0 tx=0).stop 2024-09-19T06:06:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e90647640 1 -- 172.21.15.5:0/3167518902 shutdown_connections 2024-09-19T06:06:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e90647640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88112b90 0x7f9e8819e7c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e90647640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e88074eb0 0x7f9e881a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e90647640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9e740a3430 0x7f9e740a58f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e90647640 1 --2- 172.21.15.5:0/3167518902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88074590 0x7f9e881a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e90647640 1 -- 172.21.15.5:0/3167518902 >> 172.21.15.5:0/3167518902 conn(0x7f9e880fd8e0 msgr2=0x7f9e880ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e90647640 1 -- 172.21.15.5:0/3167518902 shutdown_connections 2024-09-19T06:06:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:30.673+0000 7f9e90647640 1 -- 172.21.15.5:0/3167518902 wait complete. 2024-09-19T06:06:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:30 smithi157 ceph-mon[35346]: pgmap v98: 97 pgs: 97 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-19T06:06:30.824 INFO:tasks.workunit.client.0.smithi005.stdout: 1 60698 2.98 MB/sec warmup 116 sec latency 38.864 ms 2024-09-19T06:06:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:30 smithi005 ceph-mon[30955]: pgmap v98: 97 pgs: 97 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-19T06:06:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:30 smithi049 ceph-mon[34640]: pgmap v98: 97 pgs: 97 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-19T06:06:31.537 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:06:31.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 61041 2.96 MB/sec warmup 117 sec latency 11.251 ms 2024-09-19T06:06:31.861 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:06:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.351+0000 7f85b83e8640 1 -- 172.21.15.5:0/2413635521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b0102ef0 msgr2=0x7f85b010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.351+0000 7f85b83e8640 1 --2- 172.21.15.5:0/2413635521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b0102ef0 0x7f85b010da40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009ca0 tx=0x7f85a802a950 comp rx=0 tx=0).stop 2024-09-19T06:06:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.351+0000 7f85b83e8640 1 -- 172.21.15.5:0/2413635521 shutdown_connections 2024-09-19T06:06:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.351+0000 7f85b83e8640 1 --2- 172.21.15.5:0/2413635521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b0102ef0 0x7f85b010da40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.352+0000 7f85b83e8640 1 --2- 172.21.15.5:0/2413635521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0102570 0x7f85b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.352+0000 7f85b83e8640 1 --2- 172.21.15.5:0/2413635521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b0100e20 0x7f85b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.352+0000 7f85b83e8640 1 -- 172.21.15.5:0/2413635521 >> 172.21.15.5:0/2413635521 conn(0x7f85b00fc7d0 msgr2=0x7f85b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.352+0000 7f85b83e8640 1 -- 172.21.15.5:0/2413635521 shutdown_connections 2024-09-19T06:06:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.352+0000 7f85b83e8640 1 -- 172.21.15.5:0/2413635521 wait complete. 2024-09-19T06:06:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.352+0000 7f85b83e8640 1 Processor -- start 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.352+0000 7f85b83e8640 1 -- start start 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b83e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b0100e20 0x7f85b01a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b83e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b0102570 0x7f85b01a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b83e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0102ef0 0x7f85b019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b83e8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0110e30 con 0x7f85b0100e20 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b83e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0110cb0 con 0x7f85b0102ef0 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b83e8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0110fb0 con 0x7f85b0102570 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b6184640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b0100e20 0x7f85b01a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b6985640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0102ef0 0x7f85b019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b6184640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b0100e20 0x7f85b01a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34408/0 (socket says 172.21.15.5:34408) 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b6985640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0102ef0 0x7f85b019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48132/0 (socket says 172.21.15.5:48132) 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b6184640 1 -- 172.21.15.5:0/514893407 learned_addr learned my addr 172.21.15.5:0/514893407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b5983640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b0102570 0x7f85b01a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b6985640 1 -- 172.21.15.5:0/514893407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b0102570 msgr2=0x7f85b01a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b6985640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b0102570 0x7f85b01a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b6985640 1 -- 172.21.15.5:0/514893407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b0100e20 msgr2=0x7f85b01a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b6985640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b0100e20 0x7f85b01a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b6985640 1 -- 172.21.15.5:0/514893407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b019f230 con 0x7f85b0102ef0 2024-09-19T06:06:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b6184640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b0100e20 0x7f85b01a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.353+0000 7f85b6985640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0102ef0 0x7f85b019eab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f85a8005cc0 tx=0x7f85a802f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.354+0000 7f85a77fe640 1 -- 172.21.15.5:0/514893407 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8034040 con 0x7f85b0102ef0 2024-09-19T06:06:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.354+0000 7f85a77fe640 1 -- 172.21.15.5:0/514893407 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85a80308f0 con 0x7f85b0102ef0 2024-09-19T06:06:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.354+0000 7f85b83e8640 1 -- 172.21.15.5:0/514893407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b019f4c0 con 0x7f85b0102ef0 2024-09-19T06:06:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.354+0000 7f85a77fe640 1 -- 172.21.15.5:0/514893407 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a80045d0 con 0x7f85b0102ef0 2024-09-19T06:06:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.354+0000 7f85b83e8640 1 -- 172.21.15.5:0/514893407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b01ab490 con 0x7f85b0102ef0 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.356+0000 7f85a77fe640 1 -- 172.21.15.5:0/514893407 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f85a8037020 con 0x7f85b0102ef0 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.356+0000 7f85ba7c1740 1 Processor -- start 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85ba7c1740 1 -- start start 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85ba7c1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cadb40 0x55ffb2cac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85ba7c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cac5d0 0x55ffb2caca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85ba7c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85ba7c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffb2c122f0 con 0x55ffb2cb7370 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85ba7c1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ffb2bb9cf0 con 0x55ffb2cadb40 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85ba7c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ffb2c0e2b0 con 0x55ffb2cac5d0 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85b5983640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85b6184640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cac5d0 0x55ffb2caca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85b6184640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cac5d0 0x55ffb2caca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40550/0 (socket says 172.21.15.5:40550) 2024-09-19T06:06:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85b6184640 1 -- 172.21.15.5:0/3468482957 learned_addr learned my addr 172.21.15.5:0/3468482957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:32.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85b5983640 1 --2- 172.21.15.5:0/3468482957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cadb40 0x55ffb2cac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85b5983640 1 -- 172.21.15.5:0/3468482957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cac5d0 msgr2=0x55ffb2caca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:32.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85b5983640 1 --2- 172.21.15.5:0/3468482957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cac5d0 0x55ffb2caca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85b5983640 1 -- 172.21.15.5:0/3468482957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cadb40 msgr2=0x55ffb2cac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:32.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85a77fe640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f859c0a3430 0x7f859c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85b5983640 1 --2- 172.21.15.5:0/3468482957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cadb40 0x55ffb2cac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.357+0000 7f85b5983640 1 -- 172.21.15.5:0/3468482957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffb2b9e3a0 con 0x55ffb2cb7370 2024-09-19T06:06:32.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.358+0000 7f85b6985640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f859c0a3430 0x7f859c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.358+0000 7f85b5983640 1 --2- 172.21.15.5:0/3468482957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb9760 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009730 tx=0x7f85a00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9e2e18c153f483 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:32.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.358+0000 7f85a77fe640 1 -- 172.21.15.5:0/514893407 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f85a80f0a20 con 0x7f85b0102ef0 2024-09-19T06:06:32.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.358+0000 7f85a4dea640 1 -- 172.21.15.5:0/3468482957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0004050 con 0x55ffb2cb7370 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.358+0000 7f85a4dea640 1 -- 172.21.15.5:0/3468482957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85a000fc40 con 0x55ffb2cb7370 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.358+0000 7f85a4dea640 1 -- 172.21.15.5:0/3468482957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00038c0 con 0x55ffb2cb7370 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.359+0000 7f85ba7c1740 1 -- 172.21.15.5:0/3468482957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 msgr2=0x55ffb2cb9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.359+0000 7f85b6985640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f859c0a3430 0x7f859c0a58f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f85a80ee780 tx=0x7f85a802a950 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.359+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/3468482957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb9760 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009730 tx=0x7f85a00096c0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.359+0000 7f85ba7c1740 1 -- 172.21.15.5:0/3468482957 shutdown_connections 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.359+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/3468482957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb9760 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.359+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/3468482957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cac5d0 0x55ffb2caca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.359+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/3468482957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cadb40 0x55ffb2cac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.359+0000 7f85ba7c1740 1 -- 172.21.15.5:0/3468482957 >> 172.21.15.5:0/3468482957 conn(0x55ffb2c75310 msgr2=0x55ffb2ca8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.359+0000 7f85ba7c1740 1 -- 172.21.15.5:0/3468482957 shutdown_connections 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.359+0000 7f85ba7c1740 1 -- 172.21.15.5:0/3468482957 wait complete. 2024-09-19T06:06:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.359+0000 7f85ba7c1740 1 -- start start 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.359+0000 7f85ba7c1740 1 Processor -- start 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85ba7c1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 0x55ffb2cb6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85ba7c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cadb40 0x55ffb2cb4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85ba7c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85ba7c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffb2cba070 con 0x55ffb2cb7370 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85ba7c1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ffb2bcd060 con 0x55ffb2cac5d0 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85ba7c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ffb2cba1f0 con 0x55ffb2cadb40 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85ba7c1740 10 client.0 _ll_register_callbacks cb 0x55ffb29208c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85b5983640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 0x55ffb2cb6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85b5983640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 0x55ffb2cb6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48156/0 (socket says 172.21.15.5:48156) 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85b5983640 1 -- 172.21.15.5:0/4263078487 learned_addr learned my addr 172.21.15.5:0/4263078487 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85b6184640 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cadb40 0x55ffb2cb4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85b5983640 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85b6184640 1 -- 172.21.15.5:0/4263078487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 msgr2=0x55ffb2cb6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85b6184640 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 0x55ffb2cb6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85b6184640 1 -- 172.21.15.5:0/4263078487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 msgr2=0x55ffb2cb4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85b6184640 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85b6184640 1 -- 172.21.15.5:0/4263078487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffb29833c0 con 0x55ffb2cadb40 2024-09-19T06:06:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85b5983640 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 0x55ffb2cb6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f85b6184640 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cadb40 0x55ffb2cb4020 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f85ac005fd0 tx=0x7f85ac0039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ea78f501ac70676 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.360+0000 7f8599ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:06:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.361+0000 7f8599ffb640 1 -- 172.21.15.5:0/4263078487 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0117d0 con 0x55ffb2cadb40 2024-09-19T06:06:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.361+0000 7f8599ffb640 1 -- 172.21.15.5:0/4263078487 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f85ac011970 con 0x55ffb2cadb40 2024-09-19T06:06:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.361+0000 7f85ba7c1740 1 -- 172.21.15.5:0/4263078487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a000fc40 con 0x55ffb2cadb40 2024-09-19T06:06:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.361+0000 7f8599ffb640 1 -- 172.21.15.5:0/4263078487 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f85ac011b70 con 0x55ffb2cadb40 2024-09-19T06:06:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.361+0000 7f85ba7c1740 10 client.25723 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.361+0000 7f85ba7c1740 1 -- 172.21.15.5:0/4263078487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ffb2cc5b70 con 0x55ffb2cadb40 2024-09-19T06:06:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.361+0000 7f8599ffb640 1 -- 172.21.15.5:0/4263078487 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85a000fc40 con 0x55ffb2cadb40 2024-09-19T06:06:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.361+0000 7f8599ffb640 1 -- 172.21.15.5:0/4263078487 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f85ac010440 con 0x55ffb2cadb40 2024-09-19T06:06:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.361+0000 7f85ba7c1740 10 client.25723 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.362+0000 7f85ba7c1740 10 client.25723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:06:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.362+0000 7f85ba7c1740 20 client.25723 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.362+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55ffb2cc7080 0x55ffb2cc94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.362+0000 7f85ba7c1740 4 client.25723 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:06:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.362+0000 7f85ba7c1740 1 -- 172.21.15.5:0/4263078487 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffb2c1f860 con 0x55ffb2cc7080 2024-09-19T06:06:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.362+0000 7f85b5983640 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55ffb2cc7080 0x55ffb2cc94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.362+0000 7f8599ffb640 10 client.25723 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:06:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.362+0000 7f85b5983640 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55ffb2cc7080 0x55ffb2cc94a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d194a64da639ce2 server_cookie=c237b82ca75d5a12 in_seq=0 out_seq=0 2024-09-19T06:06:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f8599ffb640 1 -- 172.21.15.5:0/4263078487 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffb2c1f860 con 0x55ffb2cc7080 2024-09-19T06:06:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f8599ffb640 10 client.25723 handle_command_reply: tid=0 2024-09-19T06:06:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f85ba7c1740 1 client.25723 shutdown 2024-09-19T06:06:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f85ba7c1740 1 -- 172.21.15.5:0/4263078487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cadb40 msgr2=0x55ffb2cb4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cadb40 0x55ffb2cb4020 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f85ac005fd0 tx=0x7f85ac0039c0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f85ba7c1740 1 -- 172.21.15.5:0/4263078487 shutdown_connections 2024-09-19T06:06:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55ffb2cc7080 0x55ffb2cc94a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cadb40 0x55ffb2cb4020 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f8599ffb640 0 client.25723 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:06:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/4263078487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 0x55ffb2cb6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f85ba7c1740 1 -- 172.21.15.5:0/4263078487 >> 172.21.15.5:0/4263078487 conn(0x55ffb2c75310 msgr2=0x55ffb2cae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:32.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f85ba7c1740 1 -- 172.21.15.5:0/4263078487 shutdown_connections 2024-09-19T06:06:32.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f85ba7c1740 1 -- 172.21.15.5:0/4263078487 wait complete. 2024-09-19T06:06:32.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.363+0000 7f85ba7c1740 20 client.25723 trim_cache size 0 max 16384 2024-09-19T06:06:32.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.378+0000 7f85ba7c1740 1 Processor -- start 2024-09-19T06:06:32.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.378+0000 7f85ba7c1740 1 -- start start 2024-09-19T06:06:32.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85ba7c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cac5d0 0x55ffb2ca8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85ba7c1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cadb40 0x55ffb2cc3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85ba7c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85ba7c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01b3d60 con 0x55ffb2cac5d0 2024-09-19T06:06:32.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85ba7c1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01b5900 con 0x55ffb2cadb40 2024-09-19T06:06:32.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85ba7c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01b4430 con 0x55ffb2cb7370 2024-09-19T06:06:32.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85b5983640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cac5d0 0x55ffb2ca8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85b5983640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cac5d0 0x55ffb2ca8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34430/0 (socket says 172.21.15.5:34430) 2024-09-19T06:06:32.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85b6184640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cadb40 0x55ffb2cc3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85b5983640 1 -- 172.21.15.5:0/3285221656 learned_addr learned my addr 172.21.15.5:0/3285221656 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:32.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85b5983640 1 --2- 172.21.15.5:0/3285221656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85b6184640 1 -- 172.21.15.5:0/3285221656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cb7370 msgr2=0x55ffb2cb3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:32.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85b6184640 1 --2- 172.21.15.5:0/3285221656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85b6184640 1 -- 172.21.15.5:0/3285221656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cac5d0 msgr2=0x55ffb2ca8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85b6184640 1 --2- 172.21.15.5:0/3285221656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cac5d0 0x55ffb2ca8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85b6184640 1 -- 172.21.15.5:0/3285221656 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffb2cc9810 con 0x55ffb2cadb40 2024-09-19T06:06:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85b5983640 1 --2- 172.21.15.5:0/3285221656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:06:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.379+0000 7f85b6184640 1 --2- 172.21.15.5:0/3285221656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cadb40 0x55ffb2cc3310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0108e0 tx=0x7f85ac005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=999cf26d376e8544 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.380+0000 7f859bfff640 1 -- 172.21.15.5:0/3285221656 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac028040 con 0x55ffb2cadb40 2024-09-19T06:06:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.380+0000 7f859bfff640 1 -- 172.21.15.5:0/3285221656 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85ac003ec0 con 0x55ffb2cadb40 2024-09-19T06:06:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.380+0000 7f859bfff640 1 -- 172.21.15.5:0/3285221656 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac010ad0 con 0x55ffb2cadb40 2024-09-19T06:06:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.380+0000 7f85ba7c1740 1 -- 172.21.15.5:0/3285221656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cadb40 msgr2=0x55ffb2cc3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.380+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/3285221656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cadb40 0x55ffb2cc3310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0108e0 tx=0x7f85ac005900 comp rx=0 tx=0).stop 2024-09-19T06:06:32.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 -- 172.21.15.5:0/3285221656 shutdown_connections 2024-09-19T06:06:32.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/3285221656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/3285221656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cadb40 0x55ffb2cc3310 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/3285221656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cac5d0 0x55ffb2ca8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 -- 172.21.15.5:0/3285221656 >> 172.21.15.5:0/3285221656 conn(0x55ffb2c75310 msgr2=0x55ffb2cb2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:32.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 -- 172.21.15.5:0/3285221656 shutdown_connections 2024-09-19T06:06:32.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 -- 172.21.15.5:0/3285221656 wait complete. 2024-09-19T06:06:32.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 -- start start 2024-09-19T06:06:32.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 Processor -- start 2024-09-19T06:06:32.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 0x55ffb2cbf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cadb40 0x55ffb2cbfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01b5230 con 0x55ffb2cb7370 2024-09-19T06:06:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01b4b00 con 0x55ffb2cac5d0 2024-09-19T06:06:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.381+0000 7f85ba7c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01b1d40 con 0x55ffb2cadb40 2024-09-19T06:06:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85ba7c1740 10 client.0 _ll_register_callbacks cb 0x7f85ac00be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85b5983640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 0x55ffb2cbf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85b5983640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 0x55ffb2cbf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48168/0 (socket says 172.21.15.5:48168) 2024-09-19T06:06:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85b5983640 1 -- 172.21.15.5:0/1282401315 learned_addr learned my addr 172.21.15.5:0/1282401315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85b6184640 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cadb40 0x55ffb2cbfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85b5983640 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85b5983640 1 -- 172.21.15.5:0/1282401315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cadb40 msgr2=0x55ffb2cbfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85b5983640 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cadb40 0x55ffb2cbfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85b5983640 1 -- 172.21.15.5:0/1282401315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 msgr2=0x55ffb2cb0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85b5983640 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85b5983640 1 -- 172.21.15.5:0/1282401315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffb2cb0f10 con 0x55ffb2cac5d0 2024-09-19T06:06:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85b6184640 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cadb40 0x55ffb2cbfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f85b5983640 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 0x55ffb2cbf660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009a90 tx=0x7f85a00096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e40db795f0ec8f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f8599ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:06:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.382+0000 7f8599ffb640 1 -- 172.21.15.5:0/1282401315 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a000fb70 con 0x55ffb2cac5d0 2024-09-19T06:06:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f8599ffb640 1 -- 172.21.15.5:0/1282401315 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f85a0005030 con 0x55ffb2cac5d0 2024-09-19T06:06:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f85ba7c1740 1 -- 172.21.15.5:0/1282401315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ac003ec0 con 0x55ffb2cac5d0 2024-09-19T06:06:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f8599ffb640 1 -- 172.21.15.5:0/1282401315 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f85a000fd10 con 0x55ffb2cac5d0 2024-09-19T06:06:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f85ba7c1740 10 client.25805 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f85ba7c1740 1 -- 172.21.15.5:0/1282401315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ffb2d31650 con 0x55ffb2cac5d0 2024-09-19T06:06:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f8599ffb640 1 -- 172.21.15.5:0/1282401315 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85ac003ec0 con 0x55ffb2cac5d0 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f8599ffb640 1 -- 172.21.15.5:0/1282401315 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f85a002f040 con 0x55ffb2cac5d0 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f85ba7c1740 10 client.25805 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f85ba7c1740 10 client.25805 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f85ba7c1740 20 client.25805 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55ffb2d31bd0 0x55ffb2d33fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f85ba7c1740 4 client.25805 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f85ba7c1740 1 -- 172.21.15.5:0/1282401315 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffb2c19d70 con 0x55ffb2d31bd0 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.383+0000 7f85b6184640 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55ffb2d31bd0 0x55ffb2d33fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f8599ffb640 10 client.25805 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f85b6184640 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55ffb2d31bd0 0x55ffb2d33fc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b9bd6c6d9f5a126 server_cookie=9f1f60040d367d91 in_seq=0 out_seq=0 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f8599ffb640 1 -- 172.21.15.5:0/1282401315 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffb2c19d70 con 0x55ffb2d31bd0 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f8599ffb640 10 client.25805 handle_command_reply: tid=0 2024-09-19T06:06:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f85ba7c1740 1 client.25805 shutdown 2024-09-19T06:06:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f85ba7c1740 1 -- 172.21.15.5:0/1282401315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 msgr2=0x55ffb2cbf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 0x55ffb2cbf660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009a90 tx=0x7f85a00096c0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f85ba7c1740 1 -- 172.21.15.5:0/1282401315 shutdown_connections 2024-09-19T06:06:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55ffb2d31bd0 0x55ffb2d33fc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffb2cb7370 0x55ffb2cb0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f8599ffb640 0 client.25805 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:06:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffb2cadb40 0x55ffb2cbfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f85ba7c1740 1 --2- 172.21.15.5:0/1282401315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffb2cac5d0 0x55ffb2cbf660 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.384+0000 7f85ba7c1740 1 -- 172.21.15.5:0/1282401315 >> 172.21.15.5:0/1282401315 conn(0x55ffb2c75310 msgr2=0x55ffb2cb3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85ba7c1740 1 -- 172.21.15.5:0/1282401315 shutdown_connections 2024-09-19T06:06:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85ba7c1740 1 -- 172.21.15.5:0/1282401315 wait complete. 2024-09-19T06:06:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85ba7c1740 20 client.25805 trim_cache size 0 max 16384 2024-09-19T06:06:32.401 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:06:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85b83e8640 1 -- 172.21.15.5:0/514893407 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f859c0a3430 msgr2=0x7f859c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85b83e8640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f859c0a3430 0x7f859c0a58f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f85a80ee780 tx=0x7f85a802a950 comp rx=0 tx=0).stop 2024-09-19T06:06:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85b83e8640 1 -- 172.21.15.5:0/514893407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0102ef0 msgr2=0x7f85b019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85b83e8640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0102ef0 0x7f85b019eab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f85a8005cc0 tx=0x7f85a802f3a0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85b83e8640 1 -- 172.21.15.5:0/514893407 shutdown_connections 2024-09-19T06:06:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85b83e8640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0102ef0 0x7f85b019eab0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85b83e8640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f859c0a3430 0x7f859c0a58f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85b83e8640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b0102570 0x7f85b01a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85b83e8640 1 --2- 172.21.15.5:0/514893407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b0100e20 0x7f85b01a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.385+0000 7f85b83e8640 1 -- 172.21.15.5:0/514893407 >> 172.21.15.5:0/514893407 conn(0x7f85b00fc7d0 msgr2=0x7f85b006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.386+0000 7f85b83e8640 1 -- 172.21.15.5:0/514893407 shutdown_connections 2024-09-19T06:06:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:32.386+0000 7f85b83e8640 1 -- 172.21.15.5:0/514893407 wait complete. 2024-09-19T06:06:32.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:32 smithi005 ceph-mon[30955]: pgmap v99: 97 pgs: 97 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-19T06:06:32.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:32 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:06:32.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:32 smithi157 ceph-mon[35346]: pgmap v99: 97 pgs: 97 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-19T06:06:32.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:32 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:06:32.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 61516 2.95 MB/sec warmup 118 sec latency 15.344 ms 2024-09-19T06:06:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:32 smithi049 ceph-mon[34640]: pgmap v99: 97 pgs: 97 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-19T06:06:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:32 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:06:33.228 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:06:33.511 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:06:33.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:33 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:06:33.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:33 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:06:33.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:33 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:06:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:33 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:06:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:33 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:06:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:33 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:06:33.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 62161 2.95 MB/sec warmup 119 sec latency 15.673 ms 2024-09-19T06:06:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:33 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:06:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:33 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:06:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:33 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:06:34.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.032+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1498708181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90ac102ef0 msgr2=0x7f90ac10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:34.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.032+0000 7f90b1dcb640 1 --2- 172.21.15.5:0/1498708181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90ac102ef0 0x7f90ac10da40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009ca0 tx=0x7f90a402a950 comp rx=0 tx=0).stop 2024-09-19T06:06:34.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.032+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1498708181 shutdown_connections 2024-09-19T06:06:34.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.032+0000 7f90b1dcb640 1 --2- 172.21.15.5:0/1498708181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90ac102ef0 0x7f90ac10da40 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.032+0000 7f90b1dcb640 1 --2- 172.21.15.5:0/1498708181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac102570 0x7f90ac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.033+0000 7f90b1dcb640 1 --2- 172.21.15.5:0/1498708181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90ac100e20 0x7f90ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.033+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1498708181 >> 172.21.15.5:0/1498708181 conn(0x7f90ac0fc7d0 msgr2=0x7f90ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:34.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.033+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1498708181 shutdown_connections 2024-09-19T06:06:34.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.033+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1498708181 wait complete. 2024-09-19T06:06:34.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.033+0000 7f90b1dcb640 1 Processor -- start 2024-09-19T06:06:34.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.033+0000 7f90b1dcb640 1 -- start start 2024-09-19T06:06:34.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.033+0000 7f90b1dcb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac100e20 0x7f90ac198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.033+0000 7f90b1dcb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90ac102570 0x7f90ac1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.033+0000 7f90b1dcb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90ac102ef0 0x7f90ac19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.033+0000 7f90b1dcb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac110e30 con 0x7f90ac102570 2024-09-19T06:06:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.033+0000 7f90b1dcb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac110cb0 con 0x7f90ac100e20 2024-09-19T06:06:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.033+0000 7f90b1dcb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac110fb0 con 0x7f90ac102ef0 2024-09-19T06:06:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90ac102570 0x7f90ac1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90ab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac100e20 0x7f90ac198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90ac102570 0x7f90ac1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34464/0 (socket says 172.21.15.5:34464) 2024-09-19T06:06:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90ac102ef0 0x7f90ac19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90a3fff640 1 -- 172.21.15.5:0/1388900823 learned_addr learned my addr 172.21.15.5:0/1388900823 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90abfff640 1 -- 172.21.15.5:0/1388900823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac100e20 msgr2=0x7f90ac198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90abfff640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac100e20 0x7f90ac198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90abfff640 1 -- 172.21.15.5:0/1388900823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90ac102570 msgr2=0x7f90ac1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90abfff640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90ac102570 0x7f90ac1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90abfff640 1 -- 172.21.15.5:0/1388900823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac19d330 con 0x7f90ac102ef0 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90a3fff640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90ac102570 0x7f90ac1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90abfff640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90ac102ef0 0x7f90ac19cc50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f90a4005cc0 tx=0x7f90a4031f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90a97fa640 1 -- 172.21.15.5:0/1388900823 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4034040 con 0x7f90ac102ef0 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90a97fa640 1 -- 172.21.15.5:0/1388900823 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90a402f6b0 con 0x7f90ac102ef0 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1388900823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ac19d5c0 con 0x7f90ac102ef0 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.034+0000 7f90a97fa640 1 -- 172.21.15.5:0/1388900823 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4004430 con 0x7f90ac102ef0 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.035+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1388900823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ac1a4f00 con 0x7f90ac102ef0 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90a97fa640 1 -- 172.21.15.5:0/1388900823 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f90a4037020 con 0x7f90ac102ef0 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90b41a4740 1 Processor -- start 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90b41a4740 1 -- start start 2024-09-19T06:06:34.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90b41a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad199090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90b41a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad199a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90b41a4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1aab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90b41a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c2ad0ff2f0 con 0x55c2ad1995d0 2024-09-19T06:06:34.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90b41a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c2ad0a6cf0 con 0x55c2ad19ab40 2024-09-19T06:06:34.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90b41a4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c2ad0fb2b0 con 0x55c2ad1a8780 2024-09-19T06:06:34.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90a97fa640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90980a33b0 0x7f90980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34474/0 (socket says 172.21.15.5:34474) 2024-09-19T06:06:34.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.037+0000 7f90ab7fe640 1 -- 172.21.15.5:0/3961049940 learned_addr learned my addr 172.21.15.5:0/3961049940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:34.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90a3fff640 1 --2- 172.21.15.5:0/3961049940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90abfff640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90980a33b0 0x7f90980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90a97fa640 1 -- 172.21.15.5:0/1388900823 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f90a40f0970 con 0x7f90ac102ef0 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90ab7fe640 1 -- 172.21.15.5:0/3961049940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 msgr2=0x55c2ad1aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90ab7fe640 1 --2- 172.21.15.5:0/3961049940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1aab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90ab7fe640 1 -- 172.21.15.5:0/3961049940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 msgr2=0x55c2ad199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90ab7fe640 1 --2- 172.21.15.5:0/3961049940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90ab7fe640 1 -- 172.21.15.5:0/3961049940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2ad08b3a0 con 0x55c2ad1995d0 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90a3fff640 1 --2- 172.21.15.5:0/3961049940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90ab7fe640 1 --2- 172.21.15.5:0/3961049940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad199a10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f909c009d20 tx=0x7f909c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d144f850c96f50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90abfff640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90980a33b0 0x7f90980a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f90a40b9170 tx=0x7f90a408ba00 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90a1ffb640 1 -- 172.21.15.5:0/3961049940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c01e070 con 0x55c2ad1995d0 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90a1ffb640 1 -- 172.21.15.5:0/3961049940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f909c019ae0 con 0x55c2ad1995d0 2024-09-19T06:06:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.038+0000 7f90a1ffb640 1 -- 172.21.15.5:0/3961049940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c00ea10 con 0x55c2ad1995d0 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.039+0000 7f90b41a4740 1 -- 172.21.15.5:0/3961049940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 msgr2=0x55c2ad199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.039+0000 7f90b41a4740 1 --2- 172.21.15.5:0/3961049940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad199a10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f909c009d20 tx=0x7f909c011810 comp rx=0 tx=0).stop 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.039+0000 7f90b41a4740 1 -- 172.21.15.5:0/3961049940 shutdown_connections 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.039+0000 7f90b41a4740 1 --2- 172.21.15.5:0/3961049940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.039+0000 7f90b41a4740 1 --2- 172.21.15.5:0/3961049940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad199a10 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.039+0000 7f90b41a4740 1 --2- 172.21.15.5:0/3961049940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.039+0000 7f90b41a4740 1 -- 172.21.15.5:0/3961049940 >> 172.21.15.5:0/3961049940 conn(0x55c2ad162310 msgr2=0x55c2ad195c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.039+0000 7f90b41a4740 1 -- 172.21.15.5:0/3961049940 shutdown_connections 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.039+0000 7f90b41a4740 1 -- 172.21.15.5:0/3961049940 wait complete. 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90b41a4740 1 -- start start 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90b41a4740 1 Processor -- start 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90b41a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad1a8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90b41a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1a5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90b41a4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90b41a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c2ad1ab480 con 0x55c2ad1995d0 2024-09-19T06:06:34.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90b41a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c2ad0ba060 con 0x55c2ad19ab40 2024-09-19T06:06:34.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90b41a4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c2ad1ab600 con 0x55c2ad1a8780 2024-09-19T06:06:34.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90b41a4740 10 client.0 _ll_register_callbacks cb 0x55c2ace0d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:34.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad1a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90ab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad1a8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34488/0 (socket says 172.21.15.5:34488) 2024-09-19T06:06:34.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.040+0000 7f90a3fff640 1 -- 172.21.15.5:0/3852435533 learned_addr learned my addr 172.21.15.5:0/3852435533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:34.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.041+0000 7f90ab7fe640 1 -- 172.21.15.5:0/3852435533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 msgr2=0x55c2ad1a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:34.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.041+0000 7f90ab7fe640 1 --2- 172.21.15.5:0/3852435533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.041+0000 7f90ab7fe640 1 -- 172.21.15.5:0/3852435533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 msgr2=0x55c2ad1a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.041+0000 7f90ab7fe640 1 --2- 172.21.15.5:0/3852435533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad1a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.041+0000 7f90ab7fe640 1 -- 172.21.15.5:0/3852435533 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2ace703c0 con 0x55c2ad19ab40 2024-09-19T06:06:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.041+0000 7f90ab7fe640 1 --2- 172.21.15.5:0/3852435533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1a5400 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f909c01c040 tx=0x7f909c019660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e66c34cbfa4ea09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.041+0000 7f908affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:06:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.041+0000 7f90a3fff640 1 --2- 172.21.15.5:0/3852435533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad1a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.041+0000 7f908affd640 1 -- 172.21.15.5:0/3852435533 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c027020 con 0x55c2ad19ab40 2024-09-19T06:06:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.041+0000 7f908affd640 1 -- 172.21.15.5:0/3852435533 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f909c005250 con 0x55c2ad19ab40 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.041+0000 7f90b41a4740 1 -- 172.21.15.5:0/3852435533 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f909c019ae0 con 0x55c2ad19ab40 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.042+0000 7f908affd640 1 -- 172.21.15.5:0/3852435533 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f909c032050 con 0x55c2ad19ab40 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.042+0000 7f90b41a4740 10 client.25817 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.042+0000 7f90b41a4740 1 -- 172.21.15.5:0/3852435533 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c2ad1b6ef0 con 0x55c2ad19ab40 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.042+0000 7f908affd640 1 -- 172.21.15.5:0/3852435533 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f909c01f040 con 0x55c2ad19ab40 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.042+0000 7f908affd640 1 -- 172.21.15.5:0/3852435533 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f909c0046c0 con 0x55c2ad19ab40 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.042+0000 7f90b41a4740 10 client.25817 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.042+0000 7f90b41a4740 10 client.25817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.042+0000 7f90b41a4740 20 client.25817 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.042+0000 7f90b41a4740 1 --2- 172.21.15.5:0/3852435533 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c2ad1b82f0 0x55c2ad1ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.042+0000 7f90b41a4740 4 client.25817 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.042+0000 7f90b41a4740 1 -- 172.21.15.5:0/3852435533 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f909c019ae0 con 0x55c2ad1b82f0 2024-09-19T06:06:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.042+0000 7f90a3fff640 1 --2- 172.21.15.5:0/3852435533 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c2ad1b82f0 0x55c2ad1ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.043+0000 7f908affd640 10 client.25817 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.043+0000 7f90a3fff640 1 --2- 172.21.15.5:0/3852435533 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c2ad1b82f0 0x55c2ad1ba6e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2b9655668a992f server_cookie=3e06bed161991ffb in_seq=0 out_seq=0 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f908affd640 1 -- 172.21.15.5:0/3852435533 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f909c019ae0 con 0x55c2ad1b82f0 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f908affd640 10 client.25817 handle_command_reply: tid=0 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f90b41a4740 1 client.25817 shutdown 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f90b41a4740 1 -- 172.21.15.5:0/3852435533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 msgr2=0x55c2ad1a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f90b41a4740 1 --2- 172.21.15.5:0/3852435533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1a5400 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f909c01c040 tx=0x7f909c019660 comp rx=0 tx=0).stop 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f90b41a4740 1 -- 172.21.15.5:0/3852435533 shutdown_connections 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f90b41a4740 1 --2- 172.21.15.5:0/3852435533 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c2ad1b82f0 0x55c2ad1ba6e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f908affd640 0 client.25817 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f90b41a4740 1 --2- 172.21.15.5:0/3852435533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f90b41a4740 1 --2- 172.21.15.5:0/3852435533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1a5400 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f90b41a4740 1 --2- 172.21.15.5:0/3852435533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad1a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f90b41a4740 1 -- 172.21.15.5:0/3852435533 >> 172.21.15.5:0/3852435533 conn(0x55c2ad162310 msgr2=0x55c2ad19b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:34.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f90b41a4740 1 -- 172.21.15.5:0/3852435533 shutdown_connections 2024-09-19T06:06:34.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f90b41a4740 1 -- 172.21.15.5:0/3852435533 wait complete. 2024-09-19T06:06:34.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.044+0000 7f90b41a4740 20 client.25817 trim_cache size 0 max 16384 2024-09-19T06:06:34.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90b41a4740 1 Processor -- start 2024-09-19T06:06:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90b41a4740 1 -- start start 2024-09-19T06:06:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90b41a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad195e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90b41a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90b41a4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1bfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90b41a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1ab430 con 0x55c2ad1995d0 2024-09-19T06:06:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90b41a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1acfd0 con 0x55c2ad19ab40 2024-09-19T06:06:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90b41a4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1abb00 con 0x55c2ad1a8780 2024-09-19T06:06:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34492/0 (socket says 172.21.15.5:34492) 2024-09-19T06:06:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90a3fff640 1 -- 172.21.15.5:0/1796157515 learned_addr learned my addr 172.21.15.5:0/1796157515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90ab7fe640 1 --2- 172.21.15.5:0/1796157515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90a3fff640 1 --2- 172.21.15.5:0/1796157515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1bfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90a3fff640 1 -- 172.21.15.5:0/1796157515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 msgr2=0x55c2ad1bfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:34.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90a3fff640 1 --2- 172.21.15.5:0/1796157515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1bfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90a3fff640 1 -- 172.21.15.5:0/1796157515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 msgr2=0x55c2ad1b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:34.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90a3fff640 1 --2- 172.21.15.5:0/1796157515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.062+0000 7f90a3fff640 1 -- 172.21.15.5:0/1796157515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2ad1a84c0 con 0x55c2ad1995d0 2024-09-19T06:06:34.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.063+0000 7f90a3fff640 1 --2- 172.21.15.5:0/1796157515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad195e10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9094009f60 tx=0x7f9094009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895a1b034fe0b04c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:34.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.063+0000 7f90a17fa640 1 -- 172.21.15.5:0/1796157515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094024070 con 0x55c2ad1995d0 2024-09-19T06:06:34.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.063+0000 7f90a17fa640 1 -- 172.21.15.5:0/1796157515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f909400fca0 con 0x55c2ad1995d0 2024-09-19T06:06:34.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.063+0000 7f90a17fa640 1 -- 172.21.15.5:0/1796157515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094005310 con 0x55c2ad1995d0 2024-09-19T06:06:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.064+0000 7f90b41a4740 1 -- 172.21.15.5:0/1796157515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 msgr2=0x55c2ad195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.064+0000 7f90b41a4740 1 --2- 172.21.15.5:0/1796157515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad195e10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9094009f60 tx=0x7f9094009f90 comp rx=0 tx=0).stop 2024-09-19T06:06:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.064+0000 7f90b41a4740 1 -- 172.21.15.5:0/1796157515 shutdown_connections 2024-09-19T06:06:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.064+0000 7f90b41a4740 1 --2- 172.21.15.5:0/1796157515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1bfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.064+0000 7f90b41a4740 1 --2- 172.21.15.5:0/1796157515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.064+0000 7f90b41a4740 1 --2- 172.21.15.5:0/1796157515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad195e10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.064+0000 7f90b41a4740 1 -- 172.21.15.5:0/1796157515 >> 172.21.15.5:0/1796157515 conn(0x55c2ad162310 msgr2=0x55c2ad195700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:34.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.064+0000 7f90b41a4740 1 -- 172.21.15.5:0/1796157515 shutdown_connections 2024-09-19T06:06:34.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.064+0000 7f90b41a4740 1 -- 172.21.15.5:0/1796157515 wait complete. 2024-09-19T06:06:34.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.064+0000 7f90b41a4740 1 -- start start 2024-09-19T06:06:34.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90b41a4740 1 Processor -- start 2024-09-19T06:06:34.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90b41a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad1b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90b41a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1a2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90b41a4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90b41a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1ac900 con 0x55c2ad1995d0 2024-09-19T06:06:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90b41a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1ac1d0 con 0x55c2ad19ab40 2024-09-19T06:06:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90b41a4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1a9410 con 0x55c2ad1a8780 2024-09-19T06:06:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90b41a4740 10 client.0 _ll_register_callbacks cb 0x7f909c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad1b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad1b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34498/0 (socket says 172.21.15.5:34498) 2024-09-19T06:06:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90a3fff640 1 -- 172.21.15.5:0/598185877 learned_addr learned my addr 172.21.15.5:0/598185877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90a3fff640 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90ab7fe640 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90a3fff640 1 -- 172.21.15.5:0/598185877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 msgr2=0x55c2ad1a2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90a3fff640 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1a2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90a3fff640 1 -- 172.21.15.5:0/598185877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 msgr2=0x55c2ad1a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90a3fff640 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.065+0000 7f90a3fff640 1 -- 172.21.15.5:0/598185877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2ad221e80 con 0x55c2ad1995d0 2024-09-19T06:06:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.066+0000 7f90ab7fe640 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:06:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.066+0000 7f90a3fff640 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad1b1180 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f909400ffb0 tx=0x7f9094011360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa2b6f728ab5536 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.066+0000 7f908affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:06:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.066+0000 7f908affd640 1 -- 172.21.15.5:0/598185877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094024050 con 0x55c2ad1995d0 2024-09-19T06:06:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.066+0000 7f908affd640 1 -- 172.21.15.5:0/598185877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f9094003e40 con 0x55c2ad1995d0 2024-09-19T06:06:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.066+0000 7f90b41a4740 1 -- 172.21.15.5:0/598185877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f909400fca0 con 0x55c2ad1995d0 2024-09-19T06:06:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.066+0000 7f908affd640 1 -- 172.21.15.5:0/598185877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f909403e050 con 0x55c2ad1995d0 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.066+0000 7f90b41a4740 10 client.16224 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.066+0000 7f90b41a4740 1 -- 172.21.15.5:0/598185877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c2ad223020 con 0x55c2ad1995d0 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.066+0000 7f908affd640 1 -- 172.21.15.5:0/598185877 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f909402e070 con 0x55c2ad1995d0 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.067+0000 7f908affd640 1 -- 172.21.15.5:0/598185877 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f9094020040 con 0x55c2ad1995d0 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.067+0000 7f90b41a4740 10 client.16224 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.067+0000 7f90b41a4740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.067+0000 7f90b41a4740 20 client.16224 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.067+0000 7f90b41a4740 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c2ad1b82f0 0x55c2ad225890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.067+0000 7f90b41a4740 4 client.16224 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.067+0000 7f90b41a4740 1 -- 172.21.15.5:0/598185877 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f909400fca0 con 0x55c2ad1b82f0 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.067+0000 7f90ab7fe640 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c2ad1b82f0 0x55c2ad225890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f90ab7fe640 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c2ad1b82f0 0x55c2ad225890 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=858d664af63a4737 server_cookie=a344beeedf95f11a in_seq=0 out_seq=0 2024-09-19T06:06:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f908affd640 10 client.16224 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f908affd640 1 -- 172.21.15.5:0/598185877 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f909400fca0 con 0x55c2ad1b82f0 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f908affd640 10 client.16224 handle_command_reply: tid=0 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f90b41a4740 1 client.16224 shutdown 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f90b41a4740 1 -- 172.21.15.5:0/598185877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 msgr2=0x55c2ad1b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f90b41a4740 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad1b1180 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f909400ffb0 tx=0x7f9094011360 comp rx=0 tx=0).stop 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f90b41a4740 1 -- 172.21.15.5:0/598185877 shutdown_connections 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f90b41a4740 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c2ad1b82f0 0x55c2ad225890 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f908affd640 0 client.16224 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f90b41a4740 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ad1a8780 0x55c2ad1a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f90b41a4740 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2ad19ab40 0x55c2ad1a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f90b41a4740 1 --2- 172.21.15.5:0/598185877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2ad1995d0 0x55c2ad1b1180 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f90b41a4740 1 -- 172.21.15.5:0/598185877 >> 172.21.15.5:0/598185877 conn(0x55c2ad162310 msgr2=0x55c2ad19b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f90b41a4740 1 -- 172.21.15.5:0/598185877 shutdown_connections 2024-09-19T06:06:34.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.068+0000 7f90b41a4740 1 -- 172.21.15.5:0/598185877 wait complete. 2024-09-19T06:06:34.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b41a4740 20 client.16224 trim_cache size 0 max 16384 2024-09-19T06:06:34.077 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:06:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1388900823 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90980a33b0 msgr2=0x7f90980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b1dcb640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90980a33b0 0x7f90980a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f90a40b9170 tx=0x7f90a408ba00 comp rx=0 tx=0).stop 2024-09-19T06:06:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1388900823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90ac102ef0 msgr2=0x7f90ac19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b1dcb640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90ac102ef0 0x7f90ac19cc50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f90a4005cc0 tx=0x7f90a4031f40 comp rx=0 tx=0).stop 2024-09-19T06:06:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1388900823 shutdown_connections 2024-09-19T06:06:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b1dcb640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90980a33b0 0x7f90980a5870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b1dcb640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90ac102ef0 0x7f90ac19cc50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b1dcb640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90ac102570 0x7f90ac1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b1dcb640 1 --2- 172.21.15.5:0/1388900823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac100e20 0x7f90ac198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1388900823 >> 172.21.15.5:0/1388900823 conn(0x7f90ac0fc7d0 msgr2=0x7f90ac06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1388900823 shutdown_connections 2024-09-19T06:06:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:34.069+0000 7f90b1dcb640 1 -- 172.21.15.5:0/1388900823 wait complete. 2024-09-19T06:06:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:34 smithi157 ceph-mon[35346]: pgmap v100: 97 pgs: 97 active+clean; 49 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-09-19T06:06:34.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:34 smithi005 ceph-mon[30955]: pgmap v100: 97 pgs: 97 active+clean; 49 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-09-19T06:06:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:34 smithi049 ceph-mon[34640]: pgmap v100: 97 pgs: 97 active+clean; 49 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-09-19T06:06:35.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 63536 6.18 MB/sec execute 1 sec latency 15.078 ms 2024-09-19T06:06:35.880 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:06:35.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:06:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.084+0000 7fcbe5533640 1 -- 172.21.15.5:0/1004514656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbe01009a0 msgr2=0x7fcbe0100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.084+0000 7fcbe5533640 1 --2- 172.21.15.5:0/1004514656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbe01009a0 0x7fcbe0100e00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009930 tx=0x7fcbd402a420 comp rx=0 tx=0).stop 2024-09-19T06:06:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.084+0000 7fcbe5533640 1 -- 172.21.15.5:0/1004514656 shutdown_connections 2024-09-19T06:06:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.084+0000 7fcbe5533640 1 --2- 172.21.15.5:0/1004514656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbe0101340 0x7fcbe010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.084+0000 7fcbe5533640 1 --2- 172.21.15.5:0/1004514656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbe01009a0 0x7fcbe0100e00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.084+0000 7fcbe5533640 1 --2- 172.21.15.5:0/1004514656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe01069b0 0x7fcbe0106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.084+0000 7fcbe5533640 1 -- 172.21.15.5:0/1004514656 >> 172.21.15.5:0/1004514656 conn(0x7fcbe00779a0 msgr2=0x7fcbe00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.084+0000 7fcbe5533640 1 -- 172.21.15.5:0/1004514656 shutdown_connections 2024-09-19T06:06:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.084+0000 7fcbe5533640 1 -- 172.21.15.5:0/1004514656 wait complete. 2024-09-19T06:06:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbe5533640 1 Processor -- start 2024-09-19T06:06:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbe5533640 1 -- start start 2024-09-19T06:06:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbe5533640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbe01009a0 0x7fcbe00fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbe5533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe0101340 0x7fcbe00fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbe5533640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbe01069b0 0x7fcbe00fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbe5533640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0110cb0 con 0x7fcbe01009a0 2024-09-19T06:06:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbe5533640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0110b30 con 0x7fcbe01069b0 2024-09-19T06:06:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbe5533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0110e30 con 0x7fcbe0101340 2024-09-19T06:06:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbdeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbe01009a0 0x7fcbe00fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbdeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbe01009a0 0x7fcbe00fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52098/0 (socket says 172.21.15.5:52098) 2024-09-19T06:06:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbdeffd640 1 -- 172.21.15.5:0/1065291633 learned_addr learned my addr 172.21.15.5:0/1065291633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbe01069b0 0x7fcbe00fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.085+0000 7fcbde7fc640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe0101340 0x7fcbe00fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.086+0000 7fcbdeffd640 1 -- 172.21.15.5:0/1065291633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe0101340 msgr2=0x7fcbe00fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.086+0000 7fcbdeffd640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe0101340 0x7fcbe00fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.086+0000 7fcbdeffd640 1 -- 172.21.15.5:0/1065291633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbe01069b0 msgr2=0x7fcbe00fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.086+0000 7fcbdeffd640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbe01069b0 0x7fcbe00fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.086+0000 7fcbdeffd640 1 -- 172.21.15.5:0/1065291633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe01a6880 con 0x7fcbe01009a0 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.086+0000 7fcbdeffd640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbe01009a0 0x7fcbe00fcde0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000ea60 tx=0x7fcbd000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.086+0000 7fcbc7fff640 1 -- 172.21.15.5:0/1065291633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000cde0 con 0x7fcbe01009a0 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.086+0000 7fcbc7fff640 1 -- 172.21.15.5:0/1065291633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbd00035f0 con 0x7fcbe01009a0 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.086+0000 7fcbe5533640 1 -- 172.21.15.5:0/1065291633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe01a6b70 con 0x7fcbe01009a0 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.086+0000 7fcbc7fff640 1 -- 172.21.15.5:0/1065291633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0010690 con 0x7fcbe01009a0 2024-09-19T06:06:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.086+0000 7fcbe5533640 1 -- 172.21.15.5:0/1065291633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe01a70e0 con 0x7fcbe01009a0 2024-09-19T06:06:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbe82c8740 1 Processor -- start 2024-09-19T06:06:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbe82c8740 1 -- start start 2024-09-19T06:06:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbc7fff640 1 -- 172.21.15.5:0/1065291633 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcbd0010830 con 0x7fcbe01009a0 2024-09-19T06:06:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbe82c8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda0f7af0 0x559dda0f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbe82c8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda102770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbe82c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda102cb0 0x559dda1050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbe82c8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559dda05c2f0 con 0x559dda0f8410 2024-09-19T06:06:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbe82c8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559dda004270 con 0x559dda0f7af0 2024-09-19T06:06:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbe82c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559dda0582b0 con 0x559dda102cb0 2024-09-19T06:06:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbdeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda0f7af0 0x559dda0f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda0f7af0 0x559dda0f7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57468/0 (socket says 172.21.15.5:57468) 2024-09-19T06:06:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbdf7fe640 1 -- 172.21.15.5:0/3098483694 learned_addr learned my addr 172.21.15.5:0/3098483694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.089+0000 7fcbde7fc640 1 --2- 172.21.15.5:0/3098483694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda102cb0 0x559dda1050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.090+0000 7fcbdf7fe640 1 -- 172.21.15.5:0/3098483694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda102cb0 msgr2=0x559dda1050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.090+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/3098483694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda102cb0 0x559dda1050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.090+0000 7fcbdf7fe640 1 -- 172.21.15.5:0/3098483694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 msgr2=0x559dda102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.090+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/3098483694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.090+0000 7fcbdf7fe640 1 -- 172.21.15.5:0/3098483694 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dda003340 con 0x559dda0f7af0 2024-09-19T06:06:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.090+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/3098483694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda0f7af0 0x559dda0f7ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009fd0 tx=0x7fcbd80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75841df493870a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.090+0000 7fcbc7fff640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcbc80a3430 0x7fcbc80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.090+0000 7fcbc57fa640 1 -- 172.21.15.5:0/3098483694 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8004d20 con 0x559dda0f7af0 2024-09-19T06:06:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.090+0000 7fcbc57fa640 1 -- 172.21.15.5:0/3098483694 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbd8004050 con 0x559dda0f7af0 2024-09-19T06:06:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.090+0000 7fcbc57fa640 1 -- 172.21.15.5:0/3098483694 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8005280 con 0x559dda0f7af0 2024-09-19T06:06:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.090+0000 7fcbdeffd640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcbc80a3430 0x7fcbc80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.090+0000 7fcbc7fff640 1 -- 172.21.15.5:0/1065291633 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fcbd00cfe80 con 0x7fcbe01009a0 2024-09-19T06:06:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.091+0000 7fcbe82c8740 1 -- 172.21.15.5:0/3098483694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda0f7af0 msgr2=0x559dda0f7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.091+0000 7fcbdeffd640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcbc80a3430 0x7fcbc80a58f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00cd690 tx=0x7fcbd00bc040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.091+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/3098483694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda0f7af0 0x559dda0f7ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009fd0 tx=0x7fcbd80118d0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.091+0000 7fcbe82c8740 1 -- 172.21.15.5:0/3098483694 shutdown_connections 2024-09-19T06:06:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.091+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/3098483694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda102cb0 0x559dda1050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.091+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/3098483694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.091+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/3098483694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda0f7af0 0x559dda0f7ed0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.091+0000 7fcbe82c8740 1 -- 172.21.15.5:0/3098483694 >> 172.21.15.5:0/3098483694 conn(0x559dda0f4060 msgr2=0x559dda0f4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.091+0000 7fcbe82c8740 1 -- 172.21.15.5:0/3098483694 shutdown_connections 2024-09-19T06:06:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.091+0000 7fcbe82c8740 1 -- 172.21.15.5:0/3098483694 wait complete. 2024-09-19T06:06:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbe82c8740 1 -- start start 2024-09-19T06:06:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbe82c8740 1 Processor -- start 2024-09-19T06:06:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbe82c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f7af0 0x559dda0ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbe82c8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda100040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbe82c8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda100580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbe82c8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559dda105880 con 0x559dda0f8410 2024-09-19T06:06:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbe82c8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559dd9fe5290 con 0x559dda102cb0 2024-09-19T06:06:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbe82c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559dda105a00 con 0x559dda0f7af0 2024-09-19T06:06:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbe82c8740 10 client.0 _ll_register_callbacks cb 0x559dda016140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbde7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbde7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52124/0 (socket says 172.21.15.5:52124) 2024-09-19T06:06:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbde7fc640 1 -- 172.21.15.5:0/4077009431 learned_addr learned my addr 172.21.15.5:0/4077009431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.092+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f7af0 0x559dda0ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbde7fc640 1 -- 172.21.15.5:0/4077009431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f7af0 msgr2=0x559dda0ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbde7fc640 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f7af0 0x559dda0ffb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbde7fc640 1 -- 172.21.15.5:0/4077009431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 msgr2=0x559dda100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbde7fc640 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbde7fc640 1 -- 172.21.15.5:0/4077009431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd9d6f210 con 0x559dda0f8410 2024-09-19T06:06:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbde7fc640 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda100040 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcbd402aa10 tx=0x7fcbd40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8722e5f3ebc4aab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:06:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbc2ff5640 1 -- 172.21.15.5:0/4077009431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4035040 con 0x559dda0f8410 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbc2ff5640 1 -- 172.21.15.5:0/4077009431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fcbd4005450 con 0x559dda0f8410 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbe82c8740 1 -- 172.21.15.5:0/4077009431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbd8004050 con 0x559dda0f8410 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbc2ff5640 1 -- 172.21.15.5:0/4077009431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcbd4005650 con 0x559dda0f8410 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbe82c8740 10 client.16236 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbe82c8740 1 -- 172.21.15.5:0/4077009431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559dd9e3b620 con 0x559dda0f8410 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.093+0000 7fcbc2ff5640 1 -- 172.21.15.5:0/4077009431 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbd8004050 con 0x559dda0f8410 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.094+0000 7fcbc2ff5640 1 -- 172.21.15.5:0/4077009431 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fcbd4003a70 con 0x559dda0f8410 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.094+0000 7fcbe82c8740 10 client.16236 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.094+0000 7fcbe82c8740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.094+0000 7fcbe82c8740 20 client.16236 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.094+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559dda112be0 0x559dda114fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.094+0000 7fcbe82c8740 4 client.16236 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:06:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.094+0000 7fcbe82c8740 1 -- 172.21.15.5:0/4077009431 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dda069a20 con 0x559dda112be0 2024-09-19T06:06:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.094+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559dda112be0 0x559dda114fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.094+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559dda112be0 0x559dda114fd0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c155067de4e96f93 server_cookie=804821837e0207fe in_seq=0 out_seq=0 2024-09-19T06:06:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.094+0000 7fcbc2ff5640 10 client.16236 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbc2ff5640 1 -- 172.21.15.5:0/4077009431 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559dda069a20 con 0x559dda112be0 2024-09-19T06:06:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbc2ff5640 10 client.16236 handle_command_reply: tid=0 2024-09-19T06:06:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbe82c8740 1 client.16236 shutdown 2024-09-19T06:06:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbe82c8740 1 -- 172.21.15.5:0/4077009431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 msgr2=0x559dda100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda100040 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcbd402aa10 tx=0x7fcbd40043b0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbe82c8740 1 -- 172.21.15.5:0/4077009431 shutdown_connections 2024-09-19T06:06:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559dda112be0 0x559dda114fd0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda100040 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbc2ff5640 0 client.16236 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/4077009431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f7af0 0x559dda0ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbe82c8740 1 -- 172.21.15.5:0/4077009431 >> 172.21.15.5:0/4077009431 conn(0x559dda0f4060 msgr2=0x559dda0f9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.095+0000 7fcbe82c8740 1 -- 172.21.15.5:0/4077009431 shutdown_connections 2024-09-19T06:06:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.096+0000 7fcbe82c8740 1 -- 172.21.15.5:0/4077009431 wait complete. 2024-09-19T06:06:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.096+0000 7fcbe82c8740 20 client.16236 trim_cache size 0 max 16384 2024-09-19T06:06:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.121+0000 7fcbe82c8740 1 Processor -- start 2024-09-19T06:06:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.121+0000 7fcbe82c8740 1 -- start start 2024-09-19T06:06:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.121+0000 7fcbe82c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f7af0 0x559dda1003b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.121+0000 7fcbe82c8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda11a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.121+0000 7fcbe82c8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda11ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.121+0000 7fcbe82c8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01af950 con 0x559dda0f8410 2024-09-19T06:06:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.121+0000 7fcbe82c8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01b14f0 con 0x559dda102cb0 2024-09-19T06:06:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.121+0000 7fcbe82c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01b0020 con 0x559dda0f7af0 2024-09-19T06:06:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbde7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda11a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda11ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbde7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda11a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52136/0 (socket says 172.21.15.5:52136) 2024-09-19T06:06:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbde7fc640 1 -- 172.21.15.5:0/1316963541 learned_addr learned my addr 172.21.15.5:0/1316963541 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda11ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57478/0 (socket says 172.21.15.5:57478) 2024-09-19T06:06:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/1316963541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f7af0 0x559dda1003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbde7fc640 1 -- 172.21.15.5:0/1316963541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f7af0 msgr2=0x559dda1003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbde7fc640 1 --2- 172.21.15.5:0/1316963541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f7af0 0x559dda1003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbde7fc640 1 -- 172.21.15.5:0/1316963541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 msgr2=0x559dda11ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbde7fc640 1 --2- 172.21.15.5:0/1316963541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda11ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbde7fc640 1 -- 172.21.15.5:0/1316963541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dda0f3e90 con 0x559dda0f8410 2024-09-19T06:06:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/1316963541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda11ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbde7fc640 1 --2- 172.21.15.5:0/1316963541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda11a8a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4005ab0 tx=0x7fcbd402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7c4b42ea73787a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbc4ff9640 1 -- 172.21.15.5:0/1316963541 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4035030 con 0x559dda0f8410 2024-09-19T06:06:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbc4ff9640 1 -- 172.21.15.5:0/1316963541 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbd4041070 con 0x559dda0f8410 2024-09-19T06:06:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.122+0000 7fcbc4ff9640 1 -- 172.21.15.5:0/1316963541 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40054b0 con 0x559dda0f8410 2024-09-19T06:06:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.123+0000 7fcbe82c8740 1 -- 172.21.15.5:0/1316963541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 msgr2=0x559dda11a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.123+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/1316963541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda11a8a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4005ab0 tx=0x7fcbd402a420 comp rx=0 tx=0).stop 2024-09-19T06:06:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.123+0000 7fcbe82c8740 1 -- 172.21.15.5:0/1316963541 shutdown_connections 2024-09-19T06:06:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.123+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/1316963541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda11ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.123+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/1316963541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f8410 0x559dda11a8a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.123+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/1316963541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f7af0 0x559dda1003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.123+0000 7fcbe82c8740 1 -- 172.21.15.5:0/1316963541 >> 172.21.15.5:0/1316963541 conn(0x559dda0f4060 msgr2=0x559dda101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.123+0000 7fcbe82c8740 1 -- 172.21.15.5:0/1316963541 shutdown_connections 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.123+0000 7fcbe82c8740 1 -- 172.21.15.5:0/1316963541 wait complete. 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbe82c8740 1 -- start start 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbe82c8740 1 Processor -- start 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbe82c8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f7af0 0x559dda10b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbe82c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f8410 0x559dda0fc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbe82c8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda0fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbe82c8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01b0e20 con 0x559dda0f7af0 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbe82c8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01b06f0 con 0x559dda102cb0 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbe82c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01ad930 con 0x559dda0f8410 2024-09-19T06:06:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbe82c8740 10 client.0 _ll_register_callbacks cb 0x7fcbd4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f7af0 0x559dda10b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f7af0 0x559dda10b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52144/0 (socket says 172.21.15.5:52144) 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbdf7fe640 1 -- 172.21.15.5:0/2541430250 learned_addr learned my addr 172.21.15.5:0/2541430250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbde7fc640 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f8410 0x559dda0fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.124+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda0fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbde7fc640 1 -- 172.21.15.5:0/2541430250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 msgr2=0x559dda0fcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbde7fc640 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda0fcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbde7fc640 1 -- 172.21.15.5:0/2541430250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f7af0 msgr2=0x559dda10b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbde7fc640 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f7af0 0x559dda10b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbde7fc640 1 -- 172.21.15.5:0/2541430250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dda0ef2d0 con 0x559dda0f8410 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f7af0 0x559dda10b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbde7fc640 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f8410 0x559dda0fc620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x559dda0a26e0 tx=0x7fcbd4005480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f85420cad926f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbc2ff5640 1 -- 172.21.15.5:0/2541430250 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4031a80 con 0x559dda0f8410 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbe82c8740 1 -- 172.21.15.5:0/2541430250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbd4041070 con 0x559dda0f8410 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbc2ff5640 1 -- 172.21.15.5:0/2541430250 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fcbd4031c20 con 0x559dda0f8410 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbc2ff5640 1 -- 172.21.15.5:0/2541430250 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcbd4031e20 con 0x559dda0f8410 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbe82c8740 10 client.25759 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbe82c8740 1 -- 172.21.15.5:0/2541430250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559dda0fa940 con 0x559dda0f8410 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.125+0000 7fcbc2ff5640 1 -- 172.21.15.5:0/2541430250 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbd4041070 con 0x559dda0f8410 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.126+0000 7fcbc2ff5640 1 -- 172.21.15.5:0/2541430250 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fcbd4004a30 con 0x559dda0f8410 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.126+0000 7fcbe82c8740 10 client.25759 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.126+0000 7fcbe82c8740 10 client.25759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.126+0000 7fcbe82c8740 20 client.25759 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.126+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559dda17d940 0x559dda17fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.126+0000 7fcbe82c8740 4 client.25759 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:06:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.126+0000 7fcbe82c8740 1 -- 172.21.15.5:0/2541430250 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559dda063d70 con 0x559dda17d940 2024-09-19T06:06:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.126+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559dda17d940 0x559dda17fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.126+0000 7fcbc2ff5640 10 client.25759 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.126+0000 7fcbdf7fe640 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559dda17d940 0x559dda17fd60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf985852d73ed2b6 server_cookie=bbbe767cbbb304e7 in_seq=0 out_seq=0 2024-09-19T06:06:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.146+0000 7fcbc2ff5640 1 -- 172.21.15.5:0/2541430250 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559dda063d70 con 0x559dda17d940 2024-09-19T06:06:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.146+0000 7fcbc2ff5640 10 client.25759 handle_command_reply: tid=0 2024-09-19T06:06:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.146+0000 7fcbe82c8740 1 client.25759 shutdown 2024-09-19T06:06:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.146+0000 7fcbe82c8740 1 -- 172.21.15.5:0/2541430250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f8410 msgr2=0x559dda0fc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.146+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f8410 0x559dda0fc620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x559dda0a26e0 tx=0x7fcbd4005480 comp rx=0 tx=0).stop 2024-09-19T06:06:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.146+0000 7fcbe82c8740 1 -- 172.21.15.5:0/2541430250 shutdown_connections 2024-09-19T06:06:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.146+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559dda17d940 0x559dda17fd60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.146+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dda102cb0 0x559dda0fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.146+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559dda0f8410 0x559dda0fc620 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.146+0000 7fcbe82c8740 1 --2- 172.21.15.5:0/2541430250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559dda0f7af0 0x559dda10b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.146+0000 7fcbc2ff5640 0 client.25759 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.146+0000 7fcbe82c8740 1 -- 172.21.15.5:0/2541430250 >> 172.21.15.5:0/2541430250 conn(0x559dda0f4060 msgr2=0x559dda0f29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe82c8740 1 -- 172.21.15.5:0/2541430250 shutdown_connections 2024-09-19T06:06:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe82c8740 1 -- 172.21.15.5:0/2541430250 wait complete. 2024-09-19T06:06:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe82c8740 20 client.25759 trim_cache size 0 max 16384 2024-09-19T06:06:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:06:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:06:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:06:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:06:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe5533640 1 -- 172.21.15.5:0/1065291633 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcbc80a3430 msgr2=0x7fcbc80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe5533640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcbc80a3430 0x7fcbc80a58f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00cd690 tx=0x7fcbd00bc040 comp rx=0 tx=0).stop 2024-09-19T06:06:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe5533640 1 -- 172.21.15.5:0/1065291633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbe01009a0 msgr2=0x7fcbe00fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe5533640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbe01009a0 0x7fcbe00fcde0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000ea60 tx=0x7fcbd000ef20 comp rx=0 tx=0).stop 2024-09-19T06:06:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe5533640 1 -- 172.21.15.5:0/1065291633 shutdown_connections 2024-09-19T06:06:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe5533640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbe01069b0 0x7fcbe00fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe5533640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcbc80a3430 0x7fcbc80a58f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe5533640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe0101340 0x7fcbe00fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe5533640 1 --2- 172.21.15.5:0/1065291633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbe01009a0 0x7fcbe00fcde0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.147+0000 7fcbe5533640 1 -- 172.21.15.5:0/1065291633 >> 172.21.15.5:0/1065291633 conn(0x7fcbe00779a0 msgr2=0x7fcbe0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.148+0000 7fcbe5533640 1 -- 172.21.15.5:0/1065291633 shutdown_connections 2024-09-19T06:06:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:36.148+0000 7fcbe5533640 1 -- 172.21.15.5:0/1065291633 wait complete. 2024-09-19T06:06:36.173 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:06:36.588 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:36 smithi005 ceph-mon[30955]: pgmap v101: 97 pgs: 97 active+clean; 49 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-09-19T06:06:36.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.680+0000 7fa97a329640 1 -- 172.21.15.5:0/3567728766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa97410c160 msgr2=0x7fa97410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.680+0000 7fa97a329640 1 --2- 172.21.15.5:0/3567728766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa97410c160 0x7fa97410e600 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009ca0 tx=0x7fa96c02a950 comp rx=0 tx=0).stop 2024-09-19T06:06:36.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.680+0000 7fa97a329640 1 -- 172.21.15.5:0/3567728766 shutdown_connections 2024-09-19T06:06:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.680+0000 7fa97a329640 1 --2- 172.21.15.5:0/3567728766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa97410c160 0x7fa97410e600 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.680+0000 7fa97a329640 1 --2- 172.21.15.5:0/3567728766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa974103650 0x7fa97410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.680+0000 7fa97a329640 1 --2- 172.21.15.5:0/3567728766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa974102d30 0x7fa974103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.680+0000 7fa97a329640 1 -- 172.21.15.5:0/3567728766 >> 172.21.15.5:0/3567728766 conn(0x7fa9740fc790 msgr2=0x7fa9740febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.680+0000 7fa97a329640 1 -- 172.21.15.5:0/3567728766 shutdown_connections 2024-09-19T06:06:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.680+0000 7fa97a329640 1 -- 172.21.15.5:0/3567728766 wait complete. 2024-09-19T06:06:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa97a329640 1 Processor -- start 2024-09-19T06:06:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa97a329640 1 -- start start 2024-09-19T06:06:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa97a329640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa974102d30 0x7fa9741982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa97a329640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa974103650 0x7fa974198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa97a329640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97410c160 0x7fa97419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa97a329640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa974111df0 con 0x7fa974103650 2024-09-19T06:06:36.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa97a329640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa974111c70 con 0x7fa974102d30 2024-09-19T06:06:36.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa97a329640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa974111f70 con 0x7fa97410c160 2024-09-19T06:06:36.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa9737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa974103650 0x7fa974198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa9737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa974103650 0x7fa974198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52162/0 (socket says 172.21.15.5:52162) 2024-09-19T06:06:36.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa9737fe640 1 -- 172.21.15.5:0/3797560410 learned_addr learned my addr 172.21.15.5:0/3797560410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa9788c6640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97410c160 0x7fa97419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.681+0000 7fa973fff640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa974102d30 0x7fa9741982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.682+0000 7fa9737fe640 1 -- 172.21.15.5:0/3797560410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97410c160 msgr2=0x7fa97419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.682+0000 7fa9737fe640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97410c160 0x7fa97419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.682+0000 7fa9737fe640 1 -- 172.21.15.5:0/3797560410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa974102d30 msgr2=0x7fa9741982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.682+0000 7fa9737fe640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa974102d30 0x7fa9741982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.682+0000 7fa9737fe640 1 -- 172.21.15.5:0/3797560410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97419d2a0 con 0x7fa974103650 2024-09-19T06:06:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.682+0000 7fa9788c6640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97410c160 0x7fa97419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.682+0000 7fa9737fe640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa974103650 0x7fa974198830 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa96800e9f0 tx=0x7fa96800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.682+0000 7fa9717fa640 1 -- 172.21.15.5:0/3797560410 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96800ccc0 con 0x7fa974103650 2024-09-19T06:06:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.682+0000 7fa97a329640 1 -- 172.21.15.5:0/3797560410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97419d590 con 0x7fa974103650 2024-09-19T06:06:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.682+0000 7fa9717fa640 1 -- 172.21.15.5:0/3797560410 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa96800ce60 con 0x7fa974103650 2024-09-19T06:06:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.682+0000 7fa9717fa640 1 -- 172.21.15.5:0/3797560410 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968010430 con 0x7fa974103650 2024-09-19T06:06:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.682+0000 7fa97a329640 1 -- 172.21.15.5:0/3797560410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9741a4e70 con 0x7fa974103650 2024-09-19T06:06:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa97c702740 1 Processor -- start 2024-09-19T06:06:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa9717fa640 1 -- 172.21.15.5:0/3797560410 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa968005020 con 0x7fa974103650 2024-09-19T06:06:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa97c702740 1 -- start start 2024-09-19T06:06:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa97c702740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6beb60 0x56028f6bef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa97c702740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa97c702740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6c9d40 0x56028f6cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa97c702740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56028f624280 con 0x56028f6c9d40 2024-09-19T06:06:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa97c702740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56028f5cbb10 con 0x56028f6bf480 2024-09-19T06:06:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa97c702740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56028f620240 con 0x56028f6beb60 2024-09-19T06:06:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa9788c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6beb60 0x56028f6bef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa9737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6c9d40 0x56028f6cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa9788c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6beb60 0x56028f6bef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36502/0 (socket says 172.21.15.5:36502) 2024-09-19T06:06:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.685+0000 7fa9788c6640 1 -- 172.21.15.5:0/2556434542 learned_addr learned my addr 172.21.15.5:0/2556434542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa973fff640 1 --2- 172.21.15.5:0/2556434542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa973fff640 1 -- 172.21.15.5:0/2556434542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6beb60 msgr2=0x56028f6bef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa973fff640 1 --2- 172.21.15.5:0/2556434542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6beb60 0x56028f6bef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa973fff640 1 -- 172.21.15.5:0/2556434542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6c9d40 msgr2=0x56028f6cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa973fff640 1 --2- 172.21.15.5:0/2556434542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6c9d40 0x56028f6cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa973fff640 1 -- 172.21.15.5:0/2556434542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56028f5d81f0 con 0x56028f6bf480 2024-09-19T06:06:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa973fff640 1 --2- 172.21.15.5:0/2556434542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c9800 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa964005fd0 tx=0x7fa9640107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f57c11b972f984f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa9717fa640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa95c0a3430 0x7fa95c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa95a7fc640 1 -- 172.21.15.5:0/2556434542 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96401c480 con 0x56028f6bf480 2024-09-19T06:06:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa9788c6640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa95c0a3430 0x7fa95c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa95a7fc640 1 -- 172.21.15.5:0/2556434542 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa96401cad0 con 0x56028f6bf480 2024-09-19T06:06:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa95a7fc640 1 -- 172.21.15.5:0/2556434542 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964011b70 con 0x56028f6bf480 2024-09-19T06:06:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.686+0000 7fa9717fa640 1 -- 172.21.15.5:0/3797560410 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa9680d00c0 con 0x7fa974103650 2024-09-19T06:06:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.687+0000 7fa97c702740 1 -- 172.21.15.5:0/2556434542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 msgr2=0x56028f6c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.687+0000 7fa9788c6640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa95c0a3430 0x7fa95c0a58f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa96c02fcc0 tx=0x7fa96c005c50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.687+0000 7fa97c702740 1 --2- 172.21.15.5:0/2556434542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c9800 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa964005fd0 tx=0x7fa9640107a0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.687+0000 7fa97c702740 1 -- 172.21.15.5:0/2556434542 shutdown_connections 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.687+0000 7fa97c702740 1 --2- 172.21.15.5:0/2556434542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6c9d40 0x56028f6cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.687+0000 7fa97c702740 1 --2- 172.21.15.5:0/2556434542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c9800 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.687+0000 7fa97c702740 1 --2- 172.21.15.5:0/2556434542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6beb60 0x56028f6bef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.687+0000 7fa97c702740 1 -- 172.21.15.5:0/2556434542 >> 172.21.15.5:0/2556434542 conn(0x56028f6bb0f0 msgr2=0x56028f6bb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.687+0000 7fa97c702740 1 -- 172.21.15.5:0/2556434542 shutdown_connections 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.687+0000 7fa97c702740 1 -- 172.21.15.5:0/2556434542 wait complete. 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa97c702740 1 -- start start 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa97c702740 1 Processor -- start 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa97c702740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 0x56028f6c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa97c702740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa97c702740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6c9d40 0x56028f6c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa97c702740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56028f6cc910 con 0x56028f6beb60 2024-09-19T06:06:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa97c702740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56028f5def10 con 0x56028f6bf480 2024-09-19T06:06:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa97c702740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56028f6cca90 con 0x56028f6c9d40 2024-09-19T06:06:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa97c702740 10 client.0 _ll_register_callbacks cb 0x56028f5a7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa973fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 0x56028f6c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa973fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 0x56028f6c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52178/0 (socket says 172.21.15.5:52178) 2024-09-19T06:06:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa973fff640 1 -- 172.21.15.5:0/1075566352 learned_addr learned my addr 172.21.15.5:0/1075566352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa9737fe640 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.688+0000 7fa9788c6640 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6c9d40 0x56028f6c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.689+0000 7fa973fff640 1 -- 172.21.15.5:0/1075566352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6c9d40 msgr2=0x56028f6c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.689+0000 7fa973fff640 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6c9d40 0x56028f6c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.689+0000 7fa973fff640 1 -- 172.21.15.5:0/1075566352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 msgr2=0x56028f6c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.689+0000 7fa973fff640 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.689+0000 7fa973fff640 1 -- 172.21.15.5:0/1075566352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56028f3956a0 con 0x56028f6beb60 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.689+0000 7fa9737fe640 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.689+0000 7fa973fff640 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 0x56028f6c6bc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa96400bfb0 tx=0x7fa964005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e609de838bbb5a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.689+0000 7fa9537fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.689+0000 7fa9537fe640 1 -- 172.21.15.5:0/1075566352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964033060 con 0x56028f6beb60 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.689+0000 7fa9537fe640 1 -- 172.21.15.5:0/1075566352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa96401c890 con 0x56028f6beb60 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.689+0000 7fa97c702740 1 -- 172.21.15.5:0/1075566352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa96401cad0 con 0x56028f6beb60 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.690+0000 7fa9537fe640 1 -- 172.21.15.5:0/1075566352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9640118d0 con 0x56028f6beb60 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.690+0000 7fa97c702740 10 client.16254 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.690+0000 7fa97c702740 1 -- 172.21.15.5:0/1075566352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56028f6d84c0 con 0x56028f6beb60 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.690+0000 7fa9537fe640 1 -- 172.21.15.5:0/1075566352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa96401cad0 con 0x56028f6beb60 2024-09-19T06:06:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.690+0000 7fa9537fe640 1 -- 172.21.15.5:0/1075566352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fa96402d030 con 0x56028f6beb60 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.690+0000 7fa97c702740 10 client.16254 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.690+0000 7fa97c702740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.690+0000 7fa97c702740 20 client.16254 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.690+0000 7fa97c702740 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56028f6d9a80 0x56028f6dbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.690+0000 7fa97c702740 4 client.16254 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.690+0000 7fa97c702740 1 -- 172.21.15.5:0/1075566352 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56028f6317f0 con 0x56028f6d9a80 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.690+0000 7fa9737fe640 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56028f6d9a80 0x56028f6dbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.691+0000 7fa9737fe640 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56028f6d9a80 0x56028f6dbf10 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1611b89e7298d19b server_cookie=8024600a27ab2fe0 in_seq=0 out_seq=0 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.691+0000 7fa9537fe640 10 client.16254 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa9537fe640 1 -- 172.21.15.5:0/1075566352 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56028f6317f0 con 0x56028f6d9a80 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa9537fe640 10 client.16254 handle_command_reply: tid=0 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa97c702740 1 client.16254 shutdown 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa97c702740 1 -- 172.21.15.5:0/1075566352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 msgr2=0x56028f6c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa97c702740 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 0x56028f6c6bc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa96400bfb0 tx=0x7fa964005700 comp rx=0 tx=0).stop 2024-09-19T06:06:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa97c702740 1 -- 172.21.15.5:0/1075566352 shutdown_connections 2024-09-19T06:06:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa97c702740 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56028f6d9a80 0x56028f6dbf10 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa97c702740 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6c9d40 0x56028f6c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa97c702740 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa97c702740 1 --2- 172.21.15.5:0/1075566352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 0x56028f6c6bc0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa9537fe640 0 client.16254 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa97c702740 1 -- 172.21.15.5:0/1075566352 >> 172.21.15.5:0/1075566352 conn(0x56028f6bb0f0 msgr2=0x56028f6c0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa97c702740 1 -- 172.21.15.5:0/1075566352 shutdown_connections 2024-09-19T06:06:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.692+0000 7fa97c702740 1 -- 172.21.15.5:0/1075566352 wait complete. 2024-09-19T06:06:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.693+0000 7fa97c702740 20 client.16254 trim_cache size 0 max 16384 2024-09-19T06:06:36.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.713+0000 7fa97c702740 1 Processor -- start 2024-09-19T06:06:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.713+0000 7fa97c702740 1 -- start start 2024-09-19T06:06:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa97c702740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6beb60 0x56028f6c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa97c702740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa97c702740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6c9d40 0x56028f6e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa97c702740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741ab530 con 0x56028f6c9d40 2024-09-19T06:06:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa97c702740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741ad0d0 con 0x56028f6bf480 2024-09-19T06:06:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa97c702740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741abc00 con 0x56028f6beb60 2024-09-19T06:06:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa9737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa9788c6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6c9d40 0x56028f6e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa973fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6beb60 0x56028f6c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa9737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57530/0 (socket says 172.21.15.5:57530) 2024-09-19T06:06:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa9737fe640 1 -- 172.21.15.5:0/406929263 learned_addr learned my addr 172.21.15.5:0/406929263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa9788c6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6c9d40 0x56028f6e15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52190/0 (socket says 172.21.15.5:52190) 2024-09-19T06:06:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa9737fe640 1 -- 172.21.15.5:0/406929263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6beb60 msgr2=0x56028f6c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa9737fe640 1 --2- 172.21.15.5:0/406929263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6beb60 0x56028f6c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa9737fe640 1 -- 172.21.15.5:0/406929263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6c9d40 msgr2=0x56028f6e15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa9737fe640 1 --2- 172.21.15.5:0/406929263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6c9d40 0x56028f6e15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa9737fe640 1 -- 172.21.15.5:0/406929263 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56028f6c9490 con 0x56028f6bf480 2024-09-19T06:06:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa9788c6640 1 --2- 172.21.15.5:0/406929263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6c9d40 0x56028f6e15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:36.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa9737fe640 1 --2- 172.21.15.5:0/406929263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c8a20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa96809c0b0 tx=0x7fa9680d3c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24492a396dc207fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:36.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.714+0000 7fa959ffb640 1 -- 172.21.15.5:0/406929263 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96809bef0 con 0x56028f6bf480 2024-09-19T06:06:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.715+0000 7fa959ffb640 1 -- 172.21.15.5:0/406929263 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa96809cab0 con 0x56028f6bf480 2024-09-19T06:06:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.715+0000 7fa959ffb640 1 -- 172.21.15.5:0/406929263 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96809bd10 con 0x56028f6bf480 2024-09-19T06:06:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.715+0000 7fa97c702740 1 -- 172.21.15.5:0/406929263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 msgr2=0x56028f6c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.715+0000 7fa97c702740 1 --2- 172.21.15.5:0/406929263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c8a20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa96809c0b0 tx=0x7fa9680d3c10 comp rx=0 tx=0).stop 2024-09-19T06:06:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.715+0000 7fa97c702740 1 -- 172.21.15.5:0/406929263 shutdown_connections 2024-09-19T06:06:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.715+0000 7fa97c702740 1 --2- 172.21.15.5:0/406929263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6c9d40 0x56028f6e15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.715+0000 7fa97c702740 1 --2- 172.21.15.5:0/406929263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6c8a20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.715+0000 7fa97c702740 1 --2- 172.21.15.5:0/406929263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6beb60 0x56028f6c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.715+0000 7fa97c702740 1 -- 172.21.15.5:0/406929263 >> 172.21.15.5:0/406929263 conn(0x56028f6bb0f0 msgr2=0x56028f6c8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.715+0000 7fa97c702740 1 -- 172.21.15.5:0/406929263 shutdown_connections 2024-09-19T06:06:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.715+0000 7fa97c702740 1 -- 172.21.15.5:0/406929263 wait complete. 2024-09-19T06:06:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.716+0000 7fa97c702740 1 -- start start 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.716+0000 7fa97c702740 1 Processor -- start 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa97c702740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 0x56028f6d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa97c702740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa97c702740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6c9d40 0x56028f6cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa97c702740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741aca00 con 0x56028f6beb60 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa97c702740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741ac2d0 con 0x56028f6bf480 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa97c702740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741a9510 con 0x56028f6c9d40 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa973fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 0x56028f6d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa973fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 0x56028f6d20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52206/0 (socket says 172.21.15.5:52206) 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa9737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa973fff640 1 -- 172.21.15.5:0/2269532078 learned_addr learned my addr 172.21.15.5:0/2269532078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa97c702740 10 client.0 _ll_register_callbacks cb 0x7fa9680d3ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa9788c6640 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6c9d40 0x56028f6cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa9737fe640 1 -- 172.21.15.5:0/2269532078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6c9d40 msgr2=0x56028f6cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa9737fe640 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6c9d40 0x56028f6cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa9737fe640 1 -- 172.21.15.5:0/2269532078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 msgr2=0x56028f6d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa9737fe640 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 0x56028f6d20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa9737fe640 1 -- 172.21.15.5:0/2269532078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56028f6cdee0 con 0x56028f6bf480 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa973fff640 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 0x56028f6d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa9737fe640 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6cd460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa96809c340 tx=0x7fa96800c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46c53ffb64c4c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa9537fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa9537fe640 1 -- 172.21.15.5:0/2269532078 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9680a5020 con 0x56028f6bf480 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa9537fe640 1 -- 172.21.15.5:0/2269532078 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa968098a20 con 0x56028f6bf480 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.717+0000 7fa97c702740 1 -- 172.21.15.5:0/2269532078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa96809cab0 con 0x56028f6bf480 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.718+0000 7fa9537fe640 1 -- 172.21.15.5:0/2269532078 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa96809c3b0 con 0x56028f6bf480 2024-09-19T06:06:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.718+0000 7fa97c702740 10 client.25856 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.718+0000 7fa97c702740 1 -- 172.21.15.5:0/2269532078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56028f744280 con 0x56028f6bf480 2024-09-19T06:06:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.718+0000 7fa9537fe640 1 -- 172.21.15.5:0/2269532078 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa96809cab0 con 0x56028f6bf480 2024-09-19T06:06:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.718+0000 7fa9537fe640 1 -- 172.21.15.5:0/2269532078 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fa9680d2070 con 0x56028f6bf480 2024-09-19T06:06:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.718+0000 7fa97c702740 10 client.25856 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.718+0000 7fa97c702740 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.718+0000 7fa97c702740 20 client.25856 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.718+0000 7fa97c702740 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56028f744a10 0x56028f746e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.718+0000 7fa97c702740 4 client.25856 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:06:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.718+0000 7fa97c702740 1 -- 172.21.15.5:0/2269532078 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56028f62bd00 con 0x56028f744a10 2024-09-19T06:06:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.718+0000 7fa973fff640 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56028f744a10 0x56028f746e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.719+0000 7fa973fff640 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56028f744a10 0x56028f746e30 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22bf120bd177c0c3 server_cookie=69d82684b432a49d in_seq=0 out_seq=0 2024-09-19T06:06:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.719+0000 7fa9537fe640 10 client.25856 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.720+0000 7fa9537fe640 1 -- 172.21.15.5:0/2269532078 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56028f62bd00 con 0x56028f744a10 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.720+0000 7fa9537fe640 10 client.25856 handle_command_reply: tid=0 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.720+0000 7fa97c702740 1 client.25856 shutdown 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.720+0000 7fa97c702740 1 -- 172.21.15.5:0/2269532078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 msgr2=0x56028f6cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.720+0000 7fa97c702740 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6cd460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa96809c340 tx=0x7fa96800c7d0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.720+0000 7fa97c702740 1 -- 172.21.15.5:0/2269532078 shutdown_connections 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.720+0000 7fa97c702740 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56028f744a10 0x56028f746e30 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.720+0000 7fa9537fe640 0 client.25856 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.720+0000 7fa97c702740 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028f6c9d40 0x56028f6cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.720+0000 7fa97c702740 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56028f6bf480 0x56028f6cd460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.720+0000 7fa97c702740 1 --2- 172.21.15.5:0/2269532078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56028f6beb60 0x56028f6d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.720+0000 7fa97c702740 1 -- 172.21.15.5:0/2269532078 >> 172.21.15.5:0/2269532078 conn(0x56028f6bb0f0 msgr2=0x56028f6c0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97c702740 1 -- 172.21.15.5:0/2269532078 shutdown_connections 2024-09-19T06:06:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97c702740 1 -- 172.21.15.5:0/2269532078 wait complete. 2024-09-19T06:06:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97c702740 20 client.25856 trim_cache size 0 max 16384 2024-09-19T06:06:36.727 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:06:36.727 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:06:36.727 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "acbe2a6e-39bd-4b38-84b2-147c7c36df77", 2024-09-19T06:06:36.727 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:06:36.727 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:06:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97a329640 1 -- 172.21.15.5:0/3797560410 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa95c0a3430 msgr2=0x7fa95c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97a329640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa95c0a3430 0x7fa95c0a58f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa96c02fcc0 tx=0x7fa96c005c50 comp rx=0 tx=0).stop 2024-09-19T06:06:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97a329640 1 -- 172.21.15.5:0/3797560410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa974103650 msgr2=0x7fa974198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97a329640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa974103650 0x7fa974198830 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa96800e9f0 tx=0x7fa96800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97a329640 1 -- 172.21.15.5:0/3797560410 shutdown_connections 2024-09-19T06:06:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97a329640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa95c0a3430 0x7fa95c0a58f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97a329640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97410c160 0x7fa97419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97a329640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa974103650 0x7fa974198830 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97a329640 1 --2- 172.21.15.5:0/3797560410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa974102d30 0x7fa9741982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97a329640 1 -- 172.21.15.5:0/3797560410 >> 172.21.15.5:0/3797560410 conn(0x7fa9740fc790 msgr2=0x7fa9740fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.721+0000 7fa97a329640 1 -- 172.21.15.5:0/3797560410 shutdown_connections 2024-09-19T06:06:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:36.722+0000 7fa97a329640 1 -- 172.21.15.5:0/3797560410 wait complete. 2024-09-19T06:06:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:36 smithi157 ceph-mon[35346]: pgmap v101: 97 pgs: 97 active+clean; 49 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-09-19T06:06:36.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 63955 5.26 MB/sec execute 2 sec latency 25.335 ms 2024-09-19T06:06:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:36 smithi049 ceph-mon[34640]: pgmap v101: 97 pgs: 97 active+clean; 49 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-09-19T06:06:37.532 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:06:37.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:37 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:06:37.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:06:37.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:06:37.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:06:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:06:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:06:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:37 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:06:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:06:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:06:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:06:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:06:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:06:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.806 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:06:37.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 64271 3.99 MB/sec execute 3 sec latency 58.638 ms 2024-09-19T06:06:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:37 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:06:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:06:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:06:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:06:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:06:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:06:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:38.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.299+0000 7f8311991640 1 -- 172.21.15.5:0/44197000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c102790 msgr2=0x7f830c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:38.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.299+0000 7f8311991640 1 --2- 172.21.15.5:0/44197000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c102790 0x7f830c1067e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8300009900 tx=0x7f830002a410 comp rx=0 tx=0).stop 2024-09-19T06:06:38.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.299+0000 7f8311991640 1 -- 172.21.15.5:0/44197000 shutdown_connections 2024-09-19T06:06:38.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.299+0000 7f8311991640 1 --2- 172.21.15.5:0/44197000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f830c106f10 0x7f830c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.299+0000 7f8311991640 1 --2- 172.21.15.5:0/44197000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c102790 0x7f830c1067e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.299+0000 7f8311991640 1 --2- 172.21.15.5:0/44197000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f830c101de0 0x7f830c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.299+0000 7f8311991640 1 -- 172.21.15.5:0/44197000 >> 172.21.15.5:0/44197000 conn(0x7f830c0fd8e0 msgr2=0x7f830c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:38.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.299+0000 7f8311991640 1 -- 172.21.15.5:0/44197000 shutdown_connections 2024-09-19T06:06:38.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.299+0000 7f8311991640 1 -- 172.21.15.5:0/44197000 wait complete. 2024-09-19T06:06:38.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.300+0000 7f8311991640 1 Processor -- start 2024-09-19T06:06:38.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.300+0000 7f8311991640 1 -- start start 2024-09-19T06:06:38.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f8311991640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c101de0 0x7f830c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f8311991640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f830c102790 0x7f830c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f8311991640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f830c106f10 0x7f830c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c101de0 0x7f830c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f8311991640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f830c111e20 con 0x7f830c101de0 2024-09-19T06:06:38.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f8311991640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f830c111ca0 con 0x7f830c102790 2024-09-19T06:06:38.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f8311991640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f830c111fa0 con 0x7f830c106f10 2024-09-19T06:06:38.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c101de0 0x7f830c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52238/0 (socket says 172.21.15.5:52238) 2024-09-19T06:06:38.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830affd640 1 -- 172.21.15.5:0/3277857126 learned_addr learned my addr 172.21.15.5:0/3277857126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:38.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830b7fe640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f830c106f10 0x7f830c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830a7fc640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f830c102790 0x7f830c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830affd640 1 -- 172.21.15.5:0/3277857126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f830c106f10 msgr2=0x7f830c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:38.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830affd640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f830c106f10 0x7f830c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830affd640 1 -- 172.21.15.5:0/3277857126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f830c102790 msgr2=0x7f830c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:38.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830affd640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f830c102790 0x7f830c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830affd640 1 -- 172.21.15.5:0/3277857126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c1a68e0 con 0x7f830c101de0 2024-09-19T06:06:38.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830b7fe640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f830c106f10 0x7f830c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:06:38.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830a7fc640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f830c102790 0x7f830c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:06:38.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.301+0000 7f830affd640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c101de0 0x7f830c0744f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00e9f0 tx=0x7f82fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.302+0000 7f831098f640 1 -- 172.21.15.5:0/3277857126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00cd80 con 0x7f830c101de0 2024-09-19T06:06:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.302+0000 7f8311991640 1 -- 172.21.15.5:0/3277857126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c1a6bd0 con 0x7f830c101de0 2024-09-19T06:06:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.302+0000 7f831098f640 1 -- 172.21.15.5:0/3277857126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82fc0108e0 con 0x7f830c101de0 2024-09-19T06:06:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.302+0000 7f831098f640 1 -- 172.21.15.5:0/3277857126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc018920 con 0x7f830c101de0 2024-09-19T06:06:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.302+0000 7f8311991640 1 -- 172.21.15.5:0/3277857126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c1a7160 con 0x7f830c101de0 2024-09-19T06:06:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.305+0000 7f831098f640 1 -- 172.21.15.5:0/3277857126 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f82fc010a90 con 0x7f830c101de0 2024-09-19T06:06:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.305+0000 7f8313d6a740 1 Processor -- start 2024-09-19T06:06:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.305+0000 7f8313d6a740 1 -- start start 2024-09-19T06:06:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f8313d6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5d4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f8313d6a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 0x5573cb5df140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f8313d6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5df680 0x5573cb5e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f8313d6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573cb53a2e0 con 0x5573cb5df680 2024-09-19T06:06:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f8313d6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5573cb4e1ce0 con 0x5573cb5d44a0 2024-09-19T06:06:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f8313d6a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5573cb5362a0 con 0x5573cb5d4dc0 2024-09-19T06:06:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f830a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5df680 0x5573cb5e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5d4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57550/0 (socket says 172.21.15.5:57550) 2024-09-19T06:06:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f830b7fe640 1 -- 172.21.15.5:0/574305643 learned_addr learned my addr 172.21.15.5:0/574305643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f831098f640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f82f40a3430 0x7f82f40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f830a7fc640 1 -- 172.21.15.5:0/574305643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 msgr2=0x5573cb5df140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:38.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f830a7fc640 1 --2- 172.21.15.5:0/574305643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 0x5573cb5df140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f830a7fc640 1 -- 172.21.15.5:0/574305643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 msgr2=0x5573cb5d4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:38.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f830a7fc640 1 --2- 172.21.15.5:0/574305643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5d4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f830a7fc640 1 -- 172.21.15.5:0/574305643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573cb4c6440 con 0x5573cb5df680 2024-09-19T06:06:38.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f830b7fe640 1 --2- 172.21.15.5:0/574305643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5d4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:38.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f831098f640 1 -- 172.21.15.5:0/3277857126 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f82fc0d00a0 con 0x7f830c101de0 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f830b7fe640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f82f40a3430 0x7f82f40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.306+0000 7f830a7fc640 1 --2- 172.21.15.5:0/574305643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5df680 0x5573cb5e1a70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8300000c00 tx=0x7f8300032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c37397fca643221 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.307+0000 7f830b7fe640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f82f40a3430 0x7f82f40a58f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83040126b0 tx=0x7f8304004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.307+0000 7f82f1ffb640 1 -- 172.21.15.5:0/574305643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830003f070 con 0x5573cb5df680 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.307+0000 7f82f1ffb640 1 -- 172.21.15.5:0/574305643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8300004030 con 0x5573cb5df680 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.307+0000 7f82f1ffb640 1 -- 172.21.15.5:0/574305643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300035070 con 0x5573cb5df680 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.308+0000 7f8313d6a740 1 -- 172.21.15.5:0/574305643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5df680 msgr2=0x5573cb5e1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.308+0000 7f8313d6a740 1 --2- 172.21.15.5:0/574305643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5df680 0x5573cb5e1a70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8300000c00 tx=0x7f8300032c10 comp rx=0 tx=0).stop 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.308+0000 7f8313d6a740 1 -- 172.21.15.5:0/574305643 shutdown_connections 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.308+0000 7f8313d6a740 1 --2- 172.21.15.5:0/574305643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5df680 0x5573cb5e1a70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.308+0000 7f8313d6a740 1 --2- 172.21.15.5:0/574305643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 0x5573cb5df140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.308+0000 7f8313d6a740 1 --2- 172.21.15.5:0/574305643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5d4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.308+0000 7f8313d6a740 1 -- 172.21.15.5:0/574305643 >> 172.21.15.5:0/574305643 conn(0x5573cb5d0a30 msgr2=0x5573cb5d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.308+0000 7f8313d6a740 1 -- 172.21.15.5:0/574305643 shutdown_connections 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.308+0000 7f8313d6a740 1 -- 172.21.15.5:0/574305643 wait complete. 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.308+0000 7f8313d6a740 1 -- start start 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.308+0000 7f8313d6a740 1 Processor -- start 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f8313d6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5df050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f8313d6a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 0x5573cb5dc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f8313d6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5df680 0x5573cb5dc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f8313d6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573cb5e2250 con 0x5573cb5df680 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f8313d6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5573cb4f5050 con 0x5573cb5d44a0 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f8313d6a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5573cb5e23d0 con 0x5573cb5d4dc0 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f8313d6a740 10 client.0 _ll_register_callbacks cb 0x5573cb535160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f830affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5df050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f830affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5df050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57566/0 (socket says 172.21.15.5:57566) 2024-09-19T06:06:38.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f830affd640 1 -- 172.21.15.5:0/1406013543 learned_addr learned my addr 172.21.15.5:0/1406013543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:38.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f830b7fe640 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5df680 0x5573cb5dc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f830a7fc640 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 0x5573cb5dc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f830b7fe640 1 -- 172.21.15.5:0/1406013543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 msgr2=0x5573cb5dc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:38.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f830b7fe640 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 0x5573cb5dc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f830b7fe640 1 -- 172.21.15.5:0/1406013543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 msgr2=0x5573cb5df050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:38.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f830b7fe640 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5df050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f830b7fe640 1 -- 172.21.15.5:0/1406013543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573cb2ab3c0 con 0x5573cb5df680 2024-09-19T06:06:38.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.309+0000 7f830affd640 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5df050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.310+0000 7f830b7fe640 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5df680 0x5573cb5dc8f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8304012f40 tx=0x7f8304004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac94c30802d03687 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.310+0000 7f82eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.310+0000 7f82eaffd640 1 -- 172.21.15.5:0/1406013543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83040119c0 con 0x5573cb5df680 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.310+0000 7f82eaffd640 1 -- 172.21.15.5:0/1406013543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f8304011b60 con 0x5573cb5df680 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.310+0000 7f8313d6a740 1 -- 172.21.15.5:0/1406013543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8300004030 con 0x5573cb5df680 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.310+0000 7f82eaffd640 1 -- 172.21.15.5:0/1406013543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8304011d60 con 0x5573cb5df680 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.310+0000 7f8313d6a740 10 client.16272 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.310+0000 7f8313d6a740 1 -- 172.21.15.5:0/1406013543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573cb5edce0 con 0x5573cb5df680 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.310+0000 7f82eaffd640 1 -- 172.21.15.5:0/1406013543 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8300004030 con 0x5573cb5df680 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.311+0000 7f82eaffd640 1 -- 172.21.15.5:0/1406013543 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f83040274b0 con 0x5573cb5df680 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.311+0000 7f8313d6a740 10 client.16272 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.311+0000 7f8313d6a740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:38.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.311+0000 7f8313d6a740 20 client.16272 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.311+0000 7f8313d6a740 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5573cb5ef1a0 0x5573cb5f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.311+0000 7f8313d6a740 4 client.16272 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.311+0000 7f8313d6a740 1 -- 172.21.15.5:0/1406013543 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573cb547850 con 0x5573cb5ef1a0 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.311+0000 7f830affd640 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5573cb5ef1a0 0x5573cb5f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.311+0000 7f830affd640 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5573cb5ef1a0 0x5573cb5f15c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f2e88253346ddc server_cookie=6c965b46542d2d82 in_seq=0 out_seq=0 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.311+0000 7f82eaffd640 10 client.16272 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.312+0000 7f82eaffd640 1 -- 172.21.15.5:0/1406013543 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573cb547850 con 0x5573cb5ef1a0 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.312+0000 7f82eaffd640 10 client.16272 handle_command_reply: tid=0 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.312+0000 7f8313d6a740 1 client.16272 shutdown 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.313+0000 7f8313d6a740 1 -- 172.21.15.5:0/1406013543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5df680 msgr2=0x5573cb5dc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.313+0000 7f8313d6a740 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5df680 0x5573cb5dc8f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8304012f40 tx=0x7f8304004ab0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.313+0000 7f8313d6a740 1 -- 172.21.15.5:0/1406013543 shutdown_connections 2024-09-19T06:06:38.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.313+0000 7f8313d6a740 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5573cb5ef1a0 0x5573cb5f15c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.313+0000 7f8313d6a740 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5df680 0x5573cb5dc8f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.313+0000 7f8313d6a740 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 0x5573cb5dc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.313+0000 7f82eaffd640 0 client.16272 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:38.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.313+0000 7f8313d6a740 1 --2- 172.21.15.5:0/1406013543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5df050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.313+0000 7f8313d6a740 1 -- 172.21.15.5:0/1406013543 >> 172.21.15.5:0/1406013543 conn(0x5573cb5d0a30 msgr2=0x5573cb5d6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:38.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.313+0000 7f8313d6a740 1 -- 172.21.15.5:0/1406013543 shutdown_connections 2024-09-19T06:06:38.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.313+0000 7f8313d6a740 1 -- 172.21.15.5:0/1406013543 wait complete. 2024-09-19T06:06:38.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.313+0000 7f8313d6a740 20 client.16272 trim_cache size 0 max 16384 2024-09-19T06:06:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.333+0000 7f8313d6a740 1 Processor -- start 2024-09-19T06:06:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.333+0000 7f8313d6a740 1 -- start start 2024-09-19T06:06:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f8313d6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f8313d6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5d4dc0 0x5573cb5f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f8313d6a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5df680 0x5573cb5f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f8313d6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1afa50 con 0x5573cb5d4dc0 2024-09-19T06:06:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f8313d6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1b15f0 con 0x5573cb5d44a0 2024-09-19T06:06:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f8313d6a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1b0120 con 0x5573cb5df680 2024-09-19T06:06:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5d4dc0 0x5573cb5f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5df680 0x5573cb5f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5d4dc0 0x5573cb5f6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52278/0 (socket says 172.21.15.5:52278) 2024-09-19T06:06:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5eb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57570/0 (socket says 172.21.15.5:57570) 2024-09-19T06:06:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830a7fc640 1 -- 172.21.15.5:0/3560279997 learned_addr learned my addr 172.21.15.5:0/3560279997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:38.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830b7fe640 1 -- 172.21.15.5:0/3560279997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 msgr2=0x5573cb5eb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830b7fe640 1 --2- 172.21.15.5:0/3560279997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5eb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830b7fe640 1 -- 172.21.15.5:0/3560279997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5d4dc0 msgr2=0x5573cb5f6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830b7fe640 1 --2- 172.21.15.5:0/3560279997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5d4dc0 0x5573cb5f6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830b7fe640 1 -- 172.21.15.5:0/3560279997 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573cb5df3c0 con 0x5573cb5df680 2024-09-19T06:06:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830affd640 1 --2- 172.21.15.5:0/3560279997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.334+0000 7f830b7fe640 1 --2- 172.21.15.5:0/3560279997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5df680 0x5573cb5f73b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8304027970 tx=0x7f8304005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a6261fa5895db67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.335+0000 7f82f17fa640 1 -- 172.21.15.5:0/3560279997 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304027b60 con 0x5573cb5df680 2024-09-19T06:06:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.335+0000 7f82f17fa640 1 -- 172.21.15.5:0/3560279997 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8304022030 con 0x5573cb5df680 2024-09-19T06:06:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.335+0000 7f82f17fa640 1 -- 172.21.15.5:0/3560279997 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830401e680 con 0x5573cb5df680 2024-09-19T06:06:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.335+0000 7f8313d6a740 1 -- 172.21.15.5:0/3560279997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5df680 msgr2=0x5573cb5f73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.335+0000 7f8313d6a740 1 --2- 172.21.15.5:0/3560279997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5df680 0x5573cb5f73b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8304027970 tx=0x7f8304005f70 comp rx=0 tx=0).stop 2024-09-19T06:06:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.335+0000 7f8313d6a740 1 -- 172.21.15.5:0/3560279997 shutdown_connections 2024-09-19T06:06:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.335+0000 7f8313d6a740 1 --2- 172.21.15.5:0/3560279997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5df680 0x5573cb5f73b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.335+0000 7f8313d6a740 1 --2- 172.21.15.5:0/3560279997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5d4dc0 0x5573cb5f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.335+0000 7f8313d6a740 1 --2- 172.21.15.5:0/3560279997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5d44a0 0x5573cb5eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.335+0000 7f8313d6a740 1 -- 172.21.15.5:0/3560279997 >> 172.21.15.5:0/3560279997 conn(0x5573cb5d0a30 msgr2=0x5573cb5db3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.336+0000 7f8313d6a740 1 -- 172.21.15.5:0/3560279997 shutdown_connections 2024-09-19T06:06:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.336+0000 7f8313d6a740 1 -- 172.21.15.5:0/3560279997 wait complete. 2024-09-19T06:06:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.336+0000 7f8313d6a740 1 -- start start 2024-09-19T06:06:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.336+0000 7f8313d6a740 1 Processor -- start 2024-09-19T06:06:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.336+0000 7f8313d6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5d44a0 0x5573cb5e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.336+0000 7f8313d6a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 0x5573cb5e2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.336+0000 7f8313d6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5df680 0x5573cb5e3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.336+0000 7f8313d6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1b0f20 con 0x5573cb5d44a0 2024-09-19T06:06:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f8313d6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1b07f0 con 0x5573cb5df680 2024-09-19T06:06:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f8313d6a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1ada30 con 0x5573cb5d4dc0 2024-09-19T06:06:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f8313d6a740 10 client.0 _ll_register_callbacks cb 0x7f8304009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f830affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5d44a0 0x5573cb5e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f830a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 0x5573cb5e2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5df680 0x5573cb5e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f830a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 0x5573cb5e2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36582/0 (socket says 172.21.15.5:36582) 2024-09-19T06:06:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f830affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5d44a0 0x5573cb5e7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52284/0 (socket says 172.21.15.5:52284) 2024-09-19T06:06:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f830a7fc640 1 -- 172.21.15.5:0/4088029554 learned_addr learned my addr 172.21.15.5:0/4088029554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f830b7fe640 1 -- 172.21.15.5:0/4088029554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 msgr2=0x5573cb5e2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f830b7fe640 1 --2- 172.21.15.5:0/4088029554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 0x5573cb5e2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f830b7fe640 1 -- 172.21.15.5:0/4088029554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5d44a0 msgr2=0x5573cb5e7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f830b7fe640 1 --2- 172.21.15.5:0/4088029554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5d44a0 0x5573cb5e7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f830b7fe640 1 -- 172.21.15.5:0/4088029554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573cb5e3ac0 con 0x5573cb5df680 2024-09-19T06:06:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f830b7fe640 1 --2- 172.21.15.5:0/4088029554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5df680 0x5573cb5e3450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8304009b70 tx=0x7f8304027ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=968fc8ff996d3e56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f82eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:06:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f82eaffd640 1 -- 172.21.15.5:0/4088029554 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304027420 con 0x5573cb5df680 2024-09-19T06:06:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.337+0000 7f82eaffd640 1 -- 172.21.15.5:0/4088029554 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f83040119d0 con 0x5573cb5df680 2024-09-19T06:06:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f8313d6a740 1 -- 172.21.15.5:0/4088029554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8304022030 con 0x5573cb5df680 2024-09-19T06:06:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f82eaffd640 1 -- 172.21.15.5:0/4088029554 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8304004ab0 con 0x5573cb5df680 2024-09-19T06:06:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f8313d6a740 10 client.25871 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f8313d6a740 1 -- 172.21.15.5:0/4088029554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573cb659cc0 con 0x5573cb5df680 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f82eaffd640 1 -- 172.21.15.5:0/4088029554 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8304022030 con 0x5573cb5df680 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f82eaffd640 1 -- 172.21.15.5:0/4088029554 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f83040056d0 con 0x5573cb5df680 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f8313d6a740 10 client.25871 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f8313d6a740 10 client.25871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f8313d6a740 20 client.25871 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f8313d6a740 1 --2- 172.21.15.5:0/4088029554 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5573cb5ef1a0 0x5573cb65c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f8313d6a740 4 client.25871 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f8313d6a740 1 -- 172.21.15.5:0/4088029554 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573cb541d60 con 0x5573cb5ef1a0 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.338+0000 7f830affd640 1 --2- 172.21.15.5:0/4088029554 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5573cb5ef1a0 0x5573cb65c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.339+0000 7f830affd640 1 --2- 172.21.15.5:0/4088029554 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5573cb5ef1a0 0x5573cb65c3f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0ca71d892ca6acc server_cookie=b66e97c7d3a60022 in_seq=0 out_seq=0 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.339+0000 7f82eaffd640 10 client.25871 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.339+0000 7f82eaffd640 1 -- 172.21.15.5:0/4088029554 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5573cb541d60 con 0x5573cb5ef1a0 2024-09-19T06:06:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.339+0000 7f82eaffd640 10 client.25871 handle_command_reply: tid=0 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.339+0000 7f8313d6a740 1 client.25871 shutdown 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8313d6a740 1 -- 172.21.15.5:0/4088029554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5df680 msgr2=0x5573cb5e3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8313d6a740 1 --2- 172.21.15.5:0/4088029554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5df680 0x5573cb5e3450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8304009b70 tx=0x7f8304027ec0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8313d6a740 1 -- 172.21.15.5:0/4088029554 shutdown_connections 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8313d6a740 1 --2- 172.21.15.5:0/4088029554 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5573cb5ef1a0 0x5573cb65c3f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8313d6a740 1 --2- 172.21.15.5:0/4088029554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573cb5df680 0x5573cb5e3450 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8313d6a740 1 --2- 172.21.15.5:0/4088029554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573cb5d4dc0 0x5573cb5e2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8313d6a740 1 --2- 172.21.15.5:0/4088029554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573cb5d44a0 0x5573cb5e7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f82eaffd640 0 client.25871 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8313d6a740 1 -- 172.21.15.5:0/4088029554 >> 172.21.15.5:0/4088029554 conn(0x5573cb5d0a30 msgr2=0x5573cb5ddb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8313d6a740 1 -- 172.21.15.5:0/4088029554 shutdown_connections 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8313d6a740 1 -- 172.21.15.5:0/4088029554 wait complete. 2024-09-19T06:06:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8313d6a740 20 client.25871 trim_cache size 0 max 16384 2024-09-19T06:06:38.348 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:06:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:06:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:06:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "acbe2a6e-39bd-4b38-84b2-147c7c36df77": { 2024-09-19T06:06:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:06:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "acbe2a6e-39bd-4b38-84b2-147c7c36df77", 2024-09-19T06:06:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:06:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:06:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:06:38.349 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:06:38.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8311991640 1 -- 172.21.15.5:0/3277857126 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f82f40a3430 msgr2=0x7f82f40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:38.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8311991640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f82f40a3430 0x7f82f40a58f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83040126b0 tx=0x7f8304004a40 comp rx=0 tx=0).stop 2024-09-19T06:06:38.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8311991640 1 -- 172.21.15.5:0/3277857126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c101de0 msgr2=0x7f830c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:38.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8311991640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c101de0 0x7f830c0744f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00e9f0 tx=0x7f82fc00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8311991640 1 -- 172.21.15.5:0/3277857126 shutdown_connections 2024-09-19T06:06:38.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8311991640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f82f40a3430 0x7f82f40a58f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8311991640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f830c106f10 0x7f830c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8311991640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f830c102790 0x7f830c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8311991640 1 --2- 172.21.15.5:0/3277857126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c101de0 0x7f830c0744f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:38.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.340+0000 7f8311991640 1 -- 172.21.15.5:0/3277857126 >> 172.21.15.5:0/3277857126 conn(0x7f830c0fd8e0 msgr2=0x7f830c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:38.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.341+0000 7f8311991640 1 -- 172.21.15.5:0/3277857126 shutdown_connections 2024-09-19T06:06:38.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:06:38.341+0000 7f8311991640 1 -- 172.21.15.5:0/3277857126 wait complete. 2024-09-19T06:06:38.770 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:38 smithi005 ceph-mon[30955]: pgmap v102: 97 pgs: 97 active+clean; 50 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-09-19T06:06:38.770 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:38 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:06:38.770 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:38 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:38.770 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:38 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:06:38.770 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:38 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:38.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:38 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:06:38.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:38 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:38 smithi157 ceph-mon[35346]: pgmap v102: 97 pgs: 97 active+clean; 50 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-09-19T06:06:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:38 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:06:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:38 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:38 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:06:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:38 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:38 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:06:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:38 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:38.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 64609 3.08 MB/sec execute 4 sec latency 50.227 ms 2024-09-19T06:06:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:38 smithi049 ceph-mon[34640]: pgmap v102: 97 pgs: 97 active+clean; 50 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-09-19T06:06:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:38 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:06:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:38 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:38 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:06:38.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:38 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:38.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:38 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:06:38.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:38 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:06:39.277 INFO:tasks.cephfs.filesystem:scrub status for tag:acbe2a6e-39bd-4b38-84b2-147c7c36df77 - {'path': '/', 'tag': 'acbe2a6e-39bd-4b38-84b2-147c7c36df77', 'options': 'recursive,force'} 2024-09-19T06:06:39.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:39 smithi005 ceph-mon[30955]: pgmap v103: 97 pgs: 97 active+clean; 50 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T06:06:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:39 smithi157 ceph-mon[35346]: pgmap v103: 97 pgs: 97 active+clean; 50 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T06:06:39.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 65150 2.83 MB/sec execute 5 sec latency 15.554 ms 2024-09-19T06:06:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:39 smithi049 ceph-mon[34640]: pgmap v103: 97 pgs: 97 active+clean; 50 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T06:06:40.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 65789 2.91 MB/sec execute 6 sec latency 15.433 ms 2024-09-19T06:06:41.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 66372 2.82 MB/sec execute 7 sec latency 14.507 ms 2024-09-19T06:06:42.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:41 smithi157 ceph-mon[35346]: pgmap v104: 97 pgs: 97 active+clean; 50 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T06:06:42.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:41 smithi005 ceph-mon[30955]: pgmap v104: 97 pgs: 97 active+clean; 50 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T06:06:42.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:41 smithi049 ceph-mon[34640]: pgmap v104: 97 pgs: 97 active+clean; 50 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T06:06:42.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 67161 3.29 MB/sec execute 8 sec latency 10.691 ms 2024-09-19T06:06:43.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 67634 3.42 MB/sec execute 9 sec latency 14.930 ms 2024-09-19T06:06:44.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:43 smithi157 ceph-mon[35346]: pgmap v105: 97 pgs: 97 active+clean; 50 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-19T06:06:44.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:43 smithi005 ceph-mon[30955]: pgmap v105: 97 pgs: 97 active+clean; 50 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-19T06:06:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:43 smithi049 ceph-mon[34640]: pgmap v105: 97 pgs: 97 active+clean; 50 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-19T06:06:44.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 67883 3.11 MB/sec execute 10 sec latency 15.113 ms 2024-09-19T06:06:45.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:06:45.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:06:45.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:06:45.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:06:45.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:06:45.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:06:45.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:06:45.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:06:45.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:06:45.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 68293 2.88 MB/sec execute 11 sec latency 17.908 ms 2024-09-19T06:06:46.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:45 smithi157 ceph-mon[35346]: pgmap v106: 97 pgs: 97 active+clean; 50 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-09-19T06:06:46.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:45 smithi005 ceph-mon[30955]: pgmap v106: 97 pgs: 97 active+clean; 50 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-09-19T06:06:46.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:45 smithi049 ceph-mon[34640]: pgmap v106: 97 pgs: 97 active+clean; 50 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-09-19T06:06:46.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 68832 3.01 MB/sec execute 12 sec latency 16.583 ms 2024-09-19T06:06:47.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 69466 2.83 MB/sec execute 13 sec latency 14.828 ms 2024-09-19T06:06:48.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:47 smithi157 ceph-mon[35346]: pgmap v107: 97 pgs: 97 active+clean; 55 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-09-19T06:06:48.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:47 smithi005 ceph-mon[30955]: pgmap v107: 97 pgs: 97 active+clean; 55 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-09-19T06:06:48.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:47 smithi049 ceph-mon[34640]: pgmap v107: 97 pgs: 97 active+clean; 55 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-09-19T06:06:48.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 70015 2.80 MB/sec execute 14 sec latency 11.473 ms 2024-09-19T06:06:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.242+0000 7f92bbe8b640 1 -- 172.21.15.5:0/2149340597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b41069e0 msgr2=0x7f92b4106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.242+0000 7f92bbe8b640 1 --2- 172.21.15.5:0/2149340597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b41069e0 0x7f92b4106dc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009930 tx=0x7f92b002a420 comp rx=0 tx=0).stop 2024-09-19T06:06:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.242+0000 7f92bbe8b640 1 -- 172.21.15.5:0/2149340597 shutdown_connections 2024-09-19T06:06:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.242+0000 7f92bbe8b640 1 --2- 172.21.15.5:0/2149340597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b4101370 0x7f92b410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.242+0000 7f92bbe8b640 1 --2- 172.21.15.5:0/2149340597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b41009d0 0x7f92b4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.242+0000 7f92bbe8b640 1 --2- 172.21.15.5:0/2149340597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b41069e0 0x7f92b4106dc0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.242+0000 7f92bbe8b640 1 -- 172.21.15.5:0/2149340597 >> 172.21.15.5:0/2149340597 conn(0x7f92b40fc6b0 msgr2=0x7f92b40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.242+0000 7f92bbe8b640 1 -- 172.21.15.5:0/2149340597 shutdown_connections 2024-09-19T06:06:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.242+0000 7f92bbe8b640 1 -- 172.21.15.5:0/2149340597 wait complete. 2024-09-19T06:06:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.242+0000 7f92bbe8b640 1 Processor -- start 2024-09-19T06:06:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.242+0000 7f92bbe8b640 1 -- start start 2024-09-19T06:06:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92bbe8b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b41009d0 0x7f92b4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92bbe8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b4101370 0x7f92b41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92bbe8b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b41069e0 0x7f92b419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92bbe8b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4110cd0 con 0x7f92b4101370 2024-09-19T06:06:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92bbe8b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4110b50 con 0x7f92b41009d0 2024-09-19T06:06:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92bbe8b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4110e50 con 0x7f92b41069e0 2024-09-19T06:06:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92b9426640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b4101370 0x7f92b41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92b9426640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b4101370 0x7f92b41988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59220/0 (socket says 172.21.15.5:59220) 2024-09-19T06:06:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92b9426640 1 -- 172.21.15.5:0/4228331387 learned_addr learned my addr 172.21.15.5:0/4228331387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92b9c27640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b41009d0 0x7f92b4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92ba428640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b41069e0 0x7f92b419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92b9426640 1 -- 172.21.15.5:0/4228331387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b41069e0 msgr2=0x7f92b419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92b9426640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b41069e0 0x7f92b419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92b9426640 1 -- 172.21.15.5:0/4228331387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b41009d0 msgr2=0x7f92b4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92b9426640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b41009d0 0x7f92b4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92b9426640 1 -- 172.21.15.5:0/4228331387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b419d350 con 0x7f92b4101370 2024-09-19T06:06:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92b9c27640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b41009d0 0x7f92b4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:06:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92b9426640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b4101370 0x7f92b41988a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f92a400e9f0 tx=0x7f92a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92aaffd640 1 -- 172.21.15.5:0/4228331387 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a400cd80 con 0x7f92b4101370 2024-09-19T06:06:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92aaffd640 1 -- 172.21.15.5:0/4228331387 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92a40054d0 con 0x7f92b4101370 2024-09-19T06:06:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92bbe8b640 1 -- 172.21.15.5:0/4228331387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b419d640 con 0x7f92b4101370 2024-09-19T06:06:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.243+0000 7f92aaffd640 1 -- 172.21.15.5:0/4228331387 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4010490 con 0x7f92b4101370 2024-09-19T06:06:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.244+0000 7f92bbe8b640 1 -- 172.21.15.5:0/4228331387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b41a4ed0 con 0x7f92b4101370 2024-09-19T06:06:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.246+0000 7f92aaffd640 1 -- 172.21.15.5:0/4228331387 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f92a4005020 con 0x7f92b4101370 2024-09-19T06:06:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.246+0000 7f92bec23740 1 Processor -- start 2024-09-19T06:06:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.246+0000 7f92bec23740 1 -- start start 2024-09-19T06:06:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.246+0000 7f92bec23740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c62ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.246+0000 7f92bec23740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c63410 0x556e41c6d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.246+0000 7f92bec23740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c6dcb0 0x556e41c700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92bec23740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e41bc72f0 con 0x556e41c63410 2024-09-19T06:06:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92bec23740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556e41b6f270 con 0x556e41c6dcb0 2024-09-19T06:06:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92bec23740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556e41bc32b0 con 0x556e41c62af0 2024-09-19T06:06:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92b9c27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c63410 0x556e41c6d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92ba428640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c62ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92ba428640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c62ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55822/0 (socket says 172.21.15.5:55822) 2024-09-19T06:06:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92ba428640 1 -- 172.21.15.5:0/1258115172 learned_addr learned my addr 172.21.15.5:0/1258115172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92aaffd640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92a00a33b0 0x7f92a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92b9426640 1 --2- 172.21.15.5:0/1258115172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c6dcb0 0x556e41c700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92ba428640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92a00a33b0 0x7f92a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92aaffd640 1 -- 172.21.15.5:0/4228331387 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f92a40d01f0 con 0x7f92b4101370 2024-09-19T06:06:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92b9c27640 1 -- 172.21.15.5:0/1258115172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 msgr2=0x556e41c62ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92b9c27640 1 --2- 172.21.15.5:0/1258115172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c62ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92b9c27640 1 -- 172.21.15.5:0/1258115172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c6dcb0 msgr2=0x556e41c700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92b9c27640 1 --2- 172.21.15.5:0/1258115172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c6dcb0 0x556e41c700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92b9c27640 1 -- 172.21.15.5:0/1258115172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e41b6e340 con 0x556e41c63410 2024-09-19T06:06:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92b9426640 1 --2- 172.21.15.5:0/1258115172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c6dcb0 0x556e41c700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:06:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92ba428640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92a00a33b0 0x7f92a00a5870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009fd0 tx=0x7f92ac004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f92b9c27640 1 --2- 172.21.15.5:0/1258115172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c63410 0x556e41c6d770 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009900 tx=0x7f92b0032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba6d0263abcf12e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.247+0000 7f929ffff640 1 -- 172.21.15.5:0/1258115172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b003f070 con 0x556e41c63410 2024-09-19T06:06:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.248+0000 7f929ffff640 1 -- 172.21.15.5:0/1258115172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92b0004030 con 0x556e41c63410 2024-09-19T06:06:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.248+0000 7f929ffff640 1 -- 172.21.15.5:0/1258115172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0035070 con 0x556e41c63410 2024-09-19T06:06:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.248+0000 7f92bec23740 1 -- 172.21.15.5:0/1258115172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c63410 msgr2=0x556e41c6d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.248+0000 7f92bec23740 1 --2- 172.21.15.5:0/1258115172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c63410 0x556e41c6d770 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009900 tx=0x7f92b0032c00 comp rx=0 tx=0).stop 2024-09-19T06:06:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.248+0000 7f92bec23740 1 -- 172.21.15.5:0/1258115172 shutdown_connections 2024-09-19T06:06:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.248+0000 7f92bec23740 1 --2- 172.21.15.5:0/1258115172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c6dcb0 0x556e41c700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.248+0000 7f92bec23740 1 --2- 172.21.15.5:0/1258115172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c63410 0x556e41c6d770 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.248+0000 7f92bec23740 1 --2- 172.21.15.5:0/1258115172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c62ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.248+0000 7f92bec23740 1 -- 172.21.15.5:0/1258115172 >> 172.21.15.5:0/1258115172 conn(0x556e41c5f060 msgr2=0x556e41c5f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.248+0000 7f92bec23740 1 -- 172.21.15.5:0/1258115172 shutdown_connections 2024-09-19T06:06:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.248+0000 7f92bec23740 1 -- 172.21.15.5:0/1258115172 wait complete. 2024-09-19T06:06:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92bec23740 1 -- start start 2024-09-19T06:06:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92bec23740 1 Processor -- start 2024-09-19T06:06:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92bec23740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c6abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92bec23740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c6b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92bec23740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41c6b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92bec23740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e41c70880 con 0x556e41c6dcb0 2024-09-19T06:06:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92bec23740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556e41b50290 con 0x556e41c63410 2024-09-19T06:06:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92bec23740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556e41c70a00 con 0x556e41c62af0 2024-09-19T06:06:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92bec23740 10 client.0 _ll_register_callbacks cb 0x556e41b81140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92ba428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41c6b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92b9c27640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c6abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92ba428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41c6b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59238/0 (socket says 172.21.15.5:59238) 2024-09-19T06:06:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92ba428640 1 -- 172.21.15.5:0/1306276494 learned_addr learned my addr 172.21.15.5:0/1306276494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.249+0000 7f92b9426640 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c6b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f92b9c27640 1 -- 172.21.15.5:0/1306276494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 msgr2=0x556e41c6b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f92b9c27640 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c6b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f92b9c27640 1 -- 172.21.15.5:0/1306276494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 msgr2=0x556e41c6b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f92b9c27640 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41c6b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f92b9c27640 1 -- 172.21.15.5:0/1306276494 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e418da210 con 0x556e41c62af0 2024-09-19T06:06:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f92ba428640 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41c6b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:06:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f92b9426640 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c6b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:06:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f92b9c27640 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c6abd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009900 tx=0x7f92b0005510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391539e75cc4f145 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f929d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:06:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f929d7fa640 1 -- 172.21.15.5:0/1306276494 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0035070 con 0x556e41c62af0 2024-09-19T06:06:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f929d7fa640 1 -- 172.21.15.5:0/1306276494 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f92b00316a0 con 0x556e41c62af0 2024-09-19T06:06:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f92bec23740 1 -- 172.21.15.5:0/1306276494 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92b0004030 con 0x556e41c62af0 2024-09-19T06:06:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f929d7fa640 1 -- 172.21.15.5:0/1306276494 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f92b0055050 con 0x556e41c62af0 2024-09-19T06:06:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f92bec23740 10 client.25801 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f92bec23740 1 -- 172.21.15.5:0/1306276494 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556e419a6620 con 0x556e41c62af0 2024-09-19T06:06:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.250+0000 7f929d7fa640 1 -- 172.21.15.5:0/1306276494 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92b0004030 con 0x556e41c62af0 2024-09-19T06:06:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.251+0000 7f929d7fa640 1 -- 172.21.15.5:0/1306276494 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f92b0004880 con 0x556e41c62af0 2024-09-19T06:06:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.251+0000 7f92bec23740 10 client.25801 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.251+0000 7f92bec23740 10 client.25801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.251+0000 7f92bec23740 20 client.25801 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.251+0000 7f92bec23740 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e41c7dad0 0x556e41c7fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.251+0000 7f92bec23740 4 client.25801 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:06:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.251+0000 7f92bec23740 1 -- 172.21.15.5:0/1306276494 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e41bd4a20 con 0x556e41c7dad0 2024-09-19T06:06:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.251+0000 7f92b9426640 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e41c7dad0 0x556e41c7fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.251+0000 7f92b9426640 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e41c7dad0 0x556e41c7fec0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85ea53161e429b44 server_cookie=3e68763b7eb36303 in_seq=0 out_seq=0 2024-09-19T06:06:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.251+0000 7f929d7fa640 10 client.25801 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f929d7fa640 1 -- 172.21.15.5:0/1306276494 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e41bd4a20 con 0x556e41c7dad0 2024-09-19T06:06:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f929d7fa640 10 client.25801 handle_command_reply: tid=0 2024-09-19T06:06:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f92bec23740 1 client.25801 shutdown 2024-09-19T06:06:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f92bec23740 1 -- 172.21.15.5:0/1306276494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 msgr2=0x556e41c6abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f92bec23740 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c6abd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009900 tx=0x7f92b0005510 comp rx=0 tx=0).stop 2024-09-19T06:06:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f92bec23740 1 -- 172.21.15.5:0/1306276494 shutdown_connections 2024-09-19T06:06:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f92bec23740 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e41c7dad0 0x556e41c7fec0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f92bec23740 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41c6b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f92bec23740 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c6b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f929d7fa640 0 client.25801 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f92bec23740 1 --2- 172.21.15.5:0/1306276494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c6abd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f92bec23740 1 -- 172.21.15.5:0/1306276494 >> 172.21.15.5:0/1306276494 conn(0x556e41c5f060 msgr2=0x556e41c64d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f92bec23740 1 -- 172.21.15.5:0/1306276494 shutdown_connections 2024-09-19T06:06:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.252+0000 7f92bec23740 1 -- 172.21.15.5:0/1306276494 wait complete. 2024-09-19T06:06:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.253+0000 7f92bec23740 20 client.25801 trim_cache size 0 max 16384 2024-09-19T06:06:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.267+0000 7f92bec23740 1 Processor -- start 2024-09-19T06:06:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.267+0000 7f92bec23740 1 -- start start 2024-09-19T06:06:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92bec23740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c65560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92bec23740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c6b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92bec23740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41c5e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92bec23740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41ad7a0 con 0x556e41c6dcb0 2024-09-19T06:06:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92bec23740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41af340 con 0x556e41c63410 2024-09-19T06:06:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92bec23740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41ade70 con 0x556e41c62af0 2024-09-19T06:06:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92ba428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41c5e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92b9426640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c6b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92ba428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41c5e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59252/0 (socket says 172.21.15.5:59252) 2024-09-19T06:06:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92b9426640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c6b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36624/0 (socket says 172.21.15.5:36624) 2024-09-19T06:06:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92ba428640 1 -- 172.21.15.5:0/1794074707 learned_addr learned my addr 172.21.15.5:0/1794074707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92b9c27640 1 --2- 172.21.15.5:0/1794074707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c65560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92b9426640 1 -- 172.21.15.5:0/1794074707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 msgr2=0x556e41c65560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92b9426640 1 --2- 172.21.15.5:0/1794074707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c65560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92b9426640 1 -- 172.21.15.5:0/1794074707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 msgr2=0x556e41c5e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92b9426640 1 --2- 172.21.15.5:0/1794074707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41c5e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92b9426640 1 -- 172.21.15.5:0/1794074707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e41c63240 con 0x556e41c63410 2024-09-19T06:06:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.268+0000 7f92b9426640 1 --2- 172.21.15.5:0/1794074707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c6b480 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f92a40cd950 tx=0x7f92a406f3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84c8d6a427e64b27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.269+0000 7f929f7fe640 1 -- 172.21.15.5:0/1794074707 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4005310 con 0x556e41c63410 2024-09-19T06:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.269+0000 7f929f7fe640 1 -- 172.21.15.5:0/1794074707 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92a4005680 con 0x556e41c63410 2024-09-19T06:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.269+0000 7f929f7fe640 1 -- 172.21.15.5:0/1794074707 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a409a7e0 con 0x556e41c63410 2024-09-19T06:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 -- 172.21.15.5:0/1794074707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 msgr2=0x556e41c6b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 --2- 172.21.15.5:0/1794074707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c6b480 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f92a40cd950 tx=0x7f92a406f3d0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 -- 172.21.15.5:0/1794074707 shutdown_connections 2024-09-19T06:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 --2- 172.21.15.5:0/1794074707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41c5e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 --2- 172.21.15.5:0/1794074707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c6b480 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 --2- 172.21.15.5:0/1794074707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c65560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 -- 172.21.15.5:0/1794074707 >> 172.21.15.5:0/1794074707 conn(0x556e41c5f060 msgr2=0x556e41c5d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 -- 172.21.15.5:0/1794074707 shutdown_connections 2024-09-19T06:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 -- 172.21.15.5:0/1794074707 wait complete. 2024-09-19T06:06:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 -- start start 2024-09-19T06:06:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 Processor -- start 2024-09-19T06:06:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c71210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c71750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41ce6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41aec70 con 0x556e41c6dcb0 2024-09-19T06:06:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41ae540 con 0x556e41c63410 2024-09-19T06:06:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.270+0000 7f92bec23740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41ab780 con 0x556e41c62af0 2024-09-19T06:06:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92bec23740 10 client.0 _ll_register_callbacks cb 0x7f92ac009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92ba428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41ce6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92b9426640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c71750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92ba428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41ce6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59258/0 (socket says 172.21.15.5:59258) 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92ba428640 1 -- 172.21.15.5:0/1531233779 learned_addr learned my addr 172.21.15.5:0/1531233779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92b9426640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c71750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36628/0 (socket says 172.21.15.5:36628) 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92b9c27640 1 --2- 172.21.15.5:0/1531233779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c71210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92b9426640 1 -- 172.21.15.5:0/1531233779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 msgr2=0x556e41c71210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92b9426640 1 --2- 172.21.15.5:0/1531233779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c71210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92b9426640 1 -- 172.21.15.5:0/1531233779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 msgr2=0x556e41ce6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92b9426640 1 --2- 172.21.15.5:0/1531233779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41ce6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92b9426640 1 -- 172.21.15.5:0/1531233779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e41ce6960 con 0x556e41c63410 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92b9426640 1 --2- 172.21.15.5:0/1531233779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c71750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f92a40d01c0 tx=0x7f92a40097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71cae7943929ab65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f929d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f929d7fa640 1 -- 172.21.15.5:0/1531233779 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a40b4070 con 0x556e41c63410 2024-09-19T06:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f929d7fa640 1 -- 172.21.15.5:0/1531233779 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f92a40046c0 con 0x556e41c63410 2024-09-19T06:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.271+0000 7f92bec23740 1 -- 172.21.15.5:0/1531233779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92a4005680 con 0x556e41c63410 2024-09-19T06:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.272+0000 7f929d7fa640 1 -- 172.21.15.5:0/1531233779 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f92a40a0890 con 0x556e41c63410 2024-09-19T06:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.272+0000 7f92bec23740 10 client.25889 fetch_fsmap learned FSMap version 32 2024-09-19T06:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.272+0000 7f92bec23740 1 -- 172.21.15.5:0/1531233779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556e41ce7c70 con 0x556e41c63410 2024-09-19T06:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.272+0000 7f929d7fa640 1 -- 172.21.15.5:0/1531233779 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92a4005680 con 0x556e41c63410 2024-09-19T06:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.272+0000 7f929d7fa640 1 -- 172.21.15.5:0/1531233779 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f92a4098b50 con 0x556e41c63410 2024-09-19T06:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.272+0000 7f92bec23740 10 client.25889 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.272+0000 7f92bec23740 10 client.25889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.272+0000 7f92bec23740 20 client.25889 populate_metadata read hostname 'smithi005' 2024-09-19T06:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.272+0000 7f92bec23740 1 --2- 172.21.15.5:0/1531233779 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e41ce8430 0x556e41cea850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:06:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.272+0000 7f92bec23740 4 client.25889 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:06:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.272+0000 7f92bec23740 1 -- 172.21.15.5:0/1531233779 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e41bced70 con 0x556e41ce8430 2024-09-19T06:06:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.272+0000 7f92b9c27640 1 --2- 172.21.15.5:0/1531233779 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e41ce8430 0x556e41cea850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:06:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.273+0000 7f92b9c27640 1 --2- 172.21.15.5:0/1531233779 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e41ce8430 0x556e41cea850 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d7edda8723e164 server_cookie=7a644591b09797fe in_seq=0 out_seq=0 2024-09-19T06:06:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.273+0000 7f929d7fa640 10 client.25889 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.290+0000 7f929d7fa640 1 -- 172.21.15.5:0/1531233779 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e41bced70 con 0x556e41ce8430 2024-09-19T06:06:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.290+0000 7f929d7fa640 10 client.25889 handle_command_reply: tid=0 2024-09-19T06:06:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.290+0000 7f92bec23740 1 client.25889 shutdown 2024-09-19T06:06:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.291+0000 7f92bec23740 1 -- 172.21.15.5:0/1531233779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 msgr2=0x556e41c71750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.291+0000 7f92bec23740 1 --2- 172.21.15.5:0/1531233779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c71750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f92a40d01c0 tx=0x7f92a40097a0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.291+0000 7f92bec23740 1 -- 172.21.15.5:0/1531233779 shutdown_connections 2024-09-19T06:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.291+0000 7f92bec23740 1 --2- 172.21.15.5:0/1531233779 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e41ce8430 0x556e41cea850 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.291+0000 7f92bec23740 1 --2- 172.21.15.5:0/1531233779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e41c6dcb0 0x556e41ce6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.291+0000 7f92bec23740 1 --2- 172.21.15.5:0/1531233779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e41c63410 0x556e41c71750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.291+0000 7f929d7fa640 0 client.25889 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.291+0000 7f92bec23740 1 --2- 172.21.15.5:0/1531233779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e41c62af0 0x556e41c71210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.291+0000 7f92bec23740 1 -- 172.21.15.5:0/1531233779 >> 172.21.15.5:0/1531233779 conn(0x556e41c5f060 msgr2=0x556e41c5d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.291+0000 7f92bec23740 1 -- 172.21.15.5:0/1531233779 shutdown_connections 2024-09-19T06:06:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.291+0000 7f92bec23740 1 -- 172.21.15.5:0/1531233779 wait complete. 2024-09-19T06:06:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.291+0000 7f92bec23740 20 client.25889 trim_cache size 0 max 16384 2024-09-19T06:06:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:06:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:06:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:06:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:06:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.292+0000 7f92bbe8b640 1 -- 172.21.15.5:0/4228331387 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92a00a33b0 msgr2=0x7f92a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.292+0000 7f92bbe8b640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92a00a33b0 0x7f92a00a5870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009fd0 tx=0x7f92ac004ab0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.292+0000 7f92bbe8b640 1 -- 172.21.15.5:0/4228331387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b4101370 msgr2=0x7f92b41988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:06:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.292+0000 7f92bbe8b640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b4101370 0x7f92b41988a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f92a400e9f0 tx=0x7f92a400edf0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.292+0000 7f92bbe8b640 1 -- 172.21.15.5:0/4228331387 shutdown_connections 2024-09-19T06:06:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.292+0000 7f92bbe8b640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92a00a33b0 0x7f92a00a5870 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.292+0000 7f92bbe8b640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b41069e0 0x7f92b419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.292+0000 7f92bbe8b640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b4101370 0x7f92b41988a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.292+0000 7f92bbe8b640 1 --2- 172.21.15.5:0/4228331387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b41009d0 0x7f92b4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:06:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.292+0000 7f92bbe8b640 1 -- 172.21.15.5:0/4228331387 >> 172.21.15.5:0/4228331387 conn(0x7f92b40fc6b0 msgr2=0x7f92b41048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:06:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.292+0000 7f92bbe8b640 1 -- 172.21.15.5:0/4228331387 shutdown_connections 2024-09-19T06:06:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:06:49.292+0000 7f92bbe8b640 1 -- 172.21.15.5:0/4228331387 wait complete. 2024-09-19T06:06:49.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 70726 3.03 MB/sec execute 15 sec latency 43.416 ms 2024-09-19T06:06:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:49 smithi157 ceph-mon[35346]: pgmap v108: 97 pgs: 97 active+clean; 55 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-09-19T06:06:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:49 smithi005 ceph-mon[30955]: pgmap v108: 97 pgs: 97 active+clean; 55 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-09-19T06:06:50.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:49 smithi049 ceph-mon[34640]: pgmap v108: 97 pgs: 97 active+clean; 55 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-09-19T06:06:50.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 71196 3.12 MB/sec execute 16 sec latency 15.603 ms 2024-09-19T06:06:51.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 71472 2.95 MB/sec execute 17 sec latency 11.579 ms 2024-09-19T06:06:52.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:51 smithi157 ceph-mon[35346]: pgmap v109: 97 pgs: 97 active+clean; 55 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-09-19T06:06:52.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:51 smithi005 ceph-mon[30955]: pgmap v109: 97 pgs: 97 active+clean; 55 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-09-19T06:06:52.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:51 smithi049 ceph-mon[34640]: pgmap v109: 97 pgs: 97 active+clean; 55 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-09-19T06:06:52.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 71937 2.84 MB/sec execute 18 sec latency 13.508 ms 2024-09-19T06:06:53.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 72555 2.92 MB/sec execute 19 sec latency 14.707 ms 2024-09-19T06:06:54.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:53 smithi157 ceph-mon[35346]: pgmap v110: 97 pgs: 97 active+clean; 52 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-09-19T06:06:54.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:53 smithi005 ceph-mon[30955]: pgmap v110: 97 pgs: 97 active+clean; 52 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-09-19T06:06:54.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:53 smithi049 ceph-mon[34640]: pgmap v110: 97 pgs: 97 active+clean; 52 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-09-19T06:06:54.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 73295 2.85 MB/sec execute 20 sec latency 11.420 ms 2024-09-19T06:06:55.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 73867 2.93 MB/sec execute 21 sec latency 16.964 ms 2024-09-19T06:06:56.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:55 smithi157 ceph-mon[35346]: pgmap v111: 97 pgs: 97 active+clean; 52 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-19T06:06:56.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:55 smithi005 ceph-mon[30955]: pgmap v111: 97 pgs: 97 active+clean; 52 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-19T06:06:56.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:55 smithi049 ceph-mon[34640]: pgmap v111: 97 pgs: 97 active+clean; 52 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-19T06:06:56.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 74358 2.94 MB/sec execute 22 sec latency 13.461 ms 2024-09-19T06:06:57.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 74844 3.00 MB/sec execute 23 sec latency 12.260 ms 2024-09-19T06:06:58.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:57 smithi157 ceph-mon[35346]: pgmap v112: 97 pgs: 97 active+clean; 53 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-19T06:06:58.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:57 smithi005 ceph-mon[30955]: pgmap v112: 97 pgs: 97 active+clean; 53 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-19T06:06:58.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:57 smithi049 ceph-mon[34640]: pgmap v112: 97 pgs: 97 active+clean; 53 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-19T06:06:58.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 75126 2.89 MB/sec execute 24 sec latency 15.115 ms 2024-09-19T06:06:59.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 75640 2.86 MB/sec execute 25 sec latency 17.551 ms 2024-09-19T06:07:00.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:06:59 smithi157 ceph-mon[35346]: pgmap v113: 97 pgs: 97 active+clean; 53 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-19T06:07:00.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:06:59 smithi005 ceph-mon[30955]: pgmap v113: 97 pgs: 97 active+clean; 53 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-19T06:07:00.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:06:59 smithi049 ceph-mon[34640]: pgmap v113: 97 pgs: 97 active+clean; 53 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-19T06:07:00.825 INFO:tasks.workunit.client.0.smithi005.stdout: 1 76202 2.87 MB/sec execute 26 sec latency 16.988 ms 2024-09-19T06:07:01.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 76904 2.83 MB/sec execute 27 sec latency 11.641 ms 2024-09-19T06:07:02.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:01 smithi157 ceph-mon[35346]: pgmap v114: 97 pgs: 97 active+clean; 53 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-19T06:07:02.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:01 smithi005 ceph-mon[30955]: pgmap v114: 97 pgs: 97 active+clean; 53 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-19T06:07:02.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:01 smithi049 ceph-mon[34640]: pgmap v114: 97 pgs: 97 active+clean; 53 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-19T06:07:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.390+0000 7f429c223640 1 -- 172.21.15.5:0/3093598878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f429410c0a0 msgr2=0x7f429410e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.390+0000 7f429c223640 1 --2- 172.21.15.5:0/3093598878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f429410c0a0 0x7f429410e540 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f428c009ca0 tx=0x7f428c02a950 comp rx=0 tx=0).stop 2024-09-19T06:07:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.390+0000 7f429c223640 1 -- 172.21.15.5:0/3093598878 shutdown_connections 2024-09-19T06:07:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.390+0000 7f429c223640 1 --2- 172.21.15.5:0/3093598878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f429410c0a0 0x7f429410e540 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.390+0000 7f429c223640 1 --2- 172.21.15.5:0/3093598878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42941035a0 0x7f429410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.390+0000 7f429c223640 1 --2- 172.21.15.5:0/3093598878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4294102c80 0x7f4294103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.390+0000 7f429c223640 1 -- 172.21.15.5:0/3093598878 >> 172.21.15.5:0/3093598878 conn(0x7f42940779a0 msgr2=0x7f42940fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.390+0000 7f429c223640 1 -- 172.21.15.5:0/3093598878 shutdown_connections 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.390+0000 7f429c223640 1 -- 172.21.15.5:0/3093598878 wait complete. 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f429c223640 1 Processor -- start 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f429c223640 1 -- start start 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f429c223640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4294102c80 0x7f42941008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f429c223640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42941035a0 0x7f4294100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f429c223640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f429410c0a0 0x7f4294101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f429c223640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4294111da0 con 0x7f4294102c80 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f429c223640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4294111c20 con 0x7f429410c0a0 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f429c223640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4294111f20 con 0x7f42941035a0 2024-09-19T06:07:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f4299fbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4294102c80 0x7f42941008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f4299fbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4294102c80 0x7f42941008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39904/0 (socket says 172.21.15.5:39904) 2024-09-19T06:07:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f4299fbf640 1 -- 172.21.15.5:0/3097042176 learned_addr learned my addr 172.21.15.5:0/3097042176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f42997be640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42941035a0 0x7f4294100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f429a7c0640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f429410c0a0 0x7f4294101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f4299fbf640 1 -- 172.21.15.5:0/3097042176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42941035a0 msgr2=0x7f4294100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f4299fbf640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42941035a0 0x7f4294100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f4299fbf640 1 -- 172.21.15.5:0/3097042176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f429410c0a0 msgr2=0x7f4294101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f4299fbf640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f429410c0a0 0x7f4294101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.391+0000 7f4299fbf640 1 -- 172.21.15.5:0/3097042176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42941a6790 con 0x7f4294102c80 2024-09-19T06:07:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.392+0000 7f4299fbf640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4294102c80 0x7f42941008d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f429000e9f0 tx=0x7f429000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.392+0000 7f428affd640 1 -- 172.21.15.5:0/3097042176 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429000cd80 con 0x7f4294102c80 2024-09-19T06:07:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.392+0000 7f428affd640 1 -- 172.21.15.5:0/3097042176 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42900108e0 con 0x7f4294102c80 2024-09-19T06:07:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.392+0000 7f429c223640 1 -- 172.21.15.5:0/3097042176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42941a6a80 con 0x7f4294102c80 2024-09-19T06:07:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.392+0000 7f428affd640 1 -- 172.21.15.5:0/3097042176 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290018920 con 0x7f4294102c80 2024-09-19T06:07:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.392+0000 7f429c223640 1 -- 172.21.15.5:0/3097042176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42941a7020 con 0x7f4294102c80 2024-09-19T06:07:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.394+0000 7f428affd640 1 -- 172.21.15.5:0/3097042176 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4290010a90 con 0x7f4294102c80 2024-09-19T06:07:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.394+0000 7f429efbe740 1 Processor -- start 2024-09-19T06:07:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f429efbe740 1 -- start start 2024-09-19T06:07:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f429efbe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358977af0 0x561358977ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f429efbe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 0x561358982770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f429efbe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358982cb0 0x5613589850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f429efbe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5613588dc2f0 con 0x561358978410 2024-09-19T06:07:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f429efbe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561358884270 con 0x561358982cb0 2024-09-19T06:07:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f429efbe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5613588d82b0 con 0x561358977af0 2024-09-19T06:07:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f4299fbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 0x561358982770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f42997be640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358982cb0 0x5613589850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f4299fbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 0x561358982770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39916/0 (socket says 172.21.15.5:39916) 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f4299fbf640 1 -- 172.21.15.5:0/2412424649 learned_addr learned my addr 172.21.15.5:0/2412424649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f429a7c0640 1 --2- 172.21.15.5:0/2412424649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358977af0 0x561358977ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f42997be640 1 -- 172.21.15.5:0/2412424649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358977af0 msgr2=0x561358977ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f42997be640 1 --2- 172.21.15.5:0/2412424649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358977af0 0x561358977ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f42997be640 1 -- 172.21.15.5:0/2412424649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 msgr2=0x561358982770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f42997be640 1 --2- 172.21.15.5:0/2412424649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 0x561358982770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f42997be640 1 -- 172.21.15.5:0/2412424649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561358883340 con 0x561358982cb0 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f4299fbf640 1 --2- 172.21.15.5:0/2412424649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 0x561358982770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.395+0000 7f428affd640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42800a3430 0x7f42800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.396+0000 7f429a7c0640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42800a3430 0x7f42800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.396+0000 7f42997be640 1 --2- 172.21.15.5:0/2412424649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358982cb0 0x5613589850a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4284009bf0 tx=0x7f4284011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b025afd333e08ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.396+0000 7f428affd640 1 -- 172.21.15.5:0/3097042176 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f42900cffa0 con 0x7f4294102c80 2024-09-19T06:07:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.396+0000 7f427ffff640 1 -- 172.21.15.5:0/2412424649 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284004310 con 0x561358982cb0 2024-09-19T06:07:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.396+0000 7f427ffff640 1 -- 172.21.15.5:0/2412424649 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42840044b0 con 0x561358982cb0 2024-09-19T06:07:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.396+0000 7f427ffff640 1 -- 172.21.15.5:0/2412424649 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284019910 con 0x561358982cb0 2024-09-19T06:07:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.397+0000 7f429efbe740 1 -- 172.21.15.5:0/2412424649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358982cb0 msgr2=0x5613589850a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.397+0000 7f429a7c0640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42800a3430 0x7f42800a58f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f428c006010 tx=0x7f428c005d80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.397+0000 7f429efbe740 1 --2- 172.21.15.5:0/2412424649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358982cb0 0x5613589850a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4284009bf0 tx=0x7f4284011440 comp rx=0 tx=0).stop 2024-09-19T06:07:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.397+0000 7f429efbe740 1 -- 172.21.15.5:0/2412424649 shutdown_connections 2024-09-19T06:07:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.397+0000 7f429efbe740 1 --2- 172.21.15.5:0/2412424649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358982cb0 0x5613589850a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.397+0000 7f429efbe740 1 --2- 172.21.15.5:0/2412424649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 0x561358982770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.397+0000 7f429efbe740 1 --2- 172.21.15.5:0/2412424649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358977af0 0x561358977ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.397+0000 7f429efbe740 1 -- 172.21.15.5:0/2412424649 >> 172.21.15.5:0/2412424649 conn(0x561358974060 msgr2=0x561358974440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.397+0000 7f429efbe740 1 -- 172.21.15.5:0/2412424649 shutdown_connections 2024-09-19T06:07:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.397+0000 7f429efbe740 1 -- 172.21.15.5:0/2412424649 wait complete. 2024-09-19T06:07:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.397+0000 7f429efbe740 1 -- start start 2024-09-19T06:07:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.397+0000 7f429efbe740 1 Processor -- start 2024-09-19T06:07:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f429efbe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 0x56135897fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f429efbe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358978410 0x561358980000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f429efbe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358982cb0 0x561358980540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f429efbe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561358985880 con 0x561358977af0 2024-09-19T06:07:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f429efbe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561358865290 con 0x561358982cb0 2024-09-19T06:07:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f429efbe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561358985a00 con 0x561358978410 2024-09-19T06:07:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f429efbe740 10 client.0 _ll_register_callbacks cb 0x561358896140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f4299fbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 0x56135897fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f4299fbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 0x56135897fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39922/0 (socket says 172.21.15.5:39922) 2024-09-19T06:07:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f4299fbf640 1 -- 172.21.15.5:0/191555095 learned_addr learned my addr 172.21.15.5:0/191555095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f429a7c0640 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358982cb0 0x561358980540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f42997be640 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358978410 0x561358980000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f4299fbf640 1 -- 172.21.15.5:0/191555095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358978410 msgr2=0x561358980000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f4299fbf640 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358978410 0x561358980000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f4299fbf640 1 -- 172.21.15.5:0/191555095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358982cb0 msgr2=0x561358980540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f4299fbf640 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358982cb0 0x561358980540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f4299fbf640 1 -- 172.21.15.5:0/191555095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613585ef210 con 0x561358977af0 2024-09-19T06:07:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f4299fbf640 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 0x56135897fac0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4290098cb0 tx=0x7f4290098670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb24a3cad86b06d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.398+0000 7f427d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:07:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f427d7fa640 1 -- 172.21.15.5:0/191555095 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42900cdb20 con 0x561358977af0 2024-09-19T06:07:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f427d7fa640 1 -- 172.21.15.5:0/191555095 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f42900a01b0 con 0x561358977af0 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f429efbe740 1 -- 172.21.15.5:0/191555095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42840044b0 con 0x561358977af0 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f427d7fa640 1 -- 172.21.15.5:0/191555095 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4290010de0 con 0x561358977af0 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f429efbe740 10 client.16308 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f429efbe740 1 -- 172.21.15.5:0/191555095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5613586bb620 con 0x561358977af0 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f427d7fa640 1 -- 172.21.15.5:0/191555095 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42840044b0 con 0x561358977af0 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f427d7fa640 1 -- 172.21.15.5:0/191555095 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f42900a4a10 con 0x561358977af0 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f429efbe740 10 client.16308 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f429efbe740 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f429efbe740 20 client.16308 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f429efbe740 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561358992c00 0x561358995010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f429efbe740 4 client.16308 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f429efbe740 1 -- 172.21.15.5:0/191555095 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613588e9a20 con 0x561358992c00 2024-09-19T06:07:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.399+0000 7f42997be640 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561358992c00 0x561358995010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.400+0000 7f42997be640 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561358992c00 0x561358995010 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eca2345167b864c server_cookie=e76115ab102a83e7 in_seq=0 out_seq=0 2024-09-19T06:07:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.400+0000 7f427d7fa640 10 client.16308 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f427d7fa640 1 -- 172.21.15.5:0/191555095 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613588e9a20 con 0x561358992c00 2024-09-19T06:07:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f427d7fa640 10 client.16308 handle_command_reply: tid=0 2024-09-19T06:07:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f429efbe740 1 client.16308 shutdown 2024-09-19T06:07:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f429efbe740 1 -- 172.21.15.5:0/191555095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 msgr2=0x56135897fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f429efbe740 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 0x56135897fac0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4290098cb0 tx=0x7f4290098670 comp rx=0 tx=0).stop 2024-09-19T06:07:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f429efbe740 1 -- 172.21.15.5:0/191555095 shutdown_connections 2024-09-19T06:07:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f429efbe740 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561358992c00 0x561358995010 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f429efbe740 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358982cb0 0x561358980540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f427d7fa640 0 client.16308 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f429efbe740 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358978410 0x561358980000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f429efbe740 1 --2- 172.21.15.5:0/191555095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 0x56135897fac0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f429efbe740 1 -- 172.21.15.5:0/191555095 >> 172.21.15.5:0/191555095 conn(0x561358974060 msgr2=0x561358979dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f429efbe740 1 -- 172.21.15.5:0/191555095 shutdown_connections 2024-09-19T06:07:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f429efbe740 1 -- 172.21.15.5:0/191555095 wait complete. 2024-09-19T06:07:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.401+0000 7f429efbe740 20 client.16308 trim_cache size 0 max 16384 2024-09-19T06:07:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.417+0000 7f429efbe740 1 Processor -- start 2024-09-19T06:07:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.417+0000 7f429efbe740 1 -- start start 2024-09-19T06:07:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.417+0000 7f429efbe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 0x561358980370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.417+0000 7f429efbe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358978410 0x5613589730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.417+0000 7f429efbe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358982cb0 0x56135899a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.417+0000 7f429efbe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42941af910 con 0x561358977af0 2024-09-19T06:07:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.417+0000 7f429efbe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f42941b14b0 con 0x561358978410 2024-09-19T06:07:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.417+0000 7f429efbe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f42941affe0 con 0x561358982cb0 2024-09-19T06:07:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f4299fbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 0x561358980370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f4299fbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 0x561358980370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39936/0 (socket says 172.21.15.5:39936) 2024-09-19T06:07:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f4299fbf640 1 -- 172.21.15.5:0/1545161685 learned_addr learned my addr 172.21.15.5:0/1545161685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f429a7c0640 1 --2- 172.21.15.5:0/1545161685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358982cb0 0x56135899a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f42997be640 1 --2- 172.21.15.5:0/1545161685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358978410 0x5613589730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f4299fbf640 1 -- 172.21.15.5:0/1545161685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358982cb0 msgr2=0x56135899a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f4299fbf640 1 --2- 172.21.15.5:0/1545161685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358982cb0 0x56135899a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f4299fbf640 1 -- 172.21.15.5:0/1545161685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358978410 msgr2=0x5613589730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f4299fbf640 1 --2- 172.21.15.5:0/1545161685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358978410 0x5613589730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f4299fbf640 1 -- 172.21.15.5:0/1545161685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561358978240 con 0x561358977af0 2024-09-19T06:07:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f4299fbf640 1 --2- 172.21.15.5:0/1545161685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 0x561358980370 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f429009bfe0 tx=0x7f4290098580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49534a98d140eb64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f427f7fe640 1 -- 172.21.15.5:0/1545161685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429009be20 con 0x561358977af0 2024-09-19T06:07:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f427f7fe640 1 -- 172.21.15.5:0/1545161685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4290096e50 con 0x561358977af0 2024-09-19T06:07:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.418+0000 7f427f7fe640 1 -- 172.21.15.5:0/1545161685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42900cd920 con 0x561358977af0 2024-09-19T06:07:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.419+0000 7f429efbe740 1 -- 172.21.15.5:0/1545161685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 msgr2=0x561358980370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.419+0000 7f429efbe740 1 --2- 172.21.15.5:0/1545161685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 0x561358980370 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f429009bfe0 tx=0x7f4290098580 comp rx=0 tx=0).stop 2024-09-19T06:07:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.419+0000 7f429efbe740 1 -- 172.21.15.5:0/1545161685 shutdown_connections 2024-09-19T06:07:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.419+0000 7f429efbe740 1 --2- 172.21.15.5:0/1545161685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358982cb0 0x56135899a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.419+0000 7f429efbe740 1 --2- 172.21.15.5:0/1545161685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358978410 0x5613589730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.419+0000 7f429efbe740 1 --2- 172.21.15.5:0/1545161685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358977af0 0x561358980370 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.419+0000 7f429efbe740 1 -- 172.21.15.5:0/1545161685 >> 172.21.15.5:0/1545161685 conn(0x561358974060 msgr2=0x5613589729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.419+0000 7f429efbe740 1 -- 172.21.15.5:0/1545161685 shutdown_connections 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.419+0000 7f429efbe740 1 -- 172.21.15.5:0/1545161685 wait complete. 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.419+0000 7f429efbe740 1 -- start start 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.419+0000 7f429efbe740 1 Processor -- start 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f429efbe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358977af0 0x56135898b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f429efbe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 0x561358983910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f429efbe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358982cb0 0x561358983e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f429efbe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42941b0de0 con 0x561358978410 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f429efbe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f42941b06b0 con 0x561358977af0 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f429efbe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f42941ad8f0 con 0x561358982cb0 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f429efbe740 10 client.0 _ll_register_callbacks cb 0x7f42900d3740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f42997be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 0x561358983910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f42997be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 0x561358983910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39938/0 (socket says 172.21.15.5:39938) 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f42997be640 1 -- 172.21.15.5:0/1178127112 learned_addr learned my addr 172.21.15.5:0/1178127112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f429a7c0640 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358982cb0 0x561358983e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f4299fbf640 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358977af0 0x56135898b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f42997be640 1 -- 172.21.15.5:0/1178127112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358982cb0 msgr2=0x561358983e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f42997be640 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358982cb0 0x561358983e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f42997be640 1 -- 172.21.15.5:0/1178127112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358977af0 msgr2=0x56135898b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f42997be640 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358977af0 0x56135898b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f42997be640 1 -- 172.21.15.5:0/1178127112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561358984440 con 0x561358978410 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f4299fbf640 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358977af0 0x56135898b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f42997be640 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 0x561358983910 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f42840095f0 tx=0x7f42840119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b41814c464298f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.420+0000 7f427d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:07:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f427d7fa640 1 -- 172.21.15.5:0/1178127112 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284033060 con 0x561358978410 2024-09-19T06:07:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f427d7fa640 1 -- 172.21.15.5:0/1178127112 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f42840035f0 con 0x561358978410 2024-09-19T06:07:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f429efbe740 1 -- 172.21.15.5:0/1178127112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4290096e50 con 0x561358978410 2024-09-19T06:07:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f427d7fa640 1 -- 172.21.15.5:0/1178127112 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f428401f040 con 0x561358978410 2024-09-19T06:07:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f429efbe740 10 client.16314 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f429efbe740 1 -- 172.21.15.5:0/1178127112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561358989b30 con 0x561358978410 2024-09-19T06:07:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f427d7fa640 1 -- 172.21.15.5:0/1178127112 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4290096e50 con 0x561358978410 2024-09-19T06:07:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f427d7fa640 1 -- 172.21.15.5:0/1178127112 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f4284018070 con 0x561358978410 2024-09-19T06:07:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f429efbe740 10 client.16314 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f429efbe740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f429efbe740 20 client.16314 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f429efbe740 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5613589fdd40 0x561358a00160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f429efbe740 4 client.16314 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:07:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f429efbe740 1 -- 172.21.15.5:0/1178127112 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613588e3d70 con 0x5613589fdd40 2024-09-19T06:07:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.421+0000 7f4299fbf640 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5613589fdd40 0x561358a00160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.422+0000 7f427d7fa640 10 client.16314 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.422+0000 7f4299fbf640 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5613589fdd40 0x561358a00160 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f438e95db7fc425 server_cookie=7bde4e1e903df79e in_seq=0 out_seq=0 2024-09-19T06:07:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f427d7fa640 1 -- 172.21.15.5:0/1178127112 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613588e3d70 con 0x5613589fdd40 2024-09-19T06:07:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f427d7fa640 10 client.16314 handle_command_reply: tid=0 2024-09-19T06:07:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f429efbe740 1 client.16314 shutdown 2024-09-19T06:07:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f429efbe740 1 -- 172.21.15.5:0/1178127112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 msgr2=0x561358983910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f429efbe740 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 0x561358983910 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f42840095f0 tx=0x7f42840119f0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f429efbe740 1 -- 172.21.15.5:0/1178127112 shutdown_connections 2024-09-19T06:07:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f429efbe740 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5613589fdd40 0x561358a00160 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f427d7fa640 0 client.16314 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f429efbe740 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561358982cb0 0x561358983e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f429efbe740 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561358978410 0x561358983910 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f429efbe740 1 --2- 172.21.15.5:0/1178127112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561358977af0 0x56135898b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f429efbe740 1 -- 172.21.15.5:0/1178127112 >> 172.21.15.5:0/1178127112 conn(0x561358974060 msgr2=0x5613589731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f429efbe740 1 -- 172.21.15.5:0/1178127112 shutdown_connections 2024-09-19T06:07:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f429efbe740 1 -- 172.21.15.5:0/1178127112 wait complete. 2024-09-19T06:07:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.442+0000 7f429efbe740 20 client.16314 trim_cache size 0 max 16384 2024-09-19T06:07:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:07:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:07:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:07:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:07:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.443+0000 7f429c223640 1 -- 172.21.15.5:0/3097042176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42800a3430 msgr2=0x7f42800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.443+0000 7f429c223640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42800a3430 0x7f42800a58f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f428c006010 tx=0x7f428c005d80 comp rx=0 tx=0).stop 2024-09-19T06:07:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.443+0000 7f429c223640 1 -- 172.21.15.5:0/3097042176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4294102c80 msgr2=0x7f42941008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.443+0000 7f429c223640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4294102c80 0x7f42941008d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f429000e9f0 tx=0x7f429000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.443+0000 7f429c223640 1 -- 172.21.15.5:0/3097042176 shutdown_connections 2024-09-19T06:07:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.443+0000 7f429c223640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f429410c0a0 0x7f4294101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.443+0000 7f429c223640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42800a3430 0x7f42800a58f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.443+0000 7f429c223640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42941035a0 0x7f4294100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.443+0000 7f429c223640 1 --2- 172.21.15.5:0/3097042176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4294102c80 0x7f42941008d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.443+0000 7f429c223640 1 -- 172.21.15.5:0/3097042176 >> 172.21.15.5:0/3097042176 conn(0x7f42940779a0 msgr2=0x7f42940fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.443+0000 7f429c223640 1 -- 172.21.15.5:0/3097042176 shutdown_connections 2024-09-19T06:07:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:02.443+0000 7f429c223640 1 -- 172.21.15.5:0/3097042176 wait complete. 2024-09-19T06:07:02.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 77517 2.93 MB/sec execute 28 sec latency 12.891 ms 2024-09-19T06:07:03.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 78068 2.99 MB/sec execute 29 sec latency 14.334 ms 2024-09-19T06:07:04.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:03 smithi157 ceph-mon[35346]: pgmap v115: 97 pgs: 97 active+clean; 53 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.7 MiB/s wr, 208 op/s 2024-09-19T06:07:04.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:03 smithi005 ceph-mon[30955]: pgmap v115: 97 pgs: 97 active+clean; 53 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.7 MiB/s wr, 208 op/s 2024-09-19T06:07:04.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:03 smithi049 ceph-mon[34640]: pgmap v115: 97 pgs: 97 active+clean; 53 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.7 MiB/s wr, 208 op/s 2024-09-19T06:07:04.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 78444 2.95 MB/sec execute 30 sec latency 13.122 ms 2024-09-19T06:07:05.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 78743 2.86 MB/sec execute 31 sec latency 12.032 ms 2024-09-19T06:07:06.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:05 smithi157 ceph-mon[35346]: pgmap v116: 97 pgs: 97 active+clean; 53 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-09-19T06:07:06.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:05 smithi005 ceph-mon[30955]: pgmap v116: 97 pgs: 97 active+clean; 53 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-09-19T06:07:06.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:05 smithi049 ceph-mon[34640]: pgmap v116: 97 pgs: 97 active+clean; 53 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-09-19T06:07:06.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 79266 2.83 MB/sec execute 32 sec latency 15.338 ms 2024-09-19T06:07:07.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 79927 2.85 MB/sec execute 33 sec latency 12.901 ms 2024-09-19T06:07:08.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:07 smithi157 ceph-mon[35346]: pgmap v117: 97 pgs: 97 active+clean; 59 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.9 MiB/s wr, 213 op/s 2024-09-19T06:07:08.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:07 smithi005 ceph-mon[30955]: pgmap v117: 97 pgs: 97 active+clean; 59 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.9 MiB/s wr, 213 op/s 2024-09-19T06:07:08.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:07 smithi049 ceph-mon[34640]: pgmap v117: 97 pgs: 97 active+clean; 59 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.9 MiB/s wr, 213 op/s 2024-09-19T06:07:08.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 80494 2.81 MB/sec execute 34 sec latency 15.369 ms 2024-09-19T06:07:09.279 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:07:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:07:09.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 81102 2.89 MB/sec execute 35 sec latency 18.313 ms 2024-09-19T06:07:09.957 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:09 smithi005 ceph-mon[30955]: pgmap v118: 97 pgs: 97 active+clean; 59 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-19T06:07:10.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:09 smithi157 ceph-mon[35346]: pgmap v118: 97 pgs: 97 active+clean; 59 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-19T06:07:10.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:09 smithi049 ceph-mon[34640]: pgmap v118: 97 pgs: 97 active+clean; 59 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-19T06:07:10.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.176+0000 7f2dd7065640 1 -- 172.21.15.5:0/2250116396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd01027a0 msgr2=0x7f2dd01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.176+0000 7f2dd7065640 1 --2- 172.21.15.5:0/2250116396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd01027a0 0x7f2dd01067f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009900 tx=0x7f2dbc02a410 comp rx=0 tx=0).stop 2024-09-19T06:07:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.176+0000 7f2dd7065640 1 -- 172.21.15.5:0/2250116396 shutdown_connections 2024-09-19T06:07:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.176+0000 7f2dd7065640 1 --2- 172.21.15.5:0/2250116396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd0106f20 0x7f2dd010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.176+0000 7f2dd7065640 1 --2- 172.21.15.5:0/2250116396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd01027a0 0x7f2dd01067f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.176+0000 7f2dd7065640 1 --2- 172.21.15.5:0/2250116396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dd0101df0 0x7f2dd01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.176+0000 7f2dd7065640 1 -- 172.21.15.5:0/2250116396 >> 172.21.15.5:0/2250116396 conn(0x7f2dd00fd8f0 msgr2=0x7f2dd00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.176+0000 7f2dd7065640 1 -- 172.21.15.5:0/2250116396 shutdown_connections 2024-09-19T06:07:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.176+0000 7f2dd7065640 1 -- 172.21.15.5:0/2250116396 wait complete. 2024-09-19T06:07:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dd7065640 1 Processor -- start 2024-09-19T06:07:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dd7065640 1 -- start start 2024-09-19T06:07:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dd7065640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dd0101df0 0x7f2dd0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dd7065640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd01027a0 0x7f2dd01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dd7065640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd0106f20 0x7f2dd019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dd7065640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0111df0 con 0x7f2dd0101df0 2024-09-19T06:07:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dd7065640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0111c70 con 0x7f2dd0106f20 2024-09-19T06:07:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dd7065640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0111f70 con 0x7f2dd01027a0 2024-09-19T06:07:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dd4e01640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dd0101df0 0x7f2dd0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dd4e01640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dd0101df0 0x7f2dd0198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49336/0 (socket says 172.21.15.5:49336) 2024-09-19T06:07:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dd4e01640 1 -- 172.21.15.5:0/2381754817 learned_addr learned my addr 172.21.15.5:0/2381754817 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dd5602640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd0106f20 0x7f2dd019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.177+0000 7f2dcffff640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd01027a0 0x7f2dd01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.178+0000 7f2dd4e01640 1 -- 172.21.15.5:0/2381754817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd01027a0 msgr2=0x7f2dd01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.178+0000 7f2dd4e01640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd01027a0 0x7f2dd01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.178+0000 7f2dd4e01640 1 -- 172.21.15.5:0/2381754817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd0106f20 msgr2=0x7f2dd019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.178+0000 7f2dd4e01640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd0106f20 0x7f2dd019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.178+0000 7f2dd4e01640 1 -- 172.21.15.5:0/2381754817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd019d320 con 0x7f2dd0101df0 2024-09-19T06:07:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.178+0000 7f2dd4e01640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dd0101df0 0x7f2dd0198370 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400e9f0 tx=0x7f2dc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.178+0000 7f2dcdffb640 1 -- 172.21.15.5:0/2381754817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400cd80 con 0x7f2dd0101df0 2024-09-19T06:07:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.178+0000 7f2dd7065640 1 -- 172.21.15.5:0/2381754817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd019d610 con 0x7f2dd0101df0 2024-09-19T06:07:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.178+0000 7f2dcdffb640 1 -- 172.21.15.5:0/2381754817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc40108e0 con 0x7f2dd0101df0 2024-09-19T06:07:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.178+0000 7f2dcdffb640 1 -- 172.21.15.5:0/2381754817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4018920 con 0x7f2dd0101df0 2024-09-19T06:07:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.178+0000 7f2dd7065640 1 -- 172.21.15.5:0/2381754817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd01a4ef0 con 0x7f2dd0101df0 2024-09-19T06:07:10.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.181+0000 7f2dcdffb640 1 -- 172.21.15.5:0/2381754817 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2dc4010a90 con 0x7f2dd0101df0 2024-09-19T06:07:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.181+0000 7f2dd943e740 1 Processor -- start 2024-09-19T06:07:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.181+0000 7f2dd943e740 1 -- start start 2024-09-19T06:07:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dcdffb640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2dc00a33b0 0x7f2dc00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dd943e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dad380 0x5633d3dad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dd943e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dadca0 0x5633d3dbc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dd943e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dbca90 0x5633d3dbee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dd943e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633d3d132e0 con 0x5633d3dbca90 2024-09-19T06:07:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dd943e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5633d3cbace0 con 0x5633d3dadca0 2024-09-19T06:07:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dd5602640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2dc00a33b0 0x7f2dc00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dcdffb640 1 -- 172.21.15.5:0/2381754817 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f2dc40cffe0 con 0x7f2dd0101df0 2024-09-19T06:07:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dd943e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5633d3d0f2a0 con 0x5633d3dad380 2024-09-19T06:07:10.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dadca0 0x5633d3dbc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dadca0 0x5633d3dbc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54228/0 (socket says 172.21.15.5:54228) 2024-09-19T06:07:10.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dcffff640 1 -- 172.21.15.5:0/82483426 learned_addr learned my addr 172.21.15.5:0/82483426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:10.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dd5602640 1 --2- 172.21.15.5:0/82483426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dbca90 0x5633d3dbee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dd4e01640 1 --2- 172.21.15.5:0/82483426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dad380 0x5633d3dad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dcffff640 1 -- 172.21.15.5:0/82483426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dad380 msgr2=0x5633d3dad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dcffff640 1 --2- 172.21.15.5:0/82483426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dad380 0x5633d3dad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dcffff640 1 -- 172.21.15.5:0/82483426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dbca90 msgr2=0x5633d3dbee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dcffff640 1 --2- 172.21.15.5:0/82483426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dbca90 0x5633d3dbee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.182+0000 7f2dcffff640 1 -- 172.21.15.5:0/82483426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d3c9f440 con 0x5633d3dadca0 2024-09-19T06:07:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.183+0000 7f2dd5602640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2dc00a33b0 0x7f2dc00a5870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8005fd0 tx=0x7f2dc8009d70 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.183+0000 7f2dcffff640 1 --2- 172.21.15.5:0/82483426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dadca0 0x5633d3dbc550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009900 tx=0x7f2dbc032b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7fee6b2dfbfb6ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.183+0000 7f2dbaffd640 1 -- 172.21.15.5:0/82483426 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc040070 con 0x5633d3dadca0 2024-09-19T06:07:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.183+0000 7f2dbaffd640 1 -- 172.21.15.5:0/82483426 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dbc004030 con 0x5633d3dadca0 2024-09-19T06:07:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.183+0000 7f2dbaffd640 1 -- 172.21.15.5:0/82483426 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc035040 con 0x5633d3dadca0 2024-09-19T06:07:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.184+0000 7f2dd943e740 1 -- 172.21.15.5:0/82483426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dadca0 msgr2=0x5633d3dbc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.184+0000 7f2dd943e740 1 --2- 172.21.15.5:0/82483426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dadca0 0x5633d3dbc550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009900 tx=0x7f2dbc032b00 comp rx=0 tx=0).stop 2024-09-19T06:07:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.184+0000 7f2dd943e740 1 -- 172.21.15.5:0/82483426 shutdown_connections 2024-09-19T06:07:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.184+0000 7f2dd943e740 1 --2- 172.21.15.5:0/82483426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dbca90 0x5633d3dbee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.184+0000 7f2dd943e740 1 --2- 172.21.15.5:0/82483426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dadca0 0x5633d3dbc550 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.184+0000 7f2dd943e740 1 --2- 172.21.15.5:0/82483426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dad380 0x5633d3dad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.184+0000 7f2dd943e740 1 -- 172.21.15.5:0/82483426 >> 172.21.15.5:0/82483426 conn(0x5633d3da9a30 msgr2=0x5633d3da9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.184+0000 7f2dd943e740 1 -- 172.21.15.5:0/82483426 shutdown_connections 2024-09-19T06:07:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.184+0000 7f2dd943e740 1 -- 172.21.15.5:0/82483426 wait complete. 2024-09-19T06:07:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.184+0000 7f2dd943e740 1 -- start start 2024-09-19T06:07:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.184+0000 7f2dd943e740 1 Processor -- start 2024-09-19T06:07:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd943e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dad380 0x5633d3dbc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd943e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dadca0 0x5633d3db96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd943e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dbca90 0x5633d3db9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd943e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633d3dbf660 con 0x5633d3dbca90 2024-09-19T06:07:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd943e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5633d3cce050 con 0x5633d3dad380 2024-09-19T06:07:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd943e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5633d3dbf7e0 con 0x5633d3dadca0 2024-09-19T06:07:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd943e740 10 client.0 _ll_register_callbacks cb 0x5633d3d0e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd5602640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dbca90 0x5633d3db9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd5602640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dbca90 0x5633d3db9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49352/0 (socket says 172.21.15.5:49352) 2024-09-19T06:07:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd5602640 1 -- 172.21.15.5:0/3984894268 learned_addr learned my addr 172.21.15.5:0/3984894268 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dcffff640 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dadca0 0x5633d3db96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd4e01640 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dad380 0x5633d3dbc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd4e01640 1 -- 172.21.15.5:0/3984894268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dadca0 msgr2=0x5633d3db96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:10.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.185+0000 7f2dd4e01640 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dadca0 0x5633d3db96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2dd4e01640 1 -- 172.21.15.5:0/3984894268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dbca90 msgr2=0x5633d3db9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2dd4e01640 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dbca90 0x5633d3db9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2dd4e01640 1 -- 172.21.15.5:0/3984894268 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633d3a843c0 con 0x5633d3dad380 2024-09-19T06:07:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2dd5602640 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dbca90 0x5633d3db9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2dd4e01640 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dad380 0x5633d3dbc3f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40cd740 tx=0x7f2dc4005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f32ffff45bdde5c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2daffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:07:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2daffff640 1 -- 172.21.15.5:0/3984894268 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc409c380 con 0x5633d3dad380 2024-09-19T06:07:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2daffff640 1 -- 172.21.15.5:0/3984894268 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f2dc40989f0 con 0x5633d3dad380 2024-09-19T06:07:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2dd943e740 1 -- 172.21.15.5:0/3984894268 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dbc004030 con 0x5633d3dad380 2024-09-19T06:07:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2daffff640 1 -- 172.21.15.5:0/3984894268 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2dc40a0680 con 0x5633d3dad380 2024-09-19T06:07:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2dd943e740 10 client.25916 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2dd943e740 1 -- 172.21.15.5:0/3984894268 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5633d3dcb110 con 0x5633d3dad380 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2daffff640 1 -- 172.21.15.5:0/3984894268 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dbc004030 con 0x5633d3dad380 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.186+0000 7f2daffff640 1 -- 172.21.15.5:0/3984894268 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f2dc40a4ab0 con 0x5633d3dad380 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.187+0000 7f2dd943e740 10 client.25916 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.187+0000 7f2dd943e740 10 client.25916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.187+0000 7f2dd943e740 20 client.25916 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.187+0000 7f2dd943e740 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5633d3dcc590 0x5633d3dce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.187+0000 7f2dd943e740 4 client.25916 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.187+0000 7f2dd943e740 1 -- 172.21.15.5:0/3984894268 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633d3d20850 con 0x5633d3dcc590 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.187+0000 7f2dcffff640 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5633d3dcc590 0x5633d3dce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.187+0000 7f2dcffff640 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5633d3dcc590 0x5633d3dce9e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87384356e0bb10bf server_cookie=f31ddfa390171b46 in_seq=0 out_seq=0 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.187+0000 7f2daffff640 10 client.25916 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.188+0000 7f2daffff640 1 -- 172.21.15.5:0/3984894268 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633d3d20850 con 0x5633d3dcc590 2024-09-19T06:07:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.188+0000 7f2daffff640 10 client.25916 handle_command_reply: tid=0 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.188+0000 7f2dd943e740 1 client.25916 shutdown 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.188+0000 7f2dd943e740 1 -- 172.21.15.5:0/3984894268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dad380 msgr2=0x5633d3dbc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.188+0000 7f2dd943e740 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dad380 0x5633d3dbc3f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40cd740 tx=0x7f2dc4005f10 comp rx=0 tx=0).stop 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.188+0000 7f2dd943e740 1 -- 172.21.15.5:0/3984894268 shutdown_connections 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.188+0000 7f2dd943e740 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5633d3dcc590 0x5633d3dce9e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.188+0000 7f2dd943e740 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dbca90 0x5633d3db9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.188+0000 7f2dd943e740 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dadca0 0x5633d3db96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.188+0000 7f2daffff640 0 client.25916 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.188+0000 7f2dd943e740 1 --2- 172.21.15.5:0/3984894268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dad380 0x5633d3dbc3f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.188+0000 7f2dd943e740 1 -- 172.21.15.5:0/3984894268 >> 172.21.15.5:0/3984894268 conn(0x5633d3da9a30 msgr2=0x5633d3daf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.189+0000 7f2dd943e740 1 -- 172.21.15.5:0/3984894268 shutdown_connections 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.189+0000 7f2dd943e740 1 -- 172.21.15.5:0/3984894268 wait complete. 2024-09-19T06:07:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.189+0000 7f2dd943e740 20 client.25916 trim_cache size 0 max 16384 2024-09-19T06:07:10.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.208+0000 7f2dd943e740 1 Processor -- start 2024-09-19T06:07:10.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.208+0000 7f2dd943e740 1 -- start start 2024-09-19T06:07:10.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.208+0000 7f2dd943e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dad380 0x5633d3dc8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.208+0000 7f2dd943e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dadca0 0x5633d3dd42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.208+0000 7f2dd943e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dbca90 0x5633d3dd47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.208+0000 7f2dd943e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01ab500 con 0x5633d3dad380 2024-09-19T06:07:10.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.208+0000 7f2dd943e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01ad0a0 con 0x5633d3dbca90 2024-09-19T06:07:10.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.208+0000 7f2dd943e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01abbd0 con 0x5633d3dadca0 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dd4e01640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dad380 0x5633d3dc8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dadca0 0x5633d3dd42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dd5602640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dbca90 0x5633d3dd47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dd5602640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dbca90 0x5633d3dd47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54248/0 (socket says 172.21.15.5:54248) 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dadca0 0x5633d3dd42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:58786/0 (socket says 172.21.15.5:58786) 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dd5602640 1 -- 172.21.15.5:0/3892480809 learned_addr learned my addr 172.21.15.5:0/3892480809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dcffff640 1 -- 172.21.15.5:0/3892480809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dbca90 msgr2=0x5633d3dd47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dcffff640 1 --2- 172.21.15.5:0/3892480809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dbca90 0x5633d3dd47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dcffff640 1 -- 172.21.15.5:0/3892480809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dad380 msgr2=0x5633d3dc8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dcffff640 1 --2- 172.21.15.5:0/3892480809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dad380 0x5633d3dc8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dcffff640 1 -- 172.21.15.5:0/3892480809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d3db7c40 con 0x5633d3dadca0 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dcffff640 1 --2- 172.21.15.5:0/3892480809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dadca0 0x5633d3dd42b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc000c00 tx=0x7f2dbc005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4bb5d2e9f14f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:10.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dba7fc640 1 -- 172.21.15.5:0/3892480809 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc040050 con 0x5633d3dadca0 2024-09-19T06:07:10.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dba7fc640 1 -- 172.21.15.5:0/3892480809 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dbc004040 con 0x5633d3dadca0 2024-09-19T06:07:10.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.209+0000 7f2dba7fc640 1 -- 172.21.15.5:0/3892480809 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc035040 con 0x5633d3dadca0 2024-09-19T06:07:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.210+0000 7f2dd943e740 1 -- 172.21.15.5:0/3892480809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dadca0 msgr2=0x5633d3dd42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.210+0000 7f2dd943e740 1 --2- 172.21.15.5:0/3892480809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dadca0 0x5633d3dd42b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc000c00 tx=0x7f2dbc005fd0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.210+0000 7f2dd943e740 1 -- 172.21.15.5:0/3892480809 shutdown_connections 2024-09-19T06:07:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.210+0000 7f2dd943e740 1 --2- 172.21.15.5:0/3892480809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dbca90 0x5633d3dd47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.210+0000 7f2dd943e740 1 --2- 172.21.15.5:0/3892480809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dadca0 0x5633d3dd42b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.210+0000 7f2dd943e740 1 --2- 172.21.15.5:0/3892480809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dad380 0x5633d3dc8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.210+0000 7f2dd943e740 1 -- 172.21.15.5:0/3892480809 >> 172.21.15.5:0/3892480809 conn(0x5633d3da9a30 msgr2=0x5633d3db4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.210+0000 7f2dd943e740 1 -- 172.21.15.5:0/3892480809 shutdown_connections 2024-09-19T06:07:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.211+0000 7f2dd943e740 1 -- 172.21.15.5:0/3892480809 wait complete. 2024-09-19T06:07:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.211+0000 7f2dd943e740 1 -- start start 2024-09-19T06:07:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.211+0000 7f2dd943e740 1 Processor -- start 2024-09-19T06:07:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.211+0000 7f2dd943e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dad380 0x5633d3db4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.211+0000 7f2dd943e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dadca0 0x5633d3dc4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.211+0000 7f2dd943e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dbca90 0x5633d3dc4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.211+0000 7f2dd943e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01ac9d0 con 0x5633d3dadca0 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.211+0000 7f2dd943e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01ac2a0 con 0x5633d3dad380 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.211+0000 7f2dd943e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01a94e0 con 0x5633d3dbca90 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2dd943e740 10 client.0 _ll_register_callbacks cb 0x7f2dc409bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dadca0 0x5633d3dc4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dadca0 0x5633d3dc4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49372/0 (socket says 172.21.15.5:49372) 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2dcffff640 1 -- 172.21.15.5:0/264211038 learned_addr learned my addr 172.21.15.5:0/264211038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2dd4e01640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dad380 0x5633d3db4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2dd5602640 1 --2- 172.21.15.5:0/264211038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dbca90 0x5633d3dc4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2dcffff640 1 -- 172.21.15.5:0/264211038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dbca90 msgr2=0x5633d3dc4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2dcffff640 1 --2- 172.21.15.5:0/264211038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dbca90 0x5633d3dc4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2dcffff640 1 -- 172.21.15.5:0/264211038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dad380 msgr2=0x5633d3db4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:10.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2dcffff640 1 --2- 172.21.15.5:0/264211038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dad380 0x5633d3db4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2dcffff640 1 -- 172.21.15.5:0/264211038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633d3dc55f0 con 0x5633d3dadca0 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2dcffff640 1 --2- 172.21.15.5:0/264211038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dadca0 0x5633d3dc4a70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc02a930 tx=0x7f2dbc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea0979c08025067 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.212+0000 7f2daffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2daffff640 1 -- 172.21.15.5:0/264211038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc040070 con 0x5633d3dadca0 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2daffff640 1 -- 172.21.15.5:0/264211038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f2dbc004400 con 0x5633d3dadca0 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2dd943e740 1 -- 172.21.15.5:0/264211038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dbc004040 con 0x5633d3dadca0 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2daffff640 1 -- 172.21.15.5:0/264211038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2dbc035040 con 0x5633d3dadca0 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2dd943e740 10 client.16332 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2dd943e740 1 -- 172.21.15.5:0/264211038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5633d3e3ba70 con 0x5633d3dadca0 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2daffff640 1 -- 172.21.15.5:0/264211038 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dbc004040 con 0x5633d3dadca0 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2daffff640 1 -- 172.21.15.5:0/264211038 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f2dbc0054a0 con 0x5633d3dadca0 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2dd943e740 10 client.16332 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:10.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2dd943e740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2dd943e740 20 client.16332 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2dd943e740 1 --2- 172.21.15.5:0/264211038 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5633d3e3bd40 0x5633d3e3e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.213+0000 7f2dd943e740 4 client.16332 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:07:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.214+0000 7f2dd943e740 1 -- 172.21.15.5:0/264211038 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5633d3d1ad60 con 0x5633d3e3bd40 2024-09-19T06:07:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.214+0000 7f2dd4e01640 1 --2- 172.21.15.5:0/264211038 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5633d3e3bd40 0x5633d3e3e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.214+0000 7f2daffff640 10 client.16332 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.214+0000 7f2dd4e01640 1 --2- 172.21.15.5:0/264211038 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5633d3e3bd40 0x5633d3e3e160 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea2624ca33ddc2b4 server_cookie=201a67024a5870ce in_seq=0 out_seq=0 2024-09-19T06:07:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.214+0000 7f2daffff640 1 -- 172.21.15.5:0/264211038 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5633d3d1ad60 con 0x5633d3e3bd40 2024-09-19T06:07:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.214+0000 7f2daffff640 10 client.16332 handle_command_reply: tid=0 2024-09-19T06:07:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.214+0000 7f2dd943e740 1 client.16332 shutdown 2024-09-19T06:07:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.214+0000 7f2dd943e740 1 -- 172.21.15.5:0/264211038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dadca0 msgr2=0x5633d3dc4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.214+0000 7f2dd943e740 1 --2- 172.21.15.5:0/264211038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dadca0 0x5633d3dc4a70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc02a930 tx=0x7f2dbc005020 comp rx=0 tx=0).stop 2024-09-19T06:07:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.215+0000 7f2dd943e740 1 -- 172.21.15.5:0/264211038 shutdown_connections 2024-09-19T06:07:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.215+0000 7f2dd943e740 1 --2- 172.21.15.5:0/264211038 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5633d3e3bd40 0x5633d3e3e160 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.215+0000 7f2daffff640 0 client.16332 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.215+0000 7f2dd943e740 1 --2- 172.21.15.5:0/264211038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633d3dbca90 0x5633d3dc4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.215+0000 7f2dd943e740 1 --2- 172.21.15.5:0/264211038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633d3dadca0 0x5633d3dc4a70 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.215+0000 7f2dd943e740 1 --2- 172.21.15.5:0/264211038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633d3dad380 0x5633d3db4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.215+0000 7f2dd943e740 1 -- 172.21.15.5:0/264211038 >> 172.21.15.5:0/264211038 conn(0x5633d3da9a30 msgr2=0x5633d3daee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.215+0000 7f2dd943e740 1 -- 172.21.15.5:0/264211038 shutdown_connections 2024-09-19T06:07:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.215+0000 7f2dd943e740 1 -- 172.21.15.5:0/264211038 wait complete. 2024-09-19T06:07:10.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.215+0000 7f2dd943e740 20 client.16332 trim_cache size 0 max 16384 2024-09-19T06:07:10.222 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:07:10.222 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:07:10.222 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:07:10.222 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:07:10.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.216+0000 7f2dd7065640 1 -- 172.21.15.5:0/2381754817 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2dc00a33b0 msgr2=0x7f2dc00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.216+0000 7f2dd7065640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2dc00a33b0 0x7f2dc00a5870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8005fd0 tx=0x7f2dc8009d70 comp rx=0 tx=0).stop 2024-09-19T06:07:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.216+0000 7f2dd7065640 1 -- 172.21.15.5:0/2381754817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dd0101df0 msgr2=0x7f2dd0198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.216+0000 7f2dd7065640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dd0101df0 0x7f2dd0198370 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400e9f0 tx=0x7f2dc400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.216+0000 7f2dd7065640 1 -- 172.21.15.5:0/2381754817 shutdown_connections 2024-09-19T06:07:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.216+0000 7f2dd7065640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd0106f20 0x7f2dd019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.216+0000 7f2dd7065640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2dc00a33b0 0x7f2dc00a5870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.216+0000 7f2dd7065640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd01027a0 0x7f2dd01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.216+0000 7f2dd7065640 1 --2- 172.21.15.5:0/2381754817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dd0101df0 0x7f2dd0198370 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.216+0000 7f2dd7065640 1 -- 172.21.15.5:0/2381754817 >> 172.21.15.5:0/2381754817 conn(0x7f2dd00fd8f0 msgr2=0x7f2dd00feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.216+0000 7f2dd7065640 1 -- 172.21.15.5:0/2381754817 shutdown_connections 2024-09-19T06:07:10.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:10.216+0000 7f2dd7065640 1 -- 172.21.15.5:0/2381754817 wait complete. 2024-09-19T06:07:10.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 81623 2.94 MB/sec execute 36 sec latency 27.738 ms 2024-09-19T06:07:11.088 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:07:11.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:07:11.371 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:07:11.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 81997 2.91 MB/sec execute 37 sec latency 47.988 ms 2024-09-19T06:07:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.890+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3177642041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0d4100e20 msgr2=0x7fc0d4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.890+0000 7fc0dc1e5640 1 --2- 172.21.15.5:0/3177642041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0d4100e20 0x7fc0d4101200 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009900 tx=0x7fc0d002a480 comp rx=0 tx=0).stop 2024-09-19T06:07:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.890+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3177642041 shutdown_connections 2024-09-19T06:07:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.890+0000 7fc0dc1e5640 1 --2- 172.21.15.5:0/3177642041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0d4102ef0 0x7fc0d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.890+0000 7fc0dc1e5640 1 --2- 172.21.15.5:0/3177642041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0d4102570 0x7fc0d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.890+0000 7fc0dc1e5640 1 --2- 172.21.15.5:0/3177642041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0d4100e20 0x7fc0d4101200 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.890+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3177642041 >> 172.21.15.5:0/3177642041 conn(0x7fc0d40fc7d0 msgr2=0x7fc0d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.890+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3177642041 shutdown_connections 2024-09-19T06:07:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.890+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3177642041 wait complete. 2024-09-19T06:07:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0dc1e5640 1 Processor -- start 2024-09-19T06:07:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0dc1e5640 1 -- start start 2024-09-19T06:07:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0dc1e5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0d4100e20 0x7fc0d4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0dc1e5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0d4102570 0x7fc0d4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0dc1e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0d4102ef0 0x7fc0d419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0dc1e5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4110e50 con 0x7fc0d4102570 2024-09-19T06:07:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0dc1e5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4110cd0 con 0x7fc0d4100e20 2024-09-19T06:07:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0dc1e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4110fd0 con 0x7fc0d4102ef0 2024-09-19T06:07:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0d9780640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0d4102570 0x7fc0d4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0d9780640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0d4102570 0x7fc0d4198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49388/0 (socket says 172.21.15.5:49388) 2024-09-19T06:07:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0d9780640 1 -- 172.21.15.5:0/3504292756 learned_addr learned my addr 172.21.15.5:0/3504292756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0d9f81640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0d4100e20 0x7fc0d4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.891+0000 7fc0da782640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0d4102ef0 0x7fc0d419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.892+0000 7fc0d9780640 1 -- 172.21.15.5:0/3504292756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0d4102ef0 msgr2=0x7fc0d419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.892+0000 7fc0d9780640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0d4102ef0 0x7fc0d419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.892+0000 7fc0d9780640 1 -- 172.21.15.5:0/3504292756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0d4100e20 msgr2=0x7fc0d4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.892+0000 7fc0d9780640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0d4100e20 0x7fc0d4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.892+0000 7fc0d9780640 1 -- 172.21.15.5:0/3504292756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d419d3c0 con 0x7fc0d4102570 2024-09-19T06:07:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.892+0000 7fc0d9780640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0d4102570 0x7fc0d4198950 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400d950 tx=0x7fc0c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.892+0000 7fc0caffd640 1 -- 172.21.15.5:0/3504292756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4017030 con 0x7fc0d4102570 2024-09-19T06:07:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.892+0000 7fc0caffd640 1 -- 172.21.15.5:0/3504292756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0c4004020 con 0x7fc0d4102570 2024-09-19T06:07:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.892+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3504292756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d419d6b0 con 0x7fc0d4102570 2024-09-19T06:07:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.892+0000 7fc0caffd640 1 -- 172.21.15.5:0/3504292756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40047a0 con 0x7fc0d4102570 2024-09-19T06:07:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.892+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3504292756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d41a4f90 con 0x7fc0d4102570 2024-09-19T06:07:11.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.893+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3504292756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d4101200 con 0x7fc0d4102570 2024-09-19T06:07:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.895+0000 7fc0caffd640 1 -- 172.21.15.5:0/3504292756 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc0c40041d0 con 0x7fc0d4102570 2024-09-19T06:07:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.896+0000 7fc0caffd640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc0c00a33b0 0x7fc0c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.896+0000 7fc0d9f81640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc0c00a33b0 0x7fc0c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.896+0000 7fc0caffd640 1 -- 172.21.15.5:0/3504292756 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fc0c40cfe00 con 0x7fc0d4102570 2024-09-19T06:07:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.896+0000 7fc0d9f81640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc0c00a33b0 0x7fc0c00a5870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0000c00 tx=0x7fc0d0005e80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:11.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:11.899+0000 7fc0caffd640 1 -- 172.21.15.5:0/3504292756 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc0c409c7f0 con 0x7fc0d4102570 2024-09-19T06:07:12.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:11 smithi157 ceph-mon[35346]: pgmap v119: 97 pgs: 97 active+clean; 59 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-19T06:07:12.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:11 smithi005 ceph-mon[30955]: pgmap v119: 97 pgs: 97 active+clean; 59 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-19T06:07:12.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.144+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3504292756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0d4199770 con 0x7fc0d4102570 2024-09-19T06:07:12.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:11 smithi049 ceph-mon[34640]: pgmap v119: 97 pgs: 97 active+clean; 59 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-19T06:07:12.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.145+0000 7fc0caffd640 1 -- 172.21.15.5:0/3504292756 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11962 (secure 0 0 0) 0x7fc0d4199770 con 0x7fc0d4102570 2024-09-19T06:07:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:07:12.147 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"btime":"2024-09-19T06:06:24:465242+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:06:24.465223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:07:12.147 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-09-19T06:07:12.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.147+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3504292756 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc0c00a33b0 msgr2=0x7fc0c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:12.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.147+0000 7fc0dc1e5640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc0c00a33b0 0x7fc0c00a5870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0000c00 tx=0x7fc0d0005e80 comp rx=0 tx=0).stop 2024-09-19T06:07:12.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.147+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3504292756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0d4102570 msgr2=0x7fc0d4198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:12.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.147+0000 7fc0dc1e5640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0d4102570 0x7fc0d4198950 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400d950 tx=0x7fc0c400de10 comp rx=0 tx=0).stop 2024-09-19T06:07:12.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.147+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3504292756 shutdown_connections 2024-09-19T06:07:12.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.147+0000 7fc0dc1e5640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc0c00a33b0 0x7fc0c00a5870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:12.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.147+0000 7fc0dc1e5640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0d4102ef0 0x7fc0d419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:12.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.147+0000 7fc0dc1e5640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0d4102570 0x7fc0d4198950 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:12.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.147+0000 7fc0dc1e5640 1 --2- 172.21.15.5:0/3504292756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0d4100e20 0x7fc0d4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:12.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.147+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3504292756 >> 172.21.15.5:0/3504292756 conn(0x7fc0d40fc7d0 msgr2=0x7fc0d406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:12.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.147+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3504292756 shutdown_connections 2024-09-19T06:07:12.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:12.147+0000 7fc0dc1e5640 1 -- 172.21.15.5:0/3504292756 wait complete. 2024-09-19T06:07:12.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 82299 2.84 MB/sec execute 38 sec latency 39.653 ms 2024-09-19T06:07:13.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:12 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3504292756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:07:13.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:12 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3504292756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:07:13.094 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:07:13.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:12 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3504292756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:07:13.373 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:07:13.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 82823 2.81 MB/sec execute 39 sec latency 15.023 ms 2024-09-19T06:07:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.901+0000 7fe721f40640 1 -- 172.21.15.5:0/3439626075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c106a80 msgr2=0x7fe71c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.901+0000 7fe721f40640 1 --2- 172.21.15.5:0/3439626075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c106a80 0x7fe71c106e60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009930 tx=0x7fe70c02a420 comp rx=0 tx=0).stop 2024-09-19T06:07:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.901+0000 7fe721f40640 1 -- 172.21.15.5:0/3439626075 shutdown_connections 2024-09-19T06:07:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.901+0000 7fe721f40640 1 --2- 172.21.15.5:0/3439626075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe71c101420 0x7fe71c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.901+0000 7fe721f40640 1 --2- 172.21.15.5:0/3439626075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c100a80 0x7fe71c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.901+0000 7fe721f40640 1 --2- 172.21.15.5:0/3439626075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c106a80 0x7fe71c106e60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.901+0000 7fe721f40640 1 -- 172.21.15.5:0/3439626075 >> 172.21.15.5:0/3439626075 conn(0x7fe71c0fc7a0 msgr2=0x7fe71c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.901+0000 7fe721f40640 1 -- 172.21.15.5:0/3439626075 shutdown_connections 2024-09-19T06:07:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.901+0000 7fe721f40640 1 -- 172.21.15.5:0/3439626075 wait complete. 2024-09-19T06:07:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.902+0000 7fe721f40640 1 Processor -- start 2024-09-19T06:07:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.902+0000 7fe721f40640 1 -- start start 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.902+0000 7fe721f40640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c100a80 0x7fe71c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.902+0000 7fe721f40640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c101420 0x7fe71c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.902+0000 7fe721f40640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe71c106a80 0x7fe71c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.902+0000 7fe721f40640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c111e50 con 0x7fe71c101420 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.902+0000 7fe721f40640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c111cd0 con 0x7fe71c106a80 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.902+0000 7fe721f40640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c111fd0 con 0x7fe71c100a80 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.903+0000 7fe71affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c101420 0x7fe71c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.903+0000 7fe71affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c101420 0x7fe71c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49408/0 (socket says 172.21.15.5:49408) 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.903+0000 7fe71b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c100a80 0x7fe71c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.903+0000 7fe71affd640 1 -- 172.21.15.5:0/958797505 learned_addr learned my addr 172.21.15.5:0/958797505 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.903+0000 7fe71bfff640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe71c106a80 0x7fe71c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.903+0000 7fe71affd640 1 -- 172.21.15.5:0/958797505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c100a80 msgr2=0x7fe71c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.903+0000 7fe71affd640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c100a80 0x7fe71c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.903+0000 7fe71affd640 1 -- 172.21.15.5:0/958797505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe71c106a80 msgr2=0x7fe71c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.903+0000 7fe71affd640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe71c106a80 0x7fe71c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.903+0000 7fe71affd640 1 -- 172.21.15.5:0/958797505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c19f5a0 con 0x7fe71c101420 2024-09-19T06:07:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.903+0000 7fe71affd640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c101420 0x7fe71c19ab30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe71000e9f0 tx=0x7fe71000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.903+0000 7fe718ff9640 1 -- 172.21.15.5:0/958797505 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71000cd80 con 0x7fe71c101420 2024-09-19T06:07:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.904+0000 7fe718ff9640 1 -- 172.21.15.5:0/958797505 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7100054d0 con 0x7fe71c101420 2024-09-19T06:07:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.904+0000 7fe718ff9640 1 -- 172.21.15.5:0/958797505 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710010490 con 0x7fe71c101420 2024-09-19T06:07:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.904+0000 7fe721f40640 1 -- 172.21.15.5:0/958797505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c19f890 con 0x7fe71c101420 2024-09-19T06:07:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.904+0000 7fe721f40640 1 -- 172.21.15.5:0/958797505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c1a7170 con 0x7fe71c101420 2024-09-19T06:07:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.907+0000 7fe718ff9640 1 -- 172.21.15.5:0/958797505 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe710005020 con 0x7fe71c101420 2024-09-19T06:07:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.907+0000 7fe724319740 1 Processor -- start 2024-09-19T06:07:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.907+0000 7fe718ff9640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe7040a33b0 0x7fe7040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.907+0000 7fe724319740 1 -- start start 2024-09-19T06:07:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe718ff9640 1 -- 172.21.15.5:0/958797505 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fe7100d0070 con 0x7fe71c101420 2024-09-19T06:07:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71bfff640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe7040a33b0 0x7fe7040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe724319740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2243090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe724319740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d2243a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe724319740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d2250760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe724319740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5632d21a92f0 con 0x5632d224e370 2024-09-19T06:07:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe724319740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5632d2150cf0 con 0x5632d2244b40 2024-09-19T06:07:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe724319740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5632d21a52b0 con 0x5632d22435d0 2024-09-19T06:07:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2243090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2243090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54302/0 (socket says 172.21.15.5:54302) 2024-09-19T06:07:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71b7fe640 1 -- 172.21.15.5:0/1143131698 learned_addr learned my addr 172.21.15.5:0/1143131698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71bfff640 1 --2- 172.21.15.5:0/1143131698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d2250760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71affd640 1 --2- 172.21.15.5:0/1143131698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d2243a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71b7fe640 1 -- 172.21.15.5:0/1143131698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 msgr2=0x5632d2243a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71b7fe640 1 --2- 172.21.15.5:0/1143131698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d2243a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71b7fe640 1 -- 172.21.15.5:0/1143131698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 msgr2=0x5632d2250760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71b7fe640 1 --2- 172.21.15.5:0/1143131698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d2250760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71b7fe640 1 -- 172.21.15.5:0/1143131698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632d21353a0 con 0x5632d2244b40 2024-09-19T06:07:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71affd640 1 --2- 172.21.15.5:0/1143131698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d2243a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:07:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71bfff640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe7040a33b0 0x7fe7040a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe714005fd0 tx=0x7fe714013ae0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:13.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.908+0000 7fe71b7fe640 1 --2- 172.21.15.5:0/1143131698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2243090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe70c02a940 tx=0x7fe70c005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=967af5db6b5cc9d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:13.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.909+0000 7fe701ffb640 1 -- 172.21.15.5:0/1143131698 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c040070 con 0x5632d2244b40 2024-09-19T06:07:13.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.909+0000 7fe701ffb640 1 -- 172.21.15.5:0/1143131698 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe70c004030 con 0x5632d2244b40 2024-09-19T06:07:13.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.909+0000 7fe701ffb640 1 -- 172.21.15.5:0/1143131698 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c035040 con 0x5632d2244b40 2024-09-19T06:07:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.909+0000 7fe724319740 1 -- 172.21.15.5:0/1143131698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 msgr2=0x5632d2243090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.909+0000 7fe724319740 1 --2- 172.21.15.5:0/1143131698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2243090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe70c02a940 tx=0x7fe70c005be0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.910+0000 7fe724319740 1 -- 172.21.15.5:0/1143131698 shutdown_connections 2024-09-19T06:07:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.910+0000 7fe724319740 1 --2- 172.21.15.5:0/1143131698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d2250760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.910+0000 7fe724319740 1 --2- 172.21.15.5:0/1143131698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d2243a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.910+0000 7fe724319740 1 --2- 172.21.15.5:0/1143131698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2243090 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.910+0000 7fe724319740 1 -- 172.21.15.5:0/1143131698 >> 172.21.15.5:0/1143131698 conn(0x5632d220c310 msgr2=0x5632d223fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.910+0000 7fe724319740 1 -- 172.21.15.5:0/1143131698 shutdown_connections 2024-09-19T06:07:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.910+0000 7fe724319740 1 -- 172.21.15.5:0/1143131698 wait complete. 2024-09-19T06:07:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.910+0000 7fe724319740 1 -- start start 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.910+0000 7fe724319740 1 Processor -- start 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe724319740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d22435d0 0x5632d224dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe724319740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d2244b40 0x5632d224afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe724319740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d224e370 0x5632d224b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe724319740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5632d2251010 con 0x5632d22435d0 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe724319740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5632d2164060 con 0x5632d224e370 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe724319740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5632d2251190 con 0x5632d2244b40 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe71b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d22435d0 0x5632d224dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe724319740 10 client.0 _ll_register_callbacks cb 0x5632d1eb78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe71b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d22435d0 0x5632d224dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49426/0 (socket says 172.21.15.5:49426) 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe71b7fe640 1 -- 172.21.15.5:0/3006844134 learned_addr learned my addr 172.21.15.5:0/3006844134 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe71affd640 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d2244b40 0x5632d224afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe71bfff640 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d224e370 0x5632d224b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe71b7fe640 1 -- 172.21.15.5:0/3006844134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d2244b40 msgr2=0x5632d224afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe71b7fe640 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d2244b40 0x5632d224afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe71b7fe640 1 -- 172.21.15.5:0/3006844134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d224e370 msgr2=0x5632d224b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe71b7fe640 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d224e370 0x5632d224b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe71b7fe640 1 -- 172.21.15.5:0/3006844134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632d1f1a3c0 con 0x5632d22435d0 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.911+0000 7fe71bfff640 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d224e370 0x5632d224b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.912+0000 7fe71b7fe640 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d22435d0 0x5632d224dd10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe70c005de0 tx=0x7fe70c009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9cf0f2bead20fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.912+0000 7fe6faffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.912+0000 7fe6faffd640 1 -- 172.21.15.5:0/3006844134 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c040030 con 0x5632d22435d0 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.912+0000 7fe724319740 1 -- 172.21.15.5:0/3006844134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70c004030 con 0x5632d22435d0 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.912+0000 7fe6faffd640 1 -- 172.21.15.5:0/3006844134 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fe70c005230 con 0x5632d22435d0 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.912+0000 7fe6faffd640 1 -- 172.21.15.5:0/3006844134 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe70c035070 con 0x5632d22435d0 2024-09-19T06:07:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.912+0000 7fe724319740 10 client.16350 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.912+0000 7fe724319740 1 -- 172.21.15.5:0/3006844134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5632d225ca60 con 0x5632d22435d0 2024-09-19T06:07:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.912+0000 7fe6faffd640 1 -- 172.21.15.5:0/3006844134 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe70c004030 con 0x5632d22435d0 2024-09-19T06:07:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.913+0000 7fe6faffd640 1 -- 172.21.15.5:0/3006844134 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fe70c004a10 con 0x5632d22435d0 2024-09-19T06:07:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.913+0000 7fe724319740 10 client.16350 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.913+0000 7fe724319740 10 client.16350 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:07:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.913+0000 7fe724319740 20 client.16350 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.913+0000 7fe724319740 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5632d225ddb0 0x5632d22601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.913+0000 7fe724319740 4 client.16350 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.913+0000 7fe724319740 1 -- 172.21.15.5:0/3006844134 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632d21b6860 con 0x5632d225ddb0 2024-09-19T06:07:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.913+0000 7fe71affd640 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5632d225ddb0 0x5632d22601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.913+0000 7fe71affd640 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5632d225ddb0 0x5632d22601a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=91bdf069d2f01dae server_cookie=5c12f860d296d415 in_seq=0 out_seq=0 2024-09-19T06:07:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.913+0000 7fe6faffd640 10 client.16350 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe6faffd640 1 -- 172.21.15.5:0/3006844134 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632d21b6860 con 0x5632d225ddb0 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe6faffd640 10 client.16350 handle_command_reply: tid=0 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe724319740 1 client.16350 shutdown 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe724319740 1 -- 172.21.15.5:0/3006844134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d22435d0 msgr2=0x5632d224dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe724319740 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d22435d0 0x5632d224dd10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe70c005de0 tx=0x7fe70c009590 comp rx=0 tx=0).stop 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe724319740 1 -- 172.21.15.5:0/3006844134 shutdown_connections 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe724319740 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5632d225ddb0 0x5632d22601a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe724319740 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d224e370 0x5632d224b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe724319740 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d2244b40 0x5632d224afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe6faffd640 0 client.16350 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe724319740 1 --2- 172.21.15.5:0/3006844134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d22435d0 0x5632d224dd10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe724319740 1 -- 172.21.15.5:0/3006844134 >> 172.21.15.5:0/3006844134 conn(0x5632d220c310 msgr2=0x5632d2245960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.914+0000 7fe724319740 1 -- 172.21.15.5:0/3006844134 shutdown_connections 2024-09-19T06:07:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.915+0000 7fe724319740 1 -- 172.21.15.5:0/3006844134 wait complete. 2024-09-19T06:07:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.915+0000 7fe724319740 20 client.16350 trim_cache size 0 max 16384 2024-09-19T06:07:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.929+0000 7fe724319740 1 Processor -- start 2024-09-19T06:07:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.929+0000 7fe724319740 1 -- start start 2024-09-19T06:07:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe724319740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d2246120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe724319740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2246660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe724319740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d22631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe724319740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1afa40 con 0x5632d224e370 2024-09-19T06:07:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe724319740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b15e0 con 0x5632d2244b40 2024-09-19T06:07:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe724319740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b0110 con 0x5632d22435d0 2024-09-19T06:07:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d22631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d22631c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49428/0 (socket says 172.21.15.5:49428) 2024-09-19T06:07:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2246660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71bfff640 1 -- 172.21.15.5:0/847181657 learned_addr learned my addr 172.21.15.5:0/847181657 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71b7fe640 1 --2- 172.21.15.5:0/847181657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d2246120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71bfff640 1 -- 172.21.15.5:0/847181657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 msgr2=0x5632d2246120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71bfff640 1 --2- 172.21.15.5:0/847181657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d2246120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71bfff640 1 -- 172.21.15.5:0/847181657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 msgr2=0x5632d2246660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:13.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71bfff640 1 --2- 172.21.15.5:0/847181657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2246660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71bfff640 1 -- 172.21.15.5:0/847181657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632d224b8d0 con 0x5632d224e370 2024-09-19T06:07:13.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71affd640 1 --2- 172.21.15.5:0/847181657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2246660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:13.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71b7fe640 1 --2- 172.21.15.5:0/847181657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d2246120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:13.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.930+0000 7fe71bfff640 1 --2- 172.21.15.5:0/847181657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d22631c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe714003be0 tx=0x7fe714009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30bd89531438e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:13.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.931+0000 7fe7017fa640 1 -- 172.21.15.5:0/847181657 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714004250 con 0x5632d224e370 2024-09-19T06:07:13.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.931+0000 7fe7017fa640 1 -- 172.21.15.5:0/847181657 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe714025070 con 0x5632d224e370 2024-09-19T06:07:13.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.931+0000 7fe7017fa640 1 -- 172.21.15.5:0/847181657 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714010c30 con 0x5632d224e370 2024-09-19T06:07:13.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.931+0000 7fe724319740 1 -- 172.21.15.5:0/847181657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 msgr2=0x5632d22631c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:13.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.931+0000 7fe724319740 1 --2- 172.21.15.5:0/847181657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d22631c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe714003be0 tx=0x7fe714009500 comp rx=0 tx=0).stop 2024-09-19T06:07:13.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.932+0000 7fe724319740 1 -- 172.21.15.5:0/847181657 shutdown_connections 2024-09-19T06:07:13.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.932+0000 7fe724319740 1 --2- 172.21.15.5:0/847181657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d22631c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.932+0000 7fe724319740 1 --2- 172.21.15.5:0/847181657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2246660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.932+0000 7fe724319740 1 --2- 172.21.15.5:0/847181657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d2246120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.932+0000 7fe724319740 1 -- 172.21.15.5:0/847181657 >> 172.21.15.5:0/847181657 conn(0x5632d220c310 msgr2=0x5632d224a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.932+0000 7fe724319740 1 -- 172.21.15.5:0/847181657 shutdown_connections 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.932+0000 7fe724319740 1 -- 172.21.15.5:0/847181657 wait complete. 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.932+0000 7fe724319740 1 -- start start 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.932+0000 7fe724319740 1 Processor -- start 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.932+0000 7fe724319740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d22638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe724319740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2260c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe724319740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d2261150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe724319740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b0f10 con 0x5632d224e370 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe724319740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b07e0 con 0x5632d2244b40 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe724319740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1ada20 con 0x5632d22435d0 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe724319740 10 client.0 _ll_register_callbacks cb 0x7fe70c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2260c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d2261150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d2261150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49434/0 (socket says 172.21.15.5:49434) 2024-09-19T06:07:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71bfff640 1 -- 172.21.15.5:0/3561561997 learned_addr learned my addr 172.21.15.5:0/3561561997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d22638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71bfff640 1 -- 172.21.15.5:0/3561561997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 msgr2=0x5632d22638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71bfff640 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d22638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71bfff640 1 -- 172.21.15.5:0/3561561997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 msgr2=0x5632d2260c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71bfff640 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2260c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71bfff640 1 -- 172.21.15.5:0/3561561997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632d2261930 con 0x5632d224e370 2024-09-19T06:07:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71b7fe640 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d22638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71affd640 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2260c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe71bfff640 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d2261150 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe714012040 tx=0x7fe714004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335c7a0b2b2fa029 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.933+0000 7fe6faffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.934+0000 7fe6faffd640 1 -- 172.21.15.5:0/3561561997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714010a70 con 0x5632d224e370 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.934+0000 7fe724319740 1 -- 172.21.15.5:0/3561561997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe714025070 con 0x5632d224e370 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.934+0000 7fe6faffd640 1 -- 172.21.15.5:0/3561561997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fe714010c10 con 0x5632d224e370 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.934+0000 7fe6faffd640 1 -- 172.21.15.5:0/3561561997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe714010e10 con 0x5632d224e370 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.934+0000 7fe724319740 10 client.16356 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.934+0000 7fe724319740 1 -- 172.21.15.5:0/3561561997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5632d22c86a0 con 0x5632d224e370 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.934+0000 7fe6faffd640 1 -- 172.21.15.5:0/3561561997 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe714025070 con 0x5632d224e370 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.935+0000 7fe6faffd640 1 -- 172.21.15.5:0/3561561997 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fe7140052a0 con 0x5632d224e370 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.935+0000 7fe724319740 10 client.16356 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.935+0000 7fe724319740 10 client.16356 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.935+0000 7fe724319740 20 client.16356 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.935+0000 7fe724319740 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5632d22c8d70 0x5632d22cb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.935+0000 7fe724319740 4 client.16356 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.935+0000 7fe724319740 1 -- 172.21.15.5:0/3561561997 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632d21b0d70 con 0x5632d22c8d70 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.935+0000 7fe71b7fe640 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5632d22c8d70 0x5632d22cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.935+0000 7fe71b7fe640 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5632d22c8d70 0x5632d22cb160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=beb170a3ebf14c4 server_cookie=7f4e69a6ecbcca1a in_seq=0 out_seq=0 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.935+0000 7fe6faffd640 10 client.16356 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.935+0000 7fe6faffd640 1 -- 172.21.15.5:0/3561561997 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632d21b0d70 con 0x5632d22c8d70 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe6faffd640 10 client.16356 handle_command_reply: tid=0 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe724319740 1 client.16356 shutdown 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe724319740 1 -- 172.21.15.5:0/3561561997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 msgr2=0x5632d2261150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe724319740 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d2261150 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe714012040 tx=0x7fe714004c70 comp rx=0 tx=0).stop 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe724319740 1 -- 172.21.15.5:0/3561561997 shutdown_connections 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe724319740 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5632d22c8d70 0x5632d22cb160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe724319740 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632d224e370 0x5632d2261150 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe724319740 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632d2244b40 0x5632d2260c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe724319740 1 --2- 172.21.15.5:0/3561561997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632d22435d0 0x5632d22638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe724319740 1 -- 172.21.15.5:0/3561561997 >> 172.21.15.5:0/3561561997 conn(0x5632d220c310 msgr2=0x5632d224c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe6faffd640 0 client.16356 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:07:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe724319740 1 -- 172.21.15.5:0/3561561997 shutdown_connections 2024-09-19T06:07:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe724319740 1 -- 172.21.15.5:0/3561561997 wait complete. 2024-09-19T06:07:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.936+0000 7fe724319740 20 client.16356 trim_cache size 0 max 16384 2024-09-19T06:07:13.944 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:07:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.937+0000 7fe721f40640 1 -- 172.21.15.5:0/958797505 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe7040a33b0 msgr2=0x7fe7040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.937+0000 7fe721f40640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe7040a33b0 0x7fe7040a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe714005fd0 tx=0x7fe714013ae0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.937+0000 7fe721f40640 1 -- 172.21.15.5:0/958797505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c101420 msgr2=0x7fe71c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.937+0000 7fe721f40640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c101420 0x7fe71c19ab30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe71000e9f0 tx=0x7fe71000edf0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.937+0000 7fe721f40640 1 -- 172.21.15.5:0/958797505 shutdown_connections 2024-09-19T06:07:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.937+0000 7fe721f40640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe71c106a80 0x7fe71c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.937+0000 7fe721f40640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c101420 0x7fe71c19ab30 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.937+0000 7fe721f40640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe7040a33b0 0x7fe7040a5870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.937+0000 7fe721f40640 1 --2- 172.21.15.5:0/958797505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c100a80 0x7fe71c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.937+0000 7fe721f40640 1 -- 172.21.15.5:0/958797505 >> 172.21.15.5:0/958797505 conn(0x7fe71c0fc7a0 msgr2=0x7fe71c10c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.937+0000 7fe721f40640 1 -- 172.21.15.5:0/958797505 shutdown_connections 2024-09-19T06:07:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:13.937+0000 7fe721f40640 1 -- 172.21.15.5:0/958797505 wait complete. 2024-09-19T06:07:14.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:13 smithi157 ceph-mon[35346]: pgmap v120: 97 pgs: 97 active+clean; 60 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 2.0 MiB/s wr, 213 op/s 2024-09-19T06:07:14.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:13 smithi005 ceph-mon[30955]: pgmap v120: 97 pgs: 97 active+clean; 60 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 2.0 MiB/s wr, 213 op/s 2024-09-19T06:07:14.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:13 smithi049 ceph-mon[34640]: pgmap v120: 97 pgs: 97 active+clean; 60 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 2.0 MiB/s wr, 213 op/s 2024-09-19T06:07:14.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:07:14.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 83462 2.83 MB/sec execute 40 sec latency 16.298 ms 2024-09-19T06:07:14.993 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:07:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.536+0000 7f67416ca640 1 -- 172.21.15.5:0/3040843782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673c101de0 msgr2=0x7f673c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.536+0000 7f67416ca640 1 --2- 172.21.15.5:0/3040843782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673c101de0 0x7f673c1021c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f672c009900 tx=0x7f672c02a410 comp rx=0 tx=0).stop 2024-09-19T06:07:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.536+0000 7f67416ca640 1 -- 172.21.15.5:0/3040843782 shutdown_connections 2024-09-19T06:07:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.536+0000 7f67416ca640 1 --2- 172.21.15.5:0/3040843782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c106f10 0x7f673c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.536+0000 7f67416ca640 1 --2- 172.21.15.5:0/3040843782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f673c102790 0x7f673c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.536+0000 7f67416ca640 1 --2- 172.21.15.5:0/3040843782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673c101de0 0x7f673c1021c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.536+0000 7f67416ca640 1 -- 172.21.15.5:0/3040843782 >> 172.21.15.5:0/3040843782 conn(0x7f673c0fd8c0 msgr2=0x7f673c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.536+0000 7f67416ca640 1 -- 172.21.15.5:0/3040843782 shutdown_connections 2024-09-19T06:07:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.536+0000 7f67416ca640 1 -- 172.21.15.5:0/3040843782 wait complete. 2024-09-19T06:07:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f67416ca640 1 Processor -- start 2024-09-19T06:07:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f67416ca640 1 -- start start 2024-09-19T06:07:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f67416ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673c101de0 0x7f673c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f67416ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f673c102790 0x7f673c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f67416ca640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c106f10 0x7f673c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f67416ca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f673c111e20 con 0x7f673c101de0 2024-09-19T06:07:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f67416ca640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f673c111ca0 con 0x7f673c106f10 2024-09-19T06:07:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f67416ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f673c111fa0 con 0x7f673c102790 2024-09-19T06:07:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f673affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673c101de0 0x7f673c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f673affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673c101de0 0x7f673c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36444/0 (socket says 172.21.15.5:36444) 2024-09-19T06:07:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f673b7fe640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c106f10 0x7f673c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f673affd640 1 -- 172.21.15.5:0/759427809 learned_addr learned my addr 172.21.15.5:0/759427809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f6733fff640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f673c102790 0x7f673c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f673b7fe640 1 -- 172.21.15.5:0/759427809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f673c102790 msgr2=0x7f673c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f673b7fe640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f673c102790 0x7f673c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f673b7fe640 1 -- 172.21.15.5:0/759427809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673c101de0 msgr2=0x7f673c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.537+0000 7f673b7fe640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673c101de0 0x7f673c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.538+0000 7f673b7fe640 1 -- 172.21.15.5:0/759427809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c1a67f0 con 0x7f673c106f10 2024-09-19T06:07:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.538+0000 7f673affd640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673c101de0 0x7f673c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.538+0000 7f673b7fe640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c106f10 0x7f673c074f70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6734009fd0 tx=0x7f673400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.538+0000 7f6738ff9640 1 -- 172.21.15.5:0/759427809 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673400ce20 con 0x7f673c106f10 2024-09-19T06:07:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.538+0000 7f67416ca640 1 -- 172.21.15.5:0/759427809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673c1a6ae0 con 0x7f673c106f10 2024-09-19T06:07:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.538+0000 7f67416ca640 1 -- 172.21.15.5:0/759427809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673c1a7020 con 0x7f673c106f10 2024-09-19T06:07:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.538+0000 7f6738ff9640 1 -- 172.21.15.5:0/759427809 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6734005020 con 0x7f673c106f10 2024-09-19T06:07:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.538+0000 7f6738ff9640 1 -- 172.21.15.5:0/759427809 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734010690 con 0x7f673c106f10 2024-09-19T06:07:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.541+0000 7f6738ff9640 1 -- 172.21.15.5:0/759427809 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f67340051d0 con 0x7f673c106f10 2024-09-19T06:07:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.541+0000 7f6743aa3740 1 Processor -- start 2024-09-19T06:07:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.541+0000 7f6743aa3740 1 -- start start 2024-09-19T06:07:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.541+0000 7f6743aa3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd27b40 0x5571edd26090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6743aa3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd265d0 0x5571edd26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6743aa3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd31370 0x5571edd33760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6743aa3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571edc8c2f0 con 0x5571edd27b40 2024-09-19T06:07:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6743aa3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5571edc33cf0 con 0x5571edd265d0 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6743aa3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5571edc882b0 con 0x5571edd31370 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6733fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd27b40 0x5571edd26090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6733fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd31370 0x5571edd33760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6733fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd31370 0x5571edd33760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52334/0 (socket says 172.21.15.5:52334) 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6733fff640 1 -- 172.21.15.5:0/173283141 learned_addr learned my addr 172.21.15.5:0/173283141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f673affd640 1 --2- 172.21.15.5:0/173283141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd265d0 0x5571edd26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6733fff640 1 -- 172.21.15.5:0/173283141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd265d0 msgr2=0x5571edd26a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6733fff640 1 --2- 172.21.15.5:0/173283141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd265d0 0x5571edd26a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6733fff640 1 -- 172.21.15.5:0/173283141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd27b40 msgr2=0x5571edd26090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6733fff640 1 --2- 172.21.15.5:0/173283141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd27b40 0x5571edd26090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6733fff640 1 -- 172.21.15.5:0/173283141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571edc183a0 con 0x5571edd31370 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f673affd640 1 --2- 172.21.15.5:0/173283141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd265d0 0x5571edd26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:07:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.542+0000 7f6738ff9640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67280a33b0 0x7f67280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.543+0000 7f673affd640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67280a33b0 0x7f67280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.543+0000 7f6733fff640 1 --2- 172.21.15.5:0/173283141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd31370 0x5571edd33760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6724014a20 tx=0x7f6724005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9a5d4036570d550 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:15.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.543+0000 7f67317fa640 1 -- 172.21.15.5:0/173283141 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240142d0 con 0x5571edd31370 2024-09-19T06:07:15.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.543+0000 7f67317fa640 1 -- 172.21.15.5:0/173283141 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67240117e0 con 0x5571edd31370 2024-09-19T06:07:15.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.543+0000 7f6738ff9640 1 -- 172.21.15.5:0/759427809 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f67340d0400 con 0x7f673c106f10 2024-09-19T06:07:15.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.543+0000 7f67317fa640 1 -- 172.21.15.5:0/173283141 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724003830 con 0x5571edd31370 2024-09-19T06:07:15.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.544+0000 7f6743aa3740 1 -- 172.21.15.5:0/173283141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd31370 msgr2=0x5571edd33760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.544+0000 7f673affd640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67280a33b0 0x7f67280a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f672c031440 tx=0x7f672c0058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:15.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.544+0000 7f6743aa3740 1 --2- 172.21.15.5:0/173283141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd31370 0x5571edd33760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6724014a20 tx=0x7f6724005850 comp rx=0 tx=0).stop 2024-09-19T06:07:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.544+0000 7f6743aa3740 1 -- 172.21.15.5:0/173283141 shutdown_connections 2024-09-19T06:07:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.544+0000 7f6743aa3740 1 --2- 172.21.15.5:0/173283141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd31370 0x5571edd33760 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.544+0000 7f6743aa3740 1 --2- 172.21.15.5:0/173283141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd265d0 0x5571edd26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.544+0000 7f6743aa3740 1 --2- 172.21.15.5:0/173283141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd27b40 0x5571edd26090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.544+0000 7f6743aa3740 1 -- 172.21.15.5:0/173283141 >> 172.21.15.5:0/173283141 conn(0x5571edcef310 msgr2=0x5571edd22c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.544+0000 7f6743aa3740 1 -- 172.21.15.5:0/173283141 shutdown_connections 2024-09-19T06:07:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.544+0000 7f6743aa3740 1 -- 172.21.15.5:0/173283141 wait complete. 2024-09-19T06:07:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.545+0000 7f6743aa3740 1 -- start start 2024-09-19T06:07:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.545+0000 7f6743aa3740 1 Processor -- start 2024-09-19T06:07:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.545+0000 7f6743aa3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd265d0 0x5571edd30db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.546+0000 7fc12b845640 1 -- 172.21.15.5:0/1934195592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc124102490 msgr2=0x7fc12410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.546+0000 7fc12b845640 1 --2- 172.21.15.5:0/1934195592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc124102490 0x7fc12410e9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009be0 tx=0x7fc11c02a6c0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.545+0000 7f6743aa3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd27b40 0x5571edd2e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.545+0000 7f6743aa3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd31370 0x5571edd2e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.545+0000 7f673affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd265d0 0x5571edd30db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.545+0000 7f6743aa3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571edd34010 con 0x5571edd265d0 2024-09-19T06:07:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.545+0000 7f6743aa3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5571edc47060 con 0x5571edd31370 2024-09-19T06:07:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.545+0000 7f6743aa3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5571edd34190 con 0x5571edd27b40 2024-09-19T06:07:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.545+0000 7f673affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd265d0 0x5571edd30db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36468/0 (socket says 172.21.15.5:36468) 2024-09-19T06:07:15.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.545+0000 7f673affd640 1 -- 172.21.15.5:0/161193846 learned_addr learned my addr 172.21.15.5:0/161193846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:15.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.546+0000 7f6743aa3740 10 client.0 _ll_register_callbacks cb 0x5571ed99a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:15.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.546+0000 7f6733fff640 1 --2- 172.21.15.5:0/161193846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd27b40 0x5571edd2e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.546+0000 7fc12b845640 1 -- 172.21.15.5:0/1934195592 shutdown_connections 2024-09-19T06:07:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.546+0000 7fc12b845640 1 --2- 172.21.15.5:0/1934195592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc124102490 0x7fc12410e9c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.546+0000 7fc12b845640 1 --2- 172.21.15.5:0/1934195592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc124101af0 0x7fc124101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.546+0000 7fc12b845640 1 --2- 172.21.15.5:0/1934195592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc124107b90 0x7fc124107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.546+0000 7fc12b845640 1 -- 172.21.15.5:0/1934195592 >> 172.21.15.5:0/1934195592 conn(0x7fc1240fd810 msgr2=0x7fc1240ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.546+0000 7f673affd640 1 -- 172.21.15.5:0/161193846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd27b40 msgr2=0x5571edd2e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.546+0000 7f673affd640 1 --2- 172.21.15.5:0/161193846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd27b40 0x5571edd2e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.546+0000 7f673affd640 1 -- 172.21.15.5:0/161193846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd31370 msgr2=0x5571edd2e7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:07:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.546+0000 7f673affd640 1 --2- 172.21.15.5:0/161193846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd31370 0x5571edd2e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.546+0000 7f673affd640 1 -- 172.21.15.5:0/161193846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ed9fd3c0 con 0x5571edd265d0 2024-09-19T06:07:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.546+0000 7fc12b845640 1 -- 172.21.15.5:0/1934195592 shutdown_connections 2024-09-19T06:07:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.546+0000 7fc12b845640 1 -- 172.21.15.5:0/1934195592 wait complete. 2024-09-19T06:07:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.546+0000 7f673affd640 1 --2- 172.21.15.5:0/161193846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd265d0 0x5571edd30db0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f672c009a20 tx=0x7f672c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d483be7cd9d9aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.546+0000 7f67227fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:07:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.546+0000 7f67227fc640 1 -- 172.21.15.5:0/161193846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c04c030 con 0x5571edd265d0 2024-09-19T06:07:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f67227fc640 1 -- 172.21.15.5:0/161193846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f672c0035f0 con 0x5571edd265d0 2024-09-19T06:07:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f6743aa3740 1 -- 172.21.15.5:0/161193846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67240117e0 con 0x5571edd265d0 2024-09-19T06:07:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.547+0000 7fc12b845640 1 Processor -- start 2024-09-19T06:07:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f67227fc640 1 -- 172.21.15.5:0/161193846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f672c035040 con 0x5571edd265d0 2024-09-19T06:07:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f6743aa3740 10 client.16371 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f6743aa3740 1 -- 172.21.15.5:0/161193846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5571edd3fa50 con 0x5571edd265d0 2024-09-19T06:07:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.547+0000 7fc12b845640 1 -- start start 2024-09-19T06:07:15.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f67227fc640 1 -- 172.21.15.5:0/161193846 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67240117e0 con 0x5571edd265d0 2024-09-19T06:07:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.547+0000 7fc12b845640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc124101af0 0x7fc124198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f67227fc640 1 -- 172.21.15.5:0/161193846 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f672c05a7e0 con 0x5571edd265d0 2024-09-19T06:07:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.547+0000 7fc1295e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc124101af0 0x7fc124198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.547+0000 7fc1295e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc124101af0 0x7fc124198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36472/0 (socket says 172.21.15.5:36472) 2024-09-19T06:07:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.547+0000 7fc12b845640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc124102490 0x7fc124198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.547+0000 7fc1295e1640 1 -- 172.21.15.5:0/3848075979 learned_addr learned my addr 172.21.15.5:0/3848075979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc12b845640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc124107b90 0x7fc12419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc12b845640 1 -- 172.21.15.5:0/3848075979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc124111db0 con 0x7fc124101af0 2024-09-19T06:07:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc12b845640 1 -- 172.21.15.5:0/3848075979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc124111c30 con 0x7fc124102490 2024-09-19T06:07:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc12b845640 1 -- 172.21.15.5:0/3848075979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc124111f30 con 0x7fc124107b90 2024-09-19T06:07:15.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f6743aa3740 10 client.16371 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:15.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f6743aa3740 10 client.16371 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:07:15.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f6743aa3740 20 client.16371 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:15.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f6743aa3740 1 --2- 172.21.15.5:0/161193846 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5571edd40e60 0x5571edd43320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f6743aa3740 4 client.16371 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:15.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.547+0000 7f6743aa3740 1 -- 172.21.15.5:0/161193846 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571edc99860 con 0x5571edd40e60 2024-09-19T06:07:15.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.548+0000 7f6733fff640 1 --2- 172.21.15.5:0/161193846 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5571edd40e60 0x5571edd43320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc129de2640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc124107b90 0x7fc12419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc128de0640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc124102490 0x7fc124198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc1295e1640 1 -- 172.21.15.5:0/3848075979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc124107b90 msgr2=0x7fc12419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc1295e1640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc124107b90 0x7fc12419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc1295e1640 1 -- 172.21.15.5:0/3848075979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc124102490 msgr2=0x7fc124198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc1295e1640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc124102490 0x7fc124198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc1295e1640 1 -- 172.21.15.5:0/3848075979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12419d330 con 0x7fc124101af0 2024-09-19T06:07:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.548+0000 7f6733fff640 1 --2- 172.21.15.5:0/161193846 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5571edd40e60 0x5571edd43320 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c4b2d0badc831fa9 server_cookie=bf6689914c0c0a14 in_seq=0 out_seq=0 2024-09-19T06:07:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.548+0000 7f67227fc640 10 client.16371 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:07:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc129de2640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc124107b90 0x7fc12419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:07:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc128de0640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc124102490 0x7fc124198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:07:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.548+0000 7fc1295e1640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc124101af0 0x7fc124198340 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc12000e9c0 tx=0x7fc12000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.549+0000 7fc11a7fc640 1 -- 172.21.15.5:0/3848075979 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12000ccc0 con 0x7fc124101af0 2024-09-19T06:07:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.549+0000 7fc11a7fc640 1 -- 172.21.15.5:0/3848075979 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1200054d0 con 0x7fc124101af0 2024-09-19T06:07:15.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f67227fc640 1 -- 172.21.15.5:0/161193846 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571edc99860 con 0x5571edd40e60 2024-09-19T06:07:15.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f67227fc640 10 client.16371 handle_command_reply: tid=0 2024-09-19T06:07:15.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f6743aa3740 1 client.16371 shutdown 2024-09-19T06:07:15.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f6743aa3740 1 -- 172.21.15.5:0/161193846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd265d0 msgr2=0x5571edd30db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f6743aa3740 1 --2- 172.21.15.5:0/161193846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd265d0 0x5571edd30db0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f672c009a20 tx=0x7f672c02a410 comp rx=0 tx=0).stop 2024-09-19T06:07:15.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f6743aa3740 1 -- 172.21.15.5:0/161193846 shutdown_connections 2024-09-19T06:07:15.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f6743aa3740 1 --2- 172.21.15.5:0/161193846 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5571edd40e60 0x5571edd43320 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f6743aa3740 1 --2- 172.21.15.5:0/161193846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd31370 0x5571edd2e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f67227fc640 0 client.16371 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:07:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f6743aa3740 1 --2- 172.21.15.5:0/161193846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd27b40 0x5571edd2e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f6743aa3740 1 --2- 172.21.15.5:0/161193846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd265d0 0x5571edd30db0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f6743aa3740 1 -- 172.21.15.5:0/161193846 >> 172.21.15.5:0/161193846 conn(0x5571edcef310 msgr2=0x5571edd28930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f6743aa3740 1 -- 172.21.15.5:0/161193846 shutdown_connections 2024-09-19T06:07:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f6743aa3740 1 -- 172.21.15.5:0/161193846 wait complete. 2024-09-19T06:07:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.549+0000 7fc11a7fc640 1 -- 172.21.15.5:0/3848075979 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120010430 con 0x7fc124101af0 2024-09-19T06:07:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.549+0000 7fc12b845640 1 -- 172.21.15.5:0/3848075979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12419d620 con 0x7fc124101af0 2024-09-19T06:07:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.550+0000 7fc12b845640 1 -- 172.21.15.5:0/3848075979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1241a4eb0 con 0x7fc124101af0 2024-09-19T06:07:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.549+0000 7f6743aa3740 20 client.16371 trim_cache size 0 max 16384 2024-09-19T06:07:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.552+0000 7fc11a7fc640 1 -- 172.21.15.5:0/3848075979 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc120004f50 con 0x7fc124101af0 2024-09-19T06:07:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.552+0000 7fc12e5da740 1 Processor -- start 2024-09-19T06:07:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.552+0000 7fc12e5da740 1 -- start start 2024-09-19T06:07:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc12e5da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 0x563aab9a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc12e5da740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab99c660 0x563aab99cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc12e5da740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99d000 0x563aab9a89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc12e5da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563aab8f72f0 con 0x563aab9a0cc0 2024-09-19T06:07:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc12e5da740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563aab89f270 con 0x563aab99c660 2024-09-19T06:07:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc12e5da740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563aab8f32b0 con 0x563aab99d000 2024-09-19T06:07:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc129de2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 0x563aab9a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc129de2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 0x563aab9a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36474/0 (socket says 172.21.15.5:36474) 2024-09-19T06:07:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc129de2640 1 -- 172.21.15.5:0/2788447478 learned_addr learned my addr 172.21.15.5:0/2788447478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc128de0640 1 --2- 172.21.15.5:0/2788447478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99d000 0x563aab9a89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc1295e1640 1 --2- 172.21.15.5:0/2788447478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab99c660 0x563aab99cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc11a7fc640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1100a3480 0x7fc1100a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc128de0640 1 -- 172.21.15.5:0/2788447478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab99c660 msgr2=0x563aab99cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc128de0640 1 --2- 172.21.15.5:0/2788447478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab99c660 0x563aab99cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc128de0640 1 -- 172.21.15.5:0/2788447478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 msgr2=0x563aab9a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.553+0000 7fc128de0640 1 --2- 172.21.15.5:0/2788447478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 0x563aab9a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.554+0000 7fc128de0640 1 -- 172.21.15.5:0/2788447478 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aab89e340 con 0x563aab99d000 2024-09-19T06:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.554+0000 7fc129de2640 1 --2- 172.21.15.5:0/2788447478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 0x563aab9a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.554+0000 7fc129de2640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1100a3480 0x7fc1100a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.554+0000 7fc11a7fc640 1 -- 172.21.15.5:0/3848075979 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fc1200cffd0 con 0x7fc124101af0 2024-09-19T06:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.554+0000 7fc128de0640 1 --2- 172.21.15.5:0/2788447478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99d000 0x563aab9a89c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc114009c10 tx=0x7fc114010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=714cd72312501de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.554+0000 7fc10f7fe640 1 -- 172.21.15.5:0/2788447478 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114004030 con 0x563aab99d000 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.554+0000 7fc129de2640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1100a3480 0x7fc1100a5940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc11c004af0 tx=0x7fc11c02a6c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.554+0000 7fc10f7fe640 1 -- 172.21.15.5:0/2788447478 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1140035f0 con 0x563aab99d000 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.554+0000 7fc10f7fe640 1 -- 172.21.15.5:0/2788447478 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140126b0 con 0x563aab99d000 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.555+0000 7fc12e5da740 1 -- 172.21.15.5:0/2788447478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99d000 msgr2=0x563aab9a89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.555+0000 7fc12e5da740 1 --2- 172.21.15.5:0/2788447478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99d000 0x563aab9a89c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc114009c10 tx=0x7fc114010740 comp rx=0 tx=0).stop 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.555+0000 7fc12e5da740 1 -- 172.21.15.5:0/2788447478 shutdown_connections 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.555+0000 7fc12e5da740 1 --2- 172.21.15.5:0/2788447478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99d000 0x563aab9a89c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.555+0000 7fc12e5da740 1 --2- 172.21.15.5:0/2788447478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab99c660 0x563aab99cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.555+0000 7fc12e5da740 1 --2- 172.21.15.5:0/2788447478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 0x563aab9a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.555+0000 7fc12e5da740 1 -- 172.21.15.5:0/2788447478 >> 172.21.15.5:0/2788447478 conn(0x563aab98eb10 msgr2=0x563aab98eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.555+0000 7fc12e5da740 1 -- 172.21.15.5:0/2788447478 shutdown_connections 2024-09-19T06:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.555+0000 7fc12e5da740 1 -- 172.21.15.5:0/2788447478 wait complete. 2024-09-19T06:07:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.555+0000 7fc12e5da740 1 -- start start 2024-09-19T06:07:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc12e5da740 1 Processor -- start 2024-09-19T06:07:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc12e5da740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99c660 0x563aab9a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc12e5da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab99d000 0x563aab9a6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc12e5da740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab9a0cc0 0x563aab9a6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc12e5da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563aab9a90b0 con 0x563aab99d000 2024-09-19T06:07:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc12e5da740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563aab880290 con 0x563aab9a0cc0 2024-09-19T06:07:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc12e5da740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563aab9a9230 con 0x563aab99c660 2024-09-19T06:07:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc12e5da740 10 client.0 _ll_register_callbacks cb 0x563aab8b1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc1295e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99c660 0x563aab9a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc1295e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99c660 0x563aab9a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52378/0 (socket says 172.21.15.5:52378) 2024-09-19T06:07:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc1295e1640 1 -- 172.21.15.5:0/1608208786 learned_addr learned my addr 172.21.15.5:0/1608208786 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc128de0640 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab99d000 0x563aab9a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc129de2640 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab9a0cc0 0x563aab9a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc1295e1640 1 -- 172.21.15.5:0/1608208786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab9a0cc0 msgr2=0x563aab9a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc1295e1640 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab9a0cc0 0x563aab9a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.556+0000 7fc1295e1640 1 -- 172.21.15.5:0/1608208786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab99d000 msgr2=0x563aab9a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc1295e1640 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab99d000 0x563aab9a6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc1295e1640 1 -- 172.21.15.5:0/1608208786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aab60a210 con 0x563aab99c660 2024-09-19T06:07:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc128de0640 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab99d000 0x563aab9a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc1295e1640 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99c660 0x563aab9a5e00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc12009c250 tx=0x7fc120009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a936dd01fcfccaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc10cff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:07:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc10cff9640 1 -- 172.21.15.5:0/1608208786 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12000ce60 con 0x563aab99c660 2024-09-19T06:07:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc10cff9640 1 -- 172.21.15.5:0/1608208786 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fc1200046c0 con 0x563aab99c660 2024-09-19T06:07:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc12e5da740 1 -- 172.21.15.5:0/1608208786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1140035f0 con 0x563aab99c660 2024-09-19T06:07:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc10cff9640 1 -- 172.21.15.5:0/1608208786 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc1200b52a0 con 0x563aab99c660 2024-09-19T06:07:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc12e5da740 10 client.25891 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc12e5da740 1 -- 172.21.15.5:0/1608208786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563aab6d6620 con 0x563aab99c660 2024-09-19T06:07:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc10cff9640 1 -- 172.21.15.5:0/1608208786 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1140035f0 con 0x563aab99c660 2024-09-19T06:07:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.557+0000 7fc10cff9640 1 -- 172.21.15.5:0/1608208786 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fc1200d3cd0 con 0x563aab99c660 2024-09-19T06:07:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.558+0000 7fc12e5da740 10 client.25891 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.558+0000 7fc12e5da740 10 client.25891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.558+0000 7fc12e5da740 20 client.25891 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.558+0000 7fc12e5da740 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563aab9b6450 0x563aab9b8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.558+0000 7fc12e5da740 4 client.25891 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.558+0000 7fc12e5da740 1 -- 172.21.15.5:0/1608208786 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aab904a20 con 0x563aab9b6450 2024-09-19T06:07:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.558+0000 7fc128de0640 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563aab9b6450 0x563aab9b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.558+0000 7fc128de0640 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563aab9b6450 0x563aab9b8860 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1efff4e7f086006 server_cookie=efe33e5ea0a0384 in_seq=0 out_seq=0 2024-09-19T06:07:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.558+0000 7fc10cff9640 10 client.25891 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc10cff9640 1 -- 172.21.15.5:0/1608208786 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563aab904a20 con 0x563aab9b6450 2024-09-19T06:07:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc10cff9640 10 client.25891 handle_command_reply: tid=0 2024-09-19T06:07:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc12e5da740 1 client.25891 shutdown 2024-09-19T06:07:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc12e5da740 1 -- 172.21.15.5:0/1608208786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99c660 msgr2=0x563aab9a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc12e5da740 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99c660 0x563aab9a5e00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc12009c250 tx=0x7fc120009f90 comp rx=0 tx=0).stop 2024-09-19T06:07:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc12e5da740 1 -- 172.21.15.5:0/1608208786 shutdown_connections 2024-09-19T06:07:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc12e5da740 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563aab9b6450 0x563aab9b8860 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc12e5da740 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab9a0cc0 0x563aab9a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc12e5da740 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab99d000 0x563aab9a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc10cff9640 0 client.25891 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc12e5da740 1 --2- 172.21.15.5:0/1608208786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99c660 0x563aab9a5e00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc12e5da740 1 -- 172.21.15.5:0/1608208786 >> 172.21.15.5:0/1608208786 conn(0x563aab98eb10 msgr2=0x563aab99be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc12e5da740 1 -- 172.21.15.5:0/1608208786 shutdown_connections 2024-09-19T06:07:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc12e5da740 1 -- 172.21.15.5:0/1608208786 wait complete. 2024-09-19T06:07:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.559+0000 7fc12e5da740 20 client.25891 trim_cache size 0 max 16384 2024-09-19T06:07:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.568+0000 7f6743aa3740 1 Processor -- start 2024-09-19T06:07:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.568+0000 7f6743aa3740 1 -- start start 2024-09-19T06:07:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6743aa3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd265d0 0x5571edd2d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6743aa3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 0x5571edd21bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6743aa3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd31370 0x5571edd22130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6743aa3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1afad0 con 0x5571edd265d0 2024-09-19T06:07:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6743aa3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1b1670 con 0x5571edd27b40 2024-09-19T06:07:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6743aa3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1b01a0 con 0x5571edd31370 2024-09-19T06:07:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6733fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 0x5571edd21bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f673affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd265d0 0x5571edd2d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6733fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 0x5571edd21bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56336/0 (socket says 172.21.15.5:56336) 2024-09-19T06:07:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6733fff640 1 -- 172.21.15.5:0/2404789517 learned_addr learned my addr 172.21.15.5:0/2404789517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd31370 0x5571edd22130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6733fff640 1 -- 172.21.15.5:0/2404789517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd31370 msgr2=0x5571edd22130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6733fff640 1 --2- 172.21.15.5:0/2404789517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd31370 0x5571edd22130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6733fff640 1 -- 172.21.15.5:0/2404789517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd265d0 msgr2=0x5571edd2d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6733fff640 1 --2- 172.21.15.5:0/2404789517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd265d0 0x5571edd2d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6733fff640 1 -- 172.21.15.5:0/2404789517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571edd2eb20 con 0x5571edd27b40 2024-09-19T06:07:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.569+0000 7f6733fff640 1 --2- 172.21.15.5:0/2404789517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 0x5571edd21bf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67240149c0 tx=0x7f6724005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f304ced4353e5257 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.570+0000 7f6730ff9640 1 -- 172.21.15.5:0/2404789517 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672402e070 con 0x5571edd27b40 2024-09-19T06:07:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.570+0000 7f6730ff9640 1 -- 172.21.15.5:0/2404789517 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6724005a70 con 0x5571edd27b40 2024-09-19T06:07:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.570+0000 7f6730ff9640 1 -- 172.21.15.5:0/2404789517 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240054b0 con 0x5571edd27b40 2024-09-19T06:07:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.571+0000 7f6743aa3740 1 -- 172.21.15.5:0/2404789517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 msgr2=0x5571edd21bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.571+0000 7f6743aa3740 1 --2- 172.21.15.5:0/2404789517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 0x5571edd21bf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67240149c0 tx=0x7f6724005f90 comp rx=0 tx=0).stop 2024-09-19T06:07:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.571+0000 7f6743aa3740 1 -- 172.21.15.5:0/2404789517 shutdown_connections 2024-09-19T06:07:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.571+0000 7f6743aa3740 1 --2- 172.21.15.5:0/2404789517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd31370 0x5571edd22130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.571+0000 7f6743aa3740 1 --2- 172.21.15.5:0/2404789517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 0x5571edd21bf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.571+0000 7f6743aa3740 1 --2- 172.21.15.5:0/2404789517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd265d0 0x5571edd2d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.571+0000 7f6743aa3740 1 -- 172.21.15.5:0/2404789517 >> 172.21.15.5:0/2404789517 conn(0x5571edcef310 msgr2=0x5571edd214e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.571+0000 7f6743aa3740 1 -- 172.21.15.5:0/2404789517 shutdown_connections 2024-09-19T06:07:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.571+0000 7f6743aa3740 1 -- 172.21.15.5:0/2404789517 wait complete. 2024-09-19T06:07:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.572+0000 7f6743aa3740 1 -- start start 2024-09-19T06:07:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.572+0000 7f6743aa3740 1 Processor -- start 2024-09-19T06:07:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.572+0000 7f6743aa3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd265d0 0x5571edd35120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.572+0000 7f6743aa3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 0x5571edd35660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f6743aa3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd31370 0x5571edd35ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f6733fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 0x5571edd35660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f6733fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 0x5571edd35660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56352/0 (socket says 172.21.15.5:56352) 2024-09-19T06:07:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f6733fff640 1 -- 172.21.15.5:0/3252978340 learned_addr learned my addr 172.21.15.5:0/3252978340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f673affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd265d0 0x5571edd35120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f673b7fe640 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd31370 0x5571edd35ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f6743aa3740 1 -- 172.21.15.5:0/3252978340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1b0fa0 con 0x5571edd31370 2024-09-19T06:07:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f6743aa3740 1 -- 172.21.15.5:0/3252978340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1b0870 con 0x5571edd27b40 2024-09-19T06:07:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f6743aa3740 1 -- 172.21.15.5:0/3252978340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1adab0 con 0x5571edd265d0 2024-09-19T06:07:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f6743aa3740 10 client.0 _ll_register_callbacks cb 0x7f672c040050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f673affd640 1 -- 172.21.15.5:0/3252978340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 msgr2=0x5571edd35660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f673affd640 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 0x5571edd35660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f673affd640 1 -- 172.21.15.5:0/3252978340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd31370 msgr2=0x5571edd35ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f673affd640 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd31370 0x5571edd35ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f673affd640 1 -- 172.21.15.5:0/3252978340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571eddaac10 con 0x5571edd265d0 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.573+0000 7f673b7fe640 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd31370 0x5571edd35ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.574+0000 7f673affd640 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd265d0 0x5571edd35120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f672c05ad70 tx=0x7f672c05aa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f050528491757070 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.574+0000 7f67227fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.574+0000 7f67227fc640 1 -- 172.21.15.5:0/3252978340 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c04c040 con 0x5571edd265d0 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.574+0000 7f6743aa3740 1 -- 172.21.15.5:0/3252978340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6724005a70 con 0x5571edd265d0 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.574+0000 7f67227fc640 1 -- 172.21.15.5:0/3252978340 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f672c064890 con 0x5571edd265d0 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.574+0000 7f67227fc640 1 -- 172.21.15.5:0/3252978340 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f672c071050 con 0x5571edd265d0 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.574+0000 7f6743aa3740 10 client.25897 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.574+0000 7f6743aa3740 1 -- 172.21.15.5:0/3252978340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5571eddabe90 con 0x5571edd265d0 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.574+0000 7f67227fc640 1 -- 172.21.15.5:0/3252978340 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6724005a70 con 0x5571edd265d0 2024-09-19T06:07:15.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.574+0000 7f67227fc640 1 -- 172.21.15.5:0/3252978340 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f672c003960 con 0x5571edd265d0 2024-09-19T06:07:15.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.575+0000 7f6743aa3740 10 client.25897 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:15.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.575+0000 7f6743aa3740 10 client.25897 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:07:15.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.575+0000 7f6743aa3740 20 client.25897 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:15.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.575+0000 7f6743aa3740 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5571edd40e60 0x5571eddae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.575+0000 7f6743aa3740 4 client.25897 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:07:15.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.575+0000 7f6743aa3740 1 -- 172.21.15.5:0/3252978340 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571edc93d70 con 0x5571edd40e60 2024-09-19T06:07:15.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.575+0000 7f6733fff640 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5571edd40e60 0x5571eddae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.575+0000 7f6733fff640 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5571edd40e60 0x5571eddae610 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5a394f26e813396b server_cookie=9cd91ff4f2200840 in_seq=0 out_seq=0 2024-09-19T06:07:15.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.575+0000 7f67227fc640 10 client.25897 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:07:15.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.575+0000 7f67227fc640 1 -- 172.21.15.5:0/3252978340 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571edc93d70 con 0x5571edd40e60 2024-09-19T06:07:15.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.575+0000 7f67227fc640 10 client.25897 handle_command_reply: tid=0 2024-09-19T06:07:15.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.575+0000 7f6743aa3740 1 client.25897 shutdown 2024-09-19T06:07:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.576+0000 7f6743aa3740 1 -- 172.21.15.5:0/3252978340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd265d0 msgr2=0x5571edd35120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.576+0000 7f6743aa3740 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd265d0 0x5571edd35120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f672c05ad70 tx=0x7f672c05aa30 comp rx=0 tx=0).stop 2024-09-19T06:07:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.576+0000 7f6743aa3740 1 -- 172.21.15.5:0/3252978340 shutdown_connections 2024-09-19T06:07:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.576+0000 7f6743aa3740 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5571edd40e60 0x5571eddae610 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.576+0000 7f6743aa3740 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571edd31370 0x5571edd35ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.576+0000 7f6743aa3740 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571edd27b40 0x5571edd35660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.576+0000 7f67227fc640 0 client.25897 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:07:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.576+0000 7f6743aa3740 1 --2- 172.21.15.5:0/3252978340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571edd265d0 0x5571edd35120 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.576+0000 7f6743aa3740 1 -- 172.21.15.5:0/3252978340 >> 172.21.15.5:0/3252978340 conn(0x5571edcef310 msgr2=0x5571edd3c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.576+0000 7f6743aa3740 1 -- 172.21.15.5:0/3252978340 shutdown_connections 2024-09-19T06:07:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.576+0000 7f6743aa3740 1 -- 172.21.15.5:0/3252978340 wait complete. 2024-09-19T06:07:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.576+0000 7f6743aa3740 20 client.25897 trim_cache size 0 max 16384 2024-09-19T06:07:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:07:15.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.577+0000 7f67416ca640 1 -- 172.21.15.5:0/759427809 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67280a33b0 msgr2=0x7f67280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.577+0000 7f67416ca640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67280a33b0 0x7f67280a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f672c031440 tx=0x7f672c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.577+0000 7f67416ca640 1 -- 172.21.15.5:0/759427809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c106f10 msgr2=0x7f673c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.577+0000 7f67416ca640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c106f10 0x7f673c074f70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6734009fd0 tx=0x7f673400ef10 comp rx=0 tx=0).stop 2024-09-19T06:07:15.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.577+0000 7f67416ca640 1 -- 172.21.15.5:0/759427809 shutdown_connections 2024-09-19T06:07:15.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.577+0000 7f67416ca640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c106f10 0x7f673c074f70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.577+0000 7f67416ca640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67280a33b0 0x7f67280a5870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.577+0000 7f67416ca640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f673c102790 0x7f673c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.577+0000 7f67416ca640 1 --2- 172.21.15.5:0/759427809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673c101de0 0x7f673c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.577+0000 7f67416ca640 1 -- 172.21.15.5:0/759427809 >> 172.21.15.5:0/759427809 conn(0x7f673c0fd8c0 msgr2=0x7f673c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:15.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.577+0000 7f67416ca640 1 -- 172.21.15.5:0/759427809 shutdown_connections 2024-09-19T06:07:15.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:15.577+0000 7f67416ca640 1 -- 172.21.15.5:0/759427809 wait complete. 2024-09-19T06:07:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc12e5da740 1 Processor -- start 2024-09-19T06:07:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc12e5da740 1 -- start start 2024-09-19T06:07:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc12e5da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab99c660 0x563aab9a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc12e5da740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99d000 0x563aab9a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc12e5da740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab9a0cc0 0x563aab9be170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc12e5da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241ab610 con 0x563aab99c660 2024-09-19T06:07:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc12e5da740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241ad1b0 con 0x563aab9a0cc0 2024-09-19T06:07:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc12e5da740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241abce0 con 0x563aab99d000 2024-09-19T06:07:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc129de2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab9a0cc0 0x563aab9be170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc1295e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab99c660 0x563aab9a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc129de2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab9a0cc0 0x563aab9be170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56364/0 (socket says 172.21.15.5:56364) 2024-09-19T06:07:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc129de2640 1 -- 172.21.15.5:0/2939435488 learned_addr learned my addr 172.21.15.5:0/2939435488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc128de0640 1 --2- 172.21.15.5:0/2939435488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99d000 0x563aab9a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc129de2640 1 -- 172.21.15.5:0/2939435488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99d000 msgr2=0x563aab9a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc129de2640 1 --2- 172.21.15.5:0/2939435488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99d000 0x563aab9a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc129de2640 1 -- 172.21.15.5:0/2939435488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab99c660 msgr2=0x563aab9a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc129de2640 1 --2- 172.21.15.5:0/2939435488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab99c660 0x563aab9a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc129de2640 1 -- 172.21.15.5:0/2939435488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aab9a1410 con 0x563aab9a0cc0 2024-09-19T06:07:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.581+0000 7fc1295e1640 1 --2- 172.21.15.5:0/2939435488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab99c660 0x563aab9a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.582+0000 7fc129de2640 1 --2- 172.21.15.5:0/2939435488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab9a0cc0 0x563aab9be170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009be0 tx=0x7fc11c0303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80575c21cd699f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.582+0000 7fc10effd640 1 -- 172.21.15.5:0/2939435488 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c036040 con 0x563aab9a0cc0 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.582+0000 7fc10effd640 1 -- 172.21.15.5:0/2939435488 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc11c052070 con 0x563aab9a0cc0 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.582+0000 7fc10effd640 1 -- 172.21.15.5:0/2939435488 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c031440 con 0x563aab9a0cc0 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.583+0000 7fc12e5da740 1 -- 172.21.15.5:0/2939435488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab9a0cc0 msgr2=0x563aab9be170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.583+0000 7fc12e5da740 1 --2- 172.21.15.5:0/2939435488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab9a0cc0 0x563aab9be170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009be0 tx=0x7fc11c0303a0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.583+0000 7fc12e5da740 1 -- 172.21.15.5:0/2939435488 shutdown_connections 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.583+0000 7fc12e5da740 1 --2- 172.21.15.5:0/2939435488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab9a0cc0 0x563aab9be170 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.583+0000 7fc12e5da740 1 --2- 172.21.15.5:0/2939435488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99d000 0x563aab9a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.583+0000 7fc12e5da740 1 --2- 172.21.15.5:0/2939435488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab99c660 0x563aab9a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.583+0000 7fc12e5da740 1 -- 172.21.15.5:0/2939435488 >> 172.21.15.5:0/2939435488 conn(0x563aab98eb10 msgr2=0x563aab9a7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.583+0000 7fc12e5da740 1 -- 172.21.15.5:0/2939435488 shutdown_connections 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.583+0000 7fc12e5da740 1 -- 172.21.15.5:0/2939435488 wait complete. 2024-09-19T06:07:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.583+0000 7fc12e5da740 1 -- start start 2024-09-19T06:07:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.583+0000 7fc12e5da740 1 Processor -- start 2024-09-19T06:07:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc12e5da740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99c660 0x563aab9ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc12e5da740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab99d000 0x563aab9aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc12e5da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 0x563aab9a2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc12e5da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241acae0 con 0x563aab9a0cc0 2024-09-19T06:07:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc12e5da740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241ac3b0 con 0x563aab99d000 2024-09-19T06:07:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc12e5da740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241a95f0 con 0x563aab99c660 2024-09-19T06:07:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc12e5da740 10 client.0 _ll_register_callbacks cb 0x7fc120098560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc129de2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 0x563aab9a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc128de0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab99d000 0x563aab9aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc129de2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 0x563aab9a2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36514/0 (socket says 172.21.15.5:36514) 2024-09-19T06:07:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc128de0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab99d000 0x563aab9aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56368/0 (socket says 172.21.15.5:56368) 2024-09-19T06:07:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc129de2640 1 -- 172.21.15.5:0/4111337819 learned_addr learned my addr 172.21.15.5:0/4111337819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc1295e1640 1 --2- 172.21.15.5:0/4111337819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99c660 0x563aab9ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc128de0640 1 -- 172.21.15.5:0/4111337819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99c660 msgr2=0x563aab9ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc128de0640 1 --2- 172.21.15.5:0/4111337819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99c660 0x563aab9ae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc128de0640 1 -- 172.21.15.5:0/4111337819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 msgr2=0x563aab9a2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc128de0640 1 --2- 172.21.15.5:0/4111337819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 0x563aab9a2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.584+0000 7fc128de0640 1 -- 172.21.15.5:0/4111337819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aab9a2590 con 0x563aab99d000 2024-09-19T06:07:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc128de0640 1 --2- 172.21.15.5:0/4111337819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab99d000 0x563aab9aea00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc114010c40 tx=0x7fc1140035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23491cd79ce54449 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc10cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:07:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc10cff9640 1 -- 172.21.15.5:0/4111337819 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11402c070 con 0x563aab99d000 2024-09-19T06:07:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc10cff9640 1 -- 172.21.15.5:0/4111337819 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fc1140050b0 con 0x563aab99d000 2024-09-19T06:07:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc12e5da740 1 -- 172.21.15.5:0/4111337819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc11c052070 con 0x563aab99d000 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc10cff9640 1 -- 172.21.15.5:0/4111337819 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc114009810 con 0x563aab99d000 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc12e5da740 10 client.25973 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc12e5da740 1 -- 172.21.15.5:0/4111337819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563aab9a3810 con 0x563aab99d000 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc10cff9640 1 -- 172.21.15.5:0/4111337819 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc11c052070 con 0x563aab99d000 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc10cff9640 1 -- 172.21.15.5:0/4111337819 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fc11400f040 con 0x563aab99d000 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc12e5da740 10 client.25973 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc12e5da740 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc12e5da740 20 client.25973 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc12e5da740 1 --2- 172.21.15.5:0/4111337819 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563aab9b6450 0x563aaba24160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc12e5da740 4 client.25973 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.585+0000 7fc12e5da740 1 -- 172.21.15.5:0/4111337819 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563aab8fed70 con 0x563aab9b6450 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.586+0000 7fc1295e1640 1 --2- 172.21.15.5:0/4111337819 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563aab9b6450 0x563aaba24160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.586+0000 7fc10cff9640 10 client.25973 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.586+0000 7fc1295e1640 1 --2- 172.21.15.5:0/4111337819 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563aab9b6450 0x563aaba24160 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b5b2a33aef33d28 server_cookie=157286fbce2e507f in_seq=0 out_seq=0 2024-09-19T06:07:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.604+0000 7fc10cff9640 1 -- 172.21.15.5:0/4111337819 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563aab8fed70 con 0x563aab9b6450 2024-09-19T06:07:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.604+0000 7fc10cff9640 10 client.25973 handle_command_reply: tid=0 2024-09-19T06:07:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.604+0000 7fc12e5da740 1 client.25973 shutdown 2024-09-19T06:07:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.604+0000 7fc12e5da740 1 -- 172.21.15.5:0/4111337819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab99d000 msgr2=0x563aab9aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.604+0000 7fc12e5da740 1 --2- 172.21.15.5:0/4111337819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab99d000 0x563aab9aea00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc114010c40 tx=0x7fc1140035f0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.604+0000 7fc12e5da740 1 -- 172.21.15.5:0/4111337819 shutdown_connections 2024-09-19T06:07:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.604+0000 7fc12e5da740 1 --2- 172.21.15.5:0/4111337819 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563aab9b6450 0x563aaba24160 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.604+0000 7fc12e5da740 1 --2- 172.21.15.5:0/4111337819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563aab9a0cc0 0x563aab9a2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.604+0000 7fc12e5da740 1 --2- 172.21.15.5:0/4111337819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aab99d000 0x563aab9aea00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.604+0000 7fc12e5da740 1 --2- 172.21.15.5:0/4111337819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aab99c660 0x563aab9ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.604+0000 7fc12e5da740 1 -- 172.21.15.5:0/4111337819 >> 172.21.15.5:0/4111337819 conn(0x563aab98eb10 msgr2=0x563aab99be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.604+0000 7fc10cff9640 0 client.25973 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.605+0000 7fc12e5da740 1 -- 172.21.15.5:0/4111337819 shutdown_connections 2024-09-19T06:07:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.605+0000 7fc12e5da740 1 -- 172.21.15.5:0/4111337819 wait complete. 2024-09-19T06:07:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.605+0000 7fc12e5da740 20 client.25973 trim_cache size 0 max 16384 2024-09-19T06:07:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:07:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:07:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:07:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:07:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.605+0000 7fc12b845640 1 -- 172.21.15.5:0/3848075979 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1100a3480 msgr2=0x7fc1100a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.605+0000 7fc12b845640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1100a3480 0x7fc1100a5940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc11c004af0 tx=0x7fc11c02a6c0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.606+0000 7fc12b845640 1 -- 172.21.15.5:0/3848075979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc124101af0 msgr2=0x7fc124198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.606+0000 7fc12b845640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc124101af0 0x7fc124198340 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc12000e9c0 tx=0x7fc12000ee80 comp rx=0 tx=0).stop 2024-09-19T06:07:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.606+0000 7fc12b845640 1 -- 172.21.15.5:0/3848075979 shutdown_connections 2024-09-19T06:07:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.606+0000 7fc12b845640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1100a3480 0x7fc1100a5940 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.606+0000 7fc12b845640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc124107b90 0x7fc12419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.606+0000 7fc12b845640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc124102490 0x7fc124198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.607+0000 7fc12b845640 1 --2- 172.21.15.5:0/3848075979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc124101af0 0x7fc124198340 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.607+0000 7fc12b845640 1 -- 172.21.15.5:0/3848075979 >> 172.21.15.5:0/3848075979 conn(0x7fc1240fd810 msgr2=0x7fc1240fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.607+0000 7fc12b845640 1 -- 172.21.15.5:0/3848075979 shutdown_connections 2024-09-19T06:07:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:15.607+0000 7fc12b845640 1 -- 172.21.15.5:0/3848075979 wait complete. 2024-09-19T06:07:15.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 84031 2.80 MB/sec execute 41 sec latency 18.922 ms 2024-09-19T06:07:16.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:15 smithi157 ceph-mon[35346]: pgmap v121: 97 pgs: 97 active+clean; 60 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-19T06:07:16.096 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:15 smithi005 ceph-mon[30955]: pgmap v121: 97 pgs: 97 active+clean; 60 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-19T06:07:16.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:15 smithi049 ceph-mon[34640]: pgmap v121: 97 pgs: 97 active+clean; 60 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-19T06:07:16.534 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:07:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:07:16.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 84653 2.86 MB/sec execute 42 sec latency 15.471 ms 2024-09-19T06:07:17.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.304+0000 7f53db8c9640 1 -- 172.21.15.5:0/212513267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4100e20 msgr2=0x7f53d4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:17.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.304+0000 7f53db8c9640 1 --2- 172.21.15.5:0/212513267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4100e20 0x7f53d4101200 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009900 tx=0x7f53d002a480 comp rx=0 tx=0).stop 2024-09-19T06:07:17.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.304+0000 7f53db8c9640 1 -- 172.21.15.5:0/212513267 shutdown_connections 2024-09-19T06:07:17.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.304+0000 7f53db8c9640 1 --2- 172.21.15.5:0/212513267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4102ef0 0x7f53d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.304+0000 7f53db8c9640 1 --2- 172.21.15.5:0/212513267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d4102570 0x7f53d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.304+0000 7f53db8c9640 1 --2- 172.21.15.5:0/212513267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4100e20 0x7f53d4101200 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.304+0000 7f53db8c9640 1 -- 172.21.15.5:0/212513267 >> 172.21.15.5:0/212513267 conn(0x7f53d40fc7d0 msgr2=0x7f53d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:17.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.304+0000 7f53db8c9640 1 -- 172.21.15.5:0/212513267 shutdown_connections 2024-09-19T06:07:17.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.304+0000 7f53db8c9640 1 -- 172.21.15.5:0/212513267 wait complete. 2024-09-19T06:07:17.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.304+0000 7f53db8c9640 1 Processor -- start 2024-09-19T06:07:17.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.304+0000 7f53db8c9640 1 -- start start 2024-09-19T06:07:17.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53db8c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4100e20 0x7f53d4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53db8c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4102570 0x7f53d4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53db8c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d4102ef0 0x7f53d419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53db8c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4110e30 con 0x7f53d4100e20 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53db8c9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4110cb0 con 0x7f53d4102570 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53db8c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4110fb0 con 0x7f53d4102ef0 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53d9665640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4100e20 0x7f53d4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53d8e64640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4102570 0x7f53d4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53d8e64640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4102570 0x7f53d4198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56384/0 (socket says 172.21.15.5:56384) 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53d9665640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4100e20 0x7f53d4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36536/0 (socket says 172.21.15.5:36536) 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53d8e64640 1 -- 172.21.15.5:0/3826399737 learned_addr learned my addr 172.21.15.5:0/3826399737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53d9e66640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d4102ef0 0x7f53d419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53d8e64640 1 -- 172.21.15.5:0/3826399737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d4102ef0 msgr2=0x7f53d419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53d8e64640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d4102ef0 0x7f53d419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53d8e64640 1 -- 172.21.15.5:0/3826399737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4100e20 msgr2=0x7f53d4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:17.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53d8e64640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4100e20 0x7f53d4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53d8e64640 1 -- 172.21.15.5:0/3826399737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d419d3e0 con 0x7f53d4102570 2024-09-19T06:07:17.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.305+0000 7f53d9665640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4100e20 0x7f53d4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:17.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.306+0000 7f53d8e64640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4102570 0x7f53d4198970 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53c400e9f0 tx=0x7f53c400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:17.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.306+0000 7f53ca7fc640 1 -- 172.21.15.5:0/3826399737 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c400cd80 con 0x7f53d4102570 2024-09-19T06:07:17.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.306+0000 7f53db8c9640 1 -- 172.21.15.5:0/3826399737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d419d6d0 con 0x7f53d4102570 2024-09-19T06:07:17.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.306+0000 7f53ca7fc640 1 -- 172.21.15.5:0/3826399737 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53c40054d0 con 0x7f53d4102570 2024-09-19T06:07:17.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.306+0000 7f53ca7fc640 1 -- 172.21.15.5:0/3826399737 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4010490 con 0x7f53d4102570 2024-09-19T06:07:17.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.306+0000 7f53db8c9640 1 -- 172.21.15.5:0/3826399737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d41a4fb0 con 0x7f53d4102570 2024-09-19T06:07:17.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53ddca2740 1 Processor -- start 2024-09-19T06:07:17.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53ca7fc640 1 -- 172.21.15.5:0/3826399737 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f53c4005020 con 0x7f53d4102570 2024-09-19T06:07:17.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53ddca2740 1 -- start start 2024-09-19T06:07:17.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53ddca2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd5b40 0x55634ffd4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53ddca2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd45d0 0x55634ffd4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53ddca2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffe5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53ddca2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55634ff3a2f0 con 0x55634ffd45d0 2024-09-19T06:07:17.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53ddca2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55634fee1cf0 con 0x55634ffe3780 2024-09-19T06:07:17.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53ddca2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55634ff362b0 con 0x55634ffd5b40 2024-09-19T06:07:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53d9665640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd45d0 0x55634ffd4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53d9e66640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd5b40 0x55634ffd4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53d9665640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd45d0 0x55634ffd4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36548/0 (socket says 172.21.15.5:36548) 2024-09-19T06:07:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53d9665640 1 -- 172.21.15.5:0/4061773187 learned_addr learned my addr 172.21.15.5:0/4061773187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.309+0000 7f53d8e64640 1 --2- 172.21.15.5:0/4061773187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffe5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53d9665640 1 -- 172.21.15.5:0/4061773187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd5b40 msgr2=0x55634ffd4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53d9665640 1 --2- 172.21.15.5:0/4061773187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd5b40 0x55634ffd4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53d9665640 1 -- 172.21.15.5:0/4061773187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 msgr2=0x55634ffe5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53d9665640 1 --2- 172.21.15.5:0/4061773187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffe5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53d9665640 1 -- 172.21.15.5:0/4061773187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55634fec63a0 con 0x55634ffd45d0 2024-09-19T06:07:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53d8e64640 1 --2- 172.21.15.5:0/4061773187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffe5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:07:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53ca7fc640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53c00a33b0 0x7f53c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53d9e66640 1 --2- 172.21.15.5:0/4061773187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd5b40 0x55634ffd4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53d9e66640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53c00a33b0 0x7f53c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53d9665640 1 --2- 172.21.15.5:0/4061773187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd45d0 0x55634ffd4a10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009900 tx=0x7f53d0032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb5d9ccdb053b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53ca7fc640 1 -- 172.21.15.5:0/3826399737 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f53c40d00a0 con 0x7f53d4102570 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53bf7fe640 1 -- 172.21.15.5:0/4061773187 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0035040 con 0x55634ffd45d0 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53bf7fe640 1 -- 172.21.15.5:0/4061773187 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53d00316b0 con 0x55634ffd45d0 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.310+0000 7f53bf7fe640 1 -- 172.21.15.5:0/4061773187 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0003d90 con 0x55634ffd45d0 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.311+0000 7f53ddca2740 1 -- 172.21.15.5:0/4061773187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd45d0 msgr2=0x55634ffd4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.311+0000 7f53d9e66640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53c00a33b0 0x7f53c00a5870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009fd0 tx=0x7f53cc009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.311+0000 7f53ddca2740 1 --2- 172.21.15.5:0/4061773187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd45d0 0x55634ffd4a10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009900 tx=0x7f53d0032e60 comp rx=0 tx=0).stop 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.311+0000 7f53ddca2740 1 -- 172.21.15.5:0/4061773187 shutdown_connections 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.311+0000 7f53ddca2740 1 --2- 172.21.15.5:0/4061773187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffe5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.311+0000 7f53ddca2740 1 --2- 172.21.15.5:0/4061773187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd45d0 0x55634ffd4a10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.311+0000 7f53ddca2740 1 --2- 172.21.15.5:0/4061773187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd5b40 0x55634ffd4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.311+0000 7f53ddca2740 1 -- 172.21.15.5:0/4061773187 >> 172.21.15.5:0/4061773187 conn(0x55634ff9d310 msgr2=0x55634ffd0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.311+0000 7f53ddca2740 1 -- 172.21.15.5:0/4061773187 shutdown_connections 2024-09-19T06:07:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.311+0000 7f53ddca2740 1 -- 172.21.15.5:0/4061773187 wait complete. 2024-09-19T06:07:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.312+0000 7f53ddca2740 1 -- start start 2024-09-19T06:07:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.312+0000 7f53ddca2740 1 Processor -- start 2024-09-19T06:07:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.312+0000 7f53ddca2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634ffe30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.312+0000 7f53ddca2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd5b40 0x55634ffe0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.312+0000 7f53ddca2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffe0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.312+0000 7f53ddca2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55634ffe6420 con 0x55634ffd5b40 2024-09-19T06:07:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.312+0000 7f53ddca2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55634fef5060 con 0x55634ffe3780 2024-09-19T06:07:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.312+0000 7f53ddca2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55634ffe65a0 con 0x55634ffd45d0 2024-09-19T06:07:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.312+0000 7f53ddca2740 10 client.0 _ll_register_callbacks cb 0x55634fc488c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.313+0000 7f53d9e66640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffe0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.313+0000 7f53d9e66640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffe0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56404/0 (socket says 172.21.15.5:56404) 2024-09-19T06:07:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.313+0000 7f53d9e66640 1 -- 172.21.15.5:0/1291581224 learned_addr learned my addr 172.21.15.5:0/1291581224 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.313+0000 7f53d8e64640 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd5b40 0x55634ffe0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.313+0000 7f53d9665640 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634ffe30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.313+0000 7f53d9e66640 1 -- 172.21.15.5:0/1291581224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 msgr2=0x55634ffe30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.313+0000 7f53d9e66640 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634ffe30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.313+0000 7f53d9e66640 1 -- 172.21.15.5:0/1291581224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd5b40 msgr2=0x55634ffe0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.313+0000 7f53d9e66640 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd5b40 0x55634ffe0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.313+0000 7f53d9e66640 1 -- 172.21.15.5:0/1291581224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55634fcab3c0 con 0x55634ffe3780 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.313+0000 7f53d9e66640 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffe0890 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53cc011e40 tx=0x7f53cc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3222d61951a1a699 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.313+0000 7f53bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.314+0000 7f53bcff9640 1 -- 172.21.15.5:0/1291581224 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc004b20 con 0x55634ffe3780 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.314+0000 7f53bcff9640 1 -- 172.21.15.5:0/1291581224 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f53cc004cc0 con 0x55634ffe3780 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.314+0000 7f53ddca2740 1 -- 172.21.15.5:0/1291581224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d00316b0 con 0x55634ffe3780 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.314+0000 7f53bcff9640 1 -- 172.21.15.5:0/1291581224 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53cc01f9b0 con 0x55634ffe3780 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.314+0000 7f53ddca2740 10 client.25985 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.314+0000 7f53ddca2740 1 -- 172.21.15.5:0/1291581224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55634fff1e70 con 0x55634ffe3780 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.314+0000 7f53bcff9640 1 -- 172.21.15.5:0/1291581224 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53d00316b0 con 0x55634ffe3780 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.314+0000 7f53bcff9640 1 -- 172.21.15.5:0/1291581224 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f53cc026070 con 0x55634ffe3780 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.314+0000 7f53ddca2740 10 client.25985 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.314+0000 7f53ddca2740 10 client.25985 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.315+0000 7f53ddca2740 20 client.25985 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.315+0000 7f53ddca2740 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55634fff37a0 0x55634fff5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.315+0000 7f53ddca2740 4 client.25985 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.315+0000 7f53ddca2740 1 -- 172.21.15.5:0/1291581224 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55634ff47860 con 0x55634fff37a0 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.315+0000 7f53d9665640 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55634fff37a0 0x55634fff5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.315+0000 7f53d9665640 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55634fff37a0 0x55634fff5c40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4a578c2c4a42c64 server_cookie=5e05dcd02db1ecd0 in_seq=0 out_seq=0 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.315+0000 7f53bcff9640 10 client.25985 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.316+0000 7f53bcff9640 1 -- 172.21.15.5:0/1291581224 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55634ff47860 con 0x55634fff37a0 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.316+0000 7f53bcff9640 10 client.25985 handle_command_reply: tid=0 2024-09-19T06:07:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.316+0000 7f53ddca2740 1 client.25985 shutdown 2024-09-19T06:07:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.316+0000 7f53ddca2740 1 -- 172.21.15.5:0/1291581224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 msgr2=0x55634ffe0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.316+0000 7f53ddca2740 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffe0890 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53cc011e40 tx=0x7f53cc004ab0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.316+0000 7f53ddca2740 1 -- 172.21.15.5:0/1291581224 shutdown_connections 2024-09-19T06:07:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.316+0000 7f53ddca2740 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55634fff37a0 0x55634fff5c40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.316+0000 7f53ddca2740 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffe0890 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.316+0000 7f53ddca2740 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd5b40 0x55634ffe0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.316+0000 7f53ddca2740 1 --2- 172.21.15.5:0/1291581224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634ffe30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.316+0000 7f53ddca2740 1 -- 172.21.15.5:0/1291581224 >> 172.21.15.5:0/1291581224 conn(0x55634ff9d310 msgr2=0x55634ffd6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.317+0000 7f53bcff9640 0 client.25985 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:07:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.317+0000 7f53ddca2740 1 -- 172.21.15.5:0/1291581224 shutdown_connections 2024-09-19T06:07:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.317+0000 7f53ddca2740 1 -- 172.21.15.5:0/1291581224 wait complete. 2024-09-19T06:07:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.317+0000 7f53ddca2740 20 client.25985 trim_cache size 0 max 16384 2024-09-19T06:07:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.331+0000 7f53ddca2740 1 Processor -- start 2024-09-19T06:07:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.331+0000 7f53ddca2740 1 -- start start 2024-09-19T06:07:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.331+0000 7f53ddca2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634ffef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.331+0000 7f53ddca2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffd5b40 0x55634ffe0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.331+0000 7f53ddca2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffe3780 0x55634ffe0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.331+0000 7f53ddca2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41ab560 con 0x55634ffe3780 2024-09-19T06:07:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.331+0000 7f53ddca2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41ad100 con 0x55634ffd5b40 2024-09-19T06:07:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.331+0000 7f53ddca2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41abc30 con 0x55634ffd45d0 2024-09-19T06:07:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53d9e66640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffe3780 0x55634ffe0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53d8e64640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffd5b40 0x55634ffe0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53d9e66640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffe3780 0x55634ffe0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36562/0 (socket says 172.21.15.5:36562) 2024-09-19T06:07:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53d9e66640 1 -- 172.21.15.5:0/311743318 learned_addr learned my addr 172.21.15.5:0/311743318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53d9665640 1 --2- 172.21.15.5:0/311743318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634ffef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53d8e64640 1 -- 172.21.15.5:0/311743318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 msgr2=0x55634ffef6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53d8e64640 1 --2- 172.21.15.5:0/311743318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634ffef6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53d8e64640 1 -- 172.21.15.5:0/311743318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffe3780 msgr2=0x55634ffe0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53d8e64640 1 --2- 172.21.15.5:0/311743318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffe3780 0x55634ffe0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53d8e64640 1 -- 172.21.15.5:0/311743318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55634ffd0ed0 con 0x55634ffd5b40 2024-09-19T06:07:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53d9e66640 1 --2- 172.21.15.5:0/311743318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffe3780 0x55634ffe0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53d8e64640 1 --2- 172.21.15.5:0/311743318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffd5b40 0x55634ffe0310 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f53c40cd800 tx=0x7f53c40d3700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0eb6cad7cace4e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53beffd640 1 -- 172.21.15.5:0/311743318 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c409bed0 con 0x55634ffd5b40 2024-09-19T06:07:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53beffd640 1 -- 172.21.15.5:0/311743318 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53c4005310 con 0x55634ffd5b40 2024-09-19T06:07:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.332+0000 7f53beffd640 1 -- 172.21.15.5:0/311743318 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c409bcf0 con 0x55634ffd5b40 2024-09-19T06:07:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.333+0000 7f53ddca2740 1 -- 172.21.15.5:0/311743318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffd5b40 msgr2=0x55634ffe0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.333+0000 7f53ddca2740 1 --2- 172.21.15.5:0/311743318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffd5b40 0x55634ffe0310 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f53c40cd800 tx=0x7f53c40d3700 comp rx=0 tx=0).stop 2024-09-19T06:07:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.333+0000 7f53ddca2740 1 -- 172.21.15.5:0/311743318 shutdown_connections 2024-09-19T06:07:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.333+0000 7f53ddca2740 1 --2- 172.21.15.5:0/311743318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffe3780 0x55634ffe0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.333+0000 7f53ddca2740 1 --2- 172.21.15.5:0/311743318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffd5b40 0x55634ffe0310 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.333+0000 7f53ddca2740 1 --2- 172.21.15.5:0/311743318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634ffef6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.333+0000 7f53ddca2740 1 -- 172.21.15.5:0/311743318 >> 172.21.15.5:0/311743318 conn(0x55634ff9d310 msgr2=0x55634ffdb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.333+0000 7f53ddca2740 1 -- 172.21.15.5:0/311743318 shutdown_connections 2024-09-19T06:07:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.333+0000 7f53ddca2740 1 -- 172.21.15.5:0/311743318 wait complete. 2024-09-19T06:07:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53ddca2740 1 -- start start 2024-09-19T06:07:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53ddca2740 1 Processor -- start 2024-09-19T06:07:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53ddca2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634fffab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53ddca2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd5b40 0x55634fffb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53ddca2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffea860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53ddca2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41aca30 con 0x55634ffd5b40 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53ddca2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41ac300 con 0x55634ffe3780 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53ddca2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41a9560 con 0x55634ffd45d0 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53ddca2740 10 client.0 _ll_register_callbacks cb 0x7f53cc009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53d9e66640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53d9e66640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffea860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56412/0 (socket says 172.21.15.5:56412) 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53d8e64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd5b40 0x55634fffb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.334+0000 7f53d9e66640 1 -- 172.21.15.5:0/1111662586 learned_addr learned my addr 172.21.15.5:0/1111662586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53d9665640 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634fffab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53d9e66640 1 -- 172.21.15.5:0/1111662586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 msgr2=0x55634fffab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53d9e66640 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634fffab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53d9e66640 1 -- 172.21.15.5:0/1111662586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd5b40 msgr2=0x55634fffb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53d9e66640 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd5b40 0x55634fffb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53d9e66640 1 -- 172.21.15.5:0/1111662586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55634ffeae20 con 0x55634ffe3780 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53d9665640 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634fffab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53d8e64640 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd5b40 0x55634fffb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53d9e66640 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffea860 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f53cc018040 tx=0x7f53cc009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fd00b7267ffab5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53bcff9640 1 -- 172.21.15.5:0/1111662586 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc004b20 con 0x55634ffe3780 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53ddca2740 1 -- 172.21.15.5:0/1111662586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55634ff41d70 con 0x55634ffe3780 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53bcff9640 1 -- 172.21.15.5:0/1111662586 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f53cc004cc0 con 0x55634ffe3780 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53bcff9640 1 -- 172.21.15.5:0/1111662586 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53cc003970 con 0x55634ffe3780 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53ddca2740 10 client.25991 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.335+0000 7f53ddca2740 1 -- 172.21.15.5:0/1111662586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55634ffec0c0 con 0x55634ffe3780 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.336+0000 7f53bcff9640 1 -- 172.21.15.5:0/1111662586 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55634ff41d70 con 0x55634ffe3780 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.336+0000 7f53bcff9640 1 -- 172.21.15.5:0/1111662586 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f53cc026040 con 0x55634ffe3780 2024-09-19T06:07:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.336+0000 7f53ddca2740 10 client.25991 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.336+0000 7f53ddca2740 10 client.25991 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.336+0000 7f53ddca2740 20 client.25991 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.336+0000 7f53ddca2740 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55635005e680 0x556350060aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.336+0000 7f53ddca2740 4 client.25991 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.336+0000 7f53ddca2740 1 -- 172.21.15.5:0/1111662586 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55634febe170 con 0x55635005e680 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.336+0000 7f53d9665640 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55635005e680 0x556350060aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53d9665640 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55635005e680 0x556350060aa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4125f571c19350bb server_cookie=b4da6d608d1ec618 in_seq=0 out_seq=0 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53bcff9640 10 client.25991 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53bcff9640 1 -- 172.21.15.5:0/1111662586 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55634febe170 con 0x55635005e680 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53bcff9640 10 client.25991 handle_command_reply: tid=0 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53ddca2740 1 client.25991 shutdown 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53ddca2740 1 -- 172.21.15.5:0/1111662586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 msgr2=0x55634ffea860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53ddca2740 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffea860 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f53cc018040 tx=0x7f53cc009600 comp rx=0 tx=0).stop 2024-09-19T06:07:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53ddca2740 1 -- 172.21.15.5:0/1111662586 shutdown_connections 2024-09-19T06:07:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53ddca2740 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55635005e680 0x556350060aa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53ddca2740 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634ffe3780 0x55634ffea860 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53ddca2740 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634ffd5b40 0x55634fffb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53bcff9640 0 client.25991 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:07:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53ddca2740 1 --2- 172.21.15.5:0/1111662586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634ffd45d0 0x55634fffab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53ddca2740 1 -- 172.21.15.5:0/1111662586 >> 172.21.15.5:0/1111662586 conn(0x55634ff9d310 msgr2=0x55634ffe02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53ddca2740 1 -- 172.21.15.5:0/1111662586 shutdown_connections 2024-09-19T06:07:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53ddca2740 1 -- 172.21.15.5:0/1111662586 wait complete. 2024-09-19T06:07:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.337+0000 7f53ddca2740 20 client.25991 trim_cache size 0 max 16384 2024-09-19T06:07:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:07:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.338+0000 7f53db8c9640 1 -- 172.21.15.5:0/3826399737 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53c00a33b0 msgr2=0x7f53c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.338+0000 7f53db8c9640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53c00a33b0 0x7f53c00a5870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009fd0 tx=0x7f53cc009500 comp rx=0 tx=0).stop 2024-09-19T06:07:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.338+0000 7f53db8c9640 1 -- 172.21.15.5:0/3826399737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4102570 msgr2=0x7f53d4198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.338+0000 7f53db8c9640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4102570 0x7f53d4198970 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53c400e9f0 tx=0x7f53c400edf0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.338+0000 7f53db8c9640 1 -- 172.21.15.5:0/3826399737 shutdown_connections 2024-09-19T06:07:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.338+0000 7f53db8c9640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53c00a33b0 0x7f53c00a5870 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.338+0000 7f53db8c9640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d4102ef0 0x7f53d419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.338+0000 7f53db8c9640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4102570 0x7f53d4198970 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.338+0000 7f53db8c9640 1 --2- 172.21.15.5:0/3826399737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4100e20 0x7f53d4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.338+0000 7f53db8c9640 1 -- 172.21.15.5:0/3826399737 >> 172.21.15.5:0/3826399737 conn(0x7f53d40fc7d0 msgr2=0x7f53d406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.338+0000 7f53db8c9640 1 -- 172.21.15.5:0/3826399737 shutdown_connections 2024-09-19T06:07:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:17.338+0000 7f53db8c9640 1 -- 172.21.15.5:0/3826399737 wait complete. 2024-09-19T06:07:17.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 85179 2.91 MB/sec execute 43 sec latency 66.816 ms 2024-09-19T06:07:18.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:17 smithi157 ceph-mon[35346]: pgmap v122: 97 pgs: 97 active+clean; 58 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 2.1 MiB/s wr, 198 op/s 2024-09-19T06:07:18.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:17 smithi005 ceph-mon[30955]: pgmap v122: 97 pgs: 97 active+clean; 58 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 2.1 MiB/s wr, 198 op/s 2024-09-19T06:07:18.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:17 smithi049 ceph-mon[34640]: pgmap v122: 97 pgs: 97 active+clean; 58 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 2.1 MiB/s wr, 198 op/s 2024-09-19T06:07:18.530 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:07:18.813 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:07:18.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 85555 2.88 MB/sec execute 44 sec latency 12.170 ms 2024-09-19T06:07:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.310+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/2487822086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1cc101dd0 msgr2=0x7fd1cc1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.310+0000 7fd1d2c5d640 1 --2- 172.21.15.5:0/2487822086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1cc101dd0 0x7fd1cc1021b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009ca0 tx=0x7fd1bc02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.310+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/2487822086 shutdown_connections 2024-09-19T06:07:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.310+0000 7fd1d2c5d640 1 --2- 172.21.15.5:0/2487822086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1cc106f00 0x7fd1cc10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.310+0000 7fd1d2c5d640 1 --2- 172.21.15.5:0/2487822086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1cc102780 0x7fd1cc1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.310+0000 7fd1d2c5d640 1 --2- 172.21.15.5:0/2487822086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1cc101dd0 0x7fd1cc1021b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.310+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/2487822086 >> 172.21.15.5:0/2487822086 conn(0x7fd1cc0fd850 msgr2=0x7fd1cc0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.311+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/2487822086 shutdown_connections 2024-09-19T06:07:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.311+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/2487822086 wait complete. 2024-09-19T06:07:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.311+0000 7fd1d2c5d640 1 Processor -- start 2024-09-19T06:07:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.311+0000 7fd1d2c5d640 1 -- start start 2024-09-19T06:07:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d2c5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1cc101dd0 0x7fd1cc19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d2c5d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1cc102780 0x7fd1cc19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d2c5d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1cc106f00 0x7fd1cc19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d2c5d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc111e10 con 0x7fd1cc101dd0 2024-09-19T06:07:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d2c5d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc111c90 con 0x7fd1cc102780 2024-09-19T06:07:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d2c5d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc111f90 con 0x7fd1cc106f00 2024-09-19T06:07:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d09f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1cc101dd0 0x7fd1cc19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d09f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1cc101dd0 0x7fd1cc19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36576/0 (socket says 172.21.15.5:36576) 2024-09-19T06:07:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d09f9640 1 -- 172.21.15.5:0/1693689863 learned_addr learned my addr 172.21.15.5:0/1693689863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1cbfff640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1cc102780 0x7fd1cc19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d11fa640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1cc106f00 0x7fd1cc19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d09f9640 1 -- 172.21.15.5:0/1693689863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1cc106f00 msgr2=0x7fd1cc19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1cc106f00 0x7fd1cc19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d09f9640 1 -- 172.21.15.5:0/1693689863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1cc102780 msgr2=0x7fd1cc19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1cc102780 0x7fd1cc19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d09f9640 1 -- 172.21.15.5:0/1693689863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc19f580 con 0x7fd1cc101dd0 2024-09-19T06:07:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1cbfff640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1cc102780 0x7fd1cc19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.312+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1cc101dd0 0x7fd1cc19a5d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc02ade0 tx=0x7fd1bc032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.313+0000 7fd1c9ffb640 1 -- 172.21.15.5:0/1693689863 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc003b70 con 0x7fd1cc101dd0 2024-09-19T06:07:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.313+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/1693689863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc19f810 con 0x7fd1cc101dd0 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.313+0000 7fd1c9ffb640 1 -- 172.21.15.5:0/1693689863 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1bc003d10 con 0x7fd1cc101dd0 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.313+0000 7fd1c9ffb640 1 -- 172.21.15.5:0/1693689863 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc030b80 con 0x7fd1cc101dd0 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.313+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/1693689863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc1a7150 con 0x7fd1cc101dd0 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1c9ffb640 1 -- 172.21.15.5:0/1693689863 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd1bc004040 con 0x7fd1cc101dd0 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1d5036740 1 Processor -- start 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1d5036740 1 -- start start 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1d5036740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fa6b40 0x5573c1fa5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1d5036740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fa5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1d5036740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fb0370 0x5573c1fb2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1d5036740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573c1f0b2f0 con 0x5573c1fb0370 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1d5036740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5573c1eb2cf0 con 0x5573c1fa55d0 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1d5036740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5573c1f072b0 con 0x5573c1fa6b40 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fb0370 0x5573c1fb2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fb0370 0x5573c1fb2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36584/0 (socket says 172.21.15.5:36584) 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1cbfff640 1 -- 172.21.15.5:0/3782795349 learned_addr learned my addr 172.21.15.5:0/3782795349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/3782795349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fa5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.316+0000 7fd1d11fa640 1 --2- 172.21.15.5:0/3782795349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fa6b40 0x5573c1fa5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.317+0000 7fd1cbfff640 1 -- 172.21.15.5:0/3782795349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fa6b40 msgr2=0x5573c1fa5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.317+0000 7fd1c9ffb640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1b40a33b0 0x7fd1b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.317+0000 7fd1cbfff640 1 --2- 172.21.15.5:0/3782795349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fa6b40 0x5573c1fa5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.317+0000 7fd1cbfff640 1 -- 172.21.15.5:0/3782795349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 msgr2=0x5573c1fa5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.317+0000 7fd1cbfff640 1 --2- 172.21.15.5:0/3782795349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fa5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.317+0000 7fd1cbfff640 1 -- 172.21.15.5:0/3782795349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c1e973a0 con 0x5573c1fb0370 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.317+0000 7fd1d11fa640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1b40a33b0 0x7fd1b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.317+0000 7fd1c9ffb640 1 -- 172.21.15.5:0/1693689863 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fd1bc0f0f40 con 0x7fd1cc101dd0 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.317+0000 7fd1cbfff640 1 --2- 172.21.15.5:0/3782795349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fb0370 0x5573c1fb2760 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00114f0 tx=0x7fd1c00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424413d2fdd3fe90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.317+0000 7fd1b2ffd640 1 -- 172.21.15.5:0/3782795349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c001e070 con 0x5573c1fb0370 2024-09-19T06:07:19.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.317+0000 7fd1b2ffd640 1 -- 172.21.15.5:0/3782795349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1c00035f0 con 0x5573c1fb0370 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.317+0000 7fd1b2ffd640 1 -- 172.21.15.5:0/3782795349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c000f7c0 con 0x5573c1fb0370 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.318+0000 7fd1d5036740 1 -- 172.21.15.5:0/3782795349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fb0370 msgr2=0x5573c1fb2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.318+0000 7fd1d11fa640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1b40a33b0 0x7fd1b40a5870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40058c0 tx=0x7fd1c40046c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.318+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3782795349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fb0370 0x5573c1fb2760 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00114f0 tx=0x7fd1c00119b0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.318+0000 7fd1d5036740 1 -- 172.21.15.5:0/3782795349 shutdown_connections 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.318+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3782795349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fb0370 0x5573c1fb2760 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.318+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3782795349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fa5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.318+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3782795349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fa6b40 0x5573c1fa5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.318+0000 7fd1d5036740 1 -- 172.21.15.5:0/3782795349 >> 172.21.15.5:0/3782795349 conn(0x5573c1f6e310 msgr2=0x5573c1fa1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.318+0000 7fd1d5036740 1 -- 172.21.15.5:0/3782795349 shutdown_connections 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.318+0000 7fd1d5036740 1 -- 172.21.15.5:0/3782795349 wait complete. 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.319+0000 7fd1d5036740 1 -- start start 2024-09-19T06:07:19.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.319+0000 7fd1d5036740 1 Processor -- start 2024-09-19T06:07:19.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.319+0000 7fd1d5036740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.319+0000 7fd1d5036740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fa6b40 0x5573c1fad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.319+0000 7fd1d5036740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fadcb0 0x5573c1fbc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.319+0000 7fd1d5036740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573c1fb3070 con 0x5573c1fadcb0 2024-09-19T06:07:19.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.319+0000 7fd1d5036740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5573c1ec6060 con 0x5573c1fa55d0 2024-09-19T06:07:19.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.319+0000 7fd1d5036740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5573c1fb31f0 con 0x5573c1fa6b40 2024-09-19T06:07:19.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.319+0000 7fd1d5036740 10 client.0 _ll_register_callbacks cb 0x5573c1c198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:19.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1d11fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fadcb0 0x5573c1fbc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1d09f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1d11fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fadcb0 0x5573c1fbc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36594/0 (socket says 172.21.15.5:36594) 2024-09-19T06:07:19.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1d11fa640 1 -- 172.21.15.5:0/3031719279 learned_addr learned my addr 172.21.15.5:0/3031719279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:19.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1cbfff640 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fa6b40 0x5573c1fad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1d09f9640 1 -- 172.21.15.5:0/3031719279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fa6b40 msgr2=0x5573c1fad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:19.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fa6b40 0x5573c1fad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1d09f9640 1 -- 172.21.15.5:0/3031719279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fadcb0 msgr2=0x5573c1fbc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:19.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fadcb0 0x5573c1fbc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1d09f9640 1 -- 172.21.15.5:0/3031719279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c1c7c3c0 con 0x5573c1fa55d0 2024-09-19T06:07:19.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1d11fa640 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fadcb0 0x5573c1fbc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fad230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0ee7b0 tx=0x7fd1bc0f4680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20cebef7c77c7efe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.320+0000 7fd1abfff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:07:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1abfff640 1 -- 172.21.15.5:0/3031719279 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0bd1a0 con 0x5573c1fa55d0 2024-09-19T06:07:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1abfff640 1 -- 172.21.15.5:0/3031719279 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fd1bc0ee920 con 0x5573c1fa55d0 2024-09-19T06:07:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1d5036740 1 -- 172.21.15.5:0/3031719279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c00035f0 con 0x5573c1fa55d0 2024-09-19T06:07:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1abfff640 1 -- 172.21.15.5:0/3031719279 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd1bc0bcd30 con 0x5573c1fa55d0 2024-09-19T06:07:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1d5036740 10 client.26003 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1d5036740 1 -- 172.21.15.5:0/3031719279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573c1fb3070 con 0x5573c1fa55d0 2024-09-19T06:07:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1abfff640 1 -- 172.21.15.5:0/3031719279 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1c00035f0 con 0x5573c1fa55d0 2024-09-19T06:07:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1abfff640 1 -- 172.21.15.5:0/3031719279 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fd1bc0b9a40 con 0x5573c1fa55d0 2024-09-19T06:07:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1d5036740 10 client.26003 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1d5036740 10 client.26003 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:07:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1d5036740 20 client.26003 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5573c1fc05d0 0x5573c1fc2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1d5036740 4 client.26003 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.321+0000 7fd1d5036740 1 -- 172.21.15.5:0/3031719279 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573c1f18860 con 0x5573c1fc05d0 2024-09-19T06:07:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.322+0000 7fd1cbfff640 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5573c1fc05d0 0x5573c1fc2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.322+0000 7fd1cbfff640 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5573c1fc05d0 0x5573c1fc2a20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50b7f33f76a72840 server_cookie=1411a489544c427c in_seq=0 out_seq=0 2024-09-19T06:07:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.322+0000 7fd1abfff640 10 client.26003 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:07:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1abfff640 1 -- 172.21.15.5:0/3031719279 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573c1f18860 con 0x5573c1fc05d0 2024-09-19T06:07:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1abfff640 10 client.26003 handle_command_reply: tid=0 2024-09-19T06:07:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1d5036740 1 client.26003 shutdown 2024-09-19T06:07:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1d5036740 1 -- 172.21.15.5:0/3031719279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 msgr2=0x5573c1fad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fad230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0ee7b0 tx=0x7fd1bc0f4680 comp rx=0 tx=0).stop 2024-09-19T06:07:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1d5036740 1 -- 172.21.15.5:0/3031719279 shutdown_connections 2024-09-19T06:07:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5573c1fc05d0 0x5573c1fc2a20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fadcb0 0x5573c1fbc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fa6b40 0x5573c1fad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3031719279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fad230 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1abfff640 0 client.26003 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:07:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1d5036740 1 -- 172.21.15.5:0/3031719279 >> 172.21.15.5:0/3031719279 conn(0x5573c1f6e310 msgr2=0x5573c1fa7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1d5036740 1 -- 172.21.15.5:0/3031719279 shutdown_connections 2024-09-19T06:07:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1d5036740 1 -- 172.21.15.5:0/3031719279 wait complete. 2024-09-19T06:07:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.323+0000 7fd1d5036740 20 client.26003 trim_cache size 0 max 16384 2024-09-19T06:07:19.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.339+0000 7fd1d5036740 1 Processor -- start 2024-09-19T06:07:19.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.339+0000 7fd1d5036740 1 -- start start 2024-09-19T06:07:19.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d5036740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fbc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d5036740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa6b40 0x5573c1fbc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d5036740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fadcb0 0x5573c1fb1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d5036740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1af940 con 0x5573c1fa6b40 2024-09-19T06:07:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d5036740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1b14e0 con 0x5573c1fa55d0 2024-09-19T06:07:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d5036740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1b0010 con 0x5573c1fadcb0 2024-09-19T06:07:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa6b40 0x5573c1fbc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d09f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fbc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa6b40 0x5573c1fbc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36604/0 (socket says 172.21.15.5:36604) 2024-09-19T06:07:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1cbfff640 1 -- 172.21.15.5:0/909034492 learned_addr learned my addr 172.21.15.5:0/909034492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d11fa640 1 --2- 172.21.15.5:0/909034492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fadcb0 0x5573c1fb1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d09f9640 1 -- 172.21.15.5:0/909034492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fadcb0 msgr2=0x5573c1fb1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/909034492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fadcb0 0x5573c1fb1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d09f9640 1 -- 172.21.15.5:0/909034492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa6b40 msgr2=0x5573c1fbc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/909034492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa6b40 0x5573c1fbc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.340+0000 7fd1d09f9640 1 -- 172.21.15.5:0/909034492 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c1fa2960 con 0x5573c1fa55d0 2024-09-19T06:07:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.341+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/909034492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fbc310 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0b9830 tx=0x7fd1bc0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f26caabdaee01eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.341+0000 7fd1b27fc640 1 -- 172.21.15.5:0/909034492 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0bcf10 con 0x5573c1fa55d0 2024-09-19T06:07:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.341+0000 7fd1b27fc640 1 -- 172.21.15.5:0/909034492 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1bc036070 con 0x5573c1fa55d0 2024-09-19T06:07:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.341+0000 7fd1b27fc640 1 -- 172.21.15.5:0/909034492 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc004710 con 0x5573c1fa55d0 2024-09-19T06:07:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.342+0000 7fd1d5036740 1 -- 172.21.15.5:0/909034492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 msgr2=0x5573c1fbc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.342+0000 7fd1d5036740 1 --2- 172.21.15.5:0/909034492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fbc310 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0b9830 tx=0x7fd1bc0095c0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.342+0000 7fd1d5036740 1 -- 172.21.15.5:0/909034492 shutdown_connections 2024-09-19T06:07:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.342+0000 7fd1d5036740 1 --2- 172.21.15.5:0/909034492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fadcb0 0x5573c1fb1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.342+0000 7fd1d5036740 1 --2- 172.21.15.5:0/909034492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa6b40 0x5573c1fbc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.342+0000 7fd1d5036740 1 --2- 172.21.15.5:0/909034492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa55d0 0x5573c1fbc310 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.342+0000 7fd1d5036740 1 -- 172.21.15.5:0/909034492 >> 172.21.15.5:0/909034492 conn(0x5573c1f6e310 msgr2=0x5573c1fac320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.342+0000 7fd1d5036740 1 -- 172.21.15.5:0/909034492 shutdown_connections 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.342+0000 7fd1d5036740 1 -- 172.21.15.5:0/909034492 wait complete. 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.342+0000 7fd1d5036740 1 -- start start 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.342+0000 7fd1d5036740 1 Processor -- start 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.343+0000 7fd1d5036740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa55d0 0x5573c1fb8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.343+0000 7fd1d5036740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa6b40 0x5573c1fb8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.343+0000 7fd1d5036740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fadcb0 0x5573c1fb0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.343+0000 7fd1d5036740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1b0e10 con 0x5573c1fa55d0 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.343+0000 7fd1d5036740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1b06e0 con 0x5573c1fa6b40 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.343+0000 7fd1d5036740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1ad920 con 0x5573c1fadcb0 2024-09-19T06:07:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.343+0000 7fd1d5036740 10 client.0 _ll_register_callbacks cb 0x7fd1bc0f4780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.343+0000 7fd1d09f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa55d0 0x5573c1fb8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.343+0000 7fd1d09f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa55d0 0x5573c1fb8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36612/0 (socket says 172.21.15.5:36612) 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.343+0000 7fd1d09f9640 1 -- 172.21.15.5:0/3132004053 learned_addr learned my addr 172.21.15.5:0/3132004053 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.343+0000 7fd1cbfff640 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa6b40 0x5573c1fb8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.343+0000 7fd1d11fa640 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fadcb0 0x5573c1fb0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.344+0000 7fd1d09f9640 1 -- 172.21.15.5:0/3132004053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fadcb0 msgr2=0x5573c1fb0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.344+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fadcb0 0x5573c1fb0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.344+0000 7fd1d09f9640 1 -- 172.21.15.5:0/3132004053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa6b40 msgr2=0x5573c1fb8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.344+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa6b40 0x5573c1fb8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.344+0000 7fd1d09f9640 1 -- 172.21.15.5:0/3132004053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c1fb1560 con 0x5573c1fa55d0 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.344+0000 7fd1d11fa640 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fadcb0 0x5573c1fb0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.344+0000 7fd1d09f9640 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa55d0 0x5573c1fb8720 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc031040 tx=0x7fd1bc0f48b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87db89a11ce58270 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.344+0000 7fd1abfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:07:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.344+0000 7fd1abfff640 1 -- 172.21.15.5:0/3132004053 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0bcf10 con 0x5573c1fa55d0 2024-09-19T06:07:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.344+0000 7fd1abfff640 1 -- 172.21.15.5:0/3132004053 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fd1bc0f4400 con 0x5573c1fa55d0 2024-09-19T06:07:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.344+0000 7fd1d5036740 1 -- 172.21.15.5:0/3132004053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1bc036070 con 0x5573c1fa55d0 2024-09-19T06:07:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.345+0000 7fd1abfff640 1 -- 172.21.15.5:0/3132004053 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd1bc0ee860 con 0x5573c1fa55d0 2024-09-19T06:07:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.345+0000 7fd1d5036740 10 client.16428 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.345+0000 7fd1d5036740 1 -- 172.21.15.5:0/3132004053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573c1fb1560 con 0x5573c1fa55d0 2024-09-19T06:07:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.345+0000 7fd1abfff640 1 -- 172.21.15.5:0/3132004053 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1bc036070 con 0x5573c1fa55d0 2024-09-19T06:07:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.345+0000 7fd1abfff640 1 -- 172.21.15.5:0/3132004053 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fd1bc0c5b20 con 0x5573c1fa55d0 2024-09-19T06:07:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.345+0000 7fd1d5036740 10 client.16428 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.345+0000 7fd1d5036740 10 client.16428 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:07:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.345+0000 7fd1d5036740 20 client.16428 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.345+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5573c202b2d0 0x5573c202d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.345+0000 7fd1d5036740 4 client.16428 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.345+0000 7fd1d5036740 1 -- 172.21.15.5:0/3132004053 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573c1f12d70 con 0x5573c202b2d0 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.346+0000 7fd1cbfff640 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5573c202b2d0 0x5573c202d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.346+0000 7fd1cbfff640 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5573c202b2d0 0x5573c202d6c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84cf231662d6ba21 server_cookie=4b5138b1d57ecd4f in_seq=0 out_seq=0 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.346+0000 7fd1abfff640 10 client.16428 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.346+0000 7fd1abfff640 1 -- 172.21.15.5:0/3132004053 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573c1f12d70 con 0x5573c202b2d0 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.346+0000 7fd1abfff640 10 client.16428 handle_command_reply: tid=0 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.346+0000 7fd1d5036740 1 client.16428 shutdown 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.346+0000 7fd1d5036740 1 -- 172.21.15.5:0/3132004053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa55d0 msgr2=0x5573c1fb8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.346+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa55d0 0x5573c1fb8720 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc031040 tx=0x7fd1bc0f48b0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.346+0000 7fd1d5036740 1 -- 172.21.15.5:0/3132004053 shutdown_connections 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.346+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5573c202b2d0 0x5573c202d6c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.347+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573c1fadcb0 0x5573c1fb0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.347+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573c1fa6b40 0x5573c1fb8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.347+0000 7fd1d5036740 1 --2- 172.21.15.5:0/3132004053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c1fa55d0 0x5573c1fb8720 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.347+0000 7fd1d5036740 1 -- 172.21.15.5:0/3132004053 >> 172.21.15.5:0/3132004053 conn(0x5573c1f6e310 msgr2=0x5573c1fa04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.347+0000 7fd1abfff640 0 client.16428 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:07:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.347+0000 7fd1d5036740 1 -- 172.21.15.5:0/3132004053 shutdown_connections 2024-09-19T06:07:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.347+0000 7fd1d5036740 1 -- 172.21.15.5:0/3132004053 wait complete. 2024-09-19T06:07:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.347+0000 7fd1d5036740 20 client.16428 trim_cache size 0 max 16384 2024-09-19T06:07:19.355 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:07:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.348+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/1693689863 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1b40a33b0 msgr2=0x7fd1b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.348+0000 7fd1d2c5d640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1b40a33b0 0x7fd1b40a5870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40058c0 tx=0x7fd1c40046c0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.348+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/1693689863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1cc101dd0 msgr2=0x7fd1cc19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.348+0000 7fd1d2c5d640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1cc101dd0 0x7fd1cc19a5d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc02ade0 tx=0x7fd1bc032ff0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.348+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/1693689863 shutdown_connections 2024-09-19T06:07:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.348+0000 7fd1d2c5d640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1b40a33b0 0x7fd1b40a5870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.348+0000 7fd1d2c5d640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1cc106f00 0x7fd1cc19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.348+0000 7fd1d2c5d640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1cc102780 0x7fd1cc19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.348+0000 7fd1d2c5d640 1 --2- 172.21.15.5:0/1693689863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1cc101dd0 0x7fd1cc19a5d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.348+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/1693689863 >> 172.21.15.5:0/1693689863 conn(0x7fd1cc0fd850 msgr2=0x7fd1cc0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.348+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/1693689863 shutdown_connections 2024-09-19T06:07:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:19.348+0000 7fd1d2c5d640 1 -- 172.21.15.5:0/1693689863 wait complete. 2024-09-19T06:07:19.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 85843 2.82 MB/sec execute 45 sec latency 48.876 ms 2024-09-19T06:07:20.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:19 smithi157 ceph-mon[35346]: pgmap v123: 97 pgs: 97 active+clean; 58 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:07:20.127 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:19 smithi005 ceph-mon[30955]: pgmap v123: 97 pgs: 97 active+clean; 58 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:07:20.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:19 smithi049 ceph-mon[34640]: pgmap v123: 97 pgs: 97 active+clean; 58 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:07:20.157 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:07:20.437 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:07:20.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 86364 2.80 MB/sec execute 46 sec latency 13.629 ms 2024-09-19T06:07:20.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.937+0000 7fa4ea526640 1 -- 172.21.15.5:0/3262301833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e4102d30 msgr2=0x7fa4e4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:20.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.937+0000 7fa4ea526640 1 --2- 172.21.15.5:0/3262301833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e4102d30 0x7fa4e4103110 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009930 tx=0x7fa4d402a420 comp rx=0 tx=0).stop 2024-09-19T06:07:20.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.937+0000 7fa4ea526640 1 -- 172.21.15.5:0/3262301833 shutdown_connections 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.937+0000 7fa4ea526640 1 --2- 172.21.15.5:0/3262301833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e41082a0 0x7fa4e410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.937+0000 7fa4ea526640 1 --2- 172.21.15.5:0/3262301833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4e41036e0 0x7fa4e4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.937+0000 7fa4ea526640 1 --2- 172.21.15.5:0/3262301833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e4102d30 0x7fa4e4103110 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.937+0000 7fa4ea526640 1 -- 172.21.15.5:0/3262301833 >> 172.21.15.5:0/3262301833 conn(0x7fa4e40fc7d0 msgr2=0x7fa4e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.937+0000 7fa4ea526640 1 -- 172.21.15.5:0/3262301833 shutdown_connections 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.937+0000 7fa4ea526640 1 -- 172.21.15.5:0/3262301833 wait complete. 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.938+0000 7fa4ea526640 1 Processor -- start 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.938+0000 7fa4ea526640 1 -- start start 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.938+0000 7fa4ea526640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4e4102d30 0x7fa4e4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.938+0000 7fa4ea526640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e41036e0 0x7fa4e4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.938+0000 7fa4ea526640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e41082a0 0x7fa4e419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4ea526640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4111f50 con 0x7fa4e41036e0 2024-09-19T06:07:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4ea526640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4111dd0 con 0x7fa4e4102d30 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4ea526640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41120d0 con 0x7fa4e41082a0 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4e4102d30 0x7fa4e4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e41036e0 0x7fa4e4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4e4102d30 0x7fa4e4198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56480/0 (socket says 172.21.15.5:56480) 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e3fff640 1 -- 172.21.15.5:0/2860824522 learned_addr learned my addr 172.21.15.5:0/2860824522 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e8ac3640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e41082a0 0x7fa4e419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e3fff640 1 -- 172.21.15.5:0/2860824522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e41082a0 msgr2=0x7fa4e419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e41082a0 0x7fa4e419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e3fff640 1 -- 172.21.15.5:0/2860824522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e41036e0 msgr2=0x7fa4e4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e41036e0 0x7fa4e4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e3fff640 1 -- 172.21.15.5:0/2860824522 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e419d4c0 con 0x7fa4e4102d30 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4e4102d30 0x7fa4e4198510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4005940 tx=0x7fa4d40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e17fa640 1 -- 172.21.15.5:0/2860824522 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d403f030 con 0x7fa4e4102d30 2024-09-19T06:07:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.939+0000 7fa4e17fa640 1 -- 172.21.15.5:0/2860824522 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d40318a0 con 0x7fa4e4102d30 2024-09-19T06:07:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.940+0000 7fa4ea526640 1 -- 172.21.15.5:0/2860824522 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e419d750 con 0x7fa4e4102d30 2024-09-19T06:07:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.940+0000 7fa4e17fa640 1 -- 172.21.15.5:0/2860824522 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4035070 con 0x7fa4e4102d30 2024-09-19T06:07:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.940+0000 7fa4ea526640 1 -- 172.21.15.5:0/2860824522 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e41a5090 con 0x7fa4e4102d30 2024-09-19T06:07:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.942+0000 7fa4e17fa640 1 -- 172.21.15.5:0/2860824522 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa4d4042030 con 0x7fa4e4102d30 2024-09-19T06:07:20.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4ec8ff740 1 Processor -- start 2024-09-19T06:07:20.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4ec8ff740 1 -- start start 2024-09-19T06:07:20.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4ec8ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be540090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4ec8ff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be540a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4ec8ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 0x5609be54d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4e17fa640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa4cc0a3430 0x7fa4cc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4ec8ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609be4a62f0 con 0x5609be541b40 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4ec8ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5609be44dcf0 con 0x5609be54b370 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4ec8ff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5609be4a22b0 con 0x5609be5405d0 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be540a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4e8ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be540090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4e8ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be540090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36624/0 (socket says 172.21.15.5:36624) 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.943+0000 7fa4e8ac3640 1 -- 172.21.15.5:0/650707879 learned_addr learned my addr 172.21.15.5:0/650707879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4e37fe640 1 --2- 172.21.15.5:0/650707879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 0x5609be54d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4e17fa640 1 -- 172.21.15.5:0/2860824522 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa4d40f0440 con 0x7fa4e4102d30 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4e3fff640 1 -- 172.21.15.5:0/650707879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 msgr2=0x5609be54d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/650707879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 0x5609be54d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4e3fff640 1 -- 172.21.15.5:0/650707879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 msgr2=0x5609be540090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/650707879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be540090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4e3fff640 1 -- 172.21.15.5:0/650707879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609be4323a0 con 0x5609be5405d0 2024-09-19T06:07:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4e8ac3640 1 --2- 172.21.15.5:0/650707879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be540090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4e8ac3640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa4cc0a3430 0x7fa4cc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/650707879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be540a10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40edba0 tx=0x7fa4d408f560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f8d8cd4cdfc790f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4ca7fc640 1 -- 172.21.15.5:0/650707879 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4049070 con 0x5609be5405d0 2024-09-19T06:07:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4ca7fc640 1 -- 172.21.15.5:0/650707879 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d4003e20 con 0x5609be5405d0 2024-09-19T06:07:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.944+0000 7fa4ca7fc640 1 -- 172.21.15.5:0/650707879 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40b8da0 con 0x5609be5405d0 2024-09-19T06:07:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.945+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/650707879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 msgr2=0x5609be540a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.945+0000 7fa4e8ac3640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa4cc0a3430 0x7fa4cc0a58f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc005c40 tx=0x7fa4dc004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.945+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/650707879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be540a10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40edba0 tx=0x7fa4d408f560 comp rx=0 tx=0).stop 2024-09-19T06:07:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.945+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/650707879 shutdown_connections 2024-09-19T06:07:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.945+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/650707879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 0x5609be54d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.945+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/650707879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be540a10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.945+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/650707879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be540090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.945+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/650707879 >> 172.21.15.5:0/650707879 conn(0x5609be509310 msgr2=0x5609be53cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.945+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/650707879 shutdown_connections 2024-09-19T06:07:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.945+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/650707879 wait complete. 2024-09-19T06:07:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.946+0000 7fa4ec8ff740 1 -- start start 2024-09-19T06:07:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.946+0000 7fa4ec8ff740 1 Processor -- start 2024-09-19T06:07:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.946+0000 7fa4ec8ff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be54ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.946+0000 7fa4ec8ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be541b40 0x5609be548040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.946+0000 7fa4ec8ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be54b370 0x5609be548650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.946+0000 7fa4ec8ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609be4a22b0 con 0x5609be54b370 2024-09-19T06:07:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.946+0000 7fa4ec8ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5609be461060 con 0x5609be541b40 2024-09-19T06:07:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.946+0000 7fa4ec8ff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5609be54de30 con 0x5609be5405d0 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.946+0000 7fa4e8ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be54b370 0x5609be548650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.946+0000 7fa4e8ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be54b370 0x5609be548650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36636/0 (socket says 172.21.15.5:36636) 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.946+0000 7fa4e8ac3640 1 -- 172.21.15.5:0/2266520929 learned_addr learned my addr 172.21.15.5:0/2266520929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.946+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be54ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4ec8ff740 10 client.0 _ll_register_callbacks cb 0x5609be1b48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4e37fe640 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be541b40 0x5609be548040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4e3fff640 1 -- 172.21.15.5:0/2266520929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be541b40 msgr2=0x5609be548040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be541b40 0x5609be548040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4e3fff640 1 -- 172.21.15.5:0/2266520929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be54b370 msgr2=0x5609be548650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be54b370 0x5609be548650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4e3fff640 1 -- 172.21.15.5:0/2266520929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609be2173c0 con 0x5609be5405d0 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4e8ac3640 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be54b370 0x5609be548650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be54ad90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40bc6c0 tx=0x7fa4d40ee050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b372a13c87a7563f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4c37fe640 1 -- 172.21.15.5:0/2266520929 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4049040 con 0x5609be5405d0 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4c37fe640 1 -- 172.21.15.5:0/2266520929 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa4d40bc990 con 0x5609be5405d0 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.947+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2266520929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5609be4b3860 con 0x5609be5405d0 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.948+0000 7fa4c37fe640 1 -- 172.21.15.5:0/2266520929 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa4d40bce30 con 0x5609be5405d0 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.948+0000 7fa4ec8ff740 10 client.25951 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.948+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2266520929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5609be559780 con 0x5609be5405d0 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.948+0000 7fa4c37fe640 1 -- 172.21.15.5:0/2266520929 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5609be4b3860 con 0x5609be5405d0 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.948+0000 7fa4c37fe640 1 -- 172.21.15.5:0/2266520929 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fa4d40ef030 con 0x5609be5405d0 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.948+0000 7fa4ec8ff740 10 client.25951 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.948+0000 7fa4ec8ff740 10 client.25951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.948+0000 7fa4ec8ff740 20 client.25951 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.948+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609be55aad0 0x5609be55cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.948+0000 7fa4ec8ff740 4 client.25951 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.948+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2266520929 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609be4add70 con 0x5609be55aad0 2024-09-19T06:07:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.948+0000 7fa4e37fe640 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609be55aad0 0x5609be55cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.949+0000 7fa4e37fe640 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609be55aad0 0x5609be55cef0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91afb1a484b2e808 server_cookie=1f7a3df8270c7546 in_seq=0 out_seq=0 2024-09-19T06:07:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.949+0000 7fa4c37fe640 10 client.25951 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.949+0000 7fa4c37fe640 1 -- 172.21.15.5:0/2266520929 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609be4add70 con 0x5609be55aad0 2024-09-19T06:07:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.949+0000 7fa4c37fe640 10 client.25951 handle_command_reply: tid=0 2024-09-19T06:07:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4ec8ff740 1 client.25951 shutdown 2024-09-19T06:07:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2266520929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 msgr2=0x5609be54ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be54ad90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40bc6c0 tx=0x7fa4d40ee050 comp rx=0 tx=0).stop 2024-09-19T06:07:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2266520929 shutdown_connections 2024-09-19T06:07:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609be55aad0 0x5609be55cef0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be54b370 0x5609be548650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be541b40 0x5609be548040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/2266520929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be54ad90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4c37fe640 0 client.25951 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2266520929 >> 172.21.15.5:0/2266520929 conn(0x5609be509310 msgr2=0x5609be542960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2266520929 shutdown_connections 2024-09-19T06:07:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2266520929 wait complete. 2024-09-19T06:07:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.950+0000 7fa4ec8ff740 20 client.25951 trim_cache size 0 max 16384 2024-09-19T06:07:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4ec8ff740 1 Processor -- start 2024-09-19T06:07:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4ec8ff740 1 -- start start 2024-09-19T06:07:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4ec8ff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be542820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4ec8ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be542d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4ec8ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 0x5609be5627c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4ec8ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41ab5e0 con 0x5609be541b40 2024-09-19T06:07:20.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4ec8ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41ad180 con 0x5609be54b370 2024-09-19T06:07:20.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4ec8ff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41abcb0 con 0x5609be5405d0 2024-09-19T06:07:20.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be542d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be542820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be542820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52582/0 (socket says 172.21.15.5:52582) 2024-09-19T06:07:20.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be542d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36648/0 (socket says 172.21.15.5:36648) 2024-09-19T06:07:20.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4e3fff640 1 -- 172.21.15.5:0/2759944044 learned_addr learned my addr 172.21.15.5:0/2759944044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:20.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.964+0000 7fa4e8ac3640 1 --2- 172.21.15.5:0/2759944044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 0x5609be5627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.965+0000 7fa4e3fff640 1 -- 172.21.15.5:0/2759944044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 msgr2=0x5609be5627c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:20.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.965+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/2759944044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 0x5609be5627c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.965+0000 7fa4e3fff640 1 -- 172.21.15.5:0/2759944044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 msgr2=0x5609be542d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:20.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.965+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/2759944044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be542d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.965+0000 7fa4e3fff640 1 -- 172.21.15.5:0/2759944044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609be55d550 con 0x5609be5405d0 2024-09-19T06:07:20.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.965+0000 7fa4e37fe640 1 --2- 172.21.15.5:0/2759944044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be542d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:20.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.965+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/2759944044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be542820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40b9100 tx=0x7fa4d4004560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbac96bb3108a476 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:20.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.965+0000 7fa4c9ffb640 1 -- 172.21.15.5:0/2759944044 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4049070 con 0x5609be5405d0 2024-09-19T06:07:20.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.965+0000 7fa4c9ffb640 1 -- 172.21.15.5:0/2759944044 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d40c5190 con 0x5609be5405d0 2024-09-19T06:07:20.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.965+0000 7fa4c9ffb640 1 -- 172.21.15.5:0/2759944044 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40bcbf0 con 0x5609be5405d0 2024-09-19T06:07:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.966+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2759944044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 msgr2=0x5609be542820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.966+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/2759944044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be542820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40b9100 tx=0x7fa4d4004560 comp rx=0 tx=0).stop 2024-09-19T06:07:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.966+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2759944044 shutdown_connections 2024-09-19T06:07:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.966+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/2759944044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 0x5609be5627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.966+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/2759944044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be542d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.966+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/2759944044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be542820 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.966+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2759944044 >> 172.21.15.5:0/2759944044 conn(0x5609be509310 msgr2=0x5609be542110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.966+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2759944044 shutdown_connections 2024-09-19T06:07:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.966+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/2759944044 wait complete. 2024-09-19T06:07:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.966+0000 7fa4ec8ff740 1 -- start start 2024-09-19T06:07:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.966+0000 7fa4ec8ff740 1 Processor -- start 2024-09-19T06:07:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4ec8ff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be553810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4ec8ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be544dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4ec8ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 0x5609be545300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4ec8ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41acab0 con 0x5609be541b40 2024-09-19T06:07:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4ec8ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41ac380 con 0x5609be54b370 2024-09-19T06:07:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4ec8ff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41a95c0 con 0x5609be5405d0 2024-09-19T06:07:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be544dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be544dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36652/0 (socket says 172.21.15.5:36652) 2024-09-19T06:07:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4e37fe640 1 -- 172.21.15.5:0/205454739 learned_addr learned my addr 172.21.15.5:0/205454739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be553810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4ec8ff740 10 client.0 _ll_register_callbacks cb 0x7fa4dc009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4e8ac3640 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 0x5609be545300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4e3fff640 1 -- 172.21.15.5:0/205454739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 msgr2=0x5609be545300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 0x5609be545300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4e3fff640 1 -- 172.21.15.5:0/205454739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 msgr2=0x5609be544dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be544dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4e3fff640 1 -- 172.21.15.5:0/205454739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609be545b30 con 0x5609be5405d0 2024-09-19T06:07:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4e37fe640 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be544dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.967+0000 7fa4e3fff640 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be553810 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40bc6c0 tx=0x7fa4d40bc5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86a3c7dcba9ff0e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.968+0000 7fa4c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:07:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.968+0000 7fa4c37fe640 1 -- 172.21.15.5:0/205454739 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4049070 con 0x5609be5405d0 2024-09-19T06:07:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.968+0000 7fa4c37fe640 1 -- 172.21.15.5:0/205454739 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa4d40bbd80 con 0x5609be5405d0 2024-09-19T06:07:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.968+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/205454739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d40c5190 con 0x5609be5405d0 2024-09-19T06:07:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.968+0000 7fa4c37fe640 1 -- 172.21.15.5:0/205454739 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa4d40b8480 con 0x5609be5405d0 2024-09-19T06:07:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.968+0000 7fa4ec8ff740 10 client.25957 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.968+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/205454739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5609be545b30 con 0x5609be5405d0 2024-09-19T06:07:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.969+0000 7fa4c37fe640 1 -- 172.21.15.5:0/205454739 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d40c5190 con 0x5609be5405d0 2024-09-19T06:07:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.969+0000 7fa4c37fe640 1 -- 172.21.15.5:0/205454739 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fa4d40ef070 con 0x5609be5405d0 2024-09-19T06:07:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.969+0000 7fa4ec8ff740 10 client.25957 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.969+0000 7fa4ec8ff740 10 client.25957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.969+0000 7fa4ec8ff740 20 client.25957 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.969+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609be5c5b00 0x5609be5c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.969+0000 7fa4ec8ff740 4 client.25957 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:07:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.969+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/205454739 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609be42a170 con 0x5609be5c5b00 2024-09-19T06:07:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.969+0000 7fa4e37fe640 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609be5c5b00 0x5609be5c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4e37fe640 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609be5c5b00 0x5609be5c7f20 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361c35a19399dffc server_cookie=efcea2438ec73a21 in_seq=0 out_seq=0 2024-09-19T06:07:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4c37fe640 10 client.25957 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4c37fe640 1 -- 172.21.15.5:0/205454739 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609be42a170 con 0x5609be5c5b00 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4c37fe640 10 client.25957 handle_command_reply: tid=0 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4ec8ff740 1 client.25957 shutdown 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/205454739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 msgr2=0x5609be553810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be553810 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40bc6c0 tx=0x7fa4d40bc5c0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/205454739 shutdown_connections 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609be5c5b00 0x5609be5c7f20 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609be54b370 0x5609be545300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4c37fe640 0 client.25957 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609be541b40 0x5609be544dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4ec8ff740 1 --2- 172.21.15.5:0/205454739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609be5405d0 0x5609be553810 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.970+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/205454739 >> 172.21.15.5:0/205454739 conn(0x5609be509310 msgr2=0x5609be542930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.971+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/205454739 shutdown_connections 2024-09-19T06:07:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.971+0000 7fa4ec8ff740 1 -- 172.21.15.5:0/205454739 wait complete. 2024-09-19T06:07:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.971+0000 7fa4ec8ff740 20 client.25957 trim_cache size 0 max 16384 2024-09-19T06:07:20.980 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:07:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.971+0000 7fa4ea526640 1 -- 172.21.15.5:0/2860824522 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa4cc0a3430 msgr2=0x7fa4cc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.971+0000 7fa4ea526640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa4cc0a3430 0x7fa4cc0a58f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc005c40 tx=0x7fa4dc004ab0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.971+0000 7fa4ea526640 1 -- 172.21.15.5:0/2860824522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4e4102d30 msgr2=0x7fa4e4198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.971+0000 7fa4ea526640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4e4102d30 0x7fa4e4198510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4005940 tx=0x7fa4d40042f0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.972+0000 7fa4ea526640 1 -- 172.21.15.5:0/2860824522 shutdown_connections 2024-09-19T06:07:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.972+0000 7fa4ea526640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa4cc0a3430 0x7fa4cc0a58f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.972+0000 7fa4ea526640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e41082a0 0x7fa4e419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.972+0000 7fa4ea526640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e41036e0 0x7fa4e4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.972+0000 7fa4ea526640 1 --2- 172.21.15.5:0/2860824522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4e4102d30 0x7fa4e4198510 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.972+0000 7fa4ea526640 1 -- 172.21.15.5:0/2860824522 >> 172.21.15.5:0/2860824522 conn(0x7fa4e40fc7d0 msgr2=0x7fa4e4105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.972+0000 7fa4ea526640 1 -- 172.21.15.5:0/2860824522 shutdown_connections 2024-09-19T06:07:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:20.972+0000 7fa4ea526640 1 -- 172.21.15.5:0/2860824522 wait complete. 2024-09-19T06:07:21.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 87022 2.81 MB/sec execute 47 sec latency 10.912 ms 2024-09-19T06:07:22.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:21 smithi157 ceph-mon[35346]: pgmap v124: 97 pgs: 97 active+clean; 58 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:07:22.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:21 smithi005 ceph-mon[30955]: pgmap v124: 97 pgs: 97 active+clean; 58 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:07:22.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:21 smithi049 ceph-mon[34640]: pgmap v124: 97 pgs: 97 active+clean; 58 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:07:22.810 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:07:22.811 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:07:22.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 87622 2.79 MB/sec execute 48 sec latency 14.825 ms 2024-09-19T06:07:23.090 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:07:23.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 88385 2.86 MB/sec execute 49 sec latency 12.593 ms 2024-09-19T06:07:24.024 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:23 smithi005 ceph-mon[30955]: pgmap v125: 97 pgs: 97 active+clean; 58 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-09-19T06:07:24.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:23 smithi157 ceph-mon[35346]: pgmap v125: 97 pgs: 97 active+clean; 58 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-09-19T06:07:24.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:23 smithi049 ceph-mon[34640]: pgmap v125: 97 pgs: 97 active+clean; 58 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-09-19T06:07:24.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 88818 2.89 MB/sec execute 50 sec latency 23.927 ms 2024-09-19T06:07:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.016+0000 7fbce519c640 1 -- 172.21.15.5:0/638661145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbce0100e20 msgr2=0x7fbce0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.016+0000 7fbce519c640 1 --2- 172.21.15.5:0/638661145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbce0100e20 0x7fbce0101200 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009860 tx=0x7fbcd002a3d0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.016+0000 7fbce519c640 1 -- 172.21.15.5:0/638661145 shutdown_connections 2024-09-19T06:07:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.016+0000 7fbce519c640 1 --2- 172.21.15.5:0/638661145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbce0102ef0 0x7fbce010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.016+0000 7fbce519c640 1 --2- 172.21.15.5:0/638661145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbce0102570 0x7fbce01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.016+0000 7fbce519c640 1 --2- 172.21.15.5:0/638661145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbce0100e20 0x7fbce0101200 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.016+0000 7fbce519c640 1 -- 172.21.15.5:0/638661145 >> 172.21.15.5:0/638661145 conn(0x7fbce00fc7d0 msgr2=0x7fbce00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.016+0000 7fbce519c640 1 -- 172.21.15.5:0/638661145 shutdown_connections 2024-09-19T06:07:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.016+0000 7fbce519c640 1 -- 172.21.15.5:0/638661145 wait complete. 2024-09-19T06:07:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.017+0000 7fbce519c640 1 Processor -- start 2024-09-19T06:07:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.017+0000 7fbce519c640 1 -- start start 2024-09-19T06:07:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.017+0000 7fbce519c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbce0100e20 0x7fbce0103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.017+0000 7fbce519c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbce0102570 0x7fbce0103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.017+0000 7fbce519c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbce0102ef0 0x7fbce01041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.017+0000 7fbce519c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0110e30 con 0x7fbce0102570 2024-09-19T06:07:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.017+0000 7fbce519c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0110cb0 con 0x7fbce0100e20 2024-09-19T06:07:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.017+0000 7fbce519c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0110fb0 con 0x7fbce0102ef0 2024-09-19T06:07:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.017+0000 7fbcde59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbce0102570 0x7fbce0103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcded9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbce0100e20 0x7fbce0103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcde59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbce0102570 0x7fbce0103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36668/0 (socket says 172.21.15.5:36668) 2024-09-19T06:07:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcde59c640 1 -- 172.21.15.5:0/711623678 learned_addr learned my addr 172.21.15.5:0/711623678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcdf59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbce0102ef0 0x7fbce01041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcde59c640 1 -- 172.21.15.5:0/711623678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbce0102ef0 msgr2=0x7fbce01041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcde59c640 1 --2- 172.21.15.5:0/711623678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbce0102ef0 0x7fbce01041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcde59c640 1 -- 172.21.15.5:0/711623678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbce0100e20 msgr2=0x7fbce0103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcde59c640 1 --2- 172.21.15.5:0/711623678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbce0100e20 0x7fbce0103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcde59c640 1 -- 172.21.15.5:0/711623678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce01aadc0 con 0x7fbce0102570 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcde59c640 1 --2- 172.21.15.5:0/711623678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbce0102570 0x7fbce0103c90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400e9c0 tx=0x7fbcd400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcc7fff640 1 -- 172.21.15.5:0/711623678 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400ccc0 con 0x7fbce0102570 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbce519c640 1 -- 172.21.15.5:0/711623678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce01aaff0 con 0x7fbce0102570 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcc7fff640 1 -- 172.21.15.5:0/711623678 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd40054d0 con 0x7fbce0102570 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbcc7fff640 1 -- 172.21.15.5:0/711623678 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4010430 con 0x7fbce0102570 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.018+0000 7fbce519c640 1 -- 172.21.15.5:0/711623678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce01ab440 con 0x7fbce0102570 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.021+0000 7fbcc7fff640 1 -- 172.21.15.5:0/711623678 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbcd40046c0 con 0x7fbce0102570 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.021+0000 7fbce7575740 1 Processor -- start 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.021+0000 7fbce7575740 1 -- start start 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.021+0000 7fbce7575740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835e9b60 0x55d6835e9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.021+0000 7fbce7575740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d6835f4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.021+0000 7fbce7575740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d6835f7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.021+0000 7fbce7575740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d68354f280 con 0x55d6835e9b60 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.021+0000 7fbce7575740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d6834f6b10 con 0x55d6835ea480 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.021+0000 7fbce7575740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d68354b240 con 0x55d6835f4d40 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcdf59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835e9b60 0x55d6835e9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcded9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d6835f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcc7fff640 1 --2- 172.21.15.5:0/711623678 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbcc80a33b0 0x7fbcc80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcded9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d6835f4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56532/0 (socket says 172.21.15.5:56532) 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcded9d640 1 -- 172.21.15.5:0/120372252 learned_addr learned my addr 172.21.15.5:0/120372252 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcde59c640 1 --2- 172.21.15.5:0/120372252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d6835f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcdf59e640 1 --2- 172.21.15.5:0/711623678 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbcc80a33b0 0x7fbcc80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcdf59e640 1 -- 172.21.15.5:0/120372252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 msgr2=0x55d6835f7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcdf59e640 1 --2- 172.21.15.5:0/120372252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d6835f7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcdf59e640 1 -- 172.21.15.5:0/120372252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 msgr2=0x55d6835f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcdf59e640 1 --2- 172.21.15.5:0/120372252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d6835f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcc7fff640 1 -- 172.21.15.5:0/711623678 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fbcd40cff60 con 0x7fbce0102570 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcdf59e640 1 -- 172.21.15.5:0/120372252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6835031f0 con 0x55d6835e9b60 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcded9d640 1 --2- 172.21.15.5:0/120372252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d6835f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcdf59e640 1 --2- 172.21.15.5:0/120372252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835e9b60 0x55d6835e9f40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8005fd0 tx=0x7fbcd8013e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57269a4df246df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcdf59e640 1 --2- 172.21.15.5:0/711623678 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbcc80a33b0 0x7fbcc80a5870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbcd801e000 tx=0x7fbcd8020000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.022+0000 7fbcc57fa640 1 -- 172.21.15.5:0/120372252 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd800fc60 con 0x55d6835e9b60 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.023+0000 7fbcc57fa640 1 -- 172.21.15.5:0/120372252 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd8023030 con 0x55d6835e9b60 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.023+0000 7fbcc57fa640 1 -- 172.21.15.5:0/120372252 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8010b40 con 0x55d6835e9b60 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.024+0000 7fbce7575740 1 -- 172.21.15.5:0/120372252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835e9b60 msgr2=0x55d6835e9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.024+0000 7fbce7575740 1 --2- 172.21.15.5:0/120372252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835e9b60 0x55d6835e9f40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8005fd0 tx=0x7fbcd8013e50 comp rx=0 tx=0).stop 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.024+0000 7fbce7575740 1 -- 172.21.15.5:0/120372252 shutdown_connections 2024-09-19T06:07:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.024+0000 7fbce7575740 1 --2- 172.21.15.5:0/120372252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d6835f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.024+0000 7fbce7575740 1 --2- 172.21.15.5:0/120372252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d6835f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.024+0000 7fbce7575740 1 --2- 172.21.15.5:0/120372252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835e9b60 0x55d6835e9f40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.024+0000 7fbce7575740 1 -- 172.21.15.5:0/120372252 >> 172.21.15.5:0/120372252 conn(0x55d6835e60f0 msgr2=0x55d6835e64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.024+0000 7fbce7575740 1 -- 172.21.15.5:0/120372252 shutdown_connections 2024-09-19T06:07:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.024+0000 7fbce7575740 1 -- 172.21.15.5:0/120372252 wait complete. 2024-09-19T06:07:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.024+0000 7fbce7575740 1 -- start start 2024-09-19T06:07:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.024+0000 7fbce7575740 1 Processor -- start 2024-09-19T06:07:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbce7575740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835e9b60 0x55d6835f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbce7575740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835ea480 0x55d6835f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbce7575740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d6835f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbce7575740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d6835f7910 con 0x55d6835ea480 2024-09-19T06:07:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbce7575740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d683509f10 con 0x55d6835e9b60 2024-09-19T06:07:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbce7575740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d6835f7a90 con 0x55d6835f4d40 2024-09-19T06:07:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcde59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835ea480 0x55d6835f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbce7575740 10 client.0 _ll_register_callbacks cb 0x55d6834d2a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcde59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835ea480 0x55d6835f2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36692/0 (socket says 172.21.15.5:36692) 2024-09-19T06:07:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcded9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835e9b60 0x55d6835f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcde59c640 1 -- 172.21.15.5:0/1074163617 learned_addr learned my addr 172.21.15.5:0/1074163617 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcdf59e640 1 --2- 172.21.15.5:0/1074163617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d6835f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcde59c640 1 -- 172.21.15.5:0/1074163617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 msgr2=0x55d6835f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcde59c640 1 --2- 172.21.15.5:0/1074163617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d6835f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcde59c640 1 -- 172.21.15.5:0/1074163617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835e9b60 msgr2=0x55d6835f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcde59c640 1 --2- 172.21.15.5:0/1074163617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835e9b60 0x55d6835f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcde59c640 1 -- 172.21.15.5:0/1074163617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6832c06a0 con 0x55d6835ea480 2024-09-19T06:07:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcde59c640 1 --2- 172.21.15.5:0/1074163617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835ea480 0x55d6835f2220 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4000c00 tx=0x7fbcd400efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bc493cb7e974af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:07:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcc2ff5640 1 -- 172.21.15.5:0/1074163617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd409c250 con 0x55d6835ea480 2024-09-19T06:07:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.025+0000 7fbcc2ff5640 1 -- 172.21.15.5:0/1074163617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fbcd40988c0 con 0x55d6835ea480 2024-09-19T06:07:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbce7575740 1 -- 172.21.15.5:0/1074163617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd8023030 con 0x55d6835ea480 2024-09-19T06:07:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbcc2ff5640 1 -- 172.21.15.5:0/1074163617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbcd40a0600 con 0x55d6835ea480 2024-09-19T06:07:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbce7575740 10 client.16458 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbce7575740 1 -- 172.21.15.5:0/1074163617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d6836035d0 con 0x55d6835ea480 2024-09-19T06:07:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbcc2ff5640 1 -- 172.21.15.5:0/1074163617 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd8023030 con 0x55d6835ea480 2024-09-19T06:07:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbcc2ff5640 1 -- 172.21.15.5:0/1074163617 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fbcd409c4b0 con 0x55d6835ea480 2024-09-19T06:07:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbce7575740 10 client.16458 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbce7575740 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbce7575740 20 client.16458 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbce7575740 1 --2- 172.21.15.5:0/1074163617 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d6836049e0 0x55d683606e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbce7575740 4 client.16458 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbce7575740 1 -- 172.21.15.5:0/1074163617 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d68355c7f0 con 0x55d6836049e0 2024-09-19T06:07:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.026+0000 7fbcded9d640 1 --2- 172.21.15.5:0/1074163617 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d6836049e0 0x55d683606e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.027+0000 7fbcded9d640 1 --2- 172.21.15.5:0/1074163617 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d6836049e0 0x55d683606e60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b08aa6f3fdea42 server_cookie=f292b2d22fecf1a8 in_seq=0 out_seq=0 2024-09-19T06:07:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.027+0000 7fbcc2ff5640 10 client.16458 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbcc2ff5640 1 -- 172.21.15.5:0/1074163617 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d68355c7f0 con 0x55d6836049e0 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbcc2ff5640 10 client.16458 handle_command_reply: tid=0 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbce7575740 1 client.16458 shutdown 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbce7575740 1 -- 172.21.15.5:0/1074163617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835ea480 msgr2=0x55d6835f2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbce7575740 1 --2- 172.21.15.5:0/1074163617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835ea480 0x55d6835f2220 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4000c00 tx=0x7fbcd400efb0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbce7575740 1 -- 172.21.15.5:0/1074163617 shutdown_connections 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbce7575740 1 --2- 172.21.15.5:0/1074163617 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d6836049e0 0x55d683606e60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbce7575740 1 --2- 172.21.15.5:0/1074163617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d6835f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbce7575740 1 --2- 172.21.15.5:0/1074163617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835ea480 0x55d6835f2220 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbcc2ff5640 0 client.16458 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbce7575740 1 --2- 172.21.15.5:0/1074163617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835e9b60 0x55d6835f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbce7575740 1 -- 172.21.15.5:0/1074163617 >> 172.21.15.5:0/1074163617 conn(0x55d6835e60f0 msgr2=0x55d6835e5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbce7575740 1 -- 172.21.15.5:0/1074163617 shutdown_connections 2024-09-19T06:07:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbce7575740 1 -- 172.21.15.5:0/1074163617 wait complete. 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.028+0000 7fbce7575740 20 client.16458 trim_cache size 0 max 16384 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbce7575740 1 Processor -- start 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbce7575740 1 -- start start 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbce7575740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835e9b60 0x55d6835ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbce7575740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d68360c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbce7575740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835f4d40 0x55d68360cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbce7575740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01b3e00 con 0x55d6835f4d40 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbce7575740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01b59a0 con 0x55d6835ea480 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbce7575740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01b44d0 con 0x55d6835e9b60 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbcdf59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835f4d40 0x55d68360cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbcdf59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835f4d40 0x55d68360cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36694/0 (socket says 172.21.15.5:36694) 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbcdf59e640 1 -- 172.21.15.5:0/290856612 learned_addr learned my addr 172.21.15.5:0/290856612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbcde59c640 1 --2- 172.21.15.5:0/290856612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d68360c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbcded9d640 1 --2- 172.21.15.5:0/290856612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835e9b60 0x55d6835ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbcdf59e640 1 -- 172.21.15.5:0/290856612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835e9b60 msgr2=0x55d6835ec5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbcdf59e640 1 --2- 172.21.15.5:0/290856612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835e9b60 0x55d6835ec5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbcdf59e640 1 -- 172.21.15.5:0/290856612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 msgr2=0x55d68360c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbcdf59e640 1 --2- 172.21.15.5:0/290856612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d68360c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.049+0000 7fbcdf59e640 1 -- 172.21.15.5:0/290856612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6835e71a0 con 0x55d6835f4d40 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.050+0000 7fbcdf59e640 1 --2- 172.21.15.5:0/290856612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835f4d40 0x55d68360cc70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8024040 tx=0x7fbcd800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff18c95dec3926f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.050+0000 7fbcc4ff9640 1 -- 172.21.15.5:0/290856612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8037070 con 0x55d6835f4d40 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.050+0000 7fbcc4ff9640 1 -- 172.21.15.5:0/290856612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd80135c0 con 0x55d6835f4d40 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.050+0000 7fbcc4ff9640 1 -- 172.21.15.5:0/290856612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80053c0 con 0x55d6835f4d40 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.051+0000 7fbce7575740 1 -- 172.21.15.5:0/290856612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835f4d40 msgr2=0x55d68360cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.051+0000 7fbce7575740 1 --2- 172.21.15.5:0/290856612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835f4d40 0x55d68360cc70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8024040 tx=0x7fbcd800f790 comp rx=0 tx=0).stop 2024-09-19T06:07:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.051+0000 7fbce7575740 1 -- 172.21.15.5:0/290856612 shutdown_connections 2024-09-19T06:07:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.051+0000 7fbce7575740 1 --2- 172.21.15.5:0/290856612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835f4d40 0x55d68360cc70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.051+0000 7fbce7575740 1 --2- 172.21.15.5:0/290856612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d68360c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.051+0000 7fbce7575740 1 --2- 172.21.15.5:0/290856612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835e9b60 0x55d6835ec5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.051+0000 7fbce7575740 1 -- 172.21.15.5:0/290856612 >> 172.21.15.5:0/290856612 conn(0x55d6835e60f0 msgr2=0x55d6835f3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.051+0000 7fbce7575740 1 -- 172.21.15.5:0/290856612 shutdown_connections 2024-09-19T06:07:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.051+0000 7fbce7575740 1 -- 172.21.15.5:0/290856612 wait complete. 2024-09-19T06:07:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.051+0000 7fbce7575740 1 -- start start 2024-09-19T06:07:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.051+0000 7fbce7575740 1 Processor -- start 2024-09-19T06:07:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbce7575740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835e9b60 0x55d6835f9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbce7575740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d6835f9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbce7575740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d68366e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbce7575740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01b52d0 con 0x55d6835e9b60 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbcde59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d6835f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbce7575740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01b4ba0 con 0x55d6835ea480 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbcde59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d6835f9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56546/0 (socket says 172.21.15.5:56546) 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbce7575740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01b1de0 con 0x55d6835f4d40 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbcde59c640 1 -- 172.21.15.5:0/3149848596 learned_addr learned my addr 172.21.15.5:0/3149848596 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbcded9d640 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835e9b60 0x55d6835f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbcdf59e640 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d68366e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbce7575740 10 client.0 _ll_register_callbacks cb 0x7fbcd400cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbcde59c640 1 -- 172.21.15.5:0/3149848596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 msgr2=0x55d68366e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbcde59c640 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d68366e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbcde59c640 1 -- 172.21.15.5:0/3149848596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835e9b60 msgr2=0x55d6835f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbcde59c640 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835e9b60 0x55d6835f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbcde59c640 1 -- 172.21.15.5:0/3149848596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d68366ed40 con 0x55d6835ea480 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbcdf59e640 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d68366e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.052+0000 7fbcded9d640 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835e9b60 0x55d6835f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbcde59c640 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d6835f9770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40986b0 tx=0x7fbcd40cdb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a49b65cb1de676 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbcc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbcc2ff5640 1 -- 172.21.15.5:0/3149848596 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd409bd90 con 0x55d6835ea480 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbcc2ff5640 1 -- 172.21.15.5:0/3149848596 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fbcd40a0320 con 0x55d6835ea480 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbce7575740 1 -- 172.21.15.5:0/3149848596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd80135c0 con 0x55d6835ea480 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbcc2ff5640 1 -- 172.21.15.5:0/3149848596 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbcd40984f0 con 0x55d6835ea480 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbce7575740 10 client.26045 fetch_fsmap learned FSMap version 32 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbce7575740 1 -- 172.21.15.5:0/3149848596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d683670020 con 0x55d6835ea480 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbcc2ff5640 1 -- 172.21.15.5:0/3149848596 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd80135c0 con 0x55d6835ea480 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbcc2ff5640 1 -- 172.21.15.5:0/3149848596 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fbcd40850a0 con 0x55d6835ea480 2024-09-19T06:07:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbce7575740 10 client.26045 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbce7575740 10 client.26045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbce7575740 20 client.26045 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.053+0000 7fbce7575740 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d683670690 0x55d683672ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.054+0000 7fbce7575740 4 client.26045 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.054+0000 7fbce7575740 1 -- 172.21.15.5:0/3149848596 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d683556d00 con 0x55d683670690 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.054+0000 7fbcded9d640 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d683670690 0x55d683672ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.054+0000 7fbcc2ff5640 10 client.26045 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.054+0000 7fbcded9d640 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d683670690 0x55d683672ab0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d82dde517609664b server_cookie=cdd8fe140ac1116b in_seq=0 out_seq=0 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.055+0000 7fbcc2ff5640 1 -- 172.21.15.5:0/3149848596 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d683556d00 con 0x55d683670690 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.055+0000 7fbcc2ff5640 10 client.26045 handle_command_reply: tid=0 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.055+0000 7fbce7575740 1 client.26045 shutdown 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.055+0000 7fbce7575740 1 -- 172.21.15.5:0/3149848596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 msgr2=0x55d6835f9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.055+0000 7fbce7575740 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d6835f9770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40986b0 tx=0x7fbcd40cdb70 comp rx=0 tx=0).stop 2024-09-19T06:07:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.055+0000 7fbce7575740 1 -- 172.21.15.5:0/3149848596 shutdown_connections 2024-09-19T06:07:24.905 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:07:24.905 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:07:24.905 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "22ed6c5d-d3f0-47fe-a891-a75636dd014a", 2024-09-19T06:07:24.905 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:07:24.905 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:07:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.055+0000 7fbce7575740 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d683670690 0x55d683672ab0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.055+0000 7fbce7575740 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6835f4d40 0x55d68366e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.055+0000 7fbcc2ff5640 0 client.26045 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.055+0000 7fbce7575740 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d6835ea480 0x55d6835f9770 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce7575740 1 --2- 172.21.15.5:0/3149848596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6835e9b60 0x55d6835f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce7575740 1 -- 172.21.15.5:0/3149848596 >> 172.21.15.5:0/3149848596 conn(0x55d6835e60f0 msgr2=0x55d6835e4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce7575740 1 -- 172.21.15.5:0/3149848596 shutdown_connections 2024-09-19T06:07:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce7575740 1 -- 172.21.15.5:0/3149848596 wait complete. 2024-09-19T06:07:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce7575740 20 client.26045 trim_cache size 0 max 16384 2024-09-19T06:07:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce519c640 1 -- 172.21.15.5:0/711623678 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbcc80a33b0 msgr2=0x7fbcc80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce519c640 1 --2- 172.21.15.5:0/711623678 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbcc80a33b0 0x7fbcc80a5870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbcd801e000 tx=0x7fbcd8020000 comp rx=0 tx=0).stop 2024-09-19T06:07:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce519c640 1 -- 172.21.15.5:0/711623678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbce0102570 msgr2=0x7fbce0103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce519c640 1 --2- 172.21.15.5:0/711623678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbce0102570 0x7fbce0103c90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400e9c0 tx=0x7fbcd400ee80 comp rx=0 tx=0).stop 2024-09-19T06:07:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce519c640 1 -- 172.21.15.5:0/711623678 shutdown_connections 2024-09-19T06:07:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce519c640 1 --2- 172.21.15.5:0/711623678 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbcc80a33b0 0x7fbcc80a5870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce519c640 1 --2- 172.21.15.5:0/711623678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbce0102ef0 0x7fbce01041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce519c640 1 --2- 172.21.15.5:0/711623678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbce0102570 0x7fbce0103c90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce519c640 1 --2- 172.21.15.5:0/711623678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbce0100e20 0x7fbce0103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.056+0000 7fbce519c640 1 -- 172.21.15.5:0/711623678 >> 172.21.15.5:0/711623678 conn(0x7fbce00fc7d0 msgr2=0x7fbce006a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.057+0000 7fbce519c640 1 -- 172.21.15.5:0/711623678 shutdown_connections 2024-09-19T06:07:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:24.057+0000 7fbce519c640 1 -- 172.21.15.5:0/711623678 wait complete. 2024-09-19T06:07:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:24 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:07:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:24 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:24 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:07:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:24 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:07:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:24 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:24 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:24 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:07:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:24 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:07:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:24 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:24 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:24 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:07:25.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:24 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:24 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:07:25.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:24 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:24 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:07:25.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:24 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:07:25.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:24 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:24 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:24 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:07:25.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:24 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:07:25.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:24 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:24 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:24 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:07:25.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:24 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:24 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:07:25.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:24 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:24 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:07:25.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:24 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:07:25.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:24 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:24 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:24 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:07:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:24 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:07:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:24 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:24 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:24 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:07:25.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:24 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:25.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 89093 2.86 MB/sec execute 51 sec latency 64.279 ms 2024-09-19T06:07:26.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:25 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:07:26.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:25 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:26.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:25 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:07:26.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:25 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:26.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:25 smithi157 ceph-mon[35346]: pgmap v126: 97 pgs: 97 active+clean; 58 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-09-19T06:07:26.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:25 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:07:26.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:25 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:07:26.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:25 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:26.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:25 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:07:26.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:25 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:26.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:25 smithi005 ceph-mon[30955]: pgmap v126: 97 pgs: 97 active+clean; 58 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-09-19T06:07:26.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:25 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:07:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:25 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:07:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:25 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:25 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:07:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:25 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:07:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:25 smithi049 ceph-mon[34640]: pgmap v126: 97 pgs: 97 active+clean; 58 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-09-19T06:07:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:25 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:07:26.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 89383 2.81 MB/sec execute 52 sec latency 65.254 ms 2024-09-19T06:07:27.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 89902 2.79 MB/sec execute 53 sec latency 64.486 ms 2024-09-19T06:07:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:27 smithi005 ceph-mon[30955]: pgmap v127: 97 pgs: 97 active+clean; 63 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.8 MiB/s wr, 199 op/s 2024-09-19T06:07:28.040 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:07:28.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:27 smithi157 ceph-mon[35346]: pgmap v127: 97 pgs: 97 active+clean; 63 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.8 MiB/s wr, 199 op/s 2024-09-19T06:07:28.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:27 smithi049 ceph-mon[34640]: pgmap v127: 97 pgs: 97 active+clean; 63 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.8 MiB/s wr, 199 op/s 2024-09-19T06:07:28.360 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:07:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.709+0000 7fec4694b640 1 -- 172.21.15.5:0/1017242054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec401069b0 msgr2=0x7fec40106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.709+0000 7fec4694b640 1 --2- 172.21.15.5:0/1017242054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec401069b0 0x7fec40106d90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fec30009930 tx=0x7fec3002a420 comp rx=0 tx=0).stop 2024-09-19T06:07:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.710+0000 7fec4694b640 1 -- 172.21.15.5:0/1017242054 shutdown_connections 2024-09-19T06:07:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.710+0000 7fec4694b640 1 --2- 172.21.15.5:0/1017242054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec40101340 0x7fec4010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.710+0000 7fec4694b640 1 --2- 172.21.15.5:0/1017242054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec401009a0 0x7fec40100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.710+0000 7fec4694b640 1 --2- 172.21.15.5:0/1017242054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec401069b0 0x7fec40106d90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.710+0000 7fec4694b640 1 -- 172.21.15.5:0/1017242054 >> 172.21.15.5:0/1017242054 conn(0x7fec400779a0 msgr2=0x7fec400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.710+0000 7fec4694b640 1 -- 172.21.15.5:0/1017242054 shutdown_connections 2024-09-19T06:07:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.710+0000 7fec4694b640 1 -- 172.21.15.5:0/1017242054 wait complete. 2024-09-19T06:07:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.710+0000 7fec4694b640 1 Processor -- start 2024-09-19T06:07:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.710+0000 7fec4694b640 1 -- start start 2024-09-19T06:07:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec4694b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec401009a0 0x7fec40102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec4694b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec40101340 0x7fec40102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec4694b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec401069b0 0x7fec40102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec4694b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec40110ca0 con 0x7fec401069b0 2024-09-19T06:07:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec4694b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec40110b20 con 0x7fec401009a0 2024-09-19T06:07:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec4694b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec40110e20 con 0x7fec40101340 2024-09-19T06:07:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec44ee8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec401069b0 0x7fec40102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec44ee8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec401069b0 0x7fec40102ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36772/0 (socket says 172.21.15.5:36772) 2024-09-19T06:07:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec44ee8640 1 -- 172.21.15.5:0/3498095292 learned_addr learned my addr 172.21.15.5:0/3498095292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec3ffff640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec401009a0 0x7fec40102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec40101340 0x7fec40102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec44ee8640 1 -- 172.21.15.5:0/3498095292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec40101340 msgr2=0x7fec40102ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec44ee8640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec40101340 0x7fec40102ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec44ee8640 1 -- 172.21.15.5:0/3498095292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec401009a0 msgr2=0x7fec40102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec44ee8640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec401009a0 0x7fec40102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec44ee8640 1 -- 172.21.15.5:0/3498095292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec401a6850 con 0x7fec401069b0 2024-09-19T06:07:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec44ee8640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec401069b0 0x7fec40102ff0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fec3800eaa0 tx=0x7fec3800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec3d7fa640 1 -- 172.21.15.5:0/3498095292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3800cdb0 con 0x7fec401069b0 2024-09-19T06:07:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec3d7fa640 1 -- 172.21.15.5:0/3498095292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec38005020 con 0x7fec401069b0 2024-09-19T06:07:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec4694b640 1 -- 172.21.15.5:0/3498095292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec401a6b40 con 0x7fec401069b0 2024-09-19T06:07:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.711+0000 7fec3d7fa640 1 -- 172.21.15.5:0/3498095292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec380116d0 con 0x7fec401069b0 2024-09-19T06:07:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.712+0000 7fec4694b640 1 -- 172.21.15.5:0/3498095292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec401a7040 con 0x7fec401069b0 2024-09-19T06:07:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec3d7fa640 1 -- 172.21.15.5:0/3498095292 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fec380051d0 con 0x7fec401069b0 2024-09-19T06:07:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec497c7740 1 Processor -- start 2024-09-19T06:07:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec497c7740 1 -- start start 2024-09-19T06:07:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec497c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fbded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec497c7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbe410 0x555ef1fc8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec497c7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fc8cb0 0x555ef1fcb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec497c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555ef1f222f0 con 0x555ef1fbdaf0 2024-09-19T06:07:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec497c7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555ef1eca270 con 0x555ef1fbe410 2024-09-19T06:07:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec497c7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555ef1f1e2b0 con 0x555ef1fc8cb0 2024-09-19T06:07:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fbded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbe410 0x555ef1fc8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fbded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36774/0 (socket says 172.21.15.5:36774) 2024-09-19T06:07:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec3f7fe640 1 -- 172.21.15.5:0/2699763830 learned_addr learned my addr 172.21.15.5:0/2699763830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.715+0000 7fec3d7fa640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fec2c0a33b0 0x7fec2c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/2699763830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fc8cb0 0x555ef1fcb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec44ee8640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fec2c0a33b0 0x7fec2c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec3ffff640 1 -- 172.21.15.5:0/2699763830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fc8cb0 msgr2=0x555ef1fcb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec3d7fa640 1 -- 172.21.15.5:0/3498095292 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fec380cfea0 con 0x7fec401069b0 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec3ffff640 1 --2- 172.21.15.5:0/2699763830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fc8cb0 0x555ef1fcb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec3ffff640 1 -- 172.21.15.5:0/2699763830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 msgr2=0x555ef1fbded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec3ffff640 1 --2- 172.21.15.5:0/2699763830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fbded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec3ffff640 1 -- 172.21.15.5:0/2699763830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef1ec9340 con 0x555ef1fbe410 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/2699763830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fbded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec3ffff640 1 --2- 172.21.15.5:0/2699763830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbe410 0x555ef1fc8770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fec30009900 tx=0x7fec30032cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b18c36ce129918b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec44ee8640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fec2c0a33b0 0x7fec2c0a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fec3809c120 tx=0x7fec38015820 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec367fc640 1 -- 172.21.15.5:0/2699763830 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3003f070 con 0x555ef1fbe410 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec367fc640 1 -- 172.21.15.5:0/2699763830 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec300044d0 con 0x555ef1fbe410 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.716+0000 7fec367fc640 1 -- 172.21.15.5:0/2699763830 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30035070 con 0x555ef1fbe410 2024-09-19T06:07:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.717+0000 7fec497c7740 1 -- 172.21.15.5:0/2699763830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbe410 msgr2=0x555ef1fc8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.717+0000 7fec497c7740 1 --2- 172.21.15.5:0/2699763830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbe410 0x555ef1fc8770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fec30009900 tx=0x7fec30032cc0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.717+0000 7fec497c7740 1 -- 172.21.15.5:0/2699763830 shutdown_connections 2024-09-19T06:07:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.717+0000 7fec497c7740 1 --2- 172.21.15.5:0/2699763830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fc8cb0 0x555ef1fcb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.717+0000 7fec497c7740 1 --2- 172.21.15.5:0/2699763830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbe410 0x555ef1fc8770 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.717+0000 7fec497c7740 1 --2- 172.21.15.5:0/2699763830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fbded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.717+0000 7fec497c7740 1 -- 172.21.15.5:0/2699763830 >> 172.21.15.5:0/2699763830 conn(0x555ef1fba060 msgr2=0x555ef1fba440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.717+0000 7fec497c7740 1 -- 172.21.15.5:0/2699763830 shutdown_connections 2024-09-19T06:07:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.717+0000 7fec497c7740 1 -- 172.21.15.5:0/2699763830 wait complete. 2024-09-19T06:07:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec497c7740 1 -- start start 2024-09-19T06:07:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec497c7740 1 Processor -- start 2024-09-19T06:07:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec497c7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbdaf0 0x555ef1fc5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec497c7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fbe410 0x555ef1fc60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec497c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fc8cb0 0x555ef1fc65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec497c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555ef1fcb880 con 0x555ef1fc8cb0 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec497c7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555ef1eab290 con 0x555ef1fbdaf0 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec497c7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555ef1fcba00 con 0x555ef1fbe410 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec3f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbdaf0 0x555ef1fc5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fbe410 0x555ef1fc60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec3f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbdaf0 0x555ef1fc5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50276/0 (socket says 172.21.15.5:50276) 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec3f7fe640 1 -- 172.21.15.5:0/2279755093 learned_addr learned my addr 172.21.15.5:0/2279755093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fc8cb0 0x555ef1fc65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.718+0000 7fec497c7740 10 client.0 _ll_register_callbacks cb 0x555ef1edc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec3f7fe640 1 -- 172.21.15.5:0/2279755093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fbe410 msgr2=0x555ef1fc60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fbe410 0x555ef1fc60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec3f7fe640 1 -- 172.21.15.5:0/2279755093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fc8cb0 msgr2=0x555ef1fc65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fc8cb0 0x555ef1fc65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec3f7fe640 1 -- 172.21.15.5:0/2279755093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef1c35210 con 0x555ef1fbdaf0 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec3ffff640 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fbe410 0x555ef1fc60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbdaf0 0x555ef1fc5b60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fec28015e70 tx=0x7fec2800a2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9aa8913ef73a76b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec1f7fe640 1 -- 172.21.15.5:0/2279755093 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28015110 con 0x555ef1fbdaf0 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec1f7fe640 1 -- 172.21.15.5:0/2279755093 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fec280152b0 con 0x555ef1fbdaf0 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec497c7740 1 -- 172.21.15.5:0/2279755093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec300044d0 con 0x555ef1fbdaf0 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec1f7fe640 1 -- 172.21.15.5:0/2279755093 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fec28004020 con 0x555ef1fbdaf0 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec497c7740 10 client.26054 fetch_fsmap learned FSMap version 33 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.719+0000 7fec497c7740 1 -- 172.21.15.5:0/2279755093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555ef1d01620 con 0x555ef1fbdaf0 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.720+0000 7fec1f7fe640 1 -- 172.21.15.5:0/2279755093 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec300044d0 con 0x555ef1fbdaf0 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.720+0000 7fec1f7fe640 1 -- 172.21.15.5:0/2279755093 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fec280035f0 con 0x555ef1fbdaf0 2024-09-19T06:07:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.720+0000 7fec497c7740 10 client.26054 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:07:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.720+0000 7fec497c7740 10 client.26054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.720+0000 7fec497c7740 20 client.26054 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.720+0000 7fec497c7740 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555ef1fd8b50 0x555ef1fdaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.720+0000 7fec497c7740 4 client.26054 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.720+0000 7fec497c7740 1 -- 172.21.15.5:0/2279755093 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ef1f2fa20 con 0x555ef1fd8b50 2024-09-19T06:07:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.720+0000 7fec3ffff640 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555ef1fd8b50 0x555ef1fdaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.720+0000 7fec3ffff640 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555ef1fd8b50 0x555ef1fdaf40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c2ef682cddce25 server_cookie=27114ff7a6ed6f6a in_seq=0 out_seq=0 2024-09-19T06:07:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.720+0000 7fec1f7fe640 10 client.26054 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.721+0000 7fec1f7fe640 1 -- 172.21.15.5:0/2279755093 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ef1f2fa20 con 0x555ef1fd8b50 2024-09-19T06:07:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.721+0000 7fec1f7fe640 10 client.26054 handle_command_reply: tid=0 2024-09-19T06:07:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec497c7740 1 client.26054 shutdown 2024-09-19T06:07:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec497c7740 1 -- 172.21.15.5:0/2279755093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbdaf0 msgr2=0x555ef1fc5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec497c7740 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbdaf0 0x555ef1fc5b60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fec28015e70 tx=0x7fec2800a2d0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec497c7740 1 -- 172.21.15.5:0/2279755093 shutdown_connections 2024-09-19T06:07:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec497c7740 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555ef1fd8b50 0x555ef1fdaf40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec497c7740 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fc8cb0 0x555ef1fc65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec497c7740 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fbe410 0x555ef1fc60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec497c7740 1 --2- 172.21.15.5:0/2279755093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbdaf0 0x555ef1fc5b60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec1f7fe640 0 client.26054 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec497c7740 1 -- 172.21.15.5:0/2279755093 >> 172.21.15.5:0/2279755093 conn(0x555ef1fba060 msgr2=0x555ef1fbfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec497c7740 1 -- 172.21.15.5:0/2279755093 shutdown_connections 2024-09-19T06:07:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec497c7740 1 -- 172.21.15.5:0/2279755093 wait complete. 2024-09-19T06:07:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.722+0000 7fec497c7740 20 client.26054 trim_cache size 0 max 16384 2024-09-19T06:07:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.737+0000 7fec497c7740 1 Processor -- start 2024-09-19T06:07:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.737+0000 7fec497c7740 1 -- start start 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec497c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fc6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec497c7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fbe410 0x555ef1fe0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec497c7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fc8cb0 0x555ef1fe0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec497c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec401ab590 con 0x555ef1fbdaf0 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec497c7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec401ad0d0 con 0x555ef1fc8cb0 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec497c7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec401abc60 con 0x555ef1fbe410 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fc6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fc6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36790/0 (socket says 172.21.15.5:36790) 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec3f7fe640 1 -- 172.21.15.5:0/2802932119 learned_addr learned my addr 172.21.15.5:0/2802932119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec3ffff640 1 --2- 172.21.15.5:0/2802932119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fbe410 0x555ef1fe0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/2802932119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fc8cb0 0x555ef1fe0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec3f7fe640 1 -- 172.21.15.5:0/2802932119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fbe410 msgr2=0x555ef1fe0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/2802932119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fbe410 0x555ef1fe0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec3f7fe640 1 -- 172.21.15.5:0/2802932119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fc8cb0 msgr2=0x555ef1fe0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/2802932119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fc8cb0 0x555ef1fe0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec3f7fe640 1 -- 172.21.15.5:0/2802932119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef1fb9e90 con 0x555ef1fbdaf0 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec3ffff640 1 --2- 172.21.15.5:0/2802932119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fbe410 0x555ef1fe0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.738+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/2802932119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fc6410 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fec28018040 tx=0x7fec28015bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c530ca0df782ecce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.739+0000 7fec35ffb640 1 -- 172.21.15.5:0/2802932119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec280041c0 con 0x555ef1fbdaf0 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.739+0000 7fec35ffb640 1 -- 172.21.15.5:0/2802932119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec28004410 con 0x555ef1fbdaf0 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.739+0000 7fec35ffb640 1 -- 172.21.15.5:0/2802932119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28015110 con 0x555ef1fbdaf0 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.739+0000 7fec497c7740 1 -- 172.21.15.5:0/2802932119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 msgr2=0x555ef1fc6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.739+0000 7fec497c7740 1 --2- 172.21.15.5:0/2802932119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fc6410 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fec28018040 tx=0x7fec28015bc0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 -- 172.21.15.5:0/2802932119 shutdown_connections 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 --2- 172.21.15.5:0/2802932119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fc8cb0 0x555ef1fe0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 --2- 172.21.15.5:0/2802932119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fbe410 0x555ef1fe0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 --2- 172.21.15.5:0/2802932119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fc6410 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 -- 172.21.15.5:0/2802932119 >> 172.21.15.5:0/2802932119 conn(0x555ef1fba060 msgr2=0x555ef1fc7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 -- 172.21.15.5:0/2802932119 shutdown_connections 2024-09-19T06:07:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 -- 172.21.15.5:0/2802932119 wait complete. 2024-09-19T06:07:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 -- start start 2024-09-19T06:07:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 Processor -- start 2024-09-19T06:07:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fd1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbe410 0x555ef1fd1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fc8cb0 0x555ef1fc26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec401aca00 con 0x555ef1fbdaf0 2024-09-19T06:07:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec401ac330 con 0x555ef1fbe410 2024-09-19T06:07:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.740+0000 7fec497c7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec401a95b0 con 0x555ef1fc8cb0 2024-09-19T06:07:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec497c7740 10 client.0 _ll_register_callbacks cb 0x7fec280154e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fd1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fd1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36794/0 (socket says 172.21.15.5:36794) 2024-09-19T06:07:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec3f7fe640 1 -- 172.21.15.5:0/3293643741 learned_addr learned my addr 172.21.15.5:0/3293643741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbe410 0x555ef1fd1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/3293643741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fc8cb0 0x555ef1fc26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec3ffff640 1 -- 172.21.15.5:0/3293643741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fc8cb0 msgr2=0x555ef1fc26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec3ffff640 1 --2- 172.21.15.5:0/3293643741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fc8cb0 0x555ef1fc26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec3ffff640 1 -- 172.21.15.5:0/3293643741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 msgr2=0x555ef1fd1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec3ffff640 1 --2- 172.21.15.5:0/3293643741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fd1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec3ffff640 1 -- 172.21.15.5:0/3293643741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef1fc2eb0 con 0x555ef1fbe410 2024-09-19T06:07:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec3ffff640 1 --2- 172.21.15.5:0/3293643741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbe410 0x555ef1fd1570 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fec30000c00 tx=0x7fec300058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6b5b2bccaff2a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:07:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec1f7fe640 1 -- 172.21.15.5:0/3293643741 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3005a030 con 0x555ef1fbe410 2024-09-19T06:07:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec497c7740 1 -- 172.21.15.5:0/3293643741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec28004410 con 0x555ef1fbe410 2024-09-19T06:07:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec1f7fe640 1 -- 172.21.15.5:0/3293643741 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fec30004450 con 0x555ef1fbe410 2024-09-19T06:07:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec1f7fe640 1 -- 172.21.15.5:0/3293643741 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fec30035040 con 0x555ef1fbe410 2024-09-19T06:07:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.741+0000 7fec497c7740 10 client.26060 fetch_fsmap learned FSMap version 33 2024-09-19T06:07:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.742+0000 7fec497c7740 1 -- 172.21.15.5:0/3293643741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555ef20433d0 con 0x555ef1fbe410 2024-09-19T06:07:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.742+0000 7fec1f7fe640 1 -- 172.21.15.5:0/3293643741 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec28004410 con 0x555ef1fbe410 2024-09-19T06:07:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.742+0000 7fec1f7fe640 1 -- 172.21.15.5:0/3293643741 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fec300564c0 con 0x555ef1fbe410 2024-09-19T06:07:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.742+0000 7fec497c7740 10 client.26060 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:07:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.742+0000 7fec497c7740 10 client.26060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.742+0000 7fec497c7740 20 client.26060 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.742+0000 7fec497c7740 1 --2- 172.21.15.5:0/3293643741 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555ef2043700 0x555ef2045af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.742+0000 7fec497c7740 4 client.26060 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:07:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.742+0000 7fec497c7740 1 -- 172.21.15.5:0/3293643741 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ef1f29d70 con 0x555ef2043700 2024-09-19T06:07:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.742+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/3293643741 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555ef2043700 0x555ef2045af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.742+0000 7fec1f7fe640 10 client.26060 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.742+0000 7fec3f7fe640 1 --2- 172.21.15.5:0/3293643741 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555ef2043700 0x555ef2045af0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dacb2d05370cbe server_cookie=b7904b1b662791ee in_seq=0 out_seq=0 2024-09-19T06:07:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec1f7fe640 1 -- 172.21.15.5:0/3293643741 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ef1f29d70 con 0x555ef2043700 2024-09-19T06:07:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec1f7fe640 10 client.26060 handle_command_reply: tid=0 2024-09-19T06:07:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec497c7740 1 client.26060 shutdown 2024-09-19T06:07:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec497c7740 1 -- 172.21.15.5:0/3293643741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbe410 msgr2=0x555ef1fd1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec497c7740 1 --2- 172.21.15.5:0/3293643741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbe410 0x555ef1fd1570 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fec30000c00 tx=0x7fec300058d0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec497c7740 1 -- 172.21.15.5:0/3293643741 shutdown_connections 2024-09-19T06:07:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec497c7740 1 --2- 172.21.15.5:0/3293643741 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555ef2043700 0x555ef2045af0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec497c7740 1 --2- 172.21.15.5:0/3293643741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ef1fc8cb0 0x555ef1fc26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec497c7740 1 --2- 172.21.15.5:0/3293643741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ef1fbe410 0x555ef1fd1570 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec1f7fe640 0 client.26060 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec497c7740 1 --2- 172.21.15.5:0/3293643741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ef1fbdaf0 0x555ef1fd1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec497c7740 1 -- 172.21.15.5:0/3293643741 >> 172.21.15.5:0/3293643741 conn(0x555ef1fba060 msgr2=0x555ef1fbf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec497c7740 1 -- 172.21.15.5:0/3293643741 shutdown_connections 2024-09-19T06:07:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec497c7740 1 -- 172.21.15.5:0/3293643741 wait complete. 2024-09-19T06:07:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.762+0000 7fec497c7740 20 client.26060 trim_cache size 0 max 16384 2024-09-19T06:07:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:07:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:07:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:07:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:07:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.763+0000 7fec4694b640 1 -- 172.21.15.5:0/3498095292 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fec2c0a33b0 msgr2=0x7fec2c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.763+0000 7fec4694b640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fec2c0a33b0 0x7fec2c0a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fec3809c120 tx=0x7fec38015820 comp rx=0 tx=0).stop 2024-09-19T06:07:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.763+0000 7fec4694b640 1 -- 172.21.15.5:0/3498095292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec401069b0 msgr2=0x7fec40102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.763+0000 7fec4694b640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec401069b0 0x7fec40102ff0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fec3800eaa0 tx=0x7fec3800ef60 comp rx=0 tx=0).stop 2024-09-19T06:07:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.763+0000 7fec4694b640 1 -- 172.21.15.5:0/3498095292 shutdown_connections 2024-09-19T06:07:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.763+0000 7fec4694b640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec401069b0 0x7fec40102ff0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.763+0000 7fec4694b640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fec2c0a33b0 0x7fec2c0a5870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.763+0000 7fec4694b640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec40101340 0x7fec40102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.763+0000 7fec4694b640 1 --2- 172.21.15.5:0/3498095292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec401009a0 0x7fec40102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.763+0000 7fec4694b640 1 -- 172.21.15.5:0/3498095292 >> 172.21.15.5:0/3498095292 conn(0x7fec400779a0 msgr2=0x7fec40069bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.763+0000 7fec4694b640 1 -- 172.21.15.5:0/3498095292 shutdown_connections 2024-09-19T06:07:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:28.763+0000 7fec4694b640 1 -- 172.21.15.5:0/3498095292 wait complete. 2024-09-19T06:07:28.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 90502 2.80 MB/sec execute 54 sec latency 38.311 ms 2024-09-19T06:07:28.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.886+0000 7fd604222640 1 -- 172.21.15.5:0/3530172827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc102ef0 msgr2=0x7fd5fc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.886+0000 7fd604222640 1 --2- 172.21.15.5:0/3530172827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc102ef0 0x7fd5fc10da40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009ca0 tx=0x7fd5f402a950 comp rx=0 tx=0).stop 2024-09-19T06:07:28.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.886+0000 7fd604222640 1 -- 172.21.15.5:0/3530172827 shutdown_connections 2024-09-19T06:07:28.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.886+0000 7fd604222640 1 --2- 172.21.15.5:0/3530172827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc102ef0 0x7fd5fc10da40 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.886+0000 7fd604222640 1 --2- 172.21.15.5:0/3530172827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc102570 0x7fd5fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.886+0000 7fd604222640 1 --2- 172.21.15.5:0/3530172827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5fc100e20 0x7fd5fc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.886+0000 7fd604222640 1 -- 172.21.15.5:0/3530172827 >> 172.21.15.5:0/3530172827 conn(0x7fd5fc0fc7d0 msgr2=0x7fd5fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.886+0000 7fd604222640 1 -- 172.21.15.5:0/3530172827 shutdown_connections 2024-09-19T06:07:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.886+0000 7fd604222640 1 -- 172.21.15.5:0/3530172827 wait complete. 2024-09-19T06:07:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd604222640 1 Processor -- start 2024-09-19T06:07:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd604222640 1 -- start start 2024-09-19T06:07:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd604222640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc100e20 0x7fd5fc197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd604222640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc102570 0x7fd5fc1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd604222640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5fc102ef0 0x7fd5fc19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd604222640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc110e30 con 0x7fd5fc100e20 2024-09-19T06:07:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd604222640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc110cb0 con 0x7fd5fc102ef0 2024-09-19T06:07:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd604222640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc110fb0 con 0x7fd5fc102570 2024-09-19T06:07:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd601fbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc100e20 0x7fd5fc197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd601fbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc100e20 0x7fd5fc197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36816/0 (socket says 172.21.15.5:36816) 2024-09-19T06:07:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd601fbe640 1 -- 172.21.15.5:0/1752033902 learned_addr learned my addr 172.21.15.5:0/1752033902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd6017bd640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc102570 0x7fd5fc1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.887+0000 7fd6027bf640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5fc102ef0 0x7fd5fc19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.888+0000 7fd601fbe640 1 -- 172.21.15.5:0/1752033902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc102570 msgr2=0x7fd5fc1978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.888+0000 7fd601fbe640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc102570 0x7fd5fc1978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.888+0000 7fd601fbe640 1 -- 172.21.15.5:0/1752033902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5fc102ef0 msgr2=0x7fd5fc19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.888+0000 7fd601fbe640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5fc102ef0 0x7fd5fc19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.888+0000 7fd601fbe640 1 -- 172.21.15.5:0/1752033902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc19c330 con 0x7fd5fc100e20 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.888+0000 7fd6017bd640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc102570 0x7fd5fc1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.888+0000 7fd601fbe640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc100e20 0x7fd5fc197380 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800eb00 tx=0x7fd5f8009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.888+0000 7fd5f2ffd640 1 -- 172.21.15.5:0/1752033902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f801c070 con 0x7fd5fc100e20 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.888+0000 7fd5f2ffd640 1 -- 172.21.15.5:0/1752033902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5f800bc30 con 0x7fd5fc100e20 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.888+0000 7fd604222640 1 -- 172.21.15.5:0/1752033902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5fc19c620 con 0x7fd5fc100e20 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.888+0000 7fd5f2ffd640 1 -- 172.21.15.5:0/1752033902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8016610 con 0x7fd5fc100e20 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.888+0000 7fd604222640 1 -- 172.21.15.5:0/1752033902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5fc069f70 con 0x7fd5fc100e20 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.891+0000 7fd5f2ffd640 1 -- 172.21.15.5:0/1752033902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd5f801f070 con 0x7fd5fc100e20 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.891+0000 7fd6065fb740 1 Processor -- start 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.891+0000 7fd6065fb740 1 -- start start 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6065fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb03380 0x55df6cb03760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6065fb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03ca0 0x55df6cb12550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6065fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb12a90 0x55df6cb14e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6065fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55df6ca692e0 con 0x55df6cb12a90 2024-09-19T06:07:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6065fb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55df6ca10ce0 con 0x55df6cb03ca0 2024-09-19T06:07:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6065fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55df6ca652a0 con 0x55df6cb03380 2024-09-19T06:07:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd5f2ffd640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd5e80a33b0 0x7fd5e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6017bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb12a90 0x55df6cb14e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6017bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb12a90 0x55df6cb14e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36826/0 (socket says 172.21.15.5:36826) 2024-09-19T06:07:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6017bd640 1 -- 172.21.15.5:0/2294933412 learned_addr learned my addr 172.21.15.5:0/2294933412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6027bf640 1 --2- 172.21.15.5:0/2294933412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb03380 0x55df6cb03760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd601fbe640 1 --2- 172.21.15.5:0/2294933412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03ca0 0x55df6cb12550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6027bf640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd5e80a33b0 0x7fd5e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6027bf640 1 -- 172.21.15.5:0/2294933412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03ca0 msgr2=0x55df6cb12550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6027bf640 1 --2- 172.21.15.5:0/2294933412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03ca0 0x55df6cb12550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6027bf640 1 -- 172.21.15.5:0/2294933412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb12a90 msgr2=0x55df6cb14e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd5f2ffd640 1 -- 172.21.15.5:0/1752033902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fd5f80d1330 con 0x7fd5fc100e20 2024-09-19T06:07:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6027bf640 1 --2- 172.21.15.5:0/2294933412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb12a90 0x55df6cb14e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6027bf640 1 -- 172.21.15.5:0/2294933412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6c9f5440 con 0x55df6cb03380 2024-09-19T06:07:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.892+0000 7fd6017bd640 1 --2- 172.21.15.5:0/2294933412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb12a90 0x55df6cb14e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.893+0000 7fd6027bf640 1 --2- 172.21.15.5:0/2294933412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb03380 0x55df6cb03760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4005fa0 tx=0x7fd5f402fbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e788a20dd872109 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.893+0000 7fd6027bf640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd5e80a33b0 0x7fd5e80a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4047000 tx=0x7fd5f4049000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.893+0000 7fd5e7fff640 1 -- 172.21.15.5:0/2294933412 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f404e070 con 0x55df6cb03380 2024-09-19T06:07:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.893+0000 7fd5e7fff640 1 -- 172.21.15.5:0/2294933412 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5f402f6e0 con 0x55df6cb03380 2024-09-19T06:07:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.893+0000 7fd5e7fff640 1 -- 172.21.15.5:0/2294933412 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4034040 con 0x55df6cb03380 2024-09-19T06:07:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.894+0000 7fd6065fb740 1 -- 172.21.15.5:0/2294933412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb03380 msgr2=0x55df6cb03760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.894+0000 7fd6065fb740 1 --2- 172.21.15.5:0/2294933412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb03380 0x55df6cb03760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4005fa0 tx=0x7fd5f402fbe0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.894+0000 7fd6065fb740 1 -- 172.21.15.5:0/2294933412 shutdown_connections 2024-09-19T06:07:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.894+0000 7fd6065fb740 1 --2- 172.21.15.5:0/2294933412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb12a90 0x55df6cb14e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.894+0000 7fd6065fb740 1 --2- 172.21.15.5:0/2294933412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03ca0 0x55df6cb12550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.894+0000 7fd6065fb740 1 --2- 172.21.15.5:0/2294933412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb03380 0x55df6cb03760 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.894+0000 7fd6065fb740 1 -- 172.21.15.5:0/2294933412 >> 172.21.15.5:0/2294933412 conn(0x55df6caffa30 msgr2=0x55df6caffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.894+0000 7fd6065fb740 1 -- 172.21.15.5:0/2294933412 shutdown_connections 2024-09-19T06:07:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.894+0000 7fd6065fb740 1 -- 172.21.15.5:0/2294933412 wait complete. 2024-09-19T06:07:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.894+0000 7fd6065fb740 1 -- start start 2024-09-19T06:07:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd6065fb740 1 Processor -- start 2024-09-19T06:07:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd6065fb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03380 0x55df6cb12400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd6065fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 0x55df6cb0f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd6065fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb12a90 0x55df6cb0fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd6065fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55df6cb15660 con 0x55df6cb03ca0 2024-09-19T06:07:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd6065fb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55df6ca24050 con 0x55df6cb03380 2024-09-19T06:07:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd6065fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55df6cb157e0 con 0x55df6cb12a90 2024-09-19T06:07:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd6065fb740 10 client.0 _ll_register_callbacks cb 0x55df6ca64160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd6017bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 0x55df6cb0f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd6017bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 0x55df6cb0f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36832/0 (socket says 172.21.15.5:36832) 2024-09-19T06:07:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd6017bd640 1 -- 172.21.15.5:0/1631057080 learned_addr learned my addr 172.21.15.5:0/1631057080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd6027bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb12a90 0x55df6cb0fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.895+0000 7fd601fbe640 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03380 0x55df6cb12400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.896+0000 7fd6017bd640 1 -- 172.21.15.5:0/1631057080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb12a90 msgr2=0x55df6cb0fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.896+0000 7fd6017bd640 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb12a90 0x55df6cb0fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.896+0000 7fd6017bd640 1 -- 172.21.15.5:0/1631057080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03380 msgr2=0x55df6cb12400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.896+0000 7fd6017bd640 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03380 0x55df6cb12400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.896+0000 7fd6017bd640 1 -- 172.21.15.5:0/1631057080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6c7da3c0 con 0x55df6cb03ca0 2024-09-19T06:07:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.896+0000 7fd601fbe640 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03380 0x55df6cb12400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.896+0000 7fd6027bf640 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb12a90 0x55df6cb0fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.896+0000 7fd6017bd640 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 0x55df6cb0f700 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec012d20 tx=0x7fd5ec00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4643e7550f2a2a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.896+0000 7fd5e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:07:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.896+0000 7fd5e57fa640 1 -- 172.21.15.5:0/1631057080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec012110 con 0x55df6cb03ca0 2024-09-19T06:07:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.896+0000 7fd5e57fa640 1 -- 172.21.15.5:0/1631057080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fd5ec004050 con 0x55df6cb03ca0 2024-09-19T06:07:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.896+0000 7fd6065fb740 1 -- 172.21.15.5:0/1631057080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5f402f6e0 con 0x55df6cb03ca0 2024-09-19T06:07:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.897+0000 7fd5e57fa640 1 -- 172.21.15.5:0/1631057080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd5ec0122b0 con 0x55df6cb03ca0 2024-09-19T06:07:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.897+0000 7fd6065fb740 10 client.16494 fetch_fsmap learned FSMap version 33 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.897+0000 7fd6065fb740 1 -- 172.21.15.5:0/1631057080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55df6cb21160 con 0x55df6cb03ca0 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.897+0000 7fd5e57fa640 1 -- 172.21.15.5:0/1631057080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5f402f6e0 con 0x55df6cb03ca0 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.898+0000 7fd5e57fa640 1 -- 172.21.15.5:0/1631057080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fd5ec025470 con 0x55df6cb03ca0 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.898+0000 7fd6065fb740 10 client.16494 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.898+0000 7fd6065fb740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.898+0000 7fd6065fb740 20 client.16494 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.898+0000 7fd6065fb740 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55df6cb22570 0x55df6cb24990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.898+0000 7fd6065fb740 4 client.16494 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.898+0000 7fd6065fb740 1 -- 172.21.15.5:0/1631057080 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df6ca76850 con 0x55df6cb22570 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.898+0000 7fd601fbe640 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55df6cb22570 0x55df6cb24990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.898+0000 7fd5e57fa640 10 client.16494 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.898+0000 7fd601fbe640 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55df6cb22570 0x55df6cb24990 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=783a5345997a6628 server_cookie=9a00f834034cc48b in_seq=0 out_seq=0 2024-09-19T06:07:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd5e57fa640 1 -- 172.21.15.5:0/1631057080 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df6ca76850 con 0x55df6cb22570 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd5e57fa640 10 client.16494 handle_command_reply: tid=0 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd6065fb740 1 client.16494 shutdown 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd6065fb740 1 -- 172.21.15.5:0/1631057080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 msgr2=0x55df6cb0f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd6065fb740 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 0x55df6cb0f700 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec012d20 tx=0x7fd5ec00f790 comp rx=0 tx=0).stop 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd6065fb740 1 -- 172.21.15.5:0/1631057080 shutdown_connections 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd6065fb740 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55df6cb22570 0x55df6cb24990 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd6065fb740 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb12a90 0x55df6cb0fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd6065fb740 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 0x55df6cb0f700 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd5e57fa640 0 client.16494 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd6065fb740 1 --2- 172.21.15.5:0/1631057080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03380 0x55df6cb12400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd6065fb740 1 -- 172.21.15.5:0/1631057080 >> 172.21.15.5:0/1631057080 conn(0x55df6caffa30 msgr2=0x55df6cb05790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd6065fb740 1 -- 172.21.15.5:0/1631057080 shutdown_connections 2024-09-19T06:07:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.899+0000 7fd6065fb740 1 -- 172.21.15.5:0/1631057080 wait complete. 2024-09-19T06:07:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.900+0000 7fd6065fb740 20 client.16494 trim_cache size 0 max 16384 2024-09-19T06:07:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd6065fb740 1 Processor -- start 2024-09-19T06:07:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd6065fb740 1 -- start start 2024-09-19T06:07:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd6065fb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03380 0x55df6cb1e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd6065fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 0x55df6cb2a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd6065fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb12a90 0x55df6cb2a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd6065fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1ab630 con 0x55df6cb03ca0 2024-09-19T06:07:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd6065fb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1ad170 con 0x55df6cb03380 2024-09-19T06:07:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd6065fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1abd00 con 0x55df6cb12a90 2024-09-19T06:07:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd6017bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 0x55df6cb2a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd601fbe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03380 0x55df6cb1e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd6017bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 0x55df6cb2a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36848/0 (socket says 172.21.15.5:36848) 2024-09-19T06:07:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd6017bd640 1 -- 172.21.15.5:0/527179559 learned_addr learned my addr 172.21.15.5:0/527179559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd6027bf640 1 --2- 172.21.15.5:0/527179559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb12a90 0x55df6cb2a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd601fbe640 1 -- 172.21.15.5:0/527179559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb12a90 msgr2=0x55df6cb2a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd601fbe640 1 --2- 172.21.15.5:0/527179559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb12a90 0x55df6cb2a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd601fbe640 1 -- 172.21.15.5:0/527179559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 msgr2=0x55df6cb2a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.918+0000 7fd601fbe640 1 --2- 172.21.15.5:0/527179559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 0x55df6cb2a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.919+0000 7fd601fbe640 1 -- 172.21.15.5:0/527179559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6cb0dc50 con 0x55df6cb03380 2024-09-19T06:07:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.919+0000 7fd6017bd640 1 --2- 172.21.15.5:0/527179559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 0x55df6cb2a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.919+0000 7fd601fbe640 1 --2- 172.21.15.5:0/527179559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03380 0x55df6cb1e900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80cea30 tx=0x7fd5f80704a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=467f92b0a8f2076e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.919+0000 7fd5e77fe640 1 -- 172.21.15.5:0/527179559 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80a1970 con 0x55df6cb03380 2024-09-19T06:07:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.919+0000 7fd5e77fe640 1 -- 172.21.15.5:0/527179559 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5f80d0030 con 0x55df6cb03380 2024-09-19T06:07:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.919+0000 7fd5e77fe640 1 -- 172.21.15.5:0/527179559 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80cecb0 con 0x55df6cb03380 2024-09-19T06:07:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.920+0000 7fd6065fb740 1 -- 172.21.15.5:0/527179559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03380 msgr2=0x55df6cb1e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.920+0000 7fd6065fb740 1 --2- 172.21.15.5:0/527179559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03380 0x55df6cb1e900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80cea30 tx=0x7fd5f80704a0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.920+0000 7fd6065fb740 1 -- 172.21.15.5:0/527179559 shutdown_connections 2024-09-19T06:07:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.920+0000 7fd6065fb740 1 --2- 172.21.15.5:0/527179559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb12a90 0x55df6cb2a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.920+0000 7fd6065fb740 1 --2- 172.21.15.5:0/527179559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03ca0 0x55df6cb2a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.920+0000 7fd6065fb740 1 --2- 172.21.15.5:0/527179559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb03380 0x55df6cb1e900 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.920+0000 7fd6065fb740 1 -- 172.21.15.5:0/527179559 >> 172.21.15.5:0/527179559 conn(0x55df6caffa30 msgr2=0x55df6cb0e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.920+0000 7fd6065fb740 1 -- 172.21.15.5:0/527179559 shutdown_connections 2024-09-19T06:07:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.920+0000 7fd6065fb740 1 -- 172.21.15.5:0/527179559 wait complete. 2024-09-19T06:07:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.920+0000 7fd6065fb740 1 -- start start 2024-09-19T06:07:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.920+0000 7fd6065fb740 1 Processor -- start 2024-09-19T06:07:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6065fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03380 0x55df6cb161b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6065fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb03ca0 0x55df6cb166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6065fb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb12a90 0x55df6cb85260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6065fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1acaa0 con 0x55df6cb03380 2024-09-19T06:07:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6065fb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1ac3d0 con 0x55df6cb12a90 2024-09-19T06:07:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6065fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1a9610 con 0x55df6cb03ca0 2024-09-19T06:07:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6065fb740 10 client.0 _ll_register_callbacks cb 0x7fd5ec009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd601fbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03380 0x55df6cb161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd601fbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03380 0x55df6cb161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36858/0 (socket says 172.21.15.5:36858) 2024-09-19T06:07:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd601fbe640 1 -- 172.21.15.5:0/436070608 learned_addr learned my addr 172.21.15.5:0/436070608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6017bd640 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb03ca0 0x55df6cb166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6027bf640 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb12a90 0x55df6cb85260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6017bd640 1 -- 172.21.15.5:0/436070608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb12a90 msgr2=0x55df6cb85260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6017bd640 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb12a90 0x55df6cb85260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6017bd640 1 -- 172.21.15.5:0/436070608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03380 msgr2=0x55df6cb161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6017bd640 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03380 0x55df6cb161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.921+0000 7fd6017bd640 1 -- 172.21.15.5:0/436070608 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6cb858a0 con 0x55df6cb03ca0 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.922+0000 7fd601fbe640 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03380 0x55df6cb161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.922+0000 7fd6017bd640 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb03ca0 0x55df6cb166f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0059f0 tx=0x7fd5ec025c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503b3da401ac164d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.922+0000 7fd5e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.922+0000 7fd5e57fa640 1 -- 172.21.15.5:0/436070608 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec005320 con 0x55df6cb03ca0 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.922+0000 7fd5e57fa640 1 -- 172.21.15.5:0/436070608 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fd5ec0054c0 con 0x55df6cb03ca0 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.922+0000 7fd6065fb740 1 -- 172.21.15.5:0/436070608 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5f80d0030 con 0x55df6cb03ca0 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.922+0000 7fd5e57fa640 1 -- 172.21.15.5:0/436070608 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd5ec0056c0 con 0x55df6cb03ca0 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.922+0000 7fd6065fb740 10 client.26011 fetch_fsmap learned FSMap version 33 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.922+0000 7fd6065fb740 1 -- 172.21.15.5:0/436070608 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55df6cb8cc00 con 0x55df6cb03ca0 2024-09-19T06:07:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.922+0000 7fd5e57fa640 1 -- 172.21.15.5:0/436070608 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5f80d0030 con 0x55df6cb03ca0 2024-09-19T06:07:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd5e57fa640 1 -- 172.21.15.5:0/436070608 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fd5ec038500 con 0x55df6cb03ca0 2024-09-19T06:07:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd6065fb740 10 client.26011 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:07:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd6065fb740 10 client.26011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd6065fb740 20 client.26011 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd6065fb740 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55df6cb8d2f0 0x55df6cb8f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd6065fb740 4 client.26011 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:07:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd6065fb740 1 -- 172.21.15.5:0/436070608 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df6ca70d60 con 0x55df6cb8d2f0 2024-09-19T06:07:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd601fbe640 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55df6cb8d2f0 0x55df6cb8f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd5e57fa640 10 client.26011 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd601fbe640 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55df6cb8d2f0 0x55df6cb8f710 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc354affaccb7c3f server_cookie=f8998dae6fe71114 in_seq=0 out_seq=0 2024-09-19T06:07:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd5e57fa640 1 -- 172.21.15.5:0/436070608 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55df6ca70d60 con 0x55df6cb8d2f0 2024-09-19T06:07:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd5e57fa640 10 client.26011 handle_command_reply: tid=0 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.923+0000 7fd6065fb740 1 client.26011 shutdown 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.924+0000 7fd6065fb740 1 -- 172.21.15.5:0/436070608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb03ca0 msgr2=0x55df6cb166f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.924+0000 7fd6065fb740 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb03ca0 0x55df6cb166f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0059f0 tx=0x7fd5ec025c90 comp rx=0 tx=0).stop 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.924+0000 7fd6065fb740 1 -- 172.21.15.5:0/436070608 shutdown_connections 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.924+0000 7fd6065fb740 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55df6cb8d2f0 0x55df6cb8f710 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.924+0000 7fd6065fb740 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df6cb12a90 0x55df6cb85260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.924+0000 7fd6065fb740 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df6cb03ca0 0x55df6cb166f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.924+0000 7fd6065fb740 1 --2- 172.21.15.5:0/436070608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df6cb03380 0x55df6cb161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.924+0000 7fd5e57fa640 0 client.26011 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.924+0000 7fd6065fb740 1 -- 172.21.15.5:0/436070608 >> 172.21.15.5:0/436070608 conn(0x55df6caffa30 msgr2=0x55df6cb0a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.924+0000 7fd6065fb740 1 -- 172.21.15.5:0/436070608 shutdown_connections 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.924+0000 7fd6065fb740 1 -- 172.21.15.5:0/436070608 wait complete. 2024-09-19T06:07:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.924+0000 7fd6065fb740 20 client.26011 trim_cache size 0 max 16384 2024-09-19T06:07:28.932 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:07:28.932 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:07:28.932 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:07:28.932 INFO:teuthology.orchestra.run.smithi005.stdout: "22ed6c5d-d3f0-47fe-a891-a75636dd014a": { 2024-09-19T06:07:28.932 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:07:28.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "22ed6c5d-d3f0-47fe-a891-a75636dd014a", 2024-09-19T06:07:28.932 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:07:28.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:07:28.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:07:28.932 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:07:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.925+0000 7fd604222640 1 -- 172.21.15.5:0/1752033902 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd5e80a33b0 msgr2=0x7fd5e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.925+0000 7fd604222640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd5e80a33b0 0x7fd5e80a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4047000 tx=0x7fd5f4049000 comp rx=0 tx=0).stop 2024-09-19T06:07:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.925+0000 7fd604222640 1 -- 172.21.15.5:0/1752033902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc100e20 msgr2=0x7fd5fc197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.925+0000 7fd604222640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc100e20 0x7fd5fc197380 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800eb00 tx=0x7fd5f8009f20 comp rx=0 tx=0).stop 2024-09-19T06:07:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.925+0000 7fd604222640 1 -- 172.21.15.5:0/1752033902 shutdown_connections 2024-09-19T06:07:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.925+0000 7fd604222640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5fc102ef0 0x7fd5fc19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.925+0000 7fd604222640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd5e80a33b0 0x7fd5e80a5870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.925+0000 7fd604222640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc102570 0x7fd5fc1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.925+0000 7fd604222640 1 --2- 172.21.15.5:0/1752033902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc100e20 0x7fd5fc197380 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.925+0000 7fd604222640 1 -- 172.21.15.5:0/1752033902 >> 172.21.15.5:0/1752033902 conn(0x7fd5fc0fc7d0 msgr2=0x7fd5fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.925+0000 7fd604222640 1 -- 172.21.15.5:0/1752033902 shutdown_connections 2024-09-19T06:07:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:07:28.925+0000 7fd604222640 1 -- 172.21.15.5:0/1752033902 wait complete. 2024-09-19T06:07:29.710 INFO:tasks.cephfs.filesystem:scrub status for tag:22ed6c5d-d3f0-47fe-a891-a75636dd014a - {'path': '/', 'tag': '22ed6c5d-d3f0-47fe-a891-a75636dd014a', 'options': 'recursive,force'} 2024-09-19T06:07:29.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 91126 2.78 MB/sec execute 55 sec latency 15.184 ms 2024-09-19T06:07:30.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:29 smithi157 ceph-mon[35346]: pgmap v128: 97 pgs: 97 active+clean; 63 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:07:30.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:29 smithi005 ceph-mon[30955]: pgmap v128: 97 pgs: 97 active+clean; 63 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:07:30.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:29 smithi049 ceph-mon[34640]: pgmap v128: 97 pgs: 97 active+clean; 63 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:07:30.826 INFO:tasks.workunit.client.0.smithi005.stdout: 1 91893 2.84 MB/sec execute 56 sec latency 11.586 ms 2024-09-19T06:07:31.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 92354 2.87 MB/sec execute 57 sec latency 15.045 ms 2024-09-19T06:07:32.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:31 smithi157 ceph-mon[35346]: pgmap v129: 97 pgs: 97 active+clean; 63 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:07:32.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:31 smithi005 ceph-mon[30955]: pgmap v129: 97 pgs: 97 active+clean; 63 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:07:32.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:31 smithi049 ceph-mon[34640]: pgmap v129: 97 pgs: 97 active+clean; 63 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:07:32.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 92668 2.84 MB/sec execute 58 sec latency 15.132 ms 2024-09-19T06:07:33.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:32 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:07:33.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:32 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:07:33.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:32 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:07:33.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 92992 2.80 MB/sec execute 59 sec latency 15.171 ms 2024-09-19T06:07:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:33 smithi157 ceph-mon[35346]: pgmap v130: 97 pgs: 97 active+clean; 60 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-19T06:07:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:33 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:07:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:33 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:07:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:33 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:07:34.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:33 smithi005 ceph-mon[30955]: pgmap v130: 97 pgs: 97 active+clean; 60 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-19T06:07:34.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:33 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:07:34.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:33 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:07:34.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:33 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:07:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:33 smithi049 ceph-mon[34640]: pgmap v130: 97 pgs: 97 active+clean; 60 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-19T06:07:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:33 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:07:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:33 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:07:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:33 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:07:34.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 93528 2.79 MB/sec execute 60 sec latency 15.142 ms 2024-09-19T06:07:35.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:34 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:07:35.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:34 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:07:35.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:34 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:07:35.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:34 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:07:35.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:34 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:07:35.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:34 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:07:35.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:34 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:07:35.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:34 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:07:35.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:34 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:07:35.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 94199 2.80 MB/sec execute 61 sec latency 15.236 ms 2024-09-19T06:07:36.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:35 smithi005 ceph-mon[30955]: pgmap v131: 97 pgs: 97 active+clean; 60 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-19T06:07:36.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:35 smithi049 ceph-mon[34640]: pgmap v131: 97 pgs: 97 active+clean; 60 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-19T06:07:36.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:35 smithi157 ceph-mon[35346]: pgmap v131: 97 pgs: 97 active+clean; 60 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-19T06:07:36.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 94821 2.79 MB/sec execute 62 sec latency 12.669 ms 2024-09-19T06:07:37.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 95566 2.85 MB/sec execute 63 sec latency 15.065 ms 2024-09-19T06:07:38.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:37 smithi005 ceph-mon[30955]: pgmap v132: 97 pgs: 97 active+clean; 60 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-09-19T06:07:38.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:37 smithi049 ceph-mon[34640]: pgmap v132: 97 pgs: 97 active+clean; 60 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-09-19T06:07:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:37 smithi157 ceph-mon[35346]: pgmap v132: 97 pgs: 97 active+clean; 60 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-09-19T06:07:38.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 96039 2.87 MB/sec execute 64 sec latency 12.048 ms 2024-09-19T06:07:39.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 96288 2.83 MB/sec execute 65 sec latency 12.930 ms 2024-09-19T06:07:40.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:39 smithi005 ceph-mon[30955]: pgmap v133: 97 pgs: 97 active+clean; 60 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-09-19T06:07:40.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:39 smithi049 ceph-mon[34640]: pgmap v133: 97 pgs: 97 active+clean; 60 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-09-19T06:07:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:39 smithi157 ceph-mon[35346]: pgmap v133: 97 pgs: 97 active+clean; 60 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-09-19T06:07:40.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 96748 2.80 MB/sec execute 66 sec latency 11.597 ms 2024-09-19T06:07:41.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 97325 2.82 MB/sec execute 67 sec latency 14.607 ms 2024-09-19T06:07:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.857+0000 7f569a9fd640 1 -- 172.21.15.5:0/1813692615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694106a00 msgr2=0x7f5694106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.857+0000 7f569a9fd640 1 --2- 172.21.15.5:0/1813692615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694106a00 0x7f5694106de0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5684009930 tx=0x7f568402a420 comp rx=0 tx=0).stop 2024-09-19T06:07:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.857+0000 7f569a9fd640 1 -- 172.21.15.5:0/1813692615 shutdown_connections 2024-09-19T06:07:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.857+0000 7f569a9fd640 1 --2- 172.21.15.5:0/1813692615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5694101390 0x7f569410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.857+0000 7f569a9fd640 1 --2- 172.21.15.5:0/1813692615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56941009f0 0x7f5694100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.857+0000 7f569a9fd640 1 --2- 172.21.15.5:0/1813692615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694106a00 0x7f5694106de0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.857+0000 7f569a9fd640 1 -- 172.21.15.5:0/1813692615 >> 172.21.15.5:0/1813692615 conn(0x7f56940fc6f0 msgr2=0x7f56940feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.857+0000 7f569a9fd640 1 -- 172.21.15.5:0/1813692615 shutdown_connections 2024-09-19T06:07:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.857+0000 7f569a9fd640 1 -- 172.21.15.5:0/1813692615 wait complete. 2024-09-19T06:07:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f569a9fd640 1 Processor -- start 2024-09-19T06:07:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f569a9fd640 1 -- start start 2024-09-19T06:07:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f569a9fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56941009f0 0x7f5694198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f569a9fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694101390 0x7f56941988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f569a9fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5694106a00 0x7f569419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f569a9fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5694110cf0 con 0x7f5694101390 2024-09-19T06:07:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f569a9fd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5694110b70 con 0x7f5694106a00 2024-09-19T06:07:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f569a9fd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5694110e70 con 0x7f56941009f0 2024-09-19T06:07:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f56937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694101390 0x7f56941988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f56937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694101390 0x7f56941988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42624/0 (socket says 172.21.15.5:42624) 2024-09-19T06:07:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f56937fe640 1 -- 172.21.15.5:0/3024019497 learned_addr learned my addr 172.21.15.5:0/3024019497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f5698f9a640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5694106a00 0x7f569419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.858+0000 7f5693fff640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56941009f0 0x7f5694198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.859+0000 7f56937fe640 1 -- 172.21.15.5:0/3024019497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56941009f0 msgr2=0x7f5694198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.859+0000 7f56937fe640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56941009f0 0x7f5694198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.859+0000 7f56937fe640 1 -- 172.21.15.5:0/3024019497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5694106a00 msgr2=0x7f569419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.859+0000 7f56937fe640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5694106a00 0x7f569419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.859+0000 7f56937fe640 1 -- 172.21.15.5:0/3024019497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569419d350 con 0x7f5694101390 2024-09-19T06:07:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.859+0000 7f5698f9a640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5694106a00 0x7f569419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.859+0000 7f56937fe640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694101390 0x7f56941988a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f568800e9f0 tx=0x7f568800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.859+0000 7f56917fa640 1 -- 172.21.15.5:0/3024019497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568800cd80 con 0x7f5694101390 2024-09-19T06:07:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.859+0000 7f56917fa640 1 -- 172.21.15.5:0/3024019497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56880054d0 con 0x7f5694101390 2024-09-19T06:07:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.859+0000 7f569a9fd640 1 -- 172.21.15.5:0/3024019497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569419d640 con 0x7f5694101390 2024-09-19T06:07:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.859+0000 7f56917fa640 1 -- 172.21.15.5:0/3024019497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688010490 con 0x7f5694101390 2024-09-19T06:07:41.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.859+0000 7f569a9fd640 1 -- 172.21.15.5:0/3024019497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56941a4ed0 con 0x7f5694101390 2024-09-19T06:07:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.861+0000 7f56917fa640 1 -- 172.21.15.5:0/3024019497 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5688005020 con 0x7f5694101390 2024-09-19T06:07:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f569d87e740 1 Processor -- start 2024-09-19T06:07:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f569d87e740 1 -- start start 2024-09-19T06:07:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f569d87e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e612daf0 0x55b7e612ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f569d87e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612e410 0x55b7e6138770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f569d87e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e6138cb0 0x55b7e613b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f569d87e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b7e60922f0 con 0x55b7e612e410 2024-09-19T06:07:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f569d87e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b7e603a270 con 0x55b7e6138cb0 2024-09-19T06:07:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f569d87e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b7e608e2b0 con 0x55b7e612daf0 2024-09-19T06:07:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f56917fa640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f567c0a33b0 0x7f567c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f5693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612e410 0x55b7e6138770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f5698f9a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e612daf0 0x55b7e612ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f5693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612e410 0x55b7e6138770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42626/0 (socket says 172.21.15.5:42626) 2024-09-19T06:07:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f5693fff640 1 -- 172.21.15.5:0/733714553 learned_addr learned my addr 172.21.15.5:0/733714553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.862+0000 7f56937fe640 1 --2- 172.21.15.5:0/733714553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e6138cb0 0x55b7e613b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f56917fa640 1 -- 172.21.15.5:0/3024019497 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f56880d0070 con 0x7f5694101390 2024-09-19T06:07:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f5698f9a640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f567c0a33b0 0x7f567c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f5693fff640 1 -- 172.21.15.5:0/733714553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e612daf0 msgr2=0x55b7e612ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f5693fff640 1 --2- 172.21.15.5:0/733714553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e612daf0 0x55b7e612ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f5693fff640 1 -- 172.21.15.5:0/733714553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e6138cb0 msgr2=0x55b7e613b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f5693fff640 1 --2- 172.21.15.5:0/733714553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e6138cb0 0x55b7e613b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f5693fff640 1 -- 172.21.15.5:0/733714553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7e6039340 con 0x55b7e612e410 2024-09-19T06:07:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f56937fe640 1 --2- 172.21.15.5:0/733714553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e6138cb0 0x55b7e613b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f5693fff640 1 --2- 172.21.15.5:0/733714553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612e410 0x55b7e6138770 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f568402a940 tx=0x7f5684005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aafd498309fe7cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f5698f9a640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f567c0a33b0 0x7f567c0a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f568c0094d0 tx=0x7f568c004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f567a7fc640 1 -- 172.21.15.5:0/733714553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684040070 con 0x55b7e612e410 2024-09-19T06:07:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f567a7fc640 1 -- 172.21.15.5:0/733714553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5684004030 con 0x55b7e612e410 2024-09-19T06:07:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.863+0000 7f567a7fc640 1 -- 172.21.15.5:0/733714553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684035040 con 0x55b7e612e410 2024-09-19T06:07:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.864+0000 7f569d87e740 1 -- 172.21.15.5:0/733714553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612e410 msgr2=0x55b7e6138770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.864+0000 7f569d87e740 1 --2- 172.21.15.5:0/733714553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612e410 0x55b7e6138770 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f568402a940 tx=0x7f5684005be0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.864+0000 7f569d87e740 1 -- 172.21.15.5:0/733714553 shutdown_connections 2024-09-19T06:07:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.864+0000 7f569d87e740 1 --2- 172.21.15.5:0/733714553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e6138cb0 0x55b7e613b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.864+0000 7f569d87e740 1 --2- 172.21.15.5:0/733714553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612e410 0x55b7e6138770 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.864+0000 7f569d87e740 1 --2- 172.21.15.5:0/733714553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e612daf0 0x55b7e612ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.864+0000 7f569d87e740 1 -- 172.21.15.5:0/733714553 >> 172.21.15.5:0/733714553 conn(0x55b7e612a060 msgr2=0x55b7e612a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.864+0000 7f569d87e740 1 -- 172.21.15.5:0/733714553 shutdown_connections 2024-09-19T06:07:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.864+0000 7f569d87e740 1 -- 172.21.15.5:0/733714553 wait complete. 2024-09-19T06:07:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.864+0000 7f569d87e740 1 -- start start 2024-09-19T06:07:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f569d87e740 1 Processor -- start 2024-09-19T06:07:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f569d87e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e6135b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f569d87e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e612e410 0x55b7e6136070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f569d87e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e6138cb0 0x55b7e61365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f569d87e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b7e613b880 con 0x55b7e612daf0 2024-09-19T06:07:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f569d87e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b7e601b290 con 0x55b7e612e410 2024-09-19T06:07:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f569d87e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b7e613ba00 con 0x55b7e6138cb0 2024-09-19T06:07:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f569d87e740 10 client.0 _ll_register_callbacks cb 0x55b7e604c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f5693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e6135b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f56937fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e612e410 0x55b7e6136070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f5693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e6135b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42628/0 (socket says 172.21.15.5:42628) 2024-09-19T06:07:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f5693fff640 1 -- 172.21.15.5:0/1830972276 learned_addr learned my addr 172.21.15.5:0/1830972276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f5698f9a640 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e6138cb0 0x55b7e61365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f56937fe640 1 -- 172.21.15.5:0/1830972276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e6138cb0 msgr2=0x55b7e61365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f56937fe640 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e6138cb0 0x55b7e61365b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f56937fe640 1 -- 172.21.15.5:0/1830972276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 msgr2=0x55b7e6135b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f56937fe640 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e6135b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f56937fe640 1 -- 172.21.15.5:0/1830972276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7e5da5210 con 0x55b7e612e410 2024-09-19T06:07:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.865+0000 7f5693fff640 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e6135b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f56937fe640 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e612e410 0x55b7e6136070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f56880cd7d0 tx=0x7f568806c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f5ad90ead55dd90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f56737fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:07:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f56737fe640 1 -- 172.21.15.5:0/1830972276 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688005680 con 0x55b7e612e410 2024-09-19T06:07:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f56737fe640 1 -- 172.21.15.5:0/1830972276 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f56880989d0 con 0x55b7e612e410 2024-09-19T06:07:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f569d87e740 1 -- 172.21.15.5:0/1830972276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5684004030 con 0x55b7e612e410 2024-09-19T06:07:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f56737fe640 1 -- 172.21.15.5:0/1830972276 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f568809ca60 con 0x55b7e612e410 2024-09-19T06:07:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f569d87e740 10 client.26090 fetch_fsmap learned FSMap version 33 2024-09-19T06:07:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f569d87e740 1 -- 172.21.15.5:0/1830972276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b7e5e71620 con 0x55b7e612e410 2024-09-19T06:07:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f56737fe640 1 -- 172.21.15.5:0/1830972276 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5684004030 con 0x55b7e612e410 2024-09-19T06:07:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f56737fe640 1 -- 172.21.15.5:0/1830972276 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f56880d20d0 con 0x55b7e612e410 2024-09-19T06:07:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f569d87e740 10 client.26090 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:07:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f569d87e740 10 client.26090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f569d87e740 20 client.26090 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f569d87e740 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b7e6148b30 0x55b7e614af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f569d87e740 4 client.26090 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.866+0000 7f569d87e740 1 -- 172.21.15.5:0/1830972276 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7e609fa20 con 0x55b7e6148b30 2024-09-19T06:07:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.867+0000 7f5693fff640 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b7e6148b30 0x55b7e614af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.867+0000 7f5693fff640 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b7e6148b30 0x55b7e614af40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1f1b0ff3d72fe23 server_cookie=ece1f31198b5fcad in_seq=0 out_seq=0 2024-09-19T06:07:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.867+0000 7f56737fe640 10 client.26090 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f56737fe640 1 -- 172.21.15.5:0/1830972276 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7e609fa20 con 0x55b7e6148b30 2024-09-19T06:07:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f56737fe640 10 client.26090 handle_command_reply: tid=0 2024-09-19T06:07:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f569d87e740 1 client.26090 shutdown 2024-09-19T06:07:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f569d87e740 1 -- 172.21.15.5:0/1830972276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e612e410 msgr2=0x55b7e6136070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f569d87e740 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e612e410 0x55b7e6136070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f56880cd7d0 tx=0x7f568806c040 comp rx=0 tx=0).stop 2024-09-19T06:07:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f569d87e740 1 -- 172.21.15.5:0/1830972276 shutdown_connections 2024-09-19T06:07:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f569d87e740 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b7e6148b30 0x55b7e614af40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f569d87e740 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e6138cb0 0x55b7e61365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f569d87e740 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e612e410 0x55b7e6136070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f569d87e740 1 --2- 172.21.15.5:0/1830972276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e6135b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f56737fe640 0 client.26090 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f569d87e740 1 -- 172.21.15.5:0/1830972276 >> 172.21.15.5:0/1830972276 conn(0x55b7e612a060 msgr2=0x55b7e612fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f569d87e740 1 -- 172.21.15.5:0/1830972276 shutdown_connections 2024-09-19T06:07:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f569d87e740 1 -- 172.21.15.5:0/1830972276 wait complete. 2024-09-19T06:07:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.868+0000 7f569d87e740 20 client.26090 trim_cache size 0 max 16384 2024-09-19T06:07:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.887+0000 7f569d87e740 1 Processor -- start 2024-09-19T06:07:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.887+0000 7f569d87e740 1 -- start start 2024-09-19T06:07:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.887+0000 7f569d87e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e61363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.887+0000 7f569d87e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e612e410 0x55b7e6150820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.887+0000 7f569d87e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e6138cb0 0x55b7e6150d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.887+0000 7f569d87e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56941ab5b0 con 0x55b7e612daf0 2024-09-19T06:07:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.887+0000 7f569d87e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56941ad0f0 con 0x55b7e612e410 2024-09-19T06:07:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.887+0000 7f569d87e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56941abc80 con 0x55b7e6138cb0 2024-09-19T06:07:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e61363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e61363e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42644/0 (socket says 172.21.15.5:42644) 2024-09-19T06:07:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5693fff640 1 -- 172.21.15.5:0/532395670 learned_addr learned my addr 172.21.15.5:0/532395670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f56937fe640 1 --2- 172.21.15.5:0/532395670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e612e410 0x55b7e6150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5698f9a640 1 --2- 172.21.15.5:0/532395670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e6138cb0 0x55b7e6150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5693fff640 1 -- 172.21.15.5:0/532395670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e6138cb0 msgr2=0x55b7e6150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5693fff640 1 --2- 172.21.15.5:0/532395670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e6138cb0 0x55b7e6150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5693fff640 1 -- 172.21.15.5:0/532395670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e612e410 msgr2=0x55b7e6150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5693fff640 1 --2- 172.21.15.5:0/532395670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e612e410 0x55b7e6150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5693fff640 1 -- 172.21.15.5:0/532395670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7e614b590 con 0x55b7e612daf0 2024-09-19T06:07:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5698f9a640 1 --2- 172.21.15.5:0/532395670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e6138cb0 0x55b7e6150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:07:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5693fff640 1 --2- 172.21.15.5:0/532395670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e61363e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x55b7e614b7c0 tx=0x7f5684047040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961ef4a73c26d473 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5679ffb640 1 -- 172.21.15.5:0/532395670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684056070 con 0x55b7e612daf0 2024-09-19T06:07:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5679ffb640 1 -- 172.21.15.5:0/532395670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56840053c0 con 0x55b7e612daf0 2024-09-19T06:07:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.888+0000 7f5679ffb640 1 -- 172.21.15.5:0/532395670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684040040 con 0x55b7e612daf0 2024-09-19T06:07:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.889+0000 7f569d87e740 1 -- 172.21.15.5:0/532395670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 msgr2=0x55b7e61363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.889+0000 7f569d87e740 1 --2- 172.21.15.5:0/532395670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e61363e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x55b7e614b7c0 tx=0x7f5684047040 comp rx=0 tx=0).stop 2024-09-19T06:07:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.889+0000 7f569d87e740 1 -- 172.21.15.5:0/532395670 shutdown_connections 2024-09-19T06:07:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.889+0000 7f569d87e740 1 --2- 172.21.15.5:0/532395670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e6138cb0 0x55b7e6150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.889+0000 7f569d87e740 1 --2- 172.21.15.5:0/532395670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e612e410 0x55b7e6150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.889+0000 7f569d87e740 1 --2- 172.21.15.5:0/532395670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e61363e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.889+0000 7f569d87e740 1 -- 172.21.15.5:0/532395670 >> 172.21.15.5:0/532395670 conn(0x55b7e612a060 msgr2=0x55b7e6137bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.889+0000 7f569d87e740 1 -- 172.21.15.5:0/532395670 shutdown_connections 2024-09-19T06:07:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.889+0000 7f569d87e740 1 -- 172.21.15.5:0/532395670 wait complete. 2024-09-19T06:07:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.889+0000 7f569d87e740 1 -- start start 2024-09-19T06:07:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f569d87e740 1 Processor -- start 2024-09-19T06:07:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f569d87e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e6141350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f569d87e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e612e410 0x55b7e6132800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f569d87e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e6138cb0 0x55b7e6132d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f569d87e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56941aca20 con 0x55b7e612daf0 2024-09-19T06:07:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f569d87e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56941ac350 con 0x55b7e6138cb0 2024-09-19T06:07:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f569d87e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56941a95b0 con 0x55b7e612e410 2024-09-19T06:07:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f569d87e740 10 client.0 _ll_register_callbacks cb 0x7f5688004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f5693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e6141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f5693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e6141350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42660/0 (socket says 172.21.15.5:42660) 2024-09-19T06:07:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f5693fff640 1 -- 172.21.15.5:0/449978075 learned_addr learned my addr 172.21.15.5:0/449978075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f5698f9a640 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e6138cb0 0x55b7e6132d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f56937fe640 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e612e410 0x55b7e6132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f5693fff640 1 -- 172.21.15.5:0/449978075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e612e410 msgr2=0x55b7e6132800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f5693fff640 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e612e410 0x55b7e6132800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f5693fff640 1 -- 172.21.15.5:0/449978075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e6138cb0 msgr2=0x55b7e6132d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f5693fff640 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e6138cb0 0x55b7e6132d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.890+0000 7f5693fff640 1 -- 172.21.15.5:0/449978075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7e6129e90 con 0x55b7e612daf0 2024-09-19T06:07:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.891+0000 7f56937fe640 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e612e410 0x55b7e6132800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:07:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.891+0000 7f5693fff640 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e6141350 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5684006010 tx=0x7f5684009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5699320f6d23868 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.891+0000 7f56737fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:07:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.891+0000 7f56737fe640 1 -- 172.21.15.5:0/449978075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684056070 con 0x55b7e612daf0 2024-09-19T06:07:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.891+0000 7f56737fe640 1 -- 172.21.15.5:0/449978075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f5684005570 con 0x55b7e612daf0 2024-09-19T06:07:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.891+0000 7f569d87e740 1 -- 172.21.15.5:0/449978075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56840053c0 con 0x55b7e612daf0 2024-09-19T06:07:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.891+0000 7f56737fe640 1 -- 172.21.15.5:0/449978075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5684040040 con 0x55b7e612daf0 2024-09-19T06:07:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.891+0000 7f569d87e740 10 client.16518 fetch_fsmap learned FSMap version 33 2024-09-19T06:07:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.891+0000 7f569d87e740 1 -- 172.21.15.5:0/449978075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b7e613fd30 con 0x55b7e612daf0 2024-09-19T06:07:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.891+0000 7f56737fe640 1 -- 172.21.15.5:0/449978075 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56840053c0 con 0x55b7e612daf0 2024-09-19T06:07:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.891+0000 7f56737fe640 1 -- 172.21.15.5:0/449978075 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f56840048f0 con 0x55b7e612daf0 2024-09-19T06:07:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.891+0000 7f569d87e740 10 client.16518 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:07:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.892+0000 7f569d87e740 10 client.16518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.892+0000 7f569d87e740 20 client.16518 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.892+0000 7f569d87e740 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b7e61b38a0 0x55b7e61b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.892+0000 7f569d87e740 4 client.16518 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:07:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.892+0000 7f569d87e740 1 -- 172.21.15.5:0/449978075 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7e6099d70 con 0x55b7e61b38a0 2024-09-19T06:07:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.892+0000 7f56937fe640 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b7e61b38a0 0x55b7e61b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.892+0000 7f56937fe640 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b7e61b38a0 0x55b7e61b5c90 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8c2fc930fa5e25 server_cookie=8813bdf66fec63e1 in_seq=0 out_seq=0 2024-09-19T06:07:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.892+0000 7f56737fe640 10 client.16518 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.909+0000 7f56737fe640 1 -- 172.21.15.5:0/449978075 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b7e6099d70 con 0x55b7e61b38a0 2024-09-19T06:07:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.909+0000 7f56737fe640 10 client.16518 handle_command_reply: tid=0 2024-09-19T06:07:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.909+0000 7f569d87e740 1 client.16518 shutdown 2024-09-19T06:07:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.910+0000 7f569d87e740 1 -- 172.21.15.5:0/449978075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 msgr2=0x55b7e6141350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.910+0000 7f569d87e740 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e6141350 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5684006010 tx=0x7f5684009880 comp rx=0 tx=0).stop 2024-09-19T06:07:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.910+0000 7f569d87e740 1 -- 172.21.15.5:0/449978075 shutdown_connections 2024-09-19T06:07:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.910+0000 7f569d87e740 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b7e61b38a0 0x55b7e61b5c90 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.910+0000 7f569d87e740 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7e6138cb0 0x55b7e6132d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.910+0000 7f569d87e740 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e612e410 0x55b7e6132800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.910+0000 7f56737fe640 0 client.16518 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.910+0000 7f569d87e740 1 --2- 172.21.15.5:0/449978075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7e612daf0 0x55b7e6141350 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.910+0000 7f569d87e740 1 -- 172.21.15.5:0/449978075 >> 172.21.15.5:0/449978075 conn(0x55b7e612a060 msgr2=0x55b7e612f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.910+0000 7f569d87e740 1 -- 172.21.15.5:0/449978075 shutdown_connections 2024-09-19T06:07:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.910+0000 7f569d87e740 1 -- 172.21.15.5:0/449978075 wait complete. 2024-09-19T06:07:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.910+0000 7f569d87e740 20 client.16518 trim_cache size 0 max 16384 2024-09-19T06:07:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:07:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:07:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:07:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:07:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.911+0000 7f569a9fd640 1 -- 172.21.15.5:0/3024019497 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f567c0a33b0 msgr2=0x7f567c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.911+0000 7f569a9fd640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f567c0a33b0 0x7f567c0a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f568c0094d0 tx=0x7f568c004ab0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.911+0000 7f569a9fd640 1 -- 172.21.15.5:0/3024019497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694101390 msgr2=0x7f56941988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.911+0000 7f569a9fd640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694101390 0x7f56941988a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f568800e9f0 tx=0x7f568800edf0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.911+0000 7f569a9fd640 1 -- 172.21.15.5:0/3024019497 shutdown_connections 2024-09-19T06:07:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.911+0000 7f569a9fd640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5694106a00 0x7f569419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.911+0000 7f569a9fd640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694101390 0x7f56941988a0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.911+0000 7f569a9fd640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f567c0a33b0 0x7f567c0a5870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.911+0000 7f569a9fd640 1 --2- 172.21.15.5:0/3024019497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56941009f0 0x7f5694198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.911+0000 7f569a9fd640 1 -- 172.21.15.5:0/3024019497 >> 172.21.15.5:0/3024019497 conn(0x7f56940fc6f0 msgr2=0x7f5694104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.911+0000 7f569a9fd640 1 -- 172.21.15.5:0/3024019497 shutdown_connections 2024-09-19T06:07:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:41.911+0000 7f569a9fd640 1 -- 172.21.15.5:0/3024019497 wait complete. 2024-09-19T06:07:42.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:41 smithi005 ceph-mon[30955]: pgmap v134: 97 pgs: 97 active+clean; 60 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-09-19T06:07:42.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:41 smithi049 ceph-mon[34640]: pgmap v134: 97 pgs: 97 active+clean; 60 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-09-19T06:07:42.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:41 smithi157 ceph-mon[35346]: pgmap v134: 97 pgs: 97 active+clean; 60 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-09-19T06:07:42.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 97994 2.79 MB/sec execute 68 sec latency 15.789 ms 2024-09-19T06:07:43.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 98586 2.83 MB/sec execute 69 sec latency 11.625 ms 2024-09-19T06:07:44.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:43 smithi005 ceph-mon[30955]: pgmap v135: 97 pgs: 97 active+clean; 65 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-19T06:07:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:43 smithi049 ceph-mon[34640]: pgmap v135: 97 pgs: 97 active+clean; 65 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-19T06:07:44.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:43 smithi157 ceph-mon[35346]: pgmap v135: 97 pgs: 97 active+clean; 65 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-19T06:07:44.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 99156 2.84 MB/sec execute 70 sec latency 14.754 ms 2024-09-19T06:07:45.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 99653 2.86 MB/sec execute 71 sec latency 15.150 ms 2024-09-19T06:07:46.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:45 smithi005 ceph-mon[30955]: pgmap v136: 97 pgs: 97 active+clean; 65 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-09-19T06:07:46.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:45 smithi049 ceph-mon[34640]: pgmap v136: 97 pgs: 97 active+clean; 65 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-09-19T06:07:46.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:45 smithi157 ceph-mon[35346]: pgmap v136: 97 pgs: 97 active+clean; 65 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-09-19T06:07:46.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 99941 2.82 MB/sec execute 72 sec latency 13.895 ms 2024-09-19T06:07:47.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 100458 2.81 MB/sec execute 73 sec latency 15.093 ms 2024-09-19T06:07:48.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:47 smithi005 ceph-mon[30955]: pgmap v137: 97 pgs: 97 active+clean; 67 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-19T06:07:48.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:47 smithi049 ceph-mon[34640]: pgmap v137: 97 pgs: 97 active+clean; 67 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-19T06:07:48.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:47 smithi157 ceph-mon[35346]: pgmap v137: 97 pgs: 97 active+clean; 67 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-19T06:07:48.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 101054 2.82 MB/sec execute 74 sec latency 16.127 ms 2024-09-19T06:07:49.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 101728 2.80 MB/sec execute 75 sec latency 20.513 ms 2024-09-19T06:07:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:49 smithi005 ceph-mon[30955]: pgmap v138: 97 pgs: 97 active+clean; 67 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-19T06:07:50.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:49 smithi049 ceph-mon[34640]: pgmap v138: 97 pgs: 97 active+clean; 67 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-19T06:07:50.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:49 smithi157 ceph-mon[35346]: pgmap v138: 97 pgs: 97 active+clean; 67 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-19T06:07:50.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 102369 2.84 MB/sec execute 76 sec latency 14.996 ms 2024-09-19T06:07:51.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 102911 2.87 MB/sec execute 77 sec latency 15.183 ms 2024-09-19T06:07:52.103 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:51 smithi157 ceph-mon[35346]: pgmap v139: 97 pgs: 97 active+clean; 67 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-19T06:07:52.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:51 smithi005 ceph-mon[30955]: pgmap v139: 97 pgs: 97 active+clean; 67 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-19T06:07:52.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:51 smithi049 ceph-mon[34640]: pgmap v139: 97 pgs: 97 active+clean; 67 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-19T06:07:52.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 103278 2.85 MB/sec execute 78 sec latency 15.302 ms 2024-09-19T06:07:53.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 103600 2.82 MB/sec execute 79 sec latency 14.550 ms 2024-09-19T06:07:54.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:53 smithi005 ceph-mon[30955]: pgmap v140: 97 pgs: 97 active+clean; 68 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-19T06:07:54.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:53 smithi049 ceph-mon[34640]: pgmap v140: 97 pgs: 97 active+clean; 68 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-19T06:07:54.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:53 smithi157 ceph-mon[35346]: pgmap v140: 97 pgs: 97 active+clean; 68 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-19T06:07:54.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 104148 2.81 MB/sec execute 80 sec latency 13.368 ms 2024-09-19T06:07:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.002+0000 7fa621817640 1 -- 172.21.15.5:0/3083513120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61c1035a0 msgr2=0x7fa61c10ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.002+0000 7fa621817640 1 --2- 172.21.15.5:0/3083513120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61c1035a0 0x7fa61c10ba80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa604009900 tx=0x7fa60402a480 comp rx=0 tx=0).stop 2024-09-19T06:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.002+0000 7fa621817640 1 -- 172.21.15.5:0/3083513120 shutdown_connections 2024-09-19T06:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.002+0000 7fa621817640 1 --2- 172.21.15.5:0/3083513120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa61c10c0a0 0x7fa61c10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.002+0000 7fa621817640 1 --2- 172.21.15.5:0/3083513120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61c1035a0 0x7fa61c10ba80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.002+0000 7fa621817640 1 --2- 172.21.15.5:0/3083513120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa61c102c80 0x7fa61c103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.002+0000 7fa621817640 1 -- 172.21.15.5:0/3083513120 >> 172.21.15.5:0/3083513120 conn(0x7fa61c0779a0 msgr2=0x7fa61c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.002+0000 7fa621817640 1 -- 172.21.15.5:0/3083513120 shutdown_connections 2024-09-19T06:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.002+0000 7fa621817640 1 -- 172.21.15.5:0/3083513120 wait complete. 2024-09-19T06:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa621817640 1 Processor -- start 2024-09-19T06:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa621817640 1 -- start start 2024-09-19T06:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa621817640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61c102c80 0x7fa61c1008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa621817640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa61c1035a0 0x7fa61c100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa621817640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa61c10c0a0 0x7fa61c101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa621817640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c111d50 con 0x7fa61c102c80 2024-09-19T06:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa621817640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c111bd0 con 0x7fa61c1035a0 2024-09-19T06:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa621817640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c111ed0 con 0x7fa61c10c0a0 2024-09-19T06:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa61affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61c102c80 0x7fa61c1008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa61affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61c102c80 0x7fa61c1008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35684/0 (socket says 172.21.15.5:35684) 2024-09-19T06:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa61affd640 1 -- 172.21.15.5:0/2568674616 learned_addr learned my addr 172.21.15.5:0/2568674616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa61a7fc640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa61c1035a0 0x7fa61c100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.003+0000 7fa61b7fe640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa61c10c0a0 0x7fa61c101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.004+0000 7fa61affd640 1 -- 172.21.15.5:0/2568674616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa61c10c0a0 msgr2=0x7fa61c101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.004+0000 7fa61affd640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa61c10c0a0 0x7fa61c101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.004+0000 7fa61affd640 1 -- 172.21.15.5:0/2568674616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa61c1035a0 msgr2=0x7fa61c100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.004+0000 7fa61affd640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa61c1035a0 0x7fa61c100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.004+0000 7fa61affd640 1 -- 172.21.15.5:0/2568674616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c1a6880 con 0x7fa61c102c80 2024-09-19T06:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.004+0000 7fa61affd640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61c102c80 0x7fa61c1008d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009d50 tx=0x7fa60c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.004+0000 7fa620815640 1 -- 172.21.15.5:0/2568674616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c003b70 con 0x7fa61c102c80 2024-09-19T06:07:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.004+0000 7fa621817640 1 -- 172.21.15.5:0/2568674616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c1a6b70 con 0x7fa61c102c80 2024-09-19T06:07:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.004+0000 7fa620815640 1 -- 172.21.15.5:0/2568674616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa60c003d10 con 0x7fa61c102c80 2024-09-19T06:07:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.004+0000 7fa620815640 1 -- 172.21.15.5:0/2568674616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0108f0 con 0x7fa61c102c80 2024-09-19T06:07:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.004+0000 7fa621817640 1 -- 172.21.15.5:0/2568674616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c1a70e0 con 0x7fa61c102c80 2024-09-19T06:07:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.007+0000 7fa620815640 1 -- 172.21.15.5:0/2568674616 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa60c004000 con 0x7fa61c102c80 2024-09-19T06:07:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.008+0000 7fa624688740 1 Processor -- start 2024-09-19T06:07:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.008+0000 7fa624688740 1 -- start start 2024-09-19T06:07:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.008+0000 7fa624688740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c3fa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa620815640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa6080a3430 0x7fa6080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa61b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c3fa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa61b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c3fa6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35688/0 (socket says 172.21.15.5:35688) 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa624688740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c3fac20 0x55ed8c404f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa624688740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c4054c0 0x55ed8c4078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa624688740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c3562f0 con 0x55ed8c3fa300 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa624688740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c2fe270 con 0x55ed8c3fac20 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa624688740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c3522b0 con 0x55ed8c4054c0 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa620815640 1 -- 172.21.15.5:0/2568674616 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa60c0d0320 con 0x7fa61c102c80 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa61b7fe640 1 -- 172.21.15.5:0/4139925979 learned_addr learned my addr 172.21.15.5:0/4139925979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa61a7fc640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa6080a3430 0x7fa6080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa61b7fe640 1 -- 172.21.15.5:0/4139925979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c4054c0 msgr2=0x55ed8c4078b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa61b7fe640 1 --2- 172.21.15.5:0/4139925979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c4054c0 0x55ed8c4078b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.009+0000 7fa61b7fe640 1 -- 172.21.15.5:0/4139925979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c3fac20 msgr2=0x55ed8c404f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.010+0000 7fa61b7fe640 1 --2- 172.21.15.5:0/4139925979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c3fac20 0x55ed8c404f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.010+0000 7fa61b7fe640 1 -- 172.21.15.5:0/4139925979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8c2fd340 con 0x55ed8c3fa300 2024-09-19T06:07:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.010+0000 7fa61b7fe640 1 --2- 172.21.15.5:0/4139925979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c3fa6e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa614009c70 tx=0x7fa614011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e2ed3c7501e162 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.010+0000 7fa61a7fc640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa6080a3430 0x7fa6080a58f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa604031440 tx=0x7fa604005b50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.010+0000 7fa611ffb640 1 -- 172.21.15.5:0/4139925979 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61401e070 con 0x55ed8c3fa300 2024-09-19T06:07:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.010+0000 7fa611ffb640 1 -- 172.21.15.5:0/4139925979 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa614009e20 con 0x55ed8c3fa300 2024-09-19T06:07:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.010+0000 7fa611ffb640 1 -- 172.21.15.5:0/4139925979 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61400f620 con 0x55ed8c3fa300 2024-09-19T06:07:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.011+0000 7fa624688740 1 -- 172.21.15.5:0/4139925979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 msgr2=0x55ed8c3fa6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.011+0000 7fa624688740 1 --2- 172.21.15.5:0/4139925979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c3fa6e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa614009c70 tx=0x7fa614011810 comp rx=0 tx=0).stop 2024-09-19T06:07:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.011+0000 7fa624688740 1 -- 172.21.15.5:0/4139925979 shutdown_connections 2024-09-19T06:07:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.011+0000 7fa624688740 1 --2- 172.21.15.5:0/4139925979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c4054c0 0x55ed8c4078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.011+0000 7fa624688740 1 --2- 172.21.15.5:0/4139925979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c3fac20 0x55ed8c404f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.011+0000 7fa624688740 1 --2- 172.21.15.5:0/4139925979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c3fa6e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.011+0000 7fa624688740 1 -- 172.21.15.5:0/4139925979 >> 172.21.15.5:0/4139925979 conn(0x55ed8c3f6870 msgr2=0x55ed8c3f6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.011+0000 7fa624688740 1 -- 172.21.15.5:0/4139925979 shutdown_connections 2024-09-19T06:07:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.011+0000 7fa624688740 1 -- 172.21.15.5:0/4139925979 wait complete. 2024-09-19T06:07:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.012+0000 7fa624688740 1 -- start start 2024-09-19T06:07:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.012+0000 7fa624688740 1 Processor -- start 2024-09-19T06:07:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.012+0000 7fa624688740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c402450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.012+0000 7fa624688740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fac20 0x55ed8c402990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.012+0000 7fa624688740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c4054c0 0x55ed8c402ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.012+0000 7fa624688740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c408090 con 0x55ed8c3fa300 2024-09-19T06:07:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.012+0000 7fa624688740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c2df290 con 0x55ed8c4054c0 2024-09-19T06:07:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.012+0000 7fa624688740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c408210 con 0x55ed8c3fac20 2024-09-19T06:07:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa624688740 10 client.0 _ll_register_callbacks cb 0x55ed8c310140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c402450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fac20 0x55ed8c402990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c4054c0 0x55ed8c402ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fac20 0x55ed8c402990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59874/0 (socket says 172.21.15.5:59874) 2024-09-19T06:07:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61a7fc640 1 -- 172.21.15.5:0/292483429 learned_addr learned my addr 172.21.15.5:0/292483429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c402450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35694/0 (socket says 172.21.15.5:35694) 2024-09-19T06:07:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61b7fe640 1 -- 172.21.15.5:0/292483429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fac20 msgr2=0x55ed8c402990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61b7fe640 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fac20 0x55ed8c402990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61b7fe640 1 -- 172.21.15.5:0/292483429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 msgr2=0x55ed8c402450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61b7fe640 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c402450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61b7fe640 1 -- 172.21.15.5:0/292483429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8c069210 con 0x55ed8c4054c0 2024-09-19T06:07:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61a7fc640 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fac20 0x55ed8c402990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61affd640 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c402450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa61b7fe640 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c4054c0 0x55ed8c402ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa61400a010 tx=0x7fa6140098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a38835cb78e96442 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.013+0000 7fa5faffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:07:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa5faffd640 1 -- 172.21.15.5:0/292483429 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61401e040 con 0x55ed8c4054c0 2024-09-19T06:07:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa624688740 1 -- 172.21.15.5:0/292483429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa614009e20 con 0x55ed8c4054c0 2024-09-19T06:07:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa5faffd640 1 -- 172.21.15.5:0/292483429 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa614016050 con 0x55ed8c4054c0 2024-09-19T06:07:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa5faffd640 1 -- 172.21.15.5:0/292483429 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa61402f050 con 0x55ed8c4054c0 2024-09-19T06:07:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa624688740 10 client.26105 fetch_fsmap learned FSMap version 33 2024-09-19T06:07:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa624688740 1 -- 172.21.15.5:0/292483429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ed8c135620 con 0x55ed8c4054c0 2024-09-19T06:07:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa5faffd640 1 -- 172.21.15.5:0/292483429 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa614009e20 con 0x55ed8c4054c0 2024-09-19T06:07:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa5faffd640 1 -- 172.21.15.5:0/292483429 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa614021070 con 0x55ed8c4054c0 2024-09-19T06:07:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa624688740 10 client.26105 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:07:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa624688740 10 client.26105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa624688740 20 client.26105 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa624688740 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed8c415420 0x55ed8c417810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa624688740 4 client.26105 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:07:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.014+0000 7fa624688740 1 -- 172.21.15.5:0/292483429 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed8c363a20 con 0x55ed8c415420 2024-09-19T06:07:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.015+0000 7fa61affd640 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed8c415420 0x55ed8c417810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.015+0000 7fa5faffd640 10 client.26105 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.015+0000 7fa61affd640 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed8c415420 0x55ed8c417810 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37291a1b7a3e5a54 server_cookie=50e8135ea5004e67 in_seq=0 out_seq=0 2024-09-19T06:07:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.015+0000 7fa5faffd640 1 -- 172.21.15.5:0/292483429 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed8c363a20 con 0x55ed8c415420 2024-09-19T06:07:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.015+0000 7fa5faffd640 10 client.26105 handle_command_reply: tid=0 2024-09-19T06:07:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa624688740 1 client.26105 shutdown 2024-09-19T06:07:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa624688740 1 -- 172.21.15.5:0/292483429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c4054c0 msgr2=0x55ed8c402ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa624688740 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c4054c0 0x55ed8c402ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa61400a010 tx=0x7fa6140098d0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa624688740 1 -- 172.21.15.5:0/292483429 shutdown_connections 2024-09-19T06:07:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa624688740 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed8c415420 0x55ed8c417810 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa624688740 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c4054c0 0x55ed8c402ed0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa624688740 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fac20 0x55ed8c402990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa5faffd640 0 client.26105 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa624688740 1 --2- 172.21.15.5:0/292483429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c402450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa624688740 1 -- 172.21.15.5:0/292483429 >> 172.21.15.5:0/292483429 conn(0x55ed8c3f6870 msgr2=0x55ed8c3fc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa624688740 1 -- 172.21.15.5:0/292483429 shutdown_connections 2024-09-19T06:07:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa624688740 1 -- 172.21.15.5:0/292483429 wait complete. 2024-09-19T06:07:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.016+0000 7fa624688740 20 client.26105 trim_cache size 0 max 16384 2024-09-19T06:07:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.033+0000 7fa624688740 1 Processor -- start 2024-09-19T06:07:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa624688740 1 -- start start 2024-09-19T06:07:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa624688740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fa300 0x55ed8c400d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa624688740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c3fac20 0x55ed8c402d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa624688740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c4054c0 0x55ed8c41cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa624688740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1ab690 con 0x55ed8c4054c0 2024-09-19T06:07:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa624688740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1ad1d0 con 0x55ed8c3fac20 2024-09-19T06:07:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa624688740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1abd60 con 0x55ed8c3fa300 2024-09-19T06:07:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c4054c0 0x55ed8c41cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c3fac20 0x55ed8c402d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c4054c0 0x55ed8c41cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35710/0 (socket says 172.21.15.5:35710) 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61b7fe640 1 -- 172.21.15.5:0/680786401 learned_addr learned my addr 172.21.15.5:0/680786401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c3fac20 0x55ed8c402d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50586/0 (socket says 172.21.15.5:50586) 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61affd640 1 --2- 172.21.15.5:0/680786401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fa300 0x55ed8c400d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61a7fc640 1 -- 172.21.15.5:0/680786401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fa300 msgr2=0x55ed8c400d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61a7fc640 1 --2- 172.21.15.5:0/680786401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fa300 0x55ed8c400d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61a7fc640 1 -- 172.21.15.5:0/680786401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c4054c0 msgr2=0x55ed8c41cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61a7fc640 1 --2- 172.21.15.5:0/680786401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c4054c0 0x55ed8c41cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61a7fc640 1 -- 172.21.15.5:0/680786401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8c3f7920 con 0x55ed8c3fac20 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61b7fe640 1 --2- 172.21.15.5:0/680786401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c4054c0 0x55ed8c41cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.034+0000 7fa61affd640 1 --2- 172.21.15.5:0/680786401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fa300 0x55ed8c400d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:07:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.035+0000 7fa61a7fc640 1 --2- 172.21.15.5:0/680786401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c3fac20 0x55ed8c402d00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa6040057c0 tx=0x7fa60403d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=603c2e169cfc80f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.035+0000 7fa6117fa640 1 -- 172.21.15.5:0/680786401 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60404d070 con 0x55ed8c3fac20 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.035+0000 7fa6117fa640 1 -- 172.21.15.5:0/680786401 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6040035f0 con 0x55ed8c3fac20 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.035+0000 7fa6117fa640 1 -- 172.21.15.5:0/680786401 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604035040 con 0x55ed8c3fac20 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.035+0000 7fa624688740 1 -- 172.21.15.5:0/680786401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c3fac20 msgr2=0x55ed8c402d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.035+0000 7fa624688740 1 --2- 172.21.15.5:0/680786401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c3fac20 0x55ed8c402d00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa6040057c0 tx=0x7fa60403d000 comp rx=0 tx=0).stop 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.035+0000 7fa624688740 1 -- 172.21.15.5:0/680786401 shutdown_connections 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.035+0000 7fa624688740 1 --2- 172.21.15.5:0/680786401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c4054c0 0x55ed8c41cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.035+0000 7fa624688740 1 --2- 172.21.15.5:0/680786401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c3fac20 0x55ed8c402d00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.035+0000 7fa624688740 1 --2- 172.21.15.5:0/680786401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fa300 0x55ed8c400d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.035+0000 7fa624688740 1 -- 172.21.15.5:0/680786401 >> 172.21.15.5:0/680786401 conn(0x55ed8c3f6870 msgr2=0x55ed8c4044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa624688740 1 -- 172.21.15.5:0/680786401 shutdown_connections 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa624688740 1 -- 172.21.15.5:0/680786401 wait complete. 2024-09-19T06:07:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa624688740 1 -- start start 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa624688740 1 Processor -- start 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa624688740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c40d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa624688740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fac20 0x55ed8c40da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa624688740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c4054c0 0x55ed8c3fea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa624688740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1acb00 con 0x55ed8c3fa300 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa624688740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1ac430 con 0x55ed8c4054c0 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa624688740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1a9670 con 0x55ed8c3fac20 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa624688740 10 client.0 _ll_register_callbacks cb 0x7fa61401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa61affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c40d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.036+0000 7fa61affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c40d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35726/0 (socket says 172.21.15.5:35726) 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa61affd640 1 -- 172.21.15.5:0/3701967976 learned_addr learned my addr 172.21.15.5:0/3701967976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa61b7fe640 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c4054c0 0x55ed8c3fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa61a7fc640 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fac20 0x55ed8c40da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa61affd640 1 -- 172.21.15.5:0/3701967976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fac20 msgr2=0x55ed8c40da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa61affd640 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fac20 0x55ed8c40da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa61affd640 1 -- 172.21.15.5:0/3701967976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c4054c0 msgr2=0x55ed8c3fea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa61affd640 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c4054c0 0x55ed8c3fea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa61affd640 1 -- 172.21.15.5:0/3701967976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8c3ff050 con 0x55ed8c3fa300 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa61affd640 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c40d4d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009d20 tx=0x7fa60c009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f179d2c9242de26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa5faffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa5faffd640 1 -- 172.21.15.5:0/3701967976 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0b4070 con 0x55ed8c3fa300 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa5faffd640 1 -- 172.21.15.5:0/3701967976 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa60c09a910 con 0x55ed8c3fa300 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa624688740 1 -- 172.21.15.5:0/3701967976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6040035f0 con 0x55ed8c3fa300 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa5faffd640 1 -- 172.21.15.5:0/3701967976 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa60c0a09c0 con 0x55ed8c3fa300 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa624688740 10 client.16536 fetch_fsmap learned FSMap version 33 2024-09-19T06:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.037+0000 7fa624688740 1 -- 172.21.15.5:0/3701967976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ed8c47f7b0 con 0x55ed8c3fa300 2024-09-19T06:07:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.038+0000 7fa5faffd640 1 -- 172.21.15.5:0/3701967976 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6040035f0 con 0x55ed8c3fa300 2024-09-19T06:07:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.038+0000 7fa5faffd640 1 -- 172.21.15.5:0/3701967976 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa60c0cdd00 con 0x55ed8c3fa300 2024-09-19T06:07:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.038+0000 7fa624688740 10 client.16536 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:07:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.038+0000 7fa624688740 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:07:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.038+0000 7fa624688740 20 client.16536 populate_metadata read hostname 'smithi005' 2024-09-19T06:07:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.038+0000 7fa624688740 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed8c47feb0 0x55ed8c4822d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:07:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.038+0000 7fa624688740 4 client.16536 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:07:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.038+0000 7fa624688740 1 -- 172.21.15.5:0/3701967976 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed8c35dd70 con 0x55ed8c47feb0 2024-09-19T06:07:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.038+0000 7fa61a7fc640 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed8c47feb0 0x55ed8c4822d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:07:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.038+0000 7fa5faffd640 10 client.16536 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.038+0000 7fa61a7fc640 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed8c47feb0 0x55ed8c4822d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ed9ee9db030a54 server_cookie=8daea619537ef6b in_seq=0 out_seq=0 2024-09-19T06:07:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa5faffd640 1 -- 172.21.15.5:0/3701967976 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed8c35dd70 con 0x55ed8c47feb0 2024-09-19T06:07:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa5faffd640 10 client.16536 handle_command_reply: tid=0 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa624688740 1 client.16536 shutdown 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa624688740 1 -- 172.21.15.5:0/3701967976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 msgr2=0x55ed8c40d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa624688740 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c40d4d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009d20 tx=0x7fa60c009820 comp rx=0 tx=0).stop 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa624688740 1 -- 172.21.15.5:0/3701967976 shutdown_connections 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa624688740 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ed8c47feb0 0x55ed8c4822d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa624688740 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed8c4054c0 0x55ed8c3fea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa624688740 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed8c3fac20 0x55ed8c40da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa624688740 1 --2- 172.21.15.5:0/3701967976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed8c3fa300 0x55ed8c40d4d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa5faffd640 0 client.16536 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa624688740 1 -- 172.21.15.5:0/3701967976 >> 172.21.15.5:0/3701967976 conn(0x55ed8c3f6870 msgr2=0x55ed8c3fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa624688740 1 -- 172.21.15.5:0/3701967976 shutdown_connections 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.058+0000 7fa624688740 1 -- 172.21.15.5:0/3701967976 wait complete. 2024-09-19T06:07:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa624688740 20 client.16536 trim_cache size 0 max 16384 2024-09-19T06:07:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:07:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:07:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:07:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa621817640 1 -- 172.21.15.5:0/2568674616 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa6080a3430 msgr2=0x7fa6080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa621817640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa6080a3430 0x7fa6080a58f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa604031440 tx=0x7fa604005b50 comp rx=0 tx=0).stop 2024-09-19T06:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa621817640 1 -- 172.21.15.5:0/2568674616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61c102c80 msgr2=0x7fa61c1008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa621817640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61c102c80 0x7fa61c1008d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009d50 tx=0x7fa60c00c6a0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa621817640 1 -- 172.21.15.5:0/2568674616 shutdown_connections 2024-09-19T06:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa621817640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa6080a3430 0x7fa6080a58f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa621817640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa61c10c0a0 0x7fa61c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa621817640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa61c1035a0 0x7fa61c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa621817640 1 --2- 172.21.15.5:0/2568674616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61c102c80 0x7fa61c1008d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:07:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa621817640 1 -- 172.21.15.5:0/2568674616 >> 172.21.15.5:0/2568674616 conn(0x7fa61c0779a0 msgr2=0x7fa61c0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:07:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa621817640 1 -- 172.21.15.5:0/2568674616 shutdown_connections 2024-09-19T06:07:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:07:55.059+0000 7fa621817640 1 -- 172.21.15.5:0/2568674616 wait complete. 2024-09-19T06:07:55.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 104787 2.81 MB/sec execute 81 sec latency 22.447 ms 2024-09-19T06:07:56.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:55 smithi005 ceph-mon[30955]: pgmap v141: 97 pgs: 97 active+clean; 68 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-19T06:07:56.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:55 smithi049 ceph-mon[34640]: pgmap v141: 97 pgs: 97 active+clean; 68 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-19T06:07:56.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:55 smithi157 ceph-mon[35346]: pgmap v141: 97 pgs: 97 active+clean; 68 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-19T06:07:56.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 105382 2.81 MB/sec execute 82 sec latency 15.252 ms 2024-09-19T06:07:57.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106117 2.84 MB/sec execute 83 sec latency 90.416 ms 2024-09-19T06:07:58.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:57 smithi005 ceph-mon[30955]: pgmap v142: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-19T06:07:58.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:57 smithi049 ceph-mon[34640]: pgmap v142: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-19T06:07:58.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:57 smithi157 ceph-mon[35346]: pgmap v142: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-19T06:07:58.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106570 2.86 MB/sec execute 84 sec latency 15.099 ms 2024-09-19T06:07:59.711 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:07:59.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106870 2.84 MB/sec execute 85 sec latency 14.732 ms 2024-09-19T06:07:59.992 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:00.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:07:59 smithi005 ceph-mon[30955]: pgmap v143: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-19T06:08:00.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:07:59 smithi049 ceph-mon[34640]: pgmap v143: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-19T06:08:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:07:59 smithi157 ceph-mon[35346]: pgmap v143: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-19T06:08:00.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 107206 2.81 MB/sec execute 86 sec latency 30.275 ms 2024-09-19T06:08:01.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 107688 2.80 MB/sec execute 87 sec latency 71.747 ms 2024-09-19T06:08:02.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:01 smithi005 ceph-mon[30955]: pgmap v144: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-19T06:08:02.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:01 smithi049 ceph-mon[34640]: pgmap v144: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-19T06:08:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.228+0000 7f392bcde640 1 -- 172.21.15.5:0/3255976289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3924103650 msgr2=0x7f392410bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.228+0000 7f392bcde640 1 --2- 172.21.15.5:0/3255976289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3924103650 0x7f392410bb40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3914009900 tx=0x7f391402a480 comp rx=0 tx=0).stop 2024-09-19T06:08:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.228+0000 7f392bcde640 1 -- 172.21.15.5:0/3255976289 shutdown_connections 2024-09-19T06:08:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.228+0000 7f392bcde640 1 --2- 172.21.15.5:0/3255976289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392410c160 0x7f392410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.228+0000 7f392bcde640 1 --2- 172.21.15.5:0/3255976289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3924103650 0x7f392410bb40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.228+0000 7f392bcde640 1 --2- 172.21.15.5:0/3255976289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3924102d30 0x7f3924103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.228+0000 7f392bcde640 1 -- 172.21.15.5:0/3255976289 >> 172.21.15.5:0/3255976289 conn(0x7f39240fc7d0 msgr2=0x7f39240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.229+0000 7f392bcde640 1 -- 172.21.15.5:0/3255976289 shutdown_connections 2024-09-19T06:08:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.229+0000 7f392bcde640 1 -- 172.21.15.5:0/3255976289 wait complete. 2024-09-19T06:08:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.229+0000 7f392bcde640 1 Processor -- start 2024-09-19T06:08:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.229+0000 7f392bcde640 1 -- start start 2024-09-19T06:08:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f392bcde640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3924102d30 0x7f3924198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f392bcde640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3924103650 0x7f39241988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f392bcde640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392410c160 0x7f392419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f392bcde640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3924111e10 con 0x7f392410c160 2024-09-19T06:08:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f392bcde640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3924111c90 con 0x7f3924102d30 2024-09-19T06:08:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f392bcde640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3924111f90 con 0x7f3924103650 2024-09-19T06:08:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f3929a7a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3924102d30 0x7f3924198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f392a27b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392410c160 0x7f392419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f3929a7a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3924102d30 0x7f3924198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50622/0 (socket says 172.21.15.5:50622) 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f3929a7a640 1 -- 172.21.15.5:0/3527065062 learned_addr learned my addr 172.21.15.5:0/3527065062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f392a27b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392410c160 0x7f392419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35748/0 (socket says 172.21.15.5:35748) 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f3929279640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3924103650 0x7f39241988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f3929a7a640 1 -- 172.21.15.5:0/3527065062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3924103650 msgr2=0x7f39241988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f3929a7a640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3924103650 0x7f39241988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f3929a7a640 1 -- 172.21.15.5:0/3527065062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392410c160 msgr2=0x7f392419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f3929a7a640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392410c160 0x7f392419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.230+0000 7f3929a7a640 1 -- 172.21.15.5:0/3527065062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392419d340 con 0x7f3924102d30 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.231+0000 7f3929a7a640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3924102d30 0x7f3924198390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f392000e9f0 tx=0x7f392000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.231+0000 7f391affd640 1 -- 172.21.15.5:0/3527065062 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392000cd80 con 0x7f3924102d30 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.231+0000 7f392bcde640 1 -- 172.21.15.5:0/3527065062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392419d630 con 0x7f3924102d30 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.231+0000 7f391affd640 1 -- 172.21.15.5:0/3527065062 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39200054d0 con 0x7f3924102d30 2024-09-19T06:08:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.231+0000 7f391affd640 1 -- 172.21.15.5:0/3527065062 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920010490 con 0x7f3924102d30 2024-09-19T06:08:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.231+0000 7f392bcde640 1 -- 172.21.15.5:0/3527065062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39241008c0 con 0x7f3924102d30 2024-09-19T06:08:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.234+0000 7f392e0b7740 1 Processor -- start 2024-09-19T06:08:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.234+0000 7f392e0b7740 1 -- start start 2024-09-19T06:08:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.234+0000 7f392e0b7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be6f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.234+0000 7f391affd640 1 -- 172.21.15.5:0/3527065062 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3920005020 con 0x7f3924102d30 2024-09-19T06:08:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.234+0000 7f392e0b7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6be7a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.234+0000 7f392e0b7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be7a680 0x562e6be7ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f392e0b7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562e6bdd52e0 con 0x562e6be6fdc0 2024-09-19T06:08:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f392e0b7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562e6bd7cce0 con 0x562e6be7a680 2024-09-19T06:08:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f392e0b7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562e6bdd12a0 con 0x562e6be6f4a0 2024-09-19T06:08:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f3929a7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6be7a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f3929a7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6be7a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35758/0 (socket says 172.21.15.5:35758) 2024-09-19T06:08:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f3929a7a640 1 -- 172.21.15.5:0/1998828720 learned_addr learned my addr 172.21.15.5:0/1998828720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f392a27b640 1 --2- 172.21.15.5:0/1998828720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be6f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f3929279640 1 --2- 172.21.15.5:0/1998828720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be7a680 0x562e6be7ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f392a27b640 1 -- 172.21.15.5:0/1998828720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be7a680 msgr2=0x562e6be7ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f392a27b640 1 --2- 172.21.15.5:0/1998828720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be7a680 0x562e6be7ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f392a27b640 1 -- 172.21.15.5:0/1998828720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 msgr2=0x562e6be7a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f392a27b640 1 --2- 172.21.15.5:0/1998828720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6be7a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f392a27b640 1 -- 172.21.15.5:0/1998828720 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6bd61440 con 0x562e6be6f4a0 2024-09-19T06:08:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f3929a7a640 1 --2- 172.21.15.5:0/1998828720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6be7a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.235+0000 7f3929279640 1 --2- 172.21.15.5:0/1998828720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be7a680 0x562e6be7ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.236+0000 7f392a27b640 1 --2- 172.21.15.5:0/1998828720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be6f880 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f391c009f00 tx=0x7f391c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa24edb1b4529e2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.236+0000 7f390ffff640 1 -- 172.21.15.5:0/1998828720 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c004d20 con 0x562e6be6f4a0 2024-09-19T06:08:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.236+0000 7f391affd640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f39100a33b0 0x7f39100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.236+0000 7f390ffff640 1 -- 172.21.15.5:0/1998828720 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f391c004050 con 0x562e6be6f4a0 2024-09-19T06:08:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.236+0000 7f390ffff640 1 -- 172.21.15.5:0/1998828720 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c005280 con 0x562e6be6f4a0 2024-09-19T06:08:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.236+0000 7f391affd640 1 -- 172.21.15.5:0/3527065062 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f39200d00c0 con 0x7f3924102d30 2024-09-19T06:08:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.236+0000 7f3929a7a640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f39100a33b0 0x7f39100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.237+0000 7f392e0b7740 1 -- 172.21.15.5:0/1998828720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 msgr2=0x562e6be6f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.237+0000 7f3929a7a640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f39100a33b0 0x7f39100a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f392009c340 tx=0x7f39200d3c10 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.237+0000 7f392e0b7740 1 --2- 172.21.15.5:0/1998828720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be6f880 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f391c009f00 tx=0x7f391c011810 comp rx=0 tx=0).stop 2024-09-19T06:08:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.237+0000 7f392e0b7740 1 -- 172.21.15.5:0/1998828720 shutdown_connections 2024-09-19T06:08:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.237+0000 7f392e0b7740 1 --2- 172.21.15.5:0/1998828720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be7a680 0x562e6be7ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.237+0000 7f392e0b7740 1 --2- 172.21.15.5:0/1998828720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6be7a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.237+0000 7f392e0b7740 1 --2- 172.21.15.5:0/1998828720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be6f880 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.237+0000 7f392e0b7740 1 -- 172.21.15.5:0/1998828720 >> 172.21.15.5:0/1998828720 conn(0x562e6be6ba30 msgr2=0x562e6be6be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.237+0000 7f392e0b7740 1 -- 172.21.15.5:0/1998828720 shutdown_connections 2024-09-19T06:08:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.237+0000 7f392e0b7740 1 -- 172.21.15.5:0/1998828720 wait complete. 2024-09-19T06:08:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.237+0000 7f392e0b7740 1 -- start start 2024-09-19T06:08:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.237+0000 7f392e0b7740 1 Processor -- start 2024-09-19T06:08:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.238+0000 7f392e0b7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be79f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.238+0000 7f392e0b7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6fdc0 0x562e6be77200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.238+0000 7f392e0b7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be7a680 0x562e6be77740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.238+0000 7f392e0b7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562e6be7d2a0 con 0x562e6be7a680 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.238+0000 7f392e0b7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562e6bd90050 con 0x562e6be6fdc0 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.238+0000 7f392e0b7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562e6be7d420 con 0x562e6be6f4a0 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.238+0000 7f392e0b7740 10 client.0 _ll_register_callbacks cb 0x562e6bdd0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.238+0000 7f392a27b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be7a680 0x562e6be77740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.238+0000 7f392a27b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be7a680 0x562e6be77740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35768/0 (socket says 172.21.15.5:35768) 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.238+0000 7f392a27b640 1 -- 172.21.15.5:0/3600529745 learned_addr learned my addr 172.21.15.5:0/3600529745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.238+0000 7f3929279640 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6fdc0 0x562e6be77200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.238+0000 7f392a27b640 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be79f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f392a27b640 1 -- 172.21.15.5:0/3600529745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6fdc0 msgr2=0x562e6be77200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f392a27b640 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6fdc0 0x562e6be77200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f392a27b640 1 -- 172.21.15.5:0/3600529745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be7a680 msgr2=0x562e6be77740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f392a27b640 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be7a680 0x562e6be77740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f392a27b640 1 -- 172.21.15.5:0/3600529745 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6bb463c0 con 0x562e6be6f4a0 2024-09-19T06:08:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f392a27b640 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be79f50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f391c005fd0 tx=0x7f391c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f36975af53b89f0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f390d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:08:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f390d7fa640 1 -- 172.21.15.5:0/3600529745 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c02c020 con 0x562e6be6f4a0 2024-09-19T06:08:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f392e0b7740 1 -- 172.21.15.5:0/3600529745 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f391c004050 con 0x562e6be6f4a0 2024-09-19T06:08:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f390d7fa640 1 -- 172.21.15.5:0/3600529745 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f391c004a40 con 0x562e6be6f4a0 2024-09-19T06:08:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f390d7fa640 1 -- 172.21.15.5:0/3600529745 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f391c037050 con 0x562e6be6f4a0 2024-09-19T06:08:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f392e0b7740 10 client.26056 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.239+0000 7f392e0b7740 1 -- 172.21.15.5:0/3600529745 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562e6be88bf0 con 0x562e6be6f4a0 2024-09-19T06:08:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.240+0000 7f390d7fa640 1 -- 172.21.15.5:0/3600529745 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f391c004050 con 0x562e6be6f4a0 2024-09-19T06:08:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.240+0000 7f390d7fa640 1 -- 172.21.15.5:0/3600529745 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f391c0286b0 con 0x562e6be6f4a0 2024-09-19T06:08:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.240+0000 7f392e0b7740 10 client.26056 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.240+0000 7f392e0b7740 10 client.26056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.240+0000 7f392e0b7740 20 client.26056 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.240+0000 7f392e0b7740 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562e6be8a3e0 0x562e6be8c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.240+0000 7f392e0b7740 4 client.26056 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.240+0000 7f392e0b7740 1 -- 172.21.15.5:0/3600529745 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e6bde2850 con 0x562e6be8a3e0 2024-09-19T06:08:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.240+0000 7f3929279640 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562e6be8a3e0 0x562e6be8c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.241+0000 7f3929279640 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562e6be8a3e0 0x562e6be8c7f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=533622dc417e4531 server_cookie=a630d0c284f29762 in_seq=0 out_seq=0 2024-09-19T06:08:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.241+0000 7f390d7fa640 10 client.26056 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f390d7fa640 1 -- 172.21.15.5:0/3600529745 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e6bde2850 con 0x562e6be8a3e0 2024-09-19T06:08:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f390d7fa640 10 client.26056 handle_command_reply: tid=0 2024-09-19T06:08:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f392e0b7740 1 client.26056 shutdown 2024-09-19T06:08:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f392e0b7740 1 -- 172.21.15.5:0/3600529745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 msgr2=0x562e6be79f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f392e0b7740 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be79f50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f391c005fd0 tx=0x7f391c000f40 comp rx=0 tx=0).stop 2024-09-19T06:08:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f392e0b7740 1 -- 172.21.15.5:0/3600529745 shutdown_connections 2024-09-19T06:08:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f392e0b7740 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562e6be8a3e0 0x562e6be8c7f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f392e0b7740 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be7a680 0x562e6be77740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f392e0b7740 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6fdc0 0x562e6be77200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f390d7fa640 0 client.26056 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f392e0b7740 1 --2- 172.21.15.5:0/3600529745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be79f50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f392e0b7740 1 -- 172.21.15.5:0/3600529745 >> 172.21.15.5:0/3600529745 conn(0x562e6be6ba30 msgr2=0x562e6be71760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f392e0b7740 1 -- 172.21.15.5:0/3600529745 shutdown_connections 2024-09-19T06:08:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f392e0b7740 1 -- 172.21.15.5:0/3600529745 wait complete. 2024-09-19T06:08:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.242+0000 7f392e0b7740 20 client.26056 trim_cache size 0 max 16384 2024-09-19T06:08:02.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.264+0000 7f392e0b7740 1 Processor -- start 2024-09-19T06:08:02.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.264+0000 7f392e0b7740 1 -- start start 2024-09-19T06:08:02.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.264+0000 7f392e0b7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6ba30 0x562e6be86500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392e0b7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392e0b7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6be920c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392e0b7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39241ab5e0 con 0x562e6be6fdc0 2024-09-19T06:08:02.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392e0b7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39241ad120 con 0x562e6be6ba30 2024-09-19T06:08:02.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392e0b7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39241abcb0 con 0x562e6be6f4a0 2024-09-19T06:08:02.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392a27b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6ba30 0x562e6be86500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392a27b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6be920c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392a27b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6ba30 0x562e6be86500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50646/0 (socket says 172.21.15.5:50646) 2024-09-19T06:08:02.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392a27b640 1 -- 172.21.15.5:0/3719156669 learned_addr learned my addr 172.21.15.5:0/3719156669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:02.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f3929279640 1 --2- 172.21.15.5:0/3719156669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392a27b640 1 -- 172.21.15.5:0/3719156669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 msgr2=0x562e6be727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:02.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392a27b640 1 --2- 172.21.15.5:0/3719156669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392a27b640 1 -- 172.21.15.5:0/3719156669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 msgr2=0x562e6be920c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:02.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392a27b640 1 --2- 172.21.15.5:0/3719156669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6be920c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392a27b640 1 -- 172.21.15.5:0/3719156669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6be6b860 con 0x562e6be6ba30 2024-09-19T06:08:02.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f3929279640 1 --2- 172.21.15.5:0/3719156669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:02.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.265+0000 7f392a27b640 1 --2- 172.21.15.5:0/3719156669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6ba30 0x562e6be86500 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f391c009f00 tx=0x7f391c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468b2d659c77a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:02.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.266+0000 7f390f7fe640 1 -- 172.21.15.5:0/3719156669 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c02c040 con 0x562e6be6ba30 2024-09-19T06:08:02.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.266+0000 7f390f7fe640 1 -- 172.21.15.5:0/3719156669 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f391c011d10 con 0x562e6be6ba30 2024-09-19T06:08:02.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.266+0000 7f390f7fe640 1 -- 172.21.15.5:0/3719156669 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c0199d0 con 0x562e6be6ba30 2024-09-19T06:08:02.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.266+0000 7f392e0b7740 1 -- 172.21.15.5:0/3719156669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6ba30 msgr2=0x562e6be86500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:02.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.266+0000 7f392e0b7740 1 --2- 172.21.15.5:0/3719156669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6ba30 0x562e6be86500 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f391c009f00 tx=0x7f391c000f40 comp rx=0 tx=0).stop 2024-09-19T06:08:02.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.266+0000 7f392e0b7740 1 -- 172.21.15.5:0/3719156669 shutdown_connections 2024-09-19T06:08:02.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.266+0000 7f392e0b7740 1 --2- 172.21.15.5:0/3719156669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6be920c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.266+0000 7f392e0b7740 1 --2- 172.21.15.5:0/3719156669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6f4a0 0x562e6be727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.266+0000 7f392e0b7740 1 --2- 172.21.15.5:0/3719156669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6ba30 0x562e6be86500 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.267+0000 7f392e0b7740 1 -- 172.21.15.5:0/3719156669 >> 172.21.15.5:0/3719156669 conn(0x562e6be778a0 msgr2=0x562e6be720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:02.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.267+0000 7f392e0b7740 1 -- 172.21.15.5:0/3719156669 shutdown_connections 2024-09-19T06:08:02.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.267+0000 7f392e0b7740 1 -- 172.21.15.5:0/3719156669 wait complete. 2024-09-19T06:08:02.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.267+0000 7f392e0b7740 1 -- start start 2024-09-19T06:08:02.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.267+0000 7f392e0b7740 1 Processor -- start 2024-09-19T06:08:02.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f392e0b7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6ba30 0x562e6be7dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f392e0b7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6f4a0 0x562e6be7e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f392e0b7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6bef3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f392e0b7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39241aca50 con 0x562e6be6fdc0 2024-09-19T06:08:02.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f392e0b7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39241ac380 con 0x562e6be6f4a0 2024-09-19T06:08:02.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f392e0b7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39241a95c0 con 0x562e6be6ba30 2024-09-19T06:08:02.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f392e0b7740 10 client.0 _ll_register_callbacks cb 0x7f391c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:02.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f392a27b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6bef3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f3929279640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6f4a0 0x562e6be7e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f392a27b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6bef3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35782/0 (socket says 172.21.15.5:35782) 2024-09-19T06:08:02.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f392a27b640 1 -- 172.21.15.5:0/1947199831 learned_addr learned my addr 172.21.15.5:0/1947199831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:02.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f392a27b640 1 --2- 172.21.15.5:0/1947199831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6ba30 0x562e6be7dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f3929279640 1 -- 172.21.15.5:0/1947199831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6ba30 msgr2=0x562e6be7dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f3929279640 1 --2- 172.21.15.5:0/1947199831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6ba30 0x562e6be7dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f3929279640 1 -- 172.21.15.5:0/1947199831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 msgr2=0x562e6bef3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f3929279640 1 --2- 172.21.15.5:0/1947199831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6bef3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f3929279640 1 -- 172.21.15.5:0/1947199831 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6be6cae0 con 0x562e6be6f4a0 2024-09-19T06:08:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f3929279640 1 --2- 172.21.15.5:0/1947199831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6f4a0 0x562e6be7e110 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3914006010 tx=0x7f3914032e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d56c0331f6deeddc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.268+0000 7f390d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:08:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.269+0000 7f390d7fa640 1 -- 172.21.15.5:0/1947199831 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914035040 con 0x562e6be6f4a0 2024-09-19T06:08:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.269+0000 7f390d7fa640 1 -- 172.21.15.5:0/1947199831 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f3914004450 con 0x562e6be6f4a0 2024-09-19T06:08:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.269+0000 7f392e0b7740 1 -- 172.21.15.5:0/1947199831 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f391c011d10 con 0x562e6be6f4a0 2024-09-19T06:08:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.269+0000 7f390d7fa640 1 -- 172.21.15.5:0/1947199831 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3914005040 con 0x562e6be6f4a0 2024-09-19T06:08:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.269+0000 7f392e0b7740 10 client.26129 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.269+0000 7f392e0b7740 1 -- 172.21.15.5:0/1947199831 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562e6bef4840 con 0x562e6be6f4a0 2024-09-19T06:08:02.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.269+0000 7f390d7fa640 1 -- 172.21.15.5:0/1947199831 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f391c011d10 con 0x562e6be6f4a0 2024-09-19T06:08:02.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.269+0000 7f390d7fa640 1 -- 172.21.15.5:0/1947199831 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f391404e020 con 0x562e6be6f4a0 2024-09-19T06:08:02.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.270+0000 7f392e0b7740 10 client.26129 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:02.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.270+0000 7f392e0b7740 10 client.26129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:02.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.270+0000 7f392e0b7740 20 client.26129 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:02.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.270+0000 7f392e0b7740 1 --2- 172.21.15.5:0/1947199831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562e6be7a680 0x562e6bef6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:02.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.270+0000 7f392e0b7740 4 client.26129 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:08:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.270+0000 7f392e0b7740 1 -- 172.21.15.5:0/1947199831 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e6bddcd60 con 0x562e6be7a680 2024-09-19T06:08:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.270+0000 7f392a27b640 1 --2- 172.21.15.5:0/1947199831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562e6be7a680 0x562e6bef6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.270+0000 7f392a27b640 1 --2- 172.21.15.5:0/1947199831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562e6be7a680 0x562e6bef6fe0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d6682a334aa172 server_cookie=171eed0d823ec7c5 in_seq=0 out_seq=0 2024-09-19T06:08:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.270+0000 7f390d7fa640 10 client.26129 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f390d7fa640 1 -- 172.21.15.5:0/1947199831 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562e6bddcd60 con 0x562e6be7a680 2024-09-19T06:08:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f390d7fa640 10 client.26129 handle_command_reply: tid=0 2024-09-19T06:08:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f392e0b7740 1 client.26129 shutdown 2024-09-19T06:08:02.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f392e0b7740 1 -- 172.21.15.5:0/1947199831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6f4a0 msgr2=0x562e6be7e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:02.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f392e0b7740 1 --2- 172.21.15.5:0/1947199831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6f4a0 0x562e6be7e110 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3914006010 tx=0x7f3914032e10 comp rx=0 tx=0).stop 2024-09-19T06:08:02.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f392e0b7740 1 -- 172.21.15.5:0/1947199831 shutdown_connections 2024-09-19T06:08:02.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f392e0b7740 1 --2- 172.21.15.5:0/1947199831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562e6be7a680 0x562e6bef6fe0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f392e0b7740 1 --2- 172.21.15.5:0/1947199831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e6be6fdc0 0x562e6bef3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f392e0b7740 1 --2- 172.21.15.5:0/1947199831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e6be6f4a0 0x562e6be7e110 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f392e0b7740 1 --2- 172.21.15.5:0/1947199831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e6be6ba30 0x562e6be7dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f390d7fa640 0 client.26129 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:02.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f392e0b7740 1 -- 172.21.15.5:0/1947199831 >> 172.21.15.5:0/1947199831 conn(0x562e6be778a0 msgr2=0x562e6be6a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:02.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f392e0b7740 1 -- 172.21.15.5:0/1947199831 shutdown_connections 2024-09-19T06:08:02.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f392e0b7740 1 -- 172.21.15.5:0/1947199831 wait complete. 2024-09-19T06:08:02.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.271+0000 7f392e0b7740 20 client.26129 trim_cache size 0 max 16384 2024-09-19T06:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:08:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.272+0000 7f392bcde640 1 -- 172.21.15.5:0/3527065062 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f39100a33b0 msgr2=0x7f39100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.272+0000 7f392bcde640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f39100a33b0 0x7f39100a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f392009c340 tx=0x7f39200d3c10 comp rx=0 tx=0).stop 2024-09-19T06:08:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.272+0000 7f392bcde640 1 -- 172.21.15.5:0/3527065062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3924102d30 msgr2=0x7f3924198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.272+0000 7f392bcde640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3924102d30 0x7f3924198390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f392000e9f0 tx=0x7f392000edf0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.272+0000 7f392bcde640 1 -- 172.21.15.5:0/3527065062 shutdown_connections 2024-09-19T06:08:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.272+0000 7f392bcde640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392410c160 0x7f392419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.272+0000 7f392bcde640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f39100a33b0 0x7f39100a5870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.272+0000 7f392bcde640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3924103650 0x7f39241988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.272+0000 7f392bcde640 1 --2- 172.21.15.5:0/3527065062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3924102d30 0x7f3924198390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.272+0000 7f392bcde640 1 -- 172.21.15.5:0/3527065062 >> 172.21.15.5:0/3527065062 conn(0x7f39240fc7d0 msgr2=0x7f39240fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.272+0000 7f392bcde640 1 -- 172.21.15.5:0/3527065062 shutdown_connections 2024-09-19T06:08:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:02.272+0000 7f392bcde640 1 -- 172.21.15.5:0/3527065062 wait complete. 2024-09-19T06:08:02.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:01 smithi157 ceph-mon[35346]: pgmap v144: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-19T06:08:02.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 108300 2.81 MB/sec execute 88 sec latency 33.831 ms 2024-09-19T06:08:03.094 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:08:03.094 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:08:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:03.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 108858 2.79 MB/sec execute 89 sec latency 19.395 ms 2024-09-19T06:08:03.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.927+0000 7f35cc63c640 1 -- 172.21.15.5:0/2985469109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c4101df0 msgr2=0x7f35c41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:03.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.927+0000 7f35cc63c640 1 --2- 172.21.15.5:0/2985469109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c4101df0 0x7f35c41021d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009900 tx=0x7f35c002a410 comp rx=0 tx=0).stop 2024-09-19T06:08:03.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.927+0000 7f35cc63c640 1 -- 172.21.15.5:0/2985469109 shutdown_connections 2024-09-19T06:08:03.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 --2- 172.21.15.5:0/2985469109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4106f20 0x7f35c410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:03.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 --2- 172.21.15.5:0/2985469109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35c41027a0 0x7f35c41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:03.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 --2- 172.21.15.5:0/2985469109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c4101df0 0x7f35c41021d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:03.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 -- 172.21.15.5:0/2985469109 >> 172.21.15.5:0/2985469109 conn(0x7f35c40fd8f0 msgr2=0x7f35c40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:03.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 -- 172.21.15.5:0/2985469109 shutdown_connections 2024-09-19T06:08:03.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 -- 172.21.15.5:0/2985469109 wait complete. 2024-09-19T06:08:03.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 Processor -- start 2024-09-19T06:08:03.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 -- start start 2024-09-19T06:08:03.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4101df0 0x7f35c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:03.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35c41027a0 0x7f35c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:03.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c4106f20 0x7f35c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:03.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35c4111e30 con 0x7f35c4106f20 2024-09-19T06:08:03.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35c4111cb0 con 0x7f35c41027a0 2024-09-19T06:08:03.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.928+0000 7f35cc63c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35c4111fb0 con 0x7f35c4101df0 2024-09-19T06:08:03.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35cabd9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c4106f20 0x7f35c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:03.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35ca3d8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4101df0 0x7f35c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:03.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35ca3d8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4101df0 0x7f35c40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59988/0 (socket says 172.21.15.5:59988) 2024-09-19T06:08:03.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35ca3d8640 1 -- 172.21.15.5:0/1498095827 learned_addr learned my addr 172.21.15.5:0/1498095827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:03.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35c9bd7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35c41027a0 0x7f35c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:03.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35ca3d8640 1 -- 172.21.15.5:0/1498095827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35c41027a0 msgr2=0x7f35c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:03.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35ca3d8640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35c41027a0 0x7f35c4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:03.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35ca3d8640 1 -- 172.21.15.5:0/1498095827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c4106f20 msgr2=0x7f35c4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:03.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35ca3d8640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c4106f20 0x7f35c4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:03.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35ca3d8640 1 -- 172.21.15.5:0/1498095827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c41a6870 con 0x7f35c4101df0 2024-09-19T06:08:03.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35c9bd7640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35c41027a0 0x7f35c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:03.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35cabd9640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c4106f20 0x7f35c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35ca3d8640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4101df0 0x7f35c40744f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f35c0005940 tx=0x7f35c00040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35bb7fe640 1 -- 172.21.15.5:0/1498095827 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c003f030 con 0x7f35c4101df0 2024-09-19T06:08:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35cc63c640 1 -- 172.21.15.5:0/1498095827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c41a6b00 con 0x7f35c4101df0 2024-09-19T06:08:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35bb7fe640 1 -- 172.21.15.5:0/1498095827 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35c00044c0 con 0x7f35c4101df0 2024-09-19T06:08:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.929+0000 7f35bb7fe640 1 -- 172.21.15.5:0/1498095827 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0035070 con 0x7f35c4101df0 2024-09-19T06:08:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.930+0000 7f35cc63c640 1 -- 172.21.15.5:0/1498095827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c41a7030 con 0x7f35c4101df0 2024-09-19T06:08:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.930+0000 7f35cc63c640 1 -- 172.21.15.5:0/1498095827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c41021d0 con 0x7f35c4101df0 2024-09-19T06:08:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.932+0000 7f35bb7fe640 1 -- 172.21.15.5:0/1498095827 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f35c0042030 con 0x7f35c4101df0 2024-09-19T06:08:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.933+0000 7f35bb7fe640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35b00a33b0 0x7f35b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.933+0000 7f35c9bd7640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35b00a33b0 0x7f35b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.933+0000 7f35bb7fe640 1 -- 172.21.15.5:0/1498095827 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f35c00f02d0 con 0x7f35c4101df0 2024-09-19T06:08:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.933+0000 7f35c9bd7640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35b00a33b0 0x7f35b00a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f35c4075f80 tx=0x7f35b4005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:03.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:03.936+0000 7f35bb7fe640 1 -- 172.21.15.5:0/1498095827 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f35c00bc610 con 0x7f35c4101df0 2024-09-19T06:08:04.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:03 smithi005 ceph-mon[30955]: pgmap v145: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-09-19T06:08:04.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:03 smithi049 ceph-mon[34640]: pgmap v145: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-09-19T06:08:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.181+0000 7f35cc63c640 1 -- 172.21.15.5:0/1498095827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35c410a100 con 0x7f35c4101df0 2024-09-19T06:08:04.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.181+0000 7f35bb7fe640 1 -- 172.21.15.5:0/1498095827 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11959 (secure 0 0 0) 0x7f35c00c0910 con 0x7f35c4101df0 2024-09-19T06:08:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:08:04.185 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-19T06:07:24:781633+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:07:24.019631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:08:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-19T06:08:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.184+0000 7f35cc63c640 1 -- 172.21.15.5:0/1498095827 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35b00a33b0 msgr2=0x7f35b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.184+0000 7f35cc63c640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35b00a33b0 0x7f35b00a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f35c4075f80 tx=0x7f35b4005f90 comp rx=0 tx=0).stop 2024-09-19T06:08:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.184+0000 7f35cc63c640 1 -- 172.21.15.5:0/1498095827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4101df0 msgr2=0x7f35c40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.184+0000 7f35cc63c640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4101df0 0x7f35c40744f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f35c0005940 tx=0x7f35c00040c0 comp rx=0 tx=0).stop 2024-09-19T06:08:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.184+0000 7f35cc63c640 1 -- 172.21.15.5:0/1498095827 shutdown_connections 2024-09-19T06:08:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.184+0000 7f35cc63c640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c4106f20 0x7f35c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.184+0000 7f35cc63c640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35c41027a0 0x7f35c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.184+0000 7f35cc63c640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35b00a33b0 0x7f35b00a5870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.184+0000 7f35cc63c640 1 --2- 172.21.15.5:0/1498095827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4101df0 0x7f35c40744f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.184+0000 7f35cc63c640 1 -- 172.21.15.5:0/1498095827 >> 172.21.15.5:0/1498095827 conn(0x7f35c40fd8f0 msgr2=0x7f35c40feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.184+0000 7f35cc63c640 1 -- 172.21.15.5:0/1498095827 shutdown_connections 2024-09-19T06:08:04.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:04.184+0000 7f35cc63c640 1 -- 172.21.15.5:0/1498095827 wait complete. 2024-09-19T06:08:04.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:03 smithi157 ceph-mon[35346]: pgmap v145: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-09-19T06:08:04.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 109623 2.83 MB/sec execute 90 sec latency 11.851 ms 2024-09-19T06:08:04.966 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:08:05.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:04 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1498095827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:08:05.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:04 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1498095827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:08:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:05.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:04 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1498095827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:08:05.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.768+0000 7fa3421d8640 1 -- 172.21.15.5:0/891360256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa33c103740 msgr2=0x7fa33c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:05.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.768+0000 7fa3421d8640 1 --2- 172.21.15.5:0/891360256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa33c103740 0x7fa33c107b80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa330009930 tx=0x7fa33002a420 comp rx=0 tx=0).stop 2024-09-19T06:08:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.768+0000 7fa3421d8640 1 -- 172.21.15.5:0/891360256 shutdown_connections 2024-09-19T06:08:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.768+0000 7fa3421d8640 1 --2- 172.21.15.5:0/891360256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c108290 0x7fa33c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.768+0000 7fa3421d8640 1 --2- 172.21.15.5:0/891360256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa33c103740 0x7fa33c107b80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.768+0000 7fa3421d8640 1 --2- 172.21.15.5:0/891360256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33c102d90 0x7fa33c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.768+0000 7fa3421d8640 1 -- 172.21.15.5:0/891360256 >> 172.21.15.5:0/891360256 conn(0x7fa33c0fc7d0 msgr2=0x7fa33c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.768+0000 7fa3421d8640 1 -- 172.21.15.5:0/891360256 shutdown_connections 2024-09-19T06:08:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.768+0000 7fa3421d8640 1 -- 172.21.15.5:0/891360256 wait complete. 2024-09-19T06:08:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa3421d8640 1 Processor -- start 2024-09-19T06:08:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa3421d8640 1 -- start start 2024-09-19T06:08:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa3421d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa33c102d90 0x7fa33c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa3421d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c103740 0x7fa33c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa3421d8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33c108290 0x7fa33c101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa3421d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c111f40 con 0x7fa33c102d90 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa3421d8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c111dc0 con 0x7fa33c103740 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa3421d8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1120c0 con 0x7fa33c108290 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa33b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa33c102d90 0x7fa33c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa33b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa33c102d90 0x7fa33c1009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35728/0 (socket says 172.21.15.5:35728) 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa33b7fe640 1 -- 172.21.15.5:0/134409853 learned_addr learned my addr 172.21.15.5:0/134409853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa33bfff640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33c108290 0x7fa33c101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.769+0000 7fa33affd640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c103740 0x7fa33c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.770+0000 7fa33b7fe640 1 -- 172.21.15.5:0/134409853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33c108290 msgr2=0x7fa33c101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.770+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33c108290 0x7fa33c101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.770+0000 7fa33b7fe640 1 -- 172.21.15.5:0/134409853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c103740 msgr2=0x7fa33c100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.770+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c103740 0x7fa33c100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.770+0000 7fa33b7fe640 1 -- 172.21.15.5:0/134409853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c1a69a0 con 0x7fa33c102d90 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.770+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa33c102d90 0x7fa33c1009f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00d950 tx=0x7fa32c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.770+0000 7fa338ff9640 1 -- 172.21.15.5:0/134409853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c017070 con 0x7fa33c102d90 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.770+0000 7fa338ff9640 1 -- 172.21.15.5:0/134409853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa32c009c30 con 0x7fa33c102d90 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.770+0000 7fa3421d8640 1 -- 172.21.15.5:0/134409853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33c1a6c90 con 0x7fa33c102d90 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.770+0000 7fa338ff9640 1 -- 172.21.15.5:0/134409853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c00f620 con 0x7fa33c102d90 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.770+0000 7fa3421d8640 1 -- 172.21.15.5:0/134409853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33c1a71a0 con 0x7fa33c102d90 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.773+0000 7fa338ff9640 1 -- 172.21.15.5:0/134409853 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa32c0109a0 con 0x7fa33c102d90 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.773+0000 7fa3445b1740 1 Processor -- start 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.773+0000 7fa3445b1740 1 -- start start 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.773+0000 7fa3445b1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d94b40 0x560081d93090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.773+0000 7fa3445b1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d935d0 0x560081d93a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.773+0000 7fa3445b1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 0x560081da0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.773+0000 7fa3445b1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560081cf92f0 con 0x560081d9e370 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.773+0000 7fa3445b1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560081ca0cf0 con 0x560081d935d0 2024-09-19T06:08:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.773+0000 7fa3445b1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560081cf52b0 con 0x560081d94b40 2024-09-19T06:08:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d94b40 0x560081d93090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 0x560081da0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d94b40 0x560081d93090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60402/0 (socket says 172.21.15.5:60402) 2024-09-19T06:08:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33bfff640 1 -- 172.21.15.5:0/3607781198 learned_addr learned my addr 172.21.15.5:0/3607781198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/3607781198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d935d0 0x560081d93a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa338ff9640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa3240a33b0 0x7fa3240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33bfff640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa3240a33b0 0x7fa3240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33affd640 1 -- 172.21.15.5:0/3607781198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d94b40 msgr2=0x560081d93090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33affd640 1 --2- 172.21.15.5:0/3607781198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d94b40 0x560081d93090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33affd640 1 -- 172.21.15.5:0/3607781198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d935d0 msgr2=0x560081d93a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33affd640 1 --2- 172.21.15.5:0/3607781198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d935d0 0x560081d93a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33affd640 1 -- 172.21.15.5:0/3607781198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560081c853a0 con 0x560081d9e370 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa338ff9640 1 -- 172.21.15.5:0/134409853 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa32c0cf890 con 0x7fa33c102d90 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33affd640 1 --2- 172.21.15.5:0/3607781198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 0x560081da0760 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa330009230 tx=0x7fa330005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d966fe9a8dc338 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa33bfff640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa3240a33b0 0x7fa3240a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa3340049d0 tx=0x7fa334004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.774+0000 7fa321ffb640 1 -- 172.21.15.5:0/3607781198 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330040070 con 0x560081d9e370 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.775+0000 7fa321ffb640 1 -- 172.21.15.5:0/3607781198 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa330004030 con 0x560081d9e370 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.775+0000 7fa321ffb640 1 -- 172.21.15.5:0/3607781198 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330035040 con 0x560081d9e370 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.775+0000 7fa3445b1740 1 -- 172.21.15.5:0/3607781198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 msgr2=0x560081da0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.775+0000 7fa3445b1740 1 --2- 172.21.15.5:0/3607781198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 0x560081da0760 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa330009230 tx=0x7fa330005fd0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.775+0000 7fa3445b1740 1 -- 172.21.15.5:0/3607781198 shutdown_connections 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.775+0000 7fa3445b1740 1 --2- 172.21.15.5:0/3607781198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 0x560081da0760 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.775+0000 7fa3445b1740 1 --2- 172.21.15.5:0/3607781198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d935d0 0x560081d93a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.775+0000 7fa3445b1740 1 --2- 172.21.15.5:0/3607781198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d94b40 0x560081d93090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.775+0000 7fa3445b1740 1 -- 172.21.15.5:0/3607781198 >> 172.21.15.5:0/3607781198 conn(0x560081d5c310 msgr2=0x560081d8fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.776+0000 7fa3445b1740 1 -- 172.21.15.5:0/3607781198 shutdown_connections 2024-09-19T06:08:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.776+0000 7fa3445b1740 1 -- 172.21.15.5:0/3607781198 wait complete. 2024-09-19T06:08:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.776+0000 7fa3445b1740 1 -- start start 2024-09-19T06:08:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.776+0000 7fa3445b1740 1 Processor -- start 2024-09-19T06:08:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa3445b1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d935d0 0x560081d9dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa3445b1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d94b40 0x560081d9b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa3445b1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 0x560081d9b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa3445b1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560081da1010 con 0x560081d9e370 2024-09-19T06:08:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa3445b1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560081cb4060 con 0x560081d935d0 2024-09-19T06:08:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa3445b1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560081da1190 con 0x560081d94b40 2024-09-19T06:08:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa3445b1740 10 client.0 _ll_register_callbacks cb 0x560081a078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 0x560081d9b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa33b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d935d0 0x560081d9dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 0x560081d9b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35742/0 (socket says 172.21.15.5:35742) 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa33bfff640 1 -- 172.21.15.5:0/1401919089 learned_addr learned my addr 172.21.15.5:0/1401919089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa33affd640 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d94b40 0x560081d9b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa33bfff640 1 -- 172.21.15.5:0/1401919089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d94b40 msgr2=0x560081d9b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa33bfff640 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d94b40 0x560081d9b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa33bfff640 1 -- 172.21.15.5:0/1401919089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d935d0 msgr2=0x560081d9dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa33bfff640 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d935d0 0x560081d9dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa33bfff640 1 -- 172.21.15.5:0/1401919089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560081a6a3c0 con 0x560081d9e370 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d935d0 0x560081d9dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa33bfff640 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 0x560081d9b600 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa334013fd0 tx=0x7fa334004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9724cbd9852f948 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.777+0000 7fa31affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa31affd640 1 -- 172.21.15.5:0/1401919089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334026070 con 0x560081d9e370 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa31affd640 1 -- 172.21.15.5:0/1401919089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa334004b20 con 0x560081d9e370 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa3445b1740 1 -- 172.21.15.5:0/1401919089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa330004030 con 0x560081d9e370 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa31affd640 1 -- 172.21.15.5:0/1401919089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa334031050 con 0x560081d9e370 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa3445b1740 10 client.16572 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa3445b1740 1 -- 172.21.15.5:0/1401919089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560081dacb60 con 0x560081d9e370 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa31affd640 1 -- 172.21.15.5:0/1401919089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa330004030 con 0x560081d9e370 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa31affd640 1 -- 172.21.15.5:0/1401919089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa334005550 con 0x560081d9e370 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa3445b1740 10 client.16572 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa3445b1740 10 client.16572 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa3445b1740 20 client.16572 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa3445b1740 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x560081dadf70 0x560081db0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa3445b1740 4 client.16572 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.778+0000 7fa3445b1740 1 -- 172.21.15.5:0/1401919089 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560081d06860 con 0x560081dadf70 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.779+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x560081dadf70 0x560081db0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.779+0000 7fa31affd640 10 client.16572 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.779+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x560081dadf70 0x560081db0410 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=162cd02a577a442f server_cookie=50d16fef08d30195 in_seq=0 out_seq=0 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa31affd640 1 -- 172.21.15.5:0/1401919089 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560081d06860 con 0x560081dadf70 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa31affd640 10 client.16572 handle_command_reply: tid=0 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa3445b1740 1 client.16572 shutdown 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa3445b1740 1 -- 172.21.15.5:0/1401919089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 msgr2=0x560081d9b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa3445b1740 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 0x560081d9b600 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa334013fd0 tx=0x7fa334004a40 comp rx=0 tx=0).stop 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa3445b1740 1 -- 172.21.15.5:0/1401919089 shutdown_connections 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa3445b1740 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x560081dadf70 0x560081db0410 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa3445b1740 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d9e370 0x560081d9b600 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa31affd640 0 client.16572 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa3445b1740 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d94b40 0x560081d9b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa3445b1740 1 --2- 172.21.15.5:0/1401919089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d935d0 0x560081d9dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa3445b1740 1 -- 172.21.15.5:0/1401919089 >> 172.21.15.5:0/1401919089 conn(0x560081d5c310 msgr2=0x560081d95960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa3445b1740 1 -- 172.21.15.5:0/1401919089 shutdown_connections 2024-09-19T06:08:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa3445b1740 1 -- 172.21.15.5:0/1401919089 wait complete. 2024-09-19T06:08:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.780+0000 7fa3445b1740 20 client.16572 trim_cache size 0 max 16384 2024-09-19T06:08:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa3445b1740 1 Processor -- start 2024-09-19T06:08:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa3445b1740 1 -- start start 2024-09-19T06:08:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa3445b1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d935d0 0x560081d8fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa3445b1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d94b40 0x560081daa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa3445b1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d9e370 0x560081d9a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa3445b1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1af990 con 0x560081d935d0 2024-09-19T06:08:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa3445b1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1b14d0 con 0x560081d94b40 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa3445b1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1b0060 con 0x560081d9e370 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa33b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d935d0 0x560081d8fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa33b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d935d0 0x560081d8fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35754/0 (socket says 172.21.15.5:35754) 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa33b7fe640 1 -- 172.21.15.5:0/766088927 learned_addr learned my addr 172.21.15.5:0/766088927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa33affd640 1 --2- 172.21.15.5:0/766088927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d94b40 0x560081daa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa33bfff640 1 --2- 172.21.15.5:0/766088927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d9e370 0x560081d9a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa33b7fe640 1 -- 172.21.15.5:0/766088927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d9e370 msgr2=0x560081d9a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/766088927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d9e370 0x560081d9a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa33b7fe640 1 -- 172.21.15.5:0/766088927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d94b40 msgr2=0x560081daa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/766088927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d94b40 0x560081daa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa33b7fe640 1 -- 172.21.15.5:0/766088927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560081d9e0d0 con 0x560081d935d0 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.795+0000 7fa33bfff640 1 --2- 172.21.15.5:0/766088927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d9e370 0x560081d9a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.796+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/766088927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d935d0 0x560081d8fe10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa32c097910 tx=0x7fa32c0d1da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c015c9c2acaf43f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.796+0000 7fa3217fa640 1 -- 172.21.15.5:0/766088927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c020040 con 0x560081d935d0 2024-09-19T06:08:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.796+0000 7fa3217fa640 1 -- 172.21.15.5:0/766088927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa32c099f30 con 0x560081d935d0 2024-09-19T06:08:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.796+0000 7fa3217fa640 1 -- 172.21.15.5:0/766088927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c086310 con 0x560081d935d0 2024-09-19T06:08:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.796+0000 7fa3445b1740 1 -- 172.21.15.5:0/766088927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d935d0 msgr2=0x560081d8fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.796+0000 7fa3445b1740 1 --2- 172.21.15.5:0/766088927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d935d0 0x560081d8fe10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa32c097910 tx=0x7fa32c0d1da0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.797+0000 7fa3445b1740 1 -- 172.21.15.5:0/766088927 shutdown_connections 2024-09-19T06:08:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.797+0000 7fa3445b1740 1 --2- 172.21.15.5:0/766088927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d9e370 0x560081d9a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.797+0000 7fa3445b1740 1 --2- 172.21.15.5:0/766088927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d94b40 0x560081daa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.797+0000 7fa3445b1740 1 --2- 172.21.15.5:0/766088927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d935d0 0x560081d8fe10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.797+0000 7fa3445b1740 1 -- 172.21.15.5:0/766088927 >> 172.21.15.5:0/766088927 conn(0x560081d5c310 msgr2=0x560081d8f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.797+0000 7fa3445b1740 1 -- 172.21.15.5:0/766088927 shutdown_connections 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.797+0000 7fa3445b1740 1 -- 172.21.15.5:0/766088927 wait complete. 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.797+0000 7fa3445b1740 1 -- start start 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.797+0000 7fa3445b1740 1 Processor -- start 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa3445b1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d935d0 0x560081da19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa3445b1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d94b40 0x560081da1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa3445b1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d9e370 0x560081e16cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa3445b1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1b0e00 con 0x560081d94b40 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa3445b1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1b0730 con 0x560081d9e370 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa3445b1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1ad970 con 0x560081d935d0 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa3445b1740 10 client.0 _ll_register_callbacks cb 0x7fa334013e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa33affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d94b40 0x560081da1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa33affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d94b40 0x560081da1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35758/0 (socket says 172.21.15.5:35758) 2024-09-19T06:08:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa33affd640 1 -- 172.21.15.5:0/2061217241 learned_addr learned my addr 172.21.15.5:0/2061217241 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa33bfff640 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d9e370 0x560081e16cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d935d0 0x560081da19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa33bfff640 1 -- 172.21.15.5:0/2061217241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d935d0 msgr2=0x560081da19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa33bfff640 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d935d0 0x560081da19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa33bfff640 1 -- 172.21.15.5:0/2061217241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d94b40 msgr2=0x560081da1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa33bfff640 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d94b40 0x560081da1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa33bfff640 1 -- 172.21.15.5:0/2061217241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560081e17230 con 0x560081d9e370 2024-09-19T06:08:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa33affd640 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d94b40 0x560081da1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa33bfff640 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d9e370 0x560081e16cf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa334010800 tx=0x7fa334022670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ba808d75b1a28ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.798+0000 7fa31affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:08:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa31affd640 1 -- 172.21.15.5:0/2061217241 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33403a060 con 0x560081d9e370 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa31affd640 1 -- 172.21.15.5:0/2061217241 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa334004d30 con 0x560081d9e370 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa3445b1740 1 -- 172.21.15.5:0/2061217241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa32c099f30 con 0x560081d9e370 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa31affd640 1 -- 172.21.15.5:0/2061217241 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa334026040 con 0x560081d9e370 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa3445b1740 10 client.26153 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa3445b1740 1 -- 172.21.15.5:0/2061217241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560081e18530 con 0x560081d9e370 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa31affd640 1 -- 172.21.15.5:0/2061217241 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa32c099f30 con 0x560081d9e370 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa31affd640 1 -- 172.21.15.5:0/2061217241 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa334022b60 con 0x560081d9e370 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa3445b1740 10 client.26153 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa3445b1740 10 client.26153 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa3445b1740 20 client.26153 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa3445b1740 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x560081dadf70 0x560081e1ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa3445b1740 4 client.26153 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:08:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa3445b1740 1 -- 172.21.15.5:0/2061217241 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560081d00d70 con 0x560081dadf70 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.799+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x560081dadf70 0x560081e1ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.800+0000 7fa31affd640 10 client.26153 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.800+0000 7fa33b7fe640 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x560081dadf70 0x560081e1ac90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f660488c7cfcc76 server_cookie=365074f4df1be8d5 in_seq=0 out_seq=0 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.800+0000 7fa31affd640 1 -- 172.21.15.5:0/2061217241 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560081d00d70 con 0x560081dadf70 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.800+0000 7fa31affd640 10 client.26153 handle_command_reply: tid=0 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.800+0000 7fa3445b1740 1 client.26153 shutdown 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3445b1740 1 -- 172.21.15.5:0/2061217241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d9e370 msgr2=0x560081e16cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3445b1740 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d9e370 0x560081e16cf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa334010800 tx=0x7fa334022670 comp rx=0 tx=0).stop 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3445b1740 1 -- 172.21.15.5:0/2061217241 shutdown_connections 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3445b1740 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x560081dadf70 0x560081e1ac90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3445b1740 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560081d9e370 0x560081e16cf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3445b1740 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081d94b40 0x560081da1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa31affd640 0 client.26153 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:08:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3445b1740 1 --2- 172.21.15.5:0/2061217241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560081d935d0 0x560081da19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3445b1740 1 -- 172.21.15.5:0/2061217241 >> 172.21.15.5:0/2061217241 conn(0x560081d5c310 msgr2=0x560081d8e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3445b1740 1 -- 172.21.15.5:0/2061217241 shutdown_connections 2024-09-19T06:08:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3445b1740 1 -- 172.21.15.5:0/2061217241 wait complete. 2024-09-19T06:08:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3445b1740 20 client.26153 trim_cache size 0 max 16384 2024-09-19T06:08:05.818 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:08:05.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3421d8640 1 -- 172.21.15.5:0/134409853 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa3240a33b0 msgr2=0x7fa3240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:05.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3421d8640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa3240a33b0 0x7fa3240a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa3340049d0 tx=0x7fa334004ab0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3421d8640 1 -- 172.21.15.5:0/134409853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa33c102d90 msgr2=0x7fa33c1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:05.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3421d8640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa33c102d90 0x7fa33c1009f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00d950 tx=0x7fa32c00de10 comp rx=0 tx=0).stop 2024-09-19T06:08:05.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3421d8640 1 -- 172.21.15.5:0/134409853 shutdown_connections 2024-09-19T06:08:05.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3421d8640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa3240a33b0 0x7fa3240a5870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.801+0000 7fa3421d8640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33c108290 0x7fa33c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.802+0000 7fa3421d8640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c103740 0x7fa33c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.802+0000 7fa3421d8640 1 --2- 172.21.15.5:0/134409853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa33c102d90 0x7fa33c1009f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:05.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.802+0000 7fa3421d8640 1 -- 172.21.15.5:0/134409853 >> 172.21.15.5:0/134409853 conn(0x7fa33c0fc7d0 msgr2=0x7fa33c105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:05.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.802+0000 7fa3421d8640 1 -- 172.21.15.5:0/134409853 shutdown_connections 2024-09-19T06:08:05.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:05.802+0000 7fa3421d8640 1 -- 172.21.15.5:0/134409853 wait complete. 2024-09-19T06:08:05.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 110060 2.85 MB/sec execute 91 sec latency 20.966 ms 2024-09-19T06:08:06.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:05 smithi005 ceph-mon[30955]: pgmap v146: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-09-19T06:08:06.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:05 smithi049 ceph-mon[34640]: pgmap v146: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-09-19T06:08:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:05 smithi157 ceph-mon[35346]: pgmap v146: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-09-19T06:08:06.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:08:06.827 INFO:tasks.workunit.client.0.smithi005.stdout: 1 110384 2.83 MB/sec execute 92 sec latency 15.322 ms 2024-09-19T06:08:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.438+0000 7fa43fb5f640 1 -- 172.21.15.5:0/3785195774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa438102ef0 msgr2=0x7fa43810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.438+0000 7fa43fb5f640 1 --2- 172.21.15.5:0/3785195774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa438102ef0 0x7fa43810da40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa430009ca0 tx=0x7fa43002a950 comp rx=0 tx=0).stop 2024-09-19T06:08:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.438+0000 7fa43fb5f640 1 -- 172.21.15.5:0/3785195774 shutdown_connections 2024-09-19T06:08:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.438+0000 7fa43fb5f640 1 --2- 172.21.15.5:0/3785195774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa438102ef0 0x7fa43810da40 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.438+0000 7fa43fb5f640 1 --2- 172.21.15.5:0/3785195774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438102570 0x7fa4381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.438+0000 7fa43fb5f640 1 --2- 172.21.15.5:0/3785195774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa438100e20 0x7fa438101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.438+0000 7fa43fb5f640 1 -- 172.21.15.5:0/3785195774 >> 172.21.15.5:0/3785195774 conn(0x7fa4380fc7d0 msgr2=0x7fa4380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.438+0000 7fa43fb5f640 1 -- 172.21.15.5:0/3785195774 shutdown_connections 2024-09-19T06:08:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.438+0000 7fa43fb5f640 1 -- 172.21.15.5:0/3785195774 wait complete. 2024-09-19T06:08:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43fb5f640 1 Processor -- start 2024-09-19T06:08:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43fb5f640 1 -- start start 2024-09-19T06:08:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43fb5f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa438100e20 0x7fa43819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43fb5f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438102570 0x7fa43819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43fb5f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa438102ef0 0x7fa43819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43fb5f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa438110e30 con 0x7fa438102ef0 2024-09-19T06:08:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43fb5f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa438110cb0 con 0x7fa438102570 2024-09-19T06:08:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43fb5f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa438110fb0 con 0x7fa438100e20 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43e0fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa438102ef0 0x7fa43819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43d8fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa438100e20 0x7fa43819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43d8fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa438100e20 0x7fa43819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60448/0 (socket says 172.21.15.5:60448) 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43d0fa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438102570 0x7fa43819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43d8fb640 1 -- 172.21.15.5:0/2828846523 learned_addr learned my addr 172.21.15.5:0/2828846523 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.439+0000 7fa43e0fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa438102ef0 0x7fa43819ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35774/0 (socket says 172.21.15.5:35774) 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.440+0000 7fa43d8fb640 1 -- 172.21.15.5:0/2828846523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438102570 msgr2=0x7fa43819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.440+0000 7fa43d8fb640 1 --2- 172.21.15.5:0/2828846523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438102570 0x7fa43819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.440+0000 7fa43d8fb640 1 -- 172.21.15.5:0/2828846523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa438102ef0 msgr2=0x7fa43819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.440+0000 7fa43d8fb640 1 --2- 172.21.15.5:0/2828846523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa438102ef0 0x7fa43819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.440+0000 7fa43d8fb640 1 -- 172.21.15.5:0/2828846523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43819f570 con 0x7fa438100e20 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.440+0000 7fa43d8fb640 1 --2- 172.21.15.5:0/2828846523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa438100e20 0x7fa43819a5c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa43400e9f0 tx=0x7fa43400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.440+0000 7fa42effd640 1 -- 172.21.15.5:0/2828846523 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43400cd80 con 0x7fa438100e20 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.440+0000 7fa42effd640 1 -- 172.21.15.5:0/2828846523 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4340054d0 con 0x7fa438100e20 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.440+0000 7fa43fb5f640 1 -- 172.21.15.5:0/2828846523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43819f860 con 0x7fa438100e20 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.440+0000 7fa42effd640 1 -- 172.21.15.5:0/2828846523 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434010490 con 0x7fa438100e20 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.440+0000 7fa43fb5f640 1 -- 172.21.15.5:0/2828846523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4381a7140 con 0x7fa438100e20 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.442+0000 7fa441f38740 1 Processor -- start 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa441f38740 1 -- start start 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa441f38740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6fb40 0x55cc9be6e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa441f38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be6e5d0 0x55cc9be6ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa441f38740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be79370 0x55cc9be7b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa441f38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cc9bdd42f0 con 0x55cc9be6e5d0 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa441f38740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cc9bd7bcf0 con 0x55cc9be6fb40 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa441f38740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cc9bdd02b0 con 0x55cc9be79370 2024-09-19T06:08:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa43e0fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6fb40 0x55cc9be6e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa43e0fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6fb40 0x55cc9be6e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39126/0 (socket says 172.21.15.5:39126) 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa43e0fc640 1 -- 172.21.15.5:0/596260992 learned_addr learned my addr 172.21.15.5:0/596260992 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa43d0fa640 1 --2- 172.21.15.5:0/596260992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be79370 0x55cc9be7b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa42effd640 1 -- 172.21.15.5:0/2828846523 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa434005020 con 0x7fa438100e20 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa43e0fc640 1 -- 172.21.15.5:0/596260992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be79370 msgr2=0x55cc9be7b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/596260992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be79370 0x55cc9be7b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.443+0000 7fa43e0fc640 1 -- 172.21.15.5:0/596260992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be6e5d0 msgr2=0x55cc9be6ea10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.444+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/596260992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be6e5d0 0x55cc9be6ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.444+0000 7fa43e0fc640 1 -- 172.21.15.5:0/596260992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc9bd603a0 con 0x55cc9be6fb40 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.444+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/596260992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6fb40 0x55cc9be6e090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa430035000 tx=0x7fa430005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a89a428bace31a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.444+0000 7fa42effd640 1 --2- 172.21.15.5:0/2828846523 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa4240a33b0 0x7fa4240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.444+0000 7fa423fff640 1 -- 172.21.15.5:0/596260992 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430048070 con 0x55cc9be6fb40 2024-09-19T06:08:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.444+0000 7fa43d8fb640 1 --2- 172.21.15.5:0/2828846523 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa4240a33b0 0x7fa4240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.444+0000 7fa423fff640 1 -- 172.21.15.5:0/596260992 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa43002f8c0 con 0x55cc9be6fb40 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.444+0000 7fa423fff640 1 -- 172.21.15.5:0/596260992 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430034040 con 0x55cc9be6fb40 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.444+0000 7fa42effd640 1 -- 172.21.15.5:0/2828846523 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa4340d00d0 con 0x7fa438100e20 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.445+0000 7fa43d8fb640 1 --2- 172.21.15.5:0/2828846523 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa4240a33b0 0x7fa4240a5870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55cc9be6bac0 tx=0x7fa4340bc040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.445+0000 7fa441f38740 1 -- 172.21.15.5:0/596260992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6fb40 msgr2=0x55cc9be6e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.445+0000 7fa441f38740 1 --2- 172.21.15.5:0/596260992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6fb40 0x55cc9be6e090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa430035000 tx=0x7fa430005fa0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.445+0000 7fa441f38740 1 -- 172.21.15.5:0/596260992 shutdown_connections 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.445+0000 7fa441f38740 1 --2- 172.21.15.5:0/596260992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be79370 0x55cc9be7b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.445+0000 7fa441f38740 1 --2- 172.21.15.5:0/596260992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be6e5d0 0x55cc9be6ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.445+0000 7fa441f38740 1 --2- 172.21.15.5:0/596260992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6fb40 0x55cc9be6e090 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.445+0000 7fa441f38740 1 -- 172.21.15.5:0/596260992 >> 172.21.15.5:0/596260992 conn(0x55cc9be37310 msgr2=0x55cc9be6ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.445+0000 7fa441f38740 1 -- 172.21.15.5:0/596260992 shutdown_connections 2024-09-19T06:08:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.446+0000 7fa441f38740 1 -- 172.21.15.5:0/596260992 wait complete. 2024-09-19T06:08:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.446+0000 7fa441f38740 1 -- start start 2024-09-19T06:08:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.446+0000 7fa441f38740 1 Processor -- start 2024-09-19T06:08:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.446+0000 7fa441f38740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6e5d0 0x55cc9be78d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.446+0000 7fa441f38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be6fb40 0x55cc9be76040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.446+0000 7fa441f38740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be79370 0x55cc9be76650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.446+0000 7fa441f38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cc9be7c010 con 0x55cc9be6fb40 2024-09-19T06:08:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.446+0000 7fa441f38740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cc9bd8f060 con 0x55cc9be79370 2024-09-19T06:08:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.446+0000 7fa441f38740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cc9be7c190 con 0x55cc9be6e5d0 2024-09-19T06:08:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.446+0000 7fa441f38740 10 client.0 _ll_register_callbacks cb 0x55cc9bae28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa43e0fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be79370 0x55cc9be76650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa43e0fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be79370 0x55cc9be76650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39134/0 (socket says 172.21.15.5:39134) 2024-09-19T06:08:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa43e0fc640 1 -- 172.21.15.5:0/2993625753 learned_addr learned my addr 172.21.15.5:0/2993625753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa43d0fa640 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be6fb40 0x55cc9be76040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6e5d0 0x55cc9be78d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa43e0fc640 1 -- 172.21.15.5:0/2993625753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6e5d0 msgr2=0x55cc9be78d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6e5d0 0x55cc9be78d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa43e0fc640 1 -- 172.21.15.5:0/2993625753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be6fb40 msgr2=0x55cc9be76040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be6fb40 0x55cc9be76040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa43e0fc640 1 -- 172.21.15.5:0/2993625753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc9bb453c0 con 0x55cc9be79370 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be79370 0x55cc9be76650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa43002ae70 tx=0x7fa430003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=527272f565ea36a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa4217fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa4217fa640 1 -- 172.21.15.5:0/2993625753 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430048070 con 0x55cc9be79370 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa441f38740 1 -- 172.21.15.5:0/2993625753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa43002f8c0 con 0x55cc9be79370 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.447+0000 7fa4217fa640 1 -- 172.21.15.5:0/2993625753 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa430004040 con 0x55cc9be79370 2024-09-19T06:08:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.448+0000 7fa4217fa640 1 -- 172.21.15.5:0/2993625753 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa430004240 con 0x55cc9be79370 2024-09-19T06:08:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.448+0000 7fa441f38740 10 client.26165 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.448+0000 7fa441f38740 1 -- 172.21.15.5:0/2993625753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cc9be90320 con 0x55cc9be79370 2024-09-19T06:08:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.448+0000 7fa4217fa640 1 -- 172.21.15.5:0/2993625753 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa43002f8c0 con 0x55cc9be79370 2024-09-19T06:08:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.448+0000 7fa4217fa640 1 -- 172.21.15.5:0/2993625753 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa4300309f0 con 0x55cc9be79370 2024-09-19T06:08:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.448+0000 7fa441f38740 10 client.26165 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.448+0000 7fa441f38740 10 client.26165 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:08:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.448+0000 7fa441f38740 20 client.26165 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.448+0000 7fa441f38740 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55cc9be91670 0x55cc9be93a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.448+0000 7fa441f38740 4 client.26165 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.448+0000 7fa441f38740 1 -- 172.21.15.5:0/2993625753 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc9bde1860 con 0x55cc9be91670 2024-09-19T06:08:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.448+0000 7fa43d0fa640 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55cc9be91670 0x55cc9be93a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.449+0000 7fa4217fa640 10 client.26165 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:08:07.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.449+0000 7fa43d0fa640 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55cc9be91670 0x55cc9be93a60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=19dc3432285d4a47 server_cookie=d6ae79d256c95b0c in_seq=0 out_seq=0 2024-09-19T06:08:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.449+0000 7fa4217fa640 1 -- 172.21.15.5:0/2993625753 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc9bde1860 con 0x55cc9be91670 2024-09-19T06:08:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.449+0000 7fa4217fa640 10 client.26165 handle_command_reply: tid=0 2024-09-19T06:08:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.449+0000 7fa441f38740 1 client.26165 shutdown 2024-09-19T06:08:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.450+0000 7fa441f38740 1 -- 172.21.15.5:0/2993625753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be79370 msgr2=0x55cc9be76650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.450+0000 7fa441f38740 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be79370 0x55cc9be76650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa43002ae70 tx=0x7fa430003c00 comp rx=0 tx=0).stop 2024-09-19T06:08:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.450+0000 7fa441f38740 1 -- 172.21.15.5:0/2993625753 shutdown_connections 2024-09-19T06:08:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.450+0000 7fa441f38740 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55cc9be91670 0x55cc9be93a60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.450+0000 7fa441f38740 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be79370 0x55cc9be76650 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.450+0000 7fa441f38740 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be6fb40 0x55cc9be76040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.450+0000 7fa441f38740 1 --2- 172.21.15.5:0/2993625753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6e5d0 0x55cc9be78d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.450+0000 7fa441f38740 1 -- 172.21.15.5:0/2993625753 >> 172.21.15.5:0/2993625753 conn(0x55cc9be37310 msgr2=0x55cc9be70930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.450+0000 7fa4217fa640 0 client.26165 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:08:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.450+0000 7fa441f38740 1 -- 172.21.15.5:0/2993625753 shutdown_connections 2024-09-19T06:08:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.450+0000 7fa441f38740 1 -- 172.21.15.5:0/2993625753 wait complete. 2024-09-19T06:08:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.450+0000 7fa441f38740 20 client.26165 trim_cache size 0 max 16384 2024-09-19T06:08:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.463+0000 7fa441f38740 1 Processor -- start 2024-09-19T06:08:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.463+0000 7fa441f38740 1 -- start start 2024-09-19T06:08:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.463+0000 7fa441f38740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 0x55cc9be78370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.463+0000 7fa441f38740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6fb40 0x55cc9be6a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.463+0000 7fa441f38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be79370 0x55cc9be6aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.463+0000 7fa441f38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4381ab5d0 con 0x55cc9be79370 2024-09-19T06:08:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.463+0000 7fa441f38740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa4381ad110 con 0x55cc9be6e5d0 2024-09-19T06:08:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.463+0000 7fa441f38740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4381abca0 con 0x55cc9be6fb40 2024-09-19T06:08:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa43e0fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be79370 0x55cc9be6aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa43e0fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 0x55cc9be78370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa43e0fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 0x55cc9be78370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39150/0 (socket says 172.21.15.5:39150) 2024-09-19T06:08:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa43e0fc640 1 -- 172.21.15.5:0/3397472429 learned_addr learned my addr 172.21.15.5:0/3397472429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa43d0fa640 1 --2- 172.21.15.5:0/3397472429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6fb40 0x55cc9be6a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa43e0fc640 1 -- 172.21.15.5:0/3397472429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6fb40 msgr2=0x55cc9be6a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/3397472429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6fb40 0x55cc9be6a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa43e0fc640 1 -- 172.21.15.5:0/3397472429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be79370 msgr2=0x55cc9be6aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/3397472429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be79370 0x55cc9be6aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa43e0fc640 1 -- 172.21.15.5:0/3397472429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc9be6ed80 con 0x55cc9be6e5d0 2024-09-19T06:08:07.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/3397472429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 0x55cc9be78370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa430003c70 tx=0x7fa430000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ff903bd8aebc89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:07.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa4237fe640 1 -- 172.21.15.5:0/3397472429 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43005d040 con 0x55cc9be6e5d0 2024-09-19T06:08:07.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa4237fe640 1 -- 172.21.15.5:0/3397472429 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa430049040 con 0x55cc9be6e5d0 2024-09-19T06:08:07.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.464+0000 7fa4237fe640 1 -- 172.21.15.5:0/3397472429 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430048040 con 0x55cc9be6e5d0 2024-09-19T06:08:07.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.465+0000 7fa441f38740 1 -- 172.21.15.5:0/3397472429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 msgr2=0x55cc9be78370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:07.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.465+0000 7fa441f38740 1 --2- 172.21.15.5:0/3397472429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 0x55cc9be78370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa430003c70 tx=0x7fa430000f40 comp rx=0 tx=0).stop 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.465+0000 7fa441f38740 1 -- 172.21.15.5:0/3397472429 shutdown_connections 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.465+0000 7fa441f38740 1 --2- 172.21.15.5:0/3397472429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be79370 0x55cc9be6aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.465+0000 7fa441f38740 1 --2- 172.21.15.5:0/3397472429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6fb40 0x55cc9be6a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.465+0000 7fa441f38740 1 --2- 172.21.15.5:0/3397472429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 0x55cc9be78370 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.465+0000 7fa441f38740 1 -- 172.21.15.5:0/3397472429 >> 172.21.15.5:0/3397472429 conn(0x55cc9be37310 msgr2=0x55cc9be70930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.465+0000 7fa441f38740 1 -- 172.21.15.5:0/3397472429 shutdown_connections 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.465+0000 7fa441f38740 1 -- 172.21.15.5:0/3397472429 wait complete. 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.466+0000 7fa441f38740 1 -- start start 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.466+0000 7fa441f38740 1 Processor -- start 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.466+0000 7fa441f38740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 0x55cc9be94810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.466+0000 7fa441f38740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6fb40 0x55cc9be94d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.466+0000 7fa441f38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be79370 0x55cc9be95290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.466+0000 7fa441f38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4381aca40 con 0x55cc9be79370 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.466+0000 7fa441f38740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa4381ac370 con 0x55cc9be6e5d0 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.466+0000 7fa441f38740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4381a95f0 con 0x55cc9be6fb40 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.466+0000 7fa441f38740 10 client.0 _ll_register_callbacks cb 0x7fa430046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.466+0000 7fa43e0fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 0x55cc9be94810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa43d0fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6fb40 0x55cc9be94d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa43d0fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6fb40 0x55cc9be94d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60490/0 (socket says 172.21.15.5:60490) 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa43d0fa640 1 -- 172.21.15.5:0/635705170 learned_addr learned my addr 172.21.15.5:0/635705170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa43e0fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 0x55cc9be94810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39154/0 (socket says 172.21.15.5:39154) 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be79370 0x55cc9be95290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa43d0fa640 1 -- 172.21.15.5:0/635705170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 msgr2=0x55cc9be94810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa43d0fa640 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 0x55cc9be94810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa43d0fa640 1 -- 172.21.15.5:0/635705170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be79370 msgr2=0x55cc9be95290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:07.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa43d0fa640 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be79370 0x55cc9be95290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa43d0fa640 1 -- 172.21.15.5:0/635705170 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc9befafb0 con 0x55cc9be6fb40 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be79370 0x55cc9be95290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa43d0fa640 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6fb40 0x55cc9be94d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa428005940 tx=0x7fa42800bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32311b01a30449b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa4217fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa4217fa640 1 -- 172.21.15.5:0/635705170 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa428005220 con 0x55cc9be6fb40 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa4217fa640 1 -- 172.21.15.5:0/635705170 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa4280053c0 con 0x55cc9be6fb40 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.467+0000 7fa441f38740 1 -- 172.21.15.5:0/635705170 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa430049040 con 0x55cc9be6fb40 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.468+0000 7fa4217fa640 1 -- 172.21.15.5:0/635705170 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa428003e90 con 0x55cc9be6fb40 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.468+0000 7fa441f38740 10 client.26104 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.468+0000 7fa441f38740 1 -- 172.21.15.5:0/635705170 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cc9befc2c0 con 0x55cc9be6fb40 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.468+0000 7fa4217fa640 1 -- 172.21.15.5:0/635705170 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4280055c0 con 0x55cc9be6fb40 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.468+0000 7fa4217fa640 1 -- 172.21.15.5:0/635705170 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa42802f7f0 con 0x55cc9be6fb40 2024-09-19T06:08:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.468+0000 7fa441f38740 10 client.26104 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.468+0000 7fa441f38740 10 client.26104 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:08:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.468+0000 7fa441f38740 20 client.26104 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.468+0000 7fa441f38740 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55cc9befcac0 0x55cc9befeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.468+0000 7fa441f38740 4 client.26104 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:08:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.468+0000 7fa441f38740 1 -- 172.21.15.5:0/635705170 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa430049040 con 0x55cc9befcac0 2024-09-19T06:08:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.468+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55cc9befcac0 0x55cc9befeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa43e0fc640 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55cc9befcac0 0x55cc9befeee0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ca7373059942490 server_cookie=7f057c63063ea9cd in_seq=0 out_seq=0 2024-09-19T06:08:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa4217fa640 10 client.26104 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:08:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa4217fa640 1 -- 172.21.15.5:0/635705170 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa430049040 con 0x55cc9befcac0 2024-09-19T06:08:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa4217fa640 10 client.26104 handle_command_reply: tid=0 2024-09-19T06:08:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa441f38740 1 client.26104 shutdown 2024-09-19T06:08:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa441f38740 1 -- 172.21.15.5:0/635705170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6fb40 msgr2=0x55cc9be94d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa441f38740 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6fb40 0x55cc9be94d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa428005940 tx=0x7fa42800bf90 comp rx=0 tx=0).stop 2024-09-19T06:08:07.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa441f38740 1 -- 172.21.15.5:0/635705170 shutdown_connections 2024-09-19T06:08:07.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa441f38740 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55cc9befcac0 0x55cc9befeee0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa441f38740 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc9be79370 0x55cc9be95290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa441f38740 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc9be6fb40 0x55cc9be94d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa4217fa640 0 client.26104 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:08:07.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa441f38740 1 --2- 172.21.15.5:0/635705170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc9be6e5d0 0x55cc9be94810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa441f38740 1 -- 172.21.15.5:0/635705170 >> 172.21.15.5:0/635705170 conn(0x55cc9be37310 msgr2=0x55cc9be75900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:07.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa441f38740 1 -- 172.21.15.5:0/635705170 shutdown_connections 2024-09-19T06:08:07.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.469+0000 7fa441f38740 1 -- 172.21.15.5:0/635705170 wait complete. 2024-09-19T06:08:07.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa441f38740 20 client.26104 trim_cache size 0 max 16384 2024-09-19T06:08:07.478 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:08:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa43fb5f640 1 -- 172.21.15.5:0/2828846523 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa4240a33b0 msgr2=0x7fa4240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa43fb5f640 1 --2- 172.21.15.5:0/2828846523 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa4240a33b0 0x7fa4240a5870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55cc9be6bac0 tx=0x7fa4340bc040 comp rx=0 tx=0).stop 2024-09-19T06:08:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa43fb5f640 1 -- 172.21.15.5:0/2828846523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa438100e20 msgr2=0x7fa43819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa43fb5f640 1 --2- 172.21.15.5:0/2828846523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa438100e20 0x7fa43819a5c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa43400e9f0 tx=0x7fa43400edf0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa43fb5f640 1 -- 172.21.15.5:0/2828846523 shutdown_connections 2024-09-19T06:08:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa43fb5f640 1 --2- 172.21.15.5:0/2828846523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa438102ef0 0x7fa43819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa43fb5f640 1 --2- 172.21.15.5:0/2828846523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438102570 0x7fa43819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa43fb5f640 1 --2- 172.21.15.5:0/2828846523 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa4240a33b0 0x7fa4240a5870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa43fb5f640 1 --2- 172.21.15.5:0/2828846523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa438100e20 0x7fa43819a5c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa43fb5f640 1 -- 172.21.15.5:0/2828846523 >> 172.21.15.5:0/2828846523 conn(0x7fa4380fc7d0 msgr2=0x7fa43806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa43fb5f640 1 -- 172.21.15.5:0/2828846523 shutdown_connections 2024-09-19T06:08:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:07.470+0000 7fa43fb5f640 1 -- 172.21.15.5:0/2828846523 wait complete. 2024-09-19T06:08:07.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 110666 2.80 MB/sec execute 93 sec latency 26.361 ms 2024-09-19T06:08:08.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:07 smithi005 ceph-mon[30955]: pgmap v147: 97 pgs: 97 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-09-19T06:08:08.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:07 smithi049 ceph-mon[34640]: pgmap v147: 97 pgs: 97 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-09-19T06:08:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.161+0000 7fe718776640 1 -- 172.21.15.5:0/92392429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710102460 msgr2=0x7fe71010e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.161+0000 7fe718776640 1 --2- 172.21.15.5:0/92392429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710102460 0x7fe71010e990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe708009cd0 tx=0x7fe70802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.161+0000 7fe718776640 1 -- 172.21.15.5:0/92392429 shutdown_connections 2024-09-19T06:08:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.161+0000 7fe718776640 1 --2- 172.21.15.5:0/92392429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710102460 0x7fe71010e990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.161+0000 7fe718776640 1 --2- 172.21.15.5:0/92392429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe710101ac0 0x7fe710101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.161+0000 7fe718776640 1 --2- 172.21.15.5:0/92392429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe710107ad0 0x7fe710107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.161+0000 7fe718776640 1 -- 172.21.15.5:0/92392429 >> 172.21.15.5:0/92392429 conn(0x7fe7100fd7a0 msgr2=0x7fe7100ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.161+0000 7fe718776640 1 -- 172.21.15.5:0/92392429 shutdown_connections 2024-09-19T06:08:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.161+0000 7fe718776640 1 -- 172.21.15.5:0/92392429 wait complete. 2024-09-19T06:08:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe718776640 1 Processor -- start 2024-09-19T06:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe718776640 1 -- start start 2024-09-19T06:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe718776640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710101ac0 0x7fe710198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe718776640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe710102460 0x7fe710198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe718776640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe710107ad0 0x7fe71019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe718776640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe710111dc0 con 0x7fe710101ac0 2024-09-19T06:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe718776640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe710111c40 con 0x7fe710107ad0 2024-09-19T06:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe718776640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe710111f40 con 0x7fe710102460 2024-09-19T06:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe716512640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710101ac0 0x7fe710198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe716512640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710101ac0 0x7fe710198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35838/0 (socket says 172.21.15.5:35838) 2024-09-19T06:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe716512640 1 -- 172.21.15.5:0/1131452256 learned_addr learned my addr 172.21.15.5:0/1131452256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe716d13640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe710107ad0 0x7fe71019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe715d11640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe710102460 0x7fe710198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe716512640 1 -- 172.21.15.5:0/1131452256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe710102460 msgr2=0x7fe710198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe716512640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe710102460 0x7fe710198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe716512640 1 -- 172.21.15.5:0/1131452256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe710107ad0 msgr2=0x7fe71019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe716512640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe710107ad0 0x7fe71019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe716512640 1 -- 172.21.15.5:0/1131452256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71019d320 con 0x7fe710101ac0 2024-09-19T06:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.162+0000 7fe716d13640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe710107ad0 0x7fe71019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.163+0000 7fe716512640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710101ac0 0x7fe710198330 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00d770 tx=0x7fe70c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.163+0000 7fe7077fe640 1 -- 172.21.15.5:0/1131452256 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c017070 con 0x7fe710101ac0 2024-09-19T06:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.163+0000 7fe7077fe640 1 -- 172.21.15.5:0/1131452256 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe70c0054d0 con 0x7fe710101ac0 2024-09-19T06:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.163+0000 7fe7077fe640 1 -- 172.21.15.5:0/1131452256 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c004cf0 con 0x7fe710101ac0 2024-09-19T06:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.163+0000 7fe718776640 1 -- 172.21.15.5:0/1131452256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71019d610 con 0x7fe710101ac0 2024-09-19T06:08:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.163+0000 7fe718776640 1 -- 172.21.15.5:0/1131452256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7101a4ea0 con 0x7fe710101ac0 2024-09-19T06:08:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.165+0000 7fe7077fe640 1 -- 172.21.15.5:0/1131452256 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe70c004e90 con 0x7fe710101ac0 2024-09-19T06:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe71b5ed740 1 Processor -- start 2024-09-19T06:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe71b5ed740 1 -- start start 2024-09-19T06:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe71b5ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f713ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe71b5ed740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f714410 0x559e5f71e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe71b5ed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f71ecb0 0x559e5f7210a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe71b5ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e5f6782f0 con 0x559e5f713af0 2024-09-19T06:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe71b5ed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559e5f620270 con 0x559e5f71ecb0 2024-09-19T06:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe71b5ed740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559e5f6742b0 con 0x559e5f714410 2024-09-19T06:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe716512640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f714410 0x559e5f71e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe716d13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f713ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe716d13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f713ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35850/0 (socket says 172.21.15.5:35850) 2024-09-19T06:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe715d11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f71ecb0 0x559e5f7210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.166+0000 7fe716d13640 1 -- 172.21.15.5:0/1943346724 learned_addr learned my addr 172.21.15.5:0/1943346724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe715d11640 1 -- 172.21.15.5:0/1943346724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f714410 msgr2=0x559e5f71e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe715d11640 1 --2- 172.21.15.5:0/1943346724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f714410 0x559e5f71e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe715d11640 1 -- 172.21.15.5:0/1943346724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 msgr2=0x559e5f713ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe715d11640 1 --2- 172.21.15.5:0/1943346724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f713ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe715d11640 1 -- 172.21.15.5:0/1943346724 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e5f61f340 con 0x559e5f71ecb0 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe716d13640 1 --2- 172.21.15.5:0/1943346724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f713ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe7077fe640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe6fc0a33b0 0x7fe6fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe715d11640 1 --2- 172.21.15.5:0/1943346724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f71ecb0 0x559e5f7210a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe700009b40 tx=0x7fe7000113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad8889f5084f00c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe716d13640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe6fc0a33b0 0x7fe6fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe7077fe640 1 -- 172.21.15.5:0/1131452256 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fe70c0cf6e0 con 0x7fe710101ac0 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe704ff9640 1 -- 172.21.15.5:0/1943346724 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700011e10 con 0x559e5f71ecb0 2024-09-19T06:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe704ff9640 1 -- 172.21.15.5:0/1943346724 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe700012900 con 0x559e5f71ecb0 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.167+0000 7fe704ff9640 1 -- 172.21.15.5:0/1943346724 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70000f9a0 con 0x559e5f71ecb0 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.168+0000 7fe716d13640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe6fc0a33b0 0x7fe6fc0a5870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe708004af0 tx=0x7fe70803d040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.168+0000 7fe71b5ed740 1 -- 172.21.15.5:0/1943346724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f71ecb0 msgr2=0x559e5f7210a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.168+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/1943346724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f71ecb0 0x559e5f7210a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe700009b40 tx=0x7fe7000113e0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.168+0000 7fe71b5ed740 1 -- 172.21.15.5:0/1943346724 shutdown_connections 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.168+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/1943346724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f71ecb0 0x559e5f7210a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.168+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/1943346724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f714410 0x559e5f71e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.168+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/1943346724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f713ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.168+0000 7fe71b5ed740 1 -- 172.21.15.5:0/1943346724 >> 172.21.15.5:0/1943346724 conn(0x559e5f710060 msgr2=0x559e5f710440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.168+0000 7fe71b5ed740 1 -- 172.21.15.5:0/1943346724 shutdown_connections 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.168+0000 7fe71b5ed740 1 -- 172.21.15.5:0/1943346724 wait complete. 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe71b5ed740 1 -- start start 2024-09-19T06:08:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe71b5ed740 1 Processor -- start 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe71b5ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f71bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe71b5ed740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f714410 0x559e5f71c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe71b5ed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f71ecb0 0x559e5f71c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe71b5ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e5f721880 con 0x559e5f713af0 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe71b5ed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559e5f601290 con 0x559e5f71ecb0 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe71b5ed740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559e5f721a00 con 0x559e5f714410 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe71b5ed740 10 client.0 _ll_register_callbacks cb 0x559e5f632140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe716512640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f71bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe716512640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f71bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35852/0 (socket says 172.21.15.5:35852) 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe716512640 1 -- 172.21.15.5:0/4039776628 learned_addr learned my addr 172.21.15.5:0/4039776628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe715d11640 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f714410 0x559e5f71c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe716d13640 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f71ecb0 0x559e5f71c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe716512640 1 -- 172.21.15.5:0/4039776628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f714410 msgr2=0x559e5f71c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe716512640 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f714410 0x559e5f71c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe716512640 1 -- 172.21.15.5:0/4039776628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f71ecb0 msgr2=0x559e5f71c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe716512640 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f71ecb0 0x559e5f71c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.169+0000 7fe716512640 1 -- 172.21.15.5:0/4039776628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5f38b210 con 0x559e5f713af0 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe716512640 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f71bb50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe70c021000 tx=0x7fe70c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55eab295ef3bc016 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe6f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe6f9ffb640 1 -- 172.21.15.5:0/4039776628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c020070 con 0x559e5f713af0 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe6f9ffb640 1 -- 172.21.15.5:0/4039776628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fe70c098040 con 0x559e5f713af0 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe71b5ed740 1 -- 172.21.15.5:0/4039776628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe700012900 con 0x559e5f713af0 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe6f9ffb640 1 -- 172.21.15.5:0/4039776628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe70c09b7a0 con 0x559e5f713af0 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe71b5ed740 10 client.16602 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe71b5ed740 1 -- 172.21.15.5:0/4039776628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559e5f457620 con 0x559e5f713af0 2024-09-19T06:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe6f9ffb640 1 -- 172.21.15.5:0/4039776628 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe700012900 con 0x559e5f713af0 2024-09-19T06:08:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe6f9ffb640 1 -- 172.21.15.5:0/4039776628 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe70c0b4bd0 con 0x559e5f713af0 2024-09-19T06:08:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe71b5ed740 10 client.16602 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe71b5ed740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.170+0000 7fe71b5ed740 20 client.16602 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.171+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559e5f72ec30 0x559e5f731040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.171+0000 7fe71b5ed740 4 client.16602 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.171+0000 7fe71b5ed740 1 -- 172.21.15.5:0/4039776628 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e5f685a20 con 0x559e5f72ec30 2024-09-19T06:08:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.171+0000 7fe715d11640 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559e5f72ec30 0x559e5f731040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.171+0000 7fe6f9ffb640 10 client.16602 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.171+0000 7fe715d11640 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559e5f72ec30 0x559e5f731040 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4520facddc17949 server_cookie=66f6b8150385a6ee in_seq=0 out_seq=0 2024-09-19T06:08:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe6f9ffb640 1 -- 172.21.15.5:0/4039776628 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e5f685a20 con 0x559e5f72ec30 2024-09-19T06:08:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe6f9ffb640 10 client.16602 handle_command_reply: tid=0 2024-09-19T06:08:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe71b5ed740 1 client.16602 shutdown 2024-09-19T06:08:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe71b5ed740 1 -- 172.21.15.5:0/4039776628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 msgr2=0x559e5f71bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f71bb50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe70c021000 tx=0x7fe70c005740 comp rx=0 tx=0).stop 2024-09-19T06:08:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe71b5ed740 1 -- 172.21.15.5:0/4039776628 shutdown_connections 2024-09-19T06:08:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559e5f72ec30 0x559e5f731040 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f71ecb0 0x559e5f71c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f714410 0x559e5f71c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/4039776628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f71bb50 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe6f9ffb640 0 client.16602 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe71b5ed740 1 -- 172.21.15.5:0/4039776628 >> 172.21.15.5:0/4039776628 conn(0x559e5f710060 msgr2=0x559e5f715dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe71b5ed740 1 -- 172.21.15.5:0/4039776628 shutdown_connections 2024-09-19T06:08:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe71b5ed740 1 -- 172.21.15.5:0/4039776628 wait complete. 2024-09-19T06:08:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.172+0000 7fe71b5ed740 20 client.16602 trim_cache size 0 max 16384 2024-09-19T06:08:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe71b5ed740 1 Processor -- start 2024-09-19T06:08:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe71b5ed740 1 -- start start 2024-09-19T06:08:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe71b5ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f71c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe71b5ed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f714410 0x559e5f71dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe71b5ed740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f71ecb0 0x559e5f7368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe71b5ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7101ad870 con 0x559e5f713af0 2024-09-19T06:08:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe71b5ed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe7101af3b0 con 0x559e5f714410 2024-09-19T06:08:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe71b5ed740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe7101adf40 con 0x559e5f71ecb0 2024-09-19T06:08:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe716512640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f71c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe716512640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f71c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35862/0 (socket says 172.21.15.5:35862) 2024-09-19T06:08:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe716d13640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f71ecb0 0x559e5f7368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe716512640 1 -- 172.21.15.5:0/3282731612 learned_addr learned my addr 172.21.15.5:0/3282731612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.189+0000 7fe715d11640 1 --2- 172.21.15.5:0/3282731612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f714410 0x559e5f71dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.190+0000 7fe716512640 1 -- 172.21.15.5:0/3282731612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f71ecb0 msgr2=0x559e5f7368b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.190+0000 7fe716512640 1 --2- 172.21.15.5:0/3282731612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f71ecb0 0x559e5f7368b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.190+0000 7fe716512640 1 -- 172.21.15.5:0/3282731612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f714410 msgr2=0x559e5f71dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.190+0000 7fe716512640 1 --2- 172.21.15.5:0/3282731612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f714410 0x559e5f71dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.190+0000 7fe716512640 1 -- 172.21.15.5:0/3282731612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e5f70fe90 con 0x559e5f713af0 2024-09-19T06:08:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.190+0000 7fe716512640 1 --2- 172.21.15.5:0/3282731612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f71c400 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe70c097e30 tx=0x7fe70c0862c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd0e7a704efd8b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.190+0000 7fe6fbfff640 1 -- 172.21.15.5:0/3282731612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c020050 con 0x559e5f713af0 2024-09-19T06:08:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.190+0000 7fe6fbfff640 1 -- 172.21.15.5:0/3282731612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe70c0b8020 con 0x559e5f713af0 2024-09-19T06:08:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.190+0000 7fe6fbfff640 1 -- 172.21.15.5:0/3282731612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c005180 con 0x559e5f713af0 2024-09-19T06:08:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3282731612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 msgr2=0x559e5f71c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/3282731612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f71c400 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe70c097e30 tx=0x7fe70c0862c0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3282731612 shutdown_connections 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/3282731612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f71ecb0 0x559e5f7368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/3282731612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f714410 0x559e5f71dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/3282731612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f713af0 0x559e5f71c400 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3282731612 >> 172.21.15.5:0/3282731612 conn(0x559e5f710060 msgr2=0x559e5f71d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3282731612 shutdown_connections 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3282731612 wait complete. 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 -- start start 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 Processor -- start 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f713af0 0x559e5f727410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.191+0000 7fe71b5ed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f714410 0x559e5f722730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe71b5ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f71ecb0 0x559e5f722c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe71b5ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7101aece0 con 0x559e5f71ecb0 2024-09-19T06:08:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe71b5ed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe7101ae610 con 0x559e5f714410 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe71b5ed740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe7101ab850 con 0x559e5f713af0 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe71b5ed740 10 client.0 _ll_register_callbacks cb 0x7fe70c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe715d11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f714410 0x559e5f722730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe716d13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f71ecb0 0x559e5f722c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe716512640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f713af0 0x559e5f727410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe715d11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f714410 0x559e5f722730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39212/0 (socket says 172.21.15.5:39212) 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe715d11640 1 -- 172.21.15.5:0/3504898723 learned_addr learned my addr 172.21.15.5:0/3504898723 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe716d13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f71ecb0 0x559e5f722c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35864/0 (socket says 172.21.15.5:35864) 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe716512640 1 -- 172.21.15.5:0/3504898723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f714410 msgr2=0x559e5f722730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe716512640 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f714410 0x559e5f722730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe716512640 1 -- 172.21.15.5:0/3504898723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f71ecb0 msgr2=0x559e5f722c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe716512640 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f71ecb0 0x559e5f722c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe716512640 1 -- 172.21.15.5:0/3504898723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5f723470 con 0x559e5f713af0 2024-09-19T06:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe715d11640 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f714410 0x559e5f722730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe716d13640 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f71ecb0 0x559e5f722c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe716512640 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f713af0 0x559e5f727410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0cd440 tx=0x7fe70c0cfbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e70954638ee50e62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe6f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe6f9ffb640 1 -- 172.21.15.5:0/3504898723 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c07f060 con 0x559e5f713af0 2024-09-19T06:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3504898723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70c0b8020 con 0x559e5f713af0 2024-09-19T06:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.192+0000 7fe6f9ffb640 1 -- 172.21.15.5:0/3504898723 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fe70c0ba2d0 con 0x559e5f713af0 2024-09-19T06:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe6f9ffb640 1 -- 172.21.15.5:0/3504898723 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe70c020030 con 0x559e5f713af0 2024-09-19T06:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe71b5ed740 10 client.26122 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3504898723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559e5f79dcd0 con 0x559e5f713af0 2024-09-19T06:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe6f9ffb640 1 -- 172.21.15.5:0/3504898723 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe70c0b8020 con 0x559e5f713af0 2024-09-19T06:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe6f9ffb640 1 -- 172.21.15.5:0/3504898723 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe70c0cd0c0 con 0x559e5f713af0 2024-09-19T06:08:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe71b5ed740 10 client.26122 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe71b5ed740 10 client.26122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe71b5ed740 20 client.26122 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559e5f79e0b0 0x559e5f7a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe71b5ed740 4 client.26122 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:08:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3504898723 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e5f67fd70 con 0x559e5f79e0b0 2024-09-19T06:08:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe715d11640 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559e5f79e0b0 0x559e5f7a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe6f9ffb640 10 client.26122 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.193+0000 7fe715d11640 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559e5f79e0b0 0x559e5f7a04a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1370b43c4ec454 server_cookie=983cd846a445413a in_seq=0 out_seq=0 2024-09-19T06:08:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe6f9ffb640 1 -- 172.21.15.5:0/3504898723 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559e5f67fd70 con 0x559e5f79e0b0 2024-09-19T06:08:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe6f9ffb640 10 client.26122 handle_command_reply: tid=0 2024-09-19T06:08:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe71b5ed740 1 client.26122 shutdown 2024-09-19T06:08:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3504898723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f713af0 msgr2=0x559e5f727410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f713af0 0x559e5f727410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0cd440 tx=0x7fe70c0cfbc0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3504898723 shutdown_connections 2024-09-19T06:08:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559e5f79e0b0 0x559e5f7a04a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5f71ecb0 0x559e5f722c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559e5f714410 0x559e5f722730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe71b5ed740 1 --2- 172.21.15.5:0/3504898723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e5f713af0 0x559e5f727410 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe6f9ffb640 0 client.26122 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3504898723 >> 172.21.15.5:0/3504898723 conn(0x559e5f710060 msgr2=0x559e5f70f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3504898723 shutdown_connections 2024-09-19T06:08:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe71b5ed740 1 -- 172.21.15.5:0/3504898723 wait complete. 2024-09-19T06:08:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.213+0000 7fe71b5ed740 20 client.26122 trim_cache size 0 max 16384 2024-09-19T06:08:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:08:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:08:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:08:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.214+0000 7fe718776640 1 -- 172.21.15.5:0/1131452256 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe6fc0a33b0 msgr2=0x7fe6fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.214+0000 7fe718776640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe6fc0a33b0 0x7fe6fc0a5870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe708004af0 tx=0x7fe70803d040 comp rx=0 tx=0).stop 2024-09-19T06:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.214+0000 7fe718776640 1 -- 172.21.15.5:0/1131452256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710101ac0 msgr2=0x7fe710198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.214+0000 7fe718776640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710101ac0 0x7fe710198330 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00d770 tx=0x7fe70c00dc30 comp rx=0 tx=0).stop 2024-09-19T06:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.214+0000 7fe718776640 1 -- 172.21.15.5:0/1131452256 shutdown_connections 2024-09-19T06:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.214+0000 7fe718776640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe710107ad0 0x7fe71019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.214+0000 7fe718776640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe6fc0a33b0 0x7fe6fc0a5870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.214+0000 7fe718776640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe710102460 0x7fe710198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.214+0000 7fe718776640 1 --2- 172.21.15.5:0/1131452256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710101ac0 0x7fe710198330 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.214+0000 7fe718776640 1 -- 172.21.15.5:0/1131452256 >> 172.21.15.5:0/1131452256 conn(0x7fe7100fd7a0 msgr2=0x7fe7100fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.214+0000 7fe718776640 1 -- 172.21.15.5:0/1131452256 shutdown_connections 2024-09-19T06:08:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:08.214+0000 7fe718776640 1 -- 172.21.15.5:0/1131452256 wait complete. 2024-09-19T06:08:08.262 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:08:08.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:07 smithi157 ceph-mon[35346]: pgmap v147: 97 pgs: 97 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-09-19T06:08:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:08.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 111176 2.79 MB/sec execute 94 sec latency 21.321 ms 2024-09-19T06:08:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.095+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/2912112740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08102ef0 msgr2=0x7f7f0810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.095+0000 7f7f0e2fe640 1 --2- 172.21.15.5:0/2912112740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08102ef0 0x7f7f0810da40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009ca0 tx=0x7f7f0002a950 comp rx=0 tx=0).stop 2024-09-19T06:08:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.095+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/2912112740 shutdown_connections 2024-09-19T06:08:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.095+0000 7f7f0e2fe640 1 --2- 172.21.15.5:0/2912112740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08102ef0 0x7f7f0810da40 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.095+0000 7f7f0e2fe640 1 --2- 172.21.15.5:0/2912112740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f08102570 0x7f7f081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.095+0000 7f7f0e2fe640 1 --2- 172.21.15.5:0/2912112740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08100e20 0x7f7f08101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.095+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/2912112740 >> 172.21.15.5:0/2912112740 conn(0x7f7f080fc7d0 msgr2=0x7f7f080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.095+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/2912112740 shutdown_connections 2024-09-19T06:08:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.095+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/2912112740 wait complete. 2024-09-19T06:08:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7f0e2fe640 1 Processor -- start 2024-09-19T06:08:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7f0e2fe640 1 -- start start 2024-09-19T06:08:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7f0e2fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08100e20 0x7f7f0819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7f0e2fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08102570 0x7f7f0819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7f0e2fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f08102ef0 0x7f7f0819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7f0e2fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f08110e30 con 0x7f7f08102570 2024-09-19T06:08:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7f0e2fe640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f08110cb0 con 0x7f7f08100e20 2024-09-19T06:08:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7f0e2fe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7f08110fb0 con 0x7f7f08102ef0 2024-09-19T06:08:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7efffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08102570 0x7f7f0819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7efffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08102570 0x7f7f0819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35888/0 (socket says 172.21.15.5:35888) 2024-09-19T06:08:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7f07fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08100e20 0x7f7f0819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7efffff640 1 -- 172.21.15.5:0/1856075460 learned_addr learned my addr 172.21.15.5:0/1856075460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.096+0000 7f7f0c89b640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f08102ef0 0x7f7f0819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.097+0000 7f7efffff640 1 -- 172.21.15.5:0/1856075460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f08102ef0 msgr2=0x7f7f0819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.097+0000 7f7efffff640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f08102ef0 0x7f7f0819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.097+0000 7f7efffff640 1 -- 172.21.15.5:0/1856075460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08100e20 msgr2=0x7f7f0819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.097+0000 7f7efffff640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08100e20 0x7f7f0819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.097+0000 7f7efffff640 1 -- 172.21.15.5:0/1856075460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0819f570 con 0x7f7f08102570 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.097+0000 7f7efffff640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08102570 0x7f7f0819ab00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000d950 tx=0x7f7ef000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.097+0000 7f7f05ffb640 1 -- 172.21.15.5:0/1856075460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0017070 con 0x7f7f08102570 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.097+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/1856075460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0819f860 con 0x7f7f08102570 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.097+0000 7f7f05ffb640 1 -- 172.21.15.5:0/1856075460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ef0009c30 con 0x7f7f08102570 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.097+0000 7f7f05ffb640 1 -- 172.21.15.5:0/1856075460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000f620 con 0x7f7f08102570 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.097+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/1856075460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f081a7140 con 0x7f7f08102570 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f05ffb640 1 -- 172.21.15.5:0/1856075460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7ef00109a0 con 0x7f7f08102570 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f106d7740 1 Processor -- start 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f106d7740 1 -- start start 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f106d7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 0x5585e795f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f106d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e795f5d0 0x5585e795fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f106d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e796c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f106d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585e78c52f0 con 0x5585e795f5d0 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f106d7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5585e786ccf0 con 0x5585e7960b40 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f106d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5585e78c12b0 con 0x5585e796a370 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f0c89b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 0x5585e795f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e795f5d0 0x5585e795fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f0c89b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 0x5585e795f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39226/0 (socket says 172.21.15.5:39226) 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e795f5d0 0x5585e795fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35894/0 (socket says 172.21.15.5:35894) 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.100+0000 7f7f0c89b640 1 -- 172.21.15.5:0/3730641142 learned_addr learned my addr 172.21.15.5:0/3730641142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.101+0000 7f7f05ffb640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7ef40a33b0 0x7f7ef40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.101+0000 7f7efffff640 1 --2- 172.21.15.5:0/3730641142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e796c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.101+0000 7f7f0c89b640 1 -- 172.21.15.5:0/3730641142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 msgr2=0x5585e796c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.101+0000 7f7f0c89b640 1 --2- 172.21.15.5:0/3730641142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e796c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.101+0000 7f7f0c89b640 1 -- 172.21.15.5:0/3730641142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e795f5d0 msgr2=0x5585e795fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.101+0000 7f7f0c89b640 1 --2- 172.21.15.5:0/3730641142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e795f5d0 0x5585e795fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.101+0000 7f7f0c89b640 1 -- 172.21.15.5:0/3730641142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585e78513a0 con 0x5585e7960b40 2024-09-19T06:08:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.101+0000 7f7f05ffb640 1 -- 172.21.15.5:0/1856075460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f7ef00cf840 con 0x7f7f08102570 2024-09-19T06:08:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.101+0000 7f7f0c89b640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7ef40a33b0 0x7f7ef40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.101+0000 7f7efffff640 1 --2- 172.21.15.5:0/3730641142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e796c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:08:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.101+0000 7f7f0c89b640 1 --2- 172.21.15.5:0/3730641142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 0x5585e795f090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7f00035000 tx=0x7f7f00005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcc81af4f7a6ff3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.102+0000 7f7efe7fc640 1 -- 172.21.15.5:0/3730641142 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00034040 con 0x5585e7960b40 2024-09-19T06:08:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.102+0000 7f7efe7fc640 1 -- 172.21.15.5:0/3730641142 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f00003b70 con 0x5585e7960b40 2024-09-19T06:08:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.102+0000 7f7efe7fc640 1 -- 172.21.15.5:0/3730641142 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0004e6d0 con 0x5585e7960b40 2024-09-19T06:08:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.102+0000 7f7f106d7740 1 -- 172.21.15.5:0/3730641142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 msgr2=0x5585e795f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.102+0000 7f7f106d7740 1 --2- 172.21.15.5:0/3730641142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 0x5585e795f090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7f00035000 tx=0x7f7f00005fa0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.102+0000 7f7f0c89b640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7ef40a33b0 0x7f7ef40a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7f00055000 tx=0x7f7f00052000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.103+0000 7f7f106d7740 1 -- 172.21.15.5:0/3730641142 shutdown_connections 2024-09-19T06:08:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.103+0000 7f7f106d7740 1 --2- 172.21.15.5:0/3730641142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e796c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.103+0000 7f7f106d7740 1 --2- 172.21.15.5:0/3730641142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e795f5d0 0x5585e795fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.103+0000 7f7f106d7740 1 --2- 172.21.15.5:0/3730641142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 0x5585e795f090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.103+0000 7f7f106d7740 1 -- 172.21.15.5:0/3730641142 >> 172.21.15.5:0/3730641142 conn(0x5585e7928310 msgr2=0x5585e795bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.103+0000 7f7f106d7740 1 -- 172.21.15.5:0/3730641142 shutdown_connections 2024-09-19T06:08:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.103+0000 7f7f106d7740 1 -- 172.21.15.5:0/3730641142 wait complete. 2024-09-19T06:08:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.103+0000 7f7f106d7740 1 -- start start 2024-09-19T06:08:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.103+0000 7f7f106d7740 1 Processor -- start 2024-09-19T06:08:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7f106d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e795f5d0 0x5585e7969c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7f106d7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 0x5585e7966f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7f106d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e796a370 0x5585e7967480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7f106d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585e796d110 con 0x5585e796a370 2024-09-19T06:08:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7f106d7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5585e7880060 con 0x5585e7960b40 2024-09-19T06:08:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7f106d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5585e796d290 con 0x5585e795f5d0 2024-09-19T06:08:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7f106d7740 10 client.0 _ll_register_callbacks cb 0x5585e75d38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7efffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 0x5585e7966f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7f07fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e795f5d0 0x5585e7969c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7efffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 0x5585e7966f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39236/0 (socket says 172.21.15.5:39236) 2024-09-19T06:08:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7efffff640 1 -- 172.21.15.5:0/2255102927 learned_addr learned my addr 172.21.15.5:0/2255102927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7f0c89b640 1 --2- 172.21.15.5:0/2255102927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e796a370 0x5585e7967480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7efffff640 1 -- 172.21.15.5:0/2255102927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e795f5d0 msgr2=0x5585e7969c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7efffff640 1 --2- 172.21.15.5:0/2255102927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e795f5d0 0x5585e7969c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7efffff640 1 -- 172.21.15.5:0/2255102927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e796a370 msgr2=0x5585e7967480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7efffff640 1 --2- 172.21.15.5:0/2255102927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e796a370 0x5585e7967480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7efffff640 1 -- 172.21.15.5:0/2255102927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585e76363c0 con 0x5585e7960b40 2024-09-19T06:08:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7efffff640 1 --2- 172.21.15.5:0/2255102927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 0x5585e7966f40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00cd5a0 tx=0x7f7ef006a8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e21cc77e166ade7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.104+0000 7f7ee77fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:08:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7ee77fe640 1 -- 172.21.15.5:0/2255102927 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0020070 con 0x5585e7960b40 2024-09-19T06:08:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7ee77fe640 1 -- 172.21.15.5:0/2255102927 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f7ef0098250 con 0x5585e7960b40 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7f106d7740 1 -- 172.21.15.5:0/2255102927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f00003b70 con 0x5585e7960b40 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7ee77fe640 1 -- 172.21.15.5:0/2255102927 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7ef009ce60 con 0x5585e7960b40 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7f106d7740 10 client.26201 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7f106d7740 1 -- 172.21.15.5:0/2255102927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5585e7978ae0 con 0x5585e7960b40 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7ee77fe640 1 -- 172.21.15.5:0/2255102927 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f00003b70 con 0x5585e7960b40 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7ee77fe640 1 -- 172.21.15.5:0/2255102927 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7ef00860c0 con 0x5585e7960b40 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7f106d7740 10 client.26201 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7f106d7740 10 client.26201 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7f106d7740 20 client.26201 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7f106d7740 1 --2- 172.21.15.5:0/2255102927 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5585e797a3c0 0x5585e797c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7f106d7740 4 client.26201 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7f106d7740 1 -- 172.21.15.5:0/2255102927 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585e78d2860 con 0x5585e797a3c0 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.105+0000 7f7f07fff640 1 --2- 172.21.15.5:0/2255102927 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5585e797a3c0 0x5585e797c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.106+0000 7f7ee77fe640 10 client.26201 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.106+0000 7f7f07fff640 1 --2- 172.21.15.5:0/2255102927 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5585e797a3c0 0x5585e797c800 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d444595c557f9476 server_cookie=7a2516b9c95b854d in_seq=0 out_seq=0 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7ee77fe640 1 -- 172.21.15.5:0/2255102927 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585e78d2860 con 0x5585e797a3c0 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7ee77fe640 10 client.26201 handle_command_reply: tid=0 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7f106d7740 1 client.26201 shutdown 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7f106d7740 1 -- 172.21.15.5:0/2255102927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 msgr2=0x5585e7966f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7f106d7740 1 --2- 172.21.15.5:0/2255102927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 0x5585e7966f40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00cd5a0 tx=0x7f7ef006a8d0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7f106d7740 1 -- 172.21.15.5:0/2255102927 shutdown_connections 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7f106d7740 1 --2- 172.21.15.5:0/2255102927 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5585e797a3c0 0x5585e797c800 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7f106d7740 1 --2- 172.21.15.5:0/2255102927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e796a370 0x5585e7967480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7f106d7740 1 --2- 172.21.15.5:0/2255102927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e7960b40 0x5585e7966f40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7f106d7740 1 --2- 172.21.15.5:0/2255102927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e795f5d0 0x5585e7969c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7f106d7740 1 -- 172.21.15.5:0/2255102927 >> 172.21.15.5:0/2255102927 conn(0x5585e7928310 msgr2=0x5585e7961930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7ee77fe640 0 client.26201 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:08:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7f106d7740 1 -- 172.21.15.5:0/2255102927 shutdown_connections 2024-09-19T06:08:09.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7f106d7740 1 -- 172.21.15.5:0/2255102927 wait complete. 2024-09-19T06:08:09.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.107+0000 7f7f106d7740 20 client.26201 trim_cache size 0 max 16384 2024-09-19T06:08:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.121+0000 7f7f106d7740 1 Processor -- start 2024-09-19T06:08:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7f106d7740 1 -- start start 2024-09-19T06:08:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7f106d7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e795f5d0 0x5585e7976400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7f106d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 0x5585e7968eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7f106d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e7982110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7f106d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f081ab5c0 con 0x5585e7960b40 2024-09-19T06:08:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7f106d7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f081ad100 con 0x5585e795f5d0 2024-09-19T06:08:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7f106d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7f081abc90 con 0x5585e796a370 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7efffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 0x5585e7968eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7efffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 0x5585e7968eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35906/0 (socket says 172.21.15.5:35906) 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7f07fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e795f5d0 0x5585e7976400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7efffff640 1 -- 172.21.15.5:0/2539923997 learned_addr learned my addr 172.21.15.5:0/2539923997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7f0c89b640 1 --2- 172.21.15.5:0/2539923997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e7982110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7efffff640 1 -- 172.21.15.5:0/2539923997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 msgr2=0x5585e7982110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7efffff640 1 --2- 172.21.15.5:0/2539923997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e7982110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7efffff640 1 -- 172.21.15.5:0/2539923997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e795f5d0 msgr2=0x5585e7976400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7efffff640 1 --2- 172.21.15.5:0/2539923997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e795f5d0 0x5585e7976400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7efffff640 1 -- 172.21.15.5:0/2539923997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585e797ce80 con 0x5585e7960b40 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7f0c89b640 1 --2- 172.21.15.5:0/2539923997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e7982110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7f07fff640 1 --2- 172.21.15.5:0/2539923997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e795f5d0 0x5585e7976400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.122+0000 7f7efffff640 1 --2- 172.21.15.5:0/2539923997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 0x5585e7968eb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7ef006d040 tx=0x7f7ef009a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dc8e7e96aeb7e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:09.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.123+0000 7f7efdffb640 1 -- 172.21.15.5:0/2539923997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0020070 con 0x5585e7960b40 2024-09-19T06:08:09.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.123+0000 7f7efdffb640 1 -- 172.21.15.5:0/2539923997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ef00a47a0 con 0x5585e7960b40 2024-09-19T06:08:09.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.123+0000 7f7efdffb640 1 -- 172.21.15.5:0/2539923997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00a4310 con 0x5585e7960b40 2024-09-19T06:08:09.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.123+0000 7f7f106d7740 1 -- 172.21.15.5:0/2539923997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 msgr2=0x5585e7968eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:09.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.123+0000 7f7f106d7740 1 --2- 172.21.15.5:0/2539923997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 0x5585e7968eb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7ef006d040 tx=0x7f7ef009a0f0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.123+0000 7f7f106d7740 1 -- 172.21.15.5:0/2539923997 shutdown_connections 2024-09-19T06:08:09.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.123+0000 7f7f106d7740 1 --2- 172.21.15.5:0/2539923997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e7982110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.123+0000 7f7f106d7740 1 --2- 172.21.15.5:0/2539923997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 0x5585e7968eb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.123+0000 7f7f106d7740 1 --2- 172.21.15.5:0/2539923997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e795f5d0 0x5585e7976400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.124+0000 7f7f106d7740 1 -- 172.21.15.5:0/2539923997 >> 172.21.15.5:0/2539923997 conn(0x5585e7928310 msgr2=0x5585e79687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.124+0000 7f7f106d7740 1 -- 172.21.15.5:0/2539923997 shutdown_connections 2024-09-19T06:08:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.124+0000 7f7f106d7740 1 -- 172.21.15.5:0/2539923997 wait complete. 2024-09-19T06:08:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.124+0000 7f7f106d7740 1 -- start start 2024-09-19T06:08:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.124+0000 7f7f106d7740 1 Processor -- start 2024-09-19T06:08:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.124+0000 7f7f106d7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e795f5d0 0x5585e7972c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.124+0000 7f7f106d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 0x5585e796df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.124+0000 7f7f106d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e796e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7f106d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f081aca30 con 0x5585e7960b40 2024-09-19T06:08:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7f106d7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f081ac360 con 0x5585e795f5d0 2024-09-19T06:08:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7f106d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7f081a95e0 con 0x5585e796a370 2024-09-19T06:08:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7efffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 0x5585e796df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7efffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 0x5585e796df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35910/0 (socket says 172.21.15.5:35910) 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7efffff640 1 -- 172.21.15.5:0/3849199221 learned_addr learned my addr 172.21.15.5:0/3849199221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7f106d7740 10 client.0 _ll_register_callbacks cb 0x7f7ef000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7f07fff640 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e795f5d0 0x5585e7972c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7f0c89b640 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e796e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7efffff640 1 -- 172.21.15.5:0/3849199221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 msgr2=0x5585e796e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7efffff640 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e796e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7efffff640 1 -- 172.21.15.5:0/3849199221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e795f5d0 msgr2=0x5585e7972c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7efffff640 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e795f5d0 0x5585e7972c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7efffff640 1 -- 172.21.15.5:0/3849199221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585e796ecb0 con 0x5585e7960b40 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7efffff640 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 0x5585e796df40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00cd5a0 tx=0x7f7ef0010690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6baa20e5306b18 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7ee77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7ee77fe640 1 -- 172.21.15.5:0/3849199221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0020070 con 0x5585e7960b40 2024-09-19T06:08:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7ee77fe640 1 -- 172.21.15.5:0/3849199221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f7ef00a4310 con 0x5585e7960b40 2024-09-19T06:08:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.125+0000 7f7f106d7740 1 -- 172.21.15.5:0/3849199221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ef00a47a0 con 0x5585e7960b40 2024-09-19T06:08:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.126+0000 7f7ee77fe640 1 -- 172.21.15.5:0/3849199221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7ef009fee0 con 0x5585e7960b40 2024-09-19T06:08:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.126+0000 7f7f106d7740 10 client.16626 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.126+0000 7f7f106d7740 1 -- 172.21.15.5:0/3849199221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5585e79e5050 con 0x5585e7960b40 2024-09-19T06:08:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.126+0000 7f7ee77fe640 1 -- 172.21.15.5:0/3849199221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ef0098250 con 0x5585e7960b40 2024-09-19T06:08:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.126+0000 7f7ee77fe640 1 -- 172.21.15.5:0/3849199221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7ef00a1070 con 0x5585e7960b40 2024-09-19T06:08:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.126+0000 7f7f106d7740 10 client.16626 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.126+0000 7f7f106d7740 10 client.16626 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:08:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.126+0000 7f7f106d7740 20 client.16626 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.126+0000 7f7f106d7740 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5585e79e5520 0x5585e79e7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.126+0000 7f7f106d7740 4 client.16626 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:08:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.126+0000 7f7f106d7740 1 -- 172.21.15.5:0/3849199221 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7ef00a47a0 con 0x5585e79e5520 2024-09-19T06:08:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.126+0000 7f7f07fff640 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5585e79e5520 0x5585e79e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7f07fff640 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5585e79e5520 0x5585e79e7910 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ef98e81c8133d4 server_cookie=373df09baa081d0f in_seq=0 out_seq=0 2024-09-19T06:08:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7ee77fe640 10 client.16626 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:08:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7ee77fe640 1 -- 172.21.15.5:0/3849199221 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7ef00a47a0 con 0x5585e79e5520 2024-09-19T06:08:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7ee77fe640 10 client.16626 handle_command_reply: tid=0 2024-09-19T06:08:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7f106d7740 1 client.16626 shutdown 2024-09-19T06:08:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7f106d7740 1 -- 172.21.15.5:0/3849199221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 msgr2=0x5585e796df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7f106d7740 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 0x5585e796df40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00cd5a0 tx=0x7f7ef0010690 comp rx=0 tx=0).stop 2024-09-19T06:08:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7f106d7740 1 -- 172.21.15.5:0/3849199221 shutdown_connections 2024-09-19T06:08:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7f106d7740 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5585e79e5520 0x5585e79e7910 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7f106d7740 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585e796a370 0x5585e796e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7f106d7740 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585e7960b40 0x5585e796df40 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7ee77fe640 0 client.16626 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:08:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7f106d7740 1 --2- 172.21.15.5:0/3849199221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585e795f5d0 0x5585e7972c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7f106d7740 1 -- 172.21.15.5:0/3849199221 >> 172.21.15.5:0/3849199221 conn(0x5585e7928310 msgr2=0x5585e795ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.127+0000 7f7f106d7740 1 -- 172.21.15.5:0/3849199221 shutdown_connections 2024-09-19T06:08:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.128+0000 7f7f106d7740 1 -- 172.21.15.5:0/3849199221 wait complete. 2024-09-19T06:08:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.128+0000 7f7f106d7740 20 client.16626 trim_cache size 0 max 16384 2024-09-19T06:08:09.135 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:08:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.128+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/1856075460 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7ef40a33b0 msgr2=0x7f7ef40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.128+0000 7f7f0e2fe640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7ef40a33b0 0x7f7ef40a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7f00055000 tx=0x7f7f00052000 comp rx=0 tx=0).stop 2024-09-19T06:08:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.128+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/1856075460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08102570 msgr2=0x7f7f0819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.128+0000 7f7f0e2fe640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08102570 0x7f7f0819ab00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000d950 tx=0x7f7ef000de10 comp rx=0 tx=0).stop 2024-09-19T06:08:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.129+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/1856075460 shutdown_connections 2024-09-19T06:08:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.129+0000 7f7f0e2fe640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7ef40a33b0 0x7f7ef40a5870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.129+0000 7f7f0e2fe640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f08102ef0 0x7f7f0819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.129+0000 7f7f0e2fe640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08102570 0x7f7f0819ab00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.129+0000 7f7f0e2fe640 1 --2- 172.21.15.5:0/1856075460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08100e20 0x7f7f0819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.129+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/1856075460 >> 172.21.15.5:0/1856075460 conn(0x7f7f080fc7d0 msgr2=0x7f7f0806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.129+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/1856075460 shutdown_connections 2024-09-19T06:08:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:09.129+0000 7f7f0e2fe640 1 -- 172.21.15.5:0/1856075460 wait complete. 2024-09-19T06:08:09.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 111704 2.80 MB/sec execute 95 sec latency 18.881 ms 2024-09-19T06:08:09.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:08:10.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:09 smithi005 ceph-mon[30955]: pgmap v148: 97 pgs: 97 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-19T06:08:10.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:09 smithi049 ceph-mon[34640]: pgmap v148: 97 pgs: 97 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-19T06:08:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:10.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:09 smithi157 ceph-mon[35346]: pgmap v148: 97 pgs: 97 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-19T06:08:10.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.752+0000 7f1af96c0640 1 -- 172.21.15.5:0/429939209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af4102ef0 msgr2=0x7f1af410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:10.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.752+0000 7f1af96c0640 1 --2- 172.21.15.5:0/429939209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af4102ef0 0x7f1af410da40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009ca0 tx=0x7f1aec02a950 comp rx=0 tx=0).stop 2024-09-19T06:08:10.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.752+0000 7f1af96c0640 1 -- 172.21.15.5:0/429939209 shutdown_connections 2024-09-19T06:08:10.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.752+0000 7f1af96c0640 1 --2- 172.21.15.5:0/429939209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af4102ef0 0x7f1af410da40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.752+0000 7f1af96c0640 1 --2- 172.21.15.5:0/429939209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1af4102570 0x7f1af41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.752+0000 7f1af96c0640 1 --2- 172.21.15.5:0/429939209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af4100e20 0x7f1af4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.752+0000 7f1af96c0640 1 -- 172.21.15.5:0/429939209 >> 172.21.15.5:0/429939209 conn(0x7f1af40fc7d0 msgr2=0x7f1af40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:10.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.752+0000 7f1af96c0640 1 -- 172.21.15.5:0/429939209 shutdown_connections 2024-09-19T06:08:10.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.752+0000 7f1af96c0640 1 -- 172.21.15.5:0/429939209 wait complete. 2024-09-19T06:08:10.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.753+0000 7f1af96c0640 1 Processor -- start 2024-09-19T06:08:10.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.753+0000 7f1af96c0640 1 -- start start 2024-09-19T06:08:10.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af96c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af4100e20 0x7f1af419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af96c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1af4102570 0x7f1af419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af96c0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af4102ef0 0x7f1af419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af96c0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4110e30 con 0x7f1af4100e20 2024-09-19T06:08:10.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af96c0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4110cb0 con 0x7f1af4102ef0 2024-09-19T06:08:10.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af96c0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4110fb0 con 0x7f1af4102570 2024-09-19T06:08:10.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af4100e20 0x7f1af419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af4100e20 0x7f1af419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35924/0 (socket says 172.21.15.5:35924) 2024-09-19T06:08:10.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af2ffd640 1 -- 172.21.15.5:0/1686026808 learned_addr learned my addr 172.21.15.5:0/1686026808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:10.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af37fe640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af4102ef0 0x7f1af419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af27fc640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1af4102570 0x7f1af419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af2ffd640 1 -- 172.21.15.5:0/1686026808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1af4102570 msgr2=0x7f1af419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:10.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af2ffd640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1af4102570 0x7f1af419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af2ffd640 1 -- 172.21.15.5:0/1686026808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af4102ef0 msgr2=0x7f1af419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af2ffd640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af4102ef0 0x7f1af419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af2ffd640 1 -- 172.21.15.5:0/1686026808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af419f550 con 0x7f1af4100e20 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af2ffd640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af4100e20 0x7f1af419a5a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400e9f0 tx=0x7f1ae400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1adbfff640 1 -- 172.21.15.5:0/1686026808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400cd80 con 0x7f1af4100e20 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1adbfff640 1 -- 172.21.15.5:0/1686026808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae40108e0 con 0x7f1af4100e20 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.754+0000 7f1af96c0640 1 -- 172.21.15.5:0/1686026808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af419f840 con 0x7f1af4100e20 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.755+0000 7f1adbfff640 1 -- 172.21.15.5:0/1686026808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4018920 con 0x7f1af4100e20 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.755+0000 7f1af96c0640 1 -- 172.21.15.5:0/1686026808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af41a7120 con 0x7f1af4100e20 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.757+0000 7f1adbfff640 1 -- 172.21.15.5:0/1686026808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1ae4010a90 con 0x7f1af4100e20 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.757+0000 7f1afba99740 1 Processor -- start 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.757+0000 7f1afba99740 1 -- start start 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.757+0000 7f1afba99740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.757+0000 7f1afba99740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3e0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.757+0000 7f1afba99740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.757+0000 7f1afba99740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cfca3462f0 con 0x55cfca3e05d0 2024-09-19T06:08:10.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.757+0000 7f1afba99740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cfca2edcf0 con 0x55cfca3e1b40 2024-09-19T06:08:10.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.757+0000 7f1afba99740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cfca3422b0 con 0x55cfca3eb370 2024-09-19T06:08:10.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39300/0 (socket says 172.21.15.5:39300) 2024-09-19T06:08:10.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af37fe640 1 -- 172.21.15.5:0/4242707390 learned_addr learned my addr 172.21.15.5:0/4242707390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:10.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af2ffd640 1 --2- 172.21.15.5:0/4242707390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3e0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af37fe640 1 -- 172.21.15.5:0/4242707390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 msgr2=0x55cfca3ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:10.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af37fe640 1 --2- 172.21.15.5:0/4242707390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3ed760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af37fe640 1 -- 172.21.15.5:0/4242707390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 msgr2=0x55cfca3e0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af37fe640 1 --2- 172.21.15.5:0/4242707390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3e0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1adbfff640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1adc0a33b0 0x7f1adc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af37fe640 1 -- 172.21.15.5:0/4242707390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfca2d23a0 con 0x55cfca3e1b40 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af2ffd640 1 --2- 172.21.15.5:0/4242707390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af27fc640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1adc0a33b0 0x7f1adc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1adbfff640 1 -- 172.21.15.5:0/1686026808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f1ae40cffe0 con 0x7f1af4100e20 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.758+0000 7f1af37fe640 1 --2- 172.21.15.5:0/4242707390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3e0090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1aec035000 tx=0x7f1aec0093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f214c71c9bf3302 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.759+0000 7f1ad97fa640 1 -- 172.21.15.5:0/4242707390 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec048070 con 0x55cfca3e1b40 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.759+0000 7f1af27fc640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1adc0a33b0 0x7f1adc0a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55cfca3e1070 tx=0x7f1ae8005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.759+0000 7f1ad97fa640 1 -- 172.21.15.5:0/4242707390 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1aec02f8c0 con 0x55cfca3e1b40 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.759+0000 7f1ad97fa640 1 -- 172.21.15.5:0/4242707390 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec034040 con 0x55cfca3e1b40 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.759+0000 7f1afba99740 1 -- 172.21.15.5:0/4242707390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 msgr2=0x55cfca3e0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:10.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.759+0000 7f1afba99740 1 --2- 172.21.15.5:0/4242707390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3e0090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1aec035000 tx=0x7f1aec0093e0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.759+0000 7f1afba99740 1 -- 172.21.15.5:0/4242707390 shutdown_connections 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.759+0000 7f1afba99740 1 --2- 172.21.15.5:0/4242707390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.759+0000 7f1afba99740 1 --2- 172.21.15.5:0/4242707390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.759+0000 7f1afba99740 1 --2- 172.21.15.5:0/4242707390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3e0090 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.759+0000 7f1afba99740 1 -- 172.21.15.5:0/4242707390 >> 172.21.15.5:0/4242707390 conn(0x55cfca3a9310 msgr2=0x55cfca3dcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.760+0000 7f1afba99740 1 -- 172.21.15.5:0/4242707390 shutdown_connections 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.760+0000 7f1afba99740 1 -- 172.21.15.5:0/4242707390 wait complete. 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.760+0000 7f1afba99740 1 -- start start 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.760+0000 7f1afba99740 1 Processor -- start 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.760+0000 7f1afba99740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3eada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.760+0000 7f1afba99740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3e8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.760+0000 7f1afba99740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3e87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.760+0000 7f1afba99740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cfca3ee010 con 0x55cfca3e05d0 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.760+0000 7f1afba99740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cfca301060 con 0x55cfca3e1b40 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.760+0000 7f1afba99740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cfca3ee190 con 0x55cfca3eb370 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.760+0000 7f1afba99740 10 client.0 _ll_register_callbacks cb 0x55cfca0548c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1af2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1af2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3eada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35940/0 (socket says 172.21.15.5:35940) 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1af2ffd640 1 -- 172.21.15.5:0/238287452 learned_addr learned my addr 172.21.15.5:0/238287452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1af27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1af37fe640 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3e87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1af27fc640 1 -- 172.21.15.5:0/238287452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 msgr2=0x55cfca3e87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1af27fc640 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3e87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1af27fc640 1 -- 172.21.15.5:0/238287452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 msgr2=0x55cfca3eada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1af27fc640 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3eada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1af27fc640 1 -- 172.21.15.5:0/238287452 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfca0b73c0 con 0x55cfca3e1b40 2024-09-19T06:08:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1af2ffd640 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3eada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1af27fc640 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3e8050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8013010 tx=0x7f1ae8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d657800abbdf591 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1ad6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:08:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1ad6ff5640 1 -- 172.21.15.5:0/238287452 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8009e60 con 0x55cfca3e1b40 2024-09-19T06:08:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1ad6ff5640 1 -- 172.21.15.5:0/238287452 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f1ae8005020 con 0x55cfca3e1b40 2024-09-19T06:08:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.761+0000 7f1afba99740 1 -- 172.21.15.5:0/238287452 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aec02f8c0 con 0x55cfca3e1b40 2024-09-19T06:08:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.762+0000 7f1ad6ff5640 1 -- 172.21.15.5:0/238287452 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1ae8005220 con 0x55cfca3e1b40 2024-09-19T06:08:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.762+0000 7f1afba99740 10 client.26219 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.762+0000 7f1afba99740 1 -- 172.21.15.5:0/238287452 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cfca3f9ba0 con 0x55cfca3e1b40 2024-09-19T06:08:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.762+0000 7f1ad6ff5640 1 -- 172.21.15.5:0/238287452 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1aec02f8c0 con 0x55cfca3e1b40 2024-09-19T06:08:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.762+0000 7f1ad6ff5640 1 -- 172.21.15.5:0/238287452 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f1ae8027070 con 0x55cfca3e1b40 2024-09-19T06:08:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.762+0000 7f1afba99740 10 client.26219 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.762+0000 7f1afba99740 10 client.26219 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:08:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.762+0000 7f1afba99740 20 client.26219 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.762+0000 7f1afba99740 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55cfca3fb060 0x55cfca3fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.762+0000 7f1afba99740 4 client.26219 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.762+0000 7f1afba99740 1 -- 172.21.15.5:0/238287452 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfca353860 con 0x55cfca3fb060 2024-09-19T06:08:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.762+0000 7f1af2ffd640 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55cfca3fb060 0x55cfca3fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.763+0000 7f1af2ffd640 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55cfca3fb060 0x55cfca3fd4b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c64fc77e263c80f server_cookie=e15bde93277748ca in_seq=0 out_seq=0 2024-09-19T06:08:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.763+0000 7f1ad6ff5640 10 client.26219 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:08:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.763+0000 7f1ad6ff5640 1 -- 172.21.15.5:0/238287452 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cfca353860 con 0x55cfca3fb060 2024-09-19T06:08:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.763+0000 7f1ad6ff5640 10 client.26219 handle_command_reply: tid=0 2024-09-19T06:08:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.763+0000 7f1afba99740 1 client.26219 shutdown 2024-09-19T06:08:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.763+0000 7f1afba99740 1 -- 172.21.15.5:0/238287452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 msgr2=0x55cfca3e8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.763+0000 7f1afba99740 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3e8050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8013010 tx=0x7f1ae8004730 comp rx=0 tx=0).stop 2024-09-19T06:08:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.764+0000 7f1afba99740 1 -- 172.21.15.5:0/238287452 shutdown_connections 2024-09-19T06:08:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.764+0000 7f1afba99740 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55cfca3fb060 0x55cfca3fd4b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.764+0000 7f1afba99740 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3e87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.764+0000 7f1afba99740 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3e8050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.764+0000 7f1afba99740 1 --2- 172.21.15.5:0/238287452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3eada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.764+0000 7f1afba99740 1 -- 172.21.15.5:0/238287452 >> 172.21.15.5:0/238287452 conn(0x55cfca3a9310 msgr2=0x55cfca3e2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.764+0000 7f1ad6ff5640 0 client.26219 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:08:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.764+0000 7f1afba99740 1 -- 172.21.15.5:0/238287452 shutdown_connections 2024-09-19T06:08:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.764+0000 7f1afba99740 1 -- 172.21.15.5:0/238287452 wait complete. 2024-09-19T06:08:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.764+0000 7f1afba99740 20 client.26219 trim_cache size 0 max 16384 2024-09-19T06:08:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.777+0000 7f1afba99740 1 Processor -- start 2024-09-19T06:08:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.777+0000 7f1afba99740 1 -- start start 2024-09-19T06:08:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.777+0000 7f1afba99740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3eada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.777+0000 7f1afba99740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3dc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.777+0000 7f1afba99740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3dcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.777+0000 7f1afba99740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1af41af890 con 0x55cfca3e05d0 2024-09-19T06:08:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.777+0000 7f1afba99740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1af41b13d0 con 0x55cfca3e1b40 2024-09-19T06:08:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.777+0000 7f1afba99740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1af41aff60 con 0x55cfca3eb370 2024-09-19T06:08:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1af27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1af2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1af27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3dc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39320/0 (socket says 172.21.15.5:39320) 2024-09-19T06:08:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1af27fc640 1 -- 172.21.15.5:0/880171054 learned_addr learned my addr 172.21.15.5:0/880171054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1af37fe640 1 --2- 172.21.15.5:0/880171054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3dcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1af27fc640 1 -- 172.21.15.5:0/880171054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 msgr2=0x55cfca3dcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1af27fc640 1 --2- 172.21.15.5:0/880171054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3dcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1af27fc640 1 -- 172.21.15.5:0/880171054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 msgr2=0x55cfca3eada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1af27fc640 1 --2- 172.21.15.5:0/880171054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3eada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1af27fc640 1 -- 172.21.15.5:0/880171054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfca3dd180 con 0x55cfca3e1b40 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1af37fe640 1 --2- 172.21.15.5:0/880171054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3dcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1af27fc640 1 --2- 172.21.15.5:0/880171054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3dc700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009710 tx=0x7f1ae801ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74f4f17fa663cd91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1ad8ff9640 1 -- 172.21.15.5:0/880171054 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8009e60 con 0x55cfca3e1b40 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1ad8ff9640 1 -- 172.21.15.5:0/880171054 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae801e680 con 0x55cfca3e1b40 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.778+0000 7f1ad8ff9640 1 -- 172.21.15.5:0/880171054 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8004eb0 con 0x55cfca3e1b40 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.779+0000 7f1afba99740 1 -- 172.21.15.5:0/880171054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 msgr2=0x55cfca3dc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.779+0000 7f1afba99740 1 --2- 172.21.15.5:0/880171054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3dc700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009710 tx=0x7f1ae801ef30 comp rx=0 tx=0).stop 2024-09-19T06:08:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.779+0000 7f1afba99740 1 -- 172.21.15.5:0/880171054 shutdown_connections 2024-09-19T06:08:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.779+0000 7f1afba99740 1 --2- 172.21.15.5:0/880171054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca3dcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.779+0000 7f1afba99740 1 --2- 172.21.15.5:0/880171054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3dc700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.779+0000 7f1afba99740 1 --2- 172.21.15.5:0/880171054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3eada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.779+0000 7f1afba99740 1 -- 172.21.15.5:0/880171054 >> 172.21.15.5:0/880171054 conn(0x55cfca3a9310 msgr2=0x55cfca3d7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.779+0000 7f1afba99740 1 -- 172.21.15.5:0/880171054 shutdown_connections 2024-09-19T06:08:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.779+0000 7f1afba99740 1 -- 172.21.15.5:0/880171054 wait complete. 2024-09-19T06:08:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1afba99740 1 -- start start 2024-09-19T06:08:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1afba99740 1 Processor -- start 2024-09-19T06:08:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1afba99740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1afba99740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3ee9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1afba99740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca463d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1afba99740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1af41b0d00 con 0x55cfca3e05d0 2024-09-19T06:08:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1afba99740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1af41b0630 con 0x55cfca3e1b40 2024-09-19T06:08:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1afba99740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1af41ad870 con 0x55cfca3eb370 2024-09-19T06:08:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1afba99740 10 client.0 _ll_register_callbacks cb 0x7f1ae8009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1af2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1af2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3ee4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35958/0 (socket says 172.21.15.5:35958) 2024-09-19T06:08:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1af2ffd640 1 -- 172.21.15.5:0/2766729392 learned_addr learned my addr 172.21.15.5:0/2766729392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1af27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3ee9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1af37fe640 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca463d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1af2ffd640 1 -- 172.21.15.5:0/2766729392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 msgr2=0x55cfca463d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1af2ffd640 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca463d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1af2ffd640 1 -- 172.21.15.5:0/2766729392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 msgr2=0x55cfca3ee9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1af2ffd640 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3ee9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1af2ffd640 1 -- 172.21.15.5:0/2766729392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfca3ef270 con 0x55cfca3e05d0 2024-09-19T06:08:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.780+0000 7f1af27fc640 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3ee9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.781+0000 7f1af2ffd640 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3ee4a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1ae409c310 tx=0x7f1ae4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b59dd626444ca46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.781+0000 7f1ad6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.781+0000 7f1ad6ff5640 1 -- 172.21.15.5:0/2766729392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40a0680 con 0x55cfca3e05d0 2024-09-19T06:08:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.781+0000 7f1ad6ff5640 1 -- 172.21.15.5:0/2766729392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f1ae40a4ab0 con 0x55cfca3e05d0 2024-09-19T06:08:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.781+0000 7f1afba99740 1 -- 172.21.15.5:0/2766729392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ae801e680 con 0x55cfca3e05d0 2024-09-19T06:08:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.781+0000 7f1ad6ff5640 1 -- 172.21.15.5:0/2766729392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1ae40a4f40 con 0x55cfca3e05d0 2024-09-19T06:08:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.781+0000 7f1afba99740 10 client.16644 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:10.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.781+0000 7f1afba99740 1 -- 172.21.15.5:0/2766729392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cfca4653f0 con 0x55cfca3e05d0 2024-09-19T06:08:10.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.781+0000 7f1ad6ff5640 1 -- 172.21.15.5:0/2766729392 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae801e680 con 0x55cfca3e05d0 2024-09-19T06:08:10.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.781+0000 7f1ad6ff5640 1 -- 172.21.15.5:0/2766729392 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f1ae40b0260 con 0x55cfca3e05d0 2024-09-19T06:08:10.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.782+0000 7f1afba99740 10 client.16644 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:10.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.782+0000 7f1afba99740 10 client.16644 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:08:10.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.782+0000 7f1afba99740 20 client.16644 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:10.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.782+0000 7f1afba99740 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55cfca465a70 0x55cfca467e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:10.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.782+0000 7f1afba99740 4 client.16644 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:08:10.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.782+0000 7f1afba99740 1 -- 172.21.15.5:0/2766729392 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfca34dd70 con 0x55cfca465a70 2024-09-19T06:08:10.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.782+0000 7f1af27fc640 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55cfca465a70 0x55cfca467e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:10.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.782+0000 7f1af27fc640 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55cfca465a70 0x55cfca467e60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c350a95ebc18899 server_cookie=f9b9edda547962f4 in_seq=0 out_seq=0 2024-09-19T06:08:10.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.782+0000 7f1ad6ff5640 10 client.16644 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:08:10.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.782+0000 7f1ad6ff5640 1 -- 172.21.15.5:0/2766729392 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfca34dd70 con 0x55cfca465a70 2024-09-19T06:08:10.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.782+0000 7f1ad6ff5640 10 client.16644 handle_command_reply: tid=0 2024-09-19T06:08:10.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.782+0000 7f1afba99740 1 client.16644 shutdown 2024-09-19T06:08:10.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1afba99740 1 -- 172.21.15.5:0/2766729392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 msgr2=0x55cfca3ee4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:10.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1afba99740 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3ee4a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1ae409c310 tx=0x7f1ae4000bc0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1afba99740 1 -- 172.21.15.5:0/2766729392 shutdown_connections 2024-09-19T06:08:10.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1afba99740 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55cfca465a70 0x55cfca467e60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1afba99740 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfca3eb370 0x55cfca463d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1afba99740 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfca3e1b40 0x55cfca3ee9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1ad6ff5640 0 client.16644 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:08:10.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1afba99740 1 --2- 172.21.15.5:0/2766729392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfca3e05d0 0x55cfca3ee4a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1afba99740 1 -- 172.21.15.5:0/2766729392 >> 172.21.15.5:0/2766729392 conn(0x55cfca3a9310 msgr2=0x55cfca3e98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:10.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1afba99740 1 -- 172.21.15.5:0/2766729392 shutdown_connections 2024-09-19T06:08:10.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1afba99740 1 -- 172.21.15.5:0/2766729392 wait complete. 2024-09-19T06:08:10.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1afba99740 20 client.16644 trim_cache size 0 max 16384 2024-09-19T06:08:10.794 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:08:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1af96c0640 1 -- 172.21.15.5:0/1686026808 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1adc0a33b0 msgr2=0x7f1adc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1af96c0640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1adc0a33b0 0x7f1adc0a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55cfca3e1070 tx=0x7f1ae8005f90 comp rx=0 tx=0).stop 2024-09-19T06:08:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1af96c0640 1 -- 172.21.15.5:0/1686026808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af4100e20 msgr2=0x7f1af419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1af96c0640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af4100e20 0x7f1af419a5a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400e9f0 tx=0x7f1ae400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1af96c0640 1 -- 172.21.15.5:0/1686026808 shutdown_connections 2024-09-19T06:08:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.783+0000 7f1af96c0640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af4102ef0 0x7f1af419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.784+0000 7f1af96c0640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1adc0a33b0 0x7f1adc0a5870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.784+0000 7f1af96c0640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1af4102570 0x7f1af419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.784+0000 7f1af96c0640 1 --2- 172.21.15.5:0/1686026808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af4100e20 0x7f1af419a5a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.784+0000 7f1af96c0640 1 -- 172.21.15.5:0/1686026808 >> 172.21.15.5:0/1686026808 conn(0x7f1af40fc7d0 msgr2=0x7f1af406a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.784+0000 7f1af96c0640 1 -- 172.21.15.5:0/1686026808 shutdown_connections 2024-09-19T06:08:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:10.784+0000 7f1af96c0640 1 -- 172.21.15.5:0/1686026808 wait complete. 2024-09-19T06:08:10.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 112307 2.79 MB/sec execute 96 sec latency 66.307 ms 2024-09-19T06:08:11.648 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:08:11.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 112876 2.80 MB/sec execute 97 sec latency 15.724 ms 2024-09-19T06:08:11.936 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:12.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:11 smithi005 ceph-mon[30955]: pgmap v149: 97 pgs: 97 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-19T06:08:12.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:11 smithi049 ceph-mon[34640]: pgmap v149: 97 pgs: 97 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-19T06:08:12.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:11 smithi157 ceph-mon[35346]: pgmap v149: 97 pgs: 97 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-19T06:08:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.499+0000 7f1684718640 1 -- 172.21.15.5:0/1451455351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f167c102ef0 msgr2=0x7f167c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.499+0000 7f1684718640 1 --2- 172.21.15.5:0/1451455351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f167c102ef0 0x7f167c10da40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1674009900 tx=0x7f167402a480 comp rx=0 tx=0).stop 2024-09-19T06:08:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.499+0000 7f1684718640 1 -- 172.21.15.5:0/1451455351 shutdown_connections 2024-09-19T06:08:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.499+0000 7f1684718640 1 --2- 172.21.15.5:0/1451455351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f167c102ef0 0x7f167c10da40 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.499+0000 7f1684718640 1 --2- 172.21.15.5:0/1451455351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f167c102570 0x7f167c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.499+0000 7f1684718640 1 --2- 172.21.15.5:0/1451455351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f167c100e20 0x7f167c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.499+0000 7f1684718640 1 -- 172.21.15.5:0/1451455351 >> 172.21.15.5:0/1451455351 conn(0x7f167c0fc7d0 msgr2=0x7f167c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.499+0000 7f1684718640 1 -- 172.21.15.5:0/1451455351 shutdown_connections 2024-09-19T06:08:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.499+0000 7f1684718640 1 -- 172.21.15.5:0/1451455351 wait complete. 2024-09-19T06:08:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.499+0000 7f1684718640 1 Processor -- start 2024-09-19T06:08:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1684718640 1 -- start start 2024-09-19T06:08:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1684718640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f167c100e20 0x7f167c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1684718640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f167c102570 0x7f167c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1684718640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f167c102ef0 0x7f167c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1684718640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f167c110e30 con 0x7f167c102570 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1684718640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f167c110cb0 con 0x7f167c102ef0 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1684718640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f167c110fb0 con 0x7f167c100e20 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1681cb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f167c102570 0x7f167c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1681cb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f167c102570 0x7f167c198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35978/0 (socket says 172.21.15.5:35978) 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1681cb3640 1 -- 172.21.15.5:0/3458368045 learned_addr learned my addr 172.21.15.5:0/3458368045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f16824b4640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f167c100e20 0x7f167c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1682cb5640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f167c102ef0 0x7f167c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1681cb3640 1 -- 172.21.15.5:0/3458368045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f167c100e20 msgr2=0x7f167c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1681cb3640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f167c100e20 0x7f167c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1681cb3640 1 -- 172.21.15.5:0/3458368045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f167c102ef0 msgr2=0x7f167c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1681cb3640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f167c102ef0 0x7f167c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.500+0000 7f1681cb3640 1 -- 172.21.15.5:0/3458368045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c19d370 con 0x7f167c102570 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.501+0000 7f1681cb3640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f167c102570 0x7f167c198900 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f167000e9f0 tx=0x7f167000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.501+0000 7f167b7fe640 1 -- 172.21.15.5:0/3458368045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167000cd80 con 0x7f167c102570 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.501+0000 7f167b7fe640 1 -- 172.21.15.5:0/3458368045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16700108e0 con 0x7f167c102570 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.501+0000 7f167b7fe640 1 -- 172.21.15.5:0/3458368045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670018920 con 0x7f167c102570 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.501+0000 7f1684718640 1 -- 172.21.15.5:0/3458368045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167c19d660 con 0x7f167c102570 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.501+0000 7f1684718640 1 -- 172.21.15.5:0/3458368045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167c1a4f40 con 0x7f167c102570 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.503+0000 7f1686af1740 1 Processor -- start 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.503+0000 7f167b7fe640 1 -- 172.21.15.5:0/3458368045 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1670010a90 con 0x7f167c102570 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f1686af1740 1 -- start start 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f1686af1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be7b40 0x55b377be6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f1686af1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be65d0 0x55b377be6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f1686af1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377bf5780 0x55b377bf7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f1686af1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b377b4c2f0 con 0x55b377be65d0 2024-09-19T06:08:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f1686af1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b377af3cf0 con 0x55b377be7b40 2024-09-19T06:08:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f1686af1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b377b482b0 con 0x55b377bf5780 2024-09-19T06:08:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f1682cb5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be7b40 0x55b377be6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f1682cb5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be7b40 0x55b377be6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39368/0 (socket says 172.21.15.5:39368) 2024-09-19T06:08:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f1682cb5640 1 -- 172.21.15.5:0/3464074276 learned_addr learned my addr 172.21.15.5:0/3464074276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f16824b4640 1 --2- 172.21.15.5:0/3464074276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be65d0 0x55b377be6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f1681cb3640 1 --2- 172.21.15.5:0/3464074276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377bf5780 0x55b377bf7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.504+0000 7f167b7fe640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f16640a3430 0x7f16640a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f1682cb5640 1 -- 172.21.15.5:0/3464074276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377bf5780 msgr2=0x55b377bf7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f1682cb5640 1 --2- 172.21.15.5:0/3464074276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377bf5780 0x55b377bf7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f1682cb5640 1 -- 172.21.15.5:0/3464074276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be65d0 msgr2=0x55b377be6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f1682cb5640 1 --2- 172.21.15.5:0/3464074276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be65d0 0x55b377be6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f1682cb5640 1 -- 172.21.15.5:0/3464074276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b377ad83a0 con 0x55b377be7b40 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f16824b4640 1 --2- 172.21.15.5:0/3464074276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be65d0 0x55b377be6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f1682cb5640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f16640a3430 0x7f16640a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f167b7fe640 1 -- 172.21.15.5:0/3458368045 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f16700d00e0 con 0x7f167c102570 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f1682cb5640 1 --2- 172.21.15.5:0/3464074276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be7b40 0x55b377be6090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1674033000 tx=0x7f1674031b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9587f345caf6dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f1678ff9640 1 -- 172.21.15.5:0/3464074276 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1674035040 con 0x55b377be7b40 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f1682cb5640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f16640a3430 0x7f16640a58f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f167404a000 tx=0x7f1674047000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f1678ff9640 1 -- 172.21.15.5:0/3464074276 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1674046aa0 con 0x55b377be7b40 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.505+0000 7f1678ff9640 1 -- 172.21.15.5:0/3464074276 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16740459b0 con 0x55b377be7b40 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.506+0000 7f1686af1740 1 -- 172.21.15.5:0/3464074276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be7b40 msgr2=0x55b377be6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.506+0000 7f1686af1740 1 --2- 172.21.15.5:0/3464074276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be7b40 0x55b377be6090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1674033000 tx=0x7f1674031b70 comp rx=0 tx=0).stop 2024-09-19T06:08:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.506+0000 7f1686af1740 1 -- 172.21.15.5:0/3464074276 shutdown_connections 2024-09-19T06:08:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.506+0000 7f1686af1740 1 --2- 172.21.15.5:0/3464074276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377bf5780 0x55b377bf7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.506+0000 7f1686af1740 1 --2- 172.21.15.5:0/3464074276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be65d0 0x55b377be6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.506+0000 7f1686af1740 1 --2- 172.21.15.5:0/3464074276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be7b40 0x55b377be6090 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.506+0000 7f1686af1740 1 -- 172.21.15.5:0/3464074276 >> 172.21.15.5:0/3464074276 conn(0x55b377baf310 msgr2=0x55b377be2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.506+0000 7f1686af1740 1 -- 172.21.15.5:0/3464074276 shutdown_connections 2024-09-19T06:08:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.506+0000 7f1686af1740 1 -- 172.21.15.5:0/3464074276 wait complete. 2024-09-19T06:08:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1686af1740 1 -- start start 2024-09-19T06:08:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1686af1740 1 Processor -- start 2024-09-19T06:08:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1686af1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be65d0 0x55b377bf51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1686af1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 0x55b377bf2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1686af1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377bf5780 0x55b377bf2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1686af1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b377bf8480 con 0x55b377be7b40 2024-09-19T06:08:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1686af1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b377b07060 con 0x55b377be65d0 2024-09-19T06:08:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1686af1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b377bf8600 con 0x55b377bf5780 2024-09-19T06:08:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1686af1740 10 client.0 _ll_register_callbacks cb 0x55b37785a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1681cb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 0x55b377bf2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1681cb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 0x55b377bf2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35996/0 (socket says 172.21.15.5:35996) 2024-09-19T06:08:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1681cb3640 1 -- 172.21.15.5:0/480116454 learned_addr learned my addr 172.21.15.5:0/480116454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f1682cb5640 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377bf5780 0x55b377bf2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.507+0000 7f16824b4640 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be65d0 0x55b377bf51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1681cb3640 1 -- 172.21.15.5:0/480116454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377bf5780 msgr2=0x55b377bf2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1681cb3640 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377bf5780 0x55b377bf2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1681cb3640 1 -- 172.21.15.5:0/480116454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be65d0 msgr2=0x55b377bf51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1681cb3640 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be65d0 0x55b377bf51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1681cb3640 1 -- 172.21.15.5:0/480116454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3778bd3c0 con 0x55b377be7b40 2024-09-19T06:08:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f16824b4640 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be65d0 0x55b377bf51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1681cb3640 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 0x55b377bf2450 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f167000e9c0 tx=0x7f16700d3820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c23660c3f7443a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1661ffb640 1 -- 172.21.15.5:0/480116454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167009bf10 con 0x55b377be7b40 2024-09-19T06:08:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1661ffb640 1 -- 172.21.15.5:0/480116454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f1670010430 con 0x55b377be7b40 2024-09-19T06:08:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1686af1740 1 -- 172.21.15.5:0/480116454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1674046aa0 con 0x55b377be7b40 2024-09-19T06:08:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1661ffb640 1 -- 172.21.15.5:0/480116454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f167009bd30 con 0x55b377be7b40 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1686af1740 10 client.16656 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.508+0000 7f1686af1740 1 -- 172.21.15.5:0/480116454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b377c03f50 con 0x55b377be7b40 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.509+0000 7f1661ffb640 1 -- 172.21.15.5:0/480116454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1670010de0 con 0x55b377be7b40 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.509+0000 7f1661ffb640 1 -- 172.21.15.5:0/480116454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f16700d20d0 con 0x55b377be7b40 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.509+0000 7f1686af1740 10 client.16656 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.509+0000 7f1686af1740 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.509+0000 7f1686af1740 20 client.16656 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.509+0000 7f1686af1740 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b377c05360 0x55b377c07770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.509+0000 7f1686af1740 4 client.16656 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.509+0000 7f1686af1740 1 -- 172.21.15.5:0/480116454 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1674046aa0 con 0x55b377c05360 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.509+0000 7f16824b4640 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b377c05360 0x55b377c07770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.510+0000 7f1661ffb640 10 client.16656 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.510+0000 7f16824b4640 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b377c05360 0x55b377c07770 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35eec31c44dcb65c server_cookie=ad2efbeb1a80d1c3 in_seq=0 out_seq=0 2024-09-19T06:08:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1661ffb640 1 -- 172.21.15.5:0/480116454 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f1674046aa0 con 0x55b377c05360 2024-09-19T06:08:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1661ffb640 10 client.16656 handle_command_reply: tid=0 2024-09-19T06:08:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1686af1740 1 client.16656 shutdown 2024-09-19T06:08:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1686af1740 1 -- 172.21.15.5:0/480116454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 msgr2=0x55b377bf2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1686af1740 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 0x55b377bf2450 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f167000e9c0 tx=0x7f16700d3820 comp rx=0 tx=0).stop 2024-09-19T06:08:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1686af1740 1 -- 172.21.15.5:0/480116454 shutdown_connections 2024-09-19T06:08:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1686af1740 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b377c05360 0x55b377c07770 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1686af1740 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377bf5780 0x55b377bf2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1686af1740 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 0x55b377bf2450 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1661ffb640 0 client.16656 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1686af1740 1 --2- 172.21.15.5:0/480116454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be65d0 0x55b377bf51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1686af1740 1 -- 172.21.15.5:0/480116454 >> 172.21.15.5:0/480116454 conn(0x55b377baf310 msgr2=0x55b377be8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1686af1740 1 -- 172.21.15.5:0/480116454 shutdown_connections 2024-09-19T06:08:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1686af1740 1 -- 172.21.15.5:0/480116454 wait complete. 2024-09-19T06:08:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.511+0000 7f1686af1740 20 client.16656 trim_cache size 0 max 16384 2024-09-19T06:08:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.525+0000 7f1686af1740 1 Processor -- start 2024-09-19T06:08:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.525+0000 7f1686af1740 1 -- start start 2024-09-19T06:08:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1686af1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377be65d0 0x55b377c01720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1686af1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 0x55b377bf43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1686af1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377bf5780 0x55b377c0d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1686af1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1ab580 con 0x55b377be7b40 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1686af1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1ad0c0 con 0x55b377bf5780 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1686af1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1abc50 con 0x55b377be65d0 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1681cb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 0x55b377bf43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1681cb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 0x55b377bf43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36002/0 (socket says 172.21.15.5:36002) 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1681cb3640 1 -- 172.21.15.5:0/868863740 learned_addr learned my addr 172.21.15.5:0/868863740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1682cb5640 1 --2- 172.21.15.5:0/868863740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377bf5780 0x55b377c0d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f16824b4640 1 --2- 172.21.15.5:0/868863740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377be65d0 0x55b377c01720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1681cb3640 1 -- 172.21.15.5:0/868863740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377be65d0 msgr2=0x55b377c01720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1681cb3640 1 --2- 172.21.15.5:0/868863740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377be65d0 0x55b377c01720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1681cb3640 1 -- 172.21.15.5:0/868863740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377bf5780 msgr2=0x55b377c0d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1681cb3640 1 --2- 172.21.15.5:0/868863740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377bf5780 0x55b377c0d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1681cb3640 1 -- 172.21.15.5:0/868863740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b377c07d50 con 0x55b377be7b40 2024-09-19T06:08:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.526+0000 7f1681cb3640 1 --2- 172.21.15.5:0/868863740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 0x55b377bf43c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f16700a4b10 tx=0x7f1670009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7060c368002342e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.527+0000 7f1663fff640 1 -- 172.21.15.5:0/868863740 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167009bf10 con 0x55b377be7b40 2024-09-19T06:08:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.527+0000 7f1663fff640 1 -- 172.21.15.5:0/868863740 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1670010de0 con 0x55b377be7b40 2024-09-19T06:08:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.527+0000 7f1663fff640 1 -- 172.21.15.5:0/868863740 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670098a40 con 0x55b377be7b40 2024-09-19T06:08:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.527+0000 7f1686af1740 1 -- 172.21.15.5:0/868863740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 msgr2=0x55b377bf43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.527+0000 7f1686af1740 1 --2- 172.21.15.5:0/868863740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 0x55b377bf43c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f16700a4b10 tx=0x7f1670009700 comp rx=0 tx=0).stop 2024-09-19T06:08:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.527+0000 7f1686af1740 1 -- 172.21.15.5:0/868863740 shutdown_connections 2024-09-19T06:08:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.527+0000 7f1686af1740 1 --2- 172.21.15.5:0/868863740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377bf5780 0x55b377c0d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.527+0000 7f1686af1740 1 --2- 172.21.15.5:0/868863740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377be7b40 0x55b377bf43c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.527+0000 7f1686af1740 1 --2- 172.21.15.5:0/868863740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377be65d0 0x55b377c01720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.527+0000 7f1686af1740 1 -- 172.21.15.5:0/868863740 >> 172.21.15.5:0/868863740 conn(0x55b377baf310 msgr2=0x55b377bf3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.527+0000 7f1686af1740 1 -- 172.21.15.5:0/868863740 shutdown_connections 2024-09-19T06:08:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.527+0000 7f1686af1740 1 -- 172.21.15.5:0/868863740 wait complete. 2024-09-19T06:08:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1686af1740 1 -- start start 2024-09-19T06:08:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1686af1740 1 Processor -- start 2024-09-19T06:08:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1686af1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377be65d0 0x55b377bf9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1686af1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be7b40 0x55b377bfa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1686af1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377bf5780 0x55b377c6efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1686af1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1ac9f0 con 0x55b377bf5780 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1686af1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1ac320 con 0x55b377be7b40 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1686af1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1a9560 con 0x55b377be65d0 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1686af1740 10 client.0 _ll_register_callbacks cb 0x7f16700d3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1682cb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377bf5780 0x55b377c6efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f16824b4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377be65d0 0x55b377bf9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1682cb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377bf5780 0x55b377c6efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36018/0 (socket says 172.21.15.5:36018) 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f16824b4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377be65d0 0x55b377bf9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60698/0 (socket says 172.21.15.5:60698) 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1682cb5640 1 -- 172.21.15.5:0/1444504125 learned_addr learned my addr 172.21.15.5:0/1444504125 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.528+0000 7f1681cb3640 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be7b40 0x55b377bfa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f16824b4640 1 -- 172.21.15.5:0/1444504125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be7b40 msgr2=0x55b377bfa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f16824b4640 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be7b40 0x55b377bfa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f16824b4640 1 -- 172.21.15.5:0/1444504125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377bf5780 msgr2=0x55b377c6efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f16824b4640 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377bf5780 0x55b377c6efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f16824b4640 1 -- 172.21.15.5:0/1444504125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b377c6f520 con 0x55b377be65d0 2024-09-19T06:08:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f1682cb5640 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377bf5780 0x55b377c6efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f16824b4640 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377be65d0 0x55b377bf9d30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55b377c07f50 tx=0x7f166c012b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d2fac1634d95cb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f1661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:08:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f1661ffb640 1 -- 172.21.15.5:0/1444504125 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c02a060 con 0x55b377be65d0 2024-09-19T06:08:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f1661ffb640 1 -- 172.21.15.5:0/1444504125 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f166c011910 con 0x55b377be65d0 2024-09-19T06:08:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f1686af1740 1 -- 172.21.15.5:0/1444504125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b377b59860 con 0x55b377be65d0 2024-09-19T06:08:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f1661ffb640 1 -- 172.21.15.5:0/1444504125 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f166c011b10 con 0x55b377be65d0 2024-09-19T06:08:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f1686af1740 10 client.26173 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.529+0000 7f1686af1740 1 -- 172.21.15.5:0/1444504125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b377c70820 con 0x55b377be65d0 2024-09-19T06:08:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.530+0000 7f1661ffb640 1 -- 172.21.15.5:0/1444504125 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b377b59860 con 0x55b377be65d0 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.530+0000 7f1661ffb640 1 -- 172.21.15.5:0/1444504125 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f166c00f040 con 0x55b377be65d0 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.530+0000 7f1686af1740 10 client.26173 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.530+0000 7f1686af1740 10 client.26173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.530+0000 7f1686af1740 20 client.26173 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.530+0000 7f1686af1740 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b377c05360 0x55b377c72f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.530+0000 7f1686af1740 4 client.26173 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.530+0000 7f1686af1740 1 -- 172.21.15.5:0/1444504125 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b377b53d70 con 0x55b377c05360 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.530+0000 7f1681cb3640 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b377c05360 0x55b377c72f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.530+0000 7f1661ffb640 10 client.26173 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.530+0000 7f1681cb3640 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b377c05360 0x55b377c72f50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a957e7c6ef104e1 server_cookie=58cce19182a80a3e in_seq=0 out_seq=0 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1661ffb640 1 -- 172.21.15.5:0/1444504125 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b377b53d70 con 0x55b377c05360 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1661ffb640 10 client.26173 handle_command_reply: tid=0 2024-09-19T06:08:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1686af1740 1 client.26173 shutdown 2024-09-19T06:08:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1686af1740 1 -- 172.21.15.5:0/1444504125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377be65d0 msgr2=0x55b377bf9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1686af1740 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377be65d0 0x55b377bf9d30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55b377c07f50 tx=0x7f166c012b10 comp rx=0 tx=0).stop 2024-09-19T06:08:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1686af1740 1 -- 172.21.15.5:0/1444504125 shutdown_connections 2024-09-19T06:08:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1686af1740 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b377c05360 0x55b377c72f50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1686af1740 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b377bf5780 0x55b377c6efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1686af1740 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b377be7b40 0x55b377bfa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1661ffb640 0 client.26173 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1686af1740 1 --2- 172.21.15.5:0/1444504125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377be65d0 0x55b377bf9d30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1686af1740 1 -- 172.21.15.5:0/1444504125 >> 172.21.15.5:0/1444504125 conn(0x55b377baf310 msgr2=0x55b377be8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1686af1740 1 -- 172.21.15.5:0/1444504125 shutdown_connections 2024-09-19T06:08:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1686af1740 1 -- 172.21.15.5:0/1444504125 wait complete. 2024-09-19T06:08:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.531+0000 7f1686af1740 20 client.26173 trim_cache size 0 max 16384 2024-09-19T06:08:12.539 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:08:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.532+0000 7f1684718640 1 -- 172.21.15.5:0/3458368045 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f16640a3430 msgr2=0x7f16640a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.532+0000 7f1684718640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f16640a3430 0x7f16640a58f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f167404a000 tx=0x7f1674047000 comp rx=0 tx=0).stop 2024-09-19T06:08:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.532+0000 7f1684718640 1 -- 172.21.15.5:0/3458368045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f167c102570 msgr2=0x7f167c198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.532+0000 7f1684718640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f167c102570 0x7f167c198900 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f167000e9f0 tx=0x7f167000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.532+0000 7f1684718640 1 -- 172.21.15.5:0/3458368045 shutdown_connections 2024-09-19T06:08:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.532+0000 7f1684718640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f167c102ef0 0x7f167c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.532+0000 7f1684718640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f167c102570 0x7f167c198900 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.532+0000 7f1684718640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f16640a3430 0x7f16640a58f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.532+0000 7f1684718640 1 --2- 172.21.15.5:0/3458368045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f167c100e20 0x7f167c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.532+0000 7f1684718640 1 -- 172.21.15.5:0/3458368045 >> 172.21.15.5:0/3458368045 conn(0x7f167c0fc7d0 msgr2=0x7f167c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.532+0000 7f1684718640 1 -- 172.21.15.5:0/3458368045 shutdown_connections 2024-09-19T06:08:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:12.532+0000 7f1684718640 1 -- 172.21.15.5:0/3458368045 wait complete. 2024-09-19T06:08:12.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 113354 2.81 MB/sec execute 98 sec latency 28.257 ms 2024-09-19T06:08:13.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 113836 2.82 MB/sec execute 99 sec latency 14.925 ms 2024-09-19T06:08:14.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:13 smithi005 ceph-mon[30955]: pgmap v150: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-19T06:08:14.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:13 smithi049 ceph-mon[34640]: pgmap v150: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-19T06:08:14.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:13 smithi157 ceph-mon[35346]: pgmap v150: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-19T06:08:14.338 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:08:14.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:08:14.613 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:14.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 114100 2.80 MB/sec execute 100 sec latency 14.750 ms 2024-09-19T06:08:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.148+0000 7f8f7af49640 1 -- 172.21.15.5:0/3994451851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f74102570 msgr2=0x7f8f741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.148+0000 7f8f7af49640 1 --2- 172.21.15.5:0/3994451851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f74102570 0x7f8f741029b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009900 tx=0x7f8f6002a480 comp rx=0 tx=0).stop 2024-09-19T06:08:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.148+0000 7f8f7af49640 1 -- 172.21.15.5:0/3994451851 shutdown_connections 2024-09-19T06:08:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.148+0000 7f8f7af49640 1 --2- 172.21.15.5:0/3994451851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f74102ef0 0x7f8f7410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.148+0000 7f8f7af49640 1 --2- 172.21.15.5:0/3994451851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f74102570 0x7f8f741029b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.148+0000 7f8f7af49640 1 --2- 172.21.15.5:0/3994451851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f74100e20 0x7f8f74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.148+0000 7f8f7af49640 1 -- 172.21.15.5:0/3994451851 >> 172.21.15.5:0/3994451851 conn(0x7f8f740fc7d0 msgr2=0x7f8f740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.148+0000 7f8f7af49640 1 -- 172.21.15.5:0/3994451851 shutdown_connections 2024-09-19T06:08:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.148+0000 7f8f7af49640 1 -- 172.21.15.5:0/3994451851 wait complete. 2024-09-19T06:08:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f7af49640 1 Processor -- start 2024-09-19T06:08:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f7af49640 1 -- start start 2024-09-19T06:08:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f7af49640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f74100e20 0x7f8f74197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f7af49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f74102570 0x7f8f74197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f7af49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f74102ef0 0x7f8f7419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f7af49640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f74110e30 con 0x7f8f74100e20 2024-09-19T06:08:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f7af49640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f74110cb0 con 0x7f8f74102570 2024-09-19T06:08:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f7af49640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f74110fb0 con 0x7f8f74102ef0 2024-09-19T06:08:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f78ce5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f74100e20 0x7f8f74197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f78ce5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f74100e20 0x7f8f74197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55152/0 (socket says 172.21.15.5:55152) 2024-09-19T06:08:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f78ce5640 1 -- 172.21.15.5:0/75743948 learned_addr learned my addr 172.21.15.5:0/75743948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f73fff640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f74102570 0x7f8f74197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.149+0000 7f8f794e6640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f74102ef0 0x7f8f7419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.150+0000 7f8f78ce5640 1 -- 172.21.15.5:0/75743948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f74102ef0 msgr2=0x7f8f7419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.150+0000 7f8f78ce5640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f74102ef0 0x7f8f7419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.150+0000 7f8f78ce5640 1 -- 172.21.15.5:0/75743948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f74102570 msgr2=0x7f8f74197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.150+0000 7f8f78ce5640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f74102570 0x7f8f74197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.150+0000 7f8f78ce5640 1 -- 172.21.15.5:0/75743948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7419c2c0 con 0x7f8f74100e20 2024-09-19T06:08:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.150+0000 7f8f73fff640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f74102570 0x7f8f74197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.150+0000 7f8f78ce5640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f74100e20 0x7f8f74197310 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800eb00 tx=0x7f8f68009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.150+0000 7f8f71ffb640 1 -- 172.21.15.5:0/75743948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6801c070 con 0x7f8f74100e20 2024-09-19T06:08:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.150+0000 7f8f71ffb640 1 -- 172.21.15.5:0/75743948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f6800bc30 con 0x7f8f74100e20 2024-09-19T06:08:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.150+0000 7f8f7af49640 1 -- 172.21.15.5:0/75743948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7419c5b0 con 0x7f8f74100e20 2024-09-19T06:08:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.150+0000 7f8f71ffb640 1 -- 172.21.15.5:0/75743948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68016610 con 0x7f8f74100e20 2024-09-19T06:08:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.150+0000 7f8f7af49640 1 -- 172.21.15.5:0/75743948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f74069f70 con 0x7f8f74100e20 2024-09-19T06:08:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f71ffb640 1 -- 172.21.15.5:0/75743948 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8f6801f070 con 0x7f8f74100e20 2024-09-19T06:08:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f7d322740 1 Processor -- start 2024-09-19T06:08:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f7d322740 1 -- start start 2024-09-19T06:08:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f7d322740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 0x555f4ebe7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f7d322740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe8480 0x555f4ebf2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f7d322740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebf5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f7d322740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555f4eb4d280 con 0x555f4ebf2d40 2024-09-19T06:08:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f7d322740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555f4eaf4b10 con 0x555f4ebe7b60 2024-09-19T06:08:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f7d322740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555f4eb49240 con 0x555f4ebe8480 2024-09-19T06:08:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f794e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 0x555f4ebe7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f78ce5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe8480 0x555f4ebf2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f794e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 0x555f4ebe7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:34788/0 (socket says 172.21.15.5:34788) 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f73fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebf5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.153+0000 7f8f794e6640 1 -- 172.21.15.5:0/1974432776 learned_addr learned my addr 172.21.15.5:0/1974432776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.154+0000 7f8f71ffb640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8f640a33b0 0x7f8f640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.154+0000 7f8f794e6640 1 -- 172.21.15.5:0/1974432776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe8480 msgr2=0x555f4ebf2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.154+0000 7f8f794e6640 1 --2- 172.21.15.5:0/1974432776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe8480 0x555f4ebf2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.154+0000 7f8f794e6640 1 -- 172.21.15.5:0/1974432776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 msgr2=0x555f4ebf5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.154+0000 7f8f794e6640 1 --2- 172.21.15.5:0/1974432776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebf5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.154+0000 7f8f794e6640 1 -- 172.21.15.5:0/1974432776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f4eb011f0 con 0x555f4ebe7b60 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.154+0000 7f8f71ffb640 1 -- 172.21.15.5:0/75743948 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f8f680d12c0 con 0x7f8f74100e20 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.154+0000 7f8f794e6640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8f640a33b0 0x7f8f640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.154+0000 7f8f794e6640 1 --2- 172.21.15.5:0/1974432776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 0x555f4ebe7f40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c010510 tx=0x7f8f6c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f68778c995fbd5c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.154+0000 7f8f5effd640 1 -- 172.21.15.5:0/1974432776 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c023070 con 0x555f4ebe7b60 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.154+0000 7f8f5effd640 1 -- 172.21.15.5:0/1974432776 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f6c0134c0 con 0x555f4ebe7b60 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.154+0000 7f8f5effd640 1 -- 172.21.15.5:0/1974432776 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0118a0 con 0x555f4ebe7b60 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.155+0000 7f8f7d322740 1 -- 172.21.15.5:0/1974432776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 msgr2=0x555f4ebe7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.155+0000 7f8f7d322740 1 --2- 172.21.15.5:0/1974432776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 0x555f4ebe7f40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c010510 tx=0x7f8f6c009f90 comp rx=0 tx=0).stop 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.155+0000 7f8f794e6640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8f640a33b0 0x7f8f640a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c020000 tx=0x7f8f6c01b000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.155+0000 7f8f7d322740 1 -- 172.21.15.5:0/1974432776 shutdown_connections 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.155+0000 7f8f7d322740 1 --2- 172.21.15.5:0/1974432776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebf5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.155+0000 7f8f7d322740 1 --2- 172.21.15.5:0/1974432776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe8480 0x555f4ebf2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.155+0000 7f8f7d322740 1 --2- 172.21.15.5:0/1974432776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 0x555f4ebe7f40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.155+0000 7f8f7d322740 1 -- 172.21.15.5:0/1974432776 >> 172.21.15.5:0/1974432776 conn(0x555f4ebe40f0 msgr2=0x555f4ebe44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.155+0000 7f8f7d322740 1 -- 172.21.15.5:0/1974432776 shutdown_connections 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.155+0000 7f8f7d322740 1 -- 172.21.15.5:0/1974432776 wait complete. 2024-09-19T06:08:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f7d322740 1 -- start start 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f7d322740 1 Processor -- start 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f7d322740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebe7b60 0x555f4ebefbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f7d322740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe8480 0x555f4ebf0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f7d322740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebf2d40 0x555f4ebf0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f7d322740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555f4ebf9d20 con 0x555f4ebe7b60 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f7d322740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555f4eb07f10 con 0x555f4ebe8480 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f7d322740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555f4ebf9ea0 con 0x555f4ebf2d40 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f7d322740 10 client.0 _ll_register_callbacks cb 0x555f4ead0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f78ce5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebe7b60 0x555f4ebefbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f78ce5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebe7b60 0x555f4ebefbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55160/0 (socket says 172.21.15.5:55160) 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f78ce5640 1 -- 172.21.15.5:0/3425446666 learned_addr learned my addr 172.21.15.5:0/3425446666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.156+0000 7f8f73fff640 1 --2- 172.21.15.5:0/3425446666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe8480 0x555f4ebf0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f78ce5640 1 -- 172.21.15.5:0/3425446666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebf2d40 msgr2=0x555f4ebf0670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f78ce5640 1 --2- 172.21.15.5:0/3425446666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebf2d40 0x555f4ebf0670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f78ce5640 1 -- 172.21.15.5:0/3425446666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe8480 msgr2=0x555f4ebf0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f78ce5640 1 --2- 172.21.15.5:0/3425446666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe8480 0x555f4ebf0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f78ce5640 1 -- 172.21.15.5:0/3425446666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f4e8be6a0 con 0x555f4ebe7b60 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f78ce5640 1 --2- 172.21.15.5:0/3425446666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebe7b60 0x555f4ebefbf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8f680cf020 tx=0x7f8f680097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728ac9f3a177b89f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f53fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f53fff640 1 -- 172.21.15.5:0/3425446666 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f680d0070 con 0x555f4ebe7b60 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f53fff640 1 -- 172.21.15.5:0/3425446666 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f8f680a5d90 con 0x555f4ebe7b60 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f7d322740 1 -- 172.21.15.5:0/3425446666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f6c0134c0 con 0x555f4ebe7b60 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f53fff640 1 -- 172.21.15.5:0/3425446666 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8f6809d660 con 0x555f4ebe7b60 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f7d322740 10 client.16674 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f7d322740 1 -- 172.21.15.5:0/3425446666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555f4ec058f0 con 0x555f4ebe7b60 2024-09-19T06:08:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f53fff640 1 -- 172.21.15.5:0/3425446666 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f6c0134c0 con 0x555f4ebe7b60 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.157+0000 7f8f53fff640 1 -- 172.21.15.5:0/3425446666 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f8f680b62f0 con 0x555f4ebe7b60 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.158+0000 7f8f7d322740 10 client.16674 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.158+0000 7f8f7d322740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.158+0000 7f8f7d322740 20 client.16674 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.158+0000 7f8f7d322740 1 --2- 172.21.15.5:0/3425446666 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555f4ec06d00 0x555f4ec09110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.158+0000 7f8f7d322740 4 client.16674 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.158+0000 7f8f7d322740 1 -- 172.21.15.5:0/3425446666 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f4eb5a7f0 con 0x555f4ec06d00 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.158+0000 7f8f73fff640 1 --2- 172.21.15.5:0/3425446666 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555f4ec06d00 0x555f4ec09110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.158+0000 7f8f73fff640 1 --2- 172.21.15.5:0/3425446666 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555f4ec06d00 0x555f4ec09110 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2cf1aa60a835a2d server_cookie=40c3e608f75820fc in_seq=0 out_seq=0 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.158+0000 7f8f53fff640 10 client.16674 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f53fff640 1 -- 172.21.15.5:0/3425446666 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f4eb5a7f0 con 0x555f4ec06d00 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f53fff640 10 client.16674 handle_command_reply: tid=0 2024-09-19T06:08:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f7d322740 1 client.16674 shutdown 2024-09-19T06:08:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f7d322740 1 -- 172.21.15.5:0/3425446666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebe7b60 msgr2=0x555f4ebefbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f7d322740 1 --2- 172.21.15.5:0/3425446666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebe7b60 0x555f4ebefbf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8f680cf020 tx=0x7f8f680097c0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f7d322740 1 -- 172.21.15.5:0/3425446666 shutdown_connections 2024-09-19T06:08:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f7d322740 1 --2- 172.21.15.5:0/3425446666 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555f4ec06d00 0x555f4ec09110 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f7d322740 1 --2- 172.21.15.5:0/3425446666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebf2d40 0x555f4ebf0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f7d322740 1 --2- 172.21.15.5:0/3425446666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe8480 0x555f4ebf0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f7d322740 1 --2- 172.21.15.5:0/3425446666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebe7b60 0x555f4ebefbf0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f53fff640 0 client.16674 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f7d322740 1 -- 172.21.15.5:0/3425446666 >> 172.21.15.5:0/3425446666 conn(0x555f4ebe40f0 msgr2=0x555f4ebe3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f7d322740 1 -- 172.21.15.5:0/3425446666 shutdown_connections 2024-09-19T06:08:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f7d322740 1 -- 172.21.15.5:0/3425446666 wait complete. 2024-09-19T06:08:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.159+0000 7f8f7d322740 20 client.16674 trim_cache size 0 max 16384 2024-09-19T06:08:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f7d322740 1 Processor -- start 2024-09-19T06:08:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f7d322740 1 -- start start 2024-09-19T06:08:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f7d322740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 0x555f4ec0e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f7d322740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe8480 0x555f4ec0eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f7d322740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebece70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f7d322740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f741ab5e0 con 0x555f4ebf2d40 2024-09-19T06:08:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f7d322740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f741ad120 con 0x555f4ebe7b60 2024-09-19T06:08:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f7d322740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f741abcb0 con 0x555f4ebe8480 2024-09-19T06:08:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f78ce5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 0x555f4ec0e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f73fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe8480 0x555f4ec0eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f78ce5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 0x555f4ec0e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:34814/0 (socket says 172.21.15.5:34814) 2024-09-19T06:08:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f73fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe8480 0x555f4ec0eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53104/0 (socket says 172.21.15.5:53104) 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.179+0000 7f8f78ce5640 1 -- 172.21.15.5:0/3082917034 learned_addr learned my addr 172.21.15.5:0/3082917034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.180+0000 7f8f794e6640 1 --2- 172.21.15.5:0/3082917034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebece70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.180+0000 7f8f78ce5640 1 -- 172.21.15.5:0/3082917034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe8480 msgr2=0x555f4ec0eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.180+0000 7f8f78ce5640 1 --2- 172.21.15.5:0/3082917034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe8480 0x555f4ec0eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.180+0000 7f8f78ce5640 1 -- 172.21.15.5:0/3082917034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 msgr2=0x555f4ebece70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.180+0000 7f8f78ce5640 1 --2- 172.21.15.5:0/3082917034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebece70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.180+0000 7f8f78ce5640 1 -- 172.21.15.5:0/3082917034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f4ebee140 con 0x555f4ebe7b60 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.180+0000 7f8f78ce5640 1 --2- 172.21.15.5:0/3082917034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 0x555f4ec0e980 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8f6809b910 tx=0x7f8f6809d080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a941c2a1939da1fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.180+0000 7f8f5e7fc640 1 -- 172.21.15.5:0/3082917034 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f680d0070 con 0x555f4ebe7b60 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.180+0000 7f8f5e7fc640 1 -- 172.21.15.5:0/3082917034 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f680bb020 con 0x555f4ebe7b60 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.180+0000 7f8f5e7fc640 1 -- 172.21.15.5:0/3082917034 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f680a5ff0 con 0x555f4ebe7b60 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.181+0000 7f8f7d322740 1 -- 172.21.15.5:0/3082917034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 msgr2=0x555f4ec0e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.181+0000 7f8f7d322740 1 --2- 172.21.15.5:0/3082917034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 0x555f4ec0e980 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8f6809b910 tx=0x7f8f6809d080 comp rx=0 tx=0).stop 2024-09-19T06:08:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.181+0000 7f8f7d322740 1 -- 172.21.15.5:0/3082917034 shutdown_connections 2024-09-19T06:08:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.181+0000 7f8f7d322740 1 --2- 172.21.15.5:0/3082917034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebece70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.181+0000 7f8f7d322740 1 --2- 172.21.15.5:0/3082917034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe8480 0x555f4ec0eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.181+0000 7f8f7d322740 1 --2- 172.21.15.5:0/3082917034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe7b60 0x555f4ec0e980 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.181+0000 7f8f7d322740 1 -- 172.21.15.5:0/3082917034 >> 172.21.15.5:0/3082917034 conn(0x555f4ebe40f0 msgr2=0x555f4ebf1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.181+0000 7f8f7d322740 1 -- 172.21.15.5:0/3082917034 shutdown_connections 2024-09-19T06:08:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.181+0000 7f8f7d322740 1 -- 172.21.15.5:0/3082917034 wait complete. 2024-09-19T06:08:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f7d322740 1 -- start start 2024-09-19T06:08:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f7d322740 1 Processor -- start 2024-09-19T06:08:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f7d322740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe7b60 0x555f4ebff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f7d322740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe8480 0x555f4ebff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f7d322740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebfc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f7d322740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f741aca50 con 0x555f4ebf2d40 2024-09-19T06:08:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f7d322740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f741ac380 con 0x555f4ebe8480 2024-09-19T06:08:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f7d322740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f741a95c0 con 0x555f4ebe7b60 2024-09-19T06:08:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f7d322740 10 client.0 _ll_register_callbacks cb 0x7f8f6809d520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f794e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebfc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f78ce5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe7b60 0x555f4ebff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f794e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebfc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55168/0 (socket says 172.21.15.5:55168) 2024-09-19T06:08:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f794e6640 1 -- 172.21.15.5:0/2812267480 learned_addr learned my addr 172.21.15.5:0/2812267480 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.182+0000 7f8f73fff640 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe8480 0x555f4ebff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f78ce5640 1 -- 172.21.15.5:0/2812267480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe8480 msgr2=0x555f4ebff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f78ce5640 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe8480 0x555f4ebff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f78ce5640 1 -- 172.21.15.5:0/2812267480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 msgr2=0x555f4ebfc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f78ce5640 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebfc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f78ce5640 1 -- 172.21.15.5:0/2812267480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f4ebfc600 con 0x555f4ebe7b60 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f794e6640 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebfc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f78ce5640 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe7b60 0x555f4ebff0f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f6809d360 tx=0x7f8f68099270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a117341d7062169 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f53fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f53fff640 1 -- 172.21.15.5:0/2812267480 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f680d0070 con 0x555f4ebe7b60 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f53fff640 1 -- 172.21.15.5:0/2812267480 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f8f6809cab0 con 0x555f4ebe7b60 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f7d322740 1 -- 172.21.15.5:0/2812267480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f680bb020 con 0x555f4ebe7b60 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f53fff640 1 -- 172.21.15.5:0/2812267480 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8f680a1960 con 0x555f4ebe7b60 2024-09-19T06:08:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f7d322740 10 client.26188 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.183+0000 7f8f7d322740 1 -- 172.21.15.5:0/2812267480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555f4ec72080 con 0x555f4ebe7b60 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.184+0000 7f8f53fff640 1 -- 172.21.15.5:0/2812267480 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f680bb020 con 0x555f4ebe7b60 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.184+0000 7f8f53fff640 1 -- 172.21.15.5:0/2812267480 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f8f6809c520 con 0x555f4ebe7b60 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.184+0000 7f8f7d322740 10 client.26188 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.184+0000 7f8f7d322740 10 client.26188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.184+0000 7f8f7d322740 20 client.26188 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.184+0000 7f8f7d322740 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555f4ec72500 0x555f4ec748f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.184+0000 7f8f7d322740 4 client.26188 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.184+0000 7f8f7d322740 1 -- 172.21.15.5:0/2812267480 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555f4eb54d00 con 0x555f4ec72500 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.184+0000 7f8f73fff640 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555f4ec72500 0x555f4ec748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.184+0000 7f8f53fff640 10 client.26188 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.184+0000 7f8f73fff640 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555f4ec72500 0x555f4ec748f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5d8cace0075d7dc server_cookie=570e68f409e0926d in_seq=0 out_seq=0 2024-09-19T06:08:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f53fff640 1 -- 172.21.15.5:0/2812267480 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555f4eb54d00 con 0x555f4ec72500 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f53fff640 10 client.26188 handle_command_reply: tid=0 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f7d322740 1 client.26188 shutdown 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f7d322740 1 -- 172.21.15.5:0/2812267480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe7b60 msgr2=0x555f4ebff0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f7d322740 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe7b60 0x555f4ebff0f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f6809d360 tx=0x7f8f68099270 comp rx=0 tx=0).stop 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f7d322740 1 -- 172.21.15.5:0/2812267480 shutdown_connections 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f7d322740 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555f4ec72500 0x555f4ec748f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f7d322740 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f4ebf2d40 0x555f4ebfc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f7d322740 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f4ebe8480 0x555f4ebff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f53fff640 0 client.26188 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f7d322740 1 --2- 172.21.15.5:0/2812267480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f4ebe7b60 0x555f4ebff0f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f7d322740 1 -- 172.21.15.5:0/2812267480 >> 172.21.15.5:0/2812267480 conn(0x555f4ebe40f0 msgr2=0x555f4ebe2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f7d322740 1 -- 172.21.15.5:0/2812267480 shutdown_connections 2024-09-19T06:08:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f7d322740 1 -- 172.21.15.5:0/2812267480 wait complete. 2024-09-19T06:08:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.186+0000 7f8f7d322740 20 client.26188 trim_cache size 0 max 16384 2024-09-19T06:08:15.199 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:08:15.199 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:08:15.199 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "6d16f2de-e850-4ccd-9ed9-a9ba25c49a11", 2024-09-19T06:08:15.199 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:08:15.199 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:08:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.187+0000 7f8f7af49640 1 -- 172.21.15.5:0/75743948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8f640a33b0 msgr2=0x7f8f640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.187+0000 7f8f7af49640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8f640a33b0 0x7f8f640a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c020000 tx=0x7f8f6c01b000 comp rx=0 tx=0).stop 2024-09-19T06:08:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.187+0000 7f8f7af49640 1 -- 172.21.15.5:0/75743948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f74100e20 msgr2=0x7f8f74197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.187+0000 7f8f7af49640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f74100e20 0x7f8f74197310 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800eb00 tx=0x7f8f68009f20 comp rx=0 tx=0).stop 2024-09-19T06:08:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.187+0000 7f8f7af49640 1 -- 172.21.15.5:0/75743948 shutdown_connections 2024-09-19T06:08:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.187+0000 7f8f7af49640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8f640a33b0 0x7f8f640a5870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.187+0000 7f8f7af49640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f74102ef0 0x7f8f7419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.187+0000 7f8f7af49640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f74102570 0x7f8f74197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.187+0000 7f8f7af49640 1 --2- 172.21.15.5:0/75743948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f74100e20 0x7f8f74197310 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:15.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.187+0000 7f8f7af49640 1 -- 172.21.15.5:0/75743948 >> 172.21.15.5:0/75743948 conn(0x7f8f740fc7d0 msgr2=0x7f8f740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:15.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.187+0000 7f8f7af49640 1 -- 172.21.15.5:0/75743948 shutdown_connections 2024-09-19T06:08:15.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:15.187+0000 7f8f7af49640 1 -- 172.21.15.5:0/75743948 wait complete. 2024-09-19T06:08:15.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 114547 2.78 MB/sec execute 101 sec latency 18.867 ms 2024-09-19T06:08:16.014 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:08:16.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: pgmap v151: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-19T06:08:16.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:08:16.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:08:16.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:08:16.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:08:16.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:08:16.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:08:16.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:15 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: pgmap v151: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-19T06:08:16.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:08:16.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:08:16.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:08:16.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:08:16.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:08:16.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:08:16.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:15 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: pgmap v151: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-19T06:08:16.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:08:16.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:08:16.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:08:16.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:08:16.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:08:16.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:08:16.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:15 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:16.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 115076 2.79 MB/sec execute 102 sec latency 46.562 ms 2024-09-19T06:08:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.794+0000 7f3b561fe640 1 -- 172.21.15.5:0/3451343893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b50102570 msgr2=0x7f3b501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.794+0000 7f3b561fe640 1 --2- 172.21.15.5:0/3451343893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b50102570 0x7f3b501029b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009890 tx=0x7f3b3802a410 comp rx=0 tx=0).stop 2024-09-19T06:08:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.794+0000 7f3b561fe640 1 -- 172.21.15.5:0/3451343893 shutdown_connections 2024-09-19T06:08:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.794+0000 7f3b561fe640 1 --2- 172.21.15.5:0/3451343893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b50102ef0 0x7f3b5010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.794+0000 7f3b561fe640 1 --2- 172.21.15.5:0/3451343893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b50102570 0x7f3b501029b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.794+0000 7f3b561fe640 1 --2- 172.21.15.5:0/3451343893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50100e20 0x7f3b50101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.794+0000 7f3b561fe640 1 -- 172.21.15.5:0/3451343893 >> 172.21.15.5:0/3451343893 conn(0x7f3b500fc7d0 msgr2=0x7f3b500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.794+0000 7f3b561fe640 1 -- 172.21.15.5:0/3451343893 shutdown_connections 2024-09-19T06:08:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.794+0000 7f3b561fe640 1 -- 172.21.15.5:0/3451343893 wait complete. 2024-09-19T06:08:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b561fe640 1 Processor -- start 2024-09-19T06:08:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b561fe640 1 -- start start 2024-09-19T06:08:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b561fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50100e20 0x7f3b50197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b561fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b50102570 0x7f3b50197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b561fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b50102ef0 0x7f3b5019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b561fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3b50110e50 con 0x7f3b50100e20 2024-09-19T06:08:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50100e20 0x7f3b50197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b561fe640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3b50110cd0 con 0x7f3b50102570 2024-09-19T06:08:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b561fe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b50110fd0 con 0x7f3b50102ef0 2024-09-19T06:08:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50100e20 0x7f3b50197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55186/0 (socket says 172.21.15.5:55186) 2024-09-19T06:08:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/800005856 learned_addr learned my addr 172.21.15.5:0/800005856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b47fff640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b50102570 0x7f3b50197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.795+0000 7f3b4ffff640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b50102ef0 0x7f3b5019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.796+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/800005856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b50102ef0 msgr2=0x7f3b5019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.796+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b50102ef0 0x7f3b5019bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.796+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/800005856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b50102570 msgr2=0x7f3b50197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.796+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b50102570 0x7f3b50197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.796+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/800005856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5019c2e0 con 0x7f3b50100e20 2024-09-19T06:08:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.796+0000 7f3b47fff640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b50102570 0x7f3b50197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.796+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50100e20 0x7f3b50197330 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e9c0 tx=0x7f3b4000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.796+0000 7f3b4d7fa640 1 -- 172.21.15.5:0/800005856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4000cde0 con 0x7f3b50100e20 2024-09-19T06:08:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.796+0000 7f3b561fe640 1 -- 172.21.15.5:0/800005856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5019c5d0 con 0x7f3b50100e20 2024-09-19T06:08:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.796+0000 7f3b4d7fa640 1 -- 172.21.15.5:0/800005856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b400035f0 con 0x7f3b50100e20 2024-09-19T06:08:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.796+0000 7f3b4d7fa640 1 -- 172.21.15.5:0/800005856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40010690 con 0x7f3b50100e20 2024-09-19T06:08:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.796+0000 7f3b561fe640 1 -- 172.21.15.5:0/800005856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b501a3eb0 con 0x7f3b50100e20 2024-09-19T06:08:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.798+0000 7f3b4d7fa640 1 -- 172.21.15.5:0/800005856 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3b400037a0 con 0x7f3b50100e20 2024-09-19T06:08:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.799+0000 7f3b585d7740 1 Processor -- start 2024-09-19T06:08:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.799+0000 7f3b585d7740 1 -- start start 2024-09-19T06:08:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.799+0000 7f3b4d7fa640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3b3c0a33b0 0x7f3b3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.799+0000 7f3b585d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1360 0x55d9c75d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.799+0000 7f3b585d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1c80 0x55d9c75de350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.799+0000 7f3b585d7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75e0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.799+0000 7f3b585d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d9c75372e0 con 0x55d9c75d1c80 2024-09-19T06:08:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.799+0000 7f3b585d7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d9c74dece0 con 0x55d9c75de890 2024-09-19T06:08:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.799+0000 7f3b585d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d9c75332a0 con 0x55d9c75d1360 2024-09-19T06:08:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.799+0000 7f3b4ffff640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3b3c0a33b0 0x7f3b3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.799+0000 7f3b4d7fa640 1 -- 172.21.15.5:0/800005856 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f3b400d0090 con 0x7f3b50100e20 2024-09-19T06:08:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b4ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75e0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b4f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1360 0x55d9c75d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b4f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1360 0x55d9c75d1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53138/0 (socket says 172.21.15.5:53138) 2024-09-19T06:08:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/3391398635 learned_addr learned my addr 172.21.15.5:0/3391398635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/3391398635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 msgr2=0x55d9c75e0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/3391398635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75e0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/3391398635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1c80 msgr2=0x55d9c75de350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:16.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b47fff640 1 --2- 172.21.15.5:0/3391398635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1c80 0x55d9c75de350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b4ffff640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3b3c0a33b0 0x7f3b3c0a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3b48005fd0 tx=0x7f3b48009b50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:16.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/3391398635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1c80 0x55d9c75de350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/3391398635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c74c3440 con 0x55d9c75d1360 2024-09-19T06:08:16.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b47fff640 1 --2- 172.21.15.5:0/3391398635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1c80 0x55d9c75de350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:08:16.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/3391398635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1360 0x55d9c75d1740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3b400a4ac0 tx=0x7f3b4009c280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=982f4f0d33232a0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:16.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b45ffb640 1 -- 172.21.15.5:0/3391398635 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4009ca80 con 0x55d9c75d1360 2024-09-19T06:08:16.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b45ffb640 1 -- 172.21.15.5:0/3391398635 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b4009c380 con 0x55d9c75d1360 2024-09-19T06:08:16.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.800+0000 7f3b45ffb640 1 -- 172.21.15.5:0/3391398635 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b400cda70 con 0x55d9c75d1360 2024-09-19T06:08:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.801+0000 7f3b585d7740 1 -- 172.21.15.5:0/3391398635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1360 msgr2=0x55d9c75d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.801+0000 7f3b585d7740 1 --2- 172.21.15.5:0/3391398635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1360 0x55d9c75d1740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3b400a4ac0 tx=0x7f3b4009c280 comp rx=0 tx=0).stop 2024-09-19T06:08:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.801+0000 7f3b585d7740 1 -- 172.21.15.5:0/3391398635 shutdown_connections 2024-09-19T06:08:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.801+0000 7f3b585d7740 1 --2- 172.21.15.5:0/3391398635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75e0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.801+0000 7f3b585d7740 1 --2- 172.21.15.5:0/3391398635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1c80 0x55d9c75de350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.801+0000 7f3b585d7740 1 --2- 172.21.15.5:0/3391398635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1360 0x55d9c75d1740 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.801+0000 7f3b585d7740 1 -- 172.21.15.5:0/3391398635 >> 172.21.15.5:0/3391398635 conn(0x55d9c75cda30 msgr2=0x55d9c75cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.801+0000 7f3b585d7740 1 -- 172.21.15.5:0/3391398635 shutdown_connections 2024-09-19T06:08:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.801+0000 7f3b585d7740 1 -- 172.21.15.5:0/3391398635 wait complete. 2024-09-19T06:08:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.802+0000 7f3b585d7740 1 -- start start 2024-09-19T06:08:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.802+0000 7f3b585d7740 1 Processor -- start 2024-09-19T06:08:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.802+0000 7f3b585d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 0x55d9c75de2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.802+0000 7f3b585d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75db550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.802+0000 7f3b585d7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75dbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.802+0000 7f3b585d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d9c75e1460 con 0x55d9c75d1360 2024-09-19T06:08:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.802+0000 7f3b585d7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d9c74f2050 con 0x55d9c75de890 2024-09-19T06:08:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.802+0000 7f3b585d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d9c75e15e0 con 0x55d9c75d1c80 2024-09-19T06:08:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 0x55d9c75de2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.802+0000 7f3b585d7740 10 client.0 _ll_register_callbacks cb 0x55d9c7532160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 0x55d9c75de2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55206/0 (socket says 172.21.15.5:55206) 2024-09-19T06:08:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/446839445 learned_addr learned my addr 172.21.15.5:0/446839445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b47fff640 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75db550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b4ffff640 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75dbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/446839445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 msgr2=0x55d9c75db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/446839445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 msgr2=0x55d9c75dbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75dbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/446839445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c72a83c0 con 0x55d9c75d1360 2024-09-19T06:08:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 0x55d9c75de2a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3b40098d50 tx=0x7f3b4009a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9a0953754fe27c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b2effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b2effd640 1 -- 172.21.15.5:0/446839445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b400a0730 con 0x55d9c75d1360 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b2effd640 1 -- 172.21.15.5:0/446839445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f3b40086e00 con 0x55d9c75d1360 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b585d7740 1 -- 172.21.15.5:0/446839445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b4009c380 con 0x55d9c75d1360 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b2effd640 1 -- 172.21.15.5:0/446839445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3b400a4ff0 con 0x55d9c75d1360 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b585d7740 10 client.16689 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.803+0000 7f3b585d7740 1 -- 172.21.15.5:0/446839445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d9c75ecea0 con 0x55d9c75d1360 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.804+0000 7f3b2effd640 1 -- 172.21.15.5:0/446839445 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b4009c380 con 0x55d9c75d1360 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.804+0000 7f3b2effd640 1 -- 172.21.15.5:0/446839445 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3b4008d020 con 0x55d9c75d1360 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.804+0000 7f3b585d7740 10 client.16689 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.804+0000 7f3b585d7740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.804+0000 7f3b585d7740 20 client.16689 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.804+0000 7f3b585d7740 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9c75ee1f0 0x55d9c75f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.804+0000 7f3b585d7740 4 client.16689 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.804+0000 7f3b585d7740 1 -- 172.21.15.5:0/446839445 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9c7544850 con 0x55d9c75ee1f0 2024-09-19T06:08:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.804+0000 7f3b47fff640 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9c75ee1f0 0x55d9c75f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.805+0000 7f3b2effd640 10 client.16689 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.805+0000 7f3b47fff640 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9c75ee1f0 0x55d9c75f05e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e9ccd8b96c51d server_cookie=835dfb6390d0e78e in_seq=0 out_seq=0 2024-09-19T06:08:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.805+0000 7f3b2effd640 1 -- 172.21.15.5:0/446839445 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9c7544850 con 0x55d9c75ee1f0 2024-09-19T06:08:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.805+0000 7f3b2effd640 10 client.16689 handle_command_reply: tid=0 2024-09-19T06:08:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.805+0000 7f3b585d7740 1 client.16689 shutdown 2024-09-19T06:08:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.806+0000 7f3b585d7740 1 -- 172.21.15.5:0/446839445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 msgr2=0x55d9c75de2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.806+0000 7f3b585d7740 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 0x55d9c75de2a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3b40098d50 tx=0x7f3b4009a890 comp rx=0 tx=0).stop 2024-09-19T06:08:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.806+0000 7f3b585d7740 1 -- 172.21.15.5:0/446839445 shutdown_connections 2024-09-19T06:08:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.806+0000 7f3b585d7740 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9c75ee1f0 0x55d9c75f05e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.806+0000 7f3b585d7740 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75dbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.806+0000 7f3b585d7740 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75db550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.806+0000 7f3b585d7740 1 --2- 172.21.15.5:0/446839445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 0x55d9c75de2a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.806+0000 7f3b585d7740 1 -- 172.21.15.5:0/446839445 >> 172.21.15.5:0/446839445 conn(0x55d9c75cda30 msgr2=0x55d9c75d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.806+0000 7f3b2effd640 0 client.16689 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.806+0000 7f3b585d7740 1 -- 172.21.15.5:0/446839445 shutdown_connections 2024-09-19T06:08:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.806+0000 7f3b585d7740 1 -- 172.21.15.5:0/446839445 wait complete. 2024-09-19T06:08:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.806+0000 7f3b585d7740 20 client.16689 trim_cache size 0 max 16384 2024-09-19T06:08:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.829+0000 7f3b585d7740 1 Processor -- start 2024-09-19T06:08:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.829+0000 7f3b585d7740 1 -- start start 2024-09-19T06:08:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.829+0000 7f3b585d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 0x55d9c75ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.829+0000 7f3b585d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75d9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.829+0000 7f3b585d7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.829+0000 7f3b585d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3b501ab620 con 0x55d9c75d1360 2024-09-19T06:08:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.829+0000 7f3b585d7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3b501ad160 con 0x55d9c75de890 2024-09-19T06:08:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.829+0000 7f3b585d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b501abcf0 con 0x55d9c75d1c80 2024-09-19T06:08:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.829+0000 7f3b4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 0x55d9c75ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b4ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b4ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:34874/0 (socket says 172.21.15.5:34874) 2024-09-19T06:08:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b47fff640 1 --2- 172.21.15.5:0/2738133783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75d9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b4ffff640 1 -- 172.21.15.5:0/2738133783 learned_addr learned my addr 172.21.15.5:0/2738133783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/2738133783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 msgr2=0x55d9c75d9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/2738133783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75d9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/2738133783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 msgr2=0x55d9c75cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/2738133783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b4f7fe640 1 -- 172.21.15.5:0/2738133783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c75ceb50 con 0x55d9c75d1360 2024-09-19T06:08:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b47fff640 1 --2- 172.21.15.5:0/2738133783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75d9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/2738133783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 0x55d9c75ea880 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3b400987e0 tx=0x7f3b40098620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f355559081777901 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b457fa640 1 -- 172.21.15.5:0/2738133783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40003b10 con 0x55d9c75d1360 2024-09-19T06:08:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b457fa640 1 -- 172.21.15.5:0/2738133783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b400a0730 con 0x55d9c75d1360 2024-09-19T06:08:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.830+0000 7f3b457fa640 1 -- 172.21.15.5:0/2738133783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b400980d0 con 0x55d9c75d1360 2024-09-19T06:08:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.831+0000 7f3b585d7740 1 -- 172.21.15.5:0/2738133783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 msgr2=0x55d9c75ea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.831+0000 7f3b585d7740 1 --2- 172.21.15.5:0/2738133783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 0x55d9c75ea880 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3b400987e0 tx=0x7f3b40098620 comp rx=0 tx=0).stop 2024-09-19T06:08:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.831+0000 7f3b585d7740 1 -- 172.21.15.5:0/2738133783 shutdown_connections 2024-09-19T06:08:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.831+0000 7f3b585d7740 1 --2- 172.21.15.5:0/2738133783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75de890 0x55d9c75cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.831+0000 7f3b585d7740 1 --2- 172.21.15.5:0/2738133783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75d9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.831+0000 7f3b585d7740 1 --2- 172.21.15.5:0/2738133783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75d1360 0x55d9c75ea880 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.831+0000 7f3b585d7740 1 -- 172.21.15.5:0/2738133783 >> 172.21.15.5:0/2738133783 conn(0x55d9c75cda30 msgr2=0x55d9c75cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.831+0000 7f3b585d7740 1 -- 172.21.15.5:0/2738133783 shutdown_connections 2024-09-19T06:08:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.831+0000 7f3b585d7740 1 -- 172.21.15.5:0/2738133783 wait complete. 2024-09-19T06:08:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.832+0000 7f3b585d7740 1 -- start start 2024-09-19T06:08:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.832+0000 7f3b585d7740 1 Processor -- start 2024-09-19T06:08:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.832+0000 7f3b585d7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75d1360 0x55d9c75f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.832+0000 7f3b585d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.832+0000 7f3b585d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75de890 0x55d9c75f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.832+0000 7f3b585d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3b501aca90 con 0x55d9c75de890 2024-09-19T06:08:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.832+0000 7f3b585d7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3b501ac3c0 con 0x55d9c75d1360 2024-09-19T06:08:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b585d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b501a9600 con 0x55d9c75d1c80 2024-09-19T06:08:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75de890 0x55d9c75f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b47fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b585d7740 10 client.0 _ll_register_callbacks cb 0x7f3b48009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b47fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53164/0 (socket says 172.21.15.5:53164) 2024-09-19T06:08:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75de890 0x55d9c75f1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55226/0 (socket says 172.21.15.5:55226) 2024-09-19T06:08:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b47fff640 1 -- 172.21.15.5:0/2110418861 learned_addr learned my addr 172.21.15.5:0/2110418861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75d1360 0x55d9c75f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b4ffff640 1 -- 172.21.15.5:0/2110418861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 msgr2=0x55d9c75f16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b4ffff640 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75f16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b4ffff640 1 -- 172.21.15.5:0/2110418861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75d1360 msgr2=0x55d9c75f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b4ffff640 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75d1360 0x55d9c75f1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b4ffff640 1 -- 172.21.15.5:0/2110418861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c7657d30 con 0x55d9c75de890 2024-09-19T06:08:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b47fff640 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b4ffff640 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75de890 0x55d9c75f1be0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009fd0 tx=0x7f3b48004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4595e9c053f41f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b2effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b2effd640 1 -- 172.21.15.5:0/2110418861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4802f060 con 0x55d9c75de890 2024-09-19T06:08:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b2effd640 1 -- 172.21.15.5:0/2110418861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f3b48020440 con 0x55d9c75de890 2024-09-19T06:08:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.833+0000 7f3b585d7740 1 -- 172.21.15.5:0/2110418861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b400a0730 con 0x55d9c75de890 2024-09-19T06:08:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.834+0000 7f3b2effd640 1 -- 172.21.15.5:0/2110418861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3b48020640 con 0x55d9c75de890 2024-09-19T06:08:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.834+0000 7f3b585d7740 10 client.16695 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.834+0000 7f3b585d7740 1 -- 172.21.15.5:0/2110418861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d9c75d4600 con 0x55d9c75de890 2024-09-19T06:08:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.834+0000 7f3b2effd640 1 -- 172.21.15.5:0/2110418861 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b400a0730 con 0x55d9c75de890 2024-09-19T06:08:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.834+0000 7f3b2effd640 1 -- 172.21.15.5:0/2110418861 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3b48029070 con 0x55d9c75de890 2024-09-19T06:08:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.834+0000 7f3b585d7740 10 client.16695 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.834+0000 7f3b585d7740 10 client.16695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.834+0000 7f3b585d7740 20 client.16695 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.834+0000 7f3b585d7740 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9c75d4d50 0x55d9c75d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.834+0000 7f3b585d7740 4 client.16695 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:08:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.834+0000 7f3b585d7740 1 -- 172.21.15.5:0/2110418861 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9c753ed60 con 0x55d9c75d4d50 2024-09-19T06:08:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.834+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9c75d4d50 0x55d9c75d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.835+0000 7f3b4f7fe640 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9c75d4d50 0x55d9c75d5160 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870beffba5436744 server_cookie=b9774c99a8b1bdf3 in_seq=0 out_seq=0 2024-09-19T06:08:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.835+0000 7f3b2effd640 10 client.16695 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.835+0000 7f3b2effd640 1 -- 172.21.15.5:0/2110418861 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d9c753ed60 con 0x55d9c75d4d50 2024-09-19T06:08:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.835+0000 7f3b2effd640 10 client.16695 handle_command_reply: tid=0 2024-09-19T06:08:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.835+0000 7f3b585d7740 1 client.16695 shutdown 2024-09-19T06:08:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.835+0000 7f3b585d7740 1 -- 172.21.15.5:0/2110418861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75de890 msgr2=0x55d9c75f1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.835+0000 7f3b585d7740 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75de890 0x55d9c75f1be0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009fd0 tx=0x7f3b48004ab0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.835+0000 7f3b585d7740 1 -- 172.21.15.5:0/2110418861 shutdown_connections 2024-09-19T06:08:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.835+0000 7f3b585d7740 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d9c75d4d50 0x55d9c75d5160 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.835+0000 7f3b585d7740 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9c75de890 0x55d9c75f1be0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.835+0000 7f3b585d7740 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9c75d1c80 0x55d9c75f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.835+0000 7f3b585d7740 1 --2- 172.21.15.5:0/2110418861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9c75d1360 0x55d9c75f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b585d7740 1 -- 172.21.15.5:0/2110418861 >> 172.21.15.5:0/2110418861 conn(0x55d9c75cda30 msgr2=0x55d9c75ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:16.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b2effd640 0 client.16695 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:16.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b585d7740 1 -- 172.21.15.5:0/2110418861 shutdown_connections 2024-09-19T06:08:16.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b585d7740 1 -- 172.21.15.5:0/2110418861 wait complete. 2024-09-19T06:08:16.876 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:08:16.876 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:08:16.876 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:08:16.876 INFO:teuthology.orchestra.run.smithi005.stdout: "6d16f2de-e850-4ccd-9ed9-a9ba25c49a11": { 2024-09-19T06:08:16.876 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:08:16.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "6d16f2de-e850-4ccd-9ed9-a9ba25c49a11", 2024-09-19T06:08:16.877 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:08:16.877 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:08:16.877 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:08:16.877 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:08:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b585d7740 20 client.16695 trim_cache size 0 max 16384 2024-09-19T06:08:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b561fe640 1 -- 172.21.15.5:0/800005856 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3b3c0a33b0 msgr2=0x7f3b3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b561fe640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3b3c0a33b0 0x7f3b3c0a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3b48005fd0 tx=0x7f3b48009b50 comp rx=0 tx=0).stop 2024-09-19T06:08:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b561fe640 1 -- 172.21.15.5:0/800005856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50100e20 msgr2=0x7f3b50197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b561fe640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50100e20 0x7f3b50197330 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e9c0 tx=0x7f3b4000ee80 comp rx=0 tx=0).stop 2024-09-19T06:08:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b561fe640 1 -- 172.21.15.5:0/800005856 shutdown_connections 2024-09-19T06:08:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b561fe640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3b3c0a33b0 0x7f3b3c0a5870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b561fe640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b50102ef0 0x7f3b5019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b561fe640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b50102570 0x7f3b50197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.836+0000 7f3b561fe640 1 --2- 172.21.15.5:0/800005856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50100e20 0x7f3b50197330 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.837+0000 7f3b561fe640 1 -- 172.21.15.5:0/800005856 >> 172.21.15.5:0/800005856 conn(0x7f3b500fc7d0 msgr2=0x7f3b50104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.837+0000 7f3b561fe640 1 -- 172.21.15.5:0/800005856 shutdown_connections 2024-09-19T06:08:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:16.837+0000 7f3b561fe640 1 -- 172.21.15.5:0/800005856 wait complete. 2024-09-19T06:08:17.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:16 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:08:17.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:16 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:17.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:16 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:08:17.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:16 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:17.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:16 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:08:17.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:16 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:17.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:16 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:08:17.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:16 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:17.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:16 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:08:17.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:16 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:17.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:16 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:08:17.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:16 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:08:17.694 INFO:tasks.cephfs.filesystem:scrub status for tag:6d16f2de-e850-4ccd-9ed9-a9ba25c49a11 - {'path': '/', 'tag': '6d16f2de-e850-4ccd-9ed9-a9ba25c49a11', 'options': 'recursive,force'} 2024-09-19T06:08:17.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 115640 2.77 MB/sec execute 103 sec latency 20.968 ms 2024-09-19T06:08:18.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:17 smithi005 ceph-mon[30955]: pgmap v152: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-09-19T06:08:18.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:17 smithi049 ceph-mon[34640]: pgmap v152: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-09-19T06:08:18.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:17 smithi157 ceph-mon[35346]: pgmap v152: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-09-19T06:08:18.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 116147 2.77 MB/sec execute 104 sec latency 17.106 ms 2024-09-19T06:08:19.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 116858 2.80 MB/sec execute 105 sec latency 16.942 ms 2024-09-19T06:08:20.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:19 smithi005 ceph-mon[30955]: pgmap v153: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-19T06:08:20.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:19 smithi049 ceph-mon[34640]: pgmap v153: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-19T06:08:20.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:19 smithi157 ceph-mon[35346]: pgmap v153: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-19T06:08:20.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 117327 2.82 MB/sec execute 106 sec latency 14.650 ms 2024-09-19T06:08:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.310+0000 7f336f10a640 1 -- 172.21.15.5:0/1077829791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3368101380 msgr2=0x7f336810d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.310+0000 7f336f10a640 1 --2- 172.21.15.5:0/1077829791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3368101380 0x7f336810d8b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3360009cd0 tx=0x7f336002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.310+0000 7f336f10a640 1 -- 172.21.15.5:0/1077829791 shutdown_connections 2024-09-19T06:08:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.310+0000 7f336f10a640 1 --2- 172.21.15.5:0/1077829791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3368101380 0x7f336810d8b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.310+0000 7f336f10a640 1 --2- 172.21.15.5:0/1077829791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33681009e0 0x7f3368100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.310+0000 7f336f10a640 1 --2- 172.21.15.5:0/1077829791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33681069f0 0x7f3368106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.310+0000 7f336f10a640 1 -- 172.21.15.5:0/1077829791 >> 172.21.15.5:0/1077829791 conn(0x7f33680fc720 msgr2=0x7f33680feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.310+0000 7f336f10a640 1 -- 172.21.15.5:0/1077829791 shutdown_connections 2024-09-19T06:08:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.310+0000 7f336f10a640 1 -- 172.21.15.5:0/1077829791 wait complete. 2024-09-19T06:08:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336f10a640 1 Processor -- start 2024-09-19T06:08:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336f10a640 1 -- start start 2024-09-19T06:08:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336f10a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33681009e0 0x7f3368198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336f10a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3368101380 0x7f33681988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336f10a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33681069f0 0x7f336819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336f10a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3368110ce0 con 0x7f33681009e0 2024-09-19T06:08:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336f10a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3368110b60 con 0x7f33681069f0 2024-09-19T06:08:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336f10a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3368110e60 con 0x7f3368101380 2024-09-19T06:08:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336cea6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33681009e0 0x7f3368198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336cea6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33681009e0 0x7f3368198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55240/0 (socket says 172.21.15.5:55240) 2024-09-19T06:08:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336cea6640 1 -- 172.21.15.5:0/2123323775 learned_addr learned my addr 172.21.15.5:0/2123323775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336d6a7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33681069f0 0x7f336819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f3367fff640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3368101380 0x7f33681988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336cea6640 1 -- 172.21.15.5:0/2123323775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3368101380 msgr2=0x7f33681988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336cea6640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3368101380 0x7f33681988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336cea6640 1 -- 172.21.15.5:0/2123323775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33681069f0 msgr2=0x7f336819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336cea6640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33681069f0 0x7f336819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336cea6640 1 -- 172.21.15.5:0/2123323775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336819d380 con 0x7f33681009e0 2024-09-19T06:08:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336d6a7640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33681069f0 0x7f336819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.311+0000 7f336cea6640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33681009e0 0x7f3368198390 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f335c00e9f0 tx=0x7f335c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.312+0000 7f3365ffb640 1 -- 172.21.15.5:0/2123323775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c00cd80 con 0x7f33681009e0 2024-09-19T06:08:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.312+0000 7f3365ffb640 1 -- 172.21.15.5:0/2123323775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f335c0054d0 con 0x7f33681009e0 2024-09-19T06:08:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.312+0000 7f336f10a640 1 -- 172.21.15.5:0/2123323775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336819d670 con 0x7f33681009e0 2024-09-19T06:08:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.312+0000 7f3365ffb640 1 -- 172.21.15.5:0/2123323775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c010490 con 0x7f33681009e0 2024-09-19T06:08:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.312+0000 7f336f10a640 1 -- 172.21.15.5:0/2123323775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33681a4f00 con 0x7f33681009e0 2024-09-19T06:08:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.315+0000 7f3365ffb640 1 -- 172.21.15.5:0/2123323775 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f335c005020 con 0x7f33681009e0 2024-09-19T06:08:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.315+0000 7f3371ea1740 1 Processor -- start 2024-09-19T06:08:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.315+0000 7f3371ea1740 1 -- start start 2024-09-19T06:08:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.315+0000 7f3371ea1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f021ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.315+0000 7f3371ea1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 0x55bf9f02c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.315+0000 7f3371ea1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f02ccb0 0x55bf9f02f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.315+0000 7f3371ea1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bf9ef862f0 con 0x55bf9f022410 2024-09-19T06:08:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.315+0000 7f3371ea1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bf9ef2e270 con 0x55bf9f02ccb0 2024-09-19T06:08:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.315+0000 7f3371ea1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bf9ef822b0 con 0x55bf9f021af0 2024-09-19T06:08:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f336cea6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 0x55bf9f02c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f336d6a7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f021ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f3367fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f02ccb0 0x55bf9f02f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f336d6a7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f021ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53184/0 (socket says 172.21.15.5:53184) 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f336cea6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 0x55bf9f02c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55248/0 (socket says 172.21.15.5:55248) 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f3367fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f02ccb0 0x55bf9f02f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:34902/0 (socket says 172.21.15.5:34902) 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f336d6a7640 1 -- 172.21.15.5:0/3647046695 learned_addr learned my addr 172.21.15.5:0/3647046695 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f3365ffb640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f33580a33b0 0x7f33580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f336d6a7640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f33580a33b0 0x7f33580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f3367fff640 1 -- 172.21.15.5:0/3647046695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 msgr2=0x55bf9f021ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f3367fff640 1 --2- 172.21.15.5:0/3647046695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f021ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f3365ffb640 1 -- 172.21.15.5:0/2123323775 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f335c0cffd0 con 0x7f33681009e0 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f3367fff640 1 -- 172.21.15.5:0/3647046695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 msgr2=0x55bf9f02c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f3367fff640 1 --2- 172.21.15.5:0/3647046695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 0x55bf9f02c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f3367fff640 1 -- 172.21.15.5:0/3647046695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf9ef2d340 con 0x55bf9f02ccb0 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f336cea6640 1 --2- 172.21.15.5:0/3647046695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 0x55bf9f02c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f3367fff640 1 --2- 172.21.15.5:0/3647046695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f02ccb0 0x55bf9f02f0a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3354009bf0 tx=0x7f3354011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f46a8d4fad205453 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f336d6a7640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f33580a33b0 0x7f33580a5870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3360004af0 tx=0x7f336002a8e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.316+0000 7f3352ffd640 1 -- 172.21.15.5:0/3647046695 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354004050 con 0x55bf9f02ccb0 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.317+0000 7f3352ffd640 1 -- 172.21.15.5:0/3647046695 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33540054d0 con 0x55bf9f02ccb0 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.317+0000 7f3352ffd640 1 -- 172.21.15.5:0/3647046695 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354004d60 con 0x55bf9f02ccb0 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.317+0000 7f3371ea1740 1 -- 172.21.15.5:0/3647046695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f02ccb0 msgr2=0x55bf9f02f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.317+0000 7f3371ea1740 1 --2- 172.21.15.5:0/3647046695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f02ccb0 0x55bf9f02f0a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3354009bf0 tx=0x7f3354011810 comp rx=0 tx=0).stop 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.317+0000 7f3371ea1740 1 -- 172.21.15.5:0/3647046695 shutdown_connections 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.317+0000 7f3371ea1740 1 --2- 172.21.15.5:0/3647046695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f02ccb0 0x55bf9f02f0a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.318+0000 7f3371ea1740 1 --2- 172.21.15.5:0/3647046695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 0x55bf9f02c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.318+0000 7f3371ea1740 1 --2- 172.21.15.5:0/3647046695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f021ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.318+0000 7f3371ea1740 1 -- 172.21.15.5:0/3647046695 >> 172.21.15.5:0/3647046695 conn(0x55bf9f01e060 msgr2=0x55bf9f01e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.318+0000 7f3371ea1740 1 -- 172.21.15.5:0/3647046695 shutdown_connections 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.318+0000 7f3371ea1740 1 -- 172.21.15.5:0/3647046695 wait complete. 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.318+0000 7f3371ea1740 1 -- start start 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.318+0000 7f3371ea1740 1 Processor -- start 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.318+0000 7f3371ea1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f029b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.318+0000 7f3371ea1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f022410 0x55bf9f02a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.318+0000 7f3371ea1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f02ccb0 0x55bf9f02a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.318+0000 7f3371ea1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bf9f02f880 con 0x55bf9f02ccb0 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f3371ea1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bf9ef0f290 con 0x55bf9f022410 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f3371ea1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bf9f02fa00 con 0x55bf9f021af0 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f3371ea1740 10 client.0 _ll_register_callbacks cb 0x55bf9ef40140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f336cea6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f029b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f336cea6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f029b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53196/0 (socket says 172.21.15.5:53196) 2024-09-19T06:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f336cea6640 1 -- 172.21.15.5:0/4179262858 learned_addr learned my addr 172.21.15.5:0/4179262858 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f336d6a7640 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f02ccb0 0x55bf9f02a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f3367fff640 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f022410 0x55bf9f02a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f336cea6640 1 -- 172.21.15.5:0/4179262858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f022410 msgr2=0x55bf9f02a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f336cea6640 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f022410 0x55bf9f02a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f336cea6640 1 -- 172.21.15.5:0/4179262858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f02ccb0 msgr2=0x55bf9f02a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f336cea6640 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f02ccb0 0x55bf9f02a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f336cea6640 1 -- 172.21.15.5:0/4179262858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf9ec99210 con 0x55bf9f021af0 2024-09-19T06:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f336cea6640 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f029b00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f335c00c6a0 tx=0x7f335c0d3630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60f7509915bd42ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f3347fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f3347fff640 1 -- 172.21.15.5:0/4179262858 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c09c090 con 0x55bf9f021af0 2024-09-19T06:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f3347fff640 1 -- 172.21.15.5:0/4179262858 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f335c0d20d0 con 0x55bf9f021af0 2024-09-19T06:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.319+0000 7f3371ea1740 1 -- 172.21.15.5:0/4179262858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33540054d0 con 0x55bf9f021af0 2024-09-19T06:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.320+0000 7f3347fff640 1 -- 172.21.15.5:0/4179262858 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f335c005310 con 0x55bf9f021af0 2024-09-19T06:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.320+0000 7f3371ea1740 10 client.26218 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.320+0000 7f3371ea1740 1 -- 172.21.15.5:0/4179262858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bf9ed65620 con 0x55bf9f021af0 2024-09-19T06:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.320+0000 7f3347fff640 1 -- 172.21.15.5:0/4179262858 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33540054d0 con 0x55bf9f021af0 2024-09-19T06:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.320+0000 7f3347fff640 1 -- 172.21.15.5:0/4179262858 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f335c0a4aa0 con 0x55bf9f021af0 2024-09-19T06:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.320+0000 7f3371ea1740 10 client.26218 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.320+0000 7f3371ea1740 10 client.26218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.320+0000 7f3371ea1740 20 client.26218 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.320+0000 7f3371ea1740 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bf9f03cbd0 0x55bf9f03f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.320+0000 7f3371ea1740 4 client.26218 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.320+0000 7f3371ea1740 1 -- 172.21.15.5:0/4179262858 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf9ef93a20 con 0x55bf9f03cbd0 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.320+0000 7f3367fff640 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bf9f03cbd0 0x55bf9f03f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.321+0000 7f3367fff640 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bf9f03cbd0 0x55bf9f03f000 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11b320ebc7d3a42 server_cookie=aeafb6626921e059 in_seq=0 out_seq=0 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.321+0000 7f3347fff640 10 client.26218 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.321+0000 7f3347fff640 1 -- 172.21.15.5:0/4179262858 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf9ef93a20 con 0x55bf9f03cbd0 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.321+0000 7f3347fff640 10 client.26218 handle_command_reply: tid=0 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.321+0000 7f3371ea1740 1 client.26218 shutdown 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.322+0000 7f3371ea1740 1 -- 172.21.15.5:0/4179262858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 msgr2=0x55bf9f029b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.322+0000 7f3371ea1740 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f029b00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f335c00c6a0 tx=0x7f335c0d3630 comp rx=0 tx=0).stop 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.322+0000 7f3371ea1740 1 -- 172.21.15.5:0/4179262858 shutdown_connections 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.322+0000 7f3371ea1740 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bf9f03cbd0 0x55bf9f03f000 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.322+0000 7f3371ea1740 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f02ccb0 0x55bf9f02a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.322+0000 7f3371ea1740 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f022410 0x55bf9f02a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.322+0000 7f3371ea1740 1 --2- 172.21.15.5:0/4179262858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f029b00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.322+0000 7f3347fff640 0 client.26218 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.322+0000 7f3371ea1740 1 -- 172.21.15.5:0/4179262858 >> 172.21.15.5:0/4179262858 conn(0x55bf9f01e060 msgr2=0x55bf9f023dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.322+0000 7f3371ea1740 1 -- 172.21.15.5:0/4179262858 shutdown_connections 2024-09-19T06:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.322+0000 7f3371ea1740 1 -- 172.21.15.5:0/4179262858 wait complete. 2024-09-19T06:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.322+0000 7f3371ea1740 20 client.26218 trim_cache size 0 max 16384 2024-09-19T06:08:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.338+0000 7f3371ea1740 1 Processor -- start 2024-09-19T06:08:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.338+0000 7f3371ea1740 1 -- start start 2024-09-19T06:08:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.338+0000 7f3371ea1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f02ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.338+0000 7f3371ea1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 0x55bf9f02a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.338+0000 7f3371ea1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f02ccb0 0x55bf9f044930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.338+0000 7f3371ea1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33681ad7b0 con 0x55bf9f022410 2024-09-19T06:08:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.338+0000 7f3371ea1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33681af2f0 con 0x55bf9f02ccb0 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.338+0000 7f3371ea1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33681ade80 con 0x55bf9f021af0 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f3367fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 0x55bf9f02a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f336d6a7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f02ccb0 0x55bf9f044930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f3367fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 0x55bf9f02a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55272/0 (socket says 172.21.15.5:55272) 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f3367fff640 1 -- 172.21.15.5:0/311122946 learned_addr learned my addr 172.21.15.5:0/311122946 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f336cea6640 1 --2- 172.21.15.5:0/311122946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f02ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f3367fff640 1 -- 172.21.15.5:0/311122946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 msgr2=0x55bf9f02ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f3367fff640 1 --2- 172.21.15.5:0/311122946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f02ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f3367fff640 1 -- 172.21.15.5:0/311122946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f02ccb0 msgr2=0x55bf9f044930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f3367fff640 1 --2- 172.21.15.5:0/311122946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f02ccb0 0x55bf9f044930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f3367fff640 1 -- 172.21.15.5:0/311122946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf9f01de90 con 0x55bf9f022410 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f336cea6640 1 --2- 172.21.15.5:0/311122946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f02ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f3367fff640 1 --2- 172.21.15.5:0/311122946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 0x55bf9f02a380 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3354009730 tx=0x7f33540096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea1047f7fdc62ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f33527fc640 1 -- 172.21.15.5:0/311122946 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354034070 con 0x55bf9f022410 2024-09-19T06:08:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f33527fc640 1 -- 172.21.15.5:0/311122946 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3354004500 con 0x55bf9f022410 2024-09-19T06:08:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.339+0000 7f33527fc640 1 -- 172.21.15.5:0/311122946 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354003df0 con 0x55bf9f022410 2024-09-19T06:08:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.340+0000 7f3371ea1740 1 -- 172.21.15.5:0/311122946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 msgr2=0x55bf9f02a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.340+0000 7f3371ea1740 1 --2- 172.21.15.5:0/311122946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 0x55bf9f02a380 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3354009730 tx=0x7f33540096c0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.340+0000 7f3371ea1740 1 -- 172.21.15.5:0/311122946 shutdown_connections 2024-09-19T06:08:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.340+0000 7f3371ea1740 1 --2- 172.21.15.5:0/311122946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f02ccb0 0x55bf9f044930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.340+0000 7f3371ea1740 1 --2- 172.21.15.5:0/311122946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f022410 0x55bf9f02a380 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.340+0000 7f3371ea1740 1 --2- 172.21.15.5:0/311122946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f021af0 0x55bf9f02ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.340+0000 7f3371ea1740 1 -- 172.21.15.5:0/311122946 >> 172.21.15.5:0/311122946 conn(0x55bf9f01e060 msgr2=0x55bf9f02b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.340+0000 7f3371ea1740 1 -- 172.21.15.5:0/311122946 shutdown_connections 2024-09-19T06:08:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.340+0000 7f3371ea1740 1 -- 172.21.15.5:0/311122946 wait complete. 2024-09-19T06:08:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.340+0000 7f3371ea1740 1 -- start start 2024-09-19T06:08:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.340+0000 7f3371ea1740 1 Processor -- start 2024-09-19T06:08:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f3371ea1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f021af0 0x55bf9f035500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f3371ea1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f022410 0x55bf9f030820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f3371ea1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f02ccb0 0x55bf9f030e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f3371ea1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33681aec20 con 0x55bf9f02ccb0 2024-09-19T06:08:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f3371ea1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33681ae550 con 0x55bf9f021af0 2024-09-19T06:08:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f3371ea1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33681ab790 con 0x55bf9f022410 2024-09-19T06:08:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f3371ea1740 10 client.0 _ll_register_callbacks cb 0x7f335c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f336d6a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f02ccb0 0x55bf9f030e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f3367fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f022410 0x55bf9f030820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f336d6a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f02ccb0 0x55bf9f030e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55280/0 (socket says 172.21.15.5:55280) 2024-09-19T06:08:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f336cea6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f021af0 0x55bf9f035500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f336d6a7640 1 -- 172.21.15.5:0/2360068655 learned_addr learned my addr 172.21.15.5:0/2360068655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f336d6a7640 1 -- 172.21.15.5:0/2360068655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f022410 msgr2=0x55bf9f030820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f336d6a7640 1 --2- 172.21.15.5:0/2360068655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f022410 0x55bf9f030820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f336d6a7640 1 -- 172.21.15.5:0/2360068655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f021af0 msgr2=0x55bf9f035500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f336d6a7640 1 --2- 172.21.15.5:0/2360068655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f021af0 0x55bf9f035500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f336d6a7640 1 -- 172.21.15.5:0/2360068655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf9f031640 con 0x55bf9f02ccb0 2024-09-19T06:08:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f3367fff640 1 --2- 172.21.15.5:0/2360068655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f022410 0x55bf9f030820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f336d6a7640 1 --2- 172.21.15.5:0/2360068655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f02ccb0 0x55bf9f030e10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3360035000 tx=0x7f33600095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fd3dcfa9d75d5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.341+0000 7f3347fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3347fff640 1 -- 172.21.15.5:0/2360068655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336005a060 con 0x55bf9f02ccb0 2024-09-19T06:08:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3347fff640 1 -- 172.21.15.5:0/2360068655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f33600305a0 con 0x55bf9f02ccb0 2024-09-19T06:08:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3371ea1740 1 -- 172.21.15.5:0/2360068655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3354004500 con 0x55bf9f02ccb0 2024-09-19T06:08:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3347fff640 1 -- 172.21.15.5:0/2360068655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f33600307a0 con 0x55bf9f02ccb0 2024-09-19T06:08:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3371ea1740 10 client.16713 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3371ea1740 1 -- 172.21.15.5:0/2360068655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bf9f0a7a20 con 0x55bf9f02ccb0 2024-09-19T06:08:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3347fff640 1 -- 172.21.15.5:0/2360068655 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3354004500 con 0x55bf9f02ccb0 2024-09-19T06:08:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3347fff640 1 -- 172.21.15.5:0/2360068655 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3360054070 con 0x55bf9f02ccb0 2024-09-19T06:08:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3371ea1740 10 client.16713 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3371ea1740 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3371ea1740 20 client.16713 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3371ea1740 1 --2- 172.21.15.5:0/2360068655 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bf9f0a81a0 0x55bf9f0aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3371ea1740 4 client.16713 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f3371ea1740 1 -- 172.21.15.5:0/2360068655 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf9ef8dd70 con 0x55bf9f0a81a0 2024-09-19T06:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.342+0000 7f336cea6640 1 --2- 172.21.15.5:0/2360068655 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bf9f0a81a0 0x55bf9f0aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.343+0000 7f336cea6640 1 --2- 172.21.15.5:0/2360068655 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bf9f0a81a0 0x55bf9f0aa5c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e65d9d747ea8f37 server_cookie=914d9824354ac6fe in_seq=0 out_seq=0 2024-09-19T06:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.343+0000 7f3347fff640 10 client.16713 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.360+0000 7f3347fff640 1 -- 172.21.15.5:0/2360068655 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf9ef8dd70 con 0x55bf9f0a81a0 2024-09-19T06:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.360+0000 7f3347fff640 10 client.16713 handle_command_reply: tid=0 2024-09-19T06:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.360+0000 7f3371ea1740 1 client.16713 shutdown 2024-09-19T06:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.361+0000 7f3371ea1740 1 -- 172.21.15.5:0/2360068655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f02ccb0 msgr2=0x55bf9f030e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.361+0000 7f3371ea1740 1 --2- 172.21.15.5:0/2360068655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f02ccb0 0x55bf9f030e10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3360035000 tx=0x7f33600095c0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.361+0000 7f3371ea1740 1 -- 172.21.15.5:0/2360068655 shutdown_connections 2024-09-19T06:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.361+0000 7f3371ea1740 1 --2- 172.21.15.5:0/2360068655 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bf9f0a81a0 0x55bf9f0aa5c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.361+0000 7f3371ea1740 1 --2- 172.21.15.5:0/2360068655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9f02ccb0 0x55bf9f030e10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.361+0000 7f3347fff640 0 client.16713 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.361+0000 7f3371ea1740 1 --2- 172.21.15.5:0/2360068655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf9f022410 0x55bf9f030820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.361+0000 7f3371ea1740 1 --2- 172.21.15.5:0/2360068655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf9f021af0 0x55bf9f035500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.361+0000 7f3371ea1740 1 -- 172.21.15.5:0/2360068655 >> 172.21.15.5:0/2360068655 conn(0x55bf9f01e060 msgr2=0x55bf9f01d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.361+0000 7f3371ea1740 1 -- 172.21.15.5:0/2360068655 shutdown_connections 2024-09-19T06:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.361+0000 7f3371ea1740 1 -- 172.21.15.5:0/2360068655 wait complete. 2024-09-19T06:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.361+0000 7f3371ea1740 20 client.16713 trim_cache size 0 max 16384 2024-09-19T06:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:08:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.362+0000 7f336f10a640 1 -- 172.21.15.5:0/2123323775 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f33580a33b0 msgr2=0x7f33580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.362+0000 7f336f10a640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f33580a33b0 0x7f33580a5870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3360004af0 tx=0x7f336002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.362+0000 7f336f10a640 1 -- 172.21.15.5:0/2123323775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33681009e0 msgr2=0x7f3368198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.362+0000 7f336f10a640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33681009e0 0x7f3368198390 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f335c00e9f0 tx=0x7f335c00edf0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.362+0000 7f336f10a640 1 -- 172.21.15.5:0/2123323775 shutdown_connections 2024-09-19T06:08:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.362+0000 7f336f10a640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33681069f0 0x7f336819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.362+0000 7f336f10a640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f33580a33b0 0x7f33580a5870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.362+0000 7f336f10a640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3368101380 0x7f33681988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.362+0000 7f336f10a640 1 --2- 172.21.15.5:0/2123323775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33681009e0 0x7f3368198390 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.362+0000 7f336f10a640 1 -- 172.21.15.5:0/2123323775 >> 172.21.15.5:0/2123323775 conn(0x7f33680fc720 msgr2=0x7f336806a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.362+0000 7f336f10a640 1 -- 172.21.15.5:0/2123323775 shutdown_connections 2024-09-19T06:08:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:21.362+0000 7f336f10a640 1 -- 172.21.15.5:0/2123323775 wait complete. 2024-09-19T06:08:21.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 117563 2.79 MB/sec execute 107 sec latency 19.363 ms 2024-09-19T06:08:22.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:21 smithi005 ceph-mon[30955]: pgmap v154: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-19T06:08:22.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:21 smithi049 ceph-mon[34640]: pgmap v154: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-19T06:08:22.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:21 smithi157 ceph-mon[35346]: pgmap v154: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-19T06:08:22.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 118016 2.77 MB/sec execute 108 sec latency 17.407 ms 2024-09-19T06:08:23.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 118550 2.79 MB/sec execute 109 sec latency 23.047 ms 2024-09-19T06:08:24.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:23 smithi005 ceph-mon[30955]: pgmap v155: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-19T06:08:24.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:23 smithi049 ceph-mon[34640]: pgmap v155: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-19T06:08:24.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:23 smithi157 ceph-mon[35346]: pgmap v155: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-19T06:08:24.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 119087 2.77 MB/sec execute 110 sec latency 22.699 ms 2024-09-19T06:08:25.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:24 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:08:25.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:24 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:08:25.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:24 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:08:25.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:24 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:08:25.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:24 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:08:25.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:24 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:08:25.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:24 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:08:25.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:24 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:08:25.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:24 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:08:25.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 119646 2.76 MB/sec execute 111 sec latency 22.940 ms 2024-09-19T06:08:26.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:25 smithi005 ceph-mon[30955]: pgmap v156: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-09-19T06:08:26.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:25 smithi049 ceph-mon[34640]: pgmap v156: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-09-19T06:08:26.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:25 smithi157 ceph-mon[35346]: pgmap v156: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-09-19T06:08:26.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 120382 2.80 MB/sec execute 112 sec latency 15.301 ms 2024-09-19T06:08:27.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 120837 2.81 MB/sec execute 113 sec latency 14.919 ms 2024-09-19T06:08:28.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:27 smithi005 ceph-mon[30955]: pgmap v157: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-09-19T06:08:28.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:27 smithi049 ceph-mon[34640]: pgmap v157: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-09-19T06:08:28.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:27 smithi157 ceph-mon[35346]: pgmap v157: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-09-19T06:08:28.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 121097 2.79 MB/sec execute 114 sec latency 11.293 ms 2024-09-19T06:08:29.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 121507 2.77 MB/sec execute 115 sec latency 15.669 ms 2024-09-19T06:08:30.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:29 smithi005 ceph-mon[30955]: pgmap v158: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-09-19T06:08:30.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:29 smithi049 ceph-mon[34640]: pgmap v158: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-09-19T06:08:30.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:29 smithi157 ceph-mon[35346]: pgmap v158: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-09-19T06:08:30.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 122084 2.78 MB/sec execute 116 sec latency 19.764 ms 2024-09-19T06:08:31.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 122614 2.76 MB/sec execute 117 sec latency 21.623 ms 2024-09-19T06:08:32.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:31 smithi005 ceph-mon[30955]: pgmap v159: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-09-19T06:08:32.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:31 smithi049 ceph-mon[34640]: pgmap v159: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-09-19T06:08:32.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:31 smithi157 ceph-mon[35346]: pgmap v159: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-09-19T06:08:32.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 123134 2.76 MB/sec execute 118 sec latency 17.005 ms 2024-09-19T06:08:33.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 123916 2.79 MB/sec execute 119 sec latency 16.023 ms 2024-09-19T06:08:34.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:33 smithi005 ceph-mon[30955]: pgmap v160: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-19T06:08:34.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:33 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:08:34.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:33 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:08:34.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:33 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:08:34.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:33 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:08:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:33 smithi049 ceph-mon[34640]: pgmap v160: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-19T06:08:34.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:33 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:08:34.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:33 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:08:34.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:33 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:08:34.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:33 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:08:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:33 smithi157 ceph-mon[35346]: pgmap v160: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-19T06:08:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:33 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:08:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:33 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:08:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:33 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:08:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:33 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:08:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.458+0000 7f4a6558d640 1 -- 172.21.15.5:0/2696534293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a601069b0 msgr2=0x7f4a60106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.458+0000 7f4a6558d640 1 --2- 172.21.15.5:0/2696534293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a601069b0 0x7f4a60106d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009930 tx=0x7f4a5002a420 comp rx=0 tx=0).stop 2024-09-19T06:08:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.458+0000 7f4a6558d640 1 -- 172.21.15.5:0/2696534293 shutdown_connections 2024-09-19T06:08:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.458+0000 7f4a6558d640 1 --2- 172.21.15.5:0/2696534293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a60101340 0x7f4a6010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.458+0000 7f4a6558d640 1 --2- 172.21.15.5:0/2696534293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a601009a0 0x7f4a60100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.458+0000 7f4a6558d640 1 --2- 172.21.15.5:0/2696534293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a601069b0 0x7f4a60106d90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.458+0000 7f4a6558d640 1 -- 172.21.15.5:0/2696534293 >> 172.21.15.5:0/2696534293 conn(0x7f4a600779a0 msgr2=0x7f4a600fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.458+0000 7f4a6558d640 1 -- 172.21.15.5:0/2696534293 shutdown_connections 2024-09-19T06:08:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.458+0000 7f4a6558d640 1 -- 172.21.15.5:0/2696534293 wait complete. 2024-09-19T06:08:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a6558d640 1 Processor -- start 2024-09-19T06:08:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a6558d640 1 -- start start 2024-09-19T06:08:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a6558d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a601009a0 0x7f4a60196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a6558d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60101340 0x7f4a60197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a6558d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a601069b0 0x7f4a601979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a6558d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60110c70 con 0x7f4a60101340 2024-09-19T06:08:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a6558d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60110af0 con 0x7f4a601069b0 2024-09-19T06:08:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a6558d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60110df0 con 0x7f4a601009a0 2024-09-19T06:08:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a5e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60101340 0x7f4a60197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a5e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60101340 0x7f4a60197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41004/0 (socket says 172.21.15.5:41004) 2024-09-19T06:08:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a5e7fc640 1 -- 172.21.15.5:0/2539476677 learned_addr learned my addr 172.21.15.5:0/2539476677 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a5effd640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a601009a0 0x7f4a60196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a601069b0 0x7f4a601979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a5e7fc640 1 -- 172.21.15.5:0/2539476677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a601009a0 msgr2=0x7f4a60196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a5e7fc640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a601009a0 0x7f4a60196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a5e7fc640 1 -- 172.21.15.5:0/2539476677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a601069b0 msgr2=0x7f4a601979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a5e7fc640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a601069b0 0x7f4a601979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a5e7fc640 1 -- 172.21.15.5:0/2539476677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a601a67b0 con 0x7f4a60101340 2024-09-19T06:08:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.459+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a601069b0 0x7f4a601979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.460+0000 7f4a5e7fc640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60101340 0x7f4a60197480 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e9f0 tx=0x7f4a5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.460+0000 7f4a47fff640 1 -- 172.21.15.5:0/2539476677 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400cd80 con 0x7f4a60101340 2024-09-19T06:08:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.460+0000 7f4a6558d640 1 -- 172.21.15.5:0/2539476677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a601a6aa0 con 0x7f4a60101340 2024-09-19T06:08:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.460+0000 7f4a47fff640 1 -- 172.21.15.5:0/2539476677 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a540108e0 con 0x7f4a60101340 2024-09-19T06:08:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.460+0000 7f4a47fff640 1 -- 172.21.15.5:0/2539476677 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54018920 con 0x7f4a60101340 2024-09-19T06:08:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.460+0000 7f4a6558d640 1 -- 172.21.15.5:0/2539476677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a601a7040 con 0x7f4a60101340 2024-09-19T06:08:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.462+0000 7f4a47fff640 1 -- 172.21.15.5:0/2539476677 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4a54018ac0 con 0x7f4a60101340 2024-09-19T06:08:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.462+0000 7f4a68403740 1 Processor -- start 2024-09-19T06:08:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a68403740 1 -- start start 2024-09-19T06:08:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a68403740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a68403740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a3410 0x5630ef1ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a68403740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1adcb0 0x5630ef1b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a68403740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630ef1072f0 con 0x5630ef1a2af0 2024-09-19T06:08:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a68403740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5630ef0af270 con 0x5630ef1adcb0 2024-09-19T06:08:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a68403740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5630ef1032b0 con 0x5630ef1a3410 2024-09-19T06:08:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a5e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1adcb0 0x5630ef1b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1a2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41012/0 (socket says 172.21.15.5:41012) 2024-09-19T06:08:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a5f7fe640 1 -- 172.21.15.5:0/780051274 learned_addr learned my addr 172.21.15.5:0/780051274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a5effd640 1 --2- 172.21.15.5:0/780051274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a3410 0x5630ef1ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a47fff640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a480a3430 0x7f4a480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a5f7fe640 1 -- 172.21.15.5:0/780051274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a3410 msgr2=0x5630ef1ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/780051274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a3410 0x5630ef1ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a5f7fe640 1 -- 172.21.15.5:0/780051274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1adcb0 msgr2=0x5630ef1b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/780051274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1adcb0 0x5630ef1b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a5f7fe640 1 -- 172.21.15.5:0/780051274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ef0ae340 con 0x5630ef1a2af0 2024-09-19T06:08:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a5e7fc640 1 --2- 172.21.15.5:0/780051274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1adcb0 0x5630ef1b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a480a3430 0x7f4a480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.463+0000 7f4a47fff640 1 -- 172.21.15.5:0/2539476677 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f4a540d0060 con 0x7f4a60101340 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.464+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/780051274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1a2ed0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4a58011880 tx=0x7f4a58011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88464ecadea6bd5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.464+0000 7f4a457fa640 1 -- 172.21.15.5:0/780051274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58017050 con 0x5630ef1a2af0 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.464+0000 7f4a457fa640 1 -- 172.21.15.5:0/780051274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a58026440 con 0x5630ef1a2af0 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.464+0000 7f4a457fa640 1 -- 172.21.15.5:0/780051274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800e610 con 0x5630ef1a2af0 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a68403740 1 -- 172.21.15.5:0/780051274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 msgr2=0x5630ef1a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a480a3430 0x7f4a480a58f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4a58023000 tx=0x7f4a58024000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a68403740 1 --2- 172.21.15.5:0/780051274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1a2ed0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4a58011880 tx=0x7f4a58011d40 comp rx=0 tx=0).stop 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a68403740 1 -- 172.21.15.5:0/780051274 shutdown_connections 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a68403740 1 --2- 172.21.15.5:0/780051274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1adcb0 0x5630ef1b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a68403740 1 --2- 172.21.15.5:0/780051274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a3410 0x5630ef1ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a68403740 1 --2- 172.21.15.5:0/780051274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1a2ed0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a68403740 1 -- 172.21.15.5:0/780051274 >> 172.21.15.5:0/780051274 conn(0x5630ef19f060 msgr2=0x5630ef19f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a68403740 1 -- 172.21.15.5:0/780051274 shutdown_connections 2024-09-19T06:08:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a68403740 1 -- 172.21.15.5:0/780051274 wait complete. 2024-09-19T06:08:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a68403740 1 -- start start 2024-09-19T06:08:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a68403740 1 Processor -- start 2024-09-19T06:08:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.465+0000 7f4a68403740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a2af0 0x5630ef1aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a68403740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1a3410 0x5630ef1ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a68403740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1adcb0 0x5630ef1ab630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a68403740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630ef1b0880 con 0x5630ef1adcb0 2024-09-19T06:08:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a68403740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5630ef090290 con 0x5630ef1a3410 2024-09-19T06:08:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a68403740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5630ef1b0a00 con 0x5630ef1a2af0 2024-09-19T06:08:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a68403740 10 client.0 _ll_register_callbacks cb 0x5630ef0c1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1adcb0 0x5630ef1ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a2af0 0x5630ef1aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1adcb0 0x5630ef1ab630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41026/0 (socket says 172.21.15.5:41026) 2024-09-19T06:08:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5f7fe640 1 -- 172.21.15.5:0/3095360460 learned_addr learned my addr 172.21.15.5:0/3095360460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a2af0 0x5630ef1aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57784/0 (socket says 172.21.15.5:57784) 2024-09-19T06:08:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5e7fc640 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1a3410 0x5630ef1ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5f7fe640 1 -- 172.21.15.5:0/3095360460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a2af0 msgr2=0x5630ef1aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a2af0 0x5630ef1aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5f7fe640 1 -- 172.21.15.5:0/3095360460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1a3410 msgr2=0x5630ef1ab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1a3410 0x5630ef1ab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5f7fe640 1 -- 172.21.15.5:0/3095360460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630eee1a210 con 0x5630ef1adcb0 2024-09-19T06:08:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5effd640 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a2af0 0x5630ef1aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1adcb0 0x5630ef1ab630 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800f8a0 tx=0x7f4a5800f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b3aa64ec214715 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a42ff5640 1 -- 172.21.15.5:0/3095360460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5802b040 con 0x5630ef1adcb0 2024-09-19T06:08:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a42ff5640 1 -- 172.21.15.5:0/3095360460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f4a58003b70 con 0x5630ef1adcb0 2024-09-19T06:08:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.466+0000 7f4a68403740 1 -- 172.21.15.5:0/3095360460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a58026440 con 0x5630ef1adcb0 2024-09-19T06:08:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.467+0000 7f4a42ff5640 1 -- 172.21.15.5:0/3095360460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4a5803c050 con 0x5630ef1adcb0 2024-09-19T06:08:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.467+0000 7f4a68403740 10 client.16725 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.467+0000 7f4a68403740 1 -- 172.21.15.5:0/3095360460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5630eeee6620 con 0x5630ef1adcb0 2024-09-19T06:08:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.467+0000 7f4a42ff5640 1 -- 172.21.15.5:0/3095360460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a58026440 con 0x5630ef1adcb0 2024-09-19T06:08:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.467+0000 7f4a42ff5640 1 -- 172.21.15.5:0/3095360460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f4a58037030 con 0x5630ef1adcb0 2024-09-19T06:08:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.467+0000 7f4a68403740 10 client.16725 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.467+0000 7f4a68403740 10 client.16725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.467+0000 7f4a68403740 20 client.16725 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.467+0000 7f4a68403740 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5630ef1bdbc0 0x5630ef1bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.467+0000 7f4a68403740 4 client.16725 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.467+0000 7f4a68403740 1 -- 172.21.15.5:0/3095360460 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630ef114a20 con 0x5630ef1bdbc0 2024-09-19T06:08:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.467+0000 7f4a5effd640 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5630ef1bdbc0 0x5630ef1bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.468+0000 7f4a42ff5640 10 client.16725 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.468+0000 7f4a5effd640 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5630ef1bdbc0 0x5630ef1bffb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7760d15c0d205b4a server_cookie=8f07b6bb4abf22f5 in_seq=0 out_seq=0 2024-09-19T06:08:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.468+0000 7f4a42ff5640 1 -- 172.21.15.5:0/3095360460 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630ef114a20 con 0x5630ef1bdbc0 2024-09-19T06:08:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.468+0000 7f4a42ff5640 10 client.16725 handle_command_reply: tid=0 2024-09-19T06:08:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.468+0000 7f4a68403740 1 client.16725 shutdown 2024-09-19T06:08:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.468+0000 7f4a68403740 1 -- 172.21.15.5:0/3095360460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1adcb0 msgr2=0x5630ef1ab630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.468+0000 7f4a68403740 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1adcb0 0x5630ef1ab630 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800f8a0 tx=0x7f4a5800f8d0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.469+0000 7f4a68403740 1 -- 172.21.15.5:0/3095360460 shutdown_connections 2024-09-19T06:08:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.469+0000 7f4a68403740 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5630ef1bdbc0 0x5630ef1bffb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.469+0000 7f4a68403740 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1adcb0 0x5630ef1ab630 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.469+0000 7f4a68403740 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1a3410 0x5630ef1ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.469+0000 7f4a42ff5640 0 client.16725 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.469+0000 7f4a68403740 1 --2- 172.21.15.5:0/3095360460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a2af0 0x5630ef1aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.469+0000 7f4a68403740 1 -- 172.21.15.5:0/3095360460 >> 172.21.15.5:0/3095360460 conn(0x5630ef19f060 msgr2=0x5630ef1a4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.469+0000 7f4a68403740 1 -- 172.21.15.5:0/3095360460 shutdown_connections 2024-09-19T06:08:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.469+0000 7f4a68403740 1 -- 172.21.15.5:0/3095360460 wait complete. 2024-09-19T06:08:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.469+0000 7f4a68403740 20 client.16725 trim_cache size 0 max 16384 2024-09-19T06:08:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a68403740 1 Processor -- start 2024-09-19T06:08:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a68403740 1 -- start start 2024-09-19T06:08:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a68403740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1ab460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a68403740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a3410 0x5630ef1acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a68403740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1adcb0 0x5630ef1c5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a68403740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601af940 con 0x5630ef1a2af0 2024-09-19T06:08:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a68403740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b1480 con 0x5630ef1adcb0 2024-09-19T06:08:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a68403740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b0010 con 0x5630ef1a3410 2024-09-19T06:08:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1ab460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41042/0 (socket says 172.21.15.5:41042) 2024-09-19T06:08:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a5effd640 1 -- 172.21.15.5:0/331645201 learned_addr learned my addr 172.21.15.5:0/331645201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/331645201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1adcb0 0x5630ef1c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.486+0000 7f4a5e7fc640 1 --2- 172.21.15.5:0/331645201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a3410 0x5630ef1acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.487+0000 7f4a5effd640 1 -- 172.21.15.5:0/331645201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a3410 msgr2=0x5630ef1acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.487+0000 7f4a5effd640 1 --2- 172.21.15.5:0/331645201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a3410 0x5630ef1acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.487+0000 7f4a5effd640 1 -- 172.21.15.5:0/331645201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1adcb0 msgr2=0x5630ef1c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.487+0000 7f4a5effd640 1 --2- 172.21.15.5:0/331645201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1adcb0 0x5630ef1c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.487+0000 7f4a5effd640 1 -- 172.21.15.5:0/331645201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ef1c0550 con 0x5630ef1a2af0 2024-09-19T06:08:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.487+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/331645201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1adcb0 0x5630ef1c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.487+0000 7f4a5effd640 1 --2- 172.21.15.5:0/331645201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1ab460 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009900 tx=0x7f4a50000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e489d3549702245 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.487+0000 7f4a44ff9640 1 -- 172.21.15.5:0/331645201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50049030 con 0x5630ef1a2af0 2024-09-19T06:08:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.487+0000 7f4a44ff9640 1 -- 172.21.15.5:0/331645201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a500314a0 con 0x5630ef1a2af0 2024-09-19T06:08:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.487+0000 7f4a44ff9640 1 -- 172.21.15.5:0/331645201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50035040 con 0x5630ef1a2af0 2024-09-19T06:08:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.488+0000 7f4a68403740 1 -- 172.21.15.5:0/331645201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 msgr2=0x5630ef1ab460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.488+0000 7f4a68403740 1 --2- 172.21.15.5:0/331645201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1ab460 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009900 tx=0x7f4a50000bc0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.488+0000 7f4a68403740 1 -- 172.21.15.5:0/331645201 shutdown_connections 2024-09-19T06:08:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.488+0000 7f4a68403740 1 --2- 172.21.15.5:0/331645201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1adcb0 0x5630ef1c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.488+0000 7f4a68403740 1 --2- 172.21.15.5:0/331645201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1a3410 0x5630ef1acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.488+0000 7f4a68403740 1 --2- 172.21.15.5:0/331645201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1ab460 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.488+0000 7f4a68403740 1 -- 172.21.15.5:0/331645201 >> 172.21.15.5:0/331645201 conn(0x5630ef19f060 msgr2=0x5630ef1ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.488+0000 7f4a68403740 1 -- 172.21.15.5:0/331645201 shutdown_connections 2024-09-19T06:08:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.488+0000 7f4a68403740 1 -- 172.21.15.5:0/331645201 wait complete. 2024-09-19T06:08:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.488+0000 7f4a68403740 1 -- start start 2024-09-19T06:08:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.488+0000 7f4a68403740 1 Processor -- start 2024-09-19T06:08:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a68403740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a68403740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1a3410 0x5630ef1b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a68403740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1adcb0 0x5630ef1a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a68403740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b0db0 con 0x5630ef1a2af0 2024-09-19T06:08:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a68403740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b06e0 con 0x5630ef1a3410 2024-09-19T06:08:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a68403740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601ad920 con 0x5630ef1adcb0 2024-09-19T06:08:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41048/0 (socket says 172.21.15.5:41048) 2024-09-19T06:08:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a68403740 10 client.0 _ll_register_callbacks cb 0x7f4a540d3960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a5effd640 1 -- 172.21.15.5:0/858968409 learned_addr learned my addr 172.21.15.5:0/858968409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a5e7fc640 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1a3410 0x5630ef1b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a5f7fe640 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1adcb0 0x5630ef1a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a5effd640 1 -- 172.21.15.5:0/858968409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1adcb0 msgr2=0x5630ef1a71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a5effd640 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1adcb0 0x5630ef1a71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a5effd640 1 -- 172.21.15.5:0/858968409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1a3410 msgr2=0x5630ef1b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a5effd640 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1a3410 0x5630ef1b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a5effd640 1 -- 172.21.15.5:0/858968409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ef1a3240 con 0x5630ef1a2af0 2024-09-19T06:08:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a5effd640 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1b5c90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4a50031890 tx=0x7f4a500318c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ff746072676de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a42ff5640 1 -- 172.21.15.5:0/858968409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50052030 con 0x5630ef1a2af0 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a42ff5640 1 -- 172.21.15.5:0/858968409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f4a500042d0 con 0x5630ef1a2af0 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.489+0000 7f4a68403740 1 -- 172.21.15.5:0/858968409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a500314a0 con 0x5630ef1a2af0 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.490+0000 7f4a42ff5640 1 -- 172.21.15.5:0/858968409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4a50049050 con 0x5630ef1a2af0 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.490+0000 7f4a68403740 10 client.16731 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.490+0000 7f4a68403740 1 -- 172.21.15.5:0/858968409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5630ef227ec0 con 0x5630ef1a2af0 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.490+0000 7f4a42ff5640 1 -- 172.21.15.5:0/858968409 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a500314a0 con 0x5630ef1a2af0 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.490+0000 7f4a42ff5640 1 -- 172.21.15.5:0/858968409 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f4a5004e6a0 con 0x5630ef1a2af0 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.490+0000 7f4a68403740 10 client.16731 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.490+0000 7f4a68403740 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.490+0000 7f4a68403740 20 client.16731 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.490+0000 7f4a68403740 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5630ef2285e0 0x5630ef22aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.490+0000 7f4a68403740 4 client.16731 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:08:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.490+0000 7f4a68403740 1 -- 172.21.15.5:0/858968409 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630ef10ed70 con 0x5630ef2285e0 2024-09-19T06:08:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.490+0000 7f4a5e7fc640 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5630ef2285e0 0x5630ef22aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.491+0000 7f4a5e7fc640 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5630ef2285e0 0x5630ef22aa00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f6386e540cd3329 server_cookie=daee431c353a6ccc in_seq=0 out_seq=0 2024-09-19T06:08:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.491+0000 7f4a42ff5640 10 client.16731 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a42ff5640 1 -- 172.21.15.5:0/858968409 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630ef10ed70 con 0x5630ef2285e0 2024-09-19T06:08:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a42ff5640 10 client.16731 handle_command_reply: tid=0 2024-09-19T06:08:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a68403740 1 client.16731 shutdown 2024-09-19T06:08:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a68403740 1 -- 172.21.15.5:0/858968409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 msgr2=0x5630ef1b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a68403740 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1b5c90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4a50031890 tx=0x7f4a500318c0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a68403740 1 -- 172.21.15.5:0/858968409 shutdown_connections 2024-09-19T06:08:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a68403740 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5630ef2285e0 0x5630ef22aa00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a68403740 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630ef1adcb0 0x5630ef1a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a42ff5640 0 client.16731 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a68403740 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630ef1a3410 0x5630ef1b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a68403740 1 --2- 172.21.15.5:0/858968409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630ef1a2af0 0x5630ef1b5c90 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a68403740 1 -- 172.21.15.5:0/858968409 >> 172.21.15.5:0/858968409 conn(0x5630ef19f060 msgr2=0x5630ef19e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a68403740 1 -- 172.21.15.5:0/858968409 shutdown_connections 2024-09-19T06:08:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a68403740 1 -- 172.21.15.5:0/858968409 wait complete. 2024-09-19T06:08:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.511+0000 7f4a68403740 20 client.16731 trim_cache size 0 max 16384 2024-09-19T06:08:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:08:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:08:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:08:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:08:34.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.512+0000 7f4a6558d640 1 -- 172.21.15.5:0/2539476677 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a480a3430 msgr2=0x7f4a480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:34.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.512+0000 7f4a6558d640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a480a3430 0x7f4a480a58f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4a58023000 tx=0x7f4a58024000 comp rx=0 tx=0).stop 2024-09-19T06:08:34.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.512+0000 7f4a6558d640 1 -- 172.21.15.5:0/2539476677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60101340 msgr2=0x7f4a60197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:34.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.512+0000 7f4a6558d640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60101340 0x7f4a60197480 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e9f0 tx=0x7f4a5400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.512+0000 7f4a6558d640 1 -- 172.21.15.5:0/2539476677 shutdown_connections 2024-09-19T06:08:34.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.512+0000 7f4a6558d640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a601069b0 0x7f4a601979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.512+0000 7f4a6558d640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60101340 0x7f4a60197480 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.512+0000 7f4a6558d640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a480a3430 0x7f4a480a58f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.512+0000 7f4a6558d640 1 --2- 172.21.15.5:0/2539476677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a601009a0 0x7f4a60196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.512+0000 7f4a6558d640 1 -- 172.21.15.5:0/2539476677 >> 172.21.15.5:0/2539476677 conn(0x7f4a600779a0 msgr2=0x7f4a600fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.512+0000 7f4a6558d640 1 -- 172.21.15.5:0/2539476677 shutdown_connections 2024-09-19T06:08:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:34.512+0000 7f4a6558d640 1 -- 172.21.15.5:0/2539476677 wait complete. 2024-09-19T06:08:34.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 124318 2.80 MB/sec execute 120 sec latency 15.439 ms 2024-09-19T06:08:35.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 124610 2.79 MB/sec execute 121 sec latency 15.837 ms 2024-09-19T06:08:36.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:35 smithi005 ceph-mon[30955]: pgmap v161: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-19T06:08:36.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:35 smithi049 ceph-mon[34640]: pgmap v161: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-19T06:08:36.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:35 smithi157 ceph-mon[35346]: pgmap v161: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-19T06:08:36.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 125011 2.77 MB/sec execute 122 sec latency 17.405 ms 2024-09-19T06:08:37.828 INFO:tasks.workunit.client.0.smithi005.stdout: 1 125588 2.78 MB/sec execute 123 sec latency 18.197 ms 2024-09-19T06:08:38.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:37 smithi005 ceph-mon[30955]: pgmap v162: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-19T06:08:38.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:37 smithi049 ceph-mon[34640]: pgmap v162: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-19T06:08:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:37 smithi157 ceph-mon[35346]: pgmap v162: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-19T06:08:38.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 126061 2.76 MB/sec execute 124 sec latency 20.378 ms 2024-09-19T06:08:39.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 126600 2.75 MB/sec execute 125 sec latency 23.612 ms 2024-09-19T06:08:40.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:39 smithi005 ceph-mon[30955]: pgmap v163: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-19T06:08:40.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:39 smithi049 ceph-mon[34640]: pgmap v163: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-19T06:08:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:39 smithi157 ceph-mon[35346]: pgmap v163: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-19T06:08:40.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 127370 2.78 MB/sec execute 126 sec latency 11.020 ms 2024-09-19T06:08:41.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 127818 2.79 MB/sec execute 127 sec latency 12.408 ms 2024-09-19T06:08:42.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:41 smithi049 ceph-mon[34640]: pgmap v164: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-19T06:08:42.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:41 smithi157 ceph-mon[35346]: pgmap v164: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-19T06:08:42.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:41 smithi005 ceph-mon[30955]: pgmap v164: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-19T06:08:42.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 128137 2.78 MB/sec execute 128 sec latency 12.039 ms 2024-09-19T06:08:43.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 128486 2.76 MB/sec execute 129 sec latency 18.259 ms 2024-09-19T06:08:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:43 smithi049 ceph-mon[34640]: pgmap v165: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-19T06:08:44.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:43 smithi157 ceph-mon[35346]: pgmap v165: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-19T06:08:44.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:43 smithi005 ceph-mon[30955]: pgmap v165: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-19T06:08:44.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 129012 2.76 MB/sec execute 130 sec latency 14.964 ms 2024-09-19T06:08:45.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 129559 2.76 MB/sec execute 131 sec latency 21.329 ms 2024-09-19T06:08:46.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:45 smithi049 ceph-mon[34640]: pgmap v166: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-09-19T06:08:46.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:45 smithi157 ceph-mon[35346]: pgmap v166: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-09-19T06:08:46.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:45 smithi005 ceph-mon[30955]: pgmap v166: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-09-19T06:08:46.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 130138 2.75 MB/sec execute 132 sec latency 18.384 ms 2024-09-19T06:08:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.610+0000 7fef82cd9640 1 -- 172.21.15.5:0/3319915020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c102490 msgr2=0x7fef7c10e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.610+0000 7fef82cd9640 1 --2- 172.21.15.5:0/3319915020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c102490 0x7fef7c10e9c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fef74009cd0 tx=0x7fef7402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.610+0000 7fef82cd9640 1 -- 172.21.15.5:0/3319915020 shutdown_connections 2024-09-19T06:08:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.610+0000 7fef82cd9640 1 --2- 172.21.15.5:0/3319915020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c102490 0x7fef7c10e9c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.611+0000 7fef82cd9640 1 --2- 172.21.15.5:0/3319915020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7c101af0 0x7fef7c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.611+0000 7fef82cd9640 1 --2- 172.21.15.5:0/3319915020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c107b90 0x7fef7c107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.611+0000 7fef82cd9640 1 -- 172.21.15.5:0/3319915020 >> 172.21.15.5:0/3319915020 conn(0x7fef7c0fd810 msgr2=0x7fef7c0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.611+0000 7fef82cd9640 1 -- 172.21.15.5:0/3319915020 shutdown_connections 2024-09-19T06:08:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.611+0000 7fef82cd9640 1 -- 172.21.15.5:0/3319915020 wait complete. 2024-09-19T06:08:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.611+0000 7fef82cd9640 1 Processor -- start 2024-09-19T06:08:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.611+0000 7fef82cd9640 1 -- start start 2024-09-19T06:08:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef82cd9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c101af0 0x7fef7c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef82cd9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c102490 0x7fef7c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef82cd9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7c107b90 0x7fef7c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef82cd9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c111df0 con 0x7fef7c102490 2024-09-19T06:08:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef82cd9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c111c70 con 0x7fef7c107b90 2024-09-19T06:08:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef82cd9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c111f70 con 0x7fef7c101af0 2024-09-19T06:08:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c102490 0x7fef7c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c102490 0x7fef7c198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54870/0 (socket says 172.21.15.5:54870) 2024-09-19T06:08:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef7bfff640 1 -- 172.21.15.5:0/2320171866 learned_addr learned my addr 172.21.15.5:0/2320171866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef80a75640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c101af0 0x7fef7c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef81276640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7c107b90 0x7fef7c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef7bfff640 1 -- 172.21.15.5:0/2320171866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c101af0 msgr2=0x7fef7c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef7bfff640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c101af0 0x7fef7c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef7bfff640 1 -- 172.21.15.5:0/2320171866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7c107b90 msgr2=0x7fef7c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef7bfff640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7c107b90 0x7fef7c19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef7bfff640 1 -- 172.21.15.5:0/2320171866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c19d310 con 0x7fef7c102490 2024-09-19T06:08:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef7bfff640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c102490 0x7fef7c198860 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fef6800e9f0 tx=0x7fef6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef79ffb640 1 -- 172.21.15.5:0/2320171866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6800cd80 con 0x7fef7c102490 2024-09-19T06:08:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef79ffb640 1 -- 172.21.15.5:0/2320171866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef680108e0 con 0x7fef7c102490 2024-09-19T06:08:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.612+0000 7fef82cd9640 1 -- 172.21.15.5:0/2320171866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c19d600 con 0x7fef7c102490 2024-09-19T06:08:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.613+0000 7fef79ffb640 1 -- 172.21.15.5:0/2320171866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68018920 con 0x7fef7c102490 2024-09-19T06:08:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.613+0000 7fef82cd9640 1 -- 172.21.15.5:0/2320171866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c1a4e90 con 0x7fef7c102490 2024-09-19T06:08:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef79ffb640 1 -- 172.21.15.5:0/2320171866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fef68010a90 con 0x7fef7c102490 2024-09-19T06:08:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef85a6e740 1 Processor -- start 2024-09-19T06:08:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef85a6e740 1 -- start start 2024-09-19T06:08:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef85a6e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 0x5593ce21f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef85a6e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 0x5593ce229b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef85a6e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce22c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef85a6e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593ce17f2f0 con 0x5593ce21f820 2024-09-19T06:08:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef85a6e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5593ce127270 con 0x5593ce22a0c0 2024-09-19T06:08:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef85a6e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5593ce17b2b0 con 0x5593ce21ef00 2024-09-19T06:08:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef80a75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 0x5593ce229b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef81276640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 0x5593ce21f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef80a75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 0x5593ce229b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54876/0 (socket says 172.21.15.5:54876) 2024-09-19T06:08:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef80a75640 1 -- 172.21.15.5:0/3060706779 learned_addr learned my addr 172.21.15.5:0/3060706779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.616+0000 7fef7bfff640 1 --2- 172.21.15.5:0/3060706779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce22c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.617+0000 7fef81276640 1 -- 172.21.15.5:0/3060706779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 msgr2=0x5593ce22c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.617+0000 7fef81276640 1 --2- 172.21.15.5:0/3060706779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce22c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.617+0000 7fef81276640 1 -- 172.21.15.5:0/3060706779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 msgr2=0x5593ce229b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.617+0000 7fef81276640 1 --2- 172.21.15.5:0/3060706779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 0x5593ce229b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.617+0000 7fef81276640 1 -- 172.21.15.5:0/3060706779 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ce126340 con 0x5593ce21ef00 2024-09-19T06:08:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.617+0000 7fef79ffb640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fef6c0a33b0 0x7fef6c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.617+0000 7fef80a75640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fef6c0a33b0 0x7fef6c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.617+0000 7fef81276640 1 --2- 172.21.15.5:0/3060706779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 0x5593ce21f2e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fef74006010 tx=0x7fef74009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dac52b6622377006 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.617+0000 7fef79ffb640 1 -- 172.21.15.5:0/2320171866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fef680d0090 con 0x7fef7c102490 2024-09-19T06:08:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.617+0000 7fef66ffd640 1 -- 172.21.15.5:0/3060706779 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74031630 con 0x5593ce21ef00 2024-09-19T06:08:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.617+0000 7fef66ffd640 1 -- 172.21.15.5:0/3060706779 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef74036040 con 0x5593ce21ef00 2024-09-19T06:08:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.617+0000 7fef66ffd640 1 -- 172.21.15.5:0/3060706779 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef740043b0 con 0x5593ce21ef00 2024-09-19T06:08:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.618+0000 7fef85a6e740 1 -- 172.21.15.5:0/3060706779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 msgr2=0x5593ce21f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.618+0000 7fef85a6e740 1 --2- 172.21.15.5:0/3060706779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 0x5593ce21f2e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fef74006010 tx=0x7fef74009a20 comp rx=0 tx=0).stop 2024-09-19T06:08:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.618+0000 7fef80a75640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fef6c0a33b0 0x7fef6c0a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fef70004650 tx=0x7fef7000b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.618+0000 7fef85a6e740 1 -- 172.21.15.5:0/3060706779 shutdown_connections 2024-09-19T06:08:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.618+0000 7fef85a6e740 1 --2- 172.21.15.5:0/3060706779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce22c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.618+0000 7fef85a6e740 1 --2- 172.21.15.5:0/3060706779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 0x5593ce229b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.618+0000 7fef85a6e740 1 --2- 172.21.15.5:0/3060706779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 0x5593ce21f2e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.618+0000 7fef85a6e740 1 -- 172.21.15.5:0/3060706779 >> 172.21.15.5:0/3060706779 conn(0x5593ce21b470 msgr2=0x5593ce21b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.619+0000 7fef85a6e740 1 -- 172.21.15.5:0/3060706779 shutdown_connections 2024-09-19T06:08:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.619+0000 7fef85a6e740 1 -- 172.21.15.5:0/3060706779 wait complete. 2024-09-19T06:08:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.619+0000 7fef85a6e740 1 -- start start 2024-09-19T06:08:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.619+0000 7fef85a6e740 1 Processor -- start 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.619+0000 7fef85a6e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 0x5593ce227030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.619+0000 7fef85a6e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 0x5593ce227570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.619+0000 7fef85a6e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce227ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.619+0000 7fef85a6e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593ce22cc90 con 0x5593ce21f820 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.619+0000 7fef85a6e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5593ce108290 con 0x5593ce22a0c0 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.619+0000 7fef85a6e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5593ce22ce10 con 0x5593ce21ef00 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef85a6e740 10 client.0 _ll_register_callbacks cb 0x5593ce139140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 0x5593ce227570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef80a75640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 0x5593ce227030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef81276640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce227ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 0x5593ce227570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54892/0 (socket says 172.21.15.5:54892) 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef7bfff640 1 -- 172.21.15.5:0/1243734085 learned_addr learned my addr 172.21.15.5:0/1243734085 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef80a75640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 0x5593ce227030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40618/0 (socket says 172.21.15.5:40618) 2024-09-19T06:08:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef81276640 1 -- 172.21.15.5:0/1243734085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 msgr2=0x5593ce227030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef81276640 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 0x5593ce227030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef81276640 1 -- 172.21.15.5:0/1243734085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 msgr2=0x5593ce227570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef81276640 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 0x5593ce227570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef81276640 1 -- 172.21.15.5:0/1243734085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593cde92210 con 0x5593ce22a0c0 2024-09-19T06:08:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef80a75640 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 0x5593ce227030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef7bfff640 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 0x5593ce227570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef81276640 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce227ab0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef74009bf0 tx=0x7fef74009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=876f4fa31befc495 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:08:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef5bfff640 1 -- 172.21.15.5:0/1243734085 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74031a50 con 0x5593ce22a0c0 2024-09-19T06:08:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.620+0000 7fef5bfff640 1 -- 172.21.15.5:0/1243734085 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fef74031bf0 con 0x5593ce22a0c0 2024-09-19T06:08:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef85a6e740 1 -- 172.21.15.5:0/1243734085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef74036040 con 0x5593ce22a0c0 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef5bfff640 1 -- 172.21.15.5:0/1243734085 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fef74031640 con 0x5593ce22a0c0 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef85a6e740 10 client.26327 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef85a6e740 1 -- 172.21.15.5:0/1243734085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5593cdf5e620 con 0x5593ce22a0c0 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef5bfff640 1 -- 172.21.15.5:0/1243734085 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef74036040 con 0x5593ce22a0c0 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef5bfff640 1 -- 172.21.15.5:0/1243734085 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fef74030070 con 0x5593ce22a0c0 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef85a6e740 10 client.26327 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef85a6e740 10 client.26327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef85a6e740 20 client.26327 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593ce239f10 0x5593ce23c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef85a6e740 4 client.26327 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef85a6e740 1 -- 172.21.15.5:0/1243734085 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593ce18ca20 con 0x5593ce239f10 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.621+0000 7fef80a75640 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593ce239f10 0x5593ce23c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.622+0000 7fef80a75640 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593ce239f10 0x5593ce23c300 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0d3d74e24807d7 server_cookie=db70c40022e85b3a in_seq=0 out_seq=0 2024-09-19T06:08:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.622+0000 7fef5bfff640 10 client.26327 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.622+0000 7fef5bfff640 1 -- 172.21.15.5:0/1243734085 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593ce18ca20 con 0x5593ce239f10 2024-09-19T06:08:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.622+0000 7fef5bfff640 10 client.26327 handle_command_reply: tid=0 2024-09-19T06:08:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.622+0000 7fef85a6e740 1 client.26327 shutdown 2024-09-19T06:08:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.623+0000 7fef85a6e740 1 -- 172.21.15.5:0/1243734085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 msgr2=0x5593ce227ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.623+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce227ab0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef74009bf0 tx=0x7fef74009920 comp rx=0 tx=0).stop 2024-09-19T06:08:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.623+0000 7fef85a6e740 1 -- 172.21.15.5:0/1243734085 shutdown_connections 2024-09-19T06:08:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.623+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593ce239f10 0x5593ce23c300 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.623+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce227ab0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.623+0000 7fef5bfff640 0 client.26327 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.623+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21f820 0x5593ce227570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.623+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1243734085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21ef00 0x5593ce227030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.623+0000 7fef85a6e740 1 -- 172.21.15.5:0/1243734085 >> 172.21.15.5:0/1243734085 conn(0x5593ce21b470 msgr2=0x5593ce2161c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.623+0000 7fef85a6e740 1 -- 172.21.15.5:0/1243734085 shutdown_connections 2024-09-19T06:08:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.623+0000 7fef85a6e740 1 -- 172.21.15.5:0/1243734085 wait complete. 2024-09-19T06:08:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.623+0000 7fef85a6e740 20 client.26327 trim_cache size 0 max 16384 2024-09-19T06:08:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef85a6e740 1 Processor -- start 2024-09-19T06:08:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef85a6e740 1 -- start start 2024-09-19T06:08:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef85a6e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce21ef00 0x5593ce2258e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef85a6e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21f820 0x5593ce2278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef85a6e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce22a0c0 0x5593ce23f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef85a6e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1ab5e0 con 0x5593ce22a0c0 2024-09-19T06:08:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef85a6e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1ad120 con 0x5593ce21ef00 2024-09-19T06:08:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef85a6e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1abcb0 con 0x5593ce21f820 2024-09-19T06:08:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef81276640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce22a0c0 0x5593ce23f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef80a75640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce21ef00 0x5593ce2258e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef80a75640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce21ef00 0x5593ce2258e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33710/0 (socket says 172.21.15.5:33710) 2024-09-19T06:08:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21f820 0x5593ce2278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef80a75640 1 -- 172.21.15.5:0/1095558549 learned_addr learned my addr 172.21.15.5:0/1095558549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef80a75640 1 -- 172.21.15.5:0/1095558549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21f820 msgr2=0x5593ce2278e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef80a75640 1 --2- 172.21.15.5:0/1095558549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21f820 0x5593ce2278e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef80a75640 1 -- 172.21.15.5:0/1095558549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce22a0c0 msgr2=0x5593ce23f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.640+0000 7fef80a75640 1 --2- 172.21.15.5:0/1095558549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce22a0c0 0x5593ce23f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.641+0000 7fef80a75640 1 -- 172.21.15.5:0/1095558549 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ce21c520 con 0x5593ce21ef00 2024-09-19T06:08:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.641+0000 7fef81276640 1 --2- 172.21.15.5:0/1095558549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce22a0c0 0x5593ce23f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.641+0000 7fef80a75640 1 --2- 172.21.15.5:0/1095558549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce21ef00 0x5593ce2258e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef70005850 tx=0x7fef70019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39a12e5da3523f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.641+0000 7fef667fc640 1 -- 172.21.15.5:0/1095558549 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7001e9b0 con 0x5593ce21ef00 2024-09-19T06:08:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.641+0000 7fef667fc640 1 -- 172.21.15.5:0/1095558549 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef7002a420 con 0x5593ce21ef00 2024-09-19T06:08:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.641+0000 7fef667fc640 1 -- 172.21.15.5:0/1095558549 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7001d670 con 0x5593ce21ef00 2024-09-19T06:08:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.642+0000 7fef85a6e740 1 -- 172.21.15.5:0/1095558549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce21ef00 msgr2=0x5593ce2258e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.642+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1095558549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce21ef00 0x5593ce2258e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef70005850 tx=0x7fef70019000 comp rx=0 tx=0).stop 2024-09-19T06:08:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.642+0000 7fef85a6e740 1 -- 172.21.15.5:0/1095558549 shutdown_connections 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.642+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1095558549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce22a0c0 0x5593ce23f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.642+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1095558549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21f820 0x5593ce2278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.642+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1095558549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce21ef00 0x5593ce2258e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.642+0000 7fef85a6e740 1 -- 172.21.15.5:0/1095558549 >> 172.21.15.5:0/1095558549 conn(0x5593ce21b470 msgr2=0x5593ce2290b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.642+0000 7fef85a6e740 1 -- 172.21.15.5:0/1095558549 shutdown_connections 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.642+0000 7fef85a6e740 1 -- 172.21.15.5:0/1095558549 wait complete. 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.642+0000 7fef85a6e740 1 -- start start 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.642+0000 7fef85a6e740 1 Processor -- start 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef85a6e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21ef00 0x5593ce23c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef85a6e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21f820 0x5593ce23cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef85a6e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce2a2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef85a6e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1aca50 con 0x5593ce21ef00 2024-09-19T06:08:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef85a6e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1ac380 con 0x5593ce22a0c0 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef85a6e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1a95c0 con 0x5593ce21f820 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef85a6e740 10 client.0 _ll_register_callbacks cb 0x7fef700040e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef80a75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21ef00 0x5593ce23c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef80a75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21ef00 0x5593ce23c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54910/0 (socket says 172.21.15.5:54910) 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef80a75640 1 -- 172.21.15.5:0/1765647285 learned_addr learned my addr 172.21.15.5:0/1765647285 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef81276640 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce2a2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef7bfff640 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21f820 0x5593ce23cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef80a75640 1 -- 172.21.15.5:0/1765647285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21f820 msgr2=0x5593ce23cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef80a75640 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21f820 0x5593ce23cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef80a75640 1 -- 172.21.15.5:0/1765647285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 msgr2=0x5593ce2a2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef80a75640 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce2a2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef80a75640 1 -- 172.21.15.5:0/1765647285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593ce2122d0 con 0x5593ce21ef00 2024-09-19T06:08:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef7bfff640 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21f820 0x5593ce23cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef81276640 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce2a2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef80a75640 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21ef00 0x5593ce23c880 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fef70003e70 tx=0x7fef70003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a7c33f716bab89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef5bfff640 1 -- 172.21.15.5:0/1765647285 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7002a890 con 0x5593ce21ef00 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef5bfff640 1 -- 172.21.15.5:0/1765647285 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fef7001e9b0 con 0x5593ce21ef00 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.643+0000 7fef85a6e740 1 -- 172.21.15.5:0/1765647285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef7002a420 con 0x5593ce21ef00 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.644+0000 7fef5bfff640 1 -- 172.21.15.5:0/1765647285 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fef7001ebb0 con 0x5593ce21ef00 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.644+0000 7fef85a6e740 10 client.16749 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.644+0000 7fef85a6e740 1 -- 172.21.15.5:0/1765647285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5593ce21c100 con 0x5593ce21ef00 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.644+0000 7fef5bfff640 1 -- 172.21.15.5:0/1765647285 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef7002a420 con 0x5593ce21ef00 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.644+0000 7fef5bfff640 1 -- 172.21.15.5:0/1765647285 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fef7003c020 con 0x5593ce21ef00 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.644+0000 7fef85a6e740 10 client.16749 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.644+0000 7fef85a6e740 10 client.16749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.644+0000 7fef85a6e740 20 client.16749 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.644+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593ce2a44a0 0x5593ce2a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.644+0000 7fef85a6e740 4 client.16749 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:08:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.644+0000 7fef85a6e740 1 -- 172.21.15.5:0/1765647285 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593ce186d70 con 0x5593ce2a44a0 2024-09-19T06:08:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.644+0000 7fef7bfff640 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593ce2a44a0 0x5593ce2a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.645+0000 7fef5bfff640 10 client.16749 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.645+0000 7fef7bfff640 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593ce2a44a0 0x5593ce2a68c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0a311fba15a3b49 server_cookie=dc07d1339a538738 in_seq=0 out_seq=0 2024-09-19T06:08:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef5bfff640 1 -- 172.21.15.5:0/1765647285 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5593ce186d70 con 0x5593ce2a44a0 2024-09-19T06:08:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef5bfff640 10 client.16749 handle_command_reply: tid=0 2024-09-19T06:08:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef85a6e740 1 client.16749 shutdown 2024-09-19T06:08:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef85a6e740 1 -- 172.21.15.5:0/1765647285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21ef00 msgr2=0x5593ce23c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21ef00 0x5593ce23c880 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fef70003e70 tx=0x7fef70003bc0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef85a6e740 1 -- 172.21.15.5:0/1765647285 shutdown_connections 2024-09-19T06:08:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593ce2a44a0 0x5593ce2a68c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593ce22a0c0 0x5593ce2a2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593ce21f820 0x5593ce23cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef85a6e740 1 --2- 172.21.15.5:0/1765647285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ce21ef00 0x5593ce23c880 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef85a6e740 1 -- 172.21.15.5:0/1765647285 >> 172.21.15.5:0/1765647285 conn(0x5593ce21b470 msgr2=0x5593ce228890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef5bfff640 0 client.16749 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef85a6e740 1 -- 172.21.15.5:0/1765647285 shutdown_connections 2024-09-19T06:08:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef85a6e740 1 -- 172.21.15.5:0/1765647285 wait complete. 2024-09-19T06:08:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.665+0000 7fef85a6e740 20 client.16749 trim_cache size 0 max 16384 2024-09-19T06:08:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:08:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:08:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:08:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:08:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.666+0000 7fef82cd9640 1 -- 172.21.15.5:0/2320171866 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fef6c0a33b0 msgr2=0x7fef6c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.666+0000 7fef82cd9640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fef6c0a33b0 0x7fef6c0a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fef70004650 tx=0x7fef7000b390 comp rx=0 tx=0).stop 2024-09-19T06:08:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.666+0000 7fef82cd9640 1 -- 172.21.15.5:0/2320171866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c102490 msgr2=0x7fef7c198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.666+0000 7fef82cd9640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c102490 0x7fef7c198860 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fef6800e9f0 tx=0x7fef6800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.666+0000 7fef82cd9640 1 -- 172.21.15.5:0/2320171866 shutdown_connections 2024-09-19T06:08:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.666+0000 7fef82cd9640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7c107b90 0x7fef7c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.666+0000 7fef82cd9640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c102490 0x7fef7c198860 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.666+0000 7fef82cd9640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fef6c0a33b0 0x7fef6c0a5870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.666+0000 7fef82cd9640 1 --2- 172.21.15.5:0/2320171866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c101af0 0x7fef7c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.666+0000 7fef82cd9640 1 -- 172.21.15.5:0/2320171866 >> 172.21.15.5:0/2320171866 conn(0x7fef7c0fd810 msgr2=0x7fef7c0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.666+0000 7fef82cd9640 1 -- 172.21.15.5:0/2320171866 shutdown_connections 2024-09-19T06:08:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:08:47.666+0000 7fef82cd9640 1 -- 172.21.15.5:0/2320171866 wait complete. 2024-09-19T06:08:47.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:08:47.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 130747 2.77 MB/sec execute 133 sec latency 15.991 ms 2024-09-19T06:08:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:48.109 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:47 smithi005 ceph-mon[30955]: pgmap v167: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.9 MiB/s wr, 188 op/s 2024-09-19T06:08:48.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:47 smithi049 ceph-mon[34640]: pgmap v167: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.9 MiB/s wr, 188 op/s 2024-09-19T06:08:48.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:47 smithi157 ceph-mon[35346]: pgmap v167: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.9 MiB/s wr, 188 op/s 2024-09-19T06:08:48.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.509+0000 7fa168691640 1 -- 172.21.15.5:0/2107227720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160100e20 msgr2=0x7fa160101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:48.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.509+0000 7fa168691640 1 --2- 172.21.15.5:0/2107227720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160100e20 0x7fa160101200 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009900 tx=0x7fa15c02a480 comp rx=0 tx=0).stop 2024-09-19T06:08:48.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.510+0000 7fa168691640 1 -- 172.21.15.5:0/2107227720 shutdown_connections 2024-09-19T06:08:48.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.510+0000 7fa168691640 1 --2- 172.21.15.5:0/2107227720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102ef0 0x7fa16010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.510+0000 7fa168691640 1 --2- 172.21.15.5:0/2107227720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa160102570 0x7fa1601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.510+0000 7fa168691640 1 --2- 172.21.15.5:0/2107227720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160100e20 0x7fa160101200 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.510+0000 7fa168691640 1 -- 172.21.15.5:0/2107227720 >> 172.21.15.5:0/2107227720 conn(0x7fa1600fc7d0 msgr2=0x7fa1600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:48.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.510+0000 7fa168691640 1 -- 172.21.15.5:0/2107227720 shutdown_connections 2024-09-19T06:08:48.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.510+0000 7fa168691640 1 -- 172.21.15.5:0/2107227720 wait complete. 2024-09-19T06:08:48.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.510+0000 7fa168691640 1 Processor -- start 2024-09-19T06:08:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.510+0000 7fa168691640 1 -- start start 2024-09-19T06:08:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.511+0000 7fa168691640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160100e20 0x7fa160197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.511+0000 7fa16642d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160100e20 0x7fa160197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.511+0000 7fa16642d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160100e20 0x7fa160197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54936/0 (socket says 172.21.15.5:54936) 2024-09-19T06:08:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.511+0000 7fa168691640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa160102570 0x7fa1601975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.511+0000 7fa16642d640 1 -- 172.21.15.5:0/1568588101 learned_addr learned my addr 172.21.15.5:0/1568588101 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.511+0000 7fa168691640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102ef0 0x7fa160197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.511+0000 7fa168691640 1 -- 172.21.15.5:0/1568588101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa160110e30 con 0x7fa160100e20 2024-09-19T06:08:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.511+0000 7fa165c2c640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa160102570 0x7fa1601975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.511+0000 7fa166c2e640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102ef0 0x7fa160197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.511+0000 7fa168691640 1 -- 172.21.15.5:0/1568588101 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa160110cb0 con 0x7fa160102ef0 2024-09-19T06:08:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.511+0000 7fa168691640 1 -- 172.21.15.5:0/1568588101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa160110fb0 con 0x7fa160102570 2024-09-19T06:08:48.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.512+0000 7fa165c2c640 1 -- 172.21.15.5:0/1568588101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102ef0 msgr2=0x7fa160197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:48.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.512+0000 7fa165c2c640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102ef0 0x7fa160197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.512+0000 7fa165c2c640 1 -- 172.21.15.5:0/1568588101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160100e20 msgr2=0x7fa160197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:48.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.512+0000 7fa165c2c640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160100e20 0x7fa160197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.512+0000 7fa165c2c640 1 -- 172.21.15.5:0/1568588101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1601a6900 con 0x7fa160102570 2024-09-19T06:08:48.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.512+0000 7fa166c2e640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102ef0 0x7fa160197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:48.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.512+0000 7fa16642d640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160100e20 0x7fa160197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:48.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.512+0000 7fa165c2c640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa160102570 0x7fa1601975d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa15000d720 tx=0x7fa15000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:48.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.513+0000 7fa1577fe640 1 -- 172.21.15.5:0/1568588101 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150017070 con 0x7fa160102570 2024-09-19T06:08:48.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.513+0000 7fa168691640 1 -- 172.21.15.5:0/1568588101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1601a6bf0 con 0x7fa160102570 2024-09-19T06:08:48.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.513+0000 7fa1577fe640 1 -- 172.21.15.5:0/1568588101 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa15000de30 con 0x7fa160102570 2024-09-19T06:08:48.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.513+0000 7fa1577fe640 1 -- 172.21.15.5:0/1568588101 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1500047a0 con 0x7fa160102570 2024-09-19T06:08:48.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.513+0000 7fa168691640 1 -- 172.21.15.5:0/1568588101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1601a7130 con 0x7fa160102570 2024-09-19T06:08:48.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.515+0000 7fa1577fe640 1 -- 172.21.15.5:0/1568588101 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa1500099d0 con 0x7fa160102570 2024-09-19T06:08:48.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa16aa6a740 1 Processor -- start 2024-09-19T06:08:48.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa16aa6a740 1 -- start start 2024-09-19T06:08:48.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa16aa6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 0x5617712e0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa1577fe640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa14c0a3430 0x7fa14c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa16aa6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e0dc0 0x5617712eb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa16aa6a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 0x5617712eda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa16aa6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617712462e0 con 0x5617712e04a0 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa16aa6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5617711edce0 con 0x5617712e0dc0 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa16aa6a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5617712422a0 con 0x5617712eb680 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa166c2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 0x5617712e0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa166c2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 0x5617712e0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54940/0 (socket says 172.21.15.5:54940) 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa166c2e640 1 -- 172.21.15.5:0/1582789192 learned_addr learned my addr 172.21.15.5:0/1582789192 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa16642d640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa14c0a3430 0x7fa14c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.516+0000 7fa166c2e640 1 --2- 172.21.15.5:0/1582789192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 0x5617712eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.517+0000 7fa166c2e640 1 -- 172.21.15.5:0/1582789192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 msgr2=0x5617712eda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.517+0000 7fa166c2e640 1 --2- 172.21.15.5:0/1582789192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 0x5617712eda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.517+0000 7fa166c2e640 1 -- 172.21.15.5:0/1582789192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e0dc0 msgr2=0x5617712eb140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.517+0000 7fa166c2e640 1 --2- 172.21.15.5:0/1582789192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e0dc0 0x5617712eb140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.517+0000 7fa166c2e640 1 -- 172.21.15.5:0/1582789192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617711d2440 con 0x5617712e04a0 2024-09-19T06:08:48.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.517+0000 7fa1577fe640 1 -- 172.21.15.5:0/1568588101 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa1500cf850 con 0x7fa160102570 2024-09-19T06:08:48.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.517+0000 7fa166c2e640 1 --2- 172.21.15.5:0/1582789192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 0x5617712e0880 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa158009fd0 tx=0x7fa158009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2be31445e4e6fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:48.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.517+0000 7fa16642d640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa14c0a3430 0x7fa14c0a58f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0096c0 tx=0x7fa15c005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:48.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.517+0000 7fa154ff9640 1 -- 172.21.15.5:0/1582789192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158011d30 con 0x5617712e04a0 2024-09-19T06:08:48.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.517+0000 7fa154ff9640 1 -- 172.21.15.5:0/1582789192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa158003e20 con 0x5617712e04a0 2024-09-19T06:08:48.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.517+0000 7fa154ff9640 1 -- 172.21.15.5:0/1582789192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158020420 con 0x5617712e04a0 2024-09-19T06:08:48.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.518+0000 7fa16aa6a740 1 -- 172.21.15.5:0/1582789192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 msgr2=0x5617712e0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:48.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.518+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/1582789192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 0x5617712e0880 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa158009fd0 tx=0x7fa158009f20 comp rx=0 tx=0).stop 2024-09-19T06:08:48.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.518+0000 7fa16aa6a740 1 -- 172.21.15.5:0/1582789192 shutdown_connections 2024-09-19T06:08:48.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.518+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/1582789192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 0x5617712eda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.518+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/1582789192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e0dc0 0x5617712eb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.518+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/1582789192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 0x5617712e0880 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.518+0000 7fa16aa6a740 1 -- 172.21.15.5:0/1582789192 >> 172.21.15.5:0/1582789192 conn(0x5617712dca30 msgr2=0x5617712dce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.518+0000 7fa16aa6a740 1 -- 172.21.15.5:0/1582789192 shutdown_connections 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.518+0000 7fa16aa6a740 1 -- 172.21.15.5:0/1582789192 wait complete. 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa16aa6a740 1 -- start start 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa16aa6a740 1 Processor -- start 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa16aa6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e04a0 0x5617712eb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa16aa6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e0dc0 0x5617712e8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa16aa6a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 0x5617712e8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa16aa6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617712ee250 con 0x5617712e0dc0 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa16aa6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561771201050 con 0x5617712e04a0 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa16aa6a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5617712ee3d0 con 0x5617712eb680 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa16aa6a740 10 client.0 _ll_register_callbacks cb 0x561771241160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa165c2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e0dc0 0x5617712e8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa16642d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e04a0 0x5617712eb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa16642d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e04a0 0x5617712eb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33776/0 (socket says 172.21.15.5:33776) 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa16642d640 1 -- 172.21.15.5:0/3885875525 learned_addr learned my addr 172.21.15.5:0/3885875525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa165c2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e0dc0 0x5617712e8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54956/0 (socket says 172.21.15.5:54956) 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.519+0000 7fa166c2e640 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 0x5617712e8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa16642d640 1 -- 172.21.15.5:0/3885875525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 msgr2=0x5617712e8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa16642d640 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 0x5617712e8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa16642d640 1 -- 172.21.15.5:0/3885875525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e0dc0 msgr2=0x5617712e8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa16642d640 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e0dc0 0x5617712e8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa16642d640 1 -- 172.21.15.5:0/3885875525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561770fb73c0 con 0x5617712e04a0 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa165c2c640 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e0dc0 0x5617712e8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa16642d640 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e04a0 0x5617712eb0d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009a20 tx=0x7fa15c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=522f781f0c31f0dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa149ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:08:48.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa149ffb640 1 -- 172.21.15.5:0/3885875525 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c04c020 con 0x5617712e04a0 2024-09-19T06:08:48.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa149ffb640 1 -- 172.21.15.5:0/3885875525 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa15c005320 con 0x5617712e04a0 2024-09-19T06:08:48.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa16aa6a740 1 -- 172.21.15.5:0/3885875525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa158003e20 con 0x5617712e04a0 2024-09-19T06:08:48.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa149ffb640 1 -- 172.21.15.5:0/3885875525 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa15c057050 con 0x5617712e04a0 2024-09-19T06:08:48.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa16aa6a740 10 client.26342 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:48.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.520+0000 7fa16aa6a740 1 -- 172.21.15.5:0/3885875525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5617712f9d70 con 0x5617712e04a0 2024-09-19T06:08:48.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.521+0000 7fa149ffb640 1 -- 172.21.15.5:0/3885875525 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa158003e20 con 0x5617712e04a0 2024-09-19T06:08:48.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.521+0000 7fa149ffb640 1 -- 172.21.15.5:0/3885875525 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa15c003710 con 0x5617712e04a0 2024-09-19T06:08:48.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.521+0000 7fa16aa6a740 10 client.26342 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:48.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.521+0000 7fa16aa6a740 10 client.26342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:48.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.521+0000 7fa16aa6a740 20 client.26342 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:48.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.521+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5617712fb230 0x5617712fd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.521+0000 7fa16aa6a740 4 client.26342 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.521+0000 7fa16aa6a740 1 -- 172.21.15.5:0/3885875525 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561771253850 con 0x5617712fb230 2024-09-19T06:08:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.521+0000 7fa165c2c640 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5617712fb230 0x5617712fd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.521+0000 7fa165c2c640 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5617712fb230 0x5617712fd640 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc662fe1c00e525d server_cookie=b31d11bba67cd79d in_seq=0 out_seq=0 2024-09-19T06:08:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.521+0000 7fa149ffb640 10 client.26342 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:48.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa149ffb640 1 -- 172.21.15.5:0/3885875525 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561771253850 con 0x5617712fb230 2024-09-19T06:08:48.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa149ffb640 10 client.26342 handle_command_reply: tid=0 2024-09-19T06:08:48.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa16aa6a740 1 client.26342 shutdown 2024-09-19T06:08:48.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa16aa6a740 1 -- 172.21.15.5:0/3885875525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e04a0 msgr2=0x5617712eb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:48.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e04a0 0x5617712eb0d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009a20 tx=0x7fa15c009260 comp rx=0 tx=0).stop 2024-09-19T06:08:48.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa16aa6a740 1 -- 172.21.15.5:0/3885875525 shutdown_connections 2024-09-19T06:08:48.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5617712fb230 0x5617712fd640 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 0x5617712e8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e0dc0 0x5617712e8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/3885875525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e04a0 0x5617712eb0d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa16aa6a740 1 -- 172.21.15.5:0/3885875525 >> 172.21.15.5:0/3885875525 conn(0x5617712dca30 msgr2=0x5617712e2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:48.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa149ffb640 0 client.26342 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:48.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa16aa6a740 1 -- 172.21.15.5:0/3885875525 shutdown_connections 2024-09-19T06:08:48.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa16aa6a740 1 -- 172.21.15.5:0/3885875525 wait complete. 2024-09-19T06:08:48.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.523+0000 7fa16aa6a740 20 client.26342 trim_cache size 0 max 16384 2024-09-19T06:08:48.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.543+0000 7fa16aa6a740 1 Processor -- start 2024-09-19T06:08:48.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.543+0000 7fa16aa6a740 1 -- start start 2024-09-19T06:08:48.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.543+0000 7fa16aa6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 0x5617712dce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa16aa6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e0dc0 0x5617712e2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa16aa6a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 0x561771302f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa16aa6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601af900 con 0x5617712e04a0 2024-09-19T06:08:48.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa16aa6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601b1440 con 0x5617712e0dc0 2024-09-19T06:08:48.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa16aa6a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601affd0 con 0x5617712eb680 2024-09-19T06:08:48.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa165c2c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e0dc0 0x5617712e2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa16642d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 0x5617712dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa165c2c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e0dc0 0x5617712e2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33782/0 (socket says 172.21.15.5:33782) 2024-09-19T06:08:48.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa16642d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 0x5617712dce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54962/0 (socket says 172.21.15.5:54962) 2024-09-19T06:08:48.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa165c2c640 1 -- 172.21.15.5:0/963408605 learned_addr learned my addr 172.21.15.5:0/963408605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:48.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa166c2e640 1 --2- 172.21.15.5:0/963408605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 0x561771302f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa165c2c640 1 -- 172.21.15.5:0/963408605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 msgr2=0x561771302f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:48.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa165c2c640 1 --2- 172.21.15.5:0/963408605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 0x561771302f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa165c2c640 1 -- 172.21.15.5:0/963408605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 msgr2=0x5617712dce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:48.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa165c2c640 1 --2- 172.21.15.5:0/963408605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 0x5617712dce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa165c2c640 1 -- 172.21.15.5:0/963408605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617712e8e80 con 0x5617712e0dc0 2024-09-19T06:08:48.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.544+0000 7fa165c2c640 1 --2- 172.21.15.5:0/963408605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e0dc0 0x5617712e2650 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa15009b840 tx=0x7fa150086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd516e82a51a6e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:48.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.545+0000 7fa14bfff640 1 -- 172.21.15.5:0/963408605 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15009fef0 con 0x5617712e0dc0 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.545+0000 7fa14bfff640 1 -- 172.21.15.5:0/963408605 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1500a2490 con 0x5617712e0dc0 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.545+0000 7fa14bfff640 1 -- 172.21.15.5:0/963408605 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150099be0 con 0x5617712e0dc0 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.545+0000 7fa16aa6a740 1 -- 172.21.15.5:0/963408605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e0dc0 msgr2=0x5617712e2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.545+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/963408605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e0dc0 0x5617712e2650 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa15009b840 tx=0x7fa150086040 comp rx=0 tx=0).stop 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.545+0000 7fa16aa6a740 1 -- 172.21.15.5:0/963408605 shutdown_connections 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.545+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/963408605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712eb680 0x561771302f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.545+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/963408605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e0dc0 0x5617712e2650 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.545+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/963408605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712e04a0 0x5617712dce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.545+0000 7fa16aa6a740 1 -- 172.21.15.5:0/963408605 >> 172.21.15.5:0/963408605 conn(0x5617712dca30 msgr2=0x5617712e1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa16aa6a740 1 -- 172.21.15.5:0/963408605 shutdown_connections 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa16aa6a740 1 -- 172.21.15.5:0/963408605 wait complete. 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa16aa6a740 1 -- start start 2024-09-19T06:08:48.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa16aa6a740 1 Processor -- start 2024-09-19T06:08:48.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa16aa6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e04a0 0x5617712f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa16aa6a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712e0dc0 0x5617712eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa16aa6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712eb680 0x5617712ef330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa16aa6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601b0d70 con 0x5617712eb680 2024-09-19T06:08:48.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa16aa6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601b06a0 con 0x5617712e04a0 2024-09-19T06:08:48.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa16aa6a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601ad8e0 con 0x5617712e0dc0 2024-09-19T06:08:48.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa16aa6a740 10 client.0 _ll_register_callbacks cb 0x7fa15c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:48.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa166c2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712eb680 0x5617712ef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa16642d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e04a0 0x5617712f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa166c2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712eb680 0x5617712ef330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54972/0 (socket says 172.21.15.5:54972) 2024-09-19T06:08:48.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.546+0000 7fa166c2e640 1 -- 172.21.15.5:0/257003402 learned_addr learned my addr 172.21.15.5:0/257003402 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:48.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.547+0000 7fa165c2c640 1 --2- 172.21.15.5:0/257003402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712e0dc0 0x5617712eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.547+0000 7fa166c2e640 1 -- 172.21.15.5:0/257003402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712e0dc0 msgr2=0x5617712eedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.547+0000 7fa166c2e640 1 --2- 172.21.15.5:0/257003402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712e0dc0 0x5617712eedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.547+0000 7fa166c2e640 1 -- 172.21.15.5:0/257003402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e04a0 msgr2=0x5617712f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.547+0000 7fa166c2e640 1 --2- 172.21.15.5:0/257003402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e04a0 0x5617712f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.547+0000 7fa166c2e640 1 -- 172.21.15.5:0/257003402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617712ef870 con 0x5617712eb680 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.547+0000 7fa166c2e640 1 --2- 172.21.15.5:0/257003402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712eb680 0x5617712ef330 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa158009fd0 tx=0x7fa158003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f06d4a4b6e13bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.547+0000 7fa149ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.547+0000 7fa149ffb640 1 -- 172.21.15.5:0/257003402 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1580050c0 con 0x5617712eb680 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.547+0000 7fa149ffb640 1 -- 172.21.15.5:0/257003402 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa158005260 con 0x5617712eb680 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.547+0000 7fa16aa6a740 1 -- 172.21.15.5:0/257003402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1500a2490 con 0x5617712eb680 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.547+0000 7fa149ffb640 1 -- 172.21.15.5:0/257003402 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa158005460 con 0x5617712eb680 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.548+0000 7fa16aa6a740 10 client.16767 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:48.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.548+0000 7fa16aa6a740 1 -- 172.21.15.5:0/257003402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561771365be0 con 0x5617712eb680 2024-09-19T06:08:48.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.548+0000 7fa149ffb640 1 -- 172.21.15.5:0/257003402 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1500a2490 con 0x5617712eb680 2024-09-19T06:08:48.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.548+0000 7fa149ffb640 1 -- 172.21.15.5:0/257003402 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa158033be0 con 0x5617712eb680 2024-09-19T06:08:48.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.548+0000 7fa16aa6a740 10 client.16767 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:48.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.548+0000 7fa16aa6a740 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:48.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.548+0000 7fa16aa6a740 20 client.16767 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:48.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.548+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/257003402 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561771365fc0 0x5617713683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:48.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.548+0000 7fa16aa6a740 4 client.16767 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:08:48.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.548+0000 7fa16aa6a740 1 -- 172.21.15.5:0/257003402 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56177124dd60 con 0x561771365fc0 2024-09-19T06:08:48.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.548+0000 7fa16642d640 1 --2- 172.21.15.5:0/257003402 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561771365fc0 0x5617713683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:48.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16642d640 1 --2- 172.21.15.5:0/257003402 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561771365fc0 0x5617713683e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7edd99b391b75469 server_cookie=a2cc080cf71baef0 in_seq=0 out_seq=0 2024-09-19T06:08:48.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa149ffb640 10 client.16767 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:48.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa149ffb640 1 -- 172.21.15.5:0/257003402 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56177124dd60 con 0x561771365fc0 2024-09-19T06:08:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa149ffb640 10 client.16767 handle_command_reply: tid=0 2024-09-19T06:08:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16aa6a740 1 client.16767 shutdown 2024-09-19T06:08:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16aa6a740 1 -- 172.21.15.5:0/257003402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712eb680 msgr2=0x5617712ef330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/257003402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712eb680 0x5617712ef330 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa158009fd0 tx=0x7fa158003ba0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16aa6a740 1 -- 172.21.15.5:0/257003402 shutdown_connections 2024-09-19T06:08:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/257003402 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561771365fc0 0x5617713683e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/257003402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617712eb680 0x5617712ef330 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/257003402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617712e0dc0 0x5617712eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16aa6a740 1 --2- 172.21.15.5:0/257003402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617712e04a0 0x5617712f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16aa6a740 1 -- 172.21.15.5:0/257003402 >> 172.21.15.5:0/257003402 conn(0x5617712dca30 msgr2=0x5617712e2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa149ffb640 0 client.16767 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16aa6a740 1 -- 172.21.15.5:0/257003402 shutdown_connections 2024-09-19T06:08:48.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16aa6a740 1 -- 172.21.15.5:0/257003402 wait complete. 2024-09-19T06:08:48.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.549+0000 7fa16aa6a740 20 client.16767 trim_cache size 0 max 16384 2024-09-19T06:08:48.557 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:08:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:08:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:08:48.557 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:08:48.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.550+0000 7fa168691640 1 -- 172.21.15.5:0/1568588101 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa14c0a3430 msgr2=0x7fa14c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:48.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.550+0000 7fa168691640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa14c0a3430 0x7fa14c0a58f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0096c0 tx=0x7fa15c005a20 comp rx=0 tx=0).stop 2024-09-19T06:08:48.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.550+0000 7fa168691640 1 -- 172.21.15.5:0/1568588101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa160102570 msgr2=0x7fa1601975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:48.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.550+0000 7fa168691640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa160102570 0x7fa1601975d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa15000d720 tx=0x7fa15000dbe0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.550+0000 7fa168691640 1 -- 172.21.15.5:0/1568588101 shutdown_connections 2024-09-19T06:08:48.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.550+0000 7fa168691640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102ef0 0x7fa160197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.550+0000 7fa168691640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa14c0a3430 0x7fa14c0a58f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.550+0000 7fa168691640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa160102570 0x7fa1601975d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.550+0000 7fa168691640 1 --2- 172.21.15.5:0/1568588101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160100e20 0x7fa160197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:48.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.550+0000 7fa168691640 1 -- 172.21.15.5:0/1568588101 >> 172.21.15.5:0/1568588101 conn(0x7fa1600fc7d0 msgr2=0x7fa1600fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:48.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.550+0000 7fa168691640 1 -- 172.21.15.5:0/1568588101 shutdown_connections 2024-09-19T06:08:48.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:48.550+0000 7fa168691640 1 -- 172.21.15.5:0/1568588101 wait complete. 2024-09-19T06:08:48.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 131280 2.79 MB/sec execute 134 sec latency 15.776 ms 2024-09-19T06:08:49.428 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:08:49.428 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:08:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:49.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 131642 2.78 MB/sec execute 135 sec latency 14.993 ms 2024-09-19T06:08:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:49 smithi005 ceph-mon[30955]: pgmap v168: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-09-19T06:08:50.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:49 smithi049 ceph-mon[34640]: pgmap v168: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-09-19T06:08:50.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.244+0000 7f17aa639640 1 -- 172.21.15.5:0/2382137061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a4103740 msgr2=0x7f17a4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:50.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.244+0000 7f17aa639640 1 --2- 172.21.15.5:0/2382137061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a4103740 0x7f17a4107b80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f178c009930 tx=0x7f178c02a420 comp rx=0 tx=0).stop 2024-09-19T06:08:50.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.244+0000 7f17aa639640 1 -- 172.21.15.5:0/2382137061 shutdown_connections 2024-09-19T06:08:50.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.244+0000 7f17aa639640 1 --2- 172.21.15.5:0/2382137061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a4108290 0x7f17a410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:50.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.244+0000 7f17aa639640 1 --2- 172.21.15.5:0/2382137061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a4103740 0x7f17a4107b80 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:50.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.244+0000 7f17aa639640 1 --2- 172.21.15.5:0/2382137061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17a4102d90 0x7f17a4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:50.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.244+0000 7f17aa639640 1 -- 172.21.15.5:0/2382137061 >> 172.21.15.5:0/2382137061 conn(0x7f17a40fc7d0 msgr2=0x7f17a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:50.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.244+0000 7f17aa639640 1 -- 172.21.15.5:0/2382137061 shutdown_connections 2024-09-19T06:08:50.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.244+0000 7f17aa639640 1 -- 172.21.15.5:0/2382137061 wait complete. 2024-09-19T06:08:50.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f17aa639640 1 Processor -- start 2024-09-19T06:08:50.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f17aa639640 1 -- start start 2024-09-19T06:08:50.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f17aa639640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17a4102d90 0x7f17a41009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:50.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f17aa639640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a4103740 0x7f17a4100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:50.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f17aa639640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a4108290 0x7f17a4101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:50.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f17aa639640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4111f40 con 0x7f17a4103740 2024-09-19T06:08:50.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f17aa639640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4111dc0 con 0x7f17a4102d90 2024-09-19T06:08:50.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f17aa639640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41120c0 con 0x7f17a4108290 2024-09-19T06:08:50.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f179bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a4103740 0x7f17a4100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f17a8bd6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a4108290 0x7f17a4101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f179bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a4103740 0x7f17a4100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54980/0 (socket says 172.21.15.5:54980) 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f179bfff640 1 -- 172.21.15.5:0/2052999925 learned_addr learned my addr 172.21.15.5:0/2052999925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.245+0000 7f17a3fff640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17a4102d90 0x7f17a41009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f17a8bd6640 1 -- 172.21.15.5:0/2052999925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17a4102d90 msgr2=0x7f17a41009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f17a8bd6640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17a4102d90 0x7f17a41009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f17a8bd6640 1 -- 172.21.15.5:0/2052999925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a4103740 msgr2=0x7f17a4100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f17a8bd6640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a4103740 0x7f17a4100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f17a8bd6640 1 -- 172.21.15.5:0/2052999925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a41a69c0 con 0x7f17a4108290 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f17a3fff640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17a4102d90 0x7f17a41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f179bfff640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a4103740 0x7f17a4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f17a8bd6640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a4108290 0x7f17a4101470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f179c009fd0 tx=0x7f179c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f17a1ffb640 1 -- 172.21.15.5:0/2052999925 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c00ce20 con 0x7f17a4108290 2024-09-19T06:08:50.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f17a1ffb640 1 -- 172.21.15.5:0/2052999925 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f179c005020 con 0x7f17a4108290 2024-09-19T06:08:50.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f17aa639640 1 -- 172.21.15.5:0/2052999925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a41a6cb0 con 0x7f17a4108290 2024-09-19T06:08:50.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f17a1ffb640 1 -- 172.21.15.5:0/2052999925 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c010690 con 0x7f17a4108290 2024-09-19T06:08:50.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.246+0000 7f17aa639640 1 -- 172.21.15.5:0/2052999925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a41a71f0 con 0x7f17a4108290 2024-09-19T06:08:50.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.247+0000 7f17aa639640 1 -- 172.21.15.5:0/2052999925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a4103170 con 0x7f17a4108290 2024-09-19T06:08:50.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.249+0000 7f17a1ffb640 1 -- 172.21.15.5:0/2052999925 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f179c0051d0 con 0x7f17a4108290 2024-09-19T06:08:50.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.249+0000 7f17a1ffb640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f17900a33b0 0x7f17900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:50.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.250+0000 7f17a3fff640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f17900a33b0 0x7f17900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:50.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.250+0000 7f17a1ffb640 1 -- 172.21.15.5:0/2052999925 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f179c0d03d0 con 0x7f17a4108290 2024-09-19T06:08:50.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.250+0000 7f17a3fff640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f17900a33b0 0x7f17900a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1794009c20 tx=0x7f1794005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.254+0000 7f17a1ffb640 1 -- 172.21.15.5:0/2052999925 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f179c09cdc0 con 0x7f17a4108290 2024-09-19T06:08:50.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:49 smithi157 ceph-mon[35346]: pgmap v168: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-09-19T06:08:50.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.495+0000 7f17aa639640 1 -- 172.21.15.5:0/2052999925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17a410a2a0 con 0x7f17a4108290 2024-09-19T06:08:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.496+0000 7f17a1ffb640 1 -- 172.21.15.5:0/2052999925 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11959 (secure 0 0 0) 0x7f179c09c6c0 con 0x7f17a4108290 2024-09-19T06:08:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:08:50.500 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-19T06:07:24:781633+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:07:24.019631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:08:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-19T06:08:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.499+0000 7f17aa639640 1 -- 172.21.15.5:0/2052999925 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f17900a33b0 msgr2=0x7f17900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.499+0000 7f17aa639640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f17900a33b0 0x7f17900a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1794009c20 tx=0x7f1794005f90 comp rx=0 tx=0).stop 2024-09-19T06:08:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.499+0000 7f17aa639640 1 -- 172.21.15.5:0/2052999925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a4108290 msgr2=0x7f17a4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.499+0000 7f17aa639640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a4108290 0x7f17a4101470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f179c009fd0 tx=0x7f179c00ef10 comp rx=0 tx=0).stop 2024-09-19T06:08:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.499+0000 7f17aa639640 1 -- 172.21.15.5:0/2052999925 shutdown_connections 2024-09-19T06:08:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.499+0000 7f17aa639640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f17900a33b0 0x7f17900a5870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.499+0000 7f17aa639640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a4108290 0x7f17a4101470 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.499+0000 7f17aa639640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a4103740 0x7f17a4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.499+0000 7f17aa639640 1 --2- 172.21.15.5:0/2052999925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17a4102d90 0x7f17a41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.499+0000 7f17aa639640 1 -- 172.21.15.5:0/2052999925 >> 172.21.15.5:0/2052999925 conn(0x7f17a40fc7d0 msgr2=0x7f17a4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.499+0000 7f17aa639640 1 -- 172.21.15.5:0/2052999925 shutdown_connections 2024-09-19T06:08:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:50.499+0000 7f17aa639640 1 -- 172.21.15.5:0/2052999925 wait complete. 2024-09-19T06:08:50.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 131903 2.76 MB/sec execute 136 sec latency 17.296 ms 2024-09-19T06:08:51.120 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:50 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2052999925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:08:51.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:50 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2052999925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:08:51.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:08:51.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:50 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2052999925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:08:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:51.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 132431 2.75 MB/sec execute 137 sec latency 39.868 ms 2024-09-19T06:08:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.947+0000 7f0cc33d7640 1 -- 172.21.15.5:0/4239812306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc102d90 msgr2=0x7f0cbc103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.947+0000 7f0cc33d7640 1 --2- 172.21.15.5:0/4239812306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc102d90 0x7f0cbc103170 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009930 tx=0x7f0cb802a420 comp rx=0 tx=0).stop 2024-09-19T06:08:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.947+0000 7f0cc33d7640 1 -- 172.21.15.5:0/4239812306 shutdown_connections 2024-09-19T06:08:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.947+0000 7f0cc33d7640 1 --2- 172.21.15.5:0/4239812306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cbc108290 0x7f0cbc10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.947+0000 7f0cc33d7640 1 --2- 172.21.15.5:0/4239812306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc103740 0x7f0cbc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.947+0000 7f0cc33d7640 1 --2- 172.21.15.5:0/4239812306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc102d90 0x7f0cbc103170 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.947+0000 7f0cc33d7640 1 -- 172.21.15.5:0/4239812306 >> 172.21.15.5:0/4239812306 conn(0x7f0cbc0fc7d0 msgr2=0x7f0cbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.947+0000 7f0cc33d7640 1 -- 172.21.15.5:0/4239812306 shutdown_connections 2024-09-19T06:08:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.947+0000 7f0cc33d7640 1 -- 172.21.15.5:0/4239812306 wait complete. 2024-09-19T06:08:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.948+0000 7f0cc33d7640 1 Processor -- start 2024-09-19T06:08:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.948+0000 7f0cc33d7640 1 -- start start 2024-09-19T06:08:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.948+0000 7f0cc33d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc102d90 0x7f0cbc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.948+0000 7f0cc33d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc103740 0x7f0cbc198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.948+0000 7f0cc33d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cbc108290 0x7f0cbc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc1173640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc102d90 0x7f0cbc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc33d7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc111f40 con 0x7f0cbc102d90 2024-09-19T06:08:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc33d7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc111dc0 con 0x7f0cbc103740 2024-09-19T06:08:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc33d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1120c0 con 0x7f0cbc108290 2024-09-19T06:08:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc1173640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc102d90 0x7f0cbc1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54998/0 (socket says 172.21.15.5:54998) 2024-09-19T06:08:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc1173640 1 -- 172.21.15.5:0/44091728 learned_addr learned my addr 172.21.15.5:0/44091728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc0972640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc103740 0x7f0cbc198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc1974640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cbc108290 0x7f0cbc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc1173640 1 -- 172.21.15.5:0/44091728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cbc108290 msgr2=0x7f0cbc19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc1173640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cbc108290 0x7f0cbc19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc1173640 1 -- 172.21.15.5:0/44091728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc103740 msgr2=0x7f0cbc198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc1173640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc103740 0x7f0cbc198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc1173640 1 -- 172.21.15.5:0/44091728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cbc19d490 con 0x7f0cbc102d90 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc0972640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc103740 0x7f0cbc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.949+0000 7f0cc1173640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc102d90 0x7f0cbc1984e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8005940 tx=0x7f0cb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.950+0000 7f0cb27fc640 1 -- 172.21.15.5:0/44091728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb803f030 con 0x7f0cbc102d90 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.950+0000 7f0cb27fc640 1 -- 172.21.15.5:0/44091728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cb80318a0 con 0x7f0cbc102d90 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.950+0000 7f0cb27fc640 1 -- 172.21.15.5:0/44091728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8035070 con 0x7f0cbc102d90 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.950+0000 7f0cc33d7640 1 -- 172.21.15.5:0/44091728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cbc19d720 con 0x7f0cbc102d90 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.950+0000 7f0cc33d7640 1 -- 172.21.15.5:0/44091728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cbc19dbb0 con 0x7f0cbc102d90 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc57b0740 1 Processor -- start 2024-09-19T06:08:51.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc57b0740 1 -- start start 2024-09-19T06:08:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cb27fc640 1 -- 172.21.15.5:0/44091728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0cb8042030 con 0x7f0cbc102d90 2024-09-19T06:08:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc57b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 0x564a48467090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc57b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a484675d0 0x564a48467a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc57b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a48472370 0x564a48474760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc57b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a483cd2f0 con 0x564a48468b40 2024-09-19T06:08:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc57b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564a48374cf0 con 0x564a48472370 2024-09-19T06:08:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc57b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564a483c92b0 con 0x564a484675d0 2024-09-19T06:08:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc1974640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 0x564a48467090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc1974640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 0x564a48467090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55008/0 (socket says 172.21.15.5:55008) 2024-09-19T06:08:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc1974640 1 -- 172.21.15.5:0/3632414407 learned_addr learned my addr 172.21.15.5:0/3632414407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc1173640 1 --2- 172.21.15.5:0/3632414407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a484675d0 0x564a48467a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.953+0000 7f0cc0972640 1 --2- 172.21.15.5:0/3632414407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a48472370 0x564a48474760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.954+0000 7f0cc1974640 1 -- 172.21.15.5:0/3632414407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a484675d0 msgr2=0x564a48467a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.954+0000 7f0cc1974640 1 --2- 172.21.15.5:0/3632414407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a484675d0 0x564a48467a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.954+0000 7f0cc1974640 1 -- 172.21.15.5:0/3632414407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a48472370 msgr2=0x564a48474760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.954+0000 7f0cc1974640 1 --2- 172.21.15.5:0/3632414407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a48472370 0x564a48474760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.954+0000 7f0cc1974640 1 -- 172.21.15.5:0/3632414407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a483593a0 con 0x564a48468b40 2024-09-19T06:08:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.954+0000 7f0cc1974640 1 --2- 172.21.15.5:0/3632414407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 0x564a48467090 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009f90 tx=0x7f0cb4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa78b4110b132d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.954+0000 7f0cb27fc640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0ca80a3430 0x7f0ca80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.954+0000 7f0ca77fe640 1 -- 172.21.15.5:0/3632414407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4011700 con 0x564a48468b40 2024-09-19T06:08:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.954+0000 7f0cc1173640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0ca80a3430 0x7f0ca80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.954+0000 7f0ca77fe640 1 -- 172.21.15.5:0/3632414407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cb4011d50 con 0x564a48468b40 2024-09-19T06:08:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.954+0000 7f0ca77fe640 1 -- 172.21.15.5:0/3632414407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4019d10 con 0x564a48468b40 2024-09-19T06:08:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.954+0000 7f0cb27fc640 1 -- 172.21.15.5:0/44091728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f0cb80f0380 con 0x7f0cbc102d90 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.955+0000 7f0cc57b0740 1 -- 172.21.15.5:0/3632414407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 msgr2=0x564a48467090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.955+0000 7f0cc1173640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0ca80a3430 0x7f0ca80a58f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80edb90 tx=0x7f0cb80f3ed0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.955+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/3632414407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 0x564a48467090 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009f90 tx=0x7f0cb4010860 comp rx=0 tx=0).stop 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.955+0000 7f0cc57b0740 1 -- 172.21.15.5:0/3632414407 shutdown_connections 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.955+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/3632414407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a48472370 0x564a48474760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.955+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/3632414407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a484675d0 0x564a48467a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.955+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/3632414407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 0x564a48467090 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.955+0000 7f0cc57b0740 1 -- 172.21.15.5:0/3632414407 >> 172.21.15.5:0/3632414407 conn(0x564a48430310 msgr2=0x564a48463c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.955+0000 7f0cc57b0740 1 -- 172.21.15.5:0/3632414407 shutdown_connections 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.955+0000 7f0cc57b0740 1 -- 172.21.15.5:0/3632414407 wait complete. 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc57b0740 1 -- start start 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc57b0740 1 Processor -- start 2024-09-19T06:08:51.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc57b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a484675d0 0x564a48471d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc57b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48468b40 0x564a4846efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc57b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48472370 0x564a4846f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc57b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a48475070 con 0x564a48472370 2024-09-19T06:08:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc57b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564a48388060 con 0x564a484675d0 2024-09-19T06:08:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc57b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564a484751f0 con 0x564a48468b40 2024-09-19T06:08:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc57b0740 10 client.0 _ll_register_callbacks cb 0x564a480db8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc0972640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48468b40 0x564a4846efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc1974640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48472370 0x564a4846f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc0972640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48468b40 0x564a4846efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40750/0 (socket says 172.21.15.5:40750) 2024-09-19T06:08:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.956+0000 7f0cc0972640 1 -- 172.21.15.5:0/124292722 learned_addr learned my addr 172.21.15.5:0/124292722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0cc1974640 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a484675d0 0x564a48471d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0cc0972640 1 -- 172.21.15.5:0/124292722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a484675d0 msgr2=0x564a48471d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0cc0972640 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a484675d0 0x564a48471d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0cc0972640 1 -- 172.21.15.5:0/124292722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48472370 msgr2=0x564a4846f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0cc0972640 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48472370 0x564a4846f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0cc0972640 1 -- 172.21.15.5:0/124292722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a4813e3c0 con 0x564a48468b40 2024-09-19T06:08:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0cc1974640 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48472370 0x564a4846f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0cc0972640 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48468b40 0x564a4846efd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0cac012d20 tx=0x7f0cac00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77eccbc356b9b445 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0ca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:08:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0ca4ff9640 1 -- 172.21.15.5:0/124292722 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac012110 con 0x564a48468b40 2024-09-19T06:08:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0ca4ff9640 1 -- 172.21.15.5:0/124292722 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f0cac004050 con 0x564a48468b40 2024-09-19T06:08:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0cc57b0740 1 -- 172.21.15.5:0/124292722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb4011d50 con 0x564a48468b40 2024-09-19T06:08:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0ca4ff9640 1 -- 172.21.15.5:0/124292722 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0cac0122b0 con 0x564a48468b40 2024-09-19T06:08:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0cc57b0740 10 client.26296 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0cc57b0740 1 -- 172.21.15.5:0/124292722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a48489360 con 0x564a48468b40 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.957+0000 7f0ca4ff9640 1 -- 172.21.15.5:0/124292722 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cb4011d50 con 0x564a48468b40 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.958+0000 7f0ca4ff9640 1 -- 172.21.15.5:0/124292722 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f0cac010430 con 0x564a48468b40 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.958+0000 7f0cc57b0740 10 client.26296 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.958+0000 7f0cc57b0740 10 client.26296 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.958+0000 7f0cc57b0740 20 client.26296 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.958+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564a4848a870 0x564a4848cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.958+0000 7f0cc57b0740 4 client.26296 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.958+0000 7f0cc57b0740 1 -- 172.21.15.5:0/124292722 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a483da860 con 0x564a4848a870 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.958+0000 7f0cc1974640 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564a4848a870 0x564a4848cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.958+0000 7f0cc1974640 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564a4848a870 0x564a4848cc90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d12062198592cedf server_cookie=1dacb49d74592e1e in_seq=0 out_seq=0 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.958+0000 7f0ca4ff9640 10 client.26296 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:08:51.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0ca4ff9640 1 -- 172.21.15.5:0/124292722 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a483da860 con 0x564a4848a870 2024-09-19T06:08:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0ca4ff9640 10 client.26296 handle_command_reply: tid=0 2024-09-19T06:08:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0cc57b0740 1 client.26296 shutdown 2024-09-19T06:08:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0cc57b0740 1 -- 172.21.15.5:0/124292722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48468b40 msgr2=0x564a4846efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48468b40 0x564a4846efd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0cac012d20 tx=0x7f0cac00f790 comp rx=0 tx=0).stop 2024-09-19T06:08:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0cc57b0740 1 -- 172.21.15.5:0/124292722 shutdown_connections 2024-09-19T06:08:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564a4848a870 0x564a4848cc90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48472370 0x564a4846f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48468b40 0x564a4846efd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/124292722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a484675d0 0x564a48471d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0cc57b0740 1 -- 172.21.15.5:0/124292722 >> 172.21.15.5:0/124292722 conn(0x564a48430310 msgr2=0x564a48469930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0ca4ff9640 0 client.26296 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:08:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0cc57b0740 1 -- 172.21.15.5:0/124292722 shutdown_connections 2024-09-19T06:08:51.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0cc57b0740 1 -- 172.21.15.5:0/124292722 wait complete. 2024-09-19T06:08:51.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.959+0000 7f0cc57b0740 20 client.26296 trim_cache size 0 max 16384 2024-09-19T06:08:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc57b0740 1 Processor -- start 2024-09-19T06:08:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc57b0740 1 -- start start 2024-09-19T06:08:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc57b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a484675d0 0x564a4846efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc57b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a48468b40 0x564a48486bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc57b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48472370 0x564a48492540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc57b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1ab670 con 0x564a48472370 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc57b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1ad1b0 con 0x564a48468b40 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc57b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1abd40 con 0x564a484675d0 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc1974640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48472370 0x564a48492540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc0972640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a48468b40 0x564a48486bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc0972640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a48468b40 0x564a48486bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33842/0 (socket says 172.21.15.5:33842) 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc0972640 1 -- 172.21.15.5:0/3071843303 learned_addr learned my addr 172.21.15.5:0/3071843303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.975+0000 7f0cc1974640 1 --2- 172.21.15.5:0/3071843303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a484675d0 0x564a4846efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.976+0000 7f0cc0972640 1 -- 172.21.15.5:0/3071843303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a484675d0 msgr2=0x564a4846efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.976+0000 7f0cc0972640 1 --2- 172.21.15.5:0/3071843303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a484675d0 0x564a4846efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.976+0000 7f0cc0972640 1 -- 172.21.15.5:0/3071843303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48472370 msgr2=0x564a48492540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.976+0000 7f0cc0972640 1 --2- 172.21.15.5:0/3071843303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48472370 0x564a48492540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.976+0000 7f0cc0972640 1 -- 172.21.15.5:0/3071843303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a4846a120 con 0x564a48468b40 2024-09-19T06:08:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.976+0000 7f0cc0972640 1 --2- 172.21.15.5:0/3071843303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a48468b40 0x564a48486bd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0cac000c00 tx=0x7f0cac005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f6c0756dcfc0608 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.976+0000 7f0ca6ffd640 1 -- 172.21.15.5:0/3071843303 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac000e60 con 0x564a48468b40 2024-09-19T06:08:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.976+0000 7f0ca6ffd640 1 -- 172.21.15.5:0/3071843303 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cac00fe80 con 0x564a48468b40 2024-09-19T06:08:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.976+0000 7f0ca6ffd640 1 -- 172.21.15.5:0/3071843303 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac029070 con 0x564a48468b40 2024-09-19T06:08:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.977+0000 7f0cc57b0740 1 -- 172.21.15.5:0/3071843303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a48468b40 msgr2=0x564a48486bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.977+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/3071843303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a48468b40 0x564a48486bd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0cac000c00 tx=0x7f0cac005d30 comp rx=0 tx=0).stop 2024-09-19T06:08:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.977+0000 7f0cc57b0740 1 -- 172.21.15.5:0/3071843303 shutdown_connections 2024-09-19T06:08:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.977+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/3071843303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48472370 0x564a48492540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.977+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/3071843303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a48468b40 0x564a48486bd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.977+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/3071843303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a484675d0 0x564a4846efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.977+0000 7f0cc57b0740 1 -- 172.21.15.5:0/3071843303 >> 172.21.15.5:0/3071843303 conn(0x564a48430310 msgr2=0x564a4846e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.977+0000 7f0cc57b0740 1 -- 172.21.15.5:0/3071843303 shutdown_connections 2024-09-19T06:08:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.977+0000 7f0cc57b0740 1 -- 172.21.15.5:0/3071843303 wait complete. 2024-09-19T06:08:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.977+0000 7f0cc57b0740 1 -- start start 2024-09-19T06:08:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.977+0000 7f0cc57b0740 1 Processor -- start 2024-09-19T06:08:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc57b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a484675d0 0x564a4847e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc57b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 0x564a4847ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc57b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48472370 0x564a484ed4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc57b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1acae0 con 0x564a48468b40 2024-09-19T06:08:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc57b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1ac410 con 0x564a484675d0 2024-09-19T06:08:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc57b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1a9650 con 0x564a48472370 2024-09-19T06:08:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc57b0740 10 client.0 _ll_register_callbacks cb 0x7f0cac009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc0972640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 0x564a4847ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc0972640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 0x564a4847ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55036/0 (socket says 172.21.15.5:55036) 2024-09-19T06:08:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc0972640 1 -- 172.21.15.5:0/921601445 learned_addr learned my addr 172.21.15.5:0/921601445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc1974640 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48472370 0x564a484ed4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc1974640 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a484675d0 0x564a4847e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc0972640 1 -- 172.21.15.5:0/921601445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48472370 msgr2=0x564a484ed4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc0972640 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48472370 0x564a484ed4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc0972640 1 -- 172.21.15.5:0/921601445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a484675d0 msgr2=0x564a4847e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:51.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc0972640 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a484675d0 0x564a4847e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc0972640 1 -- 172.21.15.5:0/921601445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a484edbc0 con 0x564a48468b40 2024-09-19T06:08:51.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.978+0000 7f0cc1974640 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48472370 0x564a484ed4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:51.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.979+0000 7f0cc0972640 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 0x564a4847ea80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0cac005bc0 tx=0x7f0cac0106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f948fb8067f1202 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:51.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.979+0000 7f0ca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.979+0000 7f0ca4ff9640 1 -- 172.21.15.5:0/921601445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac031020 con 0x564a48468b40 2024-09-19T06:08:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.979+0000 7f0cc57b0740 1 -- 172.21.15.5:0/921601445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564a483d4d70 con 0x564a48468b40 2024-09-19T06:08:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.979+0000 7f0ca4ff9640 1 -- 172.21.15.5:0/921601445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f0cac005250 con 0x564a48468b40 2024-09-19T06:08:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.979+0000 7f0ca4ff9640 1 -- 172.21.15.5:0/921601445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0cac029070 con 0x564a48468b40 2024-09-19T06:08:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.979+0000 7f0cc57b0740 10 client.16791 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.979+0000 7f0cc57b0740 1 -- 172.21.15.5:0/921601445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a484f4eb0 con 0x564a48468b40 2024-09-19T06:08:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.980+0000 7f0ca4ff9640 1 -- 172.21.15.5:0/921601445 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564a483d4d70 con 0x564a48468b40 2024-09-19T06:08:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.980+0000 7f0ca4ff9640 1 -- 172.21.15.5:0/921601445 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f0cac025420 con 0x564a48468b40 2024-09-19T06:08:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.980+0000 7f0cc57b0740 10 client.16791 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.980+0000 7f0cc57b0740 10 client.16791 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:08:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.980+0000 7f0cc57b0740 20 client.16791 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.980+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564a484f5570 0x564a484f7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.980+0000 7f0cc57b0740 4 client.16791 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:08:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.980+0000 7f0cc57b0740 1 -- 172.21.15.5:0/921601445 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a48351170 con 0x564a484f5570 2024-09-19T06:08:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.980+0000 7f0cc1974640 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564a484f5570 0x564a484f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc1974640 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564a484f5570 0x564a484f7990 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=58772ec28f626d10 server_cookie=e41d5e59f7c91024 in_seq=0 out_seq=0 2024-09-19T06:08:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0ca4ff9640 10 client.16791 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:08:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0ca4ff9640 1 -- 172.21.15.5:0/921601445 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a48351170 con 0x564a484f5570 2024-09-19T06:08:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0ca4ff9640 10 client.16791 handle_command_reply: tid=0 2024-09-19T06:08:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc57b0740 1 client.16791 shutdown 2024-09-19T06:08:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc57b0740 1 -- 172.21.15.5:0/921601445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 msgr2=0x564a4847ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 0x564a4847ea80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0cac005bc0 tx=0x7f0cac0106e0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc57b0740 1 -- 172.21.15.5:0/921601445 shutdown_connections 2024-09-19T06:08:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564a484f5570 0x564a484f7990 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0ca4ff9640 0 client.16791 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:08:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a48472370 0x564a484ed4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a48468b40 0x564a4847ea80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc57b0740 1 --2- 172.21.15.5:0/921601445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a484675d0 0x564a4847e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc57b0740 1 -- 172.21.15.5:0/921601445 >> 172.21.15.5:0/921601445 conn(0x564a48430310 msgr2=0x564a48462d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc57b0740 1 -- 172.21.15.5:0/921601445 shutdown_connections 2024-09-19T06:08:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc57b0740 1 -- 172.21.15.5:0/921601445 wait complete. 2024-09-19T06:08:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.981+0000 7f0cc57b0740 20 client.16791 trim_cache size 0 max 16384 2024-09-19T06:08:51.997 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:08:51.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.982+0000 7f0cc33d7640 1 -- 172.21.15.5:0/44091728 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0ca80a3430 msgr2=0x7f0ca80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.982+0000 7f0cc33d7640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0ca80a3430 0x7f0ca80a58f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80edb90 tx=0x7f0cb80f3ed0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.982+0000 7f0cc33d7640 1 -- 172.21.15.5:0/44091728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc102d90 msgr2=0x7f0cbc1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.982+0000 7f0cc33d7640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc102d90 0x7f0cbc1984e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8005940 tx=0x7f0cb80042f0 comp rx=0 tx=0).stop 2024-09-19T06:08:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.982+0000 7f0cc33d7640 1 -- 172.21.15.5:0/44091728 shutdown_connections 2024-09-19T06:08:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.982+0000 7f0cc33d7640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0ca80a3430 0x7f0ca80a58f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.982+0000 7f0cc33d7640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cbc108290 0x7f0cbc19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.982+0000 7f0cc33d7640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc103740 0x7f0cbc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.982+0000 7f0cc33d7640 1 --2- 172.21.15.5:0/44091728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc102d90 0x7f0cbc1984e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.982+0000 7f0cc33d7640 1 -- 172.21.15.5:0/44091728 >> 172.21.15.5:0/44091728 conn(0x7f0cbc0fc7d0 msgr2=0x7f0cbc105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.982+0000 7f0cc33d7640 1 -- 172.21.15.5:0/44091728 shutdown_connections 2024-09-19T06:08:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:51.982+0000 7f0cc33d7640 1 -- 172.21.15.5:0/44091728 wait complete. 2024-09-19T06:08:52.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:51 smithi005 ceph-mon[30955]: pgmap v169: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-09-19T06:08:52.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:51 smithi049 ceph-mon[34640]: pgmap v169: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-09-19T06:08:52.202 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:51 smithi157 ceph-mon[35346]: pgmap v169: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-09-19T06:08:52.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 132937 2.76 MB/sec execute 138 sec latency 19.364 ms 2024-09-19T06:08:52.905 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:08:53.186 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.716+0000 7f6297256640 1 -- 172.21.15.5:0/1397197412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6290102570 msgr2=0x7f62901029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.716+0000 7f6297256640 1 --2- 172.21.15.5:0/1397197412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6290102570 0x7f62901029b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f627c009900 tx=0x7f627c02a480 comp rx=0 tx=0).stop 2024-09-19T06:08:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.716+0000 7f6297256640 1 -- 172.21.15.5:0/1397197412 shutdown_connections 2024-09-19T06:08:53.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.716+0000 7f6297256640 1 --2- 172.21.15.5:0/1397197412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6290102ef0 0x7f629010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.716+0000 7f6297256640 1 --2- 172.21.15.5:0/1397197412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6290102570 0x7f62901029b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.716+0000 7f6297256640 1 --2- 172.21.15.5:0/1397197412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6290100e20 0x7f6290101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.716+0000 7f6297256640 1 -- 172.21.15.5:0/1397197412 >> 172.21.15.5:0/1397197412 conn(0x7f62900fc7d0 msgr2=0x7f62900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:53.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.716+0000 7f6297256640 1 -- 172.21.15.5:0/1397197412 shutdown_connections 2024-09-19T06:08:53.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.716+0000 7f6297256640 1 -- 172.21.15.5:0/1397197412 wait complete. 2024-09-19T06:08:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.717+0000 7f6297256640 1 Processor -- start 2024-09-19T06:08:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.717+0000 7f6297256640 1 -- start start 2024-09-19T06:08:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f6297256640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6290100e20 0x7f6290199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f6297256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6290102570 0x7f62901997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f6297256640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6290102ef0 0x7f62901a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f6297256640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6290110e30 con 0x7f6290102570 2024-09-19T06:08:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f6297256640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6290110cb0 con 0x7f6290100e20 2024-09-19T06:08:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f6297256640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6290110fb0 con 0x7f6290102ef0 2024-09-19T06:08:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f628ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6290102570 0x7f62901997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f6294ff2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6290100e20 0x7f6290199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f62957f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6290102ef0 0x7f62901a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f628ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6290102570 0x7f62901997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55056/0 (socket says 172.21.15.5:55056) 2024-09-19T06:08:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f628ffff640 1 -- 172.21.15.5:0/3001131201 learned_addr learned my addr 172.21.15.5:0/3001131201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f62957f3640 1 -- 172.21.15.5:0/3001131201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6290100e20 msgr2=0x7f6290199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f62957f3640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6290100e20 0x7f6290199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f62957f3640 1 -- 172.21.15.5:0/3001131201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6290102570 msgr2=0x7f62901997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f62957f3640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6290102570 0x7f62901997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f62957f3640 1 -- 172.21.15.5:0/3001131201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62901a4820 con 0x7f6290102ef0 2024-09-19T06:08:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f628ffff640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6290102570 0x7f62901997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.718+0000 7f62957f3640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6290102ef0 0x7f62901a42e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f628800da10 tx=0x7f628800ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.719+0000 7f628dffb640 1 -- 172.21.15.5:0/3001131201 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628800bd70 con 0x7f6290102ef0 2024-09-19T06:08:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.719+0000 7f628dffb640 1 -- 172.21.15.5:0/3001131201 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6288003e20 con 0x7f6290102ef0 2024-09-19T06:08:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.719+0000 7f6297256640 1 -- 172.21.15.5:0/3001131201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62901a49f0 con 0x7f6290102ef0 2024-09-19T06:08:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.719+0000 7f628dffb640 1 -- 172.21.15.5:0/3001131201 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62880106c0 con 0x7f6290102ef0 2024-09-19T06:08:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.719+0000 7f6297256640 1 -- 172.21.15.5:0/3001131201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62901a4f00 con 0x7f6290102ef0 2024-09-19T06:08:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.721+0000 7f628dffb640 1 -- 172.21.15.5:0/3001131201 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6288003970 con 0x7f6290102ef0 2024-09-19T06:08:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.721+0000 7f629962f740 1 Processor -- start 2024-09-19T06:08:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f629962f740 1 -- start start 2024-09-19T06:08:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f629962f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f28f50 0x561db3f274a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f629962f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f279e0 0x561db3f27e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f629962f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 0x561db3f34b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f629962f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561db3e892f0 con 0x561db3f28f50 2024-09-19T06:08:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f629962f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561db3e30cf0 con 0x561db3f279e0 2024-09-19T06:08:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f629962f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561db3e852b0 con 0x561db3f32780 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f628ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f28f50 0x561db3f274a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f628ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f28f50 0x561db3f274a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55068/0 (socket says 172.21.15.5:55068) 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f628ffff640 1 -- 172.21.15.5:0/924847129 learned_addr learned my addr 172.21.15.5:0/924847129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f6294ff2640 1 --2- 172.21.15.5:0/924847129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f279e0 0x561db3f27e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f628ffff640 1 --2- 172.21.15.5:0/924847129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 0x561db3f34b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f6294ff2640 1 -- 172.21.15.5:0/924847129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 msgr2=0x561db3f34b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.722+0000 7f6294ff2640 1 --2- 172.21.15.5:0/924847129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 0x561db3f34b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.723+0000 7f6294ff2640 1 -- 172.21.15.5:0/924847129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f28f50 msgr2=0x561db3f274a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.723+0000 7f6294ff2640 1 --2- 172.21.15.5:0/924847129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f28f50 0x561db3f274a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.723+0000 7f6294ff2640 1 -- 172.21.15.5:0/924847129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db3e153a0 con 0x561db3f279e0 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.723+0000 7f628ffff640 1 --2- 172.21.15.5:0/924847129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f28f50 0x561db3f274a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.723+0000 7f628dffb640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f62800a3430 0x7f62800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.723+0000 7f628ffff640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f62800a3430 0x7f62800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.723+0000 7f6294ff2640 1 --2- 172.21.15.5:0/924847129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f279e0 0x561db3f27e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f628400bbf0 tx=0x7f62840107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c9fe9b15ad129e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.723+0000 7f628dffb640 1 -- 172.21.15.5:0/3001131201 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f62880cffe0 con 0x7f6290102ef0 2024-09-19T06:08:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.723+0000 7f627affd640 1 -- 172.21.15.5:0/924847129 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284004030 con 0x561db3f279e0 2024-09-19T06:08:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.723+0000 7f627affd640 1 -- 172.21.15.5:0/924847129 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62840035f0 con 0x561db3f279e0 2024-09-19T06:08:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.723+0000 7f627affd640 1 -- 172.21.15.5:0/924847129 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62840116a0 con 0x561db3f279e0 2024-09-19T06:08:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.724+0000 7f629962f740 1 -- 172.21.15.5:0/924847129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f279e0 msgr2=0x561db3f27e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.724+0000 7f629962f740 1 --2- 172.21.15.5:0/924847129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f279e0 0x561db3f27e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f628400bbf0 tx=0x7f62840107a0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.724+0000 7f628ffff640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f62800a3430 0x7f62800a58f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f627c005c40 tx=0x7f627c040040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.724+0000 7f629962f740 1 -- 172.21.15.5:0/924847129 shutdown_connections 2024-09-19T06:08:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.724+0000 7f629962f740 1 --2- 172.21.15.5:0/924847129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 0x561db3f34b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.724+0000 7f629962f740 1 --2- 172.21.15.5:0/924847129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f279e0 0x561db3f27e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.724+0000 7f629962f740 1 --2- 172.21.15.5:0/924847129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f28f50 0x561db3f274a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.724+0000 7f629962f740 1 -- 172.21.15.5:0/924847129 >> 172.21.15.5:0/924847129 conn(0x561db3eec310 msgr2=0x561db3f24070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.724+0000 7f629962f740 1 -- 172.21.15.5:0/924847129 shutdown_connections 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.724+0000 7f629962f740 1 -- 172.21.15.5:0/924847129 wait complete. 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.724+0000 7f629962f740 1 -- start start 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f629962f740 1 Processor -- start 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f629962f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f32150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f629962f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f28f50 0x561db3f2f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f629962f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f32780 0x561db3f2f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f629962f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561db3f35480 con 0x561db3f279e0 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f629962f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561db3e44060 con 0x561db3f32780 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f629962f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561db3f35600 con 0x561db3f28f50 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f629962f740 10 client.0 _ll_register_callbacks cb 0x561db3b978c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f628ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f28f50 0x561db3f2f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f62957f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f32780 0x561db3f2f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f6294ff2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f32150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f6294ff2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f32150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55078/0 (socket says 172.21.15.5:55078) 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f628ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f28f50 0x561db3f2f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40794/0 (socket says 172.21.15.5:40794) 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.725+0000 7f6294ff2640 1 -- 172.21.15.5:0/3435886235 learned_addr learned my addr 172.21.15.5:0/3435886235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f628ffff640 1 -- 172.21.15.5:0/3435886235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f32780 msgr2=0x561db3f2f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f628ffff640 1 --2- 172.21.15.5:0/3435886235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f32780 0x561db3f2f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f628ffff640 1 -- 172.21.15.5:0/3435886235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 msgr2=0x561db3f32150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f628ffff640 1 --2- 172.21.15.5:0/3435886235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f32150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f628ffff640 1 -- 172.21.15.5:0/3435886235 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561db3bfa3c0 con 0x561db3f28f50 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f6294ff2640 1 --2- 172.21.15.5:0/3435886235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f32150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f628ffff640 1 --2- 172.21.15.5:0/3435886235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f28f50 0x561db3f2f400 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x561db3f28480 tx=0x7f627c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d863e9262d338715 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f626ffff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f626ffff640 1 -- 172.21.15.5:0/3435886235 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c044070 con 0x561db3f28f50 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f626ffff640 1 -- 172.21.15.5:0/3435886235 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f627c004960 con 0x561db3f28f50 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f629962f740 1 -- 172.21.15.5:0/3435886235 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62840035f0 con 0x561db3f28f50 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f626ffff640 1 -- 172.21.15.5:0/3435886235 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f627c035040 con 0x561db3f28f50 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f629962f740 10 client.26314 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f629962f740 1 -- 172.21.15.5:0/3435886235 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561db3f40f20 con 0x561db3f28f50 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.726+0000 7f626ffff640 1 -- 172.21.15.5:0/3435886235 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62840035f0 con 0x561db3f28f50 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.727+0000 7f626ffff640 1 -- 172.21.15.5:0/3435886235 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f627c003c80 con 0x561db3f28f50 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.727+0000 7f629962f740 10 client.26314 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.727+0000 7f629962f740 10 client.26314 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.727+0000 7f629962f740 20 client.26314 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.727+0000 7f629962f740 1 --2- 172.21.15.5:0/3435886235 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561db3f42380 0x561db3f447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.727+0000 7f629962f740 4 client.26314 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.727+0000 7f629962f740 1 -- 172.21.15.5:0/3435886235 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561db3e96860 con 0x561db3f42380 2024-09-19T06:08:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.727+0000 7f6294ff2640 1 --2- 172.21.15.5:0/3435886235 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561db3f42380 0x561db3f447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f626ffff640 10 client.26314 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:08:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f6294ff2640 1 --2- 172.21.15.5:0/3435886235 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561db3f42380 0x561db3f447f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e07fa569368e43c4 server_cookie=281f5d20a5b93959 in_seq=0 out_seq=0 2024-09-19T06:08:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f626ffff640 1 -- 172.21.15.5:0/3435886235 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561db3e96860 con 0x561db3f42380 2024-09-19T06:08:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f626ffff640 10 client.26314 handle_command_reply: tid=0 2024-09-19T06:08:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f629962f740 1 client.26314 shutdown 2024-09-19T06:08:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f629962f740 1 -- 172.21.15.5:0/3435886235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f28f50 msgr2=0x561db3f2f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f629962f740 1 --2- 172.21.15.5:0/3435886235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f28f50 0x561db3f2f400 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x561db3f28480 tx=0x7f627c009580 comp rx=0 tx=0).stop 2024-09-19T06:08:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f629962f740 1 -- 172.21.15.5:0/3435886235 shutdown_connections 2024-09-19T06:08:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f629962f740 1 --2- 172.21.15.5:0/3435886235 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561db3f42380 0x561db3f447f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f629962f740 1 --2- 172.21.15.5:0/3435886235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f32780 0x561db3f2f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f629962f740 1 --2- 172.21.15.5:0/3435886235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f28f50 0x561db3f2f400 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f626ffff640 0 client.26314 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:08:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f629962f740 1 --2- 172.21.15.5:0/3435886235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f32150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.728+0000 7f629962f740 1 -- 172.21.15.5:0/3435886235 >> 172.21.15.5:0/3435886235 conn(0x561db3eec310 msgr2=0x561db3f29d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.729+0000 7f629962f740 1 -- 172.21.15.5:0/3435886235 shutdown_connections 2024-09-19T06:08:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.729+0000 7f629962f740 1 -- 172.21.15.5:0/3435886235 wait complete. 2024-09-19T06:08:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.729+0000 7f629962f740 20 client.26314 trim_cache size 0 max 16384 2024-09-19T06:08:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.743+0000 7f629962f740 1 Processor -- start 2024-09-19T06:08:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.743+0000 7f629962f740 1 -- start start 2024-09-19T06:08:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.743+0000 7f629962f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f1ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.743+0000 7f629962f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f28f50 0x561db3f2a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.743+0000 7f629962f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 0x561db3f2aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f629962f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62901ab5f0 con 0x561db3f279e0 2024-09-19T06:08:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f629962f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f62901ad130 con 0x561db3f28f50 2024-09-19T06:08:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f629962f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f62901abcc0 con 0x561db3f32780 2024-09-19T06:08:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f628ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f28f50 0x561db3f2a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f6294ff2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f1ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f628ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f28f50 0x561db3f2a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33884/0 (socket says 172.21.15.5:33884) 2024-09-19T06:08:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f628ffff640 1 -- 172.21.15.5:0/667859058 learned_addr learned my addr 172.21.15.5:0/667859058 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f62957f3640 1 --2- 172.21.15.5:0/667859058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 0x561db3f2aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f628ffff640 1 -- 172.21.15.5:0/667859058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 msgr2=0x561db3f2aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f628ffff640 1 --2- 172.21.15.5:0/667859058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 0x561db3f2aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f628ffff640 1 -- 172.21.15.5:0/667859058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 msgr2=0x561db3f1ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f628ffff640 1 --2- 172.21.15.5:0/667859058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f1ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f628ffff640 1 -- 172.21.15.5:0/667859058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db3f44e70 con 0x561db3f28f50 2024-09-19T06:08:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f628ffff640 1 --2- 172.21.15.5:0/667859058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f28f50 0x561db3f2a530 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f627c055530 tx=0x7f627c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb03a6828d4649f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.744+0000 7f627a7fc640 1 -- 172.21.15.5:0/667859058 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c044070 con 0x561db3f28f50 2024-09-19T06:08:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.745+0000 7f627a7fc640 1 -- 172.21.15.5:0/667859058 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f627c0552d0 con 0x561db3f28f50 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.745+0000 7f627a7fc640 1 -- 172.21.15.5:0/667859058 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c004c40 con 0x561db3f28f50 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.745+0000 7f629962f740 1 -- 172.21.15.5:0/667859058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f28f50 msgr2=0x561db3f2a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.745+0000 7f629962f740 1 --2- 172.21.15.5:0/667859058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f28f50 0x561db3f2a530 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f627c055530 tx=0x7f627c005fd0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.745+0000 7f629962f740 1 -- 172.21.15.5:0/667859058 shutdown_connections 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.745+0000 7f629962f740 1 --2- 172.21.15.5:0/667859058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 0x561db3f2aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.745+0000 7f629962f740 1 --2- 172.21.15.5:0/667859058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f28f50 0x561db3f2a530 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.745+0000 7f629962f740 1 --2- 172.21.15.5:0/667859058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f1ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.745+0000 7f629962f740 1 -- 172.21.15.5:0/667859058 >> 172.21.15.5:0/667859058 conn(0x561db3eec310 msgr2=0x561db3f1e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.745+0000 7f629962f740 1 -- 172.21.15.5:0/667859058 shutdown_connections 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.745+0000 7f629962f740 1 -- 172.21.15.5:0/667859058 wait complete. 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.746+0000 7f629962f740 1 -- start start 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.746+0000 7f629962f740 1 Processor -- start 2024-09-19T06:08:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.746+0000 7f629962f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f3b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.746+0000 7f629962f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f28f50 0x561db3f36470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.746+0000 7f629962f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 0x561db3f36a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.746+0000 7f629962f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62901aca60 con 0x561db3f279e0 2024-09-19T06:08:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.746+0000 7f629962f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f62901ac390 con 0x561db3f28f50 2024-09-19T06:08:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.746+0000 7f629962f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f62901a95d0 con 0x561db3f32780 2024-09-19T06:08:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f629962f740 10 client.0 _ll_register_callbacks cb 0x7f627c0317e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f6294ff2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f3b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f6294ff2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f3b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55082/0 (socket says 172.21.15.5:55082) 2024-09-19T06:08:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f6294ff2640 1 -- 172.21.15.5:0/4129470990 learned_addr learned my addr 172.21.15.5:0/4129470990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f628ffff640 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f28f50 0x561db3f36470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f62957f3640 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 0x561db3f36a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f6294ff2640 1 -- 172.21.15.5:0/4129470990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 msgr2=0x561db3f36a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f6294ff2640 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 0x561db3f36a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f6294ff2640 1 -- 172.21.15.5:0/4129470990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f28f50 msgr2=0x561db3f36470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f6294ff2640 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f28f50 0x561db3f36470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f6294ff2640 1 -- 172.21.15.5:0/4129470990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561db3f372b0 con 0x561db3f279e0 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f6294ff2640 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f3b140 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6284010ca0 tx=0x7f6284005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1b547ce825c2ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f626ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f626ffff640 1 -- 172.21.15.5:0/4129470990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284005040 con 0x561db3f279e0 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f626ffff640 1 -- 172.21.15.5:0/4129470990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f62840051e0 con 0x561db3f279e0 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.747+0000 7f629962f740 1 -- 172.21.15.5:0/4129470990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f627c0552d0 con 0x561db3f279e0 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.748+0000 7f626ffff640 1 -- 172.21.15.5:0/4129470990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f62840053e0 con 0x561db3f279e0 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.748+0000 7f629962f740 10 client.16809 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.748+0000 7f629962f740 1 -- 172.21.15.5:0/4129470990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561db3fad1c0 con 0x561db3f279e0 2024-09-19T06:08:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.748+0000 7f626ffff640 1 -- 172.21.15.5:0/4129470990 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f627c0552d0 con 0x561db3f279e0 2024-09-19T06:08:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.748+0000 7f626ffff640 1 -- 172.21.15.5:0/4129470990 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f62840115e0 con 0x561db3f279e0 2024-09-19T06:08:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.748+0000 7f629962f740 10 client.16809 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.748+0000 7f629962f740 10 client.16809 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:08:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.748+0000 7f629962f740 20 client.16809 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.748+0000 7f629962f740 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561db3fad7d0 0x561db3fafbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.748+0000 7f629962f740 4 client.16809 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:08:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.748+0000 7f629962f740 1 -- 172.21.15.5:0/4129470990 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561db3e90d70 con 0x561db3fad7d0 2024-09-19T06:08:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.748+0000 7f628ffff640 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561db3fad7d0 0x561db3fafbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f628ffff640 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561db3fad7d0 0x561db3fafbf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=148136358c1762a6 server_cookie=566c0c4145826eae in_seq=0 out_seq=0 2024-09-19T06:08:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f626ffff640 10 client.16809 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:08:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f626ffff640 1 -- 172.21.15.5:0/4129470990 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561db3e90d70 con 0x561db3fad7d0 2024-09-19T06:08:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f626ffff640 10 client.16809 handle_command_reply: tid=0 2024-09-19T06:08:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f629962f740 1 client.16809 shutdown 2024-09-19T06:08:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f629962f740 1 -- 172.21.15.5:0/4129470990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 msgr2=0x561db3f3b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f629962f740 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f3b140 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6284010ca0 tx=0x7f6284005850 comp rx=0 tx=0).stop 2024-09-19T06:08:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f629962f740 1 -- 172.21.15.5:0/4129470990 shutdown_connections 2024-09-19T06:08:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f629962f740 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561db3fad7d0 0x561db3fafbf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f629962f740 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561db3f32780 0x561db3f36a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f629962f740 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561db3f28f50 0x561db3f36470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f626ffff640 0 client.16809 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:08:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f629962f740 1 --2- 172.21.15.5:0/4129470990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561db3f279e0 0x561db3f3b140 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f629962f740 1 -- 172.21.15.5:0/4129470990 >> 172.21.15.5:0/4129470990 conn(0x561db3eec310 msgr2=0x561db3f1ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f629962f740 1 -- 172.21.15.5:0/4129470990 shutdown_connections 2024-09-19T06:08:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f629962f740 1 -- 172.21.15.5:0/4129470990 wait complete. 2024-09-19T06:08:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.749+0000 7f629962f740 20 client.16809 trim_cache size 0 max 16384 2024-09-19T06:08:53.757 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:08:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.750+0000 7f6297256640 1 -- 172.21.15.5:0/3001131201 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f62800a3430 msgr2=0x7f62800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.750+0000 7f6297256640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f62800a3430 0x7f62800a58f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f627c005c40 tx=0x7f627c040040 comp rx=0 tx=0).stop 2024-09-19T06:08:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.750+0000 7f6297256640 1 -- 172.21.15.5:0/3001131201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6290102ef0 msgr2=0x7f62901a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.750+0000 7f6297256640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6290102ef0 0x7f62901a42e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f628800da10 tx=0x7f628800ded0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.750+0000 7f6297256640 1 -- 172.21.15.5:0/3001131201 shutdown_connections 2024-09-19T06:08:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.750+0000 7f6297256640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f62800a3430 0x7f62800a58f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.750+0000 7f6297256640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6290102ef0 0x7f62901a42e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.750+0000 7f6297256640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6290102570 0x7f62901997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.750+0000 7f6297256640 1 --2- 172.21.15.5:0/3001131201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6290100e20 0x7f6290199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.750+0000 7f6297256640 1 -- 172.21.15.5:0/3001131201 >> 172.21.15.5:0/3001131201 conn(0x7f62900fc7d0 msgr2=0x7f62900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.750+0000 7f6297256640 1 -- 172.21.15.5:0/3001131201 shutdown_connections 2024-09-19T06:08:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:53.750+0000 7f6297256640 1 -- 172.21.15.5:0/3001131201 wait complete. 2024-09-19T06:08:53.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 133572 2.75 MB/sec execute 139 sec latency 20.205 ms 2024-09-19T06:08:54.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:53 smithi049 ceph-mon[34640]: pgmap v170: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 2.0 MiB/s wr, 188 op/s 2024-09-19T06:08:54.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:53 smithi005 ceph-mon[30955]: pgmap v170: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 2.0 MiB/s wr, 188 op/s 2024-09-19T06:08:54.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:53 smithi157 ceph-mon[35346]: pgmap v170: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 2.0 MiB/s wr, 188 op/s 2024-09-19T06:08:54.496 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:08:54.785 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:54.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 134171 2.77 MB/sec execute 140 sec latency 15.954 ms 2024-09-19T06:08:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.283+0000 7fa56dd7c640 1 -- 172.21.15.5:0/2035951791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa568102d30 msgr2=0x7fa568103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.283+0000 7fa56dd7c640 1 --2- 172.21.15.5:0/2035951791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa568102d30 0x7fa568103110 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa558009930 tx=0x7fa55802a420 comp rx=0 tx=0).stop 2024-09-19T06:08:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.283+0000 7fa56dd7c640 1 -- 172.21.15.5:0/2035951791 shutdown_connections 2024-09-19T06:08:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.283+0000 7fa56dd7c640 1 --2- 172.21.15.5:0/2035951791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5681082a0 0x7fa56810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.283+0000 7fa56dd7c640 1 --2- 172.21.15.5:0/2035951791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5681036e0 0x7fa568107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.283+0000 7fa56dd7c640 1 --2- 172.21.15.5:0/2035951791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa568102d30 0x7fa568103110 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.283+0000 7fa56dd7c640 1 -- 172.21.15.5:0/2035951791 >> 172.21.15.5:0/2035951791 conn(0x7fa5680fc7d0 msgr2=0x7fa5680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.283+0000 7fa56dd7c640 1 -- 172.21.15.5:0/2035951791 shutdown_connections 2024-09-19T06:08:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.283+0000 7fa56dd7c640 1 -- 172.21.15.5:0/2035951791 wait complete. 2024-09-19T06:08:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.284+0000 7fa56dd7c640 1 Processor -- start 2024-09-19T06:08:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.284+0000 7fa56dd7c640 1 -- start start 2024-09-19T06:08:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa56dd7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa568102d30 0x7fa568198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa56dd7c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5681036e0 0x7fa568198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa56dd7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5681082a0 0x7fa56819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa56dd7c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa568111f50 con 0x7fa5681082a0 2024-09-19T06:08:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa56dd7c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa568111dd0 con 0x7fa5681036e0 2024-09-19T06:08:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa56dd7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681120d0 con 0x7fa568102d30 2024-09-19T06:08:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa567fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5681082a0 0x7fa56819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa566ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5681036e0 0x7fa568198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa5677fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa568102d30 0x7fa568198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa567fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5681082a0 0x7fa56819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34640/0 (socket says 172.21.15.5:34640) 2024-09-19T06:08:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa567fff640 1 -- 172.21.15.5:0/4252374199 learned_addr learned my addr 172.21.15.5:0/4252374199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa566ffd640 1 -- 172.21.15.5:0/4252374199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa568102d30 msgr2=0x7fa568198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa566ffd640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa568102d30 0x7fa568198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa566ffd640 1 -- 172.21.15.5:0/4252374199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5681082a0 msgr2=0x7fa56819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa566ffd640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5681082a0 0x7fa56819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa566ffd640 1 -- 172.21.15.5:0/4252374199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56819d4c0 con 0x7fa5681036e0 2024-09-19T06:08:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.285+0000 7fa567fff640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5681082a0 0x7fa56819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.286+0000 7fa566ffd640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5681036e0 0x7fa568198a50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa56806a370 tx=0x7fa55c00dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.286+0000 7fa564ff9640 1 -- 172.21.15.5:0/4252374199 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c017070 con 0x7fa5681036e0 2024-09-19T06:08:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.286+0000 7fa56dd7c640 1 -- 172.21.15.5:0/4252374199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56819d7b0 con 0x7fa5681036e0 2024-09-19T06:08:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.286+0000 7fa564ff9640 1 -- 172.21.15.5:0/4252374199 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa55c0099a0 con 0x7fa5681036e0 2024-09-19T06:08:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.286+0000 7fa564ff9640 1 -- 172.21.15.5:0/4252374199 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0047a0 con 0x7fa5681036e0 2024-09-19T06:08:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.286+0000 7fa56dd7c640 1 -- 172.21.15.5:0/4252374199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5681a5090 con 0x7fa5681036e0 2024-09-19T06:08:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.288+0000 7fa570155740 1 Processor -- start 2024-09-19T06:08:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.288+0000 7fa570155740 1 -- start start 2024-09-19T06:08:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa570155740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa570155740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 0x560da41baa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa570155740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa570155740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560da41202f0 con 0x560da41c5370 2024-09-19T06:08:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa570155740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560da40c7cf0 con 0x560da41ba5d0 2024-09-19T06:08:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa570155740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560da411c2b0 con 0x560da41bbb40 2024-09-19T06:08:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa564ff9640 1 -- 172.21.15.5:0/4252374199 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa55c009b50 con 0x7fa5681036e0 2024-09-19T06:08:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa5677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 0x560da41baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa5677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 0x560da41baa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37618/0 (socket says 172.21.15.5:37618) 2024-09-19T06:08:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa5677fe640 1 -- 172.21.15.5:0/720262296 learned_addr learned my addr 172.21.15.5:0/720262296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa567fff640 1 --2- 172.21.15.5:0/720262296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa566ffd640 1 --2- 172.21.15.5:0/720262296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa5677fe640 1 -- 172.21.15.5:0/720262296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 msgr2=0x560da41ba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa5677fe640 1 --2- 172.21.15.5:0/720262296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41ba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa5677fe640 1 -- 172.21.15.5:0/720262296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 msgr2=0x560da41c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa5677fe640 1 --2- 172.21.15.5:0/720262296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa5677fe640 1 -- 172.21.15.5:0/720262296 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da40ac3a0 con 0x560da41ba5d0 2024-09-19T06:08:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.289+0000 7fa567fff640 1 --2- 172.21.15.5:0/720262296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.290+0000 7fa5677fe640 1 --2- 172.21.15.5:0/720262296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 0x560da41baa10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa55802a940 tx=0x7fa558031ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7edd207dd3c9ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.290+0000 7fa564ff9640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa5500a33b0 0x7fa5500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.290+0000 7fa54dffb640 1 -- 172.21.15.5:0/720262296 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558040070 con 0x560da41ba5d0 2024-09-19T06:08:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.290+0000 7fa54dffb640 1 -- 172.21.15.5:0/720262296 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa558004030 con 0x560da41ba5d0 2024-09-19T06:08:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.290+0000 7fa54dffb640 1 -- 172.21.15.5:0/720262296 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558035040 con 0x560da41ba5d0 2024-09-19T06:08:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.290+0000 7fa567fff640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa5500a33b0 0x7fa5500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.290+0000 7fa564ff9640 1 -- 172.21.15.5:0/4252374199 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa55c0cf8e0 con 0x7fa5681036e0 2024-09-19T06:08:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.291+0000 7fa570155740 1 -- 172.21.15.5:0/720262296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 msgr2=0x560da41baa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.291+0000 7fa567fff640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa5500a33b0 0x7fa5500a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5600049d0 tx=0x7fa560009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.291+0000 7fa570155740 1 --2- 172.21.15.5:0/720262296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 0x560da41baa10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa55802a940 tx=0x7fa558031ba0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.291+0000 7fa570155740 1 -- 172.21.15.5:0/720262296 shutdown_connections 2024-09-19T06:08:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.291+0000 7fa570155740 1 --2- 172.21.15.5:0/720262296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.291+0000 7fa570155740 1 --2- 172.21.15.5:0/720262296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 0x560da41baa10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.291+0000 7fa570155740 1 --2- 172.21.15.5:0/720262296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.291+0000 7fa570155740 1 -- 172.21.15.5:0/720262296 >> 172.21.15.5:0/720262296 conn(0x560da4183310 msgr2=0x560da41b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.291+0000 7fa570155740 1 -- 172.21.15.5:0/720262296 shutdown_connections 2024-09-19T06:08:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.291+0000 7fa570155740 1 -- 172.21.15.5:0/720262296 wait complete. 2024-09-19T06:08:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.291+0000 7fa570155740 1 -- start start 2024-09-19T06:08:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.291+0000 7fa570155740 1 Processor -- start 2024-09-19T06:08:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa570155740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41ba5d0 0x560da41c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa570155740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41bbb40 0x560da41c2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa570155740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa570155740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560da41cc420 con 0x560da41c5370 2024-09-19T06:08:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa570155740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560da40db060 con 0x560da41bbb40 2024-09-19T06:08:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa570155740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560da41cc5a0 con 0x560da41ba5d0 2024-09-19T06:08:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa570155740 10 client.0 _ll_register_callbacks cb 0x560da3e2e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa567fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa567fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41c2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34658/0 (socket says 172.21.15.5:34658) 2024-09-19T06:08:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa567fff640 1 -- 172.21.15.5:0/1504059882 learned_addr learned my addr 172.21.15.5:0/1504059882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa566ffd640 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41bbb40 0x560da41c2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa5677fe640 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41ba5d0 0x560da41c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa567fff640 1 -- 172.21.15.5:0/1504059882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41ba5d0 msgr2=0x560da41c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa567fff640 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41ba5d0 0x560da41c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa567fff640 1 -- 172.21.15.5:0/1504059882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41bbb40 msgr2=0x560da41c2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa567fff640 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41bbb40 0x560da41c2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa567fff640 1 -- 172.21.15.5:0/1504059882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da3e913c0 con 0x560da41c5370 2024-09-19T06:08:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa566ffd640 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41bbb40 0x560da41c2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa567fff640 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41c2640 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa560012260 tx=0x7fa560004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469082c2d8b0a2b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.292+0000 7fa546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa546ffd640 1 -- 172.21.15.5:0/1504059882 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560011c50 con 0x560da41c5370 2024-09-19T06:08:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa546ffd640 1 -- 172.21.15.5:0/1504059882 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa560011df0 con 0x560da41c5370 2024-09-19T06:08:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa570155740 1 -- 172.21.15.5:0/1504059882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa558004030 con 0x560da41c5370 2024-09-19T06:08:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa546ffd640 1 -- 172.21.15.5:0/1504059882 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa560029440 con 0x560da41c5370 2024-09-19T06:08:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa570155740 10 client.16821 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa570155740 1 -- 172.21.15.5:0/1504059882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560da41d7f50 con 0x560da41c5370 2024-09-19T06:08:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa546ffd640 1 -- 172.21.15.5:0/1504059882 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa558004030 con 0x560da41c5370 2024-09-19T06:08:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa546ffd640 1 -- 172.21.15.5:0/1504059882 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa560027070 con 0x560da41c5370 2024-09-19T06:08:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa570155740 10 client.16821 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa570155740 10 client.16821 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:08:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa570155740 20 client.16821 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa570155740 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x560da41d9410 0x560da41db8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa570155740 4 client.16821 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.293+0000 7fa570155740 1 -- 172.21.15.5:0/1504059882 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560da412d860 con 0x560da41d9410 2024-09-19T06:08:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.294+0000 7fa5677fe640 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x560da41d9410 0x560da41db8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.294+0000 7fa5677fe640 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x560da41d9410 0x560da41db8b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71bbced87b2450fd server_cookie=3b232ab01703687b in_seq=0 out_seq=0 2024-09-19T06:08:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.294+0000 7fa546ffd640 10 client.16821 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:08:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa546ffd640 1 -- 172.21.15.5:0/1504059882 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560da412d860 con 0x560da41d9410 2024-09-19T06:08:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa546ffd640 10 client.16821 handle_command_reply: tid=0 2024-09-19T06:08:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa570155740 1 client.16821 shutdown 2024-09-19T06:08:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa570155740 1 -- 172.21.15.5:0/1504059882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 msgr2=0x560da41c2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa570155740 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41c2640 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa560012260 tx=0x7fa560004ab0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa570155740 1 -- 172.21.15.5:0/1504059882 shutdown_connections 2024-09-19T06:08:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa570155740 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x560da41d9410 0x560da41db8b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa570155740 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41c2640 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa570155740 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41bbb40 0x560da41c2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa570155740 1 --2- 172.21.15.5:0/1504059882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41ba5d0 0x560da41c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa546ffd640 0 client.16821 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:08:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa570155740 1 -- 172.21.15.5:0/1504059882 >> 172.21.15.5:0/1504059882 conn(0x560da4183310 msgr2=0x560da41bc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa570155740 1 -- 172.21.15.5:0/1504059882 shutdown_connections 2024-09-19T06:08:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa570155740 1 -- 172.21.15.5:0/1504059882 wait complete. 2024-09-19T06:08:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.295+0000 7fa570155740 20 client.16821 trim_cache size 0 max 16384 2024-09-19T06:08:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa570155740 1 Processor -- start 2024-09-19T06:08:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa570155740 1 -- start start 2024-09-19T06:08:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa570155740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 0x560da41b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa570155740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa570155740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41bd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa570155740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681ab6a0 con 0x560da41c5370 2024-09-19T06:08:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa570155740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681ad1e0 con 0x560da41ba5d0 2024-09-19T06:08:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa570155740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681abd70 con 0x560da41bbb40 2024-09-19T06:08:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa567fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41bd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa5677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 0x560da41b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa566ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa567fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41bd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34662/0 (socket says 172.21.15.5:34662) 2024-09-19T06:08:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.310+0000 7fa567fff640 1 -- 172.21.15.5:0/1227881849 learned_addr learned my addr 172.21.15.5:0/1227881849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.311+0000 7fa567fff640 1 -- 172.21.15.5:0/1227881849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 msgr2=0x560da41bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.311+0000 7fa567fff640 1 --2- 172.21.15.5:0/1227881849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41bd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.311+0000 7fa567fff640 1 -- 172.21.15.5:0/1227881849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 msgr2=0x560da41b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.311+0000 7fa567fff640 1 --2- 172.21.15.5:0/1227881849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 0x560da41b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.311+0000 7fa567fff640 1 -- 172.21.15.5:0/1227881849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da41c0580 con 0x560da41c5370 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.311+0000 7fa5677fe640 1 --2- 172.21.15.5:0/1227881849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 0x560da41b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.311+0000 7fa567fff640 1 --2- 172.21.15.5:0/1227881849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41bd660 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa56000f8f0 tx=0x7fa56001fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fe28103ffcc9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.311+0000 7fa54d7fa640 1 -- 172.21.15.5:0/1227881849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56001f620 con 0x560da41c5370 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.311+0000 7fa54d7fa640 1 -- 172.21.15.5:0/1227881849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5600295e0 con 0x560da41c5370 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.311+0000 7fa54d7fa640 1 -- 172.21.15.5:0/1227881849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56001d630 con 0x560da41c5370 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.312+0000 7fa570155740 1 -- 172.21.15.5:0/1227881849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 msgr2=0x560da41bd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.312+0000 7fa570155740 1 --2- 172.21.15.5:0/1227881849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41bd660 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa56000f8f0 tx=0x7fa56001fd10 comp rx=0 tx=0).stop 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.312+0000 7fa570155740 1 -- 172.21.15.5:0/1227881849 shutdown_connections 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.312+0000 7fa570155740 1 --2- 172.21.15.5:0/1227881849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41c5370 0x560da41bd660 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.312+0000 7fa570155740 1 --2- 172.21.15.5:0/1227881849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.312+0000 7fa570155740 1 --2- 172.21.15.5:0/1227881849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41ba5d0 0x560da41b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.312+0000 7fa570155740 1 -- 172.21.15.5:0/1227881849 >> 172.21.15.5:0/1227881849 conn(0x560da4183310 msgr2=0x560da41b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.312+0000 7fa570155740 1 -- 172.21.15.5:0/1227881849 shutdown_connections 2024-09-19T06:08:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.312+0000 7fa570155740 1 -- 172.21.15.5:0/1227881849 wait complete. 2024-09-19T06:08:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.312+0000 7fa570155740 1 -- start start 2024-09-19T06:08:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa570155740 1 Processor -- start 2024-09-19T06:08:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa570155740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41ba5d0 0x560da41bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa570155740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41bf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa570155740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41c5370 0x560da41bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa570155740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681acb10 con 0x560da41ba5d0 2024-09-19T06:08:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa570155740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681ac440 con 0x560da41c5370 2024-09-19T06:08:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa570155740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681a9680 con 0x560da41bbb40 2024-09-19T06:08:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa570155740 10 client.0 _ll_register_callbacks cb 0x7fa5600128e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa5677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41ba5d0 0x560da41bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa566ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41bf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa567fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41c5370 0x560da41bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa5677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41ba5d0 0x560da41bf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34668/0 (socket says 172.21.15.5:34668) 2024-09-19T06:08:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa566ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41bf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53316/0 (socket says 172.21.15.5:53316) 2024-09-19T06:08:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa5677fe640 1 -- 172.21.15.5:0/218044749 learned_addr learned my addr 172.21.15.5:0/218044749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa567fff640 1 -- 172.21.15.5:0/218044749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 msgr2=0x560da41bf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa567fff640 1 --2- 172.21.15.5:0/218044749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41bf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa567fff640 1 -- 172.21.15.5:0/218044749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41ba5d0 msgr2=0x560da41bf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa567fff640 1 --2- 172.21.15.5:0/218044749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41ba5d0 0x560da41bf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.313+0000 7fa567fff640 1 -- 172.21.15.5:0/218044749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da4243300 con 0x560da41c5370 2024-09-19T06:08:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.314+0000 7fa567fff640 1 --2- 172.21.15.5:0/218044749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41c5370 0x560da41bfc70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa560012260 tx=0x7fa560029da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f17679e5eca3b84d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.314+0000 7fa546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:08:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.314+0000 7fa546ffd640 1 -- 172.21.15.5:0/218044749 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56003e030 con 0x560da41c5370 2024-09-19T06:08:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.314+0000 7fa546ffd640 1 -- 172.21.15.5:0/218044749 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa5600299a0 con 0x560da41c5370 2024-09-19T06:08:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.314+0000 7fa570155740 1 -- 172.21.15.5:0/218044749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5600295e0 con 0x560da41c5370 2024-09-19T06:08:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.314+0000 7fa546ffd640 1 -- 172.21.15.5:0/218044749 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa56004a050 con 0x560da41c5370 2024-09-19T06:08:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.314+0000 7fa570155740 10 client.26408 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.314+0000 7fa570155740 1 -- 172.21.15.5:0/218044749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560da42445e0 con 0x560da41c5370 2024-09-19T06:08:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.314+0000 7fa546ffd640 1 -- 172.21.15.5:0/218044749 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5600295e0 con 0x560da41c5370 2024-09-19T06:08:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa546ffd640 1 -- 172.21.15.5:0/218044749 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa560027070 con 0x560da41c5370 2024-09-19T06:08:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa570155740 10 client.26408 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa570155740 10 client.26408 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:08:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa570155740 20 client.26408 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa570155740 1 --2- 172.21.15.5:0/218044749 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x560da4244bf0 0x560da4246fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa570155740 4 client.26408 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:08:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa570155740 1 -- 172.21.15.5:0/218044749 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560da4127d70 con 0x560da4244bf0 2024-09-19T06:08:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa5677fe640 1 --2- 172.21.15.5:0/218044749 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x560da4244bf0 0x560da4246fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa5677fe640 1 --2- 172.21.15.5:0/218044749 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x560da4244bf0 0x560da4246fe0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8449bd05f9385f9 server_cookie=9cdbb78b4671dd44 in_seq=0 out_seq=0 2024-09-19T06:08:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa546ffd640 10 client.26408 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:08:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa546ffd640 1 -- 172.21.15.5:0/218044749 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560da4127d70 con 0x560da4244bf0 2024-09-19T06:08:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa546ffd640 10 client.26408 handle_command_reply: tid=0 2024-09-19T06:08:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.315+0000 7fa570155740 1 client.26408 shutdown 2024-09-19T06:08:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa570155740 1 -- 172.21.15.5:0/218044749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41c5370 msgr2=0x560da41bfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa570155740 1 --2- 172.21.15.5:0/218044749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41c5370 0x560da41bfc70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa560012260 tx=0x7fa560029da0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa570155740 1 -- 172.21.15.5:0/218044749 shutdown_connections 2024-09-19T06:08:55.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa570155740 1 --2- 172.21.15.5:0/218044749 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x560da4244bf0 0x560da4246fe0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa570155740 1 --2- 172.21.15.5:0/218044749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560da41c5370 0x560da41bfc70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa570155740 1 --2- 172.21.15.5:0/218044749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560da41bbb40 0x560da41bf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa546ffd640 0 client.26408 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:08:55.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa570155740 1 --2- 172.21.15.5:0/218044749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560da41ba5d0 0x560da41bf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa570155740 1 -- 172.21.15.5:0/218044749 >> 172.21.15.5:0/218044749 conn(0x560da4183310 msgr2=0x560da41d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:55.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa570155740 1 -- 172.21.15.5:0/218044749 shutdown_connections 2024-09-19T06:08:55.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa570155740 1 -- 172.21.15.5:0/218044749 wait complete. 2024-09-19T06:08:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa570155740 20 client.26408 trim_cache size 0 max 16384 2024-09-19T06:08:55.335 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:08:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa56dd7c640 1 -- 172.21.15.5:0/4252374199 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa5500a33b0 msgr2=0x7fa5500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa56dd7c640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa5500a33b0 0x7fa5500a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5600049d0 tx=0x7fa560009500 comp rx=0 tx=0).stop 2024-09-19T06:08:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa56dd7c640 1 -- 172.21.15.5:0/4252374199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5681036e0 msgr2=0x7fa568198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.316+0000 7fa56dd7c640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5681036e0 0x7fa568198a50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa56806a370 tx=0x7fa55c00dbb0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.317+0000 7fa56dd7c640 1 -- 172.21.15.5:0/4252374199 shutdown_connections 2024-09-19T06:08:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.317+0000 7fa56dd7c640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5681082a0 0x7fa56819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.317+0000 7fa56dd7c640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5681036e0 0x7fa568198a50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.317+0000 7fa56dd7c640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa5500a33b0 0x7fa5500a5870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.317+0000 7fa56dd7c640 1 --2- 172.21.15.5:0/4252374199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa568102d30 0x7fa568198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.317+0000 7fa56dd7c640 1 -- 172.21.15.5:0/4252374199 >> 172.21.15.5:0/4252374199 conn(0x7fa5680fc7d0 msgr2=0x7fa568105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.317+0000 7fa56dd7c640 1 -- 172.21.15.5:0/4252374199 shutdown_connections 2024-09-19T06:08:55.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:55.317+0000 7fa56dd7c640 1 -- 172.21.15.5:0/4252374199 wait complete. 2024-09-19T06:08:55.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 134632 2.76 MB/sec execute 141 sec latency 17.508 ms 2024-09-19T06:08:56.040 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:08:56.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:55 smithi157 ceph-mon[35346]: pgmap v171: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-19T06:08:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:56.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:55 smithi005 ceph-mon[30955]: pgmap v171: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-19T06:08:56.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:55 smithi049 ceph-mon[34640]: pgmap v171: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-19T06:08:56.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 135117 2.78 MB/sec execute 142 sec latency 13.872 ms 2024-09-19T06:08:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.856+0000 7f26340ba640 1 -- 172.21.15.5:0/416651565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c1082a0 msgr2=0x7f262c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.856+0000 7f26340ba640 1 --2- 172.21.15.5:0/416651565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c1082a0 0x7f262c10ec40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2624009cd0 tx=0x7f262402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.856+0000 7f26340ba640 1 -- 172.21.15.5:0/416651565 shutdown_connections 2024-09-19T06:08:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.856+0000 7f26340ba640 1 --2- 172.21.15.5:0/416651565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c1082a0 0x7f262c10ec40 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.856+0000 7f26340ba640 1 --2- 172.21.15.5:0/416651565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c1036e0 0x7f262c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.856+0000 7f26340ba640 1 --2- 172.21.15.5:0/416651565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262c102d30 0x7f262c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.856+0000 7f26340ba640 1 -- 172.21.15.5:0/416651565 >> 172.21.15.5:0/416651565 conn(0x7f262c0fc7d0 msgr2=0x7f262c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.856+0000 7f26340ba640 1 -- 172.21.15.5:0/416651565 shutdown_connections 2024-09-19T06:08:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.856+0000 7f26340ba640 1 -- 172.21.15.5:0/416651565 wait complete. 2024-09-19T06:08:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f26340ba640 1 Processor -- start 2024-09-19T06:08:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f26340ba640 1 -- start start 2024-09-19T06:08:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f26340ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262c102d30 0x7f262c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f26340ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c1036e0 0x7f262c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f26340ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c1082a0 0x7f262c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f26340ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f262c111f50 con 0x7f262c1082a0 2024-09-19T06:08:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f26340ba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f262c111dd0 con 0x7f262c102d30 2024-09-19T06:08:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f26340ba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1120d0 con 0x7f262c1036e0 2024-09-19T06:08:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f2631e56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262c102d30 0x7f262c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f2632657640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c1082a0 0x7f262c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f2631655640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c1036e0 0x7f262c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f2631e56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262c102d30 0x7f262c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37652/0 (socket says 172.21.15.5:37652) 2024-09-19T06:08:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f2631e56640 1 -- 172.21.15.5:0/508659890 learned_addr learned my addr 172.21.15.5:0/508659890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.857+0000 7f2631655640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c1036e0 0x7f262c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53334/0 (socket says 172.21.15.5:53334) 2024-09-19T06:08:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f2631655640 1 -- 172.21.15.5:0/508659890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262c102d30 msgr2=0x7f262c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f2631655640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262c102d30 0x7f262c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f2631655640 1 -- 172.21.15.5:0/508659890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c1082a0 msgr2=0x7f262c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f2631655640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c1082a0 0x7f262c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f2631655640 1 -- 172.21.15.5:0/508659890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c1a6960 con 0x7f262c1036e0 2024-09-19T06:08:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f2632657640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c1082a0 0x7f262c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f2631e56640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262c102d30 0x7f262c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f2631655640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c1036e0 0x7f262c100ed0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f262c0697d0 tx=0x7f261c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f2622ffd640 1 -- 172.21.15.5:0/508659890 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c00cc40 con 0x7f262c1036e0 2024-09-19T06:08:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f2622ffd640 1 -- 172.21.15.5:0/508659890 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f261c00cde0 con 0x7f262c1036e0 2024-09-19T06:08:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f26340ba640 1 -- 172.21.15.5:0/508659890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262c1a6c50 con 0x7f262c1036e0 2024-09-19T06:08:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f2622ffd640 1 -- 172.21.15.5:0/508659890 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c004a00 con 0x7f262c1036e0 2024-09-19T06:08:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.858+0000 7f26340ba640 1 -- 172.21.15.5:0/508659890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262c1a7140 con 0x7f262c1036e0 2024-09-19T06:08:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.861+0000 7f2622ffd640 1 -- 172.21.15.5:0/508659890 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f261c004ba0 con 0x7f262c1036e0 2024-09-19T06:08:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2636493740 1 Processor -- start 2024-09-19T06:08:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2636493740 1 -- start start 2024-09-19T06:08:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2636493740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 0x55eccde0a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2636493740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0a5d0 0x55eccde0aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2636493740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde17760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2636493740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eccdd702f0 con 0x55eccde0bb40 2024-09-19T06:08:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2636493740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eccdd17cf0 con 0x55eccde15370 2024-09-19T06:08:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2636493740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eccdd6c2b0 con 0x55eccde0a5d0 2024-09-19T06:08:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2632657640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 0x55eccde0a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2622ffd640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26180a33b0 0x7f26180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2632657640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 0x55eccde0a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34676/0 (socket says 172.21.15.5:34676) 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2632657640 1 -- 172.21.15.5:0/3426259157 learned_addr learned my addr 172.21.15.5:0/3426259157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2631655640 1 --2- 172.21.15.5:0/3426259157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde17760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.862+0000 7f2631e56640 1 --2- 172.21.15.5:0/3426259157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0a5d0 0x55eccde0aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2632657640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26180a33b0 0x7f26180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2631655640 1 -- 172.21.15.5:0/3426259157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0a5d0 msgr2=0x55eccde0aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2631655640 1 --2- 172.21.15.5:0/3426259157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0a5d0 0x55eccde0aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2631655640 1 -- 172.21.15.5:0/3426259157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 msgr2=0x55eccde0a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2631655640 1 --2- 172.21.15.5:0/3426259157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 0x55eccde0a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2631655640 1 -- 172.21.15.5:0/3426259157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eccdcfc3a0 con 0x55eccde15370 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2631e56640 1 --2- 172.21.15.5:0/3426259157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0a5d0 0x55eccde0aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2632657640 1 --2- 172.21.15.5:0/3426259157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 0x55eccde0a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2622ffd640 1 -- 172.21.15.5:0/508659890 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f261c0cfbc0 con 0x7f262c1036e0 2024-09-19T06:08:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2631655640 1 --2- 172.21.15.5:0/3426259157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde17760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f261c0cd320 tx=0x7f261c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4519fc48459d124a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2632657640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26180a33b0 0x7f26180a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2624005e30 tx=0x7f262402a8e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2617fff640 1 -- 172.21.15.5:0/3426259157 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c01a110 con 0x55eccde15370 2024-09-19T06:08:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2617fff640 1 -- 172.21.15.5:0/3426259157 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f261c005020 con 0x55eccde15370 2024-09-19T06:08:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.863+0000 7f2617fff640 1 -- 172.21.15.5:0/3426259157 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c09c5b0 con 0x55eccde15370 2024-09-19T06:08:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.864+0000 7f2636493740 1 -- 172.21.15.5:0/3426259157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 msgr2=0x55eccde17760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.864+0000 7f2636493740 1 --2- 172.21.15.5:0/3426259157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde17760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f261c0cd320 tx=0x7f261c01f000 comp rx=0 tx=0).stop 2024-09-19T06:08:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.864+0000 7f2636493740 1 -- 172.21.15.5:0/3426259157 shutdown_connections 2024-09-19T06:08:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.864+0000 7f2636493740 1 --2- 172.21.15.5:0/3426259157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde17760 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.864+0000 7f2636493740 1 --2- 172.21.15.5:0/3426259157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0a5d0 0x55eccde0aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.864+0000 7f2636493740 1 --2- 172.21.15.5:0/3426259157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 0x55eccde0a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.864+0000 7f2636493740 1 -- 172.21.15.5:0/3426259157 >> 172.21.15.5:0/3426259157 conn(0x55eccddd3310 msgr2=0x55eccde06c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.864+0000 7f2636493740 1 -- 172.21.15.5:0/3426259157 shutdown_connections 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.864+0000 7f2636493740 1 -- 172.21.15.5:0/3426259157 wait complete. 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.865+0000 7f2636493740 1 -- start start 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.865+0000 7f2636493740 1 Processor -- start 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.865+0000 7f2636493740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde0a5d0 0x55eccde14d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.865+0000 7f2636493740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 0x55eccde11fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.865+0000 7f2636493740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde15370 0x55eccde125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.865+0000 7f2636493740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eccde18010 con 0x55eccde0bb40 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.865+0000 7f2636493740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eccdd2b060 con 0x55eccde0a5d0 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.865+0000 7f2636493740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eccde18190 con 0x55eccde15370 2024-09-19T06:08:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.865+0000 7f2636493740 10 client.0 _ll_register_callbacks cb 0x55eccda7e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.865+0000 7f2631655640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 0x55eccde11fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.865+0000 7f2631e56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde0a5d0 0x55eccde14d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2631e56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde0a5d0 0x55eccde14d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37672/0 (socket says 172.21.15.5:37672) 2024-09-19T06:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2631e56640 1 -- 172.21.15.5:0/2012212349 learned_addr learned my addr 172.21.15.5:0/2012212349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2632657640 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde15370 0x55eccde125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2631e56640 1 -- 172.21.15.5:0/2012212349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde15370 msgr2=0x55eccde125d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2631e56640 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde15370 0x55eccde125d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2631e56640 1 -- 172.21.15.5:0/2012212349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 msgr2=0x55eccde11fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2631e56640 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 0x55eccde11fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2631e56640 1 -- 172.21.15.5:0/2012212349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eccdae13c0 con 0x55eccde0a5d0 2024-09-19T06:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2631655640 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 0x55eccde11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2631e56640 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde0a5d0 0x55eccde14d30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2628012d20 tx=0x7f262800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be96d8d4489d428 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f26157fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f26157fa640 1 -- 172.21.15.5:0/2012212349 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628005020 con 0x55eccde0a5d0 2024-09-19T06:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f26157fa640 1 -- 172.21.15.5:0/2012212349 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f26280051c0 con 0x55eccde0a5d0 2024-09-19T06:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2636493740 1 -- 172.21.15.5:0/2012212349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eccdd7d860 con 0x55eccde0a5d0 2024-09-19T06:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f26157fa640 1 -- 172.21.15.5:0/2012212349 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f26280053c0 con 0x55eccde0a5d0 2024-09-19T06:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2636493740 10 client.26420 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.866+0000 7f2636493740 1 -- 172.21.15.5:0/2012212349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eccde23ad0 con 0x55eccde0a5d0 2024-09-19T06:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.867+0000 7f26157fa640 1 -- 172.21.15.5:0/2012212349 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55eccdd7d860 con 0x55eccde0a5d0 2024-09-19T06:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.867+0000 7f26157fa640 1 -- 172.21.15.5:0/2012212349 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2628025470 con 0x55eccde0a5d0 2024-09-19T06:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.867+0000 7f2636493740 10 client.26420 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.867+0000 7f2636493740 10 client.26420 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.867+0000 7f2636493740 20 client.26420 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.867+0000 7f2636493740 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55eccde24ee0 0x55eccde272f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.867+0000 7f2636493740 4 client.26420 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.867+0000 7f2636493740 1 -- 172.21.15.5:0/2012212349 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eccdd77d70 con 0x55eccde24ee0 2024-09-19T06:08:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.867+0000 7f2631655640 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55eccde24ee0 0x55eccde272f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.868+0000 7f2631655640 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55eccde24ee0 0x55eccde272f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2c206caf5929c3b server_cookie=e23eee4d9a26451c in_seq=0 out_seq=0 2024-09-19T06:08:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.868+0000 7f26157fa640 10 client.26420 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:08:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.868+0000 7f26157fa640 1 -- 172.21.15.5:0/2012212349 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eccdd77d70 con 0x55eccde24ee0 2024-09-19T06:08:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.868+0000 7f26157fa640 10 client.26420 handle_command_reply: tid=0 2024-09-19T06:08:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.868+0000 7f2636493740 1 client.26420 shutdown 2024-09-19T06:08:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.868+0000 7f2636493740 1 -- 172.21.15.5:0/2012212349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde0a5d0 msgr2=0x55eccde14d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.868+0000 7f2636493740 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde0a5d0 0x55eccde14d30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2628012d20 tx=0x7f262800f790 comp rx=0 tx=0).stop 2024-09-19T06:08:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.869+0000 7f2636493740 1 -- 172.21.15.5:0/2012212349 shutdown_connections 2024-09-19T06:08:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.869+0000 7f2636493740 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55eccde24ee0 0x55eccde272f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.869+0000 7f2636493740 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde15370 0x55eccde125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.869+0000 7f2636493740 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0bb40 0x55eccde11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.869+0000 7f2636493740 1 --2- 172.21.15.5:0/2012212349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde0a5d0 0x55eccde14d30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.869+0000 7f26157fa640 0 client.26420 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:08:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.869+0000 7f2636493740 1 -- 172.21.15.5:0/2012212349 >> 172.21.15.5:0/2012212349 conn(0x55eccddd3310 msgr2=0x55eccde0c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.869+0000 7f2636493740 1 -- 172.21.15.5:0/2012212349 shutdown_connections 2024-09-19T06:08:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.869+0000 7f2636493740 1 -- 172.21.15.5:0/2012212349 wait complete. 2024-09-19T06:08:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.869+0000 7f2636493740 20 client.26420 trim_cache size 0 max 16384 2024-09-19T06:08:56.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2636493740 1 Processor -- start 2024-09-19T06:08:56.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2636493740 1 -- start start 2024-09-19T06:08:56.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2636493740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 0x55eccde06e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2636493740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0bb40 0x55eccde0d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2636493740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde0d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2636493740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1afa60 con 0x55eccde0a5d0 2024-09-19T06:08:56.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2636493740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1b15a0 con 0x55eccde15370 2024-09-19T06:08:56.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2636493740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1b0130 con 0x55eccde0bb40 2024-09-19T06:08:56.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2631e56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 0x55eccde06e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2631e56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 0x55eccde06e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34708/0 (socket says 172.21.15.5:34708) 2024-09-19T06:08:56.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2631e56640 1 -- 172.21.15.5:0/3773854016 learned_addr learned my addr 172.21.15.5:0/3773854016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:56.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2632657640 1 --2- 172.21.15.5:0/3773854016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde0d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.883+0000 7f2631655640 1 --2- 172.21.15.5:0/3773854016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0bb40 0x55eccde0d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.884+0000 7f2631e56640 1 -- 172.21.15.5:0/3773854016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0bb40 msgr2=0x55eccde0d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:56.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.884+0000 7f2631e56640 1 --2- 172.21.15.5:0/3773854016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0bb40 0x55eccde0d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.884+0000 7f2631e56640 1 -- 172.21.15.5:0/3773854016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 msgr2=0x55eccde0d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:56.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.884+0000 7f2631e56640 1 --2- 172.21.15.5:0/3773854016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde0d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.884+0000 7f2631e56640 1 -- 172.21.15.5:0/3773854016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eccde07960 con 0x55eccde0a5d0 2024-09-19T06:08:56.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.884+0000 7f2632657640 1 --2- 172.21.15.5:0/3773854016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde0d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:56.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.884+0000 7f2631e56640 1 --2- 172.21.15.5:0/3773854016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 0x55eccde06e10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2628012990 tx=0x7f26280129c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e993fc8e3f0fe33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:56.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.884+0000 7f26177fe640 1 -- 172.21.15.5:0/3773854016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262802a030 con 0x55eccde0a5d0 2024-09-19T06:08:56.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.884+0000 7f26177fe640 1 -- 172.21.15.5:0/3773854016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2628005560 con 0x55eccde0a5d0 2024-09-19T06:08:56.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.884+0000 7f26177fe640 1 -- 172.21.15.5:0/3773854016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628025470 con 0x55eccde0a5d0 2024-09-19T06:08:56.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.885+0000 7f2636493740 1 -- 172.21.15.5:0/3773854016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 msgr2=0x55eccde06e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:56.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.885+0000 7f2636493740 1 --2- 172.21.15.5:0/3773854016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 0x55eccde06e10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2628012990 tx=0x7f26280129c0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.885+0000 7f2636493740 1 -- 172.21.15.5:0/3773854016 shutdown_connections 2024-09-19T06:08:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.885+0000 7f2636493740 1 --2- 172.21.15.5:0/3773854016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde0d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.885+0000 7f2636493740 1 --2- 172.21.15.5:0/3773854016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0bb40 0x55eccde0d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.885+0000 7f2636493740 1 --2- 172.21.15.5:0/3773854016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 0x55eccde06e10 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.885+0000 7f2636493740 1 -- 172.21.15.5:0/3773854016 >> 172.21.15.5:0/3773854016 conn(0x55eccddd3310 msgr2=0x55eccde06700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.885+0000 7f2636493740 1 -- 172.21.15.5:0/3773854016 shutdown_connections 2024-09-19T06:08:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.885+0000 7f2636493740 1 -- 172.21.15.5:0/3773854016 wait complete. 2024-09-19T06:08:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.885+0000 7f2636493740 1 -- start start 2024-09-19T06:08:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.885+0000 7f2636493740 1 Processor -- start 2024-09-19T06:08:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2636493740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 0x55eccde19440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2636493740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0bb40 0x55eccde19980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2636493740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde19ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2636493740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1b0ed0 con 0x55eccde0a5d0 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2636493740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1b0800 con 0x55eccde15370 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2636493740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1ada40 con 0x55eccde0bb40 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2636493740 10 client.0 _ll_register_callbacks cb 0x7f2628009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2631e56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 0x55eccde19440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2631e56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 0x55eccde19440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34716/0 (socket says 172.21.15.5:34716) 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2631e56640 1 -- 172.21.15.5:0/735771384 learned_addr learned my addr 172.21.15.5:0/735771384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2631655640 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0bb40 0x55eccde19980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2632657640 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde19ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2631e56640 1 -- 172.21.15.5:0/735771384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0bb40 msgr2=0x55eccde19980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2631e56640 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0bb40 0x55eccde19980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2631e56640 1 -- 172.21.15.5:0/735771384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 msgr2=0x55eccde19ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2631e56640 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde19ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2631e56640 1 -- 172.21.15.5:0/735771384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eccde975f0 con 0x55eccde0a5d0 2024-09-19T06:08:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.886+0000 7f2631655640 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0bb40 0x55eccde19980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.887+0000 7f2631e56640 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 0x55eccde19440 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2628005390 tx=0x7f2628005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fbde927bb6ac7f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.887+0000 7f26157fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:08:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.887+0000 7f26157fa640 1 -- 172.21.15.5:0/735771384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628032020 con 0x55eccde0a5d0 2024-09-19T06:08:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.887+0000 7f26157fa640 1 -- 172.21.15.5:0/735771384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f2628025bf0 con 0x55eccde0a5d0 2024-09-19T06:08:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.887+0000 7f2636493740 1 -- 172.21.15.5:0/735771384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2628005560 con 0x55eccde0a5d0 2024-09-19T06:08:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.887+0000 7f26157fa640 1 -- 172.21.15.5:0/735771384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f262803d050 con 0x55eccde0a5d0 2024-09-19T06:08:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.887+0000 7f2636493740 10 client.16845 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.887+0000 7f2636493740 1 -- 172.21.15.5:0/735771384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eccde988f0 con 0x55eccde0a5d0 2024-09-19T06:08:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.887+0000 7f26157fa640 1 -- 172.21.15.5:0/735771384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2628005560 con 0x55eccde0a5d0 2024-09-19T06:08:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f26157fa640 1 -- 172.21.15.5:0/735771384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f262801c470 con 0x55eccde0a5d0 2024-09-19T06:08:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f2636493740 10 client.16845 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f2636493740 10 client.16845 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:08:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f2636493740 20 client.16845 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f2636493740 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55eccde98f40 0x55eccde9b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f2636493740 4 client.16845 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:08:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f2636493740 1 -- 172.21.15.5:0/735771384 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eccdcf4170 con 0x55eccde98f40 2024-09-19T06:08:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f2631655640 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55eccde98f40 0x55eccde9b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f26157fa640 10 client.16845 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:08:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f2631655640 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55eccde98f40 0x55eccde9b330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa53efb32fee9ea5 server_cookie=37c0a295b0e27dd9 in_seq=0 out_seq=0 2024-09-19T06:08:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f26157fa640 1 -- 172.21.15.5:0/735771384 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eccdcf4170 con 0x55eccde98f40 2024-09-19T06:08:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f26157fa640 10 client.16845 handle_command_reply: tid=0 2024-09-19T06:08:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.888+0000 7f2636493740 1 client.16845 shutdown 2024-09-19T06:08:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.889+0000 7f2636493740 1 -- 172.21.15.5:0/735771384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 msgr2=0x55eccde19440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.889+0000 7f2636493740 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 0x55eccde19440 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2628005390 tx=0x7f2628005f90 comp rx=0 tx=0).stop 2024-09-19T06:08:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.889+0000 7f2636493740 1 -- 172.21.15.5:0/735771384 shutdown_connections 2024-09-19T06:08:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.889+0000 7f2636493740 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55eccde98f40 0x55eccde9b330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.889+0000 7f2636493740 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eccde15370 0x55eccde19ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.889+0000 7f2636493740 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccde0bb40 0x55eccde19980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.889+0000 7f2636493740 1 --2- 172.21.15.5:0/735771384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eccde0a5d0 0x55eccde19440 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.889+0000 7f2636493740 1 -- 172.21.15.5:0/735771384 >> 172.21.15.5:0/735771384 conn(0x55eccddd3310 msgr2=0x55eccde054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.889+0000 7f26157fa640 0 client.16845 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:08:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.889+0000 7f2636493740 1 -- 172.21.15.5:0/735771384 shutdown_connections 2024-09-19T06:08:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.889+0000 7f2636493740 1 -- 172.21.15.5:0/735771384 wait complete. 2024-09-19T06:08:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.889+0000 7f2636493740 20 client.16845 trim_cache size 0 max 16384 2024-09-19T06:08:56.898 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:08:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.890+0000 7f26340ba640 1 -- 172.21.15.5:0/508659890 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26180a33b0 msgr2=0x7f26180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.890+0000 7f26340ba640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26180a33b0 0x7f26180a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2624005e30 tx=0x7f262402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.890+0000 7f26340ba640 1 -- 172.21.15.5:0/508659890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c1036e0 msgr2=0x7f262c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.890+0000 7f26340ba640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c1036e0 0x7f262c100ed0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f262c0697d0 tx=0x7f261c00ee50 comp rx=0 tx=0).stop 2024-09-19T06:08:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.890+0000 7f26340ba640 1 -- 172.21.15.5:0/508659890 shutdown_connections 2024-09-19T06:08:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.890+0000 7f26340ba640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c1082a0 0x7f262c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.890+0000 7f26340ba640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26180a33b0 0x7f26180a5870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.890+0000 7f26340ba640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c1036e0 0x7f262c100ed0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.890+0000 7f26340ba640 1 --2- 172.21.15.5:0/508659890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262c102d30 0x7f262c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.890+0000 7f26340ba640 1 -- 172.21.15.5:0/508659890 >> 172.21.15.5:0/508659890 conn(0x7f262c0fc7d0 msgr2=0x7f262c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.890+0000 7f26340ba640 1 -- 172.21.15.5:0/508659890 shutdown_connections 2024-09-19T06:08:56.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:56.890+0000 7f26340ba640 1 -- 172.21.15.5:0/508659890 wait complete. 2024-09-19T06:08:57.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:08:57.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 135406 2.76 MB/sec execute 143 sec latency 11.791 ms 2024-09-19T06:08:57.880 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:08:58.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:57 smithi005 ceph-mon[30955]: pgmap v172: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-09-19T06:08:58.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:57 smithi157 ceph-mon[35346]: pgmap v172: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-09-19T06:08:58.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:57 smithi049 ceph-mon[34640]: pgmap v172: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-09-19T06:08:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.462+0000 7f8c958b5640 1 -- 172.21.15.5:0/2905752035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100e20 msgr2=0x7f8c90101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.462+0000 7f8c958b5640 1 --2- 172.21.15.5:0/2905752035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100e20 0x7f8c90101200 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009900 tx=0x7f8c8002a480 comp rx=0 tx=0).stop 2024-09-19T06:08:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.462+0000 7f8c958b5640 1 -- 172.21.15.5:0/2905752035 shutdown_connections 2024-09-19T06:08:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.462+0000 7f8c958b5640 1 --2- 172.21.15.5:0/2905752035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c90102ef0 0x7f8c9010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.462+0000 7f8c958b5640 1 --2- 172.21.15.5:0/2905752035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c90102570 0x7f8c901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.462+0000 7f8c958b5640 1 --2- 172.21.15.5:0/2905752035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100e20 0x7f8c90101200 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.462+0000 7f8c958b5640 1 -- 172.21.15.5:0/2905752035 >> 172.21.15.5:0/2905752035 conn(0x7f8c900fc7d0 msgr2=0x7f8c900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.462+0000 7f8c958b5640 1 -- 172.21.15.5:0/2905752035 shutdown_connections 2024-09-19T06:08:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.462+0000 7f8c958b5640 1 -- 172.21.15.5:0/2905752035 wait complete. 2024-09-19T06:08:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.463+0000 7f8c958b5640 1 Processor -- start 2024-09-19T06:08:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.463+0000 7f8c958b5640 1 -- start start 2024-09-19T06:08:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c958b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100e20 0x7f8c90198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c958b5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c90102570 0x7f8c90198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c958b5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c90102ef0 0x7f8c9019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c958b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c90110e30 con 0x7f8c90100e20 2024-09-19T06:08:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c958b5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8c90110cb0 con 0x7f8c90102570 2024-09-19T06:08:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c958b5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c90110fb0 con 0x7f8c90102ef0 2024-09-19T06:08:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c90102570 0x7f8c90198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100e20 0x7f8c90198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c90102570 0x7f8c90198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37692/0 (socket says 172.21.15.5:37692) 2024-09-19T06:08:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100e20 0x7f8c90198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34732/0 (socket says 172.21.15.5:34732) 2024-09-19T06:08:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8e7fc640 1 -- 172.21.15.5:0/3636628560 learned_addr learned my addr 172.21.15.5:0/3636628560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8f7fe640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c90102ef0 0x7f8c9019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8e7fc640 1 -- 172.21.15.5:0/3636628560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c90102ef0 msgr2=0x7f8c9019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8e7fc640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c90102ef0 0x7f8c9019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8e7fc640 1 -- 172.21.15.5:0/3636628560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100e20 msgr2=0x7f8c90198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8e7fc640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100e20 0x7f8c90198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8e7fc640 1 -- 172.21.15.5:0/3636628560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9019d3e0 con 0x7f8c90102570 2024-09-19T06:08:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8effd640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100e20 0x7f8c90198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.464+0000 7f8c8f7fe640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c90102ef0 0x7f8c9019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:08:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.465+0000 7f8c8e7fc640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c90102570 0x7f8c90198970 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400e9f0 tx=0x7f8c8400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.465+0000 7f8c948b3640 1 -- 172.21.15.5:0/3636628560 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8400cd80 con 0x7f8c90102570 2024-09-19T06:08:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.465+0000 7f8c958b5640 1 -- 172.21.15.5:0/3636628560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9019d6d0 con 0x7f8c90102570 2024-09-19T06:08:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.465+0000 7f8c948b3640 1 -- 172.21.15.5:0/3636628560 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c840054d0 con 0x7f8c90102570 2024-09-19T06:08:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.465+0000 7f8c948b3640 1 -- 172.21.15.5:0/3636628560 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84010490 con 0x7f8c90102570 2024-09-19T06:08:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.465+0000 7f8c958b5640 1 -- 172.21.15.5:0/3636628560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c901a4fb0 con 0x7f8c90102570 2024-09-19T06:08:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.468+0000 7f8c97c8e740 1 Processor -- start 2024-09-19T06:08:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.468+0000 7f8c97c8e740 1 -- start start 2024-09-19T06:08:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c97c8e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c948b3640 1 -- 172.21.15.5:0/3636628560 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8c84005020 con 0x7f8c90102570 2024-09-19T06:08:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c97c8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c5eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c97c8e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 0x55ef0c5f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c97c8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ef0c5522f0 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c97c8e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ef0c4f9cf0 con 0x55ef0c5edb40 2024-09-19T06:08:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c97c8e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ef0c54e2b0 con 0x55ef0c5f7370 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c8f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c8f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5ec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37702/0 (socket says 172.21.15.5:37702) 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c8f7fe640 1 -- 172.21.15.5:0/2658489038 learned_addr learned my addr 172.21.15.5:0/2658489038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c8f7fe640 1 -- 172.21.15.5:0/2658489038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 msgr2=0x55ef0c5f9760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c8f7fe640 1 --2- 172.21.15.5:0/2658489038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 0x55ef0c5f9760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c8f7fe640 1 -- 172.21.15.5:0/2658489038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5ec5d0 msgr2=0x55ef0c5eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c8f7fe640 1 --2- 172.21.15.5:0/2658489038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c5eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.469+0000 7f8c8f7fe640 1 -- 172.21.15.5:0/2658489038 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef0c4de3a0 con 0x55ef0c5edb40 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.470+0000 7f8c8f7fe640 1 --2- 172.21.15.5:0/2658489038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5ec090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009fd0 tx=0x7f8c880108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dab14549edcc640 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.470+0000 7f8c948b3640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8c780a33b0 0x7f8c780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.470+0000 7f8c75ffb640 1 -- 172.21.15.5:0/2658489038 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c88015b70 con 0x55ef0c5edb40 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.470+0000 7f8c75ffb640 1 -- 172.21.15.5:0/2658489038 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c88015d10 con 0x55ef0c5edb40 2024-09-19T06:08:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.470+0000 7f8c75ffb640 1 -- 172.21.15.5:0/2658489038 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8800e6f0 con 0x55ef0c5edb40 2024-09-19T06:08:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.470+0000 7f8c948b3640 1 -- 172.21.15.5:0/3636628560 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f8c840d0120 con 0x7f8c90102570 2024-09-19T06:08:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.470+0000 7f8c8effd640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8c780a33b0 0x7f8c780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.471+0000 7f8c97c8e740 1 -- 172.21.15.5:0/2658489038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 msgr2=0x55ef0c5ec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.471+0000 7f8c8effd640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8c780a33b0 0x7f8c780a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009230 tx=0x7f8c80000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.471+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/2658489038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5ec090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009fd0 tx=0x7f8c880108e0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.471+0000 7f8c97c8e740 1 -- 172.21.15.5:0/2658489038 shutdown_connections 2024-09-19T06:08:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.471+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/2658489038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 0x55ef0c5f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.471+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/2658489038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c5eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.471+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/2658489038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5ec090 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.471+0000 7f8c97c8e740 1 -- 172.21.15.5:0/2658489038 >> 172.21.15.5:0/2658489038 conn(0x55ef0c5b5310 msgr2=0x55ef0c5e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.471+0000 7f8c97c8e740 1 -- 172.21.15.5:0/2658489038 shutdown_connections 2024-09-19T06:08:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.471+0000 7f8c97c8e740 1 -- 172.21.15.5:0/2658489038 wait complete. 2024-09-19T06:08:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.472+0000 7f8c97c8e740 1 -- start start 2024-09-19T06:08:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.472+0000 7f8c97c8e740 1 Processor -- start 2024-09-19T06:08:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.472+0000 7f8c97c8e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c5f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.472+0000 7f8c97c8e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5f4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.472+0000 7f8c97c8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5f7370 0x55ef0c5f4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c97c8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ef0c5fa070 con 0x55ef0c5f7370 2024-09-19T06:08:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c97c8e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ef0c50d060 con 0x55ef0c5edb40 2024-09-19T06:08:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c97c8e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ef0c5fa1f0 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c97c8e740 10 client.0 _ll_register_callbacks cb 0x55ef0c2608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c8e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c8e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5f4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37716/0 (socket says 172.21.15.5:37716) 2024-09-19T06:08:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c8e7fc640 1 -- 172.21.15.5:0/3958108649 learned_addr learned my addr 172.21.15.5:0/3958108649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c8f7fe640 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5f7370 0x55ef0c5f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c8e7fc640 1 -- 172.21.15.5:0/3958108649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5ec5d0 msgr2=0x55ef0c5f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c8e7fc640 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c5f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c8e7fc640 1 -- 172.21.15.5:0/3958108649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5f7370 msgr2=0x55ef0c5f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c8e7fc640 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5f7370 0x55ef0c5f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c8e7fc640 1 -- 172.21.15.5:0/3958108649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0c2c33c0 con 0x55ef0c5edb40 2024-09-19T06:08:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.473+0000 7f8c8f7fe640 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5f7370 0x55ef0c5f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c8e7fc640 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5f4010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c840cde80 tx=0x7f8c8409be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=592b4a996db17eef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c6effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:08:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c6effd640 1 -- 172.21.15.5:0/3958108649 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c840b4020 con 0x55ef0c5edb40 2024-09-19T06:08:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c6effd640 1 -- 172.21.15.5:0/3958108649 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f8c840b5250 con 0x55ef0c5edb40 2024-09-19T06:08:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c97c8e740 1 -- 172.21.15.5:0/3958108649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c88015d10 con 0x55ef0c5edb40 2024-09-19T06:08:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c6effd640 1 -- 172.21.15.5:0/3958108649 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8c8409cb10 con 0x55ef0c5edb40 2024-09-19T06:08:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c97c8e740 10 client.26438 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c97c8e740 1 -- 172.21.15.5:0/3958108649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ef0c605b70 con 0x55ef0c5edb40 2024-09-19T06:08:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c6effd640 1 -- 172.21.15.5:0/3958108649 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c88015d10 con 0x55ef0c5edb40 2024-09-19T06:08:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c6effd640 1 -- 172.21.15.5:0/3958108649 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f8c840a4bf0 con 0x55ef0c5edb40 2024-09-19T06:08:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c97c8e740 10 client.26438 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c97c8e740 10 client.26438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c97c8e740 20 client.26438 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ef0c607080 0x55ef0c609470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c97c8e740 4 client.26438 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:08:58.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.474+0000 7f8c97c8e740 1 -- 172.21.15.5:0/3958108649 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef0c55f860 con 0x55ef0c607080 2024-09-19T06:08:58.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.475+0000 7f8c8effd640 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ef0c607080 0x55ef0c609470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.475+0000 7f8c8effd640 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ef0c607080 0x55ef0c609470 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1a283bb627cce1 server_cookie=b387e3faeef41ce6 in_seq=0 out_seq=0 2024-09-19T06:08:58.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.475+0000 7f8c6effd640 10 client.26438 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c6effd640 1 -- 172.21.15.5:0/3958108649 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef0c55f860 con 0x55ef0c607080 2024-09-19T06:08:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c6effd640 10 client.26438 handle_command_reply: tid=0 2024-09-19T06:08:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c97c8e740 1 client.26438 shutdown 2024-09-19T06:08:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c97c8e740 1 -- 172.21.15.5:0/3958108649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 msgr2=0x55ef0c5f4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5f4010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c840cde80 tx=0x7f8c8409be30 comp rx=0 tx=0).stop 2024-09-19T06:08:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c97c8e740 1 -- 172.21.15.5:0/3958108649 shutdown_connections 2024-09-19T06:08:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ef0c607080 0x55ef0c609470 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5f7370 0x55ef0c5f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5f4010 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/3958108649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c5f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c6effd640 0 client.26438 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c97c8e740 1 -- 172.21.15.5:0/3958108649 >> 172.21.15.5:0/3958108649 conn(0x55ef0c5b5310 msgr2=0x55ef0c5ee930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c97c8e740 1 -- 172.21.15.5:0/3958108649 shutdown_connections 2024-09-19T06:08:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c97c8e740 1 -- 172.21.15.5:0/3958108649 wait complete. 2024-09-19T06:08:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.476+0000 7f8c97c8e740 20 client.26438 trim_cache size 0 max 16384 2024-09-19T06:08:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c97c8e740 1 Processor -- start 2024-09-19T06:08:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c97c8e740 1 -- start start 2024-09-19T06:08:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c97c8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c603310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c97c8e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c97c8e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 0x55ef0c5ef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c97c8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901ab5e0 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c97c8e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901ad120 con 0x55ef0c5edb40 2024-09-19T06:08:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c97c8e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901abcb0 con 0x55ef0c5f7370 2024-09-19T06:08:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c8effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c603310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c8effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c603310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34754/0 (socket says 172.21.15.5:34754) 2024-09-19T06:08:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c8effd640 1 -- 172.21.15.5:0/4221416209 learned_addr learned my addr 172.21.15.5:0/4221416209 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c8e7fc640 1 --2- 172.21.15.5:0/4221416209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c8f7fe640 1 --2- 172.21.15.5:0/4221416209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 0x55ef0c5ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c8effd640 1 -- 172.21.15.5:0/4221416209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 msgr2=0x55ef0c5ef660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c8effd640 1 --2- 172.21.15.5:0/4221416209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 0x55ef0c5ef660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c8effd640 1 -- 172.21.15.5:0/4221416209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 msgr2=0x55ef0c5ef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c8effd640 1 --2- 172.21.15.5:0/4221416209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5ef120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.491+0000 7f8c8effd640 1 -- 172.21.15.5:0/4221416209 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef0c5f4970 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.492+0000 7f8c8effd640 1 --2- 172.21.15.5:0/4221416209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c603310 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8c80031740 tx=0x7f8c8003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c43daa764814990 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.492+0000 7f8c757fa640 1 -- 172.21.15.5:0/4221416209 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80035040 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.492+0000 7f8c757fa640 1 -- 172.21.15.5:0/4221416209 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c800317f0 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.492+0000 7f8c757fa640 1 -- 172.21.15.5:0/4221416209 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8003c710 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.493+0000 7f8c97c8e740 1 -- 172.21.15.5:0/4221416209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5ec5d0 msgr2=0x55ef0c603310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.493+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/4221416209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c603310 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8c80031740 tx=0x7f8c8003d000 comp rx=0 tx=0).stop 2024-09-19T06:08:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.493+0000 7f8c97c8e740 1 -- 172.21.15.5:0/4221416209 shutdown_connections 2024-09-19T06:08:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.493+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/4221416209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 0x55ef0c5ef660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.493+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/4221416209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5edb40 0x55ef0c5ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.493+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/4221416209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c603310 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.493+0000 7f8c97c8e740 1 -- 172.21.15.5:0/4221416209 >> 172.21.15.5:0/4221416209 conn(0x55ef0c5b5310 msgr2=0x55ef0c5f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.493+0000 7f8c97c8e740 1 -- 172.21.15.5:0/4221416209 shutdown_connections 2024-09-19T06:08:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.493+0000 7f8c97c8e740 1 -- 172.21.15.5:0/4221416209 wait complete. 2024-09-19T06:08:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.493+0000 7f8c97c8e740 1 -- start start 2024-09-19T06:08:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.493+0000 7f8c97c8e740 1 Processor -- start 2024-09-19T06:08:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c97c8e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c5ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c97c8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5edb40 0x55ef0c5f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c97c8e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 0x55ef0c5f1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c97c8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901aca50 con 0x55ef0c5edb40 2024-09-19T06:08:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c97c8e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901ac380 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c97c8e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901a95e0 con 0x55ef0c5f7370 2024-09-19T06:08:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c97c8e740 10 client.0 _ll_register_callbacks cb 0x7f8c8409c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c5ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5edb40 0x55ef0c5f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c5ffae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37730/0 (socket says 172.21.15.5:37730) 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5edb40 0x55ef0c5f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34762/0 (socket says 172.21.15.5:34762) 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8effd640 1 -- 172.21.15.5:0/926732552 learned_addr learned my addr 172.21.15.5:0/926732552 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8f7fe640 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 0x55ef0c5f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8effd640 1 -- 172.21.15.5:0/926732552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 msgr2=0x55ef0c5f1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8effd640 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 0x55ef0c5f1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8effd640 1 -- 172.21.15.5:0/926732552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5edb40 msgr2=0x55ef0c5f0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8effd640 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5edb40 0x55ef0c5f0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8effd640 1 -- 172.21.15.5:0/926732552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0c5f17e0 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8e7fc640 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5edb40 0x55ef0c5f0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:08:58.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c8effd640 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c5ffae0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8c800092a0 tx=0x7f8c80005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3563e8dd8f33222 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.494+0000 7f8c6effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c6effd640 1 -- 172.21.15.5:0/926732552 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80035030 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c6effd640 1 -- 172.21.15.5:0/926732552 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f8c80005250 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c97c8e740 1 -- 172.21.15.5:0/926732552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c800317f0 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c6effd640 1 -- 172.21.15.5:0/926732552 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8c80066050 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c97c8e740 10 client.26444 fetch_fsmap learned FSMap version 33 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c97c8e740 1 -- 172.21.15.5:0/926732552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ef0c671b80 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c6effd640 1 -- 172.21.15.5:0/926732552 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c800317f0 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c6effd640 1 -- 172.21.15.5:0/926732552 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f8c80003740 con 0x55ef0c5ec5d0 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c97c8e740 10 client.26444 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c97c8e740 10 client.26444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c97c8e740 20 client.26444 populate_metadata read hostname 'smithi005' 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ef0c672290 0x55ef0c6746b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:08:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c97c8e740 4 client.26444 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:08:58.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.495+0000 7f8c97c8e740 1 -- 172.21.15.5:0/926732552 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef0c559d70 con 0x55ef0c672290 2024-09-19T06:08:58.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c8e7fc640 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ef0c672290 0x55ef0c6746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:08:58.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c8e7fc640 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ef0c672290 0x55ef0c6746b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f50ff09a7ebe69ab server_cookie=de5020d743fb3952 in_seq=0 out_seq=0 2024-09-19T06:08:58.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c6effd640 10 client.26444 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:58.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c6effd640 1 -- 172.21.15.5:0/926732552 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef0c559d70 con 0x55ef0c672290 2024-09-19T06:08:58.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c6effd640 10 client.26444 handle_command_reply: tid=0 2024-09-19T06:08:58.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c97c8e740 1 client.26444 shutdown 2024-09-19T06:08:58.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c97c8e740 1 -- 172.21.15.5:0/926732552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5ec5d0 msgr2=0x55ef0c5ffae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:58.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c5ffae0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8c800092a0 tx=0x7f8c80005be0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c97c8e740 1 -- 172.21.15.5:0/926732552 shutdown_connections 2024-09-19T06:08:58.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ef0c672290 0x55ef0c6746b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef0c5f7370 0x55ef0c5f1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef0c5edb40 0x55ef0c5f0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c6effd640 0 client.26444 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:08:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c97c8e740 1 --2- 172.21.15.5:0/926732552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ef0c5ec5d0 0x55ef0c5ffae0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.496+0000 7f8c97c8e740 1 -- 172.21.15.5:0/926732552 >> 172.21.15.5:0/926732552 conn(0x55ef0c5b5310 msgr2=0x55ef0c5f5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c97c8e740 1 -- 172.21.15.5:0/926732552 shutdown_connections 2024-09-19T06:08:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c97c8e740 1 -- 172.21.15.5:0/926732552 wait complete. 2024-09-19T06:08:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c97c8e740 20 client.26444 trim_cache size 0 max 16384 2024-09-19T06:08:58.505 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:08:58.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c958b5640 1 -- 172.21.15.5:0/3636628560 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8c780a33b0 msgr2=0x7f8c780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:58.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c958b5640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8c780a33b0 0x7f8c780a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009230 tx=0x7f8c80000bc0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c958b5640 1 -- 172.21.15.5:0/3636628560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c90102570 msgr2=0x7f8c90198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:08:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c958b5640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c90102570 0x7f8c90198970 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400e9f0 tx=0x7f8c8400edf0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c958b5640 1 -- 172.21.15.5:0/3636628560 shutdown_connections 2024-09-19T06:08:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c958b5640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8c780a33b0 0x7f8c780a5870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c958b5640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c90102ef0 0x7f8c9019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c958b5640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c90102570 0x7f8c90198970 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c958b5640 1 --2- 172.21.15.5:0/3636628560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100e20 0x7f8c90198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:08:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c958b5640 1 -- 172.21.15.5:0/3636628560 >> 172.21.15.5:0/3636628560 conn(0x7f8c900fc7d0 msgr2=0x7f8c9006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:08:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c958b5640 1 -- 172.21.15.5:0/3636628560 shutdown_connections 2024-09-19T06:08:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:08:58.497+0000 7f8c958b5640 1 -- 172.21.15.5:0/3636628560 wait complete. 2024-09-19T06:08:58.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 135901 2.75 MB/sec execute 144 sec latency 44.042 ms 2024-09-19T06:08:59.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 136412 2.76 MB/sec execute 145 sec latency 21.681 ms 2024-09-19T06:09:00.203 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:09:00.204 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:09:00.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:08:59 smithi005 ceph-mon[30955]: pgmap v173: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-19T06:09:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:08:59 smithi157 ceph-mon[35346]: pgmap v173: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-19T06:09:00.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:08:59 smithi049 ceph-mon[34640]: pgmap v173: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-19T06:09:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:09:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.766+0000 7f6266027640 1 -- 172.21.15.5:0/4273896322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f626010c100 msgr2=0x7f626010e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.766+0000 7f6266027640 1 --2- 172.21.15.5:0/4273896322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f626010c100 0x7f626010e580 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6258009ca0 tx=0x7f625802a950 comp rx=0 tx=0).stop 2024-09-19T06:09:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.766+0000 7f6266027640 1 -- 172.21.15.5:0/4273896322 shutdown_connections 2024-09-19T06:09:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.766+0000 7f6266027640 1 --2- 172.21.15.5:0/4273896322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f626010c100 0x7f626010e580 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.766+0000 7f6266027640 1 --2- 172.21.15.5:0/4273896322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62601035d0 0x7f626010bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.766+0000 7f6266027640 1 --2- 172.21.15.5:0/4273896322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6260102cb0 0x7f6260103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.766+0000 7f6266027640 1 -- 172.21.15.5:0/4273896322 >> 172.21.15.5:0/4273896322 conn(0x7f62600fc6e0 msgr2=0x7f62600feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.766+0000 7f6266027640 1 -- 172.21.15.5:0/4273896322 shutdown_connections 2024-09-19T06:09:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.766+0000 7f6266027640 1 -- 172.21.15.5:0/4273896322 wait complete. 2024-09-19T06:09:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.766+0000 7f6266027640 1 Processor -- start 2024-09-19T06:09:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f6266027640 1 -- start start 2024-09-19T06:09:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f6266027640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6260102cb0 0x7f6260100900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f6266027640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62601035d0 0x7f6260100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f6266027640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f626010c100 0x7f6260101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f6266027640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6260111dd0 con 0x7f626010c100 2024-09-19T06:09:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f6266027640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6260111c50 con 0x7f6260102cb0 2024-09-19T06:09:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f6266027640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6260111f50 con 0x7f62601035d0 2024-09-19T06:09:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f625ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f626010c100 0x7f6260101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f625f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6260102cb0 0x7f6260100900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f625ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f626010c100 0x7f6260101380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34772/0 (socket says 172.21.15.5:34772) 2024-09-19T06:09:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f625ffff640 1 -- 172.21.15.5:0/790563681 learned_addr learned my addr 172.21.15.5:0/790563681 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f625effd640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62601035d0 0x7f6260100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f625ffff640 1 -- 172.21.15.5:0/790563681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62601035d0 msgr2=0x7f6260100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f625ffff640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62601035d0 0x7f6260100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f625ffff640 1 -- 172.21.15.5:0/790563681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6260102cb0 msgr2=0x7f6260100900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f625ffff640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6260102cb0 0x7f6260100900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f625ffff640 1 -- 172.21.15.5:0/790563681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62601a6810 con 0x7f626010c100 2024-09-19T06:09:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f625effd640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62601035d0 0x7f6260100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:09:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.767+0000 7f625f7fe640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6260102cb0 0x7f6260100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.768+0000 7f625ffff640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f626010c100 0x7f6260101380 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6258004af0 tx=0x7f625802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.768+0000 7f625cff9640 1 -- 172.21.15.5:0/790563681 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258034040 con 0x7f626010c100 2024-09-19T06:09:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.768+0000 7f625cff9640 1 -- 172.21.15.5:0/790563681 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f625802fe80 con 0x7f626010c100 2024-09-19T06:09:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.768+0000 7f6266027640 1 -- 172.21.15.5:0/790563681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62601a6aa0 con 0x7f626010c100 2024-09-19T06:09:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.768+0000 7f625cff9640 1 -- 172.21.15.5:0/790563681 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62580046a0 con 0x7f626010c100 2024-09-19T06:09:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.768+0000 7f6266027640 1 -- 172.21.15.5:0/790563681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62601a6fe0 con 0x7f626010c100 2024-09-19T06:09:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.771+0000 7f625cff9640 1 -- 172.21.15.5:0/790563681 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6258037020 con 0x7f626010c100 2024-09-19T06:09:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.771+0000 7f6268dc5740 1 Processor -- start 2024-09-19T06:09:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.771+0000 7f6268dc5740 1 -- start start 2024-09-19T06:09:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.771+0000 7f6268dc5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6dc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.771+0000 7f6268dc5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.771+0000 7f6268dc5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.771+0000 7f6268dc5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc7c6382f0 con 0x55dc7c6dc300 2024-09-19T06:09:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.771+0000 7f6268dc5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc7c5e0270 con 0x55dc7c6e74c0 2024-09-19T06:09:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.771+0000 7f6268dc5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dc7c6342b0 con 0x55dc7c6dcc20 2024-09-19T06:09:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6dc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6dc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34784/0 (socket says 172.21.15.5:34784) 2024-09-19T06:09:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625effd640 1 -- 172.21.15.5:0/1406934220 learned_addr learned my addr 172.21.15.5:0/1406934220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625cff9640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f62480a34b0 0x7f62480a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625f7fe640 1 --2- 172.21.15.5:0/1406934220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625effd640 1 --2- 172.21.15.5:0/1406934220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625effd640 1 -- 172.21.15.5:0/1406934220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6dcc20 msgr2=0x55dc7c6e6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625effd640 1 --2- 172.21.15.5:0/1406934220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6e6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625effd640 1 -- 172.21.15.5:0/1406934220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6e74c0 msgr2=0x55dc7c6e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625effd640 1 --2- 172.21.15.5:0/1406934220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6e98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625effd640 1 -- 172.21.15.5:0/1406934220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7c5df340 con 0x55dc7c6dc300 2024-09-19T06:09:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625ffff640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f62480a34b0 0x7f62480a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625f7fe640 1 --2- 172.21.15.5:0/1406934220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6e6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:09:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625cff9640 1 -- 172.21.15.5:0/790563681 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f62580f0ac0 con 0x7f626010c100 2024-09-19T06:09:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.772+0000 7f625effd640 1 --2- 172.21.15.5:0/1406934220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6dc6e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6254007d80 tx=0x7f6254007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8dfac3085dfcf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.773+0000 7f6245ffb640 1 -- 172.21.15.5:0/1406934220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625401c070 con 0x55dc7c6dc300 2024-09-19T06:09:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.773+0000 7f6245ffb640 1 -- 172.21.15.5:0/1406934220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6254017400 con 0x55dc7c6dc300 2024-09-19T06:09:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.773+0000 7f6245ffb640 1 -- 172.21.15.5:0/1406934220 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540145e0 con 0x55dc7c6dc300 2024-09-19T06:09:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.773+0000 7f625ffff640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f62480a34b0 0x7f62480a5970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f625802ae70 tx=0x7f625802a950 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.774+0000 7f6268dc5740 1 -- 172.21.15.5:0/1406934220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 msgr2=0x55dc7c6dc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.774+0000 7f6268dc5740 1 --2- 172.21.15.5:0/1406934220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6dc6e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6254007d80 tx=0x7f6254007600 comp rx=0 tx=0).stop 2024-09-19T06:09:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.774+0000 7f6268dc5740 1 -- 172.21.15.5:0/1406934220 shutdown_connections 2024-09-19T06:09:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.774+0000 7f6268dc5740 1 --2- 172.21.15.5:0/1406934220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.774+0000 7f6268dc5740 1 --2- 172.21.15.5:0/1406934220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6e6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.774+0000 7f6268dc5740 1 --2- 172.21.15.5:0/1406934220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6dc6e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.774+0000 7f6268dc5740 1 -- 172.21.15.5:0/1406934220 >> 172.21.15.5:0/1406934220 conn(0x55dc7c6d8870 msgr2=0x55dc7c6d8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.774+0000 7f6268dc5740 1 -- 172.21.15.5:0/1406934220 shutdown_connections 2024-09-19T06:09:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.774+0000 7f6268dc5740 1 -- 172.21.15.5:0/1406934220 wait complete. 2024-09-19T06:09:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.775+0000 7f6268dc5740 1 -- start start 2024-09-19T06:09:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.775+0000 7f6268dc5740 1 Processor -- start 2024-09-19T06:09:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f6268dc5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6e4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f6268dc5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6e4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f6268dc5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f6268dc5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc7c6ea090 con 0x55dc7c6dc300 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f6268dc5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc7c5c1290 con 0x55dc7c6dcc20 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f6268dc5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dc7c6ea210 con 0x55dc7c6e74c0 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f6268dc5740 10 client.0 _ll_register_callbacks cb 0x55dc7c5f2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f625f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6e4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f625effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f625f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6e4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37768/0 (socket says 172.21.15.5:37768) 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f625f7fe640 1 -- 172.21.15.5:0/68382870 learned_addr learned my addr 172.21.15.5:0/68382870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f625effd640 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6e4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f625f7fe640 1 -- 172.21.15.5:0/68382870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 msgr2=0x55dc7c6e4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f625f7fe640 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6e4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f625f7fe640 1 -- 172.21.15.5:0/68382870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 msgr2=0x55dc7c6e4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f625f7fe640 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6e4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f625f7fe640 1 -- 172.21.15.5:0/68382870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7c34b210 con 0x55dc7c6dcc20 2024-09-19T06:09:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.776+0000 7f625effd640 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6e4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.777+0000 7f625f7fe640 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6e4950 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6250012d50 tx=0x7f625000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0704c02775f08fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.777+0000 7f623effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.777+0000 7f623effd640 1 -- 172.21.15.5:0/68382870 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500120d0 con 0x55dc7c6dcc20 2024-09-19T06:09:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.777+0000 7f623effd640 1 -- 172.21.15.5:0/68382870 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f6250010430 con 0x55dc7c6dcc20 2024-09-19T06:09:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.777+0000 7f6268dc5740 1 -- 172.21.15.5:0/68382870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6254017400 con 0x55dc7c6dcc20 2024-09-19T06:09:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.777+0000 7f623effd640 1 -- 172.21.15.5:0/68382870 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6250012270 con 0x55dc7c6dcc20 2024-09-19T06:09:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.777+0000 7f6268dc5740 10 client.26456 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.777+0000 7f6268dc5740 1 -- 172.21.15.5:0/68382870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dc7c417620 con 0x55dc7c6dcc20 2024-09-19T06:09:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.777+0000 7f623effd640 1 -- 172.21.15.5:0/68382870 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6254017400 con 0x55dc7c6dcc20 2024-09-19T06:09:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.778+0000 7f623effd640 1 -- 172.21.15.5:0/68382870 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f625001c720 con 0x55dc7c6dcc20 2024-09-19T06:09:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.778+0000 7f6268dc5740 10 client.26456 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.778+0000 7f6268dc5740 10 client.26456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.778+0000 7f6268dc5740 20 client.26456 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.778+0000 7f6268dc5740 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dc7c6f7510 0x55dc7c6f9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.778+0000 7f6268dc5740 4 client.26456 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.778+0000 7f6268dc5740 1 -- 172.21.15.5:0/68382870 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc7c645a20 con 0x55dc7c6f7510 2024-09-19T06:09:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.778+0000 7f625effd640 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dc7c6f7510 0x55dc7c6f9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.778+0000 7f623effd640 10 client.26456 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.778+0000 7f625effd640 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dc7c6f7510 0x55dc7c6f9940 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48ddf858264022c7 server_cookie=5d9a9c4facaf85bd in_seq=0 out_seq=0 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.779+0000 7f623effd640 1 -- 172.21.15.5:0/68382870 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc7c645a20 con 0x55dc7c6f7510 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.779+0000 7f623effd640 10 client.26456 handle_command_reply: tid=0 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.779+0000 7f6268dc5740 1 client.26456 shutdown 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.779+0000 7f6268dc5740 1 -- 172.21.15.5:0/68382870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 msgr2=0x55dc7c6e4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.779+0000 7f6268dc5740 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6e4950 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6250012d50 tx=0x7f625000f790 comp rx=0 tx=0).stop 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.779+0000 7f6268dc5740 1 -- 172.21.15.5:0/68382870 shutdown_connections 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.779+0000 7f6268dc5740 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dc7c6f7510 0x55dc7c6f9940 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.779+0000 7f6268dc5740 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.779+0000 7f6268dc5740 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6e4950 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.779+0000 7f623effd640 0 client.26456 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.779+0000 7f6268dc5740 1 --2- 172.21.15.5:0/68382870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6e4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.779+0000 7f6268dc5740 1 -- 172.21.15.5:0/68382870 >> 172.21.15.5:0/68382870 conn(0x55dc7c6d8870 msgr2=0x55dc7c6de5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.780+0000 7f6268dc5740 1 -- 172.21.15.5:0/68382870 shutdown_connections 2024-09-19T06:09:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.780+0000 7f6268dc5740 1 -- 172.21.15.5:0/68382870 wait complete. 2024-09-19T06:09:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.780+0000 7f6268dc5740 20 client.26456 trim_cache size 0 max 16384 2024-09-19T06:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.796+0000 7f6268dc5740 1 Processor -- start 2024-09-19T06:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.796+0000 7f6268dc5740 1 -- start start 2024-09-19T06:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.796+0000 7f6268dc5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6d8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.796+0000 7f6268dc5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6ff250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.796+0000 7f6268dc5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6ff790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.796+0000 7f6268dc5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62601ab510 con 0x55dc7c6dc300 2024-09-19T06:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.796+0000 7f6268dc5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f62601ad050 con 0x55dc7c6dcc20 2024-09-19T06:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.796+0000 7f6268dc5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f62601abbe0 con 0x55dc7c6e74c0 2024-09-19T06:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.797+0000 7f625effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6d8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.797+0000 7f625f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.797+0000 7f625effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6d8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34812/0 (socket says 172.21.15.5:34812) 2024-09-19T06:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.797+0000 7f625effd640 1 -- 172.21.15.5:0/4215333998 learned_addr learned my addr 172.21.15.5:0/4215333998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.797+0000 7f625effd640 1 --2- 172.21.15.5:0/4215333998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.797+0000 7f625f7fe640 1 -- 172.21.15.5:0/4215333998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 msgr2=0x55dc7c6ff790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.797+0000 7f625f7fe640 1 --2- 172.21.15.5:0/4215333998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6ff790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.797+0000 7f625f7fe640 1 -- 172.21.15.5:0/4215333998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 msgr2=0x55dc7c6d8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.797+0000 7f625f7fe640 1 --2- 172.21.15.5:0/4215333998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6d8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.797+0000 7f625f7fe640 1 -- 172.21.15.5:0/4215333998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7c6e33a0 con 0x55dc7c6dcc20 2024-09-19T06:09:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.813+0000 7f625f7fe640 1 --2- 172.21.15.5:0/4215333998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6ff250 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f625001cd50 tx=0x7f6250009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d37d0db620b35f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.813+0000 7f62457fa640 1 -- 172.21.15.5:0/4215333998 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625001c420 con 0x55dc7c6dcc20 2024-09-19T06:09:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.813+0000 7f62457fa640 1 -- 172.21.15.5:0/4215333998 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62500050b0 con 0x55dc7c6dcc20 2024-09-19T06:09:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.813+0000 7f62457fa640 1 -- 172.21.15.5:0/4215333998 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500038c0 con 0x55dc7c6dcc20 2024-09-19T06:09:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.814+0000 7f6268dc5740 1 -- 172.21.15.5:0/4215333998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 msgr2=0x55dc7c6ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.814+0000 7f6268dc5740 1 --2- 172.21.15.5:0/4215333998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6ff250 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f625001cd50 tx=0x7f6250009b50 comp rx=0 tx=0).stop 2024-09-19T06:09:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.814+0000 7f6268dc5740 1 -- 172.21.15.5:0/4215333998 shutdown_connections 2024-09-19T06:09:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.814+0000 7f6268dc5740 1 --2- 172.21.15.5:0/4215333998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.814+0000 7f6268dc5740 1 --2- 172.21.15.5:0/4215333998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6ff250 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.814+0000 7f6268dc5740 1 --2- 172.21.15.5:0/4215333998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6d8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.814+0000 7f6268dc5740 1 -- 172.21.15.5:0/4215333998 >> 172.21.15.5:0/4215333998 conn(0x55dc7c6d8870 msgr2=0x55dc7c6e6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.814+0000 7f6268dc5740 1 -- 172.21.15.5:0/4215333998 shutdown_connections 2024-09-19T06:09:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.814+0000 7f6268dc5740 1 -- 172.21.15.5:0/4215333998 wait complete. 2024-09-19T06:09:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.814+0000 7f6268dc5740 1 -- start start 2024-09-19T06:09:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f6268dc5740 1 Processor -- start 2024-09-19T06:09:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f6268dc5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f6268dc5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6eb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f6268dc5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6ebd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f6268dc5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62601ac980 con 0x55dc7c6dc300 2024-09-19T06:09:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f6268dc5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f62601ac2b0 con 0x55dc7c6dcc20 2024-09-19T06:09:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f6268dc5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f62601a94f0 con 0x55dc7c6e74c0 2024-09-19T06:09:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f6268dc5740 10 client.0 _ll_register_callbacks cb 0x7f6250009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f625effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f625f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6eb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f625effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6eb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34818/0 (socket says 172.21.15.5:34818) 2024-09-19T06:09:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f625effd640 1 -- 172.21.15.5:0/2972949988 learned_addr learned my addr 172.21.15.5:0/2972949988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f625effd640 1 -- 172.21.15.5:0/2972949988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 msgr2=0x55dc7c6ebd40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:09:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f625effd640 1 --2- 172.21.15.5:0/2972949988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6ebd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f625effd640 1 -- 172.21.15.5:0/2972949988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 msgr2=0x55dc7c6eb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f625effd640 1 --2- 172.21.15.5:0/2972949988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6eb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f625effd640 1 -- 172.21.15.5:0/2972949988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7c6dca50 con 0x55dc7c6dc300 2024-09-19T06:09:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.815+0000 7f625f7fe640 1 --2- 172.21.15.5:0/2972949988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6eb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.816+0000 7f625effd640 1 --2- 172.21.15.5:0/2972949988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6eb2c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f625401a040 tx=0x7f62540047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e725a5d1185259 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.816+0000 7f623effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:09:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.816+0000 7f623effd640 1 -- 172.21.15.5:0/2972949988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625401c040 con 0x55dc7c6dc300 2024-09-19T06:09:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.816+0000 7f623effd640 1 -- 172.21.15.5:0/2972949988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f6254017900 con 0x55dc7c6dc300 2024-09-19T06:09:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.816+0000 7f6268dc5740 1 -- 172.21.15.5:0/2972949988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62500050b0 con 0x55dc7c6dc300 2024-09-19T06:09:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.816+0000 7f623effd640 1 -- 172.21.15.5:0/2972949988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6254017b00 con 0x55dc7c6dc300 2024-09-19T06:09:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.816+0000 7f6268dc5740 10 client.16878 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.816+0000 7f6268dc5740 1 -- 172.21.15.5:0/2972949988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dc7c7623f0 con 0x55dc7c6dc300 2024-09-19T06:09:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.816+0000 7f623effd640 1 -- 172.21.15.5:0/2972949988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62500050b0 con 0x55dc7c6dc300 2024-09-19T06:09:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.816+0000 7f623effd640 1 -- 172.21.15.5:0/2972949988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f625402f3f0 con 0x55dc7c6dc300 2024-09-19T06:09:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.817+0000 7f6268dc5740 10 client.16878 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.817+0000 7f6268dc5740 10 client.16878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.817+0000 7f6268dc5740 20 client.16878 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.817+0000 7f6268dc5740 1 --2- 172.21.15.5:0/2972949988 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dc7c762740 0x55dc7c764b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.817+0000 7f6268dc5740 4 client.16878 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:09:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.817+0000 7f6268dc5740 1 -- 172.21.15.5:0/2972949988 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc7c63fd70 con 0x55dc7c762740 2024-09-19T06:09:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.817+0000 7f625f7fe640 1 --2- 172.21.15.5:0/2972949988 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dc7c762740 0x55dc7c764b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.817+0000 7f623effd640 10 client.16878 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.817+0000 7f625f7fe640 1 --2- 172.21.15.5:0/2972949988 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dc7c762740 0x55dc7c764b30 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=147cebd5d8cb0487 server_cookie=b3d27e0c315b9ac2 in_seq=0 out_seq=0 2024-09-19T06:09:00.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 136920 2.74 MB/sec execute 146 sec latency 21.798 ms 2024-09-19T06:09:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f623effd640 1 -- 172.21.15.5:0/2972949988 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc7c63fd70 con 0x55dc7c762740 2024-09-19T06:09:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f623effd640 10 client.16878 handle_command_reply: tid=0 2024-09-19T06:09:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f6268dc5740 1 client.16878 shutdown 2024-09-19T06:09:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f6268dc5740 1 -- 172.21.15.5:0/2972949988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 msgr2=0x55dc7c6eb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f6268dc5740 1 --2- 172.21.15.5:0/2972949988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6eb2c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f625401a040 tx=0x7f62540047d0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f6268dc5740 1 -- 172.21.15.5:0/2972949988 shutdown_connections 2024-09-19T06:09:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f6268dc5740 1 --2- 172.21.15.5:0/2972949988 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dc7c762740 0x55dc7c764b30 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f6268dc5740 1 --2- 172.21.15.5:0/2972949988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc7c6e74c0 0x55dc7c6ebd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f6268dc5740 1 --2- 172.21.15.5:0/2972949988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7c6dcc20 0x55dc7c6eb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f6268dc5740 1 --2- 172.21.15.5:0/2972949988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c6dc300 0x55dc7c6eb2c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f6268dc5740 1 -- 172.21.15.5:0/2972949988 >> 172.21.15.5:0/2972949988 conn(0x55dc7c6d8870 msgr2=0x55dc7c6e43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f623effd640 0 client.16878 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f6268dc5740 1 -- 172.21.15.5:0/2972949988 shutdown_connections 2024-09-19T06:09:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f6268dc5740 1 -- 172.21.15.5:0/2972949988 wait complete. 2024-09-19T06:09:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.836+0000 7f6268dc5740 20 client.16878 trim_cache size 0 max 16384 2024-09-19T06:09:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:09:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:09:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:09:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:09:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.837+0000 7f6266027640 1 -- 172.21.15.5:0/790563681 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f62480a34b0 msgr2=0x7f62480a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.837+0000 7f6266027640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f62480a34b0 0x7f62480a5970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f625802ae70 tx=0x7f625802a950 comp rx=0 tx=0).stop 2024-09-19T06:09:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.837+0000 7f6266027640 1 -- 172.21.15.5:0/790563681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f626010c100 msgr2=0x7f6260101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.837+0000 7f6266027640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f626010c100 0x7f6260101380 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6258004af0 tx=0x7f625802f5d0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.837+0000 7f6266027640 1 -- 172.21.15.5:0/790563681 shutdown_connections 2024-09-19T06:09:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.837+0000 7f6266027640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f626010c100 0x7f6260101380 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.837+0000 7f6266027640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f62480a34b0 0x7f62480a5970 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.837+0000 7f6266027640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62601035d0 0x7f6260100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.837+0000 7f6266027640 1 --2- 172.21.15.5:0/790563681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6260102cb0 0x7f6260100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.837+0000 7f6266027640 1 -- 172.21.15.5:0/790563681 >> 172.21.15.5:0/790563681 conn(0x7f62600fc6e0 msgr2=0x7f62600fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.837+0000 7f6266027640 1 -- 172.21.15.5:0/790563681 shutdown_connections 2024-09-19T06:09:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:00.837+0000 7f6266027640 1 -- 172.21.15.5:0/790563681 wait complete. 2024-09-19T06:09:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.011+0000 7f6379de3640 1 -- 172.21.15.5:0/2365162995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63741082a0 msgr2=0x7f637410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.011+0000 7f6379de3640 1 --2- 172.21.15.5:0/2365162995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63741082a0 0x7f637410ec40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f636c009cd0 tx=0x7f636c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.011+0000 7f6379de3640 1 -- 172.21.15.5:0/2365162995 shutdown_connections 2024-09-19T06:09:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.011+0000 7f6379de3640 1 --2- 172.21.15.5:0/2365162995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63741082a0 0x7f637410ec40 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.011+0000 7f6379de3640 1 --2- 172.21.15.5:0/2365162995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63741036e0 0x7f6374107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.011+0000 7f6379de3640 1 --2- 172.21.15.5:0/2365162995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6374102d30 0x7f6374103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.011+0000 7f6379de3640 1 -- 172.21.15.5:0/2365162995 >> 172.21.15.5:0/2365162995 conn(0x7f63740fc7d0 msgr2=0x7f63740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.012+0000 7f6379de3640 1 -- 172.21.15.5:0/2365162995 shutdown_connections 2024-09-19T06:09:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.012+0000 7f6379de3640 1 -- 172.21.15.5:0/2365162995 wait complete. 2024-09-19T06:09:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.012+0000 7f6379de3640 1 Processor -- start 2024-09-19T06:09:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.012+0000 7f6379de3640 1 -- start start 2024-09-19T06:09:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f6379de3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374102d30 0x7f63741984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f6379de3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63741036e0 0x7f6374198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f6379de3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63741082a0 0x7f637419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f6379de3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6374111f50 con 0x7f63741036e0 2024-09-19T06:09:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f6379de3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6374111dd0 con 0x7f6374102d30 2024-09-19T06:09:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f6379de3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f63741120d0 con 0x7f63741082a0 2024-09-19T06:09:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f6372ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63741036e0 0x7f6374198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f63737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374102d30 0x7f63741984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f63737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374102d30 0x7f63741984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37816/0 (socket says 172.21.15.5:37816) 2024-09-19T06:09:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f63737fe640 1 -- 172.21.15.5:0/3465885699 learned_addr learned my addr 172.21.15.5:0/3465885699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f6372ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63741036e0 0x7f6374198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34836/0 (socket says 172.21.15.5:34836) 2024-09-19T06:09:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f6373fff640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63741082a0 0x7f637419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f63737fe640 1 -- 172.21.15.5:0/3465885699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63741082a0 msgr2=0x7f637419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f63737fe640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63741082a0 0x7f637419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f63737fe640 1 -- 172.21.15.5:0/3465885699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63741036e0 msgr2=0x7f6374198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f63737fe640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63741036e0 0x7f6374198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f63737fe640 1 -- 172.21.15.5:0/3465885699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637419d480 con 0x7f6374102d30 2024-09-19T06:09:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.013+0000 7f6373fff640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63741082a0 0x7f637419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:09:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.014+0000 7f63737fe640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374102d30 0x7f63741984d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f636400e9f0 tx=0x7f636400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.014+0000 7f6370ff9640 1 -- 172.21.15.5:0/3465885699 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636400cd80 con 0x7f6374102d30 2024-09-19T06:09:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.014+0000 7f6379de3640 1 -- 172.21.15.5:0/3465885699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637419d770 con 0x7f6374102d30 2024-09-19T06:09:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.014+0000 7f6370ff9640 1 -- 172.21.15.5:0/3465885699 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63640108e0 con 0x7f6374102d30 2024-09-19T06:09:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.014+0000 7f6370ff9640 1 -- 172.21.15.5:0/3465885699 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364018920 con 0x7f6374102d30 2024-09-19T06:09:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.014+0000 7f6379de3640 1 -- 172.21.15.5:0/3465885699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741a5050 con 0x7f6374102d30 2024-09-19T06:09:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.016+0000 7f6370ff9640 1 -- 172.21.15.5:0/3465885699 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6364010a90 con 0x7f6374102d30 2024-09-19T06:09:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.016+0000 7f637c1bc740 1 Processor -- start 2024-09-19T06:09:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f637c1bc740 1 -- start start 2024-09-19T06:09:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f637c1bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad82fb60 0x558fad82ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f637c1bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad830480 0x558fad83a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f637c1bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad83d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f637c1bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558fad795280 con 0x558fad82fb60 2024-09-19T06:09:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f637c1bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558fad73cb10 con 0x558fad83ad40 2024-09-19T06:09:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f637c1bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558fad791240 con 0x558fad830480 2024-09-19T06:09:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f6373fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad82fb60 0x558fad82ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f6373fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad82fb60 0x558fad82ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34846/0 (socket says 172.21.15.5:34846) 2024-09-19T06:09:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f6373fff640 1 -- 172.21.15.5:0/214119606 learned_addr learned my addr 172.21.15.5:0/214119606 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f6370ff9640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f635c0a33b0 0x7f635c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f6372ffd640 1 --2- 172.21.15.5:0/214119606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad83d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f63737fe640 1 --2- 172.21.15.5:0/214119606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad830480 0x558fad83a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f6373fff640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f635c0a33b0 0x7f635c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f6372ffd640 1 -- 172.21.15.5:0/214119606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad830480 msgr2=0x558fad83a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f6372ffd640 1 --2- 172.21.15.5:0/214119606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad830480 0x558fad83a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f6372ffd640 1 -- 172.21.15.5:0/214119606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad82fb60 msgr2=0x558fad82ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f6372ffd640 1 --2- 172.21.15.5:0/214119606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad82fb60 0x558fad82ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f6372ffd640 1 -- 172.21.15.5:0/214119606 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fad7491f0 con 0x558fad83ad40 2024-09-19T06:09:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f63737fe640 1 --2- 172.21.15.5:0/214119606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad830480 0x558fad83a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:09:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.017+0000 7f6370ff9640 1 -- 172.21.15.5:0/3465885699 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f63640d0090 con 0x7f6374102d30 2024-09-19T06:09:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.018+0000 7f6372ffd640 1 --2- 172.21.15.5:0/214119606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad83d130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f637406a370 tx=0x7f6368011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4ebaa8e04695187 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.018+0000 7f6359ffb640 1 -- 172.21.15.5:0/214119606 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368004050 con 0x558fad83ad40 2024-09-19T06:09:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.018+0000 7f6373fff640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f635c0a33b0 0x7f635c0a5870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f636c004af0 tx=0x7f636c02a8e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.018+0000 7f6359ffb640 1 -- 172.21.15.5:0/214119606 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63680054d0 con 0x558fad83ad40 2024-09-19T06:09:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.018+0000 7f6359ffb640 1 -- 172.21.15.5:0/214119606 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368004c60 con 0x558fad83ad40 2024-09-19T06:09:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.019+0000 7f637c1bc740 1 -- 172.21.15.5:0/214119606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 msgr2=0x558fad83d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.019+0000 7f637c1bc740 1 --2- 172.21.15.5:0/214119606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad83d130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f637406a370 tx=0x7f6368011810 comp rx=0 tx=0).stop 2024-09-19T06:09:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.019+0000 7f637c1bc740 1 -- 172.21.15.5:0/214119606 shutdown_connections 2024-09-19T06:09:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.019+0000 7f637c1bc740 1 --2- 172.21.15.5:0/214119606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad83d130 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.019+0000 7f637c1bc740 1 --2- 172.21.15.5:0/214119606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad830480 0x558fad83a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.019+0000 7f637c1bc740 1 --2- 172.21.15.5:0/214119606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad82fb60 0x558fad82ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.019+0000 7f637c1bc740 1 -- 172.21.15.5:0/214119606 >> 172.21.15.5:0/214119606 conn(0x558fad82c0f0 msgr2=0x558fad82c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.019+0000 7f637c1bc740 1 -- 172.21.15.5:0/214119606 shutdown_connections 2024-09-19T06:09:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.019+0000 7f637c1bc740 1 -- 172.21.15.5:0/214119606 wait complete. 2024-09-19T06:09:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.019+0000 7f637c1bc740 1 -- start start 2024-09-19T06:09:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.019+0000 7f637c1bc740 1 Processor -- start 2024-09-19T06:09:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f637c1bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad837bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f637c1bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 0x558fad838130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f637c1bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad838670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f637c1bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558fad83d910 con 0x558fad830480 2024-09-19T06:09:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f637c1bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558fad74ff10 con 0x558fad83ad40 2024-09-19T06:09:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f637c1bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558fad83da90 con 0x558fad82fb60 2024-09-19T06:09:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f637c1bc740 10 client.0 _ll_register_callbacks cb 0x558fad718a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f6372ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 0x558fad838130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f6373fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad838670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f6373fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad838670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37836/0 (socket says 172.21.15.5:37836) 2024-09-19T06:09:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f6373fff640 1 -- 172.21.15.5:0/170629763 learned_addr learned my addr 172.21.15.5:0/170629763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f6372ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 0x558fad838130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34860/0 (socket says 172.21.15.5:34860) 2024-09-19T06:09:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f63737fe640 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad837bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f6373fff640 1 -- 172.21.15.5:0/170629763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 msgr2=0x558fad837bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f6373fff640 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad837bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f6373fff640 1 -- 172.21.15.5:0/170629763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 msgr2=0x558fad838130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.020+0000 7f6373fff640 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 0x558fad838130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.021+0000 7f6373fff640 1 -- 172.21.15.5:0/170629763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fad5066a0 con 0x558fad83ad40 2024-09-19T06:09:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.021+0000 7f6372ffd640 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 0x558fad838130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.021+0000 7f63737fe640 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad837bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.021+0000 7f6373fff640 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad838670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f636c031d50 tx=0x7f636c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d784bfbf3aa4780 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.021+0000 7f6352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.021+0000 7f6352ffd640 1 -- 172.21.15.5:0/170629763 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c004540 con 0x558fad83ad40 2024-09-19T06:09:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.021+0000 7f6352ffd640 1 -- 172.21.15.5:0/170629763 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f636c0046e0 con 0x558fad83ad40 2024-09-19T06:09:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.021+0000 7f637c1bc740 1 -- 172.21.15.5:0/170629763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63680054d0 con 0x558fad83ad40 2024-09-19T06:09:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.021+0000 7f6352ffd640 1 -- 172.21.15.5:0/170629763 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f636c0305a0 con 0x558fad83ad40 2024-09-19T06:09:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.021+0000 7f637c1bc740 10 client.26474 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.021+0000 7f637c1bc740 1 -- 172.21.15.5:0/170629763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558fad849480 con 0x558fad83ad40 2024-09-19T06:09:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.021+0000 7f6352ffd640 1 -- 172.21.15.5:0/170629763 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63680054d0 con 0x558fad83ad40 2024-09-19T06:09:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.022+0000 7f6352ffd640 1 -- 172.21.15.5:0/170629763 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f636c030070 con 0x558fad83ad40 2024-09-19T06:09:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.022+0000 7f637c1bc740 10 client.26474 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.022+0000 7f637c1bc740 10 client.26474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.022+0000 7f637c1bc740 20 client.26474 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.022+0000 7f637c1bc740 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558fad84a940 0x558fad84cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.022+0000 7f637c1bc740 4 client.26474 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.022+0000 7f637c1bc740 1 -- 172.21.15.5:0/170629763 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fad7a27f0 con 0x558fad84a940 2024-09-19T06:09:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.022+0000 7f63737fe640 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558fad84a940 0x558fad84cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.022+0000 7f63737fe640 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558fad84a940 0x558fad84cde0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2b7556879b5ee37 server_cookie=6bf7fb391b3e8078 in_seq=0 out_seq=0 2024-09-19T06:09:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.022+0000 7f6352ffd640 10 client.26474 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:01.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.023+0000 7f6352ffd640 1 -- 172.21.15.5:0/170629763 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fad7a27f0 con 0x558fad84a940 2024-09-19T06:09:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.023+0000 7f6352ffd640 10 client.26474 handle_command_reply: tid=0 2024-09-19T06:09:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.023+0000 7f637c1bc740 1 client.26474 shutdown 2024-09-19T06:09:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.023+0000 7f637c1bc740 1 -- 172.21.15.5:0/170629763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 msgr2=0x558fad838670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.023+0000 7f637c1bc740 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad838670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f636c031d50 tx=0x7f636c0095c0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.023+0000 7f637c1bc740 1 -- 172.21.15.5:0/170629763 shutdown_connections 2024-09-19T06:09:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.023+0000 7f637c1bc740 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558fad84a940 0x558fad84cde0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.023+0000 7f637c1bc740 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad838670 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.023+0000 7f637c1bc740 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 0x558fad838130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.023+0000 7f637c1bc740 1 --2- 172.21.15.5:0/170629763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad837bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.023+0000 7f637c1bc740 1 -- 172.21.15.5:0/170629763 >> 172.21.15.5:0/170629763 conn(0x558fad82c0f0 msgr2=0x558fad831e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:01.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.023+0000 7f6352ffd640 0 client.26474 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:01.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.024+0000 7f637c1bc740 1 -- 172.21.15.5:0/170629763 shutdown_connections 2024-09-19T06:09:01.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.024+0000 7f637c1bc740 1 -- 172.21.15.5:0/170629763 wait complete. 2024-09-19T06:09:01.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.024+0000 7f637c1bc740 20 client.26474 trim_cache size 0 max 16384 2024-09-19T06:09:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.044+0000 7f637c1bc740 1 Processor -- start 2024-09-19T06:09:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.044+0000 7f637c1bc740 1 -- start start 2024-09-19T06:09:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f637c1bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad831d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f637c1bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 0x558fad852700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f637c1bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad852c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f637c1bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ad8c0 con 0x558fad830480 2024-09-19T06:09:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f637c1bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63741af400 con 0x558fad83ad40 2024-09-19T06:09:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f637c1bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f63741adf90 con 0x558fad82fb60 2024-09-19T06:09:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f6373fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad852c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f6373fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad852c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37838/0 (socket says 172.21.15.5:37838) 2024-09-19T06:09:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f6373fff640 1 -- 172.21.15.5:0/3728071598 learned_addr learned my addr 172.21.15.5:0/3728071598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f63737fe640 1 --2- 172.21.15.5:0/3728071598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad831d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f6372ffd640 1 --2- 172.21.15.5:0/3728071598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 0x558fad852700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f6373fff640 1 -- 172.21.15.5:0/3728071598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 msgr2=0x558fad831d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f6373fff640 1 --2- 172.21.15.5:0/3728071598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad831d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f6373fff640 1 -- 172.21.15.5:0/3728071598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 msgr2=0x558fad852700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f6373fff640 1 --2- 172.21.15.5:0/3728071598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 0x558fad852700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f6373fff640 1 -- 172.21.15.5:0/3728071598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fad83a4c0 con 0x558fad83ad40 2024-09-19T06:09:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.045+0000 7f6372ffd640 1 --2- 172.21.15.5:0/3728071598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 0x558fad852700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:09:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.046+0000 7f6373fff640 1 --2- 172.21.15.5:0/3728071598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad852c40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f636c031d20 tx=0x7f636c031450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa9d6aceff2e6ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.046+0000 7f63597fa640 1 -- 172.21.15.5:0/3728071598 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c04a460 con 0x558fad83ad40 2024-09-19T06:09:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.046+0000 7f63597fa640 1 -- 172.21.15.5:0/3728071598 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f636c036040 con 0x558fad83ad40 2024-09-19T06:09:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.046+0000 7f63597fa640 1 -- 172.21.15.5:0/3728071598 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c030a90 con 0x558fad83ad40 2024-09-19T06:09:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.047+0000 7f637c1bc740 1 -- 172.21.15.5:0/3728071598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 msgr2=0x558fad852c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.047+0000 7f637c1bc740 1 --2- 172.21.15.5:0/3728071598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad852c40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f636c031d20 tx=0x7f636c031450 comp rx=0 tx=0).stop 2024-09-19T06:09:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.047+0000 7f637c1bc740 1 -- 172.21.15.5:0/3728071598 shutdown_connections 2024-09-19T06:09:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.047+0000 7f637c1bc740 1 --2- 172.21.15.5:0/3728071598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad83ad40 0x558fad852c40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.047+0000 7f637c1bc740 1 --2- 172.21.15.5:0/3728071598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad830480 0x558fad852700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.047+0000 7f637c1bc740 1 --2- 172.21.15.5:0/3728071598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad831d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.047+0000 7f637c1bc740 1 -- 172.21.15.5:0/3728071598 >> 172.21.15.5:0/3728071598 conn(0x558fad82c0f0 msgr2=0x558fad831600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.047+0000 7f637c1bc740 1 -- 172.21.15.5:0/3728071598 shutdown_connections 2024-09-19T06:09:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.047+0000 7f637c1bc740 1 -- 172.21.15.5:0/3728071598 wait complete. 2024-09-19T06:09:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.047+0000 7f637c1bc740 1 -- start start 2024-09-19T06:09:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.047+0000 7f637c1bc740 1 Processor -- start 2024-09-19T06:09:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f637c1bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad83f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f637c1bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad830480 0x558fad83f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f637c1bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad83ad40 0x558fad8b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f637c1bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63741aed30 con 0x558fad83ad40 2024-09-19T06:09:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f637c1bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ae660 con 0x558fad830480 2024-09-19T06:09:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f637c1bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ab8a0 con 0x558fad82fb60 2024-09-19T06:09:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f637c1bc740 10 client.0 _ll_register_callbacks cb 0x7f6368005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f6373fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad83ad40 0x558fad8b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f6373fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad83ad40 0x558fad8b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34880/0 (socket says 172.21.15.5:34880) 2024-09-19T06:09:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f6373fff640 1 -- 172.21.15.5:0/2750478767 learned_addr learned my addr 172.21.15.5:0/2750478767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f63737fe640 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad83f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f6372ffd640 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad830480 0x558fad83f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f63737fe640 1 -- 172.21.15.5:0/2750478767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad830480 msgr2=0x558fad83f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f63737fe640 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad830480 0x558fad83f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f63737fe640 1 -- 172.21.15.5:0/2750478767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad83ad40 msgr2=0x558fad8b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f63737fe640 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad83ad40 0x558fad8b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f63737fe640 1 -- 172.21.15.5:0/2750478767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fad8b9020 con 0x558fad82fb60 2024-09-19T06:09:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f6373fff640 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad83ad40 0x558fad8b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.048+0000 7f63737fe640 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad83f0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f63640cd7f0 tx=0x7f636406b130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b10e7e4f18d5b2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f6352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:09:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f6352ffd640 1 -- 172.21.15.5:0/2750478767 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636406e020 con 0x558fad82fb60 2024-09-19T06:09:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f6352ffd640 1 -- 172.21.15.5:0/2750478767 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f63640a4b60 con 0x558fad82fb60 2024-09-19T06:09:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f637c1bc740 1 -- 172.21.15.5:0/2750478767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f636c036040 con 0x558fad82fb60 2024-09-19T06:09:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f6352ffd640 1 -- 172.21.15.5:0/2750478767 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6364026040 con 0x558fad82fb60 2024-09-19T06:09:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f637c1bc740 10 client.26392 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f637c1bc740 1 -- 172.21.15.5:0/2750478767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558fad8ba270 con 0x558fad82fb60 2024-09-19T06:09:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f6352ffd640 1 -- 172.21.15.5:0/2750478767 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f636c036040 con 0x558fad82fb60 2024-09-19T06:09:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f6352ffd640 1 -- 172.21.15.5:0/2750478767 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f6364010430 con 0x558fad82fb60 2024-09-19T06:09:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f637c1bc740 10 client.26392 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f637c1bc740 10 client.26392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f637c1bc740 20 client.26392 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f637c1bc740 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558fad8ba9e0 0x558fad8bcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f637c1bc740 4 client.26392 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:09:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.049+0000 7f637c1bc740 1 -- 172.21.15.5:0/2750478767 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558fad79cd00 con 0x558fad8ba9e0 2024-09-19T06:09:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.050+0000 7f6372ffd640 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558fad8ba9e0 0x558fad8bcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.050+0000 7f6372ffd640 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558fad8ba9e0 0x558fad8bcdd0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df4c18297c5fae08 server_cookie=9b1d5a2d5ba3c44e in_seq=0 out_seq=0 2024-09-19T06:09:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.050+0000 7f6352ffd640 10 client.26392 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f6352ffd640 1 -- 172.21.15.5:0/2750478767 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558fad79cd00 con 0x558fad8ba9e0 2024-09-19T06:09:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f6352ffd640 10 client.26392 handle_command_reply: tid=0 2024-09-19T06:09:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f637c1bc740 1 client.26392 shutdown 2024-09-19T06:09:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f637c1bc740 1 -- 172.21.15.5:0/2750478767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 msgr2=0x558fad83f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f637c1bc740 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad83f0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f63640cd7f0 tx=0x7f636406b130 comp rx=0 tx=0).stop 2024-09-19T06:09:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f637c1bc740 1 -- 172.21.15.5:0/2750478767 shutdown_connections 2024-09-19T06:09:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f637c1bc740 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558fad8ba9e0 0x558fad8bcdd0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f637c1bc740 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fad83ad40 0x558fad8b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f637c1bc740 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fad830480 0x558fad83f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f637c1bc740 1 --2- 172.21.15.5:0/2750478767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fad82fb60 0x558fad83f0c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f6352ffd640 0 client.26392 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f637c1bc740 1 -- 172.21.15.5:0/2750478767 >> 172.21.15.5:0/2750478767 conn(0x558fad82c0f0 msgr2=0x558fad82b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f637c1bc740 1 -- 172.21.15.5:0/2750478767 shutdown_connections 2024-09-19T06:09:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.051+0000 7f637c1bc740 1 -- 172.21.15.5:0/2750478767 wait complete. 2024-09-19T06:09:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f637c1bc740 20 client.26392 trim_cache size 0 max 16384 2024-09-19T06:09:01.060 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:09:01.060 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:09:01.060 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "86d4975c-4157-4137-8d41-6852f8a90e65", 2024-09-19T06:09:01.060 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:09:01.060 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:09:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f6379de3640 1 -- 172.21.15.5:0/3465885699 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f635c0a33b0 msgr2=0x7f635c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f6379de3640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f635c0a33b0 0x7f635c0a5870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f636c004af0 tx=0x7f636c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f6379de3640 1 -- 172.21.15.5:0/3465885699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374102d30 msgr2=0x7f63741984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f6379de3640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374102d30 0x7f63741984d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f636400e9f0 tx=0x7f636400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f6379de3640 1 -- 172.21.15.5:0/3465885699 shutdown_connections 2024-09-19T06:09:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f6379de3640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f635c0a33b0 0x7f635c0a5870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f6379de3640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63741082a0 0x7f637419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f6379de3640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63741036e0 0x7f6374198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f6379de3640 1 --2- 172.21.15.5:0/3465885699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374102d30 0x7f63741984d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f6379de3640 1 -- 172.21.15.5:0/3465885699 >> 172.21.15.5:0/3465885699 conn(0x7f63740fc7d0 msgr2=0x7f6374105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f6379de3640 1 -- 172.21.15.5:0/3465885699 shutdown_connections 2024-09-19T06:09:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:01.052+0000 7f6379de3640 1 -- 172.21.15.5:0/3465885699 wait complete. 2024-09-19T06:09:01.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 137429 2.74 MB/sec execute 147 sec latency 21.424 ms 2024-09-19T06:09:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: pgmap v174: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-19T06:09:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:09:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:01.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:01.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:01.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:01.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:01 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.033 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:09:02.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: pgmap v174: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-19T06:09:02.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:09:02.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:02.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:02.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:02.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:02.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:02.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:02.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:01 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:09:02.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: pgmap v174: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:01 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:02.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 138134 2.76 MB/sec execute 148 sec latency 15.994 ms 2024-09-19T06:09:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.858+0000 7fd1b1785640 1 -- 172.21.15.5:0/3963400964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1ac101de0 msgr2=0x7fd1ac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.858+0000 7fd1b1785640 1 --2- 172.21.15.5:0/3963400964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1ac101de0 0x7fd1ac1021c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009900 tx=0x7fd19c02a410 comp rx=0 tx=0).stop 2024-09-19T06:09:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.859+0000 7fd1b1785640 1 -- 172.21.15.5:0/3963400964 shutdown_connections 2024-09-19T06:09:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.859+0000 7fd1b1785640 1 --2- 172.21.15.5:0/3963400964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1ac106f10 0x7fd1ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.859+0000 7fd1b1785640 1 --2- 172.21.15.5:0/3963400964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1ac102790 0x7fd1ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.859+0000 7fd1b1785640 1 --2- 172.21.15.5:0/3963400964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1ac101de0 0x7fd1ac1021c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.859+0000 7fd1b1785640 1 -- 172.21.15.5:0/3963400964 >> 172.21.15.5:0/3963400964 conn(0x7fd1ac0fd8c0 msgr2=0x7fd1ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.859+0000 7fd1b1785640 1 -- 172.21.15.5:0/3963400964 shutdown_connections 2024-09-19T06:09:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.859+0000 7fd1b1785640 1 -- 172.21.15.5:0/3963400964 wait complete. 2024-09-19T06:09:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.859+0000 7fd1b1785640 1 Processor -- start 2024-09-19T06:09:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.859+0000 7fd1b1785640 1 -- start start 2024-09-19T06:09:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1b1785640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1ac101de0 0x7fd1ac19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1b1785640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1ac102790 0x7fd1ac19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1b1785640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1ac106f10 0x7fd1ac19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1b1785640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac111e20 con 0x7fd1ac101de0 2024-09-19T06:09:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1b1785640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac111ca0 con 0x7fd1ac102790 2024-09-19T06:09:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1b1785640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac111fa0 con 0x7fd1ac106f10 2024-09-19T06:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1ac101de0 0x7fd1ac19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1ac101de0 0x7fd1ac19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34906/0 (socket says 172.21.15.5:34906) 2024-09-19T06:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1aaffd640 1 -- 172.21.15.5:0/2843400263 learned_addr learned my addr 172.21.15.5:0/2843400263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1aa7fc640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1ac102790 0x7fd1ac19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1ab7fe640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1ac106f10 0x7fd1ac19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1aaffd640 1 -- 172.21.15.5:0/2843400263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1ac106f10 msgr2=0x7fd1ac19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1ac106f10 0x7fd1ac19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1aaffd640 1 -- 172.21.15.5:0/2843400263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1ac102790 msgr2=0x7fd1ac19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1ac102790 0x7fd1ac19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.860+0000 7fd1aaffd640 1 -- 172.21.15.5:0/2843400263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac19f570 con 0x7fd1ac101de0 2024-09-19T06:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.861+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1ac101de0 0x7fd1ac19a5c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd19c005940 tx=0x7fd19c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.861+0000 7fd193fff640 1 -- 172.21.15.5:0/2843400263 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c03f030 con 0x7fd1ac101de0 2024-09-19T06:09:02.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.861+0000 7fd1b1785640 1 -- 172.21.15.5:0/2843400263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ac19f800 con 0x7fd1ac101de0 2024-09-19T06:09:02.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.861+0000 7fd193fff640 1 -- 172.21.15.5:0/2843400263 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd19c0044c0 con 0x7fd1ac101de0 2024-09-19T06:09:02.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.861+0000 7fd193fff640 1 -- 172.21.15.5:0/2843400263 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c035070 con 0x7fd1ac101de0 2024-09-19T06:09:02.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.861+0000 7fd1b1785640 1 -- 172.21.15.5:0/2843400263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ac1a7140 con 0x7fd1ac101de0 2024-09-19T06:09:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.863+0000 7fd193fff640 1 -- 172.21.15.5:0/2843400263 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd19c042030 con 0x7fd1ac101de0 2024-09-19T06:09:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1b3b5e740 1 Processor -- start 2024-09-19T06:09:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1b3b5e740 1 -- start start 2024-09-19T06:09:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1b3b5e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 0x564a9ee49090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1b3b5e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 0x564a9ee53950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1b3b5e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee53e90 0x564a9ee56280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1b3b5e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a9eda62e0 con 0x564a9ee48cb0 2024-09-19T06:09:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 0x564a9ee49090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 0x564a9ee49090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34910/0 (socket says 172.21.15.5:34910) 2024-09-19T06:09:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1b3b5e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564a9ed4dce0 con 0x564a9ee495d0 2024-09-19T06:09:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1ab7fe640 1 -- 172.21.15.5:0/2113057519 learned_addr learned my addr 172.21.15.5:0/2113057519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2113057519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564a9eda22a0 con 0x564a9ee53e90 2024-09-19T06:09:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/2113057519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 0x564a9ee53950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.864+0000 7fd1aa7fc640 1 --2- 172.21.15.5:0/2113057519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee53e90 0x564a9ee56280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd193fff640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1940a33b0 0x7fd1940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd1ab7fe640 1 -- 172.21.15.5:0/2113057519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee53e90 msgr2=0x564a9ee56280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd1ab7fe640 1 --2- 172.21.15.5:0/2113057519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee53e90 0x564a9ee56280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd1ab7fe640 1 -- 172.21.15.5:0/2113057519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 msgr2=0x564a9ee53950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd1ab7fe640 1 --2- 172.21.15.5:0/2113057519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 0x564a9ee53950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd1ab7fe640 1 -- 172.21.15.5:0/2113057519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a9ed32440 con 0x564a9ee48cb0 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/2113057519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 0x564a9ee53950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd193fff640 1 -- 172.21.15.5:0/2843400263 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fd19c0efce0 con 0x7fd1ac101de0 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd1ab7fe640 1 --2- 172.21.15.5:0/2113057519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 0x564a9ee49090 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009f50 tx=0x7fd1a40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21418cd128024cd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd1ab7fe640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1940a33b0 0x7fd1940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd1917fa640 1 -- 172.21.15.5:0/2113057519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a401e070 con 0x564a9ee48cb0 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd1917fa640 1 -- 172.21.15.5:0/2113057519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1a400fc30 con 0x564a9ee48cb0 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.865+0000 7fd1917fa640 1 -- 172.21.15.5:0/2113057519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400e3b0 con 0x564a9ee48cb0 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.866+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2113057519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 msgr2=0x564a9ee49090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.866+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/2113057519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 0x564a9ee49090 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009f50 tx=0x7fd1a40118d0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.866+0000 7fd1ab7fe640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1940a33b0 0x7fd1940a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd1a401b000 tx=0x7fd1a400ead0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:02.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.866+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2113057519 shutdown_connections 2024-09-19T06:09:02.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.866+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/2113057519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee53e90 0x564a9ee56280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.866+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/2113057519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 0x564a9ee53950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.866+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/2113057519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 0x564a9ee49090 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.866+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2113057519 >> 172.21.15.5:0/2113057519 conn(0x564a9ee45240 msgr2=0x564a9ee45620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:02.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.866+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2113057519 shutdown_connections 2024-09-19T06:09:02.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.866+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2113057519 wait complete. 2024-09-19T06:09:02.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1b3b5e740 1 -- start start 2024-09-19T06:09:02.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1b3b5e740 1 Processor -- start 2024-09-19T06:09:02.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1b3b5e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee48cb0 0x564a9ee537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1b3b5e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 0x564a9ee50ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1b3b5e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee53e90 0x564a9ee51010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1b3b5e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a9ee56a60 con 0x564a9ee53e90 2024-09-19T06:09:02.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1b3b5e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564a9ed61050 con 0x564a9ee495d0 2024-09-19T06:09:02.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1b3b5e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564a9ee56be0 con 0x564a9ee48cb0 2024-09-19T06:09:02.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1aa7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 0x564a9ee50ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1aa7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 0x564a9ee50ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37888/0 (socket says 172.21.15.5:37888) 2024-09-19T06:09:02.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1aa7fc640 1 -- 172.21.15.5:0/2038868013 learned_addr learned my addr 172.21.15.5:0/2038868013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:02.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee48cb0 0x564a9ee537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1ab7fe640 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee53e90 0x564a9ee51010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.867+0000 7fd1b3b5e740 10 client.0 _ll_register_callbacks cb 0x564a9eda1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:02.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd1aa7fc640 1 -- 172.21.15.5:0/2038868013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee48cb0 msgr2=0x564a9ee537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd1aa7fc640 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee48cb0 0x564a9ee537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd1aa7fc640 1 -- 172.21.15.5:0/2038868013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee53e90 msgr2=0x564a9ee51010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd1aa7fc640 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee53e90 0x564a9ee51010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd1aa7fc640 1 -- 172.21.15.5:0/2038868013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a9eb173c0 con 0x564a9ee495d0 2024-09-19T06:09:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd1ab7fe640 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee53e90 0x564a9ee51010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd1aa7fc640 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 0x564a9ee50ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0012720 tx=0x7fd1a0012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e776e6f4c0f72d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd18eff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd18eff5640 1 -- 172.21.15.5:0/2038868013 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000f920 con 0x564a9ee495d0 2024-09-19T06:09:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2038868013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a400fc30 con 0x564a9ee495d0 2024-09-19T06:09:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd18eff5640 1 -- 172.21.15.5:0/2038868013 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fd1a000fac0 con 0x564a9ee495d0 2024-09-19T06:09:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd18eff5640 1 -- 172.21.15.5:0/2038868013 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd1a000fcc0 con 0x564a9ee495d0 2024-09-19T06:09:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd1b3b5e740 10 client.26492 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.868+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2038868013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a9ee624c0 con 0x564a9ee495d0 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.869+0000 7fd18eff5640 1 -- 172.21.15.5:0/2038868013 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1a400fc30 con 0x564a9ee495d0 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.869+0000 7fd18eff5640 1 -- 172.21.15.5:0/2038868013 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fd1a001c420 con 0x564a9ee495d0 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.869+0000 7fd1b3b5e740 10 client.26492 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.869+0000 7fd1b3b5e740 10 client.26492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.869+0000 7fd1b3b5e740 20 client.26492 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.869+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564a9ee63d60 0x564a9ee661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.869+0000 7fd1b3b5e740 4 client.26492 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.869+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2038868013 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a9edb3850 con 0x564a9ee63d60 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.869+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564a9ee63d60 0x564a9ee661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.869+0000 7fd18eff5640 10 client.26492 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.870+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564a9ee63d60 0x564a9ee661c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3af2cb59487990f server_cookie=190c63d194a96a67 in_seq=0 out_seq=0 2024-09-19T06:09:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd18eff5640 1 -- 172.21.15.5:0/2038868013 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a9edb3850 con 0x564a9ee63d60 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd18eff5640 10 client.26492 handle_command_reply: tid=0 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd1b3b5e740 1 client.26492 shutdown 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2038868013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 msgr2=0x564a9ee50ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 0x564a9ee50ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0012720 tx=0x7fd1a0012be0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2038868013 shutdown_connections 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564a9ee63d60 0x564a9ee661c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee53e90 0x564a9ee51010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee495d0 0x564a9ee50ab0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/2038868013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee48cb0 0x564a9ee537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd18eff5640 0 client.26492 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2038868013 >> 172.21.15.5:0/2038868013 conn(0x564a9ee45240 msgr2=0x564a9ee3bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2038868013 shutdown_connections 2024-09-19T06:09:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/2038868013 wait complete. 2024-09-19T06:09:02.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.871+0000 7fd1b3b5e740 20 client.26492 trim_cache size 0 max 16384 2024-09-19T06:09:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1b3b5e740 1 Processor -- start 2024-09-19T06:09:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1b3b5e740 1 -- start start 2024-09-19T06:09:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1b3b5e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 0x564a9ee5fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1b3b5e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee495d0 0x564a9ee50370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1b3b5e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee53e90 0x564a9ee4c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1b3b5e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1ab5c0 con 0x564a9ee48cb0 2024-09-19T06:09:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1b3b5e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1ad100 con 0x564a9ee53e90 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1b3b5e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1abc90 con 0x564a9ee495d0 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 0x564a9ee5fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 0x564a9ee5fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34928/0 (socket says 172.21.15.5:34928) 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1aaffd640 1 -- 172.21.15.5:0/1260089595 learned_addr learned my addr 172.21.15.5:0/1260089595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1aa7fc640 1 --2- 172.21.15.5:0/1260089595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee495d0 0x564a9ee50370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1ab7fe640 1 --2- 172.21.15.5:0/1260089595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee53e90 0x564a9ee4c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1aaffd640 1 -- 172.21.15.5:0/1260089595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee495d0 msgr2=0x564a9ee50370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/1260089595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee495d0 0x564a9ee50370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1aaffd640 1 -- 172.21.15.5:0/1260089595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee53e90 msgr2=0x564a9ee4c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/1260089595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee53e90 0x564a9ee4c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.893+0000 7fd1aaffd640 1 -- 172.21.15.5:0/1260089595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a9ee49400 con 0x564a9ee48cb0 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.894+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/1260089595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 0x564a9ee5fcc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0ed4a0 tx=0x7fd19c0f3830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b15b1018ddaa505 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.894+0000 7fd190ff9640 1 -- 172.21.15.5:0/1260089595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c049040 con 0x564a9ee48cb0 2024-09-19T06:09:02.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.894+0000 7fd190ff9640 1 -- 172.21.15.5:0/1260089595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd19c0dc0b0 con 0x564a9ee48cb0 2024-09-19T06:09:02.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.894+0000 7fd190ff9640 1 -- 172.21.15.5:0/1260089595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c003d60 con 0x564a9ee48cb0 2024-09-19T06:09:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.895+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/1260089595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 msgr2=0x564a9ee5fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.895+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/1260089595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 0x564a9ee5fcc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0ed4a0 tx=0x7fd19c0f3830 comp rx=0 tx=0).stop 2024-09-19T06:09:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.895+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/1260089595 shutdown_connections 2024-09-19T06:09:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.895+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/1260089595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee53e90 0x564a9ee4c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.895+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/1260089595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee495d0 0x564a9ee50370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.895+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/1260089595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee48cb0 0x564a9ee5fcc0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.895+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/1260089595 >> 172.21.15.5:0/1260089595 conn(0x564a9ee45240 msgr2=0x564a9ee4b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.895+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/1260089595 shutdown_connections 2024-09-19T06:09:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.895+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/1260089595 wait complete. 2024-09-19T06:09:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.895+0000 7fd1b3b5e740 1 -- start start 2024-09-19T06:09:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.895+0000 7fd1b3b5e740 1 Processor -- start 2024-09-19T06:09:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.895+0000 7fd1b3b5e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee48cb0 0x564a9ee5c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1b3b5e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee495d0 0x564a9ee57760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1b3b5e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee53e90 0x564a9ee57ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1b3b5e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1aca30 con 0x564a9ee53e90 2024-09-19T06:09:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1b3b5e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1ac360 con 0x564a9ee48cb0 2024-09-19T06:09:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1b3b5e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1a95c0 con 0x564a9ee495d0 2024-09-19T06:09:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1b3b5e740 10 client.0 _ll_register_callbacks cb 0x7fd1a0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee53e90 0x564a9ee57ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee53e90 0x564a9ee57ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34930/0 (socket says 172.21.15.5:34930) 2024-09-19T06:09:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1ab7fe640 1 -- 172.21.15.5:0/3487805569 learned_addr learned my addr 172.21.15.5:0/3487805569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee48cb0 0x564a9ee5c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1aa7fc640 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee495d0 0x564a9ee57760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1aaffd640 1 -- 172.21.15.5:0/3487805569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee495d0 msgr2=0x564a9ee57760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee495d0 0x564a9ee57760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1aaffd640 1 -- 172.21.15.5:0/3487805569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee53e90 msgr2=0x564a9ee57ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee53e90 0x564a9ee57ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1aaffd640 1 -- 172.21.15.5:0/3487805569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a9ee58310 con 0x564a9ee48cb0 2024-09-19T06:09:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1aa7fc640 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee495d0 0x564a9ee57760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:09:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd1aaffd640 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee48cb0 0x564a9ee5c3e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0ed4a0 tx=0x7fd19c0ed420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f3a2de32c4dc47b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd18eff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd18eff5640 1 -- 172.21.15.5:0/3487805569 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c049040 con 0x564a9ee48cb0 2024-09-19T06:09:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.896+0000 7fd18eff5640 1 -- 172.21.15.5:0/3487805569 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fd19c0ed6c0 con 0x564a9ee48cb0 2024-09-19T06:09:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/3487805569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564a9edadd60 con 0x564a9ee48cb0 2024-09-19T06:09:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd18eff5640 1 -- 172.21.15.5:0/3487805569 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd19c0c0380 con 0x564a9ee48cb0 2024-09-19T06:09:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd1b3b5e740 10 client.26498 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/3487805569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a9eece920 con 0x564a9ee48cb0 2024-09-19T06:09:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd18eff5640 1 -- 172.21.15.5:0/3487805569 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564a9edadd60 con 0x564a9ee48cb0 2024-09-19T06:09:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd18eff5640 1 -- 172.21.15.5:0/3487805569 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fd19c04c060 con 0x564a9ee48cb0 2024-09-19T06:09:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd1b3b5e740 10 client.26498 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd1b3b5e740 10 client.26498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd1b3b5e740 20 client.26498 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564a9eecefd0 0x564a9eed13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd1b3b5e740 4 client.26498 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:09:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/3487805569 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a9ed2a210 con 0x564a9eecefd0 2024-09-19T06:09:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.897+0000 7fd1aa7fc640 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564a9eecefd0 0x564a9eed13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd18eff5640 10 client.26498 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd1aa7fc640 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564a9eecefd0 0x564a9eed13c0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c14201baa3fa4622 server_cookie=84301e4ed56d4153 in_seq=0 out_seq=0 2024-09-19T06:09:02.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd18eff5640 1 -- 172.21.15.5:0/3487805569 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564a9ed2a210 con 0x564a9eecefd0 2024-09-19T06:09:02.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd18eff5640 10 client.26498 handle_command_reply: tid=0 2024-09-19T06:09:02.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd1b3b5e740 1 client.26498 shutdown 2024-09-19T06:09:02.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/3487805569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee48cb0 msgr2=0x564a9ee5c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:02.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee48cb0 0x564a9ee5c3e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0ed4a0 tx=0x7fd19c0ed420 comp rx=0 tx=0).stop 2024-09-19T06:09:02.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/3487805569 shutdown_connections 2024-09-19T06:09:02.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564a9eecefd0 0x564a9eed13c0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a9ee53e90 0x564a9ee57ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a9ee495d0 0x564a9ee57760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd18eff5640 0 client.26498 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:02.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd1b3b5e740 1 --2- 172.21.15.5:0/3487805569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a9ee48cb0 0x564a9ee5c3e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.898+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/3487805569 >> 172.21.15.5:0/3487805569 conn(0x564a9ee45240 msgr2=0x564a9ee4c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/3487805569 shutdown_connections 2024-09-19T06:09:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b3b5e740 1 -- 172.21.15.5:0/3487805569 wait complete. 2024-09-19T06:09:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b3b5e740 20 client.26498 trim_cache size 0 max 16384 2024-09-19T06:09:02.908 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:09:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:09:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:09:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "86d4975c-4157-4137-8d41-6852f8a90e65": { 2024-09-19T06:09:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:09:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "86d4975c-4157-4137-8d41-6852f8a90e65", 2024-09-19T06:09:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:09:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:09:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:09:02.909 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:09:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b1785640 1 -- 172.21.15.5:0/2843400263 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1940a33b0 msgr2=0x7fd1940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b1785640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1940a33b0 0x7fd1940a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd1a401b000 tx=0x7fd1a400ead0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b1785640 1 -- 172.21.15.5:0/2843400263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1ac101de0 msgr2=0x7fd1ac19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b1785640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1ac101de0 0x7fd1ac19a5c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd19c005940 tx=0x7fd19c0040c0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b1785640 1 -- 172.21.15.5:0/2843400263 shutdown_connections 2024-09-19T06:09:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b1785640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1940a33b0 0x7fd1940a5870 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b1785640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1ac106f10 0x7fd1ac19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b1785640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1ac102790 0x7fd1ac19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b1785640 1 --2- 172.21.15.5:0/2843400263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1ac101de0 0x7fd1ac19a5c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.899+0000 7fd1b1785640 1 -- 172.21.15.5:0/2843400263 >> 172.21.15.5:0/2843400263 conn(0x7fd1ac0fd8c0 msgr2=0x7fd1ac0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.900+0000 7fd1b1785640 1 -- 172.21.15.5:0/2843400263 shutdown_connections 2024-09-19T06:09:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:02.900+0000 7fd1b1785640 1 -- 172.21.15.5:0/2843400263 wait complete. 2024-09-19T06:09:03.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:02 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:03.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:02 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:03.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:02 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:03.735 INFO:tasks.cephfs.filesystem:scrub status for tag:86d4975c-4157-4137-8d41-6852f8a90e65 - {'path': '/', 'tag': '86d4975c-4157-4137-8d41-6852f8a90e65', 'options': 'recursive,force'} 2024-09-19T06:09:03.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 138610 2.77 MB/sec execute 149 sec latency 12.358 ms 2024-09-19T06:09:04.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:03 smithi005 ceph-mon[30955]: pgmap v175: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-09-19T06:09:04.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:03 smithi157 ceph-mon[35346]: pgmap v175: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-09-19T06:09:04.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:03 smithi049 ceph-mon[34640]: pgmap v175: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-09-19T06:09:04.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 138872 2.76 MB/sec execute 150 sec latency 15.918 ms 2024-09-19T06:09:05.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 139341 2.74 MB/sec execute 151 sec latency 15.575 ms 2024-09-19T06:09:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:05 smithi157 ceph-mon[35346]: pgmap v176: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-09-19T06:09:06.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:05 smithi005 ceph-mon[30955]: pgmap v176: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-09-19T06:09:06.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:05 smithi049 ceph-mon[34640]: pgmap v176: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-09-19T06:09:06.829 INFO:tasks.workunit.client.0.smithi005.stdout: 1 139878 2.75 MB/sec execute 152 sec latency 18.813 ms 2024-09-19T06:09:07.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 140430 2.74 MB/sec execute 153 sec latency 20.737 ms 2024-09-19T06:09:08.181 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:07 smithi005 ceph-mon[30955]: pgmap v177: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-09-19T06:09:08.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:07 smithi157 ceph-mon[35346]: pgmap v177: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-09-19T06:09:08.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:07 smithi049 ceph-mon[34640]: pgmap v177: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-09-19T06:09:08.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 140944 2.74 MB/sec execute 154 sec latency 22.893 ms 2024-09-19T06:09:09.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 141673 2.76 MB/sec execute 155 sec latency 15.487 ms 2024-09-19T06:09:10.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:09 smithi157 ceph-mon[35346]: pgmap v178: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-19T06:09:10.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:09 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:09:10.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:09 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:09:10.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:09 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:09:10.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:09 smithi005 ceph-mon[30955]: pgmap v178: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-19T06:09:10.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:09 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:09:10.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:09 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:09:10.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:09 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:09:10.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:09 smithi049 ceph-mon[34640]: pgmap v178: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-19T06:09:10.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:09 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:09:10.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:09 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:09:10.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:09 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:09:10.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 142138 2.77 MB/sec execute 156 sec latency 15.342 ms 2024-09-19T06:09:11.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 142418 2.76 MB/sec execute 157 sec latency 12.125 ms 2024-09-19T06:09:12.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:11 smithi157 ceph-mon[35346]: pgmap v179: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-19T06:09:12.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:11 smithi005 ceph-mon[30955]: pgmap v179: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-19T06:09:12.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:11 smithi049 ceph-mon[34640]: pgmap v179: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-19T06:09:12.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 142891 2.74 MB/sec execute 158 sec latency 14.996 ms 2024-09-19T06:09:13.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 143407 2.75 MB/sec execute 159 sec latency 61.546 ms 2024-09-19T06:09:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.936+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/3551376533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb58105ea0 msgr2=0x7fcb58106330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.936+0000 7fcb5ef5f640 1 --2- 172.21.15.5:0/3551376533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb58105ea0 0x7fcb58106330 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009cd0 tx=0x7fcb5002a970 comp rx=0 tx=0).stop 2024-09-19T06:09:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.936+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/3551376533 shutdown_connections 2024-09-19T06:09:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.936+0000 7fcb5ef5f640 1 --2- 172.21.15.5:0/3551376533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb58105ea0 0x7fcb58106330 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.936+0000 7fcb5ef5f640 1 --2- 172.21.15.5:0/3551376533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb58101390 0x7fcb58105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.936+0000 7fcb5ef5f640 1 --2- 172.21.15.5:0/3551376533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb58100a70 0x7fcb58100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.936+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/3551376533 >> 172.21.15.5:0/3551376533 conn(0x7fcb580779a0 msgr2=0x7fcb580fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.936+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/3551376533 shutdown_connections 2024-09-19T06:09:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.936+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/3551376533 wait complete. 2024-09-19T06:09:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ef5f640 1 Processor -- start 2024-09-19T06:09:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ef5f640 1 -- start start 2024-09-19T06:09:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ef5f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb58100a70 0x7fcb581a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ef5f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb58101390 0x7fcb581a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ef5f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb58105ea0 0x7fcb5819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ef5f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb58118340 con 0x7fcb58101390 2024-09-19T06:09:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ef5f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581181c0 con 0x7fcb58100a70 2024-09-19T06:09:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ef5f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581184c0 con 0x7fcb58105ea0 2024-09-19T06:09:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb58101390 0x7fcb581a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb58101390 0x7fcb581a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48298/0 (socket says 172.21.15.5:48298) 2024-09-19T06:09:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb57fff640 1 -- 172.21.15.5:0/4052266168 learned_addr learned my addr 172.21.15.5:0/4052266168 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb58100a70 0x7fcb581a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5d4fc640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb58105ea0 0x7fcb5819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/4052266168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb58105ea0 msgr2=0x7fcb5819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb58105ea0 0x7fcb5819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/4052266168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb58101390 msgr2=0x7fcb581a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb58101390 0x7fcb581a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/4052266168 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5819f000 con 0x7fcb58100a70 2024-09-19T06:09:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.937+0000 7fcb57fff640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb58101390 0x7fcb581a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.938+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb58100a70 0x7fcb581a4680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00e9f0 tx=0x7fcb4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.938+0000 7fcb55ffb640 1 -- 172.21.15.5:0/4052266168 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c00ccc0 con 0x7fcb58100a70 2024-09-19T06:09:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.938+0000 7fcb55ffb640 1 -- 172.21.15.5:0/4052266168 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb4c005020 con 0x7fcb58100a70 2024-09-19T06:09:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.938+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/4052266168 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5819f2f0 con 0x7fcb58100a70 2024-09-19T06:09:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.938+0000 7fcb55ffb640 1 -- 172.21.15.5:0/4052266168 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c004a00 con 0x7fcb58100a70 2024-09-19T06:09:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.938+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/4052266168 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb581ab600 con 0x7fcb58100a70 2024-09-19T06:09:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb61dd5740 1 Processor -- start 2024-09-19T06:09:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb55ffb640 1 -- 172.21.15.5:0/4052266168 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcb4c004ba0 con 0x7fcb58100a70 2024-09-19T06:09:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb61dd5740 1 -- start start 2024-09-19T06:09:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb61dd5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa91af0 0x5565baa91ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb61dd5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa92410 0x5565baa9c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb61dd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa9ccb0 0x5565baa9f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb61dd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565ba9f62f0 con 0x5565baa9ccb0 2024-09-19T06:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb61dd5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5565ba99e270 con 0x5565baa92410 2024-09-19T06:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb61dd5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5565ba9f22b0 con 0x5565baa91af0 2024-09-19T06:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb5ccfb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa92410 0x5565baa9c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa9ccb0 0x5565baa9f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb5d4fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa91af0 0x5565baa91ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb5ccfb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa92410 0x5565baa9c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59236/0 (socket says 172.21.15.5:59236) 2024-09-19T06:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/2653332162 learned_addr learned my addr 172.21.15.5:0/2653332162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/2653332162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa91af0 msgr2=0x5565baa91ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/2653332162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa91af0 0x5565baa91ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/2653332162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa9ccb0 msgr2=0x5565baa9f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/2653332162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa9ccb0 0x5565baa9f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/2653332162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565ba99d340 con 0x5565baa92410 2024-09-19T06:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.941+0000 7fcb57fff640 1 --2- 172.21.15.5:0/2653332162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa9ccb0 0x5565baa9f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.942+0000 7fcb55ffb640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcb480a33b0 0x7fcb480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.942+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/2653332162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa92410 0x5565baa9c770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0cd340 tx=0x7fcb4c0d3f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccaf0dc2bea7e42a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.942+0000 7fcb5d4fc640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcb480a33b0 0x7fcb480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.942+0000 7fcb55ffb640 1 -- 172.21.15.5:0/4052266168 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fcb4c0cfc70 con 0x7fcb58100a70 2024-09-19T06:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.942+0000 7fcb42ffd640 1 -- 172.21.15.5:0/2653332162 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c098200 con 0x5565baa92410 2024-09-19T06:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.942+0000 7fcb42ffd640 1 -- 172.21.15.5:0/2653332162 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb4c0dd070 con 0x5565baa92410 2024-09-19T06:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.942+0000 7fcb42ffd640 1 -- 172.21.15.5:0/2653332162 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c09b8c0 con 0x5565baa92410 2024-09-19T06:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.943+0000 7fcb61dd5740 1 -- 172.21.15.5:0/2653332162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa92410 msgr2=0x5565baa9c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.943+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/2653332162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa92410 0x5565baa9c770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0cd340 tx=0x7fcb4c0d3f40 comp rx=0 tx=0).stop 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.943+0000 7fcb5d4fc640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcb480a33b0 0x7fcb480a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcb50006010 tx=0x7fcb5003c040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.943+0000 7fcb61dd5740 1 -- 172.21.15.5:0/2653332162 shutdown_connections 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.943+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/2653332162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa9ccb0 0x5565baa9f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.943+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/2653332162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa92410 0x5565baa9c770 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.943+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/2653332162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa91af0 0x5565baa91ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.943+0000 7fcb61dd5740 1 -- 172.21.15.5:0/2653332162 >> 172.21.15.5:0/2653332162 conn(0x5565baa8e060 msgr2=0x5565baa8e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.943+0000 7fcb61dd5740 1 -- 172.21.15.5:0/2653332162 shutdown_connections 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.943+0000 7fcb61dd5740 1 -- 172.21.15.5:0/2653332162 wait complete. 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.943+0000 7fcb61dd5740 1 -- start start 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.943+0000 7fcb61dd5740 1 Processor -- start 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb61dd5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 0x5565baa99c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb61dd5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa92410 0x5565baa9a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb61dd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa9ccb0 0x5565baa9a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb61dd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565baa9f880 con 0x5565baa9ccb0 2024-09-19T06:09:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb61dd5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5565ba97f290 con 0x5565baa91af0 2024-09-19T06:09:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb61dd5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5565baa9fa00 con 0x5565baa92410 2024-09-19T06:09:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb61dd5740 10 client.0 _ll_register_callbacks cb 0x5565ba9b0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb5ccfb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 0x5565baa99c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb5ccfb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 0x5565baa99c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59244/0 (socket says 172.21.15.5:59244) 2024-09-19T06:09:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/1293646033 learned_addr learned my addr 172.21.15.5:0/1293646033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb5d4fc640 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa9ccb0 0x5565baa9a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb57fff640 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa92410 0x5565baa9a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/1293646033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa92410 msgr2=0x5565baa9a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa92410 0x5565baa9a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/1293646033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa9ccb0 msgr2=0x5565baa9a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa9ccb0 0x5565baa9a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/1293646033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565ba709210 con 0x5565baa91af0 2024-09-19T06:09:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 0x5565baa99c80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0a46a0 tx=0x7fcb4c0d3740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bfce7e9e1ca22c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.944+0000 7fcb37fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb37fff640 1 -- 172.21.15.5:0/1293646033 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c01c040 con 0x5565baa91af0 2024-09-19T06:09:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb37fff640 1 -- 172.21.15.5:0/1293646033 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fcb4c09a260 con 0x5565baa91af0 2024-09-19T06:09:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb61dd5740 1 -- 172.21.15.5:0/1293646033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb4c0dd070 con 0x5565baa91af0 2024-09-19T06:09:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb37fff640 1 -- 172.21.15.5:0/1293646033 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcb4c09baa0 con 0x5565baa91af0 2024-09-19T06:09:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb61dd5740 10 client.26510 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb61dd5740 1 -- 172.21.15.5:0/1293646033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5565ba7d5620 con 0x5565baa91af0 2024-09-19T06:09:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb37fff640 1 -- 172.21.15.5:0/1293646033 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb4c0dd070 con 0x5565baa91af0 2024-09-19T06:09:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb37fff640 1 -- 172.21.15.5:0/1293646033 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fcb4c09c1c0 con 0x5565baa91af0 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb61dd5740 10 client.26510 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb61dd5740 10 client.26510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb61dd5740 20 client.26510 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5565baaacb90 0x5565baaaef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb61dd5740 4 client.26510 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb61dd5740 1 -- 172.21.15.5:0/1293646033 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565baa03a20 con 0x5565baaacb90 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.945+0000 7fcb57fff640 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5565baaacb90 0x5565baaaef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.946+0000 7fcb37fff640 10 client.26510 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.946+0000 7fcb57fff640 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5565baaacb90 0x5565baaaef80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9a1f03526bdeda2 server_cookie=b1a891871d34e972 in_seq=0 out_seq=0 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.946+0000 7fcb37fff640 1 -- 172.21.15.5:0/1293646033 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565baa03a20 con 0x5565baaacb90 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.946+0000 7fcb37fff640 10 client.26510 handle_command_reply: tid=0 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.946+0000 7fcb61dd5740 1 client.26510 shutdown 2024-09-19T06:09:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.947+0000 7fcb61dd5740 1 -- 172.21.15.5:0/1293646033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 msgr2=0x5565baa99c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.947+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 0x5565baa99c80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0a46a0 tx=0x7fcb4c0d3740 comp rx=0 tx=0).stop 2024-09-19T06:09:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.947+0000 7fcb61dd5740 1 -- 172.21.15.5:0/1293646033 shutdown_connections 2024-09-19T06:09:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.947+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5565baaacb90 0x5565baaaef80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.947+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa9ccb0 0x5565baa9a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.947+0000 7fcb37fff640 0 client.26510 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.947+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa92410 0x5565baa9a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.947+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/1293646033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 0x5565baa99c80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.947+0000 7fcb61dd5740 1 -- 172.21.15.5:0/1293646033 >> 172.21.15.5:0/1293646033 conn(0x5565baa8e060 msgr2=0x5565baa93d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.947+0000 7fcb61dd5740 1 -- 172.21.15.5:0/1293646033 shutdown_connections 2024-09-19T06:09:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.947+0000 7fcb61dd5740 1 -- 172.21.15.5:0/1293646033 wait complete. 2024-09-19T06:09:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.947+0000 7fcb61dd5740 20 client.26510 trim_cache size 0 max 16384 2024-09-19T06:09:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.964+0000 7fcb61dd5740 1 Processor -- start 2024-09-19T06:09:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.964+0000 7fcb61dd5740 1 -- start start 2024-09-19T06:09:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.964+0000 7fcb61dd5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 0x5565baa9a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.964+0000 7fcb61dd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa92410 0x5565baab1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb61dd5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa9ccb0 0x5565baab24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb61dd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581b3e50 con 0x5565baa92410 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb61dd5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581b5990 con 0x5565baa91af0 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb61dd5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581b4520 con 0x5565baa9ccb0 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa92410 0x5565baab1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb5ccfb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 0x5565baa9a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa92410 0x5565baab1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48330/0 (socket says 172.21.15.5:48330) 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb57fff640 1 -- 172.21.15.5:0/129022843 learned_addr learned my addr 172.21.15.5:0/129022843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb5d4fc640 1 --2- 172.21.15.5:0/129022843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa9ccb0 0x5565baab24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb57fff640 1 -- 172.21.15.5:0/129022843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa9ccb0 msgr2=0x5565baab24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb57fff640 1 --2- 172.21.15.5:0/129022843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa9ccb0 0x5565baab24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb57fff640 1 -- 172.21.15.5:0/129022843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 msgr2=0x5565baa9a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb57fff640 1 --2- 172.21.15.5:0/129022843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 0x5565baa9a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb57fff640 1 -- 172.21.15.5:0/129022843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565baa91920 con 0x5565baa92410 2024-09-19T06:09:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/129022843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 0x5565baa9a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb57fff640 1 --2- 172.21.15.5:0/129022843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa92410 0x5565baab1fa0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcb5819ff90 tx=0x7fcb44005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e1fc67f57cdb4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb427fc640 1 -- 172.21.15.5:0/129022843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400faf0 con 0x5565baa92410 2024-09-19T06:09:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb427fc640 1 -- 172.21.15.5:0/129022843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb4400fc90 con 0x5565baa92410 2024-09-19T06:09:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.965+0000 7fcb427fc640 1 -- 172.21.15.5:0/129022843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44026600 con 0x5565baa92410 2024-09-19T06:09:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.966+0000 7fcb61dd5740 1 -- 172.21.15.5:0/129022843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa92410 msgr2=0x5565baab1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.966+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/129022843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa92410 0x5565baab1fa0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcb5819ff90 tx=0x7fcb44005f90 comp rx=0 tx=0).stop 2024-09-19T06:09:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.966+0000 7fcb61dd5740 1 -- 172.21.15.5:0/129022843 shutdown_connections 2024-09-19T06:09:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.966+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/129022843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa9ccb0 0x5565baab24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.966+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/129022843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa92410 0x5565baab1fa0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.966+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/129022843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa91af0 0x5565baa9a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.966+0000 7fcb61dd5740 1 -- 172.21.15.5:0/129022843 >> 172.21.15.5:0/129022843 conn(0x5565baa8e060 msgr2=0x5565baa9bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.966+0000 7fcb61dd5740 1 -- 172.21.15.5:0/129022843 shutdown_connections 2024-09-19T06:09:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.966+0000 7fcb61dd5740 1 -- 172.21.15.5:0/129022843 wait complete. 2024-09-19T06:09:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb61dd5740 1 -- start start 2024-09-19T06:09:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb61dd5740 1 Processor -- start 2024-09-19T06:09:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb61dd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa91af0 0x5565baab0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb61dd5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa92410 0x5565baab0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb61dd5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa9ccb0 0x5565bab164d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb61dd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581b52c0 con 0x5565baa91af0 2024-09-19T06:09:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb61dd5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581b4bf0 con 0x5565baa9ccb0 2024-09-19T06:09:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb61dd5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581b1e30 con 0x5565baa92410 2024-09-19T06:09:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb61dd5740 10 client.0 _ll_register_callbacks cb 0x7fcb44009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb5ccfb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa91af0 0x5565baab0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb5ccfb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa91af0 0x5565baab0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48342/0 (socket says 172.21.15.5:48342) 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/739429184 learned_addr learned my addr 172.21.15.5:0/739429184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb57fff640 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa92410 0x5565baab0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb5d4fc640 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa9ccb0 0x5565bab164d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/739429184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa92410 msgr2=0x5565baab0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa92410 0x5565baab0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/739429184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa9ccb0 msgr2=0x5565bab164d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa9ccb0 0x5565bab164d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.967+0000 7fcb5ccfb640 1 -- 172.21.15.5:0/739429184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565baa9aa70 con 0x5565baa91af0 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb5ccfb640 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa91af0 0x5565baab0150 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0cd9d0 tx=0x7fcb4c09bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04daf4617293473 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb37fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb37fff640 1 -- 172.21.15.5:0/739429184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c01c070 con 0x5565baa91af0 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb37fff640 1 -- 172.21.15.5:0/739429184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fcb4c0a4740 con 0x5565baa91af0 2024-09-19T06:09:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb61dd5740 1 -- 172.21.15.5:0/739429184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb4400fc90 con 0x5565baa91af0 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb37fff640 1 -- 172.21.15.5:0/739429184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcb4c0ab050 con 0x5565baa91af0 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb61dd5740 10 client.16932 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb61dd5740 1 -- 172.21.15.5:0/739429184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5565bab17a30 con 0x5565baa91af0 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb37fff640 1 -- 172.21.15.5:0/739429184 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb4400fc90 con 0x5565baa91af0 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb37fff640 1 -- 172.21.15.5:0/739429184 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fcb4c0baba0 con 0x5565baa91af0 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb61dd5740 10 client.16932 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb61dd5740 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb61dd5740 20 client.16932 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5565baaacb90 0x5565bab19fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb61dd5740 4 client.16932 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.968+0000 7fcb61dd5740 1 -- 172.21.15.5:0/739429184 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565ba9fdd70 con 0x5565baaacb90 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.969+0000 7fcb57fff640 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5565baaacb90 0x5565bab19fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.969+0000 7fcb57fff640 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5565baaacb90 0x5565bab19fd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50e83bb539b03ae8 server_cookie=8a471edf5d5da47d in_seq=0 out_seq=0 2024-09-19T06:09:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.969+0000 7fcb37fff640 10 client.16932 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb37fff640 1 -- 172.21.15.5:0/739429184 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565ba9fdd70 con 0x5565baaacb90 2024-09-19T06:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb37fff640 10 client.16932 handle_command_reply: tid=0 2024-09-19T06:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb61dd5740 1 client.16932 shutdown 2024-09-19T06:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb61dd5740 1 -- 172.21.15.5:0/739429184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa91af0 msgr2=0x5565baab0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa91af0 0x5565baab0150 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0cd9d0 tx=0x7fcb4c09bf60 comp rx=0 tx=0).stop 2024-09-19T06:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb61dd5740 1 -- 172.21.15.5:0/739429184 shutdown_connections 2024-09-19T06:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5565baaacb90 0x5565bab19fd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565baa9ccb0 0x5565bab164d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565baa92410 0x5565baab0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb37fff640 0 client.16932 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb61dd5740 1 --2- 172.21.15.5:0/739429184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565baa91af0 0x5565baab0150 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb61dd5740 1 -- 172.21.15.5:0/739429184 >> 172.21.15.5:0/739429184 conn(0x5565baa8e060 msgr2=0x5565baa93550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb61dd5740 1 -- 172.21.15.5:0/739429184 shutdown_connections 2024-09-19T06:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb61dd5740 1 -- 172.21.15.5:0/739429184 wait complete. 2024-09-19T06:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.991+0000 7fcb61dd5740 20 client.16932 trim_cache size 0 max 16384 2024-09-19T06:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:09:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.992+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/4052266168 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcb480a33b0 msgr2=0x7fcb480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.992+0000 7fcb5ef5f640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcb480a33b0 0x7fcb480a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcb50006010 tx=0x7fcb5003c040 comp rx=0 tx=0).stop 2024-09-19T06:09:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.992+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/4052266168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb58100a70 msgr2=0x7fcb581a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.992+0000 7fcb5ef5f640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb58100a70 0x7fcb581a4680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00e9f0 tx=0x7fcb4c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.992+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/4052266168 shutdown_connections 2024-09-19T06:09:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.992+0000 7fcb5ef5f640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcb480a33b0 0x7fcb480a5870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.992+0000 7fcb5ef5f640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb58105ea0 0x7fcb5819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.992+0000 7fcb5ef5f640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb58101390 0x7fcb581a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.992+0000 7fcb5ef5f640 1 --2- 172.21.15.5:0/4052266168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb58100a70 0x7fcb581a4680 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.992+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/4052266168 >> 172.21.15.5:0/4052266168 conn(0x7fcb580779a0 msgr2=0x7fcb5810f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.992+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/4052266168 shutdown_connections 2024-09-19T06:09:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:13.992+0000 7fcb5ef5f640 1 -- 172.21.15.5:0/4052266168 wait complete. 2024-09-19T06:09:14.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:13 smithi157 ceph-mon[35346]: pgmap v180: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 2.0 MiB/s wr, 212 op/s 2024-09-19T06:09:14.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:13 smithi005 ceph-mon[30955]: pgmap v180: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 2.0 MiB/s wr, 212 op/s 2024-09-19T06:09:14.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:13 smithi049 ceph-mon[34640]: pgmap v180: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 2.0 MiB/s wr, 212 op/s 2024-09-19T06:09:14.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 143967 2.74 MB/sec execute 160 sec latency 21.716 ms 2024-09-19T06:09:15.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 144496 2.74 MB/sec execute 161 sec latency 20.146 ms 2024-09-19T06:09:16.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:15 smithi157 ceph-mon[35346]: pgmap v181: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T06:09:16.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:15 smithi005 ceph-mon[30955]: pgmap v181: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T06:09:16.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:15 smithi049 ceph-mon[34640]: pgmap v181: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T06:09:16.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 145220 2.76 MB/sec execute 162 sec latency 11.069 ms 2024-09-19T06:09:17.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 145687 2.77 MB/sec execute 163 sec latency 15.441 ms 2024-09-19T06:09:18.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:17 smithi157 ceph-mon[35346]: pgmap v182: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-09-19T06:09:18.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:17 smithi005 ceph-mon[30955]: pgmap v182: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-09-19T06:09:18.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:17 smithi049 ceph-mon[34640]: pgmap v182: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-09-19T06:09:18.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 145969 2.76 MB/sec execute 164 sec latency 11.850 ms 2024-09-19T06:09:19.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 146428 2.74 MB/sec execute 165 sec latency 19.160 ms 2024-09-19T06:09:20.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:19 smithi157 ceph-mon[35346]: pgmap v183: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T06:09:20.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:19 smithi005 ceph-mon[30955]: pgmap v183: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T06:09:20.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:19 smithi049 ceph-mon[34640]: pgmap v183: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T06:09:20.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 146979 2.75 MB/sec execute 166 sec latency 18.957 ms 2024-09-19T06:09:21.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 147555 2.74 MB/sec execute 167 sec latency 20.121 ms 2024-09-19T06:09:22.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:21 smithi157 ceph-mon[35346]: pgmap v184: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T06:09:22.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:21 smithi005 ceph-mon[30955]: pgmap v184: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T06:09:22.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:21 smithi049 ceph-mon[34640]: pgmap v184: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T06:09:22.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 148068 2.74 MB/sec execute 168 sec latency 19.610 ms 2024-09-19T06:09:23.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 148779 2.76 MB/sec execute 169 sec latency 10.858 ms 2024-09-19T06:09:24.198 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:23 smithi049 ceph-mon[34640]: pgmap v185: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-09-19T06:09:24.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:23 smithi157 ceph-mon[35346]: pgmap v185: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-09-19T06:09:24.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:23 smithi005 ceph-mon[30955]: pgmap v185: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-09-19T06:09:24.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 149261 2.77 MB/sec execute 170 sec latency 15.063 ms 2024-09-19T06:09:25.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 149545 2.75 MB/sec execute 171 sec latency 12.843 ms 2024-09-19T06:09:26.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:25 smithi157 ceph-mon[35346]: pgmap v186: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:09:26.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:25 smithi005 ceph-mon[30955]: pgmap v186: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:09:26.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:25 smithi049 ceph-mon[34640]: pgmap v186: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-19T06:09:26.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 150043 2.75 MB/sec execute 172 sec latency 18.787 ms 2024-09-19T06:09:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.087+0000 7f572a2f8640 1 -- 172.21.15.5:0/2383251254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57241035a0 msgr2=0x7f572410ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.087+0000 7f572a2f8640 1 --2- 172.21.15.5:0/2383251254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57241035a0 0x7f572410ba80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5718009900 tx=0x7f571802a480 comp rx=0 tx=0).stop 2024-09-19T06:09:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.087+0000 7f572a2f8640 1 -- 172.21.15.5:0/2383251254 shutdown_connections 2024-09-19T06:09:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.087+0000 7f572a2f8640 1 --2- 172.21.15.5:0/2383251254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f572410c0a0 0x7f572410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.087+0000 7f572a2f8640 1 --2- 172.21.15.5:0/2383251254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57241035a0 0x7f572410ba80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.087+0000 7f572a2f8640 1 --2- 172.21.15.5:0/2383251254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5724102c80 0x7f5724103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.087+0000 7f572a2f8640 1 -- 172.21.15.5:0/2383251254 >> 172.21.15.5:0/2383251254 conn(0x7f57240779a0 msgr2=0x7f57240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.087+0000 7f572a2f8640 1 -- 172.21.15.5:0/2383251254 shutdown_connections 2024-09-19T06:09:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.087+0000 7f572a2f8640 1 -- 172.21.15.5:0/2383251254 wait complete. 2024-09-19T06:09:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.087+0000 7f572a2f8640 1 Processor -- start 2024-09-19T06:09:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.087+0000 7f572a2f8640 1 -- start start 2024-09-19T06:09:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f572a2f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5724102c80 0x7f57241008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f572a2f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57241035a0 0x7f5724100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f572a2f8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f572410c0a0 0x7f5724101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f572a2f8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5724111d50 con 0x7f572410c0a0 2024-09-19T06:09:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f572a2f8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5724111bd0 con 0x7f5724102c80 2024-09-19T06:09:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f572a2f8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5724111ed0 con 0x7f57241035a0 2024-09-19T06:09:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f5723fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5724102c80 0x7f57241008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f5723fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5724102c80 0x7f57241008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57198/0 (socket says 172.21.15.5:57198) 2024-09-19T06:09:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f5723fff640 1 -- 172.21.15.5:0/250729545 learned_addr learned my addr 172.21.15.5:0/250729545 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f57237fe640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57241035a0 0x7f5724100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f5728895640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f572410c0a0 0x7f5724101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f5723fff640 1 -- 172.21.15.5:0/250729545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57241035a0 msgr2=0x7f5724100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f5723fff640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57241035a0 0x7f5724100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f5723fff640 1 -- 172.21.15.5:0/250729545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f572410c0a0 msgr2=0x7f5724101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f5723fff640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f572410c0a0 0x7f5724101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.088+0000 7f5723fff640 1 -- 172.21.15.5:0/250729545 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57241a6840 con 0x7f5724102c80 2024-09-19T06:09:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.089+0000 7f5723fff640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5724102c80 0x7f57241008d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f571400e9f0 tx=0x7f571400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.089+0000 7f57217fa640 1 -- 172.21.15.5:0/250729545 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571400cd80 con 0x7f5724102c80 2024-09-19T06:09:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.089+0000 7f572a2f8640 1 -- 172.21.15.5:0/250729545 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57241a6b30 con 0x7f5724102c80 2024-09-19T06:09:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.089+0000 7f57217fa640 1 -- 172.21.15.5:0/250729545 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57140054d0 con 0x7f5724102c80 2024-09-19T06:09:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.089+0000 7f57217fa640 1 -- 172.21.15.5:0/250729545 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714010490 con 0x7f5724102c80 2024-09-19T06:09:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.089+0000 7f572a2f8640 1 -- 172.21.15.5:0/250729545 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57241a7050 con 0x7f5724102c80 2024-09-19T06:09:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.091+0000 7f57217fa640 1 -- 172.21.15.5:0/250729545 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5714005020 con 0x7f5724102c80 2024-09-19T06:09:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f572d096740 1 Processor -- start 2024-09-19T06:09:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f572d096740 1 -- start start 2024-09-19T06:09:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f572d096740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415495af0 0x557415495ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f572d096740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415496410 0x5574154a0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f572d096740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 0x5574154a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f57217fa640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f570c0a33b0 0x7f570c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f572d096740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574153fa2f0 con 0x5574154a0cb0 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f572d096740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5574153a2270 con 0x557415496410 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f572d096740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5574153f62b0 con 0x557415495af0 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f5723fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415496410 0x5574154a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f57237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 0x5574154a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f5728895640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415495af0 0x557415495ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f57237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 0x5574154a30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48812/0 (socket says 172.21.15.5:48812) 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f57237fe640 1 -- 172.21.15.5:0/2704663920 learned_addr learned my addr 172.21.15.5:0/2704663920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.092+0000 7f5728895640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f570c0a33b0 0x7f570c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.093+0000 7f57217fa640 1 -- 172.21.15.5:0/250729545 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f57140d0070 con 0x7f5724102c80 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.093+0000 7f5723fff640 1 -- 172.21.15.5:0/2704663920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415495af0 msgr2=0x557415495ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.093+0000 7f5723fff640 1 --2- 172.21.15.5:0/2704663920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415495af0 0x557415495ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.093+0000 7f5723fff640 1 -- 172.21.15.5:0/2704663920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 msgr2=0x5574154a30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.093+0000 7f5723fff640 1 --2- 172.21.15.5:0/2704663920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 0x5574154a30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.093+0000 7f5723fff640 1 -- 172.21.15.5:0/2704663920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574153a1340 con 0x557415496410 2024-09-19T06:09:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.093+0000 7f5723fff640 1 --2- 172.21.15.5:0/2704663920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415496410 0x5574154a0770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57140cd7d0 tx=0x7f571409c130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9de48c4d97dee60d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.093+0000 7f5728895640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f570c0a33b0 0x7f570c0a5870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f571c005fd0 tx=0x7f571c009f50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.093+0000 7f570a7fc640 1 -- 172.21.15.5:0/2704663920 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714005310 con 0x557415496410 2024-09-19T06:09:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.093+0000 7f570a7fc640 1 -- 172.21.15.5:0/2704663920 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5714005680 con 0x557415496410 2024-09-19T06:09:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.093+0000 7f570a7fc640 1 -- 172.21.15.5:0/2704663920 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57140989d0 con 0x557415496410 2024-09-19T06:09:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.094+0000 7f572d096740 1 -- 172.21.15.5:0/2704663920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415496410 msgr2=0x5574154a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.094+0000 7f572d096740 1 --2- 172.21.15.5:0/2704663920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415496410 0x5574154a0770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57140cd7d0 tx=0x7f571409c130 comp rx=0 tx=0).stop 2024-09-19T06:09:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.094+0000 7f572d096740 1 -- 172.21.15.5:0/2704663920 shutdown_connections 2024-09-19T06:09:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.094+0000 7f572d096740 1 --2- 172.21.15.5:0/2704663920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 0x5574154a30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.094+0000 7f572d096740 1 --2- 172.21.15.5:0/2704663920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415496410 0x5574154a0770 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.094+0000 7f572d096740 1 --2- 172.21.15.5:0/2704663920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415495af0 0x557415495ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.094+0000 7f572d096740 1 -- 172.21.15.5:0/2704663920 >> 172.21.15.5:0/2704663920 conn(0x557415492060 msgr2=0x557415492440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.094+0000 7f572d096740 1 -- 172.21.15.5:0/2704663920 shutdown_connections 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.094+0000 7f572d096740 1 -- 172.21.15.5:0/2704663920 wait complete. 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.094+0000 7f572d096740 1 -- start start 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f572d096740 1 Processor -- start 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f572d096740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415495af0 0x55741549dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f572d096740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415496410 0x55741549e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f572d096740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574154a0cb0 0x55741549e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f572d096740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574154a3880 con 0x557415496410 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f572d096740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557415383290 con 0x5574154a0cb0 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f572d096740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5574154a3a00 con 0x557415495af0 2024-09-19T06:09:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f572d096740 10 client.0 _ll_register_callbacks cb 0x5574153b4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f57237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415496410 0x55741549e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f5728895640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574154a0cb0 0x55741549e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f57237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415496410 0x55741549e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48814/0 (socket says 172.21.15.5:48814) 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f57237fe640 1 -- 172.21.15.5:0/787700749 learned_addr learned my addr 172.21.15.5:0/787700749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f5723fff640 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415495af0 0x55741549dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f57237fe640 1 -- 172.21.15.5:0/787700749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415495af0 msgr2=0x55741549dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f57237fe640 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415495af0 0x55741549dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f57237fe640 1 -- 172.21.15.5:0/787700749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574154a0cb0 msgr2=0x55741549e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f57237fe640 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574154a0cb0 0x55741549e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f57237fe640 1 -- 172.21.15.5:0/787700749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55741510d210 con 0x557415496410 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f5728895640 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574154a0cb0 0x55741549e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f57237fe640 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415496410 0x55741549e130 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5718033000 tx=0x7f5718032e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78e1c67e7a0cb3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.095+0000 7f57037fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:09:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f57037fe640 1 -- 172.21.15.5:0/787700749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718035040 con 0x557415496410 2024-09-19T06:09:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f57037fe640 1 -- 172.21.15.5:0/787700749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f57180316a0 con 0x557415496410 2024-09-19T06:09:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f572d096740 1 -- 172.21.15.5:0/787700749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5714005680 con 0x557415496410 2024-09-19T06:09:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f57037fe640 1 -- 172.21.15.5:0/787700749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f57180318a0 con 0x557415496410 2024-09-19T06:09:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f572d096740 10 client.16944 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f572d096740 1 -- 172.21.15.5:0/787700749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5574151d9620 con 0x557415496410 2024-09-19T06:09:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f57037fe640 1 -- 172.21.15.5:0/787700749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5714005680 con 0x557415496410 2024-09-19T06:09:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f57037fe640 1 -- 172.21.15.5:0/787700749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5718004750 con 0x557415496410 2024-09-19T06:09:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f572d096740 10 client.16944 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f572d096740 10 client.16944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f572d096740 20 client.16944 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f572d096740 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5574154b0ca0 0x5574154b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f572d096740 4 client.16944 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.096+0000 7f572d096740 1 -- 172.21.15.5:0/787700749 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557415407a20 con 0x5574154b0ca0 2024-09-19T06:09:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.097+0000 7f5723fff640 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5574154b0ca0 0x5574154b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.097+0000 7f5723fff640 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5574154b0ca0 0x5574154b3090 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908679f017476f2a server_cookie=9bca75f922e731f8 in_seq=0 out_seq=0 2024-09-19T06:09:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.097+0000 7f57037fe640 10 client.16944 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f57037fe640 1 -- 172.21.15.5:0/787700749 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557415407a20 con 0x5574154b0ca0 2024-09-19T06:09:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f57037fe640 10 client.16944 handle_command_reply: tid=0 2024-09-19T06:09:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f572d096740 1 client.16944 shutdown 2024-09-19T06:09:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f572d096740 1 -- 172.21.15.5:0/787700749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415496410 msgr2=0x55741549e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f572d096740 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415496410 0x55741549e130 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5718033000 tx=0x7f5718032e30 comp rx=0 tx=0).stop 2024-09-19T06:09:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f572d096740 1 -- 172.21.15.5:0/787700749 shutdown_connections 2024-09-19T06:09:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f572d096740 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5574154b0ca0 0x5574154b3090 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f572d096740 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574154a0cb0 0x55741549e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f572d096740 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415496410 0x55741549e130 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f57037fe640 0 client.16944 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f572d096740 1 --2- 172.21.15.5:0/787700749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415495af0 0x55741549dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f572d096740 1 -- 172.21.15.5:0/787700749 >> 172.21.15.5:0/787700749 conn(0x557415492060 msgr2=0x557415497d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f572d096740 1 -- 172.21.15.5:0/787700749 shutdown_connections 2024-09-19T06:09:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f572d096740 1 -- 172.21.15.5:0/787700749 wait complete. 2024-09-19T06:09:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.098+0000 7f572d096740 20 client.16944 trim_cache size 0 max 16384 2024-09-19T06:09:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.113+0000 7f572d096740 1 Processor -- start 2024-09-19T06:09:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.113+0000 7f572d096740 1 -- start start 2024-09-19T06:09:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.113+0000 7f572d096740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415495af0 0x55741549e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.113+0000 7f572d096740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415496410 0x5574154b8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.113+0000 7f572d096740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 0x5574154b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.113+0000 7f572d096740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57241af870 con 0x5574154a0cb0 2024-09-19T06:09:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.113+0000 7f572d096740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f57241b13b0 con 0x557415495af0 2024-09-19T06:09:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.113+0000 7f572d096740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57241aff40 con 0x557415496410 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.113+0000 7f5728895640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 0x5574154b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.113+0000 7f5728895640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 0x5574154b8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48822/0 (socket says 172.21.15.5:48822) 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.113+0000 7f5728895640 1 -- 172.21.15.5:0/2344594535 learned_addr learned my addr 172.21.15.5:0/2344594535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.113+0000 7f5723fff640 1 --2- 172.21.15.5:0/2344594535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415495af0 0x55741549e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.114+0000 7f57237fe640 1 --2- 172.21.15.5:0/2344594535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415496410 0x5574154b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.114+0000 7f5723fff640 1 -- 172.21.15.5:0/2344594535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415496410 msgr2=0x5574154b8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.114+0000 7f5723fff640 1 --2- 172.21.15.5:0/2344594535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415496410 0x5574154b8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.114+0000 7f5723fff640 1 -- 172.21.15.5:0/2344594535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 msgr2=0x5574154b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.114+0000 7f5723fff640 1 --2- 172.21.15.5:0/2344594535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 0x5574154b8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.114+0000 7f5723fff640 1 -- 172.21.15.5:0/2344594535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557415496240 con 0x557415495af0 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.114+0000 7f5728895640 1 --2- 172.21.15.5:0/2344594535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 0x5574154b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.114+0000 7f5723fff640 1 --2- 172.21.15.5:0/2344594535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415495af0 0x55741549e4a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f57140cd800 tx=0x7f57140bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a42c93ba448f078 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.114+0000 7f5709ffb640 1 -- 172.21.15.5:0/2344594535 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714005680 con 0x557415495af0 2024-09-19T06:09:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.114+0000 7f5709ffb640 1 -- 172.21.15.5:0/2344594535 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f571409ca60 con 0x557415495af0 2024-09-19T06:09:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.114+0000 7f5709ffb640 1 -- 172.21.15.5:0/2344594535 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57140cda50 con 0x557415495af0 2024-09-19T06:09:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.115+0000 7f572d096740 1 -- 172.21.15.5:0/2344594535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415495af0 msgr2=0x55741549e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.115+0000 7f572d096740 1 --2- 172.21.15.5:0/2344594535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415495af0 0x55741549e4a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f57140cd800 tx=0x7f57140bc040 comp rx=0 tx=0).stop 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.115+0000 7f572d096740 1 -- 172.21.15.5:0/2344594535 shutdown_connections 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.115+0000 7f572d096740 1 --2- 172.21.15.5:0/2344594535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574154a0cb0 0x5574154b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.115+0000 7f572d096740 1 --2- 172.21.15.5:0/2344594535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557415496410 0x5574154b8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.115+0000 7f572d096740 1 --2- 172.21.15.5:0/2344594535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415495af0 0x55741549e4a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.115+0000 7f572d096740 1 -- 172.21.15.5:0/2344594535 >> 172.21.15.5:0/2344594535 conn(0x557415492060 msgr2=0x55741549fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.115+0000 7f572d096740 1 -- 172.21.15.5:0/2344594535 shutdown_connections 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.115+0000 7f572d096740 1 -- 172.21.15.5:0/2344594535 wait complete. 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.115+0000 7f572d096740 1 -- start start 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.115+0000 7f572d096740 1 Processor -- start 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f572d096740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415495af0 0x5574154a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f572d096740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415496410 0x5574154a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f572d096740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574154a0cb0 0x5574154a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f572d096740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57241b0ce0 con 0x557415495af0 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f572d096740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f57241b0610 con 0x557415496410 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f572d096740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57241ad850 con 0x5574154a0cb0 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f572d096740 10 client.0 _ll_register_callbacks cb 0x7f5718009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f5723fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415495af0 0x5574154a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f5723fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415495af0 0x5574154a93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48828/0 (socket says 172.21.15.5:48828) 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f5723fff640 1 -- 172.21.15.5:0/1029365979 learned_addr learned my addr 172.21.15.5:0/1029365979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f57237fe640 1 --2- 172.21.15.5:0/1029365979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415496410 0x5574154a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f5723fff640 1 -- 172.21.15.5:0/1029365979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574154a0cb0 msgr2=0x5574154a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f5723fff640 1 --2- 172.21.15.5:0/1029365979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574154a0cb0 0x5574154a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f5723fff640 1 -- 172.21.15.5:0/1029365979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415496410 msgr2=0x5574154a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f5723fff640 1 --2- 172.21.15.5:0/1029365979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415496410 0x5574154a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f5723fff640 1 -- 172.21.15.5:0/1029365979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574154a5420 con 0x557415495af0 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f5723fff640 1 --2- 172.21.15.5:0/1029365979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415495af0 0x5574154a93e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f57140b0010 tx=0x7f5714005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c414ff8b6e8203 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f57037fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f57037fe640 1 -- 172.21.15.5:0/1029365979 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714005680 con 0x557415495af0 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f57037fe640 1 -- 172.21.15.5:0/1029365979 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f57140b52a0 con 0x557415495af0 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.116+0000 7f572d096740 1 -- 172.21.15.5:0/1029365979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f571409ca60 con 0x557415495af0 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.117+0000 7f57037fe640 1 -- 172.21.15.5:0/1029365979 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f57140a0710 con 0x557415495af0 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.117+0000 7f572d096740 10 client.16950 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.117+0000 7f572d096740 1 -- 172.21.15.5:0/1029365979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55741551b940 con 0x557415495af0 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.117+0000 7f57037fe640 1 -- 172.21.15.5:0/1029365979 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f571409ca60 con 0x557415495af0 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.117+0000 7f57037fe640 1 -- 172.21.15.5:0/1029365979 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f57140a4b40 con 0x557415495af0 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.117+0000 7f572d096740 10 client.16950 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.117+0000 7f572d096740 10 client.16950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.117+0000 7f572d096740 20 client.16950 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.117+0000 7f572d096740 1 --2- 172.21.15.5:0/1029365979 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55741551bbc0 0x55741551dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.117+0000 7f572d096740 4 client.16950 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:09:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.117+0000 7f572d096740 1 -- 172.21.15.5:0/1029365979 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557415401d70 con 0x55741551bbc0 2024-09-19T06:09:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.117+0000 7f57237fe640 1 --2- 172.21.15.5:0/1029365979 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55741551bbc0 0x55741551dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.118+0000 7f57037fe640 10 client.16950 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.118+0000 7f57237fe640 1 --2- 172.21.15.5:0/1029365979 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55741551bbc0 0x55741551dfb0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f99810a0f7bfed server_cookie=437665c6164f0908 in_seq=0 out_seq=0 2024-09-19T06:09:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.139+0000 7f57037fe640 1 -- 172.21.15.5:0/1029365979 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557415401d70 con 0x55741551bbc0 2024-09-19T06:09:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.139+0000 7f57037fe640 10 client.16950 handle_command_reply: tid=0 2024-09-19T06:09:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.139+0000 7f572d096740 1 client.16950 shutdown 2024-09-19T06:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.139+0000 7f572d096740 1 -- 172.21.15.5:0/1029365979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415495af0 msgr2=0x5574154a93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.139+0000 7f572d096740 1 --2- 172.21.15.5:0/1029365979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415495af0 0x5574154a93e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f57140b0010 tx=0x7f5714005b30 comp rx=0 tx=0).stop 2024-09-19T06:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.140+0000 7f572d096740 1 -- 172.21.15.5:0/1029365979 shutdown_connections 2024-09-19T06:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.140+0000 7f572d096740 1 --2- 172.21.15.5:0/1029365979 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55741551bbc0 0x55741551dfb0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.140+0000 7f572d096740 1 --2- 172.21.15.5:0/1029365979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574154a0cb0 0x5574154a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.140+0000 7f572d096740 1 --2- 172.21.15.5:0/1029365979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557415496410 0x5574154a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.140+0000 7f572d096740 1 --2- 172.21.15.5:0/1029365979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557415495af0 0x5574154a93e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.140+0000 7f572d096740 1 -- 172.21.15.5:0/1029365979 >> 172.21.15.5:0/1029365979 conn(0x557415492060 msgr2=0x55741549fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.140+0000 7f57037fe640 0 client.16950 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.140+0000 7f572d096740 1 -- 172.21.15.5:0/1029365979 shutdown_connections 2024-09-19T06:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.140+0000 7f572d096740 1 -- 172.21.15.5:0/1029365979 wait complete. 2024-09-19T06:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.140+0000 7f572d096740 20 client.16950 trim_cache size 0 max 16384 2024-09-19T06:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.141+0000 7f572a2f8640 1 -- 172.21.15.5:0/250729545 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f570c0a33b0 msgr2=0x7f570c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.141+0000 7f572a2f8640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f570c0a33b0 0x7f570c0a5870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f571c005fd0 tx=0x7f571c009f50 comp rx=0 tx=0).stop 2024-09-19T06:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.141+0000 7f572a2f8640 1 -- 172.21.15.5:0/250729545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5724102c80 msgr2=0x7f57241008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.141+0000 7f572a2f8640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5724102c80 0x7f57241008d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f571400e9f0 tx=0x7f571400edf0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.141+0000 7f572a2f8640 1 -- 172.21.15.5:0/250729545 shutdown_connections 2024-09-19T06:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.141+0000 7f572a2f8640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f572410c0a0 0x7f5724101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.141+0000 7f572a2f8640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f570c0a33b0 0x7f570c0a5870 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.141+0000 7f572a2f8640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57241035a0 0x7f5724100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.141+0000 7f572a2f8640 1 --2- 172.21.15.5:0/250729545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5724102c80 0x7f57241008d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.141+0000 7f572a2f8640 1 -- 172.21.15.5:0/250729545 >> 172.21.15.5:0/250729545 conn(0x7f57240779a0 msgr2=0x7f57240fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.141+0000 7f572a2f8640 1 -- 172.21.15.5:0/250729545 shutdown_connections 2024-09-19T06:09:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:27.141+0000 7f572a2f8640 1 -- 172.21.15.5:0/250729545 wait complete. 2024-09-19T06:09:27.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 150561 2.75 MB/sec execute 173 sec latency 22.349 ms 2024-09-19T06:09:28.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:27 smithi157 ceph-mon[35346]: pgmap v187: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-09-19T06:09:28.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:27 smithi005 ceph-mon[30955]: pgmap v187: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-09-19T06:09:28.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:27 smithi049 ceph-mon[34640]: pgmap v187: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-09-19T06:09:28.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 151161 2.74 MB/sec execute 174 sec latency 21.753 ms 2024-09-19T06:09:29.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 151714 2.75 MB/sec execute 175 sec latency 15.941 ms 2024-09-19T06:09:30.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:29 smithi157 ceph-mon[35346]: pgmap v188: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-19T06:09:30.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:29 smithi005 ceph-mon[30955]: pgmap v188: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-19T06:09:30.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:29 smithi049 ceph-mon[34640]: pgmap v188: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-19T06:09:30.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 152326 2.76 MB/sec execute 176 sec latency 15.231 ms 2024-09-19T06:09:31.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 152811 2.77 MB/sec execute 177 sec latency 12.146 ms 2024-09-19T06:09:32.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:31 smithi157 ceph-mon[35346]: pgmap v189: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-19T06:09:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:31 smithi005 ceph-mon[30955]: pgmap v189: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-19T06:09:32.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:31 smithi049 ceph-mon[34640]: pgmap v189: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-19T06:09:32.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 153098 2.75 MB/sec execute 178 sec latency 11.997 ms 2024-09-19T06:09:33.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:09:33.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 153597 2.75 MB/sec execute 179 sec latency 15.773 ms 2024-09-19T06:09:34.045 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:09:34.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:33 smithi005 ceph-mon[30955]: pgmap v190: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-19T06:09:34.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:33 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:09:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:33 smithi157 ceph-mon[35346]: pgmap v190: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-19T06:09:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:33 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:09:34.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:33 smithi049 ceph-mon[34640]: pgmap v190: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-19T06:09:34.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:33 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:09:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.711+0000 7fba44bc7640 1 -- 172.21.15.5:0/1546946092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40100a90 msgr2=0x7fba40100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.711+0000 7fba44bc7640 1 --2- 172.21.15.5:0/1546946092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40100a90 0x7fba40100ef0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fba34009930 tx=0x7fba3402a440 comp rx=0 tx=0).stop 2024-09-19T06:09:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.711+0000 7fba44bc7640 1 -- 172.21.15.5:0/1546946092 shutdown_connections 2024-09-19T06:09:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.711+0000 7fba44bc7640 1 --2- 172.21.15.5:0/1546946092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba40101430 0x7fba4010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.711+0000 7fba44bc7640 1 --2- 172.21.15.5:0/1546946092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40100a90 0x7fba40100ef0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.711+0000 7fba44bc7640 1 --2- 172.21.15.5:0/1546946092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba40106a90 0x7fba40106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.711+0000 7fba44bc7640 1 -- 172.21.15.5:0/1546946092 >> 172.21.15.5:0/1546946092 conn(0x7fba400fc7d0 msgr2=0x7fba400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.711+0000 7fba44bc7640 1 -- 172.21.15.5:0/1546946092 shutdown_connections 2024-09-19T06:09:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.711+0000 7fba44bc7640 1 -- 172.21.15.5:0/1546946092 wait complete. 2024-09-19T06:09:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.712+0000 7fba44bc7640 1 Processor -- start 2024-09-19T06:09:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.712+0000 7fba44bc7640 1 -- start start 2024-09-19T06:09:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.712+0000 7fba44bc7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba40100a90 0x7fba4019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba44bc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101430 0x7fba4019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba44bc7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba40106a90 0x7fba4019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba44bc7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba40110d50 con 0x7fba40101430 2024-09-19T06:09:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba44bc7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fba40110bd0 con 0x7fba40106a90 2024-09-19T06:09:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba44bc7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba40110ed0 con 0x7fba40100a90 2024-09-19T06:09:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba3dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101430 0x7fba4019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba3e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba40100a90 0x7fba4019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba3dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101430 0x7fba4019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42354/0 (socket says 172.21.15.5:42354) 2024-09-19T06:09:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba3dd9b640 1 -- 172.21.15.5:0/3715953274 learned_addr learned my addr 172.21.15.5:0/3715953274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba3e59c640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba40100a90 0x7fba4019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50700/0 (socket says 172.21.15.5:50700) 2024-09-19T06:09:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba3ed9d640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba40106a90 0x7fba4019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba3dd9b640 1 -- 172.21.15.5:0/3715953274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba40100a90 msgr2=0x7fba4019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba3dd9b640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba40100a90 0x7fba4019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba3dd9b640 1 -- 172.21.15.5:0/3715953274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba40106a90 msgr2=0x7fba4019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba3dd9b640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba40106a90 0x7fba4019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba3dd9b640 1 -- 172.21.15.5:0/3715953274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4019f560 con 0x7fba40101430 2024-09-19T06:09:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.713+0000 7fba3ed9d640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba40106a90 0x7fba4019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.714+0000 7fba3dd9b640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101430 0x7fba4019aaf0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fba34005ab0 tx=0x7fba34003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.714+0000 7fba2f7fe640 1 -- 172.21.15.5:0/3715953274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3403f070 con 0x7fba40101430 2024-09-19T06:09:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.714+0000 7fba44bc7640 1 -- 172.21.15.5:0/3715953274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4019f7f0 con 0x7fba40101430 2024-09-19T06:09:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.714+0000 7fba2f7fe640 1 -- 172.21.15.5:0/3715953274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba34031b50 con 0x7fba40101430 2024-09-19T06:09:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.714+0000 7fba2f7fe640 1 -- 172.21.15.5:0/3715953274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34035030 con 0x7fba40101430 2024-09-19T06:09:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.714+0000 7fba44bc7640 1 -- 172.21.15.5:0/3715953274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba401a7130 con 0x7fba40101430 2024-09-19T06:09:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.717+0000 7fba2f7fe640 1 -- 172.21.15.5:0/3715953274 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fba34042070 con 0x7fba40101430 2024-09-19T06:09:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.717+0000 7fba46fa0740 1 Processor -- start 2024-09-19T06:09:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.717+0000 7fba46fa0740 1 -- start start 2024-09-19T06:09:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.717+0000 7fba46fa0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fd1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.717+0000 7fba46fa0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd1dc0 0x55dda3fdc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba46fa0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3fdea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba46fa0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dda3f372e0 con 0x55dda3fdc680 2024-09-19T06:09:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba46fa0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dda3edece0 con 0x55dda3fd1dc0 2024-09-19T06:09:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba46fa0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dda3f332a0 con 0x55dda3fd14a0 2024-09-19T06:09:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba3e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd1dc0 0x55dda3fdc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba3e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd1dc0 0x55dda3fdc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41626/0 (socket says 172.21.15.5:41626) 2024-09-19T06:09:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba3e59c640 1 -- 172.21.15.5:0/819422489 learned_addr learned my addr 172.21.15.5:0/819422489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba3dd9b640 1 --2- 172.21.15.5:0/819422489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3fdea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba3e59c640 1 -- 172.21.15.5:0/819422489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 msgr2=0x55dda3fd1880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:09:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba3e59c640 1 --2- 172.21.15.5:0/819422489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fd1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba3e59c640 1 -- 172.21.15.5:0/819422489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 msgr2=0x55dda3fdea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba3e59c640 1 --2- 172.21.15.5:0/819422489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3fdea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba3e59c640 1 -- 172.21.15.5:0/819422489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dda3ec3440 con 0x55dda3fd1dc0 2024-09-19T06:09:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.718+0000 7fba3dd9b640 1 --2- 172.21.15.5:0/819422489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3fdea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:09:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.719+0000 7fba2f7fe640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fba280a3430 0x7fba280a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.719+0000 7fba3ed9d640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fba280a3430 0x7fba280a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.719+0000 7fba3e59c640 1 --2- 172.21.15.5:0/819422489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd1dc0 0x55dda3fdc140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fba30009c70 tx=0x7fba30011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad96e3a3e41d78a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.719+0000 7fba2f7fe640 1 -- 172.21.15.5:0/3715953274 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fba340f0310 con 0x7fba40101430 2024-09-19T06:09:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.719+0000 7fba2cff9640 1 -- 172.21.15.5:0/819422489 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30004230 con 0x55dda3fd1dc0 2024-09-19T06:09:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.719+0000 7fba2cff9640 1 -- 172.21.15.5:0/819422489 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba300043d0 con 0x55dda3fd1dc0 2024-09-19T06:09:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.719+0000 7fba2cff9640 1 -- 172.21.15.5:0/819422489 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba300196b0 con 0x55dda3fd1dc0 2024-09-19T06:09:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.720+0000 7fba46fa0740 1 -- 172.21.15.5:0/819422489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd1dc0 msgr2=0x55dda3fdc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.720+0000 7fba3ed9d640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fba280a3430 0x7fba280a58f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba38005fd0 tx=0x7fba38009ed0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.720+0000 7fba46fa0740 1 --2- 172.21.15.5:0/819422489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd1dc0 0x55dda3fdc140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fba30009c70 tx=0x7fba30011810 comp rx=0 tx=0).stop 2024-09-19T06:09:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.720+0000 7fba46fa0740 1 -- 172.21.15.5:0/819422489 shutdown_connections 2024-09-19T06:09:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.720+0000 7fba46fa0740 1 --2- 172.21.15.5:0/819422489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3fdea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.720+0000 7fba46fa0740 1 --2- 172.21.15.5:0/819422489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd1dc0 0x55dda3fdc140 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.720+0000 7fba46fa0740 1 --2- 172.21.15.5:0/819422489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fd1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.720+0000 7fba46fa0740 1 -- 172.21.15.5:0/819422489 >> 172.21.15.5:0/819422489 conn(0x55dda3fcda30 msgr2=0x55dda3fcde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.720+0000 7fba46fa0740 1 -- 172.21.15.5:0/819422489 shutdown_connections 2024-09-19T06:09:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.720+0000 7fba46fa0740 1 -- 172.21.15.5:0/819422489 wait complete. 2024-09-19T06:09:34.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.721+0000 7fba46fa0740 1 -- start start 2024-09-19T06:09:34.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.721+0000 7fba46fa0740 1 Processor -- start 2024-09-19T06:09:34.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.721+0000 7fba46fa0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fd94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.721+0000 7fba46fa0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fdc680 0x55dda3fd9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.721+0000 7fba46fa0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fd9f40 0x55dda3ff0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.721+0000 7fba46fa0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dda3fdf250 con 0x55dda3fd9f40 2024-09-19T06:09:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.721+0000 7fba46fa0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dda3ef2050 con 0x55dda3fdc680 2024-09-19T06:09:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.721+0000 7fba46fa0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dda3fdf3d0 con 0x55dda3fd14a0 2024-09-19T06:09:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.722+0000 7fba46fa0740 10 client.0 _ll_register_callbacks cb 0x55dda3f32160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.722+0000 7fba3ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fd9f40 0x55dda3ff0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.722+0000 7fba3dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fdc680 0x55dda3fd9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.722+0000 7fba3ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fd9f40 0x55dda3ff0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42372/0 (socket says 172.21.15.5:42372) 2024-09-19T06:09:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.722+0000 7fba3e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fd94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.722+0000 7fba3ed9d640 1 -- 172.21.15.5:0/3309981395 learned_addr learned my addr 172.21.15.5:0/3309981395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.722+0000 7fba3dd9b640 1 -- 172.21.15.5:0/3309981395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 msgr2=0x55dda3fd94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:34.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.722+0000 7fba3dd9b640 1 --2- 172.21.15.5:0/3309981395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fd94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.722+0000 7fba3dd9b640 1 -- 172.21.15.5:0/3309981395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fd9f40 msgr2=0x55dda3ff0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:34.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.722+0000 7fba3dd9b640 1 --2- 172.21.15.5:0/3309981395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fd9f40 0x55dda3ff0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.722+0000 7fba3dd9b640 1 -- 172.21.15.5:0/3309981395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dda3ca83c0 con 0x55dda3fdc680 2024-09-19T06:09:34.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.723+0000 7fba3dd9b640 1 --2- 172.21.15.5:0/3309981395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fdc680 0x55dda3fd9a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba340eda70 tx=0x7fba34009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d52da247fa0fea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:34.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.723+0000 7fba25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:34.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.723+0000 7fba25ffb640 1 -- 172.21.15.5:0/3309981395 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34049070 con 0x55dda3fdc680 2024-09-19T06:09:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.723+0000 7fba25ffb640 1 -- 172.21.15.5:0/3309981395 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fba340b8d20 con 0x55dda3fdc680 2024-09-19T06:09:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.723+0000 7fba46fa0740 1 -- 172.21.15.5:0/3309981395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba300043d0 con 0x55dda3fdc680 2024-09-19T06:09:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.724+0000 7fba25ffb640 1 -- 172.21.15.5:0/3309981395 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fba340bc6b0 con 0x55dda3fdc680 2024-09-19T06:09:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.724+0000 7fba46fa0740 10 client.26546 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.724+0000 7fba46fa0740 1 -- 172.21.15.5:0/3309981395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dda3ff3a30 con 0x55dda3fdc680 2024-09-19T06:09:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.724+0000 7fba25ffb640 1 -- 172.21.15.5:0/3309981395 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba300043d0 con 0x55dda3fdc680 2024-09-19T06:09:34.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.724+0000 7fba25ffb640 1 -- 172.21.15.5:0/3309981395 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fba340c4de0 con 0x55dda3fdc680 2024-09-19T06:09:34.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.724+0000 7fba46fa0740 10 client.26546 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:34.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.724+0000 7fba46fa0740 10 client.26546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:34.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.724+0000 7fba46fa0740 20 client.26546 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:34.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.724+0000 7fba46fa0740 1 --2- 172.21.15.5:0/3309981395 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dda3ff4e20 0x55dda3ff72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.724+0000 7fba46fa0740 4 client.26546 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:34.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.724+0000 7fba46fa0740 1 -- 172.21.15.5:0/3309981395 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dda3f44850 con 0x55dda3ff4e20 2024-09-19T06:09:34.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.725+0000 7fba3e59c640 1 --2- 172.21.15.5:0/3309981395 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dda3ff4e20 0x55dda3ff72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.725+0000 7fba25ffb640 10 client.26546 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:34.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.725+0000 7fba3e59c640 1 --2- 172.21.15.5:0/3309981395 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dda3ff4e20 0x55dda3ff72b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d6c7236dfe1862 server_cookie=241ac1ac20d7a80d in_seq=0 out_seq=0 2024-09-19T06:09:34.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.726+0000 7fba25ffb640 1 -- 172.21.15.5:0/3309981395 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dda3f44850 con 0x55dda3ff4e20 2024-09-19T06:09:34.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.726+0000 7fba25ffb640 10 client.26546 handle_command_reply: tid=0 2024-09-19T06:09:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.726+0000 7fba46fa0740 1 client.26546 shutdown 2024-09-19T06:09:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.726+0000 7fba46fa0740 1 -- 172.21.15.5:0/3309981395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fdc680 msgr2=0x55dda3fd9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.726+0000 7fba46fa0740 1 --2- 172.21.15.5:0/3309981395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fdc680 0x55dda3fd9a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba340eda70 tx=0x7fba34009260 comp rx=0 tx=0).stop 2024-09-19T06:09:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.726+0000 7fba46fa0740 1 -- 172.21.15.5:0/3309981395 shutdown_connections 2024-09-19T06:09:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.726+0000 7fba46fa0740 1 --2- 172.21.15.5:0/3309981395 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dda3ff4e20 0x55dda3ff72b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.727+0000 7fba25ffb640 0 client.26546 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.727+0000 7fba46fa0740 1 --2- 172.21.15.5:0/3309981395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fd9f40 0x55dda3ff0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.727+0000 7fba46fa0740 1 --2- 172.21.15.5:0/3309981395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fdc680 0x55dda3fd9a00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.727+0000 7fba46fa0740 1 --2- 172.21.15.5:0/3309981395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fd94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.727+0000 7fba46fa0740 1 -- 172.21.15.5:0/3309981395 >> 172.21.15.5:0/3309981395 conn(0x55dda3fcda30 msgr2=0x55dda3fd3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.727+0000 7fba46fa0740 1 -- 172.21.15.5:0/3309981395 shutdown_connections 2024-09-19T06:09:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.727+0000 7fba46fa0740 1 -- 172.21.15.5:0/3309981395 wait complete. 2024-09-19T06:09:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.727+0000 7fba46fa0740 20 client.26546 trim_cache size 0 max 16384 2024-09-19T06:09:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.748+0000 7fba46fa0740 1 Processor -- start 2024-09-19T06:09:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.748+0000 7fba46fa0740 1 -- start start 2024-09-19T06:09:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.748+0000 7fba46fa0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fdb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.748+0000 7fba46fa0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd9f40 0x55dda3ff0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.748+0000 7fba46fa0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3ff1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.748+0000 7fba46fa0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba401ab6a0 con 0x55dda3fdc680 2024-09-19T06:09:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.748+0000 7fba46fa0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fba401ad1e0 con 0x55dda3fd9f40 2024-09-19T06:09:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.748+0000 7fba46fa0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba401abd70 con 0x55dda3fd14a0 2024-09-19T06:09:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.748+0000 7fba3ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3ff1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.748+0000 7fba3ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3ff1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42384/0 (socket says 172.21.15.5:42384) 2024-09-19T06:09:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.748+0000 7fba3ed9d640 1 -- 172.21.15.5:0/1969950959 learned_addr learned my addr 172.21.15.5:0/1969950959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.748+0000 7fba3dd9b640 1 --2- 172.21.15.5:0/1969950959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd9f40 0x55dda3ff0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.749+0000 7fba3e59c640 1 --2- 172.21.15.5:0/1969950959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fdb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.749+0000 7fba3ed9d640 1 -- 172.21.15.5:0/1969950959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 msgr2=0x55dda3fdb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:34.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.749+0000 7fba3ed9d640 1 --2- 172.21.15.5:0/1969950959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fdb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.749+0000 7fba3ed9d640 1 -- 172.21.15.5:0/1969950959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd9f40 msgr2=0x55dda3ff0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:34.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.749+0000 7fba3ed9d640 1 --2- 172.21.15.5:0/1969950959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd9f40 0x55dda3ff0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.749+0000 7fba3ed9d640 1 -- 172.21.15.5:0/1969950959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dda3fd1bf0 con 0x55dda3fdc680 2024-09-19T06:09:34.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.749+0000 7fba3ed9d640 1 --2- 172.21.15.5:0/1969950959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3ff1280 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fba38010bc0 tx=0x7fba38013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333e033e0b9661c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:34.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.749+0000 7fba27fff640 1 -- 172.21.15.5:0/1969950959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba38004d20 con 0x55dda3fdc680 2024-09-19T06:09:34.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.749+0000 7fba27fff640 1 -- 172.21.15.5:0/1969950959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba38023070 con 0x55dda3fdc680 2024-09-19T06:09:34.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.749+0000 7fba27fff640 1 -- 172.21.15.5:0/1969950959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba38005880 con 0x55dda3fdc680 2024-09-19T06:09:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.750+0000 7fba46fa0740 1 -- 172.21.15.5:0/1969950959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 msgr2=0x55dda3ff1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.750+0000 7fba46fa0740 1 --2- 172.21.15.5:0/1969950959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3ff1280 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fba38010bc0 tx=0x7fba38013000 comp rx=0 tx=0).stop 2024-09-19T06:09:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.750+0000 7fba46fa0740 1 -- 172.21.15.5:0/1969950959 shutdown_connections 2024-09-19T06:09:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.750+0000 7fba46fa0740 1 --2- 172.21.15.5:0/1969950959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3ff1280 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.750+0000 7fba46fa0740 1 --2- 172.21.15.5:0/1969950959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd9f40 0x55dda3ff0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.750+0000 7fba46fa0740 1 --2- 172.21.15.5:0/1969950959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fdb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.750+0000 7fba46fa0740 1 -- 172.21.15.5:0/1969950959 >> 172.21.15.5:0/1969950959 conn(0x55dda3fcda30 msgr2=0x55dda3fdad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.750+0000 7fba46fa0740 1 -- 172.21.15.5:0/1969950959 shutdown_connections 2024-09-19T06:09:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.750+0000 7fba46fa0740 1 -- 172.21.15.5:0/1969950959 wait complete. 2024-09-19T06:09:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba46fa0740 1 -- start start 2024-09-19T06:09:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba46fa0740 1 Processor -- start 2024-09-19T06:09:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba46fa0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba46fa0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd9f40 0x55dda3fd6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba46fa0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3fd6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba46fa0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba401acb10 con 0x55dda3fdc680 2024-09-19T06:09:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba46fa0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fba401ac440 con 0x55dda3fd9f40 2024-09-19T06:09:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba46fa0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba401a96e0 con 0x55dda3fd14a0 2024-09-19T06:09:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba46fa0740 10 client.0 _ll_register_callbacks cb 0x7fba34005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba3e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba3dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd9f40 0x55dda3fd6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba3e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fed4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50746/0 (socket says 172.21.15.5:50746) 2024-09-19T06:09:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba3dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd9f40 0x55dda3fd6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41648/0 (socket says 172.21.15.5:41648) 2024-09-19T06:09:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.751+0000 7fba3e59c640 1 -- 172.21.15.5:0/1229217603 learned_addr learned my addr 172.21.15.5:0/1229217603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.752+0000 7fba3ed9d640 1 --2- 172.21.15.5:0/1229217603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3fd6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.752+0000 7fba3e59c640 1 -- 172.21.15.5:0/1229217603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd9f40 msgr2=0x55dda3fd6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.752+0000 7fba3e59c640 1 --2- 172.21.15.5:0/1229217603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd9f40 0x55dda3fd6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.752+0000 7fba3e59c640 1 -- 172.21.15.5:0/1229217603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 msgr2=0x55dda3fd6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.752+0000 7fba3e59c640 1 --2- 172.21.15.5:0/1229217603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3fd6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.752+0000 7fba3e59c640 1 -- 172.21.15.5:0/1229217603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dda3fce6c0 con 0x55dda3fd14a0 2024-09-19T06:09:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.752+0000 7fba3e59c640 1 --2- 172.21.15.5:0/1229217603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fed4e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fba30000c00 tx=0x7fba30009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdfffe27eb11c63d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.752+0000 7fba25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:09:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.752+0000 7fba25ffb640 1 -- 172.21.15.5:0/1229217603 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba300050a0 con 0x55dda3fd14a0 2024-09-19T06:09:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.752+0000 7fba46fa0740 1 -- 172.21.15.5:0/1229217603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba38023070 con 0x55dda3fd14a0 2024-09-19T06:09:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.752+0000 7fba25ffb640 1 -- 172.21.15.5:0/1229217603 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fba30005240 con 0x55dda3fd14a0 2024-09-19T06:09:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.753+0000 7fba25ffb640 1 -- 172.21.15.5:0/1229217603 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fba3003a050 con 0x55dda3fd14a0 2024-09-19T06:09:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.753+0000 7fba46fa0740 10 client.26458 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.753+0000 7fba46fa0740 1 -- 172.21.15.5:0/1229217603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dda405f8c0 con 0x55dda3fd14a0 2024-09-19T06:09:34.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.753+0000 7fba25ffb640 1 -- 172.21.15.5:0/1229217603 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba38023070 con 0x55dda3fd14a0 2024-09-19T06:09:34.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.753+0000 7fba25ffb640 1 -- 172.21.15.5:0/1229217603 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fba30018070 con 0x55dda3fd14a0 2024-09-19T06:09:34.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.753+0000 7fba46fa0740 10 client.26458 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:34.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.753+0000 7fba46fa0740 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:34.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.753+0000 7fba46fa0740 20 client.26458 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:34.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.753+0000 7fba46fa0740 1 --2- 172.21.15.5:0/1229217603 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dda3ff4e20 0x55dda4061e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:34.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.753+0000 7fba46fa0740 4 client.26458 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:09:34.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.753+0000 7fba46fa0740 1 -- 172.21.15.5:0/1229217603 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dda3fd9d90 con 0x55dda3ff4e20 2024-09-19T06:09:34.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.753+0000 7fba3dd9b640 1 --2- 172.21.15.5:0/1229217603 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dda3ff4e20 0x55dda4061e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:34.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba25ffb640 10 client.26458 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:34.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba3dd9b640 1 --2- 172.21.15.5:0/1229217603 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dda3ff4e20 0x55dda4061e60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7c1c9b5486f71 server_cookie=2c147e8b3ab072f in_seq=0 out_seq=0 2024-09-19T06:09:34.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba25ffb640 1 -- 172.21.15.5:0/1229217603 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dda3fd9d90 con 0x55dda3ff4e20 2024-09-19T06:09:34.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba25ffb640 10 client.26458 handle_command_reply: tid=0 2024-09-19T06:09:34.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba46fa0740 1 client.26458 shutdown 2024-09-19T06:09:34.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba46fa0740 1 -- 172.21.15.5:0/1229217603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 msgr2=0x55dda3fed4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:34.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba46fa0740 1 --2- 172.21.15.5:0/1229217603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fed4e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fba30000c00 tx=0x7fba30009770 comp rx=0 tx=0).stop 2024-09-19T06:09:34.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba46fa0740 1 -- 172.21.15.5:0/1229217603 shutdown_connections 2024-09-19T06:09:34.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba46fa0740 1 --2- 172.21.15.5:0/1229217603 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dda3ff4e20 0x55dda4061e60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba46fa0740 1 --2- 172.21.15.5:0/1229217603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda3fdc680 0x55dda3fd6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba46fa0740 1 --2- 172.21.15.5:0/1229217603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dda3fd9f40 0x55dda3fd6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba46fa0740 1 --2- 172.21.15.5:0/1229217603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dda3fd14a0 0x55dda3fed4e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba46fa0740 1 -- 172.21.15.5:0/1229217603 >> 172.21.15.5:0/1229217603 conn(0x55dda3fcda30 msgr2=0x55dda3fd3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:34.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba25ffb640 0 client.26458 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:34.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba46fa0740 1 -- 172.21.15.5:0/1229217603 shutdown_connections 2024-09-19T06:09:34.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.754+0000 7fba46fa0740 1 -- 172.21.15.5:0/1229217603 wait complete. 2024-09-19T06:09:34.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.755+0000 7fba46fa0740 20 client.26458 trim_cache size 0 max 16384 2024-09-19T06:09:34.772 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:09:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:09:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:09:34.772 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:09:34.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.755+0000 7fba44bc7640 1 -- 172.21.15.5:0/3715953274 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fba280a3430 msgr2=0x7fba280a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:34.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.755+0000 7fba44bc7640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fba280a3430 0x7fba280a58f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba38005fd0 tx=0x7fba38009ed0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.755+0000 7fba44bc7640 1 -- 172.21.15.5:0/3715953274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101430 msgr2=0x7fba4019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:34.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.755+0000 7fba44bc7640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101430 0x7fba4019aaf0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fba34005ab0 tx=0x7fba34003b90 comp rx=0 tx=0).stop 2024-09-19T06:09:34.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.756+0000 7fba44bc7640 1 -- 172.21.15.5:0/3715953274 shutdown_connections 2024-09-19T06:09:34.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.756+0000 7fba44bc7640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba40106a90 0x7fba4019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.756+0000 7fba44bc7640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101430 0x7fba4019aaf0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.756+0000 7fba44bc7640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fba280a3430 0x7fba280a58f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.756+0000 7fba44bc7640 1 --2- 172.21.15.5:0/3715953274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba40100a90 0x7fba4019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:34.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.756+0000 7fba44bc7640 1 -- 172.21.15.5:0/3715953274 >> 172.21.15.5:0/3715953274 conn(0x7fba400fc7d0 msgr2=0x7fba4006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:34.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.756+0000 7fba44bc7640 1 -- 172.21.15.5:0/3715953274 shutdown_connections 2024-09-19T06:09:34.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:34.756+0000 7fba44bc7640 1 -- 172.21.15.5:0/3715953274 wait complete. 2024-09-19T06:09:34.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 154069 2.75 MB/sec execute 180 sec latency 34.695 ms 2024-09-19T06:09:35.037 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:34 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:09:35.037 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:34 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:09:35.037 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:34 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:09:35.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:34 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:09:35.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:34 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:09:35.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:34 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:09:35.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:34 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:09:35.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:34 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:09:35.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:34 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:09:35.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 154632 2.74 MB/sec execute 181 sec latency 20.075 ms 2024-09-19T06:09:35.932 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:09:35.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:09:36.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:35 smithi005 ceph-mon[30955]: pgmap v191: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-09-19T06:09:36.230 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:09:36.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:35 smithi157 ceph-mon[35346]: pgmap v191: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-09-19T06:09:36.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:35 smithi049 ceph-mon[34640]: pgmap v191: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-09-19T06:09:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.756+0000 7f78c8500640 1 -- 172.21.15.5:0/2920046739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c0100b60 msgr2=0x7f78c0100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.756+0000 7f78c8500640 1 --2- 172.21.15.5:0/2920046739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c0100b60 0x7f78c0100f40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009900 tx=0x7f78bc02a410 comp rx=0 tx=0).stop 2024-09-19T06:09:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.756+0000 7f78c8500640 1 -- 172.21.15.5:0/2920046739 shutdown_connections 2024-09-19T06:09:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.756+0000 7f78c8500640 1 --2- 172.21.15.5:0/2920046739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c010e540 0x7f78c0110930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.756+0000 7f78c8500640 1 --2- 172.21.15.5:0/2920046739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78c0101480 0x7f78c010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.756+0000 7f78c8500640 1 --2- 172.21.15.5:0/2920046739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c0100b60 0x7f78c0100f40 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.756+0000 7f78c8500640 1 -- 172.21.15.5:0/2920046739 >> 172.21.15.5:0/2920046739 conn(0x7f78c00fc7d0 msgr2=0x7f78c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.756+0000 7f78c8500640 1 -- 172.21.15.5:0/2920046739 shutdown_connections 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.756+0000 7f78c8500640 1 -- 172.21.15.5:0/2920046739 wait complete. 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.756+0000 7f78c8500640 1 Processor -- start 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c8500640 1 -- start start 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c8500640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c0100b60 0x7f78c019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c8500640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78c0101480 0x7f78c019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c8500640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c010e540 0x7f78c019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c8500640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0114120 con 0x7f78c010e540 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c8500640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0113fa0 con 0x7f78c0101480 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c8500640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78c01142a0 con 0x7f78c0100b60 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c629c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c0100b60 0x7f78c019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c5a9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78c0101480 0x7f78c019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c629c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c0100b60 0x7f78c019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50762/0 (socket says 172.21.15.5:50762) 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c5a9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78c0101480 0x7f78c019abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41668/0 (socket says 172.21.15.5:41668) 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c629c640 1 -- 172.21.15.5:0/1149849622 learned_addr learned my addr 172.21.15.5:0/1149849622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c6a9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c010e540 0x7f78c019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c629c640 1 -- 172.21.15.5:0/1149849622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78c0101480 msgr2=0x7f78c019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c629c640 1 --2- 172.21.15.5:0/1149849622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78c0101480 0x7f78c019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c629c640 1 -- 172.21.15.5:0/1149849622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c010e540 msgr2=0x7f78c019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c629c640 1 --2- 172.21.15.5:0/1149849622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c010e540 0x7f78c019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.757+0000 7f78c629c640 1 -- 172.21.15.5:0/1149849622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c019f630 con 0x7f78c0100b60 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.758+0000 7f78c629c640 1 --2- 172.21.15.5:0/1149849622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c0100b60 0x7f78c019a680 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f78bc005940 tx=0x7f78bc0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.758+0000 7f78b77fe640 1 -- 172.21.15.5:0/1149849622 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc03f030 con 0x7f78c0100b60 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.758+0000 7f78b77fe640 1 -- 172.21.15.5:0/1149849622 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78bc0044c0 con 0x7f78c0100b60 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.758+0000 7f78b77fe640 1 -- 172.21.15.5:0/1149849622 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc035070 con 0x7f78c0100b60 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.758+0000 7f78c8500640 1 -- 172.21.15.5:0/1149849622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c019f8c0 con 0x7f78c0100b60 2024-09-19T06:09:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.758+0000 7f78c8500640 1 -- 172.21.15.5:0/1149849622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c019fd50 con 0x7f78c0100b60 2024-09-19T06:09:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.759+0000 7f78c8500640 1 -- 172.21.15.5:0/1149849622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c006a810 con 0x7f78c0100b60 2024-09-19T06:09:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.761+0000 7f78b77fe640 1 -- 172.21.15.5:0/1149849622 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f78bc042030 con 0x7f78c0100b60 2024-09-19T06:09:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.761+0000 7f78b77fe640 1 --2- 172.21.15.5:0/1149849622 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f78ac0a33b0 0x7f78ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.761+0000 7f78c5a9b640 1 --2- 172.21.15.5:0/1149849622 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f78ac0a33b0 0x7f78ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.762+0000 7f78b77fe640 1 -- 172.21.15.5:0/1149849622 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f78bc0efd50 con 0x7f78c0100b60 2024-09-19T06:09:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.763+0000 7f78c5a9b640 1 --2- 172.21.15.5:0/1149849622 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f78ac0a33b0 0x7f78ac0a5870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f78c010e9f0 tx=0x7f78b0005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:36.766+0000 7f78b77fe640 1 -- 172.21.15.5:0/1149849622 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f78bc0bc090 con 0x7f78c0100b60 2024-09-19T06:09:36.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 155111 2.74 MB/sec execute 182 sec latency 20.087 ms 2024-09-19T06:09:37.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.014+0000 7f78c8500640 1 -- 172.21.15.5:0/1149849622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78c0100f40 con 0x7f78c0100b60 2024-09-19T06:09:37.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.015+0000 7f78b77fe640 1 -- 172.21.15.5:0/1149849622 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11959 (secure 0 0 0) 0x7f78bc0c0390 con 0x7f78c0100b60 2024-09-19T06:09:37.017 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:09:37.017 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-19T06:07:24:781633+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:07:24.019631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:09:37.017 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-19T06:09:37.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.017+0000 7f78c8500640 1 -- 172.21.15.5:0/1149849622 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f78ac0a33b0 msgr2=0x7f78ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:37.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.017+0000 7f78c8500640 1 --2- 172.21.15.5:0/1149849622 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f78ac0a33b0 0x7f78ac0a5870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f78c010e9f0 tx=0x7f78b0005f90 comp rx=0 tx=0).stop 2024-09-19T06:09:37.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.017+0000 7f78c8500640 1 -- 172.21.15.5:0/1149849622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c0100b60 msgr2=0x7f78c019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:37.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.017+0000 7f78c8500640 1 --2- 172.21.15.5:0/1149849622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c0100b60 0x7f78c019a680 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f78bc005940 tx=0x7f78bc0040c0 comp rx=0 tx=0).stop 2024-09-19T06:09:37.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.017+0000 7f78c8500640 1 -- 172.21.15.5:0/1149849622 shutdown_connections 2024-09-19T06:09:37.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.017+0000 7f78c8500640 1 --2- 172.21.15.5:0/1149849622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c010e540 0x7f78c019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:37.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.017+0000 7f78c8500640 1 --2- 172.21.15.5:0/1149849622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78c0101480 0x7f78c019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:37.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.017+0000 7f78c8500640 1 --2- 172.21.15.5:0/1149849622 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f78ac0a33b0 0x7f78ac0a5870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:37.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.017+0000 7f78c8500640 1 --2- 172.21.15.5:0/1149849622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c0100b60 0x7f78c019a680 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:37.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.017+0000 7f78c8500640 1 -- 172.21.15.5:0/1149849622 >> 172.21.15.5:0/1149849622 conn(0x7f78c00fc7d0 msgr2=0x7f78c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:37.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.017+0000 7f78c8500640 1 -- 172.21.15.5:0/1149849622 shutdown_connections 2024-09-19T06:09:37.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:37.017+0000 7f78c8500640 1 -- 172.21.15.5:0/1149849622 wait complete. 2024-09-19T06:09:37.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:09:37.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 155844 2.76 MB/sec execute 183 sec latency 15.155 ms 2024-09-19T06:09:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:09:38.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:37 smithi005 ceph-mon[30955]: pgmap v192: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-09-19T06:09:38.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:37 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1149849622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:09:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:37 smithi157 ceph-mon[35346]: pgmap v192: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-09-19T06:09:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:37 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1149849622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:09:38.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:37 smithi049 ceph-mon[34640]: pgmap v192: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-09-19T06:09:38.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:37 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1149849622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:09:38.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.524+0000 7fb58d11a640 1 -- 172.21.15.5:0/3952408703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58810c170 msgr2=0x7fb58810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:38.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.524+0000 7fb58d11a640 1 --2- 172.21.15.5:0/3952408703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58810c170 0x7fb58810e610 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb580009ca0 tx=0x7fb58002a950 comp rx=0 tx=0).stop 2024-09-19T06:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.524+0000 7fb58d11a640 1 -- 172.21.15.5:0/3952408703 shutdown_connections 2024-09-19T06:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.524+0000 7fb58d11a640 1 --2- 172.21.15.5:0/3952408703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58810c170 0x7fb58810e610 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.524+0000 7fb58d11a640 1 --2- 172.21.15.5:0/3952408703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588103660 0x7fb58810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.524+0000 7fb58d11a640 1 --2- 172.21.15.5:0/3952408703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb588102d40 0x7fb588103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.524+0000 7fb58d11a640 1 -- 172.21.15.5:0/3952408703 >> 172.21.15.5:0/3952408703 conn(0x7fb5880fc7a0 msgr2=0x7fb5880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.524+0000 7fb58d11a640 1 -- 172.21.15.5:0/3952408703 shutdown_connections 2024-09-19T06:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.524+0000 7fb58d11a640 1 -- 172.21.15.5:0/3952408703 wait complete. 2024-09-19T06:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.525+0000 7fb58d11a640 1 Processor -- start 2024-09-19T06:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.525+0000 7fb58d11a640 1 -- start start 2024-09-19T06:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb58d11a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588102d40 0x7fb5881009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb58d11a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb588103660 0x7fb588100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb58d11a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58810c170 0x7fb588101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb58d11a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb588111de0 con 0x7fb58810c170 2024-09-19T06:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb58d11a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb588111c60 con 0x7fb588102d40 2024-09-19T06:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb58d11a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb588111f60 con 0x7fb588103660 2024-09-19T06:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb586d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588102d40 0x7fb5881009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb586d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588102d40 0x7fb5881009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41692/0 (socket says 172.21.15.5:41692) 2024-09-19T06:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb586d9d640 1 -- 172.21.15.5:0/3123186628 learned_addr learned my addr 172.21.15.5:0/3123186628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb58759e640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58810c170 0x7fb588101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb58659c640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb588103660 0x7fb588100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb586d9d640 1 -- 172.21.15.5:0/3123186628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb588103660 msgr2=0x7fb588100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb586d9d640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb588103660 0x7fb588100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb586d9d640 1 -- 172.21.15.5:0/3123186628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58810c170 msgr2=0x7fb588101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb586d9d640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58810c170 0x7fb588101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb586d9d640 1 -- 172.21.15.5:0/3123186628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5881a67e0 con 0x7fb588102d40 2024-09-19T06:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb58659c640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb588103660 0x7fb588100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb58759e640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58810c170 0x7fb588101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.526+0000 7fb586d9d640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588102d40 0x7fb5881009a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb57800e9f0 tx=0x7fb57800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.527+0000 7fb56ffff640 1 -- 172.21.15.5:0/3123186628 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57800cd80 con 0x7fb588102d40 2024-09-19T06:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.527+0000 7fb58d11a640 1 -- 172.21.15.5:0/3123186628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5881a6ad0 con 0x7fb588102d40 2024-09-19T06:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.527+0000 7fb56ffff640 1 -- 172.21.15.5:0/3123186628 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5780108e0 con 0x7fb588102d40 2024-09-19T06:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.527+0000 7fb56ffff640 1 -- 172.21.15.5:0/3123186628 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578018920 con 0x7fb588102d40 2024-09-19T06:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.527+0000 7fb58d11a640 1 -- 172.21.15.5:0/3123186628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5881a7060 con 0x7fb588102d40 2024-09-19T06:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.529+0000 7fb58f4f3740 1 Processor -- start 2024-09-19T06:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.529+0000 7fb58f4f3740 1 -- start start 2024-09-19T06:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.529+0000 7fb58f4f3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf71b40 0x55e65bf70090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.529+0000 7fb58f4f3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf705d0 0x55e65bf70a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58f4f3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf7b370 0x55e65bf7d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58f4f3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e65bed62f0 con 0x55e65bf7b370 2024-09-19T06:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58f4f3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e65be7dcf0 con 0x55e65bf705d0 2024-09-19T06:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58f4f3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e65bed22b0 con 0x55e65bf71b40 2024-09-19T06:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf71b40 0x55e65bf70090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf71b40 0x55e65bf70090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50796/0 (socket says 172.21.15.5:50796) 2024-09-19T06:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58759e640 1 -- 172.21.15.5:0/3705480919 learned_addr learned my addr 172.21.15.5:0/3705480919 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58659c640 1 --2- 172.21.15.5:0/3705480919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf7b370 0x55e65bf7d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58759e640 1 -- 172.21.15.5:0/3705480919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf705d0 msgr2=0x55e65bf70a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:09:38.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58759e640 1 --2- 172.21.15.5:0/3705480919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf705d0 0x55e65bf70a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58759e640 1 -- 172.21.15.5:0/3705480919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf7b370 msgr2=0x55e65bf7d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:38.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58759e640 1 --2- 172.21.15.5:0/3705480919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf7b370 0x55e65bf7d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58759e640 1 -- 172.21.15.5:0/3705480919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e65be623a0 con 0x55e65bf71b40 2024-09-19T06:09:38.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb58659c640 1 --2- 172.21.15.5:0/3705480919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf7b370 0x55e65bf7d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:09:38.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.530+0000 7fb56ffff640 1 -- 172.21.15.5:0/3123186628 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb578010a90 con 0x7fb588102d40 2024-09-19T06:09:38.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.531+0000 7fb58759e640 1 --2- 172.21.15.5:0/3705480919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf71b40 0x55e65bf70090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb580009ca0 tx=0x7fb580009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da549f806a19956 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:38.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.531+0000 7fb56d7fa640 1 -- 172.21.15.5:0/3705480919 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580048070 con 0x55e65bf71b40 2024-09-19T06:09:38.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.531+0000 7fb56d7fa640 1 -- 172.21.15.5:0/3705480919 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb580003e20 con 0x55e65bf71b40 2024-09-19T06:09:38.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.531+0000 7fb56d7fa640 1 -- 172.21.15.5:0/3705480919 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580034040 con 0x55e65bf71b40 2024-09-19T06:09:38.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.531+0000 7fb56ffff640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5700a33b0 0x7fb5700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.532+0000 7fb586d9d640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5700a33b0 0x7fb5700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.532+0000 7fb56ffff640 1 -- 172.21.15.5:0/3123186628 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fb5780d0090 con 0x7fb588102d40 2024-09-19T06:09:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.532+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3705480919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf71b40 msgr2=0x55e65bf70090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.532+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3705480919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf71b40 0x55e65bf70090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb580009ca0 tx=0x7fb580009360 comp rx=0 tx=0).stop 2024-09-19T06:09:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.532+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3705480919 shutdown_connections 2024-09-19T06:09:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.532+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3705480919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf7b370 0x55e65bf7d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.532+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3705480919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf705d0 0x55e65bf70a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.532+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3705480919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf71b40 0x55e65bf70090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.532+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3705480919 >> 172.21.15.5:0/3705480919 conn(0x55e65bf39310 msgr2=0x55e65bf6cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.532+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3705480919 shutdown_connections 2024-09-19T06:09:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.532+0000 7fb586d9d640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5700a33b0 0x7fb5700a5870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55e65bf6dac0 tx=0x7fb578025b40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.532+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3705480919 wait complete. 2024-09-19T06:09:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.533+0000 7fb58f4f3740 1 -- start start 2024-09-19T06:09:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.533+0000 7fb58f4f3740 1 Processor -- start 2024-09-19T06:09:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.533+0000 7fb58f4f3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf705d0 0x55e65bf7ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.533+0000 7fb58f4f3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 0x55e65bf78020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.533+0000 7fb58f4f3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf7b370 0x55e65bf78610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.533+0000 7fb58f4f3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e65bf7e010 con 0x55e65bf705d0 2024-09-19T06:09:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58f4f3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e65be91060 con 0x55e65bf71b40 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58f4f3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e65bf7e190 con 0x55e65bf7b370 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58f4f3740 10 client.0 _ll_register_callbacks cb 0x55e65bbe48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 0x55e65bf78020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 0x55e65bf78020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41706/0 (socket says 172.21.15.5:41706) 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58659c640 1 -- 172.21.15.5:0/761203477 learned_addr learned my addr 172.21.15.5:0/761203477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58759e640 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf7b370 0x55e65bf78610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58759e640 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf705d0 0x55e65bf7ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58659c640 1 -- 172.21.15.5:0/761203477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf7b370 msgr2=0x55e65bf78610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58659c640 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf7b370 0x55e65bf78610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58659c640 1 -- 172.21.15.5:0/761203477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf705d0 msgr2=0x55e65bf7ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58659c640 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf705d0 0x55e65bf7ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58659c640 1 -- 172.21.15.5:0/761203477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65bc473c0 con 0x55e65bf71b40 2024-09-19T06:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58759e640 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf705d0 0x55e65bf7ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58659c640 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 0x55e65bf78020 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb57c012780 tx=0x7fb57c012c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd88ab59f93409be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb56aff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb56aff5640 1 -- 172.21.15.5:0/761203477 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c010dc0 con 0x55e65bf71b40 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb56aff5640 1 -- 172.21.15.5:0/761203477 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fb57c004030 con 0x55e65bf71b40 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.534+0000 7fb58f4f3740 1 -- 172.21.15.5:0/761203477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb580003e20 con 0x55e65bf71b40 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.535+0000 7fb56aff5640 1 -- 172.21.15.5:0/761203477 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb57c004230 con 0x55e65bf71b40 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.535+0000 7fb58f4f3740 10 client.26567 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.535+0000 7fb58f4f3740 1 -- 172.21.15.5:0/761203477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e65bf922e0 con 0x55e65bf71b40 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.535+0000 7fb56aff5640 1 -- 172.21.15.5:0/761203477 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb57c0043d0 con 0x55e65bf71b40 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.535+0000 7fb56aff5640 1 -- 172.21.15.5:0/761203477 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb57c0235e0 con 0x55e65bf71b40 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.535+0000 7fb58f4f3740 10 client.26567 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.535+0000 7fb58f4f3740 10 client.26567 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:09:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.535+0000 7fb58f4f3740 20 client.26567 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.535+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55e65bf93840 0x55e65bf95c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.535+0000 7fb58f4f3740 4 client.26567 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.535+0000 7fb58f4f3740 1 -- 172.21.15.5:0/761203477 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb580003e20 con 0x55e65bf93840 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.536+0000 7fb58759e640 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55e65bf93840 0x55e65bf95c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.536+0000 7fb58759e640 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55e65bf93840 0x55e65bf95c60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16ef4a64143d0615 server_cookie=8aef05bef90ac044 in_seq=0 out_seq=0 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.536+0000 7fb56aff5640 10 client.26567 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.536+0000 7fb56aff5640 1 -- 172.21.15.5:0/761203477 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fb580003e20 con 0x55e65bf93840 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb56aff5640 10 client.26567 handle_command_reply: tid=0 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb58f4f3740 1 client.26567 shutdown 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb58f4f3740 1 -- 172.21.15.5:0/761203477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 msgr2=0x55e65bf78020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 0x55e65bf78020 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb57c012780 tx=0x7fb57c012c40 comp rx=0 tx=0).stop 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb58f4f3740 1 -- 172.21.15.5:0/761203477 shutdown_connections 2024-09-19T06:09:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55e65bf93840 0x55e65bf95c60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf7b370 0x55e65bf78610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 0x55e65bf78020 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/761203477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf705d0 0x55e65bf7ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb56aff5640 0 client.26567 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:09:38.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb58f4f3740 1 -- 172.21.15.5:0/761203477 >> 172.21.15.5:0/761203477 conn(0x55e65bf39310 msgr2=0x55e65bf72960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:38.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb58f4f3740 1 -- 172.21.15.5:0/761203477 shutdown_connections 2024-09-19T06:09:38.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb58f4f3740 1 -- 172.21.15.5:0/761203477 wait complete. 2024-09-19T06:09:38.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.537+0000 7fb58f4f3740 20 client.26567 trim_cache size 0 max 16384 2024-09-19T06:09:38.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.553+0000 7fb58f4f3740 1 Processor -- start 2024-09-19T06:09:38.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.553+0000 7fb58f4f3740 1 -- start start 2024-09-19T06:09:38.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.553+0000 7fb58f4f3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf705d0 0x55e65bf8fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.553+0000 7fb58f4f3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf71b40 0x55e65bf6c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.553+0000 7fb58f4f3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf7b370 0x55e65bf6cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.553+0000 7fb58f4f3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881ab620 con 0x55e65bf71b40 2024-09-19T06:09:38.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.553+0000 7fb58f4f3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881ad160 con 0x55e65bf7b370 2024-09-19T06:09:38.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.553+0000 7fb58f4f3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881abcf0 con 0x55e65bf705d0 2024-09-19T06:09:38.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb58759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf705d0 0x55e65bf8fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb58659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf71b40 0x55e65bf6c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb58759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf705d0 0x55e65bf8fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50812/0 (socket says 172.21.15.5:50812) 2024-09-19T06:09:38.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb58759e640 1 -- 172.21.15.5:0/3638087845 learned_addr learned my addr 172.21.15.5:0/3638087845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:38.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb58759e640 1 --2- 172.21.15.5:0/3638087845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf7b370 0x55e65bf6cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb58759e640 1 -- 172.21.15.5:0/3638087845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf7b370 msgr2=0x55e65bf6cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:38.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb58759e640 1 --2- 172.21.15.5:0/3638087845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf7b370 0x55e65bf6cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb58759e640 1 -- 172.21.15.5:0/3638087845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf71b40 msgr2=0x55e65bf6c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:38.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb58759e640 1 --2- 172.21.15.5:0/3638087845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf71b40 0x55e65bf6c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb58759e640 1 -- 172.21.15.5:0/3638087845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e65bf7b0e0 con 0x55e65bf705d0 2024-09-19T06:09:38.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb58659c640 1 --2- 172.21.15.5:0/3638087845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf71b40 0x55e65bf6c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:38.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb58759e640 1 --2- 172.21.15.5:0/3638087845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf705d0 0x55e65bf8fb40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb580030040 tx=0x7fb580005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f42c48733de8763 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:38.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb56cff9640 1 -- 172.21.15.5:0/3638087845 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580048040 con 0x55e65bf705d0 2024-09-19T06:09:38.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb56cff9640 1 -- 172.21.15.5:0/3638087845 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb580049040 con 0x55e65bf705d0 2024-09-19T06:09:38.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.554+0000 7fb56cff9640 1 -- 172.21.15.5:0/3638087845 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580034040 con 0x55e65bf705d0 2024-09-19T06:09:38.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.555+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3638087845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf705d0 msgr2=0x55e65bf8fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:38.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.555+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3638087845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf705d0 0x55e65bf8fb40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb580030040 tx=0x7fb580005c50 comp rx=0 tx=0).stop 2024-09-19T06:09:38.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.555+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3638087845 shutdown_connections 2024-09-19T06:09:38.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.555+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3638087845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf7b370 0x55e65bf6cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.555+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3638087845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf71b40 0x55e65bf6c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.555+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3638087845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf705d0 0x55e65bf8fb40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.555+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3638087845 >> 172.21.15.5:0/3638087845 conn(0x55e65bf39310 msgr2=0x55e65bf73260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.555+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3638087845 shutdown_connections 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.555+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3638087845 wait complete. 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58f4f3740 1 -- start start 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58f4f3740 1 Processor -- start 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58f4f3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf705d0 0x55e65bf8c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58f4f3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 0x55e65bf877f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58f4f3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf7b370 0x55e65bf87d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58f4f3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881aca90 con 0x55e65bf7b370 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58f4f3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881ac3c0 con 0x55e65bf71b40 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58f4f3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881a9660 con 0x55e65bf705d0 2024-09-19T06:09:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58f4f3740 10 client.0 _ll_register_callbacks cb 0x7fb58806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:38.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 0x55e65bf877f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf7b370 0x55e65bf87d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 0x55e65bf877f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41718/0 (socket says 172.21.15.5:41718) 2024-09-19T06:09:38.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58659c640 1 -- 172.21.15.5:0/3828968686 learned_addr learned my addr 172.21.15.5:0/3828968686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:38.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf7b370 0x55e65bf87d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42468/0 (socket says 172.21.15.5:42468) 2024-09-19T06:09:38.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.556+0000 7fb58759e640 1 --2- 172.21.15.5:0/3828968686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf705d0 0x55e65bf8c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb58659c640 1 -- 172.21.15.5:0/3828968686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf705d0 msgr2=0x55e65bf8c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:38.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb58659c640 1 --2- 172.21.15.5:0/3828968686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf705d0 0x55e65bf8c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb58659c640 1 -- 172.21.15.5:0/3828968686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf7b370 msgr2=0x55e65bf87d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:38.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb58659c640 1 --2- 172.21.15.5:0/3828968686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf7b370 0x55e65bf87d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb58659c640 1 -- 172.21.15.5:0/3828968686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65bf88580 con 0x55e65bf71b40 2024-09-19T06:09:38.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb58659c640 1 --2- 172.21.15.5:0/3828968686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 0x55e65bf877f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00a010 tx=0x7fb57c009e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fde5013b93d7e5ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb56aff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb56aff5640 1 -- 172.21.15.5:0/3828968686 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c031070 con 0x55e65bf71b40 2024-09-19T06:09:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb56aff5640 1 -- 172.21.15.5:0/3828968686 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fb57c023650 con 0x55e65bf71b40 2024-09-19T06:09:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3828968686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb580049040 con 0x55e65bf71b40 2024-09-19T06:09:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb56aff5640 1 -- 172.21.15.5:0/3828968686 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb57c03d050 con 0x55e65bf71b40 2024-09-19T06:09:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb58f4f3740 10 client.26573 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.557+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3828968686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e65bffe570 con 0x55e65bf71b40 2024-09-19T06:09:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.558+0000 7fb56aff5640 1 -- 172.21.15.5:0/3828968686 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb580049040 con 0x55e65bf71b40 2024-09-19T06:09:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.558+0000 7fb56aff5640 1 -- 172.21.15.5:0/3828968686 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb57c036440 con 0x55e65bf71b40 2024-09-19T06:09:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.558+0000 7fb58f4f3740 10 client.26573 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.558+0000 7fb58f4f3740 10 client.26573 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:09:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.558+0000 7fb58f4f3740 20 client.26573 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:38.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.558+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3828968686 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55e65bffec90 0x55e65c001080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:38.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.558+0000 7fb58f4f3740 4 client.26573 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:09:38.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.558+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3828968686 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e65bee3860 con 0x55e65bffec90 2024-09-19T06:09:38.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.558+0000 7fb58759e640 1 --2- 172.21.15.5:0/3828968686 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55e65bffec90 0x55e65c001080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:38.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.558+0000 7fb58759e640 1 --2- 172.21.15.5:0/3828968686 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55e65bffec90 0x55e65c001080 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c98da1c73cbcf7c1 server_cookie=b8be67fe564415c2 in_seq=0 out_seq=0 2024-09-19T06:09:38.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.558+0000 7fb56aff5640 10 client.26573 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:09:38.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb56aff5640 1 -- 172.21.15.5:0/3828968686 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e65bee3860 con 0x55e65bffec90 2024-09-19T06:09:38.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb56aff5640 10 client.26573 handle_command_reply: tid=0 2024-09-19T06:09:38.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb58f4f3740 1 client.26573 shutdown 2024-09-19T06:09:38.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3828968686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 msgr2=0x55e65bf877f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:38.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3828968686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 0x55e65bf877f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00a010 tx=0x7fb57c009e10 comp rx=0 tx=0).stop 2024-09-19T06:09:38.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3828968686 shutdown_connections 2024-09-19T06:09:38.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3828968686 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55e65bffec90 0x55e65c001080 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3828968686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e65bf7b370 0x55e65bf87d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3828968686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e65bf71b40 0x55e65bf877f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb56aff5640 0 client.26573 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:09:38.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb58f4f3740 1 --2- 172.21.15.5:0/3828968686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65bf705d0 0x55e65bf8c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3828968686 >> 172.21.15.5:0/3828968686 conn(0x55e65bf39310 msgr2=0x55e65bf764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:38.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3828968686 shutdown_connections 2024-09-19T06:09:38.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb58f4f3740 1 -- 172.21.15.5:0/3828968686 wait complete. 2024-09-19T06:09:38.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.559+0000 7fb58f4f3740 20 client.26573 trim_cache size 0 max 16384 2024-09-19T06:09:38.566 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:09:38.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.560+0000 7fb58d11a640 1 -- 172.21.15.5:0/3123186628 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5700a33b0 msgr2=0x7fb5700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:38.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.560+0000 7fb58d11a640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5700a33b0 0x7fb5700a5870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55e65bf6dac0 tx=0x7fb578025b40 comp rx=0 tx=0).stop 2024-09-19T06:09:38.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.560+0000 7fb58d11a640 1 -- 172.21.15.5:0/3123186628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588102d40 msgr2=0x7fb5881009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:38.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.560+0000 7fb58d11a640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588102d40 0x7fb5881009a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb57800e9f0 tx=0x7fb57800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.560+0000 7fb58d11a640 1 -- 172.21.15.5:0/3123186628 shutdown_connections 2024-09-19T06:09:38.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.560+0000 7fb58d11a640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58810c170 0x7fb588101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.560+0000 7fb58d11a640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5700a33b0 0x7fb5700a5870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.560+0000 7fb58d11a640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb588103660 0x7fb588100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.560+0000 7fb58d11a640 1 --2- 172.21.15.5:0/3123186628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588102d40 0x7fb5881009a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:38.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.560+0000 7fb58d11a640 1 -- 172.21.15.5:0/3123186628 >> 172.21.15.5:0/3123186628 conn(0x7fb5880fc7a0 msgr2=0x7fb5880fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:38.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.560+0000 7fb58d11a640 1 -- 172.21.15.5:0/3123186628 shutdown_connections 2024-09-19T06:09:38.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:38.560+0000 7fb58d11a640 1 -- 172.21.15.5:0/3123186628 wait complete. 2024-09-19T06:09:38.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 156302 2.77 MB/sec execute 184 sec latency 15.113 ms 2024-09-19T06:09:39.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:09:39.561 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:09:39.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 156561 2.75 MB/sec execute 185 sec latency 12.852 ms 2024-09-19T06:09:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.108+0000 7fa194ecb640 1 -- 172.21.15.5:0/3397356914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa190102570 msgr2=0x7fa1901029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.108+0000 7fa194ecb640 1 --2- 172.21.15.5:0/3397356914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa190102570 0x7fa1901029b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa178009900 tx=0x7fa17802a480 comp rx=0 tx=0).stop 2024-09-19T06:09:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.108+0000 7fa194ecb640 1 -- 172.21.15.5:0/3397356914 shutdown_connections 2024-09-19T06:09:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.108+0000 7fa194ecb640 1 --2- 172.21.15.5:0/3397356914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa190102ef0 0x7fa19010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.108+0000 7fa194ecb640 1 --2- 172.21.15.5:0/3397356914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa190102570 0x7fa1901029b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.108+0000 7fa194ecb640 1 --2- 172.21.15.5:0/3397356914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa190100e20 0x7fa190101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.108+0000 7fa194ecb640 1 -- 172.21.15.5:0/3397356914 >> 172.21.15.5:0/3397356914 conn(0x7fa1900fc7d0 msgr2=0x7fa1900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.108+0000 7fa194ecb640 1 -- 172.21.15.5:0/3397356914 shutdown_connections 2024-09-19T06:09:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.108+0000 7fa194ecb640 1 -- 172.21.15.5:0/3397356914 wait complete. 2024-09-19T06:09:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa194ecb640 1 Processor -- start 2024-09-19T06:09:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa194ecb640 1 -- start start 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa194ecb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa190100e20 0x7fa1900fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa194ecb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa190102570 0x7fa1900fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa194ecb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa190102ef0 0x7fa1900fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa194ecb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa190110e30 con 0x7fa190100e20 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa194ecb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa190110cb0 con 0x7fa190102570 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa194ecb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa190110fb0 con 0x7fa190102ef0 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa18e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa190100e20 0x7fa1900fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa18e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa190100e20 0x7fa1900fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42478/0 (socket says 172.21.15.5:42478) 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa18e59c640 1 -- 172.21.15.5:0/1534318996 learned_addr learned my addr 172.21.15.5:0/1534318996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa187fff640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa190102570 0x7fa1900fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa18ed9d640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa190102ef0 0x7fa1900fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa18e59c640 1 -- 172.21.15.5:0/1534318996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa190102ef0 msgr2=0x7fa1900fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa18e59c640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa190102ef0 0x7fa1900fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa18e59c640 1 -- 172.21.15.5:0/1534318996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa190102570 msgr2=0x7fa1900fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa18e59c640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa190102570 0x7fa1900fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.109+0000 7fa18e59c640 1 -- 172.21.15.5:0/1534318996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1901a69a0 con 0x7fa190100e20 2024-09-19T06:09:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.110+0000 7fa18e59c640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa190100e20 0x7fa1900fcc80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa18000e9f0 tx=0x7fa18000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.110+0000 7fa1877fe640 1 -- 172.21.15.5:0/1534318996 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18000cd80 con 0x7fa190100e20 2024-09-19T06:09:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.110+0000 7fa194ecb640 1 -- 172.21.15.5:0/1534318996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1901a6c90 con 0x7fa190100e20 2024-09-19T06:09:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.110+0000 7fa1877fe640 1 -- 172.21.15.5:0/1534318996 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1800054d0 con 0x7fa190100e20 2024-09-19T06:09:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.110+0000 7fa1877fe640 1 -- 172.21.15.5:0/1534318996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180010490 con 0x7fa190100e20 2024-09-19T06:09:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.110+0000 7fa194ecb640 1 -- 172.21.15.5:0/1534318996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1901a71a0 con 0x7fa190100e20 2024-09-19T06:09:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.112+0000 7fa1877fe640 1 -- 172.21.15.5:0/1534318996 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa180005020 con 0x7fa190100e20 2024-09-19T06:09:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.113+0000 7fa1972a4740 1 Processor -- start 2024-09-19T06:09:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.113+0000 7fa1972a4740 1 -- start start 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa1972a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 0x5603e0a78090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa1972a4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a78a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa1972a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a83370 0x5603e0a85760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa1972a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5603e09de2f0 con 0x5603e0a83370 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa1972a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5603e0985cf0 con 0x5603e0a79b40 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa1972a4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5603e09da2b0 con 0x5603e0a785d0 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa1877fe640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa17c0a33b0 0x7fa17c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa18ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 0x5603e0a78090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa18e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a78a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa18ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 0x5603e0a78090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41740/0 (socket says 172.21.15.5:41740) 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa18ed9d640 1 -- 172.21.15.5:0/3004662016 learned_addr learned my addr 172.21.15.5:0/3004662016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa187fff640 1 --2- 172.21.15.5:0/3004662016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a83370 0x5603e0a85760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa18ed9d640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa17c0a33b0 0x7fa17c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa18e59c640 1 -- 172.21.15.5:0/3004662016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 msgr2=0x5603e0a78090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa18e59c640 1 --2- 172.21.15.5:0/3004662016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 0x5603e0a78090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa18e59c640 1 -- 172.21.15.5:0/3004662016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a83370 msgr2=0x5603e0a85760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa18e59c640 1 --2- 172.21.15.5:0/3004662016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a83370 0x5603e0a85760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa18e59c640 1 -- 172.21.15.5:0/3004662016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e096a3a0 con 0x5603e0a785d0 2024-09-19T06:09:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa18ed9d640 1 --2- 172.21.15.5:0/3004662016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 0x5603e0a78090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:09:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.114+0000 7fa1877fe640 1 -- 172.21.15.5:0/1534318996 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa1800d0070 con 0x7fa190100e20 2024-09-19T06:09:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.115+0000 7fa18e59c640 1 --2- 172.21.15.5:0/3004662016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a78a10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1800cd7d0 tx=0x7fa18009bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3f80bdc93c89cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.115+0000 7fa184ff9640 1 -- 172.21.15.5:0/3004662016 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180005680 con 0x5603e0a785d0 2024-09-19T06:09:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.115+0000 7fa184ff9640 1 -- 172.21.15.5:0/3004662016 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa180019e50 con 0x5603e0a785d0 2024-09-19T06:09:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.115+0000 7fa184ff9640 1 -- 172.21.15.5:0/3004662016 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1800ccc90 con 0x5603e0a785d0 2024-09-19T06:09:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.116+0000 7fa1972a4740 1 -- 172.21.15.5:0/3004662016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 msgr2=0x5603e0a78a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.116+0000 7fa1972a4740 1 --2- 172.21.15.5:0/3004662016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a78a10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1800cd7d0 tx=0x7fa18009bdf0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.116+0000 7fa18ed9d640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa17c0a33b0 0x7fa17c0a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa188013840 tx=0x7fa188004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.116+0000 7fa1972a4740 1 -- 172.21.15.5:0/3004662016 shutdown_connections 2024-09-19T06:09:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.116+0000 7fa1972a4740 1 --2- 172.21.15.5:0/3004662016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a83370 0x5603e0a85760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.116+0000 7fa1972a4740 1 --2- 172.21.15.5:0/3004662016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a78a10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.116+0000 7fa1972a4740 1 --2- 172.21.15.5:0/3004662016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 0x5603e0a78090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.116+0000 7fa1972a4740 1 -- 172.21.15.5:0/3004662016 >> 172.21.15.5:0/3004662016 conn(0x5603e0a41310 msgr2=0x5603e0a74c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.116+0000 7fa1972a4740 1 -- 172.21.15.5:0/3004662016 shutdown_connections 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.116+0000 7fa1972a4740 1 -- 172.21.15.5:0/3004662016 wait complete. 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.116+0000 7fa1972a4740 1 -- start start 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.116+0000 7fa1972a4740 1 Processor -- start 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa1972a4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a82d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa1972a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a79b40 0x5603e0a80020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa1972a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a83370 0x5603e0a80610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa1972a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5603e0a86010 con 0x5603e0a79b40 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa1972a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5603e0999060 con 0x5603e0a83370 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa1972a4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5603e0a86190 con 0x5603e0a785d0 2024-09-19T06:09:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa1972a4740 10 client.0 _ll_register_callbacks cb 0x5603e06ec8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa187fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a79b40 0x5603e0a80020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa187fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a79b40 0x5603e0a80020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42500/0 (socket says 172.21.15.5:42500) 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa18ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a83370 0x5603e0a80610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa187fff640 1 -- 172.21.15.5:0/329050649 learned_addr learned my addr 172.21.15.5:0/329050649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa18e59c640 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a82d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa187fff640 1 -- 172.21.15.5:0/329050649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 msgr2=0x5603e0a82d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa187fff640 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a82d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa187fff640 1 -- 172.21.15.5:0/329050649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a83370 msgr2=0x5603e0a80610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa187fff640 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a83370 0x5603e0a80610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa187fff640 1 -- 172.21.15.5:0/329050649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e074f3c0 con 0x5603e0a79b40 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa18e59c640 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a82d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa18ed9d640 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a83370 0x5603e0a80610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa187fff640 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a79b40 0x5603e0a80020 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa17802a9a0 tx=0x7fa178031e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08146fd0cb5b48e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.117+0000 7fa175ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:09:40.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.118+0000 7fa175ffb640 1 -- 172.21.15.5:0/329050649 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178035040 con 0x5603e0a79b40 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.118+0000 7fa175ffb640 1 -- 172.21.15.5:0/329050649 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa178031630 con 0x5603e0a79b40 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.118+0000 7fa1972a4740 1 -- 172.21.15.5:0/329050649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa180019e50 con 0x5603e0a79b40 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.118+0000 7fa175ffb640 1 -- 172.21.15.5:0/329050649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa178031830 con 0x5603e0a79b40 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.118+0000 7fa1972a4740 10 client.17001 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.118+0000 7fa1972a4740 1 -- 172.21.15.5:0/329050649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5603e0a91b00 con 0x5603e0a79b40 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.118+0000 7fa175ffb640 1 -- 172.21.15.5:0/329050649 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa180019e50 con 0x5603e0a79b40 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.118+0000 7fa175ffb640 1 -- 172.21.15.5:0/329050649 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa1780047a0 con 0x5603e0a79b40 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.118+0000 7fa1972a4740 10 client.17001 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.118+0000 7fa1972a4740 10 client.17001 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.118+0000 7fa1972a4740 20 client.17001 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.119+0000 7fa1972a4740 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603e0a92f10 0x5603e0a95370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.119+0000 7fa1972a4740 4 client.17001 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.119+0000 7fa1972a4740 1 -- 172.21.15.5:0/329050649 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603e09eb860 con 0x5603e0a92f10 2024-09-19T06:09:40.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.119+0000 7fa18e59c640 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603e0a92f10 0x5603e0a95370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.119+0000 7fa175ffb640 10 client.17001 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.119+0000 7fa18e59c640 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603e0a92f10 0x5603e0a95370 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f0191053ba9d3b0d server_cookie=af614b927bce90ef in_seq=0 out_seq=0 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa175ffb640 1 -- 172.21.15.5:0/329050649 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603e09eb860 con 0x5603e0a92f10 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa175ffb640 10 client.17001 handle_command_reply: tid=0 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa1972a4740 1 client.17001 shutdown 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa1972a4740 1 -- 172.21.15.5:0/329050649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a79b40 msgr2=0x5603e0a80020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa1972a4740 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a79b40 0x5603e0a80020 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa17802a9a0 tx=0x7fa178031e10 comp rx=0 tx=0).stop 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa1972a4740 1 -- 172.21.15.5:0/329050649 shutdown_connections 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa1972a4740 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603e0a92f10 0x5603e0a95370 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa1972a4740 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a83370 0x5603e0a80610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa1972a4740 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a79b40 0x5603e0a80020 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa1972a4740 1 --2- 172.21.15.5:0/329050649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a82d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa1972a4740 1 -- 172.21.15.5:0/329050649 >> 172.21.15.5:0/329050649 conn(0x5603e0a41310 msgr2=0x5603e0a7a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa175ffb640 0 client.17001 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:09:40.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa1972a4740 1 -- 172.21.15.5:0/329050649 shutdown_connections 2024-09-19T06:09:40.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa1972a4740 1 -- 172.21.15.5:0/329050649 wait complete. 2024-09-19T06:09:40.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.120+0000 7fa1972a4740 20 client.17001 trim_cache size 0 max 16384 2024-09-19T06:09:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa1972a4740 1 Processor -- start 2024-09-19T06:09:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa1972a4740 1 -- start start 2024-09-19T06:09:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa1972a4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a74e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa1972a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 0x5603e0a8f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa1972a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a83370 0x5603e0a7f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa1972a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901af8d0 con 0x5603e0a83370 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa1972a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901b1410 con 0x5603e0a79b40 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa1972a4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901affa0 con 0x5603e0a785d0 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa18ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a83370 0x5603e0a7f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa18e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a74e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa187fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 0x5603e0a8f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa187fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 0x5603e0a8f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41758/0 (socket says 172.21.15.5:41758) 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa18ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a83370 0x5603e0a7f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42506/0 (socket says 172.21.15.5:42506) 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa18e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a74e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50854/0 (socket says 172.21.15.5:50854) 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.134+0000 7fa187fff640 1 -- 172.21.15.5:0/3658547902 learned_addr learned my addr 172.21.15.5:0/3658547902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.135+0000 7fa187fff640 1 -- 172.21.15.5:0/3658547902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 msgr2=0x5603e0a74e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.135+0000 7fa187fff640 1 --2- 172.21.15.5:0/3658547902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a74e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.135+0000 7fa187fff640 1 -- 172.21.15.5:0/3658547902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a83370 msgr2=0x5603e0a7f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.135+0000 7fa187fff640 1 --2- 172.21.15.5:0/3658547902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a83370 0x5603e0a7f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.135+0000 7fa187fff640 1 -- 172.21.15.5:0/3658547902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e0a7eff0 con 0x5603e0a79b40 2024-09-19T06:09:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.135+0000 7fa187fff640 1 --2- 172.21.15.5:0/3658547902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 0x5603e0a8f2b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa178004ce0 tx=0x7fa178005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9caee8424b6b96fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.135+0000 7fa177fff640 1 -- 172.21.15.5:0/3658547902 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178035070 con 0x5603e0a79b40 2024-09-19T06:09:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.135+0000 7fa177fff640 1 -- 172.21.15.5:0/3658547902 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa178031630 con 0x5603e0a79b40 2024-09-19T06:09:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.135+0000 7fa177fff640 1 -- 172.21.15.5:0/3658547902 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178046480 con 0x5603e0a79b40 2024-09-19T06:09:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.136+0000 7fa1972a4740 1 -- 172.21.15.5:0/3658547902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 msgr2=0x5603e0a8f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.136+0000 7fa1972a4740 1 --2- 172.21.15.5:0/3658547902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 0x5603e0a8f2b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa178004ce0 tx=0x7fa178005d50 comp rx=0 tx=0).stop 2024-09-19T06:09:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.136+0000 7fa1972a4740 1 -- 172.21.15.5:0/3658547902 shutdown_connections 2024-09-19T06:09:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.136+0000 7fa1972a4740 1 --2- 172.21.15.5:0/3658547902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a83370 0x5603e0a7f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.136+0000 7fa1972a4740 1 --2- 172.21.15.5:0/3658547902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a79b40 0x5603e0a8f2b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.136+0000 7fa1972a4740 1 --2- 172.21.15.5:0/3658547902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a785d0 0x5603e0a74e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.136+0000 7fa1972a4740 1 -- 172.21.15.5:0/3658547902 >> 172.21.15.5:0/3658547902 conn(0x5603e0a41310 msgr2=0x5603e0a74700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.136+0000 7fa1972a4740 1 -- 172.21.15.5:0/3658547902 shutdown_connections 2024-09-19T06:09:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.136+0000 7fa1972a4740 1 -- 172.21.15.5:0/3658547902 wait complete. 2024-09-19T06:09:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.136+0000 7fa1972a4740 1 -- start start 2024-09-19T06:09:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.136+0000 7fa1972a4740 1 Processor -- start 2024-09-19T06:09:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa1972a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a785d0 0x5603e0a8b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa1972a4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a79b40 0x5603e0a8bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa1972a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a83370 0x5603e0a7c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa1972a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901b0d40 con 0x5603e0a785d0 2024-09-19T06:09:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa1972a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901b0670 con 0x5603e0a83370 2024-09-19T06:09:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa1972a4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901ad8b0 con 0x5603e0a79b40 2024-09-19T06:09:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa1972a4740 10 client.0 _ll_register_callbacks cb 0x7fa178009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa18e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a785d0 0x5603e0a8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa18e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a785d0 0x5603e0a8b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42510/0 (socket says 172.21.15.5:42510) 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa18e59c640 1 -- 172.21.15.5:0/4273185125 learned_addr learned my addr 172.21.15.5:0/4273185125 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa187fff640 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a79b40 0x5603e0a8bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa18ed9d640 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a83370 0x5603e0a7c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa187fff640 1 -- 172.21.15.5:0/4273185125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a83370 msgr2=0x5603e0a7c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa187fff640 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a83370 0x5603e0a7c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa187fff640 1 -- 172.21.15.5:0/4273185125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a785d0 msgr2=0x5603e0a8b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa187fff640 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a785d0 0x5603e0a8b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa187fff640 1 -- 172.21.15.5:0/4273185125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e0a7d080 con 0x5603e0a79b40 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.137+0000 7fa18e59c640 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a785d0 0x5603e0a8b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa187fff640 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a79b40 0x5603e0a8bb40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa178004ce0 tx=0x7fa1780058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0b0cca3cbd679f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa175ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa175ffb640 1 -- 172.21.15.5:0/4273185125 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178035070 con 0x5603e0a79b40 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa175ffb640 1 -- 172.21.15.5:0/4273185125 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fa1780048c0 con 0x5603e0a79b40 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa1972a4740 1 -- 172.21.15.5:0/4273185125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa178031630 con 0x5603e0a79b40 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa175ffb640 1 -- 172.21.15.5:0/4273185125 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa178067050 con 0x5603e0a79b40 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa1972a4740 10 client.26500 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa1972a4740 1 -- 172.21.15.5:0/4273185125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5603e0afd750 con 0x5603e0a79b40 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa175ffb640 1 -- 172.21.15.5:0/4273185125 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa178031630 con 0x5603e0a79b40 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa175ffb640 1 -- 172.21.15.5:0/4273185125 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa17805f020 con 0x5603e0a79b40 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa1972a4740 10 client.26500 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa1972a4740 10 client.26500 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa1972a4740 20 client.26500 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa1972a4740 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603e0afdd50 0x5603e0b00140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.138+0000 7fa1972a4740 4 client.26500 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:09:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.139+0000 7fa1972a4740 1 -- 172.21.15.5:0/4273185125 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603e09e5d70 con 0x5603e0afdd50 2024-09-19T06:09:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.139+0000 7fa18e59c640 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603e0afdd50 0x5603e0b00140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.139+0000 7fa175ffb640 10 client.26500 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:09:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.139+0000 7fa18e59c640 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603e0afdd50 0x5603e0b00140 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3e239d822fe61243 server_cookie=455c53304b42e99e in_seq=0 out_seq=0 2024-09-19T06:09:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.139+0000 7fa175ffb640 1 -- 172.21.15.5:0/4273185125 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603e09e5d70 con 0x5603e0afdd50 2024-09-19T06:09:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.139+0000 7fa175ffb640 10 client.26500 handle_command_reply: tid=0 2024-09-19T06:09:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.139+0000 7fa1972a4740 1 client.26500 shutdown 2024-09-19T06:09:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.139+0000 7fa1972a4740 1 -- 172.21.15.5:0/4273185125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a79b40 msgr2=0x5603e0a8bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.139+0000 7fa1972a4740 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a79b40 0x5603e0a8bb40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa178004ce0 tx=0x7fa1780058d0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa1972a4740 1 -- 172.21.15.5:0/4273185125 shutdown_connections 2024-09-19T06:09:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa1972a4740 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603e0afdd50 0x5603e0b00140 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa1972a4740 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603e0a83370 0x5603e0a7c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa1972a4740 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603e0a79b40 0x5603e0a8bb40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa175ffb640 0 client.26500 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:09:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa1972a4740 1 --2- 172.21.15.5:0/4273185125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603e0a785d0 0x5603e0a8b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa1972a4740 1 -- 172.21.15.5:0/4273185125 >> 172.21.15.5:0/4273185125 conn(0x5603e0a41310 msgr2=0x5603e0a734e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa1972a4740 1 -- 172.21.15.5:0/4273185125 shutdown_connections 2024-09-19T06:09:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa1972a4740 1 -- 172.21.15.5:0/4273185125 wait complete. 2024-09-19T06:09:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa1972a4740 20 client.26500 trim_cache size 0 max 16384 2024-09-19T06:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:09:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa194ecb640 1 -- 172.21.15.5:0/1534318996 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa17c0a33b0 msgr2=0x7fa17c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa194ecb640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa17c0a33b0 0x7fa17c0a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa188013840 tx=0x7fa188004ab0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa194ecb640 1 -- 172.21.15.5:0/1534318996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa190100e20 msgr2=0x7fa1900fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa194ecb640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa190100e20 0x7fa1900fcc80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa18000e9f0 tx=0x7fa18000edf0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa194ecb640 1 -- 172.21.15.5:0/1534318996 shutdown_connections 2024-09-19T06:09:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa194ecb640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa17c0a33b0 0x7fa17c0a5870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa194ecb640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa190102ef0 0x7fa1900fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.140+0000 7fa194ecb640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa190102570 0x7fa1900fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.141+0000 7fa194ecb640 1 --2- 172.21.15.5:0/1534318996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa190100e20 0x7fa1900fcc80 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.141+0000 7fa194ecb640 1 -- 172.21.15.5:0/1534318996 >> 172.21.15.5:0/1534318996 conn(0x7fa1900fc7d0 msgr2=0x7fa19006a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.141+0000 7fa194ecb640 1 -- 172.21.15.5:0/1534318996 shutdown_connections 2024-09-19T06:09:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:40.141+0000 7fa194ecb640 1 -- 172.21.15.5:0/1534318996 wait complete. 2024-09-19T06:09:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.240+0000 7f668d05e640 1 -- 172.21.15.5:0/960974774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668810c0a0 msgr2=0x7f668810e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.240+0000 7f668d05e640 1 --2- 172.21.15.5:0/960974774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668810c0a0 0x7f668810e540 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6680009ca0 tx=0x7f668002a950 comp rx=0 tx=0).stop 2024-09-19T06:09:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.240+0000 7f668d05e640 1 -- 172.21.15.5:0/960974774 shutdown_connections 2024-09-19T06:09:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.240+0000 7f668d05e640 1 --2- 172.21.15.5:0/960974774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668810c0a0 0x7f668810e540 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.240+0000 7f668d05e640 1 --2- 172.21.15.5:0/960974774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66881035a0 0x7f668810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.240+0000 7f668d05e640 1 --2- 172.21.15.5:0/960974774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6688102c80 0x7f6688103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.240+0000 7f668d05e640 1 -- 172.21.15.5:0/960974774 >> 172.21.15.5:0/960974774 conn(0x7f66880779a0 msgr2=0x7f66880fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.240+0000 7f668d05e640 1 -- 172.21.15.5:0/960974774 shutdown_connections 2024-09-19T06:09:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.240+0000 7f668d05e640 1 -- 172.21.15.5:0/960974774 wait complete. 2024-09-19T06:09:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f668d05e640 1 Processor -- start 2024-09-19T06:09:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f668d05e640 1 -- start start 2024-09-19T06:09:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f668d05e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6688102c80 0x7f66881982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f668d05e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66881035a0 0x7f6688198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f668d05e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668810c0a0 0x7f668819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f668d05e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6688111d50 con 0x7f668810c0a0 2024-09-19T06:09:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f668d05e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6688111bd0 con 0x7f6688102c80 2024-09-19T06:09:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f668d05e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6688111ed0 con 0x7f66881035a0 2024-09-19T06:09:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f668759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668810c0a0 0x7f668819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f6686d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6688102c80 0x7f66881982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f668759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668810c0a0 0x7f668819cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42536/0 (socket says 172.21.15.5:42536) 2024-09-19T06:09:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f668659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66881035a0 0x7f6688198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.241+0000 7f668759e640 1 -- 172.21.15.5:0/897456387 learned_addr learned my addr 172.21.15.5:0/897456387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f6686d9d640 1 -- 172.21.15.5:0/897456387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66881035a0 msgr2=0x7f6688198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f6686d9d640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66881035a0 0x7f6688198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f6686d9d640 1 -- 172.21.15.5:0/897456387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668810c0a0 msgr2=0x7f668819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f6686d9d640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668810c0a0 0x7f668819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f6686d9d640 1 -- 172.21.15.5:0/897456387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668819d2b0 con 0x7f6688102c80 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f668759e640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668810c0a0 0x7f668819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f668659c640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66881035a0 0x7f6688198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f6686d9d640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6688102c80 0x7f66881982c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f667800e9f0 tx=0x7f667800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f666ffff640 1 -- 172.21.15.5:0/897456387 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667800ccc0 con 0x7f6688102c80 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f668d05e640 1 -- 172.21.15.5:0/897456387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668819d5a0 con 0x7f6688102c80 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f666ffff640 1 -- 172.21.15.5:0/897456387 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6678005020 con 0x7f6688102c80 2024-09-19T06:09:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f666ffff640 1 -- 172.21.15.5:0/897456387 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678004a00 con 0x7f6688102c80 2024-09-19T06:09:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.242+0000 7f668d05e640 1 -- 172.21.15.5:0/897456387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66881a4e30 con 0x7f6688102c80 2024-09-19T06:09:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.244+0000 7f666ffff640 1 -- 172.21.15.5:0/897456387 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6678004ba0 con 0x7f6688102c80 2024-09-19T06:09:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f668fdfe740 1 Processor -- start 2024-09-19T06:09:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f668fdfe740 1 -- start start 2024-09-19T06:09:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f668fdfe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36daf0 0x55bacd36ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f668fdfe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 0x55bacd378770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f668fdfe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd378cb0 0x55bacd37b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f668fdfe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bacd2d22f0 con 0x55bacd36e410 2024-09-19T06:09:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f668fdfe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bacd27a270 con 0x55bacd36daf0 2024-09-19T06:09:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f668fdfe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bacd2ce2b0 con 0x55bacd378cb0 2024-09-19T06:09:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f6686d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 0x55bacd378770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f668759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36daf0 0x55bacd36ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f6686d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 0x55bacd378770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42542/0 (socket says 172.21.15.5:42542) 2024-09-19T06:09:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f6686d9d640 1 -- 172.21.15.5:0/3504779407 learned_addr learned my addr 172.21.15.5:0/3504779407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f668659c640 1 --2- 172.21.15.5:0/3504779407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd378cb0 0x55bacd37b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f666ffff640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f66700a33b0 0x7f66700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.245+0000 7f668759e640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f66700a33b0 0x7f66700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f666ffff640 1 -- 172.21.15.5:0/897456387 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f66780cfc70 con 0x7f6688102c80 2024-09-19T06:09:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f668759e640 1 -- 172.21.15.5:0/3504779407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd378cb0 msgr2=0x55bacd37b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f668759e640 1 --2- 172.21.15.5:0/3504779407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd378cb0 0x55bacd37b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f668759e640 1 -- 172.21.15.5:0/3504779407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 msgr2=0x55bacd378770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f668759e640 1 --2- 172.21.15.5:0/3504779407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 0x55bacd378770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f668759e640 1 -- 172.21.15.5:0/3504779407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacd279340 con 0x55bacd36daf0 2024-09-19T06:09:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f668659c640 1 --2- 172.21.15.5:0/3504779407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd378cb0 0x55bacd37b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f6686d9d640 1 --2- 172.21.15.5:0/3504779407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 0x55bacd378770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f668759e640 1 --2- 172.21.15.5:0/3504779407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36daf0 0x55bacd36ded0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f668002ae70 tx=0x7f668002f530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1921e4789bb65169 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f668759e640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f66700a33b0 0x7f66700a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6680048000 tx=0x7f668004a000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f666d7fa640 1 -- 172.21.15.5:0/3504779407 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680043050 con 0x55bacd36daf0 2024-09-19T06:09:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f666d7fa640 1 -- 172.21.15.5:0/3504779407 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f668004d020 con 0x55bacd36daf0 2024-09-19T06:09:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.246+0000 7f666d7fa640 1 -- 172.21.15.5:0/3504779407 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680034040 con 0x55bacd36daf0 2024-09-19T06:09:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.247+0000 7f668fdfe740 1 -- 172.21.15.5:0/3504779407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36daf0 msgr2=0x55bacd36ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.247+0000 7f668fdfe740 1 --2- 172.21.15.5:0/3504779407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36daf0 0x55bacd36ded0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f668002ae70 tx=0x7f668002f530 comp rx=0 tx=0).stop 2024-09-19T06:09:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.247+0000 7f668fdfe740 1 -- 172.21.15.5:0/3504779407 shutdown_connections 2024-09-19T06:09:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.247+0000 7f668fdfe740 1 --2- 172.21.15.5:0/3504779407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd378cb0 0x55bacd37b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.247+0000 7f668fdfe740 1 --2- 172.21.15.5:0/3504779407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 0x55bacd378770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.247+0000 7f668fdfe740 1 --2- 172.21.15.5:0/3504779407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36daf0 0x55bacd36ded0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.247+0000 7f668fdfe740 1 -- 172.21.15.5:0/3504779407 >> 172.21.15.5:0/3504779407 conn(0x55bacd36a060 msgr2=0x55bacd36a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.247+0000 7f668fdfe740 1 -- 172.21.15.5:0/3504779407 shutdown_connections 2024-09-19T06:09:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.247+0000 7f668fdfe740 1 -- 172.21.15.5:0/3504779407 wait complete. 2024-09-19T06:09:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.247+0000 7f668fdfe740 1 -- start start 2024-09-19T06:09:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.247+0000 7f668fdfe740 1 Processor -- start 2024-09-19T06:09:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668fdfe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd375bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668fdfe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36e410 0x55bacd376110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668fdfe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 0x55bacd376650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668fdfe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bacd37b880 con 0x55bacd378cb0 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668fdfe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bacd25b290 con 0x55bacd36e410 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668fdfe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bacd37ba00 con 0x55bacd36daf0 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668fdfe740 10 client.0 _ll_register_callbacks cb 0x55bacd28c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 0x55bacd376650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36e410 0x55bacd376110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36e410 0x55bacd376110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41798/0 (socket says 172.21.15.5:41798) 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 0x55bacd376650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42552/0 (socket says 172.21.15.5:42552) 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668659c640 1 -- 172.21.15.5:0/2204029482 learned_addr learned my addr 172.21.15.5:0/2204029482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f6686d9d640 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd375bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668659c640 1 -- 172.21.15.5:0/2204029482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 msgr2=0x55bacd375bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668659c640 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd375bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668659c640 1 -- 172.21.15.5:0/2204029482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 msgr2=0x55bacd376650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668659c640 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 0x55bacd376650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668659c640 1 -- 172.21.15.5:0/2204029482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baccfe5210 con 0x55bacd36e410 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668759e640 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 0x55bacd376650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f668659c640 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36e410 0x55bacd376110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f667c012d20 tx=0x7f667c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a7f3193bf36260 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.248+0000 7f666aff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f666aff5640 1 -- 172.21.15.5:0/2204029482 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c0120d0 con 0x55bacd36e410 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f666aff5640 1 -- 172.21.15.5:0/2204029482 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f667c004030 con 0x55bacd36e410 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f668fdfe740 1 -- 172.21.15.5:0/2204029482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f668004d020 con 0x55bacd36e410 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f666aff5640 1 -- 172.21.15.5:0/2204029482 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f667c012270 con 0x55bacd36e410 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f668fdfe740 10 client.26603 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f668fdfe740 1 -- 172.21.15.5:0/2204029482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bacd0b1620 con 0x55bacd36e410 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f666aff5640 1 -- 172.21.15.5:0/2204029482 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f668004d020 con 0x55bacd36e410 2024-09-19T06:09:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f666aff5640 1 -- 172.21.15.5:0/2204029482 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f667c01c460 con 0x55bacd36e410 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f668fdfe740 10 client.26603 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f668fdfe740 10 client.26603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f668fdfe740 20 client.26603 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f668fdfe740 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bacd388d40 0x55bacd38b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f668fdfe740 4 client.26603 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f668fdfe740 1 -- 172.21.15.5:0/2204029482 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bacd2dfa20 con 0x55bacd388d40 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.249+0000 7f6686d9d640 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bacd388d40 0x55bacd38b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.250+0000 7f6686d9d640 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bacd388d40 0x55bacd38b130 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3db42b13882030ba server_cookie=7bca6a795d00bad8 in_seq=0 out_seq=0 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.250+0000 7f666aff5640 10 client.26603 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.250+0000 7f666aff5640 1 -- 172.21.15.5:0/2204029482 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bacd2dfa20 con 0x55bacd388d40 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.250+0000 7f666aff5640 10 client.26603 handle_command_reply: tid=0 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.250+0000 7f668fdfe740 1 client.26603 shutdown 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.250+0000 7f668fdfe740 1 -- 172.21.15.5:0/2204029482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36e410 msgr2=0x55bacd376110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.251+0000 7f668fdfe740 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36e410 0x55bacd376110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f667c012d20 tx=0x7f667c00f790 comp rx=0 tx=0).stop 2024-09-19T06:09:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.251+0000 7f668fdfe740 1 -- 172.21.15.5:0/2204029482 shutdown_connections 2024-09-19T06:09:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.251+0000 7f668fdfe740 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bacd388d40 0x55bacd38b130 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.251+0000 7f668fdfe740 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 0x55bacd376650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.251+0000 7f668fdfe740 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36e410 0x55bacd376110 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.251+0000 7f666aff5640 0 client.26603 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.251+0000 7f668fdfe740 1 --2- 172.21.15.5:0/2204029482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd375bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.251+0000 7f668fdfe740 1 -- 172.21.15.5:0/2204029482 >> 172.21.15.5:0/2204029482 conn(0x55bacd36a060 msgr2=0x55bacd36fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.251+0000 7f668fdfe740 1 -- 172.21.15.5:0/2204029482 shutdown_connections 2024-09-19T06:09:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.251+0000 7f668fdfe740 1 -- 172.21.15.5:0/2204029482 wait complete. 2024-09-19T06:09:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.251+0000 7f668fdfe740 20 client.26603 trim_cache size 0 max 16384 2024-09-19T06:09:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668fdfe740 1 Processor -- start 2024-09-19T06:09:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668fdfe740 1 -- start start 2024-09-19T06:09:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668fdfe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd376480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668fdfe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 0x55bacd3909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668fdfe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd378cb0 0x55bacd390f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668fdfe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66881ad750 con 0x55bacd36e410 2024-09-19T06:09:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668fdfe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f66881af290 con 0x55bacd378cb0 2024-09-19T06:09:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668fdfe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f66881ade20 con 0x55bacd36daf0 2024-09-19T06:09:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 0x55bacd3909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 0x55bacd3909f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42564/0 (socket says 172.21.15.5:42564) 2024-09-19T06:09:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668659c640 1 -- 172.21.15.5:0/2878337303 learned_addr learned my addr 172.21.15.5:0/2878337303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f6686d9d640 1 --2- 172.21.15.5:0/2878337303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd376480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668759e640 1 --2- 172.21.15.5:0/2878337303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd378cb0 0x55bacd390f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668659c640 1 -- 172.21.15.5:0/2878337303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 msgr2=0x55bacd376480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668659c640 1 --2- 172.21.15.5:0/2878337303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd376480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668659c640 1 -- 172.21.15.5:0/2878337303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd378cb0 msgr2=0x55bacd390f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668659c640 1 --2- 172.21.15.5:0/2878337303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd378cb0 0x55bacd390f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668659c640 1 -- 172.21.15.5:0/2878337303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacd36acf0 con 0x55bacd36e410 2024-09-19T06:09:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f6686d9d640 1 --2- 172.21.15.5:0/2878337303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd376480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.267+0000 7f668659c640 1 --2- 172.21.15.5:0/2878337303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 0x55bacd3909f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f667c01ca70 tx=0x7f667c005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fd9a084b21edba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.268+0000 7f666cff9640 1 -- 172.21.15.5:0/2878337303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c029030 con 0x55bacd36e410 2024-09-19T06:09:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.268+0000 7f666cff9640 1 -- 172.21.15.5:0/2878337303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f667c00fe80 con 0x55bacd36e410 2024-09-19T06:09:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.268+0000 7f666cff9640 1 -- 172.21.15.5:0/2878337303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c005590 con 0x55bacd36e410 2024-09-19T06:09:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.268+0000 7f668fdfe740 1 -- 172.21.15.5:0/2878337303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 msgr2=0x55bacd3909f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.268+0000 7f668fdfe740 1 --2- 172.21.15.5:0/2878337303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 0x55bacd3909f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f667c01ca70 tx=0x7f667c005970 comp rx=0 tx=0).stop 2024-09-19T06:09:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.268+0000 7f668fdfe740 1 -- 172.21.15.5:0/2878337303 shutdown_connections 2024-09-19T06:09:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.268+0000 7f668fdfe740 1 --2- 172.21.15.5:0/2878337303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd378cb0 0x55bacd390f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.268+0000 7f668fdfe740 1 --2- 172.21.15.5:0/2878337303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd36e410 0x55bacd3909f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.268+0000 7f668fdfe740 1 --2- 172.21.15.5:0/2878337303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd376480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.268+0000 7f668fdfe740 1 -- 172.21.15.5:0/2878337303 >> 172.21.15.5:0/2878337303 conn(0x55bacd36a060 msgr2=0x55bacd377c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668fdfe740 1 -- 172.21.15.5:0/2878337303 shutdown_connections 2024-09-19T06:09:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668fdfe740 1 -- 172.21.15.5:0/2878337303 wait complete. 2024-09-19T06:09:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668fdfe740 1 -- start start 2024-09-19T06:09:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668fdfe740 1 Processor -- start 2024-09-19T06:09:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668fdfe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd381580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668fdfe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36e410 0x55bacd37c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668fdfe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 0x55bacd37d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668fdfe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66881aebc0 con 0x55bacd378cb0 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668fdfe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f66881ae4f0 con 0x55bacd36e410 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668fdfe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f66881ab730 con 0x55bacd36daf0 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668fdfe740 10 client.0 _ll_register_callbacks cb 0x7f667c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 0x55bacd37d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f6686d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd381580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 0x55bacd37d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42570/0 (socket says 172.21.15.5:42570) 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36e410 0x55bacd37c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.269+0000 7f668759e640 1 -- 172.21.15.5:0/3668089635 learned_addr learned my addr 172.21.15.5:0/3668089635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f6686d9d640 1 -- 172.21.15.5:0/3668089635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36e410 msgr2=0x55bacd37c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f6686d9d640 1 --2- 172.21.15.5:0/3668089635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36e410 0x55bacd37c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f6686d9d640 1 -- 172.21.15.5:0/3668089635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 msgr2=0x55bacd37d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f6686d9d640 1 --2- 172.21.15.5:0/3668089635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 0x55bacd37d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f6686d9d640 1 -- 172.21.15.5:0/3668089635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacd378760 con 0x55bacd36daf0 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f668759e640 1 --2- 172.21.15.5:0/3668089635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 0x55bacd37d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f6686d9d640 1 --2- 172.21.15.5:0/3668089635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd381580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55bacd36aec0 tx=0x7f66780054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=852b167f0af9d1da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f666aff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f666aff5640 1 -- 172.21.15.5:0/3668089635 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667801c040 con 0x55bacd36daf0 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f668fdfe740 1 -- 172.21.15.5:0/3668089635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bacd2d9d70 con 0x55bacd36daf0 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f666aff5640 1 -- 172.21.15.5:0/3668089635 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f66780051d0 con 0x55bacd36daf0 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f666aff5640 1 -- 172.21.15.5:0/3668089635 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f667809bf60 con 0x55bacd36daf0 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f668fdfe740 10 client.26518 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f668fdfe740 1 -- 172.21.15.5:0/3668089635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bacd3f3980 con 0x55bacd36daf0 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.270+0000 7f666aff5640 1 -- 172.21.15.5:0/3668089635 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6678069020 con 0x55bacd36daf0 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.271+0000 7f666aff5640 1 -- 172.21.15.5:0/3668089635 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f66780985d0 con 0x55bacd36daf0 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.271+0000 7f668fdfe740 10 client.26518 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.271+0000 7f668fdfe740 10 client.26518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.271+0000 7f668fdfe740 20 client.26518 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.271+0000 7f668fdfe740 1 --2- 172.21.15.5:0/3668089635 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bacd3f4050 0x55bacd3f6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.271+0000 7f668fdfe740 4 client.26518 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:09:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.271+0000 7f668fdfe740 1 -- 172.21.15.5:0/3668089635 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bacd256900 con 0x55bacd3f4050 2024-09-19T06:09:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.271+0000 7f668659c640 1 --2- 172.21.15.5:0/3668089635 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bacd3f4050 0x55bacd3f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.271+0000 7f668659c640 1 --2- 172.21.15.5:0/3668089635 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bacd3f4050 0x55bacd3f6440 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712c9aa80a0a68e5 server_cookie=e29854797346108a in_seq=0 out_seq=0 2024-09-19T06:09:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.271+0000 7f666aff5640 10 client.26518 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f666aff5640 1 -- 172.21.15.5:0/3668089635 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bacd256900 con 0x55bacd3f4050 2024-09-19T06:09:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f666aff5640 10 client.26518 handle_command_reply: tid=0 2024-09-19T06:09:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f668fdfe740 1 client.26518 shutdown 2024-09-19T06:09:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f668fdfe740 1 -- 172.21.15.5:0/3668089635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 msgr2=0x55bacd381580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f668fdfe740 1 --2- 172.21.15.5:0/3668089635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd381580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55bacd36aec0 tx=0x7f66780054d0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f668fdfe740 1 -- 172.21.15.5:0/3668089635 shutdown_connections 2024-09-19T06:09:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f668fdfe740 1 --2- 172.21.15.5:0/3668089635 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bacd3f4050 0x55bacd3f6440 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f668fdfe740 1 --2- 172.21.15.5:0/3668089635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd378cb0 0x55bacd37d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f666aff5640 0 client.26518 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f668fdfe740 1 --2- 172.21.15.5:0/3668089635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bacd36e410 0x55bacd37c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f668fdfe740 1 --2- 172.21.15.5:0/3668089635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bacd36daf0 0x55bacd381580 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f668fdfe740 1 -- 172.21.15.5:0/3668089635 >> 172.21.15.5:0/3668089635 conn(0x55bacd36a060 msgr2=0x55bacd3689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f668fdfe740 1 -- 172.21.15.5:0/3668089635 shutdown_connections 2024-09-19T06:09:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f668fdfe740 1 -- 172.21.15.5:0/3668089635 wait complete. 2024-09-19T06:09:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.289+0000 7f668fdfe740 20 client.26518 trim_cache size 0 max 16384 2024-09-19T06:09:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:09:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:09:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:09:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:09:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.290+0000 7f668d05e640 1 -- 172.21.15.5:0/897456387 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f66700a33b0 msgr2=0x7f66700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.290+0000 7f668d05e640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f66700a33b0 0x7f66700a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6680048000 tx=0x7f668004a000 comp rx=0 tx=0).stop 2024-09-19T06:09:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.290+0000 7f668d05e640 1 -- 172.21.15.5:0/897456387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6688102c80 msgr2=0x7f66881982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.290+0000 7f668d05e640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6688102c80 0x7f66881982c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f667800e9f0 tx=0x7f667800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.290+0000 7f668d05e640 1 -- 172.21.15.5:0/897456387 shutdown_connections 2024-09-19T06:09:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.290+0000 7f668d05e640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668810c0a0 0x7f668819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.290+0000 7f668d05e640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f66700a33b0 0x7f66700a5870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.290+0000 7f668d05e640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66881035a0 0x7f6688198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.290+0000 7f668d05e640 1 --2- 172.21.15.5:0/897456387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6688102c80 0x7f66881982c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.290+0000 7f668d05e640 1 -- 172.21.15.5:0/897456387 >> 172.21.15.5:0/897456387 conn(0x7f66880779a0 msgr2=0x7f66880feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.290+0000 7f668d05e640 1 -- 172.21.15.5:0/897456387 shutdown_connections 2024-09-19T06:09:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:40.290+0000 7f668d05e640 1 -- 172.21.15.5:0/897456387 wait complete. 2024-09-19T06:09:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:39 smithi157 ceph-mon[35346]: pgmap v193: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-09-19T06:09:40.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:39 smithi005 ceph-mon[30955]: pgmap v193: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-09-19T06:09:40.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:39 smithi049 ceph-mon[34640]: pgmap v193: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-09-19T06:09:40.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 156965 2.74 MB/sec execute 186 sec latency 46.339 ms 2024-09-19T06:09:41.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 157497 2.75 MB/sec execute 187 sec latency 21.250 ms 2024-09-19T06:09:42.062 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:09:42.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:41 smithi157 ceph-mon[35346]: pgmap v194: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-09-19T06:09:42.349 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:09:42.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:41 smithi005 ceph-mon[30955]: pgmap v194: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-09-19T06:09:42.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:41 smithi049 ceph-mon[34640]: pgmap v194: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-09-19T06:09:42.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 157925 2.74 MB/sec execute 188 sec latency 93.222 ms 2024-09-19T06:09:43.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 158429 2.73 MB/sec execute 189 sec latency 99.234 ms 2024-09-19T06:09:44.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:43 smithi157 ceph-mon[35346]: pgmap v195: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-19T06:09:44.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:43 smithi005 ceph-mon[30955]: pgmap v195: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-19T06:09:44.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:43 smithi049 ceph-mon[34640]: pgmap v195: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-19T06:09:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.578+0000 7fe5244e0640 1 -- 172.21.15.5:0/4130263086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe51c102d90 msgr2=0x7fe51c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.578+0000 7fe5244e0640 1 --2- 172.21.15.5:0/4130263086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe51c102d90 0x7fe51c103170 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe518009930 tx=0x7fe51802a420 comp rx=0 tx=0).stop 2024-09-19T06:09:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.578+0000 7fe5244e0640 1 -- 172.21.15.5:0/4130263086 shutdown_connections 2024-09-19T06:09:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.578+0000 7fe5244e0640 1 --2- 172.21.15.5:0/4130263086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe51c108290 0x7fe51c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.578+0000 7fe5244e0640 1 --2- 172.21.15.5:0/4130263086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe51c103740 0x7fe51c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.578+0000 7fe5244e0640 1 --2- 172.21.15.5:0/4130263086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe51c102d90 0x7fe51c103170 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.578+0000 7fe5244e0640 1 -- 172.21.15.5:0/4130263086 >> 172.21.15.5:0/4130263086 conn(0x7fe51c0fc7d0 msgr2=0x7fe51c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.578+0000 7fe5244e0640 1 -- 172.21.15.5:0/4130263086 shutdown_connections 2024-09-19T06:09:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.578+0000 7fe5244e0640 1 -- 172.21.15.5:0/4130263086 wait complete. 2024-09-19T06:09:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.578+0000 7fe5244e0640 1 Processor -- start 2024-09-19T06:09:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe5244e0640 1 -- start start 2024-09-19T06:09:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe5244e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe51c102d90 0x7fe51c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe5244e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe51c103740 0x7fe51c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe5244e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe51c108290 0x7fe51c101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe5244e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c111f80 con 0x7fe51c108290 2024-09-19T06:09:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe5244e0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c111e00 con 0x7fe51c102d90 2024-09-19T06:09:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe5244e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c112100 con 0x7fe51c103740 2024-09-19T06:09:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe522a7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe51c108290 0x7fe51c101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe52227c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe51c102d90 0x7fe51c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe522a7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe51c108290 0x7fe51c101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50540/0 (socket says 172.21.15.5:50540) 2024-09-19T06:09:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe52227c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe51c102d90 0x7fe51c1009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35502/0 (socket says 172.21.15.5:35502) 2024-09-19T06:09:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe522a7d640 1 -- 172.21.15.5:0/1246541820 learned_addr learned my addr 172.21.15.5:0/1246541820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.579+0000 7fe521a7b640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe51c103740 0x7fe51c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.580+0000 7fe522a7d640 1 -- 172.21.15.5:0/1246541820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe51c103740 msgr2=0x7fe51c100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.580+0000 7fe522a7d640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe51c103740 0x7fe51c100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.580+0000 7fe522a7d640 1 -- 172.21.15.5:0/1246541820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe51c102d90 msgr2=0x7fe51c1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.580+0000 7fe522a7d640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe51c102d90 0x7fe51c1009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.580+0000 7fe522a7d640 1 -- 172.21.15.5:0/1246541820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c1a69e0 con 0x7fe51c108290 2024-09-19T06:09:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.580+0000 7fe522a7d640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe51c108290 0x7fe51c101470 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe51400eb10 tx=0x7fe51400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.580+0000 7fe5137fe640 1 -- 172.21.15.5:0/1246541820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514004b20 con 0x7fe51c108290 2024-09-19T06:09:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.580+0000 7fe5244e0640 1 -- 172.21.15.5:0/1246541820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c1a6cd0 con 0x7fe51c108290 2024-09-19T06:09:44.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.580+0000 7fe5137fe640 1 -- 172.21.15.5:0/1246541820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe514004cc0 con 0x7fe51c108290 2024-09-19T06:09:44.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.580+0000 7fe5137fe640 1 -- 172.21.15.5:0/1246541820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514010690 con 0x7fe51c108290 2024-09-19T06:09:44.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.580+0000 7fe5244e0640 1 -- 172.21.15.5:0/1246541820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c1a7210 con 0x7fe51c108290 2024-09-19T06:09:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.582+0000 7fe5137fe640 1 -- 172.21.15.5:0/1246541820 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe514005020 con 0x7fe51c108290 2024-09-19T06:09:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.583+0000 7fe5268b9740 1 Processor -- start 2024-09-19T06:09:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.583+0000 7fe5268b9740 1 -- start start 2024-09-19T06:09:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.583+0000 7fe5268b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fdeb40 0x5592e7fdd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.583+0000 7fe5268b9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdd5d0 0x5592e7fdda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.583+0000 7fe5268b9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fe8370 0x5592e7fea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.583+0000 7fe5268b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5592e7f432f0 con 0x5592e7fdeb40 2024-09-19T06:09:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.583+0000 7fe5268b9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5592e7eeacf0 con 0x5592e7fe8370 2024-09-19T06:09:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.583+0000 7fe5268b9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5592e7f3f2b0 con 0x5592e7fdd5d0 2024-09-19T06:09:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe5137fe640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe5080a3430 0x7fe5080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe521a7b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fdeb40 0x5592e7fdd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe521a7b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fdeb40 0x5592e7fdd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50556/0 (socket says 172.21.15.5:50556) 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe521a7b640 1 -- 172.21.15.5:0/658831792 learned_addr learned my addr 172.21.15.5:0/658831792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe52227c640 1 --2- 172.21.15.5:0/658831792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdd5d0 0x5592e7fdda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe5137fe640 1 -- 172.21.15.5:0/1246541820 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fe5140d0100 con 0x7fe51c108290 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe522a7d640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe5080a3430 0x7fe5080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe52227c640 1 -- 172.21.15.5:0/658831792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fe8370 msgr2=0x5592e7fea760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe52227c640 1 --2- 172.21.15.5:0/658831792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fe8370 0x5592e7fea760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe52227c640 1 -- 172.21.15.5:0/658831792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fdeb40 msgr2=0x5592e7fdd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe52227c640 1 --2- 172.21.15.5:0/658831792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fdeb40 0x5592e7fdd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe52227c640 1 -- 172.21.15.5:0/658831792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e7ecf3a0 con 0x5592e7fdd5d0 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe521a7b640 1 --2- 172.21.15.5:0/658831792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fdeb40 0x5592e7fdd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe52227c640 1 --2- 172.21.15.5:0/658831792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdd5d0 0x5592e7fdda10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe51802a940 tx=0x7fe518005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53d64a58e3fe6552 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.584+0000 7fe522a7d640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe5080a3430 0x7fe5080a58f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe5140cd9b0 tx=0x7fe514098040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.585+0000 7fe510ff9640 1 -- 172.21.15.5:0/658831792 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518035040 con 0x5592e7fdd5d0 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.585+0000 7fe510ff9640 1 -- 172.21.15.5:0/658831792 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5180054f0 con 0x5592e7fdd5d0 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.585+0000 7fe510ff9640 1 -- 172.21.15.5:0/658831792 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180035f0 con 0x5592e7fdd5d0 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.585+0000 7fe5268b9740 1 -- 172.21.15.5:0/658831792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdd5d0 msgr2=0x5592e7fdda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.585+0000 7fe5268b9740 1 --2- 172.21.15.5:0/658831792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdd5d0 0x5592e7fdda10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe51802a940 tx=0x7fe518005c00 comp rx=0 tx=0).stop 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.585+0000 7fe5268b9740 1 -- 172.21.15.5:0/658831792 shutdown_connections 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.585+0000 7fe5268b9740 1 --2- 172.21.15.5:0/658831792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fe8370 0x5592e7fea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.585+0000 7fe5268b9740 1 --2- 172.21.15.5:0/658831792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdd5d0 0x5592e7fdda10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.585+0000 7fe5268b9740 1 --2- 172.21.15.5:0/658831792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fdeb40 0x5592e7fdd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.585+0000 7fe5268b9740 1 -- 172.21.15.5:0/658831792 >> 172.21.15.5:0/658831792 conn(0x5592e7fa6310 msgr2=0x5592e7fd9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.586+0000 7fe5268b9740 1 -- 172.21.15.5:0/658831792 shutdown_connections 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.586+0000 7fe5268b9740 1 -- 172.21.15.5:0/658831792 wait complete. 2024-09-19T06:09:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.586+0000 7fe5268b9740 1 -- start start 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.586+0000 7fe5268b9740 1 Processor -- start 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.586+0000 7fe5268b9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7fe7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.586+0000 7fe5268b9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdeb40 0x5592e7fe5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.586+0000 7fe5268b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 0x5592e7fe5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.586+0000 7fe5268b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5592e7feb010 con 0x5592e7fe8370 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.586+0000 7fe5268b9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5592e7efe060 con 0x5592e7fdd5d0 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.586+0000 7fe5268b9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5592e7feb190 con 0x5592e7fdeb40 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.586+0000 7fe5268b9740 10 client.0 _ll_register_callbacks cb 0x5592e7c518c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe521a7b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 0x5592e7fe5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe521a7b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 0x5592e7fe5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50560/0 (socket says 172.21.15.5:50560) 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe521a7b640 1 -- 172.21.15.5:0/1907001149 learned_addr learned my addr 172.21.15.5:0/1907001149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe52227c640 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdeb40 0x5592e7fe5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe521a7b640 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7fe7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe521a7b640 1 -- 172.21.15.5:0/1907001149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdeb40 msgr2=0x5592e7fe5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe521a7b640 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdeb40 0x5592e7fe5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe521a7b640 1 -- 172.21.15.5:0/1907001149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 msgr2=0x5592e7fe7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe521a7b640 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7fe7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe521a7b640 1 -- 172.21.15.5:0/1907001149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e7cb43c0 con 0x5592e7fe8370 2024-09-19T06:09:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe521a7b640 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 0x5592e7fe5610 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00bfa0 tx=0x7fe50c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7459a92f837ff5d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:09:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe505ffb640 1 -- 172.21.15.5:0/1907001149 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c015020 con 0x5592e7fe8370 2024-09-19T06:09:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe505ffb640 1 -- 172.21.15.5:0/1907001149 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fe50c010640 con 0x5592e7fe8370 2024-09-19T06:09:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.587+0000 7fe5268b9740 1 -- 172.21.15.5:0/1907001149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5180054f0 con 0x5592e7fe8370 2024-09-19T06:09:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.588+0000 7fe505ffb640 1 -- 172.21.15.5:0/1907001149 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe50c028050 con 0x5592e7fe8370 2024-09-19T06:09:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.588+0000 7fe5268b9740 10 client.17037 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.588+0000 7fe5268b9740 1 -- 172.21.15.5:0/1907001149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5592e7ff6b70 con 0x5592e7fe8370 2024-09-19T06:09:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.588+0000 7fe505ffb640 1 -- 172.21.15.5:0/1907001149 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5180054f0 con 0x5592e7fe8370 2024-09-19T06:09:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.588+0000 7fe505ffb640 1 -- 172.21.15.5:0/1907001149 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe50c011850 con 0x5592e7fe8370 2024-09-19T06:09:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.588+0000 7fe5268b9740 10 client.17037 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.588+0000 7fe5268b9740 10 client.17037 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:09:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.588+0000 7fe5268b9740 20 client.17037 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.588+0000 7fe5268b9740 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5592e7ff7f80 0x5592e7ffa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.588+0000 7fe5268b9740 4 client.17037 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.588+0000 7fe5268b9740 1 -- 172.21.15.5:0/1907001149 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592e7f50860 con 0x5592e7ff7f80 2024-09-19T06:09:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.588+0000 7fe52227c640 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5592e7ff7f80 0x5592e7ffa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.589+0000 7fe505ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:09:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.589+0000 7fe52227c640 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5592e7ff7f80 0x5592e7ffa3a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f38f2afeaa8019e server_cookie=e3518bcdf0204d35 in_seq=0 out_seq=0 2024-09-19T06:09:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe505ffb640 1 -- 172.21.15.5:0/1907001149 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592e7f50860 con 0x5592e7ff7f80 2024-09-19T06:09:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe505ffb640 10 client.17037 handle_command_reply: tid=0 2024-09-19T06:09:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe5268b9740 1 client.17037 shutdown 2024-09-19T06:09:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe5268b9740 1 -- 172.21.15.5:0/1907001149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 msgr2=0x5592e7fe5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe5268b9740 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 0x5592e7fe5610 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00bfa0 tx=0x7fe50c0046c0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe5268b9740 1 -- 172.21.15.5:0/1907001149 shutdown_connections 2024-09-19T06:09:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe5268b9740 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5592e7ff7f80 0x5592e7ffa3a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe5268b9740 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 0x5592e7fe5610 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe5268b9740 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdeb40 0x5592e7fe5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe5268b9740 1 --2- 172.21.15.5:0/1907001149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7fe7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe5268b9740 1 -- 172.21.15.5:0/1907001149 >> 172.21.15.5:0/1907001149 conn(0x5592e7fa6310 msgr2=0x5592e7fdf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe505ffb640 0 client.17037 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:09:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe5268b9740 1 -- 172.21.15.5:0/1907001149 shutdown_connections 2024-09-19T06:09:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe5268b9740 1 -- 172.21.15.5:0/1907001149 wait complete. 2024-09-19T06:09:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.590+0000 7fe5268b9740 20 client.17037 trim_cache size 0 max 16384 2024-09-19T06:09:44.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe5268b9740 1 Processor -- start 2024-09-19T06:09:44.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe5268b9740 1 -- start start 2024-09-19T06:09:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe5268b9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7ff4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe5268b9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdeb40 0x5592e7fe4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe5268b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 0x5592e7fffc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe5268b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1afa40 con 0x5592e7fe8370 2024-09-19T06:09:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe5268b9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1b1580 con 0x5592e7fdd5d0 2024-09-19T06:09:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe5268b9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1b0110 con 0x5592e7fdeb40 2024-09-19T06:09:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe521a7b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 0x5592e7fffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe521a7b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 0x5592e7fffc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50568/0 (socket says 172.21.15.5:50568) 2024-09-19T06:09:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe521a7b640 1 -- 172.21.15.5:0/81537931 learned_addr learned my addr 172.21.15.5:0/81537931 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe521a7b640 1 --2- 172.21.15.5:0/81537931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7ff4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe52227c640 1 --2- 172.21.15.5:0/81537931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdeb40 0x5592e7fe4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe521a7b640 1 -- 172.21.15.5:0/81537931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdeb40 msgr2=0x5592e7fe4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.605+0000 7fe521a7b640 1 --2- 172.21.15.5:0/81537931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdeb40 0x5592e7fe4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.606+0000 7fe521a7b640 1 -- 172.21.15.5:0/81537931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 msgr2=0x5592e7ff4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.606+0000 7fe521a7b640 1 --2- 172.21.15.5:0/81537931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7ff4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.606+0000 7fe521a7b640 1 -- 172.21.15.5:0/81537931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e7fe3570 con 0x5592e7fe8370 2024-09-19T06:09:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.606+0000 7fe521a7b640 1 --2- 172.21.15.5:0/81537931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 0x5592e7fffc30 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0059d0 tx=0x7fe50c00bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4651e1b63a58cae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.606+0000 7fe507fff640 1 -- 172.21.15.5:0/81537931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c015040 con 0x5592e7fe8370 2024-09-19T06:09:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.606+0000 7fe507fff640 1 -- 172.21.15.5:0/81537931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe50c00f3c0 con 0x5592e7fe8370 2024-09-19T06:09:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.606+0000 7fe507fff640 1 -- 172.21.15.5:0/81537931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c005520 con 0x5592e7fe8370 2024-09-19T06:09:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.607+0000 7fe5268b9740 1 -- 172.21.15.5:0/81537931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 msgr2=0x5592e7fffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.607+0000 7fe5268b9740 1 --2- 172.21.15.5:0/81537931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 0x5592e7fffc30 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0059d0 tx=0x7fe50c00bc00 comp rx=0 tx=0).stop 2024-09-19T06:09:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.607+0000 7fe5268b9740 1 -- 172.21.15.5:0/81537931 shutdown_connections 2024-09-19T06:09:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.607+0000 7fe5268b9740 1 --2- 172.21.15.5:0/81537931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fe8370 0x5592e7fffc30 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.607+0000 7fe5268b9740 1 --2- 172.21.15.5:0/81537931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fdeb40 0x5592e7fe4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.607+0000 7fe5268b9740 1 --2- 172.21.15.5:0/81537931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7ff4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.607+0000 7fe5268b9740 1 -- 172.21.15.5:0/81537931 >> 172.21.15.5:0/81537931 conn(0x5592e7fa6310 msgr2=0x5592e7fe38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.607+0000 7fe5268b9740 1 -- 172.21.15.5:0/81537931 shutdown_connections 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.607+0000 7fe5268b9740 1 -- 172.21.15.5:0/81537931 wait complete. 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.607+0000 7fe5268b9740 1 -- start start 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.607+0000 7fe5268b9740 1 Processor -- start 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe5268b9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7feb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe5268b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fdeb40 0x5592e7feb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe5268b9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fe8370 0x5592e7febec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe5268b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1b0eb0 con 0x5592e7fdeb40 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe5268b9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1b07e0 con 0x5592e7fdd5d0 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe5268b9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1ada20 con 0x5592e7fe8370 2024-09-19T06:09:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe521a7b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7feb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe52227c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fdeb40 0x5592e7feb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe521a7b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7feb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35524/0 (socket says 172.21.15.5:35524) 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe521a7b640 1 -- 172.21.15.5:0/1817511573 learned_addr learned my addr 172.21.15.5:0/1817511573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe5268b9740 10 client.0 _ll_register_callbacks cb 0x7fe50c00b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe521a7b640 1 --2- 172.21.15.5:0/1817511573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fe8370 0x5592e7febec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe521a7b640 1 -- 172.21.15.5:0/1817511573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fe8370 msgr2=0x5592e7febec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe521a7b640 1 --2- 172.21.15.5:0/1817511573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fe8370 0x5592e7febec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe521a7b640 1 -- 172.21.15.5:0/1817511573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fdeb40 msgr2=0x5592e7feb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe521a7b640 1 --2- 172.21.15.5:0/1817511573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fdeb40 0x5592e7feb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe521a7b640 1 -- 172.21.15.5:0/1817511573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e8060d20 con 0x5592e7fdd5d0 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe521a7b640 1 --2- 172.21.15.5:0/1817511573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7feb440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe50c005fd0 tx=0x7fe50c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95024a5b3eea753 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.608+0000 7fe505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe505ffb640 1 -- 172.21.15.5:0/1817511573 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c015030 con 0x5592e7fdd5d0 2024-09-19T06:09:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe5268b9740 1 -- 172.21.15.5:0/1817511573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5592e7f4ad70 con 0x5592e7fdd5d0 2024-09-19T06:09:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe505ffb640 1 -- 172.21.15.5:0/1817511573 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fe50c011630 con 0x5592e7fdd5d0 2024-09-19T06:09:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe505ffb640 1 -- 172.21.15.5:0/1817511573 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe50c011830 con 0x5592e7fdd5d0 2024-09-19T06:09:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe5268b9740 10 client.26621 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe5268b9740 1 -- 172.21.15.5:0/1817511573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5592e8061fc0 con 0x5592e7fdd5d0 2024-09-19T06:09:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe505ffb640 1 -- 172.21.15.5:0/1817511573 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5592e7f4ad70 con 0x5592e7fdd5d0 2024-09-19T06:09:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe505ffb640 1 -- 172.21.15.5:0/1817511573 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe50c038040 con 0x5592e7fdd5d0 2024-09-19T06:09:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe5268b9740 10 client.26621 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe5268b9740 10 client.26621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:09:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe5268b9740 20 client.26621 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe5268b9740 1 --2- 172.21.15.5:0/1817511573 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5592e8062670 0x5592e8064a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe5268b9740 4 client.26621 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.609+0000 7fe5268b9740 1 -- 172.21.15.5:0/1817511573 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592e7ec7170 con 0x5592e8062670 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe52227c640 1 --2- 172.21.15.5:0/1817511573 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5592e8062670 0x5592e8064a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe505ffb640 10 client.26621 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe52227c640 1 --2- 172.21.15.5:0/1817511573 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5592e8062670 0x5592e8064a90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad51c258f01877b3 server_cookie=f9a9a43ffa1a0cba in_seq=0 out_seq=0 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe505ffb640 1 -- 172.21.15.5:0/1817511573 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592e7ec7170 con 0x5592e8062670 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe505ffb640 10 client.26621 handle_command_reply: tid=0 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe5268b9740 1 client.26621 shutdown 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe5268b9740 1 -- 172.21.15.5:0/1817511573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 msgr2=0x5592e7feb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe5268b9740 1 --2- 172.21.15.5:0/1817511573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7feb440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe50c005fd0 tx=0x7fe50c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe5268b9740 1 -- 172.21.15.5:0/1817511573 shutdown_connections 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe5268b9740 1 --2- 172.21.15.5:0/1817511573 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5592e8062670 0x5592e8064a90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe5268b9740 1 --2- 172.21.15.5:0/1817511573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592e7fe8370 0x5592e7febec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe5268b9740 1 --2- 172.21.15.5:0/1817511573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592e7fdeb40 0x5592e7feb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe5268b9740 1 --2- 172.21.15.5:0/1817511573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592e7fdd5d0 0x5592e7feb440 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe505ffb640 0 client.26621 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:09:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe5268b9740 1 -- 172.21.15.5:0/1817511573 >> 172.21.15.5:0/1817511573 conn(0x5592e7fa6310 msgr2=0x5592e7fe4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe5268b9740 1 -- 172.21.15.5:0/1817511573 shutdown_connections 2024-09-19T06:09:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.610+0000 7fe5268b9740 1 -- 172.21.15.5:0/1817511573 wait complete. 2024-09-19T06:09:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5268b9740 20 client.26621 trim_cache size 0 max 16384 2024-09-19T06:09:44.619 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:09:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5244e0640 1 -- 172.21.15.5:0/1246541820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe5080a3430 msgr2=0x7fe5080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5244e0640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe5080a3430 0x7fe5080a58f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe5140cd9b0 tx=0x7fe514098040 comp rx=0 tx=0).stop 2024-09-19T06:09:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5244e0640 1 -- 172.21.15.5:0/1246541820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe51c108290 msgr2=0x7fe51c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5244e0640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe51c108290 0x7fe51c101470 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe51400eb10 tx=0x7fe51400c2d0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5244e0640 1 -- 172.21.15.5:0/1246541820 shutdown_connections 2024-09-19T06:09:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5244e0640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe51c108290 0x7fe51c101470 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5244e0640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe5080a3430 0x7fe5080a58f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5244e0640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe51c103740 0x7fe51c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5244e0640 1 --2- 172.21.15.5:0/1246541820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe51c102d90 0x7fe51c1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5244e0640 1 -- 172.21.15.5:0/1246541820 >> 172.21.15.5:0/1246541820 conn(0x7fe51c0fc7d0 msgr2=0x7fe51c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5244e0640 1 -- 172.21.15.5:0/1246541820 shutdown_connections 2024-09-19T06:09:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:44.611+0000 7fe5244e0640 1 -- 172.21.15.5:0/1246541820 wait complete. 2024-09-19T06:09:44.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 158968 2.74 MB/sec execute 190 sec latency 99.238 ms 2024-09-19T06:09:45.398 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:09:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:09:45.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 159449 2.74 MB/sec execute 191 sec latency 12.774 ms 2024-09-19T06:09:46.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:45 smithi005 ceph-mon[30955]: pgmap v196: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-19T06:09:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.204+0000 7f3917de3640 1 -- 172.21.15.5:0/1209663331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3910102570 msgr2=0x7f39101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.204+0000 7f3917de3640 1 --2- 172.21.15.5:0/1209663331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3910102570 0x7f39101029b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3900009900 tx=0x7f390002a480 comp rx=0 tx=0).stop 2024-09-19T06:09:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.204+0000 7f3917de3640 1 -- 172.21.15.5:0/1209663331 shutdown_connections 2024-09-19T06:09:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.204+0000 7f3917de3640 1 --2- 172.21.15.5:0/1209663331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3910102ef0 0x7f391010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.204+0000 7f3917de3640 1 --2- 172.21.15.5:0/1209663331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3910102570 0x7f39101029b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.204+0000 7f3917de3640 1 --2- 172.21.15.5:0/1209663331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3910100e20 0x7f3910101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.204+0000 7f3917de3640 1 -- 172.21.15.5:0/1209663331 >> 172.21.15.5:0/1209663331 conn(0x7f39100fc7d0 msgr2=0x7f39100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.204+0000 7f3917de3640 1 -- 172.21.15.5:0/1209663331 shutdown_connections 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.204+0000 7f3917de3640 1 -- 172.21.15.5:0/1209663331 wait complete. 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.205+0000 7f3917de3640 1 Processor -- start 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.205+0000 7f3917de3640 1 -- start start 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f3917de3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3910100e20 0x7f39100fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f3917de3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3910102570 0x7f39100fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f3917de3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3910102ef0 0x7f39100fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f3917de3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3910110e00 con 0x7f3910102570 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f3917de3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3910110c80 con 0x7f3910102ef0 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f3917de3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3910110f80 con 0x7f3910100e20 2024-09-19T06:09:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f391537e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3910102570 0x7f39100fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f391537e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3910102570 0x7f39100fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50582/0 (socket says 172.21.15.5:50582) 2024-09-19T06:09:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f391537e640 1 -- 172.21.15.5:0/980458320 learned_addr learned my addr 172.21.15.5:0/980458320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f3916380640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3910102ef0 0x7f39100fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f3915b7f640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3910100e20 0x7f39100fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f391537e640 1 -- 172.21.15.5:0/980458320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3910100e20 msgr2=0x7f39100fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f391537e640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3910100e20 0x7f39100fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f391537e640 1 -- 172.21.15.5:0/980458320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3910102ef0 msgr2=0x7f39100fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f391537e640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3910102ef0 0x7f39100fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f391537e640 1 -- 172.21.15.5:0/980458320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39101a68d0 con 0x7f3910102570 2024-09-19T06:09:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.206+0000 7f3915b7f640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3910100e20 0x7f39100fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.207+0000 7f3916380640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3910102ef0 0x7f39100fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.207+0000 7f391537e640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3910102570 0x7f39100fd1c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3900006010 tx=0x7f3900004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.207+0000 7f3906ffd640 1 -- 172.21.15.5:0/980458320 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390003f030 con 0x7f3910102570 2024-09-19T06:09:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.207+0000 7f3906ffd640 1 -- 172.21.15.5:0/980458320 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3900003aa0 con 0x7f3910102570 2024-09-19T06:09:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.207+0000 7f3917de3640 1 -- 172.21.15.5:0/980458320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39101a6b60 con 0x7f3910102570 2024-09-19T06:09:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.207+0000 7f3906ffd640 1 -- 172.21.15.5:0/980458320 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900035070 con 0x7f3910102570 2024-09-19T06:09:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.207+0000 7f3917de3640 1 -- 172.21.15.5:0/980458320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39101a7090 con 0x7f3910102570 2024-09-19T06:09:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.210+0000 7f391a1bc740 1 Processor -- start 2024-09-19T06:09:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.210+0000 7f3906ffd640 1 -- 172.21.15.5:0/980458320 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3900042030 con 0x7f3910102570 2024-09-19T06:09:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.210+0000 7f391a1bc740 1 -- start start 2024-09-19T06:09:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.210+0000 7f391a1bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6469b40 0x564af6468090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.210+0000 7f391a1bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 0x564af6468a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.210+0000 7f391a1bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af6475760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f3915b7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 0x564af6468a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f3916380640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6469b40 0x564af6468090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f3915b7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 0x564af6468a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38400/0 (socket says 172.21.15.5:38400) 2024-09-19T06:09:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f3915b7f640 1 -- 172.21.15.5:0/4029720107 learned_addr learned my addr 172.21.15.5:0/4029720107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f391537e640 1 --2- 172.21.15.5:0/4029720107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af6475760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f3906ffd640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f38fc0a3430 0x7f38fc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f391a1bc740 1 -- 172.21.15.5:0/4029720107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564af63ce2f0 con 0x564af6469b40 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f391a1bc740 1 -- 172.21.15.5:0/4029720107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564af6375cf0 con 0x564af6473370 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f391a1bc740 1 -- 172.21.15.5:0/4029720107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564af63ca2b0 con 0x564af64685d0 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f3906ffd640 1 -- 172.21.15.5:0/980458320 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f39000f0100 con 0x7f3910102570 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f3915b7f640 1 -- 172.21.15.5:0/4029720107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 msgr2=0x564af6475760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f3915b7f640 1 --2- 172.21.15.5:0/4029720107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af6475760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f3915b7f640 1 -- 172.21.15.5:0/4029720107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6469b40 msgr2=0x564af6468090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f3916380640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f38fc0a3430 0x7f38fc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f3915b7f640 1 --2- 172.21.15.5:0/4029720107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6469b40 0x564af6468090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.211+0000 7f3915b7f640 1 -- 172.21.15.5:0/4029720107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af635a3a0 con 0x564af64685d0 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.212+0000 7f3915b7f640 1 --2- 172.21.15.5:0/4029720107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 0x564af6468a10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f390c0115f0 tx=0x7f390c011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b717e69427e96d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.212+0000 7f3916380640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f38fc0a3430 0x7f38fc0a58f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3908009fd0 tx=0x7f3908004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.212+0000 7f38fbfff640 1 -- 172.21.15.5:0/4029720107 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c01e070 con 0x564af64685d0 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.212+0000 7f38fbfff640 1 -- 172.21.15.5:0/4029720107 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f390c00faf0 con 0x564af64685d0 2024-09-19T06:09:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.212+0000 7f38fbfff640 1 -- 172.21.15.5:0/4029720107 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c019b60 con 0x564af64685d0 2024-09-19T06:09:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.213+0000 7f391a1bc740 1 -- 172.21.15.5:0/4029720107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 msgr2=0x564af6468a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.213+0000 7f391a1bc740 1 --2- 172.21.15.5:0/4029720107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 0x564af6468a10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f390c0115f0 tx=0x7f390c011ab0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.213+0000 7f391a1bc740 1 -- 172.21.15.5:0/4029720107 shutdown_connections 2024-09-19T06:09:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.213+0000 7f391a1bc740 1 --2- 172.21.15.5:0/4029720107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af6475760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.213+0000 7f391a1bc740 1 --2- 172.21.15.5:0/4029720107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 0x564af6468a10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.213+0000 7f391a1bc740 1 --2- 172.21.15.5:0/4029720107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6469b40 0x564af6468090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.213+0000 7f391a1bc740 1 -- 172.21.15.5:0/4029720107 >> 172.21.15.5:0/4029720107 conn(0x564af6431310 msgr2=0x564af6464c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.213+0000 7f391a1bc740 1 -- 172.21.15.5:0/4029720107 shutdown_connections 2024-09-19T06:09:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.213+0000 7f391a1bc740 1 -- 172.21.15.5:0/4029720107 wait complete. 2024-09-19T06:09:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.213+0000 7f391a1bc740 1 -- start start 2024-09-19T06:09:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391a1bc740 1 Processor -- start 2024-09-19T06:09:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391a1bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 0x564af6472d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391a1bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6469b40 0x564af646ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391a1bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af6470530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391a1bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564af6476070 con 0x564af6469b40 2024-09-19T06:09:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391a1bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564af6389060 con 0x564af6473370 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391a1bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564af64761f0 con 0x564af64685d0 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391a1bc740 10 client.0 _ll_register_callbacks cb 0x564af60dc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391537e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6469b40 0x564af646ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f3915b7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 0x564af6472d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391537e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6469b40 0x564af646ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50600/0 (socket says 172.21.15.5:50600) 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f3915b7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 0x564af6472d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38412/0 (socket says 172.21.15.5:38412) 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391537e640 1 -- 172.21.15.5:0/1060599716 learned_addr learned my addr 172.21.15.5:0/1060599716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f3916380640 1 --2- 172.21.15.5:0/1060599716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af6470530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391537e640 1 -- 172.21.15.5:0/1060599716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 msgr2=0x564af6472d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391537e640 1 --2- 172.21.15.5:0/1060599716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 0x564af6472d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391537e640 1 -- 172.21.15.5:0/1060599716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 msgr2=0x564af6470530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391537e640 1 --2- 172.21.15.5:0/1060599716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af6470530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.214+0000 7f391537e640 1 -- 172.21.15.5:0/1060599716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af613f3c0 con 0x564af6469b40 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.215+0000 7f391537e640 1 --2- 172.21.15.5:0/1060599716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6469b40 0x564af646ffb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f39000c4bd0 tx=0x7f39000bc3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b107df7533a23999 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.215+0000 7f38f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.215+0000 7f38f97fa640 1 -- 172.21.15.5:0/1060599716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900049040 con 0x564af6469b40 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.215+0000 7f38f97fa640 1 -- 172.21.15.5:0/1060599716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f39000b8bb0 con 0x564af6469b40 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.215+0000 7f391a1bc740 1 -- 172.21.15.5:0/1060599716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f390c00faf0 con 0x564af6469b40 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.215+0000 7f38f97fa640 1 -- 172.21.15.5:0/1060599716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f39000c0840 con 0x564af6469b40 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.215+0000 7f391a1bc740 10 client.17055 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.215+0000 7f391a1bc740 1 -- 172.21.15.5:0/1060599716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564af648a220 con 0x564af6469b40 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.215+0000 7f38f97fa640 1 -- 172.21.15.5:0/1060599716 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f390c00faf0 con 0x564af6469b40 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.216+0000 7f38f97fa640 1 -- 172.21.15.5:0/1060599716 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3900003c50 con 0x564af6469b40 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.216+0000 7f391a1bc740 10 client.17055 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.216+0000 7f391a1bc740 10 client.17055 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.216+0000 7f391a1bc740 20 client.17055 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.216+0000 7f391a1bc740 1 --2- 172.21.15.5:0/1060599716 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564af648b630 0x564af648da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.216+0000 7f391a1bc740 4 client.17055 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.216+0000 7f391a1bc740 1 -- 172.21.15.5:0/1060599716 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564af63db860 con 0x564af648b630 2024-09-19T06:09:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.216+0000 7f3915b7f640 1 --2- 172.21.15.5:0/1060599716 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564af648b630 0x564af648da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.216+0000 7f38f97fa640 10 client.17055 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:09:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.216+0000 7f3915b7f640 1 --2- 172.21.15.5:0/1060599716 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564af648b630 0x564af648da20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62b40ca42a8540e5 server_cookie=3398c0a114bac9c3 in_seq=0 out_seq=0 2024-09-19T06:09:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.217+0000 7f38f97fa640 1 -- 172.21.15.5:0/1060599716 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564af63db860 con 0x564af648b630 2024-09-19T06:09:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.217+0000 7f38f97fa640 10 client.17055 handle_command_reply: tid=0 2024-09-19T06:09:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.217+0000 7f391a1bc740 1 client.17055 shutdown 2024-09-19T06:09:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.217+0000 7f391a1bc740 1 -- 172.21.15.5:0/1060599716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6469b40 msgr2=0x564af646ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.217+0000 7f391a1bc740 1 --2- 172.21.15.5:0/1060599716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6469b40 0x564af646ffb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f39000c4bd0 tx=0x7f39000bc3d0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.218+0000 7f391a1bc740 1 -- 172.21.15.5:0/1060599716 shutdown_connections 2024-09-19T06:09:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.218+0000 7f391a1bc740 1 --2- 172.21.15.5:0/1060599716 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564af648b630 0x564af648da20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.218+0000 7f391a1bc740 1 --2- 172.21.15.5:0/1060599716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af6470530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.218+0000 7f391a1bc740 1 --2- 172.21.15.5:0/1060599716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6469b40 0x564af646ffb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.218+0000 7f38f97fa640 0 client.17055 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:09:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.218+0000 7f391a1bc740 1 --2- 172.21.15.5:0/1060599716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af64685d0 0x564af6472d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.218+0000 7f391a1bc740 1 -- 172.21.15.5:0/1060599716 >> 172.21.15.5:0/1060599716 conn(0x564af6431310 msgr2=0x564af646a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.218+0000 7f391a1bc740 1 -- 172.21.15.5:0/1060599716 shutdown_connections 2024-09-19T06:09:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.218+0000 7f391a1bc740 1 -- 172.21.15.5:0/1060599716 wait complete. 2024-09-19T06:09:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.218+0000 7f391a1bc740 20 client.17055 trim_cache size 0 max 16384 2024-09-19T06:09:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f391a1bc740 1 Processor -- start 2024-09-19T06:09:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f391a1bc740 1 -- start start 2024-09-19T06:09:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f391a1bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af64685d0 0x564af6464e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f391a1bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af6469b40 0x564af6487ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f391a1bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6473370 0x564af64932e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f391a1bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39101ab550 con 0x564af6473370 2024-09-19T06:09:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f391a1bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39101ad090 con 0x564af64685d0 2024-09-19T06:09:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f391a1bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39101abc20 con 0x564af6469b40 2024-09-19T06:09:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f3916380640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6473370 0x564af64932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f3915b7f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af64685d0 0x564af6464e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f3916380640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6473370 0x564af64932e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50612/0 (socket says 172.21.15.5:50612) 2024-09-19T06:09:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f3916380640 1 -- 172.21.15.5:0/751864375 learned_addr learned my addr 172.21.15.5:0/751864375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f391537e640 1 --2- 172.21.15.5:0/751864375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af6469b40 0x564af6487ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f3915b7f640 1 -- 172.21.15.5:0/751864375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af6469b40 msgr2=0x564af6487ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f3915b7f640 1 --2- 172.21.15.5:0/751864375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af6469b40 0x564af6487ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f3915b7f640 1 -- 172.21.15.5:0/751864375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6473370 msgr2=0x564af64932e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f3915b7f640 1 --2- 172.21.15.5:0/751864375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6473370 0x564af64932e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.233+0000 7f3915b7f640 1 -- 172.21.15.5:0/751864375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af646e500 con 0x564af64685d0 2024-09-19T06:09:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.234+0000 7f391537e640 1 --2- 172.21.15.5:0/751864375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af6469b40 0x564af6487ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:09:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.234+0000 7f3915b7f640 1 --2- 172.21.15.5:0/751864375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af64685d0 0x564af6464e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f390c01c040 tx=0x7f390c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cc645d6470e5944 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:46.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.234+0000 7f38fb7fe640 1 -- 172.21.15.5:0/751864375 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c01e040 con 0x564af64685d0 2024-09-19T06:09:46.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.234+0000 7f38fb7fe640 1 -- 172.21.15.5:0/751864375 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f390c00faf0 con 0x564af64685d0 2024-09-19T06:09:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.234+0000 7f38fb7fe640 1 -- 172.21.15.5:0/751864375 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c019810 con 0x564af64685d0 2024-09-19T06:09:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.235+0000 7f391a1bc740 1 -- 172.21.15.5:0/751864375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af64685d0 msgr2=0x564af6464e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.235+0000 7f391a1bc740 1 --2- 172.21.15.5:0/751864375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af64685d0 0x564af6464e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f390c01c040 tx=0x7f390c005f90 comp rx=0 tx=0).stop 2024-09-19T06:09:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.235+0000 7f391a1bc740 1 -- 172.21.15.5:0/751864375 shutdown_connections 2024-09-19T06:09:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.235+0000 7f391a1bc740 1 --2- 172.21.15.5:0/751864375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af6473370 0x564af64932e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.235+0000 7f391a1bc740 1 --2- 172.21.15.5:0/751864375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af6469b40 0x564af6487ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.235+0000 7f391a1bc740 1 --2- 172.21.15.5:0/751864375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af64685d0 0x564af6464e10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.235+0000 7f391a1bc740 1 -- 172.21.15.5:0/751864375 >> 172.21.15.5:0/751864375 conn(0x564af6431310 msgr2=0x564af6464700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.235+0000 7f391a1bc740 1 -- 172.21.15.5:0/751864375 shutdown_connections 2024-09-19T06:09:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.235+0000 7f391a1bc740 1 -- 172.21.15.5:0/751864375 wait complete. 2024-09-19T06:09:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.235+0000 7f391a1bc740 1 -- start start 2024-09-19T06:09:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.235+0000 7f391a1bc740 1 Processor -- start 2024-09-19T06:09:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f391a1bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af64685d0 0x564af647fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f391a1bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af6469b40 0x564af64802f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f391a1bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af64f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f391a1bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39101ac9c0 con 0x564af64685d0 2024-09-19T06:09:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f391a1bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39101ac2f0 con 0x564af6473370 2024-09-19T06:09:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f391a1bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39101a9590 con 0x564af6469b40 2024-09-19T06:09:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f391a1bc740 10 client.0 _ll_register_callbacks cb 0x7f3900005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f3915b7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af64685d0 0x564af647fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f3916380640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af64f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f3915b7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af64685d0 0x564af647fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50616/0 (socket says 172.21.15.5:50616) 2024-09-19T06:09:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f3915b7f640 1 -- 172.21.15.5:0/1589927691 learned_addr learned my addr 172.21.15.5:0/1589927691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f391537e640 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af6469b40 0x564af64802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f3915b7f640 1 -- 172.21.15.5:0/1589927691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af6469b40 msgr2=0x564af64802f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f3915b7f640 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af6469b40 0x564af64802f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f3915b7f640 1 -- 172.21.15.5:0/1589927691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 msgr2=0x564af64f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f3915b7f640 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af64f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f3915b7f640 1 -- 172.21.15.5:0/1589927691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af6480bd0 con 0x564af64685d0 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.236+0000 7f3916380640 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af64f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.237+0000 7f3915b7f640 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af64685d0 0x564af647fdb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f390c009560 tx=0x7f390c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4236b00d224ab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.237+0000 7f38f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.237+0000 7f38f97fa640 1 -- 172.21.15.5:0/1589927691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c01e040 con 0x564af64685d0 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.237+0000 7f38f97fa640 1 -- 172.21.15.5:0/1589927691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f390c0053c0 con 0x564af64685d0 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.237+0000 7f391a1bc740 1 -- 172.21.15.5:0/1589927691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f390c00faf0 con 0x564af64685d0 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.237+0000 7f38f97fa640 1 -- 172.21.15.5:0/1589927691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f390c0055c0 con 0x564af64685d0 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.237+0000 7f391a1bc740 10 client.17061 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.237+0000 7f391a1bc740 1 -- 172.21.15.5:0/1589927691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564af64f68d0 con 0x564af64685d0 2024-09-19T06:09:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.237+0000 7f38f97fa640 1 -- 172.21.15.5:0/1589927691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f390c00faf0 con 0x564af64685d0 2024-09-19T06:09:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f38f97fa640 1 -- 172.21.15.5:0/1589927691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f390c003bf0 con 0x564af64685d0 2024-09-19T06:09:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f391a1bc740 10 client.17061 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f391a1bc740 10 client.17061 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:09:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f391a1bc740 20 client.17061 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f391a1bc740 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564af64f6da0 0x564af64f9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f391a1bc740 4 client.17061 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:09:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f391a1bc740 1 -- 172.21.15.5:0/1589927691 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564af63d5d70 con 0x564af64f6da0 2024-09-19T06:09:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f391537e640 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564af64f6da0 0x564af64f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f391537e640 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564af64f6da0 0x564af64f9190 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=620cd45c68622369 server_cookie=b8b2bbd659cbda26 in_seq=0 out_seq=0 2024-09-19T06:09:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f38f97fa640 10 client.17061 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:09:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f38f97fa640 1 -- 172.21.15.5:0/1589927691 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564af63d5d70 con 0x564af64f6da0 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f38f97fa640 10 client.17061 handle_command_reply: tid=0 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.238+0000 7f391a1bc740 1 client.17061 shutdown 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.239+0000 7f391a1bc740 1 -- 172.21.15.5:0/1589927691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af64685d0 msgr2=0x564af647fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.239+0000 7f391a1bc740 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af64685d0 0x564af647fdb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f390c009560 tx=0x7f390c005db0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.239+0000 7f391a1bc740 1 -- 172.21.15.5:0/1589927691 shutdown_connections 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.239+0000 7f391a1bc740 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564af64f6da0 0x564af64f9190 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.239+0000 7f391a1bc740 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564af6473370 0x564af64f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.239+0000 7f391a1bc740 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564af6469b40 0x564af64802f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.239+0000 7f38f97fa640 0 client.17061 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.239+0000 7f391a1bc740 1 --2- 172.21.15.5:0/1589927691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564af64685d0 0x564af647fdb0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.239+0000 7f391a1bc740 1 -- 172.21.15.5:0/1589927691 >> 172.21.15.5:0/1589927691 conn(0x564af6431310 msgr2=0x564af64634e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.239+0000 7f391a1bc740 1 -- 172.21.15.5:0/1589927691 shutdown_connections 2024-09-19T06:09:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.239+0000 7f391a1bc740 1 -- 172.21.15.5:0/1589927691 wait complete. 2024-09-19T06:09:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.239+0000 7f391a1bc740 20 client.17061 trim_cache size 0 max 16384 2024-09-19T06:09:46.247 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:09:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.240+0000 7f3917de3640 1 -- 172.21.15.5:0/980458320 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f38fc0a3430 msgr2=0x7f38fc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.240+0000 7f3917de3640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f38fc0a3430 0x7f38fc0a58f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3908009fd0 tx=0x7f3908004ab0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.240+0000 7f3917de3640 1 -- 172.21.15.5:0/980458320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3910102570 msgr2=0x7f39100fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:46.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.240+0000 7f3917de3640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3910102570 0x7f39100fd1c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3900006010 tx=0x7f3900004270 comp rx=0 tx=0).stop 2024-09-19T06:09:46.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.240+0000 7f3917de3640 1 -- 172.21.15.5:0/980458320 shutdown_connections 2024-09-19T06:09:46.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.240+0000 7f3917de3640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3910102ef0 0x7f39100fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.240+0000 7f3917de3640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3910102570 0x7f39100fd1c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.240+0000 7f3917de3640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f38fc0a3430 0x7f38fc0a58f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.240+0000 7f3917de3640 1 --2- 172.21.15.5:0/980458320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3910100e20 0x7f39100fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:46.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.240+0000 7f3917de3640 1 -- 172.21.15.5:0/980458320 >> 172.21.15.5:0/980458320 conn(0x7f39100fc7d0 msgr2=0x7f3910069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:46.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.240+0000 7f3917de3640 1 -- 172.21.15.5:0/980458320 shutdown_connections 2024-09-19T06:09:46.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:46.240+0000 7f3917de3640 1 -- 172.21.15.5:0/980458320 wait complete. 2024-09-19T06:09:46.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:45 smithi157 ceph-mon[35346]: pgmap v196: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-19T06:09:46.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:45 smithi049 ceph-mon[34640]: pgmap v196: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-19T06:09:46.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 159933 2.75 MB/sec execute 192 sec latency 12.679 ms 2024-09-19T06:09:46.927 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:09:47.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.760+0000 7ff35391e640 1 -- 172.21.15.5:0/3792869684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c102d90 msgr2=0x7ff34c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:47.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.760+0000 7ff35391e640 1 --2- 172.21.15.5:0/3792869684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c102d90 0x7ff34c103170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009900 tx=0x7ff33c02a410 comp rx=0 tx=0).stop 2024-09-19T06:09:47.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.760+0000 7ff35391e640 1 -- 172.21.15.5:0/3792869684 shutdown_connections 2024-09-19T06:09:47.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.760+0000 7ff35391e640 1 --2- 172.21.15.5:0/3792869684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c108290 0x7ff34c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.760+0000 7ff35391e640 1 --2- 172.21.15.5:0/3792869684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff34c103740 0x7ff34c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.760+0000 7ff35391e640 1 --2- 172.21.15.5:0/3792869684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c102d90 0x7ff34c103170 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.760+0000 7ff35391e640 1 -- 172.21.15.5:0/3792869684 >> 172.21.15.5:0/3792869684 conn(0x7ff34c0fc7d0 msgr2=0x7ff34c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:47.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.760+0000 7ff35391e640 1 -- 172.21.15.5:0/3792869684 shutdown_connections 2024-09-19T06:09:47.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.760+0000 7ff35391e640 1 -- 172.21.15.5:0/3792869684 wait complete. 2024-09-19T06:09:47.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.760+0000 7ff35391e640 1 Processor -- start 2024-09-19T06:09:47.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.760+0000 7ff35391e640 1 -- start start 2024-09-19T06:09:47.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff35391e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff34c102d90 0x7ff34c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff35391e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c103740 0x7ff34c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff35391e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c108290 0x7ff34c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff35391e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c111f60 con 0x7ff34c103740 2024-09-19T06:09:47.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff35391e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c111de0 con 0x7ff34c102d90 2024-09-19T06:09:47.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff35391e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1120e0 con 0x7ff34c108290 2024-09-19T06:09:47.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff350eb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c103740 0x7ff34c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff3516ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff34c102d90 0x7ff34c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff351ebb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c108290 0x7ff34c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff3516ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff34c102d90 0x7ff34c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35588/0 (socket says 172.21.15.5:35588) 2024-09-19T06:09:47.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff3516ba640 1 -- 172.21.15.5:0/3888546446 learned_addr learned my addr 172.21.15.5:0/3888546446 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:47.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff3516ba640 1 -- 172.21.15.5:0/3888546446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c108290 msgr2=0x7ff34c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:47.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff3516ba640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c108290 0x7ff34c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff3516ba640 1 -- 172.21.15.5:0/3888546446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c103740 msgr2=0x7ff34c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:47.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff3516ba640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c103740 0x7ff34c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff3516ba640 1 -- 172.21.15.5:0/3888546446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c19d440 con 0x7ff34c102d90 2024-09-19T06:09:47.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff351ebb640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c108290 0x7ff34c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:47.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.761+0000 7ff350eb9640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c103740 0x7ff34c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:47.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.762+0000 7ff3516ba640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff34c102d90 0x7ff34c198490 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff33c02aa00 tx=0x7ff33c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:47.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.762+0000 7ff34a7fc640 1 -- 172.21.15.5:0/3888546446 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c03f070 con 0x7ff34c102d90 2024-09-19T06:09:47.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.762+0000 7ff35391e640 1 -- 172.21.15.5:0/3888546446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c19d6d0 con 0x7ff34c102d90 2024-09-19T06:09:47.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.762+0000 7ff34a7fc640 1 -- 172.21.15.5:0/3888546446 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff33c03bb20 con 0x7ff34c102d90 2024-09-19T06:09:47.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.762+0000 7ff34a7fc640 1 -- 172.21.15.5:0/3888546446 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c035030 con 0x7ff34c102d90 2024-09-19T06:09:47.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.762+0000 7ff35391e640 1 -- 172.21.15.5:0/3888546446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c1a5010 con 0x7ff34c102d90 2024-09-19T06:09:47.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.765+0000 7ff355cf7740 1 Processor -- start 2024-09-19T06:09:47.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.765+0000 7ff355cf7740 1 -- start start 2024-09-19T06:09:47.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff34a7fc640 1 -- 172.21.15.5:0/3888546446 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff33c042070 con 0x7ff34c102d90 2024-09-19T06:09:47.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff355cf7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff355cf7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fad5d0 0x558dd8fada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff355cf7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fbc780 0x558dd8fbeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff355cf7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558dd8f132f0 con 0x558dd8fbc780 2024-09-19T06:09:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff355cf7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558dd8ebacf0 con 0x558dd8faeb40 2024-09-19T06:09:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff355cf7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558dd8f0f2b0 con 0x558dd8fad5d0 2024-09-19T06:09:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff3516ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fad5d0 0x558dd8fada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff350eb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fbc780 0x558dd8fbeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff3516ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fad5d0 0x558dd8fada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38448/0 (socket says 172.21.15.5:38448) 2024-09-19T06:09:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff3516ba640 1 -- 172.21.15.5:0/1021704872 learned_addr learned my addr 172.21.15.5:0/1021704872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff351ebb640 1 --2- 172.21.15.5:0/1021704872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff3516ba640 1 -- 172.21.15.5:0/1021704872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 msgr2=0x558dd8fad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff3516ba640 1 --2- 172.21.15.5:0/1021704872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff3516ba640 1 -- 172.21.15.5:0/1021704872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fbc780 msgr2=0x558dd8fbeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff3516ba640 1 --2- 172.21.15.5:0/1021704872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fbc780 0x558dd8fbeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.766+0000 7ff3516ba640 1 -- 172.21.15.5:0/1021704872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dd8e9f3a0 con 0x558dd8fad5d0 2024-09-19T06:09:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.767+0000 7ff34a7fc640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff3340a33b0 0x7ff3340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.767+0000 7ff3516ba640 1 --2- 172.21.15.5:0/1021704872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fad5d0 0x558dd8fada10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0eda40 tx=0x7ff33c0f8630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad8a602df93a167f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.767+0000 7ff351ebb640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff3340a33b0 0x7ff3340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.767+0000 7ff34a7fc640 1 -- 172.21.15.5:0/3888546446 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7ff33c0f0370 con 0x7ff34c102d90 2024-09-19T06:09:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.767+0000 7ff3337fe640 1 -- 172.21.15.5:0/1021704872 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0b89b0 con 0x558dd8fad5d0 2024-09-19T06:09:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.767+0000 7ff3337fe640 1 -- 172.21.15.5:0/1021704872 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff33c031430 con 0x558dd8fad5d0 2024-09-19T06:09:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.767+0000 7ff3337fe640 1 -- 172.21.15.5:0/1021704872 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0bc070 con 0x558dd8fad5d0 2024-09-19T06:09:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.768+0000 7ff355cf7740 1 -- 172.21.15.5:0/1021704872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fad5d0 msgr2=0x558dd8fada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.768+0000 7ff351ebb640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff3340a33b0 0x7ff3340a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff344009fd0 tx=0x7ff344009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.768+0000 7ff355cf7740 1 --2- 172.21.15.5:0/1021704872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fad5d0 0x558dd8fada10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0eda40 tx=0x7ff33c0f8630 comp rx=0 tx=0).stop 2024-09-19T06:09:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.768+0000 7ff355cf7740 1 -- 172.21.15.5:0/1021704872 shutdown_connections 2024-09-19T06:09:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.768+0000 7ff355cf7740 1 --2- 172.21.15.5:0/1021704872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fbc780 0x558dd8fbeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.768+0000 7ff355cf7740 1 --2- 172.21.15.5:0/1021704872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fad5d0 0x558dd8fada10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.768+0000 7ff355cf7740 1 --2- 172.21.15.5:0/1021704872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.768+0000 7ff355cf7740 1 -- 172.21.15.5:0/1021704872 >> 172.21.15.5:0/1021704872 conn(0x558dd8f76310 msgr2=0x558dd8fa9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.768+0000 7ff355cf7740 1 -- 172.21.15.5:0/1021704872 shutdown_connections 2024-09-19T06:09:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.768+0000 7ff355cf7740 1 -- 172.21.15.5:0/1021704872 wait complete. 2024-09-19T06:09:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.768+0000 7ff355cf7740 1 -- start start 2024-09-19T06:09:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.768+0000 7ff355cf7740 1 Processor -- start 2024-09-19T06:09:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff355cf7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fad5d0 0x558dd8fbc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff355cf7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fb9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff355cf7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fbc780 0x558dd8fb99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff355cf7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558dd8fbf240 con 0x558dd8fbc780 2024-09-19T06:09:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff355cf7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558dd8ece060 con 0x558dd8faeb40 2024-09-19T06:09:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff355cf7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558dd8fbf3c0 con 0x558dd8fad5d0 2024-09-19T06:09:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff351ebb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fbc780 0x558dd8fb99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff355cf7740 10 client.0 _ll_register_callbacks cb 0x558dd8c218c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff350eb9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fb9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff351ebb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fbc780 0x558dd8fb99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50642/0 (socket says 172.21.15.5:50642) 2024-09-19T06:09:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff351ebb640 1 -- 172.21.15.5:0/3043672968 learned_addr learned my addr 172.21.15.5:0/3043672968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff350eb9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fb9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35604/0 (socket says 172.21.15.5:35604) 2024-09-19T06:09:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.769+0000 7ff3516ba640 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fad5d0 0x558dd8fbc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff350eb9640 1 -- 172.21.15.5:0/3043672968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fad5d0 msgr2=0x558dd8fbc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff350eb9640 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fad5d0 0x558dd8fbc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff350eb9640 1 -- 172.21.15.5:0/3043672968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fbc780 msgr2=0x558dd8fb99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff350eb9640 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fbc780 0x558dd8fb99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff350eb9640 1 -- 172.21.15.5:0/3043672968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dd8c843c0 con 0x558dd8faeb40 2024-09-19T06:09:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff351ebb640 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fbc780 0x558dd8fb99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff350eb9640 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fb9400 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0fee70 tx=0x7ff34000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=170596c583c4fedb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff330ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff330ff9640 1 -- 172.21.15.5:0/3043672968 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340012110 con 0x558dd8faeb40 2024-09-19T06:09:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff330ff9640 1 -- 172.21.15.5:0/3043672968 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7ff340004050 con 0x558dd8faeb40 2024-09-19T06:09:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff355cf7740 1 -- 172.21.15.5:0/3043672968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558dd8f20860 con 0x558dd8faeb40 2024-09-19T06:09:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff330ff9640 1 -- 172.21.15.5:0/3043672968 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff3400122b0 con 0x558dd8faeb40 2024-09-19T06:09:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff355cf7740 10 client.26651 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.770+0000 7ff355cf7740 1 -- 172.21.15.5:0/3043672968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558dd8fcac90 con 0x558dd8faeb40 2024-09-19T06:09:47.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.771+0000 7ff330ff9640 1 -- 172.21.15.5:0/3043672968 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x558dd8f20860 con 0x558dd8faeb40 2024-09-19T06:09:47.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.771+0000 7ff330ff9640 1 -- 172.21.15.5:0/3043672968 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff340010430 con 0x558dd8faeb40 2024-09-19T06:09:47.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.771+0000 7ff355cf7740 10 client.26651 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:47.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.771+0000 7ff355cf7740 10 client.26651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:47.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.771+0000 7ff355cf7740 20 client.26651 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:47.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.771+0000 7ff355cf7740 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd8fcc0f0 0x558dd8fce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.771+0000 7ff355cf7740 4 client.26651 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:47.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.771+0000 7ff355cf7740 1 -- 172.21.15.5:0/3043672968 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dd8f1ad70 con 0x558dd8fcc0f0 2024-09-19T06:09:47.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.771+0000 7ff3516ba640 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd8fcc0f0 0x558dd8fce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.771+0000 7ff330ff9640 10 client.26651 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:47.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.772+0000 7ff3516ba640 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd8fcc0f0 0x558dd8fce530 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49314b33e1bcaab5 server_cookie=b4b0c5c7b0514a59 in_seq=0 out_seq=0 2024-09-19T06:09:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.772+0000 7ff330ff9640 1 -- 172.21.15.5:0/3043672968 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558dd8f1ad70 con 0x558dd8fcc0f0 2024-09-19T06:09:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.772+0000 7ff330ff9640 10 client.26651 handle_command_reply: tid=0 2024-09-19T06:09:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.772+0000 7ff355cf7740 1 client.26651 shutdown 2024-09-19T06:09:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.773+0000 7ff355cf7740 1 -- 172.21.15.5:0/3043672968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 msgr2=0x558dd8fb9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.773+0000 7ff355cf7740 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fb9400 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0fee70 tx=0x7ff34000f790 comp rx=0 tx=0).stop 2024-09-19T06:09:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.773+0000 7ff355cf7740 1 -- 172.21.15.5:0/3043672968 shutdown_connections 2024-09-19T06:09:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.773+0000 7ff355cf7740 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd8fcc0f0 0x558dd8fce530 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.773+0000 7ff355cf7740 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fbc780 0x558dd8fb99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.773+0000 7ff355cf7740 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fb9400 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.773+0000 7ff355cf7740 1 --2- 172.21.15.5:0/3043672968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fad5d0 0x558dd8fbc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.773+0000 7ff330ff9640 0 client.26651 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.773+0000 7ff355cf7740 1 -- 172.21.15.5:0/3043672968 >> 172.21.15.5:0/3043672968 conn(0x558dd8f76310 msgr2=0x558dd8faf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.773+0000 7ff355cf7740 1 -- 172.21.15.5:0/3043672968 shutdown_connections 2024-09-19T06:09:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.773+0000 7ff355cf7740 1 -- 172.21.15.5:0/3043672968 wait complete. 2024-09-19T06:09:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.773+0000 7ff355cf7740 20 client.26651 trim_cache size 0 max 16384 2024-09-19T06:09:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.787+0000 7ff355cf7740 1 Processor -- start 2024-09-19T06:09:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.787+0000 7ff355cf7740 1 -- start start 2024-09-19T06:09:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.787+0000 7ff355cf7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8fad5d0 0x558dd8fc84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff355cf7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8faeb40 0x558dd8fa8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff355cf7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fbc780 0x558dd8fd3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff355cf7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1ab5f0 con 0x558dd8faeb40 2024-09-19T06:09:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff355cf7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1ad130 con 0x558dd8fad5d0 2024-09-19T06:09:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff355cf7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1abcc0 con 0x558dd8fbc780 2024-09-19T06:09:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff3516ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8fad5d0 0x558dd8fc84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff350eb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8faeb40 0x558dd8fa8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff3516ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8fad5d0 0x558dd8fc84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35608/0 (socket says 172.21.15.5:35608) 2024-09-19T06:09:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff3516ba640 1 -- 172.21.15.5:0/1118478351 learned_addr learned my addr 172.21.15.5:0/1118478351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff351ebb640 1 --2- 172.21.15.5:0/1118478351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fbc780 0x558dd8fd3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff3516ba640 1 -- 172.21.15.5:0/1118478351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fbc780 msgr2=0x558dd8fd3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff3516ba640 1 --2- 172.21.15.5:0/1118478351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fbc780 0x558dd8fd3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff3516ba640 1 -- 172.21.15.5:0/1118478351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8faeb40 msgr2=0x558dd8fa8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff3516ba640 1 --2- 172.21.15.5:0/1118478351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8faeb40 0x558dd8fa8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff3516ba640 1 -- 172.21.15.5:0/1118478351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dd8fb7950 con 0x558dd8fad5d0 2024-09-19T06:09:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff351ebb640 1 --2- 172.21.15.5:0/1118478351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fbc780 0x558dd8fd3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:09:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff350eb9640 1 --2- 172.21.15.5:0/1118478351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8faeb40 0x558dd8fa8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.788+0000 7ff3516ba640 1 --2- 172.21.15.5:0/1118478351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8fad5d0 0x558dd8fc84a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0bc410 tx=0x7ff33c046040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d9720ca238052f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.789+0000 7ff332ffd640 1 -- 172.21.15.5:0/1118478351 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c049040 con 0x558dd8fad5d0 2024-09-19T06:09:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.789+0000 7ff332ffd640 1 -- 172.21.15.5:0/1118478351 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff33c0bc710 con 0x558dd8fad5d0 2024-09-19T06:09:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.789+0000 7ff332ffd640 1 -- 172.21.15.5:0/1118478351 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0bc250 con 0x558dd8fad5d0 2024-09-19T06:09:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.789+0000 7ff355cf7740 1 -- 172.21.15.5:0/1118478351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8fad5d0 msgr2=0x558dd8fc84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.789+0000 7ff355cf7740 1 --2- 172.21.15.5:0/1118478351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8fad5d0 0x558dd8fc84a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0bc410 tx=0x7ff33c046040 comp rx=0 tx=0).stop 2024-09-19T06:09:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.789+0000 7ff355cf7740 1 -- 172.21.15.5:0/1118478351 shutdown_connections 2024-09-19T06:09:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.789+0000 7ff355cf7740 1 --2- 172.21.15.5:0/1118478351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fbc780 0x558dd8fd3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.789+0000 7ff355cf7740 1 --2- 172.21.15.5:0/1118478351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8faeb40 0x558dd8fa8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.789+0000 7ff355cf7740 1 --2- 172.21.15.5:0/1118478351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8fad5d0 0x558dd8fc84a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.789+0000 7ff355cf7740 1 -- 172.21.15.5:0/1118478351 >> 172.21.15.5:0/1118478351 conn(0x558dd8f76310 msgr2=0x558dd8fa84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.790+0000 7ff355cf7740 1 -- 172.21.15.5:0/1118478351 shutdown_connections 2024-09-19T06:09:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.790+0000 7ff355cf7740 1 -- 172.21.15.5:0/1118478351 wait complete. 2024-09-19T06:09:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.790+0000 7ff355cf7740 1 -- start start 2024-09-19T06:09:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.790+0000 7ff355cf7740 1 Processor -- start 2024-09-19T06:09:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.790+0000 7ff355cf7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fad5d0 0x558dd8fc4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.790+0000 7ff355cf7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fc4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.790+0000 7ff355cf7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fbc780 0x558dd8fb1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.790+0000 7ff355cf7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1aca60 con 0x558dd8fad5d0 2024-09-19T06:09:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.790+0000 7ff355cf7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1ac390 con 0x558dd8faeb40 2024-09-19T06:09:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.790+0000 7ff355cf7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1a95d0 con 0x558dd8fbc780 2024-09-19T06:09:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff355cf7740 10 client.0 _ll_register_callbacks cb 0x7ff340009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff350eb9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fc4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff3516ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fad5d0 0x558dd8fc4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff3516ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fad5d0 0x558dd8fc4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50666/0 (socket says 172.21.15.5:50666) 2024-09-19T06:09:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff3516ba640 1 -- 172.21.15.5:0/518863083 learned_addr learned my addr 172.21.15.5:0/518863083 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff351ebb640 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fbc780 0x558dd8fb1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff3516ba640 1 -- 172.21.15.5:0/518863083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fbc780 msgr2=0x558dd8fb1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff3516ba640 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fbc780 0x558dd8fb1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff3516ba640 1 -- 172.21.15.5:0/518863083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 msgr2=0x558dd8fc4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff3516ba640 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fc4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff3516ba640 1 -- 172.21.15.5:0/518863083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dd8fb4660 con 0x558dd8fad5d0 2024-09-19T06:09:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff351ebb640 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fbc780 0x558dd8fb1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff3516ba640 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fad5d0 0x558dd8fc4610 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff33c000c00 tx=0x7ff33c0045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2b6151eb5a8c15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff330ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:09:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.791+0000 7ff330ff9640 1 -- 172.21.15.5:0/518863083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0d5060 con 0x558dd8fad5d0 2024-09-19T06:09:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff330ff9640 1 -- 172.21.15.5:0/518863083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7ff33c0baa10 con 0x558dd8fad5d0 2024-09-19T06:09:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff355cf7740 1 -- 172.21.15.5:0/518863083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff33c0bc710 con 0x558dd8fad5d0 2024-09-19T06:09:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff330ff9640 1 -- 172.21.15.5:0/518863083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff33c049070 con 0x558dd8fad5d0 2024-09-19T06:09:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff355cf7740 10 client.17079 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff355cf7740 1 -- 172.21.15.5:0/518863083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558dd8fb4660 con 0x558dd8fad5d0 2024-09-19T06:09:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff330ff9640 1 -- 172.21.15.5:0/518863083 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff33c0bc710 con 0x558dd8fad5d0 2024-09-19T06:09:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff330ff9640 1 -- 172.21.15.5:0/518863083 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff33c09c080 con 0x558dd8fad5d0 2024-09-19T06:09:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff355cf7740 10 client.17079 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff355cf7740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff355cf7740 20 client.17079 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff355cf7740 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd903bb10 0x558dd903df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff355cf7740 4 client.17079 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:09:47.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff355cf7740 1 -- 172.21.15.5:0/518863083 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558dd8e97170 con 0x558dd903bb10 2024-09-19T06:09:47.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.792+0000 7ff350eb9640 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd903bb10 0x558dd903df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:47.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.793+0000 7ff330ff9640 10 client.17079 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:47.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.793+0000 7ff350eb9640 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd903bb10 0x558dd903df00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b7be86db9d816b server_cookie=5c1896fdb461047d in_seq=0 out_seq=0 2024-09-19T06:09:47.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff330ff9640 1 -- 172.21.15.5:0/518863083 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558dd8e97170 con 0x558dd903bb10 2024-09-19T06:09:47.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff330ff9640 10 client.17079 handle_command_reply: tid=0 2024-09-19T06:09:47.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff355cf7740 1 client.17079 shutdown 2024-09-19T06:09:47.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff355cf7740 1 -- 172.21.15.5:0/518863083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fad5d0 msgr2=0x558dd8fc4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:47.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff355cf7740 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fad5d0 0x558dd8fc4610 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff33c000c00 tx=0x7ff33c0045a0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff355cf7740 1 -- 172.21.15.5:0/518863083 shutdown_connections 2024-09-19T06:09:47.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff355cf7740 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd903bb10 0x558dd903df00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff355cf7740 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd8fbc780 0x558dd8fb1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff355cf7740 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd8faeb40 0x558dd8fc4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff355cf7740 1 --2- 172.21.15.5:0/518863083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd8fad5d0 0x558dd8fc4610 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff330ff9640 0 client.17079 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:47.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff355cf7740 1 -- 172.21.15.5:0/518863083 >> 172.21.15.5:0/518863083 conn(0x558dd8f76310 msgr2=0x558dd8fbb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:47.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff355cf7740 1 -- 172.21.15.5:0/518863083 shutdown_connections 2024-09-19T06:09:47.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff355cf7740 1 -- 172.21.15.5:0/518863083 wait complete. 2024-09-19T06:09:47.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.794+0000 7ff355cf7740 20 client.17079 trim_cache size 0 max 16384 2024-09-19T06:09:47.803 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:09:47.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.795+0000 7ff35391e640 1 -- 172.21.15.5:0/3888546446 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff3340a33b0 msgr2=0x7ff3340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.795+0000 7ff35391e640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff3340a33b0 0x7ff3340a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff344009fd0 tx=0x7ff344009500 comp rx=0 tx=0).stop 2024-09-19T06:09:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.795+0000 7ff35391e640 1 -- 172.21.15.5:0/3888546446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff34c102d90 msgr2=0x7ff34c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.795+0000 7ff35391e640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff34c102d90 0x7ff34c198490 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff33c02aa00 tx=0x7ff33c003c10 comp rx=0 tx=0).stop 2024-09-19T06:09:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.795+0000 7ff35391e640 1 -- 172.21.15.5:0/3888546446 shutdown_connections 2024-09-19T06:09:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.795+0000 7ff35391e640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff3340a33b0 0x7ff3340a5870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.795+0000 7ff35391e640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c108290 0x7ff34c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.795+0000 7ff35391e640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c103740 0x7ff34c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.795+0000 7ff35391e640 1 --2- 172.21.15.5:0/3888546446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff34c102d90 0x7ff34c198490 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.795+0000 7ff35391e640 1 -- 172.21.15.5:0/3888546446 >> 172.21.15.5:0/3888546446 conn(0x7ff34c0fc7d0 msgr2=0x7ff34c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.795+0000 7ff35391e640 1 -- 172.21.15.5:0/3888546446 shutdown_connections 2024-09-19T06:09:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:47.795+0000 7ff35391e640 1 -- 172.21.15.5:0/3888546446 wait complete. 2024-09-19T06:09:47.830 INFO:tasks.workunit.client.0.smithi005.stdout: 1 160196 2.74 MB/sec execute 193 sec latency 24.948 ms 2024-09-19T06:09:48.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:47 smithi157 ceph-mon[35346]: pgmap v197: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-09-19T06:09:48.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:47 smithi005 ceph-mon[30955]: pgmap v197: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-09-19T06:09:48.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:47 smithi049 ceph-mon[34640]: pgmap v197: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-09-19T06:09:48.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 160696 2.73 MB/sec execute 194 sec latency 16.914 ms 2024-09-19T06:09:49.478 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:09:49.479 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:09:49.756 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:09:49.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 161198 2.74 MB/sec execute 195 sec latency 22.590 ms 2024-09-19T06:09:50.177 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:49 smithi005 ceph-mon[30955]: pgmap v198: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-19T06:09:50.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.275+0000 7f5224489640 1 -- 172.21.15.5:0/3583238852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c106a90 msgr2=0x7f521c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:50.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.275+0000 7f5224489640 1 --2- 172.21.15.5:0/3583238852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c106a90 0x7f521c106e70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5218009930 tx=0x7f521802a420 comp rx=0 tx=0).stop 2024-09-19T06:09:50.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.275+0000 7f5224489640 1 -- 172.21.15.5:0/3583238852 shutdown_connections 2024-09-19T06:09:50.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.275+0000 7f5224489640 1 --2- 172.21.15.5:0/3583238852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f521c101430 0x7f521c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.275+0000 7f5224489640 1 --2- 172.21.15.5:0/3583238852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f521c100a90 0x7f521c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.275+0000 7f5224489640 1 --2- 172.21.15.5:0/3583238852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c106a90 0x7f521c106e70 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.275+0000 7f5224489640 1 -- 172.21.15.5:0/3583238852 >> 172.21.15.5:0/3583238852 conn(0x7f521c0fc7d0 msgr2=0x7f521c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:50.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.275+0000 7f5224489640 1 -- 172.21.15.5:0/3583238852 shutdown_connections 2024-09-19T06:09:50.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.275+0000 7f5224489640 1 -- 172.21.15.5:0/3583238852 wait complete. 2024-09-19T06:09:50.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5224489640 1 Processor -- start 2024-09-19T06:09:50.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5224489640 1 -- start start 2024-09-19T06:09:50.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5224489640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c100a90 0x7f521c0fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5224489640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f521c101430 0x7f521c0fd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5224489640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f521c106a90 0x7f521c0fd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5224489640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f521c110d40 con 0x7f521c100a90 2024-09-19T06:09:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5224489640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f521c110bc0 con 0x7f521c106a90 2024-09-19T06:09:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5224489640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f521c110ec0 con 0x7f521c101430 2024-09-19T06:09:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5222225640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c100a90 0x7f521c0fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5221a24640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f521c101430 0x7f521c0fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5221a24640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f521c101430 0x7f521c0fd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38460/0 (socket says 172.21.15.5:38460) 2024-09-19T06:09:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5222225640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c100a90 0x7f521c0fcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50690/0 (socket says 172.21.15.5:50690) 2024-09-19T06:09:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5221a24640 1 -- 172.21.15.5:0/869775 learned_addr learned my addr 172.21.15.5:0/869775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.276+0000 7f5222a26640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f521c106a90 0x7f521c0fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.277+0000 7f5221a24640 1 -- 172.21.15.5:0/869775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f521c106a90 msgr2=0x7f521c0fd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.277+0000 7f5221a24640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f521c106a90 0x7f521c0fd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.277+0000 7f5221a24640 1 -- 172.21.15.5:0/869775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c100a90 msgr2=0x7f521c0fcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.277+0000 7f5221a24640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c100a90 0x7f521c0fcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.277+0000 7f5221a24640 1 -- 172.21.15.5:0/869775 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521c1a6860 con 0x7f521c101430 2024-09-19T06:09:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.277+0000 7f5222225640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c100a90 0x7f521c0fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.277+0000 7f5221a24640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f521c101430 0x7f521c0fd440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f520c00d770 tx=0x7f520c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.277+0000 7f52137fe640 1 -- 172.21.15.5:0/869775 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c017070 con 0x7f521c101430 2024-09-19T06:09:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.277+0000 7f5224489640 1 -- 172.21.15.5:0/869775 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521c1a6b50 con 0x7f521c101430 2024-09-19T06:09:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.277+0000 7f52137fe640 1 -- 172.21.15.5:0/869775 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f520c0054d0 con 0x7f521c101430 2024-09-19T06:09:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.277+0000 7f52137fe640 1 -- 172.21.15.5:0/869775 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c004cf0 con 0x7f521c101430 2024-09-19T06:09:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.277+0000 7f5224489640 1 -- 172.21.15.5:0/869775 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f521c1a7090 con 0x7f521c101430 2024-09-19T06:09:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.280+0000 7f52137fe640 1 -- 172.21.15.5:0/869775 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f520c004e90 con 0x7f521c101430 2024-09-19T06:09:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.280+0000 7f5226862740 1 Processor -- start 2024-09-19T06:09:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.280+0000 7f5226862740 1 -- start start 2024-09-19T06:09:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.280+0000 7f5226862740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 0x5561de3cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5226862740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 0x5561de3d6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5226862740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3d6d40 0x5561de3d9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5226862740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5561de331280 con 0x5561de3d6d40 2024-09-19T06:09:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5226862740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5561de2d8b10 con 0x5561de3cc480 2024-09-19T06:09:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5226862740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5561de32d240 con 0x5561de3cbb60 2024-09-19T06:09:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f52137fe640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52080a33b0 0x7f52080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5222225640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 0x5561de3d6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5222225640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 0x5561de3d6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35646/0 (socket says 172.21.15.5:35646) 2024-09-19T06:09:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5222225640 1 -- 172.21.15.5:0/3035575732 learned_addr learned my addr 172.21.15.5:0/3035575732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5221a24640 1 --2- 172.21.15.5:0/3035575732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3d6d40 0x5561de3d9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5222a26640 1 --2- 172.21.15.5:0/3035575732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 0x5561de3cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5222225640 1 -- 172.21.15.5:0/3035575732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 msgr2=0x5561de3cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5222225640 1 --2- 172.21.15.5:0/3035575732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 0x5561de3cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5222225640 1 -- 172.21.15.5:0/3035575732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3d6d40 msgr2=0x5561de3d9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5222225640 1 --2- 172.21.15.5:0/3035575732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3d6d40 0x5561de3d9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5222225640 1 -- 172.21.15.5:0/3035575732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561de2e51f0 con 0x5561de3cc480 2024-09-19T06:09:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f52137fe640 1 -- 172.21.15.5:0/869775 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f520c0cf730 con 0x7f521c101430 2024-09-19T06:09:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5222a26640 1 --2- 172.21.15.5:0/3035575732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 0x5561de3cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:09:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5222a26640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52080a33b0 0x7f52080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.281+0000 7f5222225640 1 --2- 172.21.15.5:0/3035575732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 0x5561de3d6800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5218006010 tx=0x7f5218032f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a917aa61fd0a20f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.282+0000 7f5210ff9640 1 -- 172.21.15.5:0/3035575732 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218035040 con 0x5561de3cc480 2024-09-19T06:09:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.282+0000 7f5210ff9640 1 -- 172.21.15.5:0/3035575732 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52180318e0 con 0x5561de3cc480 2024-09-19T06:09:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.282+0000 7f5210ff9640 1 -- 172.21.15.5:0/3035575732 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218004a00 con 0x5561de3cc480 2024-09-19T06:09:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.283+0000 7f5226862740 1 -- 172.21.15.5:0/3035575732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 msgr2=0x5561de3d6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.283+0000 7f5222a26640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52080a33b0 0x7f52080a5870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5214005fd0 tx=0x7f5214004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.283+0000 7f5226862740 1 --2- 172.21.15.5:0/3035575732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 0x5561de3d6800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5218006010 tx=0x7f5218032f60 comp rx=0 tx=0).stop 2024-09-19T06:09:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.283+0000 7f5226862740 1 -- 172.21.15.5:0/3035575732 shutdown_connections 2024-09-19T06:09:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.283+0000 7f5226862740 1 --2- 172.21.15.5:0/3035575732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3d6d40 0x5561de3d9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.283+0000 7f5226862740 1 --2- 172.21.15.5:0/3035575732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 0x5561de3d6800 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.283+0000 7f5226862740 1 --2- 172.21.15.5:0/3035575732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 0x5561de3cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.283+0000 7f5226862740 1 -- 172.21.15.5:0/3035575732 >> 172.21.15.5:0/3035575732 conn(0x5561de3c80f0 msgr2=0x5561de3c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.283+0000 7f5226862740 1 -- 172.21.15.5:0/3035575732 shutdown_connections 2024-09-19T06:09:50.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.283+0000 7f5226862740 1 -- 172.21.15.5:0/3035575732 wait complete. 2024-09-19T06:09:50.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.283+0000 7f5226862740 1 -- start start 2024-09-19T06:09:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5226862740 1 Processor -- start 2024-09-19T06:09:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5226862740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3cbb60 0x5561de3d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5226862740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 0x5561de3d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5226862740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3d6d40 0x5561de3d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5226862740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5561de3d9910 con 0x5561de3cbb60 2024-09-19T06:09:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5226862740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5561de2ebf10 con 0x5561de3cc480 2024-09-19T06:09:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5226862740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5561de3d9a90 con 0x5561de3d6d40 2024-09-19T06:09:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5226862740 10 client.0 _ll_register_callbacks cb 0x5561de2b4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5222a26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3d6d40 0x5561de3d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5221a24640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 0x5561de3d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5222225640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3cbb60 0x5561de3d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5221a24640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 0x5561de3d40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35658/0 (socket says 172.21.15.5:35658) 2024-09-19T06:09:50.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5222a26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3d6d40 0x5561de3d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38464/0 (socket says 172.21.15.5:38464) 2024-09-19T06:09:50.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5222225640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3cbb60 0x5561de3d3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50710/0 (socket says 172.21.15.5:50710) 2024-09-19T06:09:50.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.284+0000 7f5221a24640 1 -- 172.21.15.5:0/2105481583 learned_addr learned my addr 172.21.15.5:0/2105481583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:50.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5221a24640 1 -- 172.21.15.5:0/2105481583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3d6d40 msgr2=0x5561de3d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:50.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5221a24640 1 --2- 172.21.15.5:0/2105481583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3d6d40 0x5561de3d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5221a24640 1 -- 172.21.15.5:0/2105481583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3cbb60 msgr2=0x5561de3d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:50.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5221a24640 1 --2- 172.21.15.5:0/2105481583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3cbb60 0x5561de3d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5221a24640 1 -- 172.21.15.5:0/2105481583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561de0a26a0 con 0x5561de3cc480 2024-09-19T06:09:50.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5222a26640 1 --2- 172.21.15.5:0/2105481583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3d6d40 0x5561de3d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:50.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5221a24640 1 --2- 172.21.15.5:0/2105481583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 0x5561de3d40b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f520c0cce90 tx=0x7f520c09b950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=458099b965d7685d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5205ffb640 1 -- 172.21.15.5:0/2105481583 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c020070 con 0x5561de3cc480 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5226862740 1 -- 172.21.15.5:0/2105481583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5561de33e7f0 con 0x5561de3cc480 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5205ffb640 1 -- 172.21.15.5:0/2105481583 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f520c0cd0b0 con 0x5561de3cc480 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5205ffb640 1 -- 172.21.15.5:0/2105481583 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f520c0cd2b0 con 0x5561de3cc480 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5226862740 10 client.26669 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5226862740 1 -- 172.21.15.5:0/2105481583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5561de3e53d0 con 0x5561de3cc480 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.285+0000 7f5205ffb640 1 -- 172.21.15.5:0/2105481583 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5561de33e7f0 con 0x5561de3cc480 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.286+0000 7f5205ffb640 1 -- 172.21.15.5:0/2105481583 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f520c0cfb20 con 0x5561de3cc480 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.286+0000 7f5226862740 10 client.26669 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.286+0000 7f5226862740 10 client.26669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.286+0000 7f5226862740 20 client.26669 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:50.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.286+0000 7f5226862740 1 --2- 172.21.15.5:0/2105481583 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5561de3e67e0 0x5561de3e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.286+0000 7f5226862740 4 client.26669 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.286+0000 7f5226862740 1 -- 172.21.15.5:0/2105481583 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561de338d00 con 0x5561de3e67e0 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.286+0000 7f5222225640 1 --2- 172.21.15.5:0/2105481583 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5561de3e67e0 0x5561de3e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.286+0000 7f5222225640 1 --2- 172.21.15.5:0/2105481583 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5561de3e67e0 0x5561de3e8c30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c2c85178cfb367f server_cookie=ce297c52d29fcd0 in_seq=0 out_seq=0 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.286+0000 7f5205ffb640 10 client.26669 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5205ffb640 1 -- 172.21.15.5:0/2105481583 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561de338d00 con 0x5561de3e67e0 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5205ffb640 10 client.26669 handle_command_reply: tid=0 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5226862740 1 client.26669 shutdown 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5226862740 1 -- 172.21.15.5:0/2105481583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 msgr2=0x5561de3d40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5226862740 1 --2- 172.21.15.5:0/2105481583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 0x5561de3d40b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f520c0cce90 tx=0x7f520c09b950 comp rx=0 tx=0).stop 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5226862740 1 -- 172.21.15.5:0/2105481583 shutdown_connections 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5226862740 1 --2- 172.21.15.5:0/2105481583 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5561de3e67e0 0x5561de3e8c30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5226862740 1 --2- 172.21.15.5:0/2105481583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3d6d40 0x5561de3d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5226862740 1 --2- 172.21.15.5:0/2105481583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cc480 0x5561de3d40b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5226862740 1 --2- 172.21.15.5:0/2105481583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3cbb60 0x5561de3d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5226862740 1 -- 172.21.15.5:0/2105481583 >> 172.21.15.5:0/2105481583 conn(0x5561de3c80f0 msgr2=0x5561de3c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5205ffb640 0 client.26669 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5226862740 1 -- 172.21.15.5:0/2105481583 shutdown_connections 2024-09-19T06:09:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.287+0000 7f5226862740 1 -- 172.21.15.5:0/2105481583 wait complete. 2024-09-19T06:09:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.288+0000 7f5226862740 20 client.26669 trim_cache size 0 max 16384 2024-09-19T06:09:50.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:49 smithi157 ceph-mon[35346]: pgmap v198: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-19T06:09:50.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.308+0000 7f5226862740 1 Processor -- start 2024-09-19T06:09:50.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.308+0000 7f5226862740 1 -- start start 2024-09-19T06:09:50.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5226862740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 0x5561de3d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5226862740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3cc480 0x5561de3ee500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5226862740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3d6d40 0x5561de3eea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5226862740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f521c1af920 con 0x5561de3cc480 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5226862740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f521c1b1460 con 0x5561de3d6d40 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5226862740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f521c1afff0 con 0x5561de3cbb60 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5222225640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 0x5561de3d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5221a24640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3cc480 0x5561de3ee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5222225640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 0x5561de3d59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38468/0 (socket says 172.21.15.5:38468) 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5222225640 1 -- 172.21.15.5:0/3359402656 learned_addr learned my addr 172.21.15.5:0/3359402656 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5222a26640 1 --2- 172.21.15.5:0/3359402656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3d6d40 0x5561de3eea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5222225640 1 -- 172.21.15.5:0/3359402656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3d6d40 msgr2=0x5561de3eea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5222225640 1 --2- 172.21.15.5:0/3359402656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3d6d40 0x5561de3eea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5222225640 1 -- 172.21.15.5:0/3359402656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3cc480 msgr2=0x5561de3ee500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5222225640 1 --2- 172.21.15.5:0/3359402656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3cc480 0x5561de3ee500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5222225640 1 -- 172.21.15.5:0/3359402656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561de3d2b20 con 0x5561de3cbb60 2024-09-19T06:09:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5221a24640 1 --2- 172.21.15.5:0/3359402656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3cc480 0x5561de3ee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.309+0000 7f5222225640 1 --2- 172.21.15.5:0/3359402656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 0x5561de3d59d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5218009930 tx=0x7f5218009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba6ba8637a4c63ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.310+0000 7f5207fff640 1 -- 172.21.15.5:0/3359402656 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218035040 con 0x5561de3cbb60 2024-09-19T06:09:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.310+0000 7f5207fff640 1 -- 172.21.15.5:0/3359402656 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52180035f0 con 0x5561de3cbb60 2024-09-19T06:09:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.310+0000 7f5207fff640 1 -- 172.21.15.5:0/3359402656 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218032290 con 0x5561de3cbb60 2024-09-19T06:09:50.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.310+0000 7f5226862740 1 -- 172.21.15.5:0/3359402656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 msgr2=0x5561de3d59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.310+0000 7f5226862740 1 --2- 172.21.15.5:0/3359402656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 0x5561de3d59d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5218009930 tx=0x7f5218009680 comp rx=0 tx=0).stop 2024-09-19T06:09:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.310+0000 7f5226862740 1 -- 172.21.15.5:0/3359402656 shutdown_connections 2024-09-19T06:09:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.310+0000 7f5226862740 1 --2- 172.21.15.5:0/3359402656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3d6d40 0x5561de3eea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.310+0000 7f5226862740 1 --2- 172.21.15.5:0/3359402656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3cc480 0x5561de3ee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.310+0000 7f5226862740 1 --2- 172.21.15.5:0/3359402656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cbb60 0x5561de3d59d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.310+0000 7f5226862740 1 -- 172.21.15.5:0/3359402656 >> 172.21.15.5:0/3359402656 conn(0x5561de3c80f0 msgr2=0x5561de3d52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.311+0000 7f5226862740 1 -- 172.21.15.5:0/3359402656 shutdown_connections 2024-09-19T06:09:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.311+0000 7f5226862740 1 -- 172.21.15.5:0/3359402656 wait complete. 2024-09-19T06:09:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.311+0000 7f5226862740 1 -- start start 2024-09-19T06:09:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.311+0000 7f5226862740 1 Processor -- start 2024-09-19T06:09:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.311+0000 7f5226862740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cbb60 0x5561de3dab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.311+0000 7f5226862740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cc480 0x5561de3db050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.311+0000 7f5226862740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3d6d40 0x5561de3db590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.311+0000 7f5226862740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f521c1b0d90 con 0x5561de3d6d40 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.311+0000 7f5226862740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f521c1b06c0 con 0x5561de3cbb60 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.311+0000 7f5226862740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f521c1ad900 con 0x5561de3cc480 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.311+0000 7f5226862740 10 client.0 _ll_register_callbacks cb 0x7f520c00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5222a26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3d6d40 0x5561de3db590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5221a24640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cc480 0x5561de3db050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5222a26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3d6d40 0x5561de3db590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50718/0 (socket says 172.21.15.5:50718) 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5222a26640 1 -- 172.21.15.5:0/4204927244 learned_addr learned my addr 172.21.15.5:0/4204927244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5222225640 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cbb60 0x5561de3dab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5222a26640 1 -- 172.21.15.5:0/4204927244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cc480 msgr2=0x5561de3db050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5222a26640 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cc480 0x5561de3db050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5222a26640 1 -- 172.21.15.5:0/4204927244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cbb60 msgr2=0x5561de3dab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5222a26640 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cbb60 0x5561de3dab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5222a26640 1 -- 172.21.15.5:0/4204927244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561de3c3830 con 0x5561de3d6d40 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5221a24640 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cc480 0x5561de3db050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5222a26640 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3d6d40 0x5561de3db590 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5214012d70 tx=0x7f5214004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107dea9a3372f115 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5205ffb640 1 -- 172.21.15.5:0/4204927244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214012e60 con 0x5561de3d6d40 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5205ffb640 1 -- 172.21.15.5:0/4204927244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f5214010670 con 0x5561de3d6d40 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.312+0000 7f5226862740 1 -- 172.21.15.5:0/4204927244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52180035f0 con 0x5561de3d6d40 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.313+0000 7f5205ffb640 1 -- 172.21.15.5:0/4204927244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5214010870 con 0x5561de3d6d40 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.313+0000 7f5226862740 10 client.17097 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.313+0000 7f5226862740 1 -- 172.21.15.5:0/4204927244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5561de3cc2b0 con 0x5561de3d6d40 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.313+0000 7f5205ffb640 1 -- 172.21.15.5:0/4204927244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52180035f0 con 0x5561de3d6d40 2024-09-19T06:09:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.313+0000 7f5205ffb640 1 -- 172.21.15.5:0/4204927244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5214033c50 con 0x5561de3d6d40 2024-09-19T06:09:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.313+0000 7f5226862740 10 client.17097 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.313+0000 7f5226862740 10 client.17097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.313+0000 7f5226862740 20 client.17097 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.313+0000 7f5226862740 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5561de451cb0 0x5561de4540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.313+0000 7f5226862740 4 client.17097 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:09:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.313+0000 7f5226862740 1 -- 172.21.15.5:0/4204927244 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5561de2b5110 con 0x5561de451cb0 2024-09-19T06:09:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.313+0000 7f5222225640 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5561de451cb0 0x5561de4540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.314+0000 7f5222225640 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5561de451cb0 0x5561de4540f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf915ecac3c200c server_cookie=c1f6479297d3ff4b in_seq=0 out_seq=0 2024-09-19T06:09:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.314+0000 7f5205ffb640 10 client.17097 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.314+0000 7f5205ffb640 1 -- 172.21.15.5:0/4204927244 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5561de2b5110 con 0x5561de451cb0 2024-09-19T06:09:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.314+0000 7f5205ffb640 10 client.17097 handle_command_reply: tid=0 2024-09-19T06:09:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.314+0000 7f5226862740 1 client.17097 shutdown 2024-09-19T06:09:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.315+0000 7f5226862740 1 -- 172.21.15.5:0/4204927244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3d6d40 msgr2=0x5561de3db590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.315+0000 7f5226862740 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3d6d40 0x5561de3db590 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5214012d70 tx=0x7f5214004ab0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.315+0000 7f5226862740 1 -- 172.21.15.5:0/4204927244 shutdown_connections 2024-09-19T06:09:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.315+0000 7f5226862740 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5561de451cb0 0x5561de4540f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.315+0000 7f5226862740 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561de3d6d40 0x5561de3db590 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.315+0000 7f5226862740 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561de3cc480 0x5561de3db050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.315+0000 7f5226862740 1 --2- 172.21.15.5:0/4204927244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561de3cbb60 0x5561de3dab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.315+0000 7f5205ffb640 0 client.17097 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.315+0000 7f5226862740 1 -- 172.21.15.5:0/4204927244 >> 172.21.15.5:0/4204927244 conn(0x5561de3c80f0 msgr2=0x5561de3c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.315+0000 7f5226862740 1 -- 172.21.15.5:0/4204927244 shutdown_connections 2024-09-19T06:09:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.315+0000 7f5226862740 1 -- 172.21.15.5:0/4204927244 wait complete. 2024-09-19T06:09:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.315+0000 7f5226862740 20 client.17097 trim_cache size 0 max 16384 2024-09-19T06:09:50.322 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:09:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:09:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "a9e4b8d9-43b8-4f60-9c6a-ce2b7b5cba3f", 2024-09-19T06:09:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:09:50.322 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:09:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.316+0000 7f5224489640 1 -- 172.21.15.5:0/869775 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52080a33b0 msgr2=0x7f52080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.316+0000 7f5224489640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52080a33b0 0x7f52080a5870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5214005fd0 tx=0x7f5214004a40 comp rx=0 tx=0).stop 2024-09-19T06:09:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.316+0000 7f5224489640 1 -- 172.21.15.5:0/869775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f521c101430 msgr2=0x7f521c0fd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.316+0000 7f5224489640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f521c101430 0x7f521c0fd440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f520c00d770 tx=0x7f520c00dc30 comp rx=0 tx=0).stop 2024-09-19T06:09:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.316+0000 7f5224489640 1 -- 172.21.15.5:0/869775 shutdown_connections 2024-09-19T06:09:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.316+0000 7f5224489640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f521c106a90 0x7f521c0fd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.316+0000 7f5224489640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52080a33b0 0x7f52080a5870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.316+0000 7f5224489640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f521c101430 0x7f521c0fd440 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.316+0000 7f5224489640 1 --2- 172.21.15.5:0/869775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c100a90 0x7f521c0fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.316+0000 7f5224489640 1 -- 172.21.15.5:0/869775 >> 172.21.15.5:0/869775 conn(0x7f521c0fc7d0 msgr2=0x7f521c06aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.316+0000 7f5224489640 1 -- 172.21.15.5:0/869775 shutdown_connections 2024-09-19T06:09:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:50.316+0000 7f5224489640 1 -- 172.21.15.5:0/869775 wait complete. 2024-09-19T06:09:50.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:49 smithi049 ceph-mon[34640]: pgmap v198: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-19T06:09:50.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 161783 2.72 MB/sec execute 196 sec latency 26.054 ms 2024-09-19T06:09:51.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:50 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:51.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:50 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:50 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:51.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:50 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:51.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:50 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:50 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:50 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:51.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:50 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:51.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:50 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:50 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:51.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:50 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:50 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.196 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:09:51.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:50 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:51.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:50 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:50 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:51.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:50 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:51.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:50 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:50 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:50 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:51.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:50 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:51.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:50 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:50 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:51.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:50 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:50 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:50 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:51.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:50 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:50 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:51.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:50 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:51.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:50 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:50 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:50 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:51.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:50 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:51.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:50 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:50 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:51.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:50 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:50 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:09:51.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 162263 2.72 MB/sec execute 197 sec latency 12.436 ms 2024-09-19T06:09:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.994+0000 7fb0498d9640 1 -- 172.21.15.5:0/3926441468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb044103730 msgr2=0x7fb04410bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.994+0000 7fb0498d9640 1 --2- 172.21.15.5:0/3926441468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb044103730 0x7fb04410bc20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009900 tx=0x7fb02c02a480 comp rx=0 tx=0).stop 2024-09-19T06:09:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.994+0000 7fb0498d9640 1 -- 172.21.15.5:0/3926441468 shutdown_connections 2024-09-19T06:09:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.994+0000 7fb0498d9640 1 --2- 172.21.15.5:0/3926441468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb04410c1b0 0x7fb04410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.994+0000 7fb0498d9640 1 --2- 172.21.15.5:0/3926441468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb044103730 0x7fb04410bc20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.994+0000 7fb0498d9640 1 --2- 172.21.15.5:0/3926441468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb044102e10 0x7fb0441031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.994+0000 7fb0498d9640 1 -- 172.21.15.5:0/3926441468 >> 172.21.15.5:0/3926441468 conn(0x7fb0440fc7d0 msgr2=0x7fb0440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.994+0000 7fb0498d9640 1 -- 172.21.15.5:0/3926441468 shutdown_connections 2024-09-19T06:09:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.994+0000 7fb0498d9640 1 -- 172.21.15.5:0/3926441468 wait complete. 2024-09-19T06:09:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb0498d9640 1 Processor -- start 2024-09-19T06:09:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb0498d9640 1 -- start start 2024-09-19T06:09:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb0498d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb044102e10 0x7fb044198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb0498d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb04410c1b0 0x7fb044198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb0498d9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb04419ce20 0x7fb04419d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb0498d9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb044111e30 con 0x7fb04410c1b0 2024-09-19T06:09:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb0498d9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb044111cb0 con 0x7fb04419ce20 2024-09-19T06:09:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb0498d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb044111fb0 con 0x7fb044102e10 2024-09-19T06:09:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb042ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb044102e10 0x7fb044198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb0427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb04410c1b0 0x7fb044198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb0427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb04410c1b0 0x7fb044198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50732/0 (socket says 172.21.15.5:50732) 2024-09-19T06:09:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb0427fc640 1 -- 172.21.15.5:0/2665592611 learned_addr learned my addr 172.21.15.5:0/2665592611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.995+0000 7fb0437fe640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb04419ce20 0x7fb04419d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.996+0000 7fb042ffd640 1 -- 172.21.15.5:0/2665592611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb04419ce20 msgr2=0x7fb04419d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.996+0000 7fb042ffd640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb04419ce20 0x7fb04419d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.996+0000 7fb042ffd640 1 -- 172.21.15.5:0/2665592611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb04410c1b0 msgr2=0x7fb044198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.996+0000 7fb042ffd640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb04410c1b0 0x7fb044198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.996+0000 7fb042ffd640 1 -- 172.21.15.5:0/2665592611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04419d8e0 con 0x7fb044102e10 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.996+0000 7fb042ffd640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb044102e10 0x7fb044198550 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb03400eb00 tx=0x7fb03400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.996+0000 7fb0488d7640 1 -- 172.21.15.5:0/2665592611 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034010430 con 0x7fb044102e10 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.996+0000 7fb0488d7640 1 -- 172.21.15.5:0/2665592611 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb034010a80 con 0x7fb044102e10 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.996+0000 7fb0498d9640 1 -- 172.21.15.5:0/2665592611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04419dbd0 con 0x7fb044102e10 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.996+0000 7fb0488d7640 1 -- 172.21.15.5:0/2665592611 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034017ad0 con 0x7fb044102e10 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.996+0000 7fb0498d9640 1 -- 172.21.15.5:0/2665592611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0441a53b0 con 0x7fb044102e10 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.999+0000 7fb0488d7640 1 -- 172.21.15.5:0/2665592611 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb034010c30 con 0x7fb044102e10 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.999+0000 7fb04bcb2740 1 Processor -- start 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.999+0000 7fb04bcb2740 1 -- start start 2024-09-19T06:09:52.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.999+0000 7fb04bcb2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58c380 0x556e8b58c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.999+0000 7fb04bcb2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b58cca0 0x556e8b59b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:51.999+0000 7fb04bcb2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b59ba90 0x556e8b59de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb04bcb2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e8b4f22e0 con 0x556e8b58cca0 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb04bcb2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556e8b499ce0 con 0x556e8b58c380 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb04bcb2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556e8b4ee2a0 con 0x556e8b59ba90 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb042ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b58cca0 0x556e8b59b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb042ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b58cca0 0x556e8b59b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50742/0 (socket says 172.21.15.5:50742) 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb042ffd640 1 -- 172.21.15.5:0/2378384699 learned_addr learned my addr 172.21.15.5:0/2378384699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb0427fc640 1 --2- 172.21.15.5:0/2378384699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b59ba90 0x556e8b59de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb0437fe640 1 --2- 172.21.15.5:0/2378384699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58c380 0x556e8b58c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb0427fc640 1 -- 172.21.15.5:0/2378384699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58c380 msgr2=0x556e8b58c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb0427fc640 1 --2- 172.21.15.5:0/2378384699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58c380 0x556e8b58c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb0427fc640 1 -- 172.21.15.5:0/2378384699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b58cca0 msgr2=0x556e8b59b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:52.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb0427fc640 1 --2- 172.21.15.5:0/2378384699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b58cca0 0x556e8b59b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb0427fc640 1 -- 172.21.15.5:0/2378384699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e8b47e440 con 0x556e8b59ba90 2024-09-19T06:09:52.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb0427fc640 1 --2- 172.21.15.5:0/2378384699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b59ba90 0x556e8b59de80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb02c000c00 tx=0x7fb02c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dcc663915671478 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:52.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.000+0000 7fb0488d7640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb0300a33b0 0x7fb0300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb0437fe640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb0300a33b0 0x7fb0300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb039ffb640 1 -- 172.21.15.5:0/2378384699 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c040070 con 0x556e8b59ba90 2024-09-19T06:09:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb0488d7640 1 -- 172.21.15.5:0/2665592611 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fb0340d06d0 con 0x7fb044102e10 2024-09-19T06:09:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb039ffb640 1 -- 172.21.15.5:0/2378384699 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb02c004030 con 0x556e8b59ba90 2024-09-19T06:09:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb039ffb640 1 -- 172.21.15.5:0/2378384699 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c035040 con 0x556e8b59ba90 2024-09-19T06:09:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb04bcb2740 1 -- 172.21.15.5:0/2378384699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b59ba90 msgr2=0x556e8b59de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/2378384699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b59ba90 0x556e8b59de80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb02c000c00 tx=0x7fb02c005bd0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb0437fe640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb0300a33b0 0x7fb0300a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009d20 tx=0x7fb03c005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb04bcb2740 1 -- 172.21.15.5:0/2378384699 shutdown_connections 2024-09-19T06:09:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/2378384699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b59ba90 0x556e8b59de80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/2378384699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b58cca0 0x556e8b59b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/2378384699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58c380 0x556e8b58c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.001+0000 7fb04bcb2740 1 -- 172.21.15.5:0/2378384699 >> 172.21.15.5:0/2378384699 conn(0x556e8b588a30 msgr2=0x556e8b588e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.002+0000 7fb04bcb2740 1 -- 172.21.15.5:0/2378384699 shutdown_connections 2024-09-19T06:09:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.002+0000 7fb04bcb2740 1 -- 172.21.15.5:0/2378384699 wait complete. 2024-09-19T06:09:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.002+0000 7fb04bcb2740 1 -- start start 2024-09-19T06:09:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.002+0000 7fb04bcb2740 1 Processor -- start 2024-09-19T06:09:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.002+0000 7fb04bcb2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 0x556e8b59b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.002+0000 7fb04bcb2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58cca0 0x556e8b598630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.002+0000 7fb04bcb2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b59ba90 0x556e8b598b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.002+0000 7fb04bcb2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e8b59e660 con 0x556e8b59ba90 2024-09-19T06:09:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.002+0000 7fb04bcb2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556e8b4ad050 con 0x556e8b58cca0 2024-09-19T06:09:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.002+0000 7fb04bcb2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556e8b59e7e0 con 0x556e8b58c380 2024-09-19T06:09:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.002+0000 7fb04bcb2740 10 client.0 _ll_register_callbacks cb 0x556e8b4ed160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb0437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b59ba90 0x556e8b598b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb042ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 0x556e8b59b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb0427fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58cca0 0x556e8b598630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb042ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 0x556e8b59b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38518/0 (socket says 172.21.15.5:38518) 2024-09-19T06:09:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb042ffd640 1 -- 172.21.15.5:0/1127034119 learned_addr learned my addr 172.21.15.5:0/1127034119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb0437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b59ba90 0x556e8b598b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50754/0 (socket says 172.21.15.5:50754) 2024-09-19T06:09:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb042ffd640 1 -- 172.21.15.5:0/1127034119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58cca0 msgr2=0x556e8b598630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb042ffd640 1 --2- 172.21.15.5:0/1127034119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58cca0 0x556e8b598630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb042ffd640 1 -- 172.21.15.5:0/1127034119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b59ba90 msgr2=0x556e8b598b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:52.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb042ffd640 1 --2- 172.21.15.5:0/1127034119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b59ba90 0x556e8b598b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb042ffd640 1 -- 172.21.15.5:0/1127034119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8b2633c0 con 0x556e8b58c380 2024-09-19T06:09:52.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb042ffd640 1 --2- 172.21.15.5:0/1127034119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 0x556e8b59b380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb03400cfd0 tx=0x7fb0340cddf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7044aab017332247 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:52.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb022ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:09:52.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb022ffd640 1 -- 172.21.15.5:0/1127034119 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03409c2c0 con 0x556e8b58c380 2024-09-19T06:09:52.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb022ffd640 1 -- 172.21.15.5:0/1127034119 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fb0340ce050 con 0x556e8b58c380 2024-09-19T06:09:52.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.003+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1127034119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb02c004030 con 0x556e8b58c380 2024-09-19T06:09:52.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.004+0000 7fb022ffd640 1 -- 172.21.15.5:0/1127034119 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb0340ce250 con 0x556e8b58c380 2024-09-19T06:09:52.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.004+0000 7fb04bcb2740 10 client.26599 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.004+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1127034119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556e8b5aa070 con 0x556e8b58c380 2024-09-19T06:09:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.004+0000 7fb022ffd640 1 -- 172.21.15.5:0/1127034119 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb02c004030 con 0x556e8b58c380 2024-09-19T06:09:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.004+0000 7fb022ffd640 1 -- 172.21.15.5:0/1127034119 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb034017c70 con 0x556e8b58c380 2024-09-19T06:09:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.004+0000 7fb04bcb2740 10 client.26599 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.004+0000 7fb04bcb2740 10 client.26599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.004+0000 7fb04bcb2740 20 client.26599 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.004+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/1127034119 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e8b5ab480 0x556e8b5ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.004+0000 7fb04bcb2740 4 client.26599 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.004+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1127034119 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e8b4ff850 con 0x556e8b5ab480 2024-09-19T06:09:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.004+0000 7fb0427fc640 1 --2- 172.21.15.5:0/1127034119 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e8b5ab480 0x556e8b5ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.005+0000 7fb0427fc640 1 --2- 172.21.15.5:0/1127034119 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e8b5ab480 0x556e8b5ad8f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7936b290873153d9 server_cookie=71b8032108cc876e in_seq=0 out_seq=0 2024-09-19T06:09:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.005+0000 7fb022ffd640 10 client.26599 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.005+0000 7fb022ffd640 1 -- 172.21.15.5:0/1127034119 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e8b4ff850 con 0x556e8b5ab480 2024-09-19T06:09:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.005+0000 7fb022ffd640 10 client.26599 handle_command_reply: tid=0 2024-09-19T06:09:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.005+0000 7fb04bcb2740 1 client.26599 shutdown 2024-09-19T06:09:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.006+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1127034119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 msgr2=0x556e8b59b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.006+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/1127034119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 0x556e8b59b380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb03400cfd0 tx=0x7fb0340cddf0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.006+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1127034119 shutdown_connections 2024-09-19T06:09:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.006+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/1127034119 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e8b5ab480 0x556e8b5ad8f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.006+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/1127034119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b59ba90 0x556e8b598b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.006+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/1127034119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58cca0 0x556e8b598630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.006+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/1127034119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 0x556e8b59b380 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.006+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1127034119 >> 172.21.15.5:0/1127034119 conn(0x556e8b588a30 msgr2=0x556e8b58e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.006+0000 7fb022ffd640 0 client.26599 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.006+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1127034119 shutdown_connections 2024-09-19T06:09:52.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.006+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1127034119 wait complete. 2024-09-19T06:09:52.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.006+0000 7fb04bcb2740 20 client.26599 trim_cache size 0 max 16384 2024-09-19T06:09:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb04bcb2740 1 Processor -- start 2024-09-19T06:09:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb04bcb2740 1 -- start start 2024-09-19T06:09:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb04bcb2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b588a30 0x556e8b5a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb04bcb2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b58c380 0x556e8b593c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb04bcb2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58cca0 0x556e8b5b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb04bcb2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441abb50 con 0x556e8b58c380 2024-09-19T06:09:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb04bcb2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441ad690 con 0x556e8b588a30 2024-09-19T06:09:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb04bcb2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441ac220 con 0x556e8b58cca0 2024-09-19T06:09:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb0427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b58c380 0x556e8b593c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb042ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b588a30 0x556e8b5a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb0437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58cca0 0x556e8b5b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb0427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b58c380 0x556e8b593c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50756/0 (socket says 172.21.15.5:50756) 2024-09-19T06:09:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.025+0000 7fb0427fc640 1 -- 172.21.15.5:0/1742344795 learned_addr learned my addr 172.21.15.5:0/1742344795 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.026+0000 7fb0437fe640 1 -- 172.21.15.5:0/1742344795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b588a30 msgr2=0x556e8b5a7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.026+0000 7fb0437fe640 1 --2- 172.21.15.5:0/1742344795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b588a30 0x556e8b5a7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.026+0000 7fb0437fe640 1 -- 172.21.15.5:0/1742344795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b58c380 msgr2=0x556e8b593c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.026+0000 7fb0437fe640 1 --2- 172.21.15.5:0/1742344795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b58c380 0x556e8b593c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.026+0000 7fb0437fe640 1 -- 172.21.15.5:0/1742344795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e8b594140 con 0x556e8b58cca0 2024-09-19T06:09:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.026+0000 7fb042ffd640 1 --2- 172.21.15.5:0/1742344795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b588a30 0x556e8b5a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.026+0000 7fb0437fe640 1 --2- 172.21.15.5:0/1742344795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58cca0 0x556e8b5b31c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb03c012cb0 tx=0x7fb03c01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcce937d09330c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.026+0000 7fb0397fa640 1 -- 172.21.15.5:0/1742344795 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c02e070 con 0x556e8b58cca0 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.026+0000 7fb0397fa640 1 -- 172.21.15.5:0/1742344795 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb03c003c70 con 0x556e8b58cca0 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.026+0000 7fb0397fa640 1 -- 172.21.15.5:0/1742344795 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c029680 con 0x556e8b58cca0 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.027+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1742344795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58cca0 msgr2=0x556e8b5b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.027+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/1742344795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58cca0 0x556e8b5b31c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb03c012cb0 tx=0x7fb03c01b000 comp rx=0 tx=0).stop 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.027+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1742344795 shutdown_connections 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.027+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/1742344795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58cca0 0x556e8b5b31c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.027+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/1742344795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b58c380 0x556e8b593c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.027+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/1742344795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b588a30 0x556e8b5a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.027+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1742344795 >> 172.21.15.5:0/1742344795 conn(0x556e8b598cd0 msgr2=0x556e8b5934f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.027+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1742344795 shutdown_connections 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.027+0000 7fb04bcb2740 1 -- 172.21.15.5:0/1742344795 wait complete. 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.027+0000 7fb04bcb2740 1 -- start start 2024-09-19T06:09:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb04bcb2740 1 Processor -- start 2024-09-19T06:09:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb04bcb2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b588a30 0x556e8b5a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb04bcb2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 0x556e8b5951f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb04bcb2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58cca0 0x556e8b595730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb04bcb2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441acfc0 con 0x556e8b588a30 2024-09-19T06:09:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb04bcb2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441ac8f0 con 0x556e8b58cca0 2024-09-19T06:09:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb04bcb2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441a9b30 con 0x556e8b58c380 2024-09-19T06:09:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb04bcb2740 10 client.0 _ll_register_callbacks cb 0x7fb034005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb042ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b588a30 0x556e8b5a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb042ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b588a30 0x556e8b5a3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50760/0 (socket says 172.21.15.5:50760) 2024-09-19T06:09:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb042ffd640 1 -- 172.21.15.5:0/3628395840 learned_addr learned my addr 172.21.15.5:0/3628395840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb0437fe640 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58cca0 0x556e8b595730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb0427fc640 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 0x556e8b5951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb042ffd640 1 -- 172.21.15.5:0/3628395840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 msgr2=0x556e8b5951f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb042ffd640 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 0x556e8b5951f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb042ffd640 1 -- 172.21.15.5:0/3628395840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58cca0 msgr2=0x556e8b595730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb042ffd640 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58cca0 0x556e8b595730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb042ffd640 1 -- 172.21.15.5:0/3628395840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8b595d20 con 0x556e8b588a30 2024-09-19T06:09:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.028+0000 7fb0427fc640 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 0x556e8b5951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb042ffd640 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b588a30 0x556e8b5a3fc0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb0340b5ce0 tx=0x7fb0340ce300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80260b7882e2569 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb022ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:09:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb022ffd640 1 -- 172.21.15.5:0/3628395840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03409c030 con 0x556e8b588a30 2024-09-19T06:09:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb022ffd640 1 -- 172.21.15.5:0/3628395840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7fb0340105d0 con 0x556e8b588a30 2024-09-19T06:09:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb04bcb2740 1 -- 172.21.15.5:0/3628395840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb03c003c70 con 0x556e8b588a30 2024-09-19T06:09:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb022ffd640 1 -- 172.21.15.5:0/3628395840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb034017e80 con 0x556e8b588a30 2024-09-19T06:09:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb04bcb2740 10 client.17115 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb04bcb2740 1 -- 172.21.15.5:0/3628395840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556e8b615fa0 con 0x556e8b588a30 2024-09-19T06:09:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb022ffd640 1 -- 172.21.15.5:0/3628395840 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb03c003c70 con 0x556e8b588a30 2024-09-19T06:09:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb022ffd640 1 -- 172.21.15.5:0/3628395840 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb03409b820 con 0x556e8b588a30 2024-09-19T06:09:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb04bcb2740 10 client.17115 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb04bcb2740 10 client.17115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb04bcb2740 20 client.17115 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e8b59ba90 0x556e8b618670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb04bcb2740 4 client.17115 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:09:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.029+0000 7fb04bcb2740 1 -- 172.21.15.5:0/3628395840 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e8b4f9d60 con 0x556e8b59ba90 2024-09-19T06:09:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.030+0000 7fb0427fc640 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e8b59ba90 0x556e8b618670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.030+0000 7fb0427fc640 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e8b59ba90 0x556e8b618670 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f9c40791854c9f server_cookie=ab643412519f0032 in_seq=0 out_seq=0 2024-09-19T06:09:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.030+0000 7fb022ffd640 10 client.17115 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.030+0000 7fb022ffd640 1 -- 172.21.15.5:0/3628395840 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556e8b4f9d60 con 0x556e8b59ba90 2024-09-19T06:09:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.030+0000 7fb022ffd640 10 client.17115 handle_command_reply: tid=0 2024-09-19T06:09:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.030+0000 7fb04bcb2740 1 client.17115 shutdown 2024-09-19T06:09:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.030+0000 7fb04bcb2740 1 -- 172.21.15.5:0/3628395840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b588a30 msgr2=0x556e8b5a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.030+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b588a30 0x556e8b5a3fc0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb0340b5ce0 tx=0x7fb0340ce300 comp rx=0 tx=0).stop 2024-09-19T06:09:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb04bcb2740 1 -- 172.21.15.5:0/3628395840 shutdown_connections 2024-09-19T06:09:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556e8b59ba90 0x556e8b618670 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e8b58cca0 0x556e8b595730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb022ffd640 0 client.17115 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e8b58c380 0x556e8b5951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb04bcb2740 1 --2- 172.21.15.5:0/3628395840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e8b588a30 0x556e8b5a3fc0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb04bcb2740 1 -- 172.21.15.5:0/3628395840 >> 172.21.15.5:0/3628395840 conn(0x556e8b598cd0 msgr2=0x556e8b587350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb04bcb2740 1 -- 172.21.15.5:0/3628395840 shutdown_connections 2024-09-19T06:09:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb04bcb2740 1 -- 172.21.15.5:0/3628395840 wait complete. 2024-09-19T06:09:52.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb04bcb2740 20 client.17115 trim_cache size 0 max 16384 2024-09-19T06:09:52.040 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:09:52.040 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:09:52.040 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:09:52.040 INFO:teuthology.orchestra.run.smithi005.stdout: "a9e4b8d9-43b8-4f60-9c6a-ce2b7b5cba3f": { 2024-09-19T06:09:52.040 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:09:52.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "a9e4b8d9-43b8-4f60-9c6a-ce2b7b5cba3f", 2024-09-19T06:09:52.041 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:09:52.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:09:52.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:09:52.041 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:09:52.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb0498d9640 1 -- 172.21.15.5:0/2665592611 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb0300a33b0 msgr2=0x7fb0300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:52.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb0498d9640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb0300a33b0 0x7fb0300a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009d20 tx=0x7fb03c005f90 comp rx=0 tx=0).stop 2024-09-19T06:09:52.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb0498d9640 1 -- 172.21.15.5:0/2665592611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb044102e10 msgr2=0x7fb044198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:52.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.031+0000 7fb0498d9640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb044102e10 0x7fb044198550 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb03400eb00 tx=0x7fb03400c2d0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.032+0000 7fb0498d9640 1 -- 172.21.15.5:0/2665592611 shutdown_connections 2024-09-19T06:09:52.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.032+0000 7fb0498d9640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb04419ce20 0x7fb04419d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.032+0000 7fb0498d9640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb04410c1b0 0x7fb044198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.032+0000 7fb0498d9640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb0300a33b0 0x7fb0300a5870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.032+0000 7fb0498d9640 1 --2- 172.21.15.5:0/2665592611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb044102e10 0x7fb044198550 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:52.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.032+0000 7fb0498d9640 1 -- 172.21.15.5:0/2665592611 >> 172.21.15.5:0/2665592611 conn(0x7fb0440fc7d0 msgr2=0x7fb0440fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:52.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.032+0000 7fb0498d9640 1 -- 172.21.15.5:0/2665592611 shutdown_connections 2024-09-19T06:09:52.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:09:52.032+0000 7fb0498d9640 1 -- 172.21.15.5:0/2665592611 wait complete. 2024-09-19T06:09:52.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:51 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:09:52.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:51 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:52.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:51 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:09:52.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:51 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:52.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:51 smithi157 ceph-mon[35346]: pgmap v199: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-19T06:09:52.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:51 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:09:52.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:51 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:52.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:51 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:09:52.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:51 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:52.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:51 smithi005 ceph-mon[30955]: pgmap v199: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-19T06:09:52.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:51 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:09:52.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:51 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:52.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:51 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:09:52.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:51 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:09:52.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:51 smithi049 ceph-mon[34640]: pgmap v199: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-19T06:09:52.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 162964 2.74 MB/sec execute 198 sec latency 15.142 ms 2024-09-19T06:09:52.916 INFO:tasks.cephfs.filesystem:scrub status for tag:a9e4b8d9-43b8-4f60-9c6a-ce2b7b5cba3f - {'path': '/', 'tag': 'a9e4b8d9-43b8-4f60-9c6a-ce2b7b5cba3f', 'options': 'recursive,force'} 2024-09-19T06:09:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.389+0000 7f2011d31640 1 -- 172.21.15.5:0/2376954226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f200c1009a0 msgr2=0x7f200c100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.389+0000 7f2011d31640 1 --2- 172.21.15.5:0/2376954226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f200c1009a0 0x7f200c100e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2000009930 tx=0x7f200002a420 comp rx=0 tx=0).stop 2024-09-19T06:09:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.389+0000 7f2011d31640 1 -- 172.21.15.5:0/2376954226 shutdown_connections 2024-09-19T06:09:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.389+0000 7f2011d31640 1 --2- 172.21.15.5:0/2376954226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200c101340 0x7f200c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.389+0000 7f2011d31640 1 --2- 172.21.15.5:0/2376954226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f200c1009a0 0x7f200c100e00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.389+0000 7f2011d31640 1 --2- 172.21.15.5:0/2376954226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c1069b0 0x7f200c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.389+0000 7f2011d31640 1 -- 172.21.15.5:0/2376954226 >> 172.21.15.5:0/2376954226 conn(0x7f200c0779a0 msgr2=0x7f200c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.389+0000 7f2011d31640 1 -- 172.21.15.5:0/2376954226 shutdown_connections 2024-09-19T06:09:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.389+0000 7f2011d31640 1 -- 172.21.15.5:0/2376954226 wait complete. 2024-09-19T06:09:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.389+0000 7f2011d31640 1 Processor -- start 2024-09-19T06:09:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f2011d31640 1 -- start start 2024-09-19T06:09:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f2011d31640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c1009a0 0x7f200c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f2011d31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f200c101340 0x7f200c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f2011d31640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200c1069b0 0x7f200c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f2011d31640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f200c110ca0 con 0x7f200c101340 2024-09-19T06:09:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f2011d31640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f200c110b20 con 0x7f200c1069b0 2024-09-19T06:09:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f2011d31640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f200c110e20 con 0x7f200c1009a0 2024-09-19T06:09:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f200c101340 0x7f200c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c1009a0 0x7f200c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c1009a0 0x7f200c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38548/0 (socket says 172.21.15.5:38548) 2024-09-19T06:09:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f200c101340 0x7f200c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50786/0 (socket says 172.21.15.5:50786) 2024-09-19T06:09:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200b7fe640 1 -- 172.21.15.5:0/387485661 learned_addr learned my addr 172.21.15.5:0/387485661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200bfff640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200c1069b0 0x7f200c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200affd640 1 -- 172.21.15.5:0/387485661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c1009a0 msgr2=0x7f200c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200affd640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c1009a0 0x7f200c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200affd640 1 -- 172.21.15.5:0/387485661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200c1069b0 msgr2=0x7f200c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200affd640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200c1069b0 0x7f200c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200affd640 1 -- 172.21.15.5:0/387485661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c19d3d0 con 0x7f200c101340 2024-09-19T06:09:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200b7fe640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c1009a0 0x7f200c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.390+0000 7f200bfff640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200c1069b0 0x7f200c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.391+0000 7f200affd640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f200c101340 0x7f200c198920 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2000031440 tx=0x7f2000004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.391+0000 7f2008ff9640 1 -- 172.21.15.5:0/387485661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200003f070 con 0x7f200c101340 2024-09-19T06:09:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.391+0000 7f2008ff9640 1 -- 172.21.15.5:0/387485661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2000003aa0 con 0x7f200c101340 2024-09-19T06:09:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.391+0000 7f2008ff9640 1 -- 172.21.15.5:0/387485661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000035070 con 0x7f200c101340 2024-09-19T06:09:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.391+0000 7f2011d31640 1 -- 172.21.15.5:0/387485661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200c19d660 con 0x7f200c101340 2024-09-19T06:09:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.391+0000 7f2011d31640 1 -- 172.21.15.5:0/387485661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200c19dad0 con 0x7f200c101340 2024-09-19T06:09:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f2014ad2740 1 Processor -- start 2024-09-19T06:09:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f2008ff9640 1 -- 172.21.15.5:0/387485661 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2000003c50 con 0x7f200c101340 2024-09-19T06:09:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f2014ad2740 1 -- start start 2024-09-19T06:09:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f2014ad2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542cdaf0 0x5655542cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f2014ad2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542ce410 0x5655542d8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f2014ad2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 0x5655542db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f2014ad2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5655542322f0 con 0x5655542d8cb0 2024-09-19T06:09:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f2014ad2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5655541da270 con 0x5655542cdaf0 2024-09-19T06:09:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f2014ad2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56555422e2b0 con 0x5655542ce410 2024-09-19T06:09:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f200affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 0x5655542db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f200affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 0x5655542db0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50792/0 (socket says 172.21.15.5:50792) 2024-09-19T06:09:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f200affd640 1 -- 172.21.15.5:0/2717630609 learned_addr learned my addr 172.21.15.5:0/2717630609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f200b7fe640 1 --2- 172.21.15.5:0/2717630609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542ce410 0x5655542d8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.394+0000 7f200bfff640 1 --2- 172.21.15.5:0/2717630609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542cdaf0 0x5655542cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.395+0000 7f200affd640 1 -- 172.21.15.5:0/2717630609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542ce410 msgr2=0x5655542d8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.395+0000 7f200affd640 1 --2- 172.21.15.5:0/2717630609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542ce410 0x5655542d8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.395+0000 7f200affd640 1 -- 172.21.15.5:0/2717630609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542cdaf0 msgr2=0x5655542cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.395+0000 7f200affd640 1 --2- 172.21.15.5:0/2717630609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542cdaf0 0x5655542cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.395+0000 7f200affd640 1 -- 172.21.15.5:0/2717630609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655541d9340 con 0x5655542d8cb0 2024-09-19T06:09:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.395+0000 7f2008ff9640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1ff40a3430 0x7f1ff40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.395+0000 7f200bfff640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1ff40a3430 0x7f1ff40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.395+0000 7f200affd640 1 --2- 172.21.15.5:0/2717630609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 0x5655542db0a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f20000ed870 tx=0x7f20000f8660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5816d37ea4d480ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.395+0000 7f2008ff9640 1 -- 172.21.15.5:0/387485661 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f20000f0100 con 0x7f200c101340 2024-09-19T06:09:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.395+0000 7f1ff1ffb640 1 -- 172.21.15.5:0/2717630609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20000bc310 con 0x5655542d8cb0 2024-09-19T06:09:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.395+0000 7f1ff1ffb640 1 -- 172.21.15.5:0/2717630609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20000bc540 con 0x5655542d8cb0 2024-09-19T06:09:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.395+0000 7f1ff1ffb640 1 -- 172.21.15.5:0/2717630609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20000bbea0 con 0x5655542d8cb0 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.396+0000 7f2014ad2740 1 -- 172.21.15.5:0/2717630609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 msgr2=0x5655542db0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.396+0000 7f200bfff640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1ff40a3430 0x7f1ff40a58f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2004009fd0 tx=0x7f2004009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.396+0000 7f2014ad2740 1 --2- 172.21.15.5:0/2717630609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 0x5655542db0a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f20000ed870 tx=0x7f20000f8660 comp rx=0 tx=0).stop 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.396+0000 7f2014ad2740 1 -- 172.21.15.5:0/2717630609 shutdown_connections 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.396+0000 7f2014ad2740 1 --2- 172.21.15.5:0/2717630609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 0x5655542db0a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.396+0000 7f2014ad2740 1 --2- 172.21.15.5:0/2717630609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542ce410 0x5655542d8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.396+0000 7f2014ad2740 1 --2- 172.21.15.5:0/2717630609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542cdaf0 0x5655542cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.396+0000 7f2014ad2740 1 -- 172.21.15.5:0/2717630609 >> 172.21.15.5:0/2717630609 conn(0x5655542ca060 msgr2=0x5655542ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.396+0000 7f2014ad2740 1 -- 172.21.15.5:0/2717630609 shutdown_connections 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.396+0000 7f2014ad2740 1 -- 172.21.15.5:0/2717630609 wait complete. 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.396+0000 7f2014ad2740 1 -- start start 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f2014ad2740 1 Processor -- start 2024-09-19T06:09:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f2014ad2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542cdaf0 0x5655542d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f2014ad2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542ce410 0x5655542d60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f2014ad2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 0x5655542d6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f2014ad2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5655542db640 con 0x5655542d8cb0 2024-09-19T06:09:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f2014ad2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5655541bb290 con 0x5655542ce410 2024-09-19T06:09:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f2014ad2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5655542db7c0 con 0x5655542cdaf0 2024-09-19T06:09:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f2014ad2740 10 client.0 _ll_register_callbacks cb 0x5655541ec140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f200bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 0x5655542d6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f200bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 0x5655542d6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50804/0 (socket says 172.21.15.5:50804) 2024-09-19T06:09:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f200bfff640 1 -- 172.21.15.5:0/3463897646 learned_addr learned my addr 172.21.15.5:0/3463897646 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f200b7fe640 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542cdaf0 0x5655542d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f200affd640 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542ce410 0x5655542d60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f200affd640 1 -- 172.21.15.5:0/3463897646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542cdaf0 msgr2=0x5655542d5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f200affd640 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542cdaf0 0x5655542d5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f200affd640 1 -- 172.21.15.5:0/3463897646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 msgr2=0x5655542d6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f200affd640 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 0x5655542d6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f200affd640 1 -- 172.21.15.5:0/3463897646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565553f45210 con 0x5655542ce410 2024-09-19T06:09:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.397+0000 7f200bfff640 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 0x5655542d6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:09:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.398+0000 7f200affd640 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542ce410 0x5655542d60c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f20000ed870 tx=0x7f20000b88a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f84758c847121444 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.398+0000 7f1feaffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:09:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.398+0000 7f1feaffd640 1 -- 172.21.15.5:0/3463897646 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000048040 con 0x5655542ce410 2024-09-19T06:09:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.398+0000 7f2014ad2740 1 -- 172.21.15.5:0/3463897646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20000bc540 con 0x5655542ce410 2024-09-19T06:09:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.398+0000 7f1feaffd640 1 -- 172.21.15.5:0/3463897646 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f20000edb80 con 0x5655542ce410 2024-09-19T06:09:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.398+0000 7f1feaffd640 1 -- 172.21.15.5:0/3463897646 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f20000bc080 con 0x5655542ce410 2024-09-19T06:09:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.398+0000 7f2014ad2740 10 client.26705 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.398+0000 7f2014ad2740 1 -- 172.21.15.5:0/3463897646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x565554011620 con 0x5655542ce410 2024-09-19T06:09:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.398+0000 7f1feaffd640 1 -- 172.21.15.5:0/3463897646 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20000bc540 con 0x5655542ce410 2024-09-19T06:09:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.398+0000 7f1feaffd640 1 -- 172.21.15.5:0/3463897646 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f200004b060 con 0x5655542ce410 2024-09-19T06:09:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.399+0000 7f2014ad2740 10 client.26705 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.399+0000 7f2014ad2740 10 client.26705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.399+0000 7f2014ad2740 20 client.26705 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.399+0000 7f2014ad2740 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5655542e8800 0x5655542eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.399+0000 7f2014ad2740 4 client.26705 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:09:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.399+0000 7f2014ad2740 1 -- 172.21.15.5:0/3463897646 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56555423fa20 con 0x5655542e8800 2024-09-19T06:09:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.399+0000 7f200b7fe640 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5655542e8800 0x5655542eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.399+0000 7f1feaffd640 10 client.26705 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.399+0000 7f200b7fe640 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5655542e8800 0x5655542eabf0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c582695ad627a8 server_cookie=582a06687d6eb4bd in_seq=0 out_seq=0 2024-09-19T06:09:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.400+0000 7f1feaffd640 1 -- 172.21.15.5:0/3463897646 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56555423fa20 con 0x5655542e8800 2024-09-19T06:09:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.400+0000 7f1feaffd640 10 client.26705 handle_command_reply: tid=0 2024-09-19T06:09:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.400+0000 7f2014ad2740 1 client.26705 shutdown 2024-09-19T06:09:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.401+0000 7f2014ad2740 1 -- 172.21.15.5:0/3463897646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542ce410 msgr2=0x5655542d60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.401+0000 7f2014ad2740 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542ce410 0x5655542d60c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f20000ed870 tx=0x7f20000b88a0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.401+0000 7f2014ad2740 1 -- 172.21.15.5:0/3463897646 shutdown_connections 2024-09-19T06:09:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.401+0000 7f2014ad2740 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5655542e8800 0x5655542eabf0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.401+0000 7f2014ad2740 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542d8cb0 0x5655542d6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.401+0000 7f2014ad2740 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542ce410 0x5655542d60c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.401+0000 7f1feaffd640 0 client.26705 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.401+0000 7f2014ad2740 1 --2- 172.21.15.5:0/3463897646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542cdaf0 0x5655542d5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.401+0000 7f2014ad2740 1 -- 172.21.15.5:0/3463897646 >> 172.21.15.5:0/3463897646 conn(0x5655542ca060 msgr2=0x5655542cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.401+0000 7f2014ad2740 1 -- 172.21.15.5:0/3463897646 shutdown_connections 2024-09-19T06:09:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.401+0000 7f2014ad2740 1 -- 172.21.15.5:0/3463897646 wait complete. 2024-09-19T06:09:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.401+0000 7f2014ad2740 20 client.26705 trim_cache size 0 max 16384 2024-09-19T06:09:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f2014ad2740 1 Processor -- start 2024-09-19T06:09:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f2014ad2740 1 -- start start 2024-09-19T06:09:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f2014ad2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542cdaf0 0x5655542d6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f2014ad2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542ce410 0x5655542f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f2014ad2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542d8cb0 0x5655542f0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f2014ad2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1ad6c0 con 0x5655542ce410 2024-09-19T06:09:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f2014ad2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1af200 con 0x5655542d8cb0 2024-09-19T06:09:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f2014ad2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1add90 con 0x5655542cdaf0 2024-09-19T06:09:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f200affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542ce410 0x5655542f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f200affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542ce410 0x5655542f0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50820/0 (socket says 172.21.15.5:50820) 2024-09-19T06:09:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f200affd640 1 -- 172.21.15.5:0/873740556 learned_addr learned my addr 172.21.15.5:0/873740556 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f200bfff640 1 --2- 172.21.15.5:0/873740556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542d8cb0 0x5655542f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f200b7fe640 1 --2- 172.21.15.5:0/873740556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542cdaf0 0x5655542d6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f200affd640 1 -- 172.21.15.5:0/873740556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542cdaf0 msgr2=0x5655542d6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f200affd640 1 --2- 172.21.15.5:0/873740556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542cdaf0 0x5655542d6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f200affd640 1 -- 172.21.15.5:0/873740556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542d8cb0 msgr2=0x5655542f0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f200affd640 1 --2- 172.21.15.5:0/873740556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542d8cb0 0x5655542f0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f200affd640 1 -- 172.21.15.5:0/873740556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655542d1740 con 0x5655542ce410 2024-09-19T06:09:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.418+0000 7f200affd640 1 --2- 172.21.15.5:0/873740556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542ce410 0x5655542f0400 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2000006010 tx=0x7f20000bbea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8db410a41bd364 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.419+0000 7f1ff17fa640 1 -- 172.21.15.5:0/873740556 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000048070 con 0x5655542ce410 2024-09-19T06:09:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.419+0000 7f1ff17fa640 1 -- 172.21.15.5:0/873740556 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20000ed8a0 con 0x5655542ce410 2024-09-19T06:09:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.419+0000 7f1ff17fa640 1 -- 172.21.15.5:0/873740556 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200003b460 con 0x5655542ce410 2024-09-19T06:09:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.419+0000 7f2014ad2740 1 -- 172.21.15.5:0/873740556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542ce410 msgr2=0x5655542f0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.419+0000 7f2014ad2740 1 --2- 172.21.15.5:0/873740556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542ce410 0x5655542f0400 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2000006010 tx=0x7f20000bbea0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.419+0000 7f2014ad2740 1 -- 172.21.15.5:0/873740556 shutdown_connections 2024-09-19T06:09:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.419+0000 7f2014ad2740 1 --2- 172.21.15.5:0/873740556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542d8cb0 0x5655542f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.419+0000 7f2014ad2740 1 --2- 172.21.15.5:0/873740556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542ce410 0x5655542f0400 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.419+0000 7f2014ad2740 1 --2- 172.21.15.5:0/873740556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542cdaf0 0x5655542d6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.419+0000 7f2014ad2740 1 -- 172.21.15.5:0/873740556 >> 172.21.15.5:0/873740556 conn(0x5655542ca060 msgr2=0x5655542d7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.420+0000 7f2014ad2740 1 -- 172.21.15.5:0/873740556 shutdown_connections 2024-09-19T06:09:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.420+0000 7f2014ad2740 1 -- 172.21.15.5:0/873740556 wait complete. 2024-09-19T06:09:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.420+0000 7f2014ad2740 1 -- start start 2024-09-19T06:09:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.420+0000 7f2014ad2740 1 Processor -- start 2024-09-19T06:09:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.420+0000 7f2014ad2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542cdaf0 0x5655542e1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.420+0000 7f2014ad2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542ce410 0x5655542d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.420+0000 7f2014ad2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542d8cb0 0x5655542d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.420+0000 7f2014ad2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1aeb30 con 0x5655542cdaf0 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.420+0000 7f2014ad2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1ae460 con 0x5655542ce410 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.420+0000 7f2014ad2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1ab6a0 con 0x5655542d8cb0 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f2014ad2740 10 client.0 _ll_register_callbacks cb 0x7f2004009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f200b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542cdaf0 0x5655542e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f200b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542cdaf0 0x5655542e1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50830/0 (socket says 172.21.15.5:50830) 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f200b7fe640 1 -- 172.21.15.5:0/2141401206 learned_addr learned my addr 172.21.15.5:0/2141401206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f200affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542ce410 0x5655542d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f200bfff640 1 --2- 172.21.15.5:0/2141401206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542d8cb0 0x5655542d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f200b7fe640 1 -- 172.21.15.5:0/2141401206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542d8cb0 msgr2=0x5655542d2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f200b7fe640 1 --2- 172.21.15.5:0/2141401206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542d8cb0 0x5655542d2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f200b7fe640 1 -- 172.21.15.5:0/2141401206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542ce410 msgr2=0x5655542d2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f200b7fe640 1 --2- 172.21.15.5:0/2141401206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542ce410 0x5655542d2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f200b7fe640 1 -- 172.21.15.5:0/2141401206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655542d35d0 con 0x5655542cdaf0 2024-09-19T06:09:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f200b7fe640 1 --2- 172.21.15.5:0/2141401206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542cdaf0 0x5655542e1130 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0129c0 tx=0x7f1ffc012e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae0f563029c5aae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:09:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f1feaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:09:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f1feaffd640 1 -- 172.21.15.5:0/2141401206 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc01fb50 con 0x5655542cdaf0 2024-09-19T06:09:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f1feaffd640 1 -- 172.21.15.5:0/2141401206 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8506+0+0 (secure 0 0 0) 0x7f1ffc01fcf0 con 0x5655542cdaf0 2024-09-19T06:09:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f2014ad2740 1 -- 172.21.15.5:0/2141401206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20000ed8a0 con 0x5655542cdaf0 2024-09-19T06:09:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f1feaffd640 1 -- 172.21.15.5:0/2141401206 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1ffc005020 con 0x5655542cdaf0 2024-09-19T06:09:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f2014ad2740 10 client.17133 fetch_fsmap learned FSMap version 33 2024-09-19T06:09:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.421+0000 7f2014ad2740 1 -- 172.21.15.5:0/2141401206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5655543533b0 con 0x5655542cdaf0 2024-09-19T06:09:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.422+0000 7f1feaffd640 1 -- 172.21.15.5:0/2141401206 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20000ed8a0 con 0x5655542cdaf0 2024-09-19T06:09:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.422+0000 7f1feaffd640 1 -- 172.21.15.5:0/2141401206 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f1ffc004c90 con 0x5655542cdaf0 2024-09-19T06:09:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.422+0000 7f2014ad2740 10 client.17133 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:09:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.422+0000 7f2014ad2740 10 client.17133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:09:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.422+0000 7f2014ad2740 20 client.17133 populate_metadata read hostname 'smithi005' 2024-09-19T06:09:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.422+0000 7f2014ad2740 1 --2- 172.21.15.5:0/2141401206 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5655542e8800 0x565554355b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:09:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.422+0000 7f2014ad2740 4 client.17133 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:09:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.422+0000 7f2014ad2740 1 -- 172.21.15.5:0/2141401206 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565554239d70 con 0x5655542e8800 2024-09-19T06:09:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.422+0000 7f200affd640 1 --2- 172.21.15.5:0/2141401206 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5655542e8800 0x565554355b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:09:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.422+0000 7f1feaffd640 10 client.17133 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.422+0000 7f200affd640 1 --2- 172.21.15.5:0/2141401206 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5655542e8800 0x565554355b40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef72014e8da353f0 server_cookie=40ae863781227b7d in_seq=0 out_seq=0 2024-09-19T06:09:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.444+0000 7f1feaffd640 1 -- 172.21.15.5:0/2141401206 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565554239d70 con 0x5655542e8800 2024-09-19T06:09:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.444+0000 7f1feaffd640 10 client.17133 handle_command_reply: tid=0 2024-09-19T06:09:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.444+0000 7f2014ad2740 1 client.17133 shutdown 2024-09-19T06:09:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2014ad2740 1 -- 172.21.15.5:0/2141401206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542cdaf0 msgr2=0x5655542e1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2014ad2740 1 --2- 172.21.15.5:0/2141401206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542cdaf0 0x5655542e1130 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0129c0 tx=0x7f1ffc012e80 comp rx=0 tx=0).stop 2024-09-19T06:09:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2014ad2740 1 -- 172.21.15.5:0/2141401206 shutdown_connections 2024-09-19T06:09:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2014ad2740 1 --2- 172.21.15.5:0/2141401206 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5655542e8800 0x565554355b40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2014ad2740 1 --2- 172.21.15.5:0/2141401206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655542d8cb0 0x5655542d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2014ad2740 1 --2- 172.21.15.5:0/2141401206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5655542ce410 0x5655542d2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2014ad2740 1 --2- 172.21.15.5:0/2141401206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655542cdaf0 0x5655542e1130 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f1feaffd640 0 client.17133 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2014ad2740 1 -- 172.21.15.5:0/2141401206 >> 172.21.15.5:0/2141401206 conn(0x5655542ca060 msgr2=0x5655542c89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2014ad2740 1 -- 172.21.15.5:0/2141401206 shutdown_connections 2024-09-19T06:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2014ad2740 1 -- 172.21.15.5:0/2141401206 wait complete. 2024-09-19T06:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2014ad2740 20 client.17133 trim_cache size 0 max 16384 2024-09-19T06:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:09:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:09:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2011d31640 1 -- 172.21.15.5:0/387485661 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1ff40a3430 msgr2=0x7f1ff40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2011d31640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1ff40a3430 0x7f1ff40a58f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2004009fd0 tx=0x7f2004009500 comp rx=0 tx=0).stop 2024-09-19T06:09:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2011d31640 1 -- 172.21.15.5:0/387485661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f200c101340 msgr2=0x7f200c198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.445+0000 7f2011d31640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f200c101340 0x7f200c198920 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2000031440 tx=0x7f2000004270 comp rx=0 tx=0).stop 2024-09-19T06:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.446+0000 7f2011d31640 1 -- 172.21.15.5:0/387485661 shutdown_connections 2024-09-19T06:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.446+0000 7f2011d31640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200c1069b0 0x7f200c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.446+0000 7f2011d31640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f200c101340 0x7f200c198920 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.446+0000 7f2011d31640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1ff40a3430 0x7f1ff40a58f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.446+0000 7f2011d31640 1 --2- 172.21.15.5:0/387485661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c1009a0 0x7f200c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.446+0000 7f2011d31640 1 -- 172.21.15.5:0/387485661 >> 172.21.15.5:0/387485661 conn(0x7f200c0779a0 msgr2=0x7f200c069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.446+0000 7f2011d31640 1 -- 172.21.15.5:0/387485661 shutdown_connections 2024-09-19T06:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:09:53.446+0000 7f2011d31640 1 -- 172.21.15.5:0/387485661 wait complete. 2024-09-19T06:09:53.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 163442 2.75 MB/sec execute 199 sec latency 15.011 ms 2024-09-19T06:09:54.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:54 smithi157 ceph-mon[35346]: pgmap v200: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-19T06:09:54.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:54 smithi005 ceph-mon[30955]: pgmap v200: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-19T06:09:54.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:54 smithi049 ceph-mon[34640]: pgmap v200: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-19T06:09:54.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 163710 2.74 MB/sec execute 200 sec latency 15.398 ms 2024-09-19T06:09:55.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 164170 2.73 MB/sec execute 201 sec latency 13.665 ms 2024-09-19T06:09:56.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:56 smithi157 ceph-mon[35346]: pgmap v201: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-09-19T06:09:56.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:56 smithi005 ceph-mon[30955]: pgmap v201: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-09-19T06:09:56.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:56 smithi049 ceph-mon[34640]: pgmap v201: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-09-19T06:09:56.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 164716 2.74 MB/sec execute 202 sec latency 19.872 ms 2024-09-19T06:09:57.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 165274 2.72 MB/sec execute 203 sec latency 19.390 ms 2024-09-19T06:09:58.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:58 smithi157 ceph-mon[35346]: pgmap v202: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-19T06:09:58.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:58 smithi005 ceph-mon[30955]: pgmap v202: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-19T06:09:58.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:58 smithi049 ceph-mon[34640]: pgmap v202: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-19T06:09:58.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 165790 2.72 MB/sec execute 204 sec latency 19.028 ms 2024-09-19T06:09:59.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:09:59 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:09:59.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:09:59 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:09:59.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:09:59 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:09:59.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 166497 2.74 MB/sec execute 205 sec latency 15.409 ms 2024-09-19T06:10:00.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:00 smithi157 ceph-mon[35346]: pgmap v203: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-09-19T06:10:00.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:00 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:10:00.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:00 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:10:00.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:00 smithi157 ceph-mon[35346]: overall HEALTH_OK 2024-09-19T06:10:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:00 smithi005 ceph-mon[30955]: pgmap v203: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-09-19T06:10:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:00 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:10:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:00 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:10:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:00 smithi005 ceph-mon[30955]: overall HEALTH_OK 2024-09-19T06:10:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:00 smithi049 ceph-mon[34640]: pgmap v203: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-09-19T06:10:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:00 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:10:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:00 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:10:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:00 smithi049 ceph-mon[34640]: overall HEALTH_OK 2024-09-19T06:10:00.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 166963 2.75 MB/sec execute 206 sec latency 16.407 ms 2024-09-19T06:10:01.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 167218 2.74 MB/sec execute 207 sec latency 12.381 ms 2024-09-19T06:10:02.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:02 smithi005 ceph-mon[30955]: pgmap v204: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-09-19T06:10:02.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:02 smithi049 ceph-mon[34640]: pgmap v204: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-09-19T06:10:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:02 smithi157 ceph-mon[35346]: pgmap v204: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-09-19T06:10:02.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 167676 2.73 MB/sec execute 208 sec latency 17.303 ms 2024-09-19T06:10:03.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 168241 2.74 MB/sec execute 209 sec latency 19.814 ms 2024-09-19T06:10:04.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:04 smithi005 ceph-mon[30955]: pgmap v205: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-09-19T06:10:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:04 smithi049 ceph-mon[34640]: pgmap v205: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-09-19T06:10:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:04 smithi157 ceph-mon[35346]: pgmap v205: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-09-19T06:10:04.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 168797 2.72 MB/sec execute 210 sec latency 20.920 ms 2024-09-19T06:10:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:05 smithi005 ceph-mon[30955]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T06:10:05.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:05 smithi049 ceph-mon[34640]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T06:10:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:05 smithi157 ceph-mon[35346]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T06:10:05.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 169313 2.72 MB/sec execute 211 sec latency 20.607 ms 2024-09-19T06:10:06.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.542+0000 7f069645e640 1 -- 172.21.15.5:0/2615333221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0690102460 msgr2=0x7f069010e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:06.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.542+0000 7f069645e640 1 --2- 172.21.15.5:0/2615333221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0690102460 0x7f069010e990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0688009cd0 tx=0x7f068802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.542+0000 7f069645e640 1 -- 172.21.15.5:0/2615333221 shutdown_connections 2024-09-19T06:10:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.542+0000 7f069645e640 1 --2- 172.21.15.5:0/2615333221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0690102460 0x7f069010e990 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.542+0000 7f069645e640 1 --2- 172.21.15.5:0/2615333221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0690101ac0 0x7f0690101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.542+0000 7f069645e640 1 --2- 172.21.15.5:0/2615333221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0690107ad0 0x7f0690107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.542+0000 7f069645e640 1 -- 172.21.15.5:0/2615333221 >> 172.21.15.5:0/2615333221 conn(0x7f06900fd7a0 msgr2=0x7f06900ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.543+0000 7f069645e640 1 -- 172.21.15.5:0/2615333221 shutdown_connections 2024-09-19T06:10:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.543+0000 7f069645e640 1 -- 172.21.15.5:0/2615333221 wait complete. 2024-09-19T06:10:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.543+0000 7f069645e640 1 Processor -- start 2024-09-19T06:10:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.543+0000 7f069645e640 1 -- start start 2024-09-19T06:10:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.543+0000 7f069645e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0690101ac0 0x7f06901982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.543+0000 7f069645e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0690102460 0x7f0690198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.543+0000 7f069645e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0690107ad0 0x7f069019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.543+0000 7f069645e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0690111e10 con 0x7f0690107ad0 2024-09-19T06:10:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.543+0000 7f069645e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0690111c90 con 0x7f0690101ac0 2024-09-19T06:10:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.543+0000 7f069645e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0690111f90 con 0x7f0690102460 2024-09-19T06:10:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f06949fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0690107ad0 0x7f069019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f06949fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0690107ad0 0x7f069019cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35198/0 (socket says 172.21.15.5:35198) 2024-09-19T06:10:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f06949fb640 1 -- 172.21.15.5:0/1122284683 learned_addr learned my addr 172.21.15.5:0/1122284683 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f068ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0690101ac0 0x7f06901982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f068f7fe640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0690102460 0x7f0690198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f068ffff640 1 -- 172.21.15.5:0/1122284683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0690102460 msgr2=0x7f0690198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f068ffff640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0690102460 0x7f0690198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f068ffff640 1 -- 172.21.15.5:0/1122284683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0690107ad0 msgr2=0x7f069019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f068ffff640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0690107ad0 0x7f069019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f068ffff640 1 -- 172.21.15.5:0/1122284683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069019d2d0 con 0x7f0690101ac0 2024-09-19T06:10:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f06949fb640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0690107ad0 0x7f069019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f068ffff640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0690101ac0 0x7f06901982e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f068000e9f0 tx=0x7f068000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f068d7fa640 1 -- 172.21.15.5:0/1122284683 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068000ccc0 con 0x7f0690101ac0 2024-09-19T06:10:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f068d7fa640 1 -- 172.21.15.5:0/1122284683 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06800108e0 con 0x7f0690101ac0 2024-09-19T06:10:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f069645e640 1 -- 172.21.15.5:0/1122284683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069019d5c0 con 0x7f0690101ac0 2024-09-19T06:10:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f068d7fa640 1 -- 172.21.15.5:0/1122284683 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680018880 con 0x7f0690101ac0 2024-09-19T06:10:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.544+0000 7f069645e640 1 -- 172.21.15.5:0/1122284683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06901a4e50 con 0x7f0690101ac0 2024-09-19T06:10:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f068d7fa640 1 -- 172.21.15.5:0/1122284683 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0680010a90 con 0x7f0690101ac0 2024-09-19T06:10:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f06991f6740 1 Processor -- start 2024-09-19T06:10:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f06991f6740 1 -- start start 2024-09-19T06:10:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f06991f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9d3f00 0x555fec9d42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f06991f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 0x555fec9deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f06991f6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9df0c0 0x555fec9e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f06991f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555fec9342f0 con 0x555fec9d4820 2024-09-19T06:10:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f06991f6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555fec8dc270 con 0x555fec9df0c0 2024-09-19T06:10:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f06991f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555fec9302b0 con 0x555fec9d3f00 2024-09-19T06:10:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f068ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 0x555fec9deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f06949fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9d3f00 0x555fec9d42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f068ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 0x555fec9deb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35204/0 (socket says 172.21.15.5:35204) 2024-09-19T06:10:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.547+0000 7f068ffff640 1 -- 172.21.15.5:0/2364173348 learned_addr learned my addr 172.21.15.5:0/2364173348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f068d7fa640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f06780a33b0 0x7f06780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f068f7fe640 1 --2- 172.21.15.5:0/2364173348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9df0c0 0x555fec9e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f06949fb640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f06780a33b0 0x7f06780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f068d7fa640 1 -- 172.21.15.5:0/1122284683 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8594+0+0 (secure 0 0 0) 0x7f06800d0030 con 0x7f0690101ac0 2024-09-19T06:10:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f068ffff640 1 -- 172.21.15.5:0/2364173348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9d3f00 msgr2=0x555fec9d42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f068ffff640 1 --2- 172.21.15.5:0/2364173348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9d3f00 0x555fec9d42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f068ffff640 1 -- 172.21.15.5:0/2364173348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9df0c0 msgr2=0x555fec9e14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f068ffff640 1 --2- 172.21.15.5:0/2364173348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9df0c0 0x555fec9e14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f068ffff640 1 -- 172.21.15.5:0/2364173348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fec8db340 con 0x555fec9d4820 2024-09-19T06:10:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f06949fb640 1 --2- 172.21.15.5:0/2364173348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9d3f00 0x555fec9d42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f068f7fe640 1 --2- 172.21.15.5:0/2364173348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9df0c0 0x555fec9e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f068ffff640 1 --2- 172.21.15.5:0/2364173348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 0x555fec9deb80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0680098d40 tx=0x7f0680009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cc01cdffe6ab7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f06767fc640 1 -- 172.21.15.5:0/2364173348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068009c370 con 0x555fec9d4820 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f06949fb640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f06780a33b0 0x7f06780a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0688009a60 tx=0x7f068802a8e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f06767fc640 1 -- 172.21.15.5:0/2364173348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06800a0670 con 0x555fec9d4820 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.548+0000 7f06767fc640 1 -- 172.21.15.5:0/2364173348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06800bc0e0 con 0x555fec9d4820 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.549+0000 7f06991f6740 1 -- 172.21.15.5:0/2364173348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 msgr2=0x555fec9deb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.549+0000 7f06991f6740 1 --2- 172.21.15.5:0/2364173348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 0x555fec9deb80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0680098d40 tx=0x7f0680009700 comp rx=0 tx=0).stop 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.549+0000 7f06991f6740 1 -- 172.21.15.5:0/2364173348 shutdown_connections 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.549+0000 7f06991f6740 1 --2- 172.21.15.5:0/2364173348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9df0c0 0x555fec9e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.549+0000 7f06991f6740 1 --2- 172.21.15.5:0/2364173348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 0x555fec9deb80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.549+0000 7f06991f6740 1 --2- 172.21.15.5:0/2364173348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9d3f00 0x555fec9d42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.549+0000 7f06991f6740 1 -- 172.21.15.5:0/2364173348 >> 172.21.15.5:0/2364173348 conn(0x555fec9d0470 msgr2=0x555fec9d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.549+0000 7f06991f6740 1 -- 172.21.15.5:0/2364173348 shutdown_connections 2024-09-19T06:10:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.549+0000 7f06991f6740 1 -- 172.21.15.5:0/2364173348 wait complete. 2024-09-19T06:10:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.549+0000 7f06991f6740 1 -- start start 2024-09-19T06:10:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f06991f6740 1 Processor -- start 2024-09-19T06:10:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f06991f6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9d3f00 0x555fec9dc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f06991f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 0x555fec9dc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f06991f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9df0c0 0x555fec9dca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f06991f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555fec9e1c90 con 0x555fec9d4820 2024-09-19T06:10:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f06991f6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555fec8bd290 con 0x555fec9d3f00 2024-09-19T06:10:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f06991f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555fec9e1e10 con 0x555fec9df0c0 2024-09-19T06:10:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f06991f6740 10 client.0 _ll_register_callbacks cb 0x555fec8ee140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f068f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 0x555fec9dc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f068ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9d3f00 0x555fec9dc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f06949fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9df0c0 0x555fec9dca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f068ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9d3f00 0x555fec9dc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:47774/0 (socket says 172.21.15.5:47774) 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f068f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 0x555fec9dc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35218/0 (socket says 172.21.15.5:35218) 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f068ffff640 1 -- 172.21.15.5:0/2305612421 learned_addr learned my addr 172.21.15.5:0/2305612421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f068ffff640 1 -- 172.21.15.5:0/2305612421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9df0c0 msgr2=0x555fec9dca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f068ffff640 1 --2- 172.21.15.5:0/2305612421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9df0c0 0x555fec9dca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f068ffff640 1 -- 172.21.15.5:0/2305612421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 msgr2=0x555fec9dc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f068ffff640 1 --2- 172.21.15.5:0/2305612421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 0x555fec9dc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f068ffff640 1 -- 172.21.15.5:0/2305612421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fec647210 con 0x555fec9d3f00 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f068ffff640 1 --2- 172.21.15.5:0/2305612421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9d3f00 0x555fec9dc010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06800cd7e0 tx=0x7f06800983a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a82a2a2e2fdbf25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.550+0000 7f066f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f066f7fe640 1 -- 172.21.15.5:0/2305612421 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06800a4f30 con 0x555fec9d3f00 2024-09-19T06:10:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f066f7fe640 1 -- 172.21.15.5:0/2305612421 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8594+0+0 (secure 0 0 0) 0x7f06800cd9b0 con 0x555fec9d3f00 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f06991f6740 1 -- 172.21.15.5:0/2305612421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06800a0670 con 0x555fec9d3f00 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f066f7fe640 1 -- 172.21.15.5:0/2305612421 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f06800b52b0 con 0x555fec9d3f00 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f06991f6740 10 client.26723 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f06991f6740 1 -- 172.21.15.5:0/2305612421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555fec713620 con 0x555fec9d3f00 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f066f7fe640 1 -- 172.21.15.5:0/2305612421 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06800a0670 con 0x555fec9d3f00 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f066f7fe640 1 -- 172.21.15.5:0/2305612421 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f068009b950 con 0x555fec9d3f00 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f06991f6740 10 client.26723 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f06991f6740 10 client.26723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f06991f6740 20 client.26723 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f06991f6740 1 --2- 172.21.15.5:0/2305612421 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555fec9eef10 0x555fec9f1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f06991f6740 4 client.26723 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f06991f6740 1 -- 172.21.15.5:0/2305612421 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fec941a20 con 0x555fec9eef10 2024-09-19T06:10:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.551+0000 7f068f7fe640 1 --2- 172.21.15.5:0/2305612421 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555fec9eef10 0x555fec9f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.552+0000 7f066f7fe640 10 client.26723 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.552+0000 7f068f7fe640 1 --2- 172.21.15.5:0/2305612421 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555fec9eef10 0x555fec9f1300 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c1dda028732035e server_cookie=a189e44841b194f in_seq=0 out_seq=0 2024-09-19T06:10:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.552+0000 7f066f7fe640 1 -- 172.21.15.5:0/2305612421 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555fec941a20 con 0x555fec9eef10 2024-09-19T06:10:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.552+0000 7f066f7fe640 10 client.26723 handle_command_reply: tid=0 2024-09-19T06:10:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f06991f6740 1 client.26723 shutdown 2024-09-19T06:10:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f06991f6740 1 -- 172.21.15.5:0/2305612421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9d3f00 msgr2=0x555fec9dc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f06991f6740 1 --2- 172.21.15.5:0/2305612421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9d3f00 0x555fec9dc010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06800cd7e0 tx=0x7f06800983a0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f06991f6740 1 -- 172.21.15.5:0/2305612421 shutdown_connections 2024-09-19T06:10:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f06991f6740 1 --2- 172.21.15.5:0/2305612421 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555fec9eef10 0x555fec9f1300 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f06991f6740 1 --2- 172.21.15.5:0/2305612421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9df0c0 0x555fec9dca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f066f7fe640 0 client.26723 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f06991f6740 1 --2- 172.21.15.5:0/2305612421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d4820 0x555fec9dc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f06991f6740 1 --2- 172.21.15.5:0/2305612421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9d3f00 0x555fec9dc010 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f06991f6740 1 -- 172.21.15.5:0/2305612421 >> 172.21.15.5:0/2305612421 conn(0x555fec9d0470 msgr2=0x555fec9d6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f06991f6740 1 -- 172.21.15.5:0/2305612421 shutdown_connections 2024-09-19T06:10:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f06991f6740 1 -- 172.21.15.5:0/2305612421 wait complete. 2024-09-19T06:10:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.553+0000 7f06991f6740 20 client.26723 trim_cache size 0 max 16384 2024-09-19T06:10:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.568+0000 7f06991f6740 1 Processor -- start 2024-09-19T06:10:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.568+0000 7f06991f6740 1 -- start start 2024-09-19T06:10:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.568+0000 7f06991f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 0x555fec9cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.568+0000 7f06991f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9d4820 0x555fec9f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.568+0000 7f06991f6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9df0c0 0x555fec9f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.568+0000 7f06991f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06901ab6c0 con 0x555fec9d3f00 2024-09-19T06:10:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.568+0000 7f06991f6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f06901ad200 con 0x555fec9df0c0 2024-09-19T06:10:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.568+0000 7f06991f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f06901abd90 con 0x555fec9d4820 2024-09-19T06:10:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.568+0000 7f068ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 0x555fec9cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f06949fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9df0c0 0x555fec9f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f068ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 0x555fec9cb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35232/0 (socket says 172.21.15.5:35232) 2024-09-19T06:10:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f068f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9d4820 0x555fec9f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f068ffff640 1 -- 172.21.15.5:0/4208551762 learned_addr learned my addr 172.21.15.5:0/4208551762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f06949fb640 1 -- 172.21.15.5:0/4208551762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9d4820 msgr2=0x555fec9f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f06949fb640 1 --2- 172.21.15.5:0/4208551762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9d4820 0x555fec9f68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f06949fb640 1 -- 172.21.15.5:0/4208551762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 msgr2=0x555fec9cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f06949fb640 1 --2- 172.21.15.5:0/4208551762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 0x555fec9cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f06949fb640 1 -- 172.21.15.5:0/4208551762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fec9d4650 con 0x555fec9df0c0 2024-09-19T06:10:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f068ffff640 1 --2- 172.21.15.5:0/4208551762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 0x555fec9cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f06949fb640 1 --2- 172.21.15.5:0/4208551762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9df0c0 0x555fec9f6de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0688006010 tx=0x7f0688005f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb3f1b1f4b227766 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f0675ffb640 1 -- 172.21.15.5:0/4208551762 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068804f070 con 0x555fec9df0c0 2024-09-19T06:10:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f0675ffb640 1 -- 172.21.15.5:0/4208551762 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0688036050 con 0x555fec9df0c0 2024-09-19T06:10:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.569+0000 7f0675ffb640 1 -- 172.21.15.5:0/4208551762 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068804b680 con 0x555fec9df0c0 2024-09-19T06:10:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.570+0000 7f06991f6740 1 -- 172.21.15.5:0/4208551762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9df0c0 msgr2=0x555fec9f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.570+0000 7f06991f6740 1 --2- 172.21.15.5:0/4208551762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9df0c0 0x555fec9f6de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0688006010 tx=0x7f0688005f50 comp rx=0 tx=0).stop 2024-09-19T06:10:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.570+0000 7f06991f6740 1 -- 172.21.15.5:0/4208551762 shutdown_connections 2024-09-19T06:10:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.570+0000 7f06991f6740 1 --2- 172.21.15.5:0/4208551762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9df0c0 0x555fec9f6de0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.570+0000 7f06991f6740 1 --2- 172.21.15.5:0/4208551762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9d4820 0x555fec9f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.570+0000 7f06991f6740 1 --2- 172.21.15.5:0/4208551762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 0x555fec9cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.570+0000 7f06991f6740 1 -- 172.21.15.5:0/4208551762 >> 172.21.15.5:0/4208551762 conn(0x555fec9d0470 msgr2=0x555fec9ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.570+0000 7f06991f6740 1 -- 172.21.15.5:0/4208551762 shutdown_connections 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.570+0000 7f06991f6740 1 -- 172.21.15.5:0/4208551762 wait complete. 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.570+0000 7f06991f6740 1 -- start start 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f06991f6740 1 Processor -- start 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f06991f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 0x555fec9e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f06991f6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9d4820 0x555fec9e3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f06991f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9df0c0 0x555feca52840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f06991f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06901acb30 con 0x555fec9d3f00 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f06991f6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f06901ac460 con 0x555fec9d4820 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f06991f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f06901a96c0 con 0x555fec9df0c0 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f06991f6740 10 client.0 _ll_register_callbacks cb 0x7f068400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f068ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 0x555fec9e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f068ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 0x555fec9e36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35234/0 (socket says 172.21.15.5:35234) 2024-09-19T06:10:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f068ffff640 1 -- 172.21.15.5:0/71341405 learned_addr learned my addr 172.21.15.5:0/71341405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f06949fb640 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9df0c0 0x555feca52840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f068f7fe640 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9d4820 0x555fec9e3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f068ffff640 1 -- 172.21.15.5:0/71341405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9df0c0 msgr2=0x555feca52840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f068ffff640 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9df0c0 0x555feca52840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f068ffff640 1 -- 172.21.15.5:0/71341405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9d4820 msgr2=0x555fec9e3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.571+0000 7f068ffff640 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9d4820 0x555fec9e3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f068ffff640 1 -- 172.21.15.5:0/71341405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555feca52e80 con 0x555fec9d3f00 2024-09-19T06:10:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f06949fb640 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9df0c0 0x555feca52840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f068ffff640 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 0x555fec9e36a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f06800cdd30 tx=0x7f0680098000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4930f43f381594 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f066f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:10:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f066f7fe640 1 -- 172.21.15.5:0/71341405 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068009b5e0 con 0x555fec9d3f00 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f066f7fe640 1 -- 172.21.15.5:0/71341405 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8594+0+0 (secure 0 0 0) 0x7f06800a0170 con 0x555fec9d3f00 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f06991f6740 1 -- 172.21.15.5:0/71341405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0688036050 con 0x555fec9d3f00 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f066f7fe640 1 -- 172.21.15.5:0/71341405 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f06800b52b0 con 0x555fec9d3f00 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f06991f6740 10 client.17151 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f06991f6740 1 -- 172.21.15.5:0/71341405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555feca5a240 con 0x555fec9d3f00 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f066f7fe640 1 -- 172.21.15.5:0/71341405 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0688036050 con 0x555fec9d3f00 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f066f7fe640 1 -- 172.21.15.5:0/71341405 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f068009b7c0 con 0x555fec9d3f00 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f06991f6740 10 client.17151 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f06991f6740 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f06991f6740 20 client.17151 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f06991f6740 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555feca5a720 0x555feca5cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f06991f6740 4 client.17151 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:10:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.572+0000 7f06991f6740 1 -- 172.21.15.5:0/71341405 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555fec93bd70 con 0x555feca5a720 2024-09-19T06:10:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.573+0000 7f068f7fe640 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555feca5a720 0x555feca5cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.573+0000 7f066f7fe640 10 client.17151 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.573+0000 7f068f7fe640 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555feca5a720 0x555feca5cb10 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7107715efb2c1ea5 server_cookie=9e39b1ffc2c5e1d8 in_seq=0 out_seq=0 2024-09-19T06:10:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f066f7fe640 1 -- 172.21.15.5:0/71341405 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555fec93bd70 con 0x555feca5a720 2024-09-19T06:10:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f066f7fe640 10 client.17151 handle_command_reply: tid=0 2024-09-19T06:10:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f06991f6740 1 client.17151 shutdown 2024-09-19T06:10:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f06991f6740 1 -- 172.21.15.5:0/71341405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 msgr2=0x555fec9e36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f06991f6740 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 0x555fec9e36a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f06800cdd30 tx=0x7f0680098000 comp rx=0 tx=0).stop 2024-09-19T06:10:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f06991f6740 1 -- 172.21.15.5:0/71341405 shutdown_connections 2024-09-19T06:10:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f06991f6740 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555feca5a720 0x555feca5cb10 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f06991f6740 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fec9df0c0 0x555feca52840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f066f7fe640 0 client.17151 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f06991f6740 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555fec9d4820 0x555fec9e3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f06991f6740 1 --2- 172.21.15.5:0/71341405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fec9d3f00 0x555fec9e36a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f06991f6740 1 -- 172.21.15.5:0/71341405 >> 172.21.15.5:0/71341405 conn(0x555fec9d0470 msgr2=0x555fec9ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f06991f6740 1 -- 172.21.15.5:0/71341405 shutdown_connections 2024-09-19T06:10:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f06991f6740 1 -- 172.21.15.5:0/71341405 wait complete. 2024-09-19T06:10:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.596+0000 7f06991f6740 20 client.17151 trim_cache size 0 max 16384 2024-09-19T06:10:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:10:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:10:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:10:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:10:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.597+0000 7f069645e640 1 -- 172.21.15.5:0/1122284683 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f06780a33b0 msgr2=0x7f06780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.597+0000 7f069645e640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f06780a33b0 0x7f06780a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0688009a60 tx=0x7f068802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.597+0000 7f069645e640 1 -- 172.21.15.5:0/1122284683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0690101ac0 msgr2=0x7f06901982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.597+0000 7f069645e640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0690101ac0 0x7f06901982e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f068000e9f0 tx=0x7f068000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.597+0000 7f069645e640 1 -- 172.21.15.5:0/1122284683 shutdown_connections 2024-09-19T06:10:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.597+0000 7f069645e640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0690107ad0 0x7f069019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.597+0000 7f069645e640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f06780a33b0 0x7f06780a5870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.597+0000 7f069645e640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0690102460 0x7f0690198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.597+0000 7f069645e640 1 --2- 172.21.15.5:0/1122284683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0690101ac0 0x7f06901982e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.597+0000 7f069645e640 1 -- 172.21.15.5:0/1122284683 >> 172.21.15.5:0/1122284683 conn(0x7f06900fd7a0 msgr2=0x7f06900fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.597+0000 7f069645e640 1 -- 172.21.15.5:0/1122284683 shutdown_connections 2024-09-19T06:10:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:06.597+0000 7f069645e640 1 -- 172.21.15.5:0/1122284683 wait complete. 2024-09-19T06:10:06.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:06 smithi005 ceph-mon[30955]: pgmap v207: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-09-19T06:10:06.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:06 smithi049 ceph-mon[34640]: pgmap v207: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-09-19T06:10:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:06 smithi157 ceph-mon[35346]: pgmap v207: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-09-19T06:10:06.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 170037 2.74 MB/sec execute 212 sec latency 21.326 ms 2024-09-19T06:10:07.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 170507 2.75 MB/sec execute 213 sec latency 12.466 ms 2024-09-19T06:10:08.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:08 smithi005 ceph-mon[30955]: pgmap v208: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-19T06:10:08.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:08 smithi005 ceph-mon[30955]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T06:10:08.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:08 smithi049 ceph-mon[34640]: pgmap v208: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-19T06:10:08.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:08 smithi049 ceph-mon[34640]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T06:10:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:08 smithi157 ceph-mon[35346]: pgmap v208: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-19T06:10:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:08 smithi157 ceph-mon[35346]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T06:10:08.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 170744 2.74 MB/sec execute 214 sec latency 15.459 ms 2024-09-19T06:10:09.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 171147 2.73 MB/sec execute 215 sec latency 17.262 ms 2024-09-19T06:10:10.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:10 smithi005 ceph-mon[30955]: pgmap v210: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.2 MiB/s wr, 214 op/s 2024-09-19T06:10:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:10 smithi049 ceph-mon[34640]: pgmap v210: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.2 MiB/s wr, 214 op/s 2024-09-19T06:10:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:10 smithi157 ceph-mon[35346]: pgmap v210: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.2 MiB/s wr, 214 op/s 2024-09-19T06:10:10.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 171698 2.73 MB/sec execute 216 sec latency 18.305 ms 2024-09-19T06:10:11.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 172191 2.72 MB/sec execute 217 sec latency 20.133 ms 2024-09-19T06:10:12.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:12 smithi005 ceph-mon[30955]: pgmap v211: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-09-19T06:10:12.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:12 smithi049 ceph-mon[34640]: pgmap v211: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-09-19T06:10:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:12 smithi157 ceph-mon[35346]: pgmap v211: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-09-19T06:10:12.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 172728 2.72 MB/sec execute 218 sec latency 23.087 ms 2024-09-19T06:10:13.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 173481 2.74 MB/sec execute 219 sec latency 15.821 ms 2024-09-19T06:10:14.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:14 smithi005 ceph-mon[30955]: pgmap v212: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-09-19T06:10:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:14 smithi049 ceph-mon[34640]: pgmap v212: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-09-19T06:10:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:14 smithi157 ceph-mon[35346]: pgmap v212: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-09-19T06:10:14.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 173915 2.74 MB/sec execute 220 sec latency 16.569 ms 2024-09-19T06:10:15.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 174238 2.74 MB/sec execute 221 sec latency 12.900 ms 2024-09-19T06:10:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:16 smithi005 ceph-mon[30955]: pgmap v213: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-09-19T06:10:16.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:16 smithi049 ceph-mon[34640]: pgmap v213: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-09-19T06:10:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:16 smithi157 ceph-mon[35346]: pgmap v213: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-09-19T06:10:16.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 174637 2.73 MB/sec execute 222 sec latency 15.364 ms 2024-09-19T06:10:17.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 175210 2.73 MB/sec execute 223 sec latency 19.597 ms 2024-09-19T06:10:18.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:18 smithi005 ceph-mon[30955]: pgmap v214: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.6 MiB/s wr, 150 op/s 2024-09-19T06:10:18.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:18 smithi049 ceph-mon[34640]: pgmap v214: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.6 MiB/s wr, 150 op/s 2024-09-19T06:10:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:18 smithi157 ceph-mon[35346]: pgmap v214: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.6 MiB/s wr, 150 op/s 2024-09-19T06:10:18.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 175693 2.72 MB/sec execute 224 sec latency 20.212 ms 2024-09-19T06:10:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.694+0000 7fc04feba640 1 -- 172.21.15.5:0/905016623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc048107ad0 msgr2=0x7fc048107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.694+0000 7fc04feba640 1 --2- 172.21.15.5:0/905016623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc048107ad0 0x7fc048107eb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc044009930 tx=0x7fc04402a420 comp rx=0 tx=0).stop 2024-09-19T06:10:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.694+0000 7fc04feba640 1 -- 172.21.15.5:0/905016623 shutdown_connections 2024-09-19T06:10:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.694+0000 7fc04feba640 1 --2- 172.21.15.5:0/905016623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc048102460 0x7fc04810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.694+0000 7fc04feba640 1 --2- 172.21.15.5:0/905016623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc048101ac0 0x7fc048101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.694+0000 7fc04feba640 1 --2- 172.21.15.5:0/905016623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc048107ad0 0x7fc048107eb0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.694+0000 7fc04feba640 1 -- 172.21.15.5:0/905016623 >> 172.21.15.5:0/905016623 conn(0x7fc0480fd7a0 msgr2=0x7fc0480ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.694+0000 7fc04feba640 1 -- 172.21.15.5:0/905016623 shutdown_connections 2024-09-19T06:10:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.694+0000 7fc04feba640 1 -- 172.21.15.5:0/905016623 wait complete. 2024-09-19T06:10:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.695+0000 7fc04feba640 1 Processor -- start 2024-09-19T06:10:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.695+0000 7fc04feba640 1 -- start start 2024-09-19T06:10:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.695+0000 7fc04feba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc048101ac0 0x7fc048198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.695+0000 7fc04feba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc048102460 0x7fc0481988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.695+0000 7fc04feba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc048107ad0 0x7fc04819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.695+0000 7fc04feba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc048111dc0 con 0x7fc048107ad0 2024-09-19T06:10:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.695+0000 7fc04feba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc048111c40 con 0x7fc048101ac0 2024-09-19T06:10:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.695+0000 7fc04feba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc048111f40 con 0x7fc048102460 2024-09-19T06:10:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.695+0000 7fc04dc56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc048101ac0 0x7fc048198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.695+0000 7fc04e457640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc048107ad0 0x7fc04819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.695+0000 7fc04dc56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc048101ac0 0x7fc048198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55620/0 (socket says 172.21.15.5:55620) 2024-09-19T06:10:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.695+0000 7fc04dc56640 1 -- 172.21.15.5:0/3253509479 learned_addr learned my addr 172.21.15.5:0/3253509479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc04d455640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc048102460 0x7fc0481988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc04dc56640 1 -- 172.21.15.5:0/3253509479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc048102460 msgr2=0x7fc0481988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc04dc56640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc048102460 0x7fc0481988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc04dc56640 1 -- 172.21.15.5:0/3253509479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc048107ad0 msgr2=0x7fc04819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc04dc56640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc048107ad0 0x7fc04819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc04dc56640 1 -- 172.21.15.5:0/3253509479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04819d360 con 0x7fc048101ac0 2024-09-19T06:10:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc04e457640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc048107ad0 0x7fc04819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc04dc56640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc048101ac0 0x7fc048198370 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc044005940 tx=0x7fc0440042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc03effd640 1 -- 172.21.15.5:0/3253509479 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04403f030 con 0x7fc048101ac0 2024-09-19T06:10:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc03effd640 1 -- 172.21.15.5:0/3253509479 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0440318a0 con 0x7fc048101ac0 2024-09-19T06:10:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc04feba640 1 -- 172.21.15.5:0/3253509479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04819d5f0 con 0x7fc048101ac0 2024-09-19T06:10:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc03effd640 1 -- 172.21.15.5:0/3253509479 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc044035070 con 0x7fc048101ac0 2024-09-19T06:10:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.696+0000 7fc04feba640 1 -- 172.21.15.5:0/3253509479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0481a4ee0 con 0x7fc048101ac0 2024-09-19T06:10:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.699+0000 7fc03effd640 1 -- 172.21.15.5:0/3253509479 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc044042030 con 0x7fc048101ac0 2024-09-19T06:10:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.699+0000 7fc052c58740 1 Processor -- start 2024-09-19T06:10:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.699+0000 7fc052c58740 1 -- start start 2024-09-19T06:10:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.699+0000 7fc052c58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 0x55a57f8eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.699+0000 7fc052c58740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8eb2f0 0x55a57f8f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.699+0000 7fc052c58740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f8fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.699+0000 7fc052c58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a57f84f2f0 con 0x55a57f8ea9d0 2024-09-19T06:10:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.699+0000 7fc052c58740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a57f7f7270 con 0x55a57f8fa0c0 2024-09-19T06:10:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.699+0000 7fc052c58740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a57f84b2b0 con 0x55a57f8eb2f0 2024-09-19T06:10:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.699+0000 7fc04dc56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8eb2f0 0x55a57f8f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04d455640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f8fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04dc56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8eb2f0 0x55a57f8f9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:51600/0 (socket says 172.21.15.5:51600) 2024-09-19T06:10:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04dc56640 1 -- 172.21.15.5:0/1770496977 learned_addr learned my addr 172.21.15.5:0/1770496977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04d455640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f8fc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55626/0 (socket says 172.21.15.5:55626) 2024-09-19T06:10:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04e457640 1 --2- 172.21.15.5:0/1770496977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 0x55a57f8eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04dc56640 1 -- 172.21.15.5:0/1770496977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 msgr2=0x55a57f8fc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04dc56640 1 --2- 172.21.15.5:0/1770496977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f8fc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04dc56640 1 -- 172.21.15.5:0/1770496977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 msgr2=0x55a57f8eadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc03effd640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc0340a33b0 0x7fc0340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04dc56640 1 --2- 172.21.15.5:0/1770496977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 0x55a57f8eadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04dc56640 1 -- 172.21.15.5:0/1770496977 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57f7f6340 con 0x55a57f8eb2f0 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04e457640 1 --2- 172.21.15.5:0/1770496977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 0x55a57f8eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04d455640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc0340a33b0 0x7fc0340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc03effd640 1 -- 172.21.15.5:0/3253509479 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fc0440f04e0 con 0x7fc048101ac0 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc04dc56640 1 --2- 172.21.15.5:0/1770496977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8eb2f0 0x55a57f8f9b80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc0440edb80 tx=0x7fc0440b8b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=282895f66079293f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.700+0000 7fc033fff640 1 -- 172.21.15.5:0/1770496977 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc044049070 con 0x55a57f8eb2f0 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.701+0000 7fc033fff640 1 -- 172.21.15.5:0/1770496977 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc044003e20 con 0x55a57f8eb2f0 2024-09-19T06:10:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.701+0000 7fc033fff640 1 -- 172.21.15.5:0/1770496977 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0440bc570 con 0x55a57f8eb2f0 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.701+0000 7fc052c58740 1 -- 172.21.15.5:0/1770496977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8eb2f0 msgr2=0x55a57f8f9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.701+0000 7fc04d455640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc0340a33b0 0x7fc0340a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55a57f8e83e0 tx=0x7fc038005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.701+0000 7fc052c58740 1 --2- 172.21.15.5:0/1770496977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8eb2f0 0x55a57f8f9b80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc0440edb80 tx=0x7fc0440b8b00 comp rx=0 tx=0).stop 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.701+0000 7fc052c58740 1 -- 172.21.15.5:0/1770496977 shutdown_connections 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.701+0000 7fc052c58740 1 --2- 172.21.15.5:0/1770496977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f8fc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.701+0000 7fc052c58740 1 --2- 172.21.15.5:0/1770496977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8eb2f0 0x55a57f8f9b80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.701+0000 7fc052c58740 1 --2- 172.21.15.5:0/1770496977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 0x55a57f8eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.701+0000 7fc052c58740 1 -- 172.21.15.5:0/1770496977 >> 172.21.15.5:0/1770496977 conn(0x55a57f8e7060 msgr2=0x55a57f8e7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.701+0000 7fc052c58740 1 -- 172.21.15.5:0/1770496977 shutdown_connections 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.701+0000 7fc052c58740 1 -- 172.21.15.5:0/1770496977 wait complete. 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.702+0000 7fc052c58740 1 -- start start 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.702+0000 7fc052c58740 1 Processor -- start 2024-09-19T06:10:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.702+0000 7fc052c58740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8ea9d0 0x55a57f8f7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.702+0000 7fc052c58740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8eb2f0 0x55a57f8f7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.702+0000 7fc052c58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8fa0c0 0x55a57f8f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.702+0000 7fc052c58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a57f8fcab0 con 0x55a57f8fa0c0 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.702+0000 7fc052c58740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a57f7d8290 con 0x55a57f8eb2f0 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.702+0000 7fc052c58740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a57f8fcc30 con 0x55a57f8ea9d0 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.702+0000 7fc052c58740 10 client.0 _ll_register_callbacks cb 0x55a57f809140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.702+0000 7fc04d455640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8eb2f0 0x55a57f8f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.702+0000 7fc04dc56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8ea9d0 0x55a57f8f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc04d455640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8eb2f0 0x55a57f8f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55636/0 (socket says 172.21.15.5:55636) 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc04dc56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8ea9d0 0x55a57f8f7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:51612/0 (socket says 172.21.15.5:51612) 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc04e457640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8fa0c0 0x55a57f8f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc04d455640 1 -- 172.21.15.5:0/2042191318 learned_addr learned my addr 172.21.15.5:0/2042191318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc04d455640 1 -- 172.21.15.5:0/2042191318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8ea9d0 msgr2=0x55a57f8f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc04d455640 1 --2- 172.21.15.5:0/2042191318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8ea9d0 0x55a57f8f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc04d455640 1 -- 172.21.15.5:0/2042191318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8fa0c0 msgr2=0x55a57f8f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc04d455640 1 --2- 172.21.15.5:0/2042191318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8fa0c0 0x55a57f8f7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc04d455640 1 -- 172.21.15.5:0/2042191318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57f562210 con 0x55a57f8eb2f0 2024-09-19T06:10:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc04d455640 1 --2- 172.21.15.5:0/2042191318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8eb2f0 0x55a57f8f7540 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc038020a10 tx=0x7fc038004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=439c946d2b37fa75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc0317fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:10:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc0317fa640 1 -- 172.21.15.5:0/2042191318 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038011940 con 0x55a57f8eb2f0 2024-09-19T06:10:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.703+0000 7fc0317fa640 1 -- 172.21.15.5:0/2042191318 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fc038011ae0 con 0x55a57f8eb2f0 2024-09-19T06:10:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.704+0000 7fc052c58740 1 -- 172.21.15.5:0/2042191318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a57f85ca20 con 0x55a57f8eb2f0 2024-09-19T06:10:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.704+0000 7fc0317fa640 1 -- 172.21.15.5:0/2042191318 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc038011ce0 con 0x55a57f8eb2f0 2024-09-19T06:10:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.704+0000 7fc052c58740 10 client.26741 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.704+0000 7fc052c58740 1 -- 172.21.15.5:0/2042191318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a57f62e620 con 0x55a57f8eb2f0 2024-09-19T06:10:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.704+0000 7fc0317fa640 1 -- 172.21.15.5:0/2042191318 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a57f85ca20 con 0x55a57f8eb2f0 2024-09-19T06:10:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.704+0000 7fc0317fa640 1 -- 172.21.15.5:0/2042191318 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc038027070 con 0x55a57f8eb2f0 2024-09-19T06:10:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.704+0000 7fc052c58740 10 client.26741 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.704+0000 7fc052c58740 10 client.26741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.704+0000 7fc052c58740 20 client.26741 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.704+0000 7fc052c58740 1 --2- 172.21.15.5:0/2042191318 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a57f909e90 0x55a57f90c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.704+0000 7fc052c58740 4 client.26741 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.704+0000 7fc052c58740 1 -- 172.21.15.5:0/2042191318 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a57f856d70 con 0x55a57f909e90 2024-09-19T06:10:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.705+0000 7fc04dc56640 1 --2- 172.21.15.5:0/2042191318 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a57f909e90 0x55a57f90c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.705+0000 7fc04dc56640 1 --2- 172.21.15.5:0/2042191318 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a57f909e90 0x55a57f90c280 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce01f3ae8c461fbe server_cookie=aa91109c09715d3e in_seq=0 out_seq=0 2024-09-19T06:10:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.705+0000 7fc0317fa640 10 client.26741 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc0317fa640 1 -- 172.21.15.5:0/2042191318 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a57f856d70 con 0x55a57f909e90 2024-09-19T06:10:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc0317fa640 10 client.26741 handle_command_reply: tid=0 2024-09-19T06:10:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc052c58740 1 client.26741 shutdown 2024-09-19T06:10:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc052c58740 1 -- 172.21.15.5:0/2042191318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8eb2f0 msgr2=0x55a57f8f7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc052c58740 1 --2- 172.21.15.5:0/2042191318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8eb2f0 0x55a57f8f7540 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc038020a10 tx=0x7fc038004730 comp rx=0 tx=0).stop 2024-09-19T06:10:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc052c58740 1 -- 172.21.15.5:0/2042191318 shutdown_connections 2024-09-19T06:10:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc052c58740 1 --2- 172.21.15.5:0/2042191318 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a57f909e90 0x55a57f90c280 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc052c58740 1 --2- 172.21.15.5:0/2042191318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8fa0c0 0x55a57f8f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc0317fa640 0 client.26741 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc052c58740 1 --2- 172.21.15.5:0/2042191318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8eb2f0 0x55a57f8f7540 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc052c58740 1 --2- 172.21.15.5:0/2042191318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8ea9d0 0x55a57f8f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc052c58740 1 -- 172.21.15.5:0/2042191318 >> 172.21.15.5:0/2042191318 conn(0x55a57f8e7060 msgr2=0x55a57f8ecdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc052c58740 1 -- 172.21.15.5:0/2042191318 shutdown_connections 2024-09-19T06:10:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc052c58740 1 -- 172.21.15.5:0/2042191318 wait complete. 2024-09-19T06:10:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.706+0000 7fc052c58740 20 client.26741 trim_cache size 0 max 16384 2024-09-19T06:10:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.722+0000 7fc052c58740 1 Processor -- start 2024-09-19T06:10:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.722+0000 7fc052c58740 1 -- start start 2024-09-19T06:10:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc052c58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 0x55a57f8f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc052c58740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8eb2f0 0x55a57f8e60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc052c58740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f911b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc052c58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0481ab6c0 con 0x55a57f8ea9d0 2024-09-19T06:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc052c58740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0481ad200 con 0x55a57f8fa0c0 2024-09-19T06:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc052c58740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc0481abd90 con 0x55a57f8eb2f0 2024-09-19T06:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc04dc56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 0x55a57f8f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc04dc56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 0x55a57f8f78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39136/0 (socket says 172.21.15.5:39136) 2024-09-19T06:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc04dc56640 1 -- 172.21.15.5:0/3241907929 learned_addr learned my addr 172.21.15.5:0/3241907929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc04d455640 1 --2- 172.21.15.5:0/3241907929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8eb2f0 0x55a57f8e60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc04e457640 1 --2- 172.21.15.5:0/3241907929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f911b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc04dc56640 1 -- 172.21.15.5:0/3241907929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8eb2f0 msgr2=0x55a57f8e60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc04dc56640 1 --2- 172.21.15.5:0/3241907929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8eb2f0 0x55a57f8e60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc04dc56640 1 -- 172.21.15.5:0/3241907929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 msgr2=0x55a57f911b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc04dc56640 1 --2- 172.21.15.5:0/3241907929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f911b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc04dc56640 1 -- 172.21.15.5:0/3241907929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57f90c5f0 con 0x55a57f8ea9d0 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc04e457640 1 --2- 172.21.15.5:0/3241907929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f911b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc04dc56640 1 --2- 172.21.15.5:0/3241907929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 0x55a57f8f78b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc0440ee210 tx=0x7fc04402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205db4bf4e94a0b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc0337fe640 1 -- 172.21.15.5:0/3241907929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc044049040 con 0x55a57f8ea9d0 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.723+0000 7fc0337fe640 1 -- 172.21.15.5:0/3241907929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc04403b460 con 0x55a57f8ea9d0 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.724+0000 7fc0337fe640 1 -- 172.21.15.5:0/3241907929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0440bc100 con 0x55a57f8ea9d0 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.724+0000 7fc052c58740 1 -- 172.21.15.5:0/3241907929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 msgr2=0x55a57f8f78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.724+0000 7fc052c58740 1 --2- 172.21.15.5:0/3241907929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 0x55a57f8f78b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc0440ee210 tx=0x7fc04402a420 comp rx=0 tx=0).stop 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.724+0000 7fc052c58740 1 -- 172.21.15.5:0/3241907929 shutdown_connections 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.724+0000 7fc052c58740 1 --2- 172.21.15.5:0/3241907929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f911b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.724+0000 7fc052c58740 1 --2- 172.21.15.5:0/3241907929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8eb2f0 0x55a57f8e60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.724+0000 7fc052c58740 1 --2- 172.21.15.5:0/3241907929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8ea9d0 0x55a57f8f78b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.724+0000 7fc052c58740 1 -- 172.21.15.5:0/3241907929 >> 172.21.15.5:0/3241907929 conn(0x55a57f8e7060 msgr2=0x55a57f8e59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.724+0000 7fc052c58740 1 -- 172.21.15.5:0/3241907929 shutdown_connections 2024-09-19T06:10:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.724+0000 7fc052c58740 1 -- 172.21.15.5:0/3241907929 wait complete. 2024-09-19T06:10:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.725+0000 7fc052c58740 1 -- start start 2024-09-19T06:10:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.725+0000 7fc052c58740 1 Processor -- start 2024-09-19T06:10:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.725+0000 7fc052c58740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8ea9d0 0x55a57f8fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.725+0000 7fc052c58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8eb2f0 0x55a57f8fe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.725+0000 7fc052c58740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f8fe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.725+0000 7fc052c58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0481acb30 con 0x55a57f8eb2f0 2024-09-19T06:10:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.725+0000 7fc052c58740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0481ac460 con 0x55a57f8fa0c0 2024-09-19T06:10:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.725+0000 7fc052c58740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc0481a96c0 con 0x55a57f8ea9d0 2024-09-19T06:10:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.725+0000 7fc052c58740 10 client.0 _ll_register_callbacks cb 0x7fc0380096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.725+0000 7fc04d455640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8eb2f0 0x55a57f8fe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc04d455640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8eb2f0 0x55a57f8fe1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39150/0 (socket says 172.21.15.5:39150) 2024-09-19T06:10:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc04d455640 1 -- 172.21.15.5:0/4095292896 learned_addr learned my addr 172.21.15.5:0/4095292896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc04e457640 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f8fe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc04dc56640 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8ea9d0 0x55a57f8fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc04d455640 1 -- 172.21.15.5:0/4095292896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8ea9d0 msgr2=0x55a57f8fdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc04d455640 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8ea9d0 0x55a57f8fdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc04d455640 1 -- 172.21.15.5:0/4095292896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 msgr2=0x55a57f8fe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc04d455640 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f8fe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc04d455640 1 -- 172.21.15.5:0/4095292896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57f973be0 con 0x55a57f8eb2f0 2024-09-19T06:10:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc04d455640 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8eb2f0 0x55a57f8fe1d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc0380209e0 tx=0x7fc038028d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e85e1fe3559a490 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc0317fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:10:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc0317fa640 1 -- 172.21.15.5:0/4095292896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0380207d0 con 0x55a57f8eb2f0 2024-09-19T06:10:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc0317fa640 1 -- 172.21.15.5:0/4095292896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fc038032470 con 0x55a57f8eb2f0 2024-09-19T06:10:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc052c58740 1 -- 172.21.15.5:0/4095292896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a57f7d3900 con 0x55a57f8eb2f0 2024-09-19T06:10:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc0317fa640 1 -- 172.21.15.5:0/4095292896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc038011e80 con 0x55a57f8eb2f0 2024-09-19T06:10:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc052c58740 10 client.17169 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc052c58740 1 -- 172.21.15.5:0/4095292896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a57f974ed0 con 0x55a57f8eb2f0 2024-09-19T06:10:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.726+0000 7fc0317fa640 1 -- 172.21.15.5:0/4095292896 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a57f7d3900 con 0x55a57f8eb2f0 2024-09-19T06:10:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.727+0000 7fc0317fa640 1 -- 172.21.15.5:0/4095292896 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc038027040 con 0x55a57f8eb2f0 2024-09-19T06:10:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.727+0000 7fc052c58740 10 client.17169 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.727+0000 7fc052c58740 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.727+0000 7fc052c58740 20 client.17169 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.727+0000 7fc052c58740 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a57f975620 0x55a57f977a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.727+0000 7fc052c58740 4 client.17169 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:10:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.727+0000 7fc052c58740 1 -- 172.21.15.5:0/4095292896 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a57f820820 con 0x55a57f975620 2024-09-19T06:10:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.727+0000 7fc04dc56640 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a57f975620 0x55a57f977a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.727+0000 7fc04dc56640 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a57f975620 0x55a57f977a40 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b6f13dc255d785 server_cookie=ef6425cb724dd704 in_seq=0 out_seq=0 2024-09-19T06:10:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.727+0000 7fc0317fa640 10 client.17169 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc0317fa640 1 -- 172.21.15.5:0/4095292896 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a57f820820 con 0x55a57f975620 2024-09-19T06:10:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc0317fa640 10 client.17169 handle_command_reply: tid=0 2024-09-19T06:10:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc052c58740 1 client.17169 shutdown 2024-09-19T06:10:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc052c58740 1 -- 172.21.15.5:0/4095292896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8eb2f0 msgr2=0x55a57f8fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc052c58740 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8eb2f0 0x55a57f8fe1d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc0380209e0 tx=0x7fc038028d70 comp rx=0 tx=0).stop 2024-09-19T06:10:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc052c58740 1 -- 172.21.15.5:0/4095292896 shutdown_connections 2024-09-19T06:10:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc052c58740 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a57f975620 0x55a57f977a40 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc052c58740 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a57f8fa0c0 0x55a57f8fe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc0317fa640 0 client.17169 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc052c58740 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a57f8eb2f0 0x55a57f8fe1d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc052c58740 1 --2- 172.21.15.5:0/4095292896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a57f8ea9d0 0x55a57f8fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc052c58740 1 -- 172.21.15.5:0/4095292896 >> 172.21.15.5:0/4095292896 conn(0x55a57f8e7060 msgr2=0x55a57f8e61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc052c58740 1 -- 172.21.15.5:0/4095292896 shutdown_connections 2024-09-19T06:10:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc052c58740 1 -- 172.21.15.5:0/4095292896 wait complete. 2024-09-19T06:10:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.749+0000 7fc052c58740 20 client.17169 trim_cache size 0 max 16384 2024-09-19T06:10:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:10:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:10:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:10:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:10:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.750+0000 7fc04feba640 1 -- 172.21.15.5:0/3253509479 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc0340a33b0 msgr2=0x7fc0340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.750+0000 7fc04feba640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc0340a33b0 0x7fc0340a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55a57f8e83e0 tx=0x7fc038005f90 comp rx=0 tx=0).stop 2024-09-19T06:10:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.750+0000 7fc04feba640 1 -- 172.21.15.5:0/3253509479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc048101ac0 msgr2=0x7fc048198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.750+0000 7fc04feba640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc048101ac0 0x7fc048198370 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc044005940 tx=0x7fc0440042f0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.750+0000 7fc04feba640 1 -- 172.21.15.5:0/3253509479 shutdown_connections 2024-09-19T06:10:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.750+0000 7fc04feba640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc048107ad0 0x7fc04819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.750+0000 7fc04feba640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc0340a33b0 0x7fc0340a5870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.750+0000 7fc04feba640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc048102460 0x7fc0481988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.750+0000 7fc04feba640 1 --2- 172.21.15.5:0/3253509479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc048101ac0 0x7fc048198370 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.750+0000 7fc04feba640 1 -- 172.21.15.5:0/3253509479 >> 172.21.15.5:0/3253509479 conn(0x7fc0480fd7a0 msgr2=0x7fc0480fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.750+0000 7fc04feba640 1 -- 172.21.15.5:0/3253509479 shutdown_connections 2024-09-19T06:10:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:19.750+0000 7fc04feba640 1 -- 172.21.15.5:0/3253509479 wait complete. 2024-09-19T06:10:19.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 176264 2.72 MB/sec execute 225 sec latency 19.901 ms 2024-09-19T06:10:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:20 smithi005 ceph-mon[30955]: pgmap v215: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-19T06:10:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:20 smithi049 ceph-mon[34640]: pgmap v215: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-19T06:10:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:20 smithi157 ceph-mon[35346]: pgmap v215: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-19T06:10:20.831 INFO:tasks.workunit.client.0.smithi005.stdout: 1 177024 2.74 MB/sec execute 226 sec latency 15.970 ms 2024-09-19T06:10:21.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 177480 2.74 MB/sec execute 227 sec latency 11.954 ms 2024-09-19T06:10:22.602 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:22 smithi157 ceph-mon[35346]: pgmap v216: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T06:10:22.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:22 smithi005 ceph-mon[30955]: pgmap v216: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T06:10:22.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:22 smithi049 ceph-mon[34640]: pgmap v216: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T06:10:22.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 177782 2.74 MB/sec execute 228 sec latency 14.353 ms 2024-09-19T06:10:22.918 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:10:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:10:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.714+0000 7fa23245e640 1 -- 172.21.15.5:0/3810820681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c104990 msgr2=0x7fa22c1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.714+0000 7fa23245e640 1 --2- 172.21.15.5:0/3810820681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c104990 0x7fa22c1089e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa220009900 tx=0x7fa22002a410 comp rx=0 tx=0).stop 2024-09-19T06:10:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.714+0000 7fa23245e640 1 -- 172.21.15.5:0/3810820681 shutdown_connections 2024-09-19T06:10:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.714+0000 7fa23245e640 1 --2- 172.21.15.5:0/3810820681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c109110 0x7fa22c110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.714+0000 7fa23245e640 1 --2- 172.21.15.5:0/3810820681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c104990 0x7fa22c1089e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.714+0000 7fa23245e640 1 --2- 172.21.15.5:0/3810820681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c103fe0 0x7fa22c1043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.714+0000 7fa23245e640 1 -- 172.21.15.5:0/3810820681 >> 172.21.15.5:0/3810820681 conn(0x7fa22c0ffa20 msgr2=0x7fa22c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.714+0000 7fa23245e640 1 -- 172.21.15.5:0/3810820681 shutdown_connections 2024-09-19T06:10:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.715+0000 7fa23245e640 1 -- 172.21.15.5:0/3810820681 wait complete. 2024-09-19T06:10:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.715+0000 7fa23245e640 1 Processor -- start 2024-09-19T06:10:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.716+0000 7fa23245e640 1 -- start start 2024-09-19T06:10:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.716+0000 7fa23245e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c103fe0 0x7fa22c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.716+0000 7fa23245e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c104990 0x7fa22c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.716+0000 7fa23245e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c109110 0x7fa22c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.716+0000 7fa23245e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c114030 con 0x7fa22c109110 2024-09-19T06:10:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.716+0000 7fa23245e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c113eb0 con 0x7fa22c103fe0 2024-09-19T06:10:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.716+0000 7fa23245e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1141b0 con 0x7fa22c104990 2024-09-19T06:10:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.716+0000 7fa22bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c103fe0 0x7fa22c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.717+0000 7fa22bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c103fe0 0x7fa22c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55664/0 (socket says 172.21.15.5:55664) 2024-09-19T06:10:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.717+0000 7fa22bfff640 1 -- 172.21.15.5:0/66121167 learned_addr learned my addr 172.21.15.5:0/66121167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.717+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c104990 0x7fa22c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.717+0000 7fa2309fb640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c109110 0x7fa22c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.717+0000 7fa22bfff640 1 -- 172.21.15.5:0/66121167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c104990 msgr2=0x7fa22c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.717+0000 7fa22bfff640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c104990 0x7fa22c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.717+0000 7fa22bfff640 1 -- 172.21.15.5:0/66121167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c109110 msgr2=0x7fa22c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.717+0000 7fa22bfff640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c109110 0x7fa22c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.717+0000 7fa22bfff640 1 -- 172.21.15.5:0/66121167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c19f5c0 con 0x7fa22c103fe0 2024-09-19T06:10:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.717+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c104990 0x7fa22c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.717+0000 7fa22bfff640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c103fe0 0x7fa22c19a610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00e9f0 tx=0x7fa21c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.717+0000 7fa2297fa640 1 -- 172.21.15.5:0/66121167 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c00cd80 con 0x7fa22c103fe0 2024-09-19T06:10:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.718+0000 7fa2297fa640 1 -- 172.21.15.5:0/66121167 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa21c0054d0 con 0x7fa22c103fe0 2024-09-19T06:10:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.718+0000 7fa23245e640 1 -- 172.21.15.5:0/66121167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22c19f8b0 con 0x7fa22c103fe0 2024-09-19T06:10:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.718+0000 7fa2297fa640 1 -- 172.21.15.5:0/66121167 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c010490 con 0x7fa22c103fe0 2024-09-19T06:10:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.718+0000 7fa23245e640 1 -- 172.21.15.5:0/66121167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22c1a7190 con 0x7fa22c103fe0 2024-09-19T06:10:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.720+0000 7fa234837740 1 Processor -- start 2024-09-19T06:10:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.720+0000 7fa234837740 1 -- start start 2024-09-19T06:10:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa234837740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19ba4a0 0x563fe19ba880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa234837740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19badc0 0x563fe19c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa234837740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe19c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa234837740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563fe19202e0 con 0x563fe19ba4a0 2024-09-19T06:10:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa234837740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563fe18c7ce0 con 0x563fe19badc0 2024-09-19T06:10:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa234837740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563fe191c2a0 con 0x563fe19c5680 2024-09-19T06:10:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa2309fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19ba4a0 0x563fe19ba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa2309fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19ba4a0 0x563fe19ba880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39192/0 (socket says 172.21.15.5:39192) 2024-09-19T06:10:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa2309fb640 1 -- 172.21.15.5:0/145307824 learned_addr learned my addr 172.21.15.5:0/145307824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/145307824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe19c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa2309fb640 1 -- 172.21.15.5:0/145307824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 msgr2=0x563fe19c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa2309fb640 1 --2- 172.21.15.5:0/145307824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe19c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa2309fb640 1 -- 172.21.15.5:0/145307824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19badc0 msgr2=0x563fe19c5140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:10:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa2309fb640 1 --2- 172.21.15.5:0/145307824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19badc0 0x563fe19c5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa2309fb640 1 -- 172.21.15.5:0/145307824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe18ac440 con 0x563fe19ba4a0 2024-09-19T06:10:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/145307824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe19c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:10:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.721+0000 7fa2309fb640 1 --2- 172.21.15.5:0/145307824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19ba4a0 0x563fe19ba880 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa224009f90 tx=0x7fa224010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f586fbdc0bea2f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.722+0000 7fa2127fc640 1 -- 172.21.15.5:0/145307824 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224011700 con 0x563fe19ba4a0 2024-09-19T06:10:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.722+0000 7fa2297fa640 1 -- 172.21.15.5:0/66121167 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa21c005020 con 0x7fa22c103fe0 2024-09-19T06:10:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.722+0000 7fa2127fc640 1 -- 172.21.15.5:0/145307824 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa224011d50 con 0x563fe19ba4a0 2024-09-19T06:10:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.722+0000 7fa2127fc640 1 -- 172.21.15.5:0/145307824 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224019d40 con 0x563fe19ba4a0 2024-09-19T06:10:23.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.723+0000 7fa2297fa640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2140a3430 0x7fa2140a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.723+0000 7fa22bfff640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2140a3430 0x7fa2140a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.723+0000 7fa234837740 1 -- 172.21.15.5:0/145307824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19ba4a0 msgr2=0x563fe19ba880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:23.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.723+0000 7fa234837740 1 --2- 172.21.15.5:0/145307824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19ba4a0 0x563fe19ba880 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa224009f90 tx=0x7fa224010860 comp rx=0 tx=0).stop 2024-09-19T06:10:23.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.723+0000 7fa2297fa640 1 -- 172.21.15.5:0/66121167 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fa21c0d0020 con 0x7fa22c103fe0 2024-09-19T06:10:23.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.723+0000 7fa234837740 1 -- 172.21.15.5:0/145307824 shutdown_connections 2024-09-19T06:10:23.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.723+0000 7fa234837740 1 --2- 172.21.15.5:0/145307824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe19c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.723+0000 7fa234837740 1 --2- 172.21.15.5:0/145307824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19badc0 0x563fe19c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.723+0000 7fa234837740 1 --2- 172.21.15.5:0/145307824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19ba4a0 0x563fe19ba880 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.723+0000 7fa234837740 1 -- 172.21.15.5:0/145307824 >> 172.21.15.5:0/145307824 conn(0x563fe19b6a30 msgr2=0x563fe19b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.723+0000 7fa234837740 1 -- 172.21.15.5:0/145307824 shutdown_connections 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.723+0000 7fa234837740 1 -- 172.21.15.5:0/145307824 wait complete. 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.724+0000 7fa22bfff640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2140a3430 0x7fa2140a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x563fe19b7890 tx=0x7fa21c0bc040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.724+0000 7fa234837740 1 -- start start 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.724+0000 7fa234837740 1 Processor -- start 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.724+0000 7fa234837740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 0x563fe19c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.724+0000 7fa234837740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19badc0 0x563fe19c2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa234837740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe19c2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa234837740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563fe19c8250 con 0x563fe19badc0 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa234837740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563fe18db050 con 0x563fe19ba4a0 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa234837740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563fe19c83d0 con 0x563fe19c5680 2024-09-19T06:10:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa234837740 10 client.0 _ll_register_callbacks cb 0x563fe191b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa2309fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 0x563fe19c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa2309fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 0x563fe19c4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55666/0 (socket says 172.21.15.5:55666) 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa2309fb640 1 -- 172.21.15.5:0/843859222 learned_addr learned my addr 172.21.15.5:0/843859222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa2309fb640 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe19c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19badc0 0x563fe19c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa2309fb640 1 -- 172.21.15.5:0/843859222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 msgr2=0x563fe19c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa2309fb640 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe19c2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa2309fb640 1 -- 172.21.15.5:0/843859222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19badc0 msgr2=0x563fe19c2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa2309fb640 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19badc0 0x563fe19c2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa2309fb640 1 -- 172.21.15.5:0/843859222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe16913c0 con 0x563fe19ba4a0 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa2309fb640 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 0x563fe19c4f50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa224005f40 tx=0x7fa224013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f796a8f5aa0effa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.725+0000 7fa20b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:10:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa20b7fe640 1 -- 172.21.15.5:0/843859222 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224011910 con 0x563fe19ba4a0 2024-09-19T06:10:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa234837740 1 -- 172.21.15.5:0/843859222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa224011d50 con 0x563fe19ba4a0 2024-09-19T06:10:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa20b7fe640 1 -- 172.21.15.5:0/843859222 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fa224011ab0 con 0x563fe19ba4a0 2024-09-19T06:10:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa20b7fe640 1 -- 172.21.15.5:0/843859222 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa224019e50 con 0x563fe19ba4a0 2024-09-19T06:10:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa234837740 10 client.26756 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa234837740 1 -- 172.21.15.5:0/843859222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563fe19d3ba0 con 0x563fe19ba4a0 2024-09-19T06:10:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa20b7fe640 1 -- 172.21.15.5:0/843859222 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa224011d50 con 0x563fe19ba4a0 2024-09-19T06:10:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa20b7fe640 1 -- 172.21.15.5:0/843859222 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa2240044a0 con 0x563fe19ba4a0 2024-09-19T06:10:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa234837740 10 client.26756 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa234837740 10 client.26756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa234837740 20 client.26756 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa234837740 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563fe19d53e0 0x563fe19d77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa234837740 4 client.26756 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.726+0000 7fa234837740 1 -- 172.21.15.5:0/843859222 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fe192d850 con 0x563fe19d53e0 2024-09-19T06:10:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.727+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563fe19d53e0 0x563fe19d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.727+0000 7fa20b7fe640 10 client.26756 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.727+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563fe19d53e0 0x563fe19d77f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6715073e0eca83e7 server_cookie=92961af52f318bb9 in_seq=0 out_seq=0 2024-09-19T06:10:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa20b7fe640 1 -- 172.21.15.5:0/843859222 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fe192d850 con 0x563fe19d53e0 2024-09-19T06:10:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa20b7fe640 10 client.26756 handle_command_reply: tid=0 2024-09-19T06:10:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa234837740 1 client.26756 shutdown 2024-09-19T06:10:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa234837740 1 -- 172.21.15.5:0/843859222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 msgr2=0x563fe19c4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa234837740 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 0x563fe19c4f50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa224005f40 tx=0x7fa224013040 comp rx=0 tx=0).stop 2024-09-19T06:10:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa234837740 1 -- 172.21.15.5:0/843859222 shutdown_connections 2024-09-19T06:10:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa234837740 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563fe19d53e0 0x563fe19d77f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa234837740 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe19c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa234837740 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19badc0 0x563fe19c2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa234837740 1 --2- 172.21.15.5:0/843859222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 0x563fe19c4f50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa234837740 1 -- 172.21.15.5:0/843859222 >> 172.21.15.5:0/843859222 conn(0x563fe19b6a30 msgr2=0x563fe19bc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa20b7fe640 0 client.26756 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.728+0000 7fa234837740 1 -- 172.21.15.5:0/843859222 shutdown_connections 2024-09-19T06:10:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.729+0000 7fa234837740 1 -- 172.21.15.5:0/843859222 wait complete. 2024-09-19T06:10:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.729+0000 7fa234837740 20 client.26756 trim_cache size 0 max 16384 2024-09-19T06:10:23.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa234837740 1 Processor -- start 2024-09-19T06:10:23.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa234837740 1 -- start start 2024-09-19T06:10:23.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa234837740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19ba4a0 0x563fe19b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa234837740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19badc0 0x563fe19dd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa234837740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19c5680 0x563fe19dd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa234837740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1afb30 con 0x563fe19c5680 2024-09-19T06:10:23.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa234837740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1b1670 con 0x563fe19badc0 2024-09-19T06:10:23.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa234837740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1b0200 con 0x563fe19ba4a0 2024-09-19T06:10:23.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa2309fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19ba4a0 0x563fe19b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19badc0 0x563fe19dd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa2309fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19ba4a0 0x563fe19b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:51686/0 (socket says 172.21.15.5:51686) 2024-09-19T06:10:23.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa2309fb640 1 -- 172.21.15.5:0/3898433211 learned_addr learned my addr 172.21.15.5:0/3898433211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:23.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19badc0 0x563fe19dd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55670/0 (socket says 172.21.15.5:55670) 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.751+0000 7fa2309fb640 1 --2- 172.21.15.5:0/3898433211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19c5680 0x563fe19dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.752+0000 7fa2309fb640 1 -- 172.21.15.5:0/3898433211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19badc0 msgr2=0x563fe19dd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.752+0000 7fa2309fb640 1 --2- 172.21.15.5:0/3898433211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19badc0 0x563fe19dd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.752+0000 7fa2309fb640 1 -- 172.21.15.5:0/3898433211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19c5680 msgr2=0x563fe19dd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.752+0000 7fa2309fb640 1 --2- 172.21.15.5:0/3898433211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19c5680 0x563fe19dd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.752+0000 7fa2309fb640 1 -- 172.21.15.5:0/3898433211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe19b6860 con 0x563fe19ba4a0 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.752+0000 7fa2309fb640 1 --2- 172.21.15.5:0/3898433211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19ba4a0 0x563fe19b5e20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa224009720 tx=0x7fa224010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89c0b0edec3f47a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.752+0000 7fa211ffb640 1 -- 172.21.15.5:0/3898433211 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224037020 con 0x563fe19ba4a0 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.752+0000 7fa211ffb640 1 -- 172.21.15.5:0/3898433211 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa224020040 con 0x563fe19ba4a0 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.752+0000 7fa211ffb640 1 -- 172.21.15.5:0/3898433211 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22401a630 con 0x563fe19ba4a0 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.753+0000 7fa234837740 1 -- 172.21.15.5:0/3898433211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19ba4a0 msgr2=0x563fe19b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.753+0000 7fa234837740 1 --2- 172.21.15.5:0/3898433211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19ba4a0 0x563fe19b5e20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa224009720 tx=0x7fa224010860 comp rx=0 tx=0).stop 2024-09-19T06:10:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.753+0000 7fa234837740 1 -- 172.21.15.5:0/3898433211 shutdown_connections 2024-09-19T06:10:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.753+0000 7fa234837740 1 --2- 172.21.15.5:0/3898433211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19c5680 0x563fe19dd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.753+0000 7fa234837740 1 --2- 172.21.15.5:0/3898433211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19badc0 0x563fe19dd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.753+0000 7fa234837740 1 --2- 172.21.15.5:0/3898433211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19ba4a0 0x563fe19b5e20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.753+0000 7fa234837740 1 -- 172.21.15.5:0/3898433211 >> 172.21.15.5:0/3898433211 conn(0x563fe19b6a30 msgr2=0x563fe19c22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.753+0000 7fa234837740 1 -- 172.21.15.5:0/3898433211 shutdown_connections 2024-09-19T06:10:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.753+0000 7fa234837740 1 -- 172.21.15.5:0/3898433211 wait complete. 2024-09-19T06:10:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.753+0000 7fa234837740 1 -- start start 2024-09-19T06:10:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa234837740 1 Processor -- start 2024-09-19T06:10:23.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa234837740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 0x563fe19cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa234837740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19badc0 0x563fe19cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa234837740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe1a3e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa234837740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1b0fa0 con 0x563fe19badc0 2024-09-19T06:10:23.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa234837740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1b08d0 con 0x563fe19ba4a0 2024-09-19T06:10:23.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa234837740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1adb30 con 0x563fe19c5680 2024-09-19T06:10:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa234837740 10 client.0 _ll_register_callbacks cb 0x7fa224009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19badc0 0x563fe19cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa2309fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 0x563fe19cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19badc0 0x563fe19cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39214/0 (socket says 172.21.15.5:39214) 2024-09-19T06:10:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa22b7fe640 1 -- 172.21.15.5:0/2548326474 learned_addr learned my addr 172.21.15.5:0/2548326474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa2309fb640 1 --2- 172.21.15.5:0/2548326474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe1a3e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa2309fb640 1 -- 172.21.15.5:0/2548326474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 msgr2=0x563fe1a3e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa2309fb640 1 --2- 172.21.15.5:0/2548326474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe1a3e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa2309fb640 1 -- 172.21.15.5:0/2548326474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19badc0 msgr2=0x563fe19cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa2309fb640 1 --2- 172.21.15.5:0/2548326474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19badc0 0x563fe19cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.754+0000 7fa2309fb640 1 -- 172.21.15.5:0/2548326474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe19b7ae0 con 0x563fe19ba4a0 2024-09-19T06:10:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.755+0000 7fa2309fb640 1 --2- 172.21.15.5:0/2548326474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 0x563fe19cab90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa224004870 tx=0x7fa224009a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f3817bdba6a096 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.755+0000 7fa20b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.755+0000 7fa20b7fe640 1 -- 172.21.15.5:0/2548326474 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224037070 con 0x563fe19ba4a0 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.755+0000 7fa234837740 1 -- 172.21.15.5:0/2548326474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa224020040 con 0x563fe19ba4a0 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.755+0000 7fa20b7fe640 1 -- 172.21.15.5:0/2548326474 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fa224019d40 con 0x563fe19ba4a0 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.755+0000 7fa20b7fe640 1 -- 172.21.15.5:0/2548326474 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa224011ab0 con 0x563fe19ba4a0 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.755+0000 7fa234837740 10 client.26762 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.755+0000 7fa234837740 1 -- 172.21.15.5:0/2548326474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563fe1a3f780 con 0x563fe19ba4a0 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.756+0000 7fa20b7fe640 1 -- 172.21.15.5:0/2548326474 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa224020040 con 0x563fe19ba4a0 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.756+0000 7fa20b7fe640 1 -- 172.21.15.5:0/2548326474 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa224004ab0 con 0x563fe19ba4a0 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.756+0000 7fa234837740 10 client.26762 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.756+0000 7fa234837740 10 client.26762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.756+0000 7fa234837740 20 client.26762 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.756+0000 7fa234837740 1 --2- 172.21.15.5:0/2548326474 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563fe1a3fdf0 0x563fe1a421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.756+0000 7fa234837740 4 client.26762 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:10:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.756+0000 7fa234837740 1 -- 172.21.15.5:0/2548326474 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563fe19b75e0 con 0x563fe1a3fdf0 2024-09-19T06:10:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.756+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/2548326474 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563fe1a3fdf0 0x563fe1a421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/2548326474 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563fe1a3fdf0 0x563fe1a421e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=505dd990c1776ff3 server_cookie=c1c23f66689adcb8 in_seq=0 out_seq=0 2024-09-19T06:10:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa20b7fe640 10 client.26762 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa20b7fe640 1 -- 172.21.15.5:0/2548326474 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563fe19b75e0 con 0x563fe1a3fdf0 2024-09-19T06:10:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa20b7fe640 10 client.26762 handle_command_reply: tid=0 2024-09-19T06:10:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa234837740 1 client.26762 shutdown 2024-09-19T06:10:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa234837740 1 -- 172.21.15.5:0/2548326474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 msgr2=0x563fe19cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa234837740 1 --2- 172.21.15.5:0/2548326474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 0x563fe19cab90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa224004870 tx=0x7fa224009a00 comp rx=0 tx=0).stop 2024-09-19T06:10:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa234837740 1 -- 172.21.15.5:0/2548326474 shutdown_connections 2024-09-19T06:10:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa234837740 1 --2- 172.21.15.5:0/2548326474 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563fe1a3fdf0 0x563fe1a421e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa234837740 1 --2- 172.21.15.5:0/2548326474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563fe19c5680 0x563fe1a3e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa234837740 1 --2- 172.21.15.5:0/2548326474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fe19badc0 0x563fe19cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa234837740 1 --2- 172.21.15.5:0/2548326474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fe19ba4a0 0x563fe19cab90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa234837740 1 -- 172.21.15.5:0/2548326474 >> 172.21.15.5:0/2548326474 conn(0x563fe19b6a30 msgr2=0x563fe19c3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:23.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.757+0000 7fa20b7fe640 0 client.26762 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:23.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa234837740 1 -- 172.21.15.5:0/2548326474 shutdown_connections 2024-09-19T06:10:23.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa234837740 1 -- 172.21.15.5:0/2548326474 wait complete. 2024-09-19T06:10:23.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa234837740 20 client.26762 trim_cache size 0 max 16384 2024-09-19T06:10:23.766 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:10:23.767 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:10:23.767 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:10:23.767 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:10:23.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa23245e640 1 -- 172.21.15.5:0/66121167 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2140a3430 msgr2=0x7fa2140a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:23.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa23245e640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2140a3430 0x7fa2140a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x563fe19b7890 tx=0x7fa21c0bc040 comp rx=0 tx=0).stop 2024-09-19T06:10:23.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa23245e640 1 -- 172.21.15.5:0/66121167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c103fe0 msgr2=0x7fa22c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:23.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa23245e640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c103fe0 0x7fa22c19a610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00e9f0 tx=0x7fa21c00edf0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa23245e640 1 -- 172.21.15.5:0/66121167 shutdown_connections 2024-09-19T06:10:23.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa23245e640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c109110 0x7fa22c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa23245e640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2140a3430 0x7fa2140a58f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa23245e640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c104990 0x7fa22c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa23245e640 1 --2- 172.21.15.5:0/66121167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c103fe0 0x7fa22c19a610 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:23.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.758+0000 7fa23245e640 1 -- 172.21.15.5:0/66121167 >> 172.21.15.5:0/66121167 conn(0x7fa22c0ffa20 msgr2=0x7fa22c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:23.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.759+0000 7fa23245e640 1 -- 172.21.15.5:0/66121167 shutdown_connections 2024-09-19T06:10:23.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:23.759+0000 7fa23245e640 1 -- 172.21.15.5:0/66121167 wait complete. 2024-09-19T06:10:23.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 178089 2.73 MB/sec execute 229 sec latency 49.438 ms 2024-09-19T06:10:24.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:24 smithi005 ceph-mon[30955]: pgmap v217: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-09-19T06:10:24.591 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:10:24.591 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:10:24.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:24 smithi049 ceph-mon[34640]: pgmap v217: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-09-19T06:10:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:24 smithi157 ceph-mon[35346]: pgmap v217: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-09-19T06:10:24.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 178603 2.72 MB/sec execute 230 sec latency 12.455 ms 2024-09-19T06:10:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:10:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.417+0000 7ff1bbab3640 1 -- 172.21.15.5:0/1107722038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b4102d30 msgr2=0x7ff1b4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.417+0000 7ff1bbab3640 1 --2- 172.21.15.5:0/1107722038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b4102d30 0x7ff1b4103110 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009930 tx=0x7ff1b002a420 comp rx=0 tx=0).stop 2024-09-19T06:10:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.417+0000 7ff1bbab3640 1 -- 172.21.15.5:0/1107722038 shutdown_connections 2024-09-19T06:10:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.417+0000 7ff1bbab3640 1 --2- 172.21.15.5:0/1107722038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1b41082a0 0x7ff1b410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.417+0000 7ff1bbab3640 1 --2- 172.21.15.5:0/1107722038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1b41036e0 0x7ff1b4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.417+0000 7ff1bbab3640 1 --2- 172.21.15.5:0/1107722038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b4102d30 0x7ff1b4103110 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.417+0000 7ff1bbab3640 1 -- 172.21.15.5:0/1107722038 >> 172.21.15.5:0/1107722038 conn(0x7ff1b40fc7d0 msgr2=0x7ff1b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.417+0000 7ff1bbab3640 1 -- 172.21.15.5:0/1107722038 shutdown_connections 2024-09-19T06:10:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.417+0000 7ff1bbab3640 1 -- 172.21.15.5:0/1107722038 wait complete. 2024-09-19T06:10:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.418+0000 7ff1bbab3640 1 Processor -- start 2024-09-19T06:10:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.418+0000 7ff1bbab3640 1 -- start start 2024-09-19T06:10:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.418+0000 7ff1bbab3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1b4102d30 0x7ff1b4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.418+0000 7ff1bbab3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b41036e0 0x7ff1b4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.418+0000 7ff1bbab3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1b41082a0 0x7ff1b419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.418+0000 7ff1bbab3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4111f50 con 0x7ff1b41036e0 2024-09-19T06:10:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.418+0000 7ff1bbab3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4111dd0 con 0x7ff1b41082a0 2024-09-19T06:10:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.418+0000 7ff1bbab3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41120d0 con 0x7ff1b4102d30 2024-09-19T06:10:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.418+0000 7ff1b904e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b41036e0 0x7ff1b4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.418+0000 7ff1b904e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b41036e0 0x7ff1b4198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56750/0 (socket says 172.21.15.5:56750) 2024-09-19T06:10:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.418+0000 7ff1b904e640 1 -- 172.21.15.5:0/3537984102 learned_addr learned my addr 172.21.15.5:0/3537984102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.418+0000 7ff1b984f640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1b4102d30 0x7ff1b4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.419+0000 7ff1b904e640 1 -- 172.21.15.5:0/3537984102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1b4102d30 msgr2=0x7ff1b4198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.419+0000 7ff1b904e640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1b4102d30 0x7ff1b4198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.419+0000 7ff1b904e640 1 -- 172.21.15.5:0/3537984102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1b41082a0 msgr2=0x7ff1b419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.419+0000 7ff1b904e640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1b41082a0 0x7ff1b419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.419+0000 7ff1b904e640 1 -- 172.21.15.5:0/3537984102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b419d4c0 con 0x7ff1b41036e0 2024-09-19T06:10:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.419+0000 7ff1b904e640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b41036e0 0x7ff1b4198a50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400e9c0 tx=0x7ff1a400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.419+0000 7ff1aaffd640 1 -- 172.21.15.5:0/3537984102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a400ccc0 con 0x7ff1b41036e0 2024-09-19T06:10:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.419+0000 7ff1aaffd640 1 -- 172.21.15.5:0/3537984102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1a40054d0 con 0x7ff1b41036e0 2024-09-19T06:10:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.419+0000 7ff1bbab3640 1 -- 172.21.15.5:0/3537984102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b419d7b0 con 0x7ff1b41036e0 2024-09-19T06:10:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.419+0000 7ff1bbab3640 1 -- 172.21.15.5:0/3537984102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b41a5090 con 0x7ff1b41036e0 2024-09-19T06:10:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.419+0000 7ff1aaffd640 1 -- 172.21.15.5:0/3537984102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4010430 con 0x7ff1b41036e0 2024-09-19T06:10:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.420+0000 7ff1bbab3640 1 -- 172.21.15.5:0/3537984102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b406a850 con 0x7ff1b41036e0 2024-09-19T06:10:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.422+0000 7ff1aaffd640 1 -- 172.21.15.5:0/3537984102 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff1a4005000 con 0x7ff1b41036e0 2024-09-19T06:10:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.422+0000 7ff1aaffd640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff1a00a33b0 0x7ff1a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.422+0000 7ff1b984f640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff1a00a33b0 0x7ff1a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.422+0000 7ff1aaffd640 1 -- 172.21.15.5:0/3537984102 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7ff1a40d0930 con 0x7ff1b41036e0 2024-09-19T06:10:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.423+0000 7ff1b984f640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff1a00a33b0 0x7ff1a00a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009900 tx=0x7ff1b0000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.426+0000 7ff1aaffd640 1 -- 172.21.15.5:0/3537984102 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff1a40052f0 con 0x7ff1b41036e0 2024-09-19T06:10:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.665+0000 7ff1bbab3640 1 -- 172.21.15.5:0/3537984102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1b410a310 con 0x7ff1b41036e0 2024-09-19T06:10:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.666+0000 7ff1aaffd640 1 -- 172.21.15.5:0/3537984102 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11959 (secure 0 0 0) 0x7ff1a409d2f0 con 0x7ff1b41036e0 2024-09-19T06:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-19T06:07:24:781633+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:07:24.019631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:10:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-19T06:10:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.669+0000 7ff1bbab3640 1 -- 172.21.15.5:0/3537984102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff1a00a33b0 msgr2=0x7ff1a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.669+0000 7ff1bbab3640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff1a00a33b0 0x7ff1a00a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009900 tx=0x7ff1b0000bc0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.669+0000 7ff1bbab3640 1 -- 172.21.15.5:0/3537984102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b41036e0 msgr2=0x7ff1b4198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.669+0000 7ff1bbab3640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b41036e0 0x7ff1b4198a50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400e9c0 tx=0x7ff1a400ee80 comp rx=0 tx=0).stop 2024-09-19T06:10:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.669+0000 7ff1bbab3640 1 -- 172.21.15.5:0/3537984102 shutdown_connections 2024-09-19T06:10:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.669+0000 7ff1bbab3640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1b41082a0 0x7ff1b419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.669+0000 7ff1bbab3640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b41036e0 0x7ff1b4198a50 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.669+0000 7ff1bbab3640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff1a00a33b0 0x7ff1a00a5870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.669+0000 7ff1bbab3640 1 --2- 172.21.15.5:0/3537984102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1b4102d30 0x7ff1b4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.669+0000 7ff1bbab3640 1 -- 172.21.15.5:0/3537984102 >> 172.21.15.5:0/3537984102 conn(0x7ff1b40fc7d0 msgr2=0x7ff1b4105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.669+0000 7ff1bbab3640 1 -- 172.21.15.5:0/3537984102 shutdown_connections 2024-09-19T06:10:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:25.669+0000 7ff1bbab3640 1 -- 172.21.15.5:0/3537984102 wait complete. 2024-09-19T06:10:25.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 179131 2.72 MB/sec execute 231 sec latency 21.369 ms 2024-09-19T06:10:26.426 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:26 smithi005 ceph-mon[30955]: pgmap v218: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-09-19T06:10:26.426 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:26 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3537984102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:10:26.458 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:10:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:26 smithi049 ceph-mon[34640]: pgmap v218: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-09-19T06:10:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:26 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3537984102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:10:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:10:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:26 smithi157 ceph-mon[35346]: pgmap v218: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-09-19T06:10:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:26 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3537984102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:10:26.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 179725 2.72 MB/sec execute 232 sec latency 20.843 ms 2024-09-19T06:10:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.285+0000 7fa164e47640 1 -- 172.21.15.5:0/1956031029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa16010c180 msgr2=0x7fa16010e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.285+0000 7fa164e47640 1 --2- 172.21.15.5:0/1956031029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa16010c180 0x7fa16010e620 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa158009ca0 tx=0x7fa15802a950 comp rx=0 tx=0).stop 2024-09-19T06:10:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.285+0000 7fa164e47640 1 -- 172.21.15.5:0/1956031029 shutdown_connections 2024-09-19T06:10:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.285+0000 7fa164e47640 1 --2- 172.21.15.5:0/1956031029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa16010c180 0x7fa16010e620 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.285+0000 7fa164e47640 1 --2- 172.21.15.5:0/1956031029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160103670 0x7fa16010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.285+0000 7fa164e47640 1 --2- 172.21.15.5:0/1956031029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa160102d50 0x7fa160103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.285+0000 7fa164e47640 1 -- 172.21.15.5:0/1956031029 >> 172.21.15.5:0/1956031029 conn(0x7fa1600fc7d0 msgr2=0x7fa1600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.285+0000 7fa164e47640 1 -- 172.21.15.5:0/1956031029 shutdown_connections 2024-09-19T06:10:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.285+0000 7fa164e47640 1 -- 172.21.15.5:0/1956031029 wait complete. 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa164e47640 1 Processor -- start 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa164e47640 1 -- start start 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa164e47640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102d50 0x7fa1601009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa164e47640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160103670 0x7fa160100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa164e47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa16010c180 0x7fa160101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa164e47640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa160111df0 con 0x7fa160103670 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa164e47640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa160111c70 con 0x7fa160102d50 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa164e47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa160111f70 con 0x7fa16010c180 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa15e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102d50 0x7fa1601009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa157fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160103670 0x7fa160100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa15e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102d50 0x7fa1601009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44470/0 (socket says 172.21.15.5:44470) 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa15e59c640 1 -- 172.21.15.5:0/2866835185 learned_addr learned my addr 172.21.15.5:0/2866835185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.286+0000 7fa15ed9d640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa16010c180 0x7fa160101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.287+0000 7fa15e59c640 1 -- 172.21.15.5:0/2866835185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa16010c180 msgr2=0x7fa160101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:27.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.287+0000 7fa15e59c640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa16010c180 0x7fa160101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.287+0000 7fa15e59c640 1 -- 172.21.15.5:0/2866835185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160103670 msgr2=0x7fa160100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:27.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.287+0000 7fa15e59c640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160103670 0x7fa160100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.287+0000 7fa15e59c640 1 -- 172.21.15.5:0/2866835185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1601a6890 con 0x7fa160102d50 2024-09-19T06:10:27.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.287+0000 7fa15e59c640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102d50 0x7fa1601009b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa15000d950 tx=0x7fa15000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:27.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.287+0000 7fa1577fe640 1 -- 172.21.15.5:0/2866835185 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150017030 con 0x7fa160102d50 2024-09-19T06:10:27.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.287+0000 7fa1577fe640 1 -- 172.21.15.5:0/2866835185 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa150004020 con 0x7fa160102d50 2024-09-19T06:10:27.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.288+0000 7fa1577fe640 1 -- 172.21.15.5:0/2866835185 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1500047a0 con 0x7fa160102d50 2024-09-19T06:10:27.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.288+0000 7fa164e47640 1 -- 172.21.15.5:0/2866835185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1601a6b80 con 0x7fa160102d50 2024-09-19T06:10:27.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.288+0000 7fa164e47640 1 -- 172.21.15.5:0/2866835185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1601a7110 con 0x7fa160102d50 2024-09-19T06:10:27.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.290+0000 7fa1577fe640 1 -- 172.21.15.5:0/2866835185 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa1500041d0 con 0x7fa160102d50 2024-09-19T06:10:27.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.290+0000 7fa167220740 1 Processor -- start 2024-09-19T06:10:27.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa167220740 1 -- start start 2024-09-19T06:10:27.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa167220740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9db40 0x555f92b9c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa167220740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 0x555f92b9ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa167220740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92ba7370 0x555f92ba9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa167220740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555f92b022f0 con 0x555f92b9db40 2024-09-19T06:10:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa167220740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555f92aa9cf0 con 0x555f92b9c5d0 2024-09-19T06:10:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa167220740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555f92afe2b0 con 0x555f92ba7370 2024-09-19T06:10:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa15e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 0x555f92b9ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa15e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 0x555f92b9ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44486/0 (socket says 172.21.15.5:44486) 2024-09-19T06:10:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa15e59c640 1 -- 172.21.15.5:0/4272436531 learned_addr learned my addr 172.21.15.5:0/4272436531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa15ed9d640 1 --2- 172.21.15.5:0/4272436531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9db40 0x555f92b9c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa157fff640 1 --2- 172.21.15.5:0/4272436531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92ba7370 0x555f92ba9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa15e59c640 1 -- 172.21.15.5:0/4272436531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92ba7370 msgr2=0x555f92ba9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa15e59c640 1 --2- 172.21.15.5:0/4272436531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92ba7370 0x555f92ba9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa15e59c640 1 -- 172.21.15.5:0/4272436531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9db40 msgr2=0x555f92b9c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:27.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa15e59c640 1 --2- 172.21.15.5:0/4272436531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9db40 0x555f92b9c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.291+0000 7fa15e59c640 1 -- 172.21.15.5:0/4272436531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f92a8e3a0 con 0x555f92b9c5d0 2024-09-19T06:10:27.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.292+0000 7fa157fff640 1 --2- 172.21.15.5:0/4272436531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92ba7370 0x555f92ba9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:10:27.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.292+0000 7fa1577fe640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa14c0a3430 0x7fa14c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.292+0000 7fa15ed9d640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa14c0a3430 0x7fa14c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.292+0000 7fa1577fe640 1 -- 172.21.15.5:0/2866835185 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fa1500cf6e0 con 0x7fa160102d50 2024-09-19T06:10:27.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.292+0000 7fa15e59c640 1 --2- 172.21.15.5:0/4272436531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 0x555f92b9ca10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa1500cccf0 tx=0x7fa1500d5ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcca3f7e50f78908 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:27.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.292+0000 7fa15ed9d640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa14c0a3430 0x7fa14c0a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa158005e30 tx=0x7fa158005c50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:27.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.292+0000 7fa154ff9640 1 -- 172.21.15.5:0/4272436531 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150097be0 con 0x555f92b9c5d0 2024-09-19T06:10:27.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.293+0000 7fa154ff9640 1 -- 172.21.15.5:0/4272436531 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1500860d0 con 0x555f92b9c5d0 2024-09-19T06:10:27.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.293+0000 7fa154ff9640 1 -- 172.21.15.5:0/4272436531 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15009b2a0 con 0x555f92b9c5d0 2024-09-19T06:10:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.293+0000 7fa167220740 1 -- 172.21.15.5:0/4272436531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 msgr2=0x555f92b9ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.293+0000 7fa167220740 1 --2- 172.21.15.5:0/4272436531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 0x555f92b9ca10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa1500cccf0 tx=0x7fa1500d5ef0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.293+0000 7fa167220740 1 -- 172.21.15.5:0/4272436531 shutdown_connections 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.293+0000 7fa167220740 1 --2- 172.21.15.5:0/4272436531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92ba7370 0x555f92ba9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.293+0000 7fa167220740 1 --2- 172.21.15.5:0/4272436531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 0x555f92b9ca10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.293+0000 7fa167220740 1 --2- 172.21.15.5:0/4272436531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9db40 0x555f92b9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.293+0000 7fa167220740 1 -- 172.21.15.5:0/4272436531 >> 172.21.15.5:0/4272436531 conn(0x555f92b65310 msgr2=0x555f92b98c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.294+0000 7fa167220740 1 -- 172.21.15.5:0/4272436531 shutdown_connections 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.294+0000 7fa167220740 1 -- 172.21.15.5:0/4272436531 wait complete. 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.294+0000 7fa167220740 1 -- start start 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.294+0000 7fa167220740 1 Processor -- start 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.294+0000 7fa167220740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 0x555f92ba6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.294+0000 7fa167220740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92b9db40 0x555f92ba3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.294+0000 7fa167220740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92ba7370 0x555f92ba45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.294+0000 7fa167220740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555f92baa070 con 0x555f92b9c5d0 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.294+0000 7fa167220740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555f92abd060 con 0x555f92ba7370 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.294+0000 7fa167220740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555f92baa1f0 con 0x555f92b9db40 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.294+0000 7fa167220740 10 client.0 _ll_register_callbacks cb 0x555f928108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.294+0000 7fa15e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 0x555f92ba6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa15ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92ba7370 0x555f92ba45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa15e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 0x555f92ba6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56774/0 (socket says 172.21.15.5:56774) 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa15e59c640 1 -- 172.21.15.5:0/1163214108 learned_addr learned my addr 172.21.15.5:0/1163214108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa157fff640 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92b9db40 0x555f92ba3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa15ed9d640 1 -- 172.21.15.5:0/1163214108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92b9db40 msgr2=0x555f92ba3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa15ed9d640 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92b9db40 0x555f92ba3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa15ed9d640 1 -- 172.21.15.5:0/1163214108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 msgr2=0x555f92ba6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa15ed9d640 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 0x555f92ba6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa15ed9d640 1 -- 172.21.15.5:0/1163214108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f928733c0 con 0x555f92ba7370 2024-09-19T06:10:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa157fff640 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92b9db40 0x555f92ba3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:10:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa15ed9d640 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92ba7370 0x555f92ba45d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa158005cc0 tx=0x7fa1580095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b79b4b268a5b3627 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa145ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:10:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa145ffb640 1 -- 172.21.15.5:0/1163214108 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158055060 con 0x555f92ba7370 2024-09-19T06:10:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa145ffb640 1 -- 172.21.15.5:0/1163214108 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fa158004250 con 0x555f92ba7370 2024-09-19T06:10:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.295+0000 7fa167220740 1 -- 172.21.15.5:0/1163214108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1500860d0 con 0x555f92ba7370 2024-09-19T06:10:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.296+0000 7fa145ffb640 1 -- 172.21.15.5:0/1163214108 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa158034040 con 0x555f92ba7370 2024-09-19T06:10:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.296+0000 7fa167220740 10 client.26777 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.296+0000 7fa167220740 1 -- 172.21.15.5:0/1163214108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555f92bb5b00 con 0x555f92ba7370 2024-09-19T06:10:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.296+0000 7fa145ffb640 1 -- 172.21.15.5:0/1163214108 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1500860d0 con 0x555f92ba7370 2024-09-19T06:10:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.296+0000 7fa145ffb640 1 -- 172.21.15.5:0/1163214108 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa1580309b0 con 0x555f92ba7370 2024-09-19T06:10:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.296+0000 7fa167220740 10 client.26777 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.296+0000 7fa167220740 10 client.26777 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:10:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.296+0000 7fa167220740 20 client.26777 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.296+0000 7fa167220740 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f92bb6fc0 0x555f92bb93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.296+0000 7fa167220740 4 client.26777 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.296+0000 7fa167220740 1 -- 172.21.15.5:0/1163214108 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f92b0f860 con 0x555f92bb6fc0 2024-09-19T06:10:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.296+0000 7fa15e59c640 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f92bb6fc0 0x555f92bb93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.297+0000 7fa15e59c640 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f92bb6fc0 0x555f92bb93f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=406b19b66d1ed948 server_cookie=1ca93fd264958966 in_seq=0 out_seq=0 2024-09-19T06:10:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.297+0000 7fa145ffb640 10 client.26777 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:10:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.297+0000 7fa145ffb640 1 -- 172.21.15.5:0/1163214108 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f92b0f860 con 0x555f92bb6fc0 2024-09-19T06:10:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.297+0000 7fa145ffb640 10 client.26777 handle_command_reply: tid=0 2024-09-19T06:10:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.297+0000 7fa167220740 1 client.26777 shutdown 2024-09-19T06:10:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.297+0000 7fa167220740 1 -- 172.21.15.5:0/1163214108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92ba7370 msgr2=0x555f92ba45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.297+0000 7fa167220740 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92ba7370 0x555f92ba45d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa158005cc0 tx=0x7fa1580095c0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.298+0000 7fa167220740 1 -- 172.21.15.5:0/1163214108 shutdown_connections 2024-09-19T06:10:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.298+0000 7fa167220740 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f92bb6fc0 0x555f92bb93f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.298+0000 7fa167220740 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92ba7370 0x555f92ba45d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.298+0000 7fa167220740 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92b9db40 0x555f92ba3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.298+0000 7fa145ffb640 0 client.26777 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:10:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.298+0000 7fa167220740 1 --2- 172.21.15.5:0/1163214108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 0x555f92ba6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.298+0000 7fa167220740 1 -- 172.21.15.5:0/1163214108 >> 172.21.15.5:0/1163214108 conn(0x555f92b65310 msgr2=0x555f92b9e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.298+0000 7fa167220740 1 -- 172.21.15.5:0/1163214108 shutdown_connections 2024-09-19T06:10:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.298+0000 7fa167220740 1 -- 172.21.15.5:0/1163214108 wait complete. 2024-09-19T06:10:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.298+0000 7fa167220740 20 client.26777 trim_cache size 0 max 16384 2024-09-19T06:10:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.311+0000 7fa167220740 1 Processor -- start 2024-09-19T06:10:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.311+0000 7fa167220740 1 -- start start 2024-09-19T06:10:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa167220740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 0x555f92b98e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa167220740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92b9db40 0x555f92bb3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa167220740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92ba7370 0x555f92ba38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa167220740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601afad0 con 0x555f92ba7370 2024-09-19T06:10:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa167220740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601b1610 con 0x555f92b9c5d0 2024-09-19T06:10:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa167220740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601b01a0 con 0x555f92b9db40 2024-09-19T06:10:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa15e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 0x555f92b98e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa15e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 0x555f92b98e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44516/0 (socket says 172.21.15.5:44516) 2024-09-19T06:10:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa15e59c640 1 -- 172.21.15.5:0/3415212419 learned_addr learned my addr 172.21.15.5:0/3415212419 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa15ed9d640 1 --2- 172.21.15.5:0/3415212419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92ba7370 0x555f92ba38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa157fff640 1 --2- 172.21.15.5:0/3415212419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92b9db40 0x555f92bb3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa15e59c640 1 -- 172.21.15.5:0/3415212419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92b9db40 msgr2=0x555f92bb3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa15e59c640 1 --2- 172.21.15.5:0/3415212419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92b9db40 0x555f92bb3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa15e59c640 1 -- 172.21.15.5:0/3415212419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92ba7370 msgr2=0x555f92ba38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa15e59c640 1 --2- 172.21.15.5:0/3415212419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92ba7370 0x555f92ba38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa15e59c640 1 -- 172.21.15.5:0/3415212419 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f92ba2f90 con 0x555f92b9c5d0 2024-09-19T06:10:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa157fff640 1 --2- 172.21.15.5:0/3415212419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92b9db40 0x555f92bb3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:10:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa15ed9d640 1 --2- 172.21.15.5:0/3415212419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92ba7370 0x555f92ba38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.312+0000 7fa15e59c640 1 --2- 172.21.15.5:0/3415212419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 0x555f92b98e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa150005c60 tx=0x7fa150000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7e3678285cdd31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.313+0000 7fa147fff640 1 -- 172.21.15.5:0/3415212419 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1500b3020 con 0x555f92b9c5d0 2024-09-19T06:10:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.313+0000 7fa147fff640 1 -- 172.21.15.5:0/3415212419 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa15009c040 con 0x555f92b9c5d0 2024-09-19T06:10:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.313+0000 7fa147fff640 1 -- 172.21.15.5:0/3415212419 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1500afe60 con 0x555f92b9c5d0 2024-09-19T06:10:27.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.313+0000 7fa167220740 1 -- 172.21.15.5:0/3415212419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 msgr2=0x555f92b98e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:27.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.313+0000 7fa167220740 1 --2- 172.21.15.5:0/3415212419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 0x555f92b98e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa150005c60 tx=0x7fa150000bc0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.313+0000 7fa167220740 1 -- 172.21.15.5:0/3415212419 shutdown_connections 2024-09-19T06:10:27.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.313+0000 7fa167220740 1 --2- 172.21.15.5:0/3415212419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92ba7370 0x555f92ba38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.313+0000 7fa167220740 1 --2- 172.21.15.5:0/3415212419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92b9db40 0x555f92bb3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.313+0000 7fa167220740 1 --2- 172.21.15.5:0/3415212419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9c5d0 0x555f92b98e10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.314+0000 7fa167220740 1 -- 172.21.15.5:0/3415212419 >> 172.21.15.5:0/3415212419 conn(0x555f92b65310 msgr2=0x555f92b98700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:27.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.314+0000 7fa167220740 1 -- 172.21.15.5:0/3415212419 shutdown_connections 2024-09-19T06:10:27.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.314+0000 7fa167220740 1 -- 172.21.15.5:0/3415212419 wait complete. 2024-09-19T06:10:27.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.314+0000 7fa167220740 1 -- start start 2024-09-19T06:10:27.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.314+0000 7fa167220740 1 Processor -- start 2024-09-19T06:10:27.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.314+0000 7fa167220740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 0x555f92bafdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.314+0000 7fa167220740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9db40 0x555f92bab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.314+0000 7fa167220740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92ba7370 0x555f92bab890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.314+0000 7fa167220740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601b0f40 con 0x555f92b9c5d0 2024-09-19T06:10:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.314+0000 7fa167220740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601b0870 con 0x555f92b9db40 2024-09-19T06:10:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.314+0000 7fa167220740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601adad0 con 0x555f92ba7370 2024-09-19T06:10:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.314+0000 7fa167220740 10 client.0 _ll_register_callbacks cb 0x7fa158009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa15e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 0x555f92bafdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa157fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9db40 0x555f92bab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa15ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92ba7370 0x555f92bab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa15e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 0x555f92bafdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56786/0 (socket says 172.21.15.5:56786) 2024-09-19T06:10:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa15e59c640 1 -- 172.21.15.5:0/2053333359 learned_addr learned my addr 172.21.15.5:0/2053333359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa157fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9db40 0x555f92bab100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44530/0 (socket says 172.21.15.5:44530) 2024-09-19T06:10:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa15ed9d640 1 -- 172.21.15.5:0/2053333359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9db40 msgr2=0x555f92bab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa15ed9d640 1 --2- 172.21.15.5:0/2053333359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9db40 0x555f92bab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa15ed9d640 1 -- 172.21.15.5:0/2053333359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 msgr2=0x555f92bafdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa15ed9d640 1 --2- 172.21.15.5:0/2053333359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 0x555f92bafdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa15ed9d640 1 -- 172.21.15.5:0/2053333359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f92c20c90 con 0x555f92ba7370 2024-09-19T06:10:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa15e59c640 1 --2- 172.21.15.5:0/2053333359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 0x555f92bafdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa15ed9d640 1 --2- 172.21.15.5:0/2053333359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92ba7370 0x555f92bab890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa158030d50 tx=0x7fa15803a6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad8224b7c06b42b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.315+0000 7fa145ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:10:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa145ffb640 1 -- 172.21.15.5:0/2053333359 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158055070 con 0x555f92ba7370 2024-09-19T06:10:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa167220740 1 -- 172.21.15.5:0/2053333359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa15009c040 con 0x555f92ba7370 2024-09-19T06:10:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa145ffb640 1 -- 172.21.15.5:0/2053333359 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fa158004460 con 0x555f92ba7370 2024-09-19T06:10:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa145ffb640 1 -- 172.21.15.5:0/2053333359 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa158004660 con 0x555f92ba7370 2024-09-19T06:10:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa167220740 10 client.26701 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa167220740 1 -- 172.21.15.5:0/2053333359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555f92c21e70 con 0x555f92ba7370 2024-09-19T06:10:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa145ffb640 1 -- 172.21.15.5:0/2053333359 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa15009c040 con 0x555f92ba7370 2024-09-19T06:10:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa145ffb640 1 -- 172.21.15.5:0/2053333359 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa1580305b0 con 0x555f92ba7370 2024-09-19T06:10:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa167220740 10 client.26701 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa167220740 10 client.26701 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa167220740 20 client.26701 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa167220740 1 --2- 172.21.15.5:0/2053333359 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f92c22590 0x555f92c249b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa167220740 4 client.26701 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.316+0000 7fa167220740 1 -- 172.21.15.5:0/2053333359 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f92b09d70 con 0x555f92c22590 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa15e59c640 1 --2- 172.21.15.5:0/2053333359 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f92c22590 0x555f92c249b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa145ffb640 10 client.26701 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa15e59c640 1 --2- 172.21.15.5:0/2053333359 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f92c22590 0x555f92c249b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c2b200d15de2879c server_cookie=97d5ee61dcd5759e in_seq=0 out_seq=0 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa145ffb640 1 -- 172.21.15.5:0/2053333359 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f92b09d70 con 0x555f92c22590 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa145ffb640 10 client.26701 handle_command_reply: tid=0 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa167220740 1 client.26701 shutdown 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa167220740 1 -- 172.21.15.5:0/2053333359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92ba7370 msgr2=0x555f92bab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa167220740 1 --2- 172.21.15.5:0/2053333359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92ba7370 0x555f92bab890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa158030d50 tx=0x7fa15803a6e0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa167220740 1 -- 172.21.15.5:0/2053333359 shutdown_connections 2024-09-19T06:10:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa167220740 1 --2- 172.21.15.5:0/2053333359 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f92c22590 0x555f92c249b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa145ffb640 0 client.26701 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:10:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa167220740 1 --2- 172.21.15.5:0/2053333359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f92ba7370 0x555f92bab890 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa167220740 1 --2- 172.21.15.5:0/2053333359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f92b9db40 0x555f92bab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.317+0000 7fa167220740 1 --2- 172.21.15.5:0/2053333359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f92b9c5d0 0x555f92bafdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa167220740 1 -- 172.21.15.5:0/2053333359 >> 172.21.15.5:0/2053333359 conn(0x555f92b65310 msgr2=0x555f92ba5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa167220740 1 -- 172.21.15.5:0/2053333359 shutdown_connections 2024-09-19T06:10:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa167220740 1 -- 172.21.15.5:0/2053333359 wait complete. 2024-09-19T06:10:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa167220740 20 client.26701 trim_cache size 0 max 16384 2024-09-19T06:10:27.329 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:10:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa164e47640 1 -- 172.21.15.5:0/2866835185 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa14c0a3430 msgr2=0x7fa14c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa164e47640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa14c0a3430 0x7fa14c0a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa158005e30 tx=0x7fa158005c50 comp rx=0 tx=0).stop 2024-09-19T06:10:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa164e47640 1 -- 172.21.15.5:0/2866835185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102d50 msgr2=0x7fa1601009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa164e47640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102d50 0x7fa1601009b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa15000d950 tx=0x7fa15000de10 comp rx=0 tx=0).stop 2024-09-19T06:10:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa164e47640 1 -- 172.21.15.5:0/2866835185 shutdown_connections 2024-09-19T06:10:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa164e47640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa14c0a3430 0x7fa14c0a58f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa164e47640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa16010c180 0x7fa160101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa164e47640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa160103670 0x7fa160100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa164e47640 1 --2- 172.21.15.5:0/2866835185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa160102d50 0x7fa1601009b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.318+0000 7fa164e47640 1 -- 172.21.15.5:0/2866835185 >> 172.21.15.5:0/2866835185 conn(0x7fa1600fc7d0 msgr2=0x7fa1600fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.319+0000 7fa164e47640 1 -- 172.21.15.5:0/2866835185 shutdown_connections 2024-09-19T06:10:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:27.319+0000 7fa164e47640 1 -- 172.21.15.5:0/2866835185 wait complete. 2024-09-19T06:10:27.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 180300 2.73 MB/sec execute 233 sec latency 19.916 ms 2024-09-19T06:10:28.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:10:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:10:28.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:28 smithi005 ceph-mon[30955]: pgmap v219: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-19T06:10:28.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:28 smithi049 ceph-mon[34640]: pgmap v219: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-19T06:10:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:28 smithi157 ceph-mon[35346]: pgmap v219: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-19T06:10:28.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 180723 2.73 MB/sec execute 234 sec latency 69.878 ms 2024-09-19T06:10:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.916+0000 7fed49f48640 1 -- 172.21.15.5:0/3622065851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed44106f10 msgr2=0x7fed4410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.916+0000 7fed49f48640 1 --2- 172.21.15.5:0/3622065851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed44106f10 0x7fed4410ea90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009ca0 tx=0x7fed3c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.916+0000 7fed49f48640 1 -- 172.21.15.5:0/3622065851 shutdown_connections 2024-09-19T06:10:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.916+0000 7fed49f48640 1 --2- 172.21.15.5:0/3622065851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed44106f10 0x7fed4410ea90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.916+0000 7fed49f48640 1 --2- 172.21.15.5:0/3622065851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed44102790 0x7fed441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.916+0000 7fed49f48640 1 --2- 172.21.15.5:0/3622065851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed44101de0 0x7fed441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.916+0000 7fed49f48640 1 -- 172.21.15.5:0/3622065851 >> 172.21.15.5:0/3622065851 conn(0x7fed440fd8c0 msgr2=0x7fed440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.916+0000 7fed49f48640 1 -- 172.21.15.5:0/3622065851 shutdown_connections 2024-09-19T06:10:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.916+0000 7fed49f48640 1 -- 172.21.15.5:0/3622065851 wait complete. 2024-09-19T06:10:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.917+0000 7fed49f48640 1 Processor -- start 2024-09-19T06:10:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.917+0000 7fed49f48640 1 -- start start 2024-09-19T06:10:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed49f48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed44101de0 0x7fed44198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed49f48640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed44102790 0x7fed44198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed49f48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed44106f10 0x7fed4419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed49f48640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed44111e20 con 0x7fed44101de0 2024-09-19T06:10:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed49f48640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed44111ca0 con 0x7fed44102790 2024-09-19T06:10:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed49f48640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed44111fa0 con 0x7fed44106f10 2024-09-19T06:10:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed44101de0 0x7fed44198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed44102790 0x7fed44198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed44101de0 0x7fed44198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56792/0 (socket says 172.21.15.5:56792) 2024-09-19T06:10:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed437fe640 1 -- 172.21.15.5:0/3798943450 learned_addr learned my addr 172.21.15.5:0/3798943450 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed44102790 0x7fed44198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44550/0 (socket says 172.21.15.5:44550) 2024-09-19T06:10:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed43fff640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed44106f10 0x7fed4419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed437fe640 1 -- 172.21.15.5:0/3798943450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed44106f10 msgr2=0x7fed4419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed437fe640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed44106f10 0x7fed4419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed437fe640 1 -- 172.21.15.5:0/3798943450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed44102790 msgr2=0x7fed44198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed437fe640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed44102790 0x7fed44198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.918+0000 7fed437fe640 1 -- 172.21.15.5:0/3798943450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4419d2d0 con 0x7fed44101de0 2024-09-19T06:10:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.919+0000 7fed437fe640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed44101de0 0x7fed44198320 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fed3400e9f0 tx=0x7fed3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.919+0000 7fed40ff9640 1 -- 172.21.15.5:0/3798943450 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3400cd00 con 0x7fed44101de0 2024-09-19T06:10:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.919+0000 7fed40ff9640 1 -- 172.21.15.5:0/3798943450 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed340054d0 con 0x7fed44101de0 2024-09-19T06:10:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.919+0000 7fed40ff9640 1 -- 172.21.15.5:0/3798943450 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34010430 con 0x7fed44101de0 2024-09-19T06:10:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.919+0000 7fed49f48640 1 -- 172.21.15.5:0/3798943450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4419d5c0 con 0x7fed44101de0 2024-09-19T06:10:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.919+0000 7fed49f48640 1 -- 172.21.15.5:0/3798943450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed441a4ea0 con 0x7fed44101de0 2024-09-19T06:10:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.921+0000 7fed40ff9640 1 -- 172.21.15.5:0/3798943450 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fed34005020 con 0x7fed44101de0 2024-09-19T06:10:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed4c321740 1 Processor -- start 2024-09-19T06:10:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed4c321740 1 -- start start 2024-09-19T06:10:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed4c321740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5afb40 0x564a1b5ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed4c321740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5ae5d0 0x564a1b5aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed4c321740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed4c321740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a1b5142f0 con 0x564a1b5afb40 2024-09-19T06:10:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed4c321740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564a1b4bbcf0 con 0x564a1b5bd780 2024-09-19T06:10:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed4c321740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564a1b5102b0 con 0x564a1b5ae5d0 2024-09-19T06:10:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed40ff9640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fed2c0a3400 0x7fed2c0a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5bfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44562/0 (socket says 172.21.15.5:44562) 2024-09-19T06:10:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed42ffd640 1 -- 172.21.15.5:0/682610519 learned_addr learned my addr 172.21.15.5:0/682610519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed437fe640 1 --2- 172.21.15.5:0/682610519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5ae5d0 0x564a1b5aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.922+0000 7fed43fff640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fed2c0a3400 0x7fed2c0a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed40ff9640 1 -- 172.21.15.5:0/3798943450 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fed340d00c0 con 0x7fed44101de0 2024-09-19T06:10:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed43fff640 1 --2- 172.21.15.5:0/682610519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5afb40 0x564a1b5ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed42ffd640 1 -- 172.21.15.5:0/682610519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5ae5d0 msgr2=0x564a1b5aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed42ffd640 1 --2- 172.21.15.5:0/682610519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5ae5d0 0x564a1b5aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed42ffd640 1 -- 172.21.15.5:0/682610519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5afb40 msgr2=0x564a1b5ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed42ffd640 1 --2- 172.21.15.5:0/682610519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5afb40 0x564a1b5ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed42ffd640 1 -- 172.21.15.5:0/682610519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1b4a03a0 con 0x564a1b5bd780 2024-09-19T06:10:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed43fff640 1 --2- 172.21.15.5:0/682610519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5afb40 0x564a1b5ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:10:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed42ffd640 1 --2- 172.21.15.5:0/682610519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5bfb70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fed38009c10 tx=0x7fed38011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=911db4d4a6216352 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed29ffb640 1 -- 172.21.15.5:0/682610519 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38004230 con 0x564a1b5bd780 2024-09-19T06:10:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed43fff640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fed2c0a3400 0x7fed2c0a58c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fed3c006010 tx=0x7fed3c03f040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed29ffb640 1 -- 172.21.15.5:0/682610519 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed380043d0 con 0x564a1b5bd780 2024-09-19T06:10:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.923+0000 7fed29ffb640 1 -- 172.21.15.5:0/682610519 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38004d30 con 0x564a1b5bd780 2024-09-19T06:10:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.924+0000 7fed4c321740 1 -- 172.21.15.5:0/682610519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 msgr2=0x564a1b5bfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.924+0000 7fed4c321740 1 --2- 172.21.15.5:0/682610519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5bfb70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fed38009c10 tx=0x7fed38011810 comp rx=0 tx=0).stop 2024-09-19T06:10:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.924+0000 7fed4c321740 1 -- 172.21.15.5:0/682610519 shutdown_connections 2024-09-19T06:10:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.924+0000 7fed4c321740 1 --2- 172.21.15.5:0/682610519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5bfb70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.924+0000 7fed4c321740 1 --2- 172.21.15.5:0/682610519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5ae5d0 0x564a1b5aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.924+0000 7fed4c321740 1 --2- 172.21.15.5:0/682610519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5afb40 0x564a1b5ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.924+0000 7fed4c321740 1 -- 172.21.15.5:0/682610519 >> 172.21.15.5:0/682610519 conn(0x564a1b577310 msgr2=0x564a1b5aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.924+0000 7fed4c321740 1 -- 172.21.15.5:0/682610519 shutdown_connections 2024-09-19T06:10:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.924+0000 7fed4c321740 1 -- 172.21.15.5:0/682610519 wait complete. 2024-09-19T06:10:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.924+0000 7fed4c321740 1 -- start start 2024-09-19T06:10:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed4c321740 1 Processor -- start 2024-09-19T06:10:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed4c321740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 0x564a1b5bd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed4c321740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5afb40 0x564a1b5ba380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed4c321740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5bd780 0x564a1b5ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed4c321740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a1b5c0480 con 0x564a1b5ae5d0 2024-09-19T06:10:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed4c321740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564a1b4cf060 con 0x564a1b5afb40 2024-09-19T06:10:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed4c321740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564a1b5c0600 con 0x564a1b5bd780 2024-09-19T06:10:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed4c321740 10 client.0 _ll_register_callbacks cb 0x564a1b2228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5afb40 0x564a1b5ba380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5afb40 0x564a1b5ba380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44576/0 (socket says 172.21.15.5:44576) 2024-09-19T06:10:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed42ffd640 1 -- 172.21.15.5:0/735010537 learned_addr learned my addr 172.21.15.5:0/735010537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed437fe640 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 0x564a1b5bd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.925+0000 7fed43fff640 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5bd780 0x564a1b5ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed437fe640 1 -- 172.21.15.5:0/735010537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5bd780 msgr2=0x564a1b5ba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed437fe640 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5bd780 0x564a1b5ba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed437fe640 1 -- 172.21.15.5:0/735010537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5afb40 msgr2=0x564a1b5ba380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed437fe640 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5afb40 0x564a1b5ba380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed437fe640 1 -- 172.21.15.5:0/735010537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a1b2853c0 con 0x564a1b5ae5d0 2024-09-19T06:10:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed42ffd640 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5afb40 0x564a1b5ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed437fe640 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 0x564a1b5bd0d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fed340cd7f0 tx=0x7fed3400a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f22e7d59cf16c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:10:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed22ffd640 1 -- 172.21.15.5:0/735010537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3409c7e0 con 0x564a1b5ae5d0 2024-09-19T06:10:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed22ffd640 1 -- 172.21.15.5:0/735010537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fed340a4b60 con 0x564a1b5ae5d0 2024-09-19T06:10:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed4c321740 1 -- 172.21.15.5:0/735010537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed380043d0 con 0x564a1b5ae5d0 2024-09-19T06:10:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed22ffd640 1 -- 172.21.15.5:0/735010537 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fed340046c0 con 0x564a1b5ae5d0 2024-09-19T06:10:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed4c321740 10 client.17220 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.926+0000 7fed4c321740 1 -- 172.21.15.5:0/735010537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a1b5cbe00 con 0x564a1b5ae5d0 2024-09-19T06:10:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.927+0000 7fed22ffd640 1 -- 172.21.15.5:0/735010537 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed380043d0 con 0x564a1b5ae5d0 2024-09-19T06:10:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.927+0000 7fed22ffd640 1 -- 172.21.15.5:0/735010537 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fed3401bba0 con 0x564a1b5ae5d0 2024-09-19T06:10:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.927+0000 7fed4c321740 10 client.17220 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.927+0000 7fed4c321740 10 client.17220 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:10:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.927+0000 7fed4c321740 20 client.17220 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.927+0000 7fed4c321740 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x564a1b5cd2c0 0x564a1b5cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.927+0000 7fed4c321740 4 client.17220 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.927+0000 7fed4c321740 1 -- 172.21.15.5:0/735010537 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a1b521860 con 0x564a1b5cd2c0 2024-09-19T06:10:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.927+0000 7fed42ffd640 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x564a1b5cd2c0 0x564a1b5cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.928+0000 7fed22ffd640 10 client.17220 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:10:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.928+0000 7fed42ffd640 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x564a1b5cd2c0 0x564a1b5cf730 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3e90a6499ae8b2de server_cookie=c04ee36a75305063 in_seq=0 out_seq=0 2024-09-19T06:10:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed22ffd640 1 -- 172.21.15.5:0/735010537 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a1b521860 con 0x564a1b5cd2c0 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed22ffd640 10 client.17220 handle_command_reply: tid=0 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed4c321740 1 client.17220 shutdown 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed4c321740 1 -- 172.21.15.5:0/735010537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 msgr2=0x564a1b5bd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed4c321740 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 0x564a1b5bd0d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fed340cd7f0 tx=0x7fed3400a440 comp rx=0 tx=0).stop 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed4c321740 1 -- 172.21.15.5:0/735010537 shutdown_connections 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed4c321740 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x564a1b5cd2c0 0x564a1b5cf730 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed4c321740 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5bd780 0x564a1b5ba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed4c321740 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5afb40 0x564a1b5ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed4c321740 1 --2- 172.21.15.5:0/735010537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 0x564a1b5bd0d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed22ffd640 0 client.17220 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed4c321740 1 -- 172.21.15.5:0/735010537 >> 172.21.15.5:0/735010537 conn(0x564a1b577310 msgr2=0x564a1b5b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed4c321740 1 -- 172.21.15.5:0/735010537 shutdown_connections 2024-09-19T06:10:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed4c321740 1 -- 172.21.15.5:0/735010537 wait complete. 2024-09-19T06:10:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.929+0000 7fed4c321740 20 client.17220 trim_cache size 0 max 16384 2024-09-19T06:10:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.945+0000 7fed4c321740 1 Processor -- start 2024-09-19T06:10:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.945+0000 7fed4c321740 1 -- start start 2024-09-19T06:10:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.945+0000 7fed4c321740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 0x564a1b5aae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.945+0000 7fed4c321740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5afb40 0x564a1b5c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.945+0000 7fed4c321740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.945+0000 7fed4c321740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ab550 con 0x564a1b5ae5d0 2024-09-19T06:10:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.945+0000 7fed4c321740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ad090 con 0x564a1b5bd780 2024-09-19T06:10:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.945+0000 7fed4c321740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed441abc20 con 0x564a1b5afb40 2024-09-19T06:10:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 0x564a1b5aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 0x564a1b5aae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56824/0 (socket says 172.21.15.5:56824) 2024-09-19T06:10:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed437fe640 1 -- 172.21.15.5:0/2988320721 learned_addr learned my addr 172.21.15.5:0/2988320721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed43fff640 1 --2- 172.21.15.5:0/2988320721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed42ffd640 1 --2- 172.21.15.5:0/2988320721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5afb40 0x564a1b5c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed437fe640 1 -- 172.21.15.5:0/2988320721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5afb40 msgr2=0x564a1b5c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed437fe640 1 --2- 172.21.15.5:0/2988320721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5afb40 0x564a1b5c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed437fe640 1 -- 172.21.15.5:0/2988320721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 msgr2=0x564a1b5b9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed437fe640 1 --2- 172.21.15.5:0/2988320721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5b9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed437fe640 1 -- 172.21.15.5:0/2988320721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1b5b9330 con 0x564a1b5ae5d0 2024-09-19T06:10:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed43fff640 1 --2- 172.21.15.5:0/2988320721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5b9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed437fe640 1 --2- 172.21.15.5:0/2988320721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 0x564a1b5aae10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fed340cddf0 tx=0x7fed3409a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2701932315b12a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed297fa640 1 -- 172.21.15.5:0/2988320721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340b4070 con 0x564a1b5ae5d0 2024-09-19T06:10:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed297fa640 1 -- 172.21.15.5:0/2988320721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed34005310 con 0x564a1b5ae5d0 2024-09-19T06:10:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.946+0000 7fed297fa640 1 -- 172.21.15.5:0/2988320721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340a0730 con 0x564a1b5ae5d0 2024-09-19T06:10:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.947+0000 7fed4c321740 1 -- 172.21.15.5:0/2988320721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 msgr2=0x564a1b5aae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.947+0000 7fed4c321740 1 --2- 172.21.15.5:0/2988320721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 0x564a1b5aae10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fed340cddf0 tx=0x7fed3409a890 comp rx=0 tx=0).stop 2024-09-19T06:10:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.947+0000 7fed4c321740 1 -- 172.21.15.5:0/2988320721 shutdown_connections 2024-09-19T06:10:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.947+0000 7fed4c321740 1 --2- 172.21.15.5:0/2988320721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5b9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.947+0000 7fed4c321740 1 --2- 172.21.15.5:0/2988320721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5afb40 0x564a1b5c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.947+0000 7fed4c321740 1 --2- 172.21.15.5:0/2988320721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5ae5d0 0x564a1b5aae10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.947+0000 7fed4c321740 1 -- 172.21.15.5:0/2988320721 >> 172.21.15.5:0/2988320721 conn(0x564a1b577310 msgr2=0x564a1b5aa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.947+0000 7fed4c321740 1 -- 172.21.15.5:0/2988320721 shutdown_connections 2024-09-19T06:10:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.947+0000 7fed4c321740 1 -- 172.21.15.5:0/2988320721 wait complete. 2024-09-19T06:10:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed4c321740 1 -- start start 2024-09-19T06:10:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed4c321740 1 Processor -- start 2024-09-19T06:10:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed4c321740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5ae5d0 0x564a1b5c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed4c321740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5afb40 0x564a1b5c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed4c321740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed4c321740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ac9c0 con 0x564a1b5afb40 2024-09-19T06:10:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed4c321740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ac2f0 con 0x564a1b5bd780 2024-09-19T06:10:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed4c321740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed441a9550 con 0x564a1b5ae5d0 2024-09-19T06:10:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed4c321740 10 client.0 _ll_register_callbacks cb 0x7fed3400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5ae5d0 0x564a1b5c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5ae5d0 0x564a1b5c5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38508/0 (socket says 172.21.15.5:38508) 2024-09-19T06:10:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed437fe640 1 -- 172.21.15.5:0/1507723265 learned_addr learned my addr 172.21.15.5:0/1507723265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.948+0000 7fed43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.949+0000 7fed42ffd640 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5afb40 0x564a1b5c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.949+0000 7fed43fff640 1 -- 172.21.15.5:0/1507723265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5ae5d0 msgr2=0x564a1b5c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.949+0000 7fed43fff640 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5ae5d0 0x564a1b5c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.949+0000 7fed43fff640 1 -- 172.21.15.5:0/1507723265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5afb40 msgr2=0x564a1b5c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.949+0000 7fed43fff640 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5afb40 0x564a1b5c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.949+0000 7fed43fff640 1 -- 172.21.15.5:0/1507723265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a1b5c17c0 con 0x564a1b5bd780 2024-09-19T06:10:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.949+0000 7fed437fe640 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5ae5d0 0x564a1b5c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.949+0000 7fed43fff640 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5c0fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fed3c031040 tx=0x7fed3c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc70cdd2b1f8330 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.949+0000 7fed22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:10:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.949+0000 7fed22ffd640 1 -- 172.21.15.5:0/1507723265 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c004250 con 0x564a1b5bd780 2024-09-19T06:10:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.949+0000 7fed4c321740 1 -- 172.21.15.5:0/1507723265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564a1b51bd70 con 0x564a1b5bd780 2024-09-19T06:10:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.949+0000 7fed22ffd640 1 -- 172.21.15.5:0/1507723265 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fed3c043070 con 0x564a1b5bd780 2024-09-19T06:10:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.950+0000 7fed22ffd640 1 -- 172.21.15.5:0/1507723265 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fed3c0043f0 con 0x564a1b5bd780 2024-09-19T06:10:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.950+0000 7fed4c321740 10 client.26801 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.950+0000 7fed4c321740 1 -- 172.21.15.5:0/1507723265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a1b637b40 con 0x564a1b5bd780 2024-09-19T06:10:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.950+0000 7fed22ffd640 1 -- 172.21.15.5:0/1507723265 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564a1b51bd70 con 0x564a1b5bd780 2024-09-19T06:10:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.950+0000 7fed22ffd640 1 -- 172.21.15.5:0/1507723265 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fed3c03b430 con 0x564a1b5bd780 2024-09-19T06:10:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.950+0000 7fed4c321740 10 client.26801 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.950+0000 7fed4c321740 10 client.26801 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:10:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.950+0000 7fed4c321740 20 client.26801 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.950+0000 7fed4c321740 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x564a1b638260 0x564a1b63a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.950+0000 7fed4c321740 4 client.26801 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:10:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.950+0000 7fed4c321740 1 -- 172.21.15.5:0/1507723265 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a1b498170 con 0x564a1b638260 2024-09-19T06:10:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.950+0000 7fed437fe640 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x564a1b638260 0x564a1b63a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed437fe640 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x564a1b638260 0x564a1b63a680 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52b07fb3b85f6173 server_cookie=49557c21f13c99a6 in_seq=0 out_seq=0 2024-09-19T06:10:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed22ffd640 10 client.26801 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:10:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed22ffd640 1 -- 172.21.15.5:0/1507723265 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a1b498170 con 0x564a1b638260 2024-09-19T06:10:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed22ffd640 10 client.26801 handle_command_reply: tid=0 2024-09-19T06:10:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed4c321740 1 client.26801 shutdown 2024-09-19T06:10:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed4c321740 1 -- 172.21.15.5:0/1507723265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 msgr2=0x564a1b5c0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed4c321740 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5c0fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fed3c031040 tx=0x7fed3c004a40 comp rx=0 tx=0).stop 2024-09-19T06:10:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed4c321740 1 -- 172.21.15.5:0/1507723265 shutdown_connections 2024-09-19T06:10:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed4c321740 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x564a1b638260 0x564a1b63a680 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed4c321740 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a1b5bd780 0x564a1b5c0fc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed4c321740 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a1b5afb40 0x564a1b5c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed4c321740 1 --2- 172.21.15.5:0/1507723265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a1b5ae5d0 0x564a1b5c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed22ffd640 0 client.26801 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:10:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.951+0000 7fed4c321740 1 -- 172.21.15.5:0/1507723265 >> 172.21.15.5:0/1507723265 conn(0x564a1b577310 msgr2=0x564a1b5a94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed4c321740 1 -- 172.21.15.5:0/1507723265 shutdown_connections 2024-09-19T06:10:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed4c321740 1 -- 172.21.15.5:0/1507723265 wait complete. 2024-09-19T06:10:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed4c321740 20 client.26801 trim_cache size 0 max 16384 2024-09-19T06:10:28.961 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:10:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed49f48640 1 -- 172.21.15.5:0/3798943450 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fed2c0a3400 msgr2=0x7fed2c0a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed49f48640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fed2c0a3400 0x7fed2c0a58c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fed3c006010 tx=0x7fed3c03f040 comp rx=0 tx=0).stop 2024-09-19T06:10:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed49f48640 1 -- 172.21.15.5:0/3798943450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed44101de0 msgr2=0x7fed44198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed49f48640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed44101de0 0x7fed44198320 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fed3400e9f0 tx=0x7fed3400edf0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed49f48640 1 -- 172.21.15.5:0/3798943450 shutdown_connections 2024-09-19T06:10:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed49f48640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fed2c0a3400 0x7fed2c0a58c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed49f48640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed44106f10 0x7fed4419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed49f48640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed44102790 0x7fed44198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed49f48640 1 --2- 172.21.15.5:0/3798943450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed44101de0 0x7fed44198320 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed49f48640 1 -- 172.21.15.5:0/3798943450 >> 172.21.15.5:0/3798943450 conn(0x7fed440fd8c0 msgr2=0x7fed440fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed49f48640 1 -- 172.21.15.5:0/3798943450 shutdown_connections 2024-09-19T06:10:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:28.952+0000 7fed49f48640 1 -- 172.21.15.5:0/3798943450 wait complete. 2024-09-19T06:10:29.701 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:10:29.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181206 2.73 MB/sec execute 235 sec latency 14.796 ms 2024-09-19T06:10:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:10:30.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:30 smithi005 ceph-mon[30955]: pgmap v220: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-19T06:10:30.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.520+0000 7f5859433640 1 -- 172.21.15.5:0/1362806696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5854101de0 msgr2=0x7f58541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:30.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.520+0000 7f5859433640 1 --2- 172.21.15.5:0/1362806696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5854101de0 0x7f58541021c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5844009900 tx=0x7f584402a410 comp rx=0 tx=0).stop 2024-09-19T06:10:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.520+0000 7f5859433640 1 -- 172.21.15.5:0/1362806696 shutdown_connections 2024-09-19T06:10:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.520+0000 7f5859433640 1 --2- 172.21.15.5:0/1362806696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854106f10 0x7f585410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.520+0000 7f5859433640 1 --2- 172.21.15.5:0/1362806696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5854102790 0x7f58541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.520+0000 7f5859433640 1 --2- 172.21.15.5:0/1362806696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5854101de0 0x7f58541021c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.520+0000 7f5859433640 1 -- 172.21.15.5:0/1362806696 >> 172.21.15.5:0/1362806696 conn(0x7f58540fd8e0 msgr2=0x7f58540ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.520+0000 7f5859433640 1 -- 172.21.15.5:0/1362806696 shutdown_connections 2024-09-19T06:10:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.520+0000 7f5859433640 1 -- 172.21.15.5:0/1362806696 wait complete. 2024-09-19T06:10:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.520+0000 7f5859433640 1 Processor -- start 2024-09-19T06:10:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f5859433640 1 -- start start 2024-09-19T06:10:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f5859433640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5854101de0 0x7f58541983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f5859433640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854102790 0x7f5854198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f5859433640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5854106f10 0x7f585419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f5859433640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5854111e20 con 0x7f5854106f10 2024-09-19T06:10:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f5859433640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5854111ca0 con 0x7f5854102790 2024-09-19T06:10:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f5859433640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5854111fa0 con 0x7f5854101de0 2024-09-19T06:10:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5854101de0 0x7f58541983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f58537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5854106f10 0x7f585419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f58527fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854102790 0x7f5854198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5854101de0 0x7f58541983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38528/0 (socket says 172.21.15.5:38528) 2024-09-19T06:10:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f58537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5854106f10 0x7f585419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56838/0 (socket says 172.21.15.5:56838) 2024-09-19T06:10:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.521+0000 7f5852ffd640 1 -- 172.21.15.5:0/2635077451 learned_addr learned my addr 172.21.15.5:0/2635077451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.522+0000 7f58537fe640 1 -- 172.21.15.5:0/2635077451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5854101de0 msgr2=0x7f58541983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.522+0000 7f58537fe640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5854101de0 0x7f58541983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.522+0000 7f58537fe640 1 -- 172.21.15.5:0/2635077451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854102790 msgr2=0x7f5854198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.522+0000 7f58537fe640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854102790 0x7f5854198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.522+0000 7f58537fe640 1 -- 172.21.15.5:0/2635077451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585419d370 con 0x7f5854106f10 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.522+0000 7f58527fc640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854102790 0x7f5854198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.522+0000 7f58537fe640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5854106f10 0x7f585419cc90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f584c009fd0 tx=0x7f584c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.522+0000 7f583bfff640 1 -- 172.21.15.5:0/2635077451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c00ce20 con 0x7f5854106f10 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.522+0000 7f5859433640 1 -- 172.21.15.5:0/2635077451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585419d660 con 0x7f5854106f10 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.522+0000 7f583bfff640 1 -- 172.21.15.5:0/2635077451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f584c005020 con 0x7f5854106f10 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.522+0000 7f583bfff640 1 -- 172.21.15.5:0/2635077451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c010690 con 0x7f5854106f10 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.522+0000 7f5859433640 1 -- 172.21.15.5:0/2635077451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5854074420 con 0x7f5854106f10 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f583bfff640 1 -- 172.21.15.5:0/2635077451 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f584c0051d0 con 0x7f5854106f10 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f585b80c740 1 Processor -- start 2024-09-19T06:10:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f585b80c740 1 -- start start 2024-09-19T06:10:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f585b80c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea8b40 0x5570b9ea7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f585b80c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9ea7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f585b80c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9eb2370 0x5570b9eb4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f585b80c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5570b9e0d2f0 con 0x5570b9ea8b40 2024-09-19T06:10:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f585b80c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5570b9db4cf0 con 0x5570b9ea75d0 2024-09-19T06:10:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f585b80c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5570b9e092b0 con 0x5570b9eb2370 2024-09-19T06:10:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f58527fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea8b40 0x5570b9ea7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f58527fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea8b40 0x5570b9ea7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56846/0 (socket says 172.21.15.5:56846) 2024-09-19T06:10:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f58527fc640 1 -- 172.21.15.5:0/2730220918 learned_addr learned my addr 172.21.15.5:0/2730220918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f5852ffd640 1 --2- 172.21.15.5:0/2730220918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9ea7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f583bfff640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f583c0a33b0 0x7f583c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.525+0000 7f58527fc640 1 --2- 172.21.15.5:0/2730220918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9eb2370 0x5570b9eb4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f58537fe640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f583c0a33b0 0x7f583c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f5852ffd640 1 -- 172.21.15.5:0/2730220918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9eb2370 msgr2=0x5570b9eb4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f5852ffd640 1 --2- 172.21.15.5:0/2730220918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9eb2370 0x5570b9eb4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f5852ffd640 1 -- 172.21.15.5:0/2730220918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea8b40 msgr2=0x5570b9ea7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f5852ffd640 1 --2- 172.21.15.5:0/2730220918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea8b40 0x5570b9ea7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f5852ffd640 1 -- 172.21.15.5:0/2730220918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570b9d993a0 con 0x5570b9ea75d0 2024-09-19T06:10:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f583bfff640 1 -- 172.21.15.5:0/2635077451 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7f584c0d0380 con 0x7f5854106f10 2024-09-19T06:10:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f58527fc640 1 --2- 172.21.15.5:0/2730220918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea8b40 0x5570b9ea7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f5852ffd640 1 --2- 172.21.15.5:0/2730220918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9ea7a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5844004770 tx=0x7f5844031c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce7e83ba6dfde7b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f58537fe640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f583c0a33b0 0x7f583c0a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f584c0cdb60 tx=0x7f584c005f70 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f58397fa640 1 -- 172.21.15.5:0/2730220918 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844040070 con 0x5570b9ea75d0 2024-09-19T06:10:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f58397fa640 1 -- 172.21.15.5:0/2730220918 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58440316a0 con 0x5570b9ea75d0 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.526+0000 7f58397fa640 1 -- 172.21.15.5:0/2730220918 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844035040 con 0x5570b9ea75d0 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.527+0000 7f585b80c740 1 -- 172.21.15.5:0/2730220918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 msgr2=0x5570b9ea7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.527+0000 7f585b80c740 1 --2- 172.21.15.5:0/2730220918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9ea7a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5844004770 tx=0x7f5844031c20 comp rx=0 tx=0).stop 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.527+0000 7f585b80c740 1 -- 172.21.15.5:0/2730220918 shutdown_connections 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.527+0000 7f585b80c740 1 --2- 172.21.15.5:0/2730220918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9eb2370 0x5570b9eb4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.527+0000 7f585b80c740 1 --2- 172.21.15.5:0/2730220918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9ea7a10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.527+0000 7f585b80c740 1 --2- 172.21.15.5:0/2730220918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea8b40 0x5570b9ea7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.527+0000 7f585b80c740 1 -- 172.21.15.5:0/2730220918 >> 172.21.15.5:0/2730220918 conn(0x5570b9e70310 msgr2=0x5570b9ea3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.527+0000 7f585b80c740 1 -- 172.21.15.5:0/2730220918 shutdown_connections 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.527+0000 7f585b80c740 1 -- 172.21.15.5:0/2730220918 wait complete. 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.527+0000 7f585b80c740 1 -- start start 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.527+0000 7f585b80c740 1 Processor -- start 2024-09-19T06:10:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f585b80c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9eb1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f585b80c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9eaefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f585b80c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9eb2370 0x5570b9eaf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f585b80c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5570b9eb5010 con 0x5570b9eb2370 2024-09-19T06:10:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f585b80c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5570b9dc8060 con 0x5570b9ea75d0 2024-09-19T06:10:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f585b80c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5570b9eb5190 con 0x5570b9ea8b40 2024-09-19T06:10:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f585b80c740 10 client.0 _ll_register_callbacks cb 0x5570b9b1b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f58527fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9eb2370 0x5570b9eaf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9eaefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f58527fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9eb2370 0x5570b9eaf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56850/0 (socket says 172.21.15.5:56850) 2024-09-19T06:10:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f58527fc640 1 -- 172.21.15.5:0/160910017 learned_addr learned my addr 172.21.15.5:0/160910017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f58527fc640 1 --2- 172.21.15.5:0/160910017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9eb1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f5852ffd640 1 -- 172.21.15.5:0/160910017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 msgr2=0x5570b9eb1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f5852ffd640 1 --2- 172.21.15.5:0/160910017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9eb1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f5852ffd640 1 -- 172.21.15.5:0/160910017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9eb2370 msgr2=0x5570b9eaf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f5852ffd640 1 --2- 172.21.15.5:0/160910017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9eb2370 0x5570b9eaf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.528+0000 7f5852ffd640 1 -- 172.21.15.5:0/160910017 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570b9b7e3c0 con 0x5570b9ea8b40 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f5852ffd640 1 --2- 172.21.15.5:0/160910017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9eaefe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5844006010 tx=0x7f58440092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44bbbc4befa09f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f5836ff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f5836ff5640 1 -- 172.21.15.5:0/160910017 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584404b070 con 0x5570b9ea8b40 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f585b80c740 1 -- 172.21.15.5:0/160910017 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58440316a0 con 0x5570b9ea8b40 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f5836ff5640 1 -- 172.21.15.5:0/160910017 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7f5844005440 con 0x5570b9ea8b40 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f5836ff5640 1 -- 172.21.15.5:0/160910017 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5844057050 con 0x5570b9ea8b40 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f585b80c740 10 client.26731 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f585b80c740 1 -- 172.21.15.5:0/160910017 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5570b9eb5010 con 0x5570b9ea8b40 2024-09-19T06:10:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f5836ff5640 1 -- 172.21.15.5:0/160910017 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58440316a0 con 0x5570b9ea8b40 2024-09-19T06:10:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f5836ff5640 1 -- 172.21.15.5:0/160910017 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5844004ba0 con 0x5570b9ea8b40 2024-09-19T06:10:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f585b80c740 10 client.26731 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f585b80c740 10 client.26731 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:10:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f585b80c740 20 client.26731 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f585b80c740 1 --2- 172.21.15.5:0/160910017 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5570b9ec1bf0 0x5570b9ec3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f585b80c740 4 client.26731 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.529+0000 7f585b80c740 1 -- 172.21.15.5:0/160910017 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570b9e1a860 con 0x5570b9ec1bf0 2024-09-19T06:10:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.530+0000 7f58527fc640 1 --2- 172.21.15.5:0/160910017 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5570b9ec1bf0 0x5570b9ec3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.530+0000 7f58527fc640 1 --2- 172.21.15.5:0/160910017 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5570b9ec1bf0 0x5570b9ec3fe0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1ff3988ee3a2e50 server_cookie=157eedca2a5fb3bd in_seq=0 out_seq=0 2024-09-19T06:10:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.530+0000 7f5836ff5640 10 client.26731 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:10:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f5836ff5640 1 -- 172.21.15.5:0/160910017 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570b9e1a860 con 0x5570b9ec1bf0 2024-09-19T06:10:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f5836ff5640 10 client.26731 handle_command_reply: tid=0 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f585b80c740 1 client.26731 shutdown 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f585b80c740 1 -- 172.21.15.5:0/160910017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 msgr2=0x5570b9eaefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f585b80c740 1 --2- 172.21.15.5:0/160910017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9eaefe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5844006010 tx=0x7f58440092d0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f585b80c740 1 -- 172.21.15.5:0/160910017 shutdown_connections 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f585b80c740 1 --2- 172.21.15.5:0/160910017 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5570b9ec1bf0 0x5570b9ec3fe0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f585b80c740 1 --2- 172.21.15.5:0/160910017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9eb2370 0x5570b9eaf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f585b80c740 1 --2- 172.21.15.5:0/160910017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9eaefe0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f585b80c740 1 --2- 172.21.15.5:0/160910017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9eb1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f5836ff5640 0 client.26731 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f585b80c740 1 -- 172.21.15.5:0/160910017 >> 172.21.15.5:0/160910017 conn(0x5570b9e70310 msgr2=0x5570b9ea9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f585b80c740 1 -- 172.21.15.5:0/160910017 shutdown_connections 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f585b80c740 1 -- 172.21.15.5:0/160910017 wait complete. 2024-09-19T06:10:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.531+0000 7f585b80c740 20 client.26731 trim_cache size 0 max 16384 2024-09-19T06:10:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.544+0000 7f585b80c740 1 Processor -- start 2024-09-19T06:10:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f585b80c740 1 -- start start 2024-09-19T06:10:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f585b80c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9ebe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f585b80c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9e9ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f585b80c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9eb2370 0x5570b9ea2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f585b80c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58541af940 con 0x5570b9eb2370 2024-09-19T06:10:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f585b80c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58541b1480 con 0x5570b9ea75d0 2024-09-19T06:10:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f585b80c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58541b0010 con 0x5570b9ea8b40 2024-09-19T06:10:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9e9ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f58527fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9eb2370 0x5570b9ea2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9e9ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38574/0 (socket says 172.21.15.5:38574) 2024-09-19T06:10:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f5852ffd640 1 -- 172.21.15.5:0/3042606569 learned_addr learned my addr 172.21.15.5:0/3042606569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f58527fc640 1 --2- 172.21.15.5:0/3042606569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9ebe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f5852ffd640 1 -- 172.21.15.5:0/3042606569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 msgr2=0x5570b9ebe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f5852ffd640 1 --2- 172.21.15.5:0/3042606569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9ebe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f5852ffd640 1 -- 172.21.15.5:0/3042606569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9eb2370 msgr2=0x5570b9ea2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f5852ffd640 1 --2- 172.21.15.5:0/3042606569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9eb2370 0x5570b9ea2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f5852ffd640 1 -- 172.21.15.5:0/3042606569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570b9ea3de0 con 0x5570b9ea8b40 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.545+0000 7f58527fc640 1 --2- 172.21.15.5:0/3042606569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9eb2370 0x5570b9ea2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.546+0000 7f5852ffd640 1 --2- 172.21.15.5:0/3042606569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9e9ead0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5844009a20 tx=0x7f5844031ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11e64868839ebbb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.546+0000 7f5838ff9640 1 -- 172.21.15.5:0/3042606569 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584404b040 con 0x5570b9ea8b40 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.546+0000 7f5838ff9640 1 -- 172.21.15.5:0/3042606569 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5844004c20 con 0x5570b9ea8b40 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.546+0000 7f5838ff9640 1 -- 172.21.15.5:0/3042606569 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844035030 con 0x5570b9ea8b40 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 -- 172.21.15.5:0/3042606569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 msgr2=0x5570b9e9ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 --2- 172.21.15.5:0/3042606569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9e9ead0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5844009a20 tx=0x7f5844031ca0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 -- 172.21.15.5:0/3042606569 shutdown_connections 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 --2- 172.21.15.5:0/3042606569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9eb2370 0x5570b9ea2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 --2- 172.21.15.5:0/3042606569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9e9ead0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 --2- 172.21.15.5:0/3042606569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9ea75d0 0x5570b9ebe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 -- 172.21.15.5:0/3042606569 >> 172.21.15.5:0/3042606569 conn(0x5570b9e70310 msgr2=0x5570b9ea2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 -- 172.21.15.5:0/3042606569 shutdown_connections 2024-09-19T06:10:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 -- 172.21.15.5:0/3042606569 wait complete. 2024-09-19T06:10:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 -- start start 2024-09-19T06:10:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 Processor -- start 2024-09-19T06:10:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea75d0 0x5570b9eb5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9eb60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9eb2370 0x5570b9f24750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58541b0db0 con 0x5570b9ea75d0 2024-09-19T06:10:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58541b06e0 con 0x5570b9eb2370 2024-09-19T06:10:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58541ad940 con 0x5570b9ea8b40 2024-09-19T06:10:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f585b80c740 10 client.0 _ll_register_callbacks cb 0x7f584800bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.547+0000 7f58527fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea75d0 0x5570b9eb5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f58527fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea75d0 0x5570b9eb5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56872/0 (socket says 172.21.15.5:56872) 2024-09-19T06:10:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f58527fc640 1 -- 172.21.15.5:0/1176584322 learned_addr learned my addr 172.21.15.5:0/1176584322 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f5852ffd640 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9eb60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f58527fc640 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9eb2370 0x5570b9f24750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f58527fc640 1 -- 172.21.15.5:0/1176584322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 msgr2=0x5570b9eb60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f58527fc640 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9eb60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f58527fc640 1 -- 172.21.15.5:0/1176584322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9eb2370 msgr2=0x5570b9f24750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f58527fc640 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9eb2370 0x5570b9f24750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f58527fc640 1 -- 172.21.15.5:0/1176584322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570b9f24d40 con 0x5570b9ea75d0 2024-09-19T06:10:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f58527fc640 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea75d0 0x5570b9eb5b60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5848005bc0 tx=0x7f5848004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486ddeec28d6851c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f5836ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f5836ff5640 1 -- 172.21.15.5:0/1176584322 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848000e60 con 0x5570b9ea75d0 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f5836ff5640 1 -- 172.21.15.5:0/1176584322 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7f5848004020 con 0x5570b9ea75d0 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f585b80c740 1 -- 172.21.15.5:0/1176584322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5844004c20 con 0x5570b9ea75d0 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f5836ff5640 1 -- 172.21.15.5:0/1176584322 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5848039050 con 0x5570b9ea75d0 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f585b80c740 10 client.17244 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.548+0000 7f585b80c740 1 -- 172.21.15.5:0/1176584322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5570b9f2c120 con 0x5570b9ea75d0 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.549+0000 7f5836ff5640 1 -- 172.21.15.5:0/1176584322 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5844004c20 con 0x5570b9ea75d0 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.549+0000 7f5836ff5640 1 -- 172.21.15.5:0/1176584322 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5848003b30 con 0x5570b9ea75d0 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.549+0000 7f585b80c740 10 client.17244 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.549+0000 7f585b80c740 10 client.17244 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.549+0000 7f585b80c740 20 client.17244 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.549+0000 7f585b80c740 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5570b9f2c840 0x5570b9f2ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.549+0000 7f585b80c740 4 client.17244 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:10:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.549+0000 7f585b80c740 1 -- 172.21.15.5:0/1176584322 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570b9e14d70 con 0x5570b9f2c840 2024-09-19T06:10:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.549+0000 7f5852ffd640 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5570b9f2c840 0x5570b9f2ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f5836ff5640 10 client.17244 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:10:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f5852ffd640 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5570b9f2c840 0x5570b9f2ec30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e64728befa783c4 server_cookie=a1bfa5cd5340875c in_seq=0 out_seq=0 2024-09-19T06:10:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f5836ff5640 1 -- 172.21.15.5:0/1176584322 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570b9e14d70 con 0x5570b9f2c840 2024-09-19T06:10:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f5836ff5640 10 client.17244 handle_command_reply: tid=0 2024-09-19T06:10:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f585b80c740 1 client.17244 shutdown 2024-09-19T06:10:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f585b80c740 1 -- 172.21.15.5:0/1176584322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea75d0 msgr2=0x5570b9eb5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f585b80c740 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea75d0 0x5570b9eb5b60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5848005bc0 tx=0x7f5848004730 comp rx=0 tx=0).stop 2024-09-19T06:10:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f585b80c740 1 -- 172.21.15.5:0/1176584322 shutdown_connections 2024-09-19T06:10:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f585b80c740 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5570b9f2c840 0x5570b9f2ec30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f585b80c740 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5570b9eb2370 0x5570b9f24750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f585b80c740 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570b9ea8b40 0x5570b9eb60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f585b80c740 1 --2- 172.21.15.5:0/1176584322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570b9ea75d0 0x5570b9eb5b60 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f5836ff5640 0 client.17244 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:10:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f585b80c740 1 -- 172.21.15.5:0/1176584322 >> 172.21.15.5:0/1176584322 conn(0x5570b9e70310 msgr2=0x5570b9ea9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f585b80c740 1 -- 172.21.15.5:0/1176584322 shutdown_connections 2024-09-19T06:10:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f585b80c740 1 -- 172.21.15.5:0/1176584322 wait complete. 2024-09-19T06:10:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.550+0000 7f585b80c740 20 client.17244 trim_cache size 0 max 16384 2024-09-19T06:10:30.561 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:10:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.551+0000 7f5859433640 1 -- 172.21.15.5:0/2635077451 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f583c0a33b0 msgr2=0x7f583c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.551+0000 7f5859433640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f583c0a33b0 0x7f583c0a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f584c0cdb60 tx=0x7f584c005f70 comp rx=0 tx=0).stop 2024-09-19T06:10:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.551+0000 7f5859433640 1 -- 172.21.15.5:0/2635077451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5854106f10 msgr2=0x7f585419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.551+0000 7f5859433640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5854106f10 0x7f585419cc90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f584c009fd0 tx=0x7f584c00ef10 comp rx=0 tx=0).stop 2024-09-19T06:10:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.551+0000 7f5859433640 1 -- 172.21.15.5:0/2635077451 shutdown_connections 2024-09-19T06:10:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.551+0000 7f5859433640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5854106f10 0x7f585419cc90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.551+0000 7f5859433640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854102790 0x7f5854198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.551+0000 7f5859433640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f583c0a33b0 0x7f583c0a5870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.551+0000 7f5859433640 1 --2- 172.21.15.5:0/2635077451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5854101de0 0x7f58541983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.551+0000 7f5859433640 1 -- 172.21.15.5:0/2635077451 >> 172.21.15.5:0/2635077451 conn(0x7f58540fd8e0 msgr2=0x7f58540fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.551+0000 7f5859433640 1 -- 172.21.15.5:0/2635077451 shutdown_connections 2024-09-19T06:10:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:30.551+0000 7f5859433640 1 -- 172.21.15.5:0/2635077451 wait complete. 2024-09-19T06:10:30.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:30 smithi049 ceph-mon[34640]: pgmap v220: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-19T06:10:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:30 smithi157 ceph-mon[35346]: pgmap v220: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-19T06:10:30.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181475 2.72 MB/sec execute 236 sec latency 19.787 ms 2024-09-19T06:10:31.269 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:10:31.548 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:10:31.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181967 2.72 MB/sec execute 237 sec latency 18.816 ms 2024-09-19T06:10:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.087+0000 7fb754911640 1 -- 172.21.15.5:0/2018024612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb750103740 msgr2=0x7fb750107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.087+0000 7fb754911640 1 --2- 172.21.15.5:0/2018024612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb750103740 0x7fb750107b80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb738009930 tx=0x7fb73802a420 comp rx=0 tx=0).stop 2024-09-19T06:10:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.087+0000 7fb754911640 1 -- 172.21.15.5:0/2018024612 shutdown_connections 2024-09-19T06:10:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.087+0000 7fb754911640 1 --2- 172.21.15.5:0/2018024612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb750108290 0x7fb75010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.087+0000 7fb754911640 1 --2- 172.21.15.5:0/2018024612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb750103740 0x7fb750107b80 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.087+0000 7fb754911640 1 --2- 172.21.15.5:0/2018024612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb750102d90 0x7fb750103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.087+0000 7fb754911640 1 -- 172.21.15.5:0/2018024612 >> 172.21.15.5:0/2018024612 conn(0x7fb7500fc7d0 msgr2=0x7fb7500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.087+0000 7fb754911640 1 -- 172.21.15.5:0/2018024612 shutdown_connections 2024-09-19T06:10:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.087+0000 7fb754911640 1 -- 172.21.15.5:0/2018024612 wait complete. 2024-09-19T06:10:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb754911640 1 Processor -- start 2024-09-19T06:10:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb754911640 1 -- start start 2024-09-19T06:10:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb754911640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb750102d90 0x7fb750198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb754911640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb750103740 0x7fb750198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb754911640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb750108290 0x7fb75019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb754911640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb750111f30 con 0x7fb750103740 2024-09-19T06:10:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb754911640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb750111db0 con 0x7fb750102d90 2024-09-19T06:10:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb754911640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501120b0 con 0x7fb750108290 2024-09-19T06:10:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb747fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb750103740 0x7fb750198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb74ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb750108290 0x7fb75019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb747fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb750103740 0x7fb750198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56890/0 (socket says 172.21.15.5:56890) 2024-09-19T06:10:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb74ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb750108290 0x7fb75019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38606/0 (socket says 172.21.15.5:38606) 2024-09-19T06:10:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.088+0000 7fb747fff640 1 -- 172.21.15.5:0/1795529060 learned_addr learned my addr 172.21.15.5:0/1795529060 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.089+0000 7fb74e59c640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb750102d90 0x7fb750198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.089+0000 7fb74ed9d640 1 -- 172.21.15.5:0/1795529060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb750102d90 msgr2=0x7fb750198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.089+0000 7fb74ed9d640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb750102d90 0x7fb750198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.089+0000 7fb74ed9d640 1 -- 172.21.15.5:0/1795529060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb750103740 msgr2=0x7fb750198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.089+0000 7fb74ed9d640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb750103740 0x7fb750198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.089+0000 7fb74ed9d640 1 -- 172.21.15.5:0/1795529060 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75019d4c0 con 0x7fb750108290 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.089+0000 7fb747fff640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb750103740 0x7fb750198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.089+0000 7fb74ed9d640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb750108290 0x7fb75019cde0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb748007d10 tx=0x7fb748010420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.089+0000 7fb7477fe640 1 -- 172.21.15.5:0/1795529060 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74801b020 con 0x7fb750108290 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.089+0000 7fb7477fe640 1 -- 172.21.15.5:0/1795529060 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb748010d40 con 0x7fb750108290 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.090+0000 7fb7477fe640 1 -- 172.21.15.5:0/1795529060 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74800cab0 con 0x7fb750108290 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.090+0000 7fb754911640 1 -- 172.21.15.5:0/1795529060 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75019d7b0 con 0x7fb750108290 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.090+0000 7fb754911640 1 -- 172.21.15.5:0/1795529060 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb750100920 con 0x7fb750108290 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.093+0000 7fb7477fe640 1 -- 172.21.15.5:0/1795529060 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb74800cc50 con 0x7fb750108290 2024-09-19T06:10:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.093+0000 7fb756cea740 1 Processor -- start 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.093+0000 7fb756cea740 1 -- start start 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.093+0000 7fb756cea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bba4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.093+0000 7fb756cea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bba45d0 0x56388bba4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.093+0000 7fb756cea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bbaf370 0x56388bbb1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.093+0000 7fb756cea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56388bb0a2f0 con 0x56388bba5b40 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.093+0000 7fb756cea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56388bab1cf0 con 0x56388bba45d0 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.093+0000 7fb756cea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56388bb062b0 con 0x56388bbaf370 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb747fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bba4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb74e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bba45d0 0x56388bba4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb747fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bba4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56900/0 (socket says 172.21.15.5:56900) 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb747fff640 1 -- 172.21.15.5:0/3233501476 learned_addr learned my addr 172.21.15.5:0/3233501476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb747fff640 1 --2- 172.21.15.5:0/3233501476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bbaf370 0x56388bbb1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb74e59c640 1 -- 172.21.15.5:0/3233501476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bbaf370 msgr2=0x56388bbb1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb74e59c640 1 --2- 172.21.15.5:0/3233501476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bbaf370 0x56388bbb1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb74e59c640 1 -- 172.21.15.5:0/3233501476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 msgr2=0x56388bba4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb74e59c640 1 --2- 172.21.15.5:0/3233501476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bba4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb7477fe640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb73c0a33b0 0x7fb73c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb74e59c640 1 -- 172.21.15.5:0/3233501476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56388ba963a0 con 0x56388bba45d0 2024-09-19T06:10:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb747fff640 1 --2- 172.21.15.5:0/3233501476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bba4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:10:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb7477fe640 1 -- 172.21.15.5:0/1795529060 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fb7480d3690 con 0x7fb750108290 2024-09-19T06:10:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb747fff640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb73c0a33b0 0x7fb73c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.094+0000 7fb74e59c640 1 --2- 172.21.15.5:0/3233501476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bba45d0 0x56388bba4a10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb740009c20 tx=0x7fb740011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a2bb8e87042c934 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.095+0000 7fb744ff9640 1 -- 172.21.15.5:0/3233501476 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740004050 con 0x56388bba45d0 2024-09-19T06:10:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.095+0000 7fb744ff9640 1 -- 172.21.15.5:0/3233501476 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7400054d0 con 0x56388bba45d0 2024-09-19T06:10:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.095+0000 7fb744ff9640 1 -- 172.21.15.5:0/3233501476 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740004cf0 con 0x56388bba45d0 2024-09-19T06:10:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.096+0000 7fb756cea740 1 -- 172.21.15.5:0/3233501476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bba45d0 msgr2=0x56388bba4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.096+0000 7fb747fff640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb73c0a33b0 0x7fb73c0a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb7380096c0 tx=0x7fb738000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.096+0000 7fb756cea740 1 --2- 172.21.15.5:0/3233501476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bba45d0 0x56388bba4a10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb740009c20 tx=0x7fb740011810 comp rx=0 tx=0).stop 2024-09-19T06:10:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.096+0000 7fb756cea740 1 -- 172.21.15.5:0/3233501476 shutdown_connections 2024-09-19T06:10:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.096+0000 7fb756cea740 1 --2- 172.21.15.5:0/3233501476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bbaf370 0x56388bbb1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.096+0000 7fb756cea740 1 --2- 172.21.15.5:0/3233501476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bba45d0 0x56388bba4a10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.096+0000 7fb756cea740 1 --2- 172.21.15.5:0/3233501476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bba4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.096+0000 7fb756cea740 1 -- 172.21.15.5:0/3233501476 >> 172.21.15.5:0/3233501476 conn(0x56388bb6d310 msgr2=0x56388bba0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.096+0000 7fb756cea740 1 -- 172.21.15.5:0/3233501476 shutdown_connections 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.096+0000 7fb756cea740 1 -- 172.21.15.5:0/3233501476 wait complete. 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.096+0000 7fb756cea740 1 -- start start 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb756cea740 1 Processor -- start 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb756cea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbaed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb756cea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bba5b40 0x56388bbabfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb756cea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bbaf370 0x56388bbac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb756cea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56388bbb2070 con 0x56388bbaf370 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb756cea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56388bac5060 con 0x56388bba5b40 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb756cea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56388bbb21f0 con 0x56388bba45d0 2024-09-19T06:10:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb756cea740 10 client.0 _ll_register_callbacks cb 0x56388b8188c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb74e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbaed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb74e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbaed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38618/0 (socket says 172.21.15.5:38618) 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb74e59c640 1 -- 172.21.15.5:0/2699420599 learned_addr learned my addr 172.21.15.5:0/2699420599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb74ed9d640 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bbaf370 0x56388bbac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.097+0000 7fb747fff640 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bba5b40 0x56388bbabfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb74e59c640 1 -- 172.21.15.5:0/2699420599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bba5b40 msgr2=0x56388bbabfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb74e59c640 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bba5b40 0x56388bbabfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb74e59c640 1 -- 172.21.15.5:0/2699420599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bbaf370 msgr2=0x56388bbac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb74e59c640 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bbaf370 0x56388bbac560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb74e59c640 1 -- 172.21.15.5:0/2699420599 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56388b87b3c0 con 0x56388bba45d0 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb74e59c640 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbaed10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb740000c00 tx=0x7fb740005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84cfd73ebf3b8757 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb735ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb735ffb640 1 -- 172.21.15.5:0/2699420599 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740029070 con 0x56388bba45d0 2024-09-19T06:10:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb756cea740 1 -- 172.21.15.5:0/2699420599 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7400054d0 con 0x56388bba45d0 2024-09-19T06:10:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb735ffb640 1 -- 172.21.15.5:0/2699420599 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fb740003a00 con 0x56388bba45d0 2024-09-19T06:10:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb735ffb640 1 -- 172.21.15.5:0/2699420599 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb740035050 con 0x56388bba45d0 2024-09-19T06:10:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb756cea740 10 client.26749 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.098+0000 7fb756cea740 1 -- 172.21.15.5:0/2699420599 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56388bbbdac0 con 0x56388bba45d0 2024-09-19T06:10:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.099+0000 7fb735ffb640 1 -- 172.21.15.5:0/2699420599 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7400054d0 con 0x56388bba45d0 2024-09-19T06:10:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.099+0000 7fb735ffb640 1 -- 172.21.15.5:0/2699420599 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb74002e440 con 0x56388bba45d0 2024-09-19T06:10:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.099+0000 7fb756cea740 10 client.26749 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.099+0000 7fb756cea740 10 client.26749 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:10:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.099+0000 7fb756cea740 20 client.26749 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.099+0000 7fb756cea740 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56388bbbee70 0x56388bbc1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.099+0000 7fb756cea740 4 client.26749 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.099+0000 7fb756cea740 1 -- 172.21.15.5:0/2699420599 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56388bb17860 con 0x56388bbbee70 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.099+0000 7fb747fff640 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56388bbbee70 0x56388bbc1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.099+0000 7fb747fff640 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56388bbbee70 0x56388bbc1260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51b4318f53741c63 server_cookie=ea78a00600c51661 in_seq=0 out_seq=0 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.099+0000 7fb735ffb640 10 client.26749 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.100+0000 7fb735ffb640 1 -- 172.21.15.5:0/2699420599 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56388bb17860 con 0x56388bbbee70 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.100+0000 7fb735ffb640 10 client.26749 handle_command_reply: tid=0 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.100+0000 7fb756cea740 1 client.26749 shutdown 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.100+0000 7fb756cea740 1 -- 172.21.15.5:0/2699420599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 msgr2=0x56388bbaed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.100+0000 7fb756cea740 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbaed10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb740000c00 tx=0x7fb740005db0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.100+0000 7fb756cea740 1 -- 172.21.15.5:0/2699420599 shutdown_connections 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.100+0000 7fb756cea740 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56388bbbee70 0x56388bbc1260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.100+0000 7fb735ffb640 0 client.26749 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.100+0000 7fb756cea740 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bbaf370 0x56388bbac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.101+0000 7fb756cea740 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bba5b40 0x56388bbabfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.101+0000 7fb756cea740 1 --2- 172.21.15.5:0/2699420599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbaed10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.101+0000 7fb756cea740 1 -- 172.21.15.5:0/2699420599 >> 172.21.15.5:0/2699420599 conn(0x56388bb6d310 msgr2=0x56388bba6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.101+0000 7fb756cea740 1 -- 172.21.15.5:0/2699420599 shutdown_connections 2024-09-19T06:10:32.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.101+0000 7fb756cea740 1 -- 172.21.15.5:0/2699420599 wait complete. 2024-09-19T06:10:32.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.101+0000 7fb756cea740 20 client.26749 trim_cache size 0 max 16384 2024-09-19T06:10:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb756cea740 1 Processor -- start 2024-09-19T06:10:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb756cea740 1 -- start start 2024-09-19T06:10:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb756cea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbbb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb756cea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bbab880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb756cea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bbaf370 0x56388bbc6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb756cea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501ab790 con 0x56388bba5b40 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb756cea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501ad2d0 con 0x56388bbaf370 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb756cea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501abe60 con 0x56388bba45d0 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb747fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bbab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb747fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bbab880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56926/0 (socket says 172.21.15.5:56926) 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb747fff640 1 -- 172.21.15.5:0/4061431394 learned_addr learned my addr 172.21.15.5:0/4061431394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb74ed9d640 1 --2- 172.21.15.5:0/4061431394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bbaf370 0x56388bbc6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb74e59c640 1 --2- 172.21.15.5:0/4061431394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbbb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb74ed9d640 1 -- 172.21.15.5:0/4061431394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 msgr2=0x56388bbbb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb74ed9d640 1 --2- 172.21.15.5:0/4061431394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbbb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb74ed9d640 1 -- 172.21.15.5:0/4061431394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 msgr2=0x56388bbab880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb74ed9d640 1 --2- 172.21.15.5:0/4061431394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bbab880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb74ed9d640 1 -- 172.21.15.5:0/4061431394 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56388bba0de0 con 0x56388bbaf370 2024-09-19T06:10:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.115+0000 7fb747fff640 1 --2- 172.21.15.5:0/4061431394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bbab880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:32.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.116+0000 7fb74ed9d640 1 --2- 172.21.15.5:0/4061431394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bbaf370 0x56388bbc6980 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb7480d0dc0 tx=0x7fb74800e860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caed338a09f467e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.116+0000 7fb737fff640 1 -- 172.21.15.5:0/4061431394 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7480a3d90 con 0x56388bbaf370 2024-09-19T06:10:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.116+0000 7fb737fff640 1 -- 172.21.15.5:0/4061431394 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7480a85c0 con 0x56388bbaf370 2024-09-19T06:10:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.116+0000 7fb737fff640 1 -- 172.21.15.5:0/4061431394 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74809c050 con 0x56388bbaf370 2024-09-19T06:10:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 -- 172.21.15.5:0/4061431394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bbaf370 msgr2=0x56388bbc6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 --2- 172.21.15.5:0/4061431394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bbaf370 0x56388bbc6980 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb7480d0dc0 tx=0x7fb74800e860 comp rx=0 tx=0).stop 2024-09-19T06:10:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 -- 172.21.15.5:0/4061431394 shutdown_connections 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 --2- 172.21.15.5:0/4061431394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bbaf370 0x56388bbc6980 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 --2- 172.21.15.5:0/4061431394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bbab880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 --2- 172.21.15.5:0/4061431394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbbb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 -- 172.21.15.5:0/4061431394 >> 172.21.15.5:0/4061431394 conn(0x56388bb6d310 msgr2=0x56388bb9fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 -- 172.21.15.5:0/4061431394 shutdown_connections 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 -- 172.21.15.5:0/4061431394 wait complete. 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 -- start start 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 Processor -- start 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbb7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bba8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bbaf370 0x56388bba9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501acc00 con 0x56388bba5b40 2024-09-19T06:10:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.117+0000 7fb756cea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501ac530 con 0x56388bbaf370 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb756cea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501a9790 con 0x56388bba45d0 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb756cea740 10 client.0 _ll_register_callbacks cb 0x7fb740009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb74ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bbaf370 0x56388bba9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb747fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bba8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb74e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbb7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb74ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bbaf370 0x56388bba9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44686/0 (socket says 172.21.15.5:44686) 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb74ed9d640 1 -- 172.21.15.5:0/1556355266 learned_addr learned my addr 172.21.15.5:0/1556355266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb747fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bba8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56930/0 (socket says 172.21.15.5:56930) 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb74e59c640 1 -- 172.21.15.5:0/1556355266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bbaf370 msgr2=0x56388bba9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb74e59c640 1 --2- 172.21.15.5:0/1556355266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bbaf370 0x56388bba9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb74e59c640 1 -- 172.21.15.5:0/1556355266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 msgr2=0x56388bba8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb74e59c640 1 --2- 172.21.15.5:0/1556355266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bba8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb74e59c640 1 -- 172.21.15.5:0/1556355266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56388bc28950 con 0x56388bba45d0 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb747fff640 1 --2- 172.21.15.5:0/1556355266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bba8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb74e59c640 1 --2- 172.21.15.5:0/1556355266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbb7d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb740011d10 tx=0x7fb74002ea40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4c86c7cf176ad79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb735ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb735ffb640 1 -- 172.21.15.5:0/1556355266 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740029040 con 0x56388bba45d0 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb735ffb640 1 -- 172.21.15.5:0/1556355266 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fb740003800 con 0x56388bba45d0 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.118+0000 7fb756cea740 1 -- 172.21.15.5:0/1556355266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7480a85c0 con 0x56388bba45d0 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.119+0000 7fb735ffb640 1 -- 172.21.15.5:0/1556355266 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb740004560 con 0x56388bba45d0 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.119+0000 7fb756cea740 10 client.26755 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.119+0000 7fb756cea740 1 -- 172.21.15.5:0/1556355266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56388bc29a90 con 0x56388bba45d0 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.119+0000 7fb735ffb640 1 -- 172.21.15.5:0/1556355266 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb740032030 con 0x56388bba45d0 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.119+0000 7fb735ffb640 1 -- 172.21.15.5:0/1556355266 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb7400238b0 con 0x56388bba45d0 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.119+0000 7fb756cea740 10 client.26755 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.119+0000 7fb756cea740 10 client.26755 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.119+0000 7fb756cea740 20 client.26755 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.119+0000 7fb756cea740 1 --2- 172.21.15.5:0/1556355266 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56388bbbee70 0x56388bc2c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.119+0000 7fb756cea740 4 client.26755 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.119+0000 7fb756cea740 1 -- 172.21.15.5:0/1556355266 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb7480a85c0 con 0x56388bbbee70 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.119+0000 7fb747fff640 1 --2- 172.21.15.5:0/1556355266 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56388bbbee70 0x56388bc2c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb747fff640 1 --2- 172.21.15.5:0/1556355266 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56388bbbee70 0x56388bc2c120 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5cdf7b20b49ce067 server_cookie=f52ad0d64741ef1f in_seq=0 out_seq=0 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb735ffb640 10 client.26755 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb735ffb640 1 -- 172.21.15.5:0/1556355266 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb7480a85c0 con 0x56388bbbee70 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb735ffb640 10 client.26755 handle_command_reply: tid=0 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb756cea740 1 client.26755 shutdown 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb756cea740 1 -- 172.21.15.5:0/1556355266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 msgr2=0x56388bbb7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb756cea740 1 --2- 172.21.15.5:0/1556355266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbb7d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb740011d10 tx=0x7fb74002ea40 comp rx=0 tx=0).stop 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb756cea740 1 -- 172.21.15.5:0/1556355266 shutdown_connections 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb756cea740 1 --2- 172.21.15.5:0/1556355266 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56388bbbee70 0x56388bc2c120 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb756cea740 1 --2- 172.21.15.5:0/1556355266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56388bbaf370 0x56388bba9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb756cea740 1 --2- 172.21.15.5:0/1556355266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56388bba5b40 0x56388bba8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb756cea740 1 --2- 172.21.15.5:0/1556355266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56388bba45d0 0x56388bbb7d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb735ffb640 0 client.26755 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:10:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb756cea740 1 -- 172.21.15.5:0/1556355266 >> 172.21.15.5:0/1556355266 conn(0x56388bb6d310 msgr2=0x56388bba6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb756cea740 1 -- 172.21.15.5:0/1556355266 shutdown_connections 2024-09-19T06:10:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb756cea740 1 -- 172.21.15.5:0/1556355266 wait complete. 2024-09-19T06:10:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.120+0000 7fb756cea740 20 client.26755 trim_cache size 0 max 16384 2024-09-19T06:10:32.128 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:10:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.121+0000 7fb754911640 1 -- 172.21.15.5:0/1795529060 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb73c0a33b0 msgr2=0x7fb73c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.121+0000 7fb754911640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb73c0a33b0 0x7fb73c0a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb7380096c0 tx=0x7fb738000bc0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.121+0000 7fb754911640 1 -- 172.21.15.5:0/1795529060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb750108290 msgr2=0x7fb75019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.121+0000 7fb754911640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb750108290 0x7fb75019cde0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb748007d10 tx=0x7fb748010420 comp rx=0 tx=0).stop 2024-09-19T06:10:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.121+0000 7fb754911640 1 -- 172.21.15.5:0/1795529060 shutdown_connections 2024-09-19T06:10:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.121+0000 7fb754911640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb73c0a33b0 0x7fb73c0a5870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.121+0000 7fb754911640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb750108290 0x7fb75019cde0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.121+0000 7fb754911640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb750103740 0x7fb750198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.121+0000 7fb754911640 1 --2- 172.21.15.5:0/1795529060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb750102d90 0x7fb750198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.121+0000 7fb754911640 1 -- 172.21.15.5:0/1795529060 >> 172.21.15.5:0/1795529060 conn(0x7fb7500fc7d0 msgr2=0x7fb750105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.121+0000 7fb754911640 1 -- 172.21.15.5:0/1795529060 shutdown_connections 2024-09-19T06:10:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:32.121+0000 7fb754911640 1 -- 172.21.15.5:0/1795529060 wait complete. 2024-09-19T06:10:32.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:32 smithi005 ceph-mon[30955]: pgmap v221: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-19T06:10:32.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:32 smithi049 ceph-mon[34640]: pgmap v221: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-19T06:10:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:32 smithi157 ceph-mon[35346]: pgmap v221: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-19T06:10:32.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 182445 2.72 MB/sec execute 238 sec latency 67.127 ms 2024-09-19T06:10:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.851+0000 7f42695a6640 1 -- 172.21.15.5:0/4227862636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4264107ad0 msgr2=0x7f4264107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.851+0000 7f42695a6640 1 --2- 172.21.15.5:0/4227862636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4264107ad0 0x7f4264107eb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4254009cd0 tx=0x7f425402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.852+0000 7f42695a6640 1 -- 172.21.15.5:0/4227862636 shutdown_connections 2024-09-19T06:10:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.852+0000 7f42695a6640 1 --2- 172.21.15.5:0/4227862636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264102460 0x7f426410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.852+0000 7f42695a6640 1 --2- 172.21.15.5:0/4227862636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4264101ac0 0x7f4264101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.852+0000 7f42695a6640 1 --2- 172.21.15.5:0/4227862636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4264107ad0 0x7f4264107eb0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.852+0000 7f42695a6640 1 -- 172.21.15.5:0/4227862636 >> 172.21.15.5:0/4227862636 conn(0x7f42640fd7a0 msgr2=0x7f42640ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.852+0000 7f42695a6640 1 -- 172.21.15.5:0/4227862636 shutdown_connections 2024-09-19T06:10:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.852+0000 7f42695a6640 1 -- 172.21.15.5:0/4227862636 wait complete. 2024-09-19T06:10:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.852+0000 7f42695a6640 1 Processor -- start 2024-09-19T06:10:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42695a6640 1 -- start start 2024-09-19T06:10:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42695a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4264101ac0 0x7f42640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42695a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4264102460 0x7f4264074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42695a6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264107ad0 0x7f4264074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42695a6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4264111e10 con 0x7f4264101ac0 2024-09-19T06:10:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42695a6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4264111c90 con 0x7f4264107ad0 2024-09-19T06:10:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42695a6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4264111f90 con 0x7f4264102460 2024-09-19T06:10:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f4262ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4264101ac0 0x7f42640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42627fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4264102460 0x7f4264074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f4262ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4264101ac0 0x7f42640744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56944/0 (socket says 172.21.15.5:56944) 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f4262ffd640 1 -- 172.21.15.5:0/303976027 learned_addr learned my addr 172.21.15.5:0/303976027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42637fe640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264107ad0 0x7f4264074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42627fc640 1 -- 172.21.15.5:0/303976027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264107ad0 msgr2=0x7f4264074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42627fc640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264107ad0 0x7f4264074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42627fc640 1 -- 172.21.15.5:0/303976027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4264101ac0 msgr2=0x7f42640744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.853+0000 7f42627fc640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4264101ac0 0x7f42640744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.854+0000 7f42627fc640 1 -- 172.21.15.5:0/303976027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42641a6890 con 0x7f4264102460 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.854+0000 7f42627fc640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4264102460 0x7f4264074a30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f424c005fd0 tx=0x7f424c00dd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.854+0000 7f425bfff640 1 -- 172.21.15.5:0/303976027 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c009c30 con 0x7f4264102460 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.854+0000 7f425bfff640 1 -- 172.21.15.5:0/303976027 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f424c00be30 con 0x7f4264102460 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.854+0000 7f42695a6640 1 -- 172.21.15.5:0/303976027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42641a6b80 con 0x7f4264102460 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.854+0000 7f425bfff640 1 -- 172.21.15.5:0/303976027 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c003da0 con 0x7f4264102460 2024-09-19T06:10:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.854+0000 7f42695a6640 1 -- 172.21.15.5:0/303976027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42641a70d0 con 0x7f4264102460 2024-09-19T06:10:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.856+0000 7f425bfff640 1 -- 172.21.15.5:0/303976027 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f424c003f40 con 0x7f4264102460 2024-09-19T06:10:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.857+0000 7f426c422740 1 Processor -- start 2024-09-19T06:10:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.857+0000 7f426c422740 1 -- start start 2024-09-19T06:10:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.857+0000 7f426c422740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590eaf0 0x55ae7590eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.857+0000 7f426c422740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590f410 0x55ae75919770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.857+0000 7f426c422740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae75919cb0 0x55ae7591c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.857+0000 7f426c422740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ae758732f0 con 0x55ae7590eaf0 2024-09-19T06:10:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.857+0000 7f425bfff640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42500a3400 0x7f42500a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.857+0000 7f426c422740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ae7581b270 con 0x55ae7590f410 2024-09-19T06:10:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.857+0000 7f426c422740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ae7586f2b0 con 0x55ae75919cb0 2024-09-19T06:10:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.857+0000 7f4262ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590f410 0x55ae75919770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f42637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590eaf0 0x55ae7590eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f4262ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590f410 0x55ae75919770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44720/0 (socket says 172.21.15.5:44720) 2024-09-19T06:10:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f42627fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae75919cb0 0x55ae7591c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f4262ffd640 1 -- 172.21.15.5:0/199495252 learned_addr learned my addr 172.21.15.5:0/199495252 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f42637fe640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42500a3400 0x7f42500a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f425bfff640 1 -- 172.21.15.5:0/303976027 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7f424c0cf7f0 con 0x7f4264102460 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f4262ffd640 1 -- 172.21.15.5:0/199495252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae75919cb0 msgr2=0x55ae7591c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f4262ffd640 1 --2- 172.21.15.5:0/199495252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae75919cb0 0x55ae7591c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f4262ffd640 1 -- 172.21.15.5:0/199495252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590eaf0 msgr2=0x55ae7590eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f4262ffd640 1 --2- 172.21.15.5:0/199495252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590eaf0 0x55ae7590eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f4262ffd640 1 -- 172.21.15.5:0/199495252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae7581a340 con 0x55ae7590f410 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f42637fe640 1 --2- 172.21.15.5:0/199495252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590eaf0 0x55ae7590eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f42637fe640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42500a3400 0x7f42500a58c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f425c0118e0 tx=0x7f425c005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f4262ffd640 1 --2- 172.21.15.5:0/199495252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590f410 0x55ae75919770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f425402aed0 tx=0x7f4254031430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c25d6627fb31ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f42597fa640 1 -- 172.21.15.5:0/199495252 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254048070 con 0x55ae7590f410 2024-09-19T06:10:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.858+0000 7f42597fa640 1 -- 172.21.15.5:0/199495252 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4254036070 con 0x55ae7590f410 2024-09-19T06:10:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.859+0000 7f42597fa640 1 -- 172.21.15.5:0/199495252 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254004330 con 0x55ae7590f410 2024-09-19T06:10:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.859+0000 7f426c422740 1 -- 172.21.15.5:0/199495252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590f410 msgr2=0x55ae75919770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.859+0000 7f426c422740 1 --2- 172.21.15.5:0/199495252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590f410 0x55ae75919770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f425402aed0 tx=0x7f4254031430 comp rx=0 tx=0).stop 2024-09-19T06:10:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.859+0000 7f426c422740 1 -- 172.21.15.5:0/199495252 shutdown_connections 2024-09-19T06:10:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.859+0000 7f426c422740 1 --2- 172.21.15.5:0/199495252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae75919cb0 0x55ae7591c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.859+0000 7f426c422740 1 --2- 172.21.15.5:0/199495252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590f410 0x55ae75919770 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.859+0000 7f426c422740 1 --2- 172.21.15.5:0/199495252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590eaf0 0x55ae7590eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.859+0000 7f426c422740 1 -- 172.21.15.5:0/199495252 >> 172.21.15.5:0/199495252 conn(0x55ae7590b060 msgr2=0x55ae7590b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.859+0000 7f426c422740 1 -- 172.21.15.5:0/199495252 shutdown_connections 2024-09-19T06:10:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f426c422740 1 -- 172.21.15.5:0/199495252 wait complete. 2024-09-19T06:10:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f426c422740 1 -- start start 2024-09-19T06:10:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f426c422740 1 Processor -- start 2024-09-19T06:10:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f426c422740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 0x55ae75916c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f426c422740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 0x55ae75917180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f426c422740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae75919cb0 0x55ae759176c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f426c422740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ae7591c880 con 0x55ae7590f410 2024-09-19T06:10:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f426c422740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ae757fc290 con 0x55ae7590eaf0 2024-09-19T06:10:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f426c422740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ae7591ca00 con 0x55ae75919cb0 2024-09-19T06:10:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f426c422740 10 client.0 _ll_register_callbacks cb 0x55ae7582d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f4262ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 0x55ae75916c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f42627fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 0x55ae75917180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f4262ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 0x55ae75916c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44728/0 (socket says 172.21.15.5:44728) 2024-09-19T06:10:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f4262ffd640 1 -- 172.21.15.5:0/3452694418 learned_addr learned my addr 172.21.15.5:0/3452694418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.860+0000 7f42637fe640 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae75919cb0 0x55ae759176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.861+0000 7f4262ffd640 1 -- 172.21.15.5:0/3452694418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae75919cb0 msgr2=0x55ae759176c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.861+0000 7f4262ffd640 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae75919cb0 0x55ae759176c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.861+0000 7f4262ffd640 1 -- 172.21.15.5:0/3452694418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 msgr2=0x55ae75917180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.861+0000 7f4262ffd640 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 0x55ae75917180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.861+0000 7f4262ffd640 1 -- 172.21.15.5:0/3452694418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae75586210 con 0x55ae7590eaf0 2024-09-19T06:10:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.861+0000 7f42627fc640 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 0x55ae75917180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.861+0000 7f4262ffd640 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 0x55ae75916c40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4254006010 tx=0x7f425402a8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5decc12d4d4b8698 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.861+0000 7f424a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:10:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.861+0000 7f424a7fc640 1 -- 172.21.15.5:0/3452694418 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254048040 con 0x55ae7590eaf0 2024-09-19T06:10:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.861+0000 7f426c422740 1 -- 172.21.15.5:0/3452694418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4254036070 con 0x55ae7590eaf0 2024-09-19T06:10:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.861+0000 7f424a7fc640 1 -- 172.21.15.5:0/3452694418 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7f4254031a60 con 0x55ae7590eaf0 2024-09-19T06:10:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.861+0000 7f424a7fc640 1 -- 172.21.15.5:0/3452694418 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f425405d050 con 0x55ae7590eaf0 2024-09-19T06:10:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.862+0000 7f426c422740 10 client.26846 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.862+0000 7f426c422740 1 -- 172.21.15.5:0/3452694418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ae75652620 con 0x55ae7590eaf0 2024-09-19T06:10:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.862+0000 7f424a7fc640 1 -- 172.21.15.5:0/3452694418 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4254036070 con 0x55ae7590eaf0 2024-09-19T06:10:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.862+0000 7f424a7fc640 1 -- 172.21.15.5:0/3452694418 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f4254042030 con 0x55ae7590eaf0 2024-09-19T06:10:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.862+0000 7f426c422740 10 client.26846 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.862+0000 7f426c422740 10 client.26846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.862+0000 7f426c422740 20 client.26846 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.862+0000 7f426c422740 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ae75929b20 0x55ae7592bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.862+0000 7f426c422740 4 client.26846 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.862+0000 7f426c422740 1 -- 172.21.15.5:0/3452694418 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae75880a20 con 0x55ae75929b20 2024-09-19T06:10:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.862+0000 7f42627fc640 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ae75929b20 0x55ae7592bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.863+0000 7f424a7fc640 10 client.26846 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.863+0000 7f42627fc640 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ae75929b20 0x55ae7592bf10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb4f397e2f552764 server_cookie=76af2528eea67ad6 in_seq=0 out_seq=0 2024-09-19T06:10:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.863+0000 7f424a7fc640 1 -- 172.21.15.5:0/3452694418 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae75880a20 con 0x55ae75929b20 2024-09-19T06:10:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.863+0000 7f424a7fc640 10 client.26846 handle_command_reply: tid=0 2024-09-19T06:10:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.863+0000 7f426c422740 1 client.26846 shutdown 2024-09-19T06:10:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.864+0000 7f426c422740 1 -- 172.21.15.5:0/3452694418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 msgr2=0x55ae75916c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.864+0000 7f426c422740 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 0x55ae75916c40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4254006010 tx=0x7f425402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.864+0000 7f426c422740 1 -- 172.21.15.5:0/3452694418 shutdown_connections 2024-09-19T06:10:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.864+0000 7f426c422740 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ae75929b20 0x55ae7592bf10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.864+0000 7f424a7fc640 0 client.26846 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.864+0000 7f426c422740 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae75919cb0 0x55ae759176c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.864+0000 7f426c422740 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 0x55ae75917180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.864+0000 7f426c422740 1 --2- 172.21.15.5:0/3452694418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 0x55ae75916c40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.864+0000 7f426c422740 1 -- 172.21.15.5:0/3452694418 >> 172.21.15.5:0/3452694418 conn(0x55ae7590b060 msgr2=0x55ae75910dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.864+0000 7f426c422740 1 -- 172.21.15.5:0/3452694418 shutdown_connections 2024-09-19T06:10:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.864+0000 7f426c422740 1 -- 172.21.15.5:0/3452694418 wait complete. 2024-09-19T06:10:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.864+0000 7f426c422740 20 client.26846 trim_cache size 0 max 16384 2024-09-19T06:10:32.893 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:10:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.891+0000 7f426c422740 1 Processor -- start 2024-09-19T06:10:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.891+0000 7f426c422740 1 -- start start 2024-09-19T06:10:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.891+0000 7f426c422740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 0x55ae759174f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.891+0000 7f426c422740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae7590f410 0x55ae759314b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.891+0000 7f426c422740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae75919cb0 0x55ae759319f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.891+0000 7f426c422740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42641af930 con 0x55ae75919cb0 2024-09-19T06:10:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.891+0000 7f426c422740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f42641b1470 con 0x55ae7590eaf0 2024-09-19T06:10:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.891+0000 7f426c422740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f42641b0000 con 0x55ae7590f410 2024-09-19T06:10:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f4262ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 0x55ae759174f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f42637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae75919cb0 0x55ae759319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f42627fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae7590f410 0x55ae759314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f4262ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 0x55ae759174f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44734/0 (socket says 172.21.15.5:44734) 2024-09-19T06:10:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f42637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae75919cb0 0x55ae759319f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56968/0 (socket says 172.21.15.5:56968) 2024-09-19T06:10:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f4262ffd640 1 -- 172.21.15.5:0/4004295669 learned_addr learned my addr 172.21.15.5:0/4004295669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f4262ffd640 1 -- 172.21.15.5:0/4004295669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae7590f410 msgr2=0x55ae759314b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f4262ffd640 1 --2- 172.21.15.5:0/4004295669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae7590f410 0x55ae759314b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f4262ffd640 1 -- 172.21.15.5:0/4004295669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae75919cb0 msgr2=0x55ae759319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f4262ffd640 1 --2- 172.21.15.5:0/4004295669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae75919cb0 0x55ae759319f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f4262ffd640 1 -- 172.21.15.5:0/4004295669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae7592c280 con 0x55ae7590eaf0 2024-09-19T06:10:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f42637fe640 1 --2- 172.21.15.5:0/4004295669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae75919cb0 0x55ae759319f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f4262ffd640 1 --2- 172.21.15.5:0/4004295669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 0x55ae759174f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f42540305c0 tx=0x7f4254003ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f34f5ed1f50020 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f4258ff9640 1 -- 172.21.15.5:0/4004295669 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254030c90 con 0x55ae7590eaf0 2024-09-19T06:10:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f4258ff9640 1 -- 172.21.15.5:0/4004295669 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4254036040 con 0x55ae7590eaf0 2024-09-19T06:10:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.892+0000 7f4258ff9640 1 -- 172.21.15.5:0/4004295669 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42540043f0 con 0x55ae7590eaf0 2024-09-19T06:10:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.893+0000 7f426c422740 1 -- 172.21.15.5:0/4004295669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 msgr2=0x55ae759174f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.893+0000 7f426c422740 1 --2- 172.21.15.5:0/4004295669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 0x55ae759174f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f42540305c0 tx=0x7f4254003ed0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.893+0000 7f426c422740 1 -- 172.21.15.5:0/4004295669 shutdown_connections 2024-09-19T06:10:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.893+0000 7f426c422740 1 --2- 172.21.15.5:0/4004295669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae75919cb0 0x55ae759319f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.893+0000 7f426c422740 1 --2- 172.21.15.5:0/4004295669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae7590f410 0x55ae759314b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.893+0000 7f426c422740 1 --2- 172.21.15.5:0/4004295669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae7590eaf0 0x55ae759174f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.893+0000 7f426c422740 1 -- 172.21.15.5:0/4004295669 >> 172.21.15.5:0/4004295669 conn(0x55ae7590b060 msgr2=0x55ae75915d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.893+0000 7f426c422740 1 -- 172.21.15.5:0/4004295669 shutdown_connections 2024-09-19T06:10:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.893+0000 7f426c422740 1 -- 172.21.15.5:0/4004295669 wait complete. 2024-09-19T06:10:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f426c422740 1 -- start start 2024-09-19T06:10:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f426c422740 1 Processor -- start 2024-09-19T06:10:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f426c422740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae7590eaf0 0x55ae75921ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f426c422740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 0x55ae759221e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f426c422740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae75919cb0 0x55ae7591d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f426c422740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42641b0da0 con 0x55ae7590f410 2024-09-19T06:10:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f426c422740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f42641b06d0 con 0x55ae75919cb0 2024-09-19T06:10:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f426c422740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f42641ad930 con 0x55ae7590eaf0 2024-09-19T06:10:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f426c422740 10 client.0 _ll_register_callbacks cb 0x7f4254046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f42627fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 0x55ae759221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f4262ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae7590eaf0 0x55ae75921ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f42627fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 0x55ae759221e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56974/0 (socket says 172.21.15.5:56974) 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f42627fc640 1 -- 172.21.15.5:0/1872831580 learned_addr learned my addr 172.21.15.5:0/1872831580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.894+0000 7f42637fe640 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae75919cb0 0x55ae7591d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f4262ffd640 1 -- 172.21.15.5:0/1872831580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae75919cb0 msgr2=0x55ae7591d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f4262ffd640 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae75919cb0 0x55ae7591d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f4262ffd640 1 -- 172.21.15.5:0/1872831580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 msgr2=0x55ae759221e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f4262ffd640 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 0x55ae759221e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f4262ffd640 1 -- 172.21.15.5:0/1872831580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae7591da40 con 0x55ae7590eaf0 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f42627fc640 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 0x55ae759221e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f4262ffd640 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae7590eaf0 0x55ae75921ca0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4254030640 tx=0x7f4254009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd74cb8387e9ee09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f424a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f424a7fc640 1 -- 172.21.15.5:0/1872831580 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254048070 con 0x55ae7590eaf0 2024-09-19T06:10:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f426c422740 1 -- 172.21.15.5:0/1872831580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4254036040 con 0x55ae7590eaf0 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f424a7fc640 1 -- 172.21.15.5:0/1872831580 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7f4254030c90 con 0x55ae7590eaf0 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f424a7fc640 1 -- 172.21.15.5:0/1872831580 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4254030e90 con 0x55ae7590eaf0 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f426c422740 10 client.26773 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.895+0000 7f426c422740 1 -- 172.21.15.5:0/1872831580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ae75993f30 con 0x55ae7590eaf0 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.896+0000 7f424a7fc640 1 -- 172.21.15.5:0/1872831580 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4254036040 con 0x55ae7590eaf0 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.896+0000 7f424a7fc640 1 -- 172.21.15.5:0/1872831580 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f4254042070 con 0x55ae7590eaf0 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.896+0000 7f426c422740 10 client.26773 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.896+0000 7f426c422740 10 client.26773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.896+0000 7f426c422740 20 client.26773 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.896+0000 7f426c422740 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ae75929b20 0x55ae759965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.896+0000 7f426c422740 4 client.26773 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.896+0000 7f426c422740 1 -- 172.21.15.5:0/1872831580 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae7587ad70 con 0x55ae75929b20 2024-09-19T06:10:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.896+0000 7f42627fc640 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ae75929b20 0x55ae759965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.896+0000 7f424a7fc640 10 client.26773 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.896+0000 7f42627fc640 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ae75929b20 0x55ae759965f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe679ad48e3197aa server_cookie=335e66c027e242a in_seq=0 out_seq=0 2024-09-19T06:10:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f424a7fc640 1 -- 172.21.15.5:0/1872831580 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae7587ad70 con 0x55ae75929b20 2024-09-19T06:10:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f424a7fc640 10 client.26773 handle_command_reply: tid=0 2024-09-19T06:10:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f426c422740 1 client.26773 shutdown 2024-09-19T06:10:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f426c422740 1 -- 172.21.15.5:0/1872831580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae7590eaf0 msgr2=0x55ae75921ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f426c422740 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae7590eaf0 0x55ae75921ca0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4254030640 tx=0x7f4254009360 comp rx=0 tx=0).stop 2024-09-19T06:10:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f426c422740 1 -- 172.21.15.5:0/1872831580 shutdown_connections 2024-09-19T06:10:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f426c422740 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ae75929b20 0x55ae759965f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f426c422740 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ae75919cb0 0x55ae7591d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f426c422740 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae7590f410 0x55ae759221e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f424a7fc640 0 client.26773 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f426c422740 1 --2- 172.21.15.5:0/1872831580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae7590eaf0 0x55ae75921ca0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f426c422740 1 -- 172.21.15.5:0/1872831580 >> 172.21.15.5:0/1872831580 conn(0x55ae7590b060 msgr2=0x55ae75917500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f426c422740 1 -- 172.21.15.5:0/1872831580 shutdown_connections 2024-09-19T06:10:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f426c422740 1 -- 172.21.15.5:0/1872831580 wait complete. 2024-09-19T06:10:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.918+0000 7f426c422740 20 client.26773 trim_cache size 0 max 16384 2024-09-19T06:10:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:10:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:10:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:10:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:10:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.919+0000 7f42695a6640 1 -- 172.21.15.5:0/303976027 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42500a3400 msgr2=0x7f42500a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.919+0000 7f42695a6640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42500a3400 0x7f42500a58c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f425c0118e0 tx=0x7f425c005f90 comp rx=0 tx=0).stop 2024-09-19T06:10:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.919+0000 7f42695a6640 1 -- 172.21.15.5:0/303976027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4264102460 msgr2=0x7f4264074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.919+0000 7f42695a6640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4264102460 0x7f4264074a30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f424c005fd0 tx=0x7f424c00dd20 comp rx=0 tx=0).stop 2024-09-19T06:10:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.919+0000 7f42695a6640 1 -- 172.21.15.5:0/303976027 shutdown_connections 2024-09-19T06:10:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.919+0000 7f42695a6640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264107ad0 0x7f4264074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.919+0000 7f42695a6640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f42500a3400 0x7f42500a58c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.919+0000 7f42695a6640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4264102460 0x7f4264074a30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.919+0000 7f42695a6640 1 --2- 172.21.15.5:0/303976027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4264101ac0 0x7f42640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.919+0000 7f42695a6640 1 -- 172.21.15.5:0/303976027 >> 172.21.15.5:0/303976027 conn(0x7f42640fd7a0 msgr2=0x7f42640ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.920+0000 7f42695a6640 1 -- 172.21.15.5:0/303976027 shutdown_connections 2024-09-19T06:10:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:32.920+0000 7f42695a6640 1 -- 172.21.15.5:0/303976027 wait complete. 2024-09-19T06:10:33.177 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:10:33.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.720+0000 7fd195e4f640 1 -- 172.21.15.5:0/472251079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1901009b0 msgr2=0x7fd190100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:33.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.720+0000 7fd195e4f640 1 --2- 172.21.15.5:0/472251079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1901009b0 0x7fd190100d90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd180009900 tx=0x7fd18002a410 comp rx=0 tx=0).stop 2024-09-19T06:10:33.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.720+0000 7fd195e4f640 1 -- 172.21.15.5:0/472251079 shutdown_connections 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.720+0000 7fd195e4f640 1 --2- 172.21.15.5:0/472251079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd19010e4d0 0x7fd1901108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.720+0000 7fd195e4f640 1 --2- 172.21.15.5:0/472251079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd190101360 0x7fd19010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.720+0000 7fd195e4f640 1 --2- 172.21.15.5:0/472251079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1901009b0 0x7fd190100d90 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.720+0000 7fd195e4f640 1 -- 172.21.15.5:0/472251079 >> 172.21.15.5:0/472251079 conn(0x7fd1900fc7d0 msgr2=0x7fd1900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.720+0000 7fd195e4f640 1 -- 172.21.15.5:0/472251079 shutdown_connections 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.720+0000 7fd195e4f640 1 -- 172.21.15.5:0/472251079 wait complete. 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.720+0000 7fd195e4f640 1 Processor -- start 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd195e4f640 1 -- start start 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd195e4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1901009b0 0x7fd19019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd195e4f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd190101360 0x7fd19019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd195e4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd19010e4d0 0x7fd19019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd195e4f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1901140c0 con 0x7fd19010e4d0 2024-09-19T06:10:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd195e4f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd190113f40 con 0x7fd1901009b0 2024-09-19T06:10:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd195e4f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd190114240 con 0x7fd190101360 2024-09-19T06:10:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd18ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd19010e4d0 0x7fd19019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd18ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd19010e4d0 0x7fd19019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56998/0 (socket says 172.21.15.5:56998) 2024-09-19T06:10:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd18effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd190101360 0x7fd19019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd18ffff640 1 -- 172.21.15.5:0/2038266427 learned_addr learned my addr 172.21.15.5:0/2038266427 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1901009b0 0x7fd19019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd18effd640 1 -- 172.21.15.5:0/2038266427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1901009b0 msgr2=0x7fd19019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd18effd640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1901009b0 0x7fd19019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd18effd640 1 -- 172.21.15.5:0/2038266427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd19010e4d0 msgr2=0x7fd19019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd18effd640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd19010e4d0 0x7fd19019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.721+0000 7fd18effd640 1 -- 172.21.15.5:0/2038266427 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19019f630 con 0x7fd190101360 2024-09-19T06:10:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.722+0000 7fd18effd640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd190101360 0x7fd19019abc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d6d0 tx=0x7fd18400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.722+0000 7fd18cff9640 1 -- 172.21.15.5:0/2038266427 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184017070 con 0x7fd190101360 2024-09-19T06:10:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.722+0000 7fd195e4f640 1 -- 172.21.15.5:0/2038266427 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19019f920 con 0x7fd190101360 2024-09-19T06:10:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.722+0000 7fd18cff9640 1 -- 172.21.15.5:0/2038266427 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd184009820 con 0x7fd190101360 2024-09-19T06:10:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.722+0000 7fd18cff9640 1 -- 172.21.15.5:0/2038266427 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184004c40 con 0x7fd190101360 2024-09-19T06:10:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.722+0000 7fd195e4f640 1 -- 172.21.15.5:0/2038266427 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1901a7200 con 0x7fd190101360 2024-09-19T06:10:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.724+0000 7fd198228740 1 Processor -- start 2024-09-19T06:10:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.724+0000 7fd198228740 1 -- start start 2024-09-19T06:10:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.724+0000 7fd18cff9640 1 -- 172.21.15.5:0/2038266427 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd1840099d0 con 0x7fd190101360 2024-09-19T06:10:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd198228740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 0x56340547a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd198228740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x56340547aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd198228740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563405485370 0x563405487760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd198228740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5634053e02f0 con 0x563405485370 2024-09-19T06:10:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd198228740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563405387cf0 con 0x56340547a5d0 2024-09-19T06:10:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd198228740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5634053dc2b0 con 0x56340547bb40 2024-09-19T06:10:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd18f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x56340547aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd18f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x56340547aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44766/0 (socket says 172.21.15.5:44766) 2024-09-19T06:10:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd18f7fe640 1 -- 172.21.15.5:0/3190981139 learned_addr learned my addr 172.21.15.5:0/3190981139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd18effd640 1 --2- 172.21.15.5:0/3190981139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563405485370 0x563405487760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd18ffff640 1 --2- 172.21.15.5:0/3190981139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 0x56340547a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd18f7fe640 1 -- 172.21.15.5:0/3190981139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 msgr2=0x56340547a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/3190981139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 0x56340547a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd18f7fe640 1 -- 172.21.15.5:0/3190981139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563405485370 msgr2=0x563405487760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/3190981139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563405485370 0x563405487760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.725+0000 7fd18f7fe640 1 -- 172.21.15.5:0/3190981139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56340536c3a0 con 0x56340547a5d0 2024-09-19T06:10:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.726+0000 7fd18cff9640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1780a33b0 0x7fd1780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.726+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/3190981139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x56340547aa10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd180009900 tx=0x7fd180032c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2919ffeaaf79a6be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.726+0000 7fd18ffff640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1780a33b0 0x7fd1780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.726+0000 7fd18cff9640 1 -- 172.21.15.5:0/2038266427 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fd1840cf640 con 0x7fd190101360 2024-09-19T06:10:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.726+0000 7fd175ffb640 1 -- 172.21.15.5:0/3190981139 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18003f070 con 0x56340547a5d0 2024-09-19T06:10:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.726+0000 7fd175ffb640 1 -- 172.21.15.5:0/3190981139 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1800316a0 con 0x56340547a5d0 2024-09-19T06:10:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.726+0000 7fd175ffb640 1 -- 172.21.15.5:0/3190981139 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180035070 con 0x56340547a5d0 2024-09-19T06:10:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.727+0000 7fd198228740 1 -- 172.21.15.5:0/3190981139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 msgr2=0x56340547aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.727+0000 7fd198228740 1 --2- 172.21.15.5:0/3190981139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x56340547aa10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd180009900 tx=0x7fd180032c20 comp rx=0 tx=0).stop 2024-09-19T06:10:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.727+0000 7fd18ffff640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1780a33b0 0x7fd1780a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd188009370 tx=0x7fd188009760 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.727+0000 7fd198228740 1 -- 172.21.15.5:0/3190981139 shutdown_connections 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.727+0000 7fd198228740 1 --2- 172.21.15.5:0/3190981139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563405485370 0x563405487760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.727+0000 7fd198228740 1 --2- 172.21.15.5:0/3190981139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x56340547aa10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.727+0000 7fd198228740 1 --2- 172.21.15.5:0/3190981139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 0x56340547a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.727+0000 7fd198228740 1 -- 172.21.15.5:0/3190981139 >> 172.21.15.5:0/3190981139 conn(0x563405443310 msgr2=0x563405476c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.727+0000 7fd198228740 1 -- 172.21.15.5:0/3190981139 shutdown_connections 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.727+0000 7fd198228740 1 -- 172.21.15.5:0/3190981139 wait complete. 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.727+0000 7fd198228740 1 -- start start 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.727+0000 7fd198228740 1 Processor -- start 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd198228740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x563405484d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd198228740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 0x563405481ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd198228740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563405485370 0x5634054825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd198228740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563405488060 con 0x563405485370 2024-09-19T06:10:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd198228740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56340539b060 con 0x56340547a5d0 2024-09-19T06:10:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd198228740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5634054881e0 con 0x56340547bb40 2024-09-19T06:10:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd198228740 10 client.0 _ll_register_callbacks cb 0x5634050ee8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd18f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x563405484d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd18ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563405485370 0x5634054825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd18f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x563405484d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44776/0 (socket says 172.21.15.5:44776) 2024-09-19T06:10:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd18f7fe640 1 -- 172.21.15.5:0/731670373 learned_addr learned my addr 172.21.15.5:0/731670373 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd18effd640 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 0x563405481ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd18f7fe640 1 -- 172.21.15.5:0/731670373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 msgr2=0x563405481ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 0x563405481ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd18f7fe640 1 -- 172.21.15.5:0/731670373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563405485370 msgr2=0x5634054825e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563405485370 0x5634054825e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd18f7fe640 1 -- 172.21.15.5:0/731670373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634051513c0 con 0x56340547a5d0 2024-09-19T06:10:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd18ffff640 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563405485370 0x5634054825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x563405484d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd180031440 tx=0x7fd18002aa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac35e6d95cca770 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.728+0000 7fd16effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:10:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd16effd640 1 -- 172.21.15.5:0/731670373 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18004a020 con 0x56340547a5d0 2024-09-19T06:10:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd198228740 1 -- 172.21.15.5:0/731670373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1800316a0 con 0x56340547a5d0 2024-09-19T06:10:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd16effd640 1 -- 172.21.15.5:0/731670373 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fd180004250 con 0x56340547a5d0 2024-09-19T06:10:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd16effd640 1 -- 172.21.15.5:0/731670373 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd180055050 con 0x56340547a5d0 2024-09-19T06:10:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd198228740 10 client.26864 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd198228740 1 -- 172.21.15.5:0/731670373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563405493ad0 con 0x56340547a5d0 2024-09-19T06:10:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd16effd640 1 -- 172.21.15.5:0/731670373 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1800316a0 con 0x56340547a5d0 2024-09-19T06:10:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd16effd640 1 -- 172.21.15.5:0/731670373 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fd180004850 con 0x56340547a5d0 2024-09-19T06:10:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd198228740 10 client.26864 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd198228740 10 client.26864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd198228740 20 client.26864 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd198228740 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563405494e20 0x563405497210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd198228740 4 client.26864 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd198228740 1 -- 172.21.15.5:0/731670373 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634053ed860 con 0x563405494e20 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.729+0000 7fd18effd640 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563405494e20 0x563405497210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.730+0000 7fd18effd640 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563405494e20 0x563405497210 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59957dd7649d0948 server_cookie=997dc02a37b1b4c6 in_seq=0 out_seq=0 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.730+0000 7fd16effd640 10 client.26864 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd16effd640 1 -- 172.21.15.5:0/731670373 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634053ed860 con 0x563405494e20 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd16effd640 10 client.26864 handle_command_reply: tid=0 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd198228740 1 client.26864 shutdown 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd198228740 1 -- 172.21.15.5:0/731670373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 msgr2=0x563405484d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd198228740 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x563405484d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd180031440 tx=0x7fd18002aa90 comp rx=0 tx=0).stop 2024-09-19T06:10:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd198228740 1 -- 172.21.15.5:0/731670373 shutdown_connections 2024-09-19T06:10:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd198228740 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563405494e20 0x563405497210 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd198228740 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563405485370 0x5634054825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd198228740 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 0x563405481ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd16effd640 0 client.26864 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd198228740 1 --2- 172.21.15.5:0/731670373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x563405484d40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd198228740 1 -- 172.21.15.5:0/731670373 >> 172.21.15.5:0/731670373 conn(0x563405443310 msgr2=0x56340547c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd198228740 1 -- 172.21.15.5:0/731670373 shutdown_connections 2024-09-19T06:10:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd198228740 1 -- 172.21.15.5:0/731670373 wait complete. 2024-09-19T06:10:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.731+0000 7fd198228740 20 client.26864 trim_cache size 0 max 16384 2024-09-19T06:10:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd198228740 1 Processor -- start 2024-09-19T06:10:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd198228740 1 -- start start 2024-09-19T06:10:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd198228740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x563405491300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd198228740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547bb40 0x563405475d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd198228740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563405485370 0x56340547d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd198228740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1901afa50 con 0x56340547bb40 2024-09-19T06:10:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd198228740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1901b1590 con 0x56340547a5d0 2024-09-19T06:10:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd198228740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1901b0120 con 0x563405485370 2024-09-19T06:10:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd18f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x563405491300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd18f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x563405491300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44792/0 (socket says 172.21.15.5:44792) 2024-09-19T06:10:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd18f7fe640 1 -- 172.21.15.5:0/2953159217 learned_addr learned my addr 172.21.15.5:0/2953159217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd18effd640 1 --2- 172.21.15.5:0/2953159217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547bb40 0x563405475d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.746+0000 7fd18ffff640 1 --2- 172.21.15.5:0/2953159217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563405485370 0x56340547d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.747+0000 7fd18f7fe640 1 -- 172.21.15.5:0/2953159217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563405485370 msgr2=0x56340547d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.747+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/2953159217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563405485370 0x56340547d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.747+0000 7fd18f7fe640 1 -- 172.21.15.5:0/2953159217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547bb40 msgr2=0x563405475d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.747+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/2953159217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547bb40 0x563405475d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.747+0000 7fd18f7fe640 1 -- 172.21.15.5:0/2953159217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563405480540 con 0x56340547a5d0 2024-09-19T06:10:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.747+0000 7fd18effd640 1 --2- 172.21.15.5:0/2953159217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547bb40 0x563405475d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.747+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/2953159217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x563405491300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd180000c00 tx=0x7fd1800317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eda828ea3f8f578d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.747+0000 7fd1757fa640 1 -- 172.21.15.5:0/2953159217 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18004a070 con 0x56340547a5d0 2024-09-19T06:10:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.747+0000 7fd1757fa640 1 -- 172.21.15.5:0/2953159217 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd180005060 con 0x56340547a5d0 2024-09-19T06:10:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.747+0000 7fd1757fa640 1 -- 172.21.15.5:0/2953159217 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18003f070 con 0x56340547a5d0 2024-09-19T06:10:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.748+0000 7fd198228740 1 -- 172.21.15.5:0/2953159217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 msgr2=0x563405491300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.748+0000 7fd198228740 1 --2- 172.21.15.5:0/2953159217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x563405491300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd180000c00 tx=0x7fd1800317e0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.748+0000 7fd198228740 1 -- 172.21.15.5:0/2953159217 shutdown_connections 2024-09-19T06:10:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.748+0000 7fd198228740 1 --2- 172.21.15.5:0/2953159217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563405485370 0x56340547d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.748+0000 7fd198228740 1 --2- 172.21.15.5:0/2953159217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547bb40 0x563405475d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.748+0000 7fd198228740 1 --2- 172.21.15.5:0/2953159217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56340547a5d0 0x563405491300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.748+0000 7fd198228740 1 -- 172.21.15.5:0/2953159217 >> 172.21.15.5:0/2953159217 conn(0x563405443310 msgr2=0x563405471a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.748+0000 7fd198228740 1 -- 172.21.15.5:0/2953159217 shutdown_connections 2024-09-19T06:10:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.748+0000 7fd198228740 1 -- 172.21.15.5:0/2953159217 wait complete. 2024-09-19T06:10:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd198228740 1 -- start start 2024-09-19T06:10:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd198228740 1 Processor -- start 2024-09-19T06:10:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd198228740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547a5d0 0x563405488470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd198228740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 0x5634054889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd198228740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563405485370 0x563405488ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd198228740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1901b0ec0 con 0x56340547a5d0 2024-09-19T06:10:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd198228740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1901b07f0 con 0x563405485370 2024-09-19T06:10:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd198228740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1901ada50 con 0x56340547bb40 2024-09-19T06:10:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd198228740 10 client.0 _ll_register_callbacks cb 0x7fd1840d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd18f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547a5d0 0x563405488470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd18ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563405485370 0x563405488ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd18f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547a5d0 0x563405488470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57040/0 (socket says 172.21.15.5:57040) 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd18f7fe640 1 -- 172.21.15.5:0/736283036 learned_addr learned my addr 172.21.15.5:0/736283036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.749+0000 7fd18effd640 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 0x5634054889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd18f7fe640 1 -- 172.21.15.5:0/736283036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 msgr2=0x5634054889b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 0x5634054889b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd18f7fe640 1 -- 172.21.15.5:0/736283036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563405485370 msgr2=0x563405488ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563405485370 0x563405488ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd18f7fe640 1 -- 172.21.15.5:0/736283036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634054fda10 con 0x56340547a5d0 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd18ffff640 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563405485370 0x563405488ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd18f7fe640 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547a5d0 0x563405488470 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd18002aad0 tx=0x7fd180031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e271249b5c68f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd16effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd16effd640 1 -- 172.21.15.5:0/736283036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18003f070 con 0x56340547a5d0 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd16effd640 1 -- 172.21.15.5:0/736283036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fd180004450 con 0x56340547a5d0 2024-09-19T06:10:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd198228740 1 -- 172.21.15.5:0/736283036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd180005060 con 0x56340547a5d0 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd16effd640 1 -- 172.21.15.5:0/736283036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd18004a070 con 0x56340547a5d0 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd198228740 10 client.17298 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.750+0000 7fd198228740 1 -- 172.21.15.5:0/736283036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5634054fecd0 con 0x56340547a5d0 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.751+0000 7fd16effd640 1 -- 172.21.15.5:0/736283036 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd180005060 con 0x56340547a5d0 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.751+0000 7fd16effd640 1 -- 172.21.15.5:0/736283036 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fd18005f020 con 0x56340547a5d0 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.751+0000 7fd198228740 10 client.17298 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.751+0000 7fd198228740 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.751+0000 7fd198228740 20 client.17298 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.751+0000 7fd198228740 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563405494e20 0x563405501400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.751+0000 7fd198228740 4 client.17298 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.751+0000 7fd198228740 1 -- 172.21.15.5:0/736283036 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634053e7d70 con 0x563405494e20 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.751+0000 7fd18effd640 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563405494e20 0x563405501400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:33.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd16effd640 10 client.17298 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd18effd640 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563405494e20 0x563405501400 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278a01da8cd80484 server_cookie=6d93602c16d2095c in_seq=0 out_seq=0 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd16effd640 1 -- 172.21.15.5:0/736283036 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634053e7d70 con 0x563405494e20 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd16effd640 10 client.17298 handle_command_reply: tid=0 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd198228740 1 client.17298 shutdown 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd198228740 1 -- 172.21.15.5:0/736283036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547a5d0 msgr2=0x563405488470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd198228740 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547a5d0 0x563405488470 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd18002aad0 tx=0x7fd180031510 comp rx=0 tx=0).stop 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd198228740 1 -- 172.21.15.5:0/736283036 shutdown_connections 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd198228740 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563405494e20 0x563405501400 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd198228740 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563405485370 0x563405488ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd198228740 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56340547bb40 0x5634054889b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd16effd640 0 client.17298 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd198228740 1 --2- 172.21.15.5:0/736283036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56340547a5d0 0x563405488470 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd198228740 1 -- 172.21.15.5:0/736283036 >> 172.21.15.5:0/736283036 conn(0x563405443310 msgr2=0x563405480fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd198228740 1 -- 172.21.15.5:0/736283036 shutdown_connections 2024-09-19T06:10:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd198228740 1 -- 172.21.15.5:0/736283036 wait complete. 2024-09-19T06:10:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.752+0000 7fd198228740 20 client.17298 trim_cache size 0 max 16384 2024-09-19T06:10:33.761 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:10:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.753+0000 7fd195e4f640 1 -- 172.21.15.5:0/2038266427 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1780a33b0 msgr2=0x7fd1780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.753+0000 7fd195e4f640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1780a33b0 0x7fd1780a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd188009370 tx=0x7fd188009760 comp rx=0 tx=0).stop 2024-09-19T06:10:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.753+0000 7fd195e4f640 1 -- 172.21.15.5:0/2038266427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd190101360 msgr2=0x7fd19019abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.753+0000 7fd195e4f640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd190101360 0x7fd19019abc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d6d0 tx=0x7fd18400db90 comp rx=0 tx=0).stop 2024-09-19T06:10:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.753+0000 7fd195e4f640 1 -- 172.21.15.5:0/2038266427 shutdown_connections 2024-09-19T06:10:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.753+0000 7fd195e4f640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd19010e4d0 0x7fd19019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.753+0000 7fd195e4f640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1780a33b0 0x7fd1780a5870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.753+0000 7fd195e4f640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd190101360 0x7fd19019abc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.753+0000 7fd195e4f640 1 --2- 172.21.15.5:0/2038266427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1901009b0 0x7fd19019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.753+0000 7fd195e4f640 1 -- 172.21.15.5:0/2038266427 >> 172.21.15.5:0/2038266427 conn(0x7fd1900fc7d0 msgr2=0x7fd1900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.753+0000 7fd195e4f640 1 -- 172.21.15.5:0/2038266427 shutdown_connections 2024-09-19T06:10:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:33.753+0000 7fd195e4f640 1 -- 172.21.15.5:0/2038266427 wait complete. 2024-09-19T06:10:33.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 182968 2.71 MB/sec execute 239 sec latency 20.287 ms 2024-09-19T06:10:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:34 smithi005 ceph-mon[30955]: pgmap v222: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 2.0 MiB/s wr, 188 op/s 2024-09-19T06:10:34.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:34 smithi049 ceph-mon[34640]: pgmap v222: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 2.0 MiB/s wr, 188 op/s 2024-09-19T06:10:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:34 smithi157 ceph-mon[35346]: pgmap v222: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 2.0 MiB/s wr, 188 op/s 2024-09-19T06:10:34.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 183489 2.71 MB/sec execute 240 sec latency 24.152 ms 2024-09-19T06:10:35.554 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:10:35.554 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:10:35.578 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:35 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:10:35.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:35 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:10:35.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:35 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:10:35.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:35 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:10:35.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:35 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:10:35.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:35 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:10:35.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:35 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:10:35.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:35 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:10:35.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:35 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:10:35.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:35 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:10:35.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:35 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:10:35.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:35 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:10:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:10:35.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 184209 2.73 MB/sec execute 241 sec latency 16.076 ms 2024-09-19T06:10:36.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.327+0000 7f18024ea640 1 -- 172.21.15.5:0/1807254690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc100aa0 msgr2=0x7f17fc10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.327+0000 7f18024ea640 1 --2- 172.21.15.5:0/1807254690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc100aa0 0x7f17fc10ea20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009ca0 tx=0x7f17f402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.327+0000 7f18024ea640 1 -- 172.21.15.5:0/1807254690 shutdown_connections 2024-09-19T06:10:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.327+0000 7f18024ea640 1 --2- 172.21.15.5:0/1807254690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc100aa0 0x7f17fc10ea20 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.327+0000 7f18024ea640 1 --2- 172.21.15.5:0/1807254690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc100100 0x7f17fc100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.327+0000 7f18024ea640 1 --2- 172.21.15.5:0/1807254690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17fc106b80 0x7f17fc106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.327+0000 7f18024ea640 1 -- 172.21.15.5:0/1807254690 >> 172.21.15.5:0/1807254690 conn(0x7f17fc077230 msgr2=0x7f17fc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.327+0000 7f18024ea640 1 -- 172.21.15.5:0/1807254690 shutdown_connections 2024-09-19T06:10:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.327+0000 7f18024ea640 1 -- 172.21.15.5:0/1807254690 wait complete. 2024-09-19T06:10:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f18024ea640 1 Processor -- start 2024-09-19T06:10:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f18024ea640 1 -- start start 2024-09-19T06:10:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f18024ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc100100 0x7f17fc1012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f18024ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17fc100aa0 0x7f17fc101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f18024ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc106b80 0x7f17fc101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f18024ea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc111f00 con 0x7f17fc106b80 2024-09-19T06:10:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f18024ea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc111d80 con 0x7f17fc100aa0 2024-09-19T06:10:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f18024ea640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc112080 con 0x7f17fc100100 2024-09-19T06:10:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f1800a87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc106b80 0x7f17fc101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f1800a87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc106b80 0x7f17fc101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46244/0 (socket says 172.21.15.5:46244) 2024-09-19T06:10:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f17f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17fc100aa0 0x7f17fc101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f1800a87640 1 -- 172.21.15.5:0/3848941312 learned_addr learned my addr 172.21.15.5:0/3848941312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f17fbfff640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc100100 0x7f17fc1012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f17f3fff640 1 -- 172.21.15.5:0/3848941312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc100100 msgr2=0x7f17fc1012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f17f3fff640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc100100 0x7f17fc1012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f17f3fff640 1 -- 172.21.15.5:0/3848941312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc106b80 msgr2=0x7f17fc101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f17f3fff640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc106b80 0x7f17fc101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f17f3fff640 1 -- 172.21.15.5:0/3848941312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc1a6900 con 0x7f17fc100aa0 2024-09-19T06:10:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.328+0000 7f1800a87640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc106b80 0x7f17fc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.329+0000 7f17f3fff640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17fc100aa0 0x7f17fc101820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f17e400d720 tx=0x7f17e400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.329+0000 7f17f9ffb640 1 -- 172.21.15.5:0/3848941312 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4017070 con 0x7f17fc100aa0 2024-09-19T06:10:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.329+0000 7f17f9ffb640 1 -- 172.21.15.5:0/3848941312 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e400de30 con 0x7f17fc100aa0 2024-09-19T06:10:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.329+0000 7f18024ea640 1 -- 172.21.15.5:0/3848941312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc1a6bf0 con 0x7f17fc100aa0 2024-09-19T06:10:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.329+0000 7f17f9ffb640 1 -- 172.21.15.5:0/3848941312 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e40047a0 con 0x7f17fc100aa0 2024-09-19T06:10:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.329+0000 7f18024ea640 1 -- 172.21.15.5:0/3848941312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc1a7130 con 0x7f17fc100aa0 2024-09-19T06:10:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.331+0000 7f17f9ffb640 1 -- 172.21.15.5:0/3848941312 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f17e40099d0 con 0x7f17fc100aa0 2024-09-19T06:10:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.331+0000 7f18048c3740 1 Processor -- start 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f18048c3740 1 -- start start 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f18048c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 0x5558a552ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f18048c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552f480 0x5558a5539800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f18048c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a5539d40 0x5558a553c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f18048c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5558a5494280 con 0x5558a552eb60 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f18048c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5558a543bb10 con 0x5558a552f480 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f18048c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5558a5490240 con 0x5558a5539d40 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f1800a87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 0x5558a552ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f1800a87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 0x5558a552ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46254/0 (socket says 172.21.15.5:46254) 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f1800a87640 1 -- 172.21.15.5:0/3000279065 learned_addr learned my addr 172.21.15.5:0/3000279065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f17fbfff640 1 --2- 172.21.15.5:0/3000279065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552f480 0x5558a5539800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f17f3fff640 1 --2- 172.21.15.5:0/3000279065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a5539d40 0x5558a553c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f1800a87640 1 -- 172.21.15.5:0/3000279065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a5539d40 msgr2=0x5558a553c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f1800a87640 1 --2- 172.21.15.5:0/3000279065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a5539d40 0x5558a553c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f1800a87640 1 -- 172.21.15.5:0/3000279065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552f480 msgr2=0x5558a5539800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f1800a87640 1 --2- 172.21.15.5:0/3000279065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552f480 0x5558a5539800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f1800a87640 1 -- 172.21.15.5:0/3000279065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a54481f0 con 0x5558a552eb60 2024-09-19T06:10:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.332+0000 7f17fbfff640 1 --2- 172.21.15.5:0/3000279065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552f480 0x5558a5539800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.333+0000 7f17f9ffb640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f17e80a3430 0x7f17e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.333+0000 7f17fbfff640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f17e80a3430 0x7f17e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.333+0000 7f17f9ffb640 1 -- 172.21.15.5:0/3848941312 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7f17e40cf880 con 0x7f17fc100aa0 2024-09-19T06:10:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.333+0000 7f1800a87640 1 --2- 172.21.15.5:0/3000279065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 0x5558a552ef40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f17f40098b0 tx=0x7f17f40098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce56eaf81f89c2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.333+0000 7f17f27fc640 1 -- 172.21.15.5:0/3000279065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4031c40 con 0x5558a552eb60 2024-09-19T06:10:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.333+0000 7f17f27fc640 1 -- 172.21.15.5:0/3000279065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17f4036030 con 0x5558a552eb60 2024-09-19T06:10:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.333+0000 7f17f27fc640 1 -- 172.21.15.5:0/3000279065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4004820 con 0x5558a552eb60 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.334+0000 7f18048c3740 1 -- 172.21.15.5:0/3000279065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 msgr2=0x5558a552ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.334+0000 7f18048c3740 1 --2- 172.21.15.5:0/3000279065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 0x5558a552ef40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f17f40098b0 tx=0x7f17f40098e0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.334+0000 7f17fbfff640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f17e80a3430 0x7f17e80a58f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5558a552bf50 tx=0x7f17ec005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.334+0000 7f18048c3740 1 -- 172.21.15.5:0/3000279065 shutdown_connections 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.334+0000 7f18048c3740 1 --2- 172.21.15.5:0/3000279065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a5539d40 0x5558a553c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.334+0000 7f18048c3740 1 --2- 172.21.15.5:0/3000279065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552f480 0x5558a5539800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.334+0000 7f18048c3740 1 --2- 172.21.15.5:0/3000279065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 0x5558a552ef40 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.334+0000 7f18048c3740 1 -- 172.21.15.5:0/3000279065 >> 172.21.15.5:0/3000279065 conn(0x5558a552b0f0 msgr2=0x5558a552b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.334+0000 7f18048c3740 1 -- 172.21.15.5:0/3000279065 shutdown_connections 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.334+0000 7f18048c3740 1 -- 172.21.15.5:0/3000279065 wait complete. 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.334+0000 7f18048c3740 1 -- start start 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f18048c3740 1 Processor -- start 2024-09-19T06:10:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f18048c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552eb60 0x5558a5536c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f18048c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552f480 0x5558a5537140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f18048c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 0x5558a5537680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f18048c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5558a553c790 con 0x5558a5539d40 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f18048c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5558a544ef10 con 0x5558a552f480 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f18048c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5558a553c910 con 0x5558a552eb60 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f18048c3740 10 client.0 _ll_register_callbacks cb 0x5558a5417a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f1800a87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 0x5558a5537680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f1800a87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 0x5558a5537680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46266/0 (socket says 172.21.15.5:46266) 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f1800a87640 1 -- 172.21.15.5:0/3209742635 learned_addr learned my addr 172.21.15.5:0/3209742635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f17fbfff640 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552eb60 0x5558a5536c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.335+0000 7f17f3fff640 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552f480 0x5558a5537140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f1800a87640 1 -- 172.21.15.5:0/3209742635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552eb60 msgr2=0x5558a5536c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f1800a87640 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552eb60 0x5558a5536c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f1800a87640 1 -- 172.21.15.5:0/3209742635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552f480 msgr2=0x5558a5537140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f1800a87640 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552f480 0x5558a5537140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f1800a87640 1 -- 172.21.15.5:0/3209742635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a52056a0 con 0x5558a5539d40 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f17fbfff640 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552eb60 0x5558a5536c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f1800a87640 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 0x5558a5537680 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009bf0 tx=0x7f17f4031cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d714a50bbc704694 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f17db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f17db7fe640 1 -- 172.21.15.5:0/3209742635 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f404a040 con 0x5558a5539d40 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f18048c3740 1 -- 172.21.15.5:0/3209742635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f4036030 con 0x5558a5539d40 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f17db7fe640 1 -- 172.21.15.5:0/3209742635 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7f17f4003970 con 0x5558a5539d40 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f17db7fe640 1 -- 172.21.15.5:0/3209742635 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f17f4003bb0 con 0x5558a5539d40 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f18048c3740 10 client.17310 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.336+0000 7f18048c3740 1 -- 172.21.15.5:0/3209742635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5558a553c790 con 0x5558a5539d40 2024-09-19T06:10:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.337+0000 7f17db7fe640 1 -- 172.21.15.5:0/3209742635 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17f4036030 con 0x5558a5539d40 2024-09-19T06:10:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.337+0000 7f17db7fe640 1 -- 172.21.15.5:0/3209742635 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f17f404d070 con 0x5558a5539d40 2024-09-19T06:10:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.337+0000 7f18048c3740 10 client.17310 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.337+0000 7f18048c3740 10 client.17310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.337+0000 7f18048c3740 20 client.17310 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.337+0000 7f18048c3740 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5558a55494f0 0x5558a554b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.337+0000 7f18048c3740 4 client.17310 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.337+0000 7f18048c3740 1 -- 172.21.15.5:0/3209742635 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558a54a17f0 con 0x5558a55494f0 2024-09-19T06:10:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.337+0000 7f17fbfff640 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5558a55494f0 0x5558a554b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.337+0000 7f17fbfff640 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5558a55494f0 0x5558a554b8e0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a0021e23584834 server_cookie=ea950a3213504b36 in_seq=0 out_seq=0 2024-09-19T06:10:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.337+0000 7f17db7fe640 10 client.17310 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.338+0000 7f17db7fe640 1 -- 172.21.15.5:0/3209742635 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558a54a17f0 con 0x5558a55494f0 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.338+0000 7f17db7fe640 10 client.17310 handle_command_reply: tid=0 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.338+0000 7f18048c3740 1 client.17310 shutdown 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.338+0000 7f18048c3740 1 -- 172.21.15.5:0/3209742635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 msgr2=0x5558a5537680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.338+0000 7f18048c3740 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 0x5558a5537680 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009bf0 tx=0x7f17f4031cf0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.339+0000 7f18048c3740 1 -- 172.21.15.5:0/3209742635 shutdown_connections 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.339+0000 7f18048c3740 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5558a55494f0 0x5558a554b8e0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.339+0000 7f18048c3740 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 0x5558a5537680 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.339+0000 7f18048c3740 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552f480 0x5558a5537140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.339+0000 7f18048c3740 1 --2- 172.21.15.5:0/3209742635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552eb60 0x5558a5536c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.339+0000 7f18048c3740 1 -- 172.21.15.5:0/3209742635 >> 172.21.15.5:0/3209742635 conn(0x5558a552b0f0 msgr2=0x5558a552a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.339+0000 7f17db7fe640 0 client.17310 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.339+0000 7f18048c3740 1 -- 172.21.15.5:0/3209742635 shutdown_connections 2024-09-19T06:10:36.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.339+0000 7f18048c3740 1 -- 172.21.15.5:0/3209742635 wait complete. 2024-09-19T06:10:36.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.339+0000 7f18048c3740 20 client.17310 trim_cache size 0 max 16384 2024-09-19T06:10:36.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.358+0000 7f18048c3740 1 Processor -- start 2024-09-19T06:10:36.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f18048c3740 1 -- start start 2024-09-19T06:10:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f18048c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 0x5558a55315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f18048c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552f480 0x5558a5538e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f18048c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a5539d40 0x5558a5536160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f18048c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1af900 con 0x5558a552eb60 2024-09-19T06:10:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f18048c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1b1440 con 0x5558a5539d40 2024-09-19T06:10:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f18048c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1affd0 con 0x5558a552f480 2024-09-19T06:10:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 0x5558a55315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 0x5558a55315c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46268/0 (socket says 172.21.15.5:46268) 2024-09-19T06:10:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f17fbfff640 1 -- 172.21.15.5:0/2416636895 learned_addr learned my addr 172.21.15.5:0/2416636895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f17f3fff640 1 --2- 172.21.15.5:0/2416636895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552f480 0x5558a5538e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f1800a87640 1 --2- 172.21.15.5:0/2416636895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a5539d40 0x5558a5536160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f17fbfff640 1 -- 172.21.15.5:0/2416636895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552f480 msgr2=0x5558a5538e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:36.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f17fbfff640 1 --2- 172.21.15.5:0/2416636895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552f480 0x5558a5538e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f17fbfff640 1 -- 172.21.15.5:0/2416636895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a5539d40 msgr2=0x5558a5536160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:36.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f17fbfff640 1 --2- 172.21.15.5:0/2416636895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a5539d40 0x5558a5536160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.359+0000 7f17fbfff640 1 -- 172.21.15.5:0/2416636895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a552f2b0 con 0x5558a552eb60 2024-09-19T06:10:36.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.360+0000 7f17fbfff640 1 --2- 172.21.15.5:0/2416636895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 0x5558a55315c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009a10 tx=0x7f17ec01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223530671b255929 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:36.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.360+0000 7f17f1ffb640 1 -- 172.21.15.5:0/2416636895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec005020 con 0x5558a552eb60 2024-09-19T06:10:36.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.360+0000 7f17f1ffb640 1 -- 172.21.15.5:0/2416636895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17ec0047a0 con 0x5558a552eb60 2024-09-19T06:10:36.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.360+0000 7f17f1ffb640 1 -- 172.21.15.5:0/2416636895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec02b860 con 0x5558a552eb60 2024-09-19T06:10:36.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.361+0000 7f18048c3740 1 -- 172.21.15.5:0/2416636895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 msgr2=0x5558a55315c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:36.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.361+0000 7f18048c3740 1 --2- 172.21.15.5:0/2416636895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 0x5558a55315c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009a10 tx=0x7f17ec01b000 comp rx=0 tx=0).stop 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.361+0000 7f18048c3740 1 -- 172.21.15.5:0/2416636895 shutdown_connections 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.361+0000 7f18048c3740 1 --2- 172.21.15.5:0/2416636895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a5539d40 0x5558a5536160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.361+0000 7f18048c3740 1 --2- 172.21.15.5:0/2416636895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552f480 0x5558a5538e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.361+0000 7f18048c3740 1 --2- 172.21.15.5:0/2416636895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a552eb60 0x5558a55315c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.361+0000 7f18048c3740 1 -- 172.21.15.5:0/2416636895 >> 172.21.15.5:0/2416636895 conn(0x5558a552b0f0 msgr2=0x5558a552a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.361+0000 7f18048c3740 1 -- 172.21.15.5:0/2416636895 shutdown_connections 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.361+0000 7f18048c3740 1 -- 172.21.15.5:0/2416636895 wait complete. 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.361+0000 7f18048c3740 1 -- start start 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.361+0000 7f18048c3740 1 Processor -- start 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f18048c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552eb60 0x5558a5542080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f18048c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552f480 0x5558a553d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f18048c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 0x5558a553d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f18048c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1b0d70 con 0x5558a5539d40 2024-09-19T06:10:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f18048c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1b06a0 con 0x5558a552eb60 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f18048c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1ad900 con 0x5558a552f480 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f18048c3740 10 client.0 _ll_register_callbacks cb 0x7f17ec0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552eb60 0x5558a5542080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f17f3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552f480 0x5558a553d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552eb60 0x5558a5542080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56798/0 (socket says 172.21.15.5:56798) 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f17fbfff640 1 -- 172.21.15.5:0/731862110 learned_addr learned my addr 172.21.15.5:0/731862110 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f1800a87640 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 0x5558a553d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f17f3fff640 1 -- 172.21.15.5:0/731862110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552eb60 msgr2=0x5558a5542080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f17f3fff640 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552eb60 0x5558a5542080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f17f3fff640 1 -- 172.21.15.5:0/731862110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 msgr2=0x5558a553d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f17f3fff640 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 0x5558a553d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f17f3fff640 1 -- 172.21.15.5:0/731862110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a553df00 con 0x5558a552f480 2024-09-19T06:10:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f17fbfff640 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552eb60 0x5558a5542080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f1800a87640 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 0x5558a553d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f17f3fff640 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552f480 0x5558a553d400 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f17e40ccfb0 tx=0x7f17e409b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54ac4d832579ce51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.362+0000 7f17db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f17db7fe640 1 -- 172.21.15.5:0/731862110 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4004220 con 0x5558a552f480 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f17db7fe640 1 -- 172.21.15.5:0/731862110 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7f17e40981b0 con 0x5558a552f480 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f18048c3740 1 -- 172.21.15.5:0/731862110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ec0047a0 con 0x5558a552f480 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f17db7fe640 1 -- 172.21.15.5:0/731862110 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f17e409bb40 con 0x5558a552f480 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f18048c3740 10 client.26809 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f18048c3740 1 -- 172.21.15.5:0/731862110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5558a55b3ff0 con 0x5558a552f480 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f17db7fe640 1 -- 172.21.15.5:0/731862110 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e409fef0 con 0x5558a552f480 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f17db7fe640 1 -- 172.21.15.5:0/731862110 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f17e40a43b0 con 0x5558a552f480 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f18048c3740 10 client.26809 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f18048c3740 10 client.26809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f18048c3740 20 client.26809 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f18048c3740 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5558a55b4720 0x5558a55b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f18048c3740 4 client.26809 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f18048c3740 1 -- 172.21.15.5:0/731862110 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f17ec0047a0 con 0x5558a55b4720 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.363+0000 7f17fbfff640 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5558a55b4720 0x5558a55b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.364+0000 7f17fbfff640 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5558a55b4720 0x5558a55b6b40 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d95a778e1f57ea22 server_cookie=d996a773fd403336 in_seq=0 out_seq=0 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.364+0000 7f17db7fe640 10 client.26809 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f17db7fe640 1 -- 172.21.15.5:0/731862110 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f17ec0047a0 con 0x5558a55b4720 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f17db7fe640 10 client.26809 handle_command_reply: tid=0 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f18048c3740 1 client.26809 shutdown 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f18048c3740 1 -- 172.21.15.5:0/731862110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552f480 msgr2=0x5558a553d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f18048c3740 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552f480 0x5558a553d400 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f17e40ccfb0 tx=0x7f17e409b9d0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f18048c3740 1 -- 172.21.15.5:0/731862110 shutdown_connections 2024-09-19T06:10:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f18048c3740 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5558a55b4720 0x5558a55b6b40 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f18048c3740 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558a5539d40 0x5558a553d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f18048c3740 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558a552f480 0x5558a553d400 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f18048c3740 1 --2- 172.21.15.5:0/731862110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5558a552eb60 0x5558a5542080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f17db7fe640 0 client.26809 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f18048c3740 1 -- 172.21.15.5:0/731862110 >> 172.21.15.5:0/731862110 conn(0x5558a552b0f0 msgr2=0x5558a552a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f18048c3740 1 -- 172.21.15.5:0/731862110 shutdown_connections 2024-09-19T06:10:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.365+0000 7f18048c3740 1 -- 172.21.15.5:0/731862110 wait complete. 2024-09-19T06:10:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18048c3740 20 client.26809 trim_cache size 0 max 16384 2024-09-19T06:10:36.373 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:10:36.374 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:10:36.374 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "614c5ebf-975b-43e2-9d59-4b3c5268f6ab", 2024-09-19T06:10:36.374 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:10:36.374 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:10:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18024ea640 1 -- 172.21.15.5:0/3848941312 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f17e80a3430 msgr2=0x7f17e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18024ea640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f17e80a3430 0x7f17e80a58f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5558a552bf50 tx=0x7f17ec005f90 comp rx=0 tx=0).stop 2024-09-19T06:10:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18024ea640 1 -- 172.21.15.5:0/3848941312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17fc100aa0 msgr2=0x7f17fc101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18024ea640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17fc100aa0 0x7f17fc101820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f17e400d720 tx=0x7f17e400dbe0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18024ea640 1 -- 172.21.15.5:0/3848941312 shutdown_connections 2024-09-19T06:10:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18024ea640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc106b80 0x7f17fc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18024ea640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17fc100aa0 0x7f17fc101820 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18024ea640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f17e80a3430 0x7f17e80a58f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18024ea640 1 --2- 172.21.15.5:0/3848941312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc100100 0x7f17fc1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18024ea640 1 -- 172.21.15.5:0/3848941312 >> 172.21.15.5:0/3848941312 conn(0x7f17fc077230 msgr2=0x7f17fc104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18024ea640 1 -- 172.21.15.5:0/3848941312 shutdown_connections 2024-09-19T06:10:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:36.366+0000 7f18024ea640 1 -- 172.21.15.5:0/3848941312 wait complete. 2024-09-19T06:10:36.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:36 smithi005 ceph-mon[30955]: pgmap v223: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T06:10:36.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:36 smithi049 ceph-mon[34640]: pgmap v223: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T06:10:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:36 smithi157 ceph-mon[35346]: pgmap v223: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T06:10:36.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 184616 2.73 MB/sec execute 242 sec latency 68.087 ms 2024-09-19T06:10:37.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:10:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:10:37.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:10:37.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:10:37.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:10:37.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:10:37.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:10:37.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:10:37.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:10:37.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:10:37.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:10:37.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:10:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:10:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:10:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:10:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:10:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:10:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:10:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:10:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:10:37.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 184896 2.72 MB/sec execute 243 sec latency 15.716 ms 2024-09-19T06:10:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.040+0000 7fad2826b640 1 -- 172.21.15.5:0/3476353697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad20106f20 msgr2=0x7fad2010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.040+0000 7fad2826b640 1 --2- 172.21.15.5:0/3476353697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad20106f20 0x7fad2010eaa0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fad18009900 tx=0x7fad1802a410 comp rx=0 tx=0).stop 2024-09-19T06:10:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.040+0000 7fad2826b640 1 -- 172.21.15.5:0/3476353697 shutdown_connections 2024-09-19T06:10:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.040+0000 7fad2826b640 1 --2- 172.21.15.5:0/3476353697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad20106f20 0x7fad2010eaa0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.040+0000 7fad2826b640 1 --2- 172.21.15.5:0/3476353697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad201027a0 0x7fad201067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.040+0000 7fad2826b640 1 --2- 172.21.15.5:0/3476353697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad20101df0 0x7fad201021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.040+0000 7fad2826b640 1 -- 172.21.15.5:0/3476353697 >> 172.21.15.5:0/3476353697 conn(0x7fad200fd8f0 msgr2=0x7fad200ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.040+0000 7fad2826b640 1 -- 172.21.15.5:0/3476353697 shutdown_connections 2024-09-19T06:10:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.040+0000 7fad2826b640 1 -- 172.21.15.5:0/3476353697 wait complete. 2024-09-19T06:10:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad2826b640 1 Processor -- start 2024-09-19T06:10:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad2826b640 1 -- start start 2024-09-19T06:10:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad2826b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad20101df0 0x7fad2019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad2826b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad201027a0 0x7fad2019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad2826b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad20106f20 0x7fad2019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad2826b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad20111e30 con 0x7fad20101df0 2024-09-19T06:10:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad2826b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fad20111cb0 con 0x7fad20106f20 2024-09-19T06:10:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad2826b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad20111fb0 con 0x7fad201027a0 2024-09-19T06:10:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad26007640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad20101df0 0x7fad2019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad26007640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad20101df0 0x7fad2019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46294/0 (socket says 172.21.15.5:46294) 2024-09-19T06:10:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad26007640 1 -- 172.21.15.5:0/2187778924 learned_addr learned my addr 172.21.15.5:0/2187778924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad26808640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad20106f20 0x7fad2019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.041+0000 7fad25806640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad201027a0 0x7fad2019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.042+0000 7fad26007640 1 -- 172.21.15.5:0/2187778924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad201027a0 msgr2=0x7fad2019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.042+0000 7fad26007640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad201027a0 0x7fad2019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.042+0000 7fad26007640 1 -- 172.21.15.5:0/2187778924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad20106f20 msgr2=0x7fad2019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.042+0000 7fad26007640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad20106f20 0x7fad2019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.042+0000 7fad26007640 1 -- 172.21.15.5:0/2187778924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2019f560 con 0x7fad20101df0 2024-09-19T06:10:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.042+0000 7fad26007640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad20101df0 0x7fad2019a5b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00eb90 tx=0x7fad1c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.042+0000 7fad177fe640 1 -- 172.21.15.5:0/2187778924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c004d20 con 0x7fad20101df0 2024-09-19T06:10:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.042+0000 7fad177fe640 1 -- 172.21.15.5:0/2187778924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad1c00ee90 con 0x7fad20101df0 2024-09-19T06:10:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.042+0000 7fad2826b640 1 -- 172.21.15.5:0/2187778924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2019f850 con 0x7fad20101df0 2024-09-19T06:10:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.042+0000 7fad177fe640 1 -- 172.21.15.5:0/2187778924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c010670 con 0x7fad20101df0 2024-09-19T06:10:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.042+0000 7fad2826b640 1 -- 172.21.15.5:0/2187778924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad201a7130 con 0x7fad20101df0 2024-09-19T06:10:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.045+0000 7fad177fe640 1 -- 172.21.15.5:0/2187778924 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fad1c005020 con 0x7fad20101df0 2024-09-19T06:10:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.045+0000 7fad2a644740 1 Processor -- start 2024-09-19T06:10:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad2a644740 1 -- start start 2024-09-19T06:10:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad2a644740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d044a0 0x555899d04880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad2a644740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d04dc0 0x555899d0f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad2a644740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d0f680 0x555899d11a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad2a644740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555899c6a2e0 con 0x555899d0f680 2024-09-19T06:10:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad2a644740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555899c11ce0 con 0x555899d04dc0 2024-09-19T06:10:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad2a644740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555899c662a0 con 0x555899d044a0 2024-09-19T06:10:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad26007640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d04dc0 0x555899d0f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad25806640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d0f680 0x555899d11a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad25806640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d0f680 0x555899d11a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46304/0 (socket says 172.21.15.5:46304) 2024-09-19T06:10:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad25806640 1 -- 172.21.15.5:0/3790892439 learned_addr learned my addr 172.21.15.5:0/3790892439 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad177fe640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fad0c0a3430 0x7fad0c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.046+0000 7fad26808640 1 --2- 172.21.15.5:0/3790892439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d044a0 0x555899d04880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad26808640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fad0c0a3430 0x7fad0c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad177fe640 1 -- 172.21.15.5:0/2187778924 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fad1c0d0310 con 0x7fad20101df0 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad25806640 1 -- 172.21.15.5:0/3790892439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d044a0 msgr2=0x555899d04880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad25806640 1 --2- 172.21.15.5:0/3790892439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d044a0 0x555899d04880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad25806640 1 -- 172.21.15.5:0/3790892439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d04dc0 msgr2=0x555899d0f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad25806640 1 --2- 172.21.15.5:0/3790892439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d04dc0 0x555899d0f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad25806640 1 -- 172.21.15.5:0/3790892439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555899bf6440 con 0x555899d0f680 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad26007640 1 --2- 172.21.15.5:0/3790892439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d04dc0 0x555899d0f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad25806640 1 --2- 172.21.15.5:0/3790892439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d0f680 0x555899d11a70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fad10009c10 tx=0x7fad10011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540e351091442516 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad14dea640 1 -- 172.21.15.5:0/3790892439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10004230 con 0x555899d0f680 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad14dea640 1 -- 172.21.15.5:0/3790892439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad100043d0 con 0x555899d0f680 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad26808640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fad0c0a3430 0x7fad0c0a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fad18006010 tx=0x7fad1802a410 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.047+0000 7fad14dea640 1 -- 172.21.15.5:0/3790892439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10019660 con 0x555899d0f680 2024-09-19T06:10:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.048+0000 7fad2a644740 1 -- 172.21.15.5:0/3790892439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d0f680 msgr2=0x555899d11a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.048+0000 7fad2a644740 1 --2- 172.21.15.5:0/3790892439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d0f680 0x555899d11a70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fad10009c10 tx=0x7fad10011810 comp rx=0 tx=0).stop 2024-09-19T06:10:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.048+0000 7fad2a644740 1 -- 172.21.15.5:0/3790892439 shutdown_connections 2024-09-19T06:10:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.048+0000 7fad2a644740 1 --2- 172.21.15.5:0/3790892439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d0f680 0x555899d11a70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.048+0000 7fad2a644740 1 --2- 172.21.15.5:0/3790892439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d04dc0 0x555899d0f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.048+0000 7fad2a644740 1 --2- 172.21.15.5:0/3790892439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d044a0 0x555899d04880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.048+0000 7fad2a644740 1 -- 172.21.15.5:0/3790892439 >> 172.21.15.5:0/3790892439 conn(0x555899d00a30 msgr2=0x555899d00e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.048+0000 7fad2a644740 1 -- 172.21.15.5:0/3790892439 shutdown_connections 2024-09-19T06:10:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.048+0000 7fad2a644740 1 -- 172.21.15.5:0/3790892439 wait complete. 2024-09-19T06:10:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad2a644740 1 -- start start 2024-09-19T06:10:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad2a644740 1 Processor -- start 2024-09-19T06:10:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad2a644740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d044a0 0x555899d0efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad2a644740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d04dc0 0x555899d0c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad2a644740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d0f680 0x555899d0c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad2a644740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555899d12250 con 0x555899d044a0 2024-09-19T06:10:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad2a644740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555899c25050 con 0x555899d0f680 2024-09-19T06:10:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad2a644740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555899d123d0 con 0x555899d04dc0 2024-09-19T06:10:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad2a644740 10 client.0 _ll_register_callbacks cb 0x555899c65160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad26808640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d0f680 0x555899d0c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad26808640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d0f680 0x555899d0c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56842/0 (socket says 172.21.15.5:56842) 2024-09-19T06:10:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad26007640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d044a0 0x555899d0efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad26808640 1 -- 172.21.15.5:0/3672296849 learned_addr learned my addr 172.21.15.5:0/3672296849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.049+0000 7fad25806640 1 --2- 172.21.15.5:0/3672296849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d04dc0 0x555899d0c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad26808640 1 -- 172.21.15.5:0/3672296849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d04dc0 msgr2=0x555899d0c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad26808640 1 --2- 172.21.15.5:0/3672296849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d04dc0 0x555899d0c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad26808640 1 -- 172.21.15.5:0/3672296849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d044a0 msgr2=0x555899d0efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad26808640 1 --2- 172.21.15.5:0/3672296849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d044a0 0x555899d0efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad26808640 1 -- 172.21.15.5:0/3672296849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558999db3c0 con 0x555899d0f680 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad26808640 1 --2- 172.21.15.5:0/3672296849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d0f680 0x555899d0c790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad18033000 tx=0x7fad18009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ddf80c9418d19d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad09ffb640 1 -- 172.21.15.5:0/3672296849 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18050060 con 0x555899d0f680 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad09ffb640 1 -- 172.21.15.5:0/3672296849 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fad18004320 con 0x555899d0f680 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad2a644740 1 -- 172.21.15.5:0/3672296849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad100043d0 con 0x555899d0f680 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad09ffb640 1 -- 172.21.15.5:0/3672296849 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fad18035040 con 0x555899d0f680 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad2a644740 10 client.26900 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.050+0000 7fad2a644740 1 -- 172.21.15.5:0/3672296849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555899d1dbd0 con 0x555899d0f680 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.051+0000 7fad09ffb640 1 -- 172.21.15.5:0/3672296849 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad100043d0 con 0x555899d0f680 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.051+0000 7fad09ffb640 1 -- 172.21.15.5:0/3672296849 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fad18004b50 con 0x555899d0f680 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.051+0000 7fad2a644740 10 client.26900 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.051+0000 7fad2a644740 10 client.26900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.051+0000 7fad2a644740 20 client.26900 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.051+0000 7fad2a644740 1 --2- 172.21.15.5:0/3672296849 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555899d1f090 0x555899d21500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.051+0000 7fad2a644740 4 client.26900 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.051+0000 7fad2a644740 1 -- 172.21.15.5:0/3672296849 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555899c77850 con 0x555899d1f090 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.051+0000 7fad26007640 1 --2- 172.21.15.5:0/3672296849 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555899d1f090 0x555899d21500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.052+0000 7fad26007640 1 --2- 172.21.15.5:0/3672296849 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555899d1f090 0x555899d21500 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991e353cb7fc033d server_cookie=a5217a5edd4e7163 in_seq=0 out_seq=0 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.052+0000 7fad09ffb640 10 client.26900 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.052+0000 7fad09ffb640 1 -- 172.21.15.5:0/3672296849 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555899c77850 con 0x555899d1f090 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.052+0000 7fad09ffb640 10 client.26900 handle_command_reply: tid=0 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.052+0000 7fad2a644740 1 client.26900 shutdown 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.053+0000 7fad2a644740 1 -- 172.21.15.5:0/3672296849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d0f680 msgr2=0x555899d0c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.053+0000 7fad2a644740 1 --2- 172.21.15.5:0/3672296849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d0f680 0x555899d0c790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad18033000 tx=0x7fad18009260 comp rx=0 tx=0).stop 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.053+0000 7fad2a644740 1 -- 172.21.15.5:0/3672296849 shutdown_connections 2024-09-19T06:10:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.053+0000 7fad2a644740 1 --2- 172.21.15.5:0/3672296849 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555899d1f090 0x555899d21500 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.053+0000 7fad2a644740 1 --2- 172.21.15.5:0/3672296849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d0f680 0x555899d0c790 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.053+0000 7fad2a644740 1 --2- 172.21.15.5:0/3672296849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d04dc0 0x555899d0c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.053+0000 7fad09ffb640 0 client.26900 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:38.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.053+0000 7fad2a644740 1 --2- 172.21.15.5:0/3672296849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d044a0 0x555899d0efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.053+0000 7fad2a644740 1 -- 172.21.15.5:0/3672296849 >> 172.21.15.5:0/3672296849 conn(0x555899d00a30 msgr2=0x555899d06710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:38.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.053+0000 7fad2a644740 1 -- 172.21.15.5:0/3672296849 shutdown_connections 2024-09-19T06:10:38.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.053+0000 7fad2a644740 1 -- 172.21.15.5:0/3672296849 wait complete. 2024-09-19T06:10:38.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.053+0000 7fad2a644740 20 client.26900 trim_cache size 0 max 16384 2024-09-19T06:10:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.072+0000 7fad2a644740 1 Processor -- start 2024-09-19T06:10:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.072+0000 7fad2a644740 1 -- start start 2024-09-19T06:10:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.072+0000 7fad2a644740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d00a30 0x555899d1b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad2a644740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d044a0 0x555899d0c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad2a644740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d04dc0 0x555899d077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad2a644740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad201afaa0 con 0x555899d00a30 2024-09-19T06:10:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad2a644740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fad201b15e0 con 0x555899d04dc0 2024-09-19T06:10:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad2a644740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad201b0170 con 0x555899d044a0 2024-09-19T06:10:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad26007640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d00a30 0x555899d1b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad26808640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d04dc0 0x555899d077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad26007640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d00a30 0x555899d1b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46314/0 (socket says 172.21.15.5:46314) 2024-09-19T06:10:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad26007640 1 -- 172.21.15.5:0/2734261320 learned_addr learned my addr 172.21.15.5:0/2734261320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad25806640 1 --2- 172.21.15.5:0/2734261320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d044a0 0x555899d0c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad26808640 1 -- 172.21.15.5:0/2734261320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d044a0 msgr2=0x555899d0c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad26808640 1 --2- 172.21.15.5:0/2734261320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d044a0 0x555899d0c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad26808640 1 -- 172.21.15.5:0/2734261320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d00a30 msgr2=0x555899d1b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad26808640 1 --2- 172.21.15.5:0/2734261320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d00a30 0x555899d1b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad26808640 1 -- 172.21.15.5:0/2734261320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555899d12130 con 0x555899d04dc0 2024-09-19T06:10:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad26007640 1 --2- 172.21.15.5:0/2734261320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d00a30 0x555899d1b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad26808640 1 --2- 172.21.15.5:0/2734261320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d04dc0 0x555899d077f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fad1802a930 tx=0x7fad18032b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0519ed69be365bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad0bfff640 1 -- 172.21.15.5:0/2734261320 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18050070 con 0x555899d04dc0 2024-09-19T06:10:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.073+0000 7fad0bfff640 1 -- 172.21.15.5:0/2734261320 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad180317e0 con 0x555899d04dc0 2024-09-19T06:10:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.074+0000 7fad0bfff640 1 -- 172.21.15.5:0/2734261320 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18004920 con 0x555899d04dc0 2024-09-19T06:10:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.074+0000 7fad2a644740 1 -- 172.21.15.5:0/2734261320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d04dc0 msgr2=0x555899d077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.074+0000 7fad2a644740 1 --2- 172.21.15.5:0/2734261320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d04dc0 0x555899d077f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fad1802a930 tx=0x7fad18032b80 comp rx=0 tx=0).stop 2024-09-19T06:10:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.074+0000 7fad2a644740 1 -- 172.21.15.5:0/2734261320 shutdown_connections 2024-09-19T06:10:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.074+0000 7fad2a644740 1 --2- 172.21.15.5:0/2734261320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d04dc0 0x555899d077f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.074+0000 7fad2a644740 1 --2- 172.21.15.5:0/2734261320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d044a0 0x555899d0c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.074+0000 7fad2a644740 1 --2- 172.21.15.5:0/2734261320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d00a30 0x555899d1b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.074+0000 7fad2a644740 1 -- 172.21.15.5:0/2734261320 >> 172.21.15.5:0/2734261320 conn(0x555899d0c8f0 msgr2=0x555899d070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.074+0000 7fad2a644740 1 -- 172.21.15.5:0/2734261320 shutdown_connections 2024-09-19T06:10:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.074+0000 7fad2a644740 1 -- 172.21.15.5:0/2734261320 wait complete. 2024-09-19T06:10:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad2a644740 1 -- start start 2024-09-19T06:10:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad2a644740 1 Processor -- start 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad2a644740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d00a30 0x555899d13460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad2a644740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d044a0 0x555899d139a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad2a644740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d04dc0 0x555899d13ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad2a644740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad201b0f10 con 0x555899d04dc0 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad2a644740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fad201b0840 con 0x555899d044a0 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad2a644740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad201adaa0 con 0x555899d00a30 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad2a644740 10 client.0 _ll_register_callbacks cb 0x7fad18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad26808640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d04dc0 0x555899d13ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad26808640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d04dc0 0x555899d13ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46330/0 (socket says 172.21.15.5:46330) 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad26808640 1 -- 172.21.15.5:0/3380851580 learned_addr learned my addr 172.21.15.5:0/3380851580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad25806640 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d044a0 0x555899d139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.075+0000 7fad26007640 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d00a30 0x555899d13460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad26808640 1 -- 172.21.15.5:0/3380851580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d00a30 msgr2=0x555899d13460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad26808640 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d00a30 0x555899d13460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad26808640 1 -- 172.21.15.5:0/3380851580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d044a0 msgr2=0x555899d139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad26808640 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d044a0 0x555899d139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad26808640 1 -- 172.21.15.5:0/3380851580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555899d88e50 con 0x555899d04dc0 2024-09-19T06:10:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad25806640 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d044a0 0x555899d139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad26808640 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d04dc0 0x555899d13ee0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fad18005e20 tx=0x7fad18031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5151d9c265fdc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:10:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad09ffb640 1 -- 172.21.15.5:0/3380851580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18063060 con 0x555899d04dc0 2024-09-19T06:10:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad09ffb640 1 -- 172.21.15.5:0/3380851580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fad180046c0 con 0x555899d04dc0 2024-09-19T06:10:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad2a644740 1 -- 172.21.15.5:0/3380851580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad180317e0 con 0x555899d04dc0 2024-09-19T06:10:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad09ffb640 1 -- 172.21.15.5:0/3380851580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fad18050030 con 0x555899d04dc0 2024-09-19T06:10:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad2a644740 10 client.17334 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.076+0000 7fad2a644740 1 -- 172.21.15.5:0/3380851580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555899d8a120 con 0x555899d04dc0 2024-09-19T06:10:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.077+0000 7fad09ffb640 1 -- 172.21.15.5:0/3380851580 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad180317e0 con 0x555899d04dc0 2024-09-19T06:10:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.077+0000 7fad09ffb640 1 -- 172.21.15.5:0/3380851580 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fad1805c070 con 0x555899d04dc0 2024-09-19T06:10:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.077+0000 7fad2a644740 10 client.17334 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.077+0000 7fad2a644740 10 client.17334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.077+0000 7fad2a644740 20 client.17334 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.077+0000 7fad2a644740 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555899d0f680 0x555899d8c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.077+0000 7fad2a644740 4 client.17334 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:10:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.077+0000 7fad2a644740 1 -- 172.21.15.5:0/3380851580 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555899c71d60 con 0x555899d0f680 2024-09-19T06:10:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.077+0000 7fad26007640 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555899d0f680 0x555899d8c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.078+0000 7fad26007640 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555899d0f680 0x555899d8c710 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0518263cd4bf105 server_cookie=60ca7c20180decc7 in_seq=0 out_seq=0 2024-09-19T06:10:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.078+0000 7fad09ffb640 10 client.17334 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.078+0000 7fad09ffb640 1 -- 172.21.15.5:0/3380851580 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555899c71d60 con 0x555899d0f680 2024-09-19T06:10:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.078+0000 7fad09ffb640 10 client.17334 handle_command_reply: tid=0 2024-09-19T06:10:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.078+0000 7fad2a644740 1 client.17334 shutdown 2024-09-19T06:10:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2a644740 1 -- 172.21.15.5:0/3380851580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d04dc0 msgr2=0x555899d13ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2a644740 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d04dc0 0x555899d13ee0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fad18005e20 tx=0x7fad18031510 comp rx=0 tx=0).stop 2024-09-19T06:10:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2a644740 1 -- 172.21.15.5:0/3380851580 shutdown_connections 2024-09-19T06:10:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2a644740 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555899d0f680 0x555899d8c710 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2a644740 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555899d04dc0 0x555899d13ee0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2a644740 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555899d044a0 0x555899d139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2a644740 1 --2- 172.21.15.5:0/3380851580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555899d00a30 0x555899d13460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2a644740 1 -- 172.21.15.5:0/3380851580 >> 172.21.15.5:0/3380851580 conn(0x555899d0c8f0 msgr2=0x555899cffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad09ffb640 0 client.17334 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2a644740 1 -- 172.21.15.5:0/3380851580 shutdown_connections 2024-09-19T06:10:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2a644740 1 -- 172.21.15.5:0/3380851580 wait complete. 2024-09-19T06:10:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2a644740 20 client.17334 trim_cache size 0 max 16384 2024-09-19T06:10:38.086 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:10:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:10:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:10:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "614c5ebf-975b-43e2-9d59-4b3c5268f6ab": { 2024-09-19T06:10:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:10:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "614c5ebf-975b-43e2-9d59-4b3c5268f6ab", 2024-09-19T06:10:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:10:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:10:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:10:38.087 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:10:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2826b640 1 -- 172.21.15.5:0/2187778924 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fad0c0a3430 msgr2=0x7fad0c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2826b640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fad0c0a3430 0x7fad0c0a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fad18006010 tx=0x7fad1802a410 comp rx=0 tx=0).stop 2024-09-19T06:10:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2826b640 1 -- 172.21.15.5:0/2187778924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad20101df0 msgr2=0x7fad2019a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.079+0000 7fad2826b640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad20101df0 0x7fad2019a5b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00eb90 tx=0x7fad1c00c6a0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.080+0000 7fad2826b640 1 -- 172.21.15.5:0/2187778924 shutdown_connections 2024-09-19T06:10:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.080+0000 7fad2826b640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad20106f20 0x7fad2019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.080+0000 7fad2826b640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fad0c0a3430 0x7fad0c0a58f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.080+0000 7fad2826b640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad201027a0 0x7fad2019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.080+0000 7fad2826b640 1 --2- 172.21.15.5:0/2187778924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad20101df0 0x7fad2019a5b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.080+0000 7fad2826b640 1 -- 172.21.15.5:0/2187778924 >> 172.21.15.5:0/2187778924 conn(0x7fad200fd8f0 msgr2=0x7fad200fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.080+0000 7fad2826b640 1 -- 172.21.15.5:0/2187778924 shutdown_connections 2024-09-19T06:10:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:10:38.080+0000 7fad2826b640 1 -- 172.21.15.5:0/2187778924 wait complete. 2024-09-19T06:10:38.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:38 smithi005 ceph-mon[30955]: pgmap v224: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-09-19T06:10:38.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:38 smithi049 ceph-mon[34640]: pgmap v224: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-09-19T06:10:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:38 smithi157 ceph-mon[35346]: pgmap v224: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-09-19T06:10:38.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 185280 2.72 MB/sec execute 244 sec latency 36.247 ms 2024-09-19T06:10:38.963 INFO:tasks.cephfs.filesystem:scrub status for tag:614c5ebf-975b-43e2-9d59-4b3c5268f6ab - {'path': '/', 'tag': '614c5ebf-975b-43e2-9d59-4b3c5268f6ab', 'options': 'recursive,force'} 2024-09-19T06:10:39.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 185850 2.72 MB/sec execute 245 sec latency 19.586 ms 2024-09-19T06:10:40.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:40 smithi005 ceph-mon[30955]: pgmap v225: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-09-19T06:10:40.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:40 smithi049 ceph-mon[34640]: pgmap v225: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-09-19T06:10:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:40 smithi157 ceph-mon[35346]: pgmap v225: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-09-19T06:10:40.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 186346 2.71 MB/sec execute 246 sec latency 20.663 ms 2024-09-19T06:10:41.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 186875 2.71 MB/sec execute 247 sec latency 21.072 ms 2024-09-19T06:10:42.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:42 smithi005 ceph-mon[30955]: pgmap v226: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-09-19T06:10:42.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:42 smithi049 ceph-mon[34640]: pgmap v226: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-09-19T06:10:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:42 smithi157 ceph-mon[35346]: pgmap v226: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-09-19T06:10:42.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 187620 2.72 MB/sec execute 248 sec latency 14.768 ms 2024-09-19T06:10:43.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 188009 2.73 MB/sec execute 249 sec latency 15.832 ms 2024-09-19T06:10:44.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:44 smithi005 ceph-mon[30955]: pgmap v227: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-09-19T06:10:44.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:10:44.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:44 smithi049 ceph-mon[34640]: pgmap v227: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-09-19T06:10:44.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:10:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:44 smithi157 ceph-mon[35346]: pgmap v227: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-09-19T06:10:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:10:44.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 188368 2.73 MB/sec execute 250 sec latency 15.266 ms 2024-09-19T06:10:45.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:45 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:10:45.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:45 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:10:45.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:45 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:10:45.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:45 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:10:45.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:45 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:10:45.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:45 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:10:45.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 188662 2.71 MB/sec execute 251 sec latency 19.617 ms 2024-09-19T06:10:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.021+0000 7f5e57703640 1 -- 172.21.15.5:0/596871932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e50102d20 msgr2=0x7f5e50103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.021+0000 7f5e57703640 1 --2- 172.21.15.5:0/596871932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e50102d20 0x7f5e50103100 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009900 tx=0x7f5e4c02a480 comp rx=0 tx=0).stop 2024-09-19T06:10:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.022+0000 7f5e57703640 1 -- 172.21.15.5:0/596871932 shutdown_connections 2024-09-19T06:10:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.022+0000 7f5e57703640 1 --2- 172.21.15.5:0/596871932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e5010c0f0 0x7f5e5010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.022+0000 7f5e57703640 1 --2- 172.21.15.5:0/596871932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e50103640 0x7f5e5010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.022+0000 7f5e57703640 1 --2- 172.21.15.5:0/596871932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e50102d20 0x7f5e50103100 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.022+0000 7f5e57703640 1 -- 172.21.15.5:0/596871932 >> 172.21.15.5:0/596871932 conn(0x7f5e500779a0 msgr2=0x7f5e500fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.022+0000 7f5e57703640 1 -- 172.21.15.5:0/596871932 shutdown_connections 2024-09-19T06:10:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.022+0000 7f5e57703640 1 -- 172.21.15.5:0/596871932 wait complete. 2024-09-19T06:10:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.022+0000 7f5e57703640 1 Processor -- start 2024-09-19T06:10:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.022+0000 7f5e57703640 1 -- start start 2024-09-19T06:10:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.022+0000 7f5e57703640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e50102d20 0x7f5e501983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.022+0000 7f5e57703640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e50103640 0x7f5e501988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e57703640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e5010c0f0 0x7f5e5019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e57703640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e50111d50 con 0x7f5e50102d20 2024-09-19T06:10:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e57703640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5e50111bd0 con 0x7f5e5010c0f0 2024-09-19T06:10:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e57703640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5e50111ed0 con 0x7f5e50103640 2024-09-19T06:10:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e5549f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e50102d20 0x7f5e501983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e5549f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e50102d20 0x7f5e501983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42164/0 (socket says 172.21.15.5:42164) 2024-09-19T06:10:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e5549f640 1 -- 172.21.15.5:0/1572874249 learned_addr learned my addr 172.21.15.5:0/1572874249 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e55ca0640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e5010c0f0 0x7f5e5019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e54c9e640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e50103640 0x7f5e501988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e5549f640 1 -- 172.21.15.5:0/1572874249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e50103640 msgr2=0x7f5e501988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e5549f640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e50103640 0x7f5e501988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e5549f640 1 -- 172.21.15.5:0/1572874249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e5010c0f0 msgr2=0x7f5e5019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e5549f640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e5010c0f0 0x7f5e5019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e5549f640 1 -- 172.21.15.5:0/1572874249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5019d390 con 0x7f5e50102d20 2024-09-19T06:10:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e55ca0640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e5010c0f0 0x7f5e5019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e5549f640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e50102d20 0x7f5e501983a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c005940 tx=0x7f5e4c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e467fc640 1 -- 172.21.15.5:0/1572874249 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c03f030 con 0x7f5e50102d20 2024-09-19T06:10:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e57703640 1 -- 172.21.15.5:0/1572874249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5019d620 con 0x7f5e50102d20 2024-09-19T06:10:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e467fc640 1 -- 172.21.15.5:0/1572874249 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c0044c0 con 0x7f5e50102d20 2024-09-19T06:10:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.023+0000 7f5e467fc640 1 -- 172.21.15.5:0/1572874249 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c035070 con 0x7f5e50102d20 2024-09-19T06:10:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.024+0000 7f5e57703640 1 -- 172.21.15.5:0/1572874249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e501a4f10 con 0x7f5e50102d20 2024-09-19T06:10:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.026+0000 7f5e467fc640 1 -- 172.21.15.5:0/1572874249 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5e4c042030 con 0x7f5e50102d20 2024-09-19T06:10:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.026+0000 7f5e5a498740 1 Processor -- start 2024-09-19T06:10:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.026+0000 7f5e5a498740 1 -- start start 2024-09-19T06:10:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5a498740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 0x5615a6d6fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5a498740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d70410 0x5615a6d7a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5a498740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d7acb0 0x5615a6d7d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5a498740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5615a6cd42f0 con 0x5615a6d6faf0 2024-09-19T06:10:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5a498740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5615a6c7c270 con 0x5615a6d70410 2024-09-19T06:10:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5a498740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5615a6cd02b0 con 0x5615a6d7acb0 2024-09-19T06:10:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5549f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d70410 0x5615a6d7a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e55ca0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 0x5615a6d6fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5549f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d70410 0x5615a6d7a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:47236/0 (socket says 172.21.15.5:47236) 2024-09-19T06:10:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5549f640 1 -- 172.21.15.5:0/535627536 learned_addr learned my addr 172.21.15.5:0/535627536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e55ca0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 0x5615a6d6fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42170/0 (socket says 172.21.15.5:42170) 2024-09-19T06:10:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e54c9e640 1 --2- 172.21.15.5:0/535627536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d7acb0 0x5615a6d7d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e467fc640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5e3c0a33b0 0x7f5e3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e55ca0640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5e3c0a33b0 0x7f5e3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5549f640 1 -- 172.21.15.5:0/535627536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d7acb0 msgr2=0x5615a6d7d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5549f640 1 --2- 172.21.15.5:0/535627536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d7acb0 0x5615a6d7d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5549f640 1 -- 172.21.15.5:0/535627536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 msgr2=0x5615a6d6fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5549f640 1 --2- 172.21.15.5:0/535627536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 0x5615a6d6fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e5549f640 1 -- 172.21.15.5:0/535627536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615a6c7b340 con 0x5615a6d70410 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.027+0000 7f5e467fc640 1 -- 172.21.15.5:0/1572874249 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7f5e4c0efd00 con 0x7f5e50102d20 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.028+0000 7f5e5549f640 1 --2- 172.21.15.5:0/535627536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d70410 0x5615a6d7a770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0bbd70 tx=0x7f5e4c02a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6010ab744c8f9650 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.028+0000 7f5e55ca0640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5e3c0a33b0 0x7f5e3c0a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e480094d0 tx=0x7f5e48004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.028+0000 7f5e3b7fe640 1 -- 172.21.15.5:0/535627536 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c049070 con 0x5615a6d70410 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.028+0000 7f5e3b7fe640 1 -- 172.21.15.5:0/535627536 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c0bc070 con 0x5615a6d70410 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.028+0000 7f5e3b7fe640 1 -- 172.21.15.5:0/535627536 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0ed6b0 con 0x5615a6d70410 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.028+0000 7f5e5a498740 1 -- 172.21.15.5:0/535627536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d70410 msgr2=0x5615a6d7a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.028+0000 7f5e5a498740 1 --2- 172.21.15.5:0/535627536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d70410 0x5615a6d7a770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0bbd70 tx=0x7f5e4c02a480 comp rx=0 tx=0).stop 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 -- 172.21.15.5:0/535627536 shutdown_connections 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 --2- 172.21.15.5:0/535627536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d7acb0 0x5615a6d7d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 --2- 172.21.15.5:0/535627536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d70410 0x5615a6d7a770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 --2- 172.21.15.5:0/535627536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 0x5615a6d6fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 -- 172.21.15.5:0/535627536 >> 172.21.15.5:0/535627536 conn(0x5615a6d6c060 msgr2=0x5615a6d6c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 -- 172.21.15.5:0/535627536 shutdown_connections 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 -- 172.21.15.5:0/535627536 wait complete. 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 -- start start 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 Processor -- start 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 0x5615a6d77bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 0x5615a6d780f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d78630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5615a6d7d640 con 0x5615a6d70410 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5615a6c5d290 con 0x5615a6d7acb0 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.029+0000 7f5e5a498740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5615a6d7d7c0 con 0x5615a6d6faf0 2024-09-19T06:10:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e5a498740 10 client.0 _ll_register_callbacks cb 0x5615a6c8e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e54c9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 0x5615a6d780f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e54c9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 0x5615a6d780f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42184/0 (socket says 172.21.15.5:42184) 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e55ca0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d78630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e54c9e640 1 -- 172.21.15.5:0/2529447748 learned_addr learned my addr 172.21.15.5:0/2529447748 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e5549f640 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 0x5615a6d77bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e55ca0640 1 -- 172.21.15.5:0/2529447748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 msgr2=0x5615a6d77bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e55ca0640 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 0x5615a6d77bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e55ca0640 1 -- 172.21.15.5:0/2529447748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 msgr2=0x5615a6d780f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e55ca0640 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 0x5615a6d780f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e55ca0640 1 -- 172.21.15.5:0/2529447748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615a69e7210 con 0x5615a6d7acb0 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e5549f640 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 0x5615a6d77bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e55ca0640 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d78630 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800ffc0 tx=0x7f5e48004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9652c28bcd7c4635 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:10:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e38ff9640 1 -- 172.21.15.5:0/2529447748 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48026030 con 0x5615a6d7acb0 2024-09-19T06:10:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e38ff9640 1 -- 172.21.15.5:0/2529447748 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7f5e480047e0 con 0x5615a6d7acb0 2024-09-19T06:10:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.030+0000 7f5e5a498740 1 -- 172.21.15.5:0/2529447748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e4c0bc070 con 0x5615a6d7acb0 2024-09-19T06:10:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.031+0000 7f5e38ff9640 1 -- 172.21.15.5:0/2529447748 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5e48030050 con 0x5615a6d7acb0 2024-09-19T06:10:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.031+0000 7f5e5a498740 10 client.26918 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.031+0000 7f5e5a498740 1 -- 172.21.15.5:0/2529447748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5615a6ab3620 con 0x5615a6d7acb0 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.031+0000 7f5e38ff9640 1 -- 172.21.15.5:0/2529447748 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c0bc070 con 0x5615a6d7acb0 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.031+0000 7f5e38ff9640 1 -- 172.21.15.5:0/2529447748 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5e4801f070 con 0x5615a6d7acb0 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.031+0000 7f5e5a498740 10 client.26918 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.031+0000 7f5e5a498740 10 client.26918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.031+0000 7f5e5a498740 20 client.26918 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.031+0000 7f5e5a498740 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5615a6d8aa40 0x5615a6d8ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.031+0000 7f5e5a498740 4 client.26918 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.031+0000 7f5e5a498740 1 -- 172.21.15.5:0/2529447748 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615a6ce1a20 con 0x5615a6d8aa40 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.031+0000 7f5e5549f640 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5615a6d8aa40 0x5615a6d8ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e38ff9640 10 client.26918 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e5549f640 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5615a6d8aa40 0x5615a6d8ce50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba0cf98e6b7fd47 server_cookie=85bb5d3a88e03721 in_seq=0 out_seq=0 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e38ff9640 1 -- 172.21.15.5:0/2529447748 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615a6ce1a20 con 0x5615a6d8aa40 2024-09-19T06:10:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e38ff9640 10 client.26918 handle_command_reply: tid=0 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e5a498740 1 client.26918 shutdown 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e5a498740 1 -- 172.21.15.5:0/2529447748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 msgr2=0x5615a6d78630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e5a498740 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d78630 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800ffc0 tx=0x7f5e48004a40 comp rx=0 tx=0).stop 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e5a498740 1 -- 172.21.15.5:0/2529447748 shutdown_connections 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e5a498740 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5615a6d8aa40 0x5615a6d8ce50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e5a498740 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d78630 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e38ff9640 0 client.26918 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e5a498740 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 0x5615a6d780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e5a498740 1 --2- 172.21.15.5:0/2529447748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 0x5615a6d77bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.032+0000 7f5e5a498740 1 -- 172.21.15.5:0/2529447748 >> 172.21.15.5:0/2529447748 conn(0x5615a6d6c060 msgr2=0x5615a6d71d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.033+0000 7f5e5a498740 1 -- 172.21.15.5:0/2529447748 shutdown_connections 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.033+0000 7f5e5a498740 1 -- 172.21.15.5:0/2529447748 wait complete. 2024-09-19T06:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.033+0000 7f5e5a498740 20 client.26918 trim_cache size 0 max 16384 2024-09-19T06:10:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e5a498740 1 Processor -- start 2024-09-19T06:10:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e5a498740 1 -- start start 2024-09-19T06:10:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e5a498740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 0x5615a6d72560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e5a498740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d70410 0x5615a6d78460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e5a498740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d92760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e5a498740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e501ad860 con 0x5615a6d6faf0 2024-09-19T06:10:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e5a498740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5e501af3a0 con 0x5615a6d7acb0 2024-09-19T06:10:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e5a498740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5e501adf30 con 0x5615a6d70410 2024-09-19T06:10:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e5549f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 0x5615a6d72560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e54c9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d70410 0x5615a6d78460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e5549f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 0x5615a6d72560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42198/0 (socket says 172.21.15.5:42198) 2024-09-19T06:10:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e5549f640 1 -- 172.21.15.5:0/2296538948 learned_addr learned my addr 172.21.15.5:0/2296538948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.049+0000 7f5e55ca0640 1 --2- 172.21.15.5:0/2296538948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d92760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.050+0000 7f5e5549f640 1 -- 172.21.15.5:0/2296538948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d70410 msgr2=0x5615a6d78460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.050+0000 7f5e5549f640 1 --2- 172.21.15.5:0/2296538948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d70410 0x5615a6d78460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.050+0000 7f5e5549f640 1 -- 172.21.15.5:0/2296538948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 msgr2=0x5615a6d92760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.050+0000 7f5e5549f640 1 --2- 172.21.15.5:0/2296538948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d92760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.050+0000 7f5e5549f640 1 -- 172.21.15.5:0/2296538948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615a6d6f920 con 0x5615a6d6faf0 2024-09-19T06:10:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.050+0000 7f5e5549f640 1 --2- 172.21.15.5:0/2296538948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 0x5615a6d72560 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0b8a40 tx=0x7f5e4c0b81f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4594d7d34778a9f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.050+0000 7f5e3affd640 1 -- 172.21.15.5:0/2296538948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c049040 con 0x5615a6d6faf0 2024-09-19T06:10:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.050+0000 7f5e3affd640 1 -- 172.21.15.5:0/2296538948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c0bc070 con 0x5615a6d6faf0 2024-09-19T06:10:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.050+0000 7f5e3affd640 1 -- 172.21.15.5:0/2296538948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c008730 con 0x5615a6d6faf0 2024-09-19T06:10:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.051+0000 7f5e5a498740 1 -- 172.21.15.5:0/2296538948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 msgr2=0x5615a6d72560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.051+0000 7f5e5a498740 1 --2- 172.21.15.5:0/2296538948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 0x5615a6d72560 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0b8a40 tx=0x7f5e4c0b81f0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.051+0000 7f5e5a498740 1 -- 172.21.15.5:0/2296538948 shutdown_connections 2024-09-19T06:10:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.051+0000 7f5e5a498740 1 --2- 172.21.15.5:0/2296538948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d92760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.051+0000 7f5e5a498740 1 --2- 172.21.15.5:0/2296538948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d70410 0x5615a6d78460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.051+0000 7f5e5a498740 1 --2- 172.21.15.5:0/2296538948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d6faf0 0x5615a6d72560 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.051+0000 7f5e5a498740 1 -- 172.21.15.5:0/2296538948 >> 172.21.15.5:0/2296538948 conn(0x5615a6d6c060 msgr2=0x5615a6d79c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.051+0000 7f5e5a498740 1 -- 172.21.15.5:0/2296538948 shutdown_connections 2024-09-19T06:10:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.051+0000 7f5e5a498740 1 -- 172.21.15.5:0/2296538948 wait complete. 2024-09-19T06:10:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.051+0000 7f5e5a498740 1 -- start start 2024-09-19T06:10:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.051+0000 7f5e5a498740 1 Processor -- start 2024-09-19T06:10:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e5a498740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 0x5615a6d832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e5a498740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 0x5615a6d7e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e5a498740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d7ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e5a498740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e501aecd0 con 0x5615a6d70410 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e5a498740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5e501ae600 con 0x5615a6d7acb0 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e5a498740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5e501ab860 con 0x5615a6d6faf0 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e5a498740 10 client.0 _ll_register_callbacks cb 0x7f5e480136e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e54c9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 0x5615a6d7e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e5549f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 0x5615a6d832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e54c9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 0x5615a6d7e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42210/0 (socket says 172.21.15.5:42210) 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e5549f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 0x5615a6d832b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52384/0 (socket says 172.21.15.5:52384) 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e54c9e640 1 -- 172.21.15.5:0/961525944 learned_addr learned my addr 172.21.15.5:0/961525944 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e55ca0640 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d7ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e54c9e640 1 -- 172.21.15.5:0/961525944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 msgr2=0x5615a6d832b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e54c9e640 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 0x5615a6d832b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e54c9e640 1 -- 172.21.15.5:0/961525944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 msgr2=0x5615a6d7ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e54c9e640 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d7ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e54c9e640 1 -- 172.21.15.5:0/961525944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615a6df4730 con 0x5615a6d70410 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e5549f640 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 0x5615a6d832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e55ca0640 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d7ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e54c9e640 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 0x5615a6d7e5d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x5615a6d77420 tx=0x7f5e40012c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abda72c2f5e1b2d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.052+0000 7f5e38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e38ff9640 1 -- 172.21.15.5:0/961525944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40023060 con 0x5615a6d70410 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e5a498740 1 -- 172.21.15.5:0/961525944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e4c0bc070 con 0x5615a6d70410 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e38ff9640 1 -- 172.21.15.5:0/961525944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7f5e40004230 con 0x5615a6d70410 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e38ff9640 1 -- 172.21.15.5:0/961525944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5e40004430 con 0x5615a6d70410 2024-09-19T06:10:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e5a498740 10 client.17352 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e5a498740 1 -- 172.21.15.5:0/961525944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5615a6df5920 con 0x5615a6d70410 2024-09-19T06:10:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e38ff9640 1 -- 172.21.15.5:0/961525944 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c0bc070 con 0x5615a6d70410 2024-09-19T06:10:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e38ff9640 1 -- 172.21.15.5:0/961525944 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5e40003c30 con 0x5615a6d70410 2024-09-19T06:10:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e5a498740 10 client.17352 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e5a498740 10 client.17352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e5a498740 20 client.17352 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e5a498740 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5615a6df6100 0x5615a6df8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e5a498740 4 client.17352 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:10:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e5a498740 1 -- 172.21.15.5:0/961525944 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615a6cdbd70 con 0x5615a6df6100 2024-09-19T06:10:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.053+0000 7f5e5549f640 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5615a6df6100 0x5615a6df8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.054+0000 7f5e38ff9640 10 client.17352 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.054+0000 7f5e5549f640 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5615a6df6100 0x5615a6df8520 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89a1c8c01280a038 server_cookie=f0ea2671755efa90 in_seq=0 out_seq=0 2024-09-19T06:10:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e38ff9640 1 -- 172.21.15.5:0/961525944 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615a6cdbd70 con 0x5615a6df6100 2024-09-19T06:10:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e38ff9640 10 client.17352 handle_command_reply: tid=0 2024-09-19T06:10:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e5a498740 1 client.17352 shutdown 2024-09-19T06:10:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e5a498740 1 -- 172.21.15.5:0/961525944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 msgr2=0x5615a6d7e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e5a498740 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 0x5615a6d7e5d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x5615a6d77420 tx=0x7f5e40012c50 comp rx=0 tx=0).stop 2024-09-19T06:10:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e5a498740 1 -- 172.21.15.5:0/961525944 shutdown_connections 2024-09-19T06:10:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e5a498740 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5615a6df6100 0x5615a6df8520 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e5a498740 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615a6d7acb0 0x5615a6d7ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e38ff9640 0 client.17352 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e5a498740 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615a6d70410 0x5615a6d7e5d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e5a498740 1 --2- 172.21.15.5:0/961525944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615a6d6faf0 0x5615a6d832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e5a498740 1 -- 172.21.15.5:0/961525944 >> 172.21.15.5:0/961525944 conn(0x5615a6d6c060 msgr2=0x5615a6d79410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e5a498740 1 -- 172.21.15.5:0/961525944 shutdown_connections 2024-09-19T06:10:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e5a498740 1 -- 172.21.15.5:0/961525944 wait complete. 2024-09-19T06:10:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.078+0000 7f5e5a498740 20 client.17352 trim_cache size 0 max 16384 2024-09-19T06:10:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:10:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:10:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:10:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:10:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.079+0000 7f5e57703640 1 -- 172.21.15.5:0/1572874249 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5e3c0a33b0 msgr2=0x7f5e3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.079+0000 7f5e57703640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5e3c0a33b0 0x7f5e3c0a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e480094d0 tx=0x7f5e48004ab0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.079+0000 7f5e57703640 1 -- 172.21.15.5:0/1572874249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e50102d20 msgr2=0x7f5e501983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.079+0000 7f5e57703640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e50102d20 0x7f5e501983a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c005940 tx=0x7f5e4c0040c0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.079+0000 7f5e57703640 1 -- 172.21.15.5:0/1572874249 shutdown_connections 2024-09-19T06:10:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.079+0000 7f5e57703640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e5010c0f0 0x7f5e5019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.079+0000 7f5e57703640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5e3c0a33b0 0x7f5e3c0a5870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.079+0000 7f5e57703640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e50103640 0x7f5e501988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.079+0000 7f5e57703640 1 --2- 172.21.15.5:0/1572874249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e50102d20 0x7f5e501983a0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.079+0000 7f5e57703640 1 -- 172.21.15.5:0/1572874249 >> 172.21.15.5:0/1572874249 conn(0x7f5e500779a0 msgr2=0x7f5e500fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.079+0000 7f5e57703640 1 -- 172.21.15.5:0/1572874249 shutdown_connections 2024-09-19T06:10:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:46.079+0000 7f5e57703640 1 -- 172.21.15.5:0/1572874249 wait complete. 2024-09-19T06:10:46.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:46 smithi005 ceph-mon[30955]: pgmap v228: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-19T06:10:46.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:46 smithi049 ceph-mon[34640]: pgmap v228: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-19T06:10:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:46 smithi157 ceph-mon[35346]: pgmap v228: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-19T06:10:46.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 189151 2.71 MB/sec execute 252 sec latency 16.049 ms 2024-09-19T06:10:47.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 189656 2.71 MB/sec execute 253 sec latency 19.750 ms 2024-09-19T06:10:48.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:48 smithi005 ceph-mon[30955]: pgmap v229: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-09-19T06:10:48.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:48 smithi049 ceph-mon[34640]: pgmap v229: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-09-19T06:10:48.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:48 smithi157 ceph-mon[35346]: pgmap v229: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-09-19T06:10:48.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 190301 2.71 MB/sec execute 254 sec latency 20.818 ms 2024-09-19T06:10:49.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 190899 2.72 MB/sec execute 255 sec latency 14.974 ms 2024-09-19T06:10:50.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:50 smithi005 ceph-mon[30955]: pgmap v230: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:10:50.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:50 smithi049 ceph-mon[34640]: pgmap v230: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:10:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:50 smithi157 ceph-mon[35346]: pgmap v230: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:10:50.832 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191375 2.72 MB/sec execute 256 sec latency 14.674 ms 2024-09-19T06:10:51.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191850 2.72 MB/sec execute 257 sec latency 13.687 ms 2024-09-19T06:10:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:52 smithi005 ceph-mon[30955]: pgmap v231: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:10:52.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:52 smithi049 ceph-mon[34640]: pgmap v231: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:10:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:52 smithi157 ceph-mon[35346]: pgmap v231: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:10:52.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 192139 2.71 MB/sec execute 258 sec latency 13.625 ms 2024-09-19T06:10:53.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 192643 2.71 MB/sec execute 259 sec latency 17.371 ms 2024-09-19T06:10:54.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:54 smithi005 ceph-mon[30955]: pgmap v232: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-19T06:10:54.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:54 smithi049 ceph-mon[34640]: pgmap v232: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-19T06:10:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:54 smithi157 ceph-mon[35346]: pgmap v232: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-19T06:10:54.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 193169 2.71 MB/sec execute 260 sec latency 21.256 ms 2024-09-19T06:10:55.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 193770 2.71 MB/sec execute 261 sec latency 20.617 ms 2024-09-19T06:10:56.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:56 smithi005 ceph-mon[30955]: pgmap v233: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-09-19T06:10:56.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:56 smithi049 ceph-mon[34640]: pgmap v233: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-09-19T06:10:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:56 smithi157 ceph-mon[35346]: pgmap v233: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-09-19T06:10:56.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 194421 2.72 MB/sec execute 262 sec latency 16.005 ms 2024-09-19T06:10:57.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 194899 2.72 MB/sec execute 263 sec latency 15.106 ms 2024-09-19T06:10:58.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:10:58 smithi005 ceph-mon[30955]: pgmap v234: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-19T06:10:58.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:10:58 smithi049 ceph-mon[34640]: pgmap v234: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-19T06:10:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:10:58 smithi157 ceph-mon[35346]: pgmap v234: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-19T06:10:58.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 195395 2.72 MB/sec execute 264 sec latency 11.074 ms 2024-09-19T06:10:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.175+0000 7fbda4531640 1 -- 172.21.15.5:0/189139846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd9c102c80 msgr2=0x7fbd9c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.175+0000 7fbda4531640 1 --2- 172.21.15.5:0/189139846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd9c102c80 0x7fbd9c103060 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009900 tx=0x7fbd9802a480 comp rx=0 tx=0).stop 2024-09-19T06:10:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.176+0000 7fbda4531640 1 -- 172.21.15.5:0/189139846 shutdown_connections 2024-09-19T06:10:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.176+0000 7fbda4531640 1 --2- 172.21.15.5:0/189139846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd9c10c0a0 0x7fbd9c10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.176+0000 7fbda4531640 1 --2- 172.21.15.5:0/189139846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd9c1035a0 0x7fbd9c10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.176+0000 7fbda4531640 1 --2- 172.21.15.5:0/189139846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd9c102c80 0x7fbd9c103060 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.176+0000 7fbda4531640 1 -- 172.21.15.5:0/189139846 >> 172.21.15.5:0/189139846 conn(0x7fbd9c0779a0 msgr2=0x7fbd9c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.176+0000 7fbda4531640 1 -- 172.21.15.5:0/189139846 shutdown_connections 2024-09-19T06:10:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.176+0000 7fbda4531640 1 -- 172.21.15.5:0/189139846 wait complete. 2024-09-19T06:10:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.176+0000 7fbda4531640 1 Processor -- start 2024-09-19T06:10:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.176+0000 7fbda4531640 1 -- start start 2024-09-19T06:10:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.176+0000 7fbda4531640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd9c102c80 0x7fbd9c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.176+0000 7fbda4531640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd9c1035a0 0x7fbd9c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda4531640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd9c10c0a0 0x7fbd9c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda4531640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c111d50 con 0x7fbd9c10c0a0 2024-09-19T06:10:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda4531640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c111bd0 con 0x7fbd9c102c80 2024-09-19T06:10:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda4531640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c111ed0 con 0x7fbd9c1035a0 2024-09-19T06:10:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda2ace640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd9c10c0a0 0x7fbd9c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda2ace640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd9c10c0a0 0x7fbd9c19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44412/0 (socket says 172.21.15.5:44412) 2024-09-19T06:10:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda2ace640 1 -- 172.21.15.5:0/1547682336 learned_addr learned my addr 172.21.15.5:0/1547682336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda1acc640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd9c1035a0 0x7fbd9c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda22cd640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd9c102c80 0x7fbd9c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda2ace640 1 -- 172.21.15.5:0/1547682336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd9c1035a0 msgr2=0x7fbd9c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda2ace640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd9c1035a0 0x7fbd9c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda2ace640 1 -- 172.21.15.5:0/1547682336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd9c102c80 msgr2=0x7fbd9c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda2ace640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd9c102c80 0x7fbd9c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda2ace640 1 -- 172.21.15.5:0/1547682336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c19d310 con 0x7fbd9c10c0a0 2024-09-19T06:10:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda2ace640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd9c10c0a0 0x7fbd9c19cbe0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400eb10 tx=0x7fbd9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbd937fe640 1 -- 172.21.15.5:0/1547682336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94004b20 con 0x7fbd9c10c0a0 2024-09-19T06:10:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbd937fe640 1 -- 172.21.15.5:0/1547682336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd94004cc0 con 0x7fbd9c10c0a0 2024-09-19T06:10:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbd937fe640 1 -- 172.21.15.5:0/1547682336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94010690 con 0x7fbd9c10c0a0 2024-09-19T06:10:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda4531640 1 -- 172.21.15.5:0/1547682336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c19d600 con 0x7fbd9c10c0a0 2024-09-19T06:10:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.177+0000 7fbda4531640 1 -- 172.21.15.5:0/1547682336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c1a4e90 con 0x7fbd9c10c0a0 2024-09-19T06:10:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.180+0000 7fbd937fe640 1 -- 172.21.15.5:0/1547682336 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbd94005020 con 0x7fbd9c10c0a0 2024-09-19T06:10:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda72d1740 1 Processor -- start 2024-09-19T06:10:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbd937fe640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbd880a3430 0x7fbd880a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda72d1740 1 -- start start 2024-09-19T06:10:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda72d1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4ef00 0x5591c9e4f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda1acc640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbd880a3430 0x7fbd880a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda72d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4f820 0x5591c9e59b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda72d1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e5a0c0 0x5591c9e5c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbd937fe640 1 -- 172.21.15.5:0/1547682336 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fbd940d0310 con 0x7fbd9c10c0a0 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda72d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5591c9daf2f0 con 0x5591c9e4f820 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda72d1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5591c9d57270 con 0x5591c9e5a0c0 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda72d1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5591c9dab2b0 con 0x5591c9e4ef00 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda1acc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4f820 0x5591c9e59b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda22cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4ef00 0x5591c9e4f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda1acc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4f820 0x5591c9e59b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44420/0 (socket says 172.21.15.5:44420) 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda22cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4ef00 0x5591c9e4f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60294/0 (socket says 172.21.15.5:60294) 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda1acc640 1 -- 172.21.15.5:0/1967685741 learned_addr learned my addr 172.21.15.5:0/1967685741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.181+0000 7fbda2ace640 1 --2- 172.21.15.5:0/1967685741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e5a0c0 0x5591c9e5c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.182+0000 7fbda22cd640 1 -- 172.21.15.5:0/1967685741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e5a0c0 msgr2=0x5591c9e5c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.182+0000 7fbda22cd640 1 --2- 172.21.15.5:0/1967685741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e5a0c0 0x5591c9e5c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.182+0000 7fbda22cd640 1 -- 172.21.15.5:0/1967685741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4f820 msgr2=0x5591c9e59b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.182+0000 7fbda22cd640 1 --2- 172.21.15.5:0/1967685741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4f820 0x5591c9e59b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.182+0000 7fbda22cd640 1 -- 172.21.15.5:0/1967685741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c9d56340 con 0x5591c9e4ef00 2024-09-19T06:10:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.182+0000 7fbda2ace640 1 --2- 172.21.15.5:0/1967685741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e5a0c0 0x5591c9e5c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:10:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.182+0000 7fbda1acc640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbd880a3430 0x7fbd880a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0139d0 tx=0x7fbd8c009b60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.182+0000 7fbda22cd640 1 --2- 172.21.15.5:0/1967685741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4ef00 0x5591c9e4f2e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009230 tx=0x7fbd98032ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff28c9e5a1223bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.182+0000 7fbd90bfe640 1 -- 172.21.15.5:0/1967685741 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98040070 con 0x5591c9e4ef00 2024-09-19T06:10:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.182+0000 7fbd90bfe640 1 -- 172.21.15.5:0/1967685741 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd98032d30 con 0x5591c9e4ef00 2024-09-19T06:10:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.182+0000 7fbd90bfe640 1 -- 172.21.15.5:0/1967685741 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98035040 con 0x5591c9e4ef00 2024-09-19T06:10:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.183+0000 7fbda72d1740 1 -- 172.21.15.5:0/1967685741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4ef00 msgr2=0x5591c9e4f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.183+0000 7fbda72d1740 1 --2- 172.21.15.5:0/1967685741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4ef00 0x5591c9e4f2e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009230 tx=0x7fbd98032ac0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.183+0000 7fbda72d1740 1 -- 172.21.15.5:0/1967685741 shutdown_connections 2024-09-19T06:10:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.183+0000 7fbda72d1740 1 --2- 172.21.15.5:0/1967685741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e5a0c0 0x5591c9e5c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.183+0000 7fbda72d1740 1 --2- 172.21.15.5:0/1967685741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4f820 0x5591c9e59b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.183+0000 7fbda72d1740 1 --2- 172.21.15.5:0/1967685741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4ef00 0x5591c9e4f2e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.183+0000 7fbda72d1740 1 -- 172.21.15.5:0/1967685741 >> 172.21.15.5:0/1967685741 conn(0x5591c9e4b470 msgr2=0x5591c9e4b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.183+0000 7fbda72d1740 1 -- 172.21.15.5:0/1967685741 shutdown_connections 2024-09-19T06:10:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.183+0000 7fbda72d1740 1 -- 172.21.15.5:0/1967685741 wait complete. 2024-09-19T06:10:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.183+0000 7fbda72d1740 1 -- start start 2024-09-19T06:10:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda72d1740 1 Processor -- start 2024-09-19T06:10:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda72d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 0x5591c9e56fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda72d1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e4f820 0x5591c9e574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda72d1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e5a0c0 0x5591c9e57a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda72d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5591c9e5cc90 con 0x5591c9e4ef00 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda72d1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5591c9d38290 con 0x5591c9e4f820 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda72d1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5591c9e5ce10 con 0x5591c9e5a0c0 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda72d1740 10 client.0 _ll_register_callbacks cb 0x5591c9d69140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda22cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 0x5591c9e56fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda22cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 0x5591c9e56fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44426/0 (socket says 172.21.15.5:44426) 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda22cd640 1 -- 172.21.15.5:0/37882585 learned_addr learned my addr 172.21.15.5:0/37882585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda1acc640 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e4f820 0x5591c9e574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda2ace640 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e5a0c0 0x5591c9e57a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda22cd640 1 -- 172.21.15.5:0/37882585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e5a0c0 msgr2=0x5591c9e57a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda22cd640 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e5a0c0 0x5591c9e57a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda22cd640 1 -- 172.21.15.5:0/37882585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e4f820 msgr2=0x5591c9e574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda22cd640 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e4f820 0x5591c9e574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda22cd640 1 -- 172.21.15.5:0/37882585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c9ac2210 con 0x5591c9e4ef00 2024-09-19T06:10:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbda22cd640 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 0x5591c9e56fb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbd98031750 tx=0x7fbd98031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f606f112ad17f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.184+0000 7fbd85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:10:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbd85ffb640 1 -- 172.21.15.5:0/37882585 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9804b070 con 0x5591c9e4ef00 2024-09-19T06:10:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbd85ffb640 1 -- 172.21.15.5:0/37882585 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fbd98005250 con 0x5591c9e4ef00 2024-09-19T06:10:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbda72d1740 1 -- 172.21.15.5:0/37882585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd98032d30 con 0x5591c9e4ef00 2024-09-19T06:10:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbd85ffb640 1 -- 172.21.15.5:0/37882585 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbd98057050 con 0x5591c9e4ef00 2024-09-19T06:10:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbda72d1740 10 client.17364 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbda72d1740 1 -- 172.21.15.5:0/37882585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5591c9b8e620 con 0x5591c9e4ef00 2024-09-19T06:10:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbd85ffb640 1 -- 172.21.15.5:0/37882585 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd98032d30 con 0x5591c9e4ef00 2024-09-19T06:10:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbd85ffb640 1 -- 172.21.15.5:0/37882585 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fbd9803c440 con 0x5591c9e4ef00 2024-09-19T06:10:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbda72d1740 10 client.17364 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbda72d1740 10 client.17364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbda72d1740 20 client.17364 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbda72d1740 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5591c9e69ea0 0x5591c9e6c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbda72d1740 4 client.17364 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:10:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbda72d1740 1 -- 172.21.15.5:0/37882585 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591c9dbca20 con 0x5591c9e69ea0 2024-09-19T06:10:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.185+0000 7fbda1acc640 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5591c9e69ea0 0x5591c9e6c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.186+0000 7fbd85ffb640 10 client.17364 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.186+0000 7fbda1acc640 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5591c9e69ea0 0x5591c9e6c290 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ddf405060ad65e server_cookie=5d6cdfccb7fffeb0 in_seq=0 out_seq=0 2024-09-19T06:10:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.186+0000 7fbd85ffb640 1 -- 172.21.15.5:0/37882585 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591c9dbca20 con 0x5591c9e69ea0 2024-09-19T06:10:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.186+0000 7fbd85ffb640 10 client.17364 handle_command_reply: tid=0 2024-09-19T06:10:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.186+0000 7fbda72d1740 1 client.17364 shutdown 2024-09-19T06:10:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.187+0000 7fbda72d1740 1 -- 172.21.15.5:0/37882585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 msgr2=0x5591c9e56fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.187+0000 7fbda72d1740 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 0x5591c9e56fb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbd98031750 tx=0x7fbd98031400 comp rx=0 tx=0).stop 2024-09-19T06:10:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.187+0000 7fbda72d1740 1 -- 172.21.15.5:0/37882585 shutdown_connections 2024-09-19T06:10:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.187+0000 7fbda72d1740 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5591c9e69ea0 0x5591c9e6c290 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.187+0000 7fbda72d1740 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e5a0c0 0x5591c9e57a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.187+0000 7fbda72d1740 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e4f820 0x5591c9e574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.187+0000 7fbda72d1740 1 --2- 172.21.15.5:0/37882585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 0x5591c9e56fb0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.187+0000 7fbda72d1740 1 -- 172.21.15.5:0/37882585 >> 172.21.15.5:0/37882585 conn(0x5591c9e4b470 msgr2=0x5591c9e461c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.187+0000 7fbd85ffb640 0 client.17364 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.187+0000 7fbda72d1740 1 -- 172.21.15.5:0/37882585 shutdown_connections 2024-09-19T06:10:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.187+0000 7fbda72d1740 1 -- 172.21.15.5:0/37882585 wait complete. 2024-09-19T06:10:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.187+0000 7fbda72d1740 20 client.17364 trim_cache size 0 max 16384 2024-09-19T06:10:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda72d1740 1 Processor -- start 2024-09-19T06:10:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda72d1740 1 -- start start 2024-09-19T06:10:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda72d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 0x5591c9e51970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda72d1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4f820 0x5591c9e55860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda72d1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e5a0c0 0x5591c9e718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda72d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1ab630 con 0x5591c9e4ef00 2024-09-19T06:10:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda72d1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1ad170 con 0x5591c9e5a0c0 2024-09-19T06:10:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda72d1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1abd00 con 0x5591c9e4f820 2024-09-19T06:10:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda22cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 0x5591c9e51970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda22cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 0x5591c9e51970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44428/0 (socket says 172.21.15.5:44428) 2024-09-19T06:10:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda22cd640 1 -- 172.21.15.5:0/2718781357 learned_addr learned my addr 172.21.15.5:0/2718781357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda2ace640 1 --2- 172.21.15.5:0/2718781357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e5a0c0 0x5591c9e718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda1acc640 1 --2- 172.21.15.5:0/2718781357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4f820 0x5591c9e55860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda22cd640 1 -- 172.21.15.5:0/2718781357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4f820 msgr2=0x5591c9e55860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda22cd640 1 --2- 172.21.15.5:0/2718781357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4f820 0x5591c9e55860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda22cd640 1 -- 172.21.15.5:0/2718781357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e5a0c0 msgr2=0x5591c9e718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda22cd640 1 --2- 172.21.15.5:0/2718781357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e5a0c0 0x5591c9e718e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.204+0000 7fbda22cd640 1 -- 172.21.15.5:0/2718781357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c9e55f40 con 0x5591c9e4ef00 2024-09-19T06:10:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.205+0000 7fbda22cd640 1 --2- 172.21.15.5:0/2718781357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 0x5591c9e51970 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbd9803c840 tx=0x7fbd9802ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201a075319463b38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.205+0000 7fbd87fff640 1 -- 172.21.15.5:0/2718781357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9804b070 con 0x5591c9e4ef00 2024-09-19T06:10:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.205+0000 7fbd87fff640 1 -- 172.21.15.5:0/2718781357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd9803c440 con 0x5591c9e4ef00 2024-09-19T06:10:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.205+0000 7fbd87fff640 1 -- 172.21.15.5:0/2718781357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98035070 con 0x5591c9e4ef00 2024-09-19T06:10:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.206+0000 7fbda72d1740 1 -- 172.21.15.5:0/2718781357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 msgr2=0x5591c9e51970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.206+0000 7fbda72d1740 1 --2- 172.21.15.5:0/2718781357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 0x5591c9e51970 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbd9803c840 tx=0x7fbd9802ac20 comp rx=0 tx=0).stop 2024-09-19T06:10:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.206+0000 7fbda72d1740 1 -- 172.21.15.5:0/2718781357 shutdown_connections 2024-09-19T06:10:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.206+0000 7fbda72d1740 1 --2- 172.21.15.5:0/2718781357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e5a0c0 0x5591c9e718e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.206+0000 7fbda72d1740 1 --2- 172.21.15.5:0/2718781357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4f820 0x5591c9e55860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.206+0000 7fbda72d1740 1 --2- 172.21.15.5:0/2718781357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e4ef00 0x5591c9e51970 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.206+0000 7fbda72d1740 1 -- 172.21.15.5:0/2718781357 >> 172.21.15.5:0/2718781357 conn(0x5591c9e4b470 msgr2=0x5591c9e59030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.206+0000 7fbda72d1740 1 -- 172.21.15.5:0/2718781357 shutdown_connections 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.206+0000 7fbda72d1740 1 -- 172.21.15.5:0/2718781357 wait complete. 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.206+0000 7fbda72d1740 1 -- start start 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.206+0000 7fbda72d1740 1 Processor -- start 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda72d1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4ef00 0x5591c9e5de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda72d1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e4f820 0x5591c9e5e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda72d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e5a0c0 0x5591c9e5e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda72d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1acaa0 con 0x5591c9e5a0c0 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda72d1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1ac3d0 con 0x5591c9e4f820 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda72d1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1a9630 con 0x5591c9e4ef00 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda72d1740 10 client.0 _ll_register_callbacks cb 0x7fbd8c013520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:10:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda1acc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e4f820 0x5591c9e5e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda1acc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e4f820 0x5591c9e5e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57608/0 (socket says 172.21.15.5:57608) 2024-09-19T06:10:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda1acc640 1 -- 172.21.15.5:0/1980358134 learned_addr learned my addr 172.21.15.5:0/1980358134 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:10:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda22cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4ef00 0x5591c9e5de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda2ace640 1 --2- 172.21.15.5:0/1980358134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e5a0c0 0x5591c9e5e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda1acc640 1 -- 172.21.15.5:0/1980358134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4ef00 msgr2=0x5591c9e5de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda1acc640 1 --2- 172.21.15.5:0/1980358134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4ef00 0x5591c9e5de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda1acc640 1 -- 172.21.15.5:0/1980358134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e5a0c0 msgr2=0x5591c9e5e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:10:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda1acc640 1 --2- 172.21.15.5:0/1980358134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e5a0c0 0x5591c9e5e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.207+0000 7fbda1acc640 1 -- 172.21.15.5:0/1980358134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c9e4f650 con 0x5591c9e4f820 2024-09-19T06:10:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.208+0000 7fbda1acc640 1 --2- 172.21.15.5:0/1980358134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e4f820 0x5591c9e5e340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5591c9e4c2d0 tx=0x7fbd8c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5291b04f6b5700c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:10:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.208+0000 7fbd85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:10:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.208+0000 7fbd85ffb640 1 -- 172.21.15.5:0/1980358134 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c013180 con 0x5591c9e4f820 2024-09-19T06:10:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.208+0000 7fbd85ffb640 1 -- 172.21.15.5:0/1980358134 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8550+0+0 (secure 0 0 0) 0x7fbd8c013320 con 0x5591c9e4f820 2024-09-19T06:10:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.208+0000 7fbda72d1740 1 -- 172.21.15.5:0/1980358134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd9803c440 con 0x5591c9e4f820 2024-09-19T06:10:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.208+0000 7fbd85ffb640 1 -- 172.21.15.5:0/1980358134 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbd8c03a050 con 0x5591c9e4f820 2024-09-19T06:10:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.208+0000 7fbda72d1740 10 client.26942 fetch_fsmap learned FSMap version 33 2024-09-19T06:10:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.208+0000 7fbda72d1740 1 -- 172.21.15.5:0/1980358134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5591c9ed4a10 con 0x5591c9e4f820 2024-09-19T06:10:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.208+0000 7fbd85ffb640 1 -- 172.21.15.5:0/1980358134 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd9803c440 con 0x5591c9e4f820 2024-09-19T06:10:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.209+0000 7fbd85ffb640 1 -- 172.21.15.5:0/1980358134 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fbd8c01f040 con 0x5591c9e4f820 2024-09-19T06:10:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.209+0000 7fbda72d1740 10 client.26942 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:10:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.209+0000 7fbda72d1740 10 client.26942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:10:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.209+0000 7fbda72d1740 20 client.26942 populate_metadata read hostname 'smithi005' 2024-09-19T06:10:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.209+0000 7fbda72d1740 1 --2- 172.21.15.5:0/1980358134 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5591c9ed51f0 0x5591c9ed7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:10:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.209+0000 7fbda72d1740 4 client.26942 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:10:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.209+0000 7fbda72d1740 1 -- 172.21.15.5:0/1980358134 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591c9db6d70 con 0x5591c9ed51f0 2024-09-19T06:10:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.209+0000 7fbda22cd640 1 --2- 172.21.15.5:0/1980358134 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5591c9ed51f0 0x5591c9ed7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:10:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.209+0000 7fbd85ffb640 10 client.26942 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.209+0000 7fbda22cd640 1 --2- 172.21.15.5:0/1980358134 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5591c9ed51f0 0x5591c9ed7610 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e192ab4a42101ae8 server_cookie=fb39821c0c37c3ec in_seq=0 out_seq=0 2024-09-19T06:10:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbd85ffb640 1 -- 172.21.15.5:0/1980358134 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591c9db6d70 con 0x5591c9ed51f0 2024-09-19T06:10:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbd85ffb640 10 client.26942 handle_command_reply: tid=0 2024-09-19T06:10:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbda72d1740 1 client.26942 shutdown 2024-09-19T06:10:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbda72d1740 1 -- 172.21.15.5:0/1980358134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e4f820 msgr2=0x5591c9e5e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbda72d1740 1 --2- 172.21.15.5:0/1980358134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e4f820 0x5591c9e5e340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5591c9e4c2d0 tx=0x7fbd8c004730 comp rx=0 tx=0).stop 2024-09-19T06:10:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbda72d1740 1 -- 172.21.15.5:0/1980358134 shutdown_connections 2024-09-19T06:10:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbda72d1740 1 --2- 172.21.15.5:0/1980358134 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5591c9ed51f0 0x5591c9ed7610 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbd85ffb640 0 client.26942 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:10:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbda72d1740 1 --2- 172.21.15.5:0/1980358134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591c9e5a0c0 0x5591c9e5e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbda72d1740 1 --2- 172.21.15.5:0/1980358134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591c9e4f820 0x5591c9e5e340 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbda72d1740 1 --2- 172.21.15.5:0/1980358134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591c9e4ef00 0x5591c9e5de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbda72d1740 1 -- 172.21.15.5:0/1980358134 >> 172.21.15.5:0/1980358134 conn(0x5591c9e4b470 msgr2=0x5591c9e461c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbda72d1740 1 -- 172.21.15.5:0/1980358134 shutdown_connections 2024-09-19T06:10:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbda72d1740 1 -- 172.21.15.5:0/1980358134 wait complete. 2024-09-19T06:10:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.228+0000 7fbda72d1740 20 client.26942 trim_cache size 0 max 16384 2024-09-19T06:10:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:10:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:10:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:10:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:10:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.229+0000 7fbda4531640 1 -- 172.21.15.5:0/1547682336 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbd880a3430 msgr2=0x7fbd880a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.229+0000 7fbda4531640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbd880a3430 0x7fbd880a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0139d0 tx=0x7fbd8c009b60 comp rx=0 tx=0).stop 2024-09-19T06:10:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.229+0000 7fbda4531640 1 -- 172.21.15.5:0/1547682336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd9c10c0a0 msgr2=0x7fbd9c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:10:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.229+0000 7fbda4531640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd9c10c0a0 0x7fbd9c19cbe0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400eb10 tx=0x7fbd9400c2d0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.229+0000 7fbda4531640 1 -- 172.21.15.5:0/1547682336 shutdown_connections 2024-09-19T06:10:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.229+0000 7fbda4531640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd9c10c0a0 0x7fbd9c19cbe0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.229+0000 7fbda4531640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbd880a3430 0x7fbd880a58f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.229+0000 7fbda4531640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd9c1035a0 0x7fbd9c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.229+0000 7fbda4531640 1 --2- 172.21.15.5:0/1547682336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd9c102c80 0x7fbd9c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:10:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.229+0000 7fbda4531640 1 -- 172.21.15.5:0/1547682336 >> 172.21.15.5:0/1547682336 conn(0x7fbd9c0779a0 msgr2=0x7fbd9c0fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:10:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.229+0000 7fbda4531640 1 -- 172.21.15.5:0/1547682336 shutdown_connections 2024-09-19T06:10:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:10:59.229+0000 7fbda4531640 1 -- 172.21.15.5:0/1547682336 wait complete. 2024-09-19T06:10:59.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 195673 2.72 MB/sec execute 265 sec latency 16.277 ms 2024-09-19T06:11:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:00 smithi005 ceph-mon[30955]: pgmap v235: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-09-19T06:11:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:00 smithi049 ceph-mon[34640]: pgmap v235: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-09-19T06:11:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:00 smithi157 ceph-mon[35346]: pgmap v235: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-09-19T06:11:00.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 196155 2.71 MB/sec execute 266 sec latency 43.606 ms 2024-09-19T06:11:01.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 196705 2.71 MB/sec execute 267 sec latency 17.776 ms 2024-09-19T06:11:02.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:02 smithi005 ceph-mon[30955]: pgmap v236: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-09-19T06:11:02.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:02 smithi049 ceph-mon[34640]: pgmap v236: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-09-19T06:11:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:02 smithi157 ceph-mon[35346]: pgmap v236: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-09-19T06:11:02.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 197273 2.71 MB/sec execute 268 sec latency 23.244 ms 2024-09-19T06:11:03.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 197839 2.72 MB/sec execute 269 sec latency 15.316 ms 2024-09-19T06:11:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:04 smithi049 ceph-mon[34640]: pgmap v237: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-09-19T06:11:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:04 smithi157 ceph-mon[35346]: pgmap v237: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-09-19T06:11:04.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 198417 2.72 MB/sec execute 270 sec latency 24.793 ms 2024-09-19T06:11:04.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:04 smithi005 ceph-mon[30955]: pgmap v237: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-09-19T06:11:05.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:05 smithi049 ceph-mon[34640]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T06:11:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:05 smithi157 ceph-mon[35346]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T06:11:05.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 198893 2.72 MB/sec execute 271 sec latency 15.805 ms 2024-09-19T06:11:05.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:05 smithi005 ceph-mon[30955]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T06:11:06.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:06 smithi049 ceph-mon[34640]: pgmap v239: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-09-19T06:11:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:06 smithi157 ceph-mon[35346]: pgmap v239: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-09-19T06:11:06.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 199179 2.72 MB/sec execute 272 sec latency 11.334 ms 2024-09-19T06:11:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:06 smithi005 ceph-mon[30955]: pgmap v239: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-09-19T06:11:07.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 199615 2.71 MB/sec execute 273 sec latency 18.796 ms 2024-09-19T06:11:08.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:08 smithi049 ceph-mon[34640]: pgmap v240: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-19T06:11:08.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:08 smithi049 ceph-mon[34640]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T06:11:08.680 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:08 smithi005 ceph-mon[30955]: pgmap v240: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-19T06:11:08.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:08 smithi005 ceph-mon[30955]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T06:11:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:08 smithi157 ceph-mon[35346]: pgmap v240: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-19T06:11:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:08 smithi157 ceph-mon[35346]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T06:11:08.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 200145 2.71 MB/sec execute 274 sec latency 20.118 ms 2024-09-19T06:11:08.964 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:11:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:11:09.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.785+0000 7f07c5f14640 1 -- 172.21.15.5:0/63150635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0102570 msgr2=0x7f07c01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:09.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.785+0000 7f07c5f14640 1 --2- 172.21.15.5:0/63150635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0102570 0x7f07c01029b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009900 tx=0x7f07b402a480 comp rx=0 tx=0).stop 2024-09-19T06:11:09.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.785+0000 7f07c5f14640 1 -- 172.21.15.5:0/63150635 shutdown_connections 2024-09-19T06:11:09.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.785+0000 7f07c5f14640 1 --2- 172.21.15.5:0/63150635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c0102ef0 0x7f07c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.785+0000 7f07c5f14640 1 --2- 172.21.15.5:0/63150635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0102570 0x7f07c01029b0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.785+0000 7f07c5f14640 1 --2- 172.21.15.5:0/63150635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c0100e20 0x7f07c0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.785+0000 7f07c5f14640 1 -- 172.21.15.5:0/63150635 >> 172.21.15.5:0/63150635 conn(0x7f07c00fc7d0 msgr2=0x7f07c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:09.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.785+0000 7f07c5f14640 1 -- 172.21.15.5:0/63150635 shutdown_connections 2024-09-19T06:11:09.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.785+0000 7f07c5f14640 1 -- 172.21.15.5:0/63150635 wait complete. 2024-09-19T06:11:09.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.786+0000 7f07c5f14640 1 Processor -- start 2024-09-19T06:11:09.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.786+0000 7f07c5f14640 1 -- start start 2024-09-19T06:11:09.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07c5f14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0100e20 0x7f07c0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07c5f14640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c0102570 0x7f07c0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07c5f14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c0102ef0 0x7f07c019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0100e20 0x7f07c0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0100e20 0x7f07c0198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43918/0 (socket says 172.21.15.5:43918) 2024-09-19T06:11:09.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07bf7fe640 1 -- 172.21.15.5:0/178513345 learned_addr learned my addr 172.21.15.5:0/178513345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:09.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07beffd640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c0102570 0x7f07c0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07c5f14640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0110e30 con 0x7f07c0100e20 2024-09-19T06:11:09.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07c5f14640 1 -- 172.21.15.5:0/178513345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0110cb0 con 0x7f07c0102ef0 2024-09-19T06:11:09.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07c5f14640 1 -- 172.21.15.5:0/178513345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0110fb0 con 0x7f07c0102570 2024-09-19T06:11:09.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07bffff640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c0102ef0 0x7f07c019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07bf7fe640 1 -- 172.21.15.5:0/178513345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c0102570 msgr2=0x7f07c0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:09.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07bf7fe640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c0102570 0x7f07c0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07bf7fe640 1 -- 172.21.15.5:0/178513345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c0102ef0 msgr2=0x7f07c019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:09.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07bf7fe640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c0102ef0 0x7f07c019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07bf7fe640 1 -- 172.21.15.5:0/178513345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c019d3f0 con 0x7f07c0100e20 2024-09-19T06:11:09.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.787+0000 7f07beffd640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c0102570 0x7f07c0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:09.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.788+0000 7f07bf7fe640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0100e20 0x7f07c0198440 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f07b000e9f0 tx=0x7f07b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:09.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.788+0000 7f07bcff9640 1 -- 172.21.15.5:0/178513345 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b000cd80 con 0x7f07c0100e20 2024-09-19T06:11:09.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.788+0000 7f07bcff9640 1 -- 172.21.15.5:0/178513345 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07b00054d0 con 0x7f07c0100e20 2024-09-19T06:11:09.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.788+0000 7f07c5f14640 1 -- 172.21.15.5:0/178513345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c019d6e0 con 0x7f07c0100e20 2024-09-19T06:11:09.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.788+0000 7f07bcff9640 1 -- 172.21.15.5:0/178513345 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0010490 con 0x7f07c0100e20 2024-09-19T06:11:09.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.788+0000 7f07c5f14640 1 -- 172.21.15.5:0/178513345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c01a4fc0 con 0x7f07c0100e20 2024-09-19T06:11:09.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.790+0000 7f07bcff9640 1 -- 172.21.15.5:0/178513345 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f07b0005020 con 0x7f07c0100e20 2024-09-19T06:11:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.790+0000 7f07c82ed740 1 Processor -- start 2024-09-19T06:11:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07c82ed740 1 -- start start 2024-09-19T06:11:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07c82ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e23a0 0x5600571e2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07c82ed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e2cc0 0x5600571f1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07c82ed740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571f1a90 0x5600571f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07c82ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5600571482e0 con 0x5600571e23a0 2024-09-19T06:11:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07c82ed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5600570efce0 con 0x5600571e2cc0 2024-09-19T06:11:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07c82ed740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5600571442a0 con 0x5600571f1a90 2024-09-19T06:11:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e23a0 0x5600571e2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e23a0 0x5600571e2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43924/0 (socket says 172.21.15.5:43924) 2024-09-19T06:11:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07bffff640 1 -- 172.21.15.5:0/2102757497 learned_addr learned my addr 172.21.15.5:0/2102757497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07bf7fe640 1 --2- 172.21.15.5:0/2102757497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e2cc0 0x5600571f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07beffd640 1 --2- 172.21.15.5:0/2102757497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571f1a90 0x5600571f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07bffff640 1 -- 172.21.15.5:0/2102757497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571f1a90 msgr2=0x5600571f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07bffff640 1 --2- 172.21.15.5:0/2102757497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571f1a90 0x5600571f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07bffff640 1 -- 172.21.15.5:0/2102757497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e2cc0 msgr2=0x5600571f1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07bffff640 1 --2- 172.21.15.5:0/2102757497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e2cc0 0x5600571f1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07bffff640 1 -- 172.21.15.5:0/2102757497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600570d4440 con 0x5600571e23a0 2024-09-19T06:11:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.791+0000 7f07bf7fe640 1 --2- 172.21.15.5:0/2102757497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e2cc0 0x5600571f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.792+0000 7f07bcff9640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f07a80a3430 0x7f07a80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.792+0000 7f07bffff640 1 --2- 172.21.15.5:0/2102757497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e23a0 0x5600571e2780 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f07b80039f0 tx=0x7f07b8003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14afcbe7f028ffef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.792+0000 7f07bf7fe640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f07a80a3430 0x7f07a80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.792+0000 7f07bcff9640 1 -- 172.21.15.5:0/178513345 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f07b00d0040 con 0x7f07c0100e20 2024-09-19T06:11:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.792+0000 7f07a5ffb640 1 -- 172.21.15.5:0/2102757497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b801a470 con 0x5600571e23a0 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.792+0000 7f07a5ffb640 1 -- 172.21.15.5:0/2102757497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07b8015070 con 0x5600571e23a0 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.792+0000 7f07a5ffb640 1 -- 172.21.15.5:0/2102757497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8010800 con 0x5600571e23a0 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.793+0000 7f07c82ed740 1 -- 172.21.15.5:0/2102757497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e23a0 msgr2=0x5600571e2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.793+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2102757497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e23a0 0x5600571e2780 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f07b80039f0 tx=0x7f07b8003eb0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.793+0000 7f07bf7fe640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f07a80a3430 0x7f07a80a58f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f07b00cd820 tx=0x7f07b00d3b90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.793+0000 7f07c82ed740 1 -- 172.21.15.5:0/2102757497 shutdown_connections 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.793+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2102757497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571f1a90 0x5600571f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.793+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2102757497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e2cc0 0x5600571f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.793+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2102757497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e23a0 0x5600571e2780 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.793+0000 7f07c82ed740 1 -- 172.21.15.5:0/2102757497 >> 172.21.15.5:0/2102757497 conn(0x5600571dea30 msgr2=0x5600571dee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.793+0000 7f07c82ed740 1 -- 172.21.15.5:0/2102757497 shutdown_connections 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.793+0000 7f07c82ed740 1 -- 172.21.15.5:0/2102757497 wait complete. 2024-09-19T06:11:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.793+0000 7f07c82ed740 1 -- start start 2024-09-19T06:11:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07c82ed740 1 Processor -- start 2024-09-19T06:11:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07c82ed740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571e23a0 0x5600571f1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07c82ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 0x5600571ee730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07c82ed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571f1a90 0x5600571eed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07c82ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5600571f46c0 con 0x5600571e2cc0 2024-09-19T06:11:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07c82ed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560057103050 con 0x5600571f1a90 2024-09-19T06:11:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07c82ed740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5600571f4840 con 0x5600571e23a0 2024-09-19T06:11:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07c82ed740 10 client.0 _ll_register_callbacks cb 0x560057143160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 0x5600571ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 0x5600571ee730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43936/0 (socket says 172.21.15.5:43936) 2024-09-19T06:11:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07beffd640 1 -- 172.21.15.5:0/3665857827 learned_addr learned my addr 172.21.15.5:0/3665857827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07bffff640 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571f1a90 0x5600571eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07bffff640 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571e23a0 0x5600571f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07beffd640 1 -- 172.21.15.5:0/3665857827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571e23a0 msgr2=0x5600571f1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07beffd640 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571e23a0 0x5600571f1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07beffd640 1 -- 172.21.15.5:0/3665857827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571f1a90 msgr2=0x5600571eed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07beffd640 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571f1a90 0x5600571eed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.794+0000 7f07beffd640 1 -- 172.21.15.5:0/3665857827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560056eb93c0 con 0x5600571e2cc0 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.795+0000 7f07beffd640 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 0x5600571ee730 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009900 tx=0x7f07b40316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55b5184a7233f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.795+0000 7f079effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.795+0000 7f079effd640 1 -- 172.21.15.5:0/3665857827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4035040 con 0x5600571e2cc0 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.795+0000 7f079effd640 1 -- 172.21.15.5:0/3665857827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f07b4004450 con 0x5600571e2cc0 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.795+0000 7f07c82ed740 1 -- 172.21.15.5:0/3665857827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b8015070 con 0x5600571e2cc0 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.795+0000 7f079effd640 1 -- 172.21.15.5:0/3665857827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f07b4005020 con 0x5600571e2cc0 2024-09-19T06:11:09.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.795+0000 7f07c82ed740 10 client.17382 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.795+0000 7f07c82ed740 1 -- 172.21.15.5:0/3665857827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560057200160 con 0x5600571e2cc0 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.795+0000 7f079effd640 1 -- 172.21.15.5:0/3665857827 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07b8015070 con 0x5600571e2cc0 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.796+0000 7f079effd640 1 -- 172.21.15.5:0/3665857827 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f07b40035f0 con 0x5600571e2cc0 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.796+0000 7f07c82ed740 10 client.17382 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.796+0000 7f07c82ed740 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.796+0000 7f07c82ed740 20 client.17382 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.796+0000 7f07c82ed740 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560057201620 0x560057203a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.796+0000 7f07c82ed740 4 client.17382 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.796+0000 7f07c82ed740 1 -- 172.21.15.5:0/3665857827 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560057155850 con 0x560057201620 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.796+0000 7f07bffff640 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560057201620 0x560057203a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.796+0000 7f079effd640 10 client.17382 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.796+0000 7f07bffff640 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560057201620 0x560057203a40 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2730f7b962ebf1f9 server_cookie=2af3f1763f13dd77 in_seq=0 out_seq=0 2024-09-19T06:11:09.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f079effd640 1 -- 172.21.15.5:0/3665857827 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560057155850 con 0x560057201620 2024-09-19T06:11:09.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f079effd640 10 client.17382 handle_command_reply: tid=0 2024-09-19T06:11:09.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f07c82ed740 1 client.17382 shutdown 2024-09-19T06:11:09.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f07c82ed740 1 -- 172.21.15.5:0/3665857827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 msgr2=0x5600571ee730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:09.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f07c82ed740 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 0x5600571ee730 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009900 tx=0x7f07b40316d0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f07c82ed740 1 -- 172.21.15.5:0/3665857827 shutdown_connections 2024-09-19T06:11:09.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f07c82ed740 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560057201620 0x560057203a40 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f07c82ed740 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571f1a90 0x5600571eed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f07c82ed740 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 0x5600571ee730 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f079effd640 0 client.17382 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:09.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f07c82ed740 1 --2- 172.21.15.5:0/3665857827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571e23a0 0x5600571f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f07c82ed740 1 -- 172.21.15.5:0/3665857827 >> 172.21.15.5:0/3665857827 conn(0x5600571dea30 msgr2=0x5600571e4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:09.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f07c82ed740 1 -- 172.21.15.5:0/3665857827 shutdown_connections 2024-09-19T06:11:09.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.797+0000 7f07c82ed740 1 -- 172.21.15.5:0/3665857827 wait complete. 2024-09-19T06:11:09.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.798+0000 7f07c82ed740 20 client.17382 trim_cache size 0 max 16384 2024-09-19T06:11:09.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.818+0000 7f07c82ed740 1 Processor -- start 2024-09-19T06:11:09.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.818+0000 7f07c82ed740 1 -- start start 2024-09-19T06:11:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07c82ed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e23a0 0x5600571fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07c82ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 0x5600571ed000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07c82ed740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571f1a90 0x5600571f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07c82ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01ab5c0 con 0x5600571e2cc0 2024-09-19T06:11:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07c82ed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01ad100 con 0x5600571e23a0 2024-09-19T06:11:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07c82ed740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01abc90 con 0x5600571f1a90 2024-09-19T06:11:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 0x5600571ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07bffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e23a0 0x5600571fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07bffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e23a0 0x5600571fd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60956/0 (socket says 172.21.15.5:60956) 2024-09-19T06:11:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07bffff640 1 -- 172.21.15.5:0/2955212707 learned_addr learned my addr 172.21.15.5:0/2955212707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 0x5600571ed000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43948/0 (socket says 172.21.15.5:43948) 2024-09-19T06:11:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07bffff640 1 --2- 172.21.15.5:0/2955212707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571f1a90 0x5600571f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07bffff640 1 -- 172.21.15.5:0/2955212707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571f1a90 msgr2=0x5600571f06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:09.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07bffff640 1 --2- 172.21.15.5:0/2955212707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571f1a90 0x5600571f06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07bffff640 1 -- 172.21.15.5:0/2955212707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 msgr2=0x5600571ed000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:09.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07bffff640 1 --2- 172.21.15.5:0/2955212707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 0x5600571ed000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07bffff640 1 -- 172.21.15.5:0/2955212707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600571eead0 con 0x5600571e23a0 2024-09-19T06:11:09.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07bffff640 1 --2- 172.21.15.5:0/2955212707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e23a0 0x5600571fd960 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f07b8006010 tx=0x7f07b8010440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb485ba149d1408b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:09.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.819+0000 7f07a57fa640 1 -- 172.21.15.5:0/2955212707 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8023040 con 0x5600571e23a0 2024-09-19T06:11:09.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.820+0000 7f07a57fa640 1 -- 172.21.15.5:0/2955212707 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07b8015070 con 0x5600571e23a0 2024-09-19T06:11:09.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.820+0000 7f07a57fa640 1 -- 172.21.15.5:0/2955212707 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80052b0 con 0x5600571e23a0 2024-09-19T06:11:09.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.820+0000 7f07c82ed740 1 -- 172.21.15.5:0/2955212707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e23a0 msgr2=0x5600571fd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.820+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2955212707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e23a0 0x5600571fd960 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f07b8006010 tx=0x7f07b8010440 comp rx=0 tx=0).stop 2024-09-19T06:11:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.820+0000 7f07c82ed740 1 -- 172.21.15.5:0/2955212707 shutdown_connections 2024-09-19T06:11:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.820+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2955212707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571f1a90 0x5600571f06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.820+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2955212707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571e2cc0 0x5600571ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.820+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2955212707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e23a0 0x5600571fd960 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.820+0000 7f07c82ed740 1 -- 172.21.15.5:0/2955212707 >> 172.21.15.5:0/2955212707 conn(0x5600571dea30 msgr2=0x5600571eff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.820+0000 7f07c82ed740 1 -- 172.21.15.5:0/2955212707 shutdown_connections 2024-09-19T06:11:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.821+0000 7f07c82ed740 1 -- 172.21.15.5:0/2955212707 wait complete. 2024-09-19T06:11:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.821+0000 7f07c82ed740 1 -- start start 2024-09-19T06:11:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.821+0000 7f07c82ed740 1 Processor -- start 2024-09-19T06:11:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.821+0000 7f07c82ed740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571e23a0 0x5600571f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.821+0000 7f07c82ed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e2cc0 0x5600571f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.821+0000 7f07c82ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571f1a90 0x56005726b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.821+0000 7f07c82ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01aca30 con 0x5600571f1a90 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.821+0000 7f07c82ed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01ac360 con 0x5600571e2cc0 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.821+0000 7f07c82ed740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01a95c0 con 0x5600571e23a0 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.821+0000 7f07c82ed740 10 client.0 _ll_register_callbacks cb 0x7f07b4032ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f07bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571f1a90 0x56005726b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f07bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571f1a90 0x56005726b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43964/0 (socket says 172.21.15.5:43964) 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f07beffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e2cc0 0x5600571f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f07bffff640 1 -- 172.21.15.5:0/2874093491 learned_addr learned my addr 172.21.15.5:0/2874093491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f07bffff640 1 --2- 172.21.15.5:0/2874093491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571e23a0 0x5600571f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f07beffd640 1 -- 172.21.15.5:0/2874093491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571e23a0 msgr2=0x5600571f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f07beffd640 1 --2- 172.21.15.5:0/2874093491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571e23a0 0x5600571f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f07beffd640 1 -- 172.21.15.5:0/2874093491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571f1a90 msgr2=0x56005726b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:09.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f07beffd640 1 --2- 172.21.15.5:0/2874093491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571f1a90 0x56005726b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f07beffd640 1 -- 172.21.15.5:0/2874093491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005726b820 con 0x5600571e2cc0 2024-09-19T06:11:09.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f07beffd640 1 --2- 172.21.15.5:0/2874093491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e2cc0 0x5600571f61b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5600571efce0 tx=0x7f07b4003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d3b4906e9b901e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:09.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f079effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:11:09.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.822+0000 7f079effd640 1 -- 172.21.15.5:0/2874093491 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4035070 con 0x5600571e2cc0 2024-09-19T06:11:09.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f079effd640 1 -- 172.21.15.5:0/2874093491 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f07b4004450 con 0x5600571e2cc0 2024-09-19T06:11:09.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f07c82ed740 1 -- 172.21.15.5:0/2874093491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b8015070 con 0x5600571e2cc0 2024-09-19T06:11:09.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f079effd640 1 -- 172.21.15.5:0/2874093491 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f07b4054040 con 0x5600571e2cc0 2024-09-19T06:11:09.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f07c82ed740 10 client.26960 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:09.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f07c82ed740 1 -- 172.21.15.5:0/2874093491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56005726ca10 con 0x5600571e2cc0 2024-09-19T06:11:09.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f079effd640 1 -- 172.21.15.5:0/2874093491 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07b8015070 con 0x5600571e2cc0 2024-09-19T06:11:09.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f079effd640 1 -- 172.21.15.5:0/2874093491 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f07b4057070 con 0x5600571e2cc0 2024-09-19T06:11:09.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f07c82ed740 10 client.26960 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:09.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f07c82ed740 10 client.26960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:09.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f07c82ed740 20 client.26960 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:09.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2874093491 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56005726d1e0 0x56005726f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:09.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f07c82ed740 4 client.26960 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:11:09.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f07c82ed740 1 -- 172.21.15.5:0/2874093491 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56005714fd60 con 0x56005726d1e0 2024-09-19T06:11:09.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.823+0000 7f07bffff640 1 --2- 172.21.15.5:0/2874093491 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56005726d1e0 0x56005726f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:09.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f079effd640 10 client.26960 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:09.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f07bffff640 1 --2- 172.21.15.5:0/2874093491 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56005726d1e0 0x56005726f600 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c4a732366a98f server_cookie=2b8939bb4e361102 in_seq=0 out_seq=0 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f079effd640 1 -- 172.21.15.5:0/2874093491 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56005714fd60 con 0x56005726d1e0 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f079effd640 10 client.26960 handle_command_reply: tid=0 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f07c82ed740 1 client.26960 shutdown 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f07c82ed740 1 -- 172.21.15.5:0/2874093491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e2cc0 msgr2=0x5600571f61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2874093491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e2cc0 0x5600571f61b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5600571efce0 tx=0x7f07b4003e90 comp rx=0 tx=0).stop 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f07c82ed740 1 -- 172.21.15.5:0/2874093491 shutdown_connections 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2874093491 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56005726d1e0 0x56005726f600 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2874093491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600571f1a90 0x56005726b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2874093491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5600571e2cc0 0x5600571f61b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f079effd640 0 client.26960 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f07c82ed740 1 --2- 172.21.15.5:0/2874093491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600571e23a0 0x5600571f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f07c82ed740 1 -- 172.21.15.5:0/2874093491 >> 172.21.15.5:0/2874093491 conn(0x5600571dea30 msgr2=0x5600571f07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f07c82ed740 1 -- 172.21.15.5:0/2874093491 shutdown_connections 2024-09-19T06:11:09.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.824+0000 7f07c82ed740 1 -- 172.21.15.5:0/2874093491 wait complete. 2024-09-19T06:11:09.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c82ed740 20 client.26960 trim_cache size 0 max 16384 2024-09-19T06:11:09.832 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:11:09.832 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:11:09.832 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:11:09.832 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:11:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c5f14640 1 -- 172.21.15.5:0/178513345 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f07a80a3430 msgr2=0x7f07a80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c5f14640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f07a80a3430 0x7f07a80a58f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f07b00cd820 tx=0x7f07b00d3b90 comp rx=0 tx=0).stop 2024-09-19T06:11:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c5f14640 1 -- 172.21.15.5:0/178513345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0100e20 msgr2=0x7f07c0198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c5f14640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0100e20 0x7f07c0198440 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f07b000e9f0 tx=0x7f07b000edf0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c5f14640 1 -- 172.21.15.5:0/178513345 shutdown_connections 2024-09-19T06:11:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c5f14640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c0102ef0 0x7f07c019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c5f14640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f07a80a3430 0x7f07a80a58f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c5f14640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c0102570 0x7f07c0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c5f14640 1 --2- 172.21.15.5:0/178513345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0100e20 0x7f07c0198440 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c5f14640 1 -- 172.21.15.5:0/178513345 >> 172.21.15.5:0/178513345 conn(0x7f07c00fc7d0 msgr2=0x7f07c006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c5f14640 1 -- 172.21.15.5:0/178513345 shutdown_connections 2024-09-19T06:11:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:09.825+0000 7f07c5f14640 1 -- 172.21.15.5:0/178513345 wait complete. 2024-09-19T06:11:09.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 200702 2.71 MB/sec execute 275 sec latency 19.167 ms 2024-09-19T06:11:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:10 smithi049 ceph-mon[34640]: pgmap v242: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 2.3 MiB/s wr, 219 op/s 2024-09-19T06:11:10.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:10 smithi005 ceph-mon[30955]: pgmap v242: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 2.3 MiB/s wr, 219 op/s 2024-09-19T06:11:10.664 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:11:10.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:11:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:10 smithi157 ceph-mon[35346]: pgmap v242: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 2.3 MiB/s wr, 219 op/s 2024-09-19T06:11:10.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 201230 2.71 MB/sec execute 276 sec latency 19.320 ms 2024-09-19T06:11:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:11:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.523+0000 7f6c6c611640 1 -- 172.21.15.5:0/1644829236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64102ef0 msgr2=0x7f6c6410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.523+0000 7f6c6c611640 1 --2- 172.21.15.5:0/1644829236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64102ef0 0x7f6c6410da40 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009ca0 tx=0x7f6c5c02a950 comp rx=0 tx=0).stop 2024-09-19T06:11:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.524+0000 7f6c6c611640 1 -- 172.21.15.5:0/1644829236 shutdown_connections 2024-09-19T06:11:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.524+0000 7f6c6c611640 1 --2- 172.21.15.5:0/1644829236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64102ef0 0x7f6c6410da40 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.524+0000 7f6c6c611640 1 --2- 172.21.15.5:0/1644829236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64102570 0x7f6c641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.524+0000 7f6c6c611640 1 --2- 172.21.15.5:0/1644829236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64100e20 0x7f6c64101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.524+0000 7f6c6c611640 1 -- 172.21.15.5:0/1644829236 >> 172.21.15.5:0/1644829236 conn(0x7f6c640fc7d0 msgr2=0x7f6c640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.524+0000 7f6c6c611640 1 -- 172.21.15.5:0/1644829236 shutdown_connections 2024-09-19T06:11:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.524+0000 7f6c6c611640 1 -- 172.21.15.5:0/1644829236 wait complete. 2024-09-19T06:11:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.524+0000 7f6c6c611640 1 Processor -- start 2024-09-19T06:11:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.524+0000 7f6c6c611640 1 -- start start 2024-09-19T06:11:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6c611640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64100e20 0x7f6c640fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6c611640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64102570 0x7f6c640fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6c611640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64102ef0 0x7f6c640fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6c611640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64110e30 con 0x7f6c64102ef0 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6c611640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64110cb0 con 0x7f6c64100e20 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6c611640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64110fb0 con 0x7f6c64102570 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6a3ad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64100e20 0x7f6c640fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6a3ad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64100e20 0x7f6c640fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60970/0 (socket says 172.21.15.5:60970) 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6a3ad640 1 -- 172.21.15.5:0/2770164608 learned_addr learned my addr 172.21.15.5:0/2770164608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6abae640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64102ef0 0x7f6c640fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c69bac640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64102570 0x7f6c640fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6a3ad640 1 -- 172.21.15.5:0/2770164608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64102570 msgr2=0x7f6c640fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6a3ad640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64102570 0x7f6c640fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6a3ad640 1 -- 172.21.15.5:0/2770164608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64102ef0 msgr2=0x7f6c640fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6a3ad640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64102ef0 0x7f6c640fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6a3ad640 1 -- 172.21.15.5:0/2770164608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c641a6840 con 0x7f6c64100e20 2024-09-19T06:11:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c6a3ad640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64100e20 0x7f6c640fcc80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000d770 tx=0x7f6c6000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.525+0000 7f6c5b7fe640 1 -- 172.21.15.5:0/2770164608 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60017070 con 0x7f6c64100e20 2024-09-19T06:11:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.526+0000 7f6c6c611640 1 -- 172.21.15.5:0/2770164608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c641a6b30 con 0x7f6c64100e20 2024-09-19T06:11:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.526+0000 7f6c5b7fe640 1 -- 172.21.15.5:0/2770164608 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c600054d0 con 0x7f6c64100e20 2024-09-19T06:11:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.526+0000 7f6c5b7fe640 1 -- 172.21.15.5:0/2770164608 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60004cf0 con 0x7f6c64100e20 2024-09-19T06:11:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.526+0000 7f6c6c611640 1 -- 172.21.15.5:0/2770164608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c641a7070 con 0x7f6c64100e20 2024-09-19T06:11:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.527+0000 7f6c6c611640 1 -- 172.21.15.5:0/2770164608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c64101200 con 0x7f6c64100e20 2024-09-19T06:11:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.528+0000 7f6c5b7fe640 1 -- 172.21.15.5:0/2770164608 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6c60004e90 con 0x7f6c64100e20 2024-09-19T06:11:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.529+0000 7f6c5b7fe640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6c500a33b0 0x7f6c500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.529+0000 7f6c69bac640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6c500a33b0 0x7f6c500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.529+0000 7f6c5b7fe640 1 -- 172.21.15.5:0/2770164608 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f6c600cf730 con 0x7f6c64100e20 2024-09-19T06:11:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.529+0000 7f6c69bac640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6c500a33b0 0x7f6c500a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c640fe710 tx=0x7f6c54005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.532+0000 7f6c5b7fe640 1 -- 172.21.15.5:0/2770164608 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6c6009ba40 con 0x7f6c64100e20 2024-09-19T06:11:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.781+0000 7f6c6c611640 1 -- 172.21.15.5:0/2770164608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c6410a0a0 con 0x7f6c64100e20 2024-09-19T06:11:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.782+0000 7f6c5b7fe640 1 -- 172.21.15.5:0/2770164608 <== mon.2 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11959 (secure 0 0 0) 0x7f6c6009fd40 con 0x7f6c64100e20 2024-09-19T06:11:11.784 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:11:11.785 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-19T06:07:24:781633+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:07:24.019631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:11:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-19T06:11:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.784+0000 7f6c6c611640 1 -- 172.21.15.5:0/2770164608 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6c500a33b0 msgr2=0x7f6c500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.784+0000 7f6c6c611640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6c500a33b0 0x7f6c500a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c640fe710 tx=0x7f6c54005f90 comp rx=0 tx=0).stop 2024-09-19T06:11:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.784+0000 7f6c6c611640 1 -- 172.21.15.5:0/2770164608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64100e20 msgr2=0x7f6c640fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.784+0000 7f6c6c611640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64100e20 0x7f6c640fcc80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000d770 tx=0x7f6c6000dc30 comp rx=0 tx=0).stop 2024-09-19T06:11:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.784+0000 7f6c6c611640 1 -- 172.21.15.5:0/2770164608 shutdown_connections 2024-09-19T06:11:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.784+0000 7f6c6c611640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64102ef0 0x7f6c640fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.785+0000 7f6c6c611640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6c500a33b0 0x7f6c500a5870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.785+0000 7f6c6c611640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64102570 0x7f6c640fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.785+0000 7f6c6c611640 1 --2- 172.21.15.5:0/2770164608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64100e20 0x7f6c640fcc80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.785+0000 7f6c6c611640 1 -- 172.21.15.5:0/2770164608 >> 172.21.15.5:0/2770164608 conn(0x7f6c640fc7d0 msgr2=0x7f6c6406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.785+0000 7f6c6c611640 1 -- 172.21.15.5:0/2770164608 shutdown_connections 2024-09-19T06:11:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:11.785+0000 7f6c6c611640 1 -- 172.21.15.5:0/2770164608 wait complete. 2024-09-19T06:11:11.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 201934 2.72 MB/sec execute 277 sec latency 49.707 ms 2024-09-19T06:11:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.329+0000 7f9946835640 1 -- 172.21.15.5:0/1399132510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9940101340 msgr2=0x7f994010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.329+0000 7f9946835640 1 --2- 172.21.15.5:0/1399132510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9940101340 0x7f994010d870 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9938009cd0 tx=0x7f993802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.329+0000 7f9946835640 1 -- 172.21.15.5:0/1399132510 shutdown_connections 2024-09-19T06:11:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.329+0000 7f9946835640 1 --2- 172.21.15.5:0/1399132510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9940101340 0x7f994010d870 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.329+0000 7f9946835640 1 --2- 172.21.15.5:0/1399132510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99401009a0 0x7f9940100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.329+0000 7f9946835640 1 --2- 172.21.15.5:0/1399132510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99401069b0 0x7f9940106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.329+0000 7f9946835640 1 -- 172.21.15.5:0/1399132510 >> 172.21.15.5:0/1399132510 conn(0x7f99400779a0 msgr2=0x7f99400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.329+0000 7f9946835640 1 -- 172.21.15.5:0/1399132510 shutdown_connections 2024-09-19T06:11:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.329+0000 7f9946835640 1 -- 172.21.15.5:0/1399132510 wait complete. 2024-09-19T06:11:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.330+0000 7f9946835640 1 Processor -- start 2024-09-19T06:11:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.330+0000 7f9946835640 1 -- start start 2024-09-19T06:11:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.330+0000 7f9946835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401009a0 0x7f99401982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.330+0000 7f9946835640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9940101340 0x7f99401987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.330+0000 7f9946835640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99401069b0 0x7f994019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.330+0000 7f9946835640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9940110ca0 con 0x7f99401009a0 2024-09-19T06:11:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.330+0000 7f9946835640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9940110b20 con 0x7f9940101340 2024-09-19T06:11:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.330+0000 7f9946835640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9940110e20 con 0x7f99401069b0 2024-09-19T06:11:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401009a0 0x7f99401982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401009a0 0x7f99401982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44008/0 (socket says 172.21.15.5:44008) 2024-09-19T06:11:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993ffff640 1 -- 172.21.15.5:0/1552326374 learned_addr learned my addr 172.21.15.5:0/1552326374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9940101340 0x7f99401987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f9944dd2640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99401069b0 0x7f994019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993ffff640 1 -- 172.21.15.5:0/1552326374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99401069b0 msgr2=0x7f994019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993ffff640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99401069b0 0x7f994019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993ffff640 1 -- 172.21.15.5:0/1552326374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9940101340 msgr2=0x7f99401987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993ffff640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9940101340 0x7f99401987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993ffff640 1 -- 172.21.15.5:0/1552326374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994019d2a0 con 0x7f99401009a0 2024-09-19T06:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993f7fe640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9940101340 0x7f99401987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993ffff640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401009a0 0x7f99401982b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f993000c920 tx=0x7f993000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993d7fa640 1 -- 172.21.15.5:0/1552326374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930007c70 con 0x7f99401009a0 2024-09-19T06:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f993d7fa640 1 -- 172.21.15.5:0/1552326374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9930007e10 con 0x7f99401009a0 2024-09-19T06:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.331+0000 7f9946835640 1 -- 172.21.15.5:0/1552326374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994019d590 con 0x7f99401009a0 2024-09-19T06:11:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.332+0000 7f993d7fa640 1 -- 172.21.15.5:0/1552326374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930018820 con 0x7f99401009a0 2024-09-19T06:11:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.332+0000 7f9946835640 1 -- 172.21.15.5:0/1552326374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99401a4e20 con 0x7f99401009a0 2024-09-19T06:11:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f99495d0740 1 Processor -- start 2024-09-19T06:11:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f99495d0740 1 -- start start 2024-09-19T06:11:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f993d7fa640 1 -- 172.21.15.5:0/1552326374 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f99300189c0 con 0x7f99401009a0 2024-09-19T06:11:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f99495d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 0x55bbd20bc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f99495d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bc820 0x55bbd20c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f99495d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20c70c0 0x55bbd20c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f99495d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bbd201c2f0 con 0x55bbd20c70c0 2024-09-19T06:11:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f99495d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bbd1fc4270 con 0x55bbd20bbf00 2024-09-19T06:11:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f99495d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bbd20182b0 con 0x55bbd20bc820 2024-09-19T06:11:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f9944dd2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 0x55bbd20bc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20c70c0 0x55bbd20c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f9944dd2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 0x55bbd20bc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60992/0 (socket says 172.21.15.5:60992) 2024-09-19T06:11:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f9944dd2640 1 -- 172.21.15.5:0/619661409 learned_addr learned my addr 172.21.15.5:0/619661409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.335+0000 7f993ffff640 1 --2- 172.21.15.5:0/619661409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bc820 0x55bbd20c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.336+0000 7f9944dd2640 1 -- 172.21.15.5:0/619661409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bc820 msgr2=0x55bbd20c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.336+0000 7f9944dd2640 1 --2- 172.21.15.5:0/619661409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bc820 0x55bbd20c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.336+0000 7f9944dd2640 1 -- 172.21.15.5:0/619661409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20c70c0 msgr2=0x55bbd20c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.336+0000 7f9944dd2640 1 --2- 172.21.15.5:0/619661409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20c70c0 0x55bbd20c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.336+0000 7f9944dd2640 1 -- 172.21.15.5:0/619661409 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbd1fc3340 con 0x55bbd20bbf00 2024-09-19T06:11:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.336+0000 7f9944dd2640 1 --2- 172.21.15.5:0/619661409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 0x55bbd20bc2e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9938009cd0 tx=0x7f9938009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=663a2392cd03f0f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.336+0000 7f993d7fa640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f99280a33b0 0x7f99280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.336+0000 7f99267fc640 1 -- 172.21.15.5:0/619661409 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938049070 con 0x55bbd20bbf00 2024-09-19T06:11:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.336+0000 7f993ffff640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f99280a33b0 0x7f99280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.336+0000 7f99267fc640 1 -- 172.21.15.5:0/619661409 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9938036040 con 0x55bbd20bbf00 2024-09-19T06:11:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.336+0000 7f993d7fa640 1 -- 172.21.15.5:0/1552326374 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f99300cfd50 con 0x7f99401009a0 2024-09-19T06:11:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.336+0000 7f99267fc640 1 -- 172.21.15.5:0/619661409 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99380042a0 con 0x55bbd20bbf00 2024-09-19T06:11:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.337+0000 7f99495d0740 1 -- 172.21.15.5:0/619661409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 msgr2=0x55bbd20bc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.337+0000 7f99495d0740 1 --2- 172.21.15.5:0/619661409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 0x55bbd20bc2e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9938009cd0 tx=0x7f9938009a20 comp rx=0 tx=0).stop 2024-09-19T06:11:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.337+0000 7f993ffff640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f99280a33b0 0x7f99280a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f99300cd530 tx=0x7f99300d38a0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.337+0000 7f99495d0740 1 -- 172.21.15.5:0/619661409 shutdown_connections 2024-09-19T06:11:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.337+0000 7f99495d0740 1 --2- 172.21.15.5:0/619661409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20c70c0 0x55bbd20c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.337+0000 7f99495d0740 1 --2- 172.21.15.5:0/619661409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bc820 0x55bbd20c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.337+0000 7f99495d0740 1 --2- 172.21.15.5:0/619661409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 0x55bbd20bc2e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.337+0000 7f99495d0740 1 -- 172.21.15.5:0/619661409 >> 172.21.15.5:0/619661409 conn(0x55bbd20b8470 msgr2=0x55bbd20b8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.337+0000 7f99495d0740 1 -- 172.21.15.5:0/619661409 shutdown_connections 2024-09-19T06:11:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.337+0000 7f99495d0740 1 -- 172.21.15.5:0/619661409 wait complete. 2024-09-19T06:11:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.337+0000 7f99495d0740 1 -- start start 2024-09-19T06:11:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f99495d0740 1 Processor -- start 2024-09-19T06:11:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f99495d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 0x55bbd20c4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f99495d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20c4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f99495d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20c70c0 0x55bbd20c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f99495d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bbd20c9c90 con 0x55bbd20bc820 2024-09-19T06:11:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f99495d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bbd1fa5290 con 0x55bbd20bbf00 2024-09-19T06:11:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f99495d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bbd20c9e10 con 0x55bbd20c70c0 2024-09-19T06:11:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f99495d0740 10 client.0 _ll_register_callbacks cb 0x55bbd1fd6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f9944dd2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 0x55bbd20c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20c4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44018/0 (socket says 172.21.15.5:44018) 2024-09-19T06:11:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f993f7fe640 1 -- 172.21.15.5:0/1811763119 learned_addr learned my addr 172.21.15.5:0/1811763119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.338+0000 7f9944dd2640 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20c70c0 0x55bbd20c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f993f7fe640 1 -- 172.21.15.5:0/1811763119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20c70c0 msgr2=0x55bbd20c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f993f7fe640 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20c70c0 0x55bbd20c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f993f7fe640 1 -- 172.21.15.5:0/1811763119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 msgr2=0x55bbd20c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f993f7fe640 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 0x55bbd20c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f993f7fe640 1 -- 172.21.15.5:0/1811763119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbd1d2f210 con 0x55bbd20bc820 2024-09-19T06:11:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f9944dd2640 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 0x55bbd20c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f993f7fe640 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20c4570 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9934012d20 tx=0x7f993400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761bfd59f6be0b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f991f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f991f7fe640 1 -- 172.21.15.5:0/1811763119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934012110 con 0x55bbd20bc820 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f991f7fe640 1 -- 172.21.15.5:0/1811763119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f9934004050 con 0x55bbd20bc820 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f99495d0740 1 -- 172.21.15.5:0/1811763119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9938036040 con 0x55bbd20bc820 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f991f7fe640 1 -- 172.21.15.5:0/1811763119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f99340122b0 con 0x55bbd20bc820 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f99495d0740 10 client.17406 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f99495d0740 1 -- 172.21.15.5:0/1811763119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bbd1dfb620 con 0x55bbd20bc820 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.339+0000 7f991f7fe640 1 -- 172.21.15.5:0/1811763119 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9938036040 con 0x55bbd20bc820 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.340+0000 7f991f7fe640 1 -- 172.21.15.5:0/1811763119 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f9934010430 con 0x55bbd20bc820 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.340+0000 7f99495d0740 10 client.17406 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.340+0000 7f99495d0740 10 client.17406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.340+0000 7f99495d0740 20 client.17406 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.340+0000 7f99495d0740 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bbd20d7130 0x55bbd20d9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.340+0000 7f99495d0740 4 client.17406 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.340+0000 7f99495d0740 1 -- 172.21.15.5:0/1811763119 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbd2029a20 con 0x55bbd20d7130 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.340+0000 7f9944dd2640 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bbd20d7130 0x55bbd20d9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.340+0000 7f991f7fe640 10 client.17406 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.340+0000 7f9944dd2640 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bbd20d7130 0x55bbd20d9520 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879ff3018d985998 server_cookie=8065a49fa0a183da in_seq=0 out_seq=0 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f991f7fe640 1 -- 172.21.15.5:0/1811763119 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbd2029a20 con 0x55bbd20d7130 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f991f7fe640 10 client.17406 handle_command_reply: tid=0 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f99495d0740 1 client.17406 shutdown 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f99495d0740 1 -- 172.21.15.5:0/1811763119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 msgr2=0x55bbd20c4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f99495d0740 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20c4570 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9934012d20 tx=0x7f993400f790 comp rx=0 tx=0).stop 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f99495d0740 1 -- 172.21.15.5:0/1811763119 shutdown_connections 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f99495d0740 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bbd20d7130 0x55bbd20d9520 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f99495d0740 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20c70c0 0x55bbd20c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f991f7fe640 0 client.17406 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f99495d0740 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20c4570 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f99495d0740 1 --2- 172.21.15.5:0/1811763119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20bbf00 0x55bbd20c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f99495d0740 1 -- 172.21.15.5:0/1811763119 >> 172.21.15.5:0/1811763119 conn(0x55bbd20b8470 msgr2=0x55bbd20b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f99495d0740 1 -- 172.21.15.5:0/1811763119 shutdown_connections 2024-09-19T06:11:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f99495d0740 1 -- 172.21.15.5:0/1811763119 wait complete. 2024-09-19T06:11:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.341+0000 7f99495d0740 20 client.17406 trim_cache size 0 max 16384 2024-09-19T06:11:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.357+0000 7f99495d0740 1 Processor -- start 2024-09-19T06:11:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.357+0000 7f99495d0740 1 -- start start 2024-09-19T06:11:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f99495d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 0x55bbd20be970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f99495d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f99495d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20c70c0 0x55bbd20dee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f99495d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f99401ab4b0 con 0x55bbd20bc820 2024-09-19T06:11:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f99495d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f99401acff0 con 0x55bbd20c70c0 2024-09-19T06:11:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f99495d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99401abb80 con 0x55bbd20bbf00 2024-09-19T06:11:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f9944dd2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 0x55bbd20be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44022/0 (socket says 172.21.15.5:44022) 2024-09-19T06:11:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f993f7fe640 1 -- 172.21.15.5:0/1491639602 learned_addr learned my addr 172.21.15.5:0/1491639602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f9944dd2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 0x55bbd20be970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47118/0 (socket says 172.21.15.5:47118) 2024-09-19T06:11:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f9944dd2640 1 --2- 172.21.15.5:0/1491639602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20c70c0 0x55bbd20dee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f993f7fe640 1 -- 172.21.15.5:0/1491639602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 msgr2=0x55bbd20be970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f993f7fe640 1 --2- 172.21.15.5:0/1491639602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 0x55bbd20be970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f993f7fe640 1 -- 172.21.15.5:0/1491639602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20c70c0 msgr2=0x55bbd20dee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f993f7fe640 1 --2- 172.21.15.5:0/1491639602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20c70c0 0x55bbd20dee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f993f7fe640 1 -- 172.21.15.5:0/1491639602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbd20d9890 con 0x55bbd20bc820 2024-09-19T06:11:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f993f7fe640 1 --2- 172.21.15.5:0/1491639602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20c5fa0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9934012cf0 tx=0x7f9934009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a627e0146f6c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f9925ffb640 1 -- 172.21.15.5:0/1491639602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934029070 con 0x55bbd20bc820 2024-09-19T06:11:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f9925ffb640 1 -- 172.21.15.5:0/1491639602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f993400fe80 con 0x55bbd20bc820 2024-09-19T06:11:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.358+0000 7f9925ffb640 1 -- 172.21.15.5:0/1491639602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99340035f0 con 0x55bbd20bc820 2024-09-19T06:11:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.359+0000 7f99495d0740 1 -- 172.21.15.5:0/1491639602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 msgr2=0x55bbd20c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.359+0000 7f99495d0740 1 --2- 172.21.15.5:0/1491639602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20c5fa0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9934012cf0 tx=0x7f9934009c00 comp rx=0 tx=0).stop 2024-09-19T06:11:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.359+0000 7f99495d0740 1 -- 172.21.15.5:0/1491639602 shutdown_connections 2024-09-19T06:11:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.359+0000 7f99495d0740 1 --2- 172.21.15.5:0/1491639602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20c70c0 0x55bbd20dee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.359+0000 7f99495d0740 1 --2- 172.21.15.5:0/1491639602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20c5fa0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.359+0000 7f99495d0740 1 --2- 172.21.15.5:0/1491639602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 0x55bbd20be970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.359+0000 7f99495d0740 1 -- 172.21.15.5:0/1491639602 >> 172.21.15.5:0/1491639602 conn(0x55bbd20b8470 msgr2=0x55bbd20c5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.359+0000 7f99495d0740 1 -- 172.21.15.5:0/1491639602 shutdown_connections 2024-09-19T06:11:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.359+0000 7f99495d0740 1 -- 172.21.15.5:0/1491639602 wait complete. 2024-09-19T06:11:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f99495d0740 1 -- start start 2024-09-19T06:11:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f99495d0740 1 Processor -- start 2024-09-19T06:11:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f99495d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 0x55bbd20cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f99495d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f99495d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20c70c0 0x55bbd20c0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f99495d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f99401ac920 con 0x55bbd20bc820 2024-09-19T06:11:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f99495d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f99401ac250 con 0x55bbd20c70c0 2024-09-19T06:11:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f99495d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99401a94b0 con 0x55bbd20bbf00 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f99495d0740 10 client.0 _ll_register_callbacks cb 0x7f9934009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f9944dd2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 0x55bbd20cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f9944dd2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 0x55bbd20cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47122/0 (socket says 172.21.15.5:47122) 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44038/0 (socket says 172.21.15.5:44038) 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.360+0000 7f9944dd2640 1 -- 172.21.15.5:0/2809534443 learned_addr learned my addr 172.21.15.5:0/2809534443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f9944dd2640 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20c70c0 0x55bbd20c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f993f7fe640 1 -- 172.21.15.5:0/2809534443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 msgr2=0x55bbd20cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f993f7fe640 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 0x55bbd20cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f993f7fe640 1 -- 172.21.15.5:0/2809534443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20c70c0 msgr2=0x55bbd20c0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f993f7fe640 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20c70c0 0x55bbd20c0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f993f7fe640 1 -- 172.21.15.5:0/2809534443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbd20d3a70 con 0x55bbd20bc820 2024-09-19T06:11:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f9944dd2640 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 0x55bbd20cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f993f7fe640 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20cf5e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55bbd20898a0 tx=0x7f9934010780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d3df1cba6dc135 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f991f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f991f7fe640 1 -- 172.21.15.5:0/2809534443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934029030 con 0x55bbd20bc820 2024-09-19T06:11:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f991f7fe640 1 -- 172.21.15.5:0/2809534443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f99340050b0 con 0x55bbd20bc820 2024-09-19T06:11:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f99495d0740 1 -- 172.21.15.5:0/2809534443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bbd2023d70 con 0x55bbd20bc820 2024-09-19T06:11:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f991f7fe640 1 -- 172.21.15.5:0/2809534443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f99340052b0 con 0x55bbd20bc820 2024-09-19T06:11:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f99495d0740 10 client.17412 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.361+0000 7f99495d0740 1 -- 172.21.15.5:0/2809534443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bbd2141470 con 0x55bbd20bc820 2024-09-19T06:11:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.362+0000 7f991f7fe640 1 -- 172.21.15.5:0/2809534443 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55bbd2023d70 con 0x55bbd20bc820 2024-09-19T06:11:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.362+0000 7f991f7fe640 1 -- 172.21.15.5:0/2809534443 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f9934025450 con 0x55bbd20bc820 2024-09-19T06:11:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.362+0000 7f99495d0740 10 client.17412 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.362+0000 7f99495d0740 10 client.17412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.362+0000 7f99495d0740 20 client.17412 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.362+0000 7f99495d0740 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bbd2141ad0 0x55bbd2143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.362+0000 7f99495d0740 4 client.17412 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:11:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.362+0000 7f99495d0740 1 -- 172.21.15.5:0/2809534443 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbd1fa0900 con 0x55bbd2141ad0 2024-09-19T06:11:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.362+0000 7f9944dd2640 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bbd2141ad0 0x55bbd2143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.362+0000 7f9944dd2640 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bbd2141ad0 0x55bbd2143ef0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c88a3fe5b6ac4708 server_cookie=124f8c58ffd07129 in_seq=0 out_seq=0 2024-09-19T06:11:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.362+0000 7f991f7fe640 10 client.17412 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.385+0000 7f991f7fe640 1 -- 172.21.15.5:0/2809534443 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bbd1fa0900 con 0x55bbd2141ad0 2024-09-19T06:11:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.385+0000 7f991f7fe640 10 client.17412 handle_command_reply: tid=0 2024-09-19T06:11:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.385+0000 7f99495d0740 1 client.17412 shutdown 2024-09-19T06:11:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f99495d0740 1 -- 172.21.15.5:0/2809534443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 msgr2=0x55bbd20cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f99495d0740 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20cf5e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55bbd20898a0 tx=0x7f9934010780 comp rx=0 tx=0).stop 2024-09-19T06:11:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f99495d0740 1 -- 172.21.15.5:0/2809534443 shutdown_connections 2024-09-19T06:11:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f99495d0740 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bbd2141ad0 0x55bbd2143ef0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f99495d0740 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bbd20c70c0 0x55bbd20c0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f991f7fe640 0 client.17412 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f99495d0740 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbd20bc820 0x55bbd20cf5e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f99495d0740 1 --2- 172.21.15.5:0/2809534443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbd20bbf00 0x55bbd20cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f99495d0740 1 -- 172.21.15.5:0/2809534443 >> 172.21.15.5:0/2809534443 conn(0x55bbd20b8470 msgr2=0x55bbd20c5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f99495d0740 1 -- 172.21.15.5:0/2809534443 shutdown_connections 2024-09-19T06:11:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f99495d0740 1 -- 172.21.15.5:0/2809534443 wait complete. 2024-09-19T06:11:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f99495d0740 20 client.17412 trim_cache size 0 max 16384 2024-09-19T06:11:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:11:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:11:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:11:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:11:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f9946835640 1 -- 172.21.15.5:0/1552326374 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f99280a33b0 msgr2=0x7f99280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f9946835640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f99280a33b0 0x7f99280a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f99300cd530 tx=0x7f99300d38a0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f9946835640 1 -- 172.21.15.5:0/1552326374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401009a0 msgr2=0x7f99401982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.386+0000 7f9946835640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401009a0 0x7f99401982b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f993000c920 tx=0x7f993000cde0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.387+0000 7f9946835640 1 -- 172.21.15.5:0/1552326374 shutdown_connections 2024-09-19T06:11:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.387+0000 7f9946835640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f99280a33b0 0x7f99280a5870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.387+0000 7f9946835640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99401069b0 0x7f994019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.387+0000 7f9946835640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9940101340 0x7f99401987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.387+0000 7f9946835640 1 --2- 172.21.15.5:0/1552326374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401009a0 0x7f99401982b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.387+0000 7f9946835640 1 -- 172.21.15.5:0/1552326374 >> 172.21.15.5:0/1552326374 conn(0x7f99400779a0 msgr2=0x7f9940069ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.387+0000 7f9946835640 1 -- 172.21.15.5:0/1552326374 shutdown_connections 2024-09-19T06:11:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:12.387+0000 7f9946835640 1 -- 172.21.15.5:0/1552326374 wait complete. 2024-09-19T06:11:12.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:12 smithi005 ceph-mon[30955]: pgmap v243: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.1 MiB/s wr, 90 op/s 2024-09-19T06:11:12.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:12 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2770164608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:11:12.591 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:11:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:12 smithi157 ceph-mon[35346]: pgmap v243: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.1 MiB/s wr, 90 op/s 2024-09-19T06:11:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:12 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2770164608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:11:12.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 202398 2.72 MB/sec execute 278 sec latency 15.660 ms 2024-09-19T06:11:12.882 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:11:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:12 smithi049 ceph-mon[34640]: pgmap v243: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.1 MiB/s wr, 90 op/s 2024-09-19T06:11:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:12 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2770164608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:11:13.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.394+0000 7f3470807640 1 -- 172.21.15.5:0/2282892986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f346c102d90 msgr2=0x7f346c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:13.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.394+0000 7f3470807640 1 --2- 172.21.15.5:0/2282892986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f346c102d90 0x7f346c103170 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f345c009930 tx=0x7f345c02a420 comp rx=0 tx=0).stop 2024-09-19T06:11:13.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.394+0000 7f3470807640 1 -- 172.21.15.5:0/2282892986 shutdown_connections 2024-09-19T06:11:13.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.394+0000 7f3470807640 1 --2- 172.21.15.5:0/2282892986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f346c108290 0x7f346c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.394+0000 7f3470807640 1 --2- 172.21.15.5:0/2282892986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346c103740 0x7f346c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.394+0000 7f3470807640 1 --2- 172.21.15.5:0/2282892986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f346c102d90 0x7f346c103170 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.394+0000 7f3470807640 1 -- 172.21.15.5:0/2282892986 >> 172.21.15.5:0/2282892986 conn(0x7f346c0fc7d0 msgr2=0x7f346c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.394+0000 7f3470807640 1 -- 172.21.15.5:0/2282892986 shutdown_connections 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.394+0000 7f3470807640 1 -- 172.21.15.5:0/2282892986 wait complete. 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.395+0000 7f3470807640 1 Processor -- start 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f3470807640 1 -- start start 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f3470807640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346c102d90 0x7f346c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f3470807640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f346c103740 0x7f346c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f3470807640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f346c108290 0x7f346c101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f3470807640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f346c111f60 con 0x7f346c108290 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f3470807640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f346c111de0 con 0x7f346c103740 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f3470807640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1120e0 con 0x7f346c102d90 2024-09-19T06:11:13.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f346a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346c102d90 0x7f346c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f346ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f346c108290 0x7f346c101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f3469d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f346c103740 0x7f346c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f346a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346c102d90 0x7f346c1009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47148/0 (socket says 172.21.15.5:47148) 2024-09-19T06:11:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f346a59c640 1 -- 172.21.15.5:0/3532260505 learned_addr learned my addr 172.21.15.5:0/3532260505 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f346a59c640 1 -- 172.21.15.5:0/3532260505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f346c103740 msgr2=0x7f346c100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f346a59c640 1 --2- 172.21.15.5:0/3532260505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f346c103740 0x7f346c100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.396+0000 7f346a59c640 1 -- 172.21.15.5:0/3532260505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f346c108290 msgr2=0x7f346c101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.397+0000 7f346a59c640 1 --2- 172.21.15.5:0/3532260505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f346c108290 0x7f346c101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.397+0000 7f346a59c640 1 -- 172.21.15.5:0/3532260505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c1a6990 con 0x7f346c102d90 2024-09-19T06:11:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.397+0000 7f346a59c640 1 --2- 172.21.15.5:0/3532260505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346c102d90 0x7f346c1009f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f345c005ab0 tx=0x7f345c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.397+0000 7f345b7fe640 1 -- 172.21.15.5:0/3532260505 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c03f070 con 0x7f346c102d90 2024-09-19T06:11:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.397+0000 7f345b7fe640 1 -- 172.21.15.5:0/3532260505 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f345c031b50 con 0x7f346c102d90 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.397+0000 7f3470807640 1 -- 172.21.15.5:0/3532260505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346c1a6c20 con 0x7f346c102d90 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.398+0000 7f345b7fe640 1 -- 172.21.15.5:0/3532260505 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c035030 con 0x7f346c102d90 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.398+0000 7f3470807640 1 -- 172.21.15.5:0/3532260505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346c1a7150 con 0x7f346c102d90 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.400+0000 7f345b7fe640 1 -- 172.21.15.5:0/3532260505 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f345c042070 con 0x7f346c102d90 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.400+0000 7f3472be0740 1 Processor -- start 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f3472be0740 1 -- start start 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f3472be0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f5b40 0x5605cc4f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f3472be0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 0x5605cc4f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f3472be0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc501760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f3472be0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5605cc45a2f0 con 0x5605cc4f45d0 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f3472be0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5605cc401cf0 con 0x5605cc4f5b40 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f3472be0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5605cc4562b0 con 0x5605cc4ff370 2024-09-19T06:11:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f346ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f5b40 0x5605cc4f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f346a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 0x5605cc4f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f346ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f5b40 0x5605cc4f4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:32806/0 (socket says 172.21.15.5:32806) 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f346ad9d640 1 -- 172.21.15.5:0/1535893982 learned_addr learned my addr 172.21.15.5:0/1535893982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f345b7fe640 1 --2- 172.21.15.5:0/3532260505 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f34540a3430 0x7f34540a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.401+0000 7f3469d9b640 1 --2- 172.21.15.5:0/1535893982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc501760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.402+0000 7f346a59c640 1 -- 172.21.15.5:0/1535893982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 msgr2=0x5605cc501760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.402+0000 7f346a59c640 1 --2- 172.21.15.5:0/1535893982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc501760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.402+0000 7f346a59c640 1 -- 172.21.15.5:0/1535893982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f5b40 msgr2=0x5605cc4f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.402+0000 7f346a59c640 1 --2- 172.21.15.5:0/1535893982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f5b40 0x5605cc4f4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.402+0000 7f346ad9d640 1 --2- 172.21.15.5:0/3532260505 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f34540a3430 0x7f34540a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.402+0000 7f345b7fe640 1 -- 172.21.15.5:0/3532260505 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f345c0f02a0 con 0x7f346c102d90 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.402+0000 7f346a59c640 1 -- 172.21.15.5:0/1535893982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605cc3e63a0 con 0x5605cc4f45d0 2024-09-19T06:11:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.402+0000 7f346a59c640 1 --2- 172.21.15.5:0/1535893982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 0x5605cc4f4a10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f345c0ed970 tx=0x7f345c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4cd429b4ca4394 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.402+0000 7f3458ff9640 1 -- 172.21.15.5:0/1535893982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c049070 con 0x5605cc4f45d0 2024-09-19T06:11:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.402+0000 7f346ad9d640 1 --2- 172.21.15.5:0/3532260505 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f34540a3430 0x7f34540a58f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3464011a00 tx=0x7f3464004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.402+0000 7f3458ff9640 1 -- 172.21.15.5:0/1535893982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f345c0f3e40 con 0x5605cc4f45d0 2024-09-19T06:11:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.402+0000 7f3458ff9640 1 -- 172.21.15.5:0/1535893982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c0c08b0 con 0x5605cc4f45d0 2024-09-19T06:11:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.403+0000 7f3472be0740 1 -- 172.21.15.5:0/1535893982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 msgr2=0x5605cc4f4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.403+0000 7f3472be0740 1 --2- 172.21.15.5:0/1535893982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 0x5605cc4f4a10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f345c0ed970 tx=0x7f345c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.403+0000 7f3472be0740 1 -- 172.21.15.5:0/1535893982 shutdown_connections 2024-09-19T06:11:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.403+0000 7f3472be0740 1 --2- 172.21.15.5:0/1535893982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc501760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.403+0000 7f3472be0740 1 --2- 172.21.15.5:0/1535893982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 0x5605cc4f4a10 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.403+0000 7f3472be0740 1 --2- 172.21.15.5:0/1535893982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f5b40 0x5605cc4f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.403+0000 7f3472be0740 1 -- 172.21.15.5:0/1535893982 >> 172.21.15.5:0/1535893982 conn(0x5605cc4bd310 msgr2=0x5605cc4f0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.403+0000 7f3472be0740 1 -- 172.21.15.5:0/1535893982 shutdown_connections 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.403+0000 7f3472be0740 1 -- 172.21.15.5:0/1535893982 wait complete. 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.403+0000 7f3472be0740 1 -- start start 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f3472be0740 1 Processor -- start 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f3472be0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 0x5605cc4fed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f3472be0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f5b40 0x5605cc4fc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f3472be0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc4fc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f346a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 0x5605cc4fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f3472be0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5605cc501e30 con 0x5605cc4f45d0 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f3472be0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5605cc415060 con 0x5605cc4f5b40 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f3472be0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5605cc501fb0 con 0x5605cc4ff370 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f346a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 0x5605cc4fed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44076/0 (socket says 172.21.15.5:44076) 2024-09-19T06:11:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f346a59c640 1 -- 172.21.15.5:0/3945540171 learned_addr learned my addr 172.21.15.5:0/3945540171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f3469d9b640 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f5b40 0x5605cc4fc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f3472be0740 10 client.0 _ll_register_callbacks cb 0x5605cc1688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f346ad9d640 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc4fc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f346a59c640 1 -- 172.21.15.5:0/3945540171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 msgr2=0x5605cc4fc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f346a59c640 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc4fc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f346a59c640 1 -- 172.21.15.5:0/3945540171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f5b40 msgr2=0x5605cc4fc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f346a59c640 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f5b40 0x5605cc4fc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f346a59c640 1 -- 172.21.15.5:0/3945540171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605cc1cb3c0 con 0x5605cc4f45d0 2024-09-19T06:11:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.404+0000 7f346ad9d640 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc4fc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:11:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.405+0000 7f346a59c640 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 0x5605cc4fed50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f345c0c4c40 tx=0x7f345c03b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd4578fa8311469 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.405+0000 7f3451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.405+0000 7f3451ffb640 1 -- 172.21.15.5:0/3945540171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c049050 con 0x5605cc4f45d0 2024-09-19T06:11:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.405+0000 7f3451ffb640 1 -- 172.21.15.5:0/3945540171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f345c0b8c20 con 0x5605cc4f45d0 2024-09-19T06:11:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.405+0000 7f3472be0740 1 -- 172.21.15.5:0/3945540171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5605cc467860 con 0x5605cc4f45d0 2024-09-19T06:11:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.405+0000 7f3451ffb640 1 -- 172.21.15.5:0/3945540171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f345c0bc0f0 con 0x5605cc4f45d0 2024-09-19T06:11:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.405+0000 7f3472be0740 10 client.17424 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.405+0000 7f3472be0740 1 -- 172.21.15.5:0/3945540171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5605cc501e30 con 0x5605cc4f45d0 2024-09-19T06:11:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.406+0000 7f3451ffb640 1 -- 172.21.15.5:0/3945540171 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5605cc467860 con 0x5605cc4f45d0 2024-09-19T06:11:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.406+0000 7f3451ffb640 1 -- 172.21.15.5:0/3945540171 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f345c0bc810 con 0x5605cc4f45d0 2024-09-19T06:11:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.406+0000 7f3472be0740 10 client.17424 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.406+0000 7f3472be0740 10 client.17424 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:11:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.406+0000 7f3472be0740 20 client.17424 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.406+0000 7f3472be0740 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5605cc517220 0x5605cc519660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.406+0000 7f3472be0740 4 client.17424 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:11:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.406+0000 7f3472be0740 1 -- 172.21.15.5:0/3945540171 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605cc461d70 con 0x5605cc517220 2024-09-19T06:11:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.406+0000 7f3469d9b640 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5605cc517220 0x5605cc519660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.407+0000 7f3469d9b640 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5605cc517220 0x5605cc519660 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b0b04913c85d1e36 server_cookie=9dcc19b50e29bc5f in_seq=0 out_seq=0 2024-09-19T06:11:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.407+0000 7f3451ffb640 10 client.17424 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:11:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.407+0000 7f3451ffb640 1 -- 172.21.15.5:0/3945540171 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605cc461d70 con 0x5605cc517220 2024-09-19T06:11:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.407+0000 7f3451ffb640 10 client.17424 handle_command_reply: tid=0 2024-09-19T06:11:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.407+0000 7f3472be0740 1 client.17424 shutdown 2024-09-19T06:11:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.407+0000 7f3472be0740 1 -- 172.21.15.5:0/3945540171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 msgr2=0x5605cc4fed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.407+0000 7f3472be0740 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 0x5605cc4fed50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f345c0c4c40 tx=0x7f345c03b690 comp rx=0 tx=0).stop 2024-09-19T06:11:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.407+0000 7f3472be0740 1 -- 172.21.15.5:0/3945540171 shutdown_connections 2024-09-19T06:11:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.407+0000 7f3472be0740 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5605cc517220 0x5605cc519660 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.407+0000 7f3472be0740 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc4fc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.407+0000 7f3472be0740 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f5b40 0x5605cc4fc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.407+0000 7f3472be0740 1 --2- 172.21.15.5:0/3945540171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f45d0 0x5605cc4fed50 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.408+0000 7f3472be0740 1 -- 172.21.15.5:0/3945540171 >> 172.21.15.5:0/3945540171 conn(0x5605cc4bd310 msgr2=0x5605cc4f6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.408+0000 7f3451ffb640 0 client.17424 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:11:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.408+0000 7f3472be0740 1 -- 172.21.15.5:0/3945540171 shutdown_connections 2024-09-19T06:11:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.408+0000 7f3472be0740 1 -- 172.21.15.5:0/3945540171 wait complete. 2024-09-19T06:11:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.408+0000 7f3472be0740 20 client.17424 trim_cache size 0 max 16384 2024-09-19T06:11:13.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.422+0000 7f3472be0740 1 Processor -- start 2024-09-19T06:11:13.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.422+0000 7f3472be0740 1 -- start start 2024-09-19T06:11:13.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f3472be0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 0x5605cc50b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f3472be0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 0x5605cc4fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f3472be0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc4f7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f3472be0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ab6a0 con 0x5605cc4f5b40 2024-09-19T06:11:13.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f3472be0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ad1e0 con 0x5605cc4f45d0 2024-09-19T06:11:13.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f3472be0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1abd70 con 0x5605cc4ff370 2024-09-19T06:11:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f346a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 0x5605cc50b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f3469d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 0x5605cc4fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f346a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 0x5605cc50b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:32814/0 (socket says 172.21.15.5:32814) 2024-09-19T06:11:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f346a59c640 1 -- 172.21.15.5:0/2259617384 learned_addr learned my addr 172.21.15.5:0/2259617384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f3469d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 0x5605cc4fafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44092/0 (socket says 172.21.15.5:44092) 2024-09-19T06:11:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f346ad9d640 1 --2- 172.21.15.5:0/2259617384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc4f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f346a59c640 1 -- 172.21.15.5:0/2259617384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 msgr2=0x5605cc4f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f346a59c640 1 --2- 172.21.15.5:0/2259617384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc4f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f346a59c640 1 -- 172.21.15.5:0/2259617384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 msgr2=0x5605cc4fafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f346a59c640 1 --2- 172.21.15.5:0/2259617384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 0x5605cc4fafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f346a59c640 1 -- 172.21.15.5:0/2259617384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605cc4fc940 con 0x5605cc4f45d0 2024-09-19T06:11:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f346ad9d640 1 --2- 172.21.15.5:0/2259617384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc4f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:11:13.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.423+0000 7f3469d9b640 1 --2- 172.21.15.5:0/2259617384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 0x5605cc4fafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:13.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.424+0000 7f346a59c640 1 --2- 172.21.15.5:0/2259617384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 0x5605cc50b000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f345c0bc540 tx=0x7f345c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb1c0d71511ceff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:13.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.424+0000 7f3453fff640 1 -- 172.21.15.5:0/2259617384 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c049070 con 0x5605cc4f45d0 2024-09-19T06:11:13.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.424+0000 7f3453fff640 1 -- 172.21.15.5:0/2259617384 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f345c0f3b00 con 0x5605cc4f45d0 2024-09-19T06:11:13.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.424+0000 7f3453fff640 1 -- 172.21.15.5:0/2259617384 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c0bbf10 con 0x5605cc4f45d0 2024-09-19T06:11:13.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.424+0000 7f3472be0740 1 -- 172.21.15.5:0/2259617384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 msgr2=0x5605cc50b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:13.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.424+0000 7f3472be0740 1 --2- 172.21.15.5:0/2259617384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 0x5605cc50b000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f345c0bc540 tx=0x7f345c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 -- 172.21.15.5:0/2259617384 shutdown_connections 2024-09-19T06:11:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 --2- 172.21.15.5:0/2259617384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc4f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 --2- 172.21.15.5:0/2259617384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 0x5605cc4fafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 --2- 172.21.15.5:0/2259617384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 0x5605cc50b000 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 -- 172.21.15.5:0/2259617384 >> 172.21.15.5:0/2259617384 conn(0x5605cc4bd310 msgr2=0x5605cc4fa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 -- 172.21.15.5:0/2259617384 shutdown_connections 2024-09-19T06:11:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 -- 172.21.15.5:0/2259617384 wait complete. 2024-09-19T06:11:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 -- start start 2024-09-19T06:11:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 Processor -- start 2024-09-19T06:11:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 0x5605cc5020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 0x5605cc502620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc502b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1acb10 con 0x5605cc4f5b40 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.425+0000 7f3472be0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ac440 con 0x5605cc4f45d0 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3472be0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1a96a0 con 0x5605cc4ff370 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3469d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 0x5605cc502620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f346a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 0x5605cc5020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3469d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 0x5605cc502620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44098/0 (socket says 172.21.15.5:44098) 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f346a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 0x5605cc5020e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:32824/0 (socket says 172.21.15.5:32824) 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3469d9b640 1 -- 172.21.15.5:0/4216814948 learned_addr learned my addr 172.21.15.5:0/4216814948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3472be0740 10 client.0 _ll_register_callbacks cb 0x7f3464009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f346ad9d640 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc502b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3469d9b640 1 -- 172.21.15.5:0/4216814948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 msgr2=0x5605cc502b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3469d9b640 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc502b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3469d9b640 1 -- 172.21.15.5:0/4216814948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 msgr2=0x5605cc5020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3469d9b640 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 0x5605cc5020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3469d9b640 1 -- 172.21.15.5:0/4216814948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605cc5800a0 con 0x5605cc4f5b40 2024-09-19T06:11:13.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f346a59c640 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 0x5605cc5020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:13.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3469d9b640 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 0x5605cc502620 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x5605cc4f5070 tx=0x7f34600096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e45329fe8f44c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:13.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:13.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3451ffb640 1 -- 172.21.15.5:0/4216814948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460004310 con 0x5605cc4f5b40 2024-09-19T06:11:13.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3451ffb640 1 -- 172.21.15.5:0/4216814948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f34600035f0 con 0x5605cc4f5b40 2024-09-19T06:11:13.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.426+0000 7f3472be0740 1 -- 172.21.15.5:0/4216814948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f345c0f3b00 con 0x5605cc4f5b40 2024-09-19T06:11:13.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.427+0000 7f3451ffb640 1 -- 172.21.15.5:0/4216814948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f34600044b0 con 0x5605cc4f5b40 2024-09-19T06:11:13.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.427+0000 7f3472be0740 10 client.17430 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:13.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.427+0000 7f3472be0740 1 -- 172.21.15.5:0/4216814948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5605cc5813f0 con 0x5605cc4f5b40 2024-09-19T06:11:13.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.427+0000 7f3451ffb640 1 -- 172.21.15.5:0/4216814948 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f345c0f3b00 con 0x5605cc4f5b40 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.428+0000 7f3451ffb640 1 -- 172.21.15.5:0/4216814948 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3460027430 con 0x5605cc4f5b40 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.428+0000 7f3472be0740 10 client.17430 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.428+0000 7f3472be0740 10 client.17430 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.428+0000 7f3472be0740 20 client.17430 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.428+0000 7f3472be0740 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5605cc581bc0 0x5605cc583fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.428+0000 7f3472be0740 4 client.17430 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.428+0000 7f3472be0740 1 -- 172.21.15.5:0/4216814948 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605cc3de170 con 0x5605cc581bc0 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.428+0000 7f346a59c640 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5605cc581bc0 0x5605cc583fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.428+0000 7f346a59c640 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5605cc581bc0 0x5605cc583fe0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=892923c1b332b2ab server_cookie=ddc49bf836a8b34d in_seq=0 out_seq=0 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.428+0000 7f3451ffb640 10 client.17430 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3451ffb640 1 -- 172.21.15.5:0/4216814948 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605cc3de170 con 0x5605cc581bc0 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3451ffb640 10 client.17430 handle_command_reply: tid=0 2024-09-19T06:11:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3472be0740 1 client.17430 shutdown 2024-09-19T06:11:13.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3472be0740 1 -- 172.21.15.5:0/4216814948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 msgr2=0x5605cc502620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:13.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3472be0740 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 0x5605cc502620 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x5605cc4f5070 tx=0x7f34600096f0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3472be0740 1 -- 172.21.15.5:0/4216814948 shutdown_connections 2024-09-19T06:11:13.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3472be0740 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5605cc581bc0 0x5605cc583fe0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3472be0740 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605cc4ff370 0x5605cc502b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3472be0740 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605cc4f5b40 0x5605cc502620 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3472be0740 1 --2- 172.21.15.5:0/4216814948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605cc4f45d0 0x5605cc5020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3472be0740 1 -- 172.21.15.5:0/4216814948 >> 172.21.15.5:0/4216814948 conn(0x5605cc4bd310 msgr2=0x5605cc4fb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:13.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3451ffb640 0 client.17430 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:11:13.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3472be0740 1 -- 172.21.15.5:0/4216814948 shutdown_connections 2024-09-19T06:11:13.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3472be0740 1 -- 172.21.15.5:0/4216814948 wait complete. 2024-09-19T06:11:13.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.429+0000 7f3472be0740 20 client.17430 trim_cache size 0 max 16384 2024-09-19T06:11:13.437 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:11:13.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.430+0000 7f3470807640 1 -- 172.21.15.5:0/3532260505 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f34540a3430 msgr2=0x7f34540a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:13.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.430+0000 7f3470807640 1 --2- 172.21.15.5:0/3532260505 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f34540a3430 0x7f34540a58f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3464011a00 tx=0x7f3464004ab0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.430+0000 7f3470807640 1 -- 172.21.15.5:0/3532260505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346c102d90 msgr2=0x7f346c1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.430+0000 7f3470807640 1 --2- 172.21.15.5:0/3532260505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346c102d90 0x7f346c1009f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f345c005ab0 tx=0x7f345c003b90 comp rx=0 tx=0).stop 2024-09-19T06:11:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.430+0000 7f3470807640 1 -- 172.21.15.5:0/3532260505 shutdown_connections 2024-09-19T06:11:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.430+0000 7f3470807640 1 --2- 172.21.15.5:0/3532260505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f346c108290 0x7f346c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.430+0000 7f3470807640 1 --2- 172.21.15.5:0/3532260505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f346c103740 0x7f346c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.430+0000 7f3470807640 1 --2- 172.21.15.5:0/3532260505 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f34540a3430 0x7f34540a58f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.430+0000 7f3470807640 1 --2- 172.21.15.5:0/3532260505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346c102d90 0x7f346c1009f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.430+0000 7f3470807640 1 -- 172.21.15.5:0/3532260505 >> 172.21.15.5:0/3532260505 conn(0x7f346c0fc7d0 msgr2=0x7f346c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.430+0000 7f3470807640 1 -- 172.21.15.5:0/3532260505 shutdown_connections 2024-09-19T06:11:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:13.430+0000 7f3470807640 1 -- 172.21.15.5:0/3532260505 wait complete. 2024-09-19T06:11:13.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 202648 2.72 MB/sec execute 279 sec latency 12.384 ms 2024-09-19T06:11:14.091 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:11:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:11:14.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:14 smithi005 ceph-mon[30955]: pgmap v244: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-19T06:11:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:14 smithi157 ceph-mon[35346]: pgmap v244: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-19T06:11:14.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 203053 2.71 MB/sec execute 280 sec latency 18.881 ms 2024-09-19T06:11:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:14 smithi049 ceph-mon[34640]: pgmap v244: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-19T06:11:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.900+0000 7f61f29f2640 1 -- 172.21.15.5:0/3877118311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61ec1082a0 msgr2=0x7f61ec10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.900+0000 7f61f29f2640 1 --2- 172.21.15.5:0/3877118311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61ec1082a0 0x7f61ec10ec40 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009cd0 tx=0x7f61e402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.900+0000 7f61f29f2640 1 -- 172.21.15.5:0/3877118311 shutdown_connections 2024-09-19T06:11:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.900+0000 7f61f29f2640 1 --2- 172.21.15.5:0/3877118311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61ec1082a0 0x7f61ec10ec40 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.900+0000 7f61f29f2640 1 --2- 172.21.15.5:0/3877118311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61ec1036e0 0x7f61ec107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.900+0000 7f61f29f2640 1 --2- 172.21.15.5:0/3877118311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61ec102d30 0x7f61ec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.900+0000 7f61f29f2640 1 -- 172.21.15.5:0/3877118311 >> 172.21.15.5:0/3877118311 conn(0x7f61ec0fc7d0 msgr2=0x7f61ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:14.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.900+0000 7f61f29f2640 1 -- 172.21.15.5:0/3877118311 shutdown_connections 2024-09-19T06:11:14.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.900+0000 7f61f29f2640 1 -- 172.21.15.5:0/3877118311 wait complete. 2024-09-19T06:11:14.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.901+0000 7f61f29f2640 1 Processor -- start 2024-09-19T06:11:14.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.901+0000 7f61f29f2640 1 -- start start 2024-09-19T06:11:14.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.901+0000 7f61f29f2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61ec102d30 0x7f61ec1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.901+0000 7f61f29f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61ec1036e0 0x7f61ec1967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61f29f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61ec1082a0 0x7f61ec19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61ec1036e0 0x7f61ec1967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61ec1036e0 0x7f61ec1967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36610/0 (socket says 172.21.15.5:36610) 2024-09-19T06:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61eb7fe640 1 -- 172.21.15.5:0/509679916 learned_addr learned my addr 172.21.15.5:0/509679916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61f29f2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec10fd90 con 0x7f61ec1036e0 2024-09-19T06:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61f29f2640 1 -- 172.21.15.5:0/509679916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec10fc10 con 0x7f61ec102d30 2024-09-19T06:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61ebfff640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61ec102d30 0x7f61ec1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61f29f2640 1 -- 172.21.15.5:0/509679916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec10ff10 con 0x7f61ec1082a0 2024-09-19T06:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61f0f8f640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61ec1082a0 0x7f61ec19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61ebfff640 1 -- 172.21.15.5:0/509679916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61ec1082a0 msgr2=0x7f61ec19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61ebfff640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61ec1082a0 0x7f61ec19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61ebfff640 1 -- 172.21.15.5:0/509679916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61ec1036e0 msgr2=0x7f61ec1967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:14.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61ebfff640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61ec1036e0 0x7f61ec1967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61ebfff640 1 -- 172.21.15.5:0/509679916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec19b250 con 0x7f61ec102d30 2024-09-19T06:11:14.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.902+0000 7f61eb7fe640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61ec1036e0 0x7f61ec1967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:14.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.903+0000 7f61ebfff640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61ec102d30 0x7f61ec1962a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009af0 tx=0x7f61dc0071c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:14.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.903+0000 7f61e97fa640 1 -- 172.21.15.5:0/509679916 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc007e00 con 0x7f61ec102d30 2024-09-19T06:11:14.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.903+0000 7f61e97fa640 1 -- 172.21.15.5:0/509679916 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61dc01e070 con 0x7f61ec102d30 2024-09-19T06:11:14.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.903+0000 7f61f29f2640 1 -- 172.21.15.5:0/509679916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ec19b540 con 0x7f61ec102d30 2024-09-19T06:11:14.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.903+0000 7f61e97fa640 1 -- 172.21.15.5:0/509679916 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc018770 con 0x7f61ec102d30 2024-09-19T06:11:14.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.903+0000 7f61f29f2640 1 -- 172.21.15.5:0/509679916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ec1a2e20 con 0x7f61ec102d30 2024-09-19T06:11:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.905+0000 7f61e97fa640 1 -- 172.21.15.5:0/509679916 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f61dc018910 con 0x7f61ec102d30 2024-09-19T06:11:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f4dcb740 1 Processor -- start 2024-09-19T06:11:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f4dcb740 1 -- start start 2024-09-19T06:11:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f4dcb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 0x55d5f1b56090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f4dcb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b56a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f4dcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b61370 0x55d5f1b63760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f4dcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d5f1abc2f0 con 0x55d5f1b61370 2024-09-19T06:11:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f4dcb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d5f1a63cf0 con 0x55d5f1b57b40 2024-09-19T06:11:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f4dcb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d5f1ab82b0 con 0x55d5f1b565d0 2024-09-19T06:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f0f8f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 0x55d5f1b56090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b61370 0x55d5f1b63760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f0f8f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 0x55d5f1b56090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39090/0 (socket says 172.21.15.5:39090) 2024-09-19T06:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f0f8f640 1 -- 172.21.15.5:0/1059457158 learned_addr learned my addr 172.21.15.5:0/1059457158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61ebfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b56a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61e97fa640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61d40a33b0 0x7f61d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f0f8f640 1 -- 172.21.15.5:0/1059457158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 msgr2=0x55d5f1b56a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f0f8f640 1 --2- 172.21.15.5:0/1059457158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b56a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f0f8f640 1 -- 172.21.15.5:0/1059457158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b61370 msgr2=0x55d5f1b63760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f0f8f640 1 --2- 172.21.15.5:0/1059457158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b61370 0x55d5f1b63760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61f0f8f640 1 -- 172.21.15.5:0/1059457158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f1a483a0 con 0x55d5f1b57b40 2024-09-19T06:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.906+0000 7f61eb7fe640 1 --2- 172.21.15.5:0/1059457158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b61370 0x55d5f1b63760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.907+0000 7f61f0f8f640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61d40a33b0 0x7f61d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.907+0000 7f61e97fa640 1 -- 172.21.15.5:0/509679916 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f61dc0d3c90 con 0x7f61ec102d30 2024-09-19T06:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.907+0000 7f61f0f8f640 1 --2- 172.21.15.5:0/1059457158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 0x55d5f1b56090 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f61e402aed0 tx=0x7f61e40093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a8edafce75c79ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.907+0000 7f61d27fc640 1 -- 172.21.15.5:0/1059457158 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4041040 con 0x55d5f1b57b40 2024-09-19T06:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.907+0000 7f61d27fc640 1 -- 172.21.15.5:0/1059457158 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61e4043070 con 0x55d5f1b57b40 2024-09-19T06:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.907+0000 7f61d27fc640 1 -- 172.21.15.5:0/1059457158 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e40309d0 con 0x55d5f1b57b40 2024-09-19T06:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.907+0000 7f61f0f8f640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61d40a33b0 0x7f61d40a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f61e404c000 tx=0x7f61e404d000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.908+0000 7f61f4dcb740 1 -- 172.21.15.5:0/1059457158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 msgr2=0x55d5f1b56090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.908+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/1059457158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 0x55d5f1b56090 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f61e402aed0 tx=0x7f61e40093e0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.908+0000 7f61f4dcb740 1 -- 172.21.15.5:0/1059457158 shutdown_connections 2024-09-19T06:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.908+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/1059457158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b61370 0x55d5f1b63760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.908+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/1059457158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b56a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.908+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/1059457158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 0x55d5f1b56090 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.908+0000 7f61f4dcb740 1 -- 172.21.15.5:0/1059457158 >> 172.21.15.5:0/1059457158 conn(0x55d5f1b1f310 msgr2=0x55d5f1b52c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.908+0000 7f61f4dcb740 1 -- 172.21.15.5:0/1059457158 shutdown_connections 2024-09-19T06:11:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.908+0000 7f61f4dcb740 1 -- 172.21.15.5:0/1059457158 wait complete. 2024-09-19T06:11:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.908+0000 7f61f4dcb740 1 -- start start 2024-09-19T06:11:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61f4dcb740 1 Processor -- start 2024-09-19T06:11:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61f4dcb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b60d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61f4dcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61f4dcb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b61370 0x55d5f1b5e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61f4dcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d5f1b64070 con 0x55d5f1b57b40 2024-09-19T06:11:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61f4dcb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d5f1a77060 con 0x55d5f1b61370 2024-09-19T06:11:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61f4dcb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d5f1b641f0 con 0x55d5f1b565d0 2024-09-19T06:11:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61f4dcb740 10 client.0 _ll_register_callbacks cb 0x55d5f17ca8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:14.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61f0f8f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b61370 0x55d5f1b5e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61ebfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b60d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36618/0 (socket says 172.21.15.5:36618) 2024-09-19T06:11:14.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61f0f8f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b61370 0x55d5f1b5e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39096/0 (socket says 172.21.15.5:39096) 2024-09-19T06:11:14.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61eb7fe640 1 -- 172.21.15.5:0/4072779412 learned_addr learned my addr 172.21.15.5:0/4072779412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:14.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.909+0000 7f61ebfff640 1 -- 172.21.15.5:0/4072779412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b61370 msgr2=0x55d5f1b5e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:14.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61ebfff640 1 --2- 172.21.15.5:0/4072779412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b61370 0x55d5f1b5e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61ebfff640 1 -- 172.21.15.5:0/4072779412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 msgr2=0x55d5f1b5e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:14.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61ebfff640 1 --2- 172.21.15.5:0/4072779412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61ebfff640 1 -- 172.21.15.5:0/4072779412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5f182d3c0 con 0x55d5f1b565d0 2024-09-19T06:11:14.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61ebfff640 1 --2- 172.21.15.5:0/4072779412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b60d50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0d1360 tx=0x7f61dc09fab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5848c660c28e5c32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:14.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:11:14.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61cb7fe640 1 -- 172.21.15.5:0/4072779412 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0a42a0 con 0x55d5f1b565d0 2024-09-19T06:11:14.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61cb7fe640 1 -- 172.21.15.5:0/4072779412 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f61dc0d15e0 con 0x55d5f1b565d0 2024-09-19T06:11:14.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61f4dcb740 1 -- 172.21.15.5:0/4072779412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e4043070 con 0x55d5f1b565d0 2024-09-19T06:11:14.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61cb7fe640 1 -- 172.21.15.5:0/4072779412 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f61dc0d17e0 con 0x55d5f1b565d0 2024-09-19T06:11:14.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61f4dcb740 10 client.26932 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:14.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61f4dcb740 1 -- 172.21.15.5:0/4072779412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d5f1b6faf0 con 0x55d5f1b565d0 2024-09-19T06:11:14.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.910+0000 7f61cb7fe640 1 -- 172.21.15.5:0/4072779412 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61e4043070 con 0x55d5f1b565d0 2024-09-19T06:11:14.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.911+0000 7f61cb7fe640 1 -- 172.21.15.5:0/4072779412 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f61dc09f340 con 0x55d5f1b565d0 2024-09-19T06:11:14.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.911+0000 7f61f4dcb740 10 client.26932 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.911+0000 7f61f4dcb740 10 client.26932 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:11:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.911+0000 7f61f4dcb740 20 client.26932 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.911+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/4072779412 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d5f1b70f00 0x55d5f1b73360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.911+0000 7f61f4dcb740 4 client.26932 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:11:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.911+0000 7f61f4dcb740 1 -- 172.21.15.5:0/4072779412 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5f1ac9860 con 0x55d5f1b70f00 2024-09-19T06:11:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.911+0000 7f61eb7fe640 1 --2- 172.21.15.5:0/4072779412 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d5f1b70f00 0x55d5f1b73360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.911+0000 7f61cb7fe640 10 client.26932 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:11:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.911+0000 7f61eb7fe640 1 --2- 172.21.15.5:0/4072779412 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d5f1b70f00 0x55d5f1b73360 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=877c30d1b6e3a793 server_cookie=531cac153259d8eb in_seq=0 out_seq=0 2024-09-19T06:11:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.912+0000 7f61cb7fe640 1 -- 172.21.15.5:0/4072779412 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5f1ac9860 con 0x55d5f1b70f00 2024-09-19T06:11:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.912+0000 7f61cb7fe640 10 client.26932 handle_command_reply: tid=0 2024-09-19T06:11:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.912+0000 7f61f4dcb740 1 client.26932 shutdown 2024-09-19T06:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.912+0000 7f61f4dcb740 1 -- 172.21.15.5:0/4072779412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 msgr2=0x55d5f1b60d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.912+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/4072779412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b60d50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0d1360 tx=0x7f61dc09fab0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.912+0000 7f61f4dcb740 1 -- 172.21.15.5:0/4072779412 shutdown_connections 2024-09-19T06:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.912+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/4072779412 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d5f1b70f00 0x55d5f1b73360 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.912+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/4072779412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b61370 0x55d5f1b5e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.912+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/4072779412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.912+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/4072779412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b60d50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.912+0000 7f61f4dcb740 1 -- 172.21.15.5:0/4072779412 >> 172.21.15.5:0/4072779412 conn(0x55d5f1b1f310 msgr2=0x55d5f1b58960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.912+0000 7f61cb7fe640 0 client.26932 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.913+0000 7f61f4dcb740 1 -- 172.21.15.5:0/4072779412 shutdown_connections 2024-09-19T06:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.913+0000 7f61f4dcb740 1 -- 172.21.15.5:0/4072779412 wait complete. 2024-09-19T06:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.913+0000 7f61f4dcb740 20 client.26932 trim_cache size 0 max 16384 2024-09-19T06:11:14.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61f4dcb740 1 Processor -- start 2024-09-19T06:11:14.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61f4dcb740 1 -- start start 2024-09-19T06:11:14.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61f4dcb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b565d0 0x55d5f1b5cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61f4dcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61f4dcb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b61370 0x55d5f1b59120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61f4dcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1ab6c0 con 0x55d5f1b57b40 2024-09-19T06:11:14.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61f4dcb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1ad200 con 0x55d5f1b565d0 2024-09-19T06:11:14.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61f4dcb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1abd90 con 0x55d5f1b61370 2024-09-19T06:11:14.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36620/0 (socket says 172.21.15.5:36620) 2024-09-19T06:11:14.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61ebfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b565d0 0x55d5f1b5cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61eb7fe640 1 -- 172.21.15.5:0/1741902786 learned_addr learned my addr 172.21.15.5:0/1741902786 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:14.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.928+0000 7f61f0f8f640 1 --2- 172.21.15.5:0/1741902786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b61370 0x55d5f1b59120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.929+0000 7f61eb7fe640 1 -- 172.21.15.5:0/1741902786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b61370 msgr2=0x55d5f1b59120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:14.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.929+0000 7f61eb7fe640 1 --2- 172.21.15.5:0/1741902786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b61370 0x55d5f1b59120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.929+0000 7f61eb7fe640 1 -- 172.21.15.5:0/1741902786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b565d0 msgr2=0x55d5f1b5cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:14.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.929+0000 7f61eb7fe640 1 --2- 172.21.15.5:0/1741902786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b565d0 0x55d5f1b5cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.929+0000 7f61eb7fe640 1 -- 172.21.15.5:0/1741902786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f1b610c0 con 0x55d5f1b57b40 2024-09-19T06:11:14.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.929+0000 7f61ebfff640 1 --2- 172.21.15.5:0/1741902786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b565d0 0x55d5f1b5cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:14.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.929+0000 7f61eb7fe640 1 --2- 172.21.15.5:0/1741902786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5ff70 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x55d5f1b5d890 tx=0x7f61e0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2943f47b33c04a2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:14.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.929+0000 7f61d1ffb640 1 -- 172.21.15.5:0/1741902786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0022070 con 0x55d5f1b57b40 2024-09-19T06:11:14.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.929+0000 7f61d1ffb640 1 -- 172.21.15.5:0/1741902786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61e0010910 con 0x55d5f1b57b40 2024-09-19T06:11:14.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.929+0000 7f61d1ffb640 1 -- 172.21.15.5:0/1741902786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e00054e0 con 0x55d5f1b57b40 2024-09-19T06:11:14.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.930+0000 7f61f4dcb740 1 -- 172.21.15.5:0/1741902786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 msgr2=0x55d5f1b5ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:14.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.930+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/1741902786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5ff70 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x55d5f1b5d890 tx=0x7f61e0005f90 comp rx=0 tx=0).stop 2024-09-19T06:11:14.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.930+0000 7f61f4dcb740 1 -- 172.21.15.5:0/1741902786 shutdown_connections 2024-09-19T06:11:14.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.930+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/1741902786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b61370 0x55d5f1b59120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.930+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/1741902786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5ff70 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.930+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/1741902786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b565d0 0x55d5f1b5cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.930+0000 7f61f4dcb740 1 -- 172.21.15.5:0/1741902786 >> 172.21.15.5:0/1741902786 conn(0x55d5f1b1f310 msgr2=0x55d5f1b5f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.930+0000 7f61f4dcb740 1 -- 172.21.15.5:0/1741902786 shutdown_connections 2024-09-19T06:11:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.930+0000 7f61f4dcb740 1 -- 172.21.15.5:0/1741902786 wait complete. 2024-09-19T06:11:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61f4dcb740 1 -- start start 2024-09-19T06:11:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61f4dcb740 1 Processor -- start 2024-09-19T06:11:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61f4dcb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61f4dcb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61f4dcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b61370 0x55d5f1b5b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61f4dcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1acb30 con 0x55d5f1b61370 2024-09-19T06:11:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61f4dcb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1ac460 con 0x55d5f1b57b40 2024-09-19T06:11:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61f4dcb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1a96c0 con 0x55d5f1b565d0 2024-09-19T06:11:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61f4dcb740 10 client.0 _ll_register_callbacks cb 0x7f61dc09fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61eb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61eb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39108/0 (socket says 172.21.15.5:39108) 2024-09-19T06:11:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61eb7fe640 1 -- 172.21.15.5:0/578707131 learned_addr learned my addr 172.21.15.5:0/578707131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.931+0000 7f61f0f8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b61370 0x55d5f1b5b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61ebfff640 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61f0f8f640 1 -- 172.21.15.5:0/578707131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 msgr2=0x55d5f1b699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61f0f8f640 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61f0f8f640 1 -- 172.21.15.5:0/578707131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 msgr2=0x55d5f1b5abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61f0f8f640 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61f0f8f640 1 -- 172.21.15.5:0/578707131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5f1b5b760 con 0x55d5f1b61370 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61eb7fe640 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61ebfff640 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61f0f8f640 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b61370 0x55d5f1b5b120 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009ca0 tx=0x7f61e40315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02b1884fa04edef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61cb7fe640 1 -- 172.21.15.5:0/578707131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4041070 con 0x55d5f1b61370 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61cb7fe640 1 -- 172.21.15.5:0/578707131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f61e4030920 con 0x55d5f1b61370 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.932+0000 7f61f4dcb740 1 -- 172.21.15.5:0/578707131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e0010910 con 0x55d5f1b61370 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.933+0000 7f61cb7fe640 1 -- 172.21.15.5:0/578707131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f61e4069050 con 0x55d5f1b61370 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.933+0000 7f61f4dcb740 10 client.17448 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.933+0000 7f61f4dcb740 1 -- 172.21.15.5:0/578707131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d5f1bdba50 con 0x55d5f1b61370 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.933+0000 7f61cb7fe640 1 -- 172.21.15.5:0/578707131 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61e4060070 con 0x55d5f1b61370 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.933+0000 7f61cb7fe640 1 -- 172.21.15.5:0/578707131 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f61e4030070 con 0x55d5f1b61370 2024-09-19T06:11:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.933+0000 7f61f4dcb740 10 client.17448 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.933+0000 7f61f4dcb740 10 client.17448 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:11:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.933+0000 7f61f4dcb740 20 client.17448 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.933+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d5f1bdbda0 0x55d5f1bde1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.933+0000 7f61f4dcb740 4 client.17448 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:11:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.933+0000 7f61f4dcb740 1 -- 172.21.15.5:0/578707131 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f61e0010910 con 0x55d5f1bdbda0 2024-09-19T06:11:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.934+0000 7f61ebfff640 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d5f1bdbda0 0x55d5f1bde1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.934+0000 7f61ebfff640 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d5f1bdbda0 0x55d5f1bde1c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7f71f10925495438 server_cookie=5a2b17c71da15942 in_seq=0 out_seq=0 2024-09-19T06:11:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.934+0000 7f61cb7fe640 10 client.17448 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:11:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.934+0000 7f61cb7fe640 1 -- 172.21.15.5:0/578707131 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f61e0010910 con 0x55d5f1bdbda0 2024-09-19T06:11:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.934+0000 7f61cb7fe640 10 client.17448 handle_command_reply: tid=0 2024-09-19T06:11:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.934+0000 7f61f4dcb740 1 client.17448 shutdown 2024-09-19T06:11:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.935+0000 7f61f4dcb740 1 -- 172.21.15.5:0/578707131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b61370 msgr2=0x55d5f1b5b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.935+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b61370 0x55d5f1b5b120 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009ca0 tx=0x7f61e40315b0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.935+0000 7f61f4dcb740 1 -- 172.21.15.5:0/578707131 shutdown_connections 2024-09-19T06:11:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.935+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d5f1bdbda0 0x55d5f1bde1c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.935+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5f1b61370 0x55d5f1b5b120 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.935+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5f1b57b40 0x55d5f1b5abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.935+0000 7f61f4dcb740 1 --2- 172.21.15.5:0/578707131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5f1b565d0 0x55d5f1b699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.935+0000 7f61f4dcb740 1 -- 172.21.15.5:0/578707131 >> 172.21.15.5:0/578707131 conn(0x55d5f1b1f310 msgr2=0x55d5f1b60080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.935+0000 7f61cb7fe640 0 client.17448 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:11:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.935+0000 7f61f4dcb740 1 -- 172.21.15.5:0/578707131 shutdown_connections 2024-09-19T06:11:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.935+0000 7f61f4dcb740 1 -- 172.21.15.5:0/578707131 wait complete. 2024-09-19T06:11:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.935+0000 7f61f4dcb740 20 client.17448 trim_cache size 0 max 16384 2024-09-19T06:11:14.943 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:11:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.936+0000 7f61f29f2640 1 -- 172.21.15.5:0/509679916 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61d40a33b0 msgr2=0x7f61d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.936+0000 7f61f29f2640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61d40a33b0 0x7f61d40a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f61e404c000 tx=0x7f61e404d000 comp rx=0 tx=0).stop 2024-09-19T06:11:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.936+0000 7f61f29f2640 1 -- 172.21.15.5:0/509679916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61ec102d30 msgr2=0x7f61ec1962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.936+0000 7f61f29f2640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61ec102d30 0x7f61ec1962a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009af0 tx=0x7f61dc0071c0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.936+0000 7f61f29f2640 1 -- 172.21.15.5:0/509679916 shutdown_connections 2024-09-19T06:11:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.936+0000 7f61f29f2640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61d40a33b0 0x7f61d40a5870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.936+0000 7f61f29f2640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61ec1082a0 0x7f61ec19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.936+0000 7f61f29f2640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61ec1036e0 0x7f61ec1967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.936+0000 7f61f29f2640 1 --2- 172.21.15.5:0/509679916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61ec102d30 0x7f61ec1962a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.936+0000 7f61f29f2640 1 -- 172.21.15.5:0/509679916 >> 172.21.15.5:0/509679916 conn(0x7f61ec0fc7d0 msgr2=0x7f61ec105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.937+0000 7f61f29f2640 1 -- 172.21.15.5:0/509679916 shutdown_connections 2024-09-19T06:11:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:14.937+0000 7f61f29f2640 1 -- 172.21.15.5:0/509679916 wait complete. 2024-09-19T06:11:15.612 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:11:15.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 203592 2.71 MB/sec execute 281 sec latency 19.465 ms 2024-09-19T06:11:15.889 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:11:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.387+0000 7f526c707640 1 -- 172.21.15.5:0/2184854452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52641082a0 msgr2=0x7f526410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.387+0000 7f526c707640 1 --2- 172.21.15.5:0/2184854452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52641082a0 0x7f526410ec40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f525c009c10 tx=0x7f525c02a6e0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.388+0000 7f526c707640 1 -- 172.21.15.5:0/2184854452 shutdown_connections 2024-09-19T06:11:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.388+0000 7f526c707640 1 --2- 172.21.15.5:0/2184854452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52641082a0 0x7f526410ec40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.388+0000 7f526c707640 1 --2- 172.21.15.5:0/2184854452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52641036e0 0x7f5264107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.388+0000 7f526c707640 1 --2- 172.21.15.5:0/2184854452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5264102d30 0x7f5264103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.388+0000 7f526c707640 1 -- 172.21.15.5:0/2184854452 >> 172.21.15.5:0/2184854452 conn(0x7f52640fc7d0 msgr2=0x7f52640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.388+0000 7f526c707640 1 -- 172.21.15.5:0/2184854452 shutdown_connections 2024-09-19T06:11:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.388+0000 7f526c707640 1 -- 172.21.15.5:0/2184854452 wait complete. 2024-09-19T06:11:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.388+0000 7f526c707640 1 Processor -- start 2024-09-19T06:11:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.388+0000 7f526c707640 1 -- start start 2024-09-19T06:11:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526c707640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5264102d30 0x7f52641984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526c707640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52641036e0 0x7f52641989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526c707640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52641082a0 0x7f526419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526c707640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5264111f50 con 0x7f5264102d30 2024-09-19T06:11:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526c707640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5264111dd0 con 0x7f52641036e0 2024-09-19T06:11:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526c707640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52641120d0 con 0x7f52641082a0 2024-09-19T06:11:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526a4a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5264102d30 0x7f52641984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526a4a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5264102d30 0x7f52641984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36648/0 (socket says 172.21.15.5:36648) 2024-09-19T06:11:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526a4a3640 1 -- 172.21.15.5:0/2321381641 learned_addr learned my addr 172.21.15.5:0/2321381641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f5269ca2640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52641036e0 0x7f52641989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526aca4640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52641082a0 0x7f526419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526a4a3640 1 -- 172.21.15.5:0/2321381641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52641082a0 msgr2=0x7f526419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526a4a3640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52641082a0 0x7f526419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526a4a3640 1 -- 172.21.15.5:0/2321381641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52641036e0 msgr2=0x7f52641989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526a4a3640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52641036e0 0x7f52641989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526a4a3640 1 -- 172.21.15.5:0/2321381641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526419d460 con 0x7f5264102d30 2024-09-19T06:11:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f5269ca2640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52641036e0 0x7f52641989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.389+0000 7f526aca4640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52641082a0 0x7f526419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.390+0000 7f526a4a3640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5264102d30 0x7f52641984b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f526000e9f0 tx=0x7f526000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.390+0000 7f525b7fe640 1 -- 172.21.15.5:0/2321381641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526000cd80 con 0x7f5264102d30 2024-09-19T06:11:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.390+0000 7f525b7fe640 1 -- 172.21.15.5:0/2321381641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52600108e0 con 0x7f5264102d30 2024-09-19T06:11:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.390+0000 7f526c707640 1 -- 172.21.15.5:0/2321381641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526419d750 con 0x7f5264102d30 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.390+0000 7f525b7fe640 1 -- 172.21.15.5:0/2321381641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260018920 con 0x7f5264102d30 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.390+0000 7f526c707640 1 -- 172.21.15.5:0/2321381641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52641a5030 con 0x7f5264102d30 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.393+0000 7f525b7fe640 1 -- 172.21.15.5:0/2321381641 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5260010a90 con 0x7f5264102d30 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.393+0000 7f526eae0740 1 Processor -- start 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.393+0000 7f526eae0740 1 -- start start 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.393+0000 7f526eae0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c919b40 0x56432c918090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.393+0000 7f526eae0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c9185d0 0x56432c918a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.393+0000 7f526eae0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 0x56432c925760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.393+0000 7f526eae0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56432c87e2f0 con 0x56432c923370 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.393+0000 7f526eae0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56432c825cf0 con 0x56432c9185d0 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.393+0000 7f526eae0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56432c87a2b0 con 0x56432c919b40 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f526a4a3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c9185d0 0x56432c918a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f5269ca2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 0x56432c925760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f525b7fe640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52500a33b0 0x7f52500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f526a4a3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c9185d0 0x56432c918a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39138/0 (socket says 172.21.15.5:39138) 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f526a4a3640 1 -- 172.21.15.5:0/2932486971 learned_addr learned my addr 172.21.15.5:0/2932486971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f5269ca2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 0x56432c925760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36664/0 (socket says 172.21.15.5:36664) 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f526aca4640 1 --2- 172.21.15.5:0/2932486971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c919b40 0x56432c918090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f526aca4640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52500a33b0 0x7f52500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f525b7fe640 1 -- 172.21.15.5:0/2321381641 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f52600d0010 con 0x7f5264102d30 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f526a4a3640 1 -- 172.21.15.5:0/2932486971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c919b40 msgr2=0x56432c918090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f526a4a3640 1 --2- 172.21.15.5:0/2932486971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c919b40 0x56432c918090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f526a4a3640 1 -- 172.21.15.5:0/2932486971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 msgr2=0x56432c925760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f526a4a3640 1 --2- 172.21.15.5:0/2932486971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 0x56432c925760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f526a4a3640 1 -- 172.21.15.5:0/2932486971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56432c80a3a0 con 0x56432c9185d0 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f526a4a3640 1 --2- 172.21.15.5:0/2932486971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c9185d0 0x56432c918a10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52600baf10 tx=0x7f52600baf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8be71d84b4ebb0d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:16.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.394+0000 7f526aca4640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52500a33b0 0x7f52500a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f525c004af0 tx=0x7f525c005e70 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.395+0000 7f5258ff9640 1 -- 172.21.15.5:0/2932486971 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52600a0680 con 0x56432c9185d0 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.395+0000 7f5258ff9640 1 -- 172.21.15.5:0/2932486971 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52600a4f40 con 0x56432c9185d0 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.395+0000 7f5258ff9640 1 -- 172.21.15.5:0/2932486971 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52600a4ab0 con 0x56432c9185d0 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.395+0000 7f526eae0740 1 -- 172.21.15.5:0/2932486971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c9185d0 msgr2=0x56432c918a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.395+0000 7f526eae0740 1 --2- 172.21.15.5:0/2932486971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c9185d0 0x56432c918a10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52600baf10 tx=0x7f52600baf40 comp rx=0 tx=0).stop 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.395+0000 7f526eae0740 1 -- 172.21.15.5:0/2932486971 shutdown_connections 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.395+0000 7f526eae0740 1 --2- 172.21.15.5:0/2932486971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 0x56432c925760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.395+0000 7f526eae0740 1 --2- 172.21.15.5:0/2932486971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c9185d0 0x56432c918a10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.395+0000 7f526eae0740 1 --2- 172.21.15.5:0/2932486971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c919b40 0x56432c918090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.395+0000 7f526eae0740 1 -- 172.21.15.5:0/2932486971 >> 172.21.15.5:0/2932486971 conn(0x56432c8e1310 msgr2=0x56432c914c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.396+0000 7f526eae0740 1 -- 172.21.15.5:0/2932486971 shutdown_connections 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.396+0000 7f526eae0740 1 -- 172.21.15.5:0/2932486971 wait complete. 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.396+0000 7f526eae0740 1 -- start start 2024-09-19T06:11:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.396+0000 7f526eae0740 1 Processor -- start 2024-09-19T06:11:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.396+0000 7f526eae0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c9185d0 0x56432c922d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.396+0000 7f526eae0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 0x56432c920030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.396+0000 7f526eae0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c923370 0x56432c920640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.396+0000 7f526eae0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56432c92a420 con 0x56432c9185d0 2024-09-19T06:11:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.396+0000 7f526eae0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56432c839060 con 0x56432c919b40 2024-09-19T06:11:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f526eae0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56432c92a5a0 con 0x56432c923370 2024-09-19T06:11:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f526a4a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c9185d0 0x56432c922d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f526eae0740 10 client.0 _ll_register_callbacks cb 0x56432c58c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f526a4a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c9185d0 0x56432c922d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36674/0 (socket says 172.21.15.5:36674) 2024-09-19T06:11:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f5269ca2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 0x56432c920030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f526a4a3640 1 -- 172.21.15.5:0/3195301469 learned_addr learned my addr 172.21.15.5:0/3195301469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f526aca4640 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c923370 0x56432c920640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f5269ca2640 1 -- 172.21.15.5:0/3195301469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c923370 msgr2=0x56432c920640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f5269ca2640 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c923370 0x56432c920640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f5269ca2640 1 -- 172.21.15.5:0/3195301469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c9185d0 msgr2=0x56432c922d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f5269ca2640 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c9185d0 0x56432c922d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f5269ca2640 1 -- 172.21.15.5:0/3195301469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56432c5ef3c0 con 0x56432c919b40 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f526a4a3640 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c9185d0 0x56432c922d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f526aca4640 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c923370 0x56432c920640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f5269ca2640 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 0x56432c920030 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f52640feea0 tx=0x7f525400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b7f855c941d3cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f524dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.397+0000 7f524dffb640 1 -- 172.21.15.5:0/3195301469 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525401f060 con 0x56432c919b40 2024-09-19T06:11:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.398+0000 7f526eae0740 1 -- 172.21.15.5:0/3195301469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52600a4f40 con 0x56432c919b40 2024-09-19T06:11:16.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.398+0000 7f524dffb640 1 -- 172.21.15.5:0/3195301469 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f5254010430 con 0x56432c919b40 2024-09-19T06:11:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.401+0000 7f524dffb640 1 -- 172.21.15.5:0/3195301469 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5254010630 con 0x56432c919b40 2024-09-19T06:11:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.401+0000 7f526eae0740 10 client.27029 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.401+0000 7f526eae0740 1 -- 172.21.15.5:0/3195301469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56432c935f50 con 0x56432c919b40 2024-09-19T06:11:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.401+0000 7f524dffb640 1 -- 172.21.15.5:0/3195301469 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52600a4f40 con 0x56432c919b40 2024-09-19T06:11:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.402+0000 7f524dffb640 1 -- 172.21.15.5:0/3195301469 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5254022830 con 0x56432c919b40 2024-09-19T06:11:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.402+0000 7f526eae0740 10 client.27029 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.402+0000 7f526eae0740 10 client.27029 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:11:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.402+0000 7f526eae0740 20 client.27029 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.402+0000 7f526eae0740 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56432c937410 0x56432c939800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.402+0000 7f526eae0740 4 client.27029 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:11:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.402+0000 7f526eae0740 1 -- 172.21.15.5:0/3195301469 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56432c88b860 con 0x56432c937410 2024-09-19T06:11:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.402+0000 7f526a4a3640 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56432c937410 0x56432c939800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.402+0000 7f526a4a3640 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56432c937410 0x56432c939800 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b86eea8e122cbec server_cookie=c17ad0eb466bfbe1 in_seq=0 out_seq=0 2024-09-19T06:11:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.402+0000 7f524dffb640 10 client.27029 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:11:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f524dffb640 1 -- 172.21.15.5:0/3195301469 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56432c88b860 con 0x56432c937410 2024-09-19T06:11:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f524dffb640 10 client.27029 handle_command_reply: tid=0 2024-09-19T06:11:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f526eae0740 1 client.27029 shutdown 2024-09-19T06:11:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f526eae0740 1 -- 172.21.15.5:0/3195301469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 msgr2=0x56432c920030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f526eae0740 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 0x56432c920030 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f52640feea0 tx=0x7f525400f790 comp rx=0 tx=0).stop 2024-09-19T06:11:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f526eae0740 1 -- 172.21.15.5:0/3195301469 shutdown_connections 2024-09-19T06:11:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f526eae0740 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56432c937410 0x56432c939800 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f524dffb640 0 client.27029 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:11:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f526eae0740 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c923370 0x56432c920640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f526eae0740 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 0x56432c920030 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f526eae0740 1 --2- 172.21.15.5:0/3195301469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c9185d0 0x56432c922d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f526eae0740 1 -- 172.21.15.5:0/3195301469 >> 172.21.15.5:0/3195301469 conn(0x56432c8e1310 msgr2=0x56432c91a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f526eae0740 1 -- 172.21.15.5:0/3195301469 shutdown_connections 2024-09-19T06:11:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.403+0000 7f526eae0740 1 -- 172.21.15.5:0/3195301469 wait complete. 2024-09-19T06:11:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.404+0000 7f526eae0740 20 client.27029 trim_cache size 0 max 16384 2024-09-19T06:11:16.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.418+0000 7f526eae0740 1 Processor -- start 2024-09-19T06:11:16.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526eae0740 1 -- start start 2024-09-19T06:11:16.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526eae0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c9185d0 0x56432c914e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526eae0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 0x56432c9336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526eae0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 0x56432c93f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526eae0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52641ab7d0 con 0x56432c923370 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526eae0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f52641ad310 con 0x56432c919b40 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526eae0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52641abea0 con 0x56432c9185d0 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f5269ca2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 0x56432c9336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526aca4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 0x56432c93f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f5269ca2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 0x56432c9336c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39144/0 (socket says 172.21.15.5:39144) 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526aca4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 0x56432c93f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36682/0 (socket says 172.21.15.5:36682) 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f5269ca2640 1 -- 172.21.15.5:0/35263917 learned_addr learned my addr 172.21.15.5:0/35263917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526a4a3640 1 --2- 172.21.15.5:0/35263917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c9185d0 0x56432c914e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526aca4640 1 -- 172.21.15.5:0/35263917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c9185d0 msgr2=0x56432c914e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526aca4640 1 --2- 172.21.15.5:0/35263917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c9185d0 0x56432c914e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526aca4640 1 -- 172.21.15.5:0/35263917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 msgr2=0x56432c9336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526aca4640 1 --2- 172.21.15.5:0/35263917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 0x56432c9336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f526aca4640 1 -- 172.21.15.5:0/35263917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56432c915960 con 0x56432c923370 2024-09-19T06:11:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.419+0000 7f5269ca2640 1 --2- 172.21.15.5:0/35263917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 0x56432c9336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.420+0000 7f526aca4640 1 --2- 172.21.15.5:0/35263917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 0x56432c93f0b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f525c034040 tx=0x7f525c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ecdf4e0cbed548 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:16.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.420+0000 7f524ffff640 1 -- 172.21.15.5:0/35263917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c051070 con 0x56432c923370 2024-09-19T06:11:16.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.420+0000 7f524ffff640 1 -- 172.21.15.5:0/35263917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f525c037040 con 0x56432c923370 2024-09-19T06:11:16.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.420+0000 7f524ffff640 1 -- 172.21.15.5:0/35263917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c036040 con 0x56432c923370 2024-09-19T06:11:16.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 -- 172.21.15.5:0/35263917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 msgr2=0x56432c93f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:16.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 --2- 172.21.15.5:0/35263917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 0x56432c93f0b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f525c034040 tx=0x7f525c03c000 comp rx=0 tx=0).stop 2024-09-19T06:11:16.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 -- 172.21.15.5:0/35263917 shutdown_connections 2024-09-19T06:11:16.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 --2- 172.21.15.5:0/35263917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c923370 0x56432c93f0b0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 --2- 172.21.15.5:0/35263917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c919b40 0x56432c9336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 --2- 172.21.15.5:0/35263917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c9185d0 0x56432c914e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 -- 172.21.15.5:0/35263917 >> 172.21.15.5:0/35263917 conn(0x56432c8e1310 msgr2=0x56432c914700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:16.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 -- 172.21.15.5:0/35263917 shutdown_connections 2024-09-19T06:11:16.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 -- 172.21.15.5:0/35263917 wait complete. 2024-09-19T06:11:16.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 -- start start 2024-09-19T06:11:16.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 Processor -- start 2024-09-19T06:11:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c9185d0 0x56432c92f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c919b40 0x56432c92fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c923370 0x56432c92c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52641acc40 con 0x56432c919b40 2024-09-19T06:11:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f52641ac570 con 0x56432c923370 2024-09-19T06:11:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.421+0000 7f526eae0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52641a97d0 con 0x56432c9185d0 2024-09-19T06:11:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f526eae0740 10 client.0 _ll_register_callbacks cb 0x7f5254009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f5269ca2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c919b40 0x56432c92fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f5269ca2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c919b40 0x56432c92fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36692/0 (socket says 172.21.15.5:36692) 2024-09-19T06:11:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f5269ca2640 1 -- 172.21.15.5:0/3984512599 learned_addr learned my addr 172.21.15.5:0/3984512599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f526aca4640 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c923370 0x56432c92c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f526a4a3640 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c9185d0 0x56432c92f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f5269ca2640 1 -- 172.21.15.5:0/3984512599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c9185d0 msgr2=0x56432c92f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f5269ca2640 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c9185d0 0x56432c92f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f5269ca2640 1 -- 172.21.15.5:0/3984512599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c923370 msgr2=0x56432c92c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f5269ca2640 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c923370 0x56432c92c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f5269ca2640 1 -- 172.21.15.5:0/3984512599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56432c91dff0 con 0x56432c919b40 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f526aca4640 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c923370 0x56432c92c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f5269ca2640 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c919b40 0x56432c92fd70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5254000c00 tx=0x7f5254022600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bf3d52fbaae294 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f524dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f524dffb640 1 -- 172.21.15.5:0/3984512599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254022e90 con 0x56432c919b40 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f524dffb640 1 -- 172.21.15.5:0/3984512599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f5254010430 con 0x56432c919b40 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.422+0000 7f526eae0740 1 -- 172.21.15.5:0/3984512599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f525c037040 con 0x56432c919b40 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.423+0000 7f524dffb640 1 -- 172.21.15.5:0/3984512599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5254022a00 con 0x56432c919b40 2024-09-19T06:11:16.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.423+0000 7f526eae0740 10 client.17466 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.423+0000 7f526eae0740 1 -- 172.21.15.5:0/3984512599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56432c9a2600 con 0x56432c919b40 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.423+0000 7f524dffb640 1 -- 172.21.15.5:0/3984512599 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f525c037040 con 0x56432c919b40 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.423+0000 7f524dffb640 1 -- 172.21.15.5:0/3984512599 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5254005040 con 0x56432c919b40 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.423+0000 7f526eae0740 10 client.17466 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.423+0000 7f526eae0740 10 client.17466 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.423+0000 7f526eae0740 20 client.17466 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.423+0000 7f526eae0740 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56432c9a2e00 0x56432c9a5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.423+0000 7f526eae0740 4 client.17466 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.423+0000 7f526eae0740 1 -- 172.21.15.5:0/3984512599 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56432c885d70 con 0x56432c9a2e00 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.423+0000 7f526a4a3640 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56432c9a2e00 0x56432c9a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f524dffb640 10 client.17466 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526a4a3640 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56432c9a2e00 0x56432c9a5220 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7bc1022509e554e1 server_cookie=f0be0f3ed8c16e4a in_seq=0 out_seq=0 2024-09-19T06:11:16.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f524dffb640 1 -- 172.21.15.5:0/3984512599 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56432c885d70 con 0x56432c9a2e00 2024-09-19T06:11:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f524dffb640 10 client.17466 handle_command_reply: tid=0 2024-09-19T06:11:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526eae0740 1 client.17466 shutdown 2024-09-19T06:11:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526eae0740 1 -- 172.21.15.5:0/3984512599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c919b40 msgr2=0x56432c92fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526eae0740 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c919b40 0x56432c92fd70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5254000c00 tx=0x7f5254022600 comp rx=0 tx=0).stop 2024-09-19T06:11:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526eae0740 1 -- 172.21.15.5:0/3984512599 shutdown_connections 2024-09-19T06:11:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526eae0740 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56432c9a2e00 0x56432c9a5220 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526eae0740 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56432c923370 0x56432c92c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526eae0740 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56432c919b40 0x56432c92fd70 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526eae0740 1 --2- 172.21.15.5:0/3984512599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56432c9185d0 0x56432c92f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f524dffb640 0 client.17466 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:11:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526eae0740 1 -- 172.21.15.5:0/3984512599 >> 172.21.15.5:0/3984512599 conn(0x56432c8e1310 msgr2=0x56432c921890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526eae0740 1 -- 172.21.15.5:0/3984512599 shutdown_connections 2024-09-19T06:11:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526eae0740 1 -- 172.21.15.5:0/3984512599 wait complete. 2024-09-19T06:11:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.424+0000 7f526eae0740 20 client.17466 trim_cache size 0 max 16384 2024-09-19T06:11:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:11:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.425+0000 7f526c707640 1 -- 172.21.15.5:0/2321381641 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52500a33b0 msgr2=0x7f52500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.425+0000 7f526c707640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52500a33b0 0x7f52500a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f525c004af0 tx=0x7f525c005e70 comp rx=0 tx=0).stop 2024-09-19T06:11:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.425+0000 7f526c707640 1 -- 172.21.15.5:0/2321381641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5264102d30 msgr2=0x7f52641984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.425+0000 7f526c707640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5264102d30 0x7f52641984b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f526000e9f0 tx=0x7f526000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.425+0000 7f526c707640 1 -- 172.21.15.5:0/2321381641 shutdown_connections 2024-09-19T06:11:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.425+0000 7f526c707640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52500a33b0 0x7f52500a5870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.425+0000 7f526c707640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52641082a0 0x7f526419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.425+0000 7f526c707640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52641036e0 0x7f52641989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.425+0000 7f526c707640 1 --2- 172.21.15.5:0/2321381641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5264102d30 0x7f52641984b0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.425+0000 7f526c707640 1 -- 172.21.15.5:0/2321381641 >> 172.21.15.5:0/2321381641 conn(0x7f52640fc7d0 msgr2=0x7f5264105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:16.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.425+0000 7f526c707640 1 -- 172.21.15.5:0/2321381641 shutdown_connections 2024-09-19T06:11:16.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:16.425+0000 7f526c707640 1 -- 172.21.15.5:0/2321381641 wait complete. 2024-09-19T06:11:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:16 smithi005 ceph-mon[30955]: pgmap v245: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-09-19T06:11:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:16 smithi157 ceph-mon[35346]: pgmap v245: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-09-19T06:11:16.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204061 2.71 MB/sec execute 282 sec latency 69.000 ms 2024-09-19T06:11:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:16 smithi049 ceph-mon[34640]: pgmap v245: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-09-19T06:11:17.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:11:17.395 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:11:17.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204625 2.70 MB/sec execute 283 sec latency 22.608 ms 2024-09-19T06:11:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.016+0000 7f1003fcb640 1 -- 172.21.15.5:0/617316228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc106a80 msgr2=0x7f0ffc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.016+0000 7f1003fcb640 1 --2- 172.21.15.5:0/617316228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc106a80 0x7f0ffc106e60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009930 tx=0x7f0ff802a420 comp rx=0 tx=0).stop 2024-09-19T06:11:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.016+0000 7f1003fcb640 1 -- 172.21.15.5:0/617316228 shutdown_connections 2024-09-19T06:11:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.016+0000 7f1003fcb640 1 --2- 172.21.15.5:0/617316228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ffc101420 0x7f0ffc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.016+0000 7f1003fcb640 1 --2- 172.21.15.5:0/617316228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc100a80 0x7f0ffc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.016+0000 7f1003fcb640 1 --2- 172.21.15.5:0/617316228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc106a80 0x7f0ffc106e60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.016+0000 7f1003fcb640 1 -- 172.21.15.5:0/617316228 >> 172.21.15.5:0/617316228 conn(0x7f0ffc0fc7a0 msgr2=0x7f0ffc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.016+0000 7f1003fcb640 1 -- 172.21.15.5:0/617316228 shutdown_connections 2024-09-19T06:11:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.016+0000 7f1003fcb640 1 -- 172.21.15.5:0/617316228 wait complete. 2024-09-19T06:11:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.017+0000 7f1003fcb640 1 Processor -- start 2024-09-19T06:11:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.017+0000 7f1003fcb640 1 -- start start 2024-09-19T06:11:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.017+0000 7f1003fcb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc100a80 0x7f0ffc1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.017+0000 7f1003fcb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc101420 0x7f0ffc199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.017+0000 7f1003fcb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ffc106a80 0x7f0ffc1a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.017+0000 7f1003fcb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc110cf0 con 0x7f0ffc100a80 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.017+0000 7f1003fcb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc110b70 con 0x7f0ffc106a80 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.017+0000 7f1003fcb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc110e70 con 0x7f0ffc101420 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.017+0000 7f1001d67640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc100a80 0x7f0ffc1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.017+0000 7f1001d67640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc100a80 0x7f0ffc1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36720/0 (socket says 172.21.15.5:36720) 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.017+0000 7f1001d67640 1 -- 172.21.15.5:0/1762076187 learned_addr learned my addr 172.21.15.5:0/1762076187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f1001566640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc101420 0x7f0ffc199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f1002568640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ffc106a80 0x7f0ffc1a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f1001d67640 1 -- 172.21.15.5:0/1762076187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc101420 msgr2=0x7f0ffc199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f1001d67640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc101420 0x7f0ffc199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f1001d67640 1 -- 172.21.15.5:0/1762076187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ffc106a80 msgr2=0x7f0ffc1a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f1001d67640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ffc106a80 0x7f0ffc1a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f1001d67640 1 -- 172.21.15.5:0/1762076187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc1a47a0 con 0x7f0ffc100a80 2024-09-19T06:11:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f1001d67640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc100a80 0x7f0ffc1991e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8005940 tx=0x7f0ff80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f0ff2ffd640 1 -- 172.21.15.5:0/1762076187 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff803f030 con 0x7f0ffc100a80 2024-09-19T06:11:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f0ff2ffd640 1 -- 172.21.15.5:0/1762076187 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ff80318a0 con 0x7f0ffc100a80 2024-09-19T06:11:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f1003fcb640 1 -- 172.21.15.5:0/1762076187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc1a4970 con 0x7f0ffc100a80 2024-09-19T06:11:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f0ff2ffd640 1 -- 172.21.15.5:0/1762076187 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8035070 con 0x7f0ffc100a80 2024-09-19T06:11:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.018+0000 7f1003fcb640 1 -- 172.21.15.5:0/1762076187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1a4e20 con 0x7f0ffc100a80 2024-09-19T06:11:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.022+0000 7f0ff2ffd640 1 -- 172.21.15.5:0/1762076187 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0ff8042030 con 0x7f0ffc100a80 2024-09-19T06:11:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.022+0000 7f10063a4740 1 Processor -- start 2024-09-19T06:11:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.022+0000 7f10063a4740 1 -- start start 2024-09-19T06:11:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.022+0000 7f10063a4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbfe2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.022+0000 7f10063a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbfe2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.022+0000 7f10063a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becbfef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.022+0000 7f10063a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55becbf482f0 con 0x55becbfed370 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.022+0000 7f10063a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55becbeefcf0 con 0x55becbfe25d0 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.022+0000 7f10063a4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55becbf442b0 con 0x55becbfe3b40 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.022+0000 7f1001d67640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbfe2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.022+0000 7f1001566640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becbfef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.022+0000 7f1002568640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbfe2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f1001d67640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbfe2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39174/0 (socket says 172.21.15.5:39174) 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f1002568640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbfe2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50002/0 (socket says 172.21.15.5:50002) 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f1001d67640 1 -- 172.21.15.5:0/3682392445 learned_addr learned my addr 172.21.15.5:0/3682392445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f0ff2ffd640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0fe80a33b0 0x7f0fe80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f1001d67640 1 -- 172.21.15.5:0/3682392445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 msgr2=0x55becbfe2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f1001d67640 1 --2- 172.21.15.5:0/3682392445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbfe2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f1001d67640 1 -- 172.21.15.5:0/3682392445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 msgr2=0x55becbfef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f1001d67640 1 --2- 172.21.15.5:0/3682392445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becbfef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f1001d67640 1 -- 172.21.15.5:0/3682392445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55becbed43a0 con 0x55becbfe25d0 2024-09-19T06:11:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f1002568640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0fe80a33b0 0x7f0fe80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f0ff2ffd640 1 -- 172.21.15.5:0/1762076187 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f0ff80f0430 con 0x7f0ffc100a80 2024-09-19T06:11:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f1001d67640 1 --2- 172.21.15.5:0/3682392445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbfe2a10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80edad0 tx=0x7f0ff80e74c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5103b84e529ae7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f0fe7fff640 1 -- 172.21.15.5:0/3682392445 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80bc2e0 con 0x55becbfe25d0 2024-09-19T06:11:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.023+0000 7f0fe7fff640 1 -- 172.21.15.5:0/3682392445 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ff8003e20 con 0x55becbfe25d0 2024-09-19T06:11:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.024+0000 7f0fe7fff640 1 -- 172.21.15.5:0/3682392445 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80bc100 con 0x55becbfe25d0 2024-09-19T06:11:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.024+0000 7f1002568640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0fe80a33b0 0x7f0fe80a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4005c40 tx=0x7f0ff4004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.024+0000 7f10063a4740 1 -- 172.21.15.5:0/3682392445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 msgr2=0x55becbfe2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.024+0000 7f10063a4740 1 --2- 172.21.15.5:0/3682392445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbfe2a10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80edad0 tx=0x7f0ff80e74c0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.024+0000 7f10063a4740 1 -- 172.21.15.5:0/3682392445 shutdown_connections 2024-09-19T06:11:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.024+0000 7f10063a4740 1 --2- 172.21.15.5:0/3682392445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becbfef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.024+0000 7f10063a4740 1 --2- 172.21.15.5:0/3682392445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbfe2a10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.024+0000 7f10063a4740 1 --2- 172.21.15.5:0/3682392445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbfe2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.024+0000 7f10063a4740 1 -- 172.21.15.5:0/3682392445 >> 172.21.15.5:0/3682392445 conn(0x55becbfab310 msgr2=0x55becbfdec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.025+0000 7f10063a4740 1 -- 172.21.15.5:0/3682392445 shutdown_connections 2024-09-19T06:11:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.025+0000 7f10063a4740 1 -- 172.21.15.5:0/3682392445 wait complete. 2024-09-19T06:11:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.025+0000 7f10063a4740 1 -- start start 2024-09-19T06:11:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.025+0000 7f10063a4740 1 Processor -- start 2024-09-19T06:11:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.025+0000 7f10063a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbff1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.025+0000 7f10063a4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbff1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.025+0000 7f10063a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becbff5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.025+0000 7f10063a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55becbfefe30 con 0x55becbfed370 2024-09-19T06:11:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.025+0000 7f10063a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55becbf03060 con 0x55becbfe25d0 2024-09-19T06:11:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.025+0000 7f10063a4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55becbfeffb0 con 0x55becbfe3b40 2024-09-19T06:11:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f10063a4740 10 client.0 _ll_register_callbacks cb 0x55becbc568c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f1001d67640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbff1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f1001d67640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbff1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39180/0 (socket says 172.21.15.5:39180) 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f1001d67640 1 -- 172.21.15.5:0/3930667933 learned_addr learned my addr 172.21.15.5:0/3930667933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f1001566640 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbff1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f1002568640 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becbff5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f1001d67640 1 -- 172.21.15.5:0/3930667933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 msgr2=0x55becbff1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f1001d67640 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbff1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f1001d67640 1 -- 172.21.15.5:0/3930667933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 msgr2=0x55becbff5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f1001d67640 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becbff5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f1001d67640 1 -- 172.21.15.5:0/3930667933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55becbcb93c0 con 0x55becbfe25d0 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f1002568640 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becbff5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f1001d67640 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbff1410 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8000c00 tx=0x7f0ff80edad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3344fdc49fcda030 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f0fe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f0fe57fa640 1 -- 172.21.15.5:0/3930667933 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80bc100 con 0x55becbfe25d0 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f10063a4740 1 -- 172.21.15.5:0/3930667933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55becbf55860 con 0x55becbfe25d0 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.026+0000 7f0fe57fa640 1 -- 172.21.15.5:0/3930667933 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f0ff80baaa0 con 0x55becbfe25d0 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.027+0000 7f0fe57fa640 1 -- 172.21.15.5:0/3930667933 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0ff80bc2e0 con 0x55becbfe25d0 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.027+0000 7f10063a4740 10 client.27047 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.027+0000 7f10063a4740 1 -- 172.21.15.5:0/3930667933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55becbfefe30 con 0x55becbfe25d0 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.027+0000 7f0fe57fa640 1 -- 172.21.15.5:0/3930667933 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55becbf55860 con 0x55becbfe25d0 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.027+0000 7f0fe57fa640 1 -- 172.21.15.5:0/3930667933 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f0ff80b8e10 con 0x55becbfe25d0 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.027+0000 7f10063a4740 10 client.27047 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.027+0000 7f10063a4740 10 client.27047 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.027+0000 7f10063a4740 20 client.27047 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.027+0000 7f10063a4740 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55becc000ef0 0x55becc003330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.027+0000 7f10063a4740 4 client.27047 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:11:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.027+0000 7f10063a4740 1 -- 172.21.15.5:0/3930667933 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55becbf4fd70 con 0x55becc000ef0 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.027+0000 7f1001566640 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55becc000ef0 0x55becc003330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.028+0000 7f0fe57fa640 10 client.27047 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.028+0000 7f1001566640 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55becc000ef0 0x55becc003330 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8cfcf4a4b2514136 server_cookie=c7c54c87b534d312 in_seq=0 out_seq=0 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.028+0000 7f0fe57fa640 1 -- 172.21.15.5:0/3930667933 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55becbf4fd70 con 0x55becc000ef0 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.028+0000 7f0fe57fa640 10 client.27047 handle_command_reply: tid=0 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f10063a4740 1 client.27047 shutdown 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f10063a4740 1 -- 172.21.15.5:0/3930667933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 msgr2=0x55becbff1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f10063a4740 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbff1410 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8000c00 tx=0x7f0ff80edad0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f10063a4740 1 -- 172.21.15.5:0/3930667933 shutdown_connections 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f10063a4740 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55becc000ef0 0x55becc003330 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f10063a4740 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becbff5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f10063a4740 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbff1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f10063a4740 1 --2- 172.21.15.5:0/3930667933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbff1410 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f10063a4740 1 -- 172.21.15.5:0/3930667933 >> 172.21.15.5:0/3930667933 conn(0x55becbfab310 msgr2=0x55becbfe4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f0fe57fa640 0 client.27047 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:11:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f10063a4740 1 -- 172.21.15.5:0/3930667933 shutdown_connections 2024-09-19T06:11:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f10063a4740 1 -- 172.21.15.5:0/3930667933 wait complete. 2024-09-19T06:11:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.029+0000 7f10063a4740 20 client.27047 trim_cache size 0 max 16384 2024-09-19T06:11:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.045+0000 7f10063a4740 1 Processor -- start 2024-09-19T06:11:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.045+0000 7f10063a4740 1 -- start start 2024-09-19T06:11:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f10063a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbfddbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f10063a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfe3b40 0x55becc008c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f10063a4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfed370 0x55becc009140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f10063a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1ab620 con 0x55becbfe3b40 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f10063a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1ad160 con 0x55becbfe25d0 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f10063a4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1abcf0 con 0x55becbfed370 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f1001566640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfe3b40 0x55becc008c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f1001566640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfe3b40 0x55becc008c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36746/0 (socket says 172.21.15.5:36746) 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f1001d67640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbfddbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f1001566640 1 -- 172.21.15.5:0/1335886121 learned_addr learned my addr 172.21.15.5:0/1335886121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f1002568640 1 --2- 172.21.15.5:0/1335886121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfed370 0x55becc009140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f1001566640 1 -- 172.21.15.5:0/1335886121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfed370 msgr2=0x55becc009140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f1001566640 1 --2- 172.21.15.5:0/1335886121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfed370 0x55becc009140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f1001566640 1 -- 172.21.15.5:0/1335886121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 msgr2=0x55becbfddbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f1001566640 1 --2- 172.21.15.5:0/1335886121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbfddbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f1001566640 1 -- 172.21.15.5:0/1335886121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55becbfec940 con 0x55becbfe3b40 2024-09-19T06:11:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f1001d67640 1 --2- 172.21.15.5:0/1335886121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbfddbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.046+0000 7f1001566640 1 --2- 172.21.15.5:0/1335886121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfe3b40 0x55becc008c00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0fec000c00 tx=0x7f0fec00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c51594e306da80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.047+0000 7f0fe77fe640 1 -- 172.21.15.5:0/1335886121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec003d00 con 0x55becbfe3b40 2024-09-19T06:11:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.047+0000 7f0fe77fe640 1 -- 172.21.15.5:0/1335886121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fec0054d0 con 0x55becbfe3b40 2024-09-19T06:11:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.047+0000 7f0fe77fe640 1 -- 172.21.15.5:0/1335886121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec011440 con 0x55becbfe3b40 2024-09-19T06:11:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.048+0000 7f10063a4740 1 -- 172.21.15.5:0/1335886121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfe3b40 msgr2=0x55becc008c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.048+0000 7f10063a4740 1 --2- 172.21.15.5:0/1335886121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfe3b40 0x55becc008c00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0fec000c00 tx=0x7f0fec00b390 comp rx=0 tx=0).stop 2024-09-19T06:11:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.048+0000 7f10063a4740 1 -- 172.21.15.5:0/1335886121 shutdown_connections 2024-09-19T06:11:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.048+0000 7f10063a4740 1 --2- 172.21.15.5:0/1335886121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfed370 0x55becc009140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.048+0000 7f10063a4740 1 --2- 172.21.15.5:0/1335886121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfe3b40 0x55becc008c00 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.048+0000 7f10063a4740 1 --2- 172.21.15.5:0/1335886121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbfddbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.048+0000 7f10063a4740 1 -- 172.21.15.5:0/1335886121 >> 172.21.15.5:0/1335886121 conn(0x55becbfab310 msgr2=0x55becbfdd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.048+0000 7f10063a4740 1 -- 172.21.15.5:0/1335886121 shutdown_connections 2024-09-19T06:11:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.048+0000 7f10063a4740 1 -- 172.21.15.5:0/1335886121 wait complete. 2024-09-19T06:11:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.048+0000 7f10063a4740 1 -- start start 2024-09-19T06:11:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.048+0000 7f10063a4740 1 Processor -- start 2024-09-19T06:11:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f10063a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbff9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f10063a4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbffa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f10063a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becc063b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f10063a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1aca90 con 0x55becbfed370 2024-09-19T06:11:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f10063a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1ac3c0 con 0x55becbfe25d0 2024-09-19T06:11:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f10063a4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1a9620 con 0x55becbfe3b40 2024-09-19T06:11:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f1001d67640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbff9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f1001566640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbffa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f1001d67640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbff9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39190/0 (socket says 172.21.15.5:39190) 2024-09-19T06:11:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f1001d67640 1 -- 172.21.15.5:0/109970406 learned_addr learned my addr 172.21.15.5:0/109970406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f1002568640 1 --2- 172.21.15.5:0/109970406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becc063b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f10063a4740 10 client.0 _ll_register_callbacks cb 0x7f0fec00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f1001566640 1 -- 172.21.15.5:0/109970406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 msgr2=0x55becbff9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f1001566640 1 --2- 172.21.15.5:0/109970406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbff9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f1001566640 1 -- 172.21.15.5:0/109970406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 msgr2=0x55becc063b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f1001566640 1 --2- 172.21.15.5:0/109970406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becc063b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.049+0000 7f1001566640 1 -- 172.21.15.5:0/109970406 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55becc064390 con 0x55becbfe3b40 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.050+0000 7f1001566640 1 --2- 172.21.15.5:0/109970406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbffa120 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55becbff6780 tx=0x7f0fec004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33efe16ec3b3afa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.050+0000 7f0fe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.050+0000 7f0fe57fa640 1 -- 172.21.15.5:0/109970406 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec004730 con 0x55becbfe3b40 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.050+0000 7f0fe57fa640 1 -- 172.21.15.5:0/109970406 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f0fec0048d0 con 0x55becbfe3b40 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.050+0000 7f10063a4740 1 -- 172.21.15.5:0/109970406 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fec0054d0 con 0x55becbfe3b40 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.050+0000 7f0fe57fa640 1 -- 172.21.15.5:0/109970406 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0fec003b00 con 0x55becbfe3b40 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.050+0000 7f10063a4740 10 client.26974 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.050+0000 7f10063a4740 1 -- 172.21.15.5:0/109970406 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55becc06c730 con 0x55becbfe3b40 2024-09-19T06:11:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.050+0000 7f0fe57fa640 1 -- 172.21.15.5:0/109970406 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fec0054d0 con 0x55becbfe3b40 2024-09-19T06:11:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f0fe57fa640 1 -- 172.21.15.5:0/109970406 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f0fec011860 con 0x55becbfe3b40 2024-09-19T06:11:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f10063a4740 10 client.26974 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f10063a4740 10 client.26974 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:11:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f10063a4740 20 client.26974 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f10063a4740 1 --2- 172.21.15.5:0/109970406 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55becc06cda0 0x55becc06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f10063a4740 4 client.26974 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:11:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f10063a4740 1 -- 172.21.15.5:0/109970406 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55becbecc170 con 0x55becc06cda0 2024-09-19T06:11:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f1001d67640 1 --2- 172.21.15.5:0/109970406 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55becc06cda0 0x55becc06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f1001d67640 1 --2- 172.21.15.5:0/109970406 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55becc06cda0 0x55becc06f1c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88d95360447e91aa server_cookie=a50e6c55179d8611 in_seq=0 out_seq=0 2024-09-19T06:11:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f0fe57fa640 10 client.26974 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:11:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f0fe57fa640 1 -- 172.21.15.5:0/109970406 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55becbecc170 con 0x55becc06cda0 2024-09-19T06:11:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f0fe57fa640 10 client.26974 handle_command_reply: tid=0 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.051+0000 7f10063a4740 1 client.26974 shutdown 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.052+0000 7f10063a4740 1 -- 172.21.15.5:0/109970406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 msgr2=0x55becbffa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.052+0000 7f10063a4740 1 --2- 172.21.15.5:0/109970406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbffa120 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55becbff6780 tx=0x7f0fec004420 comp rx=0 tx=0).stop 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.052+0000 7f10063a4740 1 -- 172.21.15.5:0/109970406 shutdown_connections 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.052+0000 7f10063a4740 1 --2- 172.21.15.5:0/109970406 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55becc06cda0 0x55becc06f1c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.052+0000 7f10063a4740 1 --2- 172.21.15.5:0/109970406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55becbfed370 0x55becc063b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.052+0000 7f10063a4740 1 --2- 172.21.15.5:0/109970406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55becbfe3b40 0x55becbffa120 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.052+0000 7f10063a4740 1 --2- 172.21.15.5:0/109970406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55becbfe25d0 0x55becbff9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.052+0000 7f0fe57fa640 0 client.26974 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.052+0000 7f10063a4740 1 -- 172.21.15.5:0/109970406 >> 172.21.15.5:0/109970406 conn(0x55becbfab310 msgr2=0x55becbfe4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.052+0000 7f10063a4740 1 -- 172.21.15.5:0/109970406 shutdown_connections 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.052+0000 7f10063a4740 1 -- 172.21.15.5:0/109970406 wait complete. 2024-09-19T06:11:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.052+0000 7f10063a4740 20 client.26974 trim_cache size 0 max 16384 2024-09-19T06:11:18.062 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:11:18.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.053+0000 7f1003fcb640 1 -- 172.21.15.5:0/1762076187 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0fe80a33b0 msgr2=0x7f0fe80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:18.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.053+0000 7f1003fcb640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0fe80a33b0 0x7f0fe80a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4005c40 tx=0x7f0ff4004ab0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.053+0000 7f1003fcb640 1 -- 172.21.15.5:0/1762076187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc100a80 msgr2=0x7f0ffc1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.053+0000 7f1003fcb640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc100a80 0x7f0ffc1991e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8005940 tx=0x7f0ff80042f0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.053+0000 7f1003fcb640 1 -- 172.21.15.5:0/1762076187 shutdown_connections 2024-09-19T06:11:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.053+0000 7f1003fcb640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ffc106a80 0x7f0ffc1a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.053+0000 7f1003fcb640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0fe80a33b0 0x7f0fe80a5870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.053+0000 7f1003fcb640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ffc101420 0x7f0ffc199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.053+0000 7f1003fcb640 1 --2- 172.21.15.5:0/1762076187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ffc100a80 0x7f0ffc1991e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.053+0000 7f1003fcb640 1 -- 172.21.15.5:0/1762076187 >> 172.21.15.5:0/1762076187 conn(0x7f0ffc0fc7a0 msgr2=0x7f0ffc0fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.053+0000 7f1003fcb640 1 -- 172.21.15.5:0/1762076187 shutdown_connections 2024-09-19T06:11:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:18.053+0000 7f1003fcb640 1 -- 172.21.15.5:0/1762076187 wait complete. 2024-09-19T06:11:18.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:18 smithi005 ceph-mon[30955]: pgmap v246: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-09-19T06:11:18.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:11:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:18 smithi157 ceph-mon[35346]: pgmap v246: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-09-19T06:11:18.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 205373 2.71 MB/sec execute 284 sec latency 15.391 ms 2024-09-19T06:11:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:18 smithi049 ceph-mon[34640]: pgmap v246: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-09-19T06:11:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:11:19.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.519+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/1440144168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88102570 msgr2=0x7f2e881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:19.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.519+0000 7f2e8ce6d640 1 --2- 172.21.15.5:0/1440144168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88102570 0x7f2e881029b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009900 tx=0x7f2e7c02a480 comp rx=0 tx=0).stop 2024-09-19T06:11:19.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.519+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/1440144168 shutdown_connections 2024-09-19T06:11:19.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.519+0000 7f2e8ce6d640 1 --2- 172.21.15.5:0/1440144168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88102ef0 0x7f2e8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.519+0000 7f2e8ce6d640 1 --2- 172.21.15.5:0/1440144168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88102570 0x7f2e881029b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.519+0000 7f2e8ce6d640 1 --2- 172.21.15.5:0/1440144168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e88100e20 0x7f2e88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.519+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/1440144168 >> 172.21.15.5:0/1440144168 conn(0x7f2e880fc7d0 msgr2=0x7f2e880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:19.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.519+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/1440144168 shutdown_connections 2024-09-19T06:11:19.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.519+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/1440144168 wait complete. 2024-09-19T06:11:19.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e8ce6d640 1 Processor -- start 2024-09-19T06:11:19.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e8ce6d640 1 -- start start 2024-09-19T06:11:19.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e8ce6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88100e20 0x7f2e88198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e8ce6d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e88102570 0x7f2e881989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e8ce6d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88102ef0 0x7f2e8819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e8ce6d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88110e30 con 0x7f2e88100e20 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e8ce6d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88110cb0 con 0x7f2e88102570 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e8ce6d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88110fb0 con 0x7f2e88102ef0 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e8659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88100e20 0x7f2e88198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e85d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e88102570 0x7f2e881989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e8659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88100e20 0x7f2e88198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36768/0 (socket says 172.21.15.5:36768) 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e8659c640 1 -- 172.21.15.5:0/3345194161 learned_addr learned my addr 172.21.15.5:0/3345194161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.520+0000 7f2e86d9d640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88102ef0 0x7f2e8819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.521+0000 7f2e8659c640 1 -- 172.21.15.5:0/3345194161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88102ef0 msgr2=0x7f2e8819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.521+0000 7f2e8659c640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88102ef0 0x7f2e8819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.521+0000 7f2e8659c640 1 -- 172.21.15.5:0/3345194161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e88102570 msgr2=0x7f2e881989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.521+0000 7f2e8659c640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e88102570 0x7f2e881989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.521+0000 7f2e8659c640 1 -- 172.21.15.5:0/3345194161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8819d410 con 0x7f2e88100e20 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.521+0000 7f2e86d9d640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88102ef0 0x7f2e8819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.521+0000 7f2e8659c640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88100e20 0x7f2e88198460 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e9f0 tx=0x7f2e7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.521+0000 7f2e777fe640 1 -- 172.21.15.5:0/3345194161 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7800cd80 con 0x7f2e88100e20 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.521+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/3345194161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8819d700 con 0x7f2e88100e20 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.521+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/3345194161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e881a4fe0 con 0x7f2e88100e20 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.521+0000 7f2e777fe640 1 -- 172.21.15.5:0/3345194161 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e780108e0 con 0x7f2e88100e20 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.521+0000 7f2e777fe640 1 -- 172.21.15.5:0/3345194161 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78018920 con 0x7f2e88100e20 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.524+0000 7f2e777fe640 1 -- 172.21.15.5:0/3345194161 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2e78010430 con 0x7f2e88100e20 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.524+0000 7f2e8f246740 1 Processor -- start 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.524+0000 7f2e8f246740 1 -- start start 2024-09-19T06:11:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e8f246740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443aa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e8f246740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443aa5d0 0x55ab443aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e8f246740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443bbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e8f246740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab443102f0 con 0x55ab443abb40 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e8f246740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ab442b7cf0 con 0x55ab443b9780 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e8f246740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ab4430c2b0 con 0x55ab443aa5d0 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e86d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e86d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443aa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36780/0 (socket says 172.21.15.5:36780) 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e8659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443aa5d0 0x55ab443aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e86d9d640 1 -- 172.21.15.5:0/2292645518 learned_addr learned my addr 172.21.15.5:0/2292645518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e85d9b640 1 --2- 172.21.15.5:0/2292645518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443bbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e777fe640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2e700a3430 0x7f2e700a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e85d9b640 1 -- 172.21.15.5:0/2292645518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443aa5d0 msgr2=0x55ab443aaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:19.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e85d9b640 1 --2- 172.21.15.5:0/2292645518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443aa5d0 0x55ab443aaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e85d9b640 1 -- 172.21.15.5:0/2292645518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 msgr2=0x55ab443aa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e85d9b640 1 --2- 172.21.15.5:0/2292645518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443aa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.525+0000 7f2e85d9b640 1 -- 172.21.15.5:0/2292645518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab4429c3a0 con 0x55ab443b9780 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.526+0000 7f2e86d9d640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2e700a3430 0x7f2e700a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.526+0000 7f2e777fe640 1 -- 172.21.15.5:0/3345194161 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f2e780cffe0 con 0x7f2e88100e20 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.526+0000 7f2e85d9b640 1 --2- 172.21.15.5:0/2292645518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443bbb70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c005940 tx=0x7f2e7c005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc6fb375a146fe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.526+0000 7f2e74ff9640 1 -- 172.21.15.5:0/2292645518 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c040070 con 0x55ab443b9780 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.526+0000 7f2e74ff9640 1 -- 172.21.15.5:0/2292645518 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e7c004030 con 0x55ab443b9780 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.526+0000 7f2e74ff9640 1 -- 172.21.15.5:0/2292645518 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c035040 con 0x55ab443b9780 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.527+0000 7f2e86d9d640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2e700a3430 0x7f2e700a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e80005c40 tx=0x7f2e80004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.527+0000 7f2e8f246740 1 -- 172.21.15.5:0/2292645518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 msgr2=0x55ab443bbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.527+0000 7f2e8f246740 1 --2- 172.21.15.5:0/2292645518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443bbb70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c005940 tx=0x7f2e7c005bd0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.527+0000 7f2e8f246740 1 -- 172.21.15.5:0/2292645518 shutdown_connections 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.527+0000 7f2e8f246740 1 --2- 172.21.15.5:0/2292645518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443bbb70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.527+0000 7f2e8f246740 1 --2- 172.21.15.5:0/2292645518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443aa5d0 0x55ab443aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.527+0000 7f2e8f246740 1 --2- 172.21.15.5:0/2292645518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443aa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.527+0000 7f2e8f246740 1 -- 172.21.15.5:0/2292645518 >> 172.21.15.5:0/2292645518 conn(0x55ab44373310 msgr2=0x55ab443a6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.527+0000 7f2e8f246740 1 -- 172.21.15.5:0/2292645518 shutdown_connections 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.527+0000 7f2e8f246740 1 -- 172.21.15.5:0/2292645518 wait complete. 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.527+0000 7f2e8f246740 1 -- start start 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e8f246740 1 Processor -- start 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e8f246740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443aa5d0 0x55ab443b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e8f246740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443abb40 0x55ab443b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e8f246740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e8f246740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab443bc420 con 0x55ab443aa5d0 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e8f246740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ab442cb060 con 0x55ab443b9780 2024-09-19T06:11:19.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e8f246740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ab443bc5a0 con 0x55ab443abb40 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e8f246740 10 client.0 _ll_register_callbacks cb 0x55ab4401e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e85d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443abb40 0x55ab443b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e8659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443aa5d0 0x55ab443b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e85d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443abb40 0x55ab443b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50054/0 (socket says 172.21.15.5:50054) 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e86d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e85d9b640 1 -- 172.21.15.5:0/87129605 learned_addr learned my addr 172.21.15.5:0/87129605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e85d9b640 1 -- 172.21.15.5:0/87129605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 msgr2=0x55ab443b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e85d9b640 1 --2- 172.21.15.5:0/87129605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e85d9b640 1 -- 172.21.15.5:0/87129605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443aa5d0 msgr2=0x55ab443b8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e85d9b640 1 --2- 172.21.15.5:0/87129605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443aa5d0 0x55ab443b8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e85d9b640 1 -- 172.21.15.5:0/87129605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab440813c0 con 0x55ab443abb40 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.528+0000 7f2e8659c640 1 --2- 172.21.15.5:0/87129605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443aa5d0 0x55ab443b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e85d9b640 1 --2- 172.21.15.5:0/87129605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443abb40 0x55ab443b1fa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c006010 tx=0x7f2e7c0043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dee2629c45ea0bda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e6dffb640 1 -- 172.21.15.5:0/87129605 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c040070 con 0x55ab443abb40 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e6dffb640 1 -- 172.21.15.5:0/87129605 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f2e7c031640 con 0x55ab443abb40 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e8f246740 1 -- 172.21.15.5:0/87129605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e7c004030 con 0x55ab443abb40 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e6dffb640 1 -- 172.21.15.5:0/87129605 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2e7c055050 con 0x55ab443abb40 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e8f246740 10 client.26986 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e8f246740 1 -- 172.21.15.5:0/87129605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ab443bc450 con 0x55ab443abb40 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e6dffb640 1 -- 172.21.15.5:0/87129605 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e7c005640 con 0x55ab443abb40 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e6dffb640 1 -- 172.21.15.5:0/87129605 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2e7c031ce0 con 0x55ab443abb40 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e8f246740 10 client.26986 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e8f246740 10 client.26986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.529+0000 7f2e8f246740 20 client.26986 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.530+0000 7f2e8f246740 1 --2- 172.21.15.5:0/87129605 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab443cd460 0x55ab443cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.530+0000 7f2e8f246740 4 client.26986 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.530+0000 7f2e8f246740 1 -- 172.21.15.5:0/87129605 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2e7c004030 con 0x55ab443cd460 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.530+0000 7f2e8659c640 1 --2- 172.21.15.5:0/87129605 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab443cd460 0x55ab443cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.530+0000 7f2e6dffb640 10 client.26986 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.530+0000 7f2e8659c640 1 --2- 172.21.15.5:0/87129605 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab443cd460 0x55ab443cf850 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8a20c8d8b5071d server_cookie=e98a898de6732ebd in_seq=0 out_seq=0 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e6dffb640 1 -- 172.21.15.5:0/87129605 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2e7c004030 con 0x55ab443cd460 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e6dffb640 10 client.26986 handle_command_reply: tid=0 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e8f246740 1 client.26986 shutdown 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e8f246740 1 -- 172.21.15.5:0/87129605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443abb40 msgr2=0x55ab443b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e8f246740 1 --2- 172.21.15.5:0/87129605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443abb40 0x55ab443b1fa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c006010 tx=0x7f2e7c0043f0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e8f246740 1 -- 172.21.15.5:0/87129605 shutdown_connections 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e8f246740 1 --2- 172.21.15.5:0/87129605 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab443cd460 0x55ab443cf850 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e6dffb640 0 client.26986 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e8f246740 1 --2- 172.21.15.5:0/87129605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e8f246740 1 --2- 172.21.15.5:0/87129605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443abb40 0x55ab443b1fa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e8f246740 1 --2- 172.21.15.5:0/87129605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443aa5d0 0x55ab443b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e8f246740 1 -- 172.21.15.5:0/87129605 >> 172.21.15.5:0/87129605 conn(0x55ab44373310 msgr2=0x55ab443ac930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e8f246740 1 -- 172.21.15.5:0/87129605 shutdown_connections 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e8f246740 1 -- 172.21.15.5:0/87129605 wait complete. 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.531+0000 7f2e8f246740 20 client.26986 trim_cache size 0 max 16384 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.546+0000 7f2e8f246740 1 Processor -- start 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.546+0000 7f2e8f246740 1 -- start start 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8f246740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443aa5d0 0x55ab443b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8f246740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8f246740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443d5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8f246740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6800e7e0 con 0x55ab443abb40 2024-09-19T06:11:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8f246740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2e68010320 con 0x55ab443b9780 2024-09-19T06:11:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8f246740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6800eeb0 con 0x55ab443aa5d0 2024-09-19T06:11:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e85d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443aa5d0 0x55ab443b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e85d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443d5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36798/0 (socket says 172.21.15.5:36798) 2024-09-19T06:11:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e85d9b640 1 -- 172.21.15.5:0/1080497075 learned_addr learned my addr 172.21.15.5:0/1080497075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e86d9d640 1 --2- 172.21.15.5:0/1080497075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443d5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8659c640 1 -- 172.21.15.5:0/1080497075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 msgr2=0x55ab443d5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8659c640 1 --2- 172.21.15.5:0/1080497075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443d5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8659c640 1 -- 172.21.15.5:0/1080497075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 msgr2=0x55ab443d5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8659c640 1 --2- 172.21.15.5:0/1080497075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443d5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8659c640 1 -- 172.21.15.5:0/1080497075 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab443b0510 con 0x55ab443aa5d0 2024-09-19T06:11:19.567 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:11:19.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e8659c640 1 --2- 172.21.15.5:0/1080497075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443aa5d0 0x55ab443b5f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e780cd6f0 tx=0x7f2e780bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad07c9ba807d7e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:19.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e6ffff640 1 -- 172.21.15.5:0/1080497075 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7809bbe0 con 0x55ab443aa5d0 2024-09-19T06:11:19.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.547+0000 7f2e6ffff640 1 -- 172.21.15.5:0/1080497075 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e78010720 con 0x55ab443aa5d0 2024-09-19T06:11:19.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.548+0000 7f2e6ffff640 1 -- 172.21.15.5:0/1080497075 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7809a610 con 0x55ab443aa5d0 2024-09-19T06:11:19.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.548+0000 7f2e8f246740 1 -- 172.21.15.5:0/1080497075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443aa5d0 msgr2=0x55ab443b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:19.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.548+0000 7f2e8f246740 1 --2- 172.21.15.5:0/1080497075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443aa5d0 0x55ab443b5f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e780cd6f0 tx=0x7f2e780bc040 comp rx=0 tx=0).stop 2024-09-19T06:11:19.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.548+0000 7f2e8f246740 1 -- 172.21.15.5:0/1080497075 shutdown_connections 2024-09-19T06:11:19.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.548+0000 7f2e8f246740 1 --2- 172.21.15.5:0/1080497075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443b9780 0x55ab443d5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.548+0000 7f2e8f246740 1 --2- 172.21.15.5:0/1080497075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.548+0000 7f2e8f246740 1 --2- 172.21.15.5:0/1080497075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443aa5d0 0x55ab443b5f20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.548+0000 7f2e8f246740 1 -- 172.21.15.5:0/1080497075 >> 172.21.15.5:0/1080497075 conn(0x55ab44373310 msgr2=0x55ab443b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.548+0000 7f2e8f246740 1 -- 172.21.15.5:0/1080497075 shutdown_connections 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.548+0000 7f2e8f246740 1 -- 172.21.15.5:0/1080497075 wait complete. 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e8f246740 1 -- start start 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e8f246740 1 Processor -- start 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e8f246740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443aa5d0 0x55ab443caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e8f246740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443aef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e8f246740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443b9780 0x55ab443af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e8f246740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6800fc50 con 0x55ab443abb40 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e8f246740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6800f580 con 0x55ab443aa5d0 2024-09-19T06:11:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e8f246740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6800c7e0 con 0x55ab443b9780 2024-09-19T06:11:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e8f246740 10 client.0 _ll_register_callbacks cb 0x7f2e80009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e85d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e85d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36812/0 (socket says 172.21.15.5:36812) 2024-09-19T06:11:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e8659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443aa5d0 0x55ab443caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e85d9b640 1 -- 172.21.15.5:0/2570384874 learned_addr learned my addr 172.21.15.5:0/2570384874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.549+0000 7f2e86d9d640 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443b9780 0x55ab443af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e8659c640 1 -- 172.21.15.5:0/2570384874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443b9780 msgr2=0x55ab443af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e8659c640 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443b9780 0x55ab443af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e8659c640 1 -- 172.21.15.5:0/2570384874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 msgr2=0x55ab443aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e8659c640 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e8659c640 1 -- 172.21.15.5:0/2570384874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab443afda0 con 0x55ab443aa5d0 2024-09-19T06:11:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e85d9b640 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e8659c640 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443aa5d0 0x55ab443caae0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2e7809c1c0 tx=0x7f2e780d3ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63bce1267bd50e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e6dffb640 1 -- 172.21.15.5:0/2570384874 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7809c980 con 0x55ab443aa5d0 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e6dffb640 1 -- 172.21.15.5:0/2570384874 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f2e780cd970 con 0x55ab443aa5d0 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e8f246740 1 -- 172.21.15.5:0/2570384874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ab4431d860 con 0x55ab443aa5d0 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e6dffb640 1 -- 172.21.15.5:0/2570384874 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2e780cdb70 con 0x55ab443aa5d0 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e8f246740 10 client.27071 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e8f246740 1 -- 172.21.15.5:0/2570384874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ab444380f0 con 0x55ab443aa5d0 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.550+0000 7f2e6dffb640 1 -- 172.21.15.5:0/2570384874 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ab4431d860 con 0x55ab443aa5d0 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e6dffb640 1 -- 172.21.15.5:0/2570384874 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2e780d3ce0 con 0x55ab443aa5d0 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e8f246740 10 client.27071 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e8f246740 10 client.27071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e8f246740 20 client.27071 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e8f246740 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab443cd460 0x55ab4443a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e8f246740 4 client.27071 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e8f246740 1 -- 172.21.15.5:0/2570384874 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab44317d70 con 0x55ab443cd460 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e85d9b640 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab443cd460 0x55ab4443a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e85d9b640 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab443cd460 0x55ab4443a7a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0f8516579a6f3f server_cookie=63c53ea3461332b3 in_seq=0 out_seq=0 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e6dffb640 10 client.27071 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e6dffb640 1 -- 172.21.15.5:0/2570384874 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab44317d70 con 0x55ab443cd460 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e6dffb640 10 client.27071 handle_command_reply: tid=0 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e8f246740 1 client.27071 shutdown 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e8f246740 1 -- 172.21.15.5:0/2570384874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443aa5d0 msgr2=0x55ab443caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.551+0000 7f2e8f246740 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443aa5d0 0x55ab443caae0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2e7809c1c0 tx=0x7f2e780d3ad0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8f246740 1 -- 172.21.15.5:0/2570384874 shutdown_connections 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8f246740 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab443cd460 0x55ab4443a7a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8f246740 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab443b9780 0x55ab443af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e6dffb640 0 client.27071 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8f246740 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab443abb40 0x55ab443aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8f246740 1 --2- 172.21.15.5:0/2570384874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab443aa5d0 0x55ab443caae0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8f246740 1 -- 172.21.15.5:0/2570384874 >> 172.21.15.5:0/2570384874 conn(0x55ab44373310 msgr2=0x55ab443a5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8f246740 1 -- 172.21.15.5:0/2570384874 shutdown_connections 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8f246740 1 -- 172.21.15.5:0/2570384874 wait complete. 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8f246740 20 client.27071 trim_cache size 0 max 16384 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/3345194161 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2e700a3430 msgr2=0x7f2e700a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8ce6d640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2e700a3430 0x7f2e700a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e80005c40 tx=0x7f2e80004a40 comp rx=0 tx=0).stop 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/3345194161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88100e20 msgr2=0x7f2e88198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8ce6d640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88100e20 0x7f2e88198460 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e9f0 tx=0x7f2e7800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/3345194161 shutdown_connections 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8ce6d640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2e700a3430 0x7f2e700a58f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8ce6d640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88102ef0 0x7f2e8819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8ce6d640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e88102570 0x7f2e881989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8ce6d640 1 --2- 172.21.15.5:0/3345194161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88100e20 0x7f2e88198460 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:19.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.552+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/3345194161 >> 172.21.15.5:0/3345194161 conn(0x7f2e880fc7d0 msgr2=0x7f2e8806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:19.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.553+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/3345194161 shutdown_connections 2024-09-19T06:11:19.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:19.553+0000 7f2e8ce6d640 1 -- 172.21.15.5:0/3345194161 wait complete. 2024-09-19T06:11:19.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 205749 2.72 MB/sec execute 285 sec latency 69.329 ms 2024-09-19T06:11:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:20 smithi005 ceph-mon[30955]: pgmap v247: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-09-19T06:11:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:20 smithi157 ceph-mon[35346]: pgmap v247: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-09-19T06:11:20.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 206109 2.72 MB/sec execute 286 sec latency 12.618 ms 2024-09-19T06:11:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:20 smithi049 ceph-mon[34640]: pgmap v247: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-09-19T06:11:21.268 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:11:21.269 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:11:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:11:21.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 206397 2.71 MB/sec execute 287 sec latency 13.898 ms 2024-09-19T06:11:22.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.084+0000 7ff47338d640 1 -- 172.21.15.5:0/1580124429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff46c1036e0 msgr2=0x7ff46c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:22.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.084+0000 7ff47338d640 1 --2- 172.21.15.5:0/1580124429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff46c1036e0 0x7ff46c107b90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009930 tx=0x7ff45c02a420 comp rx=0 tx=0).stop 2024-09-19T06:11:22.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.084+0000 7ff47338d640 1 -- 172.21.15.5:0/1580124429 shutdown_connections 2024-09-19T06:11:22.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.084+0000 7ff47338d640 1 --2- 172.21.15.5:0/1580124429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1082a0 0x7ff46c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.084+0000 7ff47338d640 1 --2- 172.21.15.5:0/1580124429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff46c1036e0 0x7ff46c107b90 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.084+0000 7ff47338d640 1 --2- 172.21.15.5:0/1580124429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff46c102d30 0x7ff46c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.084+0000 7ff47338d640 1 -- 172.21.15.5:0/1580124429 >> 172.21.15.5:0/1580124429 conn(0x7ff46c0fc7d0 msgr2=0x7ff46c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.085+0000 7ff47338d640 1 -- 172.21.15.5:0/1580124429 shutdown_connections 2024-09-19T06:11:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.085+0000 7ff47338d640 1 -- 172.21.15.5:0/1580124429 wait complete. 2024-09-19T06:11:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.085+0000 7ff47338d640 1 Processor -- start 2024-09-19T06:11:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.085+0000 7ff47338d640 1 -- start start 2024-09-19T06:11:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.085+0000 7ff47338d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff46c102d30 0x7ff46c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.085+0000 7ff47338d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1036e0 0x7ff46c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff47338d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff46c1082a0 0x7ff46c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff47338d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c111f60 con 0x7ff46c1082a0 2024-09-19T06:11:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff47338d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c111de0 con 0x7ff46c1036e0 2024-09-19T06:11:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff47338d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1120e0 con 0x7ff46c102d30 2024-09-19T06:11:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff471129640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff46c102d30 0x7ff46c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff470928640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1036e0 0x7ff46c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff471129640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff46c102d30 0x7ff46c198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50088/0 (socket says 172.21.15.5:50088) 2024-09-19T06:11:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff471129640 1 -- 172.21.15.5:0/745327167 learned_addr learned my addr 172.21.15.5:0/745327167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff47192a640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff46c1082a0 0x7ff46c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff471129640 1 -- 172.21.15.5:0/745327167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1036e0 msgr2=0x7ff46c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff471129640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1036e0 0x7ff46c198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff471129640 1 -- 172.21.15.5:0/745327167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff46c1082a0 msgr2=0x7ff46c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff471129640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff46c1082a0 0x7ff46c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff471129640 1 -- 172.21.15.5:0/745327167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c19d510 con 0x7ff46c102d30 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff471129640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff46c102d30 0x7ff46c198560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff46800e9f0 tx=0x7ff46800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff4627fc640 1 -- 172.21.15.5:0/745327167 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46800ccc0 con 0x7ff46c102d30 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff4627fc640 1 -- 172.21.15.5:0/745327167 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff46800ce60 con 0x7ff46c102d30 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff4627fc640 1 -- 172.21.15.5:0/745327167 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff468010430 con 0x7ff46c102d30 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.086+0000 7ff47338d640 1 -- 172.21.15.5:0/745327167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46c19d800 con 0x7ff46c102d30 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.087+0000 7ff47338d640 1 -- 172.21.15.5:0/745327167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46c1a50e0 con 0x7ff46c102d30 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.089+0000 7ff475766740 1 Processor -- start 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.089+0000 7ff4627fc640 1 -- 172.21.15.5:0/745327167 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff468005020 con 0x7ff46c102d30 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.089+0000 7ff475766740 1 -- start start 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.089+0000 7ff475766740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.089+0000 7ff475766740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9a6480 0x55606e9b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff475766740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9b0d40 0x55606e9b3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff475766740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55606e90b280 con 0x55606e9a6480 2024-09-19T06:11:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff475766740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55606e8b2b10 con 0x55606e9a5b60 2024-09-19T06:11:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff475766740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55606e907240 con 0x55606e9b0d40 2024-09-19T06:11:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff471129640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9a6480 0x55606e9b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff470928640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9b0d40 0x55606e9b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff470928640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9b0d40 0x55606e9b3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50096/0 (socket says 172.21.15.5:50096) 2024-09-19T06:11:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff470928640 1 -- 172.21.15.5:0/184532869 learned_addr learned my addr 172.21.15.5:0/184532869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff470928640 1 -- 172.21.15.5:0/184532869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 msgr2=0x55606e9a5f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:11:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff470928640 1 --2- 172.21.15.5:0/184532869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9a5f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff470928640 1 -- 172.21.15.5:0/184532869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9a6480 msgr2=0x55606e9b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff470928640 1 --2- 172.21.15.5:0/184532869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9a6480 0x55606e9b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff470928640 1 -- 172.21.15.5:0/184532869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606e8bf1f0 con 0x55606e9b0d40 2024-09-19T06:11:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.090+0000 7ff470928640 1 --2- 172.21.15.5:0/184532869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9b0d40 0x55606e9b3130 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009900 tx=0x7ff45c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63121d7c6854a7b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.091+0000 7ff4627fc640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4580a3430 0x7ff4580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.091+0000 7ff4577fe640 1 -- 172.21.15.5:0/184532869 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c035040 con 0x55606e9b0d40 2024-09-19T06:11:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.091+0000 7ff47192a640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4580a3430 0x7ff4580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.091+0000 7ff4577fe640 1 -- 172.21.15.5:0/184532869 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff45c004430 con 0x55606e9b0d40 2024-09-19T06:11:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.091+0000 7ff4627fc640 1 -- 172.21.15.5:0/745327167 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7ff4680d0100 con 0x7ff46c102d30 2024-09-19T06:11:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.091+0000 7ff4577fe640 1 -- 172.21.15.5:0/184532869 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0035f0 con 0x55606e9b0d40 2024-09-19T06:11:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.092+0000 7ff475766740 1 -- 172.21.15.5:0/184532869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9b0d40 msgr2=0x55606e9b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.092+0000 7ff475766740 1 --2- 172.21.15.5:0/184532869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9b0d40 0x55606e9b3130 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009900 tx=0x7ff45c004040 comp rx=0 tx=0).stop 2024-09-19T06:11:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.092+0000 7ff47192a640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4580a3430 0x7ff4580a58f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff4640099a0 tx=0x7ff464009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.092+0000 7ff475766740 1 -- 172.21.15.5:0/184532869 shutdown_connections 2024-09-19T06:11:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.092+0000 7ff475766740 1 --2- 172.21.15.5:0/184532869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9b0d40 0x55606e9b3130 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.092+0000 7ff475766740 1 --2- 172.21.15.5:0/184532869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9a6480 0x55606e9b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.092+0000 7ff475766740 1 --2- 172.21.15.5:0/184532869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.092+0000 7ff475766740 1 -- 172.21.15.5:0/184532869 >> 172.21.15.5:0/184532869 conn(0x55606e9a20f0 msgr2=0x55606e9a24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.092+0000 7ff475766740 1 -- 172.21.15.5:0/184532869 shutdown_connections 2024-09-19T06:11:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.092+0000 7ff475766740 1 -- 172.21.15.5:0/184532869 wait complete. 2024-09-19T06:11:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.092+0000 7ff475766740 1 -- start start 2024-09-19T06:11:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.092+0000 7ff475766740 1 Processor -- start 2024-09-19T06:11:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff475766740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff475766740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9a6480 0x55606e9ae150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff475766740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9b0d40 0x55606e9ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff475766740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55606e9b39b0 con 0x55606e9b0d40 2024-09-19T06:11:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff475766740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55606e8c5f10 con 0x55606e9a5b60 2024-09-19T06:11:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff475766740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55606e9b3b30 con 0x55606e9a6480 2024-09-19T06:11:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff475766740 10 client.0 _ll_register_callbacks cb 0x55606e88ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff471129640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff47192a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9b0d40 0x55606e9ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff471129640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39268/0 (socket says 172.21.15.5:39268) 2024-09-19T06:11:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff471129640 1 -- 172.21.15.5:0/2439264325 learned_addr learned my addr 172.21.15.5:0/2439264325 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff470928640 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9a6480 0x55606e9ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff471129640 1 -- 172.21.15.5:0/2439264325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9a6480 msgr2=0x55606e9ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff471129640 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9a6480 0x55606e9ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff471129640 1 -- 172.21.15.5:0/2439264325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9b0d40 msgr2=0x55606e9ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff471129640 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9b0d40 0x55606e9ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff471129640 1 -- 172.21.15.5:0/2439264325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606e67c6a0 con 0x55606e9a5b60 2024-09-19T06:11:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff470928640 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9a6480 0x55606e9ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:11:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff471129640 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9adc10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff46800e9c0 tx=0x7ff468098600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ac7e11545f6955 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff454ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:11:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.093+0000 7ff454ff9640 1 -- 172.21.15.5:0/2439264325 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46809bcc0 con 0x55606e9a5b60 2024-09-19T06:11:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff475766740 1 -- 172.21.15.5:0/2439264325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff45c004430 con 0x55606e9a5b60 2024-09-19T06:11:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff454ff9640 1 -- 172.21.15.5:0/2439264325 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7ff4680cda50 con 0x55606e9a5b60 2024-09-19T06:11:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff454ff9640 1 -- 172.21.15.5:0/2439264325 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff4680cdc50 con 0x55606e9a5b60 2024-09-19T06:11:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff475766740 10 client.27080 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff475766740 1 -- 172.21.15.5:0/2439264325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55606e9bf600 con 0x55606e9a5b60 2024-09-19T06:11:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff454ff9640 1 -- 172.21.15.5:0/2439264325 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff45c004430 con 0x55606e9a5b60 2024-09-19T06:11:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff454ff9640 1 -- 172.21.15.5:0/2439264325 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff468005310 con 0x55606e9a5b60 2024-09-19T06:11:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff475766740 10 client.27080 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff475766740 10 client.27080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff475766740 20 client.27080 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff475766740 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55606e9c0a10 0x55606e9c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff475766740 4 client.27080 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:11:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff475766740 1 -- 172.21.15.5:0/2439264325 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55606e9187f0 con 0x55606e9c0a10 2024-09-19T06:11:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.094+0000 7ff470928640 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55606e9c0a10 0x55606e9c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.095+0000 7ff454ff9640 10 client.27080 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:22.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.095+0000 7ff470928640 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55606e9c0a10 0x55606e9c2ea0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3647025f06be076f server_cookie=bbfae5fa3a3877ec in_seq=0 out_seq=0 2024-09-19T06:11:22.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff454ff9640 1 -- 172.21.15.5:0/2439264325 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55606e9187f0 con 0x55606e9c0a10 2024-09-19T06:11:22.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff454ff9640 10 client.27080 handle_command_reply: tid=0 2024-09-19T06:11:22.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff475766740 1 client.27080 shutdown 2024-09-19T06:11:22.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff475766740 1 -- 172.21.15.5:0/2439264325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 msgr2=0x55606e9adc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:22.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff475766740 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9adc10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff46800e9c0 tx=0x7ff468098600 comp rx=0 tx=0).stop 2024-09-19T06:11:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff475766740 1 -- 172.21.15.5:0/2439264325 shutdown_connections 2024-09-19T06:11:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff475766740 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55606e9c0a10 0x55606e9c2ea0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff475766740 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9b0d40 0x55606e9ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff475766740 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9a6480 0x55606e9ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff475766740 1 --2- 172.21.15.5:0/2439264325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9adc10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff454ff9640 0 client.27080 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff475766740 1 -- 172.21.15.5:0/2439264325 >> 172.21.15.5:0/2439264325 conn(0x55606e9a20f0 msgr2=0x55606e9a7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff475766740 1 -- 172.21.15.5:0/2439264325 shutdown_connections 2024-09-19T06:11:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff475766740 1 -- 172.21.15.5:0/2439264325 wait complete. 2024-09-19T06:11:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.096+0000 7ff475766740 20 client.27080 trim_cache size 0 max 16384 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.114+0000 7ff475766740 1 Processor -- start 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff475766740 1 -- start start 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff475766740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9a9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff475766740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9a6480 0x55606e9acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff475766740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9b0d40 0x55606e9c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff475766740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1ad7e0 con 0x55606e9a6480 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff475766740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1af320 con 0x55606e9a5b60 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff475766740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1adeb0 con 0x55606e9b0d40 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff470928640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9a6480 0x55606e9acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff470928640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9a6480 0x55606e9acac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36870/0 (socket says 172.21.15.5:36870) 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff470928640 1 -- 172.21.15.5:0/3772701724 learned_addr learned my addr 172.21.15.5:0/3772701724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff471129640 1 --2- 172.21.15.5:0/3772701724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff47192a640 1 --2- 172.21.15.5:0/3772701724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9b0d40 0x55606e9c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff470928640 1 -- 172.21.15.5:0/3772701724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9b0d40 msgr2=0x55606e9c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff470928640 1 --2- 172.21.15.5:0/3772701724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9b0d40 0x55606e9c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff470928640 1 -- 172.21.15.5:0/3772701724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 msgr2=0x55606e9a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff470928640 1 --2- 172.21.15.5:0/3772701724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff470928640 1 -- 172.21.15.5:0/3772701724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606e9a1f20 con 0x55606e9a6480 2024-09-19T06:11:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff471129640 1 --2- 172.21.15.5:0/3772701724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.115+0000 7ff470928640 1 --2- 172.21.15.5:0/3772701724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9a6480 0x55606e9acac0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff45c006010 tx=0x7ff45c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a20b9994c9b279 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.116+0000 7ff456ffd640 1 -- 172.21.15.5:0/3772701724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c035040 con 0x55606e9a6480 2024-09-19T06:11:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.116+0000 7ff456ffd640 1 -- 172.21.15.5:0/3772701724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff45c003ca0 con 0x55606e9a6480 2024-09-19T06:11:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.116+0000 7ff456ffd640 1 -- 172.21.15.5:0/3772701724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c03cb30 con 0x55606e9a6480 2024-09-19T06:11:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.116+0000 7ff475766740 1 -- 172.21.15.5:0/3772701724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9a6480 msgr2=0x55606e9acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.116+0000 7ff475766740 1 --2- 172.21.15.5:0/3772701724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9a6480 0x55606e9acac0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff45c006010 tx=0x7ff45c0045e0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.116+0000 7ff475766740 1 -- 172.21.15.5:0/3772701724 shutdown_connections 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.116+0000 7ff475766740 1 --2- 172.21.15.5:0/3772701724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9b0d40 0x55606e9c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.116+0000 7ff475766740 1 --2- 172.21.15.5:0/3772701724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9a6480 0x55606e9acac0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.116+0000 7ff475766740 1 --2- 172.21.15.5:0/3772701724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a5b60 0x55606e9a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.116+0000 7ff475766740 1 -- 172.21.15.5:0/3772701724 >> 172.21.15.5:0/3772701724 conn(0x55606e9a20f0 msgr2=0x55606e9ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff475766740 1 -- 172.21.15.5:0/3772701724 shutdown_connections 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff475766740 1 -- 172.21.15.5:0/3772701724 wait complete. 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff475766740 1 -- start start 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff475766740 1 Processor -- start 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff475766740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9a5b60 0x55606e9b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff475766740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a6480 0x55606e9b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff475766740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9b0d40 0x55606e9b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff475766740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1aec50 con 0x55606e9b0d40 2024-09-19T06:11:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff475766740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1ae580 con 0x55606e9a6480 2024-09-19T06:11:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff475766740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1ab7e0 con 0x55606e9a5b60 2024-09-19T06:11:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff475766740 10 client.0 _ll_register_callbacks cb 0x7ff4680d38a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff470928640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a6480 0x55606e9b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.117+0000 7ff47192a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9b0d40 0x55606e9b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.175 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:11:22.175 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:11:22.175 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8ef354e7-e6be-43ae-b908-bf8d58eaf5f5", 2024-09-19T06:11:22.176 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:11:22.176 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:11:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff47192a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9b0d40 0x55606e9b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36878/0 (socket says 172.21.15.5:36878) 2024-09-19T06:11:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff470928640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a6480 0x55606e9b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39286/0 (socket says 172.21.15.5:39286) 2024-09-19T06:11:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff47192a640 1 -- 172.21.15.5:0/2523605509 learned_addr learned my addr 172.21.15.5:0/2523605509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff471129640 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9a5b60 0x55606e9b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff47192a640 1 -- 172.21.15.5:0/2523605509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9a5b60 msgr2=0x55606e9b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff47192a640 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9a5b60 0x55606e9b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff47192a640 1 -- 172.21.15.5:0/2523605509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a6480 msgr2=0x55606e9b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff47192a640 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a6480 0x55606e9b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff47192a640 1 -- 172.21.15.5:0/2523605509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606e99d830 con 0x55606e9b0d40 2024-09-19T06:11:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff470928640 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a6480 0x55606e9b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff47192a640 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9b0d40 0x55606e9b56d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff464004310 tx=0x7ff464004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836bf2403a2d18d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff454ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff454ff9640 1 -- 172.21.15.5:0/2523605509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46402d060 con 0x55606e9b0d40 2024-09-19T06:11:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff454ff9640 1 -- 172.21.15.5:0/2523605509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7ff46401fb40 con 0x55606e9b0d40 2024-09-19T06:11:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.118+0000 7ff475766740 1 -- 172.21.15.5:0/2523605509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff45c003ca0 con 0x55606e9b0d40 2024-09-19T06:11:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.119+0000 7ff454ff9640 1 -- 172.21.15.5:0/2523605509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff46401fd40 con 0x55606e9b0d40 2024-09-19T06:11:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.119+0000 7ff475766740 10 client.17520 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.119+0000 7ff475766740 1 -- 172.21.15.5:0/2523605509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55606ea2baa0 con 0x55606e9b0d40 2024-09-19T06:11:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.119+0000 7ff454ff9640 1 -- 172.21.15.5:0/2523605509 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff45c003ca0 con 0x55606e9b0d40 2024-09-19T06:11:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.119+0000 7ff454ff9640 1 -- 172.21.15.5:0/2523605509 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff4640303d0 con 0x55606e9b0d40 2024-09-19T06:11:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.119+0000 7ff475766740 10 client.17520 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.119+0000 7ff475766740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.119+0000 7ff475766740 20 client.17520 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.119+0000 7ff475766740 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55606ea2c220 0x55606ea2e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.119+0000 7ff475766740 4 client.17520 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:11:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.119+0000 7ff475766740 1 -- 172.21.15.5:0/2523605509 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55606e912d00 con 0x55606ea2c220 2024-09-19T06:11:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.119+0000 7ff471129640 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55606ea2c220 0x55606ea2e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.120+0000 7ff454ff9640 10 client.17520 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.120+0000 7ff471129640 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55606ea2c220 0x55606ea2e640 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33e76bfa6d271e73 server_cookie=512b5dc4a7860228 in_seq=0 out_seq=0 2024-09-19T06:11:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.120+0000 7ff454ff9640 1 -- 172.21.15.5:0/2523605509 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55606e912d00 con 0x55606ea2c220 2024-09-19T06:11:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.120+0000 7ff454ff9640 10 client.17520 handle_command_reply: tid=0 2024-09-19T06:11:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff475766740 1 client.17520 shutdown 2024-09-19T06:11:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff475766740 1 -- 172.21.15.5:0/2523605509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9b0d40 msgr2=0x55606e9b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff475766740 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9b0d40 0x55606e9b56d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff464004310 tx=0x7ff464004ab0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff475766740 1 -- 172.21.15.5:0/2523605509 shutdown_connections 2024-09-19T06:11:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff475766740 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55606ea2c220 0x55606ea2e640 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff454ff9640 0 client.17520 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff475766740 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55606e9b0d40 0x55606e9b56d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff475766740 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55606e9a6480 0x55606e9b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff475766740 1 --2- 172.21.15.5:0/2523605509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55606e9a5b60 0x55606e9b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff475766740 1 -- 172.21.15.5:0/2523605509 >> 172.21.15.5:0/2523605509 conn(0x55606e9a20f0 msgr2=0x55606e9ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff475766740 1 -- 172.21.15.5:0/2523605509 shutdown_connections 2024-09-19T06:11:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff475766740 1 -- 172.21.15.5:0/2523605509 wait complete. 2024-09-19T06:11:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.121+0000 7ff475766740 20 client.17520 trim_cache size 0 max 16384 2024-09-19T06:11:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.122+0000 7ff47338d640 1 -- 172.21.15.5:0/745327167 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4580a3430 msgr2=0x7ff4580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.122+0000 7ff47338d640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4580a3430 0x7ff4580a58f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff4640099a0 tx=0x7ff464009500 comp rx=0 tx=0).stop 2024-09-19T06:11:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.122+0000 7ff47338d640 1 -- 172.21.15.5:0/745327167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff46c102d30 msgr2=0x7ff46c198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.122+0000 7ff47338d640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff46c102d30 0x7ff46c198560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff46800e9f0 tx=0x7ff46800edf0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.122+0000 7ff47338d640 1 -- 172.21.15.5:0/745327167 shutdown_connections 2024-09-19T06:11:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.122+0000 7ff47338d640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff46c1082a0 0x7ff46c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.122+0000 7ff47338d640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1036e0 0x7ff46c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.122+0000 7ff47338d640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4580a3430 0x7ff4580a58f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.122+0000 7ff47338d640 1 --2- 172.21.15.5:0/745327167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff46c102d30 0x7ff46c198560 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.122+0000 7ff47338d640 1 -- 172.21.15.5:0/745327167 >> 172.21.15.5:0/745327167 conn(0x7ff46c0fc7d0 msgr2=0x7ff46c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.122+0000 7ff47338d640 1 -- 172.21.15.5:0/745327167 shutdown_connections 2024-09-19T06:11:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:22.122+0000 7ff47338d640 1 -- 172.21.15.5:0/745327167 wait complete. 2024-09-19T06:11:22.708 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:22 smithi005 ceph-mon[30955]: pgmap v248: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:11:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:22 smithi157 ceph-mon[35346]: pgmap v248: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:11:22.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 206886 2.70 MB/sec execute 288 sec latency 74.036 ms 2024-09-19T06:11:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:22 smithi049 ceph-mon[34640]: pgmap v248: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-19T06:11:23.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:11:23.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:11:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:11:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:11:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:11:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:11:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:11:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:11:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:11:23.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:23 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 207352 2.71 MB/sec execute 289 sec latency 57.851 ms 2024-09-19T06:11:23.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:11:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:11:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:11:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:11:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:11:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:11:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:11:23.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:11:23.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:11:23.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:23 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:11:23.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:11:23.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:11:23.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:11:23.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:11:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:11:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:11:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:11:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:11:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:23.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:23 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:11:24.797 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:24 smithi049 ceph-mon[34640]: pgmap v249: 97 pgs: 97 active+clean; 120 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.0 MiB/s wr, 188 op/s 2024-09-19T06:11:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:24 smithi157 ceph-mon[35346]: pgmap v249: 97 pgs: 97 active+clean; 120 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.0 MiB/s wr, 188 op/s 2024-09-19T06:11:24.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 207912 2.70 MB/sec execute 290 sec latency 37.744 ms 2024-09-19T06:11:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:24 smithi005 ceph-mon[30955]: pgmap v249: 97 pgs: 97 active+clean; 120 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.0 MiB/s wr, 188 op/s 2024-09-19T06:11:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.480+0000 7fb8700d0640 1 -- 172.21.15.5:0/361500718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb868107ad0 msgr2=0x7fb868107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.480+0000 7fb8700d0640 1 --2- 172.21.15.5:0/361500718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb868107ad0 0x7fb868107eb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb864009930 tx=0x7fb86402a420 comp rx=0 tx=0).stop 2024-09-19T06:11:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.480+0000 7fb8700d0640 1 -- 172.21.15.5:0/361500718 shutdown_connections 2024-09-19T06:11:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.480+0000 7fb8700d0640 1 --2- 172.21.15.5:0/361500718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb868102460 0x7fb86810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.480+0000 7fb8700d0640 1 --2- 172.21.15.5:0/361500718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb868101ac0 0x7fb868101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.480+0000 7fb8700d0640 1 --2- 172.21.15.5:0/361500718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb868107ad0 0x7fb868107eb0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.480+0000 7fb8700d0640 1 -- 172.21.15.5:0/361500718 >> 172.21.15.5:0/361500718 conn(0x7fb8680fd7e0 msgr2=0x7fb8680ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.480+0000 7fb8700d0640 1 -- 172.21.15.5:0/361500718 shutdown_connections 2024-09-19T06:11:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.480+0000 7fb8700d0640 1 -- 172.21.15.5:0/361500718 wait complete. 2024-09-19T06:11:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb8700d0640 1 Processor -- start 2024-09-19T06:11:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb8700d0640 1 -- start start 2024-09-19T06:11:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb8700d0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb868101ac0 0x7fb8681983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb8700d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb868102460 0x7fb868198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb8700d0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb868107ad0 0x7fb86819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb8700d0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb868111dd0 con 0x7fb868102460 2024-09-19T06:11:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb8700d0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb868111c50 con 0x7fb868101ac0 2024-09-19T06:11:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb8700d0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb868111f50 con 0x7fb868107ad0 2024-09-19T06:11:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb86d66b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb868102460 0x7fb868198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb86d66b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb868102460 0x7fb868198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39246/0 (socket says 172.21.15.5:39246) 2024-09-19T06:11:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb86d66b640 1 -- 172.21.15.5:0/1450102648 learned_addr learned my addr 172.21.15.5:0/1450102648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb86de6c640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb868101ac0 0x7fb8681983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb86e66d640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb868107ad0 0x7fb86819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb86d66b640 1 -- 172.21.15.5:0/1450102648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb868107ad0 msgr2=0x7fb86819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb86d66b640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb868107ad0 0x7fb86819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb86d66b640 1 -- 172.21.15.5:0/1450102648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb868101ac0 msgr2=0x7fb8681983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb86d66b640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb868101ac0 0x7fb8681983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.481+0000 7fb86d66b640 1 -- 172.21.15.5:0/1450102648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86819d3b0 con 0x7fb868102460 2024-09-19T06:11:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.482+0000 7fb86d66b640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb868102460 0x7fb868198900 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb85800d950 tx=0x7fb85800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.482+0000 7fb85effd640 1 -- 172.21.15.5:0/1450102648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858017070 con 0x7fb868102460 2024-09-19T06:11:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.482+0000 7fb85effd640 1 -- 172.21.15.5:0/1450102648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb858009c30 con 0x7fb868102460 2024-09-19T06:11:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.482+0000 7fb8700d0640 1 -- 172.21.15.5:0/1450102648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86819d6a0 con 0x7fb868102460 2024-09-19T06:11:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.482+0000 7fb85effd640 1 -- 172.21.15.5:0/1450102648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85800f620 con 0x7fb868102460 2024-09-19T06:11:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.482+0000 7fb8700d0640 1 -- 172.21.15.5:0/1450102648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8681a4f30 con 0x7fb868102460 2024-09-19T06:11:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.484+0000 7fb872e6a740 1 Processor -- start 2024-09-19T06:11:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.484+0000 7fb85effd640 1 -- 172.21.15.5:0/1450102648 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb8580109a0 con 0x7fb868102460 2024-09-19T06:11:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb872e6a740 1 -- start start 2024-09-19T06:11:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb872e6a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa07af0 0x561a7fa07ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb872e6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa08410 0x561a7fa12770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb872e6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 0x561a7fa150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb872e6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a7f96c2f0 con 0x561a7fa12cb0 2024-09-19T06:11:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb872e6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561a7f914270 con 0x561a7fa08410 2024-09-19T06:11:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb872e6a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561a7f9682b0 con 0x561a7fa07af0 2024-09-19T06:11:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb86d66b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 0x561a7fa150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb86d66b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 0x561a7fa150a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39254/0 (socket says 172.21.15.5:39254) 2024-09-19T06:11:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb86d66b640 1 -- 172.21.15.5:0/3072661354 learned_addr learned my addr 172.21.15.5:0/3072661354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb86e66d640 1 --2- 172.21.15.5:0/3072661354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa07af0 0x561a7fa07ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb86de6c640 1 --2- 172.21.15.5:0/3072661354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa08410 0x561a7fa12770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb85effd640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb8540a33b0 0x7fb8540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb86d66b640 1 -- 172.21.15.5:0/3072661354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa07af0 msgr2=0x561a7fa07ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb86d66b640 1 --2- 172.21.15.5:0/3072661354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa07af0 0x561a7fa07ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb86d66b640 1 -- 172.21.15.5:0/3072661354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa08410 msgr2=0x561a7fa12770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb86d66b640 1 --2- 172.21.15.5:0/3072661354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa08410 0x561a7fa12770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb86d66b640 1 -- 172.21.15.5:0/3072661354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a7f913340 con 0x561a7fa12cb0 2024-09-19T06:11:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb86e66d640 1 --2- 172.21.15.5:0/3072661354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa07af0 0x561a7fa07ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.485+0000 7fb86e66d640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb8540a33b0 0x7fb8540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.486+0000 7fb85effd640 1 -- 172.21.15.5:0/1450102648 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7fb8580cf8c0 con 0x7fb868102460 2024-09-19T06:11:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.486+0000 7fb86d66b640 1 --2- 172.21.15.5:0/3072661354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 0x561a7fa150a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb8580ccf60 tx=0x7fb85805bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f66670f7a7c8b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.486+0000 7fb853fff640 1 -- 172.21.15.5:0/3072661354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85809ba00 con 0x561a7fa12cb0 2024-09-19T06:11:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.486+0000 7fb86e66d640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb8540a33b0 0x7fb8540a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb860005fd0 tx=0x7fb860004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.486+0000 7fb853fff640 1 -- 172.21.15.5:0/3072661354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb858009de0 con 0x561a7fa12cb0 2024-09-19T06:11:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.486+0000 7fb853fff640 1 -- 172.21.15.5:0/3072661354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85809b590 con 0x561a7fa12cb0 2024-09-19T06:11:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.487+0000 7fb872e6a740 1 -- 172.21.15.5:0/3072661354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 msgr2=0x561a7fa150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.487+0000 7fb872e6a740 1 --2- 172.21.15.5:0/3072661354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 0x561a7fa150a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb8580ccf60 tx=0x7fb85805bac0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.487+0000 7fb872e6a740 1 -- 172.21.15.5:0/3072661354 shutdown_connections 2024-09-19T06:11:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.487+0000 7fb872e6a740 1 --2- 172.21.15.5:0/3072661354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 0x561a7fa150a0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.487+0000 7fb872e6a740 1 --2- 172.21.15.5:0/3072661354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa08410 0x561a7fa12770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.487+0000 7fb872e6a740 1 --2- 172.21.15.5:0/3072661354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa07af0 0x561a7fa07ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.487+0000 7fb872e6a740 1 -- 172.21.15.5:0/3072661354 >> 172.21.15.5:0/3072661354 conn(0x561a7fa04060 msgr2=0x561a7fa04440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.487+0000 7fb872e6a740 1 -- 172.21.15.5:0/3072661354 shutdown_connections 2024-09-19T06:11:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.487+0000 7fb872e6a740 1 -- 172.21.15.5:0/3072661354 wait complete. 2024-09-19T06:11:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.487+0000 7fb872e6a740 1 -- start start 2024-09-19T06:11:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.487+0000 7fb872e6a740 1 Processor -- start 2024-09-19T06:11:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb872e6a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa07af0 0x561a7fa0fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb872e6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa08410 0x561a7fa10050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb872e6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 0x561a7fa10590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb872e6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a7fa15820 con 0x561a7fa12cb0 2024-09-19T06:11:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb872e6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561a7f8f5290 con 0x561a7fa08410 2024-09-19T06:11:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb872e6a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561a7fa159a0 con 0x561a7fa07af0 2024-09-19T06:11:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb872e6a740 10 client.0 _ll_register_callbacks cb 0x561a7f926140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb86e66d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 0x561a7fa10590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb86e66d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 0x561a7fa10590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39264/0 (socket says 172.21.15.5:39264) 2024-09-19T06:11:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb86e66d640 1 -- 172.21.15.5:0/2257686537 learned_addr learned my addr 172.21.15.5:0/2257686537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb86d66b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa08410 0x561a7fa10050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb86de6c640 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa07af0 0x561a7fa0fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb86e66d640 1 -- 172.21.15.5:0/2257686537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa07af0 msgr2=0x561a7fa0fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb86e66d640 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa07af0 0x561a7fa0fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb86e66d640 1 -- 172.21.15.5:0/2257686537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa08410 msgr2=0x561a7fa10050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb86e66d640 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa08410 0x561a7fa10050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb86e66d640 1 -- 172.21.15.5:0/2257686537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7f67f210 con 0x561a7fa12cb0 2024-09-19T06:11:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb86d66b640 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa08410 0x561a7fa10050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb86e66d640 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 0x561a7fa10590 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb860009db0 tx=0x7fb860004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7606936f7b8ebe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb8517fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb8517fa640 1 -- 172.21.15.5:0/2257686537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860011ac0 con 0x561a7fa12cb0 2024-09-19T06:11:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb8517fa640 1 -- 172.21.15.5:0/2257686537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7fb860011c60 con 0x561a7fa12cb0 2024-09-19T06:11:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.488+0000 7fb872e6a740 1 -- 172.21.15.5:0/2257686537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb858009de0 con 0x561a7fa12cb0 2024-09-19T06:11:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.489+0000 7fb8517fa640 1 -- 172.21.15.5:0/2257686537 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb860028420 con 0x561a7fa12cb0 2024-09-19T06:11:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.489+0000 7fb872e6a740 10 client.17532 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.489+0000 7fb872e6a740 1 -- 172.21.15.5:0/2257686537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561a7f74b620 con 0x561a7fa12cb0 2024-09-19T06:11:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.489+0000 7fb8517fa640 1 -- 172.21.15.5:0/2257686537 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb858009de0 con 0x561a7fa12cb0 2024-09-19T06:11:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.489+0000 7fb8517fa640 1 -- 172.21.15.5:0/2257686537 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb86001f560 con 0x561a7fa12cb0 2024-09-19T06:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.489+0000 7fb872e6a740 10 client.17532 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.489+0000 7fb872e6a740 10 client.17532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.489+0000 7fb872e6a740 20 client.17532 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.489+0000 7fb872e6a740 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a7fa22b70 0x561a7fa24fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.489+0000 7fb872e6a740 4 client.17532 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.489+0000 7fb872e6a740 1 -- 172.21.15.5:0/2257686537 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a7f979a20 con 0x561a7fa22b70 2024-09-19T06:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.489+0000 7fb86de6c640 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a7fa22b70 0x561a7fa24fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.490+0000 7fb86de6c640 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a7fa22b70 0x561a7fa24fa0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=483e5203c5fcd036 server_cookie=66fe5b306e31df69 in_seq=0 out_seq=0 2024-09-19T06:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.490+0000 7fb8517fa640 10 client.17532 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.490+0000 7fb8517fa640 1 -- 172.21.15.5:0/2257686537 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a7f979a20 con 0x561a7fa22b70 2024-09-19T06:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.490+0000 7fb8517fa640 10 client.17532 handle_command_reply: tid=0 2024-09-19T06:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.490+0000 7fb872e6a740 1 client.17532 shutdown 2024-09-19T06:11:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.490+0000 7fb872e6a740 1 -- 172.21.15.5:0/2257686537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 msgr2=0x561a7fa10590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.490+0000 7fb872e6a740 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 0x561a7fa10590 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb860009db0 tx=0x7fb860004ab0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.491+0000 7fb872e6a740 1 -- 172.21.15.5:0/2257686537 shutdown_connections 2024-09-19T06:11:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.491+0000 7fb872e6a740 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a7fa22b70 0x561a7fa24fa0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.491+0000 7fb872e6a740 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa12cb0 0x561a7fa10590 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.491+0000 7fb872e6a740 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa08410 0x561a7fa10050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.491+0000 7fb8517fa640 0 client.17532 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.491+0000 7fb872e6a740 1 --2- 172.21.15.5:0/2257686537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa07af0 0x561a7fa0fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.491+0000 7fb872e6a740 1 -- 172.21.15.5:0/2257686537 >> 172.21.15.5:0/2257686537 conn(0x561a7fa04060 msgr2=0x561a7fa09dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.491+0000 7fb872e6a740 1 -- 172.21.15.5:0/2257686537 shutdown_connections 2024-09-19T06:11:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.491+0000 7fb872e6a740 1 -- 172.21.15.5:0/2257686537 wait complete. 2024-09-19T06:11:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.491+0000 7fb872e6a740 20 client.17532 trim_cache size 0 max 16384 2024-09-19T06:11:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb872e6a740 1 Processor -- start 2024-09-19T06:11:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb872e6a740 1 -- start start 2024-09-19T06:11:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb872e6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa07af0 0x561a7fa0a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb872e6a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa08410 0x561a7fa103c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb872e6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa12cb0 0x561a7fa1b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb872e6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8681ad8d0 con 0x561a7fa07af0 2024-09-19T06:11:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb872e6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb8681af410 con 0x561a7fa12cb0 2024-09-19T06:11:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb872e6a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb8681adfa0 con 0x561a7fa08410 2024-09-19T06:11:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb86de6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa07af0 0x561a7fa0a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb86de6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa07af0 0x561a7fa0a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39272/0 (socket says 172.21.15.5:39272) 2024-09-19T06:11:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb86de6c640 1 -- 172.21.15.5:0/1394190596 learned_addr learned my addr 172.21.15.5:0/1394190596 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb86d66b640 1 --2- 172.21.15.5:0/1394190596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa08410 0x561a7fa103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.505+0000 7fb86e66d640 1 --2- 172.21.15.5:0/1394190596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa12cb0 0x561a7fa1b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.506+0000 7fb86de6c640 1 -- 172.21.15.5:0/1394190596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa08410 msgr2=0x561a7fa103c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.506+0000 7fb86de6c640 1 --2- 172.21.15.5:0/1394190596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa08410 0x561a7fa103c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.506+0000 7fb86de6c640 1 -- 172.21.15.5:0/1394190596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa12cb0 msgr2=0x561a7fa1b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.506+0000 7fb86de6c640 1 --2- 172.21.15.5:0/1394190596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa12cb0 0x561a7fa1b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.506+0000 7fb86de6c640 1 -- 172.21.15.5:0/1394190596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a7f9fe2f0 con 0x561a7fa07af0 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.506+0000 7fb86de6c640 1 --2- 172.21.15.5:0/1394190596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa07af0 0x561a7fa0a560 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb864000c00 tx=0x7fb864005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187b43a3273e9287 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.506+0000 7fb8537fe640 1 -- 172.21.15.5:0/1394190596 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864035040 con 0x561a7fa07af0 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.506+0000 7fb8537fe640 1 -- 172.21.15.5:0/1394190596 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb86404b070 con 0x561a7fa07af0 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.506+0000 7fb8537fe640 1 -- 172.21.15.5:0/1394190596 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864003850 con 0x561a7fa07af0 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.506+0000 7fb872e6a740 1 -- 172.21.15.5:0/1394190596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa07af0 msgr2=0x561a7fa0a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.506+0000 7fb872e6a740 1 --2- 172.21.15.5:0/1394190596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa07af0 0x561a7fa0a560 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb864000c00 tx=0x7fb864005a20 comp rx=0 tx=0).stop 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 -- 172.21.15.5:0/1394190596 shutdown_connections 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 --2- 172.21.15.5:0/1394190596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa12cb0 0x561a7fa1b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 --2- 172.21.15.5:0/1394190596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa08410 0x561a7fa103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 --2- 172.21.15.5:0/1394190596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa07af0 0x561a7fa0a560 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 -- 172.21.15.5:0/1394190596 >> 172.21.15.5:0/1394190596 conn(0x561a7fa04060 msgr2=0x561a7fa11b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 -- 172.21.15.5:0/1394190596 shutdown_connections 2024-09-19T06:11:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 -- 172.21.15.5:0/1394190596 wait complete. 2024-09-19T06:11:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 -- start start 2024-09-19T06:11:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 Processor -- start 2024-09-19T06:11:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa07af0 0x561a7fa0bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa08410 0x561a7fa0c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa12cb0 0x561a7fa0c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8681aed40 con 0x561a7fa08410 2024-09-19T06:11:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb8681ae670 con 0x561a7fa07af0 2024-09-19T06:11:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb8681ab8d0 con 0x561a7fa12cb0 2024-09-19T06:11:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb872e6a740 10 client.0 _ll_register_callbacks cb 0x7fb860009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb86d66b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa08410 0x561a7fa0c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb86d66b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa08410 0x561a7fa0c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39274/0 (socket says 172.21.15.5:39274) 2024-09-19T06:11:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.507+0000 7fb86d66b640 1 -- 172.21.15.5:0/4157125018 learned_addr learned my addr 172.21.15.5:0/4157125018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb86de6c640 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa07af0 0x561a7fa0bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb86e66d640 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa12cb0 0x561a7fa0c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb86d66b640 1 -- 172.21.15.5:0/4157125018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa12cb0 msgr2=0x561a7fa0c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb86d66b640 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa12cb0 0x561a7fa0c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb86d66b640 1 -- 172.21.15.5:0/4157125018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa07af0 msgr2=0x561a7fa0bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb86d66b640 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa07af0 0x561a7fa0bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb86d66b640 1 -- 172.21.15.5:0/4157125018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7fa0cea0 con 0x561a7fa08410 2024-09-19T06:11:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb86d66b640 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa08410 0x561a7fa0c2a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb858000c00 tx=0x7fb85800d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e517246b1057050e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb8517fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb8517fa640 1 -- 172.21.15.5:0/4157125018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858020040 con 0x561a7fa08410 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb8517fa640 1 -- 172.21.15.5:0/4157125018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7fb8580982a0 con 0x561a7fa08410 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb872e6a740 1 -- 172.21.15.5:0/4157125018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb86404b070 con 0x561a7fa08410 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb8517fa640 1 -- 172.21.15.5:0/4157125018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb85809b770 con 0x561a7fa08410 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb872e6a740 10 client.17538 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb872e6a740 1 -- 172.21.15.5:0/4157125018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561a7fa91d60 con 0x561a7fa08410 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.508+0000 7fb8517fa640 1 -- 172.21.15.5:0/4157125018 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb86404b070 con 0x561a7fa08410 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.509+0000 7fb8517fa640 1 -- 172.21.15.5:0/4157125018 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb85809be90 con 0x561a7fa08410 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.509+0000 7fb872e6a740 10 client.17538 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.509+0000 7fb872e6a740 10 client.17538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.509+0000 7fb872e6a740 20 client.17538 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.509+0000 7fb872e6a740 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a7fa92110 0x561a7fa94500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.509+0000 7fb872e6a740 4 client.17538 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:11:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.509+0000 7fb872e6a740 1 -- 172.21.15.5:0/4157125018 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a7f973d70 con 0x561a7fa92110 2024-09-19T06:11:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.509+0000 7fb86de6c640 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a7fa92110 0x561a7fa94500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.509+0000 7fb8517fa640 10 client.17538 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.509+0000 7fb86de6c640 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a7fa92110 0x561a7fa94500 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a31ac6865956a93a server_cookie=52292159cbf60ce4 in_seq=0 out_seq=0 2024-09-19T06:11:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb8517fa640 1 -- 172.21.15.5:0/4157125018 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a7f973d70 con 0x561a7fa92110 2024-09-19T06:11:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb8517fa640 10 client.17538 handle_command_reply: tid=0 2024-09-19T06:11:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb872e6a740 1 client.17538 shutdown 2024-09-19T06:11:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb872e6a740 1 -- 172.21.15.5:0/4157125018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa08410 msgr2=0x561a7fa0c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb872e6a740 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa08410 0x561a7fa0c2a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb858000c00 tx=0x7fb85800d8d0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb872e6a740 1 -- 172.21.15.5:0/4157125018 shutdown_connections 2024-09-19T06:11:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb872e6a740 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a7fa92110 0x561a7fa94500 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb8517fa640 0 client.17538 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb872e6a740 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a7fa12cb0 0x561a7fa0c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb872e6a740 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a7fa08410 0x561a7fa0c2a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb872e6a740 1 --2- 172.21.15.5:0/4157125018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a7fa07af0 0x561a7fa0bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb872e6a740 1 -- 172.21.15.5:0/4157125018 >> 172.21.15.5:0/4157125018 conn(0x561a7fa04060 msgr2=0x561a7fa031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb872e6a740 1 -- 172.21.15.5:0/4157125018 shutdown_connections 2024-09-19T06:11:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb872e6a740 1 -- 172.21.15.5:0/4157125018 wait complete. 2024-09-19T06:11:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.530+0000 7fb872e6a740 20 client.17538 trim_cache size 0 max 16384 2024-09-19T06:11:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:11:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:11:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:11:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:11:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.531+0000 7fb8700d0640 1 -- 172.21.15.5:0/1450102648 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb8540a33b0 msgr2=0x7fb8540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.531+0000 7fb8700d0640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb8540a33b0 0x7fb8540a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb860005fd0 tx=0x7fb860004a40 comp rx=0 tx=0).stop 2024-09-19T06:11:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.531+0000 7fb8700d0640 1 -- 172.21.15.5:0/1450102648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb868102460 msgr2=0x7fb868198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.531+0000 7fb8700d0640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb868102460 0x7fb868198900 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb85800d950 tx=0x7fb85800de10 comp rx=0 tx=0).stop 2024-09-19T06:11:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.531+0000 7fb8700d0640 1 -- 172.21.15.5:0/1450102648 shutdown_connections 2024-09-19T06:11:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.531+0000 7fb8700d0640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb8540a33b0 0x7fb8540a5870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.531+0000 7fb8700d0640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb868107ad0 0x7fb86819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.531+0000 7fb8700d0640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb868102460 0x7fb868198900 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.531+0000 7fb8700d0640 1 --2- 172.21.15.5:0/1450102648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb868101ac0 0x7fb8681983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.531+0000 7fb8700d0640 1 -- 172.21.15.5:0/1450102648 >> 172.21.15.5:0/1450102648 conn(0x7fb8680fd7e0 msgr2=0x7fb8680fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.531+0000 7fb8700d0640 1 -- 172.21.15.5:0/1450102648 shutdown_connections 2024-09-19T06:11:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:25.531+0000 7fb8700d0640 1 -- 172.21.15.5:0/1450102648 wait complete. 2024-09-19T06:11:25.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 208328 2.70 MB/sec execute 291 sec latency 81.737 ms 2024-09-19T06:11:26.172 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:11:26.458 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:11:26.720 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:26 smithi005 ceph-mon[30955]: pgmap v250: 97 pgs: 97 active+clean; 120 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-19T06:11:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:26 smithi157 ceph-mon[35346]: pgmap v250: 97 pgs: 97 active+clean; 120 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-19T06:11:26.833 INFO:tasks.workunit.client.0.smithi005.stdout: 1 209033 2.71 MB/sec execute 292 sec latency 29.585 ms 2024-09-19T06:11:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:26 smithi049 ceph-mon[34640]: pgmap v250: 97 pgs: 97 active+clean; 120 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-19T06:11:26.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.992+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/1707029171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc28102d30 msgr2=0x7fcc28103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:26.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.992+0000 7fcc2e6ae640 1 --2- 172.21.15.5:0/1707029171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc28102d30 0x7fcc28103110 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009930 tx=0x7fcc1802a420 comp rx=0 tx=0).stop 2024-09-19T06:11:26.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.992+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/1707029171 shutdown_connections 2024-09-19T06:11:26.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.992+0000 7fcc2e6ae640 1 --2- 172.21.15.5:0/1707029171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc281082a0 0x7fcc2810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:26.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.992+0000 7fcc2e6ae640 1 --2- 172.21.15.5:0/1707029171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc281036e0 0x7fcc28107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:26.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.992+0000 7fcc2e6ae640 1 --2- 172.21.15.5:0/1707029171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc28102d30 0x7fcc28103110 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:26.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.992+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/1707029171 >> 172.21.15.5:0/1707029171 conn(0x7fcc280fc7d0 msgr2=0x7fcc280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:26.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.992+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/1707029171 shutdown_connections 2024-09-19T06:11:26.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.993+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/1707029171 wait complete. 2024-09-19T06:11:26.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.993+0000 7fcc2e6ae640 1 Processor -- start 2024-09-19T06:11:26.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.993+0000 7fcc2e6ae640 1 -- start start 2024-09-19T06:11:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2e6ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc28102d30 0x7fcc28198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2e6ae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc281036e0 0x7fcc28198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2e6ae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc281082a0 0x7fcc2819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2e6ae640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc28111f50 con 0x7fcc281082a0 2024-09-19T06:11:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2e6ae640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcc28111dd0 con 0x7fcc281036e0 2024-09-19T06:11:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2e6ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281120d0 con 0x7fcc28102d30 2024-09-19T06:11:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2cc4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc281082a0 0x7fcc2819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc281036e0 0x7fcc28198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2cc4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc281082a0 0x7fcc2819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39288/0 (socket says 172.21.15.5:39288) 2024-09-19T06:11:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc281036e0 0x7fcc28198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33416/0 (socket says 172.21.15.5:33416) 2024-09-19T06:11:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2cc4b640 1 -- 172.21.15.5:0/4234298445 learned_addr learned my addr 172.21.15.5:0/4234298445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc27fff640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc28102d30 0x7fcc28198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2cc4b640 1 -- 172.21.15.5:0/4234298445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc28102d30 msgr2=0x7fcc28198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2cc4b640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc28102d30 0x7fcc28198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2cc4b640 1 -- 172.21.15.5:0/4234298445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc281036e0 msgr2=0x7fcc28198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2cc4b640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc281036e0 0x7fcc28198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc2cc4b640 1 -- 172.21.15.5:0/4234298445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2819d4c0 con 0x7fcc281082a0 2024-09-19T06:11:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.994+0000 7fcc277fe640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc281036e0 0x7fcc28198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.995+0000 7fcc2cc4b640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc281082a0 0x7fcc2819cde0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000eb10 tx=0x7fcc2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.995+0000 7fcc257fa640 1 -- 172.21.15.5:0/4234298445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20004b20 con 0x7fcc281082a0 2024-09-19T06:11:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.995+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/4234298445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2819d7b0 con 0x7fcc281082a0 2024-09-19T06:11:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.995+0000 7fcc257fa640 1 -- 172.21.15.5:0/4234298445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc20004cc0 con 0x7fcc281082a0 2024-09-19T06:11:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.995+0000 7fcc257fa640 1 -- 172.21.15.5:0/4234298445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20010690 con 0x7fcc281082a0 2024-09-19T06:11:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.995+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/4234298445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc281a5090 con 0x7fcc281082a0 2024-09-19T06:11:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc257fa640 1 -- 172.21.15.5:0/4234298445 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcc20005020 con 0x7fcc281082a0 2024-09-19T06:11:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc30a87740 1 Processor -- start 2024-09-19T06:11:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc30a87740 1 -- start start 2024-09-19T06:11:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc30a87740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334e34a0 0x556b334e3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc30a87740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334ee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc30a87740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334ee680 0x556b334f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc30a87740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556b334492e0 con 0x556b334e3dc0 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc30a87740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556b333f0ce0 con 0x556b334ee680 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc30a87740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556b334452a0 con 0x556b334e34a0 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334ee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334ee140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39300/0 (socket says 172.21.15.5:39300) 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc27fff640 1 -- 172.21.15.5:0/3860090148 learned_addr learned my addr 172.21.15.5:0/3860090148 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc277fe640 1 --2- 172.21.15.5:0/3860090148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334e34a0 0x556b334e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.998+0000 7fcc277fe640 1 --2- 172.21.15.5:0/3860090148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334ee680 0x556b334f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.999+0000 7fcc27fff640 1 -- 172.21.15.5:0/3860090148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334e34a0 msgr2=0x556b334e3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.999+0000 7fcc27fff640 1 --2- 172.21.15.5:0/3860090148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334e34a0 0x556b334e3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.999+0000 7fcc27fff640 1 -- 172.21.15.5:0/3860090148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334ee680 msgr2=0x556b334f0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.999+0000 7fcc27fff640 1 --2- 172.21.15.5:0/3860090148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334ee680 0x556b334f0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.999+0000 7fcc27fff640 1 -- 172.21.15.5:0/3860090148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b333d5440 con 0x556b334e3dc0 2024-09-19T06:11:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.999+0000 7fcc257fa640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcc100a33b0 0x7fcc100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.999+0000 7fcc27fff640 1 --2- 172.21.15.5:0/3860090148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334ee140 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fcc1802a940 tx=0x7fcc18005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3efb527ad0059ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.999+0000 7fcc277fe640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcc100a33b0 0x7fcc100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.999+0000 7fcc257fa640 1 -- 172.21.15.5:0/4234298445 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7fcc200d0250 con 0x7fcc281082a0 2024-09-19T06:11:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.999+0000 7fcc0e7fc640 1 -- 172.21.15.5:0/3860090148 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18035040 con 0x556b334e3dc0 2024-09-19T06:11:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.999+0000 7fcc0e7fc640 1 -- 172.21.15.5:0/3860090148 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc18031db0 con 0x556b334e3dc0 2024-09-19T06:11:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:26.999+0000 7fcc0e7fc640 1 -- 172.21.15.5:0/3860090148 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18005220 con 0x556b334e3dc0 2024-09-19T06:11:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.000+0000 7fcc30a87740 1 -- 172.21.15.5:0/3860090148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 msgr2=0x556b334ee140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.000+0000 7fcc277fe640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcc100a33b0 0x7fcc100a5870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x556b334e0db0 tx=0x7fcc1c005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.000+0000 7fcc30a87740 1 --2- 172.21.15.5:0/3860090148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334ee140 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fcc1802a940 tx=0x7fcc18005be0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.000+0000 7fcc30a87740 1 -- 172.21.15.5:0/3860090148 shutdown_connections 2024-09-19T06:11:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.000+0000 7fcc30a87740 1 --2- 172.21.15.5:0/3860090148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334ee680 0x556b334f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.000+0000 7fcc30a87740 1 --2- 172.21.15.5:0/3860090148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334ee140 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.000+0000 7fcc30a87740 1 --2- 172.21.15.5:0/3860090148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334e34a0 0x556b334e3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.000+0000 7fcc30a87740 1 -- 172.21.15.5:0/3860090148 >> 172.21.15.5:0/3860090148 conn(0x556b334dfa30 msgr2=0x556b334dfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.000+0000 7fcc30a87740 1 -- 172.21.15.5:0/3860090148 shutdown_connections 2024-09-19T06:11:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.000+0000 7fcc30a87740 1 -- 172.21.15.5:0/3860090148 wait complete. 2024-09-19T06:11:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc30a87740 1 -- start start 2024-09-19T06:11:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc30a87740 1 Processor -- start 2024-09-19T06:11:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc30a87740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 0x556b334edf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc30a87740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334eb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc30a87740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334ee680 0x556b334eb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc30a87740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556b334f1250 con 0x556b334e3dc0 2024-09-19T06:11:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc30a87740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556b33404050 con 0x556b334e34a0 2024-09-19T06:11:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc30a87740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556b334f13d0 con 0x556b334ee680 2024-09-19T06:11:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc30a87740 10 client.0 _ll_register_callbacks cb 0x556b33444160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334eb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 0x556b334edf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334eb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39306/0 (socket says 172.21.15.5:39306) 2024-09-19T06:11:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 0x556b334edf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33446/0 (socket says 172.21.15.5:33446) 2024-09-19T06:11:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc277fe640 1 -- 172.21.15.5:0/1486437157 learned_addr learned my addr 172.21.15.5:0/1486437157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.001+0000 7fcc2cc4b640 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334ee680 0x556b334eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.002+0000 7fcc27fff640 1 -- 172.21.15.5:0/1486437157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334ee680 msgr2=0x556b334eb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.002+0000 7fcc27fff640 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334ee680 0x556b334eb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.002+0000 7fcc27fff640 1 -- 172.21.15.5:0/1486437157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 msgr2=0x556b334eb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.002+0000 7fcc27fff640 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334eb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.002+0000 7fcc27fff640 1 -- 172.21.15.5:0/1486437157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b331ba3c0 con 0x556b334e34a0 2024-09-19T06:11:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.002+0000 7fcc277fe640 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334eb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.002+0000 7fcc27fff640 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 0x556b334edf00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009900 tx=0x7fcc18031750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c44cf4461f66323 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.002+0000 7fcc077fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.002+0000 7fcc077fe640 1 -- 172.21.15.5:0/1486437157 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18057020 con 0x556b334e34a0 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.002+0000 7fcc30a87740 1 -- 172.21.15.5:0/1486437157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc18031db0 con 0x556b334e34a0 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.002+0000 7fcc077fe640 1 -- 172.21.15.5:0/1486437157 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7fcc18005020 con 0x556b334e34a0 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.003+0000 7fcc077fe640 1 -- 172.21.15.5:0/1486437157 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcc18062050 con 0x556b334e34a0 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.003+0000 7fcc30a87740 10 client.27116 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.003+0000 7fcc30a87740 1 -- 172.21.15.5:0/1486437157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556b334f1250 con 0x556b334e34a0 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.003+0000 7fcc077fe640 1 -- 172.21.15.5:0/1486437157 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc18031db0 con 0x556b334e34a0 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.003+0000 7fcc077fe640 1 -- 172.21.15.5:0/1486437157 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fcc18052030 con 0x556b334e34a0 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.003+0000 7fcc30a87740 10 client.27116 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.003+0000 7fcc30a87740 10 client.27116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.003+0000 7fcc30a87740 20 client.27116 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.003+0000 7fcc30a87740 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556b334fdd20 0x556b33500110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.003+0000 7fcc30a87740 4 client.27116 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:11:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.003+0000 7fcc30a87740 1 -- 172.21.15.5:0/1486437157 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b33456850 con 0x556b334fdd20 2024-09-19T06:11:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.003+0000 7fcc277fe640 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556b334fdd20 0x556b33500110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.004+0000 7fcc077fe640 10 client.27116 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.004+0000 7fcc277fe640 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556b334fdd20 0x556b33500110 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=680c21bcc8c6e8f5 server_cookie=5e1fd5beff95ed5b in_seq=0 out_seq=0 2024-09-19T06:11:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.004+0000 7fcc077fe640 1 -- 172.21.15.5:0/1486437157 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b33456850 con 0x556b334fdd20 2024-09-19T06:11:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.004+0000 7fcc077fe640 10 client.27116 handle_command_reply: tid=0 2024-09-19T06:11:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc30a87740 1 client.27116 shutdown 2024-09-19T06:11:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc30a87740 1 -- 172.21.15.5:0/1486437157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 msgr2=0x556b334edf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc30a87740 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 0x556b334edf00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009900 tx=0x7fcc18031750 comp rx=0 tx=0).stop 2024-09-19T06:11:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc30a87740 1 -- 172.21.15.5:0/1486437157 shutdown_connections 2024-09-19T06:11:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc30a87740 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556b334fdd20 0x556b33500110 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc30a87740 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334ee680 0x556b334eb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc077fe640 0 client.27116 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc30a87740 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334eb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc30a87740 1 --2- 172.21.15.5:0/1486437157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 0x556b334edf00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc30a87740 1 -- 172.21.15.5:0/1486437157 >> 172.21.15.5:0/1486437157 conn(0x556b334dfa30 msgr2=0x556b334e5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc30a87740 1 -- 172.21.15.5:0/1486437157 shutdown_connections 2024-09-19T06:11:27.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc30a87740 1 -- 172.21.15.5:0/1486437157 wait complete. 2024-09-19T06:11:27.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.005+0000 7fcc30a87740 20 client.27116 trim_cache size 0 max 16384 2024-09-19T06:11:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc30a87740 1 Processor -- start 2024-09-19T06:11:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc30a87740 1 -- start start 2024-09-19T06:11:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc30a87740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e34a0 0x556b334e99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc30a87740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334e3dc0 0x556b335058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc30a87740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334ee680 0x556b33505e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc30a87740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281ab670 con 0x556b334e34a0 2024-09-19T06:11:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc30a87740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281ad1b0 con 0x556b334ee680 2024-09-19T06:11:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc30a87740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281abd40 con 0x556b334e3dc0 2024-09-19T06:11:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e34a0 0x556b334e99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e34a0 0x556b334e99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39322/0 (socket says 172.21.15.5:39322) 2024-09-19T06:11:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc27fff640 1 -- 172.21.15.5:0/3063264198 learned_addr learned my addr 172.21.15.5:0/3063264198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc2cc4b640 1 --2- 172.21.15.5:0/3063264198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334ee680 0x556b33505e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc277fe640 1 --2- 172.21.15.5:0/3063264198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334e3dc0 0x556b335058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc27fff640 1 -- 172.21.15.5:0/3063264198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334e3dc0 msgr2=0x556b335058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc27fff640 1 --2- 172.21.15.5:0/3063264198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334e3dc0 0x556b335058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc27fff640 1 -- 172.21.15.5:0/3063264198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334ee680 msgr2=0x556b33505e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc27fff640 1 --2- 172.21.15.5:0/3063264198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334ee680 0x556b33505e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc27fff640 1 -- 172.21.15.5:0/3063264198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b334e9750 con 0x556b334e34a0 2024-09-19T06:11:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.023+0000 7fcc2cc4b640 1 --2- 172.21.15.5:0/3063264198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334ee680 0x556b33505e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.024+0000 7fcc27fff640 1 --2- 172.21.15.5:0/3063264198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e34a0 0x556b334e99a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fcc18000c00 tx=0x7fcc18031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2cabe460e2ff79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.024+0000 7fcc0dffb640 1 -- 172.21.15.5:0/3063264198 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18057070 con 0x556b334e34a0 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.024+0000 7fcc0dffb640 1 -- 172.21.15.5:0/3063264198 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc180317c0 con 0x556b334e34a0 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.024+0000 7fcc0dffb640 1 -- 172.21.15.5:0/3063264198 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1804f040 con 0x556b334e34a0 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.025+0000 7fcc30a87740 1 -- 172.21.15.5:0/3063264198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e34a0 msgr2=0x556b334e99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.025+0000 7fcc30a87740 1 --2- 172.21.15.5:0/3063264198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e34a0 0x556b334e99a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fcc18000c00 tx=0x7fcc18031400 comp rx=0 tx=0).stop 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.025+0000 7fcc30a87740 1 -- 172.21.15.5:0/3063264198 shutdown_connections 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.025+0000 7fcc30a87740 1 --2- 172.21.15.5:0/3063264198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334ee680 0x556b33505e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.025+0000 7fcc30a87740 1 --2- 172.21.15.5:0/3063264198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334e3dc0 0x556b335058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.025+0000 7fcc30a87740 1 --2- 172.21.15.5:0/3063264198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e34a0 0x556b334e99a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.025+0000 7fcc30a87740 1 -- 172.21.15.5:0/3063264198 >> 172.21.15.5:0/3063264198 conn(0x556b334dfa30 msgr2=0x556b334deb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.025+0000 7fcc30a87740 1 -- 172.21.15.5:0/3063264198 shutdown_connections 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.025+0000 7fcc30a87740 1 -- 172.21.15.5:0/3063264198 wait complete. 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.025+0000 7fcc30a87740 1 -- start start 2024-09-19T06:11:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.025+0000 7fcc30a87740 1 Processor -- start 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc30a87740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 0x556b334f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc30a87740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc30a87740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334ee680 0x556b334f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc30a87740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281acae0 con 0x556b334e3dc0 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc30a87740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281ac410 con 0x556b334e34a0 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc30a87740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281a9670 con 0x556b334ee680 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc30a87740 10 client.0 _ll_register_callbacks cb 0x7fcc1c013630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 0x556b334f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334f40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39324/0 (socket says 172.21.15.5:39324) 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc277fe640 1 -- 172.21.15.5:0/4266280168 learned_addr learned my addr 172.21.15.5:0/4266280168 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc2cc4b640 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334ee680 0x556b334f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc277fe640 1 -- 172.21.15.5:0/4266280168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334ee680 msgr2=0x556b334f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc277fe640 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334ee680 0x556b334f45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc277fe640 1 -- 172.21.15.5:0/4266280168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 msgr2=0x556b334f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc277fe640 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 0x556b334f6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc277fe640 1 -- 172.21.15.5:0/4266280168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b334f4e10 con 0x556b334e3dc0 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc27fff640 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 0x556b334f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc2cc4b640 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334ee680 0x556b334f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.026+0000 7fcc277fe640 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334f40a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x556b33500870 tx=0x7fcc1c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e59ba1b502c914b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc077fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc077fe640 1 -- 172.21.15.5:0/4266280168 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c010c40 con 0x556b334e3dc0 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc30a87740 1 -- 172.21.15.5:0/4266280168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc180317c0 con 0x556b334e3dc0 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc077fe640 1 -- 172.21.15.5:0/4266280168 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7fcc1c010de0 con 0x556b334e3dc0 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc077fe640 1 -- 172.21.15.5:0/4266280168 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcc1c036480 con 0x556b334e3dc0 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc30a87740 10 client.17556 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc30a87740 1 -- 172.21.15.5:0/4266280168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556b33568910 con 0x556b334e3dc0 2024-09-19T06:11:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc077fe640 1 -- 172.21.15.5:0/4266280168 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc180317c0 con 0x556b334e3dc0 2024-09-19T06:11:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc077fe640 1 -- 172.21.15.5:0/4266280168 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fcc1c02d6b0 con 0x556b334e3dc0 2024-09-19T06:11:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc30a87740 10 client.17556 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc30a87740 10 client.17556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc30a87740 20 client.17556 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc30a87740 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556b334fdd20 0x556b3356b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc30a87740 4 client.17556 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:11:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.027+0000 7fcc30a87740 1 -- 172.21.15.5:0/4266280168 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b33450d60 con 0x556b334fdd20 2024-09-19T06:11:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc27fff640 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556b334fdd20 0x556b3356b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc077fe640 10 client.17556 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc27fff640 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556b334fdd20 0x556b3356b020 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d8a28373cc0684 server_cookie=a0afcd8368fe22e1 in_seq=0 out_seq=0 2024-09-19T06:11:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc077fe640 1 -- 172.21.15.5:0/4266280168 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556b33450d60 con 0x556b334fdd20 2024-09-19T06:11:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc077fe640 10 client.17556 handle_command_reply: tid=0 2024-09-19T06:11:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc30a87740 1 client.17556 shutdown 2024-09-19T06:11:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc30a87740 1 -- 172.21.15.5:0/4266280168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 msgr2=0x556b334f40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc30a87740 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334f40a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x556b33500870 tx=0x7fcc1c005850 comp rx=0 tx=0).stop 2024-09-19T06:11:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc30a87740 1 -- 172.21.15.5:0/4266280168 shutdown_connections 2024-09-19T06:11:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc30a87740 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556b334fdd20 0x556b3356b020 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc30a87740 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b334ee680 0x556b334f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc30a87740 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b334e3dc0 0x556b334f40a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc30a87740 1 --2- 172.21.15.5:0/4266280168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b334e34a0 0x556b334f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.028+0000 7fcc30a87740 1 -- 172.21.15.5:0/4266280168 >> 172.21.15.5:0/4266280168 conn(0x556b334dfa30 msgr2=0x556b334eca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc077fe640 0 client.17556 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc30a87740 1 -- 172.21.15.5:0/4266280168 shutdown_connections 2024-09-19T06:11:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc30a87740 1 -- 172.21.15.5:0/4266280168 wait complete. 2024-09-19T06:11:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc30a87740 20 client.17556 trim_cache size 0 max 16384 2024-09-19T06:11:27.036 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:11:27.037 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:11:27.037 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:11:27.037 INFO:teuthology.orchestra.run.smithi005.stdout: "8ef354e7-e6be-43ae-b908-bf8d58eaf5f5": { 2024-09-19T06:11:27.037 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:11:27.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8ef354e7-e6be-43ae-b908-bf8d58eaf5f5", 2024-09-19T06:11:27.037 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:11:27.037 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:11:27.037 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:11:27.037 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:11:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/4234298445 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcc100a33b0 msgr2=0x7fcc100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc2e6ae640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcc100a33b0 0x7fcc100a5870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x556b334e0db0 tx=0x7fcc1c005f90 comp rx=0 tx=0).stop 2024-09-19T06:11:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/4234298445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc281082a0 msgr2=0x7fcc2819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc2e6ae640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc281082a0 0x7fcc2819cde0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000eb10 tx=0x7fcc2000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/4234298445 shutdown_connections 2024-09-19T06:11:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc2e6ae640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc281082a0 0x7fcc2819cde0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc2e6ae640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc281036e0 0x7fcc28198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc2e6ae640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcc100a33b0 0x7fcc100a5870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc2e6ae640 1 --2- 172.21.15.5:0/4234298445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc28102d30 0x7fcc28198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.029+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/4234298445 >> 172.21.15.5:0/4234298445 conn(0x7fcc280fc7d0 msgr2=0x7fcc28105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.030+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/4234298445 shutdown_connections 2024-09-19T06:11:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:11:27.030+0000 7fcc2e6ae640 1 -- 172.21.15.5:0/4234298445 wait complete. 2024-09-19T06:11:27.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 209502 2.72 MB/sec execute 293 sec latency 11.289 ms 2024-09-19T06:11:28.008 INFO:tasks.cephfs.filesystem:scrub status for tag:8ef354e7-e6be-43ae-b908-bf8d58eaf5f5 - {'path': '/', 'tag': '8ef354e7-e6be-43ae-b908-bf8d58eaf5f5', 'options': 'recursive,force'} 2024-09-19T06:11:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:28 smithi157 ceph-mon[35346]: pgmap v251: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-19T06:11:28.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 209738 2.71 MB/sec execute 294 sec latency 14.804 ms 2024-09-19T06:11:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:28 smithi005 ceph-mon[30955]: pgmap v251: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-19T06:11:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:28 smithi049 ceph-mon[34640]: pgmap v251: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-19T06:11:29.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:29 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:11:29.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 210152 2.70 MB/sec execute 295 sec latency 18.526 ms 2024-09-19T06:11:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:29 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:11:29.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:29 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:11:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:30 smithi157 ceph-mon[35346]: pgmap v252: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-19T06:11:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:30 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:11:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:30 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:11:30.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 210678 2.71 MB/sec execute 296 sec latency 36.674 ms 2024-09-19T06:11:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:30 smithi005 ceph-mon[30955]: pgmap v252: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-19T06:11:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:30 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:11:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:30 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:11:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:30 smithi049 ceph-mon[34640]: pgmap v252: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-19T06:11:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:30 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:11:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:30 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:11:31.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 211159 2.70 MB/sec execute 297 sec latency 21.838 ms 2024-09-19T06:11:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:32 smithi157 ceph-mon[35346]: pgmap v253: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-19T06:11:32.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 211712 2.70 MB/sec execute 298 sec latency 23.792 ms 2024-09-19T06:11:32.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:32 smithi005 ceph-mon[30955]: pgmap v253: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-19T06:11:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:32 smithi049 ceph-mon[34640]: pgmap v253: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-19T06:11:33.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 212446 2.71 MB/sec execute 299 sec latency 15.527 ms 2024-09-19T06:11:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:34 smithi157 ceph-mon[35346]: pgmap v254: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-09-19T06:11:34.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 212804 2.71 MB/sec execute 300 sec latency 16.121 ms 2024-09-19T06:11:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:34 smithi005 ceph-mon[30955]: pgmap v254: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-09-19T06:11:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:34 smithi049 ceph-mon[34640]: pgmap v254: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-09-19T06:11:35.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:35 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:11:35.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 213160 2.71 MB/sec execute 301 sec latency 15.139 ms 2024-09-19T06:11:35.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:35 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:11:35.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:35 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:11:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:36 smithi157 ceph-mon[35346]: pgmap v255: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-09-19T06:11:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:36 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:36 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:11:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:36 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:11:36.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 213438 2.70 MB/sec execute 302 sec latency 18.794 ms 2024-09-19T06:11:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:36 smithi005 ceph-mon[30955]: pgmap v255: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-09-19T06:11:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:36 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:36 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:11:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:36 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:11:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:36 smithi049 ceph-mon[34640]: pgmap v255: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-09-19T06:11:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:36 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:11:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:36 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:11:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:36 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:11:37.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 214003 2.70 MB/sec execute 303 sec latency 10.944 ms 2024-09-19T06:11:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.627+0000 7f39bd853640 1 -- 172.21.15.5:0/1292483496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b81009a0 msgr2=0x7f39b8100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.627+0000 7f39bd853640 1 --2- 172.21.15.5:0/1292483496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b81009a0 0x7f39b8100e00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009930 tx=0x7f39ac02a420 comp rx=0 tx=0).stop 2024-09-19T06:11:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.627+0000 7f39bd853640 1 -- 172.21.15.5:0/1292483496 shutdown_connections 2024-09-19T06:11:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.627+0000 7f39bd853640 1 --2- 172.21.15.5:0/1292483496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39b8101340 0x7f39b810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.627+0000 7f39bd853640 1 --2- 172.21.15.5:0/1292483496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b81009a0 0x7f39b8100e00 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.627+0000 7f39bd853640 1 --2- 172.21.15.5:0/1292483496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b81069b0 0x7f39b8106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.627+0000 7f39bd853640 1 -- 172.21.15.5:0/1292483496 >> 172.21.15.5:0/1292483496 conn(0x7f39b80779a0 msgr2=0x7f39b80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.627+0000 7f39bd853640 1 -- 172.21.15.5:0/1292483496 shutdown_connections 2024-09-19T06:11:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.627+0000 7f39bd853640 1 -- 172.21.15.5:0/1292483496 wait complete. 2024-09-19T06:11:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.628+0000 7f39bd853640 1 Processor -- start 2024-09-19T06:11:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.628+0000 7f39bd853640 1 -- start start 2024-09-19T06:11:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.628+0000 7f39bd853640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b81009a0 0x7f39b8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.628+0000 7f39bd853640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39b8101340 0x7f39b81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.628+0000 7f39bd853640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b81069b0 0x7f39b819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.628+0000 7f39bd853640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39b8110c70 con 0x7f39b81009a0 2024-09-19T06:11:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.628+0000 7f39bd853640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39b8110af0 con 0x7f39b81069b0 2024-09-19T06:11:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.628+0000 7f39bd853640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39b8110df0 con 0x7f39b8101340 2024-09-19T06:11:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.628+0000 7f39b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b81009a0 0x7f39b8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b81009a0 0x7f39b8198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39202/0 (socket says 172.21.15.5:39202) 2024-09-19T06:11:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39b6ffd640 1 -- 172.21.15.5:0/1297023766 learned_addr learned my addr 172.21.15.5:0/1297023766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39b77fe640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b81069b0 0x7f39b819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39b67fc640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39b8101340 0x7f39b81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39b6ffd640 1 -- 172.21.15.5:0/1297023766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39b8101340 msgr2=0x7f39b81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39b8101340 0x7f39b81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39b6ffd640 1 -- 172.21.15.5:0/1297023766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b81069b0 msgr2=0x7f39b819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b81069b0 0x7f39b819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39b6ffd640 1 -- 172.21.15.5:0/1297023766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b819d380 con 0x7f39b81009a0 2024-09-19T06:11:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39b77fe640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b81069b0 0x7f39b819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b81009a0 0x7f39b8198390 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f39a800e9f0 tx=0x7f39a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39bc851640 1 -- 172.21.15.5:0/1297023766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a800cd80 con 0x7f39b81009a0 2024-09-19T06:11:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39bc851640 1 -- 172.21.15.5:0/1297023766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39a80108e0 con 0x7f39b81009a0 2024-09-19T06:11:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39bc851640 1 -- 172.21.15.5:0/1297023766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8018920 con 0x7f39b81009a0 2024-09-19T06:11:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39bd853640 1 -- 172.21.15.5:0/1297023766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b819d670 con 0x7f39b81009a0 2024-09-19T06:11:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.629+0000 7f39bd853640 1 -- 172.21.15.5:0/1297023766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b81a4f00 con 0x7f39b81009a0 2024-09-19T06:11:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.632+0000 7f39bc851640 1 -- 172.21.15.5:0/1297023766 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f39a8010a90 con 0x7f39b81009a0 2024-09-19T06:11:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.632+0000 7f39c05e8740 1 Processor -- start 2024-09-19T06:11:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.632+0000 7f39c05e8740 1 -- start start 2024-09-19T06:11:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39c05e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a208af0 0x56292a208ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39c05e8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a209410 0x56292a213770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39c05e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 0x56292a2160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39c05e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56292a16d2f0 con 0x56292a213cb0 2024-09-19T06:11:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39c05e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56292a115270 con 0x56292a208af0 2024-09-19T06:11:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39c05e8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56292a1692b0 con 0x56292a209410 2024-09-19T06:11:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39bc851640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f39a00a33b0 0x7f39a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 0x56292a2160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b77fe640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f39a00a33b0 0x7f39a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 0x56292a2160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39204/0 (socket says 172.21.15.5:39204) 2024-09-19T06:11:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b67fc640 1 -- 172.21.15.5:0/1508451658 learned_addr learned my addr 172.21.15.5:0/1508451658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39bc851640 1 -- 172.21.15.5:0/1297023766 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f39a80cffd0 con 0x7f39b81009a0 2024-09-19T06:11:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b77fe640 1 --2- 172.21.15.5:0/1508451658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a208af0 0x56292a208ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/1508451658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a209410 0x56292a213770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b67fc640 1 -- 172.21.15.5:0/1508451658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a209410 msgr2=0x56292a213770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b67fc640 1 --2- 172.21.15.5:0/1508451658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a209410 0x56292a213770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b67fc640 1 -- 172.21.15.5:0/1508451658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a208af0 msgr2=0x56292a208ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b67fc640 1 --2- 172.21.15.5:0/1508451658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a208af0 0x56292a208ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b67fc640 1 -- 172.21.15.5:0/1508451658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56292a114340 con 0x56292a213cb0 2024-09-19T06:11:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b67fc640 1 --2- 172.21.15.5:0/1508451658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 0x56292a2160a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f39ac02a940 tx=0x7f39ac031ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d79c8d7b736646 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f39b77fe640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f39a00a33b0 0x7f39a00a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f39b0005fd0 tx=0x7f39b00125b0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.633+0000 7f399dffb640 1 -- 172.21.15.5:0/1508451658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac040070 con 0x56292a213cb0 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.634+0000 7f399dffb640 1 -- 172.21.15.5:0/1508451658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39ac031e50 con 0x56292a213cb0 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.634+0000 7f399dffb640 1 -- 172.21.15.5:0/1508451658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac035040 con 0x56292a213cb0 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.634+0000 7f39c05e8740 1 -- 172.21.15.5:0/1508451658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 msgr2=0x56292a2160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.634+0000 7f39c05e8740 1 --2- 172.21.15.5:0/1508451658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 0x56292a2160a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f39ac02a940 tx=0x7f39ac031ba0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.634+0000 7f39c05e8740 1 -- 172.21.15.5:0/1508451658 shutdown_connections 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.634+0000 7f39c05e8740 1 --2- 172.21.15.5:0/1508451658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 0x56292a2160a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.634+0000 7f39c05e8740 1 --2- 172.21.15.5:0/1508451658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a209410 0x56292a213770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.634+0000 7f39c05e8740 1 --2- 172.21.15.5:0/1508451658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a208af0 0x56292a208ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.634+0000 7f39c05e8740 1 -- 172.21.15.5:0/1508451658 >> 172.21.15.5:0/1508451658 conn(0x56292a205060 msgr2=0x56292a205440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.634+0000 7f39c05e8740 1 -- 172.21.15.5:0/1508451658 shutdown_connections 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.634+0000 7f39c05e8740 1 -- 172.21.15.5:0/1508451658 wait complete. 2024-09-19T06:11:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39c05e8740 1 -- start start 2024-09-19T06:11:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39c05e8740 1 Processor -- start 2024-09-19T06:11:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39c05e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a208af0 0x56292a210b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39c05e8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a209410 0x56292a211050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39c05e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 0x56292a211590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39c05e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56292a216880 con 0x56292a213cb0 2024-09-19T06:11:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39c05e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56292a0f6290 con 0x56292a208af0 2024-09-19T06:11:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39c05e8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56292a216a00 con 0x56292a209410 2024-09-19T06:11:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39c05e8740 10 client.0 _ll_register_callbacks cb 0x56292a127140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 0x56292a211590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 0x56292a211590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39210/0 (socket says 172.21.15.5:39210) 2024-09-19T06:11:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39b77fe640 1 -- 172.21.15.5:0/2702123214 learned_addr learned my addr 172.21.15.5:0/2702123214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39b6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a208af0 0x56292a210b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.635+0000 7f39b67fc640 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a209410 0x56292a211050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f39b6ffd640 1 -- 172.21.15.5:0/2702123214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a209410 msgr2=0x56292a211050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a209410 0x56292a211050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f39b6ffd640 1 -- 172.21.15.5:0/2702123214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 msgr2=0x56292a211590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 0x56292a211590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f39b6ffd640 1 -- 172.21.15.5:0/2702123214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562929e80210 con 0x56292a208af0 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f39b77fe640 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 0x56292a211590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a208af0 0x56292a210b10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f39a80cd700 tx=0x7f39a809be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a6a186e2d2c5aa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f3996ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f3996ffd640 1 -- 172.21.15.5:0/2702123214 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a80cee60 con 0x56292a208af0 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f3996ffd640 1 -- 172.21.15.5:0/2702123214 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f39a80bc2b0 con 0x56292a208af0 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f39c05e8740 1 -- 172.21.15.5:0/2702123214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39ac031e50 con 0x56292a208af0 2024-09-19T06:11:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f3996ffd640 1 -- 172.21.15.5:0/2702123214 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f39a809c340 con 0x56292a208af0 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f39c05e8740 10 client.27134 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f39c05e8740 1 -- 172.21.15.5:0/2702123214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562929f4c620 con 0x56292a208af0 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.636+0000 7f3996ffd640 1 -- 172.21.15.5:0/2702123214 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39ac031e50 con 0x56292a208af0 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.637+0000 7f3996ffd640 1 -- 172.21.15.5:0/2702123214 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f39a80a02a0 con 0x56292a208af0 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.637+0000 7f39c05e8740 10 client.27134 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.637+0000 7f39c05e8740 10 client.27134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.637+0000 7f39c05e8740 20 client.27134 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.637+0000 7f39c05e8740 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56292a223b20 0x56292a225f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.637+0000 7f39c05e8740 4 client.27134 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.637+0000 7f39c05e8740 1 -- 172.21.15.5:0/2702123214 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56292a17aa20 con 0x56292a223b20 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.637+0000 7f39b67fc640 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56292a223b20 0x56292a225f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.637+0000 7f3996ffd640 10 client.27134 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.637+0000 7f39b67fc640 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56292a223b20 0x56292a225f50 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c5e13b6f74a0f4d server_cookie=d80759e2b52292b3 in_seq=0 out_seq=0 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f3996ffd640 1 -- 172.21.15.5:0/2702123214 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56292a17aa20 con 0x56292a223b20 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f3996ffd640 10 client.27134 handle_command_reply: tid=0 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f39c05e8740 1 client.27134 shutdown 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f39c05e8740 1 -- 172.21.15.5:0/2702123214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a208af0 msgr2=0x56292a210b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f39c05e8740 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a208af0 0x56292a210b10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f39a80cd700 tx=0x7f39a809be80 comp rx=0 tx=0).stop 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f39c05e8740 1 -- 172.21.15.5:0/2702123214 shutdown_connections 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f39c05e8740 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56292a223b20 0x56292a225f50 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f39c05e8740 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a213cb0 0x56292a211590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f39c05e8740 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a209410 0x56292a211050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f39c05e8740 1 --2- 172.21.15.5:0/2702123214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a208af0 0x56292a210b10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f3996ffd640 0 client.27134 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f39c05e8740 1 -- 172.21.15.5:0/2702123214 >> 172.21.15.5:0/2702123214 conn(0x56292a205060 msgr2=0x56292a20ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f39c05e8740 1 -- 172.21.15.5:0/2702123214 shutdown_connections 2024-09-19T06:11:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.638+0000 7f39c05e8740 1 -- 172.21.15.5:0/2702123214 wait complete. 2024-09-19T06:11:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.639+0000 7f39c05e8740 20 client.27134 trim_cache size 0 max 16384 2024-09-19T06:11:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39c05e8740 1 Processor -- start 2024-09-19T06:11:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39c05e8740 1 -- start start 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39c05e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a208af0 0x56292a2112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39c05e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a209410 0x56292a22b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39c05e8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a213cb0 0x56292a22bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39c05e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39b81ad7d0 con 0x56292a208af0 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39c05e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39b81af310 con 0x56292a209410 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39c05e8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39b81adea0 con 0x56292a213cb0 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a208af0 0x56292a2112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a208af0 0x56292a2112f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39212/0 (socket says 172.21.15.5:39212) 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39b6ffd640 1 -- 172.21.15.5:0/3453011629 learned_addr learned my addr 172.21.15.5:0/3453011629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39b67fc640 1 --2- 172.21.15.5:0/3453011629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a209410 0x56292a22b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39b6ffd640 1 -- 172.21.15.5:0/3453011629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a213cb0 msgr2=0x56292a22bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/3453011629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a213cb0 0x56292a22bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39b6ffd640 1 -- 172.21.15.5:0/3453011629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a209410 msgr2=0x56292a22b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/3453011629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a209410 0x56292a22b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39b6ffd640 1 -- 172.21.15.5:0/3453011629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56292a1ff2f0 con 0x56292a208af0 2024-09-19T06:11:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.655+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/3453011629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a208af0 0x56292a2112f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f39a800cfd0 tx=0x7f39a809bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd2942436aef646 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.656+0000 7f399d7fa640 1 -- 172.21.15.5:0/3453011629 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a809bca0 con 0x56292a208af0 2024-09-19T06:11:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.656+0000 7f399d7fa640 1 -- 172.21.15.5:0/3453011629 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39a80cde50 con 0x56292a208af0 2024-09-19T06:11:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.656+0000 7f399d7fa640 1 -- 172.21.15.5:0/3453011629 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a80bc310 con 0x56292a208af0 2024-09-19T06:11:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.656+0000 7f39c05e8740 1 -- 172.21.15.5:0/3453011629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a208af0 msgr2=0x56292a2112f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.656+0000 7f39c05e8740 1 --2- 172.21.15.5:0/3453011629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a208af0 0x56292a2112f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f39a800cfd0 tx=0x7f39a809bfb0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.656+0000 7f39c05e8740 1 -- 172.21.15.5:0/3453011629 shutdown_connections 2024-09-19T06:11:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.656+0000 7f39c05e8740 1 --2- 172.21.15.5:0/3453011629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a213cb0 0x56292a22bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.656+0000 7f39c05e8740 1 --2- 172.21.15.5:0/3453011629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a209410 0x56292a22b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.656+0000 7f39c05e8740 1 --2- 172.21.15.5:0/3453011629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a208af0 0x56292a2112f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.656+0000 7f39c05e8740 1 -- 172.21.15.5:0/3453011629 >> 172.21.15.5:0/3453011629 conn(0x56292a205060 msgr2=0x56292a212b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.656+0000 7f39c05e8740 1 -- 172.21.15.5:0/3453011629 shutdown_connections 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.656+0000 7f39c05e8740 1 -- 172.21.15.5:0/3453011629 wait complete. 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39c05e8740 1 -- start start 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39c05e8740 1 Processor -- start 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39c05e8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a208af0 0x56292a21c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39c05e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a209410 0x56292a20d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39c05e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a213cb0 0x56292a20dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39c05e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39b81aec40 con 0x56292a209410 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39c05e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39b81ae570 con 0x56292a213cb0 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39c05e8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39b81ab7d0 con 0x56292a208af0 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39c05e8740 10 client.0 _ll_register_callbacks cb 0x7f39a8098410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39b67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a209410 0x56292a20d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39b67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a209410 0x56292a20d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39224/0 (socket says 172.21.15.5:39224) 2024-09-19T06:11:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39b67fc640 1 -- 172.21.15.5:0/2543703590 learned_addr learned my addr 172.21.15.5:0/2543703590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a208af0 0x56292a21c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.657+0000 7f39b77fe640 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a213cb0 0x56292a20dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f39b67fc640 1 -- 172.21.15.5:0/2543703590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a208af0 msgr2=0x56292a21c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f39b67fc640 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a208af0 0x56292a21c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f39b67fc640 1 -- 172.21.15.5:0/2543703590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a213cb0 msgr2=0x56292a20dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f39b67fc640 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a213cb0 0x56292a20dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f39b67fc640 1 -- 172.21.15.5:0/2543703590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56292a20e450 con 0x56292a209410 2024-09-19T06:11:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a208af0 0x56292a21c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f39b67fc640 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a209410 0x56292a20d6e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f39ac004650 tx=0x7f39ac005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5091ff0a7fb771c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f3996ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f3996ffd640 1 -- 172.21.15.5:0/2543703590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac040040 con 0x56292a209410 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f3996ffd640 1 -- 172.21.15.5:0/2543703590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f39ac0316a0 con 0x56292a209410 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f39c05e8740 1 -- 172.21.15.5:0/2543703590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39a80cde50 con 0x56292a209410 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f3996ffd640 1 -- 172.21.15.5:0/2543703590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f39ac05f050 con 0x56292a209410 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f39c05e8740 10 client.17574 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f39c05e8740 1 -- 172.21.15.5:0/2543703590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56292a28e770 con 0x56292a209410 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.658+0000 7f3996ffd640 1 -- 172.21.15.5:0/2543703590 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39a80cde50 con 0x56292a209410 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.659+0000 7f3996ffd640 1 -- 172.21.15.5:0/2543703590 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f39ac04e070 con 0x56292a209410 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.659+0000 7f39c05e8740 10 client.17574 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.659+0000 7f39c05e8740 10 client.17574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.659+0000 7f39c05e8740 20 client.17574 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.659+0000 7f39c05e8740 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56292a28ebc0 0x56292a290fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.659+0000 7f39c05e8740 4 client.17574 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:11:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.659+0000 7f39c05e8740 1 -- 172.21.15.5:0/2543703590 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56292a174d70 con 0x56292a28ebc0 2024-09-19T06:11:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.659+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56292a28ebc0 0x56292a290fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.659+0000 7f39b6ffd640 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56292a28ebc0 0x56292a290fb0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df750d727dbaf04c server_cookie=76b6974e4036aa35 in_seq=0 out_seq=0 2024-09-19T06:11:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.659+0000 7f3996ffd640 10 client.17574 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:38.680 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:38 smithi005 ceph-mon[30955]: pgmap v256: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-09-19T06:11:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.679+0000 7f3996ffd640 1 -- 172.21.15.5:0/2543703590 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56292a174d70 con 0x56292a28ebc0 2024-09-19T06:11:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.679+0000 7f3996ffd640 10 client.17574 handle_command_reply: tid=0 2024-09-19T06:11:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.679+0000 7f39c05e8740 1 client.17574 shutdown 2024-09-19T06:11:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.680+0000 7f39c05e8740 1 -- 172.21.15.5:0/2543703590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a209410 msgr2=0x56292a20d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.680+0000 7f39c05e8740 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a209410 0x56292a20d6e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f39ac004650 tx=0x7f39ac005020 comp rx=0 tx=0).stop 2024-09-19T06:11:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.680+0000 7f39c05e8740 1 -- 172.21.15.5:0/2543703590 shutdown_connections 2024-09-19T06:11:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.680+0000 7f39c05e8740 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56292a28ebc0 0x56292a290fb0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.680+0000 7f39c05e8740 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56292a213cb0 0x56292a20dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.680+0000 7f39c05e8740 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56292a209410 0x56292a20d6e0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.680+0000 7f3996ffd640 0 client.17574 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.680+0000 7f39c05e8740 1 --2- 172.21.15.5:0/2543703590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56292a208af0 0x56292a21c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.680+0000 7f39c05e8740 1 -- 172.21.15.5:0/2543703590 >> 172.21.15.5:0/2543703590 conn(0x56292a205060 msgr2=0x56292a20a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.680+0000 7f39c05e8740 1 -- 172.21.15.5:0/2543703590 shutdown_connections 2024-09-19T06:11:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.680+0000 7f39c05e8740 1 -- 172.21.15.5:0/2543703590 wait complete. 2024-09-19T06:11:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.680+0000 7f39c05e8740 20 client.17574 trim_cache size 0 max 16384 2024-09-19T06:11:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:11:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:11:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:11:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:11:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.681+0000 7f39bd853640 1 -- 172.21.15.5:0/1297023766 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f39a00a33b0 msgr2=0x7f39a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.681+0000 7f39bd853640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f39a00a33b0 0x7f39a00a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f39b0005fd0 tx=0x7f39b00125b0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.681+0000 7f39bd853640 1 -- 172.21.15.5:0/1297023766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b81009a0 msgr2=0x7f39b8198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.681+0000 7f39bd853640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b81009a0 0x7f39b8198390 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f39a800e9f0 tx=0x7f39a800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.681+0000 7f39bd853640 1 -- 172.21.15.5:0/1297023766 shutdown_connections 2024-09-19T06:11:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.681+0000 7f39bd853640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b81069b0 0x7f39b819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.681+0000 7f39bd853640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f39a00a33b0 0x7f39a00a5870 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.681+0000 7f39bd853640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39b8101340 0x7f39b81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.681+0000 7f39bd853640 1 --2- 172.21.15.5:0/1297023766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b81009a0 0x7f39b8198390 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.681+0000 7f39bd853640 1 -- 172.21.15.5:0/1297023766 >> 172.21.15.5:0/1297023766 conn(0x7f39b80779a0 msgr2=0x7f39b8069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.681+0000 7f39bd853640 1 -- 172.21.15.5:0/1297023766 shutdown_connections 2024-09-19T06:11:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:38.681+0000 7f39bd853640 1 -- 172.21.15.5:0/1297023766 wait complete. 2024-09-19T06:11:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:38 smithi157 ceph-mon[35346]: pgmap v256: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-09-19T06:11:38.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 214513 2.70 MB/sec execute 304 sec latency 20.925 ms 2024-09-19T06:11:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:38 smithi049 ceph-mon[34640]: pgmap v256: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-09-19T06:11:39.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 215169 2.70 MB/sec execute 305 sec latency 23.656 ms 2024-09-19T06:11:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:40 smithi157 ceph-mon[35346]: pgmap v257: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-19T06:11:40.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 215774 2.70 MB/sec execute 306 sec latency 11.378 ms 2024-09-19T06:11:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:40 smithi005 ceph-mon[30955]: pgmap v257: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-19T06:11:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:40 smithi049 ceph-mon[34640]: pgmap v257: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-19T06:11:41.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 216259 2.70 MB/sec execute 307 sec latency 13.714 ms 2024-09-19T06:11:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:42 smithi157 ceph-mon[35346]: pgmap v258: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-19T06:11:42.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 216746 2.71 MB/sec execute 308 sec latency 12.157 ms 2024-09-19T06:11:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:42 smithi005 ceph-mon[30955]: pgmap v258: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-19T06:11:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:42 smithi049 ceph-mon[34640]: pgmap v258: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-19T06:11:43.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 217034 2.70 MB/sec execute 309 sec latency 13.648 ms 2024-09-19T06:11:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:44 smithi157 ceph-mon[35346]: pgmap v259: 97 pgs: 97 active+clean; 122 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-09-19T06:11:44.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 217564 2.70 MB/sec execute 310 sec latency 15.813 ms 2024-09-19T06:11:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:44 smithi005 ceph-mon[30955]: pgmap v259: 97 pgs: 97 active+clean; 122 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-09-19T06:11:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:44 smithi049 ceph-mon[34640]: pgmap v259: 97 pgs: 97 active+clean; 122 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-09-19T06:11:45.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 218072 2.70 MB/sec execute 311 sec latency 20.807 ms 2024-09-19T06:11:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:46 smithi157 ceph-mon[35346]: pgmap v260: 97 pgs: 97 active+clean; 122 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-19T06:11:46.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 218719 2.70 MB/sec execute 312 sec latency 20.892 ms 2024-09-19T06:11:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:46 smithi005 ceph-mon[30955]: pgmap v260: 97 pgs: 97 active+clean; 122 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-19T06:11:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:46 smithi049 ceph-mon[34640]: pgmap v260: 97 pgs: 97 active+clean; 122 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-19T06:11:47.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 219322 2.70 MB/sec execute 313 sec latency 14.533 ms 2024-09-19T06:11:48.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:48 smithi157 ceph-mon[35346]: pgmap v261: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-19T06:11:48.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 219795 2.70 MB/sec execute 314 sec latency 17.258 ms 2024-09-19T06:11:48.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:48 smithi005 ceph-mon[30955]: pgmap v261: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-19T06:11:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:48 smithi049 ceph-mon[34640]: pgmap v261: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-19T06:11:49.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 220278 2.71 MB/sec execute 315 sec latency 12.154 ms 2024-09-19T06:11:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:50 smithi157 ceph-mon[35346]: pgmap v262: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-19T06:11:50.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 220559 2.70 MB/sec execute 316 sec latency 15.646 ms 2024-09-19T06:11:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:50 smithi005 ceph-mon[30955]: pgmap v262: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-19T06:11:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:50 smithi049 ceph-mon[34640]: pgmap v262: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-19T06:11:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.782+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1315252479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e01069b0 msgr2=0x7fd3e0106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.782+0000 7fd3e71ef640 1 --2- 172.21.15.5:0/1315252479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e01069b0 0x7fd3e0106d90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009930 tx=0x7fd3d402a420 comp rx=0 tx=0).stop 2024-09-19T06:11:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.782+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1315252479 shutdown_connections 2024-09-19T06:11:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.782+0000 7fd3e71ef640 1 --2- 172.21.15.5:0/1315252479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e0101340 0x7fd3e010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.782+0000 7fd3e71ef640 1 --2- 172.21.15.5:0/1315252479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3e01009a0 0x7fd3e0100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.782+0000 7fd3e71ef640 1 --2- 172.21.15.5:0/1315252479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e01069b0 0x7fd3e0106d90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.782+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1315252479 >> 172.21.15.5:0/1315252479 conn(0x7fd3e00779a0 msgr2=0x7fd3e00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.782+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1315252479 shutdown_connections 2024-09-19T06:11:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.782+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1315252479 wait complete. 2024-09-19T06:11:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.782+0000 7fd3e71ef640 1 Processor -- start 2024-09-19T06:11:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.782+0000 7fd3e71ef640 1 -- start start 2024-09-19T06:11:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e71ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e01009a0 0x7fd3e0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e71ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e0101340 0x7fd3e0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e71ef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3e01069b0 0x7fd3e019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e71ef640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e0110cf0 con 0x7fd3e01009a0 2024-09-19T06:11:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e71ef640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e0110b70 con 0x7fd3e01069b0 2024-09-19T06:11:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e71ef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e0110e70 con 0x7fd3e0101340 2024-09-19T06:11:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e4f8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e01009a0 0x7fd3e0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e4f8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e01009a0 0x7fd3e0198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44090/0 (socket says 172.21.15.5:44090) 2024-09-19T06:11:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e4f8b640 1 -- 172.21.15.5:0/1302573295 learned_addr learned my addr 172.21.15.5:0/1302573295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e578c640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3e01069b0 0x7fd3e019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3dffff640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e0101340 0x7fd3e0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e4f8b640 1 -- 172.21.15.5:0/1302573295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e0101340 msgr2=0x7fd3e0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e4f8b640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e0101340 0x7fd3e0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e4f8b640 1 -- 172.21.15.5:0/1302573295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3e01069b0 msgr2=0x7fd3e019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e4f8b640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3e01069b0 0x7fd3e019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e4f8b640 1 -- 172.21.15.5:0/1302573295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e019d320 con 0x7fd3e01009a0 2024-09-19T06:11:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e578c640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3e01069b0 0x7fd3e019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.783+0000 7fd3e4f8b640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e01009a0 0x7fd3e0198330 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4005ab0 tx=0x7fd3d4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.784+0000 7fd3ddffb640 1 -- 172.21.15.5:0/1302573295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d403f070 con 0x7fd3e01009a0 2024-09-19T06:11:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.784+0000 7fd3ddffb640 1 -- 172.21.15.5:0/1302573295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3d4031b50 con 0x7fd3e01009a0 2024-09-19T06:11:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.784+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1302573295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e019d5b0 con 0x7fd3e01009a0 2024-09-19T06:11:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.784+0000 7fd3ddffb640 1 -- 172.21.15.5:0/1302573295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4035030 con 0x7fd3e01009a0 2024-09-19T06:11:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.784+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1302573295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e01a4ea0 con 0x7fd3e01009a0 2024-09-19T06:11:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.786+0000 7fd3ddffb640 1 -- 172.21.15.5:0/1302573295 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd3d4042070 con 0x7fd3e01009a0 2024-09-19T06:11:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3ea065740 1 Processor -- start 2024-09-19T06:11:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3ea065740 1 -- start start 2024-09-19T06:11:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3ddffb640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd3d00a33b0 0x7fd3d00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3ea065740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c72af0 0x558514c72ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3ea065740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c73410 0x558514c7d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3ea065740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c7dcb0 0x558514c800a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3ea065740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558514bd72f0 con 0x558514c72af0 2024-09-19T06:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3ea065740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558514b7f270 con 0x558514c7dcb0 2024-09-19T06:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3ea065740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558514bd32b0 con 0x558514c73410 2024-09-19T06:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3e4f8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c72af0 0x558514c72ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3e578c640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd3d00a33b0 0x7fd3d00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3e4f8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c72af0 0x558514c72ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44092/0 (socket says 172.21.15.5:44092) 2024-09-19T06:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3e4f8b640 1 -- 172.21.15.5:0/1534923414 learned_addr learned my addr 172.21.15.5:0/1534923414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3e578c640 1 --2- 172.21.15.5:0/1534923414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c7dcb0 0x558514c800a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3ddffb640 1 -- 172.21.15.5:0/1302573295 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7fd3d40f02b0 con 0x7fd3e01009a0 2024-09-19T06:11:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.787+0000 7fd3dffff640 1 --2- 172.21.15.5:0/1534923414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c73410 0x558514c7d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.788+0000 7fd3e4f8b640 1 -- 172.21.15.5:0/1534923414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c73410 msgr2=0x558514c7d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.788+0000 7fd3e4f8b640 1 --2- 172.21.15.5:0/1534923414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c73410 0x558514c7d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.788+0000 7fd3e4f8b640 1 -- 172.21.15.5:0/1534923414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c7dcb0 msgr2=0x558514c800a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.788+0000 7fd3e4f8b640 1 --2- 172.21.15.5:0/1534923414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c7dcb0 0x558514c800a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.788+0000 7fd3e4f8b640 1 -- 172.21.15.5:0/1534923414 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558514b7e340 con 0x558514c72af0 2024-09-19T06:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.788+0000 7fd3e4f8b640 1 --2- 172.21.15.5:0/1534923414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c72af0 0x558514c72ed0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4031440 tx=0x7fd3d4034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd137bb71ff47f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.788+0000 7fd3e578c640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd3d00a33b0 0x7fd3d00a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8005fd0 tx=0x7fd3d80116d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.788+0000 7fd3caffd640 1 -- 172.21.15.5:0/1534923414 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4049040 con 0x558514c72af0 2024-09-19T06:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.788+0000 7fd3caffd640 1 -- 172.21.15.5:0/1534923414 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3d40bcc70 con 0x558514c72af0 2024-09-19T06:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.788+0000 7fd3caffd640 1 -- 172.21.15.5:0/1534923414 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d40d5eb0 con 0x558514c72af0 2024-09-19T06:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.789+0000 7fd3ea065740 1 -- 172.21.15.5:0/1534923414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c72af0 msgr2=0x558514c72ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.789+0000 7fd3ea065740 1 --2- 172.21.15.5:0/1534923414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c72af0 0x558514c72ed0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4031440 tx=0x7fd3d4034000 comp rx=0 tx=0).stop 2024-09-19T06:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.789+0000 7fd3ea065740 1 -- 172.21.15.5:0/1534923414 shutdown_connections 2024-09-19T06:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.789+0000 7fd3ea065740 1 --2- 172.21.15.5:0/1534923414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c7dcb0 0x558514c800a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.789+0000 7fd3ea065740 1 --2- 172.21.15.5:0/1534923414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c73410 0x558514c7d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.789+0000 7fd3ea065740 1 --2- 172.21.15.5:0/1534923414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c72af0 0x558514c72ed0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.789+0000 7fd3ea065740 1 -- 172.21.15.5:0/1534923414 >> 172.21.15.5:0/1534923414 conn(0x558514c6f060 msgr2=0x558514c6f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.789+0000 7fd3ea065740 1 -- 172.21.15.5:0/1534923414 shutdown_connections 2024-09-19T06:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.789+0000 7fd3ea065740 1 -- 172.21.15.5:0/1534923414 wait complete. 2024-09-19T06:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.789+0000 7fd3ea065740 1 -- start start 2024-09-19T06:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.789+0000 7fd3ea065740 1 Processor -- start 2024-09-19T06:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3ea065740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c72af0 0x558514c7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3ea065740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c73410 0x558514c7b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3ea065740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c7dcb0 0x558514c7b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3ea065740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558514c80640 con 0x558514c73410 2024-09-19T06:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3ea065740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558514b60290 con 0x558514c7dcb0 2024-09-19T06:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3ea065740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558514c807c0 con 0x558514c72af0 2024-09-19T06:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3ea065740 10 client.0 _ll_register_callbacks cb 0x558514b91140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c73410 0x558514c7b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3e578c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c7dcb0 0x558514c7b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c73410 0x558514c7b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44100/0 (socket says 172.21.15.5:44100) 2024-09-19T06:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3dffff640 1 -- 172.21.15.5:0/2407608718 learned_addr learned my addr 172.21.15.5:0/2407608718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3e4f8b640 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c72af0 0x558514c7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3dffff640 1 -- 172.21.15.5:0/2407608718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c72af0 msgr2=0x558514c7ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3dffff640 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c72af0 0x558514c7ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3dffff640 1 -- 172.21.15.5:0/2407608718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c7dcb0 msgr2=0x558514c7b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3dffff640 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c7dcb0 0x558514c7b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3dffff640 1 -- 172.21.15.5:0/2407608718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585148ea210 con 0x558514c73410 2024-09-19T06:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3e578c640 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c7dcb0 0x558514c7b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.790+0000 7fd3dffff640 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c73410 0x558514c7b040 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc012750 tx=0x7fd3cc012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739ae5f1ae74d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3bffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3bffff640 1 -- 172.21.15.5:0/2407608718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc00f8c0 con 0x558514c73410 2024-09-19T06:11:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3bffff640 1 -- 172.21.15.5:0/2407608718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7fd3cc00fa60 con 0x558514c73410 2024-09-19T06:11:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3ea065740 1 -- 172.21.15.5:0/2407608718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d40bcc70 con 0x558514c73410 2024-09-19T06:11:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3bffff640 1 -- 172.21.15.5:0/2407608718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd3cc00fca0 con 0x558514c73410 2024-09-19T06:11:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3ea065740 10 client.17586 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3ea065740 1 -- 172.21.15.5:0/2407608718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5585149b6620 con 0x558514c73410 2024-09-19T06:11:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3bffff640 1 -- 172.21.15.5:0/2407608718 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3d40bcc70 con 0x558514c73410 2024-09-19T06:11:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3bffff640 1 -- 172.21.15.5:0/2407608718 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fd3cc010430 con 0x558514c73410 2024-09-19T06:11:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3ea065740 10 client.17586 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3ea065740 10 client.17586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3ea065740 20 client.17586 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3ea065740 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558514c8d9d0 0x558514c8fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3ea065740 4 client.17586 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.791+0000 7fd3ea065740 1 -- 172.21.15.5:0/2407608718 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558514be4a20 con 0x558514c8d9d0 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.792+0000 7fd3e4f8b640 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558514c8d9d0 0x558514c8fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.792+0000 7fd3e4f8b640 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558514c8d9d0 0x558514c8fdc0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ffba39d237756e6 server_cookie=48a3da9ad83b9144 in_seq=0 out_seq=0 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.792+0000 7fd3bffff640 10 client.17586 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.792+0000 7fd3bffff640 1 -- 172.21.15.5:0/2407608718 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558514be4a20 con 0x558514c8d9d0 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.792+0000 7fd3bffff640 10 client.17586 handle_command_reply: tid=0 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3ea065740 1 client.17586 shutdown 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3ea065740 1 -- 172.21.15.5:0/2407608718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c73410 msgr2=0x558514c7b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3ea065740 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c73410 0x558514c7b040 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc012750 tx=0x7fd3cc012b50 comp rx=0 tx=0).stop 2024-09-19T06:11:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3ea065740 1 -- 172.21.15.5:0/2407608718 shutdown_connections 2024-09-19T06:11:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3ea065740 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558514c8d9d0 0x558514c8fdc0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3ea065740 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c7dcb0 0x558514c7b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3ea065740 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c73410 0x558514c7b040 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3ea065740 1 --2- 172.21.15.5:0/2407608718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c72af0 0x558514c7ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3ea065740 1 -- 172.21.15.5:0/2407608718 >> 172.21.15.5:0/2407608718 conn(0x558514c6f060 msgr2=0x558514c74dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3bffff640 0 client.17586 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3ea065740 1 -- 172.21.15.5:0/2407608718 shutdown_connections 2024-09-19T06:11:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3ea065740 1 -- 172.21.15.5:0/2407608718 wait complete. 2024-09-19T06:11:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.793+0000 7fd3ea065740 20 client.17586 trim_cache size 0 max 16384 2024-09-19T06:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.809+0000 7fd3ea065740 1 Processor -- start 2024-09-19T06:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.809+0000 7fd3ea065740 1 -- start start 2024-09-19T06:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.809+0000 7fd3ea065740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c72af0 0x558514c7b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.809+0000 7fd3ea065740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c73410 0x558514c95610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.809+0000 7fd3ea065740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c7dcb0 0x558514c95b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.809+0000 7fd3ea065740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01ad880 con 0x558514c72af0 2024-09-19T06:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.809+0000 7fd3ea065740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01af3c0 con 0x558514c73410 2024-09-19T06:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.809+0000 7fd3ea065740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01adf50 con 0x558514c7dcb0 2024-09-19T06:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.809+0000 7fd3dffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c73410 0x558514c95610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.809+0000 7fd3e4f8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c72af0 0x558514c7b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.810+0000 7fd3dffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c73410 0x558514c95610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54336/0 (socket says 172.21.15.5:54336) 2024-09-19T06:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.810+0000 7fd3e578c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c7dcb0 0x558514c95b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.810+0000 7fd3dffff640 1 -- 172.21.15.5:0/3757840974 learned_addr learned my addr 172.21.15.5:0/3757840974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.810+0000 7fd3dffff640 1 -- 172.21.15.5:0/3757840974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c7dcb0 msgr2=0x558514c95b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.810+0000 7fd3dffff640 1 --2- 172.21.15.5:0/3757840974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c7dcb0 0x558514c95b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.810+0000 7fd3dffff640 1 -- 172.21.15.5:0/3757840974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c72af0 msgr2=0x558514c7b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.810+0000 7fd3dffff640 1 --2- 172.21.15.5:0/3757840974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c72af0 0x558514c7b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.810+0000 7fd3dffff640 1 -- 172.21.15.5:0/3757840974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558514c6ee90 con 0x558514c73410 2024-09-19T06:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.810+0000 7fd3dffff640 1 --2- 172.21.15.5:0/3757840974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c73410 0x558514c95610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc010920 tx=0x7fd3cc00fec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd5f8ec843b300f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.810+0000 7fd3ca7fc640 1 -- 172.21.15.5:0/3757840974 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc028020 con 0x558514c73410 2024-09-19T06:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.810+0000 7fd3ca7fc640 1 -- 172.21.15.5:0/3757840974 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3cc010c10 con 0x558514c73410 2024-09-19T06:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.810+0000 7fd3ca7fc640 1 -- 172.21.15.5:0/3757840974 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc005540 con 0x558514c73410 2024-09-19T06:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.811+0000 7fd3ea065740 1 -- 172.21.15.5:0/3757840974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c73410 msgr2=0x558514c95610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.811+0000 7fd3ea065740 1 --2- 172.21.15.5:0/3757840974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c73410 0x558514c95610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc010920 tx=0x7fd3cc00fec0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.811+0000 7fd3ea065740 1 -- 172.21.15.5:0/3757840974 shutdown_connections 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.811+0000 7fd3ea065740 1 --2- 172.21.15.5:0/3757840974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c7dcb0 0x558514c95b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.811+0000 7fd3ea065740 1 --2- 172.21.15.5:0/3757840974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c73410 0x558514c95610 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.811+0000 7fd3ea065740 1 --2- 172.21.15.5:0/3757840974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c72af0 0x558514c7b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.811+0000 7fd3ea065740 1 -- 172.21.15.5:0/3757840974 >> 172.21.15.5:0/3757840974 conn(0x558514c6f060 msgr2=0x558514c7cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.811+0000 7fd3ea065740 1 -- 172.21.15.5:0/3757840974 shutdown_connections 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.811+0000 7fd3ea065740 1 -- 172.21.15.5:0/3757840974 wait complete. 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.811+0000 7fd3ea065740 1 -- start start 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3ea065740 1 Processor -- start 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3ea065740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c72af0 0x558514c85db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3ea065740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c73410 0x558514c862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3ea065740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c7dcb0 0x558514c812d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3ea065740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01aecf0 con 0x558514c7dcb0 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3ea065740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01ae620 con 0x558514c72af0 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3ea065740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01ab880 con 0x558514c73410 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3ea065740 10 client.0 _ll_register_callbacks cb 0x7fd3cc009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3dffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c73410 0x558514c862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3e4f8b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c72af0 0x558514c85db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3e578c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c7dcb0 0x558514c812d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3dffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c73410 0x558514c862f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50806/0 (socket says 172.21.15.5:50806) 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3dffff640 1 -- 172.21.15.5:0/3447374705 learned_addr learned my addr 172.21.15.5:0/3447374705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3dffff640 1 -- 172.21.15.5:0/3447374705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c72af0 msgr2=0x558514c85db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3dffff640 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c72af0 0x558514c85db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3dffff640 1 -- 172.21.15.5:0/3447374705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c7dcb0 msgr2=0x558514c812d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:11:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3dffff640 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c7dcb0 0x558514c812d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3dffff640 1 -- 172.21.15.5:0/3447374705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558514c898a0 con 0x558514c73410 2024-09-19T06:11:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3e578c640 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c7dcb0 0x558514c812d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.812+0000 7fd3e4f8b640 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c72af0 0x558514c85db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:11:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.813+0000 7fd3dffff640 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c73410 0x558514c862f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc010010 tx=0x7fd3cc00fa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=440808e546e10d93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:11:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.813+0000 7fd3bffff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:11:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.813+0000 7fd3bffff640 1 -- 172.21.15.5:0/3447374705 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc032060 con 0x558514c73410 2024-09-19T06:11:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.813+0000 7fd3ea065740 1 -- 172.21.15.5:0/3447374705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3cc010c10 con 0x558514c73410 2024-09-19T06:11:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.813+0000 7fd3bffff640 1 -- 172.21.15.5:0/3447374705 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7fd3cc005450 con 0x558514c73410 2024-09-19T06:11:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.813+0000 7fd3bffff640 1 -- 172.21.15.5:0/3447374705 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd3cc028030 con 0x558514c73410 2024-09-19T06:11:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.813+0000 7fd3ea065740 10 client.27076 fetch_fsmap learned FSMap version 33 2024-09-19T06:11:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.813+0000 7fd3ea065740 1 -- 172.21.15.5:0/3447374705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558514cf8090 con 0x558514c73410 2024-09-19T06:11:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.813+0000 7fd3bffff640 1 -- 172.21.15.5:0/3447374705 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3cc010c10 con 0x558514c73410 2024-09-19T06:11:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.814+0000 7fd3bffff640 1 -- 172.21.15.5:0/3447374705 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fd3cc01c470 con 0x558514c73410 2024-09-19T06:11:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.814+0000 7fd3ea065740 10 client.27076 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:11:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.814+0000 7fd3ea065740 10 client.27076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:11:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.814+0000 7fd3ea065740 20 client.27076 populate_metadata read hostname 'smithi005' 2024-09-19T06:11:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.814+0000 7fd3ea065740 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558514cf8670 0x558514cfaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:11:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.814+0000 7fd3ea065740 4 client.27076 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:11:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.814+0000 7fd3ea065740 1 -- 172.21.15.5:0/3447374705 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558514bded70 con 0x558514cf8670 2024-09-19T06:11:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.814+0000 7fd3e4f8b640 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558514cf8670 0x558514cfaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:11:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.814+0000 7fd3e4f8b640 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558514cf8670 0x558514cfaa90 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5780fb804bffa73 server_cookie=568814f526110ed3 in_seq=0 out_seq=0 2024-09-19T06:11:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.814+0000 7fd3bffff640 10 client.27076 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:51.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 221087 2.70 MB/sec execute 317 sec latency 17.905 ms 2024-09-19T06:11:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3bffff640 1 -- 172.21.15.5:0/3447374705 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558514bded70 con 0x558514cf8670 2024-09-19T06:11:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3bffff640 10 client.27076 handle_command_reply: tid=0 2024-09-19T06:11:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3ea065740 1 client.27076 shutdown 2024-09-19T06:11:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3ea065740 1 -- 172.21.15.5:0/3447374705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c73410 msgr2=0x558514c862f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3ea065740 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c73410 0x558514c862f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc010010 tx=0x7fd3cc00fa40 comp rx=0 tx=0).stop 2024-09-19T06:11:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3ea065740 1 -- 172.21.15.5:0/3447374705 shutdown_connections 2024-09-19T06:11:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3ea065740 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558514cf8670 0x558514cfaa90 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3ea065740 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558514c7dcb0 0x558514c812d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3ea065740 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558514c73410 0x558514c862f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3bffff640 0 client.27076 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:11:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3ea065740 1 --2- 172.21.15.5:0/3447374705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558514c72af0 0x558514c85db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3ea065740 1 -- 172.21.15.5:0/3447374705 >> 172.21.15.5:0/3447374705 conn(0x558514c6f060 msgr2=0x558514c7cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3ea065740 1 -- 172.21.15.5:0/3447374705 shutdown_connections 2024-09-19T06:11:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3ea065740 1 -- 172.21.15.5:0/3447374705 wait complete. 2024-09-19T06:11:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.836+0000 7fd3ea065740 20 client.27076 trim_cache size 0 max 16384 2024-09-19T06:11:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:11:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:11:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:11:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:11:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.837+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1302573295 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd3d00a33b0 msgr2=0x7fd3d00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.837+0000 7fd3e71ef640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd3d00a33b0 0x7fd3d00a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8005fd0 tx=0x7fd3d80116d0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.837+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1302573295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e01009a0 msgr2=0x7fd3e0198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:11:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.837+0000 7fd3e71ef640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e01009a0 0x7fd3e0198330 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4005ab0 tx=0x7fd3d4003b90 comp rx=0 tx=0).stop 2024-09-19T06:11:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.837+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1302573295 shutdown_connections 2024-09-19T06:11:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.837+0000 7fd3e71ef640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3e01069b0 0x7fd3e019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.837+0000 7fd3e71ef640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd3d00a33b0 0x7fd3d00a5870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.837+0000 7fd3e71ef640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e0101340 0x7fd3e0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.837+0000 7fd3e71ef640 1 --2- 172.21.15.5:0/1302573295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e01009a0 0x7fd3e0198330 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:11:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.837+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1302573295 >> 172.21.15.5:0/1302573295 conn(0x7fd3e00779a0 msgr2=0x7fd3e0069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:11:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.837+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1302573295 shutdown_connections 2024-09-19T06:11:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:11:51.837+0000 7fd3e71ef640 1 -- 172.21.15.5:0/1302573295 wait complete. 2024-09-19T06:11:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:52 smithi157 ceph-mon[35346]: pgmap v263: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-19T06:11:52.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 221614 2.70 MB/sec execute 318 sec latency 21.406 ms 2024-09-19T06:11:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:52 smithi005 ceph-mon[30955]: pgmap v263: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-19T06:11:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:52 smithi049 ceph-mon[34640]: pgmap v263: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-19T06:11:53.834 INFO:tasks.workunit.client.0.smithi005.stdout: 1 222217 2.69 MB/sec execute 319 sec latency 22.965 ms 2024-09-19T06:11:54.697 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:54 smithi049 ceph-mon[34640]: pgmap v264: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-19T06:11:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:54 smithi157 ceph-mon[35346]: pgmap v264: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-19T06:11:54.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 222842 2.70 MB/sec execute 320 sec latency 14.845 ms 2024-09-19T06:11:54.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:54 smithi005 ceph-mon[30955]: pgmap v264: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-19T06:11:55.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 223334 2.70 MB/sec execute 321 sec latency 10.743 ms 2024-09-19T06:11:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:56 smithi157 ceph-mon[35346]: pgmap v265: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-19T06:11:56.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 223824 2.71 MB/sec execute 322 sec latency 11.774 ms 2024-09-19T06:11:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:56 smithi005 ceph-mon[30955]: pgmap v265: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-19T06:11:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:56 smithi049 ceph-mon[34640]: pgmap v265: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-19T06:11:57.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 224110 2.70 MB/sec execute 323 sec latency 18.683 ms 2024-09-19T06:11:58.010 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:11:58.307 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:11:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:11:58 smithi157 ceph-mon[35346]: pgmap v266: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-09-19T06:11:58.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 224635 2.70 MB/sec execute 324 sec latency 12.472 ms 2024-09-19T06:11:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:11:58 smithi005 ceph-mon[30955]: pgmap v266: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-09-19T06:11:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:11:58 smithi049 ceph-mon[34640]: pgmap v266: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-09-19T06:11:59.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 225104 2.70 MB/sec execute 325 sec latency 84.617 ms 2024-09-19T06:12:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:00 smithi005 ceph-mon[30955]: pgmap v267: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-09-19T06:12:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.789+0000 7fede8e09640 1 -- 172.21.15.5:0/285620843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede4102d30 msgr2=0x7fede4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.789+0000 7fede8e09640 1 --2- 172.21.15.5:0/285620843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede4102d30 0x7fede4103110 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009930 tx=0x7fedd402a420 comp rx=0 tx=0).stop 2024-09-19T06:12:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.789+0000 7fede8e09640 1 -- 172.21.15.5:0/285620843 shutdown_connections 2024-09-19T06:12:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.789+0000 7fede8e09640 1 --2- 172.21.15.5:0/285620843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fede41082a0 0x7fede410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.789+0000 7fede8e09640 1 --2- 172.21.15.5:0/285620843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fede41036e0 0x7fede4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.789+0000 7fede8e09640 1 --2- 172.21.15.5:0/285620843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede4102d30 0x7fede4103110 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.789+0000 7fede8e09640 1 -- 172.21.15.5:0/285620843 >> 172.21.15.5:0/285620843 conn(0x7fede40fc7d0 msgr2=0x7fede40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.789+0000 7fede8e09640 1 -- 172.21.15.5:0/285620843 shutdown_connections 2024-09-19T06:12:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.789+0000 7fede8e09640 1 -- 172.21.15.5:0/285620843 wait complete. 2024-09-19T06:12:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.790+0000 7fede8e09640 1 Processor -- start 2024-09-19T06:12:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.790+0000 7fede8e09640 1 -- start start 2024-09-19T06:12:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.790+0000 7fede8e09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede4102d30 0x7fede41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.790+0000 7fede8e09640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fede41036e0 0x7fede4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.790+0000 7fede8e09640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fede41082a0 0x7fede419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.790+0000 7fede8e09640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fede4111fa0 con 0x7fede4102d30 2024-09-19T06:12:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.790+0000 7fede8e09640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fede4111e20 con 0x7fede41082a0 2024-09-19T06:12:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.790+0000 7fede8e09640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fede4112120 con 0x7fede41036e0 2024-09-19T06:12:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.790+0000 7fede259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede4102d30 0x7fede41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.790+0000 7fede259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede4102d30 0x7fede41984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60996/0 (socket says 172.21.15.5:60996) 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.790+0000 7fede259c640 1 -- 172.21.15.5:0/2792124102 learned_addr learned my addr 172.21.15.5:0/2792124102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fede2d9d640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fede41082a0 0x7fede419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fede1d9b640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fede41036e0 0x7fede4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fede259c640 1 -- 172.21.15.5:0/2792124102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fede41036e0 msgr2=0x7fede4198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fede259c640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fede41036e0 0x7fede4198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fede259c640 1 -- 172.21.15.5:0/2792124102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fede41082a0 msgr2=0x7fede419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fede259c640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fede41082a0 0x7fede419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fede259c640 1 -- 172.21.15.5:0/2792124102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede419d490 con 0x7fede4102d30 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fede2d9d640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fede41082a0 0x7fede419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fede259c640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede4102d30 0x7fede41984e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fedd4005ab0 tx=0x7fedd4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fedd37fe640 1 -- 172.21.15.5:0/2792124102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd403f070 con 0x7fede4102d30 2024-09-19T06:12:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fedd37fe640 1 -- 172.21.15.5:0/2792124102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedd4031b50 con 0x7fede4102d30 2024-09-19T06:12:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fede8e09640 1 -- 172.21.15.5:0/2792124102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede419d720 con 0x7fede4102d30 2024-09-19T06:12:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.791+0000 7fedd37fe640 1 -- 172.21.15.5:0/2792124102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4035030 con 0x7fede4102d30 2024-09-19T06:12:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.792+0000 7fede8e09640 1 -- 172.21.15.5:0/2792124102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede41a5060 con 0x7fede4102d30 2024-09-19T06:12:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.794+0000 7fedd37fe640 1 -- 172.21.15.5:0/2792124102 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fedd4042070 con 0x7fede4102d30 2024-09-19T06:12:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.794+0000 7fedeb1e2740 1 Processor -- start 2024-09-19T06:12:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fedeb1e2740 1 -- start start 2024-09-19T06:12:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fedeb1e2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321bbb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fedeb1e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 0x561321bca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fedeb1e2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bcaa90 0x561321bcce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fedeb1e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561321b212e0 con 0x561321bbbca0 2024-09-19T06:12:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fedeb1e2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561321ac8ce0 con 0x561321bbb380 2024-09-19T06:12:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fedeb1e2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561321b1d2a0 con 0x561321bcaa90 2024-09-19T06:12:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fede259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 0x561321bca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fede259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 0x561321bca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32770/0 (socket says 172.21.15.5:32770) 2024-09-19T06:12:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fede259c640 1 -- 172.21.15.5:0/1439695577 learned_addr learned my addr 172.21.15.5:0/1439695577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fede1d9b640 1 --2- 172.21.15.5:0/1439695577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bcaa90 0x561321bcce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fede2d9d640 1 --2- 172.21.15.5:0/1439695577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321bbb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.795+0000 7fedd37fe640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fedcc0a3430 0x7fedcc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.796+0000 7fede259c640 1 -- 172.21.15.5:0/1439695577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bcaa90 msgr2=0x561321bcce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.796+0000 7fede259c640 1 --2- 172.21.15.5:0/1439695577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bcaa90 0x561321bcce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.796+0000 7fede259c640 1 -- 172.21.15.5:0/1439695577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 msgr2=0x561321bbb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.796+0000 7fede259c640 1 --2- 172.21.15.5:0/1439695577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321bbb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.796+0000 7fede259c640 1 -- 172.21.15.5:0/1439695577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561321aad440 con 0x561321bbbca0 2024-09-19T06:12:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.796+0000 7fede2d9d640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fedcc0a3430 0x7fedcc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.796+0000 7fede1d9b640 1 --2- 172.21.15.5:0/1439695577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bcaa90 0x561321bcce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.796+0000 7fede259c640 1 --2- 172.21.15.5:0/1439695577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 0x561321bca550 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fedd40ed640 tx=0x7fedd40b7e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17e28995d4212b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.796+0000 7fedd37fe640 1 -- 172.21.15.5:0/2792124102 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7fedd40effa0 con 0x7fede4102d30 2024-09-19T06:12:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.796+0000 7fedd0ff9640 1 -- 172.21.15.5:0/1439695577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40bc0e0 con 0x561321bbbca0 2024-09-19T06:12:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.796+0000 7fedd0ff9640 1 -- 172.21.15.5:0/1439695577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedd4084070 con 0x561321bbbca0 2024-09-19T06:12:00.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.796+0000 7fedd0ff9640 1 -- 172.21.15.5:0/1439695577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40bbc70 con 0x561321bbbca0 2024-09-19T06:12:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:00 smithi157 ceph-mon[35346]: pgmap v267: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-09-19T06:12:00.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.797+0000 7fede2d9d640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fedcc0a3430 0x7fedcc0a58f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feddc005c40 tx=0x7feddc004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:00.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.797+0000 7fedeb1e2740 1 -- 172.21.15.5:0/1439695577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 msgr2=0x561321bca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:00.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.797+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/1439695577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 0x561321bca550 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fedd40ed640 tx=0x7fedd40b7e80 comp rx=0 tx=0).stop 2024-09-19T06:12:00.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.797+0000 7fedeb1e2740 1 -- 172.21.15.5:0/1439695577 shutdown_connections 2024-09-19T06:12:00.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.797+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/1439695577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bcaa90 0x561321bcce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.797+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/1439695577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 0x561321bca550 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.797+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/1439695577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321bbb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.797+0000 7fedeb1e2740 1 -- 172.21.15.5:0/1439695577 >> 172.21.15.5:0/1439695577 conn(0x561321bb7a30 msgr2=0x561321bb7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:00.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.797+0000 7fedeb1e2740 1 -- 172.21.15.5:0/1439695577 shutdown_connections 2024-09-19T06:12:00.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.797+0000 7fedeb1e2740 1 -- 172.21.15.5:0/1439695577 wait complete. 2024-09-19T06:12:00.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fedeb1e2740 1 -- start start 2024-09-19T06:12:00.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fedeb1e2740 1 Processor -- start 2024-09-19T06:12:00.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fedeb1e2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321bca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fedeb1e2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bbbca0 0x561321bc7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fedeb1e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bcaa90 0x561321bc7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fedeb1e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561321bcd420 con 0x561321bcaa90 2024-09-19T06:12:00.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fedeb1e2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561321adc050 con 0x561321bbb380 2024-09-19T06:12:00.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fedeb1e2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561321bcd5a0 con 0x561321bbbca0 2024-09-19T06:12:00.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fedeb1e2740 10 client.0 _ll_register_callbacks cb 0x561321b1c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:00.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fede259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321bca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fede259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321bca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60888/0 (socket says 172.21.15.5:60888) 2024-09-19T06:12:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fede259c640 1 -- 172.21.15.5:0/3792018140 learned_addr learned my addr 172.21.15.5:0/3792018140 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fede1d9b640 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bbbca0 0x561321bc7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.798+0000 7fede2d9d640 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bcaa90 0x561321bc7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.799+0000 7fede259c640 1 -- 172.21.15.5:0/3792018140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bbbca0 msgr2=0x561321bc7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.799+0000 7fede259c640 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bbbca0 0x561321bc7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.799+0000 7fede259c640 1 -- 172.21.15.5:0/3792018140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bcaa90 msgr2=0x561321bc7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.799+0000 7fede259c640 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bcaa90 0x561321bc7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.799+0000 7fede259c640 1 -- 172.21.15.5:0/3792018140 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613218923c0 con 0x561321bbb380 2024-09-19T06:12:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.799+0000 7fede259c640 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321bca370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fedd40c49a0 tx=0x7fedd4003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65c1a9c603dd15db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.799+0000 7fedc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:12:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.799+0000 7fedc9ffb640 1 -- 172.21.15.5:0/3792018140 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4049030 con 0x561321bbb380 2024-09-19T06:12:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.799+0000 7fedc9ffb640 1 -- 172.21.15.5:0/3792018140 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7fedd40ed950 con 0x561321bbb380 2024-09-19T06:12:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.799+0000 7fedeb1e2740 1 -- 172.21.15.5:0/3792018140 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd4084070 con 0x561321bbb380 2024-09-19T06:12:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.799+0000 7fedc9ffb640 1 -- 172.21.15.5:0/3792018140 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fedd40bbe50 con 0x561321bbb380 2024-09-19T06:12:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.799+0000 7fedeb1e2740 10 client.27170 fetch_fsmap learned FSMap version 33 2024-09-19T06:12:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.800+0000 7fedeb1e2740 1 -- 172.21.15.5:0/3792018140 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561321bcd420 con 0x561321bbb380 2024-09-19T06:12:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.800+0000 7fedc9ffb640 1 -- 172.21.15.5:0/3792018140 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedd4084070 con 0x561321bbb380 2024-09-19T06:12:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.800+0000 7fedc9ffb640 1 -- 172.21.15.5:0/3792018140 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fedd404c060 con 0x561321bbb380 2024-09-19T06:12:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.800+0000 7fedeb1e2740 10 client.27170 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:12:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.800+0000 7fedeb1e2740 10 client.27170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.800+0000 7fedeb1e2740 20 client.27170 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.800+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561321bd9f70 0x561321bdc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.800+0000 7fedeb1e2740 4 client.27170 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.800+0000 7fedeb1e2740 1 -- 172.21.15.5:0/3792018140 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561321b2e850 con 0x561321bd9f70 2024-09-19T06:12:00.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.800+0000 7fede1d9b640 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561321bd9f70 0x561321bdc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.801+0000 7fede1d9b640 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561321bd9f70 0x561321bdc3c0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4190de6615dbe0e server_cookie=55becaea6afe05d6 in_seq=0 out_seq=0 2024-09-19T06:12:00.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.801+0000 7fedc9ffb640 10 client.27170 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:00.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.801+0000 7fedc9ffb640 1 -- 172.21.15.5:0/3792018140 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561321b2e850 con 0x561321bd9f70 2024-09-19T06:12:00.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.801+0000 7fedc9ffb640 10 client.27170 handle_command_reply: tid=0 2024-09-19T06:12:00.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.801+0000 7fedeb1e2740 1 client.27170 shutdown 2024-09-19T06:12:00.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.802+0000 7fedeb1e2740 1 -- 172.21.15.5:0/3792018140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 msgr2=0x561321bca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:00.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.802+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321bca370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fedd40c49a0 tx=0x7fedd4003f90 comp rx=0 tx=0).stop 2024-09-19T06:12:00.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.802+0000 7fedeb1e2740 1 -- 172.21.15.5:0/3792018140 shutdown_connections 2024-09-19T06:12:00.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.802+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561321bd9f70 0x561321bdc3c0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.802+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bcaa90 0x561321bc7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.802+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bbbca0 0x561321bc7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.802+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/3792018140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321bca370 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.802+0000 7fedeb1e2740 1 -- 172.21.15.5:0/3792018140 >> 172.21.15.5:0/3792018140 conn(0x561321bb7a30 msgr2=0x561321bbd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.802+0000 7fedc9ffb640 0 client.27170 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.802+0000 7fedeb1e2740 1 -- 172.21.15.5:0/3792018140 shutdown_connections 2024-09-19T06:12:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.802+0000 7fedeb1e2740 1 -- 172.21.15.5:0/3792018140 wait complete. 2024-09-19T06:12:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.802+0000 7fedeb1e2740 20 client.27170 trim_cache size 0 max 16384 2024-09-19T06:12:00.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.823+0000 7fedeb1e2740 1 Processor -- start 2024-09-19T06:12:00.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.823+0000 7fedeb1e2740 1 -- start start 2024-09-19T06:12:00.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fedeb1e2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bb7a30 0x561321bd6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fedeb1e2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321be1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fedeb1e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 0x561321be21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fedeb1e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fede41ab760 con 0x561321bbbca0 2024-09-19T06:12:00.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fedeb1e2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fede41ad2a0 con 0x561321bbb380 2024-09-19T06:12:00.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fedeb1e2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fede41abe30 con 0x561321bb7a30 2024-09-19T06:12:00.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 0x561321be21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bb7a30 0x561321bd6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 0x561321be21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32788/0 (socket says 172.21.15.5:32788) 2024-09-19T06:12:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bb7a30 0x561321bd6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46290/0 (socket says 172.21.15.5:46290) 2024-09-19T06:12:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede2d9d640 1 -- 172.21.15.5:0/1955238087 learned_addr learned my addr 172.21.15.5:0/1955238087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede1d9b640 1 --2- 172.21.15.5:0/1955238087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321be1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede1d9b640 1 -- 172.21.15.5:0/1955238087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bb7a30 msgr2=0x561321bd6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede1d9b640 1 --2- 172.21.15.5:0/1955238087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bb7a30 0x561321bd6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede1d9b640 1 -- 172.21.15.5:0/1955238087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 msgr2=0x561321be21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede1d9b640 1 --2- 172.21.15.5:0/1955238087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 0x561321be21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede1d9b640 1 -- 172.21.15.5:0/1955238087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561321bbbad0 con 0x561321bbb380 2024-09-19T06:12:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede259c640 1 --2- 172.21.15.5:0/1955238087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bb7a30 0x561321bd6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:00.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.824+0000 7fede2d9d640 1 --2- 172.21.15.5:0/1955238087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 0x561321be21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.825+0000 7fede1d9b640 1 --2- 172.21.15.5:0/1955238087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321be1c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x561321bb8db0 tx=0x7fedd8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea5ab0392667de00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.825+0000 7fedcbfff640 1 -- 172.21.15.5:0/1955238087 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8004030 con 0x561321bbb380 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.825+0000 7fedcbfff640 1 -- 172.21.15.5:0/1955238087 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedd8005020 con 0x561321bbb380 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.825+0000 7fedcbfff640 1 -- 172.21.15.5:0/1955238087 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8004a90 con 0x561321bbb380 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 -- 172.21.15.5:0/1955238087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 msgr2=0x561321be1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/1955238087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321be1c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x561321bb8db0 tx=0x7fedd8005f90 comp rx=0 tx=0).stop 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 -- 172.21.15.5:0/1955238087 shutdown_connections 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/1955238087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bbbca0 0x561321be21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/1955238087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbb380 0x561321be1c90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/1955238087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bb7a30 0x561321bd6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 -- 172.21.15.5:0/1955238087 >> 172.21.15.5:0/1955238087 conn(0x561321bc7d10 msgr2=0x561321bc6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 -- 172.21.15.5:0/1955238087 shutdown_connections 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 -- 172.21.15.5:0/1955238087 wait complete. 2024-09-19T06:12:00.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 -- start start 2024-09-19T06:12:00.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 Processor -- start 2024-09-19T06:12:00.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bb7a30 0x561321bc1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bbb380 0x561321bd27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbbca0 0x561321bd2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fede41acbd0 con 0x561321bb7a30 2024-09-19T06:12:00.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fede41ac500 con 0x561321bbbca0 2024-09-19T06:12:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fede41a9760 con 0x561321bbb380 2024-09-19T06:12:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.826+0000 7fedeb1e2740 10 client.0 _ll_register_callbacks cb 0x7fedd80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.827+0000 7fede259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bb7a30 0x561321bc1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.827+0000 7fede259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bb7a30 0x561321bc1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32792/0 (socket says 172.21.15.5:32792) 2024-09-19T06:12:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.827+0000 7fede259c640 1 -- 172.21.15.5:0/2495383470 learned_addr learned my addr 172.21.15.5:0/2495383470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.827+0000 7fede1d9b640 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bbb380 0x561321bd27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.827+0000 7fede2d9d640 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbbca0 0x561321bd2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.827+0000 7fede259c640 1 -- 172.21.15.5:0/2495383470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bbb380 msgr2=0x561321bd27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.827+0000 7fede259c640 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bbb380 0x561321bd27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.827+0000 7fede259c640 1 -- 172.21.15.5:0/2495383470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbbca0 msgr2=0x561321bd2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.827+0000 7fede259c640 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbbca0 0x561321bd2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.827+0000 7fede259c640 1 -- 172.21.15.5:0/2495383470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561321bbf250 con 0x561321bb7a30 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.828+0000 7fede259c640 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bb7a30 0x561321bc1f00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fedd40ed730 tx=0x7fedd40b8980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394c30460cba844e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.828+0000 7fedc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.828+0000 7fedc9ffb640 1 -- 172.21.15.5:0/2495383470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4049070 con 0x561321bb7a30 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.828+0000 7fedc9ffb640 1 -- 172.21.15.5:0/2495383470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7fedd40b81a0 con 0x561321bb7a30 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.828+0000 7fedeb1e2740 1 -- 172.21.15.5:0/2495383470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd8005020 con 0x561321bb7a30 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.828+0000 7fedc9ffb640 1 -- 172.21.15.5:0/2495383470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fedd40bbc70 con 0x561321bb7a30 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.828+0000 7fedeb1e2740 10 client.17610 fetch_fsmap learned FSMap version 33 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.828+0000 7fedeb1e2740 1 -- 172.21.15.5:0/2495383470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561321bbf250 con 0x561321bb7a30 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.828+0000 7fedc9ffb640 1 -- 172.21.15.5:0/2495383470 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedd8005020 con 0x561321bb7a30 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.829+0000 7fedc9ffb640 1 -- 172.21.15.5:0/2495383470 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fedd40f38a0 con 0x561321bb7a30 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.829+0000 7fedeb1e2740 10 client.17610 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.829+0000 7fedeb1e2740 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.829+0000 7fedeb1e2740 20 client.17610 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.829+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561321bcaa90 0x561321c4ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.829+0000 7fedeb1e2740 4 client.17610 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.829+0000 7fedeb1e2740 1 -- 172.21.15.5:0/2495383470 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561321b28d60 con 0x561321bcaa90 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.829+0000 7fede1d9b640 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561321bcaa90 0x561321c4ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.829+0000 7fede1d9b640 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561321bcaa90 0x561321c4ba90 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb64d2b8c336567 server_cookie=deea8b33750b0c88 in_seq=0 out_seq=0 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.829+0000 7fedc9ffb640 10 client.17610 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.830+0000 7fedc9ffb640 1 -- 172.21.15.5:0/2495383470 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561321b28d60 con 0x561321bcaa90 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.830+0000 7fedc9ffb640 10 client.17610 handle_command_reply: tid=0 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.830+0000 7fedeb1e2740 1 client.17610 shutdown 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.830+0000 7fedeb1e2740 1 -- 172.21.15.5:0/2495383470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bb7a30 msgr2=0x561321bc1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.830+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bb7a30 0x561321bc1f00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fedd40ed730 tx=0x7fedd40b8980 comp rx=0 tx=0).stop 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.830+0000 7fedeb1e2740 1 -- 172.21.15.5:0/2495383470 shutdown_connections 2024-09-19T06:12:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.830+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561321bcaa90 0x561321c4ba90 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.830+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561321bbbca0 0x561321bd2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.830+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561321bbb380 0x561321bd27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.830+0000 7fedeb1e2740 1 --2- 172.21.15.5:0/2495383470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561321bb7a30 0x561321bc1f00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.830+0000 7fedeb1e2740 1 -- 172.21.15.5:0/2495383470 >> 172.21.15.5:0/2495383470 conn(0x561321bc7d10 msgr2=0x561321bb6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.830+0000 7fedc9ffb640 0 client.17610 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.831+0000 7fedeb1e2740 1 -- 172.21.15.5:0/2495383470 shutdown_connections 2024-09-19T06:12:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.831+0000 7fedeb1e2740 1 -- 172.21.15.5:0/2495383470 wait complete. 2024-09-19T06:12:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.831+0000 7fedeb1e2740 20 client.17610 trim_cache size 0 max 16384 2024-09-19T06:12:00.836 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:12:00.836 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:12:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:12:00.837 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:12:00.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.831+0000 7fede8e09640 1 -- 172.21.15.5:0/2792124102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fedcc0a3430 msgr2=0x7fedcc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:00.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.831+0000 7fede8e09640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fedcc0a3430 0x7fedcc0a58f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feddc005c40 tx=0x7feddc004a40 comp rx=0 tx=0).stop 2024-09-19T06:12:00.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.831+0000 7fede8e09640 1 -- 172.21.15.5:0/2792124102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede4102d30 msgr2=0x7fede41984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:00.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.831+0000 7fede8e09640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede4102d30 0x7fede41984e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fedd4005ab0 tx=0x7fedd4003b90 comp rx=0 tx=0).stop 2024-09-19T06:12:00.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.831+0000 7fede8e09640 1 -- 172.21.15.5:0/2792124102 shutdown_connections 2024-09-19T06:12:00.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.832+0000 7fede8e09640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fede41082a0 0x7fede419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.832+0000 7fede8e09640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fedcc0a3430 0x7fedcc0a58f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.832+0000 7fede8e09640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fede41036e0 0x7fede4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.832+0000 7fede8e09640 1 --2- 172.21.15.5:0/2792124102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede4102d30 0x7fede41984e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:00.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.832+0000 7fede8e09640 1 -- 172.21.15.5:0/2792124102 >> 172.21.15.5:0/2792124102 conn(0x7fede40fc7d0 msgr2=0x7fede4105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:00.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.832+0000 7fede8e09640 1 -- 172.21.15.5:0/2792124102 shutdown_connections 2024-09-19T06:12:00.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:00.832+0000 7fede8e09640 1 -- 172.21.15.5:0/2792124102 wait complete. 2024-09-19T06:12:00.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 225624 2.69 MB/sec execute 326 sec latency 21.685 ms 2024-09-19T06:12:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:00 smithi049 ceph-mon[34640]: pgmap v267: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-09-19T06:12:01.658 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:12:01.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:12:01.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 226069 2.69 MB/sec execute 327 sec latency 23.437 ms 2024-09-19T06:12:01.962 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:02.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.527+0000 7ff54937f640 1 -- 172.21.15.5:0/412622420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff544102d90 msgr2=0x7ff544103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.527+0000 7ff54937f640 1 --2- 172.21.15.5:0/412622420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff544102d90 0x7ff544103170 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff534009930 tx=0x7ff53402a420 comp rx=0 tx=0).stop 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.527+0000 7ff54937f640 1 -- 172.21.15.5:0/412622420 shutdown_connections 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.527+0000 7ff54937f640 1 --2- 172.21.15.5:0/412622420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff544108290 0x7ff54410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.527+0000 7ff54937f640 1 --2- 172.21.15.5:0/412622420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff544103740 0x7ff544107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.527+0000 7ff54937f640 1 --2- 172.21.15.5:0/412622420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff544102d90 0x7ff544103170 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.527+0000 7ff54937f640 1 -- 172.21.15.5:0/412622420 >> 172.21.15.5:0/412622420 conn(0x7ff5440fc7d0 msgr2=0x7ff5440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.527+0000 7ff54937f640 1 -- 172.21.15.5:0/412622420 shutdown_connections 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.528+0000 7ff54937f640 1 -- 172.21.15.5:0/412622420 wait complete. 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.528+0000 7ff54937f640 1 Processor -- start 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.528+0000 7ff54937f640 1 -- start start 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff54937f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff544102d90 0x7ff5441984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff54937f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff544103740 0x7ff544198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:02.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff54937f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff544108290 0x7ff54419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:02.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff54937f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff544111f70 con 0x7ff544102d90 2024-09-19T06:12:02.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff54937f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff544111df0 con 0x7ff544103740 2024-09-19T06:12:02.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff54937f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5441120f0 con 0x7ff544108290 2024-09-19T06:12:02.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff542ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff544102d90 0x7ff5441984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:02.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff542ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff544102d90 0x7ff5441984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32804/0 (socket says 172.21.15.5:32804) 2024-09-19T06:12:02.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff542ffd640 1 -- 172.21.15.5:0/65540797 learned_addr learned my addr 172.21.15.5:0/65540797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:02.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff5427fc640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff544103740 0x7ff544198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:02.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff5437fe640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff544108290 0x7ff54419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:02.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff542ffd640 1 -- 172.21.15.5:0/65540797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff544108290 msgr2=0x7ff54419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:02.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff542ffd640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff544108290 0x7ff54419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:02.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff542ffd640 1 -- 172.21.15.5:0/65540797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff544103740 msgr2=0x7ff544198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff542ffd640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff544103740 0x7ff544198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff542ffd640 1 -- 172.21.15.5:0/65540797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54419d480 con 0x7ff544102d90 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff542ffd640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff544102d90 0x7ff5441984d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff534005ab0 tx=0x7ff534003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.529+0000 7ff52bfff640 1 -- 172.21.15.5:0/65540797 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53403f070 con 0x7ff544102d90 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.530+0000 7ff52bfff640 1 -- 172.21.15.5:0/65540797 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff534031b50 con 0x7ff544102d90 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.530+0000 7ff54937f640 1 -- 172.21.15.5:0/65540797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54419d710 con 0x7ff544102d90 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.530+0000 7ff52bfff640 1 -- 172.21.15.5:0/65540797 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534035030 con 0x7ff544102d90 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.530+0000 7ff54937f640 1 -- 172.21.15.5:0/65540797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5441a5050 con 0x7ff544102d90 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.531+0000 7ff54937f640 1 -- 172.21.15.5:0/65540797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff54406a7f0 con 0x7ff544102d90 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.532+0000 7ff52bfff640 1 -- 172.21.15.5:0/65540797 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff534042070 con 0x7ff544102d90 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.532+0000 7ff52bfff640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff52c0a33b0 0x7ff52c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.533+0000 7ff5427fc640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff52c0a33b0 0x7ff52c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:02.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.533+0000 7ff52bfff640 1 -- 172.21.15.5:0/65540797 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7ff5340eff90 con 0x7ff544102d90 2024-09-19T06:12:02.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.533+0000 7ff5427fc640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff52c0a33b0 0x7ff52c0a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff5440fee10 tx=0x7ff538005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:02.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.537+0000 7ff52bfff640 1 -- 172.21.15.5:0/65540797 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff5340bc300 con 0x7ff544102d90 2024-09-19T06:12:02.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.767+0000 7ff54937f640 1 -- 172.21.15.5:0/65540797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff54410a2a0 con 0x7ff544102d90 2024-09-19T06:12:02.769 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:02 smithi005 ceph-mon[30955]: pgmap v268: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-09-19T06:12:02.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.767+0000 7ff52bfff640 1 -- 172.21.15.5:0/65540797 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11959 (secure 0 0 0) 0x7ff5340c0600 con 0x7ff544102d90 2024-09-19T06:12:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:12:02.770 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-19T06:07:24:781633+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:07:24.019631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:12:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-19T06:12:02.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.770+0000 7ff54937f640 1 -- 172.21.15.5:0/65540797 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff52c0a33b0 msgr2=0x7ff52c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:02.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.770+0000 7ff54937f640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff52c0a33b0 0x7ff52c0a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff5440fee10 tx=0x7ff538005f90 comp rx=0 tx=0).stop 2024-09-19T06:12:02.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.770+0000 7ff54937f640 1 -- 172.21.15.5:0/65540797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff544102d90 msgr2=0x7ff5441984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:02.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.770+0000 7ff54937f640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff544102d90 0x7ff5441984d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff534005ab0 tx=0x7ff534003b90 comp rx=0 tx=0).stop 2024-09-19T06:12:02.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.770+0000 7ff54937f640 1 -- 172.21.15.5:0/65540797 shutdown_connections 2024-09-19T06:12:02.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.770+0000 7ff54937f640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff52c0a33b0 0x7ff52c0a5870 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:02.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.770+0000 7ff54937f640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff544108290 0x7ff54419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:02.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.770+0000 7ff54937f640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff544103740 0x7ff544198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:02.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.770+0000 7ff54937f640 1 --2- 172.21.15.5:0/65540797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff544102d90 0x7ff5441984d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:02.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.770+0000 7ff54937f640 1 -- 172.21.15.5:0/65540797 >> 172.21.15.5:0/65540797 conn(0x7ff5440fc7d0 msgr2=0x7ff544106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:02.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.771+0000 7ff54937f640 1 -- 172.21.15.5:0/65540797 shutdown_connections 2024-09-19T06:12:02.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:02.771+0000 7ff54937f640 1 -- 172.21.15.5:0/65540797 wait complete. 2024-09-19T06:12:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:02 smithi157 ceph-mon[35346]: pgmap v268: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-09-19T06:12:02.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 226792 2.70 MB/sec execute 328 sec latency 38.033 ms 2024-09-19T06:12:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:02 smithi049 ceph-mon[34640]: pgmap v268: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-09-19T06:12:03.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:12:03.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:03 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/65540797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:12:03.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 227151 2.71 MB/sec execute 329 sec latency 15.159 ms 2024-09-19T06:12:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:03.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:03 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/65540797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:12:03.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:03 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/65540797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:12:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.366+0000 7f03c92ce640 1 -- 172.21.15.5:0/1004079862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4106f20 msgr2=0x7f03c410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.366+0000 7f03c92ce640 1 --2- 172.21.15.5:0/1004079862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4106f20 0x7f03c410eaa0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009ca0 tx=0x7f03bc02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.366+0000 7f03c92ce640 1 -- 172.21.15.5:0/1004079862 shutdown_connections 2024-09-19T06:12:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.366+0000 7f03c92ce640 1 --2- 172.21.15.5:0/1004079862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4106f20 0x7f03c410eaa0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.366+0000 7f03c92ce640 1 --2- 172.21.15.5:0/1004079862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c41027a0 0x7f03c41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.366+0000 7f03c92ce640 1 --2- 172.21.15.5:0/1004079862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c4101df0 0x7f03c41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.366+0000 7f03c92ce640 1 -- 172.21.15.5:0/1004079862 >> 172.21.15.5:0/1004079862 conn(0x7f03c40fd8f0 msgr2=0x7f03c40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.366+0000 7f03c92ce640 1 -- 172.21.15.5:0/1004079862 shutdown_connections 2024-09-19T06:12:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.366+0000 7f03c92ce640 1 -- 172.21.15.5:0/1004079862 wait complete. 2024-09-19T06:12:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c92ce640 1 Processor -- start 2024-09-19T06:12:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c92ce640 1 -- start start 2024-09-19T06:12:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c92ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c4101df0 0x7f03c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c92ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c41027a0 0x7f03c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c92ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c4106f20 0x7f03c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c92ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4111e30 con 0x7f03c41027a0 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c92ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4111cb0 con 0x7f03c4101df0 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c92ce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4111fb0 con 0x7f03c4106f20 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c41027a0 0x7f03c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c4101df0 0x7f03c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c41027a0 0x7f03c4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32820/0 (socket says 172.21.15.5:32820) 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c27fc640 1 -- 172.21.15.5:0/1632497279 learned_addr learned my addr 172.21.15.5:0/1632497279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.367+0000 7f03c37fe640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c4106f20 0x7f03c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.368+0000 7f03c2ffd640 1 -- 172.21.15.5:0/1632497279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c4106f20 msgr2=0x7f03c4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.368+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c4106f20 0x7f03c4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.368+0000 7f03c2ffd640 1 -- 172.21.15.5:0/1632497279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c41027a0 msgr2=0x7f03c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.368+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c41027a0 0x7f03c4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.368+0000 7f03c2ffd640 1 -- 172.21.15.5:0/1632497279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c41a6830 con 0x7f03c4101df0 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.368+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c4101df0 0x7f03c40744f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f03b400e9f0 tx=0x7f03b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.368+0000 7f03abfff640 1 -- 172.21.15.5:0/1632497279 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b400ccc0 con 0x7f03c4101df0 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.368+0000 7f03c92ce640 1 -- 172.21.15.5:0/1632497279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c41a6b20 con 0x7f03c4101df0 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.368+0000 7f03abfff640 1 -- 172.21.15.5:0/1632497279 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b4005020 con 0x7f03c4101df0 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.368+0000 7f03abfff640 1 -- 172.21.15.5:0/1632497279 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4004a00 con 0x7f03c4101df0 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.368+0000 7f03c92ce640 1 -- 172.21.15.5:0/1632497279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c41a7010 con 0x7f03c4101df0 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.371+0000 7f03abfff640 1 -- 172.21.15.5:0/1632497279 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f03b4004ba0 con 0x7f03c4101df0 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.371+0000 7f03cb6a7740 1 Processor -- start 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.371+0000 7f03cb6a7740 1 -- start start 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03cb6a7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669ceb40 0x555b669cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03cb6a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 0x555b669cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03cb6a7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669d8370 0x555b669da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03cb6a7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555b669332f0 con 0x555b669ceb40 2024-09-19T06:12:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03cb6a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555b668dacf0 con 0x555b669cd5d0 2024-09-19T06:12:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03cb6a7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555b6692f2b0 con 0x555b669d8370 2024-09-19T06:12:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03abfff640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f03ac0a33b0 0x7f03ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669ceb40 0x555b669cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 0x555b669cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669ceb40 0x555b669cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32824/0 (socket says 172.21.15.5:32824) 2024-09-19T06:12:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03c37fe640 1 -- 172.21.15.5:0/4154354391 learned_addr learned my addr 172.21.15.5:0/4154354391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 0x555b669cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60940/0 (socket says 172.21.15.5:60940) 2024-09-19T06:12:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03c27fc640 1 --2- 172.21.15.5:0/4154354391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669d8370 0x555b669da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03c37fe640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f03ac0a33b0 0x7f03ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03abfff640 1 -- 172.21.15.5:0/1632497279 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f03b40cfd00 con 0x7f03c4101df0 2024-09-19T06:12:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03c2ffd640 1 -- 172.21.15.5:0/4154354391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669d8370 msgr2=0x555b669da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/4154354391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669d8370 0x555b669da760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03c2ffd640 1 -- 172.21.15.5:0/4154354391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669ceb40 msgr2=0x555b669cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/4154354391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669ceb40 0x555b669cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.372+0000 7f03c2ffd640 1 -- 172.21.15.5:0/4154354391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b668bf3a0 con 0x555b669cd5d0 2024-09-19T06:12:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.373+0000 7f03c37fe640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f03ac0a33b0 0x7f03ac0a5870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009ca0 tx=0x7f03bc0093d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.373+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/4154354391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 0x555b669cda10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f03b40cd3d0 tx=0x7f03b40096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a96823dd44bb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.373+0000 7f03a97fa640 1 -- 172.21.15.5:0/4154354391 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b401c070 con 0x555b669cd5d0 2024-09-19T06:12:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.373+0000 7f03a97fa640 1 -- 172.21.15.5:0/4154354391 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b409bf60 con 0x555b669cd5d0 2024-09-19T06:12:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.373+0000 7f03a97fa640 1 -- 172.21.15.5:0/4154354391 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b40cd650 con 0x555b669cd5d0 2024-09-19T06:12:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.374+0000 7f03cb6a7740 1 -- 172.21.15.5:0/4154354391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 msgr2=0x555b669cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.374+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/4154354391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 0x555b669cda10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f03b40cd3d0 tx=0x7f03b40096e0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.374+0000 7f03cb6a7740 1 -- 172.21.15.5:0/4154354391 shutdown_connections 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.374+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/4154354391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669d8370 0x555b669da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.374+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/4154354391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 0x555b669cda10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.374+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/4154354391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669ceb40 0x555b669cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.374+0000 7f03cb6a7740 1 -- 172.21.15.5:0/4154354391 >> 172.21.15.5:0/4154354391 conn(0x555b66996310 msgr2=0x555b669c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.374+0000 7f03cb6a7740 1 -- 172.21.15.5:0/4154354391 shutdown_connections 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.374+0000 7f03cb6a7740 1 -- 172.21.15.5:0/4154354391 wait complete. 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.374+0000 7f03cb6a7740 1 -- start start 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.374+0000 7f03cb6a7740 1 Processor -- start 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03cb6a7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669cd5d0 0x555b669d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03cb6a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669ceb40 0x555b669d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03cb6a7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669d8370 0x555b669d5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03cb6a7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555b669dafb0 con 0x555b669d8370 2024-09-19T06:12:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03cb6a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555b668ee060 con 0x555b669ceb40 2024-09-19T06:12:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03cb6a7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555b669db130 con 0x555b669cd5d0 2024-09-19T06:12:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03cb6a7740 10 client.0 _ll_register_callbacks cb 0x555b666418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669cd5d0 0x555b669d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669d8370 0x555b669d5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669cd5d0 0x555b669d7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46348/0 (socket says 172.21.15.5:46348) 2024-09-19T06:12:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c2ffd640 1 -- 172.21.15.5:0/3891771882 learned_addr learned my addr 172.21.15.5:0/3891771882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c27fc640 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669ceb40 0x555b669d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c37fe640 1 -- 172.21.15.5:0/3891771882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669cd5d0 msgr2=0x555b669d7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c37fe640 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669cd5d0 0x555b669d7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c37fe640 1 -- 172.21.15.5:0/3891771882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669ceb40 msgr2=0x555b669d5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c37fe640 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669ceb40 0x555b669d5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c37fe640 1 -- 172.21.15.5:0/3891771882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b666a43c0 con 0x555b669d8370 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669cd5d0 0x555b669d7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c27fc640 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669ceb40 0x555b669d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03c37fe640 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669d8370 0x555b669d5610 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f03bc033040 tx=0x7f03bc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d561e77eca7dca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.375+0000 7f03a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.376+0000 7f03a6ff5640 1 -- 172.21.15.5:0/3891771882 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc004040 con 0x555b669d8370 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.376+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3891771882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b409bf60 con 0x555b669d8370 2024-09-19T06:12:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.376+0000 7f03a6ff5640 1 -- 172.21.15.5:0/3891771882 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f03bc0041e0 con 0x555b669d8370 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.376+0000 7f03a6ff5640 1 -- 172.21.15.5:0/3891771882 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f03bc0043e0 con 0x555b669d8370 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.376+0000 7f03cb6a7740 10 client.17628 fetch_fsmap learned FSMap version 33 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.376+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3891771882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555b669e6a50 con 0x555b669d8370 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.376+0000 7f03a6ff5640 1 -- 172.21.15.5:0/3891771882 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b409bf60 con 0x555b669d8370 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.376+0000 7f03a6ff5640 1 -- 172.21.15.5:0/3891771882 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f03bc030c00 con 0x555b669d8370 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.376+0000 7f03cb6a7740 10 client.17628 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.377+0000 7f03cb6a7740 10 client.17628 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.377+0000 7f03cb6a7740 20 client.17628 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.377+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555b669e7eb0 0x555b669ea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.377+0000 7f03cb6a7740 4 client.17628 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.377+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3891771882 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b66940860 con 0x555b669e7eb0 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.377+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555b669e7eb0 0x555b669ea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.377+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555b669e7eb0 0x555b669ea2f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b50728fb7e0d5214 server_cookie=ce3ebb69b80c8d0a in_seq=0 out_seq=0 2024-09-19T06:12:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.377+0000 7f03a6ff5640 10 client.17628 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:12:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03a6ff5640 1 -- 172.21.15.5:0/3891771882 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b66940860 con 0x555b669e7eb0 2024-09-19T06:12:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03a6ff5640 10 client.17628 handle_command_reply: tid=0 2024-09-19T06:12:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03cb6a7740 1 client.17628 shutdown 2024-09-19T06:12:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3891771882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669d8370 msgr2=0x555b669d5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669d8370 0x555b669d5610 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f03bc033040 tx=0x7f03bc009360 comp rx=0 tx=0).stop 2024-09-19T06:12:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3891771882 shutdown_connections 2024-09-19T06:12:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555b669e7eb0 0x555b669ea2f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669d8370 0x555b669d5610 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669ceb40 0x555b669d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/3891771882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669cd5d0 0x555b669d7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03a6ff5640 0 client.17628 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:12:04.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3891771882 >> 172.21.15.5:0/3891771882 conn(0x555b66996310 msgr2=0x555b669cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:04.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3891771882 shutdown_connections 2024-09-19T06:12:04.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3891771882 wait complete. 2024-09-19T06:12:04.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.378+0000 7f03cb6a7740 20 client.17628 trim_cache size 0 max 16384 2024-09-19T06:12:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.394+0000 7f03cb6a7740 1 Processor -- start 2024-09-19T06:12:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03cb6a7740 1 -- start start 2024-09-19T06:12:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03cb6a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 0x555b669e4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03cb6a7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669ceb40 0x555b669d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03cb6a7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669d8370 0x555b669d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03cb6a7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41afa20 con 0x555b669d8370 2024-09-19T06:12:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03cb6a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41b1560 con 0x555b669cd5d0 2024-09-19T06:12:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03cb6a7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41b00f0 con 0x555b669ceb40 2024-09-19T06:12:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 0x555b669e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03c27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669ceb40 0x555b669d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669d8370 0x555b669d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 0x555b669e4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60958/0 (socket says 172.21.15.5:60958) 2024-09-19T06:12:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03c27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669ceb40 0x555b669d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46358/0 (socket says 172.21.15.5:46358) 2024-09-19T06:12:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03c2ffd640 1 -- 172.21.15.5:0/3786740084 learned_addr learned my addr 172.21.15.5:0/3786740084 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03c2ffd640 1 -- 172.21.15.5:0/3786740084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669ceb40 msgr2=0x555b669d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/3786740084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669ceb40 0x555b669d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03c2ffd640 1 -- 172.21.15.5:0/3786740084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669d8370 msgr2=0x555b669d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/3786740084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669d8370 0x555b669d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.395+0000 7f03c2ffd640 1 -- 172.21.15.5:0/3786740084 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b669d35d0 con 0x555b669cd5d0 2024-09-19T06:12:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.396+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/3786740084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 0x555b669e4250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f03b409a210 tx=0x7f03b40054d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9621dd503a0c9050 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.396+0000 7f03a8ff9640 1 -- 172.21.15.5:0/3786740084 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b401c040 con 0x555b669cd5d0 2024-09-19T06:12:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.396+0000 7f03a8ff9640 1 -- 172.21.15.5:0/3786740084 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b409bf60 con 0x555b669cd5d0 2024-09-19T06:12:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.396+0000 7f03a8ff9640 1 -- 172.21.15.5:0/3786740084 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b40cd650 con 0x555b669cd5d0 2024-09-19T06:12:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3786740084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 msgr2=0x555b669e4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/3786740084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 0x555b669e4250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f03b409a210 tx=0x7f03b40054d0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3786740084 shutdown_connections 2024-09-19T06:12:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/3786740084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669d8370 0x555b669d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/3786740084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669ceb40 0x555b669d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/3786740084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669cd5d0 0x555b669e4250 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3786740084 >> 172.21.15.5:0/3786740084 conn(0x555b66996310 msgr2=0x555b669c9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3786740084 shutdown_connections 2024-09-19T06:12:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 -- 172.21.15.5:0/3786740084 wait complete. 2024-09-19T06:12:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 -- start start 2024-09-19T06:12:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 Processor -- start 2024-09-19T06:12:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669cd5d0 0x555b669e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669ceb40 0x555b669d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669d8370 0x555b669d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41b0e90 con 0x555b669cd5d0 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41b07c0 con 0x555b669d8370 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.397+0000 7f03cb6a7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41ada20 con 0x555b669ceb40 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03cb6a7740 10 client.0 _ll_register_callbacks cb 0x7f03bc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669cd5d0 0x555b669e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669d8370 0x555b669d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669d8370 0x555b669d22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60960/0 (socket says 172.21.15.5:60960) 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669cd5d0 0x555b669e09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32850/0 (socket says 172.21.15.5:32850) 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03c37fe640 1 -- 172.21.15.5:0/1944734036 learned_addr learned my addr 172.21.15.5:0/1944734036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03c27fc640 1 --2- 172.21.15.5:0/1944734036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669ceb40 0x555b669d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03c37fe640 1 -- 172.21.15.5:0/1944734036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669ceb40 msgr2=0x555b669d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03c37fe640 1 --2- 172.21.15.5:0/1944734036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669ceb40 0x555b669d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03c37fe640 1 -- 172.21.15.5:0/1944734036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669cd5d0 msgr2=0x555b669e09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03c37fe640 1 --2- 172.21.15.5:0/1944734036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669cd5d0 0x555b669e09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03c37fe640 1 -- 172.21.15.5:0/1944734036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b669d29e0 con 0x555b669d8370 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03c37fe640 1 --2- 172.21.15.5:0/1944734036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669d8370 0x555b669d22a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f03bc02ade0 tx=0x7f03bc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df0d7e55618fba80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03a6ff5640 1 -- 172.21.15.5:0/1944734036 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc05f040 con 0x555b669d8370 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03a6ff5640 1 -- 172.21.15.5:0/1944734036 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8550+0+0 (secure 0 0 0) 0x7f03bc0305c0 con 0x555b669d8370 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.398+0000 7f03cb6a7740 1 -- 172.21.15.5:0/1944734036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b409bf60 con 0x555b669d8370 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.399+0000 7f03a6ff5640 1 -- 172.21.15.5:0/1944734036 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f03bc004840 con 0x555b669d8370 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.399+0000 7f03cb6a7740 10 client.27200 fetch_fsmap learned FSMap version 33 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.399+0000 7f03cb6a7740 1 -- 172.21.15.5:0/1944734036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555b66a52ac0 con 0x555b669d8370 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.399+0000 7f03a6ff5640 1 -- 172.21.15.5:0/1944734036 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b409bf60 con 0x555b669d8370 2024-09-19T06:12:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.399+0000 7f03a6ff5640 1 -- 172.21.15.5:0/1944734036 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f03bc04bb20 con 0x555b669d8370 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.399+0000 7f03cb6a7740 10 client.27200 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.399+0000 7f03cb6a7740 10 client.27200 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.399+0000 7f03cb6a7740 20 client.27200 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.399+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/1944734036 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555b66a53110 0x555b66a55530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.399+0000 7f03cb6a7740 4 client.27200 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.399+0000 7f03cb6a7740 1 -- 172.21.15.5:0/1944734036 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b6693ad70 con 0x555b66a53110 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.399+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/1944734036 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555b66a53110 0x555b66a55530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03a6ff5640 10 client.27200 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/1944734036 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555b66a53110 0x555b66a55530 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1c5e43606ff5355 server_cookie=40ef9f6e63daa900 in_seq=0 out_seq=0 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03a6ff5640 1 -- 172.21.15.5:0/1944734036 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b6693ad70 con 0x555b66a53110 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03a6ff5640 10 client.27200 handle_command_reply: tid=0 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03cb6a7740 1 client.27200 shutdown 2024-09-19T06:12:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03cb6a7740 1 -- 172.21.15.5:0/1944734036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669d8370 msgr2=0x555b669d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/1944734036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669d8370 0x555b669d22a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f03bc02ade0 tx=0x7f03bc004270 comp rx=0 tx=0).stop 2024-09-19T06:12:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03cb6a7740 1 -- 172.21.15.5:0/1944734036 shutdown_connections 2024-09-19T06:12:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/1944734036 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555b66a53110 0x555b66a55530 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/1944734036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b669d8370 0x555b669d22a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/1944734036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b669ceb40 0x555b669d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03cb6a7740 1 --2- 172.21.15.5:0/1944734036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b669cd5d0 0x555b669e09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03a6ff5640 0 client.27200 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:12:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03cb6a7740 1 -- 172.21.15.5:0/1944734036 >> 172.21.15.5:0/1944734036 conn(0x555b66996310 msgr2=0x555b669c84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03cb6a7740 1 -- 172.21.15.5:0/1944734036 shutdown_connections 2024-09-19T06:12:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03cb6a7740 1 -- 172.21.15.5:0/1944734036 wait complete. 2024-09-19T06:12:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.400+0000 7f03cb6a7740 20 client.27200 trim_cache size 0 max 16384 2024-09-19T06:12:04.408 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:12:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.401+0000 7f03c92ce640 1 -- 172.21.15.5:0/1632497279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f03ac0a33b0 msgr2=0x7f03ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.401+0000 7f03c92ce640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f03ac0a33b0 0x7f03ac0a5870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009ca0 tx=0x7f03bc0093d0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.401+0000 7f03c92ce640 1 -- 172.21.15.5:0/1632497279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c4101df0 msgr2=0x7f03c40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.401+0000 7f03c92ce640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c4101df0 0x7f03c40744f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f03b400e9f0 tx=0x7f03b400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.401+0000 7f03c92ce640 1 -- 172.21.15.5:0/1632497279 shutdown_connections 2024-09-19T06:12:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.401+0000 7f03c92ce640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f03ac0a33b0 0x7f03ac0a5870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.401+0000 7f03c92ce640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c4106f20 0x7f03c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.401+0000 7f03c92ce640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c41027a0 0x7f03c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.401+0000 7f03c92ce640 1 --2- 172.21.15.5:0/1632497279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c4101df0 0x7f03c40744f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.401+0000 7f03c92ce640 1 -- 172.21.15.5:0/1632497279 >> 172.21.15.5:0/1632497279 conn(0x7f03c40fd8f0 msgr2=0x7f03c40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.401+0000 7f03c92ce640 1 -- 172.21.15.5:0/1632497279 shutdown_connections 2024-09-19T06:12:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:04.401+0000 7f03c92ce640 1 -- 172.21.15.5:0/1632497279 wait complete. 2024-09-19T06:12:04.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:04 smithi005 ceph-mon[30955]: pgmap v269: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-09-19T06:12:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:04 smithi157 ceph-mon[35346]: pgmap v269: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-09-19T06:12:04.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 227440 2.70 MB/sec execute 330 sec latency 24.237 ms 2024-09-19T06:12:04.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:04 smithi049 ceph-mon[34640]: pgmap v269: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-09-19T06:12:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.933+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/3281882483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4e810c0a0 msgr2=0x7fe4e810e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.933+0000 7fe4ee0a6640 1 --2- 172.21.15.5:0/3281882483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4e810c0a0 0x7fe4e810e540 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009ca0 tx=0x7fe4e002a950 comp rx=0 tx=0).stop 2024-09-19T06:12:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.933+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/3281882483 shutdown_connections 2024-09-19T06:12:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.933+0000 7fe4ee0a6640 1 --2- 172.21.15.5:0/3281882483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4e810c0a0 0x7fe4e810e540 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.933+0000 7fe4ee0a6640 1 --2- 172.21.15.5:0/3281882483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e81035a0 0x7fe4e810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.933+0000 7fe4ee0a6640 1 --2- 172.21.15.5:0/3281882483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e8102c80 0x7fe4e8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.933+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/3281882483 >> 172.21.15.5:0/3281882483 conn(0x7fe4e80779a0 msgr2=0x7fe4e80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.933+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/3281882483 shutdown_connections 2024-09-19T06:12:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.933+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/3281882483 wait complete. 2024-09-19T06:12:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4ee0a6640 1 Processor -- start 2024-09-19T06:12:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4ee0a6640 1 -- start start 2024-09-19T06:12:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4ee0a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4e8102c80 0x7fe4e81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4ee0a6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e81035a0 0x7fe4e8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4ee0a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e810c0a0 0x7fe4e819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4ee0a6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8111d30 con 0x7fe4e8102c80 2024-09-19T06:12:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4ee0a6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8111bb0 con 0x7fe4e81035a0 2024-09-19T06:12:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4ee0a6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8111eb0 con 0x7fe4e810c0a0 2024-09-19T06:12:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4e8102c80 0x7fe4e81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4e8102c80 0x7fe4e81982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43422/0 (socket says 172.21.15.5:43422) 2024-09-19T06:12:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4e77fe640 1 -- 172.21.15.5:0/1517486459 learned_addr learned my addr 172.21.15.5:0/1517486459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4e6ffd640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e81035a0 0x7fe4e8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4e7fff640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e810c0a0 0x7fe4e819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4e77fe640 1 -- 172.21.15.5:0/1517486459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e810c0a0 msgr2=0x7fe4e819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.934+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e810c0a0 0x7fe4e819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.935+0000 7fe4e77fe640 1 -- 172.21.15.5:0/1517486459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e81035a0 msgr2=0x7fe4e8198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.935+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e81035a0 0x7fe4e8198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.935+0000 7fe4e77fe640 1 -- 172.21.15.5:0/1517486459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e819d2d0 con 0x7fe4e8102c80 2024-09-19T06:12:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.935+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4e8102c80 0x7fe4e81982e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800d950 tx=0x7fe4d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.935+0000 7fe4e4ff9640 1 -- 172.21.15.5:0/1517486459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8017030 con 0x7fe4e8102c80 2024-09-19T06:12:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.935+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/1517486459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e819d5c0 con 0x7fe4e8102c80 2024-09-19T06:12:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.935+0000 7fe4e4ff9640 1 -- 172.21.15.5:0/1517486459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4d8004020 con 0x7fe4e8102c80 2024-09-19T06:12:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.935+0000 7fe4e4ff9640 1 -- 172.21.15.5:0/1517486459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80047a0 con 0x7fe4e8102c80 2024-09-19T06:12:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.935+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/1517486459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e8100800 con 0x7fe4e8102c80 2024-09-19T06:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.938+0000 7fe4e4ff9640 1 -- 172.21.15.5:0/1517486459 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe4d80041d0 con 0x7fe4e8102c80 2024-09-19T06:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.938+0000 7fe4f0e3b740 1 Processor -- start 2024-09-19T06:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4f0e3b740 1 -- start start 2024-09-19T06:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4f0e3b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dbffaf0 0x55f55dbffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4f0e3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dc00410 0x55f55dc0a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4f0e3b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc0acb0 0x55f55dc0d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4f0e3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f55db642f0 con 0x55f55dc00410 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4f0e3b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f55db0c270 con 0x55f55dbffaf0 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4f0e3b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f55db602b0 con 0x55f55dc0acb0 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dc00410 0x55f55dc0a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc0acb0 0x55f55dc0d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dc00410 0x55f55dc0a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43424/0 (socket says 172.21.15.5:43424) 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e77fe640 1 -- 172.21.15.5:0/517461275 learned_addr learned my addr 172.21.15.5:0/517461275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e7fff640 1 --2- 172.21.15.5:0/517461275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dbffaf0 0x55f55dbffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e4ff9640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe4d00a33b0 0x7fe4d00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e7fff640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe4d00a33b0 0x7fe4d00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e77fe640 1 -- 172.21.15.5:0/517461275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc0acb0 msgr2=0x55f55dc0d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/517461275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc0acb0 0x55f55dc0d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e77fe640 1 -- 172.21.15.5:0/517461275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dbffaf0 msgr2=0x55f55dbffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/517461275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dbffaf0 0x55f55dbffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e4ff9640 1 -- 172.21.15.5:0/1517486459 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8594+0+0 (secure 0 0 0) 0x7fe4d80cf7c0 con 0x7fe4e8102c80 2024-09-19T06:12:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e77fe640 1 -- 172.21.15.5:0/517461275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f55db0b340 con 0x55f55dc00410 2024-09-19T06:12:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.939+0000 7fe4e6ffd640 1 --2- 172.21.15.5:0/517461275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc0acb0 0x55f55dc0d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.940+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/517461275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dc00410 0x55f55dc0a770 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe4d809b750 tx=0x7fe4d800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1109f9c107eb017 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.940+0000 7fe4e7fff640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe4d00a33b0 0x7fe4d00a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0030040 tx=0x7fe4e002a950 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.940+0000 7fe4cdffb640 1 -- 172.21.15.5:0/517461275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d809c150 con 0x55f55dc00410 2024-09-19T06:12:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.940+0000 7fe4cdffb640 1 -- 172.21.15.5:0/517461275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4d80a46c0 con 0x55f55dc00410 2024-09-19T06:12:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.940+0000 7fe4cdffb640 1 -- 172.21.15.5:0/517461275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80980c0 con 0x55f55dc00410 2024-09-19T06:12:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.941+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/517461275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dc00410 msgr2=0x55f55dc0a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.941+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/517461275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dc00410 0x55f55dc0a770 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe4d809b750 tx=0x7fe4d800ef30 comp rx=0 tx=0).stop 2024-09-19T06:12:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.941+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/517461275 shutdown_connections 2024-09-19T06:12:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.941+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/517461275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc0acb0 0x55f55dc0d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.941+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/517461275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dc00410 0x55f55dc0a770 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.941+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/517461275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dbffaf0 0x55f55dbffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.941+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/517461275 >> 172.21.15.5:0/517461275 conn(0x55f55dbfc060 msgr2=0x55f55dbfc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.941+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/517461275 shutdown_connections 2024-09-19T06:12:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.941+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/517461275 wait complete. 2024-09-19T06:12:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4f0e3b740 1 -- start start 2024-09-19T06:12:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4f0e3b740 1 Processor -- start 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4f0e3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 0x55f55dc07bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4f0e3b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc00410 0x55f55dc08130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4f0e3b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc0acb0 0x55f55dc08670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4f0e3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f55dc0d820 con 0x55f55dbffaf0 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4f0e3b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f55daed290 con 0x55f55dc0acb0 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4f0e3b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f55dc0d9a0 con 0x55f55dc00410 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4f0e3b740 10 client.0 _ll_register_callbacks cb 0x55f55db1e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 0x55f55dc07bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 0x55f55dc07bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43438/0 (socket says 172.21.15.5:43438) 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4e77fe640 1 -- 172.21.15.5:0/1903090904 learned_addr learned my addr 172.21.15.5:0/1903090904 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.942+0000 7fe4e6ffd640 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc00410 0x55f55dc08130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4e7fff640 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc0acb0 0x55f55dc08670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4e77fe640 1 -- 172.21.15.5:0/1903090904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc00410 msgr2=0x55f55dc08130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc00410 0x55f55dc08130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4e77fe640 1 -- 172.21.15.5:0/1903090904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc0acb0 msgr2=0x55f55dc08670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc0acb0 0x55f55dc08670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4e77fe640 1 -- 172.21.15.5:0/1903090904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f55d877210 con 0x55f55dbffaf0 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4e6ffd640 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc00410 0x55f55dc08130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 0x55f55dc07bf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80ccf70 tx=0x7fe4d8009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fced8c48aceeea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4c6ffd640 1 -- 172.21.15.5:0/1903090904 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d809ba50 con 0x55f55dbffaf0 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4c6ffd640 1 -- 172.21.15.5:0/1903090904 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8594+0+0 (secure 0 0 0) 0x7fe4d80a4230 con 0x55f55dbffaf0 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/1903090904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d80a46c0 con 0x55f55dbffaf0 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4c6ffd640 1 -- 172.21.15.5:0/1903090904 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe4d809fe00 con 0x55f55dbffaf0 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4f0e3b740 10 client.17646 fetch_fsmap learned FSMap version 33 2024-09-19T06:12:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/1903090904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f55d943620 con 0x55f55dbffaf0 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.943+0000 7fe4c6ffd640 1 -- 172.21.15.5:0/1903090904 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4d80a46c0 con 0x55f55dbffaf0 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.944+0000 7fe4c6ffd640 1 -- 172.21.15.5:0/1903090904 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe4d8013ce0 con 0x55f55dbffaf0 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.944+0000 7fe4f0e3b740 10 client.17646 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.944+0000 7fe4f0e3b740 10 client.17646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.944+0000 7fe4f0e3b740 20 client.17646 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.944+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f55dc1aa80 0x55f55dc1ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.944+0000 7fe4f0e3b740 4 client.17646 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.944+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/1903090904 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f55db71a20 con 0x55f55dc1aa80 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.944+0000 7fe4e6ffd640 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f55dc1aa80 0x55f55dc1ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.944+0000 7fe4e6ffd640 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f55dc1aa80 0x55f55dc1ce70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12e7815fee5c1e3 server_cookie=282d41a745ae1bc8 in_seq=0 out_seq=0 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.944+0000 7fe4c6ffd640 10 client.17646 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.945+0000 7fe4c6ffd640 1 -- 172.21.15.5:0/1903090904 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f55db71a20 con 0x55f55dc1aa80 2024-09-19T06:12:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.945+0000 7fe4c6ffd640 10 client.17646 handle_command_reply: tid=0 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.945+0000 7fe4f0e3b740 1 client.17646 shutdown 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.946+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/1903090904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 msgr2=0x55f55dc07bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.946+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 0x55f55dc07bf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80ccf70 tx=0x7fe4d8009740 comp rx=0 tx=0).stop 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.946+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/1903090904 shutdown_connections 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.946+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f55dc1aa80 0x55f55dc1ce70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.946+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc0acb0 0x55f55dc08670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.946+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc00410 0x55f55dc08130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.946+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/1903090904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 0x55f55dc07bf0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.946+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/1903090904 >> 172.21.15.5:0/1903090904 conn(0x55f55dbfc060 msgr2=0x55f55dc01d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.946+0000 7fe4c6ffd640 0 client.17646 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.946+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/1903090904 shutdown_connections 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.946+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/1903090904 wait complete. 2024-09-19T06:12:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.946+0000 7fe4f0e3b740 20 client.17646 trim_cache size 0 max 16384 2024-09-19T06:12:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.962+0000 7fe4f0e3b740 1 Processor -- start 2024-09-19T06:12:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.962+0000 7fe4f0e3b740 1 -- start start 2024-09-19T06:12:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4f0e3b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dbffaf0 0x55f55dc02ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4f0e3b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc00410 0x55f55dc084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4f0e3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dc0acb0 0x55f55dc22440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4f0e3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e81ab6e0 con 0x55f55dc0acb0 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4f0e3b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e81ad220 con 0x55f55dc00410 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4f0e3b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e81abdb0 con 0x55f55dbffaf0 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4e77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dbffaf0 0x55f55dc02ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dc0acb0 0x55f55dc22440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4e6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc00410 0x55f55dc084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4e77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dbffaf0 0x55f55dc02ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43264/0 (socket says 172.21.15.5:43264) 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4e77fe640 1 -- 172.21.15.5:0/3041998823 learned_addr learned my addr 172.21.15.5:0/3041998823 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4e77fe640 1 -- 172.21.15.5:0/3041998823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc00410 msgr2=0x55f55dc084a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/3041998823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc00410 0x55f55dc084a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4e77fe640 1 -- 172.21.15.5:0/3041998823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dc0acb0 msgr2=0x55f55dc22440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/3041998823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dc0acb0 0x55f55dc22440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4e77fe640 1 -- 172.21.15.5:0/3041998823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f55dbff870 con 0x55f55dbffaf0 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/3041998823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dbffaf0 0x55f55dc02ef0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80cd4c0 tx=0x7fe4d809b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f2602b3d4603bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.963+0000 7fe4cd7fa640 1 -- 172.21.15.5:0/3041998823 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d809b590 con 0x55f55dbffaf0 2024-09-19T06:12:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.964+0000 7fe4cd7fa640 1 -- 172.21.15.5:0/3041998823 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4d809c150 con 0x55f55dbffaf0 2024-09-19T06:12:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.964+0000 7fe4cd7fa640 1 -- 172.21.15.5:0/3041998823 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8097cf0 con 0x55f55dbffaf0 2024-09-19T06:12:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.964+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/3041998823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dbffaf0 msgr2=0x55f55dc02ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.964+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/3041998823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dbffaf0 0x55f55dc02ef0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80cd4c0 tx=0x7fe4d809b8b0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.964+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/3041998823 shutdown_connections 2024-09-19T06:12:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.964+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/3041998823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dc0acb0 0x55f55dc22440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.964+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/3041998823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc00410 0x55f55dc084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.964+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/3041998823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dbffaf0 0x55f55dc02ef0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.964+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/3041998823 >> 172.21.15.5:0/3041998823 conn(0x55f55dbfc060 msgr2=0x55f55dc09c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.964+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/3041998823 shutdown_connections 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.964+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/3041998823 wait complete. 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4f0e3b740 1 -- start start 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4f0e3b740 1 Processor -- start 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4f0e3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 0x55f55dc0e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4f0e3b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc00410 0x55f55dc0ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4f0e3b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc0acb0 0x55f55dc0f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4f0e3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e81acb50 con 0x55f55dbffaf0 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4f0e3b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e81ac480 con 0x55f55dc0acb0 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4f0e3b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e81a96e0 con 0x55f55dc00410 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4f0e3b740 10 client.0 _ll_register_callbacks cb 0x7fe4e0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 0x55f55dc0e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 0x55f55dc0e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43452/0 (socket says 172.21.15.5:43452) 2024-09-19T06:12:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4e77fe640 1 -- 172.21.15.5:0/410926040 learned_addr learned my addr 172.21.15.5:0/410926040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4e7fff640 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc0acb0 0x55f55dc0f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.965+0000 7fe4e6ffd640 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc00410 0x55f55dc0ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4e77fe640 1 -- 172.21.15.5:0/410926040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc00410 msgr2=0x55f55dc0ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc00410 0x55f55dc0ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4e77fe640 1 -- 172.21.15.5:0/410926040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc0acb0 msgr2=0x55f55dc0f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc0acb0 0x55f55dc0f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4e77fe640 1 -- 172.21.15.5:0/410926040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f55dc84410 con 0x55f55dbffaf0 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4e6ffd640 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc00410 0x55f55dc0ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4e7fff640 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc0acb0 0x55f55dc0f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4e77fe640 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 0x55f55dc0e8f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe4d809ffc0 tx=0x7fe4d8005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd99a7d2ef4e4baa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4c6ffd640 1 -- 172.21.15.5:0/410926040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8097cf0 con 0x55f55dbffaf0 2024-09-19T06:12:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4c6ffd640 1 -- 172.21.15.5:0/410926040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8594+0+0 (secure 0 0 0) 0x7fe4d809b0a0 con 0x55f55dbffaf0 2024-09-19T06:12:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/410926040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d809c150 con 0x55f55dbffaf0 2024-09-19T06:12:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4c6ffd640 1 -- 172.21.15.5:0/410926040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe4d809b590 con 0x55f55dbffaf0 2024-09-19T06:12:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4f0e3b740 10 client.17652 fetch_fsmap learned FSMap version 33 2024-09-19T06:12:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/410926040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f55dc84410 con 0x55f55dbffaf0 2024-09-19T06:12:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.966+0000 7fe4c6ffd640 1 -- 172.21.15.5:0/410926040 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4d809c150 con 0x55f55dbffaf0 2024-09-19T06:12:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.967+0000 7fe4c6ffd640 1 -- 172.21.15.5:0/410926040 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe4d80a0f90 con 0x55f55dbffaf0 2024-09-19T06:12:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.967+0000 7fe4f0e3b740 10 client.17652 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:12:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.967+0000 7fe4f0e3b740 10 client.17652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.967+0000 7fe4f0e3b740 20 client.17652 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.967+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f55dc85920 0x55f55dc87d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.967+0000 7fe4f0e3b740 4 client.17652 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:12:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.967+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/410926040 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f55db6bd70 con 0x55f55dc85920 2024-09-19T06:12:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.967+0000 7fe4e6ffd640 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f55dc85920 0x55f55dc87d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.967+0000 7fe4e6ffd640 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f55dc85920 0x55f55dc87d10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4301cbff4cb84b server_cookie=77a151da2187c59 in_seq=0 out_seq=0 2024-09-19T06:12:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.967+0000 7fe4c6ffd640 10 client.17652 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.985+0000 7fe4c6ffd640 1 -- 172.21.15.5:0/410926040 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f55db6bd70 con 0x55f55dc85920 2024-09-19T06:12:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.985+0000 7fe4c6ffd640 10 client.17652 handle_command_reply: tid=0 2024-09-19T06:12:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.985+0000 7fe4f0e3b740 1 client.17652 shutdown 2024-09-19T06:12:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.986+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/410926040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 msgr2=0x55f55dc0e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.986+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 0x55f55dc0e8f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe4d809ffc0 tx=0x7fe4d8005bf0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.986+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/410926040 shutdown_connections 2024-09-19T06:12:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.986+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f55dc85920 0x55f55dc87d10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.986+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f55dc0acb0 0x55f55dc0f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.986+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f55dc00410 0x55f55dc0ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.986+0000 7fe4c6ffd640 0 client.17652 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.986+0000 7fe4f0e3b740 1 --2- 172.21.15.5:0/410926040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55dbffaf0 0x55f55dc0e8f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.986+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/410926040 >> 172.21.15.5:0/410926040 conn(0x55f55dbfc060 msgr2=0x55f55dc01550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.986+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/410926040 shutdown_connections 2024-09-19T06:12:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.986+0000 7fe4f0e3b740 1 -- 172.21.15.5:0/410926040 wait complete. 2024-09-19T06:12:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.986+0000 7fe4f0e3b740 20 client.17652 trim_cache size 0 max 16384 2024-09-19T06:12:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:12:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:12:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:12:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:12:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.987+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/1517486459 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe4d00a33b0 msgr2=0x7fe4d00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.987+0000 7fe4ee0a6640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe4d00a33b0 0x7fe4d00a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0030040 tx=0x7fe4e002a950 comp rx=0 tx=0).stop 2024-09-19T06:12:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.987+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/1517486459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4e8102c80 msgr2=0x7fe4e81982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.987+0000 7fe4ee0a6640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4e8102c80 0x7fe4e81982e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800d950 tx=0x7fe4d800de10 comp rx=0 tx=0).stop 2024-09-19T06:12:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.987+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/1517486459 shutdown_connections 2024-09-19T06:12:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.987+0000 7fe4ee0a6640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe4d00a33b0 0x7fe4d00a5870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.987+0000 7fe4ee0a6640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e810c0a0 0x7fe4e819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.987+0000 7fe4ee0a6640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e81035a0 0x7fe4e8198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.987+0000 7fe4ee0a6640 1 --2- 172.21.15.5:0/1517486459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4e8102c80 0x7fe4e81982e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.987+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/1517486459 >> 172.21.15.5:0/1517486459 conn(0x7fe4e80779a0 msgr2=0x7fe4e80fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.987+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/1517486459 shutdown_connections 2024-09-19T06:12:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:04.987+0000 7fe4ee0a6640 1 -- 172.21.15.5:0/1517486459 wait complete. 2024-09-19T06:12:05.143 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:12:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:05.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:05 smithi005 ceph-mon[30955]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T06:12:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:05 smithi157 ceph-mon[35346]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T06:12:05.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 227720 2.69 MB/sec execute 331 sec latency 16.096 ms 2024-09-19T06:12:05.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:05 smithi049 ceph-mon[34640]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T06:12:06.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.005+0000 7f210df18640 1 -- 172.21.15.5:0/332946009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2108102570 msgr2=0x7f21081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:06.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.005+0000 7f210df18640 1 --2- 172.21.15.5:0/332946009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2108102570 0x7f21081029b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009900 tx=0x7f20fc02a480 comp rx=0 tx=0).stop 2024-09-19T06:12:06.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.005+0000 7f210df18640 1 -- 172.21.15.5:0/332946009 shutdown_connections 2024-09-19T06:12:06.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.005+0000 7f210df18640 1 --2- 172.21.15.5:0/332946009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2108102ef0 0x7f210810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.005+0000 7f210df18640 1 --2- 172.21.15.5:0/332946009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2108102570 0x7f21081029b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.005+0000 7f210df18640 1 --2- 172.21.15.5:0/332946009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2108100e20 0x7f2108101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.005+0000 7f210df18640 1 -- 172.21.15.5:0/332946009 >> 172.21.15.5:0/332946009 conn(0x7f21080fc7d0 msgr2=0x7f21080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:06.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.005+0000 7f210df18640 1 -- 172.21.15.5:0/332946009 shutdown_connections 2024-09-19T06:12:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.006+0000 7f210df18640 1 -- 172.21.15.5:0/332946009 wait complete. 2024-09-19T06:12:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.006+0000 7f210df18640 1 Processor -- start 2024-09-19T06:12:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.006+0000 7f210df18640 1 -- start start 2024-09-19T06:12:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f210df18640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2108100e20 0x7f21080fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f210df18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2108102570 0x7f21080fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f210df18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2108102ef0 0x7f21080fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f210df18640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2108110e30 con 0x7f2108102ef0 2024-09-19T06:12:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f210df18640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2108110cb0 con 0x7f2108102570 2024-09-19T06:12:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f210df18640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2108110fb0 con 0x7f2108100e20 2024-09-19T06:12:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f2107fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2108102ef0 0x7f21080fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f21077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2108100e20 0x7f21080fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f2106ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2108102570 0x7f21080fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f2106ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2108102570 0x7f21080fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40508/0 (socket says 172.21.15.5:40508) 2024-09-19T06:12:06.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f2106ffd640 1 -- 172.21.15.5:0/3920276331 learned_addr learned my addr 172.21.15.5:0/3920276331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:06.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f21077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2108100e20 0x7f21080fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43278/0 (socket says 172.21.15.5:43278) 2024-09-19T06:12:06.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f2107fff640 1 -- 172.21.15.5:0/3920276331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2108100e20 msgr2=0x7f21080fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:06.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f2107fff640 1 --2- 172.21.15.5:0/3920276331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2108100e20 0x7f21080fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f2107fff640 1 -- 172.21.15.5:0/3920276331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2108102570 msgr2=0x7f21080fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:06.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f2107fff640 1 --2- 172.21.15.5:0/3920276331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2108102570 0x7f21080fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.007+0000 7f2107fff640 1 -- 172.21.15.5:0/3920276331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21081a68e0 con 0x7f2108102ef0 2024-09-19T06:12:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.008+0000 7f2107fff640 1 --2- 172.21.15.5:0/3920276331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2108102ef0 0x7f21080fd700 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2100009fd0 tx=0x7f210000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.008+0000 7f2104ff9640 1 -- 172.21.15.5:0/3920276331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210000ce20 con 0x7f2108102ef0 2024-09-19T06:12:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.008+0000 7f2104ff9640 1 -- 172.21.15.5:0/3920276331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2100005020 con 0x7f2108102ef0 2024-09-19T06:12:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.008+0000 7f210df18640 1 -- 172.21.15.5:0/3920276331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21081a6bd0 con 0x7f2108102ef0 2024-09-19T06:12:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.008+0000 7f2104ff9640 1 -- 172.21.15.5:0/3920276331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100010690 con 0x7f2108102ef0 2024-09-19T06:12:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.008+0000 7f210df18640 1 -- 172.21.15.5:0/3920276331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081a7110 con 0x7f2108102ef0 2024-09-19T06:12:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.011+0000 7f2104ff9640 1 -- 172.21.15.5:0/3920276331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f21000051d0 con 0x7f2108102ef0 2024-09-19T06:12:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f21102f1740 1 Processor -- start 2024-09-19T06:12:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f21102f1740 1 -- start start 2024-09-19T06:12:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f2104ff9640 1 --2- 172.21.15.5:0/3920276331 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20f00a33b0 0x7f20f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f21102f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a6b40 0x5595cd4a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f2104ff9640 1 -- 172.21.15.5:0/3920276331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8594+0+0 (secure 0 0 0) 0x7f21000d0380 con 0x7f2108102ef0 2024-09-19T06:12:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f21102f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a55d0 0x5595cd4a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f2106ffd640 1 --2- 172.21.15.5:0/3920276331 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20f00a33b0 0x7f20f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f2106ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a55d0 0x5595cd4a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f21077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a6b40 0x5595cd4a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f2106ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a55d0 0x5595cd4a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43478/0 (socket says 172.21.15.5:43478) 2024-09-19T06:12:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f2106ffd640 1 -- 172.21.15.5:0/2317396778 learned_addr learned my addr 172.21.15.5:0/2317396778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f21102f1740 1 --2- 172.21.15.5:0/2317396778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4b0370 0x5595cd4b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f21102f1740 1 -- 172.21.15.5:0/2317396778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5595cd40b2f0 con 0x5595cd4a55d0 2024-09-19T06:12:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f21102f1740 1 -- 172.21.15.5:0/2317396778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5595cd3b2cf0 con 0x5595cd4b0370 2024-09-19T06:12:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.012+0000 7f21102f1740 1 -- 172.21.15.5:0/2317396778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5595cd4072b0 con 0x5595cd4a6b40 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f2107fff640 1 --2- 172.21.15.5:0/2317396778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4b0370 0x5595cd4b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f2106ffd640 1 -- 172.21.15.5:0/2317396778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a6b40 msgr2=0x5595cd4a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f2106ffd640 1 --2- 172.21.15.5:0/2317396778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a6b40 0x5595cd4a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f2106ffd640 1 -- 172.21.15.5:0/2317396778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4b0370 msgr2=0x5595cd4b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f2106ffd640 1 --2- 172.21.15.5:0/2317396778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4b0370 0x5595cd4b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f2106ffd640 1 -- 172.21.15.5:0/2317396778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595cd3973a0 con 0x5595cd4a55d0 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f21077fe640 1 --2- 172.21.15.5:0/2317396778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a6b40 0x5595cd4a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f2107fff640 1 --2- 172.21.15.5:0/2317396778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4b0370 0x5595cd4b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f2106ffd640 1 --2- 172.21.15.5:0/3920276331 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20f00a33b0 0x7f20f00a5870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f20fc006010 tx=0x7f20fc032b40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f2106ffd640 1 --2- 172.21.15.5:0/2317396778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a55d0 0x5595cd4a5a10 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009a20 tx=0x7f20fc046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494a13aa177afeea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f20edffb640 1 -- 172.21.15.5:0/2317396778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc005220 con 0x5595cd4a55d0 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f20edffb640 1 -- 172.21.15.5:0/2317396778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20fc0053c0 con 0x5595cd4a55d0 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.013+0000 7f20edffb640 1 -- 172.21.15.5:0/2317396778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc003850 con 0x5595cd4a55d0 2024-09-19T06:12:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.014+0000 7f21102f1740 1 -- 172.21.15.5:0/2317396778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a55d0 msgr2=0x5595cd4a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.014+0000 7f21102f1740 1 --2- 172.21.15.5:0/2317396778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a55d0 0x5595cd4a5a10 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009a20 tx=0x7f20fc046000 comp rx=0 tx=0).stop 2024-09-19T06:12:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.014+0000 7f21102f1740 1 -- 172.21.15.5:0/2317396778 shutdown_connections 2024-09-19T06:12:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.014+0000 7f21102f1740 1 --2- 172.21.15.5:0/2317396778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4b0370 0x5595cd4b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.014+0000 7f21102f1740 1 --2- 172.21.15.5:0/2317396778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a55d0 0x5595cd4a5a10 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.014+0000 7f21102f1740 1 --2- 172.21.15.5:0/2317396778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a6b40 0x5595cd4a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.014+0000 7f21102f1740 1 -- 172.21.15.5:0/2317396778 >> 172.21.15.5:0/2317396778 conn(0x5595cd46e310 msgr2=0x5595cd4a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.014+0000 7f21102f1740 1 -- 172.21.15.5:0/2317396778 shutdown_connections 2024-09-19T06:12:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.014+0000 7f21102f1740 1 -- 172.21.15.5:0/2317396778 wait complete. 2024-09-19T06:12:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.014+0000 7f21102f1740 1 -- start start 2024-09-19T06:12:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f21102f1740 1 Processor -- start 2024-09-19T06:12:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f21102f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a55d0 0x5595cd4afd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f21102f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a6b40 0x5595cd4ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f21102f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4b0370 0x5595cd4ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f21102f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5595cd4b3010 con 0x5595cd4b0370 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f21102f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5595cd3c6060 con 0x5595cd4a6b40 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f21102f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5595cd4b3190 con 0x5595cd4a55d0 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f21102f1740 10 client.0 _ll_register_callbacks cb 0x5595cd1198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f2106ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a6b40 0x5595cd4ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f2106ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a6b40 0x5595cd4ad000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40540/0 (socket says 172.21.15.5:40540) 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f2106ffd640 1 -- 172.21.15.5:0/1446440516 learned_addr learned my addr 172.21.15.5:0/1446440516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f21077fe640 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a55d0 0x5595cd4afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.015+0000 7f2107fff640 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4b0370 0x5595cd4ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f2106ffd640 1 -- 172.21.15.5:0/1446440516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a55d0 msgr2=0x5595cd4afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f2106ffd640 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a55d0 0x5595cd4afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f2106ffd640 1 -- 172.21.15.5:0/1446440516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4b0370 msgr2=0x5595cd4ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f2106ffd640 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4b0370 0x5595cd4ad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f2106ffd640 1 -- 172.21.15.5:0/1446440516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595cd17c3c0 con 0x5595cd4a6b40 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f21077fe640 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a55d0 0x5595cd4afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f2106ffd640 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a6b40 0x5595cd4ad000 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f20fc031680 tx=0x7f20fc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cb3f76a03a173cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f20e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f20e6ffd640 1 -- 172.21.15.5:0/1446440516 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0037f0 con 0x5595cd4a6b40 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f21102f1740 1 -- 172.21.15.5:0/1446440516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20fc0053c0 con 0x5595cd4a6b40 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f20e6ffd640 1 -- 172.21.15.5:0/1446440516 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8594+0+0 (secure 0 0 0) 0x7f20fc003990 con 0x5595cd4a6b40 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f20e6ffd640 1 -- 172.21.15.5:0/1446440516 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f20fc005570 con 0x5595cd4a6b40 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f21102f1740 10 client.27230 fetch_fsmap learned FSMap version 33 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.016+0000 7f21102f1740 1 -- 172.21.15.5:0/1446440516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5595cd4beaf0 con 0x5595cd4a6b40 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.017+0000 7f20e6ffd640 1 -- 172.21.15.5:0/1446440516 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20fc0053c0 con 0x5595cd4a6b40 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.017+0000 7f20e6ffd640 1 -- 172.21.15.5:0/1446440516 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f20fc03d780 con 0x5595cd4a6b40 2024-09-19T06:12:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.017+0000 7f21102f1740 10 client.27230 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:12:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.017+0000 7f21102f1740 10 client.27230 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:12:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.017+0000 7f21102f1740 20 client.27230 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.017+0000 7f21102f1740 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5595cd4bfe40 0x5595cd4c2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.017+0000 7f21102f1740 4 client.27230 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.017+0000 7f21102f1740 1 -- 172.21.15.5:0/1446440516 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595cd418860 con 0x5595cd4bfe40 2024-09-19T06:12:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.017+0000 7f21077fe640 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5595cd4bfe40 0x5595cd4c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.018+0000 7f20e6ffd640 10 client.27230 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:12:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.018+0000 7f21077fe640 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5595cd4bfe40 0x5595cd4c2230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f5125edf3de1229 server_cookie=f7aec0fc8548ceaf in_seq=0 out_seq=0 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f20e6ffd640 1 -- 172.21.15.5:0/1446440516 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595cd418860 con 0x5595cd4bfe40 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f20e6ffd640 10 client.27230 handle_command_reply: tid=0 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f21102f1740 1 client.27230 shutdown 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f21102f1740 1 -- 172.21.15.5:0/1446440516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a6b40 msgr2=0x5595cd4ad000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f21102f1740 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a6b40 0x5595cd4ad000 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f20fc031680 tx=0x7f20fc009580 comp rx=0 tx=0).stop 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f21102f1740 1 -- 172.21.15.5:0/1446440516 shutdown_connections 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f21102f1740 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5595cd4bfe40 0x5595cd4c2230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f21102f1740 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4b0370 0x5595cd4ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f21102f1740 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a6b40 0x5595cd4ad000 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f21102f1740 1 --2- 172.21.15.5:0/1446440516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a55d0 0x5595cd4afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f21102f1740 1 -- 172.21.15.5:0/1446440516 >> 172.21.15.5:0/1446440516 conn(0x5595cd46e310 msgr2=0x5595cd4a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f20e6ffd640 0 client.27230 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:12:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f21102f1740 1 -- 172.21.15.5:0/1446440516 shutdown_connections 2024-09-19T06:12:06.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f21102f1740 1 -- 172.21.15.5:0/1446440516 wait complete. 2024-09-19T06:12:06.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.019+0000 7f21102f1740 20 client.27230 trim_cache size 0 max 16384 2024-09-19T06:12:06.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f21102f1740 1 Processor -- start 2024-09-19T06:12:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f21102f1740 1 -- start start 2024-09-19T06:12:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f21102f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a55d0 0x5595cd4bc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f21102f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a6b40 0x5595cd4a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f21102f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4b0370 0x5595cd4c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f21102f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21081ab6c0 con 0x5595cd4a6b40 2024-09-19T06:12:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f21102f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f21081ad200 con 0x5595cd4b0370 2024-09-19T06:12:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f21102f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f21081abd90 con 0x5595cd4a55d0 2024-09-19T06:12:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f21077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a55d0 0x5595cd4bc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f21077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a55d0 0x5595cd4bc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43300/0 (socket says 172.21.15.5:43300) 2024-09-19T06:12:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f2107fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4b0370 0x5595cd4c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f2106ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a6b40 0x5595cd4a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.034+0000 7f21077fe640 1 -- 172.21.15.5:0/3900843007 learned_addr learned my addr 172.21.15.5:0/3900843007 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.035+0000 7f2107fff640 1 -- 172.21.15.5:0/3900843007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a55d0 msgr2=0x5595cd4bc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.035+0000 7f2107fff640 1 --2- 172.21.15.5:0/3900843007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a55d0 0x5595cd4bc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.035+0000 7f2107fff640 1 -- 172.21.15.5:0/3900843007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a6b40 msgr2=0x5595cd4a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.035+0000 7f2107fff640 1 --2- 172.21.15.5:0/3900843007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a6b40 0x5595cd4a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.035+0000 7f2107fff640 1 -- 172.21.15.5:0/3900843007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595cd4ab550 con 0x5595cd4b0370 2024-09-19T06:12:06.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.035+0000 7f2107fff640 1 --2- 172.21.15.5:0/3900843007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4b0370 0x5595cd4c7800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f21000cda80 tx=0x7f210009c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faf3dc3854326562 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:06.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.035+0000 7f20ed7fa640 1 -- 172.21.15.5:0/3900843007 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21000b9020 con 0x5595cd4b0370 2024-09-19T06:12:06.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.035+0000 7f20ed7fa640 1 -- 172.21.15.5:0/3900843007 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2100098040 con 0x5595cd4b0370 2024-09-19T06:12:06.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.035+0000 7f20ed7fa640 1 -- 172.21.15.5:0/3900843007 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100098c80 con 0x5595cd4b0370 2024-09-19T06:12:06.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.036+0000 7f21102f1740 1 -- 172.21.15.5:0/3900843007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4b0370 msgr2=0x5595cd4c7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.036+0000 7f21102f1740 1 --2- 172.21.15.5:0/3900843007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4b0370 0x5595cd4c7800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f21000cda80 tx=0x7f210009c6d0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.036+0000 7f21102f1740 1 -- 172.21.15.5:0/3900843007 shutdown_connections 2024-09-19T06:12:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.036+0000 7f21102f1740 1 --2- 172.21.15.5:0/3900843007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4b0370 0x5595cd4c7800 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.036+0000 7f21102f1740 1 --2- 172.21.15.5:0/3900843007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a6b40 0x5595cd4a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.036+0000 7f21102f1740 1 --2- 172.21.15.5:0/3900843007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4a55d0 0x5595cd4bc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.036+0000 7f21102f1740 1 -- 172.21.15.5:0/3900843007 >> 172.21.15.5:0/3900843007 conn(0x5595cd46e310 msgr2=0x5595cd4a0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.036+0000 7f21102f1740 1 -- 172.21.15.5:0/3900843007 shutdown_connections 2024-09-19T06:12:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.036+0000 7f21102f1740 1 -- 172.21.15.5:0/3900843007 wait complete. 2024-09-19T06:12:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.036+0000 7f21102f1740 1 -- start start 2024-09-19T06:12:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.036+0000 7f21102f1740 1 Processor -- start 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.037+0000 7f21102f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a55d0 0x5595cd4b8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.037+0000 7f21102f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a6b40 0x5595cd4a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.037+0000 7f21102f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4b0370 0x5595cd4a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.037+0000 7f21102f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21081acb30 con 0x5595cd4a6b40 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.037+0000 7f21102f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f21081ac460 con 0x5595cd4a55d0 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.037+0000 7f21102f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f21081a96e0 con 0x5595cd4b0370 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.037+0000 7f21102f1740 10 client.0 _ll_register_callbacks cb 0x7f21000981f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.037+0000 7f2107fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4b0370 0x5595cd4a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.037+0000 7f2107fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4b0370 0x5595cd4a9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43302/0 (socket says 172.21.15.5:43302) 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.037+0000 7f2107fff640 1 -- 172.21.15.5:0/1919191720 learned_addr learned my addr 172.21.15.5:0/1919191720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.037+0000 7f21077fe640 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a55d0 0x5595cd4b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.037+0000 7f2106ffd640 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a6b40 0x5595cd4a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.038+0000 7f21077fe640 1 -- 172.21.15.5:0/1919191720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4b0370 msgr2=0x5595cd4a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.038+0000 7f21077fe640 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4b0370 0x5595cd4a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.038+0000 7f21077fe640 1 -- 172.21.15.5:0/1919191720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a6b40 msgr2=0x5595cd4a9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.038+0000 7f21077fe640 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a6b40 0x5595cd4a9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.038+0000 7f21077fe640 1 -- 172.21.15.5:0/1919191720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595cd4aa4e0 con 0x5595cd4a55d0 2024-09-19T06:12:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.038+0000 7f21077fe640 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a55d0 0x5595cd4b8900 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009d90 tx=0x7f20f80257e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92fed618e53b108d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.038+0000 7f20e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:12:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.038+0000 7f20e6ffd640 1 -- 172.21.15.5:0/1919191720 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8005020 con 0x5595cd4a55d0 2024-09-19T06:12:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.038+0000 7f21102f1740 1 -- 172.21.15.5:0/1919191720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2100098040 con 0x5595cd4a55d0 2024-09-19T06:12:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.038+0000 7f20e6ffd640 1 -- 172.21.15.5:0/1919191720 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8594+0+0 (secure 0 0 0) 0x7f20f80051c0 con 0x5595cd4a55d0 2024-09-19T06:12:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.039+0000 7f20e6ffd640 1 -- 172.21.15.5:0/1919191720 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f20f8039050 con 0x5595cd4a55d0 2024-09-19T06:12:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.039+0000 7f21102f1740 10 client.27236 fetch_fsmap learned FSMap version 33 2024-09-19T06:12:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.039+0000 7f21102f1740 1 -- 172.21.15.5:0/1919191720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5595cd52a530 con 0x5595cd4a55d0 2024-09-19T06:12:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.039+0000 7f20e6ffd640 1 -- 172.21.15.5:0/1919191720 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2100098040 con 0x5595cd4a55d0 2024-09-19T06:12:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.039+0000 7f20e6ffd640 1 -- 172.21.15.5:0/1919191720 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f20f802c410 con 0x5595cd4a55d0 2024-09-19T06:12:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.039+0000 7f21102f1740 10 client.27236 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:12:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.039+0000 7f21102f1740 10 client.27236 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:12:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.039+0000 7f21102f1740 20 client.27236 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.039+0000 7f21102f1740 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5595cd52ad10 0x5595cd52d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.039+0000 7f21102f1740 4 client.27236 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:12:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.039+0000 7f21102f1740 1 -- 172.21.15.5:0/1919191720 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595cd412d70 con 0x5595cd52ad10 2024-09-19T06:12:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.039+0000 7f2106ffd640 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5595cd52ad10 0x5595cd52d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f2106ffd640 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5595cd52ad10 0x5595cd52d130 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=915226d78ba1e166 server_cookie=90636dbf1de8d5dc in_seq=0 out_seq=0 2024-09-19T06:12:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f20e6ffd640 10 client.27236 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:12:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f20e6ffd640 1 -- 172.21.15.5:0/1919191720 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595cd412d70 con 0x5595cd52ad10 2024-09-19T06:12:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f20e6ffd640 10 client.27236 handle_command_reply: tid=0 2024-09-19T06:12:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f21102f1740 1 client.27236 shutdown 2024-09-19T06:12:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f21102f1740 1 -- 172.21.15.5:0/1919191720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a55d0 msgr2=0x5595cd4b8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f21102f1740 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a55d0 0x5595cd4b8900 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009d90 tx=0x7f20f80257e0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f21102f1740 1 -- 172.21.15.5:0/1919191720 shutdown_connections 2024-09-19T06:12:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f21102f1740 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5595cd52ad10 0x5595cd52d130 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f21102f1740 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cd4b0370 0x5595cd4a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f21102f1740 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cd4a6b40 0x5595cd4a9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f20e6ffd640 0 client.27236 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:12:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f21102f1740 1 --2- 172.21.15.5:0/1919191720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cd4a55d0 0x5595cd4b8900 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f21102f1740 1 -- 172.21.15.5:0/1919191720 >> 172.21.15.5:0/1919191720 conn(0x5595cd46e310 msgr2=0x5595cd4abfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f21102f1740 1 -- 172.21.15.5:0/1919191720 shutdown_connections 2024-09-19T06:12:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.040+0000 7f21102f1740 1 -- 172.21.15.5:0/1919191720 wait complete. 2024-09-19T06:12:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f21102f1740 20 client.27236 trim_cache size 0 max 16384 2024-09-19T06:12:06.049 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:12:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f210df18640 1 -- 172.21.15.5:0/3920276331 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20f00a33b0 msgr2=0x7f20f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f210df18640 1 --2- 172.21.15.5:0/3920276331 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20f00a33b0 0x7f20f00a5870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f20fc006010 tx=0x7f20fc032b40 comp rx=0 tx=0).stop 2024-09-19T06:12:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f210df18640 1 -- 172.21.15.5:0/3920276331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2108102ef0 msgr2=0x7f21080fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f210df18640 1 --2- 172.21.15.5:0/3920276331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2108102ef0 0x7f21080fd700 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2100009fd0 tx=0x7f210000ef10 comp rx=0 tx=0).stop 2024-09-19T06:12:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f210df18640 1 -- 172.21.15.5:0/3920276331 shutdown_connections 2024-09-19T06:12:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f210df18640 1 --2- 172.21.15.5:0/3920276331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2108102ef0 0x7f21080fd700 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f210df18640 1 --2- 172.21.15.5:0/3920276331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2108102570 0x7f21080fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f210df18640 1 --2- 172.21.15.5:0/3920276331 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20f00a33b0 0x7f20f00a5870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f210df18640 1 --2- 172.21.15.5:0/3920276331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2108100e20 0x7f21080fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f210df18640 1 -- 172.21.15.5:0/3920276331 >> 172.21.15.5:0/3920276331 conn(0x7f21080fc7d0 msgr2=0x7f210806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f210df18640 1 -- 172.21.15.5:0/3920276331 shutdown_connections 2024-09-19T06:12:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:06.041+0000 7f210df18640 1 -- 172.21.15.5:0/3920276331 wait complete. 2024-09-19T06:12:06.784 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:06 smithi005 ceph-mon[30955]: pgmap v271: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-09-19T06:12:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:06 smithi157 ceph-mon[35346]: pgmap v271: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-09-19T06:12:06.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:12:06.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 228231 2.69 MB/sec execute 332 sec latency 17.883 ms 2024-09-19T06:12:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:06 smithi049 ceph-mon[34640]: pgmap v271: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-09-19T06:12:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.611+0000 7f35eef25640 1 -- 172.21.15.5:0/1615602698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e81082e0 msgr2=0x7f35e810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.611+0000 7f35eef25640 1 --2- 172.21.15.5:0/1615602698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e81082e0 0x7f35e810ec50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009cd0 tx=0x7f35e002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.611+0000 7f35eef25640 1 -- 172.21.15.5:0/1615602698 shutdown_connections 2024-09-19T06:12:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.611+0000 7f35eef25640 1 --2- 172.21.15.5:0/1615602698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e81082e0 0x7f35e810ec50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.611+0000 7f35eef25640 1 --2- 172.21.15.5:0/1615602698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e8103760 0x7f35e8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.611+0000 7f35eef25640 1 --2- 172.21.15.5:0/1615602698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35e8102db0 0x7f35e8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.611+0000 7f35eef25640 1 -- 172.21.15.5:0/1615602698 >> 172.21.15.5:0/1615602698 conn(0x7f35e80fc7d0 msgr2=0x7f35e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.611+0000 7f35eef25640 1 -- 172.21.15.5:0/1615602698 shutdown_connections 2024-09-19T06:12:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.611+0000 7f35eef25640 1 -- 172.21.15.5:0/1615602698 wait complete. 2024-09-19T06:12:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.612+0000 7f35eef25640 1 Processor -- start 2024-09-19T06:12:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35eef25640 1 -- start start 2024-09-19T06:12:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35eef25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8102db0 0x7f35e8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35eef25640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e8103760 0x7f35e8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35eef25640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35e81082e0 0x7f35e819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35eef25640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8111f60 con 0x7f35e8102db0 2024-09-19T06:12:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35eef25640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8111de0 con 0x7f35e8103760 2024-09-19T06:12:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35eef25640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81120e0 con 0x7f35e81082e0 2024-09-19T06:12:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35eccc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8102db0 0x7f35e8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35eccc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8102db0 0x7f35e8198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43526/0 (socket says 172.21.15.5:43526) 2024-09-19T06:12:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35e7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e8103760 0x7f35e8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35eccc1640 1 -- 172.21.15.5:0/1300090347 learned_addr learned my addr 172.21.15.5:0/1300090347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:07.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35ed4c2640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35e81082e0 0x7f35e819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35e7fff640 1 -- 172.21.15.5:0/1300090347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35e81082e0 msgr2=0x7f35e819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:07.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35e7fff640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35e81082e0 0x7f35e819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35e7fff640 1 -- 172.21.15.5:0/1300090347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8102db0 msgr2=0x7f35e8198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:07.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35e7fff640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8102db0 0x7f35e8198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35e7fff640 1 -- 172.21.15.5:0/1300090347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e819d4f0 con 0x7f35e8103760 2024-09-19T06:12:07.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.613+0000 7f35eccc1640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8102db0 0x7f35e8198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.614+0000 7f35e7fff640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e8103760 0x7f35e8198a80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f35e80697d0 tx=0x7f35d400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.614+0000 7f35e5ffb640 1 -- 172.21.15.5:0/1300090347 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d400cc40 con 0x7f35e8103760 2024-09-19T06:12:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.614+0000 7f35eef25640 1 -- 172.21.15.5:0/1300090347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e819d7e0 con 0x7f35e8103760 2024-09-19T06:12:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.614+0000 7f35e5ffb640 1 -- 172.21.15.5:0/1300090347 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35d400cde0 con 0x7f35e8103760 2024-09-19T06:12:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.614+0000 7f35e5ffb640 1 -- 172.21.15.5:0/1300090347 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4018780 con 0x7f35e8103760 2024-09-19T06:12:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.614+0000 7f35eef25640 1 -- 172.21.15.5:0/1300090347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81a50c0 con 0x7f35e8103760 2024-09-19T06:12:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.617+0000 7f35e5ffb640 1 -- 172.21.15.5:0/1300090347 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f35d4010430 con 0x7f35e8103760 2024-09-19T06:12:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35f12fe740 1 Processor -- start 2024-09-19T06:12:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35f12fe740 1 -- start start 2024-09-19T06:12:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35f12fe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad80043090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35f12fe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad80043a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35f12fe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 0x55ad80050760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35f12fe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ad7ffa92f0 con 0x55ad800435d0 2024-09-19T06:12:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35f12fe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ad7ff50cf0 con 0x55ad80044b40 2024-09-19T06:12:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35f12fe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ad7ffa52b0 con 0x55ad8004e370 2024-09-19T06:12:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35ed4c2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad80043090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35ed4c2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad80043090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40588/0 (socket says 172.21.15.5:40588) 2024-09-19T06:12:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35ed4c2640 1 -- 172.21.15.5:0/243170888 learned_addr learned my addr 172.21.15.5:0/243170888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35eccc1640 1 --2- 172.21.15.5:0/243170888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad80043a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35ed4c2640 1 -- 172.21.15.5:0/243170888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 msgr2=0x55ad80050760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35e5ffb640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35d80a3430 0x7f35d80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.618+0000 7f35ed4c2640 1 --2- 172.21.15.5:0/243170888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 0x55ad80050760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.619+0000 7f35ed4c2640 1 -- 172.21.15.5:0/243170888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 msgr2=0x55ad80043a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.619+0000 7f35ed4c2640 1 --2- 172.21.15.5:0/243170888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad80043a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.619+0000 7f35ed4c2640 1 -- 172.21.15.5:0/243170888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad7ff353a0 con 0x55ad80044b40 2024-09-19T06:12:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.619+0000 7f35eccc1640 1 --2- 172.21.15.5:0/243170888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad80043a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.619+0000 7f35e5ffb640 1 -- 172.21.15.5:0/1300090347 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8594+0+0 (secure 0 0 0) 0x7f35d40cff10 con 0x7f35e8103760 2024-09-19T06:12:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.619+0000 7f35ed4c2640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35d80a3430 0x7f35d80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.619+0000 7f35ed4c2640 1 --2- 172.21.15.5:0/243170888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad80043090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f35e0035000 tx=0x7f35e0009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e63171c9d7a809 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.619+0000 7f35d2ffd640 1 -- 172.21.15.5:0/243170888 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e004d070 con 0x55ad80044b40 2024-09-19T06:12:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.619+0000 7f35d2ffd640 1 -- 172.21.15.5:0/243170888 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35e0036040 con 0x55ad80044b40 2024-09-19T06:12:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.619+0000 7f35d2ffd640 1 -- 172.21.15.5:0/243170888 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00305a0 con 0x55ad80044b40 2024-09-19T06:12:07.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.620+0000 7f35f12fe740 1 -- 172.21.15.5:0/243170888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 msgr2=0x55ad80043090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:07.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.620+0000 7f35ed4c2640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35d80a3430 0x7f35d80a58f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f35e004a000 tx=0x7f35e004b000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.620+0000 7f35f12fe740 1 --2- 172.21.15.5:0/243170888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad80043090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f35e0035000 tx=0x7f35e0009900 comp rx=0 tx=0).stop 2024-09-19T06:12:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.620+0000 7f35f12fe740 1 -- 172.21.15.5:0/243170888 shutdown_connections 2024-09-19T06:12:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.620+0000 7f35f12fe740 1 --2- 172.21.15.5:0/243170888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 0x55ad80050760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.620+0000 7f35f12fe740 1 --2- 172.21.15.5:0/243170888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad80043a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.620+0000 7f35f12fe740 1 --2- 172.21.15.5:0/243170888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad80043090 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.620+0000 7f35f12fe740 1 -- 172.21.15.5:0/243170888 >> 172.21.15.5:0/243170888 conn(0x55ad8000c310 msgr2=0x55ad8003fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.620+0000 7f35f12fe740 1 -- 172.21.15.5:0/243170888 shutdown_connections 2024-09-19T06:12:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.620+0000 7f35f12fe740 1 -- 172.21.15.5:0/243170888 wait complete. 2024-09-19T06:12:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35f12fe740 1 -- start start 2024-09-19T06:12:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35f12fe740 1 Processor -- start 2024-09-19T06:12:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35f12fe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad800435d0 0x55ad8004ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35f12fe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad80044b40 0x55ad8004b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35f12fe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad8004e370 0x55ad8004b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35f12fe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ad80051070 con 0x55ad8004e370 2024-09-19T06:12:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35f12fe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ad7ff64060 con 0x55ad800435d0 2024-09-19T06:12:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35f12fe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ad800511f0 con 0x55ad80044b40 2024-09-19T06:12:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35f12fe740 10 client.0 _ll_register_callbacks cb 0x55ad7fcb78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35ed4c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad8004e370 0x55ad8004b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35ed4c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad8004e370 0x55ad8004b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43552/0 (socket says 172.21.15.5:43552) 2024-09-19T06:12:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35eccc1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad800435d0 0x55ad8004ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35ed4c2640 1 -- 172.21.15.5:0/3049613649 learned_addr learned my addr 172.21.15.5:0/3049613649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.621+0000 7f35e7fff640 1 --2- 172.21.15.5:0/3049613649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad80044b40 0x55ad8004b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35ed4c2640 1 -- 172.21.15.5:0/3049613649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad80044b40 msgr2=0x55ad8004b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35ed4c2640 1 --2- 172.21.15.5:0/3049613649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad80044b40 0x55ad8004b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35ed4c2640 1 -- 172.21.15.5:0/3049613649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad800435d0 msgr2=0x55ad8004ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35ed4c2640 1 --2- 172.21.15.5:0/3049613649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad800435d0 0x55ad8004ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35ed4c2640 1 -- 172.21.15.5:0/3049613649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad7fd1a3c0 con 0x55ad8004e370 2024-09-19T06:12:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35ed4c2640 1 --2- 172.21.15.5:0/3049613649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad8004e370 0x55ad8004b800 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009a60 tx=0x7f35e00043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c206e7399e65c92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35c7fff640 1 -- 172.21.15.5:0/3049613649 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e004d070 con 0x55ad8004e370 2024-09-19T06:12:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35c7fff640 1 -- 172.21.15.5:0/3049613649 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8594+0+0 (secure 0 0 0) 0x7f35e0004b40 con 0x55ad8004e370 2024-09-19T06:12:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35f12fe740 1 -- 172.21.15.5:0/3049613649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e0036040 con 0x55ad8004e370 2024-09-19T06:12:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35c7fff640 1 -- 172.21.15.5:0/3049613649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f35e00305a0 con 0x55ad8004e370 2024-09-19T06:12:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35f12fe740 10 client.17682 fetch_fsmap learned FSMap version 33 2024-09-19T06:12:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.622+0000 7f35f12fe740 1 -- 172.21.15.5:0/3049613649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ad8005cb80 con 0x55ad8004e370 2024-09-19T06:12:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.623+0000 7f35c7fff640 1 -- 172.21.15.5:0/3049613649 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35e0036040 con 0x55ad8004e370 2024-09-19T06:12:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.623+0000 7f35c7fff640 1 -- 172.21.15.5:0/3049613649 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f35e0030070 con 0x55ad8004e370 2024-09-19T06:12:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.623+0000 7f35f12fe740 10 client.17682 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:12:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.623+0000 7f35f12fe740 10 client.17682 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:12:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.623+0000 7f35f12fe740 20 client.17682 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.623+0000 7f35f12fe740 1 --2- 172.21.15.5:0/3049613649 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad8005ded0 0x55ad800602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.623+0000 7f35f12fe740 4 client.17682 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.623+0000 7f35f12fe740 1 -- 172.21.15.5:0/3049613649 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad7ffb6860 con 0x55ad8005ded0 2024-09-19T06:12:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.623+0000 7f35eccc1640 1 --2- 172.21.15.5:0/3049613649 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad8005ded0 0x55ad800602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.623+0000 7f35c7fff640 10 client.17682 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:12:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.623+0000 7f35eccc1640 1 --2- 172.21.15.5:0/3049613649 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad8005ded0 0x55ad800602c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8936344ce670dd8c server_cookie=933c7752008bd646 in_seq=0 out_seq=0 2024-09-19T06:12:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.624+0000 7f35c7fff640 1 -- 172.21.15.5:0/3049613649 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad7ffb6860 con 0x55ad8005ded0 2024-09-19T06:12:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.624+0000 7f35c7fff640 10 client.17682 handle_command_reply: tid=0 2024-09-19T06:12:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.624+0000 7f35f12fe740 1 client.17682 shutdown 2024-09-19T06:12:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.625+0000 7f35f12fe740 1 -- 172.21.15.5:0/3049613649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad8004e370 msgr2=0x55ad8004b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:07.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.625+0000 7f35f12fe740 1 --2- 172.21.15.5:0/3049613649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad8004e370 0x55ad8004b800 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009a60 tx=0x7f35e00043d0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.625+0000 7f35f12fe740 1 -- 172.21.15.5:0/3049613649 shutdown_connections 2024-09-19T06:12:07.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.625+0000 7f35f12fe740 1 --2- 172.21.15.5:0/3049613649 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad8005ded0 0x55ad800602c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.625+0000 7f35f12fe740 1 --2- 172.21.15.5:0/3049613649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad8004e370 0x55ad8004b800 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.625+0000 7f35f12fe740 1 --2- 172.21.15.5:0/3049613649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad80044b40 0x55ad8004b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.625+0000 7f35f12fe740 1 --2- 172.21.15.5:0/3049613649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad800435d0 0x55ad8004ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.625+0000 7f35f12fe740 1 -- 172.21.15.5:0/3049613649 >> 172.21.15.5:0/3049613649 conn(0x55ad8000c310 msgr2=0x55ad80045930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:07.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.625+0000 7f35c7fff640 0 client.17682 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:12:07.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.625+0000 7f35f12fe740 1 -- 172.21.15.5:0/3049613649 shutdown_connections 2024-09-19T06:12:07.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.625+0000 7f35f12fe740 1 -- 172.21.15.5:0/3049613649 wait complete. 2024-09-19T06:12:07.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.625+0000 7f35f12fe740 20 client.17682 trim_cache size 0 max 16384 2024-09-19T06:12:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35f12fe740 1 Processor -- start 2024-09-19T06:12:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35f12fe740 1 -- start start 2024-09-19T06:12:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35f12fe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad8004afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35f12fe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad8004a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35f12fe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 0x55ad80062f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35f12fe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81ad9d0 con 0x55ad800435d0 2024-09-19T06:12:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35f12fe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81af510 con 0x55ad80044b40 2024-09-19T06:12:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35f12fe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81ae0a0 con 0x55ad8004e370 2024-09-19T06:12:07.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35eccc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad8004afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35eccc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad8004afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43560/0 (socket says 172.21.15.5:43560) 2024-09-19T06:12:07.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35eccc1640 1 -- 172.21.15.5:0/4119059476 learned_addr learned my addr 172.21.15.5:0/4119059476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:07.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35ed4c2640 1 --2- 172.21.15.5:0/4119059476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 0x55ad80062f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35e7fff640 1 --2- 172.21.15.5:0/4119059476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad8004a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35eccc1640 1 -- 172.21.15.5:0/4119059476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 msgr2=0x55ad80062f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:07.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35eccc1640 1 --2- 172.21.15.5:0/4119059476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 0x55ad80062f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35eccc1640 1 -- 172.21.15.5:0/4119059476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 msgr2=0x55ad8004a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:07.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35eccc1640 1 --2- 172.21.15.5:0/4119059476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad8004a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35eccc1640 1 -- 172.21.15.5:0/4119059476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad8004e130 con 0x55ad800435d0 2024-09-19T06:12:07.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.640+0000 7f35e7fff640 1 --2- 172.21.15.5:0/4119059476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad8004a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:12:07.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.641+0000 7f35eccc1640 1 --2- 172.21.15.5:0/4119059476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad8004afa0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f35dc012700 tx=0x7f35dc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d3a062884113a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:07.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.641+0000 7f35d27fc640 1 -- 172.21.15.5:0/4119059476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc02c070 con 0x55ad800435d0 2024-09-19T06:12:07.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.641+0000 7f35d27fc640 1 -- 172.21.15.5:0/4119059476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35dc00fd50 con 0x55ad800435d0 2024-09-19T06:12:07.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.641+0000 7f35d27fc640 1 -- 172.21.15.5:0/4119059476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0049a0 con 0x55ad800435d0 2024-09-19T06:12:07.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.642+0000 7f35f12fe740 1 -- 172.21.15.5:0/4119059476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 msgr2=0x55ad8004afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:07.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.642+0000 7f35f12fe740 1 --2- 172.21.15.5:0/4119059476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad8004afa0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f35dc012700 tx=0x7f35dc005f90 comp rx=0 tx=0).stop 2024-09-19T06:12:07.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.642+0000 7f35f12fe740 1 -- 172.21.15.5:0/4119059476 shutdown_connections 2024-09-19T06:12:07.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.642+0000 7f35f12fe740 1 --2- 172.21.15.5:0/4119059476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 0x55ad80062f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.642+0000 7f35f12fe740 1 --2- 172.21.15.5:0/4119059476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad8004a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.642+0000 7f35f12fe740 1 --2- 172.21.15.5:0/4119059476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad8004afa0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.642+0000 7f35f12fe740 1 -- 172.21.15.5:0/4119059476 >> 172.21.15.5:0/4119059476 conn(0x55ad8000c310 msgr2=0x55ad80049940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:07.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.642+0000 7f35f12fe740 1 -- 172.21.15.5:0/4119059476 shutdown_connections 2024-09-19T06:12:07.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.642+0000 7f35f12fe740 1 -- 172.21.15.5:0/4119059476 wait complete. 2024-09-19T06:12:07.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.642+0000 7f35f12fe740 1 -- start start 2024-09-19T06:12:07.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.642+0000 7f35f12fe740 1 Processor -- start 2024-09-19T06:12:07.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35f12fe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad800633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35f12fe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad80060710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35f12fe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 0x55ad80060c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35f12fe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81aee40 con 0x55ad800435d0 2024-09-19T06:12:07.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35f12fe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81ae770 con 0x55ad80044b40 2024-09-19T06:12:07.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35f12fe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81ab9d0 con 0x55ad8004e370 2024-09-19T06:12:07.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35f12fe740 10 client.0 _ll_register_callbacks cb 0x7f35e004b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:07.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35eccc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad800633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35eccc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad800633c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43568/0 (socket says 172.21.15.5:43568) 2024-09-19T06:12:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35eccc1640 1 -- 172.21.15.5:0/2607576612 learned_addr learned my addr 172.21.15.5:0/2607576612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35e7fff640 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad80060710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35ed4c2640 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 0x55ad80060c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35eccc1640 1 -- 172.21.15.5:0/2607576612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 msgr2=0x55ad80060c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35eccc1640 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 0x55ad80060c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35eccc1640 1 -- 172.21.15.5:0/2607576612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 msgr2=0x55ad80060710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35eccc1640 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad80060710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35eccc1640 1 -- 172.21.15.5:0/2607576612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad800611c0 con 0x55ad800435d0 2024-09-19T06:12:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35e7fff640 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad80060710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35eccc1640 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad800633c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f35dc012620 tx=0x7f35dc012650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994f6f9ff7178d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.643+0000 7f35c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.644+0000 7f35c7fff640 1 -- 172.21.15.5:0/2607576612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc034020 con 0x55ad800435d0 2024-09-19T06:12:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.644+0000 7f35c7fff640 1 -- 172.21.15.5:0/2607576612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8594+0+0 (secure 0 0 0) 0x7f35dc00faa0 con 0x55ad800435d0 2024-09-19T06:12:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.644+0000 7f35f12fe740 1 -- 172.21.15.5:0/2607576612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35dc00fd50 con 0x55ad800435d0 2024-09-19T06:12:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.644+0000 7f35c7fff640 1 -- 172.21.15.5:0/2607576612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f35dc02c040 con 0x55ad800435d0 2024-09-19T06:12:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.644+0000 7f35f12fe740 10 client.17688 fetch_fsmap learned FSMap version 33 2024-09-19T06:12:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.644+0000 7f35f12fe740 1 -- 172.21.15.5:0/2607576612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ad800c84e0 con 0x55ad800435d0 2024-09-19T06:12:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.644+0000 7f35c7fff640 1 -- 172.21.15.5:0/2607576612 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35dc00fd50 con 0x55ad800435d0 2024-09-19T06:12:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.644+0000 7f35c7fff640 1 -- 172.21.15.5:0/2607576612 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f35dc0305d0 con 0x55ad800435d0 2024-09-19T06:12:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35f12fe740 10 client.17688 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:12:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35f12fe740 10 client.17688 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:12:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35f12fe740 20 client.17688 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:07.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35f12fe740 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad800c8bf0 0x55ad800cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:07.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35f12fe740 4 client.17688 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:12:07.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35f12fe740 1 -- 172.21.15.5:0/2607576612 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad7ffb0d70 con 0x55ad800c8bf0 2024-09-19T06:12:07.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35e7fff640 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad800c8bf0 0x55ad800cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:07.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35e7fff640 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad800c8bf0 0x55ad800cafe0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91d1a11fec0e545e server_cookie=21ced7671e049ec0 in_seq=0 out_seq=0 2024-09-19T06:12:07.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35c7fff640 10 client.17688 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:12:07.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35c7fff640 1 -- 172.21.15.5:0/2607576612 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad7ffb0d70 con 0x55ad800c8bf0 2024-09-19T06:12:07.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35c7fff640 10 client.17688 handle_command_reply: tid=0 2024-09-19T06:12:07.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35f12fe740 1 client.17688 shutdown 2024-09-19T06:12:07.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35f12fe740 1 -- 172.21.15.5:0/2607576612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 msgr2=0x55ad800633c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:07.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.645+0000 7f35f12fe740 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad800633c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f35dc012620 tx=0x7f35dc012650 comp rx=0 tx=0).stop 2024-09-19T06:12:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35f12fe740 1 -- 172.21.15.5:0/2607576612 shutdown_connections 2024-09-19T06:12:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35f12fe740 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad800c8bf0 0x55ad800cafe0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35f12fe740 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad8004e370 0x55ad80060c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35f12fe740 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad80044b40 0x55ad80060710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35f12fe740 1 --2- 172.21.15.5:0/2607576612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad800435d0 0x55ad800633c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35c7fff640 0 client.17688 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:12:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35f12fe740 1 -- 172.21.15.5:0/2607576612 >> 172.21.15.5:0/2607576612 conn(0x55ad8000c310 msgr2=0x55ad8004a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35f12fe740 1 -- 172.21.15.5:0/2607576612 shutdown_connections 2024-09-19T06:12:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35f12fe740 1 -- 172.21.15.5:0/2607576612 wait complete. 2024-09-19T06:12:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35f12fe740 20 client.17688 trim_cache size 0 max 16384 2024-09-19T06:12:07.658 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:12:07.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35eef25640 1 -- 172.21.15.5:0/1300090347 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35d80a3430 msgr2=0x7f35d80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:07.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35eef25640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35d80a3430 0x7f35d80a58f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f35e004a000 tx=0x7f35e004b000 comp rx=0 tx=0).stop 2024-09-19T06:12:07.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35eef25640 1 -- 172.21.15.5:0/1300090347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e8103760 msgr2=0x7f35e8198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:07.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35eef25640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e8103760 0x7f35e8198a80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f35e80697d0 tx=0x7f35d400ee50 comp rx=0 tx=0).stop 2024-09-19T06:12:07.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35eef25640 1 -- 172.21.15.5:0/1300090347 shutdown_connections 2024-09-19T06:12:07.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35eef25640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35d80a3430 0x7f35d80a58f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35eef25640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35e81082e0 0x7f35e819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35eef25640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e8103760 0x7f35e8198a80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35eef25640 1 --2- 172.21.15.5:0/1300090347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8102db0 0x7f35e8198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:07.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.646+0000 7f35eef25640 1 -- 172.21.15.5:0/1300090347 >> 172.21.15.5:0/1300090347 conn(0x7f35e80fc7d0 msgr2=0x7f35e8105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:07.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.647+0000 7f35eef25640 1 -- 172.21.15.5:0/1300090347 shutdown_connections 2024-09-19T06:12:07.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:07.647+0000 7f35eef25640 1 -- 172.21.15.5:0/1300090347 wait complete. 2024-09-19T06:12:07.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 228706 2.69 MB/sec execute 333 sec latency 37.195 ms 2024-09-19T06:12:08.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:12:08.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:08 smithi005 ceph-mon[30955]: pgmap v272: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-19T06:12:08.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:08 smithi005 ceph-mon[30955]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T06:12:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:08 smithi157 ceph-mon[35346]: pgmap v272: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-19T06:12:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:08 smithi157 ceph-mon[35346]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T06:12:08.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 229305 2.69 MB/sec execute 334 sec latency 20.458 ms 2024-09-19T06:12:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:08 smithi049 ceph-mon[34640]: pgmap v272: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-19T06:12:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:08 smithi049 ceph-mon[34640]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T06:12:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.183+0000 7f60fa77b640 1 -- 172.21.15.5:0/1415124839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60f41036e0 msgr2=0x7f60f4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.183+0000 7f60fa77b640 1 --2- 172.21.15.5:0/1415124839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60f41036e0 0x7f60f4107b90 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009930 tx=0x7f60e802a420 comp rx=0 tx=0).stop 2024-09-19T06:12:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.183+0000 7f60fa77b640 1 -- 172.21.15.5:0/1415124839 shutdown_connections 2024-09-19T06:12:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.183+0000 7f60fa77b640 1 --2- 172.21.15.5:0/1415124839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f41082a0 0x7f60f410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.183+0000 7f60fa77b640 1 --2- 172.21.15.5:0/1415124839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60f41036e0 0x7f60f4107b90 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.183+0000 7f60fa77b640 1 --2- 172.21.15.5:0/1415124839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f4102d30 0x7f60f4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.183+0000 7f60fa77b640 1 -- 172.21.15.5:0/1415124839 >> 172.21.15.5:0/1415124839 conn(0x7f60f40fc7d0 msgr2=0x7f60f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.183+0000 7f60fa77b640 1 -- 172.21.15.5:0/1415124839 shutdown_connections 2024-09-19T06:12:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.184+0000 7f60fa77b640 1 -- 172.21.15.5:0/1415124839 wait complete. 2024-09-19T06:12:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.184+0000 7f60fa77b640 1 Processor -- start 2024-09-19T06:12:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.184+0000 7f60fa77b640 1 -- start start 2024-09-19T06:12:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60fa77b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60f4102d30 0x7f60f4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60fa77b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f41036e0 0x7f60f4198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60fa77b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f41082a0 0x7f60f419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60fa77b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4111f10 con 0x7f60f4102d30 2024-09-19T06:12:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60fa77b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4111d90 con 0x7f60f41082a0 2024-09-19T06:12:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60fa77b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4112090 con 0x7f60f41036e0 2024-09-19T06:12:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60f4102d30 0x7f60f4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60f4102d30 0x7f60f4198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43586/0 (socket says 172.21.15.5:43586) 2024-09-19T06:12:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60f3fff640 1 -- 172.21.15.5:0/730820924 learned_addr learned my addr 172.21.15.5:0/730820924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60f8d18640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f41082a0 0x7f60f419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60f37fe640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f41036e0 0x7f60f4198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60f3fff640 1 -- 172.21.15.5:0/730820924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f41036e0 msgr2=0x7f60f4198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60f3fff640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f41036e0 0x7f60f4198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60f3fff640 1 -- 172.21.15.5:0/730820924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f41082a0 msgr2=0x7f60f419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60f3fff640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f41082a0 0x7f60f419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60f3fff640 1 -- 172.21.15.5:0/730820924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f419d4f0 con 0x7f60f4102d30 2024-09-19T06:12:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.185+0000 7f60f3fff640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60f4102d30 0x7f60f4198540 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f60e400e9f0 tx=0x7f60e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.186+0000 7f60f17fa640 1 -- 172.21.15.5:0/730820924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e400cd80 con 0x7f60f4102d30 2024-09-19T06:12:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.186+0000 7f60f17fa640 1 -- 172.21.15.5:0/730820924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60e40108e0 con 0x7f60f4102d30 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.186+0000 7f60fa77b640 1 -- 172.21.15.5:0/730820924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f419d7e0 con 0x7f60f4102d30 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.186+0000 7f60f17fa640 1 -- 172.21.15.5:0/730820924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4018920 con 0x7f60f4102d30 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.186+0000 7f60fa77b640 1 -- 172.21.15.5:0/730820924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f41008c0 con 0x7f60f4102d30 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.188+0000 7f60f17fa640 1 -- 172.21.15.5:0/730820924 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f60e4010a90 con 0x7f60f4102d30 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.188+0000 7f60fcb54740 1 Processor -- start 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60fcb54740 1 -- start start 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60fcb54740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea76f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60fcb54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea76fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60fcb54740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea77c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60fcb54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557eea6d52f0 con 0x557eea76f5d0 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60fcb54740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557eea67ccf0 con 0x557eea770b40 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60fcb54740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557eea6d12b0 con 0x557eea77a370 2024-09-19T06:12:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea76fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f8d18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea76f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f8d18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea76f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40612/0 (socket says 172.21.15.5:40612) 2024-09-19T06:12:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f8d18640 1 -- 172.21.15.5:0/2291611145 learned_addr learned my addr 172.21.15.5:0/2291611145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f37fe640 1 --2- 172.21.15.5:0/2291611145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea77c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f17fa640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60dc0a7840 0x7f60dc0a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f8d18640 1 -- 172.21.15.5:0/2291611145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 msgr2=0x557eea77c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f8d18640 1 --2- 172.21.15.5:0/2291611145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea77c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f8d18640 1 -- 172.21.15.5:0/2291611145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 msgr2=0x557eea76fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f8d18640 1 --2- 172.21.15.5:0/2291611145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea76fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f8d18640 1 -- 172.21.15.5:0/2291611145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eea6613a0 con 0x557eea770b40 2024-09-19T06:12:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f8d18640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60dc0a7840 0x7f60dc0a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.189+0000 7f60f17fa640 1 -- 172.21.15.5:0/730820924 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f60e40cffb0 con 0x7f60f4102d30 2024-09-19T06:12:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.190+0000 7f60f8d18640 1 --2- 172.21.15.5:0/2291611145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea76f090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f60ec011590 tx=0x7f60ec011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ca1897e5045b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.190+0000 7f60da7fc640 1 -- 172.21.15.5:0/2291611145 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec017050 con 0x557eea770b40 2024-09-19T06:12:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.190+0000 7f60f8d18640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60dc0a7840 0x7f60dc0a9d00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f60ec022000 tx=0x7f60ec023000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.190+0000 7f60da7fc640 1 -- 172.21.15.5:0/2291611145 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60ec01bb90 con 0x557eea770b40 2024-09-19T06:12:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.190+0000 7f60da7fc640 1 -- 172.21.15.5:0/2291611145 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec025de0 con 0x557eea770b40 2024-09-19T06:12:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.191+0000 7f60fcb54740 1 -- 172.21.15.5:0/2291611145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 msgr2=0x557eea76f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.191+0000 7f60fcb54740 1 --2- 172.21.15.5:0/2291611145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea76f090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f60ec011590 tx=0x7f60ec011a50 comp rx=0 tx=0).stop 2024-09-19T06:12:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.191+0000 7f60fcb54740 1 -- 172.21.15.5:0/2291611145 shutdown_connections 2024-09-19T06:12:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.191+0000 7f60fcb54740 1 --2- 172.21.15.5:0/2291611145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea77c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.191+0000 7f60fcb54740 1 --2- 172.21.15.5:0/2291611145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea76fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.191+0000 7f60fcb54740 1 --2- 172.21.15.5:0/2291611145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea76f090 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.191+0000 7f60fcb54740 1 -- 172.21.15.5:0/2291611145 >> 172.21.15.5:0/2291611145 conn(0x557eea738310 msgr2=0x557eea76bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:09.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.191+0000 7f60fcb54740 1 -- 172.21.15.5:0/2291611145 shutdown_connections 2024-09-19T06:12:09.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.191+0000 7f60fcb54740 1 -- 172.21.15.5:0/2291611145 wait complete. 2024-09-19T06:12:09.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.191+0000 7f60fcb54740 1 -- start start 2024-09-19T06:12:09.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.191+0000 7f60fcb54740 1 Processor -- start 2024-09-19T06:12:09.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60fcb54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea779d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60fcb54740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea777040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60fcb54740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea777650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60fcb54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557eea77d070 con 0x557eea76f5d0 2024-09-19T06:12:09.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60fcb54740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557eea690060 con 0x557eea770b40 2024-09-19T06:12:09.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60fcb54740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557eea77d1f0 con 0x557eea77a370 2024-09-19T06:12:09.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60fcb54740 10 client.0 _ll_register_callbacks cb 0x557eea3e38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:09.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea779d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea779d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43594/0 (socket says 172.21.15.5:43594) 2024-09-19T06:12:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60f3fff640 1 -- 172.21.15.5:0/15893066 learned_addr learned my addr 172.21.15.5:0/15893066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60f37fe640 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea777040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60f8d18640 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea777650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60f3fff640 1 -- 172.21.15.5:0/15893066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 msgr2=0x557eea777650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60f3fff640 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea777650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60f3fff640 1 -- 172.21.15.5:0/15893066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 msgr2=0x557eea777040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60f3fff640 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea777040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60f3fff640 1 -- 172.21.15.5:0/15893066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eea4463c0 con 0x557eea76f5d0 2024-09-19T06:12:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.192+0000 7f60f37fe640 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea777040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.193+0000 7f60f3fff640 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea779d90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f60e4098780 tx=0x7f60e40d36f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdaf8bcda94ca9e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.193+0000 7f60d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.193+0000 7f60d37fe640 1 -- 172.21.15.5:0/15893066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4010de0 con 0x557eea76f5d0 2024-09-19T06:12:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.193+0000 7f60d37fe640 1 -- 172.21.15.5:0/15893066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f60e40bc0e0 con 0x557eea76f5d0 2024-09-19T06:12:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.193+0000 7f60fcb54740 1 -- 172.21.15.5:0/15893066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60ec01bb90 con 0x557eea76f5d0 2024-09-19T06:12:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.193+0000 7f60d37fe640 1 -- 172.21.15.5:0/15893066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f60e406e1d0 con 0x557eea76f5d0 2024-09-19T06:12:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.193+0000 7f60fcb54740 10 client.17700 fetch_fsmap learned FSMap version 33 2024-09-19T06:12:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.193+0000 7f60fcb54740 1 -- 172.21.15.5:0/15893066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557eea788b40 con 0x557eea76f5d0 2024-09-19T06:12:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.193+0000 7f60d37fe640 1 -- 172.21.15.5:0/15893066 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60ec01bb90 con 0x557eea76f5d0 2024-09-19T06:12:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.193+0000 7f60d37fe640 1 -- 172.21.15.5:0/15893066 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f60e40a4a50 con 0x557eea76f5d0 2024-09-19T06:12:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.194+0000 7f60fcb54740 10 client.17700 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:12:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.194+0000 7f60fcb54740 10 client.17700 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:12:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.194+0000 7f60fcb54740 20 client.17700 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.194+0000 7f60fcb54740 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557eea789ef0 0x557eea78c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.194+0000 7f60fcb54740 4 client.17700 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.194+0000 7f60fcb54740 1 -- 172.21.15.5:0/15893066 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eea6e2860 con 0x557eea789ef0 2024-09-19T06:12:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.194+0000 7f60f37fe640 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557eea789ef0 0x557eea78c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.194+0000 7f60d37fe640 10 client.17700 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:12:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.194+0000 7f60f37fe640 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557eea789ef0 0x557eea78c320 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eac1fcba2069feb4 server_cookie=aa5c4e317ab4aabf in_seq=0 out_seq=0 2024-09-19T06:12:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60d37fe640 1 -- 172.21.15.5:0/15893066 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557eea6e2860 con 0x557eea789ef0 2024-09-19T06:12:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60d37fe640 10 client.17700 handle_command_reply: tid=0 2024-09-19T06:12:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60fcb54740 1 client.17700 shutdown 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60fcb54740 1 -- 172.21.15.5:0/15893066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 msgr2=0x557eea779d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60fcb54740 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea779d90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f60e4098780 tx=0x7f60e40d36f0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60fcb54740 1 -- 172.21.15.5:0/15893066 shutdown_connections 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60fcb54740 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557eea789ef0 0x557eea78c320 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60fcb54740 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea777650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60fcb54740 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea777040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60d37fe640 0 client.17700 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60fcb54740 1 --2- 172.21.15.5:0/15893066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea779d90 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60fcb54740 1 -- 172.21.15.5:0/15893066 >> 172.21.15.5:0/15893066 conn(0x557eea738310 msgr2=0x557eea771960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60fcb54740 1 -- 172.21.15.5:0/15893066 shutdown_connections 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60fcb54740 1 -- 172.21.15.5:0/15893066 wait complete. 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.195+0000 7f60fcb54740 20 client.17700 trim_cache size 0 max 16384 2024-09-19T06:12:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.212+0000 7f60fcb54740 1 Processor -- start 2024-09-19T06:12:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.212+0000 7f60fcb54740 1 -- start start 2024-09-19T06:12:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.213+0000 7f60fcb54740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea76f5d0 0x557eea775910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.213+0000 7f60fcb54740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea770b40 0x557eea772120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.213+0000 7f60fcb54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea77a370 0x557eea772660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.213+0000 7f60fcb54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41ab6f0 con 0x557eea77a370 2024-09-19T06:12:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.213+0000 7f60fcb54740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41ad230 con 0x557eea76f5d0 2024-09-19T06:12:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.213+0000 7f60fcb54740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41abdc0 con 0x557eea770b40 2024-09-19T06:12:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.213+0000 7f60f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea76f5d0 0x557eea775910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.213+0000 7f60f37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea770b40 0x557eea772120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.213+0000 7f60f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea76f5d0 0x557eea775910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40630/0 (socket says 172.21.15.5:40630) 2024-09-19T06:12:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.213+0000 7f60f3fff640 1 -- 172.21.15.5:0/3221207456 learned_addr learned my addr 172.21.15.5:0/3221207456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.213+0000 7f60f8d18640 1 --2- 172.21.15.5:0/3221207456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea77a370 0x557eea772660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.214+0000 7f60f3fff640 1 -- 172.21.15.5:0/3221207456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea770b40 msgr2=0x557eea772120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.214+0000 7f60f3fff640 1 --2- 172.21.15.5:0/3221207456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea770b40 0x557eea772120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.214+0000 7f60f3fff640 1 -- 172.21.15.5:0/3221207456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea77a370 msgr2=0x557eea772660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.214+0000 7f60f3fff640 1 --2- 172.21.15.5:0/3221207456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea77a370 0x557eea772660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.214+0000 7f60f3fff640 1 -- 172.21.15.5:0/3221207456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eea775ff0 con 0x557eea76f5d0 2024-09-19T06:12:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.214+0000 7f60f8d18640 1 --2- 172.21.15.5:0/3221207456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea77a370 0x557eea772660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.214+0000 7f60f3fff640 1 --2- 172.21.15.5:0/3221207456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea76f5d0 0x557eea775910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f60e409c2b0 tx=0x7f60e409b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb854a8149ac150 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.214+0000 7f60d9ffb640 1 -- 172.21.15.5:0/3221207456 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4010de0 con 0x557eea76f5d0 2024-09-19T06:12:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.214+0000 7f60d9ffb640 1 -- 172.21.15.5:0/3221207456 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60e409c320 con 0x557eea76f5d0 2024-09-19T06:12:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.214+0000 7f60d9ffb640 1 -- 172.21.15.5:0/3221207456 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e40bc070 con 0x557eea76f5d0 2024-09-19T06:12:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.215+0000 7f60fcb54740 1 -- 172.21.15.5:0/3221207456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea76f5d0 msgr2=0x557eea775910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.215+0000 7f60fcb54740 1 --2- 172.21.15.5:0/3221207456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea76f5d0 0x557eea775910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f60e409c2b0 tx=0x7f60e409b770 comp rx=0 tx=0).stop 2024-09-19T06:12:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.215+0000 7f60fcb54740 1 -- 172.21.15.5:0/3221207456 shutdown_connections 2024-09-19T06:12:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.215+0000 7f60fcb54740 1 --2- 172.21.15.5:0/3221207456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea77a370 0x557eea772660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.215+0000 7f60fcb54740 1 --2- 172.21.15.5:0/3221207456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea770b40 0x557eea772120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.215+0000 7f60fcb54740 1 --2- 172.21.15.5:0/3221207456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea76f5d0 0x557eea775910 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.215+0000 7f60fcb54740 1 -- 172.21.15.5:0/3221207456 >> 172.21.15.5:0/3221207456 conn(0x557eea738310 msgr2=0x557eea7790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.215+0000 7f60fcb54740 1 -- 172.21.15.5:0/3221207456 shutdown_connections 2024-09-19T06:12:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.215+0000 7f60fcb54740 1 -- 172.21.15.5:0/3221207456 wait complete. 2024-09-19T06:12:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.216+0000 7f60fcb54740 1 -- start start 2024-09-19T06:12:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.216+0000 7f60fcb54740 1 Processor -- start 2024-09-19T06:12:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.216+0000 7f60fcb54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea77e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.216+0000 7f60fcb54740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea77e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.216+0000 7f60fcb54740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea77ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.216+0000 7f60fcb54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41acb60 con 0x557eea76f5d0 2024-09-19T06:12:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.216+0000 7f60fcb54740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41ac490 con 0x557eea770b40 2024-09-19T06:12:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.216+0000 7f60fcb54740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41a96f0 con 0x557eea77a370 2024-09-19T06:12:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.216+0000 7f60fcb54740 10 client.0 _ll_register_callbacks cb 0x7f60e400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.216+0000 7f60f8d18640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea77ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea77e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60f8d18640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea77ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43428/0 (socket says 172.21.15.5:43428) 2024-09-19T06:12:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60f8d18640 1 -- 172.21.15.5:0/3394440334 learned_addr learned my addr 172.21.15.5:0/3394440334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea77e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60f8d18640 1 -- 172.21.15.5:0/3394440334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 msgr2=0x557eea77e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60f8d18640 1 --2- 172.21.15.5:0/3394440334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea77e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60f8d18640 1 -- 172.21.15.5:0/3394440334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 msgr2=0x557eea77e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60f8d18640 1 --2- 172.21.15.5:0/3394440334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea77e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60f8d18640 1 -- 172.21.15.5:0/3394440334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eea7fc690 con 0x557eea77a370 2024-09-19T06:12:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60f8d18640 1 --2- 172.21.15.5:0/3394440334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea77ef50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00ffd0 tx=0x7f60ec025f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73798707113b5bb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:12:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60d37fe640 1 -- 172.21.15.5:0/3394440334 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec02a040 con 0x557eea77a370 2024-09-19T06:12:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60fcb54740 1 -- 172.21.15.5:0/3394440334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60e409c320 con 0x557eea77a370 2024-09-19T06:12:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60d37fe640 1 -- 172.21.15.5:0/3394440334 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f60ec004bc0 con 0x557eea77a370 2024-09-19T06:12:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60d37fe640 1 -- 172.21.15.5:0/3394440334 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f60ec017020 con 0x557eea77a370 2024-09-19T06:12:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60fcb54740 10 client.27187 fetch_fsmap learned FSMap version 33 2024-09-19T06:12:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.217+0000 7f60fcb54740 1 -- 172.21.15.5:0/3394440334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557eea7fd9e0 con 0x557eea77a370 2024-09-19T06:12:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.218+0000 7f60d37fe640 1 -- 172.21.15.5:0/3394440334 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60e409c320 con 0x557eea77a370 2024-09-19T06:12:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.218+0000 7f60d37fe640 1 -- 172.21.15.5:0/3394440334 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f60ec038030 con 0x557eea77a370 2024-09-19T06:12:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.218+0000 7f60fcb54740 10 client.27187 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:12:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.218+0000 7f60fcb54740 10 client.27187 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:12:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.218+0000 7f60fcb54740 20 client.27187 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.218+0000 7f60fcb54740 1 --2- 172.21.15.5:0/3394440334 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557eea7fde30 0x557eea800220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.218+0000 7f60fcb54740 4 client.27187 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:12:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.218+0000 7f60fcb54740 1 -- 172.21.15.5:0/3394440334 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557eea6dcd70 con 0x557eea7fde30 2024-09-19T06:12:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.218+0000 7f60f3fff640 1 --2- 172.21.15.5:0/3394440334 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557eea7fde30 0x557eea800220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60f3fff640 1 --2- 172.21.15.5:0/3394440334 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557eea7fde30 0x557eea800220 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35969b7c8f1a61c1 server_cookie=a5c26e02688189cb in_seq=0 out_seq=0 2024-09-19T06:12:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60d37fe640 10 client.27187 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:12:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60d37fe640 1 -- 172.21.15.5:0/3394440334 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557eea6dcd70 con 0x557eea7fde30 2024-09-19T06:12:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60d37fe640 10 client.27187 handle_command_reply: tid=0 2024-09-19T06:12:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60fcb54740 1 client.27187 shutdown 2024-09-19T06:12:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60fcb54740 1 -- 172.21.15.5:0/3394440334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 msgr2=0x557eea77ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60fcb54740 1 --2- 172.21.15.5:0/3394440334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea77ef50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00ffd0 tx=0x7f60ec025f80 comp rx=0 tx=0).stop 2024-09-19T06:12:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60fcb54740 1 -- 172.21.15.5:0/3394440334 shutdown_connections 2024-09-19T06:12:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60fcb54740 1 --2- 172.21.15.5:0/3394440334 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557eea7fde30 0x557eea800220 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60fcb54740 1 --2- 172.21.15.5:0/3394440334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557eea77a370 0x557eea77ef50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60fcb54740 1 --2- 172.21.15.5:0/3394440334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557eea770b40 0x557eea77e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60d37fe640 0 client.27187 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:12:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60fcb54740 1 --2- 172.21.15.5:0/3394440334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eea76f5d0 0x557eea77e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60fcb54740 1 -- 172.21.15.5:0/3394440334 >> 172.21.15.5:0/3394440334 conn(0x557eea738310 msgr2=0x557eea7790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60fcb54740 1 -- 172.21.15.5:0/3394440334 shutdown_connections 2024-09-19T06:12:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.219+0000 7f60fcb54740 1 -- 172.21.15.5:0/3394440334 wait complete. 2024-09-19T06:12:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.220+0000 7f60fcb54740 20 client.27187 trim_cache size 0 max 16384 2024-09-19T06:12:09.237 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:12:09.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.220+0000 7f60fa77b640 1 -- 172.21.15.5:0/730820924 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60dc0a7840 msgr2=0x7f60dc0a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:09.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.220+0000 7f60fa77b640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60dc0a7840 0x7f60dc0a9d00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f60ec022000 tx=0x7f60ec023000 comp rx=0 tx=0).stop 2024-09-19T06:12:09.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.220+0000 7f60fa77b640 1 -- 172.21.15.5:0/730820924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60f4102d30 msgr2=0x7f60f4198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:09.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.220+0000 7f60fa77b640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60f4102d30 0x7f60f4198540 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f60e400e9f0 tx=0x7f60e400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.220+0000 7f60fa77b640 1 -- 172.21.15.5:0/730820924 shutdown_connections 2024-09-19T06:12:09.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.220+0000 7f60fa77b640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f41082a0 0x7f60f419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.220+0000 7f60fa77b640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60dc0a7840 0x7f60dc0a9d00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.220+0000 7f60fa77b640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f41036e0 0x7f60f4198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.220+0000 7f60fa77b640 1 --2- 172.21.15.5:0/730820924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60f4102d30 0x7f60f4198540 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.220+0000 7f60fa77b640 1 -- 172.21.15.5:0/730820924 >> 172.21.15.5:0/730820924 conn(0x7f60f40fc7d0 msgr2=0x7f60f4105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.220+0000 7f60fa77b640 1 -- 172.21.15.5:0/730820924 shutdown_connections 2024-09-19T06:12:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:09.221+0000 7f60fa77b640 1 -- 172.21.15.5:0/730820924 wait complete. 2024-09-19T06:12:09.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 229834 2.69 MB/sec execute 335 sec latency 14.854 ms 2024-09-19T06:12:09.917 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:12:10.197 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:10.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:10 smithi005 ceph-mon[30955]: pgmap v274: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.0 MiB/s wr, 211 op/s 2024-09-19T06:12:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.724+0000 7f554f823640 1 -- 172.21.15.5:0/1761543204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55481082a0 msgr2=0x7f554810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.724+0000 7f554f823640 1 --2- 172.21.15.5:0/1761543204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55481082a0 0x7f554810ec40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5540009cd0 tx=0x7f554002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.724+0000 7f554f823640 1 -- 172.21.15.5:0/1761543204 shutdown_connections 2024-09-19T06:12:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.724+0000 7f554f823640 1 --2- 172.21.15.5:0/1761543204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55481082a0 0x7f554810ec40 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.724+0000 7f554f823640 1 --2- 172.21.15.5:0/1761543204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55481036e0 0x7f5548107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.724+0000 7f554f823640 1 --2- 172.21.15.5:0/1761543204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5548102d30 0x7f5548103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.724+0000 7f554f823640 1 -- 172.21.15.5:0/1761543204 >> 172.21.15.5:0/1761543204 conn(0x7f55480fc7d0 msgr2=0x7f55480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.724+0000 7f554f823640 1 -- 172.21.15.5:0/1761543204 shutdown_connections 2024-09-19T06:12:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.724+0000 7f554f823640 1 -- 172.21.15.5:0/1761543204 wait complete. 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.725+0000 7f554f823640 1 Processor -- start 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.725+0000 7f554f823640 1 -- start start 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.725+0000 7f554f823640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5548102d30 0x7f55481984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.725+0000 7f554f823640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55481036e0 0x7f55481989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.725+0000 7f554f823640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55481082a0 0x7f554819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.725+0000 7f554f823640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5548111f50 con 0x7f55481036e0 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.725+0000 7f554f823640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5548111dd0 con 0x7f5548102d30 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.725+0000 7f554f823640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f55481120d0 con 0x7f55481082a0 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.725+0000 7f554d5bf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5548102d30 0x7f55481984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f554cdbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55481036e0 0x7f55481989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f554d5bf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5548102d30 0x7f55481984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40674/0 (socket says 172.21.15.5:40674) 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f554d5bf640 1 -- 172.21.15.5:0/2253974922 learned_addr learned my addr 172.21.15.5:0/2253974922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f554ddc0640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55481082a0 0x7f554819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f554cdbe640 1 -- 172.21.15.5:0/2253974922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55481082a0 msgr2=0x7f554819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f554cdbe640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55481082a0 0x7f554819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f554cdbe640 1 -- 172.21.15.5:0/2253974922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5548102d30 msgr2=0x7f55481984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f554cdbe640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5548102d30 0x7f55481984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f554cdbe640 1 -- 172.21.15.5:0/2253974922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554819d450 con 0x7f55481036e0 2024-09-19T06:12:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f554cdbe640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55481036e0 0x7f55481989e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f55480697d0 tx=0x7f553800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f553e7fc640 1 -- 172.21.15.5:0/2253974922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553800cc40 con 0x7f55481036e0 2024-09-19T06:12:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f553e7fc640 1 -- 172.21.15.5:0/2253974922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f553800cde0 con 0x7f55481036e0 2024-09-19T06:12:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f554f823640 1 -- 172.21.15.5:0/2253974922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554819d740 con 0x7f55481036e0 2024-09-19T06:12:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f554f823640 1 -- 172.21.15.5:0/2253974922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55481a5020 con 0x7f55481036e0 2024-09-19T06:12:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.726+0000 7f553e7fc640 1 -- 172.21.15.5:0/2253974922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538004a00 con 0x7f55481036e0 2024-09-19T06:12:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.729+0000 7f5551bfc740 1 Processor -- start 2024-09-19T06:12:10.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.729+0000 7f553e7fc640 1 -- 172.21.15.5:0/2253974922 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5538004ba0 con 0x7f55481036e0 2024-09-19T06:12:10.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.729+0000 7f5551bfc740 1 -- start start 2024-09-19T06:12:10.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.729+0000 7f5551bfc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d5b40 0x55e0e04d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f5551bfc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 0x55e0e04d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f5551bfc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04e1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f5551bfc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e0e043a2f0 con 0x55e0e04d45d0 2024-09-19T06:12:10.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f5551bfc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e0e03e1cf0 con 0x55e0e04df370 2024-09-19T06:12:10.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f5551bfc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e0e04362b0 con 0x55e0e04d5b40 2024-09-19T06:12:10.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f554d5bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 0x55e0e04d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f554d5bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 0x55e0e04d4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43638/0 (socket says 172.21.15.5:43638) 2024-09-19T06:12:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f554d5bf640 1 -- 172.21.15.5:0/4231361785 learned_addr learned my addr 172.21.15.5:0/4231361785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f554cdbe640 1 --2- 172.21.15.5:0/4231361785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f554d5bf640 1 -- 172.21.15.5:0/4231361785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d5b40 msgr2=0x55e0e04d4090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:12:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f554d5bf640 1 --2- 172.21.15.5:0/4231361785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d5b40 0x55e0e04d4090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f554d5bf640 1 -- 172.21.15.5:0/4231361785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 msgr2=0x55e0e04e1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f554d5bf640 1 --2- 172.21.15.5:0/4231361785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04e1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f554d5bf640 1 -- 172.21.15.5:0/4231361785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e03c63a0 con 0x55e0e04d45d0 2024-09-19T06:12:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f553e7fc640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f55340a3430 0x7f55340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f553e7fc640 1 -- 172.21.15.5:0/2253974922 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f55380cfb70 con 0x7f55481036e0 2024-09-19T06:12:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f554d5bf640 1 --2- 172.21.15.5:0/4231361785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 0x55e0e04d4a10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5544009dc0 tx=0x7f5544011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b09edf945a6888b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.730+0000 7f554ddc0640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f55340a3430 0x7f55340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.731+0000 7f55337fe640 1 -- 172.21.15.5:0/4231361785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5544005220 con 0x55e0e04d45d0 2024-09-19T06:12:10.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.731+0000 7f55337fe640 1 -- 172.21.15.5:0/4231361785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5544004c50 con 0x55e0e04d45d0 2024-09-19T06:12:10.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.731+0000 7f55337fe640 1 -- 172.21.15.5:0/4231361785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55440167b0 con 0x55e0e04d45d0 2024-09-19T06:12:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.732+0000 7f5551bfc740 1 -- 172.21.15.5:0/4231361785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 msgr2=0x55e0e04d4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.732+0000 7f554ddc0640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f55340a3430 0x7f55340a58f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5540005e10 tx=0x7f5540005da0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.732+0000 7f5551bfc740 1 --2- 172.21.15.5:0/4231361785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 0x55e0e04d4a10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5544009dc0 tx=0x7f5544011810 comp rx=0 tx=0).stop 2024-09-19T06:12:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.732+0000 7f5551bfc740 1 -- 172.21.15.5:0/4231361785 shutdown_connections 2024-09-19T06:12:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.732+0000 7f5551bfc740 1 --2- 172.21.15.5:0/4231361785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04e1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.732+0000 7f5551bfc740 1 --2- 172.21.15.5:0/4231361785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 0x55e0e04d4a10 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.732+0000 7f5551bfc740 1 --2- 172.21.15.5:0/4231361785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d5b40 0x55e0e04d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.732+0000 7f5551bfc740 1 -- 172.21.15.5:0/4231361785 >> 172.21.15.5:0/4231361785 conn(0x55e0e049d310 msgr2=0x55e0e04d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.732+0000 7f5551bfc740 1 -- 172.21.15.5:0/4231361785 shutdown_connections 2024-09-19T06:12:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.732+0000 7f5551bfc740 1 -- 172.21.15.5:0/4231361785 wait complete. 2024-09-19T06:12:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.732+0000 7f5551bfc740 1 -- start start 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.732+0000 7f5551bfc740 1 Processor -- start 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f5551bfc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d45d0 0x55e0e04ded10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f5551bfc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d5b40 0x55e0e04dbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f5551bfc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04dc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f5551bfc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e0e04e20c0 con 0x55e0e04d5b40 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f5551bfc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e0e03f5060 con 0x55e0e04df370 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f5551bfc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e0e04e2240 con 0x55e0e04d45d0 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f5551bfc740 10 client.0 _ll_register_callbacks cb 0x55e0e01488c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f554cdbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d5b40 0x55e0e04dbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f554cdbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d5b40 0x55e0e04dbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43652/0 (socket says 172.21.15.5:43652) 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f554cdbe640 1 -- 172.21.15.5:0/3938127114 learned_addr learned my addr 172.21.15.5:0/3938127114 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f554ddc0640 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04dc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f554d5bf640 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d45d0 0x55e0e04ded10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f554cdbe640 1 -- 172.21.15.5:0/3938127114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d45d0 msgr2=0x55e0e04ded10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f554cdbe640 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d45d0 0x55e0e04ded10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f554cdbe640 1 -- 172.21.15.5:0/3938127114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 msgr2=0x55e0e04dc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f554cdbe640 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04dc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f554cdbe640 1 -- 172.21.15.5:0/3938127114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e01ab3c0 con 0x55e0e04d5b40 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.733+0000 7f554ddc0640 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04dc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.734+0000 7f5530ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.734+0000 7f554cdbe640 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d5b40 0x55e0e04dbfc0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f553809bb30 tx=0x7f55380d32b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def4cc0eb7321e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.734+0000 7f5530ff9640 1 -- 172.21.15.5:0/3938127114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553801c070 con 0x55e0e04d5b40 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.734+0000 7f5530ff9640 1 -- 172.21.15.5:0/3938127114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f55380984a0 con 0x55e0e04d5b40 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.734+0000 7f5551bfc740 1 -- 172.21.15.5:0/3938127114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5544004c50 con 0x55e0e04d5b40 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.734+0000 7f5530ff9640 1 -- 172.21.15.5:0/3938127114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f553809bc00 con 0x55e0e04d5b40 2024-09-19T06:12:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.734+0000 7f5551bfc740 10 client.17718 fetch_fsmap learned FSMap version 34 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.734+0000 7f5551bfc740 1 -- 172.21.15.5:0/3938127114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e0e04edb70 con 0x55e0e04d5b40 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.734+0000 7f5530ff9640 1 -- 172.21.15.5:0/3938127114 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5544004c50 con 0x55e0e04d5b40 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.734+0000 7f5530ff9640 1 -- 172.21.15.5:0/3938127114 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f55380a4610 con 0x55e0e04d5b40 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.734+0000 7f5551bfc740 10 client.17718 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.735+0000 7f5551bfc740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.735+0000 7f5551bfc740 20 client.17718 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.735+0000 7f5551bfc740 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e0e04eeec0 0x55e0e04f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.735+0000 7f5551bfc740 4 client.17718 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.735+0000 7f5551bfc740 1 -- 172.21.15.5:0/3938127114 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0e0447860 con 0x55e0e04eeec0 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.735+0000 7f554d5bf640 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e0e04eeec0 0x55e0e04f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.735+0000 7f5530ff9640 10 client.17718 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.735+0000 7f554d5bf640 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e0e04eeec0 0x55e0e04f12f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb9dadd08ba12a7 server_cookie=d7d0ede14c22958c in_seq=0 out_seq=0 2024-09-19T06:12:10.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5530ff9640 1 -- 172.21.15.5:0/3938127114 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0e0447860 con 0x55e0e04eeec0 2024-09-19T06:12:10.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5530ff9640 10 client.17718 handle_command_reply: tid=0 2024-09-19T06:12:10.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5551bfc740 1 client.17718 shutdown 2024-09-19T06:12:10.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5551bfc740 1 -- 172.21.15.5:0/3938127114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d5b40 msgr2=0x55e0e04dbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:10.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5551bfc740 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d5b40 0x55e0e04dbfc0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f553809bb30 tx=0x7f55380d32b0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5551bfc740 1 -- 172.21.15.5:0/3938127114 shutdown_connections 2024-09-19T06:12:10.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5551bfc740 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e0e04eeec0 0x55e0e04f12f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5551bfc740 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04dc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5530ff9640 0 client.17718 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:10.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5551bfc740 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d5b40 0x55e0e04dbfc0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5551bfc740 1 --2- 172.21.15.5:0/3938127114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d45d0 0x55e0e04ded10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5551bfc740 1 -- 172.21.15.5:0/3938127114 >> 172.21.15.5:0/3938127114 conn(0x55e0e049d310 msgr2=0x55e0e04d6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:10.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5551bfc740 1 -- 172.21.15.5:0/3938127114 shutdown_connections 2024-09-19T06:12:10.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.736+0000 7f5551bfc740 1 -- 172.21.15.5:0/3938127114 wait complete. 2024-09-19T06:12:10.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.737+0000 7f5551bfc740 20 client.17718 trim_cache size 0 max 16384 2024-09-19T06:12:10.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f5551bfc740 1 Processor -- start 2024-09-19T06:12:10.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f5551bfc740 1 -- start start 2024-09-19T06:12:10.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f5551bfc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 0x55e0e04eb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f5551bfc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d5b40 0x55e0e04d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f5551bfc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04d7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f5551bfc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55481ab6d0 con 0x55e0e04d45d0 2024-09-19T06:12:10.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f5551bfc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55481ad210 con 0x55e0e04df370 2024-09-19T06:12:10.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f5551bfc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f55481abda0 con 0x55e0e04d5b40 2024-09-19T06:12:10.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f554d5bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 0x55e0e04eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f554d5bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 0x55e0e04eb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43664/0 (socket says 172.21.15.5:43664) 2024-09-19T06:12:10.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f554d5bf640 1 -- 172.21.15.5:0/1614268802 learned_addr learned my addr 172.21.15.5:0/1614268802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:10.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f554ddc0640 1 --2- 172.21.15.5:0/1614268802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04d7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f554cdbe640 1 --2- 172.21.15.5:0/1614268802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d5b40 0x55e0e04d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f554d5bf640 1 -- 172.21.15.5:0/1614268802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d5b40 msgr2=0x55e0e04d7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:10.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f554d5bf640 1 --2- 172.21.15.5:0/1614268802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d5b40 0x55e0e04d7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f554d5bf640 1 -- 172.21.15.5:0/1614268802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 msgr2=0x55e0e04d7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:10.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f554d5bf640 1 --2- 172.21.15.5:0/1614268802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04d7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.751+0000 7f554d5bf640 1 -- 172.21.15.5:0/1614268802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e04da510 con 0x55e0e04d45d0 2024-09-19T06:12:10.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.752+0000 7f554d5bf640 1 --2- 172.21.15.5:0/1614268802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 0x55e0e04eb360 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5544000c00 tx=0x7f5544011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d5f149a252bce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:10.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.752+0000 7f5532ffd640 1 -- 172.21.15.5:0/1614268802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5544004240 con 0x55e0e04d45d0 2024-09-19T06:12:10.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.752+0000 7f5532ffd640 1 -- 172.21.15.5:0/1614268802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55440043e0 con 0x55e0e04d45d0 2024-09-19T06:12:10.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.752+0000 7f5532ffd640 1 -- 172.21.15.5:0/1614268802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554402f6e0 con 0x55e0e04d45d0 2024-09-19T06:12:10.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.752+0000 7f5551bfc740 1 -- 172.21.15.5:0/1614268802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 msgr2=0x55e0e04eb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:10.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.752+0000 7f5551bfc740 1 --2- 172.21.15.5:0/1614268802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 0x55e0e04eb360 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5544000c00 tx=0x7f5544011e10 comp rx=0 tx=0).stop 2024-09-19T06:12:10.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.752+0000 7f5551bfc740 1 -- 172.21.15.5:0/1614268802 shutdown_connections 2024-09-19T06:12:10.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.752+0000 7f5551bfc740 1 --2- 172.21.15.5:0/1614268802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04df370 0x55e0e04d7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.752+0000 7f5551bfc740 1 --2- 172.21.15.5:0/1614268802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d5b40 0x55e0e04d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.752+0000 7f5551bfc740 1 --2- 172.21.15.5:0/1614268802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04d45d0 0x55e0e04eb360 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.752+0000 7f5551bfc740 1 -- 172.21.15.5:0/1614268802 >> 172.21.15.5:0/1614268802 conn(0x55e0e049d310 msgr2=0x55e0e04db0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:10.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.753+0000 7f5551bfc740 1 -- 172.21.15.5:0/1614268802 shutdown_connections 2024-09-19T06:12:10.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.753+0000 7f5551bfc740 1 -- 172.21.15.5:0/1614268802 wait complete. 2024-09-19T06:12:10.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.753+0000 7f5551bfc740 1 -- start start 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.753+0000 7f5551bfc740 1 Processor -- start 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f5551bfc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d45d0 0x55e0e04e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f5551bfc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04d5b40 0x55e0e04d8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f5551bfc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04df370 0x55e0e04d9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f5551bfc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55481acb40 con 0x55e0e04df370 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f5551bfc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55481ac470 con 0x55e0e04d5b40 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f5551bfc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f55481a96d0 con 0x55e0e04d45d0 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f554ddc0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04df370 0x55e0e04d9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f5551bfc740 10 client.0 _ll_register_callbacks cb 0x7f55380d33b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f554cdbe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04d5b40 0x55e0e04d8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f554ddc0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04df370 0x55e0e04d9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43670/0 (socket says 172.21.15.5:43670) 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f554ddc0640 1 -- 172.21.15.5:0/4102811460 learned_addr learned my addr 172.21.15.5:0/4102811460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f554d5bf640 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d45d0 0x55e0e04e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f554ddc0640 1 -- 172.21.15.5:0/4102811460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d45d0 msgr2=0x55e0e04e7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f554ddc0640 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d45d0 0x55e0e04e7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f554ddc0640 1 -- 172.21.15.5:0/4102811460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04d5b40 msgr2=0x55e0e04d8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f554ddc0640 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04d5b40 0x55e0e04d8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f554ddc0640 1 -- 172.21.15.5:0/4102811460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e04d9b30 con 0x55e0e04df370 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f554d5bf640 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d45d0 0x55e0e04e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f554ddc0640 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04df370 0x55e0e04d9300 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5540009cd0 tx=0x7f554003e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bdf05242e32269 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.754+0000 7f5530ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.755+0000 7f5530ff9640 1 -- 172.21.15.5:0/4102811460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540057060 con 0x55e0e04df370 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.755+0000 7f5530ff9640 1 -- 172.21.15.5:0/4102811460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f5540004710 con 0x55e0e04df370 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.755+0000 7f5551bfc740 1 -- 172.21.15.5:0/4102811460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55440043e0 con 0x55e0e04df370 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.755+0000 7f5530ff9640 1 -- 172.21.15.5:0/4102811460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f55400305a0 con 0x55e0e04df370 2024-09-19T06:12:10.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.755+0000 7f5551bfc740 10 client.17724 fetch_fsmap learned FSMap version 34 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.755+0000 7f5551bfc740 1 -- 172.21.15.5:0/4102811460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e0e0559c30 con 0x55e0e04df370 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.755+0000 7f5530ff9640 1 -- 172.21.15.5:0/4102811460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55440043e0 con 0x55e0e04df370 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.755+0000 7f5530ff9640 1 -- 172.21.15.5:0/4102811460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f5540030070 con 0x55e0e04df370 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.755+0000 7f5551bfc740 10 client.17724 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.755+0000 7f5551bfc740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.755+0000 7f5551bfc740 20 client.17724 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.756+0000 7f5551bfc740 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e0e04eeec0 0x55e0e055c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.756+0000 7f5551bfc740 4 client.17724 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.756+0000 7f5551bfc740 1 -- 172.21.15.5:0/4102811460 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0e0441d70 con 0x55e0e04eeec0 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.756+0000 7f554d5bf640 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e0e04eeec0 0x55e0e055c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.756+0000 7f554d5bf640 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e0e04eeec0 0x55e0e055c3e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b60007e6a2d48f server_cookie=b2c868edbba2e360 in_seq=0 out_seq=0 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.756+0000 7f5530ff9640 10 client.17724 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:10.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.756+0000 7f5530ff9640 1 -- 172.21.15.5:0/4102811460 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0e0441d70 con 0x55e0e04eeec0 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.756+0000 7f5530ff9640 10 client.17724 handle_command_reply: tid=0 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.756+0000 7f5551bfc740 1 client.17724 shutdown 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.757+0000 7f5551bfc740 1 -- 172.21.15.5:0/4102811460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04df370 msgr2=0x55e0e04d9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.757+0000 7f5551bfc740 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04df370 0x55e0e04d9300 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5540009cd0 tx=0x7f554003e040 comp rx=0 tx=0).stop 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.757+0000 7f5551bfc740 1 -- 172.21.15.5:0/4102811460 shutdown_connections 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.757+0000 7f5551bfc740 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e0e04eeec0 0x55e0e055c3e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.757+0000 7f5551bfc740 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e04df370 0x55e0e04d9300 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.757+0000 7f5551bfc740 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0e04d5b40 0x55e0e04d8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.757+0000 7f5530ff9640 0 client.17724 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.757+0000 7f5551bfc740 1 --2- 172.21.15.5:0/4102811460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0e04d45d0 0x55e0e04e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.757+0000 7f5551bfc740 1 -- 172.21.15.5:0/4102811460 >> 172.21.15.5:0/4102811460 conn(0x55e0e049d310 msgr2=0x55e0e04dd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.757+0000 7f5551bfc740 1 -- 172.21.15.5:0/4102811460 shutdown_connections 2024-09-19T06:12:10.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.757+0000 7f5551bfc740 1 -- 172.21.15.5:0/4102811460 wait complete. 2024-09-19T06:12:10.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.757+0000 7f5551bfc740 20 client.17724 trim_cache size 0 max 16384 2024-09-19T06:12:10.766 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:12:10.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.758+0000 7f554f823640 1 -- 172.21.15.5:0/2253974922 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f55340a3430 msgr2=0x7f55340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:10.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.758+0000 7f554f823640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f55340a3430 0x7f55340a58f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5540005e10 tx=0x7f5540005da0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.758+0000 7f554f823640 1 -- 172.21.15.5:0/2253974922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55481036e0 msgr2=0x7f55481989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:10.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.758+0000 7f554f823640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55481036e0 0x7f55481989e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f55480697d0 tx=0x7f553800ee50 comp rx=0 tx=0).stop 2024-09-19T06:12:10.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.758+0000 7f554f823640 1 -- 172.21.15.5:0/2253974922 shutdown_connections 2024-09-19T06:12:10.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.758+0000 7f554f823640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f55340a3430 0x7f55340a58f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.758+0000 7f554f823640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55481082a0 0x7f554819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.758+0000 7f554f823640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55481036e0 0x7f55481989e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.758+0000 7f554f823640 1 --2- 172.21.15.5:0/2253974922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5548102d30 0x7f55481984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:10.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.758+0000 7f554f823640 1 -- 172.21.15.5:0/2253974922 >> 172.21.15.5:0/2253974922 conn(0x7f55480fc7d0 msgr2=0x7f5548105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:10.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.758+0000 7f554f823640 1 -- 172.21.15.5:0/2253974922 shutdown_connections 2024-09-19T06:12:10.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:10.758+0000 7f554f823640 1 -- 172.21.15.5:0/2253974922 wait complete. 2024-09-19T06:12:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:10 smithi157 ceph-mon[35346]: pgmap v274: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.0 MiB/s wr, 211 op/s 2024-09-19T06:12:10.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 230386 2.70 MB/sec execute 336 sec latency 20.712 ms 2024-09-19T06:12:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:10 smithi049 ceph-mon[34640]: pgmap v274: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.0 MiB/s wr, 211 op/s 2024-09-19T06:12:11.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:11 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:12:11.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:11 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:12:11.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 230672 2.70 MB/sec execute 337 sec latency 524.219 ms 2024-09-19T06:12:11.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:11 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:12:12.577 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:12:12.577 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:12:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:12 smithi157 ceph-mon[35346]: pgmap v275: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-19T06:12:12.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 230672 2.69 MB/sec execute 338 sec latency 1524.269 ms 2024-09-19T06:12:12.850 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:12 smithi005 ceph-mon[30955]: pgmap v275: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-19T06:12:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:12 smithi049 ceph-mon[34640]: pgmap v275: 97 pgs: 97 active+clean; 125 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-19T06:12:13.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.512+0000 7f6090245640 1 -- 172.21.15.5:0/833494336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6088106a80 msgr2=0x7f6088106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:13.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.512+0000 7f6090245640 1 --2- 172.21.15.5:0/833494336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6088106a80 0x7f6088106e60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6084009930 tx=0x7f608402a420 comp rx=0 tx=0).stop 2024-09-19T06:12:13.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.512+0000 7f6090245640 1 -- 172.21.15.5:0/833494336 shutdown_connections 2024-09-19T06:12:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.512+0000 7f6090245640 1 --2- 172.21.15.5:0/833494336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6088101420 0x7f608810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.512+0000 7f6090245640 1 --2- 172.21.15.5:0/833494336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6088100a80 0x7f6088100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.512+0000 7f6090245640 1 --2- 172.21.15.5:0/833494336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6088106a80 0x7f6088106e60 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.512+0000 7f6090245640 1 -- 172.21.15.5:0/833494336 >> 172.21.15.5:0/833494336 conn(0x7f60880fc7a0 msgr2=0x7f60880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.512+0000 7f6090245640 1 -- 172.21.15.5:0/833494336 shutdown_connections 2024-09-19T06:12:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.512+0000 7f6090245640 1 -- 172.21.15.5:0/833494336 wait complete. 2024-09-19T06:12:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.512+0000 7f6090245640 1 Processor -- start 2024-09-19T06:12:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.512+0000 7f6090245640 1 -- start start 2024-09-19T06:12:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f6090245640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6088100a80 0x7f6088102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f6090245640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6088101420 0x7f6088102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f6090245640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6088106a80 0x7f60881030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f6090245640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6088110d30 con 0x7f6088106a80 2024-09-19T06:12:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f6090245640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6088110bb0 con 0x7f6088101420 2024-09-19T06:12:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f6090245640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6088110eb0 con 0x7f6088100a80 2024-09-19T06:12:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f608e7e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6088106a80 0x7f60881030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f608e7e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6088106a80 0x7f60881030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43696/0 (socket says 172.21.15.5:43696) 2024-09-19T06:12:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f608e7e2640 1 -- 172.21.15.5:0/1696092551 learned_addr learned my addr 172.21.15.5:0/1696092551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f608d7e0640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6088101420 0x7f6088102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f608dfe1640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6088100a80 0x7f6088102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f608e7e2640 1 -- 172.21.15.5:0/1696092551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6088100a80 msgr2=0x7f6088102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f608e7e2640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6088100a80 0x7f6088102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f608e7e2640 1 -- 172.21.15.5:0/1696092551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6088101420 msgr2=0x7f6088102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f608e7e2640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6088101420 0x7f6088102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f608e7e2640 1 -- 172.21.15.5:0/1696092551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60881a6800 con 0x7f6088106a80 2024-09-19T06:12:13.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f608e7e2640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6088106a80 0x7f60881030d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f608000cce0 tx=0x7f608000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:13.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.513+0000 7f607effd640 1 -- 172.21.15.5:0/1696092551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608000f450 con 0x7f6088106a80 2024-09-19T06:12:13.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.514+0000 7f607effd640 1 -- 172.21.15.5:0/1696092551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f608000faa0 con 0x7f6088106a80 2024-09-19T06:12:13.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.514+0000 7f6090245640 1 -- 172.21.15.5:0/1696092551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60881a6af0 con 0x7f6088106a80 2024-09-19T06:12:13.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.514+0000 7f607effd640 1 -- 172.21.15.5:0/1696092551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080018b70 con 0x7f6088106a80 2024-09-19T06:12:13.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.514+0000 7f6090245640 1 -- 172.21.15.5:0/1696092551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60881a7030 con 0x7f6088106a80 2024-09-19T06:12:13.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.516+0000 7f607effd640 1 -- 172.21.15.5:0/1696092551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6080018d10 con 0x7f6088106a80 2024-09-19T06:12:13.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.516+0000 7f609261e740 1 Processor -- start 2024-09-19T06:12:13.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f609261e740 1 -- start start 2024-09-19T06:12:13.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f607effd640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60740a3430 0x7f60740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f609261e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa1ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f609261e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 0x55ad4fa29800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f609261e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 0x55ad4fa2c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f609261e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ad4f984280 con 0x55ad4fa29d40 2024-09-19T06:12:13.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f609261e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ad4f92bb10 con 0x55ad4fa1f480 2024-09-19T06:12:13.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f609261e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ad4f980240 con 0x55ad4fa1eb60 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f608d7e0640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60740a3430 0x7f60740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f608e7e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 0x55ad4fa2c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f607effd640 1 -- 172.21.15.5:0/1696092551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f60800cffe0 con 0x7f6088106a80 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f608e7e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 0x55ad4fa2c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43704/0 (socket says 172.21.15.5:43704) 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f608e7e2640 1 -- 172.21.15.5:0/2079479233 learned_addr learned my addr 172.21.15.5:0/2079479233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f608dfe1640 1 --2- 172.21.15.5:0/2079479233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa1ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f608d7e0640 1 --2- 172.21.15.5:0/2079479233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 0x55ad4fa29800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f608e7e2640 1 -- 172.21.15.5:0/2079479233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa1eb60 msgr2=0x55ad4fa1ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f608e7e2640 1 --2- 172.21.15.5:0/2079479233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa1ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f608e7e2640 1 -- 172.21.15.5:0/2079479233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 msgr2=0x55ad4fa29800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f608e7e2640 1 --2- 172.21.15.5:0/2079479233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 0x55ad4fa29800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.517+0000 7f608e7e2640 1 -- 172.21.15.5:0/2079479233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad4f9381f0 con 0x55ad4fa29d40 2024-09-19T06:12:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.518+0000 7f608d7e0640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60740a3430 0x7f60740a58f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f60881040e0 tx=0x7f6078013880 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:13.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.518+0000 7f608e7e2640 1 --2- 172.21.15.5:0/2079479233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 0x55ad4fa2c130 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f608009c230 tx=0x7f608009bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c047309302729936 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:13.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.518+0000 7f6073fff640 1 -- 172.21.15.5:0/2079479233 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60800cde60 con 0x55ad4fa29d40 2024-09-19T06:12:13.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.518+0000 7f6073fff640 1 -- 172.21.15.5:0/2079479233 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f608009c9a0 con 0x55ad4fa29d40 2024-09-19T06:12:13.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.518+0000 7f6073fff640 1 -- 172.21.15.5:0/2079479233 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608009a630 con 0x55ad4fa29d40 2024-09-19T06:12:13.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.519+0000 7f609261e740 1 -- 172.21.15.5:0/2079479233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 msgr2=0x55ad4fa2c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:13.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.519+0000 7f609261e740 1 --2- 172.21.15.5:0/2079479233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 0x55ad4fa2c130 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f608009c230 tx=0x7f608009bc00 comp rx=0 tx=0).stop 2024-09-19T06:12:13.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.519+0000 7f609261e740 1 -- 172.21.15.5:0/2079479233 shutdown_connections 2024-09-19T06:12:13.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.519+0000 7f609261e740 1 --2- 172.21.15.5:0/2079479233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 0x55ad4fa2c130 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.519+0000 7f609261e740 1 --2- 172.21.15.5:0/2079479233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 0x55ad4fa29800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.519+0000 7f609261e740 1 --2- 172.21.15.5:0/2079479233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa1ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.519+0000 7f609261e740 1 -- 172.21.15.5:0/2079479233 >> 172.21.15.5:0/2079479233 conn(0x55ad4fa1b0f0 msgr2=0x55ad4fa1b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.519+0000 7f609261e740 1 -- 172.21.15.5:0/2079479233 shutdown_connections 2024-09-19T06:12:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.519+0000 7f609261e740 1 -- 172.21.15.5:0/2079479233 wait complete. 2024-09-19T06:12:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.519+0000 7f609261e740 1 -- start start 2024-09-19T06:12:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.519+0000 7f609261e740 1 Processor -- start 2024-09-19T06:12:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f609261e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa26ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f609261e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 0x55ad4fa271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f609261e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 0x55ad4fa27720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f609261e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ad4fa2c910 con 0x55ad4fa29d40 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f609261e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ad4f93ef10 con 0x55ad4fa1f480 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f609261e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ad4fa2ca90 con 0x55ad4fa1eb60 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f609261e740 10 client.0 _ll_register_callbacks cb 0x55ad4f907a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f608e7e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 0x55ad4fa27720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f608e7e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 0x55ad4fa27720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43716/0 (socket says 172.21.15.5:43716) 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f608e7e2640 1 -- 172.21.15.5:0/493659237 learned_addr learned my addr 172.21.15.5:0/493659237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f608dfe1640 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa26ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f608d7e0640 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 0x55ad4fa271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f608e7e2640 1 -- 172.21.15.5:0/493659237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa1eb60 msgr2=0x55ad4fa26ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f608e7e2640 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa26ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f608e7e2640 1 -- 172.21.15.5:0/493659237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 msgr2=0x55ad4fa271e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f608e7e2640 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 0x55ad4fa271e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f608e7e2640 1 -- 172.21.15.5:0/493659237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad4f6f56a0 con 0x55ad4fa29d40 2024-09-19T06:12:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f608e7e2640 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 0x55ad4fa27720 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f608000ab20 tx=0x7f608009c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a94558f76cea59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f60717fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f60717fa640 1 -- 172.21.15.5:0/493659237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60800cde60 con 0x55ad4fa29d40 2024-09-19T06:12:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.520+0000 7f60717fa640 1 -- 172.21.15.5:0/493659237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f608000fc50 con 0x55ad4fa29d40 2024-09-19T06:12:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f609261e740 1 -- 172.21.15.5:0/493659237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f608009c9a0 con 0x55ad4fa29d40 2024-09-19T06:12:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f60717fa640 1 -- 172.21.15.5:0/493659237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f60800a0650 con 0x55ad4fa29d40 2024-09-19T06:12:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f609261e740 10 client.17736 fetch_fsmap learned FSMap version 34 2024-09-19T06:12:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f609261e740 1 -- 172.21.15.5:0/493659237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ad4fa38590 con 0x55ad4fa29d40 2024-09-19T06:12:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f60717fa640 1 -- 172.21.15.5:0/493659237 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f608009c9a0 con 0x55ad4fa29d40 2024-09-19T06:12:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f60717fa640 1 -- 172.21.15.5:0/493659237 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f60800bc0b0 con 0x55ad4fa29d40 2024-09-19T06:12:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f609261e740 10 client.17736 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T06:12:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f609261e740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f609261e740 20 client.17736 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f609261e740 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ad4fa398e0 0x55ad4fa3bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f609261e740 4 client.17736 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f609261e740 1 -- 172.21.15.5:0/493659237 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad4f9917f0 con 0x55ad4fa398e0 2024-09-19T06:12:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.521+0000 7f608dfe1640 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ad4fa398e0 0x55ad4fa3bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.522+0000 7f608dfe1640 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ad4fa398e0 0x55ad4fa3bcd0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4298e732fe8338a7 server_cookie=fe7867c32beae02c in_seq=0 out_seq=0 2024-09-19T06:12:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.522+0000 7f60717fa640 10 client.17736 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f60717fa640 1 -- 172.21.15.5:0/493659237 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad4f9917f0 con 0x55ad4fa398e0 2024-09-19T06:12:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f60717fa640 10 client.17736 handle_command_reply: tid=0 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f609261e740 1 client.17736 shutdown 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f609261e740 1 -- 172.21.15.5:0/493659237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 msgr2=0x55ad4fa27720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f609261e740 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 0x55ad4fa27720 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f608000ab20 tx=0x7f608009c330 comp rx=0 tx=0).stop 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f609261e740 1 -- 172.21.15.5:0/493659237 shutdown_connections 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f609261e740 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ad4fa398e0 0x55ad4fa3bcd0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f609261e740 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa29d40 0x55ad4fa27720 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f60717fa640 0 client.17736 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f609261e740 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 0x55ad4fa271e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f609261e740 1 --2- 172.21.15.5:0/493659237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa26ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f609261e740 1 -- 172.21.15.5:0/493659237 >> 172.21.15.5:0/493659237 conn(0x55ad4fa1b0f0 msgr2=0x55ad4fa1a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f609261e740 1 -- 172.21.15.5:0/493659237 shutdown_connections 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f609261e740 1 -- 172.21.15.5:0/493659237 wait complete. 2024-09-19T06:12:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.523+0000 7f609261e740 20 client.17736 trim_cache size 0 max 16384 2024-09-19T06:12:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.540+0000 7f609261e740 1 Processor -- start 2024-09-19T06:12:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f609261e740 1 -- start start 2024-09-19T06:12:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f609261e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa27570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f609261e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 0x55ad4fa3ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f609261e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa29d40 0x55ad4fa3f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f609261e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60881af920 con 0x55ad4fa1eb60 2024-09-19T06:12:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f609261e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60881b1460 con 0x55ad4fa1f480 2024-09-19T06:12:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f609261e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60881afff0 con 0x55ad4fa29d40 2024-09-19T06:12:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f608dfe1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa27570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f608dfe1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa27570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43726/0 (socket says 172.21.15.5:43726) 2024-09-19T06:12:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f608dfe1640 1 -- 172.21.15.5:0/2837228772 learned_addr learned my addr 172.21.15.5:0/2837228772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f608e7e2640 1 --2- 172.21.15.5:0/2837228772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa29d40 0x55ad4fa3f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f608d7e0640 1 --2- 172.21.15.5:0/2837228772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 0x55ad4fa3ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f608dfe1640 1 -- 172.21.15.5:0/2837228772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa29d40 msgr2=0x55ad4fa3f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f608dfe1640 1 --2- 172.21.15.5:0/2837228772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa29d40 0x55ad4fa3f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f608dfe1640 1 -- 172.21.15.5:0/2837228772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 msgr2=0x55ad4fa3ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f608dfe1640 1 --2- 172.21.15.5:0/2837228772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 0x55ad4fa3ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f608dfe1640 1 -- 172.21.15.5:0/2837228772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad4fa29570 con 0x55ad4fa1eb60 2024-09-19T06:12:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f608dfe1640 1 --2- 172.21.15.5:0/2837228772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa27570 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f608402a940 tx=0x7f608402ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17529ae60325971 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.541+0000 7f60737fe640 1 -- 172.21.15.5:0/2837228772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6084035040 con 0x55ad4fa1eb60 2024-09-19T06:12:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.542+0000 7f60737fe640 1 -- 172.21.15.5:0/2837228772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6084004400 con 0x55ad4fa1eb60 2024-09-19T06:12:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.542+0000 7f60737fe640 1 -- 172.21.15.5:0/2837228772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60840035f0 con 0x55ad4fa1eb60 2024-09-19T06:12:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.542+0000 7f609261e740 1 -- 172.21.15.5:0/2837228772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1eb60 msgr2=0x55ad4fa27570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.542+0000 7f609261e740 1 --2- 172.21.15.5:0/2837228772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa27570 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f608402a940 tx=0x7f608402ab90 comp rx=0 tx=0).stop 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.542+0000 7f609261e740 1 -- 172.21.15.5:0/2837228772 shutdown_connections 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.542+0000 7f609261e740 1 --2- 172.21.15.5:0/2837228772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa29d40 0x55ad4fa3f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.542+0000 7f609261e740 1 --2- 172.21.15.5:0/2837228772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1f480 0x55ad4fa3ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.542+0000 7f609261e740 1 --2- 172.21.15.5:0/2837228772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa27570 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.542+0000 7f609261e740 1 -- 172.21.15.5:0/2837228772 >> 172.21.15.5:0/2837228772 conn(0x55ad4fa1b0f0 msgr2=0x55ad4fa28c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.542+0000 7f609261e740 1 -- 172.21.15.5:0/2837228772 shutdown_connections 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.542+0000 7f609261e740 1 -- 172.21.15.5:0/2837228772 wait complete. 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.542+0000 7f609261e740 1 -- start start 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f609261e740 1 Processor -- start 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f609261e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa3cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f609261e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1f480 0x55ad4fa3d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f609261e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa29d40 0x55ad4fa3d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f609261e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60881b0d90 con 0x55ad4fa1f480 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f609261e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60881b06c0 con 0x55ad4fa1eb60 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f609261e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60881ad920 con 0x55ad4fa29d40 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f609261e740 10 client.0 _ll_register_callbacks cb 0x7f60800a0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f608d7e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1f480 0x55ad4fa3d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f608d7e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1f480 0x55ad4fa3d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43742/0 (socket says 172.21.15.5:43742) 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f608d7e0640 1 -- 172.21.15.5:0/255209561 learned_addr learned my addr 172.21.15.5:0/255209561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f608e7e2640 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa29d40 0x55ad4fa3d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f608dfe1640 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa3cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f608d7e0640 1 -- 172.21.15.5:0/255209561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa29d40 msgr2=0x55ad4fa3d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f608d7e0640 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa29d40 0x55ad4fa3d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f608d7e0640 1 -- 172.21.15.5:0/255209561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1eb60 msgr2=0x55ad4fa3cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f608d7e0640 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa3cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.543+0000 7f608d7e0640 1 -- 172.21.15.5:0/255209561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad4faa32f0 con 0x55ad4fa1f480 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f608d7e0640 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1f480 0x55ad4fa3d080 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x55ad4fa1bf50 tx=0x7f6078019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6e65aea1f45b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f60717fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f60717fa640 1 -- 172.21.15.5:0/255209561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607801fd90 con 0x55ad4fa1f480 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f60717fa640 1 -- 172.21.15.5:0/255209561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f6078005020 con 0x55ad4fa1f480 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f609261e740 1 -- 172.21.15.5:0/255209561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6084004400 con 0x55ad4fa1f480 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f60717fa640 1 -- 172.21.15.5:0/255209561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6078005220 con 0x55ad4fa1f480 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f609261e740 10 client.17742 fetch_fsmap learned FSMap version 34 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f609261e740 1 -- 172.21.15.5:0/255209561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ad4faa4610 con 0x55ad4fa1f480 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f60717fa640 1 -- 172.21.15.5:0/255209561 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6084004400 con 0x55ad4fa1f480 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f60717fa640 1 -- 172.21.15.5:0/255209561 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f6078029440 con 0x55ad4fa1f480 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f609261e740 10 client.17742 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T06:12:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f609261e740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.544+0000 7f609261e740 20 client.17742 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.545+0000 7f609261e740 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ad4faa4e10 0x55ad4faa7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.545+0000 7f609261e740 4 client.17742 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:12:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.545+0000 7f609261e740 1 -- 172.21.15.5:0/255209561 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ad4f98bd00 con 0x55ad4faa4e10 2024-09-19T06:12:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.545+0000 7f608dfe1640 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ad4faa4e10 0x55ad4faa7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.545+0000 7f608dfe1640 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ad4faa4e10 0x55ad4faa7230 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=796fde6a59288f18 server_cookie=2ac17afd5dddf4ef in_seq=0 out_seq=0 2024-09-19T06:12:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.545+0000 7f60717fa640 10 client.17742 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.546+0000 7f60717fa640 1 -- 172.21.15.5:0/255209561 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ad4f98bd00 con 0x55ad4faa4e10 2024-09-19T06:12:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.546+0000 7f60717fa640 10 client.17742 handle_command_reply: tid=0 2024-09-19T06:12:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.546+0000 7f609261e740 1 client.17742 shutdown 2024-09-19T06:12:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f609261e740 1 -- 172.21.15.5:0/255209561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1f480 msgr2=0x55ad4fa3d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f609261e740 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1f480 0x55ad4fa3d080 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x55ad4fa1bf50 tx=0x7f6078019000 comp rx=0 tx=0).stop 2024-09-19T06:12:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f609261e740 1 -- 172.21.15.5:0/255209561 shutdown_connections 2024-09-19T06:12:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f609261e740 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ad4faa4e10 0x55ad4faa7230 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f609261e740 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4fa29d40 0x55ad4fa3d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f609261e740 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad4fa1f480 0x55ad4fa3d080 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f609261e740 1 --2- 172.21.15.5:0/255209561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad4fa1eb60 0x55ad4fa3cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f60717fa640 0 client.17742 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f609261e740 1 -- 172.21.15.5:0/255209561 >> 172.21.15.5:0/255209561 conn(0x55ad4fa1b0f0 msgr2=0x55ad4fa34ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f609261e740 1 -- 172.21.15.5:0/255209561 shutdown_connections 2024-09-19T06:12:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f609261e740 1 -- 172.21.15.5:0/255209561 wait complete. 2024-09-19T06:12:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f609261e740 20 client.17742 trim_cache size 0 max 16384 2024-09-19T06:12:13.554 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:12:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:12:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8b3cbb66-6048-4943-8e19-b81bfc9873e5", 2024-09-19T06:12:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:12:13.555 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:12:13.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f6090245640 1 -- 172.21.15.5:0/1696092551 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60740a3430 msgr2=0x7f60740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:13.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f6090245640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60740a3430 0x7f60740a58f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f60881040e0 tx=0x7f6078013880 comp rx=0 tx=0).stop 2024-09-19T06:12:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f6090245640 1 -- 172.21.15.5:0/1696092551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6088106a80 msgr2=0x7f60881030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f6090245640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6088106a80 0x7f60881030d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f608000cce0 tx=0x7f608000a620 comp rx=0 tx=0).stop 2024-09-19T06:12:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f6090245640 1 -- 172.21.15.5:0/1696092551 shutdown_connections 2024-09-19T06:12:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.547+0000 7f6090245640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6088106a80 0x7f60881030d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.548+0000 7f6090245640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6088101420 0x7f6088102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.548+0000 7f6090245640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60740a3430 0x7f60740a58f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.548+0000 7f6090245640 1 --2- 172.21.15.5:0/1696092551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6088100a80 0x7f6088102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.548+0000 7f6090245640 1 -- 172.21.15.5:0/1696092551 >> 172.21.15.5:0/1696092551 conn(0x7f60880fc7a0 msgr2=0x7f60881048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.548+0000 7f6090245640 1 -- 172.21.15.5:0/1696092551 shutdown_connections 2024-09-19T06:12:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:13.548+0000 7f6090245640 1 -- 172.21.15.5:0/1696092551 wait complete. 2024-09-19T06:12:13.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 230739 2.68 MB/sec execute 339 sec latency 2260.746 ms 2024-09-19T06:12:14.412 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:12:14.669 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:14.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:14 smithi005 ceph-mon[30955]: pgmap v276: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-09-19T06:12:14.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:14 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:12:14.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:14 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:14 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:12:14.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:14 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:12:14.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:14 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:14 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:12:14.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:14 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:14 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:12:14.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:14 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:14 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:14 smithi157 ceph-mon[35346]: pgmap v276: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-09-19T06:12:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:14 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:12:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:14 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:14 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:12:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:14 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:12:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:14 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:14 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:12:14.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:14 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:14 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:12:14.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:14 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:14 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 230739 2.67 MB/sec execute 340 sec latency 1029.088 ms 2024-09-19T06:12:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:14 smithi049 ceph-mon[34640]: pgmap v276: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-09-19T06:12:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:14 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:12:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:14 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:14 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:12:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:14 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:12:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:14 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:14 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:12:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:14 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:14 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:12:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:14 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:14.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:14 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.103+0000 7fb4432fd640 1 -- 172.21.15.5:0/1405932705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43c101de0 msgr2=0x7fb43c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.103+0000 7fb4432fd640 1 --2- 172.21.15.5:0/1405932705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43c101de0 0x7fb43c1021c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb438009900 tx=0x7fb43802a410 comp rx=0 tx=0).stop 2024-09-19T06:12:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.103+0000 7fb4432fd640 1 -- 172.21.15.5:0/1405932705 shutdown_connections 2024-09-19T06:12:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.103+0000 7fb4432fd640 1 --2- 172.21.15.5:0/1405932705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb43c106f10 0x7fb43c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.103+0000 7fb4432fd640 1 --2- 172.21.15.5:0/1405932705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb43c102790 0x7fb43c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.103+0000 7fb4432fd640 1 --2- 172.21.15.5:0/1405932705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43c101de0 0x7fb43c1021c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.103+0000 7fb4432fd640 1 -- 172.21.15.5:0/1405932705 >> 172.21.15.5:0/1405932705 conn(0x7fb43c0fd8c0 msgr2=0x7fb43c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.103+0000 7fb4432fd640 1 -- 172.21.15.5:0/1405932705 shutdown_connections 2024-09-19T06:12:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.103+0000 7fb4432fd640 1 -- 172.21.15.5:0/1405932705 wait complete. 2024-09-19T06:12:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb4432fd640 1 Processor -- start 2024-09-19T06:12:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb4432fd640 1 -- start start 2024-09-19T06:12:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb4432fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43c101de0 0x7fb43c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb4432fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb43c102790 0x7fb43c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb4432fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb43c106f10 0x7fb43c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb4432fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c111e20 con 0x7fb43c101de0 2024-09-19T06:12:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb4432fd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c111ca0 con 0x7fb43c102790 2024-09-19T06:12:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb4432fd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c111fa0 con 0x7fb43c106f10 2024-09-19T06:12:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb441099640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43c101de0 0x7fb43c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb441099640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43c101de0 0x7fb43c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35642/0 (socket says 172.21.15.5:35642) 2024-09-19T06:12:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb441099640 1 -- 172.21.15.5:0/2077873169 learned_addr learned my addr 172.21.15.5:0/2077873169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb44189a640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb43c106f10 0x7fb43c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.104+0000 7fb440898640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb43c102790 0x7fb43c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.105+0000 7fb441099640 1 -- 172.21.15.5:0/2077873169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb43c106f10 msgr2=0x7fb43c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.105+0000 7fb441099640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb43c106f10 0x7fb43c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.105+0000 7fb441099640 1 -- 172.21.15.5:0/2077873169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb43c102790 msgr2=0x7fb43c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.105+0000 7fb441099640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb43c102790 0x7fb43c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.105+0000 7fb441099640 1 -- 172.21.15.5:0/2077873169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43c19d340 con 0x7fb43c101de0 2024-09-19T06:12:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.105+0000 7fb440898640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb43c102790 0x7fb43c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:12:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.105+0000 7fb441099640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43c101de0 0x7fb43c198390 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb438005940 tx=0x7fb438004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.105+0000 7fb4327fc640 1 -- 172.21.15.5:0/2077873169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43803f030 con 0x7fb43c101de0 2024-09-19T06:12:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.105+0000 7fb4327fc640 1 -- 172.21.15.5:0/2077873169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb438003aa0 con 0x7fb43c101de0 2024-09-19T06:12:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.105+0000 7fb4432fd640 1 -- 172.21.15.5:0/2077873169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43c19d5d0 con 0x7fb43c101de0 2024-09-19T06:12:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.105+0000 7fb4327fc640 1 -- 172.21.15.5:0/2077873169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438035070 con 0x7fb43c101de0 2024-09-19T06:12:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.105+0000 7fb4432fd640 1 -- 172.21.15.5:0/2077873169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43c1a4f10 con 0x7fb43c101de0 2024-09-19T06:12:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.107+0000 7fb4327fc640 1 -- 172.21.15.5:0/2077873169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb438042030 con 0x7fb43c101de0 2024-09-19T06:12:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.108+0000 7fb4456d6740 1 Processor -- start 2024-09-19T06:12:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.108+0000 7fb4456d6740 1 -- start start 2024-09-19T06:12:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.108+0000 7fb4456d6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 0x561fd4c49880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.108+0000 7fb4456d6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c49dc0 0x561fd4c54140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.108+0000 7fb4456d6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c56a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.108+0000 7fb4456d6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561fd4baf2e0 con 0x561fd4c494a0 2024-09-19T06:12:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.108+0000 7fb4456d6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561fd4b56ce0 con 0x561fd4c49dc0 2024-09-19T06:12:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.108+0000 7fb4456d6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561fd4bab2a0 con 0x561fd4c54680 2024-09-19T06:12:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.108+0000 7fb44189a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 0x561fd4c49880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.108+0000 7fb4327fc640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb4280a33b0 0x7fb4280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.108+0000 7fb44189a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 0x561fd4c49880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35652/0 (socket says 172.21.15.5:35652) 2024-09-19T06:12:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.108+0000 7fb44189a640 1 -- 172.21.15.5:0/3321797255 learned_addr learned my addr 172.21.15.5:0/3321797255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb440898640 1 --2- 172.21.15.5:0/3321797255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c56a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb44189a640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb4280a33b0 0x7fb4280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb441099640 1 --2- 172.21.15.5:0/3321797255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c49dc0 0x561fd4c54140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb4327fc640 1 -- 172.21.15.5:0/2077873169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7fb4380f0550 con 0x7fb43c101de0 2024-09-19T06:12:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb44189a640 1 -- 172.21.15.5:0/3321797255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 msgr2=0x561fd4c56a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb44189a640 1 --2- 172.21.15.5:0/3321797255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c56a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb44189a640 1 -- 172.21.15.5:0/3321797255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c49dc0 msgr2=0x561fd4c54140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb44189a640 1 --2- 172.21.15.5:0/3321797255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c49dc0 0x561fd4c54140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb44189a640 1 -- 172.21.15.5:0/3321797255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd4b3b440 con 0x561fd4c494a0 2024-09-19T06:12:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb441099640 1 --2- 172.21.15.5:0/3321797255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c49dc0 0x561fd4c54140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:12:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb44189a640 1 --2- 172.21.15.5:0/3321797255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 0x561fd4c49880 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb434014a60 tx=0x7fb434014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55993bcc2823970 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb4277fe640 1 -- 172.21.15.5:0/3321797255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43401ec50 con 0x561fd4c494a0 2024-09-19T06:12:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb4277fe640 1 -- 172.21.15.5:0/3321797255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb434023070 con 0x561fd4c494a0 2024-09-19T06:12:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb4277fe640 1 -- 172.21.15.5:0/3321797255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43401dcd0 con 0x561fd4c494a0 2024-09-19T06:12:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.109+0000 7fb44189a640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb4280a33b0 0x7fb4280a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb434022000 tx=0x7fb43401e000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.110+0000 7fb4456d6740 1 -- 172.21.15.5:0/3321797255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 msgr2=0x561fd4c49880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.110+0000 7fb4456d6740 1 --2- 172.21.15.5:0/3321797255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 0x561fd4c49880 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb434014a60 tx=0x7fb434014f20 comp rx=0 tx=0).stop 2024-09-19T06:12:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.110+0000 7fb4456d6740 1 -- 172.21.15.5:0/3321797255 shutdown_connections 2024-09-19T06:12:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.110+0000 7fb4456d6740 1 --2- 172.21.15.5:0/3321797255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c56a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.110+0000 7fb4456d6740 1 --2- 172.21.15.5:0/3321797255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c49dc0 0x561fd4c54140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.110+0000 7fb4456d6740 1 --2- 172.21.15.5:0/3321797255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 0x561fd4c49880 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.110+0000 7fb4456d6740 1 -- 172.21.15.5:0/3321797255 >> 172.21.15.5:0/3321797255 conn(0x561fd4c45a30 msgr2=0x561fd4c45e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.110+0000 7fb4456d6740 1 -- 172.21.15.5:0/3321797255 shutdown_connections 2024-09-19T06:12:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.110+0000 7fb4456d6740 1 -- 172.21.15.5:0/3321797255 wait complete. 2024-09-19T06:12:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.110+0000 7fb4456d6740 1 -- start start 2024-09-19T06:12:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.110+0000 7fb4456d6740 1 Processor -- start 2024-09-19T06:12:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.111+0000 7fb4456d6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c494a0 0x561fd4c54000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.111+0000 7fb4456d6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 0x561fd4c512b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.111+0000 7fb4456d6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c51830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.111+0000 7fb4456d6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561fd4c57250 con 0x561fd4c49dc0 2024-09-19T06:12:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.111+0000 7fb4456d6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561fd4b6a050 con 0x561fd4c494a0 2024-09-19T06:12:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.111+0000 7fb4456d6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561fd4c573d0 con 0x561fd4c54680 2024-09-19T06:12:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.111+0000 7fb4456d6740 10 client.0 _ll_register_callbacks cb 0x561fd4baa160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.111+0000 7fb440898640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 0x561fd4c512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.111+0000 7fb440898640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 0x561fd4c512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35666/0 (socket says 172.21.15.5:35666) 2024-09-19T06:12:15.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.111+0000 7fb440898640 1 -- 172.21.15.5:0/3421099507 learned_addr learned my addr 172.21.15.5:0/3421099507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:15.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.111+0000 7fb441099640 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c494a0 0x561fd4c54000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.111+0000 7fb44189a640 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c51830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb440898640 1 -- 172.21.15.5:0/3421099507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 msgr2=0x561fd4c51830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:15.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb440898640 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c51830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb440898640 1 -- 172.21.15.5:0/3421099507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c494a0 msgr2=0x561fd4c54000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb440898640 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c494a0 0x561fd4c54000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb440898640 1 -- 172.21.15.5:0/3421099507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd49203c0 con 0x561fd4c49dc0 2024-09-19T06:12:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb440898640 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 0x561fd4c512b0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0106e0 tx=0x7fb42c010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4f8ee04f1eed5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb424ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb424ff9640 1 -- 172.21.15.5:0/3421099507 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c01d060 con 0x561fd4c49dc0 2024-09-19T06:12:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb424ff9640 1 -- 172.21.15.5:0/3421099507 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7fb42c00eb90 con 0x561fd4c49dc0 2024-09-19T06:12:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb4456d6740 1 -- 172.21.15.5:0/3421099507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb434023070 con 0x561fd4c49dc0 2024-09-19T06:12:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb424ff9640 1 -- 172.21.15.5:0/3421099507 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb42c00ed90 con 0x561fd4c49dc0 2024-09-19T06:12:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb4456d6740 10 client.17754 fetch_fsmap learned FSMap version 34 2024-09-19T06:12:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb4456d6740 1 -- 172.21.15.5:0/3421099507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561fd4c62c10 con 0x561fd4c49dc0 2024-09-19T06:12:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.112+0000 7fb424ff9640 1 -- 172.21.15.5:0/3421099507 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb434023070 con 0x561fd4c49dc0 2024-09-19T06:12:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.113+0000 7fb424ff9640 1 -- 172.21.15.5:0/3421099507 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fb42c0204f0 con 0x561fd4c49dc0 2024-09-19T06:12:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.113+0000 7fb4456d6740 10 client.17754 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T06:12:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.113+0000 7fb4456d6740 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.113+0000 7fb4456d6740 20 client.17754 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.113+0000 7fb4456d6740 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fd4c64120 0x561fd4c66560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.113+0000 7fb4456d6740 4 client.17754 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.113+0000 7fb4456d6740 1 -- 172.21.15.5:0/3421099507 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fd4bbc850 con 0x561fd4c64120 2024-09-19T06:12:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.113+0000 7fb441099640 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fd4c64120 0x561fd4c66560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.113+0000 7fb424ff9640 10 client.17754 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.113+0000 7fb441099640 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fd4c64120 0x561fd4c66560 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d86b3a876e946d server_cookie=78e036502c9ca901 in_seq=0 out_seq=0 2024-09-19T06:12:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb424ff9640 1 -- 172.21.15.5:0/3421099507 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fd4bbc850 con 0x561fd4c64120 2024-09-19T06:12:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb424ff9640 10 client.17754 handle_command_reply: tid=0 2024-09-19T06:12:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb4456d6740 1 client.17754 shutdown 2024-09-19T06:12:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb4456d6740 1 -- 172.21.15.5:0/3421099507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 msgr2=0x561fd4c512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb4456d6740 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 0x561fd4c512b0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0106e0 tx=0x7fb42c010ba0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb4456d6740 1 -- 172.21.15.5:0/3421099507 shutdown_connections 2024-09-19T06:12:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb4456d6740 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fd4c64120 0x561fd4c66560 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb4456d6740 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c51830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb4456d6740 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 0x561fd4c512b0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb4456d6740 1 --2- 172.21.15.5:0/3421099507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c494a0 0x561fd4c54000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb424ff9640 0 client.17754 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb4456d6740 1 -- 172.21.15.5:0/3421099507 >> 172.21.15.5:0/3421099507 conn(0x561fd4c45a30 msgr2=0x561fd4c44b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb4456d6740 1 -- 172.21.15.5:0/3421099507 shutdown_connections 2024-09-19T06:12:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb4456d6740 1 -- 172.21.15.5:0/3421099507 wait complete. 2024-09-19T06:12:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.114+0000 7fb4456d6740 20 client.17754 trim_cache size 0 max 16384 2024-09-19T06:12:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.133+0000 7fb4456d6740 1 Processor -- start 2024-09-19T06:12:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.133+0000 7fb4456d6740 1 -- start start 2024-09-19T06:12:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb4456d6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c494a0 0x561fd4c604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb4456d6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 0x561fd4c4fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb4456d6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c6be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb4456d6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1ad8e0 con 0x561fd4c49dc0 2024-09-19T06:12:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb4456d6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1af420 con 0x561fd4c494a0 2024-09-19T06:12:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb4456d6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1adfb0 con 0x561fd4c54680 2024-09-19T06:12:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb440898640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 0x561fd4c4fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb440898640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 0x561fd4c4fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35674/0 (socket says 172.21.15.5:35674) 2024-09-19T06:12:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb440898640 1 -- 172.21.15.5:0/4158270920 learned_addr learned my addr 172.21.15.5:0/4158270920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb441099640 1 --2- 172.21.15.5:0/4158270920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c494a0 0x561fd4c604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb44189a640 1 --2- 172.21.15.5:0/4158270920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c6be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb440898640 1 -- 172.21.15.5:0/4158270920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 msgr2=0x561fd4c6be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb440898640 1 --2- 172.21.15.5:0/4158270920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c6be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb440898640 1 -- 172.21.15.5:0/4158270920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c494a0 msgr2=0x561fd4c604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb440898640 1 --2- 172.21.15.5:0/4158270920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c494a0 0x561fd4c604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb440898640 1 -- 172.21.15.5:0/4158270920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd4c492d0 con 0x561fd4c49dc0 2024-09-19T06:12:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb44189a640 1 --2- 172.21.15.5:0/4158270920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c6be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:12:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.134+0000 7fb440898640 1 --2- 172.21.15.5:0/4158270920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 0x561fd4c4fb80 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb42c020b00 tx=0x7fb42c00efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2323e6ba0772871c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.135+0000 7fb426ffd640 1 -- 172.21.15.5:0/4158270920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c000e60 con 0x561fd4c49dc0 2024-09-19T06:12:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.135+0000 7fb426ffd640 1 -- 172.21.15.5:0/4158270920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb42c02e070 con 0x561fd4c49dc0 2024-09-19T06:12:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.135+0000 7fb426ffd640 1 -- 172.21.15.5:0/4158270920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c018670 con 0x561fd4c49dc0 2024-09-19T06:12:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.135+0000 7fb4456d6740 1 -- 172.21.15.5:0/4158270920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 msgr2=0x561fd4c4fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.135+0000 7fb4456d6740 1 --2- 172.21.15.5:0/4158270920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 0x561fd4c4fb80 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb42c020b00 tx=0x7fb42c00efb0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.135+0000 7fb4456d6740 1 -- 172.21.15.5:0/4158270920 shutdown_connections 2024-09-19T06:12:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.135+0000 7fb4456d6740 1 --2- 172.21.15.5:0/4158270920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c6be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.135+0000 7fb4456d6740 1 --2- 172.21.15.5:0/4158270920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c49dc0 0x561fd4c4fb80 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.135+0000 7fb4456d6740 1 --2- 172.21.15.5:0/4158270920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c494a0 0x561fd4c604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.135+0000 7fb4456d6740 1 -- 172.21.15.5:0/4158270920 >> 172.21.15.5:0/4158270920 conn(0x561fd4c45a30 msgr2=0x561fd4c4c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.135+0000 7fb4456d6740 1 -- 172.21.15.5:0/4158270920 shutdown_connections 2024-09-19T06:12:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.135+0000 7fb4456d6740 1 -- 172.21.15.5:0/4158270920 wait complete. 2024-09-19T06:12:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.136+0000 7fb4456d6740 1 -- start start 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.136+0000 7fb4456d6740 1 Processor -- start 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.136+0000 7fb4456d6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 0x561fd4c58600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.136+0000 7fb4456d6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c49dc0 0x561fd4c58b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb4456d6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c59080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb4456d6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1aed50 con 0x561fd4c494a0 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb441099640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 0x561fd4c58600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb4456d6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1ae680 con 0x561fd4c49dc0 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb4456d6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1ab8e0 con 0x561fd4c54680 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb4456d6740 10 client.0 _ll_register_callbacks cb 0x7fb42c00b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb441099640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 0x561fd4c58600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35690/0 (socket says 172.21.15.5:35690) 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb441099640 1 -- 172.21.15.5:0/4276928395 learned_addr learned my addr 172.21.15.5:0/4276928395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb440898640 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c49dc0 0x561fd4c58b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb44189a640 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c59080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb441099640 1 -- 172.21.15.5:0/4276928395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 msgr2=0x561fd4c59080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb441099640 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c59080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb441099640 1 -- 172.21.15.5:0/4276928395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c49dc0 msgr2=0x561fd4c58b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb441099640 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c49dc0 0x561fd4c58b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb441099640 1 -- 172.21.15.5:0/4276928395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd4cce0f0 con 0x561fd4c494a0 2024-09-19T06:12:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb440898640 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c49dc0 0x561fd4c58b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb441099640 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 0x561fd4c58600 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb438032ad0 tx=0x7fb438041f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cc3271c0dab831 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb424ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb424ff9640 1 -- 172.21.15.5:0/4276928395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438049040 con 0x561fd4c494a0 2024-09-19T06:12:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb424ff9640 1 -- 172.21.15.5:0/4276928395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7fb438003c50 con 0x561fd4c494a0 2024-09-19T06:12:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.137+0000 7fb4456d6740 1 -- 172.21.15.5:0/4276928395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb42c02e070 con 0x561fd4c494a0 2024-09-19T06:12:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.138+0000 7fb424ff9640 1 -- 172.21.15.5:0/4276928395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb4380c0b60 con 0x561fd4c494a0 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.138+0000 7fb4456d6740 10 client.17760 fetch_fsmap learned FSMap version 34 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.138+0000 7fb4456d6740 1 -- 172.21.15.5:0/4276928395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561fd4ccf350 con 0x561fd4c494a0 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.138+0000 7fb424ff9640 1 -- 172.21.15.5:0/4276928395 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb42c02e070 con 0x561fd4c494a0 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.138+0000 7fb424ff9640 1 -- 172.21.15.5:0/4276928395 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fb4380edea0 con 0x561fd4c494a0 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.138+0000 7fb4456d6740 10 client.17760 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.138+0000 7fb4456d6740 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.138+0000 7fb4456d6740 20 client.17760 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.138+0000 7fb4456d6740 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fd4ccfa20 0x561fd4cd1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.138+0000 7fb4456d6740 4 client.17760 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.138+0000 7fb4456d6740 1 -- 172.21.15.5:0/4276928395 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561fd4bb6d60 con 0x561fd4ccfa20 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.138+0000 7fb440898640 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fd4ccfa20 0x561fd4cd1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb440898640 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fd4ccfa20 0x561fd4cd1e30 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12ae92563b58e552 server_cookie=2b386ce05e070a07 in_seq=0 out_seq=0 2024-09-19T06:12:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb424ff9640 10 client.17760 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb424ff9640 1 -- 172.21.15.5:0/4276928395 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561fd4bb6d60 con 0x561fd4ccfa20 2024-09-19T06:12:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb424ff9640 10 client.17760 handle_command_reply: tid=0 2024-09-19T06:12:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb4456d6740 1 client.17760 shutdown 2024-09-19T06:12:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb4456d6740 1 -- 172.21.15.5:0/4276928395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 msgr2=0x561fd4c58600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb4456d6740 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 0x561fd4c58600 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb438032ad0 tx=0x7fb438041f70 comp rx=0 tx=0).stop 2024-09-19T06:12:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb4456d6740 1 -- 172.21.15.5:0/4276928395 shutdown_connections 2024-09-19T06:12:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb4456d6740 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fd4ccfa20 0x561fd4cd1e30 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb4456d6740 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd4c54680 0x561fd4c59080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb4456d6740 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fd4c49dc0 0x561fd4c58b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb4456d6740 1 --2- 172.21.15.5:0/4276928395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd4c494a0 0x561fd4c58600 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb424ff9640 0 client.17760 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb4456d6740 1 -- 172.21.15.5:0/4276928395 >> 172.21.15.5:0/4276928395 conn(0x561fd4c45a30 msgr2=0x561fd4c4aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:15.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb4456d6740 1 -- 172.21.15.5:0/4276928395 shutdown_connections 2024-09-19T06:12:15.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb4456d6740 1 -- 172.21.15.5:0/4276928395 wait complete. 2024-09-19T06:12:15.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.139+0000 7fb4456d6740 20 client.17760 trim_cache size 0 max 16384 2024-09-19T06:12:15.157 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:12:15.157 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:12:15.157 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:12:15.157 INFO:teuthology.orchestra.run.smithi005.stdout: "8b3cbb66-6048-4943-8e19-b81bfc9873e5": { 2024-09-19T06:12:15.157 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:12:15.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8b3cbb66-6048-4943-8e19-b81bfc9873e5", 2024-09-19T06:12:15.158 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:12:15.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:12:15.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:12:15.158 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:12:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.140+0000 7fb4432fd640 1 -- 172.21.15.5:0/2077873169 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb4280a33b0 msgr2=0x7fb4280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.140+0000 7fb4432fd640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb4280a33b0 0x7fb4280a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb434022000 tx=0x7fb43401e000 comp rx=0 tx=0).stop 2024-09-19T06:12:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.140+0000 7fb4432fd640 1 -- 172.21.15.5:0/2077873169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43c101de0 msgr2=0x7fb43c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.140+0000 7fb4432fd640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43c101de0 0x7fb43c198390 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb438005940 tx=0x7fb438004270 comp rx=0 tx=0).stop 2024-09-19T06:12:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.140+0000 7fb4432fd640 1 -- 172.21.15.5:0/2077873169 shutdown_connections 2024-09-19T06:12:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.140+0000 7fb4432fd640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb4280a33b0 0x7fb4280a5870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.140+0000 7fb4432fd640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb43c106f10 0x7fb43c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.140+0000 7fb4432fd640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb43c102790 0x7fb43c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.140+0000 7fb4432fd640 1 --2- 172.21.15.5:0/2077873169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43c101de0 0x7fb43c198390 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.140+0000 7fb4432fd640 1 -- 172.21.15.5:0/2077873169 >> 172.21.15.5:0/2077873169 conn(0x7fb43c0fd8c0 msgr2=0x7fb43c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.140+0000 7fb4432fd640 1 -- 172.21.15.5:0/2077873169 shutdown_connections 2024-09-19T06:12:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:15.140+0000 7fb4432fd640 1 -- 172.21.15.5:0/2077873169 wait complete. 2024-09-19T06:12:15.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:15 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:12:15.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:15 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:15.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 230854 2.67 MB/sec execute 341 sec latency 1657.632 ms 2024-09-19T06:12:15.868 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:15 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:12:15.869 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:15 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:15.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:15 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:12:15.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:15 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:15.898 INFO:tasks.cephfs.filesystem:scrub status for tag:8b3cbb66-6048-4943-8e19-b81bfc9873e5 - {'path': '/', 'tag': '8b3cbb66-6048-4943-8e19-b81bfc9873e5', 'options': 'recursive,force'} 2024-09-19T06:12:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:16 smithi157 ceph-mon[35346]: pgmap v277: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-09-19T06:12:16.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 231111 2.66 MB/sec execute 342 sec latency 15.185 ms 2024-09-19T06:12:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:16 smithi005 ceph-mon[30955]: pgmap v277: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-09-19T06:12:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:16 smithi049 ceph-mon[34640]: pgmap v277: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-09-19T06:12:17.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 231573 2.66 MB/sec execute 343 sec latency 15.227 ms 2024-09-19T06:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.085+0000 7f8a4af33640 1 -- 172.21.15.5:0/1061627957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a441009a0 msgr2=0x7f8a44100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.085+0000 7f8a4af33640 1 --2- 172.21.15.5:0/1061627957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a441009a0 0x7f8a44100e00 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009930 tx=0x7f8a2c02a420 comp rx=0 tx=0).stop 2024-09-19T06:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.085+0000 7f8a4af33640 1 -- 172.21.15.5:0/1061627957 shutdown_connections 2024-09-19T06:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.085+0000 7f8a4af33640 1 --2- 172.21.15.5:0/1061627957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a44101340 0x7f8a4410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.085+0000 7f8a4af33640 1 --2- 172.21.15.5:0/1061627957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a441009a0 0x7f8a44100e00 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.085+0000 7f8a4af33640 1 --2- 172.21.15.5:0/1061627957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a441069b0 0x7f8a44106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.085+0000 7f8a4af33640 1 -- 172.21.15.5:0/1061627957 >> 172.21.15.5:0/1061627957 conn(0x7f8a440779a0 msgr2=0x7f8a440fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.085+0000 7f8a4af33640 1 -- 172.21.15.5:0/1061627957 shutdown_connections 2024-09-19T06:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.085+0000 7f8a4af33640 1 -- 172.21.15.5:0/1061627957 wait complete. 2024-09-19T06:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.085+0000 7f8a4af33640 1 Processor -- start 2024-09-19T06:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.085+0000 7f8a4af33640 1 -- start start 2024-09-19T06:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a4af33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a441009a0 0x7f8a44198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a4af33640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a44101340 0x7f8a44198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a4af33640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a441069b0 0x7f8a4419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a4af33640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44110cf0 con 0x7f8a44101340 2024-09-19T06:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a4af33640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44110b70 con 0x7f8a441069b0 2024-09-19T06:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a4af33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44110e70 con 0x7f8a441009a0 2024-09-19T06:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a44101340 0x7f8a44198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a44101340 0x7f8a44198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35714/0 (socket says 172.21.15.5:35714) 2024-09-19T06:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a48ccf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a441009a0 0x7f8a44198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a43fff640 1 -- 172.21.15.5:0/2552458986 learned_addr learned my addr 172.21.15.5:0/2552458986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a494d0640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a441069b0 0x7f8a4419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a43fff640 1 -- 172.21.15.5:0/2552458986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a441009a0 msgr2=0x7f8a44198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a43fff640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a441009a0 0x7f8a44198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a43fff640 1 -- 172.21.15.5:0/2552458986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a441069b0 msgr2=0x7f8a4419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a43fff640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a441069b0 0x7f8a4419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a43fff640 1 -- 172.21.15.5:0/2552458986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4419d300 con 0x7f8a44101340 2024-09-19T06:12:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a494d0640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a441069b0 0x7f8a4419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a48ccf640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a441009a0 0x7f8a44198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.086+0000 7f8a43fff640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a44101340 0x7f8a44198850 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c005ab0 tx=0x7f8a2c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.087+0000 7f8a41ffb640 1 -- 172.21.15.5:0/2552458986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c03f070 con 0x7f8a44101340 2024-09-19T06:12:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.087+0000 7f8a4af33640 1 -- 172.21.15.5:0/2552458986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4419d590 con 0x7f8a44101340 2024-09-19T06:12:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.087+0000 7f8a41ffb640 1 -- 172.21.15.5:0/2552458986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a2c031b50 con 0x7f8a44101340 2024-09-19T06:12:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.087+0000 7f8a41ffb640 1 -- 172.21.15.5:0/2552458986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c035030 con 0x7f8a44101340 2024-09-19T06:12:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.087+0000 7f8a4af33640 1 -- 172.21.15.5:0/2552458986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a441a4e80 con 0x7f8a44101340 2024-09-19T06:12:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a41ffb640 1 -- 172.21.15.5:0/2552458986 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8a2c042070 con 0x7f8a44101340 2024-09-19T06:12:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a4dcc9740 1 Processor -- start 2024-09-19T06:12:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a4dcc9740 1 -- start start 2024-09-19T06:12:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a4dcc9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2825f29d0 0x55e2825f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a4dcc9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f32f0 0x55e282601b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a4dcc9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2826020c0 0x55e2826044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a4dcc9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e2825572f0 con 0x55e2826020c0 2024-09-19T06:12:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a4dcc9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e2824ff270 con 0x55e2825f32f0 2024-09-19T06:12:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a4dcc9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e2825532b0 con 0x55e2825f29d0 2024-09-19T06:12:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2826020c0 0x55e2826044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a494d0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2825f29d0 0x55e2825f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2826020c0 0x55e2826044b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35716/0 (socket says 172.21.15.5:35716) 2024-09-19T06:12:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a43fff640 1 -- 172.21.15.5:0/3691229815 learned_addr learned my addr 172.21.15.5:0/3691229815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a48ccf640 1 --2- 172.21.15.5:0/3691229815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f32f0 0x55e282601b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a41ffb640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8a300a33b0 0x7f8a300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a43fff640 1 -- 172.21.15.5:0/3691229815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2825f29d0 msgr2=0x55e2825f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a43fff640 1 --2- 172.21.15.5:0/3691229815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2825f29d0 0x55e2825f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a43fff640 1 -- 172.21.15.5:0/3691229815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f32f0 msgr2=0x55e282601b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.090+0000 7f8a43fff640 1 --2- 172.21.15.5:0/3691229815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f32f0 0x55e282601b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.091+0000 7f8a43fff640 1 -- 172.21.15.5:0/3691229815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2824fe340 con 0x55e2826020c0 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.091+0000 7f8a494d0640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8a300a33b0 0x7f8a300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.091+0000 7f8a41ffb640 1 -- 172.21.15.5:0/2552458986 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f8a2c0f0280 con 0x7f8a44101340 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.091+0000 7f8a43fff640 1 --2- 172.21.15.5:0/3691229815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2826020c0 0x55e2826044b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0ed8f0 tx=0x7f8a2c0b8340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae37bbb6612d6626 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.091+0000 7f8a3affd640 1 -- 172.21.15.5:0/3691229815 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c049030 con 0x55e2826020c0 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.091+0000 7f8a494d0640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8a300a33b0 0x7f8a300a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009fd0 tx=0x7f8a3c004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.091+0000 7f8a3affd640 1 -- 172.21.15.5:0/3691229815 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a2c0dc070 con 0x55e2826020c0 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.091+0000 7f8a3affd640 1 -- 172.21.15.5:0/3691229815 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0bc3c0 con 0x55e2826020c0 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.092+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/3691229815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2826020c0 msgr2=0x55e2826044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.092+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/3691229815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2826020c0 0x55e2826044b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0ed8f0 tx=0x7f8a2c0b8340 comp rx=0 tx=0).stop 2024-09-19T06:12:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.092+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/3691229815 shutdown_connections 2024-09-19T06:12:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.092+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/3691229815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2826020c0 0x55e2826044b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.092+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/3691229815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f32f0 0x55e282601b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.092+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/3691229815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2825f29d0 0x55e2825f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.092+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/3691229815 >> 172.21.15.5:0/3691229815 conn(0x55e2825ef060 msgr2=0x55e2825ef440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.092+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/3691229815 shutdown_connections 2024-09-19T06:12:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.092+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/3691229815 wait complete. 2024-09-19T06:12:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.092+0000 7f8a4dcc9740 1 -- start start 2024-09-19T06:12:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.092+0000 7f8a4dcc9740 1 Processor -- start 2024-09-19T06:12:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a4dcc9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f29d0 0x55e2825fef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a4dcc9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f32f0 0x55e2825ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a4dcc9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2826020c0 0x55e2825ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a4dcc9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e282604ab0 con 0x55e2825f32f0 2024-09-19T06:12:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a4dcc9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e2824e0290 con 0x55e2825f29d0 2024-09-19T06:12:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a4dcc9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e282604c30 con 0x55e2826020c0 2024-09-19T06:12:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a4dcc9740 10 client.0 _ll_register_callbacks cb 0x55e282511140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f32f0 0x55e2825ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f32f0 0x55e2825ff4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35722/0 (socket says 172.21.15.5:35722) 2024-09-19T06:12:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a43fff640 1 -- 172.21.15.5:0/2857319046 learned_addr learned my addr 172.21.15.5:0/2857319046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a494d0640 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2826020c0 0x55e2825ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a48ccf640 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f29d0 0x55e2825fef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a43fff640 1 -- 172.21.15.5:0/2857319046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2826020c0 msgr2=0x55e2825ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a43fff640 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2826020c0 0x55e2825ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a43fff640 1 -- 172.21.15.5:0/2857319046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f29d0 msgr2=0x55e2825fef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a43fff640 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f29d0 0x55e2825fef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a43fff640 1 -- 172.21.15.5:0/2857319046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e28226a210 con 0x55e2825f32f0 2024-09-19T06:12:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a43fff640 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f32f0 0x55e2825ff4b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0b8a50 tx=0x7f8a2c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ec0024e64d0ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a23fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.093+0000 7f8a23fff640 1 -- 172.21.15.5:0/2857319046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c049070 con 0x55e2825f32f0 2024-09-19T06:12:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a23fff640 1 -- 172.21.15.5:0/2857319046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f8a2c0d0720 con 0x55e2825f32f0 2024-09-19T06:12:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/2857319046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a2c0dc070 con 0x55e2825f32f0 2024-09-19T06:12:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a23fff640 1 -- 172.21.15.5:0/2857319046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8a2c0b8040 con 0x55e2825f32f0 2024-09-19T06:12:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a4dcc9740 10 client.17772 fetch_fsmap learned FSMap version 34 2024-09-19T06:12:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/2857319046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e282336620 con 0x55e2825f32f0 2024-09-19T06:12:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a23fff640 1 -- 172.21.15.5:0/2857319046 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a2c0dc070 con 0x55e2825f32f0 2024-09-19T06:12:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a23fff640 1 -- 172.21.15.5:0/2857319046 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f8a2c0ecaf0 con 0x55e2825f32f0 2024-09-19T06:12:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a4dcc9740 10 client.17772 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T06:12:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a4dcc9740 10 client.17772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a4dcc9740 20 client.17772 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e282611bb0 0x55e282613fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a4dcc9740 4 client.17772 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/2857319046 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e282564a20 con 0x55e282611bb0 2024-09-19T06:12:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.094+0000 7f8a48ccf640 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e282611bb0 0x55e282613fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.095+0000 7f8a23fff640 10 client.17772 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.095+0000 7f8a48ccf640 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e282611bb0 0x55e282613fa0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc3606210cc47d61 server_cookie=91d6a3d2fde2b521 in_seq=0 out_seq=0 2024-09-19T06:12:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.095+0000 7f8a23fff640 1 -- 172.21.15.5:0/2857319046 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e282564a20 con 0x55e282611bb0 2024-09-19T06:12:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.095+0000 7f8a23fff640 10 client.17772 handle_command_reply: tid=0 2024-09-19T06:12:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.095+0000 7f8a4dcc9740 1 client.17772 shutdown 2024-09-19T06:12:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.096+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/2857319046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f32f0 msgr2=0x55e2825ff4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.096+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f32f0 0x55e2825ff4b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0b8a50 tx=0x7f8a2c003f90 comp rx=0 tx=0).stop 2024-09-19T06:12:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.096+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/2857319046 shutdown_connections 2024-09-19T06:12:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.096+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e282611bb0 0x55e282613fa0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.096+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2826020c0 0x55e2825ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.096+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f32f0 0x55e2825ff4b0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.096+0000 7f8a23fff640 0 client.17772 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.096+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/2857319046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f29d0 0x55e2825fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.096+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/2857319046 >> 172.21.15.5:0/2857319046 conn(0x55e2825ef060 msgr2=0x55e2825f4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.096+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/2857319046 shutdown_connections 2024-09-19T06:12:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.096+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/2857319046 wait complete. 2024-09-19T06:12:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.096+0000 7f8a4dcc9740 20 client.17772 trim_cache size 0 max 16384 2024-09-19T06:12:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a4dcc9740 1 Processor -- start 2024-09-19T06:12:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a4dcc9740 1 -- start start 2024-09-19T06:12:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a4dcc9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 0x55e282619840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a4dcc9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f32f0 0x55e282619d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a4dcc9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2826020c0 0x55e2825fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a4dcc9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441ab5e0 con 0x55e2825f29d0 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a4dcc9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441ad120 con 0x55e2825f32f0 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a4dcc9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441abcb0 con 0x55e2826020c0 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a48ccf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 0x55e282619840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a48ccf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 0x55e282619840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35730/0 (socket says 172.21.15.5:35730) 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a48ccf640 1 -- 172.21.15.5:0/1692991193 learned_addr learned my addr 172.21.15.5:0/1692991193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f32f0 0x55e282619d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a494d0640 1 --2- 172.21.15.5:0/1692991193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2826020c0 0x55e2825fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a43fff640 1 -- 172.21.15.5:0/1692991193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2826020c0 msgr2=0x55e2825fc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a43fff640 1 --2- 172.21.15.5:0/1692991193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2826020c0 0x55e2825fc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a43fff640 1 -- 172.21.15.5:0/1692991193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 msgr2=0x55e282619840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a43fff640 1 --2- 172.21.15.5:0/1692991193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 0x55e282619840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a43fff640 1 -- 172.21.15.5:0/1692991193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e282614310 con 0x55e2825f32f0 2024-09-19T06:12:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a48ccf640 1 --2- 172.21.15.5:0/1692991193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 0x55e282619840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.114+0000 7f8a43fff640 1 --2- 172.21.15.5:0/1692991193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f32f0 0x55e282619d80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0bc580 tx=0x7f8a2c001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8654dfd0c312b91a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.115+0000 7f8a3a7fc640 1 -- 172.21.15.5:0/1692991193 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c049070 con 0x55e2825f32f0 2024-09-19T06:12:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.115+0000 7f8a3a7fc640 1 -- 172.21.15.5:0/1692991193 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a2c0dc040 con 0x55e2825f32f0 2024-09-19T06:12:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.115+0000 7f8a3a7fc640 1 -- 172.21.15.5:0/1692991193 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0c1a80 con 0x55e2825f32f0 2024-09-19T06:12:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.115+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/1692991193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f32f0 msgr2=0x55e282619d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.115+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/1692991193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f32f0 0x55e282619d80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0bc580 tx=0x7f8a2c001e70 comp rx=0 tx=0).stop 2024-09-19T06:12:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.115+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/1692991193 shutdown_connections 2024-09-19T06:12:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.115+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/1692991193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2826020c0 0x55e2825fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.115+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/1692991193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2825f32f0 0x55e282619d80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.115+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/1692991193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 0x55e282619840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.115+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/1692991193 >> 172.21.15.5:0/1692991193 conn(0x55e2825ef060 msgr2=0x55e282600ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.116+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/1692991193 shutdown_connections 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.116+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/1692991193 wait complete. 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.116+0000 7f8a4dcc9740 1 -- start start 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.116+0000 7f8a4dcc9740 1 Processor -- start 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.116+0000 7f8a4dcc9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 0x55e282605eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.116+0000 7f8a4dcc9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2825f32f0 0x55e2826063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.116+0000 7f8a4dcc9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2826020c0 0x55e28267b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.116+0000 7f8a4dcc9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441aca50 con 0x55e2825f29d0 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.116+0000 7f8a4dcc9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441ac380 con 0x55e2826020c0 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.116+0000 7f8a4dcc9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441a95e0 con 0x55e2825f32f0 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.116+0000 7f8a4dcc9740 10 client.0 _ll_register_callbacks cb 0x7f8a34009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a48ccf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 0x55e282605eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a494d0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2826020c0 0x55e28267b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a48ccf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 0x55e282605eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35736/0 (socket says 172.21.15.5:35736) 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a48ccf640 1 -- 172.21.15.5:0/701124961 learned_addr learned my addr 172.21.15.5:0/701124961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a43fff640 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2825f32f0 0x55e2826063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a48ccf640 1 -- 172.21.15.5:0/701124961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2825f32f0 msgr2=0x55e2826063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a48ccf640 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2825f32f0 0x55e2826063f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a48ccf640 1 -- 172.21.15.5:0/701124961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2826020c0 msgr2=0x55e28267b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a48ccf640 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2826020c0 0x55e28267b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a48ccf640 1 -- 172.21.15.5:0/701124961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e282606c80 con 0x55e2825f29d0 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a494d0640 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2826020c0 0x55e28267b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a43fff640 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2825f32f0 0x55e2826063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a48ccf640 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 0x55e282605eb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x55e2826147b0 tx=0x7f8a340045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788ab2efd1aa4d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a23fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a23fff640 1 -- 172.21.15.5:0/701124961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34003ac0 con 0x55e2825f29d0 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a23fff640 1 -- 172.21.15.5:0/701124961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f8a34003c60 con 0x55e2825f29d0 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/701124961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a2c0dc040 con 0x55e2825f29d0 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a23fff640 1 -- 172.21.15.5:0/701124961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8a34003e60 con 0x55e2825f29d0 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a4dcc9740 10 client.17778 fetch_fsmap learned FSMap version 34 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.117+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/701124961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e28267ce90 con 0x55e2825f29d0 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.118+0000 7f8a23fff640 1 -- 172.21.15.5:0/701124961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a2c0dc040 con 0x55e2825f29d0 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.118+0000 7f8a23fff640 1 -- 172.21.15.5:0/701124961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f8a3401e5f0 con 0x55e2825f29d0 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.118+0000 7f8a4dcc9740 10 client.17778 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.118+0000 7f8a4dcc9740 10 client.17778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.118+0000 7f8a4dcc9740 20 client.17778 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.118+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e28267d610 0x55e28267fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.118+0000 7f8a4dcc9740 4 client.17778 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.118+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/701124961 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e28255ed70 con 0x55e28267d610 2024-09-19T06:12:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.118+0000 7f8a43fff640 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e28267d610 0x55e28267fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.119+0000 7f8a23fff640 10 client.17778 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.118+0000 7f8a43fff640 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e28267d610 0x55e28267fa30 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454b2828fd47e487 server_cookie=918cbf7d98aef39e in_seq=0 out_seq=0 2024-09-19T06:12:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a23fff640 1 -- 172.21.15.5:0/701124961 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e28255ed70 con 0x55e28267d610 2024-09-19T06:12:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a23fff640 10 client.17778 handle_command_reply: tid=0 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a4dcc9740 1 client.17778 shutdown 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/701124961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 msgr2=0x55e282605eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 0x55e282605eb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x55e2826147b0 tx=0x7f8a340045e0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/701124961 shutdown_connections 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e28267d610 0x55e28267fa30 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2826020c0 0x55e28267b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a23fff640 0 client.17778 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2825f32f0 0x55e2826063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a4dcc9740 1 --2- 172.21.15.5:0/701124961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2825f29d0 0x55e282605eb0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/701124961 >> 172.21.15.5:0/701124961 conn(0x55e2825ef060 msgr2=0x55e2825fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/701124961 shutdown_connections 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a4dcc9740 1 -- 172.21.15.5:0/701124961 wait complete. 2024-09-19T06:12:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.146+0000 7f8a4dcc9740 20 client.17778 trim_cache size 0 max 16384 2024-09-19T06:12:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:12:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:12:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:12:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:12:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.147+0000 7f8a4af33640 1 -- 172.21.15.5:0/2552458986 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8a300a33b0 msgr2=0x7f8a300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.147+0000 7f8a4af33640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8a300a33b0 0x7f8a300a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009fd0 tx=0x7f8a3c004a40 comp rx=0 tx=0).stop 2024-09-19T06:12:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.147+0000 7f8a4af33640 1 -- 172.21.15.5:0/2552458986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a44101340 msgr2=0x7f8a44198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.147+0000 7f8a4af33640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a44101340 0x7f8a44198850 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c005ab0 tx=0x7f8a2c003b90 comp rx=0 tx=0).stop 2024-09-19T06:12:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.147+0000 7f8a4af33640 1 -- 172.21.15.5:0/2552458986 shutdown_connections 2024-09-19T06:12:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.147+0000 7f8a4af33640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a441069b0 0x7f8a4419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.147+0000 7f8a4af33640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a44101340 0x7f8a44198850 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.147+0000 7f8a4af33640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8a300a33b0 0x7f8a300a5870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.147+0000 7f8a4af33640 1 --2- 172.21.15.5:0/2552458986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a441009a0 0x7f8a44198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.147+0000 7f8a4af33640 1 -- 172.21.15.5:0/2552458986 >> 172.21.15.5:0/2552458986 conn(0x7f8a440779a0 msgr2=0x7f8a44069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.147+0000 7f8a4af33640 1 -- 172.21.15.5:0/2552458986 shutdown_connections 2024-09-19T06:12:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:18.147+0000 7f8a4af33640 1 -- 172.21.15.5:0/2552458986 wait complete. 2024-09-19T06:12:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:18 smithi157 ceph-mon[35346]: pgmap v278: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 992 KiB/s wr, 106 op/s 2024-09-19T06:12:18.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 232092 2.66 MB/sec execute 344 sec latency 25.642 ms 2024-09-19T06:12:18.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:18 smithi005 ceph-mon[30955]: pgmap v278: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 992 KiB/s wr, 106 op/s 2024-09-19T06:12:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:18 smithi049 ceph-mon[34640]: pgmap v278: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 992 KiB/s wr, 106 op/s 2024-09-19T06:12:19.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 232609 2.66 MB/sec execute 345 sec latency 21.278 ms 2024-09-19T06:12:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:20 smithi157 ceph-mon[35346]: pgmap v279: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 952 KiB/s wr, 102 op/s 2024-09-19T06:12:20.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 233079 2.65 MB/sec execute 346 sec latency 42.463 ms 2024-09-19T06:12:20.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:20 smithi005 ceph-mon[30955]: pgmap v279: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 952 KiB/s wr, 102 op/s 2024-09-19T06:12:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:20 smithi049 ceph-mon[34640]: pgmap v279: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 952 KiB/s wr, 102 op/s 2024-09-19T06:12:21.835 INFO:tasks.workunit.client.0.smithi005.stdout: 1 233873 2.67 MB/sec execute 347 sec latency 15.166 ms 2024-09-19T06:12:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:22 smithi157 ceph-mon[35346]: pgmap v280: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 827 KiB/s wr, 88 op/s 2024-09-19T06:12:22.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 234220 2.67 MB/sec execute 348 sec latency 77.596 ms 2024-09-19T06:12:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:22 smithi005 ceph-mon[30955]: pgmap v280: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 827 KiB/s wr, 88 op/s 2024-09-19T06:12:22.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:22 smithi049 ceph-mon[34640]: pgmap v280: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 827 KiB/s wr, 88 op/s 2024-09-19T06:12:23.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 234220 2.66 MB/sec execute 349 sec latency 1077.724 ms 2024-09-19T06:12:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:24 smithi157 ceph-mon[35346]: pgmap v281: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-19T06:12:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:24 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:12:24.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 234220 2.65 MB/sec execute 350 sec latency 2077.802 ms 2024-09-19T06:12:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:24 smithi005 ceph-mon[30955]: pgmap v281: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-19T06:12:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:24 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:12:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:24 smithi049 ceph-mon[34640]: pgmap v281: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-19T06:12:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:24 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:12:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:25 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:12:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:25 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:12:25.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 234287 2.65 MB/sec execute 351 sec latency 2709.945 ms 2024-09-19T06:12:25.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:25 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:12:25.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:25 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:12:25.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:25 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:12:25.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:25 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:12:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:26 smithi157 ceph-mon[35346]: pgmap v282: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 906 KiB/s wr, 99 op/s 2024-09-19T06:12:26.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 234287 2.64 MB/sec execute 352 sec latency 1101.578 ms 2024-09-19T06:12:26.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:26 smithi005 ceph-mon[30955]: pgmap v282: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 906 KiB/s wr, 99 op/s 2024-09-19T06:12:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:26 smithi049 ceph-mon[34640]: pgmap v282: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 906 KiB/s wr, 99 op/s 2024-09-19T06:12:27.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 234287 2.63 MB/sec execute 353 sec latency 2101.686 ms 2024-09-19T06:12:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:28 smithi157 ceph-mon[35346]: pgmap v283: 97 pgs: 97 active+clean; 131 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-19T06:12:28.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 234287 2.62 MB/sec execute 354 sec latency 3101.823 ms 2024-09-19T06:12:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:28 smithi005 ceph-mon[30955]: pgmap v283: 97 pgs: 97 active+clean; 131 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-19T06:12:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:28 smithi049 ceph-mon[34640]: pgmap v283: 97 pgs: 97 active+clean; 131 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-19T06:12:29.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 234287 2.62 MB/sec execute 355 sec latency 4101.906 ms 2024-09-19T06:12:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:30 smithi157 ceph-mon[35346]: pgmap v284: 97 pgs: 97 active+clean; 131 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 839 KiB/s wr, 85 op/s 2024-09-19T06:12:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:30 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:12:30.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 234395 2.61 MB/sec execute 356 sec latency 4727.820 ms 2024-09-19T06:12:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:30 smithi005 ceph-mon[30955]: pgmap v284: 97 pgs: 97 active+clean; 131 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 839 KiB/s wr, 85 op/s 2024-09-19T06:12:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:30 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:12:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:30 smithi049 ceph-mon[34640]: pgmap v284: 97 pgs: 97 active+clean; 131 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 839 KiB/s wr, 85 op/s 2024-09-19T06:12:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:30 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:12:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.245+0000 7fa3116ac640 1 -- 172.21.15.5:0/742272861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30c1069b0 msgr2=0x7fa30c106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.245+0000 7fa3116ac640 1 --2- 172.21.15.5:0/742272861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30c1069b0 0x7fa30c106d90 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009930 tx=0x7fa2fc02a420 comp rx=0 tx=0).stop 2024-09-19T06:12:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.245+0000 7fa3116ac640 1 -- 172.21.15.5:0/742272861 shutdown_connections 2024-09-19T06:12:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.245+0000 7fa3116ac640 1 --2- 172.21.15.5:0/742272861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa30c101340 0x7fa30c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.245+0000 7fa3116ac640 1 --2- 172.21.15.5:0/742272861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c1009a0 0x7fa30c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.245+0000 7fa3116ac640 1 --2- 172.21.15.5:0/742272861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30c1069b0 0x7fa30c106d90 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.245+0000 7fa3116ac640 1 -- 172.21.15.5:0/742272861 >> 172.21.15.5:0/742272861 conn(0x7fa30c0779a0 msgr2=0x7fa30c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.245+0000 7fa3116ac640 1 -- 172.21.15.5:0/742272861 shutdown_connections 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.245+0000 7fa3116ac640 1 -- 172.21.15.5:0/742272861 wait complete. 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.245+0000 7fa3116ac640 1 Processor -- start 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa3116ac640 1 -- start start 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa3116ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30c1009a0 0x7fa30c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa3116ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c101340 0x7fa30c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa3116ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa30c1069b0 0x7fa30c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa3116ac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c110ca0 con 0x7fa30c1009a0 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa3116ac640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c110b20 con 0x7fa30c1069b0 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa3116ac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c110e20 con 0x7fa30c101340 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa30affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30c1009a0 0x7fa30c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa30affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30c1009a0 0x7fa30c198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55410/0 (socket says 172.21.15.5:55410) 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa30affd640 1 -- 172.21.15.5:0/501060999 learned_addr learned my addr 172.21.15.5:0/501060999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa30b7fe640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa30c1069b0 0x7fa30c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa303fff640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c101340 0x7fa30c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa30affd640 1 -- 172.21.15.5:0/501060999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c101340 msgr2=0x7fa30c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa30affd640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c101340 0x7fa30c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa30affd640 1 -- 172.21.15.5:0/501060999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa30c1069b0 msgr2=0x7fa30c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa30affd640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa30c1069b0 0x7fa30c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa30affd640 1 -- 172.21.15.5:0/501060999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c19d300 con 0x7fa30c1009a0 2024-09-19T06:12:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa303fff640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c101340 0x7fa30c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.246+0000 7fa30b7fe640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa30c1069b0 0x7fa30c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.247+0000 7fa30affd640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30c1009a0 0x7fa30c198310 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc005940 tx=0x7fa2fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.247+0000 7fa308ff9640 1 -- 172.21.15.5:0/501060999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc03f030 con 0x7fa30c1009a0 2024-09-19T06:12:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.247+0000 7fa308ff9640 1 -- 172.21.15.5:0/501060999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2fc0318a0 con 0x7fa30c1009a0 2024-09-19T06:12:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.247+0000 7fa3116ac640 1 -- 172.21.15.5:0/501060999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30c19d590 con 0x7fa30c1009a0 2024-09-19T06:12:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.247+0000 7fa308ff9640 1 -- 172.21.15.5:0/501060999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc035070 con 0x7fa30c1009a0 2024-09-19T06:12:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.247+0000 7fa3116ac640 1 -- 172.21.15.5:0/501060999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30c1a4e80 con 0x7fa30c1009a0 2024-09-19T06:12:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.249+0000 7fa31444a740 1 Processor -- start 2024-09-19T06:12:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa308ff9640 1 -- 172.21.15.5:0/501060999 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa2fc042030 con 0x7fa30c1009a0 2024-09-19T06:12:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa31444a740 1 -- start start 2024-09-19T06:12:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa31444a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72b9d0 0x55946b72bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa31444a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b72c2f0 0x55946b73ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa31444a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b73b0c0 0x55946b73d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa31444a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55946b6902f0 con 0x55946b72c2f0 2024-09-19T06:12:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa31444a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55946b638270 con 0x55946b73b0c0 2024-09-19T06:12:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa31444a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55946b68c2b0 con 0x55946b72b9d0 2024-09-19T06:12:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa30b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72b9d0 0x55946b72bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa30b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72b9d0 0x55946b72bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:51014/0 (socket says 172.21.15.5:51014) 2024-09-19T06:12:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa303fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b73b0c0 0x55946b73d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa30affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b72c2f0 0x55946b73ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.250+0000 7fa30b7fe640 1 -- 172.21.15.5:0/1679762933 learned_addr learned my addr 172.21.15.5:0/1679762933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.251+0000 7fa303fff640 1 -- 172.21.15.5:0/1679762933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72b9d0 msgr2=0x55946b72bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.251+0000 7fa303fff640 1 --2- 172.21.15.5:0/1679762933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72b9d0 0x55946b72bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.251+0000 7fa303fff640 1 -- 172.21.15.5:0/1679762933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b72c2f0 msgr2=0x55946b73ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.251+0000 7fa303fff640 1 --2- 172.21.15.5:0/1679762933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b72c2f0 0x55946b73ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.251+0000 7fa303fff640 1 -- 172.21.15.5:0/1679762933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946b637340 con 0x55946b73b0c0 2024-09-19T06:12:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.251+0000 7fa308ff9640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2f80a3430 0x7fa2f80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.251+0000 7fa303fff640 1 --2- 172.21.15.5:0/1679762933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b73b0c0 0x55946b73d4b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400bc80 tx=0x7fa2f400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1046abb9de4e241d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.251+0000 7fa30b7fe640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2f80a3430 0x7fa2f80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.251+0000 7fa308ff9640 1 -- 172.21.15.5:0/501060999 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7fa2fc0f0500 con 0x7fa30c1009a0 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.251+0000 7fa3017fa640 1 -- 172.21.15.5:0/1679762933 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4015070 con 0x55946b73b0c0 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.251+0000 7fa3017fa640 1 -- 172.21.15.5:0/1679762933 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2f40118f0 con 0x55946b73b0c0 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.251+0000 7fa3017fa640 1 -- 172.21.15.5:0/1679762933 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4010890 con 0x55946b73b0c0 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.252+0000 7fa31444a740 1 -- 172.21.15.5:0/1679762933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b73b0c0 msgr2=0x55946b73d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.252+0000 7fa30b7fe640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2f80a3430 0x7fa2f80a58f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa304009fd0 tx=0x7fa304009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.252+0000 7fa31444a740 1 --2- 172.21.15.5:0/1679762933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b73b0c0 0x55946b73d4b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400bc80 tx=0x7fa2f400f730 comp rx=0 tx=0).stop 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.252+0000 7fa31444a740 1 -- 172.21.15.5:0/1679762933 shutdown_connections 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.252+0000 7fa31444a740 1 --2- 172.21.15.5:0/1679762933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b73b0c0 0x55946b73d4b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.252+0000 7fa31444a740 1 --2- 172.21.15.5:0/1679762933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b72c2f0 0x55946b73ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.252+0000 7fa31444a740 1 --2- 172.21.15.5:0/1679762933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72b9d0 0x55946b72bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.252+0000 7fa31444a740 1 -- 172.21.15.5:0/1679762933 >> 172.21.15.5:0/1679762933 conn(0x55946b728060 msgr2=0x55946b728440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.252+0000 7fa31444a740 1 -- 172.21.15.5:0/1679762933 shutdown_connections 2024-09-19T06:12:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.252+0000 7fa31444a740 1 -- 172.21.15.5:0/1679762933 wait complete. 2024-09-19T06:12:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.252+0000 7fa31444a740 1 -- start start 2024-09-19T06:12:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa31444a740 1 Processor -- start 2024-09-19T06:12:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa31444a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b738080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa31444a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72c2f0 0x55946b7385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa31444a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 0x55946b738b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa31444a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55946b73dcf0 con 0x55946b73b0c0 2024-09-19T06:12:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa31444a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55946b619290 con 0x55946b72b9d0 2024-09-19T06:12:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa31444a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55946b73de70 con 0x55946b72c2f0 2024-09-19T06:12:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa31444a740 10 client.0 _ll_register_callbacks cb 0x55946b64a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa30b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 0x55946b738b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa30b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 0x55946b738b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55422/0 (socket says 172.21.15.5:55422) 2024-09-19T06:12:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa30affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b738080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa30b7fe640 1 -- 172.21.15.5:0/1909285768 learned_addr learned my addr 172.21.15.5:0/1909285768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa303fff640 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72c2f0 0x55946b7385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa30b7fe640 1 -- 172.21.15.5:0/1909285768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72c2f0 msgr2=0x55946b7385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa30b7fe640 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72c2f0 0x55946b7385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa30b7fe640 1 -- 172.21.15.5:0/1909285768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 msgr2=0x55946b738080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa30b7fe640 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b738080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa30b7fe640 1 -- 172.21.15.5:0/1909285768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946b3a3210 con 0x55946b73b0c0 2024-09-19T06:12:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa30affd640 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b738080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.253+0000 7fa303fff640 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72c2f0 0x55946b7385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.254+0000 7fa30b7fe640 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 0x55946b738b00 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa304012010 tx=0x7fa304004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e00acf1a4c6871 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.254+0000 7fa2f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.254+0000 7fa2f27fc640 1 -- 172.21.15.5:0/1909285768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30401fc80 con 0x55946b73b0c0 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.254+0000 7fa2f27fc640 1 -- 172.21.15.5:0/1909285768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7fa304017030 con 0x55946b73b0c0 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.254+0000 7fa31444a740 1 -- 172.21.15.5:0/1909285768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f40118f0 con 0x55946b73b0c0 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.254+0000 7fa2f27fc640 1 -- 172.21.15.5:0/1909285768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa30401fe20 con 0x55946b73b0c0 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.254+0000 7fa31444a740 10 client.17790 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.254+0000 7fa31444a740 1 -- 172.21.15.5:0/1909285768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55946b46f620 con 0x55946b73b0c0 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.254+0000 7fa2f27fc640 1 -- 172.21.15.5:0/1909285768 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2f40118f0 con 0x55946b73b0c0 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.254+0000 7fa2f27fc640 1 -- 172.21.15.5:0/1909285768 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fa304027410 con 0x55946b73b0c0 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.255+0000 7fa31444a740 10 client.17790 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.255+0000 7fa31444a740 10 client.17790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.255+0000 7fa31444a740 20 client.17790 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.255+0000 7fa31444a740 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55946b74b1f0 0x55946b74d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.255+0000 7fa31444a740 4 client.17790 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.255+0000 7fa31444a740 1 -- 172.21.15.5:0/1909285768 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55946b69da20 con 0x55946b74b1f0 2024-09-19T06:12:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.255+0000 7fa30affd640 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55946b74b1f0 0x55946b74d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.255+0000 7fa2f27fc640 10 client.17790 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.255+0000 7fa30affd640 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55946b74b1f0 0x55946b74d640 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2201231e698a2f3 server_cookie=a42a6bbb29668aa3 in_seq=0 out_seq=0 2024-09-19T06:12:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa2f27fc640 1 -- 172.21.15.5:0/1909285768 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55946b69da20 con 0x55946b74b1f0 2024-09-19T06:12:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa2f27fc640 10 client.17790 handle_command_reply: tid=0 2024-09-19T06:12:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa31444a740 1 client.17790 shutdown 2024-09-19T06:12:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa31444a740 1 -- 172.21.15.5:0/1909285768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 msgr2=0x55946b738b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa31444a740 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 0x55946b738b00 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa304012010 tx=0x7fa304004ab0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa31444a740 1 -- 172.21.15.5:0/1909285768 shutdown_connections 2024-09-19T06:12:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa31444a740 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55946b74b1f0 0x55946b74d640 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa2f27fc640 0 client.17790 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa31444a740 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 0x55946b738b00 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa31444a740 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72c2f0 0x55946b7385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa31444a740 1 --2- 172.21.15.5:0/1909285768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b738080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa31444a740 1 -- 172.21.15.5:0/1909285768 >> 172.21.15.5:0/1909285768 conn(0x55946b728060 msgr2=0x55946b7271c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa31444a740 1 -- 172.21.15.5:0/1909285768 shutdown_connections 2024-09-19T06:12:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa31444a740 1 -- 172.21.15.5:0/1909285768 wait complete. 2024-09-19T06:12:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.256+0000 7fa31444a740 20 client.17790 trim_cache size 0 max 16384 2024-09-19T06:12:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa31444a740 1 Processor -- start 2024-09-19T06:12:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa31444a740 1 -- start start 2024-09-19T06:12:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa31444a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b727b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa31444a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72c2f0 0x55946b752f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa31444a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 0x55946b753490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa31444a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c1ab560 con 0x55946b73b0c0 2024-09-19T06:12:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa31444a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c1ad0d0 con 0x55946b72b9d0 2024-09-19T06:12:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa31444a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c1abc30 con 0x55946b72c2f0 2024-09-19T06:12:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa30b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 0x55946b753490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa30b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 0x55946b753490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55430/0 (socket says 172.21.15.5:55430) 2024-09-19T06:12:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa30affd640 1 --2- 172.21.15.5:0/3112627515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b727b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa30b7fe640 1 -- 172.21.15.5:0/3112627515 learned_addr learned my addr 172.21.15.5:0/3112627515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.274+0000 7fa303fff640 1 --2- 172.21.15.5:0/3112627515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72c2f0 0x55946b752f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.275+0000 7fa30affd640 1 -- 172.21.15.5:0/3112627515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72c2f0 msgr2=0x55946b752f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.275+0000 7fa30affd640 1 --2- 172.21.15.5:0/3112627515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72c2f0 0x55946b752f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.275+0000 7fa30affd640 1 -- 172.21.15.5:0/3112627515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 msgr2=0x55946b753490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.275+0000 7fa30affd640 1 --2- 172.21.15.5:0/3112627515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 0x55946b753490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.275+0000 7fa30affd640 1 -- 172.21.15.5:0/3112627515 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946b73ab80 con 0x55946b72b9d0 2024-09-19T06:12:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.275+0000 7fa30affd640 1 --2- 172.21.15.5:0/3112627515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b727b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc0edc30 tx=0x7fa2fc0f60e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace279acddbd7189 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.275+0000 7fa300ff9640 1 -- 172.21.15.5:0/3112627515 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc049040 con 0x55946b72b9d0 2024-09-19T06:12:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.275+0000 7fa300ff9640 1 -- 172.21.15.5:0/3112627515 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2fc003e20 con 0x55946b72b9d0 2024-09-19T06:12:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.275+0000 7fa300ff9640 1 -- 172.21.15.5:0/3112627515 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc0c0b70 con 0x55946b72b9d0 2024-09-19T06:12:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.276+0000 7fa31444a740 1 -- 172.21.15.5:0/3112627515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 msgr2=0x55946b727b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.276+0000 7fa31444a740 1 --2- 172.21.15.5:0/3112627515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b727b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc0edc30 tx=0x7fa2fc0f60e0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.276+0000 7fa31444a740 1 -- 172.21.15.5:0/3112627515 shutdown_connections 2024-09-19T06:12:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.276+0000 7fa31444a740 1 --2- 172.21.15.5:0/3112627515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b73b0c0 0x55946b753490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.276+0000 7fa31444a740 1 --2- 172.21.15.5:0/3112627515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b72c2f0 0x55946b752f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.276+0000 7fa31444a740 1 --2- 172.21.15.5:0/3112627515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b727b00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.276+0000 7fa31444a740 1 -- 172.21.15.5:0/3112627515 >> 172.21.15.5:0/3112627515 conn(0x55946b728060 msgr2=0x55946b7271c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.276+0000 7fa31444a740 1 -- 172.21.15.5:0/3112627515 shutdown_connections 2024-09-19T06:12:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.276+0000 7fa31444a740 1 -- 172.21.15.5:0/3112627515 wait complete. 2024-09-19T06:12:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.276+0000 7fa31444a740 1 -- start start 2024-09-19T06:12:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.276+0000 7fa31444a740 1 Processor -- start 2024-09-19T06:12:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa31444a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b73efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa31444a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b72c2f0 0x55946b73f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa31444a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b73b0c0 0x55946b73fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa31444a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c1aca00 con 0x55946b72c2f0 2024-09-19T06:12:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa31444a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c1ac300 con 0x55946b72b9d0 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa31444a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c1a9560 con 0x55946b73b0c0 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa31444a740 10 client.0 _ll_register_callbacks cb 0x7fa304009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa303fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b72c2f0 0x55946b73f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa30affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b73efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa303fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b72c2f0 0x55946b73f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55440/0 (socket says 172.21.15.5:55440) 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa303fff640 1 -- 172.21.15.5:0/2743274539 learned_addr learned my addr 172.21.15.5:0/2743274539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa30b7fe640 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b73b0c0 0x55946b73fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa30affd640 1 -- 172.21.15.5:0/2743274539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b73b0c0 msgr2=0x55946b73fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa30affd640 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b73b0c0 0x55946b73fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa30affd640 1 -- 172.21.15.5:0/2743274539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b72c2f0 msgr2=0x55946b73f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa30affd640 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b72c2f0 0x55946b73f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa30affd640 1 -- 172.21.15.5:0/2743274539 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946b7b5120 con 0x55946b72b9d0 2024-09-19T06:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa303fff640 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b72c2f0 0x55946b73f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa30affd640 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b73efe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009900 tx=0x7fa2fc0bc8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=889098fff81991a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa2f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa2f27fc640 1 -- 172.21.15.5:0/2743274539 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc049070 con 0x55946b72b9d0 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa2f27fc640 1 -- 172.21.15.5:0/2743274539 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7fa2fc0b8e30 con 0x55946b72b9d0 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.277+0000 7fa31444a740 1 -- 172.21.15.5:0/2743274539 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55946b697d70 con 0x55946b72b9d0 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.278+0000 7fa2f27fc640 1 -- 172.21.15.5:0/2743274539 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa2fc03b460 con 0x55946b72b9d0 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.278+0000 7fa31444a740 10 client.27347 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.278+0000 7fa31444a740 1 -- 172.21.15.5:0/2743274539 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55946b7b63d0 con 0x55946b72b9d0 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.278+0000 7fa2f27fc640 1 -- 172.21.15.5:0/2743274539 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55946b697d70 con 0x55946b72b9d0 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.278+0000 7fa2f27fc640 1 -- 172.21.15.5:0/2743274539 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fa2fc0d0ce0 con 0x55946b72b9d0 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.278+0000 7fa31444a740 10 client.27347 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.278+0000 7fa31444a740 10 client.27347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.278+0000 7fa31444a740 20 client.27347 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.278+0000 7fa31444a740 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55946b74b1f0 0x55946b7b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.278+0000 7fa31444a740 4 client.27347 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:12:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.278+0000 7fa31444a740 1 -- 172.21.15.5:0/2743274539 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55946b614900 con 0x55946b74b1f0 2024-09-19T06:12:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.278+0000 7fa303fff640 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55946b74b1f0 0x55946b7b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.279+0000 7fa303fff640 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55946b74b1f0 0x55946b7b8a90 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2090c999813fb46 server_cookie=e3958ccdd04b977b in_seq=0 out_seq=0 2024-09-19T06:12:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.279+0000 7fa2f27fc640 10 client.27347 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.303+0000 7fa2f27fc640 1 -- 172.21.15.5:0/2743274539 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55946b614900 con 0x55946b74b1f0 2024-09-19T06:12:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.303+0000 7fa2f27fc640 10 client.27347 handle_command_reply: tid=0 2024-09-19T06:12:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa31444a740 1 client.27347 shutdown 2024-09-19T06:12:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa31444a740 1 -- 172.21.15.5:0/2743274539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 msgr2=0x55946b73efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa31444a740 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b73efe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009900 tx=0x7fa2fc0bc8a0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa31444a740 1 -- 172.21.15.5:0/2743274539 shutdown_connections 2024-09-19T06:12:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa31444a740 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55946b74b1f0 0x55946b7b8a90 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa31444a740 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55946b73b0c0 0x55946b73fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa2f27fc640 0 client.27347 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa31444a740 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55946b72c2f0 0x55946b73f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa31444a740 1 --2- 172.21.15.5:0/2743274539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55946b72b9d0 0x55946b73efe0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa31444a740 1 -- 172.21.15.5:0/2743274539 >> 172.21.15.5:0/2743274539 conn(0x55946b728060 msgr2=0x55946b72d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa31444a740 1 -- 172.21.15.5:0/2743274539 shutdown_connections 2024-09-19T06:12:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa31444a740 1 -- 172.21.15.5:0/2743274539 wait complete. 2024-09-19T06:12:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa31444a740 20 client.27347 trim_cache size 0 max 16384 2024-09-19T06:12:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:12:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:12:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:12:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:12:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa3116ac640 1 -- 172.21.15.5:0/501060999 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2f80a3430 msgr2=0x7fa2f80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.304+0000 7fa3116ac640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2f80a3430 0x7fa2f80a58f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa304009fd0 tx=0x7fa304009500 comp rx=0 tx=0).stop 2024-09-19T06:12:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.305+0000 7fa3116ac640 1 -- 172.21.15.5:0/501060999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30c1009a0 msgr2=0x7fa30c198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.305+0000 7fa3116ac640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30c1009a0 0x7fa30c198310 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc005940 tx=0x7fa2fc0042f0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.305+0000 7fa3116ac640 1 -- 172.21.15.5:0/501060999 shutdown_connections 2024-09-19T06:12:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.305+0000 7fa3116ac640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa30c1069b0 0x7fa30c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.305+0000 7fa3116ac640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2f80a3430 0x7fa2f80a58f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.305+0000 7fa3116ac640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c101340 0x7fa30c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.305+0000 7fa3116ac640 1 --2- 172.21.15.5:0/501060999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa30c1009a0 0x7fa30c198310 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.305+0000 7fa3116ac640 1 -- 172.21.15.5:0/501060999 >> 172.21.15.5:0/501060999 conn(0x7fa30c0779a0 msgr2=0x7fa30c069bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.305+0000 7fa3116ac640 1 -- 172.21.15.5:0/501060999 shutdown_connections 2024-09-19T06:12:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:31.305+0000 7fa3116ac640 1 -- 172.21.15.5:0/501060999 wait complete. 2024-09-19T06:12:31.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 234623 2.60 MB/sec execute 357 sec latency 15.340 ms 2024-09-19T06:12:31.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:31 smithi005 ceph-mon[30955]: pgmap v285: 97 pgs: 97 active+clean; 131 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 839 KiB/s wr, 85 op/s 2024-09-19T06:12:31.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:31 smithi049 ceph-mon[34640]: pgmap v285: 97 pgs: 97 active+clean; 131 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 839 KiB/s wr, 85 op/s 2024-09-19T06:12:32.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:31 smithi157 ceph-mon[35346]: pgmap v285: 97 pgs: 97 active+clean; 131 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 839 KiB/s wr, 85 op/s 2024-09-19T06:12:32.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 235043 2.60 MB/sec execute 358 sec latency 16.973 ms 2024-09-19T06:12:33.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 235559 2.60 MB/sec execute 359 sec latency 22.226 ms 2024-09-19T06:12:34.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:33 smithi005 ceph-mon[30955]: pgmap v286: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.0 MiB/s wr, 111 op/s 2024-09-19T06:12:34.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:33 smithi049 ceph-mon[34640]: pgmap v286: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.0 MiB/s wr, 111 op/s 2024-09-19T06:12:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:33 smithi157 ceph-mon[35346]: pgmap v286: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.0 MiB/s wr, 111 op/s 2024-09-19T06:12:34.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 236065 2.60 MB/sec execute 360 sec latency 20.023 ms 2024-09-19T06:12:35.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 236619 2.60 MB/sec execute 361 sec latency 20.026 ms 2024-09-19T06:12:36.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:35 smithi005 ceph-mon[30955]: pgmap v287: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 386 KiB/s wr, 48 op/s 2024-09-19T06:12:36.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:35 smithi049 ceph-mon[34640]: pgmap v287: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 386 KiB/s wr, 48 op/s 2024-09-19T06:12:36.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:35 smithi157 ceph-mon[35346]: pgmap v287: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 386 KiB/s wr, 48 op/s 2024-09-19T06:12:36.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237354 2.61 MB/sec execute 362 sec latency 15.879 ms 2024-09-19T06:12:37.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:36 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:12:37.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:36 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:37.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:36 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:12:37.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:36 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:12:37.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:36 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:12:37.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:36 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:37.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:36 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:12:37.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:36 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:12:37.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:36 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:12:37.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:36 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:12:37.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:36 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:12:37.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:36 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:12:37.836 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237764 2.61 MB/sec execute 363 sec latency 54.867 ms 2024-09-19T06:12:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:38 smithi157 ceph-mon[35346]: pgmap v288: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:12:38.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237764 2.60 MB/sec execute 364 sec latency 1054.972 ms 2024-09-19T06:12:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:38 smithi005 ceph-mon[30955]: pgmap v288: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:12:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:38 smithi049 ceph-mon[34640]: pgmap v288: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:12:39.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237764 2.60 MB/sec execute 365 sec latency 2055.107 ms 2024-09-19T06:12:40.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:40 smithi005 ceph-mon[30955]: pgmap v289: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 860 KiB/s wr, 83 op/s 2024-09-19T06:12:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:40 smithi157 ceph-mon[35346]: pgmap v289: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 860 KiB/s wr, 83 op/s 2024-09-19T06:12:40.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237831 2.59 MB/sec execute 366 sec latency 2684.088 ms 2024-09-19T06:12:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:40 smithi049 ceph-mon[34640]: pgmap v289: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 860 KiB/s wr, 83 op/s 2024-09-19T06:12:41.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237831 2.58 MB/sec execute 367 sec latency 1117.003 ms 2024-09-19T06:12:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:42 smithi157 ceph-mon[35346]: pgmap v290: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 860 KiB/s wr, 83 op/s 2024-09-19T06:12:42.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237831 2.58 MB/sec execute 368 sec latency 2117.121 ms 2024-09-19T06:12:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:42 smithi005 ceph-mon[30955]: pgmap v290: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 860 KiB/s wr, 83 op/s 2024-09-19T06:12:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:42 smithi049 ceph-mon[34640]: pgmap v290: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 860 KiB/s wr, 83 op/s 2024-09-19T06:12:43.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237831 2.57 MB/sec execute 369 sec latency 3117.250 ms 2024-09-19T06:12:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.406+0000 7f56a7b6f640 1 -- 172.21.15.5:0/1175981763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a01008b0 msgr2=0x7f56a0100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.406+0000 7f56a7b6f640 1 --2- 172.21.15.5:0/1175981763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a01008b0 0x7f56a0100c90 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f569c009900 tx=0x7f569c02a410 comp rx=0 tx=0).stop 2024-09-19T06:12:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.406+0000 7f56a7b6f640 1 -- 172.21.15.5:0/1175981763 shutdown_connections 2024-09-19T06:12:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.406+0000 7f56a7b6f640 1 --2- 172.21.15.5:0/1175981763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a010e460 0x7f56a0110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.406+0000 7f56a7b6f640 1 --2- 172.21.15.5:0/1175981763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a0101260 0x7f56a010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.406+0000 7f56a7b6f640 1 --2- 172.21.15.5:0/1175981763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a01008b0 0x7f56a0100c90 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.406+0000 7f56a7b6f640 1 -- 172.21.15.5:0/1175981763 >> 172.21.15.5:0/1175981763 conn(0x7f56a00fc6b0 msgr2=0x7f56a00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.406+0000 7f56a7b6f640 1 -- 172.21.15.5:0/1175981763 shutdown_connections 2024-09-19T06:12:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.406+0000 7f56a7b6f640 1 -- 172.21.15.5:0/1175981763 wait complete. 2024-09-19T06:12:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a7b6f640 1 Processor -- start 2024-09-19T06:12:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a7b6f640 1 -- start start 2024-09-19T06:12:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a7b6f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a01008b0 0x7f56a019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a7b6f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a0101260 0x7f56a019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a7b6f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a010e460 0x7f56a019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a7b6f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0113f70 con 0x7f56a010e460 2024-09-19T06:12:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a7b6f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0113df0 con 0x7f56a01008b0 2024-09-19T06:12:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a7b6f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01140f0 con 0x7f56a0101260 2024-09-19T06:12:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a610c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a010e460 0x7f56a019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a510a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a0101260 0x7f56a019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a610c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a010e460 0x7f56a019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57398/0 (socket says 172.21.15.5:57398) 2024-09-19T06:12:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a610c640 1 -- 172.21.15.5:0/2896147338 learned_addr learned my addr 172.21.15.5:0/2896147338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.407+0000 7f56a590b640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a01008b0 0x7f56a019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.408+0000 7f56a610c640 1 -- 172.21.15.5:0/2896147338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a0101260 msgr2=0x7f56a019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.408+0000 7f56a610c640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a0101260 0x7f56a019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.408+0000 7f56a610c640 1 -- 172.21.15.5:0/2896147338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a01008b0 msgr2=0x7f56a019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.408+0000 7f56a610c640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a01008b0 0x7f56a019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.408+0000 7f56a610c640 1 -- 172.21.15.5:0/2896147338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a019f590 con 0x7f56a010e460 2024-09-19T06:12:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.408+0000 7f56a610c640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a010e460 0x7f56a019ee60 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f569800eb40 tx=0x7f569800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.408+0000 7f5696ffd640 1 -- 172.21.15.5:0/2896147338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698004b20 con 0x7f56a010e460 2024-09-19T06:12:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.408+0000 7f56a7b6f640 1 -- 172.21.15.5:0/2896147338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a019f880 con 0x7f56a010e460 2024-09-19T06:12:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.408+0000 7f5696ffd640 1 -- 172.21.15.5:0/2896147338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5698004cc0 con 0x7f56a010e460 2024-09-19T06:12:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.408+0000 7f5696ffd640 1 -- 172.21.15.5:0/2896147338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56980107b0 con 0x7f56a010e460 2024-09-19T06:12:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.408+0000 7f56a7b6f640 1 -- 172.21.15.5:0/2896147338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a01a7110 con 0x7f56a010e460 2024-09-19T06:12:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.411+0000 7f5696ffd640 1 -- 172.21.15.5:0/2896147338 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5698005020 con 0x7f56a010e460 2024-09-19T06:12:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.411+0000 7f56aa9e1740 1 Processor -- start 2024-09-19T06:12:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.411+0000 7f56aa9e1740 1 -- start start 2024-09-19T06:12:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.411+0000 7f56aa9e1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4defeaf0 0x562c4defeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.411+0000 7f56aa9e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4deff410 0x562c4df09770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.411+0000 7f56aa9e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4df09cb0 0x562c4df0c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56aa9e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562c4de632f0 con 0x562c4df09cb0 2024-09-19T06:12:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56aa9e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562c4de0b270 con 0x562c4deff410 2024-09-19T06:12:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56aa9e1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562c4de5f2b0 con 0x562c4defeaf0 2024-09-19T06:12:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56a510a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4df09cb0 0x562c4df0c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56a510a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4df09cb0 0x562c4df0c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57400/0 (socket says 172.21.15.5:57400) 2024-09-19T06:12:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56a510a640 1 -- 172.21.15.5:0/1718885821 learned_addr learned my addr 172.21.15.5:0/1718885821 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56a510a640 1 --2- 172.21.15.5:0/1718885821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4defeaf0 0x562c4defeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56a590b640 1 --2- 172.21.15.5:0/1718885821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4deff410 0x562c4df09770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f5696ffd640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f568c0a33b0 0x7f568c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56a510a640 1 -- 172.21.15.5:0/1718885821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4defeaf0 msgr2=0x562c4defeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56a510a640 1 --2- 172.21.15.5:0/1718885821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4defeaf0 0x562c4defeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56a510a640 1 -- 172.21.15.5:0/1718885821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4deff410 msgr2=0x562c4df09770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56a510a640 1 --2- 172.21.15.5:0/1718885821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4deff410 0x562c4df09770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56a510a640 1 -- 172.21.15.5:0/1718885821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c4de0a340 con 0x562c4df09cb0 2024-09-19T06:12:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56a610c640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f568c0a33b0 0x7f568c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f5696ffd640 1 -- 172.21.15.5:0/2896147338 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f56980d0450 con 0x7f56a010e460 2024-09-19T06:12:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.412+0000 7f56a510a640 1 --2- 172.21.15.5:0/1718885821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4df09cb0 0x562c4df0c0a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f569000b6c0 tx=0x7f569000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff332deef58aa23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.413+0000 7f568bfff640 1 -- 172.21.15.5:0/1718885821 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690012d00 con 0x562c4df09cb0 2024-09-19T06:12:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.413+0000 7f568bfff640 1 -- 172.21.15.5:0/1718885821 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5690010770 con 0x562c4df09cb0 2024-09-19T06:12:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.413+0000 7f568bfff640 1 -- 172.21.15.5:0/1718885821 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690003850 con 0x562c4df09cb0 2024-09-19T06:12:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.413+0000 7f56aa9e1740 1 -- 172.21.15.5:0/1718885821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4df09cb0 msgr2=0x562c4df0c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.413+0000 7f56a610c640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f568c0a33b0 0x7f568c0a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f56980ce180 tx=0x7f5698013fd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.413+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/1718885821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4df09cb0 0x562c4df0c0a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f569000b6c0 tx=0x7f569000b650 comp rx=0 tx=0).stop 2024-09-19T06:12:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.413+0000 7f56aa9e1740 1 -- 172.21.15.5:0/1718885821 shutdown_connections 2024-09-19T06:12:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.413+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/1718885821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4df09cb0 0x562c4df0c0a0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.413+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/1718885821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4deff410 0x562c4df09770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.413+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/1718885821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4defeaf0 0x562c4defeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.413+0000 7f56aa9e1740 1 -- 172.21.15.5:0/1718885821 >> 172.21.15.5:0/1718885821 conn(0x562c4defb060 msgr2=0x562c4defb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.413+0000 7f56aa9e1740 1 -- 172.21.15.5:0/1718885821 shutdown_connections 2024-09-19T06:12:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.414+0000 7f56aa9e1740 1 -- 172.21.15.5:0/1718885821 wait complete. 2024-09-19T06:12:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.414+0000 7f56aa9e1740 1 -- start start 2024-09-19T06:12:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.414+0000 7f56aa9e1740 1 Processor -- start 2024-09-19T06:12:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.414+0000 7f56aa9e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 0x562c4df06c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.414+0000 7f56aa9e1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4deff410 0x562c4df07140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.414+0000 7f56aa9e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 0x562c4df07680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.414+0000 7f56aa9e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562c4df0c880 con 0x562c4defeaf0 2024-09-19T06:12:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.414+0000 7f56aa9e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562c4ddec290 con 0x562c4df09cb0 2024-09-19T06:12:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.414+0000 7f56aa9e1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562c4df0ca00 con 0x562c4deff410 2024-09-19T06:12:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.414+0000 7f56aa9e1740 10 client.0 _ll_register_callbacks cb 0x562c4de1d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.414+0000 7f56a510a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 0x562c4df06c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56a510a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 0x562c4df06c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57410/0 (socket says 172.21.15.5:57410) 2024-09-19T06:12:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56a510a640 1 -- 172.21.15.5:0/2275360459 learned_addr learned my addr 172.21.15.5:0/2275360459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56a510a640 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 0x562c4df07680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56a590b640 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4deff410 0x562c4df07140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56a510a640 1 -- 172.21.15.5:0/2275360459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4deff410 msgr2=0x562c4df07140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56a510a640 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4deff410 0x562c4df07140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56a510a640 1 -- 172.21.15.5:0/2275360459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 msgr2=0x562c4df07680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56a510a640 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 0x562c4df07680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56a510a640 1 -- 172.21.15.5:0/2275360459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c4db76210 con 0x562c4defeaf0 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56a510a640 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 0x562c4df06c00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5690010fd0 tx=0x7f569000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bc3f8526ec6889 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56897fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56897fa640 1 -- 172.21.15.5:0/2275360459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690004220 con 0x562c4defeaf0 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56897fa640 1 -- 172.21.15.5:0/2275360459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f56900043c0 con 0x562c4defeaf0 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56aa9e1740 1 -- 172.21.15.5:0/2275360459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5690010770 con 0x562c4defeaf0 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56897fa640 1 -- 172.21.15.5:0/2275360459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5690031050 con 0x562c4defeaf0 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56aa9e1740 10 client.17808 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.415+0000 7f56aa9e1740 1 -- 172.21.15.5:0/2275360459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562c4dc42620 con 0x562c4defeaf0 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.416+0000 7f56897fa640 1 -- 172.21.15.5:0/2275360459 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5690010770 con 0x562c4defeaf0 2024-09-19T06:12:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.416+0000 7f56897fa640 1 -- 172.21.15.5:0/2275360459 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f5690029020 con 0x562c4defeaf0 2024-09-19T06:12:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.416+0000 7f56aa9e1740 10 client.17808 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.416+0000 7f56aa9e1740 10 client.17808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.416+0000 7f56aa9e1740 20 client.17808 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.416+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562c4df19b00 0x562c4df1bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.416+0000 7f56aa9e1740 4 client.17808 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.416+0000 7f56aa9e1740 1 -- 172.21.15.5:0/2275360459 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c4de70a20 con 0x562c4df19b00 2024-09-19T06:12:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.416+0000 7f56a590b640 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562c4df19b00 0x562c4df1bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.417+0000 7f56a590b640 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562c4df19b00 0x562c4df1bef0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3787f95fa1eaf203 server_cookie=db2ddf95d4afd4ac in_seq=0 out_seq=0 2024-09-19T06:12:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.417+0000 7f56897fa640 10 client.17808 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.417+0000 7f56897fa640 1 -- 172.21.15.5:0/2275360459 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c4de70a20 con 0x562c4df19b00 2024-09-19T06:12:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.417+0000 7f56897fa640 10 client.17808 handle_command_reply: tid=0 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.417+0000 7f56aa9e1740 1 client.17808 shutdown 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.418+0000 7f56aa9e1740 1 -- 172.21.15.5:0/2275360459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 msgr2=0x562c4df06c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.418+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 0x562c4df06c00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5690010fd0 tx=0x7f569000bf50 comp rx=0 tx=0).stop 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.418+0000 7f56aa9e1740 1 -- 172.21.15.5:0/2275360459 shutdown_connections 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.418+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562c4df19b00 0x562c4df1bef0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.418+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 0x562c4df07680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.418+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4deff410 0x562c4df07140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.418+0000 7f56897fa640 0 client.17808 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.418+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/2275360459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 0x562c4df06c00 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.418+0000 7f56aa9e1740 1 -- 172.21.15.5:0/2275360459 >> 172.21.15.5:0/2275360459 conn(0x562c4defb060 msgr2=0x562c4df00d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.418+0000 7f56aa9e1740 1 -- 172.21.15.5:0/2275360459 shutdown_connections 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.418+0000 7f56aa9e1740 1 -- 172.21.15.5:0/2275360459 wait complete. 2024-09-19T06:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.418+0000 7f56aa9e1740 20 client.17808 trim_cache size 0 max 16384 2024-09-19T06:12:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.433+0000 7f56aa9e1740 1 Processor -- start 2024-09-19T06:12:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.433+0000 7f56aa9e1740 1 -- start start 2024-09-19T06:12:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.433+0000 7f56aa9e1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4defeaf0 0x562c4df07130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.433+0000 7f56aa9e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4deff410 0x562c4df07670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.433+0000 7f56aa9e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 0x562c4df216f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.433+0000 7f56aa9e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01af9e0 con 0x562c4deff410 2024-09-19T06:12:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.433+0000 7f56aa9e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01b1550 con 0x562c4df09cb0 2024-09-19T06:12:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.433+0000 7f56aa9e1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01b00b0 con 0x562c4defeaf0 2024-09-19T06:12:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.433+0000 7f56a590b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4deff410 0x562c4df07670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.433+0000 7f56a590b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4deff410 0x562c4df07670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57414/0 (socket says 172.21.15.5:57414) 2024-09-19T06:12:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.434+0000 7f56a590b640 1 -- 172.21.15.5:0/3099455504 learned_addr learned my addr 172.21.15.5:0/3099455504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.434+0000 7f56a510a640 1 --2- 172.21.15.5:0/3099455504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4defeaf0 0x562c4df07130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.434+0000 7f56a510a640 1 --2- 172.21.15.5:0/3099455504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 0x562c4df216f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.434+0000 7f56a590b640 1 -- 172.21.15.5:0/3099455504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4defeaf0 msgr2=0x562c4df07130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.434+0000 7f56a590b640 1 --2- 172.21.15.5:0/3099455504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4defeaf0 0x562c4df07130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.434+0000 7f56a590b640 1 -- 172.21.15.5:0/3099455504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 msgr2=0x562c4df216f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.434+0000 7f56a590b640 1 --2- 172.21.15.5:0/3099455504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 0x562c4df216f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.434+0000 7f56a590b640 1 -- 172.21.15.5:0/3099455504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c4def52f0 con 0x562c4deff410 2024-09-19T06:12:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.434+0000 7f56a590b640 1 --2- 172.21.15.5:0/3099455504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4deff410 0x562c4df07670 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f569c033000 tx=0x7f569c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d06259e29076b27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.434+0000 7f568b7fe640 1 -- 172.21.15.5:0/3099455504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c035040 con 0x562c4deff410 2024-09-19T06:12:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.434+0000 7f568b7fe640 1 -- 172.21.15.5:0/3099455504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f569c0054f0 con 0x562c4deff410 2024-09-19T06:12:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.434+0000 7f568b7fe640 1 -- 172.21.15.5:0/3099455504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c004d60 con 0x562c4deff410 2024-09-19T06:12:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.435+0000 7f56aa9e1740 1 -- 172.21.15.5:0/3099455504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4deff410 msgr2=0x562c4df07670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.435+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/3099455504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4deff410 0x562c4df07670 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f569c033000 tx=0x7f569c005db0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.435+0000 7f56aa9e1740 1 -- 172.21.15.5:0/3099455504 shutdown_connections 2024-09-19T06:12:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.435+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/3099455504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 0x562c4df216f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.435+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/3099455504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4deff410 0x562c4df07670 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.435+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/3099455504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4defeaf0 0x562c4df07130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.435+0000 7f56aa9e1740 1 -- 172.21.15.5:0/3099455504 >> 172.21.15.5:0/3099455504 conn(0x562c4defb060 msgr2=0x562c4df08c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.435+0000 7f56aa9e1740 1 -- 172.21.15.5:0/3099455504 shutdown_connections 2024-09-19T06:12:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.435+0000 7f56aa9e1740 1 -- 172.21.15.5:0/3099455504 wait complete. 2024-09-19T06:12:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.435+0000 7f56aa9e1740 1 -- start start 2024-09-19T06:12:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.435+0000 7f56aa9e1740 1 Processor -- start 2024-09-19T06:12:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56aa9e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 0x562c4df0d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56aa9e1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4deff410 0x562c4df0d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56aa9e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 0x562c4df826b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56aa9e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01b0e80 con 0x562c4defeaf0 2024-09-19T06:12:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56aa9e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01b0780 con 0x562c4df09cb0 2024-09-19T06:12:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56aa9e1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01ad9e0 con 0x562c4deff410 2024-09-19T06:12:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56aa9e1740 10 client.0 _ll_register_callbacks cb 0x7f569c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56a510a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 0x562c4df0d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56a510a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 0x562c4df0d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57418/0 (socket says 172.21.15.5:57418) 2024-09-19T06:12:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56a510a640 1 -- 172.21.15.5:0/371591722 learned_addr learned my addr 172.21.15.5:0/371591722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56a590b640 1 --2- 172.21.15.5:0/371591722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4deff410 0x562c4df0d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56a510a640 1 -- 172.21.15.5:0/371591722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4deff410 msgr2=0x562c4df0d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56a510a640 1 --2- 172.21.15.5:0/371591722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4deff410 0x562c4df0d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56a510a640 1 -- 172.21.15.5:0/371591722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 msgr2=0x562c4df826b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56a510a640 1 --2- 172.21.15.5:0/371591722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 0x562c4df826b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.436+0000 7f56a510a640 1 -- 172.21.15.5:0/371591722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c4df82bf0 con 0x562c4defeaf0 2024-09-19T06:12:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56a510a640 1 --2- 172.21.15.5:0/371591722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 0x562c4df0d1f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5690010f60 tx=0x7f5690004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147badb035a1dd2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56897fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56897fa640 1 -- 172.21.15.5:0/371591722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569003a060 con 0x562c4defeaf0 2024-09-19T06:12:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56897fa640 1 -- 172.21.15.5:0/371591722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f5690004860 con 0x562c4defeaf0 2024-09-19T06:12:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56aa9e1740 1 -- 172.21.15.5:0/371591722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f569c0054f0 con 0x562c4defeaf0 2024-09-19T06:12:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56897fa640 1 -- 172.21.15.5:0/371591722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5690004eb0 con 0x562c4defeaf0 2024-09-19T06:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56aa9e1740 10 client.17814 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56aa9e1740 1 -- 172.21.15.5:0/371591722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562c4df83e80 con 0x562c4defeaf0 2024-09-19T06:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56897fa640 1 -- 172.21.15.5:0/371591722 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f569c0054f0 con 0x562c4defeaf0 2024-09-19T06:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56897fa640 1 -- 172.21.15.5:0/371591722 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f5690029040 con 0x562c4defeaf0 2024-09-19T06:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56aa9e1740 10 client.17814 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56aa9e1740 10 client.17814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56aa9e1740 20 client.17814 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/371591722 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562c4df84570 0x562c4df86990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56aa9e1740 4 client.17814 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.437+0000 7f56aa9e1740 1 -- 172.21.15.5:0/371591722 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c4de6ad70 con 0x562c4df84570 2024-09-19T06:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.438+0000 7f56a590b640 1 --2- 172.21.15.5:0/371591722 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562c4df84570 0x562c4df86990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.438+0000 7f56a590b640 1 --2- 172.21.15.5:0/371591722 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562c4df84570 0x562c4df86990 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386c8bafdadc2e0d server_cookie=eb134f9f555765df in_seq=0 out_seq=0 2024-09-19T06:12:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.438+0000 7f56897fa640 10 client.17814 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.467+0000 7f56897fa640 1 -- 172.21.15.5:0/371591722 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c4de6ad70 con 0x562c4df84570 2024-09-19T06:12:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.467+0000 7f56897fa640 10 client.17814 handle_command_reply: tid=0 2024-09-19T06:12:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.467+0000 7f56aa9e1740 1 client.17814 shutdown 2024-09-19T06:12:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.467+0000 7f56aa9e1740 1 -- 172.21.15.5:0/371591722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 msgr2=0x562c4df0d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.467+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/371591722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 0x562c4df0d1f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5690010f60 tx=0x7f5690004c70 comp rx=0 tx=0).stop 2024-09-19T06:12:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56aa9e1740 1 -- 172.21.15.5:0/371591722 shutdown_connections 2024-09-19T06:12:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/371591722 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562c4df84570 0x562c4df86990 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/371591722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4df09cb0 0x562c4df826b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/371591722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c4deff410 0x562c4df0d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56897fa640 0 client.17814 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56aa9e1740 1 --2- 172.21.15.5:0/371591722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c4defeaf0 0x562c4df0d1f0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56aa9e1740 1 -- 172.21.15.5:0/371591722 >> 172.21.15.5:0/371591722 conn(0x562c4defb060 msgr2=0x562c4df14b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56aa9e1740 1 -- 172.21.15.5:0/371591722 shutdown_connections 2024-09-19T06:12:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56aa9e1740 1 -- 172.21.15.5:0/371591722 wait complete. 2024-09-19T06:12:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56aa9e1740 20 client.17814 trim_cache size 0 max 16384 2024-09-19T06:12:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:12:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:12:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:12:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:12:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56a7b6f640 1 -- 172.21.15.5:0/2896147338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f568c0a33b0 msgr2=0x7f568c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56a7b6f640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f568c0a33b0 0x7f568c0a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f56980ce180 tx=0x7f5698013fd0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56a7b6f640 1 -- 172.21.15.5:0/2896147338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a010e460 msgr2=0x7f56a019ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56a7b6f640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a010e460 0x7f56a019ee60 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f569800eb40 tx=0x7f569800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56a7b6f640 1 -- 172.21.15.5:0/2896147338 shutdown_connections 2024-09-19T06:12:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56a7b6f640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a010e460 0x7f56a019ee60 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56a7b6f640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f568c0a33b0 0x7f568c0a5870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56a7b6f640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a0101260 0x7f56a019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56a7b6f640 1 --2- 172.21.15.5:0/2896147338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a01008b0 0x7f56a019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.468+0000 7f56a7b6f640 1 -- 172.21.15.5:0/2896147338 >> 172.21.15.5:0/2896147338 conn(0x7f56a00fc6b0 msgr2=0x7f56a00feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.469+0000 7f56a7b6f640 1 -- 172.21.15.5:0/2896147338 shutdown_connections 2024-09-19T06:12:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:44.469+0000 7f56a7b6f640 1 -- 172.21.15.5:0/2896147338 wait complete. 2024-09-19T06:12:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:44 smithi157 ceph-mon[35346]: pgmap v291: 97 pgs: 97 active+clean; 131 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:12:44.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237831 2.56 MB/sec execute 370 sec latency 4117.359 ms 2024-09-19T06:12:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:44 smithi005 ceph-mon[30955]: pgmap v291: 97 pgs: 97 active+clean; 131 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:12:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:44 smithi049 ceph-mon[34640]: pgmap v291: 97 pgs: 97 active+clean; 131 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:12:45.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237942 2.56 MB/sec execute 371 sec latency 4742.357 ms 2024-09-19T06:12:45.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:12:46.189 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:46.597 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:46 smithi005 ceph-mon[30955]: pgmap v292: 97 pgs: 97 active+clean; 131 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 826 KiB/s wr, 80 op/s 2024-09-19T06:12:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.690+0000 7fd86042b640 1 -- 172.21.15.5:0/382804217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd858108290 msgr2=0x7fd85810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.690+0000 7fd86042b640 1 --2- 172.21.15.5:0/382804217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd858108290 0x7fd85810ec30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd850009cd0 tx=0x7fd85002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.690+0000 7fd86042b640 1 -- 172.21.15.5:0/382804217 shutdown_connections 2024-09-19T06:12:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.690+0000 7fd86042b640 1 --2- 172.21.15.5:0/382804217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd858108290 0x7fd85810ec30 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.690+0000 7fd86042b640 1 --2- 172.21.15.5:0/382804217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd858103740 0x7fd858107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.690+0000 7fd86042b640 1 --2- 172.21.15.5:0/382804217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd858102d90 0x7fd858103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.690+0000 7fd86042b640 1 -- 172.21.15.5:0/382804217 >> 172.21.15.5:0/382804217 conn(0x7fd8580fc7d0 msgr2=0x7fd8580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.690+0000 7fd86042b640 1 -- 172.21.15.5:0/382804217 shutdown_connections 2024-09-19T06:12:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.690+0000 7fd86042b640 1 -- 172.21.15.5:0/382804217 wait complete. 2024-09-19T06:12:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.690+0000 7fd86042b640 1 Processor -- start 2024-09-19T06:12:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd86042b640 1 -- start start 2024-09-19T06:12:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd86042b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd858102d90 0x7fd8581009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd86042b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd858103740 0x7fd858100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd86042b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd858108290 0x7fd858101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd86042b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd858111f40 con 0x7fd858108290 2024-09-19T06:12:46.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd86042b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd858111dc0 con 0x7fd858103740 2024-09-19T06:12:46.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd86042b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581120c0 con 0x7fd858102d90 2024-09-19T06:12:46.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd85e9c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd858108290 0x7fd858101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd85e1c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd858102d90 0x7fd8581009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd85e9c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd858108290 0x7fd858101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44398/0 (socket says 172.21.15.5:44398) 2024-09-19T06:12:46.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd85e9c8640 1 -- 172.21.15.5:0/4082437779 learned_addr learned my addr 172.21.15.5:0/4082437779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:46.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd858103740 0x7fd858100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd85e9c8640 1 -- 172.21.15.5:0/4082437779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd858102d90 msgr2=0x7fd8581009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd85e9c8640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd858102d90 0x7fd8581009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd85e9c8640 1 -- 172.21.15.5:0/4082437779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd858103740 msgr2=0x7fd858100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd85e9c8640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd858103740 0x7fd858100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd85e9c8640 1 -- 172.21.15.5:0/4082437779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8581a6920 con 0x7fd858108290 2024-09-19T06:12:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.691+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd858103740 0x7fd858100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.692+0000 7fd85e9c8640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd858108290 0x7fd858101470 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd850033040 tx=0x7fd850009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.692+0000 7fd84f7fe640 1 -- 172.21.15.5:0/4082437779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850004040 con 0x7fd858108290 2024-09-19T06:12:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.692+0000 7fd86042b640 1 -- 172.21.15.5:0/4082437779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8581a6bb0 con 0x7fd858108290 2024-09-19T06:12:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.692+0000 7fd84f7fe640 1 -- 172.21.15.5:0/4082437779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd850004690 con 0x7fd858108290 2024-09-19T06:12:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.692+0000 7fd84f7fe640 1 -- 172.21.15.5:0/4082437779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850030da0 con 0x7fd858108290 2024-09-19T06:12:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.692+0000 7fd86042b640 1 -- 172.21.15.5:0/4082437779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8581a70f0 con 0x7fd858108290 2024-09-19T06:12:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.695+0000 7fd84f7fe640 1 -- 172.21.15.5:0/4082437779 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd8500041e0 con 0x7fd858108290 2024-09-19T06:12:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.695+0000 7fd862804740 1 Processor -- start 2024-09-19T06:12:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd862804740 1 -- start start 2024-09-19T06:12:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd862804740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62b4a0 0x558d6b62b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd862804740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62bdc0 0x558d6b636140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd862804740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b636680 0x558d6b638a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd862804740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558d6b5912e0 con 0x558d6b62bdc0 2024-09-19T06:12:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd84f7fe640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8440a33b0 0x7fd8440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd862804740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558d6b538ce0 con 0x558d6b62b4a0 2024-09-19T06:12:46.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd862804740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558d6b58d2a0 con 0x558d6b636680 2024-09-19T06:12:46.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd85d9c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62b4a0 0x558d6b62b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd85e9c8640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8440a33b0 0x7fd8440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd85d9c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62b4a0 0x558d6b62b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56760/0 (socket says 172.21.15.5:56760) 2024-09-19T06:12:46.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd85d9c6640 1 -- 172.21.15.5:0/2189714511 learned_addr learned my addr 172.21.15.5:0/2189714511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:46.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/2189714511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62bdc0 0x558d6b636140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.696+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/2189714511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b636680 0x558d6b638a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.697+0000 7fd84f7fe640 1 -- 172.21.15.5:0/4082437779 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7fd85002e030 con 0x7fd858108290 2024-09-19T06:12:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.697+0000 7fd85e1c7640 1 -- 172.21.15.5:0/2189714511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b636680 msgr2=0x558d6b638a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.697+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/2189714511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b636680 0x558d6b638a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.697+0000 7fd85e1c7640 1 -- 172.21.15.5:0/2189714511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62b4a0 msgr2=0x558d6b62b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.697+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/2189714511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62b4a0 0x558d6b62b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.697+0000 7fd85e1c7640 1 -- 172.21.15.5:0/2189714511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d6b51d440 con 0x558d6b62bdc0 2024-09-19T06:12:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.697+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/2189714511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62b4a0 0x558d6b62b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.697+0000 7fd85e9c8640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8440a33b0 0x7fd8440a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd8500ee350 tx=0x7fd8500bc910 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.697+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/2189714511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62bdc0 0x558d6b636140 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd854009c20 tx=0x7fd854011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f21ffe365c7d109 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.697+0000 7fd84cdea640 1 -- 172.21.15.5:0/2189714511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854004050 con 0x558d6b62bdc0 2024-09-19T06:12:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.697+0000 7fd84cdea640 1 -- 172.21.15.5:0/2189714511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8540054d0 con 0x558d6b62bdc0 2024-09-19T06:12:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.697+0000 7fd84cdea640 1 -- 172.21.15.5:0/2189714511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854004cc0 con 0x558d6b62bdc0 2024-09-19T06:12:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.698+0000 7fd862804740 1 -- 172.21.15.5:0/2189714511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62bdc0 msgr2=0x558d6b636140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.698+0000 7fd862804740 1 --2- 172.21.15.5:0/2189714511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62bdc0 0x558d6b636140 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd854009c20 tx=0x7fd854011810 comp rx=0 tx=0).stop 2024-09-19T06:12:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.698+0000 7fd862804740 1 -- 172.21.15.5:0/2189714511 shutdown_connections 2024-09-19T06:12:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.698+0000 7fd862804740 1 --2- 172.21.15.5:0/2189714511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b636680 0x558d6b638a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.698+0000 7fd862804740 1 --2- 172.21.15.5:0/2189714511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62bdc0 0x558d6b636140 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.698+0000 7fd862804740 1 --2- 172.21.15.5:0/2189714511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62b4a0 0x558d6b62b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.698+0000 7fd862804740 1 -- 172.21.15.5:0/2189714511 >> 172.21.15.5:0/2189714511 conn(0x558d6b627a30 msgr2=0x558d6b627e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.698+0000 7fd862804740 1 -- 172.21.15.5:0/2189714511 shutdown_connections 2024-09-19T06:12:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.698+0000 7fd862804740 1 -- 172.21.15.5:0/2189714511 wait complete. 2024-09-19T06:12:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd862804740 1 -- start start 2024-09-19T06:12:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd862804740 1 Processor -- start 2024-09-19T06:12:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd862804740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 0x558d6b635fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd862804740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62bdc0 0x558d6b633250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd862804740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b636680 0x558d6b633790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd862804740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558d6b639250 con 0x558d6b62b4a0 2024-09-19T06:12:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd862804740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558d6b54c050 con 0x558d6b62bdc0 2024-09-19T06:12:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd862804740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558d6b6393d0 con 0x558d6b636680 2024-09-19T06:12:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd862804740 10 client.0 _ll_register_callbacks cb 0x558d6b58c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd85d9c6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 0x558d6b635fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd85e1c7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62bdc0 0x558d6b633250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd85d9c6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 0x558d6b635fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44410/0 (socket says 172.21.15.5:44410) 2024-09-19T06:12:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd85e1c7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62bdc0 0x558d6b633250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56770/0 (socket says 172.21.15.5:56770) 2024-09-19T06:12:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd85d9c6640 1 -- 172.21.15.5:0/2717028092 learned_addr learned my addr 172.21.15.5:0/2717028092 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.699+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/2717028092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b636680 0x558d6b633790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd85e1c7640 1 -- 172.21.15.5:0/2717028092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b636680 msgr2=0x558d6b633790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/2717028092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b636680 0x558d6b633790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd85e1c7640 1 -- 172.21.15.5:0/2717028092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 msgr2=0x558d6b635fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/2717028092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 0x558d6b635fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd85e1c7640 1 -- 172.21.15.5:0/2717028092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d6b3023c0 con 0x558d6b62bdc0 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/2717028092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62bdc0 0x558d6b633250 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd854004620 tx=0x7fd854004300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cbc9807f050bb80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd841ffb640 1 -- 172.21.15.5:0/2717028092 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854029070 con 0x558d6b62bdc0 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd841ffb640 1 -- 172.21.15.5:0/2717028092 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7fd854003800 con 0x558d6b62bdc0 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd862804740 1 -- 172.21.15.5:0/2717028092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8540054d0 con 0x558d6b62bdc0 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd841ffb640 1 -- 172.21.15.5:0/2717028092 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd854035050 con 0x558d6b62bdc0 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd862804740 10 client.27371 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.700+0000 7fd862804740 1 -- 172.21.15.5:0/2717028092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558d6b644c00 con 0x558d6b62bdc0 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.701+0000 7fd841ffb640 1 -- 172.21.15.5:0/2717028092 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8540054d0 con 0x558d6b62bdc0 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.701+0000 7fd841ffb640 1 -- 172.21.15.5:0/2717028092 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fd854005050 con 0x558d6b62bdc0 2024-09-19T06:12:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.701+0000 7fd862804740 10 client.27371 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.701+0000 7fd862804740 10 client.27371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.701+0000 7fd862804740 20 client.27371 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.701+0000 7fd862804740 1 --2- 172.21.15.5:0/2717028092 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d6b645f50 0x558d6b648340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.701+0000 7fd862804740 4 client.27371 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.701+0000 7fd862804740 1 -- 172.21.15.5:0/2717028092 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d6b59e850 con 0x558d6b645f50 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.701+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/2717028092 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d6b645f50 0x558d6b648340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.701+0000 7fd841ffb640 10 client.27371 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.701+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/2717028092 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d6b645f50 0x558d6b648340 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99db9311d5bd2291 server_cookie=fd742d9eb6f16314 in_seq=0 out_seq=0 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd841ffb640 1 -- 172.21.15.5:0/2717028092 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d6b59e850 con 0x558d6b645f50 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd841ffb640 10 client.27371 handle_command_reply: tid=0 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd862804740 1 client.27371 shutdown 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd862804740 1 -- 172.21.15.5:0/2717028092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62bdc0 msgr2=0x558d6b633250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd862804740 1 --2- 172.21.15.5:0/2717028092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62bdc0 0x558d6b633250 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd854004620 tx=0x7fd854004300 comp rx=0 tx=0).stop 2024-09-19T06:12:46.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd862804740 1 -- 172.21.15.5:0/2717028092 shutdown_connections 2024-09-19T06:12:46.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd862804740 1 --2- 172.21.15.5:0/2717028092 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d6b645f50 0x558d6b648340 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd862804740 1 --2- 172.21.15.5:0/2717028092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b636680 0x558d6b633790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd862804740 1 --2- 172.21.15.5:0/2717028092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62bdc0 0x558d6b633250 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd862804740 1 --2- 172.21.15.5:0/2717028092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 0x558d6b635fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd841ffb640 0 client.27371 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:46.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd862804740 1 -- 172.21.15.5:0/2717028092 >> 172.21.15.5:0/2717028092 conn(0x558d6b627a30 msgr2=0x558d6b626b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:46.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd862804740 1 -- 172.21.15.5:0/2717028092 shutdown_connections 2024-09-19T06:12:46.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.702+0000 7fd862804740 1 -- 172.21.15.5:0/2717028092 wait complete. 2024-09-19T06:12:46.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.703+0000 7fd862804740 20 client.27371 trim_cache size 0 max 16384 2024-09-19T06:12:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd862804740 1 Processor -- start 2024-09-19T06:12:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd862804740 1 -- start start 2024-09-19T06:12:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd862804740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b627a30 0x558d6b6424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd862804740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 0x558d6b62dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd862804740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b62bdc0 0x558d6b64dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd862804740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581af980 con 0x558d6b62b4a0 2024-09-19T06:12:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd862804740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581b14f0 con 0x558d6b627a30 2024-09-19T06:12:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd862804740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581b0050 con 0x558d6b62bdc0 2024-09-19T06:12:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd85e1c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 0x558d6b62dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd85e1c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 0x558d6b62dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44426/0 (socket says 172.21.15.5:44426) 2024-09-19T06:12:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd85e1c7640 1 -- 172.21.15.5:0/719373626 learned_addr learned my addr 172.21.15.5:0/719373626 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/719373626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b627a30 0x558d6b6424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/719373626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b62bdc0 0x558d6b64dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd85e1c7640 1 -- 172.21.15.5:0/719373626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b62bdc0 msgr2=0x558d6b64dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:46.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/719373626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b62bdc0 0x558d6b64dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd85e1c7640 1 -- 172.21.15.5:0/719373626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b627a30 msgr2=0x558d6b6424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:46.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/719373626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b627a30 0x558d6b6424e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.724+0000 7fd85e1c7640 1 -- 172.21.15.5:0/719373626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d6b62bbf0 con 0x558d6b62b4a0 2024-09-19T06:12:46.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.725+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/719373626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 0x558d6b62dfc0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd8540059f0 tx=0x7fd854011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d017930c39bdb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:46.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.725+0000 7fd843fff640 1 -- 172.21.15.5:0/719373626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854032070 con 0x558d6b62b4a0 2024-09-19T06:12:46.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.725+0000 7fd843fff640 1 -- 172.21.15.5:0/719373626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8540054a0 con 0x558d6b62b4a0 2024-09-19T06:12:46.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.725+0000 7fd843fff640 1 -- 172.21.15.5:0/719373626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854029070 con 0x558d6b62b4a0 2024-09-19T06:12:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.726+0000 7fd862804740 1 -- 172.21.15.5:0/719373626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 msgr2=0x558d6b62dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.726+0000 7fd862804740 1 --2- 172.21.15.5:0/719373626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 0x558d6b62dfc0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd8540059f0 tx=0x7fd854011810 comp rx=0 tx=0).stop 2024-09-19T06:12:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.726+0000 7fd862804740 1 -- 172.21.15.5:0/719373626 shutdown_connections 2024-09-19T06:12:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.726+0000 7fd862804740 1 --2- 172.21.15.5:0/719373626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b62bdc0 0x558d6b64dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.726+0000 7fd862804740 1 --2- 172.21.15.5:0/719373626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b62b4a0 0x558d6b62dfc0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.726+0000 7fd862804740 1 --2- 172.21.15.5:0/719373626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b627a30 0x558d6b6424e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.726+0000 7fd862804740 1 -- 172.21.15.5:0/719373626 >> 172.21.15.5:0/719373626 conn(0x558d6b6338f0 msgr2=0x558d6b6352d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.726+0000 7fd862804740 1 -- 172.21.15.5:0/719373626 shutdown_connections 2024-09-19T06:12:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.726+0000 7fd862804740 1 -- 172.21.15.5:0/719373626 wait complete. 2024-09-19T06:12:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.726+0000 7fd862804740 1 -- start start 2024-09-19T06:12:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.726+0000 7fd862804740 1 Processor -- start 2024-09-19T06:12:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd862804740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b627a30 0x558d6b63a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd862804740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b62b4a0 0x558d6b63aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd862804740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62bdc0 0x558d6b63b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd862804740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581b0e20 con 0x558d6b627a30 2024-09-19T06:12:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd862804740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581b0720 con 0x558d6b62bdc0 2024-09-19T06:12:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd862804740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581ad980 con 0x558d6b62b4a0 2024-09-19T06:12:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd862804740 10 client.0 _ll_register_callbacks cb 0x7fd848009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd85d9c6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b627a30 0x558d6b63a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd85d9c6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b627a30 0x558d6b63a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44438/0 (socket says 172.21.15.5:44438) 2024-09-19T06:12:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd85d9c6640 1 -- 172.21.15.5:0/4168542129 learned_addr learned my addr 172.21.15.5:0/4168542129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b62b4a0 0x558d6b63aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62bdc0 0x558d6b63b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd85d9c6640 1 -- 172.21.15.5:0/4168542129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b62b4a0 msgr2=0x558d6b63aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b62b4a0 0x558d6b63aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd85d9c6640 1 -- 172.21.15.5:0/4168542129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62bdc0 msgr2=0x558d6b63b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62bdc0 0x558d6b63b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd85d9c6640 1 -- 172.21.15.5:0/4168542129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d6b6afd70 con 0x558d6b627a30 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd85d9c6640 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b627a30 0x558d6b63a5b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd8480058c0 tx=0x7fd848013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246706cb3e448d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd841ffb640 1 -- 172.21.15.5:0/4168542129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848033060 con 0x558d6b627a30 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd841ffb640 1 -- 172.21.15.5:0/4168542129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7fd848003b80 con 0x558d6b627a30 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.727+0000 7fd862804740 1 -- 172.21.15.5:0/4168542129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8540054a0 con 0x558d6b627a30 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.728+0000 7fd841ffb640 1 -- 172.21.15.5:0/4168542129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd84801a040 con 0x558d6b627a30 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.728+0000 7fd862804740 10 client.17832 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.728+0000 7fd862804740 1 -- 172.21.15.5:0/4168542129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558d6b6b1060 con 0x558d6b627a30 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.728+0000 7fd841ffb640 1 -- 172.21.15.5:0/4168542129 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8540054a0 con 0x558d6b627a30 2024-09-19T06:12:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.728+0000 7fd841ffb640 1 -- 172.21.15.5:0/4168542129 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fd84802a040 con 0x558d6b627a30 2024-09-19T06:12:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.728+0000 7fd862804740 10 client.17832 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.728+0000 7fd862804740 10 client.17832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.728+0000 7fd862804740 20 client.17832 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.728+0000 7fd862804740 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d6b636680 0x558d6b6b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.728+0000 7fd862804740 4 client.17832 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:12:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.728+0000 7fd862804740 1 -- 172.21.15.5:0/4168542129 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d6b62ee70 con 0x558d6b636680 2024-09-19T06:12:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.728+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d6b636680 0x558d6b6b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd85e1c7640 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d6b636680 0x558d6b6b37d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bd941263925f409 server_cookie=37246f1f1565b7d2 in_seq=0 out_seq=0 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd841ffb640 10 client.17832 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd841ffb640 1 -- 172.21.15.5:0/4168542129 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558d6b62ee70 con 0x558d6b636680 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd841ffb640 10 client.17832 handle_command_reply: tid=0 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd862804740 1 client.17832 shutdown 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd862804740 1 -- 172.21.15.5:0/4168542129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b627a30 msgr2=0x558d6b63a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd862804740 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b627a30 0x558d6b63a5b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd8480058c0 tx=0x7fd848013040 comp rx=0 tx=0).stop 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd862804740 1 -- 172.21.15.5:0/4168542129 shutdown_connections 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd862804740 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d6b636680 0x558d6b6b37d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd862804740 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d6b62bdc0 0x558d6b63b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd862804740 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d6b62b4a0 0x558d6b63aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd862804740 1 --2- 172.21.15.5:0/4168542129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6b627a30 0x558d6b63a5b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd862804740 1 -- 172.21.15.5:0/4168542129 >> 172.21.15.5:0/4168542129 conn(0x558d6b6338f0 msgr2=0x558d6b62cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd841ffb640 0 client.17832 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd862804740 1 -- 172.21.15.5:0/4168542129 shutdown_connections 2024-09-19T06:12:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.729+0000 7fd862804740 1 -- 172.21.15.5:0/4168542129 wait complete. 2024-09-19T06:12:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd862804740 20 client.17832 trim_cache size 0 max 16384 2024-09-19T06:12:46.738 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:12:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:12:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:12:46.738 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:12:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd86042b640 1 -- 172.21.15.5:0/4082437779 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8440a33b0 msgr2=0x7fd8440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd86042b640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8440a33b0 0x7fd8440a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd8500ee350 tx=0x7fd8500bc910 comp rx=0 tx=0).stop 2024-09-19T06:12:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd86042b640 1 -- 172.21.15.5:0/4082437779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd858108290 msgr2=0x7fd858101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd86042b640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd858108290 0x7fd858101470 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd850033040 tx=0x7fd850009360 comp rx=0 tx=0).stop 2024-09-19T06:12:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd86042b640 1 -- 172.21.15.5:0/4082437779 shutdown_connections 2024-09-19T06:12:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd86042b640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd858108290 0x7fd858101470 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd86042b640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd858103740 0x7fd858100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd86042b640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8440a33b0 0x7fd8440a5870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd86042b640 1 --2- 172.21.15.5:0/4082437779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd858102d90 0x7fd8581009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd86042b640 1 -- 172.21.15.5:0/4082437779 >> 172.21.15.5:0/4082437779 conn(0x7fd8580fc7d0 msgr2=0x7fd858105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd86042b640 1 -- 172.21.15.5:0/4082437779 shutdown_connections 2024-09-19T06:12:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:46.730+0000 7fd86042b640 1 -- 172.21.15.5:0/4082437779 wait complete. 2024-09-19T06:12:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:46 smithi157 ceph-mon[35346]: pgmap v292: 97 pgs: 97 active+clean; 131 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 826 KiB/s wr, 80 op/s 2024-09-19T06:12:46.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 238168 2.55 MB/sec execute 372 sec latency 15.506 ms 2024-09-19T06:12:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:46 smithi049 ceph-mon[34640]: pgmap v292: 97 pgs: 97 active+clean; 131 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 826 KiB/s wr, 80 op/s 2024-09-19T06:12:47.524 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:12:47.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:12:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:47.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 238599 2.55 MB/sec execute 373 sec latency 15.645 ms 2024-09-19T06:12:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.271+0000 7f673f692640 1 -- 172.21.15.5:0/4140856434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673810c170 msgr2=0x7f673810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.271+0000 7f673f692640 1 --2- 172.21.15.5:0/4140856434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673810c170 0x7f673810e610 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f6730009ca0 tx=0x7f673002a950 comp rx=0 tx=0).stop 2024-09-19T06:12:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.271+0000 7f673f692640 1 -- 172.21.15.5:0/4140856434 shutdown_connections 2024-09-19T06:12:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.271+0000 7f673f692640 1 --2- 172.21.15.5:0/4140856434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673810c170 0x7f673810e610 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.271+0000 7f673f692640 1 --2- 172.21.15.5:0/4140856434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6738103660 0x7f673810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.271+0000 7f673f692640 1 --2- 172.21.15.5:0/4140856434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6738102d40 0x7f6738103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.271+0000 7f673f692640 1 -- 172.21.15.5:0/4140856434 >> 172.21.15.5:0/4140856434 conn(0x7f67380fc7a0 msgr2=0x7f67380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.271+0000 7f673f692640 1 -- 172.21.15.5:0/4140856434 shutdown_connections 2024-09-19T06:12:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.271+0000 7f673f692640 1 -- 172.21.15.5:0/4140856434 wait complete. 2024-09-19T06:12:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673f692640 1 Processor -- start 2024-09-19T06:12:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673f692640 1 -- start start 2024-09-19T06:12:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673f692640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6738102d40 0x7f6738198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673f692640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6738103660 0x7f6738198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673f692640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673810c170 0x7f673819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673f692640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6738111e50 con 0x7f673810c170 2024-09-19T06:12:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673f692640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6738111cd0 con 0x7f6738102d40 2024-09-19T06:12:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673f692640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6738111fd0 con 0x7f6738103660 2024-09-19T06:12:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673dc2f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673810c170 0x7f673819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673cc2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6738103660 0x7f6738198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673dc2f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673810c170 0x7f673819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44452/0 (socket says 172.21.15.5:44452) 2024-09-19T06:12:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673dc2f640 1 -- 172.21.15.5:0/1903231420 learned_addr learned my addr 172.21.15.5:0/1903231420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.272+0000 7f673d42e640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6738102d40 0x7f6738198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.273+0000 7f673cc2d640 1 -- 172.21.15.5:0/1903231420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6738102d40 msgr2=0x7f6738198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.273+0000 7f673cc2d640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6738102d40 0x7f6738198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.273+0000 7f673cc2d640 1 -- 172.21.15.5:0/1903231420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673810c170 msgr2=0x7f673819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.273+0000 7f673cc2d640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673810c170 0x7f673819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.273+0000 7f673cc2d640 1 -- 172.21.15.5:0/1903231420 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673819d2d0 con 0x7f6738103660 2024-09-19T06:12:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.273+0000 7f673dc2f640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673810c170 0x7f673819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.273+0000 7f673cc2d640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6738103660 0x7f6738198860 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f672800d650 tx=0x7f672800db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.273+0000 7f672e7fc640 1 -- 172.21.15.5:0/1903231420 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728017070 con 0x7f6738103660 2024-09-19T06:12:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.273+0000 7f672e7fc640 1 -- 172.21.15.5:0/1903231420 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67280054d0 con 0x7f6738103660 2024-09-19T06:12:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.273+0000 7f673f692640 1 -- 172.21.15.5:0/1903231420 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673819d5c0 con 0x7f6738103660 2024-09-19T06:12:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.273+0000 7f672e7fc640 1 -- 172.21.15.5:0/1903231420 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67280047a0 con 0x7f6738103660 2024-09-19T06:12:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.273+0000 7f673f692640 1 -- 172.21.15.5:0/1903231420 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67381a4ea0 con 0x7f6738103660 2024-09-19T06:12:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.275+0000 7f673f692640 1 -- 172.21.15.5:0/1903231420 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6738103120 con 0x7f6738103660 2024-09-19T06:12:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.276+0000 7f672e7fc640 1 -- 172.21.15.5:0/1903231420 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6728005020 con 0x7f6738103660 2024-09-19T06:12:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.277+0000 7f672e7fc640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67240a3430 0x7f67240a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.277+0000 7f673d42e640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67240a3430 0x7f67240a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.277+0000 7f672e7fc640 1 -- 172.21.15.5:0/1903231420 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f67280cfbc0 con 0x7f6738103660 2024-09-19T06:12:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.277+0000 7f673d42e640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67240a3430 0x7f67240a58f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6734005d00 tx=0x7f673400b410 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.279+0000 7f672e7fc640 1 -- 172.21.15.5:0/1903231420 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6728020020 con 0x7f6738103660 2024-09-19T06:12:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.526+0000 7f673f692640 1 -- 172.21.15.5:0/1903231420 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6738199680 con 0x7f6738103660 2024-09-19T06:12:48.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:48 smithi005 ceph-mon[30955]: pgmap v293: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:12:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.527+0000 7f672e7fc640 1 -- 172.21.15.5:0/1903231420 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11961 (secure 0 0 0) 0x7f6738199680 con 0x7f6738103660 2024-09-19T06:12:48.828 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:12:48.829 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"btime":"2024-09-19T06:12:29:510015+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:12:29.029072+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:12:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 35 2024-09-19T06:12:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.529+0000 7f673f692640 1 -- 172.21.15.5:0/1903231420 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67240a3430 msgr2=0x7f67240a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.529+0000 7f673f692640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67240a3430 0x7f67240a58f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6734005d00 tx=0x7f673400b410 comp rx=0 tx=0).stop 2024-09-19T06:12:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.529+0000 7f673f692640 1 -- 172.21.15.5:0/1903231420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6738103660 msgr2=0x7f6738198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.529+0000 7f673f692640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6738103660 0x7f6738198860 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f672800d650 tx=0x7f672800db10 comp rx=0 tx=0).stop 2024-09-19T06:12:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.529+0000 7f673f692640 1 -- 172.21.15.5:0/1903231420 shutdown_connections 2024-09-19T06:12:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.529+0000 7f673f692640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673810c170 0x7f673819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.529+0000 7f673f692640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67240a3430 0x7f67240a58f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.529+0000 7f673f692640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6738103660 0x7f6738198860 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.529+0000 7f673f692640 1 --2- 172.21.15.5:0/1903231420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6738102d40 0x7f6738198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.529+0000 7f673f692640 1 -- 172.21.15.5:0/1903231420 >> 172.21.15.5:0/1903231420 conn(0x7f67380fc7a0 msgr2=0x7f67380fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.529+0000 7f673f692640 1 -- 172.21.15.5:0/1903231420 shutdown_connections 2024-09-19T06:12:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:48.529+0000 7f673f692640 1 -- 172.21.15.5:0/1903231420 wait complete. 2024-09-19T06:12:48.831 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:48 smithi157 ceph-mon[35346]: pgmap v293: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:12:48.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 239120 2.55 MB/sec execute 374 sec latency 37.837 ms 2024-09-19T06:12:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:48 smithi049 ceph-mon[34640]: pgmap v293: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:12:49.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:12:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:49.624 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:49 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1903231420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:12:49.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:49 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1903231420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:12:49.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 239600 2.55 MB/sec execute 375 sec latency 20.272 ms 2024-09-19T06:12:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:49 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1903231420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:12:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.021+0000 7f11ae053640 1 -- 172.21.15.5:0/2897132335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a8100e20 msgr2=0x7f11a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.021+0000 7f11ae053640 1 --2- 172.21.15.5:0/2897132335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a8100e20 0x7f11a8101200 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1198009900 tx=0x7f119802a480 comp rx=0 tx=0).stop 2024-09-19T06:12:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.022+0000 7f11ae053640 1 -- 172.21.15.5:0/2897132335 shutdown_connections 2024-09-19T06:12:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.022+0000 7f11ae053640 1 --2- 172.21.15.5:0/2897132335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11a8102ef0 0x7f11a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.022+0000 7f11ae053640 1 --2- 172.21.15.5:0/2897132335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a8102570 0x7f11a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.022+0000 7f11ae053640 1 --2- 172.21.15.5:0/2897132335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a8100e20 0x7f11a8101200 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.022+0000 7f11ae053640 1 -- 172.21.15.5:0/2897132335 >> 172.21.15.5:0/2897132335 conn(0x7f11a80fc7d0 msgr2=0x7f11a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.022+0000 7f11ae053640 1 -- 172.21.15.5:0/2897132335 shutdown_connections 2024-09-19T06:12:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.022+0000 7f11ae053640 1 -- 172.21.15.5:0/2897132335 wait complete. 2024-09-19T06:12:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.022+0000 7f11ae053640 1 Processor -- start 2024-09-19T06:12:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11ae053640 1 -- start start 2024-09-19T06:12:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11ae053640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a8100e20 0x7f11a810d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11ae053640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11a8102570 0x7f11a81098e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11ae053640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a8102ef0 0x7f11a8109e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11ae053640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11a810fd80 con 0x7f11a8100e20 2024-09-19T06:12:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11ae053640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f11a810fc00 con 0x7f11a8102ef0 2024-09-19T06:12:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11ae053640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f11a810ff00 con 0x7f11a8102570 2024-09-19T06:12:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a8100e20 0x7f11a810d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11a8102570 0x7f11a81098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a8100e20 0x7f11a810d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44480/0 (socket says 172.21.15.5:44480) 2024-09-19T06:12:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11a77fe640 1 -- 172.21.15.5:0/3878455356 learned_addr learned my addr 172.21.15.5:0/3878455356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11a7fff640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a8102ef0 0x7f11a8109e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11a6ffd640 1 -- 172.21.15.5:0/3878455356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a8102ef0 msgr2=0x7f11a8109e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a8102ef0 0x7f11a8109e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11a6ffd640 1 -- 172.21.15.5:0/3878455356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a8100e20 msgr2=0x7f11a810d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a8100e20 0x7f11a810d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11a6ffd640 1 -- 172.21.15.5:0/3878455356 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a810a6c0 con 0x7f11a8102570 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11a7fff640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a8102ef0 0x7f11a8109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.023+0000 7f11a77fe640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a8100e20 0x7f11a810d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.024+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11a8102570 0x7f11a81098e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f119c00e9f0 tx=0x7f119c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.024+0000 7f11a4ff9640 1 -- 172.21.15.5:0/3878455356 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c00ccc0 con 0x7f11a8102570 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.024+0000 7f11a4ff9640 1 -- 172.21.15.5:0/3878455356 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f119c0108e0 con 0x7f11a8102570 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.024+0000 7f11ae053640 1 -- 172.21.15.5:0/3878455356 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a81ab000 con 0x7f11a8102570 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.024+0000 7f11a4ff9640 1 -- 172.21.15.5:0/3878455356 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c018880 con 0x7f11a8102570 2024-09-19T06:12:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.024+0000 7f11ae053640 1 -- 172.21.15.5:0/3878455356 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a81ab540 con 0x7f11a8102570 2024-09-19T06:12:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.026+0000 7f11a4ff9640 1 -- 172.21.15.5:0/3878455356 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f119c010a90 con 0x7f11a8102570 2024-09-19T06:12:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.027+0000 7f11b042c740 1 Processor -- start 2024-09-19T06:12:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.027+0000 7f11b042c740 1 -- start start 2024-09-19T06:12:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.027+0000 7f11a4ff9640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f11900a33b0 0x7f11900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.027+0000 7f11b042c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8bb40 0x561052d8a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.027+0000 7f11b042c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 0x561052d8aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.027+0000 7f11a7fff640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f11900a33b0 0x7f11900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.027+0000 7f11b042c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d95370 0x561052d97760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.027+0000 7f11b042c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561052cf02f0 con 0x561052d8a5d0 2024-09-19T06:12:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.027+0000 7f11b042c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561052c97cf0 con 0x561052d95370 2024-09-19T06:12:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.027+0000 7f11b042c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561052cec2b0 con 0x561052d8bb40 2024-09-19T06:12:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.027+0000 7f11a4ff9640 1 -- 172.21.15.5:0/3878455356 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f119c0d0140 con 0x7f11a8102570 2024-09-19T06:12:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8bb40 0x561052d8a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 0x561052d8aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 0x561052d8aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44494/0 (socket says 172.21.15.5:44494) 2024-09-19T06:12:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8bb40 0x561052d8a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43476/0 (socket says 172.21.15.5:43476) 2024-09-19T06:12:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a6ffd640 1 -- 172.21.15.5:0/241386739 learned_addr learned my addr 172.21.15.5:0/241386739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a7fff640 1 --2- 172.21.15.5:0/241386739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d95370 0x561052d97760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a6ffd640 1 -- 172.21.15.5:0/241386739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8bb40 msgr2=0x561052d8a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/241386739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8bb40 0x561052d8a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a6ffd640 1 -- 172.21.15.5:0/241386739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d95370 msgr2=0x561052d97760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/241386739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d95370 0x561052d97760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a6ffd640 1 -- 172.21.15.5:0/241386739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561052c7c3a0 con 0x561052d8a5d0 2024-09-19T06:12:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a7fff640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f11900a33b0 0x7f11900a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009fd0 tx=0x7f11a0014c30 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.028+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/241386739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 0x561052d8aa10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f119c0a4b40 tx=0x7f119c09c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508815d0ea819a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.029+0000 7f118dffb640 1 -- 172.21.15.5:0/241386739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c0a5070 con 0x561052d8a5d0 2024-09-19T06:12:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.029+0000 7f118dffb640 1 -- 172.21.15.5:0/241386739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f119c0cdaf0 con 0x561052d8a5d0 2024-09-19T06:12:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.029+0000 7f118dffb640 1 -- 172.21.15.5:0/241386739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c09b850 con 0x561052d8a5d0 2024-09-19T06:12:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.029+0000 7f11b042c740 1 -- 172.21.15.5:0/241386739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 msgr2=0x561052d8aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.029+0000 7f11b042c740 1 --2- 172.21.15.5:0/241386739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 0x561052d8aa10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f119c0a4b40 tx=0x7f119c09c1d0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.030+0000 7f11b042c740 1 -- 172.21.15.5:0/241386739 shutdown_connections 2024-09-19T06:12:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.030+0000 7f11b042c740 1 --2- 172.21.15.5:0/241386739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d95370 0x561052d97760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.030+0000 7f11b042c740 1 --2- 172.21.15.5:0/241386739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 0x561052d8aa10 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.030+0000 7f11b042c740 1 --2- 172.21.15.5:0/241386739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8bb40 0x561052d8a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.030+0000 7f11b042c740 1 -- 172.21.15.5:0/241386739 >> 172.21.15.5:0/241386739 conn(0x561052d53310 msgr2=0x561052d86c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.030+0000 7f11b042c740 1 -- 172.21.15.5:0/241386739 shutdown_connections 2024-09-19T06:12:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.030+0000 7f11b042c740 1 -- 172.21.15.5:0/241386739 wait complete. 2024-09-19T06:12:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.030+0000 7f11b042c740 1 -- start start 2024-09-19T06:12:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.030+0000 7f11b042c740 1 Processor -- start 2024-09-19T06:12:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11b042c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8a5d0 0x561052d94e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11b042c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 0x561052d920b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11b042c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d95370 0x561052d92840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11b042c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561052d97fb0 con 0x561052d95370 2024-09-19T06:12:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11b042c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561052cab060 con 0x561052d8bb40 2024-09-19T06:12:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11b042c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561052d98130 con 0x561052d8a5d0 2024-09-19T06:12:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11b042c740 10 client.0 _ll_register_callbacks cb 0x5610529fe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 0x561052d920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 0x561052d920b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56854/0 (socket says 172.21.15.5:56854) 2024-09-19T06:12:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11a6ffd640 1 -- 172.21.15.5:0/2924064334 learned_addr learned my addr 172.21.15.5:0/2924064334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11a7fff640 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d95370 0x561052d92840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11a77fe640 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8a5d0 0x561052d94e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11a6ffd640 1 -- 172.21.15.5:0/2924064334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8a5d0 msgr2=0x561052d94e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8a5d0 0x561052d94e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11a6ffd640 1 -- 172.21.15.5:0/2924064334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d95370 msgr2=0x561052d92840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d95370 0x561052d92840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.031+0000 7f11a6ffd640 1 -- 172.21.15.5:0/2924064334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561052a613c0 con 0x561052d8bb40 2024-09-19T06:12:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.032+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 0x561052d920b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f119c098860 tx=0x7f119c0106e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=757acfa14dc169ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.032+0000 7f1186ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:12:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.032+0000 7f1186ffd640 1 -- 172.21.15.5:0/2924064334 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c0b4030 con 0x561052d8bb40 2024-09-19T06:12:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.032+0000 7f11b042c740 1 -- 172.21.15.5:0/2924064334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119c0cdaf0 con 0x561052d8bb40 2024-09-19T06:12:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.032+0000 7f1186ffd640 1 -- 172.21.15.5:0/2924064334 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f119c0a4e10 con 0x561052d8bb40 2024-09-19T06:12:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.032+0000 7f1186ffd640 1 -- 172.21.15.5:0/2924064334 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f119c0a4be0 con 0x561052d8bb40 2024-09-19T06:12:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.032+0000 7f11b042c740 10 client.27395 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.032+0000 7f11b042c740 1 -- 172.21.15.5:0/2924064334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561052da3b00 con 0x561052d8bb40 2024-09-19T06:12:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.032+0000 7f1186ffd640 1 -- 172.21.15.5:0/2924064334 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f119c0cdaf0 con 0x561052d8bb40 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.032+0000 7f1186ffd640 1 -- 172.21.15.5:0/2924064334 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f119c09b2c0 con 0x561052d8bb40 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.033+0000 7f11b042c740 10 client.27395 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.033+0000 7f11b042c740 10 client.27395 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.033+0000 7f11b042c740 20 client.27395 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.033+0000 7f11b042c740 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x561052da4e50 0x561052da7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.033+0000 7f11b042c740 4 client.27395 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.033+0000 7f11b042c740 1 -- 172.21.15.5:0/2924064334 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561052cfd860 con 0x561052da4e50 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.033+0000 7f11a77fe640 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x561052da4e50 0x561052da7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.033+0000 7f11a77fe640 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x561052da4e50 0x561052da7240 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4763375968021c32 server_cookie=abb08c61f8e177a1 in_seq=0 out_seq=0 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.033+0000 7f1186ffd640 10 client.27395 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f1186ffd640 1 -- 172.21.15.5:0/2924064334 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561052cfd860 con 0x561052da4e50 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f1186ffd640 10 client.27395 handle_command_reply: tid=0 2024-09-19T06:12:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f11b042c740 1 client.27395 shutdown 2024-09-19T06:12:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f11b042c740 1 -- 172.21.15.5:0/2924064334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 msgr2=0x561052d920b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f11b042c740 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 0x561052d920b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f119c098860 tx=0x7f119c0106e0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f11b042c740 1 -- 172.21.15.5:0/2924064334 shutdown_connections 2024-09-19T06:12:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f11b042c740 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x561052da4e50 0x561052da7240 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f11b042c740 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d95370 0x561052d92840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f11b042c740 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 0x561052d920b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f11b042c740 1 --2- 172.21.15.5:0/2924064334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8a5d0 0x561052d94e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f11b042c740 1 -- 172.21.15.5:0/2924064334 >> 172.21.15.5:0/2924064334 conn(0x561052d53310 msgr2=0x561052d8c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f1186ffd640 0 client.27395 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:12:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f11b042c740 1 -- 172.21.15.5:0/2924064334 shutdown_connections 2024-09-19T06:12:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f11b042c740 1 -- 172.21.15.5:0/2924064334 wait complete. 2024-09-19T06:12:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.034+0000 7f11b042c740 20 client.27395 trim_cache size 0 max 16384 2024-09-19T06:12:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.047+0000 7f11b042c740 1 Processor -- start 2024-09-19T06:12:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.047+0000 7f11b042c740 1 -- start start 2024-09-19T06:12:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11b042c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 0x561052da0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11b042c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 0x561052da0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11b042c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d95370 0x561052dac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11b042c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b3fc0 con 0x561052d8a5d0 2024-09-19T06:12:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11b042c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b5b30 con 0x561052d8bb40 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11b042c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b4690 con 0x561052d95370 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 0x561052da0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 0x561052da0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44516/0 (socket says 172.21.15.5:44516) 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11a77fe640 1 -- 172.21.15.5:0/3602282103 learned_addr learned my addr 172.21.15.5:0/3602282103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/3602282103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 0x561052da0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11a7fff640 1 --2- 172.21.15.5:0/3602282103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d95370 0x561052dac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11a6ffd640 1 -- 172.21.15.5:0/3602282103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d95370 msgr2=0x561052dac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/3602282103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d95370 0x561052dac810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11a6ffd640 1 -- 172.21.15.5:0/3602282103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 msgr2=0x561052da0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/3602282103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 0x561052da0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11a6ffd640 1 -- 172.21.15.5:0/3602282103 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561052d98070 con 0x561052d8bb40 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11a77fe640 1 --2- 172.21.15.5:0/3602282103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 0x561052da0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/3602282103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 0x561052da0e90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f119c09a6b0 tx=0x7f119c09c1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c69b83177d0c4781 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f118d7fa640 1 -- 172.21.15.5:0/3602282103 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c0b4070 con 0x561052d8bb40 2024-09-19T06:12:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f118d7fa640 1 -- 172.21.15.5:0/3602282103 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f119c010e00 con 0x561052d8bb40 2024-09-19T06:12:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.048+0000 7f118d7fa640 1 -- 172.21.15.5:0/3602282103 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c09b2c0 con 0x561052d8bb40 2024-09-19T06:12:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.049+0000 7f11b042c740 1 -- 172.21.15.5:0/3602282103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 msgr2=0x561052da0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.049+0000 7f11b042c740 1 --2- 172.21.15.5:0/3602282103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 0x561052da0e90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f119c09a6b0 tx=0x7f119c09c1d0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.050+0000 7f11b042c740 1 -- 172.21.15.5:0/3602282103 shutdown_connections 2024-09-19T06:12:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.050+0000 7f11b042c740 1 --2- 172.21.15.5:0/3602282103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d95370 0x561052dac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.050+0000 7f11b042c740 1 --2- 172.21.15.5:0/3602282103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d8bb40 0x561052da0e90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.050+0000 7f11b042c740 1 --2- 172.21.15.5:0/3602282103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8a5d0 0x561052da0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.050+0000 7f11b042c740 1 -- 172.21.15.5:0/3602282103 >> 172.21.15.5:0/3602282103 conn(0x561052d53310 msgr2=0x561052da0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.050+0000 7f11b042c740 1 -- 172.21.15.5:0/3602282103 shutdown_connections 2024-09-19T06:12:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.050+0000 7f11b042c740 1 -- 172.21.15.5:0/3602282103 wait complete. 2024-09-19T06:12:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.050+0000 7f11b042c740 1 -- start start 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.050+0000 7f11b042c740 1 Processor -- start 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11b042c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8a5d0 0x561052d9dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11b042c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8bb40 0x561052d8ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11b042c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d95370 0x561052d8f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11b042c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b5460 con 0x561052d8bb40 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11b042c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b4d60 con 0x561052d95370 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11b042c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b1fc0 con 0x561052d8a5d0 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8bb40 0x561052d8ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8a5d0 0x561052d9dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8bb40 0x561052d8ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44526/0 (socket says 172.21.15.5:44526) 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8a5d0 0x561052d9dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43506/0 (socket says 172.21.15.5:43506) 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a6ffd640 1 -- 172.21.15.5:0/64154563 learned_addr learned my addr 172.21.15.5:0/64154563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11b042c740 10 client.0 _ll_register_callbacks cb 0x7f11a0009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a7fff640 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d95370 0x561052d8f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a77fe640 1 -- 172.21.15.5:0/64154563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d95370 msgr2=0x561052d8f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a77fe640 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d95370 0x561052d8f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a77fe640 1 -- 172.21.15.5:0/64154563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8bb40 msgr2=0x561052d8ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a77fe640 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8bb40 0x561052d8ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a77fe640 1 -- 172.21.15.5:0/64154563 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561052d8fac0 con 0x561052d8a5d0 2024-09-19T06:12:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8bb40 0x561052d8ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f11a77fe640 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8a5d0 0x561052d9dad0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f119802aa40 tx=0x7f1198003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c84c7abd08f6caf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.051+0000 7f1186ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:12:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.052+0000 7f1186ffd640 1 -- 172.21.15.5:0/64154563 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119804a070 con 0x561052d8a5d0 2024-09-19T06:12:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.052+0000 7f1186ffd640 1 -- 172.21.15.5:0/64154563 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f1198003da0 con 0x561052d8a5d0 2024-09-19T06:12:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.052+0000 7f11b042c740 1 -- 172.21.15.5:0/64154563 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561052cf7d70 con 0x561052d8a5d0 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.052+0000 7f1186ffd640 1 -- 172.21.15.5:0/64154563 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1198035040 con 0x561052d8a5d0 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.052+0000 7f11b042c740 10 client.27325 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.052+0000 7f11b042c740 1 -- 172.21.15.5:0/64154563 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561052e0f810 con 0x561052d8a5d0 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.052+0000 7f1186ffd640 1 -- 172.21.15.5:0/64154563 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561052cf7d70 con 0x561052d8a5d0 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.053+0000 7f1186ffd640 1 -- 172.21.15.5:0/64154563 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f1198004bf0 con 0x561052d8a5d0 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.053+0000 7f11b042c740 10 client.27325 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.053+0000 7f11b042c740 10 client.27325 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.053+0000 7f11b042c740 20 client.27325 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.053+0000 7f11b042c740 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x561052e0ffc0 0x561052e123e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.053+0000 7f11b042c740 4 client.27325 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.053+0000 7f11b042c740 1 -- 172.21.15.5:0/64154563 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561052c74170 con 0x561052e0ffc0 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.053+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x561052e0ffc0 0x561052e123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.053+0000 7f1186ffd640 10 client.27325 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:12:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.053+0000 7f11a6ffd640 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x561052e0ffc0 0x561052e123e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b34c5aae667c470c server_cookie=b702b017c6e5f7da in_seq=0 out_seq=0 2024-09-19T06:12:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f1186ffd640 1 -- 172.21.15.5:0/64154563 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561052c74170 con 0x561052e0ffc0 2024-09-19T06:12:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f1186ffd640 10 client.27325 handle_command_reply: tid=0 2024-09-19T06:12:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f11b042c740 1 client.27325 shutdown 2024-09-19T06:12:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f11b042c740 1 -- 172.21.15.5:0/64154563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8a5d0 msgr2=0x561052d9dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f11b042c740 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8a5d0 0x561052d9dad0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f119802aa40 tx=0x7f1198003820 comp rx=0 tx=0).stop 2024-09-19T06:12:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f11b042c740 1 -- 172.21.15.5:0/64154563 shutdown_connections 2024-09-19T06:12:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f11b042c740 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x561052e0ffc0 0x561052e123e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f11b042c740 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561052d95370 0x561052d8f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f11b042c740 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561052d8bb40 0x561052d8ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f11b042c740 1 --2- 172.21.15.5:0/64154563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561052d8a5d0 0x561052d9dad0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f1186ffd640 0 client.27325 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:12:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f11b042c740 1 -- 172.21.15.5:0/64154563 >> 172.21.15.5:0/64154563 conn(0x561052d53310 msgr2=0x561052d8ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f11b042c740 1 -- 172.21.15.5:0/64154563 shutdown_connections 2024-09-19T06:12:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f11b042c740 1 -- 172.21.15.5:0/64154563 wait complete. 2024-09-19T06:12:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.054+0000 7f11b042c740 20 client.27325 trim_cache size 0 max 16384 2024-09-19T06:12:50.061 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:12:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.055+0000 7f11ae053640 1 -- 172.21.15.5:0/3878455356 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f11900a33b0 msgr2=0x7f11900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.055+0000 7f11ae053640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f11900a33b0 0x7f11900a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009fd0 tx=0x7f11a0014c30 comp rx=0 tx=0).stop 2024-09-19T06:12:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.055+0000 7f11ae053640 1 -- 172.21.15.5:0/3878455356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11a8102570 msgr2=0x7f11a81098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.055+0000 7f11ae053640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11a8102570 0x7f11a81098e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f119c00e9f0 tx=0x7f119c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.055+0000 7f11ae053640 1 -- 172.21.15.5:0/3878455356 shutdown_connections 2024-09-19T06:12:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.055+0000 7f11ae053640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a8102ef0 0x7f11a8109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.055+0000 7f11ae053640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f11900a33b0 0x7f11900a5870 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.055+0000 7f11ae053640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11a8102570 0x7f11a81098e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.055+0000 7f11ae053640 1 --2- 172.21.15.5:0/3878455356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a8100e20 0x7f11a810d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.055+0000 7f11ae053640 1 -- 172.21.15.5:0/3878455356 >> 172.21.15.5:0/3878455356 conn(0x7f11a80fc7d0 msgr2=0x7f11a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.055+0000 7f11ae053640 1 -- 172.21.15.5:0/3878455356 shutdown_connections 2024-09-19T06:12:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:50.055+0000 7f11ae053640 1 -- 172.21.15.5:0/3878455356 wait complete. 2024-09-19T06:12:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:50 smithi157 ceph-mon[35346]: pgmap v294: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 396 KiB/s wr, 48 op/s 2024-09-19T06:12:50.832 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:50 smithi005 ceph-mon[30955]: pgmap v294: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 396 KiB/s wr, 48 op/s 2024-09-19T06:12:50.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 240131 2.54 MB/sec execute 376 sec latency 21.355 ms 2024-09-19T06:12:50.861 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:12:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:50 smithi049 ceph-mon[34640]: pgmap v294: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 396 KiB/s wr, 48 op/s 2024-09-19T06:12:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:51.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.670+0000 7f572d356640 1 -- 172.21.15.5:0/2924423138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5728102570 msgr2=0x7f57281029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:51.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.670+0000 7f572d356640 1 --2- 172.21.15.5:0/2924423138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5728102570 0x7f57281029b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f571c009900 tx=0x7f571c02a480 comp rx=0 tx=0).stop 2024-09-19T06:12:51.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.670+0000 7f572d356640 1 -- 172.21.15.5:0/2924423138 shutdown_connections 2024-09-19T06:12:51.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.670+0000 7f572d356640 1 --2- 172.21.15.5:0/2924423138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5728102ef0 0x7f572810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.670+0000 7f572d356640 1 --2- 172.21.15.5:0/2924423138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5728102570 0x7f57281029b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.670+0000 7f572d356640 1 --2- 172.21.15.5:0/2924423138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5728100e20 0x7f5728101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.670+0000 7f572d356640 1 -- 172.21.15.5:0/2924423138 >> 172.21.15.5:0/2924423138 conn(0x7f57280fc7d0 msgr2=0x7f57280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:51.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.670+0000 7f572d356640 1 -- 172.21.15.5:0/2924423138 shutdown_connections 2024-09-19T06:12:51.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.670+0000 7f572d356640 1 -- 172.21.15.5:0/2924423138 wait complete. 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.670+0000 7f572d356640 1 Processor -- start 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f572d356640 1 -- start start 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f572d356640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5728100e20 0x7f5728197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f572d356640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5728102570 0x7f5728197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f572d356640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5728102ef0 0x7f572819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f572d356640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5728110e00 con 0x7f5728102570 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f572d356640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5728110c80 con 0x7f5728102ef0 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f572d356640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5728110f80 con 0x7f5728100e20 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f57267fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5728102570 0x7f5728197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f57267fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5728102570 0x7f5728197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44546/0 (socket says 172.21.15.5:44546) 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f57267fc640 1 -- 172.21.15.5:0/2434951966 learned_addr learned my addr 172.21.15.5:0/2434951966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f5726ffd640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5728100e20 0x7f5728197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f57277fe640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5728102ef0 0x7f572819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f57267fc640 1 -- 172.21.15.5:0/2434951966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5728100e20 msgr2=0x7f5728197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:51.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f57267fc640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5728100e20 0x7f5728197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f57267fc640 1 -- 172.21.15.5:0/2434951966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5728102ef0 msgr2=0x7f572819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:51.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f57267fc640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5728102ef0 0x7f572819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.671+0000 7f57267fc640 1 -- 172.21.15.5:0/2434951966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572819c2b0 con 0x7f5728102570 2024-09-19T06:12:51.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.672+0000 7f57267fc640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5728102570 0x7f5728197840 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f571c006010 tx=0x7f571c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:51.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.672+0000 7f570ffff640 1 -- 172.21.15.5:0/2434951966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c03f030 con 0x7f5728102570 2024-09-19T06:12:51.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.672+0000 7f572d356640 1 -- 172.21.15.5:0/2434951966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572819c540 con 0x7f5728102570 2024-09-19T06:12:51.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.672+0000 7f570ffff640 1 -- 172.21.15.5:0/2434951966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f571c003aa0 con 0x7f5728102570 2024-09-19T06:12:51.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.672+0000 7f572d356640 1 -- 172.21.15.5:0/2434951966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5728069f70 con 0x7f5728102570 2024-09-19T06:12:51.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.672+0000 7f570ffff640 1 -- 172.21.15.5:0/2434951966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c049060 con 0x7f5728102570 2024-09-19T06:12:51.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f570ffff640 1 -- 172.21.15.5:0/2434951966 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f571c042030 con 0x7f5728102570 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f572f72f740 1 Processor -- start 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f572f72f740 1 -- start start 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f572f72f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f572f72f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f572f72f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d5f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f572f72f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ec4d54c2f0 con 0x55ec4d5f1370 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f572f72f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ec4d4f3cf0 con 0x55ec4d5e65d0 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f572f72f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ec4d5482b0 con 0x55ec4d5e7b40 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f57277fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f57277fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5e6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43526/0 (socket says 172.21.15.5:43526) 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f57277fe640 1 -- 172.21.15.5:0/2238135417 learned_addr learned my addr 172.21.15.5:0/2238135417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f5726ffd640 1 --2- 172.21.15.5:0/2238135417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.675+0000 7f57267fc640 1 --2- 172.21.15.5:0/2238135417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d5f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.676+0000 7f57277fe640 1 -- 172.21.15.5:0/2238135417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e65d0 msgr2=0x55ec4d5e6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.676+0000 7f57277fe640 1 --2- 172.21.15.5:0/2238135417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5e6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.676+0000 7f57277fe640 1 -- 172.21.15.5:0/2238135417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 msgr2=0x55ec4d5f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.676+0000 7f57277fe640 1 --2- 172.21.15.5:0/2238135417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d5f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.676+0000 7f57277fe640 1 -- 172.21.15.5:0/2238135417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4d4d83a0 con 0x55ec4d5e7b40 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.676+0000 7f570ffff640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57100a33b0 0x7f57100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.676+0000 7f5726ffd640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57100a33b0 0x7f57100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.676+0000 7f57277fe640 1 --2- 172.21.15.5:0/2238135417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5e6090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f57200039f0 tx=0x7f5720003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=700b5f74b523f059 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.676+0000 7f570ffff640 1 -- 172.21.15.5:0/2434951966 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f571c0f0000 con 0x7f5728102570 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.676+0000 7f570d7fa640 1 -- 172.21.15.5:0/2238135417 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572001a470 con 0x55ec4d5e7b40 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.676+0000 7f570d7fa640 1 -- 172.21.15.5:0/2238135417 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5720015070 con 0x55ec4d5e7b40 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.676+0000 7f570d7fa640 1 -- 172.21.15.5:0/2238135417 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57200107e0 con 0x55ec4d5e7b40 2024-09-19T06:12:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.677+0000 7f572f72f740 1 -- 172.21.15.5:0/2238135417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 msgr2=0x55ec4d5e6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.677+0000 7f572f72f740 1 --2- 172.21.15.5:0/2238135417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5e6090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f57200039f0 tx=0x7f5720003eb0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.677+0000 7f5726ffd640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57100a33b0 0x7f57100a5870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55ec4d5e3ac0 tx=0x7f5718005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.677+0000 7f572f72f740 1 -- 172.21.15.5:0/2238135417 shutdown_connections 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.677+0000 7f572f72f740 1 --2- 172.21.15.5:0/2238135417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d5f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.677+0000 7f572f72f740 1 --2- 172.21.15.5:0/2238135417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.677+0000 7f572f72f740 1 --2- 172.21.15.5:0/2238135417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5e6090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.677+0000 7f572f72f740 1 -- 172.21.15.5:0/2238135417 >> 172.21.15.5:0/2238135417 conn(0x55ec4d5af310 msgr2=0x55ec4d5e2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.677+0000 7f572f72f740 1 -- 172.21.15.5:0/2238135417 shutdown_connections 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.677+0000 7f572f72f740 1 -- 172.21.15.5:0/2238135417 wait complete. 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f572f72f740 1 -- start start 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f572f72f740 1 Processor -- start 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f572f72f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f572f72f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5ee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f572f72f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d5ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f572f72f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ec4d5f4070 con 0x55ec4d5f1370 2024-09-19T06:12:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f572f72f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ec4d507060 con 0x55ec4d5e65d0 2024-09-19T06:12:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f572f72f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ec4d5f41f0 con 0x55ec4d5e7b40 2024-09-19T06:12:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f572f72f740 10 client.0 _ll_register_callbacks cb 0x55ec4d25a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f5726ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f57277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d5ee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f57267fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f57277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d5ee7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44552/0 (socket says 172.21.15.5:44552) 2024-09-19T06:12:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f57277fe640 1 -- 172.21.15.5:0/740383902 learned_addr learned my addr 172.21.15.5:0/740383902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f57267fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5ee060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43542/0 (socket says 172.21.15.5:43542) 2024-09-19T06:12:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f5726ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5f0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56918/0 (socket says 172.21.15.5:56918) 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f57267fc640 1 -- 172.21.15.5:0/740383902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e65d0 msgr2=0x55ec4d5f0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f57267fc640 1 --2- 172.21.15.5:0/740383902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5f0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f57267fc640 1 -- 172.21.15.5:0/740383902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 msgr2=0x55ec4d5ee7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f57267fc640 1 --2- 172.21.15.5:0/740383902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d5ee7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.678+0000 7f57267fc640 1 -- 172.21.15.5:0/740383902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4d2bd3c0 con 0x55ec4d5e7b40 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.679+0000 7f57267fc640 1 --2- 172.21.15.5:0/740383902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5ee060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f571c0ed730 tx=0x7f571c03b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5542474d8fa6a2f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.679+0000 7f570aff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.679+0000 7f570aff5640 1 -- 172.21.15.5:0/740383902 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c035050 con 0x55ec4d5e7b40 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.679+0000 7f570aff5640 1 -- 172.21.15.5:0/740383902 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f571c003c50 con 0x55ec4d5e7b40 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.679+0000 7f572f72f740 1 -- 172.21.15.5:0/740383902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5720015070 con 0x55ec4d5e7b40 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.679+0000 7f570aff5640 1 -- 172.21.15.5:0/740383902 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f571c0bc140 con 0x55ec4d5e7b40 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.679+0000 7f572f72f740 10 client.27337 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.679+0000 7f572f72f740 1 -- 172.21.15.5:0/740383902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ec4d5f4070 con 0x55ec4d5e7b40 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.679+0000 7f570aff5640 1 -- 172.21.15.5:0/740383902 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5720015070 con 0x55ec4d5e7b40 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.680+0000 7f570aff5640 1 -- 172.21.15.5:0/740383902 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f571c0d50d0 con 0x55ec4d5e7b40 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.680+0000 7f572f72f740 10 client.27337 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.680+0000 7f572f72f740 10 client.27337 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.680+0000 7f572f72f740 20 client.27337 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.680+0000 7f572f72f740 1 --2- 172.21.15.5:0/740383902 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ec4d600e90 0x55ec4d603280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.680+0000 7f572f72f740 4 client.27337 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.680+0000 7f572f72f740 1 -- 172.21.15.5:0/740383902 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec4d559860 con 0x55ec4d600e90 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.680+0000 7f5726ffd640 1 --2- 172.21.15.5:0/740383902 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ec4d600e90 0x55ec4d603280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.680+0000 7f570aff5640 10 client.27337 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.680+0000 7f5726ffd640 1 --2- 172.21.15.5:0/740383902 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ec4d600e90 0x55ec4d603280 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e5214b9845525f1a server_cookie=c529fb4bf9bde2b3 in_seq=0 out_seq=0 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f570aff5640 1 -- 172.21.15.5:0/740383902 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec4d559860 con 0x55ec4d600e90 2024-09-19T06:12:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f570aff5640 10 client.27337 handle_command_reply: tid=0 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f572f72f740 1 client.27337 shutdown 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f572f72f740 1 -- 172.21.15.5:0/740383902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 msgr2=0x55ec4d5ee060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f572f72f740 1 --2- 172.21.15.5:0/740383902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5ee060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f571c0ed730 tx=0x7f571c03b460 comp rx=0 tx=0).stop 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f572f72f740 1 -- 172.21.15.5:0/740383902 shutdown_connections 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f572f72f740 1 --2- 172.21.15.5:0/740383902 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ec4d600e90 0x55ec4d603280 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f572f72f740 1 --2- 172.21.15.5:0/740383902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d5ee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f572f72f740 1 --2- 172.21.15.5:0/740383902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5ee060 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f572f72f740 1 --2- 172.21.15.5:0/740383902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5f0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f570aff5640 0 client.27337 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f572f72f740 1 -- 172.21.15.5:0/740383902 >> 172.21.15.5:0/740383902 conn(0x55ec4d5af310 msgr2=0x55ec4d5e8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f572f72f740 1 -- 172.21.15.5:0/740383902 shutdown_connections 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f572f72f740 1 -- 172.21.15.5:0/740383902 wait complete. 2024-09-19T06:12:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.681+0000 7f572f72f740 20 client.27337 trim_cache size 0 max 16384 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.696+0000 7f572f72f740 1 Processor -- start 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.696+0000 7f572f72f740 1 -- start start 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.696+0000 7f572f72f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5ed040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.696+0000 7f572f72f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d608b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.696+0000 7f572f72f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5f1370 0x55ec4d609090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.696+0000 7f572f72f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57281ab5e0 con 0x55ec4d5e65d0 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.696+0000 7f572f72f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f57281ad150 con 0x55ec4d5f1370 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.696+0000 7f572f72f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57281abcb0 con 0x55ec4d5e7b40 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.696+0000 7f5726ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5ed040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.696+0000 7f57277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5f1370 0x55ec4d609090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.696+0000 7f5726ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5ed040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44558/0 (socket says 172.21.15.5:44558) 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.696+0000 7f5726ffd640 1 -- 172.21.15.5:0/217913054 learned_addr learned my addr 172.21.15.5:0/217913054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.697+0000 7f57267fc640 1 --2- 172.21.15.5:0/217913054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d608b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.697+0000 7f57277fe640 1 -- 172.21.15.5:0/217913054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 msgr2=0x55ec4d608b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.697+0000 7f57277fe640 1 --2- 172.21.15.5:0/217913054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d608b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.697+0000 7f57277fe640 1 -- 172.21.15.5:0/217913054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5e65d0 msgr2=0x55ec4d5ed040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.697+0000 7f57277fe640 1 --2- 172.21.15.5:0/217913054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5ed040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.697+0000 7f57277fe640 1 -- 172.21.15.5:0/217913054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4d5ec5b0 con 0x55ec4d5f1370 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.697+0000 7f57277fe640 1 --2- 172.21.15.5:0/217913054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5f1370 0x55ec4d609090 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5720000f80 tx=0x7f5720011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da178b95174843a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.697+0000 7f570cff9640 1 -- 172.21.15.5:0/217913054 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720011640 con 0x55ec4d5f1370 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.697+0000 7f570cff9640 1 -- 172.21.15.5:0/217913054 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5720015040 con 0x55ec4d5f1370 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.697+0000 7f570cff9640 1 -- 172.21.15.5:0/217913054 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572001ab10 con 0x55ec4d5f1370 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.698+0000 7f572f72f740 1 -- 172.21.15.5:0/217913054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5f1370 msgr2=0x55ec4d609090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.698+0000 7f572f72f740 1 --2- 172.21.15.5:0/217913054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5f1370 0x55ec4d609090 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5720000f80 tx=0x7f5720011f90 comp rx=0 tx=0).stop 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.698+0000 7f572f72f740 1 -- 172.21.15.5:0/217913054 shutdown_connections 2024-09-19T06:12:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.698+0000 7f572f72f740 1 --2- 172.21.15.5:0/217913054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5f1370 0x55ec4d609090 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.698+0000 7f572f72f740 1 --2- 172.21.15.5:0/217913054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e7b40 0x55ec4d608b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.698+0000 7f572f72f740 1 --2- 172.21.15.5:0/217913054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5ed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.698+0000 7f572f72f740 1 -- 172.21.15.5:0/217913054 >> 172.21.15.5:0/217913054 conn(0x55ec4d5af310 msgr2=0x55ec4d5ec930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.698+0000 7f572f72f740 1 -- 172.21.15.5:0/217913054 shutdown_connections 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.698+0000 7f572f72f740 1 -- 172.21.15.5:0/217913054 wait complete. 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f572f72f740 1 -- start start 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f572f72f740 1 Processor -- start 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f572f72f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5f5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f572f72f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f572f72f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d66ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f572f72f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57281aca80 con 0x55ec4d5f1370 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f572f72f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f57281ac380 con 0x55ec4d5e7b40 2024-09-19T06:12:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f572f72f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57281a95e0 con 0x55ec4d5e65d0 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f572f72f740 10 client.0 _ll_register_callbacks cb 0x7f571c005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f57277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d66ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f57277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d66ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44566/0 (socket says 172.21.15.5:44566) 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f57277fe640 1 -- 172.21.15.5:0/961979212 learned_addr learned my addr 172.21.15.5:0/961979212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f57267fc640 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f5726ffd640 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f57277fe640 1 -- 172.21.15.5:0/961979212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e65d0 msgr2=0x55ec4d5f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f57277fe640 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f57277fe640 1 -- 172.21.15.5:0/961979212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e7b40 msgr2=0x55ec4d5f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f57277fe640 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f57277fe640 1 -- 172.21.15.5:0/961979212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4d66b060 con 0x55ec4d5f1370 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f5726ffd640 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.699+0000 7f57267fc640 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.700+0000 7f57277fe640 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d66ab20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5720009990 tx=0x7f5720009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b51e6a96eb71fd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.700+0000 7f570aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.700+0000 7f570aff5640 1 -- 172.21.15.5:0/961979212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572001aae0 con 0x55ec4d5f1370 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.700+0000 7f570aff5640 1 -- 172.21.15.5:0/961979212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f572001ac80 con 0x55ec4d5f1370 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.700+0000 7f572f72f740 1 -- 172.21.15.5:0/961979212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5720015040 con 0x55ec4d5f1370 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.700+0000 7f570aff5640 1 -- 172.21.15.5:0/961979212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f572003e050 con 0x55ec4d5f1370 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.700+0000 7f572f72f740 10 client.17874 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.700+0000 7f572f72f740 1 -- 172.21.15.5:0/961979212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ec4d66c1b0 con 0x55ec4d5f1370 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.701+0000 7f570aff5640 1 -- 172.21.15.5:0/961979212 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5720015040 con 0x55ec4d5f1370 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.701+0000 7f570aff5640 1 -- 172.21.15.5:0/961979212 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f5720030510 con 0x55ec4d5f1370 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.701+0000 7f572f72f740 10 client.17874 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.701+0000 7f572f72f740 10 client.17874 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:12:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.701+0000 7f572f72f740 20 client.17874 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.701+0000 7f572f72f740 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ec4d66c620 0x55ec4d66ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.701+0000 7f572f72f740 4 client.17874 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.701+0000 7f572f72f740 1 -- 172.21.15.5:0/961979212 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec4d553d70 con 0x55ec4d66c620 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.701+0000 7f5726ffd640 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ec4d66c620 0x55ec4d66ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.701+0000 7f5726ffd640 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ec4d66c620 0x55ec4d66ea10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dfbee5505c89f0c4 server_cookie=b151dc65a57468ea in_seq=0 out_seq=0 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f570aff5640 10 client.17874 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f570aff5640 1 -- 172.21.15.5:0/961979212 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec4d553d70 con 0x55ec4d66c620 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f570aff5640 10 client.17874 handle_command_reply: tid=0 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f572f72f740 1 client.17874 shutdown 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f572f72f740 1 -- 172.21.15.5:0/961979212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 msgr2=0x55ec4d66ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f572f72f740 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d66ab20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5720009990 tx=0x7f5720009c50 comp rx=0 tx=0).stop 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f572f72f740 1 -- 172.21.15.5:0/961979212 shutdown_connections 2024-09-19T06:12:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f572f72f740 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ec4d66c620 0x55ec4d66ea10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f572f72f740 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4d5f1370 0x55ec4d66ab20 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f572f72f740 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4d5e7b40 0x55ec4d5f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f572f72f740 1 --2- 172.21.15.5:0/961979212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4d5e65d0 0x55ec4d5f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f572f72f740 1 -- 172.21.15.5:0/961979212 >> 172.21.15.5:0/961979212 conn(0x55ec4d5af310 msgr2=0x55ec4d5ef8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f570aff5640 0 client.17874 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:12:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f572f72f740 1 -- 172.21.15.5:0/961979212 shutdown_connections 2024-09-19T06:12:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f572f72f740 1 -- 172.21.15.5:0/961979212 wait complete. 2024-09-19T06:12:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.702+0000 7f572f72f740 20 client.17874 trim_cache size 0 max 16384 2024-09-19T06:12:51.721 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:12:51.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.703+0000 7f572d356640 1 -- 172.21.15.5:0/2434951966 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57100a33b0 msgr2=0x7f57100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:51.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.703+0000 7f572d356640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57100a33b0 0x7f57100a5870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55ec4d5e3ac0 tx=0x7f5718005f90 comp rx=0 tx=0).stop 2024-09-19T06:12:51.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.703+0000 7f572d356640 1 -- 172.21.15.5:0/2434951966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5728102570 msgr2=0x7f5728197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.703+0000 7f572d356640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5728102570 0x7f5728197840 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f571c006010 tx=0x7f571c004270 comp rx=0 tx=0).stop 2024-09-19T06:12:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.703+0000 7f572d356640 1 -- 172.21.15.5:0/2434951966 shutdown_connections 2024-09-19T06:12:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.703+0000 7f572d356640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5728102ef0 0x7f572819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.703+0000 7f572d356640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5728102570 0x7f5728197840 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.703+0000 7f572d356640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57100a33b0 0x7f57100a5870 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.703+0000 7f572d356640 1 --2- 172.21.15.5:0/2434951966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5728100e20 0x7f5728197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.703+0000 7f572d356640 1 -- 172.21.15.5:0/2434951966 >> 172.21.15.5:0/2434951966 conn(0x7f57280fc7d0 msgr2=0x7f57280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.703+0000 7f572d356640 1 -- 172.21.15.5:0/2434951966 shutdown_connections 2024-09-19T06:12:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:51.703+0000 7f572d356640 1 -- 172.21.15.5:0/2434951966 wait complete. 2024-09-19T06:12:51.837 INFO:tasks.workunit.client.0.smithi005.stdout: 1 240718 2.55 MB/sec execute 377 sec latency 19.303 ms 2024-09-19T06:12:52.422 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:12:52.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:52 smithi005 ceph-mon[30955]: pgmap v295: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 396 KiB/s wr, 48 op/s 2024-09-19T06:12:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:52 smithi157 ceph-mon[35346]: pgmap v295: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 396 KiB/s wr, 48 op/s 2024-09-19T06:12:52.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241237 2.56 MB/sec execute 378 sec latency 17.967 ms 2024-09-19T06:12:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:52 smithi049 ceph-mon[34640]: pgmap v295: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 396 KiB/s wr, 48 op/s 2024-09-19T06:12:52.915 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.419+0000 7f3fd4960640 1 -- 172.21.15.5:0/2130625092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd0102ef0 msgr2=0x7f3fd010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.419+0000 7f3fd4960640 1 --2- 172.21.15.5:0/2130625092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd0102ef0 0x7f3fd010da40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009ca0 tx=0x7f3fc802a950 comp rx=0 tx=0).stop 2024-09-19T06:12:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.419+0000 7f3fd4960640 1 -- 172.21.15.5:0/2130625092 shutdown_connections 2024-09-19T06:12:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.419+0000 7f3fd4960640 1 --2- 172.21.15.5:0/2130625092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd0102ef0 0x7f3fd010da40 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.419+0000 7f3fd4960640 1 --2- 172.21.15.5:0/2130625092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102570 0x7f3fd01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.419+0000 7f3fd4960640 1 --2- 172.21.15.5:0/2130625092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fd0100e20 0x7f3fd0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.419+0000 7f3fd4960640 1 -- 172.21.15.5:0/2130625092 >> 172.21.15.5:0/2130625092 conn(0x7f3fd00fc7d0 msgr2=0x7f3fd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.419+0000 7f3fd4960640 1 -- 172.21.15.5:0/2130625092 shutdown_connections 2024-09-19T06:12:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.419+0000 7f3fd4960640 1 -- 172.21.15.5:0/2130625092 wait complete. 2024-09-19T06:12:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.420+0000 7f3fd4960640 1 Processor -- start 2024-09-19T06:12:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.420+0000 7f3fd4960640 1 -- start start 2024-09-19T06:12:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.420+0000 7f3fd4960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd0100e20 0x7f3fd0107a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.420+0000 7f3fd4960640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fd0102570 0x7f3fd0107f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.420+0000 7f3fd4960640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102ef0 0x7f3fd01084d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.420+0000 7f3fd4960640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd0110e30 con 0x7f3fd0100e20 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.420+0000 7f3fd4960640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd0110cb0 con 0x7f3fd0102570 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.420+0000 7f3fd4960640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd0110fb0 con 0x7f3fd0102ef0 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.420+0000 7f3fce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd0100e20 0x7f3fd0107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.420+0000 7f3fce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd0100e20 0x7f3fd0107a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44580/0 (socket says 172.21.15.5:44580) 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.420+0000 7f3fce59c640 1 -- 172.21.15.5:0/1826792080 learned_addr learned my addr 172.21.15.5:0/1826792080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.420+0000 7f3fced9d640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102ef0 0x7f3fd01084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.421+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fd0102570 0x7f3fd0107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.421+0000 7f3fce59c640 1 -- 172.21.15.5:0/1826792080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102ef0 msgr2=0x7f3fd01084d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.421+0000 7f3fce59c640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102ef0 0x7f3fd01084d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.421+0000 7f3fce59c640 1 -- 172.21.15.5:0/1826792080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fd0102570 msgr2=0x7f3fd0107f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.421+0000 7f3fce59c640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fd0102570 0x7f3fd0107f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.421+0000 7f3fce59c640 1 -- 172.21.15.5:0/1826792080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd01aac80 con 0x7f3fd0100e20 2024-09-19T06:12:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.421+0000 7f3fce59c640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd0100e20 0x7f3fd0107a50 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000d980 tx=0x7f3fc000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.421+0000 7f3fc7fff640 1 -- 172.21.15.5:0/1826792080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0017030 con 0x7f3fd0100e20 2024-09-19T06:12:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.421+0000 7f3fc7fff640 1 -- 172.21.15.5:0/1826792080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fc0004020 con 0x7f3fd0100e20 2024-09-19T06:12:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.421+0000 7f3fd4960640 1 -- 172.21.15.5:0/1826792080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd01aaf70 con 0x7f3fd0100e20 2024-09-19T06:12:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.421+0000 7f3fc7fff640 1 -- 172.21.15.5:0/1826792080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0005640 con 0x7f3fd0100e20 2024-09-19T06:12:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.421+0000 7f3fd4960640 1 -- 172.21.15.5:0/1826792080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd01ab420 con 0x7f3fd0100e20 2024-09-19T06:12:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.423+0000 7f3fc7fff640 1 -- 172.21.15.5:0/1826792080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3fc0009a30 con 0x7f3fd0100e20 2024-09-19T06:12:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.423+0000 7f3fd6d39740 1 Processor -- start 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fd6d39740 1 -- start start 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fd6d39740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32eb40 0x55ddde32d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fd6d39740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32d5d0 0x55ddde32da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fd6d39740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde338370 0x55ddde33a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fd6d39740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ddde2932f0 con 0x55ddde32eb40 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fd6d39740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ddde23acf0 con 0x55ddde32d5d0 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fd6d39740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ddde28f2b0 con 0x55ddde338370 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32eb40 0x55ddde32d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32eb40 0x55ddde32d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44586/0 (socket says 172.21.15.5:44586) 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fced9d640 1 -- 172.21.15.5:0/1323149054 learned_addr learned my addr 172.21.15.5:0/1323149054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/1323149054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde338370 0x55ddde33a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fce59c640 1 --2- 172.21.15.5:0/1323149054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32d5d0 0x55ddde32da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fc7fff640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3fbc0a3550 0x7f3fbc0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fc5d9b640 1 -- 172.21.15.5:0/1323149054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32d5d0 msgr2=0x55ddde32da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/1323149054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32d5d0 0x55ddde32da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fc5d9b640 1 -- 172.21.15.5:0/1323149054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32eb40 msgr2=0x55ddde32d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/1323149054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32eb40 0x55ddde32d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.424+0000 7f3fc5d9b640 1 -- 172.21.15.5:0/1323149054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddde21f3a0 con 0x55ddde338370 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.425+0000 7f3fced9d640 1 --2- 172.21.15.5:0/1323149054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32eb40 0x55ddde32d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.425+0000 7f3fce59c640 1 --2- 172.21.15.5:0/1323149054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32d5d0 0x55ddde32da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.425+0000 7f3fced9d640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3fbc0a3550 0x7f3fbc0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.425+0000 7f3fc7fff640 1 -- 172.21.15.5:0/1826792080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f3fc00cf7f0 con 0x7f3fd0100e20 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.425+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/1323149054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde338370 0x55ddde33a760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3fd01092a0 tx=0x7f3fb8009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cf0e53442ee186 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.425+0000 7f3fced9d640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3fbc0a3550 0x7f3fbc0a5a10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009430 tx=0x7f3fc803c040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.425+0000 7f3fc4d99640 1 -- 172.21.15.5:0/1323149054 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8016070 con 0x55ddde338370 2024-09-19T06:12:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.425+0000 7f3fc4d99640 1 -- 172.21.15.5:0/1323149054 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb800ebc0 con 0x55ddde338370 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.425+0000 7f3fc4d99640 1 -- 172.21.15.5:0/1323149054 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8010610 con 0x55ddde338370 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.426+0000 7f3fd6d39740 1 -- 172.21.15.5:0/1323149054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde338370 msgr2=0x55ddde33a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.426+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/1323149054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde338370 0x55ddde33a760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3fd01092a0 tx=0x7f3fb8009ea0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.426+0000 7f3fd6d39740 1 -- 172.21.15.5:0/1323149054 shutdown_connections 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.426+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/1323149054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde338370 0x55ddde33a760 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.426+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/1323149054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32d5d0 0x55ddde32da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.426+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/1323149054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32eb40 0x55ddde32d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.426+0000 7f3fd6d39740 1 -- 172.21.15.5:0/1323149054 >> 172.21.15.5:0/1323149054 conn(0x55ddde2f6310 msgr2=0x55ddde329c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.426+0000 7f3fd6d39740 1 -- 172.21.15.5:0/1323149054 shutdown_connections 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.426+0000 7f3fd6d39740 1 -- 172.21.15.5:0/1323149054 wait complete. 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.426+0000 7f3fd6d39740 1 -- start start 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.426+0000 7f3fd6d39740 1 Processor -- start 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fd6d39740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32d5d0 0x55ddde337ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fd6d39740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32eb40 0x55ddde334f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fd6d39740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde338370 0x55ddde335490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fd6d39740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ddde33b070 con 0x55ddde338370 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fd6d39740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ddde24e060 con 0x55ddde32eb40 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fd6d39740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ddde33b1f0 con 0x55ddde32d5d0 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fd6d39740 10 client.0 _ll_register_callbacks cb 0x55ddddfa18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fce59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32d5d0 0x55ddde337ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde338370 0x55ddde335490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fce59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32d5d0 0x55ddde337ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43592/0 (socket says 172.21.15.5:43592) 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fce59c640 1 -- 172.21.15.5:0/3610454550 learned_addr learned my addr 172.21.15.5:0/3610454550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32eb40 0x55ddde334f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fce59c640 1 -- 172.21.15.5:0/3610454550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32eb40 msgr2=0x55ddde334f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fce59c640 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32eb40 0x55ddde334f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fce59c640 1 -- 172.21.15.5:0/3610454550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde338370 msgr2=0x55ddde335490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fce59c640 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde338370 0x55ddde335490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fce59c640 1 -- 172.21.15.5:0/3610454550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddde0043c0 con 0x55ddde32d5d0 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32eb40 0x55ddde334f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fce59c640 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32d5d0 0x55ddde337ca0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0098420 tx=0x7f3fc009b4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47aa2f4dcd67a502 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.427+0000 7f3fb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fb5ffb640 1 -- 172.21.15.5:0/3610454550 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00a46c0 con 0x55ddde32d5d0 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fb5ffb640 1 -- 172.21.15.5:0/3610454550 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f3fc00980c0 con 0x55ddde32d5d0 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fd6d39740 1 -- 172.21.15.5:0/3610454550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb800ebc0 con 0x55ddde32d5d0 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fb5ffb640 1 -- 172.21.15.5:0/3610454550 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3fc0086070 con 0x55ddde32d5d0 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fd6d39740 10 client.27355 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fd6d39740 1 -- 172.21.15.5:0/3610454550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ddde346a90 con 0x55ddde32d5d0 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fb5ffb640 1 -- 172.21.15.5:0/3610454550 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb800ebc0 con 0x55ddde32d5d0 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fb5ffb640 1 -- 172.21.15.5:0/3610454550 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f3fc00977a0 con 0x55ddde32d5d0 2024-09-19T06:12:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fd6d39740 10 client.27355 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fd6d39740 10 client.27355 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:12:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fd6d39740 20 client.27355 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ddde347ef0 0x55ddde34a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.428+0000 7f3fd6d39740 4 client.27355 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.429+0000 7f3fd6d39740 1 -- 172.21.15.5:0/3610454550 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddde2a0860 con 0x55ddde347ef0 2024-09-19T06:12:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.429+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ddde347ef0 0x55ddde34a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.429+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ddde347ef0 0x55ddde34a380 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef311573422d922 server_cookie=35f21c13fc5b146a in_seq=0 out_seq=0 2024-09-19T06:12:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.429+0000 7f3fb5ffb640 10 client.27355 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:12:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fb5ffb640 1 -- 172.21.15.5:0/3610454550 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddde2a0860 con 0x55ddde347ef0 2024-09-19T06:12:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fb5ffb640 10 client.27355 handle_command_reply: tid=0 2024-09-19T06:12:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fd6d39740 1 client.27355 shutdown 2024-09-19T06:12:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fd6d39740 1 -- 172.21.15.5:0/3610454550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32d5d0 msgr2=0x55ddde337ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32d5d0 0x55ddde337ca0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0098420 tx=0x7f3fc009b4a0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fd6d39740 1 -- 172.21.15.5:0/3610454550 shutdown_connections 2024-09-19T06:12:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ddde347ef0 0x55ddde34a380 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde338370 0x55ddde335490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32eb40 0x55ddde334f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fb5ffb640 0 client.27355 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:12:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/3610454550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32d5d0 0x55ddde337ca0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fd6d39740 1 -- 172.21.15.5:0/3610454550 >> 172.21.15.5:0/3610454550 conn(0x55ddde2f6310 msgr2=0x55ddde32f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fd6d39740 1 -- 172.21.15.5:0/3610454550 shutdown_connections 2024-09-19T06:12:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fd6d39740 1 -- 172.21.15.5:0/3610454550 wait complete. 2024-09-19T06:12:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.430+0000 7f3fd6d39740 20 client.27355 trim_cache size 0 max 16384 2024-09-19T06:12:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.443+0000 7f3fd6d39740 1 Processor -- start 2024-09-19T06:12:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.443+0000 7f3fd6d39740 1 -- start start 2024-09-19T06:12:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fd6d39740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde2f6310 0x55ddde329e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fd6d39740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32d5d0 0x55ddde344340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fd6d39740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32eb40 0x55ddde334810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fd6d39740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd01b3ec0 con 0x55ddde32d5d0 2024-09-19T06:12:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fd6d39740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd01b5a30 con 0x55ddde2f6310 2024-09-19T06:12:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fd6d39740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd01b4590 con 0x55ddde32eb40 2024-09-19T06:12:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fc5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32d5d0 0x55ddde344340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fc5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32d5d0 0x55ddde344340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44602/0 (socket says 172.21.15.5:44602) 2024-09-19T06:12:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fc5d9b640 1 -- 172.21.15.5:0/406290743 learned_addr learned my addr 172.21.15.5:0/406290743 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fce59c640 1 --2- 172.21.15.5:0/406290743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde2f6310 0x55ddde329e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fced9d640 1 --2- 172.21.15.5:0/406290743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32eb40 0x55ddde334810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fc5d9b640 1 -- 172.21.15.5:0/406290743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32eb40 msgr2=0x55ddde334810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:53.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/406290743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32eb40 0x55ddde334810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fc5d9b640 1 -- 172.21.15.5:0/406290743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde2f6310 msgr2=0x55ddde329e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:53.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/406290743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde2f6310 0x55ddde329e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fc5d9b640 1 -- 172.21.15.5:0/406290743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddde34aa00 con 0x55ddde32d5d0 2024-09-19T06:12:53.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/406290743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32d5d0 0x55ddde344340 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8011e90 tx=0x7f3fb800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fd08120a494b4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fb7fff640 1 -- 172.21.15.5:0/406290743 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb802f070 con 0x55ddde32d5d0 2024-09-19T06:12:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.444+0000 7f3fb7fff640 1 -- 172.21.15.5:0/406290743 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb800ebc0 con 0x55ddde32d5d0 2024-09-19T06:12:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.445+0000 7f3fb7fff640 1 -- 172.21.15.5:0/406290743 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8016040 con 0x55ddde32d5d0 2024-09-19T06:12:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.445+0000 7f3fd6d39740 1 -- 172.21.15.5:0/406290743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32d5d0 msgr2=0x55ddde344340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.445+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/406290743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32d5d0 0x55ddde344340 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8011e90 tx=0x7f3fb800ee70 comp rx=0 tx=0).stop 2024-09-19T06:12:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.445+0000 7f3fd6d39740 1 -- 172.21.15.5:0/406290743 shutdown_connections 2024-09-19T06:12:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.445+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/406290743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32eb40 0x55ddde334810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.445+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/406290743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde32d5d0 0x55ddde344340 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.445+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/406290743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde2f6310 0x55ddde329e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.445+0000 7f3fd6d39740 1 -- 172.21.15.5:0/406290743 >> 172.21.15.5:0/406290743 conn(0x55ddde3355f0 msgr2=0x55ddde329700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.445+0000 7f3fd6d39740 1 -- 172.21.15.5:0/406290743 shutdown_connections 2024-09-19T06:12:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.445+0000 7f3fd6d39740 1 -- 172.21.15.5:0/406290743 wait complete. 2024-09-19T06:12:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fd6d39740 1 -- start start 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fd6d39740 1 Processor -- start 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fd6d39740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde2f6310 0x55ddde331ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fd6d39740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32d5d0 0x55ddde332420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fd6d39740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32eb40 0x55ddde332960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fd6d39740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd01b5360 con 0x55ddde2f6310 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fd6d39740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd01b4c60 con 0x55ddde32d5d0 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fd6d39740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd01b1ec0 con 0x55ddde32eb40 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fd6d39740 10 client.0 _ll_register_callbacks cb 0x7f3fc000d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde2f6310 0x55ddde331ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde2f6310 0x55ddde331ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44604/0 (socket says 172.21.15.5:44604) 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fce59c640 1 -- 172.21.15.5:0/554402533 learned_addr learned my addr 172.21.15.5:0/554402533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32d5d0 0x55ddde332420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fced9d640 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32eb40 0x55ddde332960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fce59c640 1 -- 172.21.15.5:0/554402533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32eb40 msgr2=0x55ddde332960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fce59c640 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32eb40 0x55ddde332960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fce59c640 1 -- 172.21.15.5:0/554402533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32d5d0 msgr2=0x55ddde332420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fce59c640 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32d5d0 0x55ddde332420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.446+0000 7f3fce59c640 1 -- 172.21.15.5:0/554402533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddde3b1c20 con 0x55ddde2f6310 2024-09-19T06:12:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fce59c640 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde2f6310 0x55ddde331ee0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0097eb0 tx=0x7f3fc009b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5ef9a1c133242b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fb5ffb640 1 -- 172.21.15.5:0/554402533 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00979b0 con 0x55ddde2f6310 2024-09-19T06:12:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fb5ffb640 1 -- 172.21.15.5:0/554402533 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f3fc009aea0 con 0x55ddde2f6310 2024-09-19T06:12:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fd6d39740 1 -- 172.21.15.5:0/554402533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb800ebc0 con 0x55ddde2f6310 2024-09-19T06:12:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fb5ffb640 1 -- 172.21.15.5:0/554402533 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3fc00cd340 con 0x55ddde2f6310 2024-09-19T06:12:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fd6d39740 10 client.17892 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fd6d39740 1 -- 172.21.15.5:0/554402533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ddde3b2e20 con 0x55ddde2f6310 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fb5ffb640 1 -- 172.21.15.5:0/554402533 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb800ebc0 con 0x55ddde2f6310 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fb5ffb640 1 -- 172.21.15.5:0/554402533 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f3fc00a0f90 con 0x55ddde2f6310 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fd6d39740 10 client.17892 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fd6d39740 10 client.17892 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fd6d39740 20 client.17892 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ddde338370 0x55ddde3b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fd6d39740 4 client.17892 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.447+0000 7f3fd6d39740 1 -- 172.21.15.5:0/554402533 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddde29ad70 con 0x55ddde338370 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ddde338370 0x55ddde3b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fc5d9b640 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ddde338370 0x55ddde3b54e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51d9790bf1cf7c06 server_cookie=c7c7d894839bf5d7 in_seq=0 out_seq=0 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fb5ffb640 10 client.17892 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:12:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fb5ffb640 1 -- 172.21.15.5:0/554402533 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddde29ad70 con 0x55ddde338370 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fb5ffb640 10 client.17892 handle_command_reply: tid=0 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fd6d39740 1 client.17892 shutdown 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fd6d39740 1 -- 172.21.15.5:0/554402533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde2f6310 msgr2=0x55ddde331ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde2f6310 0x55ddde331ee0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0097eb0 tx=0x7f3fc009b610 comp rx=0 tx=0).stop 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fd6d39740 1 -- 172.21.15.5:0/554402533 shutdown_connections 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ddde338370 0x55ddde3b54e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddde32eb40 0x55ddde332960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddde32d5d0 0x55ddde332420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fb5ffb640 0 client.17892 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fd6d39740 1 --2- 172.21.15.5:0/554402533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ddde2f6310 0x55ddde331ee0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.448+0000 7f3fd6d39740 1 -- 172.21.15.5:0/554402533 >> 172.21.15.5:0/554402533 conn(0x55ddde3355f0 msgr2=0x55ddde32f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd6d39740 1 -- 172.21.15.5:0/554402533 shutdown_connections 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd6d39740 1 -- 172.21.15.5:0/554402533 wait complete. 2024-09-19T06:12:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd6d39740 20 client.17892 trim_cache size 0 max 16384 2024-09-19T06:12:53.456 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:12:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd4960640 1 -- 172.21.15.5:0/1826792080 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3fbc0a3550 msgr2=0x7f3fbc0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd4960640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3fbc0a3550 0x7f3fbc0a5a10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009430 tx=0x7f3fc803c040 comp rx=0 tx=0).stop 2024-09-19T06:12:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd4960640 1 -- 172.21.15.5:0/1826792080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd0100e20 msgr2=0x7f3fd0107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd4960640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd0100e20 0x7f3fd0107a50 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000d980 tx=0x7f3fc000de40 comp rx=0 tx=0).stop 2024-09-19T06:12:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd4960640 1 -- 172.21.15.5:0/1826792080 shutdown_connections 2024-09-19T06:12:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd4960640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3fbc0a3550 0x7f3fbc0a5a10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd4960640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102ef0 0x7f3fd01084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd4960640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fd0102570 0x7f3fd0107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd4960640 1 --2- 172.21.15.5:0/1826792080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd0100e20 0x7f3fd0107a50 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd4960640 1 -- 172.21.15.5:0/1826792080 >> 172.21.15.5:0/1826792080 conn(0x7f3fd00fc7d0 msgr2=0x7f3fd006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd4960640 1 -- 172.21.15.5:0/1826792080 shutdown_connections 2024-09-19T06:12:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:53.449+0000 7f3fd4960640 1 -- 172.21.15.5:0/1826792080 wait complete. 2024-09-19T06:12:53.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241310 2.55 MB/sec execute 379 sec latency 869.721 ms 2024-09-19T06:12:54.065 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:12:54.366 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:54.698 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:54 smithi049 ceph-mon[34640]: pgmap v296: 97 pgs: 97 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-09-19T06:12:54.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:54 smithi005 ceph-mon[30955]: pgmap v296: 97 pgs: 97 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-09-19T06:12:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:54 smithi157 ceph-mon[35346]: pgmap v296: 97 pgs: 97 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-09-19T06:12:54.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241310 2.54 MB/sec execute 380 sec latency 1869.748 ms 2024-09-19T06:12:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.811+0000 7f94ae594640 1 -- 172.21.15.5:0/2659548143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94a8100e20 msgr2=0x7f94a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.811+0000 7f94ae594640 1 --2- 172.21.15.5:0/2659548143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94a8100e20 0x7f94a8101200 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9498009900 tx=0x7f949802a480 comp rx=0 tx=0).stop 2024-09-19T06:12:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.811+0000 7f94ae594640 1 -- 172.21.15.5:0/2659548143 shutdown_connections 2024-09-19T06:12:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.811+0000 7f94ae594640 1 --2- 172.21.15.5:0/2659548143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94a8102ef0 0x7f94a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.811+0000 7f94ae594640 1 --2- 172.21.15.5:0/2659548143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a8102570 0x7f94a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.811+0000 7f94ae594640 1 --2- 172.21.15.5:0/2659548143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94a8100e20 0x7f94a8101200 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.811+0000 7f94ae594640 1 -- 172.21.15.5:0/2659548143 >> 172.21.15.5:0/2659548143 conn(0x7f94a80fc7d0 msgr2=0x7f94a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.811+0000 7f94ae594640 1 -- 172.21.15.5:0/2659548143 shutdown_connections 2024-09-19T06:12:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.811+0000 7f94ae594640 1 -- 172.21.15.5:0/2659548143 wait complete. 2024-09-19T06:12:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.812+0000 7f94ae594640 1 Processor -- start 2024-09-19T06:12:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.812+0000 7f94ae594640 1 -- start start 2024-09-19T06:12:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.812+0000 7f94ae594640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a8100e20 0x7f94a8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.812+0000 7f94ae594640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94a8102570 0x7f94a8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.812+0000 7f94ae594640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94a8102ef0 0x7f94a819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.812+0000 7f94ae594640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8110e30 con 0x7f94a8102570 2024-09-19T06:12:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.812+0000 7f94ae594640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8110cb0 con 0x7f94a8100e20 2024-09-19T06:12:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.812+0000 7f94ae594640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8110fb0 con 0x7f94a8102ef0 2024-09-19T06:12:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.812+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94a8102570 0x7f94a8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.812+0000 7f94a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a8100e20 0x7f94a8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.812+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94a8102570 0x7f94a8198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49268/0 (socket says 172.21.15.5:49268) 2024-09-19T06:12:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94acb31640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94a8102ef0 0x7f94a819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a8100e20 0x7f94a8198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35104/0 (socket says 172.21.15.5:35104) 2024-09-19T06:12:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94a77fe640 1 -- 172.21.15.5:0/419790220 learned_addr learned my addr 172.21.15.5:0/419790220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94a77fe640 1 -- 172.21.15.5:0/419790220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94a8102ef0 msgr2=0x7f94a819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94a77fe640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94a8102ef0 0x7f94a819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94a77fe640 1 -- 172.21.15.5:0/419790220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a8100e20 msgr2=0x7f94a8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94a77fe640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a8100e20 0x7f94a8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94a77fe640 1 -- 172.21.15.5:0/419790220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a819d400 con 0x7f94a8102570 2024-09-19T06:12:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94a7fff640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a8100e20 0x7f94a8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94a77fe640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94a8102570 0x7f94a8198990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e9f0 tx=0x7f949c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94a57fa640 1 -- 172.21.15.5:0/419790220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c00cd80 con 0x7f94a8102570 2024-09-19T06:12:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94ae594640 1 -- 172.21.15.5:0/419790220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a819d6f0 con 0x7f94a8102570 2024-09-19T06:12:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94a57fa640 1 -- 172.21.15.5:0/419790220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f949c0108e0 con 0x7f94a8102570 2024-09-19T06:12:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94a57fa640 1 -- 172.21.15.5:0/419790220 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c018920 con 0x7f94a8102570 2024-09-19T06:12:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.813+0000 7f94ae594640 1 -- 172.21.15.5:0/419790220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81a4fd0 con 0x7f94a8102570 2024-09-19T06:12:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94a57fa640 1 -- 172.21.15.5:0/419790220 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f949c010a90 con 0x7f94a8102570 2024-09-19T06:12:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94b096d740 1 Processor -- start 2024-09-19T06:12:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94b096d740 1 -- start start 2024-09-19T06:12:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94b096d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd96b40 0x55bb3cd95090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94b096d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd955d0 0x55bb3cd95a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94b096d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cda2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94b096d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ccfb2f0 con 0x55bb3cda0370 2024-09-19T06:12:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94b096d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bb3cca2cf0 con 0x55bb3cd955d0 2024-09-19T06:12:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94b096d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ccf72b0 con 0x55bb3cd96b40 2024-09-19T06:12:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94a57fa640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f94900a33b0 0x7f94900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cda2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cda2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49276/0 (socket says 172.21.15.5:49276) 2024-09-19T06:12:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94a77fe640 1 -- 172.21.15.5:0/3748910197 learned_addr learned my addr 172.21.15.5:0/3748910197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.816+0000 7f94acb31640 1 --2- 172.21.15.5:0/3748910197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd96b40 0x55bb3cd95090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f94acb31640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f94900a33b0 0x7f94900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f94a7fff640 1 --2- 172.21.15.5:0/3748910197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd955d0 0x55bb3cd95a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f94a57fa640 1 -- 172.21.15.5:0/419790220 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f949c0d0070 con 0x7f94a8102570 2024-09-19T06:12:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f94a77fe640 1 -- 172.21.15.5:0/3748910197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd96b40 msgr2=0x55bb3cd95090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f94a77fe640 1 --2- 172.21.15.5:0/3748910197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd96b40 0x55bb3cd95090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f94a77fe640 1 -- 172.21.15.5:0/3748910197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd955d0 msgr2=0x55bb3cd95a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f94a77fe640 1 --2- 172.21.15.5:0/3748910197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd955d0 0x55bb3cd95a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f94a77fe640 1 -- 172.21.15.5:0/3748910197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3cc873a0 con 0x55bb3cda0370 2024-09-19T06:12:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f94acb31640 1 --2- 172.21.15.5:0/3748910197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd96b40 0x55bb3cd95090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f94a7fff640 1 --2- 172.21.15.5:0/3748910197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd955d0 0x55bb3cd95a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f94a77fe640 1 --2- 172.21.15.5:0/3748910197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cda2760 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e9c0 tx=0x7f949c09c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f74838a17221de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f94acb31640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f94900a33b0 0x7f94900a5870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009fd0 tx=0x7f94a0009600 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f948e7fc640 1 -- 172.21.15.5:0/3748910197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c026040 con 0x55bb3cda0370 2024-09-19T06:12:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f948e7fc640 1 -- 172.21.15.5:0/3748910197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f949c0a0680 con 0x55bb3cda0370 2024-09-19T06:12:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.817+0000 7f948e7fc640 1 -- 172.21.15.5:0/3748910197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c09ca80 con 0x55bb3cda0370 2024-09-19T06:12:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.818+0000 7f94b096d740 1 -- 172.21.15.5:0/3748910197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 msgr2=0x55bb3cda2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.818+0000 7f94b096d740 1 --2- 172.21.15.5:0/3748910197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cda2760 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e9c0 tx=0x7f949c09c280 comp rx=0 tx=0).stop 2024-09-19T06:12:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.818+0000 7f94b096d740 1 -- 172.21.15.5:0/3748910197 shutdown_connections 2024-09-19T06:12:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.818+0000 7f94b096d740 1 --2- 172.21.15.5:0/3748910197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cda2760 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.818+0000 7f94b096d740 1 --2- 172.21.15.5:0/3748910197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd955d0 0x55bb3cd95a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.818+0000 7f94b096d740 1 --2- 172.21.15.5:0/3748910197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd96b40 0x55bb3cd95090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.818+0000 7f94b096d740 1 -- 172.21.15.5:0/3748910197 >> 172.21.15.5:0/3748910197 conn(0x55bb3cd5e310 msgr2=0x55bb3cd91c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.818+0000 7f94b096d740 1 -- 172.21.15.5:0/3748910197 shutdown_connections 2024-09-19T06:12:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.818+0000 7f94b096d740 1 -- 172.21.15.5:0/3748910197 wait complete. 2024-09-19T06:12:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.818+0000 7f94b096d740 1 -- start start 2024-09-19T06:12:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94b096d740 1 Processor -- start 2024-09-19T06:12:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94b096d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd955d0 0x55bb3cd9fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94b096d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd96b40 0x55bb3cd9d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94b096d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cd9d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94b096d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bb3cda3010 con 0x55bb3cda0370 2024-09-19T06:12:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94b096d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ccb6060 con 0x55bb3cd955d0 2024-09-19T06:12:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94b096d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bb3cda3190 con 0x55bb3cd96b40 2024-09-19T06:12:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94b096d740 10 client.0 _ll_register_callbacks cb 0x55bb3ca098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94acb31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cd9d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94acb31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cd9d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49280/0 (socket says 172.21.15.5:49280) 2024-09-19T06:12:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94acb31640 1 -- 172.21.15.5:0/514511121 learned_addr learned my addr 172.21.15.5:0/514511121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94a7fff640 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd955d0 0x55bb3cd9fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94a77fe640 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd96b40 0x55bb3cd9d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94acb31640 1 -- 172.21.15.5:0/514511121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd96b40 msgr2=0x55bb3cd9d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94acb31640 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd96b40 0x55bb3cd9d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94acb31640 1 -- 172.21.15.5:0/514511121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd955d0 msgr2=0x55bb3cd9fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94acb31640 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd955d0 0x55bb3cd9fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.819+0000 7f94acb31640 1 -- 172.21.15.5:0/514511121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3ca6c3c0 con 0x55bb3cda0370 2024-09-19T06:12:54.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94acb31640 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cd9d7e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f94a000abc0 tx=0x7f94a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2233556c2b0a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:54.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94877fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:54.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94877fe640 1 -- 172.21.15.5:0/514511121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0024070 con 0x55bb3cda0370 2024-09-19T06:12:54.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94877fe640 1 -- 172.21.15.5:0/514511121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f94a0010ce0 con 0x55bb3cda0370 2024-09-19T06:12:54.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94b096d740 1 -- 172.21.15.5:0/514511121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949c0a0680 con 0x55bb3cda0370 2024-09-19T06:12:54.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94877fe640 1 -- 172.21.15.5:0/514511121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f94a0004b20 con 0x55bb3cda0370 2024-09-19T06:12:54.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94b096d740 10 client.17904 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:54.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94b096d740 1 -- 172.21.15.5:0/514511121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bb3cdaebf0 con 0x55bb3cda0370 2024-09-19T06:12:54.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94877fe640 1 -- 172.21.15.5:0/514511121 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f949c0a0680 con 0x55bb3cda0370 2024-09-19T06:12:54.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94877fe640 1 -- 172.21.15.5:0/514511121 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f94a000f040 con 0x55bb3cda0370 2024-09-19T06:12:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94b096d740 10 client.17904 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94b096d740 10 client.17904 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:12:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94b096d740 20 client.17904 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94b096d740 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55bb3cdb0050 0x55bb3cdb24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94b096d740 4 client.17904 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.820+0000 7f94b096d740 1 -- 172.21.15.5:0/514511121 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb3cd08860 con 0x55bb3cdb0050 2024-09-19T06:12:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.821+0000 7f94a7fff640 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55bb3cdb0050 0x55bb3cdb24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.821+0000 7f94877fe640 10 client.17904 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:12:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.821+0000 7f94a7fff640 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55bb3cdb0050 0x55bb3cdb24c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc9dcfd6c6fbe7d0 server_cookie=744f96a6591e165 in_seq=0 out_seq=0 2024-09-19T06:12:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94877fe640 1 -- 172.21.15.5:0/514511121 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb3cd08860 con 0x55bb3cdb0050 2024-09-19T06:12:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94877fe640 10 client.17904 handle_command_reply: tid=0 2024-09-19T06:12:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94b096d740 1 client.17904 shutdown 2024-09-19T06:12:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94b096d740 1 -- 172.21.15.5:0/514511121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 msgr2=0x55bb3cd9d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94b096d740 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cd9d7e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f94a000abc0 tx=0x7f94a0004a40 comp rx=0 tx=0).stop 2024-09-19T06:12:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94b096d740 1 -- 172.21.15.5:0/514511121 shutdown_connections 2024-09-19T06:12:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94b096d740 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55bb3cdb0050 0x55bb3cdb24c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94b096d740 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cd9d7e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94b096d740 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd96b40 0x55bb3cd9d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94b096d740 1 --2- 172.21.15.5:0/514511121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd955d0 0x55bb3cd9fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94b096d740 1 -- 172.21.15.5:0/514511121 >> 172.21.15.5:0/514511121 conn(0x55bb3cd5e310 msgr2=0x55bb3cd97990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94877fe640 0 client.17904 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:12:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94b096d740 1 -- 172.21.15.5:0/514511121 shutdown_connections 2024-09-19T06:12:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94b096d740 1 -- 172.21.15.5:0/514511121 wait complete. 2024-09-19T06:12:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.822+0000 7f94b096d740 20 client.17904 trim_cache size 0 max 16384 2024-09-19T06:12:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.835+0000 7f94b096d740 1 Processor -- start 2024-09-19T06:12:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.835+0000 7f94b096d740 1 -- start start 2024-09-19T06:12:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.835+0000 7f94b096d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd955d0 0x55bb3cdac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.835+0000 7f94b096d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd96b40 0x55bb3cd98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.835+0000 7f94b096d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cd98660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.835+0000 7f94b096d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81ad810 con 0x55bb3cda0370 2024-09-19T06:12:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.835+0000 7f94b096d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81af380 con 0x55bb3cd96b40 2024-09-19T06:12:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.835+0000 7f94b096d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81adee0 con 0x55bb3cd955d0 2024-09-19T06:12:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.835+0000 7f94acb31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cd98660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.835+0000 7f94acb31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cd98660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49286/0 (socket says 172.21.15.5:49286) 2024-09-19T06:12:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.835+0000 7f94acb31640 1 -- 172.21.15.5:0/3437106438 learned_addr learned my addr 172.21.15.5:0/3437106438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.836+0000 7f94a7fff640 1 --2- 172.21.15.5:0/3437106438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd955d0 0x55bb3cdac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.836+0000 7f94a77fe640 1 --2- 172.21.15.5:0/3437106438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd96b40 0x55bb3cd98120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.836+0000 7f94acb31640 1 -- 172.21.15.5:0/3437106438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd955d0 msgr2=0x55bb3cdac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.836+0000 7f94acb31640 1 --2- 172.21.15.5:0/3437106438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd955d0 0x55bb3cdac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.836+0000 7f94acb31640 1 -- 172.21.15.5:0/3437106438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd96b40 msgr2=0x55bb3cd98120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.836+0000 7f94acb31640 1 --2- 172.21.15.5:0/3437106438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd96b40 0x55bb3cd98120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.836+0000 7f94acb31640 1 -- 172.21.15.5:0/3437106438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3cdac7f0 con 0x55bb3cda0370 2024-09-19T06:12:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.836+0000 7f94a77fe640 1 --2- 172.21.15.5:0/3437106438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd96b40 0x55bb3cd98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:12:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.836+0000 7f94acb31640 1 --2- 172.21.15.5:0/3437106438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cd98660 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f94a000a750 tx=0x7f94a0010850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfd827f3b106dc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.836+0000 7f948dffb640 1 -- 172.21.15.5:0/3437106438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0024070 con 0x55bb3cda0370 2024-09-19T06:12:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.836+0000 7f948dffb640 1 -- 172.21.15.5:0/3437106438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94a00120c0 con 0x55bb3cda0370 2024-09-19T06:12:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.836+0000 7f948dffb640 1 -- 172.21.15.5:0/3437106438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0028770 con 0x55bb3cda0370 2024-09-19T06:12:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.837+0000 7f94b096d740 1 -- 172.21.15.5:0/3437106438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 msgr2=0x55bb3cd98660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.837+0000 7f94b096d740 1 --2- 172.21.15.5:0/3437106438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cd98660 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f94a000a750 tx=0x7f94a0010850 comp rx=0 tx=0).stop 2024-09-19T06:12:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.837+0000 7f94b096d740 1 -- 172.21.15.5:0/3437106438 shutdown_connections 2024-09-19T06:12:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.837+0000 7f94b096d740 1 --2- 172.21.15.5:0/3437106438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cd98660 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.837+0000 7f94b096d740 1 --2- 172.21.15.5:0/3437106438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd96b40 0x55bb3cd98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.837+0000 7f94b096d740 1 --2- 172.21.15.5:0/3437106438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd955d0 0x55bb3cdac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.837+0000 7f94b096d740 1 -- 172.21.15.5:0/3437106438 >> 172.21.15.5:0/3437106438 conn(0x55bb3cd5e310 msgr2=0x55bb3cd9f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.837+0000 7f94b096d740 1 -- 172.21.15.5:0/3437106438 shutdown_connections 2024-09-19T06:12:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.837+0000 7f94b096d740 1 -- 172.21.15.5:0/3437106438 wait complete. 2024-09-19T06:12:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.837+0000 7f94b096d740 1 -- start start 2024-09-19T06:12:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.837+0000 7f94b096d740 1 Processor -- start 2024-09-19T06:12:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94b096d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd955d0 0x55bb3cda89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94b096d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd96b40 0x55bb3cda3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94b096d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cda42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94b096d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81aecb0 con 0x55bb3cda0370 2024-09-19T06:12:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94b096d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81ae5b0 con 0x55bb3cd96b40 2024-09-19T06:12:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94b096d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81ab810 con 0x55bb3cd955d0 2024-09-19T06:12:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94b096d740 10 client.0 _ll_register_callbacks cb 0x7f94a000a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94acb31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cda42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94acb31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cda42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49298/0 (socket says 172.21.15.5:49298) 2024-09-19T06:12:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94acb31640 1 -- 172.21.15.5:0/2559702710 learned_addr learned my addr 172.21.15.5:0/2559702710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94a77fe640 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd96b40 0x55bb3cda3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94a7fff640 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd955d0 0x55bb3cda89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94acb31640 1 -- 172.21.15.5:0/2559702710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd955d0 msgr2=0x55bb3cda89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94acb31640 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd955d0 0x55bb3cda89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94acb31640 1 -- 172.21.15.5:0/2559702710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd96b40 msgr2=0x55bb3cda3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94acb31640 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd96b40 0x55bb3cda3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94acb31640 1 -- 172.21.15.5:0/2559702710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3cda4950 con 0x55bb3cda0370 2024-09-19T06:12:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94a77fe640 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd96b40 0x55bb3cda3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94acb31640 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cda42b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009cb0 tx=0x7f94a00306d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81686ddfd8a0450 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94877fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94877fe640 1 -- 172.21.15.5:0/2559702710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a003c030 con 0x55bb3cda0370 2024-09-19T06:12:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94877fe640 1 -- 172.21.15.5:0/2559702710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f94a0028510 con 0x55bb3cda0370 2024-09-19T06:12:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94b096d740 1 -- 172.21.15.5:0/2559702710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a00120c0 con 0x55bb3cda0370 2024-09-19T06:12:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94877fe640 1 -- 172.21.15.5:0/2559702710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f94a0046050 con 0x55bb3cda0370 2024-09-19T06:12:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.838+0000 7f94b096d740 10 client.17910 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.839+0000 7f94b096d740 1 -- 172.21.15.5:0/2559702710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bb3ce1ab40 con 0x55bb3cda0370 2024-09-19T06:12:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.839+0000 7f94877fe640 1 -- 172.21.15.5:0/2559702710 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94a00120c0 con 0x55bb3cda0370 2024-09-19T06:12:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.839+0000 7f94877fe640 1 -- 172.21.15.5:0/2559702710 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f94a0010ce0 con 0x55bb3cda0370 2024-09-19T06:12:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.839+0000 7f94b096d740 10 client.17910 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.839+0000 7f94b096d740 10 client.17910 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:12:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.839+0000 7f94b096d740 20 client.17910 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.839+0000 7f94b096d740 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55bb3ce1b1f0 0x55bb3ce1d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.839+0000 7f94b096d740 4 client.17910 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:12:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.839+0000 7f94b096d740 1 -- 172.21.15.5:0/2559702710 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb3cd02d70 con 0x55bb3ce1b1f0 2024-09-19T06:12:54.918 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:12:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.839+0000 7f94a7fff640 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55bb3ce1b1f0 0x55bb3ce1d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94a7fff640 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55bb3ce1b1f0 0x55bb3ce1d5e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad7e58d5cdfdef03 server_cookie=f206792945e30ce4 in_seq=0 out_seq=0 2024-09-19T06:12:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94877fe640 10 client.17910 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:12:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94877fe640 1 -- 172.21.15.5:0/2559702710 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb3cd02d70 con 0x55bb3ce1b1f0 2024-09-19T06:12:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94877fe640 10 client.17910 handle_command_reply: tid=0 2024-09-19T06:12:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94b096d740 1 client.17910 shutdown 2024-09-19T06:12:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94b096d740 1 -- 172.21.15.5:0/2559702710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 msgr2=0x55bb3cda42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94b096d740 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cda42b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009cb0 tx=0x7f94a00306d0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94b096d740 1 -- 172.21.15.5:0/2559702710 shutdown_connections 2024-09-19T06:12:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94b096d740 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55bb3ce1b1f0 0x55bb3ce1d5e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94b096d740 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3cda0370 0x55bb3cda42b0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94b096d740 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bb3cd96b40 0x55bb3cda3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94b096d740 1 --2- 172.21.15.5:0/2559702710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb3cd955d0 0x55bb3cda89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94877fe640 0 client.17910 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:12:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94b096d740 1 -- 172.21.15.5:0/2559702710 >> 172.21.15.5:0/2559702710 conn(0x55bb3cd5e310 msgr2=0x55bb3cd9b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94b096d740 1 -- 172.21.15.5:0/2559702710 shutdown_connections 2024-09-19T06:12:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94b096d740 1 -- 172.21.15.5:0/2559702710 wait complete. 2024-09-19T06:12:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.840+0000 7f94b096d740 20 client.17910 trim_cache size 0 max 16384 2024-09-19T06:12:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.841+0000 7f94ae594640 1 -- 172.21.15.5:0/419790220 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f94900a33b0 msgr2=0x7f94900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.841+0000 7f94ae594640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f94900a33b0 0x7f94900a5870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009fd0 tx=0x7f94a0009600 comp rx=0 tx=0).stop 2024-09-19T06:12:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.841+0000 7f94ae594640 1 -- 172.21.15.5:0/419790220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94a8102570 msgr2=0x7f94a8198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.841+0000 7f94ae594640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94a8102570 0x7f94a8198990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e9f0 tx=0x7f949c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.841+0000 7f94ae594640 1 -- 172.21.15.5:0/419790220 shutdown_connections 2024-09-19T06:12:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.841+0000 7f94ae594640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f94900a33b0 0x7f94900a5870 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.841+0000 7f94ae594640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94a8102ef0 0x7f94a819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.841+0000 7f94ae594640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94a8102570 0x7f94a8198990 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.841+0000 7f94ae594640 1 --2- 172.21.15.5:0/419790220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a8100e20 0x7f94a8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.841+0000 7f94ae594640 1 -- 172.21.15.5:0/419790220 >> 172.21.15.5:0/419790220 conn(0x7f94a80fc7d0 msgr2=0x7f94a806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.841+0000 7f94ae594640 1 -- 172.21.15.5:0/419790220 shutdown_connections 2024-09-19T06:12:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:54.841+0000 7f94ae594640 1 -- 172.21.15.5:0/419790220 wait complete. 2024-09-19T06:12:55.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:12:55.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241377 2.54 MB/sec execute 381 sec latency 2502.563 ms 2024-09-19T06:12:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.323+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1036385457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d0102ef0 msgr2=0x7fb1d010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.323+0000 7fb1d6ac1640 1 --2- 172.21.15.5:0/1036385457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d0102ef0 0x7fb1d010da40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009ca0 tx=0x7fb1c002a950 comp rx=0 tx=0).stop 2024-09-19T06:12:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.323+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1036385457 shutdown_connections 2024-09-19T06:12:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.323+0000 7fb1d6ac1640 1 --2- 172.21.15.5:0/1036385457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d0102ef0 0x7fb1d010da40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.323+0000 7fb1d6ac1640 1 --2- 172.21.15.5:0/1036385457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d0102570 0x7fb1d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.323+0000 7fb1d6ac1640 1 --2- 172.21.15.5:0/1036385457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1d0100e20 0x7fb1d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.323+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1036385457 >> 172.21.15.5:0/1036385457 conn(0x7fb1d00fc7d0 msgr2=0x7fb1d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.323+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1036385457 shutdown_connections 2024-09-19T06:12:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.323+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1036385457 wait complete. 2024-09-19T06:12:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.323+0000 7fb1d6ac1640 1 Processor -- start 2024-09-19T06:12:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1d6ac1640 1 -- start start 2024-09-19T06:12:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1d6ac1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d0100e20 0x7fb1d00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1d6ac1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d0102570 0x7fb1d00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1d6ac1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1d0102ef0 0x7fb1d00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1d6ac1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0110e30 con 0x7fb1d0102570 2024-09-19T06:12:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1d6ac1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0110cb0 con 0x7fb1d0102ef0 2024-09-19T06:12:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1d6ac1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0110fb0 con 0x7fb1d0100e20 2024-09-19T06:12:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1cffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d0102570 0x7fb1d00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1cffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d0102570 0x7fb1d00fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49328/0 (socket says 172.21.15.5:49328) 2024-09-19T06:12:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1cffff640 1 -- 172.21.15.5:0/1285652373 learned_addr learned my addr 172.21.15.5:0/1285652373 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1d485d640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d0100e20 0x7fb1d00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1d505e640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1d0102ef0 0x7fb1d00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1cffff640 1 -- 172.21.15.5:0/1285652373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d0100e20 msgr2=0x7fb1d00fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1cffff640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d0100e20 0x7fb1d00fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1cffff640 1 -- 172.21.15.5:0/1285652373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1d0102ef0 msgr2=0x7fb1d00fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1cffff640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1d0102ef0 0x7fb1d00fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1cffff640 1 -- 172.21.15.5:0/1285652373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d01a68f0 con 0x7fb1d0102570 2024-09-19T06:12:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1d485d640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d0100e20 0x7fb1d00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.324+0000 7fb1cffff640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d0102570 0x7fb1d00fd1c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800c970 tx=0x7fb1b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.325+0000 7fb1cdffb640 1 -- 172.21.15.5:0/1285652373 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8007bb0 con 0x7fb1d0102570 2024-09-19T06:12:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.325+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1285652373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d01a6be0 con 0x7fb1d0102570 2024-09-19T06:12:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.325+0000 7fb1cdffb640 1 -- 172.21.15.5:0/1285652373 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1b8007d50 con 0x7fb1d0102570 2024-09-19T06:12:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.325+0000 7fb1cdffb640 1 -- 172.21.15.5:0/1285652373 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800f450 con 0x7fb1d0102570 2024-09-19T06:12:56.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.325+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1285652373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d01a70d0 con 0x7fb1d0102570 2024-09-19T06:12:56.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.327+0000 7fb1cdffb640 1 -- 172.21.15.5:0/1285652373 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb1b8004f20 con 0x7fb1d0102570 2024-09-19T06:12:56.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d8e9a740 1 Processor -- start 2024-09-19T06:12:56.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d8e9a740 1 -- start start 2024-09-19T06:12:56.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d8e9a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf738b40 0x556cdf737090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d8e9a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf737a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d8e9a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf742370 0x556cdf744760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d8e9a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cdf69d2f0 con 0x556cdf7375d0 2024-09-19T06:12:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d8e9a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556cdf644cf0 con 0x556cdf738b40 2024-09-19T06:12:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d8e9a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556cdf6992b0 con 0x556cdf742370 2024-09-19T06:12:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1cdffb640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb1bc0a33b0 0x7fb1bc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d485d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf737a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d485d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf737a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49330/0 (socket says 172.21.15.5:49330) 2024-09-19T06:12:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d485d640 1 -- 172.21.15.5:0/1638376142 learned_addr learned my addr 172.21.15.5:0/1638376142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d505e640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb1bc0a33b0 0x7fb1bc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d505e640 1 --2- 172.21.15.5:0/1638376142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf738b40 0x556cdf737090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1cffff640 1 --2- 172.21.15.5:0/1638376142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf742370 0x556cdf744760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1cdffb640 1 -- 172.21.15.5:0/1285652373 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7fb1b80cfae0 con 0x7fb1d0102570 2024-09-19T06:12:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d485d640 1 -- 172.21.15.5:0/1638376142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf742370 msgr2=0x556cdf744760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d485d640 1 --2- 172.21.15.5:0/1638376142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf742370 0x556cdf744760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d485d640 1 -- 172.21.15.5:0/1638376142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf738b40 msgr2=0x556cdf737090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d485d640 1 --2- 172.21.15.5:0/1638376142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf738b40 0x556cdf737090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d485d640 1 -- 172.21.15.5:0/1638376142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdf6293a0 con 0x556cdf7375d0 2024-09-19T06:12:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.328+0000 7fb1d505e640 1 --2- 172.21.15.5:0/1638376142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf738b40 0x556cdf737090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.329+0000 7fb1d505e640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb1bc0a33b0 0x7fb1bc0a5870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0005fa0 tx=0x7fb1c0005ef0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.329+0000 7fb1d485d640 1 --2- 172.21.15.5:0/1638376142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf737a10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800cca0 tx=0x7fb1c800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e37b8b59fdea42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.329+0000 7fb1c6ffd640 1 -- 172.21.15.5:0/1638376142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8005220 con 0x556cdf7375d0 2024-09-19T06:12:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.329+0000 7fb1c6ffd640 1 -- 172.21.15.5:0/1638376142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1c8004c50 con 0x556cdf7375d0 2024-09-19T06:12:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.329+0000 7fb1c6ffd640 1 -- 172.21.15.5:0/1638376142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80107b0 con 0x556cdf7375d0 2024-09-19T06:12:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/1638376142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 msgr2=0x556cdf737a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/1638376142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf737a10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800cca0 tx=0x7fb1c800e720 comp rx=0 tx=0).stop 2024-09-19T06:12:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/1638376142 shutdown_connections 2024-09-19T06:12:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/1638376142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf742370 0x556cdf744760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/1638376142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf737a10 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/1638376142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf738b40 0x556cdf737090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/1638376142 >> 172.21.15.5:0/1638376142 conn(0x556cdf700310 msgr2=0x556cdf733c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/1638376142 shutdown_connections 2024-09-19T06:12:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/1638376142 wait complete. 2024-09-19T06:12:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 -- start start 2024-09-19T06:12:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 Processor -- start 2024-09-19T06:12:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf7375d0 0x556cdf741d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf738b40 0x556cdf73efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf742370 0x556cdf73f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cdf745070 con 0x556cdf742370 2024-09-19T06:12:56.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556cdf658060 con 0x556cdf7375d0 2024-09-19T06:12:56.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.330+0000 7fb1d8e9a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556cdf7451f0 con 0x556cdf738b40 2024-09-19T06:12:56.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1d8e9a740 10 client.0 _ll_register_callbacks cb 0x556cdf3ab8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:56.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1d505e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf742370 0x556cdf73f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1d505e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf742370 0x556cdf73f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49336/0 (socket says 172.21.15.5:49336) 2024-09-19T06:12:56.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1d505e640 1 -- 172.21.15.5:0/159109216 learned_addr learned my addr 172.21.15.5:0/159109216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:56.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1d485d640 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf7375d0 0x556cdf741d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1cffff640 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf738b40 0x556cdf73efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1d505e640 1 -- 172.21.15.5:0/159109216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf738b40 msgr2=0x556cdf73efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:56.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1d505e640 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf738b40 0x556cdf73efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1d505e640 1 -- 172.21.15.5:0/159109216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf7375d0 msgr2=0x556cdf741d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:56.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1d505e640 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf7375d0 0x556cdf741d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1d505e640 1 -- 172.21.15.5:0/159109216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cdf40e3c0 con 0x556cdf742370 2024-09-19T06:12:56.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1cffff640 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf738b40 0x556cdf73efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:56.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1d505e640 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf742370 0x556cdf73f560 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009a20 tx=0x7fb1c002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ccfa074932d30d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:56.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1affff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:56.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1affff640 1 -- 172.21.15.5:0/159109216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c004e030 con 0x556cdf742370 2024-09-19T06:12:56.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1affff640 1 -- 172.21.15.5:0/159109216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7fb1c0044070 con 0x556cdf742370 2024-09-19T06:12:56.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.331+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/159109216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c8004c50 con 0x556cdf742370 2024-09-19T06:12:56.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.332+0000 7fb1affff640 1 -- 172.21.15.5:0/159109216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb1c0034040 con 0x556cdf742370 2024-09-19T06:12:56.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.332+0000 7fb1d8e9a740 10 client.17922 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.332+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/159109216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556cdf750ac0 con 0x556cdf742370 2024-09-19T06:12:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.332+0000 7fb1affff640 1 -- 172.21.15.5:0/159109216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1c8004c50 con 0x556cdf742370 2024-09-19T06:12:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.332+0000 7fb1affff640 1 -- 172.21.15.5:0/159109216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fb1c0047050 con 0x556cdf742370 2024-09-19T06:12:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.332+0000 7fb1d8e9a740 10 client.17922 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.332+0000 7fb1d8e9a740 10 client.17922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.332+0000 7fb1d8e9a740 20 client.17922 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.332+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556cdf751f20 0x556cdf754390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.332+0000 7fb1d8e9a740 4 client.17922 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.332+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/159109216 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cdf6aa860 con 0x556cdf751f20 2024-09-19T06:12:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.332+0000 7fb1d485d640 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556cdf751f20 0x556cdf754390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.333+0000 7fb1d485d640 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556cdf751f20 0x556cdf754390 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d71a5b10ade812 server_cookie=521626030fdbe0aa in_seq=0 out_seq=0 2024-09-19T06:12:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.333+0000 7fb1affff640 10 client.17922 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.333+0000 7fb1affff640 1 -- 172.21.15.5:0/159109216 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cdf6aa860 con 0x556cdf751f20 2024-09-19T06:12:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.333+0000 7fb1affff640 10 client.17922 handle_command_reply: tid=0 2024-09-19T06:12:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.333+0000 7fb1d8e9a740 1 client.17922 shutdown 2024-09-19T06:12:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.334+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/159109216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf742370 msgr2=0x556cdf73f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.334+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf742370 0x556cdf73f560 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009a20 tx=0x7fb1c002a950 comp rx=0 tx=0).stop 2024-09-19T06:12:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.334+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/159109216 shutdown_connections 2024-09-19T06:12:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.334+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556cdf751f20 0x556cdf754390 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.334+0000 7fb1affff640 0 client.17922 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.334+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf742370 0x556cdf73f560 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.334+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf738b40 0x556cdf73efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.334+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/159109216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf7375d0 0x556cdf741d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.334+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/159109216 >> 172.21.15.5:0/159109216 conn(0x556cdf700310 msgr2=0x556cdf739930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.334+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/159109216 shutdown_connections 2024-09-19T06:12:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.334+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/159109216 wait complete. 2024-09-19T06:12:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.334+0000 7fb1d8e9a740 20 client.17922 trim_cache size 0 max 16384 2024-09-19T06:12:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1d8e9a740 1 Processor -- start 2024-09-19T06:12:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1d8e9a740 1 -- start start 2024-09-19T06:12:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1d8e9a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf733e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1d8e9a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf738b40 0x556cdf74e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1d8e9a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf742370 0x556cdf73a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1d8e9a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01afac0 con 0x556cdf7375d0 2024-09-19T06:12:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1d8e9a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01b1630 con 0x556cdf738b40 2024-09-19T06:12:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1d8e9a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01b0190 con 0x556cdf742370 2024-09-19T06:12:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1d485d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf733e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1d485d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf733e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49342/0 (socket says 172.21.15.5:49342) 2024-09-19T06:12:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1d485d640 1 -- 172.21.15.5:0/1903201378 learned_addr learned my addr 172.21.15.5:0/1903201378 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1cffff640 1 --2- 172.21.15.5:0/1903201378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf738b40 0x556cdf74e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.347+0000 7fb1d505e640 1 --2- 172.21.15.5:0/1903201378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf742370 0x556cdf73a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.348+0000 7fb1d485d640 1 -- 172.21.15.5:0/1903201378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf742370 msgr2=0x556cdf73a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.348+0000 7fb1d485d640 1 --2- 172.21.15.5:0/1903201378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf742370 0x556cdf73a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.348+0000 7fb1d485d640 1 -- 172.21.15.5:0/1903201378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf738b40 msgr2=0x556cdf74e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.348+0000 7fb1d485d640 1 --2- 172.21.15.5:0/1903201378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf738b40 0x556cdf74e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.348+0000 7fb1d485d640 1 -- 172.21.15.5:0/1903201378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdf738760 con 0x556cdf7375d0 2024-09-19T06:12:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.348+0000 7fb1cffff640 1 --2- 172.21.15.5:0/1903201378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf738b40 0x556cdf74e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.348+0000 7fb1d485d640 1 --2- 172.21.15.5:0/1903201378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf733e10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800cca0 tx=0x7fb1c800c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7200e088e728bfc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.348+0000 7fb1c67fc640 1 -- 172.21.15.5:0/1903201378 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c802b020 con 0x556cdf7375d0 2024-09-19T06:12:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.348+0000 7fb1c67fc640 1 -- 172.21.15.5:0/1903201378 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1c8010e20 con 0x556cdf7375d0 2024-09-19T06:12:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.348+0000 7fb1c67fc640 1 -- 172.21.15.5:0/1903201378 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8026ab0 con 0x556cdf7375d0 2024-09-19T06:12:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/1903201378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 msgr2=0x556cdf733e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/1903201378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf733e10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800cca0 tx=0x7fb1c800c590 comp rx=0 tx=0).stop 2024-09-19T06:12:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/1903201378 shutdown_connections 2024-09-19T06:12:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/1903201378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf742370 0x556cdf73a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/1903201378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf738b40 0x556cdf74e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/1903201378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf733e10 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/1903201378 >> 172.21.15.5:0/1903201378 conn(0x556cdf700310 msgr2=0x556cdf733700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/1903201378 shutdown_connections 2024-09-19T06:12:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/1903201378 wait complete. 2024-09-19T06:12:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 -- start start 2024-09-19T06:12:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 Processor -- start 2024-09-19T06:12:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf73c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf738b40 0x556cdf73c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf742370 0x556cdf73cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.349+0000 7fb1d8e9a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01b0f60 con 0x556cdf7375d0 2024-09-19T06:12:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d8e9a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01b0860 con 0x556cdf742370 2024-09-19T06:12:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d8e9a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01adac0 con 0x556cdf738b40 2024-09-19T06:12:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d8e9a740 10 client.0 _ll_register_callbacks cb 0x7fb1c0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d485d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf73c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d485d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf73c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49356/0 (socket says 172.21.15.5:49356) 2024-09-19T06:12:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1cffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf738b40 0x556cdf73c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d485d640 1 -- 172.21.15.5:0/341246444 learned_addr learned my addr 172.21.15.5:0/341246444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d505e640 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf742370 0x556cdf73cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d485d640 1 -- 172.21.15.5:0/341246444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf738b40 msgr2=0x556cdf73c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d485d640 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf738b40 0x556cdf73c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d485d640 1 -- 172.21.15.5:0/341246444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf742370 msgr2=0x556cdf73cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d485d640 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf742370 0x556cdf73cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d485d640 1 -- 172.21.15.5:0/341246444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cdf7bbd90 con 0x556cdf7375d0 2024-09-19T06:12:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d505e640 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf742370 0x556cdf73cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d485d640 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf73c110 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8005d60 tx=0x7fb1c800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2183ff7c3c98f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1affff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1affff640 1 -- 172.21.15.5:0/341246444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c802b030 con 0x556cdf7375d0 2024-09-19T06:12:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1affff640 1 -- 172.21.15.5:0/341246444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7fb1c8010b80 con 0x556cdf7375d0 2024-09-19T06:12:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/341246444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c8010e20 con 0x556cdf7375d0 2024-09-19T06:12:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.350+0000 7fb1affff640 1 -- 172.21.15.5:0/341246444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb1c8030450 con 0x556cdf7375d0 2024-09-19T06:12:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.351+0000 7fb1d8e9a740 10 client.17928 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.351+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/341246444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556cdf7bd010 con 0x556cdf7375d0 2024-09-19T06:12:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.351+0000 7fb1affff640 1 -- 172.21.15.5:0/341246444 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1c8010e20 con 0x556cdf7375d0 2024-09-19T06:12:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.351+0000 7fb1affff640 1 -- 172.21.15.5:0/341246444 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fb1c80265d0 con 0x556cdf7375d0 2024-09-19T06:12:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.351+0000 7fb1d8e9a740 10 client.17928 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.351+0000 7fb1d8e9a740 10 client.17928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.351+0000 7fb1d8e9a740 20 client.17928 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.351+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556cdf7bd4e0 0x556cdf7bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.351+0000 7fb1d8e9a740 4 client.17928 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:12:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.351+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/341246444 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cdf6a4d70 con 0x556cdf7bd4e0 2024-09-19T06:12:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.351+0000 7fb1cffff640 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556cdf7bd4e0 0x556cdf7bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1affff640 10 client.17928 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1cffff640 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556cdf7bd4e0 0x556cdf7bf8d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb298935db3c1f60 server_cookie=c1eafd3e4d874552 in_seq=0 out_seq=0 2024-09-19T06:12:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1affff640 1 -- 172.21.15.5:0/341246444 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cdf6a4d70 con 0x556cdf7bd4e0 2024-09-19T06:12:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1affff640 10 client.17928 handle_command_reply: tid=0 2024-09-19T06:12:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1d8e9a740 1 client.17928 shutdown 2024-09-19T06:12:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/341246444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 msgr2=0x556cdf73c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf73c110 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8005d60 tx=0x7fb1c800c750 comp rx=0 tx=0).stop 2024-09-19T06:12:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/341246444 shutdown_connections 2024-09-19T06:12:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x556cdf7bd4e0 0x556cdf7bf8d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556cdf742370 0x556cdf73cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556cdf738b40 0x556cdf73c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1d8e9a740 1 --2- 172.21.15.5:0/341246444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cdf7375d0 0x556cdf73c110 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/341246444 >> 172.21.15.5:0/341246444 conn(0x556cdf700310 msgr2=0x556cdf7324e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1affff640 0 client.17928 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/341246444 shutdown_connections 2024-09-19T06:12:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1d8e9a740 1 -- 172.21.15.5:0/341246444 wait complete. 2024-09-19T06:12:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.352+0000 7fb1d8e9a740 20 client.17928 trim_cache size 0 max 16384 2024-09-19T06:12:56.374 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:12:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.353+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1285652373 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb1bc0a33b0 msgr2=0x7fb1bc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.353+0000 7fb1d6ac1640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb1bc0a33b0 0x7fb1bc0a5870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0005fa0 tx=0x7fb1c0005ef0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.353+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1285652373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d0102570 msgr2=0x7fb1d00fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.353+0000 7fb1d6ac1640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d0102570 0x7fb1d00fd1c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800c970 tx=0x7fb1b800ce30 comp rx=0 tx=0).stop 2024-09-19T06:12:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.353+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1285652373 shutdown_connections 2024-09-19T06:12:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.353+0000 7fb1d6ac1640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1d0102ef0 0x7fb1d00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.353+0000 7fb1d6ac1640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d0102570 0x7fb1d00fd1c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.353+0000 7fb1d6ac1640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb1bc0a33b0 0x7fb1bc0a5870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.353+0000 7fb1d6ac1640 1 --2- 172.21.15.5:0/1285652373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d0100e20 0x7fb1d00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.353+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1285652373 >> 172.21.15.5:0/1285652373 conn(0x7fb1d00fc7d0 msgr2=0x7fb1d006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.353+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1285652373 shutdown_connections 2024-09-19T06:12:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:56.353+0000 7fb1d6ac1640 1 -- 172.21.15.5:0/1285652373 wait complete. 2024-09-19T06:12:56.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:56 smithi005 ceph-mon[30955]: pgmap v297: 97 pgs: 97 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 846 KiB/s wr, 83 op/s 2024-09-19T06:12:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:56 smithi157 ceph-mon[35346]: pgmap v297: 97 pgs: 97 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 846 KiB/s wr, 83 op/s 2024-09-19T06:12:56.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241377 2.53 MB/sec execute 382 sec latency 1099.830 ms 2024-09-19T06:12:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:56 smithi049 ceph-mon[34640]: pgmap v297: 97 pgs: 97 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 846 KiB/s wr, 83 op/s 2024-09-19T06:12:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.561+0000 7f157e918640 1 -- 172.21.15.5:0/3753809628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157810c0a0 msgr2=0x7f157810e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.561+0000 7f157e918640 1 --2- 172.21.15.5:0/3753809628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157810c0a0 0x7f157810e540 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1570009ca0 tx=0x7f157002a950 comp rx=0 tx=0).stop 2024-09-19T06:12:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.561+0000 7f157e918640 1 -- 172.21.15.5:0/3753809628 shutdown_connections 2024-09-19T06:12:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.561+0000 7f157e918640 1 --2- 172.21.15.5:0/3753809628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157810c0a0 0x7f157810e540 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.561+0000 7f157e918640 1 --2- 172.21.15.5:0/3753809628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15781035a0 0x7f157810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.561+0000 7f157e918640 1 --2- 172.21.15.5:0/3753809628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1578102c80 0x7f1578103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.561+0000 7f157e918640 1 -- 172.21.15.5:0/3753809628 >> 172.21.15.5:0/3753809628 conn(0x7f15780779a0 msgr2=0x7f15780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.561+0000 7f157e918640 1 -- 172.21.15.5:0/3753809628 shutdown_connections 2024-09-19T06:12:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.561+0000 7f157e918640 1 -- 172.21.15.5:0/3753809628 wait complete. 2024-09-19T06:12:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.561+0000 7f157e918640 1 Processor -- start 2024-09-19T06:12:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.561+0000 7f157e918640 1 -- start start 2024-09-19T06:12:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f157e918640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1578102c80 0x7f1578198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f157e918640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15781035a0 0x7f1578198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f157e918640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f157810c0a0 0x7f157819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f157e918640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1578111d10 con 0x7f15781035a0 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f157e918640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1578111b90 con 0x7f157810c0a0 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f157e918640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1578111e90 con 0x7f1578102c80 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f15777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15781035a0 0x7f1578198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f15777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15781035a0 0x7f1578198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49372/0 (socket says 172.21.15.5:49372) 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f15777fe640 1 -- 172.21.15.5:0/4010865097 learned_addr learned my addr 172.21.15.5:0/4010865097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f157ceb5640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f157810c0a0 0x7f157819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f1577fff640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1578102c80 0x7f1578198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f15777fe640 1 -- 172.21.15.5:0/4010865097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1578102c80 msgr2=0x7f1578198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f15777fe640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1578102c80 0x7f1578198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f15777fe640 1 -- 172.21.15.5:0/4010865097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f157810c0a0 msgr2=0x7f157819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f15777fe640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f157810c0a0 0x7f157819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f15777fe640 1 -- 172.21.15.5:0/4010865097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157819d1f0 con 0x7f15781035a0 2024-09-19T06:12:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f1577fff640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1578102c80 0x7f1578198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:12:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.562+0000 7f15777fe640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15781035a0 0x7f1578198740 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c890 tx=0x7f156c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.563+0000 7f15757fa640 1 -- 172.21.15.5:0/4010865097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c004220 con 0x7f15781035a0 2024-09-19T06:12:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.563+0000 7f157e918640 1 -- 172.21.15.5:0/4010865097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157819d4e0 con 0x7f15781035a0 2024-09-19T06:12:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.563+0000 7f15757fa640 1 -- 172.21.15.5:0/4010865097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f156c0043c0 con 0x7f15781035a0 2024-09-19T06:12:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.563+0000 7f15757fa640 1 -- 172.21.15.5:0/4010865097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c004a00 con 0x7f15781035a0 2024-09-19T06:12:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.563+0000 7f157e918640 1 -- 172.21.15.5:0/4010865097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15781a4d70 con 0x7f15781035a0 2024-09-19T06:12:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.565+0000 7f15757fa640 1 -- 172.21.15.5:0/4010865097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f156c004ba0 con 0x7f15781035a0 2024-09-19T06:12:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.565+0000 7f15816b2740 1 Processor -- start 2024-09-19T06:12:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.565+0000 7f15816b2740 1 -- start start 2024-09-19T06:12:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f15816b2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 0x55de24880ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f15816b2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24881410 0x55de2488b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f15816b2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 0x55de2488e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f15816b2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55de247e52f0 con 0x55de24880af0 2024-09-19T06:12:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f15816b2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55de2478d270 con 0x55de24881410 2024-09-19T06:12:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f15816b2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55de247e12b0 con 0x55de2488bcb0 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f157ceb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 0x55de24880ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f157ceb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 0x55de24880ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49378/0 (socket says 172.21.15.5:49378) 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f157ceb5640 1 -- 172.21.15.5:0/3984867001 learned_addr learned my addr 172.21.15.5:0/3984867001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f15757fa640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15600a33b0 0x7f15600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f1577fff640 1 --2- 172.21.15.5:0/3984867001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24881410 0x55de2488b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f157ceb5640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15600a33b0 0x7f15600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f15777fe640 1 --2- 172.21.15.5:0/3984867001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 0x55de2488e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f15757fa640 1 -- 172.21.15.5:0/4010865097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f156c0cf9d0 con 0x7f15781035a0 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f157ceb5640 1 -- 172.21.15.5:0/3984867001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 msgr2=0x55de2488e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f157ceb5640 1 --2- 172.21.15.5:0/3984867001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 0x55de2488e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f157ceb5640 1 -- 172.21.15.5:0/3984867001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24881410 msgr2=0x55de2488b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f157ceb5640 1 --2- 172.21.15.5:0/3984867001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24881410 0x55de2488b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f157ceb5640 1 -- 172.21.15.5:0/3984867001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de2478c340 con 0x55de24880af0 2024-09-19T06:12:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f157ceb5640 1 --2- 172.21.15.5:0/3984867001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 0x55de24880ed0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1570035000 tx=0x7f157002f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeedada930957511 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.566+0000 7f155e7fc640 1 -- 172.21.15.5:0/3984867001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570043050 con 0x55de24880af0 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.567+0000 7f155e7fc640 1 -- 172.21.15.5:0/3984867001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1570044040 con 0x55de24880af0 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.567+0000 7f155e7fc640 1 -- 172.21.15.5:0/3984867001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157004d020 con 0x55de24880af0 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.567+0000 7f157ceb5640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15600a33b0 0x7f15600a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f157004e000 tx=0x7f157004c000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.567+0000 7f15816b2740 1 -- 172.21.15.5:0/3984867001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 msgr2=0x55de24880ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.567+0000 7f15816b2740 1 --2- 172.21.15.5:0/3984867001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 0x55de24880ed0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1570035000 tx=0x7f157002f420 comp rx=0 tx=0).stop 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.567+0000 7f15816b2740 1 -- 172.21.15.5:0/3984867001 shutdown_connections 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.567+0000 7f15816b2740 1 --2- 172.21.15.5:0/3984867001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 0x55de2488e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.567+0000 7f15816b2740 1 --2- 172.21.15.5:0/3984867001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24881410 0x55de2488b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.567+0000 7f15816b2740 1 --2- 172.21.15.5:0/3984867001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 0x55de24880ed0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.567+0000 7f15816b2740 1 -- 172.21.15.5:0/3984867001 >> 172.21.15.5:0/3984867001 conn(0x55de2487d060 msgr2=0x55de2487d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.567+0000 7f15816b2740 1 -- 172.21.15.5:0/3984867001 shutdown_connections 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.567+0000 7f15816b2740 1 -- 172.21.15.5:0/3984867001 wait complete. 2024-09-19T06:12:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f15816b2740 1 -- start start 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f15816b2740 1 Processor -- start 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f15816b2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24880af0 0x55de24888bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f15816b2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 0x55de24889100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f15816b2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 0x55de24889640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f15816b2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55de2488e880 con 0x55de24881410 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f15816b2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55de2476e290 con 0x55de24880af0 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f15816b2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55de2488ea00 con 0x55de2488bcb0 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f15816b2740 10 client.0 _ll_register_callbacks cb 0x55de2479f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f15777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 0x55de24889100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f15777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 0x55de24889100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49384/0 (socket says 172.21.15.5:49384) 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f15777fe640 1 -- 172.21.15.5:0/274227067 learned_addr learned my addr 172.21.15.5:0/274227067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f157ceb5640 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 0x55de24889640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.568+0000 7f1577fff640 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24880af0 0x55de24888bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15777fe640 1 -- 172.21.15.5:0/274227067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 msgr2=0x55de24889640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15777fe640 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 0x55de24889640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15777fe640 1 -- 172.21.15.5:0/274227067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24880af0 msgr2=0x55de24888bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15777fe640 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24880af0 0x55de24888bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15777fe640 1 -- 172.21.15.5:0/274227067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de244f8210 con 0x55de24881410 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15777fe640 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 0x55de24889100 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f156c09bc20 tx=0x7f156c008ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de131cc279414c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15577fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15577fe640 1 -- 172.21.15.5:0/274227067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c0a0040 con 0x55de24881410 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15577fe640 1 -- 172.21.15.5:0/274227067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f156c098300 con 0x55de24881410 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15816b2740 1 -- 172.21.15.5:0/274227067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1570044040 con 0x55de24881410 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15577fe640 1 -- 172.21.15.5:0/274227067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f156c0a4900 con 0x55de24881410 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15816b2740 10 client.17940 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15816b2740 1 -- 172.21.15.5:0/274227067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55de245c4620 con 0x55de24881410 2024-09-19T06:12:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15577fe640 1 -- 172.21.15.5:0/274227067 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1570044040 con 0x55de24881410 2024-09-19T06:12:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15577fe640 1 -- 172.21.15.5:0/274227067 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f156c096440 con 0x55de24881410 2024-09-19T06:12:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15816b2740 10 client.17940 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15816b2740 10 client.17940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15816b2740 20 client.17940 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15816b2740 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de2489bc20 0x55de2489e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15816b2740 4 client.17940 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.569+0000 7f15816b2740 1 -- 172.21.15.5:0/274227067 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de247f2a20 con 0x55de2489bc20 2024-09-19T06:12:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.570+0000 7f1577fff640 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de2489bc20 0x55de2489e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.570+0000 7f1577fff640 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de2489bc20 0x55de2489e010 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37947ead8dc022f6 server_cookie=3fc5338f58641ab1 in_seq=0 out_seq=0 2024-09-19T06:12:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.570+0000 7f15577fe640 10 client.17940 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15577fe640 1 -- 172.21.15.5:0/274227067 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de247f2a20 con 0x55de2489bc20 2024-09-19T06:12:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15577fe640 10 client.17940 handle_command_reply: tid=0 2024-09-19T06:12:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15816b2740 1 client.17940 shutdown 2024-09-19T06:12:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15816b2740 1 -- 172.21.15.5:0/274227067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 msgr2=0x55de24889100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15816b2740 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 0x55de24889100 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f156c09bc20 tx=0x7f156c008ed0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15816b2740 1 -- 172.21.15.5:0/274227067 shutdown_connections 2024-09-19T06:12:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15816b2740 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de2489bc20 0x55de2489e010 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15816b2740 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 0x55de24889640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15816b2740 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 0x55de24889100 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15577fe640 0 client.17940 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15816b2740 1 --2- 172.21.15.5:0/274227067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24880af0 0x55de24888bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15816b2740 1 -- 172.21.15.5:0/274227067 >> 172.21.15.5:0/274227067 conn(0x55de2487d060 msgr2=0x55de24882dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15816b2740 1 -- 172.21.15.5:0/274227067 shutdown_connections 2024-09-19T06:12:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.571+0000 7f15816b2740 1 -- 172.21.15.5:0/274227067 wait complete. 2024-09-19T06:12:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.572+0000 7f15816b2740 20 client.17940 trim_cache size 0 max 16384 2024-09-19T06:12:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f15816b2740 1 Processor -- start 2024-09-19T06:12:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f15816b2740 1 -- start start 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f15816b2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 0x55de248835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f15816b2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de24881410 0x55de24889470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f15816b2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de2488bcb0 0x55de248a3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f15816b2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15781ad770 con 0x55de24880af0 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f15816b2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15781af2e0 con 0x55de2488bcb0 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f15816b2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15781ade40 con 0x55de24881410 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f1577fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 0x55de248835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f1577fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 0x55de248835b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49398/0 (socket says 172.21.15.5:49398) 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f1577fff640 1 -- 172.21.15.5:0/1153429267 learned_addr learned my addr 172.21.15.5:0/1153429267 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f157ceb5640 1 --2- 172.21.15.5:0/1153429267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de2488bcb0 0x55de248a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f1577fff640 1 -- 172.21.15.5:0/1153429267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de24881410 msgr2=0x55de24889470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f1577fff640 1 --2- 172.21.15.5:0/1153429267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de24881410 0x55de24889470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f1577fff640 1 -- 172.21.15.5:0/1153429267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de2488bcb0 msgr2=0x55de248a3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f1577fff640 1 --2- 172.21.15.5:0/1153429267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de2488bcb0 0x55de248a3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f1577fff640 1 -- 172.21.15.5:0/1153429267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de2489e670 con 0x55de24880af0 2024-09-19T06:12:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f157ceb5640 1 --2- 172.21.15.5:0/1153429267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de2488bcb0 0x55de248a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:12:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.586+0000 7f1577fff640 1 --2- 172.21.15.5:0/1153429267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 0x55de248835b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f156800dbc0 tx=0x7f1568005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2606fc44930007 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f155dffb640 1 -- 172.21.15.5:0/1153429267 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156801d070 con 0x55de24880af0 2024-09-19T06:12:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f155dffb640 1 -- 172.21.15.5:0/1153429267 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1568019970 con 0x55de24880af0 2024-09-19T06:12:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f155dffb640 1 -- 172.21.15.5:0/1153429267 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156800fbb0 con 0x55de24880af0 2024-09-19T06:12:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f15816b2740 1 -- 172.21.15.5:0/1153429267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 msgr2=0x55de248835b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f15816b2740 1 --2- 172.21.15.5:0/1153429267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 0x55de248835b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f156800dbc0 tx=0x7f1568005ef0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f15816b2740 1 -- 172.21.15.5:0/1153429267 shutdown_connections 2024-09-19T06:12:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f15816b2740 1 --2- 172.21.15.5:0/1153429267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de2488bcb0 0x55de248a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f15816b2740 1 --2- 172.21.15.5:0/1153429267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de24881410 0x55de24889470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f15816b2740 1 --2- 172.21.15.5:0/1153429267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24880af0 0x55de248835b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f15816b2740 1 -- 172.21.15.5:0/1153429267 >> 172.21.15.5:0/1153429267 conn(0x55de2487d060 msgr2=0x55de2488ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f15816b2740 1 -- 172.21.15.5:0/1153429267 shutdown_connections 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f15816b2740 1 -- 172.21.15.5:0/1153429267 wait complete. 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.587+0000 7f15816b2740 1 -- start start 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15816b2740 1 Processor -- start 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15816b2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24880af0 0x55de24894190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15816b2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 0x55de2488f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15816b2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 0x55de2488f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15816b2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15781aec10 con 0x55de24881410 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15816b2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15781ae510 con 0x55de24880af0 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15816b2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15781ab770 con 0x55de2488bcb0 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15816b2740 10 client.0 _ll_register_callbacks cb 0x7f156c0058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 0x55de2488f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 0x55de2488f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49406/0 (socket says 172.21.15.5:49406) 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15777fe640 1 -- 172.21.15.5:0/2930591028 learned_addr learned my addr 172.21.15.5:0/2930591028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f157ceb5640 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 0x55de2488f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f1577fff640 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24880af0 0x55de24894190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15777fe640 1 -- 172.21.15.5:0/2930591028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 msgr2=0x55de2488f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15777fe640 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 0x55de2488f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15777fe640 1 -- 172.21.15.5:0/2930591028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24880af0 msgr2=0x55de24894190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15777fe640 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24880af0 0x55de24894190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.588+0000 7f15777fe640 1 -- 172.21.15.5:0/2930591028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de2488ffb0 con 0x55de24881410 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15777fe640 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 0x55de2488f4b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f156c09b990 tx=0x7f156c09bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bd719b2fe83009 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15577fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15577fe640 1 -- 172.21.15.5:0/2930591028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c0b9060 con 0x55de24881410 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15577fe640 1 -- 172.21.15.5:0/2930591028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f156c097b50 con 0x55de24881410 2024-09-19T06:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15816b2740 1 -- 172.21.15.5:0/2930591028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1568019970 con 0x55de24881410 2024-09-19T06:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15577fe640 1 -- 172.21.15.5:0/2930591028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f156c0a0040 con 0x55de24881410 2024-09-19T06:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15816b2740 10 client.17946 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15816b2740 1 -- 172.21.15.5:0/2930591028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55de24906600 con 0x55de24881410 2024-09-19T06:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15577fe640 1 -- 172.21.15.5:0/2930591028 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1568019970 con 0x55de24881410 2024-09-19T06:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15577fe640 1 -- 172.21.15.5:0/2930591028 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f156c0861f0 con 0x55de24881410 2024-09-19T06:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15816b2740 10 client.17946 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15816b2740 10 client.17946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15816b2740 20 client.17946 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15816b2740 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de24906b70 0x55de24908f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15816b2740 4 client.17946 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f15816b2740 1 -- 172.21.15.5:0/2930591028 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de247ecd70 con 0x55de24906b70 2024-09-19T06:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.589+0000 7f1577fff640 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de24906b70 0x55de24908f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.590+0000 7f15577fe640 10 client.17946 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.590+0000 7f1577fff640 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de24906b70 0x55de24908f90 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a327be2c7fd0315 server_cookie=8948b431633bc32c in_seq=0 out_seq=0 2024-09-19T06:12:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15577fe640 1 -- 172.21.15.5:0/2930591028 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de247ecd70 con 0x55de24906b70 2024-09-19T06:12:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15577fe640 10 client.17946 handle_command_reply: tid=0 2024-09-19T06:12:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15816b2740 1 client.17946 shutdown 2024-09-19T06:12:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15816b2740 1 -- 172.21.15.5:0/2930591028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 msgr2=0x55de2488f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15816b2740 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 0x55de2488f4b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f156c09b990 tx=0x7f156c09bb80 comp rx=0 tx=0).stop 2024-09-19T06:12:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15816b2740 1 -- 172.21.15.5:0/2930591028 shutdown_connections 2024-09-19T06:12:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15816b2740 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de24906b70 0x55de24908f90 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15816b2740 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de2488bcb0 0x55de2488f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15816b2740 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de24881410 0x55de2488f4b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15577fe640 0 client.17946 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15816b2740 1 --2- 172.21.15.5:0/2930591028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de24880af0 0x55de24894190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15816b2740 1 -- 172.21.15.5:0/2930591028 >> 172.21.15.5:0/2930591028 conn(0x55de2487d060 msgr2=0x55de2487b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15816b2740 1 -- 172.21.15.5:0/2930591028 shutdown_connections 2024-09-19T06:12:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15816b2740 1 -- 172.21.15.5:0/2930591028 wait complete. 2024-09-19T06:12:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.622+0000 7f15816b2740 20 client.17946 trim_cache size 0 max 16384 2024-09-19T06:12:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:12:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:12:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:12:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:12:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.623+0000 7f157e918640 1 -- 172.21.15.5:0/4010865097 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15600a33b0 msgr2=0x7f15600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.623+0000 7f157e918640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15600a33b0 0x7f15600a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f157004e000 tx=0x7f157004c000 comp rx=0 tx=0).stop 2024-09-19T06:12:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.623+0000 7f157e918640 1 -- 172.21.15.5:0/4010865097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15781035a0 msgr2=0x7f1578198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.623+0000 7f157e918640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15781035a0 0x7f1578198740 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c890 tx=0x7f156c00cd50 comp rx=0 tx=0).stop 2024-09-19T06:12:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.623+0000 7f157e918640 1 -- 172.21.15.5:0/4010865097 shutdown_connections 2024-09-19T06:12:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.623+0000 7f157e918640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f157810c0a0 0x7f157819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.623+0000 7f157e918640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15781035a0 0x7f1578198740 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.623+0000 7f157e918640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15600a33b0 0x7f15600a5870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.623+0000 7f157e918640 1 --2- 172.21.15.5:0/4010865097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1578102c80 0x7f1578198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.623+0000 7f157e918640 1 -- 172.21.15.5:0/4010865097 >> 172.21.15.5:0/4010865097 conn(0x7f15780779a0 msgr2=0x7f15780fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.623+0000 7f157e918640 1 -- 172.21.15.5:0/4010865097 shutdown_connections 2024-09-19T06:12:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:12:57.623+0000 7f157e918640 1 -- 172.21.15.5:0/4010865097 wait complete. 2024-09-19T06:12:57.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241377 2.52 MB/sec execute 383 sec latency 2099.945 ms 2024-09-19T06:12:58.099 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:12:58.099 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:12:58.362 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:58.748 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:58 smithi005 ceph-mon[30955]: pgmap v298: 97 pgs: 97 active+clean; 135 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-09-19T06:12:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:58 smithi157 ceph-mon[35346]: pgmap v298: 97 pgs: 97 active+clean; 135 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-09-19T06:12:58.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241377 2.52 MB/sec execute 384 sec latency 3099.990 ms 2024-09-19T06:12:58.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.834+0000 7f97e387d640 1 -- 172.21.15.5:0/752826333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97dc10c170 msgr2=0x7f97dc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:58.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.834+0000 7f97e387d640 1 --2- 172.21.15.5:0/752826333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97dc10c170 0x7f97dc10e610 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009ca0 tx=0x7f97d402a950 comp rx=0 tx=0).stop 2024-09-19T06:12:58.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.834+0000 7f97e387d640 1 -- 172.21.15.5:0/752826333 shutdown_connections 2024-09-19T06:12:58.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.834+0000 7f97e387d640 1 --2- 172.21.15.5:0/752826333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97dc10c170 0x7f97dc10e610 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.834+0000 7f97e387d640 1 --2- 172.21.15.5:0/752826333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97dc103660 0x7f97dc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.834+0000 7f97e387d640 1 --2- 172.21.15.5:0/752826333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc102d40 0x7f97dc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.834+0000 7f97e387d640 1 -- 172.21.15.5:0/752826333 >> 172.21.15.5:0/752826333 conn(0x7f97dc0fc7a0 msgr2=0x7f97dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:58.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.834+0000 7f97e387d640 1 -- 172.21.15.5:0/752826333 shutdown_connections 2024-09-19T06:12:58.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.834+0000 7f97e387d640 1 -- 172.21.15.5:0/752826333 wait complete. 2024-09-19T06:12:58.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.834+0000 7f97e387d640 1 Processor -- start 2024-09-19T06:12:58.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.834+0000 7f97e387d640 1 -- start start 2024-09-19T06:12:58.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e387d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97dc102d40 0x7f97dc1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e387d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97dc103660 0x7f97dc198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e387d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc10c170 0x7f97dc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e387d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc111e20 con 0x7f97dc103660 2024-09-19T06:12:58.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e387d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc111ca0 con 0x7f97dc102d40 2024-09-19T06:12:58.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e387d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc111fa0 con 0x7f97dc10c170 2024-09-19T06:12:58.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e0e18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97dc103660 0x7f97dc198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e0e18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97dc103660 0x7f97dc198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49434/0 (socket says 172.21.15.5:49434) 2024-09-19T06:12:58.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e0e18640 1 -- 172.21.15.5:0/1270691491 learned_addr learned my addr 172.21.15.5:0/1270691491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:58.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e1e1a640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc10c170 0x7f97dc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e1619640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97dc102d40 0x7f97dc1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e0e18640 1 -- 172.21.15.5:0/1270691491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc10c170 msgr2=0x7f97dc19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:58.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e0e18640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc10c170 0x7f97dc19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e0e18640 1 -- 172.21.15.5:0/1270691491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97dc102d40 msgr2=0x7f97dc1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:58.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e0e18640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97dc102d40 0x7f97dc1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e0e18640 1 -- 172.21.15.5:0/1270691491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc19d270 con 0x7f97dc103660 2024-09-19T06:12:58.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e1619640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97dc102d40 0x7f97dc1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:12:58.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97e0e18640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97dc103660 0x7f97dc198800 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f97d000c970 tx=0x7f97d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:58.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97ce7fc640 1 -- 172.21.15.5:0/1270691491 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0007bb0 con 0x7f97dc103660 2024-09-19T06:12:58.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.835+0000 7f97ce7fc640 1 -- 172.21.15.5:0/1270691491 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d0007d50 con 0x7f97dc103660 2024-09-19T06:12:58.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.836+0000 7f97e387d640 1 -- 172.21.15.5:0/1270691491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc19d560 con 0x7f97dc103660 2024-09-19T06:12:58.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.836+0000 7f97ce7fc640 1 -- 172.21.15.5:0/1270691491 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0018780 con 0x7f97dc103660 2024-09-19T06:12:58.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.836+0000 7f97e387d640 1 -- 172.21.15.5:0/1270691491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc1a4e40 con 0x7f97dc103660 2024-09-19T06:12:58.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.838+0000 7f97ce7fc640 1 -- 172.21.15.5:0/1270691491 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f97d000f450 con 0x7f97dc103660 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.838+0000 7f97e5c56740 1 Processor -- start 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.838+0000 7f97e5c56740 1 -- start start 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e5c56740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 0x5575886fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e5c56740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 0x55758870bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e5c56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x55758870e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e5c56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557588662280 con 0x55758870c150 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e5c56740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557588609b10 con 0x5575886fca40 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e5c56740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55758865e240 con 0x5575886fd360 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97ce7fc640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f97c80a33b0 0x7f97c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e1e1a640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f97c80a33b0 0x7f97c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97ce7fc640 1 -- 172.21.15.5:0/1270691491 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f97d00cfd70 con 0x7f97dc103660 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e1619640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 0x55758870bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e1e1a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 0x5575886fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e0e18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x55758870e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e1619640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 0x55758870bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52352/0 (socket says 172.21.15.5:52352) 2024-09-19T06:12:58.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e1619640 1 -- 172.21.15.5:0/3253740288 learned_addr learned my addr 172.21.15.5:0/3253740288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:58.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e0e18640 1 -- 172.21.15.5:0/3253740288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 msgr2=0x55758870bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:58.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e0e18640 1 --2- 172.21.15.5:0/3253740288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 0x55758870bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e0e18640 1 -- 172.21.15.5:0/3253740288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 msgr2=0x5575886fce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:58.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e0e18640 1 --2- 172.21.15.5:0/3253740288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 0x5575886fce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e0e18640 1 -- 172.21.15.5:0/3253740288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575886161f0 con 0x55758870c150 2024-09-19T06:12:58.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e1619640 1 --2- 172.21.15.5:0/3253740288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 0x55758870bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:58.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e1e1a640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f97c80a33b0 0x7f97c80a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97d4005e30 tx=0x7f97d402fa80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:58.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.839+0000 7f97e0e18640 1 --2- 172.21.15.5:0/3253740288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x55758870e540 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f97d00cd4a0 tx=0x7f97d009bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e0bbdb3d1b1d43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:58.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.840+0000 7f97c77fe640 1 -- 172.21.15.5:0/3253740288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d000fa90 con 0x55758870c150 2024-09-19T06:12:58.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.840+0000 7f97c77fe640 1 -- 172.21.15.5:0/3253740288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d009c730 con 0x55758870c150 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.840+0000 7f97c77fe640 1 -- 172.21.15.5:0/3253740288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d00a1d50 con 0x55758870c150 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.840+0000 7f97e5c56740 1 -- 172.21.15.5:0/3253740288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 msgr2=0x55758870e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.840+0000 7f97e5c56740 1 --2- 172.21.15.5:0/3253740288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x55758870e540 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f97d00cd4a0 tx=0x7f97d009bc30 comp rx=0 tx=0).stop 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.840+0000 7f97e5c56740 1 -- 172.21.15.5:0/3253740288 shutdown_connections 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.840+0000 7f97e5c56740 1 --2- 172.21.15.5:0/3253740288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x55758870e540 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.840+0000 7f97e5c56740 1 --2- 172.21.15.5:0/3253740288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 0x55758870bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.840+0000 7f97e5c56740 1 --2- 172.21.15.5:0/3253740288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 0x5575886fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.841+0000 7f97e5c56740 1 -- 172.21.15.5:0/3253740288 >> 172.21.15.5:0/3253740288 conn(0x5575886f90f0 msgr2=0x5575886f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.841+0000 7f97e5c56740 1 -- 172.21.15.5:0/3253740288 shutdown_connections 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.841+0000 7f97e5c56740 1 -- 172.21.15.5:0/3253740288 wait complete. 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.841+0000 7f97e5c56740 1 -- start start 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.841+0000 7f97e5c56740 1 Processor -- start 2024-09-19T06:12:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e5c56740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 0x5575887090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e5c56740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 0x5575887095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e5c56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x557588709b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e5c56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55758870ed20 con 0x55758870c150 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e5c56740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55758861cf10 con 0x5575886fca40 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e5c56740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55758870eea0 con 0x5575886fd360 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e5c56740 10 client.0 _ll_register_callbacks cb 0x5575885e5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e1e1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x557588709b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e1e1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x557588709b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49450/0 (socket says 172.21.15.5:49450) 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e1e1a640 1 -- 172.21.15.5:0/2977237490 learned_addr learned my addr 172.21.15.5:0/2977237490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e1619640 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 0x5575887090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e0e18640 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 0x5575887095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e1e1a640 1 -- 172.21.15.5:0/2977237490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 msgr2=0x5575887095e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:58.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e1e1a640 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 0x5575887095e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e1e1a640 1 -- 172.21.15.5:0/2977237490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 msgr2=0x5575887090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e1e1a640 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 0x5575887090a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e1e1a640 1 -- 172.21.15.5:0/2977237490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575883d36a0 con 0x55758870c150 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e0e18640 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 0x5575887095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e1e1a640 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x557588709b20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f97d4005fe0 tx=0x7f97d4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f80f192d6e5d886 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97c4ff9640 1 -- 172.21.15.5:0/2977237490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d404e070 con 0x55758870c150 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97e5c56740 1 -- 172.21.15.5:0/2977237490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d009c730 con 0x55758870c150 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.842+0000 7f97c4ff9640 1 -- 172.21.15.5:0/2977237490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f97d4004040 con 0x55758870c150 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.843+0000 7f97c4ff9640 1 -- 172.21.15.5:0/2977237490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f97d4034040 con 0x55758870c150 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.843+0000 7f97e5c56740 10 client.17958 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.843+0000 7f97e5c56740 1 -- 172.21.15.5:0/2977237490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55758871a9c0 con 0x55758870c150 2024-09-19T06:12:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.843+0000 7f97c4ff9640 1 -- 172.21.15.5:0/2977237490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d009c730 con 0x55758870c150 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.843+0000 7f97c4ff9640 1 -- 172.21.15.5:0/2977237490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f97d4053930 con 0x55758870c150 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.843+0000 7f97e5c56740 10 client.17958 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.843+0000 7f97e5c56740 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.843+0000 7f97e5c56740 20 client.17958 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.843+0000 7f97e5c56740 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55758871be80 0x55758871e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.843+0000 7f97e5c56740 4 client.17958 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.843+0000 7f97e5c56740 1 -- 172.21.15.5:0/2977237490 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55758866f7f0 con 0x55758871be80 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.843+0000 7f97e1619640 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55758871be80 0x55758871e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.844+0000 7f97c4ff9640 10 client.17958 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.844+0000 7f97e1619640 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55758871be80 0x55758871e2d0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21753f60ab535903 server_cookie=4b06584a6f04f897 in_seq=0 out_seq=0 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.844+0000 7f97c4ff9640 1 -- 172.21.15.5:0/2977237490 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55758866f7f0 con 0x55758871be80 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.844+0000 7f97c4ff9640 10 client.17958 handle_command_reply: tid=0 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97e5c56740 1 client.17958 shutdown 2024-09-19T06:12:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97e5c56740 1 -- 172.21.15.5:0/2977237490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 msgr2=0x557588709b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97e5c56740 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x557588709b20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f97d4005fe0 tx=0x7f97d4005c50 comp rx=0 tx=0).stop 2024-09-19T06:12:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97e5c56740 1 -- 172.21.15.5:0/2977237490 shutdown_connections 2024-09-19T06:12:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97e5c56740 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55758871be80 0x55758871e2d0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97e5c56740 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x557588709b20 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97e5c56740 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fd360 0x5575887095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97e5c56740 1 --2- 172.21.15.5:0/2977237490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 0x5575887090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97e5c56740 1 -- 172.21.15.5:0/2977237490 >> 172.21.15.5:0/2977237490 conn(0x5575886f90f0 msgr2=0x5575886fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97c4ff9640 0 client.17958 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97e5c56740 1 -- 172.21.15.5:0/2977237490 shutdown_connections 2024-09-19T06:12:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97e5c56740 1 -- 172.21.15.5:0/2977237490 wait complete. 2024-09-19T06:12:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.845+0000 7f97e5c56740 20 client.17958 trim_cache size 0 max 16384 2024-09-19T06:12:58.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.862+0000 7f97e5c56740 1 Processor -- start 2024-09-19T06:12:58.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.862+0000 7f97e5c56740 1 -- start start 2024-09-19T06:12:58.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e5c56740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fca40 0x557588723ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e5c56740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fd360 0x5575887240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e5c56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x557588706300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e5c56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1ab640 con 0x55758870c150 2024-09-19T06:12:58.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e5c56740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1ad1b0 con 0x5575886fd360 2024-09-19T06:12:58.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e5c56740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1abd10 con 0x5575886fca40 2024-09-19T06:12:58.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e1e1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x557588706300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e1e1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x557588706300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49454/0 (socket says 172.21.15.5:49454) 2024-09-19T06:12:58.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e1e1a640 1 -- 172.21.15.5:0/701215256 learned_addr learned my addr 172.21.15.5:0/701215256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:58.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e1619640 1 --2- 172.21.15.5:0/701215256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fca40 0x557588723ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e0e18640 1 --2- 172.21.15.5:0/701215256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fd360 0x5575887240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e1e1a640 1 -- 172.21.15.5:0/701215256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fca40 msgr2=0x557588723ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:58.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e1e1a640 1 --2- 172.21.15.5:0/701215256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fca40 0x557588723ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e1e1a640 1 -- 172.21.15.5:0/701215256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fd360 msgr2=0x5575887240e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:58.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e1e1a640 1 --2- 172.21.15.5:0/701215256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fd360 0x5575887240e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e1e1a640 1 -- 172.21.15.5:0/701215256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557588708050 con 0x55758870c150 2024-09-19T06:12:58.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e0e18640 1 --2- 172.21.15.5:0/701215256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fd360 0x5575887240e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:12:58.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.863+0000 7f97e1e1a640 1 --2- 172.21.15.5:0/701215256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x557588706300 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f97d402ae70 tx=0x7f97d402f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e2149a8be75d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:58.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.864+0000 7f97c6ffd640 1 -- 172.21.15.5:0/701215256 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4060070 con 0x55758870c150 2024-09-19T06:12:58.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.864+0000 7f97c6ffd640 1 -- 172.21.15.5:0/701215256 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d4009540 con 0x55758870c150 2024-09-19T06:12:58.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.864+0000 7f97c6ffd640 1 -- 172.21.15.5:0/701215256 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d404e040 con 0x55758870c150 2024-09-19T06:12:58.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.864+0000 7f97e5c56740 1 -- 172.21.15.5:0/701215256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 msgr2=0x557588706300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.864+0000 7f97e5c56740 1 --2- 172.21.15.5:0/701215256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x557588706300 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f97d402ae70 tx=0x7f97d402f450 comp rx=0 tx=0).stop 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.864+0000 7f97e5c56740 1 -- 172.21.15.5:0/701215256 shutdown_connections 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.864+0000 7f97e5c56740 1 --2- 172.21.15.5:0/701215256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758870c150 0x557588706300 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.864+0000 7f97e5c56740 1 --2- 172.21.15.5:0/701215256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fd360 0x5575887240e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.864+0000 7f97e5c56740 1 --2- 172.21.15.5:0/701215256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575886fca40 0x557588723ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.864+0000 7f97e5c56740 1 -- 172.21.15.5:0/701215256 >> 172.21.15.5:0/701215256 conn(0x5575886f90f0 msgr2=0x55758870b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.864+0000 7f97e5c56740 1 -- 172.21.15.5:0/701215256 shutdown_connections 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.864+0000 7f97e5c56740 1 -- 172.21.15.5:0/701215256 wait complete. 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e5c56740 1 -- start start 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e5c56740 1 Processor -- start 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e5c56740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 0x5575887100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e5c56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575886fd360 0x557588710600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e5c56740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55758870c150 0x557588710b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e5c56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1acae0 con 0x5575886fd360 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e5c56740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1ac3e0 con 0x5575886fca40 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e5c56740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1a9660 con 0x55758870c150 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e5c56740 10 client.0 _ll_register_callbacks cb 0x7f97d4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e0e18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575886fd360 0x557588710600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e0e18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575886fd360 0x557588710600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49464/0 (socket says 172.21.15.5:49464) 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e0e18640 1 -- 172.21.15.5:0/2418698298 learned_addr learned my addr 172.21.15.5:0/2418698298 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e1e1a640 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55758870c150 0x557588710b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.865+0000 7f97e1619640 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 0x5575887100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97e0e18640 1 -- 172.21.15.5:0/2418698298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55758870c150 msgr2=0x557588710b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97e0e18640 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55758870c150 0x557588710b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97e0e18640 1 -- 172.21.15.5:0/2418698298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 msgr2=0x5575887100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97e0e18640 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 0x5575887100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97e0e18640 1 -- 172.21.15.5:0/2418698298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557588785d70 con 0x5575886fd360 2024-09-19T06:12:58.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97e1e1a640 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55758870c150 0x557588710b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:12:58.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97e0e18640 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575886fd360 0x557588710600 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f97d009a2e0 tx=0x7f97d009bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d123eb8a55ed5c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:12:58.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:12:58.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97c4ff9640 1 -- 172.21.15.5:0/2418698298 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d009c730 con 0x5575886fd360 2024-09-19T06:12:58.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97c4ff9640 1 -- 172.21.15.5:0/2418698298 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f97d00a1850 con 0x5575886fd360 2024-09-19T06:12:58.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97e5c56740 1 -- 172.21.15.5:0/2418698298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d4009540 con 0x5575886fd360 2024-09-19T06:12:58.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97c4ff9640 1 -- 172.21.15.5:0/2418698298 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f97d000fa90 con 0x5575886fd360 2024-09-19T06:12:58.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97e5c56740 10 client.17964 fetch_fsmap learned FSMap version 35 2024-09-19T06:12:58.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97e5c56740 1 -- 172.21.15.5:0/2418698298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557588786fe0 con 0x5575886fd360 2024-09-19T06:12:58.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.866+0000 7f97c4ff9640 1 -- 172.21.15.5:0/2418698298 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d4009540 con 0x5575886fd360 2024-09-19T06:12:58.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.867+0000 7f97c4ff9640 1 -- 172.21.15.5:0/2418698298 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f97d0062030 con 0x5575886fd360 2024-09-19T06:12:58.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.867+0000 7f97e5c56740 10 client.17964 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:12:58.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.867+0000 7f97e5c56740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:12:58.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.867+0000 7f97e5c56740 20 client.17964 populate_metadata read hostname 'smithi005' 2024-09-19T06:12:58.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.867+0000 7f97e5c56740 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557588787370 0x557588789760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.867+0000 7f97e5c56740 4 client.17964 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.867+0000 7f97e5c56740 1 -- 172.21.15.5:0/2418698298 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557588669d00 con 0x557588787370 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.867+0000 7f97e1619640 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557588787370 0x557588789760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.867+0000 7f97c4ff9640 10 client.17964 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.867+0000 7f97e1619640 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557588787370 0x557588789760 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e92efbb0f264157d server_cookie=e314296cf50a7b92 in_seq=0 out_seq=0 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.868+0000 7f97c4ff9640 1 -- 172.21.15.5:0/2418698298 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557588669d00 con 0x557588787370 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.868+0000 7f97c4ff9640 10 client.17964 handle_command_reply: tid=0 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.868+0000 7f97e5c56740 1 client.17964 shutdown 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.868+0000 7f97e5c56740 1 -- 172.21.15.5:0/2418698298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575886fd360 msgr2=0x557588710600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.868+0000 7f97e5c56740 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575886fd360 0x557588710600 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f97d009a2e0 tx=0x7f97d009bf20 comp rx=0 tx=0).stop 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e5c56740 1 -- 172.21.15.5:0/2418698298 shutdown_connections 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e5c56740 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557588787370 0x557588789760 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e5c56740 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55758870c150 0x557588710b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e5c56740 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575886fd360 0x557588710600 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e5c56740 1 --2- 172.21.15.5:0/2418698298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575886fca40 0x5575887100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97c4ff9640 0 client.17964 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:12:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e5c56740 1 -- 172.21.15.5:0/2418698298 >> 172.21.15.5:0/2418698298 conn(0x5575886f90f0 msgr2=0x557588709060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e5c56740 1 -- 172.21.15.5:0/2418698298 shutdown_connections 2024-09-19T06:12:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e5c56740 1 -- 172.21.15.5:0/2418698298 wait complete. 2024-09-19T06:12:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e5c56740 20 client.17964 trim_cache size 0 max 16384 2024-09-19T06:12:58.875 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:12:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:12:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "854d460e-5137-4fc8-b5f5-7abf169e5857", 2024-09-19T06:12:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:12:58.876 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:12:58.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e387d640 1 -- 172.21.15.5:0/1270691491 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f97c80a33b0 msgr2=0x7f97c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e387d640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f97c80a33b0 0x7f97c80a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97d4005e30 tx=0x7f97d402fa80 comp rx=0 tx=0).stop 2024-09-19T06:12:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e387d640 1 -- 172.21.15.5:0/1270691491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97dc103660 msgr2=0x7f97dc198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:12:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e387d640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97dc103660 0x7f97dc198800 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f97d000c970 tx=0x7f97d000ce30 comp rx=0 tx=0).stop 2024-09-19T06:12:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e387d640 1 -- 172.21.15.5:0/1270691491 shutdown_connections 2024-09-19T06:12:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e387d640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f97c80a33b0 0x7f97c80a5870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e387d640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc10c170 0x7f97dc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e387d640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97dc103660 0x7f97dc198800 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e387d640 1 --2- 172.21.15.5:0/1270691491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97dc102d40 0x7f97dc1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:12:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.869+0000 7f97e387d640 1 -- 172.21.15.5:0/1270691491 >> 172.21.15.5:0/1270691491 conn(0x7f97dc0fc7a0 msgr2=0x7f97dc0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:12:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.870+0000 7f97e387d640 1 -- 172.21.15.5:0/1270691491 shutdown_connections 2024-09-19T06:12:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:12:58.870+0000 7f97e387d640 1 -- 172.21.15.5:0/1270691491 wait complete. 2024-09-19T06:12:58.883 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:58 smithi049 ceph-mon[34640]: pgmap v298: 97 pgs: 97 active+clean; 135 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-09-19T06:12:59.146 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:12:58 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:12:58.881+0000 7f9e1e5f8640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...e,head] /volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{2=1} v39155 ap=2 f(v1560 m2024-09-19T06:12:48.662582+0000 5=5+0) n(v51 rc2024-09-19T06:12:48.662582+0000 b814326 6=5+1) old_inodes=6 (inest sync->mix g=2) caps={15255=pAsLsXsFs/-@34850} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x559db3497600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v366>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v39155>,<0x10000000008/~dmtmp v36841>,<0x10000000007/client0 v1193>,<0x10000000006/clients v1169>,<0x10000000005/tmp v1093>,<0x10000000003/client.0 v1077>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v1076>,<0x10000000001/sv_0 v1074>,<0x10000000000/qa v1058>,<0x1/volumes v1046>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T06:12:48.650754+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T06:12:48.662582+0000 b814326 6=5+1)","memory_value.dirstat":"f(v1560 m2024-09-19T06:12:48.662582+0000 5=5+0)","memory_value.rstat":"n(v51 rc2024-09-19T06:12:48.662582+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T06:12:59.146 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:12:58 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:12:58.897+0000 7f9e1e5f8640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...e,head] /volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PM/ auth{2=1} v39145 ap=2 f(v4810 m2024-09-19T06:12:47.211212+0000 3=3+0) n(v51 rc2024-09-19T06:12:47.211212+0000 b780288 4=3+1) old_inodes=6 (inest sync->mix g=2) caps={15255=pAsLsXsFs/-@48103} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x559db3404580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v66>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v39145>,<0x10000000008/~dmtmp v36851>,<0x10000000007/client0 v1195>,<0x10000000006/clients v1171>,<0x10000000005/tmp v1095>,<0x10000000003/client.0 v1079>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v1078>,<0x10000000001/sv_0 v1076>,<0x10000000000/qa v1060>,<0x1/volumes v1048>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T06:12:46.835542+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T06:12:47.211212+0000 b780288 4=3+1)","memory_value.dirstat":"f(v4810 m2024-09-19T06:12:47.211212+0000 3=3+0)","memory_value.rstat":"n(v51 rc2024-09-19T06:12:47.211212+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T06:12:59.147 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:12:58 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:12:58.898+0000 7f9e1e5f8640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [...e,head] /volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ auth{2=1} v39153 ap=2 f(v1820 m2024-09-19T06:12:48.060464+0000 7=7+0) n(v53 rc2024-09-19T06:12:48.060464+0000 b1462582 8=7+1) old_inodes=6 (inest sync->mix g=2) caps={15255=pAsLsXsFs/-@34650} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x559db34c4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020f:[<0x10000000009/PWRPNT v274>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x1000000020f:[<0x10000000009/PWRPNT v39153>,<0x10000000008/~dmtmp v36851>,<0x10000000007/client0 v1195>,<0x10000000006/clients v1171>,<0x10000000005/tmp v1095>,<0x10000000003/client.0 v1079>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v1078>,<0x10000000001/sv_0 v1076>,<0x10000000000/qa v1060>,<0x1/volumes v1048>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T06:12:48.035910+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T06:12:48.060464+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v1820 m2024-09-19T06:12:48.060464+0000 7=7+0)","memory_value.rstat":"n(v53 rc2024-09-19T06:12:48.060464+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T06:12:59.546 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:59 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:12:59.546 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:59 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:59.546 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:59 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:12:59.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:59 smithi005 ceph-mon[30955]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-19T06:12:59.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:59 smithi005 ceph-mon[30955]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-09-19T06:12:59.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:59 smithi005 ceph-mon[30955]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-09-19T06:12:59.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:12:59 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:59.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:12:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:59 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:12:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:59 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:59 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:12:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:59 smithi157 ceph-mon[35346]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-19T06:12:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:59 smithi157 ceph-mon[35346]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-09-19T06:12:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:59 smithi157 ceph-mon[35346]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-09-19T06:12:59.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:12:59 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:59.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241625 2.51 MB/sec execute 385 sec latency 3158.822 ms 2024-09-19T06:12:59.852 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:12:59.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:59 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:12:59.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:59 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:12:59.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:59 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:12:59.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:59 smithi049 ceph-mon[34640]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-19T06:12:59.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:59 smithi049 ceph-mon[34640]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-09-19T06:12:59.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:59 smithi049 ceph-mon[34640]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-09-19T06:12:59.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:12:59 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.341+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1918417228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c100a80 msgr2=0x7f4e1c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:00.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.341+0000 7f4e22cd8640 1 --2- 172.21.15.5:0/1918417228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c100a80 0x7f4e1c100ee0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009930 tx=0x7f4e0802a420 comp rx=0 tx=0).stop 2024-09-19T06:13:00.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.341+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1918417228 shutdown_connections 2024-09-19T06:13:00.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.341+0000 7f4e22cd8640 1 --2- 172.21.15.5:0/1918417228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c101420 0x7f4e1c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.341+0000 7f4e22cd8640 1 --2- 172.21.15.5:0/1918417228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c100a80 0x7f4e1c100ee0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.341+0000 7f4e22cd8640 1 --2- 172.21.15.5:0/1918417228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e1c106a80 0x7f4e1c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.341+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1918417228 >> 172.21.15.5:0/1918417228 conn(0x7f4e1c0fc7a0 msgr2=0x7f4e1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:00.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.341+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1918417228 shutdown_connections 2024-09-19T06:13:00.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.341+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1918417228 wait complete. 2024-09-19T06:13:00.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e22cd8640 1 Processor -- start 2024-09-19T06:13:00.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e22cd8640 1 -- start start 2024-09-19T06:13:00.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e22cd8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e1c100a80 0x7f4e1c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e22cd8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c101420 0x7f4e1c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e22cd8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c106a80 0x7f4e1c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e22cd8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c110d30 con 0x7f4e1c106a80 2024-09-19T06:13:00.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e22cd8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c110bb0 con 0x7f4e1c101420 2024-09-19T06:13:00.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e22cd8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c110eb0 con 0x7f4e1c100a80 2024-09-19T06:13:00.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e21275640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c106a80 0x7f4e1c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e1bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c101420 0x7f4e1c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e21275640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c106a80 0x7f4e1c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49482/0 (socket says 172.21.15.5:49482) 2024-09-19T06:13:00.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e20a74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e1c100a80 0x7f4e1c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.342+0000 7f4e21275640 1 -- 172.21.15.5:0/1958155475 learned_addr learned my addr 172.21.15.5:0/1958155475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:00.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.343+0000 7f4e21275640 1 -- 172.21.15.5:0/1958155475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e1c100a80 msgr2=0x7f4e1c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:00.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.343+0000 7f4e21275640 1 --2- 172.21.15.5:0/1958155475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e1c100a80 0x7f4e1c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.343+0000 7f4e21275640 1 -- 172.21.15.5:0/1958155475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c101420 msgr2=0x7f4e1c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:00.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.343+0000 7f4e21275640 1 --2- 172.21.15.5:0/1958155475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c101420 0x7f4e1c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.343+0000 7f4e21275640 1 -- 172.21.15.5:0/1958155475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1c19f560 con 0x7f4e1c106a80 2024-09-19T06:13:00.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.343+0000 7f4e21275640 1 --2- 172.21.15.5:0/1958155475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c106a80 0x7f4e1c19ee80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009fd0 tx=0x7f4e1400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:00.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.343+0000 7f4e19ffb640 1 -- 172.21.15.5:0/1958155475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14017070 con 0x7f4e1c106a80 2024-09-19T06:13:00.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.343+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1958155475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1c19f850 con 0x7f4e1c106a80 2024-09-19T06:13:00.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.343+0000 7f4e19ffb640 1 -- 172.21.15.5:0/1958155475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e14003e20 con 0x7f4e1c106a80 2024-09-19T06:13:00.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.343+0000 7f4e19ffb640 1 -- 172.21.15.5:0/1958155475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14010460 con 0x7f4e1c106a80 2024-09-19T06:13:00.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.343+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1958155475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1c1a7130 con 0x7f4e1c106a80 2024-09-19T06:13:00.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.346+0000 7f4e19ffb640 1 -- 172.21.15.5:0/1958155475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4e14003970 con 0x7f4e1c106a80 2024-09-19T06:13:00.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.346+0000 7f4e250b1740 1 Processor -- start 2024-09-19T06:13:00.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.346+0000 7f4e250b1740 1 -- start start 2024-09-19T06:13:00.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.346+0000 7f4e19ffb640 1 --2- 172.21.15.5:0/1958155475 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4e0c0a33b0 0x7f4e0c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.346+0000 7f4e250b1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa5964a0 0x561efa596880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.346+0000 7f4e250b1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa596dc0 0x561efa5a1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.346+0000 7f4e250b1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5a1680 0x561efa5a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.346+0000 7f4e250b1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561efa4fc2e0 con 0x561efa596dc0 2024-09-19T06:13:00.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.346+0000 7f4e250b1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561efa4a3ce0 con 0x561efa5a1680 2024-09-19T06:13:00.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.346+0000 7f4e250b1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561efa4f82a0 con 0x561efa5964a0 2024-09-19T06:13:00.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e19ffb640 1 -- 172.21.15.5:0/1958155475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f4e140cfae0 con 0x7f4e1c106a80 2024-09-19T06:13:00.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/1958155475 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4e0c0a33b0 0x7f4e0c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e20a74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa5964a0 0x561efa596880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e20a74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa5964a0 0x561efa596880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52426/0 (socket says 172.21.15.5:52426) 2024-09-19T06:13:00.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e20a74640 1 -- 172.21.15.5:0/2779841189 learned_addr learned my addr 172.21.15.5:0/2779841189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e21275640 1 --2- 172.21.15.5:0/2779841189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5a1680 0x561efa5a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/2779841189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa596dc0 0x561efa5a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e20a74640 1 -- 172.21.15.5:0/2779841189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5a1680 msgr2=0x561efa5a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e20a74640 1 --2- 172.21.15.5:0/2779841189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5a1680 0x561efa5a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e20a74640 1 -- 172.21.15.5:0/2779841189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa596dc0 msgr2=0x561efa5a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e20a74640 1 --2- 172.21.15.5:0/2779841189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa596dc0 0x561efa5a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e20a74640 1 -- 172.21.15.5:0/2779841189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561efa488440 con 0x561efa5964a0 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.347+0000 7f4e21275640 1 --2- 172.21.15.5:0/2779841189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5a1680 0x561efa5a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.348+0000 7f4e20a74640 1 --2- 172.21.15.5:0/2779841189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa5964a0 0x561efa596880 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000bc90 tx=0x7f4e100107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bd23052597cf002 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.348+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/1958155475 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4e0c0a33b0 0x7f4e0c0a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009900 tx=0x7f4e08032b00 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.348+0000 7f4e06ffd640 1 -- 172.21.15.5:0/2779841189 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1001c480 con 0x561efa5964a0 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.348+0000 7f4e06ffd640 1 -- 172.21.15.5:0/2779841189 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e1001cad0 con 0x561efa5964a0 2024-09-19T06:13:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.348+0000 7f4e06ffd640 1 -- 172.21.15.5:0/2779841189 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10011a80 con 0x561efa5964a0 2024-09-19T06:13:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.349+0000 7f4e250b1740 1 -- 172.21.15.5:0/2779841189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa5964a0 msgr2=0x561efa596880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.349+0000 7f4e250b1740 1 --2- 172.21.15.5:0/2779841189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa5964a0 0x561efa596880 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000bc90 tx=0x7f4e100107a0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.349+0000 7f4e250b1740 1 -- 172.21.15.5:0/2779841189 shutdown_connections 2024-09-19T06:13:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.349+0000 7f4e250b1740 1 --2- 172.21.15.5:0/2779841189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5a1680 0x561efa5a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.349+0000 7f4e250b1740 1 --2- 172.21.15.5:0/2779841189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa596dc0 0x561efa5a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.349+0000 7f4e250b1740 1 --2- 172.21.15.5:0/2779841189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa5964a0 0x561efa596880 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.349+0000 7f4e250b1740 1 -- 172.21.15.5:0/2779841189 >> 172.21.15.5:0/2779841189 conn(0x561efa592a30 msgr2=0x561efa592e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.349+0000 7f4e250b1740 1 -- 172.21.15.5:0/2779841189 shutdown_connections 2024-09-19T06:13:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.349+0000 7f4e250b1740 1 -- 172.21.15.5:0/2779841189 wait complete. 2024-09-19T06:13:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.349+0000 7f4e250b1740 1 -- start start 2024-09-19T06:13:00.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.349+0000 7f4e250b1740 1 Processor -- start 2024-09-19T06:13:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.350+0000 7f4e250b1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa5964a0 0x561efa5a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.350+0000 7f4e250b1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa596dc0 0x561efa59e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.350+0000 7f4e250b1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5a1680 0x561efa59e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.350+0000 7f4e250b1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561efa5a4250 con 0x561efa5964a0 2024-09-19T06:13:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.350+0000 7f4e250b1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561efa4b7050 con 0x561efa5a1680 2024-09-19T06:13:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.350+0000 7f4e250b1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561efa5a43d0 con 0x561efa596dc0 2024-09-19T06:13:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.350+0000 7f4e250b1740 10 client.0 _ll_register_callbacks cb 0x561efa4f7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.350+0000 7f4e20a74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa5964a0 0x561efa5a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.350+0000 7f4e20a74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa5964a0 0x561efa5a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49508/0 (socket says 172.21.15.5:49508) 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.350+0000 7f4e20a74640 1 -- 172.21.15.5:0/1379877001 learned_addr learned my addr 172.21.15.5:0/1379877001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.350+0000 7f4e21275640 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5a1680 0x561efa59e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.350+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa596dc0 0x561efa59e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4e20a74640 1 -- 172.21.15.5:0/1379877001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa596dc0 msgr2=0x561efa59e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4e20a74640 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa596dc0 0x561efa59e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4e20a74640 1 -- 172.21.15.5:0/1379877001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5a1680 msgr2=0x561efa59e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4e20a74640 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5a1680 0x561efa59e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4e20a74640 1 -- 172.21.15.5:0/1379877001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561efa26d3c0 con 0x561efa5964a0 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa596dc0 0x561efa59e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4e21275640 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5a1680 0x561efa59e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4e20a74640 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa5964a0 0x561efa5a0f80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4e10011010 tx=0x7f4e1001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6b0ae1c985e9cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4dfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4dfbfff640 1 -- 172.21.15.5:0/1379877001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1001c920 con 0x561efa5964a0 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4dfbfff640 1 -- 172.21.15.5:0/1379877001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f4e10005250 con 0x561efa5964a0 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4e250b1740 1 -- 172.21.15.5:0/1379877001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e1001cad0 con 0x561efa5964a0 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4dfbfff640 1 -- 172.21.15.5:0/1379877001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4e10005450 con 0x561efa5964a0 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4e250b1740 10 client.17976 fetch_fsmap learned FSMap version 36 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4e250b1740 1 -- 172.21.15.5:0/1379877001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561efa5afc20 con 0x561efa5964a0 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.351+0000 7f4dfbfff640 1 -- 172.21.15.5:0/1379877001 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e1001cad0 con 0x561efa5964a0 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.352+0000 7f4dfbfff640 1 -- 172.21.15.5:0/1379877001 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f4e10004680 con 0x561efa5964a0 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.352+0000 7f4e250b1740 10 client.17976 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.352+0000 7f4e250b1740 10 client.17976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.352+0000 7f4e250b1740 20 client.17976 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.352+0000 7f4e250b1740 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561efa5b1030 0x561efa5b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.352+0000 7f4e250b1740 4 client.17976 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.352+0000 7f4e250b1740 1 -- 172.21.15.5:0/1379877001 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561efa509850 con 0x561efa5b1030 2024-09-19T06:13:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.352+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561efa5b1030 0x561efa5b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.353+0000 7f4dfbfff640 10 client.17976 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.353+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561efa5b1030 0x561efa5b3420 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d6bce81abd69769 server_cookie=45e889ca33bd8df in_seq=0 out_seq=0 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.353+0000 7f4dfbfff640 1 -- 172.21.15.5:0/1379877001 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561efa509850 con 0x561efa5b1030 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.353+0000 7f4dfbfff640 10 client.17976 handle_command_reply: tid=0 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.353+0000 7f4e250b1740 1 client.17976 shutdown 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.353+0000 7f4e250b1740 1 -- 172.21.15.5:0/1379877001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa5964a0 msgr2=0x561efa5a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.353+0000 7f4e250b1740 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa5964a0 0x561efa5a0f80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4e10011010 tx=0x7f4e1001c730 comp rx=0 tx=0).stop 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.353+0000 7f4e250b1740 1 -- 172.21.15.5:0/1379877001 shutdown_connections 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.354+0000 7f4e250b1740 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561efa5b1030 0x561efa5b3420 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.354+0000 7f4e250b1740 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5a1680 0x561efa59e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.354+0000 7f4e250b1740 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa596dc0 0x561efa59e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.354+0000 7f4e250b1740 1 --2- 172.21.15.5:0/1379877001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa5964a0 0x561efa5a0f80 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.354+0000 7f4e250b1740 1 -- 172.21.15.5:0/1379877001 >> 172.21.15.5:0/1379877001 conn(0x561efa592a30 msgr2=0x561efa598760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.354+0000 7f4dfbfff640 0 client.17976 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.354+0000 7f4e250b1740 1 -- 172.21.15.5:0/1379877001 shutdown_connections 2024-09-19T06:13:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.354+0000 7f4e250b1740 1 -- 172.21.15.5:0/1379877001 wait complete. 2024-09-19T06:13:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.354+0000 7f4e250b1740 20 client.17976 trim_cache size 0 max 16384 2024-09-19T06:13:00.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e250b1740 1 Processor -- start 2024-09-19T06:13:00.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e250b1740 1 -- start start 2024-09-19T06:13:00.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e250b1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 0x561efa5ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e250b1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa5964a0 0x561efa59cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e250b1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa596dc0 0x561efa5997f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e250b1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c1af9a0 con 0x561efa592a30 2024-09-19T06:13:00.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e250b1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c1b1540 con 0x561efa596dc0 2024-09-19T06:13:00.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e250b1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c1b0070 con 0x561efa5964a0 2024-09-19T06:13:00.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e20a74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 0x561efa5ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e20a74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 0x561efa5ad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49510/0 (socket says 172.21.15.5:49510) 2024-09-19T06:13:00.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e20a74640 1 -- 172.21.15.5:0/755155499 learned_addr learned my addr 172.21.15.5:0/755155499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:00.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e21275640 1 --2- 172.21.15.5:0/755155499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa596dc0 0x561efa5997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/755155499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa5964a0 0x561efa59cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e20a74640 1 -- 172.21.15.5:0/755155499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa5964a0 msgr2=0x561efa59cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:00.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e20a74640 1 --2- 172.21.15.5:0/755155499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa5964a0 0x561efa59cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e20a74640 1 -- 172.21.15.5:0/755155499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa596dc0 msgr2=0x561efa5997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:00.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e20a74640 1 --2- 172.21.15.5:0/755155499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa596dc0 0x561efa5997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e20a74640 1 -- 172.21.15.5:0/755155499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561efa59d1e0 con 0x561efa592a30 2024-09-19T06:13:00.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.374+0000 7f4e21275640 1 --2- 172.21.15.5:0/755155499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa596dc0 0x561efa5997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:00.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.375+0000 7f4e20a74640 1 --2- 172.21.15.5:0/755155499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 0x561efa5ad4b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4e10000c00 tx=0x7f4e10010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea20e74683e7e4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:00.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.375+0000 7f4e067fc640 1 -- 172.21.15.5:0/755155499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000b820 con 0x561efa592a30 2024-09-19T06:13:00.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.375+0000 7f4e067fc640 1 -- 172.21.15.5:0/755155499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e10011440 con 0x561efa592a30 2024-09-19T06:13:00.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.375+0000 7f4e067fc640 1 -- 172.21.15.5:0/755155499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e100055f0 con 0x561efa592a30 2024-09-19T06:13:00.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.376+0000 7f4e250b1740 1 -- 172.21.15.5:0/755155499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 msgr2=0x561efa5ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:00.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.376+0000 7f4e250b1740 1 --2- 172.21.15.5:0/755155499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 0x561efa5ad4b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4e10000c00 tx=0x7f4e10010d20 comp rx=0 tx=0).stop 2024-09-19T06:13:00.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.376+0000 7f4e250b1740 1 -- 172.21.15.5:0/755155499 shutdown_connections 2024-09-19T06:13:00.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.376+0000 7f4e250b1740 1 --2- 172.21.15.5:0/755155499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa596dc0 0x561efa5997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.376+0000 7f4e250b1740 1 --2- 172.21.15.5:0/755155499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa5964a0 0x561efa59cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.376+0000 7f4e250b1740 1 --2- 172.21.15.5:0/755155499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 0x561efa5ad4b0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.376+0000 7f4e250b1740 1 -- 172.21.15.5:0/755155499 >> 172.21.15.5:0/755155499 conn(0x561efa59e8d0 msgr2=0x561efa5990e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:00.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.376+0000 7f4e250b1740 1 -- 172.21.15.5:0/755155499 shutdown_connections 2024-09-19T06:13:00.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.376+0000 7f4e250b1740 1 -- 172.21.15.5:0/755155499 wait complete. 2024-09-19T06:13:00.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.376+0000 7f4e250b1740 1 -- start start 2024-09-19T06:13:00.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.376+0000 7f4e250b1740 1 Processor -- start 2024-09-19T06:13:00.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e250b1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 0x561efa5a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e250b1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5964a0 0x561efa5a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e250b1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa596dc0 0x561efa619b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e250b1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c1b0e70 con 0x561efa592a30 2024-09-19T06:13:00.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e250b1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c1b0740 con 0x561efa5964a0 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e250b1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c1ad9a0 con 0x561efa596dc0 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e250b1740 10 client.0 _ll_register_callbacks cb 0x7f4e08032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e20a74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 0x561efa5a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e20a74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 0x561efa5a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49526/0 (socket says 172.21.15.5:49526) 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e20a74640 1 -- 172.21.15.5:0/657939399 learned_addr learned my addr 172.21.15.5:0/657939399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/657939399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5964a0 0x561efa5a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e20a74640 1 -- 172.21.15.5:0/657939399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa596dc0 msgr2=0x561efa619b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e20a74640 1 --2- 172.21.15.5:0/657939399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa596dc0 0x561efa619b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e20a74640 1 -- 172.21.15.5:0/657939399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5964a0 msgr2=0x561efa5a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e20a74640 1 --2- 172.21.15.5:0/657939399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5964a0 0x561efa5a50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.377+0000 7f4e20a74640 1 -- 172.21.15.5:0/657939399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561efa5a12f0 con 0x561efa592a30 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4e20a74640 1 --2- 172.21.15.5:0/657939399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 0x561efa5a4b80 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000b3d0 tx=0x7f4e10004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e24d3de8f3c95f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:00.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4dfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:00.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4dfbfff640 1 -- 172.21.15.5:0/657939399 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10043060 con 0x561efa592a30 2024-09-19T06:13:00.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4dfbfff640 1 -- 172.21.15.5:0/657939399 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8550+0+0 (secure 0 0 0) 0x7f4e100055f0 con 0x561efa592a30 2024-09-19T06:13:00.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4e250b1740 1 -- 172.21.15.5:0/657939399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e10011440 con 0x561efa592a30 2024-09-19T06:13:00.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4dfbfff640 1 -- 172.21.15.5:0/657939399 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4e100057f0 con 0x561efa592a30 2024-09-19T06:13:00.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4e250b1740 10 client.17982 fetch_fsmap learned FSMap version 36 2024-09-19T06:13:00.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4e250b1740 1 -- 172.21.15.5:0/657939399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561efa61b230 con 0x561efa592a30 2024-09-19T06:13:00.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4dfbfff640 1 -- 172.21.15.5:0/657939399 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e10011440 con 0x561efa592a30 2024-09-19T06:13:00.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4dfbfff640 1 -- 172.21.15.5:0/657939399 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f4e10004030 con 0x561efa592a30 2024-09-19T06:13:00.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4e250b1740 10 client.17982 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:13:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4e250b1740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4e250b1740 20 client.17982 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4e250b1740 1 --2- 172.21.15.5:0/657939399 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561efa5a1680 0x561efa61d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4e250b1740 4 client.17982 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:13:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.378+0000 7f4e250b1740 1 -- 172.21.15.5:0/657939399 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561efa503d60 con 0x561efa5a1680 2024-09-19T06:13:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.379+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/657939399 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561efa5a1680 0x561efa61d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.379+0000 7f4dfbfff640 10 client.17982 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.379+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/657939399 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561efa5a1680 0x561efa61d980 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aefcd5e8f68ed109 server_cookie=37374981409c121 in_seq=0 out_seq=0 2024-09-19T06:13:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4dfbfff640 1 -- 172.21.15.5:0/657939399 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561efa503d60 con 0x561efa5a1680 2024-09-19T06:13:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4dfbfff640 10 client.17982 handle_command_reply: tid=0 2024-09-19T06:13:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4e250b1740 1 client.17982 shutdown 2024-09-19T06:13:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4e250b1740 1 -- 172.21.15.5:0/657939399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 msgr2=0x561efa5a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:00.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4e250b1740 1 --2- 172.21.15.5:0/657939399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 0x561efa5a4b80 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000b3d0 tx=0x7f4e10004f80 comp rx=0 tx=0).stop 2024-09-19T06:13:00.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4e250b1740 1 -- 172.21.15.5:0/657939399 shutdown_connections 2024-09-19T06:13:00.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4e250b1740 1 --2- 172.21.15.5:0/657939399 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561efa5a1680 0x561efa61d980 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4e250b1740 1 --2- 172.21.15.5:0/657939399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561efa596dc0 0x561efa619b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4e250b1740 1 --2- 172.21.15.5:0/657939399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561efa5964a0 0x561efa5a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4e250b1740 1 --2- 172.21.15.5:0/657939399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561efa592a30 0x561efa5a4b80 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4e250b1740 1 -- 172.21.15.5:0/657939399 >> 172.21.15.5:0/657939399 conn(0x561efa59e8d0 msgr2=0x561efa591350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:00.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4dfbfff640 0 client.17982 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:00.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4e250b1740 1 -- 172.21.15.5:0/657939399 shutdown_connections 2024-09-19T06:13:00.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4e250b1740 1 -- 172.21.15.5:0/657939399 wait complete. 2024-09-19T06:13:00.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.380+0000 7f4e250b1740 20 client.17982 trim_cache size 0 max 16384 2024-09-19T06:13:00.389 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:13:00.389 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:13:00.389 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:13:00.389 INFO:teuthology.orchestra.run.smithi005.stdout: "854d460e-5137-4fc8-b5f5-7abf169e5857": { 2024-09-19T06:13:00.389 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:13:00.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "854d460e-5137-4fc8-b5f5-7abf169e5857", 2024-09-19T06:13:00.389 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:13:00.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:13:00.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:13:00.389 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:13:00.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.381+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1958155475 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4e0c0a33b0 msgr2=0x7f4e0c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:00.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.381+0000 7f4e22cd8640 1 --2- 172.21.15.5:0/1958155475 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4e0c0a33b0 0x7f4e0c0a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009900 tx=0x7f4e08032b00 comp rx=0 tx=0).stop 2024-09-19T06:13:00.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.381+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1958155475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c106a80 msgr2=0x7f4e1c19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:00.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.381+0000 7f4e22cd8640 1 --2- 172.21.15.5:0/1958155475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c106a80 0x7f4e1c19ee80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009fd0 tx=0x7f4e1400b260 comp rx=0 tx=0).stop 2024-09-19T06:13:00.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.381+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1958155475 shutdown_connections 2024-09-19T06:13:00.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.381+0000 7f4e22cd8640 1 --2- 172.21.15.5:0/1958155475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c106a80 0x7f4e1c19ee80 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.381+0000 7f4e22cd8640 1 --2- 172.21.15.5:0/1958155475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c101420 0x7f4e1c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.381+0000 7f4e22cd8640 1 --2- 172.21.15.5:0/1958155475 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4e0c0a33b0 0x7f4e0c0a5870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.381+0000 7f4e22cd8640 1 --2- 172.21.15.5:0/1958155475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e1c100a80 0x7f4e1c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:00.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.381+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1958155475 >> 172.21.15.5:0/1958155475 conn(0x7f4e1c0fc7a0 msgr2=0x7f4e1c10c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:00.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.381+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1958155475 shutdown_connections 2024-09-19T06:13:00.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:00.381+0000 7f4e22cd8640 1 -- 172.21.15.5:0/1958155475 wait complete. 2024-09-19T06:13:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: pgmap v299: 97 pgs: 97 active+clean; 135 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 826 KiB/s wr, 82 op/s 2024-09-19T06:13:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:00.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:00.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:00.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:00.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:00.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:00 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:13:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: pgmap v299: 97 pgs: 97 active+clean; 135 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 826 KiB/s wr, 82 op/s 2024-09-19T06:13:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:00.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:00.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:00.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:00.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:00.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:00.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:00 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:13:00.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241848 2.51 MB/sec execute 386 sec latency 110.823 ms 2024-09-19T06:13:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: pgmap v299: 97 pgs: 97 active+clean; 135 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 826 KiB/s wr, 82 op/s 2024-09-19T06:13:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:00.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:00 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:13:01.100 INFO:tasks.cephfs.filesystem:scrub status for tag:854d460e-5137-4fc8-b5f5-7abf169e5857 - {'path': '/', 'tag': '854d460e-5137-4fc8-b5f5-7abf169e5857', 'options': 'recursive,force'} 2024-09-19T06:13:01.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241848 2.50 MB/sec execute 387 sec latency 1110.917 ms 2024-09-19T06:13:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:02 smithi157 ceph-mon[35346]: pgmap v300: 97 pgs: 97 active+clean; 135 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 826 KiB/s wr, 82 op/s 2024-09-19T06:13:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:02 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:13:02.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241848 2.50 MB/sec execute 388 sec latency 2110.999 ms 2024-09-19T06:13:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:02 smithi005 ceph-mon[30955]: pgmap v300: 97 pgs: 97 active+clean; 135 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 826 KiB/s wr, 82 op/s 2024-09-19T06:13:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:02 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:13:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:02 smithi049 ceph-mon[34640]: pgmap v300: 97 pgs: 97 active+clean; 135 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 826 KiB/s wr, 82 op/s 2024-09-19T06:13:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:02 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:13:03.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241848 2.49 MB/sec execute 389 sec latency 3111.048 ms 2024-09-19T06:13:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:04 smithi157 ceph-mon[35346]: pgmap v301: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-09-19T06:13:04.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241848 2.48 MB/sec execute 390 sec latency 4111.146 ms 2024-09-19T06:13:04.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:04 smithi005 ceph-mon[30955]: pgmap v301: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-09-19T06:13:04.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:04 smithi049 ceph-mon[34640]: pgmap v301: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-09-19T06:13:05.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 242024 2.48 MB/sec execute 391 sec latency 4747.146 ms 2024-09-19T06:13:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:06 smithi157 ceph-mon[35346]: pgmap v302: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 450 KiB/s wr, 65 op/s 2024-09-19T06:13:06.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 242522 2.48 MB/sec execute 392 sec latency 13.031 ms 2024-09-19T06:13:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:06 smithi005 ceph-mon[30955]: pgmap v302: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 450 KiB/s wr, 65 op/s 2024-09-19T06:13:06.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:06 smithi049 ceph-mon[34640]: pgmap v302: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 450 KiB/s wr, 65 op/s 2024-09-19T06:13:07.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 243060 2.48 MB/sec execute 393 sec latency 23.235 ms 2024-09-19T06:13:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:08 smithi157 ceph-mon[35346]: pgmap v303: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 705 KiB/s wr, 106 op/s 2024-09-19T06:13:08.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 243624 2.47 MB/sec execute 394 sec latency 21.963 ms 2024-09-19T06:13:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:08 smithi005 ceph-mon[30955]: pgmap v303: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 705 KiB/s wr, 106 op/s 2024-09-19T06:13:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:08 smithi049 ceph-mon[34640]: pgmap v303: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 705 KiB/s wr, 106 op/s 2024-09-19T06:13:09.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:09 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:13:09.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244225 2.48 MB/sec execute 395 sec latency 19.253 ms 2024-09-19T06:13:09.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:09 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:13:09.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:09 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:13:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.719+0000 7fde50db7640 1 -- 172.21.15.5:0/4060624322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c101340 msgr2=0x7fde4c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.719+0000 7fde50db7640 1 --2- 172.21.15.5:0/4060624322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c101340 0x7fde4c10d870 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fde44009cd0 tx=0x7fde4402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.720+0000 7fde50db7640 1 -- 172.21.15.5:0/4060624322 shutdown_connections 2024-09-19T06:13:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.720+0000 7fde50db7640 1 --2- 172.21.15.5:0/4060624322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c101340 0x7fde4c10d870 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.720+0000 7fde50db7640 1 --2- 172.21.15.5:0/4060624322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c1009a0 0x7fde4c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.720+0000 7fde50db7640 1 --2- 172.21.15.5:0/4060624322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde4c1069b0 0x7fde4c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.720+0000 7fde50db7640 1 -- 172.21.15.5:0/4060624322 >> 172.21.15.5:0/4060624322 conn(0x7fde4c0779a0 msgr2=0x7fde4c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.720+0000 7fde50db7640 1 -- 172.21.15.5:0/4060624322 shutdown_connections 2024-09-19T06:13:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.720+0000 7fde50db7640 1 -- 172.21.15.5:0/4060624322 wait complete. 2024-09-19T06:13:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.720+0000 7fde50db7640 1 Processor -- start 2024-09-19T06:13:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.720+0000 7fde50db7640 1 -- start start 2024-09-19T06:13:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde50db7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde4c1009a0 0x7fde4c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde50db7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c101340 0x7fde4c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde50db7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c1069b0 0x7fde4c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde50db7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c110ca0 con 0x7fde4c1069b0 2024-09-19T06:13:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde50db7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c110b20 con 0x7fde4c1009a0 2024-09-19T06:13:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde50db7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c110e20 con 0x7fde4c101340 2024-09-19T06:13:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde4ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c1069b0 0x7fde4c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde49d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c101340 0x7fde4c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde49d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c101340 0x7fde4c198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48522/0 (socket says 172.21.15.5:48522) 2024-09-19T06:13:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde49d9b640 1 -- 172.21.15.5:0/2347104652 learned_addr learned my addr 172.21.15.5:0/2347104652 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde4ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c1069b0 0x7fde4c19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51500/0 (socket says 172.21.15.5:51500) 2024-09-19T06:13:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde4a59c640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde4c1009a0 0x7fde4c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde4ad9d640 1 -- 172.21.15.5:0/2347104652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c101340 msgr2=0x7fde4c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde4ad9d640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c101340 0x7fde4c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde4ad9d640 1 -- 172.21.15.5:0/2347104652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde4c1009a0 msgr2=0x7fde4c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde4ad9d640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde4c1009a0 0x7fde4c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde4ad9d640 1 -- 172.21.15.5:0/2347104652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c19d2c0 con 0x7fde4c1069b0 2024-09-19T06:13:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde49d9b640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c101340 0x7fde4c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.721+0000 7fde4ad9d640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c1069b0 0x7fde4c19cb90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fde4402ae00 tx=0x7fde44031dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.722+0000 7fde3b7fe640 1 -- 172.21.15.5:0/2347104652 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde44004040 con 0x7fde4c1069b0 2024-09-19T06:13:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.722+0000 7fde3b7fe640 1 -- 172.21.15.5:0/2347104652 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde44004690 con 0x7fde4c1069b0 2024-09-19T06:13:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.722+0000 7fde50db7640 1 -- 172.21.15.5:0/2347104652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4c19d550 con 0x7fde4c1069b0 2024-09-19T06:13:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.722+0000 7fde3b7fe640 1 -- 172.21.15.5:0/2347104652 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde44030d30 con 0x7fde4c1069b0 2024-09-19T06:13:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.722+0000 7fde50db7640 1 -- 172.21.15.5:0/2347104652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4c1a4e40 con 0x7fde4c1069b0 2024-09-19T06:13:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde3b7fe640 1 -- 172.21.15.5:0/2347104652 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fde440305a0 con 0x7fde4c1069b0 2024-09-19T06:13:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde53c2b740 1 Processor -- start 2024-09-19T06:13:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde53c2b740 1 -- start start 2024-09-19T06:13:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde53c2b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee7af0 0x55eeb4ee7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde53c2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde53c2b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4ef50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde53c2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eeb4e4c2f0 con 0x55eeb4ee8410 2024-09-19T06:13:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde53c2b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eeb4df4270 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde53c2b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eeb4e482b0 con 0x55eeb4ee7af0 2024-09-19T06:13:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde49d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee7af0 0x55eeb4ee7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde49d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee7af0 0x55eeb4ee7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48528/0 (socket says 172.21.15.5:48528) 2024-09-19T06:13:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde49d9b640 1 -- 172.21.15.5:0/2744235264 learned_addr learned my addr 172.21.15.5:0/2744235264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51516/0 (socket says 172.21.15.5:51516) 2024-09-19T06:13:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.725+0000 7fde49d9b640 1 --2- 172.21.15.5:0/2744235264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4ef50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde3b7fe640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fde340a3430 0x7fde340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde4a59c640 1 -- 172.21.15.5:0/2744235264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee7af0 msgr2=0x55eeb4ee7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde4a59c640 1 --2- 172.21.15.5:0/2744235264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee7af0 0x55eeb4ee7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde4a59c640 1 -- 172.21.15.5:0/2744235264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ef2cb0 msgr2=0x55eeb4ef50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde4a59c640 1 --2- 172.21.15.5:0/2744235264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4ef50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde4a59c640 1 -- 172.21.15.5:0/2744235264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeb4df3340 con 0x55eeb4ee8410 2024-09-19T06:13:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde49d9b640 1 --2- 172.21.15.5:0/2744235264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee7af0 0x55eeb4ee7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde4ad9d640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fde340a3430 0x7fde340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde3b7fe640 1 -- 172.21.15.5:0/2347104652 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8594+0+0 (secure 0 0 0) 0x7fde4402e030 con 0x7fde4c1069b0 2024-09-19T06:13:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde4a59c640 1 --2- 172.21.15.5:0/2744235264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef2770 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009c20 tx=0x7fde3c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e28529939f63a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde38ff9640 1 -- 172.21.15.5:0/2744235264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c004050 con 0x55eeb4ee8410 2024-09-19T06:13:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde38ff9640 1 -- 172.21.15.5:0/2744235264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde3c0054d0 con 0x55eeb4ee8410 2024-09-19T06:13:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.726+0000 7fde38ff9640 1 -- 172.21.15.5:0/2744235264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c004c60 con 0x55eeb4ee8410 2024-09-19T06:13:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.727+0000 7fde53c2b740 1 -- 172.21.15.5:0/2744235264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee8410 msgr2=0x55eeb4ef2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.727+0000 7fde53c2b740 1 --2- 172.21.15.5:0/2744235264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef2770 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009c20 tx=0x7fde3c011810 comp rx=0 tx=0).stop 2024-09-19T06:13:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.727+0000 7fde4ad9d640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fde340a3430 0x7fde340a58f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde440ee800 tx=0x7fde4403d040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.727+0000 7fde53c2b740 1 -- 172.21.15.5:0/2744235264 shutdown_connections 2024-09-19T06:13:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.727+0000 7fde53c2b740 1 --2- 172.21.15.5:0/2744235264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4ef50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.727+0000 7fde53c2b740 1 --2- 172.21.15.5:0/2744235264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef2770 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.727+0000 7fde53c2b740 1 --2- 172.21.15.5:0/2744235264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee7af0 0x55eeb4ee7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.727+0000 7fde53c2b740 1 -- 172.21.15.5:0/2744235264 >> 172.21.15.5:0/2744235264 conn(0x55eeb4ee4060 msgr2=0x55eeb4ee4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.727+0000 7fde53c2b740 1 -- 172.21.15.5:0/2744235264 shutdown_connections 2024-09-19T06:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.727+0000 7fde53c2b740 1 -- 172.21.15.5:0/2744235264 wait complete. 2024-09-19T06:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.727+0000 7fde53c2b740 1 -- start start 2024-09-19T06:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde53c2b740 1 Processor -- start 2024-09-19T06:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde53c2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee7af0 0x55eeb4eefbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde53c2b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde53c2b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4ef0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde53c2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eeb4ef5880 con 0x55eeb4ee7af0 2024-09-19T06:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde53c2b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eeb4dd5290 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde53c2b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eeb4ef5a00 con 0x55eeb4ee8410 2024-09-19T06:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde53c2b740 10 client.0 _ll_register_callbacks cb 0x55eeb4e06140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde49d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee7af0 0x55eeb4eefbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde49d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee7af0 0x55eeb4eefbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51532/0 (socket says 172.21.15.5:51532) 2024-09-19T06:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde49d9b640 1 -- 172.21.15.5:0/1169207820 learned_addr learned my addr 172.21.15.5:0/1169207820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde4a59c640 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde49d9b640 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4ef0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde49d9b640 1 -- 172.21.15.5:0/1169207820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee8410 msgr2=0x55eeb4ef00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde49d9b640 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde49d9b640 1 -- 172.21.15.5:0/1169207820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ef2cb0 msgr2=0x55eeb4ef0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde49d9b640 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4ef0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde49d9b640 1 -- 172.21.15.5:0/1169207820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeb4b5f210 con 0x55eeb4ee7af0 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde49d9b640 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee7af0 0x55eeb4eefbb0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fde40009630 tx=0x7fde40005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d80d4e0d446e21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.728+0000 7fde31ffb640 1 -- 172.21.15.5:0/1169207820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40005020 con 0x55eeb4ee7af0 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde31ffb640 1 -- 172.21.15.5:0/1169207820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8594+0+0 (secure 0 0 0) 0x7fde400051c0 con 0x55eeb4ee7af0 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde53c2b740 1 -- 172.21.15.5:0/1169207820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde3c0054d0 con 0x55eeb4ee7af0 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde31ffb640 1 -- 172.21.15.5:0/1169207820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fde400053c0 con 0x55eeb4ee7af0 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde53c2b740 10 client.17994 fetch_fsmap learned FSMap version 37 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde53c2b740 1 -- 172.21.15.5:0/1169207820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eeb4c2b620 con 0x55eeb4ee7af0 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde31ffb640 1 -- 172.21.15.5:0/1169207820 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde3c0054d0 con 0x55eeb4ee7af0 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde31ffb640 1 -- 172.21.15.5:0/1169207820 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) ==== 8378+0+0 (secure 0 0 0) 0x7fde4001e5c0 con 0x55eeb4ee7af0 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde53c2b740 10 client.17994 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T06:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde53c2b740 10 client.17994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde53c2b740 20 client.17994 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde53c2b740 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb4f02c50 0x55eeb4f05060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde53c2b740 4 client.17994 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde53c2b740 1 -- 172.21.15.5:0/1169207820 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eeb4e59a20 con 0x55eeb4f02c50 2024-09-19T06:13:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.729+0000 7fde4a59c640 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb4f02c50 0x55eeb4f05060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.730+0000 7fde4a59c640 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb4f02c50 0x55eeb4f05060 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370c0feb15a4a3f server_cookie=cad3566a4652992d in_seq=0 out_seq=0 2024-09-19T06:13:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.730+0000 7fde31ffb640 10 client.17994 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.730+0000 7fde31ffb640 1 -- 172.21.15.5:0/1169207820 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eeb4e59a20 con 0x55eeb4f02c50 2024-09-19T06:13:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.730+0000 7fde31ffb640 10 client.17994 handle_command_reply: tid=0 2024-09-19T06:13:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.730+0000 7fde53c2b740 1 client.17994 shutdown 2024-09-19T06:13:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.730+0000 7fde53c2b740 1 -- 172.21.15.5:0/1169207820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee7af0 msgr2=0x55eeb4eefbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.730+0000 7fde53c2b740 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee7af0 0x55eeb4eefbb0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fde40009630 tx=0x7fde40005f90 comp rx=0 tx=0).stop 2024-09-19T06:13:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.730+0000 7fde53c2b740 1 -- 172.21.15.5:0/1169207820 shutdown_connections 2024-09-19T06:13:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.731+0000 7fde53c2b740 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb4f02c50 0x55eeb4f05060 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.731+0000 7fde53c2b740 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4ef0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.731+0000 7fde53c2b740 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.731+0000 7fde31ffb640 0 client.17994 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.731+0000 7fde53c2b740 1 --2- 172.21.15.5:0/1169207820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ee7af0 0x55eeb4eefbb0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.731+0000 7fde53c2b740 1 -- 172.21.15.5:0/1169207820 >> 172.21.15.5:0/1169207820 conn(0x55eeb4ee4060 msgr2=0x55eeb4ee9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.731+0000 7fde53c2b740 1 -- 172.21.15.5:0/1169207820 shutdown_connections 2024-09-19T06:13:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.731+0000 7fde53c2b740 1 -- 172.21.15.5:0/1169207820 wait complete. 2024-09-19T06:13:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.731+0000 7fde53c2b740 20 client.17994 trim_cache size 0 max 16384 2024-09-19T06:13:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.747+0000 7fde53c2b740 1 Processor -- start 2024-09-19T06:13:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.747+0000 7fde53c2b740 1 -- start start 2024-09-19T06:13:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.747+0000 7fde53c2b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee7af0 0x55eeb4eea560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.747+0000 7fde53c2b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.747+0000 7fde53c2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4f0a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.747+0000 7fde53c2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c1ad820 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.747+0000 7fde53c2b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c1af3f0 con 0x55eeb4ee8410 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.747+0000 7fde53c2b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c1adef0 con 0x55eeb4ee7af0 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde49d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4f0a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde49d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4f0a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51538/0 (socket says 172.21.15.5:51538) 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde49d9b640 1 -- 172.21.15.5:0/1091461473 learned_addr learned my addr 172.21.15.5:0/1091461473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde4a59c640 1 --2- 172.21.15.5:0/1091461473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde49d9b640 1 --2- 172.21.15.5:0/1091461473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee7af0 0x55eeb4eea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde49d9b640 1 -- 172.21.15.5:0/1091461473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee7af0 msgr2=0x55eeb4eea560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde49d9b640 1 --2- 172.21.15.5:0/1091461473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee7af0 0x55eeb4eea560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde49d9b640 1 -- 172.21.15.5:0/1091461473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ee8410 msgr2=0x55eeb4ef0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde49d9b640 1 --2- 172.21.15.5:0/1091461473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde49d9b640 1 -- 172.21.15.5:0/1091461473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeb4ee7920 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde4a59c640 1 --2- 172.21.15.5:0/1091461473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde49d9b640 1 --2- 172.21.15.5:0/1091461473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4f0a970 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fde4000fd20 tx=0x7fde40001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32fea69d0d0c5a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde33fff640 1 -- 172.21.15.5:0/1091461473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4001ec00 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde33fff640 1 -- 172.21.15.5:0/1091461473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde4001eda0 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.748+0000 7fde33fff640 1 -- 172.21.15.5:0/1091461473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40028af0 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.749+0000 7fde53c2b740 1 -- 172.21.15.5:0/1091461473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 msgr2=0x55eeb4f0a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.749+0000 7fde53c2b740 1 --2- 172.21.15.5:0/1091461473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4f0a970 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fde4000fd20 tx=0x7fde40001e70 comp rx=0 tx=0).stop 2024-09-19T06:13:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.749+0000 7fde53c2b740 1 -- 172.21.15.5:0/1091461473 shutdown_connections 2024-09-19T06:13:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.749+0000 7fde53c2b740 1 --2- 172.21.15.5:0/1091461473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4f0a970 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.749+0000 7fde53c2b740 1 --2- 172.21.15.5:0/1091461473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ee8410 0x55eeb4ef0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.749+0000 7fde53c2b740 1 --2- 172.21.15.5:0/1091461473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee7af0 0x55eeb4eea560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.749+0000 7fde53c2b740 1 -- 172.21.15.5:0/1091461473 >> 172.21.15.5:0/1091461473 conn(0x55eeb4ee4060 msgr2=0x55eeb4ef1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.749+0000 7fde53c2b740 1 -- 172.21.15.5:0/1091461473 shutdown_connections 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.749+0000 7fde53c2b740 1 -- 172.21.15.5:0/1091461473 wait complete. 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.749+0000 7fde53c2b740 1 -- start start 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde53c2b740 1 Processor -- start 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde53c2b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ee7af0 0x55eeb4efaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde53c2b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee8410 0x55eeb4efb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde53c2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4eec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde53c2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c1aed20 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde53c2b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c1ae5f0 con 0x55eeb4ee7af0 2024-09-19T06:13:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde53c2b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c1ab820 con 0x55eeb4ee8410 2024-09-19T06:13:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde53c2b740 10 client.0 _ll_register_callbacks cb 0x7fde40011be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde49d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4eec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde49d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4eec3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51552/0 (socket says 172.21.15.5:51552) 2024-09-19T06:13:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde49d9b640 1 -- 172.21.15.5:0/2462651306 learned_addr learned my addr 172.21.15.5:0/2462651306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde49d9b640 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ee7af0 0x55eeb4efaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde4a59c640 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee8410 0x55eeb4efb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde49d9b640 1 -- 172.21.15.5:0/2462651306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee8410 msgr2=0x55eeb4efb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde49d9b640 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee8410 0x55eeb4efb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde49d9b640 1 -- 172.21.15.5:0/2462651306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ee7af0 msgr2=0x55eeb4efaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde49d9b640 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ee7af0 0x55eeb4efaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde49d9b640 1 -- 172.21.15.5:0/2462651306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeb4eecbc0 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.750+0000 7fde49d9b640 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4eec3c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x55eeb4f0b5d0 tx=0x7fde400097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278be91206b64111 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde31ffb640 1 -- 172.21.15.5:0/2462651306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40028770 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde31ffb640 1 -- 172.21.15.5:0/2462651306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8594+0+0 (secure 0 0 0) 0x7fde40028910 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde53c2b740 1 -- 172.21.15.5:0/2462651306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde4001eda0 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde31ffb640 1 -- 172.21.15.5:0/2462651306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fde40009e10 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde53c2b740 10 client.18000 fetch_fsmap learned FSMap version 37 2024-09-19T06:13:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde53c2b740 1 -- 172.21.15.5:0/2462651306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eeb4f6d3d0 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde31ffb640 1 -- 172.21.15.5:0/2462651306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde4001eda0 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde31ffb640 1 -- 172.21.15.5:0/2462651306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) ==== 8378+0+0 (secure 0 0 0) 0x7fde40034bb0 con 0x55eeb4ef2cb0 2024-09-19T06:13:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde53c2b740 10 client.18000 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T06:13:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde53c2b740 10 client.18000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde53c2b740 20 client.18000 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde53c2b740 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb4f6da30 0x55eeb4f6fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde53c2b740 4 client.18000 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:13:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.751+0000 7fde53c2b740 1 -- 172.21.15.5:0/2462651306 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eeb4e53d70 con 0x55eeb4f6da30 2024-09-19T06:13:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.752+0000 7fde4a59c640 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb4f6da30 0x55eeb4f6fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.752+0000 7fde4a59c640 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb4f6da30 0x55eeb4f6fe50 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8549eb495b00e89 server_cookie=add164ef65307b37 in_seq=0 out_seq=0 2024-09-19T06:13:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.752+0000 7fde31ffb640 10 client.18000 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.775+0000 7fde31ffb640 1 -- 172.21.15.5:0/2462651306 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eeb4e53d70 con 0x55eeb4f6da30 2024-09-19T06:13:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.775+0000 7fde31ffb640 10 client.18000 handle_command_reply: tid=0 2024-09-19T06:13:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.775+0000 7fde53c2b740 1 client.18000 shutdown 2024-09-19T06:13:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.775+0000 7fde53c2b740 1 -- 172.21.15.5:0/2462651306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 msgr2=0x55eeb4eec3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.775+0000 7fde53c2b740 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4eec3c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x55eeb4f0b5d0 tx=0x7fde400097a0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.775+0000 7fde53c2b740 1 -- 172.21.15.5:0/2462651306 shutdown_connections 2024-09-19T06:13:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.775+0000 7fde53c2b740 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb4f6da30 0x55eeb4f6fe50 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.775+0000 7fde53c2b740 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb4ef2cb0 0x55eeb4eec3c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.775+0000 7fde53c2b740 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb4ee8410 0x55eeb4efb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.775+0000 7fde31ffb640 0 client.18000 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.775+0000 7fde53c2b740 1 --2- 172.21.15.5:0/2462651306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb4ee7af0 0x55eeb4efaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.775+0000 7fde53c2b740 1 -- 172.21.15.5:0/2462651306 >> 172.21.15.5:0/2462651306 conn(0x55eeb4ee4060 msgr2=0x55eeb4ee29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde53c2b740 1 -- 172.21.15.5:0/2462651306 shutdown_connections 2024-09-19T06:13:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde53c2b740 1 -- 172.21.15.5:0/2462651306 wait complete. 2024-09-19T06:13:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde53c2b740 20 client.18000 trim_cache size 0 max 16384 2024-09-19T06:13:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:13:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:13:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:13:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:13:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde50db7640 1 -- 172.21.15.5:0/2347104652 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fde340a3430 msgr2=0x7fde340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde50db7640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fde340a3430 0x7fde340a58f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde440ee800 tx=0x7fde4403d040 comp rx=0 tx=0).stop 2024-09-19T06:13:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde50db7640 1 -- 172.21.15.5:0/2347104652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c1069b0 msgr2=0x7fde4c19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde50db7640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c1069b0 0x7fde4c19cb90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fde4402ae00 tx=0x7fde44031dd0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde50db7640 1 -- 172.21.15.5:0/2347104652 shutdown_connections 2024-09-19T06:13:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde50db7640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c1069b0 0x7fde4c19cb90 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde50db7640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fde340a3430 0x7fde340a58f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde50db7640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c101340 0x7fde4c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde50db7640 1 --2- 172.21.15.5:0/2347104652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde4c1009a0 0x7fde4c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde50db7640 1 -- 172.21.15.5:0/2347104652 >> 172.21.15.5:0/2347104652 conn(0x7fde4c0779a0 msgr2=0x7fde4c069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde50db7640 1 -- 172.21.15.5:0/2347104652 shutdown_connections 2024-09-19T06:13:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:10.776+0000 7fde50db7640 1 -- 172.21.15.5:0/2347104652 wait complete. 2024-09-19T06:13:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:10 smithi157 ceph-mon[35346]: pgmap v304: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 511 KiB/s wr, 80 op/s 2024-09-19T06:13:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:10 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:13:10.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:10 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:13:10.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:10 smithi157 ceph-mon[35346]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T06:13:10.838 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244633 2.48 MB/sec execute 396 sec latency 17.553 ms 2024-09-19T06:13:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:10 smithi005 ceph-mon[30955]: pgmap v304: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 511 KiB/s wr, 80 op/s 2024-09-19T06:13:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:10 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:13:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:10 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:13:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:10 smithi005 ceph-mon[30955]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T06:13:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:10 smithi049 ceph-mon[34640]: pgmap v304: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 511 KiB/s wr, 80 op/s 2024-09-19T06:13:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:10 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:13:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:10 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:13:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:10 smithi049 ceph-mon[34640]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T06:13:11.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244855 2.48 MB/sec execute 397 sec latency 733.400 ms 2024-09-19T06:13:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:12 smithi157 ceph-mon[35346]: pgmap v306: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 614 KiB/s wr, 96 op/s 2024-09-19T06:13:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:12 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:13:12.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244855 2.48 MB/sec execute 398 sec latency 1733.489 ms 2024-09-19T06:13:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:12 smithi005 ceph-mon[30955]: pgmap v306: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 614 KiB/s wr, 96 op/s 2024-09-19T06:13:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:12 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:13:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:12 smithi049 ceph-mon[34640]: pgmap v306: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 614 KiB/s wr, 96 op/s 2024-09-19T06:13:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:12 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:13:13.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244855 2.47 MB/sec execute 399 sec latency 2733.597 ms 2024-09-19T06:13:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:14 smithi157 ceph-mon[35346]: pgmap v307: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-09-19T06:13:14.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244855 2.46 MB/sec execute 400 sec latency 3733.722 ms 2024-09-19T06:13:14.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:14 smithi005 ceph-mon[30955]: pgmap v307: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-09-19T06:13:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:14 smithi049 ceph-mon[34640]: pgmap v307: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-09-19T06:13:15.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244922 2.46 MB/sec execute 401 sec latency 4365.220 ms 2024-09-19T06:13:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:16 smithi157 ceph-mon[35346]: pgmap v308: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-09-19T06:13:16.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244922 2.45 MB/sec execute 402 sec latency 1111.554 ms 2024-09-19T06:13:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:16 smithi005 ceph-mon[30955]: pgmap v308: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-09-19T06:13:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:16 smithi049 ceph-mon[34640]: pgmap v308: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-09-19T06:13:17.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244922 2.45 MB/sec execute 403 sec latency 2111.628 ms 2024-09-19T06:13:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:18 smithi157 ceph-mon[35346]: pgmap v309: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 767 KiB/s wr, 74 op/s 2024-09-19T06:13:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:18 smithi157 ceph-mon[35346]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T06:13:18.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244922 2.44 MB/sec execute 404 sec latency 3111.729 ms 2024-09-19T06:13:18.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:18 smithi005 ceph-mon[30955]: pgmap v309: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 767 KiB/s wr, 74 op/s 2024-09-19T06:13:18.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:18 smithi005 ceph-mon[30955]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T06:13:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:18 smithi049 ceph-mon[34640]: pgmap v309: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 767 KiB/s wr, 74 op/s 2024-09-19T06:13:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:18 smithi049 ceph-mon[34640]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T06:13:19.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244922 2.43 MB/sec execute 405 sec latency 4111.822 ms 2024-09-19T06:13:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:20 smithi157 ceph-mon[35346]: pgmap v311: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 825 KiB/s wr, 80 op/s 2024-09-19T06:13:20.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 245028 2.43 MB/sec execute 406 sec latency 4737.048 ms 2024-09-19T06:13:20.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:20 smithi005 ceph-mon[30955]: pgmap v311: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 825 KiB/s wr, 80 op/s 2024-09-19T06:13:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:20 smithi049 ceph-mon[34640]: pgmap v311: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 825 KiB/s wr, 80 op/s 2024-09-19T06:13:21.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 245264 2.43 MB/sec execute 407 sec latency 13.128 ms 2024-09-19T06:13:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:22 smithi157 ceph-mon[35346]: pgmap v312: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 767 KiB/s wr, 74 op/s 2024-09-19T06:13:22.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 245695 2.42 MB/sec execute 408 sec latency 18.761 ms 2024-09-19T06:13:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:22 smithi005 ceph-mon[30955]: pgmap v312: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 767 KiB/s wr, 74 op/s 2024-09-19T06:13:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:22 smithi049 ceph-mon[34640]: pgmap v312: 97 pgs: 97 active+clean; 132 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 767 KiB/s wr, 74 op/s 2024-09-19T06:13:23.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 246234 2.43 MB/sec execute 409 sec latency 19.562 ms 2024-09-19T06:13:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.869+0000 7f58a408d640 1 -- 172.21.15.5:0/4212507116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c102c80 msgr2=0x7f589c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.869+0000 7f58a408d640 1 --2- 172.21.15.5:0/4212507116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c102c80 0x7f589c103060 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f5898009ca0 tx=0x7f589802a950 comp rx=0 tx=0).stop 2024-09-19T06:13:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.869+0000 7f58a408d640 1 -- 172.21.15.5:0/4212507116 shutdown_connections 2024-09-19T06:13:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.869+0000 7f58a408d640 1 --2- 172.21.15.5:0/4212507116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f589c10c0a0 0x7f589c10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.869+0000 7f58a408d640 1 --2- 172.21.15.5:0/4212507116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f589c1035a0 0x7f589c10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.869+0000 7f58a408d640 1 --2- 172.21.15.5:0/4212507116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c102c80 0x7f589c103060 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.869+0000 7f58a408d640 1 -- 172.21.15.5:0/4212507116 >> 172.21.15.5:0/4212507116 conn(0x7f589c0779a0 msgr2=0x7f589c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.869+0000 7f58a408d640 1 -- 172.21.15.5:0/4212507116 shutdown_connections 2024-09-19T06:13:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.869+0000 7f58a408d640 1 -- 172.21.15.5:0/4212507116 wait complete. 2024-09-19T06:13:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a408d640 1 Processor -- start 2024-09-19T06:13:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a408d640 1 -- start start 2024-09-19T06:13:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a408d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f589c102c80 0x7f589c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a408d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f589c1035a0 0x7f589c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a408d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c10c0a0 0x7f589c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a408d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f589c111d50 con 0x7f589c10c0a0 2024-09-19T06:13:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a408d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f589c111bd0 con 0x7f589c102c80 2024-09-19T06:13:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a408d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f589c111ed0 con 0x7f589c1035a0 2024-09-19T06:13:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a262a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c10c0a0 0x7f589c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a1e29640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f589c102c80 0x7f589c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a262a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c10c0a0 0x7f589c19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60578/0 (socket says 172.21.15.5:60578) 2024-09-19T06:13:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a262a640 1 -- 172.21.15.5:0/826423281 learned_addr learned my addr 172.21.15.5:0/826423281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.870+0000 7f58a1628640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f589c1035a0 0x7f589c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.871+0000 7f58a262a640 1 -- 172.21.15.5:0/826423281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f589c1035a0 msgr2=0x7f589c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.871+0000 7f58a262a640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f589c1035a0 0x7f589c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.871+0000 7f58a262a640 1 -- 172.21.15.5:0/826423281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f589c102c80 msgr2=0x7f589c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.871+0000 7f58a262a640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f589c102c80 0x7f589c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.871+0000 7f58a262a640 1 -- 172.21.15.5:0/826423281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c19d310 con 0x7f589c10c0a0 2024-09-19T06:13:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.871+0000 7f58a1e29640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f589c102c80 0x7f589c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.871+0000 7f58a262a640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c10c0a0 0x7f589c19cbe0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f589400d750 tx=0x7f589400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.871+0000 7f5892ffd640 1 -- 172.21.15.5:0/826423281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894017070 con 0x7f589c10c0a0 2024-09-19T06:13:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.871+0000 7f58a408d640 1 -- 172.21.15.5:0/826423281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589c19d600 con 0x7f589c10c0a0 2024-09-19T06:13:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.871+0000 7f5892ffd640 1 -- 172.21.15.5:0/826423281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58940054d0 con 0x7f589c10c0a0 2024-09-19T06:13:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.871+0000 7f5892ffd640 1 -- 172.21.15.5:0/826423281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894004c60 con 0x7f589c10c0a0 2024-09-19T06:13:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.871+0000 7f58a408d640 1 -- 172.21.15.5:0/826423281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589c1a4e90 con 0x7f589c10c0a0 2024-09-19T06:13:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.874+0000 7f58a6e29740 1 Processor -- start 2024-09-19T06:13:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.874+0000 7f5892ffd640 1 -- 172.21.15.5:0/826423281 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5894004e00 con 0x7f589c10c0a0 2024-09-19T06:13:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.874+0000 7f58a6e29740 1 -- start start 2024-09-19T06:13:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.874+0000 7f58a6e29740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d31af0 0x560604d31ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.874+0000 7f58a6e29740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d32410 0x560604d3c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.874+0000 7f58a6e29740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d3ccb0 0x560604d3f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.874+0000 7f58a6e29740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560604c962f0 con 0x560604d32410 2024-09-19T06:13:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.874+0000 7f58a6e29740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560604c3e270 con 0x560604d3ccb0 2024-09-19T06:13:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.874+0000 7f58a6e29740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560604c922b0 con 0x560604d31af0 2024-09-19T06:13:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a1e29640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d32410 0x560604d3c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a1e29640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d32410 0x560604d3c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60592/0 (socket says 172.21.15.5:60592) 2024-09-19T06:13:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a1e29640 1 -- 172.21.15.5:0/3309559971 learned_addr learned my addr 172.21.15.5:0/3309559971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a1628640 1 --2- 172.21.15.5:0/3309559971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d3ccb0 0x560604d3f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a1628640 1 --2- 172.21.15.5:0/3309559971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d31af0 0x560604d31ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a1628640 1 -- 172.21.15.5:0/3309559971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d31af0 msgr2=0x560604d31ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f5892ffd640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f58880a33b0 0x7f58880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a1628640 1 --2- 172.21.15.5:0/3309559971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d31af0 0x560604d31ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a1628640 1 -- 172.21.15.5:0/3309559971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d32410 msgr2=0x560604d3c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a1628640 1 --2- 172.21.15.5:0/3309559971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d32410 0x560604d3c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a1628640 1 -- 172.21.15.5:0/3309559971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560604c3d340 con 0x560604d3ccb0 2024-09-19T06:13:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a1e29640 1 --2- 172.21.15.5:0/3309559971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d32410 0x560604d3c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a262a640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f58880a33b0 0x7f58880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f58a1628640 1 --2- 172.21.15.5:0/3309559971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d3ccb0 0x560604d3f0a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f589c10e5d0 tx=0x7f588c009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c05aa41ffd1f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f5892ffd640 1 -- 172.21.15.5:0/826423281 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f58940cf5d0 con 0x7f589c10c0a0 2024-09-19T06:13:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f5887fff640 1 -- 172.21.15.5:0/3309559971 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c004050 con 0x560604d3ccb0 2024-09-19T06:13:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f5887fff640 1 -- 172.21.15.5:0/3309559971 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f588c0054d0 con 0x560604d3ccb0 2024-09-19T06:13:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.875+0000 7f5887fff640 1 -- 172.21.15.5:0/3309559971 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c004ca0 con 0x560604d3ccb0 2024-09-19T06:13:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.876+0000 7f58a6e29740 1 -- 172.21.15.5:0/3309559971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d3ccb0 msgr2=0x560604d3f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.876+0000 7f58a6e29740 1 --2- 172.21.15.5:0/3309559971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d3ccb0 0x560604d3f0a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f589c10e5d0 tx=0x7f588c009770 comp rx=0 tx=0).stop 2024-09-19T06:13:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.876+0000 7f58a262a640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f58880a33b0 0x7f58880a5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f589409b640 tx=0x7f589406d6c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.876+0000 7f58a6e29740 1 -- 172.21.15.5:0/3309559971 shutdown_connections 2024-09-19T06:13:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.876+0000 7f58a6e29740 1 --2- 172.21.15.5:0/3309559971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d3ccb0 0x560604d3f0a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.876+0000 7f58a6e29740 1 --2- 172.21.15.5:0/3309559971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d32410 0x560604d3c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.876+0000 7f58a6e29740 1 --2- 172.21.15.5:0/3309559971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d31af0 0x560604d31ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.876+0000 7f58a6e29740 1 -- 172.21.15.5:0/3309559971 >> 172.21.15.5:0/3309559971 conn(0x560604d2e060 msgr2=0x560604d2e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.876+0000 7f58a6e29740 1 -- 172.21.15.5:0/3309559971 shutdown_connections 2024-09-19T06:13:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.876+0000 7f58a6e29740 1 -- 172.21.15.5:0/3309559971 wait complete. 2024-09-19T06:13:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a6e29740 1 -- start start 2024-09-19T06:13:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a6e29740 1 Processor -- start 2024-09-19T06:13:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a6e29740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d31af0 0x560604d39b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a6e29740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d32410 0x560604d3a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a6e29740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d3ccb0 0x560604d3a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a6e29740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560604d3f880 con 0x560604d31af0 2024-09-19T06:13:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a6e29740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560604c1f290 con 0x560604d32410 2024-09-19T06:13:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a6e29740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560604d3fa00 con 0x560604d3ccb0 2024-09-19T06:13:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a6e29740 10 client.0 _ll_register_callbacks cb 0x560604c50140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a1628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d31af0 0x560604d39b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a1628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d31af0 0x560604d39b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60596/0 (socket says 172.21.15.5:60596) 2024-09-19T06:13:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a1e29640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d32410 0x560604d3a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.877+0000 7f58a1628640 1 -- 172.21.15.5:0/3554486162 learned_addr learned my addr 172.21.15.5:0/3554486162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58a1628640 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d3ccb0 0x560604d3a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58a1e29640 1 -- 172.21.15.5:0/3554486162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d3ccb0 msgr2=0x560604d3a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58a1e29640 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d3ccb0 0x560604d3a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58a1e29640 1 -- 172.21.15.5:0/3554486162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d31af0 msgr2=0x560604d39b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58a1e29640 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d31af0 0x560604d39b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58a1e29640 1 -- 172.21.15.5:0/3554486162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606049a9210 con 0x560604d32410 2024-09-19T06:13:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58a1628640 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d31af0 0x560604d39b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58a1e29640 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d32410 0x560604d3a090 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x560604d2eec0 tx=0x7f589802f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49b2791ca191e9ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58857fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:13:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58857fa640 1 -- 172.21.15.5:0/3554486162 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589804a060 con 0x560604d32410 2024-09-19T06:13:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58857fa640 1 -- 172.21.15.5:0/3554486162 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f5898004040 con 0x560604d32410 2024-09-19T06:13:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58a6e29740 1 -- 172.21.15.5:0/3554486162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f588c0054d0 con 0x560604d32410 2024-09-19T06:13:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58857fa640 1 -- 172.21.15.5:0/3554486162 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5898034040 con 0x560604d32410 2024-09-19T06:13:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58a6e29740 10 client.27536 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.878+0000 7f58a6e29740 1 -- 172.21.15.5:0/3554486162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560604a75620 con 0x560604d32410 2024-09-19T06:13:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.879+0000 7f58857fa640 1 -- 172.21.15.5:0/3554486162 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f588c0054d0 con 0x560604d32410 2024-09-19T06:13:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.879+0000 7f58857fa640 1 -- 172.21.15.5:0/3554486162 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f589804e8a0 con 0x560604d32410 2024-09-19T06:13:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.879+0000 7f58a6e29740 10 client.27536 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.879+0000 7f58a6e29740 10 client.27536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.879+0000 7f58a6e29740 20 client.27536 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.879+0000 7f58a6e29740 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560604d4cbf0 0x560604d4f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.879+0000 7f58a6e29740 4 client.27536 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.879+0000 7f58a6e29740 1 -- 172.21.15.5:0/3554486162 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560604ca3a20 con 0x560604d4cbf0 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.879+0000 7f58a1628640 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560604d4cbf0 0x560604d4f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.879+0000 7f58857fa640 10 client.27536 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.879+0000 7f58a1628640 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560604d4cbf0 0x560604d4f050 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31fdebd8e316ee4e server_cookie=3b92435510bdbca in_seq=0 out_seq=0 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58857fa640 1 -- 172.21.15.5:0/3554486162 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560604ca3a20 con 0x560604d4cbf0 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58857fa640 10 client.27536 handle_command_reply: tid=0 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58a6e29740 1 client.27536 shutdown 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58a6e29740 1 -- 172.21.15.5:0/3554486162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d32410 msgr2=0x560604d3a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58a6e29740 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d32410 0x560604d3a090 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x560604d2eec0 tx=0x7f589802f710 comp rx=0 tx=0).stop 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58a6e29740 1 -- 172.21.15.5:0/3554486162 shutdown_connections 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58a6e29740 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560604d4cbf0 0x560604d4f050 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58a6e29740 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d3ccb0 0x560604d3a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58a6e29740 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d32410 0x560604d3a090 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58857fa640 0 client.27536 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58a6e29740 1 --2- 172.21.15.5:0/3554486162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d31af0 0x560604d39b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58a6e29740 1 -- 172.21.15.5:0/3554486162 >> 172.21.15.5:0/3554486162 conn(0x560604d2e060 msgr2=0x560604d33dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58a6e29740 1 -- 172.21.15.5:0/3554486162 shutdown_connections 2024-09-19T06:13:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58a6e29740 1 -- 172.21.15.5:0/3554486162 wait complete. 2024-09-19T06:13:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.880+0000 7f58a6e29740 20 client.27536 trim_cache size 0 max 16384 2024-09-19T06:13:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a6e29740 1 Processor -- start 2024-09-19T06:13:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a6e29740 1 -- start start 2024-09-19T06:13:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a6e29740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d31af0 0x560604d3a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a6e29740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d32410 0x560604d54960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a6e29740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 0x560604d54ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a6e29740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f589c1ad900 con 0x560604d3ccb0 2024-09-19T06:13:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a6e29740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f589c1af500 con 0x560604d31af0 2024-09-19T06:13:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a6e29740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f589c1adfd0 con 0x560604d32410 2024-09-19T06:13:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a1628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 0x560604d54ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a1628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 0x560604d54ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60608/0 (socket says 172.21.15.5:60608) 2024-09-19T06:13:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a1628640 1 -- 172.21.15.5:0/2500930310 learned_addr learned my addr 172.21.15.5:0/2500930310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a1e29640 1 --2- 172.21.15.5:0/2500930310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d32410 0x560604d54960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a1628640 1 --2- 172.21.15.5:0/2500930310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d31af0 0x560604d3a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a1628640 1 -- 172.21.15.5:0/2500930310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d32410 msgr2=0x560604d54960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a1628640 1 --2- 172.21.15.5:0/2500930310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d32410 0x560604d54960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a1628640 1 -- 172.21.15.5:0/2500930310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d31af0 msgr2=0x560604d3a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a1628640 1 --2- 172.21.15.5:0/2500930310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d31af0 0x560604d3a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.897+0000 7f58a1628640 1 -- 172.21.15.5:0/2500930310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560604d2ecf0 con 0x560604d3ccb0 2024-09-19T06:13:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.898+0000 7f58a1628640 1 --2- 172.21.15.5:0/2500930310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 0x560604d54ea0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f588c009ec0 tx=0x7f588c009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a55502744d9da2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.898+0000 7f58877fe640 1 -- 172.21.15.5:0/2500930310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c004a20 con 0x560604d3ccb0 2024-09-19T06:13:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.898+0000 7f58877fe640 1 -- 172.21.15.5:0/2500930310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f588c004bc0 con 0x560604d3ccb0 2024-09-19T06:13:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.898+0000 7f58877fe640 1 -- 172.21.15.5:0/2500930310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c02d2d0 con 0x560604d3ccb0 2024-09-19T06:13:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.898+0000 7f58a6e29740 1 -- 172.21.15.5:0/2500930310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 msgr2=0x560604d54ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.898+0000 7f58a6e29740 1 --2- 172.21.15.5:0/2500930310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 0x560604d54ea0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f588c009ec0 tx=0x7f588c009420 comp rx=0 tx=0).stop 2024-09-19T06:13:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.898+0000 7f58a6e29740 1 -- 172.21.15.5:0/2500930310 shutdown_connections 2024-09-19T06:13:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.898+0000 7f58a6e29740 1 --2- 172.21.15.5:0/2500930310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 0x560604d54ea0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.898+0000 7f58a6e29740 1 --2- 172.21.15.5:0/2500930310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d32410 0x560604d54960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.898+0000 7f58a6e29740 1 --2- 172.21.15.5:0/2500930310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d31af0 0x560604d3a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.898+0000 7f58a6e29740 1 -- 172.21.15.5:0/2500930310 >> 172.21.15.5:0/2500930310 conn(0x560604d2e060 msgr2=0x560604d3bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.899+0000 7f58a6e29740 1 -- 172.21.15.5:0/2500930310 shutdown_connections 2024-09-19T06:13:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.899+0000 7f58a6e29740 1 -- 172.21.15.5:0/2500930310 wait complete. 2024-09-19T06:13:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.899+0000 7f58a6e29740 1 -- start start 2024-09-19T06:13:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.899+0000 7f58a6e29740 1 Processor -- start 2024-09-19T06:13:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.899+0000 7f58a6e29740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d31af0 0x560604d45560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.899+0000 7f58a6e29740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d32410 0x560604d369a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.899+0000 7f58a6e29740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 0x560604d370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.899+0000 7f58a6e29740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f589c1aee30 con 0x560604d3ccb0 2024-09-19T06:13:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.899+0000 7f58a6e29740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f589c1ae700 con 0x560604d32410 2024-09-19T06:13:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.899+0000 7f58a6e29740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f589c1ab900 con 0x560604d31af0 2024-09-19T06:13:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.899+0000 7f58a6e29740 10 client.0 _ll_register_callbacks cb 0x7f5898005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a1628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 0x560604d370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a1e29640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d32410 0x560604d369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a1628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 0x560604d370e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60624/0 (socket says 172.21.15.5:60624) 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a1628640 1 -- 172.21.15.5:0/876298670 learned_addr learned my addr 172.21.15.5:0/876298670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a1628640 1 --2- 172.21.15.5:0/876298670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d31af0 0x560604d45560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a1e29640 1 -- 172.21.15.5:0/876298670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d31af0 msgr2=0x560604d45560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a1e29640 1 --2- 172.21.15.5:0/876298670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d31af0 0x560604d45560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a1e29640 1 -- 172.21.15.5:0/876298670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 msgr2=0x560604d370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a1e29640 1 --2- 172.21.15.5:0/876298670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 0x560604d370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a1e29640 1 -- 172.21.15.5:0/876298670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560604d43070 con 0x560604d32410 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a1e29640 1 --2- 172.21.15.5:0/876298670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d32410 0x560604d369a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5898004af0 tx=0x7f589802f780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6f5b9dd2127a056 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58857fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:13:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58857fa640 1 -- 172.21.15.5:0/876298670 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589805c060 con 0x560604d32410 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a6e29740 1 -- 172.21.15.5:0/876298670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f588c004bc0 con 0x560604d32410 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58857fa640 1 -- 172.21.15.5:0/876298670 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f5898045940 con 0x560604d32410 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58857fa640 1 -- 172.21.15.5:0/876298670 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f589804a070 con 0x560604d32410 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a6e29740 10 client.27542 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.900+0000 7f58a6e29740 1 -- 172.21.15.5:0/876298670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560604db7b00 con 0x560604d32410 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.901+0000 7f58857fa640 1 -- 172.21.15.5:0/876298670 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f588c004bc0 con 0x560604d32410 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.901+0000 7f58857fa640 1 -- 172.21.15.5:0/876298670 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f5898057930 con 0x560604d32410 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.901+0000 7f58a6e29740 10 client.27542 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.901+0000 7f58a6e29740 10 client.27542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.901+0000 7f58a6e29740 20 client.27542 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.901+0000 7f58a6e29740 1 --2- 172.21.15.5:0/876298670 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560604db82e0 0x560604dba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.901+0000 7f58a6e29740 4 client.27542 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:13:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.901+0000 7f58a6e29740 1 -- 172.21.15.5:0/876298670 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560604c9dd70 con 0x560604db82e0 2024-09-19T06:13:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.901+0000 7f58a1628640 1 --2- 172.21.15.5:0/876298670 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560604db82e0 0x560604dba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.902+0000 7f58a1628640 1 --2- 172.21.15.5:0/876298670 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560604db82e0 0x560604dba700 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b988ca78a1e471 server_cookie=4152563803ecddca in_seq=0 out_seq=0 2024-09-19T06:13:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.902+0000 7f58857fa640 10 client.27542 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:23.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58857fa640 1 -- 172.21.15.5:0/876298670 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560604c9dd70 con 0x560604db82e0 2024-09-19T06:13:23.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58857fa640 10 client.27542 handle_command_reply: tid=0 2024-09-19T06:13:23.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58a6e29740 1 client.27542 shutdown 2024-09-19T06:13:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58a6e29740 1 -- 172.21.15.5:0/876298670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d32410 msgr2=0x560604d369a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58a6e29740 1 --2- 172.21.15.5:0/876298670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d32410 0x560604d369a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5898004af0 tx=0x7f589802f780 comp rx=0 tx=0).stop 2024-09-19T06:13:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58a6e29740 1 -- 172.21.15.5:0/876298670 shutdown_connections 2024-09-19T06:13:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58a6e29740 1 --2- 172.21.15.5:0/876298670 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560604db82e0 0x560604dba700 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58a6e29740 1 --2- 172.21.15.5:0/876298670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560604d3ccb0 0x560604d370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58a6e29740 1 --2- 172.21.15.5:0/876298670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560604d32410 0x560604d369a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58a6e29740 1 --2- 172.21.15.5:0/876298670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560604d31af0 0x560604d45560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58857fa640 0 client.27542 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58a6e29740 1 -- 172.21.15.5:0/876298670 >> 172.21.15.5:0/876298670 conn(0x560604d2e060 msgr2=0x560604d3bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58a6e29740 1 -- 172.21.15.5:0/876298670 shutdown_connections 2024-09-19T06:13:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.926+0000 7f58a6e29740 1 -- 172.21.15.5:0/876298670 wait complete. 2024-09-19T06:13:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a6e29740 20 client.27542 trim_cache size 0 max 16384 2024-09-19T06:13:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:13:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:13:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:13:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:13:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a408d640 1 -- 172.21.15.5:0/826423281 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f58880a33b0 msgr2=0x7f58880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a408d640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f58880a33b0 0x7f58880a5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f589409b640 tx=0x7f589406d6c0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a408d640 1 -- 172.21.15.5:0/826423281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c10c0a0 msgr2=0x7f589c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a408d640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c10c0a0 0x7f589c19cbe0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f589400d750 tx=0x7f589400dc10 comp rx=0 tx=0).stop 2024-09-19T06:13:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a408d640 1 -- 172.21.15.5:0/826423281 shutdown_connections 2024-09-19T06:13:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a408d640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c10c0a0 0x7f589c19cbe0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a408d640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f58880a33b0 0x7f58880a5870 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a408d640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f589c1035a0 0x7f589c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a408d640 1 --2- 172.21.15.5:0/826423281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f589c102c80 0x7f589c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a408d640 1 -- 172.21.15.5:0/826423281 >> 172.21.15.5:0/826423281 conn(0x7f589c0779a0 msgr2=0x7f589c0fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a408d640 1 -- 172.21.15.5:0/826423281 shutdown_connections 2024-09-19T06:13:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:23.927+0000 7f58a408d640 1 -- 172.21.15.5:0/826423281 wait complete. 2024-09-19T06:13:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:24 smithi157 ceph-mon[35346]: pgmap v313: 97 pgs: 97 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 285 KiB/s wr, 39 op/s 2024-09-19T06:13:24.839 INFO:tasks.workunit.client.0.smithi005.stdout: 1 246712 2.42 MB/sec execute 410 sec latency 23.948 ms 2024-09-19T06:13:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:24 smithi005 ceph-mon[30955]: pgmap v313: 97 pgs: 97 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 285 KiB/s wr, 39 op/s 2024-09-19T06:13:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:24 smithi049 ceph-mon[34640]: pgmap v313: 97 pgs: 97 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 285 KiB/s wr, 39 op/s 2024-09-19T06:13:25.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 247258 2.42 MB/sec execute 411 sec latency 19.350 ms 2024-09-19T06:13:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:26 smithi157 ceph-mon[35346]: pgmap v314: 97 pgs: 97 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 285 KiB/s wr, 39 op/s 2024-09-19T06:13:26.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 247995 2.43 MB/sec execute 412 sec latency 15.233 ms 2024-09-19T06:13:26.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:26 smithi005 ceph-mon[30955]: pgmap v314: 97 pgs: 97 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 285 KiB/s wr, 39 op/s 2024-09-19T06:13:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:26 smithi049 ceph-mon[34640]: pgmap v314: 97 pgs: 97 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 285 KiB/s wr, 39 op/s 2024-09-19T06:13:27.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248402 2.43 MB/sec execute 413 sec latency 34.622 ms 2024-09-19T06:13:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:28 smithi157 ceph-mon[35346]: pgmap v315: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:13:28.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248402 2.43 MB/sec execute 414 sec latency 1034.708 ms 2024-09-19T06:13:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:28 smithi005 ceph-mon[30955]: pgmap v315: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:13:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:28 smithi049 ceph-mon[34640]: pgmap v315: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:13:29.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248402 2.42 MB/sec execute 415 sec latency 2034.831 ms 2024-09-19T06:13:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:30 smithi157 ceph-mon[35346]: pgmap v316: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 1005 KiB/s wr, 96 op/s 2024-09-19T06:13:30.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248469 2.42 MB/sec execute 416 sec latency 2663.090 ms 2024-09-19T06:13:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:30 smithi005 ceph-mon[30955]: pgmap v316: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 1005 KiB/s wr, 96 op/s 2024-09-19T06:13:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:30 smithi049 ceph-mon[34640]: pgmap v316: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 1005 KiB/s wr, 96 op/s 2024-09-19T06:13:31.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:13:31.361 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:13:31.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248469 2.41 MB/sec execute 417 sec latency 1117.770 ms 2024-09-19T06:13:31.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.861+0000 7f0ef945c640 1 -- 172.21.15.5:0/2044022586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4106f10 msgr2=0x7f0ef410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:31.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.861+0000 7f0ef945c640 1 --2- 172.21.15.5:0/2044022586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4106f10 0x7f0ef410ea90 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009ca0 tx=0x7f0eec02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.861+0000 7f0ef945c640 1 -- 172.21.15.5:0/2044022586 shutdown_connections 2024-09-19T06:13:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.861+0000 7f0ef945c640 1 --2- 172.21.15.5:0/2044022586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4106f10 0x7f0ef410ea90 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.861+0000 7f0ef945c640 1 --2- 172.21.15.5:0/2044022586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4102790 0x7f0ef41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.861+0000 7f0ef945c640 1 --2- 172.21.15.5:0/2044022586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ef4101de0 0x7f0ef41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.861+0000 7f0ef945c640 1 -- 172.21.15.5:0/2044022586 >> 172.21.15.5:0/2044022586 conn(0x7f0ef40fd880 msgr2=0x7f0ef40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.861+0000 7f0ef945c640 1 -- 172.21.15.5:0/2044022586 shutdown_connections 2024-09-19T06:13:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.861+0000 7f0ef945c640 1 -- 172.21.15.5:0/2044022586 wait complete. 2024-09-19T06:13:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.862+0000 7f0ef945c640 1 Processor -- start 2024-09-19T06:13:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.862+0000 7f0ef945c640 1 -- start start 2024-09-19T06:13:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.862+0000 7f0ef945c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4101de0 0x7f0ef4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.862+0000 7f0ef945c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ef4102790 0x7f0ef41987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.862+0000 7f0ef945c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4106f10 0x7f0ef419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.862+0000 7f0ef945c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4111e20 con 0x7f0ef4101de0 2024-09-19T06:13:31.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.862+0000 7f0ef945c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4111ca0 con 0x7f0ef4106f10 2024-09-19T06:13:31.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.862+0000 7f0ef945c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4111fa0 con 0x7f0ef4102790 2024-09-19T06:13:31.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.862+0000 7f0ef2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4101de0 0x7f0ef4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.862+0000 7f0ef2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4101de0 0x7f0ef4198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52476/0 (socket says 172.21.15.5:52476) 2024-09-19T06:13:31.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0ef2ffd640 1 -- 172.21.15.5:0/2961847593 learned_addr learned my addr 172.21.15.5:0/2961847593 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:31.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0ef27fc640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ef4102790 0x7f0ef41987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0ef37fe640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4106f10 0x7f0ef419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0ef2ffd640 1 -- 172.21.15.5:0/2961847593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ef4102790 msgr2=0x7f0ef41987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:31.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ef4102790 0x7f0ef41987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0ef2ffd640 1 -- 172.21.15.5:0/2961847593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4106f10 msgr2=0x7f0ef419cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:31.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4106f10 0x7f0ef419cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0ef2ffd640 1 -- 172.21.15.5:0/2961847593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef419d230 con 0x7f0ef4101de0 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4101de0 0x7f0ef4198280 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400e9f0 tx=0x7f0ee400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0edbfff640 1 -- 172.21.15.5:0/2961847593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400ccc0 con 0x7f0ef4101de0 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0edbfff640 1 -- 172.21.15.5:0/2961847593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ee400ce60 con 0x7f0ef4101de0 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0ef945c640 1 -- 172.21.15.5:0/2961847593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef419d520 con 0x7f0ef4101de0 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0edbfff640 1 -- 172.21.15.5:0/2961847593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4010430 con 0x7f0ef4101de0 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.863+0000 7f0ef945c640 1 -- 172.21.15.5:0/2961847593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef41a4e00 con 0x7f0ef4101de0 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.866+0000 7f0edbfff640 1 -- 172.21.15.5:0/2961847593 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0ee4005020 con 0x7f0ef4101de0 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.866+0000 7f0efb835740 1 Processor -- start 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.866+0000 7f0efb835740 1 -- start start 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.866+0000 7f0efb835740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1c72880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.866+0000 7f0efb835740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c7d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.866+0000 7f0efb835740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c7d680 0x55e6c1c7fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.866+0000 7f0efb835740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e6c1bd82e0 con 0x55e6c1c724a0 2024-09-19T06:13:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.866+0000 7f0efb835740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e6c1b7fce0 con 0x55e6c1c72dc0 2024-09-19T06:13:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.866+0000 7f0efb835740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e6c1bd42a0 con 0x55e6c1c7d680 2024-09-19T06:13:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.866+0000 7f0ef37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1c72880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0ef37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1c72880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52492/0 (socket says 172.21.15.5:52492) 2024-09-19T06:13:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0ef37fe640 1 -- 172.21.15.5:0/4231811157 learned_addr learned my addr 172.21.15.5:0/4231811157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/4231811157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c7d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0ef27fc640 1 --2- 172.21.15.5:0/4231811157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c7d680 0x55e6c1c7fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0edbfff640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0edc0a3430 0x7f0edc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0ef37fe640 1 -- 172.21.15.5:0/4231811157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c7d680 msgr2=0x55e6c1c7fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0ef37fe640 1 --2- 172.21.15.5:0/4231811157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c7d680 0x55e6c1c7fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0ef37fe640 1 -- 172.21.15.5:0/4231811157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c72dc0 msgr2=0x55e6c1c7d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0ef37fe640 1 --2- 172.21.15.5:0/4231811157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c7d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0ef37fe640 1 -- 172.21.15.5:0/4231811157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c1b64440 con 0x55e6c1c724a0 2024-09-19T06:13:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/4231811157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c7d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0edbfff640 1 -- 172.21.15.5:0/2961847593 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f0ee40d0180 con 0x7f0ef4101de0 2024-09-19T06:13:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0ef37fe640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0edc0a3430 0x7f0edc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.867+0000 7f0ef37fe640 1 --2- 172.21.15.5:0/4231811157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1c72880 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0eec035000 tx=0x7f0eec031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f382db9e5c7beb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.868+0000 7f0ed97fa640 1 -- 172.21.15.5:0/4231811157 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0316d0 con 0x55e6c1c724a0 2024-09-19T06:13:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.868+0000 7f0ef37fe640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0edc0a3430 0x7f0edc0a58f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0eec04c000 tx=0x7f0eec049000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.868+0000 7f0ed97fa640 1 -- 172.21.15.5:0/4231811157 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eec036040 con 0x55e6c1c724a0 2024-09-19T06:13:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.868+0000 7f0ed97fa640 1 -- 172.21.15.5:0/4231811157 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec030820 con 0x55e6c1c724a0 2024-09-19T06:13:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.869+0000 7f0efb835740 1 -- 172.21.15.5:0/4231811157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 msgr2=0x55e6c1c72880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.869+0000 7f0efb835740 1 --2- 172.21.15.5:0/4231811157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1c72880 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0eec035000 tx=0x7f0eec031af0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.869+0000 7f0efb835740 1 -- 172.21.15.5:0/4231811157 shutdown_connections 2024-09-19T06:13:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.869+0000 7f0efb835740 1 --2- 172.21.15.5:0/4231811157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c7d680 0x55e6c1c7fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.869+0000 7f0efb835740 1 --2- 172.21.15.5:0/4231811157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c7d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.869+0000 7f0efb835740 1 --2- 172.21.15.5:0/4231811157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1c72880 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.869+0000 7f0efb835740 1 -- 172.21.15.5:0/4231811157 >> 172.21.15.5:0/4231811157 conn(0x55e6c1c6ea30 msgr2=0x55e6c1c6ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.869+0000 7f0efb835740 1 -- 172.21.15.5:0/4231811157 shutdown_connections 2024-09-19T06:13:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.869+0000 7f0efb835740 1 -- 172.21.15.5:0/4231811157 wait complete. 2024-09-19T06:13:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.869+0000 7f0efb835740 1 -- start start 2024-09-19T06:13:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.869+0000 7f0efb835740 1 Processor -- start 2024-09-19T06:13:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0efb835740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c724a0 0x55e6c1c7cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0efb835740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c7a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0efb835740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c7d680 0x55e6c1c7a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0efb835740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e6c1c844e0 con 0x55e6c1c72dc0 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0efb835740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e6c1b93050 con 0x55e6c1c724a0 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0efb835740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e6c1c84660 con 0x55e6c1c7d680 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0efb835740 10 client.0 _ll_register_callbacks cb 0x55e6c1bd3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0ef27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c7a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0ef27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c7a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52506/0 (socket says 172.21.15.5:52506) 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0ef27fc640 1 -- 172.21.15.5:0/3192675486 learned_addr learned my addr 172.21.15.5:0/3192675486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0ef37fe640 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c7d680 0x55e6c1c7a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c724a0 0x55e6c1c7cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0ef27fc640 1 -- 172.21.15.5:0/3192675486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c7d680 msgr2=0x55e6c1c7a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0ef27fc640 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c7d680 0x55e6c1c7a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0ef27fc640 1 -- 172.21.15.5:0/3192675486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c724a0 msgr2=0x55e6c1c7cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0ef27fc640 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c724a0 0x55e6c1c7cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0ef27fc640 1 -- 172.21.15.5:0/3192675486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c19493c0 con 0x55e6c1c72dc0 2024-09-19T06:13:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0ef27fc640 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c7a250 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8010820 tx=0x7f0ee8010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8e091683b7922f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.870+0000 7f0ed6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0ed6ff5640 1 -- 172.21.15.5:0/3192675486 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee800fb90 con 0x55e6c1c72dc0 2024-09-19T06:13:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0ed6ff5640 1 -- 172.21.15.5:0/3192675486 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f0ee800fd30 con 0x55e6c1c72dc0 2024-09-19T06:13:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0efb835740 1 -- 172.21.15.5:0/3192675486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eec036040 con 0x55e6c1c72dc0 2024-09-19T06:13:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0ed6ff5640 1 -- 172.21.15.5:0/3192675486 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0ee8005020 con 0x55e6c1c72dc0 2024-09-19T06:13:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0efb835740 10 client.18030 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0efb835740 1 -- 172.21.15.5:0/3192675486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e6c1c8fec0 con 0x55e6c1c72dc0 2024-09-19T06:13:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0ed6ff5640 1 -- 172.21.15.5:0/3192675486 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eec036040 con 0x55e6c1c72dc0 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0ed6ff5640 1 -- 172.21.15.5:0/3192675486 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f0ee8019410 con 0x55e6c1c72dc0 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0efb835740 10 client.18030 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0efb835740 10 client.18030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0efb835740 20 client.18030 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0efb835740 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e6c1c91440 0x55e6c1c93870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0efb835740 4 client.18030 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.871+0000 7f0efb835740 1 -- 172.21.15.5:0/3192675486 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6c1be5850 con 0x55e6c1c91440 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.872+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e6c1c91440 0x55e6c1c93870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.872+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e6c1c91440 0x55e6c1c93870 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb1c54c79595a6ef server_cookie=858b8bbe02daf373 in_seq=0 out_seq=0 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.872+0000 7f0ed6ff5640 10 client.18030 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0ed6ff5640 1 -- 172.21.15.5:0/3192675486 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6c1be5850 con 0x55e6c1c91440 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0ed6ff5640 10 client.18030 handle_command_reply: tid=0 2024-09-19T06:13:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0efb835740 1 client.18030 shutdown 2024-09-19T06:13:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0efb835740 1 -- 172.21.15.5:0/3192675486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c72dc0 msgr2=0x55e6c1c7a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0efb835740 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c7a250 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8010820 tx=0x7f0ee8010ce0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0efb835740 1 -- 172.21.15.5:0/3192675486 shutdown_connections 2024-09-19T06:13:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0efb835740 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e6c1c91440 0x55e6c1c93870 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0efb835740 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c7d680 0x55e6c1c7a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0efb835740 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c7a250 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0efb835740 1 --2- 172.21.15.5:0/3192675486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c724a0 0x55e6c1c7cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0efb835740 1 -- 172.21.15.5:0/3192675486 >> 172.21.15.5:0/3192675486 conn(0x55e6c1c6ea30 msgr2=0x55e6c1c6db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0ed6ff5640 0 client.18030 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0efb835740 1 -- 172.21.15.5:0/3192675486 shutdown_connections 2024-09-19T06:13:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0efb835740 1 -- 172.21.15.5:0/3192675486 wait complete. 2024-09-19T06:13:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.873+0000 7f0efb835740 20 client.18030 trim_cache size 0 max 16384 2024-09-19T06:13:31.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.890+0000 7f0efb835740 1 Processor -- start 2024-09-19T06:13:31.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.890+0000 7f0efb835740 1 -- start start 2024-09-19T06:13:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.890+0000 7f0efb835740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1c8d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.890+0000 7f0efb835740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c99130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0efb835740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c7d680 0x55e6c1c99670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0efb835740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef41ab6a0 con 0x55e6c1c724a0 2024-09-19T06:13:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0efb835740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef41ad2a0 con 0x55e6c1c7d680 2024-09-19T06:13:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0efb835740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef41abd70 con 0x55e6c1c72dc0 2024-09-19T06:13:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ef2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1c8d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ef2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1c8d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52512/0 (socket says 172.21.15.5:52512) 2024-09-19T06:13:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ef2ffd640 1 -- 172.21.15.5:0/4239083425 learned_addr learned my addr 172.21.15.5:0/4239083425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ef27fc640 1 --2- 172.21.15.5:0/4239083425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c99130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ef37fe640 1 --2- 172.21.15.5:0/4239083425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c7d680 0x55e6c1c99670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ef2ffd640 1 -- 172.21.15.5:0/4239083425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c72dc0 msgr2=0x55e6c1c99130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/4239083425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c99130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ef2ffd640 1 -- 172.21.15.5:0/4239083425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c7d680 msgr2=0x55e6c1c99670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/4239083425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c7d680 0x55e6c1c99670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ef2ffd640 1 -- 172.21.15.5:0/4239083425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c1c787a0 con 0x55e6c1c724a0 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ef37fe640 1 --2- 172.21.15.5:0/4239083425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c7d680 0x55e6c1c99670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/4239083425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1c8d740 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400c6a0 tx=0x7f0ee40d3cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44182f0f0f1393c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ed8ff9640 1 -- 172.21.15.5:0/4239083425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee40986e0 con 0x55e6c1c724a0 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ed8ff9640 1 -- 172.21.15.5:0/4239083425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ee409cb40 con 0x55e6c1c724a0 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.891+0000 7f0ed8ff9640 1 -- 172.21.15.5:0/4239083425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee409bda0 con 0x55e6c1c724a0 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.892+0000 7f0efb835740 1 -- 172.21.15.5:0/4239083425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 msgr2=0x55e6c1c8d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.892+0000 7f0efb835740 1 --2- 172.21.15.5:0/4239083425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1c8d740 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400c6a0 tx=0x7f0ee40d3cc0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.892+0000 7f0efb835740 1 -- 172.21.15.5:0/4239083425 shutdown_connections 2024-09-19T06:13:31.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.892+0000 7f0efb835740 1 --2- 172.21.15.5:0/4239083425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c7d680 0x55e6c1c99670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.892+0000 7f0efb835740 1 --2- 172.21.15.5:0/4239083425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c72dc0 0x55e6c1c99130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.892+0000 7f0efb835740 1 --2- 172.21.15.5:0/4239083425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1c8d740 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.892+0000 7f0efb835740 1 -- 172.21.15.5:0/4239083425 >> 172.21.15.5:0/4239083425 conn(0x55e6c1c6ea30 msgr2=0x55e6c1c79340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:31.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.892+0000 7f0efb835740 1 -- 172.21.15.5:0/4239083425 shutdown_connections 2024-09-19T06:13:31.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.892+0000 7f0efb835740 1 -- 172.21.15.5:0/4239083425 wait complete. 2024-09-19T06:13:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0efb835740 1 -- start start 2024-09-19T06:13:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0efb835740 1 Processor -- start 2024-09-19T06:13:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0efb835740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1cf3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0efb835740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c72dc0 0x55e6c1cf39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0efb835740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c7d680 0x55e6c1c86e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0efb835740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef41acbd0 con 0x55e6c1c724a0 2024-09-19T06:13:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0efb835740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef41ac4a0 con 0x55e6c1c7d680 2024-09-19T06:13:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0efb835740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef41a96a0 con 0x55e6c1c72dc0 2024-09-19T06:13:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0efb835740 10 client.0 _ll_register_callbacks cb 0x7f0ee800a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0ef2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1cf3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0ef2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1cf3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52524/0 (socket says 172.21.15.5:52524) 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0ef2ffd640 1 -- 172.21.15.5:0/196464914 learned_addr learned my addr 172.21.15.5:0/196464914 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0ef27fc640 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c72dc0 0x55e6c1cf39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0ef37fe640 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c7d680 0x55e6c1c86e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0ef2ffd640 1 -- 172.21.15.5:0/196464914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c72dc0 msgr2=0x55e6c1cf39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c72dc0 0x55e6c1cf39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0ef2ffd640 1 -- 172.21.15.5:0/196464914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c7d680 msgr2=0x55e6c1c86e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.893+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c7d680 0x55e6c1c86e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.894+0000 7f0ef2ffd640 1 -- 172.21.15.5:0/196464914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c1c87650 con 0x55e6c1c724a0 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.894+0000 7f0ef2ffd640 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1cf3460 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0ee409c140 tx=0x7f0ee40cd890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10482f1924f7ba94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.894+0000 7f0ed6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.894+0000 7f0ed6ff5640 1 -- 172.21.15.5:0/196464914 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee409bf80 con 0x55e6c1c724a0 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.894+0000 7f0ed6ff5640 1 -- 172.21.15.5:0/196464914 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f0ee4016dd0 con 0x55e6c1c724a0 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.894+0000 7f0efb835740 1 -- 172.21.15.5:0/196464914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee409cb40 con 0x55e6c1c724a0 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.894+0000 7f0ed6ff5640 1 -- 172.21.15.5:0/196464914 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0ee409c210 con 0x55e6c1c724a0 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.894+0000 7f0efb835740 10 client.18036 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.894+0000 7f0efb835740 1 -- 172.21.15.5:0/196464914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e6c1cfbba0 con 0x55e6c1c724a0 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.894+0000 7f0ed6ff5640 1 -- 172.21.15.5:0/196464914 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ee409cb40 con 0x55e6c1c724a0 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0ed6ff5640 1 -- 172.21.15.5:0/196464914 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f0ee40cdbc0 con 0x55e6c1c724a0 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0efb835740 10 client.18036 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0efb835740 10 client.18036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0efb835740 20 client.18036 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0efb835740 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e6c1cfc200 0x55e6c1cfe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0efb835740 4 client.18036 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0efb835740 1 -- 172.21.15.5:0/196464914 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e6c1bdfd60 con 0x55e6c1cfc200 2024-09-19T06:13:31.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0ef27fc640 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e6c1cfc200 0x55e6c1cfe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0ef27fc640 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e6c1cfc200 0x55e6c1cfe5f0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e3ad217d68570c9 server_cookie=516a43ac695a2f79 in_seq=0 out_seq=0 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0ed6ff5640 10 client.18036 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0ed6ff5640 1 -- 172.21.15.5:0/196464914 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e6c1bdfd60 con 0x55e6c1cfc200 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0ed6ff5640 10 client.18036 handle_command_reply: tid=0 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.895+0000 7f0efb835740 1 client.18036 shutdown 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0efb835740 1 -- 172.21.15.5:0/196464914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 msgr2=0x55e6c1cf3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0efb835740 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1cf3460 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0ee409c140 tx=0x7f0ee40cd890 comp rx=0 tx=0).stop 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0efb835740 1 -- 172.21.15.5:0/196464914 shutdown_connections 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0efb835740 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e6c1cfc200 0x55e6c1cfe5f0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0efb835740 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6c1c7d680 0x55e6c1c86e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0efb835740 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6c1c72dc0 0x55e6c1cf39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0efb835740 1 --2- 172.21.15.5:0/196464914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6c1c724a0 0x55e6c1cf3460 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0efb835740 1 -- 172.21.15.5:0/196464914 >> 172.21.15.5:0/196464914 conn(0x55e6c1c6ea30 msgr2=0x55e6c1c7bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:31.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0ed6ff5640 0 client.18036 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:31.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0efb835740 1 -- 172.21.15.5:0/196464914 shutdown_connections 2024-09-19T06:13:31.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0efb835740 1 -- 172.21.15.5:0/196464914 wait complete. 2024-09-19T06:13:31.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0efb835740 20 client.18036 trim_cache size 0 max 16384 2024-09-19T06:13:31.903 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:13:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:13:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:13:31.903 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:13:31.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0ef945c640 1 -- 172.21.15.5:0/2961847593 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0edc0a3430 msgr2=0x7f0edc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:31.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0ef945c640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0edc0a3430 0x7f0edc0a58f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0eec04c000 tx=0x7f0eec049000 comp rx=0 tx=0).stop 2024-09-19T06:13:31.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0ef945c640 1 -- 172.21.15.5:0/2961847593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4101de0 msgr2=0x7f0ef4198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:31.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.896+0000 7f0ef945c640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4101de0 0x7f0ef4198280 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400e9f0 tx=0x7f0ee400edf0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.897+0000 7f0ef945c640 1 -- 172.21.15.5:0/2961847593 shutdown_connections 2024-09-19T06:13:31.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.897+0000 7f0ef945c640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4106f10 0x7f0ef419cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.897+0000 7f0ef945c640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0edc0a3430 0x7f0edc0a58f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.897+0000 7f0ef945c640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ef4102790 0x7f0ef41987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.897+0000 7f0ef945c640 1 --2- 172.21.15.5:0/2961847593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4101de0 0x7f0ef4198280 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:31.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.897+0000 7f0ef945c640 1 -- 172.21.15.5:0/2961847593 >> 172.21.15.5:0/2961847593 conn(0x7f0ef40fd880 msgr2=0x7f0ef40feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:31.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.897+0000 7f0ef945c640 1 -- 172.21.15.5:0/2961847593 shutdown_connections 2024-09-19T06:13:31.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:31.897+0000 7f0ef945c640 1 -- 172.21.15.5:0/2961847593 wait complete. 2024-09-19T06:13:32.650 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:13:32.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:13:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:32 smithi157 ceph-mon[35346]: pgmap v317: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 873 KiB/s wr, 83 op/s 2024-09-19T06:13:32.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248469 2.40 MB/sec execute 418 sec latency 2117.799 ms 2024-09-19T06:13:32.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:32 smithi005 ceph-mon[30955]: pgmap v317: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 873 KiB/s wr, 83 op/s 2024-09-19T06:13:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:32 smithi049 ceph-mon[34640]: pgmap v317: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 873 KiB/s wr, 83 op/s 2024-09-19T06:13:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:13:33.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.358+0000 7f207979d640 1 -- 172.21.15.5:0/4098665322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207410e480 msgr2=0x7f2074110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.358+0000 7f207979d640 1 --2- 172.21.15.5:0/4098665322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207410e480 0x7f2074110870 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f206c009ca0 tx=0x7f206c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:13:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.358+0000 7f207979d640 1 -- 172.21.15.5:0/4098665322 shutdown_connections 2024-09-19T06:13:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.358+0000 7f207979d640 1 --2- 172.21.15.5:0/4098665322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207410e480 0x7f2074110870 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.358+0000 7f207979d640 1 --2- 172.21.15.5:0/4098665322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2074101280 0x7f207410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.358+0000 7f207979d640 1 --2- 172.21.15.5:0/4098665322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20741008d0 0x7f2074100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.358+0000 7f207979d640 1 -- 172.21.15.5:0/4098665322 >> 172.21.15.5:0/4098665322 conn(0x7f20740fc7a0 msgr2=0x7f20740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.358+0000 7f207979d640 1 -- 172.21.15.5:0/4098665322 shutdown_connections 2024-09-19T06:13:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.358+0000 7f207979d640 1 -- 172.21.15.5:0/4098665322 wait complete. 2024-09-19T06:13:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.358+0000 7f207979d640 1 Processor -- start 2024-09-19T06:13:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.358+0000 7f207979d640 1 -- start start 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f207979d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20741008d0 0x7f207419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f207979d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2074101280 0x7f207419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f207979d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f207410e480 0x7f207419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f207979d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2074114020 con 0x7f2074101280 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f207979d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2074113ea0 con 0x7f20741008d0 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f207979d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20741141a0 con 0x7f207410e480 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f206bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2074101280 0x7f207419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f206bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2074101280 0x7f207419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52548/0 (socket says 172.21.15.5:52548) 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f206bfff640 1 -- 172.21.15.5:0/4150234721 learned_addr learned my addr 172.21.15.5:0/4150234721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f2072ffd640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20741008d0 0x7f207419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f20737fe640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f207410e480 0x7f207419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f206bfff640 1 -- 172.21.15.5:0/4150234721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f207410e480 msgr2=0x7f207419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f206bfff640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f207410e480 0x7f207419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f206bfff640 1 -- 172.21.15.5:0/4150234721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20741008d0 msgr2=0x7f207419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f206bfff640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20741008d0 0x7f207419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f206bfff640 1 -- 172.21.15.5:0/4150234721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207419f530 con 0x7f2074101280 2024-09-19T06:13:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f206bfff640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2074101280 0x7f207419aac0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f205c00d950 tx=0x7f205c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.359+0000 7f2070ff9640 1 -- 172.21.15.5:0/4150234721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c017030 con 0x7f2074101280 2024-09-19T06:13:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.360+0000 7f2070ff9640 1 -- 172.21.15.5:0/4150234721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f205c004020 con 0x7f2074101280 2024-09-19T06:13:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.360+0000 7f207979d640 1 -- 172.21.15.5:0/4150234721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207419f820 con 0x7f2074101280 2024-09-19T06:13:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.360+0000 7f2070ff9640 1 -- 172.21.15.5:0/4150234721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c0047a0 con 0x7f2074101280 2024-09-19T06:13:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.360+0000 7f207979d640 1 -- 172.21.15.5:0/4150234721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20741a7100 con 0x7f2074101280 2024-09-19T06:13:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.361+0000 7f207979d640 1 -- 172.21.15.5:0/4150234721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2074100cb0 con 0x7f2074101280 2024-09-19T06:13:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.362+0000 7f2070ff9640 1 -- 172.21.15.5:0/4150234721 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f205c0041d0 con 0x7f2074101280 2024-09-19T06:13:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.363+0000 7f2070ff9640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20600a3430 0x7f20600a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.363+0000 7f2072ffd640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20600a3430 0x7f20600a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.363+0000 7f2070ff9640 1 -- 172.21.15.5:0/4150234721 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f205c0cf770 con 0x7f2074101280 2024-09-19T06:13:33.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.363+0000 7f2072ffd640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20600a3430 0x7f20600a58f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2064009bd0 tx=0x7f2064005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:33.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.367+0000 7f2070ff9640 1 -- 172.21.15.5:0/4150234721 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f205c09c0d0 con 0x7f2074101280 2024-09-19T06:13:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.579+0000 7f207979d640 1 -- 172.21.15.5:0/4150234721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f207402d7c0 con 0x7f2074101280 2024-09-19T06:13:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.580+0000 7f2070ff9640 1 -- 172.21.15.5:0/4150234721 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11966 (secure 0 0 0) 0x7f205c09b9d0 con 0x7f2074101280 2024-09-19T06:13:33.582 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:13:33.582 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":38,"btime":"2024-09-19T06:13:11:466735+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:13:10.551483+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:13:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 38 2024-09-19T06:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.582+0000 7f207979d640 1 -- 172.21.15.5:0/4150234721 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20600a3430 msgr2=0x7f20600a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.582+0000 7f207979d640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20600a3430 0x7f20600a58f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2064009bd0 tx=0x7f2064005f90 comp rx=0 tx=0).stop 2024-09-19T06:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.582+0000 7f207979d640 1 -- 172.21.15.5:0/4150234721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2074101280 msgr2=0x7f207419aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.582+0000 7f207979d640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2074101280 0x7f207419aac0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f205c00d950 tx=0x7f205c00de10 comp rx=0 tx=0).stop 2024-09-19T06:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.582+0000 7f207979d640 1 -- 172.21.15.5:0/4150234721 shutdown_connections 2024-09-19T06:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.582+0000 7f207979d640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20600a3430 0x7f20600a58f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.582+0000 7f207979d640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f207410e480 0x7f207419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.582+0000 7f207979d640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2074101280 0x7f207419aac0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.582+0000 7f207979d640 1 --2- 172.21.15.5:0/4150234721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20741008d0 0x7f207419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.582+0000 7f207979d640 1 -- 172.21.15.5:0/4150234721 >> 172.21.15.5:0/4150234721 conn(0x7f20740fc7a0 msgr2=0x7f20740feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.582+0000 7f207979d640 1 -- 172.21.15.5:0/4150234721 shutdown_connections 2024-09-19T06:13:33.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:33.582+0000 7f207979d640 1 -- 172.21.15.5:0/4150234721 wait complete. 2024-09-19T06:13:33.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248469 2.40 MB/sec execute 419 sec latency 3117.876 ms 2024-09-19T06:13:34.284 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:13:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:13:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:34 smithi005 ceph-mon[30955]: pgmap v318: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:13:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:34 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/4150234721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:13:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:34 smithi157 ceph-mon[35346]: pgmap v318: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:13:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:34 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/4150234721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:13:34.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248469 2.39 MB/sec execute 420 sec latency 4117.914 ms 2024-09-19T06:13:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:34 smithi049 ceph-mon[34640]: pgmap v318: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:13:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:34 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/4150234721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:13:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.017+0000 7f9444b44640 1 -- 172.21.15.5:0/2314243207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440102790 msgr2=0x7f94401067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.017+0000 7f9444b44640 1 --2- 172.21.15.5:0/2314243207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440102790 0x7f94401067e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9428009900 tx=0x7f942802a410 comp rx=0 tx=0).stop 2024-09-19T06:13:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.017+0000 7f9444b44640 1 -- 172.21.15.5:0/2314243207 shutdown_connections 2024-09-19T06:13:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.017+0000 7f9444b44640 1 --2- 172.21.15.5:0/2314243207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9440106f10 0x7f944010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.017+0000 7f9444b44640 1 --2- 172.21.15.5:0/2314243207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440102790 0x7f94401067e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.017+0000 7f9444b44640 1 --2- 172.21.15.5:0/2314243207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9440101de0 0x7f94401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.017+0000 7f9444b44640 1 -- 172.21.15.5:0/2314243207 >> 172.21.15.5:0/2314243207 conn(0x7f94400fd8c0 msgr2=0x7f94400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.017+0000 7f9444b44640 1 -- 172.21.15.5:0/2314243207 shutdown_connections 2024-09-19T06:13:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.017+0000 7f9444b44640 1 -- 172.21.15.5:0/2314243207 wait complete. 2024-09-19T06:13:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.018+0000 7f9444b44640 1 Processor -- start 2024-09-19T06:13:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9444b44640 1 -- start start 2024-09-19T06:13:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9444b44640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9440101de0 0x7f94400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9444b44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440102790 0x7f9440074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9444b44640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9440106f10 0x7f9440074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9444b44640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9440111e20 con 0x7f9440102790 2024-09-19T06:13:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9444b44640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9440111ca0 con 0x7f9440106f10 2024-09-19T06:13:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9444b44640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9440111fa0 con 0x7f9440101de0 2024-09-19T06:13:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9435d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440102790 0x7f9440074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9435d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440102790 0x7f9440074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51950/0 (socket says 172.21.15.5:51950) 2024-09-19T06:13:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9435d9b640 1 -- 172.21.15.5:0/1661432337 learned_addr learned my addr 172.21.15.5:0/1661432337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f943ed9d640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9440106f10 0x7f9440074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f943e59c640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9440101de0 0x7f94400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9435d9b640 1 -- 172.21.15.5:0/1661432337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9440101de0 msgr2=0x7f94400744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:35.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9435d9b640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9440101de0 0x7f94400744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9435d9b640 1 -- 172.21.15.5:0/1661432337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9440106f10 msgr2=0x7f9440074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:35.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9435d9b640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9440106f10 0x7f9440074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.019+0000 7f9435d9b640 1 -- 172.21.15.5:0/1661432337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94401a6840 con 0x7f9440102790 2024-09-19T06:13:35.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.020+0000 7f9435d9b640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440102790 0x7f9440074a30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9428005940 tx=0x7f94280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:35.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.020+0000 7f9437fff640 1 -- 172.21.15.5:0/1661432337 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942803f030 con 0x7f9440102790 2024-09-19T06:13:35.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.020+0000 7f9437fff640 1 -- 172.21.15.5:0/1661432337 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9428003cb0 con 0x7f9440102790 2024-09-19T06:13:35.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.020+0000 7f9444b44640 1 -- 172.21.15.5:0/1661432337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94401a6ad0 con 0x7f9440102790 2024-09-19T06:13:35.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.020+0000 7f9437fff640 1 -- 172.21.15.5:0/1661432337 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428035070 con 0x7f9440102790 2024-09-19T06:13:35.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.020+0000 7f9444b44640 1 -- 172.21.15.5:0/1661432337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94401a6fb0 con 0x7f9440102790 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.022+0000 7f9437fff640 1 -- 172.21.15.5:0/1661432337 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9428042030 con 0x7f9440102790 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.022+0000 7f9446f1d740 1 Processor -- start 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9446f1d740 1 -- start start 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9446f1d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ec6b40 0x556af4ec5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9446f1d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec55d0 0x556af4ec5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9446f1d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ed0370 0x556af4ed2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9446f1d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556af4e2b2f0 con 0x556af4ed0370 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9446f1d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556af4dd2cf0 con 0x556af4ec6b40 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9446f1d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556af4e272b0 con 0x556af4ec55d0 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9435d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ed0370 0x556af4ed2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9435d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ed0370 0x556af4ed2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51960/0 (socket says 172.21.15.5:51960) 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9435d9b640 1 -- 172.21.15.5:0/1995099226 learned_addr learned my addr 172.21.15.5:0/1995099226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f943e59c640 1 --2- 172.21.15.5:0/1995099226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec55d0 0x556af4ec5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9437fff640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f942c0a33b0 0x7f942c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f943ed9d640 1 --2- 172.21.15.5:0/1995099226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ec6b40 0x556af4ec5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f943ed9d640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f942c0a33b0 0x7f942c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9435d9b640 1 -- 172.21.15.5:0/1995099226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec55d0 msgr2=0x556af4ec5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9435d9b640 1 --2- 172.21.15.5:0/1995099226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec55d0 0x556af4ec5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9435d9b640 1 -- 172.21.15.5:0/1995099226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ec6b40 msgr2=0x556af4ec5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9435d9b640 1 --2- 172.21.15.5:0/1995099226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ec6b40 0x556af4ec5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.023+0000 7f9435d9b640 1 -- 172.21.15.5:0/1995099226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556af4db73a0 con 0x556af4ed0370 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.024+0000 7f9437fff640 1 -- 172.21.15.5:0/1661432337 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f94280f0560 con 0x7f9440102790 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.024+0000 7f9435d9b640 1 --2- 172.21.15.5:0/1995099226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ed0370 0x556af4ed2760 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f94280bc520 tx=0x7f94280bc490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e933a4e4d66bfdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.024+0000 7f943ed9d640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f942c0a33b0 0x7f942c0a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9438005fd0 tx=0x7f9438009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.024+0000 7f9434d99640 1 -- 172.21.15.5:0/1995099226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428049030 con 0x556af4ed0370 2024-09-19T06:13:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.024+0000 7f9434d99640 1 -- 172.21.15.5:0/1995099226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f942803b460 con 0x556af4ed0370 2024-09-19T06:13:35.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.024+0000 7f9434d99640 1 -- 172.21.15.5:0/1995099226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428045e60 con 0x556af4ed0370 2024-09-19T06:13:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.025+0000 7f9446f1d740 1 -- 172.21.15.5:0/1995099226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ed0370 msgr2=0x556af4ed2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.025+0000 7f9446f1d740 1 --2- 172.21.15.5:0/1995099226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ed0370 0x556af4ed2760 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f94280bc520 tx=0x7f94280bc490 comp rx=0 tx=0).stop 2024-09-19T06:13:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.025+0000 7f9446f1d740 1 -- 172.21.15.5:0/1995099226 shutdown_connections 2024-09-19T06:13:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.025+0000 7f9446f1d740 1 --2- 172.21.15.5:0/1995099226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ed0370 0x556af4ed2760 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.025+0000 7f9446f1d740 1 --2- 172.21.15.5:0/1995099226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec55d0 0x556af4ec5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.025+0000 7f9446f1d740 1 --2- 172.21.15.5:0/1995099226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ec6b40 0x556af4ec5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.025+0000 7f9446f1d740 1 -- 172.21.15.5:0/1995099226 >> 172.21.15.5:0/1995099226 conn(0x556af4e8e310 msgr2=0x556af4ec1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.025+0000 7f9446f1d740 1 -- 172.21.15.5:0/1995099226 shutdown_connections 2024-09-19T06:13:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.025+0000 7f9446f1d740 1 -- 172.21.15.5:0/1995099226 wait complete. 2024-09-19T06:13:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.025+0000 7f9446f1d740 1 -- start start 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.025+0000 7f9446f1d740 1 Processor -- start 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f9446f1d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 0x556af4ecfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f9446f1d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ec6b40 0x556af4eccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f9446f1d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ed0370 0x556af4ecd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f9446f1d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556af4e2b2f0 con 0x556af4ec55d0 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f9446f1d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556af4de6060 con 0x556af4ec6b40 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f9446f1d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556af4ed2e30 con 0x556af4ed0370 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f9446f1d740 10 client.0 _ll_register_callbacks cb 0x556af4b398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f943e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 0x556af4ecfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f943e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 0x556af4ecfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51966/0 (socket says 172.21.15.5:51966) 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f943e59c640 1 -- 172.21.15.5:0/4164313275 learned_addr learned my addr 172.21.15.5:0/4164313275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f9435d9b640 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ec6b40 0x556af4eccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f943ed9d640 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ed0370 0x556af4ecd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f943e59c640 1 -- 172.21.15.5:0/4164313275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ed0370 msgr2=0x556af4ecd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f943e59c640 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ed0370 0x556af4ecd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f943e59c640 1 -- 172.21.15.5:0/4164313275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ec6b40 msgr2=0x556af4eccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f943e59c640 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ec6b40 0x556af4eccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f943e59c640 1 -- 172.21.15.5:0/4164313275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556af4b9c3c0 con 0x556af4ec55d0 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f943e59c640 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 0x556af4ecfd10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f943000bf00 tx=0x7f9430012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9511b621278e233 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f9425ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f9425ffb640 1 -- 172.21.15.5:0/4164313275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94300108a0 con 0x556af4ec55d0 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f9425ffb640 1 -- 172.21.15.5:0/4164313275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f9430010a40 con 0x556af4ec55d0 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.026+0000 7f9446f1d740 1 -- 172.21.15.5:0/4164313275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556af4e38860 con 0x556af4ec55d0 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.027+0000 7f9425ffb640 1 -- 172.21.15.5:0/4164313275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9430010c40 con 0x556af4ec55d0 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.027+0000 7f9446f1d740 10 client.18054 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.027+0000 7f9446f1d740 1 -- 172.21.15.5:0/4164313275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556af4ede760 con 0x556af4ec55d0 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.027+0000 7f9425ffb640 1 -- 172.21.15.5:0/4164313275 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556af4e38860 con 0x556af4ec55d0 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.027+0000 7f9425ffb640 1 -- 172.21.15.5:0/4164313275 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f9430019070 con 0x556af4ec55d0 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.027+0000 7f9446f1d740 10 client.18054 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.027+0000 7f9446f1d740 10 client.18054 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.027+0000 7f9446f1d740 20 client.18054 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.027+0000 7f9446f1d740 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556af4edfc70 0x556af4ee20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.027+0000 7f9446f1d740 4 client.18054 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.027+0000 7f9446f1d740 1 -- 172.21.15.5:0/4164313275 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556af4e32d70 con 0x556af4edfc70 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.027+0000 7f9435d9b640 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556af4edfc70 0x556af4ee20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.028+0000 7f9425ffb640 10 client.18054 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.028+0000 7f9435d9b640 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556af4edfc70 0x556af4ee20b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=496ef23280032c05 server_cookie=621df52bb11fcdf6 in_seq=0 out_seq=0 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9425ffb640 1 -- 172.21.15.5:0/4164313275 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556af4e32d70 con 0x556af4edfc70 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9425ffb640 10 client.18054 handle_command_reply: tid=0 2024-09-19T06:13:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9446f1d740 1 client.18054 shutdown 2024-09-19T06:13:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9446f1d740 1 -- 172.21.15.5:0/4164313275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 msgr2=0x556af4ecfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9446f1d740 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 0x556af4ecfd10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f943000bf00 tx=0x7f9430012820 comp rx=0 tx=0).stop 2024-09-19T06:13:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9446f1d740 1 -- 172.21.15.5:0/4164313275 shutdown_connections 2024-09-19T06:13:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9446f1d740 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556af4edfc70 0x556af4ee20b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9446f1d740 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ed0370 0x556af4ecd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9446f1d740 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ec6b40 0x556af4eccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9425ffb640 0 client.18054 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:13:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9446f1d740 1 --2- 172.21.15.5:0/4164313275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 0x556af4ecfd10 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9446f1d740 1 -- 172.21.15.5:0/4164313275 >> 172.21.15.5:0/4164313275 conn(0x556af4e8e310 msgr2=0x556af4ec7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9446f1d740 1 -- 172.21.15.5:0/4164313275 shutdown_connections 2024-09-19T06:13:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9446f1d740 1 -- 172.21.15.5:0/4164313275 wait complete. 2024-09-19T06:13:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.029+0000 7f9446f1d740 20 client.18054 trim_cache size 0 max 16384 2024-09-19T06:13:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f9446f1d740 1 Processor -- start 2024-09-19T06:13:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f9446f1d740 1 -- start start 2024-09-19T06:13:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f9446f1d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 0x556af4ecc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f9446f1d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec6b40 0x556af4ec0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f9446f1d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ed0370 0x556af4ec8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f9446f1d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94401af9e0 con 0x556af4ec55d0 2024-09-19T06:13:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f9446f1d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f94401b15e0 con 0x556af4ed0370 2024-09-19T06:13:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f9446f1d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f94401b00b0 con 0x556af4ec6b40 2024-09-19T06:13:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f943e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 0x556af4ecc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f943e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 0x556af4ecc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51968/0 (socket says 172.21.15.5:51968) 2024-09-19T06:13:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f943e59c640 1 -- 172.21.15.5:0/3983598752 learned_addr learned my addr 172.21.15.5:0/3983598752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f943ed9d640 1 --2- 172.21.15.5:0/3983598752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ed0370 0x556af4ec8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f9435d9b640 1 --2- 172.21.15.5:0/3983598752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec6b40 0x556af4ec0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f943e59c640 1 -- 172.21.15.5:0/3983598752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec6b40 msgr2=0x556af4ec0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f943e59c640 1 --2- 172.21.15.5:0/3983598752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec6b40 0x556af4ec0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f943e59c640 1 -- 172.21.15.5:0/3983598752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ed0370 msgr2=0x556af4ec8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f943e59c640 1 --2- 172.21.15.5:0/3983598752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ed0370 0x556af4ec8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f943e59c640 1 -- 172.21.15.5:0/3983598752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556af4ecb550 con 0x556af4ec55d0 2024-09-19T06:13:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.043+0000 7f9435d9b640 1 --2- 172.21.15.5:0/3983598752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec6b40 0x556af4ec0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:13:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.044+0000 7f943e59c640 1 --2- 172.21.15.5:0/3983598752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 0x556af4ecc880 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f943000bed0 tx=0x7f94300057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b7a3c99fe6c2db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:35.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.044+0000 7f9427fff640 1 -- 172.21.15.5:0/3983598752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94300106a0 con 0x556af4ec55d0 2024-09-19T06:13:35.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.044+0000 7f9427fff640 1 -- 172.21.15.5:0/3983598752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9430023cb0 con 0x556af4ec55d0 2024-09-19T06:13:35.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.044+0000 7f9427fff640 1 -- 172.21.15.5:0/3983598752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943002c980 con 0x556af4ec55d0 2024-09-19T06:13:35.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.044+0000 7f9446f1d740 1 -- 172.21.15.5:0/3983598752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 msgr2=0x556af4ecc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:35.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.044+0000 7f9446f1d740 1 --2- 172.21.15.5:0/3983598752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 0x556af4ecc880 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f943000bed0 tx=0x7f94300057e0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.044+0000 7f9446f1d740 1 -- 172.21.15.5:0/3983598752 shutdown_connections 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.044+0000 7f9446f1d740 1 --2- 172.21.15.5:0/3983598752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ed0370 0x556af4ec8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.044+0000 7f9446f1d740 1 --2- 172.21.15.5:0/3983598752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec6b40 0x556af4ec0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.044+0000 7f9446f1d740 1 --2- 172.21.15.5:0/3983598752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec55d0 0x556af4ecc880 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.044+0000 7f9446f1d740 1 -- 172.21.15.5:0/3983598752 >> 172.21.15.5:0/3983598752 conn(0x556af4e8e310 msgr2=0x556af4ec04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9446f1d740 1 -- 172.21.15.5:0/3983598752 shutdown_connections 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9446f1d740 1 -- 172.21.15.5:0/3983598752 wait complete. 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9446f1d740 1 -- start start 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9446f1d740 1 Processor -- start 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9446f1d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec55d0 0x556af4ed89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9446f1d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec6b40 0x556af4ec9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9446f1d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ed0370 0x556af4eca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9446f1d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94401b0f10 con 0x556af4ec6b40 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9446f1d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f94401b07e0 con 0x556af4ed0370 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9446f1d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f94401ad9e0 con 0x556af4ec55d0 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9446f1d740 10 client.0 _ll_register_callbacks cb 0x7f943000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9435d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec6b40 0x556af4ec9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9435d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec6b40 0x556af4ec9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51970/0 (socket says 172.21.15.5:51970) 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f943e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec55d0 0x556af4ed89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.045+0000 7f9435d9b640 1 -- 172.21.15.5:0/3078264051 learned_addr learned my addr 172.21.15.5:0/3078264051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f943ed9d640 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ed0370 0x556af4eca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9435d9b640 1 -- 172.21.15.5:0/3078264051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec55d0 msgr2=0x556af4ed89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9435d9b640 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec55d0 0x556af4ed89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9435d9b640 1 -- 172.21.15.5:0/3078264051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ed0370 msgr2=0x556af4eca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9435d9b640 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ed0370 0x556af4eca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9435d9b640 1 -- 172.21.15.5:0/3078264051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556af4f49950 con 0x556af4ec6b40 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f943e59c640 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec55d0 0x556af4ed89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f943ed9d640 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ed0370 0x556af4eca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9435d9b640 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec6b40 0x556af4ec9eb0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9428003c80 tx=0x7f942802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543d21b421ec3709 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9425ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9425ffb640 1 -- 172.21.15.5:0/3078264051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428049040 con 0x556af4ec6b40 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9425ffb640 1 -- 172.21.15.5:0/3078264051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f94280b8e90 con 0x556af4ec6b40 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9446f1d740 1 -- 172.21.15.5:0/3078264051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9430023cb0 con 0x556af4ec6b40 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9425ffb640 1 -- 172.21.15.5:0/3078264051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9428045e60 con 0x556af4ec6b40 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9446f1d740 10 client.18060 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9446f1d740 1 -- 172.21.15.5:0/3078264051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556af4f4aaf0 con 0x556af4ec6b40 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.046+0000 7f9425ffb640 1 -- 172.21.15.5:0/3078264051 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9430023cb0 con 0x556af4ec6b40 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f9425ffb640 1 -- 172.21.15.5:0/3078264051 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f9428003800 con 0x556af4ec6b40 2024-09-19T06:13:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f9446f1d740 10 client.18060 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f9446f1d740 10 client.18060 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f9446f1d740 20 client.18060 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f9446f1d740 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556af4f4b0a0 0x556af4f4d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f9446f1d740 4 client.18060 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f9446f1d740 1 -- 172.21.15.5:0/3078264051 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556af4daf170 con 0x556af4f4b0a0 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f943e59c640 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556af4f4b0a0 0x556af4f4d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f943e59c640 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556af4f4b0a0 0x556af4f4d4c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f2544d6b2fffeca2 server_cookie=1a24ff337d497bdb in_seq=0 out_seq=0 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f9425ffb640 10 client.18060 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f9425ffb640 1 -- 172.21.15.5:0/3078264051 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556af4daf170 con 0x556af4f4b0a0 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f9425ffb640 10 client.18060 handle_command_reply: tid=0 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.047+0000 7f9446f1d740 1 client.18060 shutdown 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9446f1d740 1 -- 172.21.15.5:0/3078264051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec6b40 msgr2=0x556af4ec9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9446f1d740 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec6b40 0x556af4ec9eb0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9428003c80 tx=0x7f942802a410 comp rx=0 tx=0).stop 2024-09-19T06:13:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9446f1d740 1 -- 172.21.15.5:0/3078264051 shutdown_connections 2024-09-19T06:13:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9446f1d740 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556af4f4b0a0 0x556af4f4d4c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9446f1d740 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556af4ed0370 0x556af4eca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9425ffb640 0 client.18060 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:13:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9446f1d740 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af4ec6b40 0x556af4ec9eb0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9446f1d740 1 --2- 172.21.15.5:0/3078264051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556af4ec55d0 0x556af4ed89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9446f1d740 1 -- 172.21.15.5:0/3078264051 >> 172.21.15.5:0/3078264051 conn(0x556af4e8e310 msgr2=0x556af4ec0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9446f1d740 1 -- 172.21.15.5:0/3078264051 shutdown_connections 2024-09-19T06:13:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9446f1d740 1 -- 172.21.15.5:0/3078264051 wait complete. 2024-09-19T06:13:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9446f1d740 20 client.18060 trim_cache size 0 max 16384 2024-09-19T06:13:35.056 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:13:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9444b44640 1 -- 172.21.15.5:0/1661432337 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f942c0a33b0 msgr2=0x7f942c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9444b44640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f942c0a33b0 0x7f942c0a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9438005fd0 tx=0x7f9438009500 comp rx=0 tx=0).stop 2024-09-19T06:13:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9444b44640 1 -- 172.21.15.5:0/1661432337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440102790 msgr2=0x7f9440074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9444b44640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440102790 0x7f9440074a30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9428005940 tx=0x7f94280042f0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9444b44640 1 -- 172.21.15.5:0/1661432337 shutdown_connections 2024-09-19T06:13:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9444b44640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9440106f10 0x7f9440074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9444b44640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440102790 0x7f9440074a30 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9444b44640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f942c0a33b0 0x7f942c0a5870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9444b44640 1 --2- 172.21.15.5:0/1661432337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9440101de0 0x7f94400744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.048+0000 7f9444b44640 1 -- 172.21.15.5:0/1661432337 >> 172.21.15.5:0/1661432337 conn(0x7f94400fd8c0 msgr2=0x7f94400fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.049+0000 7f9444b44640 1 -- 172.21.15.5:0/1661432337 shutdown_connections 2024-09-19T06:13:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:35.049+0000 7f9444b44640 1 -- 172.21.15.5:0/1661432337 wait complete. 2024-09-19T06:13:35.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248583 2.39 MB/sec execute 421 sec latency 4742.364 ms 2024-09-19T06:13:36.288 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:13:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:13:36.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:36 smithi005 ceph-mon[30955]: pgmap v319: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 851 KiB/s wr, 85 op/s 2024-09-19T06:13:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:36 smithi157 ceph-mon[35346]: pgmap v319: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 851 KiB/s wr, 85 op/s 2024-09-19T06:13:36.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248819 2.38 MB/sec execute 422 sec latency 14.212 ms 2024-09-19T06:13:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:36 smithi049 ceph-mon[34640]: pgmap v319: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 851 KiB/s wr, 85 op/s 2024-09-19T06:13:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.019+0000 7f0bc857f640 1 -- 172.21.15.5:0/2464085016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0101ac0 msgr2=0x7f0bc0101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.019+0000 7f0bc857f640 1 --2- 172.21.15.5:0/2464085016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0101ac0 0x7f0bc0101f20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009930 tx=0x7f0bb002a420 comp rx=0 tx=0).stop 2024-09-19T06:13:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.019+0000 7f0bc857f640 1 -- 172.21.15.5:0/2464085016 shutdown_connections 2024-09-19T06:13:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.019+0000 7f0bc857f640 1 --2- 172.21.15.5:0/2464085016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc0102460 0x7f0bc010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.019+0000 7f0bc857f640 1 --2- 172.21.15.5:0/2464085016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0101ac0 0x7f0bc0101f20 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.019+0000 7f0bc857f640 1 --2- 172.21.15.5:0/2464085016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bc0107ad0 0x7f0bc0107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.019+0000 7f0bc857f640 1 -- 172.21.15.5:0/2464085016 >> 172.21.15.5:0/2464085016 conn(0x7f0bc00fd7a0 msgr2=0x7f0bc00ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.019+0000 7f0bc857f640 1 -- 172.21.15.5:0/2464085016 shutdown_connections 2024-09-19T06:13:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.019+0000 7f0bc857f640 1 -- 172.21.15.5:0/2464085016 wait complete. 2024-09-19T06:13:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.020+0000 7f0bc857f640 1 Processor -- start 2024-09-19T06:13:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.020+0000 7f0bc857f640 1 -- start start 2024-09-19T06:13:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.020+0000 7f0bc857f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0101ac0 0x7f0bc01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.020+0000 7f0bc857f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc0102460 0x7f0bc01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.020+0000 7f0bc857f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bc0107ad0 0x7f0bc019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.020+0000 7f0bc857f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0111e10 con 0x7f0bc0101ac0 2024-09-19T06:13:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.020+0000 7f0bc857f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0111c90 con 0x7f0bc0107ad0 2024-09-19T06:13:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.020+0000 7f0bc857f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0111f90 con 0x7f0bc0102460 2024-09-19T06:13:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.020+0000 7f0bc631b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0101ac0 0x7f0bc01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.020+0000 7f0bc631b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0101ac0 0x7f0bc01983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51992/0 (socket says 172.21.15.5:51992) 2024-09-19T06:13:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.020+0000 7f0bc631b640 1 -- 172.21.15.5:0/4025630733 learned_addr learned my addr 172.21.15.5:0/4025630733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.020+0000 7f0bc6b1c640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bc0107ad0 0x7f0bc019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.021+0000 7f0bc5b1a640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc0102460 0x7f0bc01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.021+0000 7f0bc631b640 1 -- 172.21.15.5:0/4025630733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc0102460 msgr2=0x7f0bc01988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.021+0000 7f0bc631b640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc0102460 0x7f0bc01988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.021+0000 7f0bc631b640 1 -- 172.21.15.5:0/4025630733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bc0107ad0 msgr2=0x7f0bc019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.021+0000 7f0bc631b640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bc0107ad0 0x7f0bc019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.021+0000 7f0bc631b640 1 -- 172.21.15.5:0/4025630733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc019d390 con 0x7f0bc0101ac0 2024-09-19T06:13:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.021+0000 7f0bc631b640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0101ac0 0x7f0bc01983a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00d950 tx=0x7f0bbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.021+0000 7f0bb77fe640 1 -- 172.21.15.5:0/4025630733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc017070 con 0x7f0bc0101ac0 2024-09-19T06:13:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.021+0000 7f0bb77fe640 1 -- 172.21.15.5:0/4025630733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bbc009c30 con 0x7f0bc0101ac0 2024-09-19T06:13:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.021+0000 7f0bc857f640 1 -- 172.21.15.5:0/4025630733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc019d680 con 0x7f0bc0101ac0 2024-09-19T06:13:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.021+0000 7f0bb77fe640 1 -- 172.21.15.5:0/4025630733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00f620 con 0x7f0bc0101ac0 2024-09-19T06:13:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.021+0000 7f0bc857f640 1 -- 172.21.15.5:0/4025630733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc01a4f10 con 0x7f0bc0101ac0 2024-09-19T06:13:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.023+0000 7f0bb77fe640 1 -- 172.21.15.5:0/4025630733 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0bbc0109a0 con 0x7f0bc0101ac0 2024-09-19T06:13:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bcb3f5740 1 Processor -- start 2024-09-19T06:13:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bcb3f5740 1 -- start start 2024-09-19T06:13:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bcb3f5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85a89f0 0x55bdc85a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bcb3f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bcb3f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85b80c0 0x55bdc85ba4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bcb3f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bdc850d2f0 con 0x55bdc85b80c0 2024-09-19T06:13:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bcb3f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bdc84b5270 con 0x55bdc85a9310 2024-09-19T06:13:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bcb3f5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bdc85092b0 con 0x55bdc85a89f0 2024-09-19T06:13:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bc5b1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85b80c0 0x55bdc85ba4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bc5b1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85b80c0 0x55bdc85ba4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51996/0 (socket says 172.21.15.5:51996) 2024-09-19T06:13:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bc5b1a640 1 -- 172.21.15.5:0/3261371017 learned_addr learned my addr 172.21.15.5:0/3261371017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bc631b640 1 --2- 172.21.15.5:0/3261371017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.024+0000 7f0bc6b1c640 1 --2- 172.21.15.5:0/3261371017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85a89f0 0x55bdc85a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.025+0000 7f0bb77fe640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0bac0a33b0 0x7f0bac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.025+0000 7f0bc5b1a640 1 -- 172.21.15.5:0/3261371017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85a89f0 msgr2=0x55bdc85a8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.025+0000 7f0bc5b1a640 1 --2- 172.21.15.5:0/3261371017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85a89f0 0x55bdc85a8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.025+0000 7f0bc5b1a640 1 -- 172.21.15.5:0/3261371017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 msgr2=0x55bdc85b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.025+0000 7f0bc5b1a640 1 --2- 172.21.15.5:0/3261371017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.025+0000 7f0bc5b1a640 1 -- 172.21.15.5:0/3261371017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc84b4340 con 0x55bdc85b80c0 2024-09-19T06:13:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.025+0000 7f0bc6b1c640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0bac0a33b0 0x7f0bac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.025+0000 7f0bb77fe640 1 -- 172.21.15.5:0/4025630733 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f0bbc0cf860 con 0x7f0bc0101ac0 2024-09-19T06:13:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.025+0000 7f0bc5b1a640 1 --2- 172.21.15.5:0/3261371017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85b80c0 0x55bdc85ba4b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0006010 tx=0x7f0bb0004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a34873e54662b3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.025+0000 7f0bb4bfe640 1 -- 172.21.15.5:0/3261371017 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0035040 con 0x55bdc85b80c0 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.025+0000 7f0bb4bfe640 1 -- 172.21.15.5:0/3261371017 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bb00054f0 con 0x55bdc85b80c0 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.025+0000 7f0bb4bfe640 1 -- 172.21.15.5:0/3261371017 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00037f0 con 0x55bdc85b80c0 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.026+0000 7f0bc6b1c640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0bac0a33b0 0x7f0bac0a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009fd0 tx=0x7f0bb8004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.026+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/3261371017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85b80c0 msgr2=0x55bdc85ba4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.026+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/3261371017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85b80c0 0x55bdc85ba4b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0006010 tx=0x7f0bb0004170 comp rx=0 tx=0).stop 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.026+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/3261371017 shutdown_connections 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.026+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/3261371017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85b80c0 0x55bdc85ba4b0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.026+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/3261371017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.026+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/3261371017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85a89f0 0x55bdc85a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.026+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/3261371017 >> 172.21.15.5:0/3261371017 conn(0x55bdc85a5060 msgr2=0x55bdc85a5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.026+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/3261371017 shutdown_connections 2024-09-19T06:13:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.026+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/3261371017 wait complete. 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.026+0000 7f0bcb3f5740 1 -- start start 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bcb3f5740 1 Processor -- start 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bcb3f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a89f0 0x55bdc85b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bcb3f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a9310 0x55bdc85b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bcb3f5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bcb3f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bdc85bacc0 con 0x55bdc85a9310 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bcb3f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bdc8496290 con 0x55bdc85a89f0 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bcb3f5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bdc85bae40 con 0x55bdc85b80c0 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bcb3f5740 10 client.0 _ll_register_callbacks cb 0x55bdc84c7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bc5b1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a9310 0x55bdc85b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bc5b1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a9310 0x55bdc85b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52000/0 (socket says 172.21.15.5:52000) 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bc631b640 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a89f0 0x55bdc85b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bc5b1a640 1 -- 172.21.15.5:0/1784585160 learned_addr learned my addr 172.21.15.5:0/1784585160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bc6b1c640 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bc5b1a640 1 -- 172.21.15.5:0/1784585160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 msgr2=0x55bdc85b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bc5b1a640 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bc5b1a640 1 -- 172.21.15.5:0/1784585160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a89f0 msgr2=0x55bdc85b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bc5b1a640 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a89f0 0x55bdc85b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bc5b1a640 1 -- 172.21.15.5:0/1784585160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdc8220210 con 0x55bdc85a9310 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.027+0000 7f0bc631b640 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a89f0 0x55bdc85b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0bc5b1a640 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a9310 0x55bdc85b54b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0004650 tx=0x7f0bb00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d549981ad01f3413 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0ba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0ba9ffb640 1 -- 172.21.15.5:0/1784585160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0053060 con 0x55bdc85a9310 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0ba9ffb640 1 -- 172.21.15.5:0/1784585160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f0bb0004da0 con 0x55bdc85a9310 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1784585160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb00054f0 con 0x55bdc85a9310 2024-09-19T06:13:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0ba9ffb640 1 -- 172.21.15.5:0/1784585160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0bb0035050 con 0x55bdc85a9310 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0bcb3f5740 10 client.18072 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1784585160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bdc82ec620 con 0x55bdc85a9310 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0ba9ffb640 1 -- 172.21.15.5:0/1784585160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bb00054f0 con 0x55bdc85a9310 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0ba9ffb640 1 -- 172.21.15.5:0/1784585160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f0bb00314a0 con 0x55bdc85a9310 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0bcb3f5740 10 client.18072 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0bcb3f5740 10 client.18072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0bcb3f5740 20 client.18072 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdc85c7e90 0x55bdc85ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0bcb3f5740 4 client.18072 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.028+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1784585160 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdc851aa20 con 0x55bdc85c7e90 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.029+0000 7f0bc631b640 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdc85c7e90 0x55bdc85ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.029+0000 7f0bc631b640 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdc85c7e90 0x55bdc85ca280 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51cce2e97d586e80 server_cookie=881eaf740ca96ab3 in_seq=0 out_seq=0 2024-09-19T06:13:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.029+0000 7f0ba9ffb640 10 client.18072 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0ba9ffb640 1 -- 172.21.15.5:0/1784585160 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdc851aa20 con 0x55bdc85c7e90 2024-09-19T06:13:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0ba9ffb640 10 client.18072 handle_command_reply: tid=0 2024-09-19T06:13:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0bcb3f5740 1 client.18072 shutdown 2024-09-19T06:13:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1784585160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a9310 msgr2=0x55bdc85b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a9310 0x55bdc85b54b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0004650 tx=0x7f0bb00058d0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1784585160 shutdown_connections 2024-09-19T06:13:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdc85c7e90 0x55bdc85ca280 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0ba9ffb640 0 client.18072 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a9310 0x55bdc85b54b0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/1784585160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a89f0 0x55bdc85b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1784585160 >> 172.21.15.5:0/1784585160 conn(0x55bdc85a5060 msgr2=0x55bdc85aadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1784585160 shutdown_connections 2024-09-19T06:13:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1784585160 wait complete. 2024-09-19T06:13:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.030+0000 7f0bcb3f5740 20 client.18072 trim_cache size 0 max 16384 2024-09-19T06:13:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.044+0000 7f0bcb3f5740 1 Processor -- start 2024-09-19T06:13:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.044+0000 7f0bcb3f5740 1 -- start start 2024-09-19T06:13:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.044+0000 7f0bcb3f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 0x55bdc85cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.044+0000 7f0bcb3f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.044+0000 7f0bcb3f5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.044+0000 7f0bcb3f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01ab670 con 0x55bdc85a89f0 2024-09-19T06:13:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.044+0000 7f0bcb3f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01ad270 con 0x55bdc85a9310 2024-09-19T06:13:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.044+0000 7f0bcb3f5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01abd40 con 0x55bdc85b80c0 2024-09-19T06:13:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc631b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 0x55bdc85cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc631b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 0x55bdc85cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52008/0 (socket says 172.21.15.5:52008) 2024-09-19T06:13:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc631b640 1 -- 172.21.15.5:0/3305962937 learned_addr learned my addr 172.21.15.5:0/3305962937 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc5b1a640 1 --2- 172.21.15.5:0/3305962937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc6b1c640 1 --2- 172.21.15.5:0/3305962937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc631b640 1 -- 172.21.15.5:0/3305962937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 msgr2=0x55bdc85b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc631b640 1 --2- 172.21.15.5:0/3305962937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc631b640 1 -- 172.21.15.5:0/3305962937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 msgr2=0x55bdc85cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc631b640 1 --2- 172.21.15.5:0/3305962937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc631b640 1 -- 172.21.15.5:0/3305962937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc85afd50 con 0x55bdc85a89f0 2024-09-19T06:13:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc5b1a640 1 --2- 172.21.15.5:0/3305962937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:13:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc6b1c640 1 --2- 172.21.15.5:0/3305962937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0bc631b640 1 --2- 172.21.15.5:0/3305962937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 0x55bdc85cf850 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc09bb60 tx=0x7f0bbc0d5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bc0604a9716724 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0babfff640 1 -- 172.21.15.5:0/3305962937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc020040 con 0x55bdc85a89f0 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0babfff640 1 -- 172.21.15.5:0/3305962937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bbc09ce50 con 0x55bdc85a89f0 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.045+0000 7f0babfff640 1 -- 172.21.15.5:0/3305962937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc098240 con 0x55bdc85a89f0 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/3305962937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 msgr2=0x55bdc85cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/3305962937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 0x55bdc85cf850 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc09bb60 tx=0x7f0bbc0d5550 comp rx=0 tx=0).stop 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/3305962937 shutdown_connections 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/3305962937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/3305962937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/3305962937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 0x55bdc85cf850 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/3305962937 >> 172.21.15.5:0/3305962937 conn(0x55bdc85a5060 msgr2=0x55bdc85b6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/3305962937 shutdown_connections 2024-09-19T06:13:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/3305962937 wait complete. 2024-09-19T06:13:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 -- start start 2024-09-19T06:13:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 Processor -- start 2024-09-19T06:13:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 0x55bdc85bbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85bc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01acba0 con 0x55bdc85a89f0 2024-09-19T06:13:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01ac470 con 0x55bdc85a9310 2024-09-19T06:13:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01a9670 con 0x55bdc85b80c0 2024-09-19T06:13:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.046+0000 7f0bcb3f5740 10 client.0 _ll_register_callbacks cb 0x7f0bbc0d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bc631b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 0x55bdc85bbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bc631b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 0x55bdc85bbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52020/0 (socket says 172.21.15.5:52020) 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bc631b640 1 -- 172.21.15.5:0/1954011199 learned_addr learned my addr 172.21.15.5:0/1954011199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bc5b1a640 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85bc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bc6b1c640 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bc631b640 1 -- 172.21.15.5:0/1954011199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 msgr2=0x55bdc85bc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bc631b640 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85bc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bc631b640 1 -- 172.21.15.5:0/1954011199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 msgr2=0x55bdc85bc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bc631b640 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85bc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bc631b640 1 -- 172.21.15.5:0/1954011199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdc8631870 con 0x55bdc85a89f0 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bc6b1c640 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85bc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bc631b640 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 0x55bdc85bbe60 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc098030 tx=0x7f0bbc0d55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c59435cdf84bf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0ba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0ba9ffb640 1 -- 172.21.15.5:0/1954011199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc020070 con 0x55bdc85a89f0 2024-09-19T06:13:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0ba9ffb640 1 -- 172.21.15.5:0/1954011199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f0bbc0860b0 con 0x55bdc85a89f0 2024-09-19T06:13:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1954011199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bbc09ce50 con 0x55bdc85a89f0 2024-09-19T06:13:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0ba9ffb640 1 -- 172.21.15.5:0/1954011199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0bbc09bbd0 con 0x55bdc85a89f0 2024-09-19T06:13:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bcb3f5740 10 client.18078 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1954011199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bdc8632ad0 con 0x55bdc85a89f0 2024-09-19T06:13:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.047+0000 7f0ba9ffb640 1 -- 172.21.15.5:0/1954011199 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bbc09ce50 con 0x55bdc85a89f0 2024-09-19T06:13:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.048+0000 7f0ba9ffb640 1 -- 172.21.15.5:0/1954011199 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f0bbc0ba330 con 0x55bdc85a89f0 2024-09-19T06:13:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.048+0000 7f0bcb3f5740 10 client.18078 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.048+0000 7f0bcb3f5740 10 client.18078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.048+0000 7f0bcb3f5740 20 client.18078 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.048+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdc86330f0 0x55bdc8635510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.048+0000 7f0bcb3f5740 4 client.18078 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:13:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.048+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1954011199 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bdc8514d70 con 0x55bdc86330f0 2024-09-19T06:13:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.048+0000 7f0bc5b1a640 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdc86330f0 0x55bdc8635510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.048+0000 7f0ba9ffb640 10 client.18078 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.048+0000 7f0bc5b1a640 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdc86330f0 0x55bdc8635510 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac928462dc21502 server_cookie=652ddd91b7a53096 in_seq=0 out_seq=0 2024-09-19T06:13:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0ba9ffb640 1 -- 172.21.15.5:0/1954011199 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bdc8514d70 con 0x55bdc86330f0 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0ba9ffb640 10 client.18078 handle_command_reply: tid=0 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0bcb3f5740 1 client.18078 shutdown 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1954011199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 msgr2=0x55bdc85bbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 0x55bdc85bbe60 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc098030 tx=0x7f0bbc0d55c0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1954011199 shutdown_connections 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdc86330f0 0x55bdc8635510 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdc85b80c0 0x55bdc85bc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdc85a9310 0x55bdc85bc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0bcb3f5740 1 --2- 172.21.15.5:0/1954011199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdc85a89f0 0x55bdc85bbe60 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1954011199 >> 172.21.15.5:0/1954011199 conn(0x55bdc85a5060 msgr2=0x55bdc85aad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0ba9ffb640 0 client.18078 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1954011199 shutdown_connections 2024-09-19T06:13:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0bcb3f5740 1 -- 172.21.15.5:0/1954011199 wait complete. 2024-09-19T06:13:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.074+0000 7f0bcb3f5740 20 client.18078 trim_cache size 0 max 16384 2024-09-19T06:13:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:13:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:13:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:13:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:13:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.075+0000 7f0bc857f640 1 -- 172.21.15.5:0/4025630733 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0bac0a33b0 msgr2=0x7f0bac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.075+0000 7f0bc857f640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0bac0a33b0 0x7f0bac0a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009fd0 tx=0x7f0bb8004a40 comp rx=0 tx=0).stop 2024-09-19T06:13:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.075+0000 7f0bc857f640 1 -- 172.21.15.5:0/4025630733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0101ac0 msgr2=0x7f0bc01983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.075+0000 7f0bc857f640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0101ac0 0x7f0bc01983a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00d950 tx=0x7f0bbc00de10 comp rx=0 tx=0).stop 2024-09-19T06:13:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.075+0000 7f0bc857f640 1 -- 172.21.15.5:0/4025630733 shutdown_connections 2024-09-19T06:13:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.075+0000 7f0bc857f640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bc0107ad0 0x7f0bc019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.075+0000 7f0bc857f640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0bac0a33b0 0x7f0bac0a5870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.075+0000 7f0bc857f640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc0102460 0x7f0bc01988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.075+0000 7f0bc857f640 1 --2- 172.21.15.5:0/4025630733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0101ac0 0x7f0bc01983a0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.075+0000 7f0bc857f640 1 -- 172.21.15.5:0/4025630733 >> 172.21.15.5:0/4025630733 conn(0x7f0bc00fd7a0 msgr2=0x7f0bc00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.075+0000 7f0bc857f640 1 -- 172.21.15.5:0/4025630733 shutdown_connections 2024-09-19T06:13:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:37.075+0000 7f0bc857f640 1 -- 172.21.15.5:0/4025630733 wait complete. 2024-09-19T06:13:37.653 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:37 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:13:37.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.746+0000 7f4a7ad75640 1 -- 172.21.15.5:0/80996239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a74102570 msgr2=0x7f4a741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.746+0000 7f4a7ad75640 1 --2- 172.21.15.5:0/80996239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a74102570 0x7f4a741029b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009900 tx=0x7f4a6002a480 comp rx=0 tx=0).stop 2024-09-19T06:13:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.746+0000 7f4a7ad75640 1 -- 172.21.15.5:0/80996239 shutdown_connections 2024-09-19T06:13:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.746+0000 7f4a7ad75640 1 --2- 172.21.15.5:0/80996239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a74102ef0 0x7f4a7410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.746+0000 7f4a7ad75640 1 --2- 172.21.15.5:0/80996239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a74102570 0x7f4a741029b0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.746+0000 7f4a7ad75640 1 --2- 172.21.15.5:0/80996239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a74100e20 0x7f4a74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.746+0000 7f4a7ad75640 1 -- 172.21.15.5:0/80996239 >> 172.21.15.5:0/80996239 conn(0x7f4a740fc7d0 msgr2=0x7f4a740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.746+0000 7f4a7ad75640 1 -- 172.21.15.5:0/80996239 shutdown_connections 2024-09-19T06:13:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.746+0000 7f4a7ad75640 1 -- 172.21.15.5:0/80996239 wait complete. 2024-09-19T06:13:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a7ad75640 1 Processor -- start 2024-09-19T06:13:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a7ad75640 1 -- start start 2024-09-19T06:13:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a7ad75640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a74100e20 0x7f4a7419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a7ad75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a74102570 0x7f4a7419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a7ad75640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a74102ef0 0x7f4a7419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a7ad75640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74110e30 con 0x7f4a74102570 2024-09-19T06:13:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a7ad75640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74110cb0 con 0x7f4a74102ef0 2024-09-19T06:13:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a7ad75640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74110fb0 con 0x7f4a74100e20 2024-09-19T06:13:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a73fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a74102570 0x7f4a7419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a73fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a74102570 0x7f4a7419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52040/0 (socket says 172.21.15.5:52040) 2024-09-19T06:13:37.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a73fff640 1 -- 172.21.15.5:0/1048303394 learned_addr learned my addr 172.21.15.5:0/1048303394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a79312640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a74102ef0 0x7f4a7419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a78b11640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a74100e20 0x7f4a7419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a73fff640 1 -- 172.21.15.5:0/1048303394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a74100e20 msgr2=0x7f4a7419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a73fff640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a74100e20 0x7f4a7419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a73fff640 1 -- 172.21.15.5:0/1048303394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a74102ef0 msgr2=0x7f4a7419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a73fff640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a74102ef0 0x7f4a7419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.747+0000 7f4a73fff640 1 -- 172.21.15.5:0/1048303394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7419f5a0 con 0x7f4a74102570 2024-09-19T06:13:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.748+0000 7f4a73fff640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a74102570 0x7f4a7419ab30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4a60006010 tx=0x7f4a600040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.748+0000 7f4a71ffb640 1 -- 172.21.15.5:0/1048303394 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6003f030 con 0x7f4a74102570 2024-09-19T06:13:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.748+0000 7f4a71ffb640 1 -- 172.21.15.5:0/1048303394 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a600044c0 con 0x7f4a74102570 2024-09-19T06:13:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.748+0000 7f4a7ad75640 1 -- 172.21.15.5:0/1048303394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7419f830 con 0x7f4a74102570 2024-09-19T06:13:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.748+0000 7f4a71ffb640 1 -- 172.21.15.5:0/1048303394 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60035070 con 0x7f4a74102570 2024-09-19T06:13:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.748+0000 7f4a7ad75640 1 -- 172.21.15.5:0/1048303394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a741a7170 con 0x7f4a74102570 2024-09-19T06:13:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.750+0000 7f4a71ffb640 1 -- 172.21.15.5:0/1048303394 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4a60042030 con 0x7f4a74102570 2024-09-19T06:13:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a7d14e740 1 Processor -- start 2024-09-19T06:13:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a7d14e740 1 -- start start 2024-09-19T06:13:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a7d14e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 0x55929c2e1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a7d14e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2e15d0 0x55929c2e1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a7d14e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2ec370 0x55929c2ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a7d14e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55929c2472f0 con 0x55929c2e2b40 2024-09-19T06:13:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a7d14e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55929c1eecf0 con 0x55929c2ec370 2024-09-19T06:13:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a7d14e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55929c2432b0 con 0x55929c2e15d0 2024-09-19T06:13:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a79312640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 0x55929c2e1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a79312640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 0x55929c2e1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52046/0 (socket says 172.21.15.5:52046) 2024-09-19T06:13:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a79312640 1 -- 172.21.15.5:0/1473601971 learned_addr learned my addr 172.21.15.5:0/1473601971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a71ffb640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a640a33b0 0x7f4a640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a78b11640 1 --2- 172.21.15.5:0/1473601971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2e15d0 0x55929c2e1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.751+0000 7f4a73fff640 1 --2- 172.21.15.5:0/1473601971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2ec370 0x55929c2ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a71ffb640 1 -- 172.21.15.5:0/1048303394 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f4a600effe0 con 0x7f4a74102570 2024-09-19T06:13:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a79312640 1 -- 172.21.15.5:0/1473601971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2e15d0 msgr2=0x55929c2e1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a79312640 1 --2- 172.21.15.5:0/1473601971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2e15d0 0x55929c2e1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a79312640 1 -- 172.21.15.5:0/1473601971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2ec370 msgr2=0x55929c2ee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a79312640 1 --2- 172.21.15.5:0/1473601971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2ec370 0x55929c2ee760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a79312640 1 -- 172.21.15.5:0/1473601971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55929c1d33a0 con 0x55929c2e2b40 2024-09-19T06:13:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a73fff640 1 --2- 172.21.15.5:0/1473601971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2ec370 0x55929c2ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:13:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a79312640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a640a33b0 0x7f4a640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a79312640 1 --2- 172.21.15.5:0/1473601971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 0x55929c2e1090 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009fd0 tx=0x7f4a6c0139f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d8c124d17ecf44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a5effd640 1 -- 172.21.15.5:0/1473601971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00f5e0 con 0x55929c2e2b40 2024-09-19T06:13:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a5effd640 1 -- 172.21.15.5:0/1473601971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a6c013150 con 0x55929c2e2b40 2024-09-19T06:13:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a79312640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a640a33b0 0x7f4a640a5870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c023000 tx=0x7f4a6c020000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.752+0000 7f4a5effd640 1 -- 172.21.15.5:0/1473601971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00e680 con 0x55929c2e2b40 2024-09-19T06:13:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.753+0000 7f4a7d14e740 1 -- 172.21.15.5:0/1473601971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 msgr2=0x55929c2e1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.753+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/1473601971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 0x55929c2e1090 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009fd0 tx=0x7f4a6c0139f0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.753+0000 7f4a7d14e740 1 -- 172.21.15.5:0/1473601971 shutdown_connections 2024-09-19T06:13:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.753+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/1473601971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2ec370 0x55929c2ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.753+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/1473601971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2e15d0 0x55929c2e1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.753+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/1473601971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 0x55929c2e1090 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.753+0000 7f4a7d14e740 1 -- 172.21.15.5:0/1473601971 >> 172.21.15.5:0/1473601971 conn(0x55929c2aa310 msgr2=0x55929c2ddc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.753+0000 7f4a7d14e740 1 -- 172.21.15.5:0/1473601971 shutdown_connections 2024-09-19T06:13:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.753+0000 7f4a7d14e740 1 -- 172.21.15.5:0/1473601971 wait complete. 2024-09-19T06:13:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.754+0000 7f4a7d14e740 1 -- start start 2024-09-19T06:13:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.754+0000 7f4a7d14e740 1 Processor -- start 2024-09-19T06:13:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.754+0000 7f4a7d14e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e15d0 0x55929c2ebd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.754+0000 7f4a7d14e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2e2b40 0x55929c2e9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.754+0000 7f4a7d14e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2ec370 0x55929c2e9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.754+0000 7f4a7d14e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55929c2ef070 con 0x55929c2ec370 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.754+0000 7f4a7d14e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55929c202060 con 0x55929c2e15d0 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.754+0000 7f4a7d14e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55929c2ef1f0 con 0x55929c2e2b40 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.754+0000 7f4a7d14e740 10 client.0 _ll_register_callbacks cb 0x55929bf558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.754+0000 7f4a79312640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2ec370 0x55929c2e9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.754+0000 7f4a79312640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2ec370 0x55929c2e9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52058/0 (socket says 172.21.15.5:52058) 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a79312640 1 -- 172.21.15.5:0/2709741735 learned_addr learned my addr 172.21.15.5:0/2709741735 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a78b11640 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e15d0 0x55929c2ebd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a73fff640 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2e2b40 0x55929c2e9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a79312640 1 -- 172.21.15.5:0/2709741735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2e2b40 msgr2=0x55929c2e9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a79312640 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2e2b40 0x55929c2e9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a79312640 1 -- 172.21.15.5:0/2709741735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e15d0 msgr2=0x55929c2ebd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a79312640 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e15d0 0x55929c2ebd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a79312640 1 -- 172.21.15.5:0/2709741735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55929bfb83c0 con 0x55929c2ec370 2024-09-19T06:13:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a79312640 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2ec370 0x55929c2e9640 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00ffd0 tx=0x7f4a6c00e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130751a1bee3d400 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a53fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a53fff640 1 -- 172.21.15.5:0/2709741735 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c004e10 con 0x55929c2ec370 2024-09-19T06:13:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a53fff640 1 -- 172.21.15.5:0/2709741735 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f4a6c00e9d0 con 0x55929c2ec370 2024-09-19T06:13:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.755+0000 7f4a7d14e740 1 -- 172.21.15.5:0/2709741735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6c013150 con 0x55929c2ec370 2024-09-19T06:13:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.756+0000 7f4a53fff640 1 -- 172.21.15.5:0/2709741735 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4a6c037050 con 0x55929c2ec370 2024-09-19T06:13:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.756+0000 7f4a7d14e740 10 client.18090 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.756+0000 7f4a7d14e740 1 -- 172.21.15.5:0/2709741735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55929c2faab0 con 0x55929c2ec370 2024-09-19T06:13:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.756+0000 7f4a53fff640 1 -- 172.21.15.5:0/2709741735 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a6c013150 con 0x55929c2ec370 2024-09-19T06:13:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.756+0000 7f4a53fff640 1 -- 172.21.15.5:0/2709741735 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f4a6c005050 con 0x55929c2ec370 2024-09-19T06:13:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.756+0000 7f4a7d14e740 10 client.18090 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.756+0000 7f4a7d14e740 10 client.18090 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.756+0000 7f4a7d14e740 20 client.18090 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.756+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55929c2fbe00 0x55929c2fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.756+0000 7f4a7d14e740 4 client.18090 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.756+0000 7f4a7d14e740 1 -- 172.21.15.5:0/2709741735 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55929c254860 con 0x55929c2fbe00 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.756+0000 7f4a78b11640 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55929c2fbe00 0x55929c2fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.757+0000 7f4a78b11640 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55929c2fbe00 0x55929c2fe1f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8e9bc436ee713ab9 server_cookie=f3e10f3229d8c6bf in_seq=0 out_seq=0 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.757+0000 7f4a53fff640 10 client.18090 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a53fff640 1 -- 172.21.15.5:0/2709741735 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55929c254860 con 0x55929c2fbe00 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a53fff640 10 client.18090 handle_command_reply: tid=0 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a7d14e740 1 client.18090 shutdown 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a7d14e740 1 -- 172.21.15.5:0/2709741735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2ec370 msgr2=0x55929c2e9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2ec370 0x55929c2e9640 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00ffd0 tx=0x7f4a6c00e650 comp rx=0 tx=0).stop 2024-09-19T06:13:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a7d14e740 1 -- 172.21.15.5:0/2709741735 shutdown_connections 2024-09-19T06:13:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55929c2fbe00 0x55929c2fe1f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2ec370 0x55929c2e9640 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2e2b40 0x55929c2e9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/2709741735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e15d0 0x55929c2ebd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a7d14e740 1 -- 172.21.15.5:0/2709741735 >> 172.21.15.5:0/2709741735 conn(0x55929c2aa310 msgr2=0x55929c2e3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a53fff640 0 client.18090 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:13:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a7d14e740 1 -- 172.21.15.5:0/2709741735 shutdown_connections 2024-09-19T06:13:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a7d14e740 1 -- 172.21.15.5:0/2709741735 wait complete. 2024-09-19T06:13:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.758+0000 7f4a7d14e740 20 client.18090 trim_cache size 0 max 16384 2024-09-19T06:13:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a7d14e740 1 Processor -- start 2024-09-19T06:13:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a7d14e740 1 -- start start 2024-09-19T06:13:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a7d14e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e15d0 0x55929c2f8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a7d14e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 0x55929c2e8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a7d14e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2ec370 0x55929c2e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a7d14e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741afaa0 con 0x55929c2e2b40 2024-09-19T06:13:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a7d14e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b16a0 con 0x55929c2e15d0 2024-09-19T06:13:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a7d14e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b0170 con 0x55929c2ec370 2024-09-19T06:13:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a73fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 0x55929c2e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a73fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 0x55929c2e8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52072/0 (socket says 172.21.15.5:52072) 2024-09-19T06:13:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a73fff640 1 -- 172.21.15.5:0/3640330043 learned_addr learned my addr 172.21.15.5:0/3640330043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a78b11640 1 --2- 172.21.15.5:0/3640330043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e15d0 0x55929c2f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.773+0000 7f4a79312640 1 --2- 172.21.15.5:0/3640330043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2ec370 0x55929c2e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.774+0000 7f4a73fff640 1 -- 172.21.15.5:0/3640330043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2ec370 msgr2=0x55929c2e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.774+0000 7f4a73fff640 1 --2- 172.21.15.5:0/3640330043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2ec370 0x55929c2e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.774+0000 7f4a73fff640 1 -- 172.21.15.5:0/3640330043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e15d0 msgr2=0x55929c2f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.774+0000 7f4a73fff640 1 --2- 172.21.15.5:0/3640330043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e15d0 0x55929c2f8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.774+0000 7f4a73fff640 1 -- 172.21.15.5:0/3640330043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55929c2e12d0 con 0x55929c2e2b40 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.774+0000 7f4a78b11640 1 --2- 172.21.15.5:0/3640330043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e15d0 0x55929c2f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.774+0000 7f4a73fff640 1 --2- 172.21.15.5:0/3640330043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 0x55929c2e8010 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4a600b8d20 tx=0x7f4a6008b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbdd9cc7f587106 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.774+0000 7f4a5e7fc640 1 -- 172.21.15.5:0/3640330043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a600ede60 con 0x55929c2e2b40 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.774+0000 7f4a5e7fc640 1 -- 172.21.15.5:0/3640330043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a600c0650 con 0x55929c2e2b40 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.774+0000 7f4a5e7fc640 1 -- 172.21.15.5:0/3640330043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60049040 con 0x55929c2e2b40 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.775+0000 7f4a7d14e740 1 -- 172.21.15.5:0/3640330043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 msgr2=0x55929c2e8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.775+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/3640330043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 0x55929c2e8010 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4a600b8d20 tx=0x7f4a6008b040 comp rx=0 tx=0).stop 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.775+0000 7f4a7d14e740 1 -- 172.21.15.5:0/3640330043 shutdown_connections 2024-09-19T06:13:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.775+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/3640330043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2ec370 0x55929c2e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.775+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/3640330043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e2b40 0x55929c2e8010 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.775+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/3640330043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e15d0 0x55929c2f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.775+0000 7f4a7d14e740 1 -- 172.21.15.5:0/3640330043 >> 172.21.15.5:0/3640330043 conn(0x55929c2aa310 msgr2=0x55929c2e7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.775+0000 7f4a7d14e740 1 -- 172.21.15.5:0/3640330043 shutdown_connections 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.775+0000 7f4a7d14e740 1 -- 172.21.15.5:0/3640330043 wait complete. 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.775+0000 7f4a7d14e740 1 -- start start 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a7d14e740 1 Processor -- start 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a7d14e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e15d0 0x55929c2ef440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a7d14e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e2b40 0x55929c2ef980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a7d14e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2ec370 0x55929c2efec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a7d14e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b0fd0 con 0x55929c2e15d0 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a7d14e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b08a0 con 0x55929c2e2b40 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a7d14e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741adaa0 con 0x55929c2ec370 2024-09-19T06:13:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a7d14e740 10 client.0 _ll_register_callbacks cb 0x7f4a680097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a78b11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e15d0 0x55929c2ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a78b11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e15d0 0x55929c2ef440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52082/0 (socket says 172.21.15.5:52082) 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a78b11640 1 -- 172.21.15.5:0/614447966 learned_addr learned my addr 172.21.15.5:0/614447966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a73fff640 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e2b40 0x55929c2ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.776+0000 7f4a79312640 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2ec370 0x55929c2efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a78b11640 1 -- 172.21.15.5:0/614447966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2ec370 msgr2=0x55929c2efec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a78b11640 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2ec370 0x55929c2efec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a78b11640 1 -- 172.21.15.5:0/614447966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e2b40 msgr2=0x55929c2ef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a78b11640 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e2b40 0x55929c2ef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a78b11640 1 -- 172.21.15.5:0/614447966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55929c2de960 con 0x55929c2e15d0 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a78b11640 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e15d0 0x55929c2ef440 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x55929c2fe9c0 tx=0x7f4a6801f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37323104239834b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a53fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a53fff640 1 -- 172.21.15.5:0/614447966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68004310 con 0x55929c2e15d0 2024-09-19T06:13:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a53fff640 1 -- 172.21.15.5:0/614447966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f4a680035f0 con 0x55929c2e15d0 2024-09-19T06:13:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a7d14e740 1 -- 172.21.15.5:0/614447966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a600c0650 con 0x55929c2e15d0 2024-09-19T06:13:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a53fff640 1 -- 172.21.15.5:0/614447966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4a680044b0 con 0x55929c2e15d0 2024-09-19T06:13:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a7d14e740 10 client.18096 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a7d14e740 1 -- 172.21.15.5:0/614447966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55929c365d50 con 0x55929c2e15d0 2024-09-19T06:13:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.777+0000 7f4a53fff640 1 -- 172.21.15.5:0/614447966 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a600c0650 con 0x55929c2e15d0 2024-09-19T06:13:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a53fff640 1 -- 172.21.15.5:0/614447966 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f4a68010b90 con 0x55929c2e15d0 2024-09-19T06:13:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a7d14e740 10 client.18096 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a7d14e740 10 client.18096 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a7d14e740 20 client.18096 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55929c366500 0x55929c368920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a7d14e740 4 client.18096 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a7d14e740 1 -- 172.21.15.5:0/614447966 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55929c24ed70 con 0x55929c366500 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a73fff640 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55929c366500 0x55929c368920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a73fff640 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55929c366500 0x55929c368920 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=80221828fc3a042e server_cookie=98f2708945ad4cad in_seq=0 out_seq=0 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a53fff640 10 client.18096 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a53fff640 1 -- 172.21.15.5:0/614447966 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55929c24ed70 con 0x55929c366500 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a53fff640 10 client.18096 handle_command_reply: tid=0 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.778+0000 7f4a7d14e740 1 client.18096 shutdown 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.779+0000 7f4a7d14e740 1 -- 172.21.15.5:0/614447966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e15d0 msgr2=0x55929c2ef440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.779+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e15d0 0x55929c2ef440 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x55929c2fe9c0 tx=0x7f4a6801f830 comp rx=0 tx=0).stop 2024-09-19T06:13:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.779+0000 7f4a7d14e740 1 -- 172.21.15.5:0/614447966 shutdown_connections 2024-09-19T06:13:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.779+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55929c366500 0x55929c368920 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.779+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55929c2ec370 0x55929c2efec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.779+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55929c2e2b40 0x55929c2ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.779+0000 7f4a7d14e740 1 --2- 172.21.15.5:0/614447966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55929c2e15d0 0x55929c2ef440 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.779+0000 7f4a7d14e740 1 -- 172.21.15.5:0/614447966 >> 172.21.15.5:0/614447966 conn(0x55929c2aa310 msgr2=0x55929c2e8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.779+0000 7f4a53fff640 0 client.18096 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:13:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.779+0000 7f4a7d14e740 1 -- 172.21.15.5:0/614447966 shutdown_connections 2024-09-19T06:13:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.779+0000 7f4a7d14e740 1 -- 172.21.15.5:0/614447966 wait complete. 2024-09-19T06:13:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.779+0000 7f4a7d14e740 20 client.18096 trim_cache size 0 max 16384 2024-09-19T06:13:37.798 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:13:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.780+0000 7f4a7ad75640 1 -- 172.21.15.5:0/1048303394 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a640a33b0 msgr2=0x7f4a640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.780+0000 7f4a7ad75640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a640a33b0 0x7f4a640a5870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c023000 tx=0x7f4a6c020000 comp rx=0 tx=0).stop 2024-09-19T06:13:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.780+0000 7f4a7ad75640 1 -- 172.21.15.5:0/1048303394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a74102570 msgr2=0x7f4a7419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.780+0000 7f4a7ad75640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a74102570 0x7f4a7419ab30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4a60006010 tx=0x7f4a600040c0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.780+0000 7f4a7ad75640 1 -- 172.21.15.5:0/1048303394 shutdown_connections 2024-09-19T06:13:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.780+0000 7f4a7ad75640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a74102ef0 0x7f4a7419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.780+0000 7f4a7ad75640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a74102570 0x7f4a7419ab30 secure :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4a60006010 tx=0x7f4a600040c0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.780+0000 7f4a7ad75640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4a640a33b0 0x7f4a640a5870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.780+0000 7f4a7ad75640 1 --2- 172.21.15.5:0/1048303394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a74100e20 0x7f4a7419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.780+0000 7f4a7ad75640 1 -- 172.21.15.5:0/1048303394 >> 172.21.15.5:0/1048303394 conn(0x7f4a740fc7d0 msgr2=0x7f4a74103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.780+0000 7f4a7ad75640 1 -- 172.21.15.5:0/1048303394 shutdown_connections 2024-09-19T06:13:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:37.780+0000 7f4a7ad75640 1 -- 172.21.15.5:0/1048303394 wait complete. 2024-09-19T06:13:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:37 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:13:37.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 249156 2.38 MB/sec execute 423 sec latency 60.342 ms 2024-09-19T06:13:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:37 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:13:38.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:13:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:38 smithi157 ceph-mon[35346]: pgmap v320: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:13:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:38 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:38 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:38 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:38 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:38 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:38 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:38 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:38 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:38.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 249714 2.39 MB/sec execute 424 sec latency 14.881 ms 2024-09-19T06:13:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:38 smithi005 ceph-mon[30955]: pgmap v320: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:13:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:38 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:38 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:38 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:38 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:38 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:38 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:38 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:38 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:38 smithi049 ceph-mon[34640]: pgmap v320: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:13:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:38 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:38 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:38 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:38 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:38 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:38 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:38.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:38 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:38.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:38 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:13:39.770 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:39 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T06:13:39.770 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:39 smithi005 ceph-mon[30955]: Unable to set osd_memory_target on smithi157 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-19T06:13:39.770 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:39 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:39 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:39 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:39 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:39 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T06:13:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:39 smithi157 ceph-mon[35346]: Unable to set osd_memory_target on smithi157 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-19T06:13:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:39 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:39 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:39 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:39 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.840 INFO:tasks.workunit.client.0.smithi005.stdout: 1 250167 2.38 MB/sec execute 425 sec latency 41.613 ms 2024-09-19T06:13:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:39 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T06:13:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:39 smithi049 ceph-mon[34640]: Unable to set osd_memory_target on smithi157 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-19T06:13:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:39 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:39 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:39 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:39 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:39.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.984+0000 7f03153b2640 1 -- 172.21.15.5:0/2266589411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0310106f10 msgr2=0x7f031010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:39.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.984+0000 7f03153b2640 1 --2- 172.21.15.5:0/2266589411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0310106f10 0x7f031010ea90 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0308009ca0 tx=0x7f030802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:13:39.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.984+0000 7f03153b2640 1 -- 172.21.15.5:0/2266589411 shutdown_connections 2024-09-19T06:13:39.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.984+0000 7f03153b2640 1 --2- 172.21.15.5:0/2266589411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0310106f10 0x7f031010ea90 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.984+0000 7f03153b2640 1 --2- 172.21.15.5:0/2266589411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0310102790 0x7f03101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.984+0000 7f03153b2640 1 --2- 172.21.15.5:0/2266589411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0310101de0 0x7f03101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.984+0000 7f03153b2640 1 -- 172.21.15.5:0/2266589411 >> 172.21.15.5:0/2266589411 conn(0x7f03100fd8c0 msgr2=0x7f03100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.984+0000 7f03153b2640 1 -- 172.21.15.5:0/2266589411 shutdown_connections 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.984+0000 7f03153b2640 1 -- 172.21.15.5:0/2266589411 wait complete. 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f03153b2640 1 Processor -- start 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f03153b2640 1 -- start start 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f03153b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0310101de0 0x7f03100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f03153b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0310102790 0x7f0310074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f03153b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0310106f10 0x7f0310074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f03153b2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0310111e20 con 0x7f0310106f10 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f03153b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0310111ca0 con 0x7f0310102790 2024-09-19T06:13:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f03153b2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0310111fa0 con 0x7f0310101de0 2024-09-19T06:13:40.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f030f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0310106f10 0x7f0310074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f030f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0310106f10 0x7f0310074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52104/0 (socket says 172.21.15.5:52104) 2024-09-19T06:13:40.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f030f7fe640 1 -- 172.21.15.5:0/1345296225 learned_addr learned my addr 172.21.15.5:0/1345296225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:40.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f030effd640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0310101de0 0x7f03100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.985+0000 7f030e7fc640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0310102790 0x7f0310074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.986+0000 7f030f7fe640 1 -- 172.21.15.5:0/1345296225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0310101de0 msgr2=0x7f03100744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:40.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.986+0000 7f030f7fe640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0310101de0 0x7f03100744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.986+0000 7f030f7fe640 1 -- 172.21.15.5:0/1345296225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0310102790 msgr2=0x7f0310074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:40.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.986+0000 7f030f7fe640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0310102790 0x7f0310074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.986+0000 7f030f7fe640 1 -- 172.21.15.5:0/1345296225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03101a6840 con 0x7f0310106f10 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.986+0000 7f030f7fe640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0310106f10 0x7f0310074f70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0308009dc0 tx=0x7f0308032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.986+0000 7f02f7fff640 1 -- 172.21.15.5:0/1345296225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308003bf0 con 0x7f0310106f10 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.986+0000 7f02f7fff640 1 -- 172.21.15.5:0/1345296225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0308003d90 con 0x7f0310106f10 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.986+0000 7f03153b2640 1 -- 172.21.15.5:0/1345296225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03101a6ad0 con 0x7f0310106f10 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.986+0000 7f02f7fff640 1 -- 172.21.15.5:0/1345296225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308030b80 con 0x7f0310106f10 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.986+0000 7f03153b2640 1 -- 172.21.15.5:0/1345296225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03101a7060 con 0x7f0310106f10 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.989+0000 7f02f7fff640 1 -- 172.21.15.5:0/1345296225 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0308004040 con 0x7f0310106f10 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.989+0000 7f031778b740 1 Processor -- start 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.989+0000 7f031778b740 1 -- start start 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.989+0000 7f02f7fff640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f02f80a3430 0x7f02f80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.989+0000 7f031778b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56207146ab40 0x562071469090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.989+0000 7f031778b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620714695d0 0x562071469a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.989+0000 7f031778b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562071474370 0x562071476760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.989+0000 7f031778b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620713cf2f0 con 0x562071474370 2024-09-19T06:13:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.989+0000 7f031778b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562071376cf0 con 0x5620714695d0 2024-09-19T06:13:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.989+0000 7f031778b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5620713cb2b0 con 0x56207146ab40 2024-09-19T06:13:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f02f7fff640 1 -- 172.21.15.5:0/1345296225 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f03080f1210 con 0x7f0310106f10 2024-09-19T06:13:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562071474370 0x562071476760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030e7fc640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f02f80a3430 0x7f02f80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562071474370 0x562071476760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52120/0 (socket says 172.21.15.5:52120) 2024-09-19T06:13:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030f7fe640 1 -- 172.21.15.5:0/410224874 learned_addr learned my addr 172.21.15.5:0/410224874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030effd640 1 --2- 172.21.15.5:0/410224874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56207146ab40 0x562071469090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030e7fc640 1 --2- 172.21.15.5:0/410224874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620714695d0 0x562071469a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030f7fe640 1 -- 172.21.15.5:0/410224874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56207146ab40 msgr2=0x562071469090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030f7fe640 1 --2- 172.21.15.5:0/410224874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56207146ab40 0x562071469090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030f7fe640 1 -- 172.21.15.5:0/410224874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620714695d0 msgr2=0x562071469a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030f7fe640 1 --2- 172.21.15.5:0/410224874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620714695d0 0x562071469a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030f7fe640 1 -- 172.21.15.5:0/410224874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56207135b3a0 con 0x562071474370 2024-09-19T06:13:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030e7fc640 1 --2- 172.21.15.5:0/410224874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620714695d0 0x562071469a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:13:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030f7fe640 1 --2- 172.21.15.5:0/410224874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562071474370 0x562071476760 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f03080b9a40 tx=0x7f03080bd1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8facc873f287ac73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f030e7fc640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f02f80a3430 0x7f02f80a58f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x562071466ac0 tx=0x7f03040043f0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f02f57fa640 1 -- 172.21.15.5:0/410224874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03080bd5e0 con 0x562071474370 2024-09-19T06:13:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f02f57fa640 1 -- 172.21.15.5:0/410224874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0308036020 con 0x562071474370 2024-09-19T06:13:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.990+0000 7f02f57fa640 1 -- 172.21.15.5:0/410224874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308030d20 con 0x562071474370 2024-09-19T06:13:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.991+0000 7f031778b740 1 -- 172.21.15.5:0/410224874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562071474370 msgr2=0x562071476760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.991+0000 7f031778b740 1 --2- 172.21.15.5:0/410224874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562071474370 0x562071476760 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f03080b9a40 tx=0x7f03080bd1e0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.991+0000 7f031778b740 1 -- 172.21.15.5:0/410224874 shutdown_connections 2024-09-19T06:13:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.991+0000 7f031778b740 1 --2- 172.21.15.5:0/410224874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562071474370 0x562071476760 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.991+0000 7f031778b740 1 --2- 172.21.15.5:0/410224874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620714695d0 0x562071469a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.991+0000 7f031778b740 1 --2- 172.21.15.5:0/410224874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56207146ab40 0x562071469090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.991+0000 7f031778b740 1 -- 172.21.15.5:0/410224874 >> 172.21.15.5:0/410224874 conn(0x562071432310 msgr2=0x562071465c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.991+0000 7f031778b740 1 -- 172.21.15.5:0/410224874 shutdown_connections 2024-09-19T06:13:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.991+0000 7f031778b740 1 -- 172.21.15.5:0/410224874 wait complete. 2024-09-19T06:13:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.991+0000 7f031778b740 1 -- start start 2024-09-19T06:13:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f031778b740 1 Processor -- start 2024-09-19T06:13:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f031778b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620714695d0 0x562071473d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f031778b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56207146ab40 0x562071471020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f031778b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562071474370 0x562071471610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f031778b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620713cf2f0 con 0x56207146ab40 2024-09-19T06:13:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f031778b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56207138a060 con 0x5620714695d0 2024-09-19T06:13:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f031778b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562071476e30 con 0x562071474370 2024-09-19T06:13:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f031778b740 10 client.0 _ll_register_callbacks cb 0x5620710dd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f030e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56207146ab40 0x562071471020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f030e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56207146ab40 0x562071471020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52130/0 (socket says 172.21.15.5:52130) 2024-09-19T06:13:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f030e7fc640 1 -- 172.21.15.5:0/1245489549 learned_addr learned my addr 172.21.15.5:0/1245489549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f030effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620714695d0 0x562071473d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f030f7fe640 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562071474370 0x562071471610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f030e7fc640 1 -- 172.21.15.5:0/1245489549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562071474370 msgr2=0x562071471610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f030e7fc640 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562071474370 0x562071471610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f030e7fc640 1 -- 172.21.15.5:0/1245489549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620714695d0 msgr2=0x562071473d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f030e7fc640 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620714695d0 0x562071473d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f030e7fc640 1 -- 172.21.15.5:0/1245489549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620711403c0 con 0x56207146ab40 2024-09-19T06:13:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.992+0000 7f030effd640 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620714695d0 0x562071473d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.993+0000 7f030e7fc640 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56207146ab40 0x562071471020 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0304003f80 tx=0x7f03040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04f8967ca51042b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.993+0000 7f02f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.993+0000 7f02f2ff5640 1 -- 172.21.15.5:0/1245489549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030401f600 con 0x56207146ab40 2024-09-19T06:13:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.993+0000 7f02f2ff5640 1 -- 172.21.15.5:0/1245489549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f030401f7a0 con 0x56207146ab40 2024-09-19T06:13:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.993+0000 7f031778b740 1 -- 172.21.15.5:0/1245489549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0308036020 con 0x56207146ab40 2024-09-19T06:13:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.993+0000 7f02f2ff5640 1 -- 172.21.15.5:0/1245489549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f030402e050 con 0x56207146ab40 2024-09-19T06:13:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.993+0000 7f031778b740 10 client.18108 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.993+0000 7f031778b740 1 -- 172.21.15.5:0/1245489549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562071482660 con 0x56207146ab40 2024-09-19T06:13:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.993+0000 7f02f2ff5640 1 -- 172.21.15.5:0/1245489549 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0308036020 con 0x56207146ab40 2024-09-19T06:13:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.994+0000 7f02f2ff5640 1 -- 172.21.15.5:0/1245489549 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f030401e040 con 0x56207146ab40 2024-09-19T06:13:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.994+0000 7f031778b740 10 client.18108 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.994+0000 7f031778b740 10 client.18108 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:13:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.994+0000 7f031778b740 20 client.18108 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.994+0000 7f031778b740 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x562071483c10 0x562071486070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.994+0000 7f031778b740 4 client.18108 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.994+0000 7f031778b740 1 -- 172.21.15.5:0/1245489549 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620713dc860 con 0x562071483c10 2024-09-19T06:13:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.994+0000 7f030effd640 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x562071483c10 0x562071486070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f02f2ff5640 10 client.18108 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:13:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f030effd640 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x562071483c10 0x562071486070 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f1a0e2d32ca77be server_cookie=9a9ccc0bedbf899b in_seq=0 out_seq=0 2024-09-19T06:13:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f02f2ff5640 1 -- 172.21.15.5:0/1245489549 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620713dc860 con 0x562071483c10 2024-09-19T06:13:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f02f2ff5640 10 client.18108 handle_command_reply: tid=0 2024-09-19T06:13:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f031778b740 1 client.18108 shutdown 2024-09-19T06:13:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f031778b740 1 -- 172.21.15.5:0/1245489549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56207146ab40 msgr2=0x562071471020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f031778b740 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56207146ab40 0x562071471020 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0304003f80 tx=0x7f03040046c0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f031778b740 1 -- 172.21.15.5:0/1245489549 shutdown_connections 2024-09-19T06:13:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f031778b740 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x562071483c10 0x562071486070 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f031778b740 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562071474370 0x562071471610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f031778b740 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56207146ab40 0x562071471020 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f031778b740 1 --2- 172.21.15.5:0/1245489549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620714695d0 0x562071473d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f031778b740 1 -- 172.21.15.5:0/1245489549 >> 172.21.15.5:0/1245489549 conn(0x562071432310 msgr2=0x56207146b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.995+0000 7f02f2ff5640 0 client.18108 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:13:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.996+0000 7f031778b740 1 -- 172.21.15.5:0/1245489549 shutdown_connections 2024-09-19T06:13:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.996+0000 7f031778b740 1 -- 172.21.15.5:0/1245489549 wait complete. 2024-09-19T06:13:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:39.996+0000 7f031778b740 20 client.18108 trim_cache size 0 max 16384 2024-09-19T06:13:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.011+0000 7f031778b740 1 Processor -- start 2024-09-19T06:13:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.011+0000 7f031778b740 1 -- start start 2024-09-19T06:13:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.011+0000 7f031778b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 0x562071470fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.011+0000 7f031778b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56207146ab40 0x56207147ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.011+0000 7f031778b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562071474370 0x562071465700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.011+0000 7f031778b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03101afa30 con 0x5620714695d0 2024-09-19T06:13:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.011+0000 7f031778b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03101b1630 con 0x56207146ab40 2024-09-19T06:13:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.011+0000 7f031778b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03101b0100 con 0x562071474370 2024-09-19T06:13:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.011+0000 7f030effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 0x562071470fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.012+0000 7f030f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562071474370 0x562071465700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.011+0000 7f030effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 0x562071470fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52132/0 (socket says 172.21.15.5:52132) 2024-09-19T06:13:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.012+0000 7f030effd640 1 -- 172.21.15.5:0/2544048745 learned_addr learned my addr 172.21.15.5:0/2544048745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.012+0000 7f030e7fc640 1 --2- 172.21.15.5:0/2544048745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56207146ab40 0x56207147ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.012+0000 7f030f7fe640 1 -- 172.21.15.5:0/2544048745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56207146ab40 msgr2=0x56207147ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.012+0000 7f030f7fe640 1 --2- 172.21.15.5:0/2544048745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56207146ab40 0x56207147ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.012+0000 7f030f7fe640 1 -- 172.21.15.5:0/2544048745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 msgr2=0x562071470fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.012+0000 7f030f7fe640 1 --2- 172.21.15.5:0/2544048745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 0x562071470fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.012+0000 7f030f7fe640 1 -- 172.21.15.5:0/2544048745 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562071480450 con 0x562071474370 2024-09-19T06:13:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.012+0000 7f030f7fe640 1 --2- 172.21.15.5:0/2544048745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562071474370 0x562071465700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f03080049d0 tx=0x7f030802a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f67c7ca4f3c845aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.012+0000 7f02f4ff9640 1 -- 172.21.15.5:0/2544048745 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03080db070 con 0x562071474370 2024-09-19T06:13:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.012+0000 7f02f4ff9640 1 -- 172.21.15.5:0/2544048745 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0308036070 con 0x562071474370 2024-09-19T06:13:40.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.012+0000 7f02f4ff9640 1 -- 172.21.15.5:0/2544048745 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03080d5070 con 0x562071474370 2024-09-19T06:13:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.013+0000 7f031778b740 1 -- 172.21.15.5:0/2544048745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562071474370 msgr2=0x562071465700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.013+0000 7f031778b740 1 --2- 172.21.15.5:0/2544048745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562071474370 0x562071465700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f03080049d0 tx=0x7f030802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 -- 172.21.15.5:0/2544048745 shutdown_connections 2024-09-19T06:13:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 --2- 172.21.15.5:0/2544048745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562071474370 0x562071465700 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 --2- 172.21.15.5:0/2544048745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56207146ab40 0x56207147ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 --2- 172.21.15.5:0/2544048745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 0x562071470fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 -- 172.21.15.5:0/2544048745 >> 172.21.15.5:0/2544048745 conn(0x562071432310 msgr2=0x56207146c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 -- 172.21.15.5:0/2544048745 shutdown_connections 2024-09-19T06:13:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 -- 172.21.15.5:0/2544048745 wait complete. 2024-09-19T06:13:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 -- start start 2024-09-19T06:13:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 Processor -- start 2024-09-19T06:13:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 0x562071477630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56207146ab40 0x562071477b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562071474370 0x5620714ec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03101b0f60 con 0x5620714695d0 2024-09-19T06:13:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03101b0830 con 0x562071474370 2024-09-19T06:13:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03101ada30 con 0x56207146ab40 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.014+0000 7f031778b740 10 client.0 _ll_register_callbacks cb 0x7f030400be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.015+0000 7f030effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 0x562071477630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.015+0000 7f030effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 0x562071477630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52138/0 (socket says 172.21.15.5:52138) 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.015+0000 7f030effd640 1 -- 172.21.15.5:0/288588052 learned_addr learned my addr 172.21.15.5:0/288588052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.015+0000 7f030e7fc640 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56207146ab40 0x562071477b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.015+0000 7f030effd640 1 -- 172.21.15.5:0/288588052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56207146ab40 msgr2=0x562071477b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.015+0000 7f030effd640 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56207146ab40 0x562071477b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.015+0000 7f030effd640 1 -- 172.21.15.5:0/288588052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562071474370 msgr2=0x5620714ec940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.015+0000 7f030f7fe640 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562071474370 0x5620714ec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.015+0000 7f030effd640 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562071474370 0x5620714ec940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.015+0000 7f030effd640 1 -- 172.21.15.5:0/288588052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620714ece80 con 0x5620714695d0 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.015+0000 7f030f7fe640 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562071474370 0x5620714ec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:13:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.015+0000 7f030effd640 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 0x562071477630 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f03000122a0 tx=0x7f0300012b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5444416f496ce7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.016+0000 7f02f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.016+0000 7f02f2ff5640 1 -- 172.21.15.5:0/288588052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300004250 con 0x5620714695d0 2024-09-19T06:13:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.016+0000 7f02f2ff5640 1 -- 172.21.15.5:0/288588052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f03000043f0 con 0x5620714695d0 2024-09-19T06:13:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.016+0000 7f031778b740 1 -- 172.21.15.5:0/288588052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0308036070 con 0x5620714695d0 2024-09-19T06:13:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.016+0000 7f02f2ff5640 1 -- 172.21.15.5:0/288588052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0300033050 con 0x5620714695d0 2024-09-19T06:13:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.016+0000 7f031778b740 10 client.18114 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.016+0000 7f031778b740 1 -- 172.21.15.5:0/288588052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5620714ee150 con 0x5620714695d0 2024-09-19T06:13:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.016+0000 7f02f2ff5640 1 -- 172.21.15.5:0/288588052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0308036070 con 0x5620714695d0 2024-09-19T06:13:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.017+0000 7f02f2ff5640 1 -- 172.21.15.5:0/288588052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f030002e030 con 0x5620714695d0 2024-09-19T06:13:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.017+0000 7f031778b740 10 client.18114 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.017+0000 7f031778b740 10 client.18114 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:13:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.017+0000 7f031778b740 20 client.18114 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.017+0000 7f031778b740 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5620714ee900 0x5620714f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.017+0000 7f031778b740 4 client.18114 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:13:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.017+0000 7f031778b740 1 -- 172.21.15.5:0/288588052 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620713d6d70 con 0x5620714ee900 2024-09-19T06:13:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.017+0000 7f030e7fc640 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5620714ee900 0x5620714f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.017+0000 7f02f2ff5640 10 client.18114 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:13:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.017+0000 7f030e7fc640 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5620714ee900 0x5620714f0d20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66569f63e89cb52 server_cookie=18d58e7675a22ce1 in_seq=0 out_seq=0 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.017+0000 7f02f2ff5640 1 -- 172.21.15.5:0/288588052 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620713d6d70 con 0x5620714ee900 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.017+0000 7f02f2ff5640 10 client.18114 handle_command_reply: tid=0 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f031778b740 1 client.18114 shutdown 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f031778b740 1 -- 172.21.15.5:0/288588052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 msgr2=0x562071477630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f031778b740 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 0x562071477630 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f03000122a0 tx=0x7f0300012b10 comp rx=0 tx=0).stop 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f031778b740 1 -- 172.21.15.5:0/288588052 shutdown_connections 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f031778b740 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5620714ee900 0x5620714f0d20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f02f2ff5640 0 client.18114 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f031778b740 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562071474370 0x5620714ec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f031778b740 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56207146ab40 0x562071477b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f031778b740 1 --2- 172.21.15.5:0/288588052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620714695d0 0x562071477630 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f031778b740 1 -- 172.21.15.5:0/288588052 >> 172.21.15.5:0/288588052 conn(0x562071432310 msgr2=0x562071465560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f031778b740 1 -- 172.21.15.5:0/288588052 shutdown_connections 2024-09-19T06:13:40.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f031778b740 1 -- 172.21.15.5:0/288588052 wait complete. 2024-09-19T06:13:40.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.018+0000 7f031778b740 20 client.18114 trim_cache size 0 max 16384 2024-09-19T06:13:40.027 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:13:40.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.019+0000 7f03153b2640 1 -- 172.21.15.5:0/1345296225 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f02f80a3430 msgr2=0x7f02f80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.019+0000 7f03153b2640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f02f80a3430 0x7f02f80a58f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x562071466ac0 tx=0x7f03040043f0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.019+0000 7f03153b2640 1 -- 172.21.15.5:0/1345296225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0310106f10 msgr2=0x7f0310074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.019+0000 7f03153b2640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0310106f10 0x7f0310074f70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0308009dc0 tx=0x7f0308032ff0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.019+0000 7f03153b2640 1 -- 172.21.15.5:0/1345296225 shutdown_connections 2024-09-19T06:13:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.019+0000 7f03153b2640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0310106f10 0x7f0310074f70 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.019+0000 7f03153b2640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0310102790 0x7f0310074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.019+0000 7f03153b2640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f02f80a3430 0x7f02f80a58f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.019+0000 7f03153b2640 1 --2- 172.21.15.5:0/1345296225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0310101de0 0x7f03100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.019+0000 7f03153b2640 1 -- 172.21.15.5:0/1345296225 >> 172.21.15.5:0/1345296225 conn(0x7f03100fd8c0 msgr2=0x7f03100febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.019+0000 7f03153b2640 1 -- 172.21.15.5:0/1345296225 shutdown_connections 2024-09-19T06:13:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:40.020+0000 7f03153b2640 1 -- 172.21.15.5:0/1345296225 wait complete. 2024-09-19T06:13:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:40 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi049 to 176.0M 2024-09-19T06:13:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:40 smithi157 ceph-mon[35346]: Unable to set osd_memory_target on smithi049 to 184553062: error parsing value: Value '184553062' is below minimum 939524096 2024-09-19T06:13:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:40 smithi157 ceph-mon[35346]: pgmap v321: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 381 KiB/s wr, 43 op/s 2024-09-19T06:13:40.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:40 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi049 to 176.0M 2024-09-19T06:13:40.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:40 smithi005 ceph-mon[30955]: Unable to set osd_memory_target on smithi049 to 184553062: error parsing value: Value '184553062' is below minimum 939524096 2024-09-19T06:13:40.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:40 smithi005 ceph-mon[30955]: pgmap v321: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 381 KiB/s wr, 43 op/s 2024-09-19T06:13:40.844 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:13:40.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 250736 2.38 MB/sec execute 426 sec latency 24.411 ms 2024-09-19T06:13:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:40 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi049 to 176.0M 2024-09-19T06:13:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:40 smithi049 ceph-mon[34640]: Unable to set osd_memory_target on smithi049 to 184553062: error parsing value: Value '184553062' is below minimum 939524096 2024-09-19T06:13:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:40 smithi049 ceph-mon[34640]: pgmap v321: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 381 KiB/s wr, 43 op/s 2024-09-19T06:13:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:13:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.735+0000 7f44dbb69640 1 -- 172.21.15.5:0/1086894972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d41036e0 msgr2=0x7f44d4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.735+0000 7f44dbb69640 1 --2- 172.21.15.5:0/1086894972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d41036e0 0x7f44d4107b90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009930 tx=0x7f44c402a420 comp rx=0 tx=0).stop 2024-09-19T06:13:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.735+0000 7f44dbb69640 1 -- 172.21.15.5:0/1086894972 shutdown_connections 2024-09-19T06:13:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.735+0000 7f44dbb69640 1 --2- 172.21.15.5:0/1086894972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44d41082a0 0x7f44d410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.735+0000 7f44dbb69640 1 --2- 172.21.15.5:0/1086894972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d41036e0 0x7f44d4107b90 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.735+0000 7f44dbb69640 1 --2- 172.21.15.5:0/1086894972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d4102d30 0x7f44d4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.735+0000 7f44dbb69640 1 -- 172.21.15.5:0/1086894972 >> 172.21.15.5:0/1086894972 conn(0x7f44d40fc7d0 msgr2=0x7f44d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.735+0000 7f44dbb69640 1 -- 172.21.15.5:0/1086894972 shutdown_connections 2024-09-19T06:13:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.735+0000 7f44dbb69640 1 -- 172.21.15.5:0/1086894972 wait complete. 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44dbb69640 1 Processor -- start 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44dbb69640 1 -- start start 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44dbb69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4102d30 0x7f44d4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44dbb69640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d41036e0 0x7f44d4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44dbb69640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44d41082a0 0x7f44d4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44dbb69640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44d4111f50 con 0x7f44d4102d30 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44dbb69640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44d4111dd0 con 0x7f44d41036e0 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44dbb69640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41120d0 con 0x7f44d41082a0 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44d9905640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4102d30 0x7f44d4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44d9905640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4102d30 0x7f44d4100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52166/0 (socket says 172.21.15.5:52166) 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44d9905640 1 -- 172.21.15.5:0/2145359731 learned_addr learned my addr 172.21.15.5:0/2145359731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44da106640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44d41082a0 0x7f44d4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.736+0000 7f44d9104640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d41036e0 0x7f44d4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.737+0000 7f44d9905640 1 -- 172.21.15.5:0/2145359731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44d41082a0 msgr2=0x7f44d4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.737+0000 7f44d9905640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44d41082a0 0x7f44d4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.737+0000 7f44d9905640 1 -- 172.21.15.5:0/2145359731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d41036e0 msgr2=0x7f44d4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.737+0000 7f44d9905640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d41036e0 0x7f44d4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.737+0000 7f44d9905640 1 -- 172.21.15.5:0/2145359731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d41a69d0 con 0x7f44d4102d30 2024-09-19T06:13:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.737+0000 7f44d9905640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4102d30 0x7f44d4100990 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f44d000e9f0 tx=0x7f44d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.737+0000 7f44caffd640 1 -- 172.21.15.5:0/2145359731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d000cd80 con 0x7f44d4102d30 2024-09-19T06:13:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.737+0000 7f44dbb69640 1 -- 172.21.15.5:0/2145359731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d41a6cc0 con 0x7f44d4102d30 2024-09-19T06:13:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.737+0000 7f44dbb69640 1 -- 172.21.15.5:0/2145359731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d41a71c0 con 0x7f44d4102d30 2024-09-19T06:13:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.737+0000 7f44caffd640 1 -- 172.21.15.5:0/2145359731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44d00108e0 con 0x7f44d4102d30 2024-09-19T06:13:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.737+0000 7f44caffd640 1 -- 172.21.15.5:0/2145359731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0018920 con 0x7f44d4102d30 2024-09-19T06:13:41.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.740+0000 7f44caffd640 1 -- 172.21.15.5:0/2145359731 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f44d0010430 con 0x7f44d4102d30 2024-09-19T06:13:41.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.741+0000 7f44caffd640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f44c00a35b0 0x7f44c00a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.741+0000 7f44caffd640 1 -- 172.21.15.5:0/2145359731 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f44d00186c0 con 0x7f44d4102d30 2024-09-19T06:13:41.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.741+0000 7f44d9104640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f44c00a35b0 0x7f44c00a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.741+0000 7f44ddf42740 1 Processor -- start 2024-09-19T06:13:41.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.741+0000 7f44d9104640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f44c00a35b0 0x7f44c00a5a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f44c4031440 tx=0x7f44c4000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:41.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44ddf42740 1 -- start start 2024-09-19T06:13:41.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44ddf42740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 0x55efde44d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44ddf42740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 0x55efde44da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44ddf42740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde45a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44ddf42740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55efde3b32f0 con 0x55efde44eb40 2024-09-19T06:13:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44ddf42740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55efde35acf0 con 0x55efde44d5d0 2024-09-19T06:13:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44ddf42740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55efde3af2b0 con 0x55efde458370 2024-09-19T06:13:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44d9905640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 0x55efde44d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44d9104640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 0x55efde44da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44d9104640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 0x55efde44da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49732/0 (socket says 172.21.15.5:49732) 2024-09-19T06:13:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44d9905640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 0x55efde44d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52178/0 (socket says 172.21.15.5:52178) 2024-09-19T06:13:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44d9104640 1 -- 172.21.15.5:0/3073813014 learned_addr learned my addr 172.21.15.5:0/3073813014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.742+0000 7f44da106640 1 --2- 172.21.15.5:0/3073813014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde45a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.743+0000 7f44d9104640 1 -- 172.21.15.5:0/3073813014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 msgr2=0x55efde45a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:41.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.743+0000 7f44d9104640 1 --2- 172.21.15.5:0/3073813014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde45a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.743+0000 7f44d9104640 1 -- 172.21.15.5:0/3073813014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 msgr2=0x55efde44d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:41.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.743+0000 7f44d9104640 1 --2- 172.21.15.5:0/3073813014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 0x55efde44d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.743+0000 7f44d9104640 1 -- 172.21.15.5:0/3073813014 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efde33f3a0 con 0x55efde44d5d0 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.743+0000 7f44d9104640 1 --2- 172.21.15.5:0/3073813014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 0x55efde44da10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f44c4033000 tx=0x7f44c403d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62eee08355f19c97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.743+0000 7f44bffff640 1 -- 172.21.15.5:0/3073813014 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4035040 con 0x55efde44d5d0 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.743+0000 7f44bffff640 1 -- 172.21.15.5:0/3073813014 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c404d070 con 0x55efde44d5d0 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.743+0000 7f44bffff640 1 -- 172.21.15.5:0/3073813014 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4004a40 con 0x55efde44d5d0 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.744+0000 7f44ddf42740 1 -- 172.21.15.5:0/3073813014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 msgr2=0x55efde44da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.744+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3073813014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 0x55efde44da10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f44c4033000 tx=0x7f44c403d000 comp rx=0 tx=0).stop 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.744+0000 7f44ddf42740 1 -- 172.21.15.5:0/3073813014 shutdown_connections 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.744+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3073813014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde45a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.744+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3073813014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 0x55efde44da10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.744+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3073813014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 0x55efde44d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.744+0000 7f44ddf42740 1 -- 172.21.15.5:0/3073813014 >> 172.21.15.5:0/3073813014 conn(0x55efde416310 msgr2=0x55efde449c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.744+0000 7f44ddf42740 1 -- 172.21.15.5:0/3073813014 shutdown_connections 2024-09-19T06:13:41.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.744+0000 7f44ddf42740 1 -- 172.21.15.5:0/3073813014 wait complete. 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.744+0000 7f44ddf42740 1 -- start start 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.745+0000 7f44ddf42740 1 Processor -- start 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.745+0000 7f44ddf42740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 0x55efde457cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.745+0000 7f44ddf42740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44eb40 0x55efde454f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.745+0000 7f44ddf42740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde4554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.745+0000 7f44ddf42740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55efde45b010 con 0x55efde44d5d0 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.745+0000 7f44ddf42740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55efde36e060 con 0x55efde44eb40 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.745+0000 7f44ddf42740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55efde45b190 con 0x55efde458370 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44d9104640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44eb40 0x55efde454f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44d9905640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 0x55efde457cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44ddf42740 10 client.0 _ll_register_callbacks cb 0x55efde0c18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44d9104640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44eb40 0x55efde454f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49746/0 (socket says 172.21.15.5:49746) 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44d9104640 1 -- 172.21.15.5:0/3171138322 learned_addr learned my addr 172.21.15.5:0/3171138322 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:41.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44da106640 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde4554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44d9104640 1 -- 172.21.15.5:0/3171138322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 msgr2=0x55efde4554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44d9104640 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde4554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44d9104640 1 -- 172.21.15.5:0/3171138322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 msgr2=0x55efde457cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44d9104640 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 0x55efde457cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44d9104640 1 -- 172.21.15.5:0/3171138322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efde1243c0 con 0x55efde44eb40 2024-09-19T06:13:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44da106640 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde4554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44d9905640 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 0x55efde457cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44d9104640 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44eb40 0x55efde454f60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f44c402a940 tx=0x7f44c4004300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89ba4afe2466504 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.746+0000 7f44bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:13:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44bd7fa640 1 -- 172.21.15.5:0/3171138322 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4035070 con 0x55efde44eb40 2024-09-19T06:13:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44ddf42740 1 -- 172.21.15.5:0/3171138322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44c404d070 con 0x55efde44eb40 2024-09-19T06:13:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44bd7fa640 1 -- 172.21.15.5:0/3171138322 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f44c4005510 con 0x55efde44eb40 2024-09-19T06:13:41.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44bd7fa640 1 -- 172.21.15.5:0/3171138322 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f44c40047e0 con 0x55efde44eb40 2024-09-19T06:13:41.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44ddf42740 10 client.27632 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44ddf42740 1 -- 172.21.15.5:0/3171138322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55efde466a90 con 0x55efde44eb40 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44bd7fa640 1 -- 172.21.15.5:0/3171138322 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c404d070 con 0x55efde44eb40 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44bd7fa640 1 -- 172.21.15.5:0/3171138322 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f44c4003bf0 con 0x55efde44eb40 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44ddf42740 10 client.27632 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44ddf42740 10 client.27632 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44ddf42740 20 client.27632 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55efde467de0 0x55efde46a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44ddf42740 4 client.27632 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.747+0000 7f44ddf42740 1 -- 172.21.15.5:0/3171138322 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efde3c0860 con 0x55efde467de0 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.748+0000 7f44d9905640 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55efde467de0 0x55efde46a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.748+0000 7f44bd7fa640 10 client.27632 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.748+0000 7f44d9905640 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55efde467de0 0x55efde46a1d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a43467eec857cc10 server_cookie=f86173c82a6ee25f in_seq=0 out_seq=0 2024-09-19T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.749+0000 7f44bd7fa640 1 -- 172.21.15.5:0/3171138322 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55efde3c0860 con 0x55efde467de0 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.749+0000 7f44bd7fa640 10 client.27632 handle_command_reply: tid=0 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.749+0000 7f44ddf42740 1 client.27632 shutdown 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.749+0000 7f44ddf42740 1 -- 172.21.15.5:0/3171138322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44eb40 msgr2=0x55efde454f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.749+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44eb40 0x55efde454f60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f44c402a940 tx=0x7f44c4004300 comp rx=0 tx=0).stop 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.749+0000 7f44ddf42740 1 -- 172.21.15.5:0/3171138322 shutdown_connections 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.749+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55efde467de0 0x55efde46a1d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.749+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde4554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.749+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44eb40 0x55efde454f60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.749+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3171138322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 0x55efde457cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.749+0000 7f44ddf42740 1 -- 172.21.15.5:0/3171138322 >> 172.21.15.5:0/3171138322 conn(0x55efde416310 msgr2=0x55efde44f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.749+0000 7f44bd7fa640 0 client.27632 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.750+0000 7f44ddf42740 1 -- 172.21.15.5:0/3171138322 shutdown_connections 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.750+0000 7f44ddf42740 1 -- 172.21.15.5:0/3171138322 wait complete. 2024-09-19T06:13:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.750+0000 7f44ddf42740 20 client.27632 trim_cache size 0 max 16384 2024-09-19T06:13:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44ddf42740 1 Processor -- start 2024-09-19T06:13:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44ddf42740 1 -- start start 2024-09-19T06:13:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44ddf42740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 0x55efde449e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44ddf42740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 0x55efde464340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44ddf42740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde46f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44ddf42740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44b800e7e0 con 0x55efde44eb40 2024-09-19T06:13:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44ddf42740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44b80103e0 con 0x55efde44d5d0 2024-09-19T06:13:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44ddf42740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44b800eeb0 con 0x55efde458370 2024-09-19T06:13:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44d9104640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 0x55efde464340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44da106640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde46f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44d9104640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 0x55efde464340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52200/0 (socket says 172.21.15.5:52200) 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44d9104640 1 -- 172.21.15.5:0/398889063 learned_addr learned my addr 172.21.15.5:0/398889063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44d9905640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 0x55efde449e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.764+0000 7f44d9104640 1 -- 172.21.15.5:0/398889063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 msgr2=0x55efde46f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.765+0000 7f44d9104640 1 --2- 172.21.15.5:0/398889063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde46f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.765+0000 7f44d9104640 1 -- 172.21.15.5:0/398889063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 msgr2=0x55efde449e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.765+0000 7f44d9104640 1 --2- 172.21.15.5:0/398889063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 0x55efde449e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.765+0000 7f44d9104640 1 -- 172.21.15.5:0/398889063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efde44a960 con 0x55efde44eb40 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.765+0000 7f44da106640 1 --2- 172.21.15.5:0/398889063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde46f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.765+0000 7f44d9905640 1 --2- 172.21.15.5:0/398889063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 0x55efde449e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.765+0000 7f44d9104640 1 --2- 172.21.15.5:0/398889063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 0x55efde464340 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f44c403f040 tx=0x7f44c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fd08204c4ebb24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.765+0000 7f44bf7fe640 1 -- 172.21.15.5:0/398889063 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4067070 con 0x55efde44eb40 2024-09-19T06:13:41.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.765+0000 7f44bf7fe640 1 -- 172.21.15.5:0/398889063 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c404d070 con 0x55efde44eb40 2024-09-19T06:13:41.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.765+0000 7f44bf7fe640 1 -- 172.21.15.5:0/398889063 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4035040 con 0x55efde44eb40 2024-09-19T06:13:41.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.766+0000 7f44ddf42740 1 -- 172.21.15.5:0/398889063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 msgr2=0x55efde464340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:41.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.766+0000 7f44ddf42740 1 --2- 172.21.15.5:0/398889063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 0x55efde464340 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f44c403f040 tx=0x7f44c4005fd0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.766+0000 7f44ddf42740 1 -- 172.21.15.5:0/398889063 shutdown_connections 2024-09-19T06:13:41.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.766+0000 7f44ddf42740 1 --2- 172.21.15.5:0/398889063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde46f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.766+0000 7f44ddf42740 1 --2- 172.21.15.5:0/398889063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44eb40 0x55efde464340 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.766+0000 7f44ddf42740 1 --2- 172.21.15.5:0/398889063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44d5d0 0x55efde449e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.766+0000 7f44ddf42740 1 -- 172.21.15.5:0/398889063 >> 172.21.15.5:0/398889063 conn(0x55efde416310 msgr2=0x55efde449700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.766+0000 7f44ddf42740 1 -- 172.21.15.5:0/398889063 shutdown_connections 2024-09-19T06:13:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.766+0000 7f44ddf42740 1 -- 172.21.15.5:0/398889063 wait complete. 2024-09-19T06:13:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.766+0000 7f44ddf42740 1 -- start start 2024-09-19T06:13:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.766+0000 7f44ddf42740 1 Processor -- start 2024-09-19T06:13:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44ddf42740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 0x55efde4604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44ddf42740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44eb40 0x55efde4609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44ddf42740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde451350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44ddf42740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44b800fd10 con 0x55efde44d5d0 2024-09-19T06:13:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44ddf42740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44b800f5e0 con 0x55efde44eb40 2024-09-19T06:13:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44ddf42740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44b800c7e0 con 0x55efde458370 2024-09-19T06:13:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44ddf42740 10 client.0 _ll_register_callbacks cb 0x7f44d0097c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44d9905640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 0x55efde4604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44da106640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde451350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44d9905640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 0x55efde4604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52202/0 (socket says 172.21.15.5:52202) 2024-09-19T06:13:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44d9905640 1 -- 172.21.15.5:0/3080689797 learned_addr learned my addr 172.21.15.5:0/3080689797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44d9104640 1 --2- 172.21.15.5:0/3080689797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44eb40 0x55efde4609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44d9905640 1 -- 172.21.15.5:0/3080689797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 msgr2=0x55efde451350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44d9905640 1 --2- 172.21.15.5:0/3080689797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde451350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44d9905640 1 -- 172.21.15.5:0/3080689797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44eb40 msgr2=0x55efde4609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44d9905640 1 --2- 172.21.15.5:0/3080689797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44eb40 0x55efde4609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.767+0000 7f44d9905640 1 -- 172.21.15.5:0/3080689797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efde451890 con 0x55efde44d5d0 2024-09-19T06:13:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.768+0000 7f44d9905640 1 --2- 172.21.15.5:0/3080689797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 0x55efde4604b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f44d009c090 tx=0x7f44d00bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b525c084ae707c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.768+0000 7f44bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.768+0000 7f44bd7fa640 1 -- 172.21.15.5:0/3080689797 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d009c100 con 0x55efde44d5d0 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.768+0000 7f44bd7fa640 1 -- 172.21.15.5:0/3080689797 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f44d00a00b0 con 0x55efde44d5d0 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.768+0000 7f44ddf42740 1 -- 172.21.15.5:0/3080689797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44c404d070 con 0x55efde44d5d0 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.768+0000 7f44bd7fa640 1 -- 172.21.15.5:0/3080689797 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f44d0098770 con 0x55efde44d5d0 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.768+0000 7f44ddf42740 10 client.18132 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.768+0000 7f44ddf42740 1 -- 172.21.15.5:0/3080689797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55efde4d1fc0 con 0x55efde44d5d0 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.768+0000 7f44bd7fa640 1 -- 172.21.15.5:0/3080689797 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c404d070 con 0x55efde44d5d0 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.769+0000 7f44bd7fa640 1 -- 172.21.15.5:0/3080689797 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f44d009afc0 con 0x55efde44d5d0 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.769+0000 7f44ddf42740 10 client.18132 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.769+0000 7f44ddf42740 10 client.18132 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.769+0000 7f44ddf42740 20 client.18132 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.769+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3080689797 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55efde4d26f0 0x55efde4d4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.769+0000 7f44ddf42740 4 client.18132 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:13:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.769+0000 7f44ddf42740 1 -- 172.21.15.5:0/3080689797 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efde3bad70 con 0x55efde4d26f0 2024-09-19T06:13:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.769+0000 7f44d9104640 1 --2- 172.21.15.5:0/3080689797 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55efde4d26f0 0x55efde4d4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.769+0000 7f44bd7fa640 10 client.18132 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:13:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.769+0000 7f44d9104640 1 --2- 172.21.15.5:0/3080689797 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55efde4d26f0 0x55efde4d4b10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd0bc6ec2c3554f8 server_cookie=72074f8533cfc8a7 in_seq=0 out_seq=0 2024-09-19T06:13:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44bd7fa640 1 -- 172.21.15.5:0/3080689797 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efde3bad70 con 0x55efde4d26f0 2024-09-19T06:13:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44bd7fa640 10 client.18132 handle_command_reply: tid=0 2024-09-19T06:13:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44ddf42740 1 client.18132 shutdown 2024-09-19T06:13:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44ddf42740 1 -- 172.21.15.5:0/3080689797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 msgr2=0x55efde4604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3080689797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 0x55efde4604b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f44d009c090 tx=0x7f44d00bc040 comp rx=0 tx=0).stop 2024-09-19T06:13:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44ddf42740 1 -- 172.21.15.5:0/3080689797 shutdown_connections 2024-09-19T06:13:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3080689797 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55efde4d26f0 0x55efde4d4b10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3080689797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efde458370 0x55efde451350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3080689797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efde44eb40 0x55efde4609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44ddf42740 1 --2- 172.21.15.5:0/3080689797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efde44d5d0 0x55efde4604b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44bd7fa640 0 client.18132 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:13:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44ddf42740 1 -- 172.21.15.5:0/3080689797 >> 172.21.15.5:0/3080689797 conn(0x55efde416310 msgr2=0x55efde453850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44ddf42740 1 -- 172.21.15.5:0/3080689797 shutdown_connections 2024-09-19T06:13:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44ddf42740 1 -- 172.21.15.5:0/3080689797 wait complete. 2024-09-19T06:13:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.770+0000 7f44ddf42740 20 client.18132 trim_cache size 0 max 16384 2024-09-19T06:13:41.779 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:13:41.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.771+0000 7f44dbb69640 1 -- 172.21.15.5:0/2145359731 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f44c00a35b0 msgr2=0x7f44c00a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:41.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.771+0000 7f44dbb69640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f44c00a35b0 0x7f44c00a5a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f44c4031440 tx=0x7f44c4000bc0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.771+0000 7f44dbb69640 1 -- 172.21.15.5:0/2145359731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4102d30 msgr2=0x7f44d4100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:41.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.771+0000 7f44dbb69640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4102d30 0x7f44d4100990 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f44d000e9f0 tx=0x7f44d000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.771+0000 7f44dbb69640 1 -- 172.21.15.5:0/2145359731 shutdown_connections 2024-09-19T06:13:41.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.771+0000 7f44dbb69640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f44c00a35b0 0x7f44c00a5a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.771+0000 7f44dbb69640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44d41082a0 0x7f44d4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.771+0000 7f44dbb69640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d41036e0 0x7f44d4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.771+0000 7f44dbb69640 1 --2- 172.21.15.5:0/2145359731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4102d30 0x7f44d4100990 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:41.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.771+0000 7f44dbb69640 1 -- 172.21.15.5:0/2145359731 >> 172.21.15.5:0/2145359731 conn(0x7f44d40fc7d0 msgr2=0x7f44d4105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:41.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.771+0000 7f44dbb69640 1 -- 172.21.15.5:0/2145359731 shutdown_connections 2024-09-19T06:13:41.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:41.771+0000 7f44dbb69640 1 -- 172.21.15.5:0/2145359731 wait complete. 2024-09-19T06:13:41.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 251299 2.39 MB/sec execute 427 sec latency 37.910 ms 2024-09-19T06:13:42.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:42 smithi049 ceph-mon[34640]: pgmap v322: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 381 KiB/s wr, 43 op/s 2024-09-19T06:13:42.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:42.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:42.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:13:42.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:42.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:42 smithi005 ceph-mon[30955]: pgmap v322: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 381 KiB/s wr, 43 op/s 2024-09-19T06:13:42.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:42.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:42.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:42.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:42.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:42.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:42.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:42.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:13:42.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:42.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:42 smithi157 ceph-mon[35346]: pgmap v322: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 381 KiB/s wr, 43 op/s 2024-09-19T06:13:42.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:42.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:42.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:42.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:42.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:13:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:13:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:13:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:13:42.569 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:13:42.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 251816 2.39 MB/sec execute 428 sec latency 15.731 ms 2024-09-19T06:13:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:13:43.275 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:43 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-19T06:13:43.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:43 smithi005 ceph-mon[30955]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T06:13:43.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.361+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/2720339820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8d0106f50 msgr2=0x7fa8d010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:43.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.361+0000 7fa8d5c6b640 1 --2- 172.21.15.5:0/2720339820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8d0106f50 0x7fa8d010eaf0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009cd0 tx=0x7fa8c802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.361+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/2720339820 shutdown_connections 2024-09-19T06:13:43.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.361+0000 7fa8d5c6b640 1 --2- 172.21.15.5:0/2720339820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8d0106f50 0x7fa8d010eaf0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.361+0000 7fa8d5c6b640 1 --2- 172.21.15.5:0/2720339820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8d0100ca0 0x7fa8d0106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.361+0000 7fa8d5c6b640 1 --2- 172.21.15.5:0/2720339820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d01002f0 0x7fa8d01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.361+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/2720339820 >> 172.21.15.5:0/2720339820 conn(0x7fa8d0077210 msgr2=0x7fa8d00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:43.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.361+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/2720339820 shutdown_connections 2024-09-19T06:13:43.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.361+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/2720339820 wait complete. 2024-09-19T06:13:43.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.362+0000 7fa8d5c6b640 1 Processor -- start 2024-09-19T06:13:43.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.362+0000 7fa8d5c6b640 1 -- start start 2024-09-19T06:13:43.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.362+0000 7fa8d5c6b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d01002f0 0x7fa8d0101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.362+0000 7fa8d5c6b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8d0100ca0 0x7fa8d01017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.362+0000 7fa8d5c6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8d0106f50 0x7fa8d0101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.362+0000 7fa8d5c6b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0111eb0 con 0x7fa8d0106f50 2024-09-19T06:13:43.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.362+0000 7fa8d5c6b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0111d30 con 0x7fa8d0100ca0 2024-09-19T06:13:43.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.362+0000 7fa8d5c6b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0112030 con 0x7fa8d01002f0 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.362+0000 7fa8cffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8d0106f50 0x7fa8d0101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.362+0000 7fa8cffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8d0106f50 0x7fa8d0101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52230/0 (socket says 172.21.15.5:52230) 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.362+0000 7fa8cffff640 1 -- 172.21.15.5:0/3516390157 learned_addr learned my addr 172.21.15.5:0/3516390157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.362+0000 7fa8cf7fe640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d01002f0 0x7fa8d0101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.363+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8d0100ca0 0x7fa8d01017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.363+0000 7fa8cffff640 1 -- 172.21.15.5:0/3516390157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d01002f0 msgr2=0x7fa8d0101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.363+0000 7fa8cffff640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d01002f0 0x7fa8d0101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.363+0000 7fa8cffff640 1 -- 172.21.15.5:0/3516390157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8d0100ca0 msgr2=0x7fa8d01017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.363+0000 7fa8cffff640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8d0100ca0 0x7fa8d01017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.363+0000 7fa8cffff640 1 -- 172.21.15.5:0/3516390157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d01a6850 con 0x7fa8d0106f50 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.363+0000 7fa8cffff640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8d0106f50 0x7fa8d0101cf0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8005cc0 tx=0x7fa8c8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.363+0000 7fa8ccff9640 1 -- 172.21.15.5:0/3516390157 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8004240 con 0x7fa8d0106f50 2024-09-19T06:13:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.363+0000 7fa8ccff9640 1 -- 172.21.15.5:0/3516390157 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c80305a0 con 0x7fa8d0106f50 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.363+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/3516390157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d01a6ae0 con 0x7fa8d0106f50 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.363+0000 7fa8ccff9640 1 -- 172.21.15.5:0/3516390157 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c80466c0 con 0x7fa8d0106f50 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.363+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/3516390157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d01a7070 con 0x7fa8d0106f50 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.366+0000 7fa8ccff9640 1 -- 172.21.15.5:0/3516390157 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa8c8030ac0 con 0x7fa8d0106f50 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.366+0000 7fa8d8044740 1 Processor -- start 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.366+0000 7fa8d8044740 1 -- start start 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.366+0000 7fa8d8044740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e4b40 0x55eb272e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.366+0000 7fa8d8044740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e35d0 0x55eb272e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.366+0000 7fa8d8044740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272ee370 0x55eb272f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.366+0000 7fa8d8044740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eb272492f0 con 0x55eb272e4b40 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.366+0000 7fa8d8044740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eb271f0cf0 con 0x55eb272ee370 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8d8044740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eb272452b0 con 0x55eb272e35d0 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e4b40 0x55eb272e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e4b40 0x55eb272e3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52236/0 (socket says 172.21.15.5:52236) 2024-09-19T06:13:43.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8ceffd640 1 -- 172.21.15.5:0/2725332126 learned_addr learned my addr 172.21.15.5:0/2725332126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:43.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8cf7fe640 1 --2- 172.21.15.5:0/2725332126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e35d0 0x55eb272e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/2725332126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272ee370 0x55eb272f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8ccff9640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa8b80a3430 0x7fa8b80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8ceffd640 1 -- 172.21.15.5:0/2725332126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e35d0 msgr2=0x55eb272e3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:43.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/2725332126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e35d0 0x55eb272e3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8ceffd640 1 -- 172.21.15.5:0/2725332126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272ee370 msgr2=0x55eb272f0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:43.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/2725332126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272ee370 0x55eb272f0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8ceffd640 1 -- 172.21.15.5:0/2725332126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb271d53a0 con 0x55eb272e4b40 2024-09-19T06:13:43.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8cffff640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa8b80a3430 0x7fa8b80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8ccff9640 1 -- 172.21.15.5:0/3516390157 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7fa8c802e030 con 0x7fa8d0106f50 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/2725332126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e4b40 0x55eb272e3090 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400ae70 tx=0x7fa8c400a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a250c3aa3f826f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8b5ffb640 1 -- 172.21.15.5:0/2725332126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c401b070 con 0x55eb272e4b40 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.367+0000 7fa8b5ffb640 1 -- 172.21.15.5:0/2725332126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c4010e50 con 0x55eb272e4b40 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.368+0000 7fa8b5ffb640 1 -- 172.21.15.5:0/2725332126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400f040 con 0x55eb272e4b40 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.368+0000 7fa8cffff640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa8b80a3430 0x7fa8b80a58f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80eef90 tx=0x7fa8c808f3b0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.368+0000 7fa8d8044740 1 -- 172.21.15.5:0/2725332126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e4b40 msgr2=0x55eb272e3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.368+0000 7fa8d8044740 1 --2- 172.21.15.5:0/2725332126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e4b40 0x55eb272e3090 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400ae70 tx=0x7fa8c400a0d0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.368+0000 7fa8d8044740 1 -- 172.21.15.5:0/2725332126 shutdown_connections 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.368+0000 7fa8d8044740 1 --2- 172.21.15.5:0/2725332126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272ee370 0x55eb272f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.368+0000 7fa8d8044740 1 --2- 172.21.15.5:0/2725332126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e35d0 0x55eb272e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.368+0000 7fa8d8044740 1 --2- 172.21.15.5:0/2725332126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e4b40 0x55eb272e3090 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.368+0000 7fa8d8044740 1 -- 172.21.15.5:0/2725332126 >> 172.21.15.5:0/2725332126 conn(0x55eb272ac310 msgr2=0x55eb272dfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.368+0000 7fa8d8044740 1 -- 172.21.15.5:0/2725332126 shutdown_connections 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.368+0000 7fa8d8044740 1 -- 172.21.15.5:0/2725332126 wait complete. 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8d8044740 1 -- start start 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8d8044740 1 Processor -- start 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8d8044740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 0x55eb272edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8d8044740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272e4b40 0x55eb272eb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8d8044740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272ee370 0x55eb272eb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8d8044740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eb272f1070 con 0x55eb272e35d0 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8d8044740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eb27204060 con 0x55eb272e4b40 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8d8044740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eb272f11f0 con 0x55eb272ee370 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8d8044740 10 client.0 _ll_register_callbacks cb 0x55eb26f578c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 0x55eb272edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 0x55eb272edd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52238/0 (socket says 172.21.15.5:52238) 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8ceffd640 1 -- 172.21.15.5:0/1333744033 learned_addr learned my addr 172.21.15.5:0/1333744033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272ee370 0x55eb272eb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.369+0000 7fa8cf7fe640 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272e4b40 0x55eb272eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8ceffd640 1 -- 172.21.15.5:0/1333744033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272ee370 msgr2=0x55eb272eb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272ee370 0x55eb272eb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8ceffd640 1 -- 172.21.15.5:0/1333744033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272e4b40 msgr2=0x55eb272eb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272e4b40 0x55eb272eb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8ceffd640 1 -- 172.21.15.5:0/1333744033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb26fba3c0 con 0x55eb272e35d0 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 0x55eb272edd50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4004700 tx=0x7fa8c4006c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfb95e9c87270d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8aeffd640 1 -- 172.21.15.5:0/1333744033 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4029060 con 0x55eb272e35d0 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8aeffd640 1 -- 172.21.15.5:0/1333744033 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7fa8c4015500 con 0x55eb272e35d0 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8d8044740 1 -- 172.21.15.5:0/1333744033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c4010e50 con 0x55eb272e35d0 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8aeffd640 1 -- 172.21.15.5:0/1333744033 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa8c401b040 con 0x55eb272e35d0 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8d8044740 10 client.18144 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8d8044740 1 -- 172.21.15.5:0/1333744033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55eb272fcb20 con 0x55eb272e35d0 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8aeffd640 1 -- 172.21.15.5:0/1333744033 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c4010e50 con 0x55eb272e35d0 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.370+0000 7fa8aeffd640 1 -- 172.21.15.5:0/1333744033 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fa8c4021070 con 0x55eb272e35d0 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.371+0000 7fa8d8044740 10 client.18144 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.371+0000 7fa8d8044740 10 client.18144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.371+0000 7fa8d8044740 20 client.18144 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.371+0000 7fa8d8044740 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eb272fde70 0x55eb27300260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.371+0000 7fa8d8044740 4 client.18144 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.371+0000 7fa8d8044740 1 -- 172.21.15.5:0/1333744033 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb27256860 con 0x55eb272fde70 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.371+0000 7fa8cf7fe640 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eb272fde70 0x55eb27300260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.371+0000 7fa8cf7fe640 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eb272fde70 0x55eb27300260 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=528cf8947ff533b3 server_cookie=c5ffd8d127eafb00 in_seq=0 out_seq=0 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.371+0000 7fa8aeffd640 10 client.18144 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8aeffd640 1 -- 172.21.15.5:0/1333744033 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb27256860 con 0x55eb272fde70 2024-09-19T06:13:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8aeffd640 10 client.18144 handle_command_reply: tid=0 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8d8044740 1 client.18144 shutdown 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8d8044740 1 -- 172.21.15.5:0/1333744033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 msgr2=0x55eb272edd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8d8044740 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 0x55eb272edd50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4004700 tx=0x7fa8c4006c30 comp rx=0 tx=0).stop 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8d8044740 1 -- 172.21.15.5:0/1333744033 shutdown_connections 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8d8044740 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eb272fde70 0x55eb27300260 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8d8044740 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272ee370 0x55eb272eb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8d8044740 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272e4b40 0x55eb272eb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8d8044740 1 --2- 172.21.15.5:0/1333744033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 0x55eb272edd50 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8aeffd640 0 client.18144 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8d8044740 1 -- 172.21.15.5:0/1333744033 >> 172.21.15.5:0/1333744033 conn(0x55eb272ac310 msgr2=0x55eb272e5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8d8044740 1 -- 172.21.15.5:0/1333744033 shutdown_connections 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8d8044740 1 -- 172.21.15.5:0/1333744033 wait complete. 2024-09-19T06:13:43.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.372+0000 7fa8d8044740 20 client.18144 trim_cache size 0 max 16384 2024-09-19T06:13:43.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.386+0000 7fa8d8044740 1 Processor -- start 2024-09-19T06:13:43.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.386+0000 7fa8d8044740 1 -- start start 2024-09-19T06:13:43.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.386+0000 7fa8d8044740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 0x55eb272ea8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.386+0000 7fa8d8044740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e4b40 0x55eb272dec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.386+0000 7fa8d8044740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272ee370 0x55eb272e6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.386+0000 7fa8d8044740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01af990 con 0x55eb272e35d0 2024-09-19T06:13:43.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.386+0000 7fa8d8044740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b1590 con 0x55eb272ee370 2024-09-19T06:13:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.386+0000 7fa8d8044740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b0060 con 0x55eb272e4b40 2024-09-19T06:13:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.386+0000 7fa8ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 0x55eb272ea8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.386+0000 7fa8ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 0x55eb272ea8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52240/0 (socket says 172.21.15.5:52240) 2024-09-19T06:13:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.386+0000 7fa8ceffd640 1 -- 172.21.15.5:0/3154673857 learned_addr learned my addr 172.21.15.5:0/3154673857 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.387+0000 7fa8cf7fe640 1 --2- 172.21.15.5:0/3154673857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e4b40 0x55eb272dec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.387+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/3154673857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272ee370 0x55eb272e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.387+0000 7fa8ceffd640 1 -- 172.21.15.5:0/3154673857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e4b40 msgr2=0x55eb272dec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.387+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/3154673857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e4b40 0x55eb272dec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.387+0000 7fa8ceffd640 1 -- 172.21.15.5:0/3154673857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272ee370 msgr2=0x55eb272e6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.387+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/3154673857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272ee370 0x55eb272e6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.387+0000 7fa8ceffd640 1 -- 172.21.15.5:0/3154673857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb272eb900 con 0x55eb272e35d0 2024-09-19T06:13:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.387+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/3154673857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 0x55eb272ea8c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4006cc0 tx=0x7fa8c4016840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e103d121ff561f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.387+0000 7fa8b57fa640 1 -- 172.21.15.5:0/3154673857 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c403b070 con 0x55eb272e35d0 2024-09-19T06:13:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.387+0000 7fa8b57fa640 1 -- 172.21.15.5:0/3154673857 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c400a440 con 0x55eb272e35d0 2024-09-19T06:13:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.387+0000 7fa8b57fa640 1 -- 172.21.15.5:0/3154673857 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4029040 con 0x55eb272e35d0 2024-09-19T06:13:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.388+0000 7fa8d8044740 1 -- 172.21.15.5:0/3154673857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 msgr2=0x55eb272ea8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.388+0000 7fa8d8044740 1 --2- 172.21.15.5:0/3154673857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 0x55eb272ea8c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4006cc0 tx=0x7fa8c4016840 comp rx=0 tx=0).stop 2024-09-19T06:13:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.388+0000 7fa8d8044740 1 -- 172.21.15.5:0/3154673857 shutdown_connections 2024-09-19T06:13:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.388+0000 7fa8d8044740 1 --2- 172.21.15.5:0/3154673857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272ee370 0x55eb272e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.388+0000 7fa8d8044740 1 --2- 172.21.15.5:0/3154673857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e4b40 0x55eb272dec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.388+0000 7fa8d8044740 1 --2- 172.21.15.5:0/3154673857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272e35d0 0x55eb272ea8c0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.388+0000 7fa8d8044740 1 -- 172.21.15.5:0/3154673857 >> 172.21.15.5:0/3154673857 conn(0x55eb272ac310 msgr2=0x55eb272de500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.388+0000 7fa8d8044740 1 -- 172.21.15.5:0/3154673857 shutdown_connections 2024-09-19T06:13:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.388+0000 7fa8d8044740 1 -- 172.21.15.5:0/3154673857 wait complete. 2024-09-19T06:13:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.388+0000 7fa8d8044740 1 -- start start 2024-09-19T06:13:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.388+0000 7fa8d8044740 1 Processor -- start 2024-09-19T06:13:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8d8044740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e35d0 0x55eb272f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8d8044740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272e4b40 0x55eb272f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8d8044740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272ee370 0x55eb272f2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8d8044740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b0ec0 con 0x55eb272ee370 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8d8044740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b0790 con 0x55eb272e4b40 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8d8044740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01ad990 con 0x55eb272e35d0 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8d8044740 10 client.0 _ll_register_callbacks cb 0x7fa8c4019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272ee370 0x55eb272f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272ee370 0x55eb272f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52242/0 (socket says 172.21.15.5:52242) 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8ceffd640 1 -- 172.21.15.5:0/4015457200 learned_addr learned my addr 172.21.15.5:0/4015457200 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8cf7fe640 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272e4b40 0x55eb272f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e35d0 0x55eb272f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8ceffd640 1 -- 172.21.15.5:0/4015457200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e35d0 msgr2=0x55eb272f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e35d0 0x55eb272f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8ceffd640 1 -- 172.21.15.5:0/4015457200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272e4b40 msgr2=0x55eb272f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272e4b40 0x55eb272f1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8ceffd640 1 -- 172.21.15.5:0/4015457200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb272f2ba0 con 0x55eb272ee370 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8ceffd640 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272ee370 0x55eb272f2370 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4001eb0 tx=0x7fa8c4018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7725039263d923 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.389+0000 7fa8aeffd640 1 -- 172.21.15.5:0/4015457200 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c403b070 con 0x55eb272ee370 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8aeffd640 1 -- 172.21.15.5:0/4015457200 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7fa8c40159b0 con 0x55eb272ee370 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8d8044740 1 -- 172.21.15.5:0/4015457200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c400a440 con 0x55eb272ee370 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8aeffd640 1 -- 172.21.15.5:0/4015457200 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa8c4010e50 con 0x55eb272ee370 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8d8044740 10 client.18150 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8d8044740 1 -- 172.21.15.5:0/4015457200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55eb273687c0 con 0x55eb272ee370 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8aeffd640 1 -- 172.21.15.5:0/4015457200 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c400a440 con 0x55eb272ee370 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8aeffd640 1 -- 172.21.15.5:0/4015457200 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fa8c4021040 con 0x55eb272ee370 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8d8044740 10 client.18150 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8d8044740 10 client.18150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8d8044740 20 client.18150 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8d8044740 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eb27368da0 0x55eb2736b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8d8044740 4 client.18150 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:13:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8d8044740 1 -- 172.21.15.5:0/4015457200 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb27250d70 con 0x55eb27368da0 2024-09-19T06:13:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.390+0000 7fa8cf7fe640 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eb27368da0 0x55eb2736b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8cf7fe640 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eb27368da0 0x55eb2736b1c0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e56ca0a2e79d53f0 server_cookie=9224972a77f0d33c in_seq=0 out_seq=0 2024-09-19T06:13:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8aeffd640 10 client.18150 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8aeffd640 1 -- 172.21.15.5:0/4015457200 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb27250d70 con 0x55eb27368da0 2024-09-19T06:13:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8aeffd640 10 client.18150 handle_command_reply: tid=0 2024-09-19T06:13:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8d8044740 1 client.18150 shutdown 2024-09-19T06:13:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8d8044740 1 -- 172.21.15.5:0/4015457200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272ee370 msgr2=0x55eb272f2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8d8044740 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272ee370 0x55eb272f2370 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4001eb0 tx=0x7fa8c4018040 comp rx=0 tx=0).stop 2024-09-19T06:13:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8d8044740 1 -- 172.21.15.5:0/4015457200 shutdown_connections 2024-09-19T06:13:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8d8044740 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eb27368da0 0x55eb2736b1c0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8d8044740 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb272ee370 0x55eb272f2370 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8d8044740 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb272e4b40 0x55eb272f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8d8044740 1 --2- 172.21.15.5:0/4015457200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb272e35d0 0x55eb272f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8d8044740 1 -- 172.21.15.5:0/4015457200 >> 172.21.15.5:0/4015457200 conn(0x55eb272ac310 msgr2=0x55eb272ded00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8aeffd640 0 client.18150 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8d8044740 1 -- 172.21.15.5:0/4015457200 shutdown_connections 2024-09-19T06:13:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8d8044740 1 -- 172.21.15.5:0/4015457200 wait complete. 2024-09-19T06:13:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.391+0000 7fa8d8044740 20 client.18150 trim_cache size 0 max 16384 2024-09-19T06:13:43.398 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:13:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.392+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/3516390157 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa8b80a3430 msgr2=0x7fa8b80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.392+0000 7fa8d5c6b640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa8b80a3430 0x7fa8b80a58f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80eef90 tx=0x7fa8c808f3b0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.392+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/3516390157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8d0106f50 msgr2=0x7fa8d0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.392+0000 7fa8d5c6b640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8d0106f50 0x7fa8d0101cf0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8005cc0 tx=0x7fa8c8003ba0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.392+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/3516390157 shutdown_connections 2024-09-19T06:13:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.392+0000 7fa8d5c6b640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8d0106f50 0x7fa8d0101cf0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.392+0000 7fa8d5c6b640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8d0100ca0 0x7fa8d01017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.392+0000 7fa8d5c6b640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa8b80a3430 0x7fa8b80a58f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.392+0000 7fa8d5c6b640 1 --2- 172.21.15.5:0/3516390157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d01002f0 0x7fa8d0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.392+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/3516390157 >> 172.21.15.5:0/3516390157 conn(0x7fa8d0077210 msgr2=0x7fa8d0105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.392+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/3516390157 shutdown_connections 2024-09-19T06:13:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:43.392+0000 7fa8d5c6b640 1 -- 172.21.15.5:0/3516390157 wait complete. 2024-09-19T06:13:43.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:43 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-19T06:13:43.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:43 smithi157 ceph-mon[35346]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T06:13:43.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:43 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-19T06:13:43.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:43 smithi049 ceph-mon[34640]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T06:13:43.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 251946 2.39 MB/sec execute 429 sec latency 738.488 ms 2024-09-19T06:13:44.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:44 smithi005 ceph-mon[30955]: pgmap v323: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 975 KiB/s wr, 104 op/s 2024-09-19T06:13:44.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:44 smithi157 ceph-mon[35346]: pgmap v323: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 975 KiB/s wr, 104 op/s 2024-09-19T06:13:44.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:44 smithi049 ceph-mon[34640]: pgmap v323: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 975 KiB/s wr, 104 op/s 2024-09-19T06:13:44.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 251946 2.38 MB/sec execute 430 sec latency 1738.598 ms 2024-09-19T06:13:45.076 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:13:45.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:13:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:13:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.810+0000 7f536e3aa640 1 -- 172.21.15.5:0/1837254575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5368106a80 msgr2=0x7f5368106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.810+0000 7f536e3aa640 1 --2- 172.21.15.5:0/1837254575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5368106a80 0x7f5368106e60 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f5358009930 tx=0x7f535802a420 comp rx=0 tx=0).stop 2024-09-19T06:13:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.810+0000 7f536e3aa640 1 -- 172.21.15.5:0/1837254575 shutdown_connections 2024-09-19T06:13:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.810+0000 7f536e3aa640 1 --2- 172.21.15.5:0/1837254575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5368101420 0x7f536810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.810+0000 7f536e3aa640 1 --2- 172.21.15.5:0/1837254575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368100a80 0x7f5368100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.810+0000 7f536e3aa640 1 --2- 172.21.15.5:0/1837254575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5368106a80 0x7f5368106e60 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.810+0000 7f536e3aa640 1 -- 172.21.15.5:0/1837254575 >> 172.21.15.5:0/1837254575 conn(0x7f53680fc7a0 msgr2=0x7f53680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.811+0000 7f536e3aa640 1 -- 172.21.15.5:0/1837254575 shutdown_connections 2024-09-19T06:13:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.811+0000 7f536e3aa640 1 -- 172.21.15.5:0/1837254575 wait complete. 2024-09-19T06:13:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.811+0000 7f536e3aa640 1 Processor -- start 2024-09-19T06:13:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.811+0000 7f536e3aa640 1 -- start start 2024-09-19T06:13:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536e3aa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5368100a80 0x7f5368199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536e3aa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368101420 0x7f53681996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536e3aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5368106a80 0x7f53681a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536e3aa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5368110cf0 con 0x7f5368106a80 2024-09-19T06:13:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536e3aa640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5368110b70 con 0x7f5368101420 2024-09-19T06:13:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536e3aa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5368110e70 con 0x7f5368100a80 2024-09-19T06:13:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536c947640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5368106a80 0x7f53681a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536c947640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5368106a80 0x7f53681a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50804/0 (socket says 172.21.15.5:50804) 2024-09-19T06:13:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536c947640 1 -- 172.21.15.5:0/3405830027 learned_addr learned my addr 172.21.15.5:0/3405830027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f53677fe640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368101420 0x7f53681996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f5367fff640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5368100a80 0x7f5368199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536c947640 1 -- 172.21.15.5:0/3405830027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5368100a80 msgr2=0x7f5368199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536c947640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5368100a80 0x7f5368199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536c947640 1 -- 172.21.15.5:0/3405830027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368101420 msgr2=0x7f53681996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536c947640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368101420 0x7f53681996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536c947640 1 -- 172.21.15.5:0/3405830027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53681a4740 con 0x7f5368106a80 2024-09-19T06:13:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536c947640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5368106a80 0x7f53681a4200 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f536000cad0 tx=0x7f536000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f53657fa640 1 -- 172.21.15.5:0/3405830027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536000ad60 con 0x7f5368106a80 2024-09-19T06:13:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536e3aa640 1 -- 172.21.15.5:0/3405830027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53681a4910 con 0x7f5368106a80 2024-09-19T06:13:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f53657fa640 1 -- 172.21.15.5:0/3405830027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5360005020 con 0x7f5368106a80 2024-09-19T06:13:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f53657fa640 1 -- 172.21.15.5:0/3405830027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536000f6b0 con 0x7f5368106a80 2024-09-19T06:13:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.812+0000 7f536e3aa640 1 -- 172.21.15.5:0/3405830027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53681a4e00 con 0x7f5368106a80 2024-09-19T06:13:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.815+0000 7f53657fa640 1 -- 172.21.15.5:0/3405830027 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f53600051d0 con 0x7f5368106a80 2024-09-19T06:13:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.815+0000 7f5370783740 1 Processor -- start 2024-09-19T06:13:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.815+0000 7f5370783740 1 -- start start 2024-09-19T06:13:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.815+0000 7f5370783740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd75fb60 0x55f3dd75ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.815+0000 7f5370783740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd760480 0x55f3dd76a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.815+0000 7f5370783740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 0x55f3dd76d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.815+0000 7f5370783740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f3dd6c5280 con 0x55f3dd76ad40 2024-09-19T06:13:45.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.815+0000 7f5370783740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f3dd66cb10 con 0x55f3dd760480 2024-09-19T06:13:45.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.815+0000 7f5370783740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f3dd6c1240 con 0x55f3dd75fb60 2024-09-19T06:13:45.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f53677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 0x55f3dd76d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f53677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 0x55f3dd76d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50806/0 (socket says 172.21.15.5:50806) 2024-09-19T06:13:45.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f53677fe640 1 -- 172.21.15.5:0/701108226 learned_addr learned my addr 172.21.15.5:0/701108226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:45.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f5367fff640 1 --2- 172.21.15.5:0/701108226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd760480 0x55f3dd76a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f53677fe640 1 --2- 172.21.15.5:0/701108226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd75fb60 0x55f3dd75ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f53657fa640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53500a33b0 0x7f53500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f53677fe640 1 -- 172.21.15.5:0/701108226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd75fb60 msgr2=0x55f3dd75ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:45.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f53677fe640 1 --2- 172.21.15.5:0/701108226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd75fb60 0x55f3dd75ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f53677fe640 1 -- 172.21.15.5:0/701108226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd760480 msgr2=0x55f3dd76a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:45.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f53677fe640 1 --2- 172.21.15.5:0/701108226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd760480 0x55f3dd76a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f53677fe640 1 -- 172.21.15.5:0/701108226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3dd6791f0 con 0x55f3dd76ad40 2024-09-19T06:13:45.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f5367fff640 1 --2- 172.21.15.5:0/701108226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd760480 0x55f3dd76a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:45.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f536c947640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53500a33b0 0x7f53500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f53677fe640 1 --2- 172.21.15.5:0/701108226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 0x55f3dd76d130 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f535c0117e0 tx=0x7f535c00a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cc6547e6eb2ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:45.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f53657fa640 1 -- 172.21.15.5:0/3405830027 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f53600cfc90 con 0x7f5368106a80 2024-09-19T06:13:45.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f534e7fc640 1 -- 172.21.15.5:0/701108226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c011e20 con 0x55f3dd76ad40 2024-09-19T06:13:45.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f534e7fc640 1 -- 172.21.15.5:0/701108226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f535c00fe80 con 0x55f3dd76ad40 2024-09-19T06:13:45.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.816+0000 7f534e7fc640 1 -- 172.21.15.5:0/701108226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c004a00 con 0x55f3dd76ad40 2024-09-19T06:13:45.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.817+0000 7f536c947640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53500a33b0 0x7f53500a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f53600cd470 tx=0x7f5360005c50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:45.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.817+0000 7f5370783740 1 -- 172.21.15.5:0/701108226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 msgr2=0x55f3dd76d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:45.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.817+0000 7f5370783740 1 --2- 172.21.15.5:0/701108226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 0x55f3dd76d130 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f535c0117e0 tx=0x7f535c00a760 comp rx=0 tx=0).stop 2024-09-19T06:13:45.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.817+0000 7f5370783740 1 -- 172.21.15.5:0/701108226 shutdown_connections 2024-09-19T06:13:45.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.817+0000 7f5370783740 1 --2- 172.21.15.5:0/701108226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 0x55f3dd76d130 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.817+0000 7f5370783740 1 --2- 172.21.15.5:0/701108226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd760480 0x55f3dd76a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.817+0000 7f5370783740 1 --2- 172.21.15.5:0/701108226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd75fb60 0x55f3dd75ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.817+0000 7f5370783740 1 -- 172.21.15.5:0/701108226 >> 172.21.15.5:0/701108226 conn(0x55f3dd75c0f0 msgr2=0x55f3dd75c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:45.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.817+0000 7f5370783740 1 -- 172.21.15.5:0/701108226 shutdown_connections 2024-09-19T06:13:45.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.817+0000 7f5370783740 1 -- 172.21.15.5:0/701108226 wait complete. 2024-09-19T06:13:45.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.818+0000 7f5370783740 1 -- start start 2024-09-19T06:13:45.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.818+0000 7f5370783740 1 Processor -- start 2024-09-19T06:13:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.818+0000 7f5370783740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd75fb60 0x55f3dd767cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.818+0000 7f5370783740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 0x55f3dd7681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.818+0000 7f5370783740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd76ad40 0x55f3dd768730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.818+0000 7f5370783740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f3dd76d910 con 0x55f3dd760480 2024-09-19T06:13:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.818+0000 7f5370783740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f3dd67ff10 con 0x55f3dd76ad40 2024-09-19T06:13:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.818+0000 7f5370783740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f3dd76da90 con 0x55f3dd75fb60 2024-09-19T06:13:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.818+0000 7f5370783740 10 client.0 _ll_register_callbacks cb 0x55f3dd648a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.818+0000 7f5367fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 0x55f3dd7681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.818+0000 7f5367fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 0x55f3dd7681f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50808/0 (socket says 172.21.15.5:50808) 2024-09-19T06:13:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.818+0000 7f5367fff640 1 -- 172.21.15.5:0/1928572736 learned_addr learned my addr 172.21.15.5:0/1928572736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f53677fe640 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd75fb60 0x55f3dd767cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f53677fe640 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd76ad40 0x55f3dd768730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f5367fff640 1 -- 172.21.15.5:0/1928572736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd75fb60 msgr2=0x55f3dd767cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f5367fff640 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd75fb60 0x55f3dd767cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f5367fff640 1 -- 172.21.15.5:0/1928572736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd76ad40 msgr2=0x55f3dd768730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f5367fff640 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd76ad40 0x55f3dd768730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f5367fff640 1 -- 172.21.15.5:0/1928572736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3dd4366a0 con 0x55f3dd760480 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f53677fe640 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd76ad40 0x55f3dd768730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f5367fff640 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 0x55f3dd7681f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5358033000 tx=0x7f53580316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111cb3b8a243b009 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f53477fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f53477fe640 1 -- 172.21.15.5:0/1928572736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358035040 con 0x55f3dd760480 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f53477fe640 1 -- 172.21.15.5:0/1928572736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f5358004050 con 0x55f3dd760480 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f5370783740 1 -- 172.21.15.5:0/1928572736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f3dd6d27f0 con 0x55f3dd760480 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f53477fe640 1 -- 172.21.15.5:0/1928572736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5358004250 con 0x55f3dd760480 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f5370783740 10 client.18162 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f5370783740 1 -- 172.21.15.5:0/1928572736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f3dd7795a0 con 0x55f3dd760480 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.819+0000 7f53477fe640 1 -- 172.21.15.5:0/1928572736 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f3dd6d27f0 con 0x55f3dd760480 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.820+0000 7f53477fe640 1 -- 172.21.15.5:0/1928572736 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f5358004a60 con 0x55f3dd760480 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.820+0000 7f5370783740 10 client.18162 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.820+0000 7f5370783740 10 client.18162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.820+0000 7f5370783740 20 client.18162 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.820+0000 7f5370783740 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f3dd77ab00 0x55f3dd77cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.820+0000 7f5370783740 4 client.18162 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.820+0000 7f5370783740 1 -- 172.21.15.5:0/1928572736 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3dd6ccd00 con 0x55f3dd77ab00 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.820+0000 7f53677fe640 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f3dd77ab00 0x55f3dd77cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.820+0000 7f53677fe640 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f3dd77ab00 0x55f3dd77cf10 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec23d6049908dc9 server_cookie=f6e63502abafab43 in_seq=0 out_seq=0 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.820+0000 7f53477fe640 10 client.18162 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f53477fe640 1 -- 172.21.15.5:0/1928572736 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3dd6ccd00 con 0x55f3dd77ab00 2024-09-19T06:13:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f53477fe640 10 client.18162 handle_command_reply: tid=0 2024-09-19T06:13:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f5370783740 1 client.18162 shutdown 2024-09-19T06:13:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f5370783740 1 -- 172.21.15.5:0/1928572736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 msgr2=0x55f3dd7681f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f5370783740 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 0x55f3dd7681f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5358033000 tx=0x7f53580316d0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f5370783740 1 -- 172.21.15.5:0/1928572736 shutdown_connections 2024-09-19T06:13:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f5370783740 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f3dd77ab00 0x55f3dd77cf10 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f5370783740 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd76ad40 0x55f3dd768730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f5370783740 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 0x55f3dd7681f0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f5370783740 1 --2- 172.21.15.5:0/1928572736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd75fb60 0x55f3dd767cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f5370783740 1 -- 172.21.15.5:0/1928572736 >> 172.21.15.5:0/1928572736 conn(0x55f3dd75c0f0 msgr2=0x55f3dd761dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f53477fe640 0 client.18162 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f5370783740 1 -- 172.21.15.5:0/1928572736 shutdown_connections 2024-09-19T06:13:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f5370783740 1 -- 172.21.15.5:0/1928572736 wait complete. 2024-09-19T06:13:45.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.821+0000 7f5370783740 20 client.18162 trim_cache size 0 max 16384 2024-09-19T06:13:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f5370783740 1 Processor -- start 2024-09-19T06:13:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f5370783740 1 -- start start 2024-09-19T06:13:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f5370783740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd75fb60 0x55f3dd768580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f5370783740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd760480 0x55f3dd7827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f5370783740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 0x55f3dd782d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f5370783740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53681ad860 con 0x55f3dd76ad40 2024-09-19T06:13:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f5370783740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53681af460 con 0x55f3dd75fb60 2024-09-19T06:13:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f5370783740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53681adf30 con 0x55f3dd760480 2024-09-19T06:13:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f53677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 0x55f3dd782d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f53677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 0x55f3dd782d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50824/0 (socket says 172.21.15.5:50824) 2024-09-19T06:13:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f53677fe640 1 -- 172.21.15.5:0/489104283 learned_addr learned my addr 172.21.15.5:0/489104283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f53677fe640 1 --2- 172.21.15.5:0/489104283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd75fb60 0x55f3dd768580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 252013 2.38 MB/sec execute 431 sec latency 2365.993 ms 2024-09-19T06:13:45.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f5367fff640 1 --2- 172.21.15.5:0/489104283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd760480 0x55f3dd7827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f53677fe640 1 -- 172.21.15.5:0/489104283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd760480 msgr2=0x55f3dd7827e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f53677fe640 1 --2- 172.21.15.5:0/489104283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd760480 0x55f3dd7827e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f53677fe640 1 -- 172.21.15.5:0/489104283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd75fb60 msgr2=0x55f3dd768580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f53677fe640 1 --2- 172.21.15.5:0/489104283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd75fb60 0x55f3dd768580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.839+0000 7f53677fe640 1 -- 172.21.15.5:0/489104283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3dd76a580 con 0x55f3dd76ad40 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f53677fe640 1 --2- 172.21.15.5:0/489104283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 0x55f3dd782d20 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x55f3dd75bb20 tx=0x7f535c00a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdf8dc95c2ddd90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f534dffb640 1 -- 172.21.15.5:0/489104283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c00f9d0 con 0x55f3dd76ad40 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f534dffb640 1 -- 172.21.15.5:0/489104283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f535c00fb70 con 0x55f3dd76ad40 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f534dffb640 1 -- 172.21.15.5:0/489104283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c028240 con 0x55f3dd76ad40 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f5370783740 1 -- 172.21.15.5:0/489104283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 msgr2=0x55f3dd782d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f5370783740 1 --2- 172.21.15.5:0/489104283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 0x55f3dd782d20 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x55f3dd75bb20 tx=0x7f535c00a760 comp rx=0 tx=0).stop 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f5370783740 1 -- 172.21.15.5:0/489104283 shutdown_connections 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f5370783740 1 --2- 172.21.15.5:0/489104283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd76ad40 0x55f3dd782d20 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f5370783740 1 --2- 172.21.15.5:0/489104283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd760480 0x55f3dd7827e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f5370783740 1 --2- 172.21.15.5:0/489104283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd75fb60 0x55f3dd768580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f5370783740 1 -- 172.21.15.5:0/489104283 >> 172.21.15.5:0/489104283 conn(0x55f3dd75c0f0 msgr2=0x55f3dd769c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f5370783740 1 -- 172.21.15.5:0/489104283 shutdown_connections 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.840+0000 7f5370783740 1 -- 172.21.15.5:0/489104283 wait complete. 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f5370783740 1 -- start start 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f5370783740 1 Processor -- start 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f5370783740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd75fb60 0x55f3dd764de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f5370783740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 0x55f3dd765320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f5370783740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd76ad40 0x55f3dd765860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f5370783740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53681aed90 con 0x55f3dd760480 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f5370783740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53681ae660 con 0x55f3dd75fb60 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f5370783740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53681ab860 con 0x55f3dd76ad40 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f5370783740 10 client.0 _ll_register_callbacks cb 0x7f5358009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:45.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f5367fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 0x55f3dd765320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f5367fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 0x55f3dd765320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50838/0 (socket says 172.21.15.5:50838) 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f5367fff640 1 -- 172.21.15.5:0/3789688462 learned_addr learned my addr 172.21.15.5:0/3789688462 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.841+0000 7f53677fe640 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd75fb60 0x55f3dd764de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f53677fe640 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd76ad40 0x55f3dd765860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f5367fff640 1 -- 172.21.15.5:0/3789688462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd76ad40 msgr2=0x55f3dd765860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f5367fff640 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd76ad40 0x55f3dd765860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f5367fff640 1 -- 172.21.15.5:0/3789688462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd75fb60 msgr2=0x55f3dd764de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f5367fff640 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd75fb60 0x55f3dd764de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f5367fff640 1 -- 172.21.15.5:0/3789688462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3dd7e8d20 con 0x55f3dd760480 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f5367fff640 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 0x55f3dd765320 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5358001eb0 tx=0x7f5358005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bccbf36cf3ee4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f53477fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f53477fe640 1 -- 172.21.15.5:0/3789688462 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358035070 con 0x55f3dd760480 2024-09-19T06:13:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f53477fe640 1 -- 172.21.15.5:0/3789688462 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f53580047a0 con 0x55f3dd760480 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f5370783740 1 -- 172.21.15.5:0/3789688462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f535c00fb70 con 0x55f3dd760480 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f53477fe640 1 -- 172.21.15.5:0/3789688462 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5358053070 con 0x55f3dd760480 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f5370783740 10 client.18168 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f5370783740 1 -- 172.21.15.5:0/3789688462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f3dd7ea010 con 0x55f3dd760480 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.842+0000 7f53477fe640 1 -- 172.21.15.5:0/3789688462 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f535c00fb70 con 0x55f3dd760480 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.843+0000 7f53477fe640 1 -- 172.21.15.5:0/3789688462 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f53580051e0 con 0x55f3dd760480 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.843+0000 7f5370783740 10 client.18168 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.843+0000 7f5370783740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.843+0000 7f5370783740 20 client.18168 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.843+0000 7f5370783740 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f3dd7ea810 0x55f3dd7ecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.843+0000 7f5370783740 4 client.18168 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.843+0000 7f5370783740 1 -- 172.21.15.5:0/3789688462 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f3dd649110 con 0x55f3dd7ea810 2024-09-19T06:13:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.843+0000 7f53677fe640 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f3dd7ea810 0x55f3dd7ecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.843+0000 7f53677fe640 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f3dd7ea810 0x55f3dd7ecc30 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27627e4d491a94f4 server_cookie=48671c81b165d2cc in_seq=0 out_seq=0 2024-09-19T06:13:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.843+0000 7f53477fe640 10 client.18168 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f53477fe640 1 -- 172.21.15.5:0/3789688462 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f3dd649110 con 0x55f3dd7ea810 2024-09-19T06:13:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f53477fe640 10 client.18168 handle_command_reply: tid=0 2024-09-19T06:13:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f5370783740 1 client.18168 shutdown 2024-09-19T06:13:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f5370783740 1 -- 172.21.15.5:0/3789688462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 msgr2=0x55f3dd765320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f5370783740 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 0x55f3dd765320 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5358001eb0 tx=0x7f5358005fd0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f5370783740 1 -- 172.21.15.5:0/3789688462 shutdown_connections 2024-09-19T06:13:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f5370783740 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f3dd7ea810 0x55f3dd7ecc30 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f5370783740 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3dd76ad40 0x55f3dd765860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f5370783740 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3dd760480 0x55f3dd765320 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f5370783740 1 --2- 172.21.15.5:0/3789688462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3dd75fb60 0x55f3dd764de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f5370783740 1 -- 172.21.15.5:0/3789688462 >> 172.21.15.5:0/3789688462 conn(0x55f3dd75c0f0 msgr2=0x55f3dd769320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f53477fe640 0 client.18168 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f5370783740 1 -- 172.21.15.5:0/3789688462 shutdown_connections 2024-09-19T06:13:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f5370783740 1 -- 172.21.15.5:0/3789688462 wait complete. 2024-09-19T06:13:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.844+0000 7f5370783740 20 client.18168 trim_cache size 0 max 16384 2024-09-19T06:13:45.852 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:13:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:13:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "54964366-acdd-41db-bd11-3c20dbe78617", 2024-09-19T06:13:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:13:45.852 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:13:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.845+0000 7f536e3aa640 1 -- 172.21.15.5:0/3405830027 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53500a33b0 msgr2=0x7f53500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.845+0000 7f536e3aa640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53500a33b0 0x7f53500a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f53600cd470 tx=0x7f5360005c50 comp rx=0 tx=0).stop 2024-09-19T06:13:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.845+0000 7f536e3aa640 1 -- 172.21.15.5:0/3405830027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5368106a80 msgr2=0x7f53681a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.845+0000 7f536e3aa640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5368106a80 0x7f53681a4200 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f536000cad0 tx=0x7f536000ced0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.845+0000 7f536e3aa640 1 -- 172.21.15.5:0/3405830027 shutdown_connections 2024-09-19T06:13:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.845+0000 7f536e3aa640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5368106a80 0x7f53681a4200 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.845+0000 7f536e3aa640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368101420 0x7f53681996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.845+0000 7f536e3aa640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53500a33b0 0x7f53500a5870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.845+0000 7f536e3aa640 1 --2- 172.21.15.5:0/3405830027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5368100a80 0x7f5368199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.845+0000 7f536e3aa640 1 -- 172.21.15.5:0/3405830027 >> 172.21.15.5:0/3405830027 conn(0x7f53680fc7a0 msgr2=0x7f53680fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.845+0000 7f536e3aa640 1 -- 172.21.15.5:0/3405830027 shutdown_connections 2024-09-19T06:13:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:45.845+0000 7f536e3aa640 1 -- 172.21.15.5:0/3405830027 wait complete. 2024-09-19T06:13:46.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:46 smithi005 ceph-mon[30955]: pgmap v324: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 793 KiB/s wr, 82 op/s 2024-09-19T06:13:46.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:46 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:13:46.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:46 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:46.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:46 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:46.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:46 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:46.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:46 smithi157 ceph-mon[35346]: pgmap v324: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 793 KiB/s wr, 82 op/s 2024-09-19T06:13:46.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:46 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:13:46.671 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:46 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:46.671 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:46 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:46.671 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:46 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:46.672 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:46 smithi049 ceph-mon[34640]: pgmap v324: 97 pgs: 97 active+clean; 136 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 793 KiB/s wr, 82 op/s 2024-09-19T06:13:46.672 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:46 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:13:46.672 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:46 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:46.672 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:46 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:46.672 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:46 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:46.673 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:13:46.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 252292 2.37 MB/sec execute 432 sec latency 135.043 ms 2024-09-19T06:13:46.956 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:13:47.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:47 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:47.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:47 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:47 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:47.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:47 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:47 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:47.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:47 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:47.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:47 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:47 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:47.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:47 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:47 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:13:47.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:47 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:47 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.450+0000 7f679db43640 1 -- 172.21.15.5:0/1539187683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798102790 msgr2=0x7f67981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:47.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.450+0000 7f679db43640 1 --2- 172.21.15.5:0/1539187683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798102790 0x7f67981067e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6780009900 tx=0x7f678002a410 comp rx=0 tx=0).stop 2024-09-19T06:13:47.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.451+0000 7f679db43640 1 -- 172.21.15.5:0/1539187683 shutdown_connections 2024-09-19T06:13:47.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.451+0000 7f679db43640 1 --2- 172.21.15.5:0/1539187683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6798106f10 0x7f679810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.451+0000 7f679db43640 1 --2- 172.21.15.5:0/1539187683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798102790 0x7f67981067e0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.451+0000 7f679db43640 1 --2- 172.21.15.5:0/1539187683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798101de0 0x7f67981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.451+0000 7f679db43640 1 -- 172.21.15.5:0/1539187683 >> 172.21.15.5:0/1539187683 conn(0x7f67980fd8c0 msgr2=0x7f67980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.451+0000 7f679db43640 1 -- 172.21.15.5:0/1539187683 shutdown_connections 2024-09-19T06:13:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.451+0000 7f679db43640 1 -- 172.21.15.5:0/1539187683 wait complete. 2024-09-19T06:13:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.451+0000 7f679db43640 1 Processor -- start 2024-09-19T06:13:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.451+0000 7f679db43640 1 -- start start 2024-09-19T06:13:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f679db43640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798101de0 0x7f6798198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f679db43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798102790 0x7f67981988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f679db43640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6798106f10 0x7f679819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f679db43640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6798111e20 con 0x7f6798102790 2024-09-19T06:13:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f679db43640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6798111ca0 con 0x7f6798101de0 2024-09-19T06:13:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f679db43640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6798111fa0 con 0x7f6798106f10 2024-09-19T06:13:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f678effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798102790 0x7f67981988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f678effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798102790 0x7f67981988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50856/0 (socket says 172.21.15.5:50856) 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f678effd640 1 -- 172.21.15.5:0/1332794981 learned_addr learned my addr 172.21.15.5:0/1332794981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f67977fe640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798101de0 0x7f6798198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f6797fff640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6798106f10 0x7f679819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f678effd640 1 -- 172.21.15.5:0/1332794981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6798106f10 msgr2=0x7f679819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f678effd640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6798106f10 0x7f679819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f678effd640 1 -- 172.21.15.5:0/1332794981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798101de0 msgr2=0x7f6798198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f678effd640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798101de0 0x7f6798198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f678effd640 1 -- 172.21.15.5:0/1332794981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679819d330 con 0x7f6798102790 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f67977fe640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798101de0 0x7f6798198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.452+0000 7f678effd640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798102790 0x7f67981988c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6780005940 tx=0x7f67800040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.453+0000 7f67957fa640 1 -- 172.21.15.5:0/1332794981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678003f030 con 0x7f6798102790 2024-09-19T06:13:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.453+0000 7f679db43640 1 -- 172.21.15.5:0/1332794981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679819d5c0 con 0x7f6798102790 2024-09-19T06:13:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.453+0000 7f67957fa640 1 -- 172.21.15.5:0/1332794981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67800044c0 con 0x7f6798102790 2024-09-19T06:13:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.453+0000 7f67957fa640 1 -- 172.21.15.5:0/1332794981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780035070 con 0x7f6798102790 2024-09-19T06:13:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.453+0000 7f679db43640 1 -- 172.21.15.5:0/1332794981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67981a4f00 con 0x7f6798102790 2024-09-19T06:13:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.455+0000 7f67957fa640 1 -- 172.21.15.5:0/1332794981 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6780042030 con 0x7f6798102790 2024-09-19T06:13:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.455+0000 7f679ff1c740 1 Processor -- start 2024-09-19T06:13:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.455+0000 7f679ff1c740 1 -- start start 2024-09-19T06:13:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.455+0000 7f679ff1c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef02880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f679ff1c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 0x559f5ef0d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f679ff1c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef0d680 0x559f5ef0fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f679ff1c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559f5ee682e0 con 0x559f5ef0d680 2024-09-19T06:13:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f679ff1c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559f5ee0fce0 con 0x559f5ef02dc0 2024-09-19T06:13:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f679ff1c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559f5ee642a0 con 0x559f5ef024a0 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f67977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 0x559f5ef0d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f6797fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef02880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f67977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 0x559f5ef0d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36992/0 (socket says 172.21.15.5:36992) 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f67977fe640 1 -- 172.21.15.5:0/3150863847 learned_addr learned my addr 172.21.15.5:0/3150863847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f678effd640 1 --2- 172.21.15.5:0/3150863847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef0d680 0x559f5ef0fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f67977fe640 1 -- 172.21.15.5:0/3150863847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 msgr2=0x559f5ef02880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f67977fe640 1 --2- 172.21.15.5:0/3150863847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef02880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f67977fe640 1 -- 172.21.15.5:0/3150863847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef0d680 msgr2=0x559f5ef0fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f67977fe640 1 --2- 172.21.15.5:0/3150863847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef0d680 0x559f5ef0fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f67977fe640 1 -- 172.21.15.5:0/3150863847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5edf4440 con 0x559f5ef02dc0 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f6797fff640 1 --2- 172.21.15.5:0/3150863847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef02880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f67957fa640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67840a33b0 0x7f67840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f67977fe640 1 --2- 172.21.15.5:0/3150863847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 0x559f5ef0d140 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f67880115f0 tx=0x7f6788011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5fa34db03631657 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.456+0000 7f6797fff640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67840a33b0 0x7f67840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.457+0000 7f67957fa640 1 -- 172.21.15.5:0/1332794981 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f67800f0060 con 0x7f6798102790 2024-09-19T06:13:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.457+0000 7f678dffb640 1 -- 172.21.15.5:0/3150863847 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678801e070 con 0x559f5ef02dc0 2024-09-19T06:13:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.457+0000 7f678dffb640 1 -- 172.21.15.5:0/3150863847 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f678800faf0 con 0x559f5ef02dc0 2024-09-19T06:13:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.457+0000 7f678dffb640 1 -- 172.21.15.5:0/3150863847 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788019ac0 con 0x559f5ef02dc0 2024-09-19T06:13:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.457+0000 7f679ff1c740 1 -- 172.21.15.5:0/3150863847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 msgr2=0x559f5ef0d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.457+0000 7f6797fff640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67840a33b0 0x7f67840a5870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6790009fd0 tx=0x7f6790009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.457+0000 7f679ff1c740 1 --2- 172.21.15.5:0/3150863847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 0x559f5ef0d140 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f67880115f0 tx=0x7f6788011ab0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.457+0000 7f679ff1c740 1 -- 172.21.15.5:0/3150863847 shutdown_connections 2024-09-19T06:13:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.457+0000 7f679ff1c740 1 --2- 172.21.15.5:0/3150863847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef0d680 0x559f5ef0fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.457+0000 7f679ff1c740 1 --2- 172.21.15.5:0/3150863847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 0x559f5ef0d140 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.457+0000 7f679ff1c740 1 --2- 172.21.15.5:0/3150863847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef02880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.457+0000 7f679ff1c740 1 -- 172.21.15.5:0/3150863847 >> 172.21.15.5:0/3150863847 conn(0x559f5eefea30 msgr2=0x559f5eefee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.458+0000 7f679ff1c740 1 -- 172.21.15.5:0/3150863847 shutdown_connections 2024-09-19T06:13:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.458+0000 7f679ff1c740 1 -- 172.21.15.5:0/3150863847 wait complete. 2024-09-19T06:13:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.458+0000 7f679ff1c740 1 -- start start 2024-09-19T06:13:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.458+0000 7f679ff1c740 1 Processor -- start 2024-09-19T06:13:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.458+0000 7f679ff1c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef0cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f679ff1c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef02dc0 0x559f5ef0a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f679ff1c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef0d680 0x559f5ef0a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f678effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef02dc0 0x559f5ef0a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f678effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef02dc0 0x559f5ef0a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50862/0 (socket says 172.21.15.5:50862) 2024-09-19T06:13:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f67977fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef0cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f678effd640 1 -- 172.21.15.5:0/558934806 learned_addr learned my addr 172.21.15.5:0/558934806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f679ff1c740 1 -- 172.21.15.5:0/558934806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559f5ef10250 con 0x559f5ef02dc0 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f679ff1c740 1 -- 172.21.15.5:0/558934806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559f5ee23050 con 0x559f5ef0d680 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f679ff1c740 1 -- 172.21.15.5:0/558934806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559f5ef103d0 con 0x559f5ef024a0 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f6797fff640 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef0d680 0x559f5ef0a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f679ff1c740 10 client.0 _ll_register_callbacks cb 0x559f5ee63160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f67977fe640 1 -- 172.21.15.5:0/558934806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef0d680 msgr2=0x559f5ef0a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f67977fe640 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef0d680 0x559f5ef0a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f67977fe640 1 -- 172.21.15.5:0/558934806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef02dc0 msgr2=0x559f5ef0a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f67977fe640 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef02dc0 0x559f5ef0a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f67977fe640 1 -- 172.21.15.5:0/558934806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5ebd93c0 con 0x559f5ef024a0 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f678effd640 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef02dc0 0x559f5ef0a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f6797fff640 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef0d680 0x559f5ef0a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f67977fe640 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef0cfd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f67880115c0 tx=0x7f678800f8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba1e88bfecd0939a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.459+0000 7f6776ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.460+0000 7f6776ffd640 1 -- 172.21.15.5:0/558934806 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788004250 con 0x559f5ef024a0 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.460+0000 7f679ff1c740 1 -- 172.21.15.5:0/558934806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f678800faf0 con 0x559f5ef024a0 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.460+0000 7f6776ffd640 1 -- 172.21.15.5:0/558934806 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f67880043f0 con 0x559f5ef024a0 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.460+0000 7f6776ffd640 1 -- 172.21.15.5:0/558934806 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6788025980 con 0x559f5ef024a0 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.460+0000 7f679ff1c740 10 client.27622 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.460+0000 7f679ff1c740 1 -- 172.21.15.5:0/558934806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559f5ef1a3c0 con 0x559f5ef024a0 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.460+0000 7f6776ffd640 1 -- 172.21.15.5:0/558934806 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f678800faf0 con 0x559f5ef024a0 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.460+0000 7f6776ffd640 1 -- 172.21.15.5:0/558934806 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f6788019840 con 0x559f5ef024a0 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.460+0000 7f679ff1c740 10 client.27622 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.460+0000 7f679ff1c740 10 client.27622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.461+0000 7f679ff1c740 20 client.27622 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.461+0000 7f679ff1c740 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559f5ef1b7d0 0x559f5ef1dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.461+0000 7f679ff1c740 4 client.27622 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.461+0000 7f679ff1c740 1 -- 172.21.15.5:0/558934806 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f5ee75850 con 0x559f5ef1b7d0 2024-09-19T06:13:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.461+0000 7f678effd640 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559f5ef1b7d0 0x559f5ef1dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.461+0000 7f678effd640 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559f5ef1b7d0 0x559f5ef1dc60 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9d9fbefa509fd1 server_cookie=8f454005d9c9f23d in_seq=0 out_seq=0 2024-09-19T06:13:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.461+0000 7f6776ffd640 10 client.27622 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f6776ffd640 1 -- 172.21.15.5:0/558934806 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f5ee75850 con 0x559f5ef1b7d0 2024-09-19T06:13:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f6776ffd640 10 client.27622 handle_command_reply: tid=0 2024-09-19T06:13:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f679ff1c740 1 client.27622 shutdown 2024-09-19T06:13:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f679ff1c740 1 -- 172.21.15.5:0/558934806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 msgr2=0x559f5ef0cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f679ff1c740 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef0cfd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f67880115c0 tx=0x7f678800f8f0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f679ff1c740 1 -- 172.21.15.5:0/558934806 shutdown_connections 2024-09-19T06:13:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f679ff1c740 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559f5ef1b7d0 0x559f5ef1dc60 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f679ff1c740 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef0d680 0x559f5ef0a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f6776ffd640 0 client.27622 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f679ff1c740 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef02dc0 0x559f5ef0a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f679ff1c740 1 --2- 172.21.15.5:0/558934806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef0cfd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f679ff1c740 1 -- 172.21.15.5:0/558934806 >> 172.21.15.5:0/558934806 conn(0x559f5eefea30 msgr2=0x559f5ef04710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.462+0000 7f679ff1c740 1 -- 172.21.15.5:0/558934806 shutdown_connections 2024-09-19T06:13:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.463+0000 7f679ff1c740 1 -- 172.21.15.5:0/558934806 wait complete. 2024-09-19T06:13:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.463+0000 7f679ff1c740 20 client.27622 trim_cache size 0 max 16384 2024-09-19T06:13:47.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.482+0000 7f679ff1c740 1 Processor -- start 2024-09-19T06:13:47.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.482+0000 7f679ff1c740 1 -- start start 2024-09-19T06:13:47.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.482+0000 7f679ff1c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.482+0000 7f679ff1c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 0x559f5ef07a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.482+0000 7f679ff1c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef0d680 0x559f5ef03ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f679ff1c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67981ab6c0 con 0x559f5ef0d680 2024-09-19T06:13:47.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f679ff1c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f67981ad2c0 con 0x559f5ef02dc0 2024-09-19T06:13:47.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f679ff1c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f67981abd90 con 0x559f5ef024a0 2024-09-19T06:13:47.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f6797fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef0d680 0x559f5ef03ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f678effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 0x559f5ef07a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f67977fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f6797fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef0d680 0x559f5ef03ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50876/0 (socket says 172.21.15.5:50876) 2024-09-19T06:13:47.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f6797fff640 1 -- 172.21.15.5:0/897740655 learned_addr learned my addr 172.21.15.5:0/897740655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f67977fe640 1 -- 172.21.15.5:0/897740655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 msgr2=0x559f5ef07a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f67977fe640 1 --2- 172.21.15.5:0/897740655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 0x559f5ef07a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f67977fe640 1 -- 172.21.15.5:0/897740655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef0d680 msgr2=0x559f5ef03ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f67977fe640 1 --2- 172.21.15.5:0/897740655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef0d680 0x559f5ef03ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f67977fe640 1 -- 172.21.15.5:0/897740655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5ef0ab50 con 0x559f5ef024a0 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f678effd640 1 --2- 172.21.15.5:0/897740655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 0x559f5ef07a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f6797fff640 1 --2- 172.21.15.5:0/897740655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef0d680 0x559f5ef03ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.483+0000 7f67977fe640 1 --2- 172.21.15.5:0/897740655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef074e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6788019e00 tx=0x7f6788005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac82383c8073db1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.484+0000 7f678d7fa640 1 -- 172.21.15.5:0/897740655 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678803a070 con 0x559f5ef024a0 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.484+0000 7f678d7fa640 1 -- 172.21.15.5:0/897740655 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6788029070 con 0x559f5ef024a0 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.484+0000 7f678d7fa640 1 -- 172.21.15.5:0/897740655 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67880035f0 con 0x559f5ef024a0 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.484+0000 7f679ff1c740 1 -- 172.21.15.5:0/897740655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 msgr2=0x559f5ef074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.484+0000 7f679ff1c740 1 --2- 172.21.15.5:0/897740655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef074e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6788019e00 tx=0x7f6788005970 comp rx=0 tx=0).stop 2024-09-19T06:13:47.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 -- 172.21.15.5:0/897740655 shutdown_connections 2024-09-19T06:13:47.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 --2- 172.21.15.5:0/897740655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef0d680 0x559f5ef03ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 --2- 172.21.15.5:0/897740655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef02dc0 0x559f5ef07a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 --2- 172.21.15.5:0/897740655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef024a0 0x559f5ef074e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 -- 172.21.15.5:0/897740655 >> 172.21.15.5:0/897740655 conn(0x559f5eefea30 msgr2=0x559f5ef09370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:47.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 -- 172.21.15.5:0/897740655 shutdown_connections 2024-09-19T06:13:47.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 -- 172.21.15.5:0/897740655 wait complete. 2024-09-19T06:13:47.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 -- start start 2024-09-19T06:13:47.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 Processor -- start 2024-09-19T06:13:47.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef024a0 0x559f5ef257a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef02dc0 0x559f5ef21b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef0d680 0x559f5ef22170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67981acbf0 con 0x559f5ef024a0 2024-09-19T06:13:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f67981ac4c0 con 0x559f5ef0d680 2024-09-19T06:13:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.485+0000 7f679ff1c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f67981a96c0 con 0x559f5ef02dc0 2024-09-19T06:13:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f679ff1c740 10 client.0 _ll_register_callbacks cb 0x7f678801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f67977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef024a0 0x559f5ef257a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f678effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef02dc0 0x559f5ef21b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f67977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef024a0 0x559f5ef257a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50890/0 (socket says 172.21.15.5:50890) 2024-09-19T06:13:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f678effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef02dc0 0x559f5ef21b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45716/0 (socket says 172.21.15.5:45716) 2024-09-19T06:13:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f67977fe640 1 -- 172.21.15.5:0/243202199 learned_addr learned my addr 172.21.15.5:0/243202199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f6797fff640 1 --2- 172.21.15.5:0/243202199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef0d680 0x559f5ef22170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f67977fe640 1 -- 172.21.15.5:0/243202199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef02dc0 msgr2=0x559f5ef21b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f67977fe640 1 --2- 172.21.15.5:0/243202199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef02dc0 0x559f5ef21b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f67977fe640 1 -- 172.21.15.5:0/243202199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef0d680 msgr2=0x559f5ef22170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f67977fe640 1 --2- 172.21.15.5:0/243202199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef0d680 0x559f5ef22170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f67977fe640 1 -- 172.21.15.5:0/243202199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5ef22970 con 0x559f5ef024a0 2024-09-19T06:13:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f67977fe640 1 --2- 172.21.15.5:0/243202199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef024a0 0x559f5ef257a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6788011510 tx=0x7f6788011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b713a6c4fa07cced server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.486+0000 7f6776ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.487+0000 7f6776ffd640 1 -- 172.21.15.5:0/243202199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678803a040 con 0x559f5ef024a0 2024-09-19T06:13:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.487+0000 7f6776ffd640 1 -- 172.21.15.5:0/243202199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7f67880052d0 con 0x559f5ef024a0 2024-09-19T06:13:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.487+0000 7f679ff1c740 1 -- 172.21.15.5:0/243202199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6788029070 con 0x559f5ef024a0 2024-09-19T06:13:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.487+0000 7f6776ffd640 1 -- 172.21.15.5:0/243202199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f678800faf0 con 0x559f5ef024a0 2024-09-19T06:13:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.487+0000 7f679ff1c740 10 client.18186 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.487+0000 7f679ff1c740 1 -- 172.21.15.5:0/243202199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559f5ef87040 con 0x559f5ef024a0 2024-09-19T06:13:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.487+0000 7f6776ffd640 1 -- 172.21.15.5:0/243202199 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6788029070 con 0x559f5ef024a0 2024-09-19T06:13:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.487+0000 7f6776ffd640 1 -- 172.21.15.5:0/243202199 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f6788003ce0 con 0x559f5ef024a0 2024-09-19T06:13:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.487+0000 7f679ff1c740 10 client.18186 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.487+0000 7f679ff1c740 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.488+0000 7f679ff1c740 20 client.18186 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.488+0000 7f679ff1c740 1 --2- 172.21.15.5:0/243202199 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559f5ef1b7d0 0x559f5ef89690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.488+0000 7f679ff1c740 4 client.18186 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:13:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.488+0000 7f679ff1c740 1 -- 172.21.15.5:0/243202199 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f5ee6fd60 con 0x559f5ef1b7d0 2024-09-19T06:13:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.488+0000 7f678effd640 1 --2- 172.21.15.5:0/243202199 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559f5ef1b7d0 0x559f5ef89690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.488+0000 7f678effd640 1 --2- 172.21.15.5:0/243202199 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559f5ef1b7d0 0x559f5ef89690 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae4c15d5514631 server_cookie=3bc5b08d707e3203 in_seq=0 out_seq=0 2024-09-19T06:13:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.488+0000 7f6776ffd640 10 client.18186 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.488+0000 7f6776ffd640 1 -- 172.21.15.5:0/243202199 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559f5ee6fd60 con 0x559f5ef1b7d0 2024-09-19T06:13:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.488+0000 7f6776ffd640 10 client.18186 handle_command_reply: tid=0 2024-09-19T06:13:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.488+0000 7f679ff1c740 1 client.18186 shutdown 2024-09-19T06:13:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.488+0000 7f679ff1c740 1 -- 172.21.15.5:0/243202199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef024a0 msgr2=0x559f5ef257a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.488+0000 7f679ff1c740 1 --2- 172.21.15.5:0/243202199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef024a0 0x559f5ef257a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6788011510 tx=0x7f6788011ab0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679ff1c740 1 -- 172.21.15.5:0/243202199 shutdown_connections 2024-09-19T06:13:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679ff1c740 1 --2- 172.21.15.5:0/243202199 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559f5ef1b7d0 0x559f5ef89690 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679ff1c740 1 --2- 172.21.15.5:0/243202199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559f5ef0d680 0x559f5ef22170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679ff1c740 1 --2- 172.21.15.5:0/243202199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f5ef02dc0 0x559f5ef21b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679ff1c740 1 --2- 172.21.15.5:0/243202199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5ef024a0 0x559f5ef257a0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679ff1c740 1 -- 172.21.15.5:0/243202199 >> 172.21.15.5:0/243202199 conn(0x559f5eefea30 msgr2=0x559f5eefd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f6776ffd640 0 client.18186 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679ff1c740 1 -- 172.21.15.5:0/243202199 shutdown_connections 2024-09-19T06:13:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679ff1c740 1 -- 172.21.15.5:0/243202199 wait complete. 2024-09-19T06:13:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679ff1c740 20 client.18186 trim_cache size 0 max 16384 2024-09-19T06:13:47.497 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:13:47.497 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:13:47.497 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:13:47.497 INFO:teuthology.orchestra.run.smithi005.stdout: "54964366-acdd-41db-bd11-3c20dbe78617": { 2024-09-19T06:13:47.497 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:13:47.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "54964366-acdd-41db-bd11-3c20dbe78617", 2024-09-19T06:13:47.497 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:13:47.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:13:47.498 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:13:47.498 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:13:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679db43640 1 -- 172.21.15.5:0/1332794981 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67840a33b0 msgr2=0x7f67840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679db43640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67840a33b0 0x7f67840a5870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6790009fd0 tx=0x7f6790009500 comp rx=0 tx=0).stop 2024-09-19T06:13:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679db43640 1 -- 172.21.15.5:0/1332794981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798102790 msgr2=0x7f67981988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.489+0000 7f679db43640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798102790 0x7f67981988c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6780005940 tx=0x7f67800040c0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.490+0000 7f679db43640 1 -- 172.21.15.5:0/1332794981 shutdown_connections 2024-09-19T06:13:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.490+0000 7f679db43640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67840a33b0 0x7f67840a5870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.490+0000 7f679db43640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6798106f10 0x7f679819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.490+0000 7f679db43640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798102790 0x7f67981988c0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.490+0000 7f679db43640 1 --2- 172.21.15.5:0/1332794981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798101de0 0x7f6798198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.490+0000 7f679db43640 1 -- 172.21.15.5:0/1332794981 >> 172.21.15.5:0/1332794981 conn(0x7f67980fd8c0 msgr2=0x7f67980fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.490+0000 7f679db43640 1 -- 172.21.15.5:0/1332794981 shutdown_connections 2024-09-19T06:13:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:13:47.490+0000 7f679db43640 1 -- 172.21.15.5:0/1332794981 wait complete. 2024-09-19T06:13:47.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:47 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:47.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:47 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:47 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:47.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:47 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:47 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:47.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:47 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:47.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:47 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:47 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:47.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:47 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:47 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:13:47.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:47 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:47 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:47 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:47.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:47 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:47 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:47.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:47 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:47 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:47.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:47 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:47.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:47 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:47 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:47.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:47 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:47 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:13:47.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:47 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:47 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:13:47.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 252617 2.37 MB/sec execute 433 sec latency 43.884 ms 2024-09-19T06:13:48.215 INFO:tasks.cephfs.filesystem:scrub status for tag:54964366-acdd-41db-bd11-3c20dbe78617 - {'path': '/', 'tag': '54964366-acdd-41db-bd11-3c20dbe78617', 'options': 'recursive,force'} 2024-09-19T06:13:48.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:48 smithi157 ceph-mon[35346]: pgmap v325: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-19T06:13:48.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:48 smithi005 ceph-mon[30955]: pgmap v325: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-19T06:13:48.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:48 smithi049 ceph-mon[34640]: pgmap v325: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-19T06:13:48.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 253138 2.37 MB/sec execute 434 sec latency 14.912 ms 2024-09-19T06:13:49.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 253706 2.37 MB/sec execute 435 sec latency 20.333 ms 2024-09-19T06:13:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.175+0000 7fc9bfef1640 1 -- 172.21.15.5:0/1732383586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b8107ad0 msgr2=0x7fc9b8107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.175+0000 7fc9bfef1640 1 --2- 172.21.15.5:0/1732383586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b8107ad0 0x7fc9b8107eb0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009930 tx=0x7fc9b402a420 comp rx=0 tx=0).stop 2024-09-19T06:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.175+0000 7fc9bfef1640 1 -- 172.21.15.5:0/1732383586 shutdown_connections 2024-09-19T06:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.175+0000 7fc9bfef1640 1 --2- 172.21.15.5:0/1732383586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9b8102460 0x7fc9b810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.175+0000 7fc9bfef1640 1 --2- 172.21.15.5:0/1732383586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8101ac0 0x7fc9b8101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.175+0000 7fc9bfef1640 1 --2- 172.21.15.5:0/1732383586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b8107ad0 0x7fc9b8107eb0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.175+0000 7fc9bfef1640 1 -- 172.21.15.5:0/1732383586 >> 172.21.15.5:0/1732383586 conn(0x7fc9b80fd7a0 msgr2=0x7fc9b80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.175+0000 7fc9bfef1640 1 -- 172.21.15.5:0/1732383586 shutdown_connections 2024-09-19T06:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.176+0000 7fc9bfef1640 1 -- 172.21.15.5:0/1732383586 wait complete. 2024-09-19T06:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.176+0000 7fc9bfef1640 1 Processor -- start 2024-09-19T06:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.176+0000 7fc9bfef1640 1 -- start start 2024-09-19T06:13:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.176+0000 7fc9bfef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9b8101ac0 0x7fc9b8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bfef1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b8102460 0x7fc9b8198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bfef1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8107ad0 0x7fc9b819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bfef1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8111dc0 con 0x7fc9b8102460 2024-09-19T06:13:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bfef1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8111c40 con 0x7fc9b8101ac0 2024-09-19T06:13:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bfef1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8111f40 con 0x7fc9b8107ad0 2024-09-19T06:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bd48c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b8102460 0x7fc9b8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bd48c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b8102460 0x7fc9b8198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50906/0 (socket says 172.21.15.5:50906) 2024-09-19T06:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bd48c640 1 -- 172.21.15.5:0/117689952 learned_addr learned my addr 172.21.15.5:0/117689952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bdc8d640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9b8101ac0 0x7fc9b8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9be48e640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8107ad0 0x7fc9b819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bd48c640 1 -- 172.21.15.5:0/117689952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8107ad0 msgr2=0x7fc9b819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bd48c640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8107ad0 0x7fc9b819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bd48c640 1 -- 172.21.15.5:0/117689952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9b8101ac0 msgr2=0x7fc9b8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bd48c640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9b8101ac0 0x7fc9b8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bd48c640 1 -- 172.21.15.5:0/117689952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b819d340 con 0x7fc9b8102460 2024-09-19T06:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bd48c640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b8102460 0x7fc9b8198890 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800e9f0 tx=0x7fc9a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9aeffd640 1 -- 172.21.15.5:0/117689952 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a800cd80 con 0x7fc9b8102460 2024-09-19T06:13:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9aeffd640 1 -- 172.21.15.5:0/117689952 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a80054d0 con 0x7fc9b8102460 2024-09-19T06:13:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9bfef1640 1 -- 172.21.15.5:0/117689952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b819d630 con 0x7fc9b8102460 2024-09-19T06:13:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.177+0000 7fc9aeffd640 1 -- 172.21.15.5:0/117689952 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8010490 con 0x7fc9b8102460 2024-09-19T06:13:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.178+0000 7fc9bfef1640 1 -- 172.21.15.5:0/117689952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b81a4ec0 con 0x7fc9b8102460 2024-09-19T06:13:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.180+0000 7fc9c2c8b740 1 Processor -- start 2024-09-19T06:13:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9c2c8b740 1 -- start start 2024-09-19T06:13:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9aeffd640 1 -- 172.21.15.5:0/117689952 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc9a8005020 con 0x7fc9b8102460 2024-09-19T06:13:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9c2c8b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1018af0 0x55cbe1018ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9c2c8b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1019410 0x55cbe1023770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9c2c8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe10260a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9c2c8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cbe0f7d2f0 con 0x55cbe1023cb0 2024-09-19T06:13:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9c2c8b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cbe0f25270 con 0x55cbe1018af0 2024-09-19T06:13:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9c2c8b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cbe0f792b0 con 0x55cbe1019410 2024-09-19T06:13:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9bd48c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe10260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9be48e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1018af0 0x55cbe1018ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9be48e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1018af0 0x55cbe1018ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37034/0 (socket says 172.21.15.5:37034) 2024-09-19T06:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9be48e640 1 -- 172.21.15.5:0/270570982 learned_addr learned my addr 172.21.15.5:0/270570982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9bdc8d640 1 --2- 172.21.15.5:0/270570982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1019410 0x55cbe1023770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9be48e640 1 -- 172.21.15.5:0/270570982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1019410 msgr2=0x55cbe1023770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9be48e640 1 --2- 172.21.15.5:0/270570982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1019410 0x55cbe1023770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9be48e640 1 -- 172.21.15.5:0/270570982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 msgr2=0x55cbe10260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9be48e640 1 --2- 172.21.15.5:0/270570982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe10260a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.181+0000 7fc9be48e640 1 -- 172.21.15.5:0/270570982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe0f24340 con 0x55cbe1018af0 2024-09-19T06:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.182+0000 7fc9be48e640 1 --2- 172.21.15.5:0/270570982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1018af0 0x55cbe1018ed0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009f40 tx=0x7fc9b0010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41679494e8fa3a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.182+0000 7fc9aeffd640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc9a40a33b0 0x7fc9a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.182+0000 7fc9a3fff640 1 -- 172.21.15.5:0/270570982 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0003b70 con 0x55cbe1018af0 2024-09-19T06:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.182+0000 7fc9a3fff640 1 -- 172.21.15.5:0/270570982 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9b0003d10 con 0x55cbe1018af0 2024-09-19T06:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.182+0000 7fc9a3fff640 1 -- 172.21.15.5:0/270570982 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0011900 con 0x55cbe1018af0 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.182+0000 7fc9bdc8d640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc9a40a33b0 0x7fc9a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.182+0000 7fc9aeffd640 1 -- 172.21.15.5:0/117689952 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7fc9a80d0220 con 0x7fc9b8102460 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.183+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/270570982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1018af0 msgr2=0x55cbe1018ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.183+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/270570982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1018af0 0x55cbe1018ed0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009f40 tx=0x7fc9b0010860 comp rx=0 tx=0).stop 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.183+0000 7fc9bdc8d640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc9a40a33b0 0x7fc9a40a5870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4000c00 tx=0x7fc9b4005a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.183+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/270570982 shutdown_connections 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.183+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/270570982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe10260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.183+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/270570982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1019410 0x55cbe1023770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.183+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/270570982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1018af0 0x55cbe1018ed0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.183+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/270570982 >> 172.21.15.5:0/270570982 conn(0x55cbe1015060 msgr2=0x55cbe1015440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.183+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/270570982 shutdown_connections 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.183+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/270570982 wait complete. 2024-09-19T06:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.183+0000 7fc9c2c8b740 1 -- start start 2024-09-19T06:13:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.183+0000 7fc9c2c8b740 1 Processor -- start 2024-09-19T06:13:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9c2c8b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1018af0 0x55cbe1020bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9c2c8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1019410 0x55cbe1021100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9c2c8b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1023cb0 0x55cbe1021640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9c2c8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cbe1026880 con 0x55cbe1019410 2024-09-19T06:13:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9c2c8b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cbe0f06290 con 0x55cbe1023cb0 2024-09-19T06:13:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9c2c8b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cbe1026a00 con 0x55cbe1018af0 2024-09-19T06:13:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9c2c8b740 10 client.0 _ll_register_callbacks cb 0x55cbe0f37140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9bd48c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1019410 0x55cbe1021100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9bdc8d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1018af0 0x55cbe1020bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9bd48c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1019410 0x55cbe1021100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50918/0 (socket says 172.21.15.5:50918) 2024-09-19T06:13:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9bdc8d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1018af0 0x55cbe1020bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45750/0 (socket says 172.21.15.5:45750) 2024-09-19T06:13:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9bd48c640 1 -- 172.21.15.5:0/1067524015 learned_addr learned my addr 172.21.15.5:0/1067524015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9be48e640 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1023cb0 0x55cbe1021640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9bd48c640 1 -- 172.21.15.5:0/1067524015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1018af0 msgr2=0x55cbe1020bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9bd48c640 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1018af0 0x55cbe1020bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9bd48c640 1 -- 172.21.15.5:0/1067524015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1023cb0 msgr2=0x55cbe1021640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9bd48c640 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1023cb0 0x55cbe1021640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9bd48c640 1 -- 172.21.15.5:0/1067524015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe0c90210 con 0x55cbe1019410 2024-09-19T06:13:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9be48e640 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1023cb0 0x55cbe1021640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:13:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9bd48c640 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1019410 0x55cbe1021100 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800e9c0 tx=0x7fc9a80d3960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3d2572f1d96d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9a17fa640 1 -- 172.21.15.5:0/1067524015 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a809c2b0 con 0x55cbe1019410 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.184+0000 7fc9a17fa640 1 -- 172.21.15.5:0/1067524015 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7fc9a8098b50 con 0x55cbe1019410 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1067524015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9b0003d10 con 0x55cbe1019410 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9a17fa640 1 -- 172.21.15.5:0/1067524015 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc9a809be40 con 0x55cbe1019410 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9c2c8b740 10 client.18198 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1067524015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55cbe0d5c620 con 0x55cbe1019410 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9a17fa640 1 -- 172.21.15.5:0/1067524015 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9b0003d10 con 0x55cbe1019410 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9a17fa640 1 -- 172.21.15.5:0/1067524015 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fc9a80a4cc0 con 0x55cbe1019410 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9c2c8b740 10 client.18198 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9c2c8b740 10 client.18198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9c2c8b740 20 client.18198 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cbe1033c30 0x55cbe1036040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9c2c8b740 4 client.18198 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1067524015 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbe0f8aa20 con 0x55cbe1033c30 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.185+0000 7fc9bdc8d640 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cbe1033c30 0x55cbe1036040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.186+0000 7fc9bdc8d640 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cbe1033c30 0x55cbe1036040 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cede78c8a420e824 server_cookie=5039f08c56d8c701 in_seq=0 out_seq=0 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.186+0000 7fc9a17fa640 10 client.18198 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.186+0000 7fc9a17fa640 1 -- 172.21.15.5:0/1067524015 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbe0f8aa20 con 0x55cbe1033c30 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.186+0000 7fc9a17fa640 10 client.18198 handle_command_reply: tid=0 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9c2c8b740 1 client.18198 shutdown 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1067524015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1019410 msgr2=0x55cbe1021100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1019410 0x55cbe1021100 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800e9c0 tx=0x7fc9a80d3960 comp rx=0 tx=0).stop 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1067524015 shutdown_connections 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cbe1033c30 0x55cbe1036040 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1023cb0 0x55cbe1021640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1019410 0x55cbe1021100 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/1067524015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1018af0 0x55cbe1020bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9a17fa640 0 client.18198 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1067524015 >> 172.21.15.5:0/1067524015 conn(0x55cbe1015060 msgr2=0x55cbe101ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1067524015 shutdown_connections 2024-09-19T06:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1067524015 wait complete. 2024-09-19T06:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.187+0000 7fc9c2c8b740 20 client.18198 trim_cache size 0 max 16384 2024-09-19T06:13:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9c2c8b740 1 Processor -- start 2024-09-19T06:13:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9c2c8b740 1 -- start start 2024-09-19T06:13:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9c2c8b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1018af0 0x55cbe101b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9c2c8b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1019410 0x55cbe1021470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9c2c8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe103b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9c2c8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81ad8d0 con 0x55cbe1023cb0 2024-09-19T06:13:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9c2c8b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81af4d0 con 0x55cbe1019410 2024-09-19T06:13:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9c2c8b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81adfa0 con 0x55cbe1018af0 2024-09-19T06:13:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9be48e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe103b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9be48e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe103b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50926/0 (socket says 172.21.15.5:50926) 2024-09-19T06:13:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9bd48c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1019410 0x55cbe1021470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9be48e640 1 -- 172.21.15.5:0/1608373482 learned_addr learned my addr 172.21.15.5:0/1608373482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9bdc8d640 1 --2- 172.21.15.5:0/1608373482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1018af0 0x55cbe101b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9bd48c640 1 -- 172.21.15.5:0/1608373482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1018af0 msgr2=0x55cbe101b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9bd48c640 1 --2- 172.21.15.5:0/1608373482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1018af0 0x55cbe101b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9bd48c640 1 -- 172.21.15.5:0/1608373482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 msgr2=0x55cbe103b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9bd48c640 1 --2- 172.21.15.5:0/1608373482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe103b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9bd48c640 1 -- 172.21.15.5:0/1608373482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe101f0f0 con 0x55cbe1019410 2024-09-19T06:13:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.207+0000 7fc9be48e640 1 --2- 172.21.15.5:0/1608373482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe103b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:13:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.208+0000 7fc9bd48c640 1 --2- 172.21.15.5:0/1608373482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1019410 0x55cbe1021470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9a809c470 tx=0x7fc9a80d05b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44d55d8b948cba07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.208+0000 7fc9a37fe640 1 -- 172.21.15.5:0/1608373482 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a809c020 con 0x55cbe1019410 2024-09-19T06:13:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.208+0000 7fc9a37fe640 1 -- 172.21.15.5:0/1608373482 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a8005310 con 0x55cbe1019410 2024-09-19T06:13:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.208+0000 7fc9a37fe640 1 -- 172.21.15.5:0/1608373482 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a809cbe0 con 0x55cbe1019410 2024-09-19T06:13:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.209+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1608373482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1019410 msgr2=0x55cbe1021470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.209+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/1608373482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1019410 0x55cbe1021470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9a809c470 tx=0x7fc9a80d05b0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.209+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1608373482 shutdown_connections 2024-09-19T06:13:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.209+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/1608373482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe103b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.209+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/1608373482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1019410 0x55cbe1021470 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.209+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/1608373482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1018af0 0x55cbe101b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.209+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1608373482 >> 172.21.15.5:0/1608373482 conn(0x55cbe1015060 msgr2=0x55cbe1022c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.209+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1608373482 shutdown_connections 2024-09-19T06:13:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.209+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/1608373482 wait complete. 2024-09-19T06:13:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.209+0000 7fc9c2c8b740 1 -- start start 2024-09-19T06:13:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.209+0000 7fc9c2c8b740 1 Processor -- start 2024-09-19T06:13:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9c2c8b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1018af0 0x55cbe1027b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9c2c8b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1019410 0x55cbe10280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9c2c8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe10285e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9c2c8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81aee00 con 0x55cbe1023cb0 2024-09-19T06:13:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9c2c8b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81ae6d0 con 0x55cbe1018af0 2024-09-19T06:13:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9c2c8b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81ab8d0 con 0x55cbe1019410 2024-09-19T06:13:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9c2c8b740 10 client.0 _ll_register_callbacks cb 0x7fc9a80d3a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:13:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9be48e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe10285e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9be48e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe10285e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50928/0 (socket says 172.21.15.5:50928) 2024-09-19T06:13:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9be48e640 1 -- 172.21.15.5:0/868568890 learned_addr learned my addr 172.21.15.5:0/868568890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:13:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9bdc8d640 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1018af0 0x55cbe1027b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9bd48c640 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1019410 0x55cbe10280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9be48e640 1 -- 172.21.15.5:0/868568890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1019410 msgr2=0x55cbe10280a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9be48e640 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1019410 0x55cbe10280a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9be48e640 1 -- 172.21.15.5:0/868568890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1018af0 msgr2=0x55cbe1027b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9be48e640 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1018af0 0x55cbe1027b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9be48e640 1 -- 172.21.15.5:0/868568890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe109dac0 con 0x55cbe1023cb0 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9be48e640 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe10285e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009fd0 tx=0x7fc9b0011730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8acdf7c3858cea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.210+0000 7fc9a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9a17fa640 1 -- 172.21.15.5:0/868568890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b002a060 con 0x55cbe1023cb0 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9a17fa640 1 -- 172.21.15.5:0/868568890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7fc9b0011d60 con 0x55cbe1023cb0 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/868568890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cbe0f84d70 con 0x55cbe1023cb0 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9a17fa640 1 -- 172.21.15.5:0/868568890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc9b0005040 con 0x55cbe1023cb0 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9c2c8b740 10 client.18204 fetch_fsmap learned FSMap version 38 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/868568890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55cbe109ed90 con 0x55cbe1023cb0 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9a17fa640 1 -- 172.21.15.5:0/868568890 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55cbe0f84d70 con 0x55cbe1023cb0 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9a17fa640 1 -- 172.21.15.5:0/868568890 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fc9b001b420 con 0x55cbe1023cb0 2024-09-19T06:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9c2c8b740 10 client.18204 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9c2c8b740 10 client.18204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9c2c8b740 20 client.18204 populate_metadata read hostname 'smithi005' 2024-09-19T06:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cbe109f250 0x55cbe10a1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9c2c8b740 4 client.18204 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.211+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/868568890 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cbe0f01900 con 0x55cbe109f250 2024-09-19T06:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.212+0000 7fc9bdc8d640 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cbe109f250 0x55cbe10a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.212+0000 7fc9bdc8d640 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cbe109f250 0x55cbe10a1670 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2506d18b315d6ad server_cookie=d138ba5b01c5d667 in_seq=0 out_seq=0 2024-09-19T06:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.212+0000 7fc9a17fa640 10 client.18204 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.239+0000 7fc9a17fa640 1 -- 172.21.15.5:0/868568890 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cbe0f01900 con 0x55cbe109f250 2024-09-19T06:13:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.239+0000 7fc9a17fa640 10 client.18204 handle_command_reply: tid=0 2024-09-19T06:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.239+0000 7fc9c2c8b740 1 client.18204 shutdown 2024-09-19T06:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.239+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/868568890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 msgr2=0x55cbe10285e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.239+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe10285e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009fd0 tx=0x7fc9b0011730 comp rx=0 tx=0).stop 2024-09-19T06:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.239+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/868568890 shutdown_connections 2024-09-19T06:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.239+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cbe109f250 0x55cbe10a1670 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.239+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe1023cb0 0x55cbe10285e0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.239+0000 7fc9a17fa640 0 client.18204 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.239+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe1019410 0x55cbe10280a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.239+0000 7fc9c2c8b740 1 --2- 172.21.15.5:0/868568890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe1018af0 0x55cbe1027b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.239+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/868568890 >> 172.21.15.5:0/868568890 conn(0x55cbe1015060 msgr2=0x55cbe10139a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/868568890 shutdown_connections 2024-09-19T06:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9c2c8b740 1 -- 172.21.15.5:0/868568890 wait complete. 2024-09-19T06:13:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9c2c8b740 20 client.18204 trim_cache size 0 max 16384 2024-09-19T06:13:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:13:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:13:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:13:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9bfef1640 1 -- 172.21.15.5:0/117689952 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc9a40a33b0 msgr2=0x7fc9a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9bfef1640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc9a40a33b0 0x7fc9a40a5870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4000c00 tx=0x7fc9b4005a40 comp rx=0 tx=0).stop 2024-09-19T06:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9bfef1640 1 -- 172.21.15.5:0/117689952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b8102460 msgr2=0x7fc9b8198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9bfef1640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b8102460 0x7fc9b8198890 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800e9f0 tx=0x7fc9a800edf0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9bfef1640 1 -- 172.21.15.5:0/117689952 shutdown_connections 2024-09-19T06:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9bfef1640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc9a40a33b0 0x7fc9a40a5870 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9bfef1640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8107ad0 0x7fc9b819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9bfef1640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b8102460 0x7fc9b8198890 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9bfef1640 1 --2- 172.21.15.5:0/117689952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9b8101ac0 0x7fc9b8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9bfef1640 1 -- 172.21.15.5:0/117689952 >> 172.21.15.5:0/117689952 conn(0x7fc9b80fd7a0 msgr2=0x7fc9b80fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:13:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9bfef1640 1 -- 172.21.15.5:0/117689952 shutdown_connections 2024-09-19T06:13:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:13:50.240+0000 7fc9bfef1640 1 -- 172.21.15.5:0/117689952 wait complete. 2024-09-19T06:13:50.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:50 smithi157 ceph-mon[35346]: pgmap v326: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1018 KiB/s wr, 104 op/s 2024-09-19T06:13:50.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:50 smithi005 ceph-mon[30955]: pgmap v326: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1018 KiB/s wr, 104 op/s 2024-09-19T06:13:50.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:50 smithi049 ceph-mon[34640]: pgmap v326: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1018 KiB/s wr, 104 op/s 2024-09-19T06:13:50.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 254264 2.37 MB/sec execute 436 sec latency 19.661 ms 2024-09-19T06:13:51.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 254876 2.38 MB/sec execute 437 sec latency 15.233 ms 2024-09-19T06:13:52.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:52 smithi157 ceph-mon[35346]: pgmap v327: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1018 KiB/s wr, 104 op/s 2024-09-19T06:13:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:52 smithi005 ceph-mon[30955]: pgmap v327: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1018 KiB/s wr, 104 op/s 2024-09-19T06:13:52.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:52 smithi049 ceph-mon[34640]: pgmap v327: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1018 KiB/s wr, 104 op/s 2024-09-19T06:13:52.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255411 2.38 MB/sec execute 438 sec latency 16.157 ms 2024-09-19T06:13:53.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255493 2.38 MB/sec execute 439 sec latency 863.596 ms 2024-09-19T06:13:54.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:54 smithi005 ceph-mon[30955]: pgmap v328: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-09-19T06:13:54.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:54 smithi049 ceph-mon[34640]: pgmap v328: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-09-19T06:13:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:54 smithi157 ceph-mon[35346]: pgmap v328: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-09-19T06:13:54.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255493 2.37 MB/sec execute 440 sec latency 1863.725 ms 2024-09-19T06:13:55.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255560 2.37 MB/sec execute 441 sec latency 2492.532 ms 2024-09-19T06:13:56.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:56 smithi005 ceph-mon[30955]: pgmap v329: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-09-19T06:13:56.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:56 smithi049 ceph-mon[34640]: pgmap v329: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-09-19T06:13:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:56 smithi157 ceph-mon[35346]: pgmap v329: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-09-19T06:13:56.841 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255560 2.36 MB/sec execute 442 sec latency 1124.856 ms 2024-09-19T06:13:57.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255560 2.36 MB/sec execute 443 sec latency 2124.999 ms 2024-09-19T06:13:58.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:58 smithi005 ceph-mon[30955]: pgmap v330: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-19T06:13:58.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:58 smithi049 ceph-mon[34640]: pgmap v330: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-19T06:13:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:58 smithi157 ceph-mon[35346]: pgmap v330: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-19T06:13:58.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255560 2.35 MB/sec execute 444 sec latency 3125.115 ms 2024-09-19T06:13:59.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:13:59 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:13:59.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:13:59 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:13:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:13:59 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:13:59.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255560 2.34 MB/sec execute 445 sec latency 4125.241 ms 2024-09-19T06:14:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:00 smithi005 ceph-mon[30955]: pgmap v331: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 865 KiB/s wr, 89 op/s 2024-09-19T06:14:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:00 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:14:00.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:00 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:14:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:00 smithi049 ceph-mon[34640]: pgmap v331: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 865 KiB/s wr, 89 op/s 2024-09-19T06:14:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:00 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:14:00.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:00 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:14:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:00 smithi157 ceph-mon[35346]: pgmap v331: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 865 KiB/s wr, 89 op/s 2024-09-19T06:14:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:00 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:14:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:00 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:14:00.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255675 2.34 MB/sec execute 446 sec latency 4745.209 ms 2024-09-19T06:14:01.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255972 2.34 MB/sec execute 447 sec latency 11.148 ms 2024-09-19T06:14:02.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:02 smithi005 ceph-mon[30955]: pgmap v332: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 865 KiB/s wr, 89 op/s 2024-09-19T06:14:02.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:02 smithi049 ceph-mon[34640]: pgmap v332: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 865 KiB/s wr, 89 op/s 2024-09-19T06:14:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:02 smithi157 ceph-mon[35346]: pgmap v332: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 865 KiB/s wr, 89 op/s 2024-09-19T06:14:02.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 256476 2.34 MB/sec execute 448 sec latency 15.886 ms 2024-09-19T06:14:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.341+0000 7fee5f2d0640 1 -- 172.21.15.5:0/3146930639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58102c80 msgr2=0x7fee58103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.341+0000 7fee5f2d0640 1 --2- 172.21.15.5:0/3146930639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58102c80 0x7fee58103060 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fee54009900 tx=0x7fee5402a480 comp rx=0 tx=0).stop 2024-09-19T06:14:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.342+0000 7fee5f2d0640 1 -- 172.21.15.5:0/3146930639 shutdown_connections 2024-09-19T06:14:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.342+0000 7fee5f2d0640 1 --2- 172.21.15.5:0/3146930639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee5810c0a0 0x7fee5810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.342+0000 7fee5f2d0640 1 --2- 172.21.15.5:0/3146930639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee581035a0 0x7fee5810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.342+0000 7fee5f2d0640 1 --2- 172.21.15.5:0/3146930639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58102c80 0x7fee58103060 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.342+0000 7fee5f2d0640 1 -- 172.21.15.5:0/3146930639 >> 172.21.15.5:0/3146930639 conn(0x7fee580779a0 msgr2=0x7fee580fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.342+0000 7fee5f2d0640 1 -- 172.21.15.5:0/3146930639 shutdown_connections 2024-09-19T06:14:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.342+0000 7fee5f2d0640 1 -- 172.21.15.5:0/3146930639 wait complete. 2024-09-19T06:14:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.343+0000 7fee5f2d0640 1 Processor -- start 2024-09-19T06:14:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.343+0000 7fee5f2d0640 1 -- start start 2024-09-19T06:14:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.343+0000 7fee5f2d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58102c80 0x7fee58198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.343+0000 7fee5f2d0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581035a0 0x7fee58198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.343+0000 7fee5f2d0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee5810c0a0 0x7fee5819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.343+0000 7fee5f2d0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee58111da0 con 0x7fee58102c80 2024-09-19T06:14:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.343+0000 7fee5f2d0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee58111c20 con 0x7fee581035a0 2024-09-19T06:14:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.343+0000 7fee5f2d0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee58111f20 con 0x7fee5810c0a0 2024-09-19T06:14:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.343+0000 7fee5d06c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58102c80 0x7fee58198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.343+0000 7fee5d06c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58102c80 0x7fee58198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37890/0 (socket says 172.21.15.5:37890) 2024-09-19T06:14:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.343+0000 7fee5d06c640 1 -- 172.21.15.5:0/252929639 learned_addr learned my addr 172.21.15.5:0/252929639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.344+0000 7fee5d86d640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee5810c0a0 0x7fee5819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.344+0000 7fee5c86b640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581035a0 0x7fee58198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.344+0000 7fee5d06c640 1 -- 172.21.15.5:0/252929639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee5810c0a0 msgr2=0x7fee5819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.344+0000 7fee5d06c640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee5810c0a0 0x7fee5819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.344+0000 7fee5d06c640 1 -- 172.21.15.5:0/252929639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581035a0 msgr2=0x7fee58198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.344+0000 7fee5d06c640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581035a0 0x7fee58198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.344+0000 7fee5d06c640 1 -- 172.21.15.5:0/252929639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5819d300 con 0x7fee58102c80 2024-09-19T06:14:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.344+0000 7fee5d06c640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58102c80 0x7fee58198310 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fee54031440 tx=0x7fee54003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.344+0000 7fee4e7fc640 1 -- 172.21.15.5:0/252929639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5403f070 con 0x7fee58102c80 2024-09-19T06:14:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.344+0000 7fee5f2d0640 1 -- 172.21.15.5:0/252929639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5819d590 con 0x7fee58102c80 2024-09-19T06:14:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.344+0000 7fee4e7fc640 1 -- 172.21.15.5:0/252929639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee54031b50 con 0x7fee58102c80 2024-09-19T06:14:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.344+0000 7fee4e7fc640 1 -- 172.21.15.5:0/252929639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54035030 con 0x7fee58102c80 2024-09-19T06:14:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.345+0000 7fee5f2d0640 1 -- 172.21.15.5:0/252929639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee581a4e80 con 0x7fee58102c80 2024-09-19T06:14:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.347+0000 7fee4e7fc640 1 -- 172.21.15.5:0/252929639 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fee54042070 con 0x7fee58102c80 2024-09-19T06:14:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee6206a740 1 Processor -- start 2024-09-19T06:14:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee6206a740 1 -- start start 2024-09-19T06:14:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee4e7fc640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fee440a3400 0x7fee440a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee6206a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c1049edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee6206a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee6206a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c104ae0c0 0x561c104b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee6206a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c104032f0 con 0x561c1049f2f0 2024-09-19T06:14:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee6206a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561c103ab270 con 0x561c1049e9d0 2024-09-19T06:14:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee6206a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561c103ff2b0 con 0x561c104ae0c0 2024-09-19T06:14:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee5d86d640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fee440a3400 0x7fee440a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee5d06c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c1049edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee5c86b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee5d06c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c1049edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43224/0 (socket says 172.21.15.5:43224) 2024-09-19T06:14:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee5c86b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37900/0 (socket says 172.21.15.5:37900) 2024-09-19T06:14:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee5d06c640 1 -- 172.21.15.5:0/1182037020 learned_addr learned my addr 172.21.15.5:0/1182037020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee5d86d640 1 --2- 172.21.15.5:0/1182037020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c104ae0c0 0x561c104b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.348+0000 7fee4e7fc640 1 -- 172.21.15.5:0/252929639 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7fee540f03d0 con 0x7fee58102c80 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.349+0000 7fee5c86b640 1 -- 172.21.15.5:0/1182037020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c104ae0c0 msgr2=0x561c104b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.349+0000 7fee5c86b640 1 --2- 172.21.15.5:0/1182037020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c104ae0c0 0x561c104b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.349+0000 7fee5c86b640 1 -- 172.21.15.5:0/1182037020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 msgr2=0x561c1049edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.349+0000 7fee5c86b640 1 --2- 172.21.15.5:0/1182037020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c1049edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.349+0000 7fee5c86b640 1 -- 172.21.15.5:0/1182037020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c103aa340 con 0x561c1049f2f0 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.349+0000 7fee5d06c640 1 --2- 172.21.15.5:0/1182037020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c1049edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.349+0000 7fee5d86d640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fee440a3400 0x7fee440a58c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fee50009fd0 tx=0x7fee50009d80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.349+0000 7fee5c86b640 1 --2- 172.21.15.5:0/1182037020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104adb80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fee48009c00 tx=0x7fee48011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa7a145cdfed5ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.349+0000 7fee437fe640 1 -- 172.21.15.5:0/1182037020 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48004230 con 0x561c1049f2f0 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.349+0000 7fee437fe640 1 -- 172.21.15.5:0/1182037020 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee480043d0 con 0x561c1049f2f0 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.349+0000 7fee437fe640 1 -- 172.21.15.5:0/1182037020 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48004c00 con 0x561c1049f2f0 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.350+0000 7fee6206a740 1 -- 172.21.15.5:0/1182037020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 msgr2=0x561c104adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.350+0000 7fee6206a740 1 --2- 172.21.15.5:0/1182037020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104adb80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fee48009c00 tx=0x7fee48011810 comp rx=0 tx=0).stop 2024-09-19T06:14:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.350+0000 7fee6206a740 1 -- 172.21.15.5:0/1182037020 shutdown_connections 2024-09-19T06:14:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.350+0000 7fee6206a740 1 --2- 172.21.15.5:0/1182037020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c104ae0c0 0x561c104b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.350+0000 7fee6206a740 1 --2- 172.21.15.5:0/1182037020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104adb80 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.350+0000 7fee6206a740 1 --2- 172.21.15.5:0/1182037020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c1049edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.350+0000 7fee6206a740 1 -- 172.21.15.5:0/1182037020 >> 172.21.15.5:0/1182037020 conn(0x561c1049b060 msgr2=0x561c1049b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.350+0000 7fee6206a740 1 -- 172.21.15.5:0/1182037020 shutdown_connections 2024-09-19T06:14:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.350+0000 7fee6206a740 1 -- 172.21.15.5:0/1182037020 wait complete. 2024-09-19T06:14:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.350+0000 7fee6206a740 1 -- start start 2024-09-19T06:14:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.350+0000 7fee6206a740 1 Processor -- start 2024-09-19T06:14:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee6206a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c1049e9d0 0x561c104ab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee6206a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104ab570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee6206a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c104ae0c0 0x561c104abab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee6206a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c104b0cf0 con 0x561c1049f2f0 2024-09-19T06:14:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee6206a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561c1038c290 con 0x561c104ae0c0 2024-09-19T06:14:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee6206a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561c104b0e70 con 0x561c1049e9d0 2024-09-19T06:14:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee6206a740 10 client.0 _ll_register_callbacks cb 0x561c103bd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee5c86b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee5d86d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c104ae0c0 0x561c104abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee5c86b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104ab570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37914/0 (socket says 172.21.15.5:37914) 2024-09-19T06:14:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee5c86b640 1 -- 172.21.15.5:0/2312283627 learned_addr learned my addr 172.21.15.5:0/2312283627 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee5d06c640 1 --2- 172.21.15.5:0/2312283627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c1049e9d0 0x561c104ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee5c86b640 1 -- 172.21.15.5:0/2312283627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c1049e9d0 msgr2=0x561c104ab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee5c86b640 1 --2- 172.21.15.5:0/2312283627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c1049e9d0 0x561c104ab030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee5c86b640 1 -- 172.21.15.5:0/2312283627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c104ae0c0 msgr2=0x561c104abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee5c86b640 1 --2- 172.21.15.5:0/2312283627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c104ae0c0 0x561c104abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee5c86b640 1 -- 172.21.15.5:0/2312283627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c10116210 con 0x561c1049f2f0 2024-09-19T06:14:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee5c86b640 1 --2- 172.21.15.5:0/2312283627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104ab570 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fee48004880 tx=0x7fee48019990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2389aaa93820779a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.351+0000 7fee40ff9640 1 -- 172.21.15.5:0/2312283627 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48026070 con 0x561c1049f2f0 2024-09-19T06:14:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee6206a740 1 -- 172.21.15.5:0/2312283627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee480043d0 con 0x561c1049f2f0 2024-09-19T06:14:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee40ff9640 1 -- 172.21.15.5:0/2312283627 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7fee48019d90 con 0x561c1049f2f0 2024-09-19T06:14:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee40ff9640 1 -- 172.21.15.5:0/2312283627 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fee48032050 con 0x561c1049f2f0 2024-09-19T06:14:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee6206a740 10 client.18216 fetch_fsmap learned FSMap version 38 2024-09-19T06:14:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee6206a740 1 -- 172.21.15.5:0/2312283627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561c101e2620 con 0x561c1049f2f0 2024-09-19T06:14:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee40ff9640 1 -- 172.21.15.5:0/2312283627 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee480043d0 con 0x561c1049f2f0 2024-09-19T06:14:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee40ff9640 1 -- 172.21.15.5:0/2312283627 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fee48018070 con 0x561c1049f2f0 2024-09-19T06:14:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee6206a740 10 client.18216 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:14:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee6206a740 10 client.18216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee6206a740 20 client.18216 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee6206a740 1 --2- 172.21.15.5:0/2312283627 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561c104bdfd0 0x561c104c03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee6206a740 4 client.18216 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee6206a740 1 -- 172.21.15.5:0/2312283627 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c10410a20 con 0x561c104bdfd0 2024-09-19T06:14:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.352+0000 7fee5d06c640 1 --2- 172.21.15.5:0/2312283627 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561c104bdfd0 0x561c104c03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.353+0000 7fee40ff9640 10 client.18216 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.353+0000 7fee5d06c640 1 --2- 172.21.15.5:0/2312283627 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561c104bdfd0 0x561c104c03c0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f05532c114ce7e server_cookie=a7e0ab697eab4263 in_seq=0 out_seq=0 2024-09-19T06:14:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee40ff9640 1 -- 172.21.15.5:0/2312283627 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c10410a20 con 0x561c104bdfd0 2024-09-19T06:14:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee40ff9640 10 client.18216 handle_command_reply: tid=0 2024-09-19T06:14:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee6206a740 1 client.18216 shutdown 2024-09-19T06:14:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee6206a740 1 -- 172.21.15.5:0/2312283627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 msgr2=0x561c104ab570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee6206a740 1 --2- 172.21.15.5:0/2312283627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104ab570 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fee48004880 tx=0x7fee48019990 comp rx=0 tx=0).stop 2024-09-19T06:14:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee6206a740 1 -- 172.21.15.5:0/2312283627 shutdown_connections 2024-09-19T06:14:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee6206a740 1 --2- 172.21.15.5:0/2312283627 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561c104bdfd0 0x561c104c03c0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee40ff9640 0 client.18216 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee6206a740 1 --2- 172.21.15.5:0/2312283627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c104ae0c0 0x561c104abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee6206a740 1 --2- 172.21.15.5:0/2312283627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104ab570 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee6206a740 1 --2- 172.21.15.5:0/2312283627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c1049e9d0 0x561c104ab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee6206a740 1 -- 172.21.15.5:0/2312283627 >> 172.21.15.5:0/2312283627 conn(0x561c1049b060 msgr2=0x561c104a0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee6206a740 1 -- 172.21.15.5:0/2312283627 shutdown_connections 2024-09-19T06:14:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee6206a740 1 -- 172.21.15.5:0/2312283627 wait complete. 2024-09-19T06:14:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.354+0000 7fee6206a740 20 client.18216 trim_cache size 0 max 16384 2024-09-19T06:14:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee6206a740 1 Processor -- start 2024-09-19T06:14:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee6206a740 1 -- start start 2024-09-19T06:14:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee6206a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c1049a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee6206a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee6206a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c104ae0c0 0x561c104c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee6206a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee581ab5c0 con 0x561c1049f2f0 2024-09-19T06:14:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee6206a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee581ad1c0 con 0x561c1049e9d0 2024-09-19T06:14:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee6206a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee581abc90 con 0x561c104ae0c0 2024-09-19T06:14:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee5c86b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee5c86b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37918/0 (socket says 172.21.15.5:37918) 2024-09-19T06:14:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee5c86b640 1 -- 172.21.15.5:0/3118152925 learned_addr learned my addr 172.21.15.5:0/3118152925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee5d06c640 1 --2- 172.21.15.5:0/3118152925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c1049a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.370+0000 7fee5d86d640 1 --2- 172.21.15.5:0/3118152925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c104ae0c0 0x561c104c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.371+0000 7fee5c86b640 1 -- 172.21.15.5:0/3118152925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c104ae0c0 msgr2=0x561c104c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.371+0000 7fee5c86b640 1 --2- 172.21.15.5:0/3118152925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c104ae0c0 0x561c104c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.371+0000 7fee5c86b640 1 -- 172.21.15.5:0/3118152925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 msgr2=0x561c1049a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.371+0000 7fee5c86b640 1 --2- 172.21.15.5:0/3118152925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c1049a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.371+0000 7fee5c86b640 1 -- 172.21.15.5:0/3118152925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c104ad720 con 0x561c1049f2f0 2024-09-19T06:14:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.371+0000 7fee5d06c640 1 --2- 172.21.15.5:0/3118152925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c1049a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.371+0000 7fee5c86b640 1 --2- 172.21.15.5:0/3118152925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104c5bc0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fee48009ad0 tx=0x7fee480096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c964602e9835b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.371+0000 7fee42ffd640 1 -- 172.21.15.5:0/3118152925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48026040 con 0x561c1049f2f0 2024-09-19T06:14:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.371+0000 7fee42ffd640 1 -- 172.21.15.5:0/3118152925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee48015070 con 0x561c1049f2f0 2024-09-19T06:14:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.371+0000 7fee42ffd640 1 -- 172.21.15.5:0/3118152925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4800f3c0 con 0x561c1049f2f0 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.372+0000 7fee6206a740 1 -- 172.21.15.5:0/3118152925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 msgr2=0x561c104c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.372+0000 7fee6206a740 1 --2- 172.21.15.5:0/3118152925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104c5bc0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fee48009ad0 tx=0x7fee480096c0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.372+0000 7fee6206a740 1 -- 172.21.15.5:0/3118152925 shutdown_connections 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.372+0000 7fee6206a740 1 --2- 172.21.15.5:0/3118152925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c104ae0c0 0x561c104c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.372+0000 7fee6206a740 1 --2- 172.21.15.5:0/3118152925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1049f2f0 0x561c104c5bc0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.372+0000 7fee6206a740 1 --2- 172.21.15.5:0/3118152925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c1049a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.372+0000 7fee6206a740 1 -- 172.21.15.5:0/3118152925 >> 172.21.15.5:0/3118152925 conn(0x561c1049b060 msgr2=0x561c104999a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.372+0000 7fee6206a740 1 -- 172.21.15.5:0/3118152925 shutdown_connections 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.372+0000 7fee6206a740 1 -- 172.21.15.5:0/3118152925 wait complete. 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.372+0000 7fee6206a740 1 -- start start 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.372+0000 7fee6206a740 1 Processor -- start 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee6206a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c104b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee6206a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c1049f2f0 0x561c104b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee6206a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c104ae0c0 0x561c104b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee6206a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee581acaf0 con 0x561c104ae0c0 2024-09-19T06:14:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee6206a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee581ac3c0 con 0x561c1049e9d0 2024-09-19T06:14:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee6206a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee581a95c0 con 0x561c1049f2f0 2024-09-19T06:14:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee6206a740 10 client.0 _ll_register_callbacks cb 0x7fee540b8320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee5d06c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c104b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee5c86b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c1049f2f0 0x561c104b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee5d86d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c104ae0c0 0x561c104b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee5d06c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c104b60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43254/0 (socket says 172.21.15.5:43254) 2024-09-19T06:14:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee5d06c640 1 -- 172.21.15.5:0/1480510293 learned_addr learned my addr 172.21.15.5:0/1480510293 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee5d06c640 1 -- 172.21.15.5:0/1480510293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c1049f2f0 msgr2=0x561c104b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee5d06c640 1 --2- 172.21.15.5:0/1480510293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c1049f2f0 0x561c104b6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee5d06c640 1 -- 172.21.15.5:0/1480510293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c104ae0c0 msgr2=0x561c104b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee5d06c640 1 --2- 172.21.15.5:0/1480510293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c104ae0c0 0x561c104b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee5d06c640 1 -- 172.21.15.5:0/1480510293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c104b35d0 con 0x561c1049e9d0 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee5c86b640 1 --2- 172.21.15.5:0/1480510293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c1049f2f0 0x561c104b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee5d06c640 1 --2- 172.21.15.5:0/1480510293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c104b60c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fee54005ab0 tx=0x7fee5402a980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8e9cdf458286bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.373+0000 7fee40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee40ff9640 1 -- 172.21.15.5:0/1480510293 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54049040 con 0x561c1049e9d0 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee40ff9640 1 -- 172.21.15.5:0/1480510293 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8550+0+0 (secure 0 0 0) 0x7fee540baa40 con 0x561c1049e9d0 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee6206a740 1 -- 172.21.15.5:0/1480510293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee48015070 con 0x561c1049e9d0 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee40ff9640 1 -- 172.21.15.5:0/1480510293 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fee540c5300 con 0x561c1049e9d0 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee6206a740 10 client.27716 fetch_fsmap learned FSMap version 38 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee6206a740 1 -- 172.21.15.5:0/1480510293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561c105291b0 con 0x561c1049e9d0 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee40ff9640 1 -- 172.21.15.5:0/1480510293 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee48015070 con 0x561c1049e9d0 2024-09-19T06:14:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee40ff9640 1 -- 172.21.15.5:0/1480510293 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fee540edd80 con 0x561c1049e9d0 2024-09-19T06:14:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee6206a740 10 client.27716 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:14:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee6206a740 10 client.27716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee6206a740 20 client.27716 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee6206a740 1 --2- 172.21.15.5:0/1480510293 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561c105298e0 0x561c1052bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee6206a740 4 client.27716 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:14:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.374+0000 7fee6206a740 1 -- 172.21.15.5:0/1480510293 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c1040ad70 con 0x561c105298e0 2024-09-19T06:14:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.375+0000 7fee5c86b640 1 --2- 172.21.15.5:0/1480510293 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561c105298e0 0x561c1052bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.375+0000 7fee5c86b640 1 --2- 172.21.15.5:0/1480510293 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561c105298e0 0x561c1052bd00 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d7a9e99249c7fe4 server_cookie=dabec8455ba4e575 in_seq=0 out_seq=0 2024-09-19T06:14:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.375+0000 7fee40ff9640 10 client.27716 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.399+0000 7fee40ff9640 1 -- 172.21.15.5:0/1480510293 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fee4802f040 con 0x561c105298e0 2024-09-19T06:14:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.399+0000 7fee40ff9640 10 client.27716 handle_command_reply: tid=0 2024-09-19T06:14:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.399+0000 7fee6206a740 1 client.27716 shutdown 2024-09-19T06:14:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.399+0000 7fee6206a740 1 -- 172.21.15.5:0/1480510293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 msgr2=0x561c104b60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.399+0000 7fee6206a740 1 --2- 172.21.15.5:0/1480510293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c104b60c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fee54005ab0 tx=0x7fee5402a980 comp rx=0 tx=0).stop 2024-09-19T06:14:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.399+0000 7fee6206a740 1 -- 172.21.15.5:0/1480510293 shutdown_connections 2024-09-19T06:14:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.399+0000 7fee6206a740 1 --2- 172.21.15.5:0/1480510293 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561c105298e0 0x561c1052bd00 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.399+0000 7fee6206a740 1 --2- 172.21.15.5:0/1480510293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c104ae0c0 0x561c104b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.399+0000 7fee6206a740 1 --2- 172.21.15.5:0/1480510293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c1049f2f0 0x561c104b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.399+0000 7fee40ff9640 0 client.27716 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.399+0000 7fee6206a740 1 --2- 172.21.15.5:0/1480510293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1049e9d0 0x561c104b60c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee6206a740 1 -- 172.21.15.5:0/1480510293 >> 172.21.15.5:0/1480510293 conn(0x561c1049b060 msgr2=0x561c1049a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee6206a740 1 -- 172.21.15.5:0/1480510293 shutdown_connections 2024-09-19T06:14:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee6206a740 1 -- 172.21.15.5:0/1480510293 wait complete. 2024-09-19T06:14:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee6206a740 20 client.27716 trim_cache size 0 max 16384 2024-09-19T06:14:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:14:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:14:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:14:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:14:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee5f2d0640 1 -- 172.21.15.5:0/252929639 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fee440a3400 msgr2=0x7fee440a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee5f2d0640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fee440a3400 0x7fee440a58c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fee50009fd0 tx=0x7fee50009d80 comp rx=0 tx=0).stop 2024-09-19T06:14:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee5f2d0640 1 -- 172.21.15.5:0/252929639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58102c80 msgr2=0x7fee58198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee5f2d0640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58102c80 0x7fee58198310 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fee54031440 tx=0x7fee54003b90 comp rx=0 tx=0).stop 2024-09-19T06:14:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee5f2d0640 1 -- 172.21.15.5:0/252929639 shutdown_connections 2024-09-19T06:14:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee5f2d0640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fee440a3400 0x7fee440a58c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee5f2d0640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee5810c0a0 0x7fee5819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee5f2d0640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581035a0 0x7fee58198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee5f2d0640 1 --2- 172.21.15.5:0/252929639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58102c80 0x7fee58198310 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.400+0000 7fee5f2d0640 1 -- 172.21.15.5:0/252929639 >> 172.21.15.5:0/252929639 conn(0x7fee580779a0 msgr2=0x7fee580fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.401+0000 7fee5f2d0640 1 -- 172.21.15.5:0/252929639 shutdown_connections 2024-09-19T06:14:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:03.401+0000 7fee5f2d0640 1 -- 172.21.15.5:0/252929639 wait complete. 2024-09-19T06:14:03.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 256982 2.34 MB/sec execute 449 sec latency 28.919 ms 2024-09-19T06:14:04.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:04 smithi005 ceph-mon[30955]: pgmap v333: 97 pgs: 97 active+clean; 140 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-09-19T06:14:04.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:04 smithi049 ceph-mon[34640]: pgmap v333: 97 pgs: 97 active+clean; 140 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-09-19T06:14:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:04 smithi157 ceph-mon[35346]: pgmap v333: 97 pgs: 97 active+clean; 140 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-09-19T06:14:04.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 257586 2.33 MB/sec execute 450 sec latency 20.742 ms 2024-09-19T06:14:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:05 smithi005 ceph-mon[30955]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T06:14:05.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:05 smithi049 ceph-mon[34640]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T06:14:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:05 smithi157 ceph-mon[35346]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T06:14:05.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 258174 2.34 MB/sec execute 451 sec latency 15.229 ms 2024-09-19T06:14:06.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:06 smithi005 ceph-mon[30955]: pgmap v335: 97 pgs: 97 active+clean; 140 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 539 KiB/s wr, 77 op/s 2024-09-19T06:14:06.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:06 smithi049 ceph-mon[34640]: pgmap v335: 97 pgs: 97 active+clean; 140 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 539 KiB/s wr, 77 op/s 2024-09-19T06:14:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:06 smithi157 ceph-mon[35346]: pgmap v335: 97 pgs: 97 active+clean; 140 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 539 KiB/s wr, 77 op/s 2024-09-19T06:14:06.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 258752 2.34 MB/sec execute 452 sec latency 14.899 ms 2024-09-19T06:14:07.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259038 2.34 MB/sec execute 453 sec latency 516.019 ms 2024-09-19T06:14:08.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:08 smithi005 ceph-mon[30955]: pgmap v336: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-09-19T06:14:08.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:08 smithi005 ceph-mon[30955]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T06:14:08.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:08 smithi049 ceph-mon[34640]: pgmap v336: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-09-19T06:14:08.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:08 smithi049 ceph-mon[34640]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T06:14:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:08 smithi157 ceph-mon[35346]: pgmap v336: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-09-19T06:14:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:08 smithi157 ceph-mon[35346]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T06:14:08.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259038 2.34 MB/sec execute 454 sec latency 1516.127 ms 2024-09-19T06:14:09.842 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259038 2.33 MB/sec execute 455 sec latency 2516.181 ms 2024-09-19T06:14:10.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:10 smithi049 ceph-mon[34640]: pgmap v338: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-09-19T06:14:10.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:10 smithi005 ceph-mon[30955]: pgmap v338: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-09-19T06:14:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:10 smithi157 ceph-mon[35346]: pgmap v338: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-09-19T06:14:10.843 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259105 2.33 MB/sec execute 456 sec latency 3141.022 ms 2024-09-19T06:14:11.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259105 2.32 MB/sec execute 457 sec latency 1130.995 ms 2024-09-19T06:14:12.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:12 smithi049 ceph-mon[34640]: pgmap v339: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-19T06:14:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:12 smithi157 ceph-mon[35346]: pgmap v339: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-19T06:14:12.843 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259105 2.32 MB/sec execute 458 sec latency 2131.138 ms 2024-09-19T06:14:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:12 smithi005 ceph-mon[30955]: pgmap v339: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-19T06:14:13.843 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259105 2.31 MB/sec execute 459 sec latency 3131.261 ms 2024-09-19T06:14:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:14 smithi049 ceph-mon[34640]: pgmap v340: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-09-19T06:14:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:14 smithi157 ceph-mon[35346]: pgmap v340: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-09-19T06:14:14.843 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259105 2.31 MB/sec execute 460 sec latency 4131.362 ms 2024-09-19T06:14:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:14 smithi005 ceph-mon[30955]: pgmap v340: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-09-19T06:14:15.843 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259221 2.31 MB/sec execute 461 sec latency 4755.220 ms 2024-09-19T06:14:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.496+0000 7f2f02e9b640 1 -- 172.21.15.5:0/4220120116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2efc1009a0 msgr2=0x7f2efc100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.496+0000 7f2f02e9b640 1 --2- 172.21.15.5:0/4220120116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2efc1009a0 0x7f2efc100e00 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009930 tx=0x7f2ee802a420 comp rx=0 tx=0).stop 2024-09-19T06:14:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.496+0000 7f2f02e9b640 1 -- 172.21.15.5:0/4220120116 shutdown_connections 2024-09-19T06:14:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.496+0000 7f2f02e9b640 1 --2- 172.21.15.5:0/4220120116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2efc101340 0x7f2efc10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.496+0000 7f2f02e9b640 1 --2- 172.21.15.5:0/4220120116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2efc1009a0 0x7f2efc100e00 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.496+0000 7f2f02e9b640 1 --2- 172.21.15.5:0/4220120116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc1069b0 0x7f2efc106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.496+0000 7f2f02e9b640 1 -- 172.21.15.5:0/4220120116 >> 172.21.15.5:0/4220120116 conn(0x7f2efc0779a0 msgr2=0x7f2efc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.496+0000 7f2f02e9b640 1 -- 172.21.15.5:0/4220120116 shutdown_connections 2024-09-19T06:14:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.496+0000 7f2f02e9b640 1 -- 172.21.15.5:0/4220120116 wait complete. 2024-09-19T06:14:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2f02e9b640 1 Processor -- start 2024-09-19T06:14:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2f02e9b640 1 -- start start 2024-09-19T06:14:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2f02e9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2efc1009a0 0x7f2efc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2f02e9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc101340 0x7f2efc198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2f02e9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2efc1069b0 0x7f2efc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2f02e9b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc110c70 con 0x7f2efc1009a0 2024-09-19T06:14:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2f02e9b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc110af0 con 0x7f2efc101340 2024-09-19T06:14:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2f02e9b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc110df0 con 0x7f2efc1069b0 2024-09-19T06:14:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2f00c37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2efc1009a0 0x7f2efc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2efbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc101340 0x7f2efc198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2f00c37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2efc1009a0 0x7f2efc198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38412/0 (socket says 172.21.15.5:38412) 2024-09-19T06:14:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2efbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc101340 0x7f2efc198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58318/0 (socket says 172.21.15.5:58318) 2024-09-19T06:14:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2f00c37640 1 -- 172.21.15.5:0/3474287455 learned_addr learned my addr 172.21.15.5:0/3474287455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.497+0000 7f2f01438640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2efc1069b0 0x7f2efc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.498+0000 7f2f00c37640 1 -- 172.21.15.5:0/3474287455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2efc1069b0 msgr2=0x7f2efc19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.498+0000 7f2f00c37640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2efc1069b0 0x7f2efc19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.498+0000 7f2f00c37640 1 -- 172.21.15.5:0/3474287455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc101340 msgr2=0x7f2efc198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.498+0000 7f2f00c37640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc101340 0x7f2efc198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.498+0000 7f2f00c37640 1 -- 172.21.15.5:0/3474287455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2efc19d340 con 0x7f2efc1009a0 2024-09-19T06:14:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.498+0000 7f2f00c37640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2efc1009a0 0x7f2efc198350 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000e9f0 tx=0x7f2ef000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.498+0000 7f2ef9ffb640 1 -- 172.21.15.5:0/3474287455 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000cd80 con 0x7f2efc1009a0 2024-09-19T06:14:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.498+0000 7f2ef9ffb640 1 -- 172.21.15.5:0/3474287455 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef00108e0 con 0x7f2efc1009a0 2024-09-19T06:14:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.498+0000 7f2ef9ffb640 1 -- 172.21.15.5:0/3474287455 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0018920 con 0x7f2efc1009a0 2024-09-19T06:14:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.498+0000 7f2f02e9b640 1 -- 172.21.15.5:0/3474287455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2efc19d630 con 0x7f2efc1009a0 2024-09-19T06:14:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.498+0000 7f2f02e9b640 1 -- 172.21.15.5:0/3474287455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2efc1a4ec0 con 0x7f2efc1009a0 2024-09-19T06:14:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.501+0000 7f2ef9ffb640 1 -- 172.21.15.5:0/3474287455 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2ef0010a90 con 0x7f2efc1009a0 2024-09-19T06:14:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.501+0000 7f2f05c35740 1 Processor -- start 2024-09-19T06:14:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.501+0000 7f2f05c35740 1 -- start start 2024-09-19T06:14:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2f05c35740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df0af0 0x560450df0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2f05c35740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450df1410 0x560450dfb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2f05c35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450dfbcb0 0x560450dfe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2f05c35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560450d552f0 con 0x560450dfbcb0 2024-09-19T06:14:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2f05c35740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560450cfd270 con 0x560450df0af0 2024-09-19T06:14:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2f05c35740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560450d512b0 con 0x560450df1410 2024-09-19T06:14:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2efbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450dfbcb0 0x560450dfe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2f01438640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df0af0 0x560450df0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2efbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450dfbcb0 0x560450dfe0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38426/0 (socket says 172.21.15.5:38426) 2024-09-19T06:14:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2f01438640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df0af0 0x560450df0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58334/0 (socket says 172.21.15.5:58334) 2024-09-19T06:14:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2efbfff640 1 -- 172.21.15.5:0/1646746248 learned_addr learned my addr 172.21.15.5:0/1646746248 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2ef9ffb640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2eec0a33b0 0x7f2eec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2f00c37640 1 --2- 172.21.15.5:0/1646746248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450df1410 0x560450dfb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2f01438640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2eec0a33b0 0x7f2eec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2ef9ffb640 1 -- 172.21.15.5:0/3474287455 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f2ef00cffd0 con 0x7f2efc1009a0 2024-09-19T06:14:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2efbfff640 1 -- 172.21.15.5:0/1646746248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450df1410 msgr2=0x560450dfb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2efbfff640 1 --2- 172.21.15.5:0/1646746248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450df1410 0x560450dfb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2efbfff640 1 -- 172.21.15.5:0/1646746248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df0af0 msgr2=0x560450df0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2efbfff640 1 --2- 172.21.15.5:0/1646746248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df0af0 0x560450df0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2efbfff640 1 -- 172.21.15.5:0/1646746248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560450cfc340 con 0x560450dfbcb0 2024-09-19T06:14:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.502+0000 7f2efbfff640 1 --2- 172.21.15.5:0/1646746248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450dfbcb0 0x560450dfe0a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f2ee802a940 tx=0x7f2ee8005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d321c510cf9e12f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.503+0000 7f2f01438640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2eec0a33b0 0x7f2eec0a5870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4005c40 tx=0x7f2ef4004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.503+0000 7f2ee6ffd640 1 -- 172.21.15.5:0/1646746248 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8035040 con 0x560450dfbcb0 2024-09-19T06:14:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.503+0000 7f2ee6ffd640 1 -- 172.21.15.5:0/1646746248 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ee8004030 con 0x560450dfbcb0 2024-09-19T06:14:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.503+0000 7f2ee6ffd640 1 -- 172.21.15.5:0/1646746248 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8005530 con 0x560450dfbcb0 2024-09-19T06:14:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.503+0000 7f2f05c35740 1 -- 172.21.15.5:0/1646746248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450dfbcb0 msgr2=0x560450dfe0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.503+0000 7f2f05c35740 1 --2- 172.21.15.5:0/1646746248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450dfbcb0 0x560450dfe0a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f2ee802a940 tx=0x7f2ee8005be0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.503+0000 7f2f05c35740 1 -- 172.21.15.5:0/1646746248 shutdown_connections 2024-09-19T06:14:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.503+0000 7f2f05c35740 1 --2- 172.21.15.5:0/1646746248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450dfbcb0 0x560450dfe0a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.504+0000 7f2f05c35740 1 --2- 172.21.15.5:0/1646746248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450df1410 0x560450dfb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.504+0000 7f2f05c35740 1 --2- 172.21.15.5:0/1646746248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df0af0 0x560450df0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.504+0000 7f2f05c35740 1 -- 172.21.15.5:0/1646746248 >> 172.21.15.5:0/1646746248 conn(0x560450ded060 msgr2=0x560450ded440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.504+0000 7f2f05c35740 1 -- 172.21.15.5:0/1646746248 shutdown_connections 2024-09-19T06:14:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.504+0000 7f2f05c35740 1 -- 172.21.15.5:0/1646746248 wait complete. 2024-09-19T06:14:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.504+0000 7f2f05c35740 1 -- start start 2024-09-19T06:14:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.504+0000 7f2f05c35740 1 Processor -- start 2024-09-19T06:14:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.504+0000 7f2f05c35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 0x560450df8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.504+0000 7f2f05c35740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df1410 0x560450df9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.504+0000 7f2f05c35740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450df95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.504+0000 7f2f05c35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560450dfe880 con 0x560450df0af0 2024-09-19T06:14:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f05c35740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560450cde290 con 0x560450df1410 2024-09-19T06:14:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f05c35740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560450dfea00 con 0x560450dfbcb0 2024-09-19T06:14:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f00c37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 0x560450df8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f05c35740 10 client.0 _ll_register_callbacks cb 0x560450d0f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f00c37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 0x560450df8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38440/0 (socket says 172.21.15.5:38440) 2024-09-19T06:14:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f00c37640 1 -- 172.21.15.5:0/3158160124 learned_addr learned my addr 172.21.15.5:0/3158160124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2efbfff640 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df1410 0x560450df9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f01438640 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450df95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f00c37640 1 -- 172.21.15.5:0/3158160124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 msgr2=0x560450df95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f00c37640 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450df95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f00c37640 1 -- 172.21.15.5:0/3158160124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df1410 msgr2=0x560450df9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f00c37640 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df1410 0x560450df9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f00c37640 1 -- 172.21.15.5:0/3158160124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560450a68210 con 0x560450df0af0 2024-09-19T06:14:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f01438640 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450df95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:14:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2efbfff640 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df1410 0x560450df9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f00c37640 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 0x560450df8b40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00cdd00 tx=0x7f2ef0067290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3456f043a209e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2edbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2edbfff640 1 -- 172.21.15.5:0/3158160124 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00d22d0 con 0x560450df0af0 2024-09-19T06:14:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2edbfff640 1 -- 172.21.15.5:0/3158160124 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f2ef00a4a70 con 0x560450df0af0 2024-09-19T06:14:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f05c35740 1 -- 172.21.15.5:0/3158160124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ee8004030 con 0x560450df0af0 2024-09-19T06:14:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2edbfff640 1 -- 172.21.15.5:0/3158160124 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2ef0010de0 con 0x560450df0af0 2024-09-19T06:14:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f05c35740 10 client.18234 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.505+0000 7f2f05c35740 1 -- 172.21.15.5:0/3158160124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560450b34620 con 0x560450df0af0 2024-09-19T06:14:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.506+0000 7f2edbfff640 1 -- 172.21.15.5:0/3158160124 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ee8004030 con 0x560450df0af0 2024-09-19T06:14:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.506+0000 7f2edbfff640 1 -- 172.21.15.5:0/3158160124 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f2ef00bc140 con 0x560450df0af0 2024-09-19T06:14:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.506+0000 7f2f05c35740 10 client.18234 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.506+0000 7f2f05c35740 10 client.18234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.506+0000 7f2f05c35740 20 client.18234 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.506+0000 7f2f05c35740 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560450e0be80 0x560450e0e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.506+0000 7f2f05c35740 4 client.18234 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.506+0000 7f2f05c35740 1 -- 172.21.15.5:0/3158160124 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560450d62a20 con 0x560450e0be80 2024-09-19T06:14:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.506+0000 7f2efbfff640 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560450e0be80 0x560450e0e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.507+0000 7f2efbfff640 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560450e0be80 0x560450e0e320 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8f566bc6a84e27d server_cookie=9cd44f807a10290e in_seq=0 out_seq=0 2024-09-19T06:14:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.507+0000 7f2edbfff640 10 client.18234 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.507+0000 7f2edbfff640 1 -- 172.21.15.5:0/3158160124 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560450d62a20 con 0x560450e0be80 2024-09-19T06:14:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.507+0000 7f2edbfff640 10 client.18234 handle_command_reply: tid=0 2024-09-19T06:14:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.507+0000 7f2f05c35740 1 client.18234 shutdown 2024-09-19T06:14:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.507+0000 7f2f05c35740 1 -- 172.21.15.5:0/3158160124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 msgr2=0x560450df8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.507+0000 7f2f05c35740 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 0x560450df8b40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00cdd00 tx=0x7f2ef0067290 comp rx=0 tx=0).stop 2024-09-19T06:14:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.508+0000 7f2f05c35740 1 -- 172.21.15.5:0/3158160124 shutdown_connections 2024-09-19T06:14:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.508+0000 7f2f05c35740 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560450e0be80 0x560450e0e320 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.508+0000 7f2f05c35740 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450df95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.508+0000 7f2f05c35740 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df1410 0x560450df9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.508+0000 7f2edbfff640 0 client.18234 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.508+0000 7f2f05c35740 1 --2- 172.21.15.5:0/3158160124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 0x560450df8b40 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.508+0000 7f2f05c35740 1 -- 172.21.15.5:0/3158160124 >> 172.21.15.5:0/3158160124 conn(0x560450ded060 msgr2=0x560450df2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.508+0000 7f2f05c35740 1 -- 172.21.15.5:0/3158160124 shutdown_connections 2024-09-19T06:14:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.508+0000 7f2f05c35740 1 -- 172.21.15.5:0/3158160124 wait complete. 2024-09-19T06:14:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.508+0000 7f2f05c35740 20 client.18234 trim_cache size 0 max 16384 2024-09-19T06:14:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.524+0000 7f2f05c35740 1 Processor -- start 2024-09-19T06:14:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.524+0000 7f2f05c35740 1 -- start start 2024-09-19T06:14:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.524+0000 7f2f05c35740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df0af0 0x560450df93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.524+0000 7f2f05c35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df1410 0x560450dfaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2f05c35740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450e13bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2f05c35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1ab680 con 0x560450df1410 2024-09-19T06:14:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2f05c35740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1ad280 con 0x560450df0af0 2024-09-19T06:14:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2f05c35740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1abd50 con 0x560450dfbcb0 2024-09-19T06:14:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2efbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df1410 0x560450dfaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2f01438640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450e13bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2f00c37640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df0af0 0x560450df93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2efbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df1410 0x560450dfaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38456/0 (socket says 172.21.15.5:38456) 2024-09-19T06:14:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2efbfff640 1 -- 172.21.15.5:0/1512786820 learned_addr learned my addr 172.21.15.5:0/1512786820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2efbfff640 1 -- 172.21.15.5:0/1512786820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 msgr2=0x560450e13bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2efbfff640 1 --2- 172.21.15.5:0/1512786820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450e13bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2efbfff640 1 -- 172.21.15.5:0/1512786820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df0af0 msgr2=0x560450df93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2efbfff640 1 --2- 172.21.15.5:0/1512786820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df0af0 0x560450df93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2efbfff640 1 -- 172.21.15.5:0/1512786820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560450dece90 con 0x560450df1410 2024-09-19T06:14:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2efbfff640 1 --2- 172.21.15.5:0/1512786820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df1410 0x560450dfaab0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80051c0 tx=0x7f2ee80051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b9ee24a29de826 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2ee67fc640 1 -- 172.21.15.5:0/1512786820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8035040 con 0x560450df1410 2024-09-19T06:14:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2ee67fc640 1 -- 172.21.15.5:0/1512786820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ee804d040 con 0x560450df1410 2024-09-19T06:14:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.525+0000 7f2ee67fc640 1 -- 172.21.15.5:0/1512786820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee803fad0 con 0x560450df1410 2024-09-19T06:14:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.526+0000 7f2f05c35740 1 -- 172.21.15.5:0/1512786820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df1410 msgr2=0x560450dfaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.526+0000 7f2f05c35740 1 --2- 172.21.15.5:0/1512786820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df1410 0x560450dfaab0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80051c0 tx=0x7f2ee80051f0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.526+0000 7f2f05c35740 1 -- 172.21.15.5:0/1512786820 shutdown_connections 2024-09-19T06:14:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.526+0000 7f2f05c35740 1 --2- 172.21.15.5:0/1512786820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450e13bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.526+0000 7f2f05c35740 1 --2- 172.21.15.5:0/1512786820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df1410 0x560450dfaab0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.526+0000 7f2f05c35740 1 --2- 172.21.15.5:0/1512786820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df0af0 0x560450df93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.526+0000 7f2f05c35740 1 -- 172.21.15.5:0/1512786820 >> 172.21.15.5:0/1512786820 conn(0x560450ded060 msgr2=0x560450dfa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.526+0000 7f2f05c35740 1 -- 172.21.15.5:0/1512786820 shutdown_connections 2024-09-19T06:14:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.526+0000 7f2f05c35740 1 -- 172.21.15.5:0/1512786820 wait complete. 2024-09-19T06:14:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f05c35740 1 -- start start 2024-09-19T06:14:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f05c35740 1 Processor -- start 2024-09-19T06:14:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f05c35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 0x560450e03cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f05c35740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df1410 0x560450e04230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f05c35740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450df4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f05c35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1acbb0 con 0x560450df0af0 2024-09-19T06:14:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f05c35740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1ac480 con 0x560450df1410 2024-09-19T06:14:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f05c35740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1a96a0 con 0x560450dfbcb0 2024-09-19T06:14:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f05c35740 10 client.0 _ll_register_callbacks cb 0x7f2ef000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f00c37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 0x560450e03cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f00c37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 0x560450e03cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38460/0 (socket says 172.21.15.5:38460) 2024-09-19T06:14:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f00c37640 1 -- 172.21.15.5:0/3780587253 learned_addr learned my addr 172.21.15.5:0/3780587253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2efbfff640 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df1410 0x560450e04230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.527+0000 7f2f01438640 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450df4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2f00c37640 1 -- 172.21.15.5:0/3780587253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 msgr2=0x560450df4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2f00c37640 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450df4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2f00c37640 1 -- 172.21.15.5:0/3780587253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df1410 msgr2=0x560450e04230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2f00c37640 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df1410 0x560450e04230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2f00c37640 1 -- 172.21.15.5:0/3780587253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560450df5450 con 0x560450df0af0 2024-09-19T06:14:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2f00c37640 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 0x560450e03cf0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00097b0 tx=0x7f2ef009bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffda0c02093098cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2edbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2edbfff640 1 -- 172.21.15.5:0/3780587253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00985e0 con 0x560450df0af0 2024-09-19T06:14:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2edbfff640 1 -- 172.21.15.5:0/3780587253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f2ef0010430 con 0x560450df0af0 2024-09-19T06:14:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2f05c35740 1 -- 172.21.15.5:0/3780587253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ee804d040 con 0x560450df0af0 2024-09-19T06:14:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2edbfff640 1 -- 172.21.15.5:0/3780587253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2ef009c340 con 0x560450df0af0 2024-09-19T06:14:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2f05c35740 10 client.18240 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2f05c35740 1 -- 172.21.15.5:0/3780587253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560450e76330 con 0x560450df0af0 2024-09-19T06:14:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.528+0000 7f2edbfff640 1 -- 172.21.15.5:0/3780587253 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ee804d040 con 0x560450df0af0 2024-09-19T06:14:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.529+0000 7f2edbfff640 1 -- 172.21.15.5:0/3780587253 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f2ef0098090 con 0x560450df0af0 2024-09-19T06:14:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.529+0000 7f2f05c35740 10 client.18240 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.529+0000 7f2f05c35740 10 client.18240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.529+0000 7f2f05c35740 20 client.18240 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.529+0000 7f2f05c35740 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560450e76970 0x560450e78d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.529+0000 7f2f05c35740 4 client.18240 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:14:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.529+0000 7f2f05c35740 1 -- 172.21.15.5:0/3780587253 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560450d5cd70 con 0x560450e76970 2024-09-19T06:14:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.529+0000 7f2efbfff640 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560450e76970 0x560450e78d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.529+0000 7f2efbfff640 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560450e76970 0x560450e78d90 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e2dac4cb85fa647 server_cookie=23405dd7a0a10e40 in_seq=0 out_seq=0 2024-09-19T06:14:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.529+0000 7f2edbfff640 10 client.18240 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2edbfff640 1 -- 172.21.15.5:0/3780587253 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560450d5cd70 con 0x560450e76970 2024-09-19T06:14:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2edbfff640 10 client.18240 handle_command_reply: tid=0 2024-09-19T06:14:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2f05c35740 1 client.18240 shutdown 2024-09-19T06:14:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2f05c35740 1 -- 172.21.15.5:0/3780587253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 msgr2=0x560450e03cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2f05c35740 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 0x560450e03cf0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00097b0 tx=0x7f2ef009bf40 comp rx=0 tx=0).stop 2024-09-19T06:14:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2f05c35740 1 -- 172.21.15.5:0/3780587253 shutdown_connections 2024-09-19T06:14:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2f05c35740 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560450e76970 0x560450e78d90 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2f05c35740 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560450dfbcb0 0x560450df4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2f05c35740 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560450df1410 0x560450e04230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2f05c35740 1 --2- 172.21.15.5:0/3780587253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560450df0af0 0x560450e03cf0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2f05c35740 1 -- 172.21.15.5:0/3780587253 >> 172.21.15.5:0/3780587253 conn(0x560450ded060 msgr2=0x560450dec1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2edbfff640 0 client.18240 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2f05c35740 1 -- 172.21.15.5:0/3780587253 shutdown_connections 2024-09-19T06:14:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2f05c35740 1 -- 172.21.15.5:0/3780587253 wait complete. 2024-09-19T06:14:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.558+0000 7f2f05c35740 20 client.18240 trim_cache size 0 max 16384 2024-09-19T06:14:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:14:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:14:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:14:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:14:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.559+0000 7f2f02e9b640 1 -- 172.21.15.5:0/3474287455 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2eec0a33b0 msgr2=0x7f2eec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.559+0000 7f2f02e9b640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2eec0a33b0 0x7f2eec0a5870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4005c40 tx=0x7f2ef4004ab0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.559+0000 7f2f02e9b640 1 -- 172.21.15.5:0/3474287455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2efc1009a0 msgr2=0x7f2efc198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.559+0000 7f2f02e9b640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2efc1009a0 0x7f2efc198350 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000e9f0 tx=0x7f2ef000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.559+0000 7f2f02e9b640 1 -- 172.21.15.5:0/3474287455 shutdown_connections 2024-09-19T06:14:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.559+0000 7f2f02e9b640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2eec0a33b0 0x7f2eec0a5870 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.559+0000 7f2f02e9b640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2efc1069b0 0x7f2efc19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.559+0000 7f2f02e9b640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc101340 0x7f2efc198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.559+0000 7f2f02e9b640 1 --2- 172.21.15.5:0/3474287455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2efc1009a0 0x7f2efc198350 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.559+0000 7f2f02e9b640 1 -- 172.21.15.5:0/3474287455 >> 172.21.15.5:0/3474287455 conn(0x7f2efc0779a0 msgr2=0x7f2efc069b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.559+0000 7f2f02e9b640 1 -- 172.21.15.5:0/3474287455 shutdown_connections 2024-09-19T06:14:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:16.559+0000 7f2f02e9b640 1 -- 172.21.15.5:0/3474287455 wait complete. 2024-09-19T06:14:16.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:16 smithi049 ceph-mon[34640]: pgmap v341: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1020 KiB/s wr, 100 op/s 2024-09-19T06:14:16.811 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:16 smithi157 ceph-mon[35346]: pgmap v341: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1020 KiB/s wr, 100 op/s 2024-09-19T06:14:16.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259500 2.30 MB/sec execute 462 sec latency 13.048 ms 2024-09-19T06:14:16.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:16 smithi005 ceph-mon[30955]: pgmap v341: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1020 KiB/s wr, 100 op/s 2024-09-19T06:14:17.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:17 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:14:17.843 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259916 2.30 MB/sec execute 463 sec latency 67.309 ms 2024-09-19T06:14:17.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:17 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:14:17.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:17 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:14:18.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:14:18.504 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:14:18.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:18 smithi005 ceph-mon[30955]: pgmap v342: 97 pgs: 97 active+clean; 142 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 455 KiB/s wr, 57 op/s 2024-09-19T06:14:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:18 smithi157 ceph-mon[35346]: pgmap v342: 97 pgs: 97 active+clean; 142 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 455 KiB/s wr, 57 op/s 2024-09-19T06:14:18.843 INFO:tasks.workunit.client.0.smithi005.stdout: 1 260444 2.30 MB/sec execute 464 sec latency 20.167 ms 2024-09-19T06:14:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:18 smithi049 ceph-mon[34640]: pgmap v342: 97 pgs: 97 active+clean; 142 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 455 KiB/s wr, 57 op/s 2024-09-19T06:14:19.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.027+0000 7f8b4c34e640 1 -- 172.21.15.5:0/489601661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b44100e20 msgr2=0x7f8b44101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:19.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.027+0000 7f8b4c34e640 1 --2- 172.21.15.5:0/489601661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b44100e20 0x7f8b44101200 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009900 tx=0x7f8b4002a480 comp rx=0 tx=0).stop 2024-09-19T06:14:19.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.027+0000 7f8b4c34e640 1 -- 172.21.15.5:0/489601661 shutdown_connections 2024-09-19T06:14:19.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.027+0000 7f8b4c34e640 1 --2- 172.21.15.5:0/489601661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b44102ef0 0x7f8b4410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.027+0000 7f8b4c34e640 1 --2- 172.21.15.5:0/489601661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b44102570 0x7f8b441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.027+0000 7f8b4c34e640 1 --2- 172.21.15.5:0/489601661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b44100e20 0x7f8b44101200 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.027+0000 7f8b4c34e640 1 -- 172.21.15.5:0/489601661 >> 172.21.15.5:0/489601661 conn(0x7f8b440fc7d0 msgr2=0x7f8b440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.027+0000 7f8b4c34e640 1 -- 172.21.15.5:0/489601661 shutdown_connections 2024-09-19T06:14:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.027+0000 7f8b4c34e640 1 -- 172.21.15.5:0/489601661 wait complete. 2024-09-19T06:14:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4c34e640 1 Processor -- start 2024-09-19T06:14:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4c34e640 1 -- start start 2024-09-19T06:14:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4c34e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b44100e20 0x7f8b44198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4c34e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b44102570 0x7f8b44198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4c34e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b44102ef0 0x7f8b4419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4c34e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b44110e30 con 0x7f8b44100e20 2024-09-19T06:14:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4c34e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8b44110cb0 con 0x7f8b44102ef0 2024-09-19T06:14:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4c34e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b44110fb0 con 0x7f8b44102570 2024-09-19T06:14:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4a0ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b44100e20 0x7f8b44198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4a8eb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b44102ef0 0x7f8b4419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4a0ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b44100e20 0x7f8b44198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38480/0 (socket says 172.21.15.5:38480) 2024-09-19T06:14:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4a0ea640 1 -- 172.21.15.5:0/3790475460 learned_addr learned my addr 172.21.15.5:0/3790475460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b498e9640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b44102570 0x7f8b44198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4a8eb640 1 -- 172.21.15.5:0/3790475460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b44102570 msgr2=0x7f8b44198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4a8eb640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b44102570 0x7f8b44198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4a8eb640 1 -- 172.21.15.5:0/3790475460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b44100e20 msgr2=0x7f8b44198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.028+0000 7f8b4a8eb640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b44100e20 0x7f8b44198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.029+0000 7f8b4a8eb640 1 -- 172.21.15.5:0/3790475460 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4419d3e0 con 0x7f8b44102ef0 2024-09-19T06:14:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.029+0000 7f8b4a8eb640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b44102ef0 0x7f8b4419cd00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009fd0 tx=0x7f8b3c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:19.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.029+0000 7f8b3b7fe640 1 -- 172.21.15.5:0/3790475460 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00ce20 con 0x7f8b44102ef0 2024-09-19T06:14:19.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.029+0000 7f8b4c34e640 1 -- 172.21.15.5:0/3790475460 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4419d6d0 con 0x7f8b44102ef0 2024-09-19T06:14:19.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.029+0000 7f8b4c34e640 1 -- 172.21.15.5:0/3790475460 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b441a4fb0 con 0x7f8b44102ef0 2024-09-19T06:14:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.029+0000 7f8b3b7fe640 1 -- 172.21.15.5:0/3790475460 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b3c010430 con 0x7f8b44102ef0 2024-09-19T06:14:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.029+0000 7f8b3b7fe640 1 -- 172.21.15.5:0/3790475460 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c003b70 con 0x7f8b44102ef0 2024-09-19T06:14:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.032+0000 7f8b3b7fe640 1 -- 172.21.15.5:0/3790475460 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8b3c003d10 con 0x7f8b44102ef0 2024-09-19T06:14:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.032+0000 7f8b4e727740 1 Processor -- start 2024-09-19T06:14:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.032+0000 7f8b3b7fe640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8b300a33b0 0x7f8b300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b4e727740 1 -- start start 2024-09-19T06:14:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b3b7fe640 1 -- 172.21.15.5:0/3790475460 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f8b3c0cff20 con 0x7f8b44102ef0 2024-09-19T06:14:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b498e9640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8b300a33b0 0x7f8b300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b4e727740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3380 0x5642c1ad3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b4e727740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3ca0 0x5642c1ae2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b4e727740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ae2a90 0x5642c1ae4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b4e727740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5642c1a392e0 con 0x5642c1ae2a90 2024-09-19T06:14:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b4e727740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5642c19e0ce0 con 0x5642c1ad3ca0 2024-09-19T06:14:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b4e727740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5642c1a352a0 con 0x5642c1ad3380 2024-09-19T06:14:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b4a8eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ae2a90 0x5642c1ae4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b498e9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3ca0 0x5642c1ae2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b4a8eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ae2a90 0x5642c1ae4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38496/0 (socket says 172.21.15.5:38496) 2024-09-19T06:14:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b4a8eb640 1 -- 172.21.15.5:0/2747554334 learned_addr learned my addr 172.21.15.5:0/2747554334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b4a0ea640 1 --2- 172.21.15.5:0/2747554334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3380 0x5642c1ad3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b498e9640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8b300a33b0 0x7f8b300a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8b440fffa0 tx=0x7f8b3400bab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b498e9640 1 -- 172.21.15.5:0/2747554334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3380 msgr2=0x5642c1ad3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b498e9640 1 --2- 172.21.15.5:0/2747554334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3380 0x5642c1ad3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b498e9640 1 -- 172.21.15.5:0/2747554334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ae2a90 msgr2=0x5642c1ae4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:19.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b498e9640 1 --2- 172.21.15.5:0/2747554334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ae2a90 0x5642c1ae4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b498e9640 1 -- 172.21.15.5:0/2747554334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642c19c5440 con 0x5642c1ad3ca0 2024-09-19T06:14:19.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.033+0000 7f8b4a0ea640 1 --2- 172.21.15.5:0/2747554334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3380 0x5642c1ad3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:19.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.034+0000 7f8b498e9640 1 --2- 172.21.15.5:0/2747554334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3ca0 0x5642c1ae2550 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8b34005fd0 tx=0x7f8b3401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f74e5b4a956152d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:19.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.034+0000 7f8b38dea640 1 -- 172.21.15.5:0/2747554334 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b340130d0 con 0x5642c1ad3ca0 2024-09-19T06:14:19.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.034+0000 7f8b38dea640 1 -- 172.21.15.5:0/2747554334 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b34013270 con 0x5642c1ad3ca0 2024-09-19T06:14:19.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.034+0000 7f8b38dea640 1 -- 172.21.15.5:0/2747554334 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3401e680 con 0x5642c1ad3ca0 2024-09-19T06:14:19.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.035+0000 7f8b4e727740 1 -- 172.21.15.5:0/2747554334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3ca0 msgr2=0x5642c1ae2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.035+0000 7f8b4e727740 1 --2- 172.21.15.5:0/2747554334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3ca0 0x5642c1ae2550 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8b34005fd0 tx=0x7f8b3401b000 comp rx=0 tx=0).stop 2024-09-19T06:14:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.035+0000 7f8b4e727740 1 -- 172.21.15.5:0/2747554334 shutdown_connections 2024-09-19T06:14:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.035+0000 7f8b4e727740 1 --2- 172.21.15.5:0/2747554334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ae2a90 0x5642c1ae4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.035+0000 7f8b4e727740 1 --2- 172.21.15.5:0/2747554334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3ca0 0x5642c1ae2550 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.035+0000 7f8b4e727740 1 --2- 172.21.15.5:0/2747554334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3380 0x5642c1ad3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.035+0000 7f8b4e727740 1 -- 172.21.15.5:0/2747554334 >> 172.21.15.5:0/2747554334 conn(0x5642c1acfa30 msgr2=0x5642c1acfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.035+0000 7f8b4e727740 1 -- 172.21.15.5:0/2747554334 shutdown_connections 2024-09-19T06:14:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.035+0000 7f8b4e727740 1 -- 172.21.15.5:0/2747554334 wait complete. 2024-09-19T06:14:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.035+0000 7f8b4e727740 1 -- start start 2024-09-19T06:14:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.035+0000 7f8b4e727740 1 Processor -- start 2024-09-19T06:14:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b4e727740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 0x5642c1ae2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b4e727740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3ca0 0x5642c1adf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b4e727740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ae2a90 0x5642c1adfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b4e727740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5642c1ae5660 con 0x5642c1ad3380 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b4e727740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5642c19f4050 con 0x5642c1ae2a90 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b4e727740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5642c1ae57e0 con 0x5642c1ad3ca0 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b4e727740 10 client.0 _ll_register_callbacks cb 0x5642c1a34160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b4a0ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 0x5642c1ae2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b4a0ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 0x5642c1ae2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38498/0 (socket says 172.21.15.5:38498) 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b4a0ea640 1 -- 172.21.15.5:0/2602654460 learned_addr learned my addr 172.21.15.5:0/2602654460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b498e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3ca0 0x5642c1adf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b4a8eb640 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ae2a90 0x5642c1adfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b498e9640 1 -- 172.21.15.5:0/2602654460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ae2a90 msgr2=0x5642c1adfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b498e9640 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ae2a90 0x5642c1adfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b498e9640 1 -- 172.21.15.5:0/2602654460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 msgr2=0x5642c1ae2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b498e9640 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 0x5642c1ae2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b498e9640 1 -- 172.21.15.5:0/2602654460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642c17aa3c0 con 0x5642c1ad3ca0 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b4a8eb640 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ae2a90 0x5642c1adfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.036+0000 7f8b498e9640 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3ca0 0x5642c1adf6d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b34005fd0 tx=0x7f8b34003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48ffd8d785ec4645 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.037+0000 7f8b2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.037+0000 7f8b2dffb640 1 -- 172.21.15.5:0/2602654460 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34011850 con 0x5642c1ad3ca0 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.037+0000 7f8b4e727740 1 -- 172.21.15.5:0/2602654460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b34013270 con 0x5642c1ad3ca0 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.037+0000 7f8b2dffb640 1 -- 172.21.15.5:0/2602654460 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f8b340119f0 con 0x5642c1ad3ca0 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.037+0000 7f8b2dffb640 1 -- 172.21.15.5:0/2602654460 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8b34011bf0 con 0x5642c1ad3ca0 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.037+0000 7f8b4e727740 10 client.27691 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.037+0000 7f8b4e727740 1 -- 172.21.15.5:0/2602654460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5642c1af10c0 con 0x5642c1ad3ca0 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.037+0000 7f8b2dffb640 1 -- 172.21.15.5:0/2602654460 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b34013270 con 0x5642c1ad3ca0 2024-09-19T06:14:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.038+0000 7f8b2dffb640 1 -- 172.21.15.5:0/2602654460 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f8b3402d070 con 0x5642c1ad3ca0 2024-09-19T06:14:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.038+0000 7f8b4e727740 10 client.27691 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.038+0000 7f8b4e727740 10 client.27691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.038+0000 7f8b4e727740 20 client.27691 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.038+0000 7f8b4e727740 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5642c1af2410 0x5642c1af4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.038+0000 7f8b4e727740 4 client.27691 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.038+0000 7f8b4e727740 1 -- 172.21.15.5:0/2602654460 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642c1a46850 con 0x5642c1af2410 2024-09-19T06:14:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.038+0000 7f8b4a0ea640 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5642c1af2410 0x5642c1af4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.039+0000 7f8b4a0ea640 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5642c1af2410 0x5642c1af4800 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a24fe2c5cc4979c server_cookie=1035e870458aab7 in_seq=0 out_seq=0 2024-09-19T06:14:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.039+0000 7f8b2dffb640 10 client.27691 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.039+0000 7f8b2dffb640 1 -- 172.21.15.5:0/2602654460 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642c1a46850 con 0x5642c1af2410 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.039+0000 7f8b2dffb640 10 client.27691 handle_command_reply: tid=0 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.039+0000 7f8b4e727740 1 client.27691 shutdown 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.039+0000 7f8b4e727740 1 -- 172.21.15.5:0/2602654460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3ca0 msgr2=0x5642c1adf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.039+0000 7f8b4e727740 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3ca0 0x5642c1adf6d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b34005fd0 tx=0x7f8b34003f60 comp rx=0 tx=0).stop 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.039+0000 7f8b4e727740 1 -- 172.21.15.5:0/2602654460 shutdown_connections 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.039+0000 7f8b4e727740 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5642c1af2410 0x5642c1af4800 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.040+0000 7f8b2dffb640 0 client.27691 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.040+0000 7f8b4e727740 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ae2a90 0x5642c1adfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.040+0000 7f8b4e727740 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3ca0 0x5642c1adf6d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.040+0000 7f8b4e727740 1 --2- 172.21.15.5:0/2602654460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 0x5642c1ae2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.040+0000 7f8b4e727740 1 -- 172.21.15.5:0/2602654460 >> 172.21.15.5:0/2602654460 conn(0x5642c1acfa30 msgr2=0x5642c1ad5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.040+0000 7f8b4e727740 1 -- 172.21.15.5:0/2602654460 shutdown_connections 2024-09-19T06:14:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.040+0000 7f8b4e727740 1 -- 172.21.15.5:0/2602654460 wait complete. 2024-09-19T06:14:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.040+0000 7f8b4e727740 20 client.27691 trim_cache size 0 max 16384 2024-09-19T06:14:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.059+0000 7f8b4e727740 1 Processor -- start 2024-09-19T06:14:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.059+0000 7f8b4e727740 1 -- start start 2024-09-19T06:14:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.059+0000 7f8b4e727740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 0x5642c1aee930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.059+0000 7f8b4e727740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3ca0 0x5642c1acea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.059+0000 7f8b4e727740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ae2a90 0x5642c1af9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.059+0000 7f8b4e727740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2800e790 con 0x5642c1ad3380 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.059+0000 7f8b4e727740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28010390 con 0x5642c1ae2a90 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.059+0000 7f8b4e727740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2800ee60 con 0x5642c1ad3ca0 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.060+0000 7f8b4a0ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 0x5642c1aee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.060+0000 7f8b4a0ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 0x5642c1aee930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38508/0 (socket says 172.21.15.5:38508) 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.060+0000 7f8b4a0ea640 1 -- 172.21.15.5:0/3279479420 learned_addr learned my addr 172.21.15.5:0/3279479420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.060+0000 7f8b498e9640 1 --2- 172.21.15.5:0/3279479420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3ca0 0x5642c1acea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.060+0000 7f8b4a8eb640 1 --2- 172.21.15.5:0/3279479420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ae2a90 0x5642c1af9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.060+0000 7f8b4a8eb640 1 -- 172.21.15.5:0/3279479420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3ca0 msgr2=0x5642c1acea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.060+0000 7f8b4a8eb640 1 --2- 172.21.15.5:0/3279479420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3ca0 0x5642c1acea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.060+0000 7f8b4a8eb640 1 -- 172.21.15.5:0/3279479420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 msgr2=0x5642c1aee930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.060+0000 7f8b4a8eb640 1 --2- 172.21.15.5:0/3279479420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 0x5642c1aee930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.060+0000 7f8b4a8eb640 1 -- 172.21.15.5:0/3279479420 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642c1adffc0 con 0x5642c1ae2a90 2024-09-19T06:14:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.060+0000 7f8b4a0ea640 1 --2- 172.21.15.5:0/3279479420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 0x5642c1aee930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.060+0000 7f8b4a8eb640 1 --2- 172.21.15.5:0/3279479420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ae2a90 0x5642c1af9f20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0a4920 tx=0x7f8b3c0bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7be918d8991a0113 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.061+0000 7f8b2ffff640 1 -- 172.21.15.5:0/3279479420 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0b4030 con 0x5642c1ae2a90 2024-09-19T06:14:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.061+0000 7f8b2ffff640 1 -- 172.21.15.5:0/3279479420 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b3c098850 con 0x5642c1ae2a90 2024-09-19T06:14:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.061+0000 7f8b2ffff640 1 -- 172.21.15.5:0/3279479420 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0a49c0 con 0x5642c1ae2a90 2024-09-19T06:14:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.062+0000 7f8b4e727740 1 -- 172.21.15.5:0/3279479420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ae2a90 msgr2=0x5642c1af9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.062+0000 7f8b4e727740 1 --2- 172.21.15.5:0/3279479420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ae2a90 0x5642c1af9f20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0a4920 tx=0x7f8b3c0bc040 comp rx=0 tx=0).stop 2024-09-19T06:14:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.062+0000 7f8b4e727740 1 -- 172.21.15.5:0/3279479420 shutdown_connections 2024-09-19T06:14:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.062+0000 7f8b4e727740 1 --2- 172.21.15.5:0/3279479420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ae2a90 0x5642c1af9f20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.062+0000 7f8b4e727740 1 --2- 172.21.15.5:0/3279479420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ad3ca0 0x5642c1acea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.062+0000 7f8b4e727740 1 --2- 172.21.15.5:0/3279479420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3380 0x5642c1aee930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.062+0000 7f8b4e727740 1 -- 172.21.15.5:0/3279479420 >> 172.21.15.5:0/3279479420 conn(0x5642c1acfa30 msgr2=0x5642c1ace370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.062+0000 7f8b4e727740 1 -- 172.21.15.5:0/3279479420 shutdown_connections 2024-09-19T06:14:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.062+0000 7f8b4e727740 1 -- 172.21.15.5:0/3279479420 wait complete. 2024-09-19T06:14:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.062+0000 7f8b4e727740 1 -- start start 2024-09-19T06:14:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.062+0000 7f8b4e727740 1 Processor -- start 2024-09-19T06:14:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b4e727740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3380 0x5642c1ad9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b4e727740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3ca0 0x5642c1aeaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b4e727740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ae2a90 0x5642c1aeafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b4e727740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2800fcc0 con 0x5642c1ad3ca0 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b4e727740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2800f590 con 0x5642c1ad3380 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b4e727740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2800c790 con 0x5642c1ae2a90 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b4e727740 10 client.0 _ll_register_callbacks cb 0x7f8b3c09bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b4a0ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3380 0x5642c1ad9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b498e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3ca0 0x5642c1aeaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b4a0ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3380 0x5642c1ad9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58418/0 (socket says 172.21.15.5:58418) 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b4a0ea640 1 -- 172.21.15.5:0/3728542972 learned_addr learned my addr 172.21.15.5:0/3728542972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b498e9640 1 -- 172.21.15.5:0/3728542972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ae2a90 msgr2=0x5642c1aeafe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b498e9640 1 --2- 172.21.15.5:0/3728542972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ae2a90 0x5642c1aeafe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b498e9640 1 -- 172.21.15.5:0/3728542972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3380 msgr2=0x5642c1ad9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b498e9640 1 --2- 172.21.15.5:0/3728542972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3380 0x5642c1ad9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b498e9640 1 -- 172.21.15.5:0/3728542972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642c1aeb520 con 0x5642c1ad3ca0 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.063+0000 7f8b4a0ea640 1 --2- 172.21.15.5:0/3728542972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3380 0x5642c1ad9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.064+0000 7f8b498e9640 1 --2- 172.21.15.5:0/3728542972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3ca0 0x5642c1aeaaa0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8b34037550 tx=0x7f8b34011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd0c5c1c98ef37b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.064+0000 7f8b2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.064+0000 7f8b2dffb640 1 -- 172.21.15.5:0/3728542972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34005250 con 0x5642c1ad3ca0 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.064+0000 7f8b2dffb640 1 -- 172.21.15.5:0/3728542972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f8b340053f0 con 0x5642c1ad3ca0 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.064+0000 7f8b4e727740 1 -- 172.21.15.5:0/3728542972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b3c098850 con 0x5642c1ad3ca0 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.064+0000 7f8b2dffb640 1 -- 172.21.15.5:0/3728542972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8b340376c0 con 0x5642c1ad3ca0 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.064+0000 7f8b4e727740 10 client.18258 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.064+0000 7f8b4e727740 1 -- 172.21.15.5:0/3728542972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5642c1b58ed0 con 0x5642c1ad3ca0 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.064+0000 7f8b2dffb640 1 -- 172.21.15.5:0/3728542972 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b3c098850 con 0x5642c1ad3ca0 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.064+0000 7f8b2dffb640 1 -- 172.21.15.5:0/3728542972 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f8b3402d040 con 0x5642c1ad3ca0 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.065+0000 7f8b4e727740 10 client.18258 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.065+0000 7f8b4e727740 10 client.18258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.065+0000 7f8b4e727740 20 client.18258 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.065+0000 7f8b4e727740 1 --2- 172.21.15.5:0/3728542972 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5642c1af2410 0x5642c1b5b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.065+0000 7f8b4e727740 4 client.18258 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:14:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.065+0000 7f8b4e727740 1 -- 172.21.15.5:0/3728542972 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5642c1a40d60 con 0x5642c1af2410 2024-09-19T06:14:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.065+0000 7f8b4a0ea640 1 --2- 172.21.15.5:0/3728542972 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5642c1af2410 0x5642c1b5b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.065+0000 7f8b2dffb640 10 client.18258 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.065+0000 7f8b4a0ea640 1 --2- 172.21.15.5:0/3728542972 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5642c1af2410 0x5642c1b5b500 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b221ad44b97e301 server_cookie=82c28c40da03ddd0 in_seq=0 out_seq=0 2024-09-19T06:14:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b2dffb640 1 -- 172.21.15.5:0/3728542972 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5642c1a40d60 con 0x5642c1af2410 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b2dffb640 10 client.18258 handle_command_reply: tid=0 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b4e727740 1 client.18258 shutdown 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b4e727740 1 -- 172.21.15.5:0/3728542972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3ca0 msgr2=0x5642c1aeaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b4e727740 1 --2- 172.21.15.5:0/3728542972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3ca0 0x5642c1aeaaa0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8b34037550 tx=0x7f8b34011ca0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b4e727740 1 -- 172.21.15.5:0/3728542972 shutdown_connections 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b4e727740 1 --2- 172.21.15.5:0/3728542972 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5642c1af2410 0x5642c1b5b500 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b4e727740 1 --2- 172.21.15.5:0/3728542972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642c1ae2a90 0x5642c1aeafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b4e727740 1 --2- 172.21.15.5:0/3728542972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642c1ad3ca0 0x5642c1aeaaa0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b4e727740 1 --2- 172.21.15.5:0/3728542972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5642c1ad3380 0x5642c1ad9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b4e727740 1 -- 172.21.15.5:0/3728542972 >> 172.21.15.5:0/3728542972 conn(0x5642c1acfa30 msgr2=0x5642c1ae1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b2dffb640 0 client.18258 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b4e727740 1 -- 172.21.15.5:0/3728542972 shutdown_connections 2024-09-19T06:14:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b4e727740 1 -- 172.21.15.5:0/3728542972 wait complete. 2024-09-19T06:14:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.066+0000 7f8b4e727740 20 client.18258 trim_cache size 0 max 16384 2024-09-19T06:14:19.073 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:14:19.073 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:14:19.073 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:14:19.073 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:14:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.067+0000 7f8b4c34e640 1 -- 172.21.15.5:0/3790475460 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8b300a33b0 msgr2=0x7f8b300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.067+0000 7f8b4c34e640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8b300a33b0 0x7f8b300a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8b440fffa0 tx=0x7f8b3400bab0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.067+0000 7f8b4c34e640 1 -- 172.21.15.5:0/3790475460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b44102ef0 msgr2=0x7f8b4419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.067+0000 7f8b4c34e640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b44102ef0 0x7f8b4419cd00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009fd0 tx=0x7f8b3c00ef10 comp rx=0 tx=0).stop 2024-09-19T06:14:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.067+0000 7f8b4c34e640 1 -- 172.21.15.5:0/3790475460 shutdown_connections 2024-09-19T06:14:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.067+0000 7f8b4c34e640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b44102ef0 0x7f8b4419cd00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.067+0000 7f8b4c34e640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8b300a33b0 0x7f8b300a5870 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.067+0000 7f8b4c34e640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b44102570 0x7f8b44198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.067+0000 7f8b4c34e640 1 --2- 172.21.15.5:0/3790475460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b44100e20 0x7f8b44198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.067+0000 7f8b4c34e640 1 -- 172.21.15.5:0/3790475460 >> 172.21.15.5:0/3790475460 conn(0x7f8b440fc7d0 msgr2=0x7f8b4406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.067+0000 7f8b4c34e640 1 -- 172.21.15.5:0/3790475460 shutdown_connections 2024-09-19T06:14:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:19.068+0000 7f8b4c34e640 1 -- 172.21.15.5:0/3790475460 wait complete. 2024-09-19T06:14:19.843 INFO:tasks.workunit.client.0.smithi005.stdout: 1 260948 2.30 MB/sec execute 465 sec latency 22.766 ms 2024-09-19T06:14:19.973 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:14:19.974 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:14:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:14:20.692 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:20 smithi005 ceph-mon[30955]: pgmap v343: 97 pgs: 97 active+clean; 142 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 436 KiB/s wr, 54 op/s 2024-09-19T06:14:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:20 smithi157 ceph-mon[35346]: pgmap v343: 97 pgs: 97 active+clean; 142 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 436 KiB/s wr, 54 op/s 2024-09-19T06:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.783+0000 7fafc70da640 1 -- 172.21.15.5:0/1926174809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc010e440 msgr2=0x7fafc0110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.783+0000 7fafc70da640 1 --2- 172.21.15.5:0/1926174809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc010e440 0x7fafc0110830 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009ca0 tx=0x7fafb802a950 comp rx=0 tx=0).stop 2024-09-19T06:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.783+0000 7fafc70da640 1 -- 172.21.15.5:0/1926174809 shutdown_connections 2024-09-19T06:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.783+0000 7fafc70da640 1 --2- 172.21.15.5:0/1926174809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc010e440 0x7fafc0110830 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.783+0000 7fafc70da640 1 --2- 172.21.15.5:0/1926174809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc0074e30 0x7fafc010df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.783+0000 7fafc70da640 1 --2- 172.21.15.5:0/1926174809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc0074510 0x7fafc00748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.783+0000 7fafc70da640 1 -- 172.21.15.5:0/1926174809 >> 172.21.15.5:0/1926174809 conn(0x7fafc00fd8c0 msgr2=0x7fafc00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.783+0000 7fafc70da640 1 -- 172.21.15.5:0/1926174809 shutdown_connections 2024-09-19T06:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.783+0000 7fafc70da640 1 -- 172.21.15.5:0/1926174809 wait complete. 2024-09-19T06:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafc70da640 1 Processor -- start 2024-09-19T06:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafc70da640 1 -- start start 2024-09-19T06:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafc70da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc0074510 0x7fafc019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafc70da640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc0074e30 0x7fafc019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafc70da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc010e440 0x7fafc019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafc70da640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fafc0114000 con 0x7fafc0074510 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafc70da640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fafc0113e80 con 0x7fafc0074e30 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafc70da640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fafc0114180 con 0x7fafc010e440 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafc4e76640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc0074510 0x7fafc019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafc4e76640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc0074510 0x7fafc019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38520/0 (socket says 172.21.15.5:38520) 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafc4e76640 1 -- 172.21.15.5:0/3537315620 learned_addr learned my addr 172.21.15.5:0/3537315620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafbffff640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc0074e30 0x7fafc019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.784+0000 7fafc5677640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc010e440 0x7fafc019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.785+0000 7fafc4e76640 1 -- 172.21.15.5:0/3537315620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc010e440 msgr2=0x7fafc019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.785+0000 7fafc4e76640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc010e440 0x7fafc019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.785+0000 7fafc4e76640 1 -- 172.21.15.5:0/3537315620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc0074e30 msgr2=0x7fafc019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.785+0000 7fafc4e76640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc0074e30 0x7fafc019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.785+0000 7fafc4e76640 1 -- 172.21.15.5:0/3537315620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc019f500 con 0x7fafc0074510 2024-09-19T06:14:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.785+0000 7fafc4e76640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc0074510 0x7fafc019a550 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e9f0 tx=0x7fafb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.785+0000 7fafbdffb640 1 -- 172.21.15.5:0/3537315620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb400cd80 con 0x7fafc0074510 2024-09-19T06:14:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.785+0000 7fafbdffb640 1 -- 172.21.15.5:0/3537315620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafb40108e0 con 0x7fafc0074510 2024-09-19T06:14:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.785+0000 7fafbdffb640 1 -- 172.21.15.5:0/3537315620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4018920 con 0x7fafc0074510 2024-09-19T06:14:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.785+0000 7fafc70da640 1 -- 172.21.15.5:0/3537315620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc019f7f0 con 0x7fafc0074510 2024-09-19T06:14:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.785+0000 7fafc70da640 1 -- 172.21.15.5:0/3537315620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc01a70d0 con 0x7fafc0074510 2024-09-19T06:14:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.786+0000 7fafc70da640 1 -- 172.21.15.5:0/3537315620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc00748f0 con 0x7fafc0074510 2024-09-19T06:14:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.788+0000 7fafbdffb640 1 -- 172.21.15.5:0/3537315620 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fafb4010a90 con 0x7fafc0074510 2024-09-19T06:14:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.788+0000 7fafbdffb640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fafb00a3430 0x7fafb00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.789+0000 7fafbffff640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fafb00a3430 0x7fafb00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.789+0000 7fafbdffb640 1 -- 172.21.15.5:0/3537315620 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fafb40d0430 con 0x7fafc0074510 2024-09-19T06:14:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.789+0000 7fafbffff640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fafb00a3430 0x7fafb00a58f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fafc010f8d0 tx=0x7fafac00b410 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:20.793+0000 7fafbdffb640 1 -- 172.21.15.5:0/3537315620 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fafb409c7a0 con 0x7fafc0074510 2024-09-19T06:14:20.843 INFO:tasks.workunit.client.0.smithi005.stdout: 1 261436 2.30 MB/sec execute 466 sec latency 31.036 ms 2024-09-19T06:14:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:20 smithi049 ceph-mon[34640]: pgmap v343: 97 pgs: 97 active+clean; 142 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 436 KiB/s wr, 54 op/s 2024-09-19T06:14:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.049+0000 7fafc70da640 1 -- 172.21.15.5:0/3537315620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafc019b8b0 con 0x7fafc0074510 2024-09-19T06:14:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.050+0000 7fafbdffb640 1 -- 172.21.15.5:0/3537315620 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11965 (secure 0 0 0) 0x7fafb40a0aa0 con 0x7fafc0074510 2024-09-19T06:14:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:14:21.052 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":39,"btime":"2024-09-19T06:14:16:386479+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:14:15.452508+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:14:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 39 2024-09-19T06:14:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.052+0000 7fafc70da640 1 -- 172.21.15.5:0/3537315620 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fafb00a3430 msgr2=0x7fafb00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.052+0000 7fafc70da640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fafb00a3430 0x7fafb00a58f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fafc010f8d0 tx=0x7fafac00b410 comp rx=0 tx=0).stop 2024-09-19T06:14:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.052+0000 7fafc70da640 1 -- 172.21.15.5:0/3537315620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc0074510 msgr2=0x7fafc019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.052+0000 7fafc70da640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc0074510 0x7fafc019a550 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e9f0 tx=0x7fafb400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:14:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.052+0000 7fafc70da640 1 -- 172.21.15.5:0/3537315620 shutdown_connections 2024-09-19T06:14:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.052+0000 7fafc70da640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fafb00a3430 0x7fafb00a58f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.052+0000 7fafc70da640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc010e440 0x7fafc019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.052+0000 7fafc70da640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc0074e30 0x7fafc019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.052+0000 7fafc70da640 1 --2- 172.21.15.5:0/3537315620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc0074510 0x7fafc019a550 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.052+0000 7fafc70da640 1 -- 172.21.15.5:0/3537315620 >> 172.21.15.5:0/3537315620 conn(0x7fafc00fd8c0 msgr2=0x7fafc00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.052+0000 7fafc70da640 1 -- 172.21.15.5:0/3537315620 shutdown_connections 2024-09-19T06:14:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:21.052+0000 7fafc70da640 1 -- 172.21.15.5:0/3537315620 wait complete. 2024-09-19T06:14:21.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:21 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3537315620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:14:21.729 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:14:21.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:21 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3537315620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:14:21.843 INFO:tasks.workunit.client.0.smithi005.stdout: 1 262177 2.31 MB/sec execute 467 sec latency 15.598 ms 2024-09-19T06:14:21.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:21 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3537315620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:14:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:14:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.515+0000 7f93c25f0640 1 -- 172.21.15.5:0/2300838062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93bc102d30 msgr2=0x7f93bc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.515+0000 7f93c25f0640 1 --2- 172.21.15.5:0/2300838062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93bc102d30 0x7f93bc103110 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009930 tx=0x7f93ac02a420 comp rx=0 tx=0).stop 2024-09-19T06:14:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.516+0000 7f93c25f0640 1 -- 172.21.15.5:0/2300838062 shutdown_connections 2024-09-19T06:14:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.516+0000 7f93c25f0640 1 --2- 172.21.15.5:0/2300838062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc1082a0 0x7f93bc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.516+0000 7f93c25f0640 1 --2- 172.21.15.5:0/2300838062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93bc1036e0 0x7f93bc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.516+0000 7f93c25f0640 1 --2- 172.21.15.5:0/2300838062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93bc102d30 0x7f93bc103110 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.516+0000 7f93c25f0640 1 -- 172.21.15.5:0/2300838062 >> 172.21.15.5:0/2300838062 conn(0x7f93bc0fc7d0 msgr2=0x7f93bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.516+0000 7f93c25f0640 1 -- 172.21.15.5:0/2300838062 shutdown_connections 2024-09-19T06:14:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.516+0000 7f93c25f0640 1 -- 172.21.15.5:0/2300838062 wait complete. 2024-09-19T06:14:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.516+0000 7f93c25f0640 1 Processor -- start 2024-09-19T06:14:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.516+0000 7f93c25f0640 1 -- start start 2024-09-19T06:14:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93c25f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93bc102d30 0x7f93bc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93c25f0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93bc1036e0 0x7f93bc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93c25f0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc1082a0 0x7f93bc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93c25f0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc111f50 con 0x7f93bc102d30 2024-09-19T06:14:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93c25f0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc111dd0 con 0x7f93bc1036e0 2024-09-19T06:14:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93c25f0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1120d0 con 0x7f93bc1082a0 2024-09-19T06:14:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93bbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93bc102d30 0x7f93bc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93bbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93bc102d30 0x7f93bc100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38530/0 (socket says 172.21.15.5:38530) 2024-09-19T06:14:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93bbfff640 1 -- 172.21.15.5:0/1861586105 learned_addr learned my addr 172.21.15.5:0/1861586105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93bc1036e0 0x7f93bc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93c0b8d640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc1082a0 0x7f93bc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93bbfff640 1 -- 172.21.15.5:0/1861586105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc1082a0 msgr2=0x7f93bc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93bbfff640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc1082a0 0x7f93bc101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93bbfff640 1 -- 172.21.15.5:0/1861586105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93bc1036e0 msgr2=0x7f93bc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93bbfff640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93bc1036e0 0x7f93bc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93bbfff640 1 -- 172.21.15.5:0/1861586105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93bc1a69b0 con 0x7f93bc102d30 2024-09-19T06:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.517+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93bc1036e0 0x7f93bc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.518+0000 7f93bbfff640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93bc102d30 0x7f93bc100990 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005940 tx=0x7f93ac0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.518+0000 7f93b97fa640 1 -- 172.21.15.5:0/1861586105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac03f030 con 0x7f93bc102d30 2024-09-19T06:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.518+0000 7f93c25f0640 1 -- 172.21.15.5:0/1861586105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93bc1a6c40 con 0x7f93bc102d30 2024-09-19T06:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.518+0000 7f93b97fa640 1 -- 172.21.15.5:0/1861586105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93ac0044c0 con 0x7f93bc102d30 2024-09-19T06:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.518+0000 7f93b97fa640 1 -- 172.21.15.5:0/1861586105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac035070 con 0x7f93bc102d30 2024-09-19T06:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.518+0000 7f93c25f0640 1 -- 172.21.15.5:0/1861586105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93bc1a7170 con 0x7f93bc102d30 2024-09-19T06:14:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93c49c9740 1 Processor -- start 2024-09-19T06:14:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93c49c9740 1 -- start start 2024-09-19T06:14:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93b97fa640 1 -- 172.21.15.5:0/1861586105 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f93ac042030 con 0x7f93bc102d30 2024-09-19T06:14:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93c49c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 0x5617a7145090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93c49c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a71455d0 0x5617a7145a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93c49c9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7150370 0x5617a7152760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93c49c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617a70ab2f0 con 0x5617a71455d0 2024-09-19T06:14:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93c49c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5617a7052cf0 con 0x5617a7146b40 2024-09-19T06:14:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93c49c9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5617a70a72b0 con 0x5617a7150370 2024-09-19T06:14:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93bbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a71455d0 0x5617a7145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93c0b8d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 0x5617a7145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93c0b8d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 0x5617a7145090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58456/0 (socket says 172.21.15.5:58456) 2024-09-19T06:14:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93c0b8d640 1 -- 172.21.15.5:0/3664800394 learned_addr learned my addr 172.21.15.5:0/3664800394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.521+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/3664800394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7150370 0x5617a7152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.522+0000 7f93bb7fe640 1 -- 172.21.15.5:0/3664800394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 msgr2=0x5617a7145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.522+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/3664800394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 0x5617a7145090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.522+0000 7f93bb7fe640 1 -- 172.21.15.5:0/3664800394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a71455d0 msgr2=0x5617a7145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.522+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/3664800394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a71455d0 0x5617a7145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.522+0000 7f93bb7fe640 1 -- 172.21.15.5:0/3664800394 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a70373a0 con 0x5617a7150370 2024-09-19T06:14:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.522+0000 7f93c0b8d640 1 --2- 172.21.15.5:0/3664800394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 0x5617a7145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.522+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/3664800394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7150370 0x5617a7152760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93bc06a370 tx=0x7f93b00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da2b27bca5d2a3e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.522+0000 7f93a27fc640 1 -- 172.21.15.5:0/3664800394 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0004030 con 0x5617a7150370 2024-09-19T06:14:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.522+0000 7f93a27fc640 1 -- 172.21.15.5:0/3664800394 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b00054d0 con 0x5617a7150370 2024-09-19T06:14:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.522+0000 7f93a27fc640 1 -- 172.21.15.5:0/3664800394 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0004ce0 con 0x5617a7150370 2024-09-19T06:14:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.522+0000 7f93b97fa640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f93a40a33b0 0x7f93a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.523+0000 7f93c0b8d640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f93a40a33b0 0x7f93a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.523+0000 7f93b97fa640 1 -- 172.21.15.5:0/1861586105 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f93ac0effd0 con 0x7f93bc102d30 2024-09-19T06:14:22.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.523+0000 7f93c49c9740 1 -- 172.21.15.5:0/3664800394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7150370 msgr2=0x5617a7152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:22.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.523+0000 7f93c49c9740 1 --2- 172.21.15.5:0/3664800394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7150370 0x5617a7152760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93bc06a370 tx=0x7f93b00118d0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.523+0000 7f93c0b8d640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f93a40a33b0 0x7f93a40a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f93b4012640 tx=0x7f93b4009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:22.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.523+0000 7f93c49c9740 1 -- 172.21.15.5:0/3664800394 shutdown_connections 2024-09-19T06:14:22.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.523+0000 7f93c49c9740 1 --2- 172.21.15.5:0/3664800394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7150370 0x5617a7152760 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.523+0000 7f93c49c9740 1 --2- 172.21.15.5:0/3664800394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a71455d0 0x5617a7145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.523+0000 7f93c49c9740 1 --2- 172.21.15.5:0/3664800394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 0x5617a7145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.523+0000 7f93c49c9740 1 -- 172.21.15.5:0/3664800394 >> 172.21.15.5:0/3664800394 conn(0x5617a710e310 msgr2=0x5617a7141c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.523+0000 7f93c49c9740 1 -- 172.21.15.5:0/3664800394 shutdown_connections 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.523+0000 7f93c49c9740 1 -- 172.21.15.5:0/3664800394 wait complete. 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.524+0000 7f93c49c9740 1 -- start start 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.524+0000 7f93c49c9740 1 Processor -- start 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.524+0000 7f93c49c9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a71455d0 0x5617a714fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.524+0000 7f93c49c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 0x5617a714cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.524+0000 7f93c49c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7150370 0x5617a714d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.524+0000 7f93c49c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617a7153070 con 0x5617a7150370 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.524+0000 7f93c49c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5617a7066060 con 0x5617a7146b40 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.524+0000 7f93c49c9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5617a71531f0 con 0x5617a71455d0 2024-09-19T06:14:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93c49c9740 10 client.0 _ll_register_callbacks cb 0x5617a6db98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93bbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a71455d0 0x5617a714fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93bb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 0x5617a714cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93c0b8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7150370 0x5617a714d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93bb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 0x5617a714cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58464/0 (socket says 172.21.15.5:58464) 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93bb7fe640 1 -- 172.21.15.5:0/3004693754 learned_addr learned my addr 172.21.15.5:0/3004693754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93bb7fe640 1 -- 172.21.15.5:0/3004693754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a71455d0 msgr2=0x5617a714fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a71455d0 0x5617a714fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93bb7fe640 1 -- 172.21.15.5:0/3004693754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7150370 msgr2=0x5617a714d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7150370 0x5617a714d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93bb7fe640 1 -- 172.21.15.5:0/3004693754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a6e1c3c0 con 0x5617a7146b40 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93bbfff640 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a71455d0 0x5617a714fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93c0b8d640 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7150370 0x5617a714d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 0x5617a714cfd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009350 tx=0x7f93b0003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bdca56bd670703a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:22.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f939b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:14:22.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.525+0000 7f939b7fe640 1 -- 172.21.15.5:0/3004693754 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b001d050 con 0x5617a7146b40 2024-09-19T06:14:22.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f93c49c9740 1 -- 172.21.15.5:0/3004693754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b00054d0 con 0x5617a7146b40 2024-09-19T06:14:22.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f939b7fe640 1 -- 172.21.15.5:0/3004693754 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f93b0016040 con 0x5617a7146b40 2024-09-19T06:14:22.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f939b7fe640 1 -- 172.21.15.5:0/3004693754 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f93b0031050 con 0x5617a7146b40 2024-09-19T06:14:22.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f93c49c9740 10 client.27770 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:22.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f93c49c9740 1 -- 172.21.15.5:0/3004693754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5617a715eac0 con 0x5617a7146b40 2024-09-19T06:14:22.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f939b7fe640 1 -- 172.21.15.5:0/3004693754 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b00054d0 con 0x5617a7146b40 2024-09-19T06:14:22.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f939b7fe640 1 -- 172.21.15.5:0/3004693754 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f93b0004e90 con 0x5617a7146b40 2024-09-19T06:14:22.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f93c49c9740 10 client.27770 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:22.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f93c49c9740 10 client.27770 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:14:22.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f93c49c9740 20 client.27770 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f93c49c9740 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5617a715fe70 0x5617a7162260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f93c49c9740 4 client.27770 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.526+0000 7f93c49c9740 1 -- 172.21.15.5:0/3004693754 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617a70b8860 con 0x5617a715fe70 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.527+0000 7f93bbfff640 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5617a715fe70 0x5617a7162260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.527+0000 7f93bbfff640 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5617a715fe70 0x5617a7162260 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1416eee00ade0302 server_cookie=46c086ab7891ff3f in_seq=0 out_seq=0 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.527+0000 7f939b7fe640 10 client.27770 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f939b7fe640 1 -- 172.21.15.5:0/3004693754 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617a70b8860 con 0x5617a715fe70 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f939b7fe640 10 client.27770 handle_command_reply: tid=0 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f93c49c9740 1 client.27770 shutdown 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f93c49c9740 1 -- 172.21.15.5:0/3004693754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 msgr2=0x5617a714cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f93c49c9740 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 0x5617a714cfd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009350 tx=0x7f93b0003820 comp rx=0 tx=0).stop 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f93c49c9740 1 -- 172.21.15.5:0/3004693754 shutdown_connections 2024-09-19T06:14:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f93c49c9740 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5617a715fe70 0x5617a7162260 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f93c49c9740 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7150370 0x5617a714d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f93c49c9740 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a7146b40 0x5617a714cfd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f93c49c9740 1 --2- 172.21.15.5:0/3004693754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a71455d0 0x5617a714fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f93c49c9740 1 -- 172.21.15.5:0/3004693754 >> 172.21.15.5:0/3004693754 conn(0x5617a710e310 msgr2=0x5617a7147930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f939b7fe640 0 client.27770 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:14:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f93c49c9740 1 -- 172.21.15.5:0/3004693754 shutdown_connections 2024-09-19T06:14:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f93c49c9740 1 -- 172.21.15.5:0/3004693754 wait complete. 2024-09-19T06:14:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.528+0000 7f93c49c9740 20 client.27770 trim_cache size 0 max 16384 2024-09-19T06:14:22.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.543+0000 7f93c49c9740 1 Processor -- start 2024-09-19T06:14:22.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.543+0000 7f93c49c9740 1 -- start start 2024-09-19T06:14:22.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93c49c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a71455d0 0x5617a715c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93c49c9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7146b40 0x5617a714c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93c49c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7150370 0x5617a714ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93c49c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1afac0 con 0x5617a7150370 2024-09-19T06:14:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93c49c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1b16c0 con 0x5617a71455d0 2024-09-19T06:14:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93c49c9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1b0190 con 0x5617a7146b40 2024-09-19T06:14:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93c0b8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7150370 0x5617a714ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93bbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a71455d0 0x5617a715c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93bbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a71455d0 0x5617a715c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58476/0 (socket says 172.21.15.5:58476) 2024-09-19T06:14:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93c0b8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7150370 0x5617a714ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38550/0 (socket says 172.21.15.5:38550) 2024-09-19T06:14:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93bbfff640 1 -- 172.21.15.5:0/1764711754 learned_addr learned my addr 172.21.15.5:0/1764711754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/1764711754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7146b40 0x5617a714c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93bbfff640 1 -- 172.21.15.5:0/1764711754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7146b40 msgr2=0x5617a714c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93bbfff640 1 --2- 172.21.15.5:0/1764711754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7146b40 0x5617a714c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93bbfff640 1 -- 172.21.15.5:0/1764711754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7150370 msgr2=0x5617a714ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:22.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93bbfff640 1 --2- 172.21.15.5:0/1764711754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7150370 0x5617a714ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93bbfff640 1 -- 172.21.15.5:0/1764711754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a7142960 con 0x5617a71455d0 2024-09-19T06:14:22.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93bbfff640 1 --2- 172.21.15.5:0/1764711754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a71455d0 0x5617a715c310 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0edd00 tx=0x7f93ac0dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a49cc251ccc29619 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:22.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.544+0000 7f93a1ffb640 1 -- 172.21.15.5:0/1764711754 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac049040 con 0x5617a71455d0 2024-09-19T06:14:22.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.545+0000 7f93a1ffb640 1 -- 172.21.15.5:0/1764711754 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93ac0ede50 con 0x5617a71455d0 2024-09-19T06:14:22.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.545+0000 7f93a1ffb640 1 -- 172.21.15.5:0/1764711754 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac003d60 con 0x5617a71455d0 2024-09-19T06:14:22.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.545+0000 7f93c49c9740 1 -- 172.21.15.5:0/1764711754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a71455d0 msgr2=0x5617a715c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:22.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.545+0000 7f93c49c9740 1 --2- 172.21.15.5:0/1764711754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a71455d0 0x5617a715c310 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0edd00 tx=0x7f93ac0dc040 comp rx=0 tx=0).stop 2024-09-19T06:14:22.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.545+0000 7f93c49c9740 1 -- 172.21.15.5:0/1764711754 shutdown_connections 2024-09-19T06:14:22.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.545+0000 7f93c49c9740 1 --2- 172.21.15.5:0/1764711754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7150370 0x5617a714ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.545+0000 7f93c49c9740 1 --2- 172.21.15.5:0/1764711754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7146b40 0x5617a714c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.545+0000 7f93c49c9740 1 --2- 172.21.15.5:0/1764711754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a71455d0 0x5617a715c310 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.545+0000 7f93c49c9740 1 -- 172.21.15.5:0/1764711754 >> 172.21.15.5:0/1764711754 conn(0x5617a710e310 msgr2=0x5617a714e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.545+0000 7f93c49c9740 1 -- 172.21.15.5:0/1764711754 shutdown_connections 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.546+0000 7f93c49c9740 1 -- 172.21.15.5:0/1764711754 wait complete. 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.546+0000 7f93c49c9740 1 -- start start 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.546+0000 7f93c49c9740 1 Processor -- start 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.546+0000 7f93c49c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a71455d0 0x5617a7153be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.546+0000 7f93c49c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7146b40 0x5617a7154120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.546+0000 7f93c49c9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7150370 0x5617a7158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.546+0000 7f93c49c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1b0ff0 con 0x5617a7146b40 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.546+0000 7f93c49c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1b08c0 con 0x5617a71455d0 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.546+0000 7f93c49c9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1adac0 con 0x5617a7150370 2024-09-19T06:14:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7146b40 0x5617a7154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93bbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a71455d0 0x5617a7153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.546+0000 7f93c49c9740 10 client.0 _ll_register_callbacks cb 0x7f93ac0f39f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7146b40 0x5617a7154120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38552/0 (socket says 172.21.15.5:38552) 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93bb7fe640 1 -- 172.21.15.5:0/1898499476 learned_addr learned my addr 172.21.15.5:0/1898499476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93c0b8d640 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7150370 0x5617a7158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93bb7fe640 1 -- 172.21.15.5:0/1898499476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7150370 msgr2=0x5617a7158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7150370 0x5617a7158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93bb7fe640 1 -- 172.21.15.5:0/1898499476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a71455d0 msgr2=0x5617a7153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a71455d0 0x5617a7153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93bb7fe640 1 -- 172.21.15.5:0/1898499476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a7158810 con 0x5617a7146b40 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93c0b8d640 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7150370 0x5617a7158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93bb7fe640 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7146b40 0x5617a7154120 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f93b0011910 tx=0x7f93b0003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176d12f3163a2cec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f939b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f939b7fe640 1 -- 172.21.15.5:0/1898499476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b001d070 con 0x5617a7146b40 2024-09-19T06:14:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f939b7fe640 1 -- 172.21.15.5:0/1898499476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f93b000f540 con 0x5617a7146b40 2024-09-19T06:14:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.547+0000 7f93c49c9740 1 -- 172.21.15.5:0/1898499476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93ac0ede50 con 0x5617a7146b40 2024-09-19T06:14:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.548+0000 7f939b7fe640 1 -- 172.21.15.5:0/1898499476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f93b000f760 con 0x5617a7146b40 2024-09-19T06:14:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.548+0000 7f93c49c9740 10 client.18282 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.548+0000 7f93c49c9740 1 -- 172.21.15.5:0/1898499476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5617a71ca4a0 con 0x5617a7146b40 2024-09-19T06:14:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.548+0000 7f939b7fe640 1 -- 172.21.15.5:0/1898499476 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93ac0ede50 con 0x5617a7146b40 2024-09-19T06:14:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.548+0000 7f939b7fe640 1 -- 172.21.15.5:0/1898499476 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f93b000fcb0 con 0x5617a7146b40 2024-09-19T06:14:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.548+0000 7f93c49c9740 10 client.18282 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.548+0000 7f93c49c9740 10 client.18282 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:14:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.548+0000 7f93c49c9740 20 client.18282 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.548+0000 7f93c49c9740 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5617a71caaa0 0x5617a71cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.548+0000 7f93c49c9740 4 client.18282 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.548+0000 7f93c49c9740 1 -- 172.21.15.5:0/1898499476 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617a70b2d70 con 0x5617a71caaa0 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f93bbfff640 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5617a71caaa0 0x5617a71cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f93bbfff640 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5617a71caaa0 0x5617a71cce90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e400bcfd48efd424 server_cookie=df2b26d8175d797d in_seq=0 out_seq=0 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f939b7fe640 10 client.18282 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f939b7fe640 1 -- 172.21.15.5:0/1898499476 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617a70b2d70 con 0x5617a71caaa0 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f939b7fe640 10 client.18282 handle_command_reply: tid=0 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f93c49c9740 1 client.18282 shutdown 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f93c49c9740 1 -- 172.21.15.5:0/1898499476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7146b40 msgr2=0x5617a7154120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f93c49c9740 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7146b40 0x5617a7154120 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f93b0011910 tx=0x7f93b0003890 comp rx=0 tx=0).stop 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f93c49c9740 1 -- 172.21.15.5:0/1898499476 shutdown_connections 2024-09-19T06:14:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f93c49c9740 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5617a71caaa0 0x5617a71cce90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f93c49c9740 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617a7150370 0x5617a7158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f93c49c9740 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a7146b40 0x5617a7154120 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f93c49c9740 1 --2- 172.21.15.5:0/1898499476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617a71455d0 0x5617a7153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f93c49c9740 1 -- 172.21.15.5:0/1898499476 >> 172.21.15.5:0/1898499476 conn(0x5617a710e310 msgr2=0x5617a714f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:22.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.549+0000 7f939b7fe640 0 client.18282 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:14:22.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c49c9740 1 -- 172.21.15.5:0/1898499476 shutdown_connections 2024-09-19T06:14:22.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c49c9740 1 -- 172.21.15.5:0/1898499476 wait complete. 2024-09-19T06:14:22.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c49c9740 20 client.18282 trim_cache size 0 max 16384 2024-09-19T06:14:22.557 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:14:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c25f0640 1 -- 172.21.15.5:0/1861586105 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f93a40a33b0 msgr2=0x7f93a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c25f0640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f93a40a33b0 0x7f93a40a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f93b4012640 tx=0x7f93b4009500 comp rx=0 tx=0).stop 2024-09-19T06:14:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c25f0640 1 -- 172.21.15.5:0/1861586105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93bc102d30 msgr2=0x7f93bc100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c25f0640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93bc102d30 0x7f93bc100990 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005940 tx=0x7f93ac0040c0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c25f0640 1 -- 172.21.15.5:0/1861586105 shutdown_connections 2024-09-19T06:14:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c25f0640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f93a40a33b0 0x7f93a40a5870 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c25f0640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc1082a0 0x7f93bc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c25f0640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93bc1036e0 0x7f93bc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c25f0640 1 --2- 172.21.15.5:0/1861586105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93bc102d30 0x7f93bc100990 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.550+0000 7f93c25f0640 1 -- 172.21.15.5:0/1861586105 >> 172.21.15.5:0/1861586105 conn(0x7f93bc0fc7d0 msgr2=0x7f93bc105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.551+0000 7f93c25f0640 1 -- 172.21.15.5:0/1861586105 shutdown_connections 2024-09-19T06:14:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:22.551+0000 7f93c25f0640 1 -- 172.21.15.5:0/1861586105 wait complete. 2024-09-19T06:14:22.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:22 smithi157 ceph-mon[35346]: pgmap v344: 97 pgs: 97 active+clean; 142 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 379 KiB/s wr, 47 op/s 2024-09-19T06:14:22.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 262584 2.31 MB/sec execute 468 sec latency 66.139 ms 2024-09-19T06:14:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:22 smithi005 ceph-mon[30955]: pgmap v344: 97 pgs: 97 active+clean; 142 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 379 KiB/s wr, 47 op/s 2024-09-19T06:14:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:22 smithi049 ceph-mon[34640]: pgmap v344: 97 pgs: 97 active+clean; 142 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 379 KiB/s wr, 47 op/s 2024-09-19T06:14:23.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:14:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:14:23.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 262584 2.31 MB/sec execute 469 sec latency 1022.874 ms 2024-09-19T06:14:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.095+0000 7fedd1b11640 1 -- 172.21.15.5:0/1682340557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0745a0 msgr2=0x7fedcc074980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.095+0000 7fedd1b11640 1 --2- 172.21.15.5:0/1682340557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0745a0 0x7fedcc074980 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009900 tx=0x7fedbc02a410 comp rx=0 tx=0).stop 2024-09-19T06:14:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.095+0000 7fedd1b11640 1 -- 172.21.15.5:0/1682340557 shutdown_connections 2024-09-19T06:14:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.095+0000 7fedd1b11640 1 --2- 172.21.15.5:0/1682340557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedcc10e470 0x7fedcc1108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.095+0000 7fedd1b11640 1 --2- 172.21.15.5:0/1682340557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedcc074f50 0x7fedcc10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.095+0000 7fedd1b11640 1 --2- 172.21.15.5:0/1682340557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0745a0 0x7fedcc074980 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.095+0000 7fedd1b11640 1 -- 172.21.15.5:0/1682340557 >> 172.21.15.5:0/1682340557 conn(0x7fedcc0fd8d0 msgr2=0x7fedcc0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.095+0000 7fedd1b11640 1 -- 172.21.15.5:0/1682340557 shutdown_connections 2024-09-19T06:14:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.095+0000 7fedd1b11640 1 -- 172.21.15.5:0/1682340557 wait complete. 2024-09-19T06:14:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.096+0000 7fedd1b11640 1 Processor -- start 2024-09-19T06:14:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.096+0000 7fedd1b11640 1 -- start start 2024-09-19T06:14:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.096+0000 7fedd1b11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedcc0745a0 0x7fedcc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.096+0000 7fedd1b11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc074f50 0x7fedcc19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.096+0000 7fedd1b11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedcc10e470 0x7fedcc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.096+0000 7fedd1b11640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc114090 con 0x7fedcc074f50 2024-09-19T06:14:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.096+0000 7fedd1b11640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc113f10 con 0x7fedcc10e470 2024-09-19T06:14:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.096+0000 7fedd1b11640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc114210 con 0x7fedcc0745a0 2024-09-19T06:14:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc074f50 0x7fedcc19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc074f50 0x7fedcc19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38564/0 (socket says 172.21.15.5:38564) 2024-09-19T06:14:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcaffd640 1 -- 172.21.15.5:0/123917208 learned_addr learned my addr 172.21.15.5:0/123917208 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedcc0745a0 0x7fedcc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcbfff640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedcc10e470 0x7fedcc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcaffd640 1 -- 172.21.15.5:0/123917208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedcc0745a0 msgr2=0x7fedcc19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcaffd640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedcc0745a0 0x7fedcc19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcaffd640 1 -- 172.21.15.5:0/123917208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedcc10e470 msgr2=0x7fedcc19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcaffd640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedcc10e470 0x7fedcc19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcaffd640 1 -- 172.21.15.5:0/123917208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedcc19f490 con 0x7fedcc074f50 2024-09-19T06:14:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedcc0745a0 0x7fedcc19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcbfff640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedcc10e470 0x7fedcc19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedcaffd640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc074f50 0x7fedcc19aa20 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fedc000c920 tx=0x7fedc000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedc8ff9640 1 -- 172.21.15.5:0/123917208 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0007c70 con 0x7fedcc074f50 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedc8ff9640 1 -- 172.21.15.5:0/123917208 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedc0007e10 con 0x7fedcc074f50 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedd1b11640 1 -- 172.21.15.5:0/123917208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedcc19f780 con 0x7fedcc074f50 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedc8ff9640 1 -- 172.21.15.5:0/123917208 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0018820 con 0x7fedcc074f50 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.097+0000 7fedd1b11640 1 -- 172.21.15.5:0/123917208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedcc1a7060 con 0x7fedcc074f50 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedc8ff9640 1 -- 172.21.15.5:0/123917208 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fedc00189c0 con 0x7fedcc074f50 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedd3eea740 1 Processor -- start 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedd3eea740 1 -- start start 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedd3eea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf371b40 0x55d1cf370090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedd3eea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf3705d0 0x55d1cf370a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedd3eea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf37b370 0x55d1cf37d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedd3eea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1cf2d62f0 con 0x55d1cf3705d0 2024-09-19T06:14:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedd3eea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d1cf27dcf0 con 0x55d1cf371b40 2024-09-19T06:14:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedd3eea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d1cf2d22b0 con 0x55d1cf37b370 2024-09-19T06:14:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedcb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf3705d0 0x55d1cf370a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedcb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf3705d0 0x55d1cf370a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38566/0 (socket says 172.21.15.5:38566) 2024-09-19T06:14:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedcb7fe640 1 -- 172.21.15.5:0/726364349 learned_addr learned my addr 172.21.15.5:0/726364349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedcbfff640 1 --2- 172.21.15.5:0/726364349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf371b40 0x55d1cf370090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.100+0000 7fedcaffd640 1 --2- 172.21.15.5:0/726364349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf37b370 0x55d1cf37d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedc8ff9640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fedb40a33b0 0x7fedb40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedcb7fe640 1 -- 172.21.15.5:0/726364349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf37b370 msgr2=0x55d1cf37d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/726364349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf37b370 0x55d1cf37d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedcb7fe640 1 -- 172.21.15.5:0/726364349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf371b40 msgr2=0x55d1cf370090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/726364349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf371b40 0x55d1cf370090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedcb7fe640 1 -- 172.21.15.5:0/726364349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cf2623a0 con 0x55d1cf3705d0 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedcbfff640 1 --2- 172.21.15.5:0/726364349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf371b40 0x55d1cf370090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedcbfff640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fedb40a33b0 0x7fedb40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedc8ff9640 1 -- 172.21.15.5:0/123917208 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fedc00cff60 con 0x7fedcc074f50 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/726364349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf3705d0 0x55d1cf370a10 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fedbc02a930 tx=0x7fedbc032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f720dde744e481 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedb1ffb640 1 -- 172.21.15.5:0/726364349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc03f070 con 0x55d1cf3705d0 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedcbfff640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fedb40a33b0 0x7fedb40a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009fd0 tx=0x7fedc4004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedb1ffb640 1 -- 172.21.15.5:0/726364349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedbc0316a0 con 0x55d1cf3705d0 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.101+0000 7fedb1ffb640 1 -- 172.21.15.5:0/726364349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc035070 con 0x55d1cf3705d0 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.102+0000 7fedd3eea740 1 -- 172.21.15.5:0/726364349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf3705d0 msgr2=0x55d1cf370a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.102+0000 7fedd3eea740 1 --2- 172.21.15.5:0/726364349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf3705d0 0x55d1cf370a10 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fedbc02a930 tx=0x7fedbc032d60 comp rx=0 tx=0).stop 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.102+0000 7fedd3eea740 1 -- 172.21.15.5:0/726364349 shutdown_connections 2024-09-19T06:14:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.102+0000 7fedd3eea740 1 --2- 172.21.15.5:0/726364349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf37b370 0x55d1cf37d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.102+0000 7fedd3eea740 1 --2- 172.21.15.5:0/726364349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf3705d0 0x55d1cf370a10 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.102+0000 7fedd3eea740 1 --2- 172.21.15.5:0/726364349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf371b40 0x55d1cf370090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.102+0000 7fedd3eea740 1 -- 172.21.15.5:0/726364349 >> 172.21.15.5:0/726364349 conn(0x55d1cf339310 msgr2=0x55d1cf36cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.102+0000 7fedd3eea740 1 -- 172.21.15.5:0/726364349 shutdown_connections 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.102+0000 7fedd3eea740 1 -- 172.21.15.5:0/726364349 wait complete. 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.102+0000 7fedd3eea740 1 -- start start 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedd3eea740 1 Processor -- start 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedd3eea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf3705d0 0x55d1cf37ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedd3eea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf371b40 0x55d1cf378030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedd3eea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 0x55d1cf378640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedd3eea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1cf37e010 con 0x55d1cf37b370 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedd3eea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d1cf291060 con 0x55d1cf3705d0 2024-09-19T06:14:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedd3eea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d1cf37e190 con 0x55d1cf371b40 2024-09-19T06:14:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedd3eea740 10 client.0 _ll_register_callbacks cb 0x55d1cefe48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 0x55d1cf378640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 0x55d1cf378640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38582/0 (socket says 172.21.15.5:38582) 2024-09-19T06:14:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedcbfff640 1 -- 172.21.15.5:0/909456384 learned_addr learned my addr 172.21.15.5:0/909456384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf3705d0 0x55d1cf37ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedcaffd640 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf371b40 0x55d1cf378030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedcbfff640 1 -- 172.21.15.5:0/909456384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf371b40 msgr2=0x55d1cf378030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedcbfff640 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf371b40 0x55d1cf378030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedcbfff640 1 -- 172.21.15.5:0/909456384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf3705d0 msgr2=0x55d1cf37ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedcbfff640 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf3705d0 0x55d1cf37ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedcbfff640 1 -- 172.21.15.5:0/909456384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cf0473c0 con 0x55d1cf37b370 2024-09-19T06:14:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.103+0000 7fedcaffd640 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf371b40 0x55d1cf378030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedcbfff640 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 0x55d1cf378640 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fedc4013b50 tx=0x7fedc4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ee9bfd90cc79fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedaaffd640 1 -- 172.21.15.5:0/909456384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc401e420 con 0x55d1cf37b370 2024-09-19T06:14:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedaaffd640 1 -- 172.21.15.5:0/909456384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fedc401e5c0 con 0x55d1cf37b370 2024-09-19T06:14:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedd3eea740 1 -- 172.21.15.5:0/909456384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedbc0316a0 con 0x55d1cf37b370 2024-09-19T06:14:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedaaffd640 1 -- 172.21.15.5:0/909456384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fedc401e7c0 con 0x55d1cf37b370 2024-09-19T06:14:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedd3eea740 10 client.18294 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedd3eea740 1 -- 172.21.15.5:0/909456384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d1cf389b40 con 0x55d1cf37b370 2024-09-19T06:14:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedaaffd640 1 -- 172.21.15.5:0/909456384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedbc0316a0 con 0x55d1cf37b370 2024-09-19T06:14:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedaaffd640 1 -- 172.21.15.5:0/909456384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fedc40289f0 con 0x55d1cf37b370 2024-09-19T06:14:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedd3eea740 10 client.18294 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedd3eea740 10 client.18294 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedd3eea740 20 client.18294 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedd3eea740 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d1cf38b000 0x55d1cf38d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedd3eea740 4 client.18294 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.104+0000 7fedd3eea740 1 -- 172.21.15.5:0/909456384 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1cf2e3860 con 0x55d1cf38b000 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.105+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d1cf38b000 0x55d1cf38d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.105+0000 7fedaaffd640 10 client.18294 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.105+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d1cf38b000 0x55d1cf38d470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=17637115a59d4c01 server_cookie=917c7110bbd6c2b1 in_seq=0 out_seq=0 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedaaffd640 1 -- 172.21.15.5:0/909456384 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1cf2e3860 con 0x55d1cf38b000 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedaaffd640 10 client.18294 handle_command_reply: tid=0 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedd3eea740 1 client.18294 shutdown 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedd3eea740 1 -- 172.21.15.5:0/909456384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 msgr2=0x55d1cf378640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedd3eea740 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 0x55d1cf378640 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fedc4013b50 tx=0x7fedc4004ab0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedd3eea740 1 -- 172.21.15.5:0/909456384 shutdown_connections 2024-09-19T06:14:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedd3eea740 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d1cf38b000 0x55d1cf38d470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedd3eea740 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 0x55d1cf378640 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedd3eea740 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf371b40 0x55d1cf378030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedd3eea740 1 --2- 172.21.15.5:0/909456384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf3705d0 0x55d1cf37ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedd3eea740 1 -- 172.21.15.5:0/909456384 >> 172.21.15.5:0/909456384 conn(0x55d1cf339310 msgr2=0x55d1cf372960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedaaffd640 0 client.18294 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:14:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedd3eea740 1 -- 172.21.15.5:0/909456384 shutdown_connections 2024-09-19T06:14:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedd3eea740 1 -- 172.21.15.5:0/909456384 wait complete. 2024-09-19T06:14:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.106+0000 7fedd3eea740 20 client.18294 trim_cache size 0 max 16384 2024-09-19T06:14:24.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.119+0000 7fedd3eea740 1 Processor -- start 2024-09-19T06:14:24.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.119+0000 7fedd3eea740 1 -- start start 2024-09-19T06:14:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.119+0000 7fedd3eea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf3705d0 0x55d1cf36ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.119+0000 7fedd3eea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf371b40 0x55d1cf3872b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.119+0000 7fedd3eea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 0x55d1cf373120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.119+0000 7fedd3eea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc1af8a0 con 0x55d1cf37b370 2024-09-19T06:14:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.119+0000 7fedd3eea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc1b14a0 con 0x55d1cf3705d0 2024-09-19T06:14:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.119+0000 7fedd3eea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc1aff70 con 0x55d1cf371b40 2024-09-19T06:14:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 0x55d1cf373120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 0x55d1cf373120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38584/0 (socket says 172.21.15.5:38584) 2024-09-19T06:14:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedcbfff640 1 -- 172.21.15.5:0/1118877496 learned_addr learned my addr 172.21.15.5:0/1118877496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedcaffd640 1 --2- 172.21.15.5:0/1118877496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf371b40 0x55d1cf3872b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/1118877496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf3705d0 0x55d1cf36ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedcbfff640 1 -- 172.21.15.5:0/1118877496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf371b40 msgr2=0x55d1cf3872b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedcbfff640 1 --2- 172.21.15.5:0/1118877496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf371b40 0x55d1cf3872b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedcbfff640 1 -- 172.21.15.5:0/1118877496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf3705d0 msgr2=0x55d1cf36ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedcbfff640 1 --2- 172.21.15.5:0/1118877496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf3705d0 0x55d1cf36ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedcbfff640 1 -- 172.21.15.5:0/1118877496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cf37b0b0 con 0x55d1cf37b370 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedcaffd640 1 --2- 172.21.15.5:0/1118877496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf371b40 0x55d1cf3872b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedcbfff640 1 --2- 172.21.15.5:0/1118877496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 0x55d1cf373120 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fedc4028e30 tx=0x7fedc401f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9985d1f3db20f58d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedb17fa640 1 -- 172.21.15.5:0/1118877496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4034040 con 0x55d1cf37b370 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedb17fa640 1 -- 172.21.15.5:0/1118877496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedc4023030 con 0x55d1cf37b370 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.120+0000 7fedb17fa640 1 -- 172.21.15.5:0/1118877496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc401f720 con 0x55d1cf37b370 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.121+0000 7fedd3eea740 1 -- 172.21.15.5:0/1118877496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 msgr2=0x55d1cf373120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.121+0000 7fedd3eea740 1 --2- 172.21.15.5:0/1118877496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 0x55d1cf373120 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fedc4028e30 tx=0x7fedc401f630 comp rx=0 tx=0).stop 2024-09-19T06:14:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.121+0000 7fedd3eea740 1 -- 172.21.15.5:0/1118877496 shutdown_connections 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.121+0000 7fedd3eea740 1 --2- 172.21.15.5:0/1118877496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf37b370 0x55d1cf373120 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.121+0000 7fedd3eea740 1 --2- 172.21.15.5:0/1118877496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf371b40 0x55d1cf3872b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.121+0000 7fedd3eea740 1 --2- 172.21.15.5:0/1118877496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf3705d0 0x55d1cf36ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.121+0000 7fedd3eea740 1 -- 172.21.15.5:0/1118877496 >> 172.21.15.5:0/1118877496 conn(0x55d1cf339310 msgr2=0x55d1cf36c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.121+0000 7fedd3eea740 1 -- 172.21.15.5:0/1118877496 shutdown_connections 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.121+0000 7fedd3eea740 1 -- 172.21.15.5:0/1118877496 wait complete. 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.121+0000 7fedd3eea740 1 -- start start 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedd3eea740 1 Processor -- start 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedd3eea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf3705d0 0x55d1cf383640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedd3eea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf371b40 0x55d1cf383b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedd3eea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf37b370 0x55d1cf3748e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedd3eea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc1b0dd0 con 0x55d1cf371b40 2024-09-19T06:14:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedd3eea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc1b06a0 con 0x55d1cf37b370 2024-09-19T06:14:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedd3eea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc1ad8a0 con 0x55d1cf3705d0 2024-09-19T06:14:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedd3eea740 10 client.0 _ll_register_callbacks cb 0x7fedc4009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedcaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf371b40 0x55d1cf383b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedcaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf371b40 0x55d1cf383b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38590/0 (socket says 172.21.15.5:38590) 2024-09-19T06:14:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedcaffd640 1 -- 172.21.15.5:0/3280865759 learned_addr learned my addr 172.21.15.5:0/3280865759 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedcbfff640 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf37b370 0x55d1cf3748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf3705d0 0x55d1cf383640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedcaffd640 1 -- 172.21.15.5:0/3280865759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf3705d0 msgr2=0x55d1cf383640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedcaffd640 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf3705d0 0x55d1cf383640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedcaffd640 1 -- 172.21.15.5:0/3280865759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf37b370 msgr2=0x55d1cf3748e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedcaffd640 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf37b370 0x55d1cf3748e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedcaffd640 1 -- 172.21.15.5:0/3280865759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cf3750c0 con 0x55d1cf371b40 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedcbfff640 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf37b370 0x55d1cf3748e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.122+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf3705d0 0x55d1cf383640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedcaffd640 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf371b40 0x55d1cf383b80 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fedc0098680 tx=0x7fedc009bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b7d876eeead7f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedaaffd640 1 -- 172.21.15.5:0/3280865759 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00bae60 con 0x55d1cf371b40 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedaaffd640 1 -- 172.21.15.5:0/3280865759 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fedc00cced0 con 0x55d1cf371b40 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedd3eea740 1 -- 172.21.15.5:0/3280865759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc4023030 con 0x55d1cf371b40 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedaaffd640 1 -- 172.21.15.5:0/3280865759 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fedc0018cb0 con 0x55d1cf371b40 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedd3eea740 10 client.18300 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedd3eea740 1 -- 172.21.15.5:0/3280865759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d1cf3f5890 con 0x55d1cf371b40 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedaaffd640 1 -- 172.21.15.5:0/3280865759 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedc000fb00 con 0x55d1cf371b40 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedaaffd640 1 -- 172.21.15.5:0/3280865759 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fedc00cd250 con 0x55d1cf371b40 2024-09-19T06:14:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedd3eea740 10 client.18300 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedd3eea740 10 client.18300 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedd3eea740 20 client.18300 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedd3eea740 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d1cf3f5f90 0x55d1cf3f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedd3eea740 4 client.18300 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.123+0000 7fedd3eea740 1 -- 172.21.15.5:0/3280865759 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fedc4023030 con 0x55d1cf3f5f90 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.124+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d1cf3f5f90 0x55d1cf3f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.124+0000 7fedcb7fe640 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d1cf3f5f90 0x55d1cf3f83b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3c079ea2c06204fe server_cookie=220b398f15b3768e in_seq=0 out_seq=0 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.124+0000 7fedaaffd640 10 client.18300 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.124+0000 7fedaaffd640 1 -- 172.21.15.5:0/3280865759 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fedc4023030 con 0x55d1cf3f5f90 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.124+0000 7fedaaffd640 10 client.18300 handle_command_reply: tid=0 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.124+0000 7fedd3eea740 1 client.18300 shutdown 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd3eea740 1 -- 172.21.15.5:0/3280865759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf371b40 msgr2=0x55d1cf383b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd3eea740 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf371b40 0x55d1cf383b80 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fedc0098680 tx=0x7fedc009bff0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd3eea740 1 -- 172.21.15.5:0/3280865759 shutdown_connections 2024-09-19T06:14:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd3eea740 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55d1cf3f5f90 0x55d1cf3f83b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd3eea740 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1cf37b370 0x55d1cf3748e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd3eea740 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1cf371b40 0x55d1cf383b80 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd3eea740 1 --2- 172.21.15.5:0/3280865759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cf3705d0 0x55d1cf383640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedaaffd640 0 client.18300 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:14:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd3eea740 1 -- 172.21.15.5:0/3280865759 >> 172.21.15.5:0/3280865759 conn(0x55d1cf339310 msgr2=0x55d1cf379890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd3eea740 1 -- 172.21.15.5:0/3280865759 shutdown_connections 2024-09-19T06:14:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd3eea740 1 -- 172.21.15.5:0/3280865759 wait complete. 2024-09-19T06:14:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd3eea740 20 client.18300 trim_cache size 0 max 16384 2024-09-19T06:14:24.136 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:14:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd1b11640 1 -- 172.21.15.5:0/123917208 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fedb40a33b0 msgr2=0x7fedb40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd1b11640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fedb40a33b0 0x7fedb40a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009fd0 tx=0x7fedc4004a40 comp rx=0 tx=0).stop 2024-09-19T06:14:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd1b11640 1 -- 172.21.15.5:0/123917208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc074f50 msgr2=0x7fedcc19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd1b11640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc074f50 0x7fedcc19aa20 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fedc000c920 tx=0x7fedc000cde0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd1b11640 1 -- 172.21.15.5:0/123917208 shutdown_connections 2024-09-19T06:14:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd1b11640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedcc10e470 0x7fedcc19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd1b11640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc074f50 0x7fedcc19aa20 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd1b11640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fedb40a33b0 0x7fedb40a5870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd1b11640 1 --2- 172.21.15.5:0/123917208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedcc0745a0 0x7fedcc19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.125+0000 7fedd1b11640 1 -- 172.21.15.5:0/123917208 >> 172.21.15.5:0/123917208 conn(0x7fedcc0fd8d0 msgr2=0x7fedcc0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:24.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.126+0000 7fedd1b11640 1 -- 172.21.15.5:0/123917208 shutdown_connections 2024-09-19T06:14:24.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:24.126+0000 7fedd1b11640 1 -- 172.21.15.5:0/123917208 wait complete. 2024-09-19T06:14:24.698 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:24 smithi049 ceph-mon[34640]: pgmap v345: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-09-19T06:14:24.767 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:24 smithi005 ceph-mon[30955]: pgmap v345: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-09-19T06:14:24.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:14:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:24 smithi157 ceph-mon[35346]: pgmap v345: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-09-19T06:14:24.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 262584 2.30 MB/sec execute 470 sec latency 2022.901 ms 2024-09-19T06:14:25.057 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:14:25.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.547+0000 7f85a6dbd640 1 -- 172.21.15.5:0/1784127986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0102570 msgr2=0x7f85a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:25.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.547+0000 7f85a6dbd640 1 --2- 172.21.15.5:0/1784127986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0102570 0x7f85a01029b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8594009900 tx=0x7f859402a480 comp rx=0 tx=0).stop 2024-09-19T06:14:25.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.547+0000 7f85a6dbd640 1 -- 172.21.15.5:0/1784127986 shutdown_connections 2024-09-19T06:14:25.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.547+0000 7f85a6dbd640 1 --2- 172.21.15.5:0/1784127986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a0102ef0 0x7f85a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.547+0000 7f85a6dbd640 1 --2- 172.21.15.5:0/1784127986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0102570 0x7f85a01029b0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.547+0000 7f85a6dbd640 1 --2- 172.21.15.5:0/1784127986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a0100e20 0x7f85a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.547+0000 7f85a6dbd640 1 -- 172.21.15.5:0/1784127986 >> 172.21.15.5:0/1784127986 conn(0x7f85a00fc7d0 msgr2=0x7f85a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:25.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.547+0000 7f85a6dbd640 1 -- 172.21.15.5:0/1784127986 shutdown_connections 2024-09-19T06:14:25.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.547+0000 7f85a6dbd640 1 -- 172.21.15.5:0/1784127986 wait complete. 2024-09-19T06:14:25.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f85a6dbd640 1 Processor -- start 2024-09-19T06:14:25.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f85a6dbd640 1 -- start start 2024-09-19T06:14:25.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f85a6dbd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a0100e20 0x7f85a019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f85a6dbd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a0102570 0x7f85a019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f85a6dbd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0102ef0 0x7f85a019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f85a6dbd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0110e30 con 0x7f85a0102ef0 2024-09-19T06:14:25.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f85a6dbd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0110cb0 con 0x7f85a0100e20 2024-09-19T06:14:25.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f85a6dbd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0110fb0 con 0x7f85a0102570 2024-09-19T06:14:25.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f85a535a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0102ef0 0x7f85a019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f85a535a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0102ef0 0x7f85a019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41688/0 (socket says 172.21.15.5:41688) 2024-09-19T06:14:25.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f85a535a640 1 -- 172.21.15.5:0/2550126283 learned_addr learned my addr 172.21.15.5:0/2550126283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:25.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f85a4b59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a0100e20 0x7f85a019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.548+0000 7f859ffff640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a0102570 0x7f85a019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.549+0000 7f85a535a640 1 -- 172.21.15.5:0/2550126283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a0102570 msgr2=0x7f85a019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:25.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.549+0000 7f85a535a640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a0102570 0x7f85a019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.549+0000 7f85a535a640 1 -- 172.21.15.5:0/2550126283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a0100e20 msgr2=0x7f85a019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:25.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.549+0000 7f85a535a640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a0100e20 0x7f85a019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.549+0000 7f85a535a640 1 -- 172.21.15.5:0/2550126283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a019f5a0 con 0x7f85a0102ef0 2024-09-19T06:14:25.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.549+0000 7f85a4b59640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a0100e20 0x7f85a019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:25.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.549+0000 7f85a535a640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0102ef0 0x7f85a019eec0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f859800eb10 tx=0x7f859800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:25.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.549+0000 7f859dffb640 1 -- 172.21.15.5:0/2550126283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598004b20 con 0x7f85a0102ef0 2024-09-19T06:14:25.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.549+0000 7f859dffb640 1 -- 172.21.15.5:0/2550126283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8598004cc0 con 0x7f85a0102ef0 2024-09-19T06:14:25.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.549+0000 7f85a6dbd640 1 -- 172.21.15.5:0/2550126283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a019f890 con 0x7f85a0102ef0 2024-09-19T06:14:25.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.549+0000 7f859dffb640 1 -- 172.21.15.5:0/2550126283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598010690 con 0x7f85a0102ef0 2024-09-19T06:14:25.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.549+0000 7f85a6dbd640 1 -- 172.21.15.5:0/2550126283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a01a7170 con 0x7f85a0102ef0 2024-09-19T06:14:25.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.552+0000 7f859dffb640 1 -- 172.21.15.5:0/2550126283 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8598005020 con 0x7f85a0102ef0 2024-09-19T06:14:25.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.552+0000 7f85a9196740 1 Processor -- start 2024-09-19T06:14:25.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a9196740 1 -- start start 2024-09-19T06:14:25.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f859dffb640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f85880a3430 0x7f85880a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a9196740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e7a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a9196740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 0x55abd2e7aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a9196740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e85370 0x55abd2e87760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a9196740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55abd2de02f0 con 0x55abd2e7a5d0 2024-09-19T06:14:25.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a9196740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55abd2d87cf0 con 0x55abd2e85370 2024-09-19T06:14:25.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a9196740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55abd2ddc2b0 con 0x55abd2e7bb40 2024-09-19T06:14:25.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f859dffb640 1 -- 172.21.15.5:0/2550126283 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f85980d0380 con 0x7f85a0102ef0 2024-09-19T06:14:25.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a4b59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e7a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a4b59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e7a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:58942/0 (socket says 172.21.15.5:58942) 2024-09-19T06:14:25.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a4b59640 1 -- 172.21.15.5:0/967064513 learned_addr learned my addr 172.21.15.5:0/967064513 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:25.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f859ffff640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f85880a3430 0x7f85880a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a535a640 1 --2- 172.21.15.5:0/967064513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e85370 0x55abd2e87760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f859ffff640 1 --2- 172.21.15.5:0/967064513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 0x55abd2e7aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a4b59640 1 -- 172.21.15.5:0/967064513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e85370 msgr2=0x55abd2e87760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:25.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a4b59640 1 --2- 172.21.15.5:0/967064513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e85370 0x55abd2e87760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.553+0000 7f85a4b59640 1 -- 172.21.15.5:0/967064513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 msgr2=0x55abd2e7aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:25.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.554+0000 7f85a4b59640 1 --2- 172.21.15.5:0/967064513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 0x55abd2e7aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.554+0000 7f85a4b59640 1 -- 172.21.15.5:0/967064513 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd2d6c3a0 con 0x55abd2e7bb40 2024-09-19T06:14:25.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.554+0000 7f859ffff640 1 --2- 172.21.15.5:0/967064513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 0x55abd2e7aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:14:25.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.554+0000 7f85a4b59640 1 --2- 172.21.15.5:0/967064513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e7a090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f85900115f0 tx=0x7f8590011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb970337004fe71a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:25.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.554+0000 7f859ffff640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f85880a3430 0x7f85880a58f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8594009230 tx=0x7f8594000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:25.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.554+0000 7f8586ffd640 1 -- 172.21.15.5:0/967064513 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859001e070 con 0x55abd2e7bb40 2024-09-19T06:14:25.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.554+0000 7f8586ffd640 1 -- 172.21.15.5:0/967064513 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f859000faf0 con 0x55abd2e7bb40 2024-09-19T06:14:25.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.554+0000 7f8586ffd640 1 -- 172.21.15.5:0/967064513 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590019ac0 con 0x55abd2e7bb40 2024-09-19T06:14:25.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.555+0000 7f85a9196740 1 -- 172.21.15.5:0/967064513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 msgr2=0x55abd2e7a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:25.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.555+0000 7f85a9196740 1 --2- 172.21.15.5:0/967064513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e7a090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f85900115f0 tx=0x7f8590011ab0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.555+0000 7f85a9196740 1 -- 172.21.15.5:0/967064513 shutdown_connections 2024-09-19T06:14:25.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.555+0000 7f85a9196740 1 --2- 172.21.15.5:0/967064513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e85370 0x55abd2e87760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.555+0000 7f85a9196740 1 --2- 172.21.15.5:0/967064513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 0x55abd2e7aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.555+0000 7f85a9196740 1 --2- 172.21.15.5:0/967064513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e7a090 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.555+0000 7f85a9196740 1 -- 172.21.15.5:0/967064513 >> 172.21.15.5:0/967064513 conn(0x55abd2e43310 msgr2=0x55abd2e76c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:25.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.555+0000 7f85a9196740 1 -- 172.21.15.5:0/967064513 shutdown_connections 2024-09-19T06:14:25.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.555+0000 7f85a9196740 1 -- 172.21.15.5:0/967064513 wait complete. 2024-09-19T06:14:25.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f85a9196740 1 -- start start 2024-09-19T06:14:25.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f85a9196740 1 Processor -- start 2024-09-19T06:14:25.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f85a9196740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7a5d0 0x55abd2e84d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f85a9196740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e82000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f85a9196740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 0x55abd2e825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f85a9196740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55abd2e88070 con 0x55abd2e85370 2024-09-19T06:14:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f85a9196740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55abd2d9b060 con 0x55abd2e7a5d0 2024-09-19T06:14:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f85a9196740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55abd2e881f0 con 0x55abd2e7bb40 2024-09-19T06:14:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f85a9196740 10 client.0 _ll_register_callbacks cb 0x55abd2aee8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f859ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e82000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f859ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e82000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:58954/0 (socket says 172.21.15.5:58954) 2024-09-19T06:14:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f85a535a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 0x55abd2e825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f859ffff640 1 -- 172.21.15.5:0/275522159 learned_addr learned my addr 172.21.15.5:0/275522159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.556+0000 7f85a4b59640 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7a5d0 0x55abd2e84d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.557+0000 7f85a4b59640 1 -- 172.21.15.5:0/275522159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 msgr2=0x55abd2e82000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.557+0000 7f85a4b59640 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e82000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.557+0000 7f85a4b59640 1 -- 172.21.15.5:0/275522159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 msgr2=0x55abd2e825f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.557+0000 7f85a4b59640 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 0x55abd2e825f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.557+0000 7f85a4b59640 1 -- 172.21.15.5:0/275522159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd2b513c0 con 0x55abd2e7a5d0 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.557+0000 7f85a535a640 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 0x55abd2e825f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.557+0000 7f859ffff640 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e82000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.557+0000 7f85a4b59640 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7a5d0 0x55abd2e84d00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f85900115c0 tx=0x7f859000f870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf3d8e0d677f5d78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.557+0000 7f857ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.557+0000 7f857ffff640 1 -- 172.21.15.5:0/275522159 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859001e040 con 0x55abd2e7a5d0 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.557+0000 7f85a9196740 1 -- 172.21.15.5:0/275522159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f859000faf0 con 0x55abd2e7a5d0 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.557+0000 7f857ffff640 1 -- 172.21.15.5:0/275522159 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f8590004040 con 0x55abd2e7a5d0 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.558+0000 7f857ffff640 1 -- 172.21.15.5:0/275522159 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8590004240 con 0x55abd2e7a5d0 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.558+0000 7f85a9196740 10 client.27806 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.558+0000 7f85a9196740 1 -- 172.21.15.5:0/275522159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55abd2e93a50 con 0x55abd2e7a5d0 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.558+0000 7f857ffff640 1 -- 172.21.15.5:0/275522159 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f859000faf0 con 0x55abd2e7a5d0 2024-09-19T06:14:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.558+0000 7f857ffff640 1 -- 172.21.15.5:0/275522159 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f85900035f0 con 0x55abd2e7a5d0 2024-09-19T06:14:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.558+0000 7f85a9196740 10 client.27806 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.558+0000 7f85a9196740 10 client.27806 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:14:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.558+0000 7f85a9196740 20 client.27806 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.558+0000 7f85a9196740 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55abd2e94e00 0x55abd2e971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.558+0000 7f85a9196740 4 client.27806 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.558+0000 7f85a9196740 1 -- 172.21.15.5:0/275522159 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd2ded860 con 0x55abd2e94e00 2024-09-19T06:14:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.558+0000 7f859ffff640 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55abd2e94e00 0x55abd2e971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.559+0000 7f859ffff640 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55abd2e94e00 0x55abd2e971f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4ad23cf53635598 server_cookie=5292849058319e91 in_seq=0 out_seq=0 2024-09-19T06:14:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.559+0000 7f857ffff640 10 client.27806 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:14:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f857ffff640 1 -- 172.21.15.5:0/275522159 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abd2ded860 con 0x55abd2e94e00 2024-09-19T06:14:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f857ffff640 10 client.27806 handle_command_reply: tid=0 2024-09-19T06:14:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f85a9196740 1 client.27806 shutdown 2024-09-19T06:14:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f85a9196740 1 -- 172.21.15.5:0/275522159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7a5d0 msgr2=0x55abd2e84d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f85a9196740 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7a5d0 0x55abd2e84d00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f85900115c0 tx=0x7f859000f870 comp rx=0 tx=0).stop 2024-09-19T06:14:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f85a9196740 1 -- 172.21.15.5:0/275522159 shutdown_connections 2024-09-19T06:14:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f85a9196740 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55abd2e94e00 0x55abd2e971f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f85a9196740 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 0x55abd2e825f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f85a9196740 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e82000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f85a9196740 1 --2- 172.21.15.5:0/275522159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7a5d0 0x55abd2e84d00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f857ffff640 0 client.27806 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:14:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f85a9196740 1 -- 172.21.15.5:0/275522159 >> 172.21.15.5:0/275522159 conn(0x55abd2e43310 msgr2=0x55abd2e7c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f85a9196740 1 -- 172.21.15.5:0/275522159 shutdown_connections 2024-09-19T06:14:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f85a9196740 1 -- 172.21.15.5:0/275522159 wait complete. 2024-09-19T06:14:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.560+0000 7f85a9196740 20 client.27806 trim_cache size 0 max 16384 2024-09-19T06:14:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.575+0000 7f85a9196740 1 Processor -- start 2024-09-19T06:14:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.575+0000 7f85a9196740 1 -- start start 2024-09-19T06:14:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.575+0000 7f85a9196740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 0x55abd2e7d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.575+0000 7f85a9196740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e7d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.575+0000 7f85a9196740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e85370 0x55abd2e9c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.575+0000 7f85a9196740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01afa30 con 0x55abd2e7a5d0 2024-09-19T06:14:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.575+0000 7f85a9196740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01b1630 con 0x55abd2e85370 2024-09-19T06:14:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.575+0000 7f85a9196740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01b0100 con 0x55abd2e7bb40 2024-09-19T06:14:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.575+0000 7f85a4b59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 0x55abd2e7d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.575+0000 7f85a4b59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 0x55abd2e7d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41708/0 (socket says 172.21.15.5:41708) 2024-09-19T06:14:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.575+0000 7f85a4b59640 1 -- 172.21.15.5:0/1981307671 learned_addr learned my addr 172.21.15.5:0/1981307671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.575+0000 7f859ffff640 1 --2- 172.21.15.5:0/1981307671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e7d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.576+0000 7f85a535a640 1 --2- 172.21.15.5:0/1981307671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e85370 0x55abd2e9c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.576+0000 7f85a4b59640 1 -- 172.21.15.5:0/1981307671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 msgr2=0x55abd2e7d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.576+0000 7f85a4b59640 1 --2- 172.21.15.5:0/1981307671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e7d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.576+0000 7f85a4b59640 1 -- 172.21.15.5:0/1981307671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e85370 msgr2=0x55abd2e9c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.576+0000 7f85a4b59640 1 --2- 172.21.15.5:0/1981307671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e85370 0x55abd2e9c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.576+0000 7f85a4b59640 1 -- 172.21.15.5:0/1981307671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd2e80550 con 0x55abd2e7a5d0 2024-09-19T06:14:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.576+0000 7f85a4b59640 1 --2- 172.21.15.5:0/1981307671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 0x55abd2e7d120 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8590005db0 tx=0x7f859000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae24fa3142bcb58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.576+0000 7f85867fc640 1 -- 172.21.15.5:0/1981307671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590004460 con 0x55abd2e7a5d0 2024-09-19T06:14:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.576+0000 7f85867fc640 1 -- 172.21.15.5:0/1981307671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8590019410 con 0x55abd2e7a5d0 2024-09-19T06:14:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.576+0000 7f85867fc640 1 -- 172.21.15.5:0/1981307671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85900197b0 con 0x55abd2e7a5d0 2024-09-19T06:14:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.577+0000 7f85a9196740 1 -- 172.21.15.5:0/1981307671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 msgr2=0x55abd2e7d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.577+0000 7f85a9196740 1 --2- 172.21.15.5:0/1981307671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 0x55abd2e7d120 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8590005db0 tx=0x7f859000f3c0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.577+0000 7f85a9196740 1 -- 172.21.15.5:0/1981307671 shutdown_connections 2024-09-19T06:14:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.577+0000 7f85a9196740 1 --2- 172.21.15.5:0/1981307671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e85370 0x55abd2e9c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.577+0000 7f85a9196740 1 --2- 172.21.15.5:0/1981307671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7bb40 0x55abd2e7d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.577+0000 7f85a9196740 1 --2- 172.21.15.5:0/1981307671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e7a5d0 0x55abd2e7d120 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.577+0000 7f85a9196740 1 -- 172.21.15.5:0/1981307671 >> 172.21.15.5:0/1981307671 conn(0x55abd2e43310 msgr2=0x55abd2e84080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.577+0000 7f85a9196740 1 -- 172.21.15.5:0/1981307671 shutdown_connections 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.577+0000 7f85a9196740 1 -- 172.21.15.5:0/1981307671 wait complete. 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.577+0000 7f85a9196740 1 -- start start 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f85a9196740 1 Processor -- start 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f85a9196740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7a5d0 0x55abd2e8d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f85a9196740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7bb40 0x55abd2e88d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f85a9196740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 0x55abd2e89240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f85a9196740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01b0f60 con 0x55abd2e85370 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f85a9196740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01b0830 con 0x55abd2e7bb40 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f85a9196740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01ada30 con 0x55abd2e7a5d0 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f85a9196740 10 client.0 _ll_register_callbacks cb 0x7f859001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f859ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7bb40 0x55abd2e88d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f85a535a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 0x55abd2e89240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f85a535a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 0x55abd2e89240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41714/0 (socket says 172.21.15.5:41714) 2024-09-19T06:14:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f85a535a640 1 -- 172.21.15.5:0/1673675880 learned_addr learned my addr 172.21.15.5:0/1673675880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f859ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7bb40 0x55abd2e88d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49486/0 (socket says 172.21.15.5:49486) 2024-09-19T06:14:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.578+0000 7f85a4b59640 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7a5d0 0x55abd2e8d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f85a535a640 1 -- 172.21.15.5:0/1673675880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7a5d0 msgr2=0x55abd2e8d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f85a535a640 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7a5d0 0x55abd2e8d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f85a535a640 1 -- 172.21.15.5:0/1673675880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7bb40 msgr2=0x55abd2e88d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f85a535a640 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7bb40 0x55abd2e88d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f85a535a640 1 -- 172.21.15.5:0/1673675880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd2e89830 con 0x55abd2e85370 2024-09-19T06:14:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f859ffff640 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7bb40 0x55abd2e88d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f85a4b59640 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7a5d0 0x55abd2e8d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f85a535a640 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 0x55abd2e89240 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8598099010 tx=0x7f859809c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d9873fdf19306b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f857ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f857ffff640 1 -- 172.21.15.5:0/1673675880 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85980ba060 con 0x55abd2e85370 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f857ffff640 1 -- 172.21.15.5:0/1673675880 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f85980b0b60 con 0x55abd2e85370 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f85a9196740 1 -- 172.21.15.5:0/1673675880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8590019410 con 0x55abd2e85370 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f857ffff640 1 -- 172.21.15.5:0/1673675880 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f859809c640 con 0x55abd2e85370 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f85a9196740 10 client.18315 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.579+0000 7f85a9196740 1 -- 172.21.15.5:0/1673675880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55abd2eff760 con 0x55abd2e85370 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f857ffff640 1 -- 172.21.15.5:0/1673675880 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8590019410 con 0x55abd2e85370 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f857ffff640 1 -- 172.21.15.5:0/1673675880 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f8598098520 con 0x55abd2e85370 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f85a9196740 10 client.18315 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f85a9196740 10 client.18315 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:14:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f85a9196740 20 client.18315 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f85a9196740 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55abd2effe30 0x55abd2f02250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f85a9196740 4 client.18315 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:14:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f85a9196740 1 -- 172.21.15.5:0/1673675880 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abd2de7d70 con 0x55abd2effe30 2024-09-19T06:14:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f85a4b59640 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55abd2effe30 0x55abd2f02250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f85a4b59640 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55abd2effe30 0x55abd2f02250 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe5764af3dccc042 server_cookie=88d6698ac5feb970 in_seq=0 out_seq=0 2024-09-19T06:14:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f857ffff640 10 client.18315 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:14:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f857ffff640 1 -- 172.21.15.5:0/1673675880 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abd2de7d70 con 0x55abd2effe30 2024-09-19T06:14:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.580+0000 7f857ffff640 10 client.18315 handle_command_reply: tid=0 2024-09-19T06:14:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f85a9196740 1 client.18315 shutdown 2024-09-19T06:14:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f85a9196740 1 -- 172.21.15.5:0/1673675880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 msgr2=0x55abd2e89240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f85a9196740 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 0x55abd2e89240 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8598099010 tx=0x7f859809c180 comp rx=0 tx=0).stop 2024-09-19T06:14:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f85a9196740 1 -- 172.21.15.5:0/1673675880 shutdown_connections 2024-09-19T06:14:25.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f85a9196740 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55abd2effe30 0x55abd2f02250 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f85a9196740 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd2e85370 0x55abd2e89240 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f85a9196740 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55abd2e7bb40 0x55abd2e88d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f85a9196740 1 --2- 172.21.15.5:0/1673675880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abd2e7a5d0 0x55abd2e8d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f85a9196740 1 -- 172.21.15.5:0/1673675880 >> 172.21.15.5:0/1673675880 conn(0x55abd2e43310 msgr2=0x55abd2e754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:25.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f857ffff640 0 client.18315 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:14:25.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f85a9196740 1 -- 172.21.15.5:0/1673675880 shutdown_connections 2024-09-19T06:14:25.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f85a9196740 1 -- 172.21.15.5:0/1673675880 wait complete. 2024-09-19T06:14:25.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.581+0000 7f85a9196740 20 client.18315 trim_cache size 0 max 16384 2024-09-19T06:14:25.589 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:14:25.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.582+0000 7f85a6dbd640 1 -- 172.21.15.5:0/2550126283 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f85880a3430 msgr2=0x7f85880a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:25.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.582+0000 7f85a6dbd640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f85880a3430 0x7f85880a58f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8594009230 tx=0x7f8594000bc0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.582+0000 7f85a6dbd640 1 -- 172.21.15.5:0/2550126283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0102ef0 msgr2=0x7f85a019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:25.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.582+0000 7f85a6dbd640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0102ef0 0x7f85a019eec0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f859800eb10 tx=0x7f859800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.582+0000 7f85a6dbd640 1 -- 172.21.15.5:0/2550126283 shutdown_connections 2024-09-19T06:14:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.582+0000 7f85a6dbd640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0102ef0 0x7f85a019eec0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.582+0000 7f85a6dbd640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f85880a3430 0x7f85880a58f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.582+0000 7f85a6dbd640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a0102570 0x7f85a019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.582+0000 7f85a6dbd640 1 --2- 172.21.15.5:0/2550126283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a0100e20 0x7f85a019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.582+0000 7f85a6dbd640 1 -- 172.21.15.5:0/2550126283 >> 172.21.15.5:0/2550126283 conn(0x7f85a00fc7d0 msgr2=0x7f85a006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.582+0000 7f85a6dbd640 1 -- 172.21.15.5:0/2550126283 shutdown_connections 2024-09-19T06:14:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:25.582+0000 7f85a6dbd640 1 -- 172.21.15.5:0/2550126283 wait complete. 2024-09-19T06:14:25.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 262651 2.30 MB/sec execute 471 sec latency 2652.220 ms 2024-09-19T06:14:26.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:14:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:14:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:26 smithi005 ceph-mon[30955]: pgmap v346: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 875 KiB/s wr, 84 op/s 2024-09-19T06:14:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:26 smithi157 ceph-mon[35346]: pgmap v346: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 875 KiB/s wr, 84 op/s 2024-09-19T06:14:26.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 262651 2.29 MB/sec execute 472 sec latency 1115.401 ms 2024-09-19T06:14:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:26 smithi049 ceph-mon[34640]: pgmap v346: 97 pgs: 97 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 875 KiB/s wr, 84 op/s 2024-09-19T06:14:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.045+0000 7fae041ae640 1 -- 172.21.15.5:0/220967678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc106a80 msgr2=0x7fadfc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.045+0000 7fae041ae640 1 --2- 172.21.15.5:0/220967678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc106a80 0x7fadfc106e60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009930 tx=0x7fadf802a420 comp rx=0 tx=0).stop 2024-09-19T06:14:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.045+0000 7fae041ae640 1 -- 172.21.15.5:0/220967678 shutdown_connections 2024-09-19T06:14:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.045+0000 7fae041ae640 1 --2- 172.21.15.5:0/220967678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadfc101420 0x7fadfc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.045+0000 7fae041ae640 1 --2- 172.21.15.5:0/220967678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadfc100a80 0x7fadfc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.045+0000 7fae041ae640 1 --2- 172.21.15.5:0/220967678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc106a80 0x7fadfc106e60 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.045+0000 7fae041ae640 1 -- 172.21.15.5:0/220967678 >> 172.21.15.5:0/220967678 conn(0x7fadfc0fc7a0 msgr2=0x7fadfc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:27.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.045+0000 7fae041ae640 1 -- 172.21.15.5:0/220967678 shutdown_connections 2024-09-19T06:14:27.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.045+0000 7fae041ae640 1 -- 172.21.15.5:0/220967678 wait complete. 2024-09-19T06:14:27.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.046+0000 7fae041ae640 1 Processor -- start 2024-09-19T06:14:27.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.046+0000 7fae041ae640 1 -- start start 2024-09-19T06:14:27.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.046+0000 7fae041ae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc100a80 0x7fadfc0fced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae041ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadfc101420 0x7fadfc0fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae041ae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadfc106a80 0x7fadfc0fd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae041ae640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc110cf0 con 0x7fadfc100a80 2024-09-19T06:14:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae041ae640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc110b70 con 0x7fadfc106a80 2024-09-19T06:14:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae041ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc110e70 con 0x7fadfc101420 2024-09-19T06:14:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae01f4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc100a80 0x7fadfc0fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae01f4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc100a80 0x7fadfc0fced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41734/0 (socket says 172.21.15.5:41734) 2024-09-19T06:14:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae01f4a640 1 -- 172.21.15.5:0/4286906550 learned_addr learned my addr 172.21.15.5:0/4286906550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae01749640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadfc101420 0x7fadfc0fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae0274b640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadfc106a80 0x7fadfc0fd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae01f4a640 1 -- 172.21.15.5:0/4286906550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadfc101420 msgr2=0x7fadfc0fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae01f4a640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadfc101420 0x7fadfc0fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae01f4a640 1 -- 172.21.15.5:0/4286906550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadfc106a80 msgr2=0x7fadfc0fd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae01f4a640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadfc106a80 0x7fadfc0fd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae01f4a640 1 -- 172.21.15.5:0/4286906550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc1aad50 con 0x7fadfc100a80 2024-09-19T06:14:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fae01f4a640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc100a80 0x7fadfc0fced0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fadf8004770 tx=0x7fadf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.047+0000 7fadf2ffd640 1 -- 172.21.15.5:0/4286906550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf803f070 con 0x7fadfc100a80 2024-09-19T06:14:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.048+0000 7fae041ae640 1 -- 172.21.15.5:0/4286906550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadfc1aaf20 con 0x7fadfc100a80 2024-09-19T06:14:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.048+0000 7fadf2ffd640 1 -- 172.21.15.5:0/4286906550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadf803b460 con 0x7fadfc100a80 2024-09-19T06:14:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.048+0000 7fadf2ffd640 1 -- 172.21.15.5:0/4286906550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8035070 con 0x7fadfc100a80 2024-09-19T06:14:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.048+0000 7fae041ae640 1 -- 172.21.15.5:0/4286906550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadfc1ab3d0 con 0x7fadfc100a80 2024-09-19T06:14:27.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fadf2ffd640 1 -- 172.21.15.5:0/4286906550 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fadf803b610 con 0x7fadfc100a80 2024-09-19T06:14:27.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fae06587740 1 Processor -- start 2024-09-19T06:14:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fae06587740 1 -- start start 2024-09-19T06:14:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fae06587740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeafb40 0x560ecfeae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fae06587740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 0x560ecfeaea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fae06587740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeb9370 0x560ecfebb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fae06587740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560ecfe142f0 con 0x560ecfeae5d0 2024-09-19T06:14:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fae06587740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560ecfdbbcf0 con 0x560ecfeb9370 2024-09-19T06:14:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fae06587740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560ecfe102b0 con 0x560ecfeafb40 2024-09-19T06:14:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fae01f4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 0x560ecfeaea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fae0274b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeafb40 0x560ecfeae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fae01f4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 0x560ecfeaea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41750/0 (socket says 172.21.15.5:41750) 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.051+0000 7fae01f4a640 1 -- 172.21.15.5:0/1281864975 learned_addr learned my addr 172.21.15.5:0/1281864975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fae01749640 1 --2- 172.21.15.5:0/1281864975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeb9370 0x560ecfebb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fae0274b640 1 -- 172.21.15.5:0/1281864975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeb9370 msgr2=0x560ecfebb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fae0274b640 1 --2- 172.21.15.5:0/1281864975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeb9370 0x560ecfebb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fae0274b640 1 -- 172.21.15.5:0/1281864975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 msgr2=0x560ecfeaea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fadf2ffd640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fade80a33b0 0x7fade80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fae0274b640 1 --2- 172.21.15.5:0/1281864975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 0x560ecfeaea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fae0274b640 1 -- 172.21.15.5:0/1281864975 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ecfda03a0 con 0x560ecfeafb40 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fae01f4a640 1 --2- 172.21.15.5:0/1281864975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 0x560ecfeaea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fae0274b640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fade80a33b0 0x7fade80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fadf2ffd640 1 -- 172.21.15.5:0/4286906550 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fadf80effc0 con 0x7fadfc100a80 2024-09-19T06:14:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fae0274b640 1 --2- 172.21.15.5:0/1281864975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeafb40 0x560ecfeae090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009eb0 tx=0x7fadf4010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=457c86b6ceefa220 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fade7fff640 1 -- 172.21.15.5:0/1281864975 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4022070 con 0x560ecfeafb40 2024-09-19T06:14:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fade7fff640 1 -- 172.21.15.5:0/1281864975 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadf4012690 con 0x560ecfeafb40 2024-09-19T06:14:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fae0274b640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fade80a33b0 0x7fade80a5870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fadf4011010 tx=0x7fadf4022000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.052+0000 7fade7fff640 1 -- 172.21.15.5:0/1281864975 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4016070 con 0x560ecfeafb40 2024-09-19T06:14:27.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.053+0000 7fae06587740 1 -- 172.21.15.5:0/1281864975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeafb40 msgr2=0x560ecfeae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:27.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.053+0000 7fae06587740 1 --2- 172.21.15.5:0/1281864975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeafb40 0x560ecfeae090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009eb0 tx=0x7fadf4010740 comp rx=0 tx=0).stop 2024-09-19T06:14:27.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.053+0000 7fae06587740 1 -- 172.21.15.5:0/1281864975 shutdown_connections 2024-09-19T06:14:27.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.053+0000 7fae06587740 1 --2- 172.21.15.5:0/1281864975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeb9370 0x560ecfebb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.053+0000 7fae06587740 1 --2- 172.21.15.5:0/1281864975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 0x560ecfeaea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.053+0000 7fae06587740 1 --2- 172.21.15.5:0/1281864975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeafb40 0x560ecfeae090 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.053+0000 7fae06587740 1 -- 172.21.15.5:0/1281864975 >> 172.21.15.5:0/1281864975 conn(0x560ecfe77310 msgr2=0x560ecfeaac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.053+0000 7fae06587740 1 -- 172.21.15.5:0/1281864975 shutdown_connections 2024-09-19T06:14:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.053+0000 7fae06587740 1 -- 172.21.15.5:0/1281864975 wait complete. 2024-09-19T06:14:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae06587740 1 -- start start 2024-09-19T06:14:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae06587740 1 Processor -- start 2024-09-19T06:14:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae06587740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeae5d0 0x560ecfeb8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae06587740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeafb40 0x560ecfeb6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae06587740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeb9370 0x560ecfeb66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae06587740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560ecfebc070 con 0x560ecfeafb40 2024-09-19T06:14:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae06587740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560ecfdcf060 con 0x560ecfeb9370 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae06587740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560ecfebc1f0 con 0x560ecfeae5d0 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae06587740 10 client.0 _ll_register_callbacks cb 0x560ecfb228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae01749640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeafb40 0x560ecfeb6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae01749640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeafb40 0x560ecfeb6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41766/0 (socket says 172.21.15.5:41766) 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae01749640 1 -- 172.21.15.5:0/1336881650 learned_addr learned my addr 172.21.15.5:0/1336881650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae01f4a640 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeae5d0 0x560ecfeb8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.054+0000 7fae0274b640 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeb9370 0x560ecfeb66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fae01749640 1 -- 172.21.15.5:0/1336881650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeae5d0 msgr2=0x560ecfeb8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fae01749640 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeae5d0 0x560ecfeb8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fae01749640 1 -- 172.21.15.5:0/1336881650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeb9370 msgr2=0x560ecfeb66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fae01749640 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeb9370 0x560ecfeb66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fae01749640 1 -- 172.21.15.5:0/1336881650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ecfb853c0 con 0x560ecfeafb40 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fae01749640 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeafb40 0x560ecfeb6080 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fadec00bdf0 tx=0x7fadec012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d0b927cb1874a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fade57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fade57fa640 1 -- 172.21.15.5:0/1336881650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec010850 con 0x560ecfeafb40 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fae06587740 1 -- 172.21.15.5:0/1336881650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ecfe21860 con 0x560ecfeafb40 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fade57fa640 1 -- 172.21.15.5:0/1336881650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fadec0109f0 con 0x560ecfeafb40 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fade57fa640 1 -- 172.21.15.5:0/1336881650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fadec010bf0 con 0x560ecfeafb40 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fae06587740 10 client.18327 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.055+0000 7fae06587740 1 -- 172.21.15.5:0/1336881650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560ecfec7bc0 con 0x560ecfeafb40 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.056+0000 7fade57fa640 1 -- 172.21.15.5:0/1336881650 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560ecfe21860 con 0x560ecfeafb40 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.056+0000 7fade57fa640 1 -- 172.21.15.5:0/1336881650 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fadec0253d0 con 0x560ecfeafb40 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.056+0000 7fae06587740 10 client.18327 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.056+0000 7fae06587740 10 client.18327 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.056+0000 7fae06587740 20 client.18327 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.056+0000 7fae06587740 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x560ecfec9120 0x560ecfecb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.056+0000 7fae06587740 4 client.18327 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.056+0000 7fae06587740 1 -- 172.21.15.5:0/1336881650 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ecfe1bd70 con 0x560ecfec9120 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.056+0000 7fae01f4a640 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x560ecfec9120 0x560ecfecb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.057+0000 7fae01f4a640 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x560ecfec9120 0x560ecfecb540 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8199f8f34120c8c server_cookie=11a70ff01772bf22 in_seq=0 out_seq=0 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.057+0000 7fade57fa640 10 client.18327 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.057+0000 7fade57fa640 1 -- 172.21.15.5:0/1336881650 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ecfe1bd70 con 0x560ecfec9120 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.057+0000 7fade57fa640 10 client.18327 handle_command_reply: tid=0 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.057+0000 7fae06587740 1 client.18327 shutdown 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.057+0000 7fae06587740 1 -- 172.21.15.5:0/1336881650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeafb40 msgr2=0x560ecfeb6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.057+0000 7fae06587740 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeafb40 0x560ecfeb6080 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fadec00bdf0 tx=0x7fadec012820 comp rx=0 tx=0).stop 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.057+0000 7fae06587740 1 -- 172.21.15.5:0/1336881650 shutdown_connections 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.058+0000 7fae06587740 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x560ecfec9120 0x560ecfecb540 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.058+0000 7fae06587740 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeb9370 0x560ecfeb66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.058+0000 7fae06587740 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeafb40 0x560ecfeb6080 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.058+0000 7fade57fa640 0 client.18327 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:14:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.058+0000 7fae06587740 1 --2- 172.21.15.5:0/1336881650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeae5d0 0x560ecfeb8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.058+0000 7fae06587740 1 -- 172.21.15.5:0/1336881650 >> 172.21.15.5:0/1336881650 conn(0x560ecfe77310 msgr2=0x560ecfeb0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.058+0000 7fae06587740 1 -- 172.21.15.5:0/1336881650 shutdown_connections 2024-09-19T06:14:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.058+0000 7fae06587740 1 -- 172.21.15.5:0/1336881650 wait complete. 2024-09-19T06:14:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.058+0000 7fae06587740 20 client.18327 trim_cache size 0 max 16384 2024-09-19T06:14:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae06587740 1 Processor -- start 2024-09-19T06:14:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae06587740 1 -- start start 2024-09-19T06:14:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae06587740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeae5d0 0x560ecfec5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae06587740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeafb40 0x560ecfeb5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae06587740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeb9370 0x560ecfeb5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae06587740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc1b3ee0 con 0x560ecfeb9370 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae06587740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc1b5ae0 con 0x560ecfeae5d0 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae06587740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc1b45b0 con 0x560ecfeafb40 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae0274b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeb9370 0x560ecfeb5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae0274b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeb9370 0x560ecfeb5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41772/0 (socket says 172.21.15.5:41772) 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae01f4a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeae5d0 0x560ecfec5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae0274b640 1 -- 172.21.15.5:0/133162146 learned_addr learned my addr 172.21.15.5:0/133162146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae01749640 1 --2- 172.21.15.5:0/133162146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeafb40 0x560ecfeb5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae0274b640 1 -- 172.21.15.5:0/133162146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeafb40 msgr2=0x560ecfeb5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae0274b640 1 --2- 172.21.15.5:0/133162146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeafb40 0x560ecfeb5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae0274b640 1 -- 172.21.15.5:0/133162146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeae5d0 msgr2=0x560ecfec5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae0274b640 1 --2- 172.21.15.5:0/133162146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeae5d0 0x560ecfec5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae0274b640 1 -- 172.21.15.5:0/133162146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ecfeaebe0 con 0x560ecfeb9370 2024-09-19T06:14:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.071+0000 7fae01749640 1 --2- 172.21.15.5:0/133162146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeafb40 0x560ecfeb5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.072+0000 7fae0274b640 1 --2- 172.21.15.5:0/133162146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeb9370 0x560ecfeb5e80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fadf4005c50 tx=0x7fadf4005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ee79552d095ad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.072+0000 7fade77fe640 1 -- 172.21.15.5:0/133162146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4016040 con 0x560ecfeb9370 2024-09-19T06:14:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.072+0000 7fade77fe640 1 -- 172.21.15.5:0/133162146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadf4035070 con 0x560ecfeb9370 2024-09-19T06:14:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.072+0000 7fade77fe640 1 -- 172.21.15.5:0/133162146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4026450 con 0x560ecfeb9370 2024-09-19T06:14:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.072+0000 7fae06587740 1 -- 172.21.15.5:0/133162146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeb9370 msgr2=0x560ecfeb5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.072+0000 7fae06587740 1 --2- 172.21.15.5:0/133162146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeb9370 0x560ecfeb5e80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fadf4005c50 tx=0x7fadf4005c80 comp rx=0 tx=0).stop 2024-09-19T06:14:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.072+0000 7fae06587740 1 -- 172.21.15.5:0/133162146 shutdown_connections 2024-09-19T06:14:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.072+0000 7fae06587740 1 --2- 172.21.15.5:0/133162146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeb9370 0x560ecfeb5e80 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.072+0000 7fae06587740 1 --2- 172.21.15.5:0/133162146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeafb40 0x560ecfeb5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.072+0000 7fae06587740 1 --2- 172.21.15.5:0/133162146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeae5d0 0x560ecfec5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.072+0000 7fae06587740 1 -- 172.21.15.5:0/133162146 >> 172.21.15.5:0/133162146 conn(0x560ecfe77310 msgr2=0x560ecfeb1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae06587740 1 -- 172.21.15.5:0/133162146 shutdown_connections 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae06587740 1 -- 172.21.15.5:0/133162146 wait complete. 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae06587740 1 -- start start 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae06587740 1 Processor -- start 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae06587740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 0x560ecfec17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae06587740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeafb40 0x560ecfec1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae06587740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeb9370 0x560ecfebcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae06587740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc1b5410 con 0x560ecfeae5d0 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae06587740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc1b4ce0 con 0x560ecfeafb40 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae06587740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc1b1ee0 con 0x560ecfeb9370 2024-09-19T06:14:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae06587740 10 client.0 _ll_register_callbacks cb 0x7fadec005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae01f4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 0x560ecfec17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.073+0000 7fae01f4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 0x560ecfec17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41776/0 (socket says 172.21.15.5:41776) 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fae01f4a640 1 -- 172.21.15.5:0/976000074 learned_addr learned my addr 172.21.15.5:0/976000074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fae01749640 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeafb40 0x560ecfec1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fae0274b640 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeb9370 0x560ecfebcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fae01f4a640 1 -- 172.21.15.5:0/976000074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeb9370 msgr2=0x560ecfebcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fae01f4a640 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeb9370 0x560ecfebcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fae01f4a640 1 -- 172.21.15.5:0/976000074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeafb40 msgr2=0x560ecfec1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fae01f4a640 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeafb40 0x560ecfec1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fae01f4a640 1 -- 172.21.15.5:0/976000074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ecfebd5a0 con 0x560ecfeae5d0 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fae01f4a640 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 0x560ecfec17f0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fadf80ed7a0 tx=0x7fadf80f3b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda0b4e6ed0a45a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fade57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fade57fa640 1 -- 172.21.15.5:0/976000074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8048040 con 0x560ecfeae5d0 2024-09-19T06:14:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fade57fa640 1 -- 172.21.15.5:0/976000074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fadf80ed970 con 0x560ecfeae5d0 2024-09-19T06:14:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fae06587740 1 -- 172.21.15.5:0/976000074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadf4035070 con 0x560ecfeae5d0 2024-09-19T06:14:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fade57fa640 1 -- 172.21.15.5:0/976000074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fadf80bbe70 con 0x560ecfeae5d0 2024-09-19T06:14:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fae06587740 10 client.18333 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.074+0000 7fae06587740 1 -- 172.21.15.5:0/976000074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560ecff33a80 con 0x560ecfeae5d0 2024-09-19T06:14:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.075+0000 7fade57fa640 1 -- 172.21.15.5:0/976000074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadf4035070 con 0x560ecfeae5d0 2024-09-19T06:14:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.075+0000 7fade57fa640 1 -- 172.21.15.5:0/976000074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fadf804b060 con 0x560ecfeae5d0 2024-09-19T06:14:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.075+0000 7fae06587740 10 client.18333 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.075+0000 7fae06587740 10 client.18333 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:14:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.075+0000 7fae06587740 20 client.18333 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.075+0000 7fae06587740 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x560ecff34130 0x560ecff36550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.075+0000 7fae06587740 4 client.18333 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.075+0000 7fae06587740 1 -- 172.21.15.5:0/976000074 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ecfd98170 con 0x560ecff34130 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.075+0000 7fae01749640 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x560ecff34130 0x560ecff36550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.075+0000 7fae01749640 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x560ecff34130 0x560ecff36550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c179d3034434ded0 server_cookie=5794397d2a453736 in_seq=0 out_seq=0 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.075+0000 7fade57fa640 10 client.18333 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fade57fa640 1 -- 172.21.15.5:0/976000074 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ecfd98170 con 0x560ecff34130 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fade57fa640 10 client.18333 handle_command_reply: tid=0 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fae06587740 1 client.18333 shutdown 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fae06587740 1 -- 172.21.15.5:0/976000074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 msgr2=0x560ecfec17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fae06587740 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 0x560ecfec17f0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fadf80ed7a0 tx=0x7fadf80f3b10 comp rx=0 tx=0).stop 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fae06587740 1 -- 172.21.15.5:0/976000074 shutdown_connections 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fae06587740 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x560ecff34130 0x560ecff36550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fae06587740 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ecfeb9370 0x560ecfebcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fae06587740 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560ecfeafb40 0x560ecfec1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fae06587740 1 --2- 172.21.15.5:0/976000074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ecfeae5d0 0x560ecfec17f0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fade57fa640 0 client.18333 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:14:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fae06587740 1 -- 172.21.15.5:0/976000074 >> 172.21.15.5:0/976000074 conn(0x560ecfe77310 msgr2=0x560ecfeb5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fae06587740 1 -- 172.21.15.5:0/976000074 shutdown_connections 2024-09-19T06:14:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fae06587740 1 -- 172.21.15.5:0/976000074 wait complete. 2024-09-19T06:14:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.076+0000 7fae06587740 20 client.18333 trim_cache size 0 max 16384 2024-09-19T06:14:27.083 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:14:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.077+0000 7fae041ae640 1 -- 172.21.15.5:0/4286906550 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fade80a33b0 msgr2=0x7fade80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.077+0000 7fae041ae640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fade80a33b0 0x7fade80a5870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fadf4011010 tx=0x7fadf4022000 comp rx=0 tx=0).stop 2024-09-19T06:14:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.077+0000 7fae041ae640 1 -- 172.21.15.5:0/4286906550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc100a80 msgr2=0x7fadfc0fced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.077+0000 7fae041ae640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc100a80 0x7fadfc0fced0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fadf8004770 tx=0x7fadf80042f0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.077+0000 7fae041ae640 1 -- 172.21.15.5:0/4286906550 shutdown_connections 2024-09-19T06:14:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.077+0000 7fae041ae640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadfc106a80 0x7fadfc0fd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.077+0000 7fae041ae640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fade80a33b0 0x7fade80a5870 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.077+0000 7fae041ae640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadfc101420 0x7fadfc0fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.077+0000 7fae041ae640 1 --2- 172.21.15.5:0/4286906550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc100a80 0x7fadfc0fced0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:27.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.077+0000 7fae041ae640 1 -- 172.21.15.5:0/4286906550 >> 172.21.15.5:0/4286906550 conn(0x7fadfc0fc7a0 msgr2=0x7fadfc06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:27.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.077+0000 7fae041ae640 1 -- 172.21.15.5:0/4286906550 shutdown_connections 2024-09-19T06:14:27.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:27.077+0000 7fae041ae640 1 -- 172.21.15.5:0/4286906550 wait complete. 2024-09-19T06:14:27.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 262651 2.29 MB/sec execute 473 sec latency 2115.489 ms 2024-09-19T06:14:27.956 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:14:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:14:28.614 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:28 smithi005 ceph-mon[30955]: pgmap v347: 97 pgs: 97 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-09-19T06:14:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.695+0000 7f20d0a2b640 1 -- 172.21.15.5:0/1435878160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20cc102d40 msgr2=0x7f20cc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.695+0000 7f20d0a2b640 1 --2- 172.21.15.5:0/1435878160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20cc102d40 0x7f20cc103120 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009900 tx=0x7f20bc02a480 comp rx=0 tx=0).stop 2024-09-19T06:14:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.696+0000 7f20d0a2b640 1 -- 172.21.15.5:0/1435878160 shutdown_connections 2024-09-19T06:14:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.696+0000 7f20d0a2b640 1 --2- 172.21.15.5:0/1435878160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20cc10c170 0x7f20cc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.696+0000 7f20d0a2b640 1 --2- 172.21.15.5:0/1435878160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20cc103660 0x7f20cc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.696+0000 7f20d0a2b640 1 --2- 172.21.15.5:0/1435878160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20cc102d40 0x7f20cc103120 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.696+0000 7f20d0a2b640 1 -- 172.21.15.5:0/1435878160 >> 172.21.15.5:0/1435878160 conn(0x7f20cc0fc7a0 msgr2=0x7f20cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.696+0000 7f20d0a2b640 1 -- 172.21.15.5:0/1435878160 shutdown_connections 2024-09-19T06:14:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.696+0000 7f20d0a2b640 1 -- 172.21.15.5:0/1435878160 wait complete. 2024-09-19T06:14:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.696+0000 7f20d0a2b640 1 Processor -- start 2024-09-19T06:14:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.696+0000 7f20d0a2b640 1 -- start start 2024-09-19T06:14:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20d0a2b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20cc102d40 0x7f20cc198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20d0a2b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20cc103660 0x7f20cc1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20d0a2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20cc10c170 0x7f20cc19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20d0a2b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc111e20 con 0x7f20cc10c170 2024-09-19T06:14:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20d0a2b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc111ca0 con 0x7f20cc102d40 2024-09-19T06:14:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20d0a2b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc111fa0 con 0x7f20cc103660 2024-09-19T06:14:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20cad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20cc10c170 0x7f20cc19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20cad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20cc10c170 0x7f20cc19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41804/0 (socket says 172.21.15.5:41804) 2024-09-19T06:14:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20cad9d640 1 -- 172.21.15.5:0/2450572968 learned_addr learned my addr 172.21.15.5:0/2450572968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20ca59c640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20cc102d40 0x7f20cc198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20cc103660 0x7f20cc1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20cad9d640 1 -- 172.21.15.5:0/2450572968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20cc103660 msgr2=0x7f20cc1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20cad9d640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20cc103660 0x7f20cc1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20cad9d640 1 -- 172.21.15.5:0/2450572968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20cc102d40 msgr2=0x7f20cc198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20cad9d640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20cc102d40 0x7f20cc198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20cad9d640 1 -- 172.21.15.5:0/2450572968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc19d330 con 0x7f20cc10c170 2024-09-19T06:14:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.697+0000 7f20cad9d640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20cc10c170 0x7f20cc19cc50 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f20c400eaa0 tx=0x7f20c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:28.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.698+0000 7f20bb7fe640 1 -- 172.21.15.5:0/2450572968 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c400cdb0 con 0x7f20cc10c170 2024-09-19T06:14:28.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.698+0000 7f20bb7fe640 1 -- 172.21.15.5:0/2450572968 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20c4005020 con 0x7f20cc10c170 2024-09-19T06:14:28.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.698+0000 7f20d0a2b640 1 -- 172.21.15.5:0/2450572968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc19d620 con 0x7f20cc10c170 2024-09-19T06:14:28.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.698+0000 7f20bb7fe640 1 -- 172.21.15.5:0/2450572968 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c40116d0 con 0x7f20cc10c170 2024-09-19T06:14:28.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.698+0000 7f20d0a2b640 1 -- 172.21.15.5:0/2450572968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc1a4f00 con 0x7f20cc10c170 2024-09-19T06:14:28.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.700+0000 7f20bb7fe640 1 -- 172.21.15.5:0/2450572968 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f20c40051d0 con 0x7f20cc10c170 2024-09-19T06:14:28.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.700+0000 7f20d2e04740 1 Processor -- start 2024-09-19T06:14:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.700+0000 7f20d2e04740 1 -- start start 2024-09-19T06:14:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20d2e04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 0x55ec4e046090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20d2e04740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 0x55ec4e046a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20d2e04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e051370 0x55ec4e053760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20d2e04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ec4dfac2f0 con 0x55ec4e047b40 2024-09-19T06:14:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20d2e04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ec4df53cf0 con 0x55ec4e051370 2024-09-19T06:14:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20d2e04740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ec4dfa82b0 con 0x55ec4e0465d0 2024-09-19T06:14:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20c9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 0x55ec4e046090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20c9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 0x55ec4e046090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41816/0 (socket says 172.21.15.5:41816) 2024-09-19T06:14:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20c9d9b640 1 -- 172.21.15.5:0/1768080733 learned_addr learned my addr 172.21.15.5:0/1768080733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20bb7fe640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20b40a33b0 0x7f20b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20ca59c640 1 --2- 172.21.15.5:0/1768080733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 0x55ec4e046a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20c9d9b640 1 -- 172.21.15.5:0/1768080733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 msgr2=0x55ec4e046a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/1768080733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 0x55ec4e046a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20c9d9b640 1 -- 172.21.15.5:0/1768080733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e051370 msgr2=0x55ec4e053760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/1768080733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e051370 0x55ec4e053760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20c9d9b640 1 -- 172.21.15.5:0/1768080733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4df383a0 con 0x55ec4e047b40 2024-09-19T06:14:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20cad9d640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20b40a33b0 0x7f20b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20bb7fe640 1 -- 172.21.15.5:0/2450572968 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f20c40cff30 con 0x7f20cc10c170 2024-09-19T06:14:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.701+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/1768080733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 0x55ec4e046090 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f20c0005d90 tx=0x7f20c0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c768e59a51efbee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.702+0000 7f20b8ff9640 1 -- 172.21.15.5:0/1768080733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0016070 con 0x55ec4e047b40 2024-09-19T06:14:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.702+0000 7f20b8ff9640 1 -- 172.21.15.5:0/1768080733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20c00047a0 con 0x55ec4e047b40 2024-09-19T06:14:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.702+0000 7f20b8ff9640 1 -- 172.21.15.5:0/1768080733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c00126e0 con 0x55ec4e047b40 2024-09-19T06:14:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.702+0000 7f20cad9d640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20b40a33b0 0x7f20b40a5870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f20c40985f0 tx=0x7f20c408a040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.702+0000 7f20d2e04740 1 -- 172.21.15.5:0/1768080733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 msgr2=0x55ec4e046090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.702+0000 7f20d2e04740 1 --2- 172.21.15.5:0/1768080733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 0x55ec4e046090 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f20c0005d90 tx=0x7f20c0010740 comp rx=0 tx=0).stop 2024-09-19T06:14:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.702+0000 7f20d2e04740 1 -- 172.21.15.5:0/1768080733 shutdown_connections 2024-09-19T06:14:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.702+0000 7f20d2e04740 1 --2- 172.21.15.5:0/1768080733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e051370 0x55ec4e053760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.702+0000 7f20d2e04740 1 --2- 172.21.15.5:0/1768080733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 0x55ec4e046a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.702+0000 7f20d2e04740 1 --2- 172.21.15.5:0/1768080733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 0x55ec4e046090 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.702+0000 7f20d2e04740 1 -- 172.21.15.5:0/1768080733 >> 172.21.15.5:0/1768080733 conn(0x55ec4e00f310 msgr2=0x55ec4e042c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20d2e04740 1 -- 172.21.15.5:0/1768080733 shutdown_connections 2024-09-19T06:14:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20d2e04740 1 -- 172.21.15.5:0/1768080733 wait complete. 2024-09-19T06:14:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20d2e04740 1 -- start start 2024-09-19T06:14:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20d2e04740 1 Processor -- start 2024-09-19T06:14:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20d2e04740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 0x55ec4e050d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20d2e04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e047b40 0x55ec4e04dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20d2e04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 0x55ec4e04e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20d2e04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e054070 con 0x55ec4e051370 2024-09-19T06:14:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20d2e04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ec4df67060 con 0x55ec4e047b40 2024-09-19T06:14:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20d2e04740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e0541f0 con 0x55ec4e0465d0 2024-09-19T06:14:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20d2e04740 10 client.0 _ll_register_callbacks cb 0x55ec4dcba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20c9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 0x55ec4e04e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20c9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 0x55ec4e04e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41818/0 (socket says 172.21.15.5:41818) 2024-09-19T06:14:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20c9d9b640 1 -- 172.21.15.5:0/1269626025 learned_addr learned my addr 172.21.15.5:0/1269626025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.703+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 0x55ec4e050d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20ca59c640 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e047b40 0x55ec4e04dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20c9d9b640 1 -- 172.21.15.5:0/1269626025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 msgr2=0x55ec4e050d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 0x55ec4e050d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20c9d9b640 1 -- 172.21.15.5:0/1269626025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e047b40 msgr2=0x55ec4e04dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e047b40 0x55ec4e04dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20c9d9b640 1 -- 172.21.15.5:0/1269626025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4dd1d3c0 con 0x55ec4e051370 2024-09-19T06:14:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20ca59c640 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e047b40 0x55ec4e04dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:14:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 0x55ec4e04e530 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f20c0004cd0 tx=0x7f20c0004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e083dc5c5fe47f06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20b1ffb640 1 -- 172.21.15.5:0/1269626025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c001e020 con 0x55ec4e051370 2024-09-19T06:14:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20d2e04740 1 -- 172.21.15.5:0/1269626025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c00047a0 con 0x55ec4e051370 2024-09-19T06:14:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20b1ffb640 1 -- 172.21.15.5:0/1269626025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f20c002b430 con 0x55ec4e051370 2024-09-19T06:14:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20b1ffb640 1 -- 172.21.15.5:0/1269626025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f20c0016070 con 0x55ec4e051370 2024-09-19T06:14:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20d2e04740 10 client.18345 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20d2e04740 1 -- 172.21.15.5:0/1269626025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ec4e05faa0 con 0x55ec4e051370 2024-09-19T06:14:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.704+0000 7f20b1ffb640 1 -- 172.21.15.5:0/1269626025 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20c00047a0 con 0x55ec4e051370 2024-09-19T06:14:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.705+0000 7f20b1ffb640 1 -- 172.21.15.5:0/1269626025 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f20c002d700 con 0x55ec4e051370 2024-09-19T06:14:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.705+0000 7f20d2e04740 10 client.18345 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.705+0000 7f20d2e04740 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.705+0000 7f20d2e04740 20 client.18345 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.705+0000 7f20d2e04740 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ec4e060eb0 0x55ec4e0632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.705+0000 7f20d2e04740 4 client.18345 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.705+0000 7f20d2e04740 1 -- 172.21.15.5:0/1269626025 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec4dfb9860 con 0x55ec4e060eb0 2024-09-19T06:14:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.705+0000 7f20ca59c640 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ec4e060eb0 0x55ec4e0632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.705+0000 7f20b1ffb640 10 client.18345 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.705+0000 7f20ca59c640 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ec4e060eb0 0x55ec4e0632a0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9cf42ce251b74eb server_cookie=a7bdfaf8345a2cf0 in_seq=0 out_seq=0 2024-09-19T06:14:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.706+0000 7f20b1ffb640 1 -- 172.21.15.5:0/1269626025 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec4dfb9860 con 0x55ec4e060eb0 2024-09-19T06:14:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.706+0000 7f20b1ffb640 10 client.18345 handle_command_reply: tid=0 2024-09-19T06:14:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.706+0000 7f20d2e04740 1 client.18345 shutdown 2024-09-19T06:14:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.706+0000 7f20d2e04740 1 -- 172.21.15.5:0/1269626025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 msgr2=0x55ec4e04e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.706+0000 7f20d2e04740 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 0x55ec4e04e530 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f20c0004cd0 tx=0x7f20c0004f30 comp rx=0 tx=0).stop 2024-09-19T06:14:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.706+0000 7f20d2e04740 1 -- 172.21.15.5:0/1269626025 shutdown_connections 2024-09-19T06:14:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.706+0000 7f20d2e04740 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ec4e060eb0 0x55ec4e0632a0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.706+0000 7f20d2e04740 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 0x55ec4e04e530 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.706+0000 7f20d2e04740 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e047b40 0x55ec4e04dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.706+0000 7f20d2e04740 1 --2- 172.21.15.5:0/1269626025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 0x55ec4e050d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.706+0000 7f20d2e04740 1 -- 172.21.15.5:0/1269626025 >> 172.21.15.5:0/1269626025 conn(0x55ec4e00f310 msgr2=0x55ec4e048960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.706+0000 7f20b1ffb640 0 client.18345 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.707+0000 7f20d2e04740 1 -- 172.21.15.5:0/1269626025 shutdown_connections 2024-09-19T06:14:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.707+0000 7f20d2e04740 1 -- 172.21.15.5:0/1269626025 wait complete. 2024-09-19T06:14:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.707+0000 7f20d2e04740 20 client.18345 trim_cache size 0 max 16384 2024-09-19T06:14:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.719+0000 7f20d2e04740 1 Processor -- start 2024-09-19T06:14:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20d2e04740 1 -- start start 2024-09-19T06:14:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20d2e04740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 0x55ec4e05d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20d2e04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 0x55ec4e059860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20d2e04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e051370 0x55ec4e059da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20d2e04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1ad900 con 0x55ec4e047b40 2024-09-19T06:14:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20d2e04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1af500 con 0x55ec4e051370 2024-09-19T06:14:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20d2e04740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1adfd0 con 0x55ec4e0465d0 2024-09-19T06:14:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20ca59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 0x55ec4e059860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20ca59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 0x55ec4e059860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41820/0 (socket says 172.21.15.5:41820) 2024-09-19T06:14:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20ca59c640 1 -- 172.21.15.5:0/2202038510 learned_addr learned my addr 172.21.15.5:0/2202038510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/2202038510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 0x55ec4e05d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/2202038510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e051370 0x55ec4e059da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20ca59c640 1 -- 172.21.15.5:0/2202038510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 msgr2=0x55ec4e05d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20ca59c640 1 --2- 172.21.15.5:0/2202038510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 0x55ec4e05d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20ca59c640 1 -- 172.21.15.5:0/2202038510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e051370 msgr2=0x55ec4e059da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20ca59c640 1 --2- 172.21.15.5:0/2202038510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e051370 0x55ec4e059da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20ca59c640 1 -- 172.21.15.5:0/2202038510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4e047750 con 0x55ec4e047b40 2024-09-19T06:14:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/2202038510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e051370 0x55ec4e059da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.720+0000 7f20ca59c640 1 --2- 172.21.15.5:0/2202038510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 0x55ec4e059860 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f20bc031440 tx=0x7f20bc0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ed76a0a45a0d4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.721+0000 7f20b3fff640 1 -- 172.21.15.5:0/2202038510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc035040 con 0x55ec4e047b40 2024-09-19T06:14:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.721+0000 7f20b3fff640 1 -- 172.21.15.5:0/2202038510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20bc005040 con 0x55ec4e047b40 2024-09-19T06:14:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.721+0000 7f20b3fff640 1 -- 172.21.15.5:0/2202038510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc004b50 con 0x55ec4e047b40 2024-09-19T06:14:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.721+0000 7f20d2e04740 1 -- 172.21.15.5:0/2202038510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 msgr2=0x55ec4e059860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.721+0000 7f20d2e04740 1 --2- 172.21.15.5:0/2202038510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 0x55ec4e059860 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f20bc031440 tx=0x7f20bc0092d0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.721+0000 7f20d2e04740 1 -- 172.21.15.5:0/2202038510 shutdown_connections 2024-09-19T06:14:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.721+0000 7f20d2e04740 1 --2- 172.21.15.5:0/2202038510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e051370 0x55ec4e059da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.721+0000 7f20d2e04740 1 --2- 172.21.15.5:0/2202038510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e047b40 0x55ec4e059860 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.721+0000 7f20d2e04740 1 --2- 172.21.15.5:0/2202038510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e0465d0 0x55ec4e05d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.721+0000 7f20d2e04740 1 -- 172.21.15.5:0/2202038510 >> 172.21.15.5:0/2202038510 conn(0x55ec4e00f310 msgr2=0x55ec4e04d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.721+0000 7f20d2e04740 1 -- 172.21.15.5:0/2202038510 shutdown_connections 2024-09-19T06:14:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.721+0000 7f20d2e04740 1 -- 172.21.15.5:0/2202038510 wait complete. 2024-09-19T06:14:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20d2e04740 1 -- start start 2024-09-19T06:14:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20d2e04740 1 Processor -- start 2024-09-19T06:14:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20d2e04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e0465d0 0x55ec4e068940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20d2e04740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e047b40 0x55ec4e068e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20d2e04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 0x55ec4e0c7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20d2e04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1aee30 con 0x55ec4e051370 2024-09-19T06:14:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20d2e04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1ae700 con 0x55ec4e0465d0 2024-09-19T06:14:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20d2e04740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1ab900 con 0x55ec4e047b40 2024-09-19T06:14:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20d2e04740 10 client.0 _ll_register_callbacks cb 0x7f20c000cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20c9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 0x55ec4e0c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20c9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 0x55ec4e0c7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41834/0 (socket says 172.21.15.5:41834) 2024-09-19T06:14:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20c9d9b640 1 -- 172.21.15.5:0/3270107265 learned_addr learned my addr 172.21.15.5:0/3270107265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e0465d0 0x55ec4e068940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20ca59c640 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e047b40 0x55ec4e068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20c9d9b640 1 -- 172.21.15.5:0/3270107265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e047b40 msgr2=0x55ec4e068e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e047b40 0x55ec4e068e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20c9d9b640 1 -- 172.21.15.5:0/3270107265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e0465d0 msgr2=0x55ec4e068940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e0465d0 0x55ec4e068940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.722+0000 7f20c9d9b640 1 -- 172.21.15.5:0/3270107265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4e0c76f0 con 0x55ec4e051370 2024-09-19T06:14:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20c9d9b640 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 0x55ec4e0c7050 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f20c002db00 tx=0x7f20c0004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46ed92c412f00a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20b1ffb640 1 -- 172.21.15.5:0/3270107265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c001e040 con 0x55ec4e051370 2024-09-19T06:14:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20b1ffb640 1 -- 172.21.15.5:0/3270107265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f20c0005330 con 0x55ec4e051370 2024-09-19T06:14:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20d2e04740 1 -- 172.21.15.5:0/3270107265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20bc005040 con 0x55ec4e051370 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20b1ffb640 1 -- 172.21.15.5:0/3270107265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f20c0024040 con 0x55ec4e051370 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20d2e04740 10 client.18351 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20d2e04740 1 -- 172.21.15.5:0/3270107265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ec4e0cfc60 con 0x55ec4e051370 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20b1ffb640 1 -- 172.21.15.5:0/3270107265 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20bc005040 con 0x55ec4e051370 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20b1ffb640 1 -- 172.21.15.5:0/3270107265 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f20c00136a0 con 0x55ec4e051370 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20d2e04740 10 client.18351 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20d2e04740 10 client.18351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20d2e04740 20 client.18351 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20d2e04740 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ec4e0d02f0 0x55ec4e0d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20d2e04740 4 client.18351 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.723+0000 7f20d2e04740 1 -- 172.21.15.5:0/3270107265 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec4dfb3d70 con 0x55ec4e0d02f0 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.724+0000 7f20ca59c640 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ec4e0d02f0 0x55ec4e0d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.724+0000 7f20b1ffb640 10 client.18351 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.724+0000 7f20ca59c640 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ec4e0d02f0 0x55ec4e0d2710 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b32ea068a79a8fd server_cookie=7d8e3ff695dbab6f in_seq=0 out_seq=0 2024-09-19T06:14:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.724+0000 7f20b1ffb640 1 -- 172.21.15.5:0/3270107265 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec4dfb3d70 con 0x55ec4e0d02f0 2024-09-19T06:14:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.724+0000 7f20b1ffb640 10 client.18351 handle_command_reply: tid=0 2024-09-19T06:14:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.724+0000 7f20d2e04740 1 client.18351 shutdown 2024-09-19T06:14:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.724+0000 7f20d2e04740 1 -- 172.21.15.5:0/3270107265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 msgr2=0x55ec4e0c7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.724+0000 7f20d2e04740 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 0x55ec4e0c7050 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f20c002db00 tx=0x7f20c0004f30 comp rx=0 tx=0).stop 2024-09-19T06:14:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d2e04740 1 -- 172.21.15.5:0/3270107265 shutdown_connections 2024-09-19T06:14:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d2e04740 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ec4e0d02f0 0x55ec4e0d2710 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20b1ffb640 0 client.18351 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d2e04740 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec4e051370 0x55ec4e0c7050 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d2e04740 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec4e047b40 0x55ec4e068e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d2e04740 1 --2- 172.21.15.5:0/3270107265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4e0465d0 0x55ec4e068940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d2e04740 1 -- 172.21.15.5:0/3270107265 >> 172.21.15.5:0/3270107265 conn(0x55ec4e00f310 msgr2=0x55ec4e04f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d2e04740 1 -- 172.21.15.5:0/3270107265 shutdown_connections 2024-09-19T06:14:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d2e04740 1 -- 172.21.15.5:0/3270107265 wait complete. 2024-09-19T06:14:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d2e04740 20 client.18351 trim_cache size 0 max 16384 2024-09-19T06:14:28.736 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:14:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d0a2b640 1 -- 172.21.15.5:0/2450572968 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20b40a33b0 msgr2=0x7f20b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d0a2b640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20b40a33b0 0x7f20b40a5870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f20c40985f0 tx=0x7f20c408a040 comp rx=0 tx=0).stop 2024-09-19T06:14:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d0a2b640 1 -- 172.21.15.5:0/2450572968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20cc10c170 msgr2=0x7f20cc19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d0a2b640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20cc10c170 0x7f20cc19cc50 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f20c400eaa0 tx=0x7f20c400ef60 comp rx=0 tx=0).stop 2024-09-19T06:14:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d0a2b640 1 -- 172.21.15.5:0/2450572968 shutdown_connections 2024-09-19T06:14:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d0a2b640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20cc10c170 0x7f20cc19cc50 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d0a2b640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20b40a33b0 0x7f20b40a5870 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d0a2b640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20cc103660 0x7f20cc1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d0a2b640 1 --2- 172.21.15.5:0/2450572968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20cc102d40 0x7f20cc198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d0a2b640 1 -- 172.21.15.5:0/2450572968 >> 172.21.15.5:0/2450572968 conn(0x7f20cc0fc7a0 msgr2=0x7f20cc0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d0a2b640 1 -- 172.21.15.5:0/2450572968 shutdown_connections 2024-09-19T06:14:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:28.725+0000 7f20d0a2b640 1 -- 172.21.15.5:0/2450572968 wait complete. 2024-09-19T06:14:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:28 smithi157 ceph-mon[35346]: pgmap v347: 97 pgs: 97 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-09-19T06:14:28.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 262651 2.28 MB/sec execute 474 sec latency 3115.594 ms 2024-09-19T06:14:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:28 smithi049 ceph-mon[34640]: pgmap v347: 97 pgs: 97 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-09-19T06:14:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.650+0000 7fc823759640 1 -- 172.21.15.5:0/278507032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c102d00 msgr2=0x7fc81c1030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.650+0000 7fc823759640 1 --2- 172.21.15.5:0/278507032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c102d00 0x7fc81c1030e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc818009900 tx=0x7fc81802a480 comp rx=0 tx=0).stop 2024-09-19T06:14:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.650+0000 7fc823759640 1 -- 172.21.15.5:0/278507032 shutdown_connections 2024-09-19T06:14:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.650+0000 7fc823759640 1 --2- 172.21.15.5:0/278507032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc81c10c0d0 0x7fc81c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.650+0000 7fc823759640 1 --2- 172.21.15.5:0/278507032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c103620 0x7fc81c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.650+0000 7fc823759640 1 --2- 172.21.15.5:0/278507032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c102d00 0x7fc81c1030e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.650+0000 7fc823759640 1 -- 172.21.15.5:0/278507032 >> 172.21.15.5:0/278507032 conn(0x7fc81c0fc6b0 msgr2=0x7fc81c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.650+0000 7fc823759640 1 -- 172.21.15.5:0/278507032 shutdown_connections 2024-09-19T06:14:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.650+0000 7fc823759640 1 -- 172.21.15.5:0/278507032 wait complete. 2024-09-19T06:14:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc823759640 1 Processor -- start 2024-09-19T06:14:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc823759640 1 -- start start 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc823759640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc81c102d00 0x7fc81c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc823759640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c103620 0x7fc81c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc823759640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c10c0d0 0x7fc81c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc823759640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c111dd0 con 0x7fc81c103620 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc823759640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c111c50 con 0x7fc81c102d00 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc823759640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c111f50 con 0x7fc81c10c0d0 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc820cf4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c103620 0x7fc81c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc820cf4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c103620 0x7fc81c198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41844/0 (socket says 172.21.15.5:41844) 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc820cf4640 1 -- 172.21.15.5:0/3919314947 learned_addr learned my addr 172.21.15.5:0/3919314947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc8214f5640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc81c102d00 0x7fc81c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc821cf6640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c10c0d0 0x7fc81c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc820cf4640 1 -- 172.21.15.5:0/3919314947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c10c0d0 msgr2=0x7fc81c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc820cf4640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c10c0d0 0x7fc81c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc820cf4640 1 -- 172.21.15.5:0/3919314947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc81c102d00 msgr2=0x7fc81c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc820cf4640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc81c102d00 0x7fc81c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.651+0000 7fc820cf4640 1 -- 172.21.15.5:0/3919314947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81c19d3b0 con 0x7fc81c103620 2024-09-19T06:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.652+0000 7fc820cf4640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c103620 0x7fc81c198900 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00d770 tx=0x7fc80c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.652+0000 7fc8127fc640 1 -- 172.21.15.5:0/3919314947 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c017070 con 0x7fc81c103620 2024-09-19T06:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.652+0000 7fc8127fc640 1 -- 172.21.15.5:0/3919314947 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc80c0054d0 con 0x7fc81c103620 2024-09-19T06:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.652+0000 7fc823759640 1 -- 172.21.15.5:0/3919314947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81c19d6a0 con 0x7fc81c103620 2024-09-19T06:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.652+0000 7fc8127fc640 1 -- 172.21.15.5:0/3919314947 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c004cf0 con 0x7fc81c103620 2024-09-19T06:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.652+0000 7fc823759640 1 -- 172.21.15.5:0/3919314947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc81c1a4f30 con 0x7fc81c103620 2024-09-19T06:14:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.655+0000 7fc8265d3740 1 Processor -- start 2024-09-19T06:14:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.655+0000 7fc8265d3740 1 -- start start 2024-09-19T06:14:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.655+0000 7fc8127fc640 1 -- 172.21.15.5:0/3919314947 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc80c004e90 con 0x7fc81c103620 2024-09-19T06:14:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.655+0000 7fc8265d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 0x55b1572d6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.655+0000 7fc8265d3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d7410 0x55b1572e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.655+0000 7fc8265d3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572e1cb0 0x55b1572e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.655+0000 7fc8265d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b15723b2f0 con 0x55b1572d6af0 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.655+0000 7fc8265d3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b1571e3270 con 0x55b1572e1cb0 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.655+0000 7fc8265d3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b1572372b0 con 0x55b1572d7410 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc821cf6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 0x55b1572d6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc821cf6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 0x55b1572d6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41846/0 (socket says 172.21.15.5:41846) 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc821cf6640 1 -- 172.21.15.5:0/2660484478 learned_addr learned my addr 172.21.15.5:0/2660484478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc8214f5640 1 --2- 172.21.15.5:0/2660484478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d7410 0x55b1572e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc820cf4640 1 --2- 172.21.15.5:0/2660484478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572e1cb0 0x55b1572e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc821cf6640 1 -- 172.21.15.5:0/2660484478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d7410 msgr2=0x55b1572e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc821cf6640 1 --2- 172.21.15.5:0/2660484478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d7410 0x55b1572e1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc821cf6640 1 -- 172.21.15.5:0/2660484478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572e1cb0 msgr2=0x55b1572e40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc821cf6640 1 --2- 172.21.15.5:0/2660484478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572e1cb0 0x55b1572e40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc821cf6640 1 -- 172.21.15.5:0/2660484478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1571e2340 con 0x55b1572d6af0 2024-09-19T06:14:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc821cf6640 1 --2- 172.21.15.5:0/2660484478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 0x55b1572d6ed0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc814009f20 tx=0x7fc81400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad63605cf90c30ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc8127fc640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc8080a3430 0x7fc8080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc8077fe640 1 -- 172.21.15.5:0/2660484478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814015070 con 0x55b1572d6af0 2024-09-19T06:14:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc8077fe640 1 -- 172.21.15.5:0/2660484478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc81401ab10 con 0x55b1572d6af0 2024-09-19T06:14:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc8077fe640 1 -- 172.21.15.5:0/2660484478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814011bb0 con 0x55b1572d6af0 2024-09-19T06:14:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc8214f5640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc8080a3430 0x7fc8080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.656+0000 7fc8127fc640 1 -- 172.21.15.5:0/3919314947 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fc80c0cf730 con 0x7fc81c103620 2024-09-19T06:14:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.657+0000 7fc8265d3740 1 -- 172.21.15.5:0/2660484478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 msgr2=0x55b1572d6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.657+0000 7fc8214f5640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc8080a3430 0x7fc8080a58f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc818005ab0 tx=0x7fc818005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.657+0000 7fc8265d3740 1 --2- 172.21.15.5:0/2660484478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 0x55b1572d6ed0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc814009f20 tx=0x7fc81400f7f0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.657+0000 7fc8265d3740 1 -- 172.21.15.5:0/2660484478 shutdown_connections 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.657+0000 7fc8265d3740 1 --2- 172.21.15.5:0/2660484478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572e1cb0 0x55b1572e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.657+0000 7fc8265d3740 1 --2- 172.21.15.5:0/2660484478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d7410 0x55b1572e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.657+0000 7fc8265d3740 1 --2- 172.21.15.5:0/2660484478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 0x55b1572d6ed0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.657+0000 7fc8265d3740 1 -- 172.21.15.5:0/2660484478 >> 172.21.15.5:0/2660484478 conn(0x55b1572d3060 msgr2=0x55b1572d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.657+0000 7fc8265d3740 1 -- 172.21.15.5:0/2660484478 shutdown_connections 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.657+0000 7fc8265d3740 1 -- 172.21.15.5:0/2660484478 wait complete. 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc8265d3740 1 -- start start 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc8265d3740 1 Processor -- start 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc8265d3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d6af0 0x55b1572deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc8265d3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572d7410 0x55b1572df0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc8265d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572e1cb0 0x55b1572df600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc8265d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b1572e4880 con 0x55b1572e1cb0 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc8265d3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b1571c4290 con 0x55b1572d7410 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc8265d3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b1572e4a00 con 0x55b1572d6af0 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc8265d3740 10 client.0 _ll_register_callbacks cb 0x55b1571f5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc821cf6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572e1cb0 0x55b1572df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc821cf6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572e1cb0 0x55b1572df600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41862/0 (socket says 172.21.15.5:41862) 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc821cf6640 1 -- 172.21.15.5:0/152414673 learned_addr learned my addr 172.21.15.5:0/152414673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc8214f5640 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d6af0 0x55b1572deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc821cf6640 1 -- 172.21.15.5:0/152414673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d6af0 msgr2=0x55b1572deb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc821cf6640 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d6af0 0x55b1572deb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc821cf6640 1 -- 172.21.15.5:0/152414673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572d7410 msgr2=0x55b1572df0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc820cf4640 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572d7410 0x55b1572df0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc821cf6640 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572d7410 0x55b1572df0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.658+0000 7fc821cf6640 1 -- 172.21.15.5:0/152414673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b156f4e210 con 0x55b1572e1cb0 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc821cf6640 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572e1cb0 0x55b1572df600 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc814000f80 tx=0x7fc814004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89770f4f29423262 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc804ff9640 1 -- 172.21.15.5:0/152414673 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814028060 con 0x55b1572e1cb0 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc804ff9640 1 -- 172.21.15.5:0/152414673 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fc81401a460 con 0x55b1572e1cb0 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc8265d3740 1 -- 172.21.15.5:0/152414673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc81401ab10 con 0x55b1572e1cb0 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc804ff9640 1 -- 172.21.15.5:0/152414673 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc814015070 con 0x55b1572e1cb0 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc8265d3740 10 client.18363 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc8265d3740 1 -- 172.21.15.5:0/152414673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b15701a620 con 0x55b1572e1cb0 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc804ff9640 1 -- 172.21.15.5:0/152414673 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc81401ab10 con 0x55b1572e1cb0 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc804ff9640 1 -- 172.21.15.5:0/152414673 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fc814005040 con 0x55b1572e1cb0 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc8265d3740 10 client.18363 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc8265d3740 10 client.18363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.659+0000 7fc8265d3740 20 client.18363 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.660+0000 7fc8265d3740 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b1572f1bf0 0x55b1572f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.660+0000 7fc8265d3740 4 client.18363 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.660+0000 7fc8265d3740 1 -- 172.21.15.5:0/152414673 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b157248a20 con 0x55b1572f1bf0 2024-09-19T06:14:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.660+0000 7fc8214f5640 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b1572f1bf0 0x55b1572f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.660+0000 7fc804ff9640 10 client.18363 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.660+0000 7fc8214f5640 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b1572f1bf0 0x55b1572f3fe0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6008ba05b9b840d7 server_cookie=6ac9376155829c77 in_seq=0 out_seq=0 2024-09-19T06:14:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc804ff9640 1 -- 172.21.15.5:0/152414673 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b157248a20 con 0x55b1572f1bf0 2024-09-19T06:14:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc804ff9640 10 client.18363 handle_command_reply: tid=0 2024-09-19T06:14:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc8265d3740 1 client.18363 shutdown 2024-09-19T06:14:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc8265d3740 1 -- 172.21.15.5:0/152414673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572e1cb0 msgr2=0x55b1572df600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc8265d3740 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572e1cb0 0x55b1572df600 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc814000f80 tx=0x7fc814004a40 comp rx=0 tx=0).stop 2024-09-19T06:14:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc8265d3740 1 -- 172.21.15.5:0/152414673 shutdown_connections 2024-09-19T06:14:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc8265d3740 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b1572f1bf0 0x55b1572f3fe0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc8265d3740 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572e1cb0 0x55b1572df600 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc8265d3740 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572d7410 0x55b1572df0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc804ff9640 0 client.18363 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc8265d3740 1 --2- 172.21.15.5:0/152414673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d6af0 0x55b1572deb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc8265d3740 1 -- 172.21.15.5:0/152414673 >> 172.21.15.5:0/152414673 conn(0x55b1572d3060 msgr2=0x55b1572d8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc8265d3740 1 -- 172.21.15.5:0/152414673 shutdown_connections 2024-09-19T06:14:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc8265d3740 1 -- 172.21.15.5:0/152414673 wait complete. 2024-09-19T06:14:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.661+0000 7fc8265d3740 20 client.18363 trim_cache size 0 max 16384 2024-09-19T06:14:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8265d3740 1 Processor -- start 2024-09-19T06:14:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8265d3740 1 -- start start 2024-09-19T06:14:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8265d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 0x55b1572d9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8265d3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d7410 0x55b1572f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8265d3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572e1cb0 0x55b1572f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8265d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1ad760 con 0x55b1572d6af0 2024-09-19T06:14:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8265d3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1af360 con 0x55b1572e1cb0 2024-09-19T06:14:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8265d3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1ade30 con 0x55b1572d7410 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8214f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 0x55b1572d9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8214f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 0x55b1572d9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41872/0 (socket says 172.21.15.5:41872) 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8214f5640 1 -- 172.21.15.5:0/3314345543 learned_addr learned my addr 172.21.15.5:0/3314345543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc821cf6640 1 --2- 172.21.15.5:0/3314345543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572e1cb0 0x55b1572f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc820cf4640 1 --2- 172.21.15.5:0/3314345543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d7410 0x55b1572f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8214f5640 1 -- 172.21.15.5:0/3314345543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d7410 msgr2=0x55b1572f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8214f5640 1 --2- 172.21.15.5:0/3314345543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d7410 0x55b1572f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8214f5640 1 -- 172.21.15.5:0/3314345543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572e1cb0 msgr2=0x55b1572f9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8214f5640 1 --2- 172.21.15.5:0/3314345543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572e1cb0 0x55b1572f9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.676+0000 7fc8214f5640 1 -- 172.21.15.5:0/3314345543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1572d7240 con 0x55b1572d6af0 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc8214f5640 1 --2- 172.21.15.5:0/3314345543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 0x55b1572d9560 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc818005c40 tx=0x7fc81803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c673337950a72806 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc806ffd640 1 -- 172.21.15.5:0/3314345543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818035040 con 0x55b1572d6af0 2024-09-19T06:14:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc806ffd640 1 -- 172.21.15.5:0/3314345543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc818004ec0 con 0x55b1572d6af0 2024-09-19T06:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc806ffd640 1 -- 172.21.15.5:0/3314345543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81803d6b0 con 0x55b1572d6af0 2024-09-19T06:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc8265d3740 1 -- 172.21.15.5:0/3314345543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 msgr2=0x55b1572d9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc8265d3740 1 --2- 172.21.15.5:0/3314345543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 0x55b1572d9560 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc818005c40 tx=0x7fc81803f000 comp rx=0 tx=0).stop 2024-09-19T06:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc8265d3740 1 -- 172.21.15.5:0/3314345543 shutdown_connections 2024-09-19T06:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc8265d3740 1 --2- 172.21.15.5:0/3314345543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572e1cb0 0x55b1572f9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc8265d3740 1 --2- 172.21.15.5:0/3314345543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572d7410 0x55b1572f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc8265d3740 1 --2- 172.21.15.5:0/3314345543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d6af0 0x55b1572d9560 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc8265d3740 1 -- 172.21.15.5:0/3314345543 >> 172.21.15.5:0/3314345543 conn(0x55b1572d3060 msgr2=0x55b1572e0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc8265d3740 1 -- 172.21.15.5:0/3314345543 shutdown_connections 2024-09-19T06:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.677+0000 7fc8265d3740 1 -- 172.21.15.5:0/3314345543 wait complete. 2024-09-19T06:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc8265d3740 1 -- start start 2024-09-19T06:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc8265d3740 1 Processor -- start 2024-09-19T06:14:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc8265d3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572d6af0 0x55b1572e7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc8265d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d7410 0x55b1572e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc8265d3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572e1cb0 0x55b15735a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc8265d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1aec90 con 0x55b1572d7410 2024-09-19T06:14:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc8265d3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1ae560 con 0x55b1572d6af0 2024-09-19T06:14:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc8265d3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1ab760 con 0x55b1572e1cb0 2024-09-19T06:14:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc8265d3740 10 client.0 _ll_register_callbacks cb 0x7fc818005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc820cf4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d7410 0x55b1572e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc820cf4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d7410 0x55b1572e7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41876/0 (socket says 172.21.15.5:41876) 2024-09-19T06:14:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc820cf4640 1 -- 172.21.15.5:0/1277645620 learned_addr learned my addr 172.21.15.5:0/1277645620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc8214f5640 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572d6af0 0x55b1572e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc821cf6640 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572e1cb0 0x55b15735a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc820cf4640 1 -- 172.21.15.5:0/1277645620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572e1cb0 msgr2=0x55b15735a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc820cf4640 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572e1cb0 0x55b15735a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc820cf4640 1 -- 172.21.15.5:0/1277645620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572d6af0 msgr2=0x55b1572e7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc820cf4640 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572d6af0 0x55b1572e7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc820cf4640 1 -- 172.21.15.5:0/1277645620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b15735ab70 con 0x55b1572d7410 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc8214f5640 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572d6af0 0x55b1572e7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.678+0000 7fc821cf6640 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572e1cb0 0x55b15735a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc820cf4640 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d7410 0x55b1572e7970 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009610 tx=0x7fc80c09b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb2ebe0b06cc084 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc804ff9640 1 -- 172.21.15.5:0/1277645620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c020040 con 0x55b1572d7410 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc804ff9640 1 -- 172.21.15.5:0/1277645620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fc80c098060 con 0x55b1572d7410 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc8265d3740 1 -- 172.21.15.5:0/1277645620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc818004ec0 con 0x55b1572d7410 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc804ff9640 1 -- 172.21.15.5:0/1277645620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc80c0b4050 con 0x55b1572d7410 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc8265d3740 10 client.18369 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc8265d3740 1 -- 172.21.15.5:0/1277645620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b15735be60 con 0x55b1572d7410 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc804ff9640 1 -- 172.21.15.5:0/1277645620 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc818004ec0 con 0x55b1572d7410 2024-09-19T06:14:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc804ff9640 1 -- 172.21.15.5:0/1277645620 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fc80c0cfbe0 con 0x55b1572d7410 2024-09-19T06:14:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc8265d3740 10 client.18369 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc8265d3740 10 client.18369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc8265d3740 20 client.18369 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc8265d3740 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b15735c640 0x55b15735ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc8265d3740 4 client.18369 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:14:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.679+0000 7fc8265d3740 1 -- 172.21.15.5:0/1277645620 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b157242d70 con 0x55b15735c640 2024-09-19T06:14:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.680+0000 7fc8214f5640 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b15735c640 0x55b15735ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.680+0000 7fc804ff9640 10 client.18369 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.680+0000 7fc8214f5640 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b15735c640 0x55b15735ea60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3447915fbc49b25 server_cookie=a87a45fb452a658f in_seq=0 out_seq=0 2024-09-19T06:14:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc804ff9640 1 -- 172.21.15.5:0/1277645620 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b157242d70 con 0x55b15735c640 2024-09-19T06:14:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc804ff9640 10 client.18369 handle_command_reply: tid=0 2024-09-19T06:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc8265d3740 1 client.18369 shutdown 2024-09-19T06:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc8265d3740 1 -- 172.21.15.5:0/1277645620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d7410 msgr2=0x55b1572e7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc8265d3740 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d7410 0x55b1572e7970 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009610 tx=0x7fc80c09b7c0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc8265d3740 1 -- 172.21.15.5:0/1277645620 shutdown_connections 2024-09-19T06:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc8265d3740 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b15735c640 0x55b15735ea60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc8265d3740 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1572e1cb0 0x55b15735a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc804ff9640 0 client.18369 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc8265d3740 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1572d7410 0x55b1572e7970 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc8265d3740 1 --2- 172.21.15.5:0/1277645620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1572d6af0 0x55b1572e7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc8265d3740 1 -- 172.21.15.5:0/1277645620 >> 172.21.15.5:0/1277645620 conn(0x55b1572d3060 msgr2=0x55b1572d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc8265d3740 1 -- 172.21.15.5:0/1277645620 shutdown_connections 2024-09-19T06:14:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc8265d3740 1 -- 172.21.15.5:0/1277645620 wait complete. 2024-09-19T06:14:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.711+0000 7fc8265d3740 20 client.18369 trim_cache size 0 max 16384 2024-09-19T06:14:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:14:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:14:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:14:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:14:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.712+0000 7fc823759640 1 -- 172.21.15.5:0/3919314947 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc8080a3430 msgr2=0x7fc8080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.712+0000 7fc823759640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc8080a3430 0x7fc8080a58f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc818005ab0 tx=0x7fc818005a20 comp rx=0 tx=0).stop 2024-09-19T06:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.712+0000 7fc823759640 1 -- 172.21.15.5:0/3919314947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c103620 msgr2=0x7fc81c198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.712+0000 7fc823759640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c103620 0x7fc81c198900 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00d770 tx=0x7fc80c00dc30 comp rx=0 tx=0).stop 2024-09-19T06:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.712+0000 7fc823759640 1 -- 172.21.15.5:0/3919314947 shutdown_connections 2024-09-19T06:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.712+0000 7fc823759640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc8080a3430 0x7fc8080a58f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.712+0000 7fc823759640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c10c0d0 0x7fc81c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.712+0000 7fc823759640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c103620 0x7fc81c198900 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.712+0000 7fc823759640 1 --2- 172.21.15.5:0/3919314947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc81c102d00 0x7fc81c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.712+0000 7fc823759640 1 -- 172.21.15.5:0/3919314947 >> 172.21.15.5:0/3919314947 conn(0x7fc81c0fc6b0 msgr2=0x7fc81c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.712+0000 7fc823759640 1 -- 172.21.15.5:0/3919314947 shutdown_connections 2024-09-19T06:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:29.712+0000 7fc823759640 1 -- 172.21.15.5:0/3919314947 wait complete. 2024-09-19T06:14:29.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 262651 2.28 MB/sec execute 475 sec latency 4115.712 ms 2024-09-19T06:14:30.457 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:14:30.457 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:14:30.738 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:14:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:30 smithi157 ceph-mon[35346]: pgmap v348: 97 pgs: 97 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 828 KiB/s wr, 80 op/s 2024-09-19T06:14:30.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 262767 2.27 MB/sec execute 476 sec latency 4735.291 ms 2024-09-19T06:14:30.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:30 smithi005 ceph-mon[30955]: pgmap v348: 97 pgs: 97 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 828 KiB/s wr, 80 op/s 2024-09-19T06:14:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:30 smithi049 ceph-mon[34640]: pgmap v348: 97 pgs: 97 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 828 KiB/s wr, 80 op/s 2024-09-19T06:14:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.245+0000 7f79934c1640 1 -- 172.21.15.5:0/846504355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f798c102d30 msgr2=0x7f798c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.245+0000 7f79934c1640 1 --2- 172.21.15.5:0/846504355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f798c102d30 0x7f798c103110 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7988009930 tx=0x7f798802a420 comp rx=0 tx=0).stop 2024-09-19T06:14:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.245+0000 7f79934c1640 1 -- 172.21.15.5:0/846504355 shutdown_connections 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.245+0000 7f79934c1640 1 --2- 172.21.15.5:0/846504355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798c1082a0 0x7f798c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.245+0000 7f79934c1640 1 --2- 172.21.15.5:0/846504355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f798c1036e0 0x7f798c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.245+0000 7f79934c1640 1 --2- 172.21.15.5:0/846504355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f798c102d30 0x7f798c103110 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.245+0000 7f79934c1640 1 -- 172.21.15.5:0/846504355 >> 172.21.15.5:0/846504355 conn(0x7f798c0fc7d0 msgr2=0x7f798c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.245+0000 7f79934c1640 1 -- 172.21.15.5:0/846504355 shutdown_connections 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.245+0000 7f79934c1640 1 -- 172.21.15.5:0/846504355 wait complete. 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.246+0000 7f79934c1640 1 Processor -- start 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.246+0000 7f79934c1640 1 -- start start 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f79934c1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798c102d30 0x7f798c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f79934c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f798c1036e0 0x7f798c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f79934c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f798c1082a0 0x7f798c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f79934c1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f798c111f50 con 0x7f798c1082a0 2024-09-19T06:14:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f79934c1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f798c111dd0 con 0x7f798c102d30 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f79934c1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1120d0 con 0x7f798c1036e0 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f7991a5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f798c1082a0 0x7f798c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f799125d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798c102d30 0x7f798c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f7991a5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f798c1082a0 0x7f798c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41900/0 (socket says 172.21.15.5:41900) 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f7991a5e640 1 -- 172.21.15.5:0/260376335 learned_addr learned my addr 172.21.15.5:0/260376335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f7990a5c640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f798c1036e0 0x7f798c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f7991a5e640 1 -- 172.21.15.5:0/260376335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f798c1036e0 msgr2=0x7f798c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f7991a5e640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f798c1036e0 0x7f798c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f7991a5e640 1 -- 172.21.15.5:0/260376335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798c102d30 msgr2=0x7f798c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f7991a5e640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798c102d30 0x7f798c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f7991a5e640 1 -- 172.21.15.5:0/260376335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c19d4e0 con 0x7f798c1082a0 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f7991a5e640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f798c1082a0 0x7f798c19ce00 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7984009fd0 tx=0x7f798400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.247+0000 7f79827fc640 1 -- 172.21.15.5:0/260376335 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984017070 con 0x7f798c1082a0 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.248+0000 7f79827fc640 1 -- 172.21.15.5:0/260376335 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7984003e20 con 0x7f798c1082a0 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.248+0000 7f79934c1640 1 -- 172.21.15.5:0/260376335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798c19d7d0 con 0x7f798c1082a0 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.248+0000 7f79827fc640 1 -- 172.21.15.5:0/260376335 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984010460 con 0x7f798c1082a0 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.248+0000 7f79934c1640 1 -- 172.21.15.5:0/260376335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798c1a50b0 con 0x7f798c1082a0 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.250+0000 7f79827fc640 1 -- 172.21.15.5:0/260376335 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7984003970 con 0x7f798c1082a0 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f799589a740 1 Processor -- start 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f799589a740 1 -- start start 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f799589a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x56162981bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f799589a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56162981c480 0x561629826800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f799589a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629826d40 0x561629829130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f799589a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561629781280 con 0x561629826d40 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f799589a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561629728b10 con 0x56162981bb60 2024-09-19T06:14:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f799589a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56162977d240 con 0x56162981c480 2024-09-19T06:14:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f7990a5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629826d40 0x561629829130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f7990a5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629826d40 0x561629829130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41914/0 (socket says 172.21.15.5:41914) 2024-09-19T06:14:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f79827fc640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f79780a3430 0x7f79780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f7990a5c640 1 -- 172.21.15.5:0/2153164956 learned_addr learned my addr 172.21.15.5:0/2153164956 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f799125d640 1 --2- 172.21.15.5:0/2153164956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56162981c480 0x561629826800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f7990a5c640 1 --2- 172.21.15.5:0/2153164956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x56162981bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.251+0000 7f7991a5e640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f79780a3430 0x7f79780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.252+0000 7f79827fc640 1 -- 172.21.15.5:0/260376335 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f79840cfac0 con 0x7f798c1082a0 2024-09-19T06:14:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.252+0000 7f7990a5c640 1 -- 172.21.15.5:0/2153164956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56162981c480 msgr2=0x561629826800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.252+0000 7f7990a5c640 1 --2- 172.21.15.5:0/2153164956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56162981c480 0x561629826800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.252+0000 7f7990a5c640 1 -- 172.21.15.5:0/2153164956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 msgr2=0x56162981bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.252+0000 7f7990a5c640 1 --2- 172.21.15.5:0/2153164956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x56162981bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.252+0000 7f7990a5c640 1 -- 172.21.15.5:0/2153164956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616297351f0 con 0x561629826d40 2024-09-19T06:14:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.252+0000 7f7990a5c640 1 --2- 172.21.15.5:0/2153164956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629826d40 0x561629829130 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f797c005bc0 tx=0x7f797c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578652a1091da5ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.252+0000 7f7991a5e640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f79780a3430 0x7f79780a58f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f79840cd2a0 tx=0x7f7984004390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.252+0000 7f79777fe640 1 -- 172.21.15.5:0/2153164956 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c00fdf0 con 0x561629826d40 2024-09-19T06:14:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.252+0000 7f79777fe640 1 -- 172.21.15.5:0/2153164956 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f797c016d10 con 0x561629826d40 2024-09-19T06:14:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.252+0000 7f79777fe640 1 -- 172.21.15.5:0/2153164956 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c0047a0 con 0x561629826d40 2024-09-19T06:14:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.253+0000 7f799589a740 1 -- 172.21.15.5:0/2153164956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629826d40 msgr2=0x561629829130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.253+0000 7f799589a740 1 --2- 172.21.15.5:0/2153164956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629826d40 0x561629829130 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f797c005bc0 tx=0x7f797c00c490 comp rx=0 tx=0).stop 2024-09-19T06:14:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.253+0000 7f799589a740 1 -- 172.21.15.5:0/2153164956 shutdown_connections 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.253+0000 7f799589a740 1 --2- 172.21.15.5:0/2153164956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629826d40 0x561629829130 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.253+0000 7f799589a740 1 --2- 172.21.15.5:0/2153164956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56162981c480 0x561629826800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.253+0000 7f799589a740 1 --2- 172.21.15.5:0/2153164956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x56162981bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.253+0000 7f799589a740 1 -- 172.21.15.5:0/2153164956 >> 172.21.15.5:0/2153164956 conn(0x5616298180f0 msgr2=0x5616298184d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.253+0000 7f799589a740 1 -- 172.21.15.5:0/2153164956 shutdown_connections 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.253+0000 7f799589a740 1 -- 172.21.15.5:0/2153164956 wait complete. 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.253+0000 7f799589a740 1 -- start start 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f799589a740 1 Processor -- start 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f799589a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x561629823c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f799589a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x561629824180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f799589a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x5616298246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f799589a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561629829910 con 0x56162981c480 2024-09-19T06:14:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f799589a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56162973bf10 con 0x56162981bb60 2024-09-19T06:14:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f799589a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561629829a90 con 0x561629826d40 2024-09-19T06:14:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f799589a740 10 client.0 _ll_register_callbacks cb 0x561629704a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f799125d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x561629824180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f799125d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x561629824180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41930/0 (socket says 172.21.15.5:41930) 2024-09-19T06:14:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f799125d640 1 -- 172.21.15.5:0/12403862 learned_addr learned my addr 172.21.15.5:0/12403862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f7990a5c640 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x561629823c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.254+0000 7f7990a5c640 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x5616298246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f799125d640 1 -- 172.21.15.5:0/12403862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 msgr2=0x5616298246c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f799125d640 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x5616298246c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f799125d640 1 -- 172.21.15.5:0/12403862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 msgr2=0x561629823c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f799125d640 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x561629823c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f799125d640 1 -- 172.21.15.5:0/12403862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616294f26a0 con 0x56162981c480 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f7990a5c640 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x5616298246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f799125d640 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x561629824180 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7988009930 tx=0x7f7988005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1877b9b456d62ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f7974ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f7974ff9640 1 -- 172.21.15.5:0/12403862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988035040 con 0x56162981c480 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f7974ff9640 1 -- 172.21.15.5:0/12403862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f7988004050 con 0x56162981c480 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f799589a740 1 -- 172.21.15.5:0/12403862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f797c016d10 con 0x56162981c480 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f7974ff9640 1 -- 172.21.15.5:0/12403862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7988004250 con 0x56162981c480 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f799589a740 10 client.18381 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.255+0000 7f799589a740 1 -- 172.21.15.5:0/12403862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561629835590 con 0x56162981c480 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.256+0000 7f7974ff9640 1 -- 172.21.15.5:0/12403862 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f797c016d10 con 0x56162981c480 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.256+0000 7f7974ff9640 1 -- 172.21.15.5:0/12403862 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f798803d500 con 0x56162981c480 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.256+0000 7f799589a740 10 client.18381 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.256+0000 7f799589a740 10 client.18381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.256+0000 7f799589a740 20 client.18381 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.256+0000 7f799589a740 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561629836af0 0x561629838f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.256+0000 7f799589a740 4 client.18381 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.256+0000 7f799589a740 1 -- 172.21.15.5:0/12403862 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162978e7f0 con 0x561629836af0 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.256+0000 7f7990a5c640 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561629836af0 0x561629838f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.256+0000 7f7990a5c640 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561629836af0 0x561629838f40 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a4b6f5033c4bb65 server_cookie=5f431ff662c3c911 in_seq=0 out_seq=0 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.256+0000 7f7974ff9640 10 client.18381 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.257+0000 7f7974ff9640 1 -- 172.21.15.5:0/12403862 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56162978e7f0 con 0x561629836af0 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.257+0000 7f7974ff9640 10 client.18381 handle_command_reply: tid=0 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.257+0000 7f799589a740 1 client.18381 shutdown 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.257+0000 7f799589a740 1 -- 172.21.15.5:0/12403862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 msgr2=0x561629824180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.257+0000 7f799589a740 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x561629824180 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7988009930 tx=0x7f7988005d90 comp rx=0 tx=0).stop 2024-09-19T06:14:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.257+0000 7f799589a740 1 -- 172.21.15.5:0/12403862 shutdown_connections 2024-09-19T06:14:31.384 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:14:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:14:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "061bc969-9bf3-4307-bcc4-ea8921cf3fba", 2024-09-19T06:14:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:14:31.384 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:14:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.257+0000 7f799589a740 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561629836af0 0x561629838f40 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.257+0000 7f799589a740 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x5616298246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.257+0000 7f799589a740 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x561629824180 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.257+0000 7f799589a740 1 --2- 172.21.15.5:0/12403862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x561629823c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.257+0000 7f799589a740 1 -- 172.21.15.5:0/12403862 >> 172.21.15.5:0/12403862 conn(0x5616298180f0 msgr2=0x56162981ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.257+0000 7f7974ff9640 0 client.18381 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.258+0000 7f799589a740 1 -- 172.21.15.5:0/12403862 shutdown_connections 2024-09-19T06:14:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.258+0000 7f799589a740 1 -- 172.21.15.5:0/12403862 wait complete. 2024-09-19T06:14:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.258+0000 7f799589a740 20 client.18381 trim_cache size 0 max 16384 2024-09-19T06:14:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799589a740 1 Processor -- start 2024-09-19T06:14:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799589a740 1 -- start start 2024-09-19T06:14:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799589a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x561629825aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799589a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x56162983e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799589a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x56162983ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799589a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1ab590 con 0x56162981c480 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799589a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1ad190 con 0x56162981bb60 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799589a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1abc60 con 0x561629826d40 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799125d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x56162983e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799125d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x56162983e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41940/0 (socket says 172.21.15.5:41940) 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799125d640 1 -- 172.21.15.5:0/3775736991 learned_addr learned my addr 172.21.15.5:0/3775736991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f7990a5c640 1 --2- 172.21.15.5:0/3775736991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x561629825aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f7990a5c640 1 --2- 172.21.15.5:0/3775736991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x56162983ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799125d640 1 -- 172.21.15.5:0/3775736991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 msgr2=0x56162983ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799125d640 1 --2- 172.21.15.5:0/3775736991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x56162983ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799125d640 1 -- 172.21.15.5:0/3775736991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 msgr2=0x561629825aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799125d640 1 --2- 172.21.15.5:0/3775736991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x561629825aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f799125d640 1 -- 172.21.15.5:0/3775736991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561629822bf0 con 0x56162981c480 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.276+0000 7f7990a5c640 1 --2- 172.21.15.5:0/3775736991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x56162983ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.277+0000 7f799125d640 1 --2- 172.21.15.5:0/3775736991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x56162983e850 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f798803d8f0 tx=0x7f79880314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13ed52743404b74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.277+0000 7f7976ffd640 1 -- 172.21.15.5:0/3775736991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988035070 con 0x56162981c480 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.277+0000 7f7976ffd640 1 -- 172.21.15.5:0/3775736991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7988041070 con 0x56162981c480 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.277+0000 7f7976ffd640 1 -- 172.21.15.5:0/3775736991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988051070 con 0x56162981c480 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.278+0000 7f799589a740 1 -- 172.21.15.5:0/3775736991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 msgr2=0x56162983e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.278+0000 7f799589a740 1 --2- 172.21.15.5:0/3775736991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x56162983e850 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f798803d8f0 tx=0x7f79880314a0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.278+0000 7f799589a740 1 -- 172.21.15.5:0/3775736991 shutdown_connections 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.278+0000 7f799589a740 1 --2- 172.21.15.5:0/3775736991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x56162983ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.278+0000 7f799589a740 1 --2- 172.21.15.5:0/3775736991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x56162983e850 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.278+0000 7f799589a740 1 --2- 172.21.15.5:0/3775736991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x561629825aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.278+0000 7f799589a740 1 -- 172.21.15.5:0/3775736991 >> 172.21.15.5:0/3775736991 conn(0x5616298180f0 msgr2=0x561629825390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.278+0000 7f799589a740 1 -- 172.21.15.5:0/3775736991 shutdown_connections 2024-09-19T06:14:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.278+0000 7f799589a740 1 -- 172.21.15.5:0/3775736991 wait complete. 2024-09-19T06:14:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.278+0000 7f799589a740 1 -- start start 2024-09-19T06:14:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.278+0000 7f799589a740 1 Processor -- start 2024-09-19T06:14:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f799589a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x56162982b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f799589a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x56162982b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f799589a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x5616298a3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f799589a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1acac0 con 0x56162981c480 2024-09-19T06:14:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f799589a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1ac390 con 0x56162981bb60 2024-09-19T06:14:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f799589a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1a9590 con 0x561629826d40 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f799589a740 10 client.0 _ll_register_callbacks cb 0x7f7988009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f799125d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x56162982b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f799125d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x56162982b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41946/0 (socket says 172.21.15.5:41946) 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f799125d640 1 -- 172.21.15.5:0/1258079648 learned_addr learned my addr 172.21.15.5:0/1258079648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f7990a5c640 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x56162982b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f7990a5c640 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x5616298a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f7990a5c640 1 -- 172.21.15.5:0/1258079648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 msgr2=0x5616298a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f7990a5c640 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x5616298a3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f7990a5c640 1 -- 172.21.15.5:0/1258079648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 msgr2=0x56162982b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f7990a5c640 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x56162982b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f7990a5c640 1 -- 172.21.15.5:0/1258079648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616298a35f0 con 0x56162981bb60 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f799125d640 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x56162982b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f7990a5c640 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x56162982b290 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f797c018fd0 tx=0x7f797c029040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a35c74058825927 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.279+0000 7f7974ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f7974ff9640 1 -- 172.21.15.5:0/1258079648 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c018070 con 0x56162981bb60 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f7974ff9640 1 -- 172.21.15.5:0/1258079648 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f797c018210 con 0x56162981bb60 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f799589a740 1 -- 172.21.15.5:0/1258079648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7988041070 con 0x56162981bb60 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f7974ff9640 1 -- 172.21.15.5:0/1258079648 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f797c018410 con 0x56162981bb60 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f799589a740 10 client.27875 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f799589a740 1 -- 172.21.15.5:0/1258079648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5616298aa9b0 con 0x56162981bb60 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f7974ff9640 1 -- 172.21.15.5:0/1258079648 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7988041070 con 0x56162981bb60 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f7974ff9640 1 -- 172.21.15.5:0/1258079648 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f797c012b40 con 0x56162981bb60 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f799589a740 10 client.27875 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f799589a740 10 client.27875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f799589a740 20 client.27875 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f799589a740 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5616298aaf40 0x5616298ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f799589a740 4 client.27875 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.280+0000 7f799589a740 1 -- 172.21.15.5:0/1258079648 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561629788d00 con 0x5616298aaf40 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.281+0000 7f799125d640 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5616298aaf40 0x5616298ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.281+0000 7f799125d640 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5616298aaf40 0x5616298ad360 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88fb3e52a9869366 server_cookie=ebff9a3383ceb199 in_seq=0 out_seq=0 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.281+0000 7f7974ff9640 10 client.27875 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f7974ff9640 1 -- 172.21.15.5:0/1258079648 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561629788d00 con 0x5616298aaf40 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f7974ff9640 10 client.27875 handle_command_reply: tid=0 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f799589a740 1 client.27875 shutdown 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f799589a740 1 -- 172.21.15.5:0/1258079648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 msgr2=0x56162982b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f799589a740 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x56162982b290 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f797c018fd0 tx=0x7f797c029040 comp rx=0 tx=0).stop 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f799589a740 1 -- 172.21.15.5:0/1258079648 shutdown_connections 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f799589a740 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5616298aaf40 0x5616298ad360 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f799589a740 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561629826d40 0x5616298a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f799589a740 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56162981c480 0x56162982b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f799589a740 1 --2- 172.21.15.5:0/1258079648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162981bb60 0x56162982b290 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f799589a740 1 -- 172.21.15.5:0/1258079648 >> 172.21.15.5:0/1258079648 conn(0x5616298180f0 msgr2=0x561629817270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f7974ff9640 0 client.27875 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f799589a740 1 -- 172.21.15.5:0/1258079648 shutdown_connections 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f799589a740 1 -- 172.21.15.5:0/1258079648 wait complete. 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f799589a740 20 client.27875 trim_cache size 0 max 16384 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f79934c1640 1 -- 172.21.15.5:0/260376335 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f79780a3430 msgr2=0x7f79780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f79934c1640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f79780a3430 0x7f79780a58f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f79840cd2a0 tx=0x7f7984004390 comp rx=0 tx=0).stop 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.282+0000 7f79934c1640 1 -- 172.21.15.5:0/260376335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f798c1082a0 msgr2=0x7f798c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.283+0000 7f79934c1640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f798c1082a0 0x7f798c19ce00 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7984009fd0 tx=0x7f798400b260 comp rx=0 tx=0).stop 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.283+0000 7f79934c1640 1 -- 172.21.15.5:0/260376335 shutdown_connections 2024-09-19T06:14:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.283+0000 7f79934c1640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f798c1082a0 0x7f798c19ce00 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.283+0000 7f79934c1640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f79780a3430 0x7f79780a58f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.283+0000 7f79934c1640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f798c1036e0 0x7f798c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.283+0000 7f79934c1640 1 --2- 172.21.15.5:0/260376335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798c102d30 0x7f798c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.283+0000 7f79934c1640 1 -- 172.21.15.5:0/260376335 >> 172.21.15.5:0/260376335 conn(0x7f798c0fc7d0 msgr2=0x7f798c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.283+0000 7f79934c1640 1 -- 172.21.15.5:0/260376335 shutdown_connections 2024-09-19T06:14:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:31.283+0000 7f79934c1640 1 -- 172.21.15.5:0/260376335 wait complete. 2024-09-19T06:14:31.646 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:14:31 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:14:31.290+0000 7f9e1e5f8640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...12,head] /volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{2=1} v42731 ap=2 f(v1704 m2024-09-19T06:14:18.523027+0000 5=5+0) n(v58 rc2024-09-19T06:14:18.523027+0000 b814326 6=5+1) old_inodes=6 (inest sync->mix g=2) caps={15255=pAsLsXsFs/-@37931} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x559db3497600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v366>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v42731>,<0x10000000008/~dmtmp v40320>,<0x10000000007/client0 v1345>,<0x10000000006/clients v1321>,<0x10000000005/tmp v1245>,<0x10000000003/client.0 v1229>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v1227>,<0x10000000001/sv_0 v1225>,<0x10000000000/qa v1210>,<0x1/volumes v1195>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T06:14:18.509688+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T06:14:18.523027+0000 b814326 6=5+1)","memory_value.dirstat":"f(v1704 m2024-09-19T06:14:18.523027+0000 5=5+0)","memory_value.rstat":"n(v58 rc2024-09-19T06:14:18.523027+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T06:14:31.646 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:14:31 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:14:31.296+0000 7f9e1e5f8640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000229 [...12,head] /volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PARADOX/ auth{1=1,2=1,3=1,4=1} fragtree_t(*^3) v42743 ap=2 f(v2520 m2024-09-19T06:14:20.282888+0000 24=24+0) n(v61 rc2024-09-19T06:14:20.282888+0000 b5230239 25=24+1) old_inodes=6 (inest sync->mix g=1,2,3,4) caps={15255=pAsLsXsFs/-@45040} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x559db34d5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000229:[<0x10000000009/PARADOX v554>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000229:[<0x10000000009/PARADOX v42743>,<0x10000000008/~dmtmp v40320>,<0x10000000007/client0 v1345>,<0x10000000006/clients v1321>,<0x10000000005/tmp v1245>,<0x10000000003/client.0 v1229>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v1227>,<0x10000000001/sv_0 v1225>,<0x10000000000/qa v1210>,<0x1/volumes v1195>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T06:14:20.266786+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T06:14:20.282888+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v2520 m2024-09-19T06:14:20.282888+0000 24=24+0)","memory_value.rstat":"n(v61 rc2024-09-19T06:14:20.282888+0000 b5230239 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T06:14:31.646 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:14:31 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:14:31.300+0000 7f9e1e5f8640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [...12,head] /volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ auth{2=1} v42729 ap=2 f(v1988 m2024-09-19T06:14:17.922989+0000 7=7+0) n(v60 rc2024-09-19T06:14:17.922989+0000 b1462582 8=7+1) old_inodes=6 (inest sync->mix g=2) caps={15255=pAsLsXsFs/-@37714} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x559db34c4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020f:[<0x10000000009/PWRPNT v274>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x1000000020f:[<0x10000000009/PWRPNT v42729>,<0x10000000008/~dmtmp v40328>,<0x10000000007/client0 v1345>,<0x10000000006/clients v1321>,<0x10000000005/tmp v1245>,<0x10000000003/client.0 v1229>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v1227>,<0x10000000001/sv_0 v1225>,<0x10000000000/qa v1210>,<0x1/volumes v1195>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T06:14:17.901382+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T06:14:17.922989+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v1988 m2024-09-19T06:14:17.922989+0000 7=7+0)","memory_value.rstat":"n(v60 rc2024-09-19T06:14:17.922989+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T06:14:31.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 263013 2.27 MB/sec execute 477 sec latency 25.784 ms 2024-09-19T06:14:32.067 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:14:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:14:32.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: pgmap v349: 97 pgs: 97 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 828 KiB/s wr, 80 op/s 2024-09-19T06:14:32.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:14:32.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:14:32.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:14:32.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:14:32.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:14:32.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-19T06:14:32.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:14:32.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T06:14:32.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:14:32.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:32 smithi005 ceph-mon[30955]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-09-19T06:14:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: pgmap v349: 97 pgs: 97 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 828 KiB/s wr, 80 op/s 2024-09-19T06:14:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:14:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:32 smithi157 ceph-mon[35346]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-09-19T06:14:32.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 263426 2.27 MB/sec execute 478 sec latency 47.079 ms 2024-09-19T06:14:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: pgmap v349: 97 pgs: 97 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 828 KiB/s wr, 80 op/s 2024-09-19T06:14:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:14:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:14:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:14:32.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:14:32.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:14:32.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-19T06:14:32.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:14:32.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T06:14:32.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:14:32.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:32.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:32 smithi049 ceph-mon[34640]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-09-19T06:14:32.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.910+0000 7fd31aa82640 1 -- 172.21.15.5:0/2071403214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd314102ef0 msgr2=0x7fd31410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:32.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.910+0000 7fd31aa82640 1 --2- 172.21.15.5:0/2071403214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd314102ef0 0x7fd31410da40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009ca0 tx=0x7fd30c02a950 comp rx=0 tx=0).stop 2024-09-19T06:14:32.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.910+0000 7fd31aa82640 1 -- 172.21.15.5:0/2071403214 shutdown_connections 2024-09-19T06:14:32.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.910+0000 7fd31aa82640 1 --2- 172.21.15.5:0/2071403214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd314102ef0 0x7fd31410da40 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.910+0000 7fd31aa82640 1 --2- 172.21.15.5:0/2071403214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd314102570 0x7fd3141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.910+0000 7fd31aa82640 1 --2- 172.21.15.5:0/2071403214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd314100e20 0x7fd314101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.910+0000 7fd31aa82640 1 -- 172.21.15.5:0/2071403214 >> 172.21.15.5:0/2071403214 conn(0x7fd3140fc7d0 msgr2=0x7fd3140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:32.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.910+0000 7fd31aa82640 1 -- 172.21.15.5:0/2071403214 shutdown_connections 2024-09-19T06:14:32.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.910+0000 7fd31aa82640 1 -- 172.21.15.5:0/2071403214 wait complete. 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31aa82640 1 Processor -- start 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31aa82640 1 -- start start 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31aa82640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd314100e20 0x7fd3140fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31aa82640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd314102570 0x7fd3140fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31aa82640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd314102ef0 0x7fd3140fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31aa82640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd314110e30 con 0x7fd314102ef0 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31aa82640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd314110cb0 con 0x7fd314100e20 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31aa82640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd314110fb0 con 0x7fd314102570 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31901f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd314102ef0 0x7fd3140fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31901f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd314102ef0 0x7fd3140fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41958/0 (socket says 172.21.15.5:41958) 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31901f640 1 -- 172.21.15.5:0/1237417722 learned_addr learned my addr 172.21.15.5:0/1237417722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31881e640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd314100e20 0x7fd3140fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd313fff640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd314102570 0x7fd3140fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31901f640 1 -- 172.21.15.5:0/1237417722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd314102570 msgr2=0x7fd3140fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.911+0000 7fd31901f640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd314102570 0x7fd3140fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.912+0000 7fd31901f640 1 -- 172.21.15.5:0/1237417722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd314100e20 msgr2=0x7fd3140fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.912+0000 7fd31901f640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd314100e20 0x7fd3140fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.912+0000 7fd31901f640 1 -- 172.21.15.5:0/1237417722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3141a6850 con 0x7fd314102ef0 2024-09-19T06:14:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.912+0000 7fd31901f640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd314102ef0 0x7fd3140fd700 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd30c004af0 tx=0x7fd30c031f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.912+0000 7fd311ffb640 1 -- 172.21.15.5:0/1237417722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c034040 con 0x7fd314102ef0 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.912+0000 7fd311ffb640 1 -- 172.21.15.5:0/1237417722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd30c0308f0 con 0x7fd314102ef0 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.912+0000 7fd31aa82640 1 -- 172.21.15.5:0/1237417722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3141a6ae0 con 0x7fd314102ef0 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.912+0000 7fd311ffb640 1 -- 172.21.15.5:0/1237417722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c004600 con 0x7fd314102ef0 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.912+0000 7fd31aa82640 1 -- 172.21.15.5:0/1237417722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3141a6fc0 con 0x7fd314102ef0 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.915+0000 7fd311ffb640 1 -- 172.21.15.5:0/1237417722 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd30c037020 con 0x7fd314102ef0 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.915+0000 7fd31ce5b740 1 Processor -- start 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.915+0000 7fd31ce5b740 1 -- start start 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.915+0000 7fd31ce5b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb54a0 0x558edafb5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.915+0000 7fd31ce5b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafb5dc0 0x558edafc0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.915+0000 7fd31ce5b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafc0680 0x558edafc2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.915+0000 7fd31ce5b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558edaf1b2e0 con 0x558edafc0680 2024-09-19T06:14:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.915+0000 7fd31ce5b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558edaec2ce0 con 0x558edafb54a0 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.915+0000 7fd31ce5b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558edaf172a0 con 0x558edafb5dc0 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.915+0000 7fd313fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafc0680 0x558edafc2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd313fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafc0680 0x558edafc2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41974/0 (socket says 172.21.15.5:41974) 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd313fff640 1 -- 172.21.15.5:0/2337295064 learned_addr learned my addr 172.21.15.5:0/2337295064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd31881e640 1 --2- 172.21.15.5:0/2337295064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafb5dc0 0x558edafc0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd313fff640 1 --2- 172.21.15.5:0/2337295064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb54a0 0x558edafb5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd311ffb640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd3040a33b0 0x7fd3040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd31881e640 1 -- 172.21.15.5:0/2337295064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb54a0 msgr2=0x558edafb5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd31881e640 1 --2- 172.21.15.5:0/2337295064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb54a0 0x558edafb5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd31881e640 1 -- 172.21.15.5:0/2337295064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafc0680 msgr2=0x558edafc2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd31901f640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd3040a33b0 0x7fd3040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd31881e640 1 --2- 172.21.15.5:0/2337295064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafc0680 0x558edafc2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd31881e640 1 -- 172.21.15.5:0/2337295064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558edaea7440 con 0x558edafb5dc0 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd313fff640 1 --2- 172.21.15.5:0/2337295064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafc0680 0x558edafc2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd311ffb640 1 -- 172.21.15.5:0/1237417722 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fd30c0f09b0 con 0x7fd314102ef0 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.916+0000 7fd31881e640 1 --2- 172.21.15.5:0/2337295064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafb5dc0 0x558edafc0140 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd308009c20 tx=0x7fd308011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1331299db4823703 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.917+0000 7fd31901f640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd3040a33b0 0x7fd3040a5870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0a6020 tx=0x7fd30c087bb0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.917+0000 7fd2feffd640 1 -- 172.21.15.5:0/2337295064 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308004050 con 0x558edafb5dc0 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.917+0000 7fd2feffd640 1 -- 172.21.15.5:0/2337295064 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3080054d0 con 0x558edafb5dc0 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.917+0000 7fd2feffd640 1 -- 172.21.15.5:0/2337295064 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308004cc0 con 0x558edafb5dc0 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.917+0000 7fd31ce5b740 1 -- 172.21.15.5:0/2337295064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafb5dc0 msgr2=0x558edafc0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.917+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/2337295064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafb5dc0 0x558edafc0140 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd308009c20 tx=0x7fd308011810 comp rx=0 tx=0).stop 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.917+0000 7fd31ce5b740 1 -- 172.21.15.5:0/2337295064 shutdown_connections 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/2337295064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafc0680 0x558edafc2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/2337295064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafb5dc0 0x558edafc0140 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/2337295064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb54a0 0x558edafb5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 -- 172.21.15.5:0/2337295064 >> 172.21.15.5:0/2337295064 conn(0x558edafb1a30 msgr2=0x558edafb1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 -- 172.21.15.5:0/2337295064 shutdown_connections 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 -- 172.21.15.5:0/2337295064 wait complete. 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 -- start start 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 Processor -- start 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb54a0 0x558edafbff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 0x558edafbd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafc0680 0x558edafbd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558edafc3250 con 0x558edafb5dc0 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558edaed6050 con 0x558edafb54a0 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.918+0000 7fd31ce5b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558edafc33d0 con 0x558edafc0680 2024-09-19T06:14:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd31ce5b740 10 client.0 _ll_register_callbacks cb 0x558edaf16160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd31881e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 0x558edafbd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd313fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb54a0 0x558edafbff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd313fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb54a0 0x558edafbff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49734/0 (socket says 172.21.15.5:49734) 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd313fff640 1 -- 172.21.15.5:0/3577874288 learned_addr learned my addr 172.21.15.5:0/3577874288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd31881e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 0x558edafbd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41986/0 (socket says 172.21.15.5:41986) 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd313fff640 1 --2- 172.21.15.5:0/3577874288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafc0680 0x558edafbd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd313fff640 1 -- 172.21.15.5:0/3577874288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafc0680 msgr2=0x558edafbd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd313fff640 1 --2- 172.21.15.5:0/3577874288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafc0680 0x558edafbd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd313fff640 1 -- 172.21.15.5:0/3577874288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 msgr2=0x558edafbd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd313fff640 1 --2- 172.21.15.5:0/3577874288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 0x558edafbd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd313fff640 1 -- 172.21.15.5:0/3577874288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558edac8c3c0 con 0x558edafb54a0 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd313fff640 1 --2- 172.21.15.5:0/3577874288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb54a0 0x558edafbff50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd300013fc0 tx=0x7fd300004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee1bd301111f9db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd2f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:14:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd2f3fff640 1 -- 172.21.15.5:0/3577874288 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3000132d0 con 0x558edafb54a0 2024-09-19T06:14:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd2f3fff640 1 -- 172.21.15.5:0/3577874288 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fd300013470 con 0x558edafb54a0 2024-09-19T06:14:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.919+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3577874288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3080054d0 con 0x558edafb54a0 2024-09-19T06:14:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.920+0000 7fd2f3fff640 1 -- 172.21.15.5:0/3577874288 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd300005020 con 0x558edafb54a0 2024-09-19T06:14:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.920+0000 7fd31ce5b740 10 client.27887 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.920+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3577874288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558edafcebd0 con 0x558edafb54a0 2024-09-19T06:14:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.920+0000 7fd2f3fff640 1 -- 172.21.15.5:0/3577874288 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3080054d0 con 0x558edafb54a0 2024-09-19T06:14:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.920+0000 7fd2f3fff640 1 -- 172.21.15.5:0/3577874288 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fd3000115d0 con 0x558edafb54a0 2024-09-19T06:14:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.920+0000 7fd31ce5b740 10 client.27887 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.920+0000 7fd31ce5b740 10 client.27887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.920+0000 7fd31ce5b740 20 client.27887 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.920+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3577874288 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558edafd0090 0x558edafd24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.920+0000 7fd31ce5b740 4 client.27887 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.920+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3577874288 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558edaf28850 con 0x558edafd0090 2024-09-19T06:14:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.920+0000 7fd31881e640 1 --2- 172.21.15.5:0/3577874288 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558edafd0090 0x558edafd24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.921+0000 7fd31881e640 1 --2- 172.21.15.5:0/3577874288 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558edafd0090 0x558edafd24e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36a4afac11ae2c8 server_cookie=d85960144603b1d6 in_seq=0 out_seq=0 2024-09-19T06:14:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.921+0000 7fd2f3fff640 10 client.27887 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.921+0000 7fd2f3fff640 1 -- 172.21.15.5:0/3577874288 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558edaf28850 con 0x558edafd0090 2024-09-19T06:14:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.921+0000 7fd2f3fff640 10 client.27887 handle_command_reply: tid=0 2024-09-19T06:14:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.921+0000 7fd31ce5b740 1 client.27887 shutdown 2024-09-19T06:14:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.922+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3577874288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb54a0 msgr2=0x558edafbff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.922+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3577874288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb54a0 0x558edafbff50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd300013fc0 tx=0x7fd300004730 comp rx=0 tx=0).stop 2024-09-19T06:14:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.922+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3577874288 shutdown_connections 2024-09-19T06:14:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.922+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3577874288 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558edafd0090 0x558edafd24e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.922+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3577874288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafc0680 0x558edafbd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.922+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3577874288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 0x558edafbd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.922+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3577874288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb54a0 0x558edafbff50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.922+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3577874288 >> 172.21.15.5:0/3577874288 conn(0x558edafb1a30 msgr2=0x558edafb7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.922+0000 7fd2f3fff640 0 client.27887 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.922+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3577874288 shutdown_connections 2024-09-19T06:14:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.922+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3577874288 wait complete. 2024-09-19T06:14:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.922+0000 7fd31ce5b740 20 client.27887 trim_cache size 0 max 16384 2024-09-19T06:14:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.946+0000 7fd31ce5b740 1 Processor -- start 2024-09-19T06:14:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.946+0000 7fd31ce5b740 1 -- start start 2024-09-19T06:14:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd31ce5b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb54a0 0x558edafd7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd31ce5b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb5dc0 0x558edafd82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd31ce5b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafc0680 0x558edafba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd31ce5b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141af9c0 con 0x558edafb54a0 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd31ce5b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141b15c0 con 0x558edafb5dc0 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd31ce5b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141b0090 con 0x558edafc0680 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd313fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb54a0 0x558edafd7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd313fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb54a0 0x558edafd7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41990/0 (socket says 172.21.15.5:41990) 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd313fff640 1 -- 172.21.15.5:0/3915904818 learned_addr learned my addr 172.21.15.5:0/3915904818 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd31881e640 1 --2- 172.21.15.5:0/3915904818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb5dc0 0x558edafd82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd313fff640 1 --2- 172.21.15.5:0/3915904818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafc0680 0x558edafba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd313fff640 1 -- 172.21.15.5:0/3915904818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafc0680 msgr2=0x558edafba4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd313fff640 1 --2- 172.21.15.5:0/3915904818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafc0680 0x558edafba4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd313fff640 1 -- 172.21.15.5:0/3915904818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb5dc0 msgr2=0x558edafd82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd313fff640 1 --2- 172.21.15.5:0/3915904818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb5dc0 0x558edafd82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd313fff640 1 -- 172.21.15.5:0/3915904818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558edafd2b60 con 0x558edafb54a0 2024-09-19T06:14:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.947+0000 7fd313fff640 1 --2- 172.21.15.5:0/3915904818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb54a0 0x558edafd7db0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd30000a010 tx=0x7fd3000098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ef21f0b37e8e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.948+0000 7fd2fe7fc640 1 -- 172.21.15.5:0/3915904818 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3000130d0 con 0x558edafb54a0 2024-09-19T06:14:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.948+0000 7fd2fe7fc640 1 -- 172.21.15.5:0/3915904818 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd300013270 con 0x558edafb54a0 2024-09-19T06:14:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.948+0000 7fd2fe7fc640 1 -- 172.21.15.5:0/3915904818 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300004d00 con 0x558edafb54a0 2024-09-19T06:14:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.948+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3915904818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb54a0 msgr2=0x558edafd7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.948+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3915904818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb54a0 0x558edafd7db0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd30000a010 tx=0x7fd3000098b0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3915904818 shutdown_connections 2024-09-19T06:14:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3915904818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafc0680 0x558edafba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3915904818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafb5dc0 0x558edafd82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3915904818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb54a0 0x558edafd7db0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3915904818 >> 172.21.15.5:0/3915904818 conn(0x558edafb1a30 msgr2=0x558edafb0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3915904818 shutdown_connections 2024-09-19T06:14:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3915904818 wait complete. 2024-09-19T06:14:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 -- start start 2024-09-19T06:14:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 Processor -- start 2024-09-19T06:14:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafb54a0 0x558edafc6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 0x558edafc6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafc0680 0x558edafc71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141b0ef0 con 0x558edafb5dc0 2024-09-19T06:14:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141b07c0 con 0x558edafc0680 2024-09-19T06:14:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.949+0000 7fd31ce5b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141ad9c0 con 0x558edafb54a0 2024-09-19T06:14:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd31ce5b740 10 client.0 _ll_register_callbacks cb 0x7fd300009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd31881e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 0x558edafc6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd31881e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 0x558edafc6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41994/0 (socket says 172.21.15.5:41994) 2024-09-19T06:14:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd31881e640 1 -- 172.21.15.5:0/3436591885 learned_addr learned my addr 172.21.15.5:0/3436591885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd313fff640 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafb54a0 0x558edafc6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd313fff640 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafc0680 0x558edafc71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd31881e640 1 -- 172.21.15.5:0/3436591885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafb54a0 msgr2=0x558edafc6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd31881e640 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafb54a0 0x558edafc6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd31881e640 1 -- 172.21.15.5:0/3436591885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafc0680 msgr2=0x558edafc71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd31881e640 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafc0680 0x558edafc71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd31881e640 1 -- 172.21.15.5:0/3436591885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558edb03a080 con 0x558edafb5dc0 2024-09-19T06:14:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd313fff640 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafb54a0 0x558edafc6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd31881e640 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 0x558edafc6c60 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd308009bf0 tx=0x7fd308011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95c4c412fb608b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd2f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd2f3fff640 1 -- 172.21.15.5:0/3436591885 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30802e4d0 con 0x558edafb5dc0 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd2f3fff640 1 -- 172.21.15.5:0/3436591885 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7fd30802e670 con 0x558edafb5dc0 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.950+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3436591885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd300013270 con 0x558edafb5dc0 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.951+0000 7fd2f3fff640 1 -- 172.21.15.5:0/3436591885 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd30803d050 con 0x558edafb5dc0 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.951+0000 7fd31ce5b740 10 client.18405 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.951+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3436591885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558edb03b390 con 0x558edafb5dc0 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.951+0000 7fd2f3fff640 1 -- 172.21.15.5:0/3436591885 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd300013270 con 0x558edafb5dc0 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.951+0000 7fd2f3fff640 1 -- 172.21.15.5:0/3436591885 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fd308019a90 con 0x558edafb5dc0 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.951+0000 7fd31ce5b740 10 client.18405 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.951+0000 7fd31ce5b740 10 client.18405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.951+0000 7fd31ce5b740 20 client.18405 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.951+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558edb03b9f0 0x558edb03de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.951+0000 7fd31ce5b740 4 client.18405 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:14:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.951+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3436591885 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558edafb1300 con 0x558edb03b9f0 2024-09-19T06:14:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.951+0000 7fd313fff640 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558edb03b9f0 0x558edb03de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd313fff640 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558edb03b9f0 0x558edb03de10 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e8d6d3f9405653 server_cookie=1f3a0ae2ba59b04d in_seq=0 out_seq=0 2024-09-19T06:14:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd2f3fff640 10 client.18405 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd2f3fff640 1 -- 172.21.15.5:0/3436591885 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558edafb1300 con 0x558edb03b9f0 2024-09-19T06:14:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd2f3fff640 10 client.18405 handle_command_reply: tid=0 2024-09-19T06:14:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd31ce5b740 1 client.18405 shutdown 2024-09-19T06:14:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3436591885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 msgr2=0x558edafc6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 0x558edafc6c60 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd308009bf0 tx=0x7fd308011d10 comp rx=0 tx=0).stop 2024-09-19T06:14:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3436591885 shutdown_connections 2024-09-19T06:14:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558edb03b9f0 0x558edb03de10 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edafc0680 0x558edafc71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd2f3fff640 0 client.18405 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edafb5dc0 0x558edafc6c60 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd31ce5b740 1 --2- 172.21.15.5:0/3436591885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558edafb54a0 0x558edafc6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.952+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3436591885 >> 172.21.15.5:0/3436591885 conn(0x558edafb1a30 msgr2=0x558edafbea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.953+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3436591885 shutdown_connections 2024-09-19T06:14:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.953+0000 7fd31ce5b740 1 -- 172.21.15.5:0/3436591885 wait complete. 2024-09-19T06:14:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.953+0000 7fd31ce5b740 20 client.18405 trim_cache size 0 max 16384 2024-09-19T06:14:32.960 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:14:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:14:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:14:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "061bc969-9bf3-4307-bcc4-ea8921cf3fba": { 2024-09-19T06:14:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:14:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "061bc969-9bf3-4307-bcc4-ea8921cf3fba", 2024-09-19T06:14:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:14:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:14:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:14:32.961 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:14:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.953+0000 7fd31aa82640 1 -- 172.21.15.5:0/1237417722 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd3040a33b0 msgr2=0x7fd3040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.953+0000 7fd31aa82640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd3040a33b0 0x7fd3040a5870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0a6020 tx=0x7fd30c087bb0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.953+0000 7fd31aa82640 1 -- 172.21.15.5:0/1237417722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd314102ef0 msgr2=0x7fd3140fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.953+0000 7fd31aa82640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd314102ef0 0x7fd3140fd700 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd30c004af0 tx=0x7fd30c031f70 comp rx=0 tx=0).stop 2024-09-19T06:14:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.954+0000 7fd31aa82640 1 -- 172.21.15.5:0/1237417722 shutdown_connections 2024-09-19T06:14:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.954+0000 7fd31aa82640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd314102ef0 0x7fd3140fd700 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.954+0000 7fd31aa82640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd3040a33b0 0x7fd3040a5870 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.954+0000 7fd31aa82640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd314102570 0x7fd3140fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.954+0000 7fd31aa82640 1 --2- 172.21.15.5:0/1237417722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd314100e20 0x7fd3140fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.954+0000 7fd31aa82640 1 -- 172.21.15.5:0/1237417722 >> 172.21.15.5:0/1237417722 conn(0x7fd3140fc7d0 msgr2=0x7fd31406a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.954+0000 7fd31aa82640 1 -- 172.21.15.5:0/1237417722 shutdown_connections 2024-09-19T06:14:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:14:32.954+0000 7fd31aa82640 1 -- 172.21.15.5:0/1237417722 wait complete. 2024-09-19T06:14:33.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:33.838 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:33.839 INFO:tasks.cephfs.filesystem:scrub status for tag:061bc969-9bf3-4307-bcc4-ea8921cf3fba - {'path': '/', 'tag': '061bc969-9bf3-4307-bcc4-ea8921cf3fba', 'options': 'recursive,force'} 2024-09-19T06:14:33.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 263967 2.27 MB/sec execute 479 sec latency 22.314 ms 2024-09-19T06:14:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:14:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:34 smithi157 ceph-mon[35346]: pgmap v350: 97 pgs: 97 active+clean; 146 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-09-19T06:14:34.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 264508 2.27 MB/sec execute 480 sec latency 21.722 ms 2024-09-19T06:14:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:34 smithi005 ceph-mon[30955]: pgmap v350: 97 pgs: 97 active+clean; 146 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-09-19T06:14:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:34 smithi049 ceph-mon[34640]: pgmap v350: 97 pgs: 97 active+clean; 146 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-09-19T06:14:35.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 265063 2.27 MB/sec execute 481 sec latency 19.435 ms 2024-09-19T06:14:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:36 smithi157 ceph-mon[35346]: pgmap v351: 97 pgs: 97 active+clean; 146 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 402 KiB/s wr, 56 op/s 2024-09-19T06:14:36.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 265795 2.28 MB/sec execute 482 sec latency 15.451 ms 2024-09-19T06:14:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:36 smithi005 ceph-mon[30955]: pgmap v351: 97 pgs: 97 active+clean; 146 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 402 KiB/s wr, 56 op/s 2024-09-19T06:14:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:36 smithi049 ceph-mon[34640]: pgmap v351: 97 pgs: 97 active+clean; 146 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 402 KiB/s wr, 56 op/s 2024-09-19T06:14:37.844 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266128 2.28 MB/sec execute 483 sec latency 202.854 ms 2024-09-19T06:14:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:38 smithi157 ceph-mon[35346]: pgmap v352: 97 pgs: 97 active+clean; 149 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-09-19T06:14:38.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266128 2.27 MB/sec execute 484 sec latency 1202.945 ms 2024-09-19T06:14:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:38 smithi005 ceph-mon[30955]: pgmap v352: 97 pgs: 97 active+clean; 149 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-09-19T06:14:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:38 smithi049 ceph-mon[34640]: pgmap v352: 97 pgs: 97 active+clean; 149 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-09-19T06:14:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:14:39.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266128 2.27 MB/sec execute 485 sec latency 2203.071 ms 2024-09-19T06:14:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:14:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:14:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:40 smithi157 ceph-mon[35346]: pgmap v353: 97 pgs: 97 active+clean; 149 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 886 KiB/s wr, 92 op/s 2024-09-19T06:14:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:40 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:14:40.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:40 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:14:40.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266195 2.26 MB/sec execute 486 sec latency 2828.513 ms 2024-09-19T06:14:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:40 smithi005 ceph-mon[30955]: pgmap v353: 97 pgs: 97 active+clean; 149 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 886 KiB/s wr, 92 op/s 2024-09-19T06:14:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:40 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:14:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:40 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:14:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:40 smithi049 ceph-mon[34640]: pgmap v353: 97 pgs: 97 active+clean; 149 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 886 KiB/s wr, 92 op/s 2024-09-19T06:14:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:40 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:14:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:40 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:14:41.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266195 2.26 MB/sec execute 487 sec latency 1120.669 ms 2024-09-19T06:14:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:42 smithi157 ceph-mon[35346]: pgmap v354: 97 pgs: 97 active+clean; 149 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 886 KiB/s wr, 92 op/s 2024-09-19T06:14:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:14:42.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:14:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.802+0000 7f88fdcb7640 1 -- 172.21.15.5:0/3798048456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f8102ca0 msgr2=0x7f88f8103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.802+0000 7f88fdcb7640 1 --2- 172.21.15.5:0/3798048456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f8102ca0 0x7f88f8103080 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009900 tx=0x7f88e802a480 comp rx=0 tx=0).stop 2024-09-19T06:14:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.803+0000 7f88fdcb7640 1 -- 172.21.15.5:0/3798048456 shutdown_connections 2024-09-19T06:14:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.803+0000 7f88fdcb7640 1 --2- 172.21.15.5:0/3798048456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f810c0f0 0x7f88f810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.803+0000 7f88fdcb7640 1 --2- 172.21.15.5:0/3798048456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f81035c0 0x7f88f810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.803+0000 7f88fdcb7640 1 --2- 172.21.15.5:0/3798048456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f8102ca0 0x7f88f8103080 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.803+0000 7f88fdcb7640 1 -- 172.21.15.5:0/3798048456 >> 172.21.15.5:0/3798048456 conn(0x7f88f80fc6b0 msgr2=0x7f88f80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.803+0000 7f88fdcb7640 1 -- 172.21.15.5:0/3798048456 shutdown_connections 2024-09-19T06:14:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.803+0000 7f88fdcb7640 1 -- 172.21.15.5:0/3798048456 wait complete. 2024-09-19T06:14:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.803+0000 7f88fdcb7640 1 Processor -- start 2024-09-19T06:14:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.803+0000 7f88fdcb7640 1 -- start start 2024-09-19T06:14:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88fdcb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f8102ca0 0x7f88f8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88fdcb7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f81035c0 0x7f88f8198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88fdcb7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f810c0f0 0x7f88f819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88fdcb7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88f8111d80 con 0x7f88f8102ca0 2024-09-19T06:14:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88fdcb7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f88f8111c00 con 0x7f88f810c0f0 2024-09-19T06:14:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88fdcb7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f88f8111f00 con 0x7f88f81035c0 2024-09-19T06:14:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88f77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f8102ca0 0x7f88f8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88f77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f8102ca0 0x7f88f8198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56972/0 (socket says 172.21.15.5:56972) 2024-09-19T06:14:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88f77fe640 1 -- 172.21.15.5:0/517467579 learned_addr learned my addr 172.21.15.5:0/517467579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88f6ffd640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f81035c0 0x7f88f8198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88f7fff640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f810c0f0 0x7f88f819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88f77fe640 1 -- 172.21.15.5:0/517467579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f81035c0 msgr2=0x7f88f8198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88f77fe640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f81035c0 0x7f88f8198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88f77fe640 1 -- 172.21.15.5:0/517467579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f810c0f0 msgr2=0x7f88f819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88f77fe640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f810c0f0 0x7f88f819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88f77fe640 1 -- 172.21.15.5:0/517467579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f819d2f0 con 0x7f88f8102ca0 2024-09-19T06:14:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88f7fff640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f810c0f0 0x7f88f819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:14:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.804+0000 7f88f77fe640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f8102ca0 0x7f88f8198300 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f88e8005940 tx=0x7f88e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.805+0000 7f88f4ff9640 1 -- 172.21.15.5:0/517467579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e803f030 con 0x7f88f8102ca0 2024-09-19T06:14:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.805+0000 7f88f4ff9640 1 -- 172.21.15.5:0/517467579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88e8003cb0 con 0x7f88f8102ca0 2024-09-19T06:14:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.805+0000 7f88fdcb7640 1 -- 172.21.15.5:0/517467579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f819d580 con 0x7f88f8102ca0 2024-09-19T06:14:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.805+0000 7f88f4ff9640 1 -- 172.21.15.5:0/517467579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8035070 con 0x7f88f8102ca0 2024-09-19T06:14:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.805+0000 7f88fdcb7640 1 -- 172.21.15.5:0/517467579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f81a4e70 con 0x7f88f8102ca0 2024-09-19T06:14:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.807+0000 7f88f4ff9640 1 -- 172.21.15.5:0/517467579 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f88e8042030 con 0x7f88f8102ca0 2024-09-19T06:14:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.807+0000 7f8900a4e740 1 Processor -- start 2024-09-19T06:14:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.807+0000 7f8900a4e740 1 -- start start 2024-09-19T06:14:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f8900a4e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc09d0 0x56039bdc0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f8900a4e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdc12f0 0x56039bdcfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f8900a4e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdd00c0 0x56039bdd24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f8900a4e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56039bd252f0 con 0x56039bdd00c0 2024-09-19T06:14:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f8900a4e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56039bccd270 con 0x56039bdc09d0 2024-09-19T06:14:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f8900a4e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56039bd212b0 con 0x56039bdc12f0 2024-09-19T06:14:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdd00c0 0x56039bdd24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdd00c0 0x56039bdd24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56982/0 (socket says 172.21.15.5:56982) 2024-09-19T06:14:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f6ffd640 1 -- 172.21.15.5:0/124837188 learned_addr learned my addr 172.21.15.5:0/124837188 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f7fff640 1 --2- 172.21.15.5:0/124837188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc09d0 0x56039bdc0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f4ff9640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f88e00a33b0 0x7f88e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f77fe640 1 --2- 172.21.15.5:0/124837188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdc12f0 0x56039bdcfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f6ffd640 1 -- 172.21.15.5:0/124837188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdc12f0 msgr2=0x56039bdcfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f6ffd640 1 --2- 172.21.15.5:0/124837188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdc12f0 0x56039bdcfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f6ffd640 1 -- 172.21.15.5:0/124837188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc09d0 msgr2=0x56039bdc0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f6ffd640 1 --2- 172.21.15.5:0/124837188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc09d0 0x56039bdc0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f6ffd640 1 -- 172.21.15.5:0/124837188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039bccc340 con 0x56039bdd00c0 2024-09-19T06:14:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f7fff640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f88e00a33b0 0x7f88e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f4ff9640 1 -- 172.21.15.5:0/517467579 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f88e80f0560 con 0x7f88f8102ca0 2024-09-19T06:14:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88f6ffd640 1 --2- 172.21.15.5:0/124837188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdd00c0 0x56039bdd24b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009cb0 tx=0x7f88ec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873aba1ca438b7fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.808+0000 7f88ddffb640 1 -- 172.21.15.5:0/124837188 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec01e070 con 0x56039bdd00c0 2024-09-19T06:14:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.809+0000 7f88ddffb640 1 -- 172.21.15.5:0/124837188 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88ec019ae0 con 0x56039bdd00c0 2024-09-19T06:14:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.809+0000 7f88ddffb640 1 -- 172.21.15.5:0/124837188 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00e9b0 con 0x56039bdd00c0 2024-09-19T06:14:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.809+0000 7f8900a4e740 1 -- 172.21.15.5:0/124837188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdd00c0 msgr2=0x56039bdd24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.809+0000 7f88f7fff640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f88e00a33b0 0x7f88e00a5870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f88f0005fd0 tx=0x7f88f0004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.809+0000 7f8900a4e740 1 --2- 172.21.15.5:0/124837188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdd00c0 0x56039bdd24b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009cb0 tx=0x7f88ec011810 comp rx=0 tx=0).stop 2024-09-19T06:14:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.809+0000 7f8900a4e740 1 -- 172.21.15.5:0/124837188 shutdown_connections 2024-09-19T06:14:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.809+0000 7f8900a4e740 1 --2- 172.21.15.5:0/124837188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdd00c0 0x56039bdd24b0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.809+0000 7f8900a4e740 1 --2- 172.21.15.5:0/124837188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdc12f0 0x56039bdcfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.809+0000 7f8900a4e740 1 --2- 172.21.15.5:0/124837188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc09d0 0x56039bdc0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.809+0000 7f8900a4e740 1 -- 172.21.15.5:0/124837188 >> 172.21.15.5:0/124837188 conn(0x56039bdbd060 msgr2=0x56039bdbd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.809+0000 7f8900a4e740 1 -- 172.21.15.5:0/124837188 shutdown_connections 2024-09-19T06:14:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.809+0000 7f8900a4e740 1 -- 172.21.15.5:0/124837188 wait complete. 2024-09-19T06:14:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f8900a4e740 1 -- start start 2024-09-19T06:14:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f8900a4e740 1 Processor -- start 2024-09-19T06:14:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f8900a4e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 0x56039bdcd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f8900a4e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc12f0 0x56039bdcd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f8900a4e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdd00c0 0x56039bdcdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f8900a4e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56039bdd2cf0 con 0x56039bdc09d0 2024-09-19T06:14:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f8900a4e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56039bcae290 con 0x56039bdc12f0 2024-09-19T06:14:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f8900a4e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56039bdd2e70 con 0x56039bdd00c0 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f8900a4e740 10 client.0 _ll_register_callbacks cb 0x56039bcdf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f88f77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 0x56039bdcd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f88f77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 0x56039bdcd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56996/0 (socket says 172.21.15.5:56996) 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f88f77fe640 1 -- 172.21.15.5:0/3994779238 learned_addr learned my addr 172.21.15.5:0/3994779238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f88f7fff640 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdd00c0 0x56039bdcdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.810+0000 7f88f6ffd640 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc12f0 0x56039bdcd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f88f77fe640 1 -- 172.21.15.5:0/3994779238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdd00c0 msgr2=0x56039bdcdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f88f77fe640 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdd00c0 0x56039bdcdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f88f77fe640 1 -- 172.21.15.5:0/3994779238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc12f0 msgr2=0x56039bdcd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f88f77fe640 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc12f0 0x56039bdcd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f88f77fe640 1 -- 172.21.15.5:0/3994779238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039ba38210 con 0x56039bdc09d0 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f88f77fe640 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 0x56039bdcd040 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f88e80b8c20 tx=0x7f88e80bc650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e04a4724d97c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f88d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f88d6ffd640 1 -- 172.21.15.5:0/3994779238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8049070 con 0x56039bdc09d0 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f88d6ffd640 1 -- 172.21.15.5:0/3994779238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f88e8003800 con 0x56039bdc09d0 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f8900a4e740 1 -- 172.21.15.5:0/3994779238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88ec019ae0 con 0x56039bdc09d0 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f88d6ffd640 1 -- 172.21.15.5:0/3994779238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f88e8005d50 con 0x56039bdc09d0 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f8900a4e740 10 client.18417 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f8900a4e740 1 -- 172.21.15.5:0/3994779238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56039bb04620 con 0x56039bdc09d0 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f88d6ffd640 1 -- 172.21.15.5:0/3994779238 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88ec019ae0 con 0x56039bdc09d0 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f88d6ffd640 1 -- 172.21.15.5:0/3994779238 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f88e80d0ce0 con 0x56039bdc09d0 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f8900a4e740 10 client.18417 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f8900a4e740 10 client.18417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f8900a4e740 20 client.18417 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f8900a4e740 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56039bde0140 0x56039bde25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f8900a4e740 4 client.18417 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.811+0000 7f8900a4e740 1 -- 172.21.15.5:0/3994779238 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56039bd32a20 con 0x56039bde0140 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.812+0000 7f88f6ffd640 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56039bde0140 0x56039bde25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.812+0000 7f88d6ffd640 10 client.18417 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.812+0000 7f88f6ffd640 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56039bde0140 0x56039bde25d0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62bcbf98209bd776 server_cookie=a7bcec23a9752602 in_seq=0 out_seq=0 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f88d6ffd640 1 -- 172.21.15.5:0/3994779238 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56039bd32a20 con 0x56039bde0140 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f88d6ffd640 10 client.18417 handle_command_reply: tid=0 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f8900a4e740 1 client.18417 shutdown 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f8900a4e740 1 -- 172.21.15.5:0/3994779238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 msgr2=0x56039bdcd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f8900a4e740 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 0x56039bdcd040 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f88e80b8c20 tx=0x7f88e80bc650 comp rx=0 tx=0).stop 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f8900a4e740 1 -- 172.21.15.5:0/3994779238 shutdown_connections 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f8900a4e740 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56039bde0140 0x56039bde25d0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f8900a4e740 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdd00c0 0x56039bdcdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f88d6ffd640 0 client.18417 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f8900a4e740 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc12f0 0x56039bdcd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f8900a4e740 1 --2- 172.21.15.5:0/3994779238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 0x56039bdcd040 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f8900a4e740 1 -- 172.21.15.5:0/3994779238 >> 172.21.15.5:0/3994779238 conn(0x56039bdbd060 msgr2=0x56039bdc2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f8900a4e740 1 -- 172.21.15.5:0/3994779238 shutdown_connections 2024-09-19T06:14:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f8900a4e740 1 -- 172.21.15.5:0/3994779238 wait complete. 2024-09-19T06:14:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.813+0000 7f8900a4e740 20 client.18417 trim_cache size 0 max 16384 2024-09-19T06:14:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f8900a4e740 1 Processor -- start 2024-09-19T06:14:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f8900a4e740 1 -- start start 2024-09-19T06:14:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f8900a4e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 0x56039bde7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f8900a4e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc12f0 0x56039bde83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f8900a4e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdd00c0 0x56039bdca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f8900a4e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88f81ab630 con 0x56039bdc09d0 2024-09-19T06:14:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f8900a4e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f88f81ad230 con 0x56039bdc12f0 2024-09-19T06:14:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f8900a4e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f88f81abd00 con 0x56039bdd00c0 2024-09-19T06:14:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f88f77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 0x56039bde7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f88f77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 0x56039bde7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57008/0 (socket says 172.21.15.5:57008) 2024-09-19T06:14:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f88f77fe640 1 -- 172.21.15.5:0/2442406533 learned_addr learned my addr 172.21.15.5:0/2442406533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f88f7fff640 1 --2- 172.21.15.5:0/2442406533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdd00c0 0x56039bdca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f88f6ffd640 1 --2- 172.21.15.5:0/2442406533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc12f0 0x56039bde83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f88f77fe640 1 -- 172.21.15.5:0/2442406533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdd00c0 msgr2=0x56039bdca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f88f77fe640 1 --2- 172.21.15.5:0/2442406533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdd00c0 0x56039bdca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f88f77fe640 1 -- 172.21.15.5:0/2442406533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc12f0 msgr2=0x56039bde83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f88f77fe640 1 --2- 172.21.15.5:0/2442406533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc12f0 0x56039bde83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.828+0000 7f88f77fe640 1 -- 172.21.15.5:0/2442406533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039bdc3500 con 0x56039bdc09d0 2024-09-19T06:14:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f88f77fe640 1 --2- 172.21.15.5:0/2442406533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 0x56039bde7ea0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f88e80c4f00 tx=0x7f88e80bacd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241409e4aedb71db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f88dd7fa640 1 -- 172.21.15.5:0/2442406533 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e80d4030 con 0x56039bdc09d0 2024-09-19T06:14:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f88dd7fa640 1 -- 172.21.15.5:0/2442406533 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88e803b460 con 0x56039bdc09d0 2024-09-19T06:14:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f88dd7fa640 1 -- 172.21.15.5:0/2442406533 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8049070 con 0x56039bdc09d0 2024-09-19T06:14:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f8900a4e740 1 -- 172.21.15.5:0/2442406533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 msgr2=0x56039bde7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f8900a4e740 1 --2- 172.21.15.5:0/2442406533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 0x56039bde7ea0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f88e80c4f00 tx=0x7f88e80bacd0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f8900a4e740 1 -- 172.21.15.5:0/2442406533 shutdown_connections 2024-09-19T06:14:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f8900a4e740 1 --2- 172.21.15.5:0/2442406533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdd00c0 0x56039bdca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f8900a4e740 1 --2- 172.21.15.5:0/2442406533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdc12f0 0x56039bde83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f8900a4e740 1 --2- 172.21.15.5:0/2442406533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc09d0 0x56039bde7ea0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f8900a4e740 1 -- 172.21.15.5:0/2442406533 >> 172.21.15.5:0/2442406533 conn(0x56039bdbd060 msgr2=0x56039bdcf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f8900a4e740 1 -- 172.21.15.5:0/2442406533 shutdown_connections 2024-09-19T06:14:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.829+0000 7f8900a4e740 1 -- 172.21.15.5:0/2442406533 wait complete. 2024-09-19T06:14:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f8900a4e740 1 -- start start 2024-09-19T06:14:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f8900a4e740 1 Processor -- start 2024-09-19T06:14:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f8900a4e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdc09d0 0x56039bdd4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f8900a4e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc12f0 0x56039bdd4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f8900a4e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdd00c0 0x56039be49e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f8900a4e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88f81acb60 con 0x56039bdc12f0 2024-09-19T06:14:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f8900a4e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f88f81ac430 con 0x56039bdd00c0 2024-09-19T06:14:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f8900a4e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f88f81a9630 con 0x56039bdc09d0 2024-09-19T06:14:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f8900a4e740 10 client.0 _ll_register_callbacks cb 0x7f88e80b8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f88f6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc12f0 0x56039bdd4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f88f6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc12f0 0x56039bdd4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57010/0 (socket says 172.21.15.5:57010) 2024-09-19T06:14:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f88f6ffd640 1 -- 172.21.15.5:0/20362021 learned_addr learned my addr 172.21.15.5:0/20362021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f88f77fe640 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdc09d0 0x56039bdd4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.830+0000 7f88f7fff640 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdd00c0 0x56039be49e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f88f6ffd640 1 -- 172.21.15.5:0/20362021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdc09d0 msgr2=0x56039bdd4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f88f6ffd640 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdc09d0 0x56039bdd4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f88f6ffd640 1 -- 172.21.15.5:0/20362021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdd00c0 msgr2=0x56039be49e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f88f6ffd640 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdd00c0 0x56039be49e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f88f6ffd640 1 -- 172.21.15.5:0/20362021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039be4a3b0 con 0x56039bdc12f0 2024-09-19T06:14:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f88f6ffd640 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc12f0 0x56039bdd4a90 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009bd0 tx=0x7f88ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93e87e5eba0b547 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f88d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f88d6ffd640 1 -- 172.21.15.5:0/20362021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec01e040 con 0x56039bdc12f0 2024-09-19T06:14:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f88d6ffd640 1 -- 172.21.15.5:0/20362021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f88ec019a00 con 0x56039bdc12f0 2024-09-19T06:14:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f8900a4e740 1 -- 172.21.15.5:0/20362021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56039bd2cd70 con 0x56039bdc12f0 2024-09-19T06:14:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f88d6ffd640 1 -- 172.21.15.5:0/20362021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f88ec0098b0 con 0x56039bdc12f0 2024-09-19T06:14:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f8900a4e740 10 client.18423 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f8900a4e740 1 -- 172.21.15.5:0/20362021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56039be4b670 con 0x56039bdc12f0 2024-09-19T06:14:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f88d6ffd640 1 -- 172.21.15.5:0/20362021 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56039bd2cd70 con 0x56039bdc12f0 2024-09-19T06:14:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f88d6ffd640 1 -- 172.21.15.5:0/20362021 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f88ec00ea00 con 0x56039bdc12f0 2024-09-19T06:14:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f8900a4e740 10 client.18423 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f8900a4e740 10 client.18423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f8900a4e740 20 client.18423 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f8900a4e740 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56039be4bcd0 0x56039be4e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f8900a4e740 4 client.18423 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:14:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.831+0000 7f8900a4e740 1 -- 172.21.15.5:0/20362021 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56039bca9900 con 0x56039be4bcd0 2024-09-19T06:14:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.832+0000 7f88f77fe640 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56039be4bcd0 0x56039be4e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.832+0000 7f88d6ffd640 10 client.18423 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.832+0000 7f88f77fe640 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56039be4bcd0 0x56039be4e0f0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d5c7b0e4fb5640e server_cookie=fcc99b2e3cb37c60 in_seq=0 out_seq=0 2024-09-19T06:14:42.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266195 2.26 MB/sec execute 488 sec latency 2120.701 ms 2024-09-19T06:14:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f88d6ffd640 1 -- 172.21.15.5:0/20362021 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56039bca9900 con 0x56039be4bcd0 2024-09-19T06:14:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f88d6ffd640 10 client.18423 handle_command_reply: tid=0 2024-09-19T06:14:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f8900a4e740 1 client.18423 shutdown 2024-09-19T06:14:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f8900a4e740 1 -- 172.21.15.5:0/20362021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc12f0 msgr2=0x56039bdd4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f8900a4e740 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc12f0 0x56039bdd4a90 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009bd0 tx=0x7f88ec009840 comp rx=0 tx=0).stop 2024-09-19T06:14:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f8900a4e740 1 -- 172.21.15.5:0/20362021 shutdown_connections 2024-09-19T06:14:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f8900a4e740 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56039be4bcd0 0x56039be4e0f0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f8900a4e740 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56039bdd00c0 0x56039be49e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f8900a4e740 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56039bdc12f0 0x56039bdd4a90 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f88d6ffd640 0 client.18423 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f8900a4e740 1 --2- 172.21.15.5:0/20362021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56039bdc09d0 0x56039bdd4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f8900a4e740 1 -- 172.21.15.5:0/20362021 >> 172.21.15.5:0/20362021 conn(0x56039bdbd060 msgr2=0x56039bdc24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f8900a4e740 1 -- 172.21.15.5:0/20362021 shutdown_connections 2024-09-19T06:14:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f8900a4e740 1 -- 172.21.15.5:0/20362021 wait complete. 2024-09-19T06:14:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.853+0000 7f8900a4e740 20 client.18423 trim_cache size 0 max 16384 2024-09-19T06:14:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:14:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:14:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:14:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:14:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.854+0000 7f88fdcb7640 1 -- 172.21.15.5:0/517467579 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f88e00a33b0 msgr2=0x7f88e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.854+0000 7f88fdcb7640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f88e00a33b0 0x7f88e00a5870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f88f0005fd0 tx=0x7f88f0004a40 comp rx=0 tx=0).stop 2024-09-19T06:14:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.854+0000 7f88fdcb7640 1 -- 172.21.15.5:0/517467579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f8102ca0 msgr2=0x7f88f8198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.854+0000 7f88fdcb7640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f8102ca0 0x7f88f8198300 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f88e8005940 tx=0x7f88e80042f0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.854+0000 7f88fdcb7640 1 -- 172.21.15.5:0/517467579 shutdown_connections 2024-09-19T06:14:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.854+0000 7f88fdcb7640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f810c0f0 0x7f88f819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.854+0000 7f88fdcb7640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f88e00a33b0 0x7f88e00a5870 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.854+0000 7f88fdcb7640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f81035c0 0x7f88f8198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.854+0000 7f88fdcb7640 1 --2- 172.21.15.5:0/517467579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f8102ca0 0x7f88f8198300 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.854+0000 7f88fdcb7640 1 -- 172.21.15.5:0/517467579 >> 172.21.15.5:0/517467579 conn(0x7f88f80fc6b0 msgr2=0x7f88f80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.854+0000 7f88fdcb7640 1 -- 172.21.15.5:0/517467579 shutdown_connections 2024-09-19T06:14:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:42.854+0000 7f88fdcb7640 1 -- 172.21.15.5:0/517467579 wait complete. 2024-09-19T06:14:42.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:42 smithi005 ceph-mon[30955]: pgmap v354: 97 pgs: 97 active+clean; 149 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 886 KiB/s wr, 92 op/s 2024-09-19T06:14:42.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:42.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:42.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:14:42.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:14:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:42 smithi049 ceph-mon[34640]: pgmap v354: 97 pgs: 97 active+clean; 149 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 886 KiB/s wr, 92 op/s 2024-09-19T06:14:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:14:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:14:42.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:14:42.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:14:43.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266195 2.25 MB/sec execute 489 sec latency 3120.817 ms 2024-09-19T06:14:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:44 smithi157 ceph-mon[35346]: pgmap v355: 97 pgs: 97 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-09-19T06:14:44.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266195 2.25 MB/sec execute 490 sec latency 4120.943 ms 2024-09-19T06:14:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:44 smithi005 ceph-mon[30955]: pgmap v355: 97 pgs: 97 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-09-19T06:14:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:44 smithi049 ceph-mon[34640]: pgmap v355: 97 pgs: 97 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-09-19T06:14:45.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266311 2.24 MB/sec execute 491 sec latency 4739.003 ms 2024-09-19T06:14:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:46 smithi157 ceph-mon[35346]: pgmap v356: 97 pgs: 97 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 829 KiB/s wr, 81 op/s 2024-09-19T06:14:46.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266596 2.24 MB/sec execute 492 sec latency 11.853 ms 2024-09-19T06:14:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:46 smithi005 ceph-mon[30955]: pgmap v356: 97 pgs: 97 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 829 KiB/s wr, 81 op/s 2024-09-19T06:14:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:46 smithi049 ceph-mon[34640]: pgmap v356: 97 pgs: 97 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 829 KiB/s wr, 81 op/s 2024-09-19T06:14:47.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 267098 2.24 MB/sec execute 493 sec latency 16.780 ms 2024-09-19T06:14:48.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:48 smithi157 ceph-mon[35346]: pgmap v357: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-09-19T06:14:48.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 267619 2.24 MB/sec execute 494 sec latency 21.500 ms 2024-09-19T06:14:48.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:48 smithi005 ceph-mon[30955]: pgmap v357: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-09-19T06:14:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:48 smithi049 ceph-mon[34640]: pgmap v357: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-09-19T06:14:49.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 268201 2.24 MB/sec execute 495 sec latency 21.751 ms 2024-09-19T06:14:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:50 smithi157 ceph-mon[35346]: pgmap v358: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 386 KiB/s wr, 48 op/s 2024-09-19T06:14:50.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 268681 2.24 MB/sec execute 496 sec latency 19.873 ms 2024-09-19T06:14:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:50 smithi005 ceph-mon[30955]: pgmap v358: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 386 KiB/s wr, 48 op/s 2024-09-19T06:14:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:50 smithi049 ceph-mon[34640]: pgmap v358: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 386 KiB/s wr, 48 op/s 2024-09-19T06:14:51.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 269382 2.25 MB/sec execute 497 sec latency 14.972 ms 2024-09-19T06:14:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:52 smithi157 ceph-mon[35346]: pgmap v359: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 386 KiB/s wr, 48 op/s 2024-09-19T06:14:52.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 269675 2.25 MB/sec execute 498 sec latency 393.129 ms 2024-09-19T06:14:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:52 smithi005 ceph-mon[30955]: pgmap v359: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 386 KiB/s wr, 48 op/s 2024-09-19T06:14:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:52 smithi049 ceph-mon[34640]: pgmap v359: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 386 KiB/s wr, 48 op/s 2024-09-19T06:14:53.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 269675 2.24 MB/sec execute 499 sec latency 1393.215 ms 2024-09-19T06:14:54.698 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:54 smithi049 ceph-mon[34640]: pgmap v360: 97 pgs: 97 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-09-19T06:14:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:54 smithi157 ceph-mon[35346]: pgmap v360: 97 pgs: 97 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-09-19T06:14:54.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 269742 2.24 MB/sec execute 500 sec latency 1592.725 ms 2024-09-19T06:14:54.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:54 smithi005 ceph-mon[30955]: pgmap v360: 97 pgs: 97 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-09-19T06:14:55.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 269866 2.24 MB/sec execute 501 sec latency 1150.308 ms 2024-09-19T06:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.949+0000 7f0d56042640 1 -- 172.21.15.5:0/933955315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d501009a0 msgr2=0x7f0d50100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.949+0000 7f0d56042640 1 --2- 172.21.15.5:0/933955315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d501009a0 0x7f0d50100e00 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009930 tx=0x7f0d4402a420 comp rx=0 tx=0).stop 2024-09-19T06:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.949+0000 7f0d56042640 1 -- 172.21.15.5:0/933955315 shutdown_connections 2024-09-19T06:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.949+0000 7f0d56042640 1 --2- 172.21.15.5:0/933955315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d50101340 0x7f0d5010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.950+0000 7f0d56042640 1 --2- 172.21.15.5:0/933955315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d501009a0 0x7f0d50100e00 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.950+0000 7f0d56042640 1 --2- 172.21.15.5:0/933955315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d501069b0 0x7f0d50106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.950+0000 7f0d56042640 1 -- 172.21.15.5:0/933955315 >> 172.21.15.5:0/933955315 conn(0x7f0d500779a0 msgr2=0x7f0d500fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.950+0000 7f0d56042640 1 -- 172.21.15.5:0/933955315 shutdown_connections 2024-09-19T06:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.950+0000 7f0d56042640 1 -- 172.21.15.5:0/933955315 wait complete. 2024-09-19T06:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.950+0000 7f0d56042640 1 Processor -- start 2024-09-19T06:14:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.950+0000 7f0d56042640 1 -- start start 2024-09-19T06:14:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d56042640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d501009a0 0x7f0d5010fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d56042640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d50101340 0x7f0d50110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d56042640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d501069b0 0x7f0d501107a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d56042640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50111e60 con 0x7f0d501009a0 2024-09-19T06:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d56042640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5006a8e0 con 0x7f0d50101340 2024-09-19T06:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d56042640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50111fe0 con 0x7f0d501069b0 2024-09-19T06:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d501009a0 0x7f0d5010fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d4effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d50101340 0x7f0d50110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d501009a0 0x7f0d5010fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54070/0 (socket says 172.21.15.5:54070) 2024-09-19T06:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d4f7fe640 1 -- 172.21.15.5:0/2470219374 learned_addr learned my addr 172.21.15.5:0/2470219374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d4ffff640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d501069b0 0x7f0d501107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d4f7fe640 1 -- 172.21.15.5:0/2470219374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d501069b0 msgr2=0x7f0d501107a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d501069b0 0x7f0d501107a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d4f7fe640 1 -- 172.21.15.5:0/2470219374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d50101340 msgr2=0x7f0d50110260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d50101340 0x7f0d50110260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d4f7fe640 1 -- 172.21.15.5:0/2470219374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d501a6860 con 0x7f0d501009a0 2024-09-19T06:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d4effd640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d50101340 0x7f0d50110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.951+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d501009a0 0x7f0d5010fd20 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000d950 tx=0x7f0d4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.952+0000 7f0d4cff9640 1 -- 172.21.15.5:0/2470219374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40017070 con 0x7f0d501009a0 2024-09-19T06:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.952+0000 7f0d4cff9640 1 -- 172.21.15.5:0/2470219374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d40009c30 con 0x7f0d501009a0 2024-09-19T06:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.952+0000 7f0d56042640 1 -- 172.21.15.5:0/2470219374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d501a6b50 con 0x7f0d501009a0 2024-09-19T06:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.952+0000 7f0d4cff9640 1 -- 172.21.15.5:0/2470219374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4000f620 con 0x7f0d501009a0 2024-09-19T06:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.952+0000 7f0d56042640 1 -- 172.21.15.5:0/2470219374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d501a7070 con 0x7f0d501009a0 2024-09-19T06:14:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.954+0000 7f0d4cff9640 1 -- 172.21.15.5:0/2470219374 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0d400109a0 con 0x7f0d501009a0 2024-09-19T06:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d58dd2740 1 Processor -- start 2024-09-19T06:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d58dd2740 1 -- start start 2024-09-19T06:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d4cff9640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0d380a33b0 0x7f0d380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d58dd2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8daf0 0x55ca89c8ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d4cff9640 1 -- 172.21.15.5:0/2470219374 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f0d400cf840 con 0x7f0d501009a0 2024-09-19T06:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d58dd2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c98770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d58dd2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 0x55ca89c9b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d58dd2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca89bf22f0 con 0x55ca89c8e410 2024-09-19T06:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d4ffff640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0d380a33b0 0x7f0d380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d58dd2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ca89b9a270 con 0x55ca89c8daf0 2024-09-19T06:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d58dd2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ca89bee2b0 con 0x55ca89c98cb0 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d4f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8daf0 0x55ca89c8ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c98770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d4f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8daf0 0x55ca89c8ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:45148/0 (socket says 172.21.15.5:45148) 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c98770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54076/0 (socket says 172.21.15.5:54076) 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d4f7fe640 1 -- 172.21.15.5:0/3037858528 learned_addr learned my addr 172.21.15.5:0/3037858528 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.955+0000 7f0d4ffff640 1 --2- 172.21.15.5:0/3037858528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 0x55ca89c9b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.956+0000 7f0d4effd640 1 -- 172.21.15.5:0/3037858528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 msgr2=0x55ca89c9b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.956+0000 7f0d4effd640 1 --2- 172.21.15.5:0/3037858528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 0x55ca89c9b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.956+0000 7f0d4effd640 1 -- 172.21.15.5:0/3037858528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8daf0 msgr2=0x55ca89c8ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.956+0000 7f0d4effd640 1 --2- 172.21.15.5:0/3037858528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8daf0 0x55ca89c8ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.956+0000 7f0d4effd640 1 -- 172.21.15.5:0/3037858528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca89b99340 con 0x55ca89c8e410 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.956+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/3037858528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8daf0 0x55ca89c8ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.956+0000 7f0d4ffff640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0d380a33b0 0x7f0d380a5870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0d48005fd0 tx=0x7f0d48010740 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.956+0000 7f0d4effd640 1 --2- 172.21.15.5:0/3037858528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c98770 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0d44031440 tx=0x7f0d440314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41e3654a68c4b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.956+0000 7f0d35ffb640 1 -- 172.21.15.5:0/3037858528 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44035040 con 0x55ca89c8e410 2024-09-19T06:14:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.956+0000 7f0d35ffb640 1 -- 172.21.15.5:0/3037858528 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d440318b0 con 0x55ca89c8e410 2024-09-19T06:14:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.956+0000 7f0d35ffb640 1 -- 172.21.15.5:0/3037858528 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440049d0 con 0x55ca89c8e410 2024-09-19T06:14:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.957+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3037858528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 msgr2=0x55ca89c98770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.957+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/3037858528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c98770 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0d44031440 tx=0x7f0d440314c0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.957+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3037858528 shutdown_connections 2024-09-19T06:14:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.957+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/3037858528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 0x55ca89c9b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.957+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/3037858528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c98770 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.957+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/3037858528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8daf0 0x55ca89c8ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.957+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3037858528 >> 172.21.15.5:0/3037858528 conn(0x55ca89c8a060 msgr2=0x55ca89c8a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.957+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3037858528 shutdown_connections 2024-09-19T06:14:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.957+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3037858528 wait complete. 2024-09-19T06:14:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.957+0000 7f0d58dd2740 1 -- start start 2024-09-19T06:14:55.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d58dd2740 1 Processor -- start 2024-09-19T06:14:55.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d58dd2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c8daf0 0x55ca89c95c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d58dd2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d58dd2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c98cb0 0x55ca89c966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d58dd2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca89c9b880 con 0x55ca89c8e410 2024-09-19T06:14:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d58dd2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ca89b7b290 con 0x55ca89c98cb0 2024-09-19T06:14:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d58dd2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ca89c9ba00 con 0x55ca89c8daf0 2024-09-19T06:14:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d58dd2740 10 client.0 _ll_register_callbacks cb 0x55ca89bac140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54078/0 (socket says 172.21.15.5:54078) 2024-09-19T06:14:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d4effd640 1 -- 172.21.15.5:0/3265701634 learned_addr learned my addr 172.21.15.5:0/3265701634 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d4ffff640 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c98cb0 0x55ca89c966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c8daf0 0x55ca89c95c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d4effd640 1 -- 172.21.15.5:0/3265701634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c8daf0 msgr2=0x55ca89c95c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d4effd640 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c8daf0 0x55ca89c95c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d4effd640 1 -- 172.21.15.5:0/3265701634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c98cb0 msgr2=0x55ca89c966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d4effd640 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c98cb0 0x55ca89c966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d4effd640 1 -- 172.21.15.5:0/3265701634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca89905210 con 0x55ca89c8e410 2024-09-19T06:14:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.958+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c8daf0 0x55ca89c95c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:14:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d4effd640 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c961b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0d44006010 tx=0x7f0d44004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cecc96f2786b877 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d2effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d2effd640 1 -- 172.21.15.5:0/3265701634 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44035070 con 0x55ca89c8e410 2024-09-19T06:14:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d2effd640 1 -- 172.21.15.5:0/3265701634 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f0d440316b0 con 0x55ca89c8e410 2024-09-19T06:14:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3265701634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d440318b0 con 0x55ca89c8e410 2024-09-19T06:14:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d2effd640 1 -- 172.21.15.5:0/3265701634 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0d440035f0 con 0x55ca89c8e410 2024-09-19T06:14:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d58dd2740 10 client.18435 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3265701634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ca899d1620 con 0x55ca89c8e410 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d2effd640 1 -- 172.21.15.5:0/3265701634 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d440318b0 con 0x55ca89c8e410 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d2effd640 1 -- 172.21.15.5:0/3265701634 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f0d44046780 con 0x55ca89c8e410 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d58dd2740 10 client.18435 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d58dd2740 10 client.18435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d58dd2740 20 client.18435 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca89ca8b50 0x55ca89caaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d58dd2740 4 client.18435 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.959+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3265701634 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca89bffa20 con 0x55ca89ca8b50 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.960+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca89ca8b50 0x55ca89caaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.960+0000 7f0d2effd640 10 client.18435 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.960+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca89ca8b50 0x55ca89caaf40 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4fc9156cf64f31c server_cookie=2c59711296b7d7a3 in_seq=0 out_seq=0 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d2effd640 1 -- 172.21.15.5:0/3265701634 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca89bffa20 con 0x55ca89ca8b50 2024-09-19T06:14:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d2effd640 10 client.18435 handle_command_reply: tid=0 2024-09-19T06:14:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d58dd2740 1 client.18435 shutdown 2024-09-19T06:14:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3265701634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 msgr2=0x55ca89c961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c961b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0d44006010 tx=0x7f0d44004260 comp rx=0 tx=0).stop 2024-09-19T06:14:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3265701634 shutdown_connections 2024-09-19T06:14:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca89ca8b50 0x55ca89caaf40 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c98cb0 0x55ca89c966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c961b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d2effd640 0 client.18435 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/3265701634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c8daf0 0x55ca89c95c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3265701634 >> 172.21.15.5:0/3265701634 conn(0x55ca89c8a060 msgr2=0x55ca89c8fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3265701634 shutdown_connections 2024-09-19T06:14:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d58dd2740 1 -- 172.21.15.5:0/3265701634 wait complete. 2024-09-19T06:14:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.961+0000 7f0d58dd2740 20 client.18435 trim_cache size 0 max 16384 2024-09-19T06:14:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.976+0000 7f0d58dd2740 1 Processor -- start 2024-09-19T06:14:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d58dd2740 1 -- start start 2024-09-19T06:14:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d58dd2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8daf0 0x55ca89c96520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d58dd2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8e410 0x55ca89cb04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d58dd2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 0x55ca89cb0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d58dd2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501afa80 con 0x55ca89c8daf0 2024-09-19T06:14:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d58dd2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501b1680 con 0x55ca89c8e410 2024-09-19T06:14:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d58dd2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501b0150 con 0x55ca89c98cb0 2024-09-19T06:14:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8daf0 0x55ca89c96520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8daf0 0x55ca89c96520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54084/0 (socket says 172.21.15.5:54084) 2024-09-19T06:14:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d4f7fe640 1 -- 172.21.15.5:0/2281012511 learned_addr learned my addr 172.21.15.5:0/2281012511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d4effd640 1 --2- 172.21.15.5:0/2281012511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8e410 0x55ca89cb04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d4ffff640 1 --2- 172.21.15.5:0/2281012511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 0x55ca89cb0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d4f7fe640 1 -- 172.21.15.5:0/2281012511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 msgr2=0x55ca89cb0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/2281012511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 0x55ca89cb0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d4f7fe640 1 -- 172.21.15.5:0/2281012511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8e410 msgr2=0x55ca89cb04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/2281012511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8e410 0x55ca89cb04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d4f7fe640 1 -- 172.21.15.5:0/2281012511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca89c96a60 con 0x55ca89c8daf0 2024-09-19T06:14:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.977+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/2281012511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8daf0 0x55ca89c96520 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0d400cd020 tx=0x7f0d400cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe48f872d8749d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.978+0000 7f0d357fa640 1 -- 172.21.15.5:0/2281012511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40020040 con 0x55ca89c8daf0 2024-09-19T06:14:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.978+0000 7f0d357fa640 1 -- 172.21.15.5:0/2281012511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d40099eb0 con 0x55ca89c8daf0 2024-09-19T06:14:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.978+0000 7f0d357fa640 1 -- 172.21.15.5:0/2281012511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d400860b0 con 0x55ca89c8daf0 2024-09-19T06:14:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.978+0000 7f0d58dd2740 1 -- 172.21.15.5:0/2281012511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8daf0 msgr2=0x55ca89c96520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.978+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/2281012511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8daf0 0x55ca89c96520 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0d400cd020 tx=0x7f0d400cf040 comp rx=0 tx=0).stop 2024-09-19T06:14:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.978+0000 7f0d58dd2740 1 -- 172.21.15.5:0/2281012511 shutdown_connections 2024-09-19T06:14:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.978+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/2281012511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 0x55ca89cb0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.978+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/2281012511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8e410 0x55ca89cb04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.978+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/2281012511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8daf0 0x55ca89c96520 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.978+0000 7f0d58dd2740 1 -- 172.21.15.5:0/2281012511 >> 172.21.15.5:0/2281012511 conn(0x55ca89c8a060 msgr2=0x55ca89c97cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.979+0000 7f0d58dd2740 1 -- 172.21.15.5:0/2281012511 shutdown_connections 2024-09-19T06:14:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.979+0000 7f0d58dd2740 1 -- 172.21.15.5:0/2281012511 wait complete. 2024-09-19T06:14:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.979+0000 7f0d58dd2740 1 -- start start 2024-09-19T06:14:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.979+0000 7f0d58dd2740 1 Processor -- start 2024-09-19T06:14:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.979+0000 7f0d58dd2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8daf0 0x55ca89ca1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.979+0000 7f0d58dd2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c9c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.979+0000 7f0d58dd2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 0x55ca89c9cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.979+0000 7f0d58dd2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501b0fb0 con 0x55ca89c8e410 2024-09-19T06:14:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.979+0000 7f0d58dd2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501b0880 con 0x55ca89c8daf0 2024-09-19T06:14:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.979+0000 7f0d58dd2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501ada80 con 0x55ca89c98cb0 2024-09-19T06:14:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.979+0000 7f0d58dd2740 10 client.0 _ll_register_callbacks cb 0x7f0d44040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:14:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c9c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c9c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54096/0 (socket says 172.21.15.5:54096) 2024-09-19T06:14:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d4effd640 1 -- 172.21.15.5:0/82121517 learned_addr learned my addr 172.21.15.5:0/82121517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:14:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8daf0 0x55ca89ca1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d4ffff640 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 0x55ca89c9cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d4effd640 1 -- 172.21.15.5:0/82121517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 msgr2=0x55ca89c9cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d4effd640 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 0x55ca89c9cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d4effd640 1 -- 172.21.15.5:0/82121517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8daf0 msgr2=0x55ca89ca1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:14:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d4effd640 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8daf0 0x55ca89ca1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d4effd640 1 -- 172.21.15.5:0/82121517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca89d124b0 con 0x55ca89c8e410 2024-09-19T06:14:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d4effd640 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c9c850 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0d44046b20 tx=0x7f0d44032870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc6c80a53d392bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:14:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d2effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:14:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d2effd640 1 -- 172.21.15.5:0/82121517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4405d070 con 0x55ca89c8e410 2024-09-19T06:14:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d58dd2740 1 -- 172.21.15.5:0/82121517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d40099eb0 con 0x55ca89c8e410 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d2effd640 1 -- 172.21.15.5:0/82121517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8550+0+0 (secure 0 0 0) 0x7f0d44046660 con 0x55ca89c8e410 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d2effd640 1 -- 172.21.15.5:0/82121517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0d44069050 con 0x55ca89c8e410 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d58dd2740 10 client.18441 fetch_fsmap learned FSMap version 39 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.980+0000 7f0d58dd2740 1 -- 172.21.15.5:0/82121517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ca89d13610 con 0x55ca89c8e410 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.981+0000 7f0d2effd640 1 -- 172.21.15.5:0/82121517 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d40099eb0 con 0x55ca89c8e410 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.981+0000 7f0d2effd640 1 -- 172.21.15.5:0/82121517 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f0d44059450 con 0x55ca89c8e410 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.981+0000 7f0d58dd2740 10 client.18441 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.981+0000 7f0d58dd2740 10 client.18441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.981+0000 7f0d58dd2740 20 client.18441 populate_metadata read hostname 'smithi005' 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.981+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca89d13d00 0x55ca89d16120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.981+0000 7f0d58dd2740 4 client.18441 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.981+0000 7f0d58dd2740 1 -- 172.21.15.5:0/82121517 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca89bf9d70 con 0x55ca89d13d00 2024-09-19T06:14:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.981+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca89d13d00 0x55ca89d16120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:14:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.981+0000 7f0d2effd640 10 client.18441 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:55.982+0000 7f0d4f7fe640 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca89d13d00 0x55ca89d16120 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617f0ebab3455da9 server_cookie=363d2c17fb3fc1df in_seq=0 out_seq=0 2024-09-19T06:14:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d2effd640 1 -- 172.21.15.5:0/82121517 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca89bf9d70 con 0x55ca89d13d00 2024-09-19T06:14:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d2effd640 10 client.18441 handle_command_reply: tid=0 2024-09-19T06:14:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d58dd2740 1 client.18441 shutdown 2024-09-19T06:14:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d58dd2740 1 -- 172.21.15.5:0/82121517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 msgr2=0x55ca89c9c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c9c850 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0d44046b20 tx=0x7f0d44032870 comp rx=0 tx=0).stop 2024-09-19T06:14:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d58dd2740 1 -- 172.21.15.5:0/82121517 shutdown_connections 2024-09-19T06:14:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca89d13d00 0x55ca89d16120 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca89c98cb0 0x55ca89c9cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca89c8e410 0x55ca89c9c850 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d58dd2740 1 --2- 172.21.15.5:0/82121517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca89c8daf0 0x55ca89ca1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d58dd2740 1 -- 172.21.15.5:0/82121517 >> 172.21.15.5:0/82121517 conn(0x55ca89c8a060 msgr2=0x55ca89c8f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d2effd640 0 client.18441 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:14:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d58dd2740 1 -- 172.21.15.5:0/82121517 shutdown_connections 2024-09-19T06:14:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d58dd2740 1 -- 172.21.15.5:0/82121517 wait complete. 2024-09-19T06:14:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.004+0000 7f0d58dd2740 20 client.18441 trim_cache size 0 max 16384 2024-09-19T06:14:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:14:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:14:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:14:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:14:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.005+0000 7f0d56042640 1 -- 172.21.15.5:0/2470219374 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0d380a33b0 msgr2=0x7f0d380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.005+0000 7f0d56042640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0d380a33b0 0x7f0d380a5870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0d48005fd0 tx=0x7f0d48010740 comp rx=0 tx=0).stop 2024-09-19T06:14:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.005+0000 7f0d56042640 1 -- 172.21.15.5:0/2470219374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d501009a0 msgr2=0x7f0d5010fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:14:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.005+0000 7f0d56042640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d501009a0 0x7f0d5010fd20 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000d950 tx=0x7f0d4000de10 comp rx=0 tx=0).stop 2024-09-19T06:14:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.005+0000 7f0d56042640 1 -- 172.21.15.5:0/2470219374 shutdown_connections 2024-09-19T06:14:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.005+0000 7f0d56042640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0d380a33b0 0x7f0d380a5870 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.005+0000 7f0d56042640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d501069b0 0x7f0d501107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.005+0000 7f0d56042640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d50101340 0x7f0d50110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.005+0000 7f0d56042640 1 --2- 172.21.15.5:0/2470219374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d501009a0 0x7f0d5010fd20 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:14:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.005+0000 7f0d56042640 1 -- 172.21.15.5:0/2470219374 >> 172.21.15.5:0/2470219374 conn(0x7f0d500779a0 msgr2=0x7f0d50103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:14:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.005+0000 7f0d56042640 1 -- 172.21.15.5:0/2470219374 shutdown_connections 2024-09-19T06:14:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:14:56.005+0000 7f0d56042640 1 -- 172.21.15.5:0/2470219374 wait complete. 2024-09-19T06:14:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:56 smithi157 ceph-mon[35346]: pgmap v361: 97 pgs: 97 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 893 KiB/s wr, 85 op/s 2024-09-19T06:14:56.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 270138 2.23 MB/sec execute 502 sec latency 11.950 ms 2024-09-19T06:14:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:56 smithi005 ceph-mon[30955]: pgmap v361: 97 pgs: 97 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 893 KiB/s wr, 85 op/s 2024-09-19T06:14:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:56 smithi049 ceph-mon[34640]: pgmap v361: 97 pgs: 97 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 893 KiB/s wr, 85 op/s 2024-09-19T06:14:57.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 270606 2.23 MB/sec execute 503 sec latency 15.691 ms 2024-09-19T06:14:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:14:58 smithi157 ceph-mon[35346]: pgmap v362: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-19T06:14:58.845 INFO:tasks.workunit.client.0.smithi005.stdout: 1 271160 2.23 MB/sec execute 504 sec latency 21.583 ms 2024-09-19T06:14:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:14:58 smithi005 ceph-mon[30955]: pgmap v362: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-19T06:14:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:14:58 smithi049 ceph-mon[34640]: pgmap v362: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-19T06:14:59.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 271743 2.23 MB/sec execute 505 sec latency 20.222 ms 2024-09-19T06:15:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:00 smithi157 ceph-mon[35346]: pgmap v363: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:00.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 272219 2.23 MB/sec execute 506 sec latency 20.632 ms 2024-09-19T06:15:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:00 smithi005 ceph-mon[30955]: pgmap v363: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:00 smithi049 ceph-mon[34640]: pgmap v363: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:01.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 272923 2.24 MB/sec execute 507 sec latency 14.794 ms 2024-09-19T06:15:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:02 smithi157 ceph-mon[35346]: pgmap v364: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:02.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273219 2.24 MB/sec execute 508 sec latency 449.497 ms 2024-09-19T06:15:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:02 smithi005 ceph-mon[30955]: pgmap v364: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:02 smithi049 ceph-mon[34640]: pgmap v364: 97 pgs: 97 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:03.840 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:15:03.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273219 2.24 MB/sec execute 509 sec latency 1449.564 ms 2024-09-19T06:15:04.113 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:04.491 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:04 smithi005 ceph-mon[30955]: pgmap v365: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-09-19T06:15:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.569+0000 7fe64a686640 1 -- 172.21.15.5:0/3004917137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644101de0 msgr2=0x7fe6441021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.569+0000 7fe64a686640 1 --2- 172.21.15.5:0/3004917137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644101de0 0x7fe6441021c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe634009900 tx=0x7fe63402a410 comp rx=0 tx=0).stop 2024-09-19T06:15:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.569+0000 7fe64a686640 1 -- 172.21.15.5:0/3004917137 shutdown_connections 2024-09-19T06:15:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.569+0000 7fe64a686640 1 --2- 172.21.15.5:0/3004917137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe644106f10 0x7fe64410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.569+0000 7fe64a686640 1 --2- 172.21.15.5:0/3004917137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe644102790 0x7fe6441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.569+0000 7fe64a686640 1 --2- 172.21.15.5:0/3004917137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644101de0 0x7fe6441021c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.569+0000 7fe64a686640 1 -- 172.21.15.5:0/3004917137 >> 172.21.15.5:0/3004917137 conn(0x7fe6440fd8c0 msgr2=0x7fe6440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.569+0000 7fe64a686640 1 -- 172.21.15.5:0/3004917137 shutdown_connections 2024-09-19T06:15:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.569+0000 7fe64a686640 1 -- 172.21.15.5:0/3004917137 wait complete. 2024-09-19T06:15:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.570+0000 7fe64a686640 1 Processor -- start 2024-09-19T06:15:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.570+0000 7fe64a686640 1 -- start start 2024-09-19T06:15:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.570+0000 7fe64a686640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe644101de0 0x7fe644198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.570+0000 7fe64a686640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe644102790 0x7fe6441988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.570+0000 7fe64a686640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644106f10 0x7fe64419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.570+0000 7fe64a686640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe644111e20 con 0x7fe644106f10 2024-09-19T06:15:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.570+0000 7fe64a686640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe644111ca0 con 0x7fe644101de0 2024-09-19T06:15:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.570+0000 7fe64a686640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe644111fa0 con 0x7fe644102790 2024-09-19T06:15:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe648c23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644106f10 0x7fe64419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe648c23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644106f10 0x7fe64419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37030/0 (socket says 172.21.15.5:37030) 2024-09-19T06:15:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe648c23640 1 -- 172.21.15.5:0/1431169765 learned_addr learned my addr 172.21.15.5:0/1431169765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe643fff640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe644101de0 0x7fe644198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe6437fe640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe644102790 0x7fe6441988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe648c23640 1 -- 172.21.15.5:0/1431169765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe644102790 msgr2=0x7fe6441988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe648c23640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe644102790 0x7fe6441988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe648c23640 1 -- 172.21.15.5:0/1431169765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe644101de0 msgr2=0x7fe644198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe648c23640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe644101de0 0x7fe644198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe648c23640 1 -- 172.21.15.5:0/1431169765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64419d310 con 0x7fe644106f10 2024-09-19T06:15:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe648c23640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644106f10 0x7fe64419cc30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00eaa0 tx=0x7fe63c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe6417fa640 1 -- 172.21.15.5:0/1431169765 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c00cdb0 con 0x7fe644106f10 2024-09-19T06:15:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe6417fa640 1 -- 172.21.15.5:0/1431169765 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe63c005020 con 0x7fe644106f10 2024-09-19T06:15:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe64a686640 1 -- 172.21.15.5:0/1431169765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64419d600 con 0x7fe644106f10 2024-09-19T06:15:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe6417fa640 1 -- 172.21.15.5:0/1431169765 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0116d0 con 0x7fe644106f10 2024-09-19T06:15:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.571+0000 7fe64a686640 1 -- 172.21.15.5:0/1431169765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6441a4ee0 con 0x7fe644106f10 2024-09-19T06:15:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe6417fa640 1 -- 172.21.15.5:0/1431169765 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe63c0051d0 con 0x7fe644106f10 2024-09-19T06:15:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe64ca5f740 1 Processor -- start 2024-09-19T06:15:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe64ca5f740 1 -- start start 2024-09-19T06:15:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe64ca5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f521c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe64ca5f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 0x55a4f5227140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe64ca5f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f5227680 0x55a4f5229a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe64ca5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a4f51822e0 con 0x55a4f521c4a0 2024-09-19T06:15:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe64ca5f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4f5129ce0 con 0x55a4f5227680 2024-09-19T06:15:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe64ca5f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a4f517e2a0 con 0x55a4f521cdc0 2024-09-19T06:15:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe6417fa640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe62c0a33b0 0x7fe62c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe6437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f521c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe6437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f521c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37036/0 (socket says 172.21.15.5:37036) 2024-09-19T06:15:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.574+0000 7fe6437fe640 1 -- 172.21.15.5:0/1704888862 learned_addr learned my addr 172.21.15.5:0/1704888862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe643fff640 1 --2- 172.21.15.5:0/1704888862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 0x55a4f5227140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe6417fa640 1 -- 172.21.15.5:0/1431169765 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8594+0+0 (secure 0 0 0) 0x7fe63c0cfed0 con 0x7fe644106f10 2024-09-19T06:15:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe648c23640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe62c0a33b0 0x7fe62c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe6437fe640 1 --2- 172.21.15.5:0/1704888862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f5227680 0x55a4f5229a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe6437fe640 1 -- 172.21.15.5:0/1704888862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 msgr2=0x55a4f5227140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe6437fe640 1 --2- 172.21.15.5:0/1704888862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 0x55a4f5227140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe6437fe640 1 -- 172.21.15.5:0/1704888862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f5227680 msgr2=0x55a4f5229a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe6437fe640 1 --2- 172.21.15.5:0/1704888862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f5227680 0x55a4f5229a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe6437fe640 1 -- 172.21.15.5:0/1704888862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f510e440 con 0x55a4f521c4a0 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe6437fe640 1 --2- 172.21.15.5:0/1704888862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f521c880 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe6380058c0 tx=0x7fe63800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca5194e63af3a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe648c23640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe62c0a33b0 0x7fe62c0a5870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00efd0 tx=0x7fe63c000f40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe62a7fc640 1 -- 172.21.15.5:0/1704888862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638019030 con 0x55a4f521c4a0 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe62a7fc640 1 -- 172.21.15.5:0/1704888862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6380035f0 con 0x55a4f521c4a0 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.575+0000 7fe62a7fc640 1 -- 172.21.15.5:0/1704888862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6380116a0 con 0x55a4f521c4a0 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.576+0000 7fe64ca5f740 1 -- 172.21.15.5:0/1704888862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 msgr2=0x55a4f521c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.576+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/1704888862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f521c880 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe6380058c0 tx=0x7fe63800b6a0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.576+0000 7fe64ca5f740 1 -- 172.21.15.5:0/1704888862 shutdown_connections 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.576+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/1704888862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f5227680 0x55a4f5229a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.576+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/1704888862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 0x55a4f5227140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.576+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/1704888862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f521c880 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.576+0000 7fe64ca5f740 1 -- 172.21.15.5:0/1704888862 >> 172.21.15.5:0/1704888862 conn(0x55a4f5218a30 msgr2=0x55a4f5218e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.576+0000 7fe64ca5f740 1 -- 172.21.15.5:0/1704888862 shutdown_connections 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.576+0000 7fe64ca5f740 1 -- 172.21.15.5:0/1704888862 wait complete. 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.576+0000 7fe64ca5f740 1 -- start start 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe64ca5f740 1 Processor -- start 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe64ca5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f5226f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe64ca5f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f521cdc0 0x55a4f5224240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe64ca5f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f5227680 0x55a4f5224780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe64ca5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a4f522a250 con 0x55a4f521c4a0 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe64ca5f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4f513d050 con 0x55a4f521cdc0 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe64ca5f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a4f522a3d0 con 0x55a4f5227680 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe64ca5f740 10 client.0 _ll_register_callbacks cb 0x55a4f517d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe6437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f5226f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe6437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f5226f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37052/0 (socket says 172.21.15.5:37052) 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe6437fe640 1 -- 172.21.15.5:0/2887509974 learned_addr learned my addr 172.21.15.5:0/2887509974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:04.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe643fff640 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f521cdc0 0x55a4f5224240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe6437fe640 1 -- 172.21.15.5:0/2887509974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f5227680 msgr2=0x55a4f5224780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:04.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe6437fe640 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f5227680 0x55a4f5224780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe6437fe640 1 -- 172.21.15.5:0/2887509974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f521cdc0 msgr2=0x55a4f5224240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:04.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe6437fe640 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f521cdc0 0x55a4f5224240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe6437fe640 1 -- 172.21.15.5:0/2887509974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f4ef33c0 con 0x55a4f521c4a0 2024-09-19T06:15:04.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe643fff640 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f521cdc0 0x55a4f5224240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:04.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe6437fe640 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f5226f90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe638016040 tx=0x7fe63800b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4865cef9082290 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:04.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.577+0000 7fe6237fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:04.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe6237fe640 1 -- 172.21.15.5:0/2887509974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638019040 con 0x55a4f521c4a0 2024-09-19T06:15:04.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe6237fe640 1 -- 172.21.15.5:0/2887509974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8594+0+0 (secure 0 0 0) 0x7fe638004000 con 0x55a4f521c4a0 2024-09-19T06:15:04.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2887509974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6380035f0 con 0x55a4f521c4a0 2024-09-19T06:15:04.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe6237fe640 1 -- 172.21.15.5:0/2887509974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe638004200 con 0x55a4f521c4a0 2024-09-19T06:15:04.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe64ca5f740 10 client.18453 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:04.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2887509974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a4f5235bd0 con 0x55a4f521c4a0 2024-09-19T06:15:04.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe6237fe640 1 -- 172.21.15.5:0/2887509974 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6380035f0 con 0x55a4f521c4a0 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe6237fe640 1 -- 172.21.15.5:0/2887509974 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fe638025070 con 0x55a4f521c4a0 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe64ca5f740 10 client.18453 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe64ca5f740 10 client.18453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe64ca5f740 20 client.18453 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4f5236f80 0x55a4f5239370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe64ca5f740 4 client.18453 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.578+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2887509974 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4f518f850 con 0x55a4f5236f80 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.579+0000 7fe643fff640 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4f5236f80 0x55a4f5239370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.579+0000 7fe643fff640 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4f5236f80 0x55a4f5239370 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9572b2cd64a93cbe server_cookie=a255e2f25ff823a8 in_seq=0 out_seq=0 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.579+0000 7fe6237fe640 10 client.18453 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe6237fe640 1 -- 172.21.15.5:0/2887509974 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4f518f850 con 0x55a4f5236f80 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe6237fe640 10 client.18453 handle_command_reply: tid=0 2024-09-19T06:15:04.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe64ca5f740 1 client.18453 shutdown 2024-09-19T06:15:04.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2887509974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 msgr2=0x55a4f5226f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:04.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f5226f90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe638016040 tx=0x7fe63800b7d0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2887509974 shutdown_connections 2024-09-19T06:15:04.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4f5236f80 0x55a4f5239370 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f5227680 0x55a4f5224780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f521cdc0 0x55a4f5224240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/2887509974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f5226f90 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2887509974 >> 172.21.15.5:0/2887509974 conn(0x55a4f5218a30 msgr2=0x55a4f521e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:04.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe6237fe640 0 client.18453 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:04.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2887509974 shutdown_connections 2024-09-19T06:15:04.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2887509974 wait complete. 2024-09-19T06:15:04.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.580+0000 7fe64ca5f740 20 client.18453 trim_cache size 0 max 16384 2024-09-19T06:15:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.597+0000 7fe64ca5f740 1 Processor -- start 2024-09-19T06:15:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.597+0000 7fe64ca5f740 1 -- start start 2024-09-19T06:15:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe64ca5f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f521c4a0 0x55a4f52334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe64ca5f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 0x55a4f522fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe64ca5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f5227680 0x55a4f522ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe64ca5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441ad9b0 con 0x55a4f5227680 2024-09-19T06:15:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe64ca5f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441af5b0 con 0x55a4f521c4a0 2024-09-19T06:15:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe64ca5f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441ae080 con 0x55a4f521cdc0 2024-09-19T06:15:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe6437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f5227680 0x55a4f522ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe6437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f5227680 0x55a4f522ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37068/0 (socket says 172.21.15.5:37068) 2024-09-19T06:15:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe6437fe640 1 -- 172.21.15.5:0/3628364319 learned_addr learned my addr 172.21.15.5:0/3628364319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe6437fe640 1 --2- 172.21.15.5:0/3628364319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f521c4a0 0x55a4f52334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe643fff640 1 --2- 172.21.15.5:0/3628364319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 0x55a4f522fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe6437fe640 1 -- 172.21.15.5:0/3628364319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 msgr2=0x55a4f522fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe6437fe640 1 --2- 172.21.15.5:0/3628364319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 0x55a4f522fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe6437fe640 1 -- 172.21.15.5:0/3628364319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f521c4a0 msgr2=0x55a4f52334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe6437fe640 1 --2- 172.21.15.5:0/3628364319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f521c4a0 0x55a4f52334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe6437fe640 1 -- 172.21.15.5:0/3628364319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f5219ae0 con 0x55a4f5227680 2024-09-19T06:15:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe6437fe640 1 --2- 172.21.15.5:0/3628364319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f5227680 0x55a4f522ff40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe638004ae0 tx=0x7fe638005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276370c345730d11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe629ffb640 1 -- 172.21.15.5:0/3628364319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638019030 con 0x55a4f5227680 2024-09-19T06:15:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe629ffb640 1 -- 172.21.15.5:0/3628364319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe63800b420 con 0x55a4f5227680 2024-09-19T06:15:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.598+0000 7fe629ffb640 1 -- 172.21.15.5:0/3628364319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638004b60 con 0x55a4f5227680 2024-09-19T06:15:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.599+0000 7fe64ca5f740 1 -- 172.21.15.5:0/3628364319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f5227680 msgr2=0x55a4f522ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.599+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/3628364319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f5227680 0x55a4f522ff40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe638004ae0 tx=0x7fe638005970 comp rx=0 tx=0).stop 2024-09-19T06:15:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.599+0000 7fe64ca5f740 1 -- 172.21.15.5:0/3628364319 shutdown_connections 2024-09-19T06:15:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.599+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/3628364319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f5227680 0x55a4f522ff40 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.599+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/3628364319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 0x55a4f522fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.599+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/3628364319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f521c4a0 0x55a4f52334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.599+0000 7fe64ca5f740 1 -- 172.21.15.5:0/3628364319 >> 172.21.15.5:0/3628364319 conn(0x55a4f5218a30 msgr2=0x55a4f5223330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.599+0000 7fe64ca5f740 1 -- 172.21.15.5:0/3628364319 shutdown_connections 2024-09-19T06:15:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.599+0000 7fe64ca5f740 1 -- 172.21.15.5:0/3628364319 wait complete. 2024-09-19T06:15:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.599+0000 7fe64ca5f740 1 -- start start 2024-09-19T06:15:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe64ca5f740 1 Processor -- start 2024-09-19T06:15:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe64ca5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f529dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe64ca5f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 0x55a4f529e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe64ca5f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f5227680 0x55a4f5298150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe64ca5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441aeee0 con 0x55a4f521c4a0 2024-09-19T06:15:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe64ca5f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441ae7b0 con 0x55a4f5227680 2024-09-19T06:15:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe64ca5f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441ab9b0 con 0x55a4f521cdc0 2024-09-19T06:15:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe64ca5f740 10 client.0 _ll_register_callbacks cb 0x7fe634005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe6437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f529dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe6437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f529dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37072/0 (socket says 172.21.15.5:37072) 2024-09-19T06:15:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe6437fe640 1 -- 172.21.15.5:0/2645041562 learned_addr learned my addr 172.21.15.5:0/2645041562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe643fff640 1 --2- 172.21.15.5:0/2645041562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 0x55a4f529e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe6437fe640 1 -- 172.21.15.5:0/2645041562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 msgr2=0x55a4f529e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe6437fe640 1 --2- 172.21.15.5:0/2645041562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 0x55a4f529e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe6437fe640 1 -- 172.21.15.5:0/2645041562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f5227680 msgr2=0x55a4f5298150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe6437fe640 1 --2- 172.21.15.5:0/2645041562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f5227680 0x55a4f5298150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.600+0000 7fe6437fe640 1 -- 172.21.15.5:0/2645041562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f5298840 con 0x55a4f521c4a0 2024-09-19T06:15:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe6437fe640 1 --2- 172.21.15.5:0/2645041562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f529dd30 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe63800b830 tx=0x7fe63800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50eeddc4a08901ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe6237fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe6237fe640 1 -- 172.21.15.5:0/2645041562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63803b020 con 0x55a4f521c4a0 2024-09-19T06:15:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe6237fe640 1 -- 172.21.15.5:0/2645041562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8594+0+0 (secure 0 0 0) 0x7fe638003ff0 con 0x55a4f521c4a0 2024-09-19T06:15:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2645041562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe63800b420 con 0x55a4f521c4a0 2024-09-19T06:15:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe6237fe640 1 -- 172.21.15.5:0/2645041562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe638019040 con 0x55a4f521c4a0 2024-09-19T06:15:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe64ca5f740 10 client.18459 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2645041562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a4f52a5550 con 0x55a4f521c4a0 2024-09-19T06:15:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe6237fe640 1 -- 172.21.15.5:0/2645041562 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe63800b420 con 0x55a4f521c4a0 2024-09-19T06:15:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe6237fe640 1 -- 172.21.15.5:0/2645041562 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fe638025070 con 0x55a4f521c4a0 2024-09-19T06:15:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe64ca5f740 10 client.18459 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe64ca5f740 10 client.18459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe64ca5f740 20 client.18459 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/2645041562 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4f52a5ba0 0x55a4f52a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe64ca5f740 4 client.18459 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:15:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.601+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2645041562 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a4f5189d60 con 0x55a4f52a5ba0 2024-09-19T06:15:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe643fff640 1 --2- 172.21.15.5:0/2645041562 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4f52a5ba0 0x55a4f52a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe643fff640 1 --2- 172.21.15.5:0/2645041562 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4f52a5ba0 0x55a4f52a7fc0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22b17910d8411727 server_cookie=f46dc91455a56a49 in_seq=0 out_seq=0 2024-09-19T06:15:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe6237fe640 10 client.18459 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe6237fe640 1 -- 172.21.15.5:0/2645041562 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a4f5189d60 con 0x55a4f52a5ba0 2024-09-19T06:15:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe6237fe640 10 client.18459 handle_command_reply: tid=0 2024-09-19T06:15:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe64ca5f740 1 client.18459 shutdown 2024-09-19T06:15:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2645041562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 msgr2=0x55a4f529dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/2645041562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f529dd30 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe63800b830 tx=0x7fe63800bfa0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2645041562 shutdown_connections 2024-09-19T06:15:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/2645041562 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4f52a5ba0 0x55a4f52a7fc0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe6237fe640 0 client.18459 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/2645041562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4f5227680 0x55a4f5298150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/2645041562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4f521cdc0 0x55a4f529e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe64ca5f740 1 --2- 172.21.15.5:0/2645041562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4f521c4a0 0x55a4f529dd30 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.602+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2645041562 >> 172.21.15.5:0/2645041562 conn(0x55a4f5218a30 msgr2=0x55a4f5217370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2645041562 shutdown_connections 2024-09-19T06:15:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64ca5f740 1 -- 172.21.15.5:0/2645041562 wait complete. 2024-09-19T06:15:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64ca5f740 20 client.18459 trim_cache size 0 max 16384 2024-09-19T06:15:04.611 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:15:04.611 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:15:04.611 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:15:04.611 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:15:04.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64a686640 1 -- 172.21.15.5:0/1431169765 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe62c0a33b0 msgr2=0x7fe62c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:04.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64a686640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe62c0a33b0 0x7fe62c0a5870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00efd0 tx=0x7fe63c000f40 comp rx=0 tx=0).stop 2024-09-19T06:15:04.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64a686640 1 -- 172.21.15.5:0/1431169765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644106f10 msgr2=0x7fe64419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:04.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64a686640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644106f10 0x7fe64419cc30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00eaa0 tx=0x7fe63c00ef60 comp rx=0 tx=0).stop 2024-09-19T06:15:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64a686640 1 -- 172.21.15.5:0/1431169765 shutdown_connections 2024-09-19T06:15:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64a686640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644106f10 0x7fe64419cc30 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64a686640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe62c0a33b0 0x7fe62c0a5870 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64a686640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe644102790 0x7fe6441988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64a686640 1 --2- 172.21.15.5:0/1431169765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe644101de0 0x7fe644198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64a686640 1 -- 172.21.15.5:0/1431169765 >> 172.21.15.5:0/1431169765 conn(0x7fe6440fd8c0 msgr2=0x7fe6440fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64a686640 1 -- 172.21.15.5:0/1431169765 shutdown_connections 2024-09-19T06:15:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:04.603+0000 7fe64a686640 1 -- 172.21.15.5:0/1431169765 wait complete. 2024-09-19T06:15:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:04 smithi157 ceph-mon[35346]: pgmap v365: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-09-19T06:15:04.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273219 2.23 MB/sec execute 510 sec latency 2449.591 ms 2024-09-19T06:15:04.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:04 smithi049 ceph-mon[34640]: pgmap v365: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-09-19T06:15:05.387 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:15:05.388 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:15:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:05 smithi005 ceph-mon[30955]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T06:15:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:05 smithi157 ceph-mon[35346]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T06:15:05.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273286 2.23 MB/sec execute 511 sec latency 3070.518 ms 2024-09-19T06:15:05.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:05 smithi049 ceph-mon[34640]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T06:15:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.165+0000 7fafce7d8640 1 -- 172.21.15.5:0/972327553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc8102d40 msgr2=0x7fafc8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.165+0000 7fafce7d8640 1 --2- 172.21.15.5:0/972327553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc8102d40 0x7fafc8103120 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009900 tx=0x7fafb802a480 comp rx=0 tx=0).stop 2024-09-19T06:15:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.165+0000 7fafce7d8640 1 -- 172.21.15.5:0/972327553 shutdown_connections 2024-09-19T06:15:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.165+0000 7fafce7d8640 1 --2- 172.21.15.5:0/972327553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc810c170 0x7fafc810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.165+0000 7fafce7d8640 1 --2- 172.21.15.5:0/972327553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc8103660 0x7fafc810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.165+0000 7fafce7d8640 1 --2- 172.21.15.5:0/972327553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc8102d40 0x7fafc8103120 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.166+0000 7fafce7d8640 1 -- 172.21.15.5:0/972327553 >> 172.21.15.5:0/972327553 conn(0x7fafc80fc7a0 msgr2=0x7fafc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.166+0000 7fafce7d8640 1 -- 172.21.15.5:0/972327553 shutdown_connections 2024-09-19T06:15:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.166+0000 7fafce7d8640 1 -- 172.21.15.5:0/972327553 wait complete. 2024-09-19T06:15:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.166+0000 7fafce7d8640 1 Processor -- start 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.166+0000 7fafce7d8640 1 -- start start 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafce7d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc8102d40 0x7fafc8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafce7d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc8103660 0x7fafc8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafce7d8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc810c170 0x7fafc819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafce7d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8111e20 con 0x7fafc8102d40 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafce7d8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8111ca0 con 0x7fafc8103660 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafce7d8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8111fa0 con 0x7fafc810c170 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc8102d40 0x7fafc8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc8102d40 0x7fafc8198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37098/0 (socket says 172.21.15.5:37098) 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc7fff640 1 -- 172.21.15.5:0/3022840990 learned_addr learned my addr 172.21.15.5:0/3022840990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc77fe640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc8103660 0x7fafc8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafccd75640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc810c170 0x7fafc819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc7fff640 1 -- 172.21.15.5:0/3022840990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc810c170 msgr2=0x7fafc819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:06.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc7fff640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc810c170 0x7fafc819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:06.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc7fff640 1 -- 172.21.15.5:0/3022840990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc8103660 msgr2=0x7fafc8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:06.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc7fff640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc8103660 0x7fafc8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:06.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc7fff640 1 -- 172.21.15.5:0/3022840990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc819d3b0 con 0x7fafc8102d40 2024-09-19T06:15:06.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc7fff640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc8102d40 0x7fafc8198400 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fafb8005940 tx=0x7fafb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:06.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc57fa640 1 -- 172.21.15.5:0/3022840990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb803f030 con 0x7fafc8102d40 2024-09-19T06:15:06.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafce7d8640 1 -- 172.21.15.5:0/3022840990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc819d640 con 0x7fafc8102d40 2024-09-19T06:15:06.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc57fa640 1 -- 172.21.15.5:0/3022840990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafb8003cb0 con 0x7fafc8102d40 2024-09-19T06:15:06.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.167+0000 7fafc57fa640 1 -- 172.21.15.5:0/3022840990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8035070 con 0x7fafc8102d40 2024-09-19T06:15:06.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.168+0000 7fafce7d8640 1 -- 172.21.15.5:0/3022840990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc81a4f80 con 0x7fafc8102d40 2024-09-19T06:15:06.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.169+0000 7fafce7d8640 1 -- 172.21.15.5:0/3022840990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc806b070 con 0x7fafc8102d40 2024-09-19T06:15:06.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.170+0000 7fafc57fa640 1 -- 172.21.15.5:0/3022840990 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fafb8042030 con 0x7fafc8102d40 2024-09-19T06:15:06.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.171+0000 7fafc57fa640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fafb00a33b0 0x7fafb00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:06.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.171+0000 7fafc77fe640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fafb00a33b0 0x7fafb00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:06.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.171+0000 7fafc57fa640 1 -- 172.21.15.5:0/3022840990 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8594+0+0 (secure 0 0 0) 0x7fafb80f0530 con 0x7fafc8102d40 2024-09-19T06:15:06.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.171+0000 7fafc77fe640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fafb00a33b0 0x7fafb00a5870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fafc80fee70 tx=0x7fafbc00b2e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:06.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.176+0000 7fafc57fa640 1 -- 172.21.15.5:0/3022840990 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fafb80bcec0 con 0x7fafc8102d40 2024-09-19T06:15:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.386+0000 7fafce7d8640 1 -- 172.21.15.5:0/3022840990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafc8107210 con 0x7fafc8102d40 2024-09-19T06:15:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.387+0000 7fafc57fa640 1 -- 172.21.15.5:0/3022840990 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11965 (secure 0 0 0) 0x7fafb80bc7c0 con 0x7fafc8102d40 2024-09-19T06:15:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:15:06.389 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":39,"btime":"2024-09-19T06:14:16:386479+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:14:15.452508+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:15:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 39 2024-09-19T06:15:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.389+0000 7fafce7d8640 1 -- 172.21.15.5:0/3022840990 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fafb00a33b0 msgr2=0x7fafb00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.389+0000 7fafce7d8640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fafb00a33b0 0x7fafb00a5870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fafc80fee70 tx=0x7fafbc00b2e0 comp rx=0 tx=0).stop 2024-09-19T06:15:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.389+0000 7fafce7d8640 1 -- 172.21.15.5:0/3022840990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc8102d40 msgr2=0x7fafc8198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.389+0000 7fafce7d8640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc8102d40 0x7fafc8198400 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fafb8005940 tx=0x7fafb80042f0 comp rx=0 tx=0).stop 2024-09-19T06:15:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.389+0000 7fafce7d8640 1 -- 172.21.15.5:0/3022840990 shutdown_connections 2024-09-19T06:15:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.389+0000 7fafce7d8640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fafb00a33b0 0x7fafb00a5870 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.389+0000 7fafce7d8640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc810c170 0x7fafc819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.389+0000 7fafce7d8640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc8103660 0x7fafc8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.389+0000 7fafce7d8640 1 --2- 172.21.15.5:0/3022840990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc8102d40 0x7fafc8198400 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.389+0000 7fafce7d8640 1 -- 172.21.15.5:0/3022840990 >> 172.21.15.5:0/3022840990 conn(0x7fafc80fc7a0 msgr2=0x7fafc80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.389+0000 7fafce7d8640 1 -- 172.21.15.5:0/3022840990 shutdown_connections 2024-09-19T06:15:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:06.389+0000 7fafce7d8640 1 -- 172.21.15.5:0/3022840990 wait complete. 2024-09-19T06:15:06.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:06 smithi005 ceph-mon[30955]: pgmap v367: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-09-19T06:15:06.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:06 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3022840990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:15:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:06 smithi157 ceph-mon[35346]: pgmap v367: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-09-19T06:15:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:06 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3022840990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:15:06.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273286 2.22 MB/sec execute 512 sec latency 1117.942 ms 2024-09-19T06:15:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:06 smithi049 ceph-mon[34640]: pgmap v367: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-09-19T06:15:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:06 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3022840990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:15:07.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:15:07.318 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.726+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/438151725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ea4106a80 msgr2=0x7f6ea4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.726+0000 7f6eaa2d9640 1 --2- 172.21.15.5:0/438151725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ea4106a80 0x7f6ea4106e60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009930 tx=0x7f6e9402a420 comp rx=0 tx=0).stop 2024-09-19T06:15:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.726+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/438151725 shutdown_connections 2024-09-19T06:15:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.726+0000 7f6eaa2d9640 1 --2- 172.21.15.5:0/438151725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ea4101420 0x7f6ea410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.726+0000 7f6eaa2d9640 1 --2- 172.21.15.5:0/438151725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ea4100a80 0x7f6ea4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.726+0000 7f6eaa2d9640 1 --2- 172.21.15.5:0/438151725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ea4106a80 0x7f6ea4106e60 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.726+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/438151725 >> 172.21.15.5:0/438151725 conn(0x7f6ea40fc7a0 msgr2=0x7f6ea40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:07.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.726+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/438151725 shutdown_connections 2024-09-19T06:15:07.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.726+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/438151725 wait complete. 2024-09-19T06:15:07.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.727+0000 7f6eaa2d9640 1 Processor -- start 2024-09-19T06:15:07.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.727+0000 7f6eaa2d9640 1 -- start start 2024-09-19T06:15:07.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.727+0000 7f6eaa2d9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ea4100a80 0x7f6ea41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.727+0000 7f6eaa2d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ea4101420 0x7f6ea4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.727+0000 7f6eaa2d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ea4106a80 0x7f6ea419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.727+0000 7f6eaa2d9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea4110d30 con 0x7f6ea4101420 2024-09-19T06:15:07.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.727+0000 7f6eaa2d9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea4110bb0 con 0x7f6ea4100a80 2024-09-19T06:15:07.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.727+0000 7f6eaa2d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea4110eb0 con 0x7f6ea4106a80 2024-09-19T06:15:07.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.727+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ea4101420 0x7f6ea4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.727+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ea4101420 0x7f6ea4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37124/0 (socket says 172.21.15.5:37124) 2024-09-19T06:15:07.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.727+0000 7f6ea37fe640 1 -- 172.21.15.5:0/1252690079 learned_addr learned my addr 172.21.15.5:0/1252690079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:07.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6ea3fff640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ea4100a80 0x7f6ea41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6ea8876640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ea4106a80 0x7f6ea419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6ea37fe640 1 -- 172.21.15.5:0/1252690079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ea4106a80 msgr2=0x7f6ea419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:07.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ea4106a80 0x7f6ea419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6ea37fe640 1 -- 172.21.15.5:0/1252690079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ea4100a80 msgr2=0x7f6ea41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:07.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ea4100a80 0x7f6ea41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6ea37fe640 1 -- 172.21.15.5:0/1252690079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea419d380 con 0x7f6ea4101420 2024-09-19T06:15:07.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6ea3fff640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ea4100a80 0x7f6ea41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:07.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ea4101420 0x7f6ea4198910 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800d950 tx=0x7f6e9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:07.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6ea17fa640 1 -- 172.21.15.5:0/1252690079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98017070 con 0x7f6ea4101420 2024-09-19T06:15:07.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6ea17fa640 1 -- 172.21.15.5:0/1252690079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e98009c30 con 0x7f6ea4101420 2024-09-19T06:15:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/1252690079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea419d670 con 0x7f6ea4101420 2024-09-19T06:15:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6ea17fa640 1 -- 172.21.15.5:0/1252690079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800f620 con 0x7f6ea4101420 2024-09-19T06:15:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.728+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/1252690079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea41a4f50 con 0x7f6ea4101420 2024-09-19T06:15:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6eac6b2740 1 Processor -- start 2024-09-19T06:15:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6ea17fa640 1 -- 172.21.15.5:0/1252690079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6e980109a0 con 0x7f6ea4101420 2024-09-19T06:15:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6eac6b2740 1 -- start start 2024-09-19T06:15:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6eac6b2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0fb40 0x564b8ca0e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6eac6b2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0e5d0 0x564b8ca0ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6eac6b2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 0x564b8ca1b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6eac6b2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564b8c9742f0 con 0x564b8ca19370 2024-09-19T06:15:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6eac6b2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564b8c91bcf0 con 0x564b8ca0e5d0 2024-09-19T06:15:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6eac6b2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564b8c9702b0 con 0x564b8ca0fb40 2024-09-19T06:15:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 0x564b8ca1b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 0x564b8ca1b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37132/0 (socket says 172.21.15.5:37132) 2024-09-19T06:15:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6ea37fe640 1 -- 172.21.15.5:0/1783858521 learned_addr learned my addr 172.21.15.5:0/1783858521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6ea3fff640 1 --2- 172.21.15.5:0/1783858521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0e5d0 0x564b8ca0ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6ea8876640 1 --2- 172.21.15.5:0/1783858521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0fb40 0x564b8ca0e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6ea37fe640 1 -- 172.21.15.5:0/1783858521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0fb40 msgr2=0x564b8ca0e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/1783858521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0fb40 0x564b8ca0e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6ea37fe640 1 -- 172.21.15.5:0/1783858521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0e5d0 msgr2=0x564b8ca0ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/1783858521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0e5d0 0x564b8ca0ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6ea37fe640 1 -- 172.21.15.5:0/1783858521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b8c9003a0 con 0x564b8ca19370 2024-09-19T06:15:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.731+0000 7f6ea8876640 1 --2- 172.21.15.5:0/1783858521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0fb40 0x564b8ca0e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.732+0000 7f6ea17fa640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6e8c0a33b0 0x7f6e8c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.732+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/1783858521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 0x564b8ca1b760 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6e980cd0e0 tx=0x7f6e980d3f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385f27167c828d20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.732+0000 7f6ea8876640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6e8c0a33b0 0x7f6e8c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.732+0000 7f6ea17fa640 1 -- 172.21.15.5:0/1252690079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8594+0+0 (secure 0 0 0) 0x7f6e980cfa70 con 0x7f6ea4101420 2024-09-19T06:15:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.732+0000 7f6e8a7fc640 1 -- 172.21.15.5:0/1783858521 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98097fa0 con 0x564b8ca19370 2024-09-19T06:15:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.732+0000 7f6e8a7fc640 1 -- 172.21.15.5:0/1783858521 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e9809c400 con 0x564b8ca19370 2024-09-19T06:15:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.732+0000 7f6e8a7fc640 1 -- 172.21.15.5:0/1783858521 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9809b660 con 0x564b8ca19370 2024-09-19T06:15:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.733+0000 7f6eac6b2740 1 -- 172.21.15.5:0/1783858521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 msgr2=0x564b8ca1b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.733+0000 7f6ea8876640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6e8c0a33b0 0x7f6e8c0a5870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c008de0 tx=0x7f6e9c0087c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.733+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/1783858521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 0x564b8ca1b760 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6e980cd0e0 tx=0x7f6e980d3f80 comp rx=0 tx=0).stop 2024-09-19T06:15:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.733+0000 7f6eac6b2740 1 -- 172.21.15.5:0/1783858521 shutdown_connections 2024-09-19T06:15:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.733+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/1783858521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 0x564b8ca1b760 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.733+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/1783858521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0e5d0 0x564b8ca0ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.733+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/1783858521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0fb40 0x564b8ca0e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.733+0000 7f6eac6b2740 1 -- 172.21.15.5:0/1783858521 >> 172.21.15.5:0/1783858521 conn(0x564b8c9d7310 msgr2=0x564b8ca0ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.733+0000 7f6eac6b2740 1 -- 172.21.15.5:0/1783858521 shutdown_connections 2024-09-19T06:15:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.733+0000 7f6eac6b2740 1 -- 172.21.15.5:0/1783858521 wait complete. 2024-09-19T06:15:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.733+0000 7f6eac6b2740 1 -- start start 2024-09-19T06:15:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.733+0000 7f6eac6b2740 1 Processor -- start 2024-09-19T06:15:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6eac6b2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0e5d0 0x564b8ca18dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6eac6b2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0fb40 0x564b8ca16070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6eac6b2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 0x564b8ca16800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6eac6b2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564b8ca1c010 con 0x564b8ca19370 2024-09-19T06:15:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6eac6b2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564b8c92f060 con 0x564b8ca0fb40 2024-09-19T06:15:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6eac6b2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564b8ca1c190 con 0x564b8ca0e5d0 2024-09-19T06:15:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6eac6b2740 10 client.0 _ll_register_callbacks cb 0x564b8c6828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6ea8876640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 0x564b8ca16800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6ea8876640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 0x564b8ca16800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37138/0 (socket says 172.21.15.5:37138) 2024-09-19T06:15:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6ea8876640 1 -- 172.21.15.5:0/2146087364 learned_addr learned my addr 172.21.15.5:0/2146087364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6ea3fff640 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0e5d0 0x564b8ca18dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0fb40 0x564b8ca16070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6ea8876640 1 -- 172.21.15.5:0/2146087364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0e5d0 msgr2=0x564b8ca18dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6ea8876640 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0e5d0 0x564b8ca18dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6ea8876640 1 -- 172.21.15.5:0/2146087364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0fb40 msgr2=0x564b8ca16070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6ea8876640 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0fb40 0x564b8ca16070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6ea8876640 1 -- 172.21.15.5:0/2146087364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b8c6e53c0 con 0x564b8ca19370 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0fb40 0x564b8ca16070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6ea8876640 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 0x564b8ca16800 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c008700 tx=0x7f6e9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f011034a19a4030 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.734+0000 7f6e837fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.735+0000 7f6e837fe640 1 -- 172.21.15.5:0/2146087364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c01e620 con 0x564b8ca19370 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.735+0000 7f6e837fe640 1 -- 172.21.15.5:0/2146087364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8594+0+0 (secure 0 0 0) 0x7f6e9c01e7c0 con 0x564b8ca19370 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.735+0000 7f6eac6b2740 1 -- 172.21.15.5:0/2146087364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e9809c400 con 0x564b8ca19370 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.735+0000 7f6e837fe640 1 -- 172.21.15.5:0/2146087364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6e9c008e10 con 0x564b8ca19370 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.735+0000 7f6eac6b2740 10 client.18477 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.735+0000 7f6eac6b2740 1 -- 172.21.15.5:0/2146087364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564b8ca27b80 con 0x564b8ca19370 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.735+0000 7f6e837fe640 1 -- 172.21.15.5:0/2146087364 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e9809c400 con 0x564b8ca19370 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.735+0000 7f6e837fe640 1 -- 172.21.15.5:0/2146087364 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f6e9c026a30 con 0x564b8ca19370 2024-09-19T06:15:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.736+0000 7f6eac6b2740 10 client.18477 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.736+0000 7f6eac6b2740 10 client.18477 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:15:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.736+0000 7f6eac6b2740 20 client.18477 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.736+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564b8ca29040 0x564b8ca2b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.736+0000 7f6eac6b2740 4 client.18477 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.736+0000 7f6eac6b2740 1 -- 172.21.15.5:0/2146087364 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b8c981860 con 0x564b8ca29040 2024-09-19T06:15:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.736+0000 7f6ea3fff640 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564b8ca29040 0x564b8ca2b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.736+0000 7f6e837fe640 10 client.18477 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:15:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.736+0000 7f6ea3fff640 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564b8ca29040 0x564b8ca2b470 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=297c299a872ec403 server_cookie=6aa7ddf3bb1b7888 in_seq=0 out_seq=0 2024-09-19T06:15:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.737+0000 7f6e837fe640 1 -- 172.21.15.5:0/2146087364 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b8c981860 con 0x564b8ca29040 2024-09-19T06:15:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.737+0000 7f6e837fe640 10 client.18477 handle_command_reply: tid=0 2024-09-19T06:15:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.737+0000 7f6eac6b2740 1 client.18477 shutdown 2024-09-19T06:15:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.737+0000 7f6eac6b2740 1 -- 172.21.15.5:0/2146087364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 msgr2=0x564b8ca16800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.737+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 0x564b8ca16800 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c008700 tx=0x7f6e9c004a40 comp rx=0 tx=0).stop 2024-09-19T06:15:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.737+0000 7f6eac6b2740 1 -- 172.21.15.5:0/2146087364 shutdown_connections 2024-09-19T06:15:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.737+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564b8ca29040 0x564b8ca2b470 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.737+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca19370 0x564b8ca16800 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.737+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0fb40 0x564b8ca16070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.737+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/2146087364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0e5d0 0x564b8ca18dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.737+0000 7f6eac6b2740 1 -- 172.21.15.5:0/2146087364 >> 172.21.15.5:0/2146087364 conn(0x564b8c9d7310 msgr2=0x564b8ca10960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.737+0000 7f6e837fe640 0 client.18477 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:15:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.738+0000 7f6eac6b2740 1 -- 172.21.15.5:0/2146087364 shutdown_connections 2024-09-19T06:15:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.738+0000 7f6eac6b2740 1 -- 172.21.15.5:0/2146087364 wait complete. 2024-09-19T06:15:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.738+0000 7f6eac6b2740 20 client.18477 trim_cache size 0 max 16384 2024-09-19T06:15:07.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6eac6b2740 1 Processor -- start 2024-09-19T06:15:07.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6eac6b2740 1 -- start start 2024-09-19T06:15:07.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6eac6b2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0e5d0 0x564b8ca112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6eac6b2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 0x564b8ca11800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6eac6b2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca19370 0x564b8ca14940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6eac6b2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41ada30 con 0x564b8ca0fb40 2024-09-19T06:15:07.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6eac6b2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41af630 con 0x564b8ca0e5d0 2024-09-19T06:15:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6eac6b2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41ae100 con 0x564b8ca19370 2024-09-19T06:15:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 0x564b8ca11800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 0x564b8ca11800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37146/0 (socket says 172.21.15.5:37146) 2024-09-19T06:15:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6ea37fe640 1 -- 172.21.15.5:0/1813919952 learned_addr learned my addr 172.21.15.5:0/1813919952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6ea3fff640 1 --2- 172.21.15.5:0/1813919952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0e5d0 0x564b8ca112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6ea37fe640 1 -- 172.21.15.5:0/1813919952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca19370 msgr2=0x564b8ca14940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/1813919952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca19370 0x564b8ca14940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6ea37fe640 1 -- 172.21.15.5:0/1813919952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0e5d0 msgr2=0x564b8ca112c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/1813919952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0e5d0 0x564b8ca112c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6ea37fe640 1 -- 172.21.15.5:0/1813919952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b8ca145c0 con 0x564b8ca0fb40 2024-09-19T06:15:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.751+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/1813919952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 0x564b8ca11800 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6e980cd220 tx=0x7f6e9800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d88a749d745d8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.752+0000 7f6e89ffb640 1 -- 172.21.15.5:0/1813919952 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e980d3020 con 0x564b8ca0fb40 2024-09-19T06:15:07.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.752+0000 7f6e89ffb640 1 -- 172.21.15.5:0/1813919952 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e98009de0 con 0x564b8ca0fb40 2024-09-19T06:15:07.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.752+0000 7f6e89ffb640 1 -- 172.21.15.5:0/1813919952 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98020070 con 0x564b8ca0fb40 2024-09-19T06:15:07.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.752+0000 7f6eac6b2740 1 -- 172.21.15.5:0/1813919952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 msgr2=0x564b8ca11800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:07.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.752+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/1813919952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 0x564b8ca11800 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6e980cd220 tx=0x7f6e9800bf60 comp rx=0 tx=0).stop 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.752+0000 7f6eac6b2740 1 -- 172.21.15.5:0/1813919952 shutdown_connections 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.752+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/1813919952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca19370 0x564b8ca14940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.752+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/1813919952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 0x564b8ca11800 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.752+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/1813919952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca0e5d0 0x564b8ca112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.752+0000 7f6eac6b2740 1 -- 172.21.15.5:0/1813919952 >> 172.21.15.5:0/1813919952 conn(0x564b8c9d7310 msgr2=0x564b8ca0a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.752+0000 7f6eac6b2740 1 -- 172.21.15.5:0/1813919952 shutdown_connections 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.752+0000 7f6eac6b2740 1 -- 172.21.15.5:0/1813919952 wait complete. 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.753+0000 7f6eac6b2740 1 -- start start 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.753+0000 7f6eac6b2740 1 Processor -- start 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.753+0000 7f6eac6b2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0e5d0 0x564b8ca21730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.753+0000 7f6eac6b2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 0x564b8ca21c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.753+0000 7f6eac6b2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca19370 0x564b8ca1a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.753+0000 7f6eac6b2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41aef60 con 0x564b8ca0fb40 2024-09-19T06:15:07.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.753+0000 7f6eac6b2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41ae830 con 0x564b8ca19370 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.753+0000 7f6eac6b2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41aba30 con 0x564b8ca0e5d0 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.753+0000 7f6eac6b2740 10 client.0 _ll_register_callbacks cb 0x7f6e9c008570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.753+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 0x564b8ca21c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 0x564b8ca21c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37162/0 (socket says 172.21.15.5:37162) 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6ea37fe640 1 -- 172.21.15.5:0/3173383735 learned_addr learned my addr 172.21.15.5:0/3173383735 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6ea3fff640 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0e5d0 0x564b8ca21730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6ea8876640 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca19370 0x564b8ca1a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6ea37fe640 1 -- 172.21.15.5:0/3173383735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0e5d0 msgr2=0x564b8ca21730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0e5d0 0x564b8ca21730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6ea37fe640 1 -- 172.21.15.5:0/3173383735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca19370 msgr2=0x564b8ca1a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca19370 0x564b8ca1a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6ea37fe640 1 -- 172.21.15.5:0/3173383735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b8ca1ac70 con 0x564b8ca0fb40 2024-09-19T06:15:07.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6ea37fe640 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 0x564b8ca21c70 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6e980058c0 tx=0x7f6e980cfd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e23428c2167f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6e837fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6e837fe640 1 -- 172.21.15.5:0/3173383735 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e980b9060 con 0x564b8ca0fb40 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6e837fe640 1 -- 172.21.15.5:0/3173383735 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8594+0+0 (secure 0 0 0) 0x7f6e98098370 con 0x564b8ca0fb40 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6eac6b2740 1 -- 172.21.15.5:0/3173383735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e98009de0 con 0x564b8ca0fb40 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6e837fe640 1 -- 172.21.15.5:0/3173383735 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6e98064070 con 0x564b8ca0fb40 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6eac6b2740 10 client.18483 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.754+0000 7f6eac6b2740 1 -- 172.21.15.5:0/3173383735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564b8ca210d0 con 0x564b8ca0fb40 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.755+0000 7f6e837fe640 1 -- 172.21.15.5:0/3173383735 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e98009de0 con 0x564b8ca0fb40 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.755+0000 7f6e837fe640 1 -- 172.21.15.5:0/3173383735 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f6e9809bf60 con 0x564b8ca0fb40 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.755+0000 7f6eac6b2740 10 client.18483 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.755+0000 7f6eac6b2740 10 client.18483 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.755+0000 7f6eac6b2740 20 client.18483 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:07.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.755+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564b8ca93d10 0x564b8ca96100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:07.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.755+0000 7f6eac6b2740 4 client.18483 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:15:07.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.755+0000 7f6eac6b2740 1 -- 172.21.15.5:0/3173383735 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b8c97bd70 con 0x564b8ca93d10 2024-09-19T06:15:07.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.755+0000 7f6ea3fff640 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564b8ca93d10 0x564b8ca96100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:07.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.755+0000 7f6ea3fff640 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564b8ca93d10 0x564b8ca96100 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6da8dc901baf2a5a server_cookie=ecdd5ffdc0203ce0 in_seq=0 out_seq=0 2024-09-19T06:15:07.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.755+0000 7f6e837fe640 10 client.18483 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:15:07.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6e837fe640 1 -- 172.21.15.5:0/3173383735 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b8c97bd70 con 0x564b8ca93d10 2024-09-19T06:15:07.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6e837fe640 10 client.18483 handle_command_reply: tid=0 2024-09-19T06:15:07.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6eac6b2740 1 client.18483 shutdown 2024-09-19T06:15:07.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6eac6b2740 1 -- 172.21.15.5:0/3173383735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 msgr2=0x564b8ca21c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:07.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 0x564b8ca21c70 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6e980058c0 tx=0x7f6e980cfd70 comp rx=0 tx=0).stop 2024-09-19T06:15:07.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6eac6b2740 1 -- 172.21.15.5:0/3173383735 shutdown_connections 2024-09-19T06:15:07.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x564b8ca93d10 0x564b8ca96100 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b8ca19370 0x564b8ca1a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b8ca0fb40 0x564b8ca21c70 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6eac6b2740 1 --2- 172.21.15.5:0/3173383735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b8ca0e5d0 0x564b8ca21730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6eac6b2740 1 -- 172.21.15.5:0/3173383735 >> 172.21.15.5:0/3173383735 conn(0x564b8c9d7310 msgr2=0x564b8ca14900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:07.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6e837fe640 0 client.18483 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:15:07.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6eac6b2740 1 -- 172.21.15.5:0/3173383735 shutdown_connections 2024-09-19T06:15:07.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6eac6b2740 1 -- 172.21.15.5:0/3173383735 wait complete. 2024-09-19T06:15:07.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.756+0000 7f6eac6b2740 20 client.18483 trim_cache size 0 max 16384 2024-09-19T06:15:07.763 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:15:07.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.757+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/1252690079 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6e8c0a33b0 msgr2=0x7f6e8c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:07.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.757+0000 7f6eaa2d9640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6e8c0a33b0 0x7f6e8c0a5870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c008de0 tx=0x7f6e9c0087c0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.757+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/1252690079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ea4101420 msgr2=0x7f6ea4198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:07.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.757+0000 7f6eaa2d9640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ea4101420 0x7f6ea4198910 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800d950 tx=0x7f6e9800de10 comp rx=0 tx=0).stop 2024-09-19T06:15:07.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.757+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/1252690079 shutdown_connections 2024-09-19T06:15:07.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.757+0000 7f6eaa2d9640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6e8c0a33b0 0x7f6e8c0a5870 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.757+0000 7f6eaa2d9640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ea4106a80 0x7f6ea419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.757+0000 7f6eaa2d9640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ea4101420 0x7f6ea4198910 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.757+0000 7f6eaa2d9640 1 --2- 172.21.15.5:0/1252690079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ea4100a80 0x7f6ea41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:07.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.757+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/1252690079 >> 172.21.15.5:0/1252690079 conn(0x7f6ea40fc7a0 msgr2=0x7f6ea406ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:07.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.757+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/1252690079 shutdown_connections 2024-09-19T06:15:07.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:07.757+0000 7f6eaa2d9640 1 -- 172.21.15.5:0/1252690079 wait complete. 2024-09-19T06:15:07.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273286 2.22 MB/sec execute 513 sec latency 2118.051 ms 2024-09-19T06:15:08.429 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:15:08.686 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:08.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:08 smithi005 ceph-mon[30955]: pgmap v368: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-09-19T06:15:08.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:08 smithi005 ceph-mon[30955]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T06:15:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:08 smithi157 ceph-mon[35346]: pgmap v368: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-09-19T06:15:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:08 smithi157 ceph-mon[35346]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T06:15:08.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273286 2.22 MB/sec execute 514 sec latency 3118.160 ms 2024-09-19T06:15:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:08 smithi049 ceph-mon[34640]: pgmap v368: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-09-19T06:15:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:08 smithi049 ceph-mon[34640]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T06:15:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.096+0000 7fbf29ad0640 1 -- 172.21.15.5:0/1992598378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf241009a0 msgr2=0x7fbf24100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.096+0000 7fbf29ad0640 1 --2- 172.21.15.5:0/1992598378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf241009a0 0x7fbf24100e00 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009930 tx=0x7fbf1802a420 comp rx=0 tx=0).stop 2024-09-19T06:15:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.096+0000 7fbf29ad0640 1 -- 172.21.15.5:0/1992598378 shutdown_connections 2024-09-19T06:15:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.096+0000 7fbf29ad0640 1 --2- 172.21.15.5:0/1992598378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf24101340 0x7fbf2410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.096+0000 7fbf29ad0640 1 --2- 172.21.15.5:0/1992598378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf241009a0 0x7fbf24100e00 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.096+0000 7fbf29ad0640 1 --2- 172.21.15.5:0/1992598378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf241069b0 0x7fbf24106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.096+0000 7fbf29ad0640 1 -- 172.21.15.5:0/1992598378 >> 172.21.15.5:0/1992598378 conn(0x7fbf240779a0 msgr2=0x7fbf240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.096+0000 7fbf29ad0640 1 -- 172.21.15.5:0/1992598378 shutdown_connections 2024-09-19T06:15:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.096+0000 7fbf29ad0640 1 -- 172.21.15.5:0/1992598378 wait complete. 2024-09-19T06:15:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.096+0000 7fbf29ad0640 1 Processor -- start 2024-09-19T06:15:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf29ad0640 1 -- start start 2024-09-19T06:15:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf29ad0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf241009a0 0x7fbf24199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf29ad0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24101340 0x7fbf241996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf29ad0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf241069b0 0x7fbf241a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf29ad0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf24110ca0 con 0x7fbf24101340 2024-09-19T06:15:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf29ad0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbf24110b20 con 0x7fbf241009a0 2024-09-19T06:15:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf29ad0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf24110e20 con 0x7fbf241069b0 2024-09-19T06:15:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf22ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24101340 0x7fbf241996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf22ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24101340 0x7fbf241996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37176/0 (socket says 172.21.15.5:37176) 2024-09-19T06:15:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf22ffd640 1 -- 172.21.15.5:0/2319090741 learned_addr learned my addr 172.21.15.5:0/2319090741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf237fe640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf241009a0 0x7fbf24199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf23fff640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf241069b0 0x7fbf241a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf22ffd640 1 -- 172.21.15.5:0/2319090741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf241069b0 msgr2=0x7fbf241a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf22ffd640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf241069b0 0x7fbf241a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf22ffd640 1 -- 172.21.15.5:0/2319090741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf241009a0 msgr2=0x7fbf24199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf22ffd640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf241009a0 0x7fbf24199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf22ffd640 1 -- 172.21.15.5:0/2319090741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf241a4750 con 0x7fbf24101340 2024-09-19T06:15:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.097+0000 7fbf22ffd640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24101340 0x7fbf241996d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fbf18004770 tx=0x7fbf180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.098+0000 7fbf20ff9640 1 -- 172.21.15.5:0/2319090741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1803f070 con 0x7fbf24101340 2024-09-19T06:15:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.098+0000 7fbf20ff9640 1 -- 172.21.15.5:0/2319090741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf1803b460 con 0x7fbf24101340 2024-09-19T06:15:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.098+0000 7fbf29ad0640 1 -- 172.21.15.5:0/2319090741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf241a4920 con 0x7fbf24101340 2024-09-19T06:15:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.098+0000 7fbf20ff9640 1 -- 172.21.15.5:0/2319090741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18035070 con 0x7fbf24101340 2024-09-19T06:15:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.098+0000 7fbf29ad0640 1 -- 172.21.15.5:0/2319090741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf241a4d80 con 0x7fbf24101340 2024-09-19T06:15:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.100+0000 7fbf20ff9640 1 -- 172.21.15.5:0/2319090741 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbf1803b610 con 0x7fbf24101340 2024-09-19T06:15:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.100+0000 7fbf2c86c740 1 Processor -- start 2024-09-19T06:15:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.100+0000 7fbf2c86c740 1 -- start start 2024-09-19T06:15:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.100+0000 7fbf2c86c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 0x563277f0fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.100+0000 7fbf2c86c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f1a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.100+0000 7fbf2c86c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f1d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.100+0000 7fbf2c86c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563277e742f0 con 0x563277f0faf0 2024-09-19T06:15:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.100+0000 7fbf2c86c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563277e1c270 con 0x563277f10410 2024-09-19T06:15:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.100+0000 7fbf2c86c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563277e702b0 con 0x563277f1acb0 2024-09-19T06:15:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.100+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 0x563277f0fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.100+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 0x563277f0fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37186/0 (socket says 172.21.15.5:37186) 2024-09-19T06:15:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.100+0000 7fbf23fff640 1 -- 172.21.15.5:0/3282421512 learned_addr learned my addr 172.21.15.5:0/3282421512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf20ff9640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbf0c0a33b0 0x7fbf0c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf22ffd640 1 --2- 172.21.15.5:0/3282421512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f1d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf237fe640 1 --2- 172.21.15.5:0/3282421512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f1a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf23fff640 1 -- 172.21.15.5:0/3282421512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 msgr2=0x563277f1d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf23fff640 1 --2- 172.21.15.5:0/3282421512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f1d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf23fff640 1 -- 172.21.15.5:0/3282421512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 msgr2=0x563277f1a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf23fff640 1 --2- 172.21.15.5:0/3282421512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f1a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf23fff640 1 -- 172.21.15.5:0/3282421512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563277e1b340 con 0x563277f0faf0 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf237fe640 1 --2- 172.21.15.5:0/3282421512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f1a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf20ff9640 1 -- 172.21.15.5:0/2319090741 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fbf180f0080 con 0x7fbf24101340 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf23fff640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbf0c0a33b0 0x7fbf0c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf23fff640 1 --2- 172.21.15.5:0/3282421512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 0x563277f0fed0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00b880 tx=0x7fbf1c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352c932aae324546 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf09ffb640 1 -- 172.21.15.5:0/3282421512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c024070 con 0x563277f0faf0 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf09ffb640 1 -- 172.21.15.5:0/3282421512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf1c010af0 con 0x563277f0faf0 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.101+0000 7fbf09ffb640 1 -- 172.21.15.5:0/3282421512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c01c810 con 0x563277f0faf0 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.102+0000 7fbf2c86c740 1 -- 172.21.15.5:0/3282421512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 msgr2=0x563277f0fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.102+0000 7fbf23fff640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbf0c0a33b0 0x7fbf0c0a5870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c021000 tx=0x7fbf1c01e000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.102+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/3282421512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 0x563277f0fed0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00b880 tx=0x7fbf1c00bd40 comp rx=0 tx=0).stop 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.102+0000 7fbf2c86c740 1 -- 172.21.15.5:0/3282421512 shutdown_connections 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.102+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/3282421512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f1d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.102+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/3282421512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f1a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.102+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/3282421512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 0x563277f0fed0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.102+0000 7fbf2c86c740 1 -- 172.21.15.5:0/3282421512 >> 172.21.15.5:0/3282421512 conn(0x563277f0c060 msgr2=0x563277f0c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.102+0000 7fbf2c86c740 1 -- 172.21.15.5:0/3282421512 shutdown_connections 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.102+0000 7fbf2c86c740 1 -- 172.21.15.5:0/3282421512 wait complete. 2024-09-19T06:15:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.102+0000 7fbf2c86c740 1 -- start start 2024-09-19T06:15:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf2c86c740 1 Processor -- start 2024-09-19T06:15:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf2c86c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f0faf0 0x563277f17bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf2c86c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f18110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf2c86c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f1acb0 0x563277f18650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf2c86c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563277f1d880 con 0x563277f1acb0 2024-09-19T06:15:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf2c86c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563277dfd290 con 0x563277f10410 2024-09-19T06:15:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf2c86c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563277f1da00 con 0x563277f0faf0 2024-09-19T06:15:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf2c86c740 10 client.0 _ll_register_callbacks cb 0x563277e2e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f1acb0 0x563277f18650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f1acb0 0x563277f18650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37202/0 (socket says 172.21.15.5:37202) 2024-09-19T06:15:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf23fff640 1 -- 172.21.15.5:0/4054131645 learned_addr learned my addr 172.21.15.5:0/4054131645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf237fe640 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f0faf0 0x563277f17bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf22ffd640 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f18110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf23fff640 1 -- 172.21.15.5:0/4054131645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f0faf0 msgr2=0x563277f17bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf23fff640 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f0faf0 0x563277f17bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf23fff640 1 -- 172.21.15.5:0/4054131645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 msgr2=0x563277f18110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf23fff640 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f18110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.103+0000 7fbf23fff640 1 -- 172.21.15.5:0/4054131645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563277b87210 con 0x563277f1acb0 2024-09-19T06:15:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf23fff640 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f1acb0 0x563277f18650 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00b320 tx=0x7fbf1c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25dbaa6ed9b67992 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf02ffd640 1 -- 172.21.15.5:0/4054131645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c024040 con 0x563277f1acb0 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf02ffd640 1 -- 172.21.15.5:0/4054131645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fbf1c01cb30 con 0x563277f1acb0 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf2c86c740 1 -- 172.21.15.5:0/4054131645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf1c010af0 con 0x563277f1acb0 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf02ffd640 1 -- 172.21.15.5:0/4054131645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbf1c01cd30 con 0x563277f1acb0 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf2c86c740 10 client.18495 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf2c86c740 1 -- 172.21.15.5:0/4054131645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563277c53620 con 0x563277f1acb0 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf02ffd640 1 -- 172.21.15.5:0/4054131645 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf1c010af0 con 0x563277f1acb0 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf02ffd640 1 -- 172.21.15.5:0/4054131645 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fbf1c005290 con 0x563277f1acb0 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf2c86c740 10 client.18495 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf2c86c740 10 client.18495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf2c86c740 20 client.18495 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563277f2abe0 0x563277f2cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf2c86c740 4 client.18495 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.104+0000 7fbf2c86c740 1 -- 172.21.15.5:0/4054131645 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563277e81a20 con 0x563277f2abe0 2024-09-19T06:15:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.105+0000 7fbf237fe640 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563277f2abe0 0x563277f2cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.105+0000 7fbf237fe640 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563277f2abe0 0x563277f2cfd0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b35741d17e0c1b8 server_cookie=6e7404e2cc0b4c29 in_seq=0 out_seq=0 2024-09-19T06:15:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.105+0000 7fbf02ffd640 10 client.18495 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.106+0000 7fbf02ffd640 1 -- 172.21.15.5:0/4054131645 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563277e81a20 con 0x563277f2abe0 2024-09-19T06:15:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.106+0000 7fbf02ffd640 10 client.18495 handle_command_reply: tid=0 2024-09-19T06:15:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.106+0000 7fbf2c86c740 1 client.18495 shutdown 2024-09-19T06:15:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.106+0000 7fbf2c86c740 1 -- 172.21.15.5:0/4054131645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f1acb0 msgr2=0x563277f18650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.107+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f1acb0 0x563277f18650 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00b320 tx=0x7fbf1c005c70 comp rx=0 tx=0).stop 2024-09-19T06:15:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.107+0000 7fbf2c86c740 1 -- 172.21.15.5:0/4054131645 shutdown_connections 2024-09-19T06:15:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.107+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563277f2abe0 0x563277f2cfd0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.107+0000 7fbf02ffd640 0 client.18495 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.107+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f1acb0 0x563277f18650 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.107+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f18110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.107+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/4054131645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f0faf0 0x563277f17bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.107+0000 7fbf2c86c740 1 -- 172.21.15.5:0/4054131645 >> 172.21.15.5:0/4054131645 conn(0x563277f0c060 msgr2=0x563277f11dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.107+0000 7fbf2c86c740 1 -- 172.21.15.5:0/4054131645 shutdown_connections 2024-09-19T06:15:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.107+0000 7fbf2c86c740 1 -- 172.21.15.5:0/4054131645 wait complete. 2024-09-19T06:15:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.107+0000 7fbf2c86c740 20 client.18495 trim_cache size 0 max 16384 2024-09-19T06:15:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.121+0000 7fbf2c86c740 1 Processor -- start 2024-09-19T06:15:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.121+0000 7fbf2c86c740 1 -- start start 2024-09-19T06:15:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.121+0000 7fbf2c86c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 0x563277f19f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.121+0000 7fbf2c86c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f18480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.121+0000 7fbf2c86c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.121+0000 7fbf2c86c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241ad7c0 con 0x563277f0faf0 2024-09-19T06:15:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.121+0000 7fbf2c86c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241af3c0 con 0x563277f10410 2024-09-19T06:15:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.121+0000 7fbf2c86c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241ade90 con 0x563277f1acb0 2024-09-19T06:15:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.121+0000 7fbf237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 0x563277f19f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.121+0000 7fbf237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 0x563277f19f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37218/0 (socket says 172.21.15.5:37218) 2024-09-19T06:15:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.121+0000 7fbf237fe640 1 -- 172.21.15.5:0/3857873824 learned_addr learned my addr 172.21.15.5:0/3857873824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf23fff640 1 --2- 172.21.15.5:0/3857873824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf237fe640 1 -- 172.21.15.5:0/3857873824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 msgr2=0x563277f327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf237fe640 1 --2- 172.21.15.5:0/3857873824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f327f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf237fe640 1 -- 172.21.15.5:0/3857873824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 msgr2=0x563277f18480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf22ffd640 1 --2- 172.21.15.5:0/3857873824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f18480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf237fe640 1 --2- 172.21.15.5:0/3857873824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f18480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf237fe640 1 -- 172.21.15.5:0/3857873824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563277f0d110 con 0x563277f0faf0 2024-09-19T06:15:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf23fff640 1 --2- 172.21.15.5:0/3857873824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf22ffd640 1 --2- 172.21.15.5:0/3857873824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f18480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:15:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf237fe640 1 --2- 172.21.15.5:0/3857873824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 0x563277f19f00 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009fd0 tx=0x7fbf14004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be48c3a23d3edae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf097fa640 1 -- 172.21.15.5:0/3857873824 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14012800 con 0x563277f0faf0 2024-09-19T06:15:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf097fa640 1 -- 172.21.15.5:0/3857873824 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf14012e50 con 0x563277f0faf0 2024-09-19T06:15:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.122+0000 7fbf097fa640 1 -- 172.21.15.5:0/3857873824 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1401e430 con 0x563277f0faf0 2024-09-19T06:15:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.123+0000 7fbf2c86c740 1 -- 172.21.15.5:0/3857873824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 msgr2=0x563277f19f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.123+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/3857873824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 0x563277f19f00 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009fd0 tx=0x7fbf14004410 comp rx=0 tx=0).stop 2024-09-19T06:15:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.123+0000 7fbf2c86c740 1 -- 172.21.15.5:0/3857873824 shutdown_connections 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.123+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/3857873824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.123+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/3857873824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f10410 0x563277f18480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.123+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/3857873824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f0faf0 0x563277f19f00 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.123+0000 7fbf2c86c740 1 -- 172.21.15.5:0/3857873824 >> 172.21.15.5:0/3857873824 conn(0x563277f0c060 msgr2=0x563277f0b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.123+0000 7fbf2c86c740 1 -- 172.21.15.5:0/3857873824 shutdown_connections 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.123+0000 7fbf2c86c740 1 -- 172.21.15.5:0/3857873824 wait complete. 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.123+0000 7fbf2c86c740 1 -- start start 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.123+0000 7fbf2c86c740 1 Processor -- start 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.123+0000 7fbf2c86c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f0faf0 0x563277f23300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf2c86c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f10410 0x563277f1e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf2c86c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f1eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf2c86c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241aecf0 con 0x563277f10410 2024-09-19T06:15:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf2c86c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241ae5c0 con 0x563277f0faf0 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf2c86c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241ab7c0 con 0x563277f1acb0 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf2c86c740 10 client.0 _ll_register_callbacks cb 0x7fbf1c022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf22ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f10410 0x563277f1e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf22ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f10410 0x563277f1e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37226/0 (socket says 172.21.15.5:37226) 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf22ffd640 1 -- 172.21.15.5:0/1930371825 learned_addr learned my addr 172.21.15.5:0/1930371825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf237fe640 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f0faf0 0x563277f23300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf23fff640 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f1eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf22ffd640 1 -- 172.21.15.5:0/1930371825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 msgr2=0x563277f1eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf22ffd640 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f1eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf22ffd640 1 -- 172.21.15.5:0/1930371825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f0faf0 msgr2=0x563277f23300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf22ffd640 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f0faf0 0x563277f23300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf22ffd640 1 -- 172.21.15.5:0/1930371825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563277f072d0 con 0x563277f10410 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf237fe640 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f0faf0 0x563277f23300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf22ffd640 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f10410 0x563277f1e620 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbf180bc320 tx=0x7fbf1808b080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3180a074214a739b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf02ffd640 1 -- 172.21.15.5:0/1930371825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18048070 con 0x563277f10410 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf02ffd640 1 -- 172.21.15.5:0/1930371825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fbf180ed9d0 con 0x563277f10410 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf2c86c740 1 -- 172.21.15.5:0/1930371825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf14012e50 con 0x563277f10410 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf02ffd640 1 -- 172.21.15.5:0/1930371825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbf180edbd0 con 0x563277f10410 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf2c86c740 10 client.18501 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.124+0000 7fbf2c86c740 1 -- 172.21.15.5:0/1930371825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563277f0f920 con 0x563277f10410 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.125+0000 7fbf02ffd640 1 -- 172.21.15.5:0/1930371825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf14012e50 con 0x563277f10410 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.125+0000 7fbf02ffd640 1 -- 172.21.15.5:0/1930371825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fbf1804b060 con 0x563277f10410 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.125+0000 7fbf2c86c740 10 client.18501 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.125+0000 7fbf2c86c740 10 client.18501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.125+0000 7fbf2c86c740 20 client.18501 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.125+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563277f2abe0 0x563277f97b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.125+0000 7fbf2c86c740 4 client.18501 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:15:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.125+0000 7fbf2c86c740 1 -- 172.21.15.5:0/1930371825 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563277f0cd10 con 0x563277f2abe0 2024-09-19T06:15:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.125+0000 7fbf237fe640 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563277f2abe0 0x563277f97b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.126+0000 7fbf237fe640 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563277f2abe0 0x563277f97b60 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b940e2dfc40239 server_cookie=37265b3b11c64bec in_seq=0 out_seq=0 2024-09-19T06:15:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.126+0000 7fbf02ffd640 10 client.18501 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf02ffd640 1 -- 172.21.15.5:0/1930371825 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563277f0cd10 con 0x563277f2abe0 2024-09-19T06:15:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf02ffd640 10 client.18501 handle_command_reply: tid=0 2024-09-19T06:15:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf2c86c740 1 client.18501 shutdown 2024-09-19T06:15:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf2c86c740 1 -- 172.21.15.5:0/1930371825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f10410 msgr2=0x563277f1e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f10410 0x563277f1e620 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbf180bc320 tx=0x7fbf1808b080 comp rx=0 tx=0).stop 2024-09-19T06:15:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf2c86c740 1 -- 172.21.15.5:0/1930371825 shutdown_connections 2024-09-19T06:15:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563277f2abe0 0x563277f97b60 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563277f1acb0 0x563277f1eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563277f10410 0x563277f1e620 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf2c86c740 1 --2- 172.21.15.5:0/1930371825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563277f0faf0 0x563277f23300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf02ffd640 0 client.18501 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf2c86c740 1 -- 172.21.15.5:0/1930371825 >> 172.21.15.5:0/1930371825 conn(0x563277f0c060 msgr2=0x563277f25b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.150+0000 7fbf2c86c740 1 -- 172.21.15.5:0/1930371825 shutdown_connections 2024-09-19T06:15:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf2c86c740 1 -- 172.21.15.5:0/1930371825 wait complete. 2024-09-19T06:15:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf2c86c740 20 client.18501 trim_cache size 0 max 16384 2024-09-19T06:15:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:15:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:15:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:15:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:15:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf29ad0640 1 -- 172.21.15.5:0/2319090741 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbf0c0a33b0 msgr2=0x7fbf0c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf29ad0640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbf0c0a33b0 0x7fbf0c0a5870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c021000 tx=0x7fbf1c01e000 comp rx=0 tx=0).stop 2024-09-19T06:15:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf29ad0640 1 -- 172.21.15.5:0/2319090741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24101340 msgr2=0x7fbf241996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf29ad0640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24101340 0x7fbf241996d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fbf18004770 tx=0x7fbf180042f0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf29ad0640 1 -- 172.21.15.5:0/2319090741 shutdown_connections 2024-09-19T06:15:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf29ad0640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbf0c0a33b0 0x7fbf0c0a5870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf29ad0640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf241069b0 0x7fbf241a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf29ad0640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24101340 0x7fbf241996d0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf29ad0640 1 --2- 172.21.15.5:0/2319090741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf241009a0 0x7fbf24199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf29ad0640 1 -- 172.21.15.5:0/2319090741 >> 172.21.15.5:0/2319090741 conn(0x7fbf240779a0 msgr2=0x7fbf240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf29ad0640 1 -- 172.21.15.5:0/2319090741 shutdown_connections 2024-09-19T06:15:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:09.151+0000 7fbf29ad0640 1 -- 172.21.15.5:0/2319090741 wait complete. 2024-09-19T06:15:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.222+0000 7f2ccba92640 1 -- 172.21.15.5:0/202403627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cc4101280 msgr2=0x7f2cc410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.222+0000 7f2ccba92640 1 --2- 172.21.15.5:0/202403627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cc4101280 0x7f2cc410df40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009900 tx=0x7f2cb402a410 comp rx=0 tx=0).stop 2024-09-19T06:15:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.222+0000 7f2ccba92640 1 -- 172.21.15.5:0/202403627 shutdown_connections 2024-09-19T06:15:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.222+0000 7f2ccba92640 1 --2- 172.21.15.5:0/202403627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc410e480 0x7f2cc4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.222+0000 7f2ccba92640 1 --2- 172.21.15.5:0/202403627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cc4101280 0x7f2cc410df40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.222+0000 7f2ccba92640 1 --2- 172.21.15.5:0/202403627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc41008d0 0x7f2cc4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.222+0000 7f2ccba92640 1 -- 172.21.15.5:0/202403627 >> 172.21.15.5:0/202403627 conn(0x7f2cc40fc7a0 msgr2=0x7f2cc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.222+0000 7f2ccba92640 1 -- 172.21.15.5:0/202403627 shutdown_connections 2024-09-19T06:15:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.223+0000 7f2ccba92640 1 -- 172.21.15.5:0/202403627 wait complete. 2024-09-19T06:15:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.223+0000 7f2ccba92640 1 Processor -- start 2024-09-19T06:15:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.223+0000 7f2ccba92640 1 -- start start 2024-09-19T06:15:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.223+0000 7f2ccba92640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc41008d0 0x7f2cc419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.223+0000 7f2ccba92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cc4101280 0x7f2cc419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2ccba92640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc410e480 0x7f2cc419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2ccba92640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4114040 con 0x7f2cc4101280 2024-09-19T06:15:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2ccba92640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4113ec0 con 0x7f2cc410e480 2024-09-19T06:15:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2ccba92640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc41141c0 con 0x7f2cc41008d0 2024-09-19T06:15:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cc902d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cc4101280 0x7f2cc419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cc902d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cc4101280 0x7f2cc419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37246/0 (socket says 172.21.15.5:37246) 2024-09-19T06:15:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cc902d640 1 -- 172.21.15.5:0/2752145655 learned_addr learned my addr 172.21.15.5:0/2752145655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cc982e640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc41008d0 0x7f2cc419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cca02f640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc410e480 0x7f2cc419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cc902d640 1 -- 172.21.15.5:0/2752145655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc41008d0 msgr2=0x7f2cc419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cc902d640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc41008d0 0x7f2cc419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cc902d640 1 -- 172.21.15.5:0/2752145655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc410e480 msgr2=0x7f2cc419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cc902d640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc410e480 0x7f2cc419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cc902d640 1 -- 172.21.15.5:0/2752145655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc419f550 con 0x7f2cc4101280 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cc902d640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cc4101280 0x7f2cc419aae0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4005940 tx=0x7f2cb4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cbaffd640 1 -- 172.21.15.5:0/2752145655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb403f030 con 0x7f2cc4101280 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cbaffd640 1 -- 172.21.15.5:0/2752145655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cb4003aa0 con 0x7f2cc4101280 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2ccba92640 1 -- 172.21.15.5:0/2752145655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc419f7e0 con 0x7f2cc4101280 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.224+0000 7f2cbaffd640 1 -- 172.21.15.5:0/2752145655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4035070 con 0x7f2cc4101280 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.225+0000 7f2ccba92640 1 -- 172.21.15.5:0/2752145655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc41a7120 con 0x7f2cc4101280 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.227+0000 7f2ccde6b740 1 Processor -- start 2024-09-19T06:15:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2ccde6b740 1 -- start start 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2cbaffd640 1 -- 172.21.15.5:0/2752145655 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2cb4042030 con 0x7f2cc4101280 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2ccde6b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ebfb40 0x55eac6ebe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2ccde6b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ebea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2ccde6b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ec9370 0x55eac6ecb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2ccde6b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eac6e242f0 con 0x55eac6ebe5d0 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2ccde6b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eac6dcbcf0 con 0x55eac6ec9370 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2ccde6b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eac6e202b0 con 0x55eac6ebfb40 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2cc982e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ebea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2cc982e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ebea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37262/0 (socket says 172.21.15.5:37262) 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2cc982e640 1 -- 172.21.15.5:0/3046860324 learned_addr learned my addr 172.21.15.5:0/3046860324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2cca02f640 1 --2- 172.21.15.5:0/3046860324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ebfb40 0x55eac6ebe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2cc902d640 1 --2- 172.21.15.5:0/3046860324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ec9370 0x55eac6ecb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2cc982e640 1 -- 172.21.15.5:0/3046860324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ebfb40 msgr2=0x55eac6ebe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2cc982e640 1 --2- 172.21.15.5:0/3046860324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ebfb40 0x55eac6ebe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2cc982e640 1 -- 172.21.15.5:0/3046860324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ec9370 msgr2=0x55eac6ecb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2cc982e640 1 --2- 172.21.15.5:0/3046860324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ec9370 0x55eac6ecb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2cc982e640 1 -- 172.21.15.5:0/3046860324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eac6db03a0 con 0x55eac6ebe5d0 2024-09-19T06:15:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.228+0000 7f2cc902d640 1 --2- 172.21.15.5:0/3046860324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ec9370 0x55eac6ecb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:15:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.229+0000 7f2cc982e640 1 --2- 172.21.15.5:0/3046860324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ebea10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000bcd0 tx=0x7f2cc00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad566681d170e692 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.229+0000 7f2cbaffd640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2cb00a33b0 0x7f2cb00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.229+0000 7f2cca02f640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2cb00a33b0 0x7f2cb00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.229+0000 7f2caffff640 1 -- 172.21.15.5:0/3046860324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0005220 con 0x55eac6ebe5d0 2024-09-19T06:15:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.229+0000 7f2cbaffd640 1 -- 172.21.15.5:0/2752145655 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f2cb40f04f0 con 0x7f2cc4101280 2024-09-19T06:15:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.229+0000 7f2caffff640 1 -- 172.21.15.5:0/3046860324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cc0004c50 con 0x55eac6ebe5d0 2024-09-19T06:15:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.229+0000 7f2caffff640 1 -- 172.21.15.5:0/3046860324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00116a0 con 0x55eac6ebe5d0 2024-09-19T06:15:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.230+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3046860324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 msgr2=0x55eac6ebea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.230+0000 7f2cca02f640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2cb00a33b0 0x7f2cb00a5870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2cc406a0f0 tx=0x7f2cbc0096e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.230+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/3046860324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ebea10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000bcd0 tx=0x7f2cc00107a0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.230+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3046860324 shutdown_connections 2024-09-19T06:15:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.230+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/3046860324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ec9370 0x55eac6ecb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.230+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/3046860324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ebea10 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.230+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/3046860324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ebfb40 0x55eac6ebe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.230+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3046860324 >> 172.21.15.5:0/3046860324 conn(0x55eac6e87310 msgr2=0x55eac6ebac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.230+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3046860324 shutdown_connections 2024-09-19T06:15:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.230+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3046860324 wait complete. 2024-09-19T06:15:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.230+0000 7f2ccde6b740 1 -- start start 2024-09-19T06:15:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.230+0000 7f2ccde6b740 1 Processor -- start 2024-09-19T06:15:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2ccde6b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebe5d0 0x55eac6ec8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2ccde6b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ebfb40 0x55eac6ec5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2ccde6b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ec9370 0x55eac6ec65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2ccde6b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eac6ecc070 con 0x55eac6ec9370 2024-09-19T06:15:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2ccde6b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eac6ddf060 con 0x55eac6ebe5d0 2024-09-19T06:15:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2ccde6b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eac6ecc1f0 con 0x55eac6ebfb40 2024-09-19T06:15:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2ccde6b740 10 client.0 _ll_register_callbacks cb 0x55eac6b328c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cca02f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ec9370 0x55eac6ec65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cca02f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ec9370 0x55eac6ec65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37268/0 (socket says 172.21.15.5:37268) 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cca02f640 1 -- 172.21.15.5:0/3290948617 learned_addr learned my addr 172.21.15.5:0/3290948617 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cc982e640 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebe5d0 0x55eac6ec8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cc902d640 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ebfb40 0x55eac6ec5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cca02f640 1 -- 172.21.15.5:0/3290948617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ebfb40 msgr2=0x55eac6ec5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cca02f640 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ebfb40 0x55eac6ec5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cca02f640 1 -- 172.21.15.5:0/3290948617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebe5d0 msgr2=0x55eac6ec8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cca02f640 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebe5d0 0x55eac6ec8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cca02f640 1 -- 172.21.15.5:0/3290948617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eac6b953c0 con 0x55eac6ec9370 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cca02f640 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ec9370 0x55eac6ec65e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc010880 tx=0x7f2cbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448c10037cafb86c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cad7fa640 1 -- 172.21.15.5:0/3290948617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc00eae0 con 0x55eac6ec9370 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2cad7fa640 1 -- 172.21.15.5:0/3290948617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f2cbc00ec80 con 0x55eac6ec9370 2024-09-19T06:15:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.231+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3290948617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc0004c50 con 0x55eac6ec9370 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.232+0000 7f2cad7fa640 1 -- 172.21.15.5:0/3290948617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2cbc027400 con 0x55eac6ec9370 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.232+0000 7f2ccde6b740 10 client.18513 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.232+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3290948617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eac6ed7ab0 con 0x55eac6ec9370 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.232+0000 7f2cad7fa640 1 -- 172.21.15.5:0/3290948617 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cc0004c50 con 0x55eac6ec9370 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.232+0000 7f2cad7fa640 1 -- 172.21.15.5:0/3290948617 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f2cbc00f040 con 0x55eac6ec9370 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.232+0000 7f2ccde6b740 10 client.18513 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.232+0000 7f2ccde6b740 10 client.18513 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.232+0000 7f2ccde6b740 20 client.18513 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.232+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55eac6ed8f70 0x55eac6edb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.232+0000 7f2ccde6b740 4 client.18513 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.232+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3290948617 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eac6e31860 con 0x55eac6ed8f70 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.232+0000 7f2cc982e640 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55eac6ed8f70 0x55eac6edb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.233+0000 7f2cc982e640 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55eac6ed8f70 0x55eac6edb3c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b3e6636ae57b43a7 server_cookie=34d0729f6ee97a14 in_seq=0 out_seq=0 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.233+0000 7f2cad7fa640 10 client.18513 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.233+0000 7f2cad7fa640 1 -- 172.21.15.5:0/3290948617 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eac6e31860 con 0x55eac6ed8f70 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.233+0000 7f2cad7fa640 10 client.18513 handle_command_reply: tid=0 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.233+0000 7f2ccde6b740 1 client.18513 shutdown 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.234+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3290948617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ec9370 msgr2=0x55eac6ec65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.234+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ec9370 0x55eac6ec65e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc010880 tx=0x7f2cbc004a40 comp rx=0 tx=0).stop 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.234+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3290948617 shutdown_connections 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.234+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55eac6ed8f70 0x55eac6edb3c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.234+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ec9370 0x55eac6ec65e0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.234+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ebfb40 0x55eac6ec5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.234+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/3290948617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebe5d0 0x55eac6ec8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.234+0000 7f2cad7fa640 0 client.18513 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:15:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.234+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3290948617 >> 172.21.15.5:0/3290948617 conn(0x55eac6e87310 msgr2=0x55eac6ec0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.234+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3290948617 shutdown_connections 2024-09-19T06:15:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.234+0000 7f2ccde6b740 1 -- 172.21.15.5:0/3290948617 wait complete. 2024-09-19T06:15:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.234+0000 7f2ccde6b740 20 client.18513 trim_cache size 0 max 16384 2024-09-19T06:15:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2ccde6b740 1 Processor -- start 2024-09-19T06:15:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2ccde6b740 1 -- start start 2024-09-19T06:15:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2ccde6b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ec58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2ccde6b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebfb40 0x55eac6ec7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2ccde6b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ec9370 0x55eac6ec1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2ccde6b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc41afa40 con 0x55eac6ebe5d0 2024-09-19T06:15:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2ccde6b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc41b1640 con 0x55eac6ebfb40 2024-09-19T06:15:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2ccde6b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc41b0110 con 0x55eac6ec9370 2024-09-19T06:15:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2cc982e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ec58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2cc982e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ec58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37278/0 (socket says 172.21.15.5:37278) 2024-09-19T06:15:09.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2cc982e640 1 -- 172.21.15.5:0/328522901 learned_addr learned my addr 172.21.15.5:0/328522901 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:09.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2cc902d640 1 --2- 172.21.15.5:0/328522901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebfb40 0x55eac6ec7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2cca02f640 1 --2- 172.21.15.5:0/328522901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ec9370 0x55eac6ec1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2cc982e640 1 -- 172.21.15.5:0/328522901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ec9370 msgr2=0x55eac6ec1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2cc982e640 1 --2- 172.21.15.5:0/328522901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ec9370 0x55eac6ec1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2cc982e640 1 -- 172.21.15.5:0/328522901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebfb40 msgr2=0x55eac6ec7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2cc982e640 1 --2- 172.21.15.5:0/328522901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebfb40 0x55eac6ec7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2cc982e640 1 -- 172.21.15.5:0/328522901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eac6ec4fa0 con 0x55eac6ebe5d0 2024-09-19T06:15:09.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.247+0000 7f2cc902d640 1 --2- 172.21.15.5:0/328522901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebfb40 0x55eac6ec7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2cc982e640 1 --2- 172.21.15.5:0/328522901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ec58b0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000ba60 tx=0x7f2cc0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce814223d638eeb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2caf7fe640 1 -- 172.21.15.5:0/328522901 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0029430 con 0x55eac6ebe5d0 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2caf7fe640 1 -- 172.21.15.5:0/328522901 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cc0029a80 con 0x55eac6ebe5d0 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2caf7fe640 1 -- 172.21.15.5:0/328522901 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0027b20 con 0x55eac6ebe5d0 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2ccde6b740 1 -- 172.21.15.5:0/328522901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 msgr2=0x55eac6ec58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/328522901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ec58b0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000ba60 tx=0x7f2cc0005740 comp rx=0 tx=0).stop 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2ccde6b740 1 -- 172.21.15.5:0/328522901 shutdown_connections 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/328522901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ec9370 0x55eac6ec1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/328522901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebfb40 0x55eac6ec7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/328522901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ec58b0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2ccde6b740 1 -- 172.21.15.5:0/328522901 >> 172.21.15.5:0/328522901 conn(0x55eac6e87310 msgr2=0x55eac6ec7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2ccde6b740 1 -- 172.21.15.5:0/328522901 shutdown_connections 2024-09-19T06:15:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.248+0000 7f2ccde6b740 1 -- 172.21.15.5:0/328522901 wait complete. 2024-09-19T06:15:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2ccde6b740 1 -- start start 2024-09-19T06:15:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2ccde6b740 1 Processor -- start 2024-09-19T06:15:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2ccde6b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ecd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2ccde6b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebfb40 0x55eac6ecd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2ccde6b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ec9370 0x55eac6ecdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2ccde6b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc41b0f70 con 0x55eac6ebe5d0 2024-09-19T06:15:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2ccde6b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc41b0840 con 0x55eac6ebfb40 2024-09-19T06:15:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2ccde6b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc41ada40 con 0x55eac6ec9370 2024-09-19T06:15:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2ccde6b740 10 client.0 _ll_register_callbacks cb 0x7f2cbc010530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2cc982e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ecd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2cc982e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ecd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37282/0 (socket says 172.21.15.5:37282) 2024-09-19T06:15:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2cc982e640 1 -- 172.21.15.5:0/420473319 learned_addr learned my addr 172.21.15.5:0/420473319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2cca02f640 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ec9370 0x55eac6ecdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.249+0000 7f2cc902d640 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebfb40 0x55eac6ecd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2cc982e640 1 -- 172.21.15.5:0/420473319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ec9370 msgr2=0x55eac6ecdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2cc982e640 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ec9370 0x55eac6ecdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2cc982e640 1 -- 172.21.15.5:0/420473319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebfb40 msgr2=0x55eac6ecd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2cc982e640 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebfb40 0x55eac6ecd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2cc982e640 1 -- 172.21.15.5:0/420473319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eac6ec90b0 con 0x55eac6ebe5d0 2024-09-19T06:15:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2cc902d640 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebfb40 0x55eac6ecd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2cc982e640 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ecd420 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005ca0 tx=0x7f2cc0029cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9432526116b5d698 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2cad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2cad7fa640 1 -- 172.21.15.5:0/420473319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0004020 con 0x55eac6ebe5d0 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2cad7fa640 1 -- 172.21.15.5:0/420473319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f2cc00041c0 con 0x55eac6ebe5d0 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2ccde6b740 1 -- 172.21.15.5:0/420473319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc0029a80 con 0x55eac6ebe5d0 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2cad7fa640 1 -- 172.21.15.5:0/420473319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2cc00043c0 con 0x55eac6ebe5d0 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2ccde6b740 10 client.18519 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2ccde6b740 1 -- 172.21.15.5:0/420473319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eac6f43f50 con 0x55eac6ebe5d0 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.250+0000 7f2cad7fa640 1 -- 172.21.15.5:0/420473319 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cc0029a80 con 0x55eac6ebe5d0 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.251+0000 7f2cad7fa640 1 -- 172.21.15.5:0/420473319 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f2cc00277a0 con 0x55eac6ebe5d0 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.251+0000 7f2ccde6b740 10 client.18519 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.251+0000 7f2ccde6b740 10 client.18519 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.251+0000 7f2ccde6b740 20 client.18519 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.251+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55eac6f44460 0x55eac6f46850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.251+0000 7f2ccde6b740 4 client.18519 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:15:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.251+0000 7f2ccde6b740 1 -- 172.21.15.5:0/420473319 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eac6e2bd70 con 0x55eac6f44460 2024-09-19T06:15:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.251+0000 7f2cc902d640 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55eac6f44460 0x55eac6f46850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.251+0000 7f2cad7fa640 10 client.18519 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:15:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.251+0000 7f2cc902d640 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55eac6f44460 0x55eac6f46850 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=99e5e9a423072075 server_cookie=5a589f57f7297ab5 in_seq=0 out_seq=0 2024-09-19T06:15:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.252+0000 7f2cad7fa640 1 -- 172.21.15.5:0/420473319 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eac6e2bd70 con 0x55eac6f44460 2024-09-19T06:15:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.252+0000 7f2cad7fa640 10 client.18519 handle_command_reply: tid=0 2024-09-19T06:15:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.252+0000 7f2ccde6b740 1 client.18519 shutdown 2024-09-19T06:15:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.252+0000 7f2ccde6b740 1 -- 172.21.15.5:0/420473319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 msgr2=0x55eac6ecd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.252+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ecd420 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005ca0 tx=0x7f2cc0029cc0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.252+0000 7f2ccde6b740 1 -- 172.21.15.5:0/420473319 shutdown_connections 2024-09-19T06:15:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.252+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55eac6f44460 0x55eac6f46850 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.252+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac6ec9370 0x55eac6ecdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.252+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac6ebfb40 0x55eac6ecd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.252+0000 7f2ccde6b740 1 --2- 172.21.15.5:0/420473319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac6ebe5d0 0x55eac6ecd420 secure :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005ca0 tx=0x7f2cc0029cc0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.252+0000 7f2ccde6b740 1 -- 172.21.15.5:0/420473319 >> 172.21.15.5:0/420473319 conn(0x55eac6e87310 msgr2=0x55eac6ec8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.252+0000 7f2cad7fa640 0 client.18519 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:15:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccde6b740 1 -- 172.21.15.5:0/420473319 shutdown_connections 2024-09-19T06:15:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccde6b740 1 -- 172.21.15.5:0/420473319 wait complete. 2024-09-19T06:15:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccde6b740 20 client.18519 trim_cache size 0 max 16384 2024-09-19T06:15:09.273 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:15:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccba92640 1 -- 172.21.15.5:0/2752145655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2cb00a33b0 msgr2=0x7f2cb00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccba92640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2cb00a33b0 0x7f2cb00a5870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2cc406a0f0 tx=0x7f2cbc0096e0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccba92640 1 -- 172.21.15.5:0/2752145655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cc4101280 msgr2=0x7f2cc419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccba92640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cc4101280 0x7f2cc419aae0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4005940 tx=0x7f2cb4004270 comp rx=0 tx=0).stop 2024-09-19T06:15:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccba92640 1 -- 172.21.15.5:0/2752145655 shutdown_connections 2024-09-19T06:15:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccba92640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc410e480 0x7f2cc419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccba92640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cc4101280 0x7f2cc419aae0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccba92640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2cb00a33b0 0x7f2cb00a5870 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccba92640 1 --2- 172.21.15.5:0/2752145655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc41008d0 0x7f2cc419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccba92640 1 -- 172.21.15.5:0/2752145655 >> 172.21.15.5:0/2752145655 conn(0x7f2cc40fc7a0 msgr2=0x7f2cc40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccba92640 1 -- 172.21.15.5:0/2752145655 shutdown_connections 2024-09-19T06:15:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:09.253+0000 7f2ccba92640 1 -- 172.21.15.5:0/2752145655 wait complete. 2024-09-19T06:15:09.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273286 2.21 MB/sec execute 515 sec latency 4118.278 ms 2024-09-19T06:15:09.998 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:15:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:10.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:10 smithi005 ceph-mon[30955]: pgmap v370: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-09-19T06:15:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.782+0000 7fa2cbe75640 1 -- 172.21.15.5:0/185983589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c4101de0 msgr2=0x7fa2c41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.782+0000 7fa2cbe75640 1 --2- 172.21.15.5:0/185983589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c4101de0 0x7fa2c41021c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009ca0 tx=0x7fa2b402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.782+0000 7fa2cbe75640 1 -- 172.21.15.5:0/185983589 shutdown_connections 2024-09-19T06:15:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.782+0000 7fa2cbe75640 1 --2- 172.21.15.5:0/185983589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c4106f10 0x7fa2c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.782+0000 7fa2cbe75640 1 --2- 172.21.15.5:0/185983589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c4102790 0x7fa2c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.782+0000 7fa2cbe75640 1 --2- 172.21.15.5:0/185983589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c4101de0 0x7fa2c41021c0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.782+0000 7fa2cbe75640 1 -- 172.21.15.5:0/185983589 >> 172.21.15.5:0/185983589 conn(0x7fa2c40fd8e0 msgr2=0x7fa2c40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.782+0000 7fa2cbe75640 1 -- 172.21.15.5:0/185983589 shutdown_connections 2024-09-19T06:15:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.782+0000 7fa2cbe75640 1 -- 172.21.15.5:0/185983589 wait complete. 2024-09-19T06:15:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2cbe75640 1 Processor -- start 2024-09-19T06:15:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2cbe75640 1 -- start start 2024-09-19T06:15:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2cbe75640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c4101de0 0x7fa2c4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2cbe75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c4102790 0x7fa2c4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2cbe75640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c4106f10 0x7fa2c419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2cbe75640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c4111de0 con 0x7fa2c4102790 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2cbe75640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c4111c60 con 0x7fa2c4101de0 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2cbe75640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c4111f60 con 0x7fa2c4106f10 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2c9410640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c4102790 0x7fa2c4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2ca412640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c4106f10 0x7fa2c419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2c9c11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c4101de0 0x7fa2c4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2ca412640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c4106f10 0x7fa2c419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45274/0 (socket says 172.21.15.5:45274) 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2c9410640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c4102790 0x7fa2c4198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37302/0 (socket says 172.21.15.5:37302) 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2ca412640 1 -- 172.21.15.5:0/4152649950 learned_addr learned my addr 172.21.15.5:0/4152649950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.783+0000 7fa2c9c11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c4101de0 0x7fa2c4198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60498/0 (socket says 172.21.15.5:60498) 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.784+0000 7fa2c9c11640 1 -- 172.21.15.5:0/4152649950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c4106f10 msgr2=0x7fa2c419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.784+0000 7fa2c9c11640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c4106f10 0x7fa2c419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.784+0000 7fa2c9c11640 1 -- 172.21.15.5:0/4152649950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c4102790 msgr2=0x7fa2c4198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.784+0000 7fa2c9c11640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c4102790 0x7fa2c4198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.784+0000 7fa2c9c11640 1 -- 172.21.15.5:0/4152649950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c419d2f0 con 0x7fa2c4101de0 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.784+0000 7fa2ca412640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c4106f10 0x7fa2c419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.784+0000 7fa2c9c11640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c4101de0 0x7fa2c4198340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009ca0 tx=0x7fa2b4031990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.784+0000 7fa2c2ffd640 1 -- 172.21.15.5:0/4152649950 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4003b70 con 0x7fa2c4101de0 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.784+0000 7fa2cbe75640 1 -- 172.21.15.5:0/4152649950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c419d580 con 0x7fa2c4101de0 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.784+0000 7fa2c2ffd640 1 -- 172.21.15.5:0/4152649950 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2b4003d10 con 0x7fa2c4101de0 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.784+0000 7fa2c2ffd640 1 -- 172.21.15.5:0/4152649950 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4030a60 con 0x7fa2c4101de0 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.784+0000 7fa2cbe75640 1 -- 172.21.15.5:0/4152649950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c41a4ec0 con 0x7fa2c4101de0 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.786+0000 7fa2c2ffd640 1 -- 172.21.15.5:0/4152649950 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa2b4030c00 con 0x7fa2c4101de0 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2ce24e740 1 Processor -- start 2024-09-19T06:15:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2ce24e740 1 -- start start 2024-09-19T06:15:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2ce24e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d1f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2ce24e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1f19e0 0x56348d1f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2ce24e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1fc780 0x56348d1feb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2ce24e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56348d1532f0 con 0x56348d1f19e0 2024-09-19T06:15:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2ce24e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56348d0facf0 con 0x56348d1f2f50 2024-09-19T06:15:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2ce24e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56348d14f2b0 con 0x56348d1fc780 2024-09-19T06:15:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2c9c11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1f19e0 0x56348d1f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2ca412640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d1f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2ca412640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d1f14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60504/0 (socket says 172.21.15.5:60504) 2024-09-19T06:15:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2c9410640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1fc780 0x56348d1feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2ca412640 1 -- 172.21.15.5:0/2011585039 learned_addr learned my addr 172.21.15.5:0/2011585039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.787+0000 7fa2c9c11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1f19e0 0x56348d1f1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37314/0 (socket says 172.21.15.5:37314) 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2c2ffd640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2ac0a33b0 0x7fa2ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2c9410640 1 -- 172.21.15.5:0/2011585039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 msgr2=0x56348d1f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2c9410640 1 --2- 172.21.15.5:0/2011585039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d1f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2c9410640 1 -- 172.21.15.5:0/2011585039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1f19e0 msgr2=0x56348d1f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2c9410640 1 --2- 172.21.15.5:0/2011585039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1f19e0 0x56348d1f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2c9410640 1 -- 172.21.15.5:0/2011585039 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56348d0df3a0 con 0x56348d1fc780 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2ca412640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2ac0a33b0 0x7fa2ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2c2ffd640 1 -- 172.21.15.5:0/4152649950 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fa2b40f0e80 con 0x7fa2c4101de0 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2c9410640 1 --2- 172.21.15.5:0/2011585039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1fc780 0x56348d1feb70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009bf0 tx=0x7fa2b8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=788c5a4c2ac90383 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2ca412640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2ac0a33b0 0x7fa2ac0a5870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009fd0 tx=0x7fa2bc0046c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2abfff640 1 -- 172.21.15.5:0/2011585039 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8004050 con 0x56348d1fc780 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2abfff640 1 -- 172.21.15.5:0/2011585039 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2b80054d0 con 0x56348d1fc780 2024-09-19T06:15:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.788+0000 7fa2abfff640 1 -- 172.21.15.5:0/2011585039 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8004cb0 con 0x56348d1fc780 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.789+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2011585039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1fc780 msgr2=0x56348d1feb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.789+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/2011585039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1fc780 0x56348d1feb70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009bf0 tx=0x7fa2b8011810 comp rx=0 tx=0).stop 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.789+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2011585039 shutdown_connections 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.789+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/2011585039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1fc780 0x56348d1feb70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.789+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/2011585039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1f19e0 0x56348d1f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.789+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/2011585039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d1f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.789+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2011585039 >> 172.21.15.5:0/2011585039 conn(0x56348d1b6310 msgr2=0x56348d1ee070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.789+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2011585039 shutdown_connections 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.789+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2011585039 wait complete. 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2ce24e740 1 -- start start 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2ce24e740 1 Processor -- start 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2ce24e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f19e0 0x56348d1fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2ce24e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1f2f50 0x56348d1f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2ce24e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 0x56348d1f9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2ce24e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56348d1ff480 con 0x56348d1fc780 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2ce24e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56348d10e060 con 0x56348d1f19e0 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2ce24e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56348d1ff600 con 0x56348d1f2f50 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2ca412640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 0x56348d1f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2ce24e740 10 client.0 _ll_register_callbacks cb 0x56348ce618c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2ca412640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 0x56348d1f9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37320/0 (socket says 172.21.15.5:37320) 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2ca412640 1 -- 172.21.15.5:0/2158420438 learned_addr learned my addr 172.21.15.5:0/2158420438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2c9c11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f19e0 0x56348d1fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.790+0000 7fa2c9410640 1 --2- 172.21.15.5:0/2158420438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1f2f50 0x56348d1f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2ca412640 1 -- 172.21.15.5:0/2158420438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1f2f50 msgr2=0x56348d1f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2ca412640 1 --2- 172.21.15.5:0/2158420438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1f2f50 0x56348d1f93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2ca412640 1 -- 172.21.15.5:0/2158420438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f19e0 msgr2=0x56348d1fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2ca412640 1 --2- 172.21.15.5:0/2158420438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f19e0 0x56348d1fc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2ca412640 1 -- 172.21.15.5:0/2158420438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56348cec43c0 con 0x56348d1fc780 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2ca412640 1 --2- 172.21.15.5:0/2158420438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 0x56348d1f9900 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00fe40 tx=0x7fa2bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa7fdd98b780724 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2a97fa640 1 -- 172.21.15.5:0/2158420438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc01ce60 con 0x56348d1fc780 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2158420438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2b80054d0 con 0x56348d1fc780 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2a97fa640 1 -- 172.21.15.5:0/2158420438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fa2bc01b7c0 con 0x56348d1fc780 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2a97fa640 1 -- 172.21.15.5:0/2158420438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa2bc01ba00 con 0x56348d1fc780 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2ce24e740 10 client.18531 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.791+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2158420438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56348d20aed0 con 0x56348d1fc780 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.792+0000 7fa2a97fa640 1 -- 172.21.15.5:0/2158420438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2b80054d0 con 0x56348d1fc780 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.792+0000 7fa2a97fa640 1 -- 172.21.15.5:0/2158420438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fa2bc030c00 con 0x56348d1fc780 2024-09-19T06:15:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.792+0000 7fa2ce24e740 10 client.18531 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:10.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.792+0000 7fa2ce24e740 10 client.18531 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:15:10.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.792+0000 7fa2ce24e740 20 client.18531 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:10.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.792+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/2158420438 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56348d20c390 0x56348d20e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.792+0000 7fa2ce24e740 4 client.18531 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:10.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.792+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2158420438 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56348d160860 con 0x56348d20c390 2024-09-19T06:15:10.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.792+0000 7fa2c9c11640 1 --2- 172.21.15.5:0/2158420438 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56348d20c390 0x56348d20e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.792+0000 7fa2c9c11640 1 --2- 172.21.15.5:0/2158420438 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56348d20c390 0x56348d20e850 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a0d6edf279f29f4 server_cookie=68920bd8f6492b3c in_seq=0 out_seq=0 2024-09-19T06:15:10.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.792+0000 7fa2a97fa640 10 client.18531 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:15:10.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.793+0000 7fa2a97fa640 1 -- 172.21.15.5:0/2158420438 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56348d160860 con 0x56348d20c390 2024-09-19T06:15:10.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.793+0000 7fa2a97fa640 10 client.18531 handle_command_reply: tid=0 2024-09-19T06:15:10.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.793+0000 7fa2ce24e740 1 client.18531 shutdown 2024-09-19T06:15:10.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.793+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2158420438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 msgr2=0x56348d1f9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.793+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/2158420438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 0x56348d1f9900 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00fe40 tx=0x7fa2bc004730 comp rx=0 tx=0).stop 2024-09-19T06:15:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.793+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2158420438 shutdown_connections 2024-09-19T06:15:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.793+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/2158420438 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56348d20c390 0x56348d20e850 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.793+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/2158420438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 0x56348d1f9900 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.793+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/2158420438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1f2f50 0x56348d1f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.793+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/2158420438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f19e0 0x56348d1fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.793+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2158420438 >> 172.21.15.5:0/2158420438 conn(0x56348d1b6310 msgr2=0x56348d1f3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.793+0000 7fa2a97fa640 0 client.18531 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:15:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.794+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2158420438 shutdown_connections 2024-09-19T06:15:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.794+0000 7fa2ce24e740 1 -- 172.21.15.5:0/2158420438 wait complete. 2024-09-19T06:15:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.794+0000 7fa2ce24e740 20 client.18531 trim_cache size 0 max 16384 2024-09-19T06:15:10.807 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:10 smithi157 ceph-mon[35346]: pgmap v370: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-09-19T06:15:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2ce24e740 1 Processor -- start 2024-09-19T06:15:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2ce24e740 1 -- start start 2024-09-19T06:15:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2ce24e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1f19e0 0x56348d1fb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2ce24e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d1f4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2ce24e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 0x56348d1f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2ce24e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c41ab680 con 0x56348d1fc780 2024-09-19T06:15:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2ce24e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c41ad280 con 0x56348d1f2f50 2024-09-19T06:15:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2ce24e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c41abd50 con 0x56348d1f19e0 2024-09-19T06:15:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2ca412640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 0x56348d1f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2c9c11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1f19e0 0x56348d1fb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2c9410640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d1f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2ca412640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 0x56348d1f4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37334/0 (socket says 172.21.15.5:37334) 2024-09-19T06:15:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2ca412640 1 -- 172.21.15.5:0/4147395484 learned_addr learned my addr 172.21.15.5:0/4147395484 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2c9410640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d1f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60526/0 (socket says 172.21.15.5:60526) 2024-09-19T06:15:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.809+0000 7fa2c9c11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1f19e0 0x56348d1fb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45294/0 (socket says 172.21.15.5:45294) 2024-09-19T06:15:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.810+0000 7fa2ca412640 1 -- 172.21.15.5:0/4147395484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1f19e0 msgr2=0x56348d1fb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.810+0000 7fa2ca412640 1 --2- 172.21.15.5:0/4147395484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1f19e0 0x56348d1fb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.810+0000 7fa2ca412640 1 -- 172.21.15.5:0/4147395484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 msgr2=0x56348d1f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.810+0000 7fa2ca412640 1 --2- 172.21.15.5:0/4147395484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d1f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.810+0000 7fa2ca412640 1 -- 172.21.15.5:0/4147395484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56348d1f4fb0 con 0x56348d1fc780 2024-09-19T06:15:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.810+0000 7fa2c9410640 1 --2- 172.21.15.5:0/4147395484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d1f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.810+0000 7fa2ca412640 1 --2- 172.21.15.5:0/4147395484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 0x56348d1f4a70 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00f430 tx=0x7fa2bc0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef6b30c30f08f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:10.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.810+0000 7fa2ab7fe640 1 -- 172.21.15.5:0/4147395484 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc01b5c0 con 0x56348d1fc780 2024-09-19T06:15:10.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.810+0000 7fa2ab7fe640 1 -- 172.21.15.5:0/4147395484 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2bc01b760 con 0x56348d1fc780 2024-09-19T06:15:10.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.810+0000 7fa2ab7fe640 1 -- 172.21.15.5:0/4147395484 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0307b0 con 0x56348d1fc780 2024-09-19T06:15:10.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.811+0000 7fa2ce24e740 1 -- 172.21.15.5:0/4147395484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 msgr2=0x56348d1f4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.811+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/4147395484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 0x56348d1f4a70 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00f430 tx=0x7fa2bc0095b0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.811+0000 7fa2ce24e740 1 -- 172.21.15.5:0/4147395484 shutdown_connections 2024-09-19T06:15:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.811+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/4147395484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1fc780 0x56348d1f4a70 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.811+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/4147395484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d1f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.811+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/4147395484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1f19e0 0x56348d1fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.811+0000 7fa2ce24e740 1 -- 172.21.15.5:0/4147395484 >> 172.21.15.5:0/4147395484 conn(0x56348d1b6310 msgr2=0x56348d1fac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.811+0000 7fa2ce24e740 1 -- 172.21.15.5:0/4147395484 shutdown_connections 2024-09-19T06:15:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.812+0000 7fa2ce24e740 1 -- 172.21.15.5:0/4147395484 wait complete. 2024-09-19T06:15:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.812+0000 7fa2ce24e740 1 -- start start 2024-09-19T06:15:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.812+0000 7fa2ce24e740 1 Processor -- start 2024-09-19T06:15:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ce24e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1f19e0 0x56348d204940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ce24e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d204e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ce24e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1fc780 0x56348d1f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ce24e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c41acbb0 con 0x56348d1f19e0 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ce24e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c41ac480 con 0x56348d1f2f50 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ce24e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c41a9680 con 0x56348d1fc780 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ce24e740 10 client.0 _ll_register_callbacks cb 0x7fa2bc010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2c9c11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1f19e0 0x56348d204940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ca412640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1fc780 0x56348d1f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2c9c11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1f19e0 0x56348d204940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37344/0 (socket says 172.21.15.5:37344) 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2c9c11640 1 -- 172.21.15.5:0/1436255371 learned_addr learned my addr 172.21.15.5:0/1436255371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2c9410640 1 --2- 172.21.15.5:0/1436255371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d204e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ca412640 1 -- 172.21.15.5:0/1436255371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 msgr2=0x56348d204e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ca412640 1 --2- 172.21.15.5:0/1436255371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d204e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ca412640 1 -- 172.21.15.5:0/1436255371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1f19e0 msgr2=0x56348d204940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ca412640 1 --2- 172.21.15.5:0/1436255371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1f19e0 0x56348d204940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ca412640 1 -- 172.21.15.5:0/1436255371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56348d1f67f0 con 0x56348d1fc780 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2ca412640 1 --2- 172.21.15.5:0/1436255371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1fc780 0x56348d1f61b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00fa70 tx=0x7fa2bc028990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a258495b1b117905 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.813+0000 7fa2a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.814+0000 7fa2a97fa640 1 -- 172.21.15.5:0/1436255371 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc01b5c0 con 0x56348d1fc780 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.814+0000 7fa2a97fa640 1 -- 172.21.15.5:0/1436255371 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fa2bc030590 con 0x56348d1fc780 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.814+0000 7fa2ce24e740 1 -- 172.21.15.5:0/1436255371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2bc01b760 con 0x56348d1fc780 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.814+0000 7fa2a97fa640 1 -- 172.21.15.5:0/1436255371 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa2bc049050 con 0x56348d1fc780 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.814+0000 7fa2ce24e740 10 client.27952 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.814+0000 7fa2ce24e740 1 -- 172.21.15.5:0/1436255371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56348d202330 con 0x56348d1fc780 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.814+0000 7fa2a97fa640 1 -- 172.21.15.5:0/1436255371 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2bc01b760 con 0x56348d1fc780 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.815+0000 7fa2a97fa640 1 -- 172.21.15.5:0/1436255371 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fa2bc037020 con 0x56348d1fc780 2024-09-19T06:15:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.815+0000 7fa2ce24e740 10 client.27952 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.815+0000 7fa2ce24e740 10 client.27952 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:15:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.815+0000 7fa2ce24e740 20 client.27952 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.815+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/1436255371 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56348d27bae0 0x56348d27ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.815+0000 7fa2ce24e740 4 client.27952 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:15:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.815+0000 7fa2ce24e740 1 -- 172.21.15.5:0/1436255371 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56348d15ad70 con 0x56348d27bae0 2024-09-19T06:15:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.815+0000 7fa2c9c11640 1 --2- 172.21.15.5:0/1436255371 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56348d27bae0 0x56348d27ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.815+0000 7fa2c9c11640 1 --2- 172.21.15.5:0/1436255371 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56348d27bae0 0x56348d27ded0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a680daca11540d0 server_cookie=4b5da2153302ecf9 in_seq=0 out_seq=0 2024-09-19T06:15:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.815+0000 7fa2a97fa640 10 client.27952 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:15:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.816+0000 7fa2a97fa640 1 -- 172.21.15.5:0/1436255371 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56348d15ad70 con 0x56348d27bae0 2024-09-19T06:15:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.816+0000 7fa2a97fa640 10 client.27952 handle_command_reply: tid=0 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.816+0000 7fa2ce24e740 1 client.27952 shutdown 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.816+0000 7fa2ce24e740 1 -- 172.21.15.5:0/1436255371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1fc780 msgr2=0x56348d1f61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.816+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/1436255371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1fc780 0x56348d1f61b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00fa70 tx=0x7fa2bc028990 comp rx=0 tx=0).stop 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2ce24e740 1 -- 172.21.15.5:0/1436255371 shutdown_connections 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/1436255371 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56348d27bae0 0x56348d27ded0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/1436255371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56348d1fc780 0x56348d1f61b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/1436255371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56348d1f2f50 0x56348d204e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2ce24e740 1 --2- 172.21.15.5:0/1436255371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56348d1f19e0 0x56348d204940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2ce24e740 1 -- 172.21.15.5:0/1436255371 >> 172.21.15.5:0/1436255371 conn(0x56348d1b6310 msgr2=0x56348d1fb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2a97fa640 0 client.27952 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2ce24e740 1 -- 172.21.15.5:0/1436255371 shutdown_connections 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2ce24e740 1 -- 172.21.15.5:0/1436255371 wait complete. 2024-09-19T06:15:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2ce24e740 20 client.27952 trim_cache size 0 max 16384 2024-09-19T06:15:10.823 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:15:10.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2cbe75640 1 -- 172.21.15.5:0/4152649950 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2ac0a33b0 msgr2=0x7fa2ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2cbe75640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2ac0a33b0 0x7fa2ac0a5870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009fd0 tx=0x7fa2bc0046c0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2cbe75640 1 -- 172.21.15.5:0/4152649950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c4101de0 msgr2=0x7fa2c4198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2cbe75640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c4101de0 0x7fa2c4198340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009ca0 tx=0x7fa2b4031990 comp rx=0 tx=0).stop 2024-09-19T06:15:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2cbe75640 1 -- 172.21.15.5:0/4152649950 shutdown_connections 2024-09-19T06:15:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2cbe75640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2ac0a33b0 0x7fa2ac0a5870 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2cbe75640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c4106f10 0x7fa2c419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2cbe75640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c4102790 0x7fa2c4198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2cbe75640 1 --2- 172.21.15.5:0/4152649950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c4101de0 0x7fa2c4198340 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.817+0000 7fa2cbe75640 1 -- 172.21.15.5:0/4152649950 >> 172.21.15.5:0/4152649950 conn(0x7fa2c40fd8e0 msgr2=0x7fa2c40fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.818+0000 7fa2cbe75640 1 -- 172.21.15.5:0/4152649950 shutdown_connections 2024-09-19T06:15:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:10.818+0000 7fa2cbe75640 1 -- 172.21.15.5:0/4152649950 wait complete. 2024-09-19T06:15:10.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273402 2.21 MB/sec execute 516 sec latency 4737.498 ms 2024-09-19T06:15:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:10 smithi049 ceph-mon[34640]: pgmap v370: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-09-19T06:15:11.557 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:15:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:11.846 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273682 2.21 MB/sec execute 517 sec latency 14.796 ms 2024-09-19T06:15:12.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.356+0000 7f92369b5640 1 -- 172.21.15.5:0/155363939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230101de0 msgr2=0x7f92301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:12.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.356+0000 7f92369b5640 1 --2- 172.21.15.5:0/155363939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230101de0 0x7f92301021c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f9220009900 tx=0x7f922002a3f0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.356+0000 7f92369b5640 1 -- 172.21.15.5:0/155363939 shutdown_connections 2024-09-19T06:15:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.356+0000 7f92369b5640 1 --2- 172.21.15.5:0/155363939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9230106f10 0x7f923010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.356+0000 7f92369b5640 1 --2- 172.21.15.5:0/155363939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9230102790 0x7f92301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.356+0000 7f92369b5640 1 --2- 172.21.15.5:0/155363939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230101de0 0x7f92301021c0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.356+0000 7f92369b5640 1 -- 172.21.15.5:0/155363939 >> 172.21.15.5:0/155363939 conn(0x7f92300fd8e0 msgr2=0x7f92300ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.356+0000 7f92369b5640 1 -- 172.21.15.5:0/155363939 shutdown_connections 2024-09-19T06:15:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.356+0000 7f92369b5640 1 -- 172.21.15.5:0/155363939 wait complete. 2024-09-19T06:15:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.356+0000 7f92369b5640 1 Processor -- start 2024-09-19T06:15:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.356+0000 7f92369b5640 1 -- start start 2024-09-19T06:15:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f92369b5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9230101de0 0x7f9230198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f92369b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230102790 0x7f9230198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f92369b5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9230106f10 0x7f923019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f92369b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9230111de0 con 0x7f9230102790 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f92369b5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9230111c60 con 0x7f9230101de0 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f92369b5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9230111f60 con 0x7f9230106f10 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230102790 0x7f9230198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230102790 0x7f9230198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37358/0 (socket says 172.21.15.5:37358) 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f922f7fe640 1 -- 172.21.15.5:0/1351908115 learned_addr learned my addr 172.21.15.5:0/1351908115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f922ffff640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9230101de0 0x7f9230198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f9234f52640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9230106f10 0x7f923019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f922f7fe640 1 -- 172.21.15.5:0/1351908115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9230106f10 msgr2=0x7f923019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f922f7fe640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9230106f10 0x7f923019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f922f7fe640 1 -- 172.21.15.5:0/1351908115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9230101de0 msgr2=0x7f9230198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f922f7fe640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9230101de0 0x7f9230198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f922f7fe640 1 -- 172.21.15.5:0/1351908115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923019d2f0 con 0x7f9230102790 2024-09-19T06:15:12.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f9234f52640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9230106f10 0x7f923019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:12.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.357+0000 7f922ffff640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9230101de0 0x7f9230198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:12.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.358+0000 7f922f7fe640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230102790 0x7f9230198880 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f922400e9f0 tx=0x7f922400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:12.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.358+0000 7f922d7fa640 1 -- 172.21.15.5:0/1351908115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922400cd80 con 0x7f9230102790 2024-09-19T06:15:12.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.358+0000 7f922d7fa640 1 -- 172.21.15.5:0/1351908115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92240108e0 con 0x7f9230102790 2024-09-19T06:15:12.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.358+0000 7f92369b5640 1 -- 172.21.15.5:0/1351908115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923019d5e0 con 0x7f9230102790 2024-09-19T06:15:12.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.358+0000 7f922d7fa640 1 -- 172.21.15.5:0/1351908115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224018920 con 0x7f9230102790 2024-09-19T06:15:12.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.358+0000 7f92369b5640 1 -- 172.21.15.5:0/1351908115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92301a4ec0 con 0x7f9230102790 2024-09-19T06:15:12.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.360+0000 7f922d7fa640 1 -- 172.21.15.5:0/1351908115 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9224010a90 con 0x7f9230102790 2024-09-19T06:15:12.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.361+0000 7f9238d8e740 1 Processor -- start 2024-09-19T06:15:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.361+0000 7f9238d8e740 1 -- start start 2024-09-19T06:15:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.361+0000 7f922d7fa640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92180a3430 0x7f92180a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.361+0000 7f9238d8e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 0x55f4efda1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.361+0000 7f9238d8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda15d0 0x55f4efda1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.361+0000 7f9234f52640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92180a3430 0x7f92180a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.361+0000 7f9238d8e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efdb0780 0x55f4efdb2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.361+0000 7f9238d8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4efd072f0 con 0x55f4efda15d0 2024-09-19T06:15:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.361+0000 7f9238d8e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f4efcaecf0 con 0x55f4efdb0780 2024-09-19T06:15:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.361+0000 7f9238d8e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f4efd032b0 con 0x55f4efda2b40 2024-09-19T06:15:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.361+0000 7f922d7fa640 1 -- 172.21.15.5:0/1351908115 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f92240cffc0 con 0x7f9230102790 2024-09-19T06:15:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.361+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda15d0 0x55f4efda1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.362+0000 7f922ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 0x55f4efda1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.362+0000 7f922ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 0x55f4efda1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45324/0 (socket says 172.21.15.5:45324) 2024-09-19T06:15:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.362+0000 7f922ffff640 1 -- 172.21.15.5:0/225055954 learned_addr learned my addr 172.21.15.5:0/225055954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.362+0000 7f9234f52640 1 --2- 172.21.15.5:0/225055954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efdb0780 0x55f4efdb2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.362+0000 7f922ffff640 1 -- 172.21.15.5:0/225055954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efdb0780 msgr2=0x55f4efdb2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.362+0000 7f922ffff640 1 --2- 172.21.15.5:0/225055954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efdb0780 0x55f4efdb2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.362+0000 7f922ffff640 1 -- 172.21.15.5:0/225055954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda15d0 msgr2=0x55f4efda1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.362+0000 7f922ffff640 1 --2- 172.21.15.5:0/225055954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda15d0 0x55f4efda1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.362+0000 7f922ffff640 1 -- 172.21.15.5:0/225055954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4efc933a0 con 0x55f4efda2b40 2024-09-19T06:15:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.362+0000 7f9234f52640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92180a3430 0x7f92180a58f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9228009fd0 tx=0x7f9228011c90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.362+0000 7f922ffff640 1 --2- 172.21.15.5:0/225055954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 0x55f4efda1090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9220009900 tx=0x7f9220005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b3a43512f4a87a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.362+0000 7f92167fc640 1 -- 172.21.15.5:0/225055954 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220040070 con 0x55f4efda2b40 2024-09-19T06:15:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.363+0000 7f92167fc640 1 -- 172.21.15.5:0/225055954 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9220031860 con 0x55f4efda2b40 2024-09-19T06:15:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.363+0000 7f92167fc640 1 -- 172.21.15.5:0/225055954 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220035040 con 0x55f4efda2b40 2024-09-19T06:15:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.363+0000 7f9238d8e740 1 -- 172.21.15.5:0/225055954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 msgr2=0x55f4efda1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.363+0000 7f9238d8e740 1 --2- 172.21.15.5:0/225055954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 0x55f4efda1090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9220009900 tx=0x7f9220005d90 comp rx=0 tx=0).stop 2024-09-19T06:15:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.363+0000 7f9238d8e740 1 -- 172.21.15.5:0/225055954 shutdown_connections 2024-09-19T06:15:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.363+0000 7f9238d8e740 1 --2- 172.21.15.5:0/225055954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efdb0780 0x55f4efdb2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.363+0000 7f9238d8e740 1 --2- 172.21.15.5:0/225055954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda15d0 0x55f4efda1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.363+0000 7f9238d8e740 1 --2- 172.21.15.5:0/225055954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 0x55f4efda1090 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.363+0000 7f9238d8e740 1 -- 172.21.15.5:0/225055954 >> 172.21.15.5:0/225055954 conn(0x55f4efd6a310 msgr2=0x55f4efd9dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.363+0000 7f9238d8e740 1 -- 172.21.15.5:0/225055954 shutdown_connections 2024-09-19T06:15:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.363+0000 7f9238d8e740 1 -- 172.21.15.5:0/225055954 wait complete. 2024-09-19T06:15:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f9238d8e740 1 -- start start 2024-09-19T06:15:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f9238d8e740 1 Processor -- start 2024-09-19T06:15:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f9238d8e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda15d0 0x55f4efdb0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f9238d8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda2b40 0x55f4efdad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f9238d8e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efdb0780 0x55f4efdad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f9238d8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4efdb3420 con 0x55f4efda2b40 2024-09-19T06:15:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f9238d8e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f4efcc2060 con 0x55f4efdb0780 2024-09-19T06:15:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f9238d8e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f4efdb35a0 con 0x55f4efda15d0 2024-09-19T06:15:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f9238d8e740 10 client.0 _ll_register_callbacks cb 0x55f4efa158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda2b40 0x55f4efdad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda2b40 0x55f4efdad3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37380/0 (socket says 172.21.15.5:37380) 2024-09-19T06:15:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f922ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda15d0 0x55f4efdb0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f922f7fe640 1 -- 172.21.15.5:0/736528909 learned_addr learned my addr 172.21.15.5:0/736528909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.364+0000 7f9234f52640 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efdb0780 0x55f4efdad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f922f7fe640 1 -- 172.21.15.5:0/736528909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda15d0 msgr2=0x55f4efdb0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f922f7fe640 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda15d0 0x55f4efdb0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f922f7fe640 1 -- 172.21.15.5:0/736528909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efdb0780 msgr2=0x55f4efdad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f922f7fe640 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efdb0780 0x55f4efdad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f922f7fe640 1 -- 172.21.15.5:0/736528909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4efa783c0 con 0x55f4efda2b40 2024-09-19T06:15:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f922ffff640 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda15d0 0x55f4efdb0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f9234f52640 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efdb0780 0x55f4efdad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f922f7fe640 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda2b40 0x55f4efdad3d0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f922400cfd0 tx=0x7f922409bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a06438b3a60452c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f920f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f920f7fe640 1 -- 172.21.15.5:0/736528909 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240b5060 con 0x55f4efda2b40 2024-09-19T06:15:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f9238d8e740 1 -- 172.21.15.5:0/736528909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9220031860 con 0x55f4efda2b40 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f920f7fe640 1 -- 172.21.15.5:0/736528909 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f92240a0170 con 0x55f4efda2b40 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f920f7fe640 1 -- 172.21.15.5:0/736528909 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f92240a4ef0 con 0x55f4efda2b40 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f9238d8e740 10 client.18549 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.365+0000 7f9238d8e740 1 -- 172.21.15.5:0/736528909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f4efdbeec0 con 0x55f4efda2b40 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.366+0000 7f920f7fe640 1 -- 172.21.15.5:0/736528909 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9220031860 con 0x55f4efda2b40 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.366+0000 7f920f7fe640 1 -- 172.21.15.5:0/736528909 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f922409b810 con 0x55f4efda2b40 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.366+0000 7f9238d8e740 10 client.18549 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.366+0000 7f9238d8e740 10 client.18549 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.366+0000 7f9238d8e740 20 client.18549 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.366+0000 7f9238d8e740 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4efdc02d0 0x55f4efdc26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.366+0000 7f9238d8e740 4 client.18549 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.366+0000 7f9238d8e740 1 -- 172.21.15.5:0/736528909 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4efd14860 con 0x55f4efdc02d0 2024-09-19T06:15:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.366+0000 7f922ffff640 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4efdc02d0 0x55f4efdc26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.366+0000 7f922ffff640 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4efdc02d0 0x55f4efdc26c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27028af293d6ece9 server_cookie=bf97be5b8848ec50 in_seq=0 out_seq=0 2024-09-19T06:15:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.366+0000 7f920f7fe640 10 client.18549 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:15:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.367+0000 7f920f7fe640 1 -- 172.21.15.5:0/736528909 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4efd14860 con 0x55f4efdc02d0 2024-09-19T06:15:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.367+0000 7f920f7fe640 10 client.18549 handle_command_reply: tid=0 2024-09-19T06:15:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.367+0000 7f9238d8e740 1 client.18549 shutdown 2024-09-19T06:15:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.367+0000 7f9238d8e740 1 -- 172.21.15.5:0/736528909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda2b40 msgr2=0x55f4efdad3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.367+0000 7f9238d8e740 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda2b40 0x55f4efdad3d0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f922400cfd0 tx=0x7f922409bf80 comp rx=0 tx=0).stop 2024-09-19T06:15:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.367+0000 7f9238d8e740 1 -- 172.21.15.5:0/736528909 shutdown_connections 2024-09-19T06:15:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.367+0000 7f9238d8e740 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4efdc02d0 0x55f4efdc26c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.367+0000 7f9238d8e740 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efdb0780 0x55f4efdad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.367+0000 7f9238d8e740 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda2b40 0x55f4efdad3d0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.367+0000 7f9238d8e740 1 --2- 172.21.15.5:0/736528909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda15d0 0x55f4efdb0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.367+0000 7f920f7fe640 0 client.18549 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:15:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.367+0000 7f9238d8e740 1 -- 172.21.15.5:0/736528909 >> 172.21.15.5:0/736528909 conn(0x55f4efd6a310 msgr2=0x55f4efda3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.368+0000 7f9238d8e740 1 -- 172.21.15.5:0/736528909 shutdown_connections 2024-09-19T06:15:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.368+0000 7f9238d8e740 1 -- 172.21.15.5:0/736528909 wait complete. 2024-09-19T06:15:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.368+0000 7f9238d8e740 20 client.18549 trim_cache size 0 max 16384 2024-09-19T06:15:12.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.382+0000 7f9238d8e740 1 Processor -- start 2024-09-19T06:15:12.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.382+0000 7f9238d8e740 1 -- start start 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f9238d8e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efda15d0 0x55f4efdbc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f9238d8e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 0x55f4efdaf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f9238d8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efdb0780 0x55f4efdc7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f9238d8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92301ab640 con 0x55f4efdb0780 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f9238d8e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f92301ad240 con 0x55f4efda15d0 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f9238d8e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f92301abd10 con 0x55f4efda2b40 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f9234f52640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efdb0780 0x55f4efdc7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 0x55f4efdaf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f922ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efda15d0 0x55f4efdbc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 0x55f4efdaf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45328/0 (socket says 172.21.15.5:45328) 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f922f7fe640 1 -- 172.21.15.5:0/984176288 learned_addr learned my addr 172.21.15.5:0/984176288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f922f7fe640 1 -- 172.21.15.5:0/984176288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efda15d0 msgr2=0x55f4efdbc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f922f7fe640 1 --2- 172.21.15.5:0/984176288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efda15d0 0x55f4efdbc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f922f7fe640 1 -- 172.21.15.5:0/984176288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efdb0780 msgr2=0x55f4efdc7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f922f7fe640 1 --2- 172.21.15.5:0/984176288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efdb0780 0x55f4efdc7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f922f7fe640 1 -- 172.21.15.5:0/984176288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4efdc2a30 con 0x55f4efda2b40 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.383+0000 7f9234f52640 1 --2- 172.21.15.5:0/984176288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efdb0780 0x55f4efdc7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.384+0000 7f922f7fe640 1 --2- 172.21.15.5:0/984176288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 0x55f4efdaf340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f922409c030 tx=0x7f9224005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9564c0e83dd9217e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.384+0000 7f9215ffb640 1 -- 172.21.15.5:0/984176288 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240b5040 con 0x55f4efda2b40 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.384+0000 7f9215ffb640 1 -- 172.21.15.5:0/984176288 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f922409b780 con 0x55f4efda2b40 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.384+0000 7f9215ffb640 1 -- 172.21.15.5:0/984176288 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922409bc90 con 0x55f4efda2b40 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.384+0000 7f9238d8e740 1 -- 172.21.15.5:0/984176288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 msgr2=0x55f4efdaf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.384+0000 7f9238d8e740 1 --2- 172.21.15.5:0/984176288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 0x55f4efdaf340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f922409c030 tx=0x7f9224005e30 comp rx=0 tx=0).stop 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.385+0000 7f9238d8e740 1 -- 172.21.15.5:0/984176288 shutdown_connections 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.385+0000 7f9238d8e740 1 --2- 172.21.15.5:0/984176288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efdb0780 0x55f4efdc7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.385+0000 7f9238d8e740 1 --2- 172.21.15.5:0/984176288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efda2b40 0x55f4efdaf340 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.385+0000 7f9238d8e740 1 --2- 172.21.15.5:0/984176288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efda15d0 0x55f4efdbc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.385+0000 7f9238d8e740 1 -- 172.21.15.5:0/984176288 >> 172.21.15.5:0/984176288 conn(0x55f4efd6a310 msgr2=0x55f4efdaec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.385+0000 7f9238d8e740 1 -- 172.21.15.5:0/984176288 shutdown_connections 2024-09-19T06:15:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.385+0000 7f9238d8e740 1 -- 172.21.15.5:0/984176288 wait complete. 2024-09-19T06:15:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.385+0000 7f9238d8e740 1 -- start start 2024-09-19T06:15:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.385+0000 7f9238d8e740 1 Processor -- start 2024-09-19T06:15:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f9238d8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda15d0 0x55f4efdb8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f9238d8e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efda2b40 0x55f4efdaa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f9238d8e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efdb0780 0x55f4efdaa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f9238d8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92301acb70 con 0x55f4efda15d0 2024-09-19T06:15:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f9238d8e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f92301ac440 con 0x55f4efda2b40 2024-09-19T06:15:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f9238d8e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f92301a9640 con 0x55f4efdb0780 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f922ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda15d0 0x55f4efdb8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efda2b40 0x55f4efdaa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f9238d8e740 10 client.0 _ll_register_callbacks cb 0x7f922400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f922ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda15d0 0x55f4efdb8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37400/0 (socket says 172.21.15.5:37400) 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f922ffff640 1 -- 172.21.15.5:0/2660446184 learned_addr learned my addr 172.21.15.5:0/2660446184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f9234f52640 1 --2- 172.21.15.5:0/2660446184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efdb0780 0x55f4efdaa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f922ffff640 1 -- 172.21.15.5:0/2660446184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efdb0780 msgr2=0x55f4efdaa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f922ffff640 1 --2- 172.21.15.5:0/2660446184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efdb0780 0x55f4efdaa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f922ffff640 1 -- 172.21.15.5:0/2660446184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efda2b40 msgr2=0x55f4efdaa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f922ffff640 1 --2- 172.21.15.5:0/2660446184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efda2b40 0x55f4efdaa1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f922ffff640 1 -- 172.21.15.5:0/2660446184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4efdb0490 con 0x55f4efda15d0 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f922ffff640 1 --2- 172.21.15.5:0/2660446184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda15d0 0x55f4efdb8f80 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f9220009900 tx=0x7f922002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8765f0b24da4ffe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.386+0000 7f920f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f920f7fe640 1 -- 172.21.15.5:0/2660446184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220035040 con 0x55f4efda15d0 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f920f7fe640 1 -- 172.21.15.5:0/2660446184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f9220004240 con 0x55f4efda15d0 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f9238d8e740 1 -- 172.21.15.5:0/2660446184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f922409b780 con 0x55f4efda15d0 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f920f7fe640 1 -- 172.21.15.5:0/2660446184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f922005f050 con 0x55f4efda15d0 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f9238d8e740 10 client.18555 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f9238d8e740 1 -- 172.21.15.5:0/2660446184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f4efe2ae50 con 0x55f4efda15d0 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f920f7fe640 1 -- 172.21.15.5:0/2660446184 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f922409b780 con 0x55f4efda15d0 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f920f7fe640 1 -- 172.21.15.5:0/2660446184 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f9220004770 con 0x55f4efda15d0 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f9238d8e740 10 client.18555 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f9238d8e740 10 client.18555 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f9238d8e740 20 client.18555 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f9238d8e740 1 --2- 172.21.15.5:0/2660446184 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4efe2b2a0 0x55f4efe2d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f9238d8e740 4 client.18555 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f9238d8e740 1 -- 172.21.15.5:0/2660446184 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4efd0ed70 con 0x55f4efe2b2a0 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.387+0000 7f922f7fe640 1 --2- 172.21.15.5:0/2660446184 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4efe2b2a0 0x55f4efe2d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f922f7fe640 1 --2- 172.21.15.5:0/2660446184 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4efe2b2a0 0x55f4efe2d6b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=793dbaa2225adf15 server_cookie=14854fa9264cd88d in_seq=0 out_seq=0 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f920f7fe640 10 client.18555 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f920f7fe640 1 -- 172.21.15.5:0/2660446184 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4efd0ed70 con 0x55f4efe2b2a0 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f920f7fe640 10 client.18555 handle_command_reply: tid=0 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f9238d8e740 1 client.18555 shutdown 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f9238d8e740 1 -- 172.21.15.5:0/2660446184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda15d0 msgr2=0x55f4efdb8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f9238d8e740 1 --2- 172.21.15.5:0/2660446184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda15d0 0x55f4efdb8f80 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f9220009900 tx=0x7f922002a3f0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f9238d8e740 1 -- 172.21.15.5:0/2660446184 shutdown_connections 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f9238d8e740 1 --2- 172.21.15.5:0/2660446184 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4efe2b2a0 0x55f4efe2d6b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f920f7fe640 0 client.18555 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:15:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f9238d8e740 1 --2- 172.21.15.5:0/2660446184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4efdb0780 0x55f4efdaa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f9238d8e740 1 --2- 172.21.15.5:0/2660446184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4efda2b40 0x55f4efdaa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f9238d8e740 1 --2- 172.21.15.5:0/2660446184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4efda15d0 0x55f4efdb8f80 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f9238d8e740 1 -- 172.21.15.5:0/2660446184 >> 172.21.15.5:0/2660446184 conn(0x55f4efd6a310 msgr2=0x55f4efdab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.388+0000 7f9238d8e740 1 -- 172.21.15.5:0/2660446184 shutdown_connections 2024-09-19T06:15:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.389+0000 7f9238d8e740 1 -- 172.21.15.5:0/2660446184 wait complete. 2024-09-19T06:15:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.389+0000 7f9238d8e740 20 client.18555 trim_cache size 0 max 16384 2024-09-19T06:15:12.400 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:15:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.389+0000 7f92369b5640 1 -- 172.21.15.5:0/1351908115 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92180a3430 msgr2=0x7f92180a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.389+0000 7f92369b5640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92180a3430 0x7f92180a58f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9228009fd0 tx=0x7f9228011c90 comp rx=0 tx=0).stop 2024-09-19T06:15:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.389+0000 7f92369b5640 1 -- 172.21.15.5:0/1351908115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230102790 msgr2=0x7f9230198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.389+0000 7f92369b5640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230102790 0x7f9230198880 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f922400e9f0 tx=0x7f922400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.389+0000 7f92369b5640 1 -- 172.21.15.5:0/1351908115 shutdown_connections 2024-09-19T06:15:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.389+0000 7f92369b5640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92180a3430 0x7f92180a58f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.389+0000 7f92369b5640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9230106f10 0x7f923019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.389+0000 7f92369b5640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230102790 0x7f9230198880 secure :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f922400e9f0 tx=0x7f922400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.389+0000 7f92369b5640 1 --2- 172.21.15.5:0/1351908115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9230101de0 0x7f9230198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:12.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.389+0000 7f92369b5640 1 -- 172.21.15.5:0/1351908115 >> 172.21.15.5:0/1351908115 conn(0x7f92300fd8e0 msgr2=0x7f92300fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:12.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.390+0000 7f92369b5640 1 -- 172.21.15.5:0/1351908115 shutdown_connections 2024-09-19T06:15:12.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:12.390+0000 7f92369b5640 1 -- 172.21.15.5:0/1351908115 wait complete. 2024-09-19T06:15:12.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:12 smithi005 ceph-mon[30955]: pgmap v371: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 244 KiB/s wr, 30 op/s 2024-09-19T06:15:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:12 smithi157 ceph-mon[35346]: pgmap v371: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 244 KiB/s wr, 30 op/s 2024-09-19T06:15:12.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 274103 2.20 MB/sec execute 518 sec latency 20.890 ms 2024-09-19T06:15:12.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:12 smithi049 ceph-mon[34640]: pgmap v371: 97 pgs: 97 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 244 KiB/s wr, 30 op/s 2024-09-19T06:15:13.449 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:15:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:13.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 274648 2.21 MB/sec execute 519 sec latency 21.576 ms 2024-09-19T06:15:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.225+0000 7fda1dd6b640 1 -- 172.21.15.5:0/1862621723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18103660 msgr2=0x7fda1810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.225+0000 7fda1dd6b640 1 --2- 172.21.15.5:0/1862621723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18103660 0x7fda1810bb50 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fda00009900 tx=0x7fda0002a480 comp rx=0 tx=0).stop 2024-09-19T06:15:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.225+0000 7fda1dd6b640 1 -- 172.21.15.5:0/1862621723 shutdown_connections 2024-09-19T06:15:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.225+0000 7fda1dd6b640 1 --2- 172.21.15.5:0/1862621723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda1810c170 0x7fda1810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.225+0000 7fda1dd6b640 1 --2- 172.21.15.5:0/1862621723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18103660 0x7fda1810bb50 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.225+0000 7fda1dd6b640 1 --2- 172.21.15.5:0/1862621723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda18102d40 0x7fda18103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.225+0000 7fda1dd6b640 1 -- 172.21.15.5:0/1862621723 >> 172.21.15.5:0/1862621723 conn(0x7fda180fc7a0 msgr2=0x7fda180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.225+0000 7fda1dd6b640 1 -- 172.21.15.5:0/1862621723 shutdown_connections 2024-09-19T06:15:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.225+0000 7fda1dd6b640 1 -- 172.21.15.5:0/1862621723 wait complete. 2024-09-19T06:15:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda1dd6b640 1 Processor -- start 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda1dd6b640 1 -- start start 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda1dd6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18102d40 0x7fda181984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda1dd6b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda18103660 0x7fda181989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda1dd6b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda1810c170 0x7fda1819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda1dd6b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda18111e70 con 0x7fda18102d40 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda1dd6b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fda18111cf0 con 0x7fda1810c170 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda1dd6b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda18111ff0 con 0x7fda18103660 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18102d40 0x7fda181984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18102d40 0x7fda181984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37426/0 (socket says 172.21.15.5:37426) 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda177fe640 1 -- 172.21.15.5:0/3867195222 learned_addr learned my addr 172.21.15.5:0/3867195222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda17fff640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda1810c170 0x7fda1819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda0ffff640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda18103660 0x7fda181989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda177fe640 1 -- 172.21.15.5:0/3867195222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda18103660 msgr2=0x7fda181989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda177fe640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda18103660 0x7fda181989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda177fe640 1 -- 172.21.15.5:0/3867195222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda1810c170 msgr2=0x7fda1819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda177fe640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda1810c170 0x7fda1819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.226+0000 7fda177fe640 1 -- 172.21.15.5:0/3867195222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1819d460 con 0x7fda18102d40 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.227+0000 7fda177fe640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18102d40 0x7fda181984b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fda0800e9f0 tx=0x7fda0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.227+0000 7fda157fa640 1 -- 172.21.15.5:0/3867195222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0800ccc0 con 0x7fda18102d40 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.227+0000 7fda157fa640 1 -- 172.21.15.5:0/3867195222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda0800ce60 con 0x7fda18102d40 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.227+0000 7fda1dd6b640 1 -- 172.21.15.5:0/3867195222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1819d750 con 0x7fda18102d40 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.227+0000 7fda157fa640 1 -- 172.21.15.5:0/3867195222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08010430 con 0x7fda18102d40 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.227+0000 7fda1dd6b640 1 -- 172.21.15.5:0/3867195222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda181a5030 con 0x7fda18102d40 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.229+0000 7fda157fa640 1 -- 172.21.15.5:0/3867195222 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fda08005020 con 0x7fda18102d40 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.229+0000 7fda20144740 1 Processor -- start 2024-09-19T06:15:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda20144740 1 -- start start 2024-09-19T06:15:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda20144740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 0x5588f38b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda20144740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38b35d0 0x5588f38b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda20144740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 0x5588f38c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda20144740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5588f38192f0 con 0x5588f38be370 2024-09-19T06:15:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda20144740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5588f37c0cf0 con 0x5588f38b35d0 2024-09-19T06:15:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda20144740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5588f38152b0 con 0x5588f38b4b40 2024-09-19T06:15:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda17fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 0x5588f38b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 0x5588f38c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda17fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 0x5588f38b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45364/0 (socket says 172.21.15.5:45364) 2024-09-19T06:15:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 0x5588f38c0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37434/0 (socket says 172.21.15.5:37434) 2024-09-19T06:15:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda17fff640 1 -- 172.21.15.5:0/2239121409 learned_addr learned my addr 172.21.15.5:0/2239121409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda157fa640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fda040a3430 0x7fda040a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda177fe640 1 --2- 172.21.15.5:0/2239121409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38b35d0 0x5588f38b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda17fff640 1 -- 172.21.15.5:0/2239121409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38b35d0 msgr2=0x5588f38b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda17fff640 1 --2- 172.21.15.5:0/2239121409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38b35d0 0x5588f38b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda17fff640 1 -- 172.21.15.5:0/2239121409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 msgr2=0x5588f38c0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda17fff640 1 --2- 172.21.15.5:0/2239121409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 0x5588f38c0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda17fff640 1 -- 172.21.15.5:0/2239121409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588f37a53a0 con 0x5588f38b4b40 2024-09-19T06:15:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda157fa640 1 -- 172.21.15.5:0/3867195222 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fda080d00a0 con 0x7fda18102d40 2024-09-19T06:15:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda177fe640 1 --2- 172.21.15.5:0/2239121409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38b35d0 0x5588f38b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda17fff640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fda040a3430 0x7fda040a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.230+0000 7fda0ffff640 1 --2- 172.21.15.5:0/2239121409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 0x5588f38c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.231+0000 7fda17fff640 1 --2- 172.21.15.5:0/2239121409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 0x5588f38b3090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fda10011590 tx=0x7fda10009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8390692ced927edb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.231+0000 7fda17fff640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fda040a3430 0x7fda040a58f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda10023000 tx=0x7fda10024000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.231+0000 7fda0dffb640 1 -- 172.21.15.5:0/2239121409 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10017050 con 0x5588f38b4b40 2024-09-19T06:15:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.231+0000 7fda0dffb640 1 -- 172.21.15.5:0/2239121409 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda1001bc70 con 0x5588f38b4b40 2024-09-19T06:15:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.231+0000 7fda0dffb640 1 -- 172.21.15.5:0/2239121409 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1000eb40 con 0x5588f38b4b40 2024-09-19T06:15:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.232+0000 7fda20144740 1 -- 172.21.15.5:0/2239121409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 msgr2=0x5588f38b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.232+0000 7fda20144740 1 --2- 172.21.15.5:0/2239121409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 0x5588f38b3090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fda10011590 tx=0x7fda10009f90 comp rx=0 tx=0).stop 2024-09-19T06:15:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.232+0000 7fda20144740 1 -- 172.21.15.5:0/2239121409 shutdown_connections 2024-09-19T06:15:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.232+0000 7fda20144740 1 --2- 172.21.15.5:0/2239121409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 0x5588f38c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.232+0000 7fda20144740 1 --2- 172.21.15.5:0/2239121409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38b35d0 0x5588f38b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.232+0000 7fda20144740 1 --2- 172.21.15.5:0/2239121409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 0x5588f38b3090 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.232+0000 7fda20144740 1 -- 172.21.15.5:0/2239121409 >> 172.21.15.5:0/2239121409 conn(0x5588f387c310 msgr2=0x5588f38afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:14.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.232+0000 7fda20144740 1 -- 172.21.15.5:0/2239121409 shutdown_connections 2024-09-19T06:15:14.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.232+0000 7fda20144740 1 -- 172.21.15.5:0/2239121409 wait complete. 2024-09-19T06:15:14.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.232+0000 7fda20144740 1 -- start start 2024-09-19T06:15:14.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda20144740 1 Processor -- start 2024-09-19T06:15:14.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda20144740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 0x5588f38bdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda20144740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38b4b40 0x5588f38bb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda20144740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 0x5588f38bb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda20144740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5588f38c1070 con 0x5588f38be370 2024-09-19T06:15:14.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda20144740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5588f37d4060 con 0x5588f38b4b40 2024-09-19T06:15:14.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda20144740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5588f38c11f0 con 0x5588f38b35d0 2024-09-19T06:15:14.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda20144740 10 client.0 _ll_register_callbacks cb 0x5588f35278c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:14.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda17fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 0x5588f38bb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 0x5588f38bdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 0x5588f38bdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45370/0 (socket says 172.21.15.5:45370) 2024-09-19T06:15:14.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38b4b40 0x5588f38bb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda17fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 0x5588f38bb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37440/0 (socket says 172.21.15.5:37440) 2024-09-19T06:15:14.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda177fe640 1 -- 172.21.15.5:0/3037456291 learned_addr learned my addr 172.21.15.5:0/3037456291 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:14.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda177fe640 1 -- 172.21.15.5:0/3037456291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38b4b40 msgr2=0x5588f38bb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:14.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda177fe640 1 --2- 172.21.15.5:0/3037456291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38b4b40 0x5588f38bb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda177fe640 1 -- 172.21.15.5:0/3037456291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 msgr2=0x5588f38bb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:14.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda177fe640 1 --2- 172.21.15.5:0/3037456291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 0x5588f38bb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.233+0000 7fda177fe640 1 -- 172.21.15.5:0/3037456291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588f358a3c0 con 0x5588f38b35d0 2024-09-19T06:15:14.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.234+0000 7fda177fe640 1 --2- 172.21.15.5:0/3037456291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 0x5588f38bdd50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fda080cd830 tx=0x7fda0809bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e77633a064a0fba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:14.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.234+0000 7fd9f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:15:14.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.234+0000 7fd9f6ffd640 1 -- 172.21.15.5:0/3037456291 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0809ca60 con 0x5588f38b35d0 2024-09-19T06:15:14.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.234+0000 7fd9f6ffd640 1 -- 172.21.15.5:0/3037456291 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fda08005310 con 0x5588f38b35d0 2024-09-19T06:15:14.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.234+0000 7fda20144740 1 -- 172.21.15.5:0/3037456291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda1001bc70 con 0x5588f38b35d0 2024-09-19T06:15:14.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.234+0000 7fd9f6ffd640 1 -- 172.21.15.5:0/3037456291 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fda08005530 con 0x5588f38b35d0 2024-09-19T06:15:14.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.234+0000 7fda20144740 10 client.27982 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:14.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.234+0000 7fda20144740 1 -- 172.21.15.5:0/3037456291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5588f38ccaf0 con 0x5588f38b35d0 2024-09-19T06:15:14.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.234+0000 7fd9f6ffd640 1 -- 172.21.15.5:0/3037456291 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda1001bc70 con 0x5588f38b35d0 2024-09-19T06:15:14.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.235+0000 7fd9f6ffd640 1 -- 172.21.15.5:0/3037456291 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fda080d2110 con 0x5588f38b35d0 2024-09-19T06:15:14.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.235+0000 7fda20144740 10 client.27982 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:14.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.235+0000 7fda20144740 10 client.27982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:14.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.235+0000 7fda20144740 20 client.27982 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:14.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.235+0000 7fda20144740 1 --2- 172.21.15.5:0/3037456291 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5588f38cdf00 0x5588f38d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.235+0000 7fda20144740 4 client.27982 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:14.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.235+0000 7fda20144740 1 -- 172.21.15.5:0/3037456291 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588f3826860 con 0x5588f38cdf00 2024-09-19T06:15:14.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.235+0000 7fda0ffff640 1 --2- 172.21.15.5:0/3037456291 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5588f38cdf00 0x5588f38d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.235+0000 7fd9f6ffd640 10 client.27982 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:14.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.235+0000 7fda0ffff640 1 --2- 172.21.15.5:0/3037456291 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5588f38cdf00 0x5588f38d0330 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dd7b561222bc35c server_cookie=e65f8d917d141f00 in_seq=0 out_seq=0 2024-09-19T06:15:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fd9f6ffd640 1 -- 172.21.15.5:0/3037456291 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588f3826860 con 0x5588f38cdf00 2024-09-19T06:15:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fd9f6ffd640 10 client.27982 handle_command_reply: tid=0 2024-09-19T06:15:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fda20144740 1 client.27982 shutdown 2024-09-19T06:15:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fda20144740 1 -- 172.21.15.5:0/3037456291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 msgr2=0x5588f38bdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fda20144740 1 --2- 172.21.15.5:0/3037456291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 0x5588f38bdd50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fda080cd830 tx=0x7fda0809bcc0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fda20144740 1 -- 172.21.15.5:0/3037456291 shutdown_connections 2024-09-19T06:15:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fda20144740 1 --2- 172.21.15.5:0/3037456291 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5588f38cdf00 0x5588f38d0330 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fda20144740 1 --2- 172.21.15.5:0/3037456291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38be370 0x5588f38bb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fda20144740 1 --2- 172.21.15.5:0/3037456291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38b4b40 0x5588f38bb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fda20144740 1 --2- 172.21.15.5:0/3037456291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 0x5588f38bdd50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fd9f6ffd640 0 client.27982 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:14.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fda20144740 1 -- 172.21.15.5:0/3037456291 >> 172.21.15.5:0/3037456291 conn(0x5588f387c310 msgr2=0x5588f38b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:14.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fda20144740 1 -- 172.21.15.5:0/3037456291 shutdown_connections 2024-09-19T06:15:14.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fda20144740 1 -- 172.21.15.5:0/3037456291 wait complete. 2024-09-19T06:15:14.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.236+0000 7fda20144740 20 client.27982 trim_cache size 0 max 16384 2024-09-19T06:15:14.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda20144740 1 Processor -- start 2024-09-19T06:15:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda20144740 1 -- start start 2024-09-19T06:15:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda20144740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 0x5588f38ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda20144740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38b4b40 0x5588f38bcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda20144740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38be370 0x5588f38b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda20144740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda181ad910 con 0x5588f38b4b40 2024-09-19T06:15:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda20144740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fda181af510 con 0x5588f38be370 2024-09-19T06:15:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda20144740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda181adfe0 con 0x5588f38b35d0 2024-09-19T06:15:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38b4b40 0x5588f38bcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 0x5588f38ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38b4b40 0x5588f38bcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37448/0 (socket says 172.21.15.5:37448) 2024-09-19T06:15:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda0ffff640 1 -- 172.21.15.5:0/2464387034 learned_addr learned my addr 172.21.15.5:0/2464387034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.254+0000 7fda17fff640 1 --2- 172.21.15.5:0/2464387034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38be370 0x5588f38b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.255+0000 7fda0ffff640 1 -- 172.21.15.5:0/2464387034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 msgr2=0x5588f38ba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.255+0000 7fda0ffff640 1 --2- 172.21.15.5:0/2464387034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 0x5588f38ba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.255+0000 7fda0ffff640 1 -- 172.21.15.5:0/2464387034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38be370 msgr2=0x5588f38b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.255+0000 7fda0ffff640 1 --2- 172.21.15.5:0/2464387034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38be370 0x5588f38b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.255+0000 7fda0ffff640 1 -- 172.21.15.5:0/2464387034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588f38b9fb0 con 0x5588f38b4b40 2024-09-19T06:15:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.255+0000 7fda177fe640 1 --2- 172.21.15.5:0/2464387034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 0x5588f38ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.255+0000 7fda0ffff640 1 --2- 172.21.15.5:0/2464387034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38b4b40 0x5588f38bcf70 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fda000096c0 tx=0x7fda00005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4350ccb2ce1bcebe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.255+0000 7fda0d7fa640 1 -- 172.21.15.5:0/2464387034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00035040 con 0x5588f38b4b40 2024-09-19T06:15:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.255+0000 7fda0d7fa640 1 -- 172.21.15.5:0/2464387034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda0004b070 con 0x5588f38b4b40 2024-09-19T06:15:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.255+0000 7fda0d7fa640 1 -- 172.21.15.5:0/2464387034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda000035f0 con 0x5588f38b4b40 2024-09-19T06:15:14.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.256+0000 7fda20144740 1 -- 172.21.15.5:0/2464387034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38b4b40 msgr2=0x5588f38bcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:14.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.256+0000 7fda20144740 1 --2- 172.21.15.5:0/2464387034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38b4b40 0x5588f38bcf70 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fda000096c0 tx=0x7fda00005a20 comp rx=0 tx=0).stop 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.256+0000 7fda20144740 1 -- 172.21.15.5:0/2464387034 shutdown_connections 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.256+0000 7fda20144740 1 --2- 172.21.15.5:0/2464387034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38be370 0x5588f38b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.256+0000 7fda20144740 1 --2- 172.21.15.5:0/2464387034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38b4b40 0x5588f38bcf70 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.256+0000 7fda20144740 1 --2- 172.21.15.5:0/2464387034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b35d0 0x5588f38ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.256+0000 7fda20144740 1 -- 172.21.15.5:0/2464387034 >> 172.21.15.5:0/2464387034 conn(0x5588f387c310 msgr2=0x5588f38bc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.256+0000 7fda20144740 1 -- 172.21.15.5:0/2464387034 shutdown_connections 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.256+0000 7fda20144740 1 -- 172.21.15.5:0/2464387034 wait complete. 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.256+0000 7fda20144740 1 -- start start 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda20144740 1 Processor -- start 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda20144740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38b35d0 0x5588f38c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda20144740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 0x5588f38c1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda20144740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38be370 0x5588f38c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda20144740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda181aee40 con 0x5588f38b35d0 2024-09-19T06:15:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda20144740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fda181ae710 con 0x5588f38be370 2024-09-19T06:15:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda20144740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda181ab910 con 0x5588f38b4b40 2024-09-19T06:15:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda20144740 10 client.0 _ll_register_callbacks cb 0x7fda0800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 0x5588f38c1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 0x5588f38c1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45380/0 (socket says 172.21.15.5:45380) 2024-09-19T06:15:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda0ffff640 1 -- 172.21.15.5:0/4194067923 learned_addr learned my addr 172.21.15.5:0/4194067923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda177fe640 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38b35d0 0x5588f38c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.257+0000 7fda17fff640 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38be370 0x5588f38c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fda0ffff640 1 -- 172.21.15.5:0/4194067923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38be370 msgr2=0x5588f38c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fda0ffff640 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38be370 0x5588f38c2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fda0ffff640 1 -- 172.21.15.5:0/4194067923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38b35d0 msgr2=0x5588f38c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fda0ffff640 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38b35d0 0x5588f38c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fda0ffff640 1 -- 172.21.15.5:0/4194067923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588f38c2a20 con 0x5588f38b4b40 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fda17fff640 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38be370 0x5588f38c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fda0ffff640 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 0x5588f38c1df0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fda0002a9a0 tx=0x7fda00005900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f86b0eb697a0557 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fd9f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fd9f6ffd640 1 -- 172.21.15.5:0/4194067923 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00035050 con 0x5588f38b4b40 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fd9f6ffd640 1 -- 172.21.15.5:0/4194067923 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fda00004dc0 con 0x5588f38b4b40 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fda20144740 1 -- 172.21.15.5:0/4194067923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda0004b070 con 0x5588f38b4b40 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fd9f6ffd640 1 -- 172.21.15.5:0/4194067923 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fda0005e050 con 0x5588f38b4b40 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fda20144740 10 client.27988 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.258+0000 7fda20144740 1 -- 172.21.15.5:0/4194067923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5588f393cec0 con 0x5588f38b4b40 2024-09-19T06:15:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.259+0000 7fd9f6ffd640 1 -- 172.21.15.5:0/4194067923 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda0004b070 con 0x5588f38b4b40 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.259+0000 7fd9f6ffd640 1 -- 172.21.15.5:0/4194067923 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fda00003ab0 con 0x5588f38b4b40 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.259+0000 7fda20144740 10 client.27988 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.259+0000 7fda20144740 10 client.27988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.259+0000 7fda20144740 20 client.27988 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.259+0000 7fda20144740 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5588f393d670 0x5588f393fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.259+0000 7fda20144740 4 client.27988 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.259+0000 7fda20144740 1 -- 172.21.15.5:0/4194067923 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588f3820d70 con 0x5588f393d670 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.259+0000 7fda177fe640 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5588f393d670 0x5588f393fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.259+0000 7fd9f6ffd640 10 client.27988 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.259+0000 7fda177fe640 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5588f393d670 0x5588f393fa90 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db1daed987172e9 server_cookie=ff23b6dd06ea0c60 in_seq=0 out_seq=0 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fd9f6ffd640 1 -- 172.21.15.5:0/4194067923 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588f3820d70 con 0x5588f393d670 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fd9f6ffd640 10 client.27988 handle_command_reply: tid=0 2024-09-19T06:15:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fda20144740 1 client.27988 shutdown 2024-09-19T06:15:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fda20144740 1 -- 172.21.15.5:0/4194067923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 msgr2=0x5588f38c1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fda20144740 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 0x5588f38c1df0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fda0002a9a0 tx=0x7fda00005900 comp rx=0 tx=0).stop 2024-09-19T06:15:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fda20144740 1 -- 172.21.15.5:0/4194067923 shutdown_connections 2024-09-19T06:15:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fda20144740 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5588f393d670 0x5588f393fa90 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fda20144740 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588f38be370 0x5588f38c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fda20144740 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f38b4b40 0x5588f38c1df0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fda20144740 1 --2- 172.21.15.5:0/4194067923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588f38b35d0 0x5588f38c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fda20144740 1 -- 172.21.15.5:0/4194067923 >> 172.21.15.5:0/4194067923 conn(0x5588f387c310 msgr2=0x5588f38bd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fd9f6ffd640 0 client.27988 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fda20144740 1 -- 172.21.15.5:0/4194067923 shutdown_connections 2024-09-19T06:15:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fda20144740 1 -- 172.21.15.5:0/4194067923 wait complete. 2024-09-19T06:15:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.260+0000 7fda20144740 20 client.27988 trim_cache size 0 max 16384 2024-09-19T06:15:14.268 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:15:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.261+0000 7fda1dd6b640 1 -- 172.21.15.5:0/3867195222 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fda040a3430 msgr2=0x7fda040a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.261+0000 7fda1dd6b640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fda040a3430 0x7fda040a58f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda10023000 tx=0x7fda10024000 comp rx=0 tx=0).stop 2024-09-19T06:15:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.261+0000 7fda1dd6b640 1 -- 172.21.15.5:0/3867195222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18102d40 msgr2=0x7fda181984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.261+0000 7fda1dd6b640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18102d40 0x7fda181984b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fda0800e9f0 tx=0x7fda0800edf0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.261+0000 7fda1dd6b640 1 -- 172.21.15.5:0/3867195222 shutdown_connections 2024-09-19T06:15:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.261+0000 7fda1dd6b640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda1810c170 0x7fda1819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.261+0000 7fda1dd6b640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fda040a3430 0x7fda040a58f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.261+0000 7fda1dd6b640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda18103660 0x7fda181989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.261+0000 7fda1dd6b640 1 --2- 172.21.15.5:0/3867195222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18102d40 0x7fda181984b0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.261+0000 7fda1dd6b640 1 -- 172.21.15.5:0/3867195222 >> 172.21.15.5:0/3867195222 conn(0x7fda180fc7a0 msgr2=0x7fda180fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.261+0000 7fda1dd6b640 1 -- 172.21.15.5:0/3867195222 shutdown_connections 2024-09-19T06:15:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:14.261+0000 7fda1dd6b640 1 -- 172.21.15.5:0/3867195222 wait complete. 2024-09-19T06:15:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:14 smithi157 ceph-mon[35346]: pgmap v372: 97 pgs: 97 active+clean; 152 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 572 KiB/s wr, 68 op/s 2024-09-19T06:15:14.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 275190 2.20 MB/sec execute 520 sec latency 22.465 ms 2024-09-19T06:15:14.874 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:14 smithi005 ceph-mon[30955]: pgmap v372: 97 pgs: 97 active+clean; 152 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 572 KiB/s wr, 68 op/s 2024-09-19T06:15:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:14 smithi049 ceph-mon[34640]: pgmap v372: 97 pgs: 97 active+clean; 152 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 572 KiB/s wr, 68 op/s 2024-09-19T06:15:15.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 275655 2.20 MB/sec execute 521 sec latency 26.317 ms 2024-09-19T06:15:15.970 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:15:15.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:15:16.253 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:16.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:16 smithi005 ceph-mon[30955]: pgmap v373: 97 pgs: 97 active+clean; 152 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 476 KiB/s wr, 56 op/s 2024-09-19T06:15:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.763+0000 7f8697bb4640 1 -- 172.21.15.5:0/1848817197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86901036e0 msgr2=0x7f8690107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.763+0000 7f8697bb4640 1 --2- 172.21.15.5:0/1848817197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86901036e0 0x7f8690107b90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f8680009930 tx=0x7f868002a420 comp rx=0 tx=0).stop 2024-09-19T06:15:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.763+0000 7f8697bb4640 1 -- 172.21.15.5:0/1848817197 shutdown_connections 2024-09-19T06:15:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.763+0000 7f8697bb4640 1 --2- 172.21.15.5:0/1848817197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86901082a0 0x7f869010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.763+0000 7f8697bb4640 1 --2- 172.21.15.5:0/1848817197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86901036e0 0x7f8690107b90 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.763+0000 7f8697bb4640 1 --2- 172.21.15.5:0/1848817197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8690102d30 0x7f8690103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.763+0000 7f8697bb4640 1 -- 172.21.15.5:0/1848817197 >> 172.21.15.5:0/1848817197 conn(0x7f86900fc7d0 msgr2=0x7f86900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.763+0000 7f8697bb4640 1 -- 172.21.15.5:0/1848817197 shutdown_connections 2024-09-19T06:15:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.763+0000 7f8697bb4640 1 -- 172.21.15.5:0/1848817197 wait complete. 2024-09-19T06:15:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8697bb4640 1 Processor -- start 2024-09-19T06:15:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8697bb4640 1 -- start start 2024-09-19T06:15:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8697bb4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8690102d30 0x7f8690198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8697bb4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86901036e0 0x7f8690198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8697bb4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86901082a0 0x7f869019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8697bb4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8690111f50 con 0x7f86901082a0 2024-09-19T06:15:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8697bb4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8690111dd0 con 0x7f86901036e0 2024-09-19T06:15:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8697bb4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86901120d0 con 0x7f8690102d30 2024-09-19T06:15:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8696151640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86901082a0 0x7f869019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8695950640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8690102d30 0x7f8690198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8695950640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8690102d30 0x7f8690198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55288/0 (socket says 172.21.15.5:55288) 2024-09-19T06:15:16.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8696151640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86901082a0 0x7f869019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35830/0 (socket says 172.21.15.5:35830) 2024-09-19T06:15:16.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8695950640 1 -- 172.21.15.5:0/564466074 learned_addr learned my addr 172.21.15.5:0/564466074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f869514f640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86901036e0 0x7f8690198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8695950640 1 -- 172.21.15.5:0/564466074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86901036e0 msgr2=0x7f8690198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8695950640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86901036e0 0x7f8690198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8695950640 1 -- 172.21.15.5:0/564466074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86901082a0 msgr2=0x7f869019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8695950640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86901082a0 0x7f869019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8695950640 1 -- 172.21.15.5:0/564466074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869019d4b0 con 0x7f8690102d30 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.764+0000 7f8696151640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86901082a0 0x7f869019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.765+0000 7f8695950640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8690102d30 0x7f8690198500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f868c00e9f0 tx=0x7f868c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.765+0000 7f8686ffd640 1 -- 172.21.15.5:0/564466074 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c00ccc0 con 0x7f8690102d30 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.765+0000 7f8697bb4640 1 -- 172.21.15.5:0/564466074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869019d7a0 con 0x7f8690102d30 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.765+0000 7f8686ffd640 1 -- 172.21.15.5:0/564466074 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f868c0108e0 con 0x7f8690102d30 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.765+0000 7f8686ffd640 1 -- 172.21.15.5:0/564466074 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c0187c0 con 0x7f8690102d30 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.765+0000 7f8697bb4640 1 -- 172.21.15.5:0/564466074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86901a5080 con 0x7f8690102d30 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.767+0000 7f8686ffd640 1 -- 172.21.15.5:0/564466074 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f868c010a90 con 0x7f8690102d30 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.768+0000 7f8699f8d740 1 Processor -- start 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.768+0000 7f8699f8d740 1 -- start start 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.768+0000 7f8699f8d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa73b60 0x5637baa73f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.768+0000 7f8699f8d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa74480 0x5637baa7e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.768+0000 7f8699f8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa81130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.768+0000 7f8699f8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5637ba9d9280 con 0x5637baa7ed40 2024-09-19T06:15:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.768+0000 7f8699f8d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5637ba980b10 con 0x5637baa74480 2024-09-19T06:15:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.768+0000 7f8699f8d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5637ba9d5240 con 0x5637baa73b60 2024-09-19T06:15:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.768+0000 7f8696151640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa73b60 0x5637baa73f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.768+0000 7f8696151640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa73b60 0x5637baa73f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55294/0 (socket says 172.21.15.5:55294) 2024-09-19T06:15:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.768+0000 7f8696151640 1 -- 172.21.15.5:0/1793294860 learned_addr learned my addr 172.21.15.5:0/1793294860 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.768+0000 7f869514f640 1 --2- 172.21.15.5:0/1793294860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa81130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f8686ffd640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f867c0a33b0 0x7f867c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f8695950640 1 --2- 172.21.15.5:0/1793294860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa74480 0x5637baa7e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f869514f640 1 -- 172.21.15.5:0/1793294860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa73b60 msgr2=0x5637baa73f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f869514f640 1 --2- 172.21.15.5:0/1793294860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa73b60 0x5637baa73f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f869514f640 1 -- 172.21.15.5:0/1793294860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa74480 msgr2=0x5637baa7e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f869514f640 1 --2- 172.21.15.5:0/1793294860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa74480 0x5637baa7e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f869514f640 1 -- 172.21.15.5:0/1793294860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ba98d1f0 con 0x5637baa7ed40 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f8695950640 1 --2- 172.21.15.5:0/1793294860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa74480 0x5637baa7e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f8696151640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f867c0a33b0 0x7f867c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f8686ffd640 1 -- 172.21.15.5:0/564466074 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f868c0d0140 con 0x7f8690102d30 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f869514f640 1 --2- 172.21.15.5:0/1793294860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa81130 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f868002a940 tx=0x7f8680005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d46af444237426 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f867bfff640 1 -- 172.21.15.5:0/1793294860 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680040070 con 0x5637baa7ed40 2024-09-19T06:15:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f867bfff640 1 -- 172.21.15.5:0/1793294860 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8680004030 con 0x5637baa7ed40 2024-09-19T06:15:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.769+0000 7f867bfff640 1 -- 172.21.15.5:0/1793294860 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680035040 con 0x5637baa7ed40 2024-09-19T06:15:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.770+0000 7f8699f8d740 1 -- 172.21.15.5:0/1793294860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 msgr2=0x5637baa81130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.770+0000 7f8699f8d740 1 --2- 172.21.15.5:0/1793294860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa81130 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f868002a940 tx=0x7f8680005c00 comp rx=0 tx=0).stop 2024-09-19T06:15:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.770+0000 7f8696151640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f867c0a33b0 0x7f867c0a5870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f86880119b0 tx=0x7f8688004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.770+0000 7f8699f8d740 1 -- 172.21.15.5:0/1793294860 shutdown_connections 2024-09-19T06:15:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.770+0000 7f8699f8d740 1 --2- 172.21.15.5:0/1793294860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa81130 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.770+0000 7f8699f8d740 1 --2- 172.21.15.5:0/1793294860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa74480 0x5637baa7e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.770+0000 7f8699f8d740 1 --2- 172.21.15.5:0/1793294860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa73b60 0x5637baa73f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.770+0000 7f8699f8d740 1 -- 172.21.15.5:0/1793294860 >> 172.21.15.5:0/1793294860 conn(0x5637baa700f0 msgr2=0x5637baa704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.770+0000 7f8699f8d740 1 -- 172.21.15.5:0/1793294860 shutdown_connections 2024-09-19T06:15:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.770+0000 7f8699f8d740 1 -- 172.21.15.5:0/1793294860 wait complete. 2024-09-19T06:15:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8699f8d740 1 -- start start 2024-09-19T06:15:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8699f8d740 1 Processor -- start 2024-09-19T06:15:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8699f8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa73b60 0x5637baa7bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8699f8d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa7c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8699f8d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa7ed40 0x5637baa7c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8699f8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5637baa85d70 con 0x5637baa73b60 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8699f8d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5637ba993f10 con 0x5637baa7ed40 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8699f8d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5637baa85ef0 con 0x5637baa74480 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8699f8d740 10 client.0 _ll_register_callbacks cb 0x5637ba95ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8695950640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa73b60 0x5637baa7bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8695950640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa73b60 0x5637baa7bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35854/0 (socket says 172.21.15.5:35854) 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8695950640 1 -- 172.21.15.5:0/1986301648 learned_addr learned my addr 172.21.15.5:0/1986301648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f8696151640 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa7ed40 0x5637baa7c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.771+0000 7f869514f640 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa7c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f8695950640 1 -- 172.21.15.5:0/1986301648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 msgr2=0x5637baa7c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f8695950640 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa7c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f8695950640 1 -- 172.21.15.5:0/1986301648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa7ed40 msgr2=0x5637baa7c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f8695950640 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa7ed40 0x5637baa7c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f8695950640 1 -- 172.21.15.5:0/1986301648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ba74a6a0 con 0x5637baa73b60 2024-09-19T06:15:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f8695950640 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa73b60 0x5637baa7bbf0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f868c0a4fe0 tx=0x7f868c010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b6b3e170df2937 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f86797fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f86797fa640 1 -- 172.21.15.5:0/1986301648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c010630 con 0x5637baa73b60 2024-09-19T06:15:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f86797fa640 1 -- 172.21.15.5:0/1986301648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f868c09b850 con 0x5637baa73b60 2024-09-19T06:15:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f8699f8d740 1 -- 172.21.15.5:0/1986301648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8680004030 con 0x5637baa73b60 2024-09-19T06:15:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f86797fa640 1 -- 172.21.15.5:0/1986301648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f868c09c400 con 0x5637baa73b60 2024-09-19T06:15:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f8699f8d740 10 client.18585 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f8699f8d740 1 -- 172.21.15.5:0/1986301648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5637baa91940 con 0x5637baa73b60 2024-09-19T06:15:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.772+0000 7f86797fa640 1 -- 172.21.15.5:0/1986301648 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8680004030 con 0x5637baa73b60 2024-09-19T06:15:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.773+0000 7f86797fa640 1 -- 172.21.15.5:0/1986301648 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f868c0981e0 con 0x5637baa73b60 2024-09-19T06:15:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.773+0000 7f8699f8d740 10 client.18585 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.773+0000 7f8699f8d740 10 client.18585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.773+0000 7f8699f8d740 20 client.18585 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.773+0000 7f8699f8d740 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5637baa92e00 0x5637baa95210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.773+0000 7f8699f8d740 4 client.18585 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.773+0000 7f8699f8d740 1 -- 172.21.15.5:0/1986301648 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637ba9e67f0 con 0x5637baa92e00 2024-09-19T06:15:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.773+0000 7f869514f640 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5637baa92e00 0x5637baa95210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.773+0000 7f869514f640 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5637baa92e00 0x5637baa95210 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6743bd4183a85ce server_cookie=4603fa37c63208f5 in_seq=0 out_seq=0 2024-09-19T06:15:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.773+0000 7f86797fa640 10 client.18585 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f86797fa640 1 -- 172.21.15.5:0/1986301648 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637ba9e67f0 con 0x5637baa92e00 2024-09-19T06:15:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f86797fa640 10 client.18585 handle_command_reply: tid=0 2024-09-19T06:15:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f8699f8d740 1 client.18585 shutdown 2024-09-19T06:15:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f8699f8d740 1 -- 172.21.15.5:0/1986301648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa73b60 msgr2=0x5637baa7bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f8699f8d740 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa73b60 0x5637baa7bbf0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f868c0a4fe0 tx=0x7f868c010e00 comp rx=0 tx=0).stop 2024-09-19T06:15:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f8699f8d740 1 -- 172.21.15.5:0/1986301648 shutdown_connections 2024-09-19T06:15:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f8699f8d740 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5637baa92e00 0x5637baa95210 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f8699f8d740 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa7ed40 0x5637baa7c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f8699f8d740 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa7c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f8699f8d740 1 --2- 172.21.15.5:0/1986301648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa73b60 0x5637baa7bbf0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f8699f8d740 1 -- 172.21.15.5:0/1986301648 >> 172.21.15.5:0/1986301648 conn(0x5637baa700f0 msgr2=0x5637baa75dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f86797fa640 0 client.18585 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f8699f8d740 1 -- 172.21.15.5:0/1986301648 shutdown_connections 2024-09-19T06:15:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f8699f8d740 1 -- 172.21.15.5:0/1986301648 wait complete. 2024-09-19T06:15:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.774+0000 7f8699f8d740 20 client.18585 trim_cache size 0 max 16384 2024-09-19T06:15:16.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.793+0000 7f8699f8d740 1 Processor -- start 2024-09-19T06:15:16.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.793+0000 7f8699f8d740 1 -- start start 2024-09-19T06:15:16.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f8699f8d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 0x5637baa75cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f8699f8d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa9aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f8699f8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa9b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f8699f8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86901ab670 con 0x5637baa7ed40 2024-09-19T06:15:16.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f8699f8d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f86901ad270 con 0x5637baa73b60 2024-09-19T06:15:16.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f8699f8d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86901abd40 con 0x5637baa74480 2024-09-19T06:15:16.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f8695950640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 0x5637baa75cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f8696151640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa9b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f8695950640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 0x5637baa75cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60764/0 (socket says 172.21.15.5:60764) 2024-09-19T06:15:16.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f8695950640 1 -- 172.21.15.5:0/434553694 learned_addr learned my addr 172.21.15.5:0/434553694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:16.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f869514f640 1 --2- 172.21.15.5:0/434553694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa9aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f869514f640 1 -- 172.21.15.5:0/434553694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 msgr2=0x5637baa75cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:16.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f869514f640 1 --2- 172.21.15.5:0/434553694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 0x5637baa75cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f869514f640 1 -- 172.21.15.5:0/434553694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 msgr2=0x5637baa9b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f869514f640 1 --2- 172.21.15.5:0/434553694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa9b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f869514f640 1 -- 172.21.15.5:0/434553694 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637baa71280 con 0x5637baa74480 2024-09-19T06:15:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f8696151640 1 --2- 172.21.15.5:0/434553694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa9b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.794+0000 7f8695950640 1 --2- 172.21.15.5:0/434553694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 0x5637baa75cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.795+0000 7f869514f640 1 --2- 172.21.15.5:0/434553694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa9aae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f86800092a0 tx=0x7f8680047040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6954bd59cf653ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.795+0000 7f867b7fe640 1 -- 172.21.15.5:0/434553694 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680040040 con 0x5637baa74480 2024-09-19T06:15:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.795+0000 7f867b7fe640 1 -- 172.21.15.5:0/434553694 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86800318d0 con 0x5637baa74480 2024-09-19T06:15:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.795+0000 7f867b7fe640 1 -- 172.21.15.5:0/434553694 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680035040 con 0x5637baa74480 2024-09-19T06:15:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.795+0000 7f8699f8d740 1 -- 172.21.15.5:0/434553694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 msgr2=0x5637baa9aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.795+0000 7f8699f8d740 1 --2- 172.21.15.5:0/434553694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa9aae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f86800092a0 tx=0x7f8680047040 comp rx=0 tx=0).stop 2024-09-19T06:15:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 -- 172.21.15.5:0/434553694 shutdown_connections 2024-09-19T06:15:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 --2- 172.21.15.5:0/434553694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa9b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 --2- 172.21.15.5:0/434553694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa9aae0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 --2- 172.21.15.5:0/434553694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 0x5637baa75cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 -- 172.21.15.5:0/434553694 >> 172.21.15.5:0/434553694 conn(0x5637baa700f0 msgr2=0x5637baa755b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 -- 172.21.15.5:0/434553694 shutdown_connections 2024-09-19T06:15:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 -- 172.21.15.5:0/434553694 wait complete. 2024-09-19T06:15:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 -- start start 2024-09-19T06:15:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 Processor -- start 2024-09-19T06:15:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 0x5637baa8ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa78a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa79170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86901acba0 con 0x5637baa7ed40 2024-09-19T06:15:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f86901ac470 con 0x5637baa73b60 2024-09-19T06:15:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86901a9650 con 0x5637baa74480 2024-09-19T06:15:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.796+0000 7f8699f8d740 10 client.0 _ll_register_callbacks cb 0x7f868c09bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8696151640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa79170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8695950640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 0x5637baa8ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8696151640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa79170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35866/0 (socket says 172.21.15.5:35866) 2024-09-19T06:15:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8695950640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 0x5637baa8ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60780/0 (socket says 172.21.15.5:60780) 2024-09-19T06:15:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8696151640 1 -- 172.21.15.5:0/451629903 learned_addr learned my addr 172.21.15.5:0/451629903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f869514f640 1 --2- 172.21.15.5:0/451629903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa78a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8695950640 1 -- 172.21.15.5:0/451629903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 msgr2=0x5637baa78a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8695950640 1 --2- 172.21.15.5:0/451629903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa78a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8695950640 1 -- 172.21.15.5:0/451629903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 msgr2=0x5637baa79170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8695950640 1 --2- 172.21.15.5:0/451629903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa79170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8695950640 1 -- 172.21.15.5:0/451629903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637baafcab0 con 0x5637baa73b60 2024-09-19T06:15:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8695950640 1 --2- 172.21.15.5:0/451629903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 0x5637baa8ba00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f868c098860 tx=0x7f868c098400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d745fe3f8b596a29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f86797fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:15:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f86797fa640 1 -- 172.21.15.5:0/451629903 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c0a07b0 con 0x5637baa73b60 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8699f8d740 1 -- 172.21.15.5:0/451629903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86800318d0 con 0x5637baa73b60 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f86797fa640 1 -- 172.21.15.5:0/451629903 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f868c010640 con 0x5637baa73b60 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f86797fa640 1 -- 172.21.15.5:0/451629903 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f868c0b5210 con 0x5637baa73b60 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8699f8d740 10 client.28058 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.797+0000 7f8699f8d740 1 -- 172.21.15.5:0/451629903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5637baafdc30 con 0x5637baa73b60 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.798+0000 7f86797fa640 1 -- 172.21.15.5:0/451629903 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86800318d0 con 0x5637baa73b60 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.798+0000 7f86797fa640 1 -- 172.21.15.5:0/451629903 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f868c0bc0b0 con 0x5637baa73b60 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.798+0000 7f8699f8d740 10 client.28058 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.798+0000 7f8699f8d740 10 client.28058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.798+0000 7f8699f8d740 20 client.28058 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.798+0000 7f8699f8d740 1 --2- 172.21.15.5:0/451629903 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5637baafe1d0 0x5637bab005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.798+0000 7f8699f8d740 4 client.28058 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:15:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.798+0000 7f8699f8d740 1 -- 172.21.15.5:0/451629903 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5637ba9e0d00 con 0x5637baafe1d0 2024-09-19T06:15:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.798+0000 7f869514f640 1 --2- 172.21.15.5:0/451629903 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5637baafe1d0 0x5637bab005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.798+0000 7f869514f640 1 --2- 172.21.15.5:0/451629903 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5637baafe1d0 0x5637bab005f0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8986555d7ee6ce server_cookie=a97fb09fd64d67a8 in_seq=0 out_seq=0 2024-09-19T06:15:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.798+0000 7f86797fa640 10 client.28058 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.799+0000 7f86797fa640 1 -- 172.21.15.5:0/451629903 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5637ba9e0d00 con 0x5637baafe1d0 2024-09-19T06:15:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.799+0000 7f86797fa640 10 client.28058 handle_command_reply: tid=0 2024-09-19T06:15:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.799+0000 7f8699f8d740 1 client.28058 shutdown 2024-09-19T06:15:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.799+0000 7f8699f8d740 1 -- 172.21.15.5:0/451629903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 msgr2=0x5637baa8ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.799+0000 7f8699f8d740 1 --2- 172.21.15.5:0/451629903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 0x5637baa8ba00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f868c098860 tx=0x7f868c098400 comp rx=0 tx=0).stop 2024-09-19T06:15:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.800+0000 7f8699f8d740 1 -- 172.21.15.5:0/451629903 shutdown_connections 2024-09-19T06:15:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.800+0000 7f8699f8d740 1 --2- 172.21.15.5:0/451629903 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5637baafe1d0 0x5637bab005f0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.800+0000 7f8699f8d740 1 --2- 172.21.15.5:0/451629903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637baa7ed40 0x5637baa79170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.800+0000 7f8699f8d740 1 --2- 172.21.15.5:0/451629903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637baa74480 0x5637baa78a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.800+0000 7f86797fa640 0 client.28058 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.800+0000 7f8699f8d740 1 --2- 172.21.15.5:0/451629903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637baa73b60 0x5637baa8ba00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.800+0000 7f8699f8d740 1 -- 172.21.15.5:0/451629903 >> 172.21.15.5:0/451629903 conn(0x5637baa700f0 msgr2=0x5637baa6f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.800+0000 7f8699f8d740 1 -- 172.21.15.5:0/451629903 shutdown_connections 2024-09-19T06:15:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.800+0000 7f8699f8d740 1 -- 172.21.15.5:0/451629903 wait complete. 2024-09-19T06:15:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.800+0000 7f8699f8d740 20 client.28058 trim_cache size 0 max 16384 2024-09-19T06:15:16.808 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:15:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:15:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "5a982fb2-efea-4095-a0cc-d138ae65c1b6", 2024-09-19T06:15:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:15:16.809 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:15:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.801+0000 7f8697bb4640 1 -- 172.21.15.5:0/564466074 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f867c0a33b0 msgr2=0x7f867c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.801+0000 7f8697bb4640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f867c0a33b0 0x7f867c0a5870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f86880119b0 tx=0x7f8688004a40 comp rx=0 tx=0).stop 2024-09-19T06:15:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.801+0000 7f8697bb4640 1 -- 172.21.15.5:0/564466074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8690102d30 msgr2=0x7f8690198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.801+0000 7f8697bb4640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8690102d30 0x7f8690198500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f868c00e9f0 tx=0x7f868c00edf0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.801+0000 7f8697bb4640 1 -- 172.21.15.5:0/564466074 shutdown_connections 2024-09-19T06:15:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.801+0000 7f8697bb4640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86901082a0 0x7f869019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.801+0000 7f8697bb4640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86901036e0 0x7f8690198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.801+0000 7f8697bb4640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f867c0a33b0 0x7f867c0a5870 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.801+0000 7f8697bb4640 1 --2- 172.21.15.5:0/564466074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8690102d30 0x7f8690198500 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.801+0000 7f8697bb4640 1 -- 172.21.15.5:0/564466074 >> 172.21.15.5:0/564466074 conn(0x7f86900fc7d0 msgr2=0x7f8690105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.801+0000 7f8697bb4640 1 -- 172.21.15.5:0/564466074 shutdown_connections 2024-09-19T06:15:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:16.801+0000 7f8697bb4640 1 -- 172.21.15.5:0/564466074 wait complete. 2024-09-19T06:15:16.812 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:16 smithi157 ceph-mon[35346]: pgmap v373: 97 pgs: 97 active+clean; 152 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 476 KiB/s wr, 56 op/s 2024-09-19T06:15:16.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 276414 2.21 MB/sec execute 522 sec latency 70.485 ms 2024-09-19T06:15:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:16 smithi049 ceph-mon[34640]: pgmap v373: 97 pgs: 97 active+clean; 152 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 476 KiB/s wr, 56 op/s 2024-09-19T06:15:17.490 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:15:17.764 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:17.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:17 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:15:17.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:17 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:17.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:17 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:15:17.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:17 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:17.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 276766 2.21 MB/sec execute 523 sec latency 56.330 ms 2024-09-19T06:15:17.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:17 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:15:17.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:17 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:17.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:17 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:15:17.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:17 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:17.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:17 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:15:17.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:17 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:17.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:17 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:15:17.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:17 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: pgmap v374: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-09-19T06:15:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:15:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:15:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:15:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:15:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:15:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:15:18.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:18 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 276766 2.21 MB/sec execute 524 sec latency 1056.425 ms 2024-09-19T06:15:18.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: pgmap v374: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-09-19T06:15:18.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:15:18.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:15:18.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:15:18.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.861 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.862 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:15:18.862 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:15:18.862 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.862 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.862 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:15:18.862 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:18 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: pgmap v374: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-09-19T06:15:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:15:18.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:15:18.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:15:18.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:15:18.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:15:18.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:18.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:15:18.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:18 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:15:19.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 276766 2.21 MB/sec execute 525 sec latency 2056.480 ms 2024-09-19T06:15:19.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.916+0000 7fcfc3264640 1 -- 172.21.15.5:0/2417070821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc102790 msgr2=0x7fcfbc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:19.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.916+0000 7fcfc3264640 1 --2- 172.21.15.5:0/2417070821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc102790 0x7fcfbc1067e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009900 tx=0x7fcfb402a410 comp rx=0 tx=0).stop 2024-09-19T06:15:19.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.916+0000 7fcfc3264640 1 -- 172.21.15.5:0/2417070821 shutdown_connections 2024-09-19T06:15:19.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.916+0000 7fcfc3264640 1 --2- 172.21.15.5:0/2417070821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfbc106f10 0x7fcfbc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.916+0000 7fcfc3264640 1 --2- 172.21.15.5:0/2417070821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc102790 0x7fcfbc1067e0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.916+0000 7fcfc3264640 1 --2- 172.21.15.5:0/2417070821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfbc101de0 0x7fcfbc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.916+0000 7fcfc3264640 1 -- 172.21.15.5:0/2417070821 >> 172.21.15.5:0/2417070821 conn(0x7fcfbc0fd8c0 msgr2=0x7fcfbc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:19.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.916+0000 7fcfc3264640 1 -- 172.21.15.5:0/2417070821 shutdown_connections 2024-09-19T06:15:19.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.916+0000 7fcfc3264640 1 -- 172.21.15.5:0/2417070821 wait complete. 2024-09-19T06:15:19.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfc3264640 1 Processor -- start 2024-09-19T06:15:19.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfc3264640 1 -- start start 2024-09-19T06:15:19.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfc3264640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc101de0 0x7fcfbc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfc3264640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfbc102790 0x7fcfbc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfc3264640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfbc106f10 0x7fcfbc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfc3264640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc111e20 con 0x7fcfbc101de0 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfc3264640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc111ca0 con 0x7fcfbc102790 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfc3264640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc111fa0 con 0x7fcfbc106f10 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfc1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc101de0 0x7fcfbc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfc1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc101de0 0x7fcfbc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35892/0 (socket says 172.21.15.5:35892) 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfc1000640 1 -- 172.21.15.5:0/4206293064 learned_addr learned my addr 172.21.15.5:0/4206293064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfc1801640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfbc106f10 0x7fcfbc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.917+0000 7fcfbbfff640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfbc102790 0x7fcfbc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.918+0000 7fcfc1000640 1 -- 172.21.15.5:0/4206293064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfbc106f10 msgr2=0x7fcfbc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.918+0000 7fcfc1000640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfbc106f10 0x7fcfbc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.918+0000 7fcfc1000640 1 -- 172.21.15.5:0/4206293064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfbc102790 msgr2=0x7fcfbc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.918+0000 7fcfc1000640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfbc102790 0x7fcfbc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.918+0000 7fcfc1000640 1 -- 172.21.15.5:0/4206293064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc1a6880 con 0x7fcfbc101de0 2024-09-19T06:15:19.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.918+0000 7fcfc1000640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc101de0 0x7fcfbc0744f0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00ebc0 tx=0x7fcfac00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:19.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.918+0000 7fcfb9ffb640 1 -- 172.21.15.5:0/4206293064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac010630 con 0x7fcfbc101de0 2024-09-19T06:15:19.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.918+0000 7fcfb9ffb640 1 -- 172.21.15.5:0/4206293064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfac010c80 con 0x7fcfbc101de0 2024-09-19T06:15:19.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.918+0000 7fcfc3264640 1 -- 172.21.15.5:0/4206293064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfbc1a6b70 con 0x7fcfbc101de0 2024-09-19T06:15:19.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.918+0000 7fcfb9ffb640 1 -- 172.21.15.5:0/4206293064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac018b30 con 0x7fcfbc101de0 2024-09-19T06:15:19.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.918+0000 7fcfc3264640 1 -- 172.21.15.5:0/4206293064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfbc1a70d0 con 0x7fcfbc101de0 2024-09-19T06:15:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.921+0000 7fcfb9ffb640 1 -- 172.21.15.5:0/4206293064 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcfac018cd0 con 0x7fcfbc101de0 2024-09-19T06:15:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.921+0000 7fcfc563d740 1 Processor -- start 2024-09-19T06:15:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.921+0000 7fcfc563d740 1 -- start start 2024-09-19T06:15:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.921+0000 7fcfc563d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 0x55ea0e07f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.921+0000 7fcfc563d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e08a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.921+0000 7fcfc563d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 0x55ea0e08ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.921+0000 7fcfc563d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea0dfe52e0 con 0x55ea0e07f4a0 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.921+0000 7fcfc563d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ea0df8cce0 con 0x55ea0e08a680 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.921+0000 7fcfc563d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ea0dfe12a0 con 0x55ea0e07fdc0 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.921+0000 7fcfb9ffb640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcfa80a3430 0x7fcfa80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.921+0000 7fcfc1801640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 0x55ea0e07f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfc1801640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 0x55ea0e07f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35906/0 (socket says 172.21.15.5:35906) 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfc1000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e08a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfc1801640 1 -- 172.21.15.5:0/2719431477 learned_addr learned my addr 172.21.15.5:0/2719431477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfc1801640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcfa80a3430 0x7fcfa80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfb9ffb640 1 -- 172.21.15.5:0/4206293064 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fcfac0d0270 con 0x7fcfbc101de0 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfbbfff640 1 --2- 172.21.15.5:0/2719431477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 0x55ea0e08ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfc1801640 1 -- 172.21.15.5:0/2719431477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 msgr2=0x55ea0e08a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfc1801640 1 --2- 172.21.15.5:0/2719431477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e08a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfc1801640 1 -- 172.21.15.5:0/2719431477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 msgr2=0x55ea0e08ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfc1801640 1 --2- 172.21.15.5:0/2719431477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 0x55ea0e08ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfc1801640 1 -- 172.21.15.5:0/2719431477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0df71440 con 0x55ea0e07f4a0 2024-09-19T06:15:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfc1801640 1 --2- 172.21.15.5:0/2719431477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 0x55ea0e07f880 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40129f0 tx=0x7fcfa4012df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1716f9af7266fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfc1801640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcfa80a3430 0x7fcfa80a58f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4010740 tx=0x7fcfa4017000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfb2ffd640 1 -- 172.21.15.5:0/2719431477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4024070 con 0x55ea0e07f4a0 2024-09-19T06:15:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfb2ffd640 1 -- 172.21.15.5:0/2719431477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfa400b9c0 con 0x55ea0e07f4a0 2024-09-19T06:15:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.922+0000 7fcfb2ffd640 1 -- 172.21.15.5:0/2719431477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4004b00 con 0x55ea0e07f4a0 2024-09-19T06:15:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.923+0000 7fcfc563d740 1 -- 172.21.15.5:0/2719431477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 msgr2=0x55ea0e07f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.923+0000 7fcfc563d740 1 --2- 172.21.15.5:0/2719431477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 0x55ea0e07f880 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40129f0 tx=0x7fcfa4012df0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.923+0000 7fcfc563d740 1 -- 172.21.15.5:0/2719431477 shutdown_connections 2024-09-19T06:15:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.923+0000 7fcfc563d740 1 --2- 172.21.15.5:0/2719431477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 0x55ea0e08ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.923+0000 7fcfc563d740 1 --2- 172.21.15.5:0/2719431477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e08a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.923+0000 7fcfc563d740 1 --2- 172.21.15.5:0/2719431477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 0x55ea0e07f880 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.923+0000 7fcfc563d740 1 -- 172.21.15.5:0/2719431477 >> 172.21.15.5:0/2719431477 conn(0x55ea0e07ba30 msgr2=0x55ea0e07be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.923+0000 7fcfc563d740 1 -- 172.21.15.5:0/2719431477 shutdown_connections 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.923+0000 7fcfc563d740 1 -- 172.21.15.5:0/2719431477 wait complete. 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfc563d740 1 -- start start 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfc563d740 1 Processor -- start 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfc563d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07f4a0 0x55ea0e089fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfc563d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07fdc0 0x55ea0e087270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfc563d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 0x55ea0e0877b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfc563d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea0e08d250 con 0x55ea0e07fdc0 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfc563d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ea0dfa0050 con 0x55ea0e08a680 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfc563d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ea0e08d3d0 con 0x55ea0e07f4a0 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfc563d740 10 client.0 _ll_register_callbacks cb 0x55ea0dfe0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfbbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07fdc0 0x55ea0e087270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfbbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07fdc0 0x55ea0e087270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35918/0 (socket says 172.21.15.5:35918) 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfbbfff640 1 -- 172.21.15.5:0/1739132922 learned_addr learned my addr 172.21.15.5:0/1739132922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.924+0000 7fcfc1000640 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07f4a0 0x55ea0e089fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcfc1801640 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 0x55ea0e0877b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcfbbfff640 1 -- 172.21.15.5:0/1739132922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07f4a0 msgr2=0x55ea0e089fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcfbbfff640 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07f4a0 0x55ea0e089fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcfbbfff640 1 -- 172.21.15.5:0/1739132922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 msgr2=0x55ea0e0877b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcfbbfff640 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 0x55ea0e0877b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcfbbfff640 1 -- 172.21.15.5:0/1739132922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0dd563c0 con 0x55ea0e07fdc0 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcfc1000640 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07f4a0 0x55ea0e089fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcfbbfff640 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07fdc0 0x55ea0e087270 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009900 tx=0x7fcfb4032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b66709a776af80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcf9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcf9bfff640 1 -- 172.21.15.5:0/1739132922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4035040 con 0x55ea0e07fdc0 2024-09-19T06:15:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcf9bfff640 1 -- 172.21.15.5:0/1739132922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fcfb40317a0 con 0x55ea0e07fdc0 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcfc563d740 1 -- 172.21.15.5:0/1739132922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa400b9c0 con 0x55ea0e07fdc0 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcf9bfff640 1 -- 172.21.15.5:0/1739132922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcfb4004040 con 0x55ea0e07fdc0 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcfc563d740 10 client.18603 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.925+0000 7fcfc563d740 1 -- 172.21.15.5:0/1739132922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ea0e098c50 con 0x55ea0e07fdc0 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.926+0000 7fcf9bfff640 1 -- 172.21.15.5:0/1739132922 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfa400b9c0 con 0x55ea0e07fdc0 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.926+0000 7fcf9bfff640 1 -- 172.21.15.5:0/1739132922 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fcfb4004870 con 0x55ea0e07fdc0 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.926+0000 7fcfc563d740 10 client.18603 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.926+0000 7fcfc563d740 10 client.18603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.926+0000 7fcfc563d740 20 client.18603 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.926+0000 7fcfc563d740 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea0e09a1b0 0x55ea0e09c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.926+0000 7fcfc563d740 4 client.18603 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.926+0000 7fcfc563d740 1 -- 172.21.15.5:0/1739132922 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea0dff2850 con 0x55ea0e09a1b0 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.926+0000 7fcfc1000640 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea0e09a1b0 0x55ea0e09c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.927+0000 7fcfc1000640 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea0e09a1b0 0x55ea0e09c5e0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2a90a3a051c5d66 server_cookie=31b5bf3f5f677d25 in_seq=0 out_seq=0 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.927+0000 7fcf9bfff640 10 client.18603 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.927+0000 7fcf9bfff640 1 -- 172.21.15.5:0/1739132922 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea0dff2850 con 0x55ea0e09a1b0 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.927+0000 7fcf9bfff640 10 client.18603 handle_command_reply: tid=0 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.927+0000 7fcfc563d740 1 client.18603 shutdown 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.927+0000 7fcfc563d740 1 -- 172.21.15.5:0/1739132922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07fdc0 msgr2=0x55ea0e087270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.927+0000 7fcfc563d740 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07fdc0 0x55ea0e087270 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009900 tx=0x7fcfb4032f90 comp rx=0 tx=0).stop 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.928+0000 7fcfc563d740 1 -- 172.21.15.5:0/1739132922 shutdown_connections 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.928+0000 7fcfc563d740 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea0e09a1b0 0x55ea0e09c5e0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.928+0000 7fcf9bfff640 0 client.18603 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.928+0000 7fcfc563d740 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 0x55ea0e0877b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.928+0000 7fcfc563d740 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07fdc0 0x55ea0e087270 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.928+0000 7fcfc563d740 1 --2- 172.21.15.5:0/1739132922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07f4a0 0x55ea0e089fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.928+0000 7fcfc563d740 1 -- 172.21.15.5:0/1739132922 >> 172.21.15.5:0/1739132922 conn(0x55ea0e07ba30 msgr2=0x55ea0e081710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.928+0000 7fcfc563d740 1 -- 172.21.15.5:0/1739132922 shutdown_connections 2024-09-19T06:15:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.928+0000 7fcfc563d740 1 -- 172.21.15.5:0/1739132922 wait complete. 2024-09-19T06:15:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.928+0000 7fcfc563d740 20 client.18603 trim_cache size 0 max 16384 2024-09-19T06:15:19.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.945+0000 7fcfc563d740 1 Processor -- start 2024-09-19T06:15:19.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.945+0000 7fcfc563d740 1 -- start start 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.945+0000 7fcfc563d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e07f4a0 0x55ea0e0964b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.945+0000 7fcfc563d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e0a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.945+0000 7fcfc563d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e08a680 0x55ea0e0a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.945+0000 7fcfc563d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc1af990 con 0x55ea0e08a680 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.945+0000 7fcfc563d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc1b1590 con 0x55ea0e07f4a0 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.945+0000 7fcfc563d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc1b0060 con 0x55ea0e07fdc0 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.945+0000 7fcfc1801640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e08a680 0x55ea0e0a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.945+0000 7fcfc1801640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e08a680 0x55ea0e0a23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35922/0 (socket says 172.21.15.5:35922) 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.945+0000 7fcfc1801640 1 -- 172.21.15.5:0/4053917104 learned_addr learned my addr 172.21.15.5:0/4053917104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfbbfff640 1 --2- 172.21.15.5:0/4053917104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e0a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfc1000640 1 --2- 172.21.15.5:0/4053917104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e07f4a0 0x55ea0e0964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfc1801640 1 -- 172.21.15.5:0/4053917104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 msgr2=0x55ea0e0a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:19.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfc1801640 1 --2- 172.21.15.5:0/4053917104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e0a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfc1801640 1 -- 172.21.15.5:0/4053917104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e07f4a0 msgr2=0x55ea0e0964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:19.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfc1801640 1 --2- 172.21.15.5:0/4053917104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e07f4a0 0x55ea0e0964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfc1801640 1 -- 172.21.15.5:0/4053917104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0e08cdb0 con 0x55ea0e08a680 2024-09-19T06:15:19.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfbbfff640 1 --2- 172.21.15.5:0/4053917104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e0a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:15:19.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfc1000640 1 --2- 172.21.15.5:0/4053917104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e07f4a0 0x55ea0e0964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:15:19.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfc1801640 1 --2- 172.21.15.5:0/4053917104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e08a680 0x55ea0e0a23f0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40126a0 tx=0x7fcfa400b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecf179840f0f15f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:19.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfb27fc640 1 -- 172.21.15.5:0/4053917104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa402c070 con 0x55ea0e08a680 2024-09-19T06:15:19.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfb27fc640 1 -- 172.21.15.5:0/4053917104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfa4005560 con 0x55ea0e08a680 2024-09-19T06:15:19.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.946+0000 7fcfb27fc640 1 -- 172.21.15.5:0/4053917104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa401b040 con 0x55ea0e08a680 2024-09-19T06:15:19.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.947+0000 7fcfc563d740 1 -- 172.21.15.5:0/4053917104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e08a680 msgr2=0x55ea0e0a23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:19.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.947+0000 7fcfc563d740 1 --2- 172.21.15.5:0/4053917104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e08a680 0x55ea0e0a23f0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40126a0 tx=0x7fcfa400b340 comp rx=0 tx=0).stop 2024-09-19T06:15:19.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.947+0000 7fcfc563d740 1 -- 172.21.15.5:0/4053917104 shutdown_connections 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.947+0000 7fcfc563d740 1 --2- 172.21.15.5:0/4053917104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e08a680 0x55ea0e0a23f0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.947+0000 7fcfc563d740 1 --2- 172.21.15.5:0/4053917104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e0a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.947+0000 7fcfc563d740 1 --2- 172.21.15.5:0/4053917104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e07f4a0 0x55ea0e0964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.947+0000 7fcfc563d740 1 -- 172.21.15.5:0/4053917104 >> 172.21.15.5:0/4053917104 conn(0x55ea0e07ba30 msgr2=0x55ea0e086360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.947+0000 7fcfc563d740 1 -- 172.21.15.5:0/4053917104 shutdown_connections 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.947+0000 7fcfc563d740 1 -- 172.21.15.5:0/4053917104 wait complete. 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.947+0000 7fcfc563d740 1 -- start start 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc563d740 1 Processor -- start 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc563d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 0x55ea0e092a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc563d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e092f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc563d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 0x55ea0e08ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc563d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc1b0ec0 con 0x55ea0e07f4a0 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc563d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc1b0790 con 0x55ea0e08a680 2024-09-19T06:15:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc563d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc1ad990 con 0x55ea0e07fdc0 2024-09-19T06:15:19.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc563d740 10 client.0 _ll_register_callbacks cb 0x7fcfb4005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:19.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 0x55ea0e092a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 0x55ea0e092a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35936/0 (socket says 172.21.15.5:35936) 2024-09-19T06:15:19.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc1000640 1 -- 172.21.15.5:0/398994290 learned_addr learned my addr 172.21.15.5:0/398994290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:19.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfbbfff640 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e092f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc1801640 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 0x55ea0e08ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc1000640 1 -- 172.21.15.5:0/398994290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 msgr2=0x55ea0e092f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:19.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc1000640 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e092f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc1000640 1 -- 172.21.15.5:0/398994290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 msgr2=0x55ea0e08ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:19.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc1000640 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 0x55ea0e08ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.948+0000 7fcfc1000640 1 -- 172.21.15.5:0/398994290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0e0907e0 con 0x55ea0e07f4a0 2024-09-19T06:15:19.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.949+0000 7fcfc1000640 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 0x55ea0e092a10 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0cda50 tx=0x7fcfac009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0aaad33c805a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:19.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.949+0000 7fcf9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:19.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.949+0000 7fcf9bfff640 1 -- 172.21.15.5:0/398994290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0bc040 con 0x55ea0e07f4a0 2024-09-19T06:15:19.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.949+0000 7fcf9bfff640 1 -- 172.21.15.5:0/398994290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fcfac098c50 con 0x55ea0e07f4a0 2024-09-19T06:15:19.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.949+0000 7fcfc563d740 1 -- 172.21.15.5:0/398994290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa4005560 con 0x55ea0e07f4a0 2024-09-19T06:15:19.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.949+0000 7fcf9bfff640 1 -- 172.21.15.5:0/398994290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcfac09c5e0 con 0x55ea0e07f4a0 2024-09-19T06:15:19.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.949+0000 7fcfc563d740 10 client.18609 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:19.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.949+0000 7fcfc563d740 1 -- 172.21.15.5:0/398994290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ea0e104b20 con 0x55ea0e07f4a0 2024-09-19T06:15:19.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.949+0000 7fcf9bfff640 1 -- 172.21.15.5:0/398994290 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfa4005560 con 0x55ea0e07f4a0 2024-09-19T06:15:19.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.949+0000 7fcf9bfff640 1 -- 172.21.15.5:0/398994290 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fcfac09ba30 con 0x55ea0e07f4a0 2024-09-19T06:15:19.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.950+0000 7fcfc563d740 10 client.18609 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:19.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.950+0000 7fcfc563d740 10 client.18609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:19.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.950+0000 7fcfc563d740 20 client.18609 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:19.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.950+0000 7fcfc563d740 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea0e09a1b0 0x55ea0e107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:19.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.950+0000 7fcfc563d740 4 client.18609 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:15:19.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.950+0000 7fcfc563d740 1 -- 172.21.15.5:0/398994290 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea0dfecd60 con 0x55ea0e09a1b0 2024-09-19T06:15:19.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.950+0000 7fcfbbfff640 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea0e09a1b0 0x55ea0e107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:19.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.950+0000 7fcfbbfff640 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea0e09a1b0 0x55ea0e107260 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4038eadbb0a37e6c server_cookie=cc39980e8178c1aa in_seq=0 out_seq=0 2024-09-19T06:15:19.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.950+0000 7fcf9bfff640 10 client.18609 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.950+0000 7fcf9bfff640 1 -- 172.21.15.5:0/398994290 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ea0dfecd60 con 0x55ea0e09a1b0 2024-09-19T06:15:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.950+0000 7fcf9bfff640 10 client.18609 handle_command_reply: tid=0 2024-09-19T06:15:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.950+0000 7fcfc563d740 1 client.18609 shutdown 2024-09-19T06:15:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc563d740 1 -- 172.21.15.5:0/398994290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 msgr2=0x55ea0e092a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc563d740 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 0x55ea0e092a10 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0cda50 tx=0x7fcfac009200 comp rx=0 tx=0).stop 2024-09-19T06:15:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc563d740 1 -- 172.21.15.5:0/398994290 shutdown_connections 2024-09-19T06:15:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc563d740 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ea0e09a1b0 0x55ea0e107260 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc563d740 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea0e08a680 0x55ea0e08ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc563d740 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0e07fdc0 0x55ea0e092f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc563d740 1 --2- 172.21.15.5:0/398994290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0e07f4a0 0x55ea0e092a10 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc563d740 1 -- 172.21.15.5:0/398994290 >> 172.21.15.5:0/398994290 conn(0x55ea0e07ba30 msgr2=0x55ea0e088ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:19.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcf9bfff640 0 client.18609 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:19.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc563d740 1 -- 172.21.15.5:0/398994290 shutdown_connections 2024-09-19T06:15:19.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc563d740 1 -- 172.21.15.5:0/398994290 wait complete. 2024-09-19T06:15:19.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc563d740 20 client.18609 trim_cache size 0 max 16384 2024-09-19T06:15:19.960 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:15:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:15:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:15:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "5a982fb2-efea-4095-a0cc-d138ae65c1b6": { 2024-09-19T06:15:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:15:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5a982fb2-efea-4095-a0cc-d138ae65c1b6", 2024-09-19T06:15:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:15:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:15:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:15:19.961 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:15:19.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc3264640 1 -- 172.21.15.5:0/4206293064 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcfa80a3430 msgr2=0x7fcfa80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:19.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc3264640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcfa80a3430 0x7fcfa80a58f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4010740 tx=0x7fcfa4017000 comp rx=0 tx=0).stop 2024-09-19T06:15:19.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc3264640 1 -- 172.21.15.5:0/4206293064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc101de0 msgr2=0x7fcfbc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:19.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.951+0000 7fcfc3264640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc101de0 0x7fcfbc0744f0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00ebc0 tx=0x7fcfac00c6a0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.952+0000 7fcfc3264640 1 -- 172.21.15.5:0/4206293064 shutdown_connections 2024-09-19T06:15:19.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.952+0000 7fcfc3264640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcfa80a3430 0x7fcfa80a58f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.952+0000 7fcfc3264640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfbc106f10 0x7fcfbc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.952+0000 7fcfc3264640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfbc102790 0x7fcfbc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.952+0000 7fcfc3264640 1 --2- 172.21.15.5:0/4206293064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc101de0 0x7fcfbc0744f0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:19.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.952+0000 7fcfc3264640 1 -- 172.21.15.5:0/4206293064 >> 172.21.15.5:0/4206293064 conn(0x7fcfbc0fd8c0 msgr2=0x7fcfbc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:19.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.952+0000 7fcfc3264640 1 -- 172.21.15.5:0/4206293064 shutdown_connections 2024-09-19T06:15:19.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:19.952+0000 7fcfc3264640 1 -- 172.21.15.5:0/4206293064 wait complete. 2024-09-19T06:15:20.618 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:20 smithi005 ceph-mon[30955]: pgmap v375: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1019 KiB/s wr, 97 op/s 2024-09-19T06:15:20.648 INFO:tasks.cephfs.filesystem:scrub status for tag:5a982fb2-efea-4095-a0cc-d138ae65c1b6 - {'path': '/', 'tag': '5a982fb2-efea-4095-a0cc-d138ae65c1b6', 'options': 'recursive,force'} 2024-09-19T06:15:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:20 smithi157 ceph-mon[35346]: pgmap v375: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1019 KiB/s wr, 97 op/s 2024-09-19T06:15:20.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 276833 2.20 MB/sec execute 526 sec latency 2677.222 ms 2024-09-19T06:15:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:20 smithi049 ceph-mon[34640]: pgmap v375: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1019 KiB/s wr, 97 op/s 2024-09-19T06:15:21.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 276833 2.20 MB/sec execute 527 sec latency 1119.087 ms 2024-09-19T06:15:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.251+0000 7f3b03152640 1 -- 172.21.15.5:0/551101866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc10c0f0 msgr2=0x7f3afc10e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.251+0000 7f3b03152640 1 --2- 172.21.15.5:0/551101866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc10c0f0 0x7f3afc10e570 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009ca0 tx=0x7f3af402a950 comp rx=0 tx=0).stop 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.251+0000 7f3b03152640 1 -- 172.21.15.5:0/551101866 shutdown_connections 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.251+0000 7f3b03152640 1 --2- 172.21.15.5:0/551101866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc10c0f0 0x7f3afc10e570 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.251+0000 7f3b03152640 1 --2- 172.21.15.5:0/551101866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc1035c0 0x7f3afc10baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.251+0000 7f3b03152640 1 --2- 172.21.15.5:0/551101866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3afc102ca0 0x7f3afc103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.251+0000 7f3b03152640 1 -- 172.21.15.5:0/551101866 >> 172.21.15.5:0/551101866 conn(0x7f3afc0fc6b0 msgr2=0x7f3afc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.252+0000 7f3b03152640 1 -- 172.21.15.5:0/551101866 shutdown_connections 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.252+0000 7f3b03152640 1 -- 172.21.15.5:0/551101866 wait complete. 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.252+0000 7f3b03152640 1 Processor -- start 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.252+0000 7f3b03152640 1 -- start start 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.252+0000 7f3b03152640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102ca0 0x7f3afc198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.252+0000 7f3b03152640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3afc1035c0 0x7f3afc1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.252+0000 7f3b03152640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc10c0f0 0x7f3afc19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.252+0000 7f3b03152640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc111dd0 con 0x7f3afc10c0f0 2024-09-19T06:15:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.252+0000 7f3b03152640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc111c50 con 0x7f3afc102ca0 2024-09-19T06:15:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.252+0000 7f3b03152640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc111f50 con 0x7f3afc1035c0 2024-09-19T06:15:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3b016ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc10c0f0 0x7f3afc19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3b016ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc10c0f0 0x7f3afc19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35954/0 (socket says 172.21.15.5:35954) 2024-09-19T06:15:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3b016ef640 1 -- 172.21.15.5:0/950775877 learned_addr learned my addr 172.21.15.5:0/950775877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3b00eee640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102ca0 0x7f3afc198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3af3fff640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3afc1035c0 0x7f3afc1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3b016ef640 1 -- 172.21.15.5:0/950775877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3afc1035c0 msgr2=0x7f3afc1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3b016ef640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3afc1035c0 0x7f3afc1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3b016ef640 1 -- 172.21.15.5:0/950775877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102ca0 msgr2=0x7f3afc198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3b016ef640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102ca0 0x7f3afc198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3b016ef640 1 -- 172.21.15.5:0/950775877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3afc19d350 con 0x7f3afc10c0f0 2024-09-19T06:15:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3b016ef640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc10c0f0 0x7f3afc19cc20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009dc0 tx=0x7f3af4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3af1ffb640 1 -- 172.21.15.5:0/950775877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4034040 con 0x7f3afc10c0f0 2024-09-19T06:15:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3af1ffb640 1 -- 172.21.15.5:0/950775877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3af4030850 con 0x7f3afc10c0f0 2024-09-19T06:15:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3b03152640 1 -- 172.21.15.5:0/950775877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3afc19d5e0 con 0x7f3afc10c0f0 2024-09-19T06:15:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.253+0000 7f3af1ffb640 1 -- 172.21.15.5:0/950775877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af40047f0 con 0x7f3afc10c0f0 2024-09-19T06:15:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.254+0000 7f3b03152640 1 -- 172.21.15.5:0/950775877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3afc1a4ed0 con 0x7f3afc10c0f0 2024-09-19T06:15:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.256+0000 7f3af1ffb640 1 -- 172.21.15.5:0/950775877 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3af4037020 con 0x7f3afc10c0f0 2024-09-19T06:15:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.256+0000 7f3b05fcc740 1 Processor -- start 2024-09-19T06:15:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.256+0000 7f3b05fcc740 1 -- start start 2024-09-19T06:15:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3b05fcc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990fe9d0 0x55e5990fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3b05fcc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e59910db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3b05fcc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e59910e0c0 0x55e5991104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3b05fcc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e5990632f0 con 0x55e5990fe9d0 2024-09-19T06:15:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3b05fcc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e59900b270 con 0x55e5990ff2f0 2024-09-19T06:15:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3b05fcc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e59905f2b0 con 0x55e59910e0c0 2024-09-19T06:15:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990fe9d0 0x55e5990fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990fe9d0 0x55e5990fedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35964/0 (socket says 172.21.15.5:35964) 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3af3fff640 1 -- 172.21.15.5:0/905131247 learned_addr learned my addr 172.21.15.5:0/905131247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3af1ffb640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3ae40a33b0 0x7f3ae40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3b00eee640 1 --2- 172.21.15.5:0/905131247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e59910db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3af3fff640 1 --2- 172.21.15.5:0/905131247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e59910e0c0 0x55e5991104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3b016ef640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3ae40a33b0 0x7f3ae40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3af3fff640 1 -- 172.21.15.5:0/905131247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e59910e0c0 msgr2=0x55e5991104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3af3fff640 1 --2- 172.21.15.5:0/905131247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e59910e0c0 0x55e5991104b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3af3fff640 1 -- 172.21.15.5:0/905131247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 msgr2=0x55e59910db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3af3fff640 1 --2- 172.21.15.5:0/905131247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e59910db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3af3fff640 1 -- 172.21.15.5:0/905131247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e59900a340 con 0x55e5990fe9d0 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3af1ffb640 1 -- 172.21.15.5:0/950775877 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f3af40f0c00 con 0x7f3afc10c0f0 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.257+0000 7f3af3fff640 1 --2- 172.21.15.5:0/905131247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990fe9d0 0x55e5990fedb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00bfa0 tx=0x7f3aec00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a374d2f407db5e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3ae2ffd640 1 -- 172.21.15.5:0/905131247 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004030 con 0x55e5990fe9d0 2024-09-19T06:15:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3ae2ffd640 1 -- 172.21.15.5:0/905131247 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3aec0041d0 con 0x55e5990fe9d0 2024-09-19T06:15:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3ae2ffd640 1 -- 172.21.15.5:0/905131247 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec00f6e0 con 0x55e5990fe9d0 2024-09-19T06:15:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3b016ef640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3ae40a33b0 0x7f3ae40a5870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3af40bcc70 tx=0x7f3af4005f40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3b05fcc740 1 -- 172.21.15.5:0/905131247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990fe9d0 msgr2=0x55e5990fedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/905131247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990fe9d0 0x55e5990fedb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00bfa0 tx=0x7f3aec00b7b0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3b05fcc740 1 -- 172.21.15.5:0/905131247 shutdown_connections 2024-09-19T06:15:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/905131247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e59910e0c0 0x55e5991104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/905131247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e59910db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/905131247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990fe9d0 0x55e5990fedb0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3b05fcc740 1 -- 172.21.15.5:0/905131247 >> 172.21.15.5:0/905131247 conn(0x55e5990fb060 msgr2=0x55e5990fb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3b05fcc740 1 -- 172.21.15.5:0/905131247 shutdown_connections 2024-09-19T06:15:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.258+0000 7f3b05fcc740 1 -- 172.21.15.5:0/905131247 wait complete. 2024-09-19T06:15:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.259+0000 7f3b05fcc740 1 -- start start 2024-09-19T06:15:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.259+0000 7f3b05fcc740 1 Processor -- start 2024-09-19T06:15:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.259+0000 7f3b05fcc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990fe9d0 0x55e59910b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.259+0000 7f3b05fcc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990ff2f0 0x55e59910b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.259+0000 7f3b05fcc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e59910e0c0 0x55e59910bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.259+0000 7f3b05fcc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e599110cf0 con 0x55e5990ff2f0 2024-09-19T06:15:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.259+0000 7f3b05fcc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e598fec290 con 0x55e5990fe9d0 2024-09-19T06:15:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.259+0000 7f3b05fcc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e599110e70 con 0x55e59910e0c0 2024-09-19T06:15:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.259+0000 7f3b05fcc740 10 client.0 _ll_register_callbacks cb 0x55e59901d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3b00eee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990ff2f0 0x55e59910b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3b00eee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990ff2f0 0x55e59910b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35974/0 (socket says 172.21.15.5:35974) 2024-09-19T06:15:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3b00eee640 1 -- 172.21.15.5:0/2162551597 learned_addr learned my addr 172.21.15.5:0/2162551597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3af3fff640 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990fe9d0 0x55e59910b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3af3fff640 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e59910e0c0 0x55e59910bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3b00eee640 1 -- 172.21.15.5:0/2162551597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e59910e0c0 msgr2=0x55e59910bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3b00eee640 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e59910e0c0 0x55e59910bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3b00eee640 1 -- 172.21.15.5:0/2162551597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990fe9d0 msgr2=0x55e59910b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3b00eee640 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990fe9d0 0x55e59910b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3b00eee640 1 -- 172.21.15.5:0/2162551597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e598d76210 con 0x55e5990ff2f0 2024-09-19T06:15:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3af3fff640 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e59910e0c0 0x55e59910bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:15:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3b00eee640 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990ff2f0 0x55e59910b570 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3af800ba10 tx=0x7f3af800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56cdc65c64300d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3adbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3adbfff640 1 -- 172.21.15.5:0/2162551597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8003e80 con 0x55e5990ff2f0 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3adbfff640 1 -- 172.21.15.5:0/2162551597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f3af8011440 con 0x55e5990ff2f0 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2162551597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aec0041d0 con 0x55e5990ff2f0 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3adbfff640 1 -- 172.21.15.5:0/2162551597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3af8011640 con 0x55e5990ff2f0 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3b05fcc740 10 client.18621 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.260+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2162551597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e598e42620 con 0x55e5990ff2f0 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.261+0000 7f3adbfff640 1 -- 172.21.15.5:0/2162551597 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3aec0041d0 con 0x55e5990ff2f0 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.261+0000 7f3adbfff640 1 -- 172.21.15.5:0/2162551597 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f3af8023940 con 0x55e5990ff2f0 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.261+0000 7f3b05fcc740 10 client.18621 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.261+0000 7f3b05fcc740 10 client.18621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.261+0000 7f3b05fcc740 20 client.18621 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.261+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e59911e230 0x55e599120640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.261+0000 7f3b05fcc740 4 client.18621 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.261+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2162551597 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e599070a20 con 0x55e59911e230 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.261+0000 7f3af3fff640 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e59911e230 0x55e599120640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.262+0000 7f3adbfff640 10 client.18621 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.262+0000 7f3af3fff640 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e59911e230 0x55e599120640 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d847ad633275779a server_cookie=2f3fc2b4c9119b16 in_seq=0 out_seq=0 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3adbfff640 1 -- 172.21.15.5:0/2162551597 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e599070a20 con 0x55e59911e230 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3adbfff640 10 client.18621 handle_command_reply: tid=0 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3b05fcc740 1 client.18621 shutdown 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2162551597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990ff2f0 msgr2=0x55e59910b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990ff2f0 0x55e59910b570 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3af800ba10 tx=0x7f3af800bed0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2162551597 shutdown_connections 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e59911e230 0x55e599120640 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e59910e0c0 0x55e59910bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5990ff2f0 0x55e59910b570 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3adbfff640 0 client.18621 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/2162551597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990fe9d0 0x55e59910b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2162551597 >> 172.21.15.5:0/2162551597 conn(0x55e5990fb060 msgr2=0x55e599100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2162551597 shutdown_connections 2024-09-19T06:15:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2162551597 wait complete. 2024-09-19T06:15:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.263+0000 7f3b05fcc740 20 client.18621 trim_cache size 0 max 16384 2024-09-19T06:15:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.277+0000 7f3b05fcc740 1 Processor -- start 2024-09-19T06:15:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3b05fcc740 1 -- start start 2024-09-19T06:15:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3b05fcc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5990fe9d0 0x55e59910cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3b05fcc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e599125f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3b05fcc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 0x55e599126450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3b05fcc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1ab530 con 0x55e59910e0c0 2024-09-19T06:15:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3b05fcc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1ad130 con 0x55e5990ff2f0 2024-09-19T06:15:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3b05fcc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1abc00 con 0x55e5990fe9d0 2024-09-19T06:15:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 0x55e599126450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 0x55e599126450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35990/0 (socket says 172.21.15.5:35990) 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3af3fff640 1 -- 172.21.15.5:0/4134617690 learned_addr learned my addr 172.21.15.5:0/4134617690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3af3fff640 1 --2- 172.21.15.5:0/4134617690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5990fe9d0 0x55e59910cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3b00eee640 1 --2- 172.21.15.5:0/4134617690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e599125f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3af3fff640 1 -- 172.21.15.5:0/4134617690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5990fe9d0 msgr2=0x55e59910cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3af3fff640 1 --2- 172.21.15.5:0/4134617690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5990fe9d0 0x55e59910cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3af3fff640 1 -- 172.21.15.5:0/4134617690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 msgr2=0x55e599125f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3af3fff640 1 --2- 172.21.15.5:0/4134617690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e599125f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3af3fff640 1 -- 172.21.15.5:0/4134617690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e59911cce0 con 0x55e59910e0c0 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3b00eee640 1 --2- 172.21.15.5:0/4134617690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e599125f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.278+0000 7f3af3fff640 1 --2- 172.21.15.5:0/4134617690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 0x55e599126450 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0124e0 tx=0x7f3aec015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f1a091290562ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3ae27fc640 1 -- 172.21.15.5:0/4134617690 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec031070 con 0x55e59910e0c0 2024-09-19T06:15:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3ae27fc640 1 -- 172.21.15.5:0/4134617690 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3aec0035f0 con 0x55e59910e0c0 2024-09-19T06:15:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3ae27fc640 1 -- 172.21.15.5:0/4134617690 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec02b580 con 0x55e59910e0c0 2024-09-19T06:15:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3b05fcc740 1 -- 172.21.15.5:0/4134617690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 msgr2=0x55e599126450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/4134617690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 0x55e599126450 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0124e0 tx=0x7f3aec015040 comp rx=0 tx=0).stop 2024-09-19T06:15:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3b05fcc740 1 -- 172.21.15.5:0/4134617690 shutdown_connections 2024-09-19T06:15:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/4134617690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 0x55e599126450 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/4134617690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e599125f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/4134617690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5990fe9d0 0x55e59910cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3b05fcc740 1 -- 172.21.15.5:0/4134617690 >> 172.21.15.5:0/4134617690 conn(0x55e5990fb060 msgr2=0x55e59910c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3b05fcc740 1 -- 172.21.15.5:0/4134617690 shutdown_connections 2024-09-19T06:15:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3b05fcc740 1 -- 172.21.15.5:0/4134617690 wait complete. 2024-09-19T06:15:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.279+0000 7f3b05fcc740 1 -- start start 2024-09-19T06:15:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3b05fcc740 1 Processor -- start 2024-09-19T06:15:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3b05fcc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5990fe9d0 0x55e599116a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3b05fcc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e599111d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3b05fcc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 0x55e599112480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3b05fcc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1aca60 con 0x55e59910e0c0 2024-09-19T06:15:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3b05fcc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1ac330 con 0x55e5990ff2f0 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3b05fcc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1a9530 con 0x55e5990fe9d0 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3b05fcc740 10 client.0 _ll_register_callbacks cb 0x7f3af80042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 0x55e599112480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 0x55e599112480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35992/0 (socket says 172.21.15.5:35992) 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3af3fff640 1 -- 172.21.15.5:0/2745646592 learned_addr learned my addr 172.21.15.5:0/2745646592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3af3fff640 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5990fe9d0 0x55e599116a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3b00eee640 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e599111d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3af3fff640 1 -- 172.21.15.5:0/2745646592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5990fe9d0 msgr2=0x55e599116a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.280+0000 7f3af3fff640 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5990fe9d0 0x55e599116a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3af3fff640 1 -- 172.21.15.5:0/2745646592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 msgr2=0x55e599111d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3af3fff640 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e599111d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3af3fff640 1 -- 172.21.15.5:0/2745646592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e599187ee0 con 0x55e59910e0c0 2024-09-19T06:15:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3af3fff640 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 0x55e599112480 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0123c0 tx=0x7f3aec00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a013f6ca80a66d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3adbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3adbfff640 1 -- 172.21.15.5:0/2745646592 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec031070 con 0x55e59910e0c0 2024-09-19T06:15:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3adbfff640 1 -- 172.21.15.5:0/2745646592 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f3aec0037a0 con 0x55e59910e0c0 2024-09-19T06:15:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2745646592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aec0035f0 con 0x55e59910e0c0 2024-09-19T06:15:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3adbfff640 1 -- 172.21.15.5:0/2745646592 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3aec0039e0 con 0x55e59910e0c0 2024-09-19T06:15:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3b05fcc740 10 client.18627 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2745646592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e599189090 con 0x55e59910e0c0 2024-09-19T06:15:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3adbfff640 1 -- 172.21.15.5:0/2745646592 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3aec0035f0 con 0x55e59910e0c0 2024-09-19T06:15:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3adbfff640 1 -- 172.21.15.5:0/2745646592 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f3aec01c020 con 0x55e59910e0c0 2024-09-19T06:15:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3b05fcc740 10 client.18627 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3b05fcc740 10 client.18627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3b05fcc740 20 client.18627 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e599189520 0x55e59918b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3b05fcc740 4 client.18627 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:15:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.281+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2745646592 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e59906ad70 con 0x55e599189520 2024-09-19T06:15:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.282+0000 7f3b00eee640 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e599189520 0x55e59918b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.282+0000 7f3b00eee640 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e599189520 0x55e59918b940 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12acd5d2d7611ccd server_cookie=77abf3419530986e in_seq=0 out_seq=0 2024-09-19T06:15:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.282+0000 7f3adbfff640 10 client.18627 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3adbfff640 1 -- 172.21.15.5:0/2745646592 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e59906ad70 con 0x55e599189520 2024-09-19T06:15:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3adbfff640 10 client.18627 handle_command_reply: tid=0 2024-09-19T06:15:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3b05fcc740 1 client.18627 shutdown 2024-09-19T06:15:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2745646592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 msgr2=0x55e599112480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 0x55e599112480 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0123c0 tx=0x7f3aec00bf10 comp rx=0 tx=0).stop 2024-09-19T06:15:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2745646592 shutdown_connections 2024-09-19T06:15:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e599189520 0x55e59918b940 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e59910e0c0 0x55e599112480 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5990ff2f0 0x55e599111d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3adbfff640 0 client.18627 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3b05fcc740 1 --2- 172.21.15.5:0/2745646592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5990fe9d0 0x55e599116a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2745646592 >> 172.21.15.5:0/2745646592 conn(0x55e5990fb060 msgr2=0x55e5990fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2745646592 shutdown_connections 2024-09-19T06:15:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3b05fcc740 1 -- 172.21.15.5:0/2745646592 wait complete. 2024-09-19T06:15:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.311+0000 7f3b05fcc740 20 client.18627 trim_cache size 0 max 16384 2024-09-19T06:15:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:15:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:15:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:15:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:15:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.312+0000 7f3b03152640 1 -- 172.21.15.5:0/950775877 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3ae40a33b0 msgr2=0x7f3ae40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.312+0000 7f3b03152640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3ae40a33b0 0x7f3ae40a5870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3af40bcc70 tx=0x7f3af4005f40 comp rx=0 tx=0).stop 2024-09-19T06:15:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.312+0000 7f3b03152640 1 -- 172.21.15.5:0/950775877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc10c0f0 msgr2=0x7f3afc19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.312+0000 7f3b03152640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc10c0f0 0x7f3afc19cc20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009dc0 tx=0x7f3af4003f10 comp rx=0 tx=0).stop 2024-09-19T06:15:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.312+0000 7f3b03152640 1 -- 172.21.15.5:0/950775877 shutdown_connections 2024-09-19T06:15:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.312+0000 7f3b03152640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc10c0f0 0x7f3afc19cc20 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.312+0000 7f3b03152640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3ae40a33b0 0x7f3ae40a5870 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.312+0000 7f3b03152640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3afc1035c0 0x7f3afc1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.312+0000 7f3b03152640 1 --2- 172.21.15.5:0/950775877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102ca0 0x7f3afc198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.312+0000 7f3b03152640 1 -- 172.21.15.5:0/950775877 >> 172.21.15.5:0/950775877 conn(0x7f3afc0fc6b0 msgr2=0x7f3afc0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.312+0000 7f3b03152640 1 -- 172.21.15.5:0/950775877 shutdown_connections 2024-09-19T06:15:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:22.312+0000 7f3b03152640 1 -- 172.21.15.5:0/950775877 wait complete. 2024-09-19T06:15:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:22 smithi157 ceph-mon[35346]: pgmap v376: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 886 KiB/s wr, 84 op/s 2024-09-19T06:15:22.847 INFO:tasks.workunit.client.0.smithi005.stdout: 1 276833 2.19 MB/sec execute 528 sec latency 2119.526 ms 2024-09-19T06:15:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:22 smithi005 ceph-mon[30955]: pgmap v376: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 886 KiB/s wr, 84 op/s 2024-09-19T06:15:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:22 smithi049 ceph-mon[34640]: pgmap v376: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 886 KiB/s wr, 84 op/s 2024-09-19T06:15:23.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 276833 2.19 MB/sec execute 529 sec latency 3119.640 ms 2024-09-19T06:15:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:24 smithi157 ceph-mon[35346]: pgmap v377: 97 pgs: 97 active+clean; 155 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-09-19T06:15:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:24 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:15:24.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 276833 2.18 MB/sec execute 530 sec latency 4119.750 ms 2024-09-19T06:15:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:24 smithi005 ceph-mon[30955]: pgmap v377: 97 pgs: 97 active+clean; 155 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-09-19T06:15:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:24 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:15:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:24 smithi049 ceph-mon[34640]: pgmap v377: 97 pgs: 97 active+clean; 155 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-09-19T06:15:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:24 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:15:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:25 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:15:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:25 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:15:25.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 276938 2.18 MB/sec execute 531 sec latency 4740.627 ms 2024-09-19T06:15:25.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:25 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:15:25.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:25 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:15:25.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:25 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:15:25.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:25 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:15:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:26 smithi157 ceph-mon[35346]: pgmap v378: 97 pgs: 97 active+clean; 155 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 835 KiB/s wr, 88 op/s 2024-09-19T06:15:26.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 277195 2.18 MB/sec execute 532 sec latency 13.864 ms 2024-09-19T06:15:26.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:26 smithi005 ceph-mon[30955]: pgmap v378: 97 pgs: 97 active+clean; 155 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 835 KiB/s wr, 88 op/s 2024-09-19T06:15:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:26 smithi049 ceph-mon[34640]: pgmap v378: 97 pgs: 97 active+clean; 155 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 835 KiB/s wr, 88 op/s 2024-09-19T06:15:27.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 277631 2.18 MB/sec execute 533 sec latency 18.308 ms 2024-09-19T06:15:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:28 smithi157 ceph-mon[35346]: pgmap v379: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-09-19T06:15:28.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 278151 2.18 MB/sec execute 534 sec latency 21.815 ms 2024-09-19T06:15:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:28 smithi005 ceph-mon[30955]: pgmap v379: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-09-19T06:15:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:28 smithi049 ceph-mon[34640]: pgmap v379: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-09-19T06:15:29.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 278650 2.18 MB/sec execute 535 sec latency 21.331 ms 2024-09-19T06:15:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:30 smithi157 ceph-mon[35346]: pgmap v380: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 407 KiB/s wr, 57 op/s 2024-09-19T06:15:30.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 279179 2.18 MB/sec execute 536 sec latency 22.435 ms 2024-09-19T06:15:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:30 smithi005 ceph-mon[30955]: pgmap v380: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 407 KiB/s wr, 57 op/s 2024-09-19T06:15:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:30 smithi049 ceph-mon[34640]: pgmap v380: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 407 KiB/s wr, 57 op/s 2024-09-19T06:15:31.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 279935 2.18 MB/sec execute 537 sec latency 14.990 ms 2024-09-19T06:15:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:32 smithi157 ceph-mon[35346]: pgmap v381: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 407 KiB/s wr, 57 op/s 2024-09-19T06:15:32.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 280310 2.19 MB/sec execute 538 sec latency 76.668 ms 2024-09-19T06:15:32.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:32 smithi005 ceph-mon[30955]: pgmap v381: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 407 KiB/s wr, 57 op/s 2024-09-19T06:15:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:32 smithi049 ceph-mon[34640]: pgmap v381: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 407 KiB/s wr, 57 op/s 2024-09-19T06:15:33.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 280310 2.18 MB/sec execute 539 sec latency 1076.699 ms 2024-09-19T06:15:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:34 smithi157 ceph-mon[35346]: pgmap v382: 97 pgs: 97 active+clean; 155 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-09-19T06:15:34.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 280377 2.18 MB/sec execute 540 sec latency 1274.907 ms 2024-09-19T06:15:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:34 smithi005 ceph-mon[30955]: pgmap v382: 97 pgs: 97 active+clean; 155 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-09-19T06:15:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:34 smithi049 ceph-mon[34640]: pgmap v382: 97 pgs: 97 active+clean; 155 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-09-19T06:15:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.411+0000 7f7087bbe640 1 -- 172.21.15.5:0/4065995804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7080107b70 msgr2=0x7f7080107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.411+0000 7f7087bbe640 1 --2- 172.21.15.5:0/4065995804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7080107b70 0x7f7080107f50 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f707c009930 tx=0x7f707c02a420 comp rx=0 tx=0).stop 2024-09-19T06:15:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.412+0000 7f7087bbe640 1 -- 172.21.15.5:0/4065995804 shutdown_connections 2024-09-19T06:15:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.412+0000 7f7087bbe640 1 --2- 172.21.15.5:0/4065995804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7080102470 0x7f708010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.412+0000 7f7087bbe640 1 --2- 172.21.15.5:0/4065995804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7080101ad0 0x7f7080101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.412+0000 7f7087bbe640 1 --2- 172.21.15.5:0/4065995804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7080107b70 0x7f7080107f50 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.412+0000 7f7087bbe640 1 -- 172.21.15.5:0/4065995804 >> 172.21.15.5:0/4065995804 conn(0x7f70800fd7d0 msgr2=0x7f70800ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.412+0000 7f7087bbe640 1 -- 172.21.15.5:0/4065995804 shutdown_connections 2024-09-19T06:15:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.412+0000 7f7087bbe640 1 -- 172.21.15.5:0/4065995804 wait complete. 2024-09-19T06:15:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.412+0000 7f7087bbe640 1 Processor -- start 2024-09-19T06:15:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.412+0000 7f7087bbe640 1 -- start start 2024-09-19T06:15:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.412+0000 7f7087bbe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7080101ad0 0x7f70801982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f7087bbe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7080102470 0x7f70801987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f7087bbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7080107b70 0x7f708019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f7087bbe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7080111d90 con 0x7f7080107b70 2024-09-19T06:15:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f7087bbe640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7080111c10 con 0x7f7080101ad0 2024-09-19T06:15:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f7087bbe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7080111f10 con 0x7f7080102470 2024-09-19T06:15:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f708615b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7080107b70 0x7f708019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f708615b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7080107b70 0x7f708019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46224/0 (socket says 172.21.15.5:46224) 2024-09-19T06:15:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f708615b640 1 -- 172.21.15.5:0/3130621650 learned_addr learned my addr 172.21.15.5:0/3130621650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f708595a640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7080101ad0 0x7f70801982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f7085159640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7080102470 0x7f70801987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f708615b640 1 -- 172.21.15.5:0/3130621650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7080102470 msgr2=0x7f70801987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f708615b640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7080102470 0x7f70801987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f708615b640 1 -- 172.21.15.5:0/3130621650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7080101ad0 msgr2=0x7f70801982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f708615b640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7080101ad0 0x7f70801982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f708615b640 1 -- 172.21.15.5:0/3130621650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708019d290 con 0x7f7080107b70 2024-09-19T06:15:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f708615b640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7080107b70 0x7f708019cb60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f707800eaa0 tx=0x7f707800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f7076ffd640 1 -- 172.21.15.5:0/3130621650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707800cdb0 con 0x7f7080107b70 2024-09-19T06:15:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f7076ffd640 1 -- 172.21.15.5:0/3130621650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7078005020 con 0x7f7080107b70 2024-09-19T06:15:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f7087bbe640 1 -- 172.21.15.5:0/3130621650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708019d580 con 0x7f7080107b70 2024-09-19T06:15:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.413+0000 7f7076ffd640 1 -- 172.21.15.5:0/3130621650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70780116d0 con 0x7f7080107b70 2024-09-19T06:15:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.414+0000 7f7087bbe640 1 -- 172.21.15.5:0/3130621650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70801a4e10 con 0x7f7080107b70 2024-09-19T06:15:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.417+0000 7f7076ffd640 1 -- 172.21.15.5:0/3130621650 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f70780051d0 con 0x7f7080107b70 2024-09-19T06:15:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.417+0000 7f708aa30740 1 Processor -- start 2024-09-19T06:15:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.417+0000 7f708aa30740 1 -- start start 2024-09-19T06:15:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708aa30740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351ed9f0 0x55af351eddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708aa30740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 0x55af351fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708aa30740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351fd0c0 0x55af351ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708aa30740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55af351522f0 con 0x55af351ee310 2024-09-19T06:15:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708aa30740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55af350fa270 con 0x55af351ed9f0 2024-09-19T06:15:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708aa30740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55af3514e2b0 con 0x55af351fd0c0 2024-09-19T06:15:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708595a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 0x55af351fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708595a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 0x55af351fcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46236/0 (socket says 172.21.15.5:46236) 2024-09-19T06:15:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f7085159640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351ed9f0 0x55af351eddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708595a640 1 -- 172.21.15.5:0/293578686 learned_addr learned my addr 172.21.15.5:0/293578686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f7076ffd640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f706c0a33b0 0x7f706c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f7085159640 1 --2- 172.21.15.5:0/293578686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351fd0c0 0x55af351ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708615b640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f706c0a33b0 0x7f706c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708595a640 1 -- 172.21.15.5:0/293578686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351fd0c0 msgr2=0x55af351ff4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708595a640 1 --2- 172.21.15.5:0/293578686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351fd0c0 0x55af351ff4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708595a640 1 -- 172.21.15.5:0/293578686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351ed9f0 msgr2=0x55af351eddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708595a640 1 --2- 172.21.15.5:0/293578686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351ed9f0 0x55af351eddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f7076ffd640 1 -- 172.21.15.5:0/3130621650 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f70780cffe0 con 0x7f7080107b70 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f708595a640 1 -- 172.21.15.5:0/293578686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af350f9340 con 0x55af351ee310 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.418+0000 7f7085159640 1 --2- 172.21.15.5:0/293578686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351ed9f0 0x55af351eddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.419+0000 7f708595a640 1 --2- 172.21.15.5:0/293578686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 0x55af351fcb80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f707c009930 tx=0x7f707c032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ddfbc318013af9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.419+0000 7f708615b640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f706c0a33b0 0x7f706c0a5870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f707800efd0 tx=0x7f7078097ea0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.419+0000 7f706bfff640 1 -- 172.21.15.5:0/293578686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c03f070 con 0x55af351ee310 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.419+0000 7f706bfff640 1 -- 172.21.15.5:0/293578686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f707c0316a0 con 0x55af351ee310 2024-09-19T06:15:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.419+0000 7f706bfff640 1 -- 172.21.15.5:0/293578686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c035070 con 0x55af351ee310 2024-09-19T06:15:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.420+0000 7f708aa30740 1 -- 172.21.15.5:0/293578686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 msgr2=0x55af351fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.420+0000 7f708aa30740 1 --2- 172.21.15.5:0/293578686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 0x55af351fcb80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f707c009930 tx=0x7f707c032cc0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.420+0000 7f708aa30740 1 -- 172.21.15.5:0/293578686 shutdown_connections 2024-09-19T06:15:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.420+0000 7f708aa30740 1 --2- 172.21.15.5:0/293578686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351fd0c0 0x55af351ff4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.420+0000 7f708aa30740 1 --2- 172.21.15.5:0/293578686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 0x55af351fcb80 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.420+0000 7f708aa30740 1 --2- 172.21.15.5:0/293578686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351ed9f0 0x55af351eddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.420+0000 7f708aa30740 1 -- 172.21.15.5:0/293578686 >> 172.21.15.5:0/293578686 conn(0x55af351ea060 msgr2=0x55af351ea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.420+0000 7f708aa30740 1 -- 172.21.15.5:0/293578686 shutdown_connections 2024-09-19T06:15:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.420+0000 7f708aa30740 1 -- 172.21.15.5:0/293578686 wait complete. 2024-09-19T06:15:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.420+0000 7f708aa30740 1 -- start start 2024-09-19T06:15:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.420+0000 7f708aa30740 1 Processor -- start 2024-09-19T06:15:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f708aa30740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 0x55af351fa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f708aa30740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ee310 0x55af351fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f708aa30740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 0x55af351faac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f708aa30740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55af351ffcc0 con 0x55af351ed9f0 2024-09-19T06:15:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f708aa30740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55af350db290 con 0x55af351fd0c0 2024-09-19T06:15:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f708aa30740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55af351ffe40 con 0x55af351ee310 2024-09-19T06:15:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f708aa30740 10 client.0 _ll_register_callbacks cb 0x55af3510c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f7085159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 0x55af351fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f7085159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 0x55af351fa040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46240/0 (socket says 172.21.15.5:46240) 2024-09-19T06:15:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f7085159640 1 -- 172.21.15.5:0/1866204459 learned_addr learned my addr 172.21.15.5:0/1866204459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f708595a640 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ee310 0x55af351fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f7085159640 1 -- 172.21.15.5:0/1866204459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ee310 msgr2=0x55af351fa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f7085159640 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ee310 0x55af351fa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f7085159640 1 -- 172.21.15.5:0/1866204459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 msgr2=0x55af351faac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f7085159640 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 0x55af351faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f7085159640 1 -- 172.21.15.5:0/1866204459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af34e65210 con 0x55af351ed9f0 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f708595a640 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ee310 0x55af351fa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f7085159640 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 0x55af351fa040 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7070005d60 tx=0x7f7070010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c9b96b5e350341 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f70697fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.421+0000 7f70697fa640 1 -- 172.21.15.5:0/1866204459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070011440 con 0x55af351ed9f0 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f708aa30740 1 -- 172.21.15.5:0/1866204459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f707c0316a0 con 0x55af351ed9f0 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f70697fa640 1 -- 172.21.15.5:0/1866204459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f70700115e0 con 0x55af351ed9f0 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f70697fa640 1 -- 172.21.15.5:0/1866204459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f70700117e0 con 0x55af351ed9f0 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f708aa30740 10 client.18639 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f708aa30740 1 -- 172.21.15.5:0/1866204459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55af34f31620 con 0x55af351ed9f0 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f70697fa640 1 -- 172.21.15.5:0/1866204459 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f707c0316a0 con 0x55af351ed9f0 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f70697fa640 1 -- 172.21.15.5:0/1866204459 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f707001e960 con 0x55af351ed9f0 2024-09-19T06:15:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f708aa30740 10 client.18639 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f708aa30740 10 client.18639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f708aa30740 20 client.18639 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f708aa30740 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55af3520d100 0x55af3520f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f708aa30740 4 client.18639 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f708aa30740 1 -- 172.21.15.5:0/1866204459 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af3515fa20 con 0x55af3520d100 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.422+0000 7f708595a640 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55af3520d100 0x55af3520f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.423+0000 7f708595a640 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55af3520d100 0x55af3520f510 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54b3196821c6ed8a server_cookie=383b9485be3a3db7 in_seq=0 out_seq=0 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.423+0000 7f70697fa640 10 client.18639 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f70697fa640 1 -- 172.21.15.5:0/1866204459 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af3515fa20 con 0x55af3520d100 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f70697fa640 10 client.18639 handle_command_reply: tid=0 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f708aa30740 1 client.18639 shutdown 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f708aa30740 1 -- 172.21.15.5:0/1866204459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 msgr2=0x55af351fa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f708aa30740 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 0x55af351fa040 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7070005d60 tx=0x7f7070010800 comp rx=0 tx=0).stop 2024-09-19T06:15:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f708aa30740 1 -- 172.21.15.5:0/1866204459 shutdown_connections 2024-09-19T06:15:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f708aa30740 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55af3520d100 0x55af3520f510 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f708aa30740 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 0x55af351faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f708aa30740 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ee310 0x55af351fa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f70697fa640 0 client.18639 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f708aa30740 1 --2- 172.21.15.5:0/1866204459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 0x55af351fa040 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f708aa30740 1 -- 172.21.15.5:0/1866204459 >> 172.21.15.5:0/1866204459 conn(0x55af351ea060 msgr2=0x55af351efdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f708aa30740 1 -- 172.21.15.5:0/1866204459 shutdown_connections 2024-09-19T06:15:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f708aa30740 1 -- 172.21.15.5:0/1866204459 wait complete. 2024-09-19T06:15:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.424+0000 7f708aa30740 20 client.18639 trim_cache size 0 max 16384 2024-09-19T06:15:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708aa30740 1 Processor -- start 2024-09-19T06:15:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708aa30740 1 -- start start 2024-09-19T06:15:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708aa30740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ed9f0 0x55af3520f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708aa30740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 0x55af35214e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708aa30740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 0x55af35215360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708aa30740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70801ab5b0 con 0x55af351ee310 2024-09-19T06:15:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708aa30740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f70801ad1b0 con 0x55af351fd0c0 2024-09-19T06:15:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708aa30740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f70801abc80 con 0x55af351ed9f0 2024-09-19T06:15:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708595a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 0x55af35214e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708595a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 0x55af35214e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46250/0 (socket says 172.21.15.5:46250) 2024-09-19T06:15:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708595a640 1 -- 172.21.15.5:0/2127878727 learned_addr learned my addr 172.21.15.5:0/2127878727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f7085159640 1 --2- 172.21.15.5:0/2127878727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 0x55af35215360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708595a640 1 -- 172.21.15.5:0/2127878727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ed9f0 msgr2=0x55af3520f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708595a640 1 --2- 172.21.15.5:0/2127878727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ed9f0 0x55af3520f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708595a640 1 -- 172.21.15.5:0/2127878727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 msgr2=0x55af35215360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708595a640 1 --2- 172.21.15.5:0/2127878727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 0x55af35215360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.439+0000 7f708595a640 1 -- 172.21.15.5:0/2127878727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af351f8fd0 con 0x55af351ee310 2024-09-19T06:15:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f708595a640 1 --2- 172.21.15.5:0/2127878727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 0x55af35214e20 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f707c005ab0 tx=0x7f707c0052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e9d54467bb651c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f706b7fe640 1 -- 172.21.15.5:0/2127878727 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c051070 con 0x55af351ee310 2024-09-19T06:15:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f706b7fe640 1 -- 172.21.15.5:0/2127878727 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f707c005510 con 0x55af351ee310 2024-09-19T06:15:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f706b7fe640 1 -- 172.21.15.5:0/2127878727 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c03f040 con 0x55af351ee310 2024-09-19T06:15:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f708aa30740 1 -- 172.21.15.5:0/2127878727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 msgr2=0x55af35214e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f708aa30740 1 --2- 172.21.15.5:0/2127878727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 0x55af35214e20 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f707c005ab0 tx=0x7f707c0052a0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f708aa30740 1 -- 172.21.15.5:0/2127878727 shutdown_connections 2024-09-19T06:15:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f708aa30740 1 --2- 172.21.15.5:0/2127878727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 0x55af35215360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f708aa30740 1 --2- 172.21.15.5:0/2127878727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ee310 0x55af35214e20 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f708aa30740 1 --2- 172.21.15.5:0/2127878727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ed9f0 0x55af3520f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f708aa30740 1 -- 172.21.15.5:0/2127878727 >> 172.21.15.5:0/2127878727 conn(0x55af351ea060 msgr2=0x55af351efd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f708aa30740 1 -- 172.21.15.5:0/2127878727 shutdown_connections 2024-09-19T06:15:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.440+0000 7f708aa30740 1 -- 172.21.15.5:0/2127878727 wait complete. 2024-09-19T06:15:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.441+0000 7f708aa30740 1 -- start start 2024-09-19T06:15:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.441+0000 7f708aa30740 1 Processor -- start 2024-09-19T06:15:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.441+0000 7f708aa30740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 0x55af35200f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.441+0000 7f708aa30740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ee310 0x55af352014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.441+0000 7f708aa30740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 0x55af35201a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.441+0000 7f708aa30740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70801acae0 con 0x55af351ed9f0 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.441+0000 7f708aa30740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f70801ac3b0 con 0x55af351fd0c0 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.441+0000 7f708aa30740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f70801a95b0 con 0x55af351ee310 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.441+0000 7f708aa30740 10 client.0 _ll_register_callbacks cb 0x7f7070013ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.441+0000 7f7085159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 0x55af35200f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.441+0000 7f7085159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 0x55af35200f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46254/0 (socket says 172.21.15.5:46254) 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.441+0000 7f7085159640 1 -- 172.21.15.5:0/3445081417 learned_addr learned my addr 172.21.15.5:0/3445081417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f708595a640 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ee310 0x55af352014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f7085159640 1 -- 172.21.15.5:0/3445081417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ee310 msgr2=0x55af352014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f7085159640 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ee310 0x55af352014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f7085159640 1 -- 172.21.15.5:0/3445081417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 msgr2=0x55af35201a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f7085159640 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 0x55af35201a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f7085159640 1 -- 172.21.15.5:0/3445081417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af35276f50 con 0x55af351ed9f0 2024-09-19T06:15:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f708595a640 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ee310 0x55af352014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f7085159640 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 0x55af35200f90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7070010840 tx=0x7f707000e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215097cddefb746 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f70697fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f70697fa640 1 -- 172.21.15.5:0/3445081417 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070036060 con 0x55af351ed9f0 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f708aa30740 1 -- 172.21.15.5:0/3445081417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f707c005510 con 0x55af351ed9f0 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f70697fa640 1 -- 172.21.15.5:0/3445081417 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f7070011980 con 0x55af351ed9f0 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f70697fa640 1 -- 172.21.15.5:0/3445081417 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f707002b040 con 0x55af351ed9f0 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f708aa30740 10 client.18645 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f708aa30740 1 -- 172.21.15.5:0/3445081417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55af35278220 con 0x55af351ed9f0 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.442+0000 7f70697fa640 1 -- 172.21.15.5:0/3445081417 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f707c005510 con 0x55af351ed9f0 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.443+0000 7f70697fa640 1 -- 172.21.15.5:0/3445081417 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f7070027450 con 0x55af351ed9f0 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.443+0000 7f708aa30740 10 client.18645 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.443+0000 7f708aa30740 10 client.18645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.443+0000 7f708aa30740 20 client.18645 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.443+0000 7f708aa30740 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55af35278940 0x55af3527ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.443+0000 7f708aa30740 4 client.18645 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:15:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.443+0000 7f708aa30740 1 -- 172.21.15.5:0/3445081417 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af35159d70 con 0x55af35278940 2024-09-19T06:15:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.443+0000 7f708595a640 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55af35278940 0x55af3527ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.443+0000 7f708595a640 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55af35278940 0x55af3527ad60 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb058a66e56f83cd server_cookie=6949faef4acc8a54 in_seq=0 out_seq=0 2024-09-19T06:15:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.443+0000 7f70697fa640 10 client.18645 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f70697fa640 1 -- 172.21.15.5:0/3445081417 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af35159d70 con 0x55af35278940 2024-09-19T06:15:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f70697fa640 10 client.18645 handle_command_reply: tid=0 2024-09-19T06:15:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f708aa30740 1 client.18645 shutdown 2024-09-19T06:15:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f708aa30740 1 -- 172.21.15.5:0/3445081417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 msgr2=0x55af35200f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f708aa30740 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 0x55af35200f90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7070010840 tx=0x7f707000e980 comp rx=0 tx=0).stop 2024-09-19T06:15:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f708aa30740 1 -- 172.21.15.5:0/3445081417 shutdown_connections 2024-09-19T06:15:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f708aa30740 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55af35278940 0x55af3527ad60 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f708aa30740 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af351fd0c0 0x55af35201a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f708aa30740 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af351ee310 0x55af352014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f708aa30740 1 --2- 172.21.15.5:0/3445081417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af351ed9f0 0x55af35200f90 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f708aa30740 1 -- 172.21.15.5:0/3445081417 >> 172.21.15.5:0/3445081417 conn(0x55af351ea060 msgr2=0x55af351e89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f70697fa640 0 client.18645 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f708aa30740 1 -- 172.21.15.5:0/3445081417 shutdown_connections 2024-09-19T06:15:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f708aa30740 1 -- 172.21.15.5:0/3445081417 wait complete. 2024-09-19T06:15:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.480+0000 7f708aa30740 20 client.18645 trim_cache size 0 max 16384 2024-09-19T06:15:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:15:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:15:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:15:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:15:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.481+0000 7f7087bbe640 1 -- 172.21.15.5:0/3130621650 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f706c0a33b0 msgr2=0x7f706c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.481+0000 7f7087bbe640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f706c0a33b0 0x7f706c0a5870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f707800efd0 tx=0x7f7078097ea0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.481+0000 7f7087bbe640 1 -- 172.21.15.5:0/3130621650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7080107b70 msgr2=0x7f708019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.481+0000 7f7087bbe640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7080107b70 0x7f708019cb60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f707800eaa0 tx=0x7f707800ef60 comp rx=0 tx=0).stop 2024-09-19T06:15:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.481+0000 7f7087bbe640 1 -- 172.21.15.5:0/3130621650 shutdown_connections 2024-09-19T06:15:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.481+0000 7f7087bbe640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7080107b70 0x7f708019cb60 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.481+0000 7f7087bbe640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f706c0a33b0 0x7f706c0a5870 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.481+0000 7f7087bbe640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7080102470 0x7f70801987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.481+0000 7f7087bbe640 1 --2- 172.21.15.5:0/3130621650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7080101ad0 0x7f70801982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.481+0000 7f7087bbe640 1 -- 172.21.15.5:0/3130621650 >> 172.21.15.5:0/3130621650 conn(0x7f70800fd7d0 msgr2=0x7f70800fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.481+0000 7f7087bbe640 1 -- 172.21.15.5:0/3130621650 shutdown_connections 2024-09-19T06:15:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:35.481+0000 7f7087bbe640 1 -- 172.21.15.5:0/3130621650 wait complete. 2024-09-19T06:15:35.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 280494 2.18 MB/sec execute 541 sec latency 1178.508 ms 2024-09-19T06:15:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:36 smithi157 ceph-mon[35346]: pgmap v383: 97 pgs: 97 active+clean; 155 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 877 KiB/s wr, 84 op/s 2024-09-19T06:15:36.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 280755 2.17 MB/sec execute 542 sec latency 12.850 ms 2024-09-19T06:15:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:36 smithi005 ceph-mon[30955]: pgmap v383: 97 pgs: 97 active+clean; 155 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 877 KiB/s wr, 84 op/s 2024-09-19T06:15:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:36 smithi049 ceph-mon[34640]: pgmap v383: 97 pgs: 97 active+clean; 155 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 877 KiB/s wr, 84 op/s 2024-09-19T06:15:37.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 281186 2.17 MB/sec execute 543 sec latency 16.153 ms 2024-09-19T06:15:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:38 smithi157 ceph-mon[35346]: pgmap v384: 97 pgs: 97 active+clean; 153 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-09-19T06:15:38.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 281725 2.18 MB/sec execute 544 sec latency 22.045 ms 2024-09-19T06:15:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:38 smithi005 ceph-mon[30955]: pgmap v384: 97 pgs: 97 active+clean; 153 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-09-19T06:15:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:38 smithi049 ceph-mon[34640]: pgmap v384: 97 pgs: 97 active+clean; 153 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-09-19T06:15:39.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 282209 2.17 MB/sec execute 545 sec latency 25.307 ms 2024-09-19T06:15:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:40 smithi157 ceph-mon[35346]: pgmap v385: 97 pgs: 97 active+clean; 153 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:40.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 282728 2.17 MB/sec execute 546 sec latency 21.653 ms 2024-09-19T06:15:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:40 smithi005 ceph-mon[30955]: pgmap v385: 97 pgs: 97 active+clean; 153 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:40 smithi049 ceph-mon[34640]: pgmap v385: 97 pgs: 97 active+clean; 153 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:41.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 283509 2.18 MB/sec execute 547 sec latency 14.971 ms 2024-09-19T06:15:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:42 smithi157 ceph-mon[35346]: pgmap v386: 97 pgs: 97 active+clean; 153 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:42 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:42.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 283856 2.18 MB/sec execute 548 sec latency 68.302 ms 2024-09-19T06:15:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:42 smithi005 ceph-mon[30955]: pgmap v386: 97 pgs: 97 active+clean; 153 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:42 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:42 smithi049 ceph-mon[34640]: pgmap v386: 97 pgs: 97 active+clean; 153 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:42 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:15:43.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 283856 2.18 MB/sec execute 549 sec latency 1068.405 ms 2024-09-19T06:15:44.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:43 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:43 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:43 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:43 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:43 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:43 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:43 smithi157 ceph-mon[35346]: pgmap v387: 97 pgs: 97 active+clean; 156 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.7 MiB/s wr, 158 op/s 2024-09-19T06:15:44.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:43 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:44.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:43 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:44.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:43 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:43 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:43 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:43 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:43 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:43 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:43 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:43 smithi005 ceph-mon[30955]: pgmap v387: 97 pgs: 97 active+clean; 156 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.7 MiB/s wr, 158 op/s 2024-09-19T06:15:44.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:43 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:44.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:43 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:44.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:43 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:43 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:43 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:43 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:43 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:43 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:43 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:43 smithi049 ceph-mon[34640]: pgmap v387: 97 pgs: 97 active+clean; 156 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.7 MiB/s wr, 158 op/s 2024-09-19T06:15:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:43 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:15:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:43 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:15:44.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:43 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:15:44.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 283856 2.17 MB/sec execute 550 sec latency 2068.488 ms 2024-09-19T06:15:45.848 INFO:tasks.workunit.client.0.smithi005.stdout: 1 283923 2.17 MB/sec execute 551 sec latency 2695.464 ms 2024-09-19T06:15:46.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:45 smithi005 ceph-mon[30955]: pgmap v388: 97 pgs: 97 active+clean; 156 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:46.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:45 smithi049 ceph-mon[34640]: pgmap v388: 97 pgs: 97 active+clean; 156 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:46.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:45 smithi157 ceph-mon[35346]: pgmap v388: 97 pgs: 97 active+clean; 156 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:15:46.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 283923 2.17 MB/sec execute 552 sec latency 1092.683 ms 2024-09-19T06:15:47.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 283923 2.16 MB/sec execute 553 sec latency 2092.796 ms 2024-09-19T06:15:48.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:47 smithi005 ceph-mon[30955]: pgmap v389: 97 pgs: 97 active+clean; 157 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-09-19T06:15:48.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:47 smithi049 ceph-mon[34640]: pgmap v389: 97 pgs: 97 active+clean; 157 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-09-19T06:15:48.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:47 smithi157 ceph-mon[35346]: pgmap v389: 97 pgs: 97 active+clean; 157 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-09-19T06:15:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.583+0000 7f043a39f640 1 -- 172.21.15.5:0/1212482936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043410c0f0 msgr2=0x7f043410e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.583+0000 7f043a39f640 1 --2- 172.21.15.5:0/1212482936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043410c0f0 0x7f043410e570 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f042c009ca0 tx=0x7f042c02a950 comp rx=0 tx=0).stop 2024-09-19T06:15:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.583+0000 7f043a39f640 1 -- 172.21.15.5:0/1212482936 shutdown_connections 2024-09-19T06:15:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.583+0000 7f043a39f640 1 --2- 172.21.15.5:0/1212482936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043410c0f0 0x7f043410e570 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.583+0000 7f043a39f640 1 --2- 172.21.15.5:0/1212482936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04341035c0 0x7f043410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.583+0000 7f043a39f640 1 --2- 172.21.15.5:0/1212482936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0434102ca0 0x7f0434103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.583+0000 7f043a39f640 1 -- 172.21.15.5:0/1212482936 >> 172.21.15.5:0/1212482936 conn(0x7f04340fc6b0 msgr2=0x7f04340fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.583+0000 7f043a39f640 1 -- 172.21.15.5:0/1212482936 shutdown_connections 2024-09-19T06:15:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.583+0000 7f043a39f640 1 -- 172.21.15.5:0/1212482936 wait complete. 2024-09-19T06:15:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.584+0000 7f043a39f640 1 Processor -- start 2024-09-19T06:15:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.584+0000 7f043a39f640 1 -- start start 2024-09-19T06:15:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.584+0000 7f043a39f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0434102ca0 0x7f04341982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.584+0000 7f043a39f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04341035c0 0x7f0434198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.584+0000 7f043a39f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043410c0f0 0x7f043419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.584+0000 7f043a39f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0434111dd0 con 0x7f043410c0f0 2024-09-19T06:15:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.584+0000 7f043a39f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0434111c50 con 0x7f0434102ca0 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.584+0000 7f043a39f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0434111f50 con 0x7f04341035c0 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.584+0000 7f043893c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043410c0f0 0x7f043419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.584+0000 7f043893c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043410c0f0 0x7f043419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38972/0 (socket says 172.21.15.5:38972) 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.584+0000 7f043893c640 1 -- 172.21.15.5:0/1471163956 learned_addr learned my addr 172.21.15.5:0/1471163956 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.584+0000 7f042bfff640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04341035c0 0x7f0434198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.585+0000 7f0433fff640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0434102ca0 0x7f04341982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.585+0000 7f043893c640 1 -- 172.21.15.5:0/1471163956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04341035c0 msgr2=0x7f0434198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.585+0000 7f043893c640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04341035c0 0x7f0434198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.585+0000 7f043893c640 1 -- 172.21.15.5:0/1471163956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0434102ca0 msgr2=0x7f04341982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.585+0000 7f043893c640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0434102ca0 0x7f04341982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.585+0000 7f043893c640 1 -- 172.21.15.5:0/1471163956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043419d2d0 con 0x7f043410c0f0 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.585+0000 7f043893c640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043410c0f0 0x7f043419cba0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f042c009dc0 tx=0x7f042c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.585+0000 7f0431ffb640 1 -- 172.21.15.5:0/1471163956 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c034040 con 0x7f043410c0f0 2024-09-19T06:15:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.585+0000 7f0431ffb640 1 -- 172.21.15.5:0/1471163956 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f042c030850 con 0x7f043410c0f0 2024-09-19T06:15:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.585+0000 7f043a39f640 1 -- 172.21.15.5:0/1471163956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043419d560 con 0x7f043410c0f0 2024-09-19T06:15:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.585+0000 7f0431ffb640 1 -- 172.21.15.5:0/1471163956 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c004670 con 0x7f043410c0f0 2024-09-19T06:15:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.585+0000 7f043a39f640 1 -- 172.21.15.5:0/1471163956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04341a4e50 con 0x7f043410c0f0 2024-09-19T06:15:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.588+0000 7f0431ffb640 1 -- 172.21.15.5:0/1471163956 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f042c037020 con 0x7f043410c0f0 2024-09-19T06:15:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.588+0000 7f043d219740 1 Processor -- start 2024-09-19T06:15:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.588+0000 7f043d219740 1 -- start start 2024-09-19T06:15:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.588+0000 7f043d219740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 0x55bc7eeaddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.588+0000 7f043d219740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eeae2f0 0x55bc7eebcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.588+0000 7f043d219740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eebd0c0 0x55bc7eebf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.588+0000 7f043d219740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bc7ee122f0 con 0x55bc7eead9d0 2024-09-19T06:15:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.588+0000 7f043d219740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bc7edba270 con 0x55bc7eeae2f0 2024-09-19T06:15:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.588+0000 7f043d219740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bc7ee0e2b0 con 0x55bc7eebd0c0 2024-09-19T06:15:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.588+0000 7f042bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 0x55bc7eeaddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.588+0000 7f042bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 0x55bc7eeaddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38976/0 (socket says 172.21.15.5:38976) 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.588+0000 7f042bfff640 1 -- 172.21.15.5:0/2313685075 learned_addr learned my addr 172.21.15.5:0/2313685075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f0433fff640 1 --2- 172.21.15.5:0/2313685075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eeae2f0 0x55bc7eebcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f0431ffb640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f04200a33b0 0x7f04200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f042bfff640 1 --2- 172.21.15.5:0/2313685075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eebd0c0 0x55bc7eebf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f042bfff640 1 -- 172.21.15.5:0/2313685075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eebd0c0 msgr2=0x55bc7eebf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f042bfff640 1 --2- 172.21.15.5:0/2313685075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eebd0c0 0x55bc7eebf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f042bfff640 1 -- 172.21.15.5:0/2313685075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eeae2f0 msgr2=0x55bc7eebcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f042bfff640 1 --2- 172.21.15.5:0/2313685075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eeae2f0 0x55bc7eebcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f042bfff640 1 -- 172.21.15.5:0/2313685075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc7edb9340 con 0x55bc7eead9d0 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f043893c640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f04200a33b0 0x7f04200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f0431ffb640 1 -- 172.21.15.5:0/1471163956 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f042c0f0ab0 con 0x7f043410c0f0 2024-09-19T06:15:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f042bfff640 1 --2- 172.21.15.5:0/2313685075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 0x55bc7eeaddb0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f041c00bb10 tx=0x7f041c0041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62899360e230a8b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f042a7fc640 1 -- 172.21.15.5:0/2313685075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c003ec0 con 0x55bc7eead9d0 2024-09-19T06:15:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f042a7fc640 1 -- 172.21.15.5:0/2313685075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f041c0054d0 con 0x55bc7eead9d0 2024-09-19T06:15:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.589+0000 7f042a7fc640 1 -- 172.21.15.5:0/2313685075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c010460 con 0x55bc7eead9d0 2024-09-19T06:15:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.590+0000 7f043d219740 1 -- 172.21.15.5:0/2313685075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 msgr2=0x55bc7eeaddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.590+0000 7f043893c640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f04200a33b0 0x7f04200a5870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f042c0bce10 tx=0x7f042c08bb70 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.590+0000 7f043d219740 1 --2- 172.21.15.5:0/2313685075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 0x55bc7eeaddb0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f041c00bb10 tx=0x7f041c0041d0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.590+0000 7f043d219740 1 -- 172.21.15.5:0/2313685075 shutdown_connections 2024-09-19T06:15:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.590+0000 7f043d219740 1 --2- 172.21.15.5:0/2313685075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eebd0c0 0x55bc7eebf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.590+0000 7f043d219740 1 --2- 172.21.15.5:0/2313685075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eeae2f0 0x55bc7eebcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.590+0000 7f043d219740 1 --2- 172.21.15.5:0/2313685075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 0x55bc7eeaddb0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.590+0000 7f043d219740 1 -- 172.21.15.5:0/2313685075 >> 172.21.15.5:0/2313685075 conn(0x55bc7eeaa060 msgr2=0x55bc7eeaa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.590+0000 7f043d219740 1 -- 172.21.15.5:0/2313685075 shutdown_connections 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.590+0000 7f043d219740 1 -- 172.21.15.5:0/2313685075 wait complete. 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.590+0000 7f043d219740 1 -- start start 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f043d219740 1 Processor -- start 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f043d219740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 0x55bc7eeba030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f043d219740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eeae2f0 0x55bc7eeba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f043d219740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eebd0c0 0x55bc7eebaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f043d219740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bc7eebfc90 con 0x55bc7eead9d0 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f043d219740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bc7ed9b290 con 0x55bc7eebd0c0 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f043d219740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bc7eebfe10 con 0x55bc7eeae2f0 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f043d219740 10 client.0 _ll_register_callbacks cb 0x55bc7edcc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f042bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 0x55bc7eeba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f042bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 0x55bc7eeba030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38988/0 (socket says 172.21.15.5:38988) 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f042bfff640 1 -- 172.21.15.5:0/4288871439 learned_addr learned my addr 172.21.15.5:0/4288871439 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f0433fff640 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eeae2f0 0x55bc7eeba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f042bfff640 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eebd0c0 0x55bc7eebaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f042bfff640 1 -- 172.21.15.5:0/4288871439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eeae2f0 msgr2=0x55bc7eeba570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.591+0000 7f042bfff640 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eeae2f0 0x55bc7eeba570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f042bfff640 1 -- 172.21.15.5:0/4288871439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eebd0c0 msgr2=0x55bc7eebaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f042bfff640 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eebd0c0 0x55bc7eebaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f042bfff640 1 -- 172.21.15.5:0/4288871439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc7eb25210 con 0x55bc7eead9d0 2024-09-19T06:15:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f042bfff640 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 0x55bc7eeba030 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f041c00b540 tx=0x7f041c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e2d272d342b5c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f04137fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f04137fe640 1 -- 172.21.15.5:0/4288871439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c029020 con 0x55bc7eead9d0 2024-09-19T06:15:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f04137fe640 1 -- 172.21.15.5:0/4288871439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f041c005000 con 0x55bc7eead9d0 2024-09-19T06:15:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f043d219740 1 -- 172.21.15.5:0/4288871439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f041c0054d0 con 0x55bc7eead9d0 2024-09-19T06:15:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f04137fe640 1 -- 172.21.15.5:0/4288871439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f041c003ec0 con 0x55bc7eead9d0 2024-09-19T06:15:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f043d219740 10 client.18657 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f043d219740 1 -- 172.21.15.5:0/4288871439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bc7ebf1620 con 0x55bc7eead9d0 2024-09-19T06:15:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f04137fe640 1 -- 172.21.15.5:0/4288871439 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f041c0054d0 con 0x55bc7eead9d0 2024-09-19T06:15:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f04137fe640 1 -- 172.21.15.5:0/4288871439 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f041c022410 con 0x55bc7eead9d0 2024-09-19T06:15:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f043d219740 10 client.18657 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f043d219740 10 client.18657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.592+0000 7f043d219740 20 client.18657 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.593+0000 7f043d219740 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bc7eecd020 0x55bc7eecf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.593+0000 7f043d219740 4 client.18657 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.593+0000 7f043d219740 1 -- 172.21.15.5:0/4288871439 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc7ee1fa20 con 0x55bc7eecd020 2024-09-19T06:15:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.593+0000 7f0433fff640 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bc7eecd020 0x55bc7eecf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.593+0000 7f0433fff640 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bc7eecd020 0x55bc7eecf410 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d23830718521cce server_cookie=76810ada49dcb476 in_seq=0 out_seq=0 2024-09-19T06:15:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.593+0000 7f04137fe640 10 client.18657 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f04137fe640 1 -- 172.21.15.5:0/4288871439 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc7ee1fa20 con 0x55bc7eecd020 2024-09-19T06:15:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f04137fe640 10 client.18657 handle_command_reply: tid=0 2024-09-19T06:15:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f043d219740 1 client.18657 shutdown 2024-09-19T06:15:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f043d219740 1 -- 172.21.15.5:0/4288871439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 msgr2=0x55bc7eeba030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f043d219740 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 0x55bc7eeba030 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f041c00b540 tx=0x7f041c005f90 comp rx=0 tx=0).stop 2024-09-19T06:15:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f043d219740 1 -- 172.21.15.5:0/4288871439 shutdown_connections 2024-09-19T06:15:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f043d219740 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bc7eecd020 0x55bc7eecf410 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f043d219740 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eebd0c0 0x55bc7eebaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f043d219740 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eeae2f0 0x55bc7eeba570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f04137fe640 0 client.18657 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f043d219740 1 --2- 172.21.15.5:0/4288871439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eead9d0 0x55bc7eeba030 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f043d219740 1 -- 172.21.15.5:0/4288871439 >> 172.21.15.5:0/4288871439 conn(0x55bc7eeaa060 msgr2=0x55bc7eeafdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f043d219740 1 -- 172.21.15.5:0/4288871439 shutdown_connections 2024-09-19T06:15:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f043d219740 1 -- 172.21.15.5:0/4288871439 wait complete. 2024-09-19T06:15:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.594+0000 7f043d219740 20 client.18657 trim_cache size 0 max 16384 2024-09-19T06:15:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.609+0000 7f043d219740 1 Processor -- start 2024-09-19T06:15:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.609+0000 7f043d219740 1 -- start start 2024-09-19T06:15:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.609+0000 7f043d219740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eead9d0 0x55bc7eed4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.609+0000 7f043d219740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 0x55bc7eed5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.609+0000 7f043d219740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eebd0c0 0x55bc7eeb7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.609+0000 7f043d219740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04341ab5b0 con 0x55bc7eeae2f0 2024-09-19T06:15:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.609+0000 7f043d219740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f04341ad1b0 con 0x55bc7eebd0c0 2024-09-19T06:15:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.609+0000 7f043d219740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f04341abc80 con 0x55bc7eead9d0 2024-09-19T06:15:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.609+0000 7f0433fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 0x55bc7eed5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.609+0000 7f0433fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 0x55bc7eed5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38994/0 (socket says 172.21.15.5:38994) 2024-09-19T06:15:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.609+0000 7f0433fff640 1 -- 172.21.15.5:0/2688337789 learned_addr learned my addr 172.21.15.5:0/2688337789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.610+0000 7f042bfff640 1 --2- 172.21.15.5:0/2688337789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eebd0c0 0x55bc7eeb7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.610+0000 7f042bfff640 1 --2- 172.21.15.5:0/2688337789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eead9d0 0x55bc7eed4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.610+0000 7f0433fff640 1 -- 172.21.15.5:0/2688337789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eead9d0 msgr2=0x55bc7eed4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.610+0000 7f0433fff640 1 --2- 172.21.15.5:0/2688337789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eead9d0 0x55bc7eed4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.610+0000 7f0433fff640 1 -- 172.21.15.5:0/2688337789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eebd0c0 msgr2=0x55bc7eeb7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.610+0000 7f0433fff640 1 --2- 172.21.15.5:0/2688337789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eebd0c0 0x55bc7eeb7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.610+0000 7f0433fff640 1 -- 172.21.15.5:0/2688337789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc7eea9e90 con 0x55bc7eeae2f0 2024-09-19T06:15:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.610+0000 7f0433fff640 1 --2- 172.21.15.5:0/2688337789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 0x55bc7eed5110 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f0424000c00 tx=0x7f0424005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50915dc32d013c9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.610+0000 7f0429ffb640 1 -- 172.21.15.5:0/2688337789 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424004250 con 0x55bc7eeae2f0 2024-09-19T06:15:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.610+0000 7f0429ffb640 1 -- 172.21.15.5:0/2688337789 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04240043f0 con 0x55bc7eeae2f0 2024-09-19T06:15:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.610+0000 7f0429ffb640 1 -- 172.21.15.5:0/2688337789 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042401e880 con 0x55bc7eeae2f0 2024-09-19T06:15:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.611+0000 7f043d219740 1 -- 172.21.15.5:0/2688337789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 msgr2=0x55bc7eed5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.611+0000 7f043d219740 1 --2- 172.21.15.5:0/2688337789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 0x55bc7eed5110 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f0424000c00 tx=0x7f0424005f90 comp rx=0 tx=0).stop 2024-09-19T06:15:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.611+0000 7f043d219740 1 -- 172.21.15.5:0/2688337789 shutdown_connections 2024-09-19T06:15:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.611+0000 7f043d219740 1 --2- 172.21.15.5:0/2688337789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eebd0c0 0x55bc7eeb7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.611+0000 7f043d219740 1 --2- 172.21.15.5:0/2688337789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 0x55bc7eed5110 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.611+0000 7f043d219740 1 --2- 172.21.15.5:0/2688337789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eead9d0 0x55bc7eed4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.611+0000 7f043d219740 1 -- 172.21.15.5:0/2688337789 >> 172.21.15.5:0/2688337789 conn(0x55bc7eeaa060 msgr2=0x55bc7eebc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.611+0000 7f043d219740 1 -- 172.21.15.5:0/2688337789 shutdown_connections 2024-09-19T06:15:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.611+0000 7f043d219740 1 -- 172.21.15.5:0/2688337789 wait complete. 2024-09-19T06:15:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.611+0000 7f043d219740 1 -- start start 2024-09-19T06:15:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.611+0000 7f043d219740 1 Processor -- start 2024-09-19T06:15:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f043d219740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eead9d0 0x55bc7eec0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f043d219740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 0x55bc7eec14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f043d219740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eebd0c0 0x55bc7eec1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f043d219740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04341acae0 con 0x55bc7eeae2f0 2024-09-19T06:15:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f043d219740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f04341ac3b0 con 0x55bc7eead9d0 2024-09-19T06:15:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f043d219740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f04341a95b0 con 0x55bc7eebd0c0 2024-09-19T06:15:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f043d219740 10 client.0 _ll_register_callbacks cb 0x7f0424009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f0433fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 0x55bc7eec14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f0433fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 0x55bc7eec14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39000/0 (socket says 172.21.15.5:39000) 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f0433fff640 1 -- 172.21.15.5:0/2329505495 learned_addr learned my addr 172.21.15.5:0/2329505495 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f042bfff640 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eead9d0 0x55bc7eec0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f042bfff640 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eebd0c0 0x55bc7eec1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f0433fff640 1 -- 172.21.15.5:0/2329505495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eebd0c0 msgr2=0x55bc7eec1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f0433fff640 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eebd0c0 0x55bc7eec1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f0433fff640 1 -- 172.21.15.5:0/2329505495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eead9d0 msgr2=0x55bc7eec0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f0433fff640 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eead9d0 0x55bc7eec0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f0433fff640 1 -- 172.21.15.5:0/2329505495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc7ef36d40 con 0x55bc7eeae2f0 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f042bfff640 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eebd0c0 0x55bc7eec1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f0433fff640 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 0x55bc7eec14f0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0424009bb0 tx=0x7f0424004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad20f077e6e4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.612+0000 7f04137fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f04137fe640 1 -- 172.21.15.5:0/2329505495 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424005030 con 0x55bc7eeae2f0 2024-09-19T06:15:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f043d219740 1 -- 172.21.15.5:0/2329505495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04240043f0 con 0x55bc7eeae2f0 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f04137fe640 1 -- 172.21.15.5:0/2329505495 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f04240051d0 con 0x55bc7eeae2f0 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f04137fe640 1 -- 172.21.15.5:0/2329505495 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f04240053d0 con 0x55bc7eeae2f0 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f043d219740 10 client.18663 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f043d219740 1 -- 172.21.15.5:0/2329505495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bc7ef38040 con 0x55bc7eeae2f0 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f04137fe640 1 -- 172.21.15.5:0/2329505495 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04240043f0 con 0x55bc7eeae2f0 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f04137fe640 1 -- 172.21.15.5:0/2329505495 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f0424003710 con 0x55bc7eeae2f0 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f043d219740 10 client.18663 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f043d219740 10 client.18663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f043d219740 20 client.18663 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f043d219740 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bc7ef38740 0x55bc7ef3ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f043d219740 4 client.18663 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f043d219740 1 -- 172.21.15.5:0/2329505495 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc7ee19d70 con 0x55bc7ef38740 2024-09-19T06:15:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.613+0000 7f042bfff640 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bc7ef38740 0x55bc7ef3ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.614+0000 7f042bfff640 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bc7ef38740 0x55bc7ef3ab60 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f076a11e36c92be server_cookie=243485fe57c05ddb in_seq=0 out_seq=0 2024-09-19T06:15:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.614+0000 7f04137fe640 10 client.18663 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.643+0000 7f04137fe640 1 -- 172.21.15.5:0/2329505495 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc7ee19d70 con 0x55bc7ef38740 2024-09-19T06:15:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.643+0000 7f04137fe640 10 client.18663 handle_command_reply: tid=0 2024-09-19T06:15:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.643+0000 7f043d219740 1 client.18663 shutdown 2024-09-19T06:15:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.644+0000 7f043d219740 1 -- 172.21.15.5:0/2329505495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 msgr2=0x55bc7eec14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.644+0000 7f043d219740 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 0x55bc7eec14f0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0424009bb0 tx=0x7f0424004050 comp rx=0 tx=0).stop 2024-09-19T06:15:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.644+0000 7f043d219740 1 -- 172.21.15.5:0/2329505495 shutdown_connections 2024-09-19T06:15:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.644+0000 7f043d219740 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bc7ef38740 0x55bc7ef3ab60 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.644+0000 7f043d219740 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc7eebd0c0 0x55bc7eec1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.644+0000 7f043d219740 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc7eeae2f0 0x55bc7eec14f0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.644+0000 7f04137fe640 0 client.18663 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.644+0000 7f043d219740 1 --2- 172.21.15.5:0/2329505495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc7eead9d0 0x55bc7eec0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.644+0000 7f043d219740 1 -- 172.21.15.5:0/2329505495 >> 172.21.15.5:0/2329505495 conn(0x55bc7eeaa060 msgr2=0x55bc7eebb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.644+0000 7f043d219740 1 -- 172.21.15.5:0/2329505495 shutdown_connections 2024-09-19T06:15:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.644+0000 7f043d219740 1 -- 172.21.15.5:0/2329505495 wait complete. 2024-09-19T06:15:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.644+0000 7f043d219740 20 client.18663 trim_cache size 0 max 16384 2024-09-19T06:15:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:15:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:15:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:15:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:15:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.645+0000 7f043a39f640 1 -- 172.21.15.5:0/1471163956 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f04200a33b0 msgr2=0x7f04200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.645+0000 7f043a39f640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f04200a33b0 0x7f04200a5870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f042c0bce10 tx=0x7f042c08bb70 comp rx=0 tx=0).stop 2024-09-19T06:15:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.645+0000 7f043a39f640 1 -- 172.21.15.5:0/1471163956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043410c0f0 msgr2=0x7f043419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.645+0000 7f043a39f640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043410c0f0 0x7f043419cba0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f042c009dc0 tx=0x7f042c003f10 comp rx=0 tx=0).stop 2024-09-19T06:15:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.645+0000 7f043a39f640 1 -- 172.21.15.5:0/1471163956 shutdown_connections 2024-09-19T06:15:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.645+0000 7f043a39f640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043410c0f0 0x7f043419cba0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.645+0000 7f043a39f640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f04200a33b0 0x7f04200a5870 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.645+0000 7f043a39f640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04341035c0 0x7f0434198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.645+0000 7f043a39f640 1 --2- 172.21.15.5:0/1471163956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0434102ca0 0x7f04341982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.645+0000 7f043a39f640 1 -- 172.21.15.5:0/1471163956 >> 172.21.15.5:0/1471163956 conn(0x7f04340fc6b0 msgr2=0x7f04340fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.645+0000 7f043a39f640 1 -- 172.21.15.5:0/1471163956 shutdown_connections 2024-09-19T06:15:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:15:48.645+0000 7f043a39f640 1 -- 172.21.15.5:0/1471163956 wait complete. 2024-09-19T06:15:48.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 283923 2.16 MB/sec execute 554 sec latency 3092.919 ms 2024-09-19T06:15:49.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 283923 2.16 MB/sec execute 555 sec latency 4092.965 ms 2024-09-19T06:15:50.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:49 smithi005 ceph-mon[30955]: pgmap v390: 97 pgs: 97 active+clean; 157 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 843 KiB/s wr, 81 op/s 2024-09-19T06:15:50.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:49 smithi049 ceph-mon[34640]: pgmap v390: 97 pgs: 97 active+clean; 157 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 843 KiB/s wr, 81 op/s 2024-09-19T06:15:50.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:49 smithi157 ceph-mon[35346]: pgmap v390: 97 pgs: 97 active+clean; 157 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 843 KiB/s wr, 81 op/s 2024-09-19T06:15:50.649 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:15:50.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 284032 2.15 MB/sec execute 556 sec latency 4712.393 ms 2024-09-19T06:15:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:51.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.420+0000 7f9143eea640 1 -- 172.21.15.5:0/3031703965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f913c106a80 msgr2=0x7f913c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:51.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.420+0000 7f9143eea640 1 --2- 172.21.15.5:0/3031703965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f913c106a80 0x7f913c106e60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f9138009930 tx=0x7f913802a420 comp rx=0 tx=0).stop 2024-09-19T06:15:51.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.420+0000 7f9143eea640 1 -- 172.21.15.5:0/3031703965 shutdown_connections 2024-09-19T06:15:51.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.420+0000 7f9143eea640 1 --2- 172.21.15.5:0/3031703965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f913c101420 0x7f913c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.420+0000 7f9143eea640 1 --2- 172.21.15.5:0/3031703965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f913c100a80 0x7f913c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.420+0000 7f9143eea640 1 --2- 172.21.15.5:0/3031703965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f913c106a80 0x7f913c106e60 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.420+0000 7f9143eea640 1 -- 172.21.15.5:0/3031703965 >> 172.21.15.5:0/3031703965 conn(0x7f913c0fc7a0 msgr2=0x7f913c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.421+0000 7f9143eea640 1 -- 172.21.15.5:0/3031703965 shutdown_connections 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.421+0000 7f9143eea640 1 -- 172.21.15.5:0/3031703965 wait complete. 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.421+0000 7f9143eea640 1 Processor -- start 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.421+0000 7f9143eea640 1 -- start start 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.421+0000 7f9143eea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f913c100a80 0x7f913c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.421+0000 7f9143eea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f913c101420 0x7f913c1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9143eea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f913c106a80 0x7f913c19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9143eea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f913c110cf0 con 0x7f913c106a80 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9143eea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f913c110b70 con 0x7f913c100a80 2024-09-19T06:15:51.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9143eea640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f913c110e70 con 0x7f913c101420 2024-09-19T06:15:51.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9142487640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f913c106a80 0x7f913c19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9141c86640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f913c100a80 0x7f913c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9142487640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f913c106a80 0x7f913c19bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39022/0 (socket says 172.21.15.5:39022) 2024-09-19T06:15:51.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9142487640 1 -- 172.21.15.5:0/790761530 learned_addr learned my addr 172.21.15.5:0/790761530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:51.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9141485640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f913c101420 0x7f913c1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9142487640 1 -- 172.21.15.5:0/790761530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f913c101420 msgr2=0x7f913c1977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:51.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9142487640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f913c101420 0x7f913c1977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9142487640 1 -- 172.21.15.5:0/790761530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f913c100a80 msgr2=0x7f913c197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:51.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9142487640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f913c100a80 0x7f913c197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9142487640 1 -- 172.21.15.5:0/790761530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c19c240 con 0x7f913c106a80 2024-09-19T06:15:51.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9141c86640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f913c100a80 0x7f913c197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:51.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9141485640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f913c101420 0x7f913c1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9142487640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f913c106a80 0x7f913c19bb60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f913400eb10 tx=0x7f913400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.422+0000 7f9132ffd640 1 -- 172.21.15.5:0/790761530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134004b20 con 0x7f913c106a80 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.423+0000 7f9132ffd640 1 -- 172.21.15.5:0/790761530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9134004cc0 con 0x7f913c106a80 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.423+0000 7f9132ffd640 1 -- 172.21.15.5:0/790761530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134010690 con 0x7f913c106a80 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.423+0000 7f9143eea640 1 -- 172.21.15.5:0/790761530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913c19c530 con 0x7f913c106a80 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.423+0000 7f9143eea640 1 -- 172.21.15.5:0/790761530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913c1a3e10 con 0x7f913c106a80 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.426+0000 7f9132ffd640 1 -- 172.21.15.5:0/790761530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9134005020 con 0x7f913c106a80 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.426+0000 7f91462c3740 1 Processor -- start 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.426+0000 7f91462c3740 1 -- start start 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f91462c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a9142b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f91462c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a91438350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f91462c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 0x561a9143ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f91462c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a913912e0 con 0x561a9142b320 2024-09-19T06:15:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f91462c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561a91338ce0 con 0x561a91438890 2024-09-19T06:15:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f91462c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561a9138d2a0 con 0x561a9142bc40 2024-09-19T06:15:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9141485640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a9142b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9141485640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a9142b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39028/0 (socket says 172.21.15.5:39028) 2024-09-19T06:15:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9141485640 1 -- 172.21.15.5:0/635552450 learned_addr learned my addr 172.21.15.5:0/635552450 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9141485640 1 --2- 172.21.15.5:0/635552450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 0x561a9143ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9141c86640 1 --2- 172.21.15.5:0/635552450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a91438350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9132ffd640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91280a33b0 0x7f91280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9142487640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91280a33b0 0x7f91280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9141485640 1 -- 172.21.15.5:0/635552450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 msgr2=0x561a91438350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9141485640 1 --2- 172.21.15.5:0/635552450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a91438350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9141485640 1 -- 172.21.15.5:0/635552450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 msgr2=0x561a9143ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:51.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9141485640 1 --2- 172.21.15.5:0/635552450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 0x561a9143ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9141485640 1 -- 172.21.15.5:0/635552450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a9131d440 con 0x561a9142b320 2024-09-19T06:15:51.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9141c86640 1 --2- 172.21.15.5:0/635552450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a91438350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:51.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.427+0000 7f9132ffd640 1 -- 172.21.15.5:0/790761530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f91340d0250 con 0x7f913c106a80 2024-09-19T06:15:51.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.428+0000 7f9142487640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91280a33b0 0x7f91280a5870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f913409c4a0 tx=0x7f913406b200 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:51.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.428+0000 7f9141485640 1 --2- 172.21.15.5:0/635552450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a9142b700 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f912c013c20 tx=0x7f912c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c05d3631514d4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:51.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.428+0000 7f9127fff640 1 -- 172.21.15.5:0/635552450 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c013d10 con 0x561a9142b320 2024-09-19T06:15:51.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.428+0000 7f9127fff640 1 -- 172.21.15.5:0/635552450 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f912c013170 con 0x561a9142b320 2024-09-19T06:15:51.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.428+0000 7f9127fff640 1 -- 172.21.15.5:0/635552450 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c004a00 con 0x561a9142b320 2024-09-19T06:15:51.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.429+0000 7f91462c3740 1 -- 172.21.15.5:0/635552450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 msgr2=0x561a9142b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:51.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.429+0000 7f91462c3740 1 --2- 172.21.15.5:0/635552450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a9142b700 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f912c013c20 tx=0x7f912c009b50 comp rx=0 tx=0).stop 2024-09-19T06:15:51.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.429+0000 7f91462c3740 1 -- 172.21.15.5:0/635552450 shutdown_connections 2024-09-19T06:15:51.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.429+0000 7f91462c3740 1 --2- 172.21.15.5:0/635552450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 0x561a9143ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.429+0000 7f91462c3740 1 --2- 172.21.15.5:0/635552450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a91438350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.429+0000 7f91462c3740 1 --2- 172.21.15.5:0/635552450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a9142b700 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.429+0000 7f91462c3740 1 -- 172.21.15.5:0/635552450 >> 172.21.15.5:0/635552450 conn(0x561a91427a30 msgr2=0x561a91427e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:51.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.429+0000 7f91462c3740 1 -- 172.21.15.5:0/635552450 shutdown_connections 2024-09-19T06:15:51.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.429+0000 7f91462c3740 1 -- 172.21.15.5:0/635552450 wait complete. 2024-09-19T06:15:51.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.429+0000 7f91462c3740 1 -- start start 2024-09-19T06:15:51.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.429+0000 7f91462c3740 1 Processor -- start 2024-09-19T06:15:51.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f91462c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f91462c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a91435460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f91462c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 0x561a914359a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f91462c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a9143b460 con 0x561a9142b320 2024-09-19T06:15:51.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f91462c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561a9134c050 con 0x561a91438890 2024-09-19T06:15:51.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f91462c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561a9143b5e0 con 0x561a9142bc40 2024-09-19T06:15:51.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f91462c3740 10 client.0 _ll_register_callbacks cb 0x561a9138c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:51.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f9141485640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f9141485640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914381b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39040/0 (socket says 172.21.15.5:39040) 2024-09-19T06:15:51.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f9141485640 1 -- 172.21.15.5:0/2144235428 learned_addr learned my addr 172.21.15.5:0/2144235428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:51.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f9141c86640 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a91435460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f9141485640 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 0x561a914359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f9141485640 1 -- 172.21.15.5:0/2144235428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 msgr2=0x561a91435460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f9141485640 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a91435460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f9141485640 1 -- 172.21.15.5:0/2144235428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 msgr2=0x561a914359a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f9141485640 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 0x561a914359a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.430+0000 7f9141485640 1 -- 172.21.15.5:0/2144235428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a911023c0 con 0x561a9142b320 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.431+0000 7f9141485640 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914381b0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f912c006010 tx=0x7f912c005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1080acc19d994703 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.431+0000 7f91257fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.431+0000 7f91257fa640 1 -- 172.21.15.5:0/2144235428 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c0046c0 con 0x561a9142b320 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.431+0000 7f91257fa640 1 -- 172.21.15.5:0/2144235428 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f912c004860 con 0x561a9142b320 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.431+0000 7f91462c3740 1 -- 172.21.15.5:0/2144235428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f912c013170 con 0x561a9142b320 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.431+0000 7f91257fa640 1 -- 172.21.15.5:0/2144235428 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f912c033050 con 0x561a9142b320 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.431+0000 7f91462c3740 10 client.18675 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.431+0000 7f91462c3740 1 -- 172.21.15.5:0/2144235428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561a91446e20 con 0x561a9142b320 2024-09-19T06:15:51.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.431+0000 7f91257fa640 1 -- 172.21.15.5:0/2144235428 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f912c013170 con 0x561a9142b320 2024-09-19T06:15:51.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.431+0000 7f91257fa640 1 -- 172.21.15.5:0/2144235428 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f912c003a40 con 0x561a9142b320 2024-09-19T06:15:51.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.431+0000 7f91462c3740 10 client.18675 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:51.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.432+0000 7f91462c3740 10 client.18675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:51.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.432+0000 7f91462c3740 20 client.18675 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:51.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.432+0000 7f91462c3740 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a91448170 0x561a9144a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.432+0000 7f91462c3740 4 client.18675 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:51.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.432+0000 7f91462c3740 1 -- 172.21.15.5:0/2144235428 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a9139e850 con 0x561a91448170 2024-09-19T06:15:51.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.432+0000 7f9141c86640 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a91448170 0x561a9144a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.432+0000 7f91257fa640 10 client.18675 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:51.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.432+0000 7f9141c86640 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a91448170 0x561a9144a560 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35a0d6724e3b4ee8 server_cookie=485389e227b5d21d in_seq=0 out_seq=0 2024-09-19T06:15:51.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91257fa640 1 -- 172.21.15.5:0/2144235428 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a9139e850 con 0x561a91448170 2024-09-19T06:15:51.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91257fa640 10 client.18675 handle_command_reply: tid=0 2024-09-19T06:15:51.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91462c3740 1 client.18675 shutdown 2024-09-19T06:15:51.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91462c3740 1 -- 172.21.15.5:0/2144235428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 msgr2=0x561a914381b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:51.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91462c3740 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914381b0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f912c006010 tx=0x7f912c005b90 comp rx=0 tx=0).stop 2024-09-19T06:15:51.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91462c3740 1 -- 172.21.15.5:0/2144235428 shutdown_connections 2024-09-19T06:15:51.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91462c3740 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a91448170 0x561a9144a560 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91462c3740 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 0x561a914359a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91462c3740 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a91435460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91257fa640 0 client.18675 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:51.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91462c3740 1 --2- 172.21.15.5:0/2144235428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914381b0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91462c3740 1 -- 172.21.15.5:0/2144235428 >> 172.21.15.5:0/2144235428 conn(0x561a91427a30 msgr2=0x561a9142d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:51.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.433+0000 7f91462c3740 1 -- 172.21.15.5:0/2144235428 shutdown_connections 2024-09-19T06:15:51.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.434+0000 7f91462c3740 1 -- 172.21.15.5:0/2144235428 wait complete. 2024-09-19T06:15:51.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.434+0000 7f91462c3740 20 client.18675 trim_cache size 0 max 16384 2024-09-19T06:15:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.454+0000 7f91462c3740 1 Processor -- start 2024-09-19T06:15:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.454+0000 7f91462c3740 1 -- start start 2024-09-19T06:15:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f91462c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914446c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f91462c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a9144fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f91462c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 0x561a914501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f91462c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f913c1ab700 con 0x561a9142b320 2024-09-19T06:15:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f91462c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f913c1ad300 con 0x561a91438890 2024-09-19T06:15:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f91462c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f913c1abdd0 con 0x561a9142bc40 2024-09-19T06:15:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f9141485640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f9141485640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914446c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39056/0 (socket says 172.21.15.5:39056) 2024-09-19T06:15:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f9141485640 1 -- 172.21.15.5:0/2366237374 learned_addr learned my addr 172.21.15.5:0/2366237374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f9141c86640 1 --2- 172.21.15.5:0/2366237374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a9144fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f9141485640 1 --2- 172.21.15.5:0/2366237374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 0x561a914501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f9141485640 1 -- 172.21.15.5:0/2366237374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 msgr2=0x561a9144fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f9141485640 1 --2- 172.21.15.5:0/2366237374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a9144fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f9141485640 1 -- 172.21.15.5:0/2366237374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 msgr2=0x561a914501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f9141485640 1 --2- 172.21.15.5:0/2366237374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 0x561a914501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f9141485640 1 -- 172.21.15.5:0/2366237374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a91431060 con 0x561a9142b320 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f9141485640 1 --2- 172.21.15.5:0/2366237374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914446c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f912c031040 tx=0x7f912c0136d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b4ad04c4c65b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.455+0000 7f91277fe640 1 -- 172.21.15.5:0/2366237374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c0054e0 con 0x561a9142b320 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.456+0000 7f91277fe640 1 -- 172.21.15.5:0/2366237374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f912c003aa0 con 0x561a9142b320 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.456+0000 7f91277fe640 1 -- 172.21.15.5:0/2366237374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c013390 con 0x561a9142b320 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.456+0000 7f91462c3740 1 -- 172.21.15.5:0/2366237374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 msgr2=0x561a914446c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.456+0000 7f91462c3740 1 --2- 172.21.15.5:0/2366237374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914446c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f912c031040 tx=0x7f912c0136d0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.456+0000 7f91462c3740 1 -- 172.21.15.5:0/2366237374 shutdown_connections 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.456+0000 7f91462c3740 1 --2- 172.21.15.5:0/2366237374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a91438890 0x561a914501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.456+0000 7f91462c3740 1 --2- 172.21.15.5:0/2366237374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9142bc40 0x561a9144fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.456+0000 7f91462c3740 1 --2- 172.21.15.5:0/2366237374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914446c0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.456+0000 7f91462c3740 1 -- 172.21.15.5:0/2366237374 >> 172.21.15.5:0/2366237374 conn(0x561a91427a30 msgr2=0x561a91434550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.457+0000 7f91462c3740 1 -- 172.21.15.5:0/2366237374 shutdown_connections 2024-09-19T06:15:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.457+0000 7f91462c3740 1 -- 172.21.15.5:0/2366237374 wait complete. 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.457+0000 7f91462c3740 1 -- start start 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.457+0000 7f91462c3740 1 Processor -- start 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.457+0000 7f91462c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.457+0000 7f91462c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9142bc40 0x561a914ae380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.457+0000 7f91462c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a91438890 0x561a91440820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.457+0000 7f91462c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f913c1acc30 con 0x561a9142b320 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.457+0000 7f91462c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f913c1ac500 con 0x561a9142bc40 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.457+0000 7f91462c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f913c1a9700 con 0x561a91438890 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f91462c3740 10 client.0 _ll_register_callbacks cb 0x7f912c013770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f9141485640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f9141c86640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9142bc40 0x561a914ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f9141c86640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9142bc40 0x561a914ae380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59954/0 (socket says 172.21.15.5:59954) 2024-09-19T06:15:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f9141c86640 1 -- 172.21.15.5:0/2848329459 learned_addr learned my addr 172.21.15.5:0/2848329459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f9141485640 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a91438890 0x561a91440820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f9141c86640 1 -- 172.21.15.5:0/2848329459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a91438890 msgr2=0x561a91440820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f9141c86640 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a91438890 0x561a91440820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f9141c86640 1 -- 172.21.15.5:0/2848329459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 msgr2=0x561a914ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f9141c86640 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f9141c86640 1 -- 172.21.15.5:0/2848329459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a91440de0 con 0x561a9142bc40 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f9141485640 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a91438890 0x561a91440820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f9141c86640 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9142bc40 0x561a914ae380 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9138000c00 tx=0x7f9138004300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9187c7f00c91fc32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f91257fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.458+0000 7f91257fa640 1 -- 172.21.15.5:0/2848329459 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138035040 con 0x561a9142bc40 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91462c3740 1 -- 172.21.15.5:0/2848329459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f912c003aa0 con 0x561a9142bc40 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91257fa640 1 -- 172.21.15.5:0/2848329459 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f9138003a00 con 0x561a9142bc40 2024-09-19T06:15:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91257fa640 1 -- 172.21.15.5:0/2848329459 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9138003c00 con 0x561a9142bc40 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91462c3740 10 client.28130 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91462c3740 1 -- 172.21.15.5:0/2848329459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561a9142f280 con 0x561a9142bc40 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91257fa640 1 -- 172.21.15.5:0/2848329459 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f912c003aa0 con 0x561a9142bc40 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91257fa640 1 -- 172.21.15.5:0/2848329459 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f9138050030 con 0x561a9142bc40 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91462c3740 10 client.28130 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91462c3740 10 client.28130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91462c3740 20 client.28130 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91462c3740 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a914b5280 0x561a914b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91462c3740 4 client.28130 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f91462c3740 1 -- 172.21.15.5:0/2848329459 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561a91398d60 con 0x561a914b5280 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.459+0000 7f9141485640 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a914b5280 0x561a914b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f9141485640 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a914b5280 0x561a914b76a0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c6e51b0cbb19f64 server_cookie=781782c5beb8e2aa in_seq=0 out_seq=0 2024-09-19T06:15:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91257fa640 10 client.28130 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91257fa640 1 -- 172.21.15.5:0/2848329459 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561a91398d60 con 0x561a914b5280 2024-09-19T06:15:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91257fa640 10 client.28130 handle_command_reply: tid=0 2024-09-19T06:15:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91462c3740 1 client.28130 shutdown 2024-09-19T06:15:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91462c3740 1 -- 172.21.15.5:0/2848329459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9142bc40 msgr2=0x561a914ae380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91462c3740 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9142bc40 0x561a914ae380 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9138000c00 tx=0x7f9138004300 comp rx=0 tx=0).stop 2024-09-19T06:15:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91462c3740 1 -- 172.21.15.5:0/2848329459 shutdown_connections 2024-09-19T06:15:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91462c3740 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561a914b5280 0x561a914b76a0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91462c3740 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a91438890 0x561a91440820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91462c3740 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9142bc40 0x561a914ae380 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91462c3740 1 --2- 172.21.15.5:0/2848329459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9142b320 0x561a914ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91462c3740 1 -- 172.21.15.5:0/2848329459 >> 172.21.15.5:0/2848329459 conn(0x561a91427a30 msgr2=0x561a91436cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91257fa640 0 client.28130 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:15:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91462c3740 1 -- 172.21.15.5:0/2848329459 shutdown_connections 2024-09-19T06:15:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91462c3740 1 -- 172.21.15.5:0/2848329459 wait complete. 2024-09-19T06:15:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.460+0000 7f91462c3740 20 client.28130 trim_cache size 0 max 16384 2024-09-19T06:15:51.470 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:15:51.470 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:15:51.470 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:15:51.471 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:15:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.461+0000 7f9143eea640 1 -- 172.21.15.5:0/790761530 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91280a33b0 msgr2=0x7f91280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.461+0000 7f9143eea640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91280a33b0 0x7f91280a5870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f913409c4a0 tx=0x7f913406b200 comp rx=0 tx=0).stop 2024-09-19T06:15:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.461+0000 7f9143eea640 1 -- 172.21.15.5:0/790761530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f913c106a80 msgr2=0x7f913c19bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.461+0000 7f9143eea640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f913c106a80 0x7f913c19bb60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f913400eb10 tx=0x7f913400c2d0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.461+0000 7f9143eea640 1 -- 172.21.15.5:0/790761530 shutdown_connections 2024-09-19T06:15:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.461+0000 7f9143eea640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f913c106a80 0x7f913c19bb60 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.461+0000 7f9143eea640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91280a33b0 0x7f91280a5870 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.461+0000 7f9143eea640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f913c101420 0x7f913c1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.461+0000 7f9143eea640 1 --2- 172.21.15.5:0/790761530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f913c100a80 0x7f913c197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.461+0000 7f9143eea640 1 -- 172.21.15.5:0/790761530 >> 172.21.15.5:0/790761530 conn(0x7f913c0fc7a0 msgr2=0x7f913c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.461+0000 7f9143eea640 1 -- 172.21.15.5:0/790761530 shutdown_connections 2024-09-19T06:15:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:51.461+0000 7f9143eea640 1 -- 172.21.15.5:0/790761530 wait complete. 2024-09-19T06:15:51.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 284255 2.15 MB/sec execute 557 sec latency 59.928 ms 2024-09-19T06:15:52.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:51 smithi005 ceph-mon[30955]: pgmap v391: 97 pgs: 97 active+clean; 157 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 843 KiB/s wr, 81 op/s 2024-09-19T06:15:52.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:51 smithi049 ceph-mon[34640]: pgmap v391: 97 pgs: 97 active+clean; 157 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 843 KiB/s wr, 81 op/s 2024-09-19T06:15:52.256 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:15:52.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:15:52.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:51 smithi157 ceph-mon[35346]: pgmap v391: 97 pgs: 97 active+clean; 157 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 843 KiB/s wr, 81 op/s 2024-09-19T06:15:52.533 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:52.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 284649 2.15 MB/sec execute 558 sec latency 16.462 ms 2024-09-19T06:15:53.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.080+0000 7f8ab9247640 1 -- 172.21.15.5:0/2956347568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4102790 msgr2=0x7f8ab41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:53.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.080+0000 7f8ab9247640 1 --2- 172.21.15.5:0/2956347568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4102790 0x7f8ab41067e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009900 tx=0x7f8aa802a410 comp rx=0 tx=0).stop 2024-09-19T06:15:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.080+0000 7f8ab9247640 1 -- 172.21.15.5:0/2956347568 shutdown_connections 2024-09-19T06:15:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.080+0000 7f8ab9247640 1 --2- 172.21.15.5:0/2956347568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106f10 0x7f8ab410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.080+0000 7f8ab9247640 1 --2- 172.21.15.5:0/2956347568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4102790 0x7f8ab41067e0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.080+0000 7f8ab9247640 1 --2- 172.21.15.5:0/2956347568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4101de0 0x7f8ab41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.080+0000 7f8ab9247640 1 -- 172.21.15.5:0/2956347568 >> 172.21.15.5:0/2956347568 conn(0x7f8ab40fd8e0 msgr2=0x7f8ab40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.080+0000 7f8ab9247640 1 -- 172.21.15.5:0/2956347568 shutdown_connections 2024-09-19T06:15:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.080+0000 7f8ab9247640 1 -- 172.21.15.5:0/2956347568 wait complete. 2024-09-19T06:15:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.081+0000 7f8ab9247640 1 Processor -- start 2024-09-19T06:15:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.081+0000 7f8ab9247640 1 -- start start 2024-09-19T06:15:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.081+0000 7f8ab9247640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4101de0 0x7f8ab4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.081+0000 7f8ab9247640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4102790 0x7f8ab41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.081+0000 7f8ab9247640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106f10 0x7f8ab419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab9247640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111e20 con 0x7f8ab4101de0 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab9247640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111ca0 con 0x7f8ab4106f10 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab9247640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111fa0 con 0x7f8ab4102790 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4101de0 0x7f8ab4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4101de0 0x7f8ab4198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39080/0 (socket says 172.21.15.5:39080) 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab2d9d640 1 -- 172.21.15.5:0/1941699806 learned_addr learned my addr 172.21.15.5:0/1941699806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab359e640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106f10 0x7f8ab419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab259c640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4102790 0x7f8ab41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab2d9d640 1 -- 172.21.15.5:0/1941699806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4102790 msgr2=0x7f8ab41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab2d9d640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4102790 0x7f8ab41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab2d9d640 1 -- 172.21.15.5:0/1941699806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106f10 msgr2=0x7f8ab419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab2d9d640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106f10 0x7f8ab419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:53.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab2d9d640 1 -- 172.21.15.5:0/1941699806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab419d310 con 0x7f8ab4101de0 2024-09-19T06:15:53.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab259c640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4102790 0x7f8ab41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:53.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab2d9d640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4101de0 0x7f8ab4198360 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400e9f0 tx=0x7f8aa400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:53.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8a9bfff640 1 -- 172.21.15.5:0/1941699806 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400cd80 con 0x7f8ab4101de0 2024-09-19T06:15:53.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8ab9247640 1 -- 172.21.15.5:0/1941699806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab419d600 con 0x7f8ab4101de0 2024-09-19T06:15:53.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8a9bfff640 1 -- 172.21.15.5:0/1941699806 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8aa40108e0 con 0x7f8ab4101de0 2024-09-19T06:15:53.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.082+0000 7f8a9bfff640 1 -- 172.21.15.5:0/1941699806 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4018920 con 0x7f8ab4101de0 2024-09-19T06:15:53.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.083+0000 7f8ab9247640 1 -- 172.21.15.5:0/1941699806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab41a4ee0 con 0x7f8ab4101de0 2024-09-19T06:15:53.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.084+0000 7f8ab9247640 1 -- 172.21.15.5:0/1941699806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab41021c0 con 0x7f8ab4101de0 2024-09-19T06:15:53.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.085+0000 7f8a9bfff640 1 -- 172.21.15.5:0/1941699806 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8aa4010a90 con 0x7f8ab4101de0 2024-09-19T06:15:53.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.086+0000 7f8a9bfff640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8a9c0a33b0 0x7f8a9c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:53.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.086+0000 7f8a9bfff640 1 -- 172.21.15.5:0/1941699806 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f8aa40d00c0 con 0x7f8ab4101de0 2024-09-19T06:15:53.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.086+0000 7f8ab259c640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8a9c0a33b0 0x7f8a9c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:53.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.086+0000 7f8ab259c640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8a9c0a33b0 0x7f8a9c0a5870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8031440 tx=0x7f8aa8005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:53.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.091+0000 7f8a9bfff640 1 -- 172.21.15.5:0/1941699806 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8aa409ca80 con 0x7f8ab4101de0 2024-09-19T06:15:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.336+0000 7f8ab9247640 1 -- 172.21.15.5:0/1941699806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ab410a150 con 0x7f8ab4101de0 2024-09-19T06:15:53.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.337+0000 7f8a9bfff640 1 -- 172.21.15.5:0/1941699806 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11965 (secure 0 0 0) 0x7f8aa409c380 con 0x7f8ab4101de0 2024-09-19T06:15:53.339 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:15:53.339 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":39,"btime":"2024-09-19T06:14:16:386479+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:14:15.452508+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:15:53.339 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 39 2024-09-19T06:15:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.340+0000 7f8ab9247640 1 -- 172.21.15.5:0/1941699806 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8a9c0a33b0 msgr2=0x7f8a9c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.340+0000 7f8ab9247640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8a9c0a33b0 0x7f8a9c0a5870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8031440 tx=0x7f8aa8005a20 comp rx=0 tx=0).stop 2024-09-19T06:15:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.340+0000 7f8ab9247640 1 -- 172.21.15.5:0/1941699806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4101de0 msgr2=0x7f8ab4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.340+0000 7f8ab9247640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4101de0 0x7f8ab4198360 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400e9f0 tx=0x7f8aa400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:15:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.340+0000 7f8ab9247640 1 -- 172.21.15.5:0/1941699806 shutdown_connections 2024-09-19T06:15:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.340+0000 7f8ab9247640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106f10 0x7f8ab419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.340+0000 7f8ab9247640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8a9c0a33b0 0x7f8a9c0a5870 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.340+0000 7f8ab9247640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4102790 0x7f8ab41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.340+0000 7f8ab9247640 1 --2- 172.21.15.5:0/1941699806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4101de0 0x7f8ab4198360 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.340+0000 7f8ab9247640 1 -- 172.21.15.5:0/1941699806 >> 172.21.15.5:0/1941699806 conn(0x7f8ab40fd8e0 msgr2=0x7f8ab40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.340+0000 7f8ab9247640 1 -- 172.21.15.5:0/1941699806 shutdown_connections 2024-09-19T06:15:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:53.340+0000 7f8ab9247640 1 -- 172.21.15.5:0/1941699806 wait complete. 2024-09-19T06:15:53.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 285175 2.15 MB/sec execute 559 sec latency 26.550 ms 2024-09-19T06:15:53.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:53 smithi005 ceph-mon[30955]: pgmap v392: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-09-19T06:15:53.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:53 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1941699806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:15:54.001 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:15:54.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:53 smithi049 ceph-mon[34640]: pgmap v392: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-09-19T06:15:54.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:53 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1941699806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:15:54.281 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:54.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:53 smithi157 ceph-mon[35346]: pgmap v392: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-09-19T06:15:54.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:53 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1941699806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:15:54.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 285606 2.15 MB/sec execute 560 sec latency 49.435 ms 2024-09-19T06:15:54.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.884+0000 7f332865f640 1 -- 172.21.15.5:0/1924421860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3320106f10 msgr2=0x7f332010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.884+0000 7f332865f640 1 --2- 172.21.15.5:0/1924421860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3320106f10 0x7f332010ea90 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3318009900 tx=0x7f331802a410 comp rx=0 tx=0).stop 2024-09-19T06:15:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.885+0000 7f332865f640 1 -- 172.21.15.5:0/1924421860 shutdown_connections 2024-09-19T06:15:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.885+0000 7f332865f640 1 --2- 172.21.15.5:0/1924421860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3320106f10 0x7f332010ea90 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.885+0000 7f332865f640 1 --2- 172.21.15.5:0/1924421860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3320102790 0x7f33201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.885+0000 7f332865f640 1 --2- 172.21.15.5:0/1924421860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320101de0 0x7f33201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.885+0000 7f332865f640 1 -- 172.21.15.5:0/1924421860 >> 172.21.15.5:0/1924421860 conn(0x7f33200fd880 msgr2=0x7f33200ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.885+0000 7f332865f640 1 -- 172.21.15.5:0/1924421860 shutdown_connections 2024-09-19T06:15:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.885+0000 7f332865f640 1 -- 172.21.15.5:0/1924421860 wait complete. 2024-09-19T06:15:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.885+0000 7f332865f640 1 Processor -- start 2024-09-19T06:15:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.885+0000 7f332865f640 1 -- start start 2024-09-19T06:15:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f332865f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3320101de0 0x7f3320198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f332865f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3320102790 0x7f3320198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f332865f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320106f10 0x7f332019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f332865f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3320111e70 con 0x7f3320102790 2024-09-19T06:15:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f332865f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3320111cf0 con 0x7f3320101de0 2024-09-19T06:15:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f332865f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3320111ff0 con 0x7f3320106f10 2024-09-19T06:15:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f33263fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3320101de0 0x7f3320198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f3325bfa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3320102790 0x7f3320198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f33263fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3320101de0 0x7f3320198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40442/0 (socket says 172.21.15.5:40442) 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f33263fb640 1 -- 172.21.15.5:0/494748655 learned_addr learned my addr 172.21.15.5:0/494748655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f3326bfc640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320106f10 0x7f332019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f33263fb640 1 -- 172.21.15.5:0/494748655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320106f10 msgr2=0x7f332019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f33263fb640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320106f10 0x7f332019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f33263fb640 1 -- 172.21.15.5:0/494748655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3320102790 msgr2=0x7f3320198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f33263fb640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3320102790 0x7f3320198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f33263fb640 1 -- 172.21.15.5:0/494748655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332019d2c0 con 0x7f3320101de0 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f3325bfa640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3320102790 0x7f3320198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f3326bfc640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320106f10 0x7f332019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.886+0000 7f33263fb640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3320101de0 0x7f3320198310 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f331c00eb90 tx=0x7f331c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.887+0000 7f33177fe640 1 -- 172.21.15.5:0/494748655 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c004d20 con 0x7f3320101de0 2024-09-19T06:15:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.887+0000 7f332865f640 1 -- 172.21.15.5:0/494748655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332019d5b0 con 0x7f3320101de0 2024-09-19T06:15:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.887+0000 7f33177fe640 1 -- 172.21.15.5:0/494748655 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f331c00ee90 con 0x7f3320101de0 2024-09-19T06:15:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.887+0000 7f33177fe640 1 -- 172.21.15.5:0/494748655 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c010670 con 0x7f3320101de0 2024-09-19T06:15:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.887+0000 7f332865f640 1 -- 172.21.15.5:0/494748655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3320074420 con 0x7f3320101de0 2024-09-19T06:15:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.889+0000 7f33177fe640 1 -- 172.21.15.5:0/494748655 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f331c005020 con 0x7f3320101de0 2024-09-19T06:15:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.889+0000 7f332aa38740 1 Processor -- start 2024-09-19T06:15:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f332aa38740 1 -- start start 2024-09-19T06:15:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f33177fe640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f330c0a33b0 0x7f330c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f332aa38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e74b40 0x556081e73090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f332aa38740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e73a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f332aa38740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e7e370 0x556081e80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f332aa38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556081dd92f0 con 0x556081e74b40 2024-09-19T06:15:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f332aa38740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556081d80cf0 con 0x556081e7e370 2024-09-19T06:15:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f332aa38740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556081dd52b0 con 0x556081e735d0 2024-09-19T06:15:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f33177fe640 1 -- 172.21.15.5:0/494748655 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f331c0d0460 con 0x7f3320101de0 2024-09-19T06:15:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f3326bfc640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f330c0a33b0 0x7f330c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f33263fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e74b40 0x556081e73090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f33263fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e74b40 0x556081e73090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58748/0 (socket says 172.21.15.5:58748) 2024-09-19T06:15:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f33263fb640 1 -- 172.21.15.5:0/2958415359 learned_addr learned my addr 172.21.15.5:0/2958415359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f3325bfa640 1 --2- 172.21.15.5:0/2958415359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e73a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.890+0000 7f3326bfc640 1 --2- 172.21.15.5:0/2958415359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e7e370 0x556081e80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.891+0000 7f3325bfa640 1 -- 172.21.15.5:0/2958415359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e7e370 msgr2=0x556081e80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.891+0000 7f3325bfa640 1 --2- 172.21.15.5:0/2958415359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e7e370 0x556081e80760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.891+0000 7f3325bfa640 1 -- 172.21.15.5:0/2958415359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e74b40 msgr2=0x556081e73090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.891+0000 7f3325bfa640 1 --2- 172.21.15.5:0/2958415359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e74b40 0x556081e73090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.891+0000 7f3325bfa640 1 -- 172.21.15.5:0/2958415359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556081d653a0 con 0x556081e735d0 2024-09-19T06:15:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.891+0000 7f3326bfc640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f330c0a33b0 0x7f330c0a5870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3318031440 tx=0x7f33180041f0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.891+0000 7f3325bfa640 1 --2- 172.21.15.5:0/2958415359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e73a10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f33200fdca0 tx=0x7f33100107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f42a2f202a647da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.891+0000 7f3314ff9640 1 -- 172.21.15.5:0/2958415359 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331001c680 con 0x556081e735d0 2024-09-19T06:15:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.891+0000 7f3314ff9640 1 -- 172.21.15.5:0/2958415359 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f331001ccd0 con 0x556081e735d0 2024-09-19T06:15:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.891+0000 7f3314ff9640 1 -- 172.21.15.5:0/2958415359 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3310011d50 con 0x556081e735d0 2024-09-19T06:15:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.892+0000 7f332aa38740 1 -- 172.21.15.5:0/2958415359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 msgr2=0x556081e73a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.892+0000 7f332aa38740 1 --2- 172.21.15.5:0/2958415359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e73a10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f33200fdca0 tx=0x7f33100107a0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.892+0000 7f332aa38740 1 -- 172.21.15.5:0/2958415359 shutdown_connections 2024-09-19T06:15:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.892+0000 7f332aa38740 1 --2- 172.21.15.5:0/2958415359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e7e370 0x556081e80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.892+0000 7f332aa38740 1 --2- 172.21.15.5:0/2958415359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e73a10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.892+0000 7f332aa38740 1 --2- 172.21.15.5:0/2958415359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e74b40 0x556081e73090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.892+0000 7f332aa38740 1 -- 172.21.15.5:0/2958415359 >> 172.21.15.5:0/2958415359 conn(0x556081e3c310 msgr2=0x556081e6fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.892+0000 7f332aa38740 1 -- 172.21.15.5:0/2958415359 shutdown_connections 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.892+0000 7f332aa38740 1 -- 172.21.15.5:0/2958415359 wait complete. 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f332aa38740 1 -- start start 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f332aa38740 1 Processor -- start 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f332aa38740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e7dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f332aa38740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e74b40 0x556081e7b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f332aa38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e7e370 0x556081e7b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f332aa38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556081e81070 con 0x556081e7e370 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f332aa38740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556081d94060 con 0x556081e74b40 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f332aa38740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556081e811f0 con 0x556081e735d0 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f332aa38740 10 client.0 _ll_register_callbacks cb 0x556081ae78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f3326bfc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e7e370 0x556081e7b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f33263fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e7dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f3326bfc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e7e370 0x556081e7b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58764/0 (socket says 172.21.15.5:58764) 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f33263fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e7dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48764/0 (socket says 172.21.15.5:48764) 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f3326bfc640 1 -- 172.21.15.5:0/2555183557 learned_addr learned my addr 172.21.15.5:0/2555183557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.893+0000 7f3325bfa640 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e74b40 0x556081e7b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f3326bfc640 1 -- 172.21.15.5:0/2555183557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 msgr2=0x556081e7dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f3326bfc640 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e7dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f3326bfc640 1 -- 172.21.15.5:0/2555183557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e74b40 msgr2=0x556081e7b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f3326bfc640 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e74b40 0x556081e7b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f3326bfc640 1 -- 172.21.15.5:0/2555183557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556081b4a3c0 con 0x556081e7e370 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f3325bfa640 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e74b40 0x556081e7b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f3326bfc640 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e7e370 0x556081e7b640 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3318009900 tx=0x7f331802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784df7952c3cd55d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f3309ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f3309ffb640 1 -- 172.21.15.5:0/2555183557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331804f070 con 0x556081e7e370 2024-09-19T06:15:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f332aa38740 1 -- 172.21.15.5:0/2555183557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f331001ccd0 con 0x556081e7e370 2024-09-19T06:15:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f3309ffb640 1 -- 172.21.15.5:0/2555183557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f33180035f0 con 0x556081e7e370 2024-09-19T06:15:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f3309ffb640 1 -- 172.21.15.5:0/2555183557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f331805b050 con 0x556081e7e370 2024-09-19T06:15:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f332aa38740 10 client.18699 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.894+0000 7f332aa38740 1 -- 172.21.15.5:0/2555183557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556081e8cb40 con 0x556081e7e370 2024-09-19T06:15:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.895+0000 7f3309ffb640 1 -- 172.21.15.5:0/2555183557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f331001ccd0 con 0x556081e7e370 2024-09-19T06:15:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.895+0000 7f3309ffb640 1 -- 172.21.15.5:0/2555183557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f3318032710 con 0x556081e7e370 2024-09-19T06:15:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.895+0000 7f332aa38740 10 client.18699 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.895+0000 7f332aa38740 10 client.18699 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:15:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.895+0000 7f332aa38740 20 client.18699 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.895+0000 7f332aa38740 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556081e8df50 0x556081e90410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.895+0000 7f332aa38740 4 client.18699 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.895+0000 7f332aa38740 1 -- 172.21.15.5:0/2555183557 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556081de6860 con 0x556081e8df50 2024-09-19T06:15:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.895+0000 7f33263fb640 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556081e8df50 0x556081e90410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.896+0000 7f33263fb640 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556081e8df50 0x556081e90410 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c5a8d6d308bf6d3b server_cookie=633e560016088a0b in_seq=0 out_seq=0 2024-09-19T06:15:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.896+0000 7f3309ffb640 10 client.18699 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.896+0000 7f3309ffb640 1 -- 172.21.15.5:0/2555183557 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556081de6860 con 0x556081e8df50 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.896+0000 7f3309ffb640 10 client.18699 handle_command_reply: tid=0 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.896+0000 7f332aa38740 1 client.18699 shutdown 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.897+0000 7f332aa38740 1 -- 172.21.15.5:0/2555183557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e7e370 msgr2=0x556081e7b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.897+0000 7f332aa38740 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e7e370 0x556081e7b640 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3318009900 tx=0x7f331802a410 comp rx=0 tx=0).stop 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.897+0000 7f332aa38740 1 -- 172.21.15.5:0/2555183557 shutdown_connections 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.897+0000 7f332aa38740 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556081e8df50 0x556081e90410 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.897+0000 7f332aa38740 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e7e370 0x556081e7b640 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.897+0000 7f332aa38740 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e74b40 0x556081e7b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.897+0000 7f332aa38740 1 --2- 172.21.15.5:0/2555183557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e7dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.897+0000 7f332aa38740 1 -- 172.21.15.5:0/2555183557 >> 172.21.15.5:0/2555183557 conn(0x556081e3c310 msgr2=0x556081e75960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.897+0000 7f3309ffb640 0 client.18699 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:15:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.897+0000 7f332aa38740 1 -- 172.21.15.5:0/2555183557 shutdown_connections 2024-09-19T06:15:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.897+0000 7f332aa38740 1 -- 172.21.15.5:0/2555183557 wait complete. 2024-09-19T06:15:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.897+0000 7f332aa38740 20 client.18699 trim_cache size 0 max 16384 2024-09-19T06:15:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.911+0000 7f332aa38740 1 Processor -- start 2024-09-19T06:15:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f332aa38740 1 -- start start 2024-09-19T06:15:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f332aa38740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e8a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f332aa38740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e74b40 0x556081e6f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f332aa38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e7e370 0x556081e6fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f332aa38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33201afa80 con 0x556081e7e370 2024-09-19T06:15:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f332aa38740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33201b1680 con 0x556081e74b40 2024-09-19T06:15:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f332aa38740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33201b0150 con 0x556081e735d0 2024-09-19T06:15:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f33263fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e8a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f33263fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e8a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48768/0 (socket says 172.21.15.5:48768) 2024-09-19T06:15:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f3326bfc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e7e370 0x556081e6fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f3325bfa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e74b40 0x556081e6f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f33263fb640 1 -- 172.21.15.5:0/1897195329 learned_addr learned my addr 172.21.15.5:0/1897195329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f33263fb640 1 -- 172.21.15.5:0/1897195329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e74b40 msgr2=0x556081e6f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f33263fb640 1 --2- 172.21.15.5:0/1897195329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e74b40 0x556081e6f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f33263fb640 1 -- 172.21.15.5:0/1897195329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e7e370 msgr2=0x556081e6fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f33263fb640 1 --2- 172.21.15.5:0/1897195329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e7e370 0x556081e6fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f33263fb640 1 -- 172.21.15.5:0/1897195329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556081e76120 con 0x556081e735d0 2024-09-19T06:15:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.912+0000 7f33263fb640 1 --2- 172.21.15.5:0/1897195329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e8a310 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f331c0ce190 tx=0x7f331c004ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78cdd3643ead4cb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.913+0000 7f330bfff640 1 -- 172.21.15.5:0/1897195329 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c0b5920 con 0x556081e735d0 2024-09-19T06:15:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.913+0000 7f330bfff640 1 -- 172.21.15.5:0/1897195329 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f331c09ce20 con 0x556081e735d0 2024-09-19T06:15:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.913+0000 7f330bfff640 1 -- 172.21.15.5:0/1897195329 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c005560 con 0x556081e735d0 2024-09-19T06:15:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.913+0000 7f332aa38740 1 -- 172.21.15.5:0/1897195329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 msgr2=0x556081e8a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.913+0000 7f332aa38740 1 --2- 172.21.15.5:0/1897195329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e8a310 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f331c0ce190 tx=0x7f331c004ec0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.913+0000 7f332aa38740 1 -- 172.21.15.5:0/1897195329 shutdown_connections 2024-09-19T06:15:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.913+0000 7f332aa38740 1 --2- 172.21.15.5:0/1897195329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e7e370 0x556081e6fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.913+0000 7f332aa38740 1 --2- 172.21.15.5:0/1897195329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e74b40 0x556081e6f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.913+0000 7f332aa38740 1 --2- 172.21.15.5:0/1897195329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e735d0 0x556081e8a310 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.913+0000 7f332aa38740 1 -- 172.21.15.5:0/1897195329 >> 172.21.15.5:0/1897195329 conn(0x556081e3c310 msgr2=0x556081e79900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:54.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.913+0000 7f332aa38740 1 -- 172.21.15.5:0/1897195329 shutdown_connections 2024-09-19T06:15:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f332aa38740 1 -- 172.21.15.5:0/1897195329 wait complete. 2024-09-19T06:15:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f332aa38740 1 -- start start 2024-09-19T06:15:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f332aa38740 1 Processor -- start 2024-09-19T06:15:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f332aa38740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e735d0 0x556081e78c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f332aa38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e74b40 0x556081e86480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f332aa38740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e7e370 0x556081e86a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f332aa38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33201b0fb0 con 0x556081e74b40 2024-09-19T06:15:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f332aa38740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33201b0880 con 0x556081e735d0 2024-09-19T06:15:54.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f332aa38740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33201ada80 con 0x556081e7e370 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f332aa38740 10 client.0 _ll_register_callbacks cb 0x7f3318009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f3326bfc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e7e370 0x556081e86a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f3325bfa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e74b40 0x556081e86480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f33263fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e735d0 0x556081e78c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f3326bfc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e7e370 0x556081e86a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48776/0 (socket says 172.21.15.5:48776) 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.914+0000 7f3326bfc640 1 -- 172.21.15.5:0/3225256803 learned_addr learned my addr 172.21.15.5:0/3225256803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f3326bfc640 1 -- 172.21.15.5:0/3225256803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e735d0 msgr2=0x556081e78c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f3326bfc640 1 --2- 172.21.15.5:0/3225256803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e735d0 0x556081e78c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f3326bfc640 1 -- 172.21.15.5:0/3225256803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e74b40 msgr2=0x556081e86480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f3326bfc640 1 --2- 172.21.15.5:0/3225256803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e74b40 0x556081e86480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f3326bfc640 1 -- 172.21.15.5:0/3225256803 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556081e772a0 con 0x556081e7e370 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f3325bfa640 1 --2- 172.21.15.5:0/3225256803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e74b40 0x556081e86480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f3326bfc640 1 --2- 172.21.15.5:0/3225256803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e7e370 0x556081e86a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3318005d00 tx=0x7f33180051d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=868057d60e0e3b5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f3309ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:15:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f3309ffb640 1 -- 172.21.15.5:0/3225256803 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318064060 con 0x556081e7e370 2024-09-19T06:15:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f3309ffb640 1 -- 172.21.15.5:0/3225256803 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f33180326b0 con 0x556081e7e370 2024-09-19T06:15:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f332aa38740 1 -- 172.21.15.5:0/3225256803 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f331c09ce20 con 0x556081e7e370 2024-09-19T06:15:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f3309ffb640 1 -- 172.21.15.5:0/3225256803 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f331804f040 con 0x556081e7e370 2024-09-19T06:15:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f332aa38740 10 client.28108 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.915+0000 7f332aa38740 1 -- 172.21.15.5:0/3225256803 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556081ef7e90 con 0x556081e7e370 2024-09-19T06:15:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.916+0000 7f3309ffb640 1 -- 172.21.15.5:0/3225256803 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f331c09ce20 con 0x556081e7e370 2024-09-19T06:15:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.916+0000 7f3309ffb640 1 -- 172.21.15.5:0/3225256803 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f331803d480 con 0x556081e7e370 2024-09-19T06:15:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.916+0000 7f332aa38740 10 client.28108 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.916+0000 7f332aa38740 10 client.28108 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:15:54.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.916+0000 7f332aa38740 20 client.28108 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:54.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.916+0000 7f332aa38740 1 --2- 172.21.15.5:0/3225256803 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556081ef8490 0x556081efa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:54.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.916+0000 7f332aa38740 4 client.28108 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:15:54.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.916+0000 7f332aa38740 1 -- 172.21.15.5:0/3225256803 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556081de0d70 con 0x556081ef8490 2024-09-19T06:15:54.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.916+0000 7f33263fb640 1 --2- 172.21.15.5:0/3225256803 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556081ef8490 0x556081efa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:54.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.916+0000 7f3309ffb640 10 client.28108 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:15:54.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.916+0000 7f33263fb640 1 --2- 172.21.15.5:0/3225256803 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556081ef8490 0x556081efa880 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=79b05cd46c7d57fe server_cookie=f8673fbc2379b2ed in_seq=0 out_seq=0 2024-09-19T06:15:54.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f3309ffb640 1 -- 172.21.15.5:0/3225256803 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556081de0d70 con 0x556081ef8490 2024-09-19T06:15:54.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f3309ffb640 10 client.28108 handle_command_reply: tid=0 2024-09-19T06:15:54.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f332aa38740 1 client.28108 shutdown 2024-09-19T06:15:54.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f332aa38740 1 -- 172.21.15.5:0/3225256803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e7e370 msgr2=0x556081e86a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f332aa38740 1 --2- 172.21.15.5:0/3225256803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e7e370 0x556081e86a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3318005d00 tx=0x7f33180051d0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f332aa38740 1 -- 172.21.15.5:0/3225256803 shutdown_connections 2024-09-19T06:15:54.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f332aa38740 1 --2- 172.21.15.5:0/3225256803 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x556081ef8490 0x556081efa880 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f332aa38740 1 --2- 172.21.15.5:0/3225256803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556081e7e370 0x556081e86a00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f3309ffb640 0 client.28108 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:15:54.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f332aa38740 1 --2- 172.21.15.5:0/3225256803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556081e74b40 0x556081e86480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f332aa38740 1 --2- 172.21.15.5:0/3225256803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556081e735d0 0x556081e78c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f332aa38740 1 -- 172.21.15.5:0/3225256803 >> 172.21.15.5:0/3225256803 conn(0x556081e3c310 msgr2=0x556081e79760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:54.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f332aa38740 1 -- 172.21.15.5:0/3225256803 shutdown_connections 2024-09-19T06:15:54.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f332aa38740 1 -- 172.21.15.5:0/3225256803 wait complete. 2024-09-19T06:15:54.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.917+0000 7f332aa38740 20 client.28108 trim_cache size 0 max 16384 2024-09-19T06:15:54.924 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:15:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.918+0000 7f332865f640 1 -- 172.21.15.5:0/494748655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f330c0a33b0 msgr2=0x7f330c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.918+0000 7f332865f640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f330c0a33b0 0x7f330c0a5870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3318031440 tx=0x7f33180041f0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.918+0000 7f332865f640 1 -- 172.21.15.5:0/494748655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3320101de0 msgr2=0x7f3320198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.918+0000 7f332865f640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3320101de0 0x7f3320198310 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f331c00eb90 tx=0x7f331c00c6a0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.918+0000 7f332865f640 1 -- 172.21.15.5:0/494748655 shutdown_connections 2024-09-19T06:15:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.918+0000 7f332865f640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f330c0a33b0 0x7f330c0a5870 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.918+0000 7f332865f640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320106f10 0x7f332019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.918+0000 7f332865f640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3320102790 0x7f3320198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.918+0000 7f332865f640 1 --2- 172.21.15.5:0/494748655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3320101de0 0x7f3320198310 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.918+0000 7f332865f640 1 -- 172.21.15.5:0/494748655 >> 172.21.15.5:0/494748655 conn(0x7f33200fd880 msgr2=0x7f33200feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.918+0000 7f332865f640 1 -- 172.21.15.5:0/494748655 shutdown_connections 2024-09-19T06:15:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:54.918+0000 7f332865f640 1 -- 172.21.15.5:0/494748655 wait complete. 2024-09-19T06:15:55.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 286175 2.15 MB/sec execute 561 sec latency 21.129 ms 2024-09-19T06:15:56.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:55 smithi005 ceph-mon[30955]: pgmap v393: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 395 KiB/s wr, 47 op/s 2024-09-19T06:15:56.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:55 smithi049 ceph-mon[34640]: pgmap v393: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 395 KiB/s wr, 47 op/s 2024-09-19T06:15:56.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:55 smithi157 ceph-mon[35346]: pgmap v393: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 395 KiB/s wr, 47 op/s 2024-09-19T06:15:56.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 286764 2.15 MB/sec execute 562 sec latency 81.421 ms 2024-09-19T06:15:57.428 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:15:57.705 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:57.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287191 2.15 MB/sec execute 563 sec latency 59.143 ms 2024-09-19T06:15:58.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:57 smithi005 ceph-mon[30955]: pgmap v394: 97 pgs: 97 active+clean; 160 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:15:58.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:57 smithi049 ceph-mon[34640]: pgmap v394: 97 pgs: 97 active+clean; 160 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:15:58.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.233+0000 7f73ab12f640 1 -- 172.21.15.5:0/3565478169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a4106f10 msgr2=0x7f73a410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:58.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.233+0000 7f73ab12f640 1 --2- 172.21.15.5:0/3565478169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a4106f10 0x7f73a410ea90 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f739c009ca0 tx=0x7f739c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.233+0000 7f73ab12f640 1 -- 172.21.15.5:0/3565478169 shutdown_connections 2024-09-19T06:15:58.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.233+0000 7f73ab12f640 1 --2- 172.21.15.5:0/3565478169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a4106f10 0x7f73a410ea90 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.233+0000 7f73ab12f640 1 --2- 172.21.15.5:0/3565478169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4102790 0x7f73a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.233+0000 7f73ab12f640 1 --2- 172.21.15.5:0/3565478169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73a4101de0 0x7f73a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.233+0000 7f73ab12f640 1 -- 172.21.15.5:0/3565478169 >> 172.21.15.5:0/3565478169 conn(0x7f73a40fd8c0 msgr2=0x7f73a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.233+0000 7f73ab12f640 1 -- 172.21.15.5:0/3565478169 shutdown_connections 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.233+0000 7f73ab12f640 1 -- 172.21.15.5:0/3565478169 wait complete. 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73ab12f640 1 Processor -- start 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73ab12f640 1 -- start start 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73ab12f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73a4101de0 0x7f73a4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73ab12f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a4102790 0x7f73a4198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73ab12f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4106f10 0x7f73a419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73ab12f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4111e70 con 0x7f73a4102790 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73ab12f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4111cf0 con 0x7f73a4106f10 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73ab12f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4111ff0 con 0x7f73a4101de0 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a4102790 0x7f73a4198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a4102790 0x7f73a4198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58796/0 (socket says 172.21.15.5:58796) 2024-09-19T06:15:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73a3fff640 1 -- 172.21.15.5:0/1335218484 learned_addr learned my addr 172.21.15.5:0/1335218484 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73a8ecb640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73a4101de0 0x7f73a4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73a3fff640 1 -- 172.21.15.5:0/1335218484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73a4101de0 msgr2=0x7f73a4198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73a3fff640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73a4101de0 0x7f73a4198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73a96cc640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4106f10 0x7f73a419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73a3fff640 1 -- 172.21.15.5:0/1335218484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4106f10 msgr2=0x7f73a419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.234+0000 7f73a3fff640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4106f10 0x7f73a419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.235+0000 7f73a3fff640 1 -- 172.21.15.5:0/1335218484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a419d2e0 con 0x7f73a4102790 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.235+0000 7f73a96cc640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4106f10 0x7f73a419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.235+0000 7f73a3fff640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a4102790 0x7f73a4198870 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f739000d950 tx=0x7f739000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.235+0000 7f73a1ffb640 1 -- 172.21.15.5:0/1335218484 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390017030 con 0x7f73a4102790 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.235+0000 7f73a1ffb640 1 -- 172.21.15.5:0/1335218484 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7390004020 con 0x7f73a4102790 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.235+0000 7f73ab12f640 1 -- 172.21.15.5:0/1335218484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a419d5d0 con 0x7f73a4102790 2024-09-19T06:15:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.235+0000 7f73a1ffb640 1 -- 172.21.15.5:0/1335218484 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73900047a0 con 0x7f73a4102790 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.235+0000 7f73ab12f640 1 -- 172.21.15.5:0/1335218484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a41a4eb0 con 0x7f73a4102790 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.238+0000 7f73a1ffb640 1 -- 172.21.15.5:0/1335218484 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f73900041d0 con 0x7f73a4102790 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.238+0000 7f73ad508740 1 Processor -- start 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.238+0000 7f73ad508740 1 -- start start 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73ad508740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe36b40 0x55655fe35090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73ad508740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe355d0 0x55655fe35a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73ad508740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe44780 0x55655fe46b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73ad508740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55655fd9b2f0 con 0x55655fe44780 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73ad508740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55655fd42cf0 con 0x55655fe355d0 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73ad508740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55655fd972b0 con 0x55655fe36b40 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a1ffb640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73940a33b0 0x7f73940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a8ecb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe355d0 0x55655fe35a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a8ecb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe355d0 0x55655fe35a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40514/0 (socket says 172.21.15.5:40514) 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a8ecb640 1 -- 172.21.15.5:0/1754781451 learned_addr learned my addr 172.21.15.5:0/1754781451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a3fff640 1 --2- 172.21.15.5:0/1754781451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe44780 0x55655fe46b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a96cc640 1 --2- 172.21.15.5:0/1754781451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe36b40 0x55655fe35090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a96cc640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73940a33b0 0x7f73940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a8ecb640 1 -- 172.21.15.5:0/1754781451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe36b40 msgr2=0x55655fe35090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a8ecb640 1 --2- 172.21.15.5:0/1754781451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe36b40 0x55655fe35090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a8ecb640 1 -- 172.21.15.5:0/1754781451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe44780 msgr2=0x55655fe46b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a8ecb640 1 --2- 172.21.15.5:0/1754781451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe44780 0x55655fe46b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a1ffb640 1 -- 172.21.15.5:0/1335218484 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f73900cf760 con 0x7f73a4102790 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a8ecb640 1 -- 172.21.15.5:0/1754781451 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55655fd273a0 con 0x55655fe355d0 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.239+0000 7f73a8ecb640 1 --2- 172.21.15.5:0/1754781451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe355d0 0x55655fe35a10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f739800bd10 tx=0x7f73980107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1ffbfbd130e8b87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f73a96cc640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73940a33b0 0x7f73940a5870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f739c02f3a0 tx=0x7f739c02a8c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:58.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f738effd640 1 -- 172.21.15.5:0/1754781451 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398004030 con 0x55655fe355d0 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f738effd640 1 -- 172.21.15.5:0/1754781451 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73980035f0 con 0x55655fe355d0 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f738effd640 1 -- 172.21.15.5:0/1754781451 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398011770 con 0x55655fe355d0 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f73ad508740 1 -- 172.21.15.5:0/1754781451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe355d0 msgr2=0x55655fe35a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f73ad508740 1 --2- 172.21.15.5:0/1754781451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe355d0 0x55655fe35a10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f739800bd10 tx=0x7f73980107a0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f73ad508740 1 -- 172.21.15.5:0/1754781451 shutdown_connections 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f73ad508740 1 --2- 172.21.15.5:0/1754781451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe44780 0x55655fe46b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f73ad508740 1 --2- 172.21.15.5:0/1754781451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe355d0 0x55655fe35a10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f73ad508740 1 --2- 172.21.15.5:0/1754781451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe36b40 0x55655fe35090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f73ad508740 1 -- 172.21.15.5:0/1754781451 >> 172.21.15.5:0/1754781451 conn(0x55655fdfe310 msgr2=0x55655fe31c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f73ad508740 1 -- 172.21.15.5:0/1754781451 shutdown_connections 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.240+0000 7f73ad508740 1 -- 172.21.15.5:0/1754781451 wait complete. 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.241+0000 7f73ad508740 1 -- start start 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.241+0000 7f73ad508740 1 Processor -- start 2024-09-19T06:15:58.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.241+0000 7f73ad508740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe355d0 0x55655fe44130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.241+0000 7f73ad508740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 0x55655fe413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.241+0000 7f73ad508740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe44780 0x55655fe41980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.241+0000 7f73ad508740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55655fe47480 con 0x55655fe36b40 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.241+0000 7f73ad508740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55655fd56060 con 0x55655fe44780 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.241+0000 7f73ad508740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55655fe47600 con 0x55655fe355d0 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.241+0000 7f73ad508740 10 client.0 _ll_register_callbacks cb 0x55655faa98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.241+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 0x55655fe413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 0x55655fe413e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58828/0 (socket says 172.21.15.5:58828) 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73a3fff640 1 -- 172.21.15.5:0/758199969 learned_addr learned my addr 172.21.15.5:0/758199969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73a96cc640 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe44780 0x55655fe41980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73a8ecb640 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe355d0 0x55655fe44130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73a3fff640 1 -- 172.21.15.5:0/758199969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe355d0 msgr2=0x55655fe44130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73a3fff640 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe355d0 0x55655fe44130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73a3fff640 1 -- 172.21.15.5:0/758199969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe44780 msgr2=0x55655fe41980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73a3fff640 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe44780 0x55655fe41980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73a3fff640 1 -- 172.21.15.5:0/758199969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55655fb0c3c0 con 0x55655fe36b40 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73a3fff640 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 0x55655fe413e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f73900983f0 tx=0x7f739009b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6767508a42159833 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f7383fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f7383fff640 1 -- 172.21.15.5:0/758199969 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739009ba20 con 0x55655fe36b40 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f7383fff640 1 -- 172.21.15.5:0/758199969 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f7390098090 con 0x55655fe36b40 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73ad508740 1 -- 172.21.15.5:0/758199969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73980035f0 con 0x55655fe36b40 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f7383fff640 1 -- 172.21.15.5:0/758199969 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f739009fdd0 con 0x55655fe36b40 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73ad508740 10 client.18717 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.242+0000 7f73ad508740 1 -- 172.21.15.5:0/758199969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55655fe52ed0 con 0x55655fe36b40 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.243+0000 7f7383fff640 1 -- 172.21.15.5:0/758199969 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73980035f0 con 0x55655fe36b40 2024-09-19T06:15:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.243+0000 7f7383fff640 1 -- 172.21.15.5:0/758199969 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f7390097770 con 0x55655fe36b40 2024-09-19T06:15:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.243+0000 7f73ad508740 10 client.18717 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.243+0000 7f73ad508740 10 client.18717 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:15:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.243+0000 7f73ad508740 20 client.18717 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.243+0000 7f73ad508740 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55655fe54330 0x55655fe56720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.243+0000 7f73ad508740 4 client.18717 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.243+0000 7f73ad508740 1 -- 172.21.15.5:0/758199969 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55655fda8860 con 0x55655fe54330 2024-09-19T06:15:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.243+0000 7f73a8ecb640 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55655fe54330 0x55655fe56720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.243+0000 7f73a8ecb640 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55655fe54330 0x55655fe56720 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c8fb0a760eeec6c1 server_cookie=dc9a781ad01ffba in_seq=0 out_seq=0 2024-09-19T06:15:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.243+0000 7f7383fff640 10 client.18717 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:15:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.244+0000 7f7383fff640 1 -- 172.21.15.5:0/758199969 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55655fda8860 con 0x55655fe54330 2024-09-19T06:15:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.244+0000 7f7383fff640 10 client.18717 handle_command_reply: tid=0 2024-09-19T06:15:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.244+0000 7f73ad508740 1 client.18717 shutdown 2024-09-19T06:15:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.244+0000 7f73ad508740 1 -- 172.21.15.5:0/758199969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 msgr2=0x55655fe413e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.244+0000 7f73ad508740 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 0x55655fe413e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f73900983f0 tx=0x7f739009b3e0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.244+0000 7f73ad508740 1 -- 172.21.15.5:0/758199969 shutdown_connections 2024-09-19T06:15:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.244+0000 7f73ad508740 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55655fe54330 0x55655fe56720 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.244+0000 7f7383fff640 0 client.18717 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:15:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.244+0000 7f73ad508740 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe44780 0x55655fe41980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.244+0000 7f73ad508740 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 0x55655fe413e0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.244+0000 7f73ad508740 1 --2- 172.21.15.5:0/758199969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe355d0 0x55655fe44130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.244+0000 7f73ad508740 1 -- 172.21.15.5:0/758199969 >> 172.21.15.5:0/758199969 conn(0x55655fdfe310 msgr2=0x55655fe37960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:58.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.245+0000 7f73ad508740 1 -- 172.21.15.5:0/758199969 shutdown_connections 2024-09-19T06:15:58.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.245+0000 7f73ad508740 1 -- 172.21.15.5:0/758199969 wait complete. 2024-09-19T06:15:58.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.245+0000 7f73ad508740 20 client.18717 trim_cache size 0 max 16384 2024-09-19T06:15:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73ad508740 1 Processor -- start 2024-09-19T06:15:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73ad508740 1 -- start start 2024-09-19T06:15:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73ad508740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe355d0 0x55655fe50720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73ad508740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 0x55655fe43350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73ad508740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe44780 0x55655fe5bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73ad508740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41ab630 con 0x55655fe36b40 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73ad508740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41ad230 con 0x55655fe355d0 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73ad508740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41abd00 con 0x55655fe44780 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 0x55655fe43350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 0x55655fe43350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58832/0 (socket says 172.21.15.5:58832) 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73a3fff640 1 -- 172.21.15.5:0/614254834 learned_addr learned my addr 172.21.15.5:0/614254834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73a96cc640 1 --2- 172.21.15.5:0/614254834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe44780 0x55655fe5bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73a8ecb640 1 --2- 172.21.15.5:0/614254834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe355d0 0x55655fe50720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73a3fff640 1 -- 172.21.15.5:0/614254834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe44780 msgr2=0x55655fe5bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73a3fff640 1 --2- 172.21.15.5:0/614254834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe44780 0x55655fe5bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73a3fff640 1 -- 172.21.15.5:0/614254834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe355d0 msgr2=0x55655fe50720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73a3fff640 1 --2- 172.21.15.5:0/614254834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe355d0 0x55655fe50720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73a3fff640 1 -- 172.21.15.5:0/614254834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55655fe43d50 con 0x55655fe36b40 2024-09-19T06:15:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.258+0000 7f73a96cc640 1 --2- 172.21.15.5:0/614254834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe44780 0x55655fe5bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:58.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f73a3fff640 1 --2- 172.21.15.5:0/614254834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 0x55655fe43350 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f73900a4160 tx=0x7f73900862c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3db5f573f49da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:58.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f738e7fc640 1 -- 172.21.15.5:0/614254834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390097cc0 con 0x55655fe36b40 2024-09-19T06:15:58.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f738e7fc640 1 -- 172.21.15.5:0/614254834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f739009ba20 con 0x55655fe36b40 2024-09-19T06:15:58.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f738e7fc640 1 -- 172.21.15.5:0/614254834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390009a30 con 0x55655fe36b40 2024-09-19T06:15:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f73ad508740 1 -- 172.21.15.5:0/614254834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 msgr2=0x55655fe43350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f73ad508740 1 --2- 172.21.15.5:0/614254834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 0x55655fe43350 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f73900a4160 tx=0x7f73900862c0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f73ad508740 1 -- 172.21.15.5:0/614254834 shutdown_connections 2024-09-19T06:15:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f73ad508740 1 --2- 172.21.15.5:0/614254834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe44780 0x55655fe5bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f73ad508740 1 --2- 172.21.15.5:0/614254834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe36b40 0x55655fe43350 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f73ad508740 1 --2- 172.21.15.5:0/614254834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe355d0 0x55655fe50720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f73ad508740 1 -- 172.21.15.5:0/614254834 >> 172.21.15.5:0/614254834 conn(0x55655fdfe310 msgr2=0x55655fe42c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f73ad508740 1 -- 172.21.15.5:0/614254834 shutdown_connections 2024-09-19T06:15:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.259+0000 7f73ad508740 1 -- 172.21.15.5:0/614254834 wait complete. 2024-09-19T06:15:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.260+0000 7f73ad508740 1 -- start start 2024-09-19T06:15:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.260+0000 7f73ad508740 1 Processor -- start 2024-09-19T06:15:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.260+0000 7f73ad508740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe355d0 0x55655fe4cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.260+0000 7f73ad508740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe36b40 0x55655fe4d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.260+0000 7f73ad508740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe44780 0x55655fe49ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.260+0000 7f73ad508740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41acb60 con 0x55655fe44780 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.260+0000 7f73ad508740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41ac430 con 0x55655fe36b40 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.260+0000 7f73ad508740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41a9630 con 0x55655fe355d0 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.260+0000 7f73ad508740 10 client.0 _ll_register_callbacks cb 0x7f73900046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.260+0000 7f73a96cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe44780 0x55655fe49ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.260+0000 7f73a96cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe44780 0x55655fe49ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58844/0 (socket says 172.21.15.5:58844) 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.260+0000 7f73a96cc640 1 -- 172.21.15.5:0/2621636698 learned_addr learned my addr 172.21.15.5:0/2621636698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f73a8ecb640 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe355d0 0x55655fe4cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f73a3fff640 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe36b40 0x55655fe4d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f73a96cc640 1 -- 172.21.15.5:0/2621636698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe355d0 msgr2=0x55655fe4cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f73a96cc640 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe355d0 0x55655fe4cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f73a96cc640 1 -- 172.21.15.5:0/2621636698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe36b40 msgr2=0x55655fe4d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f73a96cc640 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe36b40 0x55655fe4d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f73a96cc640 1 -- 172.21.15.5:0/2621636698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55655fe4a2b0 con 0x55655fe44780 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f73a8ecb640 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe355d0 0x55655fe4cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f73a96cc640 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe44780 0x55655fe49ab0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f739c031cc0 tx=0x7f739c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed21b4aecb4ac8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f7383fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f7383fff640 1 -- 172.21.15.5:0/2621636698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c004040 con 0x55655fe44780 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f7383fff640 1 -- 172.21.15.5:0/2621636698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f739c046070 con 0x55655fe44780 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f73ad508740 1 -- 172.21.15.5:0/2621636698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f739009ba20 con 0x55655fe44780 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f7383fff640 1 -- 172.21.15.5:0/2621636698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f739c004b40 con 0x55655fe44780 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f73ad508740 10 client.18723 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.261+0000 7f73ad508740 1 -- 172.21.15.5:0/2621636698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55655febfc10 con 0x55655fe44780 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.262+0000 7f7383fff640 1 -- 172.21.15.5:0/2621636698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f739009ba20 con 0x55655fe44780 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.262+0000 7f7383fff640 1 -- 172.21.15.5:0/2621636698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f739c049040 con 0x55655fe44780 2024-09-19T06:15:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.262+0000 7f73ad508740 10 client.18723 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.262+0000 7f73ad508740 10 client.18723 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.262+0000 7f73ad508740 20 client.18723 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.262+0000 7f73ad508740 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55655fe54330 0x55655fec2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.262+0000 7f73ad508740 4 client.18723 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.262+0000 7f73ad508740 1 -- 172.21.15.5:0/2621636698 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55655fda2d70 con 0x55655fe54330 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.262+0000 7f73a8ecb640 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55655fe54330 0x55655fec2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.262+0000 7f73a8ecb640 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55655fe54330 0x55655fec2420 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a9f09ba83f36b211 server_cookie=74d35bbbed0b786c in_seq=0 out_seq=0 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.262+0000 7f7383fff640 10 client.18723 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f7383fff640 1 -- 172.21.15.5:0/2621636698 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55655fda2d70 con 0x55655fe54330 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f7383fff640 10 client.18723 handle_command_reply: tid=0 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f73ad508740 1 client.18723 shutdown 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f73ad508740 1 -- 172.21.15.5:0/2621636698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe44780 msgr2=0x55655fe49ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f73ad508740 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe44780 0x55655fe49ab0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f739c031cc0 tx=0x7f739c005f30 comp rx=0 tx=0).stop 2024-09-19T06:15:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f73ad508740 1 -- 172.21.15.5:0/2621636698 shutdown_connections 2024-09-19T06:15:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f73ad508740 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55655fe54330 0x55655fec2420 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f73ad508740 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655fe44780 0x55655fe49ab0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f73ad508740 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55655fe36b40 0x55655fe4d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f73ad508740 1 --2- 172.21.15.5:0/2621636698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55655fe355d0 0x55655fe4cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f7383fff640 0 client.18723 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:15:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f73ad508740 1 -- 172.21.15.5:0/2621636698 >> 172.21.15.5:0/2621636698 conn(0x55655fdfe310 msgr2=0x55655fe43460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f73ad508740 1 -- 172.21.15.5:0/2621636698 shutdown_connections 2024-09-19T06:15:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f73ad508740 1 -- 172.21.15.5:0/2621636698 wait complete. 2024-09-19T06:15:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.263+0000 7f73ad508740 20 client.18723 trim_cache size 0 max 16384 2024-09-19T06:15:58.270 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:15:58.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.264+0000 7f73ab12f640 1 -- 172.21.15.5:0/1335218484 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73940a33b0 msgr2=0x7f73940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:58.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.264+0000 7f73ab12f640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73940a33b0 0x7f73940a5870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f739c02f3a0 tx=0x7f739c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.264+0000 7f73ab12f640 1 -- 172.21.15.5:0/1335218484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a4102790 msgr2=0x7f73a4198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:58.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.264+0000 7f73ab12f640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a4102790 0x7f73a4198870 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f739000d950 tx=0x7f739000de10 comp rx=0 tx=0).stop 2024-09-19T06:15:58.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.264+0000 7f73ab12f640 1 -- 172.21.15.5:0/1335218484 shutdown_connections 2024-09-19T06:15:58.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.264+0000 7f73ab12f640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4106f10 0x7f73a419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.264+0000 7f73ab12f640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a4102790 0x7f73a4198870 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.264+0000 7f73ab12f640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73940a33b0 0x7f73940a5870 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.264+0000 7f73ab12f640 1 --2- 172.21.15.5:0/1335218484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73a4101de0 0x7f73a4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.264+0000 7f73ab12f640 1 -- 172.21.15.5:0/1335218484 >> 172.21.15.5:0/1335218484 conn(0x7f73a40fd8c0 msgr2=0x7f73a40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.264+0000 7f73ab12f640 1 -- 172.21.15.5:0/1335218484 shutdown_connections 2024-09-19T06:15:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:58.264+0000 7f73ab12f640 1 -- 172.21.15.5:0/1335218484 wait complete. 2024-09-19T06:15:58.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:57 smithi157 ceph-mon[35346]: pgmap v394: 97 pgs: 97 active+clean; 160 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-19T06:15:58.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287401 2.15 MB/sec execute 564 sec latency 719.508 ms 2024-09-19T06:15:59.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:15:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:15:59.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.785+0000 7fb5bced7640 1 -- 172.21.15.5:0/3590310033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5b8101de0 msgr2=0x7fb5b81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.785+0000 7fb5bced7640 1 --2- 172.21.15.5:0/3590310033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5b8101de0 0x7fb5b81021c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009900 tx=0x7fb5a802a410 comp rx=0 tx=0).stop 2024-09-19T06:15:59.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.785+0000 7fb5bced7640 1 -- 172.21.15.5:0/3590310033 shutdown_connections 2024-09-19T06:15:59.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.785+0000 7fb5bced7640 1 --2- 172.21.15.5:0/3590310033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b8106f10 0x7fb5b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.785+0000 7fb5bced7640 1 --2- 172.21.15.5:0/3590310033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5b8102790 0x7fb5b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.785+0000 7fb5bced7640 1 --2- 172.21.15.5:0/3590310033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5b8101de0 0x7fb5b81021c0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.785+0000 7fb5bced7640 1 -- 172.21.15.5:0/3590310033 >> 172.21.15.5:0/3590310033 conn(0x7fb5b80fd8c0 msgr2=0x7fb5b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:59.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.785+0000 7fb5bced7640 1 -- 172.21.15.5:0/3590310033 shutdown_connections 2024-09-19T06:15:59.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.785+0000 7fb5bced7640 1 -- 172.21.15.5:0/3590310033 wait complete. 2024-09-19T06:15:59.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.786+0000 7fb5bced7640 1 Processor -- start 2024-09-19T06:15:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.786+0000 7fb5bced7640 1 -- start start 2024-09-19T06:15:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.786+0000 7fb5bced7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5b8101de0 0x7fb5b81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.786+0000 7fb5bced7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b8102790 0x7fb5b8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.786+0000 7fb5bced7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5b8106f10 0x7fb5b819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.786+0000 7fb5bced7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b8111e20 con 0x7fb5b8101de0 2024-09-19T06:15:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.786+0000 7fb5bced7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b8111ca0 con 0x7fb5b8106f10 2024-09-19T06:15:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.786+0000 7fb5bced7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b8111fa0 con 0x7fb5b8102790 2024-09-19T06:15:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.786+0000 7fb5b659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5b8101de0 0x7fb5b81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.786+0000 7fb5b659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5b8101de0 0x7fb5b81982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58866/0 (socket says 172.21.15.5:58866) 2024-09-19T06:15:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.786+0000 7fb5b659c640 1 -- 172.21.15.5:0/2143638390 learned_addr learned my addr 172.21.15.5:0/2143638390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.786+0000 7fb5b5d9b640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b8102790 0x7fb5b8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.787+0000 7fb5b6d9d640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5b8106f10 0x7fb5b819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.787+0000 7fb5b659c640 1 -- 172.21.15.5:0/2143638390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b8102790 msgr2=0x7fb5b8198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.787+0000 7fb5b659c640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b8102790 0x7fb5b8198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.787+0000 7fb5b659c640 1 -- 172.21.15.5:0/2143638390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5b8106f10 msgr2=0x7fb5b819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.787+0000 7fb5b659c640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5b8106f10 0x7fb5b819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.787+0000 7fb5b659c640 1 -- 172.21.15.5:0/2143638390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b819d290 con 0x7fb5b8101de0 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.787+0000 7fb5b659c640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5b8101de0 0x7fb5b81982e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8006010 tx=0x7fb5a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.787+0000 7fb5a77fe640 1 -- 172.21.15.5:0/2143638390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a803f030 con 0x7fb5b8101de0 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.787+0000 7fb5a77fe640 1 -- 172.21.15.5:0/2143638390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5a80035f0 con 0x7fb5b8101de0 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.787+0000 7fb5bced7640 1 -- 172.21.15.5:0/2143638390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b819d520 con 0x7fb5b8101de0 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.787+0000 7fb5a77fe640 1 -- 172.21.15.5:0/2143638390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8035070 con 0x7fb5b8101de0 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.787+0000 7fb5bced7640 1 -- 172.21.15.5:0/2143638390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b81a4e60 con 0x7fb5b8101de0 2024-09-19T06:15:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.790+0000 7fb5a77fe640 1 -- 172.21.15.5:0/2143638390 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb5a8003b10 con 0x7fb5b8101de0 2024-09-19T06:15:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.790+0000 7fb5bf2b0740 1 Processor -- start 2024-09-19T06:15:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.790+0000 7fb5bf2b0740 1 -- start start 2024-09-19T06:15:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.790+0000 7fb5bf2b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1a7b40 0x556fff1a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.790+0000 7fb5bf2b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a65d0 0x556fff1a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.790+0000 7fb5bf2b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 0x556fff1b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.790+0000 7fb5bf2b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556fff10c2f0 con 0x556fff1b1370 2024-09-19T06:15:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.790+0000 7fb5bf2b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556fff0b3cf0 con 0x556fff1a65d0 2024-09-19T06:15:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.790+0000 7fb5bf2b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556fff1082b0 con 0x556fff1a7b40 2024-09-19T06:15:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1a7b40 0x556fff1a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 0x556fff1b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1a7b40 0x556fff1a6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48826/0 (socket says 172.21.15.5:48826) 2024-09-19T06:15:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b6d9d640 1 -- 172.21.15.5:0/2809941124 learned_addr learned my addr 172.21.15.5:0/2809941124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b659c640 1 --2- 172.21.15.5:0/2809941124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a65d0 0x556fff1a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5a77fe640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5a00a3430 0x7fb5a00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b6d9d640 1 -- 172.21.15.5:0/2809941124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a65d0 msgr2=0x556fff1a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b6d9d640 1 --2- 172.21.15.5:0/2809941124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a65d0 0x556fff1a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b6d9d640 1 -- 172.21.15.5:0/2809941124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 msgr2=0x556fff1b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b6d9d640 1 --2- 172.21.15.5:0/2809941124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 0x556fff1b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b6d9d640 1 -- 172.21.15.5:0/2809941124 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fff0983a0 con 0x556fff1a7b40 2024-09-19T06:15:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b5d9b640 1 --2- 172.21.15.5:0/2809941124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 0x556fff1b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b6d9d640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5a00a3430 0x7fb5a00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5a77fe640 1 -- 172.21.15.5:0/2143638390 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fb5a80f0640 con 0x7fb5b8101de0 2024-09-19T06:15:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b6d9d640 1 --2- 172.21.15.5:0/2809941124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1a7b40 0x556fff1a6090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0008ec0 tx=0x7fb5b000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d6a73733a03025 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5a4ff9640 1 -- 172.21.15.5:0/2809941124 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0022070 con 0x556fff1a7b40 2024-09-19T06:15:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.791+0000 7fb5b6d9d640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5a00a3430 0x7fb5a00a58f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb5b001f000 tx=0x7fb5b0020000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.792+0000 7fb5a4ff9640 1 -- 172.21.15.5:0/2809941124 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5b0011d60 con 0x556fff1a7b40 2024-09-19T06:15:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.792+0000 7fb5a4ff9640 1 -- 172.21.15.5:0/2809941124 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b001ca30 con 0x556fff1a7b40 2024-09-19T06:15:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.792+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/2809941124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1a7b40 msgr2=0x556fff1a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.792+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/2809941124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1a7b40 0x556fff1a6090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0008ec0 tx=0x7fb5b000e720 comp rx=0 tx=0).stop 2024-09-19T06:15:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.792+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/2809941124 shutdown_connections 2024-09-19T06:15:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.792+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/2809941124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 0x556fff1b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.792+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/2809941124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a65d0 0x556fff1a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.792+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/2809941124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1a7b40 0x556fff1a6090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.792+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/2809941124 >> 172.21.15.5:0/2809941124 conn(0x556fff16f310 msgr2=0x556fff1a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.792+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/2809941124 shutdown_connections 2024-09-19T06:15:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.792+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/2809941124 wait complete. 2024-09-19T06:15:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5bf2b0740 1 -- start start 2024-09-19T06:15:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5bf2b0740 1 Processor -- start 2024-09-19T06:15:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5bf2b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1a65d0 0x556fff1b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5bf2b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a7b40 0x556fff1ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5bf2b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 0x556fff1ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5bf2b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556fff1b4070 con 0x556fff1b1370 2024-09-19T06:15:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5bf2b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556fff0c7060 con 0x556fff1a7b40 2024-09-19T06:15:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5bf2b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556fff1b41f0 con 0x556fff1a65d0 2024-09-19T06:15:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5bf2b0740 10 client.0 _ll_register_callbacks cb 0x556ffee1a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5b6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 0x556fff1ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5b6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 0x556fff1ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58878/0 (socket says 172.21.15.5:58878) 2024-09-19T06:15:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5b6d9d640 1 -- 172.21.15.5:0/418662614 learned_addr learned my addr 172.21.15.5:0/418662614 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5b659c640 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1a65d0 0x556fff1b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5b5d9b640 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a7b40 0x556fff1ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5b6d9d640 1 -- 172.21.15.5:0/418662614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1a65d0 msgr2=0x556fff1b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5b6d9d640 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1a65d0 0x556fff1b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5b6d9d640 1 -- 172.21.15.5:0/418662614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a7b40 msgr2=0x556fff1ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5b6d9d640 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a7b40 0x556fff1ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5b6d9d640 1 -- 172.21.15.5:0/418662614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ffee7d3c0 con 0x556fff1b1370 2024-09-19T06:15:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.793+0000 7fb5b5d9b640 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a7b40 0x556fff1ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:15:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb5b6d9d640 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 0x556fff1ae7e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0000f80 tx=0x7fb5b0004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5c9ec0420586a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb59dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:15:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb59dffb640 1 -- 172.21.15.5:0/418662614 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0022070 con 0x556fff1b1370 2024-09-19T06:15:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/418662614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b0011d60 con 0x556fff1b1370 2024-09-19T06:15:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb59dffb640 1 -- 172.21.15.5:0/418662614 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fb5b001cbc0 con 0x556fff1b1370 2024-09-19T06:15:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb59dffb640 1 -- 172.21.15.5:0/418662614 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb5b0004e60 con 0x556fff1b1370 2024-09-19T06:15:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb5bf2b0740 10 client.18735 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/418662614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556fff1c3f60 con 0x556fff1b1370 2024-09-19T06:15:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb59dffb640 1 -- 172.21.15.5:0/418662614 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5b0011d60 con 0x556fff1b1370 2024-09-19T06:15:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb59dffb640 1 -- 172.21.15.5:0/418662614 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fb5b0019040 con 0x556fff1b1370 2024-09-19T06:15:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb5bf2b0740 10 client.18735 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb5bf2b0740 10 client.18735 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:15:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb5bf2b0740 20 client.18735 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.794+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556fff1c52b0 0x556fff1c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.795+0000 7fb5bf2b0740 4 client.18735 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:15:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.795+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/418662614 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fff119860 con 0x556fff1c52b0 2024-09-19T06:15:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.795+0000 7fb5b659c640 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556fff1c52b0 0x556fff1c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.795+0000 7fb59dffb640 10 client.18735 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:15:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.795+0000 7fb5b659c640 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556fff1c52b0 0x556fff1c76a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7ea2fe4803aa9d server_cookie=82a938afdcee82cc in_seq=0 out_seq=0 2024-09-19T06:15:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb59dffb640 1 -- 172.21.15.5:0/418662614 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fff119860 con 0x556fff1c52b0 2024-09-19T06:15:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb59dffb640 10 client.18735 handle_command_reply: tid=0 2024-09-19T06:15:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb5bf2b0740 1 client.18735 shutdown 2024-09-19T06:15:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/418662614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 msgr2=0x556fff1ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 0x556fff1ae7e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0000f80 tx=0x7fb5b0004c70 comp rx=0 tx=0).stop 2024-09-19T06:15:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/418662614 shutdown_connections 2024-09-19T06:15:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556fff1c52b0 0x556fff1c76a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1b1370 0x556fff1ae7e0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a7b40 0x556fff1ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb59dffb640 0 client.18735 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:15:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/418662614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1a65d0 0x556fff1b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/418662614 >> 172.21.15.5:0/418662614 conn(0x556fff16f310 msgr2=0x556fff1a8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/418662614 shutdown_connections 2024-09-19T06:15:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/418662614 wait complete. 2024-09-19T06:15:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.796+0000 7fb5bf2b0740 20 client.18735 trim_cache size 0 max 16384 2024-09-19T06:15:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5bf2b0740 1 Processor -- start 2024-09-19T06:15:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5bf2b0740 1 -- start start 2024-09-19T06:15:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5bf2b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a65d0 0x556fff1a7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5bf2b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a7b40 0x556fff1b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5bf2b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1b1370 0x556fff1a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5bf2b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b81ab660 con 0x556fff1a65d0 2024-09-19T06:15:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5bf2b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b81ad260 con 0x556fff1a7b40 2024-09-19T06:15:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5bf2b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b81abd30 con 0x556fff1b1370 2024-09-19T06:15:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5b659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a65d0 0x556fff1a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5b659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a65d0 0x556fff1a7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58880/0 (socket says 172.21.15.5:58880) 2024-09-19T06:15:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5b659c640 1 -- 172.21.15.5:0/154476962 learned_addr learned my addr 172.21.15.5:0/154476962 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5b659c640 1 -- 172.21.15.5:0/154476962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1b1370 msgr2=0x556fff1a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5b659c640 1 --2- 172.21.15.5:0/154476962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1b1370 0x556fff1a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5b659c640 1 -- 172.21.15.5:0/154476962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a7b40 msgr2=0x556fff1b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5b5d9b640 1 --2- 172.21.15.5:0/154476962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a7b40 0x556fff1b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5b659c640 1 --2- 172.21.15.5:0/154476962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a7b40 0x556fff1b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5b659c640 1 -- 172.21.15.5:0/154476962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fff1a24f0 con 0x556fff1a65d0 2024-09-19T06:15:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.810+0000 7fb5b659c640 1 --2- 172.21.15.5:0/154476962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a65d0 0x556fff1a7300 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8005c40 tx=0x7fb5a80f2d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8c1e89d422c702 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb59ffff640 1 -- 172.21.15.5:0/154476962 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8049050 con 0x556fff1a65d0 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb59ffff640 1 -- 172.21.15.5:0/154476962 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5a80c0c50 con 0x556fff1a65d0 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb59ffff640 1 -- 172.21.15.5:0/154476962 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80bcb00 con 0x556fff1a65d0 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/154476962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a65d0 msgr2=0x556fff1a7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/154476962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a65d0 0x556fff1a7300 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8005c40 tx=0x7fb5a80f2d10 comp rx=0 tx=0).stop 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/154476962 shutdown_connections 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/154476962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1b1370 0x556fff1a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/154476962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a7b40 0x556fff1b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/154476962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a65d0 0x556fff1a7300 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/154476962 >> 172.21.15.5:0/154476962 conn(0x556fff16f310 msgr2=0x556fff19da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/154476962 shutdown_connections 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/154476962 wait complete. 2024-09-19T06:15:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.811+0000 7fb5bf2b0740 1 -- start start 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5bf2b0740 1 Processor -- start 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5bf2b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a65d0 0x556fff1b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5bf2b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a7b40 0x556fff1b9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5bf2b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1b1370 0x556fff22df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5bf2b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b81acb90 con 0x556fff1a7b40 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5bf2b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b81ac460 con 0x556fff1a65d0 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5bf2b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b81a9660 con 0x556fff1b1370 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5bf2b0740 10 client.0 _ll_register_callbacks cb 0x7fb5b0020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5b5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a7b40 0x556fff1b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5b659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a65d0 0x556fff1b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5b659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a65d0 0x556fff1b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40578/0 (socket says 172.21.15.5:40578) 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5b659c640 1 -- 172.21.15.5:0/1810912846 learned_addr learned my addr 172.21.15.5:0/1810912846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5b6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1b1370 0x556fff22df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5b659c640 1 -- 172.21.15.5:0/1810912846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1b1370 msgr2=0x556fff22df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5b659c640 1 --2- 172.21.15.5:0/1810912846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1b1370 0x556fff22df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5b659c640 1 -- 172.21.15.5:0/1810912846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a7b40 msgr2=0x556fff1b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5b659c640 1 --2- 172.21.15.5:0/1810912846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a7b40 0x556fff1b9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5b659c640 1 -- 172.21.15.5:0/1810912846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fff22e480 con 0x556fff1a65d0 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.812+0000 7fb5b5d9b640 1 --2- 172.21.15.5:0/1810912846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a7b40 0x556fff1b9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.813+0000 7fb5b659c640 1 --2- 172.21.15.5:0/1810912846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a65d0 0x556fff1b9050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80c4fe0 tx=0x7fb5a80f6150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd418007d9767d58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.813+0000 7fb59dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.813+0000 7fb59dffb640 1 -- 172.21.15.5:0/1810912846 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80ee190 con 0x556fff1a65d0 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.813+0000 7fb59dffb640 1 -- 172.21.15.5:0/1810912846 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fb5a80bcb00 con 0x556fff1a65d0 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.813+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/1810912846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a80c0c50 con 0x556fff1a65d0 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.813+0000 7fb59dffb640 1 -- 172.21.15.5:0/1810912846 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb5a8049030 con 0x556fff1a65d0 2024-09-19T06:15:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.813+0000 7fb5bf2b0740 10 client.28181 fetch_fsmap learned FSMap version 39 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.813+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/1810912846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556fff22f780 con 0x556fff1a65d0 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.813+0000 7fb59dffb640 1 -- 172.21.15.5:0/1810912846 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5a80c0c50 con 0x556fff1a65d0 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.813+0000 7fb59dffb640 1 -- 172.21.15.5:0/1810912846 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fb5a80c5110 con 0x556fff1a65d0 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb5bf2b0740 10 client.28181 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb5bf2b0740 10 client.28181 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb5bf2b0740 20 client.28181 populate_metadata read hostname 'smithi005' 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/1810912846 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556fff22fcf0 0x556fff232110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb5bf2b0740 4 client.28181 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/1810912846 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556fff113d70 con 0x556fff22fcf0 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb5b5d9b640 1 --2- 172.21.15.5:0/1810912846 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556fff22fcf0 0x556fff232110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb59dffb640 10 client.28181 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb5b5d9b640 1 --2- 172.21.15.5:0/1810912846 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556fff22fcf0 0x556fff232110 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8c261a24028d91f server_cookie=105da371a5b4a5bf in_seq=0 out_seq=0 2024-09-19T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb59dffb640 1 -- 172.21.15.5:0/1810912846 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556fff113d70 con 0x556fff22fcf0 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb59dffb640 10 client.28181 handle_command_reply: tid=0 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb5bf2b0740 1 client.28181 shutdown 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/1810912846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a65d0 msgr2=0x556fff1b9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.814+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/1810912846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a65d0 0x556fff1b9050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80c4fe0 tx=0x7fb5a80f6150 comp rx=0 tx=0).stop 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/1810912846 shutdown_connections 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/1810912846 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556fff22fcf0 0x556fff232110 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/1810912846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556fff1b1370 0x556fff22df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/1810912846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fff1a7b40 0x556fff1b9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb59dffb640 0 client.28181 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bf2b0740 1 --2- 172.21.15.5:0/1810912846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fff1a65d0 0x556fff1b9050 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/1810912846 >> 172.21.15.5:0/1810912846 conn(0x556fff16f310 msgr2=0x556fff1a1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/1810912846 shutdown_connections 2024-09-19T06:15:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bf2b0740 1 -- 172.21.15.5:0/1810912846 wait complete. 2024-09-19T06:15:59.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bf2b0740 20 client.28181 trim_cache size 0 max 16384 2024-09-19T06:15:59.826 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bced7640 1 -- 172.21.15.5:0/2143638390 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5a00a3430 msgr2=0x7fb5a00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bced7640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5a00a3430 0x7fb5a00a58f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb5b001f000 tx=0x7fb5b0020000 comp rx=0 tx=0).stop 2024-09-19T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bced7640 1 -- 172.21.15.5:0/2143638390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5b8101de0 msgr2=0x7fb5b81982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bced7640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5b8101de0 0x7fb5b81982e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8006010 tx=0x7fb5a8004270 comp rx=0 tx=0).stop 2024-09-19T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bced7640 1 -- 172.21.15.5:0/2143638390 shutdown_connections 2024-09-19T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bced7640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5b8106f10 0x7fb5b819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bced7640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb5a00a3430 0x7fb5a00a58f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bced7640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b8102790 0x7fb5b8198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bced7640 1 --2- 172.21.15.5:0/2143638390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5b8101de0 0x7fb5b81982e0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bced7640 1 -- 172.21.15.5:0/2143638390 >> 172.21.15.5:0/2143638390 conn(0x7fb5b80fd8c0 msgr2=0x7fb5b80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bced7640 1 -- 172.21.15.5:0/2143638390 shutdown_connections 2024-09-19T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:15:59.815+0000 7fb5bced7640 1 -- 172.21.15.5:0/2143638390 wait complete. 2024-09-19T06:15:59.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287468 2.15 MB/sec execute 565 sec latency 915.165 ms 2024-09-19T06:16:00.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:15:59 smithi005 ceph-mon[30955]: pgmap v395: 97 pgs: 97 active+clean; 160 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 865 KiB/s wr, 83 op/s 2024-09-19T06:16:00.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:15:59 smithi049 ceph-mon[34640]: pgmap v395: 97 pgs: 97 active+clean; 160 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 865 KiB/s wr, 83 op/s 2024-09-19T06:16:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:15:59 smithi157 ceph-mon[35346]: pgmap v395: 97 pgs: 97 active+clean; 160 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 865 KiB/s wr, 83 op/s 2024-09-19T06:16:00.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:16:00.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287567 2.15 MB/sec execute 566 sec latency 1186.501 ms 2024-09-19T06:16:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:01.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.318+0000 7fa68e1f9640 1 -- 172.21.15.5:0/3771682967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa688101420 msgr2=0x7fa68810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.318+0000 7fa68e1f9640 1 --2- 172.21.15.5:0/3771682967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa688101420 0x7fa68810d950 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fa680009cd0 tx=0x7fa68002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.318+0000 7fa68e1f9640 1 -- 172.21.15.5:0/3771682967 shutdown_connections 2024-09-19T06:16:01.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.318+0000 7fa68e1f9640 1 --2- 172.21.15.5:0/3771682967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa688101420 0x7fa68810d950 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.318+0000 7fa68e1f9640 1 --2- 172.21.15.5:0/3771682967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa688100a80 0x7fa688100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.318+0000 7fa68e1f9640 1 --2- 172.21.15.5:0/3771682967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa688106a80 0x7fa688106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.318+0000 7fa68e1f9640 1 -- 172.21.15.5:0/3771682967 >> 172.21.15.5:0/3771682967 conn(0x7fa6880fc7a0 msgr2=0x7fa6880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.318+0000 7fa68e1f9640 1 -- 172.21.15.5:0/3771682967 shutdown_connections 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.319+0000 7fa68e1f9640 1 -- 172.21.15.5:0/3771682967 wait complete. 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.319+0000 7fa68e1f9640 1 Processor -- start 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa68e1f9640 1 -- start start 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa68e1f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa688100a80 0x7fa6881982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa68e1f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa688101420 0x7fa688198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa68e1f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa688106a80 0x7fa68819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa68e1f9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa688110d30 con 0x7fa688106a80 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa68e1f9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa688110bb0 con 0x7fa688101420 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa68e1f9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa688110eb0 con 0x7fa688100a80 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa687fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa688106a80 0x7fa68819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa687fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa688106a80 0x7fa68819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58900/0 (socket says 172.21.15.5:58900) 2024-09-19T06:16:01.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa687fff640 1 -- 172.21.15.5:0/2594840183 learned_addr learned my addr 172.21.15.5:0/2594840183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:01.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa686ffd640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa688101420 0x7fa688198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa6877fe640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa688100a80 0x7fa6881982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa687fff640 1 -- 172.21.15.5:0/2594840183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa688100a80 msgr2=0x7fa6881982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa687fff640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa688100a80 0x7fa6881982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa687fff640 1 -- 172.21.15.5:0/2594840183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa688101420 msgr2=0x7fa688198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa687fff640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa688101420 0x7fa688198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.320+0000 7fa687fff640 1 -- 172.21.15.5:0/2594840183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68819d290 con 0x7fa688106a80 2024-09-19T06:16:01.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.321+0000 7fa687fff640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa688106a80 0x7fa68819cbb0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa680033040 tx=0x7fa680003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.321+0000 7fa684ff9640 1 -- 172.21.15.5:0/2594840183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680004040 con 0x7fa688106a80 2024-09-19T06:16:01.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.321+0000 7fa684ff9640 1 -- 172.21.15.5:0/2594840183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa680004690 con 0x7fa688106a80 2024-09-19T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.321+0000 7fa68e1f9640 1 -- 172.21.15.5:0/2594840183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68819d520 con 0x7fa688106a80 2024-09-19T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.321+0000 7fa684ff9640 1 -- 172.21.15.5:0/2594840183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680030e10 con 0x7fa688106a80 2024-09-19T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.321+0000 7fa68e1f9640 1 -- 172.21.15.5:0/2594840183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6881a4e60 con 0x7fa688106a80 2024-09-19T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.323+0000 7fa684ff9640 1 -- 172.21.15.5:0/2594840183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa6800305a0 con 0x7fa688106a80 2024-09-19T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.323+0000 7fa6905d2740 1 Processor -- start 2024-09-19T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa6905d2740 1 -- start start 2024-09-19T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa6905d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d306b40 0x55e41d305090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa6905d2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d3055d0 0x55e41d305a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa684ff9640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa6700a33b0 0x7fa6700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa6905d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 0x55e41d312760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa6905d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e41d26b2f0 con 0x55e41d306b40 2024-09-19T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa6905d2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e41d212cf0 con 0x55e41d3055d0 2024-09-19T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa6905d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e41d2672b0 con 0x55e41d310370 2024-09-19T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa686ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d306b40 0x55e41d305090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa686ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d306b40 0x55e41d305090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58914/0 (socket says 172.21.15.5:58914) 2024-09-19T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa686ffd640 1 -- 172.21.15.5:0/314053299 learned_addr learned my addr 172.21.15.5:0/314053299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa687fff640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa6700a33b0 0x7fa6700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa6877fe640 1 --2- 172.21.15.5:0/314053299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d3055d0 0x55e41d305a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa686ffd640 1 -- 172.21.15.5:0/314053299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 msgr2=0x55e41d312760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa686ffd640 1 --2- 172.21.15.5:0/314053299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 0x55e41d312760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa684ff9640 1 -- 172.21.15.5:0/2594840183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fa68002e030 con 0x7fa688106a80 2024-09-19T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa686ffd640 1 -- 172.21.15.5:0/314053299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d3055d0 msgr2=0x55e41d305a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa686ffd640 1 --2- 172.21.15.5:0/314053299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d3055d0 0x55e41d305a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.324+0000 7fa686ffd640 1 -- 172.21.15.5:0/314053299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e41d1f73a0 con 0x55e41d306b40 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa686ffd640 1 --2- 172.21.15.5:0/314053299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d306b40 0x55e41d305090 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009710 tx=0x7fa67c013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a80e0661c17bb28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa687fff640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa6700a33b0 0x7fa6700a5870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa6800ba0b0 tx=0x7fa6800b92c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa66dffb640 1 -- 172.21.15.5:0/314053299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c004050 con 0x55e41d306b40 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa66dffb640 1 -- 172.21.15.5:0/314053299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa67c017030 con 0x55e41d306b40 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa66dffb640 1 -- 172.21.15.5:0/314053299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c010430 con 0x55e41d306b40 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa6905d2740 1 -- 172.21.15.5:0/314053299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d306b40 msgr2=0x55e41d305090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa6905d2740 1 --2- 172.21.15.5:0/314053299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d306b40 0x55e41d305090 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009710 tx=0x7fa67c013830 comp rx=0 tx=0).stop 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa6905d2740 1 -- 172.21.15.5:0/314053299 shutdown_connections 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa6905d2740 1 --2- 172.21.15.5:0/314053299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 0x55e41d312760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa6905d2740 1 --2- 172.21.15.5:0/314053299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d3055d0 0x55e41d305a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa6905d2740 1 --2- 172.21.15.5:0/314053299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d306b40 0x55e41d305090 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa6905d2740 1 -- 172.21.15.5:0/314053299 >> 172.21.15.5:0/314053299 conn(0x55e41d2ce310 msgr2=0x55e41d301c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa6905d2740 1 -- 172.21.15.5:0/314053299 shutdown_connections 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.325+0000 7fa6905d2740 1 -- 172.21.15.5:0/314053299 wait complete. 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.326+0000 7fa6905d2740 1 -- start start 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.326+0000 7fa6905d2740 1 Processor -- start 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.326+0000 7fa6905d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d3055d0 0x55e41d30fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.326+0000 7fa6905d2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d30cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.326+0000 7fa6905d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d310370 0x55e41d30d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.326+0000 7fa6905d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e41d313070 con 0x55e41d310370 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa6905d2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e41d226060 con 0x55e41d306b40 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa6905d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e41d3131f0 con 0x55e41d3055d0 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa6905d2740 10 client.0 _ll_register_callbacks cb 0x55e41cf798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa686ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d310370 0x55e41d30d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa686ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d310370 0x55e41d30d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58918/0 (socket says 172.21.15.5:58918) 2024-09-19T06:16:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa6877fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d30cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa686ffd640 1 -- 172.21.15.5:0/1250767247 learned_addr learned my addr 172.21.15.5:0/1250767247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:01.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa686ffd640 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d3055d0 0x55e41d30fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa686ffd640 1 -- 172.21.15.5:0/1250767247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d3055d0 msgr2=0x55e41d30fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa686ffd640 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d3055d0 0x55e41d30fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa686ffd640 1 -- 172.21.15.5:0/1250767247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 msgr2=0x55e41d30cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa686ffd640 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d30cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa686ffd640 1 -- 172.21.15.5:0/1250767247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e41cfdc3c0 con 0x55e41d310370 2024-09-19T06:16:01.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa6877fe640 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d30cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:01.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa686ffd640 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d310370 0x55e41d30d5d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa67c013fd0 tx=0x7fa67c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62805239232c9cc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa666ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:01.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa666ffd640 1 -- 172.21.15.5:0/1250767247 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c013180 con 0x55e41d310370 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa666ffd640 1 -- 172.21.15.5:0/1250767247 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fa67c0035f0 con 0x55e41d310370 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.327+0000 7fa6905d2740 1 -- 172.21.15.5:0/1250767247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa67c017030 con 0x55e41d310370 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.328+0000 7fa666ffd640 1 -- 172.21.15.5:0/1250767247 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa67c0037f0 con 0x55e41d310370 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.328+0000 7fa6905d2740 10 client.18753 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.328+0000 7fa6905d2740 1 -- 172.21.15.5:0/1250767247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e41d31ead0 con 0x55e41d310370 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.328+0000 7fa666ffd640 1 -- 172.21.15.5:0/1250767247 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa67c017030 con 0x55e41d310370 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.328+0000 7fa666ffd640 1 -- 172.21.15.5:0/1250767247 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fa67c02d4f0 con 0x55e41d310370 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.328+0000 7fa6905d2740 10 client.18753 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.328+0000 7fa6905d2740 10 client.18753 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.328+0000 7fa6905d2740 20 client.18753 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.328+0000 7fa6905d2740 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55e41d31fee0 0x55e41d3222d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.328+0000 7fa6905d2740 4 client.18753 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.328+0000 7fa6905d2740 1 -- 172.21.15.5:0/1250767247 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e41d278860 con 0x55e41d31fee0 2024-09-19T06:16:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.328+0000 7fa6877fe640 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55e41d31fee0 0x55e41d3222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.329+0000 7fa6877fe640 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55e41d31fee0 0x55e41d3222d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85fd332dcc6368a2 server_cookie=8804e954d4d406fd in_seq=0 out_seq=0 2024-09-19T06:16:01.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.329+0000 7fa666ffd640 10 client.18753 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:16:01.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa666ffd640 1 -- 172.21.15.5:0/1250767247 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e41d278860 con 0x55e41d31fee0 2024-09-19T06:16:01.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa666ffd640 10 client.18753 handle_command_reply: tid=0 2024-09-19T06:16:01.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa6905d2740 1 client.18753 shutdown 2024-09-19T06:16:01.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa6905d2740 1 -- 172.21.15.5:0/1250767247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d310370 msgr2=0x55e41d30d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa6905d2740 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d310370 0x55e41d30d5d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa67c013fd0 tx=0x7fa67c004610 comp rx=0 tx=0).stop 2024-09-19T06:16:01.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa6905d2740 1 -- 172.21.15.5:0/1250767247 shutdown_connections 2024-09-19T06:16:01.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa6905d2740 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55e41d31fee0 0x55e41d3222d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa6905d2740 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d310370 0x55e41d30d5d0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa666ffd640 0 client.18753 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:16:01.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa6905d2740 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d30cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa6905d2740 1 --2- 172.21.15.5:0/1250767247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d3055d0 0x55e41d30fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa6905d2740 1 -- 172.21.15.5:0/1250767247 >> 172.21.15.5:0/1250767247 conn(0x55e41d2ce310 msgr2=0x55e41d307930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa6905d2740 1 -- 172.21.15.5:0/1250767247 shutdown_connections 2024-09-19T06:16:01.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa6905d2740 1 -- 172.21.15.5:0/1250767247 wait complete. 2024-09-19T06:16:01.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.330+0000 7fa6905d2740 20 client.18753 trim_cache size 0 max 16384 2024-09-19T06:16:01.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.343+0000 7fa6905d2740 1 Processor -- start 2024-09-19T06:16:01.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.343+0000 7fa6905d2740 1 -- start start 2024-09-19T06:16:01.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa6905d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 0x55e41d31c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa6905d2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d30bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa6905d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 0x55e41d318860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa6905d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881ad6e0 con 0x55e41d3055d0 2024-09-19T06:16:01.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa6905d2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881af2e0 con 0x55e41d306b40 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa6905d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881addb0 con 0x55e41d310370 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa686ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 0x55e41d31c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa686ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 0x55e41d31c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58928/0 (socket says 172.21.15.5:58928) 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa686ffd640 1 -- 172.21.15.5:0/175997452 learned_addr learned my addr 172.21.15.5:0/175997452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa686ffd640 1 -- 172.21.15.5:0/175997452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 msgr2=0x55e41d318860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa686ffd640 1 --2- 172.21.15.5:0/175997452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 0x55e41d318860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa686ffd640 1 -- 172.21.15.5:0/175997452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 msgr2=0x55e41d30bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa6877fe640 1 --2- 172.21.15.5:0/175997452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d30bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa686ffd640 1 --2- 172.21.15.5:0/175997452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d30bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa686ffd640 1 -- 172.21.15.5:0/175997452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e41d30b530 con 0x55e41d3055d0 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa686ffd640 1 --2- 172.21.15.5:0/175997452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 0x55e41d31c310 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0059f0 tx=0x7fa67c02ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4da70a94edfe6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa66d7fa640 1 -- 172.21.15.5:0/175997452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c037070 con 0x55e41d3055d0 2024-09-19T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa66d7fa640 1 -- 172.21.15.5:0/175997452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa67c017040 con 0x55e41d3055d0 2024-09-19T06:16:01.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.344+0000 7fa66d7fa640 1 -- 172.21.15.5:0/175997452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c02d580 con 0x55e41d3055d0 2024-09-19T06:16:01.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.345+0000 7fa6905d2740 1 -- 172.21.15.5:0/175997452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 msgr2=0x55e41d31c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.345+0000 7fa6905d2740 1 --2- 172.21.15.5:0/175997452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 0x55e41d31c310 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0059f0 tx=0x7fa67c02ef60 comp rx=0 tx=0).stop 2024-09-19T06:16:01.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.345+0000 7fa6905d2740 1 -- 172.21.15.5:0/175997452 shutdown_connections 2024-09-19T06:16:01.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.345+0000 7fa6905d2740 1 --2- 172.21.15.5:0/175997452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 0x55e41d318860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.345+0000 7fa6905d2740 1 --2- 172.21.15.5:0/175997452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d30bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.345+0000 7fa6905d2740 1 --2- 172.21.15.5:0/175997452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 0x55e41d31c310 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.345+0000 7fa6905d2740 1 -- 172.21.15.5:0/175997452 >> 172.21.15.5:0/175997452 conn(0x55e41d2ce310 msgr2=0x55e41d30b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.345+0000 7fa6905d2740 1 -- 172.21.15.5:0/175997452 shutdown_connections 2024-09-19T06:16:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.345+0000 7fa6905d2740 1 -- 172.21.15.5:0/175997452 wait complete. 2024-09-19T06:16:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.345+0000 7fa6905d2740 1 -- start start 2024-09-19T06:16:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa6905d2740 1 Processor -- start 2024-09-19T06:16:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa6905d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 0x55e41d385d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa6905d2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d386260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa6905d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 0x55e41d326f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa6905d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881aec10 con 0x55e41d3055d0 2024-09-19T06:16:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa6905d2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881ae4e0 con 0x55e41d306b40 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa6905d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881ab6e0 con 0x55e41d310370 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa6905d2740 10 client.0 _ll_register_callbacks cb 0x7fa67800b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa686ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 0x55e41d385d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa686ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 0x55e41d385d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58936/0 (socket says 172.21.15.5:58936) 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa686ffd640 1 -- 172.21.15.5:0/1338992558 learned_addr learned my addr 172.21.15.5:0/1338992558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa686ffd640 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 0x55e41d326f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa6877fe640 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d386260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa686ffd640 1 -- 172.21.15.5:0/1338992558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 msgr2=0x55e41d326f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa686ffd640 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 0x55e41d326f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa686ffd640 1 -- 172.21.15.5:0/1338992558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 msgr2=0x55e41d386260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa686ffd640 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d386260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.346+0000 7fa686ffd640 1 -- 172.21.15.5:0/1338992558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e41d327790 con 0x55e41d3055d0 2024-09-19T06:16:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa686ffd640 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 0x55e41d385d20 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa67c02d8e0 tx=0x7fa67c005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83942e155c6c6394 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa666ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa666ffd640 1 -- 172.21.15.5:0/1338992558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c037040 con 0x55e41d3055d0 2024-09-19T06:16:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa6905d2740 1 -- 172.21.15.5:0/1338992558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa67c017040 con 0x55e41d3055d0 2024-09-19T06:16:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa666ffd640 1 -- 172.21.15.5:0/1338992558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fa67c013180 con 0x55e41d3055d0 2024-09-19T06:16:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa666ffd640 1 -- 172.21.15.5:0/1338992558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa67c003a30 con 0x55e41d3055d0 2024-09-19T06:16:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa6905d2740 10 client.18759 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa6905d2740 1 -- 172.21.15.5:0/1338992558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e41d38e4e0 con 0x55e41d3055d0 2024-09-19T06:16:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa666ffd640 1 -- 172.21.15.5:0/1338992558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa67c017040 con 0x55e41d3055d0 2024-09-19T06:16:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa666ffd640 1 -- 172.21.15.5:0/1338992558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fa67c03c450 con 0x55e41d3055d0 2024-09-19T06:16:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa6905d2740 10 client.18759 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa6905d2740 10 client.18759 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:16:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa6905d2740 20 client.18759 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa6905d2740 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55e41d31fee0 0x55e41d390aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa6905d2740 4 client.18759 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:16:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.347+0000 7fa6905d2740 1 -- 172.21.15.5:0/1338992558 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e41d272d70 con 0x55e41d31fee0 2024-09-19T06:16:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.348+0000 7fa6877fe640 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55e41d31fee0 0x55e41d390aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.348+0000 7fa6877fe640 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55e41d31fee0 0x55e41d390aa0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf4a7bd688e9f49f server_cookie=d9fc94d3432e6e2 in_seq=0 out_seq=0 2024-09-19T06:16:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.348+0000 7fa666ffd640 10 client.18759 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:16:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.348+0000 7fa666ffd640 1 -- 172.21.15.5:0/1338992558 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e41d272d70 con 0x55e41d31fee0 2024-09-19T06:16:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.348+0000 7fa666ffd640 10 client.18759 handle_command_reply: tid=0 2024-09-19T06:16:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.348+0000 7fa6905d2740 1 client.18759 shutdown 2024-09-19T06:16:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.348+0000 7fa6905d2740 1 -- 172.21.15.5:0/1338992558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 msgr2=0x55e41d385d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.348+0000 7fa6905d2740 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 0x55e41d385d20 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa67c02d8e0 tx=0x7fa67c005570 comp rx=0 tx=0).stop 2024-09-19T06:16:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa6905d2740 1 -- 172.21.15.5:0/1338992558 shutdown_connections 2024-09-19T06:16:01.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa6905d2740 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55e41d31fee0 0x55e41d390aa0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa6905d2740 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e41d310370 0x55e41d326f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa6905d2740 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e41d306b40 0x55e41d386260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa6905d2740 1 --2- 172.21.15.5:0/1338992558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e41d3055d0 0x55e41d385d20 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa6905d2740 1 -- 172.21.15.5:0/1338992558 >> 172.21.15.5:0/1338992558 conn(0x55e41d2ce310 msgr2=0x55e41d30c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa666ffd640 0 client.18759 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:16:01.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa6905d2740 1 -- 172.21.15.5:0/1338992558 shutdown_connections 2024-09-19T06:16:01.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa6905d2740 1 -- 172.21.15.5:0/1338992558 wait complete. 2024-09-19T06:16:01.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa6905d2740 20 client.18759 trim_cache size 0 max 16384 2024-09-19T06:16:01.356 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:16:01.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa68e1f9640 1 -- 172.21.15.5:0/2594840183 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa6700a33b0 msgr2=0x7fa6700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa68e1f9640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa6700a33b0 0x7fa6700a5870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa6800ba0b0 tx=0x7fa6800b92c0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa68e1f9640 1 -- 172.21.15.5:0/2594840183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa688106a80 msgr2=0x7fa68819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa68e1f9640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa688106a80 0x7fa68819cbb0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa680033040 tx=0x7fa680003970 comp rx=0 tx=0).stop 2024-09-19T06:16:01.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa68e1f9640 1 -- 172.21.15.5:0/2594840183 shutdown_connections 2024-09-19T06:16:01.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa68e1f9640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa688106a80 0x7fa68819cbb0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa68e1f9640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa688101420 0x7fa688198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa68e1f9640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa6700a33b0 0x7fa6700a5870 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa68e1f9640 1 --2- 172.21.15.5:0/2594840183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa688100a80 0x7fa6881982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.349+0000 7fa68e1f9640 1 -- 172.21.15.5:0/2594840183 >> 172.21.15.5:0/2594840183 conn(0x7fa6880fc7a0 msgr2=0x7fa68806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.350+0000 7fa68e1f9640 1 -- 172.21.15.5:0/2594840183 shutdown_connections 2024-09-19T06:16:01.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:01.350+0000 7fa68e1f9640 1 -- 172.21.15.5:0/2594840183 wait complete. 2024-09-19T06:16:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.737+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1324383008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf70107ad0 msgr2=0x7fcf70107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.737+0000 7fcf75ca5640 1 --2- 172.21.15.5:0/1324383008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf70107ad0 0x7fcf70107eb0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009930 tx=0x7fcf6002a420 comp rx=0 tx=0).stop 2024-09-19T06:16:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.737+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1324383008 shutdown_connections 2024-09-19T06:16:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.737+0000 7fcf75ca5640 1 --2- 172.21.15.5:0/1324383008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf70102460 0x7fcf7010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.737+0000 7fcf75ca5640 1 --2- 172.21.15.5:0/1324383008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf70101ac0 0x7fcf70101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.737+0000 7fcf75ca5640 1 --2- 172.21.15.5:0/1324383008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf70107ad0 0x7fcf70107eb0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.737+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1324383008 >> 172.21.15.5:0/1324383008 conn(0x7fcf700fd7e0 msgr2=0x7fcf700ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.737+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1324383008 shutdown_connections 2024-09-19T06:16:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.737+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1324383008 wait complete. 2024-09-19T06:16:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.738+0000 7fcf75ca5640 1 Processor -- start 2024-09-19T06:16:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.738+0000 7fcf75ca5640 1 -- start start 2024-09-19T06:16:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.738+0000 7fcf75ca5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf70101ac0 0x7fcf701982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.738+0000 7fcf75ca5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf70102460 0x7fcf70198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.738+0000 7fcf75ca5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf70107ad0 0x7fcf7019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.738+0000 7fcf75ca5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf70111dc0 con 0x7fcf70102460 2024-09-19T06:16:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.738+0000 7fcf75ca5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcf70111c40 con 0x7fcf70101ac0 2024-09-19T06:16:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.738+0000 7fcf75ca5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf70111f40 con 0x7fcf70107ad0 2024-09-19T06:16:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.738+0000 7fcf6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf70102460 0x7fcf70198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.738+0000 7fcf6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf70102460 0x7fcf70198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58948/0 (socket says 172.21.15.5:58948) 2024-09-19T06:16:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.738+0000 7fcf6effd640 1 -- 172.21.15.5:0/1961541732 learned_addr learned my addr 172.21.15.5:0/1961541732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf6ffff640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf70107ad0 0x7fcf7019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf6f7fe640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf70101ac0 0x7fcf701982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf6effd640 1 -- 172.21.15.5:0/1961541732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf70107ad0 msgr2=0x7fcf7019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf6effd640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf70107ad0 0x7fcf7019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf6effd640 1 -- 172.21.15.5:0/1961541732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf70101ac0 msgr2=0x7fcf701982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf6effd640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf70101ac0 0x7fcf701982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf6effd640 1 -- 172.21.15.5:0/1961541732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7019d2c0 con 0x7fcf70102460 2024-09-19T06:16:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf6effd640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf70102460 0x7fcf70198810 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400c890 tx=0x7fcf6400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf6cff9640 1 -- 172.21.15.5:0/1961541732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64004220 con 0x7fcf70102460 2024-09-19T06:16:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf6cff9640 1 -- 172.21.15.5:0/1961541732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf640043c0 con 0x7fcf70102460 2024-09-19T06:16:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1961541732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7019d5b0 con 0x7fcf70102460 2024-09-19T06:16:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf6cff9640 1 -- 172.21.15.5:0/1961541732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64004a00 con 0x7fcf70102460 2024-09-19T06:16:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.739+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1961541732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf701a4e40 con 0x7fcf70102460 2024-09-19T06:16:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.741+0000 7fcf6cff9640 1 -- 172.21.15.5:0/1961541732 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcf64004ba0 con 0x7fcf70102460 2024-09-19T06:16:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf78a44740 1 Processor -- start 2024-09-19T06:16:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf78a44740 1 -- start start 2024-09-19T06:16:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf78a44740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf78a44740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf78a44740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 0x564e233d80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf78a44740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e2332f2f0 con 0x564e233cb410 2024-09-19T06:16:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf78a44740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564e232d7270 con 0x564e233caaf0 2024-09-19T06:16:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf78a44740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564e2332b2b0 con 0x564e233d5cb0 2024-09-19T06:16:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58952/0 (socket says 172.21.15.5:58952) 2024-09-19T06:16:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf6f7fe640 1 -- 172.21.15.5:0/3609149004 learned_addr learned my addr 172.21.15.5:0/3609149004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf6effd640 1 --2- 172.21.15.5:0/3609149004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 0x564e233d80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf6ffff640 1 --2- 172.21.15.5:0/3609149004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.742+0000 7fcf6cff9640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcf580a33b0 0x7fcf580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.743+0000 7fcf6f7fe640 1 -- 172.21.15.5:0/3609149004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 msgr2=0x564e233d80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.743+0000 7fcf6f7fe640 1 --2- 172.21.15.5:0/3609149004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 0x564e233d80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.743+0000 7fcf6f7fe640 1 -- 172.21.15.5:0/3609149004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 msgr2=0x564e233caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.743+0000 7fcf6f7fe640 1 --2- 172.21.15.5:0/3609149004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.743+0000 7fcf6f7fe640 1 -- 172.21.15.5:0/3609149004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e232d6340 con 0x564e233cb410 2024-09-19T06:16:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.743+0000 7fcf6ffff640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcf580a33b0 0x7fcf580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.743+0000 7fcf6cff9640 1 -- 172.21.15.5:0/1961541732 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fcf640cfa20 con 0x7fcf70102460 2024-09-19T06:16:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.743+0000 7fcf6f7fe640 1 --2- 172.21.15.5:0/3609149004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d5770 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fcf6002a940 tx=0x7fcf60031cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc318260fc32c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.743+0000 7fcf55ffb640 1 -- 172.21.15.5:0/3609149004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6003f070 con 0x564e233cb410 2024-09-19T06:16:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.743+0000 7fcf6ffff640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcf580a33b0 0x7fcf580a5870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcf68007fc0 tx=0x7fcf68004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.743+0000 7fcf55ffb640 1 -- 172.21.15.5:0/3609149004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf600316a0 con 0x564e233cb410 2024-09-19T06:16:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.743+0000 7fcf55ffb640 1 -- 172.21.15.5:0/3609149004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60035070 con 0x564e233cb410 2024-09-19T06:16:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.744+0000 7fcf78a44740 1 -- 172.21.15.5:0/3609149004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 msgr2=0x564e233d5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.744+0000 7fcf78a44740 1 --2- 172.21.15.5:0/3609149004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d5770 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fcf6002a940 tx=0x7fcf60031cb0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.744+0000 7fcf78a44740 1 -- 172.21.15.5:0/3609149004 shutdown_connections 2024-09-19T06:16:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.744+0000 7fcf78a44740 1 --2- 172.21.15.5:0/3609149004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 0x564e233d80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.744+0000 7fcf78a44740 1 --2- 172.21.15.5:0/3609149004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d5770 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.744+0000 7fcf78a44740 1 --2- 172.21.15.5:0/3609149004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.744+0000 7fcf78a44740 1 -- 172.21.15.5:0/3609149004 >> 172.21.15.5:0/3609149004 conn(0x564e233c7060 msgr2=0x564e233c7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.744+0000 7fcf78a44740 1 -- 172.21.15.5:0/3609149004 shutdown_connections 2024-09-19T06:16:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.744+0000 7fcf78a44740 1 -- 172.21.15.5:0/3609149004 wait complete. 2024-09-19T06:16:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.744+0000 7fcf78a44740 1 -- start start 2024-09-19T06:16:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.744+0000 7fcf78a44740 1 Processor -- start 2024-09-19T06:16:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf78a44740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233d2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf78a44740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf78a44740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 0x564e233d3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf78a44740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e233d8880 con 0x564e233cb410 2024-09-19T06:16:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf78a44740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564e232b8290 con 0x564e233caaf0 2024-09-19T06:16:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf78a44740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564e233d8a00 con 0x564e233d5cb0 2024-09-19T06:16:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf78a44740 10 client.0 _ll_register_callbacks cb 0x564e232e9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58958/0 (socket says 172.21.15.5:58958) 2024-09-19T06:16:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf6effd640 1 -- 172.21.15.5:0/1563509861 learned_addr learned my addr 172.21.15.5:0/1563509861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf6f7fe640 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233d2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf6ffff640 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 0x564e233d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf6effd640 1 -- 172.21.15.5:0/1563509861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 msgr2=0x564e233d3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf6effd640 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 0x564e233d3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf6effd640 1 -- 172.21.15.5:0/1563509861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 msgr2=0x564e233d2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf6effd640 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233d2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf6effd640 1 -- 172.21.15.5:0/1563509861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e23042210 con 0x564e233cb410 2024-09-19T06:16:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf6ffff640 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 0x564e233d3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf6effd640 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d3030 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fcf64098120 tx=0x7fcf64005380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8119cf6d87556e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf4effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf4effd640 1 -- 172.21.15.5:0/1563509861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6401d040 con 0x564e233cb410 2024-09-19T06:16:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf4effd640 1 -- 172.21.15.5:0/1563509861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fcf640cd3b0 con 0x564e233cb410 2024-09-19T06:16:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.745+0000 7fcf78a44740 1 -- 172.21.15.5:0/1563509861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf600316a0 con 0x564e233cb410 2024-09-19T06:16:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.746+0000 7fcf4effd640 1 -- 172.21.15.5:0/1563509861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcf6409b800 con 0x564e233cb410 2024-09-19T06:16:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.746+0000 7fcf78a44740 10 client.18771 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.746+0000 7fcf78a44740 1 -- 172.21.15.5:0/1563509861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564e2310e620 con 0x564e233cb410 2024-09-19T06:16:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.746+0000 7fcf4effd640 1 -- 172.21.15.5:0/1563509861 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf600316a0 con 0x564e233cb410 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.746+0000 7fcf4effd640 1 -- 172.21.15.5:0/1563509861 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fcf640a44a0 con 0x564e233cb410 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.746+0000 7fcf78a44740 10 client.18771 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.746+0000 7fcf78a44740 10 client.18771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.746+0000 7fcf78a44740 20 client.18771 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.746+0000 7fcf78a44740 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564e233e5c00 0x564e233e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.746+0000 7fcf78a44740 4 client.18771 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.746+0000 7fcf78a44740 1 -- 172.21.15.5:0/1563509861 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e2333ca20 con 0x564e233e5c00 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.746+0000 7fcf6f7fe640 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564e233e5c00 0x564e233e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.747+0000 7fcf4effd640 10 client.18771 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.747+0000 7fcf6f7fe640 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564e233e5c00 0x564e233e7ff0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7206a03ad9ea3f43 server_cookie=5fde07d0d42f9e59 in_seq=0 out_seq=0 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.747+0000 7fcf4effd640 1 -- 172.21.15.5:0/1563509861 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e2333ca20 con 0x564e233e5c00 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.747+0000 7fcf4effd640 10 client.18771 handle_command_reply: tid=0 2024-09-19T06:16:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.747+0000 7fcf78a44740 1 client.18771 shutdown 2024-09-19T06:16:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.748+0000 7fcf78a44740 1 -- 172.21.15.5:0/1563509861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 msgr2=0x564e233d3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.748+0000 7fcf78a44740 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d3030 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fcf64098120 tx=0x7fcf64005380 comp rx=0 tx=0).stop 2024-09-19T06:16:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.748+0000 7fcf78a44740 1 -- 172.21.15.5:0/1563509861 shutdown_connections 2024-09-19T06:16:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.748+0000 7fcf78a44740 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564e233e5c00 0x564e233e7ff0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.748+0000 7fcf78a44740 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 0x564e233d3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.748+0000 7fcf78a44740 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d3030 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.748+0000 7fcf4effd640 0 client.18771 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.748+0000 7fcf78a44740 1 --2- 172.21.15.5:0/1563509861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233d2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.748+0000 7fcf78a44740 1 -- 172.21.15.5:0/1563509861 >> 172.21.15.5:0/1563509861 conn(0x564e233c7060 msgr2=0x564e233ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.748+0000 7fcf78a44740 1 -- 172.21.15.5:0/1563509861 shutdown_connections 2024-09-19T06:16:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.748+0000 7fcf78a44740 1 -- 172.21.15.5:0/1563509861 wait complete. 2024-09-19T06:16:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.748+0000 7fcf78a44740 20 client.18771 trim_cache size 0 max 16384 2024-09-19T06:16:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.762+0000 7fcf78a44740 1 Processor -- start 2024-09-19T06:16:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.762+0000 7fcf78a44740 1 -- start start 2024-09-19T06:16:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.762+0000 7fcf78a44740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233caaf0 0x564e233d33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.762+0000 7fcf78a44740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233ed8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.762+0000 7fcf78a44740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233d5cb0 0x564e233ede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.762+0000 7fcf78a44740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf701ad740 con 0x564e233cb410 2024-09-19T06:16:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.762+0000 7fcf78a44740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcf701af340 con 0x564e233d5cb0 2024-09-19T06:16:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.762+0000 7fcf78a44740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf701ade10 con 0x564e233caaf0 2024-09-19T06:16:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233ed8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233ed8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58970/0 (socket says 172.21.15.5:58970) 2024-09-19T06:16:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf6effd640 1 -- 172.21.15.5:0/2113085952 learned_addr learned my addr 172.21.15.5:0/2113085952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf6f7fe640 1 --2- 172.21.15.5:0/2113085952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233caaf0 0x564e233d33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf6effd640 1 -- 172.21.15.5:0/2113085952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233caaf0 msgr2=0x564e233d33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf6effd640 1 --2- 172.21.15.5:0/2113085952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233caaf0 0x564e233d33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf6effd640 1 -- 172.21.15.5:0/2113085952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233d5cb0 msgr2=0x564e233ede20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf6effd640 1 --2- 172.21.15.5:0/2113085952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233d5cb0 0x564e233ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf6effd640 1 -- 172.21.15.5:0/2113085952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e233e8360 con 0x564e233cb410 2024-09-19T06:16:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf6effd640 1 --2- 172.21.15.5:0/2113085952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233ed8e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fcf64005fd0 tx=0x7fcf640053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677957a500ee7714 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf557fa640 1 -- 172.21.15.5:0/2113085952 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6401d070 con 0x564e233cb410 2024-09-19T06:16:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf557fa640 1 -- 172.21.15.5:0/2113085952 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf640cfe50 con 0x564e233cb410 2024-09-19T06:16:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.763+0000 7fcf557fa640 1 -- 172.21.15.5:0/2113085952 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6409b620 con 0x564e233cb410 2024-09-19T06:16:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.764+0000 7fcf78a44740 1 -- 172.21.15.5:0/2113085952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 msgr2=0x564e233ed8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.764+0000 7fcf78a44740 1 --2- 172.21.15.5:0/2113085952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233ed8e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fcf64005fd0 tx=0x7fcf640053f0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.764+0000 7fcf78a44740 1 -- 172.21.15.5:0/2113085952 shutdown_connections 2024-09-19T06:16:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.764+0000 7fcf78a44740 1 --2- 172.21.15.5:0/2113085952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233d5cb0 0x564e233ede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.764+0000 7fcf78a44740 1 --2- 172.21.15.5:0/2113085952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233ed8e0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.764+0000 7fcf78a44740 1 --2- 172.21.15.5:0/2113085952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233caaf0 0x564e233d33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.764+0000 7fcf78a44740 1 -- 172.21.15.5:0/2113085952 >> 172.21.15.5:0/2113085952 conn(0x564e233c7060 msgr2=0x564e233d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.764+0000 7fcf78a44740 1 -- 172.21.15.5:0/2113085952 shutdown_connections 2024-09-19T06:16:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.764+0000 7fcf78a44740 1 -- 172.21.15.5:0/2113085952 wait complete. 2024-09-19T06:16:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.764+0000 7fcf78a44740 1 -- start start 2024-09-19T06:16:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.764+0000 7fcf78a44740 1 Processor -- start 2024-09-19T06:16:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.764+0000 7fcf78a44740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233de510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf78a44740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf78a44740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 0x564e233d9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf78a44740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf701aec70 con 0x564e233cb410 2024-09-19T06:16:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf78a44740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcf701ae540 con 0x564e233caaf0 2024-09-19T06:16:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf78a44740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf701ab740 con 0x564e233d5cb0 2024-09-19T06:16:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf78a44740 10 client.0 _ll_register_callbacks cb 0x7fcf640058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58976/0 (socket says 172.21.15.5:58976) 2024-09-19T06:16:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf6effd640 1 -- 172.21.15.5:0/994718523 learned_addr learned my addr 172.21.15.5:0/994718523 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf6f7fe640 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233de510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf6effd640 1 -- 172.21.15.5:0/994718523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 msgr2=0x564e233d9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf6effd640 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 0x564e233d9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf6effd640 1 -- 172.21.15.5:0/994718523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 msgr2=0x564e233de510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf6effd640 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233de510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf6effd640 1 -- 172.21.15.5:0/994718523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e233da670 con 0x564e233cb410 2024-09-19T06:16:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf6f7fe640 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233de510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf6effd640 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d9830 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fcf640cd730 tx=0x7fcf64005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2206339402304afb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf4effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf4effd640 1 -- 172.21.15.5:0/994718523 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf640b8020 con 0x564e233cb410 2024-09-19T06:16:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf4effd640 1 -- 172.21.15.5:0/994718523 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fcf640a44a0 con 0x564e233cb410 2024-09-19T06:16:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf78a44740 1 -- 172.21.15.5:0/994718523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf640cfe50 con 0x564e233cb410 2024-09-19T06:16:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf4effd640 1 -- 172.21.15.5:0/994718523 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcf6401d050 con 0x564e233cb410 2024-09-19T06:16:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf78a44740 10 client.18777 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.765+0000 7fcf78a44740 1 -- 172.21.15.5:0/994718523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564e234509f0 con 0x564e233cb410 2024-09-19T06:16:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.766+0000 7fcf4effd640 1 -- 172.21.15.5:0/994718523 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf640cfe50 con 0x564e233cb410 2024-09-19T06:16:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.766+0000 7fcf4effd640 1 -- 172.21.15.5:0/994718523 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fcf640a1200 con 0x564e233cb410 2024-09-19T06:16:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.766+0000 7fcf78a44740 10 client.18777 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.766+0000 7fcf78a44740 10 client.18777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.766+0000 7fcf78a44740 20 client.18777 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.766+0000 7fcf78a44740 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564e23450e10 0x564e23453230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.766+0000 7fcf78a44740 4 client.18777 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:16:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.766+0000 7fcf78a44740 1 -- 172.21.15.5:0/994718523 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e23336d70 con 0x564e23450e10 2024-09-19T06:16:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.766+0000 7fcf6f7fe640 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564e23450e10 0x564e23453230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.766+0000 7fcf4effd640 10 client.18777 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.766+0000 7fcf6f7fe640 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564e23450e10 0x564e23453230 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd663dbd3128f2cf server_cookie=70bb7c9a86f6d00a in_seq=0 out_seq=0 2024-09-19T06:16:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf4effd640 1 -- 172.21.15.5:0/994718523 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e23336d70 con 0x564e23450e10 2024-09-19T06:16:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf4effd640 10 client.18777 handle_command_reply: tid=0 2024-09-19T06:16:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf78a44740 1 client.18777 shutdown 2024-09-19T06:16:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf78a44740 1 -- 172.21.15.5:0/994718523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 msgr2=0x564e233d9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf78a44740 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d9830 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fcf640cd730 tx=0x7fcf64005760 comp rx=0 tx=0).stop 2024-09-19T06:16:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf78a44740 1 -- 172.21.15.5:0/994718523 shutdown_connections 2024-09-19T06:16:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf78a44740 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564e23450e10 0x564e23453230 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf78a44740 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e233d5cb0 0x564e233d9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf78a44740 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e233cb410 0x564e233d9830 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf4effd640 0 client.18777 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf78a44740 1 --2- 172.21.15.5:0/994718523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e233caaf0 0x564e233de510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf78a44740 1 -- 172.21.15.5:0/994718523 >> 172.21.15.5:0/994718523 conn(0x564e233c7060 msgr2=0x564e233c59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf78a44740 1 -- 172.21.15.5:0/994718523 shutdown_connections 2024-09-19T06:16:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf78a44740 1 -- 172.21.15.5:0/994718523 wait complete. 2024-09-19T06:16:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.793+0000 7fcf78a44740 20 client.18777 trim_cache size 0 max 16384 2024-09-19T06:16:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:16:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:16:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:16:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:16:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.794+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1961541732 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcf580a33b0 msgr2=0x7fcf580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.794+0000 7fcf75ca5640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcf580a33b0 0x7fcf580a5870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcf68007fc0 tx=0x7fcf68004a40 comp rx=0 tx=0).stop 2024-09-19T06:16:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.794+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1961541732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf70102460 msgr2=0x7fcf70198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.794+0000 7fcf75ca5640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf70102460 0x7fcf70198810 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400c890 tx=0x7fcf6400cd50 comp rx=0 tx=0).stop 2024-09-19T06:16:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.794+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1961541732 shutdown_connections 2024-09-19T06:16:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.794+0000 7fcf75ca5640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcf580a33b0 0x7fcf580a5870 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.794+0000 7fcf75ca5640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf70107ad0 0x7fcf7019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.794+0000 7fcf75ca5640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf70102460 0x7fcf70198810 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.794+0000 7fcf75ca5640 1 --2- 172.21.15.5:0/1961541732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf70101ac0 0x7fcf701982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.794+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1961541732 >> 172.21.15.5:0/1961541732 conn(0x7fcf700fd7e0 msgr2=0x7fcf700ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.794+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1961541732 shutdown_connections 2024-09-19T06:16:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:01.794+0000 7fcf75ca5640 1 -- 172.21.15.5:0/1961541732 wait complete. 2024-09-19T06:16:01.849 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287567 2.15 MB/sec execute 567 sec latency 1089.316 ms 2024-09-19T06:16:01.983 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:01 smithi005 ceph-mon[30955]: pgmap v396: 97 pgs: 97 active+clean; 160 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 865 KiB/s wr, 83 op/s 2024-09-19T06:16:02.010 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:16:02.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:01 smithi049 ceph-mon[34640]: pgmap v396: 97 pgs: 97 active+clean; 160 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 865 KiB/s wr, 83 op/s 2024-09-19T06:16:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:02.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:01 smithi157 ceph-mon[35346]: pgmap v396: 97 pgs: 97 active+clean; 160 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 865 KiB/s wr, 83 op/s 2024-09-19T06:16:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.684+0000 7f80e1095640 1 -- 172.21.15.5:0/2252884211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80dc103660 msgr2=0x7f80dc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.684+0000 7f80e1095640 1 --2- 172.21.15.5:0/2252884211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80dc103660 0x7f80dc10bb50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009900 tx=0x7f80d002a480 comp rx=0 tx=0).stop 2024-09-19T06:16:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.684+0000 7f80e1095640 1 -- 172.21.15.5:0/2252884211 shutdown_connections 2024-09-19T06:16:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.684+0000 7f80e1095640 1 --2- 172.21.15.5:0/2252884211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc10c170 0x7f80dc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.684+0000 7f80e1095640 1 --2- 172.21.15.5:0/2252884211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80dc103660 0x7f80dc10bb50 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.684+0000 7f80e1095640 1 --2- 172.21.15.5:0/2252884211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80dc102d40 0x7f80dc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.684+0000 7f80e1095640 1 -- 172.21.15.5:0/2252884211 >> 172.21.15.5:0/2252884211 conn(0x7f80dc0fc7a0 msgr2=0x7f80dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.684+0000 7f80e1095640 1 -- 172.21.15.5:0/2252884211 shutdown_connections 2024-09-19T06:16:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.684+0000 7f80e1095640 1 -- 172.21.15.5:0/2252884211 wait complete. 2024-09-19T06:16:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.685+0000 7f80e1095640 1 Processor -- start 2024-09-19T06:16:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.685+0000 7f80e1095640 1 -- start start 2024-09-19T06:16:02.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.685+0000 7f80e1095640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80dc102d40 0x7f80dc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.685+0000 7f80e1095640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80dc103660 0x7f80dc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.685+0000 7f80e1095640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc10c170 0x7f80dc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.685+0000 7f80e1095640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc111e20 con 0x7f80dc102d40 2024-09-19T06:16:02.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.685+0000 7f80e1095640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc111ca0 con 0x7f80dc10c170 2024-09-19T06:16:02.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.685+0000 7f80e1095640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc111fa0 con 0x7f80dc103660 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80dad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80dc102d40 0x7f80dc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80dad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80dc102d40 0x7f80dc1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58982/0 (socket says 172.21.15.5:58982) 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80dad9d640 1 -- 172.21.15.5:0/1737528337 learned_addr learned my addr 172.21.15.5:0/1737528337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80db59e640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc10c170 0x7f80dc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80da59c640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80dc103660 0x7f80dc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80dad9d640 1 -- 172.21.15.5:0/1737528337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80dc103660 msgr2=0x7f80dc100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80dad9d640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80dc103660 0x7f80dc100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80dad9d640 1 -- 172.21.15.5:0/1737528337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc10c170 msgr2=0x7f80dc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80dad9d640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc10c170 0x7f80dc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80dad9d640 1 -- 172.21.15.5:0/1737528337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80dc1a6860 con 0x7f80dc102d40 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80dad9d640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80dc102d40 0x7f80dc1009a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00d770 tx=0x7f80cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80c3fff640 1 -- 172.21.15.5:0/1737528337 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc017070 con 0x7f80dc102d40 2024-09-19T06:16:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80e1095640 1 -- 172.21.15.5:0/1737528337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80dc1a6b50 con 0x7f80dc102d40 2024-09-19T06:16:02.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80c3fff640 1 -- 172.21.15.5:0/1737528337 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80cc0054d0 con 0x7f80dc102d40 2024-09-19T06:16:02.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80c3fff640 1 -- 172.21.15.5:0/1737528337 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc004cf0 con 0x7f80dc102d40 2024-09-19T06:16:02.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.686+0000 7f80e1095640 1 -- 172.21.15.5:0/1737528337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80dc1a7060 con 0x7f80dc102d40 2024-09-19T06:16:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80c3fff640 1 -- 172.21.15.5:0/1737528337 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f80cc004e90 con 0x7f80dc102d40 2024-09-19T06:16:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80e346e740 1 Processor -- start 2024-09-19T06:16:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80e346e740 1 -- start start 2024-09-19T06:16:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80e346e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 0x5623e2c84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80e346e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 0x5623e2c84a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80e346e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c8f370 0x5623e2c91760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80e346e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5623e2bea2f0 con 0x5623e2c85b40 2024-09-19T06:16:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80e346e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5623e2b91cf0 con 0x5623e2c8f370 2024-09-19T06:16:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80e346e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5623e2be62b0 con 0x5623e2c845d0 2024-09-19T06:16:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80db59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 0x5623e2c84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80db59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 0x5623e2c84090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58992/0 (socket says 172.21.15.5:58992) 2024-09-19T06:16:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80db59e640 1 -- 172.21.15.5:0/3138359070 learned_addr learned my addr 172.21.15.5:0/3138359070 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80dad9d640 1 --2- 172.21.15.5:0/3138359070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 0x5623e2c84a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.689+0000 7f80c3fff640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f80c40a33b0 0x7f80c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80da59c640 1 --2- 172.21.15.5:0/3138359070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c8f370 0x5623e2c91760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80db59e640 1 -- 172.21.15.5:0/3138359070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 msgr2=0x5623e2c84a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80db59e640 1 --2- 172.21.15.5:0/3138359070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 0x5623e2c84a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80db59e640 1 -- 172.21.15.5:0/3138359070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c8f370 msgr2=0x5623e2c91760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80db59e640 1 --2- 172.21.15.5:0/3138359070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c8f370 0x5623e2c91760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80db59e640 1 -- 172.21.15.5:0/3138359070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e2b763a0 con 0x5623e2c85b40 2024-09-19T06:16:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80da59c640 1 --2- 172.21.15.5:0/3138359070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c8f370 0x5623e2c91760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80db59e640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f80c40a33b0 0x7f80c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80c3fff640 1 -- 172.21.15.5:0/1737528337 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f80cc0cf860 con 0x7f80dc102d40 2024-09-19T06:16:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80db59e640 1 --2- 172.21.15.5:0/3138359070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 0x5623e2c84090 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009fd0 tx=0x7f80d400f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f2a121b6995e28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80c17fa640 1 -- 172.21.15.5:0/3138359070 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d401c910 con 0x5623e2c85b40 2024-09-19T06:16:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80c17fa640 1 -- 172.21.15.5:0/3138359070 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80d401b440 con 0x5623e2c85b40 2024-09-19T06:16:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.690+0000 7f80c17fa640 1 -- 172.21.15.5:0/3138359070 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d40105d0 con 0x5623e2c85b40 2024-09-19T06:16:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.691+0000 7f80e346e740 1 -- 172.21.15.5:0/3138359070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 msgr2=0x5623e2c84090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.691+0000 7f80db59e640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f80c40a33b0 0x7f80c40a5870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f80d4020000 tx=0x7f80d401d000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.691+0000 7f80e346e740 1 --2- 172.21.15.5:0/3138359070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 0x5623e2c84090 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009fd0 tx=0x7f80d400f870 comp rx=0 tx=0).stop 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.691+0000 7f80e346e740 1 -- 172.21.15.5:0/3138359070 shutdown_connections 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.691+0000 7f80e346e740 1 --2- 172.21.15.5:0/3138359070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c8f370 0x5623e2c91760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.691+0000 7f80e346e740 1 --2- 172.21.15.5:0/3138359070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 0x5623e2c84a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.691+0000 7f80e346e740 1 --2- 172.21.15.5:0/3138359070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 0x5623e2c84090 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.691+0000 7f80e346e740 1 -- 172.21.15.5:0/3138359070 >> 172.21.15.5:0/3138359070 conn(0x5623e2c4d310 msgr2=0x5623e2c80c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.691+0000 7f80e346e740 1 -- 172.21.15.5:0/3138359070 shutdown_connections 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.691+0000 7f80e346e740 1 -- 172.21.15.5:0/3138359070 wait complete. 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.691+0000 7f80e346e740 1 -- start start 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80e346e740 1 Processor -- start 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80e346e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c845d0 0x5623e2c8ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80e346e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c85b40 0x5623e2c8bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80e346e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c8f370 0x5623e2c8c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80e346e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5623e2c92070 con 0x5623e2c845d0 2024-09-19T06:16:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80e346e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5623e2ba5060 con 0x5623e2c85b40 2024-09-19T06:16:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80e346e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5623e2c921f0 con 0x5623e2c8f370 2024-09-19T06:16:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80e346e740 10 client.0 _ll_register_callbacks cb 0x5623e28f88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80dad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c845d0 0x5623e2c8ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80dad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c845d0 0x5623e2c8ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58994/0 (socket says 172.21.15.5:58994) 2024-09-19T06:16:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80dad9d640 1 -- 172.21.15.5:0/2065487544 learned_addr learned my addr 172.21.15.5:0/2065487544 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80db59e640 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c8f370 0x5623e2c8c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80da59c640 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c85b40 0x5623e2c8bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80dad9d640 1 -- 172.21.15.5:0/2065487544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c8f370 msgr2=0x5623e2c8c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80dad9d640 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c8f370 0x5623e2c8c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80dad9d640 1 -- 172.21.15.5:0/2065487544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c85b40 msgr2=0x5623e2c8bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80dad9d640 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c85b40 0x5623e2c8bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.692+0000 7f80dad9d640 1 -- 172.21.15.5:0/2065487544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e295b3c0 con 0x5623e2c845d0 2024-09-19T06:16:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80dad9d640 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c845d0 0x5623e2c8ed30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009610 tx=0x7f80cc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd0316a937a754d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80beff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80beff5640 1 -- 172.21.15.5:0/2065487544 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc020050 con 0x5623e2c845d0 2024-09-19T06:16:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80beff5640 1 -- 172.21.15.5:0/2065487544 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f80cc0a4300 con 0x5623e2c845d0 2024-09-19T06:16:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80e346e740 1 -- 172.21.15.5:0/2065487544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80d401b440 con 0x5623e2c845d0 2024-09-19T06:16:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80beff5640 1 -- 172.21.15.5:0/2065487544 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f80cc005180 con 0x5623e2c845d0 2024-09-19T06:16:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80e346e740 10 client.18789 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80e346e740 1 -- 172.21.15.5:0/2065487544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5623e2c9dad0 con 0x5623e2c845d0 2024-09-19T06:16:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80beff5640 1 -- 172.21.15.5:0/2065487544 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80d401b440 con 0x5623e2c845d0 2024-09-19T06:16:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80beff5640 1 -- 172.21.15.5:0/2065487544 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f80cc0ba2e0 con 0x5623e2c845d0 2024-09-19T06:16:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80e346e740 10 client.18789 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80e346e740 10 client.18789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80e346e740 20 client.18789 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80e346e740 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5623e2c9eee0 0x5623e2ca1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80e346e740 4 client.18789 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.693+0000 7f80e346e740 1 -- 172.21.15.5:0/2065487544 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623e2bf7860 con 0x5623e2c9eee0 2024-09-19T06:16:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.694+0000 7f80da59c640 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5623e2c9eee0 0x5623e2ca1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.694+0000 7f80da59c640 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5623e2c9eee0 0x5623e2ca1310 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857cc69d6fb5d34 server_cookie=900f7ece126070dc in_seq=0 out_seq=0 2024-09-19T06:16:02.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.694+0000 7f80beff5640 10 client.18789 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80beff5640 1 -- 172.21.15.5:0/2065487544 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623e2bf7860 con 0x5623e2c9eee0 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80beff5640 10 client.18789 handle_command_reply: tid=0 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80e346e740 1 client.18789 shutdown 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80e346e740 1 -- 172.21.15.5:0/2065487544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c845d0 msgr2=0x5623e2c8ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80e346e740 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c845d0 0x5623e2c8ed30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009610 tx=0x7f80cc005740 comp rx=0 tx=0).stop 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80e346e740 1 -- 172.21.15.5:0/2065487544 shutdown_connections 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80e346e740 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5623e2c9eee0 0x5623e2ca1310 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80e346e740 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c8f370 0x5623e2c8c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80e346e740 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c85b40 0x5623e2c8bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80e346e740 1 --2- 172.21.15.5:0/2065487544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c845d0 0x5623e2c8ed30 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80e346e740 1 -- 172.21.15.5:0/2065487544 >> 172.21.15.5:0/2065487544 conn(0x5623e2c4d310 msgr2=0x5623e2c86960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80beff5640 0 client.18789 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80e346e740 1 -- 172.21.15.5:0/2065487544 shutdown_connections 2024-09-19T06:16:02.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80e346e740 1 -- 172.21.15.5:0/2065487544 wait complete. 2024-09-19T06:16:02.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.695+0000 7f80e346e740 20 client.18789 trim_cache size 0 max 16384 2024-09-19T06:16:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.708+0000 7f80e346e740 1 Processor -- start 2024-09-19T06:16:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.708+0000 7f80e346e740 1 -- start start 2024-09-19T06:16:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.708+0000 7f80e346e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 0x5623e2c7fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.708+0000 7f80e346e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 0x5623e2c87120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80e346e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c8f370 0x5623e2c87660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80e346e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1afa60 con 0x5623e2c85b40 2024-09-19T06:16:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80e346e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1b1660 con 0x5623e2c8f370 2024-09-19T06:16:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80e346e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1b0130 con 0x5623e2c845d0 2024-09-19T06:16:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 0x5623e2c87120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 0x5623e2c87120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58996/0 (socket says 172.21.15.5:58996) 2024-09-19T06:16:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80da59c640 1 -- 172.21.15.5:0/1803602880 learned_addr learned my addr 172.21.15.5:0/1803602880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80dad9d640 1 --2- 172.21.15.5:0/1803602880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 0x5623e2c7fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80db59e640 1 --2- 172.21.15.5:0/1803602880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c8f370 0x5623e2c87660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80da59c640 1 -- 172.21.15.5:0/1803602880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 msgr2=0x5623e2c7fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80da59c640 1 --2- 172.21.15.5:0/1803602880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 0x5623e2c7fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80da59c640 1 -- 172.21.15.5:0/1803602880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c8f370 msgr2=0x5623e2c87660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80da59c640 1 --2- 172.21.15.5:0/1803602880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c8f370 0x5623e2c87660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80da59c640 1 -- 172.21.15.5:0/1803602880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e2c81960 con 0x5623e2c85b40 2024-09-19T06:16:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80da59c640 1 --2- 172.21.15.5:0/1803602880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 0x5623e2c87120 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f80d0031440 tx=0x7f80d0005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf99de63bf507ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80c0ff9640 1 -- 172.21.15.5:0/1803602880 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0048070 con 0x5623e2c85b40 2024-09-19T06:16:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80c0ff9640 1 -- 172.21.15.5:0/1803602880 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80d0004380 con 0x5623e2c85b40 2024-09-19T06:16:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.709+0000 7f80c0ff9640 1 -- 172.21.15.5:0/1803602880 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0035030 con 0x5623e2c85b40 2024-09-19T06:16:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.710+0000 7f80e346e740 1 -- 172.21.15.5:0/1803602880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 msgr2=0x5623e2c87120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.710+0000 7f80e346e740 1 --2- 172.21.15.5:0/1803602880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 0x5623e2c87120 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f80d0031440 tx=0x7f80d0005de0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.710+0000 7f80e346e740 1 -- 172.21.15.5:0/1803602880 shutdown_connections 2024-09-19T06:16:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.710+0000 7f80e346e740 1 --2- 172.21.15.5:0/1803602880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c8f370 0x5623e2c87660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.710+0000 7f80e346e740 1 --2- 172.21.15.5:0/1803602880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c85b40 0x5623e2c87120 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.710+0000 7f80e346e740 1 --2- 172.21.15.5:0/1803602880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 0x5623e2c7fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.710+0000 7f80e346e740 1 -- 172.21.15.5:0/1803602880 >> 172.21.15.5:0/1803602880 conn(0x5623e2c4d310 msgr2=0x5623e2c7f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.710+0000 7f80e346e740 1 -- 172.21.15.5:0/1803602880 shutdown_connections 2024-09-19T06:16:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.710+0000 7f80e346e740 1 -- 172.21.15.5:0/1803602880 wait complete. 2024-09-19T06:16:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.710+0000 7f80e346e740 1 -- start start 2024-09-19T06:16:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80e346e740 1 Processor -- start 2024-09-19T06:16:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80e346e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 0x5623e2c97d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80e346e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c85b40 0x5623e2c930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80e346e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c8f370 0x5623e2c93850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80e346e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1b0f90 con 0x5623e2c8f370 2024-09-19T06:16:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80e346e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1b0860 con 0x5623e2c85b40 2024-09-19T06:16:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80e346e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1ada60 con 0x5623e2c845d0 2024-09-19T06:16:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80e346e740 10 client.0 _ll_register_callbacks cb 0x7f80cc00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80db59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c8f370 0x5623e2c93850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80db59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c8f370 0x5623e2c93850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59010/0 (socket says 172.21.15.5:59010) 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80db59e640 1 -- 172.21.15.5:0/1547601093 learned_addr learned my addr 172.21.15.5:0/1547601093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80dad9d640 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 0x5623e2c97d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80da59c640 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c85b40 0x5623e2c930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80db59e640 1 -- 172.21.15.5:0/1547601093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 msgr2=0x5623e2c97d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80db59e640 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 0x5623e2c97d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80db59e640 1 -- 172.21.15.5:0/1547601093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c85b40 msgr2=0x5623e2c930c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80db59e640 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c85b40 0x5623e2c930c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.711+0000 7f80db59e640 1 -- 172.21.15.5:0/1547601093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e2d08c00 con 0x5623e2c8f370 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80db59e640 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c8f370 0x5623e2c93850 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f80d4005f00 tx=0x7f80d400f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6505993711071ec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80beff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80beff5640 1 -- 172.21.15.5:0/1547601093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4038060 con 0x5623e2c8f370 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80beff5640 1 -- 172.21.15.5:0/1547601093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f80d401c890 con 0x5623e2c8f370 2024-09-19T06:16:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80e346e740 1 -- 172.21.15.5:0/1547601093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80d0004380 con 0x5623e2c8f370 2024-09-19T06:16:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80beff5640 1 -- 172.21.15.5:0/1547601093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f80d4012650 con 0x5623e2c8f370 2024-09-19T06:16:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80e346e740 10 client.18795 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80e346e740 1 -- 172.21.15.5:0/1547601093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5623e2d09dc0 con 0x5623e2c8f370 2024-09-19T06:16:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80beff5640 1 -- 172.21.15.5:0/1547601093 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80d0004380 con 0x5623e2c8f370 2024-09-19T06:16:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80beff5640 1 -- 172.21.15.5:0/1547601093 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f80d4010b60 con 0x5623e2c8f370 2024-09-19T06:16:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80e346e740 10 client.18795 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80e346e740 10 client.18795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80e346e740 20 client.18795 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80e346e740 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5623e2d0a3d0 0x5623e2d0c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80e346e740 4 client.18795 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:16:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.712+0000 7f80e346e740 1 -- 172.21.15.5:0/1547601093 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623e2bf1d70 con 0x5623e2d0a3d0 2024-09-19T06:16:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80dad9d640 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5623e2d0a3d0 0x5623e2d0c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80dad9d640 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5623e2d0a3d0 0x5623e2d0c7f0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f54ba1611b7516c9 server_cookie=6af41960d38d6d58 in_seq=0 out_seq=0 2024-09-19T06:16:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80beff5640 10 client.18795 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80beff5640 1 -- 172.21.15.5:0/1547601093 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623e2bf1d70 con 0x5623e2d0a3d0 2024-09-19T06:16:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80beff5640 10 client.18795 handle_command_reply: tid=0 2024-09-19T06:16:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80e346e740 1 client.18795 shutdown 2024-09-19T06:16:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80e346e740 1 -- 172.21.15.5:0/1547601093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c8f370 msgr2=0x5623e2c93850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80e346e740 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c8f370 0x5623e2c93850 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f80d4005f00 tx=0x7f80d400f360 comp rx=0 tx=0).stop 2024-09-19T06:16:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80e346e740 1 -- 172.21.15.5:0/1547601093 shutdown_connections 2024-09-19T06:16:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80e346e740 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5623e2d0a3d0 0x5623e2d0c7f0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80e346e740 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e2c8f370 0x5623e2c93850 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80e346e740 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623e2c85b40 0x5623e2c930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80beff5640 0 client.18795 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80e346e740 1 --2- 172.21.15.5:0/1547601093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e2c845d0 0x5623e2c97d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80e346e740 1 -- 172.21.15.5:0/1547601093 >> 172.21.15.5:0/1547601093 conn(0x5623e2c4d310 msgr2=0x5623e2c7fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80e346e740 1 -- 172.21.15.5:0/1547601093 shutdown_connections 2024-09-19T06:16:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.713+0000 7f80e346e740 1 -- 172.21.15.5:0/1547601093 wait complete. 2024-09-19T06:16:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e346e740 20 client.18795 trim_cache size 0 max 16384 2024-09-19T06:16:02.721 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:16:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e1095640 1 -- 172.21.15.5:0/1737528337 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f80c40a33b0 msgr2=0x7f80c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e1095640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f80c40a33b0 0x7f80c40a5870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f80d4020000 tx=0x7f80d401d000 comp rx=0 tx=0).stop 2024-09-19T06:16:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e1095640 1 -- 172.21.15.5:0/1737528337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80dc102d40 msgr2=0x7f80dc1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e1095640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80dc102d40 0x7f80dc1009a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00d770 tx=0x7f80cc00dc30 comp rx=0 tx=0).stop 2024-09-19T06:16:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e1095640 1 -- 172.21.15.5:0/1737528337 shutdown_connections 2024-09-19T06:16:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e1095640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc10c170 0x7f80dc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e1095640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f80c40a33b0 0x7f80c40a5870 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e1095640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80dc103660 0x7f80dc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e1095640 1 --2- 172.21.15.5:0/1737528337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80dc102d40 0x7f80dc1009a0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e1095640 1 -- 172.21.15.5:0/1737528337 >> 172.21.15.5:0/1737528337 conn(0x7f80dc0fc7a0 msgr2=0x7f80dc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e1095640 1 -- 172.21.15.5:0/1737528337 shutdown_connections 2024-09-19T06:16:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:02.714+0000 7f80e1095640 1 -- 172.21.15.5:0/1737528337 wait complete. 2024-09-19T06:16:02.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287567 2.14 MB/sec execute 568 sec latency 2089.411 ms 2024-09-19T06:16:03.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287567 2.14 MB/sec execute 569 sec latency 3089.530 ms 2024-09-19T06:16:04.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:03 smithi005 ceph-mon[30955]: pgmap v397: 97 pgs: 97 active+clean; 161 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-09-19T06:16:04.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:03 smithi049 ceph-mon[34640]: pgmap v397: 97 pgs: 97 active+clean; 161 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-09-19T06:16:04.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:03 smithi157 ceph-mon[35346]: pgmap v397: 97 pgs: 97 active+clean; 161 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-09-19T06:16:04.505 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:16:04.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:16:04.776 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:04.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287567 2.13 MB/sec execute 570 sec latency 4089.631 ms 2024-09-19T06:16:05.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.347+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3128854607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8a0108300 msgr2=0x7fd8a010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:05.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.347+0000 7fd8a58c2640 1 --2- 172.21.15.5:0/3128854607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8a0108300 0x7fd8a010ec70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd898009cd0 tx=0x7fd89802a8d0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.347+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3128854607 shutdown_connections 2024-09-19T06:16:05.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.347+0000 7fd8a58c2640 1 --2- 172.21.15.5:0/3128854607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8a0108300 0x7fd8a010ec70 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.347+0000 7fd8a58c2640 1 --2- 172.21.15.5:0/3128854607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0103780 0x7fd8a0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.347+0000 7fd8a58c2640 1 --2- 172.21.15.5:0/3128854607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a0102dd0 0x7fd8a01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.347+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3128854607 >> 172.21.15.5:0/3128854607 conn(0x7fd8a00fc7d0 msgr2=0x7fd8a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.347+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3128854607 shutdown_connections 2024-09-19T06:16:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.347+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3128854607 wait complete. 2024-09-19T06:16:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.347+0000 7fd8a58c2640 1 Processor -- start 2024-09-19T06:16:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd8a58c2640 1 -- start start 2024-09-19T06:16:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd8a58c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a0102dd0 0x7fd8a0100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd8a58c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8a0103780 0x7fd8a0100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd8a58c2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0108300 0x7fd8a01014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd8a58c2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0111f80 con 0x7fd8a0103780 2024-09-19T06:16:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd8a58c2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0111e00 con 0x7fd8a0108300 2024-09-19T06:16:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd8a58c2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0112100 con 0x7fd8a0102dd0 2024-09-19T06:16:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd89e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8a0103780 0x7fd8a0100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd89e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8a0103780 0x7fd8a0100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52240/0 (socket says 172.21.15.5:52240) 2024-09-19T06:16:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd89e7fc640 1 -- 172.21.15.5:0/3962482188 learned_addr learned my addr 172.21.15.5:0/3962482188 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd89effd640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a0102dd0 0x7fd8a0100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd89e7fc640 1 -- 172.21.15.5:0/3962482188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a0102dd0 msgr2=0x7fd8a0100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd89e7fc640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a0102dd0 0x7fd8a0100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd89e7fc640 1 -- 172.21.15.5:0/3962482188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0108300 msgr2=0x7fd8a01014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd89e7fc640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0108300 0x7fd8a01014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd89e7fc640 1 -- 172.21.15.5:0/3962482188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a01a68a0 con 0x7fd8a0103780 2024-09-19T06:16:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd89e7fc640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8a0103780 0x7fd8a0100f70 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd89400c970 tx=0x7fd89400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.348+0000 7fd8a48c0640 1 -- 172.21.15.5:0/3962482188 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894007bb0 con 0x7fd8a0103780 2024-09-19T06:16:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.349+0000 7fd8a48c0640 1 -- 172.21.15.5:0/3962482188 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd894007d50 con 0x7fd8a0103780 2024-09-19T06:16:05.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.349+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3962482188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a01a6b90 con 0x7fd8a0103780 2024-09-19T06:16:05.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.349+0000 7fd8a48c0640 1 -- 172.21.15.5:0/3962482188 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89400f450 con 0x7fd8a0103780 2024-09-19T06:16:05.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.349+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3962482188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a01a70d0 con 0x7fd8a0103780 2024-09-19T06:16:05.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.351+0000 7fd8a48c0640 1 -- 172.21.15.5:0/3962482188 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd894004eb0 con 0x7fd8a0103780 2024-09-19T06:16:05.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.351+0000 7fd8a7c9b740 1 Processor -- start 2024-09-19T06:16:05.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.351+0000 7fd8a7c9b740 1 -- start start 2024-09-19T06:16:05.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd8a7c9b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9bb60 0x55b4caa9bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd8a7c9b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 0x55b4caaa6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd8a7c9b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caaa6d40 0x55b4caaa9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd8a7c9b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b4caa01280 con 0x55b4caa9c480 2024-09-19T06:16:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd8a7c9b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b4ca9a8b10 con 0x55b4caaa6d40 2024-09-19T06:16:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd8a7c9b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b4ca9fd240 con 0x55b4caa9bb60 2024-09-19T06:16:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd89effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 0x55b4caaa6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd89effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 0x55b4caaa6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52248/0 (socket says 172.21.15.5:52248) 2024-09-19T06:16:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd89effd640 1 -- 172.21.15.5:0/3408761820 learned_addr learned my addr 172.21.15.5:0/3408761820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd89f7fe640 1 --2- 172.21.15.5:0/3408761820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9bb60 0x55b4caa9bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd89e7fc640 1 --2- 172.21.15.5:0/3408761820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caaa6d40 0x55b4caaa9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd8a48c0640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8880a33b0 0x7fd8880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd89effd640 1 -- 172.21.15.5:0/3408761820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9bb60 msgr2=0x55b4caa9bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd89effd640 1 --2- 172.21.15.5:0/3408761820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9bb60 0x55b4caa9bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd89effd640 1 -- 172.21.15.5:0/3408761820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caaa6d40 msgr2=0x55b4caaa9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd89effd640 1 --2- 172.21.15.5:0/3408761820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caaa6d40 0x55b4caaa9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd89effd640 1 -- 172.21.15.5:0/3408761820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ca9b51f0 con 0x55b4caa9c480 2024-09-19T06:16:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd89f7fe640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8880a33b0 0x7fd8880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd89effd640 1 --2- 172.21.15.5:0/3408761820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 0x55b4caaa6800 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd89000cc60 tx=0x7fd89000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571c17800e26aaf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.352+0000 7fd8a48c0640 1 -- 172.21.15.5:0/3962482188 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fd8940cfb40 con 0x7fd8a0103780 2024-09-19T06:16:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.353+0000 7fd885ffb640 1 -- 172.21.15.5:0/3408761820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890014070 con 0x55b4caa9c480 2024-09-19T06:16:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.353+0000 7fd885ffb640 1 -- 172.21.15.5:0/3408761820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8900047a0 con 0x55b4caa9c480 2024-09-19T06:16:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.353+0000 7fd885ffb640 1 -- 172.21.15.5:0/3408761820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890010670 con 0x55b4caa9c480 2024-09-19T06:16:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.353+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3408761820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 msgr2=0x55b4caaa6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.353+0000 7fd89f7fe640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8880a33b0 0x7fd8880a5870 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd898005e30 tx=0x7fd898005d80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.353+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/3408761820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 0x55b4caaa6800 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd89000cc60 tx=0x7fd89000e720 comp rx=0 tx=0).stop 2024-09-19T06:16:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.353+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3408761820 shutdown_connections 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.353+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/3408761820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caaa6d40 0x55b4caaa9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/3408761820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 0x55b4caaa6800 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/3408761820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9bb60 0x55b4caa9bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3408761820 >> 172.21.15.5:0/3408761820 conn(0x55b4caa980f0 msgr2=0x55b4caa984d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3408761820 shutdown_connections 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3408761820 wait complete. 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 -- start start 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 Processor -- start 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caa9bb60 0x55b4caaa3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9c480 0x55b4caaa40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caaa6d40 0x55b4caaa4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b4caaa9910 con 0x55b4caaa6d40 2024-09-19T06:16:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b4ca9bbf10 con 0x55b4caa9bb60 2024-09-19T06:16:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b4caaa9a90 con 0x55b4caa9c480 2024-09-19T06:16:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd8a7c9b740 10 client.0 _ll_register_callbacks cb 0x55b4ca984a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.354+0000 7fd89f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caaa6d40 0x55b4caaa4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd89f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caaa6d40 0x55b4caaa4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52250/0 (socket says 172.21.15.5:52250) 2024-09-19T06:16:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd89f7fe640 1 -- 172.21.15.5:0/3765818576 learned_addr learned my addr 172.21.15.5:0/3765818576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd89effd640 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caa9bb60 0x55b4caaa3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd89e7fc640 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9c480 0x55b4caaa40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd89f7fe640 1 -- 172.21.15.5:0/3765818576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9c480 msgr2=0x55b4caaa40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd89f7fe640 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9c480 0x55b4caaa40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd89f7fe640 1 -- 172.21.15.5:0/3765818576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caa9bb60 msgr2=0x55b4caaa3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd89f7fe640 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caa9bb60 0x55b4caaa3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd89f7fe640 1 -- 172.21.15.5:0/3765818576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ca7726a0 con 0x55b4caaa6d40 2024-09-19T06:16:05.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd89f7fe640 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caaa6d40 0x55b4caaa4630 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd898031fc0 tx=0x7fd8980095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771fbec02dcb6795 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:05.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd87effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:05.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd87effd640 1 -- 172.21.15.5:0/3765818576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89804c070 con 0x55b4caaa6d40 2024-09-19T06:16:05.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd87effd640 1 -- 172.21.15.5:0/3765818576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fd898030790 con 0x55b4caaa6d40 2024-09-19T06:16:05.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3765818576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8900047a0 con 0x55b4caaa6d40 2024-09-19T06:16:05.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd87effd640 1 -- 172.21.15.5:0/3765818576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd898030990 con 0x55b4caaa6d40 2024-09-19T06:16:05.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd8a7c9b740 10 client.18807 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:05.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.355+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3765818576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b4caab54a0 con 0x55b4caaa6d40 2024-09-19T06:16:05.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.356+0000 7fd87effd640 1 -- 172.21.15.5:0/3765818576 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8900047a0 con 0x55b4caaa6d40 2024-09-19T06:16:05.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.356+0000 7fd87effd640 1 -- 172.21.15.5:0/3765818576 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fd89803bc60 con 0x55b4caaa6d40 2024-09-19T06:16:05.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.356+0000 7fd8a7c9b740 10 client.18807 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:05.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.356+0000 7fd8a7c9b740 10 client.18807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.356+0000 7fd8a7c9b740 20 client.18807 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.356+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b4caab6900 0x55b4caab8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.356+0000 7fd8a7c9b740 4 client.18807 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.356+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3765818576 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4caa0e7f0 con 0x55b4caab6900 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.356+0000 7fd89effd640 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b4caab6900 0x55b4caab8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.357+0000 7fd87effd640 10 client.18807 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.357+0000 7fd89effd640 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b4caab6900 0x55b4caab8da0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1215e0e8ec6a928e server_cookie=6f08e18723fb0029 in_seq=0 out_seq=0 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.357+0000 7fd87effd640 1 -- 172.21.15.5:0/3765818576 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4caa0e7f0 con 0x55b4caab6900 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd87effd640 10 client.18807 handle_command_reply: tid=0 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd8a7c9b740 1 client.18807 shutdown 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3765818576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caaa6d40 msgr2=0x55b4caaa4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caaa6d40 0x55b4caaa4630 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd898031fc0 tx=0x7fd8980095c0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3765818576 shutdown_connections 2024-09-19T06:16:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b4caab6900 0x55b4caab8da0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caaa6d40 0x55b4caaa4630 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9c480 0x55b4caaa40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/3765818576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caa9bb60 0x55b4caaa3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd87effd640 0 client.18807 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3765818576 >> 172.21.15.5:0/3765818576 conn(0x55b4caa980f0 msgr2=0x55b4caa9ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3765818576 shutdown_connections 2024-09-19T06:16:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/3765818576 wait complete. 2024-09-19T06:16:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.358+0000 7fd8a7c9b740 20 client.18807 trim_cache size 0 max 16384 2024-09-19T06:16:05.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd8a7c9b740 1 Processor -- start 2024-09-19T06:16:05.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd8a7c9b740 1 -- start start 2024-09-19T06:16:05.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd8a7c9b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caa9bb60 0x55b4caaa5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd8a7c9b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 0x55b4caabe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd8a7c9b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caaa6d40 0x55b4caabebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd8a7c9b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01afa90 con 0x55b4caa9c480 2024-09-19T06:16:05.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd8a7c9b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01b1690 con 0x55b4caa9bb60 2024-09-19T06:16:05.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd8a7c9b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01b0160 con 0x55b4caaa6d40 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd89e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 0x55b4caabe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd89e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 0x55b4caabe670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52262/0 (socket says 172.21.15.5:52262) 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd89e7fc640 1 -- 172.21.15.5:0/385914102 learned_addr learned my addr 172.21.15.5:0/385914102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd89f7fe640 1 --2- 172.21.15.5:0/385914102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caaa6d40 0x55b4caabebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd89effd640 1 --2- 172.21.15.5:0/385914102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caa9bb60 0x55b4caaa5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd89e7fc640 1 -- 172.21.15.5:0/385914102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caaa6d40 msgr2=0x55b4caabebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd89e7fc640 1 --2- 172.21.15.5:0/385914102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caaa6d40 0x55b4caabebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd89e7fc640 1 -- 172.21.15.5:0/385914102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caa9bb60 msgr2=0x55b4caaa5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd89e7fc640 1 --2- 172.21.15.5:0/385914102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caa9bb60 0x55b4caaa5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd89e7fc640 1 -- 172.21.15.5:0/385914102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4caa97f20 con 0x55b4caa9c480 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd89f7fe640 1 --2- 172.21.15.5:0/385914102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caaa6d40 0x55b4caabebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.376+0000 7fd89e7fc640 1 --2- 172.21.15.5:0/385914102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 0x55b4caabe670 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd894098200 tx=0x7fd89409b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4237e07f3c092c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.377+0000 7fd8857fa640 1 -- 172.21.15.5:0/385914102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89401d070 con 0x55b4caa9c480 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.377+0000 7fd8857fa640 1 -- 172.21.15.5:0/385914102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8940a4a10 con 0x55b4caa9c480 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.377+0000 7fd8857fa640 1 -- 172.21.15.5:0/385914102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8940a4580 con 0x55b4caa9c480 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.377+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/385914102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 msgr2=0x55b4caabe670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.377+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/385914102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 0x55b4caabe670 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd894098200 tx=0x7fd89409b7c0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.377+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/385914102 shutdown_connections 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.377+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/385914102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caaa6d40 0x55b4caabebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.377+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/385914102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9c480 0x55b4caabe670 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.377+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/385914102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caa9bb60 0x55b4caaa5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.377+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/385914102 >> 172.21.15.5:0/385914102 conn(0x55b4caa980f0 msgr2=0x55b4caaa5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.377+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/385914102 shutdown_connections 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.377+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/385914102 wait complete. 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd8a7c9b740 1 -- start start 2024-09-19T06:16:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd8a7c9b740 1 Processor -- start 2024-09-19T06:16:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd8a7c9b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9bb60 0x55b4caaaeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd8a7c9b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9c480 0x55b4caaaf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd8a7c9b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caaa6d40 0x55b4caaabcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd8a7c9b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01b0fc0 con 0x55b4caa9bb60 2024-09-19T06:16:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd8a7c9b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01b0890 con 0x55b4caaa6d40 2024-09-19T06:16:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd8a7c9b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01ada90 con 0x55b4caa9c480 2024-09-19T06:16:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd8a7c9b740 10 client.0 _ll_register_callbacks cb 0x7fd898009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd89effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9bb60 0x55b4caaaeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd89effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9bb60 0x55b4caaaeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52272/0 (socket says 172.21.15.5:52272) 2024-09-19T06:16:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd89effd640 1 -- 172.21.15.5:0/723920831 learned_addr learned my addr 172.21.15.5:0/723920831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.378+0000 7fd89e7fc640 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9c480 0x55b4caaaf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd89f7fe640 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caaa6d40 0x55b4caaabcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd89effd640 1 -- 172.21.15.5:0/723920831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9c480 msgr2=0x55b4caaaf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:05.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd89effd640 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9c480 0x55b4caaaf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd89effd640 1 -- 172.21.15.5:0/723920831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caaa6d40 msgr2=0x55b4caaabcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:05.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd89effd640 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caaa6d40 0x55b4caaabcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd89effd640 1 -- 172.21.15.5:0/723920831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4caa98d80 con 0x55b4caa9bb60 2024-09-19T06:16:05.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd89f7fe640 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caaa6d40 0x55b4caaabcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:05.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd89effd640 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9bb60 0x55b4caaaeec0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fd89000cb80 tx=0x7fd8900058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60d4e882f0bf903 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:05.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd87effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:05.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd87effd640 1 -- 172.21.15.5:0/723920831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890029020 con 0x55b4caa9bb60 2024-09-19T06:16:05.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd87effd640 1 -- 172.21.15.5:0/723920831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fd890004460 con 0x55b4caa9bb60 2024-09-19T06:16:05.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/723920831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8940a4a10 con 0x55b4caa9bb60 2024-09-19T06:16:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd87effd640 1 -- 172.21.15.5:0/723920831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd890014040 con 0x55b4caa9bb60 2024-09-19T06:16:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd8a7c9b740 10 client.18813 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/723920831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b4cab21e00 con 0x55b4caa9bb60 2024-09-19T06:16:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd87effd640 1 -- 172.21.15.5:0/723920831 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8940a4a10 con 0x55b4caa9bb60 2024-09-19T06:16:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.379+0000 7fd87effd640 1 -- 172.21.15.5:0/723920831 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fd8900244d0 con 0x55b4caa9bb60 2024-09-19T06:16:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.380+0000 7fd8a7c9b740 10 client.18813 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.380+0000 7fd8a7c9b740 10 client.18813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:05.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.380+0000 7fd8a7c9b740 20 client.18813 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:05.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.380+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b4cab222b0 0x55b4cab246d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:05.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.380+0000 7fd8a7c9b740 4 client.18813 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:16:05.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.380+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/723920831 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b4caa08d00 con 0x55b4cab222b0 2024-09-19T06:16:05.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.380+0000 7fd89e7fc640 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b4cab222b0 0x55b4cab246d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:05.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.380+0000 7fd89e7fc640 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b4cab222b0 0x55b4cab246d0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a73e706b4ba1c48 server_cookie=e8352ab3543bb110 in_seq=0 out_seq=0 2024-09-19T06:16:05.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.380+0000 7fd87effd640 10 client.18813 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:05.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.381+0000 7fd87effd640 1 -- 172.21.15.5:0/723920831 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b4caa08d00 con 0x55b4cab222b0 2024-09-19T06:16:05.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.381+0000 7fd87effd640 10 client.18813 handle_command_reply: tid=0 2024-09-19T06:16:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.381+0000 7fd8a7c9b740 1 client.18813 shutdown 2024-09-19T06:16:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/723920831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9bb60 msgr2=0x55b4caaaeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9bb60 0x55b4caaaeec0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fd89000cb80 tx=0x7fd8900058d0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/723920831 shutdown_connections 2024-09-19T06:16:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b4cab222b0 0x55b4cab246d0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4caaa6d40 0x55b4caaabcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4caa9c480 0x55b4caaaf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a7c9b740 1 --2- 172.21.15.5:0/723920831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4caa9bb60 0x55b4caaaeec0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/723920831 >> 172.21.15.5:0/723920831 conn(0x55b4caa980f0 msgr2=0x55b4caa9ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd87effd640 0 client.18813 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/723920831 shutdown_connections 2024-09-19T06:16:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a7c9b740 1 -- 172.21.15.5:0/723920831 wait complete. 2024-09-19T06:16:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a7c9b740 20 client.18813 trim_cache size 0 max 16384 2024-09-19T06:16:05.396 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:16:05.396 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:16:05.396 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "75f103c4-ce5a-49f6-9944-ab3d16413862", 2024-09-19T06:16:05.396 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:16:05.396 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:16:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3962482188 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8880a33b0 msgr2=0x7fd8880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a58c2640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8880a33b0 0x7fd8880a5870 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd898005e30 tx=0x7fd898005d80 comp rx=0 tx=0).stop 2024-09-19T06:16:05.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3962482188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8a0103780 msgr2=0x7fd8a0100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:05.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a58c2640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8a0103780 0x7fd8a0100f70 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd89400c970 tx=0x7fd89400ce30 comp rx=0 tx=0).stop 2024-09-19T06:16:05.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3962482188 shutdown_connections 2024-09-19T06:16:05.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a58c2640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0108300 0x7fd8a01014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a58c2640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8a0103780 0x7fd8a0100f70 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a58c2640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd8880a33b0 0x7fd8880a5870 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a58c2640 1 --2- 172.21.15.5:0/3962482188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a0102dd0 0x7fd8a0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:05.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.382+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3962482188 >> 172.21.15.5:0/3962482188 conn(0x7fd8a00fc7d0 msgr2=0x7fd8a0105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:05.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.383+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3962482188 shutdown_connections 2024-09-19T06:16:05.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:05.383+0000 7fd8a58c2640 1 -- 172.21.15.5:0/3962482188 wait complete. 2024-09-19T06:16:05.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287671 2.13 MB/sec execute 571 sec latency 4652.939 ms 2024-09-19T06:16:05.878 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:16:05.395+0000 7f9e1e5f8640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...14,head] /volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{2=1} v46866 ap=2 f(v1872 m2024-09-19T06:15:53.748351+0000 5=5+0) n(v66 rc2024-09-19T06:15:53.748351+0000 b814326 6=5+1) old_inodes=6 (inest sync->mix g=2) caps={15255=pAsLsXsFs/-@41497} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x559db3497600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v366>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v46866>,<0x10000000008/~dmtmp v44181>,<0x10000000007/client0 v1511>,<0x10000000006/clients v1487>,<0x10000000005/tmp v1411>,<0x10000000003/client.0 v1395>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v1393>,<0x10000000001/sv_0 v1391>,<0x10000000000/qa v1376>,<0x1/volumes v1359>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T06:15:53.736721+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T06:15:53.748351+0000 b814326 6=5+1)","memory_value.dirstat":"f(v1872 m2024-09-19T06:15:53.748351+0000 5=5+0)","memory_value.rstat":"n(v66 rc2024-09-19T06:15:53.748351+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T06:16:05.879 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:16:05.414+0000 7f9e1e5f8640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [...14,head] /volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ auth{2=1} v46862 ap=2 f(v2184 m2024-09-19T06:15:53.182871+0000 7=7+0) n(v68 rc2024-09-19T06:15:53.182871+0000 b1462582 8=7+1) old_inodes=6 (inest sync->mix g=2) caps={15255=pAsLsXsFs/-@41259} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x559db34c4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020f:[<0x10000000009/PWRPNT v274>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x1000000020f:[<0x10000000009/PWRPNT v46862>,<0x10000000008/~dmtmp v44195>,<0x10000000007/client0 v1513>,<0x10000000006/clients v1489>,<0x10000000005/tmp v1413>,<0x10000000003/client.0 v1397>,<0x10000000002/cf663d32-66ce-4414-bd62-494f4d2d56f9 v1395>,<0x10000000001/sv_0 v1393>,<0x10000000000/qa v1378>,<0x1/volumes v1361>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T06:15:53.158077+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T06:15:53.182871+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v2184 m2024-09-19T06:15:53.182871+0000 7=7+0)","memory_value.rstat":"n(v68 rc2024-09-19T06:15:53.182871+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T06:16:06.082 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:16:06.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: pgmap v398: 97 pgs: 97 active+clean; 161 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 962 KiB/s wr, 101 op/s 2024-09-19T06:16:06.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:16:06.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:16:06.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:16:06.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:16:06.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:16:06.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:16:06.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:05 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: pgmap v398: 97 pgs: 97 active+clean; 161 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 962 KiB/s wr, 101 op/s 2024-09-19T06:16:06.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:16:06.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:16:06.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:16:06.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:16:06.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:16:06.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:16:06.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:05 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: pgmap v398: 97 pgs: 97 active+clean; 161 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 962 KiB/s wr, 101 op/s 2024-09-19T06:16:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:16:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:16:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:16:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:16:06.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:16:06.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:16:06.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:05 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:06.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287928 2.13 MB/sec execute 572 sec latency 30.963 ms 2024-09-19T06:16:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.860+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/48300316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102ef0 msgr2=0x7fdcc010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.860+0000 7fdcc7bb0640 1 --2- 172.21.15.5:0/48300316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102ef0 0x7fdcc010da40 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009ca0 tx=0x7fdcb802a950 comp rx=0 tx=0).stop 2024-09-19T06:16:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.860+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/48300316 shutdown_connections 2024-09-19T06:16:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.860+0000 7fdcc7bb0640 1 --2- 172.21.15.5:0/48300316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102ef0 0x7fdcc010da40 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.860+0000 7fdcc7bb0640 1 --2- 172.21.15.5:0/48300316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcc0102570 0x7fdcc01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.860+0000 7fdcc7bb0640 1 --2- 172.21.15.5:0/48300316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcc0100e20 0x7fdcc0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.860+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/48300316 >> 172.21.15.5:0/48300316 conn(0x7fdcc00fc7d0 msgr2=0x7fdcc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.860+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/48300316 shutdown_connections 2024-09-19T06:16:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.860+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/48300316 wait complete. 2024-09-19T06:16:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc7bb0640 1 Processor -- start 2024-09-19T06:16:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc7bb0640 1 -- start start 2024-09-19T06:16:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc7bb0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcc0100e20 0x7fdcc01970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc7bb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102570 0x7fdcc01975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc7bb0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcc0102ef0 0x7fdcc0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc7bb0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0110e30 con 0x7fdcc0102570 2024-09-19T06:16:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc7bb0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0110cb0 con 0x7fdcc0100e20 2024-09-19T06:16:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc7bb0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0110fb0 con 0x7fdcc0102ef0 2024-09-19T06:16:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc514b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102570 0x7fdcc01975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc594c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcc0100e20 0x7fdcc01970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc514b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102570 0x7fdcc01975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52292/0 (socket says 172.21.15.5:52292) 2024-09-19T06:16:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc514b640 1 -- 172.21.15.5:0/22404126 learned_addr learned my addr 172.21.15.5:0/22404126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.861+0000 7fdcc614d640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcc0102ef0 0x7fdcc0197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.862+0000 7fdcc514b640 1 -- 172.21.15.5:0/22404126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcc0102ef0 msgr2=0x7fdcc0197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.862+0000 7fdcc514b640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcc0102ef0 0x7fdcc0197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.862+0000 7fdcc514b640 1 -- 172.21.15.5:0/22404126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcc0100e20 msgr2=0x7fdcc01970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.862+0000 7fdcc514b640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcc0100e20 0x7fdcc01970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.862+0000 7fdcc514b640 1 -- 172.21.15.5:0/22404126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc01a6920 con 0x7fdcc0102570 2024-09-19T06:16:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.862+0000 7fdcc594c640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcc0100e20 0x7fdcc01970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.862+0000 7fdcc514b640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102570 0x7fdcc01975f0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000e9f0 tx=0x7fdcb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.862+0000 7fdcb6ffd640 1 -- 172.21.15.5:0/22404126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb000cd80 con 0x7fdcc0102570 2024-09-19T06:16:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.862+0000 7fdcb6ffd640 1 -- 172.21.15.5:0/22404126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcb00108e0 con 0x7fdcc0102570 2024-09-19T06:16:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.862+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/22404126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc01a6c10 con 0x7fdcc0102570 2024-09-19T06:16:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.862+0000 7fdcb6ffd640 1 -- 172.21.15.5:0/22404126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0018920 con 0x7fdcc0102570 2024-09-19T06:16:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.862+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/22404126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc01a7150 con 0x7fdcc0102570 2024-09-19T06:16:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.865+0000 7fdcb6ffd640 1 -- 172.21.15.5:0/22404126 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fdcb0010a90 con 0x7fdcc0102570 2024-09-19T06:16:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.865+0000 7fdcc9f89740 1 Processor -- start 2024-09-19T06:16:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.865+0000 7fdcc9f89740 1 -- start start 2024-09-19T06:16:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.865+0000 7fdcc9f89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe07880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.865+0000 7fdcc9f89740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe07dc0 0x55566fe12140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.865+0000 7fdcc9f89740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe14a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.865+0000 7fdcc9f89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55566fd6d2e0 con 0x55566fe074a0 2024-09-19T06:16:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.865+0000 7fdcc9f89740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55566fd14ce0 con 0x55566fe12680 2024-09-19T06:16:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.865+0000 7fdcc9f89740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55566fd692a0 con 0x55566fe07dc0 2024-09-19T06:16:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.865+0000 7fdcc614d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe07880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.865+0000 7fdcc594c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe07dc0 0x55566fe12140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcc594c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe07dc0 0x55566fe12140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48314/0 (socket says 172.21.15.5:48314) 2024-09-19T06:16:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcc614d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe07880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52306/0 (socket says 172.21.15.5:52306) 2024-09-19T06:16:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcc594c640 1 -- 172.21.15.5:0/3083965580 learned_addr learned my addr 172.21.15.5:0/3083965580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcb6ffd640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdcac0a33b0 0x7fdcac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcc514b640 1 --2- 172.21.15.5:0/3083965580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe14a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcc594c640 1 -- 172.21.15.5:0/3083965580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 msgr2=0x55566fe14a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcc594c640 1 --2- 172.21.15.5:0/3083965580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe14a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcc594c640 1 -- 172.21.15.5:0/3083965580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 msgr2=0x55566fe07880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcc594c640 1 --2- 172.21.15.5:0/3083965580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe07880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcc594c640 1 -- 172.21.15.5:0/3083965580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55566fcf9440 con 0x55566fe07dc0 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcc614d640 1 --2- 172.21.15.5:0/3083965580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe07880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcb6ffd640 1 -- 172.21.15.5:0/22404126 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fdcb00cffd0 con 0x7fdcc0102570 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcc614d640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdcac0a33b0 0x7fdcac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcc594c640 1 --2- 172.21.15.5:0/3083965580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe07dc0 0x55566fe12140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0115c0 tx=0x7fdcbc011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25d751b89f0fb2f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcabfff640 1 -- 172.21.15.5:0/3083965580 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00f640 con 0x55566fe07dc0 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcabfff640 1 -- 172.21.15.5:0/3083965580 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcbc00fc90 con 0x55566fe07dc0 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.866+0000 7fdcabfff640 1 -- 172.21.15.5:0/3083965580 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc019d30 con 0x55566fe07dc0 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.867+0000 7fdcc9f89740 1 -- 172.21.15.5:0/3083965580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe07dc0 msgr2=0x55566fe12140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.867+0000 7fdcc614d640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdcac0a33b0 0x7fdcac0a5870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8005e30 tx=0x7fdcb803c040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.867+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/3083965580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe07dc0 0x55566fe12140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0115c0 tx=0x7fdcbc011a80 comp rx=0 tx=0).stop 2024-09-19T06:16:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.867+0000 7fdcc9f89740 1 -- 172.21.15.5:0/3083965580 shutdown_connections 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.867+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/3083965580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe14a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.867+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/3083965580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe07dc0 0x55566fe12140 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.867+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/3083965580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe07880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.867+0000 7fdcc9f89740 1 -- 172.21.15.5:0/3083965580 >> 172.21.15.5:0/3083965580 conn(0x55566fe03a30 msgr2=0x55566fe03e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.867+0000 7fdcc9f89740 1 -- 172.21.15.5:0/3083965580 shutdown_connections 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.867+0000 7fdcc9f89740 1 -- 172.21.15.5:0/3083965580 wait complete. 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc9f89740 1 -- start start 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc9f89740 1 Processor -- start 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc9f89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe11f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc9f89740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe07dc0 0x55566fe0f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc9f89740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe12680 0x55566fe0f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc9f89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55566fe15250 con 0x55566fe074a0 2024-09-19T06:16:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc9f89740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55566fd28050 con 0x55566fe07dc0 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc9f89740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55566fe153d0 con 0x55566fe12680 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc9f89740 10 client.0 _ll_register_callbacks cb 0x55566fd68160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc594c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe11f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc594c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe11f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52318/0 (socket says 172.21.15.5:52318) 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc594c640 1 -- 172.21.15.5:0/816500533 learned_addr learned my addr 172.21.15.5:0/816500533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc514b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe07dc0 0x55566fe0f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.868+0000 7fdcc614d640 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe12680 0x55566fe0f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdcc594c640 1 -- 172.21.15.5:0/816500533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe12680 msgr2=0x55566fe0f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdcc594c640 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe12680 0x55566fe0f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdcc594c640 1 -- 172.21.15.5:0/816500533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe07dc0 msgr2=0x55566fe0f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdcc594c640 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe07dc0 0x55566fe0f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdcc594c640 1 -- 172.21.15.5:0/816500533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55566fade3c0 con 0x55566fe074a0 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdcc514b640 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe07dc0 0x55566fe0f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdcc594c640 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe11f60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc019ce0 tx=0x7fdcbc019d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a871f08cff0a69bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdca97fa640 1 -- 172.21.15.5:0/816500533 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc026020 con 0x55566fe074a0 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdca97fa640 1 -- 172.21.15.5:0/816500533 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fdcbc00fe40 con 0x55566fe074a0 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdcc9f89740 1 -- 172.21.15.5:0/816500533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcbc00fc90 con 0x55566fe074a0 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdca97fa640 1 -- 172.21.15.5:0/816500533 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc031050 con 0x55566fe074a0 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdcc9f89740 10 client.18825 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.869+0000 7fdcc9f89740 1 -- 172.21.15.5:0/816500533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55566fe20c20 con 0x55566fe074a0 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.870+0000 7fdca97fa640 1 -- 172.21.15.5:0/816500533 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcbc00fc90 con 0x55566fe074a0 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.870+0000 7fdca97fa640 1 -- 172.21.15.5:0/816500533 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fdcbc003710 con 0x55566fe074a0 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.870+0000 7fdcc9f89740 10 client.18825 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.870+0000 7fdcc9f89740 10 client.18825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.870+0000 7fdcc9f89740 20 client.18825 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.870+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55566fe21fd0 0x55566fe243c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.870+0000 7fdcc9f89740 4 client.18825 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.870+0000 7fdcc9f89740 1 -- 172.21.15.5:0/816500533 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55566fd7a850 con 0x55566fe21fd0 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.870+0000 7fdcc514b640 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55566fe21fd0 0x55566fe243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.870+0000 7fdca97fa640 10 client.18825 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.870+0000 7fdcc514b640 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55566fe21fd0 0x55566fe243c0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c72638419fd92433 server_cookie=371dc202a8e259b4 in_seq=0 out_seq=0 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.871+0000 7fdca97fa640 1 -- 172.21.15.5:0/816500533 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55566fd7a850 con 0x55566fe21fd0 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.871+0000 7fdca97fa640 10 client.18825 handle_command_reply: tid=0 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.871+0000 7fdcc9f89740 1 client.18825 shutdown 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.871+0000 7fdcc9f89740 1 -- 172.21.15.5:0/816500533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 msgr2=0x55566fe11f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.871+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe11f60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc019ce0 tx=0x7fdcbc019d10 comp rx=0 tx=0).stop 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.871+0000 7fdcc9f89740 1 -- 172.21.15.5:0/816500533 shutdown_connections 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.871+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55566fe21fd0 0x55566fe243c0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.871+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe12680 0x55566fe0f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.871+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe07dc0 0x55566fe0f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.871+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/816500533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe11f60 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.871+0000 7fdcc9f89740 1 -- 172.21.15.5:0/816500533 >> 172.21.15.5:0/816500533 conn(0x55566fe03a30 msgr2=0x55566fe09760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.871+0000 7fdca97fa640 0 client.18825 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.872+0000 7fdcc9f89740 1 -- 172.21.15.5:0/816500533 shutdown_connections 2024-09-19T06:16:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.872+0000 7fdcc9f89740 1 -- 172.21.15.5:0/816500533 wait complete. 2024-09-19T06:16:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.872+0000 7fdcc9f89740 20 client.18825 trim_cache size 0 max 16384 2024-09-19T06:16:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc9f89740 1 Processor -- start 2024-09-19T06:16:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc9f89740 1 -- start start 2024-09-19T06:16:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc9f89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe1e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc9f89740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe07dc0 0x55566fe0a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc9f89740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe0e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc9f89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01afb00 con 0x55566fe074a0 2024-09-19T06:16:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc9f89740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01b1700 con 0x55566fe12680 2024-09-19T06:16:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc9f89740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01b01d0 con 0x55566fe07dc0 2024-09-19T06:16:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc594c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe1e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc594c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe1e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52324/0 (socket says 172.21.15.5:52324) 2024-09-19T06:16:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc594c640 1 -- 172.21.15.5:0/3814466612 learned_addr learned my addr 172.21.15.5:0/3814466612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc514b640 1 --2- 172.21.15.5:0/3814466612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe07dc0 0x55566fe0a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.893+0000 7fdcc614d640 1 --2- 172.21.15.5:0/3814466612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe0e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.894+0000 7fdcc594c640 1 -- 172.21.15.5:0/3814466612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe07dc0 msgr2=0x55566fe0a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.894+0000 7fdcc594c640 1 --2- 172.21.15.5:0/3814466612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe07dc0 0x55566fe0a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.894+0000 7fdcc594c640 1 -- 172.21.15.5:0/3814466612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 msgr2=0x55566fe0e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.894+0000 7fdcc594c640 1 --2- 172.21.15.5:0/3814466612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe0e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.894+0000 7fdcc594c640 1 -- 172.21.15.5:0/3814466612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55566fe03860 con 0x55566fe074a0 2024-09-19T06:16:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.894+0000 7fdcc594c640 1 --2- 172.21.15.5:0/3814466612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe1e510 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc019740 tx=0x7fdcbc005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84275d24580cab81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.894+0000 7fdcab7fe640 1 -- 172.21.15.5:0/3814466612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc026070 con 0x55566fe074a0 2024-09-19T06:16:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.894+0000 7fdcab7fe640 1 -- 172.21.15.5:0/3814466612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcbc003cf0 con 0x55566fe074a0 2024-09-19T06:16:06.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.894+0000 7fdcab7fe640 1 -- 172.21.15.5:0/3814466612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0055b0 con 0x55566fe074a0 2024-09-19T06:16:06.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.895+0000 7fdcc9f89740 1 -- 172.21.15.5:0/3814466612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 msgr2=0x55566fe1e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.895+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/3814466612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe1e510 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc019740 tx=0x7fdcbc005970 comp rx=0 tx=0).stop 2024-09-19T06:16:06.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.895+0000 7fdcc9f89740 1 -- 172.21.15.5:0/3814466612 shutdown_connections 2024-09-19T06:16:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.895+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/3814466612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe0e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.895+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/3814466612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe07dc0 0x55566fe0a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.895+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/3814466612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe074a0 0x55566fe1e510 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.895+0000 7fdcc9f89740 1 -- 172.21.15.5:0/3814466612 >> 172.21.15.5:0/3814466612 conn(0x55566fe03a30 msgr2=0x55566fe0dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.895+0000 7fdcc9f89740 1 -- 172.21.15.5:0/3814466612 shutdown_connections 2024-09-19T06:16:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.895+0000 7fdcc9f89740 1 -- 172.21.15.5:0/3814466612 wait complete. 2024-09-19T06:16:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.895+0000 7fdcc9f89740 1 -- start start 2024-09-19T06:16:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc9f89740 1 Processor -- start 2024-09-19T06:16:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc9f89740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe074a0 0x55566fe15dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc9f89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe07dc0 0x55566fe16300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc9f89740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe8ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc9f89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01b1030 con 0x55566fe07dc0 2024-09-19T06:16:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc9f89740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01b0900 con 0x55566fe12680 2024-09-19T06:16:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc9f89740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01adb00 con 0x55566fe074a0 2024-09-19T06:16:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc9f89740 10 client.0 _ll_register_callbacks cb 0x7fdcbc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc614d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe8ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc614d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe8ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41572/0 (socket says 172.21.15.5:41572) 2024-09-19T06:16:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc614d640 1 -- 172.21.15.5:0/60939073 learned_addr learned my addr 172.21.15.5:0/60939073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc594c640 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe074a0 0x55566fe15dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.896+0000 7fdcc514b640 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe07dc0 0x55566fe16300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdcc614d640 1 -- 172.21.15.5:0/60939073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe074a0 msgr2=0x55566fe15dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdcc614d640 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe074a0 0x55566fe15dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdcc614d640 1 -- 172.21.15.5:0/60939073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe07dc0 msgr2=0x55566fe16300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdcc614d640 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe07dc0 0x55566fe16300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdcc614d640 1 -- 172.21.15.5:0/60939073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55566fe8b170 con 0x55566fe12680 2024-09-19T06:16:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdcc514b640 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe07dc0 0x55566fe16300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdcc594c640 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe074a0 0x55566fe15dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdcc614d640 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe8ab60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009ca0 tx=0x7fdcb802a950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6476fd21edeeff7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:16:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdca97fa640 1 -- 172.21.15.5:0/60939073 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8057060 con 0x55566fe12680 2024-09-19T06:16:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdcc9f89740 1 -- 172.21.15.5:0/60939073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcbc003cf0 con 0x55566fe12680 2024-09-19T06:16:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdca97fa640 1 -- 172.21.15.5:0/60939073 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fdcb8030650 con 0x55566fe12680 2024-09-19T06:16:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdca97fa640 1 -- 172.21.15.5:0/60939073 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdcb8034040 con 0x55566fe12680 2024-09-19T06:16:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdcc9f89740 10 client.28247 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdcc9f89740 1 -- 172.21.15.5:0/60939073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55566fe8c4b0 con 0x55566fe12680 2024-09-19T06:16:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.897+0000 7fdca97fa640 1 -- 172.21.15.5:0/60939073 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcbc003cf0 con 0x55566fe12680 2024-09-19T06:16:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.898+0000 7fdca97fa640 1 -- 172.21.15.5:0/60939073 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fdcb8004020 con 0x55566fe12680 2024-09-19T06:16:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.898+0000 7fdcc9f89740 10 client.28247 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.898+0000 7fdcc9f89740 10 client.28247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.898+0000 7fdcc9f89740 20 client.28247 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.898+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55566fe8cc70 0x55566fe8f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.898+0000 7fdcc9f89740 4 client.28247 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:16:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.898+0000 7fdcc9f89740 1 -- 172.21.15.5:0/60939073 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55566fd74d60 con 0x55566fe8cc70 2024-09-19T06:16:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.898+0000 7fdcc594c640 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55566fe8cc70 0x55566fe8f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.898+0000 7fdca97fa640 10 client.28247 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.898+0000 7fdcc594c640 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55566fe8cc70 0x55566fe8f090 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4613c0065d25fdc server_cookie=51158ddd77d4b44c in_seq=0 out_seq=0 2024-09-19T06:16:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdca97fa640 1 -- 172.21.15.5:0/60939073 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55566fd74d60 con 0x55566fe8cc70 2024-09-19T06:16:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdca97fa640 10 client.28247 handle_command_reply: tid=0 2024-09-19T06:16:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdcc9f89740 1 client.28247 shutdown 2024-09-19T06:16:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdcc9f89740 1 -- 172.21.15.5:0/60939073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 msgr2=0x55566fe8ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe8ab60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009ca0 tx=0x7fdcb802a950 comp rx=0 tx=0).stop 2024-09-19T06:16:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdcc9f89740 1 -- 172.21.15.5:0/60939073 shutdown_connections 2024-09-19T06:16:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55566fe8cc70 0x55566fe8f090 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55566fe12680 0x55566fe8ab60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdca97fa640 0 client.28247 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55566fe07dc0 0x55566fe16300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdcc9f89740 1 --2- 172.21.15.5:0/60939073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55566fe074a0 0x55566fe15dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdcc9f89740 1 -- 172.21.15.5:0/60939073 >> 172.21.15.5:0/60939073 conn(0x55566fe03a30 msgr2=0x55566fe02350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdcc9f89740 1 -- 172.21.15.5:0/60939073 shutdown_connections 2024-09-19T06:16:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdcc9f89740 1 -- 172.21.15.5:0/60939073 wait complete. 2024-09-19T06:16:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.899+0000 7fdcc9f89740 20 client.28247 trim_cache size 0 max 16384 2024-09-19T06:16:06.909 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:16:06.909 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:16:06.909 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:16:06.910 INFO:teuthology.orchestra.run.smithi005.stdout: "75f103c4-ce5a-49f6-9944-ab3d16413862": { 2024-09-19T06:16:06.910 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:16:06.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "75f103c4-ce5a-49f6-9944-ab3d16413862", 2024-09-19T06:16:06.910 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:16:06.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:16:06.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:16:06.910 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:16:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.900+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/22404126 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdcac0a33b0 msgr2=0x7fdcac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.900+0000 7fdcc7bb0640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdcac0a33b0 0x7fdcac0a5870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8005e30 tx=0x7fdcb803c040 comp rx=0 tx=0).stop 2024-09-19T06:16:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.900+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/22404126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102570 msgr2=0x7fdcc01975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.900+0000 7fdcc7bb0640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102570 0x7fdcc01975f0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000e9f0 tx=0x7fdcb000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.900+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/22404126 shutdown_connections 2024-09-19T06:16:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.900+0000 7fdcc7bb0640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdcac0a33b0 0x7fdcac0a5870 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.900+0000 7fdcc7bb0640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcc0102ef0 0x7fdcc0197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.900+0000 7fdcc7bb0640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102570 0x7fdcc01975f0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.900+0000 7fdcc7bb0640 1 --2- 172.21.15.5:0/22404126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcc0100e20 0x7fdcc01970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:06.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.900+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/22404126 >> 172.21.15.5:0/22404126 conn(0x7fdcc00fc7d0 msgr2=0x7fdcc00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:06.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.900+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/22404126 shutdown_connections 2024-09-19T06:16:06.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:06.900+0000 7fdcc7bb0640 1 -- 172.21.15.5:0/22404126 wait complete. 2024-09-19T06:16:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:06 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:16:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:06 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:06 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:16:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:06 smithi005 ceph-mon[30955]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-19T06:16:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:06 smithi005 ceph-mon[30955]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-09-19T06:16:07.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:06 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:07.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:06 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:16:07.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:06 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:07.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:06 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:16:07.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:06 smithi049 ceph-mon[34640]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-19T06:16:07.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:06 smithi049 ceph-mon[34640]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-09-19T06:16:07.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:06 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:07.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:06 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:16:07.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:06 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:07.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:06 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:16:07.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:06 smithi157 ceph-mon[35346]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-19T06:16:07.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:06 smithi157 ceph-mon[35346]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-09-19T06:16:07.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:06 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:07.637 INFO:tasks.cephfs.filesystem:scrub status for tag:75f103c4-ce5a-49f6-9944-ab3d16413862 - {'path': '/', 'tag': '75f103c4-ce5a-49f6-9944-ab3d16413862', 'options': 'recursive,force'} 2024-09-19T06:16:07.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 288441 2.13 MB/sec execute 573 sec latency 11.244 ms 2024-09-19T06:16:08.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:07 smithi049 ceph-mon[34640]: pgmap v399: 97 pgs: 97 active+clean; 159 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-09-19T06:16:08.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:07 smithi157 ceph-mon[35346]: pgmap v399: 97 pgs: 97 active+clean; 159 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-09-19T06:16:08.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:07 smithi005 ceph-mon[30955]: pgmap v399: 97 pgs: 97 active+clean; 159 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-09-19T06:16:08.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 288963 2.13 MB/sec execute 574 sec latency 21.639 ms 2024-09-19T06:16:09.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 289508 2.13 MB/sec execute 575 sec latency 26.433 ms 2024-09-19T06:16:10.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:09 smithi157 ceph-mon[35346]: pgmap v400: 97 pgs: 97 active+clean; 159 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 497 KiB/s wr, 77 op/s 2024-09-19T06:16:10.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:09 smithi005 ceph-mon[30955]: pgmap v400: 97 pgs: 97 active+clean; 159 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 497 KiB/s wr, 77 op/s 2024-09-19T06:16:10.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:09 smithi049 ceph-mon[34640]: pgmap v400: 97 pgs: 97 active+clean; 159 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 497 KiB/s wr, 77 op/s 2024-09-19T06:16:10.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 289953 2.13 MB/sec execute 576 sec latency 18.489 ms 2024-09-19T06:16:11.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:10 smithi157 ceph-mon[35346]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T06:16:11.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:10 smithi005 ceph-mon[30955]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T06:16:11.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:10 smithi049 ceph-mon[34640]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T06:16:11.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 290656 2.13 MB/sec execute 577 sec latency 15.372 ms 2024-09-19T06:16:12.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:11 smithi157 ceph-mon[35346]: pgmap v402: 97 pgs: 97 active+clean; 159 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 596 KiB/s wr, 93 op/s 2024-09-19T06:16:12.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:11 smithi005 ceph-mon[30955]: pgmap v402: 97 pgs: 97 active+clean; 159 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 596 KiB/s wr, 93 op/s 2024-09-19T06:16:12.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:11 smithi049 ceph-mon[34640]: pgmap v402: 97 pgs: 97 active+clean; 159 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 596 KiB/s wr, 93 op/s 2024-09-19T06:16:12.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 290948 2.14 MB/sec execute 578 sec latency 449.107 ms 2024-09-19T06:16:13.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 290948 2.13 MB/sec execute 579 sec latency 1449.224 ms 2024-09-19T06:16:14.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:14 smithi005 ceph-mon[30955]: pgmap v403: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-09-19T06:16:14.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:14 smithi005 ceph-mon[30955]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T06:16:14.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:14 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:16:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:14 smithi049 ceph-mon[34640]: pgmap v403: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-09-19T06:16:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:14 smithi049 ceph-mon[34640]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T06:16:14.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:14 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:16:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:14 smithi157 ceph-mon[35346]: pgmap v403: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-09-19T06:16:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:14 smithi157 ceph-mon[35346]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T06:16:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:14 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:16:14.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 290948 2.13 MB/sec execute 580 sec latency 2449.282 ms 2024-09-19T06:16:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.894+0000 7fcac0d11640 1 -- 172.21.15.5:0/3600746339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcabc1069b0 msgr2=0x7fcabc106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.894+0000 7fcac0d11640 1 --2- 172.21.15.5:0/3600746339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcabc1069b0 0x7fcabc106d90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009cd0 tx=0x7fcaac02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.894+0000 7fcac0d11640 1 -- 172.21.15.5:0/3600746339 shutdown_connections 2024-09-19T06:16:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.894+0000 7fcac0d11640 1 --2- 172.21.15.5:0/3600746339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcabc101340 0x7fcabc10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.894+0000 7fcac0d11640 1 --2- 172.21.15.5:0/3600746339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc1009a0 0x7fcabc100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.894+0000 7fcac0d11640 1 --2- 172.21.15.5:0/3600746339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcabc1069b0 0x7fcabc106d90 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.894+0000 7fcac0d11640 1 -- 172.21.15.5:0/3600746339 >> 172.21.15.5:0/3600746339 conn(0x7fcabc0779a0 msgr2=0x7fcabc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.894+0000 7fcac0d11640 1 -- 172.21.15.5:0/3600746339 shutdown_connections 2024-09-19T06:16:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.894+0000 7fcac0d11640 1 -- 172.21.15.5:0/3600746339 wait complete. 2024-09-19T06:16:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.895+0000 7fcac0d11640 1 Processor -- start 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.895+0000 7fcac0d11640 1 -- start start 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.895+0000 7fcac0d11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc1009a0 0x7fcabc199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.895+0000 7fcac0d11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcabc101340 0x7fcabc1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.895+0000 7fcac0d11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcabc1069b0 0x7fcabc1a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.895+0000 7fcac0d11640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc110ca0 con 0x7fcabc1069b0 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.895+0000 7fcac0d11640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc110b20 con 0x7fcabc1009a0 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.895+0000 7fcac0d11640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc110e20 con 0x7fcabc101340 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.895+0000 7fcabad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcabc1069b0 0x7fcabc1a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.895+0000 7fcabad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcabc1069b0 0x7fcabc1a41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41616/0 (socket says 172.21.15.5:41616) 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcabad9d640 1 -- 172.21.15.5:0/2911698721 learned_addr learned my addr 172.21.15.5:0/2911698721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcaba59c640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc1009a0 0x7fcabc199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcabc101340 0x7fcabc1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcabad9d640 1 -- 172.21.15.5:0/2911698721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcabc101340 msgr2=0x7fcabc1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcabad9d640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcabc101340 0x7fcabc1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcabad9d640 1 -- 172.21.15.5:0/2911698721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc1009a0 msgr2=0x7fcabc199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcabad9d640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc1009a0 0x7fcabc199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcabad9d640 1 -- 172.21.15.5:0/2911698721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc1a4730 con 0x7fcabc1069b0 2024-09-19T06:16:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcabc101340 0x7fcabc1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcabad9d640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcabc1069b0 0x7fcabc1a41f0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fcab400d750 tx=0x7fcab400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcaab7fe640 1 -- 172.21.15.5:0/2911698721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4017070 con 0x7fcabc1069b0 2024-09-19T06:16:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcaab7fe640 1 -- 172.21.15.5:0/2911698721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcab40054d0 con 0x7fcabc1069b0 2024-09-19T06:16:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcaab7fe640 1 -- 172.21.15.5:0/2911698721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4004c60 con 0x7fcabc1069b0 2024-09-19T06:16:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcac0d11640 1 -- 172.21.15.5:0/2911698721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc1a4900 con 0x7fcabc1069b0 2024-09-19T06:16:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.896+0000 7fcac0d11640 1 -- 172.21.15.5:0/2911698721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1a4e20 con 0x7fcabc1069b0 2024-09-19T06:16:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.899+0000 7fcaab7fe640 1 -- 172.21.15.5:0/2911698721 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcab4004e00 con 0x7fcabc1069b0 2024-09-19T06:16:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.899+0000 7fcac3aa8740 1 Processor -- start 2024-09-19T06:16:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.899+0000 7fcac3aa8740 1 -- start start 2024-09-19T06:16:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcac3aa8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf7af0 0x559c9fdf7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcac3aa8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf8410 0x559c9fe02770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcac3aa8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 0x559c9fe050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcac3aa8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559c9fd5c2f0 con 0x559c9fe02cb0 2024-09-19T06:16:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcac3aa8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559c9fd04270 con 0x559c9fdf8410 2024-09-19T06:16:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcac3aa8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559c9fd582b0 con 0x559c9fdf7af0 2024-09-19T06:16:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcaab7fe640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcaa40a33b0 0x7fcaa40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcab9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 0x559c9fe050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcaba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf8410 0x559c9fe02770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcab9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 0x559c9fe050a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41620/0 (socket says 172.21.15.5:41620) 2024-09-19T06:16:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcab9d9b640 1 -- 172.21.15.5:0/979408910 learned_addr learned my addr 172.21.15.5:0/979408910 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/979408910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf7af0 0x559c9fdf7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcabad9d640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcaa40a33b0 0x7fcaa40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcaab7fe640 1 -- 172.21.15.5:0/2911698721 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fcab40cf680 con 0x7fcabc1069b0 2024-09-19T06:16:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcab9d9b640 1 -- 172.21.15.5:0/979408910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf7af0 msgr2=0x559c9fdf7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/979408910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf7af0 0x559c9fdf7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcab9d9b640 1 -- 172.21.15.5:0/979408910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf8410 msgr2=0x559c9fe02770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/979408910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf8410 0x559c9fe02770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcab9d9b640 1 -- 172.21.15.5:0/979408910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c9fd03340 con 0x559c9fe02cb0 2024-09-19T06:16:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcaba59c640 1 --2- 172.21.15.5:0/979408910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf8410 0x559c9fe02770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.900+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/979408910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 0x559c9fe050a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fcab00149f0 tx=0x7fcab000b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4841344240d1a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.901+0000 7fcabad9d640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcaa40a33b0 0x7fcaa40a5870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcab409b640 tx=0x7fcab4086040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.901+0000 7fcaa8ff9640 1 -- 172.21.15.5:0/979408910 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab00142d0 con 0x559c9fe02cb0 2024-09-19T06:16:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.901+0000 7fcaa8ff9640 1 -- 172.21.15.5:0/979408910 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcab0005020 con 0x559c9fe02cb0 2024-09-19T06:16:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.901+0000 7fcaa8ff9640 1 -- 172.21.15.5:0/979408910 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab00049a0 con 0x559c9fe02cb0 2024-09-19T06:16:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.901+0000 7fcac3aa8740 1 -- 172.21.15.5:0/979408910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 msgr2=0x559c9fe050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.901+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/979408910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 0x559c9fe050a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fcab00149f0 tx=0x7fcab000b540 comp rx=0 tx=0).stop 2024-09-19T06:16:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.901+0000 7fcac3aa8740 1 -- 172.21.15.5:0/979408910 shutdown_connections 2024-09-19T06:16:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.901+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/979408910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 0x559c9fe050a0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.901+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/979408910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf8410 0x559c9fe02770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.901+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/979408910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf7af0 0x559c9fdf7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.901+0000 7fcac3aa8740 1 -- 172.21.15.5:0/979408910 >> 172.21.15.5:0/979408910 conn(0x559c9fdf4060 msgr2=0x559c9fdf4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcac3aa8740 1 -- 172.21.15.5:0/979408910 shutdown_connections 2024-09-19T06:16:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcac3aa8740 1 -- 172.21.15.5:0/979408910 wait complete. 2024-09-19T06:16:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcac3aa8740 1 -- start start 2024-09-19T06:16:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcac3aa8740 1 Processor -- start 2024-09-19T06:16:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcac3aa8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf7af0 0x559c9fdffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcac3aa8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf8410 0x559c9fe00100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcac3aa8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fe02cb0 0x559c9fe00640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcac3aa8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559c9fe05880 con 0x559c9fdf7af0 2024-09-19T06:16:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcac3aa8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559c9fce5290 con 0x559c9fe02cb0 2024-09-19T06:16:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcac3aa8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559c9fe05a00 con 0x559c9fdf8410 2024-09-19T06:16:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcac3aa8740 10 client.0 _ll_register_callbacks cb 0x559c9fd16140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcab9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf7af0 0x559c9fdffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcab9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf7af0 0x559c9fdffbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41632/0 (socket says 172.21.15.5:41632) 2024-09-19T06:16:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.902+0000 7fcab9d9b640 1 -- 172.21.15.5:0/3055357683 learned_addr learned my addr 172.21.15.5:0/3055357683 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcaba59c640 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf8410 0x559c9fe00100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fe02cb0 0x559c9fe00640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcab9d9b640 1 -- 172.21.15.5:0/3055357683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf8410 msgr2=0x559c9fe00100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf8410 0x559c9fe00100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcab9d9b640 1 -- 172.21.15.5:0/3055357683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fe02cb0 msgr2=0x559c9fe00640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fe02cb0 0x559c9fe00640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcab9d9b640 1 -- 172.21.15.5:0/3055357683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c9fa6f210 con 0x559c9fdf7af0 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf7af0 0x559c9fdffbc0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fcab0006010 tx=0x7fcab0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5dc9662f2e2a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcaa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcaa1ffb640 1 -- 172.21.15.5:0/3055357683 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0029020 con 0x559c9fdf7af0 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcaa1ffb640 1 -- 172.21.15.5:0/3055357683 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fcab000fb70 con 0x559c9fdf7af0 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcac3aa8740 1 -- 172.21.15.5:0/3055357683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab0005020 con 0x559c9fdf7af0 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcaa1ffb640 1 -- 172.21.15.5:0/3055357683 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcab000fd90 con 0x559c9fdf7af0 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcac3aa8740 10 client.18843 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.903+0000 7fcac3aa8740 1 -- 172.21.15.5:0/3055357683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559c9fb3b620 con 0x559c9fdf7af0 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.904+0000 7fcaa1ffb640 1 -- 172.21.15.5:0/3055357683 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcab0005020 con 0x559c9fdf7af0 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.904+0000 7fcaa1ffb640 1 -- 172.21.15.5:0/3055357683 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fcab0012450 con 0x559c9fdf7af0 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.904+0000 7fcac3aa8740 10 client.18843 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.904+0000 7fcac3aa8740 10 client.18843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.904+0000 7fcac3aa8740 20 client.18843 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.904+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559c9fe12b60 0x559c9fe14f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.904+0000 7fcac3aa8740 4 client.18843 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.904+0000 7fcac3aa8740 1 -- 172.21.15.5:0/3055357683 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c9fd69a20 con 0x559c9fe12b60 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.904+0000 7fcaba59c640 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559c9fe12b60 0x559c9fe14f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcaa1ffb640 10 client.18843 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcaba59c640 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559c9fe12b60 0x559c9fe14f50 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b813d22d5ce9ee server_cookie=6ea071279449a5a3 in_seq=0 out_seq=0 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcaa1ffb640 1 -- 172.21.15.5:0/3055357683 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c9fd69a20 con 0x559c9fe12b60 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcaa1ffb640 10 client.18843 handle_command_reply: tid=0 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcac3aa8740 1 client.18843 shutdown 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcac3aa8740 1 -- 172.21.15.5:0/3055357683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf7af0 msgr2=0x559c9fdffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf7af0 0x559c9fdffbc0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fcab0006010 tx=0x7fcab0001e50 comp rx=0 tx=0).stop 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcac3aa8740 1 -- 172.21.15.5:0/3055357683 shutdown_connections 2024-09-19T06:16:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559c9fe12b60 0x559c9fe14f50 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fe02cb0 0x559c9fe00640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf8410 0x559c9fe00100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcaa1ffb640 0 client.18843 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/3055357683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf7af0 0x559c9fdffbc0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.905+0000 7fcac3aa8740 1 -- 172.21.15.5:0/3055357683 >> 172.21.15.5:0/3055357683 conn(0x559c9fdf4060 msgr2=0x559c9fdf9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.906+0000 7fcac3aa8740 1 -- 172.21.15.5:0/3055357683 shutdown_connections 2024-09-19T06:16:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.906+0000 7fcac3aa8740 1 -- 172.21.15.5:0/3055357683 wait complete. 2024-09-19T06:16:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.906+0000 7fcac3aa8740 20 client.18843 trim_cache size 0 max 16384 2024-09-19T06:16:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.920+0000 7fcac3aa8740 1 Processor -- start 2024-09-19T06:16:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.920+0000 7fcac3aa8740 1 -- start start 2024-09-19T06:16:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcac3aa8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf7af0 0x559c9fdfe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcac3aa8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf8410 0x559c9fe00470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcac3aa8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fe02cb0 0x559c9fe1a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcac3aa8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1ad860 con 0x559c9fdf8410 2024-09-19T06:16:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcac3aa8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1af460 con 0x559c9fdf7af0 2024-09-19T06:16:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcac3aa8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1adf30 con 0x559c9fe02cb0 2024-09-19T06:16:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcaba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf8410 0x559c9fe00470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcaba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf8410 0x559c9fe00470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41638/0 (socket says 172.21.15.5:41638) 2024-09-19T06:16:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcaba59c640 1 -- 172.21.15.5:0/2041512340 learned_addr learned my addr 172.21.15.5:0/2041512340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/2041512340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf7af0 0x559c9fdfe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcaba59c640 1 -- 172.21.15.5:0/2041512340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fe02cb0 msgr2=0x559c9fe1a520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:16:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcaba59c640 1 --2- 172.21.15.5:0/2041512340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fe02cb0 0x559c9fe1a520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcaba59c640 1 -- 172.21.15.5:0/2041512340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf7af0 msgr2=0x559c9fdfe470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcaba59c640 1 --2- 172.21.15.5:0/2041512340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf7af0 0x559c9fdfe470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcaba59c640 1 -- 172.21.15.5:0/2041512340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c9fdfe0f0 con 0x559c9fdf8410 2024-09-19T06:16:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcaba59c640 1 --2- 172.21.15.5:0/2041512340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf8410 0x559c9fe00470 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fcaac02ae00 tx=0x7fcaac005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759ffe105f43a32b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcaa3fff640 1 -- 172.21.15.5:0/2041512340 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac031990 con 0x559c9fdf8410 2024-09-19T06:16:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcaa3fff640 1 -- 172.21.15.5:0/2041512340 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcaac036040 con 0x559c9fdf8410 2024-09-19T06:16:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.921+0000 7fcaa3fff640 1 -- 172.21.15.5:0/2041512340 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac004600 con 0x559c9fdf8410 2024-09-19T06:16:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.922+0000 7fcac3aa8740 1 -- 172.21.15.5:0/2041512340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf8410 msgr2=0x559c9fe00470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.922+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/2041512340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf8410 0x559c9fe00470 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fcaac02ae00 tx=0x7fcaac005fa0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.922+0000 7fcac3aa8740 1 -- 172.21.15.5:0/2041512340 shutdown_connections 2024-09-19T06:16:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.922+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/2041512340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fe02cb0 0x559c9fe1a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.922+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/2041512340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fdf8410 0x559c9fe00470 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.922+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/2041512340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf7af0 0x559c9fdfe470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.922+0000 7fcac3aa8740 1 -- 172.21.15.5:0/2041512340 >> 172.21.15.5:0/2041512340 conn(0x559c9fdf4060 msgr2=0x559c9fe01c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.922+0000 7fcac3aa8740 1 -- 172.21.15.5:0/2041512340 shutdown_connections 2024-09-19T06:16:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.922+0000 7fcac3aa8740 1 -- 172.21.15.5:0/2041512340 wait complete. 2024-09-19T06:16:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.922+0000 7fcac3aa8740 1 -- start start 2024-09-19T06:16:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcac3aa8740 1 Processor -- start 2024-09-19T06:16:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcac3aa8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf7af0 0x559c9fe0ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcac3aa8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf8410 0x559c9fe0b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcac3aa8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 0x559c9fe06250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcac3aa8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1aed90 con 0x559c9fe02cb0 2024-09-19T06:16:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcac3aa8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1ae660 con 0x559c9fdf8410 2024-09-19T06:16:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcac3aa8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1ab860 con 0x559c9fdf7af0 2024-09-19T06:16:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcac3aa8740 10 client.0 _ll_register_callbacks cb 0x7fcaac004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcab9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 0x559c9fe06250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcab9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 0x559c9fe06250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41642/0 (socket says 172.21.15.5:41642) 2024-09-19T06:16:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcab9d9b640 1 -- 172.21.15.5:0/1926731545 learned_addr learned my addr 172.21.15.5:0/1926731545 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf7af0 0x559c9fe0ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcab9d9b640 1 -- 172.21.15.5:0/1926731545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf7af0 msgr2=0x559c9fe0ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf7af0 0x559c9fe0ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcab9d9b640 1 -- 172.21.15.5:0/1926731545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf8410 msgr2=0x559c9fe0b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcaba59c640 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf8410 0x559c9fe0b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf8410 0x559c9fe0b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcab9d9b640 1 -- 172.21.15.5:0/1926731545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c9fe06a80 con 0x559c9fe02cb0 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcab9d9b640 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 0x559c9fe06250 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcab0006010 tx=0x7fcab0014450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ab86fc09ab6481 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.923+0000 7fcaa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcaa1ffb640 1 -- 172.21.15.5:0/1926731545 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0029040 con 0x559c9fe02cb0 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcaa1ffb640 1 -- 172.21.15.5:0/1926731545 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fcab000fb70 con 0x559c9fe02cb0 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcac3aa8740 1 -- 172.21.15.5:0/1926731545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaac036040 con 0x559c9fe02cb0 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcaa1ffb640 1 -- 172.21.15.5:0/1926731545 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcab0012ae0 con 0x559c9fe02cb0 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcac3aa8740 10 client.18849 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcac3aa8740 1 -- 172.21.15.5:0/1926731545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559c9fe81440 con 0x559c9fe02cb0 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcaa1ffb640 1 -- 172.21.15.5:0/1926731545 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcaac036040 con 0x559c9fe02cb0 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcaa1ffb640 1 -- 172.21.15.5:0/1926731545 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fcab0003cd0 con 0x559c9fe02cb0 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcac3aa8740 10 client.18849 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcac3aa8740 10 client.18849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcac3aa8740 20 client.18849 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559c9fe81a90 0x559c9fe83eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcac3aa8740 4 client.18849 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:16:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcac3aa8740 1 -- 172.21.15.5:0/1926731545 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c9fd63d70 con 0x559c9fe81a90 2024-09-19T06:16:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.924+0000 7fcaba59c640 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559c9fe81a90 0x559c9fe83eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.925+0000 7fcaba59c640 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559c9fe81a90 0x559c9fe83eb0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc8ea3937f13a67 server_cookie=aa5383e9f8a8a23 in_seq=0 out_seq=0 2024-09-19T06:16:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.925+0000 7fcaa1ffb640 10 client.18849 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcaa1ffb640 1 -- 172.21.15.5:0/1926731545 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c9fd63d70 con 0x559c9fe81a90 2024-09-19T06:16:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcaa1ffb640 10 client.18849 handle_command_reply: tid=0 2024-09-19T06:16:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac3aa8740 1 client.18849 shutdown 2024-09-19T06:16:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac3aa8740 1 -- 172.21.15.5:0/1926731545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 msgr2=0x559c9fe06250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 0x559c9fe06250 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcab0006010 tx=0x7fcab0014450 comp rx=0 tx=0).stop 2024-09-19T06:16:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac3aa8740 1 -- 172.21.15.5:0/1926731545 shutdown_connections 2024-09-19T06:16:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559c9fe81a90 0x559c9fe83eb0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c9fe02cb0 0x559c9fe06250 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9fdf8410 0x559c9fe0b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcaa1ffb640 0 client.18849 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac3aa8740 1 --2- 172.21.15.5:0/1926731545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c9fdf7af0 0x559c9fe0ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac3aa8740 1 -- 172.21.15.5:0/1926731545 >> 172.21.15.5:0/1926731545 conn(0x559c9fdf4060 msgr2=0x559c9fdf94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac3aa8740 1 -- 172.21.15.5:0/1926731545 shutdown_connections 2024-09-19T06:16:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac3aa8740 1 -- 172.21.15.5:0/1926731545 wait complete. 2024-09-19T06:16:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac3aa8740 20 client.18849 trim_cache size 0 max 16384 2024-09-19T06:16:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:16:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:16:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:16:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:16:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac0d11640 1 -- 172.21.15.5:0/2911698721 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcaa40a33b0 msgr2=0x7fcaa40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac0d11640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcaa40a33b0 0x7fcaa40a5870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcab409b640 tx=0x7fcab4086040 comp rx=0 tx=0).stop 2024-09-19T06:16:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac0d11640 1 -- 172.21.15.5:0/2911698721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcabc1069b0 msgr2=0x7fcabc1a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac0d11640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcabc1069b0 0x7fcabc1a41f0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fcab400d750 tx=0x7fcab400dc10 comp rx=0 tx=0).stop 2024-09-19T06:16:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac0d11640 1 -- 172.21.15.5:0/2911698721 shutdown_connections 2024-09-19T06:16:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac0d11640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcabc1069b0 0x7fcabc1a41f0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac0d11640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcaa40a33b0 0x7fcaa40a5870 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac0d11640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcabc101340 0x7fcabc1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac0d11640 1 --2- 172.21.15.5:0/2911698721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc1009a0 0x7fcabc199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.949+0000 7fcac0d11640 1 -- 172.21.15.5:0/2911698721 >> 172.21.15.5:0/2911698721 conn(0x7fcabc0779a0 msgr2=0x7fcabc103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.950+0000 7fcac0d11640 1 -- 172.21.15.5:0/2911698721 shutdown_connections 2024-09-19T06:16:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:14.950+0000 7fcac0d11640 1 -- 172.21.15.5:0/2911698721 wait complete. 2024-09-19T06:16:15.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:15 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:16:15.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:15 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:16:15.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:15 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:16:15.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:15 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:16:15.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 291015 2.12 MB/sec execute 581 sec latency 3067.231 ms 2024-09-19T06:16:15.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:15 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:16:15.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:15 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:16:16.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:16 smithi049 ceph-mon[34640]: pgmap v405: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 89 op/s 2024-09-19T06:16:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:16 smithi157 ceph-mon[35346]: pgmap v405: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 89 op/s 2024-09-19T06:16:16.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 291015 2.12 MB/sec execute 582 sec latency 1133.654 ms 2024-09-19T06:16:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:16 smithi005 ceph-mon[30955]: pgmap v405: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 89 op/s 2024-09-19T06:16:17.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 291015 2.12 MB/sec execute 583 sec latency 2133.783 ms 2024-09-19T06:16:18.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:18 smithi049 ceph-mon[34640]: pgmap v406: 97 pgs: 97 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-09-19T06:16:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:18 smithi157 ceph-mon[35346]: pgmap v406: 97 pgs: 97 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-09-19T06:16:18.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 291015 2.11 MB/sec execute 584 sec latency 3133.888 ms 2024-09-19T06:16:18.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:18 smithi005 ceph-mon[30955]: pgmap v406: 97 pgs: 97 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-09-19T06:16:19.851 INFO:tasks.workunit.client.0.smithi005.stdout: 1 291015 2.11 MB/sec execute 585 sec latency 4133.972 ms 2024-09-19T06:16:20.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:20 smithi049 ceph-mon[34640]: pgmap v407: 97 pgs: 97 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-19T06:16:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:20 smithi157 ceph-mon[35346]: pgmap v407: 97 pgs: 97 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-19T06:16:20.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 291129 2.11 MB/sec execute 586 sec latency 4751.138 ms 2024-09-19T06:16:20.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:20 smithi005 ceph-mon[30955]: pgmap v407: 97 pgs: 97 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-19T06:16:21.851 INFO:tasks.workunit.client.0.smithi005.stdout: 1 291377 2.11 MB/sec execute 587 sec latency 14.826 ms 2024-09-19T06:16:22.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:22 smithi049 ceph-mon[34640]: pgmap v408: 97 pgs: 97 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:16:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:22 smithi157 ceph-mon[35346]: pgmap v408: 97 pgs: 97 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:16:22.850 INFO:tasks.workunit.client.0.smithi005.stdout: 1 291814 2.10 MB/sec execute 588 sec latency 14.912 ms 2024-09-19T06:16:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:22 smithi005 ceph-mon[30955]: pgmap v408: 97 pgs: 97 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-09-19T06:16:23.851 INFO:tasks.workunit.client.0.smithi005.stdout: 1 292347 2.11 MB/sec execute 589 sec latency 21.206 ms 2024-09-19T06:16:24.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:24 smithi049 ceph-mon[34640]: pgmap v409: 97 pgs: 97 active+clean; 160 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 447 KiB/s wr, 54 op/s 2024-09-19T06:16:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:24 smithi157 ceph-mon[35346]: pgmap v409: 97 pgs: 97 active+clean; 160 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 447 KiB/s wr, 54 op/s 2024-09-19T06:16:24.851 INFO:tasks.workunit.client.0.smithi005.stdout: 1 292854 2.10 MB/sec execute 590 sec latency 20.538 ms 2024-09-19T06:16:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:24 smithi005 ceph-mon[30955]: pgmap v409: 97 pgs: 97 active+clean; 160 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 447 KiB/s wr, 54 op/s 2024-09-19T06:16:25.851 INFO:tasks.workunit.client.0.smithi005.stdout: 1 293380 2.10 MB/sec execute 591 sec latency 20.075 ms 2024-09-19T06:16:26.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:26 smithi049 ceph-mon[34640]: pgmap v410: 97 pgs: 97 active+clean; 160 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 391 KiB/s wr, 47 op/s 2024-09-19T06:16:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:26 smithi157 ceph-mon[35346]: pgmap v410: 97 pgs: 97 active+clean; 160 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 391 KiB/s wr, 47 op/s 2024-09-19T06:16:26.851 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294155 2.11 MB/sec execute 592 sec latency 15.576 ms 2024-09-19T06:16:26.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:26 smithi005 ceph-mon[30955]: pgmap v410: 97 pgs: 97 active+clean; 160 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 391 KiB/s wr, 47 op/s 2024-09-19T06:16:27.851 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294491 2.11 MB/sec execute 593 sec latency 148.590 ms 2024-09-19T06:16:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.040+0000 7f0de9a80640 1 -- 172.21.15.5:0/2409108307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de4102d00 msgr2=0x7f0de41030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.040+0000 7f0de9a80640 1 --2- 172.21.15.5:0/2409108307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de4102d00 0x7f0de41030e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009900 tx=0x7f0dd402a480 comp rx=0 tx=0).stop 2024-09-19T06:16:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.040+0000 7f0de9a80640 1 -- 172.21.15.5:0/2409108307 shutdown_connections 2024-09-19T06:16:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.040+0000 7f0de9a80640 1 --2- 172.21.15.5:0/2409108307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de410c0d0 0x7f0de410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.040+0000 7f0de9a80640 1 --2- 172.21.15.5:0/2409108307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de4103620 0x7f0de410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.040+0000 7f0de9a80640 1 --2- 172.21.15.5:0/2409108307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de4102d00 0x7f0de41030e0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.040+0000 7f0de9a80640 1 -- 172.21.15.5:0/2409108307 >> 172.21.15.5:0/2409108307 conn(0x7f0de40779a0 msgr2=0x7f0de40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.040+0000 7f0de9a80640 1 -- 172.21.15.5:0/2409108307 shutdown_connections 2024-09-19T06:16:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.040+0000 7f0de9a80640 1 -- 172.21.15.5:0/2409108307 wait complete. 2024-09-19T06:16:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de9a80640 1 Processor -- start 2024-09-19T06:16:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de9a80640 1 -- start start 2024-09-19T06:16:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de9a80640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de4102d00 0x7f0de4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de9a80640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de4103620 0x7f0de4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de9a80640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de410c0d0 0x7f0de419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de9a80640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4111d50 con 0x7f0de4102d00 2024-09-19T06:16:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de9a80640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4111bd0 con 0x7f0de4103620 2024-09-19T06:16:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de9a80640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4111ed0 con 0x7f0de410c0d0 2024-09-19T06:16:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de4102d00 0x7f0de4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de4102d00 0x7f0de4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41618/0 (socket says 172.21.15.5:41618) 2024-09-19T06:16:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de37fe640 1 -- 172.21.15.5:0/1326559475 learned_addr learned my addr 172.21.15.5:0/1326559475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de3fff640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de410c0d0 0x7f0de419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de2ffd640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de4103620 0x7f0de4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de37fe640 1 -- 172.21.15.5:0/1326559475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de410c0d0 msgr2=0x7f0de419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de37fe640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de410c0d0 0x7f0de419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de37fe640 1 -- 172.21.15.5:0/1326559475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de4103620 msgr2=0x7f0de4198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de37fe640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de4103620 0x7f0de4198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.041+0000 7f0de37fe640 1 -- 172.21.15.5:0/1326559475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de419d2f0 con 0x7f0de4102d00 2024-09-19T06:16:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.042+0000 7f0de2ffd640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de4103620 0x7f0de4198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.042+0000 7f0de37fe640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de4102d00 0x7f0de4198300 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4006010 tx=0x7f0dd4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.042+0000 7f0de0ff9640 1 -- 172.21.15.5:0/1326559475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd403f030 con 0x7f0de4102d00 2024-09-19T06:16:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.042+0000 7f0de0ff9640 1 -- 172.21.15.5:0/1326559475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dd40035f0 con 0x7f0de4102d00 2024-09-19T06:16:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.042+0000 7f0de9a80640 1 -- 172.21.15.5:0/1326559475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de419d580 con 0x7f0de4102d00 2024-09-19T06:16:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.042+0000 7f0de0ff9640 1 -- 172.21.15.5:0/1326559475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4035070 con 0x7f0de4102d00 2024-09-19T06:16:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.042+0000 7f0de9a80640 1 -- 172.21.15.5:0/1326559475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de41a4e70 con 0x7f0de4102d00 2024-09-19T06:16:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.044+0000 7f0de0ff9640 1 -- 172.21.15.5:0/1326559475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0dd4003b10 con 0x7f0de4102d00 2024-09-19T06:16:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.044+0000 7f0dec80e740 1 Processor -- start 2024-09-19T06:16:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.044+0000 7f0dec80e740 1 -- start start 2024-09-19T06:16:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0dec80e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 0x5599454b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0dec80e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454b12d0 0x5599454bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0dec80e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454c00c0 0x5599454c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0dec80e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5599454152f0 con 0x5599454b09b0 2024-09-19T06:16:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0dec80e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5599453bd270 con 0x5599454b12d0 2024-09-19T06:16:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0dec80e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5599454112b0 con 0x5599454c00c0 2024-09-19T06:16:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 0x5599454b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 0x5599454b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41634/0 (socket says 172.21.15.5:41634) 2024-09-19T06:16:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de3fff640 1 -- 172.21.15.5:0/3781835405 learned_addr learned my addr 172.21.15.5:0/3781835405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de37fe640 1 --2- 172.21.15.5:0/3781835405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454b12d0 0x5599454bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de0ff9640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0dcc0a33b0 0x7f0dcc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de2ffd640 1 --2- 172.21.15.5:0/3781835405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454c00c0 0x5599454c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de3fff640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0dcc0a33b0 0x7f0dcc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de0ff9640 1 -- 172.21.15.5:0/1326559475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f0dd40f0740 con 0x7f0de4102d00 2024-09-19T06:16:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de3fff640 1 -- 172.21.15.5:0/3781835405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454c00c0 msgr2=0x5599454c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de3fff640 1 --2- 172.21.15.5:0/3781835405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454c00c0 0x5599454c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de3fff640 1 -- 172.21.15.5:0/3781835405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454b12d0 msgr2=0x5599454bfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de3fff640 1 --2- 172.21.15.5:0/3781835405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454b12d0 0x5599454bfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de3fff640 1 -- 172.21.15.5:0/3781835405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599453bc340 con 0x5599454b09b0 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de2ffd640 1 --2- 172.21.15.5:0/3781835405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454c00c0 0x5599454c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.045+0000 7f0de3fff640 1 --2- 172.21.15.5:0/3781835405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 0x5599454b0d90 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc012810 tx=0x7f0ddc012cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d65e1ff021a26b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0dc9ffb640 1 -- 172.21.15.5:0/3781835405 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc003d40 con 0x5599454b09b0 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0dc9ffb640 1 -- 172.21.15.5:0/3781835405 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ddc012030 con 0x5599454b09b0 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0dc9ffb640 1 -- 172.21.15.5:0/3781835405 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc004400 con 0x5599454b09b0 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0dec80e740 1 -- 172.21.15.5:0/3781835405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 msgr2=0x5599454b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0de3fff640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0dcc0a33b0 0x7f0dcc0a5870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc01f000 tx=0x7f0ddc01e000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0dec80e740 1 --2- 172.21.15.5:0/3781835405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 0x5599454b0d90 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc012810 tx=0x7f0ddc012cd0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0dec80e740 1 -- 172.21.15.5:0/3781835405 shutdown_connections 2024-09-19T06:16:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0dec80e740 1 --2- 172.21.15.5:0/3781835405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454c00c0 0x5599454c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0dec80e740 1 --2- 172.21.15.5:0/3781835405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454b12d0 0x5599454bfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0dec80e740 1 --2- 172.21.15.5:0/3781835405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 0x5599454b0d90 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0dec80e740 1 -- 172.21.15.5:0/3781835405 >> 172.21.15.5:0/3781835405 conn(0x5599454ad060 msgr2=0x5599454ad440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0dec80e740 1 -- 172.21.15.5:0/3781835405 shutdown_connections 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.046+0000 7f0dec80e740 1 -- 172.21.15.5:0/3781835405 wait complete. 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0dec80e740 1 -- start start 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0dec80e740 1 Processor -- start 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0dec80e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454b09b0 0x5599454bd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0dec80e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 0x5599454bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0dec80e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454c00c0 0x5599454bdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0dec80e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5599454c2c90 con 0x5599454b12d0 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0dec80e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55994539e290 con 0x5599454c00c0 2024-09-19T06:16:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0dec80e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5599454c2e10 con 0x5599454b09b0 2024-09-19T06:16:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0dec80e740 10 client.0 _ll_register_callbacks cb 0x5599453cf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0de2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 0x5599454bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0de2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 0x5599454bd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41636/0 (socket says 172.21.15.5:41636) 2024-09-19T06:16:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0de2ffd640 1 -- 172.21.15.5:0/3471074688 learned_addr learned my addr 172.21.15.5:0/3471074688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.047+0000 7f0de37fe640 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454b09b0 0x5599454bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0de3fff640 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454c00c0 0x5599454bdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0de2ffd640 1 -- 172.21.15.5:0/3471074688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454b09b0 msgr2=0x5599454bd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0de2ffd640 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454b09b0 0x5599454bd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0de2ffd640 1 -- 172.21.15.5:0/3471074688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454c00c0 msgr2=0x5599454bdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0de2ffd640 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454c00c0 0x5599454bdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0de2ffd640 1 -- 172.21.15.5:0/3471074688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559945128210 con 0x5599454b12d0 2024-09-19T06:16:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0de3fff640 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454c00c0 0x5599454bdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0de2ffd640 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 0x5599454bd5a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800f6d0 tx=0x7f0dd800fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4d2c0117d4e029 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0dc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/3471074688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8016070 con 0x5599454b12d0 2024-09-19T06:16:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0dec80e740 1 -- 172.21.15.5:0/3471074688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ddc012030 con 0x5599454b12d0 2024-09-19T06:16:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/3471074688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f0dd800bb00 con 0x5599454b12d0 2024-09-19T06:16:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/3471074688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0dd800bd00 con 0x5599454b12d0 2024-09-19T06:16:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0dec80e740 10 client.18861 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0dec80e740 1 -- 172.21.15.5:0/3471074688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5599451f4620 con 0x5599454b12d0 2024-09-19T06:16:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.048+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/3471074688 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ddc012030 con 0x5599454b12d0 2024-09-19T06:16:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.049+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/3471074688 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f0dd80104c0 con 0x5599454b12d0 2024-09-19T06:16:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.049+0000 7f0dec80e740 10 client.18861 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.049+0000 7f0dec80e740 10 client.18861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.049+0000 7f0dec80e740 20 client.18861 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.049+0000 7f0dec80e740 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5599454d02c0 0x5599454d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.049+0000 7f0dec80e740 4 client.18861 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.049+0000 7f0dec80e740 1 -- 172.21.15.5:0/3471074688 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559945422a20 con 0x5599454d02c0 2024-09-19T06:16:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.049+0000 7f0de37fe640 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5599454d02c0 0x5599454d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.049+0000 7f0dc2ffd640 10 client.18861 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.049+0000 7f0de37fe640 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5599454d02c0 0x5599454d2700 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8897b0439b0dd5aa server_cookie=e4eb322586a96c4a in_seq=0 out_seq=0 2024-09-19T06:16:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/3471074688 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559945422a20 con 0x5599454d02c0 2024-09-19T06:16:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dc2ffd640 10 client.18861 handle_command_reply: tid=0 2024-09-19T06:16:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dec80e740 1 client.18861 shutdown 2024-09-19T06:16:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dec80e740 1 -- 172.21.15.5:0/3471074688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 msgr2=0x5599454bd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dec80e740 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 0x5599454bd5a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800f6d0 tx=0x7f0dd800fb90 comp rx=0 tx=0).stop 2024-09-19T06:16:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dec80e740 1 -- 172.21.15.5:0/3471074688 shutdown_connections 2024-09-19T06:16:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dec80e740 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5599454d02c0 0x5599454d2700 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dec80e740 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454c00c0 0x5599454bdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dc2ffd640 0 client.18861 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dec80e740 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 0x5599454bd5a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dec80e740 1 --2- 172.21.15.5:0/3471074688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454b09b0 0x5599454bd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dec80e740 1 -- 172.21.15.5:0/3471074688 >> 172.21.15.5:0/3471074688 conn(0x5599454ad060 msgr2=0x5599454b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dec80e740 1 -- 172.21.15.5:0/3471074688 shutdown_connections 2024-09-19T06:16:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dec80e740 1 -- 172.21.15.5:0/3471074688 wait complete. 2024-09-19T06:16:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.050+0000 7f0dec80e740 20 client.18861 trim_cache size 0 max 16384 2024-09-19T06:16:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0dec80e740 1 Processor -- start 2024-09-19T06:16:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0dec80e740 1 -- start start 2024-09-19T06:16:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0dec80e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 0x5599454bd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0dec80e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454b12d0 0x5599454ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0dec80e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454c00c0 0x5599454d7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0dec80e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0de41ab630 con 0x5599454b09b0 2024-09-19T06:16:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0dec80e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0de41ad230 con 0x5599454c00c0 2024-09-19T06:16:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0dec80e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0de41abd00 con 0x5599454b12d0 2024-09-19T06:16:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0de37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 0x5599454bd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0de37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 0x5599454bd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41644/0 (socket says 172.21.15.5:41644) 2024-09-19T06:16:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0de37fe640 1 -- 172.21.15.5:0/1026612701 learned_addr learned my addr 172.21.15.5:0/1026612701 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0de2ffd640 1 --2- 172.21.15.5:0/1026612701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454b12d0 0x5599454ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0de3fff640 1 --2- 172.21.15.5:0/1026612701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454c00c0 0x5599454d7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0de37fe640 1 -- 172.21.15.5:0/1026612701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454b12d0 msgr2=0x5599454ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0de37fe640 1 --2- 172.21.15.5:0/1026612701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454b12d0 0x5599454ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0de37fe640 1 -- 172.21.15.5:0/1026612701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454c00c0 msgr2=0x5599454d7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0de37fe640 1 --2- 172.21.15.5:0/1026612701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454c00c0 0x5599454d7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0de37fe640 1 -- 172.21.15.5:0/1026612701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599454bde50 con 0x5599454b09b0 2024-09-19T06:16:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.065+0000 7f0de2ffd640 1 --2- 172.21.15.5:0/1026612701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454b12d0 0x5599454ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0de37fe640 1 --2- 172.21.15.5:0/1026612701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 0x5599454bd910 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0dd402c040 tx=0x7f0dd40f2e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e630ac9472659eae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0dc97fa640 1 -- 172.21.15.5:0/1026612701 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4049040 con 0x5599454b09b0 2024-09-19T06:16:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0dc97fa640 1 -- 172.21.15.5:0/1026612701 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dd40bd100 con 0x5599454b09b0 2024-09-19T06:16:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0dc97fa640 1 -- 172.21.15.5:0/1026612701 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd40a70b0 con 0x5599454b09b0 2024-09-19T06:16:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0dec80e740 1 -- 172.21.15.5:0/1026612701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 msgr2=0x5599454bd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0dec80e740 1 --2- 172.21.15.5:0/1026612701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 0x5599454bd910 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0dd402c040 tx=0x7f0dd40f2e60 comp rx=0 tx=0).stop 2024-09-19T06:16:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0dec80e740 1 -- 172.21.15.5:0/1026612701 shutdown_connections 2024-09-19T06:16:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0dec80e740 1 --2- 172.21.15.5:0/1026612701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454c00c0 0x5599454d7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0dec80e740 1 --2- 172.21.15.5:0/1026612701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454b12d0 0x5599454ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0dec80e740 1 --2- 172.21.15.5:0/1026612701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b09b0 0x5599454bd910 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0dec80e740 1 -- 172.21.15.5:0/1026612701 >> 172.21.15.5:0/1026612701 conn(0x5599454ad060 msgr2=0x5599454ab9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0dec80e740 1 -- 172.21.15.5:0/1026612701 shutdown_connections 2024-09-19T06:16:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.066+0000 7f0dec80e740 1 -- 172.21.15.5:0/1026612701 wait complete. 2024-09-19T06:16:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0dec80e740 1 -- start start 2024-09-19T06:16:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0dec80e740 1 Processor -- start 2024-09-19T06:16:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0dec80e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454b09b0 0x5599454c8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0dec80e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 0x5599454c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0dec80e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454c00c0 0x5599454c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0dec80e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0de41acb60 con 0x5599454b12d0 2024-09-19T06:16:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0dec80e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0de41ac430 con 0x5599454b09b0 2024-09-19T06:16:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0dec80e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0de41a9630 con 0x5599454c00c0 2024-09-19T06:16:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0dec80e740 10 client.0 _ll_register_callbacks cb 0x7f0dd800d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0de2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 0x5599454c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0de2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 0x5599454c50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41652/0 (socket says 172.21.15.5:41652) 2024-09-19T06:16:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0de2ffd640 1 -- 172.21.15.5:0/1998430463 learned_addr learned my addr 172.21.15.5:0/1998430463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0de3fff640 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454c00c0 0x5599454c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0de2ffd640 1 -- 172.21.15.5:0/1998430463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454c00c0 msgr2=0x5599454c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0de2ffd640 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454c00c0 0x5599454c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0de2ffd640 1 -- 172.21.15.5:0/1998430463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454b09b0 msgr2=0x5599454c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.067+0000 7f0de37fe640 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454b09b0 0x5599454c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0de2ffd640 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454b09b0 0x5599454c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0de2ffd640 1 -- 172.21.15.5:0/1998430463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599454c5b80 con 0x5599454b12d0 2024-09-19T06:16:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0de3fff640 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454c00c0 0x5599454c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0de37fe640 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454b09b0 0x5599454c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0de2ffd640 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 0x5599454c50d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800f6a0 tx=0x7f0dd800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a01551632fe653 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/1998430463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8016040 con 0x5599454b12d0 2024-09-19T06:16:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/1998430463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f0dd800bb00 con 0x5599454b12d0 2024-09-19T06:16:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dec80e740 1 -- 172.21.15.5:0/1998430463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dd40bd100 con 0x5599454b12d0 2024-09-19T06:16:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/1998430463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0dd800de80 con 0x5599454b12d0 2024-09-19T06:16:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dec80e740 10 client.18867 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dec80e740 1 -- 172.21.15.5:0/1998430463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55994553bd00 con 0x5599454b12d0 2024-09-19T06:16:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/1998430463 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dd40bd100 con 0x5599454b12d0 2024-09-19T06:16:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/1998430463 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f0dd8035030 con 0x5599454b12d0 2024-09-19T06:16:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dec80e740 10 client.18867 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dec80e740 10 client.18867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dec80e740 20 client.18867 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dec80e740 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55994553c560 0x55994553e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dec80e740 4 client.18867 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:16:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.068+0000 7f0dec80e740 1 -- 172.21.15.5:0/1998430463 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55994541cd70 con 0x55994553c560 2024-09-19T06:16:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.069+0000 7f0de37fe640 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55994553c560 0x55994553e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.069+0000 7f0de37fe640 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55994553c560 0x55994553e980 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=233e986c3817faab server_cookie=3ac84d74fd8a026a in_seq=0 out_seq=0 2024-09-19T06:16:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.069+0000 7f0dc2ffd640 10 client.18867 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/1998430463 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55994541cd70 con 0x55994553c560 2024-09-19T06:16:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dc2ffd640 10 client.18867 handle_command_reply: tid=0 2024-09-19T06:16:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dec80e740 1 client.18867 shutdown 2024-09-19T06:16:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dec80e740 1 -- 172.21.15.5:0/1998430463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 msgr2=0x5599454c50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dec80e740 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 0x5599454c50d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800f6a0 tx=0x7f0dd800d710 comp rx=0 tx=0).stop 2024-09-19T06:16:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dec80e740 1 -- 172.21.15.5:0/1998430463 shutdown_connections 2024-09-19T06:16:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dec80e740 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55994553c560 0x55994553e980 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dec80e740 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599454c00c0 0x5599454c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dec80e740 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599454b12d0 0x5599454c50d0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dc2ffd640 0 client.18867 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dec80e740 1 --2- 172.21.15.5:0/1998430463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599454b09b0 0x5599454c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dec80e740 1 -- 172.21.15.5:0/1998430463 >> 172.21.15.5:0/1998430463 conn(0x5599454ad060 msgr2=0x5599454ab9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dec80e740 1 -- 172.21.15.5:0/1998430463 shutdown_connections 2024-09-19T06:16:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dec80e740 1 -- 172.21.15.5:0/1998430463 wait complete. 2024-09-19T06:16:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.092+0000 7f0dec80e740 20 client.18867 trim_cache size 0 max 16384 2024-09-19T06:16:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:16:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:16:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:16:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:16:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.093+0000 7f0de9a80640 1 -- 172.21.15.5:0/1326559475 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0dcc0a33b0 msgr2=0x7f0dcc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.093+0000 7f0de9a80640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0dcc0a33b0 0x7f0dcc0a5870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc01f000 tx=0x7f0ddc01e000 comp rx=0 tx=0).stop 2024-09-19T06:16:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.093+0000 7f0de9a80640 1 -- 172.21.15.5:0/1326559475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de4102d00 msgr2=0x7f0de4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.093+0000 7f0de9a80640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de4102d00 0x7f0de4198300 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4006010 tx=0x7f0dd4004270 comp rx=0 tx=0).stop 2024-09-19T06:16:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.093+0000 7f0de9a80640 1 -- 172.21.15.5:0/1326559475 shutdown_connections 2024-09-19T06:16:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.093+0000 7f0de9a80640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0dcc0a33b0 0x7f0dcc0a5870 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.093+0000 7f0de9a80640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de410c0d0 0x7f0de419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.093+0000 7f0de9a80640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de4103620 0x7f0de4198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.093+0000 7f0de9a80640 1 --2- 172.21.15.5:0/1326559475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de4102d00 0x7f0de4198300 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.093+0000 7f0de9a80640 1 -- 172.21.15.5:0/1326559475 >> 172.21.15.5:0/1326559475 conn(0x7f0de40779a0 msgr2=0x7f0de40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.093+0000 7f0de9a80640 1 -- 172.21.15.5:0/1326559475 shutdown_connections 2024-09-19T06:16:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:28.093+0000 7f0de9a80640 1 -- 172.21.15.5:0/1326559475 wait complete. 2024-09-19T06:16:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:28 smithi157 ceph-mon[35346]: pgmap v411: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-09-19T06:16:28.851 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294491 2.11 MB/sec execute 594 sec latency 1148.687 ms 2024-09-19T06:16:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:28 smithi005 ceph-mon[30955]: pgmap v411: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-09-19T06:16:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:28 smithi049 ceph-mon[34640]: pgmap v411: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-09-19T06:16:29.851 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294491 2.11 MB/sec execute 595 sec latency 2148.839 ms 2024-09-19T06:16:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:30 smithi157 ceph-mon[35346]: pgmap v412: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 876 KiB/s wr, 80 op/s 2024-09-19T06:16:30.851 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294558 2.10 MB/sec execute 596 sec latency 2767.507 ms 2024-09-19T06:16:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:30 smithi005 ceph-mon[30955]: pgmap v412: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 876 KiB/s wr, 80 op/s 2024-09-19T06:16:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:30 smithi049 ceph-mon[34640]: pgmap v412: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 876 KiB/s wr, 80 op/s 2024-09-19T06:16:31.852 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294558 2.10 MB/sec execute 597 sec latency 1134.488 ms 2024-09-19T06:16:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:32 smithi157 ceph-mon[35346]: pgmap v413: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 876 KiB/s wr, 80 op/s 2024-09-19T06:16:32.852 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294558 2.10 MB/sec execute 598 sec latency 2134.563 ms 2024-09-19T06:16:32.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:32 smithi005 ceph-mon[30955]: pgmap v413: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 876 KiB/s wr, 80 op/s 2024-09-19T06:16:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:32 smithi049 ceph-mon[34640]: pgmap v413: 97 pgs: 97 active+clean; 162 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 876 KiB/s wr, 80 op/s 2024-09-19T06:16:33.851 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294558 2.09 MB/sec execute 599 sec latency 3134.661 ms 2024-09-19T06:16:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:34 smithi157 ceph-mon[35346]: pgmap v414: 97 pgs: 97 active+clean; 163 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-09-19T06:16:34.852 INFO:tasks.workunit.client.0.smithi005.stdout: 1 cleanup 600 sec 2024-09-19T06:16:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:34 smithi005 ceph-mon[30955]: pgmap v414: 97 pgs: 97 active+clean; 163 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-09-19T06:16:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:34 smithi049 ceph-mon[34640]: pgmap v414: 97 pgs: 97 active+clean; 163 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-09-19T06:16:35.852 INFO:tasks.workunit.client.0.smithi005.stdout: 1 cleanup 601 sec 2024-09-19T06:16:36.446 INFO:tasks.workunit.client.0.smithi005.stdout: 0 cleanup 602 sec 2024-09-19T06:16:36.447 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-19T06:16:36.447 INFO:tasks.workunit.client.0.smithi005.stdout: Operation Count AvgLat MaxLat 2024-09-19T06:16:36.447 INFO:tasks.workunit.client.0.smithi005.stdout: ---------------------------------------- 2024-09-19T06:16:36.447 INFO:tasks.workunit.client.0.smithi005.stdout: NTCreateX 40009 3.711 99.228 2024-09-19T06:16:36.447 INFO:tasks.workunit.client.0.smithi005.stdout: Close 29384 0.050 81.733 2024-09-19T06:16:36.447 INFO:tasks.workunit.client.0.smithi005.stdout: Rename 1701 77.318 4755.209 2024-09-19T06:16:36.447 INFO:tasks.workunit.client.0.smithi005.stdout: Unlink 8085 3.924 65.248 2024-09-19T06:16:36.447 INFO:tasks.workunit.client.0.smithi005.stdout: Qpathinfo 36291 3.441 4747.136 2024-09-19T06:16:36.447 INFO:tasks.workunit.client.0.smithi005.stdout: Qfileinfo 6341 0.075 1.636 2024-09-19T06:16:36.447 INFO:tasks.workunit.client.0.smithi005.stdout: Qfsinfo 6680 0.314 3.907 2024-09-19T06:16:36.447 INFO:tasks.workunit.client.0.smithi005.stdout: Sfileinfo 3256 1.292 11.038 2024-09-19T06:16:36.447 INFO:tasks.workunit.client.0.smithi005.stdout: Find 14058 10.266 96.571 2024-09-19T06:16:36.448 INFO:tasks.workunit.client.0.smithi005.stdout: WriteX 19797 0.174 9.424 2024-09-19T06:16:36.448 INFO:tasks.workunit.client.0.smithi005.stdout: ReadX 63118 0.004 0.133 2024-09-19T06:16:36.448 INFO:tasks.workunit.client.0.smithi005.stdout: LockX 132 0.750 16.539 2024-09-19T06:16:36.448 INFO:tasks.workunit.client.0.smithi005.stdout: UnlockX 132 0.629 19.298 2024-09-19T06:16:36.448 INFO:tasks.workunit.client.0.smithi005.stdout: Flush 2805 2.483 33.417 2024-09-19T06:16:36.448 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-19T06:16:36.448 INFO:tasks.workunit.client.0.smithi005.stdout:Throughput 2.0928 MB/sec 1 clients 1 procs max_latency=4755.220 ms 2024-09-19T06:16:36.448 INFO:tasks.workunit.client.0.smithi005.stderr:+ dbench 10 2024-09-19T06:16:36.449 INFO:tasks.workunit.client.0.smithi005.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-19T06:16:36.449 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-19T06:16:36.449 INFO:tasks.workunit.client.0.smithi005.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-19T06:16:36.449 INFO:tasks.workunit.client.0.smithi005.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-19T06:16:36.548 INFO:tasks.workunit.client.0.smithi005.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-19T06:16:36.548 INFO:tasks.workunit.client.0.smithi005.stdout:releasing clients 2024-09-19T06:16:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:36 smithi157 ceph-mon[35346]: pgmap v415: 97 pgs: 97 active+clean; 163 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 851 KiB/s wr, 86 op/s 2024-09-19T06:16:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:36 smithi005 ceph-mon[30955]: pgmap v415: 97 pgs: 97 active+clean; 163 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 851 KiB/s wr, 86 op/s 2024-09-19T06:16:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:36 smithi049 ceph-mon[34640]: pgmap v415: 97 pgs: 97 active+clean; 163 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 851 KiB/s wr, 86 op/s 2024-09-19T06:16:37.548 INFO:tasks.workunit.client.0.smithi005.stdout: 10 279 104.18 MB/sec warmup 1 sec latency 91.557 ms 2024-09-19T06:16:37.638 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:16:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:38.548 INFO:tasks.workunit.client.0.smithi005.stdout: 10 501 87.08 MB/sec warmup 2 sec latency 90.677 ms 2024-09-19T06:16:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.579+0000 7f84af233640 1 -- 172.21.15.5:0/3181642353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a810d530 msgr2=0x7f84a81150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.579+0000 7f84af233640 1 --2- 172.21.15.5:0/3181642353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a810d530 0x7f84a81150b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009ca0 tx=0x7f84a002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.579+0000 7f84af233640 1 -- 172.21.15.5:0/3181642353 shutdown_connections 2024-09-19T06:16:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.579+0000 7f84af233640 1 --2- 172.21.15.5:0/3181642353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a810d530 0x7f84a81150b0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.579+0000 7f84af233640 1 --2- 172.21.15.5:0/3181642353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a8108e00 0x7f84a810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.579+0000 7f84af233640 1 --2- 172.21.15.5:0/3181642353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a8108450 0x7f84a8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.579+0000 7f84af233640 1 -- 172.21.15.5:0/3181642353 >> 172.21.15.5:0/3181642353 conn(0x7f84a8077ba0 msgr2=0x7f84a8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.579+0000 7f84af233640 1 -- 172.21.15.5:0/3181642353 shutdown_connections 2024-09-19T06:16:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.580+0000 7f84af233640 1 -- 172.21.15.5:0/3181642353 wait complete. 2024-09-19T06:16:38.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.580+0000 7f84af233640 1 Processor -- start 2024-09-19T06:16:38.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.581+0000 7f84af233640 1 -- start start 2024-09-19T06:16:38.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.581+0000 7f84af233640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a8108450 0x7f84a81a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.581+0000 7f84af233640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8108e00 0x7f84a81a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.581+0000 7f84af233640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a810d530 0x7f84a819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.581+0000 7f84af233640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81183f0 con 0x7f84a8108e00 2024-09-19T06:16:38.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.581+0000 7f84af233640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f84a8118270 con 0x7f84a8108450 2024-09-19T06:16:38.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.581+0000 7f84af233640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f84a8118570 con 0x7f84a810d530 2024-09-19T06:16:38.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.581+0000 7f84a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8108e00 0x7f84a81a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.581+0000 7f84a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8108e00 0x7f84a81a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55010/0 (socket says 172.21.15.5:55010) 2024-09-19T06:16:38.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.581+0000 7f84a7fff640 1 -- 172.21.15.5:0/484273355 learned_addr learned my addr 172.21.15.5:0/484273355 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:38.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.581+0000 7f84acfcf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a8108450 0x7f84a81a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.581+0000 7f84a7fff640 1 -- 172.21.15.5:0/484273355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a810d530 msgr2=0x7f84a819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.582+0000 7f84a7fff640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a810d530 0x7f84a819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.582+0000 7f84a7fff640 1 -- 172.21.15.5:0/484273355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a8108450 msgr2=0x7f84a81a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.582+0000 7f84a7fff640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a8108450 0x7f84a81a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.582+0000 7f84a7fff640 1 -- 172.21.15.5:0/484273355 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a819ef70 con 0x7f84a8108e00 2024-09-19T06:16:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.582+0000 7f84acfcf640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a8108450 0x7f84a81a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.582+0000 7f84a7fff640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8108e00 0x7f84a81a4b70 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f849400d950 tx=0x7f849400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.582+0000 7f84a5ffb640 1 -- 172.21.15.5:0/484273355 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494017030 con 0x7f84a8108e00 2024-09-19T06:16:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.582+0000 7f84af233640 1 -- 172.21.15.5:0/484273355 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a819f260 con 0x7f84a8108e00 2024-09-19T06:16:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.583+0000 7f84a5ffb640 1 -- 172.21.15.5:0/484273355 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8494004020 con 0x7f84a8108e00 2024-09-19T06:16:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.583+0000 7f84a5ffb640 1 -- 172.21.15.5:0/484273355 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84940047a0 con 0x7f84a8108e00 2024-09-19T06:16:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.583+0000 7f84af233640 1 -- 172.21.15.5:0/484273355 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a81ab5b0 con 0x7f84a8108e00 2024-09-19T06:16:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.586+0000 7f84a5ffb640 1 -- 172.21.15.5:0/484273355 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f84940041d0 con 0x7f84a8108e00 2024-09-19T06:16:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.587+0000 7f84a5ffb640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f84980a33b0 0x7f84980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.587+0000 7f84acfcf640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f84980a33b0 0x7f84980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.587+0000 7f84a5ffb640 1 -- 172.21.15.5:0/484273355 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f84940cf7c0 con 0x7f84a8108e00 2024-09-19T06:16:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84b160c740 1 Processor -- start 2024-09-19T06:16:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84b160c740 1 -- start start 2024-09-19T06:16:38.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84b160c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd384a0 0x55750fd38880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84b160c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd38dc0 0x55750fd43140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84b160c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd45a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84b160c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55750fc9e2e0 con 0x55750fd43680 2024-09-19T06:16:38.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84b160c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55750fc45ce0 con 0x55750fd384a0 2024-09-19T06:16:38.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84b160c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55750fc9a2a0 con 0x55750fd38dc0 2024-09-19T06:16:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84acfcf640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f84980a33b0 0x7f84980a5870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f849c009fc0 tx=0x7f849c005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84ad7d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd45a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84acfcf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd384a0 0x55750fd38880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84ad7d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd45a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55018/0 (socket says 172.21.15.5:55018) 2024-09-19T06:16:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84ad7d0640 1 -- 172.21.15.5:0/3816554477 learned_addr learned my addr 172.21.15.5:0/3816554477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.588+0000 7f84a7fff640 1 --2- 172.21.15.5:0/3816554477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd38dc0 0x55750fd43140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.589+0000 7f84acfcf640 1 -- 172.21.15.5:0/3816554477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd38dc0 msgr2=0x55750fd43140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.589+0000 7f84acfcf640 1 --2- 172.21.15.5:0/3816554477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd38dc0 0x55750fd43140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.589+0000 7f84acfcf640 1 -- 172.21.15.5:0/3816554477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 msgr2=0x55750fd45a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.589+0000 7f84acfcf640 1 --2- 172.21.15.5:0/3816554477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd45a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.589+0000 7f84acfcf640 1 -- 172.21.15.5:0/3816554477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55750fc2a440 con 0x55750fd384a0 2024-09-19T06:16:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.589+0000 7f84ad7d0640 1 --2- 172.21.15.5:0/3816554477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd45a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.589+0000 7f84acfcf640 1 --2- 172.21.15.5:0/3816554477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd384a0 0x55750fd38880 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f849c011ca0 tx=0x7f849c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ee72e33bb9b38a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.589+0000 7f8492ffd640 1 -- 172.21.15.5:0/3816554477 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c00f480 con 0x55750fd384a0 2024-09-19T06:16:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.589+0000 7f8492ffd640 1 -- 172.21.15.5:0/3816554477 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f849c00fad0 con 0x55750fd384a0 2024-09-19T06:16:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.589+0000 7f8492ffd640 1 -- 172.21.15.5:0/3816554477 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c01ca70 con 0x55750fd384a0 2024-09-19T06:16:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.590+0000 7f84b160c740 1 -- 172.21.15.5:0/3816554477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd384a0 msgr2=0x55750fd38880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.590+0000 7f84b160c740 1 --2- 172.21.15.5:0/3816554477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd384a0 0x55750fd38880 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f849c011ca0 tx=0x7f849c014000 comp rx=0 tx=0).stop 2024-09-19T06:16:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.590+0000 7f84b160c740 1 -- 172.21.15.5:0/3816554477 shutdown_connections 2024-09-19T06:16:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.590+0000 7f84b160c740 1 --2- 172.21.15.5:0/3816554477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd45a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.590+0000 7f84b160c740 1 --2- 172.21.15.5:0/3816554477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd38dc0 0x55750fd43140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.590+0000 7f84b160c740 1 --2- 172.21.15.5:0/3816554477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd384a0 0x55750fd38880 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.590+0000 7f84b160c740 1 -- 172.21.15.5:0/3816554477 >> 172.21.15.5:0/3816554477 conn(0x55750fd34a30 msgr2=0x55750fd34e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.590+0000 7f84b160c740 1 -- 172.21.15.5:0/3816554477 shutdown_connections 2024-09-19T06:16:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.590+0000 7f84b160c740 1 -- 172.21.15.5:0/3816554477 wait complete. 2024-09-19T06:16:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.591+0000 7f84b160c740 1 -- start start 2024-09-19T06:16:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.591+0000 7f84b160c740 1 Processor -- start 2024-09-19T06:16:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.591+0000 7f84b160c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd384a0 0x55750fd42f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84b160c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd38dc0 0x55750fd401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84acfcf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd384a0 0x55750fd42f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84acfcf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd384a0 0x55750fd42f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55032/0 (socket says 172.21.15.5:55032) 2024-09-19T06:16:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84acfcf640 1 -- 172.21.15.5:0/3038615650 learned_addr learned my addr 172.21.15.5:0/3038615650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84b160c740 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd43680 0x55750fd40720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84b160c740 1 -- 172.21.15.5:0/3038615650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55750fd462a0 con 0x55750fd384a0 2024-09-19T06:16:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84b160c740 1 -- 172.21.15.5:0/3038615650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55750fc59050 con 0x55750fd43680 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84b160c740 1 -- 172.21.15.5:0/3038615650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55750fd46420 con 0x55750fd38dc0 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84ad7d0640 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd43680 0x55750fd40720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84b160c740 10 client.0 _ll_register_callbacks cb 0x55750fc99160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84a7fff640 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd38dc0 0x55750fd401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84acfcf640 1 -- 172.21.15.5:0/3038615650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd38dc0 msgr2=0x55750fd401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84acfcf640 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd38dc0 0x55750fd401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84acfcf640 1 -- 172.21.15.5:0/3038615650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd43680 msgr2=0x55750fd40720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84acfcf640 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd43680 0x55750fd40720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84acfcf640 1 -- 172.21.15.5:0/3038615650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55750fa0f3c0 con 0x55750fd384a0 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.592+0000 7f84a7fff640 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd38dc0 0x55750fd401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.593+0000 7f84acfcf640 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd384a0 0x55750fd42f30 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f849c01c690 tx=0x7f849c003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bc776e039b3c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.593+0000 7f8487fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.593+0000 7f8487fff640 1 -- 172.21.15.5:0/3038615650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c00fd40 con 0x55750fd384a0 2024-09-19T06:16:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.593+0000 7f8487fff640 1 -- 172.21.15.5:0/3038615650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f849c00f480 con 0x55750fd384a0 2024-09-19T06:16:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.593+0000 7f84b160c740 1 -- 172.21.15.5:0/3038615650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f849c00fad0 con 0x55750fd384a0 2024-09-19T06:16:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.593+0000 7f8487fff640 1 -- 172.21.15.5:0/3038615650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f849c00f680 con 0x55750fd384a0 2024-09-19T06:16:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.593+0000 7f84b160c740 10 client.18879 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.593+0000 7f84b160c740 1 -- 172.21.15.5:0/3038615650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55750fd51b80 con 0x55750fd384a0 2024-09-19T06:16:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.594+0000 7f8487fff640 1 -- 172.21.15.5:0/3038615650 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f849c00fad0 con 0x55750fd384a0 2024-09-19T06:16:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.594+0000 7f8487fff640 1 -- 172.21.15.5:0/3038615650 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f849c01a070 con 0x55750fd384a0 2024-09-19T06:16:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.594+0000 7f84b160c740 10 client.18879 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.594+0000 7f84b160c740 10 client.18879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.594+0000 7f84b160c740 20 client.18879 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.594+0000 7f84b160c740 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55750fd52fe0 0x55750fd553d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.594+0000 7f84b160c740 4 client.18879 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.594+0000 7f84b160c740 1 -- 172.21.15.5:0/3038615650 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55750fcab850 con 0x55750fd52fe0 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.594+0000 7f84a7fff640 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55750fd52fe0 0x55750fd553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.595+0000 7f8487fff640 10 client.18879 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.595+0000 7f84a7fff640 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55750fd52fe0 0x55750fd553d0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22660e0b2ab056c3 server_cookie=b39984d8f95e3724 in_seq=0 out_seq=0 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.596+0000 7f8487fff640 1 -- 172.21.15.5:0/3038615650 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55750fcab850 con 0x55750fd52fe0 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.596+0000 7f8487fff640 10 client.18879 handle_command_reply: tid=0 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.596+0000 7f84b160c740 1 client.18879 shutdown 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.596+0000 7f84b160c740 1 -- 172.21.15.5:0/3038615650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd384a0 msgr2=0x55750fd42f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.596+0000 7f84b160c740 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd384a0 0x55750fd42f30 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f849c01c690 tx=0x7f849c003e40 comp rx=0 tx=0).stop 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.596+0000 7f84b160c740 1 -- 172.21.15.5:0/3038615650 shutdown_connections 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.596+0000 7f84b160c740 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55750fd52fe0 0x55750fd553d0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.596+0000 7f8487fff640 0 client.18879 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.596+0000 7f84b160c740 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd43680 0x55750fd40720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.596+0000 7f84b160c740 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd38dc0 0x55750fd401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.596+0000 7f84b160c740 1 --2- 172.21.15.5:0/3038615650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd384a0 0x55750fd42f30 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.596+0000 7f84b160c740 1 -- 172.21.15.5:0/3038615650 >> 172.21.15.5:0/3038615650 conn(0x55750fd34a30 msgr2=0x55750fd3a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.597+0000 7f84b160c740 1 -- 172.21.15.5:0/3038615650 shutdown_connections 2024-09-19T06:16:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.597+0000 7f84b160c740 1 -- 172.21.15.5:0/3038615650 wait complete. 2024-09-19T06:16:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.597+0000 7f84b160c740 20 client.18879 trim_cache size 0 max 16384 2024-09-19T06:16:38.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.619+0000 7f84b160c740 1 Processor -- start 2024-09-19T06:16:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.619+0000 7f84b160c740 1 -- start start 2024-09-19T06:16:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.619+0000 7f84b160c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd384a0 0x55750fd5aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.619+0000 7f84b160c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 0x55750fd5b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.619+0000 7f84b160c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd4b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.619+0000 7f84b160c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81b3f40 con 0x55750fd43680 2024-09-19T06:16:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.619+0000 7f84b160c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81b5b40 con 0x55750fd38dc0 2024-09-19T06:16:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.619+0000 7f84b160c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81b4610 con 0x55750fd384a0 2024-09-19T06:16:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.619+0000 7f84ad7d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd4b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.619+0000 7f84a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 0x55750fd5b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.620+0000 7f84a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 0x55750fd5b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60138/0 (socket says 172.21.15.5:60138) 2024-09-19T06:16:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.620+0000 7f84a7fff640 1 -- 172.21.15.5:0/3627243972 learned_addr learned my addr 172.21.15.5:0/3627243972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.620+0000 7f84acfcf640 1 --2- 172.21.15.5:0/3627243972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd384a0 0x55750fd5aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.620+0000 7f84a7fff640 1 -- 172.21.15.5:0/3627243972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd384a0 msgr2=0x55750fd5aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.620+0000 7f84a7fff640 1 --2- 172.21.15.5:0/3627243972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd384a0 0x55750fd5aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.620+0000 7f84a7fff640 1 -- 172.21.15.5:0/3627243972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 msgr2=0x55750fd4b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.620+0000 7f84a7fff640 1 --2- 172.21.15.5:0/3627243972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd4b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.620+0000 7f84a7fff640 1 -- 172.21.15.5:0/3627243972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55750fd3e730 con 0x55750fd38dc0 2024-09-19T06:16:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.620+0000 7f84ad7d0640 1 --2- 172.21.15.5:0/3627243972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd4b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.620+0000 7f84a7fff640 1 --2- 172.21.15.5:0/3627243972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 0x55750fd5b000 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f84940983f0 tx=0x7f8494005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3815c01957c2240 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.620+0000 7f84927fc640 1 -- 172.21.15.5:0/3627243972 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84940b3070 con 0x55750fd38dc0 2024-09-19T06:16:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.620+0000 7f84927fc640 1 -- 172.21.15.5:0/3627243972 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8494014040 con 0x55750fd38dc0 2024-09-19T06:16:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.621+0000 7f84927fc640 1 -- 172.21.15.5:0/3627243972 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84940cd110 con 0x55750fd38dc0 2024-09-19T06:16:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.621+0000 7f84b160c740 1 -- 172.21.15.5:0/3627243972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 msgr2=0x55750fd5b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.621+0000 7f84b160c740 1 --2- 172.21.15.5:0/3627243972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 0x55750fd5b000 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f84940983f0 tx=0x7f8494005bf0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.622+0000 7f84b160c740 1 -- 172.21.15.5:0/3627243972 shutdown_connections 2024-09-19T06:16:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.622+0000 7f84b160c740 1 --2- 172.21.15.5:0/3627243972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd4b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.622+0000 7f84b160c740 1 --2- 172.21.15.5:0/3627243972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 0x55750fd5b000 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.622+0000 7f84b160c740 1 --2- 172.21.15.5:0/3627243972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd384a0 0x55750fd5aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.622+0000 7f84b160c740 1 -- 172.21.15.5:0/3627243972 >> 172.21.15.5:0/3627243972 conn(0x55750fd34a30 msgr2=0x55750fd3f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.622+0000 7f84b160c740 1 -- 172.21.15.5:0/3627243972 shutdown_connections 2024-09-19T06:16:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.622+0000 7f84b160c740 1 -- 172.21.15.5:0/3627243972 wait complete. 2024-09-19T06:16:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.622+0000 7f84b160c740 1 -- start start 2024-09-19T06:16:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.622+0000 7f84b160c740 1 Processor -- start 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84b160c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd384a0 0x55750fd48e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84b160c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 0x55750fd493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84b160c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd4b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84b160c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81b5470 con 0x55750fd43680 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84b160c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81b4d40 con 0x55750fd38dc0 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84b160c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81b1f40 con 0x55750fd384a0 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 0x55750fd493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 0x55750fd493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60148/0 (socket says 172.21.15.5:60148) 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84a7fff640 1 -- 172.21.15.5:0/210903310 learned_addr learned my addr 172.21.15.5:0/210903310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84ad7d0640 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd4b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84b160c740 10 client.0 _ll_register_callbacks cb 0x7f84a806b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84acfcf640 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd384a0 0x55750fd48e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84a7fff640 1 -- 172.21.15.5:0/210903310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd384a0 msgr2=0x55750fd48e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84a7fff640 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd384a0 0x55750fd48e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84a7fff640 1 -- 172.21.15.5:0/210903310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 msgr2=0x55750fd4b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84a7fff640 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd4b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.623+0000 7f84a7fff640 1 -- 172.21.15.5:0/210903310 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55750fd4bb80 con 0x55750fd38dc0 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.624+0000 7f84a7fff640 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 0x55750fd493c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8494097e80 tx=0x7f8494004540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4edb7c4c1c0e2974 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.624+0000 7f8487fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.624+0000 7f8487fff640 1 -- 172.21.15.5:0/210903310 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84940b3040 con 0x55750fd38dc0 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.624+0000 7f84b160c740 1 -- 172.21.15.5:0/210903310 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8494014040 con 0x55750fd38dc0 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.624+0000 7f8487fff640 1 -- 172.21.15.5:0/210903310 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f8494098090 con 0x55750fd38dc0 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.624+0000 7f8487fff640 1 -- 172.21.15.5:0/210903310 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f84940a4690 con 0x55750fd38dc0 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.624+0000 7f84b160c740 10 client.28286 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.624+0000 7f84b160c740 1 -- 172.21.15.5:0/210903310 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55750fdbd610 con 0x55750fd38dc0 2024-09-19T06:16:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.625+0000 7f8487fff640 1 -- 172.21.15.5:0/210903310 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8494065070 con 0x55750fd38dc0 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.625+0000 7f8487fff640 1 -- 172.21.15.5:0/210903310 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f8494060ce0 con 0x55750fd38dc0 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.625+0000 7f84b160c740 10 client.28286 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.625+0000 7f84b160c740 10 client.28286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.625+0000 7f84b160c740 20 client.28286 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.625+0000 7f84b160c740 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55750fd52fe0 0x55750fdbfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.625+0000 7f84b160c740 4 client.28286 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.625+0000 7f84b160c740 1 -- 172.21.15.5:0/210903310 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8494014040 con 0x55750fd52fe0 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.625+0000 7f84acfcf640 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55750fd52fe0 0x55750fdbfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.625+0000 7f8487fff640 10 client.28286 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f84acfcf640 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55750fd52fe0 0x55750fdbfc70 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90abf9d2d28f8fef server_cookie=4415f6fc6bbda547 in_seq=0 out_seq=0 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f8487fff640 1 -- 172.21.15.5:0/210903310 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f8494014040 con 0x55750fd52fe0 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f8487fff640 10 client.28286 handle_command_reply: tid=0 2024-09-19T06:16:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f84b160c740 1 client.28286 shutdown 2024-09-19T06:16:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f84b160c740 1 -- 172.21.15.5:0/210903310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 msgr2=0x55750fd493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f84b160c740 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 0x55750fd493c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8494097e80 tx=0x7f8494004540 comp rx=0 tx=0).stop 2024-09-19T06:16:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f84b160c740 1 -- 172.21.15.5:0/210903310 shutdown_connections 2024-09-19T06:16:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f84b160c740 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55750fd52fe0 0x55750fdbfc70 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f84b160c740 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750fd43680 0x55750fd4b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f84b160c740 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750fd38dc0 0x55750fd493c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f84b160c740 1 --2- 172.21.15.5:0/210903310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750fd384a0 0x55750fd48e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f84b160c740 1 -- 172.21.15.5:0/210903310 >> 172.21.15.5:0/210903310 conn(0x55750fd34a30 msgr2=0x55750fd332a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.626+0000 7f8487fff640 0 client.28286 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.627+0000 7f84b160c740 1 -- 172.21.15.5:0/210903310 shutdown_connections 2024-09-19T06:16:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.627+0000 7f84b160c740 1 -- 172.21.15.5:0/210903310 wait complete. 2024-09-19T06:16:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.627+0000 7f84b160c740 20 client.28286 trim_cache size 0 max 16384 2024-09-19T06:16:38.634 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:16:38.634 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:16:38.634 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:16:38.634 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:16:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.628+0000 7f84af233640 1 -- 172.21.15.5:0/484273355 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f84980a33b0 msgr2=0x7f84980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.628+0000 7f84af233640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f84980a33b0 0x7f84980a5870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f849c009fc0 tx=0x7f849c005f90 comp rx=0 tx=0).stop 2024-09-19T06:16:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.628+0000 7f84af233640 1 -- 172.21.15.5:0/484273355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8108e00 msgr2=0x7f84a81a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.628+0000 7f84af233640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8108e00 0x7f84a81a4b70 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f849400d950 tx=0x7f849400de10 comp rx=0 tx=0).stop 2024-09-19T06:16:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.628+0000 7f84af233640 1 -- 172.21.15.5:0/484273355 shutdown_connections 2024-09-19T06:16:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.628+0000 7f84af233640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f84980a33b0 0x7f84980a5870 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.628+0000 7f84af233640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a810d530 0x7f84a819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.628+0000 7f84af233640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8108e00 0x7f84a81a4b70 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.628+0000 7f84af233640 1 --2- 172.21.15.5:0/484273355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a8108450 0x7f84a81a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:38.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.628+0000 7f84af233640 1 -- 172.21.15.5:0/484273355 >> 172.21.15.5:0/484273355 conn(0x7f84a8077ba0 msgr2=0x7f84a8078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:38.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.629+0000 7f84af233640 1 -- 172.21.15.5:0/484273355 shutdown_connections 2024-09-19T06:16:38.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:38.629+0000 7f84af233640 1 -- 172.21.15.5:0/484273355 wait complete. 2024-09-19T06:16:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:38 smithi157 ceph-mon[35346]: pgmap v416: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 2.2 MiB/s wr, 111 op/s 2024-09-19T06:16:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:38 smithi005 ceph-mon[30955]: pgmap v416: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 2.2 MiB/s wr, 111 op/s 2024-09-19T06:16:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:38 smithi049 ceph-mon[34640]: pgmap v416: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 2.2 MiB/s wr, 111 op/s 2024-09-19T06:16:39.443 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:16:39.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:16:39.548 INFO:tasks.workunit.client.0.smithi005.stdout: 10 685 69.35 MB/sec warmup 3 sec latency 104.726 ms 2024-09-19T06:16:39.812 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:40.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.349+0000 7ff7c9800640 1 -- 172.21.15.5:0/1130966603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7c4101a00 msgr2=0x7ff7c4101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:40.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.349+0000 7ff7c9800640 1 --2- 172.21.15.5:0/1130966603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7c4101a00 0x7ff7c4101e60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009930 tx=0x7ff7b802a420 comp rx=0 tx=0).stop 2024-09-19T06:16:40.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.349+0000 7ff7c9800640 1 -- 172.21.15.5:0/1130966603 shutdown_connections 2024-09-19T06:16:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.349+0000 7ff7c9800640 1 --2- 172.21.15.5:0/1130966603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c41023a0 0x7ff7c410e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.349+0000 7ff7c9800640 1 --2- 172.21.15.5:0/1130966603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7c4101a00 0x7ff7c4101e60 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.349+0000 7ff7c9800640 1 --2- 172.21.15.5:0/1130966603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c4107a00 0x7ff7c4107de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.349+0000 7ff7c9800640 1 -- 172.21.15.5:0/1130966603 >> 172.21.15.5:0/1130966603 conn(0x7ff7c40fd6e0 msgr2=0x7ff7c40ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.349+0000 7ff7c9800640 1 -- 172.21.15.5:0/1130966603 shutdown_connections 2024-09-19T06:16:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.349+0000 7ff7c9800640 1 -- 172.21.15.5:0/1130966603 wait complete. 2024-09-19T06:16:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.350+0000 7ff7c9800640 1 Processor -- start 2024-09-19T06:16:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c9800640 1 -- start start 2024-09-19T06:16:40.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7c4101a00 0x7ff7c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:40.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c9800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c41023a0 0x7ff7c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:40.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c9800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c4107a00 0x7ff7c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c9800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c4111c70 con 0x7ff7c4101a00 2024-09-19T06:16:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c9800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c4111af0 con 0x7ff7c4107a00 2024-09-19T06:16:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c9800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c4111df0 con 0x7ff7c41023a0 2024-09-19T06:16:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7c4101a00 0x7ff7c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7c4101a00 0x7ff7c40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55066/0 (socket says 172.21.15.5:55066) 2024-09-19T06:16:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c2ffd640 1 -- 172.21.15.5:0/1875102677 learned_addr learned my addr 172.21.15.5:0/1875102677 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c37fe640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c4107a00 0x7ff7c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c27fc640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c41023a0 0x7ff7c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c2ffd640 1 -- 172.21.15.5:0/1875102677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c41023a0 msgr2=0x7ff7c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c2ffd640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c41023a0 0x7ff7c4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.351+0000 7ff7c2ffd640 1 -- 172.21.15.5:0/1875102677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c4107a00 msgr2=0x7ff7c4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.352+0000 7ff7c2ffd640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c4107a00 0x7ff7c4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.352+0000 7ff7c2ffd640 1 -- 172.21.15.5:0/1875102677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c41a66f0 con 0x7ff7c4101a00 2024-09-19T06:16:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.352+0000 7ff7c37fe640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c4107a00 0x7ff7c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:16:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.352+0000 7ff7c2ffd640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7c4101a00 0x7ff7c40744f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400e9f0 tx=0x7ff7b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.352+0000 7ff7abfff640 1 -- 172.21.15.5:0/1875102677 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b400cd80 con 0x7ff7c4101a00 2024-09-19T06:16:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.352+0000 7ff7abfff640 1 -- 172.21.15.5:0/1875102677 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7b40108e0 con 0x7ff7c4101a00 2024-09-19T06:16:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.352+0000 7ff7c9800640 1 -- 172.21.15.5:0/1875102677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c41a69e0 con 0x7ff7c4101a00 2024-09-19T06:16:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.352+0000 7ff7abfff640 1 -- 172.21.15.5:0/1875102677 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4018920 con 0x7ff7c4101a00 2024-09-19T06:16:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.352+0000 7ff7c9800640 1 -- 172.21.15.5:0/1875102677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c41a6f40 con 0x7ff7c4101a00 2024-09-19T06:16:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.353+0000 7ff7c9800640 1 -- 172.21.15.5:0/1875102677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c4101e60 con 0x7ff7c4101a00 2024-09-19T06:16:40.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.356+0000 7ff7abfff640 1 -- 172.21.15.5:0/1875102677 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff7b4010a90 con 0x7ff7c4101a00 2024-09-19T06:16:40.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.357+0000 7ff7abfff640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7ac0a3430 0x7ff7ac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:40.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.357+0000 7ff7abfff640 1 -- 172.21.15.5:0/1875102677 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7ff7b40d0510 con 0x7ff7c4101a00 2024-09-19T06:16:40.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.357+0000 7ff7c27fc640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7ac0a3430 0x7ff7ac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:40.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.358+0000 7ff7c27fc640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7ac0a3430 0x7ff7ac0a58f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8000c00 tx=0x7ff7b8032ba0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:40.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.360+0000 7ff7abfff640 1 -- 172.21.15.5:0/1875102677 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff7b409ced0 con 0x7ff7c4101a00 2024-09-19T06:16:40.548 INFO:tasks.workunit.client.0.smithi005.stdout: 10 842 53.85 MB/sec warmup 4 sec latency 111.413 ms 2024-09-19T06:16:40.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:40 smithi005 ceph-mon[30955]: pgmap v417: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.6 MiB/s wr, 51 op/s 2024-09-19T06:16:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.670+0000 7ff7c9800640 1 -- 172.21.15.5:0/1875102677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7c4109f90 con 0x7ff7c4101a00 2024-09-19T06:16:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.671+0000 7ff7abfff640 1 -- 172.21.15.5:0/1875102677 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11965 (secure 0 0 0) 0x7ff7b409c7d0 con 0x7ff7c4101a00 2024-09-19T06:16:40.672 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:16:40.673 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":39,"btime":"2024-09-19T06:14:16:386479+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:14:15.452508+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:16:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 39 2024-09-19T06:16:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.673+0000 7ff7c9800640 1 -- 172.21.15.5:0/1875102677 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7ac0a3430 msgr2=0x7ff7ac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.673+0000 7ff7c9800640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7ac0a3430 0x7ff7ac0a58f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8000c00 tx=0x7ff7b8032ba0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.673+0000 7ff7c9800640 1 -- 172.21.15.5:0/1875102677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7c4101a00 msgr2=0x7ff7c40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.673+0000 7ff7c9800640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7c4101a00 0x7ff7c40744f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400e9f0 tx=0x7ff7b400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.673+0000 7ff7c9800640 1 -- 172.21.15.5:0/1875102677 shutdown_connections 2024-09-19T06:16:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.673+0000 7ff7c9800640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c4107a00 0x7ff7c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.673+0000 7ff7c9800640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7ac0a3430 0x7ff7ac0a58f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.673+0000 7ff7c9800640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c41023a0 0x7ff7c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.673+0000 7ff7c9800640 1 --2- 172.21.15.5:0/1875102677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7c4101a00 0x7ff7c40744f0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:40.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.673+0000 7ff7c9800640 1 -- 172.21.15.5:0/1875102677 >> 172.21.15.5:0/1875102677 conn(0x7ff7c40fd6e0 msgr2=0x7ff7c40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:40.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.673+0000 7ff7c9800640 1 -- 172.21.15.5:0/1875102677 shutdown_connections 2024-09-19T06:16:40.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:40.674+0000 7ff7c9800640 1 -- 172.21.15.5:0/1875102677 wait complete. 2024-09-19T06:16:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:40 smithi157 ceph-mon[35346]: pgmap v417: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.6 MiB/s wr, 51 op/s 2024-09-19T06:16:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:40 smithi049 ceph-mon[34640]: pgmap v417: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.6 MiB/s wr, 51 op/s 2024-09-19T06:16:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.201+0000 7f634e4b7640 1 -- 172.21.15.5:0/3033776494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63481069b0 msgr2=0x7f6348106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.201+0000 7f634e4b7640 1 --2- 172.21.15.5:0/3033776494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63481069b0 0x7f6348106d90 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f6338009930 tx=0x7f633802a420 comp rx=0 tx=0).stop 2024-09-19T06:16:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.201+0000 7f634e4b7640 1 -- 172.21.15.5:0/3033776494 shutdown_connections 2024-09-19T06:16:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.201+0000 7f634e4b7640 1 --2- 172.21.15.5:0/3033776494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6348101340 0x7f634810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.201+0000 7f634e4b7640 1 --2- 172.21.15.5:0/3033776494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63481009a0 0x7f6348100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.201+0000 7f634e4b7640 1 --2- 172.21.15.5:0/3033776494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63481069b0 0x7f6348106d90 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.201+0000 7f634e4b7640 1 -- 172.21.15.5:0/3033776494 >> 172.21.15.5:0/3033776494 conn(0x7f63480779a0 msgr2=0x7f63480fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.202+0000 7f634e4b7640 1 -- 172.21.15.5:0/3033776494 shutdown_connections 2024-09-19T06:16:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.202+0000 7f634e4b7640 1 -- 172.21.15.5:0/3033776494 wait complete. 2024-09-19T06:16:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.202+0000 7f634e4b7640 1 Processor -- start 2024-09-19T06:16:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.202+0000 7f634e4b7640 1 -- start start 2024-09-19T06:16:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f634e4b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63481009a0 0x7f6348102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f634e4b7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348101340 0x7f6348102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f634e4b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63481069b0 0x7f6348102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f634e4b7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6348110ca0 con 0x7f63481069b0 2024-09-19T06:16:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f634e4b7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6348110b20 con 0x7f6348101340 2024-09-19T06:16:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f634e4b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6348110e20 con 0x7f63481009a0 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f63477fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348101340 0x7f6348102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f63477fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348101340 0x7f6348102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60180/0 (socket says 172.21.15.5:60180) 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f63477fe640 1 -- 172.21.15.5:0/2468155416 learned_addr learned my addr 172.21.15.5:0/2468155416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f634ca54640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63481069b0 0x7f6348102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f63477fe640 1 -- 172.21.15.5:0/2468155416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63481009a0 msgr2=0x7f6348102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f63477fe640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63481009a0 0x7f6348102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f63477fe640 1 -- 172.21.15.5:0/2468155416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63481069b0 msgr2=0x7f6348102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f63477fe640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63481069b0 0x7f6348102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.203+0000 7f63477fe640 1 -- 172.21.15.5:0/2468155416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63481a6850 con 0x7f6348101340 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.204+0000 7f63477fe640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348101340 0x7f6348102ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f633c00e9f0 tx=0x7f633c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.204+0000 7f63457fa640 1 -- 172.21.15.5:0/2468155416 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c00cd80 con 0x7f6348101340 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.204+0000 7f634e4b7640 1 -- 172.21.15.5:0/2468155416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63481a6b40 con 0x7f6348101340 2024-09-19T06:16:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.204+0000 7f63457fa640 1 -- 172.21.15.5:0/2468155416 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f633c0108e0 con 0x7f6348101340 2024-09-19T06:16:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.204+0000 7f634e4b7640 1 -- 172.21.15.5:0/2468155416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63481a7090 con 0x7f6348101340 2024-09-19T06:16:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.204+0000 7f63457fa640 1 -- 172.21.15.5:0/2468155416 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c0188c0 con 0x7f6348101340 2024-09-19T06:16:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f6351250740 1 Processor -- start 2024-09-19T06:16:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f6351250740 1 -- start start 2024-09-19T06:16:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f63457fa640 1 -- 172.21.15.5:0/2468155416 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f633c010a90 con 0x7f6348101340 2024-09-19T06:16:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f6351250740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b57647ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f6351250740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 0x557b57652770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f6351250740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b576550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f6351250740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557b575ac2f0 con 0x557b57647af0 2024-09-19T06:16:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f6351250740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557b57554270 con 0x557b57652cb0 2024-09-19T06:16:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f6351250740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557b575a82b0 con 0x557b57648410 2024-09-19T06:16:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f634ca54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b57647ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f63477fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b576550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f634ca54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b57647ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55084/0 (socket says 172.21.15.5:55084) 2024-09-19T06:16:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f634ca54640 1 -- 172.21.15.5:0/967221068 learned_addr learned my addr 172.21.15.5:0/967221068 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.207+0000 7f6347fff640 1 --2- 172.21.15.5:0/967221068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 0x557b57652770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f634ca54640 1 -- 172.21.15.5:0/967221068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 msgr2=0x557b57652770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f634ca54640 1 --2- 172.21.15.5:0/967221068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 0x557b57652770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f634ca54640 1 -- 172.21.15.5:0/967221068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 msgr2=0x557b576550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f634ca54640 1 --2- 172.21.15.5:0/967221068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b576550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f634ca54640 1 -- 172.21.15.5:0/967221068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b57553340 con 0x557b57647af0 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f63477fe640 1 --2- 172.21.15.5:0/967221068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b576550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f63457fa640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63300a33b0 0x7f63300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f634ca54640 1 --2- 172.21.15.5:0/967221068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b57647ed0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f6340009f50 tx=0x7f63400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8142a0ecf59dd9fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f6347fff640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63300a33b0 0x7f63300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f632e7fc640 1 -- 172.21.15.5:0/967221068 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340004d20 con 0x557b57647af0 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f63457fa640 1 -- 172.21.15.5:0/2468155416 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f633c0d00a0 con 0x7f6348101340 2024-09-19T06:16:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f632e7fc640 1 -- 172.21.15.5:0/967221068 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6340004050 con 0x557b57647af0 2024-09-19T06:16:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.208+0000 7f632e7fc640 1 -- 172.21.15.5:0/967221068 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340005280 con 0x557b57647af0 2024-09-19T06:16:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.209+0000 7f6351250740 1 -- 172.21.15.5:0/967221068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 msgr2=0x557b57647ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.209+0000 7f6347fff640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63300a33b0 0x7f63300a5870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6338005c20 tx=0x7f6338031400 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.209+0000 7f6351250740 1 --2- 172.21.15.5:0/967221068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b57647ed0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f6340009f50 tx=0x7f63400118d0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.209+0000 7f6351250740 1 -- 172.21.15.5:0/967221068 shutdown_connections 2024-09-19T06:16:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.209+0000 7f6351250740 1 --2- 172.21.15.5:0/967221068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b576550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.209+0000 7f6351250740 1 --2- 172.21.15.5:0/967221068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 0x557b57652770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.209+0000 7f6351250740 1 --2- 172.21.15.5:0/967221068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b57647ed0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.209+0000 7f6351250740 1 -- 172.21.15.5:0/967221068 >> 172.21.15.5:0/967221068 conn(0x557b57644060 msgr2=0x557b57644440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.209+0000 7f6351250740 1 -- 172.21.15.5:0/967221068 shutdown_connections 2024-09-19T06:16:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.209+0000 7f6351250740 1 -- 172.21.15.5:0/967221068 wait complete. 2024-09-19T06:16:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.209+0000 7f6351250740 1 -- start start 2024-09-19T06:16:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6351250740 1 Processor -- start 2024-09-19T06:16:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6351250740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b5764fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6351250740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 0x557b576500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6351250740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b576505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6351250740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557b57655880 con 0x557b57647af0 2024-09-19T06:16:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6351250740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557b57535290 con 0x557b57652cb0 2024-09-19T06:16:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6351250740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557b57655a00 con 0x557b57648410 2024-09-19T06:16:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6351250740 10 client.0 _ll_register_callbacks cb 0x557b57566140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6347fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b5764fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6347fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b5764fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55088/0 (socket says 172.21.15.5:55088) 2024-09-19T06:16:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6347fff640 1 -- 172.21.15.5:0/400145383 learned_addr learned my addr 172.21.15.5:0/400145383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f634ca54640 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b576505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f63477fe640 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 0x557b576500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6347fff640 1 -- 172.21.15.5:0/400145383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 msgr2=0x557b576500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6347fff640 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 0x557b576500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6347fff640 1 -- 172.21.15.5:0/400145383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 msgr2=0x557b576505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6347fff640 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b576505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f6347fff640 1 -- 172.21.15.5:0/400145383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b572bf210 con 0x557b57647af0 2024-09-19T06:16:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.210+0000 7f634ca54640 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b576505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.211+0000 7f6347fff640 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b5764fb70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f6338032f50 tx=0x7f6338005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9422fbacb90f637 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.211+0000 7f63277fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.211+0000 7f63277fe640 1 -- 172.21.15.5:0/400145383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633804c070 con 0x557b57647af0 2024-09-19T06:16:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.211+0000 7f6351250740 1 -- 172.21.15.5:0/400145383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6340004050 con 0x557b57647af0 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.211+0000 7f63277fe640 1 -- 172.21.15.5:0/400145383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f6338004250 con 0x557b57647af0 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.211+0000 7f63277fe640 1 -- 172.21.15.5:0/400145383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6338058050 con 0x557b57647af0 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.211+0000 7f6351250740 10 client.18903 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.211+0000 7f6351250740 1 -- 172.21.15.5:0/400145383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557b5738b620 con 0x557b57647af0 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.211+0000 7f63277fe640 1 -- 172.21.15.5:0/400145383 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6340004050 con 0x557b57647af0 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.212+0000 7f63277fe640 1 -- 172.21.15.5:0/400145383 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f633803c560 con 0x557b57647af0 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.212+0000 7f6351250740 10 client.18903 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.212+0000 7f6351250740 10 client.18903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.212+0000 7f6351250740 20 client.18903 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.212+0000 7f6351250740 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557b57662bd0 0x557b57664fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.212+0000 7f6351250740 4 client.18903 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.212+0000 7f6351250740 1 -- 172.21.15.5:0/400145383 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b575b9a20 con 0x557b57662bd0 2024-09-19T06:16:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.212+0000 7f63477fe640 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557b57662bd0 0x557b57664fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.212+0000 7f63277fe640 10 client.18903 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.212+0000 7f63477fe640 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557b57662bd0 0x557b57664fc0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde20e0490876c69 server_cookie=2bb346b95f040e3f in_seq=0 out_seq=0 2024-09-19T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.214+0000 7f63277fe640 1 -- 172.21.15.5:0/400145383 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b575b9a20 con 0x557b57662bd0 2024-09-19T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.214+0000 7f63277fe640 10 client.18903 handle_command_reply: tid=0 2024-09-19T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.214+0000 7f6351250740 1 client.18903 shutdown 2024-09-19T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.215+0000 7f6351250740 1 -- 172.21.15.5:0/400145383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 msgr2=0x557b5764fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.215+0000 7f6351250740 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b5764fb70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f6338032f50 tx=0x7f6338005fd0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.215+0000 7f6351250740 1 -- 172.21.15.5:0/400145383 shutdown_connections 2024-09-19T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.215+0000 7f6351250740 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557b57662bd0 0x557b57664fc0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.215+0000 7f6351250740 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b576505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.215+0000 7f6351250740 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 0x557b576500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.215+0000 7f63277fe640 0 client.18903 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.215+0000 7f6351250740 1 --2- 172.21.15.5:0/400145383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b5764fb70 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.215+0000 7f6351250740 1 -- 172.21.15.5:0/400145383 >> 172.21.15.5:0/400145383 conn(0x557b57644060 msgr2=0x557b576431c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.215+0000 7f6351250740 1 -- 172.21.15.5:0/400145383 shutdown_connections 2024-09-19T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.215+0000 7f6351250740 1 -- 172.21.15.5:0/400145383 wait complete. 2024-09-19T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.215+0000 7f6351250740 20 client.18903 trim_cache size 0 max 16384 2024-09-19T06:16:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.238+0000 7f6351250740 1 Processor -- start 2024-09-19T06:16:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.238+0000 7f6351250740 1 -- start start 2024-09-19T06:16:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.238+0000 7f6351250740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b5765f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.238+0000 7f6351250740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 0x557b57651ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.238+0000 7f6351250740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b57650420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.238+0000 7f6351250740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63481afb00 con 0x557b57647af0 2024-09-19T06:16:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.238+0000 7f6351250740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63481b1700 con 0x557b57652cb0 2024-09-19T06:16:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.238+0000 7f6351250740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f63481b01d0 con 0x557b57648410 2024-09-19T06:16:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.238+0000 7f6347fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b5765f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.238+0000 7f6347fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b5765f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55096/0 (socket says 172.21.15.5:55096) 2024-09-19T06:16:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.239+0000 7f6347fff640 1 -- 172.21.15.5:0/3014433042 learned_addr learned my addr 172.21.15.5:0/3014433042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.239+0000 7f634ca54640 1 --2- 172.21.15.5:0/3014433042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b57650420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.239+0000 7f63477fe640 1 --2- 172.21.15.5:0/3014433042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 0x557b57651ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.239+0000 7f6347fff640 1 -- 172.21.15.5:0/3014433042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 msgr2=0x557b57651ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.239+0000 7f6347fff640 1 --2- 172.21.15.5:0/3014433042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 0x557b57651ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.239+0000 7f6347fff640 1 -- 172.21.15.5:0/3014433042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 msgr2=0x557b57650420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.239+0000 7f6347fff640 1 --2- 172.21.15.5:0/3014433042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b57650420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.239+0000 7f6347fff640 1 -- 172.21.15.5:0/3014433042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b5764e0a0 con 0x557b57647af0 2024-09-19T06:16:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.239+0000 7f6347fff640 1 --2- 172.21.15.5:0/3014433042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b5765f210 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f6338033000 tx=0x7f633803cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5ae8c0cd6423a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.239+0000 7f632dffb640 1 -- 172.21.15.5:0/3014433042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338061070 con 0x557b57647af0 2024-09-19T06:16:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.239+0000 7f632dffb640 1 -- 172.21.15.5:0/3014433042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6338004450 con 0x557b57647af0 2024-09-19T06:16:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.239+0000 7f632dffb640 1 -- 172.21.15.5:0/3014433042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633804c040 con 0x557b57647af0 2024-09-19T06:16:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.240+0000 7f6351250740 1 -- 172.21.15.5:0/3014433042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 msgr2=0x557b5765f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.240+0000 7f6351250740 1 --2- 172.21.15.5:0/3014433042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b5765f210 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f6338033000 tx=0x7f633803cd80 comp rx=0 tx=0).stop 2024-09-19T06:16:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.240+0000 7f6351250740 1 -- 172.21.15.5:0/3014433042 shutdown_connections 2024-09-19T06:16:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.240+0000 7f6351250740 1 --2- 172.21.15.5:0/3014433042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57652cb0 0x557b57650420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.240+0000 7f6351250740 1 --2- 172.21.15.5:0/3014433042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57648410 0x557b57651ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.240+0000 7f6351250740 1 --2- 172.21.15.5:0/3014433042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57647af0 0x557b5765f210 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.240+0000 7f6351250740 1 -- 172.21.15.5:0/3014433042 >> 172.21.15.5:0/3014433042 conn(0x557b57644060 msgr2=0x557b576513d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.240+0000 7f6351250740 1 -- 172.21.15.5:0/3014433042 shutdown_connections 2024-09-19T06:16:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.240+0000 7f6351250740 1 -- 172.21.15.5:0/3014433042 wait complete. 2024-09-19T06:16:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.240+0000 7f6351250740 1 -- start start 2024-09-19T06:16:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f6351250740 1 Processor -- start 2024-09-19T06:16:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f6351250740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57647af0 0x557b5765ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f6351250740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57648410 0x557b5765b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f6351250740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57652cb0 0x557b5764c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f6351250740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63481b1030 con 0x557b57648410 2024-09-19T06:16:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f6351250740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63481b0900 con 0x557b57647af0 2024-09-19T06:16:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f6351250740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f63481adb00 con 0x557b57652cb0 2024-09-19T06:16:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f6351250740 10 client.0 _ll_register_callbacks cb 0x7f6338040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f63477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57648410 0x557b5765b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f63477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57648410 0x557b5765b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55108/0 (socket says 172.21.15.5:55108) 2024-09-19T06:16:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f63477fe640 1 -- 172.21.15.5:0/3584038901 learned_addr learned my addr 172.21.15.5:0/3584038901 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f6347fff640 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57647af0 0x557b5765ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f634ca54640 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57652cb0 0x557b5764c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f63477fe640 1 -- 172.21.15.5:0/3584038901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57652cb0 msgr2=0x557b5764c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f63477fe640 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57652cb0 0x557b5764c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f63477fe640 1 -- 172.21.15.5:0/3584038901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57647af0 msgr2=0x557b5765ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f63477fe640 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57647af0 0x557b5765ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f63477fe640 1 -- 172.21.15.5:0/3584038901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b5764c6d0 con 0x557b57648410 2024-09-19T06:16:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f634ca54640 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57652cb0 0x557b5764c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.241+0000 7f6347fff640 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57647af0 0x557b5765ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f63477fe640 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57648410 0x557b5765b1d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f633c0cddd0 tx=0x7f633c0bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d25bbeedc1451b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f63277fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f63277fe640 1 -- 172.21.15.5:0/3584038901 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c010de0 con 0x557b57648410 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f63277fe640 1 -- 172.21.15.5:0/3584038901 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f633c0989d0 con 0x557b57648410 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f6351250740 1 -- 172.21.15.5:0/3584038901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6338004450 con 0x557b57648410 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f63277fe640 1 -- 172.21.15.5:0/3584038901 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f633c09ca60 con 0x557b57648410 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f6351250740 10 client.18909 fetch_fsmap learned FSMap version 39 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f6351250740 1 -- 172.21.15.5:0/3584038901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557b576cd0d0 con 0x557b57648410 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f63277fe640 1 -- 172.21.15.5:0/3584038901 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6338004450 con 0x557b57648410 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f63277fe640 1 -- 172.21.15.5:0/3584038901 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f633c0d3c60 con 0x557b57648410 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f6351250740 10 client.18909 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f6351250740 10 client.18909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f6351250740 20 client.18909 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f6351250740 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557b576cd860 0x557b576cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.242+0000 7f6351250740 4 client.18909 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:16:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.243+0000 7f6351250740 1 -- 172.21.15.5:0/3584038901 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b575b3d70 con 0x557b576cd860 2024-09-19T06:16:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.243+0000 7f6347fff640 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557b576cd860 0x557b576cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.243+0000 7f63277fe640 10 client.18909 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.243+0000 7f6347fff640 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557b576cd860 0x557b576cfc80 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc581a78270670c8 server_cookie=5ed2ec71d7f816d1 in_seq=0 out_seq=0 2024-09-19T06:16:41.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:16:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.469+0000 7f63277fe640 1 -- 172.21.15.5:0/3584038901 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b575b3d70 con 0x557b576cd860 2024-09-19T06:16:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.469+0000 7f63277fe640 10 client.18909 handle_command_reply: tid=0 2024-09-19T06:16:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.469+0000 7f6351250740 1 client.18909 shutdown 2024-09-19T06:16:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.470+0000 7f6351250740 1 -- 172.21.15.5:0/3584038901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57648410 msgr2=0x557b5765b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.470+0000 7f6351250740 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57648410 0x557b5765b1d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f633c0cddd0 tx=0x7f633c0bc040 comp rx=0 tx=0).stop 2024-09-19T06:16:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.470+0000 7f6351250740 1 -- 172.21.15.5:0/3584038901 shutdown_connections 2024-09-19T06:16:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.470+0000 7f6351250740 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557b576cd860 0x557b576cfc80 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.470+0000 7f6351250740 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b57652cb0 0x557b5764c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.470+0000 7f6351250740 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b57648410 0x557b5765b1d0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.470+0000 7f6351250740 1 --2- 172.21.15.5:0/3584038901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b57647af0 0x557b5765ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.470+0000 7f63277fe640 0 client.18909 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.470+0000 7f6351250740 1 -- 172.21.15.5:0/3584038901 >> 172.21.15.5:0/3584038901 conn(0x557b57644060 msgr2=0x557b576431c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.470+0000 7f6351250740 1 -- 172.21.15.5:0/3584038901 shutdown_connections 2024-09-19T06:16:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.470+0000 7f6351250740 1 -- 172.21.15.5:0/3584038901 wait complete. 2024-09-19T06:16:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.470+0000 7f6351250740 20 client.18909 trim_cache size 0 max 16384 2024-09-19T06:16:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:16:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:16:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:16:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:16:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.471+0000 7f634e4b7640 1 -- 172.21.15.5:0/2468155416 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63300a33b0 msgr2=0x7f63300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.471+0000 7f634e4b7640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63300a33b0 0x7f63300a5870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6338005c20 tx=0x7f6338031400 comp rx=0 tx=0).stop 2024-09-19T06:16:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.471+0000 7f634e4b7640 1 -- 172.21.15.5:0/2468155416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348101340 msgr2=0x7f6348102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.471+0000 7f634e4b7640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348101340 0x7f6348102ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f633c00e9f0 tx=0x7f633c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.472+0000 7f634e4b7640 1 -- 172.21.15.5:0/2468155416 shutdown_connections 2024-09-19T06:16:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.472+0000 7f634e4b7640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63481069b0 0x7f6348102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.472+0000 7f634e4b7640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348101340 0x7f6348102ab0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.472+0000 7f634e4b7640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63300a33b0 0x7f63300a5870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.472+0000 7f634e4b7640 1 --2- 172.21.15.5:0/2468155416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63481009a0 0x7f6348102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.472+0000 7f634e4b7640 1 -- 172.21.15.5:0/2468155416 >> 172.21.15.5:0/2468155416 conn(0x7f63480779a0 msgr2=0x7f6348069bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.472+0000 7f634e4b7640 1 -- 172.21.15.5:0/2468155416 shutdown_connections 2024-09-19T06:16:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:41.472+0000 7f634e4b7640 1 -- 172.21.15.5:0/2468155416 wait complete. 2024-09-19T06:16:41.548 INFO:tasks.workunit.client.0.smithi005.stdout: 10 931 43.51 MB/sec warmup 5 sec latency 271.427 ms 2024-09-19T06:16:41.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:41 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1875102677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:41.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:41 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1875102677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:41.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:41 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1875102677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:16:42.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.272+0000 7fa1bafd4640 1 -- 172.21.15.5:0/3792810024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4101df0 msgr2=0x7fa1b41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:42.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.272+0000 7fa1bafd4640 1 --2- 172.21.15.5:0/3792810024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4101df0 0x7fa1b41021d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009900 tx=0x7fa1a802a410 comp rx=0 tx=0).stop 2024-09-19T06:16:42.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.272+0000 7fa1bafd4640 1 -- 172.21.15.5:0/3792810024 shutdown_connections 2024-09-19T06:16:42.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.272+0000 7fa1bafd4640 1 --2- 172.21.15.5:0/3792810024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1b4106f20 0x7fa1b410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.272+0000 7fa1bafd4640 1 --2- 172.21.15.5:0/3792810024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b41027a0 0x7fa1b41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.272+0000 7fa1bafd4640 1 --2- 172.21.15.5:0/3792810024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4101df0 0x7fa1b41021d0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.272+0000 7fa1bafd4640 1 -- 172.21.15.5:0/3792810024 >> 172.21.15.5:0/3792810024 conn(0x7fa1b40fd8f0 msgr2=0x7fa1b40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:42.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.273+0000 7fa1bafd4640 1 -- 172.21.15.5:0/3792810024 shutdown_connections 2024-09-19T06:16:42.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.273+0000 7fa1bafd4640 1 -- 172.21.15.5:0/3792810024 wait complete. 2024-09-19T06:16:42.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.273+0000 7fa1bafd4640 1 Processor -- start 2024-09-19T06:16:42.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1bafd4640 1 -- start start 2024-09-19T06:16:42.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1bafd4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1b4101df0 0x7fa1b41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1bafd4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b41027a0 0x7fa1b4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1bafd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4106f20 0x7fa1b419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1bafd4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4111df0 con 0x7fa1b4106f20 2024-09-19T06:16:42.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1bafd4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4111c70 con 0x7fa1b4101df0 2024-09-19T06:16:42.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1bafd4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4111f70 con 0x7fa1b41027a0 2024-09-19T06:16:42.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b9571640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4106f20 0x7fa1b419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b9571640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4106f20 0x7fa1b419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55124/0 (socket says 172.21.15.5:55124) 2024-09-19T06:16:42.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b8d70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1b4101df0 0x7fa1b41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b9571640 1 -- 172.21.15.5:0/256380682 learned_addr learned my addr 172.21.15.5:0/256380682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:42.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b41027a0 0x7fa1b4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b9571640 1 -- 172.21.15.5:0/256380682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b41027a0 msgr2=0x7fa1b4198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:42.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b9571640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b41027a0 0x7fa1b4198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b9571640 1 -- 172.21.15.5:0/256380682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1b4101df0 msgr2=0x7fa1b41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:42.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b9571640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1b4101df0 0x7fa1b41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b9571640 1 -- 172.21.15.5:0/256380682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b419d380 con 0x7fa1b4106f20 2024-09-19T06:16:42.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b8d70640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1b4101df0 0x7fa1b41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:42.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b41027a0 0x7fa1b4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:42.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.274+0000 7fa1b9571640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4106f20 0x7fa1b419cca0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00eb10 tx=0x7fa1ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:42.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.275+0000 7fa1b1ffb640 1 -- 172.21.15.5:0/256380682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac004b20 con 0x7fa1b4106f20 2024-09-19T06:16:42.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.275+0000 7fa1bafd4640 1 -- 172.21.15.5:0/256380682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b419d670 con 0x7fa1b4106f20 2024-09-19T06:16:42.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.275+0000 7fa1b1ffb640 1 -- 172.21.15.5:0/256380682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1ac004cc0 con 0x7fa1b4106f20 2024-09-19T06:16:42.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.275+0000 7fa1b1ffb640 1 -- 172.21.15.5:0/256380682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac010690 con 0x7fa1b4106f20 2024-09-19T06:16:42.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.275+0000 7fa1bafd4640 1 -- 172.21.15.5:0/256380682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b4074420 con 0x7fa1b4106f20 2024-09-19T06:16:42.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.278+0000 7fa1b1ffb640 1 -- 172.21.15.5:0/256380682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa1ac005020 con 0x7fa1b4106f20 2024-09-19T06:16:42.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1bd3ad740 1 Processor -- start 2024-09-19T06:16:42.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1bd3ad740 1 -- start start 2024-09-19T06:16:42.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1bd3ad740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe7296b40 0x55cbe7295090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1bd3ad740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 0x55cbe7295a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1bd3ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe72a0370 0x55cbe72a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1bd3ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cbe71fb2f0 con 0x55cbe72a0370 2024-09-19T06:16:42.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1bd3ad740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cbe71a2cf0 con 0x55cbe72955d0 2024-09-19T06:16:42.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1bd3ad740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cbe71f72b0 con 0x55cbe7296b40 2024-09-19T06:16:42.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe72a0370 0x55cbe72a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1b8d70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 0x55cbe7295a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1b8d70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 0x55cbe7295a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60254/0 (socket says 172.21.15.5:60254) 2024-09-19T06:16:42.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1b8d70640 1 -- 172.21.15.5:0/3838711824 learned_addr learned my addr 172.21.15.5:0/3838711824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:42.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.279+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/3838711824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe72a0370 0x55cbe72a2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55140/0 (socket says 172.21.15.5:55140) 2024-09-19T06:16:42.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.280+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/3838711824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe7296b40 0x55cbe7295090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.280+0000 7fa1b1ffb640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa1a40a33b0 0x7fa1a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.280+0000 7fa1b9571640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa1a40a33b0 0x7fa1a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.280+0000 7fa1b3fff640 1 -- 172.21.15.5:0/3838711824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe7296b40 msgr2=0x55cbe7295090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:42.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.280+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/3838711824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe7296b40 0x55cbe7295090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.280+0000 7fa1b3fff640 1 -- 172.21.15.5:0/3838711824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 msgr2=0x55cbe7295a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:42.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.280+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/3838711824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 0x55cbe7295a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.280+0000 7fa1b3fff640 1 -- 172.21.15.5:0/3838711824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe71873a0 con 0x55cbe72a0370 2024-09-19T06:16:42.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.280+0000 7fa1b1ffb640 1 -- 172.21.15.5:0/256380682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fa1ac0d0300 con 0x7fa1b4106f20 2024-09-19T06:16:42.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.280+0000 7fa1b8d70640 1 --2- 172.21.15.5:0/3838711824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 0x55cbe7295a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:42.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.280+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/3838711824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe72a0370 0x55cbe72a2760 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0005fd0 tx=0x7fa1a000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fa63510734d736 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:42.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.280+0000 7fa1b9571640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa1a40a33b0 0x7fa1a40a5870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac09c2c0 tx=0x7fa1ac00efd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:42.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.281+0000 7fa19effd640 1 -- 172.21.15.5:0/3838711824 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00142d0 con 0x55cbe72a0370 2024-09-19T06:16:42.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.281+0000 7fa19effd640 1 -- 172.21.15.5:0/3838711824 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1a0005020 con 0x55cbe72a0370 2024-09-19T06:16:42.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.281+0000 7fa19effd640 1 -- 172.21.15.5:0/3838711824 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00142d0 con 0x55cbe72a0370 2024-09-19T06:16:42.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.282+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/3838711824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe72a0370 msgr2=0x55cbe72a2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:42.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.282+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/3838711824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe72a0370 0x55cbe72a2760 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0005fd0 tx=0x7fa1a000b4a0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.282+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/3838711824 shutdown_connections 2024-09-19T06:16:42.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.282+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/3838711824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe72a0370 0x55cbe72a2760 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.282+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/3838711824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 0x55cbe7295a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.282+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/3838711824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe7296b40 0x55cbe7295090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.282+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/3838711824 >> 172.21.15.5:0/3838711824 conn(0x55cbe725e310 msgr2=0x55cbe7291c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:42.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.282+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/3838711824 shutdown_connections 2024-09-19T06:16:42.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.282+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/3838711824 wait complete. 2024-09-19T06:16:42.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.282+0000 7fa1bd3ad740 1 -- start start 2024-09-19T06:16:42.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.283+0000 7fa1bd3ad740 1 Processor -- start 2024-09-19T06:16:42.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.283+0000 7fa1bd3ad740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 0x55cbe729fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.283+0000 7fa1bd3ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 0x55cbe729d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.283+0000 7fa1bd3ad740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe72a0370 0x55cbe729d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.283+0000 7fa1bd3ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cbe72a3070 con 0x55cbe7296b40 2024-09-19T06:16:42.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.283+0000 7fa1bd3ad740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cbe71b6060 con 0x55cbe72955d0 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.283+0000 7fa1bd3ad740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cbe72a31f0 con 0x55cbe72a0370 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.283+0000 7fa1bd3ad740 10 client.0 _ll_register_callbacks cb 0x55cbe6f098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.283+0000 7fa1b3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 0x55cbe729fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.283+0000 7fa1b8d70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 0x55cbe729d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.283+0000 7fa1b8d70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 0x55cbe729d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55144/0 (socket says 172.21.15.5:55144) 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.283+0000 7fa1b8d70640 1 -- 172.21.15.5:0/2787791182 learned_addr learned my addr 172.21.15.5:0/2787791182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.284+0000 7fa1b3fff640 1 -- 172.21.15.5:0/2787791182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe72a0370 msgr2=0x55cbe729d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.284+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/2787791182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe72a0370 0x55cbe729d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.284+0000 7fa1b3fff640 1 -- 172.21.15.5:0/2787791182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 msgr2=0x55cbe729d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.284+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/2787791182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 0x55cbe729d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.284+0000 7fa1b3fff640 1 -- 172.21.15.5:0/2787791182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe6f6c3c0 con 0x55cbe72955d0 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.284+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/2787791182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 0x55cbe729fd70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0005fd0 tx=0x7fa1a000bdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e00bd48ed13414a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:42.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.284+0000 7fa193fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.284+0000 7fa193fff640 1 -- 172.21.15.5:0/2787791182 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0026020 con 0x55cbe72955d0 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.284+0000 7fa193fff640 1 -- 172.21.15.5:0/2787791182 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fa1a0004230 con 0x55cbe72955d0 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.284+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2787791182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a0005020 con 0x55cbe72955d0 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.285+0000 7fa193fff640 1 -- 172.21.15.5:0/2787791182 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa1a0004430 con 0x55cbe72955d0 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.285+0000 7fa1bd3ad740 10 client.28319 fetch_fsmap learned FSMap version 40 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.285+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2787791182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cbe72aeb10 con 0x55cbe72955d0 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.285+0000 7fa193fff640 1 -- 172.21.15.5:0/2787791182 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1a0005020 con 0x55cbe72955d0 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.285+0000 7fa193fff640 1 -- 172.21.15.5:0/2787791182 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fa1a0004a60 con 0x55cbe72955d0 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.285+0000 7fa1bd3ad740 10 client.28319 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.285+0000 7fa1bd3ad740 10 client.28319 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.285+0000 7fa1bd3ad740 20 client.28319 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.285+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/2787791182 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cbe72aff10 0x55cbe72b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.285+0000 7fa1bd3ad740 4 client.28319 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:42.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.285+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2787791182 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbe7208860 con 0x55cbe72aff10 2024-09-19T06:16:42.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.286+0000 7fa1b8d70640 1 --2- 172.21.15.5:0/2787791182 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cbe72aff10 0x55cbe72b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.286+0000 7fa1b8d70640 1 --2- 172.21.15.5:0/2787791182 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cbe72aff10 0x55cbe72b2300 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bfda1a9210a777d5 server_cookie=3dbb46502d1e4764 in_seq=0 out_seq=0 2024-09-19T06:16:42.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.286+0000 7fa193fff640 10 client.28319 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.287+0000 7fa193fff640 1 -- 172.21.15.5:0/2787791182 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbe7208860 con 0x55cbe72aff10 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.287+0000 7fa193fff640 10 client.28319 handle_command_reply: tid=0 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.287+0000 7fa1bd3ad740 1 client.28319 shutdown 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.287+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2787791182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 msgr2=0x55cbe729fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.287+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/2787791182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 0x55cbe729fd70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0005fd0 tx=0x7fa1a000bdc0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.287+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2787791182 shutdown_connections 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.287+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/2787791182 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cbe72aff10 0x55cbe72b2300 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.287+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/2787791182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe72a0370 0x55cbe729d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.287+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/2787791182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 0x55cbe729d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.287+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/2787791182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe72955d0 0x55cbe729fd70 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0005fd0 tx=0x7fa1a000bdc0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.287+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2787791182 >> 172.21.15.5:0/2787791182 conn(0x55cbe725e310 msgr2=0x55cbe7297960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.287+0000 7fa193fff640 0 client.28319 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:16:42.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.288+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2787791182 shutdown_connections 2024-09-19T06:16:42.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.288+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2787791182 wait complete. 2024-09-19T06:16:42.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.288+0000 7fa1bd3ad740 20 client.28319 trim_cache size 0 max 16384 2024-09-19T06:16:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.310+0000 7fa1bd3ad740 1 Processor -- start 2024-09-19T06:16:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.310+0000 7fa1bd3ad740 1 -- start start 2024-09-19T06:16:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1bd3ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 0x55cbe729bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1bd3ad740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe72a0370 0x55cbe72b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1bd3ad740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe729c430 0x55cbe72b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1bd3ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41ab6c0 con 0x55cbe7296b40 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1bd3ad740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41ad2c0 con 0x55cbe729c430 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1bd3ad740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41abd90 con 0x55cbe72a0370 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 0x55cbe729bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 0x55cbe729bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55148/0 (socket says 172.21.15.5:55148) 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1b8d70640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe72a0370 0x55cbe72b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1b3fff640 1 -- 172.21.15.5:0/4291413774 learned_addr learned my addr 172.21.15.5:0/4291413774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/4291413774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe729c430 0x55cbe72b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1b3fff640 1 -- 172.21.15.5:0/4291413774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe72a0370 msgr2=0x55cbe72b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/4291413774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe72a0370 0x55cbe72b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1b3fff640 1 -- 172.21.15.5:0/4291413774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe729c430 msgr2=0x55cbe72b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/4291413774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe729c430 0x55cbe72b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1b3fff640 1 -- 172.21.15.5:0/4291413774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe7292960 con 0x55cbe7296b40 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/4291413774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 0x55cbe729bef0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00056c0 tx=0x7fa1a0016fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8cab199237cb435 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.311+0000 7fa19e7fc640 1 -- 172.21.15.5:0/4291413774 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0026040 con 0x55cbe7296b40 2024-09-19T06:16:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.312+0000 7fa19e7fc640 1 -- 172.21.15.5:0/4291413774 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1a0003df0 con 0x55cbe7296b40 2024-09-19T06:16:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.312+0000 7fa19e7fc640 1 -- 172.21.15.5:0/4291413774 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00045f0 con 0x55cbe7296b40 2024-09-19T06:16:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.312+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/4291413774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 msgr2=0x55cbe729bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.312+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/4291413774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 0x55cbe729bef0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00056c0 tx=0x7fa1a0016fc0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.312+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/4291413774 shutdown_connections 2024-09-19T06:16:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.312+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/4291413774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe729c430 0x55cbe72b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.312+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/4291413774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe72a0370 0x55cbe72b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.312+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/4291413774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe7296b40 0x55cbe729bef0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.312+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/4291413774 >> 172.21.15.5:0/4291413774 conn(0x55cbe725e310 msgr2=0x55cbe729b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.313+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/4291413774 shutdown_connections 2024-09-19T06:16:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.313+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/4291413774 wait complete. 2024-09-19T06:16:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.313+0000 7fa1bd3ad740 1 -- start start 2024-09-19T06:16:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.313+0000 7fa1bd3ad740 1 Processor -- start 2024-09-19T06:16:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.313+0000 7fa1bd3ad740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe7296b40 0x55cbe72b5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.313+0000 7fa1bd3ad740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe729c430 0x55cbe72b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1bd3ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe72a0370 0x55cbe72b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1bd3ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41acbf0 con 0x55cbe72a0370 2024-09-19T06:16:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1bd3ad740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41ac4c0 con 0x55cbe729c430 2024-09-19T06:16:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1bd3ad740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41a96c0 con 0x55cbe7296b40 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1bd3ad740 10 client.0 _ll_register_callbacks cb 0x7fa1a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1b8d70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe729c430 0x55cbe72b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1b8d70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe729c430 0x55cbe72b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60282/0 (socket says 172.21.15.5:60282) 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1b8d70640 1 -- 172.21.15.5:0/2857508162 learned_addr learned my addr 172.21.15.5:0/2857508162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe7296b40 0x55cbe72b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe72a0370 0x55cbe72b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1b8d70640 1 -- 172.21.15.5:0/2857508162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe7296b40 msgr2=0x55cbe72b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1b8d70640 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe7296b40 0x55cbe72b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1b8d70640 1 -- 172.21.15.5:0/2857508162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe72a0370 msgr2=0x55cbe72b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1b8d70640 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe72a0370 0x55cbe72b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1b8d70640 1 -- 172.21.15.5:0/2857508162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe72b38d0 con 0x55cbe729c430 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe7296b40 0x55cbe72b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa1b8d70640 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe729c430 0x55cbe72b2ca0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa1a802a930 tx=0x7fa1a80041a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f251fa1dfed6934 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.314+0000 7fa193fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:16:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa193fff640 1 -- 172.21.15.5:0/2857508162 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8054060 con 0x55cbe729c430 2024-09-19T06:16:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2857508162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a0003df0 con 0x55cbe729c430 2024-09-19T06:16:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa193fff640 1 -- 172.21.15.5:0/2857508162 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fa1a8005240 con 0x55cbe729c430 2024-09-19T06:16:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa193fff640 1 -- 172.21.15.5:0/2857508162 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa1a8035040 con 0x55cbe729c430 2024-09-19T06:16:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa1bd3ad740 10 client.28325 fetch_fsmap learned FSMap version 40 2024-09-19T06:16:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2857508162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cbe731a6e0 con 0x55cbe729c430 2024-09-19T06:16:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa193fff640 1 -- 172.21.15.5:0/2857508162 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1a0003df0 con 0x55cbe729c430 2024-09-19T06:16:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa193fff640 1 -- 172.21.15.5:0/2857508162 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fa1a803a700 con 0x55cbe729c430 2024-09-19T06:16:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa1bd3ad740 10 client.28325 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T06:16:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa1bd3ad740 10 client.28325 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:16:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa1bd3ad740 20 client.28325 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cbe731ae40 0x55cbe731d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa1bd3ad740 4 client.28325 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:16:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.315+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2857508162 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbe7202d70 con 0x55cbe731ae40 2024-09-19T06:16:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.316+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cbe731ae40 0x55cbe731d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.316+0000 7fa1b3fff640 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cbe731ae40 0x55cbe731d260 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f7b3024434052a6 server_cookie=f4daef684fcf7b01 in_seq=0 out_seq=0 2024-09-19T06:16:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.316+0000 7fa193fff640 10 client.28325 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:16:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.316+0000 7fa193fff640 1 -- 172.21.15.5:0/2857508162 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbe7202d70 con 0x55cbe731ae40 2024-09-19T06:16:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.316+0000 7fa193fff640 10 client.28325 handle_command_reply: tid=0 2024-09-19T06:16:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.316+0000 7fa1bd3ad740 1 client.28325 shutdown 2024-09-19T06:16:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.316+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2857508162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe729c430 msgr2=0x55cbe72b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.316+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe729c430 0x55cbe72b2ca0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa1a802a930 tx=0x7fa1a80041a0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.317+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2857508162 shutdown_connections 2024-09-19T06:16:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.317+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cbe731ae40 0x55cbe731d260 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.317+0000 7fa193fff640 0 client.28325 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:16:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.317+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbe72a0370 0x55cbe72b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.317+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbe729c430 0x55cbe72b2ca0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.317+0000 7fa1bd3ad740 1 --2- 172.21.15.5:0/2857508162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbe7296b40 0x55cbe72b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.317+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2857508162 >> 172.21.15.5:0/2857508162 conn(0x55cbe725e310 msgr2=0x55cbe72904e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:42.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.317+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2857508162 shutdown_connections 2024-09-19T06:16:42.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.318+0000 7fa1bd3ad740 1 -- 172.21.15.5:0/2857508162 wait complete. 2024-09-19T06:16:42.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.318+0000 7fa1bd3ad740 20 client.28325 trim_cache size 0 max 16384 2024-09-19T06:16:42.325 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:16:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.319+0000 7fa1bafd4640 1 -- 172.21.15.5:0/256380682 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa1a40a33b0 msgr2=0x7fa1a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.319+0000 7fa1bafd4640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa1a40a33b0 0x7fa1a40a5870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac09c2c0 tx=0x7fa1ac00efd0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.319+0000 7fa1bafd4640 1 -- 172.21.15.5:0/256380682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4106f20 msgr2=0x7fa1b419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.319+0000 7fa1bafd4640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4106f20 0x7fa1b419cca0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00eb10 tx=0x7fa1ac00c2d0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.319+0000 7fa1bafd4640 1 -- 172.21.15.5:0/256380682 shutdown_connections 2024-09-19T06:16:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.319+0000 7fa1bafd4640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4106f20 0x7fa1b419cca0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.319+0000 7fa1bafd4640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa1a40a33b0 0x7fa1a40a5870 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.319+0000 7fa1bafd4640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b41027a0 0x7fa1b4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.319+0000 7fa1bafd4640 1 --2- 172.21.15.5:0/256380682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1b4101df0 0x7fa1b41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.319+0000 7fa1bafd4640 1 -- 172.21.15.5:0/256380682 >> 172.21.15.5:0/256380682 conn(0x7fa1b40fd8f0 msgr2=0x7fa1b40fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.319+0000 7fa1bafd4640 1 -- 172.21.15.5:0/256380682 shutdown_connections 2024-09-19T06:16:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:42.319+0000 7fa1bafd4640 1 -- 172.21.15.5:0/256380682 wait complete. 2024-09-19T06:16:42.548 INFO:tasks.workunit.client.0.smithi005.stdout: 10 1061 38.00 MB/sec warmup 6 sec latency 130.709 ms 2024-09-19T06:16:42.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:42 smithi005 ceph-mon[30955]: pgmap v418: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.6 MiB/s wr, 51 op/s 2024-09-19T06:16:42.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:42 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:16:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:42 smithi157 ceph-mon[35346]: pgmap v418: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.6 MiB/s wr, 51 op/s 2024-09-19T06:16:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:42 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:16:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:42 smithi049 ceph-mon[34640]: pgmap v418: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.6 MiB/s wr, 51 op/s 2024-09-19T06:16:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:42 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:16:43.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:16:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:43.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 1157 32.86 MB/sec warmup 7 sec latency 206.656 ms 2024-09-19T06:16:43.680 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:43 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:16:43.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:43 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:16:43.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:43 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:16:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.953+0000 7f0fa837d640 1 -- 172.21.15.5:0/3843266181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa01036e0 msgr2=0x7f0fa0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.953+0000 7f0fa837d640 1 --2- 172.21.15.5:0/3843266181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa01036e0 0x7f0fa0107b90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f0f90009930 tx=0x7f0f9002a420 comp rx=0 tx=0).stop 2024-09-19T06:16:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.953+0000 7f0fa837d640 1 -- 172.21.15.5:0/3843266181 shutdown_connections 2024-09-19T06:16:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.953+0000 7f0fa837d640 1 --2- 172.21.15.5:0/3843266181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fa01082a0 0x7f0fa010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.953+0000 7f0fa837d640 1 --2- 172.21.15.5:0/3843266181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa01036e0 0x7f0fa0107b90 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.953+0000 7f0fa837d640 1 --2- 172.21.15.5:0/3843266181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa0102d30 0x7f0fa0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.953+0000 7f0fa837d640 1 -- 172.21.15.5:0/3843266181 >> 172.21.15.5:0/3843266181 conn(0x7f0fa00fc7d0 msgr2=0x7f0fa00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.953+0000 7f0fa837d640 1 -- 172.21.15.5:0/3843266181 shutdown_connections 2024-09-19T06:16:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.953+0000 7f0fa837d640 1 -- 172.21.15.5:0/3843266181 wait complete. 2024-09-19T06:16:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.954+0000 7f0fa837d640 1 Processor -- start 2024-09-19T06:16:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.954+0000 7f0fa837d640 1 -- start start 2024-09-19T06:16:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.954+0000 7f0fa837d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fa0102d30 0x7f0fa01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.954+0000 7f0fa837d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa01036e0 0x7f0fa0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.954+0000 7f0fa837d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa01082a0 0x7f0fa019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.954+0000 7f0fa5918640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa01036e0 0x7f0fa0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.954+0000 7f0fa5918640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa01036e0 0x7f0fa0198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60292/0 (socket says 172.21.15.5:60292) 2024-09-19T06:16:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.954+0000 7f0fa5918640 1 -- 172.21.15.5:0/1951023302 learned_addr learned my addr 172.21.15.5:0/1951023302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.954+0000 7f0fa6119640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fa0102d30 0x7f0fa01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.954+0000 7f0fa691a640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa01082a0 0x7f0fa019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.954+0000 7f0fa837d640 1 -- 172.21.15.5:0/1951023302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa0111f50 con 0x7f0fa01082a0 2024-09-19T06:16:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.955+0000 7f0fa837d640 1 -- 172.21.15.5:0/1951023302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa0111dd0 con 0x7f0fa01036e0 2024-09-19T06:16:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.955+0000 7f0fa837d640 1 -- 172.21.15.5:0/1951023302 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa01120d0 con 0x7f0fa0102d30 2024-09-19T06:16:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.955+0000 7f0fa5918640 1 -- 172.21.15.5:0/1951023302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fa0102d30 msgr2=0x7f0fa01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.955+0000 7f0fa5918640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fa0102d30 0x7f0fa01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.955+0000 7f0fa5918640 1 -- 172.21.15.5:0/1951023302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa01082a0 msgr2=0x7f0fa019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.955+0000 7f0fa5918640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa01082a0 0x7f0fa019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.955+0000 7f0fa5918640 1 -- 172.21.15.5:0/1951023302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa019d480 con 0x7f0fa01036e0 2024-09-19T06:16:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.955+0000 7f0fa6119640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fa0102d30 0x7f0fa01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.955+0000 7f0fa5918640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa01036e0 0x7f0fa0198a10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0f90031440 tx=0x7f0f90004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.955+0000 7f0f977fe640 1 -- 172.21.15.5:0/1951023302 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9003f030 con 0x7f0fa01036e0 2024-09-19T06:16:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.956+0000 7f0f977fe640 1 -- 172.21.15.5:0/1951023302 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f90003aa0 con 0x7f0fa01036e0 2024-09-19T06:16:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.956+0000 7f0fa837d640 1 -- 172.21.15.5:0/1951023302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa019d710 con 0x7f0fa01036e0 2024-09-19T06:16:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.956+0000 7f0f977fe640 1 -- 172.21.15.5:0/1951023302 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90035070 con 0x7f0fa01036e0 2024-09-19T06:16:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.956+0000 7f0fa837d640 1 -- 172.21.15.5:0/1951023302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa01a5050 con 0x7f0fa01036e0 2024-09-19T06:16:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.959+0000 7f0faa756740 1 Processor -- start 2024-09-19T06:16:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.959+0000 7f0faa756740 1 -- start start 2024-09-19T06:16:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.959+0000 7f0f977fe640 1 -- 172.21.15.5:0/1951023302 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0f90042030 con 0x7f0fa01036e0 2024-09-19T06:16:43.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.959+0000 7f0faa756740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362087b40 0x564362086090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.959+0000 7f0faa756740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x564362086a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.959+0000 7f0faa756740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x564362093760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.959+0000 7f0faa756740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564361fec2f0 con 0x564362091370 2024-09-19T06:16:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.959+0000 7f0faa756740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564361f93cf0 con 0x5643620865d0 2024-09-19T06:16:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0faa756740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564361fe82b0 con 0x564362087b40 2024-09-19T06:16:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0fa6119640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x564362086a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0fa6119640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x564362086a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60302/0 (socket says 172.21.15.5:60302) 2024-09-19T06:16:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0fa6119640 1 -- 172.21.15.5:0/1065846997 learned_addr learned my addr 172.21.15.5:0/1065846997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0fa5918640 1 --2- 172.21.15.5:0/1065846997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x564362093760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0fa691a640 1 --2- 172.21.15.5:0/1065846997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362087b40 0x564362086090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0fa6119640 1 -- 172.21.15.5:0/1065846997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362087b40 msgr2=0x564362086090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0fa6119640 1 --2- 172.21.15.5:0/1065846997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362087b40 0x564362086090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0fa6119640 1 -- 172.21.15.5:0/1065846997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 msgr2=0x564362093760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0fa6119640 1 --2- 172.21.15.5:0/1065846997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x564362093760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0fa6119640 1 -- 172.21.15.5:0/1065846997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564361f783a0 con 0x5643620865d0 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0f977fe640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0f8c0a33b0 0x7f0f8c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0fa691a640 1 --2- 172.21.15.5:0/1065846997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362087b40 0x564362086090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0f977fe640 1 -- 172.21.15.5:0/1951023302 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f0f900f02c0 con 0x7f0fa01036e0 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.960+0000 7f0fa6119640 1 --2- 172.21.15.5:0/1065846997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x564362086a10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c013900 tx=0x7f0f9c013dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408cc0122efa0403 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.961+0000 7f0fa691a640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0f8c0a33b0 0x7f0f8c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.961+0000 7f0f94dea640 1 -- 172.21.15.5:0/1065846997 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c021070 con 0x5643620865d0 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.961+0000 7f0f94dea640 1 -- 172.21.15.5:0/1065846997 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f9c00ee40 con 0x5643620865d0 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.961+0000 7f0f94dea640 1 -- 172.21.15.5:0/1065846997 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c01b370 con 0x5643620865d0 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.962+0000 7f0faa756740 1 -- 172.21.15.5:0/1065846997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 msgr2=0x564362086a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.962+0000 7f0fa691a640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0f8c0a33b0 0x7f0f8c0a5870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005fd0 tx=0x7f0f98009600 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.962+0000 7f0faa756740 1 --2- 172.21.15.5:0/1065846997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x564362086a10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c013900 tx=0x7f0f9c013dc0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.962+0000 7f0faa756740 1 -- 172.21.15.5:0/1065846997 shutdown_connections 2024-09-19T06:16:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.962+0000 7f0faa756740 1 --2- 172.21.15.5:0/1065846997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x564362093760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.962+0000 7f0faa756740 1 --2- 172.21.15.5:0/1065846997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x564362086a10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.962+0000 7f0faa756740 1 --2- 172.21.15.5:0/1065846997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362087b40 0x564362086090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.962+0000 7f0faa756740 1 -- 172.21.15.5:0/1065846997 >> 172.21.15.5:0/1065846997 conn(0x56436204f310 msgr2=0x564362082c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.962+0000 7f0faa756740 1 -- 172.21.15.5:0/1065846997 shutdown_connections 2024-09-19T06:16:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.962+0000 7f0faa756740 1 -- 172.21.15.5:0/1065846997 wait complete. 2024-09-19T06:16:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.962+0000 7f0faa756740 1 -- start start 2024-09-19T06:16:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.962+0000 7f0faa756740 1 Processor -- start 2024-09-19T06:16:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0faa756740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x564362090d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0faa756740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362087b40 0x56436208dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0faa756740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x56436208e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0faa756740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5643620940c0 con 0x564362091370 2024-09-19T06:16:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0faa756740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564361fa7060 con 0x5643620865d0 2024-09-19T06:16:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0faa756740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564362094240 con 0x564362087b40 2024-09-19T06:16:43.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0fa5918640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362087b40 0x56436208dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0fa5918640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362087b40 0x56436208dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52648/0 (socket says 172.21.15.5:52648) 2024-09-19T06:16:43.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0fa5918640 1 -- 172.21.15.5:0/669966897 learned_addr learned my addr 172.21.15.5:0/669966897 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:43.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0faa756740 10 client.0 _ll_register_callbacks cb 0x564361cfa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:43.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0fa691a640 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x56436208e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0fa6119640 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x564362090d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0fa691a640 1 -- 172.21.15.5:0/669966897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362087b40 msgr2=0x56436208dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0fa691a640 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362087b40 0x56436208dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0fa691a640 1 -- 172.21.15.5:0/669966897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 msgr2=0x564362090d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0fa691a640 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x564362090d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.963+0000 7f0fa691a640 1 -- 172.21.15.5:0/669966897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564361d5d3c0 con 0x564362091370 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0fa691a640 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x56436208e5e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0f98013ed0 tx=0x7f0f98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f9d8f7a242b47a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0f89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0f89ffb640 1 -- 172.21.15.5:0/669966897 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98020620 con 0x564362091370 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0f89ffb640 1 -- 172.21.15.5:0/669966897 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f0f980207c0 con 0x564362091370 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0faa756740 1 -- 172.21.15.5:0/669966897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564361ff9860 con 0x564362091370 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0f89ffb640 1 -- 172.21.15.5:0/669966897 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0f98011e60 con 0x564362091370 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0faa756740 10 client.18939 fetch_fsmap learned FSMap version 40 2024-09-19T06:16:43.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0faa756740 1 -- 172.21.15.5:0/669966897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56436209fb90 con 0x564362091370 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0f89ffb640 1 -- 172.21.15.5:0/669966897 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f98031480 con 0x564362091370 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0f89ffb640 1 -- 172.21.15.5:0/669966897 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f0f9800f040 con 0x564362091370 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0faa756740 10 client.18939 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0faa756740 10 client.18939 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0faa756740 20 client.18939 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0faa756740 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5643620a1050 0x5643620a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0faa756740 4 client.18939 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.964+0000 7f0faa756740 1 -- 172.21.15.5:0/669966897 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564361ff9860 con 0x5643620a1050 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.965+0000 7f0fa6119640 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5643620a1050 0x5643620a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.965+0000 7f0fa6119640 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5643620a1050 0x5643620a34d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8821568d68efc495 server_cookie=d3295645311ba808 in_seq=0 out_seq=0 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.965+0000 7f0f89ffb640 10 client.18939 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0f89ffb640 1 -- 172.21.15.5:0/669966897 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564361ff9860 con 0x5643620a1050 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0f89ffb640 10 client.18939 handle_command_reply: tid=0 2024-09-19T06:16:43.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0faa756740 1 client.18939 shutdown 2024-09-19T06:16:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0faa756740 1 -- 172.21.15.5:0/669966897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 msgr2=0x56436208e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0faa756740 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x56436208e5e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0f98013ed0 tx=0x7f0f98004a40 comp rx=0 tx=0).stop 2024-09-19T06:16:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0faa756740 1 -- 172.21.15.5:0/669966897 shutdown_connections 2024-09-19T06:16:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0faa756740 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5643620a1050 0x5643620a34d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0faa756740 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x56436208e5e0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0faa756740 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362087b40 0x56436208dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0faa756740 1 --2- 172.21.15.5:0/669966897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x564362090d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0faa756740 1 -- 172.21.15.5:0/669966897 >> 172.21.15.5:0/669966897 conn(0x56436204f310 msgr2=0x564362088960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0f89ffb640 0 client.18939 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:16:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0faa756740 1 -- 172.21.15.5:0/669966897 shutdown_connections 2024-09-19T06:16:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0faa756740 1 -- 172.21.15.5:0/669966897 wait complete. 2024-09-19T06:16:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.966+0000 7f0faa756740 20 client.18939 trim_cache size 0 max 16384 2024-09-19T06:16:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.985+0000 7f0faa756740 1 Processor -- start 2024-09-19T06:16:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.985+0000 7f0faa756740 1 -- start start 2024-09-19T06:16:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.985+0000 7f0faa756740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5643620865d0 0x564362082e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.985+0000 7f0faa756740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564362087b40 0x56436209d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0faa756740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x564362089120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0faa756740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa01ab6f0 con 0x564362091370 2024-09-19T06:16:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0faa756740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa01ad2f0 con 0x564362087b40 2024-09-19T06:16:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0faa756740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa01abdc0 con 0x5643620865d0 2024-09-19T06:16:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0fa691a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x564362089120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0fa691a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x564362089120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55216/0 (socket says 172.21.15.5:55216) 2024-09-19T06:16:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0fa5918640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564362087b40 0x56436209d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0fa691a640 1 -- 172.21.15.5:0/1732501444 learned_addr learned my addr 172.21.15.5:0/1732501444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0fa6119640 1 --2- 172.21.15.5:0/1732501444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5643620865d0 0x564362082e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0fa691a640 1 -- 172.21.15.5:0/1732501444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5643620865d0 msgr2=0x564362082e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0fa691a640 1 --2- 172.21.15.5:0/1732501444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5643620865d0 0x564362082e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0fa691a640 1 -- 172.21.15.5:0/1732501444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564362087b40 msgr2=0x56436209d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0fa691a640 1 --2- 172.21.15.5:0/1732501444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564362087b40 0x56436209d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0fa691a640 1 -- 172.21.15.5:0/1732501444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564362083dd0 con 0x564362091370 2024-09-19T06:16:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0fa6119640 1 --2- 172.21.15.5:0/1732501444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5643620865d0 0x564362082e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.986+0000 7f0fa691a640 1 --2- 172.21.15.5:0/1732501444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x564362089120 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0f9801d3e0 tx=0x7f0f98011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4248743a3c95992 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.987+0000 7f0f8bfff640 1 -- 172.21.15.5:0/1732501444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98011a20 con 0x564362091370 2024-09-19T06:16:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.987+0000 7f0f8bfff640 1 -- 172.21.15.5:0/1732501444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f9801d8c0 con 0x564362091370 2024-09-19T06:16:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.987+0000 7f0f8bfff640 1 -- 172.21.15.5:0/1732501444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98020810 con 0x564362091370 2024-09-19T06:16:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.988+0000 7f0faa756740 1 -- 172.21.15.5:0/1732501444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 msgr2=0x564362089120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.988+0000 7f0faa756740 1 --2- 172.21.15.5:0/1732501444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x564362089120 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0f9801d3e0 tx=0x7f0f98011df0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.988+0000 7f0faa756740 1 -- 172.21.15.5:0/1732501444 shutdown_connections 2024-09-19T06:16:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.988+0000 7f0faa756740 1 --2- 172.21.15.5:0/1732501444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362091370 0x564362089120 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.988+0000 7f0faa756740 1 --2- 172.21.15.5:0/1732501444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564362087b40 0x56436209d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.988+0000 7f0faa756740 1 --2- 172.21.15.5:0/1732501444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5643620865d0 0x564362082e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.988+0000 7f0faa756740 1 -- 172.21.15.5:0/1732501444 >> 172.21.15.5:0/1732501444 conn(0x56436204f310 msgr2=0x564362082700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.988+0000 7f0faa756740 1 -- 172.21.15.5:0/1732501444 shutdown_connections 2024-09-19T06:16:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.988+0000 7f0faa756740 1 -- 172.21.15.5:0/1732501444 wait complete. 2024-09-19T06:16:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.988+0000 7f0faa756740 1 -- start start 2024-09-19T06:16:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.988+0000 7f0faa756740 1 Processor -- start 2024-09-19T06:16:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0faa756740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x5643620996b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0faa756740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362087b40 0x564362099bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0faa756740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362091370 0x56436208a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0faa756740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa01acc20 con 0x564362087b40 2024-09-19T06:16:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0faa756740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa01ac4f0 con 0x5643620865d0 2024-09-19T06:16:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0faa756740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa01a96f0 con 0x564362091370 2024-09-19T06:16:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0faa756740 10 client.0 _ll_register_callbacks cb 0x7f0f98009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0fa5918640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362087b40 0x564362099bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0fa6119640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x5643620996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0fa5918640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362087b40 0x564362099bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55224/0 (socket says 172.21.15.5:55224) 2024-09-19T06:16:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0fa5918640 1 -- 172.21.15.5:0/3189704218 learned_addr learned my addr 172.21.15.5:0/3189704218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0fa691a640 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362091370 0x56436208a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0fa5918640 1 -- 172.21.15.5:0/3189704218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362091370 msgr2=0x56436208a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0fa5918640 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362091370 0x56436208a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0fa5918640 1 -- 172.21.15.5:0/3189704218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 msgr2=0x5643620996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0fa5918640 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x5643620996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0fa5918640 1 -- 172.21.15.5:0/3189704218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56436208b090 con 0x564362087b40 2024-09-19T06:16:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0fa6119640 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x5643620996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.989+0000 7f0fa691a640 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362091370 0x56436208a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.990+0000 7f0fa5918640 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362087b40 0x564362099bf0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f0f900f3bb0 tx=0x7f0f90009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1b88f7950ce140 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.990+0000 7f0f89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.990+0000 7f0f89ffb640 1 -- 172.21.15.5:0/3189704218 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90049040 con 0x564362087b40 2024-09-19T06:16:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.990+0000 7f0f89ffb640 1 -- 172.21.15.5:0/3189704218 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f0f9003bd20 con 0x564362087b40 2024-09-19T06:16:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.990+0000 7f0faa756740 1 -- 172.21.15.5:0/3189704218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f9801d8c0 con 0x564362087b40 2024-09-19T06:16:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.990+0000 7f0f89ffb640 1 -- 172.21.15.5:0/3189704218 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0f900bc170 con 0x564362087b40 2024-09-19T06:16:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.990+0000 7f0faa756740 10 client.18945 fetch_fsmap learned FSMap version 40 2024-09-19T06:16:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.990+0000 7f0faa756740 1 -- 172.21.15.5:0/3189704218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56436210b8a0 con 0x564362087b40 2024-09-19T06:16:43.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.990+0000 7f0f89ffb640 1 -- 172.21.15.5:0/3189704218 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f900bc630 con 0x564362087b40 2024-09-19T06:16:43.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.991+0000 7f0f89ffb640 1 -- 172.21.15.5:0/3189704218 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f0f900b8ca0 con 0x564362087b40 2024-09-19T06:16:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.991+0000 7f0faa756740 10 client.18945 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T06:16:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.991+0000 7f0faa756740 10 client.18945 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:16:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.991+0000 7f0faa756740 20 client.18945 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.991+0000 7f0faa756740 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x56436210c070 0x56436210e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.991+0000 7f0faa756740 4 client.18945 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:16:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.991+0000 7f0faa756740 1 -- 172.21.15.5:0/3189704218 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0f9801d8c0 con 0x56436210c070 2024-09-19T06:16:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.991+0000 7f0fa6119640 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x56436210c070 0x56436210e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.992+0000 7f0f89ffb640 10 client.18945 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:16:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.992+0000 7f0fa6119640 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x56436210c070 0x56436210e490 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c0db5a5bfeae2d3 server_cookie=1277dfb12e1e90c2 in_seq=0 out_seq=0 2024-09-19T06:16:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.992+0000 7f0f89ffb640 1 -- 172.21.15.5:0/3189704218 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0f9801d8c0 con 0x56436210c070 2024-09-19T06:16:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.992+0000 7f0f89ffb640 10 client.18945 handle_command_reply: tid=0 2024-09-19T06:16:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.992+0000 7f0faa756740 1 client.18945 shutdown 2024-09-19T06:16:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.993+0000 7f0faa756740 1 -- 172.21.15.5:0/3189704218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362087b40 msgr2=0x564362099bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.993+0000 7f0faa756740 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362087b40 0x564362099bf0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f0f900f3bb0 tx=0x7f0f90009560 comp rx=0 tx=0).stop 2024-09-19T06:16:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.993+0000 7f0faa756740 1 -- 172.21.15.5:0/3189704218 shutdown_connections 2024-09-19T06:16:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.993+0000 7f0faa756740 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x56436210c070 0x56436210e490 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.993+0000 7f0faa756740 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564362091370 0x56436208a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.993+0000 7f0faa756740 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564362087b40 0x564362099bf0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.993+0000 7f0faa756740 1 --2- 172.21.15.5:0/3189704218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5643620865d0 0x5643620996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.993+0000 7f0faa756740 1 -- 172.21.15.5:0/3189704218 >> 172.21.15.5:0/3189704218 conn(0x56436204f310 msgr2=0x5643620814e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.993+0000 7f0f89ffb640 0 client.18945 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:16:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.993+0000 7f0faa756740 1 -- 172.21.15.5:0/3189704218 shutdown_connections 2024-09-19T06:16:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.993+0000 7f0faa756740 1 -- 172.21.15.5:0/3189704218 wait complete. 2024-09-19T06:16:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.993+0000 7f0faa756740 20 client.18945 trim_cache size 0 max 16384 2024-09-19T06:16:44.001 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:16:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.994+0000 7f0fa837d640 1 -- 172.21.15.5:0/1951023302 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0f8c0a33b0 msgr2=0x7f0f8c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.994+0000 7f0fa837d640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0f8c0a33b0 0x7f0f8c0a5870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005fd0 tx=0x7f0f98009600 comp rx=0 tx=0).stop 2024-09-19T06:16:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.994+0000 7f0fa837d640 1 -- 172.21.15.5:0/1951023302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa01036e0 msgr2=0x7f0fa0198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.994+0000 7f0fa837d640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa01036e0 0x7f0fa0198a10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0f90031440 tx=0x7f0f90004270 comp rx=0 tx=0).stop 2024-09-19T06:16:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.994+0000 7f0fa837d640 1 -- 172.21.15.5:0/1951023302 shutdown_connections 2024-09-19T06:16:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.994+0000 7f0fa837d640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa01082a0 0x7f0fa019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.994+0000 7f0fa837d640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa01036e0 0x7f0fa0198a10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.994+0000 7f0fa837d640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0f8c0a33b0 0x7f0f8c0a5870 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.994+0000 7f0fa837d640 1 --2- 172.21.15.5:0/1951023302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fa0102d30 0x7f0fa01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.994+0000 7f0fa837d640 1 -- 172.21.15.5:0/1951023302 >> 172.21.15.5:0/1951023302 conn(0x7f0fa00fc7d0 msgr2=0x7f0fa0105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.995+0000 7f0fa837d640 1 -- 172.21.15.5:0/1951023302 shutdown_connections 2024-09-19T06:16:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:43.995+0000 7f0fa837d640 1 -- 172.21.15.5:0/1951023302 wait complete. 2024-09-19T06:16:44.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 1266 28.91 MB/sec warmup 8 sec latency 212.391 ms 2024-09-19T06:16:44.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:44 smithi005 ceph-mon[30955]: pgmap v419: 97 pgs: 97 active+clean; 365 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 18 MiB/s wr, 329 op/s 2024-09-19T06:16:44.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:44 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:44.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:44 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:16:44.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:44 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:16:44.785 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:16:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:44 smithi157 ceph-mon[35346]: pgmap v419: 97 pgs: 97 active+clean; 365 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 18 MiB/s wr, 329 op/s 2024-09-19T06:16:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:44 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:44 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:16:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:44 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:16:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:44 smithi049 ceph-mon[34640]: pgmap v419: 97 pgs: 97 active+clean; 365 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 18 MiB/s wr, 329 op/s 2024-09-19T06:16:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:44 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:16:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:44 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:16:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:44 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:16:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:45.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 1440 28.60 MB/sec warmup 9 sec latency 111.448 ms 2024-09-19T06:16:45.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.684+0000 7fdcb416f640 1 -- 172.21.15.5:0/1806080964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcac101de0 msgr2=0x7fdcac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.684+0000 7fdcb416f640 1 --2- 172.21.15.5:0/1806080964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcac101de0 0x7fdcac1021c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009900 tx=0x7fdca802a410 comp rx=0 tx=0).stop 2024-09-19T06:16:45.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.684+0000 7fdcb416f640 1 -- 172.21.15.5:0/1806080964 shutdown_connections 2024-09-19T06:16:45.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.684+0000 7fdcb416f640 1 --2- 172.21.15.5:0/1806080964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcac106f10 0x7fdcac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.684+0000 7fdcb416f640 1 --2- 172.21.15.5:0/1806080964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcac102790 0x7fdcac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.684+0000 7fdcb416f640 1 --2- 172.21.15.5:0/1806080964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcac101de0 0x7fdcac1021c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.684+0000 7fdcb416f640 1 -- 172.21.15.5:0/1806080964 >> 172.21.15.5:0/1806080964 conn(0x7fdcac0fd8c0 msgr2=0x7fdcac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:45.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.684+0000 7fdcb416f640 1 -- 172.21.15.5:0/1806080964 shutdown_connections 2024-09-19T06:16:45.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.684+0000 7fdcb416f640 1 -- 172.21.15.5:0/1806080964 wait complete. 2024-09-19T06:16:45.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb416f640 1 Processor -- start 2024-09-19T06:16:45.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb416f640 1 -- start start 2024-09-19T06:16:45.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb416f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcac101de0 0x7fdcac19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb416f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcac102790 0x7fdcac19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb416f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcac106f10 0x7fdcac19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb416f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac111e70 con 0x7fdcac106f10 2024-09-19T06:16:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb416f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac111cf0 con 0x7fdcac102790 2024-09-19T06:16:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb416f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac111ff0 con 0x7fdcac101de0 2024-09-19T06:16:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb1f0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcac101de0 0x7fdcac19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb170a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcac102790 0x7fdcac19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb270c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcac106f10 0x7fdcac19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb1f0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcac101de0 0x7fdcac19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53422/0 (socket says 172.21.15.5:53422) 2024-09-19T06:16:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb170a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcac102790 0x7fdcac19ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36404/0 (socket says 172.21.15.5:36404) 2024-09-19T06:16:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.685+0000 7fdcb1f0b640 1 -- 172.21.15.5:0/386341492 learned_addr learned my addr 172.21.15.5:0/386341492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.686+0000 7fdcb1f0b640 1 -- 172.21.15.5:0/386341492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcac102790 msgr2=0x7fdcac19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.686+0000 7fdcb1f0b640 1 --2- 172.21.15.5:0/386341492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcac102790 0x7fdcac19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.686+0000 7fdcb1f0b640 1 -- 172.21.15.5:0/386341492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcac106f10 msgr2=0x7fdcac19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.686+0000 7fdcb1f0b640 1 --2- 172.21.15.5:0/386341492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcac106f10 0x7fdcac19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.686+0000 7fdcb1f0b640 1 -- 172.21.15.5:0/386341492 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac19f580 con 0x7fdcac101de0 2024-09-19T06:16:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.686+0000 7fdcb1f0b640 1 --2- 172.21.15.5:0/386341492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcac101de0 0x7fdcac19a5d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdca8005ab0 tx=0x7fdca8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.686+0000 7fdca2ffd640 1 -- 172.21.15.5:0/386341492 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca803f070 con 0x7fdcac101de0 2024-09-19T06:16:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.687+0000 7fdca2ffd640 1 -- 172.21.15.5:0/386341492 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdca8031b60 con 0x7fdcac101de0 2024-09-19T06:16:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.687+0000 7fdca2ffd640 1 -- 172.21.15.5:0/386341492 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8035030 con 0x7fdcac101de0 2024-09-19T06:16:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.687+0000 7fdcb416f640 1 -- 172.21.15.5:0/386341492 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcac19f810 con 0x7fdcac101de0 2024-09-19T06:16:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.687+0000 7fdcb416f640 1 -- 172.21.15.5:0/386341492 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcac19fca0 con 0x7fdcac101de0 2024-09-19T06:16:45.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.689+0000 7fdca2ffd640 1 -- 172.21.15.5:0/386341492 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fdca8042070 con 0x7fdcac101de0 2024-09-19T06:16:45.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.689+0000 7fdcb6548740 1 Processor -- start 2024-09-19T06:16:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.690+0000 7fdcb6548740 1 -- start start 2024-09-19T06:16:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.690+0000 7fdcb6548740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a197843b40 0x55a197842090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.690+0000 7fdcb6548740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1978425d0 0x55a197842a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.690+0000 7fdcb6548740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a19784d370 0x55a19784f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.690+0000 7fdcb6548740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a1977a82f0 con 0x55a1978425d0 2024-09-19T06:16:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.690+0000 7fdcb6548740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a19774fcf0 con 0x55a197843b40 2024-09-19T06:16:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.690+0000 7fdcb6548740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a1977a42b0 con 0x55a19784d370 2024-09-19T06:16:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.690+0000 7fdcb1f0b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1978425d0 0x55a197842a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.690+0000 7fdcb170a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a19784d370 0x55a19784f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.690+0000 7fdcb170a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a19784d370 0x55a19784f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53430/0 (socket says 172.21.15.5:53430) 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.690+0000 7fdcb170a640 1 -- 172.21.15.5:0/1749974872 learned_addr learned my addr 172.21.15.5:0/1749974872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdcb270c640 1 --2- 172.21.15.5:0/1749974872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a197843b40 0x55a197842090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdcb170a640 1 -- 172.21.15.5:0/1749974872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a197843b40 msgr2=0x55a197842090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdcb170a640 1 --2- 172.21.15.5:0/1749974872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a197843b40 0x55a197842090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdcb170a640 1 -- 172.21.15.5:0/1749974872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1978425d0 msgr2=0x55a197842a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdcb170a640 1 --2- 172.21.15.5:0/1749974872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1978425d0 0x55a197842a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdcb170a640 1 -- 172.21.15.5:0/1749974872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1977343a0 con 0x55a19784d370 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdca2ffd640 1 --2- 172.21.15.5:0/386341492 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdc980a33b0 0x7fdc980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdcb270c640 1 --2- 172.21.15.5:0/386341492 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdc980a33b0 0x7fdc980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdcb170a640 1 --2- 172.21.15.5:0/1749974872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a19784d370 0x55a19784f760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009c90 tx=0x7fdc9c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=518b715b6d51f131 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdca2ffd640 1 -- 172.21.15.5:0/386341492 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fdca80f0030 con 0x7fdcac101de0 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdc97fff640 1 -- 172.21.15.5:0/1749974872 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c01e070 con 0x55a19784d370 2024-09-19T06:16:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdc97fff640 1 -- 172.21.15.5:0/1749974872 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc9c019ae0 con 0x55a19784d370 2024-09-19T06:16:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.691+0000 7fdc97fff640 1 -- 172.21.15.5:0/1749974872 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00e960 con 0x55a19784d370 2024-09-19T06:16:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.692+0000 7fdcb6548740 1 -- 172.21.15.5:0/1749974872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a19784d370 msgr2=0x55a19784f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.692+0000 7fdcb270c640 1 --2- 172.21.15.5:0/386341492 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdc980a33b0 0x7fdc980a5870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdca4005fd0 tx=0x7fdca4009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.692+0000 7fdcb6548740 1 --2- 172.21.15.5:0/1749974872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a19784d370 0x55a19784f760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009c90 tx=0x7fdc9c011810 comp rx=0 tx=0).stop 2024-09-19T06:16:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.692+0000 7fdcb6548740 1 -- 172.21.15.5:0/1749974872 shutdown_connections 2024-09-19T06:16:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.692+0000 7fdcb6548740 1 --2- 172.21.15.5:0/1749974872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a19784d370 0x55a19784f760 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.692+0000 7fdcb6548740 1 --2- 172.21.15.5:0/1749974872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1978425d0 0x55a197842a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.692+0000 7fdcb6548740 1 --2- 172.21.15.5:0/1749974872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a197843b40 0x55a197842090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.692+0000 7fdcb6548740 1 -- 172.21.15.5:0/1749974872 >> 172.21.15.5:0/1749974872 conn(0x55a19780b310 msgr2=0x55a19783ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.692+0000 7fdcb6548740 1 -- 172.21.15.5:0/1749974872 shutdown_connections 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.692+0000 7fdcb6548740 1 -- 172.21.15.5:0/1749974872 wait complete. 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.693+0000 7fdcb6548740 1 -- start start 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.693+0000 7fdcb6548740 1 Processor -- start 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.693+0000 7fdcb6548740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1978425d0 0x55a19784cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.693+0000 7fdcb6548740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a197843b40 0x55a19784a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.693+0000 7fdcb6548740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19784d370 0x55a19784a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.693+0000 7fdcb6548740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a197850070 con 0x55a197843b40 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.693+0000 7fdcb6548740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a197763060 con 0x55a19784d370 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.693+0000 7fdcb6548740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a1978501f0 con 0x55a1978425d0 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.693+0000 7fdcb6548740 10 client.0 _ll_register_callbacks cb 0x55a1974b68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.693+0000 7fdcb170a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a197843b40 0x55a19784a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.693+0000 7fdcb1f0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1978425d0 0x55a19784cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdcb1f0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1978425d0 0x55a19784cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53438/0 (socket says 172.21.15.5:53438) 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdcb1f0b640 1 -- 172.21.15.5:0/506193757 learned_addr learned my addr 172.21.15.5:0/506193757 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdcb270c640 1 --2- 172.21.15.5:0/506193757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19784d370 0x55a19784a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdcb1f0b640 1 -- 172.21.15.5:0/506193757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19784d370 msgr2=0x55a19784a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdcb1f0b640 1 --2- 172.21.15.5:0/506193757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19784d370 0x55a19784a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdcb1f0b640 1 -- 172.21.15.5:0/506193757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a197843b40 msgr2=0x55a19784a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdcb1f0b640 1 --2- 172.21.15.5:0/506193757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a197843b40 0x55a19784a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdcb1f0b640 1 -- 172.21.15.5:0/506193757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1975193c0 con 0x55a1978425d0 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdcb1f0b640 1 --2- 172.21.15.5:0/506193757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1978425d0 0x55a19784cd60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdca8000c00 tx=0x7fdca8009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75be59ddf1045531 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdc957fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdc957fa640 1 -- 172.21.15.5:0/506193757 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8049070 con 0x55a1978425d0 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdc957fa640 1 -- 172.21.15.5:0/506193757 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fdca80b8a10 con 0x55a1978425d0 2024-09-19T06:16:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.694+0000 7fdcb6548740 1 -- 172.21.15.5:0/506193757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc9c019ae0 con 0x55a1978425d0 2024-09-19T06:16:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.695+0000 7fdc957fa640 1 -- 172.21.15.5:0/506193757 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdca80bbee0 con 0x55a1978425d0 2024-09-19T06:16:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.695+0000 7fdcb6548740 10 client.28315 fetch_fsmap learned FSMap version 40 2024-09-19T06:16:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.695+0000 7fdcb6548740 1 -- 172.21.15.5:0/506193757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a1978500d0 con 0x55a1978425d0 2024-09-19T06:16:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.695+0000 7fdc957fa640 1 -- 172.21.15.5:0/506193757 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc9c019ae0 con 0x55a1978425d0 2024-09-19T06:16:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.695+0000 7fdc957fa640 1 -- 172.21.15.5:0/506193757 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fdca804c060 con 0x55a1978425d0 2024-09-19T06:16:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.695+0000 7fdcb6548740 10 client.28315 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T06:16:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.695+0000 7fdcb6548740 10 client.28315 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:16:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.695+0000 7fdcb6548740 20 client.28315 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.695+0000 7fdcb6548740 1 --2- 172.21.15.5:0/506193757 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55a1978658c0 0x55a197867cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.695+0000 7fdcb6548740 4 client.28315 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.695+0000 7fdcb6548740 1 -- 172.21.15.5:0/506193757 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1977b5860 con 0x55a1978658c0 2024-09-19T06:16:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.696+0000 7fdcb170a640 1 --2- 172.21.15.5:0/506193757 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55a1978658c0 0x55a197867cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.696+0000 7fdc957fa640 10 client.28315 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:16:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.696+0000 7fdcb170a640 1 --2- 172.21.15.5:0/506193757 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55a1978658c0 0x55a197867cb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27860ec1864df446 server_cookie=ac0bda2bcc1410bd in_seq=0 out_seq=0 2024-09-19T06:16:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.697+0000 7fdc957fa640 1 -- 172.21.15.5:0/506193757 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1977b5860 con 0x55a1978658c0 2024-09-19T06:16:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.697+0000 7fdc957fa640 10 client.28315 handle_command_reply: tid=0 2024-09-19T06:16:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.697+0000 7fdcb6548740 1 client.28315 shutdown 2024-09-19T06:16:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.697+0000 7fdcb6548740 1 -- 172.21.15.5:0/506193757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1978425d0 msgr2=0x55a19784cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.697+0000 7fdcb6548740 1 --2- 172.21.15.5:0/506193757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1978425d0 0x55a19784cd60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdca8000c00 tx=0x7fdca8009620 comp rx=0 tx=0).stop 2024-09-19T06:16:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.697+0000 7fdcb6548740 1 -- 172.21.15.5:0/506193757 shutdown_connections 2024-09-19T06:16:45.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.698+0000 7fdcb6548740 1 --2- 172.21.15.5:0/506193757 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55a1978658c0 0x55a197867cb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.698+0000 7fdcb6548740 1 --2- 172.21.15.5:0/506193757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19784d370 0x55a19784a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.698+0000 7fdc957fa640 0 client.28315 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:16:45.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.698+0000 7fdcb6548740 1 --2- 172.21.15.5:0/506193757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a197843b40 0x55a19784a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.698+0000 7fdcb6548740 1 --2- 172.21.15.5:0/506193757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1978425d0 0x55a19784cd60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.698+0000 7fdcb6548740 1 -- 172.21.15.5:0/506193757 >> 172.21.15.5:0/506193757 conn(0x55a19780b310 msgr2=0x55a197844930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:45.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.698+0000 7fdcb6548740 1 -- 172.21.15.5:0/506193757 shutdown_connections 2024-09-19T06:16:45.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.698+0000 7fdcb6548740 1 -- 172.21.15.5:0/506193757 wait complete. 2024-09-19T06:16:45.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.698+0000 7fdcb6548740 20 client.28315 trim_cache size 0 max 16384 2024-09-19T06:16:45.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb6548740 1 Processor -- start 2024-09-19T06:16:45.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb6548740 1 -- start start 2024-09-19T06:16:45.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb6548740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1978425d0 0x55a1978498d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb6548740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a197843b40 0x55a19784bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb6548740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a19784d370 0x55a197845120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb6548740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac1ab6b0 con 0x55a19784d370 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb6548740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac1ad2b0 con 0x55a1978425d0 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb6548740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac1abd80 con 0x55a197843b40 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb270c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a19784d370 0x55a197845120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb1f0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1978425d0 0x55a1978498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb270c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a19784d370 0x55a197845120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60700/0 (socket says 172.21.15.5:60700) 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb1f0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1978425d0 0x55a1978498d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36432/0 (socket says 172.21.15.5:36432) 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb270c640 1 -- 172.21.15.5:0/1399595164 learned_addr learned my addr 172.21.15.5:0/1399595164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.716+0000 7fdcb170a640 1 --2- 172.21.15.5:0/1399595164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a197843b40 0x55a19784bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.717+0000 7fdcb270c640 1 -- 172.21.15.5:0/1399595164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a197843b40 msgr2=0x55a19784bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.717+0000 7fdcb270c640 1 --2- 172.21.15.5:0/1399595164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a197843b40 0x55a19784bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.717+0000 7fdcb270c640 1 -- 172.21.15.5:0/1399595164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1978425d0 msgr2=0x55a1978498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:45.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.717+0000 7fdcb270c640 1 --2- 172.21.15.5:0/1399595164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1978425d0 0x55a1978498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.717+0000 7fdcb270c640 1 -- 172.21.15.5:0/1399595164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19784d0d0 con 0x55a19784d370 2024-09-19T06:16:45.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.717+0000 7fdcb1f0b640 1 --2- 172.21.15.5:0/1399595164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1978425d0 0x55a1978498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:45.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.717+0000 7fdcb270c640 1 --2- 172.21.15.5:0/1399595164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a19784d370 0x55a197845120 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fdca4004060 tx=0x7fdca4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baea609f77345986 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.717+0000 7fdc977fe640 1 -- 172.21.15.5:0/1399595164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4004710 con 0x55a19784d370 2024-09-19T06:16:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.717+0000 7fdc977fe640 1 -- 172.21.15.5:0/1399595164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdca401d8d0 con 0x55a19784d370 2024-09-19T06:16:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.717+0000 7fdc977fe640 1 -- 172.21.15.5:0/1399595164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4020690 con 0x55a19784d370 2024-09-19T06:16:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.718+0000 7fdcb6548740 1 -- 172.21.15.5:0/1399595164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a19784d370 msgr2=0x55a197845120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.718+0000 7fdcb6548740 1 --2- 172.21.15.5:0/1399595164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a19784d370 0x55a197845120 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fdca4004060 tx=0x7fdca4017000 comp rx=0 tx=0).stop 2024-09-19T06:16:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.718+0000 7fdcb6548740 1 -- 172.21.15.5:0/1399595164 shutdown_connections 2024-09-19T06:16:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.718+0000 7fdcb6548740 1 --2- 172.21.15.5:0/1399595164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a19784d370 0x55a197845120 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.718+0000 7fdcb6548740 1 --2- 172.21.15.5:0/1399595164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a197843b40 0x55a19784bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.718+0000 7fdcb6548740 1 --2- 172.21.15.5:0/1399595164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1978425d0 0x55a1978498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.718+0000 7fdcb6548740 1 -- 172.21.15.5:0/1399595164 >> 172.21.15.5:0/1399595164 conn(0x55a19780b310 msgr2=0x55a19784b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.718+0000 7fdcb6548740 1 -- 172.21.15.5:0/1399595164 shutdown_connections 2024-09-19T06:16:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.718+0000 7fdcb6548740 1 -- 172.21.15.5:0/1399595164 wait complete. 2024-09-19T06:16:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.718+0000 7fdcb6548740 1 -- start start 2024-09-19T06:16:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb6548740 1 Processor -- start 2024-09-19T06:16:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb6548740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1978425d0 0x55a19785e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb6548740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a197843b40 0x55a197859860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb6548740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a19784d370 0x55a197859dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb6548740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac1acbe0 con 0x55a1978425d0 2024-09-19T06:16:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb6548740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac1ac4b0 con 0x55a197843b40 2024-09-19T06:16:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb6548740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac1a96f0 con 0x55a19784d370 2024-09-19T06:16:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb6548740 10 client.0 _ll_register_callbacks cb 0x7fdca80f3870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb1f0b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1978425d0 0x55a19785e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb1f0b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1978425d0 0x55a19785e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60702/0 (socket says 172.21.15.5:60702) 2024-09-19T06:16:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb170a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a197843b40 0x55a197859860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb1f0b640 1 -- 172.21.15.5:0/2273763768 learned_addr learned my addr 172.21.15.5:0/2273763768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.719+0000 7fdcb270c640 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a19784d370 0x55a197859dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdcb1f0b640 1 -- 172.21.15.5:0/2273763768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a19784d370 msgr2=0x55a197859dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdcb1f0b640 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a19784d370 0x55a197859dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdcb1f0b640 1 -- 172.21.15.5:0/2273763768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a197843b40 msgr2=0x55a197859860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdcb1f0b640 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a197843b40 0x55a197859860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdcb1f0b640 1 -- 172.21.15.5:0/2273763768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19785a5f0 con 0x55a1978425d0 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdcb170a640 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a197843b40 0x55a197859860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdcb1f0b640 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1978425d0 0x55a19785e4e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fdca80bc0a0 tx=0x7fdca80c06c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e66d478487571f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdc957fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdc957fa640 1 -- 172.21.15.5:0/2273763768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8049070 con 0x55a1978425d0 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdc957fa640 1 -- 172.21.15.5:0/2273763768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fdca80d5220 con 0x55a1978425d0 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdcb6548740 1 -- 172.21.15.5:0/2273763768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdca401d8d0 con 0x55a1978425d0 2024-09-19T06:16:45.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdc957fa640 1 -- 172.21.15.5:0/2273763768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdca80bbd00 con 0x55a1978425d0 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdcb6548740 10 client.18963 fetch_fsmap learned FSMap version 40 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdcb6548740 1 -- 172.21.15.5:0/2273763768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a1978d0530 con 0x55a1978425d0 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.720+0000 7fdc957fa640 1 -- 172.21.15.5:0/2273763768 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdca401d8d0 con 0x55a1978425d0 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.721+0000 7fdc957fa640 1 -- 172.21.15.5:0/2273763768 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fdca80bc600 con 0x55a1978425d0 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.721+0000 7fdcb6548740 10 client.18963 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.721+0000 7fdcb6548740 10 client.18963 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.721+0000 7fdcb6548740 20 client.18963 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.721+0000 7fdcb6548740 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55a1978d0af0 0x55a1978d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.721+0000 7fdcb6548740 4 client.18963 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.721+0000 7fdcb6548740 1 -- 172.21.15.5:0/2273763768 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1977afd70 con 0x55a1978d0af0 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.721+0000 7fdcb170a640 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55a1978d0af0 0x55a1978d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.721+0000 7fdc957fa640 10 client.18963 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:16:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.721+0000 7fdcb170a640 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55a1978d0af0 0x55a1978d2f10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0da00b16d626e87 server_cookie=5698212ae0f3f6da in_seq=0 out_seq=0 2024-09-19T06:16:45.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdc957fa640 1 -- 172.21.15.5:0/2273763768 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1977afd70 con 0x55a1978d0af0 2024-09-19T06:16:45.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdc957fa640 10 client.18963 handle_command_reply: tid=0 2024-09-19T06:16:45.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdcb6548740 1 client.18963 shutdown 2024-09-19T06:16:45.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdcb6548740 1 -- 172.21.15.5:0/2273763768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1978425d0 msgr2=0x55a19785e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdcb6548740 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1978425d0 0x55a19785e4e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fdca80bc0a0 tx=0x7fdca80c06c0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdcb6548740 1 -- 172.21.15.5:0/2273763768 shutdown_connections 2024-09-19T06:16:45.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdcb6548740 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55a1978d0af0 0x55a1978d2f10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdcb6548740 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a19784d370 0x55a197859dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdcb6548740 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a197843b40 0x55a197859860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdcb6548740 1 --2- 172.21.15.5:0/2273763768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1978425d0 0x55a19785e4e0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdcb6548740 1 -- 172.21.15.5:0/2273763768 >> 172.21.15.5:0/2273763768 conn(0x55a19780b310 msgr2=0x55a19783d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:45.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdc957fa640 0 client.18963 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:16:45.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdcb6548740 1 -- 172.21.15.5:0/2273763768 shutdown_connections 2024-09-19T06:16:45.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdcb6548740 1 -- 172.21.15.5:0/2273763768 wait complete. 2024-09-19T06:16:45.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.722+0000 7fdcb6548740 20 client.18963 trim_cache size 0 max 16384 2024-09-19T06:16:45.729 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:16:45.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.723+0000 7fdcb416f640 1 -- 172.21.15.5:0/386341492 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdc980a33b0 msgr2=0x7fdc980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.723+0000 7fdcb416f640 1 --2- 172.21.15.5:0/386341492 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdc980a33b0 0x7fdc980a5870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdca4005fd0 tx=0x7fdca4009500 comp rx=0 tx=0).stop 2024-09-19T06:16:45.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.723+0000 7fdcb416f640 1 -- 172.21.15.5:0/386341492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcac101de0 msgr2=0x7fdcac19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.723+0000 7fdcb416f640 1 --2- 172.21.15.5:0/386341492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcac101de0 0x7fdcac19a5d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdca8005ab0 tx=0x7fdca8003c10 comp rx=0 tx=0).stop 2024-09-19T06:16:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.723+0000 7fdcb416f640 1 -- 172.21.15.5:0/386341492 shutdown_connections 2024-09-19T06:16:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.723+0000 7fdcb416f640 1 --2- 172.21.15.5:0/386341492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcac106f10 0x7fdcac19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.723+0000 7fdcb416f640 1 --2- 172.21.15.5:0/386341492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcac102790 0x7fdcac19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.723+0000 7fdcb416f640 1 --2- 172.21.15.5:0/386341492 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdc980a33b0 0x7fdc980a5870 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.723+0000 7fdcb416f640 1 --2- 172.21.15.5:0/386341492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcac101de0 0x7fdcac19a5d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.723+0000 7fdcb416f640 1 -- 172.21.15.5:0/386341492 >> 172.21.15.5:0/386341492 conn(0x7fdcac0fd8c0 msgr2=0x7fdcac0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.723+0000 7fdcb416f640 1 -- 172.21.15.5:0/386341492 shutdown_connections 2024-09-19T06:16:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:45.723+0000 7fdcb416f640 1 -- 172.21.15.5:0/386341492 wait complete. 2024-09-19T06:16:46.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:16:46.548 INFO:tasks.workunit.client.0.smithi005.stdout: 10 1573 26.14 MB/sec warmup 10 sec latency 110.306 ms 2024-09-19T06:16:46.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:46 smithi005 ceph-mon[30955]: pgmap v420: 97 pgs: 97 active+clean; 365 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 18 MiB/s wr, 303 op/s 2024-09-19T06:16:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:46 smithi157 ceph-mon[35346]: pgmap v420: 97 pgs: 97 active+clean; 365 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 18 MiB/s wr, 303 op/s 2024-09-19T06:16:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:46 smithi049 ceph-mon[34640]: pgmap v420: 97 pgs: 97 active+clean; 365 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 18 MiB/s wr, 303 op/s 2024-09-19T06:16:47.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.290+0000 7f6a09a3b640 1 -- 172.21.15.5:0/782768421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a04106f10 msgr2=0x7f6a0410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:47.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.290+0000 7f6a09a3b640 1 --2- 172.21.15.5:0/782768421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a04106f10 0x7f6a0410ea90 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009ca0 tx=0x7f69fc02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.290+0000 7f6a09a3b640 1 -- 172.21.15.5:0/782768421 shutdown_connections 2024-09-19T06:16:47.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.290+0000 7f6a09a3b640 1 --2- 172.21.15.5:0/782768421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a04106f10 0x7f6a0410ea90 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.290+0000 7f6a09a3b640 1 --2- 172.21.15.5:0/782768421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a04102790 0x7f6a041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.290+0000 7f6a09a3b640 1 --2- 172.21.15.5:0/782768421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a04101de0 0x7f6a041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.290+0000 7f6a09a3b640 1 -- 172.21.15.5:0/782768421 >> 172.21.15.5:0/782768421 conn(0x7f6a040fd8c0 msgr2=0x7f6a040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:47.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.290+0000 7f6a09a3b640 1 -- 172.21.15.5:0/782768421 shutdown_connections 2024-09-19T06:16:47.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.290+0000 7f6a09a3b640 1 -- 172.21.15.5:0/782768421 wait complete. 2024-09-19T06:16:47.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.291+0000 7f6a09a3b640 1 Processor -- start 2024-09-19T06:16:47.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.291+0000 7f6a09a3b640 1 -- start start 2024-09-19T06:16:47.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.291+0000 7f6a09a3b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a04101de0 0x7f6a0410afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.291+0000 7f6a09a3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a04102790 0x7f6a0410b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.291+0000 7f6a09a3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a04106f10 0x7f6a041aa750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.291+0000 7f6a09a3b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a04075470 con 0x7f6a04102790 2024-09-19T06:16:47.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.291+0000 7f6a09a3b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6a040752f0 con 0x7f6a04106f10 2024-09-19T06:16:47.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.291+0000 7f6a09a3b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6a040755f0 con 0x7f6a04101de0 2024-09-19T06:16:47.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.291+0000 7f6a027fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a04102790 0x7f6a0410b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a027fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a04102790 0x7f6a0410b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60712/0 (socket says 172.21.15.5:60712) 2024-09-19T06:16:47.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a027fc640 1 -- 172.21.15.5:0/425625014 learned_addr learned my addr 172.21.15.5:0/425625014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:47.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a04101de0 0x7f6a0410afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a037fe640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a04106f10 0x7f6a041aa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a027fc640 1 -- 172.21.15.5:0/425625014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a04101de0 msgr2=0x7f6a0410afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:47.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a027fc640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a04101de0 0x7f6a0410afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a027fc640 1 -- 172.21.15.5:0/425625014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a04106f10 msgr2=0x7f6a041aa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:47.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a027fc640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a04106f10 0x7f6a041aa750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a027fc640 1 -- 172.21.15.5:0/425625014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a041aac90 con 0x7f6a04102790 2024-09-19T06:16:47.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a037fe640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a04106f10 0x7f6a041aa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:47.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a04101de0 0x7f6a0410afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:47.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a027fc640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a04102790 0x7f6a0410b4e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f69f800e9f0 tx=0x7f69f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:47.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a08a39640 1 -- 172.21.15.5:0/425625014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f800cd80 con 0x7f6a04102790 2024-09-19T06:16:47.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a08a39640 1 -- 172.21.15.5:0/425625014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69f80108e0 con 0x7f6a04102790 2024-09-19T06:16:47.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.292+0000 7f6a09a3b640 1 -- 172.21.15.5:0/425625014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a041aae60 con 0x7f6a04102790 2024-09-19T06:16:47.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.293+0000 7f6a08a39640 1 -- 172.21.15.5:0/425625014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8018920 con 0x7f6a04102790 2024-09-19T06:16:47.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.293+0000 7f6a09a3b640 1 -- 172.21.15.5:0/425625014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a041ab2b0 con 0x7f6a04102790 2024-09-19T06:16:47.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.296+0000 7f6a0be14740 1 Processor -- start 2024-09-19T06:16:47.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.296+0000 7f6a08a39640 1 -- 172.21.15.5:0/425625014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f69f8010a90 con 0x7f6a04102790 2024-09-19T06:16:47.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.296+0000 7f6a0be14740 1 -- start start 2024-09-19T06:16:47.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a0be14740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fdb40 0x5633bb0fc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a0be14740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fc5d0 0x5633bb0fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a0be14740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb107370 0x5633bb109760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a037fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fdb40 0x5633bb0fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fc5d0 0x5633bb0fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fc5d0 0x5633bb0fca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60722/0 (socket says 172.21.15.5:60722) 2024-09-19T06:16:47.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a02ffd640 1 -- 172.21.15.5:0/1446160964 learned_addr learned my addr 172.21.15.5:0/1446160964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:47.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a0be14740 1 -- 172.21.15.5:0/1446160964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633bb0622f0 con 0x5633bb0fc5d0 2024-09-19T06:16:47.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a0be14740 1 -- 172.21.15.5:0/1446160964 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5633bb009cf0 con 0x5633bb107370 2024-09-19T06:16:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a0be14740 1 -- 172.21.15.5:0/1446160964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5633bb05e2b0 con 0x5633bb0fdb40 2024-09-19T06:16:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a027fc640 1 --2- 172.21.15.5:0/1446160964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb107370 0x5633bb109760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a08a39640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69ec0a33b0 0x7f69ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a02ffd640 1 -- 172.21.15.5:0/1446160964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fdb40 msgr2=0x5633bb0fc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/1446160964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fdb40 0x5633bb0fc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a02ffd640 1 -- 172.21.15.5:0/1446160964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb107370 msgr2=0x5633bb109760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.297+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/1446160964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb107370 0x5633bb109760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.298+0000 7f6a02ffd640 1 -- 172.21.15.5:0/1446160964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633bafee3a0 con 0x5633bb0fc5d0 2024-09-19T06:16:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.298+0000 7f6a08a39640 1 -- 172.21.15.5:0/425625014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f69f80d0130 con 0x7f6a04102790 2024-09-19T06:16:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.298+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/1446160964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fc5d0 0x5633bb0fca10 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009d50 tx=0x7f69f40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9403a2b625528dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.298+0000 7f6a037fe640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69ec0a33b0 0x7f69ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.299+0000 7f6a037fe640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69ec0a33b0 0x7f69ec0a5870 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f69fc005e30 tx=0x7f69fc02a8c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.299+0000 7f69e9ffb640 1 -- 172.21.15.5:0/1446160964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4012650 con 0x5633bb0fc5d0 2024-09-19T06:16:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.299+0000 7f69e9ffb640 1 -- 172.21.15.5:0/1446160964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69f4009e40 con 0x5633bb0fc5d0 2024-09-19T06:16:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.299+0000 7f69e9ffb640 1 -- 172.21.15.5:0/1446160964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f400fb90 con 0x5633bb0fc5d0 2024-09-19T06:16:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.300+0000 7f6a0be14740 1 -- 172.21.15.5:0/1446160964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fc5d0 msgr2=0x5633bb0fca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.300+0000 7f6a0be14740 1 --2- 172.21.15.5:0/1446160964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fc5d0 0x5633bb0fca10 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009d50 tx=0x7f69f40117b0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.301+0000 7f6a0be14740 1 -- 172.21.15.5:0/1446160964 shutdown_connections 2024-09-19T06:16:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.301+0000 7f6a0be14740 1 --2- 172.21.15.5:0/1446160964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb107370 0x5633bb109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.301+0000 7f6a0be14740 1 --2- 172.21.15.5:0/1446160964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fc5d0 0x5633bb0fca10 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.301+0000 7f6a0be14740 1 --2- 172.21.15.5:0/1446160964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fdb40 0x5633bb0fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.301+0000 7f6a0be14740 1 -- 172.21.15.5:0/1446160964 >> 172.21.15.5:0/1446160964 conn(0x5633bb0c5310 msgr2=0x5633bb0f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.301+0000 7f6a0be14740 1 -- 172.21.15.5:0/1446160964 shutdown_connections 2024-09-19T06:16:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.301+0000 7f6a0be14740 1 -- 172.21.15.5:0/1446160964 wait complete. 2024-09-19T06:16:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.302+0000 7f6a0be14740 1 -- start start 2024-09-19T06:16:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.302+0000 7f6a0be14740 1 Processor -- start 2024-09-19T06:16:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.303+0000 7f6a0be14740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fc5d0 0x5633bb106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.303+0000 7f6a0be14740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fdb40 0x5633bb104070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.303+0000 7f6a0be14740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 0x5633bb104800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.303+0000 7f6a0be14740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633bb10a070 con 0x5633bb107370 2024-09-19T06:16:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.303+0000 7f6a0be14740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5633bb01d060 con 0x5633bb0fdb40 2024-09-19T06:16:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.303+0000 7f6a0be14740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5633bb10a1f0 con 0x5633bb0fc5d0 2024-09-19T06:16:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.303+0000 7f6a0be14740 10 client.0 _ll_register_callbacks cb 0x5633bad708c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.303+0000 7f6a037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 0x5633bb104800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.303+0000 7f6a027fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fdb40 0x5633bb104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.303+0000 7f6a037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 0x5633bb104800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60732/0 (socket says 172.21.15.5:60732) 2024-09-19T06:16:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.303+0000 7f6a037fe640 1 -- 172.21.15.5:0/1369856639 learned_addr learned my addr 172.21.15.5:0/1369856639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.303+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fc5d0 0x5633bb106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.304+0000 7f6a037fe640 1 -- 172.21.15.5:0/1369856639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fc5d0 msgr2=0x5633bb106dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.304+0000 7f6a037fe640 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fc5d0 0x5633bb106dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.304+0000 7f6a037fe640 1 -- 172.21.15.5:0/1369856639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fdb40 msgr2=0x5633bb104070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.304+0000 7f6a037fe640 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fdb40 0x5633bb104070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.304+0000 7f6a037fe640 1 -- 172.21.15.5:0/1369856639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633badd33c0 con 0x5633bb107370 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.304+0000 7f6a027fc640 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fdb40 0x5633bb104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.304+0000 7f6a037fe640 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 0x5633bb104800 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009ca0 tx=0x7f69fc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d404b995b4f853 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.304+0000 7f69e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.304+0000 7f69e2ffd640 1 -- 172.21.15.5:0/1369856639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc050030 con 0x5633bb107370 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.304+0000 7f6a0be14740 1 -- 172.21.15.5:0/1369856639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5633bb06f860 con 0x5633bb107370 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.304+0000 7f69e2ffd640 1 -- 172.21.15.5:0/1369856639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f69fc004250 con 0x5633bb107370 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.305+0000 7f69e2ffd640 1 -- 172.21.15.5:0/1369856639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f69fc004b40 con 0x5633bb107370 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.305+0000 7f6a0be14740 10 client.18975 fetch_fsmap learned FSMap version 41 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.305+0000 7f6a0be14740 1 -- 172.21.15.5:0/1369856639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5633bb115bd0 con 0x5633bb107370 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.305+0000 7f69e2ffd640 1 -- 172.21.15.5:0/1369856639 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5633bb06f860 con 0x5633bb107370 2024-09-19T06:16:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.305+0000 7f69e2ffd640 1 -- 172.21.15.5:0/1369856639 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f69fc049050 con 0x5633bb107370 2024-09-19T06:16:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.305+0000 7f6a0be14740 10 client.18975 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:16:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.305+0000 7f6a0be14740 10 client.18975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:16:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.306+0000 7f6a0be14740 20 client.18975 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.306+0000 7f6a0be14740 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5633bb117030 0x5633bb1194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.306+0000 7f6a0be14740 4 client.18975 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.306+0000 7f6a0be14740 1 -- 172.21.15.5:0/1369856639 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633bb069d70 con 0x5633bb117030 2024-09-19T06:16:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.306+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5633bb117030 0x5633bb1194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.307+0000 7f69e2ffd640 10 client.18975 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:16:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.307+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5633bb117030 0x5633bb1194d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5751e2b3a4af0433 server_cookie=e863987c23d5485a in_seq=0 out_seq=0 2024-09-19T06:16:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.308+0000 7f69e2ffd640 1 -- 172.21.15.5:0/1369856639 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633bb069d70 con 0x5633bb117030 2024-09-19T06:16:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.308+0000 7f69e2ffd640 10 client.18975 handle_command_reply: tid=0 2024-09-19T06:16:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.308+0000 7f6a0be14740 1 client.18975 shutdown 2024-09-19T06:16:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.309+0000 7f6a0be14740 1 -- 172.21.15.5:0/1369856639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 msgr2=0x5633bb104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.309+0000 7f6a0be14740 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 0x5633bb104800 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009ca0 tx=0x7f69fc0095c0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.310+0000 7f6a0be14740 1 -- 172.21.15.5:0/1369856639 shutdown_connections 2024-09-19T06:16:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.310+0000 7f6a0be14740 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5633bb117030 0x5633bb1194d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.310+0000 7f69e2ffd640 0 client.18975 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:16:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.310+0000 7f6a0be14740 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 0x5633bb104800 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.310+0000 7f6a0be14740 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fdb40 0x5633bb104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.310+0000 7f6a0be14740 1 --2- 172.21.15.5:0/1369856639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fc5d0 0x5633bb106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.310+0000 7f6a0be14740 1 -- 172.21.15.5:0/1369856639 >> 172.21.15.5:0/1369856639 conn(0x5633bb0c5310 msgr2=0x5633bb0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.310+0000 7f6a0be14740 1 -- 172.21.15.5:0/1369856639 shutdown_connections 2024-09-19T06:16:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.310+0000 7f6a0be14740 1 -- 172.21.15.5:0/1369856639 wait complete. 2024-09-19T06:16:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.311+0000 7f6a0be14740 20 client.18975 trim_cache size 0 max 16384 2024-09-19T06:16:47.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.328+0000 7f6a0be14740 1 Processor -- start 2024-09-19T06:16:47.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.328+0000 7f6a0be14740 1 -- start start 2024-09-19T06:16:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a0be14740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 0x5633bb119450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a0be14740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fdb40 0x5633bb0ff120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a0be14740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 0x5633bb0ff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a0be14740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041b3e30 con 0x5633bb107370 2024-09-19T06:16:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a0be14740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041b5a30 con 0x5633bb0fc5d0 2024-09-19T06:16:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a0be14740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041b4500 con 0x5633bb0fdb40 2024-09-19T06:16:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 0x5633bb0ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a02ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 0x5633bb119450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 0x5633bb0ff660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60740/0 (socket says 172.21.15.5:60740) 2024-09-19T06:16:47.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a02ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 0x5633bb119450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36488/0 (socket says 172.21.15.5:36488) 2024-09-19T06:16:47.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a037fe640 1 -- 172.21.15.5:0/2177861442 learned_addr learned my addr 172.21.15.5:0/2177861442 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:47.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a027fc640 1 --2- 172.21.15.5:0/2177861442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fdb40 0x5633bb0ff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a02ffd640 1 -- 172.21.15.5:0/2177861442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fdb40 msgr2=0x5633bb0ff120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:47.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/2177861442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fdb40 0x5633bb0ff120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.329+0000 7f6a02ffd640 1 -- 172.21.15.5:0/2177861442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 msgr2=0x5633bb0ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:47.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.330+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/2177861442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 0x5633bb0ff660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.330+0000 7f6a02ffd640 1 -- 172.21.15.5:0/2177861442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633bb109a90 con 0x5633bb0fc5d0 2024-09-19T06:16:47.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.330+0000 7f6a037fe640 1 --2- 172.21.15.5:0/2177861442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 0x5633bb0ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:47.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.330+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/2177861442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 0x5633bb119450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f69f400f830 tx=0x7f69f4014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e6ab0dca358fa35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:47.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.330+0000 7f69e97fa640 1 -- 172.21.15.5:0/2177861442 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f40052b0 con 0x5633bb0fc5d0 2024-09-19T06:16:47.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.330+0000 7f69e97fa640 1 -- 172.21.15.5:0/2177861442 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69f400f560 con 0x5633bb0fc5d0 2024-09-19T06:16:47.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.330+0000 7f69e97fa640 1 -- 172.21.15.5:0/2177861442 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4003850 con 0x5633bb0fc5d0 2024-09-19T06:16:47.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.331+0000 7f6a0be14740 1 -- 172.21.15.5:0/2177861442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 msgr2=0x5633bb119450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:47.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.331+0000 7f6a0be14740 1 --2- 172.21.15.5:0/2177861442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 0x5633bb119450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f69f400f830 tx=0x7f69f4014040 comp rx=0 tx=0).stop 2024-09-19T06:16:47.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.331+0000 7f6a0be14740 1 -- 172.21.15.5:0/2177861442 shutdown_connections 2024-09-19T06:16:47.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.331+0000 7f6a0be14740 1 --2- 172.21.15.5:0/2177861442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb107370 0x5633bb0ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.331+0000 7f6a0be14740 1 --2- 172.21.15.5:0/2177861442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb0fdb40 0x5633bb0ff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.331+0000 7f6a0be14740 1 --2- 172.21.15.5:0/2177861442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 0x5633bb119450 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.331+0000 7f6a0be14740 1 -- 172.21.15.5:0/2177861442 >> 172.21.15.5:0/2177861442 conn(0x5633bb0c5310 msgr2=0x5633bb0f3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.331+0000 7f6a0be14740 1 -- 172.21.15.5:0/2177861442 shutdown_connections 2024-09-19T06:16:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.331+0000 7f6a0be14740 1 -- 172.21.15.5:0/2177861442 wait complete. 2024-09-19T06:16:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a0be14740 1 -- start start 2024-09-19T06:16:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a0be14740 1 Processor -- start 2024-09-19T06:16:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a0be14740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 0x5633bb10f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a0be14740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fdb40 0x5633bb10fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a0be14740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb107370 0x5633bb100900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a0be14740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041b5360 con 0x5633bb0fdb40 2024-09-19T06:16:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a0be14740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041b4c30 con 0x5633bb0fc5d0 2024-09-19T06:16:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a0be14740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041b1e30 con 0x5633bb107370 2024-09-19T06:16:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a0be14740 10 client.0 _ll_register_callbacks cb 0x7f69f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a027fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fdb40 0x5633bb10fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a02ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 0x5633bb10f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a027fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fdb40 0x5633bb10fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60750/0 (socket says 172.21.15.5:60750) 2024-09-19T06:16:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.332+0000 7f6a027fc640 1 -- 172.21.15.5:0/669443039 learned_addr learned my addr 172.21.15.5:0/669443039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.333+0000 7f6a037fe640 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb107370 0x5633bb100900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.333+0000 7f6a02ffd640 1 -- 172.21.15.5:0/669443039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb107370 msgr2=0x5633bb100900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.333+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb107370 0x5633bb100900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.333+0000 7f6a02ffd640 1 -- 172.21.15.5:0/669443039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fdb40 msgr2=0x5633bb10fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.333+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fdb40 0x5633bb10fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.333+0000 7f6a02ffd640 1 -- 172.21.15.5:0/669443039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633bb101130 con 0x5633bb0fc5d0 2024-09-19T06:16:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.333+0000 7f6a037fe640 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb107370 0x5633bb100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.333+0000 7f6a02ffd640 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 0x5633bb10f710 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009c70 tx=0x7f69f4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8d5e7c923748ba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.333+0000 7f69e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:16:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.334+0000 7f69e2ffd640 1 -- 172.21.15.5:0/669443039 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f403c060 con 0x5633bb0fc5d0 2024-09-19T06:16:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.334+0000 7f69e2ffd640 1 -- 172.21.15.5:0/669443039 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f69f40052c0 con 0x5633bb0fc5d0 2024-09-19T06:16:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.334+0000 7f6a0be14740 1 -- 172.21.15.5:0/669443039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f400f560 con 0x5633bb0fc5d0 2024-09-19T06:16:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.334+0000 7f69e2ffd640 1 -- 172.21.15.5:0/669443039 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f69f400f860 con 0x5633bb0fc5d0 2024-09-19T06:16:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.334+0000 7f6a0be14740 10 client.28379 fetch_fsmap learned FSMap version 41 2024-09-19T06:16:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.334+0000 7f6a0be14740 1 -- 172.21.15.5:0/669443039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5633bb1818b0 con 0x5633bb0fc5d0 2024-09-19T06:16:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.334+0000 7f69e2ffd640 1 -- 172.21.15.5:0/669443039 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69f400f560 con 0x5633bb0fc5d0 2024-09-19T06:16:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.334+0000 7f69e2ffd640 1 -- 172.21.15.5:0/669443039 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f69f402d070 con 0x5633bb0fc5d0 2024-09-19T06:16:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.334+0000 7f6a0be14740 10 client.28379 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:16:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.335+0000 7f6a0be14740 10 client.28379 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:16:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.335+0000 7f6a0be14740 20 client.28379 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.335+0000 7f6a0be14740 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5633bb117030 0x5633bb183d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.335+0000 7f6a0be14740 4 client.28379 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:16:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.335+0000 7f6a0be14740 1 -- 172.21.15.5:0/669443039 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633bafe6170 con 0x5633bb117030 2024-09-19T06:16:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.335+0000 7f6a027fc640 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5633bb117030 0x5633bb183d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f69e2ffd640 10 client.28379 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:16:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f6a027fc640 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5633bb117030 0x5633bb183d50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9da11c0e81eff4ec server_cookie=2b0df25a950d0d1f in_seq=0 out_seq=0 2024-09-19T06:16:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f69e2ffd640 1 -- 172.21.15.5:0/669443039 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633bafe6170 con 0x5633bb117030 2024-09-19T06:16:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f69e2ffd640 10 client.28379 handle_command_reply: tid=0 2024-09-19T06:16:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f6a0be14740 1 client.28379 shutdown 2024-09-19T06:16:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f6a0be14740 1 -- 172.21.15.5:0/669443039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 msgr2=0x5633bb10f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f6a0be14740 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 0x5633bb10f710 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009c70 tx=0x7f69f4000bc0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f6a0be14740 1 -- 172.21.15.5:0/669443039 shutdown_connections 2024-09-19T06:16:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f6a0be14740 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5633bb117030 0x5633bb183d50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f6a0be14740 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5633bb107370 0x5633bb100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f6a0be14740 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633bb0fdb40 0x5633bb10fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f6a0be14740 1 --2- 172.21.15.5:0/669443039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5633bb0fc5d0 0x5633bb10f710 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f6a0be14740 1 -- 172.21.15.5:0/669443039 >> 172.21.15.5:0/669443039 conn(0x5633bb0c5310 msgr2=0x5633bb1058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.336+0000 7f69e2ffd640 0 client.28379 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:16:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.337+0000 7f6a0be14740 1 -- 172.21.15.5:0/669443039 shutdown_connections 2024-09-19T06:16:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.337+0000 7f6a0be14740 1 -- 172.21.15.5:0/669443039 wait complete. 2024-09-19T06:16:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.337+0000 7f6a0be14740 20 client.28379 trim_cache size 0 max 16384 2024-09-19T06:16:47.346 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:16:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.337+0000 7f6a09a3b640 1 -- 172.21.15.5:0/425625014 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69ec0a33b0 msgr2=0x7f69ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.337+0000 7f6a09a3b640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69ec0a33b0 0x7f69ec0a5870 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f69fc005e30 tx=0x7f69fc02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.337+0000 7f6a09a3b640 1 -- 172.21.15.5:0/425625014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a04102790 msgr2=0x7f6a0410b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.337+0000 7f6a09a3b640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a04102790 0x7f6a0410b4e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f69f800e9f0 tx=0x7f69f800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.338+0000 7f6a09a3b640 1 -- 172.21.15.5:0/425625014 shutdown_connections 2024-09-19T06:16:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.338+0000 7f6a09a3b640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a04106f10 0x7f6a041aa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.338+0000 7f6a09a3b640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a04102790 0x7f6a0410b4e0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.338+0000 7f6a09a3b640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69ec0a33b0 0x7f69ec0a5870 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.338+0000 7f6a09a3b640 1 --2- 172.21.15.5:0/425625014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a04101de0 0x7f6a0410afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.338+0000 7f6a09a3b640 1 -- 172.21.15.5:0/425625014 >> 172.21.15.5:0/425625014 conn(0x7f6a040fd8c0 msgr2=0x7f6a040ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.338+0000 7f6a09a3b640 1 -- 172.21.15.5:0/425625014 shutdown_connections 2024-09-19T06:16:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:47.338+0000 7f6a09a3b640 1 -- 172.21.15.5:0/425625014 wait complete. 2024-09-19T06:16:47.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 1723 23.86 MB/sec warmup 11 sec latency 112.218 ms 2024-09-19T06:16:47.707 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:47 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:16:47.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:47 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:16:47.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:47 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:16:48.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:16:48.386 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:48.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 1868 21.99 MB/sec warmup 12 sec latency 93.355 ms 2024-09-19T06:16:48.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:48 smithi157 ceph-mon[35346]: pgmap v421: 97 pgs: 97 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 19 MiB/s wr, 373 op/s 2024-09-19T06:16:48.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:48 smithi005 ceph-mon[30955]: pgmap v421: 97 pgs: 97 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 19 MiB/s wr, 373 op/s 2024-09-19T06:16:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:48 smithi049 ceph-mon[34640]: pgmap v421: 97 pgs: 97 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 19 MiB/s wr, 373 op/s 2024-09-19T06:16:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.992+0000 7f25a16e1640 1 -- 172.21.15.5:0/408263330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259c1027a0 msgr2=0x7f259c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.992+0000 7f25a16e1640 1 --2- 172.21.15.5:0/408263330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259c1027a0 0x7f259c1067f0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f2590009900 tx=0x7f259002a410 comp rx=0 tx=0).stop 2024-09-19T06:16:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.992+0000 7f25a16e1640 1 -- 172.21.15.5:0/408263330 shutdown_connections 2024-09-19T06:16:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.992+0000 7f25a16e1640 1 --2- 172.21.15.5:0/408263330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f259c106f20 0x7f259c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.992+0000 7f25a16e1640 1 --2- 172.21.15.5:0/408263330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259c1027a0 0x7f259c1067f0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.992+0000 7f25a16e1640 1 --2- 172.21.15.5:0/408263330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c101df0 0x7f259c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.992+0000 7f25a16e1640 1 -- 172.21.15.5:0/408263330 >> 172.21.15.5:0/408263330 conn(0x7f259c0fd8f0 msgr2=0x7f259c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.992+0000 7f25a16e1640 1 -- 172.21.15.5:0/408263330 shutdown_connections 2024-09-19T06:16:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.992+0000 7f25a16e1640 1 -- 172.21.15.5:0/408263330 wait complete. 2024-09-19T06:16:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.993+0000 7f25a16e1640 1 Processor -- start 2024-09-19T06:16:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.993+0000 7f25a16e1640 1 -- start start 2024-09-19T06:16:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.993+0000 7f25a16e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c101df0 0x7f259c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.993+0000 7f25a16e1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f259c1027a0 0x7f259c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.993+0000 7f25a16e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259c106f20 0x7f259c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.993+0000 7f25a16e1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f259c111e30 con 0x7f259c106f20 2024-09-19T06:16:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.993+0000 7f25a16e1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f259c111cb0 con 0x7f259c1027a0 2024-09-19T06:16:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.993+0000 7f25a16e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f259c111fb0 con 0x7f259c101df0 2024-09-19T06:16:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.994+0000 7f259b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259c106f20 0x7f259c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.994+0000 7f259b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259c106f20 0x7f259c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60776/0 (socket says 172.21.15.5:60776) 2024-09-19T06:16:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.994+0000 7f259b7fe640 1 -- 172.21.15.5:0/1956191535 learned_addr learned my addr 172.21.15.5:0/1956191535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.994+0000 7f259affd640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c101df0 0x7f259c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.994+0000 7f259a7fc640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f259c1027a0 0x7f259c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.994+0000 7f259a7fc640 1 -- 172.21.15.5:0/1956191535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c101df0 msgr2=0x7f259c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.994+0000 7f259a7fc640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c101df0 0x7f259c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.994+0000 7f259a7fc640 1 -- 172.21.15.5:0/1956191535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259c106f20 msgr2=0x7f259c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.994+0000 7f259a7fc640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259c106f20 0x7f259c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.994+0000 7f259a7fc640 1 -- 172.21.15.5:0/1956191535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c19d300 con 0x7f259c1027a0 2024-09-19T06:16:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.994+0000 7f259affd640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c101df0 0x7f259c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.994+0000 7f259a7fc640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f259c1027a0 0x7f259c198890 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f259002a930 tx=0x7f2590031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.995+0000 7f2583fff640 1 -- 172.21.15.5:0/1956191535 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590035070 con 0x7f259c1027a0 2024-09-19T06:16:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.995+0000 7f2583fff640 1 -- 172.21.15.5:0/1956191535 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2590004480 con 0x7f259c1027a0 2024-09-19T06:16:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.995+0000 7f25a16e1640 1 -- 172.21.15.5:0/1956191535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259c19d590 con 0x7f259c1027a0 2024-09-19T06:16:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.995+0000 7f2583fff640 1 -- 172.21.15.5:0/1956191535 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590003d40 con 0x7f259c1027a0 2024-09-19T06:16:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.995+0000 7f25a16e1640 1 -- 172.21.15.5:0/1956191535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259c074420 con 0x7f259c1027a0 2024-09-19T06:16:49.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.998+0000 7f2583fff640 1 -- 172.21.15.5:0/1956191535 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f259003d760 con 0x7f259c1027a0 2024-09-19T06:16:49.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.998+0000 7f25a3aba740 1 Processor -- start 2024-09-19T06:16:49.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.998+0000 7f25a3aba740 1 -- start start 2024-09-19T06:16:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f25a3aba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd691b40 0x562bbd690090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f25a3aba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd690a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f25a3aba740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd6a3b80 0x562bbd6a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f25a3aba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562bbd5f62f0 con 0x562bbd6905d0 2024-09-19T06:16:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f25a3aba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562bbd59dcf0 con 0x562bbd691b40 2024-09-19T06:16:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f25a3aba740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562bbd5f22b0 con 0x562bbd6a3b80 2024-09-19T06:16:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f259a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd6a3b80 0x562bbd6a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f259a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd6a3b80 0x562bbd6a5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53542/0 (socket says 172.21.15.5:53542) 2024-09-19T06:16:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f259a7fc640 1 -- 172.21.15.5:0/2792615428 learned_addr learned my addr 172.21.15.5:0/2792615428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f259b7fe640 1 --2- 172.21.15.5:0/2792615428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd691b40 0x562bbd690090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f259affd640 1 --2- 172.21.15.5:0/2792615428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd690a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f2583fff640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25840a34b0 0x7f25840a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f259b7fe640 1 -- 172.21.15.5:0/2792615428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd6a3b80 msgr2=0x562bbd6a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f259b7fe640 1 --2- 172.21.15.5:0/2792615428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd6a3b80 0x562bbd6a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f259b7fe640 1 -- 172.21.15.5:0/2792615428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 msgr2=0x562bbd690a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f259b7fe640 1 --2- 172.21.15.5:0/2792615428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd690a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f259b7fe640 1 -- 172.21.15.5:0/2792615428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bbd5823a0 con 0x562bbd691b40 2024-09-19T06:16:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:48.999+0000 7f2583fff640 1 -- 172.21.15.5:0/1956191535 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f25900f4550 con 0x7f259c1027a0 2024-09-19T06:16:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.000+0000 7f259b7fe640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25840a34b0 0x7f25840a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.000+0000 7f259b7fe640 1 --2- 172.21.15.5:0/2792615428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd691b40 0x562bbd690090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2594011490 tx=0x7f2594011950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333c21769482079b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.000+0000 7f25817fa640 1 -- 172.21.15.5:0/2792615428 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594017050 con 0x562bbd691b40 2024-09-19T06:16:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.000+0000 7f25817fa640 1 -- 172.21.15.5:0/2792615428 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f259401bd80 con 0x562bbd691b40 2024-09-19T06:16:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.000+0000 7f25817fa640 1 -- 172.21.15.5:0/2792615428 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594026c30 con 0x562bbd691b40 2024-09-19T06:16:49.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.001+0000 7f25a3aba740 1 -- 172.21.15.5:0/2792615428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd691b40 msgr2=0x562bbd690090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:49.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.001+0000 7f25a3aba740 1 --2- 172.21.15.5:0/2792615428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd691b40 0x562bbd690090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2594011490 tx=0x7f2594011950 comp rx=0 tx=0).stop 2024-09-19T06:16:49.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.001+0000 7f259b7fe640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25840a34b0 0x7f25840a5970 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2594023000 tx=0x7f2594024000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:49.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.001+0000 7f25a3aba740 1 -- 172.21.15.5:0/2792615428 shutdown_connections 2024-09-19T06:16:49.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.001+0000 7f25a3aba740 1 --2- 172.21.15.5:0/2792615428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd6a3b80 0x562bbd6a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.001+0000 7f25a3aba740 1 --2- 172.21.15.5:0/2792615428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd690a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.001+0000 7f25a3aba740 1 --2- 172.21.15.5:0/2792615428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd691b40 0x562bbd690090 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.001+0000 7f25a3aba740 1 -- 172.21.15.5:0/2792615428 >> 172.21.15.5:0/2792615428 conn(0x562bbd659310 msgr2=0x562bbd68cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:49.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.001+0000 7f25a3aba740 1 -- 172.21.15.5:0/2792615428 shutdown_connections 2024-09-19T06:16:49.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.001+0000 7f25a3aba740 1 -- 172.21.15.5:0/2792615428 wait complete. 2024-09-19T06:16:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.001+0000 7f25a3aba740 1 -- start start 2024-09-19T06:16:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f25a3aba740 1 Processor -- start 2024-09-19T06:16:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f25a3aba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd6a3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f25a3aba740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd691b40 0x562bbd6a0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f25a3aba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 0x562bbd6a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f25a3aba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562bbd6a6880 con 0x562bbd6905d0 2024-09-19T06:16:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f25a3aba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562bbd5b1060 con 0x562bbd6a3b80 2024-09-19T06:16:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f25a3aba740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562bbd6a6a00 con 0x562bbd691b40 2024-09-19T06:16:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f25a3aba740 10 client.0 _ll_register_callbacks cb 0x562bbd3048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f259affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd6a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f259affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd6a3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60792/0 (socket says 172.21.15.5:60792) 2024-09-19T06:16:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f259a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd691b40 0x562bbd6a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f259affd640 1 -- 172.21.15.5:0/2520065085 learned_addr learned my addr 172.21.15.5:0/2520065085 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.002+0000 7f259b7fe640 1 --2- 172.21.15.5:0/2520065085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 0x562bbd6a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.003+0000 7f259affd640 1 -- 172.21.15.5:0/2520065085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd691b40 msgr2=0x562bbd6a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.003+0000 7f259affd640 1 --2- 172.21.15.5:0/2520065085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd691b40 0x562bbd6a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.003+0000 7f259affd640 1 -- 172.21.15.5:0/2520065085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 msgr2=0x562bbd6a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.003+0000 7f259affd640 1 --2- 172.21.15.5:0/2520065085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 0x562bbd6a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.003+0000 7f259affd640 1 -- 172.21.15.5:0/2520065085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bbd3673c0 con 0x562bbd6905d0 2024-09-19T06:16:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.003+0000 7f259affd640 1 --2- 172.21.15.5:0/2520065085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd6a3420 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f258c0129e0 tx=0x7f258c012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4696cfbe93e7d792 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.003+0000 7f257eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.003+0000 7f257eff5640 1 -- 172.21.15.5:0/2520065085 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c004050 con 0x562bbd6905d0 2024-09-19T06:16:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.003+0000 7f257eff5640 1 -- 172.21.15.5:0/2520065085 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f258c0041f0 con 0x562bbd6905d0 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.003+0000 7f25a3aba740 1 -- 172.21.15.5:0/2520065085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f259401bd80 con 0x562bbd6905d0 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.004+0000 7f257eff5640 1 -- 172.21.15.5:0/2520065085 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f258c0043f0 con 0x562bbd6905d0 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.004+0000 7f25a3aba740 10 client.18993 fetch_fsmap learned FSMap version 41 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.004+0000 7f25a3aba740 1 -- 172.21.15.5:0/2520065085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562bbd6b2290 con 0x562bbd6905d0 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.004+0000 7f257eff5640 1 -- 172.21.15.5:0/2520065085 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f259401bd80 con 0x562bbd6905d0 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.004+0000 7f257eff5640 1 -- 172.21.15.5:0/2520065085 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f258c003c40 con 0x562bbd6905d0 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.004+0000 7f25a3aba740 10 client.18993 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.004+0000 7f25a3aba740 10 client.18993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.004+0000 7f25a3aba740 20 client.18993 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.004+0000 7f25a3aba740 1 --2- 172.21.15.5:0/2520065085 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bbd6b3b40 0x562bbd6b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.004+0000 7f25a3aba740 4 client.18993 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.004+0000 7f25a3aba740 1 -- 172.21.15.5:0/2520065085 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bbd603860 con 0x562bbd6b3b40 2024-09-19T06:16:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.005+0000 7f259a7fc640 1 --2- 172.21.15.5:0/2520065085 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bbd6b3b40 0x562bbd6b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.005+0000 7f259a7fc640 1 --2- 172.21.15.5:0/2520065085 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bbd6b3b40 0x562bbd6b6000 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba667e1f542d5fff server_cookie=263aa1448d7955d0 in_seq=0 out_seq=0 2024-09-19T06:16:49.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.005+0000 7f257eff5640 10 client.18993 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:49.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.006+0000 7f257eff5640 1 -- 172.21.15.5:0/2520065085 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562bbd603860 con 0x562bbd6b3b40 2024-09-19T06:16:49.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.006+0000 7f257eff5640 10 client.18993 handle_command_reply: tid=0 2024-09-19T06:16:49.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.006+0000 7f25a3aba740 1 client.18993 shutdown 2024-09-19T06:16:49.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.007+0000 7f25a3aba740 1 -- 172.21.15.5:0/2520065085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 msgr2=0x562bbd6a3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:49.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.007+0000 7f25a3aba740 1 --2- 172.21.15.5:0/2520065085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd6a3420 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f258c0129e0 tx=0x7f258c012ea0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.007+0000 7f25a3aba740 1 -- 172.21.15.5:0/2520065085 shutdown_connections 2024-09-19T06:16:49.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.007+0000 7f25a3aba740 1 --2- 172.21.15.5:0/2520065085 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bbd6b3b40 0x562bbd6b6000 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.007+0000 7f25a3aba740 1 --2- 172.21.15.5:0/2520065085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 0x562bbd6a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.007+0000 7f25a3aba740 1 --2- 172.21.15.5:0/2520065085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd691b40 0x562bbd6a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.007+0000 7f25a3aba740 1 --2- 172.21.15.5:0/2520065085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd6a3420 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.007+0000 7f25a3aba740 1 -- 172.21.15.5:0/2520065085 >> 172.21.15.5:0/2520065085 conn(0x562bbd659310 msgr2=0x562bbd692930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:49.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.007+0000 7f257eff5640 0 client.18993 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:49.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.007+0000 7f25a3aba740 1 -- 172.21.15.5:0/2520065085 shutdown_connections 2024-09-19T06:16:49.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.007+0000 7f25a3aba740 1 -- 172.21.15.5:0/2520065085 wait complete. 2024-09-19T06:16:49.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.007+0000 7f25a3aba740 20 client.18993 trim_cache size 0 max 16384 2024-09-19T06:16:49.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.026+0000 7f25a3aba740 1 Processor -- start 2024-09-19T06:16:49.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.026+0000 7f25a3aba740 1 -- start start 2024-09-19T06:16:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.026+0000 7f25a3aba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd6afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.026+0000 7f25a3aba740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd691b40 0x562bbd69eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.026+0000 7f25a3aba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 0x562bbd69f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f25a3aba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f259c1ab5b0 con 0x562bbd6905d0 2024-09-19T06:16:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f25a3aba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f259c1ad1b0 con 0x562bbd6a3b80 2024-09-19T06:16:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f25a3aba740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f259c1abc80 con 0x562bbd691b40 2024-09-19T06:16:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f259affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd6afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f259affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd6afb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60802/0 (socket says 172.21.15.5:60802) 2024-09-19T06:16:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f259affd640 1 -- 172.21.15.5:0/3661336678 learned_addr learned my addr 172.21.15.5:0/3661336678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f259a7fc640 1 --2- 172.21.15.5:0/3661336678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd691b40 0x562bbd69eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f259b7fe640 1 --2- 172.21.15.5:0/3661336678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 0x562bbd69f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f259affd640 1 -- 172.21.15.5:0/3661336678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd691b40 msgr2=0x562bbd69eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f259affd640 1 --2- 172.21.15.5:0/3661336678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd691b40 0x562bbd69eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f259affd640 1 -- 172.21.15.5:0/3661336678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 msgr2=0x562bbd69f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f259affd640 1 --2- 172.21.15.5:0/3661336678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 0x562bbd69f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f259affd640 1 -- 172.21.15.5:0/3661336678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bbd69c2f0 con 0x562bbd6905d0 2024-09-19T06:16:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.027+0000 7f259affd640 1 --2- 172.21.15.5:0/3661336678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd6afb20 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f258c009710 tx=0x7f258c010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb400885c516579 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.028+0000 7f2580ff9640 1 -- 172.21.15.5:0/3661336678 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c029040 con 0x562bbd6905d0 2024-09-19T06:16:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.028+0000 7f2580ff9640 1 -- 172.21.15.5:0/3661336678 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f258c00f9b0 con 0x562bbd6905d0 2024-09-19T06:16:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.028+0000 7f2580ff9640 1 -- 172.21.15.5:0/3661336678 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c0053f0 con 0x562bbd6905d0 2024-09-19T06:16:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.029+0000 7f25a3aba740 1 -- 172.21.15.5:0/3661336678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 msgr2=0x562bbd6afb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.029+0000 7f25a3aba740 1 --2- 172.21.15.5:0/3661336678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd6afb20 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f258c009710 tx=0x7f258c010900 comp rx=0 tx=0).stop 2024-09-19T06:16:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.029+0000 7f25a3aba740 1 -- 172.21.15.5:0/3661336678 shutdown_connections 2024-09-19T06:16:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.029+0000 7f25a3aba740 1 --2- 172.21.15.5:0/3661336678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 0x562bbd69f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.029+0000 7f25a3aba740 1 --2- 172.21.15.5:0/3661336678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd691b40 0x562bbd69eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.029+0000 7f25a3aba740 1 --2- 172.21.15.5:0/3661336678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd6905d0 0x562bbd6afb20 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.029+0000 7f25a3aba740 1 -- 172.21.15.5:0/3661336678 >> 172.21.15.5:0/3661336678 conn(0x562bbd659310 msgr2=0x562bbd6a06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.029+0000 7f25a3aba740 1 -- 172.21.15.5:0/3661336678 shutdown_connections 2024-09-19T06:16:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.029+0000 7f25a3aba740 1 -- 172.21.15.5:0/3661336678 wait complete. 2024-09-19T06:16:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.029+0000 7f25a3aba740 1 -- start start 2024-09-19T06:16:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.029+0000 7f25a3aba740 1 Processor -- start 2024-09-19T06:16:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f25a3aba740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd6905d0 0x562bbd6a7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f25a3aba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd691b40 0x562bbd6a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f25a3aba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 0x562bbd71c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f25a3aba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f259c1acae0 con 0x562bbd691b40 2024-09-19T06:16:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f25a3aba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f259c1ac3b0 con 0x562bbd6a3b80 2024-09-19T06:16:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f25a3aba740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f259c1a95b0 con 0x562bbd6905d0 2024-09-19T06:16:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f25a3aba740 10 client.0 _ll_register_callbacks cb 0x7f258c012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f259a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd691b40 0x562bbd6a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f259a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd691b40 0x562bbd6a7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60812/0 (socket says 172.21.15.5:60812) 2024-09-19T06:16:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f259a7fc640 1 -- 172.21.15.5:0/3578154687 learned_addr learned my addr 172.21.15.5:0/3578154687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f259b7fe640 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 0x562bbd71c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f259affd640 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd6905d0 0x562bbd6a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f259a7fc640 1 -- 172.21.15.5:0/3578154687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd6905d0 msgr2=0x562bbd6a7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f259a7fc640 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd6905d0 0x562bbd6a7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f259a7fc640 1 -- 172.21.15.5:0/3578154687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 msgr2=0x562bbd71c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f259a7fc640 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 0x562bbd71c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.030+0000 7f259a7fc640 1 -- 172.21.15.5:0/3578154687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bbd71ce10 con 0x562bbd691b40 2024-09-19T06:16:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.031+0000 7f259a7fc640 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd691b40 0x562bbd6a7780 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2590004650 tx=0x7f259003df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b56459f8c95786 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.031+0000 7f257eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.031+0000 7f257eff5640 1 -- 172.21.15.5:0/3578154687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25900c0400 con 0x562bbd691b40 2024-09-19T06:16:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.031+0000 7f257eff5640 1 -- 172.21.15.5:0/3578154687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f25900f1ea0 con 0x562bbd691b40 2024-09-19T06:16:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.031+0000 7f25a3aba740 1 -- 172.21.15.5:0/3578154687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f258c00f9b0 con 0x562bbd691b40 2024-09-19T06:16:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.031+0000 7f257eff5640 1 -- 172.21.15.5:0/3578154687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f25900c0220 con 0x562bbd691b40 2024-09-19T06:16:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.031+0000 7f25a3aba740 10 client.18999 fetch_fsmap learned FSMap version 41 2024-09-19T06:16:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.031+0000 7f25a3aba740 1 -- 172.21.15.5:0/3578154687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562bbd71e0e0 con 0x562bbd691b40 2024-09-19T06:16:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.032+0000 7f257eff5640 1 -- 172.21.15.5:0/3578154687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25900c0fc0 con 0x562bbd691b40 2024-09-19T06:16:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.032+0000 7f257eff5640 1 -- 172.21.15.5:0/3578154687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f259003b0d0 con 0x562bbd691b40 2024-09-19T06:16:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.032+0000 7f25a3aba740 10 client.18999 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:16:49.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.032+0000 7f25a3aba740 10 client.18999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.032+0000 7f25a3aba740 20 client.18999 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.032+0000 7f25a3aba740 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bbd71e7f0 0x562bbd720c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.032+0000 7f25a3aba740 4 client.18999 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.032+0000 7f25a3aba740 1 -- 172.21.15.5:0/3578154687 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f258c00f9b0 con 0x562bbd71e7f0 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.032+0000 7f259affd640 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bbd71e7f0 0x562bbd720c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.032+0000 7f257eff5640 10 client.18999 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.032+0000 7f259affd640 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bbd71e7f0 0x562bbd720c10 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3849a6d475a9b3f6 server_cookie=c19eba49b138ccd7 in_seq=0 out_seq=0 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.033+0000 7f257eff5640 1 -- 172.21.15.5:0/3578154687 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f258c00f9b0 con 0x562bbd71e7f0 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.033+0000 7f257eff5640 10 client.18999 handle_command_reply: tid=0 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.033+0000 7f25a3aba740 1 client.18999 shutdown 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.033+0000 7f25a3aba740 1 -- 172.21.15.5:0/3578154687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd691b40 msgr2=0x562bbd6a7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.033+0000 7f25a3aba740 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd691b40 0x562bbd6a7780 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2590004650 tx=0x7f259003df10 comp rx=0 tx=0).stop 2024-09-19T06:16:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.033+0000 7f25a3aba740 1 -- 172.21.15.5:0/3578154687 shutdown_connections 2024-09-19T06:16:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.033+0000 7f25a3aba740 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bbd71e7f0 0x562bbd720c10 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.033+0000 7f25a3aba740 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bbd6a3b80 0x562bbd71c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.033+0000 7f25a3aba740 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bbd691b40 0x562bbd6a7780 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.033+0000 7f25a3aba740 1 --2- 172.21.15.5:0/3578154687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bbd6905d0 0x562bbd6a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.033+0000 7f25a3aba740 1 -- 172.21.15.5:0/3578154687 >> 172.21.15.5:0/3578154687 conn(0x562bbd659310 msgr2=0x562bbd69ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.033+0000 7f257eff5640 0 client.18999 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.034+0000 7f25a3aba740 1 -- 172.21.15.5:0/3578154687 shutdown_connections 2024-09-19T06:16:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.034+0000 7f25a3aba740 1 -- 172.21.15.5:0/3578154687 wait complete. 2024-09-19T06:16:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.034+0000 7f25a3aba740 20 client.18999 trim_cache size 0 max 16384 2024-09-19T06:16:49.040 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:16:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.034+0000 7f25a16e1640 1 -- 172.21.15.5:0/1956191535 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25840a34b0 msgr2=0x7f25840a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.034+0000 7f25a16e1640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25840a34b0 0x7f25840a5970 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2594023000 tx=0x7f2594024000 comp rx=0 tx=0).stop 2024-09-19T06:16:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.034+0000 7f25a16e1640 1 -- 172.21.15.5:0/1956191535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f259c1027a0 msgr2=0x7f259c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.035+0000 7f25a16e1640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f259c1027a0 0x7f259c198890 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f259002a930 tx=0x7f2590031a20 comp rx=0 tx=0).stop 2024-09-19T06:16:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.035+0000 7f25a16e1640 1 -- 172.21.15.5:0/1956191535 shutdown_connections 2024-09-19T06:16:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.035+0000 7f25a16e1640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259c106f20 0x7f259c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.035+0000 7f25a16e1640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f259c1027a0 0x7f259c198890 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.035+0000 7f25a16e1640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25840a34b0 0x7f25840a5970 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.035+0000 7f25a16e1640 1 --2- 172.21.15.5:0/1956191535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c101df0 0x7f259c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:49.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.035+0000 7f25a16e1640 1 -- 172.21.15.5:0/1956191535 >> 172.21.15.5:0/1956191535 conn(0x7f259c0fd8f0 msgr2=0x7f259c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:49.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.035+0000 7f25a16e1640 1 -- 172.21.15.5:0/1956191535 shutdown_connections 2024-09-19T06:16:49.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:49.035+0000 7f25a16e1640 1 -- 172.21.15.5:0/1956191535 wait complete. 2024-09-19T06:16:49.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 2027 20.45 MB/sec warmup 13 sec latency 205.017 ms 2024-09-19T06:16:50.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 2214 19.32 MB/sec warmup 14 sec latency 69.471 ms 2024-09-19T06:16:50.753 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:16:50.753 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:16:50.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:50 smithi005 ceph-mon[30955]: pgmap v422: 97 pgs: 97 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 17 MiB/s wr, 348 op/s 2024-09-19T06:16:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:50 smithi157 ceph-mon[35346]: pgmap v422: 97 pgs: 97 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 17 MiB/s wr, 348 op/s 2024-09-19T06:16:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:50 smithi049 ceph-mon[34640]: pgmap v422: 97 pgs: 97 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 17 MiB/s wr, 348 op/s 2024-09-19T06:16:51.116 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:51.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 2371 18.60 MB/sec warmup 15 sec latency 107.157 ms 2024-09-19T06:16:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.870+0000 7fc376105640 1 -- 172.21.15.5:0/803291863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc370100e20 msgr2=0x7fc370101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:51.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.870+0000 7fc376105640 1 --2- 172.21.15.5:0/803291863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc370100e20 0x7fc370101200 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc360009900 tx=0x7fc36002a480 comp rx=0 tx=0).stop 2024-09-19T06:16:51.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.870+0000 7fc376105640 1 -- 172.21.15.5:0/803291863 shutdown_connections 2024-09-19T06:16:51.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.870+0000 7fc376105640 1 --2- 172.21.15.5:0/803291863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc370102ef0 0x7fc37010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.870+0000 7fc376105640 1 --2- 172.21.15.5:0/803291863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc370102570 0x7fc3701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.870+0000 7fc376105640 1 --2- 172.21.15.5:0/803291863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc370100e20 0x7fc370101200 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.870+0000 7fc376105640 1 -- 172.21.15.5:0/803291863 >> 172.21.15.5:0/803291863 conn(0x7fc3700fc7d0 msgr2=0x7fc3700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:51.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.871+0000 7fc376105640 1 -- 172.21.15.5:0/803291863 shutdown_connections 2024-09-19T06:16:51.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.871+0000 7fc376105640 1 -- 172.21.15.5:0/803291863 wait complete. 2024-09-19T06:16:51.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.872+0000 7fc376105640 1 Processor -- start 2024-09-19T06:16:51.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.872+0000 7fc376105640 1 -- start start 2024-09-19T06:16:51.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.872+0000 7fc376105640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc370100e20 0x7fc37019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.872+0000 7fc376105640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc370102570 0x7fc37019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.872+0000 7fc376105640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc370102ef0 0x7fc37019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.872+0000 7fc376105640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc37010fd30 con 0x7fc370102570 2024-09-19T06:16:51.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.872+0000 7fc376105640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc37010fbb0 con 0x7fc370102ef0 2024-09-19T06:16:51.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.872+0000 7fc376105640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc37010feb0 con 0x7fc370100e20 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.873+0000 7fc36effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc370102570 0x7fc37019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.873+0000 7fc36effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc370102570 0x7fc37019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60822/0 (socket says 172.21.15.5:60822) 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.873+0000 7fc36effd640 1 -- 172.21.15.5:0/1485946003 learned_addr learned my addr 172.21.15.5:0/1485946003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.873+0000 7fc36f7fe640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc370100e20 0x7fc37019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.873+0000 7fc36ffff640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc370102ef0 0x7fc37019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.873+0000 7fc36effd640 1 -- 172.21.15.5:0/1485946003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc370100e20 msgr2=0x7fc37019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.873+0000 7fc36effd640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc370100e20 0x7fc37019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.873+0000 7fc36effd640 1 -- 172.21.15.5:0/1485946003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc370102ef0 msgr2=0x7fc37019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.873+0000 7fc36effd640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc370102ef0 0x7fc37019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.873+0000 7fc36effd640 1 -- 172.21.15.5:0/1485946003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37019f650 con 0x7fc370102570 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.873+0000 7fc36ffff640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc370102ef0 0x7fc37019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.873+0000 7fc36effd640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc370102570 0x7fc37019abe0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc36400e9f0 tx=0x7fc36400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.874+0000 7fc36cff9640 1 -- 172.21.15.5:0/1485946003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36400cd80 con 0x7fc370102570 2024-09-19T06:16:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.874+0000 7fc36cff9640 1 -- 172.21.15.5:0/1485946003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3640054d0 con 0x7fc370102570 2024-09-19T06:16:51.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.874+0000 7fc36cff9640 1 -- 172.21.15.5:0/1485946003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364010490 con 0x7fc370102570 2024-09-19T06:16:51.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.874+0000 7fc376105640 1 -- 172.21.15.5:0/1485946003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37019f940 con 0x7fc370102570 2024-09-19T06:16:51.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.874+0000 7fc376105640 1 -- 172.21.15.5:0/1485946003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3701a7220 con 0x7fc370102570 2024-09-19T06:16:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.877+0000 7fc3784de740 1 Processor -- start 2024-09-19T06:16:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.877+0000 7fc3784de740 1 -- start start 2024-09-19T06:16:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.877+0000 7fc36cff9640 1 -- 172.21.15.5:0/1485946003 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc364005020 con 0x7fc370102570 2024-09-19T06:16:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.877+0000 7fc3784de740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe620f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.877+0000 7fc3784de740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe621480 0x5647fe62b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.877+0000 7fc3784de740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe62bd40 0x5647fe62e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.877+0000 7fc3784de740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647fe586280 con 0x5647fe620b60 2024-09-19T06:16:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.877+0000 7fc3784de740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5647fe52db10 con 0x5647fe621480 2024-09-19T06:16:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.877+0000 7fc3784de740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5647fe582240 con 0x5647fe62bd40 2024-09-19T06:16:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe621480 0x5647fe62b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe621480 0x5647fe62b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36564/0 (socket says 172.21.15.5:36564) 2024-09-19T06:16:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36f7fe640 1 -- 172.21.15.5:0/3521538593 learned_addr learned my addr 172.21.15.5:0/3521538593 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36ffff640 1 --2- 172.21.15.5:0/3521538593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe620f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36effd640 1 --2- 172.21.15.5:0/3521538593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe62bd40 0x5647fe62e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36f7fe640 1 -- 172.21.15.5:0/3521538593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe62bd40 msgr2=0x5647fe62e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36f7fe640 1 --2- 172.21.15.5:0/3521538593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe62bd40 0x5647fe62e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36f7fe640 1 -- 172.21.15.5:0/3521538593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 msgr2=0x5647fe620f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36f7fe640 1 --2- 172.21.15.5:0/3521538593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe620f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36f7fe640 1 -- 172.21.15.5:0/3521538593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647fe53a1f0 con 0x5647fe621480 2024-09-19T06:16:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36ffff640 1 --2- 172.21.15.5:0/3521538593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe620f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36f7fe640 1 --2- 172.21.15.5:0/3521538593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe621480 0x5647fe62b800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc360009900 tx=0x7fc360005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4af047629e368d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.878+0000 7fc36cff9640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc3580a3500 0x7fc3580a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.879+0000 7fc36ffff640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc3580a3500 0x7fc3580a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.879+0000 7fc36cff9640 1 -- 172.21.15.5:0/1485946003 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fc3640cffd0 con 0x7fc370102570 2024-09-19T06:16:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.879+0000 7fc355ffb640 1 -- 172.21.15.5:0/3521538593 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360040070 con 0x5647fe621480 2024-09-19T06:16:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.879+0000 7fc355ffb640 1 -- 172.21.15.5:0/3521538593 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc360004030 con 0x5647fe621480 2024-09-19T06:16:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.879+0000 7fc355ffb640 1 -- 172.21.15.5:0/3521538593 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360035040 con 0x5647fe621480 2024-09-19T06:16:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.880+0000 7fc3784de740 1 -- 172.21.15.5:0/3521538593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe621480 msgr2=0x5647fe62b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.880+0000 7fc36ffff640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc3580a3500 0x7fc3580a59c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc368009fd0 tx=0x7fc368009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.880+0000 7fc3784de740 1 --2- 172.21.15.5:0/3521538593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe621480 0x5647fe62b800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc360009900 tx=0x7fc360005bd0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.880+0000 7fc3784de740 1 -- 172.21.15.5:0/3521538593 shutdown_connections 2024-09-19T06:16:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.880+0000 7fc3784de740 1 --2- 172.21.15.5:0/3521538593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe62bd40 0x5647fe62e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.880+0000 7fc3784de740 1 --2- 172.21.15.5:0/3521538593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe621480 0x5647fe62b800 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.880+0000 7fc3784de740 1 --2- 172.21.15.5:0/3521538593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe620f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.880+0000 7fc3784de740 1 -- 172.21.15.5:0/3521538593 >> 172.21.15.5:0/3521538593 conn(0x5647fe61d0f0 msgr2=0x5647fe61d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.880+0000 7fc3784de740 1 -- 172.21.15.5:0/3521538593 shutdown_connections 2024-09-19T06:16:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.880+0000 7fc3784de740 1 -- 172.21.15.5:0/3521538593 wait complete. 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.881+0000 7fc3784de740 1 -- start start 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.881+0000 7fc3784de740 1 Processor -- start 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.881+0000 7fc3784de740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe628b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.881+0000 7fc3784de740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe621480 0x5647fe6290c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.881+0000 7fc3784de740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe629600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.881+0000 7fc3784de740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647fe62e910 con 0x5647fe620b60 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.881+0000 7fc3784de740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5647fe540f10 con 0x5647fe62bd40 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.881+0000 7fc3784de740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5647fe62ea90 con 0x5647fe621480 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe628b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc3784de740 10 client.0 _ll_register_callbacks cb 0x5647fe509a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc36ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe629600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc36ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe629600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36572/0 (socket says 172.21.15.5:36572) 2024-09-19T06:16:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc36effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe621480 0x5647fe6290c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc36ffff640 1 -- 172.21.15.5:0/4121435633 learned_addr learned my addr 172.21.15.5:0/4121435633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc36ffff640 1 -- 172.21.15.5:0/4121435633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe621480 msgr2=0x5647fe6290c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc36ffff640 1 --2- 172.21.15.5:0/4121435633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe621480 0x5647fe6290c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc36ffff640 1 -- 172.21.15.5:0/4121435633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 msgr2=0x5647fe628b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc36ffff640 1 --2- 172.21.15.5:0/4121435633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe628b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc36ffff640 1 -- 172.21.15.5:0/4121435633 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647fe2f76a0 con 0x5647fe62bd40 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc36ffff640 1 --2- 172.21.15.5:0/4121435633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe629600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc368012910 tx=0x7fc368004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c8361a338f3b26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc34effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.882+0000 7fc34effd640 1 -- 172.21.15.5:0/4121435633 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368011c90 con 0x5647fe62bd40 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc34effd640 1 -- 172.21.15.5:0/4121435633 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fc36801f420 con 0x5647fe62bd40 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc3784de740 1 -- 172.21.15.5:0/4121435633 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc360004030 con 0x5647fe62bd40 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc34effd640 1 -- 172.21.15.5:0/4121435633 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc368011e30 con 0x5647fe62bd40 2024-09-19T06:16:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc3784de740 10 client.28409 fetch_fsmap learned FSMap version 41 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc3784de740 1 -- 172.21.15.5:0/4121435633 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5647fe642ce0 con 0x5647fe62bd40 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc34effd640 1 -- 172.21.15.5:0/4121435633 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc360004030 con 0x5647fe62bd40 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc34effd640 1 -- 172.21.15.5:0/4121435633 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fc368027070 con 0x5647fe62bd40 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc3784de740 10 client.28409 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc3784de740 10 client.28409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc3784de740 20 client.28409 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc3784de740 1 --2- 172.21.15.5:0/4121435633 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5647fe644630 0x5647fe646ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc3784de740 4 client.28409 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.883+0000 7fc3784de740 1 -- 172.21.15.5:0/4121435633 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647fe5937f0 con 0x5647fe644630 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.884+0000 7fc36f7fe640 1 --2- 172.21.15.5:0/4121435633 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5647fe644630 0x5647fe646ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.884+0000 7fc34effd640 10 client.28409 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.884+0000 7fc36f7fe640 1 --2- 172.21.15.5:0/4121435633 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5647fe644630 0x5647fe646ad0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81843386969708c6 server_cookie=73f6c0acc9edf834 in_seq=0 out_seq=0 2024-09-19T06:16:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc34effd640 1 -- 172.21.15.5:0/4121435633 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647fe5937f0 con 0x5647fe644630 2024-09-19T06:16:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc34effd640 10 client.28409 handle_command_reply: tid=0 2024-09-19T06:16:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc3784de740 1 client.28409 shutdown 2024-09-19T06:16:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc3784de740 1 -- 172.21.15.5:0/4121435633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 msgr2=0x5647fe629600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc3784de740 1 --2- 172.21.15.5:0/4121435633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe629600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc368012910 tx=0x7fc368004ab0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc3784de740 1 -- 172.21.15.5:0/4121435633 shutdown_connections 2024-09-19T06:16:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc3784de740 1 --2- 172.21.15.5:0/4121435633 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5647fe644630 0x5647fe646ad0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc3784de740 1 --2- 172.21.15.5:0/4121435633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe629600 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc34effd640 0 client.28409 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc3784de740 1 --2- 172.21.15.5:0/4121435633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe621480 0x5647fe6290c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc3784de740 1 --2- 172.21.15.5:0/4121435633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe628b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc3784de740 1 -- 172.21.15.5:0/4121435633 >> 172.21.15.5:0/4121435633 conn(0x5647fe61d0f0 msgr2=0x5647fe622e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.885+0000 7fc3784de740 1 -- 172.21.15.5:0/4121435633 shutdown_connections 2024-09-19T06:16:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.886+0000 7fc3784de740 1 -- 172.21.15.5:0/4121435633 wait complete. 2024-09-19T06:16:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.886+0000 7fc3784de740 20 client.28409 trim_cache size 0 max 16384 2024-09-19T06:16:51.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.911+0000 7fc3784de740 1 Processor -- start 2024-09-19T06:16:51.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.911+0000 7fc3784de740 1 -- start start 2024-09-19T06:16:51.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.911+0000 7fc3784de740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe620b60 0x5647fe6373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.911+0000 7fc3784de740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe621480 0x5647fe625de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.911+0000 7fc3784de740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe626320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.911+0000 7fc3784de740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc37006fd90 con 0x5647fe621480 2024-09-19T06:16:51.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.911+0000 7fc3784de740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc370071990 con 0x5647fe62bd40 2024-09-19T06:16:51.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.911+0000 7fc3784de740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc370070460 con 0x5647fe620b60 2024-09-19T06:16:51.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.912+0000 7fc36effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe621480 0x5647fe625de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.912+0000 7fc36effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe621480 0x5647fe625de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60844/0 (socket says 172.21.15.5:60844) 2024-09-19T06:16:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.912+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe620b60 0x5647fe6373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.912+0000 7fc36effd640 1 -- 172.21.15.5:0/3953966511 learned_addr learned my addr 172.21.15.5:0/3953966511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.912+0000 7fc36ffff640 1 --2- 172.21.15.5:0/3953966511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe626320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.912+0000 7fc36f7fe640 1 -- 172.21.15.5:0/3953966511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 msgr2=0x5647fe626320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.912+0000 7fc36f7fe640 1 --2- 172.21.15.5:0/3953966511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe626320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.912+0000 7fc36f7fe640 1 -- 172.21.15.5:0/3953966511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe621480 msgr2=0x5647fe625de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.912+0000 7fc36f7fe640 1 --2- 172.21.15.5:0/3953966511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe621480 0x5647fe625de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.912+0000 7fc36f7fe640 1 -- 172.21.15.5:0/3953966511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647fe61cf20 con 0x5647fe620b60 2024-09-19T06:16:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.912+0000 7fc36effd640 1 --2- 172.21.15.5:0/3953966511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe621480 0x5647fe625de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.912+0000 7fc36f7fe640 1 --2- 172.21.15.5:0/3953966511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe620b60 0x5647fe6373f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc360005940 tx=0x7fc360032fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cc3a2a50cc92a81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.913+0000 7fc3557fa640 1 -- 172.21.15.5:0/3953966511 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360035040 con 0x5647fe620b60 2024-09-19T06:16:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.913+0000 7fc3557fa640 1 -- 172.21.15.5:0/3953966511 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3600318e0 con 0x5647fe620b60 2024-09-19T06:16:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.913+0000 7fc3557fa640 1 -- 172.21.15.5:0/3953966511 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360040040 con 0x5647fe620b60 2024-09-19T06:16:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.913+0000 7fc3784de740 1 -- 172.21.15.5:0/3953966511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe620b60 msgr2=0x5647fe6373f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.913+0000 7fc3784de740 1 --2- 172.21.15.5:0/3953966511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe620b60 0x5647fe6373f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc360005940 tx=0x7fc360032fb0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.914+0000 7fc3784de740 1 -- 172.21.15.5:0/3953966511 shutdown_connections 2024-09-19T06:16:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.914+0000 7fc3784de740 1 --2- 172.21.15.5:0/3953966511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe626320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.914+0000 7fc3784de740 1 --2- 172.21.15.5:0/3953966511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe621480 0x5647fe625de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.914+0000 7fc3784de740 1 --2- 172.21.15.5:0/3953966511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe620b60 0x5647fe6373f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.914+0000 7fc3784de740 1 -- 172.21.15.5:0/3953966511 >> 172.21.15.5:0/3953966511 conn(0x5647fe61d0f0 msgr2=0x5647fe636ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.914+0000 7fc3784de740 1 -- 172.21.15.5:0/3953966511 shutdown_connections 2024-09-19T06:16:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.914+0000 7fc3784de740 1 -- 172.21.15.5:0/3953966511 wait complete. 2024-09-19T06:16:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.914+0000 7fc3784de740 1 -- start start 2024-09-19T06:16:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.914+0000 7fc3784de740 1 Processor -- start 2024-09-19T06:16:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc3784de740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe649ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc3784de740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe621480 0x5647fe64a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc3784de740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe64aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc3784de740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3700712c0 con 0x5647fe620b60 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc3784de740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc370070b90 con 0x5647fe62bd40 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc3784de740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc37006dd90 con 0x5647fe621480 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc3784de740 10 client.0 _ll_register_callbacks cb 0x7fc368012840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe649ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe649ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60848/0 (socket says 172.21.15.5:60848) 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc36f7fe640 1 -- 172.21.15.5:0/617190434 learned_addr learned my addr 172.21.15.5:0/617190434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc36ffff640 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe64aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc36effd640 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe621480 0x5647fe64a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc36f7fe640 1 -- 172.21.15.5:0/617190434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe621480 msgr2=0x5647fe64a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:51.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc36f7fe640 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe621480 0x5647fe64a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc36f7fe640 1 -- 172.21.15.5:0/617190434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 msgr2=0x5647fe64aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc36f7fe640 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe64aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.915+0000 7fc36f7fe640 1 -- 172.21.15.5:0/617190434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647fe6ae2a0 con 0x5647fe620b60 2024-09-19T06:16:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.916+0000 7fc36f7fe640 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe649ff0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc360009850 tx=0x7fc360003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97ed571c5e6788a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.916+0000 7fc34effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.916+0000 7fc34effd640 1 -- 172.21.15.5:0/617190434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360056020 con 0x5647fe620b60 2024-09-19T06:16:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.916+0000 7fc3784de740 1 -- 172.21.15.5:0/617190434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5647fe58dd00 con 0x5647fe620b60 2024-09-19T06:16:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.916+0000 7fc34effd640 1 -- 172.21.15.5:0/617190434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fc360004040 con 0x5647fe620b60 2024-09-19T06:16:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.916+0000 7fc34effd640 1 -- 172.21.15.5:0/617190434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc360061050 con 0x5647fe620b60 2024-09-19T06:16:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.916+0000 7fc3784de740 10 client.19017 fetch_fsmap learned FSMap version 41 2024-09-19T06:16:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.916+0000 7fc3784de740 1 -- 172.21.15.5:0/617190434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5647fe6af590 con 0x5647fe620b60 2024-09-19T06:16:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.917+0000 7fc34effd640 1 -- 172.21.15.5:0/617190434 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5647fe58dd00 con 0x5647fe620b60 2024-09-19T06:16:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.917+0000 7fc34effd640 1 -- 172.21.15.5:0/617190434 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fc360005290 con 0x5647fe620b60 2024-09-19T06:16:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.917+0000 7fc3784de740 10 client.19017 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:16:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.917+0000 7fc3784de740 10 client.19017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.917+0000 7fc3784de740 20 client.19017 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.917+0000 7fc3784de740 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5647fe644630 0x5647fe6b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.917+0000 7fc3784de740 4 client.19017 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:16:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.917+0000 7fc3784de740 1 -- 172.21.15.5:0/617190434 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5647fe50a110 con 0x5647fe644630 2024-09-19T06:16:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.917+0000 7fc36effd640 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5647fe644630 0x5647fe6b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:51.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc36effd640 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5647fe644630 0x5647fe6b1920 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=530f83f3f3035c54 server_cookie=fab3369facc57d8b in_seq=0 out_seq=0 2024-09-19T06:16:51.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc34effd640 10 client.19017 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc34effd640 1 -- 172.21.15.5:0/617190434 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5647fe50a110 con 0x5647fe644630 2024-09-19T06:16:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc34effd640 10 client.19017 handle_command_reply: tid=0 2024-09-19T06:16:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc3784de740 1 client.19017 shutdown 2024-09-19T06:16:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc3784de740 1 -- 172.21.15.5:0/617190434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 msgr2=0x5647fe649ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc3784de740 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe649ff0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc360009850 tx=0x7fc360003680 comp rx=0 tx=0).stop 2024-09-19T06:16:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc3784de740 1 -- 172.21.15.5:0/617190434 shutdown_connections 2024-09-19T06:16:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc3784de740 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5647fe644630 0x5647fe6b1920 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc3784de740 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647fe62bd40 0x5647fe64aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc3784de740 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647fe621480 0x5647fe64a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc3784de740 1 --2- 172.21.15.5:0/617190434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fe620b60 0x5647fe649ff0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc3784de740 1 -- 172.21.15.5:0/617190434 >> 172.21.15.5:0/617190434 conn(0x5647fe61d0f0 msgr2=0x5647fe629360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.918+0000 7fc34effd640 0 client.19017 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:51.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc3784de740 1 -- 172.21.15.5:0/617190434 shutdown_connections 2024-09-19T06:16:51.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc3784de740 1 -- 172.21.15.5:0/617190434 wait complete. 2024-09-19T06:16:51.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc3784de740 20 client.19017 trim_cache size 0 max 16384 2024-09-19T06:16:51.925 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:16:51.925 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:16:51.925 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "d6536895-eb1a-4746-8078-27514f77cce7", 2024-09-19T06:16:51.925 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:16:51.925 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:16:51.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc376105640 1 -- 172.21.15.5:0/1485946003 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc3580a3500 msgr2=0x7fc3580a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc376105640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc3580a3500 0x7fc3580a59c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc368009fd0 tx=0x7fc368009500 comp rx=0 tx=0).stop 2024-09-19T06:16:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc376105640 1 -- 172.21.15.5:0/1485946003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc370102570 msgr2=0x7fc37019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc376105640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc370102570 0x7fc37019abe0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc36400e9f0 tx=0x7fc36400edf0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc376105640 1 -- 172.21.15.5:0/1485946003 shutdown_connections 2024-09-19T06:16:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc376105640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc370102ef0 0x7fc37019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc376105640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc370102570 0x7fc37019abe0 secure :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc36400e9f0 tx=0x7fc36400edf0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc376105640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc3580a3500 0x7fc3580a59c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc376105640 1 --2- 172.21.15.5:0/1485946003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc370100e20 0x7fc37019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.919+0000 7fc376105640 1 -- 172.21.15.5:0/1485946003 >> 172.21.15.5:0/1485946003 conn(0x7fc3700fc7d0 msgr2=0x7fc3700fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.920+0000 7fc376105640 1 -- 172.21.15.5:0/1485946003 shutdown_connections 2024-09-19T06:16:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:51.920+0000 7fc376105640 1 -- 172.21.15.5:0/1485946003 wait complete. 2024-09-19T06:16:52.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 2551 18.41 MB/sec warmup 16 sec latency 105.407 ms 2024-09-19T06:16:52.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:52 smithi005 ceph-mon[30955]: pgmap v423: 97 pgs: 97 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 17 MiB/s wr, 348 op/s 2024-09-19T06:16:52.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:52 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:16:52.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:52 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:52.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:52 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:16:52.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:16:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:52 smithi157 ceph-mon[35346]: pgmap v423: 97 pgs: 97 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 17 MiB/s wr, 348 op/s 2024-09-19T06:16:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:52 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:16:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:52 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:52 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:16:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:52 smithi049 ceph-mon[34640]: pgmap v423: 97 pgs: 97 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 17 MiB/s wr, 348 op/s 2024-09-19T06:16:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:52 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:16:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:52 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:52 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:16:53.010 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:16:53.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 2765 18.01 MB/sec warmup 17 sec latency 136.313 ms 2024-09-19T06:16:53.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.563+0000 7fa5fad74640 1 -- 172.21.15.5:0/2948737209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5f4100e20 msgr2=0x7fa5f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:53.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.563+0000 7fa5fad74640 1 --2- 172.21.15.5:0/2948737209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5f4100e20 0x7fa5f4101200 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009900 tx=0x7fa5e802a480 comp rx=0 tx=0).stop 2024-09-19T06:16:53.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.563+0000 7fa5fad74640 1 -- 172.21.15.5:0/2948737209 shutdown_connections 2024-09-19T06:16:53.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.563+0000 7fa5fad74640 1 --2- 172.21.15.5:0/2948737209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5f4102ef0 0x7fa5f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.563+0000 7fa5fad74640 1 --2- 172.21.15.5:0/2948737209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f4102570 0x7fa5f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.563+0000 7fa5fad74640 1 --2- 172.21.15.5:0/2948737209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5f4100e20 0x7fa5f4101200 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.563+0000 7fa5fad74640 1 -- 172.21.15.5:0/2948737209 >> 172.21.15.5:0/2948737209 conn(0x7fa5f40fc7d0 msgr2=0x7fa5f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.563+0000 7fa5fad74640 1 -- 172.21.15.5:0/2948737209 shutdown_connections 2024-09-19T06:16:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.563+0000 7fa5fad74640 1 -- 172.21.15.5:0/2948737209 wait complete. 2024-09-19T06:16:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.564+0000 7fa5fad74640 1 Processor -- start 2024-09-19T06:16:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.564+0000 7fa5fad74640 1 -- start start 2024-09-19T06:16:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.564+0000 7fa5fad74640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5f4100e20 0x7fa5f410d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.564+0000 7fa5fad74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f4102570 0x7fa5f4109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.564+0000 7fa5fad74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5f4102ef0 0x7fa5f4109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.564+0000 7fa5fad74640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f410fd90 con 0x7fa5f4102ef0 2024-09-19T06:16:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.564+0000 7fa5f8b10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5f4100e20 0x7fa5f410d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.564+0000 7fa5fad74640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f410fc10 con 0x7fa5f4100e20 2024-09-19T06:16:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.564+0000 7fa5fad74640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f410ff10 con 0x7fa5f4102570 2024-09-19T06:16:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.564+0000 7fa5f8b10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5f4100e20 0x7fa5f410d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36618/0 (socket says 172.21.15.5:36618) 2024-09-19T06:16:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.564+0000 7fa5f8b10640 1 -- 172.21.15.5:0/1161462678 learned_addr learned my addr 172.21.15.5:0/1161462678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.564+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f4102570 0x7fa5f4109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.565+0000 7fa5f9311640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5f4102ef0 0x7fa5f4109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.565+0000 7fa5f8b10640 1 -- 172.21.15.5:0/1161462678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f4102570 msgr2=0x7fa5f4109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.565+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f4102570 0x7fa5f4109660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.565+0000 7fa5f8b10640 1 -- 172.21.15.5:0/1161462678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5f4102ef0 msgr2=0x7fa5f4109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.565+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5f4102ef0 0x7fa5f4109ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.565+0000 7fa5f8b10640 1 -- 172.21.15.5:0/1161462678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f410a160 con 0x7fa5f4100e20 2024-09-19T06:16:53.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.565+0000 7fa5f9311640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5f4102ef0 0x7fa5f4109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:53.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.565+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5f4100e20 0x7fa5f410d5c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa5e802aa70 tx=0x7fa5e8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:53.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.566+0000 7fa5f1ffb640 1 -- 172.21.15.5:0/1161462678 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e803f070 con 0x7fa5f4100e20 2024-09-19T06:16:53.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.566+0000 7fa5f1ffb640 1 -- 172.21.15.5:0/1161462678 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5e803bb20 con 0x7fa5f4100e20 2024-09-19T06:16:53.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.566+0000 7fa5f1ffb640 1 -- 172.21.15.5:0/1161462678 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8035030 con 0x7fa5f4100e20 2024-09-19T06:16:53.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.566+0000 7fa5fad74640 1 -- 172.21.15.5:0/1161462678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f410a3f0 con 0x7fa5f4100e20 2024-09-19T06:16:53.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.566+0000 7fa5fad74640 1 -- 172.21.15.5:0/1161462678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f41ab4d0 con 0x7fa5f4100e20 2024-09-19T06:16:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.569+0000 7fa5f1ffb640 1 -- 172.21.15.5:0/1161462678 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa5e8042070 con 0x7fa5f4100e20 2024-09-19T06:16:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.569+0000 7fa5fd14d740 1 Processor -- start 2024-09-19T06:16:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.570+0000 7fa5fd14d740 1 -- start start 2024-09-19T06:16:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.570+0000 7fa5fd14d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5faa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.570+0000 7fa5fd14d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 0x55eac5fb5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.570+0000 7fa5fd14d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5fb5680 0x55eac5fb7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.570+0000 7fa5fd14d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eac5f102e0 con 0x55eac5faa4a0 2024-09-19T06:16:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.570+0000 7fa5fd14d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eac5eb7ce0 con 0x55eac5faadc0 2024-09-19T06:16:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.570+0000 7fa5fd14d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eac5f0c2a0 con 0x55eac5fb5680 2024-09-19T06:16:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.570+0000 7fa5f8b10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 0x55eac5fb5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.570+0000 7fa5f8b10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 0x55eac5fb5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36626/0 (socket says 172.21.15.5:36626) 2024-09-19T06:16:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.570+0000 7fa5f8b10640 1 -- 172.21.15.5:0/1573131798 learned_addr learned my addr 172.21.15.5:0/1573131798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.570+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/1573131798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5fb5680 0x55eac5fb7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5f3fff640 1 -- 172.21.15.5:0/1573131798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 msgr2=0x55eac5fb5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/1573131798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 0x55eac5fb5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5f3fff640 1 -- 172.21.15.5:0/1573131798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 msgr2=0x55eac5faa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/1573131798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5faa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5f3fff640 1 -- 172.21.15.5:0/1573131798 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eac5e9c440 con 0x55eac5fb5680 2024-09-19T06:16:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5f1ffb640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa5e40a3430 0x7fa5e40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/1573131798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 0x55eac5fb5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5f1ffb640 1 -- 172.21.15.5:0/1161462678 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fa5e80f03f0 con 0x7fa5f4100e20 2024-09-19T06:16:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5f9311640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa5e40a3430 0x7fa5e40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/1573131798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5fb5680 0x55eac5fb7a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa5f410aeb0 tx=0x7fa5e0009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d619ff07310accf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5deffd640 1 -- 172.21.15.5:0/1573131798 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0017070 con 0x55eac5fb5680 2024-09-19T06:16:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5deffd640 1 -- 172.21.15.5:0/1573131798 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5e0013470 con 0x55eac5fb5680 2024-09-19T06:16:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.571+0000 7fa5deffd640 1 -- 172.21.15.5:0/1573131798 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0012630 con 0x55eac5fb5680 2024-09-19T06:16:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.572+0000 7fa5fd14d740 1 -- 172.21.15.5:0/1573131798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5fb5680 msgr2=0x55eac5fb7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.572+0000 7fa5f9311640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa5e40a3430 0x7fa5e40a58f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009b80 tx=0x7fa5ec009b10 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.572+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/1573131798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5fb5680 0x55eac5fb7a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa5f410aeb0 tx=0x7fa5e0009e00 comp rx=0 tx=0).stop 2024-09-19T06:16:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.572+0000 7fa5fd14d740 1 -- 172.21.15.5:0/1573131798 shutdown_connections 2024-09-19T06:16:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.572+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/1573131798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5fb5680 0x55eac5fb7a70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.572+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/1573131798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 0x55eac5fb5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.572+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/1573131798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5faa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.572+0000 7fa5fd14d740 1 -- 172.21.15.5:0/1573131798 >> 172.21.15.5:0/1573131798 conn(0x55eac5fa6a30 msgr2=0x55eac5fa6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.572+0000 7fa5fd14d740 1 -- 172.21.15.5:0/1573131798 shutdown_connections 2024-09-19T06:16:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.572+0000 7fa5fd14d740 1 -- 172.21.15.5:0/1573131798 wait complete. 2024-09-19T06:16:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.573+0000 7fa5fd14d740 1 -- start start 2024-09-19T06:16:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.573+0000 7fa5fd14d740 1 Processor -- start 2024-09-19T06:16:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5fd14d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faa4a0 0x55eac5fb4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5fd14d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5faadc0 0x55eac5fb2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5fd14d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5fb5680 0x55eac5fb27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5fd14d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eac5fb8250 con 0x55eac5fb5680 2024-09-19T06:16:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5fd14d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eac5ecb050 con 0x55eac5faa4a0 2024-09-19T06:16:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5fd14d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eac5fb83d0 con 0x55eac5faadc0 2024-09-19T06:16:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5fd14d740 10 client.0 _ll_register_callbacks cb 0x55eac5f0b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5f8b10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faa4a0 0x55eac5fb4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5f8b10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faa4a0 0x55eac5fb4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36632/0 (socket says 172.21.15.5:36632) 2024-09-19T06:16:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5f8b10640 1 -- 172.21.15.5:0/4264999755 learned_addr learned my addr 172.21.15.5:0/4264999755 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5faadc0 0x55eac5fb2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5f9311640 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5fb5680 0x55eac5fb27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5f8b10640 1 -- 172.21.15.5:0/4264999755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5faadc0 msgr2=0x55eac5fb2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5faadc0 0x55eac5fb2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5f8b10640 1 -- 172.21.15.5:0/4264999755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5fb5680 msgr2=0x55eac5fb27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5fb5680 0x55eac5fb27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.574+0000 7fa5f8b10640 1 -- 172.21.15.5:0/4264999755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eac5c813c0 con 0x55eac5faa4a0 2024-09-19T06:16:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.575+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faa4a0 0x55eac5fb4fc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80b8ab0 tx=0x7fa5e80b82f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f01eae9f1d2a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.575+0000 7fa5d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:16:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.575+0000 7fa5d3fff640 1 -- 172.21.15.5:0/4264999755 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8049030 con 0x55eac5faa4a0 2024-09-19T06:16:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.575+0000 7fa5fd14d740 1 -- 172.21.15.5:0/4264999755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e0013470 con 0x55eac5faa4a0 2024-09-19T06:16:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.575+0000 7fa5d3fff640 1 -- 172.21.15.5:0/4264999755 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fa5e80edd40 con 0x55eac5faa4a0 2024-09-19T06:16:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.575+0000 7fa5d3fff640 1 -- 172.21.15.5:0/4264999755 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa5e80bc190 con 0x55eac5faa4a0 2024-09-19T06:16:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.575+0000 7fa5fd14d740 10 client.28427 fetch_fsmap learned FSMap version 41 2024-09-19T06:16:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.575+0000 7fa5fd14d740 1 -- 172.21.15.5:0/4264999755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eac5fc3c00 con 0x55eac5faa4a0 2024-09-19T06:16:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.575+0000 7fa5d3fff640 1 -- 172.21.15.5:0/4264999755 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5e0013470 con 0x55eac5faa4a0 2024-09-19T06:16:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.575+0000 7fa5d3fff640 1 -- 172.21.15.5:0/4264999755 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fa5e80c4e30 con 0x55eac5faa4a0 2024-09-19T06:16:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.576+0000 7fa5fd14d740 10 client.28427 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.576+0000 7fa5fd14d740 10 client.28427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.576+0000 7fa5fd14d740 20 client.28427 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.576+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eac5fc52d0 0x55eac5fc7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.576+0000 7fa5fd14d740 4 client.28427 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.576+0000 7fa5fd14d740 1 -- 172.21.15.5:0/4264999755 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eac5f1d850 con 0x55eac5fc52d0 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.576+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eac5fc52d0 0x55eac5fc7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.576+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eac5fc52d0 0x55eac5fc7720 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b88d1c0b0e2686ac server_cookie=85d2c0898775e30 in_seq=0 out_seq=0 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.576+0000 7fa5d3fff640 10 client.28427 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.577+0000 7fa5d3fff640 1 -- 172.21.15.5:0/4264999755 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eac5f1d850 con 0x55eac5fc52d0 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.577+0000 7fa5d3fff640 10 client.28427 handle_command_reply: tid=0 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.577+0000 7fa5fd14d740 1 client.28427 shutdown 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.578+0000 7fa5fd14d740 1 -- 172.21.15.5:0/4264999755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faa4a0 msgr2=0x55eac5fb4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.578+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faa4a0 0x55eac5fb4fc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80b8ab0 tx=0x7fa5e80b82f0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.578+0000 7fa5fd14d740 1 -- 172.21.15.5:0/4264999755 shutdown_connections 2024-09-19T06:16:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.578+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eac5fc52d0 0x55eac5fc7720 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.578+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5fb5680 0x55eac5fb27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.578+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5faadc0 0x55eac5fb2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.578+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/4264999755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faa4a0 0x55eac5fb4fc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.578+0000 7fa5fd14d740 1 -- 172.21.15.5:0/4264999755 >> 172.21.15.5:0/4264999755 conn(0x55eac5fa6a30 msgr2=0x55eac5fac760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.578+0000 7fa5d3fff640 0 client.28427 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.578+0000 7fa5fd14d740 1 -- 172.21.15.5:0/4264999755 shutdown_connections 2024-09-19T06:16:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.578+0000 7fa5fd14d740 1 -- 172.21.15.5:0/4264999755 wait complete. 2024-09-19T06:16:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.578+0000 7fa5fd14d740 20 client.28427 trim_cache size 0 max 16384 2024-09-19T06:16:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5fd14d740 1 Processor -- start 2024-09-19T06:16:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5fd14d740 1 -- start start 2024-09-19T06:16:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5fd14d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5fa5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5fd14d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5faadc0 0x55eac5fcd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5fd14d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5fb5680 0x55eac5fcd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5fd14d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f41b3e80 con 0x55eac5faa4a0 2024-09-19T06:16:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5fd14d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f41b5a80 con 0x55eac5fb5680 2024-09-19T06:16:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5f3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5faadc0 0x55eac5fcd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5f8b10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5fa5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5f3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5faadc0 0x55eac5fcd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53644/0 (socket says 172.21.15.5:53644) 2024-09-19T06:16:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5f3fff640 1 -- 172.21.15.5:0/2037591810 learned_addr learned my addr 172.21.15.5:0/2037591810 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/2037591810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5fa5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60876/0 (socket says 172.21.15.5:60876) 2024-09-19T06:16:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5fd14d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f41b4550 con 0x55eac5faadc0 2024-09-19T06:16:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.609+0000 7fa5f9311640 1 --2- 172.21.15.5:0/2037591810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5fb5680 0x55eac5fcd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.610+0000 7fa5f8b10640 1 -- 172.21.15.5:0/2037591810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5faadc0 msgr2=0x55eac5fcd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.610+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/2037591810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5faadc0 0x55eac5fcd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.610+0000 7fa5f8b10640 1 -- 172.21.15.5:0/2037591810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5fb5680 msgr2=0x55eac5fcd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.610+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/2037591810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5fb5680 0x55eac5fcd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.610+0000 7fa5f8b10640 1 -- 172.21.15.5:0/2037591810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eac5fb7de0 con 0x55eac5faa4a0 2024-09-19T06:16:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.610+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/2037591810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5faadc0 0x55eac5fcd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.610+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/2037591810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5fa5a60 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8000c00 tx=0x7fa5e802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a817a32eaf6cf2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.610+0000 7fa5de7fc640 1 -- 172.21.15.5:0/2037591810 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8049070 con 0x55eac5faa4a0 2024-09-19T06:16:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.610+0000 7fa5de7fc640 1 -- 172.21.15.5:0/2037591810 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5e80c0a00 con 0x55eac5faa4a0 2024-09-19T06:16:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.610+0000 7fa5de7fc640 1 -- 172.21.15.5:0/2037591810 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80bbfb0 con 0x55eac5faa4a0 2024-09-19T06:16:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.611+0000 7fa5fd14d740 1 -- 172.21.15.5:0/2037591810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 msgr2=0x55eac5fa5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.611+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/2037591810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5fa5a60 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8000c00 tx=0x7fa5e802a480 comp rx=0 tx=0).stop 2024-09-19T06:16:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.611+0000 7fa5fd14d740 1 -- 172.21.15.5:0/2037591810 shutdown_connections 2024-09-19T06:16:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.611+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/2037591810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5fb5680 0x55eac5fcd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.611+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/2037591810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5faadc0 0x55eac5fcd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.611+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/2037591810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5fa5a60 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.611+0000 7fa5fd14d740 1 -- 172.21.15.5:0/2037591810 >> 172.21.15.5:0/2037591810 conn(0x55eac5fa6a30 msgr2=0x55eac5fa5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.612+0000 7fa5fd14d740 1 -- 172.21.15.5:0/2037591810 shutdown_connections 2024-09-19T06:16:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.612+0000 7fa5fd14d740 1 -- 172.21.15.5:0/2037591810 wait complete. 2024-09-19T06:16:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.612+0000 7fa5fd14d740 1 -- start start 2024-09-19T06:16:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.612+0000 7fa5fd14d740 1 Processor -- start 2024-09-19T06:16:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5fd14d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5fbda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5fd14d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 0x55eac5fbdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5fd14d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5fb5680 0x55eac5fbafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5fd14d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f41b53b0 con 0x55eac5faa4a0 2024-09-19T06:16:53.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5fd14d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f41b4c80 con 0x55eac5faadc0 2024-09-19T06:16:53.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5fd14d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f41b1e80 con 0x55eac5fb5680 2024-09-19T06:16:53.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5f8b10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5fbda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5fd14d740 10 client.0 _ll_register_callbacks cb 0x7fa5e80f3b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:53.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5f8b10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5fbda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60880/0 (socket says 172.21.15.5:60880) 2024-09-19T06:16:53.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5f8b10640 1 -- 172.21.15.5:0/3309181475 learned_addr learned my addr 172.21.15.5:0/3309181475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5f9311640 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5fb5680 0x55eac5fbafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 0x55eac5fbdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5f8b10640 1 -- 172.21.15.5:0/3309181475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5fb5680 msgr2=0x55eac5fbafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5fb5680 0x55eac5fbafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5f8b10640 1 -- 172.21.15.5:0/3309181475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 msgr2=0x55eac5fbdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 0x55eac5fbdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5f8b10640 1 -- 172.21.15.5:0/3309181475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eac5fbb7e0 con 0x55eac5faa4a0 2024-09-19T06:16:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5f9311640 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5fb5680 0x55eac5fbafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.613+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 0x55eac5fbdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:16:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.614+0000 7fa5f8b10640 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5fbda10 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80f3ee0 tx=0x7fa5e80c52c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f447c6dfd787e975 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:53.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.614+0000 7fa5d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:16:53.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.614+0000 7fa5d3fff640 1 -- 172.21.15.5:0/3309181475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80da060 con 0x55eac5faa4a0 2024-09-19T06:16:53.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.614+0000 7fa5d3fff640 1 -- 172.21.15.5:0/3309181475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fa5e803b870 con 0x55eac5faa4a0 2024-09-19T06:16:53.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.614+0000 7fa5fd14d740 1 -- 172.21.15.5:0/3309181475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e80c0a00 con 0x55eac5faa4a0 2024-09-19T06:16:53.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.614+0000 7fa5d3fff640 1 -- 172.21.15.5:0/3309181475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa5e80ca050 con 0x55eac5faa4a0 2024-09-19T06:16:53.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.614+0000 7fa5fd14d740 10 client.19032 fetch_fsmap learned FSMap version 41 2024-09-19T06:16:53.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.614+0000 7fa5fd14d740 1 -- 172.21.15.5:0/3309181475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eac5fbb7e0 con 0x55eac5faa4a0 2024-09-19T06:16:53.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.615+0000 7fa5d3fff640 1 -- 172.21.15.5:0/3309181475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5e80c0a00 con 0x55eac5faa4a0 2024-09-19T06:16:53.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.615+0000 7fa5d3fff640 1 -- 172.21.15.5:0/3309181475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fa5e80b83a0 con 0x55eac5faa4a0 2024-09-19T06:16:53.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.615+0000 7fa5fd14d740 10 client.19032 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:16:53.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.615+0000 7fa5fd14d740 10 client.19032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:53.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.615+0000 7fa5fd14d740 20 client.19032 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:53.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.615+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eac602ff60 0x55eac6032380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:53.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.615+0000 7fa5fd14d740 4 client.19032 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:16:53.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.615+0000 7fa5fd14d740 1 -- 172.21.15.5:0/3309181475 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eac5fb2b40 con 0x55eac602ff60 2024-09-19T06:16:53.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.615+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eac602ff60 0x55eac6032380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:53.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.616+0000 7fa5d3fff640 10 client.19032 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:53.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.616+0000 7fa5f3fff640 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eac602ff60 0x55eac6032380 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce571e7239d773f server_cookie=9151412395d45f5d in_seq=0 out_seq=0 2024-09-19T06:16:53.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.616+0000 7fa5d3fff640 1 -- 172.21.15.5:0/3309181475 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eac5fb2b40 con 0x55eac602ff60 2024-09-19T06:16:53.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.616+0000 7fa5d3fff640 10 client.19032 handle_command_reply: tid=0 2024-09-19T06:16:53.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.616+0000 7fa5fd14d740 1 client.19032 shutdown 2024-09-19T06:16:53.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.616+0000 7fa5fd14d740 1 -- 172.21.15.5:0/3309181475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 msgr2=0x55eac5fbda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:53.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.616+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5fbda10 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80f3ee0 tx=0x7fa5e80c52c0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.617+0000 7fa5fd14d740 1 -- 172.21.15.5:0/3309181475 shutdown_connections 2024-09-19T06:16:53.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.617+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eac602ff60 0x55eac6032380 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.617+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eac5fb5680 0x55eac5fbafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.617+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eac5faadc0 0x55eac5fbdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.617+0000 7fa5fd14d740 1 --2- 172.21.15.5:0/3309181475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eac5faa4a0 0x55eac5fbda10 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.617+0000 7fa5fd14d740 1 -- 172.21.15.5:0/3309181475 >> 172.21.15.5:0/3309181475 conn(0x55eac5fa6a30 msgr2=0x55eac5fa5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:53.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.617+0000 7fa5d3fff640 0 client.19032 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:53.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.617+0000 7fa5fd14d740 1 -- 172.21.15.5:0/3309181475 shutdown_connections 2024-09-19T06:16:53.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.617+0000 7fa5fd14d740 1 -- 172.21.15.5:0/3309181475 wait complete. 2024-09-19T06:16:53.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.617+0000 7fa5fd14d740 20 client.19032 trim_cache size 0 max 16384 2024-09-19T06:16:53.625 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:16:53.625 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:16:53.625 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:16:53.625 INFO:teuthology.orchestra.run.smithi005.stdout: "d6536895-eb1a-4746-8078-27514f77cce7": { 2024-09-19T06:16:53.625 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:16:53.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d6536895-eb1a-4746-8078-27514f77cce7", 2024-09-19T06:16:53.625 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:16:53.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:16:53.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:16:53.626 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:16:53.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.618+0000 7fa5fad74640 1 -- 172.21.15.5:0/1161462678 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa5e40a3430 msgr2=0x7fa5e40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:53.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.618+0000 7fa5fad74640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa5e40a3430 0x7fa5e40a58f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009b80 tx=0x7fa5ec009b10 comp rx=0 tx=0).stop 2024-09-19T06:16:53.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.618+0000 7fa5fad74640 1 -- 172.21.15.5:0/1161462678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5f4100e20 msgr2=0x7fa5f410d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:53.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.618+0000 7fa5fad74640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5f4100e20 0x7fa5f410d5c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa5e802aa70 tx=0x7fa5e8003c10 comp rx=0 tx=0).stop 2024-09-19T06:16:53.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.618+0000 7fa5fad74640 1 -- 172.21.15.5:0/1161462678 shutdown_connections 2024-09-19T06:16:53.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.618+0000 7fa5fad74640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5f4102ef0 0x7fa5f4109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.618+0000 7fa5fad74640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa5e40a3430 0x7fa5e40a58f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.618+0000 7fa5fad74640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f4102570 0x7fa5f4109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.618+0000 7fa5fad74640 1 --2- 172.21.15.5:0/1161462678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5f4100e20 0x7fa5f410d5c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:53.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.618+0000 7fa5fad74640 1 -- 172.21.15.5:0/1161462678 >> 172.21.15.5:0/1161462678 conn(0x7fa5f40fc7d0 msgr2=0x7fa5f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:53.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.618+0000 7fa5fad74640 1 -- 172.21.15.5:0/1161462678 shutdown_connections 2024-09-19T06:16:53.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:16:53.619+0000 7fa5fad74640 1 -- 172.21.15.5:0/1161462678 wait complete. 2024-09-19T06:16:53.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:53 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:16:53.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:53 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:16:53.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:53 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:53.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:53 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:53.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:53 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:53.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:53 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:16:53.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:53 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:16:53.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:53 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:53.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:53 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:53.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:53 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:53.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:53 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:16:53.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:53 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:16:53.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:53 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:53.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:53 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:53.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:53 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:16:54.401 INFO:tasks.cephfs.filesystem:scrub status for tag:d6536895-eb1a-4746-8078-27514f77cce7 - {'path': '/', 'tag': 'd6536895-eb1a-4746-8078-27514f77cce7', 'options': 'recursive,force'} 2024-09-19T06:16:54.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 3032 18.74 MB/sec warmup 18 sec latency 109.968 ms 2024-09-19T06:16:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.607+0000 7f371b64c640 1 -- 172.21.15.5:0/670357287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3714101ac0 msgr2=0x7f3714101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.607+0000 7f371b64c640 1 --2- 172.21.15.5:0/670357287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3714101ac0 0x7f3714101f20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3704009860 tx=0x7f370402a340 comp rx=0 tx=0).stop 2024-09-19T06:16:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.607+0000 7f371b64c640 1 -- 172.21.15.5:0/670357287 shutdown_connections 2024-09-19T06:16:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.607+0000 7f371b64c640 1 --2- 172.21.15.5:0/670357287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3714102460 0x7f371410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.607+0000 7f371b64c640 1 --2- 172.21.15.5:0/670357287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3714101ac0 0x7f3714101f20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.607+0000 7f371b64c640 1 --2- 172.21.15.5:0/670357287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3714107ad0 0x7f3714107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.607+0000 7f371b64c640 1 -- 172.21.15.5:0/670357287 >> 172.21.15.5:0/670357287 conn(0x7f37140fd7a0 msgr2=0x7f37140ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.607+0000 7f371b64c640 1 -- 172.21.15.5:0/670357287 shutdown_connections 2024-09-19T06:16:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.607+0000 7f371b64c640 1 -- 172.21.15.5:0/670357287 wait complete. 2024-09-19T06:16:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.608+0000 7f371b64c640 1 Processor -- start 2024-09-19T06:16:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.608+0000 7f371b64c640 1 -- start start 2024-09-19T06:16:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f371b64c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3714101ac0 0x7f3714196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f371b64c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3714102460 0x7f3714196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f371b64c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3714107ad0 0x7f371419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f371b64c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f371410fbd0 con 0x7f3714107ad0 2024-09-19T06:16:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f371b64c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f371410fa50 con 0x7f3714101ac0 2024-09-19T06:16:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f371b64c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f371410fd50 con 0x7f3714102460 2024-09-19T06:16:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f3719be9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3714107ad0 0x7f371419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f37193e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3714101ac0 0x7f3714196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f3719be9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3714107ad0 0x7f371419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58574/0 (socket says 172.21.15.5:58574) 2024-09-19T06:16:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f3718be7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3714102460 0x7f3714196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f3719be9640 1 -- 172.21.15.5:0/2161558628 learned_addr learned my addr 172.21.15.5:0/2161558628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f37193e8640 1 -- 172.21.15.5:0/2161558628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3714102460 msgr2=0x7f3714196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f37193e8640 1 --2- 172.21.15.5:0/2161558628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3714102460 0x7f3714196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f37193e8640 1 -- 172.21.15.5:0/2161558628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3714107ad0 msgr2=0x7f371419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f37193e8640 1 --2- 172.21.15.5:0/2161558628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3714107ad0 0x7f371419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.609+0000 7f37193e8640 1 -- 172.21.15.5:0/2161558628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371419b100 con 0x7f3714101ac0 2024-09-19T06:16:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.610+0000 7f37193e8640 1 --2- 172.21.15.5:0/2161558628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3714101ac0 0x7f3714196110 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f371000d6d0 tx=0x7f371000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.610+0000 7f370a7fc640 1 -- 172.21.15.5:0/2161558628 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710017070 con 0x7f3714101ac0 2024-09-19T06:16:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.610+0000 7f370a7fc640 1 -- 172.21.15.5:0/2161558628 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3710009820 con 0x7f3714101ac0 2024-09-19T06:16:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.610+0000 7f371b64c640 1 -- 172.21.15.5:0/2161558628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371419b3f0 con 0x7f3714101ac0 2024-09-19T06:16:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.610+0000 7f370a7fc640 1 -- 172.21.15.5:0/2161558628 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710004c40 con 0x7f3714101ac0 2024-09-19T06:16:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.610+0000 7f371b64c640 1 -- 172.21.15.5:0/2161558628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37141a2c80 con 0x7f3714101ac0 2024-09-19T06:16:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.613+0000 7f370a7fc640 1 -- 172.21.15.5:0/2161558628 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f37100099d0 con 0x7f3714101ac0 2024-09-19T06:16:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.613+0000 7f371e3e5740 1 Processor -- start 2024-09-19T06:16:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f371e3e5740 1 -- start start 2024-09-19T06:16:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f371e3e5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 0x55b6202eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f371e3e5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202ef410 0x55b6202f9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f371e3e5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202f9cb0 0x55b6202fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f370a7fc640 1 --2- 172.21.15.5:0/2161558628 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f37000a3430 0x7f37000a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f371e3e5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b6202532f0 con 0x55b6202eeaf0 2024-09-19T06:16:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f371e3e5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b6201fb270 con 0x55b6202ef410 2024-09-19T06:16:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f371e3e5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b62024f2b0 con 0x55b6202f9cb0 2024-09-19T06:16:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f3719be9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 0x55b6202eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f37193e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202ef410 0x55b6202f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f37193e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202ef410 0x55b6202f9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60618/0 (socket says 172.21.15.5:60618) 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f3719be9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 0x55b6202eeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58576/0 (socket says 172.21.15.5:58576) 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f37193e8640 1 -- 172.21.15.5:0/977555279 learned_addr learned my addr 172.21.15.5:0/977555279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f3719be9640 1 --2- 172.21.15.5:0/2161558628 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f37000a3430 0x7f37000a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f370a7fc640 1 -- 172.21.15.5:0/2161558628 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f37100cf4f0 con 0x7f3714101ac0 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f37193e8640 1 -- 172.21.15.5:0/977555279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202f9cb0 msgr2=0x55b6202fc0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f37193e8640 1 --2- 172.21.15.5:0/977555279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202f9cb0 0x55b6202fc0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f37193e8640 1 -- 172.21.15.5:0/977555279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 msgr2=0x55b6202eeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f37193e8640 1 --2- 172.21.15.5:0/977555279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 0x55b6202eeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.614+0000 7f37193e8640 1 -- 172.21.15.5:0/977555279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6201fa340 con 0x55b6202ef410 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.615+0000 7f37193e8640 1 --2- 172.21.15.5:0/977555279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202ef410 0x55b6202f9770 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3710021000 tx=0x7f371001e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a1bd020878b89cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.615+0000 7f3719be9640 1 --2- 172.21.15.5:0/2161558628 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f37000a3430 0x7f37000a58f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f370c009c50 tx=0x7f370c009a00 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.615+0000 7f36ff7fe640 1 -- 172.21.15.5:0/977555279 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710020070 con 0x55b6202ef410 2024-09-19T06:16:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.615+0000 7f36ff7fe640 1 -- 172.21.15.5:0/977555279 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f371009fb00 con 0x55b6202ef410 2024-09-19T06:16:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.615+0000 7f36ff7fe640 1 -- 172.21.15.5:0/977555279 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710099b90 con 0x55b6202ef410 2024-09-19T06:16:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.616+0000 7f371e3e5740 1 -- 172.21.15.5:0/977555279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202ef410 msgr2=0x55b6202f9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.616+0000 7f371e3e5740 1 --2- 172.21.15.5:0/977555279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202ef410 0x55b6202f9770 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3710021000 tx=0x7f371001e000 comp rx=0 tx=0).stop 2024-09-19T06:16:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.616+0000 7f371e3e5740 1 -- 172.21.15.5:0/977555279 shutdown_connections 2024-09-19T06:16:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.616+0000 7f371e3e5740 1 --2- 172.21.15.5:0/977555279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202f9cb0 0x55b6202fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.616+0000 7f371e3e5740 1 --2- 172.21.15.5:0/977555279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202ef410 0x55b6202f9770 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.616+0000 7f371e3e5740 1 --2- 172.21.15.5:0/977555279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 0x55b6202eeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.616+0000 7f371e3e5740 1 -- 172.21.15.5:0/977555279 >> 172.21.15.5:0/977555279 conn(0x55b6202eb060 msgr2=0x55b6202eb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.616+0000 7f371e3e5740 1 -- 172.21.15.5:0/977555279 shutdown_connections 2024-09-19T06:16:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.616+0000 7f371e3e5740 1 -- 172.21.15.5:0/977555279 wait complete. 2024-09-19T06:16:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.617+0000 7f371e3e5740 1 -- start start 2024-09-19T06:16:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.617+0000 7f371e3e5740 1 Processor -- start 2024-09-19T06:16:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.617+0000 7f371e3e5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 0x55b6202f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f37193e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 0x55b6202f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f37193e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 0x55b6202f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60626/0 (socket says 172.21.15.5:60626) 2024-09-19T06:16:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.617+0000 7f371e3e5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 0x55b6202f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f37193e8640 1 -- 172.21.15.5:0/2616001809 learned_addr learned my addr 172.21.15.5:0/2616001809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202f9cb0 0x55b6202f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f371e3e5740 1 -- 172.21.15.5:0/2616001809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b6202fc880 con 0x55b6202f9cb0 2024-09-19T06:16:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f371e3e5740 1 -- 172.21.15.5:0/2616001809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b6201dc290 con 0x55b6202eeaf0 2024-09-19T06:16:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f3718be7640 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 0x55b6202f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f3719be9640 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202f9cb0 0x55b6202f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f371e3e5740 1 -- 172.21.15.5:0/2616001809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b6202fca00 con 0x55b6202ef410 2024-09-19T06:16:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f371e3e5740 10 client.0 _ll_register_callbacks cb 0x55b62020d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f37193e8640 1 -- 172.21.15.5:0/2616001809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 msgr2=0x55b6202f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f37193e8640 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 0x55b6202f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f37193e8640 1 -- 172.21.15.5:0/2616001809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202f9cb0 msgr2=0x55b6202f75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f37193e8640 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202f9cb0 0x55b6202f75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f37193e8640 1 -- 172.21.15.5:0/2616001809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b61ff66210 con 0x55b6202eeaf0 2024-09-19T06:16:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.618+0000 7f3719be9640 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202f9cb0 0x55b6202f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.619+0000 7f371e3e5740 1 -- 172.21.15.5:0/2616001809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371009fb00 con 0x55b6202eeaf0 2024-09-19T06:16:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.619+0000 7f37193e8640 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 0x55b6202f6b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f371009b500 tx=0x7f3710009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7853d8dbb0370744 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.619+0000 7f36fcff9640 10 client.28445 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:16:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.619+0000 7f36fcff9640 1 -- 172.21.15.5:0/2616001809 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710020040 con 0x55b6202eeaf0 2024-09-19T06:16:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.619+0000 7f36fcff9640 1 -- 172.21.15.5:0/2616001809 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f3710097e70 con 0x55b6202eeaf0 2024-09-19T06:16:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.619+0000 7f36fcff9640 1 -- 172.21.15.5:0/2616001809 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f37100a43c0 con 0x55b6202eeaf0 2024-09-19T06:16:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.619+0000 7f371e3e5740 10 client.28445 fetch_fsmap learned FSMap version 42 2024-09-19T06:16:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.619+0000 7f371e3e5740 1 -- 172.21.15.5:0/2616001809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55b620032620 con 0x55b6202eeaf0 2024-09-19T06:16:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.619+0000 7f36fcff9640 1 -- 172.21.15.5:0/2616001809 <== mon.2 v2:172.21.15.49:3300/0 4 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f37100cce40 con 0x55b6202eeaf0 2024-09-19T06:16:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.620+0000 7f371e3e5740 10 client.28445 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T06:16:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.620+0000 7f371e3e5740 10 client.28445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.620+0000 7f371e3e5740 20 client.28445 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.620+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b6203098d0 0x55b62030bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.620+0000 7f371e3e5740 4 client.28445 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:16:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.620+0000 7f371e3e5740 1 -- 172.21.15.5:0/2616001809 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b620260a20 con 0x55b6203098d0 2024-09-19T06:16:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.620+0000 7f3718be7640 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b6203098d0 0x55b62030bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.620+0000 7f36fcff9640 10 client.28445 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.620+0000 7f3718be7640 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b6203098d0 0x55b62030bcc0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f40070f4c206148 server_cookie=e654e54c85e42c03 in_seq=0 out_seq=0 2024-09-19T06:16:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f36fcff9640 1 -- 172.21.15.5:0/2616001809 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b620260a20 con 0x55b6203098d0 2024-09-19T06:16:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f36fcff9640 10 client.28445 handle_command_reply: tid=0 2024-09-19T06:16:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f371e3e5740 1 client.28445 shutdown 2024-09-19T06:16:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f371e3e5740 1 -- 172.21.15.5:0/2616001809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 msgr2=0x55b6202f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 0x55b6202f6b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f371009b500 tx=0x7f3710009740 comp rx=0 tx=0).stop 2024-09-19T06:16:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f371e3e5740 1 -- 172.21.15.5:0/2616001809 shutdown_connections 2024-09-19T06:16:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b6203098d0 0x55b62030bcc0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202f9cb0 0x55b6202f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 0x55b6202f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f36fcff9640 0 client.28445 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:54.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2616001809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 0x55b6202f6b40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f371e3e5740 1 -- 172.21.15.5:0/2616001809 >> 172.21.15.5:0/2616001809 conn(0x55b6202eb060 msgr2=0x55b6202f0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:54.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f371e3e5740 1 -- 172.21.15.5:0/2616001809 shutdown_connections 2024-09-19T06:16:54.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f371e3e5740 1 -- 172.21.15.5:0/2616001809 wait complete. 2024-09-19T06:16:54.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.621+0000 7f371e3e5740 20 client.28445 trim_cache size 0 max 16384 2024-09-19T06:16:54.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:54 smithi005 ceph-mon[30955]: pgmap v424: 97 pgs: 97 active+clean; 374 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 19 MiB/s wr, 427 op/s 2024-09-19T06:16:54.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.646+0000 7f371e3e5740 1 Processor -- start 2024-09-19T06:16:54.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.646+0000 7f371e3e5740 1 -- start start 2024-09-19T06:16:54.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f371e3e5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 0x55b6202f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f371e3e5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 0x55b6202f7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f371e3e5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202f9cb0 0x55b6202f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f371e3e5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37141ab680 con 0x55b6202eeaf0 2024-09-19T06:16:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f371e3e5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f37141ad2e0 con 0x55b6202f9cb0 2024-09-19T06:16:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f371e3e5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f37141abd50 con 0x55b6202ef410 2024-09-19T06:16:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f37193e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 0x55b6202f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f3719be9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202f9cb0 0x55b6202f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f37193e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 0x55b6202f8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58604/0 (socket says 172.21.15.5:58604) 2024-09-19T06:16:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f37193e8640 1 -- 172.21.15.5:0/2568780330 learned_addr learned my addr 172.21.15.5:0/2568780330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f3718be7640 1 --2- 172.21.15.5:0/2568780330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 0x55b6202f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f37193e8640 1 -- 172.21.15.5:0/2568780330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 msgr2=0x55b6202f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f37193e8640 1 --2- 172.21.15.5:0/2568780330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 0x55b6202f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f37193e8640 1 -- 172.21.15.5:0/2568780330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202f9cb0 msgr2=0x55b6202f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f37193e8640 1 --2- 172.21.15.5:0/2568780330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202f9cb0 0x55b6202f75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f37193e8640 1 -- 172.21.15.5:0/2568780330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6202f1940 con 0x55b6202eeaf0 2024-09-19T06:16:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.647+0000 7f3719be9640 1 --2- 172.21.15.5:0/2568780330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202f9cb0 0x55b6202f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.648+0000 7f37193e8640 1 --2- 172.21.15.5:0/2568780330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 0x55b6202f8ab0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f37100a4580 tx=0x7f371009e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbeafd64d1d5d82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.648+0000 7f36feffd640 1 -- 172.21.15.5:0/2568780330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710020070 con 0x55b6202eeaf0 2024-09-19T06:16:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.648+0000 7f36feffd640 1 -- 172.21.15.5:0/2568780330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f371009fb00 con 0x55b6202eeaf0 2024-09-19T06:16:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.648+0000 7f36feffd640 1 -- 172.21.15.5:0/2568780330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710097550 con 0x55b6202eeaf0 2024-09-19T06:16:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.649+0000 7f371e3e5740 1 -- 172.21.15.5:0/2568780330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 msgr2=0x55b6202f8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.649+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2568780330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 0x55b6202f8ab0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f37100a4580 tx=0x7f371009e000 comp rx=0 tx=0).stop 2024-09-19T06:16:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.649+0000 7f371e3e5740 1 -- 172.21.15.5:0/2568780330 shutdown_connections 2024-09-19T06:16:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.649+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2568780330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202f9cb0 0x55b6202f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.649+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2568780330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 0x55b6202f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.649+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2568780330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202eeaf0 0x55b6202f8ab0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.649+0000 7f371e3e5740 1 -- 172.21.15.5:0/2568780330 >> 172.21.15.5:0/2568780330 conn(0x55b6202eb060 msgr2=0x55b6202f83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.649+0000 7f371e3e5740 1 -- 172.21.15.5:0/2568780330 shutdown_connections 2024-09-19T06:16:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.649+0000 7f371e3e5740 1 -- 172.21.15.5:0/2568780330 wait complete. 2024-09-19T06:16:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.649+0000 7f371e3e5740 1 -- start start 2024-09-19T06:16:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.649+0000 7f371e3e5740 1 Processor -- start 2024-09-19T06:16:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f371e3e5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 0x55b620301e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f371e3e5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 0x55b6203023c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f371e3e5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202f9cb0 0x55b6202f3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f371e3e5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37141acc10 con 0x55b6202f9cb0 2024-09-19T06:16:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f371e3e5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f37141ac480 con 0x55b6202eeaf0 2024-09-19T06:16:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f371e3e5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f37141a9680 con 0x55b6202ef410 2024-09-19T06:16:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f371e3e5740 10 client.0 _ll_register_callbacks cb 0x7f370c00e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:16:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f37193e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 0x55b620301e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f3718be7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 0x55b6203023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f3719be9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202f9cb0 0x55b6202f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f37193e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 0x55b620301e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60636/0 (socket says 172.21.15.5:60636) 2024-09-19T06:16:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f37193e8640 1 -- 172.21.15.5:0/2881623603 learned_addr learned my addr 172.21.15.5:0/2881623603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:16:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f37193e8640 1 -- 172.21.15.5:0/2881623603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 msgr2=0x55b6203023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f37193e8640 1 --2- 172.21.15.5:0/2881623603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 0x55b6203023c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f37193e8640 1 -- 172.21.15.5:0/2881623603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202f9cb0 msgr2=0x55b6202f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:16:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f37193e8640 1 --2- 172.21.15.5:0/2881623603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202f9cb0 0x55b6202f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f37193e8640 1 -- 172.21.15.5:0/2881623603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6202f3a90 con 0x55b6202eeaf0 2024-09-19T06:16:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.650+0000 7f3719be9640 1 --2- 172.21.15.5:0/2881623603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202f9cb0 0x55b6202f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:16:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.651+0000 7f37193e8640 1 --2- 172.21.15.5:0/2881623603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 0x55b620301e80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3710097c60 tx=0x7f371009b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81f12da07a12a2ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:16:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.651+0000 7f36fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:16:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.651+0000 7f36fcff9640 1 -- 172.21.15.5:0/2881623603 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710020040 con 0x55b6202eeaf0 2024-09-19T06:16:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.651+0000 7f36fcff9640 1 -- 172.21.15.5:0/2881623603 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f3710084d00 con 0x55b6202eeaf0 2024-09-19T06:16:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.651+0000 7f371e3e5740 1 -- 172.21.15.5:0/2881623603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371009fb00 con 0x55b6202eeaf0 2024-09-19T06:16:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.651+0000 7f36fcff9640 1 -- 172.21.15.5:0/2881623603 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3710097550 con 0x55b6202eeaf0 2024-09-19T06:16:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.651+0000 7f371e3e5740 10 client.28451 fetch_fsmap learned FSMap version 42 2024-09-19T06:16:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.651+0000 7f371e3e5740 1 -- 172.21.15.5:0/2881623603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b6202f3a90 con 0x55b6202eeaf0 2024-09-19T06:16:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.651+0000 7f36fcff9640 1 -- 172.21.15.5:0/2881623603 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f371009fb00 con 0x55b6202eeaf0 2024-09-19T06:16:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.651+0000 7f36fcff9640 1 -- 172.21.15.5:0/2881623603 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f37100a0c90 con 0x55b6202eeaf0 2024-09-19T06:16:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.651+0000 7f371e3e5740 10 client.28451 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T06:16:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.652+0000 7f371e3e5740 10 client.28451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:16:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.652+0000 7f371e3e5740 20 client.28451 populate_metadata read hostname 'smithi005' 2024-09-19T06:16:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.652+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2881623603 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b620374070 0x55b620376460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:16:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.652+0000 7f371e3e5740 4 client.28451 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:16:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.652+0000 7f371e3e5740 1 -- 172.21.15.5:0/2881623603 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b62025ad70 con 0x55b620374070 2024-09-19T06:16:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.652+0000 7f3718be7640 1 --2- 172.21.15.5:0/2881623603 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b620374070 0x55b620376460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:16:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.652+0000 7f36fcff9640 10 client.28451 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.652+0000 7f3718be7640 1 --2- 172.21.15.5:0/2881623603 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b620374070 0x55b620376460 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebbee99fd6c02c34 server_cookie=23b1aae83bd0c6f9 in_seq=0 out_seq=0 2024-09-19T06:16:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.691+0000 7f36fcff9640 1 -- 172.21.15.5:0/2881623603 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b62025ad70 con 0x55b620374070 2024-09-19T06:16:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.691+0000 7f36fcff9640 10 client.28451 handle_command_reply: tid=0 2024-09-19T06:16:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.691+0000 7f371e3e5740 1 client.28451 shutdown 2024-09-19T06:16:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.691+0000 7f371e3e5740 1 -- 172.21.15.5:0/2881623603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 msgr2=0x55b620301e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.691+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2881623603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 0x55b620301e80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3710097c60 tx=0x7f371009b040 comp rx=0 tx=0).stop 2024-09-19T06:16:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.691+0000 7f371e3e5740 1 -- 172.21.15.5:0/2881623603 shutdown_connections 2024-09-19T06:16:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.691+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2881623603 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b620374070 0x55b620376460 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.691+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2881623603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6202f9cb0 0x55b6202f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.691+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2881623603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6202ef410 0x55b6203023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.691+0000 7f371e3e5740 1 --2- 172.21.15.5:0/2881623603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6202eeaf0 0x55b620301e80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.691+0000 7f371e3e5740 1 -- 172.21.15.5:0/2881623603 >> 172.21.15.5:0/2881623603 conn(0x55b6202eb060 msgr2=0x55b6202f0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.691+0000 7f36fcff9640 0 client.28451 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:16:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.692+0000 7f371e3e5740 1 -- 172.21.15.5:0/2881623603 shutdown_connections 2024-09-19T06:16:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.692+0000 7f371e3e5740 1 -- 172.21.15.5:0/2881623603 wait complete. 2024-09-19T06:16:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.692+0000 7f371e3e5740 20 client.28451 trim_cache size 0 max 16384 2024-09-19T06:16:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:16:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:16:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:16:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:16:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.693+0000 7f371b64c640 1 -- 172.21.15.5:0/2161558628 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f37000a3430 msgr2=0x7f37000a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.693+0000 7f371b64c640 1 --2- 172.21.15.5:0/2161558628 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f37000a3430 0x7f37000a58f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f370c009c50 tx=0x7f370c009a00 comp rx=0 tx=0).stop 2024-09-19T06:16:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.693+0000 7f371b64c640 1 -- 172.21.15.5:0/2161558628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3714101ac0 msgr2=0x7f3714196110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:16:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.693+0000 7f371b64c640 1 --2- 172.21.15.5:0/2161558628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3714101ac0 0x7f3714196110 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f371000d6d0 tx=0x7f371000db90 comp rx=0 tx=0).stop 2024-09-19T06:16:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.693+0000 7f371b64c640 1 -- 172.21.15.5:0/2161558628 shutdown_connections 2024-09-19T06:16:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.693+0000 7f371b64c640 1 --2- 172.21.15.5:0/2161558628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3714107ad0 0x7f371419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.693+0000 7f371b64c640 1 --2- 172.21.15.5:0/2161558628 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f37000a3430 0x7f37000a58f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.693+0000 7f371b64c640 1 --2- 172.21.15.5:0/2161558628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3714102460 0x7f3714196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.693+0000 7f371b64c640 1 --2- 172.21.15.5:0/2161558628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3714101ac0 0x7f3714196110 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:16:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.693+0000 7f371b64c640 1 -- 172.21.15.5:0/2161558628 >> 172.21.15.5:0/2161558628 conn(0x7f37140fd7a0 msgr2=0x7f37140fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:16:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.694+0000 7f371b64c640 1 -- 172.21.15.5:0/2161558628 shutdown_connections 2024-09-19T06:16:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:16:54.694+0000 7f371b64c640 1 -- 172.21.15.5:0/2161558628 wait complete. 2024-09-19T06:16:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:54 smithi157 ceph-mon[35346]: pgmap v424: 97 pgs: 97 active+clean; 374 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 19 MiB/s wr, 427 op/s 2024-09-19T06:16:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:54 smithi049 ceph-mon[34640]: pgmap v424: 97 pgs: 97 active+clean; 374 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 19 MiB/s wr, 427 op/s 2024-09-19T06:16:55.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 3265 18.40 MB/sec warmup 19 sec latency 487.501 ms 2024-09-19T06:16:55.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:55 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:16:55.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:55 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:16:55.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:55 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:16:56.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 3428 18.24 MB/sec warmup 20 sec latency 68.695 ms 2024-09-19T06:16:56.796 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:56 smithi049 ceph-mon[34640]: pgmap v425: 97 pgs: 97 active+clean; 374 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 2.6 MiB/s wr, 149 op/s 2024-09-19T06:16:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:56 smithi157 ceph-mon[35346]: pgmap v425: 97 pgs: 97 active+clean; 374 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 2.6 MiB/s wr, 149 op/s 2024-09-19T06:16:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:56 smithi005 ceph-mon[30955]: pgmap v425: 97 pgs: 97 active+clean; 374 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 2.6 MiB/s wr, 149 op/s 2024-09-19T06:16:57.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 3537 17.78 MB/sec warmup 21 sec latency 57.910 ms 2024-09-19T06:16:58.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 3727 17.78 MB/sec warmup 22 sec latency 109.736 ms 2024-09-19T06:16:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:58 smithi157 ceph-mon[35346]: pgmap v426: 97 pgs: 97 active+clean; 390 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 303 op/s 2024-09-19T06:16:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:58 smithi005 ceph-mon[30955]: pgmap v426: 97 pgs: 97 active+clean; 390 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 303 op/s 2024-09-19T06:16:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:58 smithi049 ceph-mon[34640]: pgmap v426: 97 pgs: 97 active+clean; 390 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 303 op/s 2024-09-19T06:16:59.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 3880 17.23 MB/sec warmup 23 sec latency 166.281 ms 2024-09-19T06:16:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:16:59 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:16:59.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:16:59 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:16:59.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:16:59 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:17:00.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4022 16.68 MB/sec warmup 24 sec latency 586.899 ms 2024-09-19T06:17:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:00 smithi157 ceph-mon[35346]: pgmap v427: 97 pgs: 97 active+clean; 390 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 4.1 MiB/s wr, 233 op/s 2024-09-19T06:17:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:00 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:17:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:00 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:17:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:00 smithi005 ceph-mon[30955]: pgmap v427: 97 pgs: 97 active+clean; 390 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 4.1 MiB/s wr, 233 op/s 2024-09-19T06:17:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:00 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:17:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:00 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:17:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:00 smithi049 ceph-mon[34640]: pgmap v427: 97 pgs: 97 active+clean; 390 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 4.1 MiB/s wr, 233 op/s 2024-09-19T06:17:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:00 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:17:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:00 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:17:01.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4155 16.12 MB/sec warmup 25 sec latency 76.889 ms 2024-09-19T06:17:02.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4300 15.67 MB/sec warmup 26 sec latency 137.217 ms 2024-09-19T06:17:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:02 smithi157 ceph-mon[35346]: pgmap v428: 97 pgs: 97 active+clean; 390 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 4.1 MiB/s wr, 233 op/s 2024-09-19T06:17:02.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:02 smithi005 ceph-mon[30955]: pgmap v428: 97 pgs: 97 active+clean; 390 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 4.1 MiB/s wr, 233 op/s 2024-09-19T06:17:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:02 smithi049 ceph-mon[34640]: pgmap v428: 97 pgs: 97 active+clean; 390 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 4.1 MiB/s wr, 233 op/s 2024-09-19T06:17:03.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4461 15.17 MB/sec warmup 27 sec latency 112.257 ms 2024-09-19T06:17:04.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4648 15.13 MB/sec warmup 28 sec latency 100.324 ms 2024-09-19T06:17:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:04 smithi157 ceph-mon[35346]: pgmap v429: 97 pgs: 97 active+clean; 403 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.4 MiB/s wr, 345 op/s 2024-09-19T06:17:04.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:04 smithi005 ceph-mon[30955]: pgmap v429: 97 pgs: 97 active+clean; 403 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.4 MiB/s wr, 345 op/s 2024-09-19T06:17:04.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:04 smithi049 ceph-mon[34640]: pgmap v429: 97 pgs: 97 active+clean; 403 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.4 MiB/s wr, 345 op/s 2024-09-19T06:17:05.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4814 14.79 MB/sec warmup 29 sec latency 77.806 ms 2024-09-19T06:17:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:05 smithi157 ceph-mon[35346]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T06:17:05.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:05 smithi005 ceph-mon[30955]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T06:17:05.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:05 smithi049 ceph-mon[34640]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T06:17:06.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 5008 14.67 MB/sec warmup 30 sec latency 65.472 ms 2024-09-19T06:17:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:06 smithi157 ceph-mon[35346]: pgmap v431: 97 pgs: 97 active+clean; 403 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.1 MiB/s wr, 320 op/s 2024-09-19T06:17:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:06 smithi005 ceph-mon[30955]: pgmap v431: 97 pgs: 97 active+clean; 403 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.1 MiB/s wr, 320 op/s 2024-09-19T06:17:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:06 smithi049 ceph-mon[34640]: pgmap v431: 97 pgs: 97 active+clean; 403 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.1 MiB/s wr, 320 op/s 2024-09-19T06:17:07.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 5181 14.26 MB/sec warmup 31 sec latency 62.391 ms 2024-09-19T06:17:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.819+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3157656129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb4103f40 msgr2=0x7fbbb4104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.819+0000 7fbbbc4e6640 1 --2- 172.21.15.5:0/3157656129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb4103f40 0x7fbbb4104320 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009900 tx=0x7fbbb002a3f0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.819+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3157656129 shutdown_connections 2024-09-19T06:17:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.819+0000 7fbbbc4e6640 1 --2- 172.21.15.5:0/3157656129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbb4109090 0x7fbbb4110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.819+0000 7fbbbc4e6640 1 --2- 172.21.15.5:0/3157656129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb41048f0 0x7fbbb4108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.819+0000 7fbbbc4e6640 1 --2- 172.21.15.5:0/3157656129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb4103f40 0x7fbbb4104320 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.819+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3157656129 >> 172.21.15.5:0/3157656129 conn(0x7fbbb40ff970 msgr2=0x7fbbb4101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.819+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3157656129 shutdown_connections 2024-09-19T06:17:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.819+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3157656129 wait complete. 2024-09-19T06:17:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbbc4e6640 1 Processor -- start 2024-09-19T06:17:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbbc4e6640 1 -- start start 2024-09-19T06:17:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbbc4e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb4103f40 0x7fbbb419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbbc4e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbb4109090 0x7fbbb419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbbc4e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb419eef0 0x7fbbb419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbbc4e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4113f70 con 0x7fbbb4103f40 2024-09-19T06:17:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbbc4e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4113df0 con 0x7fbbb419eef0 2024-09-19T06:17:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbbc4e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41140f0 con 0x7fbbb4109090 2024-09-19T06:17:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbba282640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb4103f40 0x7fbbb419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbba282640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb4103f40 0x7fbbb419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47702/0 (socket says 172.21.15.5:47702) 2024-09-19T06:17:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbba282640 1 -- 172.21.15.5:0/3151111071 learned_addr learned my addr 172.21.15.5:0/3151111071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbb9a81640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbb4109090 0x7fbbb419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb419eef0 0x7fbbb419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbba282640 1 -- 172.21.15.5:0/3151111071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbb4109090 msgr2=0x7fbbb419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.820+0000 7fbbba282640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbb4109090 0x7fbbb419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.821+0000 7fbbba282640 1 -- 172.21.15.5:0/3151111071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb419eef0 msgr2=0x7fbbb419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.821+0000 7fbbba282640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb419eef0 0x7fbbb419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.821+0000 7fbbba282640 1 -- 172.21.15.5:0/3151111071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb419f9e0 con 0x7fbbb4103f40 2024-09-19T06:17:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.821+0000 7fbbba282640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb4103f40 0x7fbbb419a630 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0005940 tx=0x7fbbb00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.821+0000 7fbbab7fe640 1 -- 172.21.15.5:0/3151111071 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb003f030 con 0x7fbbb4103f40 2024-09-19T06:17:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.821+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3151111071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb419fc70 con 0x7fbbb4103f40 2024-09-19T06:17:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.821+0000 7fbbab7fe640 1 -- 172.21.15.5:0/3151111071 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbb00044c0 con 0x7fbbb4103f40 2024-09-19T06:17:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.821+0000 7fbbab7fe640 1 -- 172.21.15.5:0/3151111071 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0035070 con 0x7fbbb4103f40 2024-09-19T06:17:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.821+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3151111071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb41a7490 con 0x7fbbb4103f40 2024-09-19T06:17:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.825+0000 7fbbbf277740 1 Processor -- start 2024-09-19T06:17:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.825+0000 7fbbab7fe640 1 -- 172.21.15.5:0/3151111071 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbbb0042030 con 0x7fbbb4103f40 2024-09-19T06:17:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.825+0000 7fbbbf277740 1 -- start start 2024-09-19T06:17:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.825+0000 7fbbbf277740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.825+0000 7fbbbf277740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1a2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbf277740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbf277740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56322f0fc2f0 con 0x56322f198410 2024-09-19T06:17:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbf277740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56322f0a4270 con 0x56322f1a2cb0 2024-09-19T06:17:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbf277740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56322f0f82b0 con 0x56322f197af0 2024-09-19T06:17:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbba282640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbb9a81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbaa83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbaa83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60576/0 (socket says 172.21.15.5:60576) 2024-09-19T06:17:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbaa83640 1 -- 172.21.15.5:0/520461403 learned_addr learned my addr 172.21.15.5:0/520461403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbaa83640 1 -- 172.21.15.5:0/520461403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 msgr2=0x56322f1a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/520461403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbaa83640 1 -- 172.21.15.5:0/520461403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 msgr2=0x56322f1a2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/520461403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1a2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbaa83640 1 -- 172.21.15.5:0/520461403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56322f0a3340 con 0x56322f197af0 2024-09-19T06:17:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbb9a81640 1 --2- 172.21.15.5:0/520461403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbab7fe640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbba00a33b0 0x7fbba00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.826+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/520461403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f197ed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009f50 tx=0x7fbbac0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcbf376668a34c85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.827+0000 7fbbba282640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbba00a33b0 0x7fbba00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.827+0000 7fbba8bfe640 1 -- 172.21.15.5:0/520461403 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac004b20 con 0x56322f197af0 2024-09-19T06:17:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.827+0000 7fbba8bfe640 1 -- 172.21.15.5:0/520461403 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbac004cc0 con 0x56322f197af0 2024-09-19T06:17:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.827+0000 7fbba8bfe640 1 -- 172.21.15.5:0/520461403 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac005020 con 0x56322f197af0 2024-09-19T06:17:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.827+0000 7fbbab7fe640 1 -- 172.21.15.5:0/3151111071 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8594+0+0 (secure 0 0 0) 0x7fbbb00efba0 con 0x7fbbb4103f40 2024-09-19T06:17:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.828+0000 7fbbbf277740 1 -- 172.21.15.5:0/520461403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 msgr2=0x56322f197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.828+0000 7fbbbf277740 1 --2- 172.21.15.5:0/520461403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f197ed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009f50 tx=0x7fbbac0118d0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.828+0000 7fbbba282640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbba00a33b0 0x7fbba00a5870 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x56322f194ec0 tx=0x7fbbb00f3650 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.828+0000 7fbbbf277740 1 -- 172.21.15.5:0/520461403 shutdown_connections 2024-09-19T06:17:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.828+0000 7fbbbf277740 1 --2- 172.21.15.5:0/520461403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.828+0000 7fbbbf277740 1 --2- 172.21.15.5:0/520461403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.828+0000 7fbbbf277740 1 --2- 172.21.15.5:0/520461403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f197ed0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.828+0000 7fbbbf277740 1 -- 172.21.15.5:0/520461403 >> 172.21.15.5:0/520461403 conn(0x56322f194060 msgr2=0x56322f194440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.828+0000 7fbbbf277740 1 -- 172.21.15.5:0/520461403 shutdown_connections 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.828+0000 7fbbbf277740 1 -- 172.21.15.5:0/520461403 wait complete. 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.828+0000 7fbbbf277740 1 -- start start 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.828+0000 7fbbbf277740 1 Processor -- start 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbbf277740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbbf277740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1a0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbbf277740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbbf277740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56322f1a5880 con 0x56322f198410 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbbaa83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbb9a81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbb9a81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47728/0 (socket says 172.21.15.5:47728) 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbb9a81640 1 -- 172.21.15.5:0/699794961 learned_addr learned my addr 172.21.15.5:0/699794961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbbf277740 1 -- 172.21.15.5:0/699794961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56322f085290 con 0x56322f1a2cb0 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbbf277740 1 -- 172.21.15.5:0/699794961 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56322f1a5a00 con 0x56322f197af0 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbbf277740 10 client.0 _ll_register_callbacks cb 0x56322f0b6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbb9a81640 1 -- 172.21.15.5:0/699794961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 msgr2=0x56322f19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbb9a81640 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbb9a81640 1 -- 172.21.15.5:0/699794961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 msgr2=0x56322f1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbb9a81640 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbb9a81640 1 -- 172.21.15.5:0/699794961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56322ee0f210 con 0x56322f198410 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.829+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.830+0000 7fbbb9a81640 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1a0090 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fbba4012730 tx=0x7fbba4012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a760210a9656d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.830+0000 7fbb9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.830+0000 7fbb9dffb640 1 -- 172.21.15.5:0/699794961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba4004030 con 0x56322f198410 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.830+0000 7fbb9dffb640 1 -- 172.21.15.5:0/699794961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8594+0+0 (secure 0 0 0) 0x7fbba40041d0 con 0x56322f198410 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.830+0000 7fbbbf277740 1 -- 172.21.15.5:0/699794961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbac004cc0 con 0x56322f198410 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.830+0000 7fbb9dffb640 1 -- 172.21.15.5:0/699794961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbba40043d0 con 0x56322f198410 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.830+0000 7fbbbf277740 10 client.19062 fetch_fsmap learned FSMap version 42 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.830+0000 7fbbbf277740 1 -- 172.21.15.5:0/699794961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56322eedb620 con 0x56322f198410 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.830+0000 7fbb9dffb640 1 -- 172.21.15.5:0/699794961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbac004cc0 con 0x56322f198410 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.831+0000 7fbb9dffb640 1 -- 172.21.15.5:0/699794961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7fbba4003b10 con 0x56322f198410 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.831+0000 7fbbbf277740 10 client.19062 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.831+0000 7fbbbf277740 10 client.19062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.831+0000 7fbbbf277740 20 client.19062 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.831+0000 7fbbbf277740 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56322f1b2e30 0x56322f1b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.831+0000 7fbbbf277740 4 client.19062 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:17:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.831+0000 7fbbbf277740 1 -- 172.21.15.5:0/699794961 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56322f109a20 con 0x56322f1b2e30 2024-09-19T06:17:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.831+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56322f1b2e30 0x56322f1b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.831+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56322f1b2e30 0x56322f1b5290 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63cefe7e4611b90b server_cookie=47f5e3e320b25907 in_seq=0 out_seq=0 2024-09-19T06:17:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.831+0000 7fbb9dffb640 10 client.19062 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.832+0000 7fbb9dffb640 1 -- 172.21.15.5:0/699794961 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56322f109a20 con 0x56322f1b2e30 2024-09-19T06:17:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.832+0000 7fbb9dffb640 10 client.19062 handle_command_reply: tid=0 2024-09-19T06:17:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.832+0000 7fbbbf277740 1 client.19062 shutdown 2024-09-19T06:17:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.833+0000 7fbbbf277740 1 -- 172.21.15.5:0/699794961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 msgr2=0x56322f1a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.833+0000 7fbbbf277740 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1a0090 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fbba4012730 tx=0x7fbba4012bf0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.833+0000 7fbbbf277740 1 -- 172.21.15.5:0/699794961 shutdown_connections 2024-09-19T06:17:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.833+0000 7fbbbf277740 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56322f1b2e30 0x56322f1b5290 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.833+0000 7fbbbf277740 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.833+0000 7fbbbf277740 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1a0090 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.833+0000 7fbbbf277740 1 --2- 172.21.15.5:0/699794961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.833+0000 7fbb9dffb640 0 client.19062 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.833+0000 7fbbbf277740 1 -- 172.21.15.5:0/699794961 >> 172.21.15.5:0/699794961 conn(0x56322f194060 msgr2=0x56322f199dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.833+0000 7fbbbf277740 1 -- 172.21.15.5:0/699794961 shutdown_connections 2024-09-19T06:17:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.833+0000 7fbbbf277740 1 -- 172.21.15.5:0/699794961 wait complete. 2024-09-19T06:17:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.833+0000 7fbbbf277740 20 client.19062 trim_cache size 0 max 16384 2024-09-19T06:17:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.855+0000 7fbbbf277740 1 Processor -- start 2024-09-19T06:17:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.855+0000 7fbbbf277740 1 -- start start 2024-09-19T06:17:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.855+0000 7fbbbf277740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.855+0000 7fbbbf277740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1bab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.855+0000 7fbbbf277740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1bb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.855+0000 7fbbbf277740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9800e870 con 0x56322f198410 2024-09-19T06:17:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.855+0000 7fbbbf277740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb980104d0 con 0x56322f1a2cb0 2024-09-19T06:17:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.855+0000 7fbbbf277740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9800ef40 con 0x56322f197af0 2024-09-19T06:17:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbbbaa83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbbb9a81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1bab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbbbaa83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f1a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60598/0 (socket says 172.21.15.5:60598) 2024-09-19T06:17:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbbbaa83640 1 -- 172.21.15.5:0/3423431385 learned_addr learned my addr 172.21.15.5:0/3423431385 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/3423431385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1bb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbbbaa83640 1 -- 172.21.15.5:0/3423431385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 msgr2=0x56322f1bb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/3423431385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1bb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbbbaa83640 1 -- 172.21.15.5:0/3423431385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 msgr2=0x56322f1bab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/3423431385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1bab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbbbaa83640 1 -- 172.21.15.5:0/3423431385 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56322f193e90 con 0x56322f197af0 2024-09-19T06:17:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbbb9a81640 1 --2- 172.21.15.5:0/3423431385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1bab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/3423431385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f1a0400 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009d60 tx=0x7fbbac004f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2607331aad3bff77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbb9ffff640 1 -- 172.21.15.5:0/3423431385 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0043a0 con 0x56322f197af0 2024-09-19T06:17:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.856+0000 7fbb9ffff640 1 -- 172.21.15.5:0/3423431385 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbac003ee0 con 0x56322f197af0 2024-09-19T06:17:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.857+0000 7fbb9ffff640 1 -- 172.21.15.5:0/3423431385 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac027ac0 con 0x56322f197af0 2024-09-19T06:17:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.857+0000 7fbbbf277740 1 -- 172.21.15.5:0/3423431385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 msgr2=0x56322f1a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.857+0000 7fbbbf277740 1 --2- 172.21.15.5:0/3423431385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f1a0400 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009d60 tx=0x7fbbac004f30 comp rx=0 tx=0).stop 2024-09-19T06:17:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.857+0000 7fbbbf277740 1 -- 172.21.15.5:0/3423431385 shutdown_connections 2024-09-19T06:17:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.857+0000 7fbbbf277740 1 --2- 172.21.15.5:0/3423431385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f1bb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.857+0000 7fbbbf277740 1 --2- 172.21.15.5:0/3423431385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f198410 0x56322f1bab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.857+0000 7fbbbf277740 1 --2- 172.21.15.5:0/3423431385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f197af0 0x56322f1a0400 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.857+0000 7fbbbf277740 1 -- 172.21.15.5:0/3423431385 >> 172.21.15.5:0/3423431385 conn(0x56322f194060 msgr2=0x56322f1a1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbf277740 1 -- 172.21.15.5:0/3423431385 shutdown_connections 2024-09-19T06:17:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbf277740 1 -- 172.21.15.5:0/3423431385 wait complete. 2024-09-19T06:17:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbf277740 1 -- start start 2024-09-19T06:17:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbf277740 1 Processor -- start 2024-09-19T06:17:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbf277740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f197af0 0x56322f19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbf277740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f198410 0x56322f19d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbf277740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f19d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbf277740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9800fe00 con 0x56322f197af0 2024-09-19T06:17:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbf277740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9800f670 con 0x56322f1a2cb0 2024-09-19T06:17:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbf277740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9800c870 con 0x56322f198410 2024-09-19T06:17:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbf277740 10 client.0 _ll_register_callbacks cb 0x7fbba4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbaa83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f197af0 0x56322f19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbaa83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f197af0 0x56322f19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47740/0 (socket says 172.21.15.5:47740) 2024-09-19T06:17:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbbaa83640 1 -- 172.21.15.5:0/2977006301 learned_addr learned my addr 172.21.15.5:0/2977006301 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.858+0000 7fbbb9a81640 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f198410 0x56322f19d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.859+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.859+0000 7fbbbaa83640 1 -- 172.21.15.5:0/2977006301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f198410 msgr2=0x56322f19d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.859+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f198410 0x56322f19d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.859+0000 7fbbbaa83640 1 -- 172.21.15.5:0/2977006301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 msgr2=0x56322f19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.859+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.859+0000 7fbbbaa83640 1 -- 172.21.15.5:0/2977006301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56322f21cbe0 con 0x56322f197af0 2024-09-19T06:17:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.859+0000 7fbbb9a81640 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f198410 0x56322f19d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.859+0000 7fbbbaa83640 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f197af0 0x56322f19cbb0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fbbac012fd0 tx=0x7fbbac005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941df4fba507e6ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.859+0000 7fbb9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:17:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.859+0000 7fbb9dffb640 1 -- 172.21.15.5:0/2977006301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0043a0 con 0x56322f197af0 2024-09-19T06:17:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.859+0000 7fbb9dffb640 1 -- 172.21.15.5:0/2977006301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8594+0+0 (secure 0 0 0) 0x7fbbac003a30 con 0x56322f197af0 2024-09-19T06:17:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.859+0000 7fbbbf277740 1 -- 172.21.15.5:0/2977006301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbac003ee0 con 0x56322f197af0 2024-09-19T06:17:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.860+0000 7fbb9dffb640 1 -- 172.21.15.5:0/2977006301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbbac04c050 con 0x56322f197af0 2024-09-19T06:17:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.860+0000 7fbbbf277740 10 client.19068 fetch_fsmap learned FSMap version 42 2024-09-19T06:17:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.860+0000 7fbbbf277740 1 -- 172.21.15.5:0/2977006301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56322f21deb0 con 0x56322f197af0 2024-09-19T06:17:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.860+0000 7fbb9dffb640 1 -- 172.21.15.5:0/2977006301 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbac003ee0 con 0x56322f197af0 2024-09-19T06:17:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.860+0000 7fbb9dffb640 1 -- 172.21.15.5:0/2977006301 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7fbbac00f5a0 con 0x56322f197af0 2024-09-19T06:17:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.860+0000 7fbbbf277740 10 client.19068 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T06:17:07.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.860+0000 7fbbbf277740 10 client.19068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:07.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.860+0000 7fbbbf277740 20 client.19068 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:07.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.860+0000 7fbbbf277740 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56322f21e3a0 0x56322f2207c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:07.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.860+0000 7fbbbf277740 4 client.19068 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:17:07.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.860+0000 7fbbbf277740 1 -- 172.21.15.5:0/2977006301 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56322f103d70 con 0x56322f21e3a0 2024-09-19T06:17:07.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.860+0000 7fbbb9a81640 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56322f21e3a0 0x56322f2207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:07.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.861+0000 7fbbb9a81640 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56322f21e3a0 0x56322f2207c0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f19677b7d6a7ea server_cookie=739eb6be2365cbc3 in_seq=0 out_seq=0 2024-09-19T06:17:07.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.861+0000 7fbb9dffb640 10 client.19068 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.903+0000 7fbb9dffb640 1 -- 172.21.15.5:0/2977006301 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56322f103d70 con 0x56322f21e3a0 2024-09-19T06:17:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.903+0000 7fbb9dffb640 10 client.19068 handle_command_reply: tid=0 2024-09-19T06:17:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.903+0000 7fbbbf277740 1 client.19068 shutdown 2024-09-19T06:17:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.903+0000 7fbbbf277740 1 -- 172.21.15.5:0/2977006301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f197af0 msgr2=0x56322f19cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.903+0000 7fbbbf277740 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f197af0 0x56322f19cbb0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fbbac012fd0 tx=0x7fbbac005f30 comp rx=0 tx=0).stop 2024-09-19T06:17:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.904+0000 7fbbbf277740 1 -- 172.21.15.5:0/2977006301 shutdown_connections 2024-09-19T06:17:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.904+0000 7fbbbf277740 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56322f21e3a0 0x56322f2207c0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.904+0000 7fbbbf277740 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322f1a2cb0 0x56322f19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.904+0000 7fbbbf277740 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322f198410 0x56322f19d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.904+0000 7fbbbf277740 1 --2- 172.21.15.5:0/2977006301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322f197af0 0x56322f19cbb0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.904+0000 7fbbbf277740 1 -- 172.21.15.5:0/2977006301 >> 172.21.15.5:0/2977006301 conn(0x56322f194060 msgr2=0x56322f1929a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.904+0000 7fbb9dffb640 0 client.19068 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.904+0000 7fbbbf277740 1 -- 172.21.15.5:0/2977006301 shutdown_connections 2024-09-19T06:17:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.904+0000 7fbbbf277740 1 -- 172.21.15.5:0/2977006301 wait complete. 2024-09-19T06:17:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.904+0000 7fbbbf277740 20 client.19068 trim_cache size 0 max 16384 2024-09-19T06:17:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:17:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:17:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:17:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:17:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.905+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3151111071 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbba00a33b0 msgr2=0x7fbba00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.905+0000 7fbbbc4e6640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbba00a33b0 0x7fbba00a5870 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x56322f194ec0 tx=0x7fbbb00f3650 comp rx=0 tx=0).stop 2024-09-19T06:17:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.905+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3151111071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb4103f40 msgr2=0x7fbbb419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.905+0000 7fbbbc4e6640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb4103f40 0x7fbbb419a630 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0005940 tx=0x7fbbb00040c0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.905+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3151111071 shutdown_connections 2024-09-19T06:17:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.905+0000 7fbbbc4e6640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb419eef0 0x7fbbb419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.905+0000 7fbbbc4e6640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbba00a33b0 0x7fbba00a5870 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.905+0000 7fbbbc4e6640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbb4109090 0x7fbbb419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.905+0000 7fbbbc4e6640 1 --2- 172.21.15.5:0/3151111071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb4103f40 0x7fbbb419a630 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.905+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3151111071 >> 172.21.15.5:0/3151111071 conn(0x7fbbb40ff970 msgr2=0x7fbbb4100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.905+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3151111071 shutdown_connections 2024-09-19T06:17:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:07.905+0000 7fbbbc4e6640 1 -- 172.21.15.5:0/3151111071 wait complete. 2024-09-19T06:17:08.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 5384 14.27 MB/sec warmup 32 sec latency 74.541 ms 2024-09-19T06:17:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:08 smithi157 ceph-mon[35346]: pgmap v432: 97 pgs: 97 active+clean; 411 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 306 op/s 2024-09-19T06:17:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:08 smithi157 ceph-mon[35346]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T06:17:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:08 smithi005 ceph-mon[30955]: pgmap v432: 97 pgs: 97 active+clean; 411 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 306 op/s 2024-09-19T06:17:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:08 smithi005 ceph-mon[30955]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T06:17:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:08 smithi049 ceph-mon[34640]: pgmap v432: 97 pgs: 97 active+clean; 411 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 306 op/s 2024-09-19T06:17:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:08 smithi049 ceph-mon[34640]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T06:17:09.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 5536 13.92 MB/sec warmup 33 sec latency 94.678 ms 2024-09-19T06:17:10.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 5719 13.67 MB/sec warmup 34 sec latency 69.164 ms 2024-09-19T06:17:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:10 smithi157 ceph-mon[35346]: pgmap v434: 97 pgs: 97 active+clean; 411 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 382 op/s 2024-09-19T06:17:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:10 smithi005 ceph-mon[30955]: pgmap v434: 97 pgs: 97 active+clean; 411 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 382 op/s 2024-09-19T06:17:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:10 smithi049 ceph-mon[34640]: pgmap v434: 97 pgs: 97 active+clean; 411 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 382 op/s 2024-09-19T06:17:11.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 5927 13.72 MB/sec warmup 35 sec latency 56.061 ms 2024-09-19T06:17:12.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 6157 13.67 MB/sec warmup 36 sec latency 68.154 ms 2024-09-19T06:17:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:12 smithi157 ceph-mon[35346]: pgmap v435: 97 pgs: 97 active+clean; 411 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-09-19T06:17:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:12 smithi005 ceph-mon[30955]: pgmap v435: 97 pgs: 97 active+clean; 411 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-09-19T06:17:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:12 smithi049 ceph-mon[34640]: pgmap v435: 97 pgs: 97 active+clean; 411 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-09-19T06:17:13.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 6353 13.72 MB/sec warmup 37 sec latency 109.493 ms 2024-09-19T06:17:14.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 6510 13.53 MB/sec warmup 38 sec latency 54.400 ms 2024-09-19T06:17:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:14 smithi157 ceph-mon[35346]: pgmap v436: 97 pgs: 97 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 373 op/s 2024-09-19T06:17:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:14 smithi005 ceph-mon[30955]: pgmap v436: 97 pgs: 97 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 373 op/s 2024-09-19T06:17:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:14 smithi049 ceph-mon[34640]: pgmap v436: 97 pgs: 97 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 373 op/s 2024-09-19T06:17:15.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 6708 13.72 MB/sec warmup 39 sec latency 64.370 ms 2024-09-19T06:17:16.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 6913 13.82 MB/sec warmup 40 sec latency 56.492 ms 2024-09-19T06:17:16.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:16 smithi157 ceph-mon[35346]: pgmap v437: 97 pgs: 97 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 310 op/s 2024-09-19T06:17:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:16 smithi005 ceph-mon[30955]: pgmap v437: 97 pgs: 97 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 310 op/s 2024-09-19T06:17:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:16 smithi049 ceph-mon[34640]: pgmap v437: 97 pgs: 97 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 310 op/s 2024-09-19T06:17:17.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7130 13.81 MB/sec warmup 41 sec latency 58.551 ms 2024-09-19T06:17:18.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7256 13.63 MB/sec warmup 42 sec latency 55.828 ms 2024-09-19T06:17:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:18 smithi157 ceph-mon[35346]: pgmap v438: 97 pgs: 97 active+clean; 439 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 5.4 MiB/s wr, 253 op/s 2024-09-19T06:17:18.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:18 smithi005 ceph-mon[30955]: pgmap v438: 97 pgs: 97 active+clean; 439 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 5.4 MiB/s wr, 253 op/s 2024-09-19T06:17:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:18 smithi049 ceph-mon[34640]: pgmap v438: 97 pgs: 97 active+clean; 439 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 5.4 MiB/s wr, 253 op/s 2024-09-19T06:17:19.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7384 13.41 MB/sec warmup 43 sec latency 86.945 ms 2024-09-19T06:17:19.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:19 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:17:19.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:19 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:17:19.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:19 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:17:20.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7559 13.37 MB/sec warmup 44 sec latency 103.939 ms 2024-09-19T06:17:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:20 smithi157 ceph-mon[35346]: pgmap v439: 97 pgs: 97 active+clean; 439 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 5.2 MiB/s wr, 242 op/s 2024-09-19T06:17:20.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:20 smithi005 ceph-mon[30955]: pgmap v439: 97 pgs: 97 active+clean; 439 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 5.2 MiB/s wr, 242 op/s 2024-09-19T06:17:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:20 smithi049 ceph-mon[34640]: pgmap v439: 97 pgs: 97 active+clean; 439 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 5.2 MiB/s wr, 242 op/s 2024-09-19T06:17:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.034+0000 7f9b0dffb640 1 -- 172.21.15.5:0/634191878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04031880 con 0x7f9b10100810 2024-09-19T06:17:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.035+0000 7f9b0ffff640 1 -- 172.21.15.5:0/634191878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b10100810 msgr2=0x7f9b10100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.035+0000 7f9b0ffff640 1 --2- 172.21.15.5:0/634191878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b10100810 0x7f9b10100c70 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009930 tx=0x7f9b0402a420 comp rx=0 tx=0).stop 2024-09-19T06:17:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.035+0000 7f9b0ffff640 1 -- 172.21.15.5:0/634191878 shutdown_connections 2024-09-19T06:17:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.035+0000 7f9b0ffff640 1 --2- 172.21.15.5:0/634191878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b101011b0 0x7f9b1010d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.035+0000 7f9b0ffff640 1 --2- 172.21.15.5:0/634191878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b10100810 0x7f9b10100c70 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.035+0000 7f9b0ffff640 1 --2- 172.21.15.5:0/634191878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b10106820 0x7f9b10106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.035+0000 7f9b0ffff640 1 -- 172.21.15.5:0/634191878 >> 172.21.15.5:0/634191878 conn(0x7f9b100777f0 msgr2=0x7f9b100feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.035+0000 7f9b0ffff640 1 -- 172.21.15.5:0/634191878 shutdown_connections 2024-09-19T06:17:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.035+0000 7f9b0ffff640 1 -- 172.21.15.5:0/634191878 wait complete. 2024-09-19T06:17:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.036+0000 7f9b0ffff640 1 Processor -- start 2024-09-19T06:17:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.036+0000 7f9b0ffff640 1 -- start start 2024-09-19T06:17:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.036+0000 7f9b0ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b10100810 0x7f9b1010dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.036+0000 7f9b0ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b101011b0 0x7f9b1010e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.036+0000 7f9b0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b10106820 0x7f9b1010ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.036+0000 7f9b0ffff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10111c00 con 0x7f9b10106820 2024-09-19T06:17:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.036+0000 7f9b0ffff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10111a80 con 0x7f9b10100810 2024-09-19T06:17:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.036+0000 7f9b0e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b101011b0 0x7f9b1010e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b0e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b101011b0 0x7f9b1010e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57770/0 (socket says 172.21.15.5:57770) 2024-09-19T06:17:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b0e7fc640 1 -- 172.21.15.5:0/1141680898 learned_addr learned my addr 172.21.15.5:0/1141680898 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b0f7fe640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b10106820 0x7f9b1010ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b0effd640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b10100810 0x7f9b1010dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b0ffff640 1 -- 172.21.15.5:0/1141680898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10111d80 con 0x7f9b101011b0 2024-09-19T06:17:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b0f7fe640 1 -- 172.21.15.5:0/1141680898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b101011b0 msgr2=0x7f9b1010e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b0f7fe640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b101011b0 0x7f9b1010e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b0f7fe640 1 -- 172.21.15.5:0/1141680898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b10100810 msgr2=0x7f9b1010dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b0f7fe640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b10100810 0x7f9b1010dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b0f7fe640 1 -- 172.21.15.5:0/1141680898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b101a6690 con 0x7f9b10106820 2024-09-19T06:17:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b0f7fe640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b10106820 0x7f9b1010ea70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009fd0 tx=0x7f9b0800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b14802640 1 -- 172.21.15.5:0/1141680898 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0800ce20 con 0x7f9b10106820 2024-09-19T06:17:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b0ffff640 1 -- 172.21.15.5:0/1141680898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b101a6980 con 0x7f9b10106820 2024-09-19T06:17:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b14802640 1 -- 172.21.15.5:0/1141680898 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b08005020 con 0x7f9b10106820 2024-09-19T06:17:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.037+0000 7f9b14802640 1 -- 172.21.15.5:0/1141680898 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08010690 con 0x7f9b10106820 2024-09-19T06:17:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.038+0000 7f9b0ffff640 1 -- 172.21.15.5:0/1141680898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b101a6ea0 con 0x7f9b10106820 2024-09-19T06:17:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.040+0000 7f9b14802640 1 -- 172.21.15.5:0/1141680898 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9b080051d0 con 0x7f9b10106820 2024-09-19T06:17:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.041+0000 7f9b14802640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9af80a3420 0x7f9af80a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.041+0000 7f9b0effd640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9af80a3420 0x7f9af80a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.041+0000 7f9b14802640 1 -- 172.21.15.5:0/1141680898 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f9b080d03f0 con 0x7f9b10106820 2024-09-19T06:17:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.042+0000 7f9b18678740 1 Processor -- start 2024-09-19T06:17:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.042+0000 7f9b18678740 1 -- start start 2024-09-19T06:17:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.042+0000 7f9b0effd640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9af80a3420 0x7f9af80a58e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009b50 tx=0x7f9b00005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.042+0000 7f9b18678740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 0x561fab2500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.042+0000 7f9b18678740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 0x561fab25a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.042+0000 7f9b18678740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab25ae80 0x561fab25d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.042+0000 7f9b0effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 0x561fab2500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.042+0000 7f9b0effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 0x561fab2500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57782/0 (socket says 172.21.15.5:57782) 2024-09-19T06:17:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.042+0000 7f9b0e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 0x561fab25a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.042+0000 7f9b0e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 0x561fab25a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39388/0 (socket says 172.21.15.5:39388) 2024-09-19T06:17:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9b0f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab25ae80 0x561fab25d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9b18678740 1 -- 172.21.15.5:0/740097410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561fab1b42f0 con 0x561fab25ae80 2024-09-19T06:17:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9b0effd640 1 -- 172.21.15.5:0/740097410 learned_addr learned my addr 172.21.15.5:0/740097410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9b18678740 1 -- 172.21.15.5:0/740097410 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561fab15c270 con 0x561fab2505e0 2024-09-19T06:17:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9b18678740 1 -- 172.21.15.5:0/740097410 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561fab1b02b0 con 0x561fab24fcc0 2024-09-19T06:17:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9b0f7fe640 1 -- 172.21.15.5:0/740097410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 msgr2=0x561fab2500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9b0f7fe640 1 --2- 172.21.15.5:0/740097410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 0x561fab2500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9b0f7fe640 1 -- 172.21.15.5:0/740097410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 msgr2=0x561fab25a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9b0f7fe640 1 --2- 172.21.15.5:0/740097410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 0x561fab25a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9b0f7fe640 1 -- 172.21.15.5:0/740097410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fab15b340 con 0x561fab25ae80 2024-09-19T06:17:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9b0effd640 1 --2- 172.21.15.5:0/740097410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 0x561fab2500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9b0f7fe640 1 --2- 172.21.15.5:0/740097410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab25ae80 0x561fab25d270 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9b080cdac0 tx=0x7f9b0809ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2d09bb9b007456 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9af5ffb640 1 -- 172.21.15.5:0/740097410 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080b4020 con 0x561fab25ae80 2024-09-19T06:17:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9af5ffb640 1 -- 172.21.15.5:0/740097410 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b08005540 con 0x561fab25ae80 2024-09-19T06:17:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.043+0000 7f9af5ffb640 1 -- 172.21.15.5:0/740097410 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080a5090 con 0x561fab25ae80 2024-09-19T06:17:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.044+0000 7f9b18678740 1 -- 172.21.15.5:0/740097410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab25ae80 msgr2=0x561fab25d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.044+0000 7f9b18678740 1 --2- 172.21.15.5:0/740097410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab25ae80 0x561fab25d270 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9b080cdac0 tx=0x7f9b0809ce90 comp rx=0 tx=0).stop 2024-09-19T06:17:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.045+0000 7f9b18678740 1 -- 172.21.15.5:0/740097410 shutdown_connections 2024-09-19T06:17:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.045+0000 7f9b18678740 1 --2- 172.21.15.5:0/740097410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab25ae80 0x561fab25d270 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.045+0000 7f9b18678740 1 --2- 172.21.15.5:0/740097410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 0x561fab25a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.045+0000 7f9b18678740 1 --2- 172.21.15.5:0/740097410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 0x561fab2500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.045+0000 7f9b18678740 1 -- 172.21.15.5:0/740097410 >> 172.21.15.5:0/740097410 conn(0x561fab24c230 msgr2=0x561fab24c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.045+0000 7f9b18678740 1 -- 172.21.15.5:0/740097410 shutdown_connections 2024-09-19T06:17:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.045+0000 7f9b18678740 1 -- 172.21.15.5:0/740097410 wait complete. 2024-09-19T06:17:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.045+0000 7f9b18678740 1 -- start start 2024-09-19T06:17:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.045+0000 7f9b18678740 1 Processor -- start 2024-09-19T06:17:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b18678740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab24fcc0 0x561fab257dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b18678740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab2505e0 0x561fab258300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b18678740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab25ae80 0x561fab258840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b0effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab24fcc0 0x561fab257dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b18678740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561fab25da50 con 0x561fab24fcc0 2024-09-19T06:17:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b0effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab24fcc0 0x561fab257dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48320/0 (socket says 172.21.15.5:48320) 2024-09-19T06:17:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b0effd640 1 -- 172.21.15.5:0/773706353 learned_addr learned my addr 172.21.15.5:0/773706353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b18678740 1 -- 172.21.15.5:0/773706353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561fab13d290 con 0x561fab25ae80 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b18678740 1 -- 172.21.15.5:0/773706353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561fab25dbd0 con 0x561fab2505e0 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b0e7fc640 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab2505e0 0x561fab258300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b18678740 10 client.0 _ll_register_callbacks cb 0x561fab16e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b0f7fe640 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab25ae80 0x561fab258840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b0effd640 1 -- 172.21.15.5:0/773706353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab2505e0 msgr2=0x561fab258300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b0effd640 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab2505e0 0x561fab258300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b0effd640 1 -- 172.21.15.5:0/773706353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab25ae80 msgr2=0x561fab258840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b0effd640 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab25ae80 0x561fab258840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b0effd640 1 -- 172.21.15.5:0/773706353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561faaec7210 con 0x561fab24fcc0 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.046+0000 7f9b0f7fe640 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab25ae80 0x561fab258840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.047+0000 7f9b0effd640 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab24fcc0 0x561fab257dc0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9b000118d0 tx=0x7f9b00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ef9820001cb976 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.047+0000 7f9aeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:17:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.047+0000 7f9aeeffd640 1 -- 172.21.15.5:0/773706353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0001fd90 con 0x561fab24fcc0 2024-09-19T06:17:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.047+0000 7f9aeeffd640 1 -- 172.21.15.5:0/773706353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f9b000273e0 con 0x561fab24fcc0 2024-09-19T06:17:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.047+0000 7f9b18678740 1 -- 172.21.15.5:0/773706353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b08005540 con 0x561fab24fcc0 2024-09-19T06:17:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.047+0000 7f9aeeffd640 1 -- 172.21.15.5:0/773706353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9b000275e0 con 0x561fab24fcc0 2024-09-19T06:17:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.047+0000 7f9b18678740 10 client.19080 fetch_fsmap learned FSMap version 43 2024-09-19T06:17:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.047+0000 7f9b18678740 1 -- 172.21.15.5:0/773706353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561faaf93620 con 0x561fab24fcc0 2024-09-19T06:17:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.047+0000 7f9aeeffd640 1 -- 172.21.15.5:0/773706353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b08005540 con 0x561fab24fcc0 2024-09-19T06:17:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.048+0000 7f9aeeffd640 1 -- 172.21.15.5:0/773706353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7f9b00026070 con 0x561fab24fcc0 2024-09-19T06:17:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.048+0000 7f9b18678740 10 client.19080 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:17:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.048+0000 7f9b18678740 10 client.19080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.048+0000 7f9b18678740 20 client.19080 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.048+0000 7f9b18678740 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fab26af00 0x561fab26d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.048+0000 7f9b18678740 4 client.19080 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:17:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.048+0000 7f9b18678740 1 -- 172.21.15.5:0/773706353 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fab1c1a20 con 0x561fab26af00 2024-09-19T06:17:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.048+0000 7f9b0e7fc640 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fab26af00 0x561fab26d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.049+0000 7f9b0e7fc640 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fab26af00 0x561fab26d370 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ff283801565898a server_cookie=cf4ab141a44f268d in_seq=0 out_seq=0 2024-09-19T06:17:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.049+0000 7f9aeeffd640 10 client.19080 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9aeeffd640 1 -- 172.21.15.5:0/773706353 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fab1c1a20 con 0x561fab26af00 2024-09-19T06:17:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9aeeffd640 10 client.19080 handle_command_reply: tid=0 2024-09-19T06:17:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9b18678740 1 client.19080 shutdown 2024-09-19T06:17:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9b18678740 1 -- 172.21.15.5:0/773706353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab24fcc0 msgr2=0x561fab257dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9b18678740 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab24fcc0 0x561fab257dc0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9b000118d0 tx=0x7f9b00004730 comp rx=0 tx=0).stop 2024-09-19T06:17:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9b18678740 1 -- 172.21.15.5:0/773706353 shutdown_connections 2024-09-19T06:17:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9b18678740 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fab26af00 0x561fab26d370 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9b18678740 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab25ae80 0x561fab258840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9aeeffd640 0 client.19080 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9b18678740 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab2505e0 0x561fab258300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9b18678740 1 --2- 172.21.15.5:0/773706353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab24fcc0 0x561fab257dc0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9b18678740 1 -- 172.21.15.5:0/773706353 >> 172.21.15.5:0/773706353 conn(0x561fab24c230 msgr2=0x561fab251f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.050+0000 7f9b18678740 1 -- 172.21.15.5:0/773706353 shutdown_connections 2024-09-19T06:17:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.051+0000 7f9b18678740 1 -- 172.21.15.5:0/773706353 wait complete. 2024-09-19T06:17:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.051+0000 7f9b18678740 20 client.19080 trim_cache size 0 max 16384 2024-09-19T06:17:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.074+0000 7f9b18678740 1 Processor -- start 2024-09-19T06:17:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.074+0000 7f9b18678740 1 -- start start 2024-09-19T06:17:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b18678740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab24fcc0 0x561fab258670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b18678740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab2505e0 0x561fab272c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b18678740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab25ae80 0x561fab273190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b18678740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101af830 con 0x561fab2505e0 2024-09-19T06:17:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b18678740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101b14c0 con 0x561fab24fcc0 2024-09-19T06:17:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b18678740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101aff30 con 0x561fab25ae80 2024-09-19T06:17:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b0effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab24fcc0 0x561fab258670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b0e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab2505e0 0x561fab272c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b0effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab24fcc0 0x561fab258670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39408/0 (socket says 172.21.15.5:39408) 2024-09-19T06:17:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b0effd640 1 -- 172.21.15.5:0/3504045785 learned_addr learned my addr 172.21.15.5:0/3504045785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b0f7fe640 1 --2- 172.21.15.5:0/3504045785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab25ae80 0x561fab273190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b0effd640 1 -- 172.21.15.5:0/3504045785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab25ae80 msgr2=0x561fab273190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b0effd640 1 --2- 172.21.15.5:0/3504045785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab25ae80 0x561fab273190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b0effd640 1 -- 172.21.15.5:0/3504045785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab2505e0 msgr2=0x561fab272c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b0effd640 1 --2- 172.21.15.5:0/3504045785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab2505e0 0x561fab272c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b0effd640 1 -- 172.21.15.5:0/3504045785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fab267e50 con 0x561fab24fcc0 2024-09-19T06:17:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.075+0000 7f9b0f7fe640 1 --2- 172.21.15.5:0/3504045785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab25ae80 0x561fab273190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:17:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.076+0000 7f9b0effd640 1 --2- 172.21.15.5:0/3504045785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab24fcc0 0x561fab258670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9b00004700 tx=0x7f9b00005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=213804e744b44525 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.076+0000 7f9af57fa640 1 -- 172.21.15.5:0/3504045785 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b000273e0 con 0x561fab24fcc0 2024-09-19T06:17:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.076+0000 7f9af57fa640 1 -- 172.21.15.5:0/3504045785 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b00028bd0 con 0x561fab24fcc0 2024-09-19T06:17:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.076+0000 7f9af57fa640 1 -- 172.21.15.5:0/3504045785 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00030420 con 0x561fab24fcc0 2024-09-19T06:17:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.077+0000 7f9b18678740 1 -- 172.21.15.5:0/3504045785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab24fcc0 msgr2=0x561fab258670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.077+0000 7f9b18678740 1 --2- 172.21.15.5:0/3504045785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab24fcc0 0x561fab258670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9b00004700 tx=0x7f9b00005870 comp rx=0 tx=0).stop 2024-09-19T06:17:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.077+0000 7f9b18678740 1 -- 172.21.15.5:0/3504045785 shutdown_connections 2024-09-19T06:17:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.077+0000 7f9b18678740 1 --2- 172.21.15.5:0/3504045785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab25ae80 0x561fab273190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.077+0000 7f9b18678740 1 --2- 172.21.15.5:0/3504045785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab2505e0 0x561fab272c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.077+0000 7f9b18678740 1 --2- 172.21.15.5:0/3504045785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab24fcc0 0x561fab258670 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.077+0000 7f9b18678740 1 -- 172.21.15.5:0/3504045785 >> 172.21.15.5:0/3504045785 conn(0x561fab24c230 msgr2=0x561fab256e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.077+0000 7f9b18678740 1 -- 172.21.15.5:0/3504045785 shutdown_connections 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.077+0000 7f9b18678740 1 -- 172.21.15.5:0/3504045785 wait complete. 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.077+0000 7f9b18678740 1 -- start start 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b18678740 1 Processor -- start 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b18678740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 0x561fab25e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b18678740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 0x561fab25e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b18678740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab25ae80 0x561fab2d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b18678740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101b0df0 con 0x561fab25ae80 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b18678740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101b0660 con 0x561fab2505e0 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b18678740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101ad830 con 0x561fab24fcc0 2024-09-19T06:17:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b18678740 10 client.0 _ll_register_callbacks cb 0x7f9b00004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b0e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 0x561fab25e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b0effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 0x561fab25e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b0f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab25ae80 0x561fab2d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b0e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 0x561fab25e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39414/0 (socket says 172.21.15.5:39414) 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b0effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 0x561fab25e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57812/0 (socket says 172.21.15.5:57812) 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.078+0000 7f9b0e7fc640 1 -- 172.21.15.5:0/154266375 learned_addr learned my addr 172.21.15.5:0/154266375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9b0e7fc640 1 -- 172.21.15.5:0/154266375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 msgr2=0x561fab25e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9b0e7fc640 1 --2- 172.21.15.5:0/154266375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 0x561fab25e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9b0e7fc640 1 -- 172.21.15.5:0/154266375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab25ae80 msgr2=0x561fab2d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9b0e7fc640 1 --2- 172.21.15.5:0/154266375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab25ae80 0x561fab2d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9b0e7fc640 1 -- 172.21.15.5:0/154266375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fab2d4220 con 0x561fab2505e0 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9b0effd640 1 --2- 172.21.15.5:0/154266375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 0x561fab25e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9b0e7fc640 1 --2- 172.21.15.5:0/154266375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 0x561fab25e920 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9b04000c00 tx=0x7f9b0402ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=200dda305aa8460d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9aeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:17:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9aeeffd640 1 -- 172.21.15.5:0/154266375 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0404a070 con 0x561fab2505e0 2024-09-19T06:17:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9b18678740 1 -- 172.21.15.5:0/154266375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b00028bd0 con 0x561fab2505e0 2024-09-19T06:17:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9aeeffd640 1 -- 172.21.15.5:0/154266375 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f9b040316a0 con 0x561fab2505e0 2024-09-19T06:17:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9aeeffd640 1 -- 172.21.15.5:0/154266375 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9b04035040 con 0x561fab2505e0 2024-09-19T06:17:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9b18678740 10 client.28487 fetch_fsmap learned FSMap version 43 2024-09-19T06:17:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.079+0000 7f9b18678740 1 -- 172.21.15.5:0/154266375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561fab2d55c0 con 0x561fab2505e0 2024-09-19T06:17:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.080+0000 7f9aeeffd640 1 -- 172.21.15.5:0/154266375 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b00028bd0 con 0x561fab2505e0 2024-09-19T06:17:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.080+0000 7f9aeeffd640 1 -- 172.21.15.5:0/154266375 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7f9b0403d450 con 0x561fab2505e0 2024-09-19T06:17:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.080+0000 7f9b18678740 10 client.28487 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:17:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.080+0000 7f9b18678740 10 client.28487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.080+0000 7f9b18678740 20 client.28487 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.080+0000 7f9b18678740 1 --2- 172.21.15.5:0/154266375 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fab2d5de0 0x561fab2d8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.080+0000 7f9b18678740 4 client.28487 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:17:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.080+0000 7f9b18678740 1 -- 172.21.15.5:0/154266375 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561fab1bbd70 con 0x561fab2d5de0 2024-09-19T06:17:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.080+0000 7f9b0effd640 1 --2- 172.21.15.5:0/154266375 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fab2d5de0 0x561fab2d8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.081+0000 7f9aeeffd640 10 client.28487 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.081+0000 7f9b0effd640 1 --2- 172.21.15.5:0/154266375 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fab2d5de0 0x561fab2d8200 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca76db9158c1394 server_cookie=2b729c6cb52d6cba in_seq=0 out_seq=0 2024-09-19T06:17:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.132+0000 7f9aeeffd640 1 -- 172.21.15.5:0/154266375 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561fab1bbd70 con 0x561fab2d5de0 2024-09-19T06:17:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.132+0000 7f9aeeffd640 10 client.28487 handle_command_reply: tid=0 2024-09-19T06:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.132+0000 7f9b18678740 1 client.28487 shutdown 2024-09-19T06:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.132+0000 7f9b18678740 1 -- 172.21.15.5:0/154266375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 msgr2=0x561fab25e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.132+0000 7f9b18678740 1 --2- 172.21.15.5:0/154266375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 0x561fab25e920 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9b04000c00 tx=0x7f9b0402ab90 comp rx=0 tx=0).stop 2024-09-19T06:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.132+0000 7f9b18678740 1 -- 172.21.15.5:0/154266375 shutdown_connections 2024-09-19T06:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.132+0000 7f9b18678740 1 --2- 172.21.15.5:0/154266375 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561fab2d5de0 0x561fab2d8200 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.132+0000 7f9b18678740 1 --2- 172.21.15.5:0/154266375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fab25ae80 0x561fab2d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.132+0000 7f9aeeffd640 0 client.28487 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.132+0000 7f9b18678740 1 --2- 172.21.15.5:0/154266375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fab2505e0 0x561fab25e920 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.132+0000 7f9b18678740 1 --2- 172.21.15.5:0/154266375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fab24fcc0 0x561fab25e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.133+0000 7f9b18678740 1 -- 172.21.15.5:0/154266375 >> 172.21.15.5:0/154266375 conn(0x561fab24c230 msgr2=0x561fab259620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.133+0000 7f9b18678740 1 -- 172.21.15.5:0/154266375 shutdown_connections 2024-09-19T06:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.133+0000 7f9b18678740 1 -- 172.21.15.5:0/154266375 wait complete. 2024-09-19T06:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.133+0000 7f9b18678740 20 client.28487 trim_cache size 0 max 16384 2024-09-19T06:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:17:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.133+0000 7f9b0ffff640 1 -- 172.21.15.5:0/1141680898 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9af80a3420 msgr2=0x7f9af80a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.133+0000 7f9b0ffff640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9af80a3420 0x7f9af80a58e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009b50 tx=0x7f9b00005f90 comp rx=0 tx=0).stop 2024-09-19T06:17:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.133+0000 7f9b0ffff640 1 -- 172.21.15.5:0/1141680898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b10106820 msgr2=0x7f9b1010ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.133+0000 7f9b0ffff640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b10106820 0x7f9b1010ea70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009fd0 tx=0x7f9b0800ef10 comp rx=0 tx=0).stop 2024-09-19T06:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.134+0000 7f9b0ffff640 1 -- 172.21.15.5:0/1141680898 shutdown_connections 2024-09-19T06:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.134+0000 7f9b0ffff640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b10106820 0x7f9b1010ea70 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.134+0000 7f9b0ffff640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9af80a3420 0x7f9af80a58e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.134+0000 7f9b0ffff640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b101011b0 0x7f9b1010e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.134+0000 7f9b0ffff640 1 --2- 172.21.15.5:0/1141680898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b10100810 0x7f9b1010dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.134+0000 7f9b0ffff640 1 -- 172.21.15.5:0/1141680898 >> 172.21.15.5:0/1141680898 conn(0x7f9b100777f0 msgr2=0x7f9b100feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.134+0000 7f9b0ffff640 1 -- 172.21.15.5:0/1141680898 shutdown_connections 2024-09-19T06:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:21.134+0000 7f9b0ffff640 1 -- 172.21.15.5:0/1141680898 wait complete. 2024-09-19T06:17:21.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7721 13.23 MB/sec warmup 45 sec latency 104.938 ms 2024-09-19T06:17:22.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7888 13.06 MB/sec warmup 46 sec latency 54.256 ms 2024-09-19T06:17:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:22 smithi157 ceph-mon[35346]: pgmap v440: 97 pgs: 97 active+clean; 439 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 4.5 MiB/s wr, 211 op/s 2024-09-19T06:17:22.817 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:22 smithi005 ceph-mon[30955]: pgmap v440: 97 pgs: 97 active+clean; 439 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 4.5 MiB/s wr, 211 op/s 2024-09-19T06:17:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:22 smithi049 ceph-mon[34640]: pgmap v440: 97 pgs: 97 active+clean; 439 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 4.5 MiB/s wr, 211 op/s 2024-09-19T06:17:23.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8061 13.10 MB/sec warmup 47 sec latency 90.557 ms 2024-09-19T06:17:24.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:17:24.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8202 12.86 MB/sec warmup 48 sec latency 62.057 ms 2024-09-19T06:17:24.764 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:17:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:24 smithi157 ceph-mon[35346]: pgmap v441: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.5 MiB/s wr, 344 op/s 2024-09-19T06:17:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:24 smithi005 ceph-mon[30955]: pgmap v441: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.5 MiB/s wr, 344 op/s 2024-09-19T06:17:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:24 smithi049 ceph-mon[34640]: pgmap v441: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.5 MiB/s wr, 344 op/s 2024-09-19T06:17:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.338+0000 7f931407a640 1 -- 172.21.15.5:0/1944958079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c076720 msgr2=0x7f930c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.338+0000 7f931407a640 1 --2- 172.21.15.5:0/1944958079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c076720 0x7f930c076b00 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f9308009900 tx=0x7f930802a420 comp rx=0 tx=0).stop 2024-09-19T06:17:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.339+0000 7f931407a640 1 -- 172.21.15.5:0/1944958079 shutdown_connections 2024-09-19T06:17:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.339+0000 7f931407a640 1 --2- 172.21.15.5:0/1944958079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f930c112a30 0x7f930c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.339+0000 7f931407a640 1 --2- 172.21.15.5:0/1944958079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f930c077040 0x7f930c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.339+0000 7f931407a640 1 --2- 172.21.15.5:0/1944958079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c076720 0x7f930c076b00 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.339+0000 7f931407a640 1 -- 172.21.15.5:0/1944958079 >> 172.21.15.5:0/1944958079 conn(0x7f930c0ffa90 msgr2=0x7f930c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.342+0000 7f931407a640 1 -- 172.21.15.5:0/1944958079 shutdown_connections 2024-09-19T06:17:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.342+0000 7f931407a640 1 -- 172.21.15.5:0/1944958079 wait complete. 2024-09-19T06:17:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.343+0000 7f931407a640 1 Processor -- start 2024-09-19T06:17:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.343+0000 7f931407a640 1 -- start start 2024-09-19T06:17:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.343+0000 7f931407a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c076720 0x7f930c073a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.343+0000 7f931407a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f930c077040 0x7f930c06cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.343+0000 7f931407a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f930c112a30 0x7f930c06d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.343+0000 7f931407a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f930c118530 con 0x7f930c076720 2024-09-19T06:17:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.343+0000 7f931407a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1183b0 con 0x7f930c112a30 2024-09-19T06:17:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.343+0000 7f931407a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1186b0 con 0x7f930c077040 2024-09-19T06:17:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.343+0000 7f9312617640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f930c112a30 0x7f930c06d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.343+0000 7f9311e16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c076720 0x7f930c073a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.343+0000 7f9312617640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f930c112a30 0x7f930c06d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35656/0 (socket says 172.21.15.5:35656) 2024-09-19T06:17:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.343+0000 7f9312617640 1 -- 172.21.15.5:0/917871548 learned_addr learned my addr 172.21.15.5:0/917871548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.344+0000 7f9312617640 1 -- 172.21.15.5:0/917871548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f930c077040 msgr2=0x7f930c06cae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:17:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.344+0000 7f9312617640 1 --2- 172.21.15.5:0/917871548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f930c077040 0x7f930c06cae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.344+0000 7f9312617640 1 -- 172.21.15.5:0/917871548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c076720 msgr2=0x7f930c073a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.344+0000 7f9312617640 1 --2- 172.21.15.5:0/917871548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c076720 0x7f930c073a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.344+0000 7f9312617640 1 -- 172.21.15.5:0/917871548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c06d630 con 0x7f930c112a30 2024-09-19T06:17:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.344+0000 7f9312617640 1 --2- 172.21.15.5:0/917871548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f930c112a30 0x7f930c06d020 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f930400eb10 tx=0x7f930400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.344+0000 7f9302ffd640 1 -- 172.21.15.5:0/917871548 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304004b20 con 0x7f930c112a30 2024-09-19T06:17:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.344+0000 7f931407a640 1 -- 172.21.15.5:0/917871548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c06d920 con 0x7f930c112a30 2024-09-19T06:17:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.344+0000 7f931407a640 1 -- 172.21.15.5:0/917871548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c0718f0 con 0x7f930c112a30 2024-09-19T06:17:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.344+0000 7f9302ffd640 1 -- 172.21.15.5:0/917871548 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9304004cc0 con 0x7f930c112a30 2024-09-19T06:17:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.345+0000 7f9302ffd640 1 -- 172.21.15.5:0/917871548 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040186f0 con 0x7f930c112a30 2024-09-19T06:17:25.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.347+0000 7f9302ffd640 1 -- 172.21.15.5:0/917871548 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9304018930 con 0x7f930c112a30 2024-09-19T06:17:25.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.348+0000 7f9316453740 1 Processor -- start 2024-09-19T06:17:25.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.348+0000 7f9316453740 1 -- start start 2024-09-19T06:17:25.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.348+0000 7f9316453740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2c4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.348+0000 7f9316453740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.348+0000 7f9302ffd640 1 --2- 172.21.15.5:0/917871548 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92f80a3480 0x7f92f80a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.348+0000 7f9316453740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.348+0000 7f9316453740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c7c22a2e0 con 0x560c7c2c44a0 2024-09-19T06:17:25.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.348+0000 7f9316453740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560c7c1d1ce0 con 0x560c7c2c4dc0 2024-09-19T06:17:25.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.348+0000 7f9316453740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560c7c2262a0 con 0x560c7c2cf680 2024-09-19T06:17:25.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.349+0000 7f9311615640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.349+0000 7f9302ffd640 1 -- 172.21.15.5:0/917871548 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f93040d02a0 con 0x7f930c112a30 2024-09-19T06:17:25.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.349+0000 7f9311e16640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.349+0000 7f9311615640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55760/0 (socket says 172.21.15.5:55760) 2024-09-19T06:17:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.349+0000 7f9311615640 1 -- 172.21.15.5:0/3467472746 learned_addr learned my addr 172.21.15.5:0/3467472746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.349+0000 7f9311615640 1 --2- 172.21.15.5:0/917871548 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92f80a3480 0x7f92f80a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.349+0000 7f9312617640 1 --2- 172.21.15.5:0/3467472746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.349+0000 7f9311e16640 1 -- 172.21.15.5:0/3467472746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 msgr2=0x560c7c2d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.349+0000 7f9311e16640 1 --2- 172.21.15.5:0/3467472746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.349+0000 7f9311e16640 1 -- 172.21.15.5:0/3467472746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 msgr2=0x560c7c2c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.349+0000 7f9311e16640 1 --2- 172.21.15.5:0/3467472746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.349+0000 7f9311e16640 1 -- 172.21.15.5:0/3467472746 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7c1b6440 con 0x560c7c2c4dc0 2024-09-19T06:17:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.350+0000 7f9311e16640 1 --2- 172.21.15.5:0/3467472746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2cf140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f930802aa10 tx=0x7f9308005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cf282cef70a1c2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.350+0000 7f9311615640 1 --2- 172.21.15.5:0/917871548 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92f80a3480 0x7f92f80a5940 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f92fc014460 tx=0x7f92fc00b2f0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.350+0000 7f92f7fff640 1 -- 172.21.15.5:0/3467472746 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308035040 con 0x560c7c2c4dc0 2024-09-19T06:17:25.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.350+0000 7f92f7fff640 1 -- 172.21.15.5:0/3467472746 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93080054f0 con 0x560c7c2c4dc0 2024-09-19T06:17:25.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.350+0000 7f92f7fff640 1 -- 172.21.15.5:0/3467472746 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93080035f0 con 0x560c7c2c4dc0 2024-09-19T06:17:25.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.351+0000 7f9316453740 1 -- 172.21.15.5:0/3467472746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 msgr2=0x560c7c2cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:25.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.351+0000 7f9316453740 1 --2- 172.21.15.5:0/3467472746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2cf140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f930802aa10 tx=0x7f9308005fd0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.351+0000 7f9316453740 1 -- 172.21.15.5:0/3467472746 shutdown_connections 2024-09-19T06:17:25.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.351+0000 7f9316453740 1 --2- 172.21.15.5:0/3467472746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.351+0000 7f9316453740 1 --2- 172.21.15.5:0/3467472746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2cf140 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.351+0000 7f9316453740 1 --2- 172.21.15.5:0/3467472746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.351+0000 7f9316453740 1 -- 172.21.15.5:0/3467472746 >> 172.21.15.5:0/3467472746 conn(0x560c7c2c0a30 msgr2=0x560c7c2c0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:25.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.351+0000 7f9316453740 1 -- 172.21.15.5:0/3467472746 shutdown_connections 2024-09-19T06:17:25.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.351+0000 7f9316453740 1 -- 172.21.15.5:0/3467472746 wait complete. 2024-09-19T06:17:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.352+0000 7f9316453740 1 -- start start 2024-09-19T06:17:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.352+0000 7f9316453740 1 Processor -- start 2024-09-19T06:17:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.352+0000 7f9316453740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c44a0 0x560c7c2cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.352+0000 7f9316453740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c4dc0 0x560c7c2cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.352+0000 7f9316453740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.352+0000 7f9316453740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c7c2d2250 con 0x560c7c2c4dc0 2024-09-19T06:17:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.352+0000 7f9316453740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560c7c1e5050 con 0x560c7c2c44a0 2024-09-19T06:17:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.352+0000 7f9316453740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560c7c2d23d0 con 0x560c7c2cf680 2024-09-19T06:17:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.352+0000 7f9316453740 10 client.0 _ll_register_callbacks cb 0x560c7c225160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f9311615640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c4dc0 0x560c7c2cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f9311615640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c4dc0 0x560c7c2cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55772/0 (socket says 172.21.15.5:55772) 2024-09-19T06:17:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f9311615640 1 -- 172.21.15.5:0/331978129 learned_addr learned my addr 172.21.15.5:0/331978129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f9311e16640 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c44a0 0x560c7c2cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f9312617640 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f9311615640 1 -- 172.21.15.5:0/331978129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 msgr2=0x560c7c2cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f9311615640 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f9311615640 1 -- 172.21.15.5:0/331978129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c44a0 msgr2=0x560c7c2cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f9311615640 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c44a0 0x560c7c2cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f9311615640 1 -- 172.21.15.5:0/331978129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7bf9b3c0 con 0x560c7c2c4dc0 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f9311615640 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c4dc0 0x560c7c2cc210 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f92fc006350 tx=0x7f92fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128876fc3055ad81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f92f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f92f57fa640 1 -- 172.21.15.5:0/331978129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc011040 con 0x560c7c2c4dc0 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f92f57fa640 1 -- 172.21.15.5:0/331978129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f92fc016410 con 0x560c7c2c4dc0 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.353+0000 7f9316453740 1 -- 172.21.15.5:0/331978129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93080054f0 con 0x560c7c2c4dc0 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.354+0000 7f92f57fa640 1 -- 172.21.15.5:0/331978129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f92fc007b10 con 0x560c7c2c4dc0 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.354+0000 7f9316453740 10 client.19098 fetch_fsmap learned FSMap version 43 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.354+0000 7f9316453740 1 -- 172.21.15.5:0/331978129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560c7c2ddbf0 con 0x560c7c2c4dc0 2024-09-19T06:17:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.354+0000 7f92f57fa640 1 -- 172.21.15.5:0/331978129 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93080054f0 con 0x560c7c2c4dc0 2024-09-19T06:17:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.354+0000 7f92f57fa640 1 -- 172.21.15.5:0/331978129 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7f92fc008020 con 0x560c7c2c4dc0 2024-09-19T06:17:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.354+0000 7f9316453740 10 client.19098 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:17:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.354+0000 7f9316453740 10 client.19098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.354+0000 7f9316453740 20 client.19098 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.354+0000 7f9316453740 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560c7c2df1a0 0x560c7c2e1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.354+0000 7f9316453740 4 client.19098 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:17:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.354+0000 7f9316453740 1 -- 172.21.15.5:0/331978129 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c7c237850 con 0x560c7c2df1a0 2024-09-19T06:17:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.355+0000 7f9311e16640 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560c7c2df1a0 0x560c7c2e1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.355+0000 7f9311e16640 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560c7c2df1a0 0x560c7c2e1660 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a0e7e9092f535b5 server_cookie=9080705ffea3fb84 in_seq=0 out_seq=0 2024-09-19T06:17:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.355+0000 7f92f57fa640 10 client.19098 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.356+0000 7f92f57fa640 1 -- 172.21.15.5:0/331978129 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c7c237850 con 0x560c7c2df1a0 2024-09-19T06:17:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.356+0000 7f92f57fa640 10 client.19098 handle_command_reply: tid=0 2024-09-19T06:17:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.356+0000 7f9316453740 1 client.19098 shutdown 2024-09-19T06:17:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.356+0000 7f9316453740 1 -- 172.21.15.5:0/331978129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c4dc0 msgr2=0x560c7c2cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.356+0000 7f9316453740 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c4dc0 0x560c7c2cc210 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f92fc006350 tx=0x7f92fc017000 comp rx=0 tx=0).stop 2024-09-19T06:17:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.356+0000 7f9316453740 1 -- 172.21.15.5:0/331978129 shutdown_connections 2024-09-19T06:17:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.356+0000 7f9316453740 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560c7c2df1a0 0x560c7c2e1660 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.356+0000 7f9316453740 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.356+0000 7f9316453740 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c4dc0 0x560c7c2cc210 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.356+0000 7f92f57fa640 0 client.19098 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.356+0000 7f9316453740 1 --2- 172.21.15.5:0/331978129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c44a0 0x560c7c2cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.356+0000 7f9316453740 1 -- 172.21.15.5:0/331978129 >> 172.21.15.5:0/331978129 conn(0x560c7c2c0a30 msgr2=0x560c7c2c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.357+0000 7f9316453740 1 -- 172.21.15.5:0/331978129 shutdown_connections 2024-09-19T06:17:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.357+0000 7f9316453740 1 -- 172.21.15.5:0/331978129 wait complete. 2024-09-19T06:17:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.357+0000 7f9316453740 20 client.19098 trim_cache size 0 max 16384 2024-09-19T06:17:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.388+0000 7f9316453740 1 Processor -- start 2024-09-19T06:17:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.388+0000 7f9316453740 1 -- start start 2024-09-19T06:17:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.388+0000 7f9316453740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2db4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.388+0000 7f9316453740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.388+0000 7f9316453740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2e7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.388+0000 7f9311e16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2db4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.388+0000 7f9311e16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2db4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55784/0 (socket says 172.21.15.5:55784) 2024-09-19T06:17:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.388+0000 7f9311615640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.388+0000 7f9311615640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2e6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35686/0 (socket says 172.21.15.5:35686) 2024-09-19T06:17:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.389+0000 7f9311e16640 1 -- 172.21.15.5:0/3320599834 learned_addr learned my addr 172.21.15.5:0/3320599834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.389+0000 7f9316453740 1 -- 172.21.15.5:0/3320599834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92f000e7e0 con 0x560c7c2c44a0 2024-09-19T06:17:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.389+0000 7f9316453740 1 -- 172.21.15.5:0/3320599834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0010470 con 0x560c7c2c4dc0 2024-09-19T06:17:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.389+0000 7f9316453740 1 -- 172.21.15.5:0/3320599834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f92f000eee0 con 0x560c7c2cf680 2024-09-19T06:17:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.389+0000 7f9312617640 1 --2- 172.21.15.5:0/3320599834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.389+0000 7f9311e16640 1 -- 172.21.15.5:0/3320599834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 msgr2=0x560c7c2e7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.389+0000 7f9311e16640 1 --2- 172.21.15.5:0/3320599834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2e7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.389+0000 7f9311e16640 1 -- 172.21.15.5:0/3320599834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 msgr2=0x560c7c2e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.389+0000 7f9311e16640 1 --2- 172.21.15.5:0/3320599834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2e6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.389+0000 7f9311e16640 1 -- 172.21.15.5:0/3320599834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7c2cf2d0 con 0x560c7c2c44a0 2024-09-19T06:17:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.389+0000 7f9311615640 1 --2- 172.21.15.5:0/3320599834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.389+0000 7f9311e16640 1 --2- 172.21.15.5:0/3320599834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2db4e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f930802aa10 tx=0x7f9308004910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb934e55d9e2ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.390+0000 7f92f77fe640 1 -- 172.21.15.5:0/3320599834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308035040 con 0x560c7c2c44a0 2024-09-19T06:17:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.390+0000 7f92f77fe640 1 -- 172.21.15.5:0/3320599834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9308004980 con 0x560c7c2c44a0 2024-09-19T06:17:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.390+0000 7f92f77fe640 1 -- 172.21.15.5:0/3320599834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308031820 con 0x560c7c2c44a0 2024-09-19T06:17:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.391+0000 7f9316453740 1 -- 172.21.15.5:0/3320599834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 msgr2=0x560c7c2db4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.391+0000 7f9316453740 1 --2- 172.21.15.5:0/3320599834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2db4e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f930802aa10 tx=0x7f9308004910 comp rx=0 tx=0).stop 2024-09-19T06:17:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.391+0000 7f9316453740 1 -- 172.21.15.5:0/3320599834 shutdown_connections 2024-09-19T06:17:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.391+0000 7f9316453740 1 --2- 172.21.15.5:0/3320599834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2e7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.391+0000 7f9316453740 1 --2- 172.21.15.5:0/3320599834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.391+0000 7f9316453740 1 --2- 172.21.15.5:0/3320599834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2db4e0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.391+0000 7f9316453740 1 -- 172.21.15.5:0/3320599834 >> 172.21.15.5:0/3320599834 conn(0x560c7c2c0a30 msgr2=0x560c7c2cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.391+0000 7f9316453740 1 -- 172.21.15.5:0/3320599834 shutdown_connections 2024-09-19T06:17:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.391+0000 7f9316453740 1 -- 172.21.15.5:0/3320599834 wait complete. 2024-09-19T06:17:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.391+0000 7f9316453740 1 -- start start 2024-09-19T06:17:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.391+0000 7f9316453740 1 Processor -- start 2024-09-19T06:17:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.392+0000 7f9316453740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2d79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.392+0000 7f9316453740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2d7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.392+0000 7f9311e16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2d79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.392+0000 7f9316453740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.392+0000 7f9316453740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92f000fda0 con 0x560c7c2c44a0 2024-09-19T06:17:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.392+0000 7f9316453740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f92f000f610 con 0x560c7c2c4dc0 2024-09-19T06:17:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.392+0000 7f9316453740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f92f000c7e0 con 0x560c7c2cf680 2024-09-19T06:17:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.392+0000 7f9311e16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2d79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55792/0 (socket says 172.21.15.5:55792) 2024-09-19T06:17:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.392+0000 7f9311e16640 1 -- 172.21.15.5:0/4164256851 learned_addr learned my addr 172.21.15.5:0/4164256851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.392+0000 7f9316453740 10 client.0 _ll_register_callbacks cb 0x7f92fc00b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.392+0000 7f9311615640 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2d7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.392+0000 7f9312617640 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.393+0000 7f9311e16640 1 -- 172.21.15.5:0/4164256851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 msgr2=0x560c7c2d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.393+0000 7f9311e16640 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.393+0000 7f9311e16640 1 -- 172.21.15.5:0/4164256851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 msgr2=0x560c7c2d7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.393+0000 7f9311e16640 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2d7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.393+0000 7f9311e16640 1 -- 172.21.15.5:0/4164256851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7c2d3790 con 0x560c7c2c44a0 2024-09-19T06:17:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.393+0000 7f9312617640 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:17:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.393+0000 7f9311e16640 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2d79e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f9308050040 tx=0x7f9308004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83cc6b6bb411624f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:25.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.393+0000 7f92f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:17:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.393+0000 7f92f57fa640 1 -- 172.21.15.5:0/4164256851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308035040 con 0x560c7c2c44a0 2024-09-19T06:17:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.393+0000 7f92f57fa640 1 -- 172.21.15.5:0/4164256851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f93080053d0 con 0x560c7c2c44a0 2024-09-19T06:17:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.393+0000 7f9316453740 1 -- 172.21.15.5:0/4164256851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9308004980 con 0x560c7c2c44a0 2024-09-19T06:17:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.394+0000 7f92f57fa640 1 -- 172.21.15.5:0/4164256851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f93080314a0 con 0x560c7c2c44a0 2024-09-19T06:17:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.394+0000 7f9316453740 10 client.19104 fetch_fsmap learned FSMap version 43 2024-09-19T06:17:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.394+0000 7f9316453740 1 -- 172.21.15.5:0/4164256851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560c7c349b70 con 0x560c7c2c44a0 2024-09-19T06:17:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.394+0000 7f92f57fa640 1 -- 172.21.15.5:0/4164256851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9308031af0 con 0x560c7c2c44a0 2024-09-19T06:17:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.394+0000 7f92f57fa640 1 -- 172.21.15.5:0/4164256851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7f9308057070 con 0x560c7c2c44a0 2024-09-19T06:17:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.394+0000 7f9316453740 10 client.19104 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:17:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.394+0000 7f9316453740 10 client.19104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.394+0000 7f9316453740 20 client.19104 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.394+0000 7f9316453740 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560c7c349df0 0x560c7c34c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.394+0000 7f9316453740 4 client.19104 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.394+0000 7f9316453740 1 -- 172.21.15.5:0/4164256851 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9308004980 con 0x560c7c349df0 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.395+0000 7f9311615640 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560c7c349df0 0x560c7c34c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.395+0000 7f92f57fa640 10 client.19104 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.395+0000 7f9311615640 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560c7c349df0 0x560c7c34c1e0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cfb15005c55c036 server_cookie=9cb24d6089252bab in_seq=0 out_seq=0 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.395+0000 7f92f57fa640 1 -- 172.21.15.5:0/4164256851 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f9308004980 con 0x560c7c349df0 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.395+0000 7f92f57fa640 10 client.19104 handle_command_reply: tid=0 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.395+0000 7f9316453740 1 client.19104 shutdown 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.396+0000 7f9316453740 1 -- 172.21.15.5:0/4164256851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 msgr2=0x560c7c2d79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.396+0000 7f9316453740 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2d79e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f9308050040 tx=0x7f9308004dd0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.396+0000 7f9316453740 1 -- 172.21.15.5:0/4164256851 shutdown_connections 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.396+0000 7f9316453740 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560c7c349df0 0x560c7c34c1e0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.396+0000 7f92f57fa640 0 client.19104 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.396+0000 7f9316453740 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c2cf680 0x560c7c2d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.396+0000 7f9316453740 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7c2c4dc0 0x560c7c2d7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.396+0000 7f9316453740 1 --2- 172.21.15.5:0/4164256851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c2c44a0 0x560c7c2d79e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.396+0000 7f9316453740 1 -- 172.21.15.5:0/4164256851 >> 172.21.15.5:0/4164256851 conn(0x560c7c2c0a30 msgr2=0x560c7c2cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:25.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.396+0000 7f9316453740 1 -- 172.21.15.5:0/4164256851 shutdown_connections 2024-09-19T06:17:25.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.396+0000 7f9316453740 1 -- 172.21.15.5:0/4164256851 wait complete. 2024-09-19T06:17:25.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.396+0000 7f9316453740 20 client.19104 trim_cache size 0 max 16384 2024-09-19T06:17:25.406 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:17:25.406 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:17:25.406 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:17:25.407 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:17:25.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.397+0000 7f931407a640 1 -- 172.21.15.5:0/917871548 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92f80a3480 msgr2=0x7f92f80a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:25.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.397+0000 7f931407a640 1 --2- 172.21.15.5:0/917871548 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92f80a3480 0x7f92f80a5940 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f92fc014460 tx=0x7f92fc00b2f0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.397+0000 7f931407a640 1 -- 172.21.15.5:0/917871548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f930c112a30 msgr2=0x7f930c06d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:25.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.397+0000 7f931407a640 1 --2- 172.21.15.5:0/917871548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f930c112a30 0x7f930c06d020 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f930400eb10 tx=0x7f930400c2d0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.397+0000 7f931407a640 1 -- 172.21.15.5:0/917871548 shutdown_connections 2024-09-19T06:17:25.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.397+0000 7f931407a640 1 --2- 172.21.15.5:0/917871548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f930c112a30 0x7f930c06d020 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.397+0000 7f931407a640 1 --2- 172.21.15.5:0/917871548 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f92f80a3480 0x7f92f80a5940 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.397+0000 7f931407a640 1 --2- 172.21.15.5:0/917871548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f930c077040 0x7f930c06cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.397+0000 7f931407a640 1 --2- 172.21.15.5:0/917871548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c076720 0x7f930c073a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:25.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.397+0000 7f931407a640 1 -- 172.21.15.5:0/917871548 >> 172.21.15.5:0/917871548 conn(0x7f930c0ffa90 msgr2=0x7f930c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:25.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.398+0000 7f931407a640 1 -- 172.21.15.5:0/917871548 shutdown_connections 2024-09-19T06:17:25.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:25.398+0000 7f931407a640 1 -- 172.21.15.5:0/917871548 wait complete. 2024-09-19T06:17:25.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8368 12.74 MB/sec warmup 49 sec latency 72.457 ms 2024-09-19T06:17:26.285 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:17:26.285 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:17:26.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8522 12.52 MB/sec warmup 50 sec latency 110.266 ms 2024-09-19T06:17:26.682 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:17:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:26 smithi157 ceph-mon[35346]: pgmap v442: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 228 op/s 2024-09-19T06:17:26.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:26 smithi005 ceph-mon[30955]: pgmap v442: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 228 op/s 2024-09-19T06:17:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:26 smithi049 ceph-mon[34640]: pgmap v442: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 228 op/s 2024-09-19T06:17:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.239+0000 7f477f743640 1 -- 172.21.15.5:0/701997913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4778102570 msgr2=0x7f47781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.239+0000 7f477f743640 1 --2- 172.21.15.5:0/701997913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4778102570 0x7f47781029b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4768009890 tx=0x7f476802a410 comp rx=0 tx=0).stop 2024-09-19T06:17:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.240+0000 7f477f743640 1 -- 172.21.15.5:0/701997913 shutdown_connections 2024-09-19T06:17:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.240+0000 7f477f743640 1 --2- 172.21.15.5:0/701997913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778102ef0 0x7f477810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.240+0000 7f477f743640 1 --2- 172.21.15.5:0/701997913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4778102570 0x7f47781029b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.240+0000 7f477f743640 1 --2- 172.21.15.5:0/701997913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4778100e20 0x7f4778101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.240+0000 7f477f743640 1 -- 172.21.15.5:0/701997913 >> 172.21.15.5:0/701997913 conn(0x7f47780fc7d0 msgr2=0x7f47780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:27.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.240+0000 7f477f743640 1 -- 172.21.15.5:0/701997913 shutdown_connections 2024-09-19T06:17:27.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.240+0000 7f477f743640 1 -- 172.21.15.5:0/701997913 wait complete. 2024-09-19T06:17:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.241+0000 7f477f743640 1 Processor -- start 2024-09-19T06:17:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.241+0000 7f477f743640 1 -- start start 2024-09-19T06:17:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.242+0000 7f477f743640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4778100e20 0x7f477810e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.242+0000 7f477f743640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778102570 0x7f477810e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:27.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.242+0000 7f477f743640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4778102ef0 0x7f477810edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:27.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.242+0000 7f477f743640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4778111f40 con 0x7f4778102ef0 2024-09-19T06:17:27.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.242+0000 7f477f743640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4778111dc0 con 0x7f4778102570 2024-09-19T06:17:27.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.242+0000 7f477f743640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47781120c0 con 0x7f4778100e20 2024-09-19T06:17:27.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.242+0000 7f477ccde640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778102570 0x7f477810e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:27.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.242+0000 7f477ccde640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778102570 0x7f477810e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35718/0 (socket says 172.21.15.5:35718) 2024-09-19T06:17:27.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.242+0000 7f477dce0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4778102ef0 0x7f477810edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:27.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.242+0000 7f477dce0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4778102ef0 0x7f477810edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55802/0 (socket says 172.21.15.5:55802) 2024-09-19T06:17:27.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.242+0000 7f477dce0640 1 -- 172.21.15.5:0/3552220120 learned_addr learned my addr 172.21.15.5:0/3552220120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:27.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.243+0000 7f477d4df640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4778100e20 0x7f477810e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.243+0000 7f477dce0640 1 -- 172.21.15.5:0/3552220120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4778100e20 msgr2=0x7f477810e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.243+0000 7f477dce0640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4778100e20 0x7f477810e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.243+0000 7f477dce0640 1 -- 172.21.15.5:0/3552220120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778102570 msgr2=0x7f477810e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.243+0000 7f477dce0640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778102570 0x7f477810e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.243+0000 7f477dce0640 1 -- 172.21.15.5:0/3552220120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47781a68e0 con 0x7f4778102ef0 2024-09-19T06:17:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.243+0000 7f477dce0640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4778102ef0 0x7f477810edd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f477000eb10 tx=0x7f477000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.244+0000 7f476e7fc640 1 -- 172.21.15.5:0/3552220120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770004b20 con 0x7f4778102ef0 2024-09-19T06:17:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.244+0000 7f476e7fc640 1 -- 172.21.15.5:0/3552220120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4770004cc0 con 0x7f4778102ef0 2024-09-19T06:17:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.244+0000 7f477f743640 1 -- 172.21.15.5:0/3552220120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47781981e0 con 0x7f4778102ef0 2024-09-19T06:17:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.244+0000 7f476e7fc640 1 -- 172.21.15.5:0/3552220120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770010720 con 0x7f4778102ef0 2024-09-19T06:17:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.244+0000 7f477f743640 1 -- 172.21.15.5:0/3552220120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4778198720 con 0x7f4778102ef0 2024-09-19T06:17:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.246+0000 7f477d4df640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4778100e20 0x7f477810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:17:27.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.246+0000 7f477f743640 1 -- 172.21.15.5:0/3552220120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4778101200 con 0x7f4778102ef0 2024-09-19T06:17:27.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.247+0000 7f476e7fc640 1 -- 172.21.15.5:0/3552220120 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4770005020 con 0x7f4778102ef0 2024-09-19T06:17:27.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.248+0000 7f476e7fc640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f47640abd10 0x7f47640ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:27.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.249+0000 7f476e7fc640 1 -- 172.21.15.5:0/3552220120 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f47700d0250 con 0x7f4778102ef0 2024-09-19T06:17:27.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.249+0000 7f477d4df640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f47640abd10 0x7f47640ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:27.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.250+0000 7f477d4df640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f47640abd10 0x7f47640ae1d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4774005fd0 tx=0x7f4774009260 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:27.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.256+0000 7f476e7fc640 1 -- 172.21.15.5:0/3552220120 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f477009cc10 con 0x7f4778102ef0 2024-09-19T06:17:27.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8668 12.34 MB/sec warmup 51 sec latency 115.544 ms 2024-09-19T06:17:27.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.613+0000 7f477f743640 1 -- 172.21.15.5:0/3552220120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4778109090 con 0x7f4778102ef0 2024-09-19T06:17:27.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.613+0000 7f476e7fc640 1 -- 172.21.15.5:0/3552220120 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11973 (secure 0 0 0) 0x7f477009c510 con 0x7f4778102ef0 2024-09-19T06:17:27.615 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:17:27.615 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":43,"btime":"2024-09-19T06:17:18:514546+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:17:17.959496+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:17:27.616 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 43 2024-09-19T06:17:27.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.616+0000 7f477f743640 1 -- 172.21.15.5:0/3552220120 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f47640abd10 msgr2=0x7f47640ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:27.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.616+0000 7f477f743640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f47640abd10 0x7f47640ae1d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4774005fd0 tx=0x7f4774009260 comp rx=0 tx=0).stop 2024-09-19T06:17:27.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.616+0000 7f477f743640 1 -- 172.21.15.5:0/3552220120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4778102ef0 msgr2=0x7f477810edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:27.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.616+0000 7f477f743640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4778102ef0 0x7f477810edd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f477000eb10 tx=0x7f477000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:17:27.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.616+0000 7f477f743640 1 -- 172.21.15.5:0/3552220120 shutdown_connections 2024-09-19T06:17:27.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.616+0000 7f477f743640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4778102ef0 0x7f477810edd0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:27.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.616+0000 7f477f743640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778102570 0x7f477810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.616+0000 7f477f743640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f47640abd10 0x7f47640ae1d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.616+0000 7f477f743640 1 --2- 172.21.15.5:0/3552220120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4778100e20 0x7f477810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.616+0000 7f477f743640 1 -- 172.21.15.5:0/3552220120 >> 172.21.15.5:0/3552220120 conn(0x7f47780fc7d0 msgr2=0x7f4778104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.616+0000 7f477f743640 1 -- 172.21.15.5:0/3552220120 shutdown_connections 2024-09-19T06:17:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:27.616+0000 7f477f743640 1 -- 172.21.15.5:0/3552220120 wait complete. 2024-09-19T06:17:28.375 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:17:28.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8843 12.25 MB/sec warmup 52 sec latency 63.912 ms 2024-09-19T06:17:28.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:28 smithi005 ceph-mon[30955]: pgmap v443: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 349 op/s 2024-09-19T06:17:28.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:28 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3552220120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:17:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:17:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:28 smithi157 ceph-mon[35346]: pgmap v443: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 349 op/s 2024-09-19T06:17:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:28 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3552220120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:17:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:28 smithi049 ceph-mon[34640]: pgmap v443: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 349 op/s 2024-09-19T06:17:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:28 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3552220120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:17:29.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.362+0000 7fec23eac640 1 -- 172.21.15.5:0/293857878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c103ff0 msgr2=0x7fec1c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:29.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.362+0000 7fec23eac640 1 --2- 172.21.15.5:0/293857878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c103ff0 0x7fec1c1043d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fec18009860 tx=0x7fec1802a340 comp rx=0 tx=0).stop 2024-09-19T06:17:29.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.362+0000 7fec23eac640 1 -- 172.21.15.5:0/293857878 shutdown_connections 2024-09-19T06:17:29.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.362+0000 7fec23eac640 1 --2- 172.21.15.5:0/293857878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c109120 0x7fec1c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.362+0000 7fec23eac640 1 --2- 172.21.15.5:0/293857878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec1c1049a0 0x7fec1c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.362+0000 7fec23eac640 1 --2- 172.21.15.5:0/293857878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c103ff0 0x7fec1c1043d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.362+0000 7fec23eac640 1 -- 172.21.15.5:0/293857878 >> 172.21.15.5:0/293857878 conn(0x7fec1c0ffab0 msgr2=0x7fec1c101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:29.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.362+0000 7fec23eac640 1 -- 172.21.15.5:0/293857878 shutdown_connections 2024-09-19T06:17:29.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.362+0000 7fec23eac640 1 -- 172.21.15.5:0/293857878 wait complete. 2024-09-19T06:17:29.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec23eac640 1 Processor -- start 2024-09-19T06:17:29.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec23eac640 1 -- start start 2024-09-19T06:17:29.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec23eac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c103ff0 0x7fec1c110bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec23eac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c1049a0 0x7fec1c10dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec23eac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec1c109120 0x7fec1c10e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec23eac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c113fd0 con 0x7fec1c1049a0 2024-09-19T06:17:29.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec23eac640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c113e50 con 0x7fec1c109120 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec23eac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c114150 con 0x7fec1c103ff0 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec21c48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c103ff0 0x7fec1c110bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec22449640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec1c109120 0x7fec1c10e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec22449640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec1c109120 0x7fec1c10e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35726/0 (socket says 172.21.15.5:35726) 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec21c48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c103ff0 0x7fec1c110bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45138/0 (socket says 172.21.15.5:45138) 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec21447640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c1049a0 0x7fec1c10dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.363+0000 7fec22449640 1 -- 172.21.15.5:0/1638249948 learned_addr learned my addr 172.21.15.5:0/1638249948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.364+0000 7fec21c48640 1 -- 172.21.15.5:0/1638249948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec1c109120 msgr2=0x7fec1c10e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.364+0000 7fec21c48640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec1c109120 0x7fec1c10e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.364+0000 7fec21c48640 1 -- 172.21.15.5:0/1638249948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c1049a0 msgr2=0x7fec1c10dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.364+0000 7fec21c48640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c1049a0 0x7fec1c10dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.364+0000 7fec21c48640 1 -- 172.21.15.5:0/1638249948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c1aad10 con 0x7fec1c103ff0 2024-09-19T06:17:29.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.364+0000 7fec22449640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec1c109120 0x7fec1c10e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:29.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.364+0000 7fec21c48640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c103ff0 0x7fec1c110bd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fec18035040 tx=0x7fec18030bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:29.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.364+0000 7fec12ffd640 1 -- 172.21.15.5:0/1638249948 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18040020 con 0x7fec1c103ff0 2024-09-19T06:17:29.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.364+0000 7fec12ffd640 1 -- 172.21.15.5:0/1638249948 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec18038040 con 0x7fec1c103ff0 2024-09-19T06:17:29.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.364+0000 7fec23eac640 1 -- 172.21.15.5:0/1638249948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c1aaee0 con 0x7fec1c103ff0 2024-09-19T06:17:29.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.364+0000 7fec12ffd640 1 -- 172.21.15.5:0/1638249948 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18037070 con 0x7fec1c103ff0 2024-09-19T06:17:29.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.364+0000 7fec23eac640 1 -- 172.21.15.5:0/1638249948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c1ab2d0 con 0x7fec1c103ff0 2024-09-19T06:17:29.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.367+0000 7fec12ffd640 1 -- 172.21.15.5:0/1638249948 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fec18043020 con 0x7fec1c103ff0 2024-09-19T06:17:29.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.368+0000 7fec26285740 1 Processor -- start 2024-09-19T06:17:29.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.368+0000 7fec26285740 1 -- start start 2024-09-19T06:17:29.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.368+0000 7fec26285740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf575b40 0x5630cf574090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.368+0000 7fec26285740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 0x5630cf574a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.368+0000 7fec26285740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf57f370 0x5630cf581760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.368+0000 7fec26285740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630cf4da2f0 con 0x5630cf57f370 2024-09-19T06:17:29.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.368+0000 7fec26285740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5630cf481cf0 con 0x5630cf5745d0 2024-09-19T06:17:29.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.368+0000 7fec26285740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5630cf4d62b0 con 0x5630cf575b40 2024-09-19T06:17:29.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.368+0000 7fec21c48640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 0x5630cf574a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.369+0000 7fec21447640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf57f370 0x5630cf581760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.369+0000 7fec21c48640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 0x5630cf574a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35730/0 (socket says 172.21.15.5:35730) 2024-09-19T06:17:29.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.369+0000 7fec21c48640 1 -- 172.21.15.5:0/1445924989 learned_addr learned my addr 172.21.15.5:0/1445924989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:29.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.369+0000 7fec22449640 1 --2- 172.21.15.5:0/1445924989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf575b40 0x5630cf574090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.369+0000 7fec21c48640 1 -- 172.21.15.5:0/1445924989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf575b40 msgr2=0x5630cf574090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:29.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.369+0000 7fec21c48640 1 --2- 172.21.15.5:0/1445924989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf575b40 0x5630cf574090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.369+0000 7fec12ffd640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fec080a33b0 0x7fec080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.369+0000 7fec21c48640 1 -- 172.21.15.5:0/1445924989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf57f370 msgr2=0x5630cf581760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:29.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.369+0000 7fec21c48640 1 --2- 172.21.15.5:0/1445924989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf57f370 0x5630cf581760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.369+0000 7fec21c48640 1 -- 172.21.15.5:0/1445924989 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630cf4663a0 con 0x5630cf5745d0 2024-09-19T06:17:29.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.369+0000 7fec22449640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fec080a33b0 0x7fec080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.370+0000 7fec12ffd640 1 -- 172.21.15.5:0/1638249948 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7fec180ef950 con 0x7fec1c103ff0 2024-09-19T06:17:29.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.370+0000 7fec21c48640 1 --2- 172.21.15.5:0/1445924989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 0x5630cf574a10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fec180ed100 tx=0x7fec180b8040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc5a020c8114d219 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:29.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.370+0000 7fec22449640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fec080a33b0 0x7fec080a5870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec140049d0 tx=0x7fec14009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:29.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.370+0000 7fec07fff640 1 -- 172.21.15.5:0/1445924989 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec180bbaf0 con 0x5630cf5745d0 2024-09-19T06:17:29.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.370+0000 7fec07fff640 1 -- 172.21.15.5:0/1445924989 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec180bc420 con 0x5630cf5745d0 2024-09-19T06:17:29.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.370+0000 7fec07fff640 1 -- 172.21.15.5:0/1445924989 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec180bb680 con 0x5630cf5745d0 2024-09-19T06:17:29.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.371+0000 7fec26285740 1 -- 172.21.15.5:0/1445924989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 msgr2=0x5630cf574a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:29.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.371+0000 7fec26285740 1 --2- 172.21.15.5:0/1445924989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 0x5630cf574a10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fec180ed100 tx=0x7fec180b8040 comp rx=0 tx=0).stop 2024-09-19T06:17:29.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.371+0000 7fec26285740 1 -- 172.21.15.5:0/1445924989 shutdown_connections 2024-09-19T06:17:29.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.371+0000 7fec26285740 1 --2- 172.21.15.5:0/1445924989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf57f370 0x5630cf581760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.371+0000 7fec26285740 1 --2- 172.21.15.5:0/1445924989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 0x5630cf574a10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.371+0000 7fec26285740 1 --2- 172.21.15.5:0/1445924989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf575b40 0x5630cf574090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.371+0000 7fec26285740 1 -- 172.21.15.5:0/1445924989 >> 172.21.15.5:0/1445924989 conn(0x5630cf53d310 msgr2=0x5630cf570c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:29.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.371+0000 7fec26285740 1 -- 172.21.15.5:0/1445924989 shutdown_connections 2024-09-19T06:17:29.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.371+0000 7fec26285740 1 -- 172.21.15.5:0/1445924989 wait complete. 2024-09-19T06:17:29.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.371+0000 7fec26285740 1 -- start start 2024-09-19T06:17:29.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.371+0000 7fec26285740 1 Processor -- start 2024-09-19T06:17:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.372+0000 7fec26285740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf5745d0 0x5630cf57edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.372+0000 7fec26285740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf575b40 0x5630cf57c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.372+0000 7fec26285740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf57f370 0x5630cf57c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.372+0000 7fec26285740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630cf481cf0 con 0x5630cf575b40 2024-09-19T06:17:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.372+0000 7fec26285740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5630cf495060 con 0x5630cf57f370 2024-09-19T06:17:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.372+0000 7fec26285740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5630cf581e30 con 0x5630cf5745d0 2024-09-19T06:17:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.372+0000 7fec26285740 10 client.0 _ll_register_callbacks cb 0x5630cf1e88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.372+0000 7fec21c48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf5745d0 0x5630cf57edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.372+0000 7fec21c48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf5745d0 0x5630cf57edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45154/0 (socket says 172.21.15.5:45154) 2024-09-19T06:17:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.372+0000 7fec21c48640 1 -- 172.21.15.5:0/3017575883 learned_addr learned my addr 172.21.15.5:0/3017575883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.372+0000 7fec21447640 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf575b40 0x5630cf57c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.372+0000 7fec22449640 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf57f370 0x5630cf57c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec21c48640 1 -- 172.21.15.5:0/3017575883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf57f370 msgr2=0x5630cf57c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec21c48640 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf57f370 0x5630cf57c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec21c48640 1 -- 172.21.15.5:0/3017575883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf575b40 msgr2=0x5630cf57c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec21c48640 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf575b40 0x5630cf57c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec21c48640 1 -- 172.21.15.5:0/3017575883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630cf24b3c0 con 0x5630cf5745d0 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec22449640 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf57f370 0x5630cf57c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec21c48640 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf5745d0 0x5630cf57edf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fec1802a860 tx=0x7fec1802a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7c22609979938a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec057fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec057fa640 1 -- 172.21.15.5:0/3017575883 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec180bb860 con 0x5630cf5745d0 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec057fa640 1 -- 172.21.15.5:0/3017575883 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7fec180b8390 con 0x5630cf5745d0 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec26285740 1 -- 172.21.15.5:0/3017575883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec180bc420 con 0x5630cf5745d0 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec057fa640 1 -- 172.21.15.5:0/3017575883 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fec180bb680 con 0x5630cf5745d0 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec26285740 10 client.28468 fetch_fsmap learned FSMap version 43 2024-09-19T06:17:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.373+0000 7fec26285740 1 -- 172.21.15.5:0/3017575883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5630cf58d7a0 con 0x5630cf5745d0 2024-09-19T06:17:29.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.374+0000 7fec057fa640 1 -- 172.21.15.5:0/3017575883 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec180bc420 con 0x5630cf5745d0 2024-09-19T06:17:29.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.374+0000 7fec057fa640 1 -- 172.21.15.5:0/3017575883 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7fec180b7a70 con 0x5630cf5745d0 2024-09-19T06:17:29.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.374+0000 7fec26285740 10 client.28468 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:17:29.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.374+0000 7fec26285740 10 client.28468 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:17:29.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.374+0000 7fec26285740 20 client.28468 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:29.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.374+0000 7fec26285740 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5630cf58eaf0 0x5630cf590ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.374+0000 7fec26285740 4 client.28468 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:17:29.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.374+0000 7fec26285740 1 -- 172.21.15.5:0/3017575883 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630cf4e7860 con 0x5630cf58eaf0 2024-09-19T06:17:29.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.374+0000 7fec21447640 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5630cf58eaf0 0x5630cf590ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.375+0000 7fec057fa640 10 client.28468 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:17:29.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.375+0000 7fec21447640 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5630cf58eaf0 0x5630cf590ee0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b59b99fa1ec837bd server_cookie=b47648ccb174b88f in_seq=0 out_seq=0 2024-09-19T06:17:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.376+0000 7fec057fa640 1 -- 172.21.15.5:0/3017575883 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630cf4e7860 con 0x5630cf58eaf0 2024-09-19T06:17:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.376+0000 7fec057fa640 10 client.28468 handle_command_reply: tid=0 2024-09-19T06:17:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.376+0000 7fec26285740 1 client.28468 shutdown 2024-09-19T06:17:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.376+0000 7fec26285740 1 -- 172.21.15.5:0/3017575883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf5745d0 msgr2=0x5630cf57edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.376+0000 7fec26285740 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf5745d0 0x5630cf57edf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fec1802a860 tx=0x7fec1802a340 comp rx=0 tx=0).stop 2024-09-19T06:17:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.376+0000 7fec26285740 1 -- 172.21.15.5:0/3017575883 shutdown_connections 2024-09-19T06:17:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.376+0000 7fec26285740 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5630cf58eaf0 0x5630cf590ee0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.377+0000 7fec057fa640 0 client.28468 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:17:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.377+0000 7fec26285740 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf57f370 0x5630cf57c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.377+0000 7fec26285740 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf575b40 0x5630cf57c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.377+0000 7fec26285740 1 --2- 172.21.15.5:0/3017575883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf5745d0 0x5630cf57edf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.377+0000 7fec26285740 1 -- 172.21.15.5:0/3017575883 >> 172.21.15.5:0/3017575883 conn(0x5630cf53d310 msgr2=0x5630cf5769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:29.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.377+0000 7fec26285740 1 -- 172.21.15.5:0/3017575883 shutdown_connections 2024-09-19T06:17:29.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.377+0000 7fec26285740 1 -- 172.21.15.5:0/3017575883 wait complete. 2024-09-19T06:17:29.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.377+0000 7fec26285740 20 client.28468 trim_cache size 0 max 16384 2024-09-19T06:17:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.397+0000 7fec26285740 1 Processor -- start 2024-09-19T06:17:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.397+0000 7fec26285740 1 -- start start 2024-09-19T06:17:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.397+0000 7fec26285740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 0x5630cf57b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.397+0000 7fec26285740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf575b40 0x5630cf56fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.397+0000 7fec26285740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf57f370 0x5630cf596670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.397+0000 7fec26285740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c1b3fd0 con 0x5630cf57f370 2024-09-19T06:17:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.397+0000 7fec26285740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c1b5c60 con 0x5630cf5745d0 2024-09-19T06:17:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.397+0000 7fec26285740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c1b46d0 con 0x5630cf575b40 2024-09-19T06:17:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec21c48640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 0x5630cf57b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec21c48640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 0x5630cf57b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35754/0 (socket says 172.21.15.5:35754) 2024-09-19T06:17:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec21c48640 1 -- 172.21.15.5:0/4028654869 learned_addr learned my addr 172.21.15.5:0/4028654869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec22449640 1 --2- 172.21.15.5:0/4028654869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf57f370 0x5630cf596670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec21447640 1 --2- 172.21.15.5:0/4028654869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf575b40 0x5630cf56fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec21c48640 1 -- 172.21.15.5:0/4028654869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf575b40 msgr2=0x5630cf56fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec21c48640 1 --2- 172.21.15.5:0/4028654869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf575b40 0x5630cf56fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec21c48640 1 -- 172.21.15.5:0/4028654869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf57f370 msgr2=0x5630cf596670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec21c48640 1 --2- 172.21.15.5:0/4028654869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf57f370 0x5630cf596670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec21c48640 1 -- 172.21.15.5:0/4028654869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630cf57a640 con 0x5630cf5745d0 2024-09-19T06:17:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec21447640 1 --2- 172.21.15.5:0/4028654869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf575b40 0x5630cf56fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec21c48640 1 --2- 172.21.15.5:0/4028654869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 0x5630cf57b8a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fec1802a930 tx=0x7fec180ba230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a3f2f8e8bde4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec077fe640 1 -- 172.21.15.5:0/4028654869 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec180bbaf0 con 0x5630cf5745d0 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec077fe640 1 -- 172.21.15.5:0/4028654869 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec180bc1e0 con 0x5630cf5745d0 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.398+0000 7fec077fe640 1 -- 172.21.15.5:0/4028654869 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec180bb860 con 0x5630cf5745d0 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.399+0000 7fec26285740 1 -- 172.21.15.5:0/4028654869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 msgr2=0x5630cf57b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.399+0000 7fec26285740 1 --2- 172.21.15.5:0/4028654869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 0x5630cf57b8a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fec1802a930 tx=0x7fec180ba230 comp rx=0 tx=0).stop 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.400+0000 7fec26285740 1 -- 172.21.15.5:0/4028654869 shutdown_connections 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.400+0000 7fec26285740 1 --2- 172.21.15.5:0/4028654869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf57f370 0x5630cf596670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.400+0000 7fec26285740 1 --2- 172.21.15.5:0/4028654869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf575b40 0x5630cf56fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.400+0000 7fec26285740 1 --2- 172.21.15.5:0/4028654869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf5745d0 0x5630cf57b8a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.400+0000 7fec26285740 1 -- 172.21.15.5:0/4028654869 >> 172.21.15.5:0/4028654869 conn(0x5630cf53d310 msgr2=0x5630cf570c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.400+0000 7fec26285740 1 -- 172.21.15.5:0/4028654869 shutdown_connections 2024-09-19T06:17:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.400+0000 7fec26285740 1 -- 172.21.15.5:0/4028654869 wait complete. 2024-09-19T06:17:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.400+0000 7fec26285740 1 -- start start 2024-09-19T06:17:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.400+0000 7fec26285740 1 Processor -- start 2024-09-19T06:17:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec26285740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf5745d0 0x5630cf587680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec26285740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf575b40 0x5630cf5829b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec26285740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf57f370 0x5630cf582ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec26285740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c1b5590 con 0x5630cf5745d0 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec26285740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c1b4e00 con 0x5630cf575b40 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec26285740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c1b1fd0 con 0x5630cf57f370 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec26285740 10 client.0 _ll_register_callbacks cb 0x7fec0c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec21c48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf5745d0 0x5630cf587680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec21c48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf5745d0 0x5630cf587680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55858/0 (socket says 172.21.15.5:55858) 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec21c48640 1 -- 172.21.15.5:0/555106917 learned_addr learned my addr 172.21.15.5:0/555106917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec21447640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf575b40 0x5630cf5829b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec22449640 1 --2- 172.21.15.5:0/555106917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf57f370 0x5630cf582ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec21447640 1 -- 172.21.15.5:0/555106917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf57f370 msgr2=0x5630cf582ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec21447640 1 --2- 172.21.15.5:0/555106917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf57f370 0x5630cf582ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec21447640 1 -- 172.21.15.5:0/555106917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf5745d0 msgr2=0x5630cf587680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec21447640 1 --2- 172.21.15.5:0/555106917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf5745d0 0x5630cf587680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.401+0000 7fec21447640 1 -- 172.21.15.5:0/555106917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630cf5835e0 con 0x5630cf575b40 2024-09-19T06:17:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.402+0000 7fec21447640 1 --2- 172.21.15.5:0/555106917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf575b40 0x5630cf5829b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fec1c10f440 tx=0x7fec0c010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e6a5dd258c28942 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.402+0000 7fec057fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:17:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.402+0000 7fec057fa640 1 -- 172.21.15.5:0/555106917 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c005220 con 0x5630cf575b40 2024-09-19T06:17:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.402+0000 7fec057fa640 1 -- 172.21.15.5:0/555106917 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7fec0c0053c0 con 0x5630cf575b40 2024-09-19T06:17:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.402+0000 7fec26285740 1 -- 172.21.15.5:0/555106917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec180bc1e0 con 0x5630cf575b40 2024-09-19T06:17:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.402+0000 7fec057fa640 1 -- 172.21.15.5:0/555106917 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fec0c005600 con 0x5630cf575b40 2024-09-19T06:17:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.402+0000 7fec26285740 10 client.28526 fetch_fsmap learned FSMap version 43 2024-09-19T06:17:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.402+0000 7fec26285740 1 -- 172.21.15.5:0/555106917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5630cf5f9540 con 0x5630cf575b40 2024-09-19T06:17:29.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.403+0000 7fec057fa640 1 -- 172.21.15.5:0/555106917 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec180bc1e0 con 0x5630cf575b40 2024-09-19T06:17:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.403+0000 7fec057fa640 1 -- 172.21.15.5:0/555106917 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7fec0c00fb10 con 0x5630cf575b40 2024-09-19T06:17:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.403+0000 7fec26285740 10 client.28526 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:17:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.403+0000 7fec26285740 10 client.28526 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:17:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.403+0000 7fec26285740 20 client.28526 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.403+0000 7fec26285740 1 --2- 172.21.15.5:0/555106917 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5630cf58eaf0 0x5630cf5fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.403+0000 7fec26285740 4 client.28526 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:17:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.403+0000 7fec26285740 1 -- 172.21.15.5:0/555106917 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630cf4e1d70 con 0x5630cf58eaf0 2024-09-19T06:17:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.403+0000 7fec21c48640 1 --2- 172.21.15.5:0/555106917 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5630cf58eaf0 0x5630cf5fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.404+0000 7fec21c48640 1 --2- 172.21.15.5:0/555106917 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5630cf58eaf0 0x5630cf5fbcb0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bf20ea65177bd89 server_cookie=46444fcfc4978f27 in_seq=0 out_seq=0 2024-09-19T06:17:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.404+0000 7fec057fa640 10 client.28526 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:17:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.404+0000 7fec057fa640 1 -- 172.21.15.5:0/555106917 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630cf4e1d70 con 0x5630cf58eaf0 2024-09-19T06:17:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.404+0000 7fec057fa640 10 client.28526 handle_command_reply: tid=0 2024-09-19T06:17:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.404+0000 7fec26285740 1 client.28526 shutdown 2024-09-19T06:17:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.404+0000 7fec26285740 1 -- 172.21.15.5:0/555106917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf575b40 msgr2=0x5630cf5829b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.404+0000 7fec26285740 1 --2- 172.21.15.5:0/555106917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf575b40 0x5630cf5829b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fec1c10f440 tx=0x7fec0c010ab0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.404+0000 7fec26285740 1 -- 172.21.15.5:0/555106917 shutdown_connections 2024-09-19T06:17:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.404+0000 7fec26285740 1 --2- 172.21.15.5:0/555106917 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x5630cf58eaf0 0x5630cf5fbcb0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec26285740 1 --2- 172.21.15.5:0/555106917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5630cf57f370 0x5630cf582ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec26285740 1 --2- 172.21.15.5:0/555106917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630cf575b40 0x5630cf5829b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec26285740 1 --2- 172.21.15.5:0/555106917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cf5745d0 0x5630cf587680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec26285740 1 -- 172.21.15.5:0/555106917 >> 172.21.15.5:0/555106917 conn(0x5630cf53d310 msgr2=0x5630cf56fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec057fa640 0 client.28526 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:17:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec26285740 1 -- 172.21.15.5:0/555106917 shutdown_connections 2024-09-19T06:17:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec26285740 1 -- 172.21.15.5:0/555106917 wait complete. 2024-09-19T06:17:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec26285740 20 client.28526 trim_cache size 0 max 16384 2024-09-19T06:17:29.410 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:17:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec23eac640 1 -- 172.21.15.5:0/1638249948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fec080a33b0 msgr2=0x7fec080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec23eac640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fec080a33b0 0x7fec080a5870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec140049d0 tx=0x7fec14009500 comp rx=0 tx=0).stop 2024-09-19T06:17:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec23eac640 1 -- 172.21.15.5:0/1638249948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c103ff0 msgr2=0x7fec1c110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec23eac640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c103ff0 0x7fec1c110bd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fec18035040 tx=0x7fec18030bd0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.405+0000 7fec23eac640 1 -- 172.21.15.5:0/1638249948 shutdown_connections 2024-09-19T06:17:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.406+0000 7fec23eac640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec1c109120 0x7fec1c10e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.406+0000 7fec23eac640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c1049a0 0x7fec1c10dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.406+0000 7fec23eac640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fec080a33b0 0x7fec080a5870 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.406+0000 7fec23eac640 1 --2- 172.21.15.5:0/1638249948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c103ff0 0x7fec1c110bd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.406+0000 7fec23eac640 1 -- 172.21.15.5:0/1638249948 >> 172.21.15.5:0/1638249948 conn(0x7fec1c0ffab0 msgr2=0x7fec1c100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.406+0000 7fec23eac640 1 -- 172.21.15.5:0/1638249948 shutdown_connections 2024-09-19T06:17:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:29.406+0000 7fec23eac640 1 -- 172.21.15.5:0/1638249948 wait complete. 2024-09-19T06:17:29.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8997 12.15 MB/sec warmup 53 sec latency 89.128 ms 2024-09-19T06:17:30.299 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:17:30.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 9202 12.28 MB/sec warmup 54 sec latency 128.603 ms 2024-09-19T06:17:30.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:30 smithi005 ceph-mon[30955]: pgmap v444: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-19T06:17:30.692 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:17:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:30 smithi157 ceph-mon[35346]: pgmap v444: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-19T06:17:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:30 smithi049 ceph-mon[34640]: pgmap v444: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-19T06:17:31.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.271+0000 7fea4a021640 1 -- 172.21.15.5:0/2450554501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44103780 msgr2=0x7fea44107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.271+0000 7fea4a021640 1 --2- 172.21.15.5:0/2450554501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44103780 0x7fea44107bc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fea38009930 tx=0x7fea3802a420 comp rx=0 tx=0).stop 2024-09-19T06:17:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.271+0000 7fea4a021640 1 -- 172.21.15.5:0/2450554501 shutdown_connections 2024-09-19T06:17:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.271+0000 7fea4a021640 1 --2- 172.21.15.5:0/2450554501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea44108300 0x7fea4410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.271+0000 7fea4a021640 1 --2- 172.21.15.5:0/2450554501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44103780 0x7fea44107bc0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.271+0000 7fea4a021640 1 --2- 172.21.15.5:0/2450554501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea44102dd0 0x7fea441031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.271+0000 7fea4a021640 1 -- 172.21.15.5:0/2450554501 >> 172.21.15.5:0/2450554501 conn(0x7fea440fc7d0 msgr2=0x7fea440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.272+0000 7fea4a021640 1 -- 172.21.15.5:0/2450554501 shutdown_connections 2024-09-19T06:17:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.272+0000 7fea4a021640 1 -- 172.21.15.5:0/2450554501 wait complete. 2024-09-19T06:17:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.272+0000 7fea4a021640 1 Processor -- start 2024-09-19T06:17:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.273+0000 7fea4a021640 1 -- start start 2024-09-19T06:17:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.273+0000 7fea4a021640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea44102dd0 0x7fea441a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea4a021640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44103780 0x7fea441a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea4a021640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea44108300 0x7fea4410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea4a021640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea44111f50 con 0x7fea44103780 2024-09-19T06:17:31.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea4a021640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fea44111dd0 con 0x7fea44102dd0 2024-09-19T06:17:31.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea4a021640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea441120d0 con 0x7fea44108300 2024-09-19T06:17:31.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea44102dd0 0x7fea441a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea44102dd0 0x7fea441a48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35792/0 (socket says 172.21.15.5:35792) 2024-09-19T06:17:31.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea437fe640 1 -- 172.21.15.5:0/4188697015 learned_addr learned my addr 172.21.15.5:0/4188697015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:31.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea42ffd640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44103780 0x7fea441a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea437fe640 1 -- 172.21.15.5:0/4188697015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea44108300 msgr2=0x7fea4410ded0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:17:31.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea437fe640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea44108300 0x7fea4410ded0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea437fe640 1 -- 172.21.15.5:0/4188697015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44103780 msgr2=0x7fea441a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea437fe640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44103780 0x7fea441a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.274+0000 7fea437fe640 1 -- 172.21.15.5:0/4188697015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4410e760 con 0x7fea44102dd0 2024-09-19T06:17:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.275+0000 7fea437fe640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea44102dd0 0x7fea441a48a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fea3400d950 tx=0x7fea3400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.275+0000 7fea40ff9640 1 -- 172.21.15.5:0/4188697015 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34017030 con 0x7fea44102dd0 2024-09-19T06:17:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.275+0000 7fea40ff9640 1 -- 172.21.15.5:0/4188697015 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea34004020 con 0x7fea44102dd0 2024-09-19T06:17:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.275+0000 7fea4a021640 1 -- 172.21.15.5:0/4188697015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4410ea50 con 0x7fea44102dd0 2024-09-19T06:17:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.275+0000 7fea40ff9640 1 -- 172.21.15.5:0/4188697015 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea340047a0 con 0x7fea44102dd0 2024-09-19T06:17:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.275+0000 7fea4a021640 1 -- 172.21.15.5:0/4188697015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea441ab7d0 con 0x7fea44102dd0 2024-09-19T06:17:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.278+0000 7fea40ff9640 1 -- 172.21.15.5:0/4188697015 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fea340041d0 con 0x7fea44102dd0 2024-09-19T06:17:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.279+0000 7fea4c3fa740 1 Processor -- start 2024-09-19T06:17:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.279+0000 7fea4c3fa740 1 -- start start 2024-09-19T06:17:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.279+0000 7fea4c3fa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a76090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.279+0000 7fea4c3fa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a765d0 0x55fb53a76a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.279+0000 7fea4c3fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a81370 0x55fb53a83760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.279+0000 7fea4c3fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fb539dc2f0 con 0x55fb53a81370 2024-09-19T06:17:31.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.279+0000 7fea4c3fa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fb53983cf0 con 0x55fb53a77b40 2024-09-19T06:17:31.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.279+0000 7fea4c3fa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fb539d82b0 con 0x55fb53a765d0 2024-09-19T06:17:31.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.279+0000 7fea43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a76090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.279+0000 7fea437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a765d0 0x55fb53a76a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.279+0000 7fea43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a76090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35796/0 (socket says 172.21.15.5:35796) 2024-09-19T06:17:31.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.279+0000 7fea43fff640 1 -- 172.21.15.5:0/3749851146 learned_addr learned my addr 172.21.15.5:0/3749851146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:31.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.280+0000 7fea42ffd640 1 --2- 172.21.15.5:0/3749851146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a81370 0x55fb53a83760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.280+0000 7fea40ff9640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fea2c0a34b0 0x7fea2c0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.280+0000 7fea43fff640 1 -- 172.21.15.5:0/3749851146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a765d0 msgr2=0x55fb53a76a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:31.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.280+0000 7fea43fff640 1 --2- 172.21.15.5:0/3749851146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a765d0 0x55fb53a76a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.280+0000 7fea43fff640 1 -- 172.21.15.5:0/3749851146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a81370 msgr2=0x55fb53a83760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:31.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.280+0000 7fea43fff640 1 --2- 172.21.15.5:0/3749851146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a81370 0x55fb53a83760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.280+0000 7fea43fff640 1 -- 172.21.15.5:0/3749851146 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb539683a0 con 0x55fb53a77b40 2024-09-19T06:17:31.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.280+0000 7fea43fff640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fea2c0a34b0 0x7fea2c0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.280+0000 7fea40ff9640 1 -- 172.21.15.5:0/4188697015 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7fea340cf840 con 0x7fea44102dd0 2024-09-19T06:17:31.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.281+0000 7fea43fff640 1 --2- 172.21.15.5:0/3749851146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a76090 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009ed0 tx=0x7fea3c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd2e4e36e16c96bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:31.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.281+0000 7fea29ffb640 1 -- 172.21.15.5:0/3749851146 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c021070 con 0x55fb53a77b40 2024-09-19T06:17:31.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.281+0000 7fea29ffb640 1 -- 172.21.15.5:0/3749851146 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea3c0045d0 con 0x55fb53a77b40 2024-09-19T06:17:31.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.281+0000 7fea29ffb640 1 -- 172.21.15.5:0/3749851146 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c01b440 con 0x55fb53a77b40 2024-09-19T06:17:31.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.282+0000 7fea4c3fa740 1 -- 172.21.15.5:0/3749851146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 msgr2=0x55fb53a76090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:31.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.282+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/3749851146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a76090 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009ed0 tx=0x7fea3c00e720 comp rx=0 tx=0).stop 2024-09-19T06:17:31.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.282+0000 7fea43fff640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fea2c0a34b0 0x7fea2c0a5970 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00efd0 tx=0x7fea3c01f000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:31.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.282+0000 7fea4c3fa740 1 -- 172.21.15.5:0/3749851146 shutdown_connections 2024-09-19T06:17:31.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.282+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/3749851146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a81370 0x55fb53a83760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.282+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/3749851146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a765d0 0x55fb53a76a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.282+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/3749851146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a76090 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.282+0000 7fea4c3fa740 1 -- 172.21.15.5:0/3749851146 >> 172.21.15.5:0/3749851146 conn(0x55fb53a3f310 msgr2=0x55fb53a72c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:31.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.282+0000 7fea4c3fa740 1 -- 172.21.15.5:0/3749851146 shutdown_connections 2024-09-19T06:17:31.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.282+0000 7fea4c3fa740 1 -- 172.21.15.5:0/3749851146 wait complete. 2024-09-19T06:17:31.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.282+0000 7fea4c3fa740 1 -- start start 2024-09-19T06:17:31.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea4c3fa740 1 Processor -- start 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea4c3fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a765d0 0x55fb53a80c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea4c3fa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a7df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea4c3fa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a81370 0x55fb53a7e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea4c3fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fb53a840c0 con 0x55fb53a765d0 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea4c3fa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fb53997060 con 0x55fb53a77b40 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea4c3fa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fb53a84240 con 0x55fb53a81370 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea4c3fa740 10 client.0 _ll_register_callbacks cb 0x55fb536ea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a7df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a7df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35806/0 (socket says 172.21.15.5:35806) 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea43fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a81370 0x55fb53a7e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea42ffd640 1 -- 172.21.15.5:0/573505763 learned_addr learned my addr 172.21.15.5:0/573505763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea437fe640 1 --2- 172.21.15.5:0/573505763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a765d0 0x55fb53a80c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.283+0000 7fea42ffd640 1 -- 172.21.15.5:0/573505763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a81370 msgr2=0x55fb53a7e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.284+0000 7fea42ffd640 1 --2- 172.21.15.5:0/573505763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a81370 0x55fb53a7e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.284+0000 7fea42ffd640 1 -- 172.21.15.5:0/573505763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a765d0 msgr2=0x55fb53a80c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.284+0000 7fea42ffd640 1 --2- 172.21.15.5:0/573505763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a765d0 0x55fb53a80c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.284+0000 7fea42ffd640 1 -- 172.21.15.5:0/573505763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5374d3c0 con 0x55fb53a77b40 2024-09-19T06:17:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.284+0000 7fea42ffd640 1 --2- 172.21.15.5:0/573505763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a7df00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fea38005ab0 tx=0x7fea38005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a4c7a375ce2aca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.284+0000 7fea22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:17:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.284+0000 7fea22ffd640 1 -- 172.21.15.5:0/573505763 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38035030 con 0x55fb53a77b40 2024-09-19T06:17:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.284+0000 7fea4c3fa740 1 -- 172.21.15.5:0/573505763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3c0045d0 con 0x55fb53a77b40 2024-09-19T06:17:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.284+0000 7fea22ffd640 1 -- 172.21.15.5:0/573505763 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7fea380048c0 con 0x55fb53a77b40 2024-09-19T06:17:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.285+0000 7fea22ffd640 1 -- 172.21.15.5:0/573505763 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fea38005d50 con 0x55fb53a77b40 2024-09-19T06:17:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.285+0000 7fea4c3fa740 10 client.28538 fetch_fsmap learned FSMap version 43 2024-09-19T06:17:31.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.285+0000 7fea4c3fa740 1 -- 172.21.15.5:0/573505763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fb53a8fac0 con 0x55fb53a77b40 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.285+0000 7fea22ffd640 1 -- 172.21.15.5:0/573505763 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea3c0045d0 con 0x55fb53a77b40 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.285+0000 7fea22ffd640 1 -- 172.21.15.5:0/573505763 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7fea38003bf0 con 0x55fb53a77b40 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.285+0000 7fea4c3fa740 10 client.28538 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.285+0000 7fea4c3fa740 10 client.28538 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.285+0000 7fea4c3fa740 20 client.28538 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.285+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/573505763 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55fb53a90ed0 0x55fb53a93300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.285+0000 7fea4c3fa740 4 client.28538 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.285+0000 7fea4c3fa740 1 -- 172.21.15.5:0/573505763 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb539e9860 con 0x55fb53a90ed0 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.286+0000 7fea437fe640 1 --2- 172.21.15.5:0/573505763 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55fb53a90ed0 0x55fb53a93300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.286+0000 7fea437fe640 1 --2- 172.21.15.5:0/573505763 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55fb53a90ed0 0x55fb53a93300 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cea11a0daae9d4d7 server_cookie=26406b38057d50b9 in_seq=0 out_seq=0 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.286+0000 7fea22ffd640 10 client.28538 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea22ffd640 1 -- 172.21.15.5:0/573505763 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb539e9860 con 0x55fb53a90ed0 2024-09-19T06:17:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea22ffd640 10 client.28538 handle_command_reply: tid=0 2024-09-19T06:17:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea4c3fa740 1 client.28538 shutdown 2024-09-19T06:17:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea4c3fa740 1 -- 172.21.15.5:0/573505763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 msgr2=0x55fb53a7df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/573505763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a7df00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fea38005ab0 tx=0x7fea38005270 comp rx=0 tx=0).stop 2024-09-19T06:17:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea4c3fa740 1 -- 172.21.15.5:0/573505763 shutdown_connections 2024-09-19T06:17:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/573505763 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55fb53a90ed0 0x55fb53a93300 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/573505763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a81370 0x55fb53a7e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/573505763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a7df00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea22ffd640 0 client.28538 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:17:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/573505763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a765d0 0x55fb53a80c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea4c3fa740 1 -- 172.21.15.5:0/573505763 >> 172.21.15.5:0/573505763 conn(0x55fb53a3f310 msgr2=0x55fb53a78960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea4c3fa740 1 -- 172.21.15.5:0/573505763 shutdown_connections 2024-09-19T06:17:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea4c3fa740 1 -- 172.21.15.5:0/573505763 wait complete. 2024-09-19T06:17:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.287+0000 7fea4c3fa740 20 client.28538 trim_cache size 0 max 16384 2024-09-19T06:17:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.308+0000 7fea4c3fa740 1 Processor -- start 2024-09-19T06:17:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.308+0000 7fea4c3fa740 1 -- start start 2024-09-19T06:17:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea4c3fa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a765d0 0x55fb53a8d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea4c3fa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a79120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea4c3fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a81370 0x55fb53a79660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea4c3fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea441b3f40 con 0x55fb53a81370 2024-09-19T06:17:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea4c3fa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fea441b5bd0 con 0x55fb53a77b40 2024-09-19T06:17:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea4c3fa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea441b4640 con 0x55fb53a765d0 2024-09-19T06:17:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a81370 0x55fb53a79660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a79120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a765d0 0x55fb53a8d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a81370 0x55fb53a79660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55898/0 (socket says 172.21.15.5:55898) 2024-09-19T06:17:31.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea43fff640 1 -- 172.21.15.5:0/2740769266 learned_addr learned my addr 172.21.15.5:0/2740769266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:31.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea437fe640 1 -- 172.21.15.5:0/2740769266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 msgr2=0x55fb53a79120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:31.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea437fe640 1 --2- 172.21.15.5:0/2740769266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a79120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea437fe640 1 -- 172.21.15.5:0/2740769266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a81370 msgr2=0x55fb53a79660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:31.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea437fe640 1 --2- 172.21.15.5:0/2740769266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a81370 0x55fb53a79660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.309+0000 7fea437fe640 1 -- 172.21.15.5:0/2740769266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb53a731a0 con 0x55fb53a765d0 2024-09-19T06:17:31.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.310+0000 7fea437fe640 1 --2- 172.21.15.5:0/2740769266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a765d0 0x55fb53a8d360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fea340ccf10 tx=0x7fea340d1c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edbd6ddba7fcd9f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:31.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.310+0000 7fea297fa640 1 -- 172.21.15.5:0/2740769266 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3409fe50 con 0x55fb53a765d0 2024-09-19T06:17:31.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.310+0000 7fea297fa640 1 -- 172.21.15.5:0/2740769266 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea340a4710 con 0x55fb53a765d0 2024-09-19T06:17:31.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.310+0000 7fea297fa640 1 -- 172.21.15.5:0/2740769266 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea340a4280 con 0x55fb53a765d0 2024-09-19T06:17:31.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.311+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2740769266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a765d0 msgr2=0x55fb53a8d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:31.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.311+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/2740769266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a765d0 0x55fb53a8d360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fea340ccf10 tx=0x7fea340d1c10 comp rx=0 tx=0).stop 2024-09-19T06:17:31.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.311+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2740769266 shutdown_connections 2024-09-19T06:17:31.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.311+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/2740769266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a81370 0x55fb53a79660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.311+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/2740769266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a79120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.311+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/2740769266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a765d0 0x55fb53a8d360 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.311+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2740769266 >> 172.21.15.5:0/2740769266 conn(0x55fb53a3f310 msgr2=0x55fb53a7c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:31.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.311+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2740769266 shutdown_connections 2024-09-19T06:17:31.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.311+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2740769266 wait complete. 2024-09-19T06:17:31.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.311+0000 7fea4c3fa740 1 -- start start 2024-09-19T06:17:31.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea4c3fa740 1 Processor -- start 2024-09-19T06:17:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea4c3fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a765d0 0x55fb53a853a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea4c3fa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a858e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea4c3fa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a81370 0x55fb53a85e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea4c3fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea441b5500 con 0x55fb53a765d0 2024-09-19T06:17:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea4c3fa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fea441b4d70 con 0x55fb53a77b40 2024-09-19T06:17:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea4c3fa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea441b1f40 con 0x55fb53a81370 2024-09-19T06:17:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea4c3fa740 10 client.0 _ll_register_callbacks cb 0x7fea38004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a765d0 0x55fb53a853a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a765d0 0x55fb53a853a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55914/0 (socket says 172.21.15.5:55914) 2024-09-19T06:17:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea437fe640 1 -- 172.21.15.5:0/2214826074 learned_addr learned my addr 172.21.15.5:0/2214826074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea43fff640 1 --2- 172.21.15.5:0/2214826074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a81370 0x55fb53a85e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.312+0000 7fea42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a858e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.313+0000 7fea437fe640 1 -- 172.21.15.5:0/2214826074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a81370 msgr2=0x55fb53a85e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.313+0000 7fea437fe640 1 --2- 172.21.15.5:0/2214826074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a81370 0x55fb53a85e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.313+0000 7fea437fe640 1 -- 172.21.15.5:0/2214826074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 msgr2=0x55fb53a858e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.313+0000 7fea437fe640 1 --2- 172.21.15.5:0/2214826074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a858e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.313+0000 7fea437fe640 1 -- 172.21.15.5:0/2214826074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb53a73960 con 0x55fb53a765d0 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.313+0000 7fea437fe640 1 --2- 172.21.15.5:0/2214826074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a765d0 0x55fb53a853a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fea3409b600 tx=0x7fea34097dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee39d058728fb44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.313+0000 7fea22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.313+0000 7fea22ffd640 1 -- 172.21.15.5:0/2214826074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34086070 con 0x55fb53a765d0 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.313+0000 7fea22ffd640 1 -- 172.21.15.5:0/2214826074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7fea340a4280 con 0x55fb53a765d0 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.313+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2214826074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea340a4710 con 0x55fb53a765d0 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.314+0000 7fea22ffd640 1 -- 172.21.15.5:0/2214826074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fea34086210 con 0x55fb53a765d0 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.314+0000 7fea4c3fa740 10 client.19146 fetch_fsmap learned FSMap version 43 2024-09-19T06:17:31.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.314+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2214826074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fb53afbe50 con 0x55fb53a765d0 2024-09-19T06:17:31.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.314+0000 7fea22ffd640 1 -- 172.21.15.5:0/2214826074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea340a4710 con 0x55fb53a765d0 2024-09-19T06:17:31.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.314+0000 7fea22ffd640 1 -- 172.21.15.5:0/2214826074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7fea34009bd0 con 0x55fb53a765d0 2024-09-19T06:17:31.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.314+0000 7fea4c3fa740 10 client.19146 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:17:31.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.314+0000 7fea4c3fa740 10 client.19146 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:17:31.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.314+0000 7fea4c3fa740 20 client.19146 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:31.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.314+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/2214826074 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55fb53afc4c0 0x55fb53afe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:31.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.314+0000 7fea4c3fa740 4 client.19146 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:17:31.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.314+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2214826074 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb539e3d70 con 0x55fb53afc4c0 2024-09-19T06:17:31.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.315+0000 7fea42ffd640 1 --2- 172.21.15.5:0/2214826074 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55fb53afc4c0 0x55fb53afe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:31.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.315+0000 7fea22ffd640 10 client.19146 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:17:31.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.315+0000 7fea42ffd640 1 --2- 172.21.15.5:0/2214826074 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55fb53afc4c0 0x55fb53afe8b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7206fcc90961aa4f server_cookie=574b6379cfb38f68 in_seq=0 out_seq=0 2024-09-19T06:17:31.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea22ffd640 1 -- 172.21.15.5:0/2214826074 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb539e3d70 con 0x55fb53afc4c0 2024-09-19T06:17:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea22ffd640 10 client.19146 handle_command_reply: tid=0 2024-09-19T06:17:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea4c3fa740 1 client.19146 shutdown 2024-09-19T06:17:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2214826074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a765d0 msgr2=0x55fb53a853a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/2214826074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a765d0 0x55fb53a853a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fea3409b600 tx=0x7fea34097dc0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2214826074 shutdown_connections 2024-09-19T06:17:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/2214826074 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55fb53afc4c0 0x55fb53afe8b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/2214826074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb53a81370 0x55fb53a85e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/2214826074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb53a77b40 0x55fb53a858e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea4c3fa740 1 --2- 172.21.15.5:0/2214826074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb53a765d0 0x55fb53a853a0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2214826074 >> 172.21.15.5:0/2214826074 conn(0x55fb53a3f310 msgr2=0x55fb53a7f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea22ffd640 0 client.19146 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:17:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2214826074 shutdown_connections 2024-09-19T06:17:31.490 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:17:31.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea4c3fa740 1 -- 172.21.15.5:0/2214826074 wait complete. 2024-09-19T06:17:31.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.316+0000 7fea4c3fa740 20 client.19146 trim_cache size 0 max 16384 2024-09-19T06:17:31.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.317+0000 7fea4a021640 1 -- 172.21.15.5:0/4188697015 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fea2c0a34b0 msgr2=0x7fea2c0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:31.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.317+0000 7fea4a021640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fea2c0a34b0 0x7fea2c0a5970 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00efd0 tx=0x7fea3c01f000 comp rx=0 tx=0).stop 2024-09-19T06:17:31.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.317+0000 7fea4a021640 1 -- 172.21.15.5:0/4188697015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea44102dd0 msgr2=0x7fea441a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:31.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.317+0000 7fea4a021640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea44102dd0 0x7fea441a48a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fea3400d950 tx=0x7fea3400de10 comp rx=0 tx=0).stop 2024-09-19T06:17:31.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.317+0000 7fea4a021640 1 -- 172.21.15.5:0/4188697015 shutdown_connections 2024-09-19T06:17:31.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.317+0000 7fea4a021640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fea2c0a34b0 0x7fea2c0a5970 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.317+0000 7fea4a021640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea44108300 0x7fea4410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.317+0000 7fea4a021640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44103780 0x7fea441a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.317+0000 7fea4a021640 1 --2- 172.21.15.5:0/4188697015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea44102dd0 0x7fea441a48a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:31.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.317+0000 7fea4a021640 1 -- 172.21.15.5:0/4188697015 >> 172.21.15.5:0/4188697015 conn(0x7fea440fc7d0 msgr2=0x7fea44100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:31.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.317+0000 7fea4a021640 1 -- 172.21.15.5:0/4188697015 shutdown_connections 2024-09-19T06:17:31.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:31.317+0000 7fea4a021640 1 -- 172.21.15.5:0/4188697015 wait complete. 2024-09-19T06:17:31.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 9393 12.27 MB/sec warmup 55 sec latency 83.510 ms 2024-09-19T06:17:32.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:17:32.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 9624 12.26 MB/sec warmup 56 sec latency 62.552 ms 2024-09-19T06:17:32.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:32 smithi005 ceph-mon[30955]: pgmap v445: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-19T06:17:32.699 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:17:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:32 smithi157 ceph-mon[35346]: pgmap v445: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-19T06:17:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:32 smithi049 ceph-mon[34640]: pgmap v445: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-19T06:17:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.361+0000 7ff5eaee3640 1 -- 172.21.15.5:0/2655289329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e41049a0 msgr2=0x7ff5e41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.361+0000 7ff5eaee3640 1 --2- 172.21.15.5:0/2655289329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e41049a0 0x7ff5e41089f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0009860 tx=0x7ff5d002a340 comp rx=0 tx=0).stop 2024-09-19T06:17:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.361+0000 7ff5eaee3640 1 -- 172.21.15.5:0/2655289329 shutdown_connections 2024-09-19T06:17:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.361+0000 7ff5eaee3640 1 --2- 172.21.15.5:0/2655289329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e4109120 0x7ff5e4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.361+0000 7ff5eaee3640 1 --2- 172.21.15.5:0/2655289329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e41049a0 0x7ff5e41089f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.361+0000 7ff5eaee3640 1 --2- 172.21.15.5:0/2655289329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e4103ff0 0x7ff5e41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.361+0000 7ff5eaee3640 1 -- 172.21.15.5:0/2655289329 >> 172.21.15.5:0/2655289329 conn(0x7ff5e40ffab0 msgr2=0x7ff5e4101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.361+0000 7ff5eaee3640 1 -- 172.21.15.5:0/2655289329 shutdown_connections 2024-09-19T06:17:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.361+0000 7ff5eaee3640 1 -- 172.21.15.5:0/2655289329 wait complete. 2024-09-19T06:17:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.362+0000 7ff5eaee3640 1 Processor -- start 2024-09-19T06:17:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.362+0000 7ff5eaee3640 1 -- start start 2024-09-19T06:17:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.362+0000 7ff5eaee3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e4103ff0 0x7ff5e419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.362+0000 7ff5eaee3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e41049a0 0x7ff5e419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.362+0000 7ff5eaee3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e4109120 0x7ff5e419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.362+0000 7ff5eaee3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e4113ff0 con 0x7ff5e4109120 2024-09-19T06:17:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.362+0000 7ff5eaee3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e4113e70 con 0x7ff5e41049a0 2024-09-19T06:17:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.362+0000 7ff5eaee3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e4114170 con 0x7ff5e4103ff0 2024-09-19T06:17:33.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e8c7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e4103ff0 0x7ff5e419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e9480640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e4109120 0x7ff5e419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e9480640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e4109120 0x7ff5e419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55932/0 (socket says 172.21.15.5:55932) 2024-09-19T06:17:33.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e41049a0 0x7ff5e419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e9480640 1 -- 172.21.15.5:0/1392471437 learned_addr learned my addr 172.21.15.5:0/1392471437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:33.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e3fff640 1 -- 172.21.15.5:0/1392471437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e4103ff0 msgr2=0x7ff5e419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e3fff640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e4103ff0 0x7ff5e419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e3fff640 1 -- 172.21.15.5:0/1392471437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e4109120 msgr2=0x7ff5e419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e3fff640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e4109120 0x7ff5e419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e3fff640 1 -- 172.21.15.5:0/1392471437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e419f500 con 0x7ff5e41049a0 2024-09-19T06:17:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e9480640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e4109120 0x7ff5e419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e3fff640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e41049a0 0x7ff5e419aa90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0035040 tx=0x7ff5d0030bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.363+0000 7ff5e1ffb640 1 -- 172.21.15.5:0/1392471437 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0037050 con 0x7ff5e41049a0 2024-09-19T06:17:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.364+0000 7ff5e1ffb640 1 -- 172.21.15.5:0/1392471437 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5d0038040 con 0x7ff5e41049a0 2024-09-19T06:17:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.364+0000 7ff5e1ffb640 1 -- 172.21.15.5:0/1392471437 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0003920 con 0x7ff5e41049a0 2024-09-19T06:17:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.364+0000 7ff5eaee3640 1 -- 172.21.15.5:0/1392471437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e419f790 con 0x7ff5e41049a0 2024-09-19T06:17:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.364+0000 7ff5eaee3640 1 -- 172.21.15.5:0/1392471437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e419fb60 con 0x7ff5e41049a0 2024-09-19T06:17:33.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.366+0000 7ff5e1ffb640 1 -- 172.21.15.5:0/1392471437 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff5d0045030 con 0x7ff5e41049a0 2024-09-19T06:17:33.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.367+0000 7ff5ed2bc740 1 Processor -- start 2024-09-19T06:17:33.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.367+0000 7ff5ed2bc740 1 -- start start 2024-09-19T06:17:33.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.367+0000 7ff5e1ffb640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff5d40a3430 0x7ff5d40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e9480640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff5d40a3430 0x7ff5d40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5ed2bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e1ffb640 1 -- 172.21.15.5:0/1392471437 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7ff5d00ef6d0 con 0x7ff5e41049a0 2024-09-19T06:17:33.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5ed2bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 0x55ad420c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5ed2bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 0x55ad420cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5ed2bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ad420262f0 con 0x55ad420c05d0 2024-09-19T06:17:33.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5ed2bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ad41fcdcf0 con 0x55ad420cb370 2024-09-19T06:17:33.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5ed2bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ad420222b0 con 0x55ad420c1b40 2024-09-19T06:17:33.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e9480640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 0x55ad420cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e9480640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 0x55ad420cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35870/0 (socket says 172.21.15.5:35870) 2024-09-19T06:17:33.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e9480640 1 -- 172.21.15.5:0/4235630541 learned_addr learned my addr 172.21.15.5:0/4235630541 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:33.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e8c7f640 1 --2- 172.21.15.5:0/4235630541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e3fff640 1 --2- 172.21.15.5:0/4235630541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 0x55ad420c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e8c7f640 1 -- 172.21.15.5:0/4235630541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 msgr2=0x55ad420cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:33.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e8c7f640 1 --2- 172.21.15.5:0/4235630541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 0x55ad420cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e8c7f640 1 -- 172.21.15.5:0/4235630541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 msgr2=0x55ad420c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:33.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e8c7f640 1 --2- 172.21.15.5:0/4235630541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 0x55ad420c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e8c7f640 1 -- 172.21.15.5:0/4235630541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad41fb23a0 con 0x55ad420c1b40 2024-09-19T06:17:33.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e9480640 1 --2- 172.21.15.5:0/4235630541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 0x55ad420cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:33.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e9480640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff5d40a3430 0x7ff5d40a58f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00fc40 tx=0x7ff5dc011810 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:33.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.368+0000 7ff5e3fff640 1 --2- 172.21.15.5:0/4235630541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 0x55ad420c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:17:33.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.369+0000 7ff5e8c7f640 1 --2- 172.21.15.5:0/4235630541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c0090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009c30 tx=0x7ff5d8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f1908a9ccadede1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:33.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.369+0000 7ff5ceffd640 1 -- 172.21.15.5:0/4235630541 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d801e070 con 0x55ad420c1b40 2024-09-19T06:17:33.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.369+0000 7ff5ceffd640 1 -- 172.21.15.5:0/4235630541 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5d80035f0 con 0x55ad420c1b40 2024-09-19T06:17:33.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.369+0000 7ff5ceffd640 1 -- 172.21.15.5:0/4235630541 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8019690 con 0x55ad420c1b40 2024-09-19T06:17:33.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.370+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/4235630541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 msgr2=0x55ad420c0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:33.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.370+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/4235630541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c0090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009c30 tx=0x7ff5d8011810 comp rx=0 tx=0).stop 2024-09-19T06:17:33.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.370+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/4235630541 shutdown_connections 2024-09-19T06:17:33.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.370+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/4235630541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 0x55ad420cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.370+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/4235630541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 0x55ad420c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.370+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/4235630541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c0090 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.370+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/4235630541 >> 172.21.15.5:0/4235630541 conn(0x55ad42089310 msgr2=0x55ad420bcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.370+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/4235630541 shutdown_connections 2024-09-19T06:17:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.370+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/4235630541 wait complete. 2024-09-19T06:17:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.371+0000 7ff5ed2bc740 1 -- start start 2024-09-19T06:17:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.371+0000 7ff5ed2bc740 1 Processor -- start 2024-09-19T06:17:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.371+0000 7ff5ed2bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 0x55ad420cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.371+0000 7ff5ed2bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.371+0000 7ff5ed2bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 0x55ad420c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.371+0000 7ff5e8c7f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 0x55ad420cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.371+0000 7ff5e8c7f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 0x55ad420cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35882/0 (socket says 172.21.15.5:35882) 2024-09-19T06:17:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.371+0000 7ff5e8c7f640 1 -- 172.21.15.5:0/1163383689 learned_addr learned my addr 172.21.15.5:0/1163383689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.372+0000 7ff5e3fff640 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.372+0000 7ff5e9480640 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 0x55ad420c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.371+0000 7ff5ed2bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ad420ce070 con 0x55ad420cb370 2024-09-19T06:17:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.372+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/1163383689 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ad41fe1060 con 0x55ad420c05d0 2024-09-19T06:17:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.372+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/1163383689 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ad420ce1f0 con 0x55ad420c1b40 2024-09-19T06:17:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.372+0000 7ff5ed2bc740 10 client.0 _ll_register_callbacks cb 0x55ad41d348c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.372+0000 7ff5e8c7f640 1 -- 172.21.15.5:0/1163383689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 msgr2=0x55ad420c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.372+0000 7ff5e8c7f640 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.372+0000 7ff5e8c7f640 1 -- 172.21.15.5:0/1163383689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 msgr2=0x55ad420c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.372+0000 7ff5e8c7f640 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 0x55ad420c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.372+0000 7ff5e8c7f640 1 -- 172.21.15.5:0/1163383689 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad41d973c0 con 0x55ad420c05d0 2024-09-19T06:17:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.372+0000 7ff5e9480640 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 0x55ad420c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.373+0000 7ff5e8c7f640 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 0x55ad420cad10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff5d801c040 tx=0x7ff5d8003dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dd9858c90383271 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.373+0000 7ff5c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:17:33.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.373+0000 7ff5c3fff640 1 -- 172.21.15.5:0/1163383689 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d801e050 con 0x55ad420c05d0 2024-09-19T06:17:33.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.373+0000 7ff5c3fff640 1 -- 172.21.15.5:0/1163383689 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7ff5d8004460 con 0x55ad420c05d0 2024-09-19T06:17:33.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.373+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/1163383689 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5d80035f0 con 0x55ad420c05d0 2024-09-19T06:17:33.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.374+0000 7ff5c3fff640 1 -- 172.21.15.5:0/1163383689 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff5d8033050 con 0x55ad420c05d0 2024-09-19T06:17:33.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.374+0000 7ff5ed2bc740 10 client.28556 fetch_fsmap learned FSMap version 43 2024-09-19T06:17:33.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.374+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/1163383689 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ad420d9ac0 con 0x55ad420c05d0 2024-09-19T06:17:33.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.374+0000 7ff5c3fff640 1 -- 172.21.15.5:0/1163383689 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5d802a070 con 0x55ad420c05d0 2024-09-19T06:17:33.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.374+0000 7ff5c3fff640 1 -- 172.21.15.5:0/1163383689 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7ff5d8004bf0 con 0x55ad420c05d0 2024-09-19T06:17:33.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.374+0000 7ff5ed2bc740 10 client.28556 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:17:33.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.374+0000 7ff5ed2bc740 10 client.28556 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:17:33.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.374+0000 7ff5ed2bc740 20 client.28556 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:33.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.374+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad420daec0 0x55ad420dd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.374+0000 7ff5ed2bc740 4 client.28556 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:17:33.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.374+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/1163383689 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff5d80035f0 con 0x55ad420daec0 2024-09-19T06:17:33.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.374+0000 7ff5e3fff640 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad420daec0 0x55ad420dd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.375+0000 7ff5c3fff640 10 client.28556 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:17:33.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.375+0000 7ff5e3fff640 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad420daec0 0x55ad420dd2b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=409eb98d66d612a3 server_cookie=75d7b67272b915b8 in_seq=0 out_seq=0 2024-09-19T06:17:33.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5c3fff640 1 -- 172.21.15.5:0/1163383689 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff5d80035f0 con 0x55ad420daec0 2024-09-19T06:17:33.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5c3fff640 10 client.28556 handle_command_reply: tid=0 2024-09-19T06:17:33.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5ed2bc740 1 client.28556 shutdown 2024-09-19T06:17:33.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/1163383689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 msgr2=0x55ad420cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:33.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 0x55ad420cad10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff5d801c040 tx=0x7ff5d8003dd0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/1163383689 shutdown_connections 2024-09-19T06:17:33.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad420daec0 0x55ad420dd2b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 0x55ad420c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5c3fff640 0 client.28556 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:17:33.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/1163383689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 0x55ad420cad10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/1163383689 >> 172.21.15.5:0/1163383689 conn(0x55ad42089310 msgr2=0x55ad420c2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:33.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.376+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/1163383689 shutdown_connections 2024-09-19T06:17:33.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.377+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/1163383689 wait complete. 2024-09-19T06:17:33.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.377+0000 7ff5ed2bc740 20 client.28556 trim_cache size 0 max 16384 2024-09-19T06:17:33.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.398+0000 7ff5ed2bc740 1 Processor -- start 2024-09-19T06:17:33.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.398+0000 7ff5ed2bc740 1 -- start start 2024-09-19T06:17:33.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.398+0000 7ff5ed2bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 0x55ad420bbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.398+0000 7ff5ed2bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.398+0000 7ff5ed2bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 0x55ad420c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5ed2bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41afa70 con 0x55ad420c05d0 2024-09-19T06:17:33.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5ed2bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41b1700 con 0x55ad420cb370 2024-09-19T06:17:33.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5ed2bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41b0170 con 0x55ad420c1b40 2024-09-19T06:17:33.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5e8c7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 0x55ad420bbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5e8c7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 0x55ad420bbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55962/0 (socket says 172.21.15.5:55962) 2024-09-19T06:17:33.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5e8c7f640 1 -- 172.21.15.5:0/2878534642 learned_addr learned my addr 172.21.15.5:0/2878534642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:33.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5e9480640 1 --2- 172.21.15.5:0/2878534642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 0x55ad420c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5e3fff640 1 --2- 172.21.15.5:0/2878534642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5e9480640 1 -- 172.21.15.5:0/2878534642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 msgr2=0x55ad420c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5e9480640 1 --2- 172.21.15.5:0/2878534642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5e9480640 1 -- 172.21.15.5:0/2878534642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 msgr2=0x55ad420bbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5e9480640 1 --2- 172.21.15.5:0/2878534642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 0x55ad420bbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5e9480640 1 -- 172.21.15.5:0/2878534642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad420bced0 con 0x55ad420cb370 2024-09-19T06:17:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5e8c7f640 1 --2- 172.21.15.5:0/2878534642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 0x55ad420bbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.399+0000 7ff5e9480640 1 --2- 172.21.15.5:0/2878534642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 0x55ad420c3660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009270 tx=0x7ff5dc01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46104fd2999118d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.400+0000 7ff5ce7fc640 1 -- 172.21.15.5:0/2878534642 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc02e070 con 0x55ad420cb370 2024-09-19T06:17:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.400+0000 7ff5ce7fc640 1 -- 172.21.15.5:0/2878534642 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5dc021b30 con 0x55ad420cb370 2024-09-19T06:17:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.400+0000 7ff5ce7fc640 1 -- 172.21.15.5:0/2878534642 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc01a040 con 0x55ad420cb370 2024-09-19T06:17:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.401+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2878534642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 msgr2=0x55ad420c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.401+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/2878534642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 0x55ad420c3660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009270 tx=0x7ff5dc01f000 comp rx=0 tx=0).stop 2024-09-19T06:17:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.401+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2878534642 shutdown_connections 2024-09-19T06:17:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.401+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/2878534642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420cb370 0x55ad420c3660 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.401+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/2878534642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.401+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/2878534642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420c05d0 0x55ad420bbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.401+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2878534642 >> 172.21.15.5:0/2878534642 conn(0x55ad42089310 msgr2=0x55ad420bb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.401+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2878534642 shutdown_connections 2024-09-19T06:17:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.401+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2878534642 wait complete. 2024-09-19T06:17:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.402+0000 7ff5ed2bc740 1 -- start start 2024-09-19T06:17:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.402+0000 7ff5ed2bc740 1 Processor -- start 2024-09-19T06:17:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.402+0000 7ff5ed2bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 0x55ad420d3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.402+0000 7ff5ed2bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.402+0000 7ff5ed2bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 0x55ad420c5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.402+0000 7ff5e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45286/0 (socket says 172.21.15.5:45286) 2024-09-19T06:17:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e8c7f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 0x55ad420d3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e9480640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 0x55ad420c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e8c7f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 0x55ad420d3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35896/0 (socket says 172.21.15.5:35896) 2024-09-19T06:17:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e9480640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 0x55ad420c5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55972/0 (socket says 172.21.15.5:55972) 2024-09-19T06:17:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2328608212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41b1030 con 0x55ad420cb370 2024-09-19T06:17:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e3fff640 1 -- 172.21.15.5:0/2328608212 learned_addr learned my addr 172.21.15.5:0/2328608212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2328608212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41b08a0 con 0x55ad420c05d0 2024-09-19T06:17:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2328608212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41ada70 con 0x55ad420c1b40 2024-09-19T06:17:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5ed2bc740 10 client.0 _ll_register_callbacks cb 0x7ff5d801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e3fff640 1 -- 172.21.15.5:0/2328608212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 msgr2=0x55ad420d3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e3fff640 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 0x55ad420d3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e3fff640 1 -- 172.21.15.5:0/2328608212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 msgr2=0x55ad420c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:33.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e3fff640 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 0x55ad420c5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e3fff640 1 -- 172.21.15.5:0/2328608212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad420c57f0 con 0x55ad420c1b40 2024-09-19T06:17:33.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e8c7f640 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 0x55ad420d3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-19T06:17:33.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e9480640 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 0x55ad420c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-19T06:17:33.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5e3fff640 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c4bc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00bba60 tx=0x7ff5d002a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a63e3b44e653e703 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:33.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.403+0000 7ff5c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:17:33.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.404+0000 7ff5c3fff640 1 -- 172.21.15.5:0/2328608212 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00b8140 con 0x55ad420c1b40 2024-09-19T06:17:33.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.404+0000 7ff5c3fff640 1 -- 172.21.15.5:0/2328608212 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7ff5d00ed020 con 0x55ad420c1b40 2024-09-19T06:17:33.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.404+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2328608212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5dc021b30 con 0x55ad420c1b40 2024-09-19T06:17:33.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.404+0000 7ff5c3fff640 1 -- 172.21.15.5:0/2328608212 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff5d00b82e0 con 0x55ad420c1b40 2024-09-19T06:17:33.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.404+0000 7ff5ed2bc740 10 client.28507 fetch_fsmap learned FSMap version 43 2024-09-19T06:17:33.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.404+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2328608212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ad4214e130 con 0x55ad420c1b40 2024-09-19T06:17:33.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.404+0000 7ff5c3fff640 1 -- 172.21.15.5:0/2328608212 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5dc021b30 con 0x55ad420c1b40 2024-09-19T06:17:33.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.404+0000 7ff5c3fff640 1 -- 172.21.15.5:0/2328608212 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7ff5d00a60b0 con 0x55ad420c1b40 2024-09-19T06:17:33.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.405+0000 7ff5ed2bc740 10 client.28507 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:17:33.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.405+0000 7ff5ed2bc740 10 client.28507 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:17:33.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.405+0000 7ff5ed2bc740 20 client.28507 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:33.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.405+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad420daec0 0x55ad421509a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:33.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.405+0000 7ff5ed2bc740 4 client.28507 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:17:33.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.405+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2328608212 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad42033860 con 0x55ad420daec0 2024-09-19T06:17:33.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.405+0000 7ff5e8c7f640 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad420daec0 0x55ad421509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:33.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.406+0000 7ff5c3fff640 10 client.28507 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:17:33.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.406+0000 7ff5e8c7f640 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad420daec0 0x55ad421509a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3382c5d2cf0aed6 server_cookie=ca25a8d8344cb647 in_seq=0 out_seq=0 2024-09-19T06:17:33.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.406+0000 7ff5c3fff640 1 -- 172.21.15.5:0/2328608212 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad42033860 con 0x55ad420daec0 2024-09-19T06:17:33.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.406+0000 7ff5c3fff640 10 client.28507 handle_command_reply: tid=0 2024-09-19T06:17:33.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.406+0000 7ff5ed2bc740 1 client.28507 shutdown 2024-09-19T06:17:33.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.406+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2328608212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 msgr2=0x55ad420c4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:33.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.407+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c4bc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00bba60 tx=0x7ff5d002a340 comp rx=0 tx=0).stop 2024-09-19T06:17:33.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.407+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2328608212 shutdown_connections 2024-09-19T06:17:33.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.407+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ad420daec0 0x55ad421509a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.407+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad420cb370 0x55ad420c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.407+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad420c1b40 0x55ad420c4bc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.407+0000 7ff5ed2bc740 1 --2- 172.21.15.5:0/2328608212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad420c05d0 0x55ad420d3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.407+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2328608212 >> 172.21.15.5:0/2328608212 conn(0x55ad42089310 msgr2=0x55ad420bbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:33.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.407+0000 7ff5c3fff640 0 client.28507 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:17:33.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.407+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2328608212 shutdown_connections 2024-09-19T06:17:33.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.407+0000 7ff5ed2bc740 1 -- 172.21.15.5:0/2328608212 wait complete. 2024-09-19T06:17:33.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.408+0000 7ff5ed2bc740 20 client.28507 trim_cache size 0 max 16384 2024-09-19T06:17:33.416 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:17:33.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.408+0000 7ff5eaee3640 1 -- 172.21.15.5:0/1392471437 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff5d40a3430 msgr2=0x7ff5d40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:33.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.408+0000 7ff5eaee3640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff5d40a3430 0x7ff5d40a58f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00fc40 tx=0x7ff5dc011810 comp rx=0 tx=0).stop 2024-09-19T06:17:33.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.408+0000 7ff5eaee3640 1 -- 172.21.15.5:0/1392471437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e41049a0 msgr2=0x7ff5e419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:33.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.408+0000 7ff5eaee3640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e41049a0 0x7ff5e419aa90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0035040 tx=0x7ff5d0030bd0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.408+0000 7ff5eaee3640 1 -- 172.21.15.5:0/1392471437 shutdown_connections 2024-09-19T06:17:33.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.408+0000 7ff5eaee3640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e4109120 0x7ff5e419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.408+0000 7ff5eaee3640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e41049a0 0x7ff5e419aa90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.408+0000 7ff5eaee3640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff5d40a3430 0x7ff5d40a58f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.408+0000 7ff5eaee3640 1 --2- 172.21.15.5:0/1392471437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e4103ff0 0x7ff5e419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.408+0000 7ff5eaee3640 1 -- 172.21.15.5:0/1392471437 >> 172.21.15.5:0/1392471437 conn(0x7ff5e40ffab0 msgr2=0x7ff5e4100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.409+0000 7ff5eaee3640 1 -- 172.21.15.5:0/1392471437 shutdown_connections 2024-09-19T06:17:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:33.409+0000 7ff5eaee3640 1 -- 172.21.15.5:0/1392471437 wait complete. 2024-09-19T06:17:33.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 9763 12.22 MB/sec warmup 57 sec latency 125.038 ms 2024-09-19T06:17:34.111 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:17:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.267+0000 7f53d1749640 1 -- 172.21.15.5:0/1034466856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53cc10c130 msgr2=0x7f53cc10e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.267+0000 7f53d1749640 1 --2- 172.21.15.5:0/1034466856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53cc10c130 0x7f53cc10e580 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009ca0 tx=0x7f53c402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.267+0000 7f53d1749640 1 -- 172.21.15.5:0/1034466856 shutdown_connections 2024-09-19T06:17:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.267+0000 7f53d1749640 1 --2- 172.21.15.5:0/1034466856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53cc10c130 0x7f53cc10e580 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.267+0000 7f53d1749640 1 --2- 172.21.15.5:0/1034466856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53cc1035e0 0x7f53cc10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.267+0000 7f53d1749640 1 --2- 172.21.15.5:0/1034466856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53cc102c30 0x7f53cc103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.267+0000 7f53d1749640 1 -- 172.21.15.5:0/1034466856 >> 172.21.15.5:0/1034466856 conn(0x7f53cc0fc6f0 msgr2=0x7f53cc0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.267+0000 7f53d1749640 1 -- 172.21.15.5:0/1034466856 shutdown_connections 2024-09-19T06:17:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.268+0000 7f53d1749640 1 -- 172.21.15.5:0/1034466856 wait complete. 2024-09-19T06:17:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.268+0000 7f53d1749640 1 Processor -- start 2024-09-19T06:17:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.268+0000 7f53d1749640 1 -- start start 2024-09-19T06:17:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53d1749640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53cc102c30 0x7f53cc198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53d1749640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53cc1035e0 0x7f53cc198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53d1749640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53cc10c130 0x7f53cc19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53d1749640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc111de0 con 0x7f53cc10c130 2024-09-19T06:17:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53d1749640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc111c60 con 0x7f53cc1035e0 2024-09-19T06:17:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53d1749640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc111f60 con 0x7f53cc102c30 2024-09-19T06:17:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53cc10c130 0x7f53cc19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53cc10c130 0x7f53cc19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55992/0 (socket says 172.21.15.5:55992) 2024-09-19T06:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53cb7fe640 1 -- 172.21.15.5:0/488585059 learned_addr learned my addr 172.21.15.5:0/488585059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53ca7fc640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53cc1035e0 0x7f53cc198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53caffd640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53cc102c30 0x7f53cc198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53cb7fe640 1 -- 172.21.15.5:0/488585059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53cc102c30 msgr2=0x7f53cc198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53cb7fe640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53cc102c30 0x7f53cc198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53cb7fe640 1 -- 172.21.15.5:0/488585059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53cc1035e0 msgr2=0x7f53cc198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53cb7fe640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53cc1035e0 0x7f53cc198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53cb7fe640 1 -- 172.21.15.5:0/488585059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc19d300 con 0x7f53cc10c130 2024-09-19T06:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.269+0000 7f53ca7fc640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53cc1035e0 0x7f53cc198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.270+0000 7f53cb7fe640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53cc10c130 0x7f53cc19cbd0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f53c4031040 tx=0x7f53c4032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.270+0000 7f53b3fff640 1 -- 172.21.15.5:0/488585059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4003b70 con 0x7f53cc10c130 2024-09-19T06:17:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.270+0000 7f53b3fff640 1 -- 172.21.15.5:0/488585059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53c4003d10 con 0x7f53cc10c130 2024-09-19T06:17:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.270+0000 7f53b3fff640 1 -- 172.21.15.5:0/488585059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4030b60 con 0x7f53cc10c130 2024-09-19T06:17:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.270+0000 7f53d1749640 1 -- 172.21.15.5:0/488585059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53cc19d590 con 0x7f53cc10c130 2024-09-19T06:17:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.270+0000 7f53d1749640 1 -- 172.21.15.5:0/488585059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53cc1a4e80 con 0x7f53cc10c130 2024-09-19T06:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.273+0000 7f53b3fff640 1 -- 172.21.15.5:0/488585059 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f53c4030d00 con 0x7f53cc10c130 2024-09-19T06:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.273+0000 7f53d45bd740 1 Processor -- start 2024-09-19T06:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.273+0000 7f53d45bd740 1 -- start start 2024-09-19T06:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.273+0000 7f53d45bd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86deaf0 0x55feb86deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.273+0000 7f53d45bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 0x55feb86e9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.273+0000 7f53d45bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb86ec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.273+0000 7f53d45bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55feb86432f0 con 0x55feb86df410 2024-09-19T06:17:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.273+0000 7f53d45bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55feb85eb270 con 0x55feb86e9cb0 2024-09-19T06:17:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.273+0000 7f53d45bd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55feb863f2b0 con 0x55feb86deaf0 2024-09-19T06:17:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.273+0000 7f53caffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 0x55feb86e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.273+0000 7f53caffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 0x55feb86e9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56008/0 (socket says 172.21.15.5:56008) 2024-09-19T06:17:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.273+0000 7f53caffd640 1 -- 172.21.15.5:0/429788857 learned_addr learned my addr 172.21.15.5:0/429788857 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53ca7fc640 1 --2- 172.21.15.5:0/429788857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb86ec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53ca7fc640 1 --2- 172.21.15.5:0/429788857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86deaf0 0x55feb86deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53ca7fc640 1 -- 172.21.15.5:0/429788857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86deaf0 msgr2=0x55feb86deed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53ca7fc640 1 --2- 172.21.15.5:0/429788857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86deaf0 0x55feb86deed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53ca7fc640 1 -- 172.21.15.5:0/429788857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 msgr2=0x55feb86e9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53ca7fc640 1 --2- 172.21.15.5:0/429788857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 0x55feb86e9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53ca7fc640 1 -- 172.21.15.5:0/429788857 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb85ea340 con 0x55feb86e9cb0 2024-09-19T06:17:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53caffd640 1 --2- 172.21.15.5:0/429788857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 0x55feb86e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53b3fff640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53b40a3430 0x7f53b40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53ca7fc640 1 --2- 172.21.15.5:0/429788857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb86ec0a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009710 tx=0x7f53c00096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93ef6bcec7627a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53b3fff640 1 -- 172.21.15.5:0/488585059 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f53c40f1130 con 0x7f53cc10c130 2024-09-19T06:17:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53b17fa640 1 -- 172.21.15.5:0/429788857 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0004050 con 0x55feb86e9cb0 2024-09-19T06:17:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.274+0000 7f53caffd640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53b40a3430 0x7f53b40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.275+0000 7f53b17fa640 1 -- 172.21.15.5:0/429788857 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53c000fdc0 con 0x55feb86e9cb0 2024-09-19T06:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.275+0000 7f53b17fa640 1 -- 172.21.15.5:0/429788857 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c00054a0 con 0x55feb86e9cb0 2024-09-19T06:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.275+0000 7f53caffd640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53b40a3430 0x7f53b40a58f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55feb86dbec0 tx=0x7f53bc005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.275+0000 7f53d45bd740 1 -- 172.21.15.5:0/429788857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 msgr2=0x55feb86ec0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.275+0000 7f53d45bd740 1 --2- 172.21.15.5:0/429788857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb86ec0a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009710 tx=0x7f53c00096a0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.275+0000 7f53d45bd740 1 -- 172.21.15.5:0/429788857 shutdown_connections 2024-09-19T06:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.275+0000 7f53d45bd740 1 --2- 172.21.15.5:0/429788857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb86ec0a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.275+0000 7f53d45bd740 1 --2- 172.21.15.5:0/429788857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 0x55feb86e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.275+0000 7f53d45bd740 1 --2- 172.21.15.5:0/429788857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86deaf0 0x55feb86deed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.275+0000 7f53d45bd740 1 -- 172.21.15.5:0/429788857 >> 172.21.15.5:0/429788857 conn(0x55feb86db060 msgr2=0x55feb86db440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.275+0000 7f53d45bd740 1 -- 172.21.15.5:0/429788857 shutdown_connections 2024-09-19T06:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.275+0000 7f53d45bd740 1 -- 172.21.15.5:0/429788857 wait complete. 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53d45bd740 1 -- start start 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53d45bd740 1 Processor -- start 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53d45bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86deaf0 0x55feb86e6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53d45bd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86df410 0x55feb86e7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53d45bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86e9cb0 0x55feb86e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53d45bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55feb86ec880 con 0x55feb86e9cb0 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53d45bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55feb85cc290 con 0x55feb86deaf0 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53d45bd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55feb86eca00 con 0x55feb86df410 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53d45bd740 10 client.0 _ll_register_callbacks cb 0x55feb85fd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86e9cb0 0x55feb86e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86e9cb0 0x55feb86e75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56018/0 (socket says 172.21.15.5:56018) 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53cb7fe640 1 -- 172.21.15.5:0/2653620051 learned_addr learned my addr 172.21.15.5:0/2653620051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.276+0000 7f53caffd640 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86deaf0 0x55feb86e6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.277+0000 7f53ca7fc640 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86df410 0x55feb86e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.277+0000 7f53cb7fe640 1 -- 172.21.15.5:0/2653620051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86df410 msgr2=0x55feb86e7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.277+0000 7f53cb7fe640 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86df410 0x55feb86e7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.277+0000 7f53cb7fe640 1 -- 172.21.15.5:0/2653620051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86deaf0 msgr2=0x55feb86e6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.277+0000 7f53cb7fe640 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86deaf0 0x55feb86e6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.277+0000 7f53cb7fe640 1 -- 172.21.15.5:0/2653620051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb8356210 con 0x55feb86e9cb0 2024-09-19T06:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.277+0000 7f53ca7fc640 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86df410 0x55feb86e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.277+0000 7f53cb7fe640 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86e9cb0 0x55feb86e75c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f53c40c5c40 tx=0x7f53c40bcf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126281f5b8fac79f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.277+0000 7f53aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.277+0000 7f53aeff5640 1 -- 172.21.15.5:0/2653620051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c40c18b0 con 0x55feb86e9cb0 2024-09-19T06:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.277+0000 7f53aeff5640 1 -- 172.21.15.5:0/2653620051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f53c40c5ce0 con 0x55feb86e9cb0 2024-09-19T06:17:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.278+0000 7f53d45bd740 1 -- 172.21.15.5:0/2653620051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c000fdc0 con 0x55feb86e9cb0 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.278+0000 7f53aeff5640 1 -- 172.21.15.5:0/2653620051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f53c40c6170 con 0x55feb86e9cb0 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.278+0000 7f53d45bd740 10 client.19173 fetch_fsmap learned FSMap version 44 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.278+0000 7f53d45bd740 1 -- 172.21.15.5:0/2653620051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55feb8422620 con 0x55feb86e9cb0 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.278+0000 7f53aeff5640 1 -- 172.21.15.5:0/2653620051 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53c000fdc0 con 0x55feb86e9cb0 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.279+0000 7f53aeff5640 1 -- 172.21.15.5:0/2653620051 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8402+0+0 (secure 0 0 0) 0x7f53c40bc950 con 0x55feb86e9cb0 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.279+0000 7f53d45bd740 10 client.19173 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.279+0000 7f53d45bd740 10 client.19173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.279+0000 7f53d45bd740 20 client.19173 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.279+0000 7f53d45bd740 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55feb86fe100 0x55feb8700580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.279+0000 7f53d45bd740 4 client.19173 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.279+0000 7f53d45bd740 1 -- 172.21.15.5:0/2653620051 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55feb8650a20 con 0x55feb86fe100 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.279+0000 7f53caffd640 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55feb86fe100 0x55feb8700580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.280+0000 7f53aeff5640 10 client.19173 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.280+0000 7f53caffd640 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55feb86fe100 0x55feb8700580 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=529fbe2da07f9d00 server_cookie=e8055e4cd0d227be in_seq=0 out_seq=0 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.280+0000 7f53aeff5640 1 -- 172.21.15.5:0/2653620051 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55feb8650a20 con 0x55feb86fe100 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.280+0000 7f53aeff5640 10 client.19173 handle_command_reply: tid=0 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.281+0000 7f53d45bd740 1 client.19173 shutdown 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.281+0000 7f53d45bd740 1 -- 172.21.15.5:0/2653620051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86e9cb0 msgr2=0x55feb86e75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.281+0000 7f53d45bd740 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86e9cb0 0x55feb86e75c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f53c40c5c40 tx=0x7f53c40bcf20 comp rx=0 tx=0).stop 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.281+0000 7f53d45bd740 1 -- 172.21.15.5:0/2653620051 shutdown_connections 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.281+0000 7f53d45bd740 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55feb86fe100 0x55feb8700580 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.281+0000 7f53aeff5640 0 client.19173 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.281+0000 7f53d45bd740 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86e9cb0 0x55feb86e75c0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.281+0000 7f53d45bd740 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86df410 0x55feb86e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.281+0000 7f53d45bd740 1 --2- 172.21.15.5:0/2653620051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86deaf0 0x55feb86e6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.281+0000 7f53d45bd740 1 -- 172.21.15.5:0/2653620051 >> 172.21.15.5:0/2653620051 conn(0x55feb86db060 msgr2=0x55feb86e0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.281+0000 7f53d45bd740 1 -- 172.21.15.5:0/2653620051 shutdown_connections 2024-09-19T06:17:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.281+0000 7f53d45bd740 1 -- 172.21.15.5:0/2653620051 wait complete. 2024-09-19T06:17:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.282+0000 7f53d45bd740 20 client.19173 trim_cache size 0 max 16384 2024-09-19T06:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.304+0000 7f53d45bd740 1 Processor -- start 2024-09-19T06:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.304+0000 7f53d45bd740 1 -- start start 2024-09-19T06:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53d45bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86deaf0 0x55feb86e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53d45bd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86df410 0x55feb8705e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53d45bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb8706390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53d45bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1ab5b0 con 0x55feb86deaf0 2024-09-19T06:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53d45bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1ad270 con 0x55feb86e9cb0 2024-09-19T06:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53d45bd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1abce0 con 0x55feb86df410 2024-09-19T06:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53caffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86deaf0 0x55feb86e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb8706390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53ca7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86df410 0x55feb8705e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb8706390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35930/0 (socket says 172.21.15.5:35930) 2024-09-19T06:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53caffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86deaf0 0x55feb86e73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56028/0 (socket says 172.21.15.5:56028) 2024-09-19T06:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53cb7fe640 1 -- 172.21.15.5:0/2379078734 learned_addr learned my addr 172.21.15.5:0/2379078734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53cb7fe640 1 -- 172.21.15.5:0/2379078734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86df410 msgr2=0x55feb8705e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53cb7fe640 1 --2- 172.21.15.5:0/2379078734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86df410 0x55feb8705e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53cb7fe640 1 -- 172.21.15.5:0/2379078734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86deaf0 msgr2=0x55feb86e73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53cb7fe640 1 --2- 172.21.15.5:0/2379078734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86deaf0 0x55feb86e73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.305+0000 7f53cb7fe640 1 -- 172.21.15.5:0/2379078734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb86dbcf0 con 0x55feb86e9cb0 2024-09-19T06:17:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.306+0000 7f53cb7fe640 1 --2- 172.21.15.5:0/2379078734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb8706390 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f53c4030ff0 tx=0x7f53c40bcf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=856efa085c08b2b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.306+0000 7f53b0ff9640 1 -- 172.21.15.5:0/2379078734 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c40bd2d0 con 0x55feb86e9cb0 2024-09-19T06:17:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.306+0000 7f53b0ff9640 1 -- 172.21.15.5:0/2379078734 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53c4036030 con 0x55feb86e9cb0 2024-09-19T06:17:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.306+0000 7f53b0ff9640 1 -- 172.21.15.5:0/2379078734 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c40bc950 con 0x55feb86e9cb0 2024-09-19T06:17:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.307+0000 7f53d45bd740 1 -- 172.21.15.5:0/2379078734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 msgr2=0x55feb8706390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.307+0000 7f53d45bd740 1 --2- 172.21.15.5:0/2379078734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb8706390 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f53c4030ff0 tx=0x7f53c40bcf90 comp rx=0 tx=0).stop 2024-09-19T06:17:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.307+0000 7f53d45bd740 1 -- 172.21.15.5:0/2379078734 shutdown_connections 2024-09-19T06:17:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.307+0000 7f53d45bd740 1 --2- 172.21.15.5:0/2379078734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb8706390 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.307+0000 7f53d45bd740 1 --2- 172.21.15.5:0/2379078734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86df410 0x55feb8705e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.307+0000 7f53d45bd740 1 --2- 172.21.15.5:0/2379078734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86deaf0 0x55feb86e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.307+0000 7f53d45bd740 1 -- 172.21.15.5:0/2379078734 >> 172.21.15.5:0/2379078734 conn(0x55feb86db060 msgr2=0x55feb86e5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.307+0000 7f53d45bd740 1 -- 172.21.15.5:0/2379078734 shutdown_connections 2024-09-19T06:17:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.307+0000 7f53d45bd740 1 -- 172.21.15.5:0/2379078734 wait complete. 2024-09-19T06:17:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.307+0000 7f53d45bd740 1 -- start start 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.307+0000 7f53d45bd740 1 Processor -- start 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53d45bd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86deaf0 0x55feb86f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53d45bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 0x55feb86f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53d45bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb86e3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53d45bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1acba0 con 0x55feb86df410 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53ca7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 0x55feb86f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53caffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86deaf0 0x55feb86f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53d45bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1ac410 con 0x55feb86e9cb0 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53d45bd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1a95b0 con 0x55feb86deaf0 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53caffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86deaf0 0x55feb86f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45318/0 (socket says 172.21.15.5:45318) 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53ca7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 0x55feb86f6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56040/0 (socket says 172.21.15.5:56040) 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53caffd640 1 -- 172.21.15.5:0/1616941900 learned_addr learned my addr 172.21.15.5:0/1616941900 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53cb7fe640 1 --2- 172.21.15.5:0/1616941900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb86e3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53d45bd740 10 client.0 _ll_register_callbacks cb 0x7f53c4009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53caffd640 1 -- 172.21.15.5:0/1616941900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 msgr2=0x55feb86e3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53caffd640 1 --2- 172.21.15.5:0/1616941900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb86e3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53caffd640 1 -- 172.21.15.5:0/1616941900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 msgr2=0x55feb86f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.308+0000 7f53caffd640 1 --2- 172.21.15.5:0/1616941900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 0x55feb86f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.309+0000 7f53caffd640 1 -- 172.21.15.5:0/1616941900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb86e3f50 con 0x55feb86deaf0 2024-09-19T06:17:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.309+0000 7f53caffd640 1 --2- 172.21.15.5:0/1616941900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86deaf0 0x55feb86f64d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009860 tx=0x7f53bc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7709dcf374ec3b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.309+0000 7f53aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:17:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.309+0000 7f53aeff5640 1 -- 172.21.15.5:0/1616941900 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc004030 con 0x55feb86deaf0 2024-09-19T06:17:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.309+0000 7f53aeff5640 1 -- 172.21.15.5:0/1616941900 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f53bc0041d0 con 0x55feb86deaf0 2024-09-19T06:17:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.309+0000 7f53d45bd740 1 -- 172.21.15.5:0/1616941900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c4036030 con 0x55feb86deaf0 2024-09-19T06:17:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.309+0000 7f53aeff5640 1 -- 172.21.15.5:0/1616941900 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f53bc0043d0 con 0x55feb86deaf0 2024-09-19T06:17:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.309+0000 7f53d45bd740 10 client.28522 fetch_fsmap learned FSMap version 44 2024-09-19T06:17:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.309+0000 7f53d45bd740 1 -- 172.21.15.5:0/1616941900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55feb8768b00 con 0x55feb86deaf0 2024-09-19T06:17:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.310+0000 7f53aeff5640 1 -- 172.21.15.5:0/1616941900 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53bc003aa0 con 0x55feb86deaf0 2024-09-19T06:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.310+0000 7f53aeff5640 1 -- 172.21.15.5:0/1616941900 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 44) ==== 8402+0+0 (secure 0 0 0) 0x7f53bc01f850 con 0x55feb86deaf0 2024-09-19T06:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.310+0000 7f53d45bd740 10 client.28522 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T06:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.310+0000 7f53d45bd740 10 client.28522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.310+0000 7f53d45bd740 20 client.28522 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.310+0000 7f53d45bd740 1 --2- 172.21.15.5:0/1616941900 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55feb86fe100 0x55feb876b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.310+0000 7f53d45bd740 4 client.28522 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.310+0000 7f53d45bd740 1 -- 172.21.15.5:0/1616941900 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f53c4036030 con 0x55feb86fe100 2024-09-19T06:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.310+0000 7f53ca7fc640 1 --2- 172.21.15.5:0/1616941900 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55feb86fe100 0x55feb876b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.311+0000 7f53ca7fc640 1 --2- 172.21.15.5:0/1616941900 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55feb86fe100 0x55feb876b380 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20c2492ee39c97d7 server_cookie=e72b28ab684d4355 in_seq=0 out_seq=0 2024-09-19T06:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.311+0000 7f53aeff5640 10 client.28522 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.364+0000 7f53aeff5640 1 -- 172.21.15.5:0/1616941900 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f53c4036030 con 0x55feb86fe100 2024-09-19T06:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.364+0000 7f53aeff5640 10 client.28522 handle_command_reply: tid=0 2024-09-19T06:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.364+0000 7f53d45bd740 1 client.28522 shutdown 2024-09-19T06:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.365+0000 7f53d45bd740 1 -- 172.21.15.5:0/1616941900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86deaf0 msgr2=0x55feb86f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.365+0000 7f53d45bd740 1 --2- 172.21.15.5:0/1616941900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86deaf0 0x55feb86f64d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009860 tx=0x7f53bc0046c0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.365+0000 7f53d45bd740 1 -- 172.21.15.5:0/1616941900 shutdown_connections 2024-09-19T06:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.365+0000 7f53d45bd740 1 --2- 172.21.15.5:0/1616941900 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55feb86fe100 0x55feb876b380 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.365+0000 7f53aeff5640 0 client.28522 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.365+0000 7f53d45bd740 1 --2- 172.21.15.5:0/1616941900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55feb86e9cb0 0x55feb86e3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.365+0000 7f53d45bd740 1 --2- 172.21.15.5:0/1616941900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feb86df410 0x55feb86f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.365+0000 7f53d45bd740 1 --2- 172.21.15.5:0/1616941900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55feb86deaf0 0x55feb86f64d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.365+0000 7f53d45bd740 1 -- 172.21.15.5:0/1616941900 >> 172.21.15.5:0/1616941900 conn(0x55feb86db060 msgr2=0x55feb86d99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.365+0000 7f53d45bd740 1 -- 172.21.15.5:0/1616941900 shutdown_connections 2024-09-19T06:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.365+0000 7f53d45bd740 1 -- 172.21.15.5:0/1616941900 wait complete. 2024-09-19T06:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.365+0000 7f53d45bd740 20 client.28522 trim_cache size 0 max 16384 2024-09-19T06:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:17:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:17:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.366+0000 7f53d1749640 1 -- 172.21.15.5:0/488585059 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53b40a3430 msgr2=0x7f53b40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.366+0000 7f53d1749640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53b40a3430 0x7f53b40a58f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55feb86dbec0 tx=0x7f53bc005f90 comp rx=0 tx=0).stop 2024-09-19T06:17:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.366+0000 7f53d1749640 1 -- 172.21.15.5:0/488585059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53cc10c130 msgr2=0x7f53cc19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.366+0000 7f53d1749640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53cc10c130 0x7f53cc19cbd0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f53c4031040 tx=0x7f53c4032f80 comp rx=0 tx=0).stop 2024-09-19T06:17:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.366+0000 7f53d1749640 1 -- 172.21.15.5:0/488585059 shutdown_connections 2024-09-19T06:17:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.366+0000 7f53d1749640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53cc10c130 0x7f53cc19cbd0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.366+0000 7f53d1749640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53cc1035e0 0x7f53cc198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.366+0000 7f53d1749640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f53b40a3430 0x7f53b40a58f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.366+0000 7f53d1749640 1 --2- 172.21.15.5:0/488585059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53cc102c30 0x7f53cc198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.366+0000 7f53d1749640 1 -- 172.21.15.5:0/488585059 >> 172.21.15.5:0/488585059 conn(0x7f53cc0fc6f0 msgr2=0x7f53cc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.366+0000 7f53d1749640 1 -- 172.21.15.5:0/488585059 shutdown_connections 2024-09-19T06:17:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:34.366+0000 7f53d1749640 1 -- 172.21.15.5:0/488585059 wait complete. 2024-09-19T06:17:34.501 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:17:34.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 9918 12.09 MB/sec warmup 58 sec latency 81.857 ms 2024-09-19T06:17:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:34 smithi157 ceph-mon[35346]: pgmap v446: 97 pgs: 97 active+clean; 447 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 370 op/s 2024-09-19T06:17:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:34 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:17:34.825 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:34 smithi005 ceph-mon[30955]: pgmap v446: 97 pgs: 97 active+clean; 447 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 370 op/s 2024-09-19T06:17:34.825 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:34 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:17:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:34 smithi049 ceph-mon[34640]: pgmap v446: 97 pgs: 97 active+clean; 447 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 370 op/s 2024-09-19T06:17:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:34 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:17:35.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.106+0000 7fefa22a5640 1 -- 172.21.15.5:0/817297451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef9c102ef0 msgr2=0x7fef9c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.106+0000 7fefa22a5640 1 --2- 172.21.15.5:0/817297451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef9c102ef0 0x7fef9c10da40 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fef94009be0 tx=0x7fef9402a750 comp rx=0 tx=0).stop 2024-09-19T06:17:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.106+0000 7fefa22a5640 1 -- 172.21.15.5:0/817297451 shutdown_connections 2024-09-19T06:17:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.106+0000 7fefa22a5640 1 --2- 172.21.15.5:0/817297451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef9c102ef0 0x7fef9c10da40 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.106+0000 7fefa22a5640 1 --2- 172.21.15.5:0/817297451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c102570 0x7fef9c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.106+0000 7fefa22a5640 1 --2- 172.21.15.5:0/817297451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9c100e20 0x7fef9c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.107+0000 7fefa22a5640 1 -- 172.21.15.5:0/817297451 >> 172.21.15.5:0/817297451 conn(0x7fef9c0fc7d0 msgr2=0x7fef9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.107+0000 7fefa22a5640 1 -- 172.21.15.5:0/817297451 shutdown_connections 2024-09-19T06:17:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.107+0000 7fefa22a5640 1 -- 172.21.15.5:0/817297451 wait complete. 2024-09-19T06:17:35.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.108+0000 7fefa22a5640 1 Processor -- start 2024-09-19T06:17:35.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.108+0000 7fefa22a5640 1 -- start start 2024-09-19T06:17:35.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.109+0000 7fefa22a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef9c100e20 0x7fef9c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.109+0000 7fefa22a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c102570 0x7fef9c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.109+0000 7fefa22a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9c102ef0 0x7fef9c19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.109+0000 7fefa22a5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c110e30 con 0x7fef9c100e20 2024-09-19T06:17:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.109+0000 7fef9bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef9c100e20 0x7fef9c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.109+0000 7fef9bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef9c100e20 0x7fef9c1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53502/0 (socket says 172.21.15.5:53502) 2024-09-19T06:17:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.109+0000 7fef9bfff640 1 -- 172.21.15.5:0/458616332 learned_addr learned my addr 172.21.15.5:0/458616332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.109+0000 7fefa0842640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9c102ef0 0x7fef9c19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.110+0000 7fef93fff640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c102570 0x7fef9c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.110+0000 7fefa22a5640 1 -- 172.21.15.5:0/458616332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c110cb0 con 0x7fef9c102570 2024-09-19T06:17:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.110+0000 7fefa22a5640 1 -- 172.21.15.5:0/458616332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c110fb0 con 0x7fef9c102ef0 2024-09-19T06:17:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.110+0000 7fef9bfff640 1 -- 172.21.15.5:0/458616332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9c102ef0 msgr2=0x7fef9c19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.110+0000 7fef9bfff640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9c102ef0 0x7fef9c19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.110+0000 7fef9bfff640 1 -- 172.21.15.5:0/458616332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c102570 msgr2=0x7fef9c197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.110+0000 7fef9bfff640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c102570 0x7fef9c197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.110+0000 7fef9bfff640 1 -- 172.21.15.5:0/458616332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c19c2a0 con 0x7fef9c100e20 2024-09-19T06:17:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.110+0000 7fef93fff640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c102570 0x7fef9c197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:17:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.110+0000 7fefa0842640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9c102ef0 0x7fef9c19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.111+0000 7fef9bfff640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef9c100e20 0x7fef9c1972f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00e9c0 tx=0x7fef8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.111+0000 7fef99ffb640 1 -- 172.21.15.5:0/458616332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00cde0 con 0x7fef9c100e20 2024-09-19T06:17:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.111+0000 7fef99ffb640 1 -- 172.21.15.5:0/458616332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef8c0035f0 con 0x7fef9c100e20 2024-09-19T06:17:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.111+0000 7fefa22a5640 1 -- 172.21.15.5:0/458616332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9c19c590 con 0x7fef9c100e20 2024-09-19T06:17:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.111+0000 7fefa22a5640 1 -- 172.21.15.5:0/458616332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9c069f70 con 0x7fef9c100e20 2024-09-19T06:17:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.111+0000 7fef99ffb640 1 -- 172.21.15.5:0/458616332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c010690 con 0x7fef9c100e20 2024-09-19T06:17:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa467e740 1 Processor -- start 2024-09-19T06:17:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa467e740 1 -- start start 2024-09-19T06:17:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fef99ffb640 1 -- 172.21.15.5:0/458616332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fef8c010830 con 0x7fef9c100e20 2024-09-19T06:17:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa467e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa873b40 0x5585aa872090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa467e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa8725d0 0x5585aa872a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa467e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa87d370 0x5585aa87f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa467e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585aa7d82f0 con 0x5585aa8725d0 2024-09-19T06:17:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa467e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5585aa77fcf0 con 0x5585aa873b40 2024-09-19T06:17:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa467e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5585aa7d42b0 con 0x5585aa87d370 2024-09-19T06:17:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa0842640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa873b40 0x5585aa872090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa0842640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa873b40 0x5585aa872090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40318/0 (socket says 172.21.15.5:40318) 2024-09-19T06:17:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa0842640 1 -- 172.21.15.5:0/3942532536 learned_addr learned my addr 172.21.15.5:0/3942532536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fef93fff640 1 --2- 172.21.15.5:0/3942532536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa87d370 0x5585aa87f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fef9bfff640 1 --2- 172.21.15.5:0/3942532536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa8725d0 0x5585aa872a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa0842640 1 -- 172.21.15.5:0/3942532536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa87d370 msgr2=0x5585aa87f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa0842640 1 --2- 172.21.15.5:0/3942532536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa87d370 0x5585aa87f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.115+0000 7fefa0842640 1 -- 172.21.15.5:0/3942532536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa8725d0 msgr2=0x5585aa872a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.116+0000 7fefa0842640 1 --2- 172.21.15.5:0/3942532536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa8725d0 0x5585aa872a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.116+0000 7fefa0842640 1 -- 172.21.15.5:0/3942532536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585aa7643a0 con 0x5585aa873b40 2024-09-19T06:17:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.116+0000 7fef9bfff640 1 --2- 172.21.15.5:0/3942532536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa8725d0 0x5585aa872a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:17:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.116+0000 7fef99ffb640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fef880a34b0 0x7fef880a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.116+0000 7fef9bfff640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fef880a34b0 0x7fef880a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.116+0000 7fef99ffb640 1 -- 172.21.15.5:0/458616332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7fef8c0d0100 con 0x7fef9c100e20 2024-09-19T06:17:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.116+0000 7fefa0842640 1 --2- 172.21.15.5:0/3942532536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa873b40 0x5585aa872090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fef94009be0 tx=0x7fef940039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1587dc6ebf3a191 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.116+0000 7fef927fc640 1 -- 172.21.15.5:0/3942532536 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94047070 con 0x5585aa873b40 2024-09-19T06:17:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.117+0000 7fef927fc640 1 -- 172.21.15.5:0/3942532536 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef94037040 con 0x5585aa873b40 2024-09-19T06:17:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.117+0000 7fef927fc640 1 -- 172.21.15.5:0/3942532536 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94036040 con 0x5585aa873b40 2024-09-19T06:17:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.117+0000 7fefa467e740 1 -- 172.21.15.5:0/3942532536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa873b40 msgr2=0x5585aa872090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.117+0000 7fef9bfff640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fef880a34b0 0x7fef880a5970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5585aa86fac0 tx=0x7fef8c003e10 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.117+0000 7fefa467e740 1 --2- 172.21.15.5:0/3942532536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa873b40 0x5585aa872090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fef94009be0 tx=0x7fef940039e0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.118+0000 7fefa467e740 1 -- 172.21.15.5:0/3942532536 shutdown_connections 2024-09-19T06:17:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.118+0000 7fefa467e740 1 --2- 172.21.15.5:0/3942532536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa87d370 0x5585aa87f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.118+0000 7fefa467e740 1 --2- 172.21.15.5:0/3942532536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa8725d0 0x5585aa872a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.118+0000 7fefa467e740 1 --2- 172.21.15.5:0/3942532536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa873b40 0x5585aa872090 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.118+0000 7fefa467e740 1 -- 172.21.15.5:0/3942532536 >> 172.21.15.5:0/3942532536 conn(0x5585aa83b310 msgr2=0x5585aa86ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.118+0000 7fefa467e740 1 -- 172.21.15.5:0/3942532536 shutdown_connections 2024-09-19T06:17:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.118+0000 7fefa467e740 1 -- 172.21.15.5:0/3942532536 wait complete. 2024-09-19T06:17:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.118+0000 7fefa467e740 1 -- start start 2024-09-19T06:17:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fefa467e740 1 Processor -- start 2024-09-19T06:17:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fefa467e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa8725d0 0x5585aa87cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fefa467e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa873b40 0x5585aa87a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fefa467e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa87d370 0x5585aa87a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fefa467e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585aa880060 con 0x5585aa873b40 2024-09-19T06:17:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fefa0842640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa8725d0 0x5585aa87cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fefa0842640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa8725d0 0x5585aa87cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:39626/0 (socket says 172.21.15.5:39626) 2024-09-19T06:17:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fefa0842640 1 -- 172.21.15.5:0/2375675589 learned_addr learned my addr 172.21.15.5:0/2375675589 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fef93fff640 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa873b40 0x5585aa87a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fefa467e740 1 -- 172.21.15.5:0/2375675589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5585aa793060 con 0x5585aa87d370 2024-09-19T06:17:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fefa467e740 1 -- 172.21.15.5:0/2375675589 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5585aa8801e0 con 0x5585aa8725d0 2024-09-19T06:17:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fefa467e740 10 client.0 _ll_register_callbacks cb 0x5585aa4e68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.119+0000 7fefa0842640 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa87d370 0x5585aa87a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.120+0000 7fefa0842640 1 -- 172.21.15.5:0/2375675589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa87d370 msgr2=0x5585aa87a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.120+0000 7fefa0842640 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa87d370 0x5585aa87a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.120+0000 7fefa0842640 1 -- 172.21.15.5:0/2375675589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa873b40 msgr2=0x5585aa87a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.120+0000 7fefa0842640 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa873b40 0x5585aa87a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.120+0000 7fefa0842640 1 -- 172.21.15.5:0/2375675589 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585aa5493c0 con 0x5585aa8725d0 2024-09-19T06:17:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.120+0000 7fef93fff640 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa873b40 0x5585aa87a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.120+0000 7fefa467e740 1 -- 172.21.15.5:0/2375675589 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef94037040 con 0x5585aa8725d0 2024-09-19T06:17:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.120+0000 7fefa0842640 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa8725d0 0x5585aa87cd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fef94009560 tx=0x7fef94004230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a453b6a056a95fff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.120+0000 7fef7b7fe640 10 client.28534 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:17:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.121+0000 7fef7b7fe640 1 -- 172.21.15.5:0/2375675589 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94054060 con 0x5585aa8725d0 2024-09-19T06:17:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.121+0000 7fef7b7fe640 1 -- 172.21.15.5:0/2375675589 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7fef94004310 con 0x5585aa8725d0 2024-09-19T06:17:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.121+0000 7fef7b7fe640 1 -- 172.21.15.5:0/2375675589 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fef94036070 con 0x5585aa8725d0 2024-09-19T06:17:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.121+0000 7fefa467e740 10 client.28534 fetch_fsmap learned FSMap version 44 2024-09-19T06:17:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.121+0000 7fefa467e740 1 -- 172.21.15.5:0/2375675589 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=44}) -- 0x5585aa880060 con 0x5585aa8725d0 2024-09-19T06:17:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.121+0000 7fef7b7fe640 1 -- 172.21.15.5:0/2375675589 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 44) ==== 8402+0+0 (secure 0 0 0) 0x7fef94041030 con 0x5585aa8725d0 2024-09-19T06:17:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.121+0000 7fefa467e740 10 client.28534 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T06:17:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.121+0000 7fefa467e740 10 client.28534 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:17:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.121+0000 7fefa467e740 20 client.28534 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.121+0000 7fefa467e740 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5585aa8952a0 0x5585aa897690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.121+0000 7fefa467e740 4 client.28534 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:17:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.121+0000 7fefa467e740 1 -- 172.21.15.5:0/2375675589 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585aa7e5860 con 0x5585aa8952a0 2024-09-19T06:17:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.122+0000 7fef93fff640 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5585aa8952a0 0x5585aa897690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.122+0000 7fef93fff640 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5585aa8952a0 0x5585aa897690 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ef4708d88429b0c server_cookie=d98a7680bcaeba9 in_seq=0 out_seq=0 2024-09-19T06:17:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.122+0000 7fef7b7fe640 10 client.28534 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:17:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.123+0000 7fef7b7fe640 1 -- 172.21.15.5:0/2375675589 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585aa7e5860 con 0x5585aa8952a0 2024-09-19T06:17:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.123+0000 7fef7b7fe640 10 client.28534 handle_command_reply: tid=0 2024-09-19T06:17:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.123+0000 7fefa467e740 1 client.28534 shutdown 2024-09-19T06:17:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.123+0000 7fefa467e740 1 -- 172.21.15.5:0/2375675589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa8725d0 msgr2=0x5585aa87cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.123+0000 7fefa467e740 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa8725d0 0x5585aa87cd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fef94009560 tx=0x7fef94004230 comp rx=0 tx=0).stop 2024-09-19T06:17:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.123+0000 7fefa467e740 1 -- 172.21.15.5:0/2375675589 shutdown_connections 2024-09-19T06:17:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.123+0000 7fefa467e740 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5585aa8952a0 0x5585aa897690 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.123+0000 7fefa467e740 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa87d370 0x5585aa87a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.123+0000 7fefa467e740 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa873b40 0x5585aa87a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.123+0000 7fefa467e740 1 --2- 172.21.15.5:0/2375675589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa8725d0 0x5585aa87cd50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.123+0000 7fef7b7fe640 0 client.28534 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:17:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.123+0000 7fefa467e740 1 -- 172.21.15.5:0/2375675589 >> 172.21.15.5:0/2375675589 conn(0x5585aa83b310 msgr2=0x5585aa874930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.124+0000 7fefa467e740 1 -- 172.21.15.5:0/2375675589 shutdown_connections 2024-09-19T06:17:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.124+0000 7fefa467e740 1 -- 172.21.15.5:0/2375675589 wait complete. 2024-09-19T06:17:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.124+0000 7fefa467e740 20 client.28534 trim_cache size 0 max 16384 2024-09-19T06:17:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.140+0000 7fefa467e740 1 Processor -- start 2024-09-19T06:17:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.140+0000 7fefa467e740 1 -- start start 2024-09-19T06:17:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fefa467e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa8725d0 0x5585aa86dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fefa467e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa873b40 0x5585aa86e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fefa467e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa87d370 0x5585aa875120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fefa467e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1ab520 con 0x5585aa8725d0 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fefa467e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1ad1e0 con 0x5585aa87d370 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fefa467e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1abc50 con 0x5585aa873b40 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fef93fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa873b40 0x5585aa86e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fef93fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa873b40 0x5585aa86e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:39640/0 (socket says 172.21.15.5:39640) 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fef93fff640 1 -- 172.21.15.5:0/172478648 learned_addr learned my addr 172.21.15.5:0/172478648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fefa0842640 1 --2- 172.21.15.5:0/172478648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa8725d0 0x5585aa86dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fefa0842640 1 --2- 172.21.15.5:0/172478648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa87d370 0x5585aa875120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fef93fff640 1 -- 172.21.15.5:0/172478648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa87d370 msgr2=0x5585aa875120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fef93fff640 1 --2- 172.21.15.5:0/172478648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa87d370 0x5585aa875120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fef93fff640 1 -- 172.21.15.5:0/172478648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa8725d0 msgr2=0x5585aa86dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fef93fff640 1 --2- 172.21.15.5:0/172478648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa8725d0 0x5585aa86dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fef93fff640 1 -- 172.21.15.5:0/172478648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585aa86f730 con 0x5585aa873b40 2024-09-19T06:17:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fefa0842640 1 --2- 172.21.15.5:0/172478648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa87d370 0x5585aa875120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:17:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.141+0000 7fef93fff640 1 --2- 172.21.15.5:0/172478648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa873b40 0x5585aa86e130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5585aa879890 tx=0x7fef840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cefd1edadbf8611d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.142+0000 7fef91ffb640 1 -- 172.21.15.5:0/172478648 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84004e00 con 0x5585aa873b40 2024-09-19T06:17:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.142+0000 7fef91ffb640 1 -- 172.21.15.5:0/172478648 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef840096c0 con 0x5585aa873b40 2024-09-19T06:17:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.142+0000 7fef91ffb640 1 -- 172.21.15.5:0/172478648 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84003850 con 0x5585aa873b40 2024-09-19T06:17:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.142+0000 7fefa467e740 1 -- 172.21.15.5:0/172478648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa873b40 msgr2=0x5585aa86e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.142+0000 7fefa467e740 1 --2- 172.21.15.5:0/172478648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa873b40 0x5585aa86e130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5585aa879890 tx=0x7fef840046c0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.143+0000 7fefa467e740 1 -- 172.21.15.5:0/172478648 shutdown_connections 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.143+0000 7fefa467e740 1 --2- 172.21.15.5:0/172478648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa87d370 0x5585aa875120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.143+0000 7fefa467e740 1 --2- 172.21.15.5:0/172478648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa873b40 0x5585aa86e130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.143+0000 7fefa467e740 1 --2- 172.21.15.5:0/172478648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa8725d0 0x5585aa86dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.143+0000 7fefa467e740 1 -- 172.21.15.5:0/172478648 >> 172.21.15.5:0/172478648 conn(0x5585aa83b310 msgr2=0x5585aa86d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.143+0000 7fefa467e740 1 -- 172.21.15.5:0/172478648 shutdown_connections 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.143+0000 7fefa467e740 1 -- 172.21.15.5:0/172478648 wait complete. 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.143+0000 7fefa467e740 1 -- start start 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.144+0000 7fefa467e740 1 Processor -- start 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.144+0000 7fefa467e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa8725d0 0x5585aa88e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.144+0000 7fefa467e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa873b40 0x5585aa889710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.144+0000 7fefa467e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa87d370 0x5585aa889c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.144+0000 7fefa467e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1acb10 con 0x5585aa873b40 2024-09-19T06:17:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.144+0000 7fefa0842640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa8725d0 0x5585aa88e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.144+0000 7fefa467e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1ac380 con 0x5585aa8725d0 2024-09-19T06:17:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.144+0000 7fefa467e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1a9570 con 0x5585aa87d370 2024-09-19T06:17:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.144+0000 7fefa0842640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa8725d0 0x5585aa88e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40342/0 (socket says 172.21.15.5:40342) 2024-09-19T06:17:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.144+0000 7fefa0842640 1 -- 172.21.15.5:0/3421705581 learned_addr learned my addr 172.21.15.5:0/3421705581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.144+0000 7fefa467e740 10 client.0 _ll_register_callbacks cb 0x7fef94045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.144+0000 7fef93fff640 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa873b40 0x5585aa889710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.145+0000 7fefa0842640 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa87d370 0x5585aa889c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.145+0000 7fefa0842640 1 -- 172.21.15.5:0/3421705581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa87d370 msgr2=0x5585aa889c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.145+0000 7fefa0842640 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa87d370 0x5585aa889c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.145+0000 7fefa0842640 1 -- 172.21.15.5:0/3421705581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa873b40 msgr2=0x5585aa889710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.145+0000 7fefa0842640 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa873b40 0x5585aa889710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.145+0000 7fefa0842640 1 -- 172.21.15.5:0/3421705581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585aa88a480 con 0x5585aa8725d0 2024-09-19T06:17:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.145+0000 7fefa0842640 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa8725d0 0x5585aa88e3e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fef9402ac50 tx=0x7fef9403d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85bd5b380e6c5aea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.145+0000 7fef7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:17:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.145+0000 7fef7b7fe640 1 -- 172.21.15.5:0/3421705581 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94047040 con 0x5585aa8725d0 2024-09-19T06:17:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.145+0000 7fefa467e740 1 -- 172.21.15.5:0/3421705581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef840096c0 con 0x5585aa8725d0 2024-09-19T06:17:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.146+0000 7fef7b7fe640 1 -- 172.21.15.5:0/3421705581 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7fef94030dd0 con 0x5585aa8725d0 2024-09-19T06:17:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.146+0000 7fef7b7fe640 1 -- 172.21.15.5:0/3421705581 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fef94066050 con 0x5585aa8725d0 2024-09-19T06:17:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.146+0000 7fefa467e740 10 client.28595 fetch_fsmap learned FSMap version 44 2024-09-19T06:17:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.146+0000 7fefa467e740 1 -- 172.21.15.5:0/3421705581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5585aa8ffd60 con 0x5585aa8725d0 2024-09-19T06:17:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.146+0000 7fef7b7fe640 1 -- 172.21.15.5:0/3421705581 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef840096c0 con 0x5585aa8725d0 2024-09-19T06:17:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.146+0000 7fef7b7fe640 1 -- 172.21.15.5:0/3421705581 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 44) ==== 8402+0+0 (secure 0 0 0) 0x7fef94041070 con 0x5585aa8725d0 2024-09-19T06:17:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.146+0000 7fefa467e740 10 client.28595 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T06:17:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.146+0000 7fefa467e740 10 client.28595 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:17:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.146+0000 7fefa467e740 20 client.28595 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.146+0000 7fefa467e740 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5585aa9003f0 0x5585aa902810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.147+0000 7fefa467e740 4 client.28595 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:17:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.147+0000 7fefa467e740 1 -- 172.21.15.5:0/3421705581 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5585aa7dfd70 con 0x5585aa9003f0 2024-09-19T06:17:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.147+0000 7fef93fff640 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5585aa9003f0 0x5585aa902810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.147+0000 7fef93fff640 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5585aa9003f0 0x5585aa902810 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a89d53643f29010d server_cookie=bcd99db8f2a01161 in_seq=0 out_seq=0 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.147+0000 7fef7b7fe640 10 client.28595 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.148+0000 7fef7b7fe640 1 -- 172.21.15.5:0/3421705581 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5585aa7dfd70 con 0x5585aa9003f0 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.148+0000 7fef7b7fe640 10 client.28595 handle_command_reply: tid=0 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.148+0000 7fefa467e740 1 client.28595 shutdown 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.148+0000 7fefa467e740 1 -- 172.21.15.5:0/3421705581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa8725d0 msgr2=0x5585aa88e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.148+0000 7fefa467e740 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa8725d0 0x5585aa88e3e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fef9402ac50 tx=0x7fef9403d040 comp rx=0 tx=0).stop 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.149+0000 7fefa467e740 1 -- 172.21.15.5:0/3421705581 shutdown_connections 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.149+0000 7fefa467e740 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5585aa9003f0 0x5585aa902810 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.149+0000 7fefa467e740 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585aa87d370 0x5585aa889c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.149+0000 7fefa467e740 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585aa873b40 0x5585aa889710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.149+0000 7fefa467e740 1 --2- 172.21.15.5:0/3421705581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5585aa8725d0 0x5585aa88e3e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.149+0000 7fefa467e740 1 -- 172.21.15.5:0/3421705581 >> 172.21.15.5:0/3421705581 conn(0x5585aa83b310 msgr2=0x5585aa86dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.149+0000 7fef7b7fe640 0 client.28595 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:17:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.149+0000 7fefa467e740 1 -- 172.21.15.5:0/3421705581 shutdown_connections 2024-09-19T06:17:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.149+0000 7fefa467e740 1 -- 172.21.15.5:0/3421705581 wait complete. 2024-09-19T06:17:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.149+0000 7fefa467e740 20 client.28595 trim_cache size 0 max 16384 2024-09-19T06:17:35.154 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:17:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.150+0000 7fefa22a5640 1 -- 172.21.15.5:0/458616332 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fef880a34b0 msgr2=0x7fef880a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.150+0000 7fefa22a5640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fef880a34b0 0x7fef880a5970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5585aa86fac0 tx=0x7fef8c003e10 comp rx=0 tx=0).stop 2024-09-19T06:17:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.150+0000 7fefa22a5640 1 -- 172.21.15.5:0/458616332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef9c100e20 msgr2=0x7fef9c1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.150+0000 7fefa22a5640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef9c100e20 0x7fef9c1972f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00e9c0 tx=0x7fef8c00ee80 comp rx=0 tx=0).stop 2024-09-19T06:17:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.151+0000 7fefa22a5640 1 -- 172.21.15.5:0/458616332 shutdown_connections 2024-09-19T06:17:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.151+0000 7fefa22a5640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fef880a34b0 0x7fef880a5970 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.151+0000 7fefa22a5640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9c102ef0 0x7fef9c19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.151+0000 7fefa22a5640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c102570 0x7fef9c197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.151+0000 7fefa22a5640 1 --2- 172.21.15.5:0/458616332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef9c100e20 0x7fef9c1972f0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.151+0000 7fefa22a5640 1 -- 172.21.15.5:0/458616332 >> 172.21.15.5:0/458616332 conn(0x7fef9c0fc7d0 msgr2=0x7fef9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.151+0000 7fefa22a5640 1 -- 172.21.15.5:0/458616332 shutdown_connections 2024-09-19T06:17:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:35.151+0000 7fefa22a5640 1 -- 172.21.15.5:0/458616332 wait complete. 2024-09-19T06:17:35.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 10104 12.16 MB/sec warmup 59 sec latency 65.560 ms 2024-09-19T06:17:35.921 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:17:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:17:36.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 10237 12.06 MB/sec warmup 60 sec latency 61.793 ms 2024-09-19T06:17:36.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:36 smithi005 ceph-mon[30955]: pgmap v447: 97 pgs: 97 active+clean; 447 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 3.7 MiB/s wr, 237 op/s 2024-09-19T06:17:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:36 smithi157 ceph-mon[35346]: pgmap v447: 97 pgs: 97 active+clean; 447 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 3.7 MiB/s wr, 237 op/s 2024-09-19T06:17:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:36 smithi049 ceph-mon[34640]: pgmap v447: 97 pgs: 97 active+clean; 447 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 3.7 MiB/s wr, 237 op/s 2024-09-19T06:17:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.909+0000 7f068a908640 1 -- 172.21.15.5:0/949482622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0684102d30 msgr2=0x7f0684103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.909+0000 7f068a908640 1 --2- 172.21.15.5:0/949482622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0684102d30 0x7f0684103110 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f0674009930 tx=0x7f067402a420 comp rx=0 tx=0).stop 2024-09-19T06:17:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.909+0000 7f068a908640 1 -- 172.21.15.5:0/949482622 shutdown_connections 2024-09-19T06:17:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.909+0000 7f068a908640 1 --2- 172.21.15.5:0/949482622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06841082a0 0x7f068410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.909+0000 7f068a908640 1 --2- 172.21.15.5:0/949482622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06841036e0 0x7f0684107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.909+0000 7f068a908640 1 --2- 172.21.15.5:0/949482622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0684102d30 0x7f0684103110 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.909+0000 7f068a908640 1 -- 172.21.15.5:0/949482622 >> 172.21.15.5:0/949482622 conn(0x7f06840fc7d0 msgr2=0x7f06840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.910+0000 7f068a908640 1 -- 172.21.15.5:0/949482622 shutdown_connections 2024-09-19T06:17:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.910+0000 7f068a908640 1 -- 172.21.15.5:0/949482622 wait complete. 2024-09-19T06:17:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.910+0000 7f068a908640 1 Processor -- start 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.910+0000 7f068a908640 1 -- start start 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f068a908640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0684102d30 0x7f0684100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f068a908640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06841036e0 0x7f0684100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f068a908640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06841082a0 0x7f0684101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f068a908640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0684111f90 con 0x7f06841036e0 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f068a908640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0684111e10 con 0x7f06841082a0 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f068a908640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0684112110 con 0x7f0684102d30 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f0683fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0684102d30 0x7f0684100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f06837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06841036e0 0x7f0684100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f0683fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0684102d30 0x7f0684100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:39658/0 (socket says 172.21.15.5:39658) 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f0683fff640 1 -- 172.21.15.5:0/4158431222 learned_addr learned my addr 172.21.15.5:0/4158431222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f0688ea5640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06841082a0 0x7f0684101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f06837fe640 1 -- 172.21.15.5:0/4158431222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0684102d30 msgr2=0x7f0684100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.911+0000 7f06837fe640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0684102d30 0x7f0684100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.912+0000 7f06837fe640 1 -- 172.21.15.5:0/4158431222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06841082a0 msgr2=0x7f0684101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.912+0000 7f06837fe640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06841082a0 0x7f0684101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.912+0000 7f06837fe640 1 -- 172.21.15.5:0/4158431222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06841a69c0 con 0x7f06841036e0 2024-09-19T06:17:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.912+0000 7f0688ea5640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06841082a0 0x7f0684101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:17:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.912+0000 7f06837fe640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06841036e0 0x7f0684100ed0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f06840697d0 tx=0x7f067800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.912+0000 7f06817fa640 1 -- 172.21.15.5:0/4158431222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067800cd00 con 0x7f06841036e0 2024-09-19T06:17:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.912+0000 7f068a908640 1 -- 172.21.15.5:0/4158431222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06841a6cb0 con 0x7f06841036e0 2024-09-19T06:17:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.912+0000 7f06817fa640 1 -- 172.21.15.5:0/4158431222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06780035f0 con 0x7f06841036e0 2024-09-19T06:17:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.912+0000 7f068a908640 1 -- 172.21.15.5:0/4158431222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06841a71e0 con 0x7f06841036e0 2024-09-19T06:17:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.913+0000 7f06817fa640 1 -- 172.21.15.5:0/4158431222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067800cd00 con 0x7f06841036e0 2024-09-19T06:17:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.915+0000 7f068cce1740 1 Processor -- start 2024-09-19T06:17:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.915+0000 7f068cce1740 1 -- start start 2024-09-19T06:17:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f068cce1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b103ab40 0x5621b1039090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f068cce1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 0x5621b1039a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f068cce1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b104cb80 0x5621b104ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f068cce1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5621b0f9f2f0 con 0x5621b103ab40 2024-09-19T06:17:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f068cce1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5621b0f46cf0 con 0x5621b10395d0 2024-09-19T06:17:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f0683fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 0x5621b1039a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f0683fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 0x5621b1039a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40370/0 (socket says 172.21.15.5:40370) 2024-09-19T06:17:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f0683fff640 1 -- 172.21.15.5:0/1539460592 learned_addr learned my addr 172.21.15.5:0/1539460592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f068cce1740 1 -- 172.21.15.5:0/1539460592 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5621b0f9b2b0 con 0x5621b104cb80 2024-09-19T06:17:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f0688ea5640 1 --2- 172.21.15.5:0/1539460592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b103ab40 0x5621b1039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f0683fff640 1 -- 172.21.15.5:0/1539460592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b104cb80 msgr2=0x5621b104ef70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:17:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f0683fff640 1 --2- 172.21.15.5:0/1539460592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b104cb80 0x5621b104ef70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f0683fff640 1 -- 172.21.15.5:0/1539460592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b103ab40 msgr2=0x5621b1039090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f0683fff640 1 --2- 172.21.15.5:0/1539460592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b103ab40 0x5621b1039090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f0683fff640 1 -- 172.21.15.5:0/1539460592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b0f2b3a0 con 0x5621b10395d0 2024-09-19T06:17:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.916+0000 7f06817fa640 1 -- 172.21.15.5:0/4158431222 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f06780037a0 con 0x7f06841036e0 2024-09-19T06:17:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.917+0000 7f0683fff640 1 --2- 172.21.15.5:0/1539460592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 0x5621b1039a10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0674009850 tx=0x7f0674004140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7f503e38dd35425 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.917+0000 7f066a7fc640 1 -- 172.21.15.5:0/1539460592 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674035040 con 0x5621b10395d0 2024-09-19T06:17:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.917+0000 7f066a7fc640 1 -- 172.21.15.5:0/1539460592 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0674004460 con 0x5621b10395d0 2024-09-19T06:17:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.917+0000 7f066a7fc640 1 -- 172.21.15.5:0/1539460592 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06740035f0 con 0x5621b10395d0 2024-09-19T06:17:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.917+0000 7f06817fa640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f066c0a35f0 0x7f066c0a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.918+0000 7f0688ea5640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f066c0a35f0 0x7f066c0a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.918+0000 7f06817fa640 1 -- 172.21.15.5:0/4158431222 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f06780cfc40 con 0x7f06841036e0 2024-09-19T06:17:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.918+0000 7f0688ea5640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f066c0a35f0 0x7f066c0a5ab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f067c005fd0 tx=0x7f067c009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.918+0000 7f068cce1740 1 -- 172.21.15.5:0/1539460592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 msgr2=0x5621b1039a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.918+0000 7f068cce1740 1 --2- 172.21.15.5:0/1539460592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 0x5621b1039a10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0674009850 tx=0x7f0674004140 comp rx=0 tx=0).stop 2024-09-19T06:17:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.918+0000 7f068cce1740 1 -- 172.21.15.5:0/1539460592 shutdown_connections 2024-09-19T06:17:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.918+0000 7f068cce1740 1 --2- 172.21.15.5:0/1539460592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b104cb80 0x5621b104ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.918+0000 7f068cce1740 1 --2- 172.21.15.5:0/1539460592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 0x5621b1039a10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.918+0000 7f068cce1740 1 --2- 172.21.15.5:0/1539460592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b103ab40 0x5621b1039090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.918+0000 7f068cce1740 1 -- 172.21.15.5:0/1539460592 >> 172.21.15.5:0/1539460592 conn(0x5621b1002310 msgr2=0x5621b1035c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.918+0000 7f068cce1740 1 -- 172.21.15.5:0/1539460592 shutdown_connections 2024-09-19T06:17:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.918+0000 7f068cce1740 1 -- 172.21.15.5:0/1539460592 wait complete. 2024-09-19T06:17:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.919+0000 7f068cce1740 1 -- start start 2024-09-19T06:17:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.919+0000 7f068cce1740 1 Processor -- start 2024-09-19T06:17:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.919+0000 7f068cce1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b10395d0 0x5621b104c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.919+0000 7f068cce1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b103ab40 0x5621b1049890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.919+0000 7f068cce1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b104a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.919+0000 7f068cce1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5621b104f820 con 0x5621b104cb80 2024-09-19T06:17:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.919+0000 7f068cce1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5621b0f5a060 con 0x5621b103ab40 2024-09-19T06:17:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f068cce1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5621b104f9a0 con 0x5621b10395d0 2024-09-19T06:17:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f068cce1740 10 client.0 _ll_register_callbacks cb 0x5621b0cad8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f06837fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b103ab40 0x5621b1049890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f06837fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b103ab40 0x5621b1049890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40378/0 (socket says 172.21.15.5:40378) 2024-09-19T06:17:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f06837fe640 1 -- 172.21.15.5:0/4013885893 learned_addr learned my addr 172.21.15.5:0/4013885893 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f0683fff640 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b10395d0 0x5621b104c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f0688ea5640 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b104a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f06837fe640 1 -- 172.21.15.5:0/4013885893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b10395d0 msgr2=0x5621b104c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f06837fe640 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b10395d0 0x5621b104c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f06837fe640 1 -- 172.21.15.5:0/4013885893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 msgr2=0x5621b104a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f06837fe640 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b104a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f06837fe640 1 -- 172.21.15.5:0/4013885893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b0d103c0 con 0x5621b103ab40 2024-09-19T06:17:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f06837fe640 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b103ab40 0x5621b1049890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f06780cd970 tx=0x7f067809bb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5279c54a27294096 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.920+0000 7f06637fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:17:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.921+0000 7f06637fe640 1 -- 172.21.15.5:0/4013885893 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06780b4020 con 0x5621b103ab40 2024-09-19T06:17:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.921+0000 7f068cce1740 1 -- 172.21.15.5:0/4013885893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0674004460 con 0x5621b103ab40 2024-09-19T06:17:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.921+0000 7f06637fe640 1 -- 172.21.15.5:0/4013885893 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f06780bab40 con 0x5621b103ab40 2024-09-19T06:17:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.921+0000 7f06637fe640 1 -- 172.21.15.5:0/4013885893 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f067809bf00 con 0x5621b103ab40 2024-09-19T06:17:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.921+0000 7f068cce1740 10 client.28607 fetch_fsmap learned FSMap version 44 2024-09-19T06:17:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.921+0000 7f068cce1740 1 -- 172.21.15.5:0/4013885893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5621b105b370 con 0x5621b103ab40 2024-09-19T06:17:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.921+0000 7f06637fe640 1 -- 172.21.15.5:0/4013885893 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0674004460 con 0x5621b103ab40 2024-09-19T06:17:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.921+0000 7f06637fe640 1 -- 172.21.15.5:0/4013885893 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 44) ==== 8402+0+0 (secure 0 0 0) 0x7f067807aae0 con 0x5621b103ab40 2024-09-19T06:17:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.921+0000 7f068cce1740 10 client.28607 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T06:17:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.922+0000 7f068cce1740 10 client.28607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.922+0000 7f068cce1740 20 client.28607 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.922+0000 7f068cce1740 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5621b105c950 0x5621b105ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.922+0000 7f068cce1740 4 client.28607 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:17:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.922+0000 7f068cce1740 1 -- 172.21.15.5:0/4013885893 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621b0fac860 con 0x5621b105c950 2024-09-19T06:17:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.922+0000 7f0683fff640 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5621b105c950 0x5621b105ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.922+0000 7f06637fe640 10 client.28607 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.922+0000 7f0683fff640 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5621b105c950 0x5621b105ed70 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee72a298965ad04e server_cookie=581f806a97d2ff46 in_seq=0 out_seq=0 2024-09-19T06:17:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f06637fe640 1 -- 172.21.15.5:0/4013885893 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621b0fac860 con 0x5621b105c950 2024-09-19T06:17:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f06637fe640 10 client.28607 handle_command_reply: tid=0 2024-09-19T06:17:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f068cce1740 1 client.28607 shutdown 2024-09-19T06:17:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f068cce1740 1 -- 172.21.15.5:0/4013885893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b103ab40 msgr2=0x5621b1049890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f068cce1740 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b103ab40 0x5621b1049890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f06780cd970 tx=0x7f067809bb30 comp rx=0 tx=0).stop 2024-09-19T06:17:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f068cce1740 1 -- 172.21.15.5:0/4013885893 shutdown_connections 2024-09-19T06:17:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f068cce1740 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5621b105c950 0x5621b105ed70 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f068cce1740 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b104a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f068cce1740 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b103ab40 0x5621b1049890 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f068cce1740 1 --2- 172.21.15.5:0/4013885893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b10395d0 0x5621b104c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f068cce1740 1 -- 172.21.15.5:0/4013885893 >> 172.21.15.5:0/4013885893 conn(0x5621b1002310 msgr2=0x5621b103b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f06637fe640 0 client.28607 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f068cce1740 1 -- 172.21.15.5:0/4013885893 shutdown_connections 2024-09-19T06:17:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.923+0000 7f068cce1740 1 -- 172.21.15.5:0/4013885893 wait complete. 2024-09-19T06:17:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.924+0000 7f068cce1740 20 client.28607 trim_cache size 0 max 16384 2024-09-19T06:17:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.945+0000 7f068cce1740 1 Processor -- start 2024-09-19T06:17:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.945+0000 7f068cce1740 1 -- start start 2024-09-19T06:17:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f068cce1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b10395d0 0x5621b1049150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f068cce1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b103ab40 0x5621b1034bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f068cce1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b1064600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f068cce1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06841ab6b0 con 0x5621b104cb80 2024-09-19T06:17:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f068cce1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f06841ad370 con 0x5621b103ab40 2024-09-19T06:17:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f068cce1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f06841abde0 con 0x5621b10395d0 2024-09-19T06:17:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f0688ea5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b1064600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f0683fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b10395d0 0x5621b1049150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f06837fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b103ab40 0x5621b1034bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f0683fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b10395d0 0x5621b1049150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:39670/0 (socket says 172.21.15.5:39670) 2024-09-19T06:17:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f0683fff640 1 -- 172.21.15.5:0/836490168 learned_addr learned my addr 172.21.15.5:0/836490168 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f0683fff640 1 -- 172.21.15.5:0/836490168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b103ab40 msgr2=0x5621b1034bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f0683fff640 1 --2- 172.21.15.5:0/836490168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b103ab40 0x5621b1034bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f0683fff640 1 -- 172.21.15.5:0/836490168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 msgr2=0x5621b1064600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f0683fff640 1 --2- 172.21.15.5:0/836490168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b1064600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f0683fff640 1 -- 172.21.15.5:0/836490168 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b105f3d0 con 0x5621b10395d0 2024-09-19T06:17:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f0688ea5640 1 --2- 172.21.15.5:0/836490168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b1064600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.946+0000 7f0683fff640 1 --2- 172.21.15.5:0/836490168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b10395d0 0x5621b1049150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0674009850 tx=0x7f0674009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f36fed16a76748 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.947+0000 7f0669ffb640 1 -- 172.21.15.5:0/836490168 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674035040 con 0x5621b10395d0 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.947+0000 7f0669ffb640 1 -- 172.21.15.5:0/836490168 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0674003aa0 con 0x5621b10395d0 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.947+0000 7f0669ffb640 1 -- 172.21.15.5:0/836490168 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674031b10 con 0x5621b10395d0 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.948+0000 7f068cce1740 1 -- 172.21.15.5:0/836490168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b10395d0 msgr2=0x5621b1049150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.948+0000 7f068cce1740 1 --2- 172.21.15.5:0/836490168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b10395d0 0x5621b1049150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0674009850 tx=0x7f0674009260 comp rx=0 tx=0).stop 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.948+0000 7f068cce1740 1 -- 172.21.15.5:0/836490168 shutdown_connections 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.948+0000 7f068cce1740 1 --2- 172.21.15.5:0/836490168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b1064600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.948+0000 7f068cce1740 1 --2- 172.21.15.5:0/836490168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b103ab40 0x5621b1034bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.948+0000 7f068cce1740 1 --2- 172.21.15.5:0/836490168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b10395d0 0x5621b1049150 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.948+0000 7f068cce1740 1 -- 172.21.15.5:0/836490168 >> 172.21.15.5:0/836490168 conn(0x5621b1002310 msgr2=0x5621b10344e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.948+0000 7f068cce1740 1 -- 172.21.15.5:0/836490168 shutdown_connections 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.948+0000 7f068cce1740 1 -- 172.21.15.5:0/836490168 wait complete. 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.948+0000 7f068cce1740 1 -- start start 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.948+0000 7f068cce1740 1 Processor -- start 2024-09-19T06:17:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f068cce1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 0x5621b1050300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f068cce1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b103ab40 0x5621b1050840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f068cce1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b10bf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f068cce1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06841acca0 con 0x5621b104cb80 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f068cce1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f06841ac510 con 0x5621b10395d0 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f068cce1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f06841a96b0 con 0x5621b103ab40 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f068cce1740 10 client.0 _ll_register_callbacks cb 0x7f0678009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f0683fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 0x5621b1050300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f06837fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b103ab40 0x5621b1050840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f06837fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b103ab40 0x5621b1050840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:39678/0 (socket says 172.21.15.5:39678) 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f0683fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 0x5621b1050300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40398/0 (socket says 172.21.15.5:40398) 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f06837fe640 1 -- 172.21.15.5:0/844834736 learned_addr learned my addr 172.21.15.5:0/844834736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f0688ea5640 1 --2- 172.21.15.5:0/844834736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b10bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f06837fe640 1 -- 172.21.15.5:0/844834736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 msgr2=0x5621b1050300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f06837fe640 1 --2- 172.21.15.5:0/844834736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 0x5621b1050300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f06837fe640 1 -- 172.21.15.5:0/844834736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 msgr2=0x5621b10bf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f06837fe640 1 --2- 172.21.15.5:0/844834736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b10bf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.949+0000 7f06837fe640 1 -- 172.21.15.5:0/844834736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b10bfcf0 con 0x5621b103ab40 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.950+0000 7f06837fe640 1 --2- 172.21.15.5:0/844834736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b103ab40 0x5621b1050840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0678098360 tx=0x7f067807ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=139c0d5f85f411df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.950+0000 7f06637fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.950+0000 7f06637fe640 1 -- 172.21.15.5:0/844834736 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06780b4040 con 0x5621b103ab40 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.950+0000 7f06637fe640 1 -- 172.21.15.5:0/844834736 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f067809b560 con 0x5621b103ab40 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.950+0000 7f068cce1740 1 -- 172.21.15.5:0/844834736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0674003aa0 con 0x5621b103ab40 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.950+0000 7f06637fe640 1 -- 172.21.15.5:0/844834736 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f06780a02b0 con 0x5621b103ab40 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.950+0000 7f068cce1740 10 client.28555 fetch_fsmap learned FSMap version 44 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.950+0000 7f068cce1740 1 -- 172.21.15.5:0/844834736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5621b10c7070 con 0x5621b103ab40 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.951+0000 7f06637fe640 1 -- 172.21.15.5:0/844834736 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0674003aa0 con 0x5621b103ab40 2024-09-19T06:17:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.951+0000 7f06637fe640 1 -- 172.21.15.5:0/844834736 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 44) ==== 8402+0+0 (secure 0 0 0) 0x7f06780cdc50 con 0x5621b103ab40 2024-09-19T06:17:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.951+0000 7f068cce1740 10 client.28555 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T06:17:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.951+0000 7f068cce1740 10 client.28555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.951+0000 7f068cce1740 20 client.28555 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.951+0000 7f068cce1740 1 --2- 172.21.15.5:0/844834736 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5621b10c7870 0x5621b10c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.951+0000 7f068cce1740 4 client.28555 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:17:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.951+0000 7f068cce1740 1 -- 172.21.15.5:0/844834736 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621b0fa6d70 con 0x5621b10c7870 2024-09-19T06:17:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.951+0000 7f0683fff640 1 --2- 172.21.15.5:0/844834736 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5621b10c7870 0x5621b10c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f06637fe640 10 client.28555 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f0683fff640 1 --2- 172.21.15.5:0/844834736 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5621b10c7870 0x5621b10c9c90 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c644d55d8917e164 server_cookie=1d225652fe19cc87 in_seq=0 out_seq=0 2024-09-19T06:17:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f06637fe640 1 -- 172.21.15.5:0/844834736 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621b0fa6d70 con 0x5621b10c7870 2024-09-19T06:17:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f06637fe640 10 client.28555 handle_command_reply: tid=0 2024-09-19T06:17:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f068cce1740 1 client.28555 shutdown 2024-09-19T06:17:37.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f068cce1740 1 -- 172.21.15.5:0/844834736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b103ab40 msgr2=0x5621b1050840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:37.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f068cce1740 1 --2- 172.21.15.5:0/844834736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b103ab40 0x5621b1050840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0678098360 tx=0x7f067807ad10 comp rx=0 tx=0).stop 2024-09-19T06:17:37.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f068cce1740 1 -- 172.21.15.5:0/844834736 shutdown_connections 2024-09-19T06:17:37.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f068cce1740 1 --2- 172.21.15.5:0/844834736 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5621b10c7870 0x5621b10c9c90 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f068cce1740 1 --2- 172.21.15.5:0/844834736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621b104cb80 0x5621b10bf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f068cce1740 1 --2- 172.21.15.5:0/844834736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5621b103ab40 0x5621b1050840 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f06637fe640 0 client.28555 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:37.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f068cce1740 1 --2- 172.21.15.5:0/844834736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621b10395d0 0x5621b1050300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f068cce1740 1 -- 172.21.15.5:0/844834736 >> 172.21.15.5:0/844834736 conn(0x5621b1002310 msgr2=0x5621b1034d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:37.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f068cce1740 1 -- 172.21.15.5:0/844834736 shutdown_connections 2024-09-19T06:17:37.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f068cce1740 1 -- 172.21.15.5:0/844834736 wait complete. 2024-09-19T06:17:37.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.952+0000 7f068cce1740 20 client.28555 trim_cache size 0 max 16384 2024-09-19T06:17:37.224 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:17:37.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.953+0000 7f068a908640 1 -- 172.21.15.5:0/4158431222 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f066c0a35f0 msgr2=0x7f066c0a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:37.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.953+0000 7f068a908640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f066c0a35f0 0x7f066c0a5ab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f067c005fd0 tx=0x7f067c009500 comp rx=0 tx=0).stop 2024-09-19T06:17:37.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.953+0000 7f068a908640 1 -- 172.21.15.5:0/4158431222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06841036e0 msgr2=0x7f0684100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:37.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.953+0000 7f068a908640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06841036e0 0x7f0684100ed0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f06840697d0 tx=0x7f067800ee50 comp rx=0 tx=0).stop 2024-09-19T06:17:37.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.954+0000 7f068a908640 1 -- 172.21.15.5:0/4158431222 shutdown_connections 2024-09-19T06:17:37.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.954+0000 7f068a908640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06841082a0 0x7f0684101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.954+0000 7f068a908640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06841036e0 0x7f0684100ed0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.954+0000 7f068a908640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f066c0a35f0 0x7f066c0a5ab0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.954+0000 7f068a908640 1 --2- 172.21.15.5:0/4158431222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0684102d30 0x7f0684100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.954+0000 7f068a908640 1 -- 172.21.15.5:0/4158431222 >> 172.21.15.5:0/4158431222 conn(0x7f06840fc7d0 msgr2=0x7f0684105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.954+0000 7f068a908640 1 -- 172.21.15.5:0/4158431222 shutdown_connections 2024-09-19T06:17:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:36.954+0000 7f068a908640 1 -- 172.21.15.5:0/4158431222 wait complete. 2024-09-19T06:17:37.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 10391 12.06 MB/sec warmup 61 sec latency 116.098 ms 2024-09-19T06:17:38.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 10570 12.05 MB/sec warmup 62 sec latency 125.051 ms 2024-09-19T06:17:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:38 smithi157 ceph-mon[35346]: pgmap v448: 97 pgs: 97 active+clean; 454 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 352 op/s 2024-09-19T06:17:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:38 smithi005 ceph-mon[30955]: pgmap v448: 97 pgs: 97 active+clean; 454 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 352 op/s 2024-09-19T06:17:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:38 smithi049 ceph-mon[34640]: pgmap v448: 97 pgs: 97 active+clean; 454 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 352 op/s 2024-09-19T06:17:39.550 INFO:tasks.workunit.client.0.smithi005.stdout: 10 10713 11.96 MB/sec warmup 63 sec latency 167.914 ms 2024-09-19T06:17:40.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 10871 11.86 MB/sec warmup 64 sec latency 137.799 ms 2024-09-19T06:17:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:40 smithi157 ceph-mon[35346]: pgmap v449: 97 pgs: 97 active+clean; 454 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 4.3 MiB/s wr, 231 op/s 2024-09-19T06:17:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:40 smithi005 ceph-mon[30955]: pgmap v449: 97 pgs: 97 active+clean; 454 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 4.3 MiB/s wr, 231 op/s 2024-09-19T06:17:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:40 smithi049 ceph-mon[34640]: pgmap v449: 97 pgs: 97 active+clean; 454 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 4.3 MiB/s wr, 231 op/s 2024-09-19T06:17:41.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11008 11.76 MB/sec warmup 65 sec latency 82.610 ms 2024-09-19T06:17:42.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11146 11.71 MB/sec warmup 66 sec latency 151.952 ms 2024-09-19T06:17:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:42 smithi005 ceph-mon[30955]: pgmap v450: 97 pgs: 97 active+clean; 454 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 4.3 MiB/s wr, 231 op/s 2024-09-19T06:17:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:42 smithi049 ceph-mon[34640]: pgmap v450: 97 pgs: 97 active+clean; 454 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 4.3 MiB/s wr, 231 op/s 2024-09-19T06:17:43.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:42 smithi157 ceph-mon[35346]: pgmap v450: 97 pgs: 97 active+clean; 454 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 4.3 MiB/s wr, 231 op/s 2024-09-19T06:17:43.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11283 11.68 MB/sec warmup 67 sec latency 166.025 ms 2024-09-19T06:17:44.429 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:17:44.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:17:44.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11474 11.76 MB/sec warmup 68 sec latency 145.371 ms 2024-09-19T06:17:44.836 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:17:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:44 smithi005 ceph-mon[30955]: pgmap v451: 97 pgs: 97 active+clean; 461 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.2 MiB/s wr, 334 op/s 2024-09-19T06:17:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:44 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:17:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:44 smithi049 ceph-mon[34640]: pgmap v451: 97 pgs: 97 active+clean; 461 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.2 MiB/s wr, 334 op/s 2024-09-19T06:17:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:44 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:17:45.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:44 smithi157 ceph-mon[35346]: pgmap v451: 97 pgs: 97 active+clean; 461 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.2 MiB/s wr, 334 op/s 2024-09-19T06:17:45.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:44 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:17:45.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11608 11.62 MB/sec warmup 69 sec latency 99.335 ms 2024-09-19T06:17:45.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:45 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:17:45.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:45 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:17:45.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:45 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:17:45.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:45 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:17:45.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:45 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:17:45.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:45 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:17:46.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:45 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:17:46.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:45 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:17:46.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:45 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:17:46.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11748 11.57 MB/sec warmup 70 sec latency 164.022 ms 2024-09-19T06:17:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:46 smithi005 ceph-mon[30955]: pgmap v452: 97 pgs: 97 active+clean; 461 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-09-19T06:17:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:46 smithi049 ceph-mon[34640]: pgmap v452: 97 pgs: 97 active+clean; 461 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-09-19T06:17:47.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:46 smithi157 ceph-mon[35346]: pgmap v452: 97 pgs: 97 active+clean; 461 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-09-19T06:17:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.501+0000 7f5f1cff6640 1 -- 172.21.15.5:0/4185478876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f18102c80 msgr2=0x7f5f18103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.501+0000 7f5f1cff6640 1 --2- 172.21.15.5:0/4185478876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f18102c80 0x7f5f18103060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009be0 tx=0x7f5f1002a750 comp rx=0 tx=0).stop 2024-09-19T06:17:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.501+0000 7f5f1cff6640 1 -- 172.21.15.5:0/4185478876 shutdown_connections 2024-09-19T06:17:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.501+0000 7f5f1cff6640 1 --2- 172.21.15.5:0/4185478876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1810c0a0 0x7f5f1810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.501+0000 7f5f1cff6640 1 --2- 172.21.15.5:0/4185478876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f181035a0 0x7f5f1810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.501+0000 7f5f1cff6640 1 --2- 172.21.15.5:0/4185478876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f18102c80 0x7f5f18103060 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.501+0000 7f5f1cff6640 1 -- 172.21.15.5:0/4185478876 >> 172.21.15.5:0/4185478876 conn(0x7f5f180779a0 msgr2=0x7f5f180fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.501+0000 7f5f1cff6640 1 -- 172.21.15.5:0/4185478876 shutdown_connections 2024-09-19T06:17:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.502+0000 7f5f1cff6640 1 -- 172.21.15.5:0/4185478876 wait complete. 2024-09-19T06:17:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.503+0000 7f5f1cff6640 1 Processor -- start 2024-09-19T06:17:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.503+0000 7f5f1cff6640 1 -- start start 2024-09-19T06:17:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.503+0000 7f5f1cff6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f18102c80 0x7f5f18073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.503+0000 7f5f1cff6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f181035a0 0x7f5f1806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.503+0000 7f5f1cff6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1810c0a0 0x7f5f1806d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.503+0000 7f5f1cff6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18111d50 con 0x7f5f1810c0a0 2024-09-19T06:17:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.503+0000 7f5f1cff6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18111bd0 con 0x7f5f18102c80 2024-09-19T06:17:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.503+0000 7f5f1cff6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18111ed0 con 0x7f5f181035a0 2024-09-19T06:17:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.503+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1810c0a0 0x7f5f1806d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.503+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1810c0a0 0x7f5f1806d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41968/0 (socket says 172.21.15.5:41968) 2024-09-19T06:17:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.503+0000 7f5f16d9d640 1 -- 172.21.15.5:0/2853634213 learned_addr learned my addr 172.21.15.5:0/2853634213 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.504+0000 7f5f1659c640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f18102c80 0x7f5f18073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.504+0000 7f5f16d9d640 1 -- 172.21.15.5:0/2853634213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f181035a0 msgr2=0x7f5f1806cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.504+0000 7f5f16d9d640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f181035a0 0x7f5f1806cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.504+0000 7f5f16d9d640 1 -- 172.21.15.5:0/2853634213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f18102c80 msgr2=0x7f5f18073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.504+0000 7f5f16d9d640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f18102c80 0x7f5f18073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.504+0000 7f5f16d9d640 1 -- 172.21.15.5:0/2853634213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1806d740 con 0x7f5f1810c0a0 2024-09-19T06:17:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.504+0000 7f5f16d9d640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1810c0a0 0x7f5f1806d0a0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800e970 tx=0x7f5f0800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.504+0000 7f5f0f7fe640 1 -- 172.21.15.5:0/2853634213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0800ccc0 con 0x7f5f1810c0a0 2024-09-19T06:17:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.504+0000 7f5f1cff6640 1 -- 172.21.15.5:0/2853634213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f18071600 con 0x7f5f1810c0a0 2024-09-19T06:17:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.505+0000 7f5f1cff6640 1 -- 172.21.15.5:0/2853634213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f18071b50 con 0x7f5f1810c0a0 2024-09-19T06:17:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.505+0000 7f5f0f7fe640 1 -- 172.21.15.5:0/2853634213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f0800ce60 con 0x7f5f1810c0a0 2024-09-19T06:17:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.505+0000 7f5f0f7fe640 1 -- 172.21.15.5:0/2853634213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08010430 con 0x7f5f1810c0a0 2024-09-19T06:17:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.508+0000 7f5f0f7fe640 1 -- 172.21.15.5:0/2853634213 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5f080105d0 con 0x7f5f1810c0a0 2024-09-19T06:17:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.508+0000 7f5f1fe6c740 1 Processor -- start 2024-09-19T06:17:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.509+0000 7f5f1fe6c740 1 -- start start 2024-09-19T06:17:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.509+0000 7f5f0f7fe640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5f040a3430 0x7f5f040a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.509+0000 7f5f1fe6c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x562682838d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.509+0000 7f5f15d9b640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5f040a3430 0x7f5f040a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.509+0000 7f5f0f7fe640 1 -- 172.21.15.5:0/2853634213 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f5f080cfff0 con 0x7f5f1810c0a0 2024-09-19T06:17:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.509+0000 7f5f1fe6c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828392d0 0x562682847b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.509+0000 7f5f1fe6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x56268284a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.509+0000 7f5f1fe6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56268279d2f0 con 0x5626828480c0 2024-09-19T06:17:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.509+0000 7f5f1fe6c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562682745270 con 0x5626828389b0 2024-09-19T06:17:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.509+0000 7f5f1fe6c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5626827992b0 con 0x5626828392d0 2024-09-19T06:17:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.510+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x56268284a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.510+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x562682838d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.510+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x56268284a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41970/0 (socket says 172.21.15.5:41970) 2024-09-19T06:17:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.510+0000 7f5f16d9d640 1 -- 172.21.15.5:0/3059996832 learned_addr learned my addr 172.21.15.5:0/3059996832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.510+0000 7f5f15d9b640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5f040a3430 0x7f5f040a58f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5f000058c0 tx=0x7f5f00005850 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.510+0000 7f5f15d9b640 1 --2- 172.21.15.5:0/3059996832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828392d0 0x562682847b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.510+0000 7f5f1659c640 1 -- 172.21.15.5:0/3059996832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828392d0 msgr2=0x562682847b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.510+0000 7f5f1659c640 1 --2- 172.21.15.5:0/3059996832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828392d0 0x562682847b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.510+0000 7f5f1659c640 1 -- 172.21.15.5:0/3059996832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 msgr2=0x56268284a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.510+0000 7f5f1659c640 1 --2- 172.21.15.5:0/3059996832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x56268284a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.510+0000 7f5f1659c640 1 -- 172.21.15.5:0/3059996832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562682744340 con 0x5626828389b0 2024-09-19T06:17:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.510+0000 7f5f1659c640 1 --2- 172.21.15.5:0/3059996832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x562682838d90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5f1002ad40 tx=0x7f5f10005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea5e1231e2a9cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.511+0000 7f5f0cff9640 1 -- 172.21.15.5:0/3059996832 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10041030 con 0x5626828389b0 2024-09-19T06:17:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.511+0000 7f5f0cff9640 1 -- 172.21.15.5:0/3059996832 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f10037040 con 0x5626828389b0 2024-09-19T06:17:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.511+0000 7f5f0cff9640 1 -- 172.21.15.5:0/3059996832 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10036040 con 0x5626828389b0 2024-09-19T06:17:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.511+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/3059996832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 msgr2=0x562682838d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.511+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/3059996832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x562682838d90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5f1002ad40 tx=0x7f5f10005ec0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.512+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/3059996832 shutdown_connections 2024-09-19T06:17:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.512+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/3059996832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x56268284a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.512+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/3059996832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828392d0 0x562682847b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.512+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/3059996832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x562682838d90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.512+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/3059996832 >> 172.21.15.5:0/3059996832 conn(0x562682835060 msgr2=0x562682835440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.512+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/3059996832 shutdown_connections 2024-09-19T06:17:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.512+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/3059996832 wait complete. 2024-09-19T06:17:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.512+0000 7f5f1fe6c740 1 -- start start 2024-09-19T06:17:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.512+0000 7f5f1fe6c740 1 Processor -- start 2024-09-19T06:17:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1fe6c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x562682845020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1fe6c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828392d0 0x562682845560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1fe6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x562682845aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1fe6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56268284ac30 con 0x5626828480c0 2024-09-19T06:17:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1fe6c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562682726290 con 0x5626828389b0 2024-09-19T06:17:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1fe6c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56268284adb0 con 0x5626828392d0 2024-09-19T06:17:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1fe6c740 10 client.0 _ll_register_callbacks cb 0x562682757140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x562682845aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x562682845020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x562682845aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41974/0 (socket says 172.21.15.5:41974) 2024-09-19T06:17:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f16d9d640 1 -- 172.21.15.5:0/1041302257 learned_addr learned my addr 172.21.15.5:0/1041302257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f15d9b640 1 --2- 172.21.15.5:0/1041302257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828392d0 0x562682845560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1659c640 1 -- 172.21.15.5:0/1041302257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828392d0 msgr2=0x562682845560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1659c640 1 --2- 172.21.15.5:0/1041302257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828392d0 0x562682845560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1659c640 1 -- 172.21.15.5:0/1041302257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 msgr2=0x562682845aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1659c640 1 --2- 172.21.15.5:0/1041302257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x562682845aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1659c640 1 -- 172.21.15.5:0/1041302257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626824b0210 con 0x5626828389b0 2024-09-19T06:17:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5f1659c640 1 --2- 172.21.15.5:0/1041302257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x562682845020 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5f10000f80 tx=0x7f5f10030b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d7908e7f4c101e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.513+0000 7f5efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:17:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5efdffb640 1 -- 172.21.15.5:0/1041302257 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1004a070 con 0x5626828389b0 2024-09-19T06:17:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5efdffb640 1 -- 172.21.15.5:0/1041302257 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f5f10003b70 con 0x5626828389b0 2024-09-19T06:17:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1041302257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f10037040 con 0x5626828389b0 2024-09-19T06:17:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5efdffb640 1 -- 172.21.15.5:0/1041302257 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5f10056050 con 0x5626828389b0 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5f1fe6c740 10 client.28625 fetch_fsmap learned FSMap version 44 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1041302257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56268257c620 con 0x5626828389b0 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5efdffb640 1 -- 172.21.15.5:0/1041302257 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f10037040 con 0x5626828389b0 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5efdffb640 1 -- 172.21.15.5:0/1041302257 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 44) ==== 8402+0+0 (secure 0 0 0) 0x7f5f1003c040 con 0x5626828389b0 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5f1fe6c740 10 client.28625 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5f1fe6c740 10 client.28625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5f1fe6c740 20 client.28625 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/1041302257 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562682858290 0x56268285a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5f1fe6c740 4 client.28625 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.514+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1041302257 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626827aaa20 con 0x562682858290 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.515+0000 7f5f15d9b640 1 --2- 172.21.15.5:0/1041302257 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562682858290 0x56268285a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.515+0000 7f5efdffb640 10 client.28625 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.515+0000 7f5f15d9b640 1 --2- 172.21.15.5:0/1041302257 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562682858290 0x56268285a6d0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efedb97400a0fae server_cookie=9fcf55f04aeb587d in_seq=0 out_seq=0 2024-09-19T06:17:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5efdffb640 1 -- 172.21.15.5:0/1041302257 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626827aaa20 con 0x562682858290 2024-09-19T06:17:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5efdffb640 10 client.28625 handle_command_reply: tid=0 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5f1fe6c740 1 client.28625 shutdown 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1041302257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 msgr2=0x562682845020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/1041302257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x562682845020 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5f10000f80 tx=0x7f5f10030b50 comp rx=0 tx=0).stop 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1041302257 shutdown_connections 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/1041302257 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562682858290 0x56268285a6d0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/1041302257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x562682845aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5efdffb640 0 client.28625 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/1041302257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828392d0 0x562682845560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/1041302257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x562682845020 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1041302257 >> 172.21.15.5:0/1041302257 conn(0x562682835060 msgr2=0x56268283ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1041302257 shutdown_connections 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1041302257 wait complete. 2024-09-19T06:17:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.516+0000 7f5f1fe6c740 20 client.28625 trim_cache size 0 max 16384 2024-09-19T06:17:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1fe6c740 1 Processor -- start 2024-09-19T06:17:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1fe6c740 1 -- start start 2024-09-19T06:17:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1fe6c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x56268285ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1fe6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828392d0 0x5626828604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1fe6c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828480c0 0x562682842260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1fe6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef800e7e0 con 0x5626828392d0 2024-09-19T06:17:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1fe6c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef80104a0 con 0x5626828389b0 2024-09-19T06:17:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1fe6c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef800ef10 con 0x5626828480c0 2024-09-19T06:17:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x56268285ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f15d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828392d0 0x5626828604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x56268285ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58820/0 (socket says 172.21.15.5:58820) 2024-09-19T06:17:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1659c640 1 -- 172.21.15.5:0/1293756679 learned_addr learned my addr 172.21.15.5:0/1293756679 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f16d9d640 1 --2- 172.21.15.5:0/1293756679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828480c0 0x562682842260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1659c640 1 -- 172.21.15.5:0/1293756679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828480c0 msgr2=0x562682842260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.539+0000 7f5f1659c640 1 --2- 172.21.15.5:0/1293756679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828480c0 0x562682842260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.540+0000 7f5f1659c640 1 -- 172.21.15.5:0/1293756679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828392d0 msgr2=0x5626828604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.540+0000 7f5f1659c640 1 --2- 172.21.15.5:0/1293756679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828392d0 0x5626828604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.540+0000 7f5f1659c640 1 -- 172.21.15.5:0/1293756679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268285ad50 con 0x5626828389b0 2024-09-19T06:17:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.540+0000 7f5f1659c640 1 --2- 172.21.15.5:0/1293756679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x56268285ffa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5f10005d70 tx=0x7f5f10005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=524d4e1cd952c667 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.540+0000 7f5efffff640 1 -- 172.21.15.5:0/1293756679 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10041020 con 0x5626828389b0 2024-09-19T06:17:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.540+0000 7f5efffff640 1 -- 172.21.15.5:0/1293756679 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f1002ae50 con 0x5626828389b0 2024-09-19T06:17:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.540+0000 7f5efffff640 1 -- 172.21.15.5:0/1293756679 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1004a040 con 0x5626828389b0 2024-09-19T06:17:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.541+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1293756679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 msgr2=0x56268285ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.541+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/1293756679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x56268285ffa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5f10005d70 tx=0x7f5f10005ec0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.541+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1293756679 shutdown_connections 2024-09-19T06:17:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.541+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/1293756679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828480c0 0x562682842260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.541+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/1293756679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828392d0 0x5626828604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.541+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/1293756679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828389b0 0x56268285ffa0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.541+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1293756679 >> 172.21.15.5:0/1293756679 conn(0x562682835060 msgr2=0x5626828470a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.541+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1293756679 shutdown_connections 2024-09-19T06:17:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.541+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/1293756679 wait complete. 2024-09-19T06:17:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.542+0000 7f5f1fe6c740 1 -- start start 2024-09-19T06:17:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.542+0000 7f5f1fe6c740 1 Processor -- start 2024-09-19T06:17:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.542+0000 7f5f1fe6c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828389b0 0x5626828505b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.542+0000 7f5f1fe6c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828392d0 0x56268284b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.542+0000 7f5f1fe6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x56268284be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.542+0000 7f5f15d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828392d0 0x56268284b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828389b0 0x5626828505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f15d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828392d0 0x56268284b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58834/0 (socket says 172.21.15.5:58834) 2024-09-19T06:17:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f15d9b640 1 -- 172.21.15.5:0/2026561741 learned_addr learned my addr 172.21.15.5:0/2026561741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.542+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x56268284be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/2026561741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef800fdd0 con 0x5626828480c0 2024-09-19T06:17:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/2026561741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef800f640 con 0x5626828392d0 2024-09-19T06:17:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/2026561741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef800c7e0 con 0x5626828389b0 2024-09-19T06:17:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f1fe6c740 10 client.0 _ll_register_callbacks cb 0x7f5f0809c0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f15d9b640 1 -- 172.21.15.5:0/2026561741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828389b0 msgr2=0x5626828505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f15d9b640 1 --2- 172.21.15.5:0/2026561741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828389b0 0x5626828505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f15d9b640 1 -- 172.21.15.5:0/2026561741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 msgr2=0x56268284be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f15d9b640 1 --2- 172.21.15.5:0/2026561741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x56268284be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f15d9b640 1 -- 172.21.15.5:0/2026561741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56268284c400 con 0x5626828392d0 2024-09-19T06:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f16d9d640 1 --2- 172.21.15.5:0/2026561741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x56268284be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5f15d9b640 1 --2- 172.21.15.5:0/2026561741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828392d0 0x56268284b8d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5f00005fd0 tx=0x7f5f000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9df98bad5c9e963d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.543+0000 7f5efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.544+0000 7f5efdffb640 1 -- 172.21.15.5:0/2026561741 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00036060 con 0x5626828392d0 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.544+0000 7f5efdffb640 1 -- 172.21.15.5:0/2026561741 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f5f00005230 con 0x5626828392d0 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.544+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/2026561741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f1002ae50 con 0x5626828392d0 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.544+0000 7f5efdffb640 1 -- 172.21.15.5:0/2026561741 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5f00005430 con 0x5626828392d0 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.544+0000 7f5f1fe6c740 10 client.28631 fetch_fsmap learned FSMap version 44 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.544+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/2026561741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5626828c2dd0 con 0x5626828392d0 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.544+0000 7f5efdffb640 1 -- 172.21.15.5:0/2026561741 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f1002ae50 con 0x5626828392d0 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.544+0000 7f5efdffb640 1 -- 172.21.15.5:0/2026561741 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 44) ==== 8402+0+0 (secure 0 0 0) 0x7f5f0001da60 con 0x5626828392d0 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.544+0000 7f5f1fe6c740 10 client.28631 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.544+0000 7f5f1fe6c740 10 client.28631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.544+0000 7f5f1fe6c740 20 client.28631 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.545+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/2026561741 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5626828c3640 0x5626828c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.545+0000 7f5f1fe6c740 4 client.28631 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:17:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.545+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/2026561741 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626827a4d70 con 0x5626828c3640 2024-09-19T06:17:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.545+0000 7f5f1659c640 1 --2- 172.21.15.5:0/2026561741 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5626828c3640 0x5626828c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.545+0000 7f5efdffb640 10 client.28631 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.545+0000 7f5f1659c640 1 --2- 172.21.15.5:0/2026561741 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5626828c3640 0x5626828c5a60 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebaabf937f679f02 server_cookie=aeac9e80d30494a7 in_seq=0 out_seq=0 2024-09-19T06:17:47.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11896 11.46 MB/sec warmup 71 sec latency 94.195 ms 2024-09-19T06:17:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.599+0000 7f5efdffb640 1 -- 172.21.15.5:0/2026561741 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626827a4d70 con 0x5626828c3640 2024-09-19T06:17:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.599+0000 7f5efdffb640 10 client.28631 handle_command_reply: tid=0 2024-09-19T06:17:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.599+0000 7f5f1fe6c740 1 client.28631 shutdown 2024-09-19T06:17:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.600+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/2026561741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828392d0 msgr2=0x56268284b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.600+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/2026561741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828392d0 0x56268284b8d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5f00005fd0 tx=0x7f5f000046c0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.600+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/2026561741 shutdown_connections 2024-09-19T06:17:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.600+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/2026561741 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5626828c3640 0x5626828c5a60 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.600+0000 7f5efdffb640 0 client.28631 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.600+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/2026561741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626828480c0 0x56268284be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.600+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/2026561741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626828392d0 0x56268284b8d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.600+0000 7f5f1fe6c740 1 --2- 172.21.15.5:0/2026561741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626828389b0 0x5626828505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.600+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/2026561741 >> 172.21.15.5:0/2026561741 conn(0x562682835060 msgr2=0x5626828339a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.600+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/2026561741 shutdown_connections 2024-09-19T06:17:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.600+0000 7f5f1fe6c740 1 -- 172.21.15.5:0/2026561741 wait complete. 2024-09-19T06:17:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.600+0000 7f5f1fe6c740 20 client.28631 trim_cache size 0 max 16384 2024-09-19T06:17:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:17:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:17:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:17:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:17:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.601+0000 7f5f1cff6640 1 -- 172.21.15.5:0/2853634213 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5f040a3430 msgr2=0x7f5f040a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.601+0000 7f5f1cff6640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5f040a3430 0x7f5f040a58f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5f000058c0 tx=0x7f5f00005850 comp rx=0 tx=0).stop 2024-09-19T06:17:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.601+0000 7f5f1cff6640 1 -- 172.21.15.5:0/2853634213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1810c0a0 msgr2=0x7f5f1806d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.601+0000 7f5f1cff6640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1810c0a0 0x7f5f1806d0a0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800e970 tx=0x7f5f0800ed70 comp rx=0 tx=0).stop 2024-09-19T06:17:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.601+0000 7f5f1cff6640 1 -- 172.21.15.5:0/2853634213 shutdown_connections 2024-09-19T06:17:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.601+0000 7f5f1cff6640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1810c0a0 0x7f5f1806d0a0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.601+0000 7f5f1cff6640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5f040a3430 0x7f5f040a58f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.601+0000 7f5f1cff6640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f181035a0 0x7f5f1806cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.601+0000 7f5f1cff6640 1 --2- 172.21.15.5:0/2853634213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f18102c80 0x7f5f18073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.601+0000 7f5f1cff6640 1 -- 172.21.15.5:0/2853634213 >> 172.21.15.5:0/2853634213 conn(0x7f5f180779a0 msgr2=0x7f5f180fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.601+0000 7f5f1cff6640 1 -- 172.21.15.5:0/2853634213 shutdown_connections 2024-09-19T06:17:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:17:47.601+0000 7f5f1cff6640 1 -- 172.21.15.5:0/2853634213 wait complete. 2024-09-19T06:17:47.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:47 smithi005 ceph-mon[30955]: pgmap v453: 97 pgs: 97 active+clean; 460 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 327 op/s 2024-09-19T06:17:47.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:47 smithi049 ceph-mon[34640]: pgmap v453: 97 pgs: 97 active+clean; 460 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 327 op/s 2024-09-19T06:17:48.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:47 smithi157 ceph-mon[35346]: pgmap v453: 97 pgs: 97 active+clean; 460 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 327 op/s 2024-09-19T06:17:48.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12051 11.34 MB/sec warmup 72 sec latency 109.373 ms 2024-09-19T06:17:48.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:48 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:17:48.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:48 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:48 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:17:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:48 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.147+0000 7f7ddc693640 1 -- 172.21.15.5:0/3645997112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dd41009d0 msgr2=0x7f7dd4100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.147+0000 7f7ddc693640 1 --2- 172.21.15.5:0/3645997112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dd41009d0 0x7f7dd4100db0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009900 tx=0x7f7dd002a410 comp rx=0 tx=0).stop 2024-09-19T06:17:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.147+0000 7f7ddc693640 1 -- 172.21.15.5:0/3645997112 shutdown_connections 2024-09-19T06:17:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.147+0000 7f7ddc693640 1 --2- 172.21.15.5:0/3645997112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd410e4f0 0x7f7dd4110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.147+0000 7f7ddc693640 1 --2- 172.21.15.5:0/3645997112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd4101380 0x7f7dd410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.147+0000 7f7ddc693640 1 --2- 172.21.15.5:0/3645997112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dd41009d0 0x7f7dd4100db0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.147+0000 7f7ddc693640 1 -- 172.21.15.5:0/3645997112 >> 172.21.15.5:0/3645997112 conn(0x7f7dd40fc7d0 msgr2=0x7f7dd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.147+0000 7f7ddc693640 1 -- 172.21.15.5:0/3645997112 shutdown_connections 2024-09-19T06:17:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.147+0000 7f7ddc693640 1 -- 172.21.15.5:0/3645997112 wait complete. 2024-09-19T06:17:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.147+0000 7f7ddc693640 1 Processor -- start 2024-09-19T06:17:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7ddc693640 1 -- start start 2024-09-19T06:17:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7ddc693640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dd41009d0 0x7f7dd419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7ddc693640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd4101380 0x7f7dd419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7ddc693640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd410e4f0 0x7f7dd419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7ddc693640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd4114120 con 0x7f7dd41009d0 2024-09-19T06:17:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7ddc693640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd4113fa0 con 0x7f7dd410e4f0 2024-09-19T06:17:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7ddc693640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41142a0 con 0x7f7dd4101380 2024-09-19T06:17:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7dda42f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dd41009d0 0x7f7dd419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7dda42f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dd41009d0 0x7f7dd419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42016/0 (socket says 172.21.15.5:42016) 2024-09-19T06:17:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7dda42f640 1 -- 172.21.15.5:0/431302366 learned_addr learned my addr 172.21.15.5:0/431302366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7dd9c2e640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd4101380 0x7f7dd419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7dda42f640 1 -- 172.21.15.5:0/431302366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd4101380 msgr2=0x7f7dd419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7dda42f640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd4101380 0x7f7dd419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7dda42f640 1 -- 172.21.15.5:0/431302366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd410e4f0 msgr2=0x7f7dd419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7dda42f640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd410e4f0 0x7f7dd419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7dda42f640 1 -- 172.21.15.5:0/431302366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd419f690 con 0x7f7dd41009d0 2024-09-19T06:17:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7dd9c2e640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd4101380 0x7f7dd419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:17:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7dda42f640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dd41009d0 0x7f7dd419a6e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0005940 tx=0x7f7dd00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.148+0000 7f7dcb7fe640 1 -- 172.21.15.5:0/431302366 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd003f030 con 0x7f7dd41009d0 2024-09-19T06:17:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.149+0000 7f7ddc693640 1 -- 172.21.15.5:0/431302366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd419f920 con 0x7f7dd41009d0 2024-09-19T06:17:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.149+0000 7f7dcb7fe640 1 -- 172.21.15.5:0/431302366 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dd0003cb0 con 0x7f7dd41009d0 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.149+0000 7f7dcb7fe640 1 -- 172.21.15.5:0/431302366 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0035070 con 0x7f7dd41009d0 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.149+0000 7f7ddc693640 1 -- 172.21.15.5:0/431302366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd41a7260 con 0x7f7dd41009d0 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.151+0000 7f7dcb7fe640 1 -- 172.21.15.5:0/431302366 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7dd0042030 con 0x7f7dd41009d0 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.151+0000 7f7ddea6c740 1 Processor -- start 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.151+0000 7f7ddea6c740 1 -- start start 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.151+0000 7f7ddea6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 0x5609a2268f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.151+0000 7f7ddea6c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2269480 0x5609a2273800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.151+0000 7f7ddea6c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2273d40 0x5609a2276130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddea6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609a21ce280 con 0x5609a2268b60 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddea6c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5609a2175b10 con 0x5609a2273d40 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddea6c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5609a21ca240 con 0x5609a2269480 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7dcb7fe640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7dc00a33b0 0x7f7dc00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddac30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 0x5609a2268f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7dda42f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2269480 0x5609a2273800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddac30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 0x5609a2268f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42030/0 (socket says 172.21.15.5:42030) 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddac30640 1 -- 172.21.15.5:0/1906824731 learned_addr learned my addr 172.21.15.5:0/1906824731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddac30640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7dc00a33b0 0x7f7dc00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7dcb7fe640 1 -- 172.21.15.5:0/431302366 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f7dd00f01e0 con 0x7f7dd41009d0 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7dd9c2e640 1 --2- 172.21.15.5:0/1906824731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2273d40 0x5609a2276130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddac30640 1 -- 172.21.15.5:0/1906824731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2269480 msgr2=0x5609a2273800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddac30640 1 --2- 172.21.15.5:0/1906824731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2269480 0x5609a2273800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddac30640 1 -- 172.21.15.5:0/1906824731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2273d40 msgr2=0x5609a2276130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddac30640 1 --2- 172.21.15.5:0/1906824731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2273d40 0x5609a2276130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddac30640 1 -- 172.21.15.5:0/1906824731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a21821f0 con 0x5609a2268b60 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7dda42f640 1 --2- 172.21.15.5:0/1906824731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2269480 0x5609a2273800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7dd9c2e640 1 --2- 172.21.15.5:0/1906824731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2273d40 0x5609a2276130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:17:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.152+0000 7f7ddac30640 1 --2- 172.21.15.5:0/1906824731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 0x5609a2268f40 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc012c60 tx=0x7f7dcc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56e5d19eb8dfecf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.153+0000 7f7ddac30640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7dc00a33b0 0x7f7dc00a5870 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009fd0 tx=0x7f7dcc01a000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.153+0000 7f7dc8ff9640 1 -- 172.21.15.5:0/1906824731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc020070 con 0x5609a2268b60 2024-09-19T06:17:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.153+0000 7f7dc8ff9640 1 -- 172.21.15.5:0/1906824731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dcc019ab0 con 0x5609a2268b60 2024-09-19T06:17:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.153+0000 7f7dc8ff9640 1 -- 172.21.15.5:0/1906824731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc003de0 con 0x5609a2268b60 2024-09-19T06:17:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 -- 172.21.15.5:0/1906824731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 msgr2=0x5609a2268f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/1906824731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 0x5609a2268f40 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc012c60 tx=0x7f7dcc00f730 comp rx=0 tx=0).stop 2024-09-19T06:17:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 -- 172.21.15.5:0/1906824731 shutdown_connections 2024-09-19T06:17:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/1906824731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2273d40 0x5609a2276130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/1906824731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2269480 0x5609a2273800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/1906824731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 0x5609a2268f40 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 -- 172.21.15.5:0/1906824731 >> 172.21.15.5:0/1906824731 conn(0x5609a22650f0 msgr2=0x5609a22654d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 -- 172.21.15.5:0/1906824731 shutdown_connections 2024-09-19T06:17:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 -- 172.21.15.5:0/1906824731 wait complete. 2024-09-19T06:17:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 -- start start 2024-09-19T06:17:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 Processor -- start 2024-09-19T06:17:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 0x5609a2270c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 0x5609a2271180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a22716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609a2276910 con 0x5609a2269480 2024-09-19T06:17:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5609a2188f10 con 0x5609a2268b60 2024-09-19T06:17:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5609a2276a90 con 0x5609a2273d40 2024-09-19T06:17:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.154+0000 7f7ddea6c740 10 client.0 _ll_register_callbacks cb 0x5609a2151a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dd9c2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 0x5609a2271180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7ddac30640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a22716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dda42f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 0x5609a2270c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dda42f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 0x5609a2270c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58854/0 (socket says 172.21.15.5:58854) 2024-09-19T06:17:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7ddac30640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a22716c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54674/0 (socket says 172.21.15.5:54674) 2024-09-19T06:17:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dda42f640 1 -- 172.21.15.5:0/3561039111 learned_addr learned my addr 172.21.15.5:0/3561039111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dd9c2e640 1 -- 172.21.15.5:0/3561039111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 msgr2=0x5609a22716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dd9c2e640 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a22716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dd9c2e640 1 -- 172.21.15.5:0/3561039111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 msgr2=0x5609a2270c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dd9c2e640 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 0x5609a2270c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dd9c2e640 1 -- 172.21.15.5:0/3561039111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a1f3f6a0 con 0x5609a2269480 2024-09-19T06:17:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dda42f640 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 0x5609a2270c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7ddac30640 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a22716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dd9c2e640 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 0x5609a2271180 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4013760 tx=0x7f7dc4013b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a847755935b73087 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:17:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dbdffb640 1 -- 172.21.15.5:0/3561039111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4011d70 con 0x5609a2269480 2024-09-19T06:17:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7dbdffb640 1 -- 172.21.15.5:0/3561039111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f7dc4005020 con 0x5609a2269480 2024-09-19T06:17:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.155+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3561039111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dcc019ab0 con 0x5609a2269480 2024-09-19T06:17:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.156+0000 7f7dbdffb640 1 -- 172.21.15.5:0/3561039111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7dc4005220 con 0x5609a2269480 2024-09-19T06:17:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.156+0000 7f7ddea6c740 10 client.19242 fetch_fsmap learned FSMap version 44 2024-09-19T06:17:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.156+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3561039111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5609a22825a0 con 0x5609a2269480 2024-09-19T06:17:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.156+0000 7f7dbdffb640 1 -- 172.21.15.5:0/3561039111 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dcc019ab0 con 0x5609a2269480 2024-09-19T06:17:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.156+0000 7f7dbdffb640 1 -- 172.21.15.5:0/3561039111 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8402+0+0 (secure 0 0 0) 0x7f7dc401a030 con 0x5609a2269480 2024-09-19T06:17:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.156+0000 7f7ddea6c740 10 client.19242 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T06:17:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.156+0000 7f7ddea6c740 10 client.19242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.156+0000 7f7ddea6c740 20 client.19242 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.156+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609a2283c20 0x5609a2286060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.156+0000 7f7ddea6c740 4 client.19242 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.156+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3561039111 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609a21db7f0 con 0x5609a2283c20 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.156+0000 7f7dda42f640 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609a2283c20 0x5609a2286060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7dda42f640 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609a2283c20 0x5609a2286060 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3838188d74c83e5e server_cookie=217e1f413873f171 in_seq=0 out_seq=0 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7dbdffb640 10 client.19242 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7dbdffb640 1 -- 172.21.15.5:0/3561039111 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609a21db7f0 con 0x5609a2283c20 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7dbdffb640 10 client.19242 handle_command_reply: tid=0 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7ddea6c740 1 client.19242 shutdown 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3561039111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 msgr2=0x5609a2271180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 0x5609a2271180 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4013760 tx=0x7f7dc4013b60 comp rx=0 tx=0).stop 2024-09-19T06:17:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3561039111 shutdown_connections 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609a2283c20 0x5609a2286060 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a22716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7dbdffb640 0 client.19242 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 0x5609a2271180 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/3561039111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 0x5609a2270c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.157+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3561039111 >> 172.21.15.5:0/3561039111 conn(0x5609a22650f0 msgr2=0x5609a2264270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.158+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3561039111 shutdown_connections 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.158+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3561039111 wait complete. 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.158+0000 7f7ddea6c740 20 client.19242 trim_cache size 0 max 16384 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.176+0000 7f7ddea6c740 1 Processor -- start 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.176+0000 7f7ddea6c740 1 -- start start 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7ddea6c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 0x5609a2271510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7ddea6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 0x5609a228b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7ddea6c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a228be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7ddea6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41afab0 con 0x5609a2269480 2024-09-19T06:17:48.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7ddea6c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41b1770 con 0x5609a2268b60 2024-09-19T06:17:48.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7ddea6c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41b01e0 con 0x5609a2273d40 2024-09-19T06:17:48.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7dd9c2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 0x5609a228b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7dd9c2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 0x5609a228b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42054/0 (socket says 172.21.15.5:42054) 2024-09-19T06:17:48.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7dd9c2e640 1 -- 172.21.15.5:0/3231127839 learned_addr learned my addr 172.21.15.5:0/3231127839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:48.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7dda42f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 0x5609a2271510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7ddac30640 1 --2- 172.21.15.5:0/3231127839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a228be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7dda42f640 1 -- 172.21.15.5:0/3231127839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 msgr2=0x5609a228be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:48.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7dda42f640 1 --2- 172.21.15.5:0/3231127839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a228be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7dda42f640 1 -- 172.21.15.5:0/3231127839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 msgr2=0x5609a228b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7dda42f640 1 --2- 172.21.15.5:0/3231127839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 0x5609a228b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7dda42f640 1 -- 172.21.15.5:0/3231127839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a22692b0 con 0x5609a2268b60 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7ddac30640 1 --2- 172.21.15.5:0/3231127839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a228be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.177+0000 7f7dd9c2e640 1 --2- 172.21.15.5:0/3231127839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 0x5609a228b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.178+0000 7f7dda42f640 1 --2- 172.21.15.5:0/3231127839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 0x5609a2271510 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0006010 tx=0x7f7dd0086de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6662d068a93768f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.178+0000 7f7dbffff640 1 -- 172.21.15.5:0/3231127839 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0049040 con 0x5609a2268b60 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.178+0000 7f7dbffff640 1 -- 172.21.15.5:0/3231127839 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dd003b460 con 0x5609a2268b60 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.178+0000 7f7dbffff640 1 -- 172.21.15.5:0/3231127839 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00c5110 con 0x5609a2268b60 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.179+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3231127839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 msgr2=0x5609a2271510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.179+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/3231127839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 0x5609a2271510 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0006010 tx=0x7f7dd0086de0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.179+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3231127839 shutdown_connections 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.179+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/3231127839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a228be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.179+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/3231127839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2269480 0x5609a228b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.179+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/3231127839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2268b60 0x5609a2271510 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.179+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3231127839 >> 172.21.15.5:0/3231127839 conn(0x5609a22650f0 msgr2=0x5609a2272bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.179+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3231127839 shutdown_connections 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.179+0000 7f7ddea6c740 1 -- 172.21.15.5:0/3231127839 wait complete. 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.179+0000 7f7ddea6c740 1 -- start start 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7ddea6c740 1 Processor -- start 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7ddea6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 0x5609a2277e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7ddea6c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2269480 0x5609a2278350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7ddea6c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a22ed900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7ddea6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41b10a0 con 0x5609a2268b60 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7ddea6c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41b0910 con 0x5609a2269480 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7ddea6c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41adab0 con 0x5609a2273d40 2024-09-19T06:17:48.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7ddea6c740 10 client.0 _ll_register_callbacks cb 0x7f7dc4009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7dda42f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 0x5609a2277e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7dda42f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 0x5609a2277e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42056/0 (socket says 172.21.15.5:42056) 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7dda42f640 1 -- 172.21.15.5:0/4160789203 learned_addr learned my addr 172.21.15.5:0/4160789203 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7dd9c2e640 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2269480 0x5609a2278350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7ddac30640 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a22ed900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7dda42f640 1 -- 172.21.15.5:0/4160789203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 msgr2=0x5609a22ed900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7dda42f640 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a22ed900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7dda42f640 1 -- 172.21.15.5:0/4160789203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2269480 msgr2=0x5609a2278350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7dda42f640 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2269480 0x5609a2278350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.180+0000 7f7dda42f640 1 -- 172.21.15.5:0/4160789203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a22ede40 con 0x5609a2268b60 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.181+0000 7f7dda42f640 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 0x5609a2277e10 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00ed8b0 tx=0x7f7dd002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26aa9af97a9f2395 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.181+0000 7f7dbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:17:48.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.181+0000 7f7dbdffb640 1 -- 172.21.15.5:0/4160789203 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0049040 con 0x5609a2268b60 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.181+0000 7f7dbdffb640 1 -- 172.21.15.5:0/4160789203 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f7dd00b8b10 con 0x5609a2268b60 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.181+0000 7f7ddea6c740 1 -- 172.21.15.5:0/4160789203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5609a21d5d00 con 0x5609a2268b60 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.181+0000 7f7dbdffb640 1 -- 172.21.15.5:0/4160789203 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7dd00044e0 con 0x5609a2268b60 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.181+0000 7f7ddea6c740 10 client.19248 fetch_fsmap learned FSMap version 44 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.181+0000 7f7ddea6c740 1 -- 172.21.15.5:0/4160789203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5609a22ef000 con 0x5609a2268b60 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.182+0000 7f7dbdffb640 1 -- 172.21.15.5:0/4160789203 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5609a21d5d00 con 0x5609a2268b60 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.182+0000 7f7dbdffb640 1 -- 172.21.15.5:0/4160789203 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8402+0+0 (secure 0 0 0) 0x7f7dd00d5230 con 0x5609a2268b60 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.182+0000 7f7ddea6c740 10 client.19248 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.182+0000 7f7ddea6c740 10 client.19248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.182+0000 7f7ddea6c740 20 client.19248 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.182+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609a22ef760 0x5609a22f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.182+0000 7f7ddea6c740 4 client.19248 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:17:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.182+0000 7f7ddea6c740 1 -- 172.21.15.5:0/4160789203 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5609a2152110 con 0x5609a22ef760 2024-09-19T06:17:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.182+0000 7f7dd9c2e640 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609a22ef760 0x5609a22f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.183+0000 7f7dd9c2e640 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609a22ef760 0x5609a22f1b80 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0d682f7030996b1 server_cookie=92657de9c0c7855a in_seq=0 out_seq=0 2024-09-19T06:17:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.183+0000 7f7dbdffb640 10 client.19248 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.184+0000 7f7dbdffb640 1 -- 172.21.15.5:0/4160789203 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5609a2152110 con 0x5609a22ef760 2024-09-19T06:17:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.184+0000 7f7dbdffb640 10 client.19248 handle_command_reply: tid=0 2024-09-19T06:17:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.184+0000 7f7ddea6c740 1 client.19248 shutdown 2024-09-19T06:17:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.185+0000 7f7ddea6c740 1 -- 172.21.15.5:0/4160789203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 msgr2=0x5609a2277e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.185+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 0x5609a2277e10 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00ed8b0 tx=0x7f7dd002a410 comp rx=0 tx=0).stop 2024-09-19T06:17:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.185+0000 7f7ddea6c740 1 -- 172.21.15.5:0/4160789203 shutdown_connections 2024-09-19T06:17:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.185+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5609a22ef760 0x5609a22f1b80 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.185+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a2273d40 0x5609a22ed900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.185+0000 7f7dbdffb640 0 client.19248 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.185+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609a2269480 0x5609a2278350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.185+0000 7f7ddea6c740 1 --2- 172.21.15.5:0/4160789203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609a2268b60 0x5609a2277e10 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.185+0000 7f7ddea6c740 1 -- 172.21.15.5:0/4160789203 >> 172.21.15.5:0/4160789203 conn(0x5609a22650f0 msgr2=0x5609a226a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.185+0000 7f7ddea6c740 1 -- 172.21.15.5:0/4160789203 shutdown_connections 2024-09-19T06:17:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.185+0000 7f7ddea6c740 1 -- 172.21.15.5:0/4160789203 wait complete. 2024-09-19T06:17:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.185+0000 7f7ddea6c740 20 client.19248 trim_cache size 0 max 16384 2024-09-19T06:17:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:17:48.983 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:17:48.984 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "6b0b5cf1-6674-408d-8b08-cc618722bc1c", 2024-09-19T06:17:48.984 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:17:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:17:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.186+0000 7f7ddc693640 1 -- 172.21.15.5:0/431302366 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7dc00a33b0 msgr2=0x7f7dc00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.186+0000 7f7ddc693640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7dc00a33b0 0x7f7dc00a5870 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009fd0 tx=0x7f7dcc01a000 comp rx=0 tx=0).stop 2024-09-19T06:17:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.186+0000 7f7ddc693640 1 -- 172.21.15.5:0/431302366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dd41009d0 msgr2=0x7f7dd419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.186+0000 7f7ddc693640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dd41009d0 0x7f7dd419a6e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0005940 tx=0x7f7dd00042f0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.186+0000 7f7ddc693640 1 -- 172.21.15.5:0/431302366 shutdown_connections 2024-09-19T06:17:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.186+0000 7f7ddc693640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd410e4f0 0x7f7dd419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.186+0000 7f7ddc693640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7dc00a33b0 0x7f7dc00a5870 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.186+0000 7f7ddc693640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd4101380 0x7f7dd419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.186+0000 7f7ddc693640 1 --2- 172.21.15.5:0/431302366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dd41009d0 0x7f7dd419a6e0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:48.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.186+0000 7f7ddc693640 1 -- 172.21.15.5:0/431302366 >> 172.21.15.5:0/431302366 conn(0x7f7dd40fc7d0 msgr2=0x7f7dd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:48.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.187+0000 7f7ddc693640 1 -- 172.21.15.5:0/431302366 shutdown_connections 2024-09-19T06:17:48.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:48.187+0000 7f7ddc693640 1 -- 172.21.15.5:0/431302366 wait complete. 2024-09-19T06:17:49.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:48 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:17:49.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:48 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12195 11.25 MB/sec warmup 73 sec latency 145.451 ms 2024-09-19T06:17:49.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:17:49.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:17:49.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:17:49.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:17:49.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:17:49.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:17:49.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:17:49.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:49 smithi005 ceph-mon[30955]: pgmap v454: 97 pgs: 97 active+clean; 460 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2024-09-19T06:17:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:17:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:17:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:17:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:17:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:17:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:17:49.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:49.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:17:49.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:49 smithi049 ceph-mon[34640]: pgmap v454: 97 pgs: 97 active+clean; 460 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2024-09-19T06:17:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:17:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:17:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:17:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:17:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:17:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:50.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:50.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:50.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:17:50.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:50.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:17:50.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:17:50.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:49 smithi157 ceph-mon[35346]: pgmap v454: 97 pgs: 97 active+clean; 460 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2024-09-19T06:17:50.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12325 11.16 MB/sec warmup 74 sec latency 183.038 ms 2024-09-19T06:17:51.354 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:17:51.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12507 11.15 MB/sec warmup 75 sec latency 65.566 ms 2024-09-19T06:17:51.740 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:17:52.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:51 smithi005 ceph-mon[30955]: pgmap v455: 97 pgs: 97 active+clean; 460 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2024-09-19T06:17:52.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:51 smithi049 ceph-mon[34640]: pgmap v455: 97 pgs: 97 active+clean; 460 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2024-09-19T06:17:52.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:51 smithi157 ceph-mon[35346]: pgmap v455: 97 pgs: 97 active+clean; 460 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2024-09-19T06:17:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.301+0000 7f2714bb0640 1 -- 172.21.15.5:0/209133524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2710101de0 msgr2=0x7f27101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.301+0000 7f2714bb0640 1 --2- 172.21.15.5:0/209133524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2710101de0 0x7f27101021c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2700009900 tx=0x7f270002a410 comp rx=0 tx=0).stop 2024-09-19T06:17:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.301+0000 7f2714bb0640 1 -- 172.21.15.5:0/209133524 shutdown_connections 2024-09-19T06:17:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.301+0000 7f2714bb0640 1 --2- 172.21.15.5:0/209133524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710106f10 0x7f271010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.301+0000 7f2714bb0640 1 --2- 172.21.15.5:0/209133524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2710102790 0x7f27101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.301+0000 7f2714bb0640 1 --2- 172.21.15.5:0/209133524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2710101de0 0x7f27101021c0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.301+0000 7f2714bb0640 1 -- 172.21.15.5:0/209133524 >> 172.21.15.5:0/209133524 conn(0x7f27100fd8e0 msgr2=0x7f27100ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.301+0000 7f2714bb0640 1 -- 172.21.15.5:0/209133524 shutdown_connections 2024-09-19T06:17:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.301+0000 7f2714bb0640 1 -- 172.21.15.5:0/209133524 wait complete. 2024-09-19T06:17:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.302+0000 7f2714bb0640 1 Processor -- start 2024-09-19T06:17:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.302+0000 7f2714bb0640 1 -- start start 2024-09-19T06:17:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.302+0000 7f2714bb0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2710101de0 0x7f27100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.302+0000 7f2714bb0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710102790 0x7f2710074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.302+0000 7f2714bb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2710106f10 0x7f2710074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.302+0000 7f2714bb0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2710111e20 con 0x7f2710106f10 2024-09-19T06:17:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.302+0000 7f2714bb0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2710111ca0 con 0x7f2710101de0 2024-09-19T06:17:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.302+0000 7f2714bb0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2710111fa0 con 0x7f2710102790 2024-09-19T06:17:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2710101de0 0x7f27100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710102790 0x7f2710074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2710106f10 0x7f2710074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2710101de0 0x7f27100744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58880/0 (socket says 172.21.15.5:58880) 2024-09-19T06:17:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710102790 0x7f2710074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54710/0 (socket says 172.21.15.5:54710) 2024-09-19T06:17:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270e59c640 1 -- 172.21.15.5:0/2367345025 learned_addr learned my addr 172.21.15.5:0/2367345025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270e59c640 1 -- 172.21.15.5:0/2367345025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710102790 msgr2=0x7f2710074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270e59c640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710102790 0x7f2710074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270e59c640 1 -- 172.21.15.5:0/2367345025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2710106f10 msgr2=0x7f2710074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270e59c640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2710106f10 0x7f2710074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270e59c640 1 -- 172.21.15.5:0/2367345025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27101a6880 con 0x7f2710101de0 2024-09-19T06:17:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270ed9d640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2710106f10 0x7f2710074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:17:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.303+0000 7f270e59c640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2710101de0 0x7f27100744f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2700005940 tx=0x7f27000040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.304+0000 7f26ff7fe640 1 -- 172.21.15.5:0/2367345025 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270003f030 con 0x7f2710101de0 2024-09-19T06:17:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.304+0000 7f2714bb0640 1 -- 172.21.15.5:0/2367345025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27101a6b10 con 0x7f2710101de0 2024-09-19T06:17:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.304+0000 7f2714bb0640 1 -- 172.21.15.5:0/2367345025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27101a7040 con 0x7f2710101de0 2024-09-19T06:17:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.304+0000 7f26ff7fe640 1 -- 172.21.15.5:0/2367345025 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27000044c0 con 0x7f2710101de0 2024-09-19T06:17:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.304+0000 7f26ff7fe640 1 -- 172.21.15.5:0/2367345025 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700035070 con 0x7f2710101de0 2024-09-19T06:17:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.307+0000 7f26ff7fe640 1 -- 172.21.15.5:0/2367345025 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2700042030 con 0x7f2710101de0 2024-09-19T06:17:52.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.308+0000 7f26ff7fe640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26f80a3430 0x7f26f80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.308+0000 7f26ff7fe640 1 -- 172.21.15.5:0/2367345025 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f27000efbe0 con 0x7f2710101de0 2024-09-19T06:17:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.309+0000 7f270dd9b640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26f80a3430 0x7f26f80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.309+0000 7f270dd9b640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26f80a3430 0x7f26f80a58f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2710075f80 tx=0x7f2704005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f2716f89740 1 Processor -- start 2024-09-19T06:17:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f2716f89740 1 -- start start 2024-09-19T06:17:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f2716f89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 0x55e6830f7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f2716f89740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f7dc0 0x55e683102140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f2716f89740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e683102680 0x55e683104a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f2716f89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e68305d2e0 con 0x55e6830f74a0 2024-09-19T06:17:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f2716f89740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e683004ce0 con 0x55e683102680 2024-09-19T06:17:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f2716f89740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e6830592a0 con 0x55e6830f7dc0 2024-09-19T06:17:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f270e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 0x55e6830f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f270e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 0x55e6830f7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42100/0 (socket says 172.21.15.5:42100) 2024-09-19T06:17:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f270e59c640 1 -- 172.21.15.5:0/775458730 learned_addr learned my addr 172.21.15.5:0/775458730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f270dd9b640 1 --2- 172.21.15.5:0/775458730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f7dc0 0x55e683102140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.310+0000 7f270ed9d640 1 --2- 172.21.15.5:0/775458730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e683102680 0x55e683104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.311+0000 7f270e59c640 1 -- 172.21.15.5:0/775458730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f7dc0 msgr2=0x55e683102140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.311+0000 7f270e59c640 1 --2- 172.21.15.5:0/775458730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f7dc0 0x55e683102140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.311+0000 7f270e59c640 1 -- 172.21.15.5:0/775458730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e683102680 msgr2=0x55e683104a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.311+0000 7f270e59c640 1 --2- 172.21.15.5:0/775458730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e683102680 0x55e683104a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.311+0000 7f270e59c640 1 -- 172.21.15.5:0/775458730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e682fe9440 con 0x55e6830f74a0 2024-09-19T06:17:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.311+0000 7f270ed9d640 1 --2- 172.21.15.5:0/775458730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e683102680 0x55e683104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.311+0000 7f270e59c640 1 --2- 172.21.15.5:0/775458730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 0x55e6830f7880 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f27000b8300 tx=0x7f27000bb9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db85cdf4ed28d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.311+0000 7f26fcff9640 1 -- 172.21.15.5:0/775458730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700049040 con 0x55e6830f74a0 2024-09-19T06:17:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.311+0000 7f26fcff9640 1 -- 172.21.15.5:0/775458730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27000bbea0 con 0x55e6830f74a0 2024-09-19T06:17:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.311+0000 7f26fcff9640 1 -- 172.21.15.5:0/775458730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27000bc100 con 0x55e6830f74a0 2024-09-19T06:17:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.312+0000 7f2716f89740 1 -- 172.21.15.5:0/775458730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 msgr2=0x55e6830f7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.312+0000 7f2716f89740 1 --2- 172.21.15.5:0/775458730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 0x55e6830f7880 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f27000b8300 tx=0x7f27000bb9e0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.313+0000 7f2716f89740 1 -- 172.21.15.5:0/775458730 shutdown_connections 2024-09-19T06:17:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.313+0000 7f2716f89740 1 --2- 172.21.15.5:0/775458730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e683102680 0x55e683104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.313+0000 7f2716f89740 1 --2- 172.21.15.5:0/775458730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f7dc0 0x55e683102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.313+0000 7f2716f89740 1 --2- 172.21.15.5:0/775458730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 0x55e6830f7880 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.313+0000 7f2716f89740 1 -- 172.21.15.5:0/775458730 >> 172.21.15.5:0/775458730 conn(0x55e6830f3a30 msgr2=0x55e6830f3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.313+0000 7f2716f89740 1 -- 172.21.15.5:0/775458730 shutdown_connections 2024-09-19T06:17:52.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.313+0000 7f2716f89740 1 -- 172.21.15.5:0/775458730 wait complete. 2024-09-19T06:17:52.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.313+0000 7f2716f89740 1 -- start start 2024-09-19T06:17:52.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.313+0000 7f2716f89740 1 Processor -- start 2024-09-19T06:17:52.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f2716f89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 0x55e683101f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f2716f89740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6830f7dc0 0x55e6830ff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f2716f89740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e683102680 0x55e6830ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f2716f89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e683105010 con 0x55e6830f74a0 2024-09-19T06:17:52.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f2716f89740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e683018050 con 0x55e6830f7dc0 2024-09-19T06:17:52.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f2716f89740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e683105190 con 0x55e683102680 2024-09-19T06:17:52.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f270e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 0x55e683101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f270e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 0x55e683101f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42104/0 (socket says 172.21.15.5:42104) 2024-09-19T06:17:52.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f270e59c640 1 -- 172.21.15.5:0/2092826029 learned_addr learned my addr 172.21.15.5:0/2092826029 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:52.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f2716f89740 10 client.0 _ll_register_callbacks cb 0x55e683058160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:52.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f270ed9d640 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e683102680 0x55e6830ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f270dd9b640 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6830f7dc0 0x55e6830ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f270e59c640 1 -- 172.21.15.5:0/2092826029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e683102680 msgr2=0x55e6830ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:52.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f270e59c640 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e683102680 0x55e6830ff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f270e59c640 1 -- 172.21.15.5:0/2092826029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6830f7dc0 msgr2=0x55e6830ff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:52.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f270e59c640 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6830f7dc0 0x55e6830ff220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f270e59c640 1 -- 172.21.15.5:0/2092826029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e682dce3c0 con 0x55e6830f74a0 2024-09-19T06:17:52.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.314+0000 7f270dd9b640 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6830f7dc0 0x55e6830ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:17:52.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.315+0000 7f270e59c640 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 0x55e683101f70 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f27000bbba0 tx=0x7f27000bb900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8272fe488edf95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:52.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.315+0000 7f26f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:17:52.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.315+0000 7f26f5ffb640 1 -- 172.21.15.5:0/2092826029 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700049070 con 0x55e6830f74a0 2024-09-19T06:17:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.315+0000 7f26f5ffb640 1 -- 172.21.15.5:0/2092826029 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f27000bc100 con 0x55e6830f74a0 2024-09-19T06:17:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.315+0000 7f2716f89740 1 -- 172.21.15.5:0/2092826029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27000bbea0 con 0x55e6830f74a0 2024-09-19T06:17:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.315+0000 7f26f5ffb640 1 -- 172.21.15.5:0/2092826029 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f27000c4b10 con 0x55e6830f74a0 2024-09-19T06:17:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.315+0000 7f2716f89740 10 client.19260 fetch_fsmap learned FSMap version 45 2024-09-19T06:17:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.315+0000 7f2716f89740 1 -- 172.21.15.5:0/2092826029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e683105010 con 0x55e6830f74a0 2024-09-19T06:17:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.315+0000 7f26f5ffb640 1 -- 172.21.15.5:0/2092826029 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27000bc340 con 0x55e6830f74a0 2024-09-19T06:17:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.316+0000 7f26f5ffb640 1 -- 172.21.15.5:0/2092826029 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8406+0+0 (secure 0 0 0) 0x7f27000dace0 con 0x55e6830f74a0 2024-09-19T06:17:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.316+0000 7f2716f89740 10 client.19260 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T06:17:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.316+0000 7f2716f89740 10 client.19260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.316+0000 7f2716f89740 20 client.19260 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.316+0000 7f2716f89740 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e68311a280 0x55e68311c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.316+0000 7f2716f89740 4 client.19260 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:17:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.316+0000 7f2716f89740 1 -- 172.21.15.5:0/2092826029 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f27000bbea0 con 0x55e68311a280 2024-09-19T06:17:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.316+0000 7f270dd9b640 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e68311a280 0x55e68311c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.316+0000 7f270dd9b640 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e68311a280 0x55e68311c670 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab9264cd81fcc81 server_cookie=a30300b8a83f533b in_seq=0 out_seq=0 2024-09-19T06:17:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.316+0000 7f26f5ffb640 10 client.19260 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.317+0000 7f26f5ffb640 1 -- 172.21.15.5:0/2092826029 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f27000bbea0 con 0x55e68311a280 2024-09-19T06:17:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.317+0000 7f26f5ffb640 10 client.19260 handle_command_reply: tid=0 2024-09-19T06:17:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.317+0000 7f2716f89740 1 client.19260 shutdown 2024-09-19T06:17:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.317+0000 7f2716f89740 1 -- 172.21.15.5:0/2092826029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 msgr2=0x55e683101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.317+0000 7f2716f89740 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 0x55e683101f70 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f27000bbba0 tx=0x7f27000bb900 comp rx=0 tx=0).stop 2024-09-19T06:17:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.317+0000 7f2716f89740 1 -- 172.21.15.5:0/2092826029 shutdown_connections 2024-09-19T06:17:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.317+0000 7f2716f89740 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e68311a280 0x55e68311c670 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.317+0000 7f2716f89740 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e683102680 0x55e6830ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.317+0000 7f2716f89740 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6830f7dc0 0x55e6830ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.317+0000 7f2716f89740 1 --2- 172.21.15.5:0/2092826029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f74a0 0x55e683101f70 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.317+0000 7f26f5ffb640 0 client.19260 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.317+0000 7f2716f89740 1 -- 172.21.15.5:0/2092826029 >> 172.21.15.5:0/2092826029 conn(0x55e6830f3a30 msgr2=0x55e6830f9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.318+0000 7f2716f89740 1 -- 172.21.15.5:0/2092826029 shutdown_connections 2024-09-19T06:17:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.318+0000 7f2716f89740 1 -- 172.21.15.5:0/2092826029 wait complete. 2024-09-19T06:17:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.318+0000 7f2716f89740 20 client.19260 trim_cache size 0 max 16384 2024-09-19T06:17:52.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.346+0000 7f2716f89740 1 Processor -- start 2024-09-19T06:17:52.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.347+0000 7f2716f89740 1 -- start start 2024-09-19T06:17:52.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.347+0000 7f2716f89740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6830f74a0 0x55e6830fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.347+0000 7f2716f89740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f7dc0 0x55e683121ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.347+0000 7f2716f89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e683102680 0x55e6831223e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.347+0000 7f2716f89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26f000e800 con 0x55e683102680 2024-09-19T06:17:52.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.347+0000 7f2716f89740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f26f00104f0 con 0x55e6830f74a0 2024-09-19T06:17:52.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.347+0000 7f2716f89740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26f000ef60 con 0x55e6830f7dc0 2024-09-19T06:17:52.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.347+0000 7f270ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e683102680 0x55e6831223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.347+0000 7f270ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e683102680 0x55e6831223e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42116/0 (socket says 172.21.15.5:42116) 2024-09-19T06:17:52.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.347+0000 7f270ed9d640 1 -- 172.21.15.5:0/850122255 learned_addr learned my addr 172.21.15.5:0/850122255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:52.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.347+0000 7f270e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6830f74a0 0x55e6830fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.348+0000 7f270e59c640 1 -- 172.21.15.5:0/850122255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f7dc0 msgr2=0x55e683121ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:52.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.348+0000 7f270e59c640 1 --2- 172.21.15.5:0/850122255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f7dc0 0x55e683121ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.348+0000 7f270e59c640 1 -- 172.21.15.5:0/850122255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e683102680 msgr2=0x55e6831223e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:52.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.348+0000 7f270e59c640 1 --2- 172.21.15.5:0/850122255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e683102680 0x55e6831223e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.348+0000 7f270e59c640 1 -- 172.21.15.5:0/850122255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6830fd770 con 0x55e6830f74a0 2024-09-19T06:17:52.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.348+0000 7f270e59c640 1 --2- 172.21.15.5:0/850122255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6830f74a0 0x55e6830fdaf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f27000ed910 tx=0x7f27000dc0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7854d07a2f34b696 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:52.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.348+0000 7f26f7fff640 1 -- 172.21.15.5:0/850122255 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700049070 con 0x55e6830f74a0 2024-09-19T06:17:52.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.348+0000 7f26f7fff640 1 -- 172.21.15.5:0/850122255 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27000a3020 con 0x55e6830f74a0 2024-09-19T06:17:52.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.348+0000 7f26f7fff640 1 -- 172.21.15.5:0/850122255 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27000baed0 con 0x55e6830f74a0 2024-09-19T06:17:52.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.349+0000 7f2716f89740 1 -- 172.21.15.5:0/850122255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6830f74a0 msgr2=0x55e6830fdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:52.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.349+0000 7f2716f89740 1 --2- 172.21.15.5:0/850122255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6830f74a0 0x55e6830fdaf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f27000ed910 tx=0x7f27000dc0c0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.350+0000 7f2716f89740 1 -- 172.21.15.5:0/850122255 shutdown_connections 2024-09-19T06:17:52.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.350+0000 7f2716f89740 1 --2- 172.21.15.5:0/850122255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e683102680 0x55e6831223e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.350+0000 7f2716f89740 1 --2- 172.21.15.5:0/850122255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f7dc0 0x55e683121ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.350+0000 7f2716f89740 1 --2- 172.21.15.5:0/850122255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6830f74a0 0x55e6830fdaf0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.350+0000 7f2716f89740 1 -- 172.21.15.5:0/850122255 >> 172.21.15.5:0/850122255 conn(0x55e6830f3a30 msgr2=0x55e6830f2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:52.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.350+0000 7f2716f89740 1 -- 172.21.15.5:0/850122255 shutdown_connections 2024-09-19T06:17:52.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.350+0000 7f2716f89740 1 -- 172.21.15.5:0/850122255 wait complete. 2024-09-19T06:17:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f2716f89740 1 -- start start 2024-09-19T06:17:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f2716f89740 1 Processor -- start 2024-09-19T06:17:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f2716f89740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f74a0 0x55e683113390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f2716f89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f7dc0 0x55e6831106e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f2716f89740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e683102680 0x55e683110c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f2716f89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26f000fe20 con 0x55e6830f7dc0 2024-09-19T06:17:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f7dc0 0x55e6831106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f2716f89740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f26f000f690 con 0x55e683102680 2024-09-19T06:17:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f2716f89740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26f000c800 con 0x55e6830f74a0 2024-09-19T06:17:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f270e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f74a0 0x55e683113390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f7dc0 0x55e6831106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42124/0 (socket says 172.21.15.5:42124) 2024-09-19T06:17:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f270dd9b640 1 -- 172.21.15.5:0/1418739270 learned_addr learned my addr 172.21.15.5:0/1418739270 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:17:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.351+0000 7f2716f89740 10 client.0 _ll_register_callbacks cb 0x7f27080099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:17:52.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.352+0000 7f270ed9d640 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e683102680 0x55e683110c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.352+0000 7f270dd9b640 1 -- 172.21.15.5:0/1418739270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f74a0 msgr2=0x55e683113390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.352+0000 7f270dd9b640 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f74a0 0x55e683113390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.352+0000 7f270dd9b640 1 -- 172.21.15.5:0/1418739270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e683102680 msgr2=0x55e683110c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:17:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.352+0000 7f270dd9b640 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e683102680 0x55e683110c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.352+0000 7f270dd9b640 1 -- 172.21.15.5:0/1418739270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e683111450 con 0x55e6830f7dc0 2024-09-19T06:17:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.352+0000 7f270ed9d640 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e683102680 0x55e683110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:17:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.352+0000 7f270dd9b640 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f7dc0 0x55e6831106e0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x55e6830f4890 tx=0x7f27040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257bb47e231f502b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:17:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.352+0000 7f26f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:17:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.353+0000 7f26f5ffb640 1 -- 172.21.15.5:0/1418739270 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270402d020 con 0x55e6830f7dc0 2024-09-19T06:17:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.353+0000 7f26f5ffb640 1 -- 172.21.15.5:0/1418739270 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7f270401dcd0 con 0x55e6830f7dc0 2024-09-19T06:17:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.353+0000 7f2716f89740 1 -- 172.21.15.5:0/1418739270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27000a3020 con 0x55e6830f7dc0 2024-09-19T06:17:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.354+0000 7f26f5ffb640 1 -- 172.21.15.5:0/1418739270 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f2704038050 con 0x55e6830f7dc0 2024-09-19T06:17:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.354+0000 7f2716f89740 10 client.19266 fetch_fsmap learned FSMap version 45 2024-09-19T06:17:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.354+0000 7f2716f89740 1 -- 172.21.15.5:0/1418739270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e683184f60 con 0x55e6830f7dc0 2024-09-19T06:17:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.354+0000 7f26f5ffb640 1 -- 172.21.15.5:0/1418739270 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2704003aa0 con 0x55e6830f7dc0 2024-09-19T06:17:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.354+0000 7f26f5ffb640 1 -- 172.21.15.5:0/1418739270 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8406+0+0 (secure 0 0 0) 0x7f27040286e0 con 0x55e6830f7dc0 2024-09-19T06:17:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.354+0000 7f2716f89740 10 client.19266 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T06:17:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.354+0000 7f2716f89740 10 client.19266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:17:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.354+0000 7f2716f89740 20 client.19266 populate_metadata read hostname 'smithi005' 2024-09-19T06:17:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.355+0000 7f2716f89740 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e6831857d0 0x55e683187bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:17:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.355+0000 7f2716f89740 4 client.19266 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:17:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.355+0000 7f2716f89740 1 -- 172.21.15.5:0/1418739270 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f27000a3020 con 0x55e6831857d0 2024-09-19T06:17:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.355+0000 7f270e59c640 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e6831857d0 0x55e683187bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:17:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.356+0000 7f270e59c640 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e6831857d0 0x55e683187bf0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=651452e69e4d9a0e server_cookie=a3f5f0bd3f4b0301 in_seq=0 out_seq=0 2024-09-19T06:17:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.356+0000 7f26f5ffb640 10 client.19266 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.356+0000 7f26f5ffb640 1 -- 172.21.15.5:0/1418739270 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f27000a3020 con 0x55e6831857d0 2024-09-19T06:17:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.356+0000 7f26f5ffb640 10 client.19266 handle_command_reply: tid=0 2024-09-19T06:17:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.356+0000 7f2716f89740 1 client.19266 shutdown 2024-09-19T06:17:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.357+0000 7f2716f89740 1 -- 172.21.15.5:0/1418739270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f7dc0 msgr2=0x55e6831106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.357+0000 7f2716f89740 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f7dc0 0x55e6831106e0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x55e6830f4890 tx=0x7f27040046c0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.358+0000 7f2716f89740 1 -- 172.21.15.5:0/1418739270 shutdown_connections 2024-09-19T06:17:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.358+0000 7f2716f89740 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e6831857d0 0x55e683187bf0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.358+0000 7f2716f89740 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e683102680 0x55e683110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.358+0000 7f2716f89740 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6830f7dc0 0x55e6831106e0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.358+0000 7f2716f89740 1 --2- 172.21.15.5:0/1418739270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6830f74a0 0x55e683113390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.358+0000 7f2716f89740 1 -- 172.21.15.5:0/1418739270 >> 172.21.15.5:0/1418739270 conn(0x55e6830f3a30 msgr2=0x55e683100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.358+0000 7f26f5ffb640 0 client.19266 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:17:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.358+0000 7f2716f89740 1 -- 172.21.15.5:0/1418739270 shutdown_connections 2024-09-19T06:17:52.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.359+0000 7f2716f89740 1 -- 172.21.15.5:0/1418739270 wait complete. 2024-09-19T06:17:52.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.359+0000 7f2716f89740 20 client.19266 trim_cache size 0 max 16384 2024-09-19T06:17:52.361 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:17:52.361 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:17:52.361 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:17:52.361 INFO:teuthology.orchestra.run.smithi005.stdout: "6b0b5cf1-6674-408d-8b08-cc618722bc1c": { 2024-09-19T06:17:52.361 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:17:52.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "6b0b5cf1-6674-408d-8b08-cc618722bc1c", 2024-09-19T06:17:52.361 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:17:52.361 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:17:52.361 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:17:52.361 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:17:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.360+0000 7f2714bb0640 1 -- 172.21.15.5:0/2367345025 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26f80a3430 msgr2=0x7f26f80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.360+0000 7f2714bb0640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26f80a3430 0x7f26f80a58f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2710075f80 tx=0x7f2704005f90 comp rx=0 tx=0).stop 2024-09-19T06:17:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.360+0000 7f2714bb0640 1 -- 172.21.15.5:0/2367345025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2710101de0 msgr2=0x7f27100744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:17:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.360+0000 7f2714bb0640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2710101de0 0x7f27100744f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2700005940 tx=0x7f27000040c0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.360+0000 7f2714bb0640 1 -- 172.21.15.5:0/2367345025 shutdown_connections 2024-09-19T06:17:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.360+0000 7f2714bb0640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2710106f10 0x7f2710074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.360+0000 7f2714bb0640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26f80a3430 0x7f26f80a58f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.360+0000 7f2714bb0640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710102790 0x7f2710074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.360+0000 7f2714bb0640 1 --2- 172.21.15.5:0/2367345025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2710101de0 0x7f27100744f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:17:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.360+0000 7f2714bb0640 1 -- 172.21.15.5:0/2367345025 >> 172.21.15.5:0/2367345025 conn(0x7f27100fd8e0 msgr2=0x7f27100fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:17:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.361+0000 7f2714bb0640 1 -- 172.21.15.5:0/2367345025 shutdown_connections 2024-09-19T06:17:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:17:52.361+0000 7f2714bb0640 1 -- 172.21.15.5:0/2367345025 wait complete. 2024-09-19T06:17:52.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12716 11.24 MB/sec warmup 76 sec latency 79.788 ms 2024-09-19T06:17:53.218 INFO:tasks.cephfs.filesystem:scrub status for tag:6b0b5cf1-6674-408d-8b08-cc618722bc1c - {'path': '/', 'tag': '6b0b5cf1-6674-408d-8b08-cc618722bc1c', 'options': 'recursive,force'} 2024-09-19T06:17:53.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12912 11.25 MB/sec warmup 77 sec latency 57.962 ms 2024-09-19T06:17:54.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 13087 11.26 MB/sec warmup 78 sec latency 66.228 ms 2024-09-19T06:17:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:54 smithi157 ceph-mon[35346]: pgmap v456: 97 pgs: 97 active+clean; 466 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 407 op/s 2024-09-19T06:17:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:54 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:17:54.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:54 smithi005 ceph-mon[30955]: pgmap v456: 97 pgs: 97 active+clean; 466 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 407 op/s 2024-09-19T06:17:54.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:54 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:17:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:54 smithi049 ceph-mon[34640]: pgmap v456: 97 pgs: 97 active+clean; 466 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 407 op/s 2024-09-19T06:17:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:54 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:17:55.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 13279 11.26 MB/sec warmup 79 sec latency 66.303 ms 2024-09-19T06:17:56.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 13451 11.27 MB/sec warmup 80 sec latency 53.256 ms 2024-09-19T06:17:56.696 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:56 smithi049 ceph-mon[34640]: pgmap v457: 97 pgs: 97 active+clean; 466 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 304 op/s 2024-09-19T06:17:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:56 smithi157 ceph-mon[35346]: pgmap v457: 97 pgs: 97 active+clean; 466 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 304 op/s 2024-09-19T06:17:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:56 smithi005 ceph-mon[30955]: pgmap v457: 97 pgs: 97 active+clean; 466 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 304 op/s 2024-09-19T06:17:57.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 13594 11.21 MB/sec warmup 81 sec latency 61.439 ms 2024-09-19T06:17:58.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 13763 11.21 MB/sec warmup 82 sec latency 61.091 ms 2024-09-19T06:17:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:58 smithi157 ceph-mon[35346]: pgmap v458: 97 pgs: 97 active+clean; 476 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 428 op/s 2024-09-19T06:17:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:58 smithi005 ceph-mon[30955]: pgmap v458: 97 pgs: 97 active+clean; 476 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 428 op/s 2024-09-19T06:17:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:58 smithi049 ceph-mon[34640]: pgmap v458: 97 pgs: 97 active+clean; 476 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 428 op/s 2024-09-19T06:17:59.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 13901 11.12 MB/sec warmup 83 sec latency 61.583 ms 2024-09-19T06:17:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:59 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:17:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:59 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:17:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:17:59 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:17:59.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:59 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:17:59.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:59 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:17:59.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:17:59 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:17:59.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:59 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:17:59.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:59 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:17:59.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:17:59 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:18:00.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 14066 11.08 MB/sec warmup 84 sec latency 56.614 ms 2024-09-19T06:18:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.732+0000 7fd4536af640 1 -- 172.21.15.5:0/3148735696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c101ac0 msgr2=0x7fd44c101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.732+0000 7fd4536af640 1 --2- 172.21.15.5:0/3148735696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c101ac0 0x7fd44c101f20 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009930 tx=0x7fd43c02a420 comp rx=0 tx=0).stop 2024-09-19T06:18:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.733+0000 7fd4536af640 1 -- 172.21.15.5:0/3148735696 shutdown_connections 2024-09-19T06:18:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.733+0000 7fd4536af640 1 --2- 172.21.15.5:0/3148735696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd44c102460 0x7fd44c10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.733+0000 7fd4536af640 1 --2- 172.21.15.5:0/3148735696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c101ac0 0x7fd44c101f20 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.733+0000 7fd4536af640 1 --2- 172.21.15.5:0/3148735696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c107ad0 0x7fd44c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.733+0000 7fd4536af640 1 -- 172.21.15.5:0/3148735696 >> 172.21.15.5:0/3148735696 conn(0x7fd44c0fd7a0 msgr2=0x7fd44c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.733+0000 7fd4536af640 1 -- 172.21.15.5:0/3148735696 shutdown_connections 2024-09-19T06:18:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.733+0000 7fd4536af640 1 -- 172.21.15.5:0/3148735696 wait complete. 2024-09-19T06:18:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.734+0000 7fd4536af640 1 Processor -- start 2024-09-19T06:18:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.734+0000 7fd4536af640 1 -- start start 2024-09-19T06:18:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.734+0000 7fd4536af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c101ac0 0x7fd44c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.734+0000 7fd4536af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c102460 0x7fd44c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.734+0000 7fd4536af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd44c107ad0 0x7fd44c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.734+0000 7fd4536af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c111dd0 con 0x7fd44c102460 2024-09-19T06:18:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.734+0000 7fd4536af640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c111c50 con 0x7fd44c101ac0 2024-09-19T06:18:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.734+0000 7fd4536af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c111f50 con 0x7fd44c107ad0 2024-09-19T06:18:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.734+0000 7fd450c4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c102460 0x7fd44c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.735+0000 7fd45144b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c101ac0 0x7fd44c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.735+0000 7fd450c4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c102460 0x7fd44c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46712/0 (socket says 172.21.15.5:46712) 2024-09-19T06:18:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.735+0000 7fd450c4a640 1 -- 172.21.15.5:0/3900402033 learned_addr learned my addr 172.21.15.5:0/3900402033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.735+0000 7fd451c4c640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd44c107ad0 0x7fd44c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.735+0000 7fd450c4a640 1 -- 172.21.15.5:0/3900402033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd44c107ad0 msgr2=0x7fd44c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.735+0000 7fd450c4a640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd44c107ad0 0x7fd44c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.735+0000 7fd450c4a640 1 -- 172.21.15.5:0/3900402033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c101ac0 msgr2=0x7fd44c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.735+0000 7fd450c4a640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c101ac0 0x7fd44c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.735+0000 7fd450c4a640 1 -- 172.21.15.5:0/3900402033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c19d350 con 0x7fd44c102460 2024-09-19T06:18:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.735+0000 7fd45144b640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c101ac0 0x7fd44c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.735+0000 7fd450c4a640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c102460 0x7fd44c1988d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd43c005ab0 tx=0x7fd43c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.736+0000 7fd4427fc640 1 -- 172.21.15.5:0/3900402033 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c03f070 con 0x7fd44c102460 2024-09-19T06:18:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.736+0000 7fd4536af640 1 -- 172.21.15.5:0/3900402033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44c19d5e0 con 0x7fd44c102460 2024-09-19T06:18:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.736+0000 7fd4427fc640 1 -- 172.21.15.5:0/3900402033 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd43c031b50 con 0x7fd44c102460 2024-09-19T06:18:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.736+0000 7fd4427fc640 1 -- 172.21.15.5:0/3900402033 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c035030 con 0x7fd44c102460 2024-09-19T06:18:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.736+0000 7fd4536af640 1 -- 172.21.15.5:0/3900402033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44c074420 con 0x7fd44c102460 2024-09-19T06:18:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.739+0000 7fd45643f740 1 Processor -- start 2024-09-19T06:18:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.739+0000 7fd45643f740 1 -- start start 2024-09-19T06:18:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd45643f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55562870daf0 0x55562870ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd45643f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870e410 0x555628718770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd45643f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555628718cb0 0x55562871b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd45643f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5556286722f0 con 0x555628718cb0 2024-09-19T06:18:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd45643f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55562861a270 con 0x55562870e410 2024-09-19T06:18:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd45643f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55562866e2b0 con 0x55562870daf0 2024-09-19T06:18:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd45144b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870e410 0x555628718770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd451c4c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55562870daf0 0x55562870ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd451c4c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55562870daf0 0x55562870ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59834/0 (socket says 172.21.15.5:59834) 2024-09-19T06:18:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd45144b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870e410 0x555628718770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48118/0 (socket says 172.21.15.5:48118) 2024-09-19T06:18:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd451c4c640 1 -- 172.21.15.5:0/986134723 learned_addr learned my addr 172.21.15.5:0/986134723 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd4427fc640 1 -- 172.21.15.5:0/3900402033 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd43c042070 con 0x7fd44c102460 2024-09-19T06:18:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.740+0000 7fd450c4a640 1 --2- 172.21.15.5:0/986134723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555628718cb0 0x55562871b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.741+0000 7fd45144b640 1 -- 172.21.15.5:0/986134723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55562870daf0 msgr2=0x55562870ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.741+0000 7fd45144b640 1 --2- 172.21.15.5:0/986134723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55562870daf0 0x55562870ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.741+0000 7fd45144b640 1 -- 172.21.15.5:0/986134723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555628718cb0 msgr2=0x55562871b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.741+0000 7fd45144b640 1 --2- 172.21.15.5:0/986134723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555628718cb0 0x55562871b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.741+0000 7fd45144b640 1 -- 172.21.15.5:0/986134723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555628619340 con 0x55562870e410 2024-09-19T06:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.741+0000 7fd450c4a640 1 --2- 172.21.15.5:0/986134723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555628718cb0 0x55562871b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.742+0000 7fd4427fc640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd4380a3460 0x7fd4380a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.742+0000 7fd4427fc640 1 -- 172.21.15.5:0/3900402033 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7fd43c0f02e0 con 0x7fd44c102460 2024-09-19T06:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.742+0000 7fd451c4c640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd4380a3460 0x7fd4380a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.742+0000 7fd45144b640 1 --2- 172.21.15.5:0/986134723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870e410 0x555628718770 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd448009f50 tx=0x7fd4480118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a410c478041a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.742+0000 7fd4377fe640 1 -- 172.21.15.5:0/986134723 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448004030 con 0x55562870e410 2024-09-19T06:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.742+0000 7fd4377fe640 1 -- 172.21.15.5:0/986134723 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd448005020 con 0x55562870e410 2024-09-19T06:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.742+0000 7fd4377fe640 1 -- 172.21.15.5:0/986134723 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448004ce0 con 0x55562870e410 2024-09-19T06:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.743+0000 7fd45643f740 1 -- 172.21.15.5:0/986134723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870e410 msgr2=0x555628718770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.743+0000 7fd451c4c640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd4380a3460 0x7fd4380a5920 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd444009370 tx=0x7fd4440098a0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.743+0000 7fd45643f740 1 --2- 172.21.15.5:0/986134723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870e410 0x555628718770 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd448009f50 tx=0x7fd4480118d0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.743+0000 7fd45643f740 1 -- 172.21.15.5:0/986134723 shutdown_connections 2024-09-19T06:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.743+0000 7fd45643f740 1 --2- 172.21.15.5:0/986134723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555628718cb0 0x55562871b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.743+0000 7fd45643f740 1 --2- 172.21.15.5:0/986134723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870e410 0x555628718770 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.743+0000 7fd45643f740 1 --2- 172.21.15.5:0/986134723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55562870daf0 0x55562870ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.743+0000 7fd45643f740 1 -- 172.21.15.5:0/986134723 >> 172.21.15.5:0/986134723 conn(0x55562870a060 msgr2=0x55562870a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.743+0000 7fd45643f740 1 -- 172.21.15.5:0/986134723 shutdown_connections 2024-09-19T06:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.743+0000 7fd45643f740 1 -- 172.21.15.5:0/986134723 wait complete. 2024-09-19T06:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.744+0000 7fd45643f740 1 -- start start 2024-09-19T06:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.744+0000 7fd45643f740 1 Processor -- start 2024-09-19T06:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.744+0000 7fd45643f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 0x555628715bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.744+0000 7fd45643f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 0x555628716100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.744+0000 7fd45643f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555628718cb0 0x555628716640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.744+0000 7fd45643f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55562871b880 con 0x55562870e410 2024-09-19T06:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd45643f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5556285fb290 con 0x55562870daf0 2024-09-19T06:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd45643f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55562871ba00 con 0x555628718cb0 2024-09-19T06:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd45643f740 10 client.0 _ll_register_callbacks cb 0x55562862c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd450c4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 0x555628716100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd45144b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 0x555628715bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd450c4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 0x555628716100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46724/0 (socket says 172.21.15.5:46724) 2024-09-19T06:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd451c4c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555628718cb0 0x555628716640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd450c4a640 1 -- 172.21.15.5:0/1264683626 learned_addr learned my addr 172.21.15.5:0/1264683626 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd45144b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 0x555628715bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48132/0 (socket says 172.21.15.5:48132) 2024-09-19T06:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd450c4a640 1 -- 172.21.15.5:0/1264683626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555628718cb0 msgr2=0x555628716640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd450c4a640 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555628718cb0 0x555628716640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd450c4a640 1 -- 172.21.15.5:0/1264683626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 msgr2=0x555628715bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd450c4a640 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 0x555628715bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd450c4a640 1 -- 172.21.15.5:0/1264683626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555628385210 con 0x55562870e410 2024-09-19T06:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd451c4c640 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555628718cb0 0x555628716640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd45144b640 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 0x555628715bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd450c4a640 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 0x555628716100 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd43c031440 tx=0x7fd43c0bc080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be24e8f4d9e3860d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.745+0000 7fd434ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.746+0000 7fd434ff9640 1 -- 172.21.15.5:0/1264683626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0bc540 con 0x55562870e410 2024-09-19T06:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.746+0000 7fd434ff9640 1 -- 172.21.15.5:0/1264683626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7fd43c0b8bb0 con 0x55562870e410 2024-09-19T06:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.746+0000 7fd45643f740 1 -- 172.21.15.5:0/1264683626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd448005020 con 0x55562870e410 2024-09-19T06:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.746+0000 7fd434ff9640 1 -- 172.21.15.5:0/1264683626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd43c0c08f0 con 0x55562870e410 2024-09-19T06:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.746+0000 7fd45643f740 10 client.19275 fetch_fsmap learned FSMap version 46 2024-09-19T06:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.746+0000 7fd45643f740 1 -- 172.21.15.5:0/1264683626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555628451620 con 0x55562870e410 2024-09-19T06:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.746+0000 7fd434ff9640 1 -- 172.21.15.5:0/1264683626 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd448005020 con 0x55562870e410 2024-09-19T06:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.746+0000 7fd434ff9640 1 -- 172.21.15.5:0/1264683626 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8410+0+0 (secure 0 0 0) 0x7fd43c0c4d20 con 0x55562870e410 2024-09-19T06:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.746+0000 7fd45643f740 10 client.19275 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T06:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.746+0000 7fd45643f740 10 client.19275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.746+0000 7fd45643f740 20 client.19275 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.746+0000 7fd45643f740 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555628728e20 0x55562872b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.747+0000 7fd45643f740 4 client.19275 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.747+0000 7fd45643f740 1 -- 172.21.15.5:0/1264683626 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55562867fa20 con 0x555628728e20 2024-09-19T06:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.747+0000 7fd45144b640 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555628728e20 0x55562872b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.747+0000 7fd45144b640 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555628728e20 0x55562872b2c0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc772f6a17f75d3 server_cookie=c24990194e03ca01 in_seq=0 out_seq=0 2024-09-19T06:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.747+0000 7fd434ff9640 10 client.19275 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.748+0000 7fd434ff9640 1 -- 172.21.15.5:0/1264683626 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55562867fa20 con 0x555628728e20 2024-09-19T06:18:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.748+0000 7fd434ff9640 10 client.19275 handle_command_reply: tid=0 2024-09-19T06:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.749+0000 7fd45643f740 1 client.19275 shutdown 2024-09-19T06:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.749+0000 7fd45643f740 1 -- 172.21.15.5:0/1264683626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 msgr2=0x555628716100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.749+0000 7fd45643f740 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 0x555628716100 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd43c031440 tx=0x7fd43c0bc080 comp rx=0 tx=0).stop 2024-09-19T06:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.749+0000 7fd45643f740 1 -- 172.21.15.5:0/1264683626 shutdown_connections 2024-09-19T06:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.749+0000 7fd45643f740 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555628728e20 0x55562872b2c0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.749+0000 7fd45643f740 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555628718cb0 0x555628716640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.749+0000 7fd45643f740 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 0x555628716100 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.749+0000 7fd45643f740 1 --2- 172.21.15.5:0/1264683626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 0x555628715bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.749+0000 7fd45643f740 1 -- 172.21.15.5:0/1264683626 >> 172.21.15.5:0/1264683626 conn(0x55562870a060 msgr2=0x55562870fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.749+0000 7fd434ff9640 0 client.19275 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.749+0000 7fd45643f740 1 -- 172.21.15.5:0/1264683626 shutdown_connections 2024-09-19T06:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.750+0000 7fd45643f740 1 -- 172.21.15.5:0/1264683626 wait complete. 2024-09-19T06:18:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.750+0000 7fd45643f740 20 client.19275 trim_cache size 0 max 16384 2024-09-19T06:18:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.775+0000 7fd45643f740 1 Processor -- start 2024-09-19T06:18:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.775+0000 7fd45643f740 1 -- start start 2024-09-19T06:18:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45643f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870daf0 0x555628716470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45643f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55562870e410 0x555628717b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45643f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555628718cb0 0x555628730bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45643f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b3ea0 con 0x55562870daf0 2024-09-19T06:18:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45643f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b5bc0 con 0x555628718cb0 2024-09-19T06:18:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45643f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b4630 con 0x55562870e410 2024-09-19T06:18:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45144b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870daf0 0x555628716470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45144b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870daf0 0x555628716470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46738/0 (socket says 172.21.15.5:46738) 2024-09-19T06:18:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45144b640 1 -- 172.21.15.5:0/2489556505 learned_addr learned my addr 172.21.15.5:0/2489556505 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd451c4c640 1 --2- 172.21.15.5:0/2489556505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555628718cb0 0x555628730bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd450c4a640 1 --2- 172.21.15.5:0/2489556505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55562870e410 0x555628717b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45144b640 1 -- 172.21.15.5:0/2489556505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55562870e410 msgr2=0x555628717b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45144b640 1 --2- 172.21.15.5:0/2489556505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55562870e410 0x555628717b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45144b640 1 -- 172.21.15.5:0/2489556505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555628718cb0 msgr2=0x555628730bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45144b640 1 --2- 172.21.15.5:0/2489556505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555628718cb0 0x555628730bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.776+0000 7fd45144b640 1 -- 172.21.15.5:0/2489556505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556287140f0 con 0x55562870daf0 2024-09-19T06:18:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.777+0000 7fd45144b640 1 --2- 172.21.15.5:0/2489556505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870daf0 0x555628716470 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd448009ef0 tx=0x7fd4480096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0104d2d3c3bb58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.777+0000 7fd436ffd640 1 -- 172.21.15.5:0/2489556505 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480042d0 con 0x55562870daf0 2024-09-19T06:18:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.777+0000 7fd436ffd640 1 -- 172.21.15.5:0/2489556505 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd448004f50 con 0x55562870daf0 2024-09-19T06:18:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.777+0000 7fd436ffd640 1 -- 172.21.15.5:0/2489556505 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480035f0 con 0x55562870daf0 2024-09-19T06:18:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.778+0000 7fd45643f740 1 -- 172.21.15.5:0/2489556505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870daf0 msgr2=0x555628716470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.778+0000 7fd45643f740 1 --2- 172.21.15.5:0/2489556505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870daf0 0x555628716470 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd448009ef0 tx=0x7fd4480096c0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.778+0000 7fd45643f740 1 -- 172.21.15.5:0/2489556505 shutdown_connections 2024-09-19T06:18:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.778+0000 7fd45643f740 1 --2- 172.21.15.5:0/2489556505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555628718cb0 0x555628730bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.778+0000 7fd45643f740 1 --2- 172.21.15.5:0/2489556505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55562870e410 0x555628717b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.778+0000 7fd45643f740 1 --2- 172.21.15.5:0/2489556505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870daf0 0x555628716470 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.778+0000 7fd45643f740 1 -- 172.21.15.5:0/2489556505 >> 172.21.15.5:0/2489556505 conn(0x55562870a060 msgr2=0x555628717420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.778+0000 7fd45643f740 1 -- 172.21.15.5:0/2489556505 shutdown_connections 2024-09-19T06:18:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.778+0000 7fd45643f740 1 -- 172.21.15.5:0/2489556505 wait complete. 2024-09-19T06:18:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.778+0000 7fd45643f740 1 -- start start 2024-09-19T06:18:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.778+0000 7fd45643f740 1 Processor -- start 2024-09-19T06:18:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.779+0000 7fd45643f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 0x555628721000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.779+0000 7fd45643f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 0x555628721540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.779+0000 7fd45144b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 0x555628721000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.779+0000 7fd45144b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 0x555628721000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48152/0 (socket says 172.21.15.5:48152) 2024-09-19T06:18:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.779+0000 7fd450c4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 0x555628721540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.779+0000 7fd450c4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 0x555628721540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46748/0 (socket says 172.21.15.5:46748) 2024-09-19T06:18:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.779+0000 7fd45144b640 1 -- 172.21.15.5:0/1608864721 learned_addr learned my addr 172.21.15.5:0/1608864721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.779+0000 7fd45643f740 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555628718cb0 0x55562871c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.779+0000 7fd45643f740 1 -- 172.21.15.5:0/1608864721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b54f0 con 0x55562870e410 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.779+0000 7fd45643f740 1 -- 172.21.15.5:0/1608864721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b4d60 con 0x55562870daf0 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.779+0000 7fd45643f740 1 -- 172.21.15.5:0/1608864721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b1ea0 con 0x555628718cb0 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.779+0000 7fd45643f740 10 client.0 _ll_register_callbacks cb 0x7fd43c0b8890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.780+0000 7fd451c4c640 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555628718cb0 0x55562871c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.780+0000 7fd45144b640 1 -- 172.21.15.5:0/1608864721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555628718cb0 msgr2=0x55562871c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.780+0000 7fd45144b640 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555628718cb0 0x55562871c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.780+0000 7fd45144b640 1 -- 172.21.15.5:0/1608864721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 msgr2=0x555628721540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.780+0000 7fd45144b640 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 0x555628721540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.780+0000 7fd45144b640 1 -- 172.21.15.5:0/1608864721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55562871cdf0 con 0x55562870daf0 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.780+0000 7fd45144b640 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 0x555628721000 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd448011dd0 tx=0x7fd448009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbc68b78e0c609c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.780+0000 7fd434ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:18:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.780+0000 7fd434ff9640 1 -- 172.21.15.5:0/1608864721 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480044e0 con 0x55562870daf0 2024-09-19T06:18:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.780+0000 7fd434ff9640 1 -- 172.21.15.5:0/1608864721 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8550+0+0 (secure 0 0 0) 0x7fd44802c040 con 0x55562870daf0 2024-09-19T06:18:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.780+0000 7fd45643f740 1 -- 172.21.15.5:0/1608864721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd448004f50 con 0x55562870daf0 2024-09-19T06:18:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.780+0000 7fd434ff9640 1 -- 172.21.15.5:0/1608864721 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd448004aa0 con 0x55562870daf0 2024-09-19T06:18:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.781+0000 7fd45643f740 10 client.28679 fetch_fsmap learned FSMap version 46 2024-09-19T06:18:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.781+0000 7fd45643f740 1 -- 172.21.15.5:0/1608864721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5556287937f0 con 0x55562870daf0 2024-09-19T06:18:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.781+0000 7fd434ff9640 1 -- 172.21.15.5:0/1608864721 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd448004c40 con 0x55562870daf0 2024-09-19T06:18:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.781+0000 7fd434ff9640 1 -- 172.21.15.5:0/1608864721 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 46) ==== 8410+0+0 (secure 0 0 0) 0x7fd448019450 con 0x55562870daf0 2024-09-19T06:18:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.781+0000 7fd45643f740 10 client.28679 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T06:18:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.781+0000 7fd45643f740 10 client.28679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.781+0000 7fd45643f740 20 client.28679 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.781+0000 7fd45643f740 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555628793c50 0x555628796070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.781+0000 7fd45643f740 4 client.28679 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:18:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.781+0000 7fd45643f740 1 -- 172.21.15.5:0/1608864721 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fd448004f50 con 0x555628793c50 2024-09-19T06:18:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.781+0000 7fd450c4a640 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555628793c50 0x555628796070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.782+0000 7fd434ff9640 10 client.28679 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.782+0000 7fd450c4a640 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555628793c50 0x555628796070 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f3bcfa1228e66f server_cookie=356f490c648d3d39 in_seq=0 out_seq=0 2024-09-19T06:18:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:00 smithi157 ceph-mon[35346]: pgmap v459: 97 pgs: 97 active+clean; 476 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 3.8 MiB/s wr, 318 op/s 2024-09-19T06:18:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:00 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:18:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:00 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:18:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.860+0000 7fd434ff9640 1 -- 172.21.15.5:0/1608864721 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fd448004f50 con 0x555628793c50 2024-09-19T06:18:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.860+0000 7fd434ff9640 10 client.28679 handle_command_reply: tid=0 2024-09-19T06:18:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.860+0000 7fd45643f740 1 client.28679 shutdown 2024-09-19T06:18:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.860+0000 7fd45643f740 1 -- 172.21.15.5:0/1608864721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 msgr2=0x555628721000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.860+0000 7fd45643f740 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 0x555628721000 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd448011dd0 tx=0x7fd448009c20 comp rx=0 tx=0).stop 2024-09-19T06:18:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.860+0000 7fd45643f740 1 -- 172.21.15.5:0/1608864721 shutdown_connections 2024-09-19T06:18:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.860+0000 7fd45643f740 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555628793c50 0x555628796070 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.860+0000 7fd45643f740 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555628718cb0 0x55562871c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.860+0000 7fd45643f740 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55562870e410 0x555628721540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.860+0000 7fd434ff9640 0 client.28679 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.860+0000 7fd45643f740 1 --2- 172.21.15.5:0/1608864721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55562870daf0 0x555628721000 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.860+0000 7fd45643f740 1 -- 172.21.15.5:0/1608864721 >> 172.21.15.5:0/1608864721 conn(0x55562870a060 msgr2=0x55562870fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd45643f740 1 -- 172.21.15.5:0/1608864721 shutdown_connections 2024-09-19T06:18:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd45643f740 1 -- 172.21.15.5:0/1608864721 wait complete. 2024-09-19T06:18:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd45643f740 20 client.28679 trim_cache size 0 max 16384 2024-09-19T06:18:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:18:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:18:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:18:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:18:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd4536af640 1 -- 172.21.15.5:0/3900402033 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd4380a3460 msgr2=0x7fd4380a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd4536af640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd4380a3460 0x7fd4380a5920 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd444009370 tx=0x7fd4440098a0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd4536af640 1 -- 172.21.15.5:0/3900402033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c102460 msgr2=0x7fd44c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd4536af640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c102460 0x7fd44c1988d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd43c005ab0 tx=0x7fd43c003b90 comp rx=0 tx=0).stop 2024-09-19T06:18:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd4536af640 1 -- 172.21.15.5:0/3900402033 shutdown_connections 2024-09-19T06:18:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd4536af640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd4380a3460 0x7fd4380a5920 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd4536af640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd44c107ad0 0x7fd44c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd4536af640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c102460 0x7fd44c1988d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd4536af640 1 --2- 172.21.15.5:0/3900402033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c101ac0 0x7fd44c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.861+0000 7fd4536af640 1 -- 172.21.15.5:0/3900402033 >> 172.21.15.5:0/3900402033 conn(0x7fd44c0fd7a0 msgr2=0x7fd44c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.862+0000 7fd4536af640 1 -- 172.21.15.5:0/3900402033 shutdown_connections 2024-09-19T06:18:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:00.862+0000 7fd4536af640 1 -- 172.21.15.5:0/3900402033 wait complete. 2024-09-19T06:18:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:00 smithi005 ceph-mon[30955]: pgmap v459: 97 pgs: 97 active+clean; 476 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 3.8 MiB/s wr, 318 op/s 2024-09-19T06:18:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:00 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:18:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:00 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:18:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:00 smithi049 ceph-mon[34640]: pgmap v459: 97 pgs: 97 active+clean; 476 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 3.8 MiB/s wr, 318 op/s 2024-09-19T06:18:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:00 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:18:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:00 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:18:01.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 14243 11.04 MB/sec warmup 85 sec latency 117.483 ms 2024-09-19T06:18:02.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 14395 10.98 MB/sec warmup 86 sec latency 66.244 ms 2024-09-19T06:18:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:02 smithi157 ceph-mon[35346]: pgmap v460: 97 pgs: 97 active+clean; 476 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 3.8 MiB/s wr, 318 op/s 2024-09-19T06:18:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:02 smithi005 ceph-mon[30955]: pgmap v460: 97 pgs: 97 active+clean; 476 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 3.8 MiB/s wr, 318 op/s 2024-09-19T06:18:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:02 smithi049 ceph-mon[34640]: pgmap v460: 97 pgs: 97 active+clean; 476 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 3.8 MiB/s wr, 318 op/s 2024-09-19T06:18:03.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 14578 10.98 MB/sec warmup 87 sec latency 82.528 ms 2024-09-19T06:18:04.551 INFO:tasks.workunit.client.0.smithi005.stdout: 10 14759 11.00 MB/sec warmup 88 sec latency 73.231 ms 2024-09-19T06:18:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:04 smithi157 ceph-mon[35346]: pgmap v461: 97 pgs: 97 active+clean; 483 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 470 op/s 2024-09-19T06:18:04.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:04 smithi005 ceph-mon[30955]: pgmap v461: 97 pgs: 97 active+clean; 483 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 470 op/s 2024-09-19T06:18:04.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:04 smithi049 ceph-mon[34640]: pgmap v461: 97 pgs: 97 active+clean; 483 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 470 op/s 2024-09-19T06:18:05.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 14930 10.97 MB/sec warmup 89 sec latency 65.920 ms 2024-09-19T06:18:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:05 smithi157 ceph-mon[35346]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T06:18:05.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:05 smithi005 ceph-mon[30955]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T06:18:05.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:05 smithi049 ceph-mon[34640]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T06:18:06.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15104 10.95 MB/sec warmup 90 sec latency 61.112 ms 2024-09-19T06:18:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:06 smithi157 ceph-mon[35346]: pgmap v463: 97 pgs: 97 active+clean; 483 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 331 op/s 2024-09-19T06:18:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:06 smithi005 ceph-mon[30955]: pgmap v463: 97 pgs: 97 active+clean; 483 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 331 op/s 2024-09-19T06:18:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:06 smithi049 ceph-mon[34640]: pgmap v463: 97 pgs: 97 active+clean; 483 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 331 op/s 2024-09-19T06:18:07.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15272 10.92 MB/sec warmup 91 sec latency 57.323 ms 2024-09-19T06:18:08.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15441 10.88 MB/sec warmup 92 sec latency 57.743 ms 2024-09-19T06:18:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:08 smithi157 ceph-mon[35346]: pgmap v464: 97 pgs: 97 active+clean; 493 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 340 op/s 2024-09-19T06:18:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:08 smithi157 ceph-mon[35346]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T06:18:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:08 smithi005 ceph-mon[30955]: pgmap v464: 97 pgs: 97 active+clean; 493 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 340 op/s 2024-09-19T06:18:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:08 smithi005 ceph-mon[30955]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T06:18:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:08 smithi049 ceph-mon[34640]: pgmap v464: 97 pgs: 97 active+clean; 493 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 340 op/s 2024-09-19T06:18:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:08 smithi049 ceph-mon[34640]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T06:18:09.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15627 10.84 MB/sec warmup 93 sec latency 63.976 ms 2024-09-19T06:18:10.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15797 10.82 MB/sec warmup 94 sec latency 54.534 ms 2024-09-19T06:18:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:10 smithi157 ceph-mon[35346]: pgmap v466: 97 pgs: 97 active+clean; 493 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.3 MiB/s wr, 425 op/s 2024-09-19T06:18:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:10 smithi005 ceph-mon[30955]: pgmap v466: 97 pgs: 97 active+clean; 493 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.3 MiB/s wr, 425 op/s 2024-09-19T06:18:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:10 smithi049 ceph-mon[34640]: pgmap v466: 97 pgs: 97 active+clean; 493 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.3 MiB/s wr, 425 op/s 2024-09-19T06:18:11.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15985 10.84 MB/sec warmup 95 sec latency 58.077 ms 2024-09-19T06:18:12.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 16170 10.86 MB/sec warmup 96 sec latency 70.237 ms 2024-09-19T06:18:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:12 smithi157 ceph-mon[35346]: pgmap v467: 97 pgs: 97 active+clean; 493 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-09-19T06:18:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:12 smithi005 ceph-mon[30955]: pgmap v467: 97 pgs: 97 active+clean; 493 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-09-19T06:18:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:12 smithi049 ceph-mon[34640]: pgmap v467: 97 pgs: 97 active+clean; 493 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-09-19T06:18:13.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 16342 10.86 MB/sec warmup 97 sec latency 89.154 ms 2024-09-19T06:18:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.997+0000 7f699c9c4640 1 -- 172.21.15.5:0/2939721367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699810c110 msgr2=0x7f699810e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.997+0000 7f699c9c4640 1 --2- 172.21.15.5:0/2939721367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699810c110 0x7f699810e580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6990009ca0 tx=0x7f699002a940 comp rx=0 tx=0).stop 2024-09-19T06:18:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.997+0000 7f699c9c4640 1 -- 172.21.15.5:0/2939721367 shutdown_connections 2024-09-19T06:18:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.997+0000 7f699c9c4640 1 --2- 172.21.15.5:0/2939721367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699810c110 0x7f699810e580 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.997+0000 7f699c9c4640 1 --2- 172.21.15.5:0/2939721367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6998103660 0x7f699810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.997+0000 7f699c9c4640 1 --2- 172.21.15.5:0/2939721367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6998102d40 0x7f6998103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.997+0000 7f699c9c4640 1 -- 172.21.15.5:0/2939721367 >> 172.21.15.5:0/2939721367 conn(0x7f69980fc6b0 msgr2=0x7f69980fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.997+0000 7f699c9c4640 1 -- 172.21.15.5:0/2939721367 shutdown_connections 2024-09-19T06:18:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.997+0000 7f699c9c4640 1 -- 172.21.15.5:0/2939721367 wait complete. 2024-09-19T06:18:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.998+0000 7f699c9c4640 1 Processor -- start 2024-09-19T06:18:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.998+0000 7f699c9c4640 1 -- start start 2024-09-19T06:18:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699c9c4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6998102d40 0x7f6998198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699c9c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6998103660 0x7f6998198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699c9c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699810c110 0x7f699819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699c9c4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6998111d90 con 0x7f699810c110 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699c9c4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6998111c10 con 0x7f6998102d40 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699c9c4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6998111f10 con 0x7f6998103660 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6998102d40 0x7f6998198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f6995d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6998103660 0x7f6998198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6998102d40 0x7f6998198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57256/0 (socket says 172.21.15.5:57256) 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f6996d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699810c110 0x7f699819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699659c640 1 -- 172.21.15.5:0/2568342658 learned_addr learned my addr 172.21.15.5:0/2568342658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f6995d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6998103660 0x7f6998198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38124/0 (socket says 172.21.15.5:38124) 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699659c640 1 -- 172.21.15.5:0/2568342658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6998103660 msgr2=0x7f6998198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699659c640 1 --2- 172.21.15.5:0/2568342658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6998103660 0x7f6998198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699659c640 1 -- 172.21.15.5:0/2568342658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699810c110 msgr2=0x7f699819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699659c640 1 --2- 172.21.15.5:0/2568342658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699810c110 0x7f699819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699659c640 1 -- 172.21.15.5:0/2568342658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699819d330 con 0x7f6998102d40 2024-09-19T06:18:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:13.999+0000 7f699659c640 1 --2- 172.21.15.5:0/2568342658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6998102d40 0x7f6998198340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f698800ea60 tx=0x7f698800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.000+0000 7f69877fe640 1 -- 172.21.15.5:0/2568342658 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698800cde0 con 0x7f6998102d40 2024-09-19T06:18:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.000+0000 7f69877fe640 1 -- 172.21.15.5:0/2568342658 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69880035f0 con 0x7f6998102d40 2024-09-19T06:18:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.000+0000 7f69877fe640 1 -- 172.21.15.5:0/2568342658 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988010690 con 0x7f6998102d40 2024-09-19T06:18:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.000+0000 7f699c9c4640 1 -- 172.21.15.5:0/2568342658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699819d620 con 0x7f6998102d40 2024-09-19T06:18:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.000+0000 7f699c9c4640 1 -- 172.21.15.5:0/2568342658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69981a4eb0 con 0x7f6998102d40 2024-09-19T06:18:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.003+0000 7f69877fe640 1 -- 172.21.15.5:0/2568342658 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f69880037a0 con 0x7f6998102d40 2024-09-19T06:18:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.003+0000 7f699f83f740 1 Processor -- start 2024-09-19T06:18:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.003+0000 7f699f83f740 1 -- start start 2024-09-19T06:18:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.003+0000 7f699f83f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b14ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.003+0000 7f699f83f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b14f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.003+0000 7f699f83f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.003+0000 7f699f83f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d81b0af2f0 con 0x55d81b150440 2024-09-19T06:18:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.003+0000 7f699f83f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d81b057270 con 0x55d81b15e350 2024-09-19T06:18:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.003+0000 7f699f83f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d81b0ab2b0 con 0x55d81b14f360 2024-09-19T06:18:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.004+0000 7f699659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b14f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.004+0000 7f6996d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b14ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.004+0000 7f6996d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b14ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45450/0 (socket says 172.21.15.5:45450) 2024-09-19T06:18:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.004+0000 7f6996d9d640 1 -- 172.21.15.5:0/986946830 learned_addr learned my addr 172.21.15.5:0/986946830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.004+0000 7f6995d9b640 1 --2- 172.21.15.5:0/986946830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.004+0000 7f6995d9b640 1 -- 172.21.15.5:0/986946830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 msgr2=0x55d81b14f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.004+0000 7f6995d9b640 1 --2- 172.21.15.5:0/986946830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b14f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.004+0000 7f6995d9b640 1 -- 172.21.15.5:0/986946830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 msgr2=0x55d81b14ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.004+0000 7f6995d9b640 1 --2- 172.21.15.5:0/986946830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b14ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.004+0000 7f6995d9b640 1 -- 172.21.15.5:0/986946830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81b056340 con 0x55d81b15e350 2024-09-19T06:18:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.004+0000 7f699659c640 1 --2- 172.21.15.5:0/986946830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b14f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.004+0000 7f69877fe640 1 --2- 172.21.15.5:0/2568342658 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69800a33b0 0x7f69800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.005+0000 7f6996d9d640 1 --2- 172.21.15.5:0/2568342658 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69800a33b0 0x7f69800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.005+0000 7f69877fe640 1 -- 172.21.15.5:0/2568342658 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f69880cffc0 con 0x7f6998102d40 2024-09-19T06:18:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.005+0000 7f6995d9b640 1 --2- 172.21.15.5:0/986946830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b160740 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f698c009bf0 tx=0x7f698c011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7ce931636fe8645 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.005+0000 7f6984ff9640 1 -- 172.21.15.5:0/986946830 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c004310 con 0x55d81b15e350 2024-09-19T06:18:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.005+0000 7f6984ff9640 1 -- 172.21.15.5:0/986946830 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f698c0044b0 con 0x55d81b15e350 2024-09-19T06:18:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.005+0000 7f6984ff9640 1 -- 172.21.15.5:0/986946830 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c004310 con 0x55d81b15e350 2024-09-19T06:18:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.006+0000 7f699f83f740 1 -- 172.21.15.5:0/986946830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 msgr2=0x55d81b160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.006+0000 7f699f83f740 1 --2- 172.21.15.5:0/986946830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b160740 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f698c009bf0 tx=0x7f698c011440 comp rx=0 tx=0).stop 2024-09-19T06:18:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.006+0000 7f6996d9d640 1 --2- 172.21.15.5:0/2568342658 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69800a33b0 0x7f69800a5870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6990009ca0 tx=0x7f6990005c50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.006+0000 7f699f83f740 1 -- 172.21.15.5:0/986946830 shutdown_connections 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.006+0000 7f699f83f740 1 --2- 172.21.15.5:0/986946830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b160740 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.006+0000 7f699f83f740 1 --2- 172.21.15.5:0/986946830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b14f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.006+0000 7f699f83f740 1 --2- 172.21.15.5:0/986946830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b14ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.006+0000 7f699f83f740 1 -- 172.21.15.5:0/986946830 >> 172.21.15.5:0/986946830 conn(0x55d81b14b470 msgr2=0x55d81b14b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.006+0000 7f699f83f740 1 -- 172.21.15.5:0/986946830 shutdown_connections 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.006+0000 7f699f83f740 1 -- 172.21.15.5:0/986946830 wait complete. 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f699f83f740 1 -- start start 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f699f83f740 1 Processor -- start 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f699f83f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b14f360 0x55d81b1479c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f699f83f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b150440 0x55d81b147f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f699f83f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b148440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f699f83f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d81b161110 con 0x55d81b14f360 2024-09-19T06:18:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f699f83f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d81b038290 con 0x55d81b15e350 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f699f83f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d81b161290 con 0x55d81b150440 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f699f83f740 10 client.0 _ll_register_callbacks cb 0x55d81b069140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f699659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b14f360 0x55d81b1479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f699659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b14f360 0x55d81b1479c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45454/0 (socket says 172.21.15.5:45454) 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f699659c640 1 -- 172.21.15.5:0/1965942819 learned_addr learned my addr 172.21.15.5:0/1965942819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.007+0000 7f6996d9d640 1 --2- 172.21.15.5:0/1965942819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.008+0000 7f699659c640 1 -- 172.21.15.5:0/1965942819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b150440 msgr2=0x55d81b147f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.008+0000 7f699659c640 1 --2- 172.21.15.5:0/1965942819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b150440 0x55d81b147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.008+0000 7f699659c640 1 -- 172.21.15.5:0/1965942819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 msgr2=0x55d81b148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.008+0000 7f699659c640 1 --2- 172.21.15.5:0/1965942819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b148440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.008+0000 7f699659c640 1 -- 172.21.15.5:0/1965942819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81adc2210 con 0x55d81b14f360 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.008+0000 7f699659c640 1 --2- 172.21.15.5:0/1965942819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b14f360 0x55d81b1479c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f698800ea30 tx=0x7f6988009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4108a90b3b02c77c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.008+0000 7f697dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:18:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.008+0000 7f697dffb640 1 -- 172.21.15.5:0/1965942819 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698809bdc0 con 0x55d81b14f360 2024-09-19T06:18:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.008+0000 7f699f83f740 1 -- 172.21.15.5:0/1965942819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f698c0044b0 con 0x55d81b14f360 2024-09-19T06:18:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.008+0000 7f697dffb640 1 -- 172.21.15.5:0/1965942819 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f69880988f0 con 0x55d81b14f360 2024-09-19T06:18:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.008+0000 7f697dffb640 1 -- 172.21.15.5:0/1965942819 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f698809bbe0 con 0x55d81b14f360 2024-09-19T06:18:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.009+0000 7f699f83f740 10 client.19293 fetch_fsmap learned FSMap version 46 2024-09-19T06:18:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.009+0000 7f699f83f740 1 -- 172.21.15.5:0/1965942819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d81ae8e620 con 0x55d81b14f360 2024-09-19T06:18:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.009+0000 7f697dffb640 1 -- 172.21.15.5:0/1965942819 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f698c0044b0 con 0x55d81b14f360 2024-09-19T06:18:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.009+0000 7f697dffb640 1 -- 172.21.15.5:0/1965942819 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8410+0+0 (secure 0 0 0) 0x7f69880d3cc0 con 0x55d81b14f360 2024-09-19T06:18:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.009+0000 7f699f83f740 10 client.19293 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T06:18:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.009+0000 7f699f83f740 10 client.19293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.009+0000 7f699f83f740 20 client.19293 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.009+0000 7f699f83f740 1 --2- 172.21.15.5:0/1965942819 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d81b16e460 0x55d81b170900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.009+0000 7f699f83f740 4 client.19293 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.009+0000 7f699f83f740 1 -- 172.21.15.5:0/1965942819 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d81b0bca20 con 0x55d81b16e460 2024-09-19T06:18:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.009+0000 7f6995d9b640 1 --2- 172.21.15.5:0/1965942819 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d81b16e460 0x55d81b170900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.010+0000 7f697dffb640 10 client.19293 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.010+0000 7f6995d9b640 1 --2- 172.21.15.5:0/1965942819 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d81b16e460 0x55d81b170900 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70c13cfefa13da9d server_cookie=1dc91f5f07310eaa in_seq=0 out_seq=0 2024-09-19T06:18:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f697dffb640 1 -- 172.21.15.5:0/1965942819 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d81b0bca20 con 0x55d81b16e460 2024-09-19T06:18:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f697dffb640 10 client.19293 handle_command_reply: tid=0 2024-09-19T06:18:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f699f83f740 1 client.19293 shutdown 2024-09-19T06:18:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f699f83f740 1 -- 172.21.15.5:0/1965942819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b14f360 msgr2=0x55d81b1479c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f699f83f740 1 --2- 172.21.15.5:0/1965942819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b14f360 0x55d81b1479c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f698800ea30 tx=0x7f6988009490 comp rx=0 tx=0).stop 2024-09-19T06:18:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f699f83f740 1 -- 172.21.15.5:0/1965942819 shutdown_connections 2024-09-19T06:18:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f699f83f740 1 --2- 172.21.15.5:0/1965942819 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d81b16e460 0x55d81b170900 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f699f83f740 1 --2- 172.21.15.5:0/1965942819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f699f83f740 1 --2- 172.21.15.5:0/1965942819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b150440 0x55d81b147f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f697dffb640 0 client.19293 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f699f83f740 1 --2- 172.21.15.5:0/1965942819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b14f360 0x55d81b1479c0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f699f83f740 1 -- 172.21.15.5:0/1965942819 >> 172.21.15.5:0/1965942819 conn(0x55d81b14b470 msgr2=0x55d81b14b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f699f83f740 1 -- 172.21.15.5:0/1965942819 shutdown_connections 2024-09-19T06:18:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f699f83f740 1 -- 172.21.15.5:0/1965942819 wait complete. 2024-09-19T06:18:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.011+0000 7f699f83f740 20 client.19293 trim_cache size 0 max 16384 2024-09-19T06:18:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f699f83f740 1 Processor -- start 2024-09-19T06:18:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f699f83f740 1 -- start start 2024-09-19T06:18:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f699f83f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f699f83f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b15db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f699f83f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b176210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f699f83f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69981ab680 con 0x55d81b150440 2024-09-19T06:18:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f699f83f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f69981ad3a0 con 0x55d81b15e350 2024-09-19T06:18:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f699f83f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f69981abe10 con 0x55d81b14f360 2024-09-19T06:18:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f6995d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b15db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f699659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f6995d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b15db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45462/0 (socket says 172.21.15.5:45462) 2024-09-19T06:18:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f699659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b15d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38160/0 (socket says 172.21.15.5:38160) 2024-09-19T06:18:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.033+0000 7f6995d9b640 1 -- 172.21.15.5:0/3556869110 learned_addr learned my addr 172.21.15.5:0/3556869110 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.034+0000 7f6996d9d640 1 --2- 172.21.15.5:0/3556869110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b176210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.034+0000 7f699659c640 1 -- 172.21.15.5:0/3556869110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 msgr2=0x55d81b176210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.034+0000 7f699659c640 1 --2- 172.21.15.5:0/3556869110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b176210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.034+0000 7f699659c640 1 -- 172.21.15.5:0/3556869110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 msgr2=0x55d81b15db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.034+0000 7f699659c640 1 --2- 172.21.15.5:0/3556869110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b15db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.034+0000 7f699659c640 1 -- 172.21.15.5:0/3556869110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81b16a3e0 con 0x55d81b14f360 2024-09-19T06:18:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.034+0000 7f6996d9d640 1 --2- 172.21.15.5:0/3556869110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b176210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:18:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.034+0000 7f6995d9b640 1 --2- 172.21.15.5:0/3556869110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b15db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.034+0000 7f699659c640 1 --2- 172.21.15.5:0/3556869110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b15d5e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f69880986e0 tx=0x7f69880cd7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df4484ac19b614df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.034+0000 7f697ffff640 1 -- 172.21.15.5:0/3556869110 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698809bdc0 con 0x55d81b14f360 2024-09-19T06:18:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.034+0000 7f697ffff640 1 -- 172.21.15.5:0/3556869110 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6988003b30 con 0x55d81b14f360 2024-09-19T06:18:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.034+0000 7f697ffff640 1 -- 172.21.15.5:0/3556869110 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880d3220 con 0x55d81b14f360 2024-09-19T06:18:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.035+0000 7f699f83f740 1 -- 172.21.15.5:0/3556869110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 msgr2=0x55d81b15d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.035+0000 7f699f83f740 1 --2- 172.21.15.5:0/3556869110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b15d5e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f69880986e0 tx=0x7f69880cd7c0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.035+0000 7f699f83f740 1 -- 172.21.15.5:0/3556869110 shutdown_connections 2024-09-19T06:18:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.035+0000 7f699f83f740 1 --2- 172.21.15.5:0/3556869110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b176210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.035+0000 7f699f83f740 1 --2- 172.21.15.5:0/3556869110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b15db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.035+0000 7f699f83f740 1 --2- 172.21.15.5:0/3556869110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b15d5e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.035+0000 7f699f83f740 1 -- 172.21.15.5:0/3556869110 >> 172.21.15.5:0/3556869110 conn(0x55d81b14b470 msgr2=0x55d81b15b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.035+0000 7f699f83f740 1 -- 172.21.15.5:0/3556869110 shutdown_connections 2024-09-19T06:18:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.035+0000 7f699f83f740 1 -- 172.21.15.5:0/3556869110 wait complete. 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f699f83f740 1 -- start start 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f699f83f740 1 Processor -- start 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f699f83f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b166550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f699f83f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b166a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f699f83f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b163520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f699f83f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69981accd0 con 0x55d81b150440 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f699f83f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f69981ac540 con 0x55d81b15e350 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f699f83f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f69981a9680 con 0x55d81b14f360 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f699f83f740 10 client.0 _ll_register_callbacks cb 0x7f69880d3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f6995d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b166a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f6996d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b163520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f6995d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b166a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45476/0 (socket says 172.21.15.5:45476) 2024-09-19T06:18:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.036+0000 7f6995d9b640 1 -- 172.21.15.5:0/3388458254 learned_addr learned my addr 172.21.15.5:0/3388458254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.037+0000 7f699659c640 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b166550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.037+0000 7f6996d9d640 1 -- 172.21.15.5:0/3388458254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 msgr2=0x55d81b166550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.037+0000 7f6996d9d640 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b166550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.037+0000 7f6996d9d640 1 -- 172.21.15.5:0/3388458254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 msgr2=0x55d81b166a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.037+0000 7f6996d9d640 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b166a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.037+0000 7f6996d9d640 1 -- 172.21.15.5:0/3388458254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81b1671d0 con 0x55d81b15e350 2024-09-19T06:18:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.037+0000 7f6995d9b640 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b166a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.037+0000 7f6996d9d640 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b163520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f699002ae60 tx=0x7f69900095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79ef4a64b2769b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.037+0000 7f697dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:18:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.037+0000 7f697dffb640 1 -- 172.21.15.5:0/3388458254 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990057060 con 0x55d81b15e350 2024-09-19T06:18:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.037+0000 7f699f83f740 1 -- 172.21.15.5:0/3388458254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d81b0b6d70 con 0x55d81b15e350 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.037+0000 7f697dffb640 1 -- 172.21.15.5:0/3388458254 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f6990004710 con 0x55d81b15e350 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.038+0000 7f697dffb640 1 -- 172.21.15.5:0/3388458254 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6990034040 con 0x55d81b15e350 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.038+0000 7f699f83f740 10 client.28697 fetch_fsmap learned FSMap version 46 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.038+0000 7f699f83f740 1 -- 172.21.15.5:0/3388458254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d81b1d9890 con 0x55d81b15e350 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.038+0000 7f697dffb640 1 -- 172.21.15.5:0/3388458254 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55d81b0b6d70 con 0x55d81b15e350 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.038+0000 7f697dffb640 1 -- 172.21.15.5:0/3388458254 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 46) ==== 8410+0+0 (secure 0 0 0) 0x7f6990051070 con 0x55d81b15e350 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.038+0000 7f699f83f740 10 client.28697 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.038+0000 7f699f83f740 10 client.28697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.038+0000 7f699f83f740 20 client.28697 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.038+0000 7f699f83f740 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d81b1da080 0x55d81b1dc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.038+0000 7f699f83f740 4 client.28697 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.038+0000 7f699f83f740 1 -- 172.21.15.5:0/3388458254 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d81b033900 con 0x55d81b1da080 2024-09-19T06:18:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.039+0000 7f699659c640 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d81b1da080 0x55d81b1dc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.039+0000 7f699659c640 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d81b1da080 0x55d81b1dc4a0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd2016d1570d8fd server_cookie=7455d73b8df56946 in_seq=0 out_seq=0 2024-09-19T06:18:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.039+0000 7f697dffb640 10 client.28697 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f697dffb640 1 -- 172.21.15.5:0/3388458254 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d81b033900 con 0x55d81b1da080 2024-09-19T06:18:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f697dffb640 10 client.28697 handle_command_reply: tid=0 2024-09-19T06:18:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f699f83f740 1 client.28697 shutdown 2024-09-19T06:18:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f699f83f740 1 -- 172.21.15.5:0/3388458254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 msgr2=0x55d81b163520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f699f83f740 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b163520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f699002ae60 tx=0x7f69900095c0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f699f83f740 1 -- 172.21.15.5:0/3388458254 shutdown_connections 2024-09-19T06:18:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f699f83f740 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d81b1da080 0x55d81b1dc4a0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f697dffb640 0 client.28697 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f699f83f740 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d81b15e350 0x55d81b163520 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f699f83f740 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d81b150440 0x55d81b166a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f699f83f740 1 --2- 172.21.15.5:0/3388458254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d81b14f360 0x55d81b166550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f699f83f740 1 -- 172.21.15.5:0/3388458254 >> 172.21.15.5:0/3388458254 conn(0x55d81b14b470 msgr2=0x55d81b146290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f699f83f740 1 -- 172.21.15.5:0/3388458254 shutdown_connections 2024-09-19T06:18:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.129+0000 7f699f83f740 1 -- 172.21.15.5:0/3388458254 wait complete. 2024-09-19T06:18:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.130+0000 7f699f83f740 20 client.28697 trim_cache size 0 max 16384 2024-09-19T06:18:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:18:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:18:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:18:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:18:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.130+0000 7f699c9c4640 1 -- 172.21.15.5:0/2568342658 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69800a33b0 msgr2=0x7f69800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.130+0000 7f699c9c4640 1 --2- 172.21.15.5:0/2568342658 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69800a33b0 0x7f69800a5870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6990009ca0 tx=0x7f6990005c50 comp rx=0 tx=0).stop 2024-09-19T06:18:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.130+0000 7f699c9c4640 1 -- 172.21.15.5:0/2568342658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6998102d40 msgr2=0x7f6998198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.130+0000 7f699c9c4640 1 --2- 172.21.15.5:0/2568342658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6998102d40 0x7f6998198340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f698800ea60 tx=0x7f698800ef20 comp rx=0 tx=0).stop 2024-09-19T06:18:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.131+0000 7f699c9c4640 1 -- 172.21.15.5:0/2568342658 shutdown_connections 2024-09-19T06:18:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.131+0000 7f699c9c4640 1 --2- 172.21.15.5:0/2568342658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699810c110 0x7f699819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.131+0000 7f699c9c4640 1 --2- 172.21.15.5:0/2568342658 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69800a33b0 0x7f69800a5870 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.131+0000 7f699c9c4640 1 --2- 172.21.15.5:0/2568342658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6998103660 0x7f6998198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.131+0000 7f699c9c4640 1 --2- 172.21.15.5:0/2568342658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6998102d40 0x7f6998198340 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.131+0000 7f699c9c4640 1 -- 172.21.15.5:0/2568342658 >> 172.21.15.5:0/2568342658 conn(0x7f69980fc6b0 msgr2=0x7f69980fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.131+0000 7f699c9c4640 1 -- 172.21.15.5:0/2568342658 shutdown_connections 2024-09-19T06:18:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:14.131+0000 7f699c9c4640 1 -- 172.21.15.5:0/2568342658 wait complete. 2024-09-19T06:18:14.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 16501 10.88 MB/sec warmup 98 sec latency 67.219 ms 2024-09-19T06:18:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:14 smithi157 ceph-mon[35346]: pgmap v468: 97 pgs: 97 active+clean; 509 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 361 op/s 2024-09-19T06:18:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:14 smithi005 ceph-mon[30955]: pgmap v468: 97 pgs: 97 active+clean; 509 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 361 op/s 2024-09-19T06:18:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:14 smithi049 ceph-mon[34640]: pgmap v468: 97 pgs: 97 active+clean; 509 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 361 op/s 2024-09-19T06:18:15.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 16694 10.89 MB/sec warmup 99 sec latency 61.825 ms 2024-09-19T06:18:16.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 16851 10.90 MB/sec warmup 100 sec latency 59.132 ms 2024-09-19T06:18:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:16 smithi157 ceph-mon[35346]: pgmap v469: 97 pgs: 97 active+clean; 509 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 304 op/s 2024-09-19T06:18:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:16 smithi005 ceph-mon[30955]: pgmap v469: 97 pgs: 97 active+clean; 509 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 304 op/s 2024-09-19T06:18:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:16 smithi049 ceph-mon[34640]: pgmap v469: 97 pgs: 97 active+clean; 509 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 304 op/s 2024-09-19T06:18:17.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 16972 10.82 MB/sec warmup 101 sec latency 64.632 ms 2024-09-19T06:18:18.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17145 10.82 MB/sec warmup 102 sec latency 54.110 ms 2024-09-19T06:18:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:18 smithi157 ceph-mon[35346]: pgmap v470: 97 pgs: 97 active+clean; 516 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.4 MiB/s wr, 329 op/s 2024-09-19T06:18:18.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:18 smithi005 ceph-mon[30955]: pgmap v470: 97 pgs: 97 active+clean; 516 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.4 MiB/s wr, 329 op/s 2024-09-19T06:18:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:18 smithi049 ceph-mon[34640]: pgmap v470: 97 pgs: 97 active+clean; 516 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.4 MiB/s wr, 329 op/s 2024-09-19T06:18:19.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17296 10.75 MB/sec warmup 103 sec latency 52.386 ms 2024-09-19T06:18:20.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17456 10.70 MB/sec warmup 104 sec latency 62.492 ms 2024-09-19T06:18:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:20 smithi157 ceph-mon[35346]: pgmap v471: 97 pgs: 97 active+clean; 516 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 315 op/s 2024-09-19T06:18:20.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:20 smithi005 ceph-mon[30955]: pgmap v471: 97 pgs: 97 active+clean; 516 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 315 op/s 2024-09-19T06:18:20.895 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:20 smithi049 ceph-mon[34640]: pgmap v471: 97 pgs: 97 active+clean; 516 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 315 op/s 2024-09-19T06:18:21.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17638 10.66 MB/sec warmup 105 sec latency 70.409 ms 2024-09-19T06:18:22.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17830 10.67 MB/sec warmup 106 sec latency 71.888 ms 2024-09-19T06:18:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:22 smithi157 ceph-mon[35346]: pgmap v472: 97 pgs: 97 active+clean; 516 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 274 op/s 2024-09-19T06:18:22.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:22 smithi005 ceph-mon[30955]: pgmap v472: 97 pgs: 97 active+clean; 516 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 274 op/s 2024-09-19T06:18:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:22 smithi049 ceph-mon[34640]: pgmap v472: 97 pgs: 97 active+clean; 516 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 274 op/s 2024-09-19T06:18:23.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:18:23.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17992 10.63 MB/sec warmup 107 sec latency 64.297 ms 2024-09-19T06:18:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:18:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.141+0000 7f81757a0640 1 -- 172.21.15.5:0/2332176543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170103e10 msgr2=0x7f81701041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.141+0000 7f81757a0640 1 --2- 172.21.15.5:0/2332176543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170103e10 0x7f81701041f0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8160009900 tx=0x7f816002a410 comp rx=0 tx=0).stop 2024-09-19T06:18:24.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.141+0000 7f81757a0640 1 -- 172.21.15.5:0/2332176543 shutdown_connections 2024-09-19T06:18:24.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.141+0000 7f81757a0640 1 --2- 172.21.15.5:0/2332176543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8170108f40 0x7f8170110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.141+0000 7f81757a0640 1 --2- 172.21.15.5:0/2332176543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81701047c0 0x7f8170108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.141+0000 7f81757a0640 1 --2- 172.21.15.5:0/2332176543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170103e10 0x7f81701041f0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.141+0000 7f81757a0640 1 -- 172.21.15.5:0/2332176543 >> 172.21.15.5:0/2332176543 conn(0x7f81700ff870 msgr2=0x7f8170101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:24.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.141+0000 7f81757a0640 1 -- 172.21.15.5:0/2332176543 shutdown_connections 2024-09-19T06:18:24.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.141+0000 7f81757a0640 1 -- 172.21.15.5:0/2332176543 wait complete. 2024-09-19T06:18:24.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.142+0000 7f81757a0640 1 Processor -- start 2024-09-19T06:18:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.142+0000 7f81757a0640 1 -- start start 2024-09-19T06:18:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.142+0000 7f81757a0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8170103e10 0x7f817019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.142+0000 7f81757a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81701047c0 0x7f817019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f81757a0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8170108f40 0x7f817019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f81757a0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8170113ea0 con 0x7f81701047c0 2024-09-19T06:18:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f81757a0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8170113d20 con 0x7f8170103e10 2024-09-19T06:18:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f81757a0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8170114020 con 0x7f8170108f40 2024-09-19T06:18:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f8167fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81701047c0 0x7f817019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f8167fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81701047c0 0x7f817019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60734/0 (socket says 172.21.15.5:60734) 2024-09-19T06:18:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f8167fff640 1 -- 172.21.15.5:0/2667609392 learned_addr learned my addr 172.21.15.5:0/2667609392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f816ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8170103e10 0x7f817019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f8174f9f640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8170108f40 0x7f817019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f816ffff640 1 -- 172.21.15.5:0/2667609392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8170108f40 msgr2=0x7f817019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f816ffff640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8170108f40 0x7f817019ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f816ffff640 1 -- 172.21.15.5:0/2667609392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81701047c0 msgr2=0x7f817019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f816ffff640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81701047c0 0x7f817019a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.143+0000 7f816ffff640 1 -- 172.21.15.5:0/2667609392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817019f370 con 0x7f8170103e10 2024-09-19T06:18:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.144+0000 7f816ffff640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8170103e10 0x7f817019a3c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f816002aa00 tx=0x7f8160003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.144+0000 7f816dffb640 1 -- 172.21.15.5:0/2667609392 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816003f070 con 0x7f8170103e10 2024-09-19T06:18:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.144+0000 7f816dffb640 1 -- 172.21.15.5:0/2667609392 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8160031b50 con 0x7f8170103e10 2024-09-19T06:18:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.144+0000 7f816dffb640 1 -- 172.21.15.5:0/2667609392 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160035030 con 0x7f8170103e10 2024-09-19T06:18:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.144+0000 7f81757a0640 1 -- 172.21.15.5:0/2667609392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817019f600 con 0x7f8170103e10 2024-09-19T06:18:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.144+0000 7f81757a0640 1 -- 172.21.15.5:0/2667609392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817019fa60 con 0x7f8170103e10 2024-09-19T06:18:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.147+0000 7f816dffb640 1 -- 172.21.15.5:0/2667609392 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8160042070 con 0x7f8170103e10 2024-09-19T06:18:24.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.148+0000 7f8178ddb740 1 Processor -- start 2024-09-19T06:18:24.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.148+0000 7f816dffb640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f81500a33b0 0x7f81500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.148+0000 7f8178ddb740 1 -- start start 2024-09-19T06:18:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.148+0000 7f8174f9f640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f81500a33b0 0x7f81500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.148+0000 7f8178ddb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bb1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.148+0000 7f8178ddb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1f90 0x560b45bbc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.148+0000 7f8178ddb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bbec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.148+0000 7f8178ddb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560b45b172e0 con 0x560b45bbc850 2024-09-19T06:18:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.148+0000 7f8178ddb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560b45abece0 con 0x560b45bb1f90 2024-09-19T06:18:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.148+0000 7f8178ddb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560b45b132a0 con 0x560b45bb1670 2024-09-19T06:18:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.149+0000 7f816dffb640 1 -- 172.21.15.5:0/2667609392 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f81600efec0 con 0x7f8170103e10 2024-09-19T06:18:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.149+0000 7f816ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bb1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.149+0000 7f816ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bb1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36228/0 (socket says 172.21.15.5:36228) 2024-09-19T06:18:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.149+0000 7f816ffff640 1 -- 172.21.15.5:0/1141702604 learned_addr learned my addr 172.21.15.5:0/1141702604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.149+0000 7f8167fff640 1 --2- 172.21.15.5:0/1141702604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1f90 0x560b45bbc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.149+0000 7f8174f9f640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f81500a33b0 0x7f81500a5870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8168005fd0 tx=0x7f8168009aa0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.149+0000 7f8174f9f640 1 --2- 172.21.15.5:0/1141702604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bbec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.149+0000 7f816ffff640 1 -- 172.21.15.5:0/1141702604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1f90 msgr2=0x560b45bbc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.149+0000 7f816ffff640 1 --2- 172.21.15.5:0/1141702604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1f90 0x560b45bbc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.149+0000 7f816ffff640 1 -- 172.21.15.5:0/1141702604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 msgr2=0x560b45bbec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.149+0000 7f816ffff640 1 --2- 172.21.15.5:0/1141702604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bbec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.149+0000 7f816ffff640 1 -- 172.21.15.5:0/1141702604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b45aa3440 con 0x560b45bb1670 2024-09-19T06:18:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.150+0000 7f816ffff640 1 --2- 172.21.15.5:0/1141702604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bb1a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f81600c48c0 tx=0x7f8160005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b794ff924fdca159 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.150+0000 7f81667fc640 1 -- 172.21.15.5:0/1141702604 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160049070 con 0x560b45bb1670 2024-09-19T06:18:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.150+0000 7f81667fc640 1 -- 172.21.15.5:0/1141702604 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f816003d550 con 0x560b45bb1670 2024-09-19T06:18:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.150+0000 7f81667fc640 1 -- 172.21.15.5:0/1141702604 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81600c0530 con 0x560b45bb1670 2024-09-19T06:18:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.151+0000 7f8178ddb740 1 -- 172.21.15.5:0/1141702604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 msgr2=0x560b45bb1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.151+0000 7f8178ddb740 1 --2- 172.21.15.5:0/1141702604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bb1a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f81600c48c0 tx=0x7f8160005c00 comp rx=0 tx=0).stop 2024-09-19T06:18:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.151+0000 7f8178ddb740 1 -- 172.21.15.5:0/1141702604 shutdown_connections 2024-09-19T06:18:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.151+0000 7f8178ddb740 1 --2- 172.21.15.5:0/1141702604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bbec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.151+0000 7f8178ddb740 1 --2- 172.21.15.5:0/1141702604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1f90 0x560b45bbc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.151+0000 7f8178ddb740 1 --2- 172.21.15.5:0/1141702604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bb1a50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.151+0000 7f8178ddb740 1 -- 172.21.15.5:0/1141702604 >> 172.21.15.5:0/1141702604 conn(0x560b45badc00 msgr2=0x560b45badfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.151+0000 7f8178ddb740 1 -- 172.21.15.5:0/1141702604 shutdown_connections 2024-09-19T06:18:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.151+0000 7f8178ddb740 1 -- 172.21.15.5:0/1141702604 wait complete. 2024-09-19T06:18:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.151+0000 7f8178ddb740 1 -- start start 2024-09-19T06:18:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8178ddb740 1 Processor -- start 2024-09-19T06:18:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8178ddb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bbc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8178ddb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1f90 0x560b45bb9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8178ddb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bb9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8178ddb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560b45bbf1e0 con 0x560b45bbc850 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8178ddb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560b45ad2050 con 0x560b45bb1f90 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8178ddb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560b45bbf360 con 0x560b45bb1670 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8178ddb740 10 client.0 _ll_register_callbacks cb 0x560b45b12160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8174f9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bb9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8174f9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bb9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60752/0 (socket says 172.21.15.5:60752) 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8174f9f640 1 -- 172.21.15.5:0/2985593292 learned_addr learned my addr 172.21.15.5:0/2985593292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f816ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bbc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8167fff640 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1f90 0x560b45bb9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8174f9f640 1 -- 172.21.15.5:0/2985593292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 msgr2=0x560b45bbc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8174f9f640 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bbc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.152+0000 7f8174f9f640 1 -- 172.21.15.5:0/2985593292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1f90 msgr2=0x560b45bb9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.153+0000 7f8174f9f640 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1f90 0x560b45bb9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.153+0000 7f8174f9f640 1 -- 172.21.15.5:0/2985593292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b458883c0 con 0x560b45bbc850 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.153+0000 7f8167fff640 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1f90 0x560b45bb9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.153+0000 7f8174f9f640 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bb9940 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f8168004380 tx=0x7f8168004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd292f86018200 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.153+0000 7f815f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.153+0000 7f815f7fe640 1 -- 172.21.15.5:0/2985593292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816800fa00 con 0x560b45bbc850 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.153+0000 7f815f7fe640 1 -- 172.21.15.5:0/2985593292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f816800fba0 con 0x560b45bbc850 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.153+0000 7f8178ddb740 1 -- 172.21.15.5:0/2985593292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f816003d550 con 0x560b45bbc850 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.153+0000 7f815f7fe640 1 -- 172.21.15.5:0/2985593292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f816800fda0 con 0x560b45bbc850 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.153+0000 7f8178ddb740 10 client.19311 fetch_fsmap learned FSMap version 46 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.153+0000 7f8178ddb740 1 -- 172.21.15.5:0/2985593292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560b45bd3330 con 0x560b45bbc850 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.154+0000 7f815f7fe640 1 -- 172.21.15.5:0/2985593292 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f816003d550 con 0x560b45bbc850 2024-09-19T06:18:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.154+0000 7f815f7fe640 1 -- 172.21.15.5:0/2985593292 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8410+0+0 (secure 0 0 0) 0x7f816801e520 con 0x560b45bbc850 2024-09-19T06:18:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.154+0000 7f8178ddb740 10 client.19311 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T06:18:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.154+0000 7f8178ddb740 10 client.19311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.154+0000 7f8178ddb740 20 client.19311 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.154+0000 7f8178ddb740 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b45bd4800 0x560b45bd6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.154+0000 7f8178ddb740 4 client.19311 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.154+0000 7f8178ddb740 1 -- 172.21.15.5:0/2985593292 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b45b24850 con 0x560b45bd4800 2024-09-19T06:18:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.154+0000 7f816ffff640 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b45bd4800 0x560b45bd6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.155+0000 7f816ffff640 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b45bd4800 0x560b45bd6c80 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6950fcf2e2db2b84 server_cookie=7e9a6ccb9e51a114 in_seq=0 out_seq=0 2024-09-19T06:18:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.155+0000 7f815f7fe640 10 client.19311 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.155+0000 7f815f7fe640 1 -- 172.21.15.5:0/2985593292 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b45b24850 con 0x560b45bd4800 2024-09-19T06:18:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.155+0000 7f815f7fe640 10 client.19311 handle_command_reply: tid=0 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.155+0000 7f8178ddb740 1 client.19311 shutdown 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.156+0000 7f8178ddb740 1 -- 172.21.15.5:0/2985593292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 msgr2=0x560b45bb9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.156+0000 7f8178ddb740 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bb9940 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f8168004380 tx=0x7f8168004ab0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.156+0000 7f8178ddb740 1 -- 172.21.15.5:0/2985593292 shutdown_connections 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.156+0000 7f8178ddb740 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b45bd4800 0x560b45bd6c80 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.156+0000 7f8178ddb740 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bb9940 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.156+0000 7f8178ddb740 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1f90 0x560b45bb9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.156+0000 7f815f7fe640 0 client.19311 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.156+0000 7f8178ddb740 1 --2- 172.21.15.5:0/2985593292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bbc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.156+0000 7f8178ddb740 1 -- 172.21.15.5:0/2985593292 >> 172.21.15.5:0/2985593292 conn(0x560b45badc00 msgr2=0x560b45bb3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.156+0000 7f8178ddb740 1 -- 172.21.15.5:0/2985593292 shutdown_connections 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.156+0000 7f8178ddb740 1 -- 172.21.15.5:0/2985593292 wait complete. 2024-09-19T06:18:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.156+0000 7f8178ddb740 20 client.19311 trim_cache size 0 max 16384 2024-09-19T06:18:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f8178ddb740 1 Processor -- start 2024-09-19T06:18:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f8178ddb740 1 -- start start 2024-09-19T06:18:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f8178ddb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bdc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f8178ddb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bb1f90 0x560b45bdcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f8178ddb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bbc850 0x560b45bccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f8178ddb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81701ab510 con 0x560b45bb1f90 2024-09-19T06:18:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f8178ddb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f81701ad230 con 0x560b45bbc850 2024-09-19T06:18:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f8178ddb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81701abca0 con 0x560b45bb1670 2024-09-19T06:18:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f816ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bdc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f8167fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bb1f90 0x560b45bdcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f816ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bdc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36244/0 (socket says 172.21.15.5:36244) 2024-09-19T06:18:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f8167fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bb1f90 0x560b45bdcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60762/0 (socket says 172.21.15.5:60762) 2024-09-19T06:18:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f816ffff640 1 -- 172.21.15.5:0/23429232 learned_addr learned my addr 172.21.15.5:0/23429232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.186+0000 7f8174f9f640 1 --2- 172.21.15.5:0/23429232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bbc850 0x560b45bccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.187+0000 7f816ffff640 1 -- 172.21.15.5:0/23429232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bbc850 msgr2=0x560b45bccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.187+0000 7f816ffff640 1 --2- 172.21.15.5:0/23429232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bbc850 0x560b45bccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.187+0000 7f816ffff640 1 -- 172.21.15.5:0/23429232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bb1f90 msgr2=0x560b45bdcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.187+0000 7f816ffff640 1 --2- 172.21.15.5:0/23429232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bb1f90 0x560b45bdcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.187+0000 7f816ffff640 1 -- 172.21.15.5:0/23429232 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b45bb9f40 con 0x560b45bb1670 2024-09-19T06:18:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.187+0000 7f816ffff640 1 --2- 172.21.15.5:0/23429232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bdc590 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f81600ed6a0 tx=0x7f81600bbc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f862a50847072e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.187+0000 7f8165ffb640 1 -- 172.21.15.5:0/23429232 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160049040 con 0x560b45bb1670 2024-09-19T06:18:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.187+0000 7f8165ffb640 1 -- 172.21.15.5:0/23429232 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81600c0530 con 0x560b45bb1670 2024-09-19T06:18:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.187+0000 7f8165ffb640 1 -- 172.21.15.5:0/23429232 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81600dc0f0 con 0x560b45bb1670 2024-09-19T06:18:24.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.188+0000 7f8178ddb740 1 -- 172.21.15.5:0/23429232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 msgr2=0x560b45bdc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.188+0000 7f8178ddb740 1 --2- 172.21.15.5:0/23429232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bdc590 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f81600ed6a0 tx=0x7f81600bbc90 comp rx=0 tx=0).stop 2024-09-19T06:18:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.188+0000 7f8178ddb740 1 -- 172.21.15.5:0/23429232 shutdown_connections 2024-09-19T06:18:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.188+0000 7f8178ddb740 1 --2- 172.21.15.5:0/23429232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bbc850 0x560b45bccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.188+0000 7f8178ddb740 1 --2- 172.21.15.5:0/23429232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bb1f90 0x560b45bdcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.188+0000 7f8178ddb740 1 --2- 172.21.15.5:0/23429232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1670 0x560b45bdc590 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.188+0000 7f8178ddb740 1 -- 172.21.15.5:0/23429232 >> 172.21.15.5:0/23429232 conn(0x560b45badc00 msgr2=0x560b45bb84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.188+0000 7f8178ddb740 1 -- 172.21.15.5:0/23429232 shutdown_connections 2024-09-19T06:18:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.188+0000 7f8178ddb740 1 -- 172.21.15.5:0/23429232 wait complete. 2024-09-19T06:18:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.189+0000 7f8178ddb740 1 -- start start 2024-09-19T06:18:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.189+0000 7f8178ddb740 1 Processor -- start 2024-09-19T06:18:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.189+0000 7f8178ddb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1670 0x560b45bcd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f8178ddb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1f90 0x560b45bca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f816ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1670 0x560b45bcd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f8178ddb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bcab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f816ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1670 0x560b45bcd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43290/0 (socket says 172.21.15.5:43290) 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f8167fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1f90 0x560b45bca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f8167fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1f90 0x560b45bca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36248/0 (socket says 172.21.15.5:36248) 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f8174f9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bcab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f8174f9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bcab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60770/0 (socket says 172.21.15.5:60770) 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f816ffff640 1 -- 172.21.15.5:0/3596093567 learned_addr learned my addr 172.21.15.5:0/3596093567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f8178ddb740 1 -- 172.21.15.5:0/3596093567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81701acb60 con 0x560b45bbc850 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f8178ddb740 1 -- 172.21.15.5:0/3596093567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f81701ac3d0 con 0x560b45bb1670 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f8178ddb740 1 -- 172.21.15.5:0/3596093567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81701a9510 con 0x560b45bb1f90 2024-09-19T06:18:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.190+0000 7f8178ddb740 10 client.0 _ll_register_callbacks cb 0x7f8168003a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.191+0000 7f816ffff640 1 -- 172.21.15.5:0/3596093567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1f90 msgr2=0x560b45bca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.191+0000 7f816ffff640 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1f90 0x560b45bca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.191+0000 7f816ffff640 1 -- 172.21.15.5:0/3596093567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 msgr2=0x560b45bcab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.191+0000 7f816ffff640 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bcab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.191+0000 7f816ffff640 1 -- 172.21.15.5:0/3596093567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b45baecb0 con 0x560b45bb1670 2024-09-19T06:18:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.191+0000 7f8167fff640 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1f90 0x560b45bca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.191+0000 7f8174f9f640 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bcab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.191+0000 7f816ffff640 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1670 0x560b45bcd230 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f81600b8690 tx=0x7f81600ba530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1114aec8056b30fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.191+0000 7f815f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:18:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.191+0000 7f815f7fe640 1 -- 172.21.15.5:0/3596093567 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160049070 con 0x560b45bb1670 2024-09-19T06:18:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.191+0000 7f815f7fe640 1 -- 172.21.15.5:0/3596093567 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f81600ed870 con 0x560b45bb1670 2024-09-19T06:18:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.191+0000 7f8178ddb740 1 -- 172.21.15.5:0/3596093567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81600c0530 con 0x560b45bb1670 2024-09-19T06:18:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.192+0000 7f815f7fe640 1 -- 172.21.15.5:0/3596093567 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f816003d110 con 0x560b45bb1670 2024-09-19T06:18:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.192+0000 7f8178ddb740 10 client.28712 fetch_fsmap learned FSMap version 46 2024-09-19T06:18:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.192+0000 7f8178ddb740 1 -- 172.21.15.5:0/3596093567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560b45c3f0e0 con 0x560b45bb1670 2024-09-19T06:18:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.192+0000 7f815f7fe640 1 -- 172.21.15.5:0/3596093567 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81600c0530 con 0x560b45bb1670 2024-09-19T06:18:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.192+0000 7f815f7fe640 1 -- 172.21.15.5:0/3596093567 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 46) ==== 8410+0+0 (secure 0 0 0) 0x7f816004c060 con 0x560b45bb1670 2024-09-19T06:18:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.192+0000 7f8178ddb740 10 client.28712 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T06:18:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.192+0000 7f8178ddb740 10 client.28712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.192+0000 7f8178ddb740 20 client.28712 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.192+0000 7f8178ddb740 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b45bd4800 0x560b45c41480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.192+0000 7f8178ddb740 4 client.28712 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:18:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.192+0000 7f8178ddb740 1 -- 172.21.15.5:0/3596093567 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b45b1ed60 con 0x560b45bd4800 2024-09-19T06:18:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.192+0000 7f8167fff640 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b45bd4800 0x560b45c41480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.193+0000 7f8167fff640 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b45bd4800 0x560b45c41480 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd4cebdc54fbfa5 server_cookie=6947f660c1e24da1 in_seq=0 out_seq=0 2024-09-19T06:18:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.193+0000 7f815f7fe640 10 client.28712 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.193+0000 7f815f7fe640 1 -- 172.21.15.5:0/3596093567 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560b45b1ed60 con 0x560b45bd4800 2024-09-19T06:18:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.193+0000 7f815f7fe640 10 client.28712 handle_command_reply: tid=0 2024-09-19T06:18:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.193+0000 7f8178ddb740 1 client.28712 shutdown 2024-09-19T06:18:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.194+0000 7f8178ddb740 1 -- 172.21.15.5:0/3596093567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1670 msgr2=0x560b45bcd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.194+0000 7f8178ddb740 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1670 0x560b45bcd230 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f81600b8690 tx=0x7f81600ba530 comp rx=0 tx=0).stop 2024-09-19T06:18:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.194+0000 7f8178ddb740 1 -- 172.21.15.5:0/3596093567 shutdown_connections 2024-09-19T06:18:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.194+0000 7f8178ddb740 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b45bd4800 0x560b45c41480 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.194+0000 7f815f7fe640 0 client.28712 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.194+0000 7f8178ddb740 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b45bbc850 0x560b45bcab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.194+0000 7f8178ddb740 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b45bb1f90 0x560b45bca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.194+0000 7f8178ddb740 1 --2- 172.21.15.5:0/3596093567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b45bb1670 0x560b45bcd230 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.194+0000 7f8178ddb740 1 -- 172.21.15.5:0/3596093567 >> 172.21.15.5:0/3596093567 conn(0x560b45badc00 msgr2=0x560b45bbac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.195+0000 7f8178ddb740 1 -- 172.21.15.5:0/3596093567 shutdown_connections 2024-09-19T06:18:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.195+0000 7f8178ddb740 1 -- 172.21.15.5:0/3596093567 wait complete. 2024-09-19T06:18:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.195+0000 7f8178ddb740 20 client.28712 trim_cache size 0 max 16384 2024-09-19T06:18:24.200 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:18:24.200 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:18:24.200 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:18:24.200 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:18:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.196+0000 7f81757a0640 1 -- 172.21.15.5:0/2667609392 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f81500a33b0 msgr2=0x7f81500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.196+0000 7f81757a0640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f81500a33b0 0x7f81500a5870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8168005fd0 tx=0x7f8168009aa0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.196+0000 7f81757a0640 1 -- 172.21.15.5:0/2667609392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8170103e10 msgr2=0x7f817019a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.196+0000 7f81757a0640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8170103e10 0x7f817019a3c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f816002aa00 tx=0x7f8160003b90 comp rx=0 tx=0).stop 2024-09-19T06:18:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.196+0000 7f81757a0640 1 -- 172.21.15.5:0/2667609392 shutdown_connections 2024-09-19T06:18:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.196+0000 7f81757a0640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f81500a33b0 0x7f81500a5870 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.196+0000 7f81757a0640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8170108f40 0x7f817019ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.196+0000 7f81757a0640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81701047c0 0x7f817019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.196+0000 7f81757a0640 1 --2- 172.21.15.5:0/2667609392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8170103e10 0x7f817019a3c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.196+0000 7f81757a0640 1 -- 172.21.15.5:0/2667609392 >> 172.21.15.5:0/2667609392 conn(0x7f81700ff870 msgr2=0x7f8170100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.197+0000 7f81757a0640 1 -- 172.21.15.5:0/2667609392 shutdown_connections 2024-09-19T06:18:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:24.197+0000 7f81757a0640 1 -- 172.21.15.5:0/2667609392 wait complete. 2024-09-19T06:18:24.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 18147 10.62 MB/sec warmup 108 sec latency 95.242 ms 2024-09-19T06:18:24.765 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:24 smithi005 ceph-mon[30955]: pgmap v473: 97 pgs: 97 active+clean; 519 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 403 op/s 2024-09-19T06:18:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:24 smithi157 ceph-mon[35346]: pgmap v473: 97 pgs: 97 active+clean; 519 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 403 op/s 2024-09-19T06:18:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:24 smithi049 ceph-mon[34640]: pgmap v473: 97 pgs: 97 active+clean; 519 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 403 op/s 2024-09-19T06:18:25.040 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:18:25.041 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:18:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:18:25.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 18310 10.62 MB/sec warmup 109 sec latency 87.295 ms 2024-09-19T06:18:26.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.044+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3143660758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa308112ba0 msgr2=0x7fa308114f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:26.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.044+0000 7fa30e3f6640 1 --2- 172.21.15.5:0/3143660758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa308112ba0 0x7fa308114f90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa300009ca0 tx=0x7fa30002a950 comp rx=0 tx=0).stop 2024-09-19T06:18:26.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.044+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3143660758 shutdown_connections 2024-09-19T06:18:26.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.044+0000 7fa30e3f6640 1 --2- 172.21.15.5:0/3143660758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa308112ba0 0x7fa308114f90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:26.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.044+0000 7fa30e3f6640 1 --2- 172.21.15.5:0/3143660758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa308074eb0 0x7fa308112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:26.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.044+0000 7fa30e3f6640 1 --2- 172.21.15.5:0/3143660758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa308074590 0x7fa308074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:26.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.044+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3143660758 >> 172.21.15.5:0/3143660758 conn(0x7fa3080fd8f0 msgr2=0x7fa3080ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:26.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.044+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3143660758 shutdown_connections 2024-09-19T06:18:26.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.044+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3143660758 wait complete. 2024-09-19T06:18:26.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.045+0000 7fa30e3f6640 1 Processor -- start 2024-09-19T06:18:26.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa30e3f6640 1 -- start start 2024-09-19T06:18:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa30e3f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa308074590 0x7fa3081a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa30e3f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa308074eb0 0x7fa3081a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa30e3f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa308112ba0 0x7fa30819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa30e3f6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa308118410 con 0x7fa308074590 2024-09-19T06:18:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa30e3f6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa308118290 con 0x7fa308074eb0 2024-09-19T06:18:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa30e3f6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa308118590 con 0x7fa308112ba0 2024-09-19T06:18:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa2fffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa308074eb0 0x7fa3081a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa2fffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa308074eb0 0x7fa3081a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50222/0 (socket says 172.21.15.5:50222) 2024-09-19T06:18:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa307fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa308074590 0x7fa3081a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa2fffff640 1 -- 172.21.15.5:0/3454901412 learned_addr learned my addr 172.21.15.5:0/3454901412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa30c993640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa308112ba0 0x7fa30819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa2fffff640 1 -- 172.21.15.5:0/3454901412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa308112ba0 msgr2=0x7fa30819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.046+0000 7fa2fffff640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa308112ba0 0x7fa30819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.047+0000 7fa2fffff640 1 -- 172.21.15.5:0/3454901412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa308074590 msgr2=0x7fa3081a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.047+0000 7fa2fffff640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa308074590 0x7fa3081a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.047+0000 7fa2fffff640 1 -- 172.21.15.5:0/3454901412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30819ef40 con 0x7fa308074eb0 2024-09-19T06:18:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.047+0000 7fa2fffff640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa308074eb0 0x7fa3081a4b60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000d950 tx=0x7fa2f000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.047+0000 7fa305ffb640 1 -- 172.21.15.5:0/3454901412 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0017030 con 0x7fa308074eb0 2024-09-19T06:18:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.047+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3454901412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30819f230 con 0x7fa308074eb0 2024-09-19T06:18:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.047+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3454901412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3081ab480 con 0x7fa308074eb0 2024-09-19T06:18:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.047+0000 7fa305ffb640 1 -- 172.21.15.5:0/3454901412 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2f0004020 con 0x7fa308074eb0 2024-09-19T06:18:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.047+0000 7fa305ffb640 1 -- 172.21.15.5:0/3454901412 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0005440 con 0x7fa308074eb0 2024-09-19T06:18:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.049+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3454901412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e8005290 con 0x7fa308074eb0 2024-09-19T06:18:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.051+0000 7fa305ffb640 1 -- 172.21.15.5:0/3454901412 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa2f0009a30 con 0x7fa308074eb0 2024-09-19T06:18:26.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.052+0000 7fa305ffb640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2f40a3430 0x7fa2f40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.052+0000 7fa305ffb640 1 -- 172.21.15.5:0/3454901412 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fa2f00cf640 con 0x7fa308074eb0 2024-09-19T06:18:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.052+0000 7fa307fff640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2f40a3430 0x7fa2f40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.053+0000 7fa307fff640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2f40a3430 0x7fa2f40a58f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009bd0 tx=0x7fa2f8005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.056+0000 7fa305ffb640 1 -- 172.21.15.5:0/3454901412 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa2f009bfa0 con 0x7fa308074eb0 2024-09-19T06:18:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.408+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3454901412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2e8005850 con 0x7fa308074eb0 2024-09-19T06:18:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.409+0000 7fa305ffb640 1 -- 172.21.15.5:0/3454901412 <== mon.2 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11979 (secure 0 0 0) 0x7fa2f009b8a0 con 0x7fa308074eb0 2024-09-19T06:18:26.412 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:18:26.412 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":46,"btime":"2024-09-19T06:17:53:883643+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:17:52.960770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:18:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 46 2024-09-19T06:18:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.412+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3454901412 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2f40a3430 msgr2=0x7fa2f40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.412+0000 7fa30e3f6640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2f40a3430 0x7fa2f40a58f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009bd0 tx=0x7fa2f8005f90 comp rx=0 tx=0).stop 2024-09-19T06:18:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.412+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3454901412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa308074eb0 msgr2=0x7fa3081a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.412+0000 7fa30e3f6640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa308074eb0 0x7fa3081a4b60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000d950 tx=0x7fa2f000de10 comp rx=0 tx=0).stop 2024-09-19T06:18:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.413+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3454901412 shutdown_connections 2024-09-19T06:18:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.413+0000 7fa30e3f6640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa2f40a3430 0x7fa2f40a58f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.413+0000 7fa30e3f6640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa308112ba0 0x7fa30819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.413+0000 7fa30e3f6640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa308074eb0 0x7fa3081a4b60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.413+0000 7fa30e3f6640 1 --2- 172.21.15.5:0/3454901412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa308074590 0x7fa3081a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.413+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3454901412 >> 172.21.15.5:0/3454901412 conn(0x7fa3080fd8f0 msgr2=0x7fa3080ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.413+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3454901412 shutdown_connections 2024-09-19T06:18:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:26.413+0000 7fa30e3f6640 1 -- 172.21.15.5:0/3454901412 wait complete. 2024-09-19T06:18:26.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 18486 10.57 MB/sec warmup 110 sec latency 135.783 ms 2024-09-19T06:18:26.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:26 smithi005 ceph-mon[30955]: pgmap v474: 97 pgs: 97 active+clean; 519 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 281 op/s 2024-09-19T06:18:26.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:26 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3454901412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:18:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:26 smithi157 ceph-mon[35346]: pgmap v474: 97 pgs: 97 active+clean; 519 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 281 op/s 2024-09-19T06:18:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:26 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3454901412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:18:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:26 smithi049 ceph-mon[34640]: pgmap v474: 97 pgs: 97 active+clean; 519 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 281 op/s 2024-09-19T06:18:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:26 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3454901412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:18:27.080 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:18:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.265+0000 7f50b1342640 1 -- 172.21.15.5:0/1544009897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1069b0 msgr2=0x7f50ac106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.265+0000 7f50b1342640 1 --2- 172.21.15.5:0/1544009897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1069b0 0x7f50ac106d90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f509c009930 tx=0x7f509c02a420 comp rx=0 tx=0).stop 2024-09-19T06:18:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.265+0000 7f50b1342640 1 -- 172.21.15.5:0/1544009897 shutdown_connections 2024-09-19T06:18:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.265+0000 7f50b1342640 1 --2- 172.21.15.5:0/1544009897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50ac101340 0x7f50ac10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.265+0000 7f50b1342640 1 --2- 172.21.15.5:0/1544009897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50ac1009a0 0x7f50ac100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.265+0000 7f50b1342640 1 --2- 172.21.15.5:0/1544009897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1069b0 0x7f50ac106d90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.265+0000 7f50b1342640 1 -- 172.21.15.5:0/1544009897 >> 172.21.15.5:0/1544009897 conn(0x7f50ac0779a0 msgr2=0x7f50ac0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.265+0000 7f50b1342640 1 -- 172.21.15.5:0/1544009897 shutdown_connections 2024-09-19T06:18:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.265+0000 7f50b1342640 1 -- 172.21.15.5:0/1544009897 wait complete. 2024-09-19T06:18:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.266+0000 7f50b1342640 1 Processor -- start 2024-09-19T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.266+0000 7f50b1342640 1 -- start start 2024-09-19T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.266+0000 7f50b1342640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1009a0 0x7f50ac19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.266+0000 7f50b1342640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50ac101340 0x7f50ac19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50b1342640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50ac1069b0 0x7f50ac19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50b1342640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac111d00 con 0x7f50ac101340 2024-09-19T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50b1342640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac111b80 con 0x7f50ac1069b0 2024-09-19T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50b1342640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac111e80 con 0x7f50ac1009a0 2024-09-19T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50ab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50ac1069b0 0x7f50ac19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50aaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1009a0 0x7f50ac19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50aa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50ac101340 0x7f50ac19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50ab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50ac1069b0 0x7f50ac19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50226/0 (socket says 172.21.15.5:50226) 2024-09-19T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50ab7fe640 1 -- 172.21.15.5:0/1076529979 learned_addr learned my addr 172.21.15.5:0/1076529979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50ab7fe640 1 -- 172.21.15.5:0/1076529979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1009a0 msgr2=0x7f50ac19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50ab7fe640 1 --2- 172.21.15.5:0/1076529979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1009a0 0x7f50ac19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50ab7fe640 1 -- 172.21.15.5:0/1076529979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50ac101340 msgr2=0x7f50ac19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50ab7fe640 1 --2- 172.21.15.5:0/1076529979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50ac101340 0x7f50ac19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50ab7fe640 1 -- 172.21.15.5:0/1076529979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ac19f320 con 0x7f50ac1069b0 2024-09-19T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.267+0000 7f50ab7fe640 1 --2- 172.21.15.5:0/1076529979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50ac1069b0 0x7f50ac19ebf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f50a400efd0 tx=0x7f50a400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.268+0000 7f5093fff640 1 -- 172.21.15.5:0/1076529979 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4003b70 con 0x7f50ac1069b0 2024-09-19T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.268+0000 7f5093fff640 1 -- 172.21.15.5:0/1076529979 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50a4003d10 con 0x7f50ac1069b0 2024-09-19T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.268+0000 7f5093fff640 1 -- 172.21.15.5:0/1076529979 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4010980 con 0x7f50ac1069b0 2024-09-19T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.268+0000 7f50b1342640 1 -- 172.21.15.5:0/1076529979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ac19f610 con 0x7f50ac1069b0 2024-09-19T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.268+0000 7f50b1342640 1 -- 172.21.15.5:0/1076529979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ac1a6ea0 con 0x7f50ac1069b0 2024-09-19T06:18:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.271+0000 7f50b40d9740 1 Processor -- start 2024-09-19T06:18:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.271+0000 7f5093fff640 1 -- 172.21.15.5:0/1076529979 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f50a4004020 con 0x7f50ac1069b0 2024-09-19T06:18:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.271+0000 7f50b40d9740 1 -- start start 2024-09-19T06:18:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.271+0000 7f50b40d9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9f2af0 0x561e8a9f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.271+0000 7f50b40d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f3410 0x561e8a9fd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.271+0000 7f50b40d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9fdcb0 0x561e8aa000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50b40d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e8a9572f0 con 0x561e8a9f3410 2024-09-19T06:18:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50b40d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561e8a8ff270 con 0x561e8a9fdcb0 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50b40d9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561e8a9532b0 con 0x561e8a9f2af0 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f3410 0x561e8a9fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aa7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9fdcb0 0x561e8aa000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f3410 0x561e8a9fd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60472/0 (socket says 172.21.15.5:60472) 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aaffd640 1 -- 172.21.15.5:0/1339962720 learned_addr learned my addr 172.21.15.5:0/1339962720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aa7fc640 1 --2- 172.21.15.5:0/1339962720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9f2af0 0x561e8a9f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aa7fc640 1 -- 172.21.15.5:0/1339962720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9f2af0 msgr2=0x561e8a9f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aa7fc640 1 --2- 172.21.15.5:0/1339962720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9f2af0 0x561e8a9f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aa7fc640 1 -- 172.21.15.5:0/1339962720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f3410 msgr2=0x561e8a9fd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aa7fc640 1 --2- 172.21.15.5:0/1339962720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f3410 0x561e8a9fd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aa7fc640 1 -- 172.21.15.5:0/1339962720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e8a8fe340 con 0x561e8a9fdcb0 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aaffd640 1 --2- 172.21.15.5:0/1339962720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f3410 0x561e8a9fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f5093fff640 1 --2- 172.21.15.5:0/1076529979 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f50940a33b0 0x7f50940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aaffd640 1 --2- 172.21.15.5:0/1076529979 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f50940a33b0 0x7f50940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f50aa7fc640 1 --2- 172.21.15.5:0/1339962720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9fdcb0 0x561e8aa000a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009730 tx=0x7f50a00096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e94bf2cf4d4b19f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.272+0000 7f5093fff640 1 -- 172.21.15.5:0/1076529979 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f50a40d00e0 con 0x7f50ac1069b0 2024-09-19T06:18:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.273+0000 7f50917fa640 1 -- 172.21.15.5:0/1339962720 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0004050 con 0x561e8a9fdcb0 2024-09-19T06:18:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.273+0000 7f50917fa640 1 -- 172.21.15.5:0/1339962720 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50a00054d0 con 0x561e8a9fdcb0 2024-09-19T06:18:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.273+0000 7f50917fa640 1 -- 172.21.15.5:0/1339962720 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0004d40 con 0x561e8a9fdcb0 2024-09-19T06:18:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.274+0000 7f50aaffd640 1 --2- 172.21.15.5:0/1076529979 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f50940a33b0 0x7f50940a5870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x561e8a9efec0 tx=0x7f509c00b040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.274+0000 7f50b40d9740 1 -- 172.21.15.5:0/1339962720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9fdcb0 msgr2=0x561e8aa000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.274+0000 7f50b40d9740 1 --2- 172.21.15.5:0/1339962720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9fdcb0 0x561e8aa000a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009730 tx=0x7f50a00096c0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.274+0000 7f50b40d9740 1 -- 172.21.15.5:0/1339962720 shutdown_connections 2024-09-19T06:18:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.274+0000 7f50b40d9740 1 --2- 172.21.15.5:0/1339962720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9fdcb0 0x561e8aa000a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.274+0000 7f50b40d9740 1 --2- 172.21.15.5:0/1339962720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f3410 0x561e8a9fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.274+0000 7f50b40d9740 1 --2- 172.21.15.5:0/1339962720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9f2af0 0x561e8a9f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.274+0000 7f50b40d9740 1 -- 172.21.15.5:0/1339962720 >> 172.21.15.5:0/1339962720 conn(0x561e8a9ef060 msgr2=0x561e8a9ef440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.274+0000 7f50b40d9740 1 -- 172.21.15.5:0/1339962720 shutdown_connections 2024-09-19T06:18:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.274+0000 7f50b40d9740 1 -- 172.21.15.5:0/1339962720 wait complete. 2024-09-19T06:18:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.275+0000 7f50b40d9740 1 -- start start 2024-09-19T06:18:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.275+0000 7f50b40d9740 1 Processor -- start 2024-09-19T06:18:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.276+0000 7f50b40d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 0x561e8a9faaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.276+0000 7f50b40d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9f3410 0x561e8a9fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.276+0000 7f50aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 0x561e8a9faaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.276+0000 7f50aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 0x561e8a9faaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60486/0 (socket says 172.21.15.5:60486) 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.276+0000 7f50b40d9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9fdcb0 0x561e8a9fb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.276+0000 7f50aaffd640 1 -- 172.21.15.5:0/3190507525 learned_addr learned my addr 172.21.15.5:0/3190507525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.276+0000 7f50b40d9740 1 -- 172.21.15.5:0/3190507525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e8aa00920 con 0x561e8a9f2af0 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.276+0000 7f50aa7fc640 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9f3410 0x561e8a9fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.276+0000 7f50b40d9740 1 -- 172.21.15.5:0/3190507525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561e8a8e0290 con 0x561e8a9f3410 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.276+0000 7f50b40d9740 1 -- 172.21.15.5:0/3190507525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561e8aa00aa0 con 0x561e8a9fdcb0 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.276+0000 7f50ab7fe640 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9fdcb0 0x561e8a9fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.276+0000 7f50b40d9740 10 client.0 _ll_register_callbacks cb 0x561e8a911140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.277+0000 7f50aaffd640 1 -- 172.21.15.5:0/3190507525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9fdcb0 msgr2=0x561e8a9fb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.277+0000 7f50aaffd640 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9fdcb0 0x561e8a9fb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.277+0000 7f50aaffd640 1 -- 172.21.15.5:0/3190507525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9f3410 msgr2=0x561e8a9fafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.277+0000 7f50aaffd640 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9f3410 0x561e8a9fafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.277+0000 7f50aaffd640 1 -- 172.21.15.5:0/3190507525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e8a66a210 con 0x561e8a9f2af0 2024-09-19T06:18:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.277+0000 7f50aa7fc640 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9f3410 0x561e8a9fafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.277+0000 7f50aaffd640 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 0x561e8a9faaa0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f509c031a30 tx=0x7f509c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4dd05a9e20a352 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.277+0000 7f508eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:18:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.277+0000 7f508eff5640 1 -- 172.21.15.5:0/3190507525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c035040 con 0x561e8a9f2af0 2024-09-19T06:18:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.277+0000 7f508eff5640 1 -- 172.21.15.5:0/3190507525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f509c051470 con 0x561e8a9f2af0 2024-09-19T06:18:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.277+0000 7f50b40d9740 1 -- 172.21.15.5:0/3190507525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50a00054d0 con 0x561e8a9f2af0 2024-09-19T06:18:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.278+0000 7f508eff5640 1 -- 172.21.15.5:0/3190507525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f509c03de20 con 0x561e8a9f2af0 2024-09-19T06:18:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.278+0000 7f50b40d9740 10 client.19335 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.278+0000 7f50b40d9740 1 -- 172.21.15.5:0/3190507525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561e8a736620 con 0x561e8a9f2af0 2024-09-19T06:18:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.278+0000 7f508eff5640 1 -- 172.21.15.5:0/3190507525 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50a00054d0 con 0x561e8a9f2af0 2024-09-19T06:18:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.278+0000 7f508eff5640 1 -- 172.21.15.5:0/3190507525 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7f509c044030 con 0x561e8a9f2af0 2024-09-19T06:18:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.278+0000 7f50b40d9740 10 client.19335 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.278+0000 7f50b40d9740 10 client.19335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.278+0000 7f50b40d9740 20 client.19335 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.278+0000 7f50b40d9740 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e8aa167b0 0x561e8aa18ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.278+0000 7f50b40d9740 4 client.19335 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.278+0000 7f50b40d9740 1 -- 172.21.15.5:0/3190507525 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e8a964a20 con 0x561e8aa167b0 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.278+0000 7f50aa7fc640 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e8aa167b0 0x561e8aa18ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.279+0000 7f50aa7fc640 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e8aa167b0 0x561e8aa18ba0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82d2fd28488af74c server_cookie=a7dc69be4fd5844b in_seq=0 out_seq=0 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.279+0000 7f508eff5640 10 client.19335 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f508eff5640 1 -- 172.21.15.5:0/3190507525 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e8a964a20 con 0x561e8aa167b0 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f508eff5640 10 client.19335 handle_command_reply: tid=0 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f50b40d9740 1 client.19335 shutdown 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f50b40d9740 1 -- 172.21.15.5:0/3190507525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 msgr2=0x561e8a9faaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f50b40d9740 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 0x561e8a9faaa0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f509c031a30 tx=0x7f509c031400 comp rx=0 tx=0).stop 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f50b40d9740 1 -- 172.21.15.5:0/3190507525 shutdown_connections 2024-09-19T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f50b40d9740 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e8aa167b0 0x561e8aa18ba0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f50b40d9740 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9fdcb0 0x561e8a9fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f50b40d9740 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9f3410 0x561e8a9fafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f50b40d9740 1 --2- 172.21.15.5:0/3190507525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 0x561e8a9faaa0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f50b40d9740 1 -- 172.21.15.5:0/3190507525 >> 172.21.15.5:0/3190507525 conn(0x561e8a9ef060 msgr2=0x561e8a9f4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f508eff5640 0 client.19335 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.280+0000 7f50b40d9740 1 -- 172.21.15.5:0/3190507525 shutdown_connections 2024-09-19T06:18:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.281+0000 7f50b40d9740 1 -- 172.21.15.5:0/3190507525 wait complete. 2024-09-19T06:18:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.281+0000 7f50b40d9740 20 client.19335 trim_cache size 0 max 16384 2024-09-19T06:18:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.304+0000 7f50b40d9740 1 Processor -- start 2024-09-19T06:18:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.304+0000 7f50b40d9740 1 -- start start 2024-09-19T06:18:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50b40d9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9f2af0 0x561e8a9fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50b40d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f3410 0x561e8aa1e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50b40d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9fdcb0 0x561e8aa1e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50b40d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1ab610 con 0x561e8a9f3410 2024-09-19T06:18:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50b40d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1ad300 con 0x561e8a9fdcb0 2024-09-19T06:18:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50b40d9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1abda0 con 0x561e8a9f2af0 2024-09-19T06:18:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50aaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9f2af0 0x561e8a9fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50aa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f3410 0x561e8aa1e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50aaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9f2af0 0x561e8a9fca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42262/0 (socket says 172.21.15.5:42262) 2024-09-19T06:18:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50aaffd640 1 -- 172.21.15.5:0/3811460347 learned_addr learned my addr 172.21.15.5:0/3811460347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50ab7fe640 1 --2- 172.21.15.5:0/3811460347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9fdcb0 0x561e8aa1e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50aaffd640 1 -- 172.21.15.5:0/3811460347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9fdcb0 msgr2=0x561e8aa1e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50aaffd640 1 --2- 172.21.15.5:0/3811460347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9fdcb0 0x561e8aa1e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50aaffd640 1 -- 172.21.15.5:0/3811460347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f3410 msgr2=0x561e8aa1e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50aaffd640 1 --2- 172.21.15.5:0/3811460347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f3410 0x561e8aa1e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.305+0000 7f50aaffd640 1 -- 172.21.15.5:0/3811460347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e8a9eee90 con 0x561e8a9f2af0 2024-09-19T06:18:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.306+0000 7f50aaffd640 1 --2- 172.21.15.5:0/3811460347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9f2af0 0x561e8a9fca10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f509c042040 tx=0x7f509c031730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6066a9fbe6aa8053 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.306+0000 7f5090ff9640 1 -- 172.21.15.5:0/3811460347 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c005020 con 0x561e8a9f2af0 2024-09-19T06:18:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.306+0000 7f5090ff9640 1 -- 172.21.15.5:0/3811460347 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f509c04c070 con 0x561e8a9f2af0 2024-09-19T06:18:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.306+0000 7f5090ff9640 1 -- 172.21.15.5:0/3811460347 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c004ca0 con 0x561e8a9f2af0 2024-09-19T06:18:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.307+0000 7f50b40d9740 1 -- 172.21.15.5:0/3811460347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9f2af0 msgr2=0x561e8a9fca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.307+0000 7f50b40d9740 1 --2- 172.21.15.5:0/3811460347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9f2af0 0x561e8a9fca10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f509c042040 tx=0x7f509c031730 comp rx=0 tx=0).stop 2024-09-19T06:18:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.308+0000 7f50b40d9740 1 -- 172.21.15.5:0/3811460347 shutdown_connections 2024-09-19T06:18:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.308+0000 7f50b40d9740 1 --2- 172.21.15.5:0/3811460347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9fdcb0 0x561e8aa1e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.308+0000 7f50b40d9740 1 --2- 172.21.15.5:0/3811460347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f3410 0x561e8aa1e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.308+0000 7f50b40d9740 1 --2- 172.21.15.5:0/3811460347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9f2af0 0x561e8a9fca10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.308+0000 7f50b40d9740 1 -- 172.21.15.5:0/3811460347 >> 172.21.15.5:0/3811460347 conn(0x561e8a9ef060 msgr2=0x561e8a9fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.308+0000 7f50b40d9740 1 -- 172.21.15.5:0/3811460347 shutdown_connections 2024-09-19T06:18:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.308+0000 7f50b40d9740 1 -- 172.21.15.5:0/3811460347 wait complete. 2024-09-19T06:18:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.308+0000 7f50b40d9740 1 -- start start 2024-09-19T06:18:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50b40d9740 1 Processor -- start 2024-09-19T06:18:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50b40d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 0x561e8aa0eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50b40d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9f3410 0x561e8a9f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50b40d9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9fdcb0 0x561e8a9f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50b40d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1acc30 con 0x561e8a9f2af0 2024-09-19T06:18:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50b40d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1ac4a0 con 0x561e8a9f3410 2024-09-19T06:18:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50b40d9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1a9630 con 0x561e8a9fdcb0 2024-09-19T06:18:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50b40d9740 10 client.0 _ll_register_callbacks cb 0x7f509c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 0x561e8aa0eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50aa7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9f3410 0x561e8a9f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 0x561e8aa0eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60504/0 (socket says 172.21.15.5:60504) 2024-09-19T06:18:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50aaffd640 1 -- 172.21.15.5:0/2565058222 learned_addr learned my addr 172.21.15.5:0/2565058222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.309+0000 7f50ab7fe640 1 --2- 172.21.15.5:0/2565058222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9fdcb0 0x561e8a9f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.310+0000 7f50aaffd640 1 -- 172.21.15.5:0/2565058222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9fdcb0 msgr2=0x561e8a9f7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.310+0000 7f50aaffd640 1 --2- 172.21.15.5:0/2565058222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9fdcb0 0x561e8a9f7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.310+0000 7f50aaffd640 1 -- 172.21.15.5:0/2565058222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9f3410 msgr2=0x561e8a9f74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.310+0000 7f50aaffd640 1 --2- 172.21.15.5:0/2565058222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9f3410 0x561e8a9f74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.310+0000 7f50aaffd640 1 -- 172.21.15.5:0/2565058222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e8a9f7f40 con 0x561e8a9f2af0 2024-09-19T06:18:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.310+0000 7f50aaffd640 1 --2- 172.21.15.5:0/2565058222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 0x561e8aa0eb00 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f509c048640 tx=0x7f509c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1606b9344eb7beec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.310+0000 7f508eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:18:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.310+0000 7f508eff5640 1 -- 172.21.15.5:0/2565058222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c051470 con 0x561e8a9f2af0 2024-09-19T06:18:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.310+0000 7f508eff5640 1 -- 172.21.15.5:0/2565058222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f509c051610 con 0x561e8a9f2af0 2024-09-19T06:18:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.310+0000 7f50b40d9740 1 -- 172.21.15.5:0/2565058222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f509c04c070 con 0x561e8a9f2af0 2024-09-19T06:18:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.311+0000 7f508eff5640 1 -- 172.21.15.5:0/2565058222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f509c076050 con 0x561e8a9f2af0 2024-09-19T06:18:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.311+0000 7f50b40d9740 10 client.19341 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.311+0000 7f50b40d9740 1 -- 172.21.15.5:0/2565058222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561e8aa80e90 con 0x561e8a9f2af0 2024-09-19T06:18:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.311+0000 7f508eff5640 1 -- 172.21.15.5:0/2565058222 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f509c04c070 con 0x561e8a9f2af0 2024-09-19T06:18:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.312+0000 7f508eff5640 1 -- 172.21.15.5:0/2565058222 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7f509c044040 con 0x561e8a9f2af0 2024-09-19T06:18:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.312+0000 7f50b40d9740 10 client.19341 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.312+0000 7f50b40d9740 10 client.19341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.312+0000 7f50b40d9740 20 client.19341 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.312+0000 7f50b40d9740 1 --2- 172.21.15.5:0/2565058222 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e8aa81600 0x561e8aa83a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.312+0000 7f50b40d9740 4 client.19341 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:18:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.312+0000 7f50b40d9740 1 -- 172.21.15.5:0/2565058222 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e8a95ed70 con 0x561e8aa81600 2024-09-19T06:18:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.312+0000 7f50aa7fc640 1 --2- 172.21.15.5:0/2565058222 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e8aa81600 0x561e8aa83a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.312+0000 7f508eff5640 10 client.19341 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.313+0000 7f50aa7fc640 1 --2- 172.21.15.5:0/2565058222 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e8aa81600 0x561e8aa83a20 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfdc0dd110b1050d server_cookie=79682c8ce6eacc89 in_seq=0 out_seq=0 2024-09-19T06:18:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.361+0000 7f508eff5640 1 -- 172.21.15.5:0/2565058222 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e8a95ed70 con 0x561e8aa81600 2024-09-19T06:18:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.361+0000 7f508eff5640 10 client.19341 handle_command_reply: tid=0 2024-09-19T06:18:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.361+0000 7f50b40d9740 1 client.19341 shutdown 2024-09-19T06:18:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.362+0000 7f50b40d9740 1 -- 172.21.15.5:0/2565058222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 msgr2=0x561e8aa0eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.362+0000 7f50b40d9740 1 --2- 172.21.15.5:0/2565058222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 0x561e8aa0eb00 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f509c048640 tx=0x7f509c031400 comp rx=0 tx=0).stop 2024-09-19T06:18:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.362+0000 7f50b40d9740 1 -- 172.21.15.5:0/2565058222 shutdown_connections 2024-09-19T06:18:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.362+0000 7f50b40d9740 1 --2- 172.21.15.5:0/2565058222 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e8aa81600 0x561e8aa83a20 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.362+0000 7f50b40d9740 1 --2- 172.21.15.5:0/2565058222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e8a9fdcb0 0x561e8a9f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.362+0000 7f50b40d9740 1 --2- 172.21.15.5:0/2565058222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e8a9f3410 0x561e8a9f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.362+0000 7f508eff5640 0 client.19341 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.362+0000 7f50b40d9740 1 --2- 172.21.15.5:0/2565058222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8a9f2af0 0x561e8aa0eb00 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.362+0000 7f50b40d9740 1 -- 172.21.15.5:0/2565058222 >> 172.21.15.5:0/2565058222 conn(0x561e8a9ef060 msgr2=0x561e8a9ee1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.362+0000 7f50b40d9740 1 -- 172.21.15.5:0/2565058222 shutdown_connections 2024-09-19T06:18:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.362+0000 7f50b40d9740 1 -- 172.21.15.5:0/2565058222 wait complete. 2024-09-19T06:18:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.362+0000 7f50b40d9740 20 client.19341 trim_cache size 0 max 16384 2024-09-19T06:18:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:18:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:18:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:18:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:18:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.363+0000 7f50b1342640 1 -- 172.21.15.5:0/1076529979 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f50940a33b0 msgr2=0x7f50940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.363+0000 7f50b1342640 1 --2- 172.21.15.5:0/1076529979 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f50940a33b0 0x7f50940a5870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x561e8a9efec0 tx=0x7f509c00b040 comp rx=0 tx=0).stop 2024-09-19T06:18:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.363+0000 7f50b1342640 1 -- 172.21.15.5:0/1076529979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50ac1069b0 msgr2=0x7f50ac19ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.363+0000 7f50b1342640 1 --2- 172.21.15.5:0/1076529979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50ac1069b0 0x7f50ac19ebf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f50a400efd0 tx=0x7f50a400c6a0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.363+0000 7f50b1342640 1 -- 172.21.15.5:0/1076529979 shutdown_connections 2024-09-19T06:18:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.363+0000 7f50b1342640 1 --2- 172.21.15.5:0/1076529979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50ac1069b0 0x7f50ac19ebf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.363+0000 7f50b1342640 1 --2- 172.21.15.5:0/1076529979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50ac101340 0x7f50ac19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.363+0000 7f50b1342640 1 --2- 172.21.15.5:0/1076529979 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f50940a33b0 0x7f50940a5870 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.363+0000 7f50b1342640 1 --2- 172.21.15.5:0/1076529979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1009a0 0x7f50ac19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.363+0000 7f50b1342640 1 -- 172.21.15.5:0/1076529979 >> 172.21.15.5:0/1076529979 conn(0x7f50ac0779a0 msgr2=0x7f50ac103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.364+0000 7f50b1342640 1 -- 172.21.15.5:0/1076529979 shutdown_connections 2024-09-19T06:18:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:27.364+0000 7f50b1342640 1 -- 172.21.15.5:0/1076529979 wait complete. 2024-09-19T06:18:27.477 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:18:27.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 18668 10.58 MB/sec warmup 111 sec latency 102.582 ms 2024-09-19T06:18:27.764 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:27 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:18:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:27 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:18:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:27 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:18:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.025+0000 7ff323f20640 1 -- 172.21.15.5:0/94297995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff31c102ef0 msgr2=0x7ff31c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.025+0000 7ff323f20640 1 --2- 172.21.15.5:0/94297995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff31c102ef0 0x7ff31c10da40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff314009cd0 tx=0x7ff31402a980 comp rx=0 tx=0).stop 2024-09-19T06:18:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.025+0000 7ff323f20640 1 -- 172.21.15.5:0/94297995 shutdown_connections 2024-09-19T06:18:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.025+0000 7ff323f20640 1 --2- 172.21.15.5:0/94297995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff31c102ef0 0x7ff31c10da40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.025+0000 7ff323f20640 1 --2- 172.21.15.5:0/94297995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff31c102570 0x7ff31c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.025+0000 7ff323f20640 1 --2- 172.21.15.5:0/94297995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff31c100e20 0x7ff31c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.025+0000 7ff323f20640 1 -- 172.21.15.5:0/94297995 >> 172.21.15.5:0/94297995 conn(0x7ff31c0fc7d0 msgr2=0x7ff31c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.026+0000 7ff323f20640 1 -- 172.21.15.5:0/94297995 shutdown_connections 2024-09-19T06:18:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.026+0000 7ff323f20640 1 -- 172.21.15.5:0/94297995 wait complete. 2024-09-19T06:18:28.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff323f20640 1 Processor -- start 2024-09-19T06:18:28.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff323f20640 1 -- start start 2024-09-19T06:18:28.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff323f20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff31c100e20 0x7ff31c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff323f20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff31c102570 0x7ff31c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff323f20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff31c102ef0 0x7ff31c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff323f20640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c110e30 con 0x7ff31c100e20 2024-09-19T06:18:28.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff323f20640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c110cb0 con 0x7ff31c102570 2024-09-19T06:18:28.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff323f20640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c110fb0 con 0x7ff31c102ef0 2024-09-19T06:18:28.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff3214bb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff31c102570 0x7ff31c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff321cbc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff31c100e20 0x7ff31c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff3214bb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff31c102570 0x7ff31c197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50284/0 (socket says 172.21.15.5:50284) 2024-09-19T06:18:28.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff3214bb640 1 -- 172.21.15.5:0/2873907382 learned_addr learned my addr 172.21.15.5:0/2873907382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:28.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.027+0000 7ff3224bd640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff31c102ef0 0x7ff31c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.028+0000 7ff3214bb640 1 -- 172.21.15.5:0/2873907382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff31c102ef0 msgr2=0x7ff31c19bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:28.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.028+0000 7ff3214bb640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff31c102ef0 0x7ff31c19bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.028+0000 7ff3214bb640 1 -- 172.21.15.5:0/2873907382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff31c100e20 msgr2=0x7ff31c197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:28.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.028+0000 7ff3214bb640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff31c100e20 0x7ff31c197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.028+0000 7ff3214bb640 1 -- 172.21.15.5:0/2873907382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c19c300 con 0x7ff31c102570 2024-09-19T06:18:28.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.028+0000 7ff3214bb640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff31c102570 0x7ff31c197870 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00e9f0 tx=0x7ff30c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.029+0000 7ff312ffd640 1 -- 172.21.15.5:0/2873907382 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c00cd80 con 0x7ff31c102570 2024-09-19T06:18:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.029+0000 7ff323f20640 1 -- 172.21.15.5:0/2873907382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31c19c5f0 con 0x7ff31c102570 2024-09-19T06:18:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.029+0000 7ff323f20640 1 -- 172.21.15.5:0/2873907382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31c069f70 con 0x7ff31c102570 2024-09-19T06:18:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.029+0000 7ff312ffd640 1 -- 172.21.15.5:0/2873907382 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff30c0108e0 con 0x7ff31c102570 2024-09-19T06:18:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.029+0000 7ff312ffd640 1 -- 172.21.15.5:0/2873907382 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c018920 con 0x7ff31c102570 2024-09-19T06:18:28.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.032+0000 7ff312ffd640 1 -- 172.21.15.5:0/2873907382 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff30c010430 con 0x7ff31c102570 2024-09-19T06:18:28.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.032+0000 7ff3262f9740 1 Processor -- start 2024-09-19T06:18:28.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.033+0000 7ff3262f9740 1 -- start start 2024-09-19T06:18:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.033+0000 7ff3262f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bf3b40 0x555f76bf2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.033+0000 7ff3262f9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf25d0 0x555f76bf2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.033+0000 7ff3262f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bfd370 0x555f76bff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.033+0000 7ff3262f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555f76b582f0 con 0x555f76bf3b40 2024-09-19T06:18:28.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.033+0000 7ff3262f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555f76affcf0 con 0x555f76bfd370 2024-09-19T06:18:28.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.033+0000 7ff3262f9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555f76b542b0 con 0x555f76bf25d0 2024-09-19T06:18:28.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.033+0000 7ff3224bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bf3b40 0x555f76bf2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.033+0000 7ff312ffd640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff3080a3430 0x7ff3080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.033+0000 7ff3224bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bf3b40 0x555f76bf2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60538/0 (socket says 172.21.15.5:60538) 2024-09-19T06:18:28.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.033+0000 7ff3224bd640 1 -- 172.21.15.5:0/4154186051 learned_addr learned my addr 172.21.15.5:0/4154186051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:28.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.034+0000 7ff321cbc640 1 --2- 172.21.15.5:0/4154186051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf25d0 0x555f76bf2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.034+0000 7ff3224bd640 1 -- 172.21.15.5:0/4154186051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf25d0 msgr2=0x555f76bf2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:28.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.034+0000 7ff3224bd640 1 --2- 172.21.15.5:0/4154186051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf25d0 0x555f76bf2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.034+0000 7ff3224bd640 1 -- 172.21.15.5:0/4154186051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bfd370 msgr2=0x555f76bff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:28.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.034+0000 7ff312ffd640 1 -- 172.21.15.5:0/2873907382 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7ff30c0cff40 con 0x7ff31c102570 2024-09-19T06:18:28.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.034+0000 7ff3214bb640 1 --2- 172.21.15.5:0/4154186051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bfd370 0x555f76bff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.034+0000 7ff3224bd640 1 --2- 172.21.15.5:0/4154186051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bfd370 0x555f76bff760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.034+0000 7ff3224bd640 1 -- 172.21.15.5:0/4154186051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f76ae43a0 con 0x555f76bf3b40 2024-09-19T06:18:28.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.034+0000 7ff321cbc640 1 --2- 172.21.15.5:0/4154186051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf25d0 0x555f76bf2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:18:28.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.034+0000 7ff3224bd640 1 --2- 172.21.15.5:0/4154186051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bf3b40 0x555f76bf2090 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff314009cd0 tx=0x7ff314005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206edd819b65f080 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:28.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.034+0000 7ff3224bd640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff3080a3430 0x7ff3080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.035+0000 7ff307fff640 1 -- 172.21.15.5:0/4154186051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314034040 con 0x555f76bf3b40 2024-09-19T06:18:28.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.035+0000 7ff307fff640 1 -- 172.21.15.5:0/4154186051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff31404b040 con 0x555f76bf3b40 2024-09-19T06:18:28.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.035+0000 7ff307fff640 1 -- 172.21.15.5:0/4154186051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314034040 con 0x555f76bf3b40 2024-09-19T06:18:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.036+0000 7ff3262f9740 1 -- 172.21.15.5:0/4154186051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bf3b40 msgr2=0x555f76bf2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.036+0000 7ff3224bd640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff3080a3430 0x7ff3080a58f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff314003fd0 tx=0x7ff314030bb0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.036+0000 7ff3262f9740 1 --2- 172.21.15.5:0/4154186051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bf3b40 0x555f76bf2090 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff314009cd0 tx=0x7ff314005f80 comp rx=0 tx=0).stop 2024-09-19T06:18:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.036+0000 7ff3262f9740 1 -- 172.21.15.5:0/4154186051 shutdown_connections 2024-09-19T06:18:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.036+0000 7ff3262f9740 1 --2- 172.21.15.5:0/4154186051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bfd370 0x555f76bff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.036+0000 7ff3262f9740 1 --2- 172.21.15.5:0/4154186051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf25d0 0x555f76bf2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.036+0000 7ff3262f9740 1 --2- 172.21.15.5:0/4154186051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bf3b40 0x555f76bf2090 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.036+0000 7ff3262f9740 1 -- 172.21.15.5:0/4154186051 >> 172.21.15.5:0/4154186051 conn(0x555f76bbb310 msgr2=0x555f76beec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.036+0000 7ff3262f9740 1 -- 172.21.15.5:0/4154186051 shutdown_connections 2024-09-19T06:18:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.036+0000 7ff3262f9740 1 -- 172.21.15.5:0/4154186051 wait complete. 2024-09-19T06:18:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.036+0000 7ff3262f9740 1 -- start start 2024-09-19T06:18:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.037+0000 7ff3262f9740 1 Processor -- start 2024-09-19T06:18:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.037+0000 7ff3262f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76bf61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.037+0000 7ff3262f9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf3b40 0x555f76bf6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.037+0000 7ff3262f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bfd370 0x555f76bf6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.037+0000 7ff3262f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555f76bffe80 con 0x555f76bfd370 2024-09-19T06:18:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.037+0000 7ff3262f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555f76b13060 con 0x555f76bf25d0 2024-09-19T06:18:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.037+0000 7ff3262f9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555f76c00000 con 0x555f76bf3b40 2024-09-19T06:18:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.037+0000 7ff3262f9740 10 client.0 _ll_register_callbacks cb 0x555f768668c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.037+0000 7ff3214bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf3b40 0x555f76bf6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.037+0000 7ff321cbc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76bf61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.037+0000 7ff3214bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf3b40 0x555f76bf6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42318/0 (socket says 172.21.15.5:42318) 2024-09-19T06:18:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.037+0000 7ff3214bb640 1 -- 172.21.15.5:0/552229221 learned_addr learned my addr 172.21.15.5:0/552229221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.038+0000 7ff3224bd640 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bfd370 0x555f76bf6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.038+0000 7ff321cbc640 1 -- 172.21.15.5:0/552229221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf3b40 msgr2=0x555f76bf6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.038+0000 7ff321cbc640 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf3b40 0x555f76bf6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.038+0000 7ff321cbc640 1 -- 172.21.15.5:0/552229221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bfd370 msgr2=0x555f76bf6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.038+0000 7ff321cbc640 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bfd370 0x555f76bf6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.038+0000 7ff321cbc640 1 -- 172.21.15.5:0/552229221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f768c93c0 con 0x555f76bf25d0 2024-09-19T06:18:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.038+0000 7ff3214bb640 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf3b40 0x555f76bf6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.038+0000 7ff321cbc640 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76bf61d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff31800f5e0 tx=0x7ff31800f9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efb35afe116c0f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.038+0000 7ff3057fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:18:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3057fa640 1 -- 172.21.15.5:0/552229221 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3180035f0 con 0x555f76bf25d0 2024-09-19T06:18:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3057fa640 1 -- 172.21.15.5:0/552229221 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7ff318003790 con 0x555f76bf25d0 2024-09-19T06:18:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3262f9740 1 -- 172.21.15.5:0/552229221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff31404b040 con 0x555f76bf25d0 2024-09-19T06:18:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3057fa640 1 -- 172.21.15.5:0/552229221 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff318026050 con 0x555f76bf25d0 2024-09-19T06:18:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3262f9740 10 client.28745 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3262f9740 1 -- 172.21.15.5:0/552229221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555f76c142c0 con 0x555f76bf25d0 2024-09-19T06:18:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3057fa640 1 -- 172.21.15.5:0/552229221 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff31801d070 con 0x555f76bf25d0 2024-09-19T06:18:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3057fa640 1 -- 172.21.15.5:0/552229221 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7ff3180189d0 con 0x555f76bf25d0 2024-09-19T06:18:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3262f9740 10 client.28745 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3262f9740 10 client.28745 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:18:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3262f9740 20 client.28745 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3262f9740 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f76c15890 0x555f76c17d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3262f9740 4 client.28745 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.039+0000 7ff3262f9740 1 -- 172.21.15.5:0/552229221 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff31404b040 con 0x555f76c15890 2024-09-19T06:18:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.040+0000 7ff3214bb640 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f76c15890 0x555f76c17d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.040+0000 7ff3057fa640 10 client.28745 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:18:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.040+0000 7ff3214bb640 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f76c15890 0x555f76c17d10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c323527f7b88cca server_cookie=1648602d7a236e29 in_seq=0 out_seq=0 2024-09-19T06:18:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.042+0000 7ff3057fa640 1 -- 172.21.15.5:0/552229221 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff31404b040 con 0x555f76c15890 2024-09-19T06:18:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.042+0000 7ff3057fa640 10 client.28745 handle_command_reply: tid=0 2024-09-19T06:18:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.042+0000 7ff3262f9740 1 client.28745 shutdown 2024-09-19T06:18:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.042+0000 7ff3262f9740 1 -- 172.21.15.5:0/552229221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 msgr2=0x555f76bf61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.042+0000 7ff3262f9740 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76bf61d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff31800f5e0 tx=0x7ff31800f9e0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.043+0000 7ff3262f9740 1 -- 172.21.15.5:0/552229221 shutdown_connections 2024-09-19T06:18:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.043+0000 7ff3262f9740 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f76c15890 0x555f76c17d10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.043+0000 7ff3262f9740 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bfd370 0x555f76bf6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.043+0000 7ff3262f9740 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf3b40 0x555f76bf6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.043+0000 7ff3262f9740 1 --2- 172.21.15.5:0/552229221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76bf61d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.043+0000 7ff3262f9740 1 -- 172.21.15.5:0/552229221 >> 172.21.15.5:0/552229221 conn(0x555f76bbb310 msgr2=0x555f76bf4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.043+0000 7ff3057fa640 0 client.28745 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:18:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.043+0000 7ff3262f9740 1 -- 172.21.15.5:0/552229221 shutdown_connections 2024-09-19T06:18:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.043+0000 7ff3262f9740 1 -- 172.21.15.5:0/552229221 wait complete. 2024-09-19T06:18:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.044+0000 7ff3262f9740 20 client.28745 trim_cache size 0 max 16384 2024-09-19T06:18:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.063+0000 7ff3262f9740 1 Processor -- start 2024-09-19T06:18:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.063+0000 7ff3262f9740 1 -- start start 2024-09-19T06:18:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff3262f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76beee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff3262f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bf3b40 0x555f76bf6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff3262f9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bfd370 0x555f76c1d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff3262f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff30000e7e0 con 0x555f76bf3b40 2024-09-19T06:18:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff3262f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff3000104d0 con 0x555f76bf25d0 2024-09-19T06:18:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff3262f9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff30000ef70 con 0x555f76bfd370 2024-09-19T06:18:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff321cbc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76beee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff321cbc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76beee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50304/0 (socket says 172.21.15.5:50304) 2024-09-19T06:18:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff321cbc640 1 -- 172.21.15.5:0/4135162898 learned_addr learned my addr 172.21.15.5:0/4135162898 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff3214bb640 1 --2- 172.21.15.5:0/4135162898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bf3b40 0x555f76bf6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff3224bd640 1 --2- 172.21.15.5:0/4135162898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bfd370 0x555f76c1d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff321cbc640 1 -- 172.21.15.5:0/4135162898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bfd370 msgr2=0x555f76c1d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff321cbc640 1 --2- 172.21.15.5:0/4135162898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bfd370 0x555f76c1d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff321cbc640 1 -- 172.21.15.5:0/4135162898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bf3b40 msgr2=0x555f76bf6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff321cbc640 1 --2- 172.21.15.5:0/4135162898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bf3b40 0x555f76bf6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.064+0000 7ff321cbc640 1 -- 172.21.15.5:0/4135162898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f76c03eb0 con 0x555f76bf25d0 2024-09-19T06:18:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.065+0000 7ff321cbc640 1 --2- 172.21.15.5:0/4135162898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76beee10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff318018e70 tx=0x7ff3180092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c0257b2cb905e50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.065+0000 7ff3077fe640 1 -- 172.21.15.5:0/4135162898 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3180055a0 con 0x555f76bf25d0 2024-09-19T06:18:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.065+0000 7ff3077fe640 1 -- 172.21.15.5:0/4135162898 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff31801d040 con 0x555f76bf25d0 2024-09-19T06:18:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.065+0000 7ff3077fe640 1 -- 172.21.15.5:0/4135162898 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318003a70 con 0x555f76bf25d0 2024-09-19T06:18:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.065+0000 7ff3262f9740 1 -- 172.21.15.5:0/4135162898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 msgr2=0x555f76beee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.065+0000 7ff3262f9740 1 --2- 172.21.15.5:0/4135162898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76beee10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff318018e70 tx=0x7ff3180092e0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.066+0000 7ff3262f9740 1 -- 172.21.15.5:0/4135162898 shutdown_connections 2024-09-19T06:18:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.066+0000 7ff3262f9740 1 --2- 172.21.15.5:0/4135162898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bfd370 0x555f76c1d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.066+0000 7ff3262f9740 1 --2- 172.21.15.5:0/4135162898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bf3b40 0x555f76bf6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.066+0000 7ff3262f9740 1 --2- 172.21.15.5:0/4135162898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76beee10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.066+0000 7ff3262f9740 1 -- 172.21.15.5:0/4135162898 >> 172.21.15.5:0/4135162898 conn(0x555f76bbb310 msgr2=0x555f76bee700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.066+0000 7ff3262f9740 1 -- 172.21.15.5:0/4135162898 shutdown_connections 2024-09-19T06:18:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.066+0000 7ff3262f9740 1 -- 172.21.15.5:0/4135162898 wait complete. 2024-09-19T06:18:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.066+0000 7ff3262f9740 1 -- start start 2024-09-19T06:18:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.066+0000 7ff3262f9740 1 Processor -- start 2024-09-19T06:18:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff3262f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76c11c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff3262f9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf3b40 0x555f76c121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff3262f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bfd370 0x555f76c017e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff3262f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff30000fe00 con 0x555f76bfd370 2024-09-19T06:18:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff3262f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff30000f670 con 0x555f76bf25d0 2024-09-19T06:18:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff3262f9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff30000c7e0 con 0x555f76bf3b40 2024-09-19T06:18:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff3262f9740 10 client.0 _ll_register_callbacks cb 0x7ff318009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff321cbc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76c11c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff321cbc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76c11c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50310/0 (socket says 172.21.15.5:50310) 2024-09-19T06:18:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff3214bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf3b40 0x555f76c121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff321cbc640 1 -- 172.21.15.5:0/3399466764 learned_addr learned my addr 172.21.15.5:0/3399466764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff3224bd640 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bfd370 0x555f76c017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff321cbc640 1 -- 172.21.15.5:0/3399466764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf3b40 msgr2=0x555f76c121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff321cbc640 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf3b40 0x555f76c121b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff321cbc640 1 -- 172.21.15.5:0/3399466764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bfd370 msgr2=0x555f76c017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff321cbc640 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bfd370 0x555f76c017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff321cbc640 1 -- 172.21.15.5:0/3399466764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f76c01ed0 con 0x555f76bf25d0 2024-09-19T06:18:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff3224bd640 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bfd370 0x555f76c017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:18:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff321cbc640 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76c11c70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff318003630 tx=0x7ff3180097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db7d4f4d91d10688 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.067+0000 7ff3057fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:18:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.068+0000 7ff3057fa640 1 -- 172.21.15.5:0/3399466764 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318003770 con 0x555f76bf25d0 2024-09-19T06:18:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.068+0000 7ff3057fa640 1 -- 172.21.15.5:0/3399466764 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7ff318003910 con 0x555f76bf25d0 2024-09-19T06:18:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.068+0000 7ff3262f9740 1 -- 172.21.15.5:0/3399466764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff31801d040 con 0x555f76bf25d0 2024-09-19T06:18:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.068+0000 7ff3057fa640 1 -- 172.21.15.5:0/3399466764 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff318003b10 con 0x555f76bf25d0 2024-09-19T06:18:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.068+0000 7ff3262f9740 10 client.28751 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.068+0000 7ff3262f9740 1 -- 172.21.15.5:0/3399466764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555f76c7ff90 con 0x555f76bf25d0 2024-09-19T06:18:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.068+0000 7ff3057fa640 1 -- 172.21.15.5:0/3399466764 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff31801d040 con 0x555f76bf25d0 2024-09-19T06:18:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.069+0000 7ff3057fa640 1 -- 172.21.15.5:0/3399466764 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7ff318035440 con 0x555f76bf25d0 2024-09-19T06:18:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.069+0000 7ff3262f9740 10 client.28751 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.069+0000 7ff3262f9740 10 client.28751 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:18:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.069+0000 7ff3262f9740 20 client.28751 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.069+0000 7ff3262f9740 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f76c80590 0x555f76c82980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.069+0000 7ff3262f9740 4 client.28751 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:18:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.069+0000 7ff3262f9740 1 -- 172.21.15.5:0/3399466764 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f76b65860 con 0x555f76c80590 2024-09-19T06:18:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.069+0000 7ff3214bb640 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f76c80590 0x555f76c82980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.069+0000 7ff3057fa640 10 client.28751 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:18:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3214bb640 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f76c80590 0x555f76c82980 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2b061f3291bc8b87 server_cookie=d5b0b0aa2f7dbc78 in_seq=0 out_seq=0 2024-09-19T06:18:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3057fa640 1 -- 172.21.15.5:0/3399466764 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f76b65860 con 0x555f76c80590 2024-09-19T06:18:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3057fa640 10 client.28751 handle_command_reply: tid=0 2024-09-19T06:18:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3262f9740 1 client.28751 shutdown 2024-09-19T06:18:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3262f9740 1 -- 172.21.15.5:0/3399466764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 msgr2=0x555f76c11c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3262f9740 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76c11c70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff318003630 tx=0x7ff3180097f0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3262f9740 1 -- 172.21.15.5:0/3399466764 shutdown_connections 2024-09-19T06:18:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3262f9740 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555f76c80590 0x555f76c82980 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3262f9740 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f76bfd370 0x555f76c017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3262f9740 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f76bf3b40 0x555f76c121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3057fa640 0 client.28751 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:18:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3262f9740 1 --2- 172.21.15.5:0/3399466764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555f76bf25d0 0x555f76c11c70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3262f9740 1 -- 172.21.15.5:0/3399466764 >> 172.21.15.5:0/3399466764 conn(0x555f76bbb310 msgr2=0x555f76bf4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3262f9740 1 -- 172.21.15.5:0/3399466764 shutdown_connections 2024-09-19T06:18:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3262f9740 1 -- 172.21.15.5:0/3399466764 wait complete. 2024-09-19T06:18:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.070+0000 7ff3262f9740 20 client.28751 trim_cache size 0 max 16384 2024-09-19T06:18:28.077 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:18:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.071+0000 7ff323f20640 1 -- 172.21.15.5:0/2873907382 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff3080a3430 msgr2=0x7ff3080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.071+0000 7ff323f20640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff3080a3430 0x7ff3080a58f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff314003fd0 tx=0x7ff314030bb0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.071+0000 7ff323f20640 1 -- 172.21.15.5:0/2873907382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff31c102570 msgr2=0x7ff31c197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.071+0000 7ff323f20640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff31c102570 0x7ff31c197870 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00e9f0 tx=0x7ff30c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.071+0000 7ff323f20640 1 -- 172.21.15.5:0/2873907382 shutdown_connections 2024-09-19T06:18:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.071+0000 7ff323f20640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff3080a3430 0x7ff3080a58f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.071+0000 7ff323f20640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff31c102ef0 0x7ff31c19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.071+0000 7ff323f20640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff31c102570 0x7ff31c197870 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.071+0000 7ff323f20640 1 --2- 172.21.15.5:0/2873907382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff31c100e20 0x7ff31c197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.071+0000 7ff323f20640 1 -- 172.21.15.5:0/2873907382 >> 172.21.15.5:0/2873907382 conn(0x7ff31c0fc7d0 msgr2=0x7ff31c1048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.072+0000 7ff323f20640 1 -- 172.21.15.5:0/2873907382 shutdown_connections 2024-09-19T06:18:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:28.072+0000 7ff323f20640 1 -- 172.21.15.5:0/2873907382 wait complete. 2024-09-19T06:18:28.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 18831 10.52 MB/sec warmup 112 sec latency 87.652 ms 2024-09-19T06:18:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:28 smithi157 ceph-mon[35346]: pgmap v475: 97 pgs: 97 active+clean; 520 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 406 op/s 2024-09-19T06:18:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:28 smithi005 ceph-mon[30955]: pgmap v475: 97 pgs: 97 active+clean; 520 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 406 op/s 2024-09-19T06:18:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:28 smithi049 ceph-mon[34640]: pgmap v475: 97 pgs: 97 active+clean; 520 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 406 op/s 2024-09-19T06:18:28.983 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:18:29.344 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:18:29.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 18998 10.50 MB/sec warmup 113 sec latency 69.296 ms 2024-09-19T06:18:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.994+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4123361151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08109110 msgr2=0x7f4e08110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.994+0000 7f4e0ff89640 1 --2- 172.21.15.5:0/4123361151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08109110 0x7f4e08110c90 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009ca0 tx=0x7f4e0002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:18:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.994+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4123361151 shutdown_connections 2024-09-19T06:18:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.994+0000 7f4e0ff89640 1 --2- 172.21.15.5:0/4123361151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08109110 0x7f4e08110c90 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.994+0000 7f4e0ff89640 1 --2- 172.21.15.5:0/4123361151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e08104990 0x7f4e081089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.994+0000 7f4e0ff89640 1 --2- 172.21.15.5:0/4123361151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e08103fe0 0x7f4e081043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:29.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.994+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4123361151 >> 172.21.15.5:0/4123361151 conn(0x7f4e080ffa60 msgr2=0x7f4e08101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:29.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.994+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4123361151 shutdown_connections 2024-09-19T06:18:29.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.994+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4123361151 wait complete. 2024-09-19T06:18:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.995+0000 7f4e0ff89640 1 Processor -- start 2024-09-19T06:18:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.995+0000 7f4e0ff89640 1 -- start start 2024-09-19T06:18:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.995+0000 7f4e0ff89640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e08103fe0 0x7f4e0819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.995+0000 7f4e0ff89640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08104990 0x7f4e0819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.995+0000 7f4e0ff89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e08109110 0x7f4e0819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.995+0000 7f4e0dd25640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e08103fe0 0x7f4e0819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.995+0000 7f4e0ff89640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08113fd0 con 0x7f4e08104990 2024-09-19T06:18:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.995+0000 7f4e0ff89640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08113e50 con 0x7f4e08103fe0 2024-09-19T06:18:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.995+0000 7f4e0ff89640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08114150 con 0x7f4e08109110 2024-09-19T06:18:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.995+0000 7f4e0d524640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08104990 0x7f4e0819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.995+0000 7f4e0dd25640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e08103fe0 0x7f4e0819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50326/0 (socket says 172.21.15.5:50326) 2024-09-19T06:18:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.995+0000 7f4e0dd25640 1 -- 172.21.15.5:0/4162151660 learned_addr learned my addr 172.21.15.5:0/4162151660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.996+0000 7f4e0e526640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e08109110 0x7f4e0819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.996+0000 7f4e0d524640 1 -- 172.21.15.5:0/4162151660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e08109110 msgr2=0x7f4e0819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.996+0000 7f4e0d524640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e08109110 0x7f4e0819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.996+0000 7f4e0d524640 1 -- 172.21.15.5:0/4162151660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e08103fe0 msgr2=0x7f4e0819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.996+0000 7f4e0d524640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e08103fe0 0x7f4e0819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.996+0000 7f4e0d524640 1 -- 172.21.15.5:0/4162151660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0819f460 con 0x7f4e08104990 2024-09-19T06:18:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.996+0000 7f4e0e526640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e08109110 0x7f4e0819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.997+0000 7f4e0d524640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08104990 0x7f4e0819a9f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4df800e9f0 tx=0x7f4df800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.997+0000 7f4dfeffd640 1 -- 172.21.15.5:0/4162151660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df800cd80 con 0x7f4e08104990 2024-09-19T06:18:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.997+0000 7f4dfeffd640 1 -- 172.21.15.5:0/4162151660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4df80035f0 con 0x7f4e08104990 2024-09-19T06:18:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.997+0000 7f4dfeffd640 1 -- 172.21.15.5:0/4162151660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8010690 con 0x7f4e08104990 2024-09-19T06:18:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.997+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4162151660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0819f750 con 0x7f4e08104990 2024-09-19T06:18:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:29.997+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4162151660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e08076630 con 0x7f4e08104990 2024-09-19T06:18:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.000+0000 7f4dfeffd640 1 -- 172.21.15.5:0/4162151660 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4df80037a0 con 0x7f4e08104990 2024-09-19T06:18:30.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.001+0000 7f4e12362740 1 Processor -- start 2024-09-19T06:18:30.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.001+0000 7f4e12362740 1 -- start start 2024-09-19T06:18:30.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.001+0000 7f4e12362740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6b0b40 0x55750f6af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.001+0000 7f4e12362740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6af5d0 0x55750f6afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.001+0000 7f4e12362740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6ba370 0x55750f6bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.001+0000 7f4e12362740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55750f6152f0 con 0x55750f6b0b40 2024-09-19T06:18:30.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.001+0000 7f4e12362740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55750f5bccf0 con 0x55750f6ba370 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.001+0000 7f4e12362740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55750f6112b0 con 0x55750f6af5d0 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.001+0000 7f4e0e526640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6b0b40 0x55750f6af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.001+0000 7f4e0d524640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6ba370 0x55750f6bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4e0e526640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6b0b40 0x55750f6af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60586/0 (socket says 172.21.15.5:60586) 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4e0d524640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6ba370 0x55750f6bc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50340/0 (socket says 172.21.15.5:50340) 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4e0e526640 1 -- 172.21.15.5:0/722130608 learned_addr learned my addr 172.21.15.5:0/722130608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4dfeffd640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4df40a3430 0x7f4df40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4e0dd25640 1 --2- 172.21.15.5:0/722130608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6af5d0 0x55750f6afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4e0d524640 1 -- 172.21.15.5:0/722130608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6af5d0 msgr2=0x55750f6afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4e0d524640 1 --2- 172.21.15.5:0/722130608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6af5d0 0x55750f6afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4e0d524640 1 -- 172.21.15.5:0/722130608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6b0b40 msgr2=0x55750f6af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4e0d524640 1 --2- 172.21.15.5:0/722130608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6b0b40 0x55750f6af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4e0d524640 1 -- 172.21.15.5:0/722130608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55750f5a13a0 con 0x55750f6ba370 2024-09-19T06:18:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4e0dd25640 1 --2- 172.21.15.5:0/722130608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6af5d0 0x55750f6afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:18:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4dfeffd640 1 -- 172.21.15.5:0/4162151660 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f4df80cff90 con 0x7f4e08104990 2024-09-19T06:18:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4e0d524640 1 --2- 172.21.15.5:0/722130608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6ba370 0x55750f6bc760 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4df80cd660 tx=0x7f4df8021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d544f2d3f21277 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.002+0000 7f4e0e526640 1 --2- 172.21.15.5:0/722130608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6b0b40 0x55750f6af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.003+0000 7f4df3fff640 1 -- 172.21.15.5:0/722130608 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df809bd30 con 0x55750f6ba370 2024-09-19T06:18:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.003+0000 7f4e0e526640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4df40a3430 0x7f4df40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.003+0000 7f4df3fff640 1 -- 172.21.15.5:0/722130608 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4df8003b10 con 0x55750f6ba370 2024-09-19T06:18:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.003+0000 7f4df3fff640 1 -- 172.21.15.5:0/722130608 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df809bb50 con 0x55750f6ba370 2024-09-19T06:18:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.004+0000 7f4e12362740 1 -- 172.21.15.5:0/722130608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6ba370 msgr2=0x55750f6bc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.004+0000 7f4e0e526640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4df40a3430 0x7f4df40a58f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4e00004af0 tx=0x7f4e000095c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.004+0000 7f4e12362740 1 --2- 172.21.15.5:0/722130608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6ba370 0x55750f6bc760 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4df80cd660 tx=0x7f4df8021000 comp rx=0 tx=0).stop 2024-09-19T06:18:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.004+0000 7f4e12362740 1 -- 172.21.15.5:0/722130608 shutdown_connections 2024-09-19T06:18:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.004+0000 7f4e12362740 1 --2- 172.21.15.5:0/722130608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6ba370 0x55750f6bc760 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.004+0000 7f4e12362740 1 --2- 172.21.15.5:0/722130608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6af5d0 0x55750f6afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.004+0000 7f4e12362740 1 --2- 172.21.15.5:0/722130608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6b0b40 0x55750f6af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.004+0000 7f4e12362740 1 -- 172.21.15.5:0/722130608 >> 172.21.15.5:0/722130608 conn(0x55750f678310 msgr2=0x55750f6abc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.004+0000 7f4e12362740 1 -- 172.21.15.5:0/722130608 shutdown_connections 2024-09-19T06:18:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.004+0000 7f4e12362740 1 -- 172.21.15.5:0/722130608 wait complete. 2024-09-19T06:18:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.004+0000 7f4e12362740 1 -- start start 2024-09-19T06:18:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.004+0000 7f4e12362740 1 Processor -- start 2024-09-19T06:18:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e12362740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 0x55750f6b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e12362740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6b0b40 0x55750f6b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e12362740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6ba370 0x55750f6b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e12362740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55750f6bd010 con 0x55750f6af5d0 2024-09-19T06:18:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e12362740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55750f5d0060 con 0x55750f6b0b40 2024-09-19T06:18:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e12362740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55750f6bd190 con 0x55750f6ba370 2024-09-19T06:18:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e12362740 10 client.0 _ll_register_callbacks cb 0x55750f3238c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e0e526640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6ba370 0x55750f6b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e0dd25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 0x55750f6b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e0d524640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6b0b40 0x55750f6b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e0dd25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 0x55750f6b9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60594/0 (socket says 172.21.15.5:60594) 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e0d524640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6b0b40 0x55750f6b6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50354/0 (socket says 172.21.15.5:50354) 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e0e526640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6ba370 0x55750f6b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42358/0 (socket says 172.21.15.5:42358) 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.005+0000 7f4e0dd25640 1 -- 172.21.15.5:0/3130857812 learned_addr learned my addr 172.21.15.5:0/3130857812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.006+0000 7f4e0dd25640 1 -- 172.21.15.5:0/3130857812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6ba370 msgr2=0x55750f6b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.006+0000 7f4e0dd25640 1 --2- 172.21.15.5:0/3130857812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6ba370 0x55750f6b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.006+0000 7f4e0dd25640 1 -- 172.21.15.5:0/3130857812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6b0b40 msgr2=0x55750f6b6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.006+0000 7f4e0dd25640 1 --2- 172.21.15.5:0/3130857812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6b0b40 0x55750f6b6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.006+0000 7f4e0dd25640 1 -- 172.21.15.5:0/3130857812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55750f3863c0 con 0x55750f6af5d0 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.006+0000 7f4e0dd25640 1 --2- 172.21.15.5:0/3130857812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 0x55750f6b9d10 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009f80 tx=0x7f4e0400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c2f4c4f99df30e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.006+0000 7f4df17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.006+0000 7f4df17fa640 1 -- 172.21.15.5:0/3130857812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04004020 con 0x55750f6af5d0 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.006+0000 7f4df17fa640 1 -- 172.21.15.5:0/3130857812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f4e040041c0 con 0x55750f6af5d0 2024-09-19T06:18:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.006+0000 7f4e12362740 1 -- 172.21.15.5:0/3130857812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55750f622860 con 0x55750f6af5d0 2024-09-19T06:18:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.006+0000 7f4df17fa640 1 -- 172.21.15.5:0/3130857812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4e04004400 con 0x55750f6af5d0 2024-09-19T06:18:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.007+0000 7f4e12362740 10 client.19371 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.007+0000 7f4e12362740 1 -- 172.21.15.5:0/3130857812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55750f6c8ac0 con 0x55750f6af5d0 2024-09-19T06:18:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.007+0000 7f4df17fa640 1 -- 172.21.15.5:0/3130857812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e040054d0 con 0x55750f6af5d0 2024-09-19T06:18:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.007+0000 7f4df17fa640 1 -- 172.21.15.5:0/3130857812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7f4e040035f0 con 0x55750f6af5d0 2024-09-19T06:18:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.007+0000 7f4e12362740 10 client.19371 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.007+0000 7f4e12362740 10 client.19371 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:18:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.007+0000 7f4e12362740 20 client.19371 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.007+0000 7f4e12362740 1 --2- 172.21.15.5:0/3130857812 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55750f6c9fa0 0x55750f6cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.007+0000 7f4e12362740 4 client.19371 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.007+0000 7f4e12362740 1 -- 172.21.15.5:0/3130857812 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55750f622860 con 0x55750f6c9fa0 2024-09-19T06:18:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.008+0000 7f4e0d524640 1 --2- 172.21.15.5:0/3130857812 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55750f6c9fa0 0x55750f6cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.009+0000 7f4df17fa640 10 client.19371 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:18:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.009+0000 7f4e0d524640 1 --2- 172.21.15.5:0/3130857812 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55750f6c9fa0 0x55750f6cc3c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c12b79e315f3bb40 server_cookie=e1a41dcdb4b47f7d in_seq=0 out_seq=0 2024-09-19T06:18:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4df17fa640 1 -- 172.21.15.5:0/3130857812 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55750f622860 con 0x55750f6c9fa0 2024-09-19T06:18:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4df17fa640 10 client.19371 handle_command_reply: tid=0 2024-09-19T06:18:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4e12362740 1 client.19371 shutdown 2024-09-19T06:18:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4e12362740 1 -- 172.21.15.5:0/3130857812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 msgr2=0x55750f6b9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4e12362740 1 --2- 172.21.15.5:0/3130857812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 0x55750f6b9d10 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009f80 tx=0x7f4e0400b690 comp rx=0 tx=0).stop 2024-09-19T06:18:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4e12362740 1 -- 172.21.15.5:0/3130857812 shutdown_connections 2024-09-19T06:18:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4e12362740 1 --2- 172.21.15.5:0/3130857812 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55750f6c9fa0 0x55750f6cc3c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4e12362740 1 --2- 172.21.15.5:0/3130857812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6ba370 0x55750f6b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4e12362740 1 --2- 172.21.15.5:0/3130857812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6b0b40 0x55750f6b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4e12362740 1 --2- 172.21.15.5:0/3130857812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 0x55750f6b9d10 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4e12362740 1 -- 172.21.15.5:0/3130857812 >> 172.21.15.5:0/3130857812 conn(0x55750f678310 msgr2=0x55750f6b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4df17fa640 0 client.19371 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:18:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4e12362740 1 -- 172.21.15.5:0/3130857812 shutdown_connections 2024-09-19T06:18:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4e12362740 1 -- 172.21.15.5:0/3130857812 wait complete. 2024-09-19T06:18:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.010+0000 7f4e12362740 20 client.19371 trim_cache size 0 max 16384 2024-09-19T06:18:30.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.031+0000 7f4e12362740 1 Processor -- start 2024-09-19T06:18:30.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.031+0000 7f4e12362740 1 -- start start 2024-09-19T06:18:30.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.031+0000 7f4e12362740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6af5d0 0x55750f6aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.031+0000 7f4e12362740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6b0b40 0x55750f6aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.031+0000 7f4e12362740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6ba370 0x55750f6b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.031+0000 7f4e12362740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081ab6b0 con 0x55750f6ba370 2024-09-19T06:18:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.031+0000 7f4e12362740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081ad3a0 con 0x55750f6af5d0 2024-09-19T06:18:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.031+0000 7f4e12362740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081abe40 con 0x55750f6b0b40 2024-09-19T06:18:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.031+0000 7f4e0dd25640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6af5d0 0x55750f6aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.031+0000 7f4e0e526640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6ba370 0x55750f6b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.031+0000 7f4e0dd25640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6af5d0 0x55750f6aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50362/0 (socket says 172.21.15.5:50362) 2024-09-19T06:18:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.032+0000 7f4e0dd25640 1 -- 172.21.15.5:0/1514561964 learned_addr learned my addr 172.21.15.5:0/1514561964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.032+0000 7f4e0dd25640 1 -- 172.21.15.5:0/1514561964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6b0b40 msgr2=0x55750f6aabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.032+0000 7f4e0dd25640 1 --2- 172.21.15.5:0/1514561964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6b0b40 0x55750f6aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.032+0000 7f4e0dd25640 1 -- 172.21.15.5:0/1514561964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6ba370 msgr2=0x55750f6b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.032+0000 7f4e0dd25640 1 --2- 172.21.15.5:0/1514561964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6ba370 0x55750f6b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.032+0000 7f4e0dd25640 1 -- 172.21.15.5:0/1514561964 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55750f6b5f70 con 0x55750f6af5d0 2024-09-19T06:18:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.032+0000 7f4e0dd25640 1 --2- 172.21.15.5:0/1514561964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6af5d0 0x55750f6aed90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4e04029040 tx=0x7f4e04009af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fcf88dea50978cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.033+0000 7f4df37fe640 1 -- 172.21.15.5:0/1514561964 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e040037f0 con 0x55750f6af5d0 2024-09-19T06:18:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.033+0000 7f4df37fe640 1 -- 172.21.15.5:0/1514561964 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e04004130 con 0x55750f6af5d0 2024-09-19T06:18:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.033+0000 7f4df37fe640 1 -- 172.21.15.5:0/1514561964 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e040045a0 con 0x55750f6af5d0 2024-09-19T06:18:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.034+0000 7f4e12362740 1 -- 172.21.15.5:0/1514561964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6af5d0 msgr2=0x55750f6aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.034+0000 7f4e12362740 1 --2- 172.21.15.5:0/1514561964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6af5d0 0x55750f6aed90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4e04029040 tx=0x7f4e04009af0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.034+0000 7f4e12362740 1 -- 172.21.15.5:0/1514561964 shutdown_connections 2024-09-19T06:18:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.034+0000 7f4e12362740 1 --2- 172.21.15.5:0/1514561964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6ba370 0x55750f6b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.034+0000 7f4e12362740 1 --2- 172.21.15.5:0/1514561964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6b0b40 0x55750f6aabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.034+0000 7f4e12362740 1 --2- 172.21.15.5:0/1514561964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6af5d0 0x55750f6aed90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.034+0000 7f4e12362740 1 -- 172.21.15.5:0/1514561964 >> 172.21.15.5:0/1514561964 conn(0x55750f678310 msgr2=0x55750f6aa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:30.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.034+0000 7f4e12362740 1 -- 172.21.15.5:0/1514561964 shutdown_connections 2024-09-19T06:18:30.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.034+0000 7f4e12362740 1 -- 172.21.15.5:0/1514561964 wait complete. 2024-09-19T06:18:30.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.035+0000 7f4e12362740 1 -- start start 2024-09-19T06:18:30.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.035+0000 7f4e12362740 1 Processor -- start 2024-09-19T06:18:30.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.036+0000 7f4e12362740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 0x55750f6c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.036+0000 7f4e12362740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6b0b40 0x55750f6b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.036+0000 7f4e12362740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6ba370 0x55750f6b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.036+0000 7f4e0dd25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 0x55750f6c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.036+0000 7f4e0dd25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 0x55750f6c2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60624/0 (socket says 172.21.15.5:60624) 2024-09-19T06:18:30.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.036+0000 7f4e0dd25640 1 -- 172.21.15.5:0/507899421 learned_addr learned my addr 172.21.15.5:0/507899421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.036+0000 7f4e12362740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081accd0 con 0x55750f6af5d0 2024-09-19T06:18:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.036+0000 7f4e12362740 1 -- 172.21.15.5:0/507899421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081ac540 con 0x55750f6b0b40 2024-09-19T06:18:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.036+0000 7f4e12362740 1 -- 172.21.15.5:0/507899421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081a96b0 con 0x55750f6ba370 2024-09-19T06:18:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.036+0000 7f4e0e526640 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6ba370 0x55750f6b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.036+0000 7f4e0d524640 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6b0b40 0x55750f6b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.036+0000 7f4e12362740 10 client.0 _ll_register_callbacks cb 0x7f4e04009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.037+0000 7f4e0e526640 1 -- 172.21.15.5:0/507899421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6b0b40 msgr2=0x55750f6b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.037+0000 7f4e0e526640 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6b0b40 0x55750f6b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.037+0000 7f4e0e526640 1 -- 172.21.15.5:0/507899421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 msgr2=0x55750f6c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.037+0000 7f4e0e526640 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 0x55750f6c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.037+0000 7f4e0e526640 1 -- 172.21.15.5:0/507899421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55750f6b4a00 con 0x55750f6ba370 2024-09-19T06:18:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.037+0000 7f4e0dd25640 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 0x55750f6c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.037+0000 7f4e0e526640 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6ba370 0x55750f6b4200 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e0002ade0 tx=0x7f4e0002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ca9b3179fc8f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.037+0000 7f4df17fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:18:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.037+0000 7f4df17fa640 1 -- 172.21.15.5:0/507899421 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00004650 con 0x55750f6ba370 2024-09-19T06:18:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.037+0000 7f4df17fa640 1 -- 172.21.15.5:0/507899421 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f4e000047f0 con 0x55750f6ba370 2024-09-19T06:18:30.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.037+0000 7f4e12362740 1 -- 172.21.15.5:0/507899421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e04004130 con 0x55750f6ba370 2024-09-19T06:18:30.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.038+0000 7f4df17fa640 1 -- 172.21.15.5:0/507899421 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4e00063050 con 0x55750f6ba370 2024-09-19T06:18:30.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.038+0000 7f4e12362740 10 client.28708 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:30.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.038+0000 7f4e12362740 1 -- 172.21.15.5:0/507899421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55750f734b30 con 0x55750f6ba370 2024-09-19T06:18:30.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.038+0000 7f4df17fa640 1 -- 172.21.15.5:0/507899421 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e04004130 con 0x55750f6ba370 2024-09-19T06:18:30.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.038+0000 7f4df17fa640 1 -- 172.21.15.5:0/507899421 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7f4e000557d0 con 0x55750f6ba370 2024-09-19T06:18:30.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.038+0000 7f4e12362740 10 client.28708 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:30.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.038+0000 7f4e12362740 10 client.28708 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:18:30.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.038+0000 7f4e12362740 20 client.28708 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:30.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.038+0000 7f4e12362740 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55750f7353c0 0x55750f7377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:30.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.038+0000 7f4e12362740 4 client.28708 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:18:30.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.038+0000 7f4e12362740 1 -- 172.21.15.5:0/507899421 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55750f61cd70 con 0x55750f7353c0 2024-09-19T06:18:30.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.040+0000 7f4e0dd25640 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55750f7353c0 0x55750f7377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:30.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4e0dd25640 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55750f7353c0 0x55750f7377e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8ff83548a105dfad server_cookie=647aeb8cf212ed12 in_seq=0 out_seq=0 2024-09-19T06:18:30.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4df17fa640 10 client.28708 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:18:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4df17fa640 1 -- 172.21.15.5:0/507899421 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55750f61cd70 con 0x55750f7353c0 2024-09-19T06:18:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4df17fa640 10 client.28708 handle_command_reply: tid=0 2024-09-19T06:18:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4e12362740 1 client.28708 shutdown 2024-09-19T06:18:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4e12362740 1 -- 172.21.15.5:0/507899421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6ba370 msgr2=0x55750f6b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4e12362740 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6ba370 0x55750f6b4200 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e0002ade0 tx=0x7f4e0002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4e12362740 1 -- 172.21.15.5:0/507899421 shutdown_connections 2024-09-19T06:18:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4e12362740 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55750f7353c0 0x55750f7377e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4e12362740 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55750f6ba370 0x55750f6b4200 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4e12362740 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55750f6b0b40 0x55750f6b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4e12362740 1 --2- 172.21.15.5:0/507899421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55750f6af5d0 0x55750f6c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4df17fa640 0 client.28708 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:18:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.042+0000 7f4e12362740 1 -- 172.21.15.5:0/507899421 >> 172.21.15.5:0/507899421 conn(0x55750f678310 msgr2=0x55750f6aad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.043+0000 7f4e12362740 1 -- 172.21.15.5:0/507899421 shutdown_connections 2024-09-19T06:18:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.043+0000 7f4e12362740 1 -- 172.21.15.5:0/507899421 wait complete. 2024-09-19T06:18:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.043+0000 7f4e12362740 20 client.28708 trim_cache size 0 max 16384 2024-09-19T06:18:30.046 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:18:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.043+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4162151660 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4df40a3430 msgr2=0x7f4df40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.043+0000 7f4e0ff89640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4df40a3430 0x7f4df40a58f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4e00004af0 tx=0x7f4e000095c0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.043+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4162151660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08104990 msgr2=0x7f4e0819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.043+0000 7f4e0ff89640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08104990 0x7f4e0819a9f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4df800e9f0 tx=0x7f4df800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.044+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4162151660 shutdown_connections 2024-09-19T06:18:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.044+0000 7f4e0ff89640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4df40a3430 0x7f4df40a58f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.044+0000 7f4e0ff89640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e08109110 0x7f4e0819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.044+0000 7f4e0ff89640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08104990 0x7f4e0819a9f0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.044+0000 7f4e0ff89640 1 --2- 172.21.15.5:0/4162151660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e08103fe0 0x7f4e0819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.044+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4162151660 >> 172.21.15.5:0/4162151660 conn(0x7f4e080ffa60 msgr2=0x7f4e08100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.044+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4162151660 shutdown_connections 2024-09-19T06:18:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:30.044+0000 7f4e0ff89640 1 -- 172.21.15.5:0/4162151660 wait complete. 2024-09-19T06:18:30.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 19158 10.48 MB/sec warmup 114 sec latency 94.420 ms 2024-09-19T06:18:30.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:30 smithi005 ceph-mon[30955]: pgmap v476: 97 pgs: 97 active+clean; 520 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-09-19T06:18:30.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:18:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:30 smithi157 ceph-mon[35346]: pgmap v476: 97 pgs: 97 active+clean; 520 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-09-19T06:18:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:30 smithi049 ceph-mon[34640]: pgmap v476: 97 pgs: 97 active+clean; 520 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-09-19T06:18:31.118 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:18:31.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 19355 10.52 MB/sec warmup 115 sec latency 104.662 ms 2024-09-19T06:18:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.678+0000 7fcfa0121640 1 -- 172.21.15.5:0/233775195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf98102570 msgr2=0x7fcf981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.678+0000 7fcfa0121640 1 --2- 172.21.15.5:0/233775195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf98102570 0x7fcf981029b0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009900 tx=0x7fcf8802a480 comp rx=0 tx=0).stop 2024-09-19T06:18:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.678+0000 7fcfa0121640 1 -- 172.21.15.5:0/233775195 shutdown_connections 2024-09-19T06:18:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.678+0000 7fcfa0121640 1 --2- 172.21.15.5:0/233775195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf98102ef0 0x7fcf9810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.678+0000 7fcfa0121640 1 --2- 172.21.15.5:0/233775195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf98102570 0x7fcf981029b0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.678+0000 7fcfa0121640 1 --2- 172.21.15.5:0/233775195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf98100e20 0x7fcf98101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.678+0000 7fcfa0121640 1 -- 172.21.15.5:0/233775195 >> 172.21.15.5:0/233775195 conn(0x7fcf980fc7d0 msgr2=0x7fcf980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.678+0000 7fcfa0121640 1 -- 172.21.15.5:0/233775195 shutdown_connections 2024-09-19T06:18:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.678+0000 7fcfa0121640 1 -- 172.21.15.5:0/233775195 wait complete. 2024-09-19T06:18:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcfa0121640 1 Processor -- start 2024-09-19T06:18:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcfa0121640 1 -- start start 2024-09-19T06:18:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcfa0121640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf98100e20 0x7fcf981a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcfa0121640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf98102570 0x7fcf981a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcfa0121640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf98102ef0 0x7fcf9819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcfa0121640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98110e30 con 0x7fcf98102570 2024-09-19T06:18:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcfa0121640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98110cb0 con 0x7fcf98102ef0 2024-09-19T06:18:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcfa0121640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98110fb0 con 0x7fcf98100e20 2024-09-19T06:18:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcf9d6bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf98102570 0x7fcf981a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcf9debd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf98100e20 0x7fcf981a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcf9debd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf98100e20 0x7fcf981a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42396/0 (socket says 172.21.15.5:42396) 2024-09-19T06:18:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcf9d6bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf98102570 0x7fcf981a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60642/0 (socket says 172.21.15.5:60642) 2024-09-19T06:18:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcf9d6bc640 1 -- 172.21.15.5:0/2111252380 learned_addr learned my addr 172.21.15.5:0/2111252380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.679+0000 7fcf9e6be640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf98102ef0 0x7fcf9819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.680+0000 7fcf9d6bc640 1 -- 172.21.15.5:0/2111252380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf98100e20 msgr2=0x7fcf981a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.680+0000 7fcf9d6bc640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf98100e20 0x7fcf981a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.680+0000 7fcf9d6bc640 1 -- 172.21.15.5:0/2111252380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf98102ef0 msgr2=0x7fcf9819ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.680+0000 7fcf9d6bc640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf98102ef0 0x7fcf9819ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.680+0000 7fcf9d6bc640 1 -- 172.21.15.5:0/2111252380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9819f270 con 0x7fcf98102570 2024-09-19T06:18:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.680+0000 7fcf9debd640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf98100e20 0x7fcf981a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.680+0000 7fcf9d6bc640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf98102570 0x7fcf981a4ba0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fcf88006010 tx=0x7fcf88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.680+0000 7fcf8effd640 1 -- 172.21.15.5:0/2111252380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8803f030 con 0x7fcf98102570 2024-09-19T06:18:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.680+0000 7fcf8effd640 1 -- 172.21.15.5:0/2111252380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf88003aa0 con 0x7fcf98102570 2024-09-19T06:18:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.680+0000 7fcfa0121640 1 -- 172.21.15.5:0/2111252380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9819f500 con 0x7fcf98102570 2024-09-19T06:18:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.680+0000 7fcf8effd640 1 -- 172.21.15.5:0/2111252380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88035070 con 0x7fcf98102570 2024-09-19T06:18:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.680+0000 7fcfa0121640 1 -- 172.21.15.5:0/2111252380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf981ab4c0 con 0x7fcf98102570 2024-09-19T06:18:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcfa24fa740 1 Processor -- start 2024-09-19T06:18:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcfa24fa740 1 -- start start 2024-09-19T06:18:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcf8effd640 1 -- 172.21.15.5:0/2111252380 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcf88042030 con 0x7fcf98102570 2024-09-19T06:18:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcfa24fa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee57090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcfa24fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee57a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcfa24fa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee62370 0x557cbee64760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcfa24fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557cbedbd2f0 con 0x557cbee575d0 2024-09-19T06:18:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcfa24fa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557cbed64cf0 con 0x557cbee62370 2024-09-19T06:18:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcfa24fa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557cbedb92b0 con 0x557cbee58b40 2024-09-19T06:18:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcf9e6be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee57090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcf9debd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee57a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcf9e6be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee57090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42400/0 (socket says 172.21.15.5:42400) 2024-09-19T06:18:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.684+0000 7fcf9e6be640 1 -- 172.21.15.5:0/2511425939 learned_addr learned my addr 172.21.15.5:0/2511425939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf9d6bc640 1 --2- 172.21.15.5:0/2511425939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee62370 0x557cbee64760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf9e6be640 1 -- 172.21.15.5:0/2511425939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee62370 msgr2=0x557cbee64760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf9e6be640 1 --2- 172.21.15.5:0/2511425939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee62370 0x557cbee64760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf9e6be640 1 -- 172.21.15.5:0/2511425939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 msgr2=0x557cbee57a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf9e6be640 1 --2- 172.21.15.5:0/2511425939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee57a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf9e6be640 1 -- 172.21.15.5:0/2511425939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cbed493a0 con 0x557cbee58b40 2024-09-19T06:18:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf9d6bc640 1 --2- 172.21.15.5:0/2511425939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee62370 0x557cbee64760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:18:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf9e6be640 1 --2- 172.21.15.5:0/2511425939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee57090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009f50 tx=0x7fcf900118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8205d9849401feb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf8effd640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcf840a3430 0x7fcf840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf83fff640 1 -- 172.21.15.5:0/2511425939 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90004d20 con 0x557cbee58b40 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf83fff640 1 -- 172.21.15.5:0/2511425939 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf90004050 con 0x557cbee58b40 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf83fff640 1 -- 172.21.15.5:0/2511425939 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90005280 con 0x557cbee58b40 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.685+0000 7fcf9debd640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcf840a3430 0x7fcf840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.686+0000 7fcf8effd640 1 -- 172.21.15.5:0/2111252380 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fcf880efff0 con 0x7fcf98102570 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.686+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2511425939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 msgr2=0x557cbee57090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.686+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/2511425939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee57090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009f50 tx=0x7fcf900118d0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.686+0000 7fcf9debd640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcf840a3430 0x7fcf840a58f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x557cbee54ac0 tx=0x7fcf9400b410 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.686+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2511425939 shutdown_connections 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.686+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/2511425939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee62370 0x557cbee64760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.686+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/2511425939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee57a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.686+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/2511425939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee57090 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.687+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2511425939 >> 172.21.15.5:0/2511425939 conn(0x557cbee20310 msgr2=0x557cbee53c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.687+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2511425939 shutdown_connections 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.687+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2511425939 wait complete. 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.687+0000 7fcfa24fa740 1 -- start start 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.687+0000 7fcfa24fa740 1 Processor -- start 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.688+0000 7fcfa24fa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee575d0 0x557cbee61cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.688+0000 7fcfa24fa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee5ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.688+0000 7fcfa24fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee62370 0x557cbee5f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.688+0000 7fcfa24fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557cbee65070 con 0x557cbee62370 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.688+0000 7fcfa24fa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557cbed78060 con 0x557cbee575d0 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.688+0000 7fcfa24fa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557cbee651f0 con 0x557cbee58b40 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.688+0000 7fcfa24fa740 10 client.0 _ll_register_callbacks cb 0x557cbeacb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.688+0000 7fcf9d6bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee5ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.688+0000 7fcf9d6bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee5ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42402/0 (socket says 172.21.15.5:42402) 2024-09-19T06:18:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.688+0000 7fcf9d6bc640 1 -- 172.21.15.5:0/2734663512 learned_addr learned my addr 172.21.15.5:0/2734663512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.688+0000 7fcf9e6be640 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee62370 0x557cbee5f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcf9d6bc640 1 -- 172.21.15.5:0/2734663512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee575d0 msgr2=0x557cbee61cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcf9d6bc640 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee575d0 0x557cbee61cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcf9d6bc640 1 -- 172.21.15.5:0/2734663512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee62370 msgr2=0x557cbee5f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcf9d6bc640 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee62370 0x557cbee5f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcf9d6bc640 1 -- 172.21.15.5:0/2734663512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cbeb2e3c0 con 0x557cbee58b40 2024-09-19T06:18:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcf9e6be640 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee62370 0x557cbee5f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcf9d6bc640 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee5ef80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcf880edd20 tx=0x7fcf880b7fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f53895e757149917 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcf817fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:18:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcf817fa640 1 -- 172.21.15.5:0/2734663512 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88049050 con 0x557cbee58b40 2024-09-19T06:18:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2734663512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf90004050 con 0x557cbee58b40 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcf817fa640 1 -- 172.21.15.5:0/2734663512 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fcf88003c50 con 0x557cbee58b40 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcf817fa640 1 -- 172.21.15.5:0/2734663512 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fcf880bbea0 con 0x557cbee58b40 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcfa24fa740 10 client.28720 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.689+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2734663512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557cbee650c0 con 0x557cbee58b40 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.690+0000 7fcf817fa640 1 -- 172.21.15.5:0/2734663512 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf90004050 con 0x557cbee58b40 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.690+0000 7fcf817fa640 1 -- 172.21.15.5:0/2734663512 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7fcf880d50f0 con 0x557cbee58b40 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.690+0000 7fcfa24fa740 10 client.28720 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.690+0000 7fcfa24fa740 10 client.28720 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.690+0000 7fcfa24fa740 20 client.28720 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.690+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557cbee720a0 0x557cbee74500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.690+0000 7fcfa24fa740 4 client.28720 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.690+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2734663512 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cbedca860 con 0x557cbee720a0 2024-09-19T06:18:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.690+0000 7fcf9debd640 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557cbee720a0 0x557cbee74500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.691+0000 7fcf817fa640 10 client.28720 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:18:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.691+0000 7fcf9debd640 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557cbee720a0 0x557cbee74500 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f27a080289a50a server_cookie=d619f0c5a9b85757 in_seq=0 out_seq=0 2024-09-19T06:18:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcf817fa640 1 -- 172.21.15.5:0/2734663512 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cbedca860 con 0x557cbee720a0 2024-09-19T06:18:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcf817fa640 10 client.28720 handle_command_reply: tid=0 2024-09-19T06:18:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcfa24fa740 1 client.28720 shutdown 2024-09-19T06:18:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2734663512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 msgr2=0x557cbee5ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee5ef80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcf880edd20 tx=0x7fcf880b7fa0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2734663512 shutdown_connections 2024-09-19T06:18:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557cbee720a0 0x557cbee74500 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee62370 0x557cbee5f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee5ef80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcf817fa640 0 client.28720 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:18:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/2734663512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee575d0 0x557cbee61cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2734663512 >> 172.21.15.5:0/2734663512 conn(0x557cbee20310 msgr2=0x557cbee59960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2734663512 shutdown_connections 2024-09-19T06:18:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcfa24fa740 1 -- 172.21.15.5:0/2734663512 wait complete. 2024-09-19T06:18:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.692+0000 7fcfa24fa740 20 client.28720 trim_cache size 0 max 16384 2024-09-19T06:18:31.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.714+0000 7fcfa24fa740 1 Processor -- start 2024-09-19T06:18:31.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.714+0000 7fcfa24fa740 1 -- start start 2024-09-19T06:18:31.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.715+0000 7fcfa24fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee5e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.715+0000 7fcfa24fa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee58b40 0x557cbee5a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.715+0000 7fcfa24fa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee62370 0x557cbee5a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.715+0000 7fcfa24fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b3db0 con 0x557cbee575d0 2024-09-19T06:18:31.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.715+0000 7fcfa24fa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b5aa0 con 0x557cbee58b40 2024-09-19T06:18:31.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.715+0000 7fcfa24fa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b4540 con 0x557cbee62370 2024-09-19T06:18:31.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.715+0000 7fcf9debd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee5e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.715+0000 7fcf9d6bc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee58b40 0x557cbee5a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.715+0000 7fcf9e6be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee62370 0x557cbee5a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.715+0000 7fcf9debd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee5e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60682/0 (socket says 172.21.15.5:60682) 2024-09-19T06:18:31.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.715+0000 7fcf9d6bc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee58b40 0x557cbee5a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50424/0 (socket says 172.21.15.5:50424) 2024-09-19T06:18:31.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.715+0000 7fcf9debd640 1 -- 172.21.15.5:0/1745786425 learned_addr learned my addr 172.21.15.5:0/1745786425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:31.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.716+0000 7fcf9d6bc640 1 -- 172.21.15.5:0/1745786425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee62370 msgr2=0x557cbee5a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:31.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.716+0000 7fcf9d6bc640 1 --2- 172.21.15.5:0/1745786425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee62370 0x557cbee5a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.716+0000 7fcf9d6bc640 1 -- 172.21.15.5:0/1745786425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 msgr2=0x557cbee5e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:31.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.716+0000 7fcf9d6bc640 1 --2- 172.21.15.5:0/1745786425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee5e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.716+0000 7fcf9d6bc640 1 -- 172.21.15.5:0/1745786425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cbee5df30 con 0x557cbee58b40 2024-09-19T06:18:31.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.716+0000 7fcf9e6be640 1 --2- 172.21.15.5:0/1745786425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee62370 0x557cbee5a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:31.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.716+0000 7fcf9d6bc640 1 --2- 172.21.15.5:0/1745786425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee58b40 0x557cbee5a120 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcf880bc060 tx=0x7fcf880ba870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a92075b97fa2aecb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:31.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.716+0000 7fcf837fe640 1 -- 172.21.15.5:0/1745786425 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88049070 con 0x557cbee58b40 2024-09-19T06:18:31.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.716+0000 7fcf837fe640 1 -- 172.21.15.5:0/1745786425 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf880c4f20 con 0x557cbee58b40 2024-09-19T06:18:31.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.716+0000 7fcf837fe640 1 -- 172.21.15.5:0/1745786425 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf880bbcc0 con 0x557cbee58b40 2024-09-19T06:18:31.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.717+0000 7fcfa24fa740 1 -- 172.21.15.5:0/1745786425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee58b40 msgr2=0x557cbee5a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:31.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.717+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/1745786425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee58b40 0x557cbee5a120 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcf880bc060 tx=0x7fcf880ba870 comp rx=0 tx=0).stop 2024-09-19T06:18:31.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.718+0000 7fcfa24fa740 1 -- 172.21.15.5:0/1745786425 shutdown_connections 2024-09-19T06:18:31.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.718+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/1745786425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee62370 0x557cbee5a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.718+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/1745786425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee58b40 0x557cbee5a120 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.718+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/1745786425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee5e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.718+0000 7fcfa24fa740 1 -- 172.21.15.5:0/1745786425 >> 172.21.15.5:0/1745786425 conn(0x557cbee20310 msgr2=0x557cbee52d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:31.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.718+0000 7fcfa24fa740 1 -- 172.21.15.5:0/1745786425 shutdown_connections 2024-09-19T06:18:31.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.718+0000 7fcfa24fa740 1 -- 172.21.15.5:0/1745786425 wait complete. 2024-09-19T06:18:31.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.718+0000 7fcfa24fa740 1 -- start start 2024-09-19T06:18:31.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.718+0000 7fcfa24fa740 1 Processor -- start 2024-09-19T06:18:31.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcfa24fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee6ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcfa24fa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee65f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcfa24fa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee62370 0x557cbee66470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcfa24fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b53d0 con 0x557cbee575d0 2024-09-19T06:18:31.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcfa24fa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b4c40 con 0x557cbee62370 2024-09-19T06:18:31.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcfa24fa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b1db0 con 0x557cbee58b40 2024-09-19T06:18:31.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcfa24fa740 10 client.0 _ll_register_callbacks cb 0x7fcf88009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcf9debd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee6ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcf9debd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee6ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60694/0 (socket says 172.21.15.5:60694) 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcf9debd640 1 -- 172.21.15.5:0/3890725790 learned_addr learned my addr 172.21.15.5:0/3890725790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcf9d6bc640 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee65f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcf9e6be640 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee62370 0x557cbee66470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.719+0000 7fcf9e6be640 1 -- 172.21.15.5:0/3890725790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 msgr2=0x557cbee65f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.720+0000 7fcf9e6be640 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee65f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.720+0000 7fcf9e6be640 1 -- 172.21.15.5:0/3890725790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 msgr2=0x557cbee6ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.720+0000 7fcf9e6be640 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee6ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.720+0000 7fcf9e6be640 1 -- 172.21.15.5:0/3890725790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cbee66c70 con 0x557cbee62370 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.720+0000 7fcf9e6be640 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee62370 0x557cbee66470 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcf90004760 tx=0x7fcf900045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7140f324f93435d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.720+0000 7fcf817fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:18:31.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.720+0000 7fcf817fa640 1 -- 172.21.15.5:0/3890725790 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90004ba0 con 0x557cbee62370 2024-09-19T06:18:31.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.720+0000 7fcf817fa640 1 -- 172.21.15.5:0/3890725790 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fcf90004d40 con 0x557cbee62370 2024-09-19T06:18:31.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.720+0000 7fcfa24fa740 1 -- 172.21.15.5:0/3890725790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf880c4f20 con 0x557cbee62370 2024-09-19T06:18:31.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.720+0000 7fcf817fa640 1 -- 172.21.15.5:0/3890725790 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fcf90005040 con 0x557cbee62370 2024-09-19T06:18:31.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.720+0000 7fcfa24fa740 10 client.28781 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:31.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.720+0000 7fcfa24fa740 1 -- 172.21.15.5:0/3890725790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557cbeedce10 con 0x557cbee62370 2024-09-19T06:18:31.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.721+0000 7fcf817fa640 1 -- 172.21.15.5:0/3890725790 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf880c4f20 con 0x557cbee62370 2024-09-19T06:18:31.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.721+0000 7fcf817fa640 1 -- 172.21.15.5:0/3890725790 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7fcf9002f640 con 0x557cbee62370 2024-09-19T06:18:31.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.721+0000 7fcfa24fa740 10 client.28781 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:31.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.721+0000 7fcfa24fa740 10 client.28781 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:18:31.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.721+0000 7fcfa24fa740 20 client.28781 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:31.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.721+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557cbeedd390 0x557cbeedf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:31.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.721+0000 7fcfa24fa740 4 client.28781 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:18:31.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.721+0000 7fcfa24fa740 1 -- 172.21.15.5:0/3890725790 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cbedc4d70 con 0x557cbeedd390 2024-09-19T06:18:31.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.721+0000 7fcf9debd640 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557cbeedd390 0x557cbeedf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:31.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.722+0000 7fcf9debd640 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557cbeedd390 0x557cbeedf780 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c78e334f735fe0aa server_cookie=ca92ee03be47b63 in_seq=0 out_seq=0 2024-09-19T06:18:31.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.722+0000 7fcf817fa640 10 client.28781 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:18:31.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.722+0000 7fcf817fa640 1 -- 172.21.15.5:0/3890725790 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cbedc4d70 con 0x557cbeedd390 2024-09-19T06:18:31.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.722+0000 7fcf817fa640 10 client.28781 handle_command_reply: tid=0 2024-09-19T06:18:31.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.722+0000 7fcfa24fa740 1 client.28781 shutdown 2024-09-19T06:18:31.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.722+0000 7fcfa24fa740 1 -- 172.21.15.5:0/3890725790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee62370 msgr2=0x557cbee66470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:31.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.722+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee62370 0x557cbee66470 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcf90004760 tx=0x7fcf900045e0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.722+0000 7fcfa24fa740 1 -- 172.21.15.5:0/3890725790 shutdown_connections 2024-09-19T06:18:31.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.722+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x557cbeedd390 0x557cbeedf780 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.723+0000 7fcf817fa640 0 client.28781 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:18:31.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.723+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cbee62370 0x557cbee66470 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.723+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cbee58b40 0x557cbee65f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.723+0000 7fcfa24fa740 1 --2- 172.21.15.5:0/3890725790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cbee575d0 0x557cbee6ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.723+0000 7fcfa24fa740 1 -- 172.21.15.5:0/3890725790 >> 172.21.15.5:0/3890725790 conn(0x557cbee20310 msgr2=0x557cbee607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:31.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.723+0000 7fcfa24fa740 1 -- 172.21.15.5:0/3890725790 shutdown_connections 2024-09-19T06:18:31.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.723+0000 7fcfa24fa740 1 -- 172.21.15.5:0/3890725790 wait complete. 2024-09-19T06:18:31.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.723+0000 7fcfa24fa740 20 client.28781 trim_cache size 0 max 16384 2024-09-19T06:18:31.728 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:18:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.724+0000 7fcfa0121640 1 -- 172.21.15.5:0/2111252380 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcf840a3430 msgr2=0x7fcf840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.724+0000 7fcfa0121640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcf840a3430 0x7fcf840a58f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x557cbee54ac0 tx=0x7fcf9400b410 comp rx=0 tx=0).stop 2024-09-19T06:18:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.724+0000 7fcfa0121640 1 -- 172.21.15.5:0/2111252380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf98102570 msgr2=0x7fcf981a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.724+0000 7fcfa0121640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf98102570 0x7fcf981a4ba0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fcf88006010 tx=0x7fcf88004270 comp rx=0 tx=0).stop 2024-09-19T06:18:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.724+0000 7fcfa0121640 1 -- 172.21.15.5:0/2111252380 shutdown_connections 2024-09-19T06:18:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.724+0000 7fcfa0121640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf98102ef0 0x7fcf9819ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.724+0000 7fcfa0121640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf98102570 0x7fcf981a4ba0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.724+0000 7fcfa0121640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcf840a3430 0x7fcf840a58f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.724+0000 7fcfa0121640 1 --2- 172.21.15.5:0/2111252380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf98100e20 0x7fcf981a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:31.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.724+0000 7fcfa0121640 1 -- 172.21.15.5:0/2111252380 >> 172.21.15.5:0/2111252380 conn(0x7fcf980fc7d0 msgr2=0x7fcf9806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:31.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.724+0000 7fcfa0121640 1 -- 172.21.15.5:0/2111252380 shutdown_connections 2024-09-19T06:18:31.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:31.724+0000 7fcfa0121640 1 -- 172.21.15.5:0/2111252380 wait complete. 2024-09-19T06:18:32.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 19565 10.56 MB/sec warmup 116 sec latency 60.949 ms 2024-09-19T06:18:32.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:32 smithi005 ceph-mon[30955]: pgmap v477: 97 pgs: 97 active+clean; 520 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-09-19T06:18:32.773 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:18:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:32 smithi157 ceph-mon[35346]: pgmap v477: 97 pgs: 97 active+clean; 520 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-09-19T06:18:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:32 smithi049 ceph-mon[34640]: pgmap v477: 97 pgs: 97 active+clean; 520 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-09-19T06:18:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:18:33.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 19755 10.58 MB/sec warmup 117 sec latency 77.078 ms 2024-09-19T06:18:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.694+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/662813294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb04100480 msgr2=0x7fbb04100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.694+0000 7fbb0c7a4640 1 --2- 172.21.15.5:0/662813294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb04100480 0x7fbb04100860 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000aa10 tx=0x7fbb0002b720 comp rx=0 tx=0).stop 2024-09-19T06:18:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.694+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/662813294 shutdown_connections 2024-09-19T06:18:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.694+0000 7fbb0c7a4640 1 --2- 172.21.15.5:0/662813294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb041055b0 0x7fbb0410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.694+0000 7fbb0c7a4640 1 --2- 172.21.15.5:0/662813294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb04100e30 0x7fbb04104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.694+0000 7fbb0c7a4640 1 --2- 172.21.15.5:0/662813294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb04100480 0x7fbb04100860 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.694+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/662813294 >> 172.21.15.5:0/662813294 conn(0x7fbb04077210 msgr2=0x7fbb040fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.694+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/662813294 shutdown_connections 2024-09-19T06:18:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.695+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/662813294 wait complete. 2024-09-19T06:18:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.695+0000 7fbb0c7a4640 1 Processor -- start 2024-09-19T06:18:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.695+0000 7fbb0c7a4640 1 -- start start 2024-09-19T06:18:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.696+0000 7fbb0c7a4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb04100480 0x7fbb040fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.696+0000 7fbb0c7a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb04100e30 0x7fbb040fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.696+0000 7fbb0c7a4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb041055b0 0x7fbb040fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.696+0000 7fbb0a540640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb04100480 0x7fbb040fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.696+0000 7fbb09d3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb04100e30 0x7fbb040fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.696+0000 7fbb09d3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb04100e30 0x7fbb040fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60722/0 (socket says 172.21.15.5:60722) 2024-09-19T06:18:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.696+0000 7fbb0a540640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb04100480 0x7fbb040fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50454/0 (socket says 172.21.15.5:50454) 2024-09-19T06:18:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.696+0000 7fbb09d3f640 1 -- 172.21.15.5:0/3843984295 learned_addr learned my addr 172.21.15.5:0/3843984295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.696+0000 7fbb0c7a4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0410fc90 con 0x7fbb04100e30 2024-09-19T06:18:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.696+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/3843984295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0410fb10 con 0x7fbb04100480 2024-09-19T06:18:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.696+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/3843984295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0410fe10 con 0x7fbb041055b0 2024-09-19T06:18:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.696+0000 7fbb0ad41640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb041055b0 0x7fbb040fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.697+0000 7fbb0ad41640 1 -- 172.21.15.5:0/3843984295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb04100480 msgr2=0x7fbb040fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.697+0000 7fbb0ad41640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb04100480 0x7fbb040fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.697+0000 7fbb0ad41640 1 -- 172.21.15.5:0/3843984295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb04100e30 msgr2=0x7fbb040fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.697+0000 7fbb0ad41640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb04100e30 0x7fbb040fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.697+0000 7fbb0ad41640 1 -- 172.21.15.5:0/3843984295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb041a6840 con 0x7fbb041055b0 2024-09-19T06:18:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.697+0000 7fbb09d3f640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb04100e30 0x7fbb040fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.697+0000 7fbb0ad41640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb041055b0 0x7fbb040fd110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009fd0 tx=0x7fbafc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.697+0000 7fbafb7fe640 1 -- 172.21.15.5:0/3843984295 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc00ce20 con 0x7fbb041055b0 2024-09-19T06:18:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.697+0000 7fbafb7fe640 1 -- 172.21.15.5:0/3843984295 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbafc005020 con 0x7fbb041055b0 2024-09-19T06:18:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.697+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/3843984295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb041a6b30 con 0x7fbb041055b0 2024-09-19T06:18:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.697+0000 7fbafb7fe640 1 -- 172.21.15.5:0/3843984295 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc010690 con 0x7fbb041055b0 2024-09-19T06:18:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.697+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/3843984295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb041a70c0 con 0x7fbb041055b0 2024-09-19T06:18:33.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.701+0000 7fbafb7fe640 1 -- 172.21.15.5:0/3843984295 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbafc0051d0 con 0x7fbb041055b0 2024-09-19T06:18:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.701+0000 7fbb0eb7d740 1 Processor -- start 2024-09-19T06:18:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.701+0000 7fbb0eb7d740 1 -- start start 2024-09-19T06:18:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.701+0000 7fbb0eb7d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bcceeab40 0x564bccee9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.701+0000 7fbb0eb7d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccee9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.701+0000 7fbb0eb7d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bccef4370 0x564bccef6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.701+0000 7fbb0eb7d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564bcce4f2f0 con 0x564bccef4370 2024-09-19T06:18:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.701+0000 7fbb0eb7d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564bccdf6cf0 con 0x564bccee95d0 2024-09-19T06:18:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.701+0000 7fbb0eb7d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564bcce4b2b0 con 0x564bcceeab40 2024-09-19T06:18:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbb09d3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bcceeab40 0x564bccee9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbb09d3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bcceeab40 0x564bccee9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42462/0 (socket says 172.21.15.5:42462) 2024-09-19T06:18:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbb09d3f640 1 -- 172.21.15.5:0/2804739187 learned_addr learned my addr 172.21.15.5:0/2804739187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbb09d3f640 1 --2- 172.21.15.5:0/2804739187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bccef4370 0x564bccef6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbb0a540640 1 --2- 172.21.15.5:0/2804739187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccee9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbafb7fe640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbaf00a3430 0x7fbaf00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbb0a540640 1 -- 172.21.15.5:0/2804739187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bcceeab40 msgr2=0x564bccee9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbb0a540640 1 --2- 172.21.15.5:0/2804739187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bcceeab40 0x564bccee9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbb0a540640 1 -- 172.21.15.5:0/2804739187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bccef4370 msgr2=0x564bccef6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbb0ad41640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbaf00a3430 0x7fbaf00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbb0a540640 1 --2- 172.21.15.5:0/2804739187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bccef4370 0x564bccef6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbb0a540640 1 -- 172.21.15.5:0/2804739187 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bccddb3a0 con 0x564bccee95d0 2024-09-19T06:18:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.703+0000 7fbb0a540640 1 --2- 172.21.15.5:0/2804739187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccee9a10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbb0002bd10 tx=0x7fbb00005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffa433776bd0df67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.702+0000 7fbafb7fe640 1 -- 172.21.15.5:0/3843984295 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fbafc0d0330 con 0x7fbb041055b0 2024-09-19T06:18:33.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.703+0000 7fbaf8ff9640 1 -- 172.21.15.5:0/2804739187 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00041070 con 0x564bccee95d0 2024-09-19T06:18:33.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.703+0000 7fbaf8ff9640 1 -- 172.21.15.5:0/2804739187 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb000054f0 con 0x564bccee95d0 2024-09-19T06:18:33.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.703+0000 7fbb0ad41640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbaf00a3430 0x7fbaf00a58f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0a4cd0 tx=0x7fbafc0981f0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:33.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.703+0000 7fbaf8ff9640 1 -- 172.21.15.5:0/2804739187 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00036040 con 0x564bccee95d0 2024-09-19T06:18:33.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.704+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/2804739187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 msgr2=0x564bccee9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:33.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.704+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/2804739187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccee9a10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbb0002bd10 tx=0x7fbb00005c00 comp rx=0 tx=0).stop 2024-09-19T06:18:33.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.704+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/2804739187 shutdown_connections 2024-09-19T06:18:33.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.704+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/2804739187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bccef4370 0x564bccef6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.704+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/2804739187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccee9a10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.704+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/2804739187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bcceeab40 0x564bccee9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.704+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/2804739187 >> 172.21.15.5:0/2804739187 conn(0x564bcceb2310 msgr2=0x564bccee5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:33.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.704+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/2804739187 shutdown_connections 2024-09-19T06:18:33.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.704+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/2804739187 wait complete. 2024-09-19T06:18:33.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb0eb7d740 1 -- start start 2024-09-19T06:18:33.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb0eb7d740 1 Processor -- start 2024-09-19T06:18:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb0eb7d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccef3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb0eb7d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 0x564bccef0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb0eb7d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 0x564bccef14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb09d3f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccef3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb0eb7d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564bccef7060 con 0x564bcceeab40 2024-09-19T06:18:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb0eb7d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564bcce0a060 con 0x564bccee95d0 2024-09-19T06:18:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb0eb7d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564bccef71e0 con 0x564bccef4370 2024-09-19T06:18:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb0a540640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 0x564bccef0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb09d3f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccef3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50474/0 (socket says 172.21.15.5:50474) 2024-09-19T06:18:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb09d3f640 1 -- 172.21.15.5:0/4244470074 learned_addr learned my addr 172.21.15.5:0/4244470074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.705+0000 7fbb0eb7d740 10 client.0 _ll_register_callbacks cb 0x564bccb5d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.706+0000 7fbb09d3f640 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 0x564bccef14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.706+0000 7fbb0a540640 1 -- 172.21.15.5:0/4244470074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 msgr2=0x564bccef14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.706+0000 7fbb0a540640 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 0x564bccef14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.706+0000 7fbb0a540640 1 -- 172.21.15.5:0/4244470074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 msgr2=0x564bccef3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.706+0000 7fbb0a540640 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccef3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.706+0000 7fbb0a540640 1 -- 172.21.15.5:0/4244470074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bccbc03c0 con 0x564bcceeab40 2024-09-19T06:18:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.706+0000 7fbb09d3f640 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccef3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.706+0000 7fbb0a540640 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 0x564bccef0f90 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005ab0 tx=0x7fbb00000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2508478ff10408e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.706+0000 7fbaedffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:18:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.706+0000 7fbaedffb640 1 -- 172.21.15.5:0/4244470074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00036030 con 0x564bcceeab40 2024-09-19T06:18:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.706+0000 7fbaedffb640 1 -- 172.21.15.5:0/4244470074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fbb000050c0 con 0x564bcceeab40 2024-09-19T06:18:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.706+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4244470074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb000054f0 con 0x564bcceeab40 2024-09-19T06:18:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.707+0000 7fbaedffb640 1 -- 172.21.15.5:0/4244470074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbb00041040 con 0x564bcceeab40 2024-09-19T06:18:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.707+0000 7fbb0eb7d740 10 client.19407 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.707+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4244470074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564bccf02a30 con 0x564bcceeab40 2024-09-19T06:18:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.707+0000 7fbaedffb640 1 -- 172.21.15.5:0/4244470074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb000054f0 con 0x564bcceeab40 2024-09-19T06:18:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.707+0000 7fbaedffb640 1 -- 172.21.15.5:0/4244470074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7fbb0003b040 con 0x564bcceeab40 2024-09-19T06:18:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.707+0000 7fbb0eb7d740 10 client.19407 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.707+0000 7fbb0eb7d740 10 client.19407 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:18:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.707+0000 7fbb0eb7d740 20 client.19407 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.707+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564bccf03d80 0x564bccf06170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.708+0000 7fbb0eb7d740 4 client.19407 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.708+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4244470074 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bcce5c860 con 0x564bccf03d80 2024-09-19T06:18:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.708+0000 7fbb09d3f640 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564bccf03d80 0x564bccf06170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.708+0000 7fbb09d3f640 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564bccf03d80 0x564bccf06170 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2eaee146ff56c server_cookie=bb3ee5a24fb355bb in_seq=0 out_seq=0 2024-09-19T06:18:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.708+0000 7fbaedffb640 10 client.19407 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:18:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.709+0000 7fbaedffb640 1 -- 172.21.15.5:0/4244470074 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bcce5c860 con 0x564bccf03d80 2024-09-19T06:18:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.709+0000 7fbaedffb640 10 client.19407 handle_command_reply: tid=0 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.709+0000 7fbb0eb7d740 1 client.19407 shutdown 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.709+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4244470074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 msgr2=0x564bccef0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.709+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 0x564bccef0f90 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005ab0 tx=0x7fbb00000bc0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.710+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4244470074 shutdown_connections 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.710+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564bccf03d80 0x564bccf06170 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.710+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 0x564bccef14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.710+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 0x564bccef0f90 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.710+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/4244470074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccef3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.710+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4244470074 >> 172.21.15.5:0/4244470074 conn(0x564bcceb2310 msgr2=0x564bcceeb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.710+0000 7fbaedffb640 0 client.19407 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.710+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4244470074 shutdown_connections 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.710+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4244470074 wait complete. 2024-09-19T06:18:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.710+0000 7fbb0eb7d740 20 client.19407 trim_cache size 0 max 16384 2024-09-19T06:18:33.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.729+0000 7fbb0eb7d740 1 Processor -- start 2024-09-19T06:18:33.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0eb7d740 1 -- start start 2024-09-19T06:18:33.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0eb7d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccf00300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0eb7d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 0x564bccef0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0eb7d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 0x564bccef5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0eb7d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041af9e0 con 0x564bcceeab40 2024-09-19T06:18:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0eb7d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041b16d0 con 0x564bccee95d0 2024-09-19T06:18:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0eb7d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041b0170 con 0x564bccef4370 2024-09-19T06:18:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0a540640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 0x564bccef0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0a540640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 0x564bccef0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60750/0 (socket says 172.21.15.5:60750) 2024-09-19T06:18:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0a540640 1 -- 172.21.15.5:0/2599311230 learned_addr learned my addr 172.21.15.5:0/2599311230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:33.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb09d3f640 1 --2- 172.21.15.5:0/2599311230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccf00300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb09d3f640 1 --2- 172.21.15.5:0/2599311230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 0x564bccef5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0a540640 1 -- 172.21.15.5:0/2599311230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 msgr2=0x564bccef5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0a540640 1 --2- 172.21.15.5:0/2599311230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 0x564bccef5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0a540640 1 -- 172.21.15.5:0/2599311230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 msgr2=0x564bccf00300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0a540640 1 --2- 172.21.15.5:0/2599311230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccf00300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.730+0000 7fbb0a540640 1 -- 172.21.15.5:0/2599311230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bccef71d0 con 0x564bcceeab40 2024-09-19T06:18:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.731+0000 7fbb0a540640 1 --2- 172.21.15.5:0/2599311230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 0x564bccef0850 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fbb00030550 tx=0x7fbb0002b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb2e9e94d6e438 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.731+0000 7fbaeffff640 1 -- 172.21.15.5:0/2599311230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00036040 con 0x564bcceeab40 2024-09-19T06:18:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.731+0000 7fbaeffff640 1 -- 172.21.15.5:0/2599311230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb00005850 con 0x564bcceeab40 2024-09-19T06:18:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.731+0000 7fbaeffff640 1 -- 172.21.15.5:0/2599311230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00004920 con 0x564bcceeab40 2024-09-19T06:18:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.732+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/2599311230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 msgr2=0x564bccef0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.732+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/2599311230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 0x564bccef0850 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fbb00030550 tx=0x7fbb0002b720 comp rx=0 tx=0).stop 2024-09-19T06:18:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.732+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/2599311230 shutdown_connections 2024-09-19T06:18:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.732+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/2599311230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 0x564bccef5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.732+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/2599311230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bcceeab40 0x564bccef0850 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.732+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/2599311230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bccee95d0 0x564bccf00300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.732+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/2599311230 >> 172.21.15.5:0/2599311230 conn(0x564bcceb2310 msgr2=0x564bccee0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.732+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/2599311230 shutdown_connections 2024-09-19T06:18:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.733+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/2599311230 wait complete. 2024-09-19T06:18:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.733+0000 7fbb0eb7d740 1 -- start start 2024-09-19T06:18:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.733+0000 7fbb0eb7d740 1 Processor -- start 2024-09-19T06:18:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.733+0000 7fbb0eb7d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bccee95d0 0x564bcceed300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb0eb7d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bcceeab40 0x564bcceed840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb0eb7d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 0x564bcceedd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb09d3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bccee95d0 0x564bcceed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb09d3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bccee95d0 0x564bcceed300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60762/0 (socket says 172.21.15.5:60762) 2024-09-19T06:18:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb09d3f640 1 -- 172.21.15.5:0/4070955084 learned_addr learned my addr 172.21.15.5:0/4070955084 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4070955084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041b1000 con 0x564bccee95d0 2024-09-19T06:18:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4070955084 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041b0870 con 0x564bcceeab40 2024-09-19T06:18:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4070955084 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041ad9e0 con 0x564bccef4370 2024-09-19T06:18:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb0a540640 1 --2- 172.21.15.5:0/4070955084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bcceeab40 0x564bcceed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb0eb7d740 10 client.0 _ll_register_callbacks cb 0x7fbaf40117f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb09d3f640 1 -- 172.21.15.5:0/4070955084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 msgr2=0x564bcceedd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb09d3f640 1 --2- 172.21.15.5:0/4070955084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 0x564bcceedd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb09d3f640 1 -- 172.21.15.5:0/4070955084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bcceeab40 msgr2=0x564bcceed840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb09d3f640 1 --2- 172.21.15.5:0/4070955084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bcceeab40 0x564bcceed840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb09d3f640 1 -- 172.21.15.5:0/4070955084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bccefb6e0 con 0x564bccee95d0 2024-09-19T06:18:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.734+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4070955084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb00005850 con 0x564bccee95d0 2024-09-19T06:18:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.735+0000 7fbb09d3f640 1 --2- 172.21.15.5:0/4070955084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bccee95d0 0x564bcceed300 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4011e40 tx=0x7fbaf4011560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bc141544a6e4de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.735+0000 7fbaedffb640 10 client.19413 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.735+0000 7fbaedffb640 1 -- 172.21.15.5:0/4070955084 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf402e030 con 0x564bccee95d0 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.735+0000 7fbaedffb640 1 -- 172.21.15.5:0/4070955084 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fbaf4009040 con 0x564bccee95d0 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.735+0000 7fbaedffb640 1 -- 172.21.15.5:0/4070955084 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbaf4023190 con 0x564bccee95d0 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.735+0000 7fbb0eb7d740 10 client.19413 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.735+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4070955084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=47}) -- 0x564bccefc900 con 0x564bccee95d0 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.736+0000 7fbaedffb640 1 -- 172.21.15.5:0/4070955084 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7fbaf40239a0 con 0x564bccee95d0 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.736+0000 7fbb0eb7d740 10 client.19413 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.736+0000 7fbb0eb7d740 10 client.19413 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.736+0000 7fbb0eb7d740 20 client.19413 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.736+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/4070955084 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564bccf6e7a0 0x564bccf70b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.736+0000 7fbb0eb7d740 4 client.19413 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:18:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.736+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4070955084 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bcce56d70 con 0x564bccf6e7a0 2024-09-19T06:18:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.736+0000 7fbb0a540640 1 --2- 172.21.15.5:0/4070955084 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564bccf6e7a0 0x564bccf70b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.736+0000 7fbb0a540640 1 --2- 172.21.15.5:0/4070955084 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564bccf6e7a0 0x564bccf70b90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9bfdf5810c0c65d7 server_cookie=221de525a8e713a2 in_seq=0 out_seq=0 2024-09-19T06:18:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.736+0000 7fbaedffb640 10 client.19413 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:18:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.737+0000 7fbaedffb640 1 -- 172.21.15.5:0/4070955084 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bcce56d70 con 0x564bccf6e7a0 2024-09-19T06:18:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.737+0000 7fbaedffb640 10 client.19413 handle_command_reply: tid=0 2024-09-19T06:18:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.737+0000 7fbb0eb7d740 1 client.19413 shutdown 2024-09-19T06:18:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.737+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4070955084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bccee95d0 msgr2=0x564bcceed300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.737+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/4070955084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bccee95d0 0x564bcceed300 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4011e40 tx=0x7fbaf4011560 comp rx=0 tx=0).stop 2024-09-19T06:18:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.737+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4070955084 shutdown_connections 2024-09-19T06:18:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.737+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/4070955084 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x564bccf6e7a0 0x564bccf70b90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.737+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/4070955084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bccef4370 0x564bcceedd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.737+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/4070955084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bcceeab40 0x564bcceed840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.737+0000 7fbb0eb7d740 1 --2- 172.21.15.5:0/4070955084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bccee95d0 0x564bcceed300 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.737+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4070955084 >> 172.21.15.5:0/4070955084 conn(0x564bcceb2310 msgr2=0x564bccef07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.737+0000 7fbaedffb640 0 client.19413 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:18:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.738+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4070955084 shutdown_connections 2024-09-19T06:18:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.738+0000 7fbb0eb7d740 1 -- 172.21.15.5:0/4070955084 wait complete. 2024-09-19T06:18:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.738+0000 7fbb0eb7d740 20 client.19413 trim_cache size 0 max 16384 2024-09-19T06:18:33.744 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:18:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.739+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/3843984295 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbaf00a3430 msgr2=0x7fbaf00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.739+0000 7fbb0c7a4640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbaf00a3430 0x7fbaf00a58f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0a4cd0 tx=0x7fbafc0981f0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.739+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/3843984295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb041055b0 msgr2=0x7fbb040fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.739+0000 7fbb0c7a4640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb041055b0 0x7fbb040fd110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009fd0 tx=0x7fbafc00ef10 comp rx=0 tx=0).stop 2024-09-19T06:18:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.739+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/3843984295 shutdown_connections 2024-09-19T06:18:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.739+0000 7fbb0c7a4640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbaf00a3430 0x7fbaf00a58f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.739+0000 7fbb0c7a4640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb041055b0 0x7fbb040fd110 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.739+0000 7fbb0c7a4640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb04100e30 0x7fbb040fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.739+0000 7fbb0c7a4640 1 --2- 172.21.15.5:0/3843984295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb04100480 0x7fbb040fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.739+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/3843984295 >> 172.21.15.5:0/3843984295 conn(0x7fbb04077210 msgr2=0x7fbb040fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.739+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/3843984295 shutdown_connections 2024-09-19T06:18:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:33.739+0000 7fbb0c7a4640 1 -- 172.21.15.5:0/3843984295 wait complete. 2024-09-19T06:18:34.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 19885 10.58 MB/sec warmup 118 sec latency 59.156 ms 2024-09-19T06:18:34.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:34 smithi005 ceph-mon[30955]: pgmap v478: 97 pgs: 97 active+clean; 529 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 373 op/s 2024-09-19T06:18:34.561 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:18:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:34 smithi157 ceph-mon[35346]: pgmap v478: 97 pgs: 97 active+clean; 529 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 373 op/s 2024-09-19T06:18:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:34 smithi049 ceph-mon[34640]: pgmap v478: 97 pgs: 97 active+clean; 529 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 373 op/s 2024-09-19T06:18:34.921 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:18:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.482+0000 7fca68625640 1 -- 172.21.15.5:0/3776603999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca60101de0 msgr2=0x7fca601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.482+0000 7fca68625640 1 --2- 172.21.15.5:0/3776603999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca60101de0 0x7fca601021c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009900 tx=0x7fca5c02a410 comp rx=0 tx=0).stop 2024-09-19T06:18:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.483+0000 7fca68625640 1 -- 172.21.15.5:0/3776603999 shutdown_connections 2024-09-19T06:18:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.483+0000 7fca68625640 1 --2- 172.21.15.5:0/3776603999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca60106f10 0x7fca6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.483+0000 7fca68625640 1 --2- 172.21.15.5:0/3776603999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60102790 0x7fca601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.483+0000 7fca68625640 1 --2- 172.21.15.5:0/3776603999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca60101de0 0x7fca601021c0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.483+0000 7fca68625640 1 -- 172.21.15.5:0/3776603999 >> 172.21.15.5:0/3776603999 conn(0x7fca600fd8e0 msgr2=0x7fca600ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.483+0000 7fca68625640 1 -- 172.21.15.5:0/3776603999 shutdown_connections 2024-09-19T06:18:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.483+0000 7fca68625640 1 -- 172.21.15.5:0/3776603999 wait complete. 2024-09-19T06:18:35.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.484+0000 7fca68625640 1 Processor -- start 2024-09-19T06:18:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.484+0000 7fca68625640 1 -- start start 2024-09-19T06:18:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.484+0000 7fca68625640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca60101de0 0x7fca600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.484+0000 7fca68625640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca60102790 0x7fca60074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.484+0000 7fca68625640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60106f10 0x7fca60074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.484+0000 7fca68625640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca60111e20 con 0x7fca60101de0 2024-09-19T06:18:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.484+0000 7fca68625640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fca60111ca0 con 0x7fca60102790 2024-09-19T06:18:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.484+0000 7fca68625640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca60111fa0 con 0x7fca60106f10 2024-09-19T06:18:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.484+0000 7fca65bc0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca60102790 0x7fca60074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.484+0000 7fca65bc0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca60102790 0x7fca60074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54200/0 (socket says 172.21.15.5:54200) 2024-09-19T06:18:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.484+0000 7fca65bc0640 1 -- 172.21.15.5:0/2578333257 learned_addr learned my addr 172.21.15.5:0/2578333257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.485+0000 7fca663c1640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca60101de0 0x7fca600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.485+0000 7fca66bc2640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60106f10 0x7fca60074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.485+0000 7fca65bc0640 1 -- 172.21.15.5:0/2578333257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60106f10 msgr2=0x7fca60074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.485+0000 7fca65bc0640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60106f10 0x7fca60074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.485+0000 7fca65bc0640 1 -- 172.21.15.5:0/2578333257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca60101de0 msgr2=0x7fca600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.485+0000 7fca65bc0640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca60101de0 0x7fca600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.485+0000 7fca65bc0640 1 -- 172.21.15.5:0/2578333257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca601aadb0 con 0x7fca60102790 2024-09-19T06:18:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.485+0000 7fca66bc2640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60106f10 0x7fca60074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:18:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.485+0000 7fca663c1640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca60101de0 0x7fca600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:18:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.485+0000 7fca65bc0640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca60102790 0x7fca60074a30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fca5000e9f0 tx=0x7fca5000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.486+0000 7fca577fe640 1 -- 172.21.15.5:0/2578333257 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5000cd80 con 0x7fca60102790 2024-09-19T06:18:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.486+0000 7fca577fe640 1 -- 172.21.15.5:0/2578333257 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca500054d0 con 0x7fca60102790 2024-09-19T06:18:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.486+0000 7fca68625640 1 -- 172.21.15.5:0/2578333257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca601aafe0 con 0x7fca60102790 2024-09-19T06:18:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.486+0000 7fca577fe640 1 -- 172.21.15.5:0/2578333257 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50010430 con 0x7fca60102790 2024-09-19T06:18:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.486+0000 7fca68625640 1 -- 172.21.15.5:0/2578333257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca601ab430 con 0x7fca60102790 2024-09-19T06:18:35.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.489+0000 7fca577fe640 1 -- 172.21.15.5:0/2578333257 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fca50005020 con 0x7fca60102790 2024-09-19T06:18:35.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.489+0000 7fca6a9fe740 1 Processor -- start 2024-09-19T06:18:35.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.490+0000 7fca6a9fe740 1 -- start start 2024-09-19T06:18:35.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.491+0000 7fca6a9fe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02fb40 0x56096f02e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.491+0000 7fca577fe640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fca4c0a3430 0x7fca4c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.491+0000 7fca66bc2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02fb40 0x56096f02e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.491+0000 7fca66bc2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02fb40 0x56096f02e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47270/0 (socket says 172.21.15.5:47270) 2024-09-19T06:18:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.491+0000 7fca66bc2640 1 -- 172.21.15.5:0/2774524585 learned_addr learned my addr 172.21.15.5:0/2774524585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.491+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/2774524585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02e5d0 0x56096f02ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.491+0000 7fca577fe640 1 -- 172.21.15.5:0/2578333257 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fca500d01e0 con 0x7fca60102790 2024-09-19T06:18:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.491+0000 7fca663c1640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fca4c0a3430 0x7fca4c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.491+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/2774524585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 0x56096f03b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.491+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2774524585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56096ef942f0 con 0x56096f02fb40 2024-09-19T06:18:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.491+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2774524585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56096ef3bcf0 con 0x56096f02e5d0 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.491+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2774524585 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56096ef902b0 con 0x56096f039370 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.492+0000 7fca65bc0640 1 --2- 172.21.15.5:0/2774524585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02e5d0 0x56096f02ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.492+0000 7fca66bc2640 1 --2- 172.21.15.5:0/2774524585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 0x56096f03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.492+0000 7fca66bc2640 1 -- 172.21.15.5:0/2774524585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 msgr2=0x56096f03b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.492+0000 7fca66bc2640 1 --2- 172.21.15.5:0/2774524585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 0x56096f03b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.492+0000 7fca66bc2640 1 -- 172.21.15.5:0/2774524585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02e5d0 msgr2=0x56096f02ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.492+0000 7fca66bc2640 1 --2- 172.21.15.5:0/2774524585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02e5d0 0x56096f02ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.492+0000 7fca66bc2640 1 -- 172.21.15.5:0/2774524585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56096ef203a0 con 0x56096f02fb40 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.492+0000 7fca65bc0640 1 --2- 172.21.15.5:0/2774524585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02e5d0 0x56096f02ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.492+0000 7fca66bc2640 1 --2- 172.21.15.5:0/2774524585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02fb40 0x56096f02e090 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fca58005f40 tx=0x7fca58009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8340d2b09c8f8ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.492+0000 7fca54ff9640 1 -- 172.21.15.5:0/2774524585 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5801a020 con 0x56096f02fb40 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.492+0000 7fca54ff9640 1 -- 172.21.15.5:0/2774524585 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca58003e20 con 0x56096f02fb40 2024-09-19T06:18:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.492+0000 7fca54ff9640 1 -- 172.21.15.5:0/2774524585 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58010ea0 con 0x56096f02fb40 2024-09-19T06:18:35.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.493+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2774524585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02fb40 msgr2=0x56096f02e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:35.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.493+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/2774524585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02fb40 0x56096f02e090 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fca58005f40 tx=0x7fca58009c80 comp rx=0 tx=0).stop 2024-09-19T06:18:35.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.493+0000 7fca663c1640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fca4c0a3430 0x7fca4c0a58f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fca5c031440 tx=0x7fca5c005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.493+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2774524585 shutdown_connections 2024-09-19T06:18:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.493+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/2774524585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 0x56096f03b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.493+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/2774524585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02e5d0 0x56096f02ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.493+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/2774524585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02fb40 0x56096f02e090 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.493+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2774524585 >> 172.21.15.5:0/2774524585 conn(0x56096eff7310 msgr2=0x56096f02ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.494+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2774524585 shutdown_connections 2024-09-19T06:18:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.494+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2774524585 wait complete. 2024-09-19T06:18:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.494+0000 7fca6a9fe740 1 -- start start 2024-09-19T06:18:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.494+0000 7fca6a9fe740 1 Processor -- start 2024-09-19T06:18:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca6a9fe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f038d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca6a9fe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f02fb40 0x56096f035fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca6a9fe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f039370 0x56096f0365d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca6a9fe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56096f03c070 con 0x56096f02e5d0 2024-09-19T06:18:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca6a9fe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56096ef4f060 con 0x56096f039370 2024-09-19T06:18:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca6a9fe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56096f03c1f0 con 0x56096f02fb40 2024-09-19T06:18:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca6a9fe740 10 client.0 _ll_register_callbacks cb 0x56096eca28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca65bc0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f02fb40 0x56096f035fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca663c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca65bc0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f02fb40 0x56096f035fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42306/0 (socket says 172.21.15.5:42306) 2024-09-19T06:18:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca65bc0640 1 -- 172.21.15.5:0/3024712141 learned_addr learned my addr 172.21.15.5:0/3024712141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca66bc2640 1 --2- 172.21.15.5:0/3024712141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f039370 0x56096f0365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca65bc0640 1 -- 172.21.15.5:0/3024712141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f039370 msgr2=0x56096f0365d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca65bc0640 1 --2- 172.21.15.5:0/3024712141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f039370 0x56096f0365d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca65bc0640 1 -- 172.21.15.5:0/3024712141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 msgr2=0x56096f038d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca65bc0640 1 --2- 172.21.15.5:0/3024712141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f038d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.495+0000 7fca65bc0640 1 -- 172.21.15.5:0/3024712141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56096ed053c0 con 0x56096f02fb40 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.496+0000 7fca65bc0640 1 --2- 172.21.15.5:0/3024712141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f02fb40 0x56096f035fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fca5000c6a0 tx=0x7fca5009bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b96a0734da63c4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.496+0000 7fca49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.496+0000 7fca49ffb640 1 -- 172.21.15.5:0/3024712141 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca500b4070 con 0x56096f02fb40 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.496+0000 7fca49ffb640 1 -- 172.21.15.5:0/3024712141 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fca500b5270 con 0x56096f02fb40 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.496+0000 7fca6a9fe740 1 -- 172.21.15.5:0/3024712141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca58003e20 con 0x56096f02fb40 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.496+0000 7fca49ffb640 1 -- 172.21.15.5:0/3024712141 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fca500b5470 con 0x56096f02fb40 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.496+0000 7fca6a9fe740 10 client.28750 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.496+0000 7fca6a9fe740 1 -- 172.21.15.5:0/3024712141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56096f047b30 con 0x56096f02fb40 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.497+0000 7fca49ffb640 1 -- 172.21.15.5:0/3024712141 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca58003e20 con 0x56096f02fb40 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.497+0000 7fca49ffb640 1 -- 172.21.15.5:0/3024712141 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7fca50081020 con 0x56096f02fb40 2024-09-19T06:18:35.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.497+0000 7fca6a9fe740 10 client.28750 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.497+0000 7fca6a9fe740 10 client.28750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.497+0000 7fca6a9fe740 20 client.28750 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.497+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/3024712141 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56096f0490c0 0x56096f04b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.497+0000 7fca6a9fe740 4 client.28750 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.497+0000 7fca6a9fe740 1 -- 172.21.15.5:0/3024712141 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56096efa1860 con 0x56096f0490c0 2024-09-19T06:18:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.497+0000 7fca663c1640 1 --2- 172.21.15.5:0/3024712141 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56096f0490c0 0x56096f04b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.497+0000 7fca49ffb640 10 client.28750 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.498+0000 7fca663c1640 1 --2- 172.21.15.5:0/3024712141 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56096f0490c0 0x56096f04b500 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f8bb0f8b9f10566 server_cookie=ed8f07e9dc68cd74 in_seq=0 out_seq=0 2024-09-19T06:18:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.498+0000 7fca49ffb640 1 -- 172.21.15.5:0/3024712141 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56096efa1860 con 0x56096f0490c0 2024-09-19T06:18:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.498+0000 7fca49ffb640 10 client.28750 handle_command_reply: tid=0 2024-09-19T06:18:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.498+0000 7fca6a9fe740 1 client.28750 shutdown 2024-09-19T06:18:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.499+0000 7fca6a9fe740 1 -- 172.21.15.5:0/3024712141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f02fb40 msgr2=0x56096f035fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.499+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/3024712141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f02fb40 0x56096f035fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fca5000c6a0 tx=0x7fca5009bec0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.499+0000 7fca6a9fe740 1 -- 172.21.15.5:0/3024712141 shutdown_connections 2024-09-19T06:18:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.499+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/3024712141 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56096f0490c0 0x56096f04b500 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.499+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/3024712141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f039370 0x56096f0365d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.499+0000 7fca49ffb640 0 client.28750 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.499+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/3024712141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f02fb40 0x56096f035fe0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.499+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/3024712141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f038d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.499+0000 7fca6a9fe740 1 -- 172.21.15.5:0/3024712141 >> 172.21.15.5:0/3024712141 conn(0x56096eff7310 msgr2=0x56096f030960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.499+0000 7fca6a9fe740 1 -- 172.21.15.5:0/3024712141 shutdown_connections 2024-09-19T06:18:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.499+0000 7fca6a9fe740 1 -- 172.21.15.5:0/3024712141 wait complete. 2024-09-19T06:18:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.499+0000 7fca6a9fe740 20 client.28750 trim_cache size 0 max 16384 2024-09-19T06:18:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.525+0000 7fca6a9fe740 1 Processor -- start 2024-09-19T06:18:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.525+0000 7fca6a9fe740 1 -- start start 2024-09-19T06:18:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.525+0000 7fca6a9fe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f045310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.525+0000 7fca6a9fe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 0x56096f0348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.525+0000 7fca6a9fe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 0x56096f034df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.525+0000 7fca6a9fe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca601b3e50 con 0x56096f02e5d0 2024-09-19T06:18:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.525+0000 7fca6a9fe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fca601b5b40 con 0x56096f02fb40 2024-09-19T06:18:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.525+0000 7fca6a9fe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca601b45e0 con 0x56096f039370 2024-09-19T06:18:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.526+0000 7fca65bc0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 0x56096f0348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.526+0000 7fca663c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f045310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.526+0000 7fca65bc0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 0x56096f0348b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54230/0 (socket says 172.21.15.5:54230) 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.526+0000 7fca65bc0640 1 -- 172.21.15.5:0/1102633939 learned_addr learned my addr 172.21.15.5:0/1102633939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.526+0000 7fca66bc2640 1 --2- 172.21.15.5:0/1102633939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 0x56096f034df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.526+0000 7fca65bc0640 1 -- 172.21.15.5:0/1102633939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 msgr2=0x56096f034df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.526+0000 7fca65bc0640 1 --2- 172.21.15.5:0/1102633939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 0x56096f034df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.526+0000 7fca65bc0640 1 -- 172.21.15.5:0/1102633939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 msgr2=0x56096f045310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.526+0000 7fca65bc0640 1 --2- 172.21.15.5:0/1102633939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f045310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.526+0000 7fca65bc0640 1 -- 172.21.15.5:0/1102633939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56096f0370d0 con 0x56096f02fb40 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.526+0000 7fca65bc0640 1 --2- 172.21.15.5:0/1102633939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 0x56096f0348b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fca500cdf10 tx=0x7fca500d0140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0a4793039915a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.527+0000 7fca4bfff640 1 -- 172.21.15.5:0/1102633939 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca500b4070 con 0x56096f02fb40 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.527+0000 7fca4bfff640 1 -- 172.21.15.5:0/1102633939 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca5007e020 con 0x56096f02fb40 2024-09-19T06:18:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.527+0000 7fca4bfff640 1 -- 172.21.15.5:0/1102633939 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5009cba0 con 0x56096f02fb40 2024-09-19T06:18:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.527+0000 7fca6a9fe740 1 -- 172.21.15.5:0/1102633939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 msgr2=0x56096f0348b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.527+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/1102633939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 0x56096f0348b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fca500cdf10 tx=0x7fca500d0140 comp rx=0 tx=0).stop 2024-09-19T06:18:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.527+0000 7fca6a9fe740 1 -- 172.21.15.5:0/1102633939 shutdown_connections 2024-09-19T06:18:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.528+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/1102633939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 0x56096f034df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.528+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/1102633939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 0x56096f0348b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.528+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/1102633939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f045310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.528+0000 7fca6a9fe740 1 -- 172.21.15.5:0/1102633939 >> 172.21.15.5:0/1102633939 conn(0x56096eff7310 msgr2=0x56096f0312c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.528+0000 7fca6a9fe740 1 -- 172.21.15.5:0/1102633939 shutdown_connections 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.528+0000 7fca6a9fe740 1 -- 172.21.15.5:0/1102633939 wait complete. 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.528+0000 7fca6a9fe740 1 -- start start 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.528+0000 7fca6a9fe740 1 Processor -- start 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca6a9fe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f041a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca6a9fe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 0x56096f03cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca6a9fe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 0x56096f03d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca663c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f041a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca65bc0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 0x56096f03cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca663c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f041a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47302/0 (socket says 172.21.15.5:47302) 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca65bc0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 0x56096f03cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54236/0 (socket says 172.21.15.5:54236) 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca6a9fe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca601b5470 con 0x56096f02e5d0 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2201516658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fca601b4ce0 con 0x56096f02fb40 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca663c1640 1 -- 172.21.15.5:0/2201516658 learned_addr learned my addr 172.21.15.5:0/2201516658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2201516658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca601b1e50 con 0x56096f039370 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca6a9fe740 10 client.0 _ll_register_callbacks cb 0x7fca500048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca66bc2640 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 0x56096f03d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca65bc0640 1 -- 172.21.15.5:0/2201516658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 msgr2=0x56096f03d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca65bc0640 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 0x56096f03d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca65bc0640 1 -- 172.21.15.5:0/2201516658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 msgr2=0x56096f041a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca65bc0640 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f041a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca65bc0640 1 -- 172.21.15.5:0/2201516658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56096f03d8b0 con 0x56096f02fb40 2024-09-19T06:18:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.529+0000 7fca663c1640 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f041a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.530+0000 7fca65bc0640 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 0x56096f03cd30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fca5009c3e0 tx=0x7fca50098400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebf4af53ea766e73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.530+0000 7fca49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.530+0000 7fca49ffb640 1 -- 172.21.15.5:0/2201516658 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca500b4030 con 0x56096f02fb40 2024-09-19T06:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.530+0000 7fca49ffb640 1 -- 172.21.15.5:0/2201516658 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fca500b5270 con 0x56096f02fb40 2024-09-19T06:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.530+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2201516658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5007e020 con 0x56096f02fb40 2024-09-19T06:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.530+0000 7fca49ffb640 1 -- 172.21.15.5:0/2201516658 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fca5009cba0 con 0x56096f02fb40 2024-09-19T06:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.530+0000 7fca6a9fe740 10 client.28814 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.530+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2201516658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56096f03d8b0 con 0x56096f02fb40 2024-09-19T06:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.530+0000 7fca49ffb640 1 -- 172.21.15.5:0/2201516658 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca5007e020 con 0x56096f02fb40 2024-09-19T06:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.531+0000 7fca49ffb640 1 -- 172.21.15.5:0/2201516658 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7fca50081070 con 0x56096f02fb40 2024-09-19T06:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.531+0000 7fca6a9fe740 10 client.28814 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.531+0000 7fca6a9fe740 10 client.28814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.531+0000 7fca6a9fe740 20 client.28814 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.531+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56096f0490c0 0x56096f0b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.531+0000 7fca6a9fe740 4 client.28814 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.531+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2201516658 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56096ef9bd70 con 0x56096f0490c0 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.531+0000 7fca663c1640 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56096f0490c0 0x56096f0b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.531+0000 7fca663c1640 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56096f0490c0 0x56096f0b5ea0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353216930208795b server_cookie=27474d3fd7247e71 in_seq=0 out_seq=0 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.531+0000 7fca49ffb640 10 client.28814 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca49ffb640 1 -- 172.21.15.5:0/2201516658 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56096ef9bd70 con 0x56096f0490c0 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca49ffb640 10 client.28814 handle_command_reply: tid=0 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca6a9fe740 1 client.28814 shutdown 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2201516658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 msgr2=0x56096f03cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 0x56096f03cd30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fca5009c3e0 tx=0x7fca50098400 comp rx=0 tx=0).stop 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2201516658 shutdown_connections 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56096f0490c0 0x56096f0b5ea0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56096f039370 0x56096f03d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56096f02fb40 0x56096f03cd30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca6a9fe740 1 --2- 172.21.15.5:0/2201516658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56096f02e5d0 0x56096f041a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2201516658 >> 172.21.15.5:0/2201516658 conn(0x56096eff7310 msgr2=0x56096f0347f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca49ffb640 0 client.28814 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2201516658 shutdown_connections 2024-09-19T06:18:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca6a9fe740 1 -- 172.21.15.5:0/2201516658 wait complete. 2024-09-19T06:18:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.532+0000 7fca6a9fe740 20 client.28814 trim_cache size 0 max 16384 2024-09-19T06:18:35.539 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:18:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.533+0000 7fca68625640 1 -- 172.21.15.5:0/2578333257 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fca4c0a3430 msgr2=0x7fca4c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.533+0000 7fca68625640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fca4c0a3430 0x7fca4c0a58f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fca5c031440 tx=0x7fca5c005a20 comp rx=0 tx=0).stop 2024-09-19T06:18:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.533+0000 7fca68625640 1 -- 172.21.15.5:0/2578333257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca60102790 msgr2=0x7fca60074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.533+0000 7fca68625640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca60102790 0x7fca60074a30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fca5000e9f0 tx=0x7fca5000edf0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.533+0000 7fca68625640 1 -- 172.21.15.5:0/2578333257 shutdown_connections 2024-09-19T06:18:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.533+0000 7fca68625640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fca4c0a3430 0x7fca4c0a58f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.533+0000 7fca68625640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60106f10 0x7fca60074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.533+0000 7fca68625640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca60102790 0x7fca60074a30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.533+0000 7fca68625640 1 --2- 172.21.15.5:0/2578333257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca60101de0 0x7fca600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.533+0000 7fca68625640 1 -- 172.21.15.5:0/2578333257 >> 172.21.15.5:0/2578333257 conn(0x7fca600fd8e0 msgr2=0x7fca600feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.533+0000 7fca68625640 1 -- 172.21.15.5:0/2578333257 shutdown_connections 2024-09-19T06:18:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:35.533+0000 7fca68625640 1 -- 172.21.15.5:0/2578333257 wait complete. 2024-09-19T06:18:35.552 INFO:tasks.workunit.client.0.smithi005.stdout: 10 20054 10.60 MB/sec warmup 119 sec latency 75.520 ms 2024-09-19T06:18:36.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:36 smithi005 ceph-mon[30955]: pgmap v479: 97 pgs: 97 active+clean; 529 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-09-19T06:18:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:36 smithi157 ceph-mon[35346]: pgmap v479: 97 pgs: 97 active+clean; 529 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-09-19T06:18:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:36 smithi049 ceph-mon[34640]: pgmap v479: 97 pgs: 97 active+clean; 529 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-09-19T06:18:37.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 20351 3.37 MB/sec execute 1 sec latency 111.156 ms 2024-09-19T06:18:37.733 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:18:37.734 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:18:38.103 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:18:38.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 20510 6.14 MB/sec execute 2 sec latency 82.887 ms 2024-09-19T06:18:38.580 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:38 smithi005 ceph-mon[30955]: pgmap v480: 97 pgs: 97 active+clean; 542 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 367 op/s 2024-09-19T06:18:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.706+0000 7f7126533640 1 -- 172.21.15.5:0/105608671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7120102ef0 msgr2=0x7f712010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.706+0000 7f7126533640 1 --2- 172.21.15.5:0/105608671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7120102ef0 0x7f712010da40 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f7118009ca0 tx=0x7f711802a950 comp rx=0 tx=0).stop 2024-09-19T06:18:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.707+0000 7f7126533640 1 -- 172.21.15.5:0/105608671 shutdown_connections 2024-09-19T06:18:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.707+0000 7f7126533640 1 --2- 172.21.15.5:0/105608671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7120102ef0 0x7f712010da40 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.707+0000 7f7126533640 1 --2- 172.21.15.5:0/105608671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7120102570 0x7f71201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.707+0000 7f7126533640 1 --2- 172.21.15.5:0/105608671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7120100e20 0x7f7120101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.707+0000 7f7126533640 1 -- 172.21.15.5:0/105608671 >> 172.21.15.5:0/105608671 conn(0x7f71200fc7d0 msgr2=0x7f71200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.707+0000 7f7126533640 1 -- 172.21.15.5:0/105608671 shutdown_connections 2024-09-19T06:18:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.707+0000 7f7126533640 1 -- 172.21.15.5:0/105608671 wait complete. 2024-09-19T06:18:38.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.708+0000 7f7126533640 1 Processor -- start 2024-09-19T06:18:38.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.708+0000 7f7126533640 1 -- start start 2024-09-19T06:18:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.708+0000 7f7126533640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7120100e20 0x7f712019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.708+0000 7f7126533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7120102570 0x7f712019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.708+0000 7f7126533640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7120102ef0 0x7f71201a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.708+0000 7f7126533640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7120110e80 con 0x7f7120102ef0 2024-09-19T06:18:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.708+0000 7f7126533640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7120110d00 con 0x7f7120100e20 2024-09-19T06:18:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.708+0000 7f7126533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7120111000 con 0x7f7120102570 2024-09-19T06:18:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.708+0000 7f7124ad0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7120102ef0 0x7f71201a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.708+0000 7f7124ad0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7120102ef0 0x7f71201a32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47322/0 (socket says 172.21.15.5:47322) 2024-09-19T06:18:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.708+0000 7f7124ad0640 1 -- 172.21.15.5:0/117413827 learned_addr learned my addr 172.21.15.5:0/117413827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f711ffff640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7120100e20 0x7f712019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f7124ad0640 1 -- 172.21.15.5:0/117413827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7120102570 msgr2=0x7f712019f210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:18:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f7124ad0640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7120102570 0x7f712019f210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f7124ad0640 1 -- 172.21.15.5:0/117413827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7120100e20 msgr2=0x7f712019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f7124ad0640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7120100e20 0x7f712019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f7124ad0640 1 -- 172.21.15.5:0/117413827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71201a39a0 con 0x7f7120102ef0 2024-09-19T06:18:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f711ffff640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7120100e20 0x7f712019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:18:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f7124ad0640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7120102ef0 0x7f71201a32a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f711802af40 tx=0x7f7118003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f711d7fa640 1 -- 172.21.15.5:0/117413827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118034040 con 0x7f7120102ef0 2024-09-19T06:18:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f7126533640 1 -- 172.21.15.5:0/117413827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71201a3c30 con 0x7f7120102ef0 2024-09-19T06:18:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f711d7fa640 1 -- 172.21.15.5:0/117413827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f711802f8c0 con 0x7f7120102ef0 2024-09-19T06:18:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f711d7fa640 1 -- 172.21.15.5:0/117413827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118004560 con 0x7f7120102ef0 2024-09-19T06:18:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.709+0000 7f7126533640 1 -- 172.21.15.5:0/117413827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71201ab550 con 0x7f7120102ef0 2024-09-19T06:18:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.713+0000 7f711d7fa640 1 -- 172.21.15.5:0/117413827 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7118037020 con 0x7f7120102ef0 2024-09-19T06:18:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.713+0000 7f712890c740 1 Processor -- start 2024-09-19T06:18:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.713+0000 7f712890c740 1 -- start start 2024-09-19T06:18:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f712890c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be95ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f712890c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 0x5575be96a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f712890c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 0x5575be96d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f712890c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5575be8c5280 con 0x5575be960480 2024-09-19T06:18:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f712890c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5575be86cb10 con 0x5575be95fb60 2024-09-19T06:18:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f712890c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5575be8c1240 con 0x5575be96ad40 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f711d7fa640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f71080a33b0 0x7f71080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f711d7fa640 1 -- 172.21.15.5:0/117413827 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f71180f0ac0 con 0x7f7120102ef0 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f7124ad0640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f71080a33b0 0x7f71080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f711ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 0x5575be96a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f711ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 0x5575be96a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47334/0 (socket says 172.21.15.5:47334) 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f711f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be95ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f711ffff640 1 -- 172.21.15.5:0/1438457797 learned_addr learned my addr 172.21.15.5:0/1438457797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.714+0000 7f711f7fe640 1 --2- 172.21.15.5:0/1438457797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 0x5575be96d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.715+0000 7f711f7fe640 1 -- 172.21.15.5:0/1438457797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 msgr2=0x5575be96d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.715+0000 7f711f7fe640 1 --2- 172.21.15.5:0/1438457797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 0x5575be96d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.715+0000 7f711f7fe640 1 -- 172.21.15.5:0/1438457797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 msgr2=0x5575be96a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.715+0000 7f7124ad0640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f71080a33b0 0x7f71080a5870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f71180b9860 tx=0x7f7118004040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.715+0000 7f711f7fe640 1 --2- 172.21.15.5:0/1438457797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 0x5575be96a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.715+0000 7f711f7fe640 1 -- 172.21.15.5:0/1438457797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575be8791f0 con 0x5575be95fb60 2024-09-19T06:18:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.715+0000 7f711ffff640 1 --2- 172.21.15.5:0/1438457797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 0x5575be96a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:18:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.715+0000 7f711f7fe640 1 --2- 172.21.15.5:0/1438457797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be95ff40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7114009f60 tx=0x7f7114009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecb8e13808b48724 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.716+0000 7f71067fc640 1 -- 172.21.15.5:0/1438457797 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71140130d0 con 0x5575be95fb60 2024-09-19T06:18:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.716+0000 7f71067fc640 1 -- 172.21.15.5:0/1438457797 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7114013270 con 0x5575be95fb60 2024-09-19T06:18:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.716+0000 7f71067fc640 1 -- 172.21.15.5:0/1438457797 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71140047a0 con 0x5575be95fb60 2024-09-19T06:18:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.717+0000 7f712890c740 1 -- 172.21.15.5:0/1438457797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 msgr2=0x5575be95ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.717+0000 7f712890c740 1 --2- 172.21.15.5:0/1438457797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be95ff40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7114009f60 tx=0x7f7114009620 comp rx=0 tx=0).stop 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.717+0000 7f712890c740 1 -- 172.21.15.5:0/1438457797 shutdown_connections 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.717+0000 7f712890c740 1 --2- 172.21.15.5:0/1438457797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 0x5575be96d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.717+0000 7f712890c740 1 --2- 172.21.15.5:0/1438457797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 0x5575be96a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.717+0000 7f712890c740 1 --2- 172.21.15.5:0/1438457797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be95ff40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.717+0000 7f712890c740 1 -- 172.21.15.5:0/1438457797 >> 172.21.15.5:0/1438457797 conn(0x5575be95c0f0 msgr2=0x5575be95c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.717+0000 7f712890c740 1 -- 172.21.15.5:0/1438457797 shutdown_connections 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.717+0000 7f712890c740 1 -- 172.21.15.5:0/1438457797 wait complete. 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.717+0000 7f712890c740 1 -- start start 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f712890c740 1 Processor -- start 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f712890c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be967bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f712890c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 0x5575be968110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f712890c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 0x5575be968650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f712890c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5575be96d9b0 con 0x5575be960480 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f712890c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5575be87ff10 con 0x5575be95fb60 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f712890c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5575be96db30 con 0x5575be96ad40 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f712890c740 10 client.0 _ll_register_callbacks cb 0x5575be848a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f711ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 0x5575be968110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f711ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 0x5575be968110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47338/0 (socket says 172.21.15.5:47338) 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f711ffff640 1 -- 172.21.15.5:0/1883140109 learned_addr learned my addr 172.21.15.5:0/1883140109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f711f7fe640 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 0x5575be968650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.718+0000 7f711f7fe640 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be967bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.719+0000 7f711ffff640 1 -- 172.21.15.5:0/1883140109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 msgr2=0x5575be968650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.719+0000 7f711ffff640 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 0x5575be968650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.719+0000 7f711ffff640 1 -- 172.21.15.5:0/1883140109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 msgr2=0x5575be967bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.719+0000 7f711ffff640 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be967bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.719+0000 7f711ffff640 1 -- 172.21.15.5:0/1883140109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575be6366a0 con 0x5575be960480 2024-09-19T06:18:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.719+0000 7f711f7fe640 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 0x5575be968650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.719+0000 7f711ffff640 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 0x5575be968110 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f71100126f0 tx=0x7f7110012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9095ae26e364ac20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.719+0000 7f70ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:18:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.720+0000 7f70ff7fe640 1 -- 172.21.15.5:0/1883140109 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711001f060 con 0x5575be960480 2024-09-19T06:18:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.720+0000 7f712890c740 1 -- 172.21.15.5:0/1883140109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7114013270 con 0x5575be960480 2024-09-19T06:18:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.720+0000 7f70ff7fe640 1 -- 172.21.15.5:0/1883140109 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f7110010d60 con 0x5575be960480 2024-09-19T06:18:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.720+0000 7f70ff7fe640 1 -- 172.21.15.5:0/1883140109 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7110023430 con 0x5575be960480 2024-09-19T06:18:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.720+0000 7f712890c740 10 client.19443 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.720+0000 7f712890c740 1 -- 172.21.15.5:0/1883140109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5575be9795f0 con 0x5575be960480 2024-09-19T06:18:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.720+0000 7f70ff7fe640 1 -- 172.21.15.5:0/1883140109 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7114013270 con 0x5575be960480 2024-09-19T06:18:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.721+0000 7f70ff7fe640 1 -- 172.21.15.5:0/1883140109 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7f7110019030 con 0x5575be960480 2024-09-19T06:18:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.721+0000 7f712890c740 10 client.19443 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.721+0000 7f712890c740 10 client.19443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.721+0000 7f712890c740 20 client.19443 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.721+0000 7f712890c740 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5575be97ab00 0x5575be97cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.721+0000 7f712890c740 4 client.19443 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.721+0000 7f712890c740 1 -- 172.21.15.5:0/1883140109 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575be8d27f0 con 0x5575be97ab00 2024-09-19T06:18:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.721+0000 7f711f7fe640 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5575be97ab00 0x5575be97cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.722+0000 7f70ff7fe640 10 client.19443 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.722+0000 7f711f7fe640 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5575be97ab00 0x5575be97cf50 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f13d0b6ff89519c server_cookie=3a212896455b7dee in_seq=0 out_seq=0 2024-09-19T06:18:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.722+0000 7f70ff7fe640 1 -- 172.21.15.5:0/1883140109 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575be8d27f0 con 0x5575be97ab00 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.722+0000 7f70ff7fe640 10 client.19443 handle_command_reply: tid=0 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.722+0000 7f712890c740 1 client.19443 shutdown 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.723+0000 7f712890c740 1 -- 172.21.15.5:0/1883140109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 msgr2=0x5575be968110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.723+0000 7f712890c740 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 0x5575be968110 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f71100126f0 tx=0x7f7110012bb0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.723+0000 7f712890c740 1 -- 172.21.15.5:0/1883140109 shutdown_connections 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.723+0000 7f712890c740 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5575be97ab00 0x5575be97cf50 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.723+0000 7f712890c740 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 0x5575be968650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.723+0000 7f712890c740 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be960480 0x5575be968110 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.723+0000 7f712890c740 1 --2- 172.21.15.5:0/1883140109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be967bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.723+0000 7f712890c740 1 -- 172.21.15.5:0/1883140109 >> 172.21.15.5:0/1883140109 conn(0x5575be95c0f0 msgr2=0x5575be95b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.723+0000 7f70ff7fe640 0 client.19443 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.723+0000 7f712890c740 1 -- 172.21.15.5:0/1883140109 shutdown_connections 2024-09-19T06:18:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.723+0000 7f712890c740 1 -- 172.21.15.5:0/1883140109 wait complete. 2024-09-19T06:18:38.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.723+0000 7f712890c740 20 client.19443 trim_cache size 0 max 16384 2024-09-19T06:18:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f712890c740 1 Processor -- start 2024-09-19T06:18:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f712890c740 1 -- start start 2024-09-19T06:18:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f712890c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be95fb60 0x5575be982820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f712890c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be96ad40 0x5575be982d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f712890c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 0x5575be9615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f712890c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71201b3d30 con 0x5575be9635f0 2024-09-19T06:18:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f712890c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f71201b5a20 con 0x5575be96ad40 2024-09-19T06:18:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f712890c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71201b44c0 con 0x5575be95fb60 2024-09-19T06:18:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f711f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 0x5575be9615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f711f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 0x5575be9615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47344/0 (socket says 172.21.15.5:47344) 2024-09-19T06:18:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f711ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be96ad40 0x5575be982d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f711f7fe640 1 -- 172.21.15.5:0/2867702001 learned_addr learned my addr 172.21.15.5:0/2867702001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.750+0000 7f711f7fe640 1 --2- 172.21.15.5:0/2867702001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be95fb60 0x5575be982820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.751+0000 7f711ffff640 1 -- 172.21.15.5:0/2867702001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be95fb60 msgr2=0x5575be982820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.751+0000 7f711ffff640 1 --2- 172.21.15.5:0/2867702001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be95fb60 0x5575be982820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.751+0000 7f711ffff640 1 -- 172.21.15.5:0/2867702001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 msgr2=0x5575be9615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.751+0000 7f711ffff640 1 --2- 172.21.15.5:0/2867702001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 0x5575be9615b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.751+0000 7f711ffff640 1 -- 172.21.15.5:0/2867702001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575be957830 con 0x5575be96ad40 2024-09-19T06:18:38.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.751+0000 7f711ffff640 1 --2- 172.21.15.5:0/2867702001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be96ad40 0x5575be982d60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x5575be968cc0 tx=0x7f7110009d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45e7be704b71aaad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:38.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.751+0000 7f7105ffb640 1 -- 172.21.15.5:0/2867702001 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71100227a0 con 0x5575be96ad40 2024-09-19T06:18:38.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.751+0000 7f7105ffb640 1 -- 172.21.15.5:0/2867702001 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7110022940 con 0x5575be96ad40 2024-09-19T06:18:38.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.751+0000 7f7105ffb640 1 -- 172.21.15.5:0/2867702001 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110004330 con 0x5575be96ad40 2024-09-19T06:18:38.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.752+0000 7f712890c740 1 -- 172.21.15.5:0/2867702001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be96ad40 msgr2=0x5575be982d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:38.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.752+0000 7f712890c740 1 --2- 172.21.15.5:0/2867702001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be96ad40 0x5575be982d60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x5575be968cc0 tx=0x7f7110009d30 comp rx=0 tx=0).stop 2024-09-19T06:18:38.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.752+0000 7f712890c740 1 -- 172.21.15.5:0/2867702001 shutdown_connections 2024-09-19T06:18:38.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.752+0000 7f712890c740 1 --2- 172.21.15.5:0/2867702001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 0x5575be9615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.752+0000 7f712890c740 1 --2- 172.21.15.5:0/2867702001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be96ad40 0x5575be982d60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.752+0000 7f712890c740 1 --2- 172.21.15.5:0/2867702001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be95fb60 0x5575be982820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.752+0000 7f712890c740 1 -- 172.21.15.5:0/2867702001 >> 172.21.15.5:0/2867702001 conn(0x5575be95c0f0 msgr2=0x5575be966cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.752+0000 7f712890c740 1 -- 172.21.15.5:0/2867702001 shutdown_connections 2024-09-19T06:18:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.753+0000 7f712890c740 1 -- 172.21.15.5:0/2867702001 wait complete. 2024-09-19T06:18:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.753+0000 7f712890c740 1 -- start start 2024-09-19T06:18:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.753+0000 7f712890c740 1 Processor -- start 2024-09-19T06:18:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f712890c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be973180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f712890c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 0x5575be9736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f712890c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 0x5575be96e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f712890c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71201b5350 con 0x5575be9635f0 2024-09-19T06:18:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f712890c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f71201b4bc0 con 0x5575be95fb60 2024-09-19T06:18:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f712890c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71201b1d30 con 0x5575be96ad40 2024-09-19T06:18:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f712890c740 10 client.0 _ll_register_callbacks cb 0x7f7110009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f711ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 0x5575be9736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f711f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be973180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f711ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 0x5575be9736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47358/0 (socket says 172.21.15.5:47358) 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f711ffff640 1 -- 172.21.15.5:0/1817784760 learned_addr learned my addr 172.21.15.5:0/1817784760 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f711ffff640 1 -- 172.21.15.5:0/1817784760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 msgr2=0x5575be96e6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f711ffff640 1 --2- 172.21.15.5:0/1817784760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 0x5575be96e6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f711ffff640 1 -- 172.21.15.5:0/1817784760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 msgr2=0x5575be973180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f711ffff640 1 --2- 172.21.15.5:0/1817784760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be973180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f711ffff640 1 -- 172.21.15.5:0/1817784760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575be96ef10 con 0x5575be9635f0 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f711f7fe640 1 --2- 172.21.15.5:0/1817784760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be973180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f711ffff640 1 --2- 172.21.15.5:0/1817784760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 0x5575be9736c0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f71100224b0 tx=0x7f71100059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33344d10908e7017 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.754+0000 7f70ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:18:38.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.755+0000 7f70ff7fe640 1 -- 172.21.15.5:0/1817784760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110030070 con 0x5575be9635f0 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.755+0000 7f70ff7fe640 1 -- 172.21.15.5:0/1817784760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f7110022d50 con 0x5575be9635f0 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.755+0000 7f712890c740 1 -- 172.21.15.5:0/1817784760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7110022940 con 0x5575be9635f0 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.755+0000 7f70ff7fe640 1 -- 172.21.15.5:0/1817784760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f711003c050 con 0x5575be9635f0 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.755+0000 7f712890c740 10 client.19449 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.755+0000 7f712890c740 1 -- 172.21.15.5:0/1817784760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5575be9e54c0 con 0x5575be9635f0 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.755+0000 7f70ff7fe640 1 -- 172.21.15.5:0/1817784760 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7110022940 con 0x5575be9635f0 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.756+0000 7f70ff7fe640 1 -- 172.21.15.5:0/1817784760 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7f7110019040 con 0x5575be9635f0 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.756+0000 7f712890c740 10 client.19449 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.756+0000 7f712890c740 10 client.19449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.756+0000 7f712890c740 20 client.19449 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.756+0000 7f712890c740 1 --2- 172.21.15.5:0/1817784760 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5575be9e5c40 0x5575be9e8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.756+0000 7f712890c740 4 client.19449 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:18:38.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.756+0000 7f712890c740 1 -- 172.21.15.5:0/1817784760 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5575be8ccd00 con 0x5575be9e5c40 2024-09-19T06:18:38.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.756+0000 7f711f7fe640 1 --2- 172.21.15.5:0/1817784760 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5575be9e5c40 0x5575be9e8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:38.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.757+0000 7f711f7fe640 1 --2- 172.21.15.5:0/1817784760 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5575be9e5c40 0x5575be9e8080 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eeb5168faaffd5f server_cookie=cd99fabe39d5184e in_seq=0 out_seq=0 2024-09-19T06:18:38.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.757+0000 7f70ff7fe640 10 client.19449 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:38.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f70ff7fe640 1 -- 172.21.15.5:0/1817784760 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5575be8ccd00 con 0x5575be9e5c40 2024-09-19T06:18:38.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f70ff7fe640 10 client.19449 handle_command_reply: tid=0 2024-09-19T06:18:38.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f712890c740 1 client.19449 shutdown 2024-09-19T06:18:38.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f712890c740 1 -- 172.21.15.5:0/1817784760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 msgr2=0x5575be9736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:38.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f712890c740 1 --2- 172.21.15.5:0/1817784760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 0x5575be9736c0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f71100224b0 tx=0x7f71100059b0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f712890c740 1 -- 172.21.15.5:0/1817784760 shutdown_connections 2024-09-19T06:18:38.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f712890c740 1 --2- 172.21.15.5:0/1817784760 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5575be9e5c40 0x5575be9e8080 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f712890c740 1 --2- 172.21.15.5:0/1817784760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5575be96ad40 0x5575be96e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f712890c740 1 --2- 172.21.15.5:0/1817784760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575be9635f0 0x5575be9736c0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f712890c740 1 --2- 172.21.15.5:0/1817784760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575be95fb60 0x5575be973180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f712890c740 1 -- 172.21.15.5:0/1817784760 >> 172.21.15.5:0/1817784760 conn(0x5575be95c0f0 msgr2=0x5575be95aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:38.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f70ff7fe640 0 client.19449 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:38.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.758+0000 7f712890c740 1 -- 172.21.15.5:0/1817784760 shutdown_connections 2024-09-19T06:18:38.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.759+0000 7f712890c740 1 -- 172.21.15.5:0/1817784760 wait complete. 2024-09-19T06:18:38.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.759+0000 7f712890c740 20 client.19449 trim_cache size 0 max 16384 2024-09-19T06:18:38.765 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:18:38.765 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:18:38.765 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "ca0b287f-9bc7-4d7a-a8af-b92b98bfbccf", 2024-09-19T06:18:38.765 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:18:38.766 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:18:38.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.760+0000 7f7126533640 1 -- 172.21.15.5:0/117413827 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f71080a33b0 msgr2=0x7f71080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:38.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.760+0000 7f7126533640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f71080a33b0 0x7f71080a5870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f71180b9860 tx=0x7f7118004040 comp rx=0 tx=0).stop 2024-09-19T06:18:38.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.760+0000 7f7126533640 1 -- 172.21.15.5:0/117413827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7120102ef0 msgr2=0x7f71201a32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.760+0000 7f7126533640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7120102ef0 0x7f71201a32a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f711802af40 tx=0x7f7118003c10 comp rx=0 tx=0).stop 2024-09-19T06:18:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.760+0000 7f7126533640 1 -- 172.21.15.5:0/117413827 shutdown_connections 2024-09-19T06:18:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.760+0000 7f7126533640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7120102ef0 0x7f71201a32a0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.760+0000 7f7126533640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f71080a33b0 0x7f71080a5870 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.760+0000 7f7126533640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7120102570 0x7f712019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.760+0000 7f7126533640 1 --2- 172.21.15.5:0/117413827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7120100e20 0x7f712019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.760+0000 7f7126533640 1 -- 172.21.15.5:0/117413827 >> 172.21.15.5:0/117413827 conn(0x7f71200fc7d0 msgr2=0x7f71200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.760+0000 7f7126533640 1 -- 172.21.15.5:0/117413827 shutdown_connections 2024-09-19T06:18:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:38.760+0000 7f7126533640 1 -- 172.21.15.5:0/117413827 wait complete. 2024-09-19T06:18:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:38 smithi157 ceph-mon[35346]: pgmap v480: 97 pgs: 97 active+clean; 542 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 367 op/s 2024-09-19T06:18:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:38 smithi049 ceph-mon[34640]: pgmap v480: 97 pgs: 97 active+clean; 542 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 367 op/s 2024-09-19T06:18:39.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:18:39.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:18:39.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:18:39.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:18:39.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:18:39.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:18:39.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:18:39.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:18:39.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:18:39.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:39 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 20651 5.14 MB/sec execute 3 sec latency 80.355 ms 2024-09-19T06:18:39.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:18:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:18:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:18:39.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:18:39.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:18:39.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:18:39.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:18:39.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:18:39.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:18:39.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:18:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:39 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:18:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:18:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:18:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:18:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:18:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:18:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:18:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:18:39.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:18:39.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:39 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:39.952 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:18:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.511+0000 7f285092a640 1 -- 172.21.15.5:0/3876030477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284c101340 msgr2=0x7f284c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.511+0000 7f285092a640 1 --2- 172.21.15.5:0/3876030477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284c101340 0x7f284c10d870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2844009be0 tx=0x7f284402a6c0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.511+0000 7f285092a640 1 -- 172.21.15.5:0/3876030477 shutdown_connections 2024-09-19T06:18:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.511+0000 7f285092a640 1 --2- 172.21.15.5:0/3876030477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284c101340 0x7f284c10d870 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.511+0000 7f285092a640 1 --2- 172.21.15.5:0/3876030477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f284c1009a0 0x7f284c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.511+0000 7f285092a640 1 --2- 172.21.15.5:0/3876030477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f284c1069b0 0x7f284c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.511+0000 7f285092a640 1 -- 172.21.15.5:0/3876030477 >> 172.21.15.5:0/3876030477 conn(0x7f284c0779a0 msgr2=0x7f284c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.511+0000 7f285092a640 1 -- 172.21.15.5:0/3876030477 shutdown_connections 2024-09-19T06:18:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.511+0000 7f285092a640 1 -- 172.21.15.5:0/3876030477 wait complete. 2024-09-19T06:18:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.512+0000 7f285092a640 1 Processor -- start 2024-09-19T06:18:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.512+0000 7f285092a640 1 -- start start 2024-09-19T06:18:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.513+0000 7f285092a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f284c1009a0 0x7f284c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.513+0000 7f285092a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f284c101340 0x7f284c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.513+0000 7f2849d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f284c101340 0x7f284c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.513+0000 7f2849d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f284c101340 0x7f284c199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47382/0 (socket says 172.21.15.5:47382) 2024-09-19T06:18:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.513+0000 7f284a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f284c1009a0 0x7f284c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.513+0000 7f284a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f284c1009a0 0x7f284c198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42410/0 (socket says 172.21.15.5:42410) 2024-09-19T06:18:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.513+0000 7f2849d9b640 1 -- 172.21.15.5:0/165943577 learned_addr learned my addr 172.21.15.5:0/165943577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.513+0000 7f285092a640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284c1069b0 0x7f284c1a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.513+0000 7f285092a640 1 -- 172.21.15.5:0/165943577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f284c110ca0 con 0x7f284c101340 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.513+0000 7f285092a640 1 -- 172.21.15.5:0/165943577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f284c110b20 con 0x7f284c1069b0 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.513+0000 7f285092a640 1 -- 172.21.15.5:0/165943577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f284c110e20 con 0x7f284c1009a0 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.513+0000 7f284ad9d640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284c1069b0 0x7f284c1a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.514+0000 7f2849d9b640 1 -- 172.21.15.5:0/165943577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f284c1009a0 msgr2=0x7f284c198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.514+0000 7f2849d9b640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f284c1009a0 0x7f284c198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.514+0000 7f2849d9b640 1 -- 172.21.15.5:0/165943577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284c1069b0 msgr2=0x7f284c1a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.514+0000 7f2849d9b640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284c1069b0 0x7f284c1a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.514+0000 7f2849d9b640 1 -- 172.21.15.5:0/165943577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c1a44f0 con 0x7f284c101340 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.514+0000 7f284a59c640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f284c1009a0 0x7f284c198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.514+0000 7f2849d9b640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f284c101340 0x7f284c199470 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f284000d950 tx=0x7f284000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.514+0000 7f283b7fe640 1 -- 172.21.15.5:0/165943577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840017070 con 0x7f284c101340 2024-09-19T06:18:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.514+0000 7f283b7fe640 1 -- 172.21.15.5:0/165943577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f284000de80 con 0x7f284c101340 2024-09-19T06:18:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.514+0000 7f285092a640 1 -- 172.21.15.5:0/165943577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c1a46c0 con 0x7f284c101340 2024-09-19T06:18:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.514+0000 7f283b7fe640 1 -- 172.21.15.5:0/165943577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284000f620 con 0x7f284c101340 2024-09-19T06:18:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.514+0000 7f285092a640 1 -- 172.21.15.5:0/165943577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c1a4b70 con 0x7f284c101340 2024-09-19T06:18:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.518+0000 7f283b7fe640 1 -- 172.21.15.5:0/165943577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2840009c30 con 0x7f284c101340 2024-09-19T06:18:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f28536c6740 1 Processor -- start 2024-09-19T06:18:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f28536c6740 1 -- start start 2024-09-19T06:18:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f283b7fe640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f28340a3480 0x7f28340a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f284ad9d640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f28340a3480 0x7f28340a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f28536c6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 0x5608e9409d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f28536c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e940a290 0x5608e9418b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f28536c6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e941b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f28536c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5608e936e2f0 con 0x5608e940a290 2024-09-19T06:18:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f28536c6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5608e9316270 con 0x5608e94190c0 2024-09-19T06:18:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f28536c6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5608e936a2b0 con 0x5608e9409970 2024-09-19T06:18:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f283b7fe640 1 -- 172.21.15.5:0/165943577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f28400cf890 con 0x7f284c101340 2024-09-19T06:18:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f284a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 0x5608e9409d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f284a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 0x5608e9409d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42426/0 (socket says 172.21.15.5:42426) 2024-09-19T06:18:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.519+0000 7f284a59c640 1 -- 172.21.15.5:0/3333738015 learned_addr learned my addr 172.21.15.5:0/3333738015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.520+0000 7f2849d9b640 1 --2- 172.21.15.5:0/3333738015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e940a290 0x5608e9418b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.520+0000 7f284ad9d640 1 --2- 172.21.15.5:0/3333738015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e941b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.520+0000 7f284ad9d640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f28340a3480 0x7f28340a5940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2844009560 tx=0x7f284402abc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.520+0000 7f284a59c640 1 -- 172.21.15.5:0/3333738015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 msgr2=0x5608e941b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.520+0000 7f284a59c640 1 --2- 172.21.15.5:0/3333738015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e941b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.520+0000 7f284a59c640 1 -- 172.21.15.5:0/3333738015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e940a290 msgr2=0x5608e9418b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.520+0000 7f284a59c640 1 --2- 172.21.15.5:0/3333738015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e940a290 0x5608e9418b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.520+0000 7f284a59c640 1 -- 172.21.15.5:0/3333738015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e9315340 con 0x5608e9409970 2024-09-19T06:18:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.520+0000 7f2849d9b640 1 --2- 172.21.15.5:0/3333738015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e940a290 0x5608e9418b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.520+0000 7f284a59c640 1 --2- 172.21.15.5:0/3333738015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 0x5608e9409d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f283c0114f0 tx=0x7f283c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b8b0817fb55f306 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.520+0000 7f2838ff9640 1 -- 172.21.15.5:0/3333738015 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c004020 con 0x5608e9409970 2024-09-19T06:18:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.521+0000 7f2838ff9640 1 -- 172.21.15.5:0/3333738015 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f283c0041c0 con 0x5608e9409970 2024-09-19T06:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.521+0000 7f2838ff9640 1 -- 172.21.15.5:0/3333738015 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c00f880 con 0x5608e9409970 2024-09-19T06:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.521+0000 7f28536c6740 1 -- 172.21.15.5:0/3333738015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 msgr2=0x5608e9409d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.521+0000 7f28536c6740 1 --2- 172.21.15.5:0/3333738015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 0x5608e9409d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f283c0114f0 tx=0x7f283c0119b0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.522+0000 7f28536c6740 1 -- 172.21.15.5:0/3333738015 shutdown_connections 2024-09-19T06:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.522+0000 7f28536c6740 1 --2- 172.21.15.5:0/3333738015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e941b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.522+0000 7f28536c6740 1 --2- 172.21.15.5:0/3333738015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e940a290 0x5608e9418b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.522+0000 7f28536c6740 1 --2- 172.21.15.5:0/3333738015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 0x5608e9409d50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.522+0000 7f28536c6740 1 -- 172.21.15.5:0/3333738015 >> 172.21.15.5:0/3333738015 conn(0x5608e9406060 msgr2=0x5608e9406440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.522+0000 7f28536c6740 1 -- 172.21.15.5:0/3333738015 shutdown_connections 2024-09-19T06:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.522+0000 7f28536c6740 1 -- 172.21.15.5:0/3333738015 wait complete. 2024-09-19T06:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.522+0000 7f28536c6740 1 -- start start 2024-09-19T06:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.522+0000 7f28536c6740 1 Processor -- start 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f28536c6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 0x5608e9416050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f28536c6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e940a290 0x5608e9416590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f28536c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e94190c0 0x5608e9416ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f28536c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5608e941bcf0 con 0x5608e94190c0 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f28536c6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5608e92f7290 con 0x5608e940a290 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f28536c6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5608e941be70 con 0x5608e9409970 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f28536c6740 10 client.0 _ll_register_callbacks cb 0x5608e9328140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f284a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 0x5608e9416050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f284a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 0x5608e9416050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42440/0 (socket says 172.21.15.5:42440) 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f284a59c640 1 -- 172.21.15.5:0/3969023011 learned_addr learned my addr 172.21.15.5:0/3969023011 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f284ad9d640 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e94190c0 0x5608e9416ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f2849d9b640 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e940a290 0x5608e9416590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f284a59c640 1 -- 172.21.15.5:0/3969023011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e940a290 msgr2=0x5608e9416590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f284a59c640 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e940a290 0x5608e9416590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f284a59c640 1 -- 172.21.15.5:0/3969023011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e94190c0 msgr2=0x5608e9416ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f284a59c640 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e94190c0 0x5608e9416ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f284a59c640 1 -- 172.21.15.5:0/3969023011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e9081210 con 0x5608e9409970 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f284a59c640 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 0x5608e9416050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f283c009660 tx=0x7f283c005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82aaedb4bd0e6540 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.523+0000 7f2831ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.524+0000 7f2831ffb640 1 -- 172.21.15.5:0/3969023011 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c028020 con 0x5608e9409970 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.524+0000 7f2831ffb640 1 -- 172.21.15.5:0/3969023011 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f283c004370 con 0x5608e9409970 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.524+0000 7f28536c6740 1 -- 172.21.15.5:0/3969023011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f283c0041c0 con 0x5608e9409970 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.524+0000 7f2831ffb640 1 -- 172.21.15.5:0/3969023011 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f283c033050 con 0x5608e9409970 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.524+0000 7f28536c6740 10 client.28780 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.524+0000 7f28536c6740 1 -- 172.21.15.5:0/3969023011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5608e914d620 con 0x5608e9409970 2024-09-19T06:18:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.524+0000 7f2831ffb640 1 -- 172.21.15.5:0/3969023011 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f283c0041c0 con 0x5608e9409970 2024-09-19T06:18:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.525+0000 7f2831ffb640 1 -- 172.21.15.5:0/3969023011 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7f283c004b80 con 0x5608e9409970 2024-09-19T06:18:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.525+0000 7f28536c6740 10 client.28780 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.525+0000 7f28536c6740 10 client.28780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.525+0000 7f28536c6740 20 client.28780 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.525+0000 7f28536c6740 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5608e9429060 0x5608e942b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.525+0000 7f28536c6740 4 client.28780 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.525+0000 7f28536c6740 1 -- 172.21.15.5:0/3969023011 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608e937ba20 con 0x5608e9429060 2024-09-19T06:18:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.525+0000 7f2849d9b640 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5608e9429060 0x5608e942b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.525+0000 7f2831ffb640 10 client.28780 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.525+0000 7f2849d9b640 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5608e9429060 0x5608e942b450 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13a053af372b5504 server_cookie=b8c01720b35f5d3d in_seq=0 out_seq=0 2024-09-19T06:18:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.526+0000 7f2831ffb640 1 -- 172.21.15.5:0/3969023011 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5608e937ba20 con 0x5608e9429060 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.526+0000 7f2831ffb640 10 client.28780 handle_command_reply: tid=0 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.526+0000 7f28536c6740 1 client.28780 shutdown 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.526+0000 7f28536c6740 1 -- 172.21.15.5:0/3969023011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 msgr2=0x5608e9416050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.526+0000 7f28536c6740 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 0x5608e9416050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f283c009660 tx=0x7f283c005730 comp rx=0 tx=0).stop 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.526+0000 7f28536c6740 1 -- 172.21.15.5:0/3969023011 shutdown_connections 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.526+0000 7f28536c6740 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5608e9429060 0x5608e942b450 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.526+0000 7f28536c6740 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e94190c0 0x5608e9416ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.526+0000 7f28536c6740 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e940a290 0x5608e9416590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.526+0000 7f2831ffb640 0 client.28780 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.526+0000 7f28536c6740 1 --2- 172.21.15.5:0/3969023011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e9409970 0x5608e9416050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.527+0000 7f28536c6740 1 -- 172.21.15.5:0/3969023011 >> 172.21.15.5:0/3969023011 conn(0x5608e9406060 msgr2=0x5608e940bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.527+0000 7f28536c6740 1 -- 172.21.15.5:0/3969023011 shutdown_connections 2024-09-19T06:18:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.527+0000 7f28536c6740 1 -- 172.21.15.5:0/3969023011 wait complete. 2024-09-19T06:18:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.527+0000 7f28536c6740 20 client.28780 trim_cache size 0 max 16384 2024-09-19T06:18:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.549+0000 7f6f3968a640 1 -- 172.21.15.5:0/3568932216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34102570 msgr2=0x7f6f341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.549+0000 7f6f3968a640 1 --2- 172.21.15.5:0/3568932216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34102570 0x7f6f341029b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009900 tx=0x7f6f2802a480 comp rx=0 tx=0).stop 2024-09-19T06:18:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.549+0000 7f6f3968a640 1 -- 172.21.15.5:0/3568932216 shutdown_connections 2024-09-19T06:18:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.550+0000 7f6f3968a640 1 --2- 172.21.15.5:0/3568932216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f34102ef0 0x7f6f34069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.550+0000 7f6f3968a640 1 --2- 172.21.15.5:0/3568932216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34102570 0x7f6f341029b0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.550+0000 7f6f3968a640 1 --2- 172.21.15.5:0/3568932216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f34100e20 0x7f6f34101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.550+0000 7f6f3968a640 1 -- 172.21.15.5:0/3568932216 >> 172.21.15.5:0/3568932216 conn(0x7f6f340fc7d0 msgr2=0x7f6f340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.550+0000 7f6f3968a640 1 -- 172.21.15.5:0/3568932216 shutdown_connections 2024-09-19T06:18:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.550+0000 7f6f3968a640 1 -- 172.21.15.5:0/3568932216 wait complete. 2024-09-19T06:18:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.551+0000 7f6f3968a640 1 Processor -- start 2024-09-19T06:18:40.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 20792 4.96 MB/sec execute 4 sec latency 66.617 ms 2024-09-19T06:18:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.551+0000 7f6f3968a640 1 -- start start 2024-09-19T06:18:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.552+0000 7f6f3968a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34100e20 0x7f6f341a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.552+0000 7f6f3968a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f34102ef0 0x7f6f341a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.552+0000 7f6f32ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34100e20 0x7f6f341a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.552+0000 7f6f3968a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f3419a6b0 0x7f6f3419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.552+0000 7f6f32ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34100e20 0x7f6f341a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47422/0 (socket says 172.21.15.5:47422) 2024-09-19T06:18:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.552+0000 7f6f3968a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341141e0 con 0x7f6f34100e20 2024-09-19T06:18:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.552+0000 7f6f32ffd640 1 -- 172.21.15.5:0/1140389336 learned_addr learned my addr 172.21.15.5:0/1140389336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.552+0000 7f6f3968a640 1 -- 172.21.15.5:0/1140389336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34114060 con 0x7f6f3419a6b0 2024-09-19T06:18:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.552+0000 7f6f3968a640 1 -- 172.21.15.5:0/1140389336 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34114360 con 0x7f6f34102ef0 2024-09-19T06:18:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.553+0000 7f6f337fe640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f3419a6b0 0x7f6f3419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.553+0000 7f6f327fc640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f34102ef0 0x7f6f341a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.553+0000 7f6f32ffd640 1 -- 172.21.15.5:0/1140389336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f34102ef0 msgr2=0x7f6f341a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.553+0000 7f6f32ffd640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f34102ef0 0x7f6f341a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.553+0000 7f6f32ffd640 1 -- 172.21.15.5:0/1140389336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f3419a6b0 msgr2=0x7f6f3419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.553+0000 7f6f32ffd640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f3419a6b0 0x7f6f3419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.553+0000 7f6f32ffd640 1 -- 172.21.15.5:0/1140389336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3419b260 con 0x7f6f34100e20 2024-09-19T06:18:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.554+0000 7f6f32ffd640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34100e20 0x7f6f341a05e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400d950 tx=0x7f6f2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.555+0000 7f6f1bfff640 1 -- 172.21.15.5:0/1140389336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24017070 con 0x7f6f34100e20 2024-09-19T06:18:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.555+0000 7f6f1bfff640 1 -- 172.21.15.5:0/1140389336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f24009c30 con 0x7f6f34100e20 2024-09-19T06:18:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.555+0000 7f6f1bfff640 1 -- 172.21.15.5:0/1140389336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400f620 con 0x7f6f34100e20 2024-09-19T06:18:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.555+0000 7f6f3968a640 1 -- 172.21.15.5:0/1140389336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f341a7390 con 0x7f6f34100e20 2024-09-19T06:18:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.555+0000 7f6f3968a640 1 -- 172.21.15.5:0/1140389336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3410fe20 con 0x7f6f34100e20 2024-09-19T06:18:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.557+0000 7f28536c6740 1 Processor -- start 2024-09-19T06:18:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.557+0000 7f28536c6740 1 -- start start 2024-09-19T06:18:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.557+0000 7f28536c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e9409970 0x5608e9417fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.557+0000 7f28536c6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e940a290 0x5608e9430c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.557+0000 7f28536c6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e94311b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.557+0000 7f28536c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1ab5a0 con 0x5608e9409970 2024-09-19T06:18:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.557+0000 7f28536c6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1ad290 con 0x5608e94190c0 2024-09-19T06:18:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.557+0000 7f28536c6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1abd30 con 0x5608e940a290 2024-09-19T06:18:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.558+0000 7f284ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e94311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.558+0000 7f284ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e94311b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54334/0 (socket says 172.21.15.5:54334) 2024-09-19T06:18:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.558+0000 7f284ad9d640 1 -- 172.21.15.5:0/1486222064 learned_addr learned my addr 172.21.15.5:0/1486222064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.558+0000 7f2849d9b640 1 --2- 172.21.15.5:0/1486222064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e940a290 0x5608e9430c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.558+0000 7f284ad9d640 1 -- 172.21.15.5:0/1486222064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e940a290 msgr2=0x5608e9430c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.558+0000 7f284ad9d640 1 --2- 172.21.15.5:0/1486222064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e940a290 0x5608e9430c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.558+0000 7f284ad9d640 1 -- 172.21.15.5:0/1486222064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e9409970 msgr2=0x5608e9417fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.558+0000 7f284ad9d640 1 --2- 172.21.15.5:0/1486222064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e9409970 0x5608e9417fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.558+0000 7f284ad9d640 1 -- 172.21.15.5:0/1486222064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e940a0c0 con 0x5608e94190c0 2024-09-19T06:18:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.558+0000 7f284ad9d640 1 --2- 172.21.15.5:0/1486222064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e94311b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f284402acb0 tx=0x7f2844041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=def3ed7c450cd0ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.559+0000 7f2833fff640 1 -- 172.21.15.5:0/1486222064 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844036040 con 0x5608e94190c0 2024-09-19T06:18:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.559+0000 7f2833fff640 1 -- 172.21.15.5:0/1486222064 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2844046070 con 0x5608e94190c0 2024-09-19T06:18:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.559+0000 7f2833fff640 1 -- 172.21.15.5:0/1486222064 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284403f680 con 0x5608e94190c0 2024-09-19T06:18:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.558+0000 7f6f3ba63740 1 Processor -- start 2024-09-19T06:18:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.559+0000 7f6f3ba63740 1 -- start start 2024-09-19T06:18:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.559+0000 7f6f3ba63740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250e4a0 0x558d5250e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.559+0000 7f6f3ba63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d5250edc0 0x558d52519140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.559+0000 7f6f3ba63740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d52519680 0x558d5251ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.559+0000 7f6f3ba63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558d524742e0 con 0x558d5250edc0 2024-09-19T06:18:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.559+0000 7f6f3ba63740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558d5241bce0 con 0x558d52519680 2024-09-19T06:18:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.559+0000 7f6f3ba63740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558d524702a0 con 0x558d5250e4a0 2024-09-19T06:18:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250e4a0 0x558d5250e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250e4a0 0x558d5250e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42472/0 (socket says 172.21.15.5:42472) 2024-09-19T06:18:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f6f337fe640 1 -- 172.21.15.5:0/3465617443 learned_addr learned my addr 172.21.15.5:0/3465617443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f6f327fc640 1 --2- 172.21.15.5:0/3465617443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d52519680 0x558d5251ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f28536c6740 1 -- 172.21.15.5:0/1486222064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 msgr2=0x5608e94311b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f28536c6740 1 --2- 172.21.15.5:0/1486222064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e94311b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f284402acb0 tx=0x7f2844041040 comp rx=0 tx=0).stop 2024-09-19T06:18:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f28536c6740 1 -- 172.21.15.5:0/1486222064 shutdown_connections 2024-09-19T06:18:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f28536c6740 1 --2- 172.21.15.5:0/1486222064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e94311b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f28536c6740 1 --2- 172.21.15.5:0/1486222064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e940a290 0x5608e9430c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f28536c6740 1 --2- 172.21.15.5:0/1486222064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e9409970 0x5608e9417fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f28536c6740 1 -- 172.21.15.5:0/1486222064 >> 172.21.15.5:0/1486222064 conn(0x5608e9406060 msgr2=0x5608e94178b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.561+0000 7f28536c6740 1 -- 172.21.15.5:0/1486222064 shutdown_connections 2024-09-19T06:18:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.561+0000 7f28536c6740 1 -- 172.21.15.5:0/1486222064 wait complete. 2024-09-19T06:18:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.561+0000 7f28536c6740 1 -- start start 2024-09-19T06:18:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.562+0000 7f28536c6740 1 Processor -- start 2024-09-19T06:18:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.562+0000 7f28536c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e9409970 0x5608e94211c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.562+0000 7f28536c6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e940a290 0x5608e9421700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.562+0000 7f28536c6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e941e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.562+0000 7f28536c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1acbc0 con 0x5608e9409970 2024-09-19T06:18:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.562+0000 7f28536c6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1ac430 con 0x5608e94190c0 2024-09-19T06:18:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.562+0000 7f28536c6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1a95c0 con 0x5608e940a290 2024-09-19T06:18:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.562+0000 7f28536c6740 10 client.0 _ll_register_callbacks cb 0x7f28400976f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f6f327fc640 1 -- 172.21.15.5:0/3465617443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250e4a0 msgr2=0x558d5250e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f6f327fc640 1 --2- 172.21.15.5:0/3465617443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250e4a0 0x558d5250e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f6f327fc640 1 -- 172.21.15.5:0/3465617443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d5250edc0 msgr2=0x558d52519140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:18:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f6f327fc640 1 --2- 172.21.15.5:0/3465617443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d5250edc0 0x558d52519140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.560+0000 7f6f327fc640 1 -- 172.21.15.5:0/3465617443 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d52400440 con 0x558d52519680 2024-09-19T06:18:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.561+0000 7f6f327fc640 1 --2- 172.21.15.5:0/3465617443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d52519680 0x558d5251ba70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6f28008000 tx=0x7f6f280041c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c82d6ec59cd47369 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.561+0000 7f6f1bfff640 1 -- 172.21.15.5:0/1140389336 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6f240109a0 con 0x7f6f34100e20 2024-09-19T06:18:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.561+0000 7f6f197fa640 1 -- 172.21.15.5:0/3465617443 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28040070 con 0x558d52519680 2024-09-19T06:18:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.561+0000 7f6f197fa640 1 -- 172.21.15.5:0/3465617443 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f280054f0 con 0x558d52519680 2024-09-19T06:18:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.561+0000 7f6f197fa640 1 -- 172.21.15.5:0/3465617443 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28035040 con 0x558d52519680 2024-09-19T06:18:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.562+0000 7f6f3ba63740 1 -- 172.21.15.5:0/3465617443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d52519680 msgr2=0x558d5251ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.562+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/3465617443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d52519680 0x558d5251ba70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6f28008000 tx=0x7f6f280041c0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f284a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e9409970 0x5608e94211c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f2849d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e940a290 0x5608e9421700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f2849d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e940a290 0x5608e9421700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42480/0 (socket says 172.21.15.5:42480) 2024-09-19T06:18:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f2849d9b640 1 -- 172.21.15.5:0/1549708808 learned_addr learned my addr 172.21.15.5:0/1549708808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f284ad9d640 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e941e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f6f3ba63740 1 -- 172.21.15.5:0/3465617443 shutdown_connections 2024-09-19T06:18:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/3465617443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d52519680 0x558d5251ba70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/3465617443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d5250edc0 0x558d52519140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/3465617443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250e4a0 0x558d5250e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f6f3ba63740 1 -- 172.21.15.5:0/3465617443 >> 172.21.15.5:0/3465617443 conn(0x558d5250aa30 msgr2=0x558d5250ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:40.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f6f3ba63740 1 -- 172.21.15.5:0/3465617443 shutdown_connections 2024-09-19T06:18:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f284ad9d640 1 -- 172.21.15.5:0/1549708808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e940a290 msgr2=0x5608e9421700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f284ad9d640 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e940a290 0x5608e9421700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f284ad9d640 1 -- 172.21.15.5:0/1549708808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e9409970 msgr2=0x5608e94211c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f6f3ba63740 1 -- 172.21.15.5:0/3465617443 wait complete. 2024-09-19T06:18:40.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f6f1bfff640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6f1c0a3450 0x7f6f1c0a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f6f3ba63740 1 -- start start 2024-09-19T06:18:40.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f6f3ba63740 1 Processor -- start 2024-09-19T06:18:40.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.563+0000 7f6f1bfff640 1 -- 172.21.15.5:0/1140389336 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f6f240cf840 con 0x7f6f34100e20 2024-09-19T06:18:40.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f6f327fc640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6f1c0a3450 0x7f6f1c0a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f284ad9d640 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e9409970 0x5608e94211c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f284ad9d640 1 -- 172.21.15.5:0/1549708808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e941e9c0 con 0x5608e94190c0 2024-09-19T06:18:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f2849d9b640 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e940a290 0x5608e9421700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f284a59c640 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e9409970 0x5608e94211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:18:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f284ad9d640 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e941e300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2844034040 tx=0x7f2844030b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ac209cfa752cf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f2831ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:18:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f2831ffb640 1 -- 172.21.15.5:0/1549708808 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284407c030 con 0x5608e94190c0 2024-09-19T06:18:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f2831ffb640 1 -- 172.21.15.5:0/1549708808 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f284403f420 con 0x5608e94190c0 2024-09-19T06:18:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f28536c6740 1 -- 172.21.15.5:0/1549708808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2844046070 con 0x5608e94190c0 2024-09-19T06:18:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f2831ffb640 1 -- 172.21.15.5:0/1549708808 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2844087050 con 0x5608e94190c0 2024-09-19T06:18:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f28536c6740 10 client.28853 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f28536c6740 1 -- 172.21.15.5:0/1549708808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5608e940df90 con 0x5608e94190c0 2024-09-19T06:18:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.566+0000 7f2831ffb640 1 -- 172.21.15.5:0/1549708808 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2844046070 con 0x5608e94190c0 2024-09-19T06:18:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.566+0000 7f2831ffb640 1 -- 172.21.15.5:0/1549708808 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7f284403c070 con 0x5608e94190c0 2024-09-19T06:18:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f6f3ba63740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250e4a0 0x558d52519020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f6f3ba63740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250edc0 0x558d525162d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f6f3ba63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d52519680 0x558d52516870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f6f3ba63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558d5251c250 con 0x558d52519680 2024-09-19T06:18:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f6f3ba63740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558d5242f050 con 0x558d5250e4a0 2024-09-19T06:18:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f6f3ba63740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558d5251c3d0 con 0x558d5250edc0 2024-09-19T06:18:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.564+0000 7f6f3ba63740 10 client.0 _ll_register_callbacks cb 0x558d5246f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d52519680 0x558d52516870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d52519680 0x558d52516870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47452/0 (socket says 172.21.15.5:47452) 2024-09-19T06:18:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f6f337fe640 1 -- 172.21.15.5:0/2911829420 learned_addr learned my addr 172.21.15.5:0/2911829420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f6f327fc640 1 --2- 172.21.15.5:0/2911829420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250edc0 0x558d525162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f6f327fc640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6f1c0a3450 0x7f6f1c0a5910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6f280096c0 tx=0x7f6f2802a480 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:40.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f6f337fe640 1 -- 172.21.15.5:0/2911829420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250edc0 msgr2=0x558d525162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f6f337fe640 1 --2- 172.21.15.5:0/2911829420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250edc0 0x558d525162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f6f337fe640 1 -- 172.21.15.5:0/2911829420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250e4a0 msgr2=0x558d52519020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:18:40.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f6f337fe640 1 --2- 172.21.15.5:0/2911829420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250e4a0 0x558d52519020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.565+0000 7f6f337fe640 1 -- 172.21.15.5:0/2911829420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d521e53c0 con 0x558d52519680 2024-09-19T06:18:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.566+0000 7f28536c6740 10 client.28853 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.566+0000 7f28536c6740 10 client.28853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.566+0000 7f28536c6740 20 client.28853 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.566+0000 7f28536c6740 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5608e9498660 0x5608e949aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.566+0000 7f28536c6740 4 client.28853 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:18:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.566+0000 7f28536c6740 1 -- 172.21.15.5:0/1549708808 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608e9375d70 con 0x5608e9498660 2024-09-19T06:18:40.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.566+0000 7f6f337fe640 1 --2- 172.21.15.5:0/2911829420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d52519680 0x558d52516870 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009d40 tx=0x7f6f2c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81afa6325ed5395 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:40.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.566+0000 7f6f16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:18:40.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.566+0000 7f6f16ff5640 1 -- 172.21.15.5:0/2911829420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c003b70 con 0x558d52519680 2024-09-19T06:18:40.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.566+0000 7f6f16ff5640 1 -- 172.21.15.5:0/2911829420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f6f2c003d10 con 0x558d52519680 2024-09-19T06:18:40.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.567+0000 7f6f3ba63740 1 -- 172.21.15.5:0/2911829420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f280054f0 con 0x558d52519680 2024-09-19T06:18:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.567+0000 7f284a59c640 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5608e9498660 0x5608e949aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.567+0000 7f2831ffb640 10 client.28853 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.567+0000 7f284a59c640 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5608e9498660 0x5608e949aa80 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bded414b4f07bb9 server_cookie=869a027937caf796 in_seq=0 out_seq=0 2024-09-19T06:18:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.567+0000 7f6f16ff5640 1 -- 172.21.15.5:0/2911829420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6f2c009e30 con 0x558d52519680 2024-09-19T06:18:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.567+0000 7f6f3ba63740 10 client.19470 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.567+0000 7f6f3ba63740 1 -- 172.21.15.5:0/2911829420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558d52538cd0 con 0x558d52519680 2024-09-19T06:18:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.568+0000 7f6f16ff5640 1 -- 172.21.15.5:0/2911829420 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f280054f0 con 0x558d52519680 2024-09-19T06:18:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.568+0000 7f6f16ff5640 1 -- 172.21.15.5:0/2911829420 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7f6f2c01a070 con 0x558d52519680 2024-09-19T06:18:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.568+0000 7f6f3ba63740 10 client.19470 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.568+0000 7f6f3ba63740 10 client.19470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.568+0000 7f6f3ba63740 20 client.19470 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.568+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/2911829420 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d5253a130 0x558d5253c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.568+0000 7f6f3ba63740 4 client.19470 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.568+0000 7f6f3ba63740 1 -- 172.21.15.5:0/2911829420 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d52481850 con 0x558d5253a130 2024-09-19T06:18:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.568+0000 7f6f32ffd640 1 --2- 172.21.15.5:0/2911829420 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d5253a130 0x558d5253c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.569+0000 7f6f32ffd640 1 --2- 172.21.15.5:0/2911829420 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d5253a130 0x558d5253c590 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=413165b99f569d97 server_cookie=2eebe957c89a2ede in_seq=0 out_seq=0 2024-09-19T06:18:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.569+0000 7f6f16ff5640 10 client.19470 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f16ff5640 1 -- 172.21.15.5:0/2911829420 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d52481850 con 0x558d5253a130 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f16ff5640 10 client.19470 handle_command_reply: tid=0 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f3ba63740 1 client.19470 shutdown 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f3ba63740 1 -- 172.21.15.5:0/2911829420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d52519680 msgr2=0x558d52516870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/2911829420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d52519680 0x558d52516870 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009d40 tx=0x7f6f2c0117b0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f3ba63740 1 -- 172.21.15.5:0/2911829420 shutdown_connections 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/2911829420 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d5253a130 0x558d5253c590 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/2911829420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d52519680 0x558d52516870 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/2911829420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250edc0 0x558d525162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/2911829420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250e4a0 0x558d52519020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f3ba63740 1 -- 172.21.15.5:0/2911829420 >> 172.21.15.5:0/2911829420 conn(0x558d5250aa30 msgr2=0x558d52510710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f16ff5640 0 client.19470 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f3ba63740 1 -- 172.21.15.5:0/2911829420 shutdown_connections 2024-09-19T06:18:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.570+0000 7f6f3ba63740 1 -- 172.21.15.5:0/2911829420 wait complete. 2024-09-19T06:18:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.571+0000 7f6f3ba63740 20 client.19470 trim_cache size 0 max 16384 2024-09-19T06:18:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.607+0000 7f6f3ba63740 1 Processor -- start 2024-09-19T06:18:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.607+0000 7f6f3ba63740 1 -- start start 2024-09-19T06:18:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.607+0000 7f6f3ba63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d5250e4a0 0x558d52505940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.607+0000 7f6f3ba63740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250edc0 0x558d52518240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.607+0000 7f6f3ba63740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d52519680 0x558d52541ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.607+0000 7f6f3ba63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f340703a0 con 0x558d5250e4a0 2024-09-19T06:18:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.607+0000 7f6f3ba63740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34072090 con 0x558d5250edc0 2024-09-19T06:18:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.607+0000 7f6f3ba63740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34070b30 con 0x558d52519680 2024-09-19T06:18:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.607+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d52519680 0x558d52541ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.607+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d52519680 0x558d52541ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42500/0 (socket says 172.21.15.5:42500) 2024-09-19T06:18:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.607+0000 7f6f337fe640 1 -- 172.21.15.5:0/1689724909 learned_addr learned my addr 172.21.15.5:0/1689724909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.608+0000 7f6f327fc640 1 --2- 172.21.15.5:0/1689724909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250edc0 0x558d52518240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.608+0000 7f6f337fe640 1 -- 172.21.15.5:0/1689724909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250edc0 msgr2=0x558d52518240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.608+0000 7f6f337fe640 1 --2- 172.21.15.5:0/1689724909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250edc0 0x558d52518240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.608+0000 7f6f337fe640 1 -- 172.21.15.5:0/1689724909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d5250e4a0 msgr2=0x558d52505940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.608+0000 7f6f337fe640 1 --2- 172.21.15.5:0/1689724909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d5250e4a0 0x558d52505940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.608+0000 7f6f337fe640 1 -- 172.21.15.5:0/1689724909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d525152a0 con 0x558d52519680 2024-09-19T06:18:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.608+0000 7f6f327fc640 1 --2- 172.21.15.5:0/1689724909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250edc0 0x558d52518240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:18:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.608+0000 7f6f337fe640 1 --2- 172.21.15.5:0/1689724909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d52519680 0x558d52541ea0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00a010 tx=0x7f6f2c009ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1067ae9f4c546b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:40.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.609+0000 7f6f18ff9640 1 -- 172.21.15.5:0/1689724909 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c026cd0 con 0x558d52519680 2024-09-19T06:18:40.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.609+0000 7f6f18ff9640 1 -- 172.21.15.5:0/1689724909 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f2c02b070 con 0x558d52519680 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.609+0000 7f6f18ff9640 1 -- 172.21.15.5:0/1689724909 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c005650 con 0x558d52519680 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.609+0000 7f6f3ba63740 1 -- 172.21.15.5:0/1689724909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d52519680 msgr2=0x558d52541ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.610+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/1689724909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d52519680 0x558d52541ea0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00a010 tx=0x7f6f2c009ab0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.610+0000 7f6f3ba63740 1 -- 172.21.15.5:0/1689724909 shutdown_connections 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.610+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/1689724909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d52519680 0x558d52541ea0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.610+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/1689724909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250edc0 0x558d52518240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.610+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/1689724909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d5250e4a0 0x558d52505940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.610+0000 7f6f3ba63740 1 -- 172.21.15.5:0/1689724909 >> 172.21.15.5:0/1689724909 conn(0x558d5250aa30 msgr2=0x558d52517b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.610+0000 7f6f3ba63740 1 -- 172.21.15.5:0/1689724909 shutdown_connections 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.610+0000 7f6f3ba63740 1 -- 172.21.15.5:0/1689724909 wait complete. 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.610+0000 7f6f3ba63740 1 -- start start 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.611+0000 7f6f3ba63740 1 Processor -- start 2024-09-19T06:18:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.611+0000 7f6f3ba63740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250e4a0 0x558d5251d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.611+0000 7f6f3ba63740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250edc0 0x558d5251d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.611+0000 7f6f3ba63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d52519680 0x558d5251dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.611+0000 7f6f327fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250edc0 0x558d5251d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.611+0000 7f6f327fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250edc0 0x558d5251d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42514/0 (socket says 172.21.15.5:42514) 2024-09-19T06:18:40.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.611+0000 7f6f327fc640 1 -- 172.21.15.5:0/4287839692 learned_addr learned my addr 172.21.15.5:0/4287839692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:40.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.611+0000 7f6f3ba63740 1 -- 172.21.15.5:0/4287839692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f340719c0 con 0x558d52519680 2024-09-19T06:18:40.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.612+0000 7f6f3ba63740 1 -- 172.21.15.5:0/4287839692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34071230 con 0x558d5250e4a0 2024-09-19T06:18:40.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.612+0000 7f6f3ba63740 1 -- 172.21.15.5:0/4287839692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3406e3a0 con 0x558d5250edc0 2024-09-19T06:18:40.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.612+0000 7f6f3ba63740 10 client.0 _ll_register_callbacks cb 0x7f6f2c009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:40.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.612+0000 7f6f337fe640 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d52519680 0x558d5251dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.612+0000 7f6f32ffd640 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250e4a0 0x558d5251d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.612+0000 7f6f327fc640 1 -- 172.21.15.5:0/4287839692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250e4a0 msgr2=0x558d5251d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.612+0000 7f6f327fc640 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250e4a0 0x558d5251d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.612+0000 7f6f327fc640 1 -- 172.21.15.5:0/4287839692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d52519680 msgr2=0x558d5251dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.612+0000 7f6f327fc640 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d52519680 0x558d5251dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.612+0000 7f6f327fc640 1 -- 172.21.15.5:0/4287839692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d52519390 con 0x558d5250edc0 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.612+0000 7f6f327fc640 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250edc0 0x558d5251d960 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x558d5250bdb0 tx=0x7f6f280041c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b46d8314b1be6956 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.612+0000 7f6f16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.613+0000 7f6f16ff5640 1 -- 172.21.15.5:0/4287839692 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28040070 con 0x558d5250edc0 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.613+0000 7f6f16ff5640 1 -- 172.21.15.5:0/4287839692 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7f6f280316b0 con 0x558d5250edc0 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.613+0000 7f6f3ba63740 1 -- 172.21.15.5:0/4287839692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f2c02b070 con 0x558d5250edc0 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.613+0000 7f6f16ff5640 1 -- 172.21.15.5:0/4287839692 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6f28035040 con 0x558d5250edc0 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.613+0000 7f6f3ba63740 10 client.28801 fetch_fsmap learned FSMap version 47 2024-09-19T06:18:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.613+0000 7f6f3ba63740 1 -- 172.21.15.5:0/4287839692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558d525a4fb0 con 0x558d5250edc0 2024-09-19T06:18:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.613+0000 7f6f16ff5640 1 -- 172.21.15.5:0/4287839692 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f280044d0 con 0x558d5250edc0 2024-09-19T06:18:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.614+0000 7f6f16ff5640 1 -- 172.21.15.5:0/4287839692 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 47) ==== 8406+0+0 (secure 0 0 0) 0x7f6f28046920 con 0x558d5250edc0 2024-09-19T06:18:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.614+0000 7f6f3ba63740 10 client.28801 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:18:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.614+0000 7f6f3ba63740 10 client.28801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.614+0000 7f6f3ba63740 20 client.28801 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.614+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d525a53b0 0x558d525a77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.614+0000 7f6f3ba63740 4 client.28801 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:18:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.614+0000 7f6f3ba63740 1 -- 172.21.15.5:0/4287839692 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f6f2c02b070 con 0x558d525a53b0 2024-09-19T06:18:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.614+0000 7f6f32ffd640 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d525a53b0 0x558d525a77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.614+0000 7f6f16ff5640 10 client.28801 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.615+0000 7f6f32ffd640 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d525a53b0 0x558d525a77a0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71e47178521b3367 server_cookie=18d3dc1796de0069 in_seq=0 out_seq=0 2024-09-19T06:18:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.615+0000 7f6f16ff5640 1 -- 172.21.15.5:0/4287839692 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f6f2c02b070 con 0x558d525a53b0 2024-09-19T06:18:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.615+0000 7f6f16ff5640 10 client.28801 handle_command_reply: tid=0 2024-09-19T06:18:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.615+0000 7f6f3ba63740 1 client.28801 shutdown 2024-09-19T06:18:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.615+0000 7f6f3ba63740 1 -- 172.21.15.5:0/4287839692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250edc0 msgr2=0x558d5251d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.615+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250edc0 0x558d5251d960 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x558d5250bdb0 tx=0x7f6f280041c0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.615+0000 7f6f3ba63740 1 -- 172.21.15.5:0/4287839692 shutdown_connections 2024-09-19T06:18:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.615+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558d525a53b0 0x558d525a77a0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.616+0000 7f6f16ff5640 0 client.28801 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.616+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d52519680 0x558d5251dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.616+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d5250edc0 0x558d5251d960 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.616+0000 7f6f3ba63740 1 --2- 172.21.15.5:0/4287839692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d5250e4a0 0x558d5251d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.616+0000 7f6f3ba63740 1 -- 172.21.15.5:0/4287839692 >> 172.21.15.5:0/4287839692 conn(0x558d5250aa30 msgr2=0x558d52510710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.616+0000 7f6f3ba63740 1 -- 172.21.15.5:0/4287839692 shutdown_connections 2024-09-19T06:18:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.616+0000 7f6f3ba63740 1 -- 172.21.15.5:0/4287839692 wait complete. 2024-09-19T06:18:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.616+0000 7f6f3ba63740 20 client.28801 trim_cache size 0 max 16384 2024-09-19T06:18:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:18:40.621 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:18:40.621 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:18:40.622 INFO:teuthology.orchestra.run.smithi005.stdout: "ca0b287f-9bc7-4d7a-a8af-b92b98bfbccf": { 2024-09-19T06:18:40.622 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:18:40.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "ca0b287f-9bc7-4d7a-a8af-b92b98bfbccf", 2024-09-19T06:18:40.622 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:18:40.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:18:40.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:18:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:18:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.617+0000 7f6f3968a640 1 -- 172.21.15.5:0/1140389336 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6f1c0a3450 msgr2=0x7f6f1c0a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.617+0000 7f6f3968a640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6f1c0a3450 0x7f6f1c0a5910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6f280096c0 tx=0x7f6f2802a480 comp rx=0 tx=0).stop 2024-09-19T06:18:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.617+0000 7f6f3968a640 1 -- 172.21.15.5:0/1140389336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34100e20 msgr2=0x7f6f341a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.617+0000 7f6f3968a640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34100e20 0x7f6f341a05e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400d950 tx=0x7f6f2400de10 comp rx=0 tx=0).stop 2024-09-19T06:18:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.617+0000 7f6f3968a640 1 -- 172.21.15.5:0/1140389336 shutdown_connections 2024-09-19T06:18:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.617+0000 7f6f3968a640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f3419a6b0 0x7f6f3419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.617+0000 7f6f3968a640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6f1c0a3450 0x7f6f1c0a5910 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.617+0000 7f6f3968a640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f34102ef0 0x7f6f341a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.617+0000 7f6f3968a640 1 --2- 172.21.15.5:0/1140389336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34100e20 0x7f6f341a05e0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.617+0000 7f6f3968a640 1 -- 172.21.15.5:0/1140389336 >> 172.21.15.5:0/1140389336 conn(0x7f6f340fc7d0 msgr2=0x7f6f341067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.617+0000 7f6f3968a640 1 -- 172.21.15.5:0/1140389336 shutdown_connections 2024-09-19T06:18:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:18:40.617+0000 7f6f3968a640 1 -- 172.21.15.5:0/1140389336 wait complete. 2024-09-19T06:18:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f2831ffb640 1 -- 172.21.15.5:0/1549708808 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608e9375d70 con 0x5608e9498660 2024-09-19T06:18:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f2831ffb640 10 client.28853 handle_command_reply: tid=0 2024-09-19T06:18:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f28536c6740 1 client.28853 shutdown 2024-09-19T06:18:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f28536c6740 1 -- 172.21.15.5:0/1549708808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 msgr2=0x5608e941e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f28536c6740 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e941e300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2844034040 tx=0x7f2844030b40 comp rx=0 tx=0).stop 2024-09-19T06:18:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f28536c6740 1 -- 172.21.15.5:0/1549708808 shutdown_connections 2024-09-19T06:18:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f28536c6740 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5608e9498660 0x5608e949aa80 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f28536c6740 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608e94190c0 0x5608e941e300 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f28536c6740 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e940a290 0x5608e9421700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f28536c6740 1 --2- 172.21.15.5:0/1549708808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e9409970 0x5608e94211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f28536c6740 1 -- 172.21.15.5:0/1549708808 >> 172.21.15.5:0/1549708808 conn(0x5608e9406060 msgr2=0x5608e94051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f2831ffb640 0 client.28853 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f28536c6740 1 -- 172.21.15.5:0/1549708808 shutdown_connections 2024-09-19T06:18:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f28536c6740 1 -- 172.21.15.5:0/1549708808 wait complete. 2024-09-19T06:18:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.627+0000 7f28536c6740 20 client.28853 trim_cache size 0 max 16384 2024-09-19T06:18:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:18:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:18:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:18:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:18:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.628+0000 7f285092a640 1 -- 172.21.15.5:0/165943577 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f28340a3480 msgr2=0x7f28340a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.628+0000 7f285092a640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f28340a3480 0x7f28340a5940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2844009560 tx=0x7f284402abc0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.628+0000 7f285092a640 1 -- 172.21.15.5:0/165943577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f284c101340 msgr2=0x7f284c199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.628+0000 7f285092a640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f284c101340 0x7f284c199470 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f284000d950 tx=0x7f284000dd50 comp rx=0 tx=0).stop 2024-09-19T06:18:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.628+0000 7f285092a640 1 -- 172.21.15.5:0/165943577 shutdown_connections 2024-09-19T06:18:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.628+0000 7f285092a640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284c1069b0 0x7f284c1a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.628+0000 7f285092a640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f284c101340 0x7f284c199470 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.628+0000 7f285092a640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f28340a3480 0x7f28340a5940 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.628+0000 7f285092a640 1 --2- 172.21.15.5:0/165943577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f284c1009a0 0x7f284c198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.628+0000 7f285092a640 1 -- 172.21.15.5:0/165943577 >> 172.21.15.5:0/165943577 conn(0x7f284c0779a0 msgr2=0x7f284c103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.628+0000 7f285092a640 1 -- 172.21.15.5:0/165943577 shutdown_connections 2024-09-19T06:18:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:40.628+0000 7f285092a640 1 -- 172.21.15.5:0/165943577 wait complete. 2024-09-19T06:18:40.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:40 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:18:40.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:40 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:18:40.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:40 smithi005 ceph-mon[30955]: pgmap v481: 97 pgs: 97 active+clean; 542 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 242 op/s 2024-09-19T06:18:40.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:40 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:40.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:40 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:40 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:18:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:40 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:18:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:40 smithi157 ceph-mon[35346]: pgmap v481: 97 pgs: 97 active+clean; 542 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 242 op/s 2024-09-19T06:18:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:40 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:40 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:40 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:18:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:40 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:18:40.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:40 smithi049 ceph-mon[34640]: pgmap v481: 97 pgs: 97 active+clean; 542 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 242 op/s 2024-09-19T06:18:40.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:40 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:40.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:40 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:18:41.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 20946 5.74 MB/sec execute 5 sec latency 81.693 ms 2024-09-19T06:18:42.014 INFO:tasks.cephfs.filesystem:scrub status for tag:ca0b287f-9bc7-4d7a-a8af-b92b98bfbccf - {'path': '/', 'tag': 'ca0b287f-9bc7-4d7a-a8af-b92b98bfbccf', 'options': 'recursive,force'} 2024-09-19T06:18:42.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21096 5.96 MB/sec execute 6 sec latency 70.481 ms 2024-09-19T06:18:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:42 smithi157 ceph-mon[35346]: pgmap v482: 97 pgs: 97 active+clean; 542 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 242 op/s 2024-09-19T06:18:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:42 smithi005 ceph-mon[30955]: pgmap v482: 97 pgs: 97 active+clean; 542 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 242 op/s 2024-09-19T06:18:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:42 smithi049 ceph-mon[34640]: pgmap v482: 97 pgs: 97 active+clean; 542 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 242 op/s 2024-09-19T06:18:43.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21282 5.97 MB/sec execute 7 sec latency 65.385 ms 2024-09-19T06:18:44.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21485 6.77 MB/sec execute 8 sec latency 87.169 ms 2024-09-19T06:18:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:44 smithi157 ceph-mon[35346]: pgmap v483: 97 pgs: 97 active+clean; 543 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 450 op/s 2024-09-19T06:18:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:44 smithi005 ceph-mon[30955]: pgmap v483: 97 pgs: 97 active+clean; 543 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 450 op/s 2024-09-19T06:18:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:44 smithi049 ceph-mon[34640]: pgmap v483: 97 pgs: 97 active+clean; 543 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 450 op/s 2024-09-19T06:18:45.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21658 6.72 MB/sec execute 9 sec latency 65.129 ms 2024-09-19T06:18:45.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:45 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:18:45.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:45 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:18:45.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:45 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:18:46.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21831 6.41 MB/sec execute 10 sec latency 55.101 ms 2024-09-19T06:18:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:46 smithi157 ceph-mon[35346]: pgmap v484: 97 pgs: 97 active+clean; 543 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 331 op/s 2024-09-19T06:18:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:46 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:18:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:46 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:18:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:46 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:18:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:46 smithi005 ceph-mon[30955]: pgmap v484: 97 pgs: 97 active+clean; 543 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 331 op/s 2024-09-19T06:18:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:46 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:18:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:46 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:18:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:46 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:18:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:46 smithi049 ceph-mon[34640]: pgmap v484: 97 pgs: 97 active+clean; 543 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 331 op/s 2024-09-19T06:18:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:46 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:18:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:46 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:18:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:46 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:18:47.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 22015 6.83 MB/sec execute 11 sec latency 63.218 ms 2024-09-19T06:18:48.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 22205 7.07 MB/sec execute 12 sec latency 73.197 ms 2024-09-19T06:18:48.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:48 smithi157 ceph-mon[35346]: pgmap v485: 97 pgs: 97 active+clean; 542 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.4 MiB/s wr, 463 op/s 2024-09-19T06:18:48.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:48 smithi005 ceph-mon[30955]: pgmap v485: 97 pgs: 97 active+clean; 542 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.4 MiB/s wr, 463 op/s 2024-09-19T06:18:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:48 smithi049 ceph-mon[34640]: pgmap v485: 97 pgs: 97 active+clean; 542 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.4 MiB/s wr, 463 op/s 2024-09-19T06:18:49.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 22399 7.37 MB/sec execute 13 sec latency 61.196 ms 2024-09-19T06:18:49.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:49 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:18:49.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:49 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:18:49.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:49 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:18:49.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:49 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:18:49.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:49 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:18:49.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:49 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:18:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:49 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:18:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:49 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:18:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:49 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:18:50.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 22564 7.51 MB/sec execute 14 sec latency 122.756 ms 2024-09-19T06:18:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:50 smithi157 ceph-mon[35346]: pgmap v486: 97 pgs: 97 active+clean; 542 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 340 op/s 2024-09-19T06:18:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:50 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:18:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:50 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:18:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:50 smithi005 ceph-mon[30955]: pgmap v486: 97 pgs: 97 active+clean; 542 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 340 op/s 2024-09-19T06:18:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:50 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:18:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:50 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:18:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:50 smithi049 ceph-mon[34640]: pgmap v486: 97 pgs: 97 active+clean; 542 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 340 op/s 2024-09-19T06:18:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:50 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:18:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:50 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:18:51.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 22774 7.99 MB/sec execute 15 sec latency 100.102 ms 2024-09-19T06:18:51.772 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:51 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:18:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:51 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:18:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:51 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:18:52.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 22998 8.35 MB/sec execute 16 sec latency 57.142 ms 2024-09-19T06:18:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:52 smithi157 ceph-mon[35346]: pgmap v487: 97 pgs: 97 active+clean; 542 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 340 op/s 2024-09-19T06:18:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:52 smithi005 ceph-mon[30955]: pgmap v487: 97 pgs: 97 active+clean; 542 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 340 op/s 2024-09-19T06:18:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:52 smithi049 ceph-mon[34640]: pgmap v487: 97 pgs: 97 active+clean; 542 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 340 op/s 2024-09-19T06:18:53.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 23140 8.32 MB/sec execute 17 sec latency 106.688 ms 2024-09-19T06:18:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.756+0000 7fa91d88d640 1 -- 172.21.15.5:0/725730266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181009a0 msgr2=0x7fa918100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.756+0000 7fa91d88d640 1 --2- 172.21.15.5:0/725730266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181009a0 0x7fa918100e00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009890 tx=0x7fa90c02a360 comp rx=0 tx=0).stop 2024-09-19T06:18:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.756+0000 7fa91d88d640 1 -- 172.21.15.5:0/725730266 shutdown_connections 2024-09-19T06:18:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.756+0000 7fa91d88d640 1 --2- 172.21.15.5:0/725730266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa918101340 0x7fa91810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.756+0000 7fa91d88d640 1 --2- 172.21.15.5:0/725730266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181009a0 0x7fa918100e00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.756+0000 7fa91d88d640 1 --2- 172.21.15.5:0/725730266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9181069b0 0x7fa918106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.756+0000 7fa91d88d640 1 -- 172.21.15.5:0/725730266 >> 172.21.15.5:0/725730266 conn(0x7fa9180779a0 msgr2=0x7fa9180fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.756+0000 7fa91d88d640 1 -- 172.21.15.5:0/725730266 shutdown_connections 2024-09-19T06:18:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.756+0000 7fa91d88d640 1 -- 172.21.15.5:0/725730266 wait complete. 2024-09-19T06:18:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.757+0000 7fa91d88d640 1 Processor -- start 2024-09-19T06:18:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.757+0000 7fa91d88d640 1 -- start start 2024-09-19T06:18:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa91d88d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181009a0 0x7fa91819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa91d88d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa918101340 0x7fa91819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa91d88d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9181069b0 0x7fa91819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa91d88d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa91810fc40 con 0x7fa9181069b0 2024-09-19T06:18:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa91d88d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa91810fac0 con 0x7fa918101340 2024-09-19T06:18:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa91d88d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa91810fdc0 con 0x7fa9181009a0 2024-09-19T06:18:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa916ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181009a0 0x7fa91819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa916ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181009a0 0x7fa91819afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40322/0 (socket says 172.21.15.5:40322) 2024-09-19T06:18:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa916ffd640 1 -- 172.21.15.5:0/3628368102 learned_addr learned my addr 172.21.15.5:0/3628368102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa9177fe640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9181069b0 0x7fa91819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa9167fc640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa918101340 0x7fa91819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa916ffd640 1 -- 172.21.15.5:0/3628368102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa918101340 msgr2=0x7fa91819b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa916ffd640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa918101340 0x7fa91819b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa916ffd640 1 -- 172.21.15.5:0/3628368102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9181069b0 msgr2=0x7fa91819fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.758+0000 7fa916ffd640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9181069b0 0x7fa91819fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.759+0000 7fa916ffd640 1 -- 172.21.15.5:0/3628368102 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9181a0570 con 0x7fa9181009a0 2024-09-19T06:18:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.759+0000 7fa916ffd640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181009a0 0x7fa91819afd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa90800e9c0 tx=0x7fa90800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.759+0000 7fa91c88b640 1 -- 172.21.15.5:0/3628368102 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90800ccc0 con 0x7fa9181009a0 2024-09-19T06:18:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.759+0000 7fa91c88b640 1 -- 172.21.15.5:0/3628368102 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9080054d0 con 0x7fa9181009a0 2024-09-19T06:18:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.759+0000 7fa91c88b640 1 -- 172.21.15.5:0/3628368102 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908010430 con 0x7fa9181009a0 2024-09-19T06:18:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.759+0000 7fa91d88d640 1 -- 172.21.15.5:0/3628368102 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9181a0860 con 0x7fa9181009a0 2024-09-19T06:18:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.759+0000 7fa91d88d640 1 -- 172.21.15.5:0/3628368102 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9181a6ef0 con 0x7fa9181009a0 2024-09-19T06:18:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.762+0000 7fa92062c740 1 Processor -- start 2024-09-19T06:18:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.762+0000 7fa91c88b640 1 -- 172.21.15.5:0/3628368102 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa9080105d0 con 0x7fa9181009a0 2024-09-19T06:18:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.763+0000 7fa92062c740 1 -- start start 2024-09-19T06:18:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.763+0000 7fa92062c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1853df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.763+0000 7fa92062c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd1854330 0x561dd1866f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.763+0000 7fa92062c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd18674c0 0x561dd18698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.763+0000 7fa92062c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561dd17b82f0 con 0x561dd1853a10 2024-09-19T06:18:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.763+0000 7fa92062c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561dd1760270 con 0x561dd18674c0 2024-09-19T06:18:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.763+0000 7fa92062c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561dd17b42b0 con 0x561dd1854330 2024-09-19T06:18:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.763+0000 7fa9177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1853df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.763+0000 7fa9177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1853df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51108/0 (socket says 172.21.15.5:51108) 2024-09-19T06:18:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.763+0000 7fa9177fe640 1 -- 172.21.15.5:0/2821931961 learned_addr learned my addr 172.21.15.5:0/2821931961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.763+0000 7fa916ffd640 1 --2- 172.21.15.5:0/2821931961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd1854330 0x561dd1866f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.763+0000 7fa9167fc640 1 --2- 172.21.15.5:0/2821931961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd18674c0 0x561dd18698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa9177fe640 1 -- 172.21.15.5:0/2821931961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd1854330 msgr2=0x561dd1866f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa9177fe640 1 --2- 172.21.15.5:0/2821931961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd1854330 0x561dd1866f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa9177fe640 1 -- 172.21.15.5:0/2821931961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd18674c0 msgr2=0x561dd18698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa9177fe640 1 --2- 172.21.15.5:0/2821931961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd18674c0 0x561dd18698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa9177fe640 1 -- 172.21.15.5:0/2821931961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dd175f340 con 0x561dd1853a10 2024-09-19T06:18:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa9167fc640 1 --2- 172.21.15.5:0/2821931961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd18674c0 0x561dd18698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa91c88b640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa9000a3430 0x7fa9000a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa9177fe640 1 --2- 172.21.15.5:0/2821931961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1853df0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa910009fd0 tx=0x7fa9100118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a828f42910ca4888 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa916ffd640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa9000a3430 0x7fa9000a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa91c88b640 1 -- 172.21.15.5:0/3628368102 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fa9080cfeb0 con 0x7fa9181009a0 2024-09-19T06:18:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa8fdffb640 1 -- 172.21.15.5:0/2821931961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910004d20 con 0x561dd1853a10 2024-09-19T06:18:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa8fdffb640 1 -- 172.21.15.5:0/2821931961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa910004050 con 0x561dd1853a10 2024-09-19T06:18:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.764+0000 7fa8fdffb640 1 -- 172.21.15.5:0/2821931961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910005280 con 0x561dd1853a10 2024-09-19T06:18:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.765+0000 7fa92062c740 1 -- 172.21.15.5:0/2821931961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 msgr2=0x561dd1853df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.765+0000 7fa92062c740 1 --2- 172.21.15.5:0/2821931961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1853df0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa910009fd0 tx=0x7fa9100118d0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.765+0000 7fa916ffd640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa9000a3430 0x7fa9000a58f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa9080cd5e0 tx=0x7fa908005880 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.765+0000 7fa92062c740 1 -- 172.21.15.5:0/2821931961 shutdown_connections 2024-09-19T06:18:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.765+0000 7fa92062c740 1 --2- 172.21.15.5:0/2821931961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd18674c0 0x561dd18698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.765+0000 7fa92062c740 1 --2- 172.21.15.5:0/2821931961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd1854330 0x561dd1866f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.765+0000 7fa92062c740 1 --2- 172.21.15.5:0/2821931961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1853df0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.765+0000 7fa92062c740 1 -- 172.21.15.5:0/2821931961 >> 172.21.15.5:0/2821931961 conn(0x561dd1850060 msgr2=0x561dd1850440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.765+0000 7fa92062c740 1 -- 172.21.15.5:0/2821931961 shutdown_connections 2024-09-19T06:18:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.765+0000 7fa92062c740 1 -- 172.21.15.5:0/2821931961 wait complete. 2024-09-19T06:18:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.766+0000 7fa92062c740 1 -- start start 2024-09-19T06:18:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.766+0000 7fa92062c740 1 Processor -- start 2024-09-19T06:18:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa92062c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1864420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa92062c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd1854330 0x561dd1864960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa9177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1864420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa9177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1864420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51120/0 (socket says 172.21.15.5:51120) 2024-09-19T06:18:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa9177fe640 1 -- 172.21.15.5:0/1924158891 learned_addr learned my addr 172.21.15.5:0/1924158891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa92062c740 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd18674c0 0x561dd1864ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa92062c740 1 -- 172.21.15.5:0/1924158891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561dd186a090 con 0x561dd1853a10 2024-09-19T06:18:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa92062c740 1 -- 172.21.15.5:0/1924158891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561dd1741290 con 0x561dd18674c0 2024-09-19T06:18:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa92062c740 1 -- 172.21.15.5:0/1924158891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561dd186a210 con 0x561dd1854330 2024-09-19T06:18:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa92062c740 10 client.0 _ll_register_callbacks cb 0x561dd1772140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa9167fc640 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd1854330 0x561dd1864960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa9177fe640 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd18674c0 0x561dd1864ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa9167fc640 1 -- 172.21.15.5:0/1924158891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd18674c0 msgr2=0x561dd1864ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa9167fc640 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd18674c0 0x561dd1864ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa9167fc640 1 -- 172.21.15.5:0/1924158891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 msgr2=0x561dd1864420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa9167fc640 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1864420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa9167fc640 1 -- 172.21.15.5:0/1924158891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dd14cb210 con 0x561dd1854330 2024-09-19T06:18:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa9177fe640 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd18674c0 0x561dd1864ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa9167fc640 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd1854330 0x561dd1864960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa90c02a880 tx=0x7fa90c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c827227361f46daf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.767+0000 7fa8f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.768+0000 7fa8f6ffd640 1 -- 172.21.15.5:0/1924158891 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c041020 con 0x561dd1854330 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.768+0000 7fa8f6ffd640 1 -- 172.21.15.5:0/1924158891 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fa90c030340 con 0x561dd1854330 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.768+0000 7fa92062c740 1 -- 172.21.15.5:0/1924158891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa910004050 con 0x561dd1854330 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.768+0000 7fa8f6ffd640 1 -- 172.21.15.5:0/1924158891 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa90c04c050 con 0x561dd1854330 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.768+0000 7fa92062c740 10 client.28813 fetch_fsmap learned FSMap version 48 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.768+0000 7fa92062c740 1 -- 172.21.15.5:0/1924158891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561dd1597620 con 0x561dd1854330 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.768+0000 7fa8f6ffd640 1 -- 172.21.15.5:0/1924158891 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa910004050 con 0x561dd1854330 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.769+0000 7fa8f6ffd640 1 -- 172.21.15.5:0/1924158891 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7fa90c003a00 con 0x561dd1854330 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.769+0000 7fa92062c740 10 client.28813 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.769+0000 7fa92062c740 10 client.28813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.769+0000 7fa92062c740 20 client.28813 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.769+0000 7fa92062c740 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561dd18774e0 0x561dd1879920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.769+0000 7fa92062c740 4 client.28813 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:18:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.769+0000 7fa92062c740 1 -- 172.21.15.5:0/1924158891 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dd17c5a20 con 0x561dd18774e0 2024-09-19T06:18:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.769+0000 7fa9177fe640 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561dd18774e0 0x561dd1879920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.769+0000 7fa9177fe640 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561dd18774e0 0x561dd1879920 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad23b2a5d0bf54c6 server_cookie=8b6035d1055bdbec in_seq=0 out_seq=0 2024-09-19T06:18:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.769+0000 7fa8f6ffd640 10 client.28813 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa8f6ffd640 1 -- 172.21.15.5:0/1924158891 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561dd17c5a20 con 0x561dd18774e0 2024-09-19T06:18:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa8f6ffd640 10 client.28813 handle_command_reply: tid=0 2024-09-19T06:18:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa92062c740 1 client.28813 shutdown 2024-09-19T06:18:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa92062c740 1 -- 172.21.15.5:0/1924158891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd1854330 msgr2=0x561dd1864960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa92062c740 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd1854330 0x561dd1864960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa90c02a880 tx=0x7fa90c004270 comp rx=0 tx=0).stop 2024-09-19T06:18:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa92062c740 1 -- 172.21.15.5:0/1924158891 shutdown_connections 2024-09-19T06:18:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa92062c740 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561dd18774e0 0x561dd1879920 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa92062c740 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd18674c0 0x561dd1864ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa92062c740 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd1854330 0x561dd1864960 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa92062c740 1 --2- 172.21.15.5:0/1924158891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1864420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa92062c740 1 -- 172.21.15.5:0/1924158891 >> 172.21.15.5:0/1924158891 conn(0x561dd1850060 msgr2=0x561dd1855d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa8f6ffd640 0 client.28813 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa92062c740 1 -- 172.21.15.5:0/1924158891 shutdown_connections 2024-09-19T06:18:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.770+0000 7fa92062c740 1 -- 172.21.15.5:0/1924158891 wait complete. 2024-09-19T06:18:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.771+0000 7fa92062c740 20 client.28813 trim_cache size 0 max 16384 2024-09-19T06:18:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa92062c740 1 Processor -- start 2024-09-19T06:18:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa92062c740 1 -- start start 2024-09-19T06:18:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa92062c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1862cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa92062c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd1854330 0x561dd187f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa92062c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd18674c0 0x561dd187f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa92062c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181ab690 con 0x561dd1853a10 2024-09-19T06:18:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa92062c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181ad380 con 0x561dd1854330 2024-09-19T06:18:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa92062c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181abe20 con 0x561dd18674c0 2024-09-19T06:18:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa9177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1862cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa9177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1862cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51132/0 (socket says 172.21.15.5:51132) 2024-09-19T06:18:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa9167fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd1854330 0x561dd187f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa9177fe640 1 -- 172.21.15.5:0/2169959662 learned_addr learned my addr 172.21.15.5:0/2169959662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.796+0000 7fa9177fe640 1 --2- 172.21.15.5:0/2169959662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd18674c0 0x561dd187f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.797+0000 7fa9177fe640 1 -- 172.21.15.5:0/2169959662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd18674c0 msgr2=0x561dd187f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.797+0000 7fa9177fe640 1 --2- 172.21.15.5:0/2169959662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd18674c0 0x561dd187f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.797+0000 7fa9177fe640 1 -- 172.21.15.5:0/2169959662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd1854330 msgr2=0x561dd187f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.797+0000 7fa9177fe640 1 --2- 172.21.15.5:0/2169959662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd1854330 0x561dd187f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.797+0000 7fa9177fe640 1 -- 172.21.15.5:0/2169959662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dd1879fa0 con 0x561dd1853a10 2024-09-19T06:18:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.797+0000 7fa9177fe640 1 --2- 172.21.15.5:0/2169959662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1862cd0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa910000f80 tx=0x7fa910003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ed6d7a69aac2e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.797+0000 7fa8fd7fa640 1 -- 172.21.15.5:0/2169959662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9100049e0 con 0x561dd1853a10 2024-09-19T06:18:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.797+0000 7fa8fd7fa640 1 -- 172.21.15.5:0/2169959662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa910004500 con 0x561dd1853a10 2024-09-19T06:18:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.797+0000 7fa8fd7fa640 1 -- 172.21.15.5:0/2169959662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910005310 con 0x561dd1853a10 2024-09-19T06:18:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.798+0000 7fa92062c740 1 -- 172.21.15.5:0/2169959662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 msgr2=0x561dd1862cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.798+0000 7fa92062c740 1 --2- 172.21.15.5:0/2169959662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1862cd0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa910000f80 tx=0x7fa910003f50 comp rx=0 tx=0).stop 2024-09-19T06:18:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.798+0000 7fa92062c740 1 -- 172.21.15.5:0/2169959662 shutdown_connections 2024-09-19T06:18:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.798+0000 7fa92062c740 1 --2- 172.21.15.5:0/2169959662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd18674c0 0x561dd187f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.798+0000 7fa92062c740 1 --2- 172.21.15.5:0/2169959662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd1854330 0x561dd187f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.798+0000 7fa92062c740 1 --2- 172.21.15.5:0/2169959662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd1862cd0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.798+0000 7fa92062c740 1 -- 172.21.15.5:0/2169959662 >> 172.21.15.5:0/2169959662 conn(0x561dd1850060 msgr2=0x561dd18664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.798+0000 7fa92062c740 1 -- 172.21.15.5:0/2169959662 shutdown_connections 2024-09-19T06:18:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.798+0000 7fa92062c740 1 -- 172.21.15.5:0/2169959662 wait complete. 2024-09-19T06:18:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.798+0000 7fa92062c740 1 -- start start 2024-09-19T06:18:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa92062c740 1 Processor -- start 2024-09-19T06:18:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa92062c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd186f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa92062c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd1854330 0x561dd186fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa92062c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd18674c0 0x561dd186c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa92062c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181accb0 con 0x561dd1853a10 2024-09-19T06:18:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa92062c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181ac520 con 0x561dd1854330 2024-09-19T06:18:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa92062c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181a9690 con 0x561dd18674c0 2024-09-19T06:18:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa92062c740 10 client.0 _ll_register_callbacks cb 0x7fa9100093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:18:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa9177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd186f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa9177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd186f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51134/0 (socket says 172.21.15.5:51134) 2024-09-19T06:18:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa9177fe640 1 -- 172.21.15.5:0/95616194 learned_addr learned my addr 172.21.15.5:0/95616194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:18:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa9177fe640 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd18674c0 0x561dd186c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.799+0000 7fa9167fc640 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd1854330 0x561dd186fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.800+0000 7fa9177fe640 1 -- 172.21.15.5:0/95616194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd18674c0 msgr2=0x561dd186c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.800+0000 7fa9177fe640 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd18674c0 0x561dd186c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.800+0000 7fa9177fe640 1 -- 172.21.15.5:0/95616194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd1854330 msgr2=0x561dd186fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:18:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.800+0000 7fa9177fe640 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd1854330 0x561dd186fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.800+0000 7fa9177fe640 1 -- 172.21.15.5:0/95616194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dd186ca60 con 0x561dd1853a10 2024-09-19T06:18:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.800+0000 7fa9167fc640 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd1854330 0x561dd186fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:18:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.800+0000 7fa9177fe640 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd186f4d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fa910006010 tx=0x7fa9100021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f252e1f07965a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:18:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.800+0000 7fa8f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:18:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.800+0000 7fa8f6ffd640 1 -- 172.21.15.5:0/95616194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9100049e0 con 0x561dd1853a10 2024-09-19T06:18:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.800+0000 7fa8f6ffd640 1 -- 172.21.15.5:0/95616194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8550+0+0 (secure 0 0 0) 0x7fa910004b80 con 0x561dd1853a10 2024-09-19T06:18:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.800+0000 7fa92062c740 1 -- 172.21.15.5:0/95616194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa910004500 con 0x561dd1853a10 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.800+0000 7fa8f6ffd640 1 -- 172.21.15.5:0/95616194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa910004d80 con 0x561dd1853a10 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.801+0000 7fa92062c740 10 client.19491 fetch_fsmap learned FSMap version 48 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.801+0000 7fa92062c740 1 -- 172.21.15.5:0/95616194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561dd18e2aa0 con 0x561dd1853a10 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.801+0000 7fa8f6ffd640 1 -- 172.21.15.5:0/95616194 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa910004500 con 0x561dd1853a10 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.801+0000 7fa8f6ffd640 1 -- 172.21.15.5:0/95616194 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7fa910030040 con 0x561dd1853a10 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.801+0000 7fa92062c740 10 client.19491 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.801+0000 7fa92062c740 10 client.19491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.801+0000 7fa92062c740 20 client.19491 populate_metadata read hostname 'smithi005' 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.801+0000 7fa92062c740 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561dd18e3060 0x561dd18e5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.801+0000 7fa92062c740 4 client.19491 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.801+0000 7fa92062c740 1 -- 172.21.15.5:0/95616194 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561dd17bfd70 con 0x561dd18e3060 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.801+0000 7fa9167fc640 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561dd18e3060 0x561dd18e5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:18:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.802+0000 7fa9167fc640 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561dd18e3060 0x561dd18e5480 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ebc4e225e6d814 server_cookie=9cf00f3e3533d017 in_seq=0 out_seq=0 2024-09-19T06:18:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.802+0000 7fa8f6ffd640 10 client.19491 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa8f6ffd640 1 -- 172.21.15.5:0/95616194 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561dd17bfd70 con 0x561dd18e3060 2024-09-19T06:18:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa8f6ffd640 10 client.19491 handle_command_reply: tid=0 2024-09-19T06:18:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa92062c740 1 client.19491 shutdown 2024-09-19T06:18:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa92062c740 1 -- 172.21.15.5:0/95616194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 msgr2=0x561dd186f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa92062c740 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd186f4d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fa910006010 tx=0x7fa9100021d0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa92062c740 1 -- 172.21.15.5:0/95616194 shutdown_connections 2024-09-19T06:18:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa92062c740 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561dd18e3060 0x561dd18e5480 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa92062c740 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561dd18674c0 0x561dd186c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa92062c740 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dd1854330 0x561dd186fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa8f6ffd640 0 client.19491 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:18:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa92062c740 1 --2- 172.21.15.5:0/95616194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dd1853a10 0x561dd186f4d0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa92062c740 1 -- 172.21.15.5:0/95616194 >> 172.21.15.5:0/95616194 conn(0x561dd1850060 msgr2=0x561dd184e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa92062c740 1 -- 172.21.15.5:0/95616194 shutdown_connections 2024-09-19T06:18:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa92062c740 1 -- 172.21.15.5:0/95616194 wait complete. 2024-09-19T06:18:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.877+0000 7fa92062c740 20 client.19491 trim_cache size 0 max 16384 2024-09-19T06:18:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:18:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:18:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:18:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:18:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.878+0000 7fa91d88d640 1 -- 172.21.15.5:0/3628368102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa9000a3430 msgr2=0x7fa9000a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.878+0000 7fa91d88d640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa9000a3430 0x7fa9000a58f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa9080cd5e0 tx=0x7fa908005880 comp rx=0 tx=0).stop 2024-09-19T06:18:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.878+0000 7fa91d88d640 1 -- 172.21.15.5:0/3628368102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181009a0 msgr2=0x7fa91819afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:18:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.878+0000 7fa91d88d640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181009a0 0x7fa91819afd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa90800e9c0 tx=0x7fa90800ee80 comp rx=0 tx=0).stop 2024-09-19T06:18:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.878+0000 7fa91d88d640 1 -- 172.21.15.5:0/3628368102 shutdown_connections 2024-09-19T06:18:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.878+0000 7fa91d88d640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9181069b0 0x7fa91819fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.878+0000 7fa91d88d640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa918101340 0x7fa91819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.878+0000 7fa91d88d640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa9000a3430 0x7fa9000a58f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.878+0000 7fa91d88d640 1 --2- 172.21.15.5:0/3628368102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181009a0 0x7fa91819afd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:18:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.878+0000 7fa91d88d640 1 -- 172.21.15.5:0/3628368102 >> 172.21.15.5:0/3628368102 conn(0x7fa9180779a0 msgr2=0x7fa918104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:18:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.878+0000 7fa91d88d640 1 -- 172.21.15.5:0/3628368102 shutdown_connections 2024-09-19T06:18:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:18:53.878+0000 7fa91d88d640 1 -- 172.21.15.5:0/3628368102 wait complete. 2024-09-19T06:18:54.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 23292 8.62 MB/sec execute 18 sec latency 108.623 ms 2024-09-19T06:18:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:54 smithi157 ceph-mon[35346]: pgmap v488: 97 pgs: 97 active+clean; 558 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 454 op/s 2024-09-19T06:18:54.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:54 smithi005 ceph-mon[30955]: pgmap v488: 97 pgs: 97 active+clean; 558 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 454 op/s 2024-09-19T06:18:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:54 smithi049 ceph-mon[34640]: pgmap v488: 97 pgs: 97 active+clean; 558 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 454 op/s 2024-09-19T06:18:55.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 23512 9.29 MB/sec execute 19 sec latency 74.683 ms 2024-09-19T06:18:56.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 23671 9.19 MB/sec execute 20 sec latency 143.116 ms 2024-09-19T06:18:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:56 smithi157 ceph-mon[35346]: pgmap v489: 97 pgs: 97 active+clean; 558 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 245 op/s 2024-09-19T06:18:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:56 smithi005 ceph-mon[30955]: pgmap v489: 97 pgs: 97 active+clean; 558 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 245 op/s 2024-09-19T06:18:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:56 smithi049 ceph-mon[34640]: pgmap v489: 97 pgs: 97 active+clean; 558 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 245 op/s 2024-09-19T06:18:57.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 23812 9.07 MB/sec execute 21 sec latency 54.244 ms 2024-09-19T06:18:58.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 23956 9.03 MB/sec execute 22 sec latency 72.119 ms 2024-09-19T06:18:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:18:58 smithi157 ceph-mon[35346]: pgmap v490: 97 pgs: 97 active+clean; 569 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 385 op/s 2024-09-19T06:18:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:18:58 smithi005 ceph-mon[30955]: pgmap v490: 97 pgs: 97 active+clean; 569 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 385 op/s 2024-09-19T06:18:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:18:58 smithi049 ceph-mon[34640]: pgmap v490: 97 pgs: 97 active+clean; 569 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 385 op/s 2024-09-19T06:18:59.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 24089 8.76 MB/sec execute 23 sec latency 112.816 ms 2024-09-19T06:19:00.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 24264 8.75 MB/sec execute 24 sec latency 63.545 ms 2024-09-19T06:19:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:00 smithi157 ceph-mon[35346]: pgmap v491: 97 pgs: 97 active+clean; 569 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 253 op/s 2024-09-19T06:19:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:00 smithi005 ceph-mon[30955]: pgmap v491: 97 pgs: 97 active+clean; 569 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 253 op/s 2024-09-19T06:19:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:00 smithi049 ceph-mon[34640]: pgmap v491: 97 pgs: 97 active+clean; 569 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 253 op/s 2024-09-19T06:19:01.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 24413 8.53 MB/sec execute 25 sec latency 79.831 ms 2024-09-19T06:19:02.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 24577 8.42 MB/sec execute 26 sec latency 60.344 ms 2024-09-19T06:19:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:02 smithi157 ceph-mon[35346]: pgmap v492: 97 pgs: 97 active+clean; 569 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 253 op/s 2024-09-19T06:19:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:02 smithi005 ceph-mon[30955]: pgmap v492: 97 pgs: 97 active+clean; 569 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 253 op/s 2024-09-19T06:19:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:02 smithi049 ceph-mon[34640]: pgmap v492: 97 pgs: 97 active+clean; 569 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 253 op/s 2024-09-19T06:19:03.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 24764 8.52 MB/sec execute 27 sec latency 52.338 ms 2024-09-19T06:19:04.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 24927 8.55 MB/sec execute 28 sec latency 157.066 ms 2024-09-19T06:19:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:04 smithi157 ceph-mon[35346]: pgmap v493: 97 pgs: 97 active+clean; 570 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 404 op/s 2024-09-19T06:19:04.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:04 smithi005 ceph-mon[30955]: pgmap v493: 97 pgs: 97 active+clean; 570 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 404 op/s 2024-09-19T06:19:04.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:04 smithi049 ceph-mon[34640]: pgmap v493: 97 pgs: 97 active+clean; 570 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 404 op/s 2024-09-19T06:19:05.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 25127 8.62 MB/sec execute 29 sec latency 63.867 ms 2024-09-19T06:19:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:05 smithi157 ceph-mon[35346]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T06:19:05.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:05 smithi005 ceph-mon[30955]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T06:19:05.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:05 smithi049 ceph-mon[34640]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T06:19:06.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 25335 8.77 MB/sec execute 30 sec latency 59.909 ms 2024-09-19T06:19:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:06 smithi157 ceph-mon[35346]: pgmap v495: 97 pgs: 97 active+clean; 570 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 348 op/s 2024-09-19T06:19:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:06 smithi005 ceph-mon[30955]: pgmap v495: 97 pgs: 97 active+clean; 570 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 348 op/s 2024-09-19T06:19:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:06 smithi049 ceph-mon[34640]: pgmap v495: 97 pgs: 97 active+clean; 570 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 348 op/s 2024-09-19T06:19:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.006+0000 7f12f755f640 1 -- 172.21.15.5:0/3953328513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f01035a0 msgr2=0x7f12f010ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.006+0000 7f12f755f640 1 --2- 172.21.15.5:0/3953328513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f01035a0 0x7f12f010ba80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f12e0009860 tx=0x7f12e002a3d0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.006+0000 7f12f755f640 1 -- 172.21.15.5:0/3953328513 shutdown_connections 2024-09-19T06:19:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.006+0000 7f12f755f640 1 --2- 172.21.15.5:0/3953328513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f010c0a0 0x7f12f010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.006+0000 7f12f755f640 1 --2- 172.21.15.5:0/3953328513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f01035a0 0x7f12f010ba80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.006+0000 7f12f755f640 1 --2- 172.21.15.5:0/3953328513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12f0102c80 0x7f12f0103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.006+0000 7f12f755f640 1 -- 172.21.15.5:0/3953328513 >> 172.21.15.5:0/3953328513 conn(0x7f12f00779a0 msgr2=0x7f12f00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.006+0000 7f12f755f640 1 -- 172.21.15.5:0/3953328513 shutdown_connections 2024-09-19T06:19:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.007+0000 7f12f755f640 1 -- 172.21.15.5:0/3953328513 wait complete. 2024-09-19T06:19:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.007+0000 7f12f755f640 1 Processor -- start 2024-09-19T06:19:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.007+0000 7f12f755f640 1 -- start start 2024-09-19T06:19:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f755f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12f0102c80 0x7f12f0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f755f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f01035a0 0x7f12f0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f755f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f010c0a0 0x7f12f019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f755f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0111d50 con 0x7f12f010c0a0 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f755f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0111bd0 con 0x7f12f01035a0 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f755f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0111ed0 con 0x7f12f0102c80 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f5afc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f010c0a0 0x7f12f019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f4afa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f01035a0 0x7f12f0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f5afc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f010c0a0 0x7f12f019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43538/0 (socket says 172.21.15.5:43538) 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f4afa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f01035a0 0x7f12f0198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37204/0 (socket says 172.21.15.5:37204) 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f5afc640 1 -- 172.21.15.5:0/411524477 learned_addr learned my addr 172.21.15.5:0/411524477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f52fb640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12f0102c80 0x7f12f0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f4afa640 1 -- 172.21.15.5:0/411524477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12f0102c80 msgr2=0x7f12f0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f4afa640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12f0102c80 0x7f12f0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f4afa640 1 -- 172.21.15.5:0/411524477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f010c0a0 msgr2=0x7f12f019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f4afa640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f010c0a0 0x7f12f019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f4afa640 1 -- 172.21.15.5:0/411524477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f019d310 con 0x7f12f01035a0 2024-09-19T06:19:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f5afc640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f010c0a0 0x7f12f019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.008+0000 7f12f4afa640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f01035a0 0x7f12f0198860 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f12e0000c00 tx=0x7f12e0030cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.009+0000 7f12e67fc640 1 -- 172.21.15.5:0/411524477 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0005460 con 0x7f12f01035a0 2024-09-19T06:19:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.009+0000 7f12f755f640 1 -- 172.21.15.5:0/411524477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f019d5a0 con 0x7f12f01035a0 2024-09-19T06:19:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.009+0000 7f12e67fc640 1 -- 172.21.15.5:0/411524477 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12e0005600 con 0x7f12f01035a0 2024-09-19T06:19:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.009+0000 7f12e67fc640 1 -- 172.21.15.5:0/411524477 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0004d70 con 0x7f12f01035a0 2024-09-19T06:19:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.009+0000 7f12f755f640 1 -- 172.21.15.5:0/411524477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f01a4e90 con 0x7f12f01035a0 2024-09-19T06:19:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.012+0000 7f12e67fc640 1 -- 172.21.15.5:0/411524477 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f12e0039040 con 0x7f12f01035a0 2024-09-19T06:19:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12e67fc640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f12dc0a33b0 0x7f12dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12fa3d7740 1 Processor -- start 2024-09-19T06:19:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12fa3d7740 1 -- start start 2024-09-19T06:19:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12e67fc640 1 -- 172.21.15.5:0/411524477 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8594+0+0 (secure 0 0 0) 0x7f12e00f09f0 con 0x7f12f01035a0 2024-09-19T06:19:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12fa3d7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f912395ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12fa3d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912396410 0x55f9123a0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12f52fb640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f12dc0a33b0 0x7f12dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12fa3d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f9123a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12fa3d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9122fa2f0 con 0x55f912396410 2024-09-19T06:19:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12fa3d7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f9122a2270 con 0x55f912395af0 2024-09-19T06:19:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12fa3d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f9122f62b0 con 0x55f9123a0cb0 2024-09-19T06:19:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12f5afc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f9123a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12f5afc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f9123a30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45198/0 (socket says 172.21.15.5:45198) 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12f5afc640 1 -- 172.21.15.5:0/3595927535 learned_addr learned my addr 172.21.15.5:0/3595927535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12f4afa640 1 --2- 172.21.15.5:0/3595927535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912396410 0x55f9123a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.013+0000 7f12f52fb640 1 --2- 172.21.15.5:0/3595927535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f912395ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.014+0000 7f12f5afc640 1 -- 172.21.15.5:0/3595927535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 msgr2=0x55f912395ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.014+0000 7f12f5afc640 1 --2- 172.21.15.5:0/3595927535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f912395ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.014+0000 7f12f5afc640 1 -- 172.21.15.5:0/3595927535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912396410 msgr2=0x55f9123a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.014+0000 7f12f5afc640 1 --2- 172.21.15.5:0/3595927535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912396410 0x55f9123a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.014+0000 7f12f5afc640 1 -- 172.21.15.5:0/3595927535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9122a1340 con 0x55f9123a0cb0 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.014+0000 7f12f52fb640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f12dc0a33b0 0x7f12dc0a5870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f12ec005fd0 tx=0x7f12ec011440 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.014+0000 7f12f5afc640 1 --2- 172.21.15.5:0/3595927535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f9123a30a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f12e800e830 tx=0x7f12e800ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc48d713f320da3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.014+0000 7f12db7fe640 1 -- 172.21.15.5:0/3595927535 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8004d20 con 0x55f9123a0cb0 2024-09-19T06:19:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.014+0000 7f12db7fe640 1 -- 172.21.15.5:0/3595927535 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12e8011e50 con 0x55f9123a0cb0 2024-09-19T06:19:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.014+0000 7f12db7fe640 1 -- 172.21.15.5:0/3595927535 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8005910 con 0x55f9123a0cb0 2024-09-19T06:19:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.015+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3595927535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 msgr2=0x55f9123a30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.015+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3595927535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f9123a30a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f12e800e830 tx=0x7f12e800ec30 comp rx=0 tx=0).stop 2024-09-19T06:19:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.015+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3595927535 shutdown_connections 2024-09-19T06:19:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.015+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3595927535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f9123a30a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.015+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3595927535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912396410 0x55f9123a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.015+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3595927535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f912395ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.015+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3595927535 >> 172.21.15.5:0/3595927535 conn(0x55f912392060 msgr2=0x55f912392440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.015+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3595927535 shutdown_connections 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.015+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3595927535 wait complete. 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.015+0000 7f12fa3d7740 1 -- start start 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12fa3d7740 1 Processor -- start 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12fa3d7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f91239db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12fa3d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912396410 0x55f91239e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12fa3d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f91239e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12fa3d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9123a7c90 con 0x55f912396410 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12fa3d7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f912283290 con 0x55f912395af0 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12fa3d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f9123a7e10 con 0x55f9123a0cb0 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12fa3d7740 10 client.0 _ll_register_callbacks cb 0x55f9122b4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12f52fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f91239db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12f52fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f91239db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37208/0 (socket says 172.21.15.5:37208) 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12f52fb640 1 -- 172.21.15.5:0/1302240007 learned_addr learned my addr 172.21.15.5:0/1302240007 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12f5afc640 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f91239e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.016+0000 7f12f4afa640 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912396410 0x55f91239e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12f52fb640 1 -- 172.21.15.5:0/1302240007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 msgr2=0x55f91239e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12f52fb640 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f91239e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12f52fb640 1 -- 172.21.15.5:0/1302240007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912396410 msgr2=0x55f91239e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12f52fb640 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912396410 0x55f91239e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12f52fb640 1 -- 172.21.15.5:0/1302240007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f91200d210 con 0x55f912395af0 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12f52fb640 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f91239db30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00fb30 tx=0x7f12ec004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28f21961b8f8e7fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12d8ff9640 1 -- 172.21.15.5:0/1302240007 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec017040 con 0x55f912395af0 2024-09-19T06:19:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12d8ff9640 1 -- 172.21.15.5:0/1302240007 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8594+0+0 (secure 0 0 0) 0x7f12ec01e8d0 con 0x55f912395af0 2024-09-19T06:19:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12fa3d7740 1 -- 172.21.15.5:0/1302240007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e8011e50 con 0x55f912395af0 2024-09-19T06:19:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12d8ff9640 1 -- 172.21.15.5:0/1302240007 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f12ec01ead0 con 0x55f912395af0 2024-09-19T06:19:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12fa3d7740 10 client.28883 fetch_fsmap learned FSMap version 48 2024-09-19T06:19:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.017+0000 7f12fa3d7740 1 -- 172.21.15.5:0/1302240007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f9120d9620 con 0x55f912395af0 2024-09-19T06:19:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.018+0000 7f12d8ff9640 1 -- 172.21.15.5:0/1302240007 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12e8011e50 con 0x55f912395af0 2024-09-19T06:19:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.018+0000 7f12d8ff9640 1 -- 172.21.15.5:0/1302240007 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f12ec025070 con 0x55f912395af0 2024-09-19T06:19:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.018+0000 7f12fa3d7740 10 client.28883 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:19:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.018+0000 7f12fa3d7740 10 client.28883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.018+0000 7f12fa3d7740 20 client.28883 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.018+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9123b50b0 0x55f9123b7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.018+0000 7f12fa3d7740 4 client.28883 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:19:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.018+0000 7f12fa3d7740 1 -- 172.21.15.5:0/1302240007 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f912307a20 con 0x55f9123b50b0 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.018+0000 7f12f4afa640 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9123b50b0 0x55f9123b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.019+0000 7f12f4afa640 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9123b50b0 0x55f9123b7500 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1911b87d50bf4c4 server_cookie=70808583497a9e28 in_seq=0 out_seq=0 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.019+0000 7f12d8ff9640 10 client.28883 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.019+0000 7f12d8ff9640 1 -- 172.21.15.5:0/1302240007 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f912307a20 con 0x55f9123b50b0 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.019+0000 7f12d8ff9640 10 client.28883 handle_command_reply: tid=0 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.019+0000 7f12fa3d7740 1 client.28883 shutdown 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.020+0000 7f12fa3d7740 1 -- 172.21.15.5:0/1302240007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 msgr2=0x55f91239db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.020+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f91239db30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00fb30 tx=0x7f12ec004730 comp rx=0 tx=0).stop 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.020+0000 7f12fa3d7740 1 -- 172.21.15.5:0/1302240007 shutdown_connections 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.020+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9123b50b0 0x55f9123b7500 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.020+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f91239e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.020+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912396410 0x55f91239e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.020+0000 7f12d8ff9640 0 client.28883 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.020+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/1302240007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f91239db30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.020+0000 7f12fa3d7740 1 -- 172.21.15.5:0/1302240007 >> 172.21.15.5:0/1302240007 conn(0x55f912392060 msgr2=0x55f912397dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.020+0000 7f12fa3d7740 1 -- 172.21.15.5:0/1302240007 shutdown_connections 2024-09-19T06:19:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.020+0000 7f12fa3d7740 1 -- 172.21.15.5:0/1302240007 wait complete. 2024-09-19T06:19:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.020+0000 7f12fa3d7740 20 client.28883 trim_cache size 0 max 16384 2024-09-19T06:19:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12fa3d7740 1 Processor -- start 2024-09-19T06:19:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12fa3d7740 1 -- start start 2024-09-19T06:19:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12fa3d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912395af0 0x55f9123bce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12fa3d7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912396410 0x55f9123bd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12fa3d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f9123975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12fa3d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01ab660 con 0x55f912395af0 2024-09-19T06:19:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12fa3d7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01ad350 con 0x55f912396410 2024-09-19T06:19:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12fa3d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01abdf0 con 0x55f9123a0cb0 2024-09-19T06:19:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12f4afa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912396410 0x55f9123bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12f5afc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f9123975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12f52fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912395af0 0x55f9123bce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12f4afa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912396410 0x55f9123bd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37224/0 (socket says 172.21.15.5:37224) 2024-09-19T06:19:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12f5afc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f9123975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45210/0 (socket says 172.21.15.5:45210) 2024-09-19T06:19:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12f52fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912395af0 0x55f9123bce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43564/0 (socket says 172.21.15.5:43564) 2024-09-19T06:19:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.042+0000 7f12f4afa640 1 -- 172.21.15.5:0/3287508406 learned_addr learned my addr 172.21.15.5:0/3287508406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.043+0000 7f12f4afa640 1 -- 172.21.15.5:0/3287508406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 msgr2=0x55f9123975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.043+0000 7f12f4afa640 1 --2- 172.21.15.5:0/3287508406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f9123975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.043+0000 7f12f4afa640 1 -- 172.21.15.5:0/3287508406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912395af0 msgr2=0x55f9123bce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.043+0000 7f12f4afa640 1 --2- 172.21.15.5:0/3287508406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912395af0 0x55f9123bce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.043+0000 7f12f4afa640 1 -- 172.21.15.5:0/3287508406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f912391e90 con 0x55f912396410 2024-09-19T06:19:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.043+0000 7f12f52fb640 1 --2- 172.21.15.5:0/3287508406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912395af0 0x55f9123bce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.043+0000 7f12f4afa640 1 --2- 172.21.15.5:0/3287508406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912396410 0x55f9123bd350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f12e00bcc10 tx=0x7f12e0030eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a048f2cba967ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.043+0000 7f12daffd640 1 -- 172.21.15.5:0/3287508406 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e00c58f0 con 0x55f912396410 2024-09-19T06:19:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.043+0000 7f12daffd640 1 -- 172.21.15.5:0/3287508406 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12e00bad20 con 0x55f912396410 2024-09-19T06:19:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.043+0000 7f12daffd640 1 -- 172.21.15.5:0/3287508406 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e00c5460 con 0x55f912396410 2024-09-19T06:19:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.044+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3287508406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912396410 msgr2=0x55f9123bd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.044+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3287508406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912396410 0x55f9123bd350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f12e00bcc10 tx=0x7f12e0030eb0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.044+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3287508406 shutdown_connections 2024-09-19T06:19:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.044+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3287508406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9123a0cb0 0x55f9123975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.044+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3287508406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912396410 0x55f9123bd350 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.044+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3287508406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f912395af0 0x55f9123bce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.044+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3287508406 >> 172.21.15.5:0/3287508406 conn(0x55f912392060 msgr2=0x55f91239cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.045+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3287508406 shutdown_connections 2024-09-19T06:19:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.045+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3287508406 wait complete. 2024-09-19T06:19:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.045+0000 7f12fa3d7740 1 -- start start 2024-09-19T06:19:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.045+0000 7f12fa3d7740 1 Processor -- start 2024-09-19T06:19:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12fa3d7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f91239aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12fa3d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f912396410 0x55f91239b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12fa3d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9123a0cb0 0x55f91239b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12fa3d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01acc80 con 0x55f9123a0cb0 2024-09-19T06:19:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12fa3d7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01ac4f0 con 0x55f912395af0 2024-09-19T06:19:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12fa3d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01a9660 con 0x55f912396410 2024-09-19T06:19:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12f52fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f91239aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12fa3d7740 10 client.0 _ll_register_callbacks cb 0x7f12ec00f420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12f52fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f91239aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37240/0 (socket says 172.21.15.5:37240) 2024-09-19T06:19:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12f52fb640 1 -- 172.21.15.5:0/3746663747 learned_addr learned my addr 172.21.15.5:0/3746663747 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12f5afc640 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9123a0cb0 0x55f91239b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12f4afa640 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f912396410 0x55f91239b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12f52fb640 1 -- 172.21.15.5:0/3746663747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f912396410 msgr2=0x55f91239b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12f52fb640 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f912396410 0x55f91239b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12f52fb640 1 -- 172.21.15.5:0/3746663747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9123a0cb0 msgr2=0x55f91239b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12f52fb640 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9123a0cb0 0x55f91239b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12f52fb640 1 -- 172.21.15.5:0/3746663747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f912392cf0 con 0x55f912395af0 2024-09-19T06:19:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12f5afc640 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9123a0cb0 0x55f91239b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.046+0000 7f12f52fb640 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f91239aad0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f12ec011940 tx=0x7f12ec01e270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=277c769f1aae52c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.047+0000 7f12d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:19:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.047+0000 7f12d8ff9640 1 -- 172.21.15.5:0/3746663747 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec017050 con 0x55f912395af0 2024-09-19T06:19:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.047+0000 7f12d8ff9640 1 -- 172.21.15.5:0/3746663747 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8594+0+0 (secure 0 0 0) 0x7f12ec01e8d0 con 0x55f912395af0 2024-09-19T06:19:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.047+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3746663747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e00bad20 con 0x55f912395af0 2024-09-19T06:19:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.047+0000 7f12d8ff9640 1 -- 172.21.15.5:0/3746663747 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f12ec00f940 con 0x55f912395af0 2024-09-19T06:19:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.047+0000 7f12fa3d7740 10 client.28889 fetch_fsmap learned FSMap version 48 2024-09-19T06:19:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.047+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3746663747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f91241fe80 con 0x55f912395af0 2024-09-19T06:19:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.047+0000 7f12d8ff9640 1 -- 172.21.15.5:0/3746663747 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12e00bad20 con 0x55f912395af0 2024-09-19T06:19:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.048+0000 7f12d8ff9640 1 -- 172.21.15.5:0/3746663747 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f12ec025040 con 0x55f912395af0 2024-09-19T06:19:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.048+0000 7f12fa3d7740 10 client.28889 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:19:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.048+0000 7f12fa3d7740 10 client.28889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.048+0000 7f12fa3d7740 20 client.28889 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.048+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9123b50b0 0x55f9124226a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.048+0000 7f12fa3d7740 4 client.28889 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:19:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.048+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3746663747 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f912301d70 con 0x55f9123b50b0 2024-09-19T06:19:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.048+0000 7f12f4afa640 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9123b50b0 0x55f9124226a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.049+0000 7f12d8ff9640 10 client.28889 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.049+0000 7f12f4afa640 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9123b50b0 0x55f9124226a0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=503bbe4343a2e2e9 server_cookie=7e730a61eb92f047 in_seq=0 out_seq=0 2024-09-19T06:19:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12d8ff9640 1 -- 172.21.15.5:0/3746663747 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f912301d70 con 0x55f9123b50b0 2024-09-19T06:19:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12d8ff9640 10 client.28889 handle_command_reply: tid=0 2024-09-19T06:19:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12fa3d7740 1 client.28889 shutdown 2024-09-19T06:19:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3746663747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 msgr2=0x55f91239aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f91239aad0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f12ec011940 tx=0x7f12ec01e270 comp rx=0 tx=0).stop 2024-09-19T06:19:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3746663747 shutdown_connections 2024-09-19T06:19:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9123b50b0 0x55f9124226a0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9123a0cb0 0x55f91239b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12d8ff9640 0 client.28889 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f912396410 0x55f91239b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12fa3d7740 1 --2- 172.21.15.5:0/3746663747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f912395af0 0x55f91239aad0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3746663747 >> 172.21.15.5:0/3746663747 conn(0x55f912392060 msgr2=0x55f9123975a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3746663747 shutdown_connections 2024-09-19T06:19:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12fa3d7740 1 -- 172.21.15.5:0/3746663747 wait complete. 2024-09-19T06:19:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.130+0000 7f12fa3d7740 20 client.28889 trim_cache size 0 max 16384 2024-09-19T06:19:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:19:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:19:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:19:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:19:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.131+0000 7f12f755f640 1 -- 172.21.15.5:0/411524477 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f12dc0a33b0 msgr2=0x7f12dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.131+0000 7f12f755f640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f12dc0a33b0 0x7f12dc0a5870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f12ec005fd0 tx=0x7f12ec011440 comp rx=0 tx=0).stop 2024-09-19T06:19:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.131+0000 7f12f755f640 1 -- 172.21.15.5:0/411524477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f01035a0 msgr2=0x7f12f0198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.131+0000 7f12f755f640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f01035a0 0x7f12f0198860 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f12e0000c00 tx=0x7f12e0030cc0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.131+0000 7f12f755f640 1 -- 172.21.15.5:0/411524477 shutdown_connections 2024-09-19T06:19:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.131+0000 7f12f755f640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f010c0a0 0x7f12f019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.131+0000 7f12f755f640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f01035a0 0x7f12f0198860 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.131+0000 7f12f755f640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f12dc0a33b0 0x7f12dc0a5870 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.131+0000 7f12f755f640 1 --2- 172.21.15.5:0/411524477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12f0102c80 0x7f12f0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.131+0000 7f12f755f640 1 -- 172.21.15.5:0/411524477 >> 172.21.15.5:0/411524477 conn(0x7f12f00779a0 msgr2=0x7f12f00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.131+0000 7f12f755f640 1 -- 172.21.15.5:0/411524477 shutdown_connections 2024-09-19T06:19:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:07.131+0000 7f12f755f640 1 -- 172.21.15.5:0/411524477 wait complete. 2024-09-19T06:19:07.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 25516 8.60 MB/sec execute 31 sec latency 64.050 ms 2024-09-19T06:19:08.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 25687 8.51 MB/sec execute 32 sec latency 59.064 ms 2024-09-19T06:19:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:08 smithi157 ceph-mon[35346]: pgmap v496: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 356 op/s 2024-09-19T06:19:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:08 smithi157 ceph-mon[35346]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T06:19:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:08 smithi005 ceph-mon[30955]: pgmap v496: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 356 op/s 2024-09-19T06:19:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:08 smithi005 ceph-mon[30955]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T06:19:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:08 smithi049 ceph-mon[34640]: pgmap v496: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 356 op/s 2024-09-19T06:19:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:08 smithi049 ceph-mon[34640]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T06:19:09.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 25848 8.44 MB/sec execute 33 sec latency 56.177 ms 2024-09-19T06:19:10.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 26058 8.71 MB/sec execute 34 sec latency 52.543 ms 2024-09-19T06:19:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:10 smithi157 ceph-mon[35346]: pgmap v498: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 445 op/s 2024-09-19T06:19:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:10 smithi005 ceph-mon[30955]: pgmap v498: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 445 op/s 2024-09-19T06:19:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:10 smithi049 ceph-mon[34640]: pgmap v498: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 445 op/s 2024-09-19T06:19:11.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 26244 8.80 MB/sec execute 35 sec latency 60.614 ms 2024-09-19T06:19:12.015 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:19:12.403 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:19:12.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 26428 8.82 MB/sec execute 36 sec latency 67.266 ms 2024-09-19T06:19:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:12 smithi157 ceph-mon[35346]: pgmap v499: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 219 op/s 2024-09-19T06:19:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:12 smithi005 ceph-mon[30955]: pgmap v499: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 219 op/s 2024-09-19T06:19:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:12 smithi049 ceph-mon[34640]: pgmap v499: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 219 op/s 2024-09-19T06:19:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.159+0000 7fd036979640 1 -- 172.21.15.5:0/2954217071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd030105de0 msgr2=0x7fd03010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.159+0000 7fd036979640 1 --2- 172.21.15.5:0/2954217071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd030105de0 0x7fd03010d960 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd028009ca0 tx=0x7fd02802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.159+0000 7fd036979640 1 -- 172.21.15.5:0/2954217071 shutdown_connections 2024-09-19T06:19:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.159+0000 7fd036979640 1 --2- 172.21.15.5:0/2954217071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd030105de0 0x7fd03010d960 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.159+0000 7fd036979640 1 --2- 172.21.15.5:0/2954217071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd030101e20 0x7fd0301056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.159+0000 7fd036979640 1 --2- 172.21.15.5:0/2954217071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd030101470 0x7fd030101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.159+0000 7fd036979640 1 -- 172.21.15.5:0/2954217071 >> 172.21.15.5:0/2954217071 conn(0x7fd030077230 msgr2=0x7fd0300fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.159+0000 7fd036979640 1 -- 172.21.15.5:0/2954217071 shutdown_connections 2024-09-19T06:19:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.159+0000 7fd036979640 1 -- 172.21.15.5:0/2954217071 wait complete. 2024-09-19T06:19:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.160+0000 7fd036979640 1 Processor -- start 2024-09-19T06:19:13.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.160+0000 7fd036979640 1 -- start start 2024-09-19T06:19:13.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.161+0000 7fd036979640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd030101470 0x7fd0301960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.161+0000 7fd036979640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd030101e20 0x7fd030196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.161+0000 7fd036979640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd030105de0 0x7fd03019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.161+0000 7fd036979640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd03010fbf0 con 0x7fd030101470 2024-09-19T06:19:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.161+0000 7fd036979640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd03010fa70 con 0x7fd030101e20 2024-09-19T06:19:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.161+0000 7fd036979640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd03010fd70 con 0x7fd030105de0 2024-09-19T06:19:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.161+0000 7fd034f16640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd030105de0 0x7fd03019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.161+0000 7fd034f16640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd030105de0 0x7fd03019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45220/0 (socket says 172.21.15.5:45220) 2024-09-19T06:19:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.161+0000 7fd034f16640 1 -- 172.21.15.5:0/3805556991 learned_addr learned my addr 172.21.15.5:0/3805556991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.162+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd030101e20 0x7fd030196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.162+0000 7fd02ffff640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd030101470 0x7fd0301960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.162+0000 7fd034f16640 1 -- 172.21.15.5:0/3805556991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd030101e20 msgr2=0x7fd030196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.162+0000 7fd034f16640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd030101e20 0x7fd030196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.162+0000 7fd034f16640 1 -- 172.21.15.5:0/3805556991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd030101470 msgr2=0x7fd0301960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.162+0000 7fd034f16640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd030101470 0x7fd0301960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.162+0000 7fd034f16640 1 -- 172.21.15.5:0/3805556991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03019b070 con 0x7fd030105de0 2024-09-19T06:19:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.162+0000 7fd02ffff640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd030101470 0x7fd0301960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:19:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.163+0000 7fd034f16640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd030105de0 0x7fd03019a990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd028009dc0 tx=0x7fd028032ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.163+0000 7fd02d7fa640 1 -- 172.21.15.5:0/3805556991 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028003c30 con 0x7fd030105de0 2024-09-19T06:19:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.163+0000 7fd036979640 1 -- 172.21.15.5:0/3805556991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03019b300 con 0x7fd030105de0 2024-09-19T06:19:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.163+0000 7fd036979640 1 -- 172.21.15.5:0/3805556991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0301a7050 con 0x7fd030105de0 2024-09-19T06:19:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.164+0000 7fd02d7fa640 1 -- 172.21.15.5:0/3805556991 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd028003dd0 con 0x7fd030105de0 2024-09-19T06:19:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.164+0000 7fd02d7fa640 1 -- 172.21.15.5:0/3805556991 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028030aa0 con 0x7fd030105de0 2024-09-19T06:19:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.167+0000 7fd02d7fa640 1 -- 172.21.15.5:0/3805556991 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd02802f4c0 con 0x7fd030105de0 2024-09-19T06:19:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.167+0000 7fd038d52740 1 Processor -- start 2024-09-19T06:19:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.167+0000 7fd038d52740 1 -- start start 2024-09-19T06:19:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.167+0000 7fd038d52740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 0x557a77d4d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.167+0000 7fd038d52740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 0x557a77d60950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.167+0000 7fd038d52740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d60e90 0x557a77d63280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.167+0000 7fd038d52740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557a77cb32e0 con 0x557a77d4dce0 2024-09-19T06:19:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.167+0000 7fd038d52740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557a77c5ace0 con 0x557a77d4d3c0 2024-09-19T06:19:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.167+0000 7fd038d52740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557a77caf2a0 con 0x557a77d60e90 2024-09-19T06:19:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 0x557a77d60950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 0x557a77d60950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43592/0 (socket says 172.21.15.5:43592) 2024-09-19T06:19:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02ffff640 1 -- 172.21.15.5:0/1181099566 learned_addr learned my addr 172.21.15.5:0/1181099566 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/1181099566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 0x557a77d4d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/1181099566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d60e90 0x557a77d63280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02d7fa640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0180a3430 0x7fd0180a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02d7fa640 1 -- 172.21.15.5:0/3805556991 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fd0280f1080 con 0x7fd030105de0 2024-09-19T06:19:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02f7fe640 1 -- 172.21.15.5:0/1181099566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d60e90 msgr2=0x557a77d63280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/1181099566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d60e90 0x557a77d63280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02f7fe640 1 -- 172.21.15.5:0/1181099566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 msgr2=0x557a77d60950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/1181099566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 0x557a77d60950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02f7fe640 1 -- 172.21.15.5:0/1181099566 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a77c3f440 con 0x557a77d4d3c0 2024-09-19T06:19:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd02ffff640 1 --2- 172.21.15.5:0/1181099566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 0x557a77d60950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.168+0000 7fd034f16640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0180a3430 0x7fd0180a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.169+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/1181099566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 0x557a77d4d7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd0240116b0 tx=0x7fd024004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfe87f54b263d79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.169+0000 7fd0167fc640 1 -- 172.21.15.5:0/1181099566 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024011050 con 0x557a77d4d3c0 2024-09-19T06:19:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.169+0000 7fd034f16640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0180a3430 0x7fd0180a58f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd0280ee970 tx=0x7fd028004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.169+0000 7fd0167fc640 1 -- 172.21.15.5:0/1181099566 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0240111f0 con 0x557a77d4d3c0 2024-09-19T06:19:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.169+0000 7fd0167fc640 1 -- 172.21.15.5:0/1181099566 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024010430 con 0x557a77d4d3c0 2024-09-19T06:19:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.170+0000 7fd038d52740 1 -- 172.21.15.5:0/1181099566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 msgr2=0x557a77d4d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.170+0000 7fd038d52740 1 --2- 172.21.15.5:0/1181099566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 0x557a77d4d7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd0240116b0 tx=0x7fd024004730 comp rx=0 tx=0).stop 2024-09-19T06:19:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.170+0000 7fd038d52740 1 -- 172.21.15.5:0/1181099566 shutdown_connections 2024-09-19T06:19:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.170+0000 7fd038d52740 1 --2- 172.21.15.5:0/1181099566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d60e90 0x557a77d63280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.170+0000 7fd038d52740 1 --2- 172.21.15.5:0/1181099566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 0x557a77d60950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.170+0000 7fd038d52740 1 --2- 172.21.15.5:0/1181099566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 0x557a77d4d7a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.170+0000 7fd038d52740 1 -- 172.21.15.5:0/1181099566 >> 172.21.15.5:0/1181099566 conn(0x557a77d49a30 msgr2=0x557a77d49e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.171+0000 7fd038d52740 1 -- 172.21.15.5:0/1181099566 shutdown_connections 2024-09-19T06:19:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.171+0000 7fd038d52740 1 -- 172.21.15.5:0/1181099566 wait complete. 2024-09-19T06:19:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.171+0000 7fd038d52740 1 -- start start 2024-09-19T06:19:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.171+0000 7fd038d52740 1 Processor -- start 2024-09-19T06:19:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.172+0000 7fd038d52740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4d3c0 0x557a77d60890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.172+0000 7fd02f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4d3c0 0x557a77d60890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.172+0000 7fd02f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4d3c0 0x557a77d60890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43602/0 (socket says 172.21.15.5:43602) 2024-09-19T06:19:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.172+0000 7fd02f7fe640 1 -- 172.21.15.5:0/3962167348 learned_addr learned my addr 172.21.15.5:0/3962167348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.172+0000 7fd038d52740 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4dce0 0x557a77d5db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.172+0000 7fd038d52740 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d60e90 0x557a77d5e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.172+0000 7fd038d52740 1 -- 172.21.15.5:0/3962167348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557a77d639a0 con 0x557a77d4d3c0 2024-09-19T06:19:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.172+0000 7fd038d52740 1 -- 172.21.15.5:0/3962167348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557a77c6e050 con 0x557a77d4dce0 2024-09-19T06:19:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.172+0000 7fd038d52740 1 -- 172.21.15.5:0/3962167348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557a77d63b20 con 0x557a77d60e90 2024-09-19T06:19:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.172+0000 7fd02ffff640 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4dce0 0x557a77d5db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.172+0000 7fd02ffff640 1 -- 172.21.15.5:0/3962167348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d60e90 msgr2=0x557a77d5e130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:19:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.173+0000 7fd02ffff640 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d60e90 0x557a77d5e130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.173+0000 7fd02ffff640 1 -- 172.21.15.5:0/3962167348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4d3c0 msgr2=0x557a77d60890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:13.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.173+0000 7fd038d52740 10 client.0 _ll_register_callbacks cb 0x557a77cae160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:13.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.174+0000 7fd02ffff640 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4d3c0 0x557a77d60890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.174+0000 7fd02ffff640 1 -- 172.21.15.5:0/3962167348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a77a243c0 con 0x557a77d4dce0 2024-09-19T06:19:13.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.174+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4d3c0 0x557a77d60890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:13.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.174+0000 7fd038d52740 1 -- 172.21.15.5:0/3962167348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0240111f0 con 0x557a77d4dce0 2024-09-19T06:19:13.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.175+0000 7fd02ffff640 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4dce0 0x557a77d5db40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd0200136e0 tx=0x7fd020013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dee0d1bf7878ba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:13.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.175+0000 7fd00f7fe640 10 client.28901 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:19:13.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.175+0000 7fd00f7fe640 1 -- 172.21.15.5:0/3962167348 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020011dd0 con 0x557a77d4dce0 2024-09-19T06:19:13.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.175+0000 7fd00f7fe640 1 -- 172.21.15.5:0/3962167348 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fd02001c440 con 0x557a77d4dce0 2024-09-19T06:19:13.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.175+0000 7fd00f7fe640 1 -- 172.21.15.5:0/3962167348 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd02001c680 con 0x557a77d4dce0 2024-09-19T06:19:13.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.175+0000 7fd038d52740 10 client.28901 fetch_fsmap learned FSMap version 48 2024-09-19T06:19:13.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.176+0000 7fd038d52740 1 -- 172.21.15.5:0/3962167348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=48}) -- 0x557a77d6f3e0 con 0x557a77d4dce0 2024-09-19T06:19:13.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.176+0000 7fd00f7fe640 1 -- 172.21.15.5:0/3962167348 <== mon.2 v2:172.21.15.49:3300/0 4 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7fd02001a070 con 0x557a77d4dce0 2024-09-19T06:19:13.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.176+0000 7fd038d52740 10 client.28901 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:19:13.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.176+0000 7fd038d52740 10 client.28901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:13.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.176+0000 7fd038d52740 20 client.28901 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.176+0000 7fd038d52740 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557a77d707e0 0x557a77d72c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.176+0000 7fd038d52740 4 client.28901 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:19:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.176+0000 7fd038d52740 1 -- 172.21.15.5:0/3962167348 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a77cc0850 con 0x557a77d707e0 2024-09-19T06:19:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.177+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557a77d707e0 0x557a77d72c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.177+0000 7fd00f7fe640 10 client.28901 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.177+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557a77d707e0 0x557a77d72c30 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e099dd39ac4ab6 server_cookie=b7e2903117498af9 in_seq=0 out_seq=0 2024-09-19T06:19:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.178+0000 7fd00f7fe640 1 -- 172.21.15.5:0/3962167348 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a77cc0850 con 0x557a77d707e0 2024-09-19T06:19:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.178+0000 7fd00f7fe640 10 client.28901 handle_command_reply: tid=0 2024-09-19T06:19:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.178+0000 7fd038d52740 1 client.28901 shutdown 2024-09-19T06:19:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.178+0000 7fd038d52740 1 -- 172.21.15.5:0/3962167348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4dce0 msgr2=0x557a77d5db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.178+0000 7fd038d52740 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4dce0 0x557a77d5db40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd0200136e0 tx=0x7fd020013ba0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.178+0000 7fd038d52740 1 -- 172.21.15.5:0/3962167348 shutdown_connections 2024-09-19T06:19:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.178+0000 7fd038d52740 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557a77d707e0 0x557a77d72c30 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.178+0000 7fd038d52740 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d60e90 0x557a77d5e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.178+0000 7fd038d52740 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4dce0 0x557a77d5db40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.178+0000 7fd038d52740 1 --2- 172.21.15.5:0/3962167348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4d3c0 0x557a77d60890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.178+0000 7fd038d52740 1 -- 172.21.15.5:0/3962167348 >> 172.21.15.5:0/3962167348 conn(0x557a77d49a30 msgr2=0x557a77d4f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.179+0000 7fd00f7fe640 0 client.28901 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.179+0000 7fd038d52740 1 -- 172.21.15.5:0/3962167348 shutdown_connections 2024-09-19T06:19:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.179+0000 7fd038d52740 1 -- 172.21.15.5:0/3962167348 wait complete. 2024-09-19T06:19:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.179+0000 7fd038d52740 20 client.28901 trim_cache size 0 max 16384 2024-09-19T06:19:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.205+0000 7fd038d52740 1 Processor -- start 2024-09-19T06:19:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.205+0000 7fd038d52740 1 -- start start 2024-09-19T06:19:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.205+0000 7fd038d52740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d4d3c0 0x557a77d6cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.205+0000 7fd038d52740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 0x557a77d5fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd038d52740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d60e90 0x557a77d78540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd038d52740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301ab700 con 0x557a77d4dce0 2024-09-19T06:19:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd038d52740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301ad3f0 con 0x557a77d60e90 2024-09-19T06:19:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd038d52740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301abe90 con 0x557a77d4d3c0 2024-09-19T06:19:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 0x557a77d5fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 0x557a77d5fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43604/0 (socket says 172.21.15.5:43604) 2024-09-19T06:19:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd02ffff640 1 -- 172.21.15.5:0/616205590 learned_addr learned my addr 172.21.15.5:0/616205590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd02f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d4d3c0 0x557a77d6cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/616205590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d60e90 0x557a77d78540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd02f7fe640 1 -- 172.21.15.5:0/616205590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d60e90 msgr2=0x557a77d78540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/616205590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d60e90 0x557a77d78540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd02f7fe640 1 -- 172.21.15.5:0/616205590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 msgr2=0x557a77d5fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/616205590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 0x557a77d5fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd02f7fe640 1 -- 172.21.15.5:0/616205590 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a77d5caf0 con 0x557a77d4d3c0 2024-09-19T06:19:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.206+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/616205590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d4d3c0 0x557a77d6cc70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x557a77d734e0 tx=0x7fd024004a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c7866f2d77e8749 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.207+0000 7fd015ffb640 1 -- 172.21.15.5:0/616205590 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0240040b0 con 0x557a77d4d3c0 2024-09-19T06:19:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.207+0000 7fd015ffb640 1 -- 172.21.15.5:0/616205590 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd024004f00 con 0x557a77d4d3c0 2024-09-19T06:19:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.207+0000 7fd015ffb640 1 -- 172.21.15.5:0/616205590 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024003810 con 0x557a77d4d3c0 2024-09-19T06:19:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.208+0000 7fd038d52740 1 -- 172.21.15.5:0/616205590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d4d3c0 msgr2=0x557a77d6cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.208+0000 7fd038d52740 1 --2- 172.21.15.5:0/616205590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d4d3c0 0x557a77d6cc70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x557a77d734e0 tx=0x7fd024004a50 comp rx=0 tx=0).stop 2024-09-19T06:19:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.208+0000 7fd038d52740 1 -- 172.21.15.5:0/616205590 shutdown_connections 2024-09-19T06:19:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.208+0000 7fd038d52740 1 --2- 172.21.15.5:0/616205590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d60e90 0x557a77d78540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.208+0000 7fd038d52740 1 --2- 172.21.15.5:0/616205590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d4dce0 0x557a77d5fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.208+0000 7fd038d52740 1 --2- 172.21.15.5:0/616205590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d4d3c0 0x557a77d6cc70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.208+0000 7fd038d52740 1 -- 172.21.15.5:0/616205590 >> 172.21.15.5:0/616205590 conn(0x557a77d49a30 msgr2=0x557a77d5f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.208+0000 7fd038d52740 1 -- 172.21.15.5:0/616205590 shutdown_connections 2024-09-19T06:19:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.208+0000 7fd038d52740 1 -- 172.21.15.5:0/616205590 wait complete. 2024-09-19T06:19:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.208+0000 7fd038d52740 1 -- start start 2024-09-19T06:19:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.208+0000 7fd038d52740 1 Processor -- start 2024-09-19T06:19:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd038d52740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 0x557a77d5aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd038d52740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d4dce0 0x557a77d5af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd038d52740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d60e90 0x557a77d5b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd038d52740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301acd20 con 0x557a77d60e90 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd038d52740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301ac590 con 0x557a77d4d3c0 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd038d52740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301a9720 con 0x557a77d4dce0 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd038d52740 10 client.0 _ll_register_callbacks cb 0x7fd020005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd02f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 0x557a77d5aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd02f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 0x557a77d5aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37268/0 (socket says 172.21.15.5:37268) 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd02f7fe640 1 -- 172.21.15.5:0/285325741 learned_addr learned my addr 172.21.15.5:0/285325741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d60e90 0x557a77d5b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd02ffff640 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d4dce0 0x557a77d5af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd02ffff640 1 -- 172.21.15.5:0/285325741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 msgr2=0x557a77d5aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.209+0000 7fd02ffff640 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 0x557a77d5aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.210+0000 7fd02ffff640 1 -- 172.21.15.5:0/285325741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d60e90 msgr2=0x557a77d5b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.210+0000 7fd02ffff640 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d60e90 0x557a77d5b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.210+0000 7fd02ffff640 1 -- 172.21.15.5:0/285325741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a77dda510 con 0x557a77d4dce0 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.210+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 0x557a77d5aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.210+0000 7fd02ffff640 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d4dce0 0x557a77d5af80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x557a77d3e4e0 tx=0x7fd0200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ee4b4c9b6a16acc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.210+0000 7fd00f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.210+0000 7fd00f7fe640 1 -- 172.21.15.5:0/285325741 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020004450 con 0x557a77d4dce0 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.210+0000 7fd00f7fe640 1 -- 172.21.15.5:0/285325741 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fd020026840 con 0x557a77d4dce0 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.210+0000 7fd038d52740 1 -- 172.21.15.5:0/285325741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd024004f00 con 0x557a77d4dce0 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.210+0000 7fd00f7fe640 1 -- 172.21.15.5:0/285325741 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd020026a40 con 0x557a77d4dce0 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.210+0000 7fd038d52740 10 client.28852 fetch_fsmap learned FSMap version 48 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.210+0000 7fd038d52740 1 -- 172.21.15.5:0/285325741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557a77ddb700 con 0x557a77d4dce0 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.211+0000 7fd00f7fe640 1 -- 172.21.15.5:0/285325741 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0240111f0 con 0x557a77d4dce0 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.211+0000 7fd00f7fe640 1 -- 172.21.15.5:0/285325741 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7fd02001a040 con 0x557a77d4dce0 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.211+0000 7fd038d52740 10 client.28852 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.211+0000 7fd038d52740 10 client.28852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.211+0000 7fd038d52740 20 client.28852 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.211+0000 7fd038d52740 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557a77ddbee0 0x557a77dde300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.211+0000 7fd038d52740 4 client.28852 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:19:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.211+0000 7fd038d52740 1 -- 172.21.15.5:0/285325741 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fd024004f00 con 0x557a77ddbee0 2024-09-19T06:19:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.211+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557a77ddbee0 0x557a77dde300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.212+0000 7fd00f7fe640 10 client.28852 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.212+0000 7fd02f7fe640 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557a77ddbee0 0x557a77dde300 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43afdf89103b04a1 server_cookie=98c4a34b48693e92 in_seq=0 out_seq=0 2024-09-19T06:19:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.212+0000 7fd00f7fe640 1 -- 172.21.15.5:0/285325741 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fd024004f00 con 0x557a77ddbee0 2024-09-19T06:19:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.212+0000 7fd00f7fe640 10 client.28852 handle_command_reply: tid=0 2024-09-19T06:19:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.212+0000 7fd038d52740 1 client.28852 shutdown 2024-09-19T06:19:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.212+0000 7fd038d52740 1 -- 172.21.15.5:0/285325741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d4dce0 msgr2=0x557a77d5af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.212+0000 7fd038d52740 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d4dce0 0x557a77d5af80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x557a77d3e4e0 tx=0x7fd0200046c0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.212+0000 7fd038d52740 1 -- 172.21.15.5:0/285325741 shutdown_connections 2024-09-19T06:19:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.212+0000 7fd038d52740 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557a77ddbee0 0x557a77dde300 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.213+0000 7fd038d52740 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a77d60e90 0x557a77d5b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.213+0000 7fd00f7fe640 0 client.28852 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.213+0000 7fd038d52740 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a77d4dce0 0x557a77d5af80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.213+0000 7fd038d52740 1 --2- 172.21.15.5:0/285325741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a77d4d3c0 0x557a77d5aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.213+0000 7fd038d52740 1 -- 172.21.15.5:0/285325741 >> 172.21.15.5:0/285325741 conn(0x557a77d49a30 msgr2=0x557a77d59110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.213+0000 7fd038d52740 1 -- 172.21.15.5:0/285325741 shutdown_connections 2024-09-19T06:19:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.213+0000 7fd038d52740 1 -- 172.21.15.5:0/285325741 wait complete. 2024-09-19T06:19:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.213+0000 7fd038d52740 20 client.28852 trim_cache size 0 max 16384 2024-09-19T06:19:13.220 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:19:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:19:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:19:13.221 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:19:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.214+0000 7fd036979640 1 -- 172.21.15.5:0/3805556991 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0180a3430 msgr2=0x7fd0180a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.214+0000 7fd036979640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0180a3430 0x7fd0180a58f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd0280ee970 tx=0x7fd028004a40 comp rx=0 tx=0).stop 2024-09-19T06:19:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.214+0000 7fd036979640 1 -- 172.21.15.5:0/3805556991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd030105de0 msgr2=0x7fd03019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.214+0000 7fd036979640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd030105de0 0x7fd03019a990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd028009dc0 tx=0x7fd028032ff0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.214+0000 7fd036979640 1 -- 172.21.15.5:0/3805556991 shutdown_connections 2024-09-19T06:19:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.214+0000 7fd036979640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0180a3430 0x7fd0180a58f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.214+0000 7fd036979640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd030105de0 0x7fd03019a990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.214+0000 7fd036979640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd030101e20 0x7fd030196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.214+0000 7fd036979640 1 --2- 172.21.15.5:0/3805556991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd030101470 0x7fd0301960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.214+0000 7fd036979640 1 -- 172.21.15.5:0/3805556991 >> 172.21.15.5:0/3805556991 conn(0x7fd030077230 msgr2=0x7fd030104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.214+0000 7fd036979640 1 -- 172.21.15.5:0/3805556991 shutdown_connections 2024-09-19T06:19:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:13.214+0000 7fd036979640 1 -- 172.21.15.5:0/3805556991 wait complete. 2024-09-19T06:19:13.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 26594 8.96 MB/sec execute 37 sec latency 84.870 ms 2024-09-19T06:19:14.034 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:19:14.035 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:19:14.412 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:19:14.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 26788 9.08 MB/sec execute 38 sec latency 66.702 ms 2024-09-19T06:19:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:14 smithi157 ceph-mon[35346]: pgmap v500: 97 pgs: 97 active+clean; 598 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 402 op/s 2024-09-19T06:19:14.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:14 smithi005 ceph-mon[30955]: pgmap v500: 97 pgs: 97 active+clean; 598 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 402 op/s 2024-09-19T06:19:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:14 smithi049 ceph-mon[34640]: pgmap v500: 97 pgs: 97 active+clean; 598 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 402 op/s 2024-09-19T06:19:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.987+0000 7f66eb124640 1 -- 172.21.15.5:0/3605238714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4101de0 msgr2=0x7f66e41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.987+0000 7f66eb124640 1 --2- 172.21.15.5:0/3605238714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4101de0 0x7f66e41021c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009900 tx=0x7f66d802a410 comp rx=0 tx=0).stop 2024-09-19T06:19:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.987+0000 7f66eb124640 1 -- 172.21.15.5:0/3605238714 shutdown_connections 2024-09-19T06:19:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.987+0000 7f66eb124640 1 --2- 172.21.15.5:0/3605238714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e4106f10 0x7f66e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.987+0000 7f66eb124640 1 --2- 172.21.15.5:0/3605238714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66e4102790 0x7f66e41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.987+0000 7f66eb124640 1 --2- 172.21.15.5:0/3605238714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4101de0 0x7f66e41021c0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:14.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.987+0000 7f66eb124640 1 -- 172.21.15.5:0/3605238714 >> 172.21.15.5:0/3605238714 conn(0x7f66e40fd8e0 msgr2=0x7f66e40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:14.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.987+0000 7f66eb124640 1 -- 172.21.15.5:0/3605238714 shutdown_connections 2024-09-19T06:19:14.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.987+0000 7f66eb124640 1 -- 172.21.15.5:0/3605238714 wait complete. 2024-09-19T06:19:14.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.988+0000 7f66eb124640 1 Processor -- start 2024-09-19T06:19:14.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.988+0000 7f66eb124640 1 -- start start 2024-09-19T06:19:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.989+0000 7f66eb124640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4102790 0x7f66e40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.989+0000 7f66eb124640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66e4106f10 0x7f66e4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.989+0000 7f66eb124640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e4076180 0x7f66e4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.989+0000 7f66eb124640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4111de0 con 0x7f66e4102790 2024-09-19T06:19:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.989+0000 7f66eb124640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4111c60 con 0x7f66e4106f10 2024-09-19T06:19:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.989+0000 7f66eb124640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4111f60 con 0x7f66e4076180 2024-09-19T06:19:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.989+0000 7f66e96c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e4076180 0x7f66e4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.989+0000 7f66e96c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e4076180 0x7f66e4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34282/0 (socket says 172.21.15.5:34282) 2024-09-19T06:19:14.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.989+0000 7f66e8ec0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4102790 0x7f66e40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:14.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.989+0000 7f66e96c1640 1 -- 172.21.15.5:0/1606097478 learned_addr learned my addr 172.21.15.5:0/1606097478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:14.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.989+0000 7f66e3fff640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66e4106f10 0x7f66e4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:14.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.990+0000 7f66e8ec0640 1 -- 172.21.15.5:0/1606097478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e4076180 msgr2=0x7f66e4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:14.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.990+0000 7f66e8ec0640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e4076180 0x7f66e4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:14.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.990+0000 7f66e8ec0640 1 -- 172.21.15.5:0/1606097478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66e4106f10 msgr2=0x7f66e4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:14.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.990+0000 7f66e8ec0640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66e4106f10 0x7f66e4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:14.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.990+0000 7f66e8ec0640 1 -- 172.21.15.5:0/1606097478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e419c7a0 con 0x7f66e4102790 2024-09-19T06:19:14.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.990+0000 7f66e3fff640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66e4106f10 0x7f66e4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:14.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.990+0000 7f66e8ec0640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4102790 0x7f66e40744f0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f66d8008000 tx=0x7f66d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:14.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.991+0000 7f66e1ffb640 1 -- 172.21.15.5:0/1606097478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d803f070 con 0x7f66e4102790 2024-09-19T06:19:14.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.991+0000 7f66e1ffb640 1 -- 172.21.15.5:0/1606097478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66d803b460 con 0x7f66e4102790 2024-09-19T06:19:14.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.991+0000 7f66e1ffb640 1 -- 172.21.15.5:0/1606097478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8035070 con 0x7f66e4102790 2024-09-19T06:19:14.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.991+0000 7f66eb124640 1 -- 172.21.15.5:0/1606097478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e419ca30 con 0x7f66e4102790 2024-09-19T06:19:14.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.991+0000 7f66eb124640 1 -- 172.21.15.5:0/1606097478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e419ce90 con 0x7f66e4102790 2024-09-19T06:19:14.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.993+0000 7f66eb124640 1 -- 172.21.15.5:0/1606097478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e41032c0 con 0x7f66e4102790 2024-09-19T06:19:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.994+0000 7f66e1ffb640 1 -- 172.21.15.5:0/1606097478 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f66d803b610 con 0x7f66e4102790 2024-09-19T06:19:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.995+0000 7f66e1ffb640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f66d40a3500 0x7f66d40a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.995+0000 7f66e3fff640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f66d40a3500 0x7f66d40a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.996+0000 7f66e1ffb640 1 -- 172.21.15.5:0/1606097478 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f66d80f0040 con 0x7f66e4102790 2024-09-19T06:19:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:14.996+0000 7f66e3fff640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f66d40a3500 0x7f66d40a59c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f66e4075cb0 tx=0x7f66d000b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:15.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.002+0000 7f66e1ffb640 1 -- 172.21.15.5:0/1606097478 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f66d80bc440 con 0x7f66e4102790 2024-09-19T06:19:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.378+0000 7f66eb124640 1 -- 172.21.15.5:0/1606097478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66e410a0f0 con 0x7f66e4102790 2024-09-19T06:19:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.379+0000 7f66e1ffb640 1 -- 172.21.15.5:0/1606097478 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11975 (secure 0 0 0) 0x7f66d80c0740 con 0x7f66e4102790 2024-09-19T06:19:15.381 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:19:15.381 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":48,"btime":"2024-09-19T06:18:50:496191+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:18:50.457699+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:19:15.381 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 48 2024-09-19T06:19:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.382+0000 7f66eb124640 1 -- 172.21.15.5:0/1606097478 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f66d40a3500 msgr2=0x7f66d40a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.382+0000 7f66eb124640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f66d40a3500 0x7f66d40a59c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f66e4075cb0 tx=0x7f66d000b390 comp rx=0 tx=0).stop 2024-09-19T06:19:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.382+0000 7f66eb124640 1 -- 172.21.15.5:0/1606097478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4102790 msgr2=0x7f66e40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.382+0000 7f66eb124640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4102790 0x7f66e40744f0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f66d8008000 tx=0x7f66d80042f0 comp rx=0 tx=0).stop 2024-09-19T06:19:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.382+0000 7f66eb124640 1 -- 172.21.15.5:0/1606097478 shutdown_connections 2024-09-19T06:19:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.382+0000 7f66eb124640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f66d40a3500 0x7f66d40a59c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.382+0000 7f66eb124640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e4076180 0x7f66e4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.382+0000 7f66eb124640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66e4106f10 0x7f66e4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.382+0000 7f66eb124640 1 --2- 172.21.15.5:0/1606097478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4102790 0x7f66e40744f0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.382+0000 7f66eb124640 1 -- 172.21.15.5:0/1606097478 >> 172.21.15.5:0/1606097478 conn(0x7f66e40fd8e0 msgr2=0x7f66e40fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:15.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.382+0000 7f66eb124640 1 -- 172.21.15.5:0/1606097478 shutdown_connections 2024-09-19T06:19:15.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:15.383+0000 7f66eb124640 1 -- 172.21.15.5:0/1606097478 wait complete. 2024-09-19T06:19:15.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 26985 9.11 MB/sec execute 39 sec latency 104.645 ms 2024-09-19T06:19:15.708 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:15 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1606097478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:19:15.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:15 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1606097478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:19:15.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:15 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1606097478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:19:16.073 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:19:16.454 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:19:16.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 27101 9.06 MB/sec execute 40 sec latency 81.651 ms 2024-09-19T06:19:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:16 smithi157 ceph-mon[35346]: pgmap v501: 97 pgs: 97 active+clean; 598 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 335 op/s 2024-09-19T06:19:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:16 smithi005 ceph-mon[30955]: pgmap v501: 97 pgs: 97 active+clean; 598 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 335 op/s 2024-09-19T06:19:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:16 smithi049 ceph-mon[34640]: pgmap v501: 97 pgs: 97 active+clean; 598 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 335 op/s 2024-09-19T06:19:17.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.043+0000 7f9dd3161640 1 -- 172.21.15.5:0/1389523669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc102570 msgr2=0x7f9dcc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:17.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.043+0000 7f9dd3161640 1 --2- 172.21.15.5:0/1389523669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc102570 0x7f9dcc1029b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009930 tx=0x7f9db802a4d0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.043+0000 7f9dd3161640 1 -- 172.21.15.5:0/1389523669 shutdown_connections 2024-09-19T06:19:17.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.043+0000 7f9dd3161640 1 --2- 172.21.15.5:0/1389523669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dcc102ef0 0x7f9dcc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.043+0000 7f9dd3161640 1 --2- 172.21.15.5:0/1389523669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc102570 0x7f9dcc1029b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.043+0000 7f9dd3161640 1 --2- 172.21.15.5:0/1389523669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dcc100e20 0x7f9dcc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.043+0000 7f9dd3161640 1 -- 172.21.15.5:0/1389523669 >> 172.21.15.5:0/1389523669 conn(0x7f9dcc0fc7d0 msgr2=0x7f9dcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:17.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.043+0000 7f9dd3161640 1 -- 172.21.15.5:0/1389523669 shutdown_connections 2024-09-19T06:19:17.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.043+0000 7f9dd3161640 1 -- 172.21.15.5:0/1389523669 wait complete. 2024-09-19T06:19:17.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dd3161640 1 Processor -- start 2024-09-19T06:19:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dd3161640 1 -- start start 2024-09-19T06:19:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dd3161640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dcc100e20 0x7f9dcc10e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dd3161640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc102570 0x7f9dcc10e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dd3161640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dcc102ef0 0x7f9dcc10ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dd3161640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc111f40 con 0x7f9dcc100e20 2024-09-19T06:19:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dd3161640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc111dc0 con 0x7f9dcc102ef0 2024-09-19T06:19:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dd3161640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1120c0 con 0x7f9dcc102570 2024-09-19T06:19:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dd16fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dcc102ef0 0x7f9dcc10ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dd16fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dcc102ef0 0x7f9dcc10ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49490/0 (socket says 172.21.15.5:49490) 2024-09-19T06:19:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dd16fe640 1 -- 172.21.15.5:0/693921662 learned_addr learned my addr 172.21.15.5:0/693921662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dd0efd640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dcc100e20 0x7f9dcc10e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.044+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc102570 0x7f9dcc10e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.045+0000 7f9dd16fe640 1 -- 172.21.15.5:0/693921662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc102570 msgr2=0x7f9dcc10e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.045+0000 7f9dd16fe640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc102570 0x7f9dcc10e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.045+0000 7f9dd16fe640 1 -- 172.21.15.5:0/693921662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dcc100e20 msgr2=0x7f9dcc10e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.045+0000 7f9dd16fe640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dcc100e20 0x7f9dcc10e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.045+0000 7f9dd16fe640 1 -- 172.21.15.5:0/693921662 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc1aae30 con 0x7f9dcc102ef0 2024-09-19T06:19:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.045+0000 7f9dd16fe640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dcc102ef0 0x7f9dcc10ec50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400eb10 tx=0x7f9dc400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.045+0000 7f9dc9ffb640 1 -- 172.21.15.5:0/693921662 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4004b20 con 0x7f9dcc102ef0 2024-09-19T06:19:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.046+0000 7f9dc9ffb640 1 -- 172.21.15.5:0/693921662 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dc4004cc0 con 0x7f9dcc102ef0 2024-09-19T06:19:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.046+0000 7f9dd3161640 1 -- 172.21.15.5:0/693921662 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dcc1ab060 con 0x7f9dcc102ef0 2024-09-19T06:19:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.046+0000 7f9dc9ffb640 1 -- 172.21.15.5:0/693921662 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4010690 con 0x7f9dcc102ef0 2024-09-19T06:19:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.046+0000 7f9dd3161640 1 -- 172.21.15.5:0/693921662 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dcc1ab520 con 0x7f9dcc102ef0 2024-09-19T06:19:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.049+0000 7f9dc9ffb640 1 -- 172.21.15.5:0/693921662 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9dc4005020 con 0x7f9dcc102ef0 2024-09-19T06:19:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.050+0000 7f9dd553a740 1 Processor -- start 2024-09-19T06:19:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.050+0000 7f9dd553a740 1 -- start start 2024-09-19T06:19:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.050+0000 7f9dd553a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b1b40 0x56052e6b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.050+0000 7f9dd553a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.050+0000 7f9dd553a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.050+0000 7f9dd553a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56052e6162f0 con 0x56052e6bb370 2024-09-19T06:19:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.050+0000 7f9dd553a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56052e5bdcf0 con 0x56052e6b05d0 2024-09-19T06:19:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.050+0000 7f9dd553a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56052e6122b0 con 0x56052e6b1b40 2024-09-19T06:19:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dd0efd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dd0efd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49504/0 (socket says 172.21.15.5:49504) 2024-09-19T06:19:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dd0efd640 1 -- 172.21.15.5:0/504272047 learned_addr learned my addr 172.21.15.5:0/504272047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/504272047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b1b40 0x56052e6b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dc9ffb640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9dbc0a33b0 0x7f9dbc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dcbfff640 1 -- 172.21.15.5:0/504272047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b1b40 msgr2=0x56052e6b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/504272047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b1b40 0x56052e6b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dcbfff640 1 -- 172.21.15.5:0/504272047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 msgr2=0x56052e6b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/504272047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dcbfff640 1 -- 172.21.15.5:0/504272047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052e5a23a0 con 0x56052e6bb370 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dd0efd640 1 --2- 172.21.15.5:0/504272047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.051+0000 7f9dd16fe640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9dbc0a33b0 0x7f9dbc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.052+0000 7f9dc9ffb640 1 -- 172.21.15.5:0/693921662 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f9dc40d0300 con 0x7f9dcc102ef0 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.052+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/504272047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6bd760 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f9db8031440 tx=0x7f9db8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5899ab6dbf263947 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.052+0000 7f9dd16fe640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9dbc0a33b0 0x7f9dbc0a5870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40989c0 tx=0x7f9dc4066f50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.052+0000 7f9db6ffd640 1 -- 172.21.15.5:0/504272047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8035070 con 0x56052e6bb370 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.052+0000 7f9db6ffd640 1 -- 172.21.15.5:0/504272047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9db8005020 con 0x56052e6bb370 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.052+0000 7f9db6ffd640 1 -- 172.21.15.5:0/504272047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db80044e0 con 0x56052e6bb370 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.053+0000 7f9dd553a740 1 -- 172.21.15.5:0/504272047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 msgr2=0x56052e6bd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.053+0000 7f9dd553a740 1 --2- 172.21.15.5:0/504272047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6bd760 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f9db8031440 tx=0x7f9db8005fd0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.053+0000 7f9dd553a740 1 -- 172.21.15.5:0/504272047 shutdown_connections 2024-09-19T06:19:17.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.053+0000 7f9dd553a740 1 --2- 172.21.15.5:0/504272047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6bd760 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.053+0000 7f9dd553a740 1 --2- 172.21.15.5:0/504272047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.053+0000 7f9dd553a740 1 --2- 172.21.15.5:0/504272047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b1b40 0x56052e6b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.053+0000 7f9dd553a740 1 -- 172.21.15.5:0/504272047 >> 172.21.15.5:0/504272047 conn(0x56052e679310 msgr2=0x56052e6acc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.053+0000 7f9dd553a740 1 -- 172.21.15.5:0/504272047 shutdown_connections 2024-09-19T06:19:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.053+0000 7f9dd553a740 1 -- 172.21.15.5:0/504272047 wait complete. 2024-09-19T06:19:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.054+0000 7f9dd553a740 1 -- start start 2024-09-19T06:19:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.054+0000 7f9dd553a740 1 Processor -- start 2024-09-19T06:19:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.054+0000 7f9dd553a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b05d0 0x56052e6bad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.054+0000 7f9dd553a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b1b40 0x56052e6b8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.054+0000 7f9dd553a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6b8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.054+0000 7f9dd553a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56052e6be010 con 0x56052e6bb370 2024-09-19T06:19:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.054+0000 7f9dd553a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56052e5d1060 con 0x56052e6b1b40 2024-09-19T06:19:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.054+0000 7f9dd553a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56052e6be190 con 0x56052e6b05d0 2024-09-19T06:19:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.054+0000 7f9dd553a740 10 client.0 _ll_register_callbacks cb 0x56052e3248c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6b8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37912/0 (socket says 172.21.15.5:37912) 2024-09-19T06:19:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dcbfff640 1 -- 172.21.15.5:0/1652182294 learned_addr learned my addr 172.21.15.5:0/1652182294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b05d0 0x56052e6bad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dd0efd640 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b1b40 0x56052e6b8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dcbfff640 1 -- 172.21.15.5:0/1652182294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b05d0 msgr2=0x56052e6bad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b05d0 0x56052e6bad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dcbfff640 1 -- 172.21.15.5:0/1652182294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b1b40 msgr2=0x56052e6b8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b1b40 0x56052e6b8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dcbfff640 1 -- 172.21.15.5:0/1652182294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052e3873c0 con 0x56052e6bb370 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dd0efd640 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b1b40 0x56052e6b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6b8640 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9db8004770 tx=0x7f9db8032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3101f00e29641d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.055+0000 7f9dabfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dabfff640 1 -- 172.21.15.5:0/1652182294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8035040 con 0x56052e6bb370 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dabfff640 1 -- 172.21.15.5:0/1652182294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f9db8031cf0 con 0x56052e6bb370 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dd553a740 1 -- 172.21.15.5:0/1652182294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9db8005020 con 0x56052e6bb370 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dabfff640 1 -- 172.21.15.5:0/1652182294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f9db80051d0 con 0x56052e6bb370 2024-09-19T06:19:17.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dd553a740 10 client.19542 fetch_fsmap learned FSMap version 48 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dd553a740 1 -- 172.21.15.5:0/1652182294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56052e6c9b10 con 0x56052e6bb370 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dabfff640 1 -- 172.21.15.5:0/1652182294 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9db8005020 con 0x56052e6bb370 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dabfff640 1 -- 172.21.15.5:0/1652182294 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f9db80048d0 con 0x56052e6bb370 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dd553a740 10 client.19542 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dd553a740 10 client.19542 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dd553a740 20 client.19542 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dd553a740 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56052e6cae60 0x56052e6cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dd553a740 4 client.19542 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.056+0000 7f9dd553a740 1 -- 172.21.15.5:0/1652182294 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56052e623860 con 0x56052e6cae60 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.057+0000 7f9dd0efd640 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56052e6cae60 0x56052e6cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.057+0000 7f9dabfff640 10 client.19542 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.057+0000 7f9dd0efd640 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56052e6cae60 0x56052e6cd250 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16294a7c090d543a server_cookie=f3778b750ceefab1 in_seq=0 out_seq=0 2024-09-19T06:19:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.058+0000 7f9dabfff640 1 -- 172.21.15.5:0/1652182294 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56052e623860 con 0x56052e6cae60 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.058+0000 7f9dabfff640 10 client.19542 handle_command_reply: tid=0 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.058+0000 7f9dd553a740 1 client.19542 shutdown 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.058+0000 7f9dd553a740 1 -- 172.21.15.5:0/1652182294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 msgr2=0x56052e6b8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.058+0000 7f9dd553a740 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6b8640 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9db8004770 tx=0x7f9db8032bd0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.058+0000 7f9dd553a740 1 -- 172.21.15.5:0/1652182294 shutdown_connections 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.058+0000 7f9dd553a740 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56052e6cae60 0x56052e6cd250 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.058+0000 7f9dd553a740 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6b8640 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.058+0000 7f9dd553a740 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b1b40 0x56052e6b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.058+0000 7f9dd553a740 1 --2- 172.21.15.5:0/1652182294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b05d0 0x56052e6bad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.058+0000 7f9dd553a740 1 -- 172.21.15.5:0/1652182294 >> 172.21.15.5:0/1652182294 conn(0x56052e679310 msgr2=0x56052e6b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.059+0000 7f9dabfff640 0 client.19542 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.059+0000 7f9dd553a740 1 -- 172.21.15.5:0/1652182294 shutdown_connections 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.059+0000 7f9dd553a740 1 -- 172.21.15.5:0/1652182294 wait complete. 2024-09-19T06:19:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.059+0000 7f9dd553a740 20 client.19542 trim_cache size 0 max 16384 2024-09-19T06:19:17.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.076+0000 7f9dd553a740 1 Processor -- start 2024-09-19T06:19:17.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd553a740 1 -- start start 2024-09-19T06:19:17.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd553a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6c7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd553a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b1b40 0x56052e6b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd553a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd553a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1b3f00 con 0x56052e6bb370 2024-09-19T06:19:17.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd553a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1b5bf0 con 0x56052e6b05d0 2024-09-19T06:19:17.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd553a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1b4690 con 0x56052e6b1b40 2024-09-19T06:19:17.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd0efd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b1b40 0x56052e6b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd0efd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b1b40 0x56052e6b7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34318/0 (socket says 172.21.15.5:34318) 2024-09-19T06:19:17.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd0efd640 1 -- 172.21.15.5:0/9287017 learned_addr learned my addr 172.21.15.5:0/9287017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:17.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/9287017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd0efd640 1 -- 172.21.15.5:0/9287017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 msgr2=0x56052e6c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:17.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd0efd640 1 --2- 172.21.15.5:0/9287017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd0efd640 1 -- 172.21.15.5:0/9287017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 msgr2=0x56052e6cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:17.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.077+0000 7f9dd0efd640 1 --2- 172.21.15.5:0/9287017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.078+0000 7f9dd0efd640 1 -- 172.21.15.5:0/9287017 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052e6bb0b0 con 0x56052e6b1b40 2024-09-19T06:19:17.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.078+0000 7f9dd0efd640 1 --2- 172.21.15.5:0/9287017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b1b40 0x56052e6b7050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0000c00 tx=0x7f9dc0012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b756e5b076ce24f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:17.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.078+0000 7f9db67fc640 1 -- 172.21.15.5:0/9287017 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0004030 con 0x56052e6b1b40 2024-09-19T06:19:17.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.078+0000 7f9db67fc640 1 -- 172.21.15.5:0/9287017 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dc00035f0 con 0x56052e6b1b40 2024-09-19T06:19:17.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.078+0000 7f9db67fc640 1 -- 172.21.15.5:0/9287017 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0010690 con 0x56052e6b1b40 2024-09-19T06:19:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.079+0000 7f9dd553a740 1 -- 172.21.15.5:0/9287017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b1b40 msgr2=0x56052e6b7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.079+0000 7f9dd553a740 1 --2- 172.21.15.5:0/9287017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b1b40 0x56052e6b7050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0000c00 tx=0x7f9dc0012820 comp rx=0 tx=0).stop 2024-09-19T06:19:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.079+0000 7f9dd553a740 1 -- 172.21.15.5:0/9287017 shutdown_connections 2024-09-19T06:19:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.079+0000 7f9dd553a740 1 --2- 172.21.15.5:0/9287017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6bb370 0x56052e6cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.079+0000 7f9dd553a740 1 --2- 172.21.15.5:0/9287017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6b1b40 0x56052e6b7050 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.079+0000 7f9dd553a740 1 --2- 172.21.15.5:0/9287017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.079+0000 7f9dd553a740 1 -- 172.21.15.5:0/9287017 >> 172.21.15.5:0/9287017 conn(0x56052e679310 msgr2=0x56052e6b6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.079+0000 7f9dd553a740 1 -- 172.21.15.5:0/9287017 shutdown_connections 2024-09-19T06:19:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.079+0000 7f9dd553a740 1 -- 172.21.15.5:0/9287017 wait complete. 2024-09-19T06:19:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.080+0000 7f9dd553a740 1 -- start start 2024-09-19T06:19:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.080+0000 7f9dd553a740 1 Processor -- start 2024-09-19T06:19:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.080+0000 7f9dd553a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.080+0000 7f9dd553a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6b1b40 0x56052e6cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dd553a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6bb370 0x56052e6cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dcbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dcbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49532/0 (socket says 172.21.15.5:49532) 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dd553a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1b5520 con 0x56052e6b1b40 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dd0efd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6b1b40 0x56052e6cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dcbfff640 1 -- 172.21.15.5:0/3638887725 learned_addr learned my addr 172.21.15.5:0/3638887725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dd553a740 1 -- 172.21.15.5:0/3638887725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1b4d90 con 0x56052e6b05d0 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dd553a740 1 -- 172.21.15.5:0/3638887725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1b1f00 con 0x56052e6bb370 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dd553a740 10 client.0 _ll_register_callbacks cb 0x7f9dc0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/3638887725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6bb370 0x56052e6cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dcbfff640 1 -- 172.21.15.5:0/3638887725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6bb370 msgr2=0x56052e6cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/3638887725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6bb370 0x56052e6cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dcbfff640 1 -- 172.21.15.5:0/3638887725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6b1b40 msgr2=0x56052e6cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/3638887725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6b1b40 0x56052e6cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.081+0000 7f9dcbfff640 1 -- 172.21.15.5:0/3638887725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052e6ce360 con 0x56052e6b05d0 2024-09-19T06:19:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.082+0000 7f9dcbfff640 1 --2- 172.21.15.5:0/3638887725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6d0460 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9db8004de0 tx=0x7f9db802a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e006039655968349 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.082+0000 7f9dabfff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.082+0000 7f9dabfff640 1 -- 172.21.15.5:0/3638887725 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8035040 con 0x56052e6b05d0 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.082+0000 7f9dabfff640 1 -- 172.21.15.5:0/3638887725 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f9db8032de0 con 0x56052e6b05d0 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.082+0000 7f9dd553a740 1 -- 172.21.15.5:0/3638887725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dc00035f0 con 0x56052e6b05d0 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.082+0000 7f9dabfff640 1 -- 172.21.15.5:0/3638887725 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f9db8067050 con 0x56052e6b05d0 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.082+0000 7f9dd553a740 10 client.28928 fetch_fsmap learned FSMap version 48 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.082+0000 7f9dd553a740 1 -- 172.21.15.5:0/3638887725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56052e735610 con 0x56052e6b05d0 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.082+0000 7f9dabfff640 1 -- 172.21.15.5:0/3638887725 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9db8009520 con 0x56052e6b05d0 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.082+0000 7f9dabfff640 1 -- 172.21.15.5:0/3638887725 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f9db8032790 con 0x56052e6b05d0 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.083+0000 7f9dd553a740 10 client.28928 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.083+0000 7f9dd553a740 10 client.28928 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.083+0000 7f9dd553a740 20 client.28928 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.083+0000 7f9dd553a740 1 --2- 172.21.15.5:0/3638887725 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56052e735c80 0x56052e738070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.083+0000 7f9dd553a740 4 client.28928 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:19:17.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.083+0000 7f9dd553a740 1 -- 172.21.15.5:0/3638887725 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9dc00035f0 con 0x56052e735c80 2024-09-19T06:19:17.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.083+0000 7f9dd0efd640 1 --2- 172.21.15.5:0/3638887725 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56052e735c80 0x56052e738070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.083+0000 7f9dabfff640 10 client.28928 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:19:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.083+0000 7f9dd0efd640 1 --2- 172.21.15.5:0/3638887725 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56052e735c80 0x56052e738070 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb640cb17d6de0ed server_cookie=ce940f1bfa8b167b in_seq=0 out_seq=0 2024-09-19T06:19:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.083+0000 7f9dabfff640 1 -- 172.21.15.5:0/3638887725 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9dc00035f0 con 0x56052e735c80 2024-09-19T06:19:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.083+0000 7f9dabfff640 10 client.28928 handle_command_reply: tid=0 2024-09-19T06:19:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.083+0000 7f9dd553a740 1 client.28928 shutdown 2024-09-19T06:19:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.084+0000 7f9dd553a740 1 -- 172.21.15.5:0/3638887725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 msgr2=0x56052e6d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.084+0000 7f9dd553a740 1 --2- 172.21.15.5:0/3638887725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6d0460 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9db8004de0 tx=0x7f9db802a4d0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.084+0000 7f9dd553a740 1 -- 172.21.15.5:0/3638887725 shutdown_connections 2024-09-19T06:19:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.084+0000 7f9dd553a740 1 --2- 172.21.15.5:0/3638887725 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56052e735c80 0x56052e738070 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.084+0000 7f9dd553a740 1 --2- 172.21.15.5:0/3638887725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56052e6bb370 0x56052e6cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.084+0000 7f9dabfff640 0 client.28928 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:19:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.084+0000 7f9dd553a740 1 --2- 172.21.15.5:0/3638887725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052e6b1b40 0x56052e6cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.084+0000 7f9dd553a740 1 --2- 172.21.15.5:0/3638887725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56052e6b05d0 0x56052e6d0460 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.084+0000 7f9dd553a740 1 -- 172.21.15.5:0/3638887725 >> 172.21.15.5:0/3638887725 conn(0x56052e679310 msgr2=0x56052e6b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.084+0000 7f9dd553a740 1 -- 172.21.15.5:0/3638887725 shutdown_connections 2024-09-19T06:19:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.084+0000 7f9dd553a740 1 -- 172.21.15.5:0/3638887725 wait complete. 2024-09-19T06:19:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.084+0000 7f9dd553a740 20 client.28928 trim_cache size 0 max 16384 2024-09-19T06:19:17.091 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:19:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.085+0000 7f9dd3161640 1 -- 172.21.15.5:0/693921662 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9dbc0a33b0 msgr2=0x7f9dbc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.085+0000 7f9dd3161640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9dbc0a33b0 0x7f9dbc0a5870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40989c0 tx=0x7f9dc4066f50 comp rx=0 tx=0).stop 2024-09-19T06:19:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.086+0000 7f9dd3161640 1 -- 172.21.15.5:0/693921662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dcc102ef0 msgr2=0x7f9dcc10ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.086+0000 7f9dd3161640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dcc102ef0 0x7f9dcc10ec50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400eb10 tx=0x7f9dc400c2d0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.086+0000 7f9dd3161640 1 -- 172.21.15.5:0/693921662 shutdown_connections 2024-09-19T06:19:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.086+0000 7f9dd3161640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dcc102ef0 0x7f9dcc10ec50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.086+0000 7f9dd3161640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9dbc0a33b0 0x7f9dbc0a5870 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.086+0000 7f9dd3161640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc102570 0x7f9dcc10e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.086+0000 7f9dd3161640 1 --2- 172.21.15.5:0/693921662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dcc100e20 0x7f9dcc10e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.086+0000 7f9dd3161640 1 -- 172.21.15.5:0/693921662 >> 172.21.15.5:0/693921662 conn(0x7f9dcc0fc7d0 msgr2=0x7f9dcc104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.086+0000 7f9dd3161640 1 -- 172.21.15.5:0/693921662 shutdown_connections 2024-09-19T06:19:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:17.086+0000 7f9dd3161640 1 -- 172.21.15.5:0/693921662 wait complete. 2024-09-19T06:19:17.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 27245 9.11 MB/sec execute 41 sec latency 103.261 ms 2024-09-19T06:19:17.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:19:18.210 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:19:18.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 27403 9.15 MB/sec execute 42 sec latency 222.207 ms 2024-09-19T06:19:18.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:18 smithi005 ceph-mon[30955]: pgmap v502: 97 pgs: 97 active+clean; 610 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 290 op/s 2024-09-19T06:19:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:18 smithi157 ceph-mon[35346]: pgmap v502: 97 pgs: 97 active+clean; 610 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 290 op/s 2024-09-19T06:19:18.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.888+0000 7eff5c91e640 1 -- 172.21.15.5:0/1214030781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff58102790 msgr2=0x7eff581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:18.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.888+0000 7eff5c91e640 1 --2- 172.21.15.5:0/1214030781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff58102790 0x7eff581067e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009900 tx=0x7eff4c02a410 comp rx=0 tx=0).stop 2024-09-19T06:19:18.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.888+0000 7eff5c91e640 1 -- 172.21.15.5:0/1214030781 shutdown_connections 2024-09-19T06:19:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.888+0000 7eff5c91e640 1 --2- 172.21.15.5:0/1214030781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff58106f10 0x7eff5810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.888+0000 7eff5c91e640 1 --2- 172.21.15.5:0/1214030781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff58102790 0x7eff581067e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.888+0000 7eff5c91e640 1 --2- 172.21.15.5:0/1214030781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff58101de0 0x7eff581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.888+0000 7eff5c91e640 1 -- 172.21.15.5:0/1214030781 >> 172.21.15.5:0/1214030781 conn(0x7eff580fd8c0 msgr2=0x7eff580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.888+0000 7eff5c91e640 1 -- 172.21.15.5:0/1214030781 shutdown_connections 2024-09-19T06:19:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.888+0000 7eff5c91e640 1 -- 172.21.15.5:0/1214030781 wait complete. 2024-09-19T06:19:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.889+0000 7eff5c91e640 1 Processor -- start 2024-09-19T06:19:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.889+0000 7eff5c91e640 1 -- start start 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.889+0000 7eff5c91e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff58101de0 0x7eff58198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5c91e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff58102790 0x7eff581988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5c91e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff58106f10 0x7eff5819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5c91e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff58111e70 con 0x7eff58101de0 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5c91e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff58111cf0 con 0x7eff58106f10 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5c91e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff58111ff0 con 0x7eff58102790 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff58101de0 0x7eff58198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff58101de0 0x7eff58198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37954/0 (socket says 172.21.15.5:37954) 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5659c640 1 -- 172.21.15.5:0/4077128476 learned_addr learned my addr 172.21.15.5:0/4077128476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff56d9d640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff58106f10 0x7eff5819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff55d9b640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff58102790 0x7eff581988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5659c640 1 -- 172.21.15.5:0/4077128476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff58102790 msgr2=0x7eff581988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5659c640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff58102790 0x7eff581988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5659c640 1 -- 172.21.15.5:0/4077128476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff58106f10 msgr2=0x7eff5819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5659c640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff58106f10 0x7eff5819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5659c640 1 -- 172.21.15.5:0/4077128476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5819d340 con 0x7eff58101de0 2024-09-19T06:19:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff56d9d640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff58106f10 0x7eff5819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.890+0000 7eff5659c640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff58101de0 0x7eff58198390 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7eff4800e9f0 tx=0x7eff4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.891+0000 7eff477fe640 1 -- 172.21.15.5:0/4077128476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4800ccc0 con 0x7eff58101de0 2024-09-19T06:19:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.891+0000 7eff477fe640 1 -- 172.21.15.5:0/4077128476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff480108e0 con 0x7eff58101de0 2024-09-19T06:19:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.891+0000 7eff5c91e640 1 -- 172.21.15.5:0/4077128476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5819d630 con 0x7eff58101de0 2024-09-19T06:19:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.891+0000 7eff477fe640 1 -- 172.21.15.5:0/4077128476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff480187c0 con 0x7eff58101de0 2024-09-19T06:19:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.891+0000 7eff5c91e640 1 -- 172.21.15.5:0/4077128476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff581a4f10 con 0x7eff58101de0 2024-09-19T06:19:18.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.894+0000 7eff5ecf7740 1 Processor -- start 2024-09-19T06:19:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:18 smithi049 ceph-mon[34640]: pgmap v502: 97 pgs: 97 active+clean; 610 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 290 op/s 2024-09-19T06:19:18.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.894+0000 7eff5ecf7740 1 -- start start 2024-09-19T06:19:18.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.895+0000 7eff5ecf7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e2121b40 0x5596e2120090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.895+0000 7eff477fe640 1 -- 172.21.15.5:0/4077128476 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7eff48018960 con 0x7eff58101de0 2024-09-19T06:19:18.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.895+0000 7eff5ecf7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e21205d0 0x5596e2120a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.895+0000 7eff5ecf7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e212b370 0x5596e212d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.895+0000 7eff5ecf7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5596e20862f0 con 0x5596e212b370 2024-09-19T06:19:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.895+0000 7eff5ecf7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5596e202dcf0 con 0x5596e21205d0 2024-09-19T06:19:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.895+0000 7eff5ecf7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5596e20822b0 con 0x5596e2121b40 2024-09-19T06:19:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.895+0000 7eff55d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e212b370 0x5596e212d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.895+0000 7eff55d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e212b370 0x5596e212d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37962/0 (socket says 172.21.15.5:37962) 2024-09-19T06:19:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.895+0000 7eff55d9b640 1 -- 172.21.15.5:0/451230461 learned_addr learned my addr 172.21.15.5:0/451230461 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.895+0000 7eff56d9d640 1 --2- 172.21.15.5:0/451230461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e2121b40 0x5596e2120090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.895+0000 7eff5659c640 1 --2- 172.21.15.5:0/451230461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e21205d0 0x5596e2120a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.896+0000 7eff55d9b640 1 -- 172.21.15.5:0/451230461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e2121b40 msgr2=0x5596e2120090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.896+0000 7eff55d9b640 1 --2- 172.21.15.5:0/451230461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e2121b40 0x5596e2120090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.896+0000 7eff55d9b640 1 -- 172.21.15.5:0/451230461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e21205d0 msgr2=0x5596e2120a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:18.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.896+0000 7eff55d9b640 1 --2- 172.21.15.5:0/451230461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e21205d0 0x5596e2120a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.896+0000 7eff55d9b640 1 -- 172.21.15.5:0/451230461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e20123a0 con 0x5596e212b370 2024-09-19T06:19:18.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.896+0000 7eff5659c640 1 --2- 172.21.15.5:0/451230461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e21205d0 0x5596e2120a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:19:18.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.896+0000 7eff55d9b640 1 --2- 172.21.15.5:0/451230461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e212b370 0x5596e212d760 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009900 tx=0x7eff4c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154414717c2454c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:18.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.896+0000 7eff44ff9640 1 -- 172.21.15.5:0/451230461 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c035030 con 0x5596e212b370 2024-09-19T06:19:18.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.896+0000 7eff44ff9640 1 -- 172.21.15.5:0/451230461 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff4c042070 con 0x5596e212b370 2024-09-19T06:19:18.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.896+0000 7eff44ff9640 1 -- 172.21.15.5:0/451230461 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c0046c0 con 0x5596e212b370 2024-09-19T06:19:18.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.897+0000 7eff477fe640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7eff400a33b0 0x7eff400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.897+0000 7eff56d9d640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7eff400a33b0 0x7eff400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.897+0000 7eff5ecf7740 1 -- 172.21.15.5:0/451230461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e212b370 msgr2=0x5596e212d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:18.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.897+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/451230461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e212b370 0x5596e212d760 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009900 tx=0x7eff4c0040c0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.897+0000 7eff5ecf7740 1 -- 172.21.15.5:0/451230461 shutdown_connections 2024-09-19T06:19:18.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.897+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/451230461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e212b370 0x5596e212d760 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.897+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/451230461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e21205d0 0x5596e2120a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.897+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/451230461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e2121b40 0x5596e2120090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.898+0000 7eff5ecf7740 1 -- 172.21.15.5:0/451230461 >> 172.21.15.5:0/451230461 conn(0x5596e20e9310 msgr2=0x5596e211cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:18.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.898+0000 7eff5ecf7740 1 -- 172.21.15.5:0/451230461 shutdown_connections 2024-09-19T06:19:18.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.898+0000 7eff477fe640 1 -- 172.21.15.5:0/4077128476 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7eff480d0180 con 0x7eff58101de0 2024-09-19T06:19:18.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.898+0000 7eff56d9d640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7eff400a33b0 0x7eff400a5870 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7eff50005fd0 tx=0x7eff50009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:18.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.898+0000 7eff5ecf7740 1 -- 172.21.15.5:0/451230461 wait complete. 2024-09-19T06:19:18.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.898+0000 7eff5ecf7740 1 -- start start 2024-09-19T06:19:18.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.899+0000 7eff5ecf7740 1 Processor -- start 2024-09-19T06:19:18.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.899+0000 7eff5ecf7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e21205d0 0x5596e212ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.899+0000 7eff5ecf7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e2121b40 0x5596e2128030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.899+0000 7eff5ecf7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e212b370 0x5596e2128640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.899+0000 7eff5ecf7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5596e212e010 con 0x5596e21205d0 2024-09-19T06:19:18.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.899+0000 7eff5ecf7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5596e2041060 con 0x5596e2121b40 2024-09-19T06:19:18.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.899+0000 7eff5ecf7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5596e212e190 con 0x5596e212b370 2024-09-19T06:19:18.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.899+0000 7eff5ecf7740 10 client.0 _ll_register_callbacks cb 0x5596e1d948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:18.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.899+0000 7eff5659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e21205d0 0x5596e212ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.899+0000 7eff5659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e21205d0 0x5596e212ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37966/0 (socket says 172.21.15.5:37966) 2024-09-19T06:19:18.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.899+0000 7eff5659c640 1 -- 172.21.15.5:0/2865787825 learned_addr learned my addr 172.21.15.5:0/2865787825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:18.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.900+0000 7eff5659c640 1 -- 172.21.15.5:0/2865787825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e212b370 msgr2=0x5596e2128640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:18.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.900+0000 7eff56d9d640 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e212b370 0x5596e2128640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.900+0000 7eff5659c640 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e212b370 0x5596e2128640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.900+0000 7eff55d9b640 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e2121b40 0x5596e2128030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.900+0000 7eff5659c640 1 -- 172.21.15.5:0/2865787825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e2121b40 msgr2=0x5596e2128030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.900+0000 7eff5659c640 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e2121b40 0x5596e2128030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.900+0000 7eff5659c640 1 -- 172.21.15.5:0/2865787825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e1df73c0 con 0x5596e21205d0 2024-09-19T06:19:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.900+0000 7eff56d9d640 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e212b370 0x5596e2128640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:19:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.900+0000 7eff55d9b640 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e2121b40 0x5596e2128030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:19:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.901+0000 7eff5659c640 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e21205d0 0x5596e212ad80 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7eff4800e9c0 tx=0x7eff4809c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3cf8d8c6f976f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.901+0000 7eff3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:19:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.901+0000 7eff3dffb640 1 -- 172.21.15.5:0/2865787825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4809cb80 con 0x5596e21205d0 2024-09-19T06:19:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.901+0000 7eff3dffb640 1 -- 172.21.15.5:0/2865787825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7eff48010430 con 0x5596e21205d0 2024-09-19T06:19:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.901+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2865787825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff4c042070 con 0x5596e21205d0 2024-09-19T06:19:18.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.901+0000 7eff3dffb640 1 -- 172.21.15.5:0/2865787825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7eff4809c480 con 0x5596e21205d0 2024-09-19T06:19:18.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.901+0000 7eff5ecf7740 10 client.19557 fetch_fsmap learned FSMap version 48 2024-09-19T06:19:18.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.901+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2865787825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5596e21422f0 con 0x5596e21205d0 2024-09-19T06:19:18.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.901+0000 7eff3dffb640 1 -- 172.21.15.5:0/2865787825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff4c042070 con 0x5596e21205d0 2024-09-19T06:19:18.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.902+0000 7eff3dffb640 1 -- 172.21.15.5:0/2865787825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7eff480b51f0 con 0x5596e21205d0 2024-09-19T06:19:18.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.902+0000 7eff5ecf7740 10 client.19557 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:19:18.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.902+0000 7eff5ecf7740 10 client.19557 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:19:18.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.902+0000 7eff5ecf7740 20 client.19557 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.902+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5596e2143850 0x5596e2145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.902+0000 7eff5ecf7740 4 client.19557 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.902+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2865787825 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596e2093860 con 0x5596e2143850 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.902+0000 7eff55d9b640 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5596e2143850 0x5596e2145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.903+0000 7eff55d9b640 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5596e2143850 0x5596e2145ca0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f2017ffcb8f14082 server_cookie=81f6f6b7f1a58865 in_seq=0 out_seq=0 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.903+0000 7eff3dffb640 10 client.19557 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.903+0000 7eff3dffb640 1 -- 172.21.15.5:0/2865787825 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596e2093860 con 0x5596e2143850 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.903+0000 7eff3dffb640 10 client.19557 handle_command_reply: tid=0 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.903+0000 7eff5ecf7740 1 client.19557 shutdown 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.904+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2865787825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e21205d0 msgr2=0x5596e212ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.904+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e21205d0 0x5596e212ad80 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7eff4800e9c0 tx=0x7eff4809c380 comp rx=0 tx=0).stop 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.904+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2865787825 shutdown_connections 2024-09-19T06:19:18.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.904+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5596e2143850 0x5596e2145ca0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.904+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e212b370 0x5596e2128640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.904+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e2121b40 0x5596e2128030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.904+0000 7eff3dffb640 0 client.19557 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:19:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.904+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/2865787825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e21205d0 0x5596e212ad80 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.904+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2865787825 >> 172.21.15.5:0/2865787825 conn(0x5596e20e9310 msgr2=0x5596e2122960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.904+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2865787825 shutdown_connections 2024-09-19T06:19:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.904+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2865787825 wait complete. 2024-09-19T06:19:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.904+0000 7eff5ecf7740 20 client.19557 trim_cache size 0 max 16384 2024-09-19T06:19:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.923+0000 7eff5ecf7740 1 Processor -- start 2024-09-19T06:19:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.923+0000 7eff5ecf7740 1 -- start start 2024-09-19T06:19:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.923+0000 7eff5ecf7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 0x5596e2123120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.923+0000 7eff5ecf7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e2121b40 0x5596e2123660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.923+0000 7eff5ecf7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e212b370 0x5596e214b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.923+0000 7eff5ecf7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff581ab5a0 con 0x5596e2121b40 2024-09-19T06:19:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.923+0000 7eff5ecf7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff581ad290 con 0x5596e212b370 2024-09-19T06:19:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.923+0000 7eff5ecf7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff581abd30 con 0x5596e21205d0 2024-09-19T06:19:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.923+0000 7eff5659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 0x5596e2123120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.923+0000 7eff5659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 0x5596e2123120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34350/0 (socket says 172.21.15.5:34350) 2024-09-19T06:19:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.924+0000 7eff5659c640 1 -- 172.21.15.5:0/586808470 learned_addr learned my addr 172.21.15.5:0/586808470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.924+0000 7eff55d9b640 1 --2- 172.21.15.5:0/586808470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e2121b40 0x5596e2123660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.924+0000 7eff56d9d640 1 --2- 172.21.15.5:0/586808470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e212b370 0x5596e214b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.924+0000 7eff5659c640 1 -- 172.21.15.5:0/586808470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e212b370 msgr2=0x5596e214b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.924+0000 7eff5659c640 1 --2- 172.21.15.5:0/586808470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e212b370 0x5596e214b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.924+0000 7eff5659c640 1 -- 172.21.15.5:0/586808470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e2121b40 msgr2=0x5596e2123660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.924+0000 7eff5659c640 1 --2- 172.21.15.5:0/586808470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e2121b40 0x5596e2123660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.924+0000 7eff5659c640 1 -- 172.21.15.5:0/586808470 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e2128fd0 con 0x5596e21205d0 2024-09-19T06:19:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.924+0000 7eff55d9b640 1 --2- 172.21.15.5:0/586808470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e2121b40 0x5596e2123660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.924+0000 7eff5659c640 1 --2- 172.21.15.5:0/586808470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 0x5596e2123120 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7eff4809c640 tx=0x7eff48098440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ca7cea5a7431a38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.925+0000 7eff3ffff640 1 -- 172.21.15.5:0/586808470 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4809cb80 con 0x5596e21205d0 2024-09-19T06:19:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.925+0000 7eff3ffff640 1 -- 172.21.15.5:0/586808470 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff480a50f0 con 0x5596e21205d0 2024-09-19T06:19:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.925+0000 7eff3ffff640 1 -- 172.21.15.5:0/586808470 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4809b340 con 0x5596e21205d0 2024-09-19T06:19:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.926+0000 7eff5ecf7740 1 -- 172.21.15.5:0/586808470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 msgr2=0x5596e2123120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.926+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/586808470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 0x5596e2123120 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7eff4809c640 tx=0x7eff48098440 comp rx=0 tx=0).stop 2024-09-19T06:19:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.926+0000 7eff5ecf7740 1 -- 172.21.15.5:0/586808470 shutdown_connections 2024-09-19T06:19:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.926+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/586808470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e212b370 0x5596e214b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.926+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/586808470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e2121b40 0x5596e2123660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.926+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/586808470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 0x5596e2123120 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.926+0000 7eff5ecf7740 1 -- 172.21.15.5:0/586808470 >> 172.21.15.5:0/586808470 conn(0x5596e20e9310 msgr2=0x5596e212a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.926+0000 7eff5ecf7740 1 -- 172.21.15.5:0/586808470 shutdown_connections 2024-09-19T06:19:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.926+0000 7eff5ecf7740 1 -- 172.21.15.5:0/586808470 wait complete. 2024-09-19T06:19:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.926+0000 7eff5ecf7740 1 -- start start 2024-09-19T06:19:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.926+0000 7eff5ecf7740 1 Processor -- start 2024-09-19T06:19:18.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5ecf7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 0x5596e2137cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5ecf7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e2121b40 0x5596e21381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5ecf7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e212b370 0x5596e2138730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5ecf7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff581acbc0 con 0x5596e2121b40 2024-09-19T06:19:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5ecf7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff581ac430 con 0x5596e212b370 2024-09-19T06:19:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5ecf7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff581a95a0 con 0x5596e21205d0 2024-09-19T06:19:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5ecf7740 10 client.0 _ll_register_callbacks cb 0x7eff480987d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 0x5596e2137cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 0x5596e2137cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34360/0 (socket says 172.21.15.5:34360) 2024-09-19T06:19:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5659c640 1 -- 172.21.15.5:0/2760631397 learned_addr learned my addr 172.21.15.5:0/2760631397 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff55d9b640 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e2121b40 0x5596e21381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff56d9d640 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e212b370 0x5596e2138730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5659c640 1 -- 172.21.15.5:0/2760631397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e212b370 msgr2=0x5596e2138730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5659c640 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e212b370 0x5596e2138730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5659c640 1 -- 172.21.15.5:0/2760631397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e2121b40 msgr2=0x5596e21381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5659c640 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e2121b40 0x5596e21381f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff5659c640 1 -- 172.21.15.5:0/2760631397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e21ad820 con 0x5596e21205d0 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.927+0000 7eff55d9b640 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e2121b40 0x5596e21381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.928+0000 7eff5659c640 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 0x5596e2137cb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7eff480983c0 tx=0x7eff4809c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd5be43ae25d8d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.928+0000 7eff3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.928+0000 7eff3dffb640 1 -- 172.21.15.5:0/2760631397 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48010630 con 0x5596e21205d0 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.928+0000 7eff3dffb640 1 -- 172.21.15.5:0/2760631397 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7eff4809b340 con 0x5596e21205d0 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.928+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2760631397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff480a50f0 con 0x5596e21205d0 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.928+0000 7eff3dffb640 1 -- 172.21.15.5:0/2760631397 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7eff480a0830 con 0x5596e21205d0 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.928+0000 7eff5ecf7740 10 client.28891 fetch_fsmap learned FSMap version 48 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.928+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2760631397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5596e21ad820 con 0x5596e21205d0 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.929+0000 7eff3dffb640 1 -- 172.21.15.5:0/2760631397 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff480a50f0 con 0x5596e21205d0 2024-09-19T06:19:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.929+0000 7eff3dffb640 1 -- 172.21.15.5:0/2760631397 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7eff480bc130 con 0x5596e21205d0 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.929+0000 7eff5ecf7740 10 client.28891 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.929+0000 7eff5ecf7740 10 client.28891 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.929+0000 7eff5ecf7740 20 client.28891 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.929+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5596e21aee90 0x5596e21b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.929+0000 7eff5ecf7740 4 client.28891 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.929+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2760631397 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596e208dd70 con 0x5596e21aee90 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.929+0000 7eff55d9b640 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5596e21aee90 0x5596e21b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff3dffb640 10 client.28891 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff55d9b640 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5596e21aee90 0x5596e21b12b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac6bc93b7f245bc0 server_cookie=79fd80217bd9f38f in_seq=0 out_seq=0 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff3dffb640 1 -- 172.21.15.5:0/2760631397 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596e208dd70 con 0x5596e21aee90 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff3dffb640 10 client.28891 handle_command_reply: tid=0 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff5ecf7740 1 client.28891 shutdown 2024-09-19T06:19:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2760631397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 msgr2=0x5596e2137cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 0x5596e2137cb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7eff480983c0 tx=0x7eff4809c380 comp rx=0 tx=0).stop 2024-09-19T06:19:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2760631397 shutdown_connections 2024-09-19T06:19:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5596e21aee90 0x5596e21b12b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596e212b370 0x5596e2138730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596e2121b40 0x5596e21381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff5ecf7740 1 --2- 172.21.15.5:0/2760631397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5596e21205d0 0x5596e2137cb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2760631397 >> 172.21.15.5:0/2760631397 conn(0x5596e20e9310 msgr2=0x5596e211b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.930+0000 7eff3dffb640 0 client.28891 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:19:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.931+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2760631397 shutdown_connections 2024-09-19T06:19:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.931+0000 7eff5ecf7740 1 -- 172.21.15.5:0/2760631397 wait complete. 2024-09-19T06:19:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.931+0000 7eff5ecf7740 20 client.28891 trim_cache size 0 max 16384 2024-09-19T06:19:18.937 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:19:18.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.931+0000 7eff5c91e640 1 -- 172.21.15.5:0/4077128476 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7eff400a33b0 msgr2=0x7eff400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:18.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.931+0000 7eff5c91e640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7eff400a33b0 0x7eff400a5870 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7eff50005fd0 tx=0x7eff50009500 comp rx=0 tx=0).stop 2024-09-19T06:19:18.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.932+0000 7eff5c91e640 1 -- 172.21.15.5:0/4077128476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff58101de0 msgr2=0x7eff58198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.932+0000 7eff5c91e640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff58101de0 0x7eff58198390 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7eff4800e9f0 tx=0x7eff4800edf0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.932+0000 7eff5c91e640 1 -- 172.21.15.5:0/4077128476 shutdown_connections 2024-09-19T06:19:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.932+0000 7eff5c91e640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff58106f10 0x7eff5819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.932+0000 7eff5c91e640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7eff400a33b0 0x7eff400a5870 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.932+0000 7eff5c91e640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff58102790 0x7eff581988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.932+0000 7eff5c91e640 1 --2- 172.21.15.5:0/4077128476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff58101de0 0x7eff58198390 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.932+0000 7eff5c91e640 1 -- 172.21.15.5:0/4077128476 >> 172.21.15.5:0/4077128476 conn(0x7eff580fd8c0 msgr2=0x7eff580fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.932+0000 7eff5c91e640 1 -- 172.21.15.5:0/4077128476 shutdown_connections 2024-09-19T06:19:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:18.932+0000 7eff5c91e640 1 -- 172.21.15.5:0/4077128476 wait complete. 2024-09-19T06:19:19.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 27520 9.06 MB/sec execute 43 sec latency 373.780 ms 2024-09-19T06:19:19.746 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:19:20.126 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:19:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.258+0000 7f1c69627640 1 -- 172.21.15.5:0/2361267662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c641069b0 msgr2=0x7f1c64106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.258+0000 7f1c69627640 1 --2- 172.21.15.5:0/2361267662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c641069b0 0x7f1c64106d90 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009930 tx=0x7f1c5402a420 comp rx=0 tx=0).stop 2024-09-19T06:19:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.258+0000 7f1c69627640 1 -- 172.21.15.5:0/2361267662 shutdown_connections 2024-09-19T06:19:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.258+0000 7f1c69627640 1 --2- 172.21.15.5:0/2361267662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c64101340 0x7f1c6410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.258+0000 7f1c69627640 1 --2- 172.21.15.5:0/2361267662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c641009a0 0x7f1c64100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.258+0000 7f1c69627640 1 --2- 172.21.15.5:0/2361267662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c641069b0 0x7f1c64106d90 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.258+0000 7f1c69627640 1 -- 172.21.15.5:0/2361267662 >> 172.21.15.5:0/2361267662 conn(0x7f1c640779a0 msgr2=0x7f1c640fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.258+0000 7f1c69627640 1 -- 172.21.15.5:0/2361267662 shutdown_connections 2024-09-19T06:19:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.258+0000 7f1c69627640 1 -- 172.21.15.5:0/2361267662 wait complete. 2024-09-19T06:19:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.259+0000 7f1c69627640 1 Processor -- start 2024-09-19T06:19:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.259+0000 7f1c69627640 1 -- start start 2024-09-19T06:19:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c69627640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c641009a0 0x7f1c6419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c69627640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c64101340 0x7f1c6419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c69627640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c641069b0 0x7f1c6419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c69627640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c64110c70 con 0x7f1c64101340 2024-09-19T06:19:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c69627640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c64110af0 con 0x7f1c641009a0 2024-09-19T06:19:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c69627640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c64110df0 con 0x7f1c641069b0 2024-09-19T06:19:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c627fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c64101340 0x7f1c6419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c62ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c641009a0 0x7f1c6419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c627fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c64101340 0x7f1c6419ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38010/0 (socket says 172.21.15.5:38010) 2024-09-19T06:19:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c627fc640 1 -- 172.21.15.5:0/3102121008 learned_addr learned my addr 172.21.15.5:0/3102121008 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c637fe640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c641069b0 0x7f1c6419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c627fc640 1 -- 172.21.15.5:0/3102121008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c641069b0 msgr2=0x7f1c6419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c627fc640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c641069b0 0x7f1c6419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c627fc640 1 -- 172.21.15.5:0/3102121008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c641009a0 msgr2=0x7f1c6419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c627fc640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c641009a0 0x7f1c6419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c627fc640 1 -- 172.21.15.5:0/3102121008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6419f560 con 0x7f1c64101340 2024-09-19T06:19:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c637fe640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c641069b0 0x7f1c6419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c641009a0 0x7f1c6419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.260+0000 7f1c627fc640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c64101340 0x7f1c6419ac10 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800e9f0 tx=0x7f1c5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.261+0000 7f1c4bfff640 1 -- 172.21.15.5:0/3102121008 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800cd80 con 0x7f1c64101340 2024-09-19T06:19:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.261+0000 7f1c69627640 1 -- 172.21.15.5:0/3102121008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6419f850 con 0x7f1c64101340 2024-09-19T06:19:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.261+0000 7f1c4bfff640 1 -- 172.21.15.5:0/3102121008 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c580108e0 con 0x7f1c64101340 2024-09-19T06:19:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.261+0000 7f1c4bfff640 1 -- 172.21.15.5:0/3102121008 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c58018920 con 0x7f1c64101340 2024-09-19T06:19:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.261+0000 7f1c69627640 1 -- 172.21.15.5:0/3102121008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c641a70e0 con 0x7f1c64101340 2024-09-19T06:19:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.264+0000 7f1c4bfff640 1 -- 172.21.15.5:0/3102121008 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1c58010a90 con 0x7f1c64101340 2024-09-19T06:19:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.264+0000 7f1c6c3c2740 1 Processor -- start 2024-09-19T06:19:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.264+0000 7f1c6c3c2740 1 -- start start 2024-09-19T06:19:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c6c3c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290c3af0 0x55de290c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c6c3c2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c6c3c2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 0x55de290d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c6c3c2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55de290282f0 con 0x55de290c4410 2024-09-19T06:19:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c6c3c2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55de28fd0270 con 0x55de290cecb0 2024-09-19T06:19:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c6c3c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55de290242b0 con 0x55de290c3af0 2024-09-19T06:19:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c4bfff640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1c4c0a3430 0x7f1c4c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c637fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290c3af0 0x55de290c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c627fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 0x55de290d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c637fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290c3af0 0x55de290c3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34378/0 (socket says 172.21.15.5:34378) 2024-09-19T06:19:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c637fe640 1 -- 172.21.15.5:0/320112108 learned_addr learned my addr 172.21.15.5:0/320112108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c637fe640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1c4c0a3430 0x7f1c4c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/320112108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c4bfff640 1 -- 172.21.15.5:0/3102121008 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f1c580cff70 con 0x7f1c64101340 2024-09-19T06:19:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c627fc640 1 -- 172.21.15.5:0/320112108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290c3af0 msgr2=0x55de290c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c627fc640 1 --2- 172.21.15.5:0/320112108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290c3af0 0x55de290c3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c627fc640 1 -- 172.21.15.5:0/320112108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 msgr2=0x55de290ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.265+0000 7f1c627fc640 1 --2- 172.21.15.5:0/320112108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.266+0000 7f1c627fc640 1 -- 172.21.15.5:0/320112108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de28fcf340 con 0x55de290cecb0 2024-09-19T06:19:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.266+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/320112108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.266+0000 7f1c627fc640 1 --2- 172.21.15.5:0/320112108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 0x55de290d10a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1c580cd6a0 tx=0x7f1c5806e120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b39fcac030f3a0f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.266+0000 7f1c637fe640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1c4c0a3430 0x7f1c4c0a58f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009fd0 tx=0x7f1c5c011440 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.266+0000 7f1c497fa640 1 -- 172.21.15.5:0/320112108 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c580a05e0 con 0x55de290cecb0 2024-09-19T06:19:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.266+0000 7f1c497fa640 1 -- 172.21.15.5:0/320112108 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c580a4ea0 con 0x55de290cecb0 2024-09-19T06:19:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.266+0000 7f1c497fa640 1 -- 172.21.15.5:0/320112108 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c580bc0d0 con 0x55de290cecb0 2024-09-19T06:19:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.267+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/320112108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 msgr2=0x55de290d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.267+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/320112108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 0x55de290d10a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1c580cd6a0 tx=0x7f1c5806e120 comp rx=0 tx=0).stop 2024-09-19T06:19:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.267+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/320112108 shutdown_connections 2024-09-19T06:19:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.267+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/320112108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 0x55de290d10a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.267+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/320112108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.267+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/320112108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290c3af0 0x55de290c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.267+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/320112108 >> 172.21.15.5:0/320112108 conn(0x55de290c0060 msgr2=0x55de290c0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.267+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/320112108 shutdown_connections 2024-09-19T06:19:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.267+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/320112108 wait complete. 2024-09-19T06:19:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.268+0000 7f1c6c3c2740 1 -- start start 2024-09-19T06:19:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.268+0000 7f1c6c3c2740 1 Processor -- start 2024-09-19T06:19:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.268+0000 7f1c6c3c2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c3af0 0x55de290cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.268+0000 7f1c6c3c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290c4410 0x55de290cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.268+0000 7f1c6c3c2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 0x55de290cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.268+0000 7f1c6c3c2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55de290d1880 con 0x55de290c3af0 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.268+0000 7f1c6c3c2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55de28fb1290 con 0x55de290cecb0 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.268+0000 7f1c6c3c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55de290d1a00 con 0x55de290c4410 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.268+0000 7f1c6c3c2740 10 client.0 _ll_register_callbacks cb 0x55de28fe2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c637fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 0x55de290cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c637fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 0x55de290cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49608/0 (socket says 172.21.15.5:49608) 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c637fe640 1 -- 172.21.15.5:0/3658073843 learned_addr learned my addr 172.21.15.5:0/3658073843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c3af0 0x55de290cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c637fe640 1 -- 172.21.15.5:0/3658073843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290c4410 msgr2=0x55de290cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c637fe640 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290c4410 0x55de290cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c637fe640 1 -- 172.21.15.5:0/3658073843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c3af0 msgr2=0x55de290cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c637fe640 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c3af0 0x55de290cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c637fe640 1 -- 172.21.15.5:0/3658073843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de28d3b210 con 0x55de290cecb0 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c3af0 0x55de290cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c637fe640 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 0x55de290cc600 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c003e40 tx=0x7f1c5c009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c2af3a33f2f601 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c46ff5640 1 -- 172.21.15.5:0/3658073843 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c004810 con 0x55de290cecb0 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c46ff5640 1 -- 172.21.15.5:0/3658073843 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f1c5c004b20 con 0x55de290cecb0 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.269+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/3658073843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c580a4ea0 con 0x55de290cecb0 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.270+0000 7f1c46ff5640 1 -- 172.21.15.5:0/3658073843 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1c5c032480 con 0x55de290cecb0 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.270+0000 7f1c6c3c2740 10 client.28952 fetch_fsmap learned FSMap version 48 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.270+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/3658073843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55de28e07620 con 0x55de290cecb0 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.270+0000 7f1c46ff5640 1 -- 172.21.15.5:0/3658073843 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c580a4ea0 con 0x55de290cecb0 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.270+0000 7f1c46ff5640 1 -- 172.21.15.5:0/3658073843 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f1c5c00e630 con 0x55de290cecb0 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.270+0000 7f1c6c3c2740 10 client.28952 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:19:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.270+0000 7f1c6c3c2740 10 client.28952 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.270+0000 7f1c6c3c2740 20 client.28952 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.270+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de290dede0 0x55de290e11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.270+0000 7f1c6c3c2740 4 client.28952 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:19:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.270+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/3658073843 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de29035a20 con 0x55de290dede0 2024-09-19T06:19:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.270+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de290dede0 0x55de290e11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.271+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de290dede0 0x55de290e11f0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5862d6c620b1260 server_cookie=ad4f33daa0708b9f in_seq=0 out_seq=0 2024-09-19T06:19:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.271+0000 7f1c46ff5640 10 client.28952 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c46ff5640 1 -- 172.21.15.5:0/3658073843 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de29035a20 con 0x55de290dede0 2024-09-19T06:19:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c46ff5640 10 client.28952 handle_command_reply: tid=0 2024-09-19T06:19:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c6c3c2740 1 client.28952 shutdown 2024-09-19T06:19:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/3658073843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 msgr2=0x55de290cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 0x55de290cc600 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c003e40 tx=0x7f1c5c009ee0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/3658073843 shutdown_connections 2024-09-19T06:19:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de290dede0 0x55de290e11f0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290cecb0 0x55de290cc600 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290c4410 0x55de290cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/3658073843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c3af0 0x55de290cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c46ff5640 0 client.28952 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/3658073843 >> 172.21.15.5:0/3658073843 conn(0x55de290c0060 msgr2=0x55de290bf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/3658073843 shutdown_connections 2024-09-19T06:19:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/3658073843 wait complete. 2024-09-19T06:19:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.272+0000 7f1c6c3c2740 20 client.28952 trim_cache size 0 max 16384 2024-09-19T06:19:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.297+0000 7f1c6c3c2740 1 Processor -- start 2024-09-19T06:19:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.297+0000 7f1c6c3c2740 1 -- start start 2024-09-19T06:19:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.298+0000 7f1c6c3c2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 0x55de290cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.298+0000 7f1c6c3c2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.298+0000 7f1c6c3c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290cecb0 0x55de290e7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.298+0000 7f1c6c3c2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c641afa50 con 0x55de290c4410 2024-09-19T06:19:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.298+0000 7f1c62ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 0x55de290cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.298+0000 7f1c62ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 0x55de290cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49618/0 (socket says 172.21.15.5:49618) 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.298+0000 7f1c62ffd640 1 -- 172.21.15.5:0/1569417791 learned_addr learned my addr 172.21.15.5:0/1569417791 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.298+0000 7f1c627fc640 1 --2- 172.21.15.5:0/1569417791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.298+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/1569417791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c641b1740 con 0x55de290c3af0 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.298+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/1569417791 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c641b01e0 con 0x55de290cecb0 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.298+0000 7f1c637fe640 1 --2- 172.21.15.5:0/1569417791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290cecb0 0x55de290e7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.299+0000 7f1c62ffd640 1 -- 172.21.15.5:0/1569417791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290cecb0 msgr2=0x55de290e7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.299+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/1569417791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290cecb0 0x55de290e7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.299+0000 7f1c62ffd640 1 -- 172.21.15.5:0/1569417791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 msgr2=0x55de290e6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.299+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/1569417791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290e6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.299+0000 7f1c62ffd640 1 -- 172.21.15.5:0/1569417791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de290c11f0 con 0x55de290c3af0 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.299+0000 7f1c627fc640 1 --2- 172.21.15.5:0/1569417791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.299+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/1569417791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 0x55de290cc430 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009900 tx=0x7f1c54004350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bceff0d7e59cf56e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.299+0000 7f1c48ff9640 1 -- 172.21.15.5:0/1569417791 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54035040 con 0x55de290c3af0 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.299+0000 7f1c48ff9640 1 -- 172.21.15.5:0/1569417791 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c540044c0 con 0x55de290c3af0 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.299+0000 7f1c48ff9640 1 -- 172.21.15.5:0/1569417791 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54003d00 con 0x55de290c3af0 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.300+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/1569417791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 msgr2=0x55de290cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.300+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/1569417791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 0x55de290cc430 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009900 tx=0x7f1c54004350 comp rx=0 tx=0).stop 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.301+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/1569417791 shutdown_connections 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.301+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/1569417791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290cecb0 0x55de290e7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.301+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/1569417791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.301+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/1569417791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 0x55de290cc430 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.301+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/1569417791 >> 172.21.15.5:0/1569417791 conn(0x55de290c0060 msgr2=0x55de290cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.301+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/1569417791 shutdown_connections 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.301+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/1569417791 wait complete. 2024-09-19T06:19:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.301+0000 7f1c6c3c2740 1 -- start start 2024-09-19T06:19:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.301+0000 7f1c6c3c2740 1 Processor -- start 2024-09-19T06:19:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c6c3c2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 0x55de290d7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c6c3c2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290d2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c6c3c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290cecb0 0x55de290d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c6c3c2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c641b1070 con 0x55de290c4410 2024-09-19T06:19:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c6c3c2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c641b08e0 con 0x55de290c3af0 2024-09-19T06:19:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c6c3c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c641ada50 con 0x55de290cecb0 2024-09-19T06:19:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c6c3c2740 10 client.0 _ll_register_callbacks cb 0x7f1c5c003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c62ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 0x55de290d7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c62ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 0x55de290d7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49622/0 (socket says 172.21.15.5:49622) 2024-09-19T06:19:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c62ffd640 1 -- 172.21.15.5:0/2139193424 learned_addr learned my addr 172.21.15.5:0/2139193424 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c637fe640 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290cecb0 0x55de290d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c627fc640 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c62ffd640 1 -- 172.21.15.5:0/2139193424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290cecb0 msgr2=0x55de290d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290cecb0 0x55de290d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c62ffd640 1 -- 172.21.15.5:0/2139193424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 msgr2=0x55de290d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290d2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.302+0000 7f1c62ffd640 1 -- 172.21.15.5:0/2139193424 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de290c6940 con 0x55de290c3af0 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.303+0000 7f1c62ffd640 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 0x55de290d7470 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009930 tx=0x7f1c54005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d161ebb1a20ef51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.303+0000 7f1c46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.303+0000 7f1c46ff5640 1 -- 172.21.15.5:0/2139193424 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54056020 con 0x55de290c3af0 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.303+0000 7f1c46ff5640 1 -- 172.21.15.5:0/2139193424 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f1c54005450 con 0x55de290c3af0 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.303+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/2139193424 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55de2902fd70 con 0x55de290c3af0 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.303+0000 7f1c46ff5640 1 -- 172.21.15.5:0/2139193424 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1c54061050 con 0x55de290c3af0 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.303+0000 7f1c6c3c2740 10 client.28958 fetch_fsmap learned FSMap version 48 2024-09-19T06:19:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.303+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/2139193424 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55de29152180 con 0x55de290c3af0 2024-09-19T06:19:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.304+0000 7f1c46ff5640 1 -- 172.21.15.5:0/2139193424 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55de2902fd70 con 0x55de290c3af0 2024-09-19T06:19:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.304+0000 7f1c46ff5640 1 -- 172.21.15.5:0/2139193424 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f1c54004900 con 0x55de290c3af0 2024-09-19T06:19:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.304+0000 7f1c6c3c2740 10 client.28958 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:19:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.304+0000 7f1c6c3c2740 10 client.28958 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.304+0000 7f1c6c3c2740 20 client.28958 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.304+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de29152990 0x55de29154db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.304+0000 7f1c6c3c2740 4 client.28958 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:19:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.304+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/2139193424 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de28fac900 con 0x55de29152990 2024-09-19T06:19:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.304+0000 7f1c627fc640 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de29152990 0x55de29154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.305+0000 7f1c627fc640 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de29152990 0x55de29154db0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231cad3552ecd7a2 server_cookie=43848d4858a10e8 in_seq=0 out_seq=0 2024-09-19T06:19:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.305+0000 7f1c46ff5640 10 client.28958 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.369+0000 7f1c46ff5640 1 -- 172.21.15.5:0/2139193424 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de28fac900 con 0x55de29152990 2024-09-19T06:19:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.369+0000 7f1c46ff5640 10 client.28958 handle_command_reply: tid=0 2024-09-19T06:19:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.369+0000 7f1c6c3c2740 1 client.28958 shutdown 2024-09-19T06:19:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.370+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/2139193424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 msgr2=0x55de290d7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.370+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 0x55de290d7470 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009930 tx=0x7f1c54005e50 comp rx=0 tx=0).stop 2024-09-19T06:19:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.370+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/2139193424 shutdown_connections 2024-09-19T06:19:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.370+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55de29152990 0x55de29154db0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.370+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55de290cecb0 0x55de290d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.370+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de290c4410 0x55de290d2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.370+0000 7f1c6c3c2740 1 --2- 172.21.15.5:0/2139193424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55de290c3af0 0x55de290d7470 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.370+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/2139193424 >> 172.21.15.5:0/2139193424 conn(0x55de290c0060 msgr2=0x55de290be9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.370+0000 7f1c46ff5640 0 client.28958 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.370+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/2139193424 shutdown_connections 2024-09-19T06:19:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.370+0000 7f1c6c3c2740 1 -- 172.21.15.5:0/2139193424 wait complete. 2024-09-19T06:19:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.370+0000 7f1c6c3c2740 20 client.28958 trim_cache size 0 max 16384 2024-09-19T06:19:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:19:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:19:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:19:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:19:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.371+0000 7f1c69627640 1 -- 172.21.15.5:0/3102121008 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1c4c0a3430 msgr2=0x7f1c4c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.371+0000 7f1c69627640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1c4c0a3430 0x7f1c4c0a58f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009fd0 tx=0x7f1c5c011440 comp rx=0 tx=0).stop 2024-09-19T06:19:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.371+0000 7f1c69627640 1 -- 172.21.15.5:0/3102121008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c64101340 msgr2=0x7f1c6419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.371+0000 7f1c69627640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c64101340 0x7f1c6419ac10 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800e9f0 tx=0x7f1c5800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.372+0000 7f1c69627640 1 -- 172.21.15.5:0/3102121008 shutdown_connections 2024-09-19T06:19:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.372+0000 7f1c69627640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1c4c0a3430 0x7f1c4c0a58f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.372+0000 7f1c69627640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c641069b0 0x7f1c6419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.372+0000 7f1c69627640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c64101340 0x7f1c6419ac10 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.372+0000 7f1c69627640 1 --2- 172.21.15.5:0/3102121008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c641009a0 0x7f1c6419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.372+0000 7f1c69627640 1 -- 172.21.15.5:0/3102121008 >> 172.21.15.5:0/3102121008 conn(0x7f1c640779a0 msgr2=0x7f1c641027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:20.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.372+0000 7f1c69627640 1 -- 172.21.15.5:0/3102121008 shutdown_connections 2024-09-19T06:19:20.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:20.372+0000 7f1c69627640 1 -- 172.21.15.5:0/3102121008 wait complete. 2024-09-19T06:19:20.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 27676 8.98 MB/sec execute 44 sec latency 68.729 ms 2024-09-19T06:19:20.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.696+0000 7fe15f0e3640 1 -- 172.21.15.5:0/2558021036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe158101dd0 msgr2=0x7fe1581021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.696+0000 7fe15f0e3640 1 --2- 172.21.15.5:0/2558021036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe158101dd0 0x7fe1581021b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe154009900 tx=0x7fe15402a410 comp rx=0 tx=0).stop 2024-09-19T06:19:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.696+0000 7fe15f0e3640 1 -- 172.21.15.5:0/2558021036 shutdown_connections 2024-09-19T06:19:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.696+0000 7fe15f0e3640 1 --2- 172.21.15.5:0/2558021036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe158106f00 0x7fe15810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.696+0000 7fe15f0e3640 1 --2- 172.21.15.5:0/2558021036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe158102780 0x7fe1581067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.696+0000 7fe15f0e3640 1 --2- 172.21.15.5:0/2558021036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe158101dd0 0x7fe1581021b0 secure :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe154009900 tx=0x7fe15402a410 comp rx=0 tx=0).stop 2024-09-19T06:19:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.696+0000 7fe15f0e3640 1 -- 172.21.15.5:0/2558021036 >> 172.21.15.5:0/2558021036 conn(0x7fe1580fd870 msgr2=0x7fe1580ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.697+0000 7fe15f0e3640 1 -- 172.21.15.5:0/2558021036 shutdown_connections 2024-09-19T06:19:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.697+0000 7fe15f0e3640 1 -- 172.21.15.5:0/2558021036 wait complete. 2024-09-19T06:19:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe15f0e3640 1 Processor -- start 2024-09-19T06:19:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe15f0e3640 1 -- start start 2024-09-19T06:19:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe15f0e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe158102780 0x7fe158198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe15f0e3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe158106f00 0x7fe158198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe15f0e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15819ce20 0x7fe15819d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe15f0e3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe158111e10 con 0x7fe15819ce20 2024-09-19T06:19:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe15f0e3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe158111c90 con 0x7fe158106f00 2024-09-19T06:19:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe15f0e3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe158111f90 con 0x7fe158102780 2024-09-19T06:19:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe14ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe158106f00 0x7fe158198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe15d680640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15819ce20 0x7fe15819d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe15d680640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15819ce20 0x7fe15819d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38056/0 (socket says 172.21.15.5:38056) 2024-09-19T06:19:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe15d680640 1 -- 172.21.15.5:0/1239617864 learned_addr learned my addr 172.21.15.5:0/1239617864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.698+0000 7fe15ce7f640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe158102780 0x7fe158198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.699+0000 7fe15d680640 1 -- 172.21.15.5:0/1239617864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe158102780 msgr2=0x7fe158198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.699+0000 7fe15d680640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe158102780 0x7fe158198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.699+0000 7fe15d680640 1 -- 172.21.15.5:0/1239617864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe158106f00 msgr2=0x7fe158198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.699+0000 7fe15d680640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe158106f00 0x7fe158198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.699+0000 7fe15d680640 1 -- 172.21.15.5:0/1239617864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15819d8e0 con 0x7fe15819ce20 2024-09-19T06:19:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.699+0000 7fe14ffff640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe158106f00 0x7fe158198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.699+0000 7fe15d680640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15819ce20 0x7fe15819d2d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe15000eb10 tx=0x7fe15000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.699+0000 7fe14dffb640 1 -- 172.21.15.5:0/1239617864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150004b20 con 0x7fe15819ce20 2024-09-19T06:19:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.700+0000 7fe14dffb640 1 -- 172.21.15.5:0/1239617864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe150004cc0 con 0x7fe15819ce20 2024-09-19T06:19:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.700+0000 7fe15f0e3640 1 -- 172.21.15.5:0/1239617864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15819dbd0 con 0x7fe15819ce20 2024-09-19T06:19:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.700+0000 7fe14dffb640 1 -- 172.21.15.5:0/1239617864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150010690 con 0x7fe15819ce20 2024-09-19T06:19:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.700+0000 7fe15f0e3640 1 -- 172.21.15.5:0/1239617864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1581a53b0 con 0x7fe15819ce20 2024-09-19T06:19:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.703+0000 7fe1614bc740 1 Processor -- start 2024-09-19T06:19:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.703+0000 7fe14dffb640 1 -- 172.21.15.5:0/1239617864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe150005020 con 0x7fe15819ce20 2024-09-19T06:19:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.703+0000 7fe1614bc740 1 -- start start 2024-09-19T06:19:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe1614bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51db40 0x55cd1f51c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe1614bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51c5d0 0x55cd1f51ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe1614bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f52b780 0x55cd1f52db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe1614bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cd1f4822f0 con 0x55cd1f51db40 2024-09-19T06:19:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe1614bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cd1f429cf0 con 0x55cd1f51c5d0 2024-09-19T06:19:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe1614bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cd1f47e2b0 con 0x55cd1f52b780 2024-09-19T06:19:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe14ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51db40 0x55cd1f51c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe14ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51db40 0x55cd1f51c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38066/0 (socket says 172.21.15.5:38066) 2024-09-19T06:19:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe14ffff640 1 -- 172.21.15.5:0/755378440 learned_addr learned my addr 172.21.15.5:0/755378440 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe15ce7f640 1 --2- 172.21.15.5:0/755378440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51c5d0 0x55cd1f51ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe14ffff640 1 --2- 172.21.15.5:0/755378440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f52b780 0x55cd1f52db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe14ffff640 1 -- 172.21.15.5:0/755378440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f52b780 msgr2=0x55cd1f52db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe14ffff640 1 --2- 172.21.15.5:0/755378440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f52b780 0x55cd1f52db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe14ffff640 1 -- 172.21.15.5:0/755378440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51c5d0 msgr2=0x55cd1f51ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe14ffff640 1 --2- 172.21.15.5:0/755378440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51c5d0 0x55cd1f51ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe14ffff640 1 -- 172.21.15.5:0/755378440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1f40e3a0 con 0x55cd1f51db40 2024-09-19T06:19:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.704+0000 7fe14ffff640 1 --2- 172.21.15.5:0/755378440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51db40 0x55cd1f51c090 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe148013bf0 tx=0x7fe148009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4e267aca4be892 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.705+0000 7fe14dffb640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe1400a3500 0x7fe1400a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.705+0000 7fe13effd640 1 -- 172.21.15.5:0/755378440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148013170 con 0x55cd1f51db40 2024-09-19T06:19:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.705+0000 7fe13effd640 1 -- 172.21.15.5:0/755378440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe148013ce0 con 0x55cd1f51db40 2024-09-19T06:19:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.705+0000 7fe13effd640 1 -- 172.21.15.5:0/755378440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148004a00 con 0x55cd1f51db40 2024-09-19T06:19:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.705+0000 7fe14dffb640 1 -- 172.21.15.5:0/1239617864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fe1500d0180 con 0x7fe15819ce20 2024-09-19T06:19:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.705+0000 7fe15ce7f640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe1400a3500 0x7fe1400a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.706+0000 7fe1614bc740 1 -- 172.21.15.5:0/755378440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51db40 msgr2=0x55cd1f51c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.706+0000 7fe1614bc740 1 --2- 172.21.15.5:0/755378440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51db40 0x55cd1f51c090 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe148013bf0 tx=0x7fe148009b50 comp rx=0 tx=0).stop 2024-09-19T06:19:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.706+0000 7fe15ce7f640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe1400a3500 0x7fe1400a59c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe154000c00 tx=0x7fe154005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.706+0000 7fe1614bc740 1 -- 172.21.15.5:0/755378440 shutdown_connections 2024-09-19T06:19:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.706+0000 7fe1614bc740 1 --2- 172.21.15.5:0/755378440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f52b780 0x55cd1f52db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.706+0000 7fe1614bc740 1 --2- 172.21.15.5:0/755378440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51c5d0 0x55cd1f51ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.706+0000 7fe1614bc740 1 --2- 172.21.15.5:0/755378440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51db40 0x55cd1f51c090 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.706+0000 7fe1614bc740 1 -- 172.21.15.5:0/755378440 >> 172.21.15.5:0/755378440 conn(0x55cd1f4e5310 msgr2=0x55cd1f518c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.706+0000 7fe1614bc740 1 -- 172.21.15.5:0/755378440 shutdown_connections 2024-09-19T06:19:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.706+0000 7fe1614bc740 1 -- 172.21.15.5:0/755378440 wait complete. 2024-09-19T06:19:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.707+0000 7fe1614bc740 1 -- start start 2024-09-19T06:19:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.707+0000 7fe1614bc740 1 Processor -- start 2024-09-19T06:19:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.707+0000 7fe1614bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51c5d0 0x55cd1f52b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.707+0000 7fe1614bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51db40 0x55cd1f528410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.707+0000 7fe1614bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f52b780 0x55cd1f5289e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.707+0000 7fe1614bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cd1f52e420 con 0x55cd1f51c5d0 2024-09-19T06:19:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.707+0000 7fe1614bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cd1f43d060 con 0x55cd1f52b780 2024-09-19T06:19:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.707+0000 7fe1614bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cd1f52e5a0 con 0x55cd1f51db40 2024-09-19T06:19:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.707+0000 7fe15ce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51c5d0 0x55cd1f52b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe15ce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51c5d0 0x55cd1f52b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38074/0 (socket says 172.21.15.5:38074) 2024-09-19T06:19:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe1614bc740 10 client.0 _ll_register_callbacks cb 0x55cd1f1908c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe15ce7f640 1 -- 172.21.15.5:0/1530108713 learned_addr learned my addr 172.21.15.5:0/1530108713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe14ffff640 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51db40 0x55cd1f528410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe15d680640 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f52b780 0x55cd1f5289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe15d680640 1 -- 172.21.15.5:0/1530108713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51db40 msgr2=0x55cd1f528410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe15d680640 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51db40 0x55cd1f528410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe15d680640 1 -- 172.21.15.5:0/1530108713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51c5d0 msgr2=0x55cd1f52b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe15d680640 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51c5d0 0x55cd1f52b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe15d680640 1 -- 172.21.15.5:0/1530108713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1f1f33c0 con 0x55cd1f52b780 2024-09-19T06:19:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe14ffff640 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51db40 0x55cd1f528410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe15d680640 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f52b780 0x55cd1f5289e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe1500cd850 tx=0x7fe15009bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eddf2ea5f778ec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe137fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:19:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe137fff640 1 -- 172.21.15.5:0/1530108713 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1500b4070 con 0x55cd1f52b780 2024-09-19T06:19:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe1614bc740 1 -- 172.21.15.5:0/1530108713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe148013ce0 con 0x55cd1f52b780 2024-09-19T06:19:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe137fff640 1 -- 172.21.15.5:0/1530108713 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fe1500a4bc0 con 0x55cd1f52b780 2024-09-19T06:19:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.708+0000 7fe137fff640 1 -- 172.21.15.5:0/1530108713 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe15009c3e0 con 0x55cd1f52b780 2024-09-19T06:19:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.709+0000 7fe1614bc740 10 client.28970 fetch_fsmap learned FSMap version 48 2024-09-19T06:19:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.709+0000 7fe1614bc740 1 -- 172.21.15.5:0/1530108713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55cd1f539ef0 con 0x55cd1f52b780 2024-09-19T06:19:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.709+0000 7fe137fff640 1 -- 172.21.15.5:0/1530108713 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe148013ce0 con 0x55cd1f52b780 2024-09-19T06:19:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.709+0000 7fe137fff640 1 -- 172.21.15.5:0/1530108713 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7fe1500053a0 con 0x55cd1f52b780 2024-09-19T06:19:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.709+0000 7fe1614bc740 10 client.28970 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:19:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.709+0000 7fe1614bc740 10 client.28970 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:19:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.709+0000 7fe1614bc740 20 client.28970 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.709+0000 7fe1614bc740 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd1f53b7d0 0x55cd1f53dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.709+0000 7fe1614bc740 4 client.28970 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:19:20.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.709+0000 7fe1614bc740 1 -- 172.21.15.5:0/1530108713 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd1f48f860 con 0x55cd1f53b7d0 2024-09-19T06:19:20.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.709+0000 7fe15ce7f640 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd1f53b7d0 0x55cd1f53dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.710+0000 7fe15ce7f640 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd1f53b7d0 0x55cd1f53dc00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd06737d10f9bbed server_cookie=239552bfb2a324 in_seq=0 out_seq=0 2024-09-19T06:19:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.710+0000 7fe137fff640 10 client.28970 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:19:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.710+0000 7fe137fff640 1 -- 172.21.15.5:0/1530108713 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd1f48f860 con 0x55cd1f53b7d0 2024-09-19T06:19:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.710+0000 7fe137fff640 10 client.28970 handle_command_reply: tid=0 2024-09-19T06:19:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.710+0000 7fe1614bc740 1 client.28970 shutdown 2024-09-19T06:19:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.711+0000 7fe1614bc740 1 -- 172.21.15.5:0/1530108713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f52b780 msgr2=0x55cd1f5289e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.711+0000 7fe1614bc740 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f52b780 0x55cd1f5289e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe1500cd850 tx=0x7fe15009bf20 comp rx=0 tx=0).stop 2024-09-19T06:19:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.711+0000 7fe1614bc740 1 -- 172.21.15.5:0/1530108713 shutdown_connections 2024-09-19T06:19:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.711+0000 7fe1614bc740 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd1f53b7d0 0x55cd1f53dc00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.711+0000 7fe1614bc740 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f52b780 0x55cd1f5289e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.711+0000 7fe1614bc740 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51db40 0x55cd1f528410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.711+0000 7fe1614bc740 1 --2- 172.21.15.5:0/1530108713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f51c5d0 0x55cd1f52b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.711+0000 7fe1614bc740 1 -- 172.21.15.5:0/1530108713 >> 172.21.15.5:0/1530108713 conn(0x55cd1f4e5310 msgr2=0x55cd1f51e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.711+0000 7fe137fff640 0 client.28970 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:19:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.711+0000 7fe1614bc740 1 -- 172.21.15.5:0/1530108713 shutdown_connections 2024-09-19T06:19:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.711+0000 7fe1614bc740 1 -- 172.21.15.5:0/1530108713 wait complete. 2024-09-19T06:19:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.711+0000 7fe1614bc740 20 client.28970 trim_cache size 0 max 16384 2024-09-19T06:19:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.736+0000 7fe1614bc740 1 Processor -- start 2024-09-19T06:19:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.736+0000 7fe1614bc740 1 -- start start 2024-09-19T06:19:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.736+0000 7fe1614bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 0x55cd1f5376c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.736+0000 7fe1614bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51c5d0 0x55cd1f518700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.736+0000 7fe1614bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51db40 0x55cd1f543510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.736+0000 7fe1614bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581abb70 con 0x55cd1f4e5310 2024-09-19T06:19:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.736+0000 7fe1614bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581ad860 con 0x55cd1f51c5d0 2024-09-19T06:19:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.736+0000 7fe1614bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581ac300 con 0x55cd1f51db40 2024-09-19T06:19:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.737+0000 7fe15ce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 0x55cd1f5376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.737+0000 7fe15ce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 0x55cd1f5376c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38084/0 (socket says 172.21.15.5:38084) 2024-09-19T06:19:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.737+0000 7fe15ce7f640 1 -- 172.21.15.5:0/4238684194 learned_addr learned my addr 172.21.15.5:0/4238684194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.737+0000 7fe14ffff640 1 --2- 172.21.15.5:0/4238684194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51c5d0 0x55cd1f518700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.737+0000 7fe14ffff640 1 -- 172.21.15.5:0/4238684194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51db40 msgr2=0x55cd1f543510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.737+0000 7fe15d680640 1 --2- 172.21.15.5:0/4238684194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51db40 0x55cd1f543510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.737+0000 7fe14ffff640 1 --2- 172.21.15.5:0/4238684194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51db40 0x55cd1f543510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.737+0000 7fe14ffff640 1 -- 172.21.15.5:0/4238684194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 msgr2=0x55cd1f5376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.737+0000 7fe14ffff640 1 --2- 172.21.15.5:0/4238684194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 0x55cd1f5376c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.737+0000 7fe14ffff640 1 -- 172.21.15.5:0/4238684194 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1f518ed0 con 0x55cd1f51c5d0 2024-09-19T06:19:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.737+0000 7fe15ce7f640 1 --2- 172.21.15.5:0/4238684194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 0x55cd1f5376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.737+0000 7fe14ffff640 1 --2- 172.21.15.5:0/4238684194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51c5d0 0x55cd1f518700 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe14800fd30 tx=0x7fe148005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eecb99ff36bdfce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.738+0000 7fe13e7fc640 1 -- 172.21.15.5:0/4238684194 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148024070 con 0x55cd1f51c5d0 2024-09-19T06:19:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.738+0000 7fe13e7fc640 1 -- 172.21.15.5:0/4238684194 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe14800fd60 con 0x55cd1f51c5d0 2024-09-19T06:19:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.738+0000 7fe13e7fc640 1 -- 172.21.15.5:0/4238684194 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148003d00 con 0x55cd1f51c5d0 2024-09-19T06:19:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.739+0000 7fe1614bc740 1 -- 172.21.15.5:0/4238684194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51c5d0 msgr2=0x55cd1f518700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.739+0000 7fe1614bc740 1 --2- 172.21.15.5:0/4238684194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51c5d0 0x55cd1f518700 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe14800fd30 tx=0x7fe148005fd0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.739+0000 7fe1614bc740 1 -- 172.21.15.5:0/4238684194 shutdown_connections 2024-09-19T06:19:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.739+0000 7fe1614bc740 1 --2- 172.21.15.5:0/4238684194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51db40 0x55cd1f543510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.739+0000 7fe1614bc740 1 --2- 172.21.15.5:0/4238684194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51c5d0 0x55cd1f518700 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.739+0000 7fe1614bc740 1 --2- 172.21.15.5:0/4238684194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 0x55cd1f5376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.739+0000 7fe1614bc740 1 -- 172.21.15.5:0/4238684194 >> 172.21.15.5:0/4238684194 conn(0x55cd1f523e00 msgr2=0x55cd1f526ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.739+0000 7fe1614bc740 1 -- 172.21.15.5:0/4238684194 shutdown_connections 2024-09-19T06:19:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.739+0000 7fe1614bc740 1 -- 172.21.15.5:0/4238684194 wait complete. 2024-09-19T06:19:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.740+0000 7fe1614bc740 1 -- start start 2024-09-19T06:19:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.740+0000 7fe1614bc740 1 Processor -- start 2024-09-19T06:19:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.741+0000 7fe1614bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 0x55cd1f533c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.741+0000 7fe1614bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51c5d0 0x55cd1f52efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.741+0000 7fe1614bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51db40 0x55cd1f52f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.741+0000 7fe1614bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581ad190 con 0x55cd1f4e5310 2024-09-19T06:19:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.741+0000 7fe15ce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 0x55cd1f533c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.741+0000 7fe15ce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 0x55cd1f533c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38096/0 (socket says 172.21.15.5:38096) 2024-09-19T06:19:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.741+0000 7fe15ce7f640 1 -- 172.21.15.5:0/97758070 learned_addr learned my addr 172.21.15.5:0/97758070 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.741+0000 7fe15d680640 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51db40 0x55cd1f52f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.741+0000 7fe14ffff640 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51c5d0 0x55cd1f52efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.741+0000 7fe1614bc740 1 -- 172.21.15.5:0/97758070 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581aca00 con 0x55cd1f51db40 2024-09-19T06:19:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.741+0000 7fe1614bc740 1 -- 172.21.15.5:0/97758070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581a9b50 con 0x55cd1f51c5d0 2024-09-19T06:19:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.742+0000 7fe1614bc740 10 client.0 _ll_register_callbacks cb 0x7fe150096f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.742+0000 7fe15ce7f640 1 -- 172.21.15.5:0/97758070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51c5d0 msgr2=0x55cd1f52efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.742+0000 7fe15ce7f640 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51c5d0 0x55cd1f52efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.742+0000 7fe15ce7f640 1 -- 172.21.15.5:0/97758070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51db40 msgr2=0x55cd1f52f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.742+0000 7fe15ce7f640 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51db40 0x55cd1f52f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.742+0000 7fe15ce7f640 1 -- 172.21.15.5:0/97758070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1f52b4d0 con 0x55cd1f4e5310 2024-09-19T06:19:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.742+0000 7fe15ce7f640 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 0x55cd1f533c90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe154009900 tx=0x7fe15403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f390f48e150f2705 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.742+0000 7fe137fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:19:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.743+0000 7fe137fff640 1 -- 172.21.15.5:0/97758070 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154057060 con 0x55cd1f4e5310 2024-09-19T06:19:20.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.743+0000 7fe137fff640 1 -- 172.21.15.5:0/97758070 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fe1540055d0 con 0x55cd1f4e5310 2024-09-19T06:19:20.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.743+0000 7fe1614bc740 1 -- 172.21.15.5:0/97758070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe14800fd60 con 0x55cd1f4e5310 2024-09-19T06:19:20.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.743+0000 7fe137fff640 1 -- 172.21.15.5:0/97758070 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe154035040 con 0x55cd1f4e5310 2024-09-19T06:19:20.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.743+0000 7fe1614bc740 10 client.19599 fetch_fsmap learned FSMap version 48 2024-09-19T06:19:20.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.743+0000 7fe1614bc740 1 -- 172.21.15.5:0/97758070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55cd1f5a6080 con 0x55cd1f4e5310 2024-09-19T06:19:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.744+0000 7fe137fff640 1 -- 172.21.15.5:0/97758070 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe14800fd60 con 0x55cd1f4e5310 2024-09-19T06:19:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.744+0000 7fe137fff640 1 -- 172.21.15.5:0/97758070 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7fe154044040 con 0x55cd1f4e5310 2024-09-19T06:19:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.744+0000 7fe1614bc740 10 client.19599 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T06:19:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.744+0000 7fe1614bc740 10 client.19599 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:19:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.744+0000 7fe1614bc740 20 client.19599 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.744+0000 7fe1614bc740 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd1f52b780 0x55cd1f5a8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.744+0000 7fe1614bc740 4 client.19599 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:19:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.744+0000 7fe1614bc740 1 -- 172.21.15.5:0/97758070 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd1f489d70 con 0x55cd1f52b780 2024-09-19T06:19:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.744+0000 7fe14ffff640 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd1f52b780 0x55cd1f5a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.745+0000 7fe137fff640 10 client.19599 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:19:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.745+0000 7fe14ffff640 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd1f52b780 0x55cd1f5a8840 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2218fd29a4c610c5 server_cookie=62c61e496ec5c9a9 in_seq=0 out_seq=0 2024-09-19T06:19:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.745+0000 7fe137fff640 1 -- 172.21.15.5:0/97758070 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd1f489d70 con 0x55cd1f52b780 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.745+0000 7fe137fff640 10 client.19599 handle_command_reply: tid=0 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.745+0000 7fe1614bc740 1 client.19599 shutdown 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.745+0000 7fe1614bc740 1 -- 172.21.15.5:0/97758070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 msgr2=0x55cd1f533c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.745+0000 7fe1614bc740 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 0x55cd1f533c90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe154009900 tx=0x7fe15403f040 comp rx=0 tx=0).stop 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.745+0000 7fe1614bc740 1 -- 172.21.15.5:0/97758070 shutdown_connections 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.745+0000 7fe1614bc740 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd1f52b780 0x55cd1f5a8840 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.745+0000 7fe1614bc740 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd1f51db40 0x55cd1f52f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.745+0000 7fe1614bc740 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1f51c5d0 0x55cd1f52efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.745+0000 7fe1614bc740 1 --2- 172.21.15.5:0/97758070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd1f4e5310 0x55cd1f533c90 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.746+0000 7fe1614bc740 1 -- 172.21.15.5:0/97758070 >> 172.21.15.5:0/97758070 conn(0x55cd1f523e00 msgr2=0x55cd1f529c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.746+0000 7fe137fff640 0 client.19599 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.746+0000 7fe1614bc740 1 -- 172.21.15.5:0/97758070 shutdown_connections 2024-09-19T06:19:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.746+0000 7fe1614bc740 1 -- 172.21.15.5:0/97758070 wait complete. 2024-09-19T06:19:20.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.746+0000 7fe1614bc740 20 client.19599 trim_cache size 0 max 16384 2024-09-19T06:19:20.752 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:19:20.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.747+0000 7fe15f0e3640 1 -- 172.21.15.5:0/1239617864 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe1400a3500 msgr2=0x7fe1400a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.747+0000 7fe15f0e3640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe1400a3500 0x7fe1400a59c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe154000c00 tx=0x7fe154005a20 comp rx=0 tx=0).stop 2024-09-19T06:19:20.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.747+0000 7fe15f0e3640 1 -- 172.21.15.5:0/1239617864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15819ce20 msgr2=0x7fe15819d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:20.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.747+0000 7fe15f0e3640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15819ce20 0x7fe15819d2d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe15000eb10 tx=0x7fe15000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.747+0000 7fe15f0e3640 1 -- 172.21.15.5:0/1239617864 shutdown_connections 2024-09-19T06:19:20.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.747+0000 7fe15f0e3640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15819ce20 0x7fe15819d2d0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.747+0000 7fe15f0e3640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe158106f00 0x7fe158198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.747+0000 7fe15f0e3640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe1400a3500 0x7fe1400a59c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.747+0000 7fe15f0e3640 1 --2- 172.21.15.5:0/1239617864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe158102780 0x7fe158198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:20.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.747+0000 7fe15f0e3640 1 -- 172.21.15.5:0/1239617864 >> 172.21.15.5:0/1239617864 conn(0x7fe1580fd870 msgr2=0x7fe1580fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:20.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.747+0000 7fe15f0e3640 1 -- 172.21.15.5:0/1239617864 shutdown_connections 2024-09-19T06:19:20.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:20.748+0000 7fe15f0e3640 1 -- 172.21.15.5:0/1239617864 wait complete. 2024-09-19T06:19:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:20 smithi157 ceph-mon[35346]: pgmap v503: 97 pgs: 97 active+clean; 610 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 278 op/s 2024-09-19T06:19:20.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:20 smithi005 ceph-mon[30955]: pgmap v503: 97 pgs: 97 active+clean; 610 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 278 op/s 2024-09-19T06:19:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:20 smithi049 ceph-mon[34640]: pgmap v503: 97 pgs: 97 active+clean; 610 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 278 op/s 2024-09-19T06:19:21.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 27831 8.88 MB/sec execute 45 sec latency 185.809 ms 2024-09-19T06:19:21.909 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:19:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:19:22.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 27981 8.79 MB/sec execute 46 sec latency 114.910 ms 2024-09-19T06:19:22.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:22 smithi005 ceph-mon[30955]: pgmap v504: 97 pgs: 97 active+clean; 610 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 242 op/s 2024-09-19T06:19:22.725 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:22 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.851+0000 7fd66600a640 1 -- 172.21.15.5:0/3944536773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd660102d50 msgr2=0x7fd660103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.851+0000 7fd66600a640 1 --2- 172.21.15.5:0/3944536773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd660102d50 0x7fd660103130 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fd650009900 tx=0x7fd65002a480 comp rx=0 tx=0).stop 2024-09-19T06:19:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.851+0000 7fd66600a640 1 -- 172.21.15.5:0/3944536773 shutdown_connections 2024-09-19T06:19:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.851+0000 7fd66600a640 1 --2- 172.21.15.5:0/3944536773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd66010c180 0x7fd66010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.851+0000 7fd66600a640 1 --2- 172.21.15.5:0/3944536773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd660103670 0x7fd66010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.851+0000 7fd66600a640 1 --2- 172.21.15.5:0/3944536773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd660102d50 0x7fd660103130 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.851+0000 7fd66600a640 1 -- 172.21.15.5:0/3944536773 >> 172.21.15.5:0/3944536773 conn(0x7fd6600fc7d0 msgr2=0x7fd6600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.851+0000 7fd66600a640 1 -- 172.21.15.5:0/3944536773 shutdown_connections 2024-09-19T06:19:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.851+0000 7fd66600a640 1 -- 172.21.15.5:0/3944536773 wait complete. 2024-09-19T06:19:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.852+0000 7fd66600a640 1 Processor -- start 2024-09-19T06:19:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.852+0000 7fd66600a640 1 -- start start 2024-09-19T06:19:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.852+0000 7fd66600a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd660102d50 0x7fd6601009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.852+0000 7fd66600a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd660103670 0x7fd660100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd66600a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66010c180 0x7fd660101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd66600a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd660111e30 con 0x7fd66010c180 2024-09-19T06:19:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd66600a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd660111cb0 con 0x7fd660102d50 2024-09-19T06:19:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd66600a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd660111fb0 con 0x7fd660103670 2024-09-19T06:19:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd65f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd660102d50 0x7fd6601009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd65ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66010c180 0x7fd660101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd65effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd660103670 0x7fd660100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd65f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd660102d50 0x7fd6601009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49708/0 (socket says 172.21.15.5:49708) 2024-09-19T06:19:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd65ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66010c180 0x7fd660101430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38126/0 (socket says 172.21.15.5:38126) 2024-09-19T06:19:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd65f7fe640 1 -- 172.21.15.5:0/441431976 learned_addr learned my addr 172.21.15.5:0/441431976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd65f7fe640 1 -- 172.21.15.5:0/441431976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd660103670 msgr2=0x7fd660100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd65f7fe640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd660103670 0x7fd660100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd65f7fe640 1 -- 172.21.15.5:0/441431976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66010c180 msgr2=0x7fd660101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd65f7fe640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66010c180 0x7fd660101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd65f7fe640 1 -- 172.21.15.5:0/441431976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6601a6860 con 0x7fd660102d50 2024-09-19T06:19:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.853+0000 7fd65effd640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd660103670 0x7fd660100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.854+0000 7fd65f7fe640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd660102d50 0x7fd6601009b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd650005940 tx=0x7fd6500040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.854+0000 7fd65cff9640 1 -- 172.21.15.5:0/441431976 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65003f030 con 0x7fd660102d50 2024-09-19T06:19:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.854+0000 7fd65cff9640 1 -- 172.21.15.5:0/441431976 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6500044c0 con 0x7fd660102d50 2024-09-19T06:19:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.854+0000 7fd65cff9640 1 -- 172.21.15.5:0/441431976 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650035070 con 0x7fd660102d50 2024-09-19T06:19:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.854+0000 7fd66600a640 1 -- 172.21.15.5:0/441431976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6601a6af0 con 0x7fd660102d50 2024-09-19T06:19:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.854+0000 7fd66600a640 1 -- 172.21.15.5:0/441431976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6601a6f80 con 0x7fd660102d50 2024-09-19T06:19:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.857+0000 7fd65cff9640 1 -- 172.21.15.5:0/441431976 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd650042030 con 0x7fd660102d50 2024-09-19T06:19:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd6683e3740 1 Processor -- start 2024-09-19T06:19:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd6683e3740 1 -- start start 2024-09-19T06:19:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd65cff9640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd6480a33b0 0x7fd6480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd65cff9640 1 -- 172.21.15.5:0/441431976 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fd6500f0080 con 0x7fd660102d50 2024-09-19T06:19:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd6683e3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x563507395090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd6683e3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073955d0 0x563507395a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd6683e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 0x5635073aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd6683e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5635072fb2f0 con 0x5635073a8b80 2024-09-19T06:19:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd6683e3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5635072a2cf0 con 0x563507396b40 2024-09-19T06:19:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd6683e3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5635072f72b0 con 0x5635073955d0 2024-09-19T06:19:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd65ffff640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd6480a33b0 0x7fd6480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd65effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073955d0 0x563507395a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd65f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x563507395090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd65effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073955d0 0x563507395a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34502/0 (socket says 172.21.15.5:34502) 2024-09-19T06:19:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.858+0000 7fd65effd640 1 -- 172.21.15.5:0/1323506143 learned_addr learned my addr 172.21.15.5:0/1323506143 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.859+0000 7fd65effd640 1 -- 172.21.15.5:0/1323506143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 msgr2=0x563507395090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.859+0000 7fd65effd640 1 --2- 172.21.15.5:0/1323506143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x563507395090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.859+0000 7fd65effd640 1 -- 172.21.15.5:0/1323506143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 msgr2=0x5635073aaf70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:19:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.859+0000 7fd65ffff640 1 --2- 172.21.15.5:0/1323506143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 0x5635073aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.859+0000 7fd65effd640 1 --2- 172.21.15.5:0/1323506143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 0x5635073aaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.859+0000 7fd65effd640 1 -- 172.21.15.5:0/1323506143 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635072873a0 con 0x5635073955d0 2024-09-19T06:19:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.859+0000 7fd65ffff640 1 --2- 172.21.15.5:0/1323506143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 0x5635073aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:19:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.859+0000 7fd65effd640 1 --2- 172.21.15.5:0/1323506143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073955d0 0x563507395a10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd6540114c0 tx=0x7fd654011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f69c00109d24455d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.860+0000 7fd65ffff640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd6480a33b0 0x7fd6480a5870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd658005fd0 tx=0x7fd658009c20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.860+0000 7fd645ffb640 1 -- 172.21.15.5:0/1323506143 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654004030 con 0x5635073955d0 2024-09-19T06:19:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.860+0000 7fd645ffb640 1 -- 172.21.15.5:0/1323506143 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6540041d0 con 0x5635073955d0 2024-09-19T06:19:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.860+0000 7fd645ffb640 1 -- 172.21.15.5:0/1323506143 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65400f930 con 0x5635073955d0 2024-09-19T06:19:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.861+0000 7fd6683e3740 1 -- 172.21.15.5:0/1323506143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073955d0 msgr2=0x563507395a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.861+0000 7fd6683e3740 1 --2- 172.21.15.5:0/1323506143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073955d0 0x563507395a10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd6540114c0 tx=0x7fd654011980 comp rx=0 tx=0).stop 2024-09-19T06:19:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.861+0000 7fd6683e3740 1 -- 172.21.15.5:0/1323506143 shutdown_connections 2024-09-19T06:19:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.861+0000 7fd6683e3740 1 --2- 172.21.15.5:0/1323506143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 0x5635073aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.861+0000 7fd6683e3740 1 --2- 172.21.15.5:0/1323506143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073955d0 0x563507395a10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.861+0000 7fd6683e3740 1 --2- 172.21.15.5:0/1323506143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x563507395090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.861+0000 7fd6683e3740 1 -- 172.21.15.5:0/1323506143 >> 172.21.15.5:0/1323506143 conn(0x56350735e310 msgr2=0x563507391c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.861+0000 7fd6683e3740 1 -- 172.21.15.5:0/1323506143 shutdown_connections 2024-09-19T06:19:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.861+0000 7fd6683e3740 1 -- 172.21.15.5:0/1323506143 wait complete. 2024-09-19T06:19:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.861+0000 7fd6683e3740 1 -- start start 2024-09-19T06:19:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.861+0000 7fd6683e3740 1 Processor -- start 2024-09-19T06:19:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd6683e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073955d0 0x5635073a85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd6683e3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x5635073a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd6683e3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073a8b80 0x5635073a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd6683e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5635073ab880 con 0x5635073955d0 2024-09-19T06:19:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd6683e3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5635072b6060 con 0x563507396b40 2024-09-19T06:19:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd6683e3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5635073aba00 con 0x5635073a8b80 2024-09-19T06:19:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd65f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073955d0 0x5635073a85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd65f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073955d0 0x5635073a85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38148/0 (socket says 172.21.15.5:38148) 2024-09-19T06:19:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd65f7fe640 1 -- 172.21.15.5:0/3834872216 learned_addr learned my addr 172.21.15.5:0/3834872216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd6683e3740 10 client.0 _ll_register_callbacks cb 0x5635070098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd65effd640 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x5635073a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd65ffff640 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073a8b80 0x5635073a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd65f7fe640 1 -- 172.21.15.5:0/3834872216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073a8b80 msgr2=0x5635073a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd65f7fe640 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073a8b80 0x5635073a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd65f7fe640 1 -- 172.21.15.5:0/3834872216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 msgr2=0x5635073a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd65f7fe640 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x5635073a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd65f7fe640 1 -- 172.21.15.5:0/3834872216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350706c3c0 con 0x5635073955d0 2024-09-19T06:19:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.862+0000 7fd65effd640 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x5635073a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.863+0000 7fd65f7fe640 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073955d0 0x5635073a85a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fd6500bc380 tx=0x7fd6500bbfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb9de073e59cee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.863+0000 7fd63effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:19:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.863+0000 7fd63effd640 1 -- 172.21.15.5:0/3834872216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650049040 con 0x5635073955d0 2024-09-19T06:19:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.863+0000 7fd63effd640 1 -- 172.21.15.5:0/3834872216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fd6500d52d0 con 0x5635073955d0 2024-09-19T06:19:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.863+0000 7fd6683e3740 1 -- 172.21.15.5:0/3834872216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6540041d0 con 0x5635073955d0 2024-09-19T06:19:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.863+0000 7fd63effd640 1 -- 172.21.15.5:0/3834872216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd6500c06f0 con 0x5635073955d0 2024-09-19T06:19:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.863+0000 7fd6683e3740 10 client.19608 fetch_fsmap learned FSMap version 49 2024-09-19T06:19:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.863+0000 7fd6683e3740 1 -- 172.21.15.5:0/3834872216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5635073b7300 con 0x5635073955d0 2024-09-19T06:19:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.864+0000 7fd63effd640 1 -- 172.21.15.5:0/3834872216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6540041d0 con 0x5635073955d0 2024-09-19T06:19:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.864+0000 7fd63effd640 1 -- 172.21.15.5:0/3834872216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7fd6500c0170 con 0x5635073955d0 2024-09-19T06:19:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.864+0000 7fd6683e3740 10 client.19608 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T06:19:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.864+0000 7fd6683e3740 10 client.19608 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:19:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.864+0000 7fd6683e3740 20 client.19608 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.864+0000 7fd6683e3740 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5635073b8650 0x5635073baa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.864+0000 7fd6683e3740 4 client.19608 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:19:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.864+0000 7fd6683e3740 1 -- 172.21.15.5:0/3834872216 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563507308860 con 0x5635073b8650 2024-09-19T06:19:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.865+0000 7fd65effd640 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5635073b8650 0x5635073baa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.866+0000 7fd65effd640 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5635073b8650 0x5635073baa80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a076220355900fa6 server_cookie=9999fc4a329f7623 in_seq=0 out_seq=0 2024-09-19T06:19:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.866+0000 7fd63effd640 10 client.19608 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:19:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.866+0000 7fd63effd640 1 -- 172.21.15.5:0/3834872216 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563507308860 con 0x5635073b8650 2024-09-19T06:19:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.866+0000 7fd63effd640 10 client.19608 handle_command_reply: tid=0 2024-09-19T06:19:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.867+0000 7fd6683e3740 1 client.19608 shutdown 2024-09-19T06:19:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.867+0000 7fd6683e3740 1 -- 172.21.15.5:0/3834872216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073955d0 msgr2=0x5635073a85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.867+0000 7fd6683e3740 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073955d0 0x5635073a85a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fd6500bc380 tx=0x7fd6500bbfa0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.867+0000 7fd6683e3740 1 -- 172.21.15.5:0/3834872216 shutdown_connections 2024-09-19T06:19:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.867+0000 7fd6683e3740 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5635073b8650 0x5635073baa80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.867+0000 7fd6683e3740 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073a8b80 0x5635073a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.867+0000 7fd6683e3740 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x5635073a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.867+0000 7fd6683e3740 1 --2- 172.21.15.5:0/3834872216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073955d0 0x5635073a85a0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.867+0000 7fd6683e3740 1 -- 172.21.15.5:0/3834872216 >> 172.21.15.5:0/3834872216 conn(0x56350735e310 msgr2=0x563507397960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.867+0000 7fd63effd640 0 client.19608 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:19:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.867+0000 7fd6683e3740 1 -- 172.21.15.5:0/3834872216 shutdown_connections 2024-09-19T06:19:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.867+0000 7fd6683e3740 1 -- 172.21.15.5:0/3834872216 wait complete. 2024-09-19T06:19:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.868+0000 7fd6683e3740 20 client.19608 trim_cache size 0 max 16384 2024-09-19T06:19:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.892+0000 7fd6683e3740 1 Processor -- start 2024-09-19T06:19:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd6683e3740 1 -- start start 2024-09-19T06:19:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd6683e3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x563507391e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd6683e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 0x5635073b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd6683e3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073c0380 0x5635073c07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd6683e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601ab590 con 0x5635073a8b80 2024-09-19T06:19:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd6683e3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601ad250 con 0x563507396b40 2024-09-19T06:19:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd6683e3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601abd20 con 0x5635073c0380 2024-09-19T06:19:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd65effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 0x5635073b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd65effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 0x5635073b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38164/0 (socket says 172.21.15.5:38164) 2024-09-19T06:19:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd65effd640 1 -- 172.21.15.5:0/2845501243 learned_addr learned my addr 172.21.15.5:0/2845501243 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd65f7fe640 1 --2- 172.21.15.5:0/2845501243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x563507391e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd65ffff640 1 --2- 172.21.15.5:0/2845501243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073c0380 0x5635073c07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd65f7fe640 1 -- 172.21.15.5:0/2845501243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073c0380 msgr2=0x5635073c07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd65f7fe640 1 --2- 172.21.15.5:0/2845501243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073c0380 0x5635073c07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd65f7fe640 1 -- 172.21.15.5:0/2845501243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 msgr2=0x5635073b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd65f7fe640 1 --2- 172.21.15.5:0/2845501243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 0x5635073b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd65f7fe640 1 -- 172.21.15.5:0/2845501243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635073a8910 con 0x563507396b40 2024-09-19T06:19:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.893+0000 7fd65effd640 1 --2- 172.21.15.5:0/2845501243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 0x5635073b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.894+0000 7fd65f7fe640 1 --2- 172.21.15.5:0/2845501243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x563507391e10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd6500ba6a0 tx=0x7fd6500bc570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66be79992b4c71c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.894+0000 7fd6457fa640 1 -- 172.21.15.5:0/2845501243 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650049070 con 0x563507396b40 2024-09-19T06:19:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.894+0000 7fd6457fa640 1 -- 172.21.15.5:0/2845501243 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6500c4fb0 con 0x563507396b40 2024-09-19T06:19:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.894+0000 7fd6457fa640 1 -- 172.21.15.5:0/2845501243 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6500c0170 con 0x563507396b40 2024-09-19T06:19:22.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:22 smithi049 ceph-mon[34640]: pgmap v504: 97 pgs: 97 active+clean; 610 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 242 op/s 2024-09-19T06:19:22.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:22 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.895+0000 7fd6683e3740 1 -- 172.21.15.5:0/2845501243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 msgr2=0x563507391e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.895+0000 7fd6683e3740 1 --2- 172.21.15.5:0/2845501243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x563507391e10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd6500ba6a0 tx=0x7fd6500bc570 comp rx=0 tx=0).stop 2024-09-19T06:19:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.895+0000 7fd6683e3740 1 -- 172.21.15.5:0/2845501243 shutdown_connections 2024-09-19T06:19:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.895+0000 7fd6683e3740 1 --2- 172.21.15.5:0/2845501243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073c0380 0x5635073c07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.895+0000 7fd6683e3740 1 --2- 172.21.15.5:0/2845501243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635073a8b80 0x5635073b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.895+0000 7fd6683e3740 1 --2- 172.21.15.5:0/2845501243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563507396b40 0x563507391e10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.895+0000 7fd6683e3740 1 -- 172.21.15.5:0/2845501243 >> 172.21.15.5:0/2845501243 conn(0x56350735e310 msgr2=0x563507391700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.895+0000 7fd6683e3740 1 -- 172.21.15.5:0/2845501243 shutdown_connections 2024-09-19T06:19:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.895+0000 7fd6683e3740 1 -- 172.21.15.5:0/2845501243 wait complete. 2024-09-19T06:19:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.895+0000 7fd6683e3740 1 -- start start 2024-09-19T06:19:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd6683e3740 1 Processor -- start 2024-09-19T06:19:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd6683e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563507396b40 0x5635073a2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd6683e3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635073a8b80 0x5635073a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd6683e3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073c0380 0x5635073a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd6683e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601acb80 con 0x563507396b40 2024-09-19T06:19:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd6683e3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601ac3f0 con 0x5635073a8b80 2024-09-19T06:19:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd6683e3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601a9590 con 0x5635073c0380 2024-09-19T06:19:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd65f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563507396b40 0x5635073a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd6683e3740 10 client.0 _ll_register_callbacks cb 0x7fd6500b86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd65f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563507396b40 0x5635073a2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38176/0 (socket says 172.21.15.5:38176) 2024-09-19T06:19:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd65f7fe640 1 -- 172.21.15.5:0/454673820 learned_addr learned my addr 172.21.15.5:0/454673820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd65ffff640 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073c0380 0x5635073a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.896+0000 7fd65effd640 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635073a8b80 0x5635073a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd65ffff640 1 -- 172.21.15.5:0/454673820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635073a8b80 msgr2=0x5635073a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd65ffff640 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635073a8b80 0x5635073a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd65ffff640 1 -- 172.21.15.5:0/454673820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563507396b40 msgr2=0x5635073a2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd65ffff640 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563507396b40 0x5635073a2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd65ffff640 1 -- 172.21.15.5:0/454673820 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635074223d0 con 0x5635073c0380 2024-09-19T06:19:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd65ffff640 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073c0380 0x5635073a33e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd658009fd0 tx=0x7fd658018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b1bb6a204780e49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd63effd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:19:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd63effd640 1 -- 172.21.15.5:0/454673820 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65800fd60 con 0x5635073c0380 2024-09-19T06:19:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd63effd640 1 -- 172.21.15.5:0/454673820 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fd658004b20 con 0x5635073c0380 2024-09-19T06:19:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd6683e3740 1 -- 172.21.15.5:0/454673820 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6500c4fb0 con 0x5635073c0380 2024-09-19T06:19:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd63effd640 1 -- 172.21.15.5:0/454673820 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd658004d20 con 0x5635073c0380 2024-09-19T06:19:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd6683e3740 10 client.28939 fetch_fsmap learned FSMap version 49 2024-09-19T06:19:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.897+0000 7fd6683e3740 1 -- 172.21.15.5:0/454673820 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5635074236b0 con 0x5635073c0380 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.898+0000 7fd63effd640 1 -- 172.21.15.5:0/454673820 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd658004500 con 0x5635073c0380 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.898+0000 7fd63effd640 1 -- 172.21.15.5:0/454673820 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7fd6580326c0 con 0x5635073c0380 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.898+0000 7fd6683e3740 10 client.28939 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.898+0000 7fd6683e3740 10 client.28939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.898+0000 7fd6683e3740 20 client.28939 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.898+0000 7fd6683e3740 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x563507423de0 0x563507426200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.898+0000 7fd6683e3740 4 client.28939 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.898+0000 7fd6683e3740 1 -- 172.21.15.5:0/454673820 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd6500c4fb0 con 0x563507423de0 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.898+0000 7fd65f7fe640 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x563507423de0 0x563507426200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd63effd640 10 client.28939 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd65f7fe640 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x563507423de0 0x563507426200 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d75231ed4d526482 server_cookie=b8dd2ce59deb774d in_seq=0 out_seq=0 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd63effd640 1 -- 172.21.15.5:0/454673820 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd6500c4fb0 con 0x563507423de0 2024-09-19T06:19:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd63effd640 10 client.28939 handle_command_reply: tid=0 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd6683e3740 1 client.28939 shutdown 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd6683e3740 1 -- 172.21.15.5:0/454673820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073c0380 msgr2=0x5635073a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd6683e3740 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073c0380 0x5635073a33e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd658009fd0 tx=0x7fd658018000 comp rx=0 tx=0).stop 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd6683e3740 1 -- 172.21.15.5:0/454673820 shutdown_connections 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd6683e3740 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x563507423de0 0x563507426200 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd6683e3740 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635073c0380 0x5635073a33e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd6683e3740 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635073a8b80 0x5635073a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd63effd640 0 client.28939 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd6683e3740 1 --2- 172.21.15.5:0/454673820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563507396b40 0x5635073a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.899+0000 7fd6683e3740 1 -- 172.21.15.5:0/454673820 >> 172.21.15.5:0/454673820 conn(0x56350735e310 msgr2=0x563507391700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.900+0000 7fd6683e3740 1 -- 172.21.15.5:0/454673820 shutdown_connections 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.900+0000 7fd6683e3740 1 -- 172.21.15.5:0/454673820 wait complete. 2024-09-19T06:19:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.900+0000 7fd6683e3740 20 client.28939 trim_cache size 0 max 16384 2024-09-19T06:19:22.908 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:19:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.900+0000 7fd66600a640 1 -- 172.21.15.5:0/441431976 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd6480a33b0 msgr2=0x7fd6480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.900+0000 7fd66600a640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd6480a33b0 0x7fd6480a5870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd658005fd0 tx=0x7fd658009c20 comp rx=0 tx=0).stop 2024-09-19T06:19:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.900+0000 7fd66600a640 1 -- 172.21.15.5:0/441431976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd660102d50 msgr2=0x7fd6601009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.901+0000 7fd66600a640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd660102d50 0x7fd6601009b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd650005940 tx=0x7fd6500040c0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.901+0000 7fd66600a640 1 -- 172.21.15.5:0/441431976 shutdown_connections 2024-09-19T06:19:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.901+0000 7fd66600a640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66010c180 0x7fd660101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.901+0000 7fd66600a640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd6480a33b0 0x7fd6480a5870 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.901+0000 7fd66600a640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd660103670 0x7fd660100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.901+0000 7fd66600a640 1 --2- 172.21.15.5:0/441431976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd660102d50 0x7fd6601009b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.901+0000 7fd66600a640 1 -- 172.21.15.5:0/441431976 >> 172.21.15.5:0/441431976 conn(0x7fd6600fc7d0 msgr2=0x7fd6600fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.901+0000 7fd66600a640 1 -- 172.21.15.5:0/441431976 shutdown_connections 2024-09-19T06:19:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:22.901+0000 7fd66600a640 1 -- 172.21.15.5:0/441431976 wait complete. 2024-09-19T06:19:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:22 smithi157 ceph-mon[35346]: pgmap v504: 97 pgs: 97 active+clean; 610 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 242 op/s 2024-09-19T06:19:23.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:22 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:23.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28124 8.78 MB/sec execute 47 sec latency 102.912 ms 2024-09-19T06:19:24.383 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:19:24.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28283 8.77 MB/sec execute 48 sec latency 54.118 ms 2024-09-19T06:19:24.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:24 smithi005 ceph-mon[30955]: pgmap v505: 97 pgs: 97 active+clean; 612 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 382 op/s 2024-09-19T06:19:24.766 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:19:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:24 smithi049 ceph-mon[34640]: pgmap v505: 97 pgs: 97 active+clean; 612 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 382 op/s 2024-09-19T06:19:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:24 smithi157 ceph-mon[35346]: pgmap v505: 97 pgs: 97 active+clean; 612 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 382 op/s 2024-09-19T06:19:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.322+0000 7fa85df02640 1 -- 172.21.15.5:0/1939386949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8581049a0 msgr2=0x7fa8581089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.322+0000 7fa85df02640 1 --2- 172.21.15.5:0/1939386949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8581049a0 0x7fa8581089f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009900 tx=0x7fa84c02a410 comp rx=0 tx=0).stop 2024-09-19T06:19:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.322+0000 7fa85df02640 1 -- 172.21.15.5:0/1939386949 shutdown_connections 2024-09-19T06:19:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.322+0000 7fa85df02640 1 --2- 172.21.15.5:0/1939386949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa858109120 0x7fa858110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.322+0000 7fa85df02640 1 --2- 172.21.15.5:0/1939386949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8581049a0 0x7fa8581089f0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.322+0000 7fa85df02640 1 --2- 172.21.15.5:0/1939386949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858103ff0 0x7fa8581043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.322+0000 7fa85df02640 1 -- 172.21.15.5:0/1939386949 >> 172.21.15.5:0/1939386949 conn(0x7fa8580ffab0 msgr2=0x7fa858101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.322+0000 7fa85df02640 1 -- 172.21.15.5:0/1939386949 shutdown_connections 2024-09-19T06:19:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.322+0000 7fa85df02640 1 -- 172.21.15.5:0/1939386949 wait complete. 2024-09-19T06:19:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.324+0000 7fa85df02640 1 Processor -- start 2024-09-19T06:19:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.324+0000 7fa85df02640 1 -- start start 2024-09-19T06:19:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.324+0000 7fa85df02640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858103ff0 0x7fa858076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.324+0000 7fa85df02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa858109120 0x7fa858076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.324+0000 7fa85df02640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa858077180 0x7fa8581aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.324+0000 7fa85df02640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa858113ff0 con 0x7fa858109120 2024-09-19T06:19:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.324+0000 7fa85df02640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa858113e70 con 0x7fa858103ff0 2024-09-19T06:19:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.324+0000 7fa85df02640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa858114170 con 0x7fa858077180 2024-09-19T06:19:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.325+0000 7fa856ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa858109120 0x7fa858076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.325+0000 7fa857fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa858077180 0x7fa8581aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.325+0000 7fa857fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa858077180 0x7fa8581aad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42594/0 (socket says 172.21.15.5:42594) 2024-09-19T06:19:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.325+0000 7fa857fff640 1 -- 172.21.15.5:0/2466017599 learned_addr learned my addr 172.21.15.5:0/2466017599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.325+0000 7fa856ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa858109120 0x7fa858076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33412/0 (socket says 172.21.15.5:33412) 2024-09-19T06:19:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.325+0000 7fa8577fe640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858103ff0 0x7fa858076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.325+0000 7fa857fff640 1 -- 172.21.15.5:0/2466017599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858103ff0 msgr2=0x7fa858076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.325+0000 7fa857fff640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858103ff0 0x7fa858076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.325+0000 7fa857fff640 1 -- 172.21.15.5:0/2466017599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa858109120 msgr2=0x7fa858076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:25.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.325+0000 7fa857fff640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa858109120 0x7fa858076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.325+0000 7fa857fff640 1 -- 172.21.15.5:0/2466017599 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8581ab240 con 0x7fa858077180 2024-09-19T06:19:25.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.325+0000 7fa857fff640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa858077180 0x7fa8581aad00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa85000eb10 tx=0x7fa85000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:25.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.326+0000 7fa854ff9640 1 -- 172.21.15.5:0/2466017599 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850004b20 con 0x7fa858077180 2024-09-19T06:19:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.326+0000 7fa854ff9640 1 -- 172.21.15.5:0/2466017599 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa850004cc0 con 0x7fa858077180 2024-09-19T06:19:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.326+0000 7fa854ff9640 1 -- 172.21.15.5:0/2466017599 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850010690 con 0x7fa858077180 2024-09-19T06:19:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.326+0000 7fa85df02640 1 -- 172.21.15.5:0/2466017599 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8581ab530 con 0x7fa858077180 2024-09-19T06:19:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.326+0000 7fa85df02640 1 -- 172.21.15.5:0/2466017599 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8581ab9f0 con 0x7fa858077180 2024-09-19T06:19:25.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.329+0000 7fa8602db740 1 Processor -- start 2024-09-19T06:19:25.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.329+0000 7fa854ff9640 1 -- 172.21.15.5:0/2466017599 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa850005020 con 0x7fa858077180 2024-09-19T06:19:25.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa8602db740 1 -- start start 2024-09-19T06:19:25.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa8602db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71eb40 0x5602eb71d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa8602db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 0x5602eb71da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa8602db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb72a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa8602db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5602eb6832f0 con 0x5602eb71eb40 2024-09-19T06:19:25.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa8602db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5602eb62acf0 con 0x5602eb728370 2024-09-19T06:19:25.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa8602db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5602eb67f2b0 con 0x5602eb71d5d0 2024-09-19T06:19:25.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa856ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71eb40 0x5602eb71d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa8577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 0x5602eb71da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa8577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 0x5602eb71da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42602/0 (socket says 172.21.15.5:42602) 2024-09-19T06:19:25.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa8577fe640 1 -- 172.21.15.5:0/4265884318 learned_addr learned my addr 172.21.15.5:0/4265884318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:25.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa856ffd640 1 --2- 172.21.15.5:0/4265884318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb72a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.330+0000 7fa8577fe640 1 -- 172.21.15.5:0/4265884318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 msgr2=0x5602eb72a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:25.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.331+0000 7fa8577fe640 1 --2- 172.21.15.5:0/4265884318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb72a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.331+0000 7fa8577fe640 1 -- 172.21.15.5:0/4265884318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71eb40 msgr2=0x5602eb71d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:25.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.331+0000 7fa8577fe640 1 --2- 172.21.15.5:0/4265884318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71eb40 0x5602eb71d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.331+0000 7fa8577fe640 1 -- 172.21.15.5:0/4265884318 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602eb60f3a0 con 0x5602eb71d5d0 2024-09-19T06:19:25.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.331+0000 7fa856ffd640 1 --2- 172.21.15.5:0/4265884318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb72a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:19:25.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.331+0000 7fa854ff9640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa8400a3430 0x7fa8400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.331+0000 7fa856ffd640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa8400a3430 0x7fa8400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.331+0000 7fa854ff9640 1 -- 172.21.15.5:0/2466017599 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fa8500d0340 con 0x7fa858077180 2024-09-19T06:19:25.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.331+0000 7fa8577fe640 1 --2- 172.21.15.5:0/4265884318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 0x5602eb71da10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa848009cf0 tx=0x7fa848011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eb26268a7ed9516 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:25.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.332+0000 7fa83dffb640 1 -- 172.21.15.5:0/4265884318 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84801e070 con 0x5602eb71d5d0 2024-09-19T06:19:25.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.332+0000 7fa83dffb640 1 -- 172.21.15.5:0/4265884318 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa848019ae0 con 0x5602eb71d5d0 2024-09-19T06:19:25.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.332+0000 7fa83dffb640 1 -- 172.21.15.5:0/4265884318 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84800e990 con 0x5602eb71d5d0 2024-09-19T06:19:25.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.333+0000 7fa8602db740 1 -- 172.21.15.5:0/4265884318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 msgr2=0x5602eb71da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:25.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.333+0000 7fa8602db740 1 --2- 172.21.15.5:0/4265884318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 0x5602eb71da10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa848009cf0 tx=0x7fa848011810 comp rx=0 tx=0).stop 2024-09-19T06:19:25.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.333+0000 7fa856ffd640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa8400a3430 0x7fa8400a58f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa84c005940 tx=0x7fa84c005e50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:25.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.333+0000 7fa8602db740 1 -- 172.21.15.5:0/4265884318 shutdown_connections 2024-09-19T06:19:25.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.333+0000 7fa8602db740 1 --2- 172.21.15.5:0/4265884318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb72a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.333+0000 7fa8602db740 1 --2- 172.21.15.5:0/4265884318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 0x5602eb71da10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.333+0000 7fa8602db740 1 --2- 172.21.15.5:0/4265884318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71eb40 0x5602eb71d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.333+0000 7fa8602db740 1 -- 172.21.15.5:0/4265884318 >> 172.21.15.5:0/4265884318 conn(0x5602eb6e6310 msgr2=0x5602eb719c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:25.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.333+0000 7fa8602db740 1 -- 172.21.15.5:0/4265884318 shutdown_connections 2024-09-19T06:19:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.333+0000 7fa8602db740 1 -- 172.21.15.5:0/4265884318 wait complete. 2024-09-19T06:19:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.334+0000 7fa8602db740 1 -- start start 2024-09-19T06:19:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.334+0000 7fa8602db740 1 Processor -- start 2024-09-19T06:19:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.334+0000 7fa8602db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb71d5d0 0x5602eb727d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.334+0000 7fa8602db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71eb40 0x5602eb724fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.334+0000 7fa8602db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb728370 0x5602eb725570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa8602db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5602eb72b0c0 con 0x5602eb728370 2024-09-19T06:19:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa8602db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5602eb63e060 con 0x5602eb71d5d0 2024-09-19T06:19:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa8602db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5602eb72b240 con 0x5602eb71eb40 2024-09-19T06:19:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa856ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71eb40 0x5602eb724fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa8602db740 10 client.0 _ll_register_callbacks cb 0x5602eb3918c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa8577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb71d5d0 0x5602eb727d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa857fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb728370 0x5602eb725570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa856ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71eb40 0x5602eb724fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42608/0 (socket says 172.21.15.5:42608) 2024-09-19T06:19:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa8577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb71d5d0 0x5602eb727d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59834/0 (socket says 172.21.15.5:59834) 2024-09-19T06:19:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa856ffd640 1 -- 172.21.15.5:0/461815833 learned_addr learned my addr 172.21.15.5:0/461815833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa856ffd640 1 -- 172.21.15.5:0/461815833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb71d5d0 msgr2=0x5602eb727d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa856ffd640 1 --2- 172.21.15.5:0/461815833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb71d5d0 0x5602eb727d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa856ffd640 1 -- 172.21.15.5:0/461815833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb728370 msgr2=0x5602eb725570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa856ffd640 1 --2- 172.21.15.5:0/461815833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb728370 0x5602eb725570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa856ffd640 1 -- 172.21.15.5:0/461815833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602eb3f43c0 con 0x5602eb71eb40 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.335+0000 7fa8577fe640 1 --2- 172.21.15.5:0/461815833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb71d5d0 0x5602eb727d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.336+0000 7fa856ffd640 1 --2- 172.21.15.5:0/461815833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71eb40 0x5602eb724fd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009900 tx=0x7fa84c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=647eaeb6e0079394 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.336+0000 7fa836ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.336+0000 7fa836ffd640 1 -- 172.21.15.5:0/461815833 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c035040 con 0x5602eb71eb40 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.336+0000 7fa8602db740 1 -- 172.21.15.5:0/461815833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa848019ae0 con 0x5602eb71eb40 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.336+0000 7fa836ffd640 1 -- 172.21.15.5:0/461815833 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fa84c004450 con 0x5602eb71eb40 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.337+0000 7fa836ffd640 1 -- 172.21.15.5:0/461815833 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa84c05a050 con 0x5602eb71eb40 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.337+0000 7fa8602db740 10 client.28951 fetch_fsmap learned FSMap version 50 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.337+0000 7fa8602db740 1 -- 172.21.15.5:0/461815833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5602eb736ad0 con 0x5602eb71eb40 2024-09-19T06:19:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.337+0000 7fa836ffd640 1 -- 172.21.15.5:0/461815833 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa848019ae0 con 0x5602eb71eb40 2024-09-19T06:19:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.337+0000 7fa836ffd640 1 -- 172.21.15.5:0/461815833 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 50) ==== 8394+0+0 (secure 0 0 0) 0x7fa84c004b10 con 0x5602eb71eb40 2024-09-19T06:19:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.337+0000 7fa8602db740 10 client.28951 fetch_fsmap finished waiting for FSMap version 50 2024-09-19T06:19:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.337+0000 7fa8602db740 10 client.28951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.337+0000 7fa8602db740 20 client.28951 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.337+0000 7fa8602db740 1 --2- 172.21.15.5:0/461815833 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5602eb738450 0x5602eb73a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.337+0000 7fa8602db740 4 client.28951 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:19:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.337+0000 7fa8602db740 1 -- 172.21.15.5:0/461815833 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602eb690860 con 0x5602eb738450 2024-09-19T06:19:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.337+0000 7fa8577fe640 1 --2- 172.21.15.5:0/461815833 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5602eb738450 0x5602eb73a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.338+0000 7fa836ffd640 10 client.28951 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.338+0000 7fa8577fe640 1 --2- 172.21.15.5:0/461815833 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5602eb738450 0x5602eb73a8f0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a6a19542da355bd server_cookie=d8181d50cdfc5a90 in_seq=0 out_seq=0 2024-09-19T06:19:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa836ffd640 1 -- 172.21.15.5:0/461815833 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602eb690860 con 0x5602eb738450 2024-09-19T06:19:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa836ffd640 10 client.28951 handle_command_reply: tid=0 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa8602db740 1 client.28951 shutdown 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa8602db740 1 -- 172.21.15.5:0/461815833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71eb40 msgr2=0x5602eb724fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa8602db740 1 --2- 172.21.15.5:0/461815833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71eb40 0x5602eb724fd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009900 tx=0x7fa84c02a410 comp rx=0 tx=0).stop 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa8602db740 1 -- 172.21.15.5:0/461815833 shutdown_connections 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa8602db740 1 --2- 172.21.15.5:0/461815833 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5602eb738450 0x5602eb73a8f0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa8602db740 1 --2- 172.21.15.5:0/461815833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb728370 0x5602eb725570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa8602db740 1 --2- 172.21.15.5:0/461815833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71eb40 0x5602eb724fd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa8602db740 1 --2- 172.21.15.5:0/461815833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb71d5d0 0x5602eb727d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa8602db740 1 -- 172.21.15.5:0/461815833 >> 172.21.15.5:0/461815833 conn(0x5602eb6e6310 msgr2=0x5602eb71f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa836ffd640 0 client.28951 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa8602db740 1 -- 172.21.15.5:0/461815833 shutdown_connections 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa8602db740 1 -- 172.21.15.5:0/461815833 wait complete. 2024-09-19T06:19:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.339+0000 7fa8602db740 20 client.28951 trim_cache size 0 max 16384 2024-09-19T06:19:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.355+0000 7fa8602db740 1 Processor -- start 2024-09-19T06:19:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa8602db740 1 -- start start 2024-09-19T06:19:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa8602db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 0x5602eb734360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa8602db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71eb40 0x5602eb724890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa8602db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb7202c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa8602db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b4440 con 0x5602eb71eb40 2024-09-19T06:19:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa8602db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b60d0 con 0x5602eb728370 2024-09-19T06:19:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa8602db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b4bd0 con 0x5602eb71d5d0 2024-09-19T06:19:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa857fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb7202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa856ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71eb40 0x5602eb724890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa8577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 0x5602eb734360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa856ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71eb40 0x5602eb724890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33452/0 (socket says 172.21.15.5:33452) 2024-09-19T06:19:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa857fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb7202c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59848/0 (socket says 172.21.15.5:59848) 2024-09-19T06:19:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa856ffd640 1 -- 172.21.15.5:0/4197584994 learned_addr learned my addr 172.21.15.5:0/4197584994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa856ffd640 1 -- 172.21.15.5:0/4197584994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 msgr2=0x5602eb734360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa856ffd640 1 --2- 172.21.15.5:0/4197584994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 0x5602eb734360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa856ffd640 1 -- 172.21.15.5:0/4197584994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 msgr2=0x5602eb7202c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa856ffd640 1 --2- 172.21.15.5:0/4197584994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb7202c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa856ffd640 1 -- 172.21.15.5:0/4197584994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602eb73af70 con 0x5602eb71eb40 2024-09-19T06:19:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa8577fe640 1 --2- 172.21.15.5:0/4197584994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 0x5602eb734360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.356+0000 7fa857fff640 1 --2- 172.21.15.5:0/4197584994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb7202c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.357+0000 7fa856ffd640 1 --2- 172.21.15.5:0/4197584994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71eb40 0x5602eb724890 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009870 tx=0x7fa84c031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8bcc6ca806471d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.357+0000 7fa83d7fa640 1 -- 172.21.15.5:0/4197584994 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c035070 con 0x5602eb71eb40 2024-09-19T06:19:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.357+0000 7fa83d7fa640 1 -- 172.21.15.5:0/4197584994 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa84c032e60 con 0x5602eb71eb40 2024-09-19T06:19:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.357+0000 7fa83d7fa640 1 -- 172.21.15.5:0/4197584994 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c004990 con 0x5602eb71eb40 2024-09-19T06:19:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.358+0000 7fa8602db740 1 -- 172.21.15.5:0/4197584994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71eb40 msgr2=0x5602eb724890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.358+0000 7fa8602db740 1 --2- 172.21.15.5:0/4197584994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71eb40 0x5602eb724890 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009870 tx=0x7fa84c031730 comp rx=0 tx=0).stop 2024-09-19T06:19:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.358+0000 7fa8602db740 1 -- 172.21.15.5:0/4197584994 shutdown_connections 2024-09-19T06:19:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.358+0000 7fa8602db740 1 --2- 172.21.15.5:0/4197584994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb7202c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.358+0000 7fa8602db740 1 --2- 172.21.15.5:0/4197584994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71eb40 0x5602eb724890 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.358+0000 7fa8602db740 1 --2- 172.21.15.5:0/4197584994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71d5d0 0x5602eb734360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.358+0000 7fa8602db740 1 -- 172.21.15.5:0/4197584994 >> 172.21.15.5:0/4197584994 conn(0x5602eb6e6310 msgr2=0x5602eb71cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.358+0000 7fa8602db740 1 -- 172.21.15.5:0/4197584994 shutdown_connections 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.358+0000 7fa8602db740 1 -- 172.21.15.5:0/4197584994 wait complete. 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.359+0000 7fa8602db740 1 -- start start 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.359+0000 7fa8602db740 1 Processor -- start 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.359+0000 7fa8602db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71d5d0 0x5602eb728d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.359+0000 7fa8602db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71eb40 0x5602eb729290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.359+0000 7fa8602db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb7297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.359+0000 7fa8602db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b5a00 con 0x5602eb71d5d0 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.359+0000 7fa8602db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b52a0 con 0x5602eb728370 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.359+0000 7fa8602db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b2440 con 0x5602eb71eb40 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.359+0000 7fa8602db740 10 client.0 _ll_register_callbacks cb 0x7fa84c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.359+0000 7fa8577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71d5d0 0x5602eb728d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.359+0000 7fa8577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71d5d0 0x5602eb728d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33456/0 (socket says 172.21.15.5:33456) 2024-09-19T06:19:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.359+0000 7fa8577fe640 1 -- 172.21.15.5:0/1594982194 learned_addr learned my addr 172.21.15.5:0/1594982194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.360+0000 7fa856ffd640 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71eb40 0x5602eb729290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.360+0000 7fa857fff640 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb7297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.360+0000 7fa8577fe640 1 -- 172.21.15.5:0/1594982194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71eb40 msgr2=0x5602eb729290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.360+0000 7fa8577fe640 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71eb40 0x5602eb729290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.360+0000 7fa8577fe640 1 -- 172.21.15.5:0/1594982194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 msgr2=0x5602eb7297d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.360+0000 7fa8577fe640 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb7297d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.360+0000 7fa8577fe640 1 -- 172.21.15.5:0/1594982194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602eb72f770 con 0x5602eb71d5d0 2024-09-19T06:19:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.360+0000 7fa8577fe640 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71d5d0 0x5602eb728d50 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fa848009250 tx=0x7fa848009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7176ed8427ecf0c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.360+0000 7fa836ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:19:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.360+0000 7fa836ffd640 1 -- 172.21.15.5:0/1594982194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84801e040 con 0x5602eb71d5d0 2024-09-19T06:19:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.360+0000 7fa836ffd640 1 -- 172.21.15.5:0/1594982194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fa8480055e0 con 0x5602eb71d5d0 2024-09-19T06:19:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.360+0000 7fa8602db740 1 -- 172.21.15.5:0/1594982194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa84c032e60 con 0x5602eb71d5d0 2024-09-19T06:19:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.361+0000 7fa836ffd640 1 -- 172.21.15.5:0/1594982194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa848038050 con 0x5602eb71d5d0 2024-09-19T06:19:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.361+0000 7fa8602db740 10 client.19632 fetch_fsmap learned FSMap version 50 2024-09-19T06:19:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.361+0000 7fa8602db740 1 -- 172.21.15.5:0/1594982194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5602eb730a70 con 0x5602eb71d5d0 2024-09-19T06:19:25.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.361+0000 7fa836ffd640 1 -- 172.21.15.5:0/1594982194 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa84c032e60 con 0x5602eb71d5d0 2024-09-19T06:19:25.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.361+0000 7fa836ffd640 1 -- 172.21.15.5:0/1594982194 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8394+0+0 (secure 0 0 0) 0x7fa84800e660 con 0x5602eb71d5d0 2024-09-19T06:19:25.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.361+0000 7fa8602db740 10 client.19632 fetch_fsmap finished waiting for FSMap version 50 2024-09-19T06:19:25.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.361+0000 7fa8602db740 10 client.19632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:25.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.361+0000 7fa8602db740 20 client.19632 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:25.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.361+0000 7fa8602db740 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5602eb730f60 0x5602eb7a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:25.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.361+0000 7fa8602db740 4 client.19632 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:19:25.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.361+0000 7fa8602db740 1 -- 172.21.15.5:0/1594982194 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602eb68ad70 con 0x5602eb730f60 2024-09-19T06:19:25.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.362+0000 7fa856ffd640 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5602eb730f60 0x5602eb7a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:25.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.362+0000 7fa836ffd640 10 client.19632 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:25.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.362+0000 7fa856ffd640 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5602eb730f60 0x5602eb7a51a0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34d3ef4bb8b7c975 server_cookie=fbe8d228f4db8f62 in_seq=0 out_seq=0 2024-09-19T06:19:25.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.362+0000 7fa836ffd640 1 -- 172.21.15.5:0/1594982194 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602eb68ad70 con 0x5602eb730f60 2024-09-19T06:19:25.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.362+0000 7fa836ffd640 10 client.19632 handle_command_reply: tid=0 2024-09-19T06:19:25.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.362+0000 7fa8602db740 1 client.19632 shutdown 2024-09-19T06:19:25.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.363+0000 7fa8602db740 1 -- 172.21.15.5:0/1594982194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71d5d0 msgr2=0x5602eb728d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:25.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.363+0000 7fa8602db740 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71d5d0 0x5602eb728d50 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fa848009250 tx=0x7fa848009c60 comp rx=0 tx=0).stop 2024-09-19T06:19:25.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.363+0000 7fa8602db740 1 -- 172.21.15.5:0/1594982194 shutdown_connections 2024-09-19T06:19:25.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.363+0000 7fa8602db740 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5602eb730f60 0x5602eb7a51a0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.363+0000 7fa8602db740 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602eb728370 0x5602eb7297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.363+0000 7fa8602db740 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602eb71eb40 0x5602eb729290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.363+0000 7fa8602db740 1 --2- 172.21.15.5:0/1594982194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602eb71d5d0 0x5602eb728d50 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.363+0000 7fa8602db740 1 -- 172.21.15.5:0/1594982194 >> 172.21.15.5:0/1594982194 conn(0x5602eb6e6310 msgr2=0x5602eb7202c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:25.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.363+0000 7fa836ffd640 0 client.19632 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:25.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.363+0000 7fa8602db740 1 -- 172.21.15.5:0/1594982194 shutdown_connections 2024-09-19T06:19:25.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.363+0000 7fa8602db740 1 -- 172.21.15.5:0/1594982194 wait complete. 2024-09-19T06:19:25.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.363+0000 7fa8602db740 20 client.19632 trim_cache size 0 max 16384 2024-09-19T06:19:25.369 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:19:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.364+0000 7fa85df02640 1 -- 172.21.15.5:0/2466017599 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa8400a3430 msgr2=0x7fa8400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.364+0000 7fa85df02640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa8400a3430 0x7fa8400a58f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa84c005940 tx=0x7fa84c005e50 comp rx=0 tx=0).stop 2024-09-19T06:19:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.364+0000 7fa85df02640 1 -- 172.21.15.5:0/2466017599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa858077180 msgr2=0x7fa8581aad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.364+0000 7fa85df02640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa858077180 0x7fa8581aad00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa85000eb10 tx=0x7fa85000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.364+0000 7fa85df02640 1 -- 172.21.15.5:0/2466017599 shutdown_connections 2024-09-19T06:19:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.364+0000 7fa85df02640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa8400a3430 0x7fa8400a58f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.364+0000 7fa85df02640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa858077180 0x7fa8581aad00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.364+0000 7fa85df02640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa858109120 0x7fa858076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.364+0000 7fa85df02640 1 --2- 172.21.15.5:0/2466017599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858103ff0 0x7fa858076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.364+0000 7fa85df02640 1 -- 172.21.15.5:0/2466017599 >> 172.21.15.5:0/2466017599 conn(0x7fa8580ffab0 msgr2=0x7fa858100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:25.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.364+0000 7fa85df02640 1 -- 172.21.15.5:0/2466017599 shutdown_connections 2024-09-19T06:19:25.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:25.364+0000 7fa85df02640 1 -- 172.21.15.5:0/2466017599 wait complete. 2024-09-19T06:19:25.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28461 8.77 MB/sec execute 49 sec latency 77.517 ms 2024-09-19T06:19:25.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:25 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:25.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:25 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:26.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:25 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:26.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28682 8.88 MB/sec execute 50 sec latency 58.195 ms 2024-09-19T06:19:26.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:26 smithi005 ceph-mon[30955]: pgmap v506: 97 pgs: 97 active+clean; 612 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 249 op/s 2024-09-19T06:19:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:26 smithi049 ceph-mon[34640]: pgmap v506: 97 pgs: 97 active+clean; 612 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 249 op/s 2024-09-19T06:19:27.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:26 smithi157 ceph-mon[35346]: pgmap v506: 97 pgs: 97 active+clean; 612 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 249 op/s 2024-09-19T06:19:27.164 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:19:27.164 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:19:27.539 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:19:27.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28844 8.79 MB/sec execute 51 sec latency 62.557 ms 2024-09-19T06:19:27.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:27 smithi005 ceph-mon[30955]: pgmap v507: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.3 MiB/s wr, 383 op/s 2024-09-19T06:19:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:27 smithi049 ceph-mon[34640]: pgmap v507: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.3 MiB/s wr, 383 op/s 2024-09-19T06:19:28.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:27 smithi157 ceph-mon[35346]: pgmap v507: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.3 MiB/s wr, 383 op/s 2024-09-19T06:19:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.109+0000 7fb2ca880640 1 -- 172.21.15.5:0/1933882249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4106b80 msgr2=0x7fb2c4106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.109+0000 7fb2ca880640 1 --2- 172.21.15.5:0/1933882249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4106b80 0x7fb2c4106f60 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009900 tx=0x7fb2b402a410 comp rx=0 tx=0).stop 2024-09-19T06:19:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.109+0000 7fb2ca880640 1 -- 172.21.15.5:0/1933882249 shutdown_connections 2024-09-19T06:19:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.109+0000 7fb2ca880640 1 --2- 172.21.15.5:0/1933882249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2c4100aa0 0x7fb2c410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.109+0000 7fb2ca880640 1 --2- 172.21.15.5:0/1933882249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c4100100 0x7fb2c4100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.109+0000 7fb2ca880640 1 --2- 172.21.15.5:0/1933882249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4106b80 0x7fb2c4106f60 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.109+0000 7fb2ca880640 1 -- 172.21.15.5:0/1933882249 >> 172.21.15.5:0/1933882249 conn(0x7fb2c4077230 msgr2=0x7fb2c40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:28.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.110+0000 7fb2ca880640 1 -- 172.21.15.5:0/1933882249 shutdown_connections 2024-09-19T06:19:28.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.110+0000 7fb2ca880640 1 -- 172.21.15.5:0/1933882249 wait complete. 2024-09-19T06:19:28.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.111+0000 7fb2ca880640 1 Processor -- start 2024-09-19T06:19:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.111+0000 7fb2ca880640 1 -- start start 2024-09-19T06:19:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.111+0000 7fb2ca880640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4100100 0x7fb2c4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2ca880640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2c4100aa0 0x7fb2c41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2ca880640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c4106b80 0x7fb2c419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2ca880640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4111f90 con 0x7fb2c4100100 2024-09-19T06:19:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2ca880640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4111e10 con 0x7fb2c4100aa0 2024-09-19T06:19:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2ca880640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4112110 con 0x7fb2c4106b80 2024-09-19T06:19:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4100100 0x7fb2c4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2c4100aa0 0x7fb2c41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4100100 0x7fb2c4198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33476/0 (socket says 172.21.15.5:33476) 2024-09-19T06:19:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2c3fff640 1 -- 172.21.15.5:0/2811632783 learned_addr learned my addr 172.21.15.5:0/2811632783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2c8e1d640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c4106b80 0x7fb2c419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2c8e1d640 1 -- 172.21.15.5:0/2811632783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2c4100aa0 msgr2=0x7fb2c41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2c8e1d640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2c4100aa0 0x7fb2c41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2c8e1d640 1 -- 172.21.15.5:0/2811632783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4100100 msgr2=0x7fb2c4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2c8e1d640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4100100 0x7fb2c4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2c8e1d640 1 -- 172.21.15.5:0/2811632783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c419d440 con 0x7fb2c4106b80 2024-09-19T06:19:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.112+0000 7fb2c3fff640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4100100 0x7fb2c4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.113+0000 7fb2c8e1d640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c4106b80 0x7fb2c419cd60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005fd0 tx=0x7fb2bc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.113+0000 7fb2c17fa640 1 -- 172.21.15.5:0/2811632783 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc00cdf0 con 0x7fb2c4106b80 2024-09-19T06:19:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.113+0000 7fb2c17fa640 1 -- 172.21.15.5:0/2811632783 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2bc005020 con 0x7fb2c4106b80 2024-09-19T06:19:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.113+0000 7fb2ca880640 1 -- 172.21.15.5:0/2811632783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c419d730 con 0x7fb2c4106b80 2024-09-19T06:19:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.113+0000 7fb2ca880640 1 -- 172.21.15.5:0/2811632783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c4101210 con 0x7fb2c4106b80 2024-09-19T06:19:28.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.113+0000 7fb2c17fa640 1 -- 172.21.15.5:0/2811632783 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc010690 con 0x7fb2c4106b80 2024-09-19T06:19:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.116+0000 7fb2c17fa640 1 -- 172.21.15.5:0/2811632783 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb2bc004b20 con 0x7fb2c4106b80 2024-09-19T06:19:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.116+0000 7fb2ccc59740 1 Processor -- start 2024-09-19T06:19:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.116+0000 7fb2ccc59740 1 -- start start 2024-09-19T06:19:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.116+0000 7fb2ccc59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bc5370 0x560b87bc5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.116+0000 7fb2ccc59740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5c90 0x560b87bd0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.116+0000 7fb2ccc59740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bd0550 0x560b87bd2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2ccc59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560b87b22280 con 0x560b87bc5370 2024-09-19T06:19:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2ccc59740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560b87ac9b10 con 0x560b87bc5c90 2024-09-19T06:19:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2ccc59740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560b87b1e240 con 0x560b87bd0550 2024-09-19T06:19:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2c3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5c90 0x560b87bd0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2c37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bd0550 0x560b87bd2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bc5370 0x560b87bc5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2c3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5c90 0x560b87bd0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59880/0 (socket says 172.21.15.5:59880) 2024-09-19T06:19:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2c3fff640 1 -- 172.21.15.5:0/486958619 learned_addr learned my addr 172.21.15.5:0/486958619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2c37fe640 1 -- 172.21.15.5:0/486958619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bd0550 msgr2=0x560b87bd2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2c37fe640 1 --2- 172.21.15.5:0/486958619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bd0550 0x560b87bd2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2c37fe640 1 -- 172.21.15.5:0/486958619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5c90 msgr2=0x560b87bd0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2c37fe640 1 --2- 172.21.15.5:0/486958619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5c90 0x560b87bd0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2c37fe640 1 -- 172.21.15.5:0/486958619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b87ad61f0 con 0x560b87bc5370 2024-09-19T06:19:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2c17fa640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb2ac0a3430 0x7fb2ac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.117+0000 7fb2c17fa640 1 -- 172.21.15.5:0/2811632783 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fb2bc0d0380 con 0x7fb2c4106b80 2024-09-19T06:19:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.118+0000 7fb2c37fe640 1 --2- 172.21.15.5:0/486958619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bc5370 0x560b87bc5750 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x560b87bc2c80 tx=0x7fb2b800d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ad0c1b0737a7aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.118+0000 7fb2c3fff640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb2ac0a3430 0x7fb2ac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.118+0000 7fb2aa7fc640 1 -- 172.21.15.5:0/486958619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b801b410 con 0x560b87bc5370 2024-09-19T06:19:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.118+0000 7fb2aa7fc640 1 -- 172.21.15.5:0/486958619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2b8015900 con 0x560b87bc5370 2024-09-19T06:19:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.118+0000 7fb2aa7fc640 1 -- 172.21.15.5:0/486958619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b801a530 con 0x560b87bc5370 2024-09-19T06:19:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.119+0000 7fb2ccc59740 1 -- 172.21.15.5:0/486958619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bc5370 msgr2=0x560b87bc5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.119+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/486958619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bc5370 0x560b87bc5750 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x560b87bc2c80 tx=0x7fb2b800d4f0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.119+0000 7fb2c3fff640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb2ac0a3430 0x7fb2ac0a58f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4004650 tx=0x7fb2b4005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.119+0000 7fb2ccc59740 1 -- 172.21.15.5:0/486958619 shutdown_connections 2024-09-19T06:19:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.119+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/486958619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bd0550 0x560b87bd2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.119+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/486958619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5c90 0x560b87bd0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.119+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/486958619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bc5370 0x560b87bc5750 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.119+0000 7fb2ccc59740 1 -- 172.21.15.5:0/486958619 >> 172.21.15.5:0/486958619 conn(0x560b87bc1900 msgr2=0x560b87bc1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.120+0000 7fb2ccc59740 1 -- 172.21.15.5:0/486958619 shutdown_connections 2024-09-19T06:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.120+0000 7fb2ccc59740 1 -- 172.21.15.5:0/486958619 wait complete. 2024-09-19T06:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.120+0000 7fb2ccc59740 1 -- start start 2024-09-19T06:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.120+0000 7fb2ccc59740 1 Processor -- start 2024-09-19T06:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.121+0000 7fb2ccc59740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5370 0x560b87bcd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.121+0000 7fb2ccc59740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5c90 0x560b87bcd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.121+0000 7fb2ccc59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bd0550 0x560b87bcdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.121+0000 7fb2c3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5370 0x560b87bcd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.121+0000 7fb2ccc59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560b87bd3120 con 0x560b87bd0550 2024-09-19T06:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.121+0000 7fb2ccc59740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560b87adcf10 con 0x560b87bc5370 2024-09-19T06:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.121+0000 7fb2ccc59740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560b87bd32a0 con 0x560b87bc5c90 2024-09-19T06:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.121+0000 7fb2c3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5370 0x560b87bcd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59890/0 (socket says 172.21.15.5:59890) 2024-09-19T06:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.121+0000 7fb2c3fff640 1 -- 172.21.15.5:0/3974998004 learned_addr learned my addr 172.21.15.5:0/3974998004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.121+0000 7fb2c8e1d640 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bd0550 0x560b87bcdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.121+0000 7fb2ccc59740 10 client.0 _ll_register_callbacks cb 0x560b87aa5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.121+0000 7fb2c37fe640 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5c90 0x560b87bcd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2c3fff640 1 -- 172.21.15.5:0/3974998004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5c90 msgr2=0x560b87bcd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2c3fff640 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5c90 0x560b87bcd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2c3fff640 1 -- 172.21.15.5:0/3974998004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bd0550 msgr2=0x560b87bcdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2c3fff640 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bd0550 0x560b87bcdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2c3fff640 1 -- 172.21.15.5:0/3974998004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b878936a0 con 0x560b87bc5370 2024-09-19T06:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2c3fff640 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5370 0x560b87bcd420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4000c00 tx=0x7fb2b4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0aa2e4606e1e43a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2a37fe640 1 -- 172.21.15.5:0/3974998004 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4035040 con 0x560b87bc5370 2024-09-19T06:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2a37fe640 1 -- 172.21.15.5:0/3974998004 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fb2b4031970 con 0x560b87bc5370 2024-09-19T06:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2ccc59740 1 -- 172.21.15.5:0/3974998004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b8015900 con 0x560b87bc5370 2024-09-19T06:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2a37fe640 1 -- 172.21.15.5:0/3974998004 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb2b4031b70 con 0x560b87bc5370 2024-09-19T06:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2ccc59740 10 client.29021 fetch_fsmap learned FSMap version 50 2024-09-19T06:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.122+0000 7fb2ccc59740 1 -- 172.21.15.5:0/3974998004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x560b87bded90 con 0x560b87bc5370 2024-09-19T06:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.123+0000 7fb2a37fe640 1 -- 172.21.15.5:0/3974998004 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2b8015900 con 0x560b87bc5370 2024-09-19T06:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.123+0000 7fb2a37fe640 1 -- 172.21.15.5:0/3974998004 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 50) ==== 8394+0+0 (secure 0 0 0) 0x7fb2b4044030 con 0x560b87bc5370 2024-09-19T06:19:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.123+0000 7fb2ccc59740 10 client.29021 fetch_fsmap finished waiting for FSMap version 50 2024-09-19T06:19:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.123+0000 7fb2ccc59740 10 client.29021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.123+0000 7fb2ccc59740 20 client.29021 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.123+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b87be02a0 0x560b87be26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.123+0000 7fb2ccc59740 4 client.29021 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:19:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.123+0000 7fb2ccc59740 1 -- 172.21.15.5:0/3974998004 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b87b2f7f0 con 0x560b87be02a0 2024-09-19T06:19:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.123+0000 7fb2c37fe640 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b87be02a0 0x560b87be26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.124+0000 7fb2a37fe640 10 client.29021 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.124+0000 7fb2c37fe640 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b87be02a0 0x560b87be26d0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92acd690d7c49ae9 server_cookie=a32700121e51efbf in_seq=0 out_seq=0 2024-09-19T06:19:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2a37fe640 1 -- 172.21.15.5:0/3974998004 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b87b2f7f0 con 0x560b87be02a0 2024-09-19T06:19:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2a37fe640 10 client.29021 handle_command_reply: tid=0 2024-09-19T06:19:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2ccc59740 1 client.29021 shutdown 2024-09-19T06:19:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2ccc59740 1 -- 172.21.15.5:0/3974998004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5370 msgr2=0x560b87bcd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5370 0x560b87bcd420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4000c00 tx=0x7fb2b4009260 comp rx=0 tx=0).stop 2024-09-19T06:19:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2ccc59740 1 -- 172.21.15.5:0/3974998004 shutdown_connections 2024-09-19T06:19:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b87be02a0 0x560b87be26d0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bd0550 0x560b87bcdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5c90 0x560b87bcd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2a37fe640 0 client.29021 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/3974998004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5370 0x560b87bcd420 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2ccc59740 1 -- 172.21.15.5:0/3974998004 >> 172.21.15.5:0/3974998004 conn(0x560b87bc1900 msgr2=0x560b87bb8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2ccc59740 1 -- 172.21.15.5:0/3974998004 shutdown_connections 2024-09-19T06:19:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2ccc59740 1 -- 172.21.15.5:0/3974998004 wait complete. 2024-09-19T06:19:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.125+0000 7fb2ccc59740 20 client.29021 trim_cache size 0 max 16384 2024-09-19T06:19:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2ccc59740 1 Processor -- start 2024-09-19T06:19:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2ccc59740 1 -- start start 2024-09-19T06:19:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2ccc59740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5370 0x560b87bcc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2ccc59740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5c90 0x560b87be7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2ccc59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bd0550 0x560b87be8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2ccc59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b000e7e0 con 0x560b87bd0550 2024-09-19T06:19:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2ccc59740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b0010470 con 0x560b87bc5c90 2024-09-19T06:19:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2ccc59740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b000ef70 con 0x560b87bc5370 2024-09-19T06:19:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2c3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5370 0x560b87bcc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2c3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5370 0x560b87bcc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42686/0 (socket says 172.21.15.5:42686) 2024-09-19T06:19:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2c3fff640 1 -- 172.21.15.5:0/483990057 learned_addr learned my addr 172.21.15.5:0/483990057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2c8e1d640 1 --2- 172.21.15.5:0/483990057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bd0550 0x560b87be8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.152+0000 7fb2c37fe640 1 --2- 172.21.15.5:0/483990057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5c90 0x560b87be7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.153+0000 7fb2c3fff640 1 -- 172.21.15.5:0/483990057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5c90 msgr2=0x560b87be7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.153+0000 7fb2c3fff640 1 --2- 172.21.15.5:0/483990057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5c90 0x560b87be7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.153+0000 7fb2c3fff640 1 -- 172.21.15.5:0/483990057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bd0550 msgr2=0x560b87be8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.153+0000 7fb2c3fff640 1 --2- 172.21.15.5:0/483990057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bd0550 0x560b87be8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.153+0000 7fb2c3fff640 1 -- 172.21.15.5:0/483990057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b87bc5ac0 con 0x560b87bc5370 2024-09-19T06:19:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.153+0000 7fb2c3fff640 1 --2- 172.21.15.5:0/483990057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5370 0x560b87bcc380 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4042040 tx=0x7fb2b402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95be5b8331dd23a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:28.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.153+0000 7fb2a9ffb640 1 -- 172.21.15.5:0/483990057 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4031770 con 0x560b87bc5370 2024-09-19T06:19:28.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.153+0000 7fb2a9ffb640 1 -- 172.21.15.5:0/483990057 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2b405d020 con 0x560b87bc5370 2024-09-19T06:19:28.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.153+0000 7fb2a9ffb640 1 -- 172.21.15.5:0/483990057 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40517f0 con 0x560b87bc5370 2024-09-19T06:19:28.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.154+0000 7fb2ccc59740 1 -- 172.21.15.5:0/483990057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5370 msgr2=0x560b87bcc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:28.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.154+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/483990057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5370 0x560b87bcc380 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4042040 tx=0x7fb2b402a410 comp rx=0 tx=0).stop 2024-09-19T06:19:28.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.154+0000 7fb2ccc59740 1 -- 172.21.15.5:0/483990057 shutdown_connections 2024-09-19T06:19:28.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.154+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/483990057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bd0550 0x560b87be8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.154+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/483990057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bc5c90 0x560b87be7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.154+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/483990057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5370 0x560b87bcc380 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.154+0000 7fb2ccc59740 1 -- 172.21.15.5:0/483990057 >> 172.21.15.5:0/483990057 conn(0x560b87bc1900 msgr2=0x560b87bb8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.154+0000 7fb2ccc59740 1 -- 172.21.15.5:0/483990057 shutdown_connections 2024-09-19T06:19:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.154+0000 7fb2ccc59740 1 -- 172.21.15.5:0/483990057 wait complete. 2024-09-19T06:19:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.155+0000 7fb2ccc59740 1 -- start start 2024-09-19T06:19:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.155+0000 7fb2ccc59740 1 Processor -- start 2024-09-19T06:19:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.156+0000 7fb2ccc59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bc5370 0x560b87bd8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.156+0000 7fb2ccc59740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5c90 0x560b87bd8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.156+0000 7fb2ccc59740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bd0550 0x560b87bd5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.156+0000 7fb2ccc59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b000fda0 con 0x560b87bc5370 2024-09-19T06:19:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.156+0000 7fb2ccc59740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b000f640 con 0x560b87bd0550 2024-09-19T06:19:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.156+0000 7fb2ccc59740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b000c7e0 con 0x560b87bc5c90 2024-09-19T06:19:28.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.156+0000 7fb2c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bc5370 0x560b87bd8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.156+0000 7fb2ccc59740 10 client.0 _ll_register_callbacks cb 0x7fb2b4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:28.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.156+0000 7fb2c8e1d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bd0550 0x560b87bd5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.156+0000 7fb2c8e1d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bd0550 0x560b87bd5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59914/0 (socket says 172.21.15.5:59914) 2024-09-19T06:19:28.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.156+0000 7fb2c8e1d640 1 -- 172.21.15.5:0/2583577824 learned_addr learned my addr 172.21.15.5:0/2583577824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:28.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.157+0000 7fb2c8e1d640 1 -- 172.21.15.5:0/2583577824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5c90 msgr2=0x560b87bd8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:28.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.157+0000 7fb2c8e1d640 1 --2- 172.21.15.5:0/2583577824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5c90 0x560b87bd8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.157+0000 7fb2c8e1d640 1 -- 172.21.15.5:0/2583577824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bc5370 msgr2=0x560b87bd8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:28.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.157+0000 7fb2c8e1d640 1 --2- 172.21.15.5:0/2583577824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bc5370 0x560b87bd8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.157+0000 7fb2c8e1d640 1 -- 172.21.15.5:0/2583577824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b87bd5d30 con 0x560b87bd0550 2024-09-19T06:19:28.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.157+0000 7fb2c8e1d640 1 --2- 172.21.15.5:0/2583577824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bd0550 0x560b87bd5550 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc098a40 tx=0x7fb2bc09c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4351905b327396f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:28.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.157+0000 7fb2a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:19:28.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.158+0000 7fb2a37fe640 1 -- 172.21.15.5:0/2583577824 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0cded0 con 0x560b87bd0550 2024-09-19T06:19:28.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.158+0000 7fb2a37fe640 1 -- 172.21.15.5:0/2583577824 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fb2bc0a4dc0 con 0x560b87bd0550 2024-09-19T06:19:28.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.158+0000 7fb2ccc59740 1 -- 172.21.15.5:0/2583577824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b405d020 con 0x560b87bd0550 2024-09-19T06:19:28.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.158+0000 7fb2a37fe640 1 -- 172.21.15.5:0/2583577824 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb2bc0a5250 con 0x560b87bd0550 2024-09-19T06:19:28.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.158+0000 7fb2ccc59740 10 client.29027 fetch_fsmap learned FSMap version 50 2024-09-19T06:19:28.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.158+0000 7fb2ccc59740 1 -- 172.21.15.5:0/2583577824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x560b87c4bb40 con 0x560b87bd0550 2024-09-19T06:19:28.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.159+0000 7fb2a37fe640 1 -- 172.21.15.5:0/2583577824 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2b405d020 con 0x560b87bd0550 2024-09-19T06:19:28.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.159+0000 7fb2a37fe640 1 -- 172.21.15.5:0/2583577824 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 50) ==== 8394+0+0 (secure 0 0 0) 0x7fb2bc0b5430 con 0x560b87bd0550 2024-09-19T06:19:28.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.159+0000 7fb2ccc59740 10 client.29027 fetch_fsmap finished waiting for FSMap version 50 2024-09-19T06:19:28.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.159+0000 7fb2ccc59740 10 client.29027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:28.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.159+0000 7fb2ccc59740 20 client.29027 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:28.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.159+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/2583577824 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b87c4c2a0 0x560b87c4e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:28.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.159+0000 7fb2ccc59740 4 client.29027 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:19:28.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.159+0000 7fb2ccc59740 1 -- 172.21.15.5:0/2583577824 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560b87b29d00 con 0x560b87c4c2a0 2024-09-19T06:19:28.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.159+0000 7fb2c3fff640 1 --2- 172.21.15.5:0/2583577824 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b87c4c2a0 0x560b87c4e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:28.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.160+0000 7fb2a37fe640 10 client.29027 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:28.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.160+0000 7fb2c3fff640 1 --2- 172.21.15.5:0/2583577824 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b87c4c2a0 0x560b87c4e6c0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=609c205951b09011 server_cookie=edffdc06b990b3a4 in_seq=0 out_seq=0 2024-09-19T06:19:28.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.161+0000 7fb2a37fe640 1 -- 172.21.15.5:0/2583577824 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560b87b29d00 con 0x560b87c4c2a0 2024-09-19T06:19:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.161+0000 7fb2a37fe640 10 client.29027 handle_command_reply: tid=0 2024-09-19T06:19:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.161+0000 7fb2ccc59740 1 client.29027 shutdown 2024-09-19T06:19:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.161+0000 7fb2ccc59740 1 -- 172.21.15.5:0/2583577824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bd0550 msgr2=0x560b87bd5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.161+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/2583577824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bd0550 0x560b87bd5550 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc098a40 tx=0x7fb2bc09c6a0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.161+0000 7fb2ccc59740 1 -- 172.21.15.5:0/2583577824 shutdown_connections 2024-09-19T06:19:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.161+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/2583577824 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560b87c4c2a0 0x560b87c4e6c0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.161+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/2583577824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560b87bd0550 0x560b87bd5550 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.161+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/2583577824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b87bc5c90 0x560b87bd8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.161+0000 7fb2ccc59740 1 --2- 172.21.15.5:0/2583577824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b87bc5370 0x560b87bd8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.161+0000 7fb2ccc59740 1 -- 172.21.15.5:0/2583577824 >> 172.21.15.5:0/2583577824 conn(0x560b87bc1900 msgr2=0x560b87bc6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.161+0000 7fb2a37fe640 0 client.29027 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.162+0000 7fb2ccc59740 1 -- 172.21.15.5:0/2583577824 shutdown_connections 2024-09-19T06:19:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.162+0000 7fb2ccc59740 1 -- 172.21.15.5:0/2583577824 wait complete. 2024-09-19T06:19:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.162+0000 7fb2ccc59740 20 client.29027 trim_cache size 0 max 16384 2024-09-19T06:19:28.176 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:19:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:19:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "29402121-cbd7-496c-be52-89dff12429e9", 2024-09-19T06:19:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:19:28.177 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:19:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.163+0000 7fb2ca880640 1 -- 172.21.15.5:0/2811632783 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb2ac0a3430 msgr2=0x7fb2ac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.163+0000 7fb2ca880640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb2ac0a3430 0x7fb2ac0a58f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4004650 tx=0x7fb2b4005a20 comp rx=0 tx=0).stop 2024-09-19T06:19:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.163+0000 7fb2ca880640 1 -- 172.21.15.5:0/2811632783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c4106b80 msgr2=0x7fb2c419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.163+0000 7fb2ca880640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c4106b80 0x7fb2c419cd60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005fd0 tx=0x7fb2bc00ef10 comp rx=0 tx=0).stop 2024-09-19T06:19:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.163+0000 7fb2ca880640 1 -- 172.21.15.5:0/2811632783 shutdown_connections 2024-09-19T06:19:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.163+0000 7fb2ca880640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb2ac0a3430 0x7fb2ac0a58f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.163+0000 7fb2ca880640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c4106b80 0x7fb2c419cd60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.163+0000 7fb2ca880640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2c4100aa0 0x7fb2c41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.163+0000 7fb2ca880640 1 --2- 172.21.15.5:0/2811632783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4100100 0x7fb2c4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.163+0000 7fb2ca880640 1 -- 172.21.15.5:0/2811632783 >> 172.21.15.5:0/2811632783 conn(0x7fb2c4077230 msgr2=0x7fb2c4104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.163+0000 7fb2ca880640 1 -- 172.21.15.5:0/2811632783 shutdown_connections 2024-09-19T06:19:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:28.163+0000 7fb2ca880640 1 -- 172.21.15.5:0/2811632783 wait complete. 2024-09-19T06:19:28.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28980 8.72 MB/sec execute 52 sec latency 88.114 ms 2024-09-19T06:19:28.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:19:28.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:19:28.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:28.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:19:28.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:19:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:19:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:19:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:19:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:19:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:19:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 29124 8.68 MB/sec execute 53 sec latency 115.668 ms 2024-09-19T06:19:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:29 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:19:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:29 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:29 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:19:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:29 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:19:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:29 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:19:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:29 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:29 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:19:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:29 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:29 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:29 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:29 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:29 smithi005 ceph-mon[30955]: pgmap v508: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-09-19T06:19:29.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:29 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:19:29.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:29 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:29 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:19:29.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:29 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:19:29.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:29 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:19:29.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:29 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:29 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:19:29.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:29 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:29 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:29 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:29 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:29.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:29 smithi049 ceph-mon[34640]: pgmap v508: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-09-19T06:19:30.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:29 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:19:30.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:29 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:30.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:29 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:19:30.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:29 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:19:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:29 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:19:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:29 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:29 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:19:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:29 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:29 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:29 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:29 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:19:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:29 smithi157 ceph-mon[35346]: pgmap v508: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-09-19T06:19:30.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 29301 8.85 MB/sec execute 54 sec latency 132.958 ms 2024-09-19T06:19:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:30 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:30 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:31.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:30 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:31.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 29466 8.84 MB/sec execute 55 sec latency 128.149 ms 2024-09-19T06:19:31.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:31 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:31.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:31 smithi005 ceph-mon[30955]: pgmap v509: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-09-19T06:19:31.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:31 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:31.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:31 smithi049 ceph-mon[34640]: pgmap v509: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-09-19T06:19:32.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:31 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:32.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:31 smithi157 ceph-mon[35346]: pgmap v509: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-09-19T06:19:32.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 29663 8.84 MB/sec execute 56 sec latency 57.403 ms 2024-09-19T06:19:33.048 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:19:33.425 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:19:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.508+0000 7f9beca84640 1 -- 172.21.15.5:0/3522762772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be81035a0 msgr2=0x7f9be810ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.508+0000 7f9beca84640 1 --2- 172.21.15.5:0/3522762772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be81035a0 0x7f9be810ba80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009900 tx=0x7f9bdc02a480 comp rx=0 tx=0).stop 2024-09-19T06:19:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.509+0000 7f9beca84640 1 -- 172.21.15.5:0/3522762772 shutdown_connections 2024-09-19T06:19:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.509+0000 7f9beca84640 1 --2- 172.21.15.5:0/3522762772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be810c0a0 0x7f9be810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.509+0000 7f9beca84640 1 --2- 172.21.15.5:0/3522762772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be81035a0 0x7f9be810ba80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.509+0000 7f9beca84640 1 --2- 172.21.15.5:0/3522762772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be8102c80 0x7f9be8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.509+0000 7f9beca84640 1 -- 172.21.15.5:0/3522762772 >> 172.21.15.5:0/3522762772 conn(0x7f9be80779a0 msgr2=0x7f9be80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.509+0000 7f9beca84640 1 -- 172.21.15.5:0/3522762772 shutdown_connections 2024-09-19T06:19:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.509+0000 7f9beca84640 1 -- 172.21.15.5:0/3522762772 wait complete. 2024-09-19T06:19:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9beca84640 1 Processor -- start 2024-09-19T06:19:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9beca84640 1 -- start start 2024-09-19T06:19:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9beca84640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be8102c80 0x7f9be8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9beca84640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be81035a0 0x7f9be81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9beca84640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be810c0a0 0x7f9be819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9beca84640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9be8111da0 con 0x7f9be810c0a0 2024-09-19T06:19:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9beca84640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9be8111c20 con 0x7f9be8102c80 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9beca84640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9be8111f20 con 0x7f9be81035a0 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9be659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be8102c80 0x7f9be8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9be5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be81035a0 0x7f9be81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9be5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be81035a0 0x7f9be81988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42732/0 (socket says 172.21.15.5:42732) 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9be5d9b640 1 -- 172.21.15.5:0/1761143702 learned_addr learned my addr 172.21.15.5:0/1761143702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9be659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be8102c80 0x7f9be8198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59930/0 (socket says 172.21.15.5:59930) 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.510+0000 7f9be6d9d640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be810c0a0 0x7f9be819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.511+0000 7f9be659c640 1 -- 172.21.15.5:0/1761143702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be81035a0 msgr2=0x7f9be81988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.511+0000 7f9be659c640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be81035a0 0x7f9be81988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.511+0000 7f9be659c640 1 -- 172.21.15.5:0/1761143702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be810c0a0 msgr2=0x7f9be819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.511+0000 7f9be659c640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be810c0a0 0x7f9be819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.511+0000 7f9be659c640 1 -- 172.21.15.5:0/1761143702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be819d360 con 0x7f9be8102c80 2024-09-19T06:19:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.511+0000 7f9be5d9b640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be81035a0 0x7f9be81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.511+0000 7f9be659c640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be8102c80 0x7f9be8198370 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800e9f0 tx=0x7f9bd800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.511+0000 7f9bd77fe640 1 -- 172.21.15.5:0/1761143702 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800ccc0 con 0x7f9be8102c80 2024-09-19T06:19:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.511+0000 7f9bd77fe640 1 -- 172.21.15.5:0/1761143702 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bd80035f0 con 0x7f9be8102c80 2024-09-19T06:19:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.511+0000 7f9bd77fe640 1 -- 172.21.15.5:0/1761143702 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800ce60 con 0x7f9be8102c80 2024-09-19T06:19:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.511+0000 7f9beca84640 1 -- 172.21.15.5:0/1761143702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be819d650 con 0x7f9be8102c80 2024-09-19T06:19:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.511+0000 7f9beca84640 1 -- 172.21.15.5:0/1761143702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be81a4ee0 con 0x7f9be8102c80 2024-09-19T06:19:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.514+0000 7f9bd77fe640 1 -- 172.21.15.5:0/1761143702 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9bd80037a0 con 0x7f9be8102c80 2024-09-19T06:19:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.515+0000 7f9bef81d740 1 Processor -- start 2024-09-19T06:19:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.515+0000 7f9bef81d740 1 -- start start 2024-09-19T06:19:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.515+0000 7f9bef81d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 0x5649b6c3c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.515+0000 7f9bef81d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3c820 0x5649b6c46b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.515+0000 7f9bef81d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.515+0000 7f9bef81d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5649b6b9c2f0 con 0x5649b6c3bf00 2024-09-19T06:19:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.515+0000 7f9bef81d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5649b6b44270 con 0x5649b6c470c0 2024-09-19T06:19:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.515+0000 7f9bef81d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5649b6b982b0 con 0x5649b6c3c820 2024-09-19T06:19:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9bd77fe640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9bd00a3430 0x7f9bd00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be5d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be5d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59936/0 (socket says 172.21.15.5:59936) 2024-09-19T06:19:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be5d9b640 1 -- 172.21.15.5:0/1640197320 learned_addr learned my addr 172.21.15.5:0/1640197320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 0x5649b6c3c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3c820 0x5649b6c46b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be6d9d640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9bd00a3430 0x7f9bd00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9bd77fe640 1 -- 172.21.15.5:0/1761143702 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f9bd80cfea0 con 0x7f9be8102c80 2024-09-19T06:19:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be6d9d640 1 -- 172.21.15.5:0/1640197320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3c820 msgr2=0x5649b6c46b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be6d9d640 1 --2- 172.21.15.5:0/1640197320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3c820 0x5649b6c46b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be6d9d640 1 -- 172.21.15.5:0/1640197320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 msgr2=0x5649b6c494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be6d9d640 1 --2- 172.21.15.5:0/1640197320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be6d9d640 1 -- 172.21.15.5:0/1640197320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649b6b43340 con 0x5649b6c3bf00 2024-09-19T06:19:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be5d9b640 1 --2- 172.21.15.5:0/1640197320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.516+0000 7f9be6d9d640 1 --2- 172.21.15.5:0/1640197320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 0x5649b6c3c2e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f9be0011cd0 tx=0x7f9be000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc350a778bb401e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.517+0000 7f9be6d9d640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9bd00a3430 0x7f9bd00a58f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9be000fc90 tx=0x7f9be001e000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.517+0000 7f9bd4ff9640 1 -- 172.21.15.5:0/1640197320 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be001dbf0 con 0x5649b6c3bf00 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.517+0000 7f9bd4ff9640 1 -- 172.21.15.5:0/1640197320 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9be0009cf0 con 0x5649b6c3bf00 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.517+0000 7f9bd4ff9640 1 -- 172.21.15.5:0/1640197320 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0005620 con 0x5649b6c3bf00 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.518+0000 7f9bef81d740 1 -- 172.21.15.5:0/1640197320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 msgr2=0x5649b6c3c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.518+0000 7f9bef81d740 1 --2- 172.21.15.5:0/1640197320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 0x5649b6c3c2e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f9be0011cd0 tx=0x7f9be000f790 comp rx=0 tx=0).stop 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.518+0000 7f9bef81d740 1 -- 172.21.15.5:0/1640197320 shutdown_connections 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.518+0000 7f9bef81d740 1 --2- 172.21.15.5:0/1640197320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.518+0000 7f9bef81d740 1 --2- 172.21.15.5:0/1640197320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3c820 0x5649b6c46b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.518+0000 7f9bef81d740 1 --2- 172.21.15.5:0/1640197320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 0x5649b6c3c2e0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.518+0000 7f9bef81d740 1 -- 172.21.15.5:0/1640197320 >> 172.21.15.5:0/1640197320 conn(0x5649b6c38470 msgr2=0x5649b6c38850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.518+0000 7f9bef81d740 1 -- 172.21.15.5:0/1640197320 shutdown_connections 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.518+0000 7f9bef81d740 1 -- 172.21.15.5:0/1640197320 wait complete. 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.519+0000 7f9bef81d740 1 -- start start 2024-09-19T06:19:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.519+0000 7f9bef81d740 1 Processor -- start 2024-09-19T06:19:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.519+0000 7f9bef81d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 0x5649b6c43fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.519+0000 7f9bef81d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c3c820 0x5649b6c44500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.519+0000 7f9bef81d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c470c0 0x5649b6c44a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.519+0000 7f9bef81d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5649b6c49c80 con 0x5649b6c470c0 2024-09-19T06:19:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.519+0000 7f9bef81d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5649b6b25290 con 0x5649b6c3c820 2024-09-19T06:19:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.519+0000 7f9bef81d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5649b6c49e00 con 0x5649b6c3bf00 2024-09-19T06:19:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c470c0 0x5649b6c44a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c470c0 0x5649b6c44a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33578/0 (socket says 172.21.15.5:33578) 2024-09-19T06:19:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be6d9d640 1 -- 172.21.15.5:0/4146403043 learned_addr learned my addr 172.21.15.5:0/4146403043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be5d9b640 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c3c820 0x5649b6c44500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be659c640 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 0x5649b6c43fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9bef81d740 10 client.0 _ll_register_callbacks cb 0x5649b6b56140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be659c640 1 -- 172.21.15.5:0/4146403043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c3c820 msgr2=0x5649b6c44500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be659c640 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c3c820 0x5649b6c44500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be659c640 1 -- 172.21.15.5:0/4146403043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c470c0 msgr2=0x5649b6c44a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be659c640 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c470c0 0x5649b6c44a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be659c640 1 -- 172.21.15.5:0/4146403043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649b68af210 con 0x5649b6c3bf00 2024-09-19T06:19:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be6d9d640 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c470c0 0x5649b6c44a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be5d9b640 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c3c820 0x5649b6c44500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9bef81d740 1 -- 172.21.15.5:0/4146403043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9be0009cf0 con 0x5649b6c3bf00 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9be659c640 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 0x5649b6c43fc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80cd5a0 tx=0x7f9bd8009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c68989efc7d36d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.520+0000 7f9bcdffb640 10 client.28981 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.521+0000 7f9bcdffb640 1 -- 172.21.15.5:0/4146403043 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80a0510 con 0x5649b6c3bf00 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.521+0000 7f9bcdffb640 1 -- 172.21.15.5:0/4146403043 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f9bd80bc0c0 con 0x5649b6c3bf00 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.521+0000 7f9bcdffb640 1 -- 172.21.15.5:0/4146403043 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9bd80a4dd0 con 0x5649b6c3bf00 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.521+0000 7f9bef81d740 10 client.28981 fetch_fsmap learned FSMap version 52 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.521+0000 7f9bef81d740 1 -- 172.21.15.5:0/4146403043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=52}) -- 0x5649b697b620 con 0x5649b6c3bf00 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.521+0000 7f9bcdffb640 1 -- 172.21.15.5:0/4146403043 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 52) ==== 8386+0+0 (secure 0 0 0) 0x7f9bd80a0170 con 0x5649b6c3bf00 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.521+0000 7f9bef81d740 10 client.28981 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.521+0000 7f9bef81d740 10 client.28981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.522+0000 7f9bef81d740 20 client.28981 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.522+0000 7f9bef81d740 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5649b6c56dc0 0x5649b6c59280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.522+0000 7f9bef81d740 4 client.28981 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:19:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.522+0000 7f9bef81d740 1 -- 172.21.15.5:0/4146403043 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649b6ba9a20 con 0x5649b6c56dc0 2024-09-19T06:19:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.522+0000 7f9be5d9b640 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5649b6c56dc0 0x5649b6c59280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.522+0000 7f9be5d9b640 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5649b6c56dc0 0x5649b6c59280 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da96caf1d973f0a9 server_cookie=66230062ef5276a1 in_seq=0 out_seq=0 2024-09-19T06:19:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.522+0000 7f9bcdffb640 10 client.28981 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bcdffb640 1 -- 172.21.15.5:0/4146403043 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649b6ba9a20 con 0x5649b6c56dc0 2024-09-19T06:19:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bcdffb640 10 client.28981 handle_command_reply: tid=0 2024-09-19T06:19:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bef81d740 1 client.28981 shutdown 2024-09-19T06:19:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bef81d740 1 -- 172.21.15.5:0/4146403043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 msgr2=0x5649b6c43fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bef81d740 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 0x5649b6c43fc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80cd5a0 tx=0x7f9bd8009460 comp rx=0 tx=0).stop 2024-09-19T06:19:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bef81d740 1 -- 172.21.15.5:0/4146403043 shutdown_connections 2024-09-19T06:19:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bef81d740 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5649b6c56dc0 0x5649b6c59280 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bef81d740 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c470c0 0x5649b6c44a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bef81d740 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c3c820 0x5649b6c44500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bef81d740 1 --2- 172.21.15.5:0/4146403043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 0x5649b6c43fc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bef81d740 1 -- 172.21.15.5:0/4146403043 >> 172.21.15.5:0/4146403043 conn(0x5649b6c38470 msgr2=0x5649b6c331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bcdffb640 0 client.28981 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.523+0000 7f9bef81d740 1 -- 172.21.15.5:0/4146403043 shutdown_connections 2024-09-19T06:19:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.524+0000 7f9bef81d740 1 -- 172.21.15.5:0/4146403043 wait complete. 2024-09-19T06:19:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.524+0000 7f9bef81d740 20 client.28981 trim_cache size 0 max 16384 2024-09-19T06:19:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9bef81d740 1 Processor -- start 2024-09-19T06:19:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9bef81d740 1 -- start start 2024-09-19T06:19:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9bef81d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 0x5649b6c45f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9bef81d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3c820 0x5649b6c5ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9bef81d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c5f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9bef81d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81ab440 con 0x5649b6c3c820 2024-09-19T06:19:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9bef81d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81ad0a0 con 0x5649b6c470c0 2024-09-19T06:19:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9bef81d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81abbd0 con 0x5649b6c3bf00 2024-09-19T06:19:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9be5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3c820 0x5649b6c5ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9be659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 0x5649b6c45f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9be659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 0x5649b6c45f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42762/0 (socket says 172.21.15.5:42762) 2024-09-19T06:19:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9be659c640 1 -- 172.21.15.5:0/839986504 learned_addr learned my addr 172.21.15.5:0/839986504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9be5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3c820 0x5649b6c5ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33594/0 (socket says 172.21.15.5:33594) 2024-09-19T06:19:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.544+0000 7f9be6d9d640 1 --2- 172.21.15.5:0/839986504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c5f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.545+0000 7f9be659c640 1 -- 172.21.15.5:0/839986504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 msgr2=0x5649b6c5f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.545+0000 7f9be659c640 1 --2- 172.21.15.5:0/839986504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c5f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.545+0000 7f9be659c640 1 -- 172.21.15.5:0/839986504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3c820 msgr2=0x5649b6c5ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.545+0000 7f9be659c640 1 --2- 172.21.15.5:0/839986504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3c820 0x5649b6c5ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.545+0000 7f9be659c640 1 -- 172.21.15.5:0/839986504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649b6c539b0 con 0x5649b6c3bf00 2024-09-19T06:19:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.545+0000 7f9be5d9b640 1 --2- 172.21.15.5:0/839986504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3c820 0x5649b6c5ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.545+0000 7f9be659c640 1 --2- 172.21.15.5:0/839986504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 0x5649b6c45f30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009820 tx=0x7f9bd80bc050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b6b28dcc2e44b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.545+0000 7f9bcffff640 1 -- 172.21.15.5:0/839986504 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80a0510 con 0x5649b6c3bf00 2024-09-19T06:19:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.545+0000 7f9bcffff640 1 -- 172.21.15.5:0/839986504 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9be0009cf0 con 0x5649b6c3bf00 2024-09-19T06:19:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.545+0000 7f9bcffff640 1 -- 172.21.15.5:0/839986504 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80a4b80 con 0x5649b6c3bf00 2024-09-19T06:19:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.546+0000 7f9bef81d740 1 -- 172.21.15.5:0/839986504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 msgr2=0x5649b6c45f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.546+0000 7f9bef81d740 1 --2- 172.21.15.5:0/839986504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 0x5649b6c45f30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009820 tx=0x7f9bd80bc050 comp rx=0 tx=0).stop 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.546+0000 7f9bef81d740 1 -- 172.21.15.5:0/839986504 shutdown_connections 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.546+0000 7f9bef81d740 1 --2- 172.21.15.5:0/839986504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c5f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.546+0000 7f9bef81d740 1 --2- 172.21.15.5:0/839986504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3c820 0x5649b6c5ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.546+0000 7f9bef81d740 1 --2- 172.21.15.5:0/839986504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3bf00 0x5649b6c45f30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.546+0000 7f9bef81d740 1 -- 172.21.15.5:0/839986504 >> 172.21.15.5:0/839986504 conn(0x5649b6c38470 msgr2=0x5649b6c45820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.546+0000 7f9bef81d740 1 -- 172.21.15.5:0/839986504 shutdown_connections 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.546+0000 7f9bef81d740 1 -- 172.21.15.5:0/839986504 wait complete. 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9bef81d740 1 -- start start 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9bef81d740 1 Processor -- start 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9bef81d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 0x5649b6c4f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9bef81d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3c820 0x5649b6c4f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9bef81d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c40a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9bef81d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81ac9d0 con 0x5649b6c3bf00 2024-09-19T06:19:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9bef81d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81ac2a0 con 0x5649b6c470c0 2024-09-19T06:19:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9bef81d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81a9440 con 0x5649b6c3c820 2024-09-19T06:19:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9be659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 0x5649b6c4f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9be659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 0x5649b6c4f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33598/0 (socket says 172.21.15.5:33598) 2024-09-19T06:19:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9be5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3c820 0x5649b6c4f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9be659c640 1 -- 172.21.15.5:0/3040675910 learned_addr learned my addr 172.21.15.5:0/3040675910 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9bef81d740 10 client.0 _ll_register_callbacks cb 0x7f9bd800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.547+0000 7f9be6d9d640 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c40a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9be659c640 1 -- 172.21.15.5:0/3040675910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3c820 msgr2=0x5649b6c4f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9be659c640 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3c820 0x5649b6c4f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9be659c640 1 -- 172.21.15.5:0/3040675910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 msgr2=0x5649b6c40a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9be659c640 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c40a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9be659c640 1 -- 172.21.15.5:0/3040675910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649b6c41210 con 0x5649b6c3bf00 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9be6d9d640 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c40a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9be659c640 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 0x5649b6c4f380 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009fd0 tx=0x7f9bd80a4b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04cf88cd972c1ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9bcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9bcdffb640 1 -- 172.21.15.5:0/3040675910 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80a0510 con 0x5649b6c3bf00 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9bcdffb640 1 -- 172.21.15.5:0/3040675910 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f9bd8097eb0 con 0x5649b6c3bf00 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9bef81d740 1 -- 172.21.15.5:0/3040675910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9be0009cf0 con 0x5649b6c3bf00 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9bcdffb640 1 -- 172.21.15.5:0/3040675910 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9bd80bc0c0 con 0x5649b6c3bf00 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9bef81d740 10 client.19668 fetch_fsmap learned FSMap version 52 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.548+0000 7f9bef81d740 1 -- 172.21.15.5:0/3040675910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5649b6cc1780 con 0x5649b6c3bf00 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.549+0000 7f9bcdffb640 1 -- 172.21.15.5:0/3040675910 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bd8098130 con 0x5649b6c3bf00 2024-09-19T06:19:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.549+0000 7f9bcdffb640 1 -- 172.21.15.5:0/3040675910 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) ==== 8386+0+0 (secure 0 0 0) 0x7f9bd80a1880 con 0x5649b6c3bf00 2024-09-19T06:19:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.549+0000 7f9bef81d740 10 client.19668 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T06:19:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.549+0000 7f9bef81d740 10 client.19668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.549+0000 7f9bef81d740 20 client.19668 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.549+0000 7f9bef81d740 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5649b6cc1c60 0x5649b6cc4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.549+0000 7f9bef81d740 4 client.19668 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:19:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.549+0000 7f9bef81d740 1 -- 172.21.15.5:0/3040675910 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9be0009cf0 con 0x5649b6cc1c60 2024-09-19T06:19:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.549+0000 7f9be5d9b640 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5649b6cc1c60 0x5649b6cc4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.550+0000 7f9be5d9b640 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5649b6cc1c60 0x5649b6cc4050 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c1b2e61089976b5 server_cookie=7daad8494011eae5 in_seq=0 out_seq=0 2024-09-19T06:19:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.550+0000 7f9bcdffb640 10 client.19668 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:33.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 29800 8.78 MB/sec execute 57 sec latency 80.654 ms 2024-09-19T06:19:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bcdffb640 1 -- 172.21.15.5:0/3040675910 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f9be0009cf0 con 0x5649b6cc1c60 2024-09-19T06:19:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bcdffb640 10 client.19668 handle_command_reply: tid=0 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bef81d740 1 client.19668 shutdown 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bef81d740 1 -- 172.21.15.5:0/3040675910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 msgr2=0x5649b6c4f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bef81d740 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 0x5649b6c4f380 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009fd0 tx=0x7f9bd80a4b70 comp rx=0 tx=0).stop 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bef81d740 1 -- 172.21.15.5:0/3040675910 shutdown_connections 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bef81d740 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5649b6cc1c60 0x5649b6cc4050 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bef81d740 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649b6c470c0 0x5649b6c40a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bef81d740 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649b6c3c820 0x5649b6c4f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bef81d740 1 --2- 172.21.15.5:0/3040675910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649b6c3bf00 0x5649b6c4f380 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bef81d740 1 -- 172.21.15.5:0/3040675910 >> 172.21.15.5:0/3040675910 conn(0x5649b6c38470 msgr2=0x5649b6c3e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bcdffb640 0 client.19668 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bef81d740 1 -- 172.21.15.5:0/3040675910 shutdown_connections 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.819+0000 7f9bef81d740 1 -- 172.21.15.5:0/3040675910 wait complete. 2024-09-19T06:19:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9bef81d740 20 client.19668 trim_cache size 0 max 16384 2024-09-19T06:19:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:19:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:19:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:19:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:19:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9beca84640 1 -- 172.21.15.5:0/1761143702 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9bd00a3430 msgr2=0x7f9bd00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9beca84640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9bd00a3430 0x7f9bd00a58f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9be000fc90 tx=0x7f9be001e000 comp rx=0 tx=0).stop 2024-09-19T06:19:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9beca84640 1 -- 172.21.15.5:0/1761143702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be8102c80 msgr2=0x7f9be8198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9beca84640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be8102c80 0x7f9be8198370 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800e9f0 tx=0x7f9bd800edf0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9beca84640 1 -- 172.21.15.5:0/1761143702 shutdown_connections 2024-09-19T06:19:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9beca84640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be810c0a0 0x7f9be819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9beca84640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9bd00a3430 0x7f9bd00a58f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9beca84640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be81035a0 0x7f9be81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9beca84640 1 --2- 172.21.15.5:0/1761143702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be8102c80 0x7f9be8198370 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9beca84640 1 -- 172.21.15.5:0/1761143702 >> 172.21.15.5:0/1761143702 conn(0x7f9be80779a0 msgr2=0x7f9be80fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9beca84640 1 -- 172.21.15.5:0/1761143702 shutdown_connections 2024-09-19T06:19:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:33.820+0000 7f9beca84640 1 -- 172.21.15.5:0/1761143702 wait complete. 2024-09-19T06:19:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:33 smithi157 ceph-mon[35346]: pgmap v510: 97 pgs: 97 active+clean; 625 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 449 op/s 2024-09-19T06:19:34.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:33 smithi005 ceph-mon[30955]: pgmap v510: 97 pgs: 97 active+clean; 625 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 449 op/s 2024-09-19T06:19:34.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:33 smithi049 ceph-mon[34640]: pgmap v510: 97 pgs: 97 active+clean; 625 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 449 op/s 2024-09-19T06:19:34.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 29936 8.76 MB/sec execute 58 sec latency 175.208 ms 2024-09-19T06:19:35.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 30111 8.80 MB/sec execute 59 sec latency 123.102 ms 2024-09-19T06:19:36.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:35 smithi005 ceph-mon[30955]: pgmap v511: 97 pgs: 97 active+clean; 625 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2024-09-19T06:19:36.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:35 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:36.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:35 smithi157 ceph-mon[35346]: pgmap v511: 97 pgs: 97 active+clean; 625 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2024-09-19T06:19:36.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:35 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:36.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:35 smithi049 ceph-mon[34640]: pgmap v511: 97 pgs: 97 active+clean; 625 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2024-09-19T06:19:36.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:35 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:36.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 30283 8.89 MB/sec execute 60 sec latency 110.211 ms 2024-09-19T06:19:37.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 30380 8.80 MB/sec execute 61 sec latency 141.609 ms 2024-09-19T06:19:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.778+0000 7fedf91e6640 1 -- 172.21.15.5:0/2937560534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf4101440 msgr2=0x7fedf410df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.778+0000 7fedf91e6640 1 --2- 172.21.15.5:0/2937560534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf4101440 0x7fedf410df30 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fede8009ca0 tx=0x7fede802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.778+0000 7fedf91e6640 1 -- 172.21.15.5:0/2937560534 shutdown_connections 2024-09-19T06:19:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.778+0000 7fedf91e6640 1 --2- 172.21.15.5:0/2937560534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedf410e500 0x7fedf4110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.778+0000 7fedf91e6640 1 --2- 172.21.15.5:0/2937560534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf4101440 0x7fedf410df30 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.778+0000 7fedf91e6640 1 --2- 172.21.15.5:0/2937560534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedf4100b20 0x7fedf4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.778+0000 7fedf91e6640 1 -- 172.21.15.5:0/2937560534 >> 172.21.15.5:0/2937560534 conn(0x7fedf40fc7d0 msgr2=0x7fedf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.778+0000 7fedf91e6640 1 -- 172.21.15.5:0/2937560534 shutdown_connections 2024-09-19T06:19:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.778+0000 7fedf91e6640 1 -- 172.21.15.5:0/2937560534 wait complete. 2024-09-19T06:19:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf91e6640 1 Processor -- start 2024-09-19T06:19:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf91e6640 1 -- start start 2024-09-19T06:19:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf91e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf4100b20 0x7fedf419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf91e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedf4101440 0x7fedf419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf91e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedf410e500 0x7fedf419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf91e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedf4114180 con 0x7fedf4100b20 2024-09-19T06:19:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf91e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedf4114000 con 0x7fedf4101440 2024-09-19T06:19:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf91e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fedf4114300 con 0x7fedf410e500 2024-09-19T06:19:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf4100b20 0x7fedf419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf4100b20 0x7fedf419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33514/0 (socket says 172.21.15.5:33514) 2024-09-19T06:19:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf2d9d640 1 -- 172.21.15.5:0/915695675 learned_addr learned my addr 172.21.15.5:0/915695675 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf259c640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedf4101440 0x7fedf419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.779+0000 7fedf359e640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedf410e500 0x7fedf419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.780+0000 7fedf2d9d640 1 -- 172.21.15.5:0/915695675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedf410e500 msgr2=0x7fedf419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.780+0000 7fedf2d9d640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedf410e500 0x7fedf419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.780+0000 7fedf2d9d640 1 -- 172.21.15.5:0/915695675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedf4101440 msgr2=0x7fedf419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.780+0000 7fedf2d9d640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedf4101440 0x7fedf419ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.780+0000 7fedf2d9d640 1 -- 172.21.15.5:0/915695675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf419f6e0 con 0x7fedf4100b20 2024-09-19T06:19:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.780+0000 7fedf2d9d640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf4100b20 0x7fedf419a730 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fede400b4a0 tx=0x7fede400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.780+0000 7feddbfff640 1 -- 172.21.15.5:0/915695675 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4014070 con 0x7fedf4100b20 2024-09-19T06:19:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.780+0000 7fedf91e6640 1 -- 172.21.15.5:0/915695675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf419f9d0 con 0x7fedf4100b20 2024-09-19T06:19:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.780+0000 7feddbfff640 1 -- 172.21.15.5:0/915695675 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fede4010ac0 con 0x7fedf4100b20 2024-09-19T06:19:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.780+0000 7feddbfff640 1 -- 172.21.15.5:0/915695675 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4017c30 con 0x7fedf4100b20 2024-09-19T06:19:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.780+0000 7fedf91e6640 1 -- 172.21.15.5:0/915695675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf41a72b0 con 0x7fedf4100b20 2024-09-19T06:19:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.782+0000 7feddbfff640 1 -- 172.21.15.5:0/915695675 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fede4017070 con 0x7fedf4100b20 2024-09-19T06:19:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedfb5bf740 1 Processor -- start 2024-09-19T06:19:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedfb5bf740 1 -- start start 2024-09-19T06:19:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedfb5bf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedfb5bf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedfb5bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 0x55dfcb3c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedfb5bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dfcb31b2e0 con 0x55dfcb3c0680 2024-09-19T06:19:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedfb5bf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dfcb2c2ce0 con 0x55dfcb3b54a0 2024-09-19T06:19:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedfb5bf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dfcb3172a0 con 0x55dfcb3b5dc0 2024-09-19T06:19:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7feddbfff640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7feddc0a33b0 0x7feddc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedf259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 0x55dfcb3c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedf259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 0x55dfcb3c2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33522/0 (socket says 172.21.15.5:33522) 2024-09-19T06:19:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedf259c640 1 -- 172.21.15.5:0/2800779749 learned_addr learned my addr 172.21.15.5:0/2800779749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedf359e640 1 --2- 172.21.15.5:0/2800779749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7feddbfff640 1 -- 172.21.15.5:0/915695675 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fede40cf260 con 0x7fedf4100b20 2024-09-19T06:19:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedf2d9d640 1 --2- 172.21.15.5:0/2800779749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedf359e640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7feddc0a33b0 0x7feddc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedf259c640 1 -- 172.21.15.5:0/2800779749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 msgr2=0x55dfcb3c0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedf259c640 1 --2- 172.21.15.5:0/2800779749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3c0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedf259c640 1 -- 172.21.15.5:0/2800779749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 msgr2=0x55dfcb3b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedf259c640 1 --2- 172.21.15.5:0/2800779749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.783+0000 7fedf259c640 1 -- 172.21.15.5:0/2800779749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfcb2a7440 con 0x55dfcb3c0680 2024-09-19T06:19:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.784+0000 7fedf259c640 1 --2- 172.21.15.5:0/2800779749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 0x55dfcb3c2a70 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fedf406a140 tx=0x7fede8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d410384b198210b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.784+0000 7fedd97fa640 1 -- 172.21.15.5:0/2800779749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8031450 con 0x55dfcb3c0680 2024-09-19T06:19:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.784+0000 7fedf359e640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7feddc0a33b0 0x7feddc0a5870 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fedec005fd0 tx=0x7fedec005f60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.784+0000 7fedd97fa640 1 -- 172.21.15.5:0/2800779749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fede8036030 con 0x55dfcb3c0680 2024-09-19T06:19:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.784+0000 7fedd97fa640 1 -- 172.21.15.5:0/2800779749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede80045b0 con 0x55dfcb3c0680 2024-09-19T06:19:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.784+0000 7fedfb5bf740 1 -- 172.21.15.5:0/2800779749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 msgr2=0x55dfcb3c2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.784+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/2800779749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 0x55dfcb3c2a70 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fedf406a140 tx=0x7fede8005fa0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 -- 172.21.15.5:0/2800779749 shutdown_connections 2024-09-19T06:19:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/2800779749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 0x55dfcb3c2a70 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/2800779749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3c0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/2800779749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 -- 172.21.15.5:0/2800779749 >> 172.21.15.5:0/2800779749 conn(0x55dfcb3b1a30 msgr2=0x55dfcb3b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 -- 172.21.15.5:0/2800779749 shutdown_connections 2024-09-19T06:19:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 -- 172.21.15.5:0/2800779749 wait complete. 2024-09-19T06:19:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 -- start start 2024-09-19T06:19:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 Processor -- start 2024-09-19T06:19:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3bff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3bd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3c0680 0x55dfcb3bd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dfcb3c30d0 con 0x55dfcb3b5dc0 2024-09-19T06:19:37.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dfcb2d6050 con 0x55dfcb3b54a0 2024-09-19T06:19:37.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dfcb3c3250 con 0x55dfcb3c0680 2024-09-19T06:19:37.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedfb5bf740 10 client.0 _ll_register_callbacks cb 0x55dfcb316160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:37.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedf259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3bd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedf259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3bd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33534/0 (socket says 172.21.15.5:33534) 2024-09-19T06:19:37.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedf259c640 1 -- 172.21.15.5:0/657966352 learned_addr learned my addr 172.21.15.5:0/657966352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:37.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedf359e640 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3c0680 0x55dfcb3bd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.785+0000 7fedf2d9d640 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedf259c640 1 -- 172.21.15.5:0/657966352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3c0680 msgr2=0x55dfcb3bd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:37.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedf259c640 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3c0680 0x55dfcb3bd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedf259c640 1 -- 172.21.15.5:0/657966352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 msgr2=0x55dfcb3bff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:37.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedf259c640 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3bff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedf259c640 1 -- 172.21.15.5:0/657966352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfcb08c3c0 con 0x55dfcb3b5dc0 2024-09-19T06:19:37.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedf2d9d640 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3bff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:37.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedf259c640 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3bd1e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fede802ade0 tx=0x7fede80319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9990ac316c244a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:37.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:19:37.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedd6ff5640 1 -- 172.21.15.5:0/657966352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede804a040 con 0x55dfcb3b5dc0 2024-09-19T06:19:37.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedd6ff5640 1 -- 172.21.15.5:0/657966352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fede80046e0 con 0x55dfcb3b5dc0 2024-09-19T06:19:37.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedfb5bf740 1 -- 172.21.15.5:0/657966352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fede8036030 con 0x55dfcb3b5dc0 2024-09-19T06:19:37.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedd6ff5640 1 -- 172.21.15.5:0/657966352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fede805e050 con 0x55dfcb3b5dc0 2024-09-19T06:19:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedfb5bf740 10 client.19680 fetch_fsmap learned FSMap version 53 2024-09-19T06:19:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.786+0000 7fedfb5bf740 1 -- 172.21.15.5:0/657966352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55dfcb3c30d0 con 0x55dfcb3b5dc0 2024-09-19T06:19:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.787+0000 7fedd6ff5640 1 -- 172.21.15.5:0/657966352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fede8036030 con 0x55dfcb3b5dc0 2024-09-19T06:19:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.787+0000 7fedd6ff5640 1 -- 172.21.15.5:0/657966352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) ==== 8382+0+0 (secure 0 0 0) 0x7fede804d030 con 0x55dfcb3b5dc0 2024-09-19T06:19:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.787+0000 7fedfb5bf740 10 client.19680 fetch_fsmap finished waiting for FSMap version 53 2024-09-19T06:19:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.787+0000 7fedfb5bf740 10 client.19680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.787+0000 7fedfb5bf740 20 client.19680 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.787+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dfcb3cfbc0 0x55dfcb3d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.787+0000 7fedfb5bf740 4 client.19680 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:19:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.787+0000 7fedfb5bf740 1 -- 172.21.15.5:0/657966352 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfcb328850 con 0x55dfcb3cfbc0 2024-09-19T06:19:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.787+0000 7fedf2d9d640 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dfcb3cfbc0 0x55dfcb3d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.787+0000 7fedf2d9d640 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dfcb3cfbc0 0x55dfcb3d2040 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f195891ada663cbf server_cookie=8c6cde954d60f1a7 in_seq=0 out_seq=0 2024-09-19T06:19:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.787+0000 7fedd6ff5640 10 client.19680 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedd6ff5640 1 -- 172.21.15.5:0/657966352 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfcb328850 con 0x55dfcb3cfbc0 2024-09-19T06:19:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedd6ff5640 10 client.19680 handle_command_reply: tid=0 2024-09-19T06:19:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedfb5bf740 1 client.19680 shutdown 2024-09-19T06:19:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedfb5bf740 1 -- 172.21.15.5:0/657966352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b5dc0 msgr2=0x55dfcb3bd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3bd1e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fede802ade0 tx=0x7fede80319a0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedfb5bf740 1 -- 172.21.15.5:0/657966352 shutdown_connections 2024-09-19T06:19:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dfcb3cfbc0 0x55dfcb3d2040 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3c0680 0x55dfcb3bd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3bd1e0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/657966352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3bff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedfb5bf740 1 -- 172.21.15.5:0/657966352 >> 172.21.15.5:0/657966352 conn(0x55dfcb3b1a30 msgr2=0x55dfcb3b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedd6ff5640 0 client.19680 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedfb5bf740 1 -- 172.21.15.5:0/657966352 shutdown_connections 2024-09-19T06:19:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedfb5bf740 1 -- 172.21.15.5:0/657966352 wait complete. 2024-09-19T06:19:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.788+0000 7fedfb5bf740 20 client.19680 trim_cache size 0 max 16384 2024-09-19T06:19:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.805+0000 7fedfb5bf740 1 Processor -- start 2024-09-19T06:19:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.805+0000 7fedfb5bf740 1 -- start start 2024-09-19T06:19:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedfb5bf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedfb5bf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3d7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedfb5bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 0x55dfcb3d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedfb5bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41afa00 con 0x55dfcb3c0680 2024-09-19T06:19:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedfb5bf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41b1630 con 0x55dfcb3b54a0 2024-09-19T06:19:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedfb5bf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41b0190 con 0x55dfcb3b5dc0 2024-09-19T06:19:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedf359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 0x55dfcb3d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedf359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 0x55dfcb3d7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33538/0 (socket says 172.21.15.5:33538) 2024-09-19T06:19:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedf359e640 1 -- 172.21.15.5:0/1376204861 learned_addr learned my addr 172.21.15.5:0/1376204861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedf2d9d640 1 --2- 172.21.15.5:0/1376204861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedf259c640 1 --2- 172.21.15.5:0/1376204861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedf359e640 1 -- 172.21.15.5:0/1376204861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 msgr2=0x55dfcb3d7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedf359e640 1 --2- 172.21.15.5:0/1376204861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3d7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedf359e640 1 -- 172.21.15.5:0/1376204861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 msgr2=0x55dfcb3cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:37.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedf359e640 1 --2- 172.21.15.5:0/1376204861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedf359e640 1 -- 172.21.15.5:0/1376204861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfcb3c02a0 con 0x55dfcb3c0680 2024-09-19T06:19:37.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedf259c640 1 --2- 172.21.15.5:0/1376204861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3d7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:19:37.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedf359e640 1 --2- 172.21.15.5:0/1376204861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 0x55dfcb3d7e50 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fedec00f710 tx=0x7fedec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b07381441c97346 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:37.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedd8ff9640 1 -- 172.21.15.5:0/1376204861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec017030 con 0x55dfcb3c0680 2024-09-19T06:19:37.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedd8ff9640 1 -- 172.21.15.5:0/1376204861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedec01c930 con 0x55dfcb3c0680 2024-09-19T06:19:37.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.806+0000 7fedd8ff9640 1 -- 172.21.15.5:0/1376204861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec01d910 con 0x55dfcb3c0680 2024-09-19T06:19:37.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.807+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1376204861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 msgr2=0x55dfcb3d7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:37.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.807+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/1376204861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 0x55dfcb3d7e50 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fedec00f710 tx=0x7fedec016000 comp rx=0 tx=0).stop 2024-09-19T06:19:37.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.807+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1376204861 shutdown_connections 2024-09-19T06:19:37.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.807+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/1376204861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3c0680 0x55dfcb3d7e50 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.807+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/1376204861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3d7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.807+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/1376204861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.807+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1376204861 >> 172.21.15.5:0/1376204861 conn(0x55dfcb3b1a30 msgr2=0x55dfcb3b8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.807+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1376204861 shutdown_connections 2024-09-19T06:19:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.807+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1376204861 wait complete. 2024-09-19T06:19:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.807+0000 7fedfb5bf740 1 -- start start 2024-09-19T06:19:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.807+0000 7fedfb5bf740 1 Processor -- start 2024-09-19T06:19:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedfb5bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedfb5bf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedfb5bf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3c0680 0x55dfcb4328a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedfb5bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41b0f60 con 0x55dfcb3b54a0 2024-09-19T06:19:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedfb5bf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41b0860 con 0x55dfcb3c0680 2024-09-19T06:19:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedfb5bf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41ada00 con 0x55dfcb3b5dc0 2024-09-19T06:19:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedfb5bf740 10 client.0 _ll_register_callbacks cb 0x7fede8031ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33540/0 (socket says 172.21.15.5:33540) 2024-09-19T06:19:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf2d9d640 1 -- 172.21.15.5:0/1817249410 learned_addr learned my addr 172.21.15.5:0/1817249410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf359e640 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3c0680 0x55dfcb4328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf259c640 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf2d9d640 1 -- 172.21.15.5:0/1817249410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 msgr2=0x55dfcb3c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf2d9d640 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf2d9d640 1 -- 172.21.15.5:0/1817249410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3c0680 msgr2=0x55dfcb4328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf2d9d640 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3c0680 0x55dfcb4328a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf2d9d640 1 -- 172.21.15.5:0/1817249410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfcb432f40 con 0x55dfcb3b54a0 2024-09-19T06:19:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf359e640 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3c0680 0x55dfcb4328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:19:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf259c640 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:19:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedf2d9d640 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3c3be0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fede4097b40 tx=0x7fede4009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58259b7c39710881 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:19:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedd6ff5640 1 -- 172.21.15.5:0/1817249410 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4082070 con 0x55dfcb3b54a0 2024-09-19T06:19:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedd6ff5640 1 -- 172.21.15.5:0/1817249410 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fede40ccc10 con 0x55dfcb3b54a0 2024-09-19T06:19:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.808+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1817249410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedec01c930 con 0x55dfcb3b54a0 2024-09-19T06:19:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.809+0000 7fedd6ff5640 1 -- 172.21.15.5:0/1817249410 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fede401e040 con 0x55dfcb3b54a0 2024-09-19T06:19:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.809+0000 7fedfb5bf740 10 client.19686 fetch_fsmap learned FSMap version 53 2024-09-19T06:19:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.809+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1817249410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55dfcb43a2d0 con 0x55dfcb3b54a0 2024-09-19T06:19:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.809+0000 7fedd6ff5640 1 -- 172.21.15.5:0/1817249410 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedec01c930 con 0x55dfcb3b54a0 2024-09-19T06:19:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.809+0000 7fedd6ff5640 1 -- 172.21.15.5:0/1817249410 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) ==== 8382+0+0 (secure 0 0 0) 0x7fede40a3e10 con 0x55dfcb3b54a0 2024-09-19T06:19:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.809+0000 7fedfb5bf740 10 client.19686 fetch_fsmap finished waiting for FSMap version 53 2024-09-19T06:19:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.809+0000 7fedfb5bf740 10 client.19686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.809+0000 7fedfb5bf740 20 client.19686 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.809+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dfcb43aa50 0x55dfcb43ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.809+0000 7fedfb5bf740 4 client.19686 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:19:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.809+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1817249410 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dfcb322d60 con 0x55dfcb43aa50 2024-09-19T06:19:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.809+0000 7fedf259c640 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dfcb43aa50 0x55dfcb43ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedf259c640 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dfcb43aa50 0x55dfcb43ce40 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434494a589e4f3f0 server_cookie=26f5cd7d0b350865 in_seq=0 out_seq=0 2024-09-19T06:19:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedd6ff5640 10 client.19686 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedd6ff5640 1 -- 172.21.15.5:0/1817249410 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dfcb322d60 con 0x55dfcb43aa50 2024-09-19T06:19:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedd6ff5640 10 client.19686 handle_command_reply: tid=0 2024-09-19T06:19:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedfb5bf740 1 client.19686 shutdown 2024-09-19T06:19:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1817249410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b54a0 msgr2=0x55dfcb3c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3c3be0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fede4097b40 tx=0x7fede4009390 comp rx=0 tx=0).stop 2024-09-19T06:19:37.761 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:19:37.761 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:19:37.761 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:19:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: "29402121-cbd7-496c-be52-89dff12429e9": { 2024-09-19T06:19:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:19:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "29402121-cbd7-496c-be52-89dff12429e9", 2024-09-19T06:19:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:19:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:19:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:19:37.762 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:19:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1817249410 shutdown_connections 2024-09-19T06:19:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dfcb43aa50 0x55dfcb43ce40 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfcb3c0680 0x55dfcb4328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfcb3b5dc0 0x55dfcb3c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedd6ff5640 0 client.19686 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedfb5bf740 1 --2- 172.21.15.5:0/1817249410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfcb3b54a0 0x55dfcb3c3be0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1817249410 >> 172.21.15.5:0/1817249410 conn(0x55dfcb3b1a30 msgr2=0x55dfcb3bea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1817249410 shutdown_connections 2024-09-19T06:19:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedfb5bf740 1 -- 172.21.15.5:0/1817249410 wait complete. 2024-09-19T06:19:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.810+0000 7fedfb5bf740 20 client.19686 trim_cache size 0 max 16384 2024-09-19T06:19:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.811+0000 7fedf91e6640 1 -- 172.21.15.5:0/915695675 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7feddc0a33b0 msgr2=0x7feddc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.811+0000 7fedf91e6640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7feddc0a33b0 0x7feddc0a5870 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fedec005fd0 tx=0x7fedec005f60 comp rx=0 tx=0).stop 2024-09-19T06:19:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.811+0000 7fedf91e6640 1 -- 172.21.15.5:0/915695675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf4100b20 msgr2=0x7fedf419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.811+0000 7fedf91e6640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf4100b20 0x7fedf419a730 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fede400b4a0 tx=0x7fede400b960 comp rx=0 tx=0).stop 2024-09-19T06:19:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.811+0000 7fedf91e6640 1 -- 172.21.15.5:0/915695675 shutdown_connections 2024-09-19T06:19:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.811+0000 7fedf91e6640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7feddc0a33b0 0x7feddc0a5870 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.811+0000 7fedf91e6640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedf410e500 0x7fedf419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.811+0000 7fedf91e6640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedf4101440 0x7fedf419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.811+0000 7fedf91e6640 1 --2- 172.21.15.5:0/915695675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf4100b20 0x7fedf419a730 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.811+0000 7fedf91e6640 1 -- 172.21.15.5:0/915695675 >> 172.21.15.5:0/915695675 conn(0x7fedf40fc7d0 msgr2=0x7fedf40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.811+0000 7fedf91e6640 1 -- 172.21.15.5:0/915695675 shutdown_connections 2024-09-19T06:19:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:19:36.811+0000 7fedf91e6640 1 -- 172.21.15.5:0/915695675 wait complete. 2024-09-19T06:19:38.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:37 smithi005 ceph-mon[30955]: pgmap v512: 97 pgs: 97 active+clean; 636 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 433 op/s 2024-09-19T06:19:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:37 smithi157 ceph-mon[35346]: pgmap v512: 97 pgs: 97 active+clean; 636 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 433 op/s 2024-09-19T06:19:38.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:37 smithi049 ceph-mon[34640]: pgmap v512: 97 pgs: 97 active+clean; 636 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 433 op/s 2024-09-19T06:19:38.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 30527 8.79 MB/sec execute 62 sec latency 54.846 ms 2024-09-19T06:19:39.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 30716 8.86 MB/sec execute 63 sec latency 66.502 ms 2024-09-19T06:19:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:39 smithi157 ceph-mon[35346]: pgmap v513: 97 pgs: 97 active+clean; 636 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-09-19T06:19:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:19:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:19:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:19:40.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:19:40.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:19:40.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:39 smithi005 ceph-mon[30955]: pgmap v513: 97 pgs: 97 active+clean; 636 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-09-19T06:19:40.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:19:40.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:19:40.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:19:40.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:19:40.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:19:40.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:39 smithi049 ceph-mon[34640]: pgmap v513: 97 pgs: 97 active+clean; 636 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-09-19T06:19:40.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:19:40.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:19:40.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:19:40.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:19:40.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:19:40.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 30868 8.81 MB/sec execute 64 sec latency 154.112 ms 2024-09-19T06:19:41.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:40 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:41.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:40 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:41.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:40 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:19:41.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 31021 8.78 MB/sec execute 65 sec latency 87.442 ms 2024-09-19T06:19:42.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:41 smithi005 ceph-mon[30955]: pgmap v514: 97 pgs: 97 active+clean; 636 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-09-19T06:19:42.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:41 smithi157 ceph-mon[35346]: pgmap v514: 97 pgs: 97 active+clean; 636 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-09-19T06:19:42.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:41 smithi049 ceph-mon[34640]: pgmap v514: 97 pgs: 97 active+clean; 636 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-09-19T06:19:42.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 31131 8.71 MB/sec execute 66 sec latency 152.623 ms 2024-09-19T06:19:43.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 31248 8.62 MB/sec execute 67 sec latency 137.036 ms 2024-09-19T06:19:44.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:43 smithi157 ceph-mon[35346]: pgmap v515: 97 pgs: 97 active+clean; 643 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 410 op/s 2024-09-19T06:19:44.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:43 smithi005 ceph-mon[30955]: pgmap v515: 97 pgs: 97 active+clean; 643 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 410 op/s 2024-09-19T06:19:44.356 INFO:tasks.cephfs.filesystem:scrub status for tag:29402121-cbd7-496c-be52-89dff12429e9 - {'path': '/', 'tag': '29402121-cbd7-496c-be52-89dff12429e9', 'options': 'recursive,force'} 2024-09-19T06:19:44.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:43 smithi049 ceph-mon[34640]: pgmap v515: 97 pgs: 97 active+clean; 643 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 410 op/s 2024-09-19T06:19:44.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 31410 8.67 MB/sec execute 68 sec latency 174.035 ms 2024-09-19T06:19:45.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 31602 8.73 MB/sec execute 69 sec latency 65.929 ms 2024-09-19T06:19:46.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:45 smithi157 ceph-mon[35346]: pgmap v516: 97 pgs: 97 active+clean; 643 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 235 op/s 2024-09-19T06:19:46.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:45 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:19:46.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:45 smithi005 ceph-mon[30955]: pgmap v516: 97 pgs: 97 active+clean; 643 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 235 op/s 2024-09-19T06:19:46.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:45 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:19:46.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:45 smithi049 ceph-mon[34640]: pgmap v516: 97 pgs: 97 active+clean; 643 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 235 op/s 2024-09-19T06:19:46.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:45 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:19:46.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 31786 8.76 MB/sec execute 70 sec latency 319.034 ms 2024-09-19T06:19:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.946+0000 7f362b465640 1 -- 172.21.15.5:0/3471695296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36241069b0 msgr2=0x7f3624106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.946+0000 7f362b465640 1 --2- 172.21.15.5:0/3471695296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36241069b0 0x7f3624106d90 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f3620009930 tx=0x7f362002a420 comp rx=0 tx=0).stop 2024-09-19T06:19:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.946+0000 7f362b465640 1 -- 172.21.15.5:0/3471695296 shutdown_connections 2024-09-19T06:19:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.946+0000 7f362b465640 1 --2- 172.21.15.5:0/3471695296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3624101340 0x7f362410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.946+0000 7f362b465640 1 --2- 172.21.15.5:0/3471695296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36241009a0 0x7f3624100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.946+0000 7f362b465640 1 --2- 172.21.15.5:0/3471695296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36241069b0 0x7f3624106d90 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.946+0000 7f362b465640 1 -- 172.21.15.5:0/3471695296 >> 172.21.15.5:0/3471695296 conn(0x7f36240779a0 msgr2=0x7f36240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.947+0000 7f362b465640 1 -- 172.21.15.5:0/3471695296 shutdown_connections 2024-09-19T06:19:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.947+0000 7f362b465640 1 -- 172.21.15.5:0/3471695296 wait complete. 2024-09-19T06:19:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.947+0000 7f362b465640 1 Processor -- start 2024-09-19T06:19:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.947+0000 7f362b465640 1 -- start start 2024-09-19T06:19:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f362b465640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36241009a0 0x7f36241991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f362b465640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3624101340 0x7f36241996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f362b465640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36241069b0 0x7f36241a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f362b465640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3624110cf0 con 0x7f36241009a0 2024-09-19T06:19:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f362b465640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3624110b70 con 0x7f3624101340 2024-09-19T06:19:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f362b465640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3624110e70 con 0x7f36241069b0 2024-09-19T06:19:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f3629201640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36241009a0 0x7f36241991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f3629201640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36241009a0 0x7f36241991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37448/0 (socket says 172.21.15.5:37448) 2024-09-19T06:19:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f3629201640 1 -- 172.21.15.5:0/3451813892 learned_addr learned my addr 172.21.15.5:0/3451813892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f3628a00640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3624101340 0x7f36241996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f3629a02640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36241069b0 0x7f36241a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f3629201640 1 -- 172.21.15.5:0/3451813892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36241069b0 msgr2=0x7f36241a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f3629201640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36241069b0 0x7f36241a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f3629201640 1 -- 172.21.15.5:0/3451813892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3624101340 msgr2=0x7f36241996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f3629201640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3624101340 0x7f36241996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f3629201640 1 -- 172.21.15.5:0/3451813892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36241a4760 con 0x7f36241009a0 2024-09-19T06:19:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.948+0000 7f3628a00640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3624101340 0x7f36241996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.949+0000 7f3629201640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36241009a0 0x7f36241991a0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3620031440 tx=0x7f3620003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.949+0000 7f36167fc640 1 -- 172.21.15.5:0/3451813892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362003f070 con 0x7f36241009a0 2024-09-19T06:19:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.949+0000 7f362b465640 1 -- 172.21.15.5:0/3451813892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3624196f90 con 0x7f36241009a0 2024-09-19T06:19:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.949+0000 7f36167fc640 1 -- 172.21.15.5:0/3451813892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3620031b50 con 0x7f36241009a0 2024-09-19T06:19:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.949+0000 7f36167fc640 1 -- 172.21.15.5:0/3451813892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3620035030 con 0x7f36241009a0 2024-09-19T06:19:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.949+0000 7f362b465640 1 -- 172.21.15.5:0/3451813892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3624197420 con 0x7f36241009a0 2024-09-19T06:19:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.952+0000 7f36167fc640 1 -- 172.21.15.5:0/3451813892 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3620042070 con 0x7f36241009a0 2024-09-19T06:19:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f362e207740 1 Processor -- start 2024-09-19T06:19:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f362e207740 1 -- start start 2024-09-19T06:19:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f362e207740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 0x55b2b7566ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f362e207740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7567410 0x55b2b7571770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f362e207740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7571cb0 0x55b2b75740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f362e207740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2b74cb2f0 con 0x55b2b7566af0 2024-09-19T06:19:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f362e207740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b2b7473270 con 0x55b2b7567410 2024-09-19T06:19:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f362e207740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b2b74c72b0 con 0x55b2b7571cb0 2024-09-19T06:19:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f3629a02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 0x55b2b7566ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f3629201640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7567410 0x55b2b7571770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f3629a02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 0x55b2b7566ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37452/0 (socket says 172.21.15.5:37452) 2024-09-19T06:19:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f3629a02640 1 -- 172.21.15.5:0/3132257438 learned_addr learned my addr 172.21.15.5:0/3132257438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.953+0000 7f3628a00640 1 --2- 172.21.15.5:0/3132257438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7571cb0 0x55b2b75740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.954+0000 7f3629a02640 1 -- 172.21.15.5:0/3132257438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7571cb0 msgr2=0x55b2b75740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.954+0000 7f3629a02640 1 --2- 172.21.15.5:0/3132257438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7571cb0 0x55b2b75740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.954+0000 7f3629a02640 1 -- 172.21.15.5:0/3132257438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7567410 msgr2=0x55b2b7571770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.954+0000 7f3629a02640 1 --2- 172.21.15.5:0/3132257438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7567410 0x55b2b7571770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.954+0000 7f3629a02640 1 -- 172.21.15.5:0/3132257438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b7472340 con 0x55b2b7566af0 2024-09-19T06:19:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.954+0000 7f3628a00640 1 --2- 172.21.15.5:0/3132257438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7571cb0 0x55b2b75740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:19:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.954+0000 7f3629a02640 1 --2- 172.21.15.5:0/3132257438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 0x55b2b7566ed0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f361c009f60 tx=0x7f361c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab1e214fe1cd893 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.954+0000 7f360f7fe640 1 -- 172.21.15.5:0/3132257438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c01e070 con 0x55b2b7566af0 2024-09-19T06:19:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.954+0000 7f360f7fe640 1 -- 172.21.15.5:0/3132257438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f361c003c30 con 0x55b2b7566af0 2024-09-19T06:19:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.954+0000 7f360f7fe640 1 -- 172.21.15.5:0/3132257438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c00f820 con 0x55b2b7566af0 2024-09-19T06:19:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.954+0000 7f36167fc640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36100a3400 0x7f36100a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.954+0000 7f3629201640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36100a3400 0x7f36100a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.955+0000 7f36167fc640 1 -- 172.21.15.5:0/3451813892 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f36200f03b0 con 0x7f36241009a0 2024-09-19T06:19:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.955+0000 7f3629201640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36100a3400 0x7f36100a58c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f36200f3fd0 tx=0x7f36200f3e60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.955+0000 7f362e207740 1 -- 172.21.15.5:0/3132257438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 msgr2=0x55b2b7566ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.955+0000 7f362e207740 1 --2- 172.21.15.5:0/3132257438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 0x55b2b7566ed0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f361c009f60 tx=0x7f361c009f90 comp rx=0 tx=0).stop 2024-09-19T06:19:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.956+0000 7f362e207740 1 -- 172.21.15.5:0/3132257438 shutdown_connections 2024-09-19T06:19:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.956+0000 7f362e207740 1 --2- 172.21.15.5:0/3132257438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7571cb0 0x55b2b75740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.956+0000 7f362e207740 1 --2- 172.21.15.5:0/3132257438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7567410 0x55b2b7571770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.956+0000 7f362e207740 1 --2- 172.21.15.5:0/3132257438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 0x55b2b7566ed0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.956+0000 7f362e207740 1 -- 172.21.15.5:0/3132257438 >> 172.21.15.5:0/3132257438 conn(0x55b2b7563060 msgr2=0x55b2b7563440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.956+0000 7f362e207740 1 -- 172.21.15.5:0/3132257438 shutdown_connections 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.956+0000 7f362e207740 1 -- 172.21.15.5:0/3132257438 wait complete. 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.956+0000 7f362e207740 1 -- start start 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.956+0000 7f362e207740 1 Processor -- start 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f362e207740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7566af0 0x55b2b756ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f362e207740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7567410 0x55b2b756f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f362e207740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7571cb0 0x55b2b756f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f362e207740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2b7574880 con 0x55b2b7567410 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f362e207740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b2b7454290 con 0x55b2b7571cb0 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f362e207740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b2b7574a00 con 0x55b2b7566af0 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f362e207740 10 client.0 _ll_register_callbacks cb 0x55b2b7485140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f3628a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7567410 0x55b2b756f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f3629a02640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7566af0 0x55b2b756ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f3629a02640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7566af0 0x55b2b756ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46342/0 (socket says 172.21.15.5:46342) 2024-09-19T06:19:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f3629a02640 1 -- 172.21.15.5:0/175505452 learned_addr learned my addr 172.21.15.5:0/175505452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f3629a02640 1 --2- 172.21.15.5:0/175505452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7571cb0 0x55b2b756f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f3629a02640 1 -- 172.21.15.5:0/175505452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7571cb0 msgr2=0x55b2b756f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f3629a02640 1 --2- 172.21.15.5:0/175505452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7571cb0 0x55b2b756f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f3629a02640 1 -- 172.21.15.5:0/175505452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7567410 msgr2=0x55b2b756f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f3629a02640 1 --2- 172.21.15.5:0/175505452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7567410 0x55b2b756f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f3629a02640 1 -- 172.21.15.5:0/175505452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b71de210 con 0x55b2b7566af0 2024-09-19T06:19:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f3629a02640 1 --2- 172.21.15.5:0/175505452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7566af0 0x55b2b756ebc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f361c003f20 tx=0x7f361c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faa6d2acd20b4040 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.957+0000 7f360cff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:19:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.958+0000 7f360cff9640 1 -- 172.21.15.5:0/175505452 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c01e030 con 0x55b2b7566af0 2024-09-19T06:19:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.958+0000 7f360cff9640 1 -- 172.21.15.5:0/175505452 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f361c00f4e0 con 0x55b2b7566af0 2024-09-19T06:19:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.958+0000 7f362e207740 1 -- 172.21.15.5:0/175505452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f361c003c30 con 0x55b2b7566af0 2024-09-19T06:19:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.958+0000 7f360cff9640 1 -- 172.21.15.5:0/175505452 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f361c00f6e0 con 0x55b2b7566af0 2024-09-19T06:19:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.958+0000 7f362e207740 10 client.29005 fetch_fsmap learned FSMap version 54 2024-09-19T06:19:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.958+0000 7f362e207740 1 -- 172.21.15.5:0/175505452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b2b72aa620 con 0x55b2b7566af0 2024-09-19T06:19:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.958+0000 7f360cff9640 1 -- 172.21.15.5:0/175505452 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f361c02d070 con 0x55b2b7566af0 2024-09-19T06:19:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.959+0000 7f360cff9640 1 -- 172.21.15.5:0/175505452 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f361c0054a0 con 0x55b2b7566af0 2024-09-19T06:19:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.959+0000 7f362e207740 10 client.29005 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T06:19:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.959+0000 7f362e207740 10 client.29005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.959+0000 7f362e207740 20 client.29005 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.959+0000 7f362e207740 1 --2- 172.21.15.5:0/175505452 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b2b7581b30 0x55b2b7583f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.959+0000 7f362e207740 4 client.29005 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:19:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.959+0000 7f362e207740 1 -- 172.21.15.5:0/175505452 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f361c003c30 con 0x55b2b7581b30 2024-09-19T06:19:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.959+0000 7f3628a00640 1 --2- 172.21.15.5:0/175505452 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b2b7581b30 0x55b2b7583f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.959+0000 7f360cff9640 10 client.29005 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.959+0000 7f3628a00640 1 --2- 172.21.15.5:0/175505452 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b2b7581b30 0x55b2b7583f20 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69350cbb8edb54dc server_cookie=2eee19f3681bce53 in_seq=0 out_seq=0 2024-09-19T06:19:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f360cff9640 1 -- 172.21.15.5:0/175505452 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f361c003c30 con 0x55b2b7581b30 2024-09-19T06:19:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f360cff9640 10 client.29005 handle_command_reply: tid=0 2024-09-19T06:19:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f362e207740 1 client.29005 shutdown 2024-09-19T06:19:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f362e207740 1 -- 172.21.15.5:0/175505452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7566af0 msgr2=0x55b2b756ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f362e207740 1 --2- 172.21.15.5:0/175505452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7566af0 0x55b2b756ebc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f361c003f20 tx=0x7f361c000f40 comp rx=0 tx=0).stop 2024-09-19T06:19:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f362e207740 1 -- 172.21.15.5:0/175505452 shutdown_connections 2024-09-19T06:19:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f362e207740 1 --2- 172.21.15.5:0/175505452 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b2b7581b30 0x55b2b7583f20 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f362e207740 1 --2- 172.21.15.5:0/175505452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7571cb0 0x55b2b756f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f360cff9640 0 client.29005 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f362e207740 1 --2- 172.21.15.5:0/175505452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7567410 0x55b2b756f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f362e207740 1 --2- 172.21.15.5:0/175505452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7566af0 0x55b2b756ebc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f362e207740 1 -- 172.21.15.5:0/175505452 >> 172.21.15.5:0/175505452 conn(0x55b2b7563060 msgr2=0x55b2b7568dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f362e207740 1 -- 172.21.15.5:0/175505452 shutdown_connections 2024-09-19T06:19:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.960+0000 7f362e207740 1 -- 172.21.15.5:0/175505452 wait complete. 2024-09-19T06:19:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.961+0000 7f362e207740 20 client.29005 trim_cache size 0 max 16384 2024-09-19T06:19:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.986+0000 7f362e207740 1 Processor -- start 2024-09-19T06:19:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.986+0000 7f362e207740 1 -- start start 2024-09-19T06:19:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.986+0000 7f362e207740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 0x55b2b756f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.986+0000 7f362e207740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7567410 0x55b2b7589660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f362e207740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7571cb0 0x55b2b7589ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f362e207740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36241afb00 con 0x55b2b7566af0 2024-09-19T06:19:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f362e207740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f36241b1700 con 0x55b2b7571cb0 2024-09-19T06:19:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f362e207740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f36241b0290 con 0x55b2b7567410 2024-09-19T06:19:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f3629a02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 0x55b2b756f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f3629a02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 0x55b2b756f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37462/0 (socket says 172.21.15.5:37462) 2024-09-19T06:19:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f3629a02640 1 -- 172.21.15.5:0/3876201631 learned_addr learned my addr 172.21.15.5:0/3876201631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f3628a00640 1 --2- 172.21.15.5:0/3876201631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7567410 0x55b2b7589660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f3629a02640 1 --2- 172.21.15.5:0/3876201631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7571cb0 0x55b2b7589ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f3629a02640 1 -- 172.21.15.5:0/3876201631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7567410 msgr2=0x55b2b7589660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f3629a02640 1 --2- 172.21.15.5:0/3876201631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7567410 0x55b2b7589660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f3629a02640 1 -- 172.21.15.5:0/3876201631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7571cb0 msgr2=0x55b2b7589ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f3629a02640 1 --2- 172.21.15.5:0/3876201631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7571cb0 0x55b2b7589ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f3629a02640 1 -- 172.21.15.5:0/3876201631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b733a8a0 con 0x55b2b7566af0 2024-09-19T06:19:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f3628a00640 1 --2- 172.21.15.5:0/3876201631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7567410 0x55b2b7589660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:19:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.987+0000 7f3629a02640 1 --2- 172.21.15.5:0/3876201631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 0x55b2b756f470 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f361c0099a0 tx=0x7f361c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed186c68089e053 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.988+0000 7f360effd640 1 -- 172.21.15.5:0/3876201631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c040070 con 0x55b2b7566af0 2024-09-19T06:19:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.988+0000 7f360effd640 1 -- 172.21.15.5:0/3876201631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f361c02d050 con 0x55b2b7566af0 2024-09-19T06:19:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.988+0000 7f360effd640 1 -- 172.21.15.5:0/3876201631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c01e040 con 0x55b2b7566af0 2024-09-19T06:19:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.989+0000 7f362e207740 1 -- 172.21.15.5:0/3876201631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 msgr2=0x55b2b756f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.989+0000 7f362e207740 1 --2- 172.21.15.5:0/3876201631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 0x55b2b756f470 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f361c0099a0 tx=0x7f361c005fb0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.989+0000 7f362e207740 1 -- 172.21.15.5:0/3876201631 shutdown_connections 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.989+0000 7f362e207740 1 --2- 172.21.15.5:0/3876201631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7571cb0 0x55b2b7589ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.989+0000 7f362e207740 1 --2- 172.21.15.5:0/3876201631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7567410 0x55b2b7589660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.989+0000 7f362e207740 1 --2- 172.21.15.5:0/3876201631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7566af0 0x55b2b756f470 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.989+0000 7f362e207740 1 -- 172.21.15.5:0/3876201631 >> 172.21.15.5:0/3876201631 conn(0x55b2b7563060 msgr2=0x55b2b7570c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.989+0000 7f362e207740 1 -- 172.21.15.5:0/3876201631 shutdown_connections 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.989+0000 7f362e207740 1 -- 172.21.15.5:0/3876201631 wait complete. 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.989+0000 7f362e207740 1 -- start start 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.989+0000 7f362e207740 1 Processor -- start 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.990+0000 7f362e207740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7566af0 0x55b2b757a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.990+0000 7f362e207740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7567410 0x55b2b7575450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.990+0000 7f362e207740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7571cb0 0x55b2b7575990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.990+0000 7f362e207740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36241b1030 con 0x55b2b7571cb0 2024-09-19T06:19:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.990+0000 7f362e207740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f36241b0960 con 0x55b2b7567410 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.990+0000 7f362e207740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f36241adb00 con 0x55b2b7566af0 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.990+0000 7f362e207740 10 client.0 _ll_register_callbacks cb 0x7f361c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.990+0000 7f3628a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7567410 0x55b2b7575450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.990+0000 7f3628a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7567410 0x55b2b7575450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41384/0 (socket says 172.21.15.5:41384) 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.990+0000 7f3628a00640 1 -- 172.21.15.5:0/3180936954 learned_addr learned my addr 172.21.15.5:0/3180936954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.990+0000 7f3629a02640 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7571cb0 0x55b2b7575990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.990+0000 7f3629a02640 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7566af0 0x55b2b757a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f3628a00640 1 -- 172.21.15.5:0/3180936954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7566af0 msgr2=0x55b2b757a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f3628a00640 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7566af0 0x55b2b757a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f3628a00640 1 -- 172.21.15.5:0/3180936954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7571cb0 msgr2=0x55b2b7575990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f3628a00640 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7571cb0 0x55b2b7575990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f3628a00640 1 -- 172.21.15.5:0/3180936954 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b7575ed0 con 0x55b2b7567410 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f3628a00640 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7567410 0x55b2b7575450 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55b2b7569690 tx=0x7f3618013ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55bd4238a1996c60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:19:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f360cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f360cff9640 1 -- 172.21.15.5:0/3180936954 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361801e790 con 0x55b2b7567410 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f360cff9640 1 -- 172.21.15.5:0/3180936954 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7f361801e930 con 0x55b2b7567410 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f362e207740 1 -- 172.21.15.5:0/3180936954 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f361c02d050 con 0x55b2b7567410 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f360cff9640 1 -- 172.21.15.5:0/3180936954 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f361801eb30 con 0x55b2b7567410 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f362e207740 10 client.29075 fetch_fsmap learned FSMap version 54 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.991+0000 7f362e207740 1 -- 172.21.15.5:0/3180936954 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b2b75ec3f0 con 0x55b2b7567410 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.992+0000 7f360cff9640 1 -- 172.21.15.5:0/3180936954 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f361c02d050 con 0x55b2b7567410 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.992+0000 7f360cff9640 1 -- 172.21.15.5:0/3180936954 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f3618025030 con 0x55b2b7567410 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.992+0000 7f362e207740 10 client.29075 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.992+0000 7f362e207740 10 client.29075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.992+0000 7f362e207740 20 client.29075 populate_metadata read hostname 'smithi005' 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.992+0000 7f362e207740 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b2b7581b30 0x55b2b75eeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.992+0000 7f362e207740 4 client.29075 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:19:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.992+0000 7f362e207740 1 -- 172.21.15.5:0/3180936954 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2b74d8a20 con 0x55b2b7581b30 2024-09-19T06:19:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.993+0000 7f3629a02640 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b2b7581b30 0x55b2b75eeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:19:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.993+0000 7f360cff9640 10 client.29075 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:46.993+0000 7f3629a02640 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b2b7581b30 0x55b2b75eeae0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=286d17a902a4fe9e server_cookie=a281b17bd7523414 in_seq=0 out_seq=0 2024-09-19T06:19:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f360cff9640 1 -- 172.21.15.5:0/3180936954 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2b74d8a20 con 0x55b2b7581b30 2024-09-19T06:19:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f360cff9640 10 client.29075 handle_command_reply: tid=0 2024-09-19T06:19:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f362e207740 1 client.29075 shutdown 2024-09-19T06:19:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f362e207740 1 -- 172.21.15.5:0/3180936954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7567410 msgr2=0x55b2b7575450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f362e207740 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7567410 0x55b2b7575450 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55b2b7569690 tx=0x7f3618013ab0 comp rx=0 tx=0).stop 2024-09-19T06:19:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f362e207740 1 -- 172.21.15.5:0/3180936954 shutdown_connections 2024-09-19T06:19:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f362e207740 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b2b7581b30 0x55b2b75eeae0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f362e207740 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b7571cb0 0x55b2b7575990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f362e207740 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b7567410 0x55b2b7575450 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f362e207740 1 --2- 172.21.15.5:0/3180936954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2b7566af0 0x55b2b757a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f362e207740 1 -- 172.21.15.5:0/3180936954 >> 172.21.15.5:0/3180936954 conn(0x55b2b7563060 msgr2=0x55b2b75619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f360cff9640 0 client.29075 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:19:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f362e207740 1 -- 172.21.15.5:0/3180936954 shutdown_connections 2024-09-19T06:19:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.067+0000 7f362e207740 1 -- 172.21.15.5:0/3180936954 wait complete. 2024-09-19T06:19:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.068+0000 7f362e207740 20 client.29075 trim_cache size 0 max 16384 2024-09-19T06:19:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:19:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:19:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:19:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:19:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.069+0000 7f362b465640 1 -- 172.21.15.5:0/3451813892 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36100a3400 msgr2=0x7f36100a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.069+0000 7f362b465640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36100a3400 0x7f36100a58c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f36200f3fd0 tx=0x7f36200f3e60 comp rx=0 tx=0).stop 2024-09-19T06:19:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.069+0000 7f362b465640 1 -- 172.21.15.5:0/3451813892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36241009a0 msgr2=0x7f36241991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:19:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.069+0000 7f362b465640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36241009a0 0x7f36241991a0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3620031440 tx=0x7f3620003b20 comp rx=0 tx=0).stop 2024-09-19T06:19:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.069+0000 7f362b465640 1 -- 172.21.15.5:0/3451813892 shutdown_connections 2024-09-19T06:19:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.069+0000 7f362b465640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36100a3400 0x7f36100a58c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.069+0000 7f362b465640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36241069b0 0x7f36241a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.069+0000 7f362b465640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3624101340 0x7f36241996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.069+0000 7f362b465640 1 --2- 172.21.15.5:0/3451813892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36241009a0 0x7f36241991a0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:19:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.069+0000 7f362b465640 1 -- 172.21.15.5:0/3451813892 >> 172.21.15.5:0/3451813892 conn(0x7f36240779a0 msgr2=0x7f36240ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:19:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.069+0000 7f362b465640 1 -- 172.21.15.5:0/3451813892 shutdown_connections 2024-09-19T06:19:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:19:47.069+0000 7f362b465640 1 -- 172.21.15.5:0/3451813892 wait complete. 2024-09-19T06:19:47.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 31965 8.70 MB/sec execute 71 sec latency 64.350 ms 2024-09-19T06:19:48.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:47 smithi157 ceph-mon[35346]: pgmap v517: 97 pgs: 97 active+clean; 646 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 362 op/s 2024-09-19T06:19:48.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:47 smithi005 ceph-mon[30955]: pgmap v517: 97 pgs: 97 active+clean; 646 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 362 op/s 2024-09-19T06:19:48.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:47 smithi049 ceph-mon[34640]: pgmap v517: 97 pgs: 97 active+clean; 646 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 362 op/s 2024-09-19T06:19:48.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 32120 8.70 MB/sec execute 72 sec latency 58.976 ms 2024-09-19T06:19:49.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 32264 8.67 MB/sec execute 73 sec latency 133.977 ms 2024-09-19T06:19:50.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:49 smithi157 ceph-mon[35346]: pgmap v518: 97 pgs: 97 active+clean; 646 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-09-19T06:19:50.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:49 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:19:50.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:49 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:19:50.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:49 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:19:50.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:49 smithi005 ceph-mon[30955]: pgmap v518: 97 pgs: 97 active+clean; 646 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-09-19T06:19:50.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:49 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:19:50.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:49 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:19:50.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:49 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:19:50.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:49 smithi049 ceph-mon[34640]: pgmap v518: 97 pgs: 97 active+clean; 646 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-09-19T06:19:50.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:49 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:19:50.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:49 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:19:50.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:49 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:19:50.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 32454 8.71 MB/sec execute 74 sec latency 58.900 ms 2024-09-19T06:19:51.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 32629 8.73 MB/sec execute 75 sec latency 86.625 ms 2024-09-19T06:19:52.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:51 smithi157 ceph-mon[35346]: pgmap v519: 97 pgs: 97 active+clean; 646 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-09-19T06:19:52.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:51 smithi005 ceph-mon[30955]: pgmap v519: 97 pgs: 97 active+clean; 646 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-09-19T06:19:52.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:51 smithi049 ceph-mon[34640]: pgmap v519: 97 pgs: 97 active+clean; 646 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-09-19T06:19:52.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 32809 8.74 MB/sec execute 76 sec latency 54.460 ms 2024-09-19T06:19:53.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 33023 8.81 MB/sec execute 77 sec latency 98.510 ms 2024-09-19T06:19:54.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:53 smithi157 ceph-mon[35346]: pgmap v520: 97 pgs: 97 active+clean; 650 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 385 op/s 2024-09-19T06:19:54.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:53 smithi005 ceph-mon[30955]: pgmap v520: 97 pgs: 97 active+clean; 650 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 385 op/s 2024-09-19T06:19:54.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:53 smithi049 ceph-mon[34640]: pgmap v520: 97 pgs: 97 active+clean; 650 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 385 op/s 2024-09-19T06:19:54.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 33163 8.79 MB/sec execute 78 sec latency 56.930 ms 2024-09-19T06:19:55.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 33355 8.91 MB/sec execute 79 sec latency 53.641 ms 2024-09-19T06:19:56.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:55 smithi157 ceph-mon[35346]: pgmap v521: 97 pgs: 97 active+clean; 650 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-09-19T06:19:56.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:55 smithi005 ceph-mon[30955]: pgmap v521: 97 pgs: 97 active+clean; 650 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-09-19T06:19:56.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:55 smithi049 ceph-mon[34640]: pgmap v521: 97 pgs: 97 active+clean; 650 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-09-19T06:19:56.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 33558 8.94 MB/sec execute 80 sec latency 55.731 ms 2024-09-19T06:19:57.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 33704 8.87 MB/sec execute 81 sec latency 61.728 ms 2024-09-19T06:19:58.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:57 smithi157 ceph-mon[35346]: pgmap v522: 97 pgs: 97 active+clean; 665 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.7 MiB/s wr, 403 op/s 2024-09-19T06:19:58.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:57 smithi005 ceph-mon[30955]: pgmap v522: 97 pgs: 97 active+clean; 665 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.7 MiB/s wr, 403 op/s 2024-09-19T06:19:58.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:57 smithi049 ceph-mon[34640]: pgmap v522: 97 pgs: 97 active+clean; 665 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.7 MiB/s wr, 403 op/s 2024-09-19T06:19:58.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 33834 8.83 MB/sec execute 82 sec latency 61.743 ms 2024-09-19T06:19:59.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 34007 8.82 MB/sec execute 83 sec latency 92.525 ms 2024-09-19T06:20:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.208+0000 7fbf622ba640 1 -- 172.21.15.5:0/893084325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5c1069b0 msgr2=0x7fbf5c106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.208+0000 7fbf622ba640 1 --2- 172.21.15.5:0/893084325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5c1069b0 0x7fbf5c106d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00d990 tx=0x7fbf4c02c2f0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.208+0000 7fbf622ba640 1 -- 172.21.15.5:0/893084325 shutdown_connections 2024-09-19T06:20:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.208+0000 7fbf622ba640 1 --2- 172.21.15.5:0/893084325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf5c101340 0x7fbf5c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.208+0000 7fbf622ba640 1 --2- 172.21.15.5:0/893084325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c1009a0 0x7fbf5c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.208+0000 7fbf622ba640 1 --2- 172.21.15.5:0/893084325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5c1069b0 0x7fbf5c106d90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.208+0000 7fbf622ba640 1 -- 172.21.15.5:0/893084325 >> 172.21.15.5:0/893084325 conn(0x7fbf5c0779a0 msgr2=0x7fbf5c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.209+0000 7fbf622ba640 1 -- 172.21.15.5:0/893084325 shutdown_connections 2024-09-19T06:20:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.209+0000 7fbf622ba640 1 -- 172.21.15.5:0/893084325 wait complete. 2024-09-19T06:20:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.209+0000 7fbf622ba640 1 Processor -- start 2024-09-19T06:20:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.209+0000 7fbf622ba640 1 -- start start 2024-09-19T06:20:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf622ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf5c1009a0 0x7fbf5c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf622ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5c101340 0x7fbf5c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf622ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c1069b0 0x7fbf5c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf622ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c10fba0 con 0x7fbf5c1069b0 2024-09-19T06:20:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf622ba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c10fa20 con 0x7fbf5c1009a0 2024-09-19T06:20:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf622ba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c10fd20 con 0x7fbf5c101340 2024-09-19T06:20:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf60857640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c1069b0 0x7fbf5c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf5c1009a0 0x7fbf5c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf5b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5c101340 0x7fbf5c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf5b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5c101340 0x7fbf5c19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53440/0 (socket says 172.21.15.5:53440) 2024-09-19T06:20:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/726946557 learned_addr learned my addr 172.21.15.5:0/726946557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/726946557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf5c1009a0 msgr2=0x7fbf5c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf5c1009a0 0x7fbf5c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/726946557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c1069b0 msgr2=0x7fbf5c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.210+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c1069b0 0x7fbf5c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.211+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/726946557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c19f320 con 0x7fbf5c101340 2024-09-19T06:20:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.211+0000 7fbf5bfff640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf5c1009a0 0x7fbf5c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.211+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5c101340 0x7fbf5c19a870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9f0 tx=0x7fbf4400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.211+0000 7fbf597fa640 1 -- 172.21.15.5:0/726946557 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400cd80 con 0x7fbf5c101340 2024-09-19T06:20:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.211+0000 7fbf597fa640 1 -- 172.21.15.5:0/726946557 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf440108e0 con 0x7fbf5c101340 2024-09-19T06:20:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.211+0000 7fbf622ba640 1 -- 172.21.15.5:0/726946557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5c19f610 con 0x7fbf5c101340 2024-09-19T06:20:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.211+0000 7fbf597fa640 1 -- 172.21.15.5:0/726946557 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44018920 con 0x7fbf5c101340 2024-09-19T06:20:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.211+0000 7fbf622ba640 1 -- 172.21.15.5:0/726946557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5c1a6ea0 con 0x7fbf5c101340 2024-09-19T06:20:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.215+0000 7fbf6505a740 1 Processor -- start 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.215+0000 7fbf6505a740 1 -- start start 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.215+0000 7fbf597fa640 1 -- 172.21.15.5:0/726946557 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbf44010a90 con 0x7fbf5c101340 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.215+0000 7fbf6505a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf6505a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9b2410 0x55a3bf9bc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf6505a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf6505a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a3bf9162f0 con 0x55a3bf9bccb0 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf6505a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a3bf8be270 con 0x55a3bf9b1af0 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf6505a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a3bf9122b0 con 0x55a3bf9b2410 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf60857640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf60857640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:52188/0 (socket says 172.21.15.5:52188) 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf60857640 1 -- 172.21.15.5:0/1299323678 learned_addr learned my addr 172.21.15.5:0/1299323678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf5bfff640 1 --2- 172.21.15.5:0/1299323678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9b2410 0x55a3bf9bc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf60857640 1 -- 172.21.15.5:0/1299323678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9b2410 msgr2=0x55a3bf9bc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf60857640 1 --2- 172.21.15.5:0/1299323678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9b2410 0x55a3bf9bc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf60857640 1 -- 172.21.15.5:0/1299323678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9bccb0 msgr2=0x55a3bf9bf0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:20:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/1299323678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf60857640 1 --2- 172.21.15.5:0/1299323678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9bf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf60857640 1 -- 172.21.15.5:0/1299323678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3bf8bd340 con 0x55a3bf9b1af0 2024-09-19T06:20:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.216+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/1299323678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:20:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.217+0000 7fbf60857640 1 --2- 172.21.15.5:0/1299323678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b1ed0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400e480 tx=0x7fbf540095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cf9564eb73f2846 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.217+0000 7fbf597fa640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbf480a33b0 0x7fbf480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.217+0000 7fbf527fc640 1 -- 172.21.15.5:0/1299323678 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54003970 con 0x55a3bf9b1af0 2024-09-19T06:20:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.217+0000 7fbf597fa640 1 -- 172.21.15.5:0/726946557 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fbf440d0010 con 0x7fbf5c101340 2024-09-19T06:20:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.217+0000 7fbf527fc640 1 -- 172.21.15.5:0/1299323678 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf5400d7d0 con 0x55a3bf9b1af0 2024-09-19T06:20:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.217+0000 7fbf527fc640 1 -- 172.21.15.5:0/1299323678 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf540043e0 con 0x55a3bf9b1af0 2024-09-19T06:20:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.217+0000 7fbf5bfff640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbf480a33b0 0x7fbf480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.218+0000 7fbf6505a740 1 -- 172.21.15.5:0/1299323678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b1af0 msgr2=0x55a3bf9b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.218+0000 7fbf6505a740 1 --2- 172.21.15.5:0/1299323678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b1ed0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400e480 tx=0x7fbf540095c0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.218+0000 7fbf5bfff640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbf480a33b0 0x7fbf480a5870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00d960 tx=0x7fbf4c032340 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.218+0000 7fbf6505a740 1 -- 172.21.15.5:0/1299323678 shutdown_connections 2024-09-19T06:20:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.218+0000 7fbf6505a740 1 --2- 172.21.15.5:0/1299323678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.218+0000 7fbf6505a740 1 --2- 172.21.15.5:0/1299323678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9b2410 0x55a3bf9bc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.218+0000 7fbf6505a740 1 --2- 172.21.15.5:0/1299323678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b1ed0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.218+0000 7fbf6505a740 1 -- 172.21.15.5:0/1299323678 >> 172.21.15.5:0/1299323678 conn(0x55a3bf9ae060 msgr2=0x55a3bf9ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.218+0000 7fbf6505a740 1 -- 172.21.15.5:0/1299323678 shutdown_connections 2024-09-19T06:20:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.218+0000 7fbf6505a740 1 -- 172.21.15.5:0/1299323678 wait complete. 2024-09-19T06:20:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.219+0000 7fbf6505a740 1 -- start start 2024-09-19T06:20:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.219+0000 7fbf6505a740 1 Processor -- start 2024-09-19T06:20:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.219+0000 7fbf6505a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.219+0000 7fbf6505a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b2410 0x55a3bf9ba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf6505a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9ba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf6505a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a3bf9bf880 con 0x55a3bf9b1af0 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf6505a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a3bf89f290 con 0x55a3bf9b2410 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf6505a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a3bf9bfa00 con 0x55a3bf9bccb0 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59212/0 (socket says 172.21.15.5:59212) 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf5bfff640 1 -- 172.21.15.5:0/1181927139 learned_addr learned my addr 172.21.15.5:0/1181927139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf6505a740 10 client.0 _ll_register_callbacks cb 0x55a3bf8d0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf60857640 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b2410 0x55a3bf9ba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf5bfff640 1 -- 172.21.15.5:0/1181927139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9bccb0 msgr2=0x55a3bf9ba690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf5bfff640 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9ba690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf5bfff640 1 -- 172.21.15.5:0/1181927139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b2410 msgr2=0x55a3bf9ba150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf5bfff640 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b2410 0x55a3bf9ba150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf5bfff640 1 -- 172.21.15.5:0/1181927139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3bf629210 con 0x55a3bf9b1af0 2024-09-19T06:20:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf5bfff640 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b9c10 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00d990 tx=0x7fbf4c02c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4486afbcbb5d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.220+0000 7fbf3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:20:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf3b7fe640 1 -- 172.21.15.5:0/1181927139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c047040 con 0x55a3bf9b1af0 2024-09-19T06:20:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf6505a740 1 -- 172.21.15.5:0/1181927139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf5400d7d0 con 0x55a3bf9b1af0 2024-09-19T06:20:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf3b7fe640 1 -- 172.21.15.5:0/1181927139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fbf4c003ac0 con 0x55a3bf9b1af0 2024-09-19T06:20:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf3b7fe640 1 -- 172.21.15.5:0/1181927139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fbf4c059050 con 0x55a3bf9b1af0 2024-09-19T06:20:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf6505a740 10 client.19713 fetch_fsmap learned FSMap version 54 2024-09-19T06:20:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf6505a740 1 -- 172.21.15.5:0/1181927139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a3bf6f5620 con 0x55a3bf9b1af0 2024-09-19T06:20:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf3b7fe640 1 -- 172.21.15.5:0/1181927139 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf5400d7d0 con 0x55a3bf9b1af0 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf3b7fe640 1 -- 172.21.15.5:0/1181927139 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fbf4c040c10 con 0x55a3bf9b1af0 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf6505a740 10 client.19713 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf6505a740 10 client.19713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf6505a740 20 client.19713 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf6505a740 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a3bf9d53d0 0x55a3bf9d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf6505a740 4 client.19713 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.221+0000 7fbf6505a740 1 -- 172.21.15.5:0/1181927139 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3bf923a20 con 0x55a3bf9d53d0 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.222+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a3bf9d53d0 0x55a3bf9d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.222+0000 7fbf3b7fe640 10 client.19713 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.222+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a3bf9d53d0 0x55a3bf9d77e0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a78e300831b3c73 server_cookie=10201511140e1c1b in_seq=0 out_seq=0 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf3b7fe640 1 -- 172.21.15.5:0/1181927139 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3bf923a20 con 0x55a3bf9d53d0 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf3b7fe640 10 client.19713 handle_command_reply: tid=0 2024-09-19T06:20:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf6505a740 1 client.19713 shutdown 2024-09-19T06:20:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf6505a740 1 -- 172.21.15.5:0/1181927139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 msgr2=0x55a3bf9b9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf6505a740 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b9c10 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00d990 tx=0x7fbf4c02c2f0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf6505a740 1 -- 172.21.15.5:0/1181927139 shutdown_connections 2024-09-19T06:20:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf6505a740 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a3bf9d53d0 0x55a3bf9d77e0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf6505a740 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9ba690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf6505a740 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b2410 0x55a3bf9ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf6505a740 1 --2- 172.21.15.5:0/1181927139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b9c10 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf6505a740 1 -- 172.21.15.5:0/1181927139 >> 172.21.15.5:0/1181927139 conn(0x55a3bf9ae060 msgr2=0x55a3bf9b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf3b7fe640 0 client.19713 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf6505a740 1 -- 172.21.15.5:0/1181927139 shutdown_connections 2024-09-19T06:20:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf6505a740 1 -- 172.21.15.5:0/1181927139 wait complete. 2024-09-19T06:20:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.223+0000 7fbf6505a740 20 client.19713 trim_cache size 0 max 16384 2024-09-19T06:20:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.249+0000 7fbf6505a740 1 Processor -- start 2024-09-19T06:20:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.249+0000 7fbf6505a740 1 -- start start 2024-09-19T06:20:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf6505a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf6505a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b2410 0x55a3bf9ba4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf6505a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9dd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf6505a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ab650 con 0x55a3bf9b2410 2024-09-19T06:20:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf6505a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ad250 con 0x55a3bf9bccb0 2024-09-19T06:20:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf6505a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1abde0 con 0x55a3bf9b1af0 2024-09-19T06:20:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf5b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b2410 0x55a3bf9ba4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf5b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b2410 0x55a3bf9ba4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59220/0 (socket says 172.21.15.5:59220) 2024-09-19T06:20:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/732440087 learned_addr learned my addr 172.21.15.5:0/732440087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf60857640 1 --2- 172.21.15.5:0/732440087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9dd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf5bfff640 1 --2- 172.21.15.5:0/732440087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf60857640 1 -- 172.21.15.5:0/732440087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9b1af0 msgr2=0x55a3bf9b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf60857640 1 --2- 172.21.15.5:0/732440087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf60857640 1 -- 172.21.15.5:0/732440087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b2410 msgr2=0x55a3bf9ba4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf60857640 1 --2- 172.21.15.5:0/732440087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b2410 0x55a3bf9ba4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf60857640 1 -- 172.21.15.5:0/732440087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3bf9add30 con 0x55a3bf9bccb0 2024-09-19T06:20:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.250+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/732440087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b2410 0x55a3bf9ba4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.251+0000 7fbf60857640 1 --2- 172.21.15.5:0/732440087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9dd140 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009fd0 tx=0x7fbf54000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ba274135e8a2fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.251+0000 7fbf51ffb640 1 -- 172.21.15.5:0/732440087 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400d7d0 con 0x55a3bf9bccb0 2024-09-19T06:20:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.251+0000 7fbf51ffb640 1 -- 172.21.15.5:0/732440087 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf54025020 con 0x55a3bf9bccb0 2024-09-19T06:20:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.251+0000 7fbf51ffb640 1 -- 172.21.15.5:0/732440087 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54005870 con 0x55a3bf9bccb0 2024-09-19T06:20:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.252+0000 7fbf6505a740 1 -- 172.21.15.5:0/732440087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9bccb0 msgr2=0x55a3bf9dd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.252+0000 7fbf6505a740 1 --2- 172.21.15.5:0/732440087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9dd140 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009fd0 tx=0x7fbf54000f40 comp rx=0 tx=0).stop 2024-09-19T06:20:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.252+0000 7fbf6505a740 1 -- 172.21.15.5:0/732440087 shutdown_connections 2024-09-19T06:20:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.252+0000 7fbf6505a740 1 --2- 172.21.15.5:0/732440087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9dd140 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.252+0000 7fbf6505a740 1 --2- 172.21.15.5:0/732440087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b2410 0x55a3bf9ba4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.252+0000 7fbf6505a740 1 --2- 172.21.15.5:0/732440087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.252+0000 7fbf6505a740 1 -- 172.21.15.5:0/732440087 >> 172.21.15.5:0/732440087 conn(0x55a3bf9ae060 msgr2=0x55a3bf9bbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.252+0000 7fbf6505a740 1 -- 172.21.15.5:0/732440087 shutdown_connections 2024-09-19T06:20:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.252+0000 7fbf6505a740 1 -- 172.21.15.5:0/732440087 wait complete. 2024-09-19T06:20:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf6505a740 1 -- start start 2024-09-19T06:20:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf6505a740 1 Processor -- start 2024-09-19T06:20:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf6505a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf6505a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b2410 0x55a3bf9b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf6505a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9b7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf6505a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1acb80 con 0x55a3bf9b1af0 2024-09-19T06:20:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf6505a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ac4b0 con 0x55a3bf9b2410 2024-09-19T06:20:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf6505a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1a9690 con 0x55a3bf9bccb0 2024-09-19T06:20:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf6505a740 10 client.0 _ll_register_callbacks cb 0x7fbf4c044040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59228/0 (socket says 172.21.15.5:59228) 2024-09-19T06:20:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf5bfff640 1 -- 172.21.15.5:0/844490447 learned_addr learned my addr 172.21.15.5:0/844490447 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b2410 0x55a3bf9b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.253+0000 7fbf60857640 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/844490447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9bccb0 msgr2=0x55a3bf9b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9b7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/844490447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 msgr2=0x55a3bf9b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/844490447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3bfa3f110 con 0x55a3bf9b2410 2024-09-19T06:20:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf5bfff640 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b2410 0x55a3bf9b70e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbf440cd7a0 tx=0x7fbf4400cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bcc53d0d7f5f4a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:20:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf3b7fe640 1 -- 172.21.15.5:0/844490447 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf440989f0 con 0x55a3bf9b2410 2024-09-19T06:20:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf3b7fe640 1 -- 172.21.15.5:0/844490447 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8550+0+0 (secure 0 0 0) 0x7fbf440a4ab0 con 0x55a3bf9b2410 2024-09-19T06:20:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf6505a740 1 -- 172.21.15.5:0/844490447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf54025020 con 0x55a3bf9b2410 2024-09-19T06:20:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf3b7fe640 1 -- 172.21.15.5:0/844490447 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fbf44098b90 con 0x55a3bf9b2410 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf6505a740 10 client.29093 fetch_fsmap learned FSMap version 54 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.254+0000 7fbf6505a740 1 -- 172.21.15.5:0/844490447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a3bfa403b0 con 0x55a3bf9b2410 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.255+0000 7fbf3b7fe640 1 -- 172.21.15.5:0/844490447 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf54025020 con 0x55a3bf9b2410 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.255+0000 7fbf3b7fe640 1 -- 172.21.15.5:0/844490447 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fbf440a01f0 con 0x55a3bf9b2410 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.255+0000 7fbf6505a740 10 client.29093 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.255+0000 7fbf6505a740 10 client.29093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.255+0000 7fbf6505a740 20 client.29093 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.255+0000 7fbf6505a740 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a3bfa40ad0 0x55a3bfa42ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.255+0000 7fbf6505a740 4 client.29093 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.255+0000 7fbf6505a740 1 -- 172.21.15.5:0/844490447 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3bf91dd70 con 0x55a3bfa40ad0 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.255+0000 7fbf5bfff640 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a3bfa40ad0 0x55a3bfa42ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.256+0000 7fbf5bfff640 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a3bfa40ad0 0x55a3bfa42ef0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7266e6b439cecb45 server_cookie=e50b513fffdcf994 in_seq=0 out_seq=0 2024-09-19T06:20:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.256+0000 7fbf3b7fe640 10 client.29093 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:19:59 smithi157 ceph-mon[35346]: pgmap v523: 97 pgs: 97 active+clean; 665 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.6 MiB/s wr, 276 op/s 2024-09-19T06:20:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf3b7fe640 1 -- 172.21.15.5:0/844490447 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3bf91dd70 con 0x55a3bfa40ad0 2024-09-19T06:20:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf3b7fe640 10 client.29093 handle_command_reply: tid=0 2024-09-19T06:20:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf6505a740 1 client.29093 shutdown 2024-09-19T06:20:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf6505a740 1 -- 172.21.15.5:0/844490447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b2410 msgr2=0x55a3bf9b70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf6505a740 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b2410 0x55a3bf9b70e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbf440cd7a0 tx=0x7fbf4400cf20 comp rx=0 tx=0).stop 2024-09-19T06:20:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf6505a740 1 -- 172.21.15.5:0/844490447 shutdown_connections 2024-09-19T06:20:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf6505a740 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a3bfa40ad0 0x55a3bfa42ef0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf6505a740 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3bf9bccb0 0x55a3bf9b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf6505a740 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3bf9b2410 0x55a3bf9b70e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf3b7fe640 0 client.29093 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf6505a740 1 --2- 172.21.15.5:0/844490447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3bf9b1af0 0x55a3bf9b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf6505a740 1 -- 172.21.15.5:0/844490447 >> 172.21.15.5:0/844490447 conn(0x55a3bf9ae060 msgr2=0x55a3bf9ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf6505a740 1 -- 172.21.15.5:0/844490447 shutdown_connections 2024-09-19T06:20:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.319+0000 7fbf6505a740 1 -- 172.21.15.5:0/844490447 wait complete. 2024-09-19T06:20:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.320+0000 7fbf6505a740 20 client.29093 trim_cache size 0 max 16384 2024-09-19T06:20:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:20:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:20:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:20:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:20:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.320+0000 7fbf622ba640 1 -- 172.21.15.5:0/726946557 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbf480a33b0 msgr2=0x7fbf480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.320+0000 7fbf622ba640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbf480a33b0 0x7fbf480a5870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00d960 tx=0x7fbf4c032340 comp rx=0 tx=0).stop 2024-09-19T06:20:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.320+0000 7fbf622ba640 1 -- 172.21.15.5:0/726946557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5c101340 msgr2=0x7fbf5c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.320+0000 7fbf622ba640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5c101340 0x7fbf5c19a870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9f0 tx=0x7fbf4400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.321+0000 7fbf622ba640 1 -- 172.21.15.5:0/726946557 shutdown_connections 2024-09-19T06:20:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.321+0000 7fbf622ba640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c1069b0 0x7fbf5c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.321+0000 7fbf622ba640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbf480a33b0 0x7fbf480a5870 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.321+0000 7fbf622ba640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5c101340 0x7fbf5c19a870 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.321+0000 7fbf622ba640 1 --2- 172.21.15.5:0/726946557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf5c1009a0 0x7fbf5c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.321+0000 7fbf622ba640 1 -- 172.21.15.5:0/726946557 >> 172.21.15.5:0/726946557 conn(0x7fbf5c0779a0 msgr2=0x7fbf5c103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.321+0000 7fbf622ba640 1 -- 172.21.15.5:0/726946557 shutdown_connections 2024-09-19T06:20:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:00.321+0000 7fbf622ba640 1 -- 172.21.15.5:0/726946557 wait complete. 2024-09-19T06:20:00.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:19:59 smithi005 ceph-mon[30955]: pgmap v523: 97 pgs: 97 active+clean; 665 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.6 MiB/s wr, 276 op/s 2024-09-19T06:20:00.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:19:59 smithi049 ceph-mon[34640]: pgmap v523: 97 pgs: 97 active+clean; 665 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.6 MiB/s wr, 276 op/s 2024-09-19T06:20:00.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 34169 8.78 MB/sec execute 84 sec latency 101.065 ms 2024-09-19T06:20:01.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:00 smithi157 ceph-mon[35346]: overall HEALTH_OK 2024-09-19T06:20:01.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:00 smithi005 ceph-mon[30955]: overall HEALTH_OK 2024-09-19T06:20:01.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:00 smithi049 ceph-mon[34640]: overall HEALTH_OK 2024-09-19T06:20:01.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 34331 8.76 MB/sec execute 85 sec latency 131.400 ms 2024-09-19T06:20:02.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:01 smithi157 ceph-mon[35346]: pgmap v524: 97 pgs: 97 active+clean; 665 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.6 MiB/s wr, 276 op/s 2024-09-19T06:20:02.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:01 smithi005 ceph-mon[30955]: pgmap v524: 97 pgs: 97 active+clean; 665 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.6 MiB/s wr, 276 op/s 2024-09-19T06:20:02.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:01 smithi049 ceph-mon[34640]: pgmap v524: 97 pgs: 97 active+clean; 665 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.6 MiB/s wr, 276 op/s 2024-09-19T06:20:02.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 34511 8.79 MB/sec execute 86 sec latency 57.953 ms 2024-09-19T06:20:03.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 34679 8.80 MB/sec execute 87 sec latency 55.009 ms 2024-09-19T06:20:04.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:03 smithi157 ceph-mon[35346]: pgmap v525: 97 pgs: 97 active+clean; 668 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 432 op/s 2024-09-19T06:20:04.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:03 smithi005 ceph-mon[30955]: pgmap v525: 97 pgs: 97 active+clean; 668 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 432 op/s 2024-09-19T06:20:04.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:03 smithi049 ceph-mon[34640]: pgmap v525: 97 pgs: 97 active+clean; 668 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 432 op/s 2024-09-19T06:20:04.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 34876 8.83 MB/sec execute 88 sec latency 65.049 ms 2024-09-19T06:20:05.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:04 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:20:05.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:04 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:20:05.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:04 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:20:05.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 35066 8.82 MB/sec execute 89 sec latency 76.327 ms 2024-09-19T06:20:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:05 smithi157 ceph-mon[35346]: pgmap v526: 97 pgs: 97 active+clean; 668 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 285 op/s 2024-09-19T06:20:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:05 smithi157 ceph-mon[35346]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T06:20:06.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:05 smithi005 ceph-mon[30955]: pgmap v526: 97 pgs: 97 active+clean; 668 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 285 op/s 2024-09-19T06:20:06.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:05 smithi005 ceph-mon[30955]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T06:20:06.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:05 smithi049 ceph-mon[34640]: pgmap v526: 97 pgs: 97 active+clean; 668 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 285 op/s 2024-09-19T06:20:06.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:05 smithi049 ceph-mon[34640]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T06:20:06.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 35235 8.78 MB/sec execute 90 sec latency 168.856 ms 2024-09-19T06:20:07.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 35406 8.79 MB/sec execute 91 sec latency 60.056 ms 2024-09-19T06:20:08.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:07 smithi157 ceph-mon[35346]: pgmap v528: 97 pgs: 97 active+clean; 684 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 335 op/s 2024-09-19T06:20:08.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:07 smithi005 ceph-mon[30955]: pgmap v528: 97 pgs: 97 active+clean; 684 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 335 op/s 2024-09-19T06:20:08.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:07 smithi049 ceph-mon[34640]: pgmap v528: 97 pgs: 97 active+clean; 684 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 335 op/s 2024-09-19T06:20:08.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 35569 8.81 MB/sec execute 92 sec latency 81.458 ms 2024-09-19T06:20:09.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:08 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:20:09.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:08 smithi157 ceph-mon[35346]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T06:20:09.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:08 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:20:09.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:08 smithi005 ceph-mon[30955]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T06:20:09.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:08 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:20:09.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:08 smithi049 ceph-mon[34640]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T06:20:09.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 35725 8.79 MB/sec execute 93 sec latency 58.329 ms 2024-09-19T06:20:10.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 35919 8.87 MB/sec execute 94 sec latency 154.544 ms 2024-09-19T06:20:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:10 smithi157 ceph-mon[35346]: pgmap v530: 97 pgs: 97 active+clean; 684 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 419 op/s 2024-09-19T06:20:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:10 smithi005 ceph-mon[30955]: pgmap v530: 97 pgs: 97 active+clean; 684 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 419 op/s 2024-09-19T06:20:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:10 smithi049 ceph-mon[34640]: pgmap v530: 97 pgs: 97 active+clean; 684 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 419 op/s 2024-09-19T06:20:11.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 36117 8.88 MB/sec execute 95 sec latency 79.520 ms 2024-09-19T06:20:12.556 INFO:tasks.workunit.client.0.smithi005.stdout: 10 36319 8.92 MB/sec execute 96 sec latency 61.651 ms 2024-09-19T06:20:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:12 smithi157 ceph-mon[35346]: pgmap v531: 97 pgs: 97 active+clean; 684 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 184 op/s 2024-09-19T06:20:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:12 smithi005 ceph-mon[30955]: pgmap v531: 97 pgs: 97 active+clean; 684 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 184 op/s 2024-09-19T06:20:12.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:12 smithi049 ceph-mon[34640]: pgmap v531: 97 pgs: 97 active+clean; 684 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 184 op/s 2024-09-19T06:20:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.456+0000 7fdcd143c640 1 -- 172.21.15.5:0/2454252705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdccc102460 msgr2=0x7fdccc10e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.456+0000 7fdcd143c640 1 --2- 172.21.15.5:0/2454252705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdccc102460 0x7fdccc10e990 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009cd0 tx=0x7fdcc402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.456+0000 7fdcd143c640 1 -- 172.21.15.5:0/2454252705 shutdown_connections 2024-09-19T06:20:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.456+0000 7fdcd143c640 1 --2- 172.21.15.5:0/2454252705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdccc102460 0x7fdccc10e990 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.456+0000 7fdcd143c640 1 --2- 172.21.15.5:0/2454252705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdccc101ac0 0x7fdccc101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.456+0000 7fdcd143c640 1 --2- 172.21.15.5:0/2454252705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc107ad0 0x7fdccc107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.456+0000 7fdcd143c640 1 -- 172.21.15.5:0/2454252705 >> 172.21.15.5:0/2454252705 conn(0x7fdccc0fd7a0 msgr2=0x7fdccc0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.456+0000 7fdcd143c640 1 -- 172.21.15.5:0/2454252705 shutdown_connections 2024-09-19T06:20:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.456+0000 7fdcd143c640 1 -- 172.21.15.5:0/2454252705 wait complete. 2024-09-19T06:20:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.457+0000 7fdcd143c640 1 Processor -- start 2024-09-19T06:20:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.457+0000 7fdcd143c640 1 -- start start 2024-09-19T06:20:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdcd143c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdccc101ac0 0x7fdccc1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdcd143c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc102460 0x7fdccc198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdcd143c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdccc107ad0 0x7fdccc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdcd143c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc111dc0 con 0x7fdccc107ad0 2024-09-19T06:20:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdcd143c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc111c40 con 0x7fdccc101ac0 2024-09-19T06:20:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdcd143c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc111f40 con 0x7fdccc102460 2024-09-19T06:20:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdccc101ac0 0x7fdccc1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdccc107ad0 0x7fdccc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdccc107ad0 0x7fdccc19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43348/0 (socket says 172.21.15.5:43348) 2024-09-19T06:20:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdccb7fe640 1 -- 172.21.15.5:0/3527759160 learned_addr learned my addr 172.21.15.5:0/3527759160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdcca7fc640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc102460 0x7fdccc198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdccaffd640 1 -- 172.21.15.5:0/3527759160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc102460 msgr2=0x7fdccc198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdccaffd640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc102460 0x7fdccc198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdccaffd640 1 -- 172.21.15.5:0/3527759160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdccc107ad0 msgr2=0x7fdccc19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdccaffd640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdccc107ad0 0x7fdccc19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdccaffd640 1 -- 172.21.15.5:0/3527759160 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc19d2c0 con 0x7fdccc101ac0 2024-09-19T06:20:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.458+0000 7fdcca7fc640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc102460 0x7fdccc198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.459+0000 7fdccb7fe640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdccc107ad0 0x7fdccc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.459+0000 7fdccaffd640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdccc101ac0 0x7fdccc1982d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00e9f0 tx=0x7fdcbc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.459+0000 7fdcb3fff640 1 -- 172.21.15.5:0/3527759160 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00ccc0 con 0x7fdccc101ac0 2024-09-19T06:20:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.459+0000 7fdcd143c640 1 -- 172.21.15.5:0/3527759160 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdccc19d5b0 con 0x7fdccc101ac0 2024-09-19T06:20:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.459+0000 7fdcb3fff640 1 -- 172.21.15.5:0/3527759160 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcbc005020 con 0x7fdccc101ac0 2024-09-19T06:20:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.459+0000 7fdcb3fff640 1 -- 172.21.15.5:0/3527759160 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc004a00 con 0x7fdccc101ac0 2024-09-19T06:20:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.459+0000 7fdcd143c640 1 -- 172.21.15.5:0/3527759160 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdccc074420 con 0x7fdccc101ac0 2024-09-19T06:20:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.462+0000 7fdcb3fff640 1 -- 172.21.15.5:0/3527759160 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fdcbc004ba0 con 0x7fdccc101ac0 2024-09-19T06:20:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdcd41d4740 1 Processor -- start 2024-09-19T06:20:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdcd41d4740 1 -- start start 2024-09-19T06:20:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdcd41d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5daf0 0x555860d5ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdcd41d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d5e410 0x555860d68770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdcd41d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d68cb0 0x555860d6b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdcd41d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555860cc22f0 con 0x555860d5e410 2024-09-19T06:20:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdcd41d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555860c6a270 con 0x555860d5daf0 2024-09-19T06:20:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdcd41d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555860cbe2b0 con 0x555860d68cb0 2024-09-19T06:20:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5daf0 0x555860d5ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5daf0 0x555860d5ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54492/0 (socket says 172.21.15.5:54492) 2024-09-19T06:20:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdccb7fe640 1 -- 172.21.15.5:0/2007543540 learned_addr learned my addr 172.21.15.5:0/2007543540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdccaffd640 1 --2- 172.21.15.5:0/2007543540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d5e410 0x555860d68770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.463+0000 7fdcca7fc640 1 --2- 172.21.15.5:0/2007543540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d68cb0 0x555860d6b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.464+0000 7fdcb3fff640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdcb40a3430 0x7fdcb40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.464+0000 7fdccb7fe640 1 -- 172.21.15.5:0/2007543540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d68cb0 msgr2=0x555860d6b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.464+0000 7fdccb7fe640 1 --2- 172.21.15.5:0/2007543540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d68cb0 0x555860d6b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.464+0000 7fdccb7fe640 1 -- 172.21.15.5:0/2007543540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d5e410 msgr2=0x555860d68770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.464+0000 7fdccb7fe640 1 --2- 172.21.15.5:0/2007543540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d5e410 0x555860d68770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.464+0000 7fdccb7fe640 1 -- 172.21.15.5:0/2007543540 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555860c69340 con 0x555860d5daf0 2024-09-19T06:20:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.464+0000 7fdcb3fff640 1 -- 172.21.15.5:0/3527759160 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7fdcbc0cfcc0 con 0x7fdccc101ac0 2024-09-19T06:20:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.464+0000 7fdccb7fe640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdcb40a3430 0x7fdcb40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.464+0000 7fdccb7fe640 1 --2- 172.21.15.5:0/2007543540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5daf0 0x555860d5ded0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009410 tx=0x7fdcc4005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c798c72a60176d5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.465+0000 7fdcb17fa640 1 -- 172.21.15.5:0/2007543540 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc403c920 con 0x555860d5daf0 2024-09-19T06:20:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.465+0000 7fdcb17fa640 1 -- 172.21.15.5:0/2007543540 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc4036040 con 0x555860d5daf0 2024-09-19T06:20:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.465+0000 7fdcb17fa640 1 -- 172.21.15.5:0/2007543540 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4004730 con 0x555860d5daf0 2024-09-19T06:20:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.465+0000 7fdcd41d4740 1 -- 172.21.15.5:0/2007543540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5daf0 msgr2=0x555860d5ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.465+0000 7fdccb7fe640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdcb40a3430 0x7fdcb40a58f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4051000 tx=0x7fdcc4054000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.465+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/2007543540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5daf0 0x555860d5ded0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009410 tx=0x7fdcc4005c50 comp rx=0 tx=0).stop 2024-09-19T06:20:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.466+0000 7fdcd41d4740 1 -- 172.21.15.5:0/2007543540 shutdown_connections 2024-09-19T06:20:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.466+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/2007543540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d68cb0 0x555860d6b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.466+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/2007543540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d5e410 0x555860d68770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.466+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/2007543540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5daf0 0x555860d5ded0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.466+0000 7fdcd41d4740 1 -- 172.21.15.5:0/2007543540 >> 172.21.15.5:0/2007543540 conn(0x555860d5a060 msgr2=0x555860d5a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.466+0000 7fdcd41d4740 1 -- 172.21.15.5:0/2007543540 shutdown_connections 2024-09-19T06:20:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.466+0000 7fdcd41d4740 1 -- 172.21.15.5:0/2007543540 wait complete. 2024-09-19T06:20:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.466+0000 7fdcd41d4740 1 -- start start 2024-09-19T06:20:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.466+0000 7fdcd41d4740 1 Processor -- start 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdcd41d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d5daf0 0x555860d65bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdcd41d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d5e410 0x555860d660f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdcd41d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 0x555860d66630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdcd41d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555860d6b880 con 0x555860d5daf0 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdcd41d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555860c4b290 con 0x555860d68cb0 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdcd41d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555860d6ba00 con 0x555860d5e410 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d5daf0 0x555860d65bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d5daf0 0x555860d65bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43362/0 (socket says 172.21.15.5:43362) 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdccaffd640 1 -- 172.21.15.5:0/1731310668 learned_addr learned my addr 172.21.15.5:0/1731310668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdcca7fc640 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d5e410 0x555860d660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdcd41d4740 10 client.0 _ll_register_callbacks cb 0x555860c7c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdccb7fe640 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 0x555860d66630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdccaffd640 1 -- 172.21.15.5:0/1731310668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d5e410 msgr2=0x555860d660f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.467+0000 7fdccaffd640 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d5e410 0x555860d660f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.468+0000 7fdccaffd640 1 -- 172.21.15.5:0/1731310668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 msgr2=0x555860d66630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.468+0000 7fdccaffd640 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 0x555860d66630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.468+0000 7fdccaffd640 1 -- 172.21.15.5:0/1731310668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558609d5210 con 0x555860d5daf0 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.468+0000 7fdccb7fe640 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 0x555860d66630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.468+0000 7fdccaffd640 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d5daf0 0x555860d65bb0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00c6a0 tx=0x7fdcbc0096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a5446a33cb5fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.468+0000 7fdcaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.468+0000 7fdcaeff5640 1 -- 172.21.15.5:0/1731310668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc01c070 con 0x555860d5daf0 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.468+0000 7fdcaeff5640 1 -- 172.21.15.5:0/1731310668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7fdcbc098590 con 0x555860d5daf0 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.468+0000 7fdcd41d4740 1 -- 172.21.15.5:0/1731310668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc4036040 con 0x555860d5daf0 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.468+0000 7fdcaeff5640 1 -- 172.21.15.5:0/1731310668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc09ba60 con 0x555860d5daf0 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.468+0000 7fdcd41d4740 10 client.19731 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.468+0000 7fdcd41d4740 1 -- 172.21.15.5:0/1731310668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x555860aa1620 con 0x555860d5daf0 2024-09-19T06:20:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.469+0000 7fdcaeff5640 1 -- 172.21.15.5:0/1731310668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc4036040 con 0x555860d5daf0 2024-09-19T06:20:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.469+0000 7fdcaeff5640 1 -- 172.21.15.5:0/1731310668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7fdcbc09f020 con 0x555860d5daf0 2024-09-19T06:20:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.469+0000 7fdcd41d4740 10 client.19731 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.469+0000 7fdcd41d4740 10 client.19731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.469+0000 7fdcd41d4740 20 client.19731 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.469+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555860d78c80 0x555860d7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.469+0000 7fdcd41d4740 4 client.19731 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.469+0000 7fdcd41d4740 1 -- 172.21.15.5:0/1731310668 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555860ccfa20 con 0x555860d78c80 2024-09-19T06:20:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.469+0000 7fdcca7fc640 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555860d78c80 0x555860d7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.470+0000 7fdcca7fc640 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555860d78c80 0x555860d7b090 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f5fd45f9b7c97a server_cookie=56f23e5617ed66a4 in_seq=0 out_seq=0 2024-09-19T06:20:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.470+0000 7fdcaeff5640 10 client.19731 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.471+0000 7fdcaeff5640 1 -- 172.21.15.5:0/1731310668 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555860ccfa20 con 0x555860d78c80 2024-09-19T06:20:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.471+0000 7fdcaeff5640 10 client.19731 handle_command_reply: tid=0 2024-09-19T06:20:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.471+0000 7fdcd41d4740 1 client.19731 shutdown 2024-09-19T06:20:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.472+0000 7fdcd41d4740 1 -- 172.21.15.5:0/1731310668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d5daf0 msgr2=0x555860d65bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.472+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d5daf0 0x555860d65bb0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00c6a0 tx=0x7fdcbc0096e0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.472+0000 7fdcd41d4740 1 -- 172.21.15.5:0/1731310668 shutdown_connections 2024-09-19T06:20:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.472+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555860d78c80 0x555860d7b090 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.472+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 0x555860d66630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.472+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d5e410 0x555860d660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.472+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/1731310668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d5daf0 0x555860d65bb0 secure :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00c6a0 tx=0x7fdcbc0096e0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.472+0000 7fdcd41d4740 1 -- 172.21.15.5:0/1731310668 >> 172.21.15.5:0/1731310668 conn(0x555860d5a060 msgr2=0x555860d5fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.472+0000 7fdcaeff5640 0 client.19731 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.472+0000 7fdcd41d4740 1 -- 172.21.15.5:0/1731310668 shutdown_connections 2024-09-19T06:20:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.472+0000 7fdcd41d4740 1 -- 172.21.15.5:0/1731310668 wait complete. 2024-09-19T06:20:13.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.472+0000 7fdcd41d4740 20 client.19731 trim_cache size 0 max 16384 2024-09-19T06:20:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdcd41d4740 1 Processor -- start 2024-09-19T06:20:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdcd41d4740 1 -- start start 2024-09-19T06:20:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdcd41d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d5e410 0x555860d66460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdcd41d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 0x555860d67b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdcd41d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d64c80 0x555860d809a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdcd41d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1ab720 con 0x555860d64c80 2024-09-19T06:20:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdcd41d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1ad290 con 0x555860d68cb0 2024-09-19T06:20:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdcd41d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1abe20 con 0x555860d5e410 2024-09-19T06:20:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d64c80 0x555860d809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdcca7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 0x555860d67b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d5e410 0x555860d66460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdcca7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 0x555860d67b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54512/0 (socket says 172.21.15.5:54512) 2024-09-19T06:20:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d64c80 0x555860d809a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43374/0 (socket says 172.21.15.5:43374) 2024-09-19T06:20:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.501+0000 7fdcca7fc640 1 -- 172.21.15.5:0/3966450062 learned_addr learned my addr 172.21.15.5:0/3966450062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.502+0000 7fdcca7fc640 1 -- 172.21.15.5:0/3966450062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d5e410 msgr2=0x555860d66460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.502+0000 7fdcca7fc640 1 --2- 172.21.15.5:0/3966450062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d5e410 0x555860d66460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.502+0000 7fdcca7fc640 1 -- 172.21.15.5:0/3966450062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d64c80 msgr2=0x555860d809a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.502+0000 7fdcca7fc640 1 --2- 172.21.15.5:0/3966450062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d64c80 0x555860d809a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.502+0000 7fdcca7fc640 1 -- 172.21.15.5:0/3966450062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555860d59e90 con 0x555860d68cb0 2024-09-19T06:20:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.502+0000 7fdccaffd640 1 --2- 172.21.15.5:0/3966450062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d5e410 0x555860d66460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.502+0000 7fdcca7fc640 1 --2- 172.21.15.5:0/3966450062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 0x555860d67b20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x555860d60690 tx=0x7fdcc000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace7b3e38c0a28cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.502+0000 7fdcb0ff9640 1 -- 172.21.15.5:0/3966450062 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0004220 con 0x555860d68cb0 2024-09-19T06:20:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.502+0000 7fdcb0ff9640 1 -- 172.21.15.5:0/3966450062 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc00043c0 con 0x555860d68cb0 2024-09-19T06:20:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.502+0000 7fdcb0ff9640 1 -- 172.21.15.5:0/3966450062 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00106f0 con 0x555860d68cb0 2024-09-19T06:20:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.503+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3966450062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 msgr2=0x555860d67b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.503+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/3966450062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 0x555860d67b20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x555860d60690 tx=0x7fdcc000b390 comp rx=0 tx=0).stop 2024-09-19T06:20:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.503+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3966450062 shutdown_connections 2024-09-19T06:20:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.503+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/3966450062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d64c80 0x555860d809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.503+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/3966450062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d68cb0 0x555860d67b20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.503+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/3966450062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d5e410 0x555860d66460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.503+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3966450062 >> 172.21.15.5:0/3966450062 conn(0x555860d5a060 msgr2=0x555860d67410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.503+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3966450062 shutdown_connections 2024-09-19T06:20:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.503+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3966450062 wait complete. 2024-09-19T06:20:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.504+0000 7fdcd41d4740 1 -- start start 2024-09-19T06:20:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.504+0000 7fdcd41d4740 1 Processor -- start 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.504+0000 7fdcd41d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5e410 0x555860d6cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.504+0000 7fdcd41d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d64c80 0x555860d6d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.504+0000 7fdcd41d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d68cb0 0x555860de2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.504+0000 7fdcd41d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1acbc0 con 0x555860d64c80 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.504+0000 7fdcd41d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1ac4f0 con 0x555860d5e410 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.504+0000 7fdcd41d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1a9720 con 0x555860d68cb0 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.504+0000 7fdcd41d4740 10 client.0 _ll_register_callbacks cb 0x7fdcbc004e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5e410 0x555860d6cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5e410 0x555860d6cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54522/0 (socket says 172.21.15.5:54522) 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdccaffd640 1 -- 172.21.15.5:0/3198773564 learned_addr learned my addr 172.21.15.5:0/3198773564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdccb7fe640 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d68cb0 0x555860de2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdcca7fc640 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d64c80 0x555860d6d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdccaffd640 1 -- 172.21.15.5:0/3198773564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d68cb0 msgr2=0x555860de2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdccaffd640 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d68cb0 0x555860de2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdccaffd640 1 -- 172.21.15.5:0/3198773564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d64c80 msgr2=0x555860d6d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdccaffd640 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d64c80 0x555860d6d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdccaffd640 1 -- 172.21.15.5:0/3198773564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555860d552d0 con 0x555860d5e410 2024-09-19T06:20:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdccb7fe640 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d68cb0 0x555860de2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdccaffd640 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5e410 0x555860d6cdf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0988f0 tx=0x7fdcbc0d3790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d770d4bc037d1fd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdcaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:20:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdcaeff5640 1 -- 172.21.15.5:0/3198773564 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc01c040 con 0x555860d5e410 2024-09-19T06:20:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdcaeff5640 1 -- 172.21.15.5:0/3198773564 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7fdcbc098590 con 0x555860d5e410 2024-09-19T06:20:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.505+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3198773564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc00043c0 con 0x555860d5e410 2024-09-19T06:20:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.506+0000 7fdcaeff5640 1 -- 172.21.15.5:0/3198773564 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc09b880 con 0x555860d5e410 2024-09-19T06:20:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.506+0000 7fdcd41d4740 10 client.29111 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.506+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3198773564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x555860d60940 con 0x555860d5e410 2024-09-19T06:20:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.506+0000 7fdcaeff5640 1 -- 172.21.15.5:0/3198773564 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc00043c0 con 0x555860d5e410 2024-09-19T06:20:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.506+0000 7fdcaeff5640 1 -- 172.21.15.5:0/3198773564 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7fdcbc081030 con 0x555860d5e410 2024-09-19T06:20:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.506+0000 7fdcd41d4740 10 client.29111 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.506+0000 7fdcd41d4740 10 client.29111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.506+0000 7fdcd41d4740 20 client.29111 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.506+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555860de4450 0x555860de6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.506+0000 7fdcd41d4740 4 client.29111 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:20:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.506+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3198773564 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555860cc9d70 con 0x555860de4450 2024-09-19T06:20:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.506+0000 7fdcca7fc640 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555860de4450 0x555860de6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.507+0000 7fdcaeff5640 10 client.29111 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.507+0000 7fdcca7fc640 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555860de4450 0x555860de6840 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bae3413a503b504 server_cookie=87d46b13f63486a7 in_seq=0 out_seq=0 2024-09-19T06:20:13.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 36431 8.86 MB/sec execute 97 sec latency 75.952 ms 2024-09-19T06:20:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.582+0000 7fdcaeff5640 1 -- 172.21.15.5:0/3198773564 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555860cc9d70 con 0x555860de4450 2024-09-19T06:20:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.582+0000 7fdcaeff5640 10 client.29111 handle_command_reply: tid=0 2024-09-19T06:20:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.582+0000 7fdcd41d4740 1 client.29111 shutdown 2024-09-19T06:20:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.582+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3198773564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5e410 msgr2=0x555860d6cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.582+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5e410 0x555860d6cdf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0988f0 tx=0x7fdcbc0d3790 comp rx=0 tx=0).stop 2024-09-19T06:20:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.583+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3198773564 shutdown_connections 2024-09-19T06:20:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.583+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x555860de4450 0x555860de6840 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.583+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860d68cb0 0x555860de2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.583+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555860d64c80 0x555860d6d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.583+0000 7fdcd41d4740 1 --2- 172.21.15.5:0/3198773564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555860d5e410 0x555860d6cdf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.583+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3198773564 >> 172.21.15.5:0/3198773564 conn(0x555860d5a060 msgr2=0x555860d591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.583+0000 7fdcaeff5640 0 client.29111 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.583+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3198773564 shutdown_connections 2024-09-19T06:20:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.583+0000 7fdcd41d4740 1 -- 172.21.15.5:0/3198773564 wait complete. 2024-09-19T06:20:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.583+0000 7fdcd41d4740 20 client.29111 trim_cache size 0 max 16384 2024-09-19T06:20:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:20:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:20:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:20:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:20:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.584+0000 7fdcd143c640 1 -- 172.21.15.5:0/3527759160 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdcb40a3430 msgr2=0x7fdcb40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.584+0000 7fdcd143c640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdcb40a3430 0x7fdcb40a58f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4051000 tx=0x7fdcc4054000 comp rx=0 tx=0).stop 2024-09-19T06:20:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.584+0000 7fdcd143c640 1 -- 172.21.15.5:0/3527759160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdccc101ac0 msgr2=0x7fdccc1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.584+0000 7fdcd143c640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdccc101ac0 0x7fdccc1982d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00e9f0 tx=0x7fdcbc00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.584+0000 7fdcd143c640 1 -- 172.21.15.5:0/3527759160 shutdown_connections 2024-09-19T06:20:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.584+0000 7fdcd143c640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdccc107ad0 0x7fdccc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.584+0000 7fdcd143c640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fdcb40a3430 0x7fdcb40a58f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.584+0000 7fdcd143c640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc102460 0x7fdccc198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.584+0000 7fdcd143c640 1 --2- 172.21.15.5:0/3527759160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdccc101ac0 0x7fdccc1982d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.584+0000 7fdcd143c640 1 -- 172.21.15.5:0/3527759160 >> 172.21.15.5:0/3527759160 conn(0x7fdccc0fd7a0 msgr2=0x7fdccc0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.584+0000 7fdcd143c640 1 -- 172.21.15.5:0/3527759160 shutdown_connections 2024-09-19T06:20:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:13.584+0000 7fdcd143c640 1 -- 172.21.15.5:0/3527759160 wait complete. 2024-09-19T06:20:14.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:20:14.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 36566 8.86 MB/sec execute 98 sec latency 75.169 ms 2024-09-19T06:20:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:20:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:14 smithi157 ceph-mon[35346]: pgmap v532: 97 pgs: 97 active+clean; 703 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 377 op/s 2024-09-19T06:20:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:14 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:20:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:14 smithi005 ceph-mon[30955]: pgmap v532: 97 pgs: 97 active+clean; 703 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 377 op/s 2024-09-19T06:20:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:14 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:20:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:14 smithi049 ceph-mon[34640]: pgmap v532: 97 pgs: 97 active+clean; 703 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 377 op/s 2024-09-19T06:20:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:14 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:20:15.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 36754 8.95 MB/sec execute 99 sec latency 74.533 ms 2024-09-19T06:20:16.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 36924 8.97 MB/sec execute 100 sec latency 153.727 ms 2024-09-19T06:20:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:16 smithi157 ceph-mon[35346]: pgmap v533: 97 pgs: 97 active+clean; 703 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 305 op/s 2024-09-19T06:20:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:16 smithi005 ceph-mon[30955]: pgmap v533: 97 pgs: 97 active+clean; 703 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 305 op/s 2024-09-19T06:20:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:16 smithi049 ceph-mon[34640]: pgmap v533: 97 pgs: 97 active+clean; 703 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 305 op/s 2024-09-19T06:20:17.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37038 8.95 MB/sec execute 101 sec latency 136.231 ms 2024-09-19T06:20:18.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37142 8.91 MB/sec execute 102 sec latency 131.070 ms 2024-09-19T06:20:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:18 smithi157 ceph-mon[35346]: pgmap v534: 97 pgs: 97 active+clean; 707 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 337 op/s 2024-09-19T06:20:18.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:18 smithi005 ceph-mon[30955]: pgmap v534: 97 pgs: 97 active+clean; 707 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 337 op/s 2024-09-19T06:20:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:18 smithi049 ceph-mon[34640]: pgmap v534: 97 pgs: 97 active+clean; 707 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 337 op/s 2024-09-19T06:20:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.179+0000 7ff1ff020640 1 -- 172.21.15.5:0/1730665565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f8102de0 msgr2=0x7ff1f81031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.179+0000 7ff1ff020640 1 --2- 172.21.15.5:0/1730665565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f8102de0 0x7ff1f81031c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009900 tx=0x7ff1e802a480 comp rx=0 tx=0).stop 2024-09-19T06:20:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.179+0000 7ff1ff020640 1 -- 172.21.15.5:0/1730665565 shutdown_connections 2024-09-19T06:20:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.179+0000 7ff1ff020640 1 --2- 172.21.15.5:0/1730665565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f810c180 0x7ff1f810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.179+0000 7ff1ff020640 1 --2- 172.21.15.5:0/1730665565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f8103700 0x7ff1f810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.179+0000 7ff1ff020640 1 --2- 172.21.15.5:0/1730665565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f8102de0 0x7ff1f81031c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.179+0000 7ff1ff020640 1 -- 172.21.15.5:0/1730665565 >> 172.21.15.5:0/1730665565 conn(0x7ff1f80fc7a0 msgr2=0x7ff1f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.179+0000 7ff1ff020640 1 -- 172.21.15.5:0/1730665565 shutdown_connections 2024-09-19T06:20:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.179+0000 7ff1ff020640 1 -- 172.21.15.5:0/1730665565 wait complete. 2024-09-19T06:20:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.180+0000 7ff1ff020640 1 Processor -- start 2024-09-19T06:20:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.180+0000 7ff1ff020640 1 -- start start 2024-09-19T06:20:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1ff020640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f8102de0 0x7ff1f819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1ff020640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f8103700 0x7ff1f819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1ff020640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f810c180 0x7ff1f819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1ff020640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8111e20 con 0x7ff1f8103700 2024-09-19T06:20:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1ff020640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8111ca0 con 0x7ff1f8102de0 2024-09-19T06:20:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1ff020640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8111fa0 con 0x7ff1f810c180 2024-09-19T06:20:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1fcdbc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f8102de0 0x7ff1f819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1fcdbc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f8102de0 0x7ff1f819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43222/0 (socket says 172.21.15.5:43222) 2024-09-19T06:20:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f8103700 0x7ff1f819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1fcdbc640 1 -- 172.21.15.5:0/3311453458 learned_addr learned my addr 172.21.15.5:0/3311453458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1fd5bd640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f810c180 0x7ff1f819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1fcdbc640 1 -- 172.21.15.5:0/3311453458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f810c180 msgr2=0x7ff1f819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1fcdbc640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f810c180 0x7ff1f819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1fcdbc640 1 -- 172.21.15.5:0/3311453458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f8103700 msgr2=0x7ff1f819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1fcdbc640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f8103700 0x7ff1f819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1fcdbc640 1 -- 172.21.15.5:0/3311453458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f819f5b0 con 0x7ff1f8102de0 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.181+0000 7ff1f7fff640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f8103700 0x7ff1f819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.182+0000 7ff1fcdbc640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f8102de0 0x7ff1f819a600 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005940 tx=0x7ff1e80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.182+0000 7ff1f5ffb640 1 -- 172.21.15.5:0/3311453458 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e803f030 con 0x7ff1f8102de0 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.182+0000 7ff1ff020640 1 -- 172.21.15.5:0/3311453458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f819f840 con 0x7ff1f8102de0 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.182+0000 7ff1ff020640 1 -- 172.21.15.5:0/3311453458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f81a7180 con 0x7ff1f8102de0 2024-09-19T06:20:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.182+0000 7ff1f5ffb640 1 -- 172.21.15.5:0/3311453458 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1e80044c0 con 0x7ff1f8102de0 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.183+0000 7ff1f5ffb640 1 -- 172.21.15.5:0/3311453458 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8035070 con 0x7ff1f8102de0 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.186+0000 7ff2013f9740 1 Processor -- start 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.186+0000 7ff2013f9740 1 -- start start 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.186+0000 7ff2013f9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261def2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.186+0000 7ff2013f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261def2dc0 0x56261defd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.186+0000 7ff2013f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 0x56261deffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.186+0000 7ff2013f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56261de582e0 con 0x56261def2dc0 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.186+0000 7ff2013f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56261ddffce0 con 0x56261defd680 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.186+0000 7ff2013f9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56261de542a0 con 0x56261def24a0 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.186+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 0x56261deffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.187+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 0x56261deffa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43232/0 (socket says 172.21.15.5:43232) 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.187+0000 7ff1f7fff640 1 -- 172.21.15.5:0/312296712 learned_addr learned my addr 172.21.15.5:0/312296712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.187+0000 7ff1fd5bd640 1 --2- 172.21.15.5:0/312296712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261def2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.187+0000 7ff1f7fff640 1 -- 172.21.15.5:0/312296712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 msgr2=0x56261def2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.187+0000 7ff1f7fff640 1 --2- 172.21.15.5:0/312296712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261def2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.187+0000 7ff1f7fff640 1 -- 172.21.15.5:0/312296712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261def2dc0 msgr2=0x56261defd140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.187+0000 7ff1f7fff640 1 --2- 172.21.15.5:0/312296712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261def2dc0 0x56261defd140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.187+0000 7ff1f7fff640 1 -- 172.21.15.5:0/312296712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56261dde4440 con 0x56261defd680 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.187+0000 7ff1f5ffb640 1 -- 172.21.15.5:0/3311453458 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff1e8042030 con 0x7ff1f8102de0 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.187+0000 7ff1f7fff640 1 --2- 172.21.15.5:0/312296712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 0x56261deffa70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0114f0 tx=0x7ff1ec0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919f1aef02c87700 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.187+0000 7ff1deffd640 1 -- 172.21.15.5:0/312296712 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec01e070 con 0x56261defd680 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.188+0000 7ff1deffd640 1 -- 172.21.15.5:0/312296712 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1ec00fa70 con 0x56261defd680 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.188+0000 7ff1deffd640 1 -- 172.21.15.5:0/312296712 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec019be0 con 0x56261defd680 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.188+0000 7ff1f5ffb640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff1e00a3430 0x7ff1e00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.188+0000 7ff1f5ffb640 1 -- 172.21.15.5:0/3311453458 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7ff1e80effc0 con 0x7ff1f8102de0 2024-09-19T06:20:19.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.188+0000 7ff1fd5bd640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff1e00a3430 0x7ff1e00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.189+0000 7ff2013f9740 1 -- 172.21.15.5:0/312296712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 msgr2=0x56261deffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.189+0000 7ff2013f9740 1 --2- 172.21.15.5:0/312296712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 0x56261deffa70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0114f0 tx=0x7ff1ec0119b0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.189+0000 7ff1fd5bd640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff1e00a3430 0x7ff1e00a58f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0005fd0 tx=0x7ff1f0009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.189+0000 7ff2013f9740 1 -- 172.21.15.5:0/312296712 shutdown_connections 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.189+0000 7ff2013f9740 1 --2- 172.21.15.5:0/312296712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 0x56261deffa70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.189+0000 7ff2013f9740 1 --2- 172.21.15.5:0/312296712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261def2dc0 0x56261defd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.189+0000 7ff2013f9740 1 --2- 172.21.15.5:0/312296712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261def2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.189+0000 7ff2013f9740 1 -- 172.21.15.5:0/312296712 >> 172.21.15.5:0/312296712 conn(0x56261deeea30 msgr2=0x56261deeee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.189+0000 7ff2013f9740 1 -- 172.21.15.5:0/312296712 shutdown_connections 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.189+0000 7ff2013f9740 1 -- 172.21.15.5:0/312296712 wait complete. 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.189+0000 7ff2013f9740 1 -- start start 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.190+0000 7ff2013f9740 1 Processor -- start 2024-09-19T06:20:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.190+0000 7ff2013f9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261defcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.190+0000 7ff2013f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 0x56261defa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.190+0000 7ff2013f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261defd680 0x56261defa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.190+0000 7ff1fcdbc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261defcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.190+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 0x56261defa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff1fcdbc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261defcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43040/0 (socket says 172.21.15.5:43040) 2024-09-19T06:20:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 0x56261defa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43242/0 (socket says 172.21.15.5:43242) 2024-09-19T06:20:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff1fcdbc640 1 -- 172.21.15.5:0/290245906 learned_addr learned my addr 172.21.15.5:0/290245906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff2013f9740 1 -- 172.21.15.5:0/290245906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56261df002f0 con 0x56261defd680 2024-09-19T06:20:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff2013f9740 1 -- 172.21.15.5:0/290245906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56261de13050 con 0x56261def2dc0 2024-09-19T06:20:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff2013f9740 1 -- 172.21.15.5:0/290245906 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56261df00470 con 0x56261def24a0 2024-09-19T06:20:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff2013f9740 10 client.0 _ll_register_callbacks cb 0x56261de53160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff1fd5bd640 1 --2- 172.21.15.5:0/290245906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261defd680 0x56261defa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff1fcdbc640 1 -- 172.21.15.5:0/290245906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 msgr2=0x56261defa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff1fcdbc640 1 --2- 172.21.15.5:0/290245906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 0x56261defa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff1fcdbc640 1 -- 172.21.15.5:0/290245906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261defd680 msgr2=0x56261defa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff1fcdbc640 1 --2- 172.21.15.5:0/290245906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261defd680 0x56261defa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff1fcdbc640 1 -- 172.21.15.5:0/290245906 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56261dbc93c0 con 0x56261def24a0 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff1fcdbc640 1 --2- 172.21.15.5:0/290245906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261defcff0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8006010 tx=0x7ff1e80bbd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6128a470e1e6c5ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff1d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.191+0000 7ff1d7fff640 1 -- 172.21.15.5:0/290245906 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8049040 con 0x56261def24a0 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff2013f9740 1 -- 172.21.15.5:0/290245906 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1ec00fa70 con 0x56261def24a0 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff1d7fff640 1 -- 172.21.15.5:0/290245906 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7ff1e80b89a0 con 0x56261def24a0 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff1d7fff640 1 -- 172.21.15.5:0/290245906 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7ff1e80c4ef0 con 0x56261def24a0 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff2013f9740 10 client.29059 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff2013f9740 1 -- 172.21.15.5:0/290245906 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56261df003a0 con 0x56261def24a0 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff1d7fff640 1 -- 172.21.15.5:0/290245906 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1ec00fa70 con 0x56261def24a0 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff1d7fff640 1 -- 172.21.15.5:0/290245906 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7ff1e80c0100 con 0x56261def24a0 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff2013f9740 10 client.29059 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff2013f9740 10 client.29059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff2013f9740 20 client.29059 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff2013f9740 1 --2- 172.21.15.5:0/290245906 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56261df0d0a0 0x56261df0f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff2013f9740 4 client.29059 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff2013f9740 1 -- 172.21.15.5:0/290245906 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56261de65850 con 0x56261df0d0a0 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.192+0000 7ff1f7fff640 1 --2- 172.21.15.5:0/290245906 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56261df0d0a0 0x56261df0f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.193+0000 7ff1f7fff640 1 --2- 172.21.15.5:0/290245906 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56261df0d0a0 0x56261df0f4d0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d128b2944a6c63 server_cookie=5f714a578fcbedf0 in_seq=0 out_seq=0 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.193+0000 7ff1d7fff640 10 client.29059 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff1d7fff640 1 -- 172.21.15.5:0/290245906 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56261de65850 con 0x56261df0d0a0 2024-09-19T06:20:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff1d7fff640 10 client.29059 handle_command_reply: tid=0 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff2013f9740 1 client.29059 shutdown 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff2013f9740 1 -- 172.21.15.5:0/290245906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 msgr2=0x56261defcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff2013f9740 1 --2- 172.21.15.5:0/290245906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261defcff0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8006010 tx=0x7ff1e80bbd50 comp rx=0 tx=0).stop 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff2013f9740 1 -- 172.21.15.5:0/290245906 shutdown_connections 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff2013f9740 1 --2- 172.21.15.5:0/290245906 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56261df0d0a0 0x56261df0f4d0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff2013f9740 1 --2- 172.21.15.5:0/290245906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261defd680 0x56261defa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff2013f9740 1 --2- 172.21.15.5:0/290245906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 0x56261defa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff2013f9740 1 --2- 172.21.15.5:0/290245906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261defcff0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff1d7fff640 0 client.29059 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff2013f9740 1 -- 172.21.15.5:0/290245906 >> 172.21.15.5:0/290245906 conn(0x56261deeea30 msgr2=0x56261def4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff2013f9740 1 -- 172.21.15.5:0/290245906 shutdown_connections 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.194+0000 7ff2013f9740 1 -- 172.21.15.5:0/290245906 wait complete. 2024-09-19T06:20:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.195+0000 7ff2013f9740 20 client.29059 trim_cache size 0 max 16384 2024-09-19T06:20:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.221+0000 7ff2013f9740 1 Processor -- start 2024-09-19T06:20:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.221+0000 7ff2013f9740 1 -- start start 2024-09-19T06:20:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.221+0000 7ff2013f9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261df09550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.221+0000 7ff2013f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 0x56261df14d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.221+0000 7ff2013f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261defd680 0x56261df152b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.221+0000 7ff2013f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81afa00 con 0x56261defd680 2024-09-19T06:20:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.221+0000 7ff2013f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81b1570 con 0x56261def2dc0 2024-09-19T06:20:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.221+0000 7ff2013f9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81b0100 con 0x56261def24a0 2024-09-19T06:20:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.221+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 0x56261df14d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.221+0000 7ff1fd5bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261defd680 0x56261df152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.221+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 0x56261df14d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43252/0 (socket says 172.21.15.5:43252) 2024-09-19T06:20:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.221+0000 7ff1f7fff640 1 -- 172.21.15.5:0/1638242501 learned_addr learned my addr 172.21.15.5:0/1638242501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.222+0000 7ff1fcdbc640 1 --2- 172.21.15.5:0/1638242501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261df09550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.222+0000 7ff1f7fff640 1 -- 172.21.15.5:0/1638242501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 msgr2=0x56261df09550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.222+0000 7ff1f7fff640 1 --2- 172.21.15.5:0/1638242501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261df09550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.222+0000 7ff1f7fff640 1 -- 172.21.15.5:0/1638242501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261defd680 msgr2=0x56261df152b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.222+0000 7ff1f7fff640 1 --2- 172.21.15.5:0/1638242501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261defd680 0x56261df152b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.222+0000 7ff1f7fff640 1 -- 172.21.15.5:0/1638242501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56261deee4d0 con 0x56261def2dc0 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.222+0000 7ff1fd5bd640 1 --2- 172.21.15.5:0/1638242501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261defd680 0x56261df152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.222+0000 7ff1f7fff640 1 --2- 172.21.15.5:0/1638242501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 0x56261df14d70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0097e0 tx=0x7ff1ec005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=823e292b750b3891 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.222+0000 7ff1de7fc640 1 -- 172.21.15.5:0/1638242501 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec01e040 con 0x56261def2dc0 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.222+0000 7ff1de7fc640 1 -- 172.21.15.5:0/1638242501 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1ec00fa70 con 0x56261def2dc0 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.222+0000 7ff1de7fc640 1 -- 172.21.15.5:0/1638242501 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec005610 con 0x56261def2dc0 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.223+0000 7ff2013f9740 1 -- 172.21.15.5:0/1638242501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 msgr2=0x56261df14d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.223+0000 7ff2013f9740 1 --2- 172.21.15.5:0/1638242501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 0x56261df14d70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0097e0 tx=0x7ff1ec005fd0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.223+0000 7ff2013f9740 1 -- 172.21.15.5:0/1638242501 shutdown_connections 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.223+0000 7ff2013f9740 1 --2- 172.21.15.5:0/1638242501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261defd680 0x56261df152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.223+0000 7ff2013f9740 1 --2- 172.21.15.5:0/1638242501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261def2dc0 0x56261df14d70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.223+0000 7ff2013f9740 1 --2- 172.21.15.5:0/1638242501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261df09550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.223+0000 7ff2013f9740 1 -- 172.21.15.5:0/1638242501 >> 172.21.15.5:0/1638242501 conn(0x56261deeea30 msgr2=0x56261def9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.224+0000 7ff2013f9740 1 -- 172.21.15.5:0/1638242501 shutdown_connections 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.224+0000 7ff2013f9740 1 -- 172.21.15.5:0/1638242501 wait complete. 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.224+0000 7ff2013f9740 1 -- start start 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.224+0000 7ff2013f9740 1 Processor -- start 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.225+0000 7ff2013f9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261df05c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.225+0000 7ff2013f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261def2dc0 0x56261df02f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.225+0000 7ff2013f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 0x56261df034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.225+0000 7ff2013f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81b0ea0 con 0x56261def2dc0 2024-09-19T06:20:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.225+0000 7ff2013f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81b07d0 con 0x56261defd680 2024-09-19T06:20:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.225+0000 7ff2013f9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81ada00 con 0x56261def24a0 2024-09-19T06:20:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.225+0000 7ff2013f9740 10 client.0 _ll_register_callbacks cb 0x7ff1e8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.225+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261def2dc0 0x56261df02f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.225+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261def2dc0 0x56261df02f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51220/0 (socket says 172.21.15.5:51220) 2024-09-19T06:20:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.225+0000 7ff1f7fff640 1 -- 172.21.15.5:0/479377479 learned_addr learned my addr 172.21.15.5:0/479377479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.225+0000 7ff1fcdbc640 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261df05c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.225+0000 7ff1fd5bd640 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 0x56261df034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.226+0000 7ff1f7fff640 1 -- 172.21.15.5:0/479377479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 msgr2=0x56261df05c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.226+0000 7ff1f7fff640 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261df05c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.226+0000 7ff1f7fff640 1 -- 172.21.15.5:0/479377479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 msgr2=0x56261df034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.226+0000 7ff1f7fff640 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 0x56261df034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.226+0000 7ff1f7fff640 1 -- 172.21.15.5:0/479377479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56261df03a90 con 0x56261def2dc0 2024-09-19T06:20:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.226+0000 7ff1fcdbc640 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261df05c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:20:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.226+0000 7ff1fd5bd640 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 0x56261df034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.226+0000 7ff1f7fff640 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261def2dc0 0x56261df02f60 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec011eb0 tx=0x7ff1ec0195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593d57daafb1d871 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.226+0000 7ff1d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:20:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.226+0000 7ff1d7fff640 1 -- 172.21.15.5:0/479377479 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec01e040 con 0x56261def2dc0 2024-09-19T06:20:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.226+0000 7ff1d7fff640 1 -- 172.21.15.5:0/479377479 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7ff1ec00fca0 con 0x56261def2dc0 2024-09-19T06:20:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.226+0000 7ff2013f9740 1 -- 172.21.15.5:0/479377479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1ec00fa70 con 0x56261def2dc0 2024-09-19T06:20:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.227+0000 7ff1d7fff640 1 -- 172.21.15.5:0/479377479 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7ff1ec00fea0 con 0x56261def2dc0 2024-09-19T06:20:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.227+0000 7ff2013f9740 10 client.19752 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.227+0000 7ff2013f9740 1 -- 172.21.15.5:0/479377479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56261df80370 con 0x56261def2dc0 2024-09-19T06:20:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.227+0000 7ff1d7fff640 1 -- 172.21.15.5:0/479377479 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1ec00fa70 con 0x56261def2dc0 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.227+0000 7ff1d7fff640 1 -- 172.21.15.5:0/479377479 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7ff1ec02e070 con 0x56261def2dc0 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.227+0000 7ff2013f9740 10 client.19752 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.227+0000 7ff2013f9740 10 client.19752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.227+0000 7ff2013f9740 20 client.19752 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.227+0000 7ff2013f9740 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56261df80860 0x56261df82c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.227+0000 7ff2013f9740 4 client.19752 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.227+0000 7ff2013f9740 1 -- 172.21.15.5:0/479377479 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56261de5fd60 con 0x56261df80860 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.227+0000 7ff1fcdbc640 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56261df80860 0x56261df82c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.228+0000 7ff1fcdbc640 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56261df80860 0x56261df82c80 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c27fc5c59d5663 server_cookie=73706ae4e533d3da in_seq=0 out_seq=0 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.228+0000 7ff1d7fff640 10 client.19752 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.228+0000 7ff1d7fff640 1 -- 172.21.15.5:0/479377479 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56261de5fd60 con 0x56261df80860 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.228+0000 7ff1d7fff640 10 client.19752 handle_command_reply: tid=0 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.228+0000 7ff2013f9740 1 client.19752 shutdown 2024-09-19T06:20:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.229+0000 7ff2013f9740 1 -- 172.21.15.5:0/479377479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261def2dc0 msgr2=0x56261df02f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.229+0000 7ff2013f9740 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261def2dc0 0x56261df02f60 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec011eb0 tx=0x7ff1ec0195e0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.229+0000 7ff2013f9740 1 -- 172.21.15.5:0/479377479 shutdown_connections 2024-09-19T06:20:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.229+0000 7ff2013f9740 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56261df80860 0x56261df82c80 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.229+0000 7ff2013f9740 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261defd680 0x56261df034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.229+0000 7ff1d7fff640 0 client.19752 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.229+0000 7ff2013f9740 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261def2dc0 0x56261df02f60 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.229+0000 7ff2013f9740 1 --2- 172.21.15.5:0/479377479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56261def24a0 0x56261df05c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.229+0000 7ff2013f9740 1 -- 172.21.15.5:0/479377479 >> 172.21.15.5:0/479377479 conn(0x56261deeea30 msgr2=0x56261deed350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:19.147 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:20:19.148 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:20:19.148 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:20:19.148 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:20:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.229+0000 7ff2013f9740 1 -- 172.21.15.5:0/479377479 shutdown_connections 2024-09-19T06:20:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.229+0000 7ff2013f9740 1 -- 172.21.15.5:0/479377479 wait complete. 2024-09-19T06:20:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.229+0000 7ff2013f9740 20 client.19752 trim_cache size 0 max 16384 2024-09-19T06:20:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.230+0000 7ff1ff020640 1 -- 172.21.15.5:0/3311453458 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff1e00a3430 msgr2=0x7ff1e00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.230+0000 7ff1ff020640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff1e00a3430 0x7ff1e00a58f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0005fd0 tx=0x7ff1f0009500 comp rx=0 tx=0).stop 2024-09-19T06:20:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.230+0000 7ff1ff020640 1 -- 172.21.15.5:0/3311453458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f8102de0 msgr2=0x7ff1f819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.230+0000 7ff1ff020640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f8102de0 0x7ff1f819a600 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005940 tx=0x7ff1e80040c0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.230+0000 7ff1ff020640 1 -- 172.21.15.5:0/3311453458 shutdown_connections 2024-09-19T06:20:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.230+0000 7ff1ff020640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff1e00a3430 0x7ff1e00a58f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.230+0000 7ff1ff020640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f810c180 0x7ff1f819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.230+0000 7ff1ff020640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f8103700 0x7ff1f819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.230+0000 7ff1ff020640 1 --2- 172.21.15.5:0/3311453458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f8102de0 0x7ff1f819a600 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.230+0000 7ff1ff020640 1 -- 172.21.15.5:0/3311453458 >> 172.21.15.5:0/3311453458 conn(0x7ff1f80fc7a0 msgr2=0x7ff1f8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.230+0000 7ff1ff020640 1 -- 172.21.15.5:0/3311453458 shutdown_connections 2024-09-19T06:20:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:18.230+0000 7ff1ff020640 1 -- 172.21.15.5:0/3311453458 wait complete. 2024-09-19T06:20:19.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37228 8.83 MB/sec execute 103 sec latency 144.182 ms 2024-09-19T06:20:20.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37378 8.84 MB/sec execute 104 sec latency 144.361 ms 2024-09-19T06:20:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:20 smithi157 ceph-mon[35346]: pgmap v535: 97 pgs: 97 active+clean; 707 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 323 op/s 2024-09-19T06:20:20.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:20 smithi005 ceph-mon[30955]: pgmap v535: 97 pgs: 97 active+clean; 707 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 323 op/s 2024-09-19T06:20:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:20 smithi049 ceph-mon[34640]: pgmap v535: 97 pgs: 97 active+clean; 707 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 323 op/s 2024-09-19T06:20:21.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37526 8.79 MB/sec execute 105 sec latency 102.803 ms 2024-09-19T06:20:22.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37656 8.73 MB/sec execute 106 sec latency 162.437 ms 2024-09-19T06:20:22.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:22 smithi005 ceph-mon[30955]: pgmap v536: 97 pgs: 97 active+clean; 707 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 281 op/s 2024-09-19T06:20:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:22 smithi157 ceph-mon[35346]: pgmap v536: 97 pgs: 97 active+clean; 707 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 281 op/s 2024-09-19T06:20:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:22 smithi049 ceph-mon[34640]: pgmap v536: 97 pgs: 97 active+clean; 707 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 281 op/s 2024-09-19T06:20:23.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37801 8.70 MB/sec execute 107 sec latency 148.247 ms 2024-09-19T06:20:23.611 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:20:23.611 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:20:23.996 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:20:24.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37953 8.70 MB/sec execute 108 sec latency 71.126 ms 2024-09-19T06:20:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.563+0000 7f6093f33640 1 -- 172.21.15.5:0/2490536295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608c102570 msgr2=0x7f608c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.563+0000 7f6093f33640 1 --2- 172.21.15.5:0/2490536295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608c102570 0x7f608c1029b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f607c009900 tx=0x7f607c02a480 comp rx=0 tx=0).stop 2024-09-19T06:20:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.563+0000 7f6093f33640 1 -- 172.21.15.5:0/2490536295 shutdown_connections 2024-09-19T06:20:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.563+0000 7f6093f33640 1 --2- 172.21.15.5:0/2490536295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c102ef0 0x7f608c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.563+0000 7f6093f33640 1 --2- 172.21.15.5:0/2490536295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608c102570 0x7f608c1029b0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.563+0000 7f6093f33640 1 --2- 172.21.15.5:0/2490536295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c100e20 0x7f608c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.563+0000 7f6093f33640 1 -- 172.21.15.5:0/2490536295 >> 172.21.15.5:0/2490536295 conn(0x7f608c0fc7d0 msgr2=0x7f608c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.563+0000 7f6093f33640 1 -- 172.21.15.5:0/2490536295 shutdown_connections 2024-09-19T06:20:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.563+0000 7f6093f33640 1 -- 172.21.15.5:0/2490536295 wait complete. 2024-09-19T06:20:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.564+0000 7f6093f33640 1 Processor -- start 2024-09-19T06:20:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.564+0000 7f6093f33640 1 -- start start 2024-09-19T06:20:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.564+0000 7f6093f33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c100e20 0x7f608c196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f6093f33640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c102570 0x7f608c196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f6093f33640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608c102ef0 0x7f608c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f6093f33640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f608c10fd30 con 0x7f608c102ef0 2024-09-19T06:20:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f6093f33640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f608c10fbb0 con 0x7f608c102570 2024-09-19T06:20:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f6093f33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f608c10feb0 con 0x7f608c100e20 2024-09-19T06:20:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f60914ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c102570 0x7f608c196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f6091ccf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c100e20 0x7f608c196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f60914ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c102570 0x7f608c196770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58316/0 (socket says 172.21.15.5:58316) 2024-09-19T06:20:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f6091ccf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c100e20 0x7f608c196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34500/0 (socket says 172.21.15.5:34500) 2024-09-19T06:20:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f60914ce640 1 -- 172.21.15.5:0/1287369273 learned_addr learned my addr 172.21.15.5:0/1287369273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f60924d0640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608c102ef0 0x7f608c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f60924d0640 1 -- 172.21.15.5:0/1287369273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c100e20 msgr2=0x7f608c196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f60924d0640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c100e20 0x7f608c196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f60924d0640 1 -- 172.21.15.5:0/1287369273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c102570 msgr2=0x7f608c196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f60924d0640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c102570 0x7f608c196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.565+0000 7f60924d0640 1 -- 172.21.15.5:0/1287369273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608c19b1e0 con 0x7f608c102ef0 2024-09-19T06:20:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.566+0000 7f60924d0640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608c102ef0 0x7f608c19ab00 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f6084009fd0 tx=0x7f608400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.566+0000 7f6082ffd640 1 -- 172.21.15.5:0/1287369273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608400ce20 con 0x7f608c102ef0 2024-09-19T06:20:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.566+0000 7f6082ffd640 1 -- 172.21.15.5:0/1287369273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6084005020 con 0x7f608c102ef0 2024-09-19T06:20:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.566+0000 7f6093f33640 1 -- 172.21.15.5:0/1287369273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608c19b4d0 con 0x7f608c102ef0 2024-09-19T06:20:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.566+0000 7f6082ffd640 1 -- 172.21.15.5:0/1287369273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6084010690 con 0x7f608c102ef0 2024-09-19T06:20:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.567+0000 7f6093f33640 1 -- 172.21.15.5:0/1287369273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f608c1a2db0 con 0x7f608c102ef0 2024-09-19T06:20:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.568+0000 7f6093f33640 1 -- 172.21.15.5:0/1287369273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f608c101200 con 0x7f608c102ef0 2024-09-19T06:20:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.569+0000 7f6082ffd640 1 -- 172.21.15.5:0/1287369273 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f60840051d0 con 0x7f608c102ef0 2024-09-19T06:20:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.570+0000 7f6082ffd640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60780a7840 0x7f60780a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.571+0000 7f6091ccf640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60780a7840 0x7f60780a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.571+0000 7f6082ffd640 1 -- 172.21.15.5:0/1287369273 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f60840d0380 con 0x7f608c102ef0 2024-09-19T06:20:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.571+0000 7f6091ccf640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60780a7840 0x7f60780a9d00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f60880095f0 tx=0x7f6088005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:24.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.576+0000 7f6082ffd640 1 -- 172.21.15.5:0/1287369273 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f608409cd40 con 0x7f608c102ef0 2024-09-19T06:20:24.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:24 smithi049 ceph-mon[34640]: pgmap v537: 97 pgs: 97 active+clean; 712 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.6 MiB/s wr, 386 op/s 2024-09-19T06:20:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:24 smithi157 ceph-mon[35346]: pgmap v537: 97 pgs: 97 active+clean; 712 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.6 MiB/s wr, 386 op/s 2024-09-19T06:20:24.833 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:24 smithi005 ceph-mon[30955]: pgmap v537: 97 pgs: 97 active+clean; 712 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.6 MiB/s wr, 386 op/s 2024-09-19T06:20:24.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.935+0000 7f6093f33640 1 -- 172.21.15.5:0/1287369273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f608c109090 con 0x7f608c102ef0 2024-09-19T06:20:24.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.936+0000 7f6082ffd640 1 -- 172.21.15.5:0/1287369273 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11960 (secure 0 0 0) 0x7f6084098020 con 0x7f608c102ef0 2024-09-19T06:20:24.937 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:20:24.938 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":57,"btime":"2024-09-19T06:20:13:435014+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:20:12.962963+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24448,24442,24463,24437]},"id":1}]} 2024-09-19T06:20:24.938 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 57 2024-09-19T06:20:24.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.938+0000 7f6093f33640 1 -- 172.21.15.5:0/1287369273 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60780a7840 msgr2=0x7f60780a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.939+0000 7f6093f33640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60780a7840 0x7f60780a9d00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f60880095f0 tx=0x7f6088005f90 comp rx=0 tx=0).stop 2024-09-19T06:20:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.939+0000 7f6093f33640 1 -- 172.21.15.5:0/1287369273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608c102ef0 msgr2=0x7f608c19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.939+0000 7f6093f33640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608c102ef0 0x7f608c19ab00 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f6084009fd0 tx=0x7f608400ef10 comp rx=0 tx=0).stop 2024-09-19T06:20:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.939+0000 7f6093f33640 1 -- 172.21.15.5:0/1287369273 shutdown_connections 2024-09-19T06:20:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.939+0000 7f6093f33640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608c102ef0 0x7f608c19ab00 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.939+0000 7f6093f33640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c102570 0x7f608c196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.939+0000 7f6093f33640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60780a7840 0x7f60780a9d00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.939+0000 7f6093f33640 1 --2- 172.21.15.5:0/1287369273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c100e20 0x7f608c196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.939+0000 7f6093f33640 1 -- 172.21.15.5:0/1287369273 >> 172.21.15.5:0/1287369273 conn(0x7f608c0fc7d0 msgr2=0x7f608c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.939+0000 7f6093f33640 1 -- 172.21.15.5:0/1287369273 shutdown_connections 2024-09-19T06:20:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:24.939+0000 7f6093f33640 1 -- 172.21.15.5:0/1287369273 wait complete. 2024-09-19T06:20:25.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 38114 8.69 MB/sec execute 109 sec latency 76.320 ms 2024-09-19T06:20:25.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:20:25.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:25 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1287369273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:20:25.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:25 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1287369273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:20:25.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:25 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1287369273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:20:26.071 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:20:26.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 38294 8.71 MB/sec execute 110 sec latency 72.231 ms 2024-09-19T06:20:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.715+0000 7f07cda82640 1 -- 172.21.15.5:0/877563432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c81035e0 msgr2=0x7f07c810bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.715+0000 7f07cda82640 1 --2- 172.21.15.5:0/877563432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c81035e0 0x7f07c810bac0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009900 tx=0x7f07bc02a410 comp rx=0 tx=0).stop 2024-09-19T06:20:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.715+0000 7f07cda82640 1 -- 172.21.15.5:0/877563432 shutdown_connections 2024-09-19T06:20:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.715+0000 7f07cda82640 1 --2- 172.21.15.5:0/877563432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c810c130 0x7f07c810e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.715+0000 7f07cda82640 1 --2- 172.21.15.5:0/877563432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c81035e0 0x7f07c810bac0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.715+0000 7f07cda82640 1 --2- 172.21.15.5:0/877563432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8102c30 0x7f07c8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.715+0000 7f07cda82640 1 -- 172.21.15.5:0/877563432 >> 172.21.15.5:0/877563432 conn(0x7f07c80fc6f0 msgr2=0x7f07c80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.715+0000 7f07cda82640 1 -- 172.21.15.5:0/877563432 shutdown_connections 2024-09-19T06:20:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.715+0000 7f07cda82640 1 -- 172.21.15.5:0/877563432 wait complete. 2024-09-19T06:20:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.716+0000 7f07cda82640 1 Processor -- start 2024-09-19T06:20:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.717+0000 7f07cda82640 1 -- start start 2024-09-19T06:20:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.717+0000 7f07cda82640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8102c30 0x7f07c8100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.717+0000 7f07cda82640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c81035e0 0x7f07c8100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.717+0000 7f07cda82640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c810c130 0x7f07c8101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.717+0000 7f07cda82640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8111d90 con 0x7f07c810c130 2024-09-19T06:20:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.717+0000 7f07cda82640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8111c10 con 0x7f07c8102c30 2024-09-19T06:20:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.717+0000 7f07cda82640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8111f10 con 0x7f07c81035e0 2024-09-19T06:20:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.717+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8102c30 0x7f07c8100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.717+0000 7f07c6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c81035e0 0x7f07c8100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.717+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c810c130 0x7f07c8101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.717+0000 7f07c6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c81035e0 0x7f07c8100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34518/0 (socket says 172.21.15.5:34518) 2024-09-19T06:20:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.717+0000 7f07c6ffd640 1 -- 172.21.15.5:0/487233796 learned_addr learned my addr 172.21.15.5:0/487233796 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.718+0000 7f07c77fe640 1 -- 172.21.15.5:0/487233796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c81035e0 msgr2=0x7f07c8100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.718+0000 7f07c77fe640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c81035e0 0x7f07c8100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.718+0000 7f07c77fe640 1 -- 172.21.15.5:0/487233796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c810c130 msgr2=0x7f07c8101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.718+0000 7f07c77fe640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c810c130 0x7f07c8101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.718+0000 7f07c77fe640 1 -- 172.21.15.5:0/487233796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c81aacc0 con 0x7f07c8102c30 2024-09-19T06:20:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.718+0000 7f07c6ffd640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c81035e0 0x7f07c8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.718+0000 7f07c77fe640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8102c30 0x7f07c8100910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e9f0 tx=0x7f07b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.718+0000 7f07c4ff9640 1 -- 172.21.15.5:0/487233796 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b800ccc0 con 0x7f07c8102c30 2024-09-19T06:20:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.718+0000 7f07c4ff9640 1 -- 172.21.15.5:0/487233796 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07b80108e0 con 0x7f07c8102c30 2024-09-19T06:20:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.718+0000 7f07c4ff9640 1 -- 172.21.15.5:0/487233796 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8018880 con 0x7f07c8102c30 2024-09-19T06:20:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.718+0000 7f07cda82640 1 -- 172.21.15.5:0/487233796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c81aaef0 con 0x7f07c8102c30 2024-09-19T06:20:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.718+0000 7f07cda82640 1 -- 172.21.15.5:0/487233796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c81ab360 con 0x7f07c8102c30 2024-09-19T06:20:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.721+0000 7f07c4ff9640 1 -- 172.21.15.5:0/487233796 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f07b8010a90 con 0x7f07c8102c30 2024-09-19T06:20:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07d0820740 1 Processor -- start 2024-09-19T06:20:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07d0820740 1 -- start start 2024-09-19T06:20:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07d0820740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 0x559a3041eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07d0820740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 0x559a30429770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07d0820740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a3042c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07d0820740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a303832f0 con 0x559a3041eaf0 2024-09-19T06:20:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07d0820740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559a3032b270 con 0x559a30429cb0 2024-09-19T06:20:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07d0820740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559a3037f2b0 con 0x559a3041f410 2024-09-19T06:20:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 0x559a3041eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 0x559a3041eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41158/0 (socket says 172.21.15.5:41158) 2024-09-19T06:20:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 0x559a30429770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07c7fff640 1 -- 172.21.15.5:0/3989706517 learned_addr learned my addr 172.21.15.5:0/3989706517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07c6ffd640 1 --2- 172.21.15.5:0/3989706517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a3042c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.722+0000 7f07c4ff9640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f07b00a33b0 0x7f07b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.723+0000 7f07c77fe640 1 -- 172.21.15.5:0/3989706517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 msgr2=0x559a3042c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.723+0000 7f07c77fe640 1 --2- 172.21.15.5:0/3989706517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a3042c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.723+0000 7f07c77fe640 1 -- 172.21.15.5:0/3989706517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 msgr2=0x559a3041eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.723+0000 7f07c77fe640 1 --2- 172.21.15.5:0/3989706517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 0x559a3041eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.723+0000 7f07c77fe640 1 -- 172.21.15.5:0/3989706517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a3032a340 con 0x559a3041f410 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.723+0000 7f07c7fff640 1 --2- 172.21.15.5:0/3989706517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 0x559a3041eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.723+0000 7f07c7fff640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f07b00a33b0 0x7f07b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.723+0000 7f07c4ff9640 1 -- 172.21.15.5:0/487233796 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f07b80d0090 con 0x7f07c8102c30 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.723+0000 7f07c77fe640 1 --2- 172.21.15.5:0/3989706517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 0x559a30429770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f07b80cd7c0 tx=0x7f07b8091890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a2807280b3df33b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.723+0000 7f07adffb640 1 -- 172.21.15.5:0/3989706517 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8026030 con 0x559a3041f410 2024-09-19T06:20:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.723+0000 7f07adffb640 1 -- 172.21.15.5:0/3989706517 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07b8010e00 con 0x559a3041f410 2024-09-19T06:20:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.723+0000 7f07adffb640 1 -- 172.21.15.5:0/3989706517 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80a0700 con 0x559a3041f410 2024-09-19T06:20:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.724+0000 7f07d0820740 1 -- 172.21.15.5:0/3989706517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 msgr2=0x559a30429770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.724+0000 7f07c7fff640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f07b00a33b0 0x7f07b00a5870 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f07c00107d0 tx=0x7f07c0004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.724+0000 7f07d0820740 1 --2- 172.21.15.5:0/3989706517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 0x559a30429770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f07b80cd7c0 tx=0x7f07b8091890 comp rx=0 tx=0).stop 2024-09-19T06:20:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.724+0000 7f07d0820740 1 -- 172.21.15.5:0/3989706517 shutdown_connections 2024-09-19T06:20:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.724+0000 7f07d0820740 1 --2- 172.21.15.5:0/3989706517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a3042c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.724+0000 7f07d0820740 1 --2- 172.21.15.5:0/3989706517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 0x559a30429770 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.724+0000 7f07d0820740 1 --2- 172.21.15.5:0/3989706517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 0x559a3041eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.724+0000 7f07d0820740 1 -- 172.21.15.5:0/3989706517 >> 172.21.15.5:0/3989706517 conn(0x559a3041b060 msgr2=0x559a3041b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.724+0000 7f07d0820740 1 -- 172.21.15.5:0/3989706517 shutdown_connections 2024-09-19T06:20:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.724+0000 7f07d0820740 1 -- 172.21.15.5:0/3989706517 wait complete. 2024-09-19T06:20:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.725+0000 7f07d0820740 1 -- start start 2024-09-19T06:20:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.725+0000 7f07d0820740 1 Processor -- start 2024-09-19T06:20:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.725+0000 7f07d0820740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 0x559a30426be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.725+0000 7f07d0820740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 0x559a30427120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.725+0000 7f07d0820740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a30427660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.725+0000 7f07d0820740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a3042c880 con 0x559a3041eaf0 2024-09-19T06:20:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.725+0000 7f07d0820740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559a3030c290 con 0x559a30429cb0 2024-09-19T06:20:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.725+0000 7f07d0820740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559a3042ca00 con 0x559a3041f410 2024-09-19T06:20:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.725+0000 7f07d0820740 10 client.0 _ll_register_callbacks cb 0x559a3033d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 0x559a30426be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 0x559a30426be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41172/0 (socket says 172.21.15.5:41172) 2024-09-19T06:20:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07c77fe640 1 -- 172.21.15.5:0/4095957798 learned_addr learned my addr 172.21.15.5:0/4095957798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07c7fff640 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a30427660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07c77fe640 1 -- 172.21.15.5:0/4095957798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 msgr2=0x559a30427120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07c77fe640 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 0x559a30427120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07c77fe640 1 -- 172.21.15.5:0/4095957798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 msgr2=0x559a30427660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07c77fe640 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a30427660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07c77fe640 1 -- 172.21.15.5:0/4095957798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a30096210 con 0x559a3041eaf0 2024-09-19T06:20:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07c7fff640 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a30427660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:20:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07c77fe640 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 0x559a30426be0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f07b80cddc0 tx=0x7f07b809a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9b95377654e997 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:20:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.726+0000 7f07a6ffd640 1 -- 172.21.15.5:0/4095957798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b806af00 con 0x559a3041eaf0 2024-09-19T06:20:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.727+0000 7f07d0820740 1 -- 172.21.15.5:0/4095957798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a30390a20 con 0x559a3041eaf0 2024-09-19T06:20:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.727+0000 7f07a6ffd640 1 -- 172.21.15.5:0/4095957798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f07b80a4b30 con 0x559a3041eaf0 2024-09-19T06:20:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.727+0000 7f07a6ffd640 1 -- 172.21.15.5:0/4095957798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f07b80a4fc0 con 0x559a3041eaf0 2024-09-19T06:20:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.727+0000 7f07d0820740 10 client.19770 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.727+0000 7f07d0820740 1 -- 172.21.15.5:0/4095957798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559a30162620 con 0x559a3041eaf0 2024-09-19T06:20:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.727+0000 7f07a6ffd640 1 -- 172.21.15.5:0/4095957798 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559a30390a20 con 0x559a3041eaf0 2024-09-19T06:20:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.728+0000 7f07a6ffd640 1 -- 172.21.15.5:0/4095957798 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f07b809c6f0 con 0x559a3041eaf0 2024-09-19T06:20:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.728+0000 7f07d0820740 10 client.19770 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.728+0000 7f07d0820740 10 client.19770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.728+0000 7f07d0820740 20 client.19770 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.728+0000 7f07d0820740 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559a30439da0 0x559a3043c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.728+0000 7f07d0820740 4 client.19770 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.728+0000 7f07d0820740 1 -- 172.21.15.5:0/4095957798 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a3038ad70 con 0x559a30439da0 2024-09-19T06:20:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.728+0000 7f07c6ffd640 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559a30439da0 0x559a3043c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.729+0000 7f07a6ffd640 10 client.19770 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.729+0000 7f07c6ffd640 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559a30439da0 0x559a3043c1b0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=746623ce69dac4d5 server_cookie=34d49f0412d5bca5 in_seq=0 out_seq=0 2024-09-19T06:20:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.730+0000 7f07a6ffd640 1 -- 172.21.15.5:0/4095957798 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a3038ad70 con 0x559a30439da0 2024-09-19T06:20:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.730+0000 7f07a6ffd640 10 client.19770 handle_command_reply: tid=0 2024-09-19T06:20:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.730+0000 7f07d0820740 1 client.19770 shutdown 2024-09-19T06:20:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.731+0000 7f07d0820740 1 -- 172.21.15.5:0/4095957798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 msgr2=0x559a30426be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.731+0000 7f07d0820740 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 0x559a30426be0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f07b80cddc0 tx=0x7f07b809a910 comp rx=0 tx=0).stop 2024-09-19T06:20:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.731+0000 7f07d0820740 1 -- 172.21.15.5:0/4095957798 shutdown_connections 2024-09-19T06:20:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.731+0000 7f07d0820740 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559a30439da0 0x559a3043c1b0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.731+0000 7f07d0820740 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a30427660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.731+0000 7f07d0820740 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 0x559a30427120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.731+0000 7f07d0820740 1 --2- 172.21.15.5:0/4095957798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041eaf0 0x559a30426be0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.731+0000 7f07d0820740 1 -- 172.21.15.5:0/4095957798 >> 172.21.15.5:0/4095957798 conn(0x559a3041b060 msgr2=0x559a30420dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.731+0000 7f07a6ffd640 0 client.19770 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.731+0000 7f07d0820740 1 -- 172.21.15.5:0/4095957798 shutdown_connections 2024-09-19T06:20:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.731+0000 7f07d0820740 1 -- 172.21.15.5:0/4095957798 wait complete. 2024-09-19T06:20:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.731+0000 7f07d0820740 20 client.19770 trim_cache size 0 max 16384 2024-09-19T06:20:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.756+0000 7f07d0820740 1 Processor -- start 2024-09-19T06:20:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.756+0000 7f07d0820740 1 -- start start 2024-09-19T06:20:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.756+0000 7f07d0820740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041eaf0 0x559a30421560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07d0820740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041f410 0x559a30427490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07d0820740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a30428b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07d0820740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b3ed0 con 0x559a3041f410 2024-09-19T06:20:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07d0820740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b5a40 con 0x559a30429cb0 2024-09-19T06:20:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07d0820740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b45d0 con 0x559a3041eaf0 2024-09-19T06:20:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041f410 0x559a30427490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041f410 0x559a30427490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41184/0 (socket says 172.21.15.5:41184) 2024-09-19T06:20:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c6ffd640 1 -- 172.21.15.5:0/4011330228 learned_addr learned my addr 172.21.15.5:0/4011330228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c7fff640 1 --2- 172.21.15.5:0/4011330228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a30428b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c77fe640 1 --2- 172.21.15.5:0/4011330228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041eaf0 0x559a30421560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c7fff640 1 -- 172.21.15.5:0/4011330228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041eaf0 msgr2=0x559a30421560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c7fff640 1 --2- 172.21.15.5:0/4011330228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041eaf0 0x559a30421560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c7fff640 1 -- 172.21.15.5:0/4011330228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041f410 msgr2=0x559a30427490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c7fff640 1 --2- 172.21.15.5:0/4011330228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041f410 0x559a30427490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c7fff640 1 -- 172.21.15.5:0/4011330228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a304162d0 con 0x559a30429cb0 2024-09-19T06:20:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c77fe640 1 --2- 172.21.15.5:0/4011330228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041eaf0 0x559a30421560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c6ffd640 1 --2- 172.21.15.5:0/4011330228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041f410 0x559a30427490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.757+0000 7f07c7fff640 1 --2- 172.21.15.5:0/4011330228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a30428b50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f07c000f790 tx=0x7f07c0018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f9651ab8d7d940 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.758+0000 7f07ad7fa640 1 -- 172.21.15.5:0/4011330228 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c001f600 con 0x559a30429cb0 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.758+0000 7f07ad7fa640 1 -- 172.21.15.5:0/4011330228 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07c001fc50 con 0x559a30429cb0 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.758+0000 7f07ad7fa640 1 -- 172.21.15.5:0/4011330228 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c001cc60 con 0x559a30429cb0 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.759+0000 7f07d0820740 1 -- 172.21.15.5:0/4011330228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 msgr2=0x559a30428b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.759+0000 7f07d0820740 1 --2- 172.21.15.5:0/4011330228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a30428b50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f07c000f790 tx=0x7f07c0018000 comp rx=0 tx=0).stop 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.759+0000 7f07d0820740 1 -- 172.21.15.5:0/4011330228 shutdown_connections 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.759+0000 7f07d0820740 1 --2- 172.21.15.5:0/4011330228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a30429cb0 0x559a30428b50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.759+0000 7f07d0820740 1 --2- 172.21.15.5:0/4011330228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a3041f410 0x559a30427490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.759+0000 7f07d0820740 1 --2- 172.21.15.5:0/4011330228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041eaf0 0x559a30421560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.759+0000 7f07d0820740 1 -- 172.21.15.5:0/4011330228 >> 172.21.15.5:0/4011330228 conn(0x559a3041b060 msgr2=0x559a30428440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.759+0000 7f07d0820740 1 -- 172.21.15.5:0/4011330228 shutdown_connections 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.759+0000 7f07d0820740 1 -- 172.21.15.5:0/4011330228 wait complete. 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.759+0000 7f07d0820740 1 -- start start 2024-09-19T06:20:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07d0820740 1 Processor -- start 2024-09-19T06:20:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07d0820740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a3041eaf0 0x559a30424880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07d0820740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 0x559a30431c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07d0820740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a30429cb0 0x559a30432420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07d0820740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b5370 con 0x559a30429cb0 2024-09-19T06:20:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07d0820740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b4ca0 con 0x559a3041eaf0 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07d0820740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b1ed0 con 0x559a3041f410 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a30429cb0 0x559a30432420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a3041eaf0 0x559a30424880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a3041eaf0 0x559a30424880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58358/0 (socket says 172.21.15.5:58358) 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a30429cb0 0x559a30432420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41186/0 (socket says 172.21.15.5:41186) 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07c77fe640 1 -- 172.21.15.5:0/197139278 learned_addr learned my addr 172.21.15.5:0/197139278 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07c6ffd640 1 --2- 172.21.15.5:0/197139278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 0x559a30431c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.760+0000 7f07d0820740 10 client.0 _ll_register_callbacks cb 0x7f07c0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07c77fe640 1 -- 172.21.15.5:0/197139278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 msgr2=0x559a30431c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07c77fe640 1 --2- 172.21.15.5:0/197139278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 0x559a30431c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07c77fe640 1 -- 172.21.15.5:0/197139278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a30429cb0 msgr2=0x559a30432420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07c77fe640 1 --2- 172.21.15.5:0/197139278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a30429cb0 0x559a30432420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07c77fe640 1 -- 172.21.15.5:0/197139278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a30422eb0 con 0x559a3041eaf0 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07c77fe640 1 --2- 172.21.15.5:0/197139278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a3041eaf0 0x559a30424880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f07b8000c00 tx=0x7f07b80a4d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a386cf954133f4ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07a6ffd640 1 -- 172.21.15.5:0/197139278 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80a0700 con 0x559a3041eaf0 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07a6ffd640 1 -- 172.21.15.5:0/197139278 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f07b80bc110 con 0x559a3041eaf0 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07d0820740 1 -- 172.21.15.5:0/197139278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07c001fc50 con 0x559a3041eaf0 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07a6ffd640 1 -- 172.21.15.5:0/197139278 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f07b80bc310 con 0x559a3041eaf0 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07d0820740 10 client.29153 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.761+0000 7f07d0820740 1 -- 172.21.15.5:0/197139278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559a304a3d80 con 0x559a3041eaf0 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.762+0000 7f07a6ffd640 1 -- 172.21.15.5:0/197139278 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07c001fc50 con 0x559a3041eaf0 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.762+0000 7f07a6ffd640 1 -- 172.21.15.5:0/197139278 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f07b809b2c0 con 0x559a3041eaf0 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.762+0000 7f07d0820740 10 client.29153 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.762+0000 7f07d0820740 10 client.29153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.762+0000 7f07d0820740 20 client.29153 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.762+0000 7f07d0820740 1 --2- 172.21.15.5:0/197139278 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559a304a4170 0x559a304a6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.762+0000 7f07d0820740 4 client.29153 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:20:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.762+0000 7f07d0820740 1 -- 172.21.15.5:0/197139278 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a30307900 con 0x559a304a4170 2024-09-19T06:20:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.762+0000 7f07c6ffd640 1 --2- 172.21.15.5:0/197139278 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559a304a4170 0x559a304a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.763+0000 7f07c6ffd640 1 --2- 172.21.15.5:0/197139278 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559a304a4170 0x559a304a6610 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39d42dcb4ac51df1 server_cookie=d7906dccb7f311dd in_seq=0 out_seq=0 2024-09-19T06:20:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.763+0000 7f07a6ffd640 10 client.29153 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:26.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:26 smithi005 ceph-mon[30955]: pgmap v538: 97 pgs: 97 active+clean; 712 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-09-19T06:20:26.797 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:26 smithi049 ceph-mon[34640]: pgmap v538: 97 pgs: 97 active+clean; 712 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-09-19T06:20:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:26 smithi157 ceph-mon[35346]: pgmap v538: 97 pgs: 97 active+clean; 712 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-09-19T06:20:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f07a6ffd640 1 -- 172.21.15.5:0/197139278 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a30307900 con 0x559a304a4170 2024-09-19T06:20:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f07a6ffd640 10 client.29153 handle_command_reply: tid=0 2024-09-19T06:20:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f07d0820740 1 client.29153 shutdown 2024-09-19T06:20:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f07d0820740 1 -- 172.21.15.5:0/197139278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a3041eaf0 msgr2=0x559a30424880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f07d0820740 1 --2- 172.21.15.5:0/197139278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a3041eaf0 0x559a30424880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f07b8000c00 tx=0x7f07b80a4d60 comp rx=0 tx=0).stop 2024-09-19T06:20:26.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1125304714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f98100100 msgr2=0x7f9f98100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f9f9d3c6640 1 --2- 172.21.15.5:0/1125304714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f98100100 0x7f9f98100560 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009900 tx=0x7f9f8402a410 comp rx=0 tx=0).stop 2024-09-19T06:20:26.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1125304714 shutdown_connections 2024-09-19T06:20:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.827+0000 7f07d0820740 1 -- 172.21.15.5:0/197139278 shutdown_connections 2024-09-19T06:20:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.827+0000 7f07d0820740 1 --2- 172.21.15.5:0/197139278 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x559a304a4170 0x559a304a6610 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.827+0000 7f07d0820740 1 --2- 172.21.15.5:0/197139278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a30429cb0 0x559a30432420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.827+0000 7f07d0820740 1 --2- 172.21.15.5:0/197139278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a3041f410 0x559a30431c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f9f9d3c6640 1 --2- 172.21.15.5:0/1125304714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f98100aa0 0x7f9f9810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.827+0000 7f07d0820740 1 --2- 172.21.15.5:0/197139278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a3041eaf0 0x559a30424880 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.827+0000 7f07d0820740 1 -- 172.21.15.5:0/197139278 >> 172.21.15.5:0/197139278 conn(0x559a3041b060 msgr2=0x559a3041a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f9f9d3c6640 1 --2- 172.21.15.5:0/1125304714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f98100100 0x7f9f98100560 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.827+0000 7f07a6ffd640 0 client.29153 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.827+0000 7f07d0820740 1 -- 172.21.15.5:0/197139278 shutdown_connections 2024-09-19T06:20:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f9f9d3c6640 1 --2- 172.21.15.5:0/1125304714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f98106b80 0x7f9f98106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1125304714 >> 172.21.15.5:0/1125304714 conn(0x7f9f98077230 msgr2=0x7f9f980fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1125304714 shutdown_connections 2024-09-19T06:20:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.826+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1125304714 wait complete. 2024-09-19T06:20:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:20:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:20:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:20:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:20:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.827+0000 7f07d0820740 1 -- 172.21.15.5:0/197139278 wait complete. 2024-09-19T06:20:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.827+0000 7f07d0820740 20 client.29153 trim_cache size 0 max 16384 2024-09-19T06:20:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.827+0000 7f9f9d3c6640 1 Processor -- start 2024-09-19T06:20:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f9f9d3c6640 1 -- start start 2024-09-19T06:20:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f07cda82640 1 -- 172.21.15.5:0/487233796 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f07b00a33b0 msgr2=0x7f07b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f07cda82640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f07b00a33b0 0x7f07b00a5870 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f07c00107d0 tx=0x7f07c0004a40 comp rx=0 tx=0).stop 2024-09-19T06:20:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f9f9d3c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f98100100 0x7f9f981161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f9f9d3c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f98100aa0 0x7f9f98116730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f07cda82640 1 -- 172.21.15.5:0/487233796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8102c30 msgr2=0x7f07c8100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f07cda82640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8102c30 0x7f07c8100910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e9f0 tx=0x7f07b800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f9f9d3c6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f98106b80 0x7f9f9811bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f9f9d3c6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98102e30 con 0x7f9f98106b80 2024-09-19T06:20:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f9f9d3c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98102cb0 con 0x7f9f98100aa0 2024-09-19T06:20:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f9f9d3c6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98102fb0 con 0x7f9f98100100 2024-09-19T06:20:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f07cda82640 1 -- 172.21.15.5:0/487233796 shutdown_connections 2024-09-19T06:20:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f07cda82640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c810c130 0x7f07c8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f07cda82640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f07b00a33b0 0x7f07b00a5870 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f07cda82640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c81035e0 0x7f07c8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f07cda82640 1 --2- 172.21.15.5:0/487233796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8102c30 0x7f07c8100910 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f07cda82640 1 -- 172.21.15.5:0/487233796 >> 172.21.15.5:0/487233796 conn(0x7f07c80fc6f0 msgr2=0x7f07c80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f9f967fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f98100aa0 0x7f9f98116730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f9f967fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f98100aa0 0x7f9f98116730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58376/0 (socket says 172.21.15.5:58376) 2024-09-19T06:20:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f9f967fc640 1 -- 172.21.15.5:0/1996657394 learned_addr learned my addr 172.21.15.5:0/1996657394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f9f96ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f98100100 0x7f9f981161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f9f977fe640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f98106b80 0x7f9f9811bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f07cda82640 1 -- 172.21.15.5:0/487233796 shutdown_connections 2024-09-19T06:20:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:26.828+0000 7f07cda82640 1 -- 172.21.15.5:0/487233796 wait complete. 2024-09-19T06:20:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.829+0000 7f9f96ffd640 1 -- 172.21.15.5:0/1996657394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f98100aa0 msgr2=0x7f9f98116730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.829+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f98100aa0 0x7f9f98116730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.829+0000 7f9f96ffd640 1 -- 172.21.15.5:0/1996657394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f98106b80 msgr2=0x7f9f9811bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.829+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f98106b80 0x7f9f9811bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.829+0000 7f9f96ffd640 1 -- 172.21.15.5:0/1996657394 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9811c7d0 con 0x7f9f98100100 2024-09-19T06:20:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.829+0000 7f9f977fe640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f98106b80 0x7f9f9811bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.829+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f98100100 0x7f9f981161f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800c970 tx=0x7f9f8800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.829+0000 7f9f8ffff640 1 -- 172.21.15.5:0/1996657394 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88007bb0 con 0x7f9f98100100 2024-09-19T06:20:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.829+0000 7f9f8ffff640 1 -- 172.21.15.5:0/1996657394 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f88007d50 con 0x7f9f98100100 2024-09-19T06:20:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.829+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1996657394 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9811cac0 con 0x7f9f98100100 2024-09-19T06:20:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.829+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1996657394 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f981114e0 con 0x7f9f98100100 2024-09-19T06:20:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.830+0000 7f9f8ffff640 1 -- 172.21.15.5:0/1996657394 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88004e40 con 0x7f9f98100100 2024-09-19T06:20:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.833+0000 7f9f8ffff640 1 -- 172.21.15.5:0/1996657394 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9f88004fe0 con 0x7f9f98100100 2024-09-19T06:20:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.833+0000 7f9f9f79f740 1 Processor -- start 2024-09-19T06:20:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.834+0000 7f9f9f79f740 1 -- start start 2024-09-19T06:20:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.834+0000 7f9f8ffff640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9f7c0a33b0 0x7f9f7c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f977fe640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9f7c0a33b0 0x7f9f7c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f8ffff640 1 -- 172.21.15.5:0/1996657394 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f9f880cf730 con 0x7f9f98100100 2024-09-19T06:20:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f9f79f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4ad6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f9f79f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ad65d0 0x55efd4ad6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f9f79f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ae1370 0x55efd4ae3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f9f79f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55efd4a3c2f0 con 0x55efd4ad7b40 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f9f79f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55efd49e3cf0 con 0x55efd4ad65d0 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f9f79f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55efd4a382b0 con 0x55efd4ae1370 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f977fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ae1370 0x55efd4ae3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4ad6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4ad6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41210/0 (socket says 172.21.15.5:41210) 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f96ffd640 1 -- 172.21.15.5:0/254504607 learned_addr learned my addr 172.21.15.5:0/254504607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.835+0000 7f9f967fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ad65d0 0x55efd4ad6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.836+0000 7f9f967fc640 1 -- 172.21.15.5:0/254504607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ae1370 msgr2=0x55efd4ae3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.836+0000 7f9f967fc640 1 --2- 172.21.15.5:0/254504607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ae1370 0x55efd4ae3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.836+0000 7f9f967fc640 1 -- 172.21.15.5:0/254504607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 msgr2=0x55efd4ad6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.836+0000 7f9f977fe640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9f7c0a33b0 0x7f9f7c0a5870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9f980ff880 tx=0x7f9f90011d20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:26.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.836+0000 7f9f967fc640 1 --2- 172.21.15.5:0/254504607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4ad6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.836+0000 7f9f967fc640 1 -- 172.21.15.5:0/254504607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efd49c83a0 con 0x55efd4ad65d0 2024-09-19T06:20:26.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.836+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/254504607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4ad6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:26.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.836+0000 7f9f967fc640 1 --2- 172.21.15.5:0/254504607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ad65d0 0x55efd4ad6a10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f84004770 tx=0x7f9f84031c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31f62e536aded52f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:26.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.837+0000 7f9f8d7fa640 1 -- 172.21.15.5:0/254504607 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84040070 con 0x55efd4ad65d0 2024-09-19T06:20:26.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.837+0000 7f9f8d7fa640 1 -- 172.21.15.5:0/254504607 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f840316a0 con 0x55efd4ad65d0 2024-09-19T06:20:26.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.837+0000 7f9f8d7fa640 1 -- 172.21.15.5:0/254504607 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84035040 con 0x55efd4ad65d0 2024-09-19T06:20:26.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.838+0000 7f9f9f79f740 1 -- 172.21.15.5:0/254504607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ad65d0 msgr2=0x55efd4ad6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.838+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/254504607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ad65d0 0x55efd4ad6a10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f84004770 tx=0x7f9f84031c20 comp rx=0 tx=0).stop 2024-09-19T06:20:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.838+0000 7f9f9f79f740 1 -- 172.21.15.5:0/254504607 shutdown_connections 2024-09-19T06:20:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.838+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/254504607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ae1370 0x55efd4ae3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.838+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/254504607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ad65d0 0x55efd4ad6a10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.838+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/254504607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4ad6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.838+0000 7f9f9f79f740 1 -- 172.21.15.5:0/254504607 >> 172.21.15.5:0/254504607 conn(0x55efd4a9f310 msgr2=0x55efd4ad2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.838+0000 7f9f9f79f740 1 -- 172.21.15.5:0/254504607 shutdown_connections 2024-09-19T06:20:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.838+0000 7f9f9f79f740 1 -- 172.21.15.5:0/254504607 wait complete. 2024-09-19T06:20:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f9f79f740 1 -- start start 2024-09-19T06:20:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f9f79f740 1 Processor -- start 2024-09-19T06:20:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f9f79f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4ae0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f9f79f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4addf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f9f79f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4ade480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f9f79f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55efd4ae4010 con 0x55efd4ad7b40 2024-09-19T06:20:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f9f79f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55efd49f7060 con 0x55efd4ae1370 2024-09-19T06:20:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f9f79f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55efd4ae4190 con 0x55efd4ad65d0 2024-09-19T06:20:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f96ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4ae0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f9f79f740 10 client.0 _ll_register_callbacks cb 0x55efd474a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f96ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4ae0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34586/0 (socket says 172.21.15.5:34586) 2024-09-19T06:20:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f96ffd640 1 -- 172.21.15.5:0/3853291540 learned_addr learned my addr 172.21.15.5:0/3853291540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.839+0000 7f9f967fc640 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4addf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.840+0000 7f9f977fe640 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4ade480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.840+0000 7f9f967fc640 1 -- 172.21.15.5:0/3853291540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 msgr2=0x55efd4ae0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.840+0000 7f9f967fc640 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4ae0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.840+0000 7f9f967fc640 1 -- 172.21.15.5:0/3853291540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 msgr2=0x55efd4ade480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.840+0000 7f9f967fc640 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4ade480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.840+0000 7f9f967fc640 1 -- 172.21.15.5:0/3853291540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efd47ad3c0 con 0x55efd4ad7b40 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.840+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4ae0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.840+0000 7f9f977fe640 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4ade480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.840+0000 7f9f967fc640 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4addf40 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9f84004770 tx=0x7f9f84005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3ae158a059f58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.840+0000 7f9f7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.841+0000 7f9f7a7fc640 1 -- 172.21.15.5:0/3853291540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8404c060 con 0x55efd4ad7b40 2024-09-19T06:20:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.841+0000 7f9f7a7fc640 1 -- 172.21.15.5:0/3853291540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f9f84004240 con 0x55efd4ad7b40 2024-09-19T06:20:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.841+0000 7f9f9f79f740 1 -- 172.21.15.5:0/3853291540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f840316a0 con 0x55efd4ad7b40 2024-09-19T06:20:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.841+0000 7f9f7a7fc640 1 -- 172.21.15.5:0/3853291540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f9f84035030 con 0x55efd4ad7b40 2024-09-19T06:20:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.841+0000 7f9f9f79f740 10 client.19788 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.841+0000 7f9f9f79f740 1 -- 172.21.15.5:0/3853291540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55efd4aefa30 con 0x55efd4ad7b40 2024-09-19T06:20:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.841+0000 7f9f7a7fc640 1 -- 172.21.15.5:0/3853291540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f840316a0 con 0x55efd4ad7b40 2024-09-19T06:20:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.842+0000 7f9f7a7fc640 1 -- 172.21.15.5:0/3853291540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f9f840047a0 con 0x55efd4ad7b40 2024-09-19T06:20:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.842+0000 7f9f9f79f740 10 client.19788 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.842+0000 7f9f9f79f740 10 client.19788 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:20:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.842+0000 7f9f9f79f740 20 client.19788 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.842+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55efd4af0d80 0x55efd4af3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.842+0000 7f9f9f79f740 4 client.19788 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.842+0000 7f9f9f79f740 1 -- 172.21.15.5:0/3853291540 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efd4a49860 con 0x55efd4af0d80 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.842+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55efd4af0d80 0x55efd4af3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.843+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55efd4af0d80 0x55efd4af3170 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c625e0b3a8f22a20 server_cookie=625978f2b6c2bab9 in_seq=0 out_seq=0 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.843+0000 7f9f7a7fc640 10 client.19788 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.843+0000 7f9f7a7fc640 1 -- 172.21.15.5:0/3853291540 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55efd4a49860 con 0x55efd4af0d80 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.843+0000 7f9f7a7fc640 10 client.19788 handle_command_reply: tid=0 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.843+0000 7f9f9f79f740 1 client.19788 shutdown 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.844+0000 7f9f9f79f740 1 -- 172.21.15.5:0/3853291540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 msgr2=0x55efd4addf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.844+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4addf40 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9f84004770 tx=0x7f9f84005500 comp rx=0 tx=0).stop 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.844+0000 7f9f9f79f740 1 -- 172.21.15.5:0/3853291540 shutdown_connections 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.844+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55efd4af0d80 0x55efd4af3170 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.844+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4ade480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.844+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4addf40 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.844+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/3853291540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4ae0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.844+0000 7f9f9f79f740 1 -- 172.21.15.5:0/3853291540 >> 172.21.15.5:0/3853291540 conn(0x55efd4a9f310 msgr2=0x55efd4ad8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.844+0000 7f9f7a7fc640 0 client.19788 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:20:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.844+0000 7f9f9f79f740 1 -- 172.21.15.5:0/3853291540 shutdown_connections 2024-09-19T06:20:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.844+0000 7f9f9f79f740 1 -- 172.21.15.5:0/3853291540 wait complete. 2024-09-19T06:20:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.844+0000 7f9f9f79f740 20 client.19788 trim_cache size 0 max 16384 2024-09-19T06:20:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.864+0000 7f9f9f79f740 1 Processor -- start 2024-09-19T06:20:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.864+0000 7f9f9f79f740 1 -- start start 2024-09-19T06:20:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.864+0000 7f9f9f79f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4aed310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.864+0000 7f9f9f79f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4adcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.864+0000 7f9f9f79f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4ae2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.864+0000 7f9f9f79f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8000e8a0 con 0x55efd4ad7b40 2024-09-19T06:20:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f9f79f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80010410 con 0x55efd4ae1370 2024-09-19T06:20:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f9f79f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8000efa0 con 0x55efd4ad65d0 2024-09-19T06:20:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4adcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f96ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4aed310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f96ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4aed310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34602/0 (socket says 172.21.15.5:34602) 2024-09-19T06:20:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4adcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41228/0 (socket says 172.21.15.5:41228) 2024-09-19T06:20:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f96ffd640 1 -- 172.21.15.5:0/1225514110 learned_addr learned my addr 172.21.15.5:0/1225514110 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f977fe640 1 --2- 172.21.15.5:0/1225514110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4ae2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f96ffd640 1 -- 172.21.15.5:0/1225514110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 msgr2=0x55efd4ae2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/1225514110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4ae2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f96ffd640 1 -- 172.21.15.5:0/1225514110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 msgr2=0x55efd4adcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/1225514110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4adcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f96ffd640 1 -- 172.21.15.5:0/1225514110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efd4ad2ed0 con 0x55efd4ad65d0 2024-09-19T06:20:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.865+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/1225514110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4aed310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9f8809ff60 tx=0x7f9f88086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acaa92bc32675ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.866+0000 7f9f8cff9640 1 -- 172.21.15.5:0/1225514110 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8801d040 con 0x55efd4ad65d0 2024-09-19T06:20:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.866+0000 7f9f8cff9640 1 -- 172.21.15.5:0/1225514110 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f88099e60 con 0x55efd4ad65d0 2024-09-19T06:20:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.866+0000 7f9f8cff9640 1 -- 172.21.15.5:0/1225514110 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f880977f0 con 0x55efd4ad65d0 2024-09-19T06:20:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.867+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1225514110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 msgr2=0x55efd4aed310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.867+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/1225514110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4aed310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9f8809ff60 tx=0x7f9f88086040 comp rx=0 tx=0).stop 2024-09-19T06:20:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.867+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1225514110 shutdown_connections 2024-09-19T06:20:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.867+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/1225514110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4ae2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.867+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/1225514110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4adcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.867+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/1225514110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4aed310 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.867+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1225514110 >> 172.21.15.5:0/1225514110 conn(0x55efd4a9f310 msgr2=0x55efd4adc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.867+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1225514110 shutdown_connections 2024-09-19T06:20:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.868+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1225514110 wait complete. 2024-09-19T06:20:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.868+0000 7f9f9f79f740 1 -- start start 2024-09-19T06:20:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.868+0000 7f9f9f79f740 1 Processor -- start 2024-09-19T06:20:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.868+0000 7f9f9f79f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4ae98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.868+0000 7f9f9f79f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4af7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f9f79f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4af8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f96ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4ae98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f9f79f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8000fd40 con 0x55efd4ad7b40 2024-09-19T06:20:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f96ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4ae98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34616/0 (socket says 172.21.15.5:34616) 2024-09-19T06:20:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f96ffd640 1 -- 172.21.15.5:0/1283615069 learned_addr learned my addr 172.21.15.5:0/1283615069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f967fc640 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4af7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f977fe640 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4af8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1283615069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8000f670 con 0x55efd4ae1370 2024-09-19T06:20:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1283615069 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8000c8a0 con 0x55efd4ad65d0 2024-09-19T06:20:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f9f79f740 10 client.0 _ll_register_callbacks cb 0x7f9f88007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f977fe640 1 -- 172.21.15.5:0/1283615069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 msgr2=0x55efd4ae98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f977fe640 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4ae98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f977fe640 1 -- 172.21.15.5:0/1283615069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 msgr2=0x55efd4af7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f977fe640 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4af7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.869+0000 7f9f977fe640 1 -- 172.21.15.5:0/1283615069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efd4af8780 con 0x55efd4ae1370 2024-09-19T06:20:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.870+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4ae98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.870+0000 7f9f977fe640 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4af8240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9f90011fd0 tx=0x7f9f90004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c672803ff4e0921 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.870+0000 7f9f7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:20:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.870+0000 7f9f7a7fc640 1 -- 172.21.15.5:0/1283615069 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9001e630 con 0x55efd4ae1370 2024-09-19T06:20:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.870+0000 7f9f7a7fc640 1 -- 172.21.15.5:0/1283615069 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f9f9001e7d0 con 0x55efd4ae1370 2024-09-19T06:20:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.870+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1283615069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55efd4a43d70 con 0x55efd4ae1370 2024-09-19T06:20:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.870+0000 7f9f7a7fc640 1 -- 172.21.15.5:0/1283615069 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f9f90035050 con 0x55efd4ae1370 2024-09-19T06:20:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.870+0000 7f9f9f79f740 10 client.29168 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.870+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1283615069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55efd4ae90b0 con 0x55efd4ae1370 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.871+0000 7f9f7a7fc640 1 -- 172.21.15.5:0/1283615069 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f9002c070 con 0x55efd4ae1370 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.871+0000 7f9f7a7fc640 1 -- 172.21.15.5:0/1283615069 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f9f900279c0 con 0x55efd4ae1370 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.871+0000 7f9f9f79f740 10 client.29168 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.871+0000 7f9f9f79f740 10 client.29168 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.871+0000 7f9f9f79f740 20 client.29168 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.871+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55efd4b5bad0 0x55efd4b5dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.871+0000 7f9f9f79f740 4 client.29168 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.871+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1283615069 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efd4a43d70 con 0x55efd4b5bad0 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.871+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55efd4b5bad0 0x55efd4b5dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f96ffd640 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55efd4b5bad0 0x55efd4b5dec0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb5debb858c8eded server_cookie=5c8f5f1c1f1e7948 in_seq=0 out_seq=0 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f7a7fc640 10 client.29168 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f7a7fc640 1 -- 172.21.15.5:0/1283615069 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efd4a43d70 con 0x55efd4b5bad0 2024-09-19T06:20:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f7a7fc640 10 client.29168 handle_command_reply: tid=0 2024-09-19T06:20:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f9f79f740 1 client.29168 shutdown 2024-09-19T06:20:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1283615069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 msgr2=0x55efd4af8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4af8240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9f90011fd0 tx=0x7f9f90004ab0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1283615069 shutdown_connections 2024-09-19T06:20:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55efd4b5bad0 0x55efd4b5dec0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efd4ae1370 0x55efd4af8240 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd4ad7b40 0x55efd4af7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f9f79f740 1 --2- 172.21.15.5:0/1283615069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efd4ad65d0 0x55efd4ae98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1283615069 >> 172.21.15.5:0/1283615069 conn(0x55efd4a9f310 msgr2=0x55efd4ad14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f7a7fc640 0 client.29168 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:20:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.872+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1283615069 shutdown_connections 2024-09-19T06:20:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.873+0000 7f9f9f79f740 1 -- 172.21.15.5:0/1283615069 wait complete. 2024-09-19T06:20:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.873+0000 7f9f9f79f740 20 client.29168 trim_cache size 0 max 16384 2024-09-19T06:20:26.878 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:20:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.873+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1996657394 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9f7c0a33b0 msgr2=0x7f9f7c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.873+0000 7f9f9d3c6640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9f7c0a33b0 0x7f9f7c0a5870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9f980ff880 tx=0x7f9f90011d20 comp rx=0 tx=0).stop 2024-09-19T06:20:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.873+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1996657394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f98100100 msgr2=0x7f9f981161f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.873+0000 7f9f9d3c6640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f98100100 0x7f9f981161f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800c970 tx=0x7f9f8800ce30 comp rx=0 tx=0).stop 2024-09-19T06:20:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.873+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1996657394 shutdown_connections 2024-09-19T06:20:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.874+0000 7f9f9d3c6640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f98106b80 0x7f9f9811bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.874+0000 7f9f9d3c6640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f98100aa0 0x7f9f98116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.874+0000 7f9f9d3c6640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9f7c0a33b0 0x7f9f7c0a5870 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.874+0000 7f9f9d3c6640 1 --2- 172.21.15.5:0/1996657394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f98100100 0x7f9f981161f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.874+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1996657394 >> 172.21.15.5:0/1996657394 conn(0x7f9f98077230 msgr2=0x7f9f98104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.874+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1996657394 shutdown_connections 2024-09-19T06:20:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:26.874+0000 7f9f9d3c6640 1 -- 172.21.15.5:0/1996657394 wait complete. 2024-09-19T06:20:27.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 38472 8.73 MB/sec execute 111 sec latency 101.920 ms 2024-09-19T06:20:27.655 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:20:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:20:28.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 38627 8.69 MB/sec execute 112 sec latency 82.993 ms 2024-09-19T06:20:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.634+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/2398997056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef01011a0 msgr2=0x7f5ef0101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.634+0000 7f5ef7ec4640 1 --2- 172.21.15.5:0/2398997056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef01011a0 0x7f5ef0101600 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009900 tx=0x7f5ee002a410 comp rx=0 tx=0).stop 2024-09-19T06:20:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.634+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/2398997056 shutdown_connections 2024-09-19T06:20:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.634+0000 7f5ef7ec4640 1 --2- 172.21.15.5:0/2398997056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0101b40 0x7f5ef010da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.634+0000 7f5ef7ec4640 1 --2- 172.21.15.5:0/2398997056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef01011a0 0x7f5ef0101600 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.634+0000 7f5ef7ec4640 1 --2- 172.21.15.5:0/2398997056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ef0106c30 0x7f5ef0107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.634+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/2398997056 >> 172.21.15.5:0/2398997056 conn(0x7f5ef0077210 msgr2=0x7f5ef00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.634+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/2398997056 shutdown_connections 2024-09-19T06:20:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.634+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/2398997056 wait complete. 2024-09-19T06:20:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.635+0000 7f5ef7ec4640 1 Processor -- start 2024-09-19T06:20:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.635+0000 7f5ef7ec4640 1 -- start start 2024-09-19T06:20:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.635+0000 7f5ef7ec4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ef01011a0 0x7f5ef01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.635+0000 7f5ef7ec4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0101b40 0x7f5ef0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.635+0000 7f5ef7ec4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef0106c30 0x7f5ef019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.635+0000 7f5ef7ec4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0111fd0 con 0x7f5ef0106c30 2024-09-19T06:20:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.635+0000 7f5ef7ec4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0111e50 con 0x7f5ef01011a0 2024-09-19T06:20:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.635+0000 7f5ef7ec4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0112150 con 0x7f5ef0101b40 2024-09-19T06:20:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.636+0000 7f5ef545f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0101b40 0x7f5ef0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.636+0000 7f5ef6461640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef0106c30 0x7f5ef019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.636+0000 7f5ef545f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0101b40 0x7f5ef0198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34620/0 (socket says 172.21.15.5:34620) 2024-09-19T06:20:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.636+0000 7f5ef545f640 1 -- 172.21.15.5:0/283806353 learned_addr learned my addr 172.21.15.5:0/283806353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.636+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ef01011a0 0x7f5ef01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.636+0000 7f5ef5c60640 1 -- 172.21.15.5:0/283806353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0101b40 msgr2=0x7f5ef0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.636+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0101b40 0x7f5ef0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.636+0000 7f5ef5c60640 1 -- 172.21.15.5:0/283806353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef0106c30 msgr2=0x7f5ef019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.636+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef0106c30 0x7f5ef019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.636+0000 7f5ef5c60640 1 -- 172.21.15.5:0/283806353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef019d480 con 0x7f5ef01011a0 2024-09-19T06:20:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.636+0000 7f5ef545f640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0101b40 0x7f5ef0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.636+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ef01011a0 0x7f5ef01984d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00e9f0 tx=0x7f5eec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.637+0000 7f5ee6ffd640 1 -- 172.21.15.5:0/283806353 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec00cd80 con 0x7f5ef01011a0 2024-09-19T06:20:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.637+0000 7f5ee6ffd640 1 -- 172.21.15.5:0/283806353 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5eec0035f0 con 0x7f5ef01011a0 2024-09-19T06:20:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.637+0000 7f5ee6ffd640 1 -- 172.21.15.5:0/283806353 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec010670 con 0x7f5ef01011a0 2024-09-19T06:20:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.637+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/283806353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef019d770 con 0x7f5ef01011a0 2024-09-19T06:20:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.637+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/283806353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef010e0c0 con 0x7f5ef01011a0 2024-09-19T06:20:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.639+0000 7f5ee6ffd640 1 -- 172.21.15.5:0/283806353 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5eec0037a0 con 0x7f5ef01011a0 2024-09-19T06:20:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.640+0000 7f5efa29d740 1 Processor -- start 2024-09-19T06:20:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5efa29d740 1 -- start start 2024-09-19T06:20:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5efa29d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab609b40 0x55e3ab608090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5efa29d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab608a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5efa29d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab61bb80 0x55e3ab61df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5efa29d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3ab56e2f0 con 0x55e3ab61bb80 2024-09-19T06:20:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5efa29d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e3ab515cf0 con 0x55e3ab609b40 2024-09-19T06:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5efa29d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e3ab56a2b0 con 0x55e3ab6085d0 2024-09-19T06:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5ee6ffd640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5edc0a33b0 0x7f5edc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5ef545f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab608a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5ef5c60640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab609b40 0x55e3ab608090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5ef6461640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab61bb80 0x55e3ab61df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5ef6461640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab61bb80 0x55e3ab61df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41278/0 (socket says 172.21.15.5:41278) 2024-09-19T06:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5ef5c60640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab609b40 0x55e3ab608090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58450/0 (socket says 172.21.15.5:58450) 2024-09-19T06:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.641+0000 7f5ef6461640 1 -- 172.21.15.5:0/2780056387 learned_addr learned my addr 172.21.15.5:0/2780056387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.642+0000 7f5ef6461640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5edc0a33b0 0x7f5edc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.642+0000 7f5ee6ffd640 1 -- 172.21.15.5:0/283806353 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f5eec0cff70 con 0x7f5ef01011a0 2024-09-19T06:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.642+0000 7f5ef5c60640 1 -- 172.21.15.5:0/2780056387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 msgr2=0x55e3ab608a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.642+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/2780056387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab608a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.642+0000 7f5ef5c60640 1 -- 172.21.15.5:0/2780056387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab61bb80 msgr2=0x55e3ab61df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.642+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/2780056387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab61bb80 0x55e3ab61df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.642+0000 7f5ef5c60640 1 -- 172.21.15.5:0/2780056387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ab4fa3a0 con 0x55e3ab609b40 2024-09-19T06:20:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.642+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/2780056387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab609b40 0x55e3ab608090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0cd640 tx=0x7f5eec003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f1b4481121bda8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.642+0000 7f5edbfff640 1 -- 172.21.15.5:0/2780056387 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec003b30 con 0x55e3ab609b40 2024-09-19T06:20:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.643+0000 7f5edbfff640 1 -- 172.21.15.5:0/2780056387 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5eec09c8d0 con 0x55e3ab609b40 2024-09-19T06:20:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.643+0000 7f5edbfff640 1 -- 172.21.15.5:0/2780056387 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec0bc0f0 con 0x55e3ab609b40 2024-09-19T06:20:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.643+0000 7f5ef6461640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5edc0a33b0 0x7f5edc0a5870 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55e3ab609070 tx=0x7f5ee8004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.643+0000 7f5efa29d740 1 -- 172.21.15.5:0/2780056387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab609b40 msgr2=0x55e3ab608090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.643+0000 7f5efa29d740 1 --2- 172.21.15.5:0/2780056387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab609b40 0x55e3ab608090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0cd640 tx=0x7f5eec003e30 comp rx=0 tx=0).stop 2024-09-19T06:20:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.644+0000 7f5efa29d740 1 -- 172.21.15.5:0/2780056387 shutdown_connections 2024-09-19T06:20:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.644+0000 7f5efa29d740 1 --2- 172.21.15.5:0/2780056387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab61bb80 0x55e3ab61df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.644+0000 7f5efa29d740 1 --2- 172.21.15.5:0/2780056387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab608a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.644+0000 7f5efa29d740 1 --2- 172.21.15.5:0/2780056387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab609b40 0x55e3ab608090 secure :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0cd640 tx=0x7f5eec003e30 comp rx=0 tx=0).stop 2024-09-19T06:20:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.644+0000 7f5efa29d740 1 -- 172.21.15.5:0/2780056387 >> 172.21.15.5:0/2780056387 conn(0x55e3ab5d1310 msgr2=0x55e3ab604c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.644+0000 7f5efa29d740 1 -- 172.21.15.5:0/2780056387 shutdown_connections 2024-09-19T06:20:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.644+0000 7f5efa29d740 1 -- 172.21.15.5:0/2780056387 wait complete. 2024-09-19T06:20:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.644+0000 7f5efa29d740 1 -- start start 2024-09-19T06:20:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.644+0000 7f5efa29d740 1 Processor -- start 2024-09-19T06:20:28.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.645+0000 7f5efa29d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab618ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.645+0000 7f5efa29d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab61bb80 0x55e3ab619020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.645+0000 7f5efa29d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 0x55e3ab627cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.645+0000 7f5efa29d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3ab61e820 con 0x55e3ab61bb80 2024-09-19T06:20:28.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.645+0000 7f5efa29d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e3ab529060 con 0x55e3ab619560 2024-09-19T06:20:28.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.645+0000 7f5efa29d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e3ab61e9a0 con 0x55e3ab6085d0 2024-09-19T06:20:28.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.645+0000 7f5efa29d740 10 client.0 _ll_register_callbacks cb 0x55e3ab27c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:28.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.645+0000 7f5ef5c60640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab618ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.645+0000 7f5ef6461640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 0x55e3ab627cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.645+0000 7f5ef5c60640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab618ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34630/0 (socket says 172.21.15.5:34630) 2024-09-19T06:20:28.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.645+0000 7f5ef5c60640 1 -- 172.21.15.5:0/3619401488 learned_addr learned my addr 172.21.15.5:0/3619401488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:28.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.646+0000 7f5ef6461640 1 -- 172.21.15.5:0/3619401488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 msgr2=0x55e3ab618ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:28.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.646+0000 7f5ef6461640 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab618ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.646+0000 7f5ef6461640 1 -- 172.21.15.5:0/3619401488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab61bb80 msgr2=0x55e3ab619020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:28.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.646+0000 7f5ef545f640 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab61bb80 0x55e3ab619020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.647+0000 7f5ef6461640 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab61bb80 0x55e3ab619020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.647+0000 7f5ef6461640 1 -- 172.21.15.5:0/3619401488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ab2df3c0 con 0x55e3ab619560 2024-09-19T06:20:28.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.647+0000 7f5ef545f640 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab61bb80 0x55e3ab619020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:20:28.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.647+0000 7f5ef6461640 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 0x55e3ab627cd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8005c40 tx=0x7f5ee8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdbf9233e134b573 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:28.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.647+0000 7f5ed97fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:20:28.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.647+0000 7f5ed97fa640 1 -- 172.21.15.5:0/3619401488 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8026070 con 0x55e3ab619560 2024-09-19T06:20:28.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.647+0000 7f5efa29d740 1 -- 172.21.15.5:0/3619401488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eec09c8d0 con 0x55e3ab619560 2024-09-19T06:20:28.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.648+0000 7f5ed97fa640 1 -- 172.21.15.5:0/3619401488 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f5ee8004d20 con 0x55e3ab619560 2024-09-19T06:20:28.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.648+0000 7f5ed97fa640 1 -- 172.21.15.5:0/3619401488 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f5ee8031050 con 0x55e3ab619560 2024-09-19T06:20:28.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.648+0000 7f5efa29d740 10 client.29180 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:28.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.648+0000 7f5efa29d740 1 -- 172.21.15.5:0/3619401488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55e3ab62a9c0 con 0x55e3ab619560 2024-09-19T06:20:28.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.648+0000 7f5ed97fa640 1 -- 172.21.15.5:0/3619401488 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5eec09c8d0 con 0x55e3ab619560 2024-09-19T06:20:28.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.648+0000 7f5ed97fa640 1 -- 172.21.15.5:0/3619401488 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f5ee80054d0 con 0x55e3ab619560 2024-09-19T06:20:28.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.648+0000 7f5efa29d740 10 client.29180 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:28.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.648+0000 7f5efa29d740 10 client.29180 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:20:28.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.648+0000 7f5efa29d740 20 client.29180 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:28.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.648+0000 7f5efa29d740 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55e3ab62bdd0 0x55e3ab62e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.648+0000 7f5efa29d740 4 client.29180 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:28.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.648+0000 7f5efa29d740 1 -- 172.21.15.5:0/3619401488 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3ab57b860 con 0x55e3ab62bdd0 2024-09-19T06:20:28.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.649+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55e3ab62bdd0 0x55e3ab62e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.649+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55e3ab62bdd0 0x55e3ab62e1c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32343ed53f422103 server_cookie=8136dc46ec960ff6 in_seq=0 out_seq=0 2024-09-19T06:20:28.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.649+0000 7f5ed97fa640 10 client.29180 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:20:28.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.650+0000 7f5ed97fa640 1 -- 172.21.15.5:0/3619401488 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3ab57b860 con 0x55e3ab62bdd0 2024-09-19T06:20:28.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.650+0000 7f5ed97fa640 10 client.29180 handle_command_reply: tid=0 2024-09-19T06:20:28.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.650+0000 7f5efa29d740 1 client.29180 shutdown 2024-09-19T06:20:28.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.650+0000 7f5efa29d740 1 -- 172.21.15.5:0/3619401488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 msgr2=0x55e3ab627cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:28.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.650+0000 7f5efa29d740 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 0x55e3ab627cd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8005c40 tx=0x7f5ee8004a40 comp rx=0 tx=0).stop 2024-09-19T06:20:28.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.650+0000 7f5efa29d740 1 -- 172.21.15.5:0/3619401488 shutdown_connections 2024-09-19T06:20:28.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.650+0000 7f5efa29d740 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55e3ab62bdd0 0x55e3ab62e1c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.650+0000 7f5efa29d740 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 0x55e3ab627cd0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.650+0000 7f5efa29d740 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab61bb80 0x55e3ab619020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.650+0000 7f5efa29d740 1 --2- 172.21.15.5:0/3619401488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab618ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.650+0000 7f5efa29d740 1 -- 172.21.15.5:0/3619401488 >> 172.21.15.5:0/3619401488 conn(0x55e3ab5d1310 msgr2=0x55e3ab60a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:28.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.650+0000 7f5ed97fa640 0 client.29180 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:20:28.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.651+0000 7f5efa29d740 1 -- 172.21.15.5:0/3619401488 shutdown_connections 2024-09-19T06:20:28.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.651+0000 7f5efa29d740 1 -- 172.21.15.5:0/3619401488 wait complete. 2024-09-19T06:20:28.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.651+0000 7f5efa29d740 20 client.29180 trim_cache size 0 max 16384 2024-09-19T06:20:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.670+0000 7f5efa29d740 1 Processor -- start 2024-09-19T06:20:28.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.670+0000 7f5efa29d740 1 -- start start 2024-09-19T06:20:28.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.670+0000 7f5efa29d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab5d1310 0x55e3ab627ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.670+0000 7f5efa29d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab6085d0 0x55e3ab617ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.670+0000 7f5efa29d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab619560 0x55e3ab633ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.670+0000 7f5efa29d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01ab750 con 0x55e3ab5d1310 2024-09-19T06:20:28.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5efa29d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01ad2c0 con 0x55e3ab6085d0 2024-09-19T06:20:28.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5efa29d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01abe50 con 0x55e3ab619560 2024-09-19T06:20:28.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5ef6461640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab619560 0x55e3ab633ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5ef545f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab6085d0 0x55e3ab617ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5ef6461640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab619560 0x55e3ab633ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34636/0 (socket says 172.21.15.5:34636) 2024-09-19T06:20:28.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5ef545f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab6085d0 0x55e3ab617ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58464/0 (socket says 172.21.15.5:58464) 2024-09-19T06:20:28.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5ef6461640 1 -- 172.21.15.5:0/2489669834 learned_addr learned my addr 172.21.15.5:0/2489669834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:28.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5ef6461640 1 -- 172.21.15.5:0/2489669834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab6085d0 msgr2=0x55e3ab617ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:28.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5ef6461640 1 --2- 172.21.15.5:0/2489669834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab6085d0 0x55e3ab617ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5ef6461640 1 -- 172.21.15.5:0/2489669834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab5d1310 msgr2=0x55e3ab627ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:28.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5ef6461640 1 --2- 172.21.15.5:0/2489669834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab5d1310 0x55e3ab627ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5ef6461640 1 -- 172.21.15.5:0/2489669834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ab605960 con 0x55e3ab619560 2024-09-19T06:20:28.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.671+0000 7f5ef6461640 1 --2- 172.21.15.5:0/2489669834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab619560 0x55e3ab633ad0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8006010 tx=0x7f5ee8003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a56b2133a2f7aa41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.672+0000 7f5edb7fe640 1 -- 172.21.15.5:0/2489669834 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8026050 con 0x55e3ab619560 2024-09-19T06:20:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.672+0000 7f5edb7fe640 1 -- 172.21.15.5:0/2489669834 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ee803a070 con 0x55e3ab619560 2024-09-19T06:20:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.672+0000 7f5edb7fe640 1 -- 172.21.15.5:0/2489669834 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8035db0 con 0x55e3ab619560 2024-09-19T06:20:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.673+0000 7f5efa29d740 1 -- 172.21.15.5:0/2489669834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab619560 msgr2=0x55e3ab633ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.673+0000 7f5efa29d740 1 --2- 172.21.15.5:0/2489669834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab619560 0x55e3ab633ad0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8006010 tx=0x7f5ee8003970 comp rx=0 tx=0).stop 2024-09-19T06:20:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.673+0000 7f5efa29d740 1 -- 172.21.15.5:0/2489669834 shutdown_connections 2024-09-19T06:20:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.673+0000 7f5efa29d740 1 --2- 172.21.15.5:0/2489669834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab619560 0x55e3ab633ad0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.673+0000 7f5efa29d740 1 --2- 172.21.15.5:0/2489669834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab6085d0 0x55e3ab617ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.673+0000 7f5efa29d740 1 --2- 172.21.15.5:0/2489669834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab5d1310 0x55e3ab627ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.673+0000 7f5efa29d740 1 -- 172.21.15.5:0/2489669834 >> 172.21.15.5:0/2489669834 conn(0x55e3ab619180 msgr2=0x55e3ab6173b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:28.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.673+0000 7f5efa29d740 1 -- 172.21.15.5:0/2489669834 shutdown_connections 2024-09-19T06:20:28.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.673+0000 7f5efa29d740 1 -- 172.21.15.5:0/2489669834 wait complete. 2024-09-19T06:20:28.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.673+0000 7f5efa29d740 1 -- start start 2024-09-19T06:20:28.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.673+0000 7f5efa29d740 1 Processor -- start 2024-09-19T06:20:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5efa29d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab5d1310 0x55e3ab624360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5efa29d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab61c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5efa29d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 0x55e3ab61cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5efa29d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01acbf0 con 0x55e3ab5d1310 2024-09-19T06:20:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5efa29d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01ac520 con 0x55e3ab619560 2024-09-19T06:20:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5efa29d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01a9730 con 0x55e3ab6085d0 2024-09-19T06:20:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5ef6461640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 0x55e3ab61cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5efa29d740 10 client.0 _ll_register_callbacks cb 0x7f5ee8013ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5ef545f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab61c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5ef6461640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 0x55e3ab61cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58468/0 (socket says 172.21.15.5:58468) 2024-09-19T06:20:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5ef6461640 1 -- 172.21.15.5:0/3270127258 learned_addr learned my addr 172.21.15.5:0/3270127258 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab5d1310 0x55e3ab624360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5ef6461640 1 -- 172.21.15.5:0/3270127258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 msgr2=0x55e3ab61c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5ef6461640 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab61c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5ef6461640 1 -- 172.21.15.5:0/3270127258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab5d1310 msgr2=0x55e3ab624360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5ef6461640 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab5d1310 0x55e3ab624360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5ef6461640 1 -- 172.21.15.5:0/3270127258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ab61d5d0 con 0x55e3ab619560 2024-09-19T06:20:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.674+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab5d1310 0x55e3ab624360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:20:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.675+0000 7f5ef6461640 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 0x55e3ab61cdf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80059c0 tx=0x7f5ee8009a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33808161a55b02cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.675+0000 7f5ed97fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:20:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.675+0000 7f5ed97fa640 1 -- 172.21.15.5:0/3270127258 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8042070 con 0x55e3ab619560 2024-09-19T06:20:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.675+0000 7f5ed97fa640 1 -- 172.21.15.5:0/3270127258 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f5ee8035430 con 0x55e3ab619560 2024-09-19T06:20:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.675+0000 7f5efa29d740 1 -- 172.21.15.5:0/3270127258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee803a070 con 0x55e3ab619560 2024-09-19T06:20:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.675+0000 7f5ed97fa640 1 -- 172.21.15.5:0/3270127258 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f5ee8026070 con 0x55e3ab619560 2024-09-19T06:20:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.675+0000 7f5efa29d740 10 client.29186 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.675+0000 7f5efa29d740 1 -- 172.21.15.5:0/3270127258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55e3ab696aa0 con 0x55e3ab619560 2024-09-19T06:20:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.676+0000 7f5ed97fa640 1 -- 172.21.15.5:0/3270127258 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ee803a070 con 0x55e3ab619560 2024-09-19T06:20:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.676+0000 7f5ed97fa640 1 -- 172.21.15.5:0/3270127258 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f5ee802a750 con 0x55e3ab619560 2024-09-19T06:20:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.676+0000 7f5efa29d740 10 client.29186 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.676+0000 7f5efa29d740 10 client.29186 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:20:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.676+0000 7f5efa29d740 20 client.29186 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.676+0000 7f5efa29d740 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55e3ab61bb80 0x55e3ab6990e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.676+0000 7f5efa29d740 4 client.29186 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.676+0000 7f5efa29d740 1 -- 172.21.15.5:0/3270127258 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3ab575d70 con 0x55e3ab61bb80 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.676+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55e3ab61bb80 0x55e3ab6990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5ed97fa640 10 client.29186 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5ef5c60640 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55e3ab61bb80 0x55e3ab6990e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69c61b6d734eaae5 server_cookie=fe0b02d611a66c3a in_seq=0 out_seq=0 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5ed97fa640 1 -- 172.21.15.5:0/3270127258 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3ab575d70 con 0x55e3ab61bb80 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5ed97fa640 10 client.29186 handle_command_reply: tid=0 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5efa29d740 1 client.29186 shutdown 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5efa29d740 1 -- 172.21.15.5:0/3270127258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 msgr2=0x55e3ab61cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5efa29d740 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 0x55e3ab61cdf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80059c0 tx=0x7f5ee8009a50 comp rx=0 tx=0).stop 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5efa29d740 1 -- 172.21.15.5:0/3270127258 shutdown_connections 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5efa29d740 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55e3ab61bb80 0x55e3ab6990e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5efa29d740 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3ab619560 0x55e3ab61cdf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5efa29d740 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3ab6085d0 0x55e3ab61c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5efa29d740 1 --2- 172.21.15.5:0/3270127258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ab5d1310 0x55e3ab624360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5efa29d740 1 -- 172.21.15.5:0/3270127258 >> 172.21.15.5:0/3270127258 conn(0x55e3ab619180 msgr2=0x55e3ab617bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5ed97fa640 0 client.29186 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:20:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.677+0000 7f5efa29d740 1 -- 172.21.15.5:0/3270127258 shutdown_connections 2024-09-19T06:20:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.678+0000 7f5efa29d740 1 -- 172.21.15.5:0/3270127258 wait complete. 2024-09-19T06:20:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.678+0000 7f5efa29d740 20 client.29186 trim_cache size 0 max 16384 2024-09-19T06:20:28.683 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:20:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.678+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/283806353 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5edc0a33b0 msgr2=0x7f5edc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.678+0000 7f5ef7ec4640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5edc0a33b0 0x7f5edc0a5870 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55e3ab609070 tx=0x7f5ee8004ab0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.678+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/283806353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ef01011a0 msgr2=0x7f5ef01984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.678+0000 7f5ef7ec4640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ef01011a0 0x7f5ef01984d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00e9f0 tx=0x7f5eec00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.679+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/283806353 shutdown_connections 2024-09-19T06:20:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.679+0000 7f5ef7ec4640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef0106c30 0x7f5ef019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.679+0000 7f5ef7ec4640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5edc0a33b0 0x7f5edc0a5870 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.679+0000 7f5ef7ec4640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0101b40 0x7f5ef0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.679+0000 7f5ef7ec4640 1 --2- 172.21.15.5:0/283806353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ef01011a0 0x7f5ef01984d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.679+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/283806353 >> 172.21.15.5:0/283806353 conn(0x7f5ef0077210 msgr2=0x7f5ef0103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.679+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/283806353 shutdown_connections 2024-09-19T06:20:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:28.679+0000 7f5ef7ec4640 1 -- 172.21.15.5:0/283806353 wait complete. 2024-09-19T06:20:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:28 smithi157 ceph-mon[35346]: pgmap v539: 97 pgs: 97 active+clean; 718 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.1 MiB/s wr, 379 op/s 2024-09-19T06:20:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:28 smithi005 ceph-mon[30955]: pgmap v539: 97 pgs: 97 active+clean; 718 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.1 MiB/s wr, 379 op/s 2024-09-19T06:20:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:28 smithi049 ceph-mon[34640]: pgmap v539: 97 pgs: 97 active+clean; 718 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.1 MiB/s wr, 379 op/s 2024-09-19T06:20:29.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:20:29.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 38792 8.71 MB/sec execute 113 sec latency 64.208 ms 2024-09-19T06:20:29.767 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:20:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.334+0000 7f3fb57af640 1 -- 172.21.15.5:0/1722909420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0103650 msgr2=0x7f3fb010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.334+0000 7f3fb57af640 1 --2- 172.21.15.5:0/1722909420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0103650 0x7f3fb010bb40 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009900 tx=0x7f3fa402a480 comp rx=0 tx=0).stop 2024-09-19T06:20:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.334+0000 7f3fb57af640 1 -- 172.21.15.5:0/1722909420 shutdown_connections 2024-09-19T06:20:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.334+0000 7f3fb57af640 1 --2- 172.21.15.5:0/1722909420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb010c160 0x7f3fb010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.334+0000 7f3fb57af640 1 --2- 172.21.15.5:0/1722909420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0103650 0x7f3fb010bb40 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.334+0000 7f3fb57af640 1 --2- 172.21.15.5:0/1722909420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fb0102d30 0x7f3fb0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.334+0000 7f3fb57af640 1 -- 172.21.15.5:0/1722909420 >> 172.21.15.5:0/1722909420 conn(0x7f3fb00fc790 msgr2=0x7f3fb00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:30.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.334+0000 7f3fb57af640 1 -- 172.21.15.5:0/1722909420 shutdown_connections 2024-09-19T06:20:30.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.335+0000 7f3fb57af640 1 -- 172.21.15.5:0/1722909420 wait complete. 2024-09-19T06:20:30.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.335+0000 7f3fb57af640 1 Processor -- start 2024-09-19T06:20:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3fb57af640 1 -- start start 2024-09-19T06:20:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3fb57af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0102d30 0x7f3fb0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3fb57af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0103650 0x7f3fb0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3fb57af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fb010c160 0x7f3fb0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3fb57af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0111e60 con 0x7f3fb0103650 2024-09-19T06:20:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3fb57af640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0111ce0 con 0x7f3fb010c160 2024-09-19T06:20:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3fb57af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0111fe0 con 0x7f3fb0102d30 2024-09-19T06:20:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3fae7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0103650 0x7f3fb0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3faeffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0102d30 0x7f3fb0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3fae7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0103650 0x7f3fb0100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41324/0 (socket says 172.21.15.5:41324) 2024-09-19T06:20:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3fae7fc640 1 -- 172.21.15.5:0/2879703822 learned_addr learned my addr 172.21.15.5:0/2879703822 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:30.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3faf7fe640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fb010c160 0x7f3fb0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.336+0000 7f3faeffd640 1 -- 172.21.15.5:0/2879703822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fb010c160 msgr2=0x7f3fb0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:30.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.337+0000 7f3faeffd640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fb010c160 0x7f3fb0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.337+0000 7f3faeffd640 1 -- 172.21.15.5:0/2879703822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0103650 msgr2=0x7f3fb0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:30.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.337+0000 7f3faeffd640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0103650 0x7f3fb0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.337+0000 7f3faeffd640 1 -- 172.21.15.5:0/2879703822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb01aad30 con 0x7f3fb0102d30 2024-09-19T06:20:30.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.337+0000 7f3fae7fc640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0103650 0x7f3fb0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:30.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.337+0000 7f3faeffd640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0102d30 0x7f3fb0100990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000e9f0 tx=0x7f3fa000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:30.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.337+0000 7f3f97fff640 1 -- 172.21.15.5:0/2879703822 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa000ccc0 con 0x7f3fb0102d30 2024-09-19T06:20:30.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.337+0000 7f3f97fff640 1 -- 172.21.15.5:0/2879703822 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fa00108e0 con 0x7f3fb0102d30 2024-09-19T06:20:30.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.337+0000 7f3fb57af640 1 -- 172.21.15.5:0/2879703822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb01aaf60 con 0x7f3fb0102d30 2024-09-19T06:20:30.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.337+0000 7f3f97fff640 1 -- 172.21.15.5:0/2879703822 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0018880 con 0x7f3fb0102d30 2024-09-19T06:20:30.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.337+0000 7f3fb57af640 1 -- 172.21.15.5:0/2879703822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb01ab3c0 con 0x7f3fb0102d30 2024-09-19T06:20:30.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.341+0000 7f3f97fff640 1 -- 172.21.15.5:0/2879703822 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3fa0018a20 con 0x7f3fb0102d30 2024-09-19T06:20:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.341+0000 7f3fb7b88740 1 Processor -- start 2024-09-19T06:20:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.341+0000 7f3fb7b88740 1 -- start start 2024-09-19T06:20:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.341+0000 7f3fb7b88740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cadb40 0x559a95cac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.341+0000 7f3fb7b88740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cac5d0 0x559a95caca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.341+0000 7f3fb7b88740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cb7370 0x559a95cb9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.341+0000 7f3fb7b88740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a95c122f0 con 0x559a95cadb40 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.341+0000 7f3fb7b88740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559a95bb9cf0 con 0x559a95cb7370 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.341+0000 7f3fb7b88740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559a95c0e2b0 con 0x559a95cac5d0 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.342+0000 7f3faf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cadb40 0x559a95cac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.342+0000 7f3faeffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cac5d0 0x559a95caca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.342+0000 7f3fae7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cb7370 0x559a95cb9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.342+0000 7f3fae7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cb7370 0x559a95cb9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58502/0 (socket says 172.21.15.5:58502) 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.342+0000 7f3fae7fc640 1 -- 172.21.15.5:0/3126840006 learned_addr learned my addr 172.21.15.5:0/3126840006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.342+0000 7f3f97fff640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3f980a3430 0x7f3f980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.342+0000 7f3fae7fc640 1 -- 172.21.15.5:0/3126840006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cac5d0 msgr2=0x559a95caca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.342+0000 7f3fae7fc640 1 --2- 172.21.15.5:0/3126840006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cac5d0 0x559a95caca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.342+0000 7f3fae7fc640 1 -- 172.21.15.5:0/3126840006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cadb40 msgr2=0x559a95cac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:30.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.342+0000 7f3fae7fc640 1 --2- 172.21.15.5:0/3126840006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cadb40 0x559a95cac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.342+0000 7f3fae7fc640 1 -- 172.21.15.5:0/3126840006 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a95b9e3a0 con 0x559a95cb7370 2024-09-19T06:20:30.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.342+0000 7f3faf7fe640 1 --2- 172.21.15.5:0/3126840006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cadb40 0x559a95cac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:30.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.343+0000 7f3faf7fe640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3f980a3430 0x7f3f980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.343+0000 7f3fae7fc640 1 --2- 172.21.15.5:0/3126840006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cb7370 0x559a95cb9760 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3fa402aa70 tx=0x7f3fa40044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d91136008da7c33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:30.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.343+0000 7f3f97fff640 1 -- 172.21.15.5:0/2879703822 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f3fa00cffc0 con 0x7f3fb0102d30 2024-09-19T06:20:30.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.343+0000 7f3f957fa640 1 -- 172.21.15.5:0/3126840006 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4040070 con 0x559a95cb7370 2024-09-19T06:20:30.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.343+0000 7f3f957fa640 1 -- 172.21.15.5:0/3126840006 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fa40047a0 con 0x559a95cb7370 2024-09-19T06:20:30.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.343+0000 7f3f957fa640 1 -- 172.21.15.5:0/3126840006 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4035040 con 0x559a95cb7370 2024-09-19T06:20:30.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.344+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3126840006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cb7370 msgr2=0x559a95cb9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:30.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.344+0000 7f3faf7fe640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3f980a3430 0x7f3f980a58f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8005fd0 tx=0x7f3fa8004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.344+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/3126840006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cb7370 0x559a95cb9760 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3fa402aa70 tx=0x7f3fa40044b0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.344+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3126840006 shutdown_connections 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.344+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/3126840006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cb7370 0x559a95cb9760 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.344+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/3126840006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cac5d0 0x559a95caca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.344+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/3126840006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cadb40 0x559a95cac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.344+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3126840006 >> 172.21.15.5:0/3126840006 conn(0x559a95c75310 msgr2=0x559a95ca8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.344+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3126840006 shutdown_connections 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.344+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3126840006 wait complete. 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.344+0000 7f3fb7b88740 1 -- start start 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3fb7b88740 1 Processor -- start 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3fb7b88740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 0x559a95cb6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3fb7b88740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cadb40 0x559a95cb4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3fb7b88740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cb7370 0x559a95cb4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3faeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 0x559a95cb6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3fae7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cadb40 0x559a95cb4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3fae7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cadb40 0x559a95cb4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34680/0 (socket says 172.21.15.5:34680) 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3fae7fc640 1 -- 172.21.15.5:0/1925675174 learned_addr learned my addr 172.21.15.5:0/1925675174 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3fb7b88740 1 -- 172.21.15.5:0/1925675174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a95cba010 con 0x559a95cb7370 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3fb7b88740 1 -- 172.21.15.5:0/1925675174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559a95bcd060 con 0x559a95cac5d0 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3fb7b88740 1 -- 172.21.15.5:0/1925675174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559a95cba190 con 0x559a95cadb40 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3faf7fe640 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cb7370 0x559a95cb4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.345+0000 7f3fb7b88740 10 client.0 _ll_register_callbacks cb 0x559a959208c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.346+0000 7f3fae7fc640 1 -- 172.21.15.5:0/1925675174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 msgr2=0x559a95cb6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.346+0000 7f3fae7fc640 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 0x559a95cb6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.346+0000 7f3fae7fc640 1 -- 172.21.15.5:0/1925675174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cb7370 msgr2=0x559a95cb4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.346+0000 7f3fae7fc640 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cb7370 0x559a95cb4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.346+0000 7f3fae7fc640 1 -- 172.21.15.5:0/1925675174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a959833c0 con 0x559a95cadb40 2024-09-19T06:20:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.346+0000 7f3faeffd640 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 0x559a95cb6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:20:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.346+0000 7f3fb7b88740 1 -- 172.21.15.5:0/1925675174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fa40047a0 con 0x559a95cadb40 2024-09-19T06:20:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.346+0000 7f3fae7fc640 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cadb40 0x559a95cb4020 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009a20 tx=0x7f3fa402a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4198c60b23822a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.346+0000 7f3f92ff5640 10 client.29134 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:20:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.346+0000 7f3f92ff5640 1 -- 172.21.15.5:0/1925675174 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4040070 con 0x559a95cadb40 2024-09-19T06:20:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.347+0000 7f3f92ff5640 1 -- 172.21.15.5:0/1925675174 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f3fa4005410 con 0x559a95cadb40 2024-09-19T06:20:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.347+0000 7f3f92ff5640 1 -- 172.21.15.5:0/1925675174 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f3fa4035040 con 0x559a95cadb40 2024-09-19T06:20:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.347+0000 7f3fb7b88740 10 client.29134 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.347+0000 7f3fb7b88740 1 -- 172.21.15.5:0/1925675174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=57}) -- 0x559a95cba040 con 0x559a95cadb40 2024-09-19T06:20:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.347+0000 7f3f92ff5640 1 -- 172.21.15.5:0/1925675174 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f3fa40037a0 con 0x559a95cadb40 2024-09-19T06:20:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.347+0000 7f3fb7b88740 10 client.29134 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.347+0000 7f3fb7b88740 10 client.29134 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:20:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.347+0000 7f3fb7b88740 20 client.29134 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.347+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x559a95cc6b50 0x559a95cc8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.347+0000 7f3fb7b88740 4 client.29134 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.347+0000 7f3fb7b88740 1 -- 172.21.15.5:0/1925675174 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a95c1f860 con 0x559a95cc6b50 2024-09-19T06:20:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.347+0000 7f3faeffd640 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x559a95cc6b50 0x559a95cc8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.348+0000 7f3faeffd640 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x559a95cc6b50 0x559a95cc8f40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2b0cc749fbe8240 server_cookie=d756785bfe994bf7 in_seq=0 out_seq=0 2024-09-19T06:20:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.348+0000 7f3f92ff5640 10 client.29134 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:20:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3f92ff5640 1 -- 172.21.15.5:0/1925675174 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a95c1f860 con 0x559a95cc6b50 2024-09-19T06:20:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3f92ff5640 10 client.29134 handle_command_reply: tid=0 2024-09-19T06:20:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3fb7b88740 1 client.29134 shutdown 2024-09-19T06:20:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3fb7b88740 1 -- 172.21.15.5:0/1925675174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cadb40 msgr2=0x559a95cb4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cadb40 0x559a95cb4020 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009a20 tx=0x7f3fa402a480 comp rx=0 tx=0).stop 2024-09-19T06:20:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3fb7b88740 1 -- 172.21.15.5:0/1925675174 shutdown_connections 2024-09-19T06:20:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x559a95cc6b50 0x559a95cc8f40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cb7370 0x559a95cb4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cadb40 0x559a95cb4020 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/1925675174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 0x559a95cb6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3fb7b88740 1 -- 172.21.15.5:0/1925675174 >> 172.21.15.5:0/1925675174 conn(0x559a95c75310 msgr2=0x559a95cae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3f92ff5640 0 client.29134 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:20:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3fb7b88740 1 -- 172.21.15.5:0/1925675174 shutdown_connections 2024-09-19T06:20:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.349+0000 7f3fb7b88740 1 -- 172.21.15.5:0/1925675174 wait complete. 2024-09-19T06:20:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.350+0000 7f3fb7b88740 20 client.29134 trim_cache size 0 max 16384 2024-09-19T06:20:30.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.367+0000 7f3fb7b88740 1 Processor -- start 2024-09-19T06:20:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.367+0000 7f3fb7b88740 1 -- start start 2024-09-19T06:20:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.367+0000 7f3fb7b88740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 0x559a95ca8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.367+0000 7f3fb7b88740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cadb40 0x559a95cc32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.367+0000 7f3fb7b88740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cb7370 0x559a95cb38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3fb7b88740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b3e30 con 0x559a95cadb40 2024-09-19T06:20:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3fb7b88740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b59a0 con 0x559a95cac5d0 2024-09-19T06:20:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3fb7b88740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b4530 con 0x559a95cb7370 2024-09-19T06:20:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3faeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 0x559a95ca8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3faeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 0x559a95ca8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58526/0 (socket says 172.21.15.5:58526) 2024-09-19T06:20:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3faeffd640 1 -- 172.21.15.5:0/4207278756 learned_addr learned my addr 172.21.15.5:0/4207278756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3fae7fc640 1 --2- 172.21.15.5:0/4207278756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cadb40 0x559a95cc32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3faf7fe640 1 --2- 172.21.15.5:0/4207278756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cb7370 0x559a95cb38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3faeffd640 1 -- 172.21.15.5:0/4207278756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cb7370 msgr2=0x559a95cb38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3faeffd640 1 --2- 172.21.15.5:0/4207278756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cb7370 0x559a95cb38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3faeffd640 1 -- 172.21.15.5:0/4207278756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cadb40 msgr2=0x559a95cc32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3faeffd640 1 --2- 172.21.15.5:0/4207278756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cadb40 0x559a95cc32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3faeffd640 1 -- 172.21.15.5:0/4207278756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a95cb4940 con 0x559a95cac5d0 2024-09-19T06:20:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.368+0000 7f3faeffd640 1 --2- 172.21.15.5:0/4207278756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 0x559a95ca8e10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0098790 tx=0x7f3fa00d24d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d95ec173d265b6fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.369+0000 7f3f94ff9640 1 -- 172.21.15.5:0/4207278756 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa009bc90 con 0x559a95cac5d0 2024-09-19T06:20:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.369+0000 7f3f94ff9640 1 -- 172.21.15.5:0/4207278756 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fa0010e00 con 0x559a95cac5d0 2024-09-19T06:20:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.369+0000 7f3f94ff9640 1 -- 172.21.15.5:0/4207278756 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa00989a0 con 0x559a95cac5d0 2024-09-19T06:20:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.369+0000 7f3fb7b88740 1 -- 172.21.15.5:0/4207278756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 msgr2=0x559a95ca8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.369+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/4207278756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 0x559a95ca8e10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0098790 tx=0x7f3fa00d24d0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.370+0000 7f3fb7b88740 1 -- 172.21.15.5:0/4207278756 shutdown_connections 2024-09-19T06:20:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.370+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/4207278756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cb7370 0x559a95cb38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.370+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/4207278756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cadb40 0x559a95cc32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.370+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/4207278756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cac5d0 0x559a95ca8e10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.370+0000 7f3fb7b88740 1 -- 172.21.15.5:0/4207278756 >> 172.21.15.5:0/4207278756 conn(0x559a95c75310 msgr2=0x559a95ca8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.370+0000 7f3fb7b88740 1 -- 172.21.15.5:0/4207278756 shutdown_connections 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.370+0000 7f3fb7b88740 1 -- 172.21.15.5:0/4207278756 wait complete. 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.370+0000 7f3fb7b88740 1 -- start start 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.370+0000 7f3fb7b88740 1 Processor -- start 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.370+0000 7f3fb7b88740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cac5d0 0x559a95cba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.370+0000 7f3fb7b88740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cadb40 0x559a95cbaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3fb7b88740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cb7370 0x559a95d2f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3fb7b88740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b52d0 con 0x559a95cac5d0 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3fb7b88740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b4c00 con 0x559a95cadb40 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3fb7b88740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b1e30 con 0x559a95cb7370 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3faeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cac5d0 0x559a95cba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3fb7b88740 10 client.0 _ll_register_callbacks cb 0x7f3fa8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3fae7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cadb40 0x559a95cbaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3faeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cac5d0 0x559a95cba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41364/0 (socket says 172.21.15.5:41364) 2024-09-19T06:20:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3faeffd640 1 -- 172.21.15.5:0/3425000366 learned_addr learned my addr 172.21.15.5:0/3425000366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3faf7fe640 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cb7370 0x559a95d2f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3fae7fc640 1 -- 172.21.15.5:0/3425000366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cb7370 msgr2=0x559a95d2f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3fae7fc640 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cb7370 0x559a95d2f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3fae7fc640 1 -- 172.21.15.5:0/3425000366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cac5d0 msgr2=0x559a95cba580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3fae7fc640 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cac5d0 0x559a95cba580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3fae7fc640 1 -- 172.21.15.5:0/3425000366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a95d2faa0 con 0x559a95cadb40 2024-09-19T06:20:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.371+0000 7f3faeffd640 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cac5d0 0x559a95cba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:30.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.372+0000 7f3fae7fc640 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cadb40 0x559a95cbaac0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4006010 tx=0x7f3fa4005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c03e7db20c65c123 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:30.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.372+0000 7f3f92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:20:30.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.372+0000 7f3f92ff5640 1 -- 172.21.15.5:0/3425000366 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4060060 con 0x559a95cadb40 2024-09-19T06:20:30.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.372+0000 7f3f92ff5640 1 -- 172.21.15.5:0/3425000366 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f3fa4005410 con 0x559a95cadb40 2024-09-19T06:20:30.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.372+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3425000366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a95c19d70 con 0x559a95cadb40 2024-09-19T06:20:30.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.372+0000 7f3f92ff5640 1 -- 172.21.15.5:0/3425000366 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f3fa40314a0 con 0x559a95cadb40 2024-09-19T06:20:30.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.372+0000 7f3fb7b88740 10 client.29201 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:30.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.372+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3425000366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559a95d30c60 con 0x559a95cadb40 2024-09-19T06:20:30.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.373+0000 7f3f92ff5640 1 -- 172.21.15.5:0/3425000366 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559a95c19d70 con 0x559a95cadb40 2024-09-19T06:20:30.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.373+0000 7f3f92ff5640 1 -- 172.21.15.5:0/3425000366 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f3fa405a040 con 0x559a95cadb40 2024-09-19T06:20:30.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.373+0000 7f3fb7b88740 10 client.29201 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:30.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.373+0000 7f3fb7b88740 10 client.29201 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:20:30.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.373+0000 7f3fb7b88740 20 client.29201 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.373+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x559a95d312d0 0x559a95d336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.373+0000 7f3fb7b88740 4 client.29201 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.373+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3425000366 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a95b96170 con 0x559a95d312d0 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.373+0000 7f3faeffd640 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x559a95d312d0 0x559a95d336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.374+0000 7f3faeffd640 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x559a95d312d0 0x559a95d336c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b9ed586232fa57e server_cookie=b13a10e6f171dac1 in_seq=0 out_seq=0 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.374+0000 7f3f92ff5640 10 client.29201 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.374+0000 7f3f92ff5640 1 -- 172.21.15.5:0/3425000366 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a95b96170 con 0x559a95d312d0 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.374+0000 7f3f92ff5640 10 client.29201 handle_command_reply: tid=0 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.374+0000 7f3fb7b88740 1 client.29201 shutdown 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.374+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3425000366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cadb40 msgr2=0x559a95cbaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.374+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cadb40 0x559a95cbaac0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4006010 tx=0x7f3fa4005a20 comp rx=0 tx=0).stop 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.375+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3425000366 shutdown_connections 2024-09-19T06:20:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.375+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x559a95d312d0 0x559a95d336c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.375+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a95cb7370 0x559a95d2f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.375+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a95cadb40 0x559a95cbaac0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.375+0000 7f3fb7b88740 1 --2- 172.21.15.5:0/3425000366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a95cac5d0 0x559a95cba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.375+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3425000366 >> 172.21.15.5:0/3425000366 conn(0x559a95c75310 msgr2=0x559a95cb60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:30.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.375+0000 7f3f92ff5640 0 client.29201 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:20:30.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.375+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3425000366 shutdown_connections 2024-09-19T06:20:30.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.375+0000 7f3fb7b88740 1 -- 172.21.15.5:0/3425000366 wait complete. 2024-09-19T06:20:30.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.375+0000 7f3fb7b88740 20 client.29201 trim_cache size 0 max 16384 2024-09-19T06:20:30.381 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:20:30.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.376+0000 7f3fb57af640 1 -- 172.21.15.5:0/2879703822 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3f980a3430 msgr2=0x7f3f980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:30.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.376+0000 7f3fb57af640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3f980a3430 0x7f3f980a58f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8005fd0 tx=0x7f3fa8004a40 comp rx=0 tx=0).stop 2024-09-19T06:20:30.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.376+0000 7f3fb57af640 1 -- 172.21.15.5:0/2879703822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0102d30 msgr2=0x7f3fb0100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:30.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.376+0000 7f3fb57af640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0102d30 0x7f3fb0100990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000e9f0 tx=0x7f3fa000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.376+0000 7f3fb57af640 1 -- 172.21.15.5:0/2879703822 shutdown_connections 2024-09-19T06:20:30.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.376+0000 7f3fb57af640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fb010c160 0x7f3fb0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.376+0000 7f3fb57af640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0103650 0x7f3fb0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.376+0000 7f3fb57af640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3f980a3430 0x7f3f980a58f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.376+0000 7f3fb57af640 1 --2- 172.21.15.5:0/2879703822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0102d30 0x7f3fb0100990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:30.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.376+0000 7f3fb57af640 1 -- 172.21.15.5:0/2879703822 >> 172.21.15.5:0/2879703822 conn(0x7f3fb00fc790 msgr2=0x7f3fb00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:30.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.376+0000 7f3fb57af640 1 -- 172.21.15.5:0/2879703822 shutdown_connections 2024-09-19T06:20:30.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:30.376+0000 7f3fb57af640 1 -- 172.21.15.5:0/2879703822 wait complete. 2024-09-19T06:20:30.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 38959 8.70 MB/sec execute 114 sec latency 72.269 ms 2024-09-19T06:20:30.763 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:30 smithi005 ceph-mon[30955]: pgmap v540: 97 pgs: 97 active+clean; 718 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 226 op/s 2024-09-19T06:20:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:30 smithi157 ceph-mon[35346]: pgmap v540: 97 pgs: 97 active+clean; 718 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 226 op/s 2024-09-19T06:20:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:30 smithi049 ceph-mon[34640]: pgmap v540: 97 pgs: 97 active+clean; 718 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 226 op/s 2024-09-19T06:20:31.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:20:31.481 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:20:31.558 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39132 8.71 MB/sec execute 115 sec latency 173.403 ms 2024-09-19T06:20:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.082+0000 7f28e609c640 1 -- 172.21.15.5:0/2280810027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e0102d30 msgr2=0x7f28e0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.082+0000 7f28e609c640 1 --2- 172.21.15.5:0/2280810027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e0102d30 0x7f28e0103110 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009930 tx=0x7f28d002a420 comp rx=0 tx=0).stop 2024-09-19T06:20:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.083+0000 7f28e609c640 1 -- 172.21.15.5:0/2280810027 shutdown_connections 2024-09-19T06:20:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.083+0000 7f28e609c640 1 --2- 172.21.15.5:0/2280810027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e01082a0 0x7f28e010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.083+0000 7f28e609c640 1 --2- 172.21.15.5:0/2280810027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28e01036e0 0x7f28e0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.083+0000 7f28e609c640 1 --2- 172.21.15.5:0/2280810027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e0102d30 0x7f28e0103110 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.083+0000 7f28e609c640 1 -- 172.21.15.5:0/2280810027 >> 172.21.15.5:0/2280810027 conn(0x7f28e00fc7d0 msgr2=0x7f28e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.083+0000 7f28e609c640 1 -- 172.21.15.5:0/2280810027 shutdown_connections 2024-09-19T06:20:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.083+0000 7f28e609c640 1 -- 172.21.15.5:0/2280810027 wait complete. 2024-09-19T06:20:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28e609c640 1 Processor -- start 2024-09-19T06:20:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28e609c640 1 -- start start 2024-09-19T06:20:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28e609c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e0102d30 0x7f28e019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28e609c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28e01036e0 0x7f28e019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28e609c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e01082a0 0x7f28e019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28e609c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0102070 con 0x7f28e01082a0 2024-09-19T06:20:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28e609c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0101ef0 con 0x7f28e01036e0 2024-09-19T06:20:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28e609c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01021f0 con 0x7f28e0102d30 2024-09-19T06:20:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e01082a0 0x7f28e019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e01082a0 0x7f28e019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41394/0 (socket says 172.21.15.5:41394) 2024-09-19T06:20:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28dffff640 1 -- 172.21.15.5:0/3325299790 learned_addr learned my addr 172.21.15.5:0/3325299790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28df7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e0102d30 0x7f28e019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.084+0000 7f28deffd640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28e01036e0 0x7f28e019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.085+0000 7f28dffff640 1 -- 172.21.15.5:0/3325299790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e0102d30 msgr2=0x7f28e019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.085+0000 7f28dffff640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e0102d30 0x7f28e019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.085+0000 7f28dffff640 1 -- 172.21.15.5:0/3325299790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28e01036e0 msgr2=0x7f28e019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.085+0000 7f28dffff640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28e01036e0 0x7f28e019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.085+0000 7f28dffff640 1 -- 172.21.15.5:0/3325299790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e019f660 con 0x7f28e01082a0 2024-09-19T06:20:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.085+0000 7f28df7fe640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e0102d30 0x7f28e019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.085+0000 7f28dffff640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e01082a0 0x7f28e019ef80 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f28d800eb10 tx=0x7f28d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.086+0000 7f28dcff9640 1 -- 172.21.15.5:0/3325299790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8004b20 con 0x7f28e01082a0 2024-09-19T06:20:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.086+0000 7f28dcff9640 1 -- 172.21.15.5:0/3325299790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28d8004cc0 con 0x7f28e01082a0 2024-09-19T06:20:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.086+0000 7f28dcff9640 1 -- 172.21.15.5:0/3325299790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8004b20 con 0x7f28e01082a0 2024-09-19T06:20:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.086+0000 7f28e609c640 1 -- 172.21.15.5:0/3325299790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e019f950 con 0x7f28e01082a0 2024-09-19T06:20:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.086+0000 7f28e609c640 1 -- 172.21.15.5:0/3325299790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e01a7230 con 0x7f28e01082a0 2024-09-19T06:20:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.089+0000 7f28dcff9640 1 -- 172.21.15.5:0/3325299790 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f28d8005020 con 0x7f28e01082a0 2024-09-19T06:20:32.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28e8475740 1 Processor -- start 2024-09-19T06:20:32.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28e8475740 1 -- start start 2024-09-19T06:20:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28e8475740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d0348b40 0x55b0d0347090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28e8475740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d03475d0 0x55b0d0347a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28e8475740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0352370 0x55b0d0354760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28e8475740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b0d02ad2f0 con 0x55b0d0352370 2024-09-19T06:20:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28e8475740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b0d0254cf0 con 0x55b0d0348b40 2024-09-19T06:20:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28e8475740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b0d02a92b0 con 0x55b0d03475d0 2024-09-19T06:20:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28dcff9640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f28c80a37b0 0x7f28c80a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28dffff640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f28c80a37b0 0x7f28c80a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28deffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d0348b40 0x55b0d0347090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28deffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d0348b40 0x55b0d0347090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58542/0 (socket says 172.21.15.5:58542) 2024-09-19T06:20:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28deffd640 1 -- 172.21.15.5:0/1199682214 learned_addr learned my addr 172.21.15.5:0/1199682214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28df7fe640 1 --2- 172.21.15.5:0/1199682214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d03475d0 0x55b0d0347a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28deffd640 1 --2- 172.21.15.5:0/1199682214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0352370 0x55b0d0354760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.090+0000 7f28dcff9640 1 -- 172.21.15.5:0/3325299790 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f28d800cce0 con 0x7f28e01082a0 2024-09-19T06:20:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.091+0000 7f28deffd640 1 -- 172.21.15.5:0/1199682214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d03475d0 msgr2=0x55b0d0347a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.091+0000 7f28deffd640 1 --2- 172.21.15.5:0/1199682214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d03475d0 0x55b0d0347a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.091+0000 7f28deffd640 1 -- 172.21.15.5:0/1199682214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0352370 msgr2=0x55b0d0354760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.091+0000 7f28deffd640 1 --2- 172.21.15.5:0/1199682214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0352370 0x55b0d0354760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.091+0000 7f28deffd640 1 -- 172.21.15.5:0/1199682214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0d02393a0 con 0x55b0d0348b40 2024-09-19T06:20:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.091+0000 7f28dffff640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f28c80a37b0 0x7f28c80a5c70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f28d80cd4b0 tx=0x7f28d80bc040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.091+0000 7f28deffd640 1 --2- 172.21.15.5:0/1199682214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d0348b40 0x55b0d0347090 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f28d40058c0 tx=0x7f28d4005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad24c1c15bc8c53a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.091+0000 7f28c5ffb640 1 -- 172.21.15.5:0/1199682214 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4005020 con 0x55b0d0348b40 2024-09-19T06:20:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.091+0000 7f28c5ffb640 1 -- 172.21.15.5:0/1199682214 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28d40047a0 con 0x55b0d0348b40 2024-09-19T06:20:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.091+0000 7f28c5ffb640 1 -- 172.21.15.5:0/1199682214 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d400fa20 con 0x55b0d0348b40 2024-09-19T06:20:32.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.092+0000 7f28e8475740 1 -- 172.21.15.5:0/1199682214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d0348b40 msgr2=0x55b0d0347090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.092+0000 7f28e8475740 1 --2- 172.21.15.5:0/1199682214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d0348b40 0x55b0d0347090 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f28d40058c0 tx=0x7f28d4005850 comp rx=0 tx=0).stop 2024-09-19T06:20:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.092+0000 7f28e8475740 1 -- 172.21.15.5:0/1199682214 shutdown_connections 2024-09-19T06:20:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.092+0000 7f28e8475740 1 --2- 172.21.15.5:0/1199682214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0352370 0x55b0d0354760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.092+0000 7f28e8475740 1 --2- 172.21.15.5:0/1199682214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d03475d0 0x55b0d0347a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.092+0000 7f28e8475740 1 --2- 172.21.15.5:0/1199682214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d0348b40 0x55b0d0347090 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.092+0000 7f28e8475740 1 -- 172.21.15.5:0/1199682214 >> 172.21.15.5:0/1199682214 conn(0x55b0d0310310 msgr2=0x55b0d0343c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.092+0000 7f28e8475740 1 -- 172.21.15.5:0/1199682214 shutdown_connections 2024-09-19T06:20:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.093+0000 7f28e8475740 1 -- 172.21.15.5:0/1199682214 wait complete. 2024-09-19T06:20:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.093+0000 7f28e8475740 1 -- start start 2024-09-19T06:20:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.093+0000 7f28e8475740 1 Processor -- start 2024-09-19T06:20:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28e8475740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d03475d0 0x55b0d0351d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28e8475740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 0x55b0d034f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28e8475740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0352370 0x55b0d034f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28e8475740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b0d0355070 con 0x55b0d0352370 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28e8475740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b0d0268060 con 0x55b0d03475d0 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28e8475740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b0d03551f0 con 0x55b0d0348b40 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28deffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d03475d0 0x55b0d0351d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28deffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d03475d0 0x55b0d0351d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58544/0 (socket says 172.21.15.5:58544) 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28e8475740 10 client.0 _ll_register_callbacks cb 0x55b0cffbb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28deffd640 1 -- 172.21.15.5:0/2807810966 learned_addr learned my addr 172.21.15.5:0/2807810966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28deffd640 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0352370 0x55b0d034f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28df7fe640 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 0x55b0d034f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28deffd640 1 -- 172.21.15.5:0/2807810966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 msgr2=0x55b0d034f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28deffd640 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 0x55b0d034f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28deffd640 1 -- 172.21.15.5:0/2807810966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d03475d0 msgr2=0x55b0d0351d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28deffd640 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d03475d0 0x55b0d0351d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28deffd640 1 -- 172.21.15.5:0/2807810966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0d001e3c0 con 0x55b0d0352370 2024-09-19T06:20:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.094+0000 7f28df7fe640 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 0x55b0d034f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.095+0000 7f28deffd640 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0352370 0x55b0d034f600 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f28d4004f80 tx=0x7f28d4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97e36e5b497628a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.095+0000 7f28beffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:20:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.095+0000 7f28beffd640 1 -- 172.21.15.5:0/2807810966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d40055f0 con 0x55b0d0352370 2024-09-19T06:20:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.095+0000 7f28e8475740 1 -- 172.21.15.5:0/2807810966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d40047a0 con 0x55b0d0352370 2024-09-19T06:20:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.095+0000 7f28beffd640 1 -- 172.21.15.5:0/2807810966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f28d400f790 con 0x55b0d0352370 2024-09-19T06:20:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.095+0000 7f28beffd640 1 -- 172.21.15.5:0/2807810966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f28d400f990 con 0x55b0d0352370 2024-09-19T06:20:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.095+0000 7f28e8475740 10 client.19836 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.095+0000 7f28e8475740 1 -- 172.21.15.5:0/2807810966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55b0d0360b10 con 0x55b0d0352370 2024-09-19T06:20:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.096+0000 7f28beffd640 1 -- 172.21.15.5:0/2807810966 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28d402d460 con 0x55b0d0352370 2024-09-19T06:20:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.096+0000 7f28beffd640 1 -- 172.21.15.5:0/2807810966 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f28d4035080 con 0x55b0d0352370 2024-09-19T06:20:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.096+0000 7f28e8475740 10 client.19836 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.096+0000 7f28e8475740 10 client.19836 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:20:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.096+0000 7f28e8475740 20 client.19836 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.096+0000 7f28e8475740 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b0d0361fd0 0x55b0d03643c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.096+0000 7f28e8475740 4 client.19836 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.096+0000 7f28e8475740 1 -- 172.21.15.5:0/2807810966 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f28d40047a0 con 0x55b0d0361fd0 2024-09-19T06:20:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.096+0000 7f28df7fe640 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b0d0361fd0 0x55b0d03643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.097+0000 7f28beffd640 10 client.19836 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:20:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.097+0000 7f28df7fe640 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b0d0361fd0 0x55b0d03643c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3ad171222da3a68 server_cookie=f49d71dbc6ca4aa5 in_seq=0 out_seq=0 2024-09-19T06:20:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.098+0000 7f28beffd640 1 -- 172.21.15.5:0/2807810966 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f28d40047a0 con 0x55b0d0361fd0 2024-09-19T06:20:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.098+0000 7f28beffd640 10 client.19836 handle_command_reply: tid=0 2024-09-19T06:20:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.098+0000 7f28e8475740 1 client.19836 shutdown 2024-09-19T06:20:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.098+0000 7f28e8475740 1 -- 172.21.15.5:0/2807810966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0352370 msgr2=0x55b0d034f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:32.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.098+0000 7f28e8475740 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0352370 0x55b0d034f600 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f28d4004f80 tx=0x7f28d4005f90 comp rx=0 tx=0).stop 2024-09-19T06:20:32.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.099+0000 7f28e8475740 1 -- 172.21.15.5:0/2807810966 shutdown_connections 2024-09-19T06:20:32.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.099+0000 7f28e8475740 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b0d0361fd0 0x55b0d03643c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.099+0000 7f28beffd640 0 client.19836 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:20:32.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.099+0000 7f28e8475740 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0352370 0x55b0d034f600 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.099+0000 7f28e8475740 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 0x55b0d034f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.099+0000 7f28e8475740 1 --2- 172.21.15.5:0/2807810966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d03475d0 0x55b0d0351d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.099+0000 7f28e8475740 1 -- 172.21.15.5:0/2807810966 >> 172.21.15.5:0/2807810966 conn(0x55b0d0310310 msgr2=0x55b0d0349960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:32.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.099+0000 7f28e8475740 1 -- 172.21.15.5:0/2807810966 shutdown_connections 2024-09-19T06:20:32.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.099+0000 7f28e8475740 1 -- 172.21.15.5:0/2807810966 wait complete. 2024-09-19T06:20:32.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.099+0000 7f28e8475740 20 client.19836 trim_cache size 0 max 16384 2024-09-19T06:20:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.125+0000 7f28e8475740 1 Processor -- start 2024-09-19T06:20:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.125+0000 7f28e8475740 1 -- start start 2024-09-19T06:20:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.125+0000 7f28e8475740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d03475d0 0x55b0d035e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.125+0000 7f28e8475740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 0x55b0d034a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.125+0000 7f28e8475740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d0352370 0x55b0d034a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.125+0000 7f28e8475740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01afa50 con 0x55b0d03475d0 2024-09-19T06:20:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.125+0000 7f28e8475740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b15c0 con 0x55b0d0352370 2024-09-19T06:20:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.125+0000 7f28e8475740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b0150 con 0x55b0d0348b40 2024-09-19T06:20:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.125+0000 7f28deffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d03475d0 0x55b0d035e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28df7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 0x55b0d034a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28deffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d03475d0 0x55b0d035e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41408/0 (socket says 172.21.15.5:41408) 2024-09-19T06:20:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28df7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 0x55b0d034a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34768/0 (socket says 172.21.15.5:34768) 2024-09-19T06:20:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28deffd640 1 -- 172.21.15.5:0/249036909 learned_addr learned my addr 172.21.15.5:0/249036909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28deffd640 1 --2- 172.21.15.5:0/249036909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d0352370 0x55b0d034a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28df7fe640 1 -- 172.21.15.5:0/249036909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d0352370 msgr2=0x55b0d034a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28df7fe640 1 --2- 172.21.15.5:0/249036909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d0352370 0x55b0d034a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28df7fe640 1 -- 172.21.15.5:0/249036909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d03475d0 msgr2=0x55b0d035e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28df7fe640 1 --2- 172.21.15.5:0/249036909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d03475d0 0x55b0d035e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28df7fe640 1 -- 172.21.15.5:0/249036909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0d0343ea0 con 0x55b0d0348b40 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28df7fe640 1 --2- 172.21.15.5:0/249036909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 0x55b0d034a120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f28d0031440 tx=0x7f28d0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e45f7954debd4e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28c57fa640 1 -- 172.21.15.5:0/249036909 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0035040 con 0x55b0d0348b40 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28c57fa640 1 -- 172.21.15.5:0/249036909 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28d0004490 con 0x55b0d0348b40 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.126+0000 7f28c57fa640 1 -- 172.21.15.5:0/249036909 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0003850 con 0x55b0d0348b40 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.128+0000 7f28e8475740 1 -- 172.21.15.5:0/249036909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 msgr2=0x55b0d034a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.128+0000 7f28e8475740 1 --2- 172.21.15.5:0/249036909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 0x55b0d034a120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f28d0031440 tx=0x7f28d0000bc0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.128+0000 7f28e8475740 1 -- 172.21.15.5:0/249036909 shutdown_connections 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.128+0000 7f28e8475740 1 --2- 172.21.15.5:0/249036909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d0352370 0x55b0d034a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.128+0000 7f28e8475740 1 --2- 172.21.15.5:0/249036909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0348b40 0x55b0d034a120 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.128+0000 7f28e8475740 1 --2- 172.21.15.5:0/249036909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d03475d0 0x55b0d035e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.128+0000 7f28e8475740 1 -- 172.21.15.5:0/249036909 >> 172.21.15.5:0/249036909 conn(0x55b0d0310310 msgr2=0x55b0d034d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.128+0000 7f28e8475740 1 -- 172.21.15.5:0/249036909 shutdown_connections 2024-09-19T06:20:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.128+0000 7f28e8475740 1 -- 172.21.15.5:0/249036909 wait complete. 2024-09-19T06:20:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.128+0000 7f28e8475740 1 -- start start 2024-09-19T06:20:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28e8475740 1 Processor -- start 2024-09-19T06:20:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28e8475740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d03475d0 0x55b0d035a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28e8475740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0348b40 0x55b0d035a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28e8475740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0352370 0x55b0d034b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28e8475740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b0ef0 con 0x55b0d0348b40 2024-09-19T06:20:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28e8475740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b0820 con 0x55b0d03475d0 2024-09-19T06:20:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28e8475740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01ada50 con 0x55b0d0352370 2024-09-19T06:20:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28e8475740 10 client.0 _ll_register_callbacks cb 0x7f28d4009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0348b40 0x55b0d035a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0348b40 0x55b0d035a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41418/0 (socket says 172.21.15.5:41418) 2024-09-19T06:20:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28df7fe640 1 -- 172.21.15.5:0/87039629 learned_addr learned my addr 172.21.15.5:0/87039629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28deffd640 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d03475d0 0x55b0d035a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28deffd640 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0352370 0x55b0d034b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28df7fe640 1 -- 172.21.15.5:0/87039629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0352370 msgr2=0x55b0d034b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28df7fe640 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0352370 0x55b0d034b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28df7fe640 1 -- 172.21.15.5:0/87039629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d03475d0 msgr2=0x55b0d035a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28df7fe640 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d03475d0 0x55b0d035a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.129+0000 7f28df7fe640 1 -- 172.21.15.5:0/87039629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0d035b0b0 con 0x55b0d0348b40 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.130+0000 7f28deffd640 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0352370 0x55b0d034b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.130+0000 7f28df7fe640 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0348b40 0x55b0d035a9c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f28d0005e00 tx=0x7f28d002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca93a70c1d4d1d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.130+0000 7f28beffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.130+0000 7f28beffd640 1 -- 172.21.15.5:0/87039629 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0035070 con 0x55b0d0348b40 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.130+0000 7f28beffd640 1 -- 172.21.15.5:0/87039629 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f28d0004b80 con 0x55b0d0348b40 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.130+0000 7f28e8475740 1 -- 172.21.15.5:0/87039629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d0004490 con 0x55b0d0348b40 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.130+0000 7f28beffd640 1 -- 172.21.15.5:0/87039629 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f28d0004f80 con 0x55b0d0348b40 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.130+0000 7f28e8475740 10 client.19842 fetch_fsmap learned FSMap version 57 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.130+0000 7f28e8475740 1 -- 172.21.15.5:0/87039629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55b0d03cc0b0 con 0x55b0d0348b40 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.131+0000 7f28beffd640 1 -- 172.21.15.5:0/87039629 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28d0004490 con 0x55b0d0348b40 2024-09-19T06:20:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.131+0000 7f28beffd640 1 -- 172.21.15.5:0/87039629 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f28d0003c50 con 0x55b0d0348b40 2024-09-19T06:20:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.131+0000 7f28e8475740 10 client.19842 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T06:20:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.131+0000 7f28e8475740 10 client.19842 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:20:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.131+0000 7f28e8475740 20 client.19842 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.131+0000 7f28e8475740 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b0d03cc840 0x55b0d03cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.131+0000 7f28e8475740 4 client.19842 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:20:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.131+0000 7f28e8475740 1 -- 172.21.15.5:0/87039629 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0d02ba860 con 0x55b0d03cc840 2024-09-19T06:20:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.132+0000 7f28deffd640 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b0d03cc840 0x55b0d03cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.132+0000 7f28deffd640 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b0d03cc840 0x55b0d03cec60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aada87ceb6aca9a2 server_cookie=9300a2f65195a116 in_seq=0 out_seq=0 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.132+0000 7f28beffd640 10 client.19842 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28beffd640 1 -- 172.21.15.5:0/87039629 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0d02ba860 con 0x55b0d03cc840 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28beffd640 10 client.19842 handle_command_reply: tid=0 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28e8475740 1 client.19842 shutdown 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28e8475740 1 -- 172.21.15.5:0/87039629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0348b40 msgr2=0x55b0d035a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28e8475740 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0348b40 0x55b0d035a9c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f28d0005e00 tx=0x7f28d002a420 comp rx=0 tx=0).stop 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28e8475740 1 -- 172.21.15.5:0/87039629 shutdown_connections 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28e8475740 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55b0d03cc840 0x55b0d03cec60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28e8475740 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0d0352370 0x55b0d034b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28e8475740 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d0348b40 0x55b0d035a9c0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28e8475740 1 --2- 172.21.15.5:0/87039629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b0d03475d0 0x55b0d035a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28e8475740 1 -- 172.21.15.5:0/87039629 >> 172.21.15.5:0/87039629 conn(0x55b0d0310310 msgr2=0x55b0d0351090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28beffd640 0 client.19842 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:20:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28e8475740 1 -- 172.21.15.5:0/87039629 shutdown_connections 2024-09-19T06:20:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28e8475740 1 -- 172.21.15.5:0/87039629 wait complete. 2024-09-19T06:20:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.133+0000 7f28e8475740 20 client.19842 trim_cache size 0 max 16384 2024-09-19T06:20:32.139 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:20:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.134+0000 7f28e609c640 1 -- 172.21.15.5:0/3325299790 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f28c80a37b0 msgr2=0x7f28c80a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.134+0000 7f28e609c640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f28c80a37b0 0x7f28c80a5c70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f28d80cd4b0 tx=0x7f28d80bc040 comp rx=0 tx=0).stop 2024-09-19T06:20:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.134+0000 7f28e609c640 1 -- 172.21.15.5:0/3325299790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e01082a0 msgr2=0x7f28e019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.134+0000 7f28e609c640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e01082a0 0x7f28e019ef80 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f28d800eb10 tx=0x7f28d800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.134+0000 7f28e609c640 1 -- 172.21.15.5:0/3325299790 shutdown_connections 2024-09-19T06:20:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.134+0000 7f28e609c640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e01082a0 0x7f28e019ef80 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.134+0000 7f28e609c640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28e01036e0 0x7f28e019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.134+0000 7f28e609c640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f28c80a37b0 0x7f28c80a5c70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.134+0000 7f28e609c640 1 --2- 172.21.15.5:0/3325299790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e0102d30 0x7f28e019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.134+0000 7f28e609c640 1 -- 172.21.15.5:0/3325299790 >> 172.21.15.5:0/3325299790 conn(0x7f28e00fc7d0 msgr2=0x7f28e00fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.134+0000 7f28e609c640 1 -- 172.21.15.5:0/3325299790 shutdown_connections 2024-09-19T06:20:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:32.135+0000 7f28e609c640 1 -- 172.21.15.5:0/3325299790 wait complete. 2024-09-19T06:20:32.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39271 8.67 MB/sec execute 116 sec latency 73.886 ms 2024-09-19T06:20:32.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:32 smithi005 ceph-mon[30955]: pgmap v541: 97 pgs: 97 active+clean; 718 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 226 op/s 2024-09-19T06:20:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:32 smithi157 ceph-mon[35346]: pgmap v541: 97 pgs: 97 active+clean; 718 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 226 op/s 2024-09-19T06:20:32.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:20:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:32 smithi049 ceph-mon[34640]: pgmap v541: 97 pgs: 97 active+clean; 718 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 226 op/s 2024-09-19T06:20:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:20:33.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39460 8.73 MB/sec execute 117 sec latency 58.268 ms 2024-09-19T06:20:33.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.833+0000 7f63b1911640 1 -- 172.21.15.5:0/2250075546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63ac102570 msgr2=0x7f63ac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:33.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.833+0000 7f63b1911640 1 --2- 172.21.15.5:0/2250075546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63ac102570 0x7f63ac1029b0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009900 tx=0x7f63a002a480 comp rx=0 tx=0).stop 2024-09-19T06:20:33.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.834+0000 7f63b1911640 1 -- 172.21.15.5:0/2250075546 shutdown_connections 2024-09-19T06:20:33.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.834+0000 7f63b1911640 1 --2- 172.21.15.5:0/2250075546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63ac102ef0 0x7f63ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.834+0000 7f63b1911640 1 --2- 172.21.15.5:0/2250075546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63ac102570 0x7f63ac1029b0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.834+0000 7f63b1911640 1 --2- 172.21.15.5:0/2250075546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac100e20 0x7f63ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.834+0000 7f63b1911640 1 -- 172.21.15.5:0/2250075546 >> 172.21.15.5:0/2250075546 conn(0x7f63ac0fc7d0 msgr2=0x7f63ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:33.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.834+0000 7f63b1911640 1 -- 172.21.15.5:0/2250075546 shutdown_connections 2024-09-19T06:20:33.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.834+0000 7f63b1911640 1 -- 172.21.15.5:0/2250075546 wait complete. 2024-09-19T06:20:33.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.834+0000 7f63b1911640 1 Processor -- start 2024-09-19T06:20:33.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63b1911640 1 -- start start 2024-09-19T06:20:33.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63b1911640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63ac100e20 0x7f63ac196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63b1911640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac102570 0x7f63ac1967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63b1911640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63ac102ef0 0x7f63ac19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63b1911640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac10fd50 con 0x7f63ac102ef0 2024-09-19T06:20:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63b1911640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac10fbd0 con 0x7f63ac100e20 2024-09-19T06:20:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63b1911640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac10fed0 con 0x7f63ac102570 2024-09-19T06:20:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63aa7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac102570 0x7f63ac1967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63aa7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac102570 0x7f63ac1967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34790/0 (socket says 172.21.15.5:34790) 2024-09-19T06:20:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63aa7fc640 1 -- 172.21.15.5:0/2882872432 learned_addr learned my addr 172.21.15.5:0/2882872432 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63ab7fe640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63ac102ef0 0x7f63ac19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63aaffd640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63ac100e20 0x7f63ac196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.835+0000 7f63aa7fc640 1 -- 172.21.15.5:0/2882872432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63ac100e20 msgr2=0x7f63ac196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.836+0000 7f63aa7fc640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63ac100e20 0x7f63ac196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.836+0000 7f63aa7fc640 1 -- 172.21.15.5:0/2882872432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63ac102ef0 msgr2=0x7f63ac19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:33.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.836+0000 7f63aa7fc640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63ac102ef0 0x7f63ac19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.836+0000 7f63aa7fc640 1 -- 172.21.15.5:0/2882872432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac19b220 con 0x7f63ac102570 2024-09-19T06:20:33.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.836+0000 7f63aa7fc640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac102570 0x7f63ac1967b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f63a0006010 tx=0x7f63a0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:33.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.836+0000 7f63b090f640 1 -- 172.21.15.5:0/2882872432 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a003f030 con 0x7f63ac102570 2024-09-19T06:20:33.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.836+0000 7f63b1911640 1 -- 172.21.15.5:0/2882872432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ac19b4b0 con 0x7f63ac102570 2024-09-19T06:20:33.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.836+0000 7f63b1911640 1 -- 172.21.15.5:0/2882872432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ac1a2df0 con 0x7f63ac102570 2024-09-19T06:20:33.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.836+0000 7f63b090f640 1 -- 172.21.15.5:0/2882872432 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63a0003aa0 con 0x7f63ac102570 2024-09-19T06:20:33.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.837+0000 7f63b090f640 1 -- 172.21.15.5:0/2882872432 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0049060 con 0x7f63ac102570 2024-09-19T06:20:33.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.839+0000 7f63b090f640 1 -- 172.21.15.5:0/2882872432 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f63a0042030 con 0x7f63ac102570 2024-09-19T06:20:33.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.840+0000 7f63b3cea740 1 Processor -- start 2024-09-19T06:20:33.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.840+0000 7f63b3cea740 1 -- start start 2024-09-19T06:20:33.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.840+0000 7f63b3cea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6d99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.840+0000 7f63b3cea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4280 0x55ac8b6d46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.840+0000 7f63b3cea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6e0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.840+0000 7f63b3cea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ac8b6302f0 con 0x55ac8b6d4280 2024-09-19T06:20:33.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.840+0000 7f63b3cea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ac8b5d7cf0 con 0x55ac8b6d4c00 2024-09-19T06:20:33.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.840+0000 7f63b3cea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ac8b62c2b0 con 0x55ac8b6d95d0 2024-09-19T06:20:33.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63ab7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6d99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63aa7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6e0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63ab7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6d99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34802/0 (socket says 172.21.15.5:34802) 2024-09-19T06:20:33.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4280 0x55ac8b6d46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63ab7fe640 1 -- 172.21.15.5:0/3805595450 learned_addr learned my addr 172.21.15.5:0/3805595450 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:33.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63ab7fe640 1 -- 172.21.15.5:0/3805595450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d4c00 msgr2=0x55ac8b6e0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:33.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63ab7fe640 1 --2- 172.21.15.5:0/3805595450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6e0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63ab7fe640 1 -- 172.21.15.5:0/3805595450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4280 msgr2=0x55ac8b6d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:33.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63ab7fe640 1 --2- 172.21.15.5:0/3805595450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4280 0x55ac8b6d46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63ab7fe640 1 -- 172.21.15.5:0/3805595450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac8b5bc3a0 con 0x55ac8b6d95d0 2024-09-19T06:20:33.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63b090f640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63940a33c0 0x7f63940a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63ab7fe640 1 --2- 172.21.15.5:0/3805595450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6d99b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f63a4011440 tx=0x7f63a4011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abd8071bb817dde8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:33.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f63aaffd640 1 --2- 172.21.15.5:0/3805595450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4280 0x55ac8b6d46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:33.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.841+0000 7f6391ffb640 1 -- 172.21.15.5:0/3805595450 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4004d20 con 0x55ac8b6d95d0 2024-09-19T06:20:33.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.842+0000 7f6391ffb640 1 -- 172.21.15.5:0/3805595450 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63a4004050 con 0x55ac8b6d95d0 2024-09-19T06:20:33.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.842+0000 7f63aaffd640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63940a33c0 0x7f63940a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.842+0000 7f63b090f640 1 -- 172.21.15.5:0/2882872432 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f63a00f0180 con 0x7f63ac102570 2024-09-19T06:20:33.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.842+0000 7f6391ffb640 1 -- 172.21.15.5:0/3805595450 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a400f7a0 con 0x55ac8b6d95d0 2024-09-19T06:20:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.843+0000 7f63b3cea740 1 -- 172.21.15.5:0/3805595450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d95d0 msgr2=0x55ac8b6d99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.843+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3805595450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6d99b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f63a4011440 tx=0x7f63a4011900 comp rx=0 tx=0).stop 2024-09-19T06:20:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.843+0000 7f63aaffd640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63940a33c0 0x7f63940a5880 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55ac8b6c7a40 tx=0x7f639c009110 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.843+0000 7f63b3cea740 1 -- 172.21.15.5:0/3805595450 shutdown_connections 2024-09-19T06:20:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.843+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3805595450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6e0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.843+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3805595450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4280 0x55ac8b6d46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.843+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3805595450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6d99b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.843+0000 7f63b3cea740 1 -- 172.21.15.5:0/3805595450 >> 172.21.15.5:0/3805595450 conn(0x55ac8b693310 msgr2=0x55ac8b6c66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.844+0000 7f63b3cea740 1 -- 172.21.15.5:0/3805595450 shutdown_connections 2024-09-19T06:20:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.844+0000 7f63b3cea740 1 -- 172.21.15.5:0/3805595450 wait complete. 2024-09-19T06:20:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.844+0000 7f63b3cea740 1 -- start start 2024-09-19T06:20:33.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.844+0000 7f63b3cea740 1 Processor -- start 2024-09-19T06:20:33.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63b3cea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4280 0x55ac8b6e00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63b3cea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63b3cea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6dd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63b3cea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ac8b6e0990 con 0x55ac8b6d4c00 2024-09-19T06:20:33.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63b3cea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ac8b5eb060 con 0x55ac8b6d95d0 2024-09-19T06:20:33.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63b3cea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ac8b6e0b10 con 0x55ac8b6d4280 2024-09-19T06:20:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63aa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63b3cea740 10 client.0 _ll_register_callbacks cb 0x55ac8b33e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63aaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4280 0x55ac8b6e00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63aa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6dd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41472/0 (socket says 172.21.15.5:41472) 2024-09-19T06:20:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63aa7fc640 1 -- 172.21.15.5:0/3928915660 learned_addr learned my addr 172.21.15.5:0/3928915660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63ab7fe640 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6dd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63ab7fe640 1 -- 172.21.15.5:0/3928915660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4280 msgr2=0x55ac8b6e00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63ab7fe640 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4280 0x55ac8b6e00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63ab7fe640 1 -- 172.21.15.5:0/3928915660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 msgr2=0x55ac8b6dd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63ab7fe640 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6dd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63ab7fe640 1 -- 172.21.15.5:0/3928915660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac8b3a13c0 con 0x55ac8b6d95d0 2024-09-19T06:20:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.845+0000 7f63aa7fc640 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.846+0000 7f63ab7fe640 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6dd890 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f63a4006010 tx=0x7f63a40044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e563aba24fa74042 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.846+0000 7f638affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:20:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.846+0000 7f638affd640 1 -- 172.21.15.5:0/3928915660 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a401d040 con 0x55ac8b6d95d0 2024-09-19T06:20:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.846+0000 7f638affd640 1 -- 172.21.15.5:0/3928915660 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f63a4004c40 con 0x55ac8b6d95d0 2024-09-19T06:20:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.846+0000 7f63b3cea740 1 -- 172.21.15.5:0/3928915660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a4004050 con 0x55ac8b6d95d0 2024-09-19T06:20:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.846+0000 7f638affd640 1 -- 172.21.15.5:0/3928915660 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f63a4030050 con 0x55ac8b6d95d0 2024-09-19T06:20:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.846+0000 7f63b3cea740 10 client.29231 fetch_fsmap learned FSMap version 58 2024-09-19T06:20:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.846+0000 7f63b3cea740 1 -- 172.21.15.5:0/3928915660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55ac8b6ec300 con 0x55ac8b6d95d0 2024-09-19T06:20:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.847+0000 7f638affd640 1 -- 172.21.15.5:0/3928915660 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63a4004e40 con 0x55ac8b6d95d0 2024-09-19T06:20:33.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.847+0000 7f638affd640 1 -- 172.21.15.5:0/3928915660 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f63a402b030 con 0x55ac8b6d95d0 2024-09-19T06:20:33.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.847+0000 7f63b3cea740 10 client.29231 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:20:33.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.847+0000 7f63b3cea740 10 client.29231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:33.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.847+0000 7f63b3cea740 20 client.29231 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:33.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.847+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ac8b6ed760 0x55ac8b6efb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.847+0000 7f63b3cea740 4 client.29231 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:33.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.847+0000 7f63b3cea740 1 -- 172.21.15.5:0/3928915660 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f63a4004050 con 0x55ac8b6ed760 2024-09-19T06:20:33.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.847+0000 7f63aaffd640 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ac8b6ed760 0x55ac8b6efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.848+0000 7f638affd640 10 client.29231 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.848+0000 7f63aaffd640 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ac8b6ed760 0x55ac8b6efb50 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49248b4657d665db server_cookie=e735d4ee61891618 in_seq=0 out_seq=0 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.848+0000 7f638affd640 1 -- 172.21.15.5:0/3928915660 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f63a4004050 con 0x55ac8b6ed760 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.848+0000 7f638affd640 10 client.29231 handle_command_reply: tid=0 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.848+0000 7f63b3cea740 1 client.29231 shutdown 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.849+0000 7f63b3cea740 1 -- 172.21.15.5:0/3928915660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d95d0 msgr2=0x55ac8b6dd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.849+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6dd890 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f63a4006010 tx=0x7f63a40044b0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.849+0000 7f63b3cea740 1 -- 172.21.15.5:0/3928915660 shutdown_connections 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.849+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ac8b6ed760 0x55ac8b6efb50 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.849+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6dd890 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.849+0000 7f638affd640 0 client.29231 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.849+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.849+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3928915660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4280 0x55ac8b6e00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.849+0000 7f63b3cea740 1 -- 172.21.15.5:0/3928915660 >> 172.21.15.5:0/3928915660 conn(0x55ac8b693310 msgr2=0x55ac8b6d3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:33.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.849+0000 7f63b3cea740 1 -- 172.21.15.5:0/3928915660 shutdown_connections 2024-09-19T06:20:33.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.849+0000 7f63b3cea740 1 -- 172.21.15.5:0/3928915660 wait complete. 2024-09-19T06:20:33.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.849+0000 7f63b3cea740 20 client.29231 trim_cache size 0 max 16384 2024-09-19T06:20:33.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63b3cea740 1 Processor -- start 2024-09-19T06:20:33.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63b3cea740 1 -- start start 2024-09-19T06:20:33.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63b3cea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d4280 0x55ac8b6f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63b3cea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6f5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63b3cea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6d7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63b3cea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f639800e7e0 con 0x55ac8b6d95d0 2024-09-19T06:20:33.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63b3cea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6398010320 con 0x55ac8b6d4280 2024-09-19T06:20:33.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63b3cea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f639800eeb0 con 0x55ac8b6d4c00 2024-09-19T06:20:33.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6d7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6d7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41482/0 (socket says 172.21.15.5:41482) 2024-09-19T06:20:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63aa7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63ab7fe640 1 -- 172.21.15.5:0/1755206663 learned_addr learned my addr 172.21.15.5:0/1755206663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.869+0000 7f63aaffd640 1 --2- 172.21.15.5:0/1755206663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d4280 0x55ac8b6f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.870+0000 7f63ab7fe640 1 -- 172.21.15.5:0/1755206663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4c00 msgr2=0x55ac8b6f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.870+0000 7f63ab7fe640 1 --2- 172.21.15.5:0/1755206663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.870+0000 7f63ab7fe640 1 -- 172.21.15.5:0/1755206663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d4280 msgr2=0x55ac8b6f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.870+0000 7f63ab7fe640 1 --2- 172.21.15.5:0/1755206663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d4280 0x55ac8b6f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.870+0000 7f63ab7fe640 1 -- 172.21.15.5:0/1755206663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac8b6db8a0 con 0x55ac8b6d95d0 2024-09-19T06:20:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.870+0000 7f63aaffd640 1 --2- 172.21.15.5:0/1755206663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d4280 0x55ac8b6f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:20:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.870+0000 7f63ab7fe640 1 --2- 172.21.15.5:0/1755206663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6d7250 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f63a4011440 tx=0x7f63a4005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6031d4264df86fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.870+0000 7f63917fa640 1 -- 172.21.15.5:0/1755206663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4039070 con 0x55ac8b6d95d0 2024-09-19T06:20:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.870+0000 7f63917fa640 1 -- 172.21.15.5:0/1755206663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63a4015030 con 0x55ac8b6d95d0 2024-09-19T06:20:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.870+0000 7f63917fa640 1 -- 172.21.15.5:0/1755206663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a401d070 con 0x55ac8b6d95d0 2024-09-19T06:20:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.871+0000 7f63b3cea740 1 -- 172.21.15.5:0/1755206663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d95d0 msgr2=0x55ac8b6d7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.871+0000 7f63b3cea740 1 --2- 172.21.15.5:0/1755206663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6d7250 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f63a4011440 tx=0x7f63a4005080 comp rx=0 tx=0).stop 2024-09-19T06:20:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.871+0000 7f63b3cea740 1 -- 172.21.15.5:0/1755206663 shutdown_connections 2024-09-19T06:20:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.871+0000 7f63b3cea740 1 --2- 172.21.15.5:0/1755206663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6d7250 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.871+0000 7f63b3cea740 1 --2- 172.21.15.5:0/1755206663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.871+0000 7f63b3cea740 1 --2- 172.21.15.5:0/1755206663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d4280 0x55ac8b6f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.871+0000 7f63b3cea740 1 -- 172.21.15.5:0/1755206663 >> 172.21.15.5:0/1755206663 conn(0x55ac8b693310 msgr2=0x55ac8b6dc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.871+0000 7f63b3cea740 1 -- 172.21.15.5:0/1755206663 shutdown_connections 2024-09-19T06:20:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.871+0000 7f63b3cea740 1 -- 172.21.15.5:0/1755206663 wait complete. 2024-09-19T06:20:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.872+0000 7f63b3cea740 1 -- start start 2024-09-19T06:20:33.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.872+0000 7f63b3cea740 1 Processor -- start 2024-09-19T06:20:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.872+0000 7f63b3cea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4280 0x55ac8b6e62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.872+0000 7f63b3cea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6f4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63b3cea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63b3cea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f639800fc50 con 0x55ac8b6d4c00 2024-09-19T06:20:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63b3cea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f639800f580 con 0x55ac8b6d95d0 2024-09-19T06:20:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63b3cea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f639800c7e0 con 0x55ac8b6d4280 2024-09-19T06:20:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63aa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63b3cea740 10 client.0 _ll_register_callbacks cb 0x7f639c0093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63aa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6f4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41496/0 (socket says 172.21.15.5:41496) 2024-09-19T06:20:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63aa7fc640 1 -- 172.21.15.5:0/3824462814 learned_addr learned my addr 172.21.15.5:0/3824462814 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63aaffd640 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4280 0x55ac8b6e62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63ab7fe640 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63aaffd640 1 -- 172.21.15.5:0/3824462814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d95d0 msgr2=0x55ac8b6f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:33.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63aaffd640 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63aaffd640 1 -- 172.21.15.5:0/3824462814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 msgr2=0x55ac8b6f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:33.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63aaffd640 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63aaffd640 1 -- 172.21.15.5:0/3824462814 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac8b6f5420 con 0x55ac8b6d4280 2024-09-19T06:20:33.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f63aaffd640 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4280 0x55ac8b6e62c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55ac8b6f0220 tx=0x7f639c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d43728151177bf3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:33.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.873+0000 7f638affd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:20:33.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.874+0000 7f638affd640 1 -- 172.21.15.5:0/3824462814 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c023e20 con 0x55ac8b6d4280 2024-09-19T06:20:33.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.874+0000 7f638affd640 1 -- 172.21.15.5:0/3824462814 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f639c02a4d0 con 0x55ac8b6d4280 2024-09-19T06:20:33.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.874+0000 7f63b3cea740 1 -- 172.21.15.5:0/3824462814 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a4015030 con 0x55ac8b6d4280 2024-09-19T06:20:33.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.874+0000 7f638affd640 1 -- 172.21.15.5:0/3824462814 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f639c0059c0 con 0x55ac8b6d4280 2024-09-19T06:20:33.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.874+0000 7f63b3cea740 10 client.29173 fetch_fsmap learned FSMap version 58 2024-09-19T06:20:33.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.874+0000 7f63b3cea740 1 -- 172.21.15.5:0/3824462814 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55ac8b758070 con 0x55ac8b6d4280 2024-09-19T06:20:33.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.875+0000 7f638affd640 1 -- 172.21.15.5:0/3824462814 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f639c01b040 con 0x55ac8b6d4280 2024-09-19T06:20:33.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.875+0000 7f638affd640 1 -- 172.21.15.5:0/3824462814 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f639c020040 con 0x55ac8b6d4280 2024-09-19T06:20:33.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.875+0000 7f63b3cea740 10 client.29173 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:20:33.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.875+0000 7f63b3cea740 10 client.29173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:33.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.875+0000 7f63b3cea740 20 client.29173 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:33.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.875+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ac8b6ed760 0x55ac8b75a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:33.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.875+0000 7f63b3cea740 4 client.29173 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:20:33.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.875+0000 7f63b3cea740 1 -- 172.21.15.5:0/3824462814 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f63a4015030 con 0x55ac8b6ed760 2024-09-19T06:20:33.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.875+0000 7f63aa7fc640 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ac8b6ed760 0x55ac8b75a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:33.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.875+0000 7f638affd640 10 client.29173 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:33.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.876+0000 7f63aa7fc640 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ac8b6ed760 0x55ac8b75a7b0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62b66a87e142e30 server_cookie=905a1b59d60d55a1 in_seq=0 out_seq=0 2024-09-19T06:20:33.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.876+0000 7f638affd640 1 -- 172.21.15.5:0/3824462814 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f63a4015030 con 0x55ac8b6ed760 2024-09-19T06:20:33.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.876+0000 7f638affd640 10 client.29173 handle_command_reply: tid=0 2024-09-19T06:20:33.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.876+0000 7f63b3cea740 1 client.29173 shutdown 2024-09-19T06:20:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.876+0000 7f63b3cea740 1 -- 172.21.15.5:0/3824462814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4280 msgr2=0x55ac8b6e62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.876+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4280 0x55ac8b6e62c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55ac8b6f0220 tx=0x7f639c014000 comp rx=0 tx=0).stop 2024-09-19T06:20:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.876+0000 7f63b3cea740 1 -- 172.21.15.5:0/3824462814 shutdown_connections 2024-09-19T06:20:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.877+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ac8b6ed760 0x55ac8b75a7b0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.877+0000 7f638affd640 0 client.29173 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.877+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac8b6d95d0 0x55ac8b6f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.877+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac8b6d4c00 0x55ac8b6f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.877+0000 7f63b3cea740 1 --2- 172.21.15.5:0/3824462814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac8b6d4280 0x55ac8b6e62c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.877+0000 7f63b3cea740 1 -- 172.21.15.5:0/3824462814 >> 172.21.15.5:0/3824462814 conn(0x55ac8b693310 msgr2=0x55ac8b6d7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:33.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.877+0000 7f63b3cea740 1 -- 172.21.15.5:0/3824462814 shutdown_connections 2024-09-19T06:20:33.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.877+0000 7f63b3cea740 1 -- 172.21.15.5:0/3824462814 wait complete. 2024-09-19T06:20:33.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.877+0000 7f63b3cea740 20 client.29173 trim_cache size 0 max 16384 2024-09-19T06:20:33.884 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:20:33.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.878+0000 7f63b1911640 1 -- 172.21.15.5:0/2882872432 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63940a33c0 msgr2=0x7f63940a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:33.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.878+0000 7f63b1911640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63940a33c0 0x7f63940a5880 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55ac8b6c7a40 tx=0x7f639c009110 comp rx=0 tx=0).stop 2024-09-19T06:20:33.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.878+0000 7f63b1911640 1 -- 172.21.15.5:0/2882872432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac102570 msgr2=0x7f63ac1967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:33.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.878+0000 7f63b1911640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac102570 0x7f63ac1967b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f63a0006010 tx=0x7f63a0004270 comp rx=0 tx=0).stop 2024-09-19T06:20:33.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.878+0000 7f63b1911640 1 -- 172.21.15.5:0/2882872432 shutdown_connections 2024-09-19T06:20:33.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.878+0000 7f63b1911640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63ac102ef0 0x7f63ac19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.878+0000 7f63b1911640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63940a33c0 0x7f63940a5880 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.878+0000 7f63b1911640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac102570 0x7f63ac1967b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.878+0000 7f63b1911640 1 --2- 172.21.15.5:0/2882872432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63ac100e20 0x7f63ac196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:33.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.878+0000 7f63b1911640 1 -- 172.21.15.5:0/2882872432 >> 172.21.15.5:0/2882872432 conn(0x7f63ac0fc7d0 msgr2=0x7f63ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:33.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.878+0000 7f63b1911640 1 -- 172.21.15.5:0/2882872432 shutdown_connections 2024-09-19T06:20:33.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:33.878+0000 7f63b1911640 1 -- 172.21.15.5:0/2882872432 wait complete. 2024-09-19T06:20:34.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39664 8.77 MB/sec execute 118 sec latency 68.186 ms 2024-09-19T06:20:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:34 smithi005 ceph-mon[30955]: pgmap v542: 97 pgs: 97 active+clean; 724 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 359 op/s 2024-09-19T06:20:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:34 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:20:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:34 smithi157 ceph-mon[35346]: pgmap v542: 97 pgs: 97 active+clean; 724 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 359 op/s 2024-09-19T06:20:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:34 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:20:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:34 smithi049 ceph-mon[34640]: pgmap v542: 97 pgs: 97 active+clean; 724 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 359 op/s 2024-09-19T06:20:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:34 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:20:35.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39836 8.77 MB/sec execute 119 sec latency 55.104 ms 2024-09-19T06:20:35.772 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:20:35.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:20:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:20:36.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39952 8.75 MB/sec execute 120 sec latency 98.761 ms 2024-09-19T06:20:36.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:36 smithi005 ceph-mon[30955]: pgmap v543: 97 pgs: 97 active+clean; 724 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-09-19T06:20:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:36 smithi157 ceph-mon[35346]: pgmap v543: 97 pgs: 97 active+clean; 724 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-09-19T06:20:36.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.764+0000 7f357813e640 1 -- 172.21.15.5:0/1827696594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3570100e20 msgr2=0x7f3570101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:36.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.764+0000 7f357813e640 1 --2- 172.21.15.5:0/1827696594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3570100e20 0x7f3570101200 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f356c009930 tx=0x7f356c02a4d0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.764+0000 7f357813e640 1 -- 172.21.15.5:0/1827696594 shutdown_connections 2024-09-19T06:20:36.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.764+0000 7f357813e640 1 --2- 172.21.15.5:0/1827696594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102ef0 0x7f357010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.764+0000 7f357813e640 1 --2- 172.21.15.5:0/1827696594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3570102570 0x7f35701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.764+0000 7f357813e640 1 --2- 172.21.15.5:0/1827696594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3570100e20 0x7f3570101200 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.764+0000 7f357813e640 1 -- 172.21.15.5:0/1827696594 >> 172.21.15.5:0/1827696594 conn(0x7f35700fc7d0 msgr2=0x7f35700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:36.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.764+0000 7f357813e640 1 -- 172.21.15.5:0/1827696594 shutdown_connections 2024-09-19T06:20:36.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.764+0000 7f357813e640 1 -- 172.21.15.5:0/1827696594 wait complete. 2024-09-19T06:20:36.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.765+0000 7f357813e640 1 Processor -- start 2024-09-19T06:20:36.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.765+0000 7f357813e640 1 -- start start 2024-09-19T06:20:36.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.765+0000 7f357813e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3570100e20 0x7f35701a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.765+0000 7f357813e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102570 0x7f35701a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.765+0000 7f357813e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3570102ef0 0x7f357010cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.765+0000 7f357813e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3570111f80 con 0x7f3570102570 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.765+0000 7f3575eda640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3570100e20 0x7f35701a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.765+0000 7f3575eda640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3570100e20 0x7f35701a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:45764/0 (socket says 172.21.15.5:45764) 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.765+0000 7f3575eda640 1 -- 172.21.15.5:0/441652189 learned_addr learned my addr 172.21.15.5:0/441652189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f35756d9640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102570 0x7f35701a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f357813e640 1 -- 172.21.15.5:0/441652189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3570111e00 con 0x7f3570100e20 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f35766db640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3570102ef0 0x7f357010cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f357813e640 1 -- 172.21.15.5:0/441652189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3570112100 con 0x7f3570102ef0 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f3575eda640 1 -- 172.21.15.5:0/441652189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3570102ef0 msgr2=0x7f357010cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f3575eda640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3570102ef0 0x7f357010cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f3575eda640 1 -- 172.21.15.5:0/441652189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102570 msgr2=0x7f35701a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:36.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f3575eda640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102570 0x7f35701a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f3575eda640 1 -- 172.21.15.5:0/441652189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357010d300 con 0x7f3570100e20 2024-09-19T06:20:36.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f3575eda640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3570100e20 0x7f35701a4660 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f356c005ab0 tx=0x7f356c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:36.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f3566ffd640 1 -- 172.21.15.5:0/441652189 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c03f070 con 0x7f3570100e20 2024-09-19T06:20:36.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f357813e640 1 -- 172.21.15.5:0/441652189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357010d590 con 0x7f3570100e20 2024-09-19T06:20:36.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.766+0000 7f3566ffd640 1 -- 172.21.15.5:0/441652189 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f356c031b50 con 0x7f3570100e20 2024-09-19T06:20:36.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.767+0000 7f3566ffd640 1 -- 172.21.15.5:0/441652189 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c035030 con 0x7f3570100e20 2024-09-19T06:20:36.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.767+0000 7f357813e640 1 -- 172.21.15.5:0/441652189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35701ab4c0 con 0x7f3570100e20 2024-09-19T06:20:36.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.770+0000 7f3566ffd640 1 -- 172.21.15.5:0/441652189 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f356c042070 con 0x7f3570100e20 2024-09-19T06:20:36.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.770+0000 7f357a517740 1 Processor -- start 2024-09-19T06:20:36.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.770+0000 7f357a517740 1 -- start start 2024-09-19T06:20:36.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.771+0000 7f357a517740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc211b60 0x5595cc211f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.771+0000 7f357a517740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc212480 0x5595cc21c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.771+0000 7f357a517740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc21cd40 0x5595cc21f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.771+0000 7f357a517740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5595cc177280 con 0x5595cc211b60 2024-09-19T06:20:36.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.771+0000 7f357a517740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5595cc11eb10 con 0x5595cc212480 2024-09-19T06:20:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.771+0000 7f357a517740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5595cc173240 con 0x5595cc21cd40 2024-09-19T06:20:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.771+0000 7f35756d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc21cd40 0x5595cc21f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.771+0000 7f35756d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc21cd40 0x5595cc21f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38490/0 (socket says 172.21.15.5:38490) 2024-09-19T06:20:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.771+0000 7f35756d9640 1 -- 172.21.15.5:0/333700064 learned_addr learned my addr 172.21.15.5:0/333700064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.771+0000 7f3575eda640 1 --2- 172.21.15.5:0/333700064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc212480 0x5595cc21c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.771+0000 7f35766db640 1 --2- 172.21.15.5:0/333700064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc211b60 0x5595cc211f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f35756d9640 1 -- 172.21.15.5:0/333700064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc212480 msgr2=0x5595cc21c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f35756d9640 1 --2- 172.21.15.5:0/333700064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc212480 0x5595cc21c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f35756d9640 1 -- 172.21.15.5:0/333700064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc211b60 msgr2=0x5595cc211f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f35756d9640 1 --2- 172.21.15.5:0/333700064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc211b60 0x5595cc211f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f35756d9640 1 -- 172.21.15.5:0/333700064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595cc12b1f0 con 0x5595cc21cd40 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f35766db640 1 --2- 172.21.15.5:0/333700064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc211b60 0x5595cc211f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f3566ffd640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f355c0a3430 0x7f355c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f35756d9640 1 --2- 172.21.15.5:0/333700064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc21cd40 0x5595cc21f130 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3560009c60 tx=0x7f3560011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=628553ecb47df52a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f3566ffd640 1 -- 172.21.15.5:0/441652189 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f356c0f0350 con 0x7f3570100e20 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f35766db640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f355c0a3430 0x7f355c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f355bfff640 1 -- 172.21.15.5:0/333700064 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560004230 con 0x5595cc21cd40 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f355bfff640 1 -- 172.21.15.5:0/333700064 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35600043d0 con 0x5595cc21cd40 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.772+0000 7f355bfff640 1 -- 172.21.15.5:0/333700064 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560004d40 con 0x5595cc21cd40 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.773+0000 7f357a517740 1 -- 172.21.15.5:0/333700064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc21cd40 msgr2=0x5595cc21f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.773+0000 7f357a517740 1 --2- 172.21.15.5:0/333700064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc21cd40 0x5595cc21f130 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3560009c60 tx=0x7f3560011810 comp rx=0 tx=0).stop 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.773+0000 7f35766db640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f355c0a3430 0x7f355c0a58f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3568005fd0 tx=0x7f3568009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.773+0000 7f357a517740 1 -- 172.21.15.5:0/333700064 shutdown_connections 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.773+0000 7f357a517740 1 --2- 172.21.15.5:0/333700064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc21cd40 0x5595cc21f130 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.773+0000 7f357a517740 1 --2- 172.21.15.5:0/333700064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc212480 0x5595cc21c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.773+0000 7f357a517740 1 --2- 172.21.15.5:0/333700064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc211b60 0x5595cc211f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.773+0000 7f357a517740 1 -- 172.21.15.5:0/333700064 >> 172.21.15.5:0/333700064 conn(0x5595cc20e0f0 msgr2=0x5595cc20e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.773+0000 7f357a517740 1 -- 172.21.15.5:0/333700064 shutdown_connections 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.773+0000 7f357a517740 1 -- 172.21.15.5:0/333700064 wait complete. 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.774+0000 7f357a517740 1 -- start start 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.774+0000 7f357a517740 1 Processor -- start 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.774+0000 7f357a517740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 0x5595cc219c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.774+0000 7f357a517740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc212480 0x5595cc21a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.774+0000 7f357a517740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc21a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.774+0000 7f357a517740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5595cc21f910 con 0x5595cc212480 2024-09-19T06:20:36.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.774+0000 7f357a517740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5595cc131f10 con 0x5595cc21cd40 2024-09-19T06:20:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.774+0000 7f357a517740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5595cc21fa90 con 0x5595cc211b60 2024-09-19T06:20:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f357a517740 10 client.0 _ll_register_callbacks cb 0x5595cc0faa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f3575eda640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 0x5595cc219c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f35756d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc212480 0x5595cc21a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f35766db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc21a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f35766db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc21a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:45782/0 (socket says 172.21.15.5:45782) 2024-09-19T06:20:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f35766db640 1 -- 172.21.15.5:0/3333654875 learned_addr learned my addr 172.21.15.5:0/3333654875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f3575eda640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 0x5595cc219c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38502/0 (socket says 172.21.15.5:38502) 2024-09-19T06:20:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f35766db640 1 -- 172.21.15.5:0/3333654875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 msgr2=0x5595cc219c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f35766db640 1 --2- 172.21.15.5:0/3333654875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 0x5595cc219c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f35766db640 1 -- 172.21.15.5:0/3333654875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc212480 msgr2=0x5595cc21a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f35766db640 1 --2- 172.21.15.5:0/3333654875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc212480 0x5595cc21a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f35766db640 1 -- 172.21.15.5:0/3333654875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595cbee86a0 con 0x5595cc21cd40 2024-09-19T06:20:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f35756d9640 1 --2- 172.21.15.5:0/3333654875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc212480 0x5595cc21a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:20:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f35766db640 1 --2- 172.21.15.5:0/3333654875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc21a6c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f35680099e0 tx=0x7f3568004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ed08e2021392d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.775+0000 7f35597fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:20:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.776+0000 7f35597fa640 1 -- 172.21.15.5:0/3333654875 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568011970 con 0x5595cc21cd40 2024-09-19T06:20:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.776+0000 7f357a517740 1 -- 172.21.15.5:0/3333654875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35600043d0 con 0x5595cc21cd40 2024-09-19T06:20:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.776+0000 7f35597fa640 1 -- 172.21.15.5:0/3333654875 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f3568011b10 con 0x5595cc21cd40 2024-09-19T06:20:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.776+0000 7f35597fa640 1 -- 172.21.15.5:0/3333654875 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f3568011d10 con 0x5595cc21cd40 2024-09-19T06:20:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.776+0000 7f357a517740 10 client.29249 fetch_fsmap learned FSMap version 58 2024-09-19T06:20:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.776+0000 7f357a517740 1 -- 172.21.15.5:0/3333654875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5595cc22b500 con 0x5595cc21cd40 2024-09-19T06:20:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.776+0000 7f35597fa640 1 -- 172.21.15.5:0/3333654875 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35600043d0 con 0x5595cc21cd40 2024-09-19T06:20:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.776+0000 7f35597fa640 1 -- 172.21.15.5:0/3333654875 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f3568020550 con 0x5595cc21cd40 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.776+0000 7f357a517740 10 client.29249 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.776+0000 7f357a517740 10 client.29249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.777+0000 7f357a517740 20 client.29249 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.777+0000 7f357a517740 1 --2- 172.21.15.5:0/3333654875 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5595cc22ccf0 0x5595cc22f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.777+0000 7f357a517740 4 client.29249 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.777+0000 7f357a517740 1 -- 172.21.15.5:0/3333654875 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595cc1847f0 con 0x5595cc22ccf0 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.777+0000 7f3575eda640 1 --2- 172.21.15.5:0/3333654875 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5595cc22ccf0 0x5595cc22f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.778+0000 7f3575eda640 1 --2- 172.21.15.5:0/3333654875 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5595cc22ccf0 0x5595cc22f1b0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2c8dd0a772f091d server_cookie=b3ad3c84160dcfa6 in_seq=0 out_seq=0 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.778+0000 7f35597fa640 10 client.29249 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.778+0000 7f35597fa640 1 -- 172.21.15.5:0/3333654875 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595cc1847f0 con 0x5595cc22ccf0 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.778+0000 7f35597fa640 10 client.29249 handle_command_reply: tid=0 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.778+0000 7f357a517740 1 client.29249 shutdown 2024-09-19T06:20:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.779+0000 7f357a517740 1 -- 172.21.15.5:0/3333654875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 msgr2=0x5595cc21a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.779+0000 7f357a517740 1 --2- 172.21.15.5:0/3333654875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc21a6c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f35680099e0 tx=0x7f3568004a40 comp rx=0 tx=0).stop 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.779+0000 7f357a517740 1 -- 172.21.15.5:0/3333654875 shutdown_connections 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.779+0000 7f357a517740 1 --2- 172.21.15.5:0/3333654875 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5595cc22ccf0 0x5595cc22f1b0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.779+0000 7f357a517740 1 --2- 172.21.15.5:0/3333654875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc21a6c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.779+0000 7f35597fa640 0 client.29249 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.779+0000 7f357a517740 1 --2- 172.21.15.5:0/3333654875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc212480 0x5595cc21a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.779+0000 7f357a517740 1 --2- 172.21.15.5:0/3333654875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 0x5595cc219c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.779+0000 7f357a517740 1 -- 172.21.15.5:0/3333654875 >> 172.21.15.5:0/3333654875 conn(0x5595cc20e0f0 msgr2=0x5595cc213e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.779+0000 7f357a517740 1 -- 172.21.15.5:0/3333654875 shutdown_connections 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.779+0000 7f357a517740 1 -- 172.21.15.5:0/3333654875 wait complete. 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.779+0000 7f357a517740 20 client.29249 trim_cache size 0 max 16384 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.810+0000 7f357a517740 1 Processor -- start 2024-09-19T06:20:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.810+0000 7f357a517740 1 -- start start 2024-09-19T06:20:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.810+0000 7f357a517740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 0x5595cc21baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.810+0000 7f357a517740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc212480 0x5595cc234ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f357a517740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc235000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f357a517740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35701b3dc0 con 0x5595cc212480 2024-09-19T06:20:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f357a517740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35701b5900 con 0x5595cc21cd40 2024-09-19T06:20:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f357a517740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35701b4490 con 0x5595cc211b60 2024-09-19T06:20:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f35766db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc235000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f3575eda640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 0x5595cc21baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f35756d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc212480 0x5595cc234ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f35766db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc235000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:45788/0 (socket says 172.21.15.5:45788) 2024-09-19T06:20:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f3575eda640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 0x5595cc21baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38514/0 (socket says 172.21.15.5:38514) 2024-09-19T06:20:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f35766db640 1 -- 172.21.15.5:0/1384272797 learned_addr learned my addr 172.21.15.5:0/1384272797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f3575eda640 1 -- 172.21.15.5:0/1384272797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 msgr2=0x5595cc235000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f3575eda640 1 --2- 172.21.15.5:0/1384272797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc235000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f3575eda640 1 -- 172.21.15.5:0/1384272797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc212480 msgr2=0x5595cc234ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f3575eda640 1 --2- 172.21.15.5:0/1384272797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc212480 0x5595cc234ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f3575eda640 1 -- 172.21.15.5:0/1384272797 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595cc2122b0 con 0x5595cc211b60 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.811+0000 7f3575eda640 1 --2- 172.21.15.5:0/1384272797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 0x5595cc21baa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f356c0eda20 tx=0x7f356c0f3d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a78fa856f63b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.812+0000 7f355b7fe640 1 -- 172.21.15.5:0/1384272797 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c0ef040 con 0x5595cc211b60 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.812+0000 7f355b7fe640 1 -- 172.21.15.5:0/1384272797 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f356c0bc660 con 0x5595cc211b60 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.812+0000 7f355b7fe640 1 -- 172.21.15.5:0/1384272797 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c049040 con 0x5595cc211b60 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.813+0000 7f357a517740 1 -- 172.21.15.5:0/1384272797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 msgr2=0x5595cc21baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.813+0000 7f357a517740 1 --2- 172.21.15.5:0/1384272797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 0x5595cc21baa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f356c0eda20 tx=0x7f356c0f3d80 comp rx=0 tx=0).stop 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.813+0000 7f357a517740 1 -- 172.21.15.5:0/1384272797 shutdown_connections 2024-09-19T06:20:36.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.813+0000 7f357a517740 1 --2- 172.21.15.5:0/1384272797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc235000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.813+0000 7f357a517740 1 --2- 172.21.15.5:0/1384272797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc212480 0x5595cc234ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.813+0000 7f357a517740 1 --2- 172.21.15.5:0/1384272797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc211b60 0x5595cc21baa0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.813+0000 7f357a517740 1 -- 172.21.15.5:0/1384272797 >> 172.21.15.5:0/1384272797 conn(0x5595cc20e0f0 msgr2=0x5595cc21b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:36.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.813+0000 7f357a517740 1 -- 172.21.15.5:0/1384272797 shutdown_connections 2024-09-19T06:20:36.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.813+0000 7f357a517740 1 -- 172.21.15.5:0/1384272797 wait complete. 2024-09-19T06:20:36.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.813+0000 7f357a517740 1 -- start start 2024-09-19T06:20:36.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.814+0000 7f357a517740 1 Processor -- start 2024-09-19T06:20:36.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.814+0000 7f357a517740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc211b60 0x5595cc216c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.814+0000 7f357a517740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc212480 0x5595cc2171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.814+0000 7f357a517740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc2176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.814+0000 7f357a517740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35701b5230 con 0x5595cc211b60 2024-09-19T06:20:36.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.814+0000 7f357a517740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35701b4b60 con 0x5595cc21cd40 2024-09-19T06:20:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.814+0000 7f357a517740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35701b1dc0 con 0x5595cc212480 2024-09-19T06:20:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.814+0000 7f35756d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc212480 0x5595cc2171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.814+0000 7f35766db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc2176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.814+0000 7f35756d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc212480 0x5595cc2171b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38522/0 (socket says 172.21.15.5:38522) 2024-09-19T06:20:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.814+0000 7f35766db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc2176f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:45804/0 (socket says 172.21.15.5:45804) 2024-09-19T06:20:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.814+0000 7f35756d9640 1 -- 172.21.15.5:0/583673563 learned_addr learned my addr 172.21.15.5:0/583673563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.815+0000 7f357a517740 10 client.0 _ll_register_callbacks cb 0x7f3568009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.815+0000 7f3575eda640 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc211b60 0x5595cc216c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.815+0000 7f35766db640 1 -- 172.21.15.5:0/583673563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc212480 msgr2=0x5595cc2171b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.815+0000 7f35766db640 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc212480 0x5595cc2171b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.815+0000 7f35766db640 1 -- 172.21.15.5:0/583673563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc211b60 msgr2=0x5595cc216c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.815+0000 7f35766db640 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc211b60 0x5595cc216c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.815+0000 7f35766db640 1 -- 172.21.15.5:0/583673563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595cc296a90 con 0x5595cc21cd40 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.815+0000 7f3575eda640 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc211b60 0x5595cc216c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.815+0000 7f35766db640 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc2176f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f35680099e0 tx=0x7f35680206e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=656408817f466a2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.815+0000 7f35597fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.816+0000 7f35597fa640 1 -- 172.21.15.5:0/583673563 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568011770 con 0x5595cc21cd40 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.816+0000 7f35597fa640 1 -- 172.21.15.5:0/583673563 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f3568011910 con 0x5595cc21cd40 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.816+0000 7f357a517740 1 -- 172.21.15.5:0/583673563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f356c0bc660 con 0x5595cc21cd40 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.816+0000 7f35597fa640 1 -- 172.21.15.5:0/583673563 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f3568011b10 con 0x5595cc21cd40 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.816+0000 7f357a517740 10 client.29255 fetch_fsmap learned FSMap version 58 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.816+0000 7f357a517740 1 -- 172.21.15.5:0/583673563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5595cc297d90 con 0x5595cc21cd40 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.817+0000 7f35597fa640 1 -- 172.21.15.5:0/583673563 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3568013e50 con 0x5595cc21cd40 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.817+0000 7f35597fa640 1 -- 172.21.15.5:0/583673563 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f3568029500 con 0x5595cc21cd40 2024-09-19T06:20:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.817+0000 7f357a517740 10 client.29255 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:20:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.817+0000 7f357a517740 10 client.29255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.817+0000 7f357a517740 20 client.29255 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.817+0000 7f357a517740 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5595cc2984f0 0x5595cc29a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.817+0000 7f357a517740 4 client.29255 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:20:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.817+0000 7f357a517740 1 -- 172.21.15.5:0/583673563 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f356c0bc660 con 0x5595cc2984f0 2024-09-19T06:20:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.818+0000 7f3575eda640 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5595cc2984f0 0x5595cc29a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.819+0000 7f3575eda640 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5595cc2984f0 0x5595cc29a910 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cbae1ce0f9260ba server_cookie=7f7f70f911a81fa6 in_seq=0 out_seq=0 2024-09-19T06:20:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.819+0000 7f35597fa640 10 client.29255 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f35597fa640 1 -- 172.21.15.5:0/583673563 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f356c0bc660 con 0x5595cc2984f0 2024-09-19T06:20:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f35597fa640 10 client.29255 handle_command_reply: tid=0 2024-09-19T06:20:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f357a517740 1 client.29255 shutdown 2024-09-19T06:20:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f357a517740 1 -- 172.21.15.5:0/583673563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 msgr2=0x5595cc2176f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f357a517740 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc2176f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f35680099e0 tx=0x7f35680206e0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f357a517740 1 -- 172.21.15.5:0/583673563 shutdown_connections 2024-09-19T06:20:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f357a517740 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5595cc2984f0 0x5595cc29a910 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f357a517740 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595cc21cd40 0x5595cc2176f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f357a517740 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595cc212480 0x5595cc2171b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f357a517740 1 --2- 172.21.15.5:0/583673563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595cc211b60 0x5595cc216c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f357a517740 1 -- 172.21.15.5:0/583673563 >> 172.21.15.5:0/583673563 conn(0x5595cc20e0f0 msgr2=0x5595cc213e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:36.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f35597fa640 0 client.29255 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:36.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f357a517740 1 -- 172.21.15.5:0/583673563 shutdown_connections 2024-09-19T06:20:36.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.821+0000 7f357a517740 1 -- 172.21.15.5:0/583673563 wait complete. 2024-09-19T06:20:36.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.822+0000 7f357a517740 20 client.29255 trim_cache size 0 max 16384 2024-09-19T06:20:36.839 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:20:36.839 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:20:36.839 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "227cfcdf-9f28-4609-8f0a-c8227db218bd", 2024-09-19T06:20:36.840 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:20:36.840 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:20:36.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.823+0000 7f357813e640 1 -- 172.21.15.5:0/441652189 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f355c0a3430 msgr2=0x7f355c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:36.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.823+0000 7f357813e640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f355c0a3430 0x7f355c0a58f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3568005fd0 tx=0x7f3568009500 comp rx=0 tx=0).stop 2024-09-19T06:20:36.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.823+0000 7f357813e640 1 -- 172.21.15.5:0/441652189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3570100e20 msgr2=0x7f35701a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:36.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.823+0000 7f357813e640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3570100e20 0x7f35701a4660 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f356c005ab0 tx=0x7f356c003b90 comp rx=0 tx=0).stop 2024-09-19T06:20:36.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.823+0000 7f357813e640 1 -- 172.21.15.5:0/441652189 shutdown_connections 2024-09-19T06:20:36.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.823+0000 7f357813e640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f355c0a3430 0x7f355c0a58f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.823+0000 7f357813e640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3570102ef0 0x7f357010cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.823+0000 7f357813e640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102570 0x7f35701a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.823+0000 7f357813e640 1 --2- 172.21.15.5:0/441652189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3570100e20 0x7f35701a4660 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:36.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.823+0000 7f357813e640 1 -- 172.21.15.5:0/441652189 >> 172.21.15.5:0/441652189 conn(0x7f35700fc7d0 msgr2=0x7f357010e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:36.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.823+0000 7f357813e640 1 -- 172.21.15.5:0/441652189 shutdown_connections 2024-09-19T06:20:36.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:36.823+0000 7f357813e640 1 -- 172.21.15.5:0/441652189 wait complete. 2024-09-19T06:20:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:36 smithi049 ceph-mon[34640]: pgmap v543: 97 pgs: 97 active+clean; 724 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-09-19T06:20:37.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 40132 8.81 MB/sec execute 121 sec latency 126.258 ms 2024-09-19T06:20:37.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:37 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:20:37.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:20:37.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:20:37.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:20:37.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:37 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:20:37.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.590 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:37 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.591 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:20:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:37 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:20:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:20:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:20:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:20:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:37 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:20:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:37 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:37 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:20:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:20:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:20:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:20:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:37 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:20:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:37 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:20:38.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 40311 8.85 MB/sec execute 122 sec latency 83.418 ms 2024-09-19T06:20:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.583+0000 7fd114d77640 1 -- 172.21.15.5:0/49846364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd110106f10 msgr2=0x7fd11010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.583+0000 7fd114d77640 1 --2- 172.21.15.5:0/49846364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd110106f10 0x7fd11010ea90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fd108009ca0 tx=0x7fd10802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.584+0000 7fd114d77640 1 -- 172.21.15.5:0/49846364 shutdown_connections 2024-09-19T06:20:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.584+0000 7fd114d77640 1 --2- 172.21.15.5:0/49846364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd110106f10 0x7fd11010ea90 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.584+0000 7fd114d77640 1 --2- 172.21.15.5:0/49846364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd110102790 0x7fd1101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.584+0000 7fd114d77640 1 --2- 172.21.15.5:0/49846364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd110101de0 0x7fd1101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.584+0000 7fd114d77640 1 -- 172.21.15.5:0/49846364 >> 172.21.15.5:0/49846364 conn(0x7fd1100fd8e0 msgr2=0x7fd1100ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.584+0000 7fd114d77640 1 -- 172.21.15.5:0/49846364 shutdown_connections 2024-09-19T06:20:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.584+0000 7fd114d77640 1 -- 172.21.15.5:0/49846364 wait complete. 2024-09-19T06:20:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.585+0000 7fd114d77640 1 Processor -- start 2024-09-19T06:20:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.585+0000 7fd114d77640 1 -- start start 2024-09-19T06:20:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.585+0000 7fd114d77640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd110101de0 0x7fd11010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.585+0000 7fd114d77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd110102790 0x7fd11010e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.585+0000 7fd114d77640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd110106f10 0x7fd1101a3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.585+0000 7fd114d77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd110111e20 con 0x7fd110102790 2024-09-19T06:20:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.585+0000 7fd114d77640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd110111ca0 con 0x7fd110101de0 2024-09-19T06:20:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.585+0000 7fd114d77640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd110111fa0 con 0x7fd110106f10 2024-09-19T06:20:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.586+0000 7fd10dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd110102790 0x7fd11010e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.586+0000 7fd10dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd110102790 0x7fd11010e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35404/0 (socket says 172.21.15.5:35404) 2024-09-19T06:20:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.586+0000 7fd10ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd110106f10 0x7fd1101a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.586+0000 7fd10dd9b640 1 -- 172.21.15.5:0/375520994 learned_addr learned my addr 172.21.15.5:0/375520994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.586+0000 7fd10e59c640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd110101de0 0x7fd11010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.586+0000 7fd10dd9b640 1 -- 172.21.15.5:0/375520994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd110106f10 msgr2=0x7fd1101a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:38.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.586+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd110106f10 0x7fd1101a3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.586+0000 7fd10dd9b640 1 -- 172.21.15.5:0/375520994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd110101de0 msgr2=0x7fd11010deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:38.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.586+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd110101de0 0x7fd11010deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.586+0000 7fd10dd9b640 1 -- 172.21.15.5:0/375520994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1101a3940 con 0x7fd110102790 2024-09-19T06:20:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.586+0000 7fd10ed9d640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd110106f10 0x7fd1101a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.586+0000 7fd10e59c640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd110101de0 0x7fd11010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.587+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd110102790 0x7fd11010e3f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd10000e9f0 tx=0x7fd10000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.587+0000 7fd0ff7fe640 1 -- 172.21.15.5:0/375520994 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10000cd80 con 0x7fd110102790 2024-09-19T06:20:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.587+0000 7fd0ff7fe640 1 -- 172.21.15.5:0/375520994 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1000054d0 con 0x7fd110102790 2024-09-19T06:20:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.587+0000 7fd0ff7fe640 1 -- 172.21.15.5:0/375520994 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100010490 con 0x7fd110102790 2024-09-19T06:20:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.587+0000 7fd114d77640 1 -- 172.21.15.5:0/375520994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1101a3c30 con 0x7fd110102790 2024-09-19T06:20:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.587+0000 7fd114d77640 1 -- 172.21.15.5:0/375520994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1101ab4c0 con 0x7fd110102790 2024-09-19T06:20:38.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.591+0000 7fd117150740 1 Processor -- start 2024-09-19T06:20:38.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.591+0000 7fd0ff7fe640 1 -- 172.21.15.5:0/375520994 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd100005020 con 0x7fd110102790 2024-09-19T06:20:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.591+0000 7fd117150740 1 -- start start 2024-09-19T06:20:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.591+0000 7fd117150740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636bf4a0 0x55b8636bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.591+0000 7fd117150740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 0x55b8636ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.591+0000 7fd117150740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636ca680 0x55b8636cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.591+0000 7fd117150740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b8636252e0 con 0x55b8636bf4a0 2024-09-19T06:20:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.591+0000 7fd117150740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b8635ccce0 con 0x55b8636ca680 2024-09-19T06:20:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.591+0000 7fd117150740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b8636212a0 con 0x55b8636bfdc0 2024-09-19T06:20:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd10e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 0x55b8636ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd10e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 0x55b8636ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38540/0 (socket says 172.21.15.5:38540) 2024-09-19T06:20:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd10e59c640 1 -- 172.21.15.5:0/637559710 learned_addr learned my addr 172.21.15.5:0/637559710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd10dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636ca680 0x55b8636cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd10dd9b640 1 -- 172.21.15.5:0/637559710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 msgr2=0x55b8636ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/637559710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 0x55b8636ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd10dd9b640 1 -- 172.21.15.5:0/637559710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636bf4a0 msgr2=0x55b8636bf880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:20:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd0ff7fe640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0f80a34b0 0x7fd0f80a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/637559710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636bf4a0 0x55b8636bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd10dd9b640 1 -- 172.21.15.5:0/637559710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8635b1440 con 0x55b8636ca680 2024-09-19T06:20:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd10e59c640 1 --2- 172.21.15.5:0/637559710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 0x55b8636ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd10e59c640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0f80a34b0 0x7fd0f80a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/637559710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636ca680 0x55b8636cca70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd1000cd700 tx=0x7fd100098530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a08121e179ac9fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.592+0000 7fd0ff7fe640 1 -- 172.21.15.5:0/375520994 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7fd1000d0030 con 0x7fd110102790 2024-09-19T06:20:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.593+0000 7fd0fcff9640 1 -- 172.21.15.5:0/637559710 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10009c060 con 0x55b8636ca680 2024-09-19T06:20:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.593+0000 7fd0fcff9640 1 -- 172.21.15.5:0/637559710 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd100005310 con 0x55b8636ca680 2024-09-19T06:20:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.593+0000 7fd0fcff9640 1 -- 172.21.15.5:0/637559710 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10009bbf0 con 0x55b8636ca680 2024-09-19T06:20:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.593+0000 7fd117150740 1 -- 172.21.15.5:0/637559710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636ca680 msgr2=0x55b8636cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.593+0000 7fd10e59c640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0f80a34b0 0x7fd0f80a5970 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55b8636bc890 tx=0x7fd104005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.593+0000 7fd117150740 1 --2- 172.21.15.5:0/637559710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636ca680 0x55b8636cca70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd1000cd700 tx=0x7fd100098530 comp rx=0 tx=0).stop 2024-09-19T06:20:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.594+0000 7fd117150740 1 -- 172.21.15.5:0/637559710 shutdown_connections 2024-09-19T06:20:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.594+0000 7fd117150740 1 --2- 172.21.15.5:0/637559710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636ca680 0x55b8636cca70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.594+0000 7fd117150740 1 --2- 172.21.15.5:0/637559710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 0x55b8636ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.594+0000 7fd117150740 1 --2- 172.21.15.5:0/637559710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636bf4a0 0x55b8636bf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.594+0000 7fd117150740 1 -- 172.21.15.5:0/637559710 >> 172.21.15.5:0/637559710 conn(0x55b8636bba30 msgr2=0x55b8636bbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.594+0000 7fd117150740 1 -- 172.21.15.5:0/637559710 shutdown_connections 2024-09-19T06:20:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.594+0000 7fd117150740 1 -- 172.21.15.5:0/637559710 wait complete. 2024-09-19T06:20:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.594+0000 7fd117150740 1 -- start start 2024-09-19T06:20:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.594+0000 7fd117150740 1 Processor -- start 2024-09-19T06:20:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd117150740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 0x55b8636ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd117150740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636bfdc0 0x55b8636c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd117150740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636ca680 0x55b8636c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd117150740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b8636cd250 con 0x55b8636ca680 2024-09-19T06:20:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd117150740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b8635e0050 con 0x55b8636bfdc0 2024-09-19T06:20:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd117150740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b8636cd3d0 con 0x55b8636bf4a0 2024-09-19T06:20:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd10e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 0x55b8636ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd10e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 0x55b8636ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38546/0 (socket says 172.21.15.5:38546) 2024-09-19T06:20:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd10e59c640 1 -- 172.21.15.5:0/3126799918 learned_addr learned my addr 172.21.15.5:0/3126799918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd10ed9d640 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636ca680 0x55b8636c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636bfdc0 0x55b8636c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd117150740 10 client.0 _ll_register_callbacks cb 0x55b863620160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd10e59c640 1 -- 172.21.15.5:0/3126799918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636bfdc0 msgr2=0x55b8636c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd10e59c640 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636bfdc0 0x55b8636c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd10e59c640 1 -- 172.21.15.5:0/3126799918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636ca680 msgr2=0x55b8636c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd10e59c640 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636ca680 0x55b8636c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.595+0000 7fd10e59c640 1 -- 172.21.15.5:0/3126799918 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8633963c0 con 0x55b8636bf4a0 2024-09-19T06:20:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.596+0000 7fd10e59c640 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 0x55b8636ca020 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd10400fc90 tx=0x7fd104004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69467457fbe85dab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.596+0000 7fd0f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:20:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.596+0000 7fd0f5ffb640 1 -- 172.21.15.5:0/3126799918 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10402e060 con 0x55b8636bf4a0 2024-09-19T06:20:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.596+0000 7fd0f5ffb640 1 -- 172.21.15.5:0/3126799918 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7fd104018030 con 0x55b8636bf4a0 2024-09-19T06:20:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.596+0000 7fd117150740 1 -- 172.21.15.5:0/3126799918 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b863632850 con 0x55b8636bf4a0 2024-09-19T06:20:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.596+0000 7fd0f5ffb640 1 -- 172.21.15.5:0/3126799918 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fd10400fe40 con 0x55b8636bf4a0 2024-09-19T06:20:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.596+0000 7fd117150740 10 client.29203 fetch_fsmap learned FSMap version 58 2024-09-19T06:20:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.596+0000 7fd117150740 1 -- 172.21.15.5:0/3126799918 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55b8636d8ce0 con 0x55b8636bf4a0 2024-09-19T06:20:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.597+0000 7fd0f5ffb640 1 -- 172.21.15.5:0/3126799918 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b863632850 con 0x55b8636bf4a0 2024-09-19T06:20:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.597+0000 7fd0f5ffb640 1 -- 172.21.15.5:0/3126799918 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fd104031aa0 con 0x55b8636bf4a0 2024-09-19T06:20:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.597+0000 7fd117150740 10 client.29203 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:20:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.597+0000 7fd117150740 10 client.29203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.597+0000 7fd117150740 20 client.29203 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.597+0000 7fd117150740 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b8636da1a0 0x55b8636dc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.597+0000 7fd117150740 4 client.29203 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.597+0000 7fd117150740 1 -- 172.21.15.5:0/3126799918 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b86362cd60 con 0x55b8636da1a0 2024-09-19T06:20:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.597+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b8636da1a0 0x55b8636dc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.598+0000 7fd0f5ffb640 10 client.29203 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:38.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.598+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b8636da1a0 0x55b8636dc590 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b71fa485a058a81e server_cookie=d76a9eb1d40a62e8 in_seq=0 out_seq=0 2024-09-19T06:20:38.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd0f5ffb640 1 -- 172.21.15.5:0/3126799918 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b86362cd60 con 0x55b8636da1a0 2024-09-19T06:20:38.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd0f5ffb640 10 client.29203 handle_command_reply: tid=0 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd117150740 1 client.29203 shutdown 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd117150740 1 -- 172.21.15.5:0/3126799918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 msgr2=0x55b8636ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd117150740 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 0x55b8636ca020 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd10400fc90 tx=0x7fd104004730 comp rx=0 tx=0).stop 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd117150740 1 -- 172.21.15.5:0/3126799918 shutdown_connections 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd117150740 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b8636da1a0 0x55b8636dc590 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd117150740 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636ca680 0x55b8636c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd0f5ffb640 0 client.29203 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd117150740 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636bfdc0 0x55b8636c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd117150740 1 --2- 172.21.15.5:0/3126799918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 0x55b8636ca020 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd117150740 1 -- 172.21.15.5:0/3126799918 >> 172.21.15.5:0/3126799918 conn(0x55b8636bba30 msgr2=0x55b8636c1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd117150740 1 -- 172.21.15.5:0/3126799918 shutdown_connections 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.599+0000 7fd117150740 1 -- 172.21.15.5:0/3126799918 wait complete. 2024-09-19T06:20:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.600+0000 7fd117150740 20 client.29203 trim_cache size 0 max 16384 2024-09-19T06:20:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.627+0000 7fd117150740 1 Processor -- start 2024-09-19T06:20:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.628+0000 7fd117150740 1 -- start start 2024-09-19T06:20:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.628+0000 7fd117150740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 0x55b8636d6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.628+0000 7fd117150740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636bfdc0 0x55b8636c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.628+0000 7fd117150740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636ca680 0x55b8636e1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.628+0000 7fd117150740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b3e90 con 0x55b8636ca680 2024-09-19T06:20:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.628+0000 7fd117150740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b59d0 con 0x55b8636bfdc0 2024-09-19T06:20:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.628+0000 7fd117150740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b4560 con 0x55b8636bf4a0 2024-09-19T06:20:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.628+0000 7fd10ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636ca680 0x55b8636e1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.628+0000 7fd10ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636ca680 0x55b8636e1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35426/0 (socket says 172.21.15.5:35426) 2024-09-19T06:20:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.628+0000 7fd10ed9d640 1 -- 172.21.15.5:0/409216379 learned_addr learned my addr 172.21.15.5:0/409216379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.628+0000 7fd10e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 0x55b8636d6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.628+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/409216379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636bfdc0 0x55b8636c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.629+0000 7fd10ed9d640 1 -- 172.21.15.5:0/409216379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 msgr2=0x55b8636d6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.629+0000 7fd10ed9d640 1 --2- 172.21.15.5:0/409216379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 0x55b8636d6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.629+0000 7fd10ed9d640 1 -- 172.21.15.5:0/409216379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636bfdc0 msgr2=0x55b8636c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.629+0000 7fd10ed9d640 1 --2- 172.21.15.5:0/409216379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636bfdc0 0x55b8636c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.629+0000 7fd10ed9d640 1 -- 172.21.15.5:0/409216379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8636c62c0 con 0x55b8636ca680 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.629+0000 7fd10e59c640 1 --2- 172.21.15.5:0/409216379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 0x55b8636d6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.629+0000 7fd10ed9d640 1 --2- 172.21.15.5:0/409216379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636ca680 0x55b8636e1df0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fd108009ca0 tx=0x7fd1080098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eceb7134f71a00b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.629+0000 7fd0f7fff640 1 -- 172.21.15.5:0/409216379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108031a40 con 0x55b8636ca680 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.629+0000 7fd0f7fff640 1 -- 172.21.15.5:0/409216379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd108036050 con 0x55b8636ca680 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.629+0000 7fd0f7fff640 1 -- 172.21.15.5:0/409216379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108004600 con 0x55b8636ca680 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.630+0000 7fd117150740 1 -- 172.21.15.5:0/409216379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636ca680 msgr2=0x55b8636e1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.630+0000 7fd117150740 1 --2- 172.21.15.5:0/409216379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636ca680 0x55b8636e1df0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fd108009ca0 tx=0x7fd1080098b0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.630+0000 7fd117150740 1 -- 172.21.15.5:0/409216379 shutdown_connections 2024-09-19T06:20:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.630+0000 7fd117150740 1 --2- 172.21.15.5:0/409216379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636ca680 0x55b8636e1df0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.630+0000 7fd117150740 1 --2- 172.21.15.5:0/409216379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636bfdc0 0x55b8636c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.630+0000 7fd117150740 1 --2- 172.21.15.5:0/409216379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bf4a0 0x55b8636d6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.630+0000 7fd117150740 1 -- 172.21.15.5:0/409216379 >> 172.21.15.5:0/409216379 conn(0x55b8636bba30 msgr2=0x55b8636c28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.630+0000 7fd117150740 1 -- 172.21.15.5:0/409216379 shutdown_connections 2024-09-19T06:20:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.631+0000 7fd117150740 1 -- 172.21.15.5:0/409216379 wait complete. 2024-09-19T06:20:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.631+0000 7fd117150740 1 -- start start 2024-09-19T06:20:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.631+0000 7fd117150740 1 Processor -- start 2024-09-19T06:20:38.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd117150740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636bf4a0 0x55b8636c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd117150740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 0x55b8636c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd117150740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636ca680 0x55b8636c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd117150740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b5300 con 0x55b8636bf4a0 2024-09-19T06:20:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd117150740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b4c30 con 0x55b8636ca680 2024-09-19T06:20:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd117150740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b1e90 con 0x55b8636bfdc0 2024-09-19T06:20:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd117150740 10 client.0 _ll_register_callbacks cb 0x7fd104012ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd10dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 0x55b8636c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd10e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636bf4a0 0x55b8636c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd10dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 0x55b8636c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38552/0 (socket says 172.21.15.5:38552) 2024-09-19T06:20:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd10dd9b640 1 -- 172.21.15.5:0/694292886 learned_addr learned my addr 172.21.15.5:0/694292886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd10ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636ca680 0x55b8636c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd10dd9b640 1 -- 172.21.15.5:0/694292886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636ca680 msgr2=0x55b8636c4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/694292886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636ca680 0x55b8636c4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd10dd9b640 1 -- 172.21.15.5:0/694292886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636bf4a0 msgr2=0x55b8636c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/694292886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636bf4a0 0x55b8636c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.632+0000 7fd10dd9b640 1 -- 172.21.15.5:0/694292886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8636ccde0 con 0x55b8636bfdc0 2024-09-19T06:20:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.633+0000 7fd10dd9b640 1 --2- 172.21.15.5:0/694292886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 0x55b8636c4a00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd10009bf90 tx=0x7fd1000cd6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d866fab2c0117f35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.633+0000 7fd0f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:20:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.633+0000 7fd0f5ffb640 1 -- 172.21.15.5:0/694292886 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10009bdd0 con 0x55b8636bfdc0 2024-09-19T06:20:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.633+0000 7fd0f5ffb640 1 -- 172.21.15.5:0/694292886 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7fd10009a590 con 0x55b8636bfdc0 2024-09-19T06:20:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.633+0000 7fd117150740 1 -- 172.21.15.5:0/694292886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd108036050 con 0x55b8636bfdc0 2024-09-19T06:20:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.633+0000 7fd0f5ffb640 1 -- 172.21.15.5:0/694292886 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fd10009bbf0 con 0x55b8636bfdc0 2024-09-19T06:20:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.633+0000 7fd117150740 10 client.29209 fetch_fsmap learned FSMap version 58 2024-09-19T06:20:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.633+0000 7fd117150740 1 -- 172.21.15.5:0/694292886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55b8637450e0 con 0x55b8636bfdc0 2024-09-19T06:20:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.633+0000 7fd0f5ffb640 1 -- 172.21.15.5:0/694292886 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd108036050 con 0x55b8636bfdc0 2024-09-19T06:20:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.634+0000 7fd0f5ffb640 1 -- 172.21.15.5:0/694292886 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fd100098900 con 0x55b8636bfdc0 2024-09-19T06:20:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.634+0000 7fd117150740 10 client.29209 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:20:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.634+0000 7fd117150740 10 client.29209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.634+0000 7fd117150740 20 client.29209 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.634+0000 7fd117150740 1 --2- 172.21.15.5:0/694292886 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b8636da1a0 0x55b863747430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.634+0000 7fd117150740 4 client.29209 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:20:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.634+0000 7fd117150740 1 -- 172.21.15.5:0/694292886 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b8635a9210 con 0x55b8636da1a0 2024-09-19T06:20:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.634+0000 7fd10e59c640 1 --2- 172.21.15.5:0/694292886 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b8636da1a0 0x55b863747430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd0f5ffb640 10 client.29209 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd10e59c640 1 --2- 172.21.15.5:0/694292886 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b8636da1a0 0x55b863747430 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd37e9f58ee5f684 server_cookie=f39991757fd6f879 in_seq=0 out_seq=0 2024-09-19T06:20:38.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd0f5ffb640 1 -- 172.21.15.5:0/694292886 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b8635a9210 con 0x55b8636da1a0 2024-09-19T06:20:38.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd0f5ffb640 10 client.29209 handle_command_reply: tid=0 2024-09-19T06:20:38.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd117150740 1 client.29209 shutdown 2024-09-19T06:20:38.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd117150740 1 -- 172.21.15.5:0/694292886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 msgr2=0x55b8636c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd117150740 1 --2- 172.21.15.5:0/694292886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 0x55b8636c4a00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd10009bf90 tx=0x7fd1000cd6f0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd117150740 1 -- 172.21.15.5:0/694292886 shutdown_connections 2024-09-19T06:20:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd117150740 1 --2- 172.21.15.5:0/694292886 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b8636da1a0 0x55b863747430 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd117150740 1 --2- 172.21.15.5:0/694292886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8636ca680 0x55b8636c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd0f5ffb640 0 client.29209 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd117150740 1 --2- 172.21.15.5:0/694292886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b8636bfdc0 0x55b8636c4a00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd117150740 1 --2- 172.21.15.5:0/694292886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8636bf4a0 0x55b8636c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd117150740 1 -- 172.21.15.5:0/694292886 >> 172.21.15.5:0/694292886 conn(0x55b8636bba30 msgr2=0x55b8636c2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd117150740 1 -- 172.21.15.5:0/694292886 shutdown_connections 2024-09-19T06:20:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd117150740 1 -- 172.21.15.5:0/694292886 wait complete. 2024-09-19T06:20:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.635+0000 7fd117150740 20 client.29209 trim_cache size 0 max 16384 2024-09-19T06:20:38.643 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:20:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:20:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:20:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "227cfcdf-9f28-4609-8f0a-c8227db218bd": { 2024-09-19T06:20:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:20:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "227cfcdf-9f28-4609-8f0a-c8227db218bd", 2024-09-19T06:20:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:20:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:20:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:20:38.643 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:20:38.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.636+0000 7fd114d77640 1 -- 172.21.15.5:0/375520994 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0f80a34b0 msgr2=0x7fd0f80a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:38.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.636+0000 7fd114d77640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0f80a34b0 0x7fd0f80a5970 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55b8636bc890 tx=0x7fd104005f90 comp rx=0 tx=0).stop 2024-09-19T06:20:38.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.636+0000 7fd114d77640 1 -- 172.21.15.5:0/375520994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd110102790 msgr2=0x7fd11010e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:38.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.636+0000 7fd114d77640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd110102790 0x7fd11010e3f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd10000e9f0 tx=0x7fd10000edf0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.636+0000 7fd114d77640 1 -- 172.21.15.5:0/375520994 shutdown_connections 2024-09-19T06:20:38.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.636+0000 7fd114d77640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0f80a34b0 0x7fd0f80a5970 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.636+0000 7fd114d77640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd110106f10 0x7fd1101a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.636+0000 7fd114d77640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd110102790 0x7fd11010e3f0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.636+0000 7fd114d77640 1 --2- 172.21.15.5:0/375520994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd110101de0 0x7fd11010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:38.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.636+0000 7fd114d77640 1 -- 172.21.15.5:0/375520994 >> 172.21.15.5:0/375520994 conn(0x7fd1100fd8e0 msgr2=0x7fd1100fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:38.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.636+0000 7fd114d77640 1 -- 172.21.15.5:0/375520994 shutdown_connections 2024-09-19T06:20:38.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:20:38.636+0000 7fd114d77640 1 -- 172.21.15.5:0/375520994 wait complete. 2024-09-19T06:20:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:38 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:20:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:38 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:20:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:38 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:38 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:20:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:38 smithi157 ceph-mon[35346]: pgmap v544: 97 pgs: 97 active+clean; 741 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 369 op/s 2024-09-19T06:20:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:38 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:38.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:38 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:38.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:38 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:20:38.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:38 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:20:38.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:38 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:38.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:38 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:20:38.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:38 smithi005 ceph-mon[30955]: pgmap v544: 97 pgs: 97 active+clean; 741 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 369 op/s 2024-09-19T06:20:38.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:38 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:38.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:38 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:38 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:20:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:38 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:20:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:38 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:38 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:20:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:38 smithi049 ceph-mon[34640]: pgmap v544: 97 pgs: 97 active+clean; 741 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 369 op/s 2024-09-19T06:20:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:38 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:38 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:20:39.408 INFO:tasks.cephfs.filesystem:scrub status for tag:227cfcdf-9f28-4609-8f0a-c8227db218bd - {'path': '/', 'tag': '227cfcdf-9f28-4609-8f0a-c8227db218bd', 'options': 'recursive,force'} 2024-09-19T06:20:39.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 40457 8.85 MB/sec execute 123 sec latency 60.947 ms 2024-09-19T06:20:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.961+0000 7f836fd11640 1 -- 172.21.15.5:0/2278365326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83680745c0 msgr2=0x7f83680749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.961+0000 7f836fd11640 1 --2- 172.21.15.5:0/2278365326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83680745c0 0x7f83680749a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f8364009ca0 tx=0x7f836402a950 comp rx=0 tx=0).stop 2024-09-19T06:20:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.961+0000 7f836fd11640 1 -- 172.21.15.5:0/2278365326 shutdown_connections 2024-09-19T06:20:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.961+0000 7f836fd11640 1 --2- 172.21.15.5:0/2278365326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836810e360 0x7f8368110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.961+0000 7f836fd11640 1 --2- 172.21.15.5:0/2278365326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8368074ee0 0x7f836810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.961+0000 7f836fd11640 1 --2- 172.21.15.5:0/2278365326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83680745c0 0x7f83680749a0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.961+0000 7f836fd11640 1 -- 172.21.15.5:0/2278365326 >> 172.21.15.5:0/2278365326 conn(0x7f83680fd7a0 msgr2=0x7f83680ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.961+0000 7f836fd11640 1 -- 172.21.15.5:0/2278365326 shutdown_connections 2024-09-19T06:20:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.961+0000 7f836fd11640 1 -- 172.21.15.5:0/2278365326 wait complete. 2024-09-19T06:20:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836fd11640 1 Processor -- start 2024-09-19T06:20:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836fd11640 1 -- start start 2024-09-19T06:20:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836fd11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83680745c0 0x7f836819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836fd11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8368074ee0 0x7f836819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836fd11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836810e360 0x7f836819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836fd11640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8368114000 con 0x7f83680745c0 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836fd11640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8368113e80 con 0x7f836810e360 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836fd11640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8368114180 con 0x7f8368074ee0 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836e2ae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836810e360 0x7f836819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836daad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83680745c0 0x7f836819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836e2ae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836810e360 0x7f836819ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:45872/0 (socket says 172.21.15.5:45872) 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836daad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83680745c0 0x7f836819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35466/0 (socket says 172.21.15.5:35466) 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836e2ae640 1 -- 172.21.15.5:0/1447766244 learned_addr learned my addr 172.21.15.5:0/1447766244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.962+0000 7f836d2ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8368074ee0 0x7f836819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.963+0000 7f836e2ae640 1 -- 172.21.15.5:0/1447766244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8368074ee0 msgr2=0x7f836819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.963+0000 7f836e2ae640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8368074ee0 0x7f836819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.963+0000 7f836e2ae640 1 -- 172.21.15.5:0/1447766244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83680745c0 msgr2=0x7f836819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.963+0000 7f836e2ae640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83680745c0 0x7f836819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.963+0000 7f836e2ae640 1 -- 172.21.15.5:0/1447766244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836819f5b0 con 0x7f836810e360 2024-09-19T06:20:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.963+0000 7f836daad640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83680745c0 0x7f836819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.963+0000 7f836e2ae640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836810e360 0x7f836819ee80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f836000e970 tx=0x7f836000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.964+0000 7f835effd640 1 -- 172.21.15.5:0/1447766244 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836000cd80 con 0x7f836810e360 2024-09-19T06:20:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.964+0000 7f835effd640 1 -- 172.21.15.5:0/1447766244 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83600035f0 con 0x7f836810e360 2024-09-19T06:20:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.964+0000 7f835effd640 1 -- 172.21.15.5:0/1447766244 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360010690 con 0x7f836810e360 2024-09-19T06:20:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.964+0000 7f836fd11640 1 -- 172.21.15.5:0/1447766244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836819f8a0 con 0x7f836810e360 2024-09-19T06:20:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.964+0000 7f836fd11640 1 -- 172.21.15.5:0/1447766244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83681a7130 con 0x7f836810e360 2024-09-19T06:20:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.967+0000 7f835effd640 1 -- 172.21.15.5:0/1447766244 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8360010830 con 0x7f836810e360 2024-09-19T06:20:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.969+0000 7f8372aa6740 1 Processor -- start 2024-09-19T06:20:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.970+0000 7f8372aa6740 1 -- start start 2024-09-19T06:20:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.970+0000 7f8372aa6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd12aaf0 0x55f9fd12aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.970+0000 7f8372aa6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd135770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.971+0000 7f8372aa6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd135cb0 0x55f9fd1380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.971+0000 7f8372aa6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9fd08f2f0 con 0x55f9fd135cb0 2024-09-19T06:20:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.971+0000 7f8372aa6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f9fd037270 con 0x55f9fd12b410 2024-09-19T06:20:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.971+0000 7f8372aa6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f9fd08b2b0 con 0x55f9fd12aaf0 2024-09-19T06:20:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.971+0000 7f835effd640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f83540a34b0 0x7f83540a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.971+0000 7f836d2ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd12aaf0 0x55f9fd12aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.971+0000 7f836daad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd135770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.971+0000 7f836daad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd135770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:45888/0 (socket says 172.21.15.5:45888) 2024-09-19T06:20:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.971+0000 7f836daad640 1 -- 172.21.15.5:0/316088085 learned_addr learned my addr 172.21.15.5:0/316088085 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.971+0000 7f836d2ac640 1 --2- 172.21.15.5:0/316088085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd135cb0 0x55f9fd1380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.971+0000 7f836e2ae640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f83540a34b0 0x7f83540a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.972+0000 7f836daad640 1 -- 172.21.15.5:0/316088085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd12aaf0 msgr2=0x55f9fd12aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.972+0000 7f836daad640 1 --2- 172.21.15.5:0/316088085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd12aaf0 0x55f9fd12aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.972+0000 7f836daad640 1 -- 172.21.15.5:0/316088085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd135cb0 msgr2=0x55f9fd1380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.972+0000 7f836daad640 1 --2- 172.21.15.5:0/316088085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd135cb0 0x55f9fd1380a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.972+0000 7f836daad640 1 -- 172.21.15.5:0/316088085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9fd036340 con 0x55f9fd12b410 2024-09-19T06:20:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.972+0000 7f836d2ac640 1 --2- 172.21.15.5:0/316088085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd135cb0 0x55f9fd1380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.973+0000 7f836daad640 1 --2- 172.21.15.5:0/316088085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd135770 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8364000f80 tx=0x7f83640093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49194cef13afdbaa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.973+0000 7f8353fff640 1 -- 172.21.15.5:0/316088085 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364048070 con 0x55f9fd12b410 2024-09-19T06:20:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.973+0000 7f8353fff640 1 -- 172.21.15.5:0/316088085 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f836402f870 con 0x55f9fd12b410 2024-09-19T06:20:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.973+0000 7f8353fff640 1 -- 172.21.15.5:0/316088085 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364034040 con 0x55f9fd12b410 2024-09-19T06:20:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.973+0000 7f835effd640 1 -- 172.21.15.5:0/1447766244 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f83600cff70 con 0x7f836810e360 2024-09-19T06:20:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.974+0000 7f8372aa6740 1 -- 172.21.15.5:0/316088085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 msgr2=0x55f9fd135770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.974+0000 7f836e2ae640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f83540a34b0 0x7f83540a5970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8360099530 tx=0x7f83600d3690 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.974+0000 7f8372aa6740 1 --2- 172.21.15.5:0/316088085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd135770 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8364000f80 tx=0x7f83640093f0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.974+0000 7f8372aa6740 1 -- 172.21.15.5:0/316088085 shutdown_connections 2024-09-19T06:20:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.974+0000 7f8372aa6740 1 --2- 172.21.15.5:0/316088085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd135cb0 0x55f9fd1380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.974+0000 7f8372aa6740 1 --2- 172.21.15.5:0/316088085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd135770 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.974+0000 7f8372aa6740 1 --2- 172.21.15.5:0/316088085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd12aaf0 0x55f9fd12aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.974+0000 7f8372aa6740 1 -- 172.21.15.5:0/316088085 >> 172.21.15.5:0/316088085 conn(0x55f9fd127060 msgr2=0x55f9fd127440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.975+0000 7f8372aa6740 1 -- 172.21.15.5:0/316088085 shutdown_connections 2024-09-19T06:20:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.975+0000 7f8372aa6740 1 -- 172.21.15.5:0/316088085 wait complete. 2024-09-19T06:20:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.975+0000 7f8372aa6740 1 -- start start 2024-09-19T06:20:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f8372aa6740 1 Processor -- start 2024-09-19T06:20:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f8372aa6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 0x55f9fd132bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f8372aa6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd133130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f8372aa6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd135cb0 0x55f9fd133670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f8372aa6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9fd138880 con 0x55f9fd12aaf0 2024-09-19T06:20:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f8372aa6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f9fd018290 con 0x55f9fd12b410 2024-09-19T06:20:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f8372aa6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f9fd138a00 con 0x55f9fd135cb0 2024-09-19T06:20:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f8372aa6740 10 client.0 _ll_register_callbacks cb 0x55f9fd049140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f836d2ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 0x55f9fd132bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f836daad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd133130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f836d2ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 0x55f9fd132bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35482/0 (socket says 172.21.15.5:35482) 2024-09-19T06:20:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f836daad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd133130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:45898/0 (socket says 172.21.15.5:45898) 2024-09-19T06:20:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.976+0000 7f836d2ac640 1 -- 172.21.15.5:0/4217471727 learned_addr learned my addr 172.21.15.5:0/4217471727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.977+0000 7f836daad640 1 -- 172.21.15.5:0/4217471727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd135cb0 msgr2=0x55f9fd133670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.977+0000 7f836daad640 1 --2- 172.21.15.5:0/4217471727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd135cb0 0x55f9fd133670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.977+0000 7f836daad640 1 -- 172.21.15.5:0/4217471727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 msgr2=0x55f9fd132bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.977+0000 7f836daad640 1 --2- 172.21.15.5:0/4217471727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 0x55f9fd132bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.977+0000 7f836daad640 1 -- 172.21.15.5:0/4217471727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9fcda2210 con 0x55f9fd12b410 2024-09-19T06:20:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.977+0000 7f836d2ac640 1 --2- 172.21.15.5:0/4217471727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 0x55f9fd132bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.977+0000 7f836daad640 1 --2- 172.21.15.5:0/4217471727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd133130 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8364000f80 tx=0x7f836402fc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6039cc3d646127d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.977+0000 7f83517fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:20:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.977+0000 7f83517fa640 1 -- 172.21.15.5:0/4217471727 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364051060 con 0x55f9fd12b410 2024-09-19T06:20:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.977+0000 7f83517fa640 1 -- 172.21.15.5:0/4217471727 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f8364040040 con 0x55f9fd12b410 2024-09-19T06:20:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.977+0000 7f8372aa6740 1 -- 172.21.15.5:0/4217471727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f836402f870 con 0x55f9fd12b410 2024-09-19T06:20:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.978+0000 7f83517fa640 1 -- 172.21.15.5:0/4217471727 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8364048030 con 0x55f9fd12b410 2024-09-19T06:20:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.978+0000 7f8372aa6740 10 client.29285 fetch_fsmap learned FSMap version 58 2024-09-19T06:20:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.978+0000 7f8372aa6740 1 -- 172.21.15.5:0/4217471727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f9fce6e620 con 0x55f9fd12b410 2024-09-19T06:20:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.978+0000 7f83517fa640 1 -- 172.21.15.5:0/4217471727 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f836402f870 con 0x55f9fd12b410 2024-09-19T06:20:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.978+0000 7f83517fa640 1 -- 172.21.15.5:0/4217471727 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f8364030740 con 0x55f9fd12b410 2024-09-19T06:20:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.978+0000 7f8372aa6740 10 client.29285 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:20:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.978+0000 7f8372aa6740 10 client.29285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.978+0000 7f8372aa6740 20 client.29285 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.978+0000 7f8372aa6740 1 --2- 172.21.15.5:0/4217471727 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9fd14e380 0x55f9fd150770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.978+0000 7f8372aa6740 4 client.29285 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.978+0000 7f8372aa6740 1 -- 172.21.15.5:0/4217471727 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9fd09ca20 con 0x55f9fd14e380 2024-09-19T06:20:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.978+0000 7f836d2ac640 1 --2- 172.21.15.5:0/4217471727 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9fd14e380 0x55f9fd150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.979+0000 7f83517fa640 10 client.29285 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.979+0000 7f836d2ac640 1 --2- 172.21.15.5:0/4217471727 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9fd14e380 0x55f9fd150770 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1157e046ff3f0 server_cookie=1e8b0e19fc6c94b in_seq=0 out_seq=0 2024-09-19T06:20:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f83517fa640 1 -- 172.21.15.5:0/4217471727 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9fd09ca20 con 0x55f9fd14e380 2024-09-19T06:20:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f83517fa640 10 client.29285 handle_command_reply: tid=0 2024-09-19T06:20:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f8372aa6740 1 client.29285 shutdown 2024-09-19T06:20:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f8372aa6740 1 -- 172.21.15.5:0/4217471727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 msgr2=0x55f9fd133130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f8372aa6740 1 --2- 172.21.15.5:0/4217471727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd133130 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8364000f80 tx=0x7f836402fc50 comp rx=0 tx=0).stop 2024-09-19T06:20:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f8372aa6740 1 -- 172.21.15.5:0/4217471727 shutdown_connections 2024-09-19T06:20:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f8372aa6740 1 --2- 172.21.15.5:0/4217471727 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9fd14e380 0x55f9fd150770 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f8372aa6740 1 --2- 172.21.15.5:0/4217471727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd135cb0 0x55f9fd133670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f8372aa6740 1 --2- 172.21.15.5:0/4217471727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd133130 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f8372aa6740 1 --2- 172.21.15.5:0/4217471727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 0x55f9fd132bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f8372aa6740 1 -- 172.21.15.5:0/4217471727 >> 172.21.15.5:0/4217471727 conn(0x55f9fd127060 msgr2=0x55f9fd12cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f83517fa640 0 client.29285 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f8372aa6740 1 -- 172.21.15.5:0/4217471727 shutdown_connections 2024-09-19T06:20:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f8372aa6740 1 -- 172.21.15.5:0/4217471727 wait complete. 2024-09-19T06:20:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:39.980+0000 7f8372aa6740 20 client.29285 trim_cache size 0 max 16384 2024-09-19T06:20:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.006+0000 7f8372aa6740 1 Processor -- start 2024-09-19T06:20:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.006+0000 7f8372aa6740 1 -- start start 2024-09-19T06:20:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.006+0000 7f8372aa6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd12aaf0 0x55f9fd12d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.006+0000 7f8372aa6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd1334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.006+0000 7f8372aa6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd135cb0 0x55f9fd134b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.006+0000 7f8372aa6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83681ab680 con 0x55f9fd135cb0 2024-09-19T06:20:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.006+0000 7f8372aa6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f83681ad1c0 con 0x55f9fd12b410 2024-09-19T06:20:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.006+0000 7f8372aa6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83681abd50 con 0x55f9fd12aaf0 2024-09-19T06:20:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f836d2ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd12aaf0 0x55f9fd12d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f836d2ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd12aaf0 0x55f9fd12d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38592/0 (socket says 172.21.15.5:38592) 2024-09-19T06:20:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f836d2ac640 1 -- 172.21.15.5:0/1849976094 learned_addr learned my addr 172.21.15.5:0/1849976094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f836d2ac640 1 --2- 172.21.15.5:0/1849976094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd135cb0 0x55f9fd134b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f836daad640 1 --2- 172.21.15.5:0/1849976094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd1334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f836d2ac640 1 -- 172.21.15.5:0/1849976094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 msgr2=0x55f9fd1334c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f836d2ac640 1 --2- 172.21.15.5:0/1849976094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd1334c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f836d2ac640 1 -- 172.21.15.5:0/1849976094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd135cb0 msgr2=0x55f9fd134b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f836d2ac640 1 --2- 172.21.15.5:0/1849976094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd135cb0 0x55f9fd134b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f836d2ac640 1 -- 172.21.15.5:0/1849976094 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9fd131180 con 0x55f9fd12aaf0 2024-09-19T06:20:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f836d2ac640 1 --2- 172.21.15.5:0/1849976094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd12aaf0 0x55f9fd12d560 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8358004150 tx=0x7f83580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3519e2279861230 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f83537fe640 1 -- 172.21.15.5:0/1849976094 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835801b040 con 0x55f9fd12aaf0 2024-09-19T06:20:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f83537fe640 1 -- 172.21.15.5:0/1849976094 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8358003e10 con 0x55f9fd12aaf0 2024-09-19T06:20:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.007+0000 7f83537fe640 1 -- 172.21.15.5:0/1849976094 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83580047a0 con 0x55f9fd12aaf0 2024-09-19T06:20:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.008+0000 7f8372aa6740 1 -- 172.21.15.5:0/1849976094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd12aaf0 msgr2=0x55f9fd12d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.008+0000 7f8372aa6740 1 --2- 172.21.15.5:0/1849976094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd12aaf0 0x55f9fd12d560 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8358004150 tx=0x7f83580046c0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.008+0000 7f8372aa6740 1 -- 172.21.15.5:0/1849976094 shutdown_connections 2024-09-19T06:20:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.008+0000 7f8372aa6740 1 --2- 172.21.15.5:0/1849976094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd135cb0 0x55f9fd134b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.008+0000 7f8372aa6740 1 --2- 172.21.15.5:0/1849976094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd1334c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.008+0000 7f8372aa6740 1 --2- 172.21.15.5:0/1849976094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd12aaf0 0x55f9fd12d560 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.008+0000 7f8372aa6740 1 -- 172.21.15.5:0/1849976094 >> 172.21.15.5:0/1849976094 conn(0x55f9fd127060 msgr2=0x55f9fd134450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.008+0000 7f8372aa6740 1 -- 172.21.15.5:0/1849976094 shutdown_connections 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.008+0000 7f8372aa6740 1 -- 172.21.15.5:0/1849976094 wait complete. 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.009+0000 7f8372aa6740 1 -- start start 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.009+0000 7f8372aa6740 1 Processor -- start 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.009+0000 7f8372aa6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 0x55f9fd150950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.009+0000 7f8372aa6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd150e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.009+0000 7f8372aa6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd135cb0 0x55f9fd1b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.009+0000 7f8372aa6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83681acaf0 con 0x55f9fd12aaf0 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.009+0000 7f8372aa6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f83681ac420 con 0x55f9fd12b410 2024-09-19T06:20:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.009+0000 7f8372aa6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83681a9680 con 0x55f9fd135cb0 2024-09-19T06:20:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f836d2ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 0x55f9fd150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f8372aa6740 10 client.0 _ll_register_callbacks cb 0x7f8364046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f836d2ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 0x55f9fd150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35502/0 (socket says 172.21.15.5:35502) 2024-09-19T06:20:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f836d2ac640 1 -- 172.21.15.5:0/2252213066 learned_addr learned my addr 172.21.15.5:0/2252213066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f836daad640 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f836d2ac640 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd135cb0 0x55f9fd1b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f836d2ac640 1 -- 172.21.15.5:0/2252213066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd135cb0 msgr2=0x55f9fd1b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f836d2ac640 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd135cb0 0x55f9fd1b6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f836d2ac640 1 -- 172.21.15.5:0/2252213066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 msgr2=0x55f9fd150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f836d2ac640 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f836d2ac640 1 -- 172.21.15.5:0/2252213066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9fd1b71f0 con 0x55f9fd12aaf0 2024-09-19T06:20:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f836d2ac640 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 0x55f9fd150950 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8358006010 tx=0x7f8358013be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1a2cce1a7fb12e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.010+0000 7f83517fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.011+0000 7f83517fa640 1 -- 172.21.15.5:0/2252213066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358005230 con 0x55f9fd12aaf0 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.011+0000 7f83517fa640 1 -- 172.21.15.5:0/2252213066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f83580053d0 con 0x55f9fd12aaf0 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.011+0000 7f8372aa6740 1 -- 172.21.15.5:0/2252213066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8358003e10 con 0x55f9fd12aaf0 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.011+0000 7f83517fa640 1 -- 172.21.15.5:0/2252213066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f835803f050 con 0x55f9fd12aaf0 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.011+0000 7f8372aa6740 10 client.19908 fetch_fsmap learned FSMap version 58 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.011+0000 7f8372aa6740 1 -- 172.21.15.5:0/2252213066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f9fd1b8470 con 0x55f9fd12aaf0 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.011+0000 7f83517fa640 1 -- 172.21.15.5:0/2252213066 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8358003e10 con 0x55f9fd12aaf0 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.011+0000 7f83517fa640 1 -- 172.21.15.5:0/2252213066 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f835802c450 con 0x55f9fd12aaf0 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.011+0000 7f8372aa6740 10 client.19908 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.012+0000 7f8372aa6740 10 client.19908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.012+0000 7f8372aa6740 20 client.19908 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.012+0000 7f8372aa6740 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9fd1b8a80 0x55f9fd1baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.012+0000 7f8372aa6740 4 client.19908 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:20:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.012+0000 7f8372aa6740 1 -- 172.21.15.5:0/2252213066 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9fd096d70 con 0x55f9fd1b8a80 2024-09-19T06:20:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.012+0000 7f836daad640 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9fd1b8a80 0x55f9fd1baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.012+0000 7f83517fa640 10 client.19908 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.012+0000 7f836daad640 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9fd1b8a80 0x55f9fd1baea0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c418e3196b7c3a0 server_cookie=2815097b59ef5af7 in_seq=0 out_seq=0 2024-09-19T06:20:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.082+0000 7f83517fa640 1 -- 172.21.15.5:0/2252213066 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9fd096d70 con 0x55f9fd1b8a80 2024-09-19T06:20:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.082+0000 7f83517fa640 10 client.19908 handle_command_reply: tid=0 2024-09-19T06:20:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.082+0000 7f8372aa6740 1 client.19908 shutdown 2024-09-19T06:20:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.082+0000 7f8372aa6740 1 -- 172.21.15.5:0/2252213066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 msgr2=0x55f9fd150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.082+0000 7f8372aa6740 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 0x55f9fd150950 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8358006010 tx=0x7f8358013be0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.083+0000 7f8372aa6740 1 -- 172.21.15.5:0/2252213066 shutdown_connections 2024-09-19T06:20:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.083+0000 7f8372aa6740 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f9fd1b8a80 0x55f9fd1baea0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.083+0000 7f8372aa6740 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9fd135cb0 0x55f9fd1b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.083+0000 7f8372aa6740 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9fd12b410 0x55f9fd150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.083+0000 7f8372aa6740 1 --2- 172.21.15.5:0/2252213066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fd12aaf0 0x55f9fd150950 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.083+0000 7f8372aa6740 1 -- 172.21.15.5:0/2252213066 >> 172.21.15.5:0/2252213066 conn(0x55f9fd127060 msgr2=0x55f9fd134c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.083+0000 7f83517fa640 0 client.19908 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.083+0000 7f8372aa6740 1 -- 172.21.15.5:0/2252213066 shutdown_connections 2024-09-19T06:20:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.083+0000 7f8372aa6740 1 -- 172.21.15.5:0/2252213066 wait complete. 2024-09-19T06:20:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.083+0000 7f8372aa6740 20 client.19908 trim_cache size 0 max 16384 2024-09-19T06:20:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:20:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:20:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:20:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:20:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.084+0000 7f836fd11640 1 -- 172.21.15.5:0/1447766244 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f83540a34b0 msgr2=0x7f83540a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.084+0000 7f836fd11640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f83540a34b0 0x7f83540a5970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8360099530 tx=0x7f83600d3690 comp rx=0 tx=0).stop 2024-09-19T06:20:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.084+0000 7f836fd11640 1 -- 172.21.15.5:0/1447766244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836810e360 msgr2=0x7f836819ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.084+0000 7f836fd11640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836810e360 0x7f836819ee80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f836000e970 tx=0x7f836000ee30 comp rx=0 tx=0).stop 2024-09-19T06:20:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.084+0000 7f836fd11640 1 -- 172.21.15.5:0/1447766244 shutdown_connections 2024-09-19T06:20:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.084+0000 7f836fd11640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836810e360 0x7f836819ee80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.084+0000 7f836fd11640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f83540a34b0 0x7f83540a5970 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.084+0000 7f836fd11640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8368074ee0 0x7f836819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.084+0000 7f836fd11640 1 --2- 172.21.15.5:0/1447766244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83680745c0 0x7f836819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.084+0000 7f836fd11640 1 -- 172.21.15.5:0/1447766244 >> 172.21.15.5:0/1447766244 conn(0x7f83680fd7a0 msgr2=0x7f83680ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.084+0000 7f836fd11640 1 -- 172.21.15.5:0/1447766244 shutdown_connections 2024-09-19T06:20:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:40.084+0000 7f836fd11640 1 -- 172.21.15.5:0/1447766244 wait complete. 2024-09-19T06:20:40.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 40567 8.82 MB/sec execute 124 sec latency 76.494 ms 2024-09-19T06:20:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:40 smithi157 ceph-mon[35346]: pgmap v545: 97 pgs: 97 active+clean; 741 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 247 op/s 2024-09-19T06:20:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:40 smithi005 ceph-mon[30955]: pgmap v545: 97 pgs: 97 active+clean; 741 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 247 op/s 2024-09-19T06:20:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:40 smithi049 ceph-mon[34640]: pgmap v545: 97 pgs: 97 active+clean; 741 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 247 op/s 2024-09-19T06:20:41.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 40719 8.81 MB/sec execute 125 sec latency 63.120 ms 2024-09-19T06:20:42.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 40884 8.79 MB/sec execute 126 sec latency 59.788 ms 2024-09-19T06:20:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:42 smithi157 ceph-mon[35346]: pgmap v546: 97 pgs: 97 active+clean; 741 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 247 op/s 2024-09-19T06:20:42.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:42 smithi005 ceph-mon[30955]: pgmap v546: 97 pgs: 97 active+clean; 741 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 247 op/s 2024-09-19T06:20:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:42 smithi049 ceph-mon[34640]: pgmap v546: 97 pgs: 97 active+clean; 741 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 247 op/s 2024-09-19T06:20:43.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41027 8.76 MB/sec execute 127 sec latency 64.149 ms 2024-09-19T06:20:44.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41168 8.73 MB/sec execute 128 sec latency 60.831 ms 2024-09-19T06:20:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:44 smithi157 ceph-mon[35346]: pgmap v547: 97 pgs: 97 active+clean; 749 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 439 op/s 2024-09-19T06:20:44.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:44 smithi005 ceph-mon[30955]: pgmap v547: 97 pgs: 97 active+clean; 749 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 439 op/s 2024-09-19T06:20:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:44 smithi049 ceph-mon[34640]: pgmap v547: 97 pgs: 97 active+clean; 749 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 439 op/s 2024-09-19T06:20:45.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41312 8.73 MB/sec execute 129 sec latency 88.016 ms 2024-09-19T06:20:46.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41467 8.72 MB/sec execute 130 sec latency 81.725 ms 2024-09-19T06:20:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:46 smithi157 ceph-mon[35346]: pgmap v548: 97 pgs: 97 active+clean; 749 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 306 op/s 2024-09-19T06:20:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:46 smithi005 ceph-mon[30955]: pgmap v548: 97 pgs: 97 active+clean; 749 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 306 op/s 2024-09-19T06:20:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:46 smithi049 ceph-mon[34640]: pgmap v548: 97 pgs: 97 active+clean; 749 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 306 op/s 2024-09-19T06:20:47.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41639 8.70 MB/sec execute 131 sec latency 64.737 ms 2024-09-19T06:20:48.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41823 8.70 MB/sec execute 132 sec latency 69.672 ms 2024-09-19T06:20:48.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:48 smithi157 ceph-mon[35346]: pgmap v549: 97 pgs: 97 active+clean; 750 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.9 MiB/s wr, 436 op/s 2024-09-19T06:20:48.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:48 smithi005 ceph-mon[30955]: pgmap v549: 97 pgs: 97 active+clean; 750 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.9 MiB/s wr, 436 op/s 2024-09-19T06:20:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:48 smithi049 ceph-mon[34640]: pgmap v549: 97 pgs: 97 active+clean; 750 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.9 MiB/s wr, 436 op/s 2024-09-19T06:20:49.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41973 8.66 MB/sec execute 133 sec latency 59.161 ms 2024-09-19T06:20:49.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:49 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:20:49.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:49 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:20:49.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:49 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:20:49.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:49 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:20:49.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:49 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:20:49.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:49 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:20:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:49 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:20:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:49 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:20:49.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:49 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:20:50.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 42142 8.68 MB/sec execute 134 sec latency 84.347 ms 2024-09-19T06:20:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:50 smithi157 ceph-mon[35346]: pgmap v550: 97 pgs: 97 active+clean; 750 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 321 op/s 2024-09-19T06:20:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:50 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:20:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:50 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:20:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:50 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:20:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:50 smithi005 ceph-mon[30955]: pgmap v550: 97 pgs: 97 active+clean; 750 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 321 op/s 2024-09-19T06:20:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:50 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:20:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:50 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:20:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:50 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:20:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:50 smithi049 ceph-mon[34640]: pgmap v550: 97 pgs: 97 active+clean; 750 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 321 op/s 2024-09-19T06:20:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:50 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:20:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:50 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:20:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:50 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:20:51.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 42325 8.69 MB/sec execute 135 sec latency 68.277 ms 2024-09-19T06:20:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:51 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:20:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:51 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:20:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:51 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:20:51.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:51 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:20:51.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:51 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:20:51.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:51 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:20:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:51 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:20:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:51 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:20:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:51 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:20:52.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 42483 8.68 MB/sec execute 136 sec latency 63.447 ms 2024-09-19T06:20:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:52 smithi005 ceph-mon[30955]: pgmap v551: 97 pgs: 97 active+clean; 750 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 321 op/s 2024-09-19T06:20:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:52 smithi049 ceph-mon[34640]: pgmap v551: 97 pgs: 97 active+clean; 750 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 321 op/s 2024-09-19T06:20:53.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:52 smithi157 ceph-mon[35346]: pgmap v551: 97 pgs: 97 active+clean; 750 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 321 op/s 2024-09-19T06:20:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.272+0000 7f01f8647640 1 -- 172.21.15.5:0/464253743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f01069b0 msgr2=0x7f01f0106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.272+0000 7f01f8647640 1 --2- 172.21.15.5:0/464253743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f01069b0 0x7f01f0106d90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009930 tx=0x7f01ec02a420 comp rx=0 tx=0).stop 2024-09-19T06:20:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.272+0000 7f01f8647640 1 -- 172.21.15.5:0/464253743 shutdown_connections 2024-09-19T06:20:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.272+0000 7f01f8647640 1 --2- 172.21.15.5:0/464253743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f0101340 0x7f01f010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.272+0000 7f01f8647640 1 --2- 172.21.15.5:0/464253743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f01009a0 0x7f01f0100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.272+0000 7f01f8647640 1 --2- 172.21.15.5:0/464253743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f01069b0 0x7f01f0106d90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.272+0000 7f01f8647640 1 -- 172.21.15.5:0/464253743 >> 172.21.15.5:0/464253743 conn(0x7f01f00779a0 msgr2=0x7f01f00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.272+0000 7f01f8647640 1 -- 172.21.15.5:0/464253743 shutdown_connections 2024-09-19T06:20:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.272+0000 7f01f8647640 1 -- 172.21.15.5:0/464253743 wait complete. 2024-09-19T06:20:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f8647640 1 Processor -- start 2024-09-19T06:20:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f8647640 1 -- start start 2024-09-19T06:20:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f8647640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f01009a0 0x7f01f0196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f8647640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f0101340 0x7f01f0196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f8647640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f01069b0 0x7f01f019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f8647640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01f010fbe0 con 0x7f01f0101340 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f8647640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f01f010fa60 con 0x7f01f01009a0 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f8647640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01f010fd60 con 0x7f01f01069b0 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f5be2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f0101340 0x7f01f0196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f63e3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f01009a0 0x7f01f0196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f5be2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f0101340 0x7f01f0196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47852/0 (socket says 172.21.15.5:47852) 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f5be2640 1 -- 172.21.15.5:0/2752512558 learned_addr learned my addr 172.21.15.5:0/2752512558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f63e3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f01009a0 0x7f01f0196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:51060/0 (socket says 172.21.15.5:51060) 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f6be4640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f01069b0 0x7f01f019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f5be2640 1 -- 172.21.15.5:0/2752512558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f01069b0 msgr2=0x7f01f019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f5be2640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f01069b0 0x7f01f019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f5be2640 1 -- 172.21.15.5:0/2752512558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f01009a0 msgr2=0x7f01f0196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.273+0000 7f01f5be2640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f01009a0 0x7f01f0196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.274+0000 7f01f5be2640 1 -- 172.21.15.5:0/2752512558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f019b120 con 0x7f01f0101340 2024-09-19T06:20:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.274+0000 7f01f63e3640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f01009a0 0x7f01f0196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.274+0000 7f01f5be2640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f0101340 0x7f01f0196670 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f01e0007860 tx=0x7f01e0007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.274+0000 7f01e77fe640 1 -- 172.21.15.5:0/2752512558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0019020 con 0x7f01f0101340 2024-09-19T06:20:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.274+0000 7f01f8647640 1 -- 172.21.15.5:0/2752512558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f019b410 con 0x7f01f0101340 2024-09-19T06:20:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.274+0000 7f01f8647640 1 -- 172.21.15.5:0/2752512558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f01a2ca0 con 0x7f01f0101340 2024-09-19T06:20:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.274+0000 7f01e77fe640 1 -- 172.21.15.5:0/2752512558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01e0015470 con 0x7f01f0101340 2024-09-19T06:20:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.274+0000 7f01e77fe640 1 -- 172.21.15.5:0/2752512558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e001e660 con 0x7f01f0101340 2024-09-19T06:20:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.277+0000 7f01e77fe640 1 -- 172.21.15.5:0/2752512558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f01e001c020 con 0x7f01f0101340 2024-09-19T06:20:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.278+0000 7f01e77fe640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f01dc0a3430 0x7f01dc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.278+0000 7f01f63e3640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f01dc0a3430 0x7f01dc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.278+0000 7f01e77fe640 1 -- 172.21.15.5:0/2752512558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f01e00cfec0 con 0x7f01f0101340 2024-09-19T06:20:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.279+0000 7f01f63e3640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f01dc0a3430 0x7f01dc0a58f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f01ec031440 tx=0x7f01ec005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.303+0000 7f01fb3dd740 1 Processor -- start 2024-09-19T06:20:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.303+0000 7f01fb3dd740 1 -- start start 2024-09-19T06:20:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.303+0000 7f01fb3dd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d1af0 0x55aebb3d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.303+0000 7f01fb3dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3d2410 0x55aebb3dc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.303+0000 7f01fb3dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3dccb0 0x55aebb3df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.303+0000 7f01fb3dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aebb3362f0 con 0x55aebb3dccb0 2024-09-19T06:20:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.303+0000 7f01fb3dd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aebb2de270 con 0x55aebb3d1af0 2024-09-19T06:20:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.303+0000 7f01fb3dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55aebb3322b0 con 0x55aebb3d2410 2024-09-19T06:20:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01f5be2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3d2410 0x55aebb3dc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01f6be4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3dccb0 0x55aebb3df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01f5be2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3d2410 0x55aebb3dc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57104/0 (socket says 172.21.15.5:57104) 2024-09-19T06:20:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01f63e3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d1af0 0x55aebb3d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01f5be2640 1 -- 172.21.15.5:0/967481305 learned_addr learned my addr 172.21.15.5:0/967481305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01f63e3640 1 -- 172.21.15.5:0/967481305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3d2410 msgr2=0x55aebb3dc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01f63e3640 1 --2- 172.21.15.5:0/967481305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3d2410 0x55aebb3dc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01f63e3640 1 -- 172.21.15.5:0/967481305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3dccb0 msgr2=0x55aebb3df0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01f63e3640 1 --2- 172.21.15.5:0/967481305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3dccb0 0x55aebb3df0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01f63e3640 1 -- 172.21.15.5:0/967481305 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aebb2dd340 con 0x55aebb3d1af0 2024-09-19T06:20:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01f6be4640 1 --2- 172.21.15.5:0/967481305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3dccb0 0x55aebb3df0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01f63e3640 1 --2- 172.21.15.5:0/967481305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d1af0 0x55aebb3d1ed0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f01ec02a940 tx=0x7f01ec03d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbeeb84bfb52f7ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.304+0000 7f01e4bfe640 1 -- 172.21.15.5:0/967481305 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec035040 con 0x55aebb3d1af0 2024-09-19T06:20:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.305+0000 7f01e4bfe640 1 -- 172.21.15.5:0/967481305 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01ec004de0 con 0x55aebb3d1af0 2024-09-19T06:20:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.305+0000 7f01e4bfe640 1 -- 172.21.15.5:0/967481305 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0324b0 con 0x55aebb3d1af0 2024-09-19T06:20:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.305+0000 7f01fb3dd740 1 -- 172.21.15.5:0/967481305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d1af0 msgr2=0x55aebb3d1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.305+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/967481305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d1af0 0x55aebb3d1ed0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f01ec02a940 tx=0x7f01ec03d000 comp rx=0 tx=0).stop 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.305+0000 7f01fb3dd740 1 -- 172.21.15.5:0/967481305 shutdown_connections 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.305+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/967481305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3dccb0 0x55aebb3df0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.305+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/967481305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3d2410 0x55aebb3dc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.305+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/967481305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d1af0 0x55aebb3d1ed0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.305+0000 7f01fb3dd740 1 -- 172.21.15.5:0/967481305 >> 172.21.15.5:0/967481305 conn(0x55aebb3ce060 msgr2=0x55aebb3ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.306+0000 7f01fb3dd740 1 -- 172.21.15.5:0/967481305 shutdown_connections 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.306+0000 7f01fb3dd740 1 -- 172.21.15.5:0/967481305 wait complete. 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.306+0000 7f01fb3dd740 1 -- start start 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.306+0000 7f01fb3dd740 1 Processor -- start 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01fb3dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3d1af0 0x55aebb3d9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01fb3dd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3da140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01fb3dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3dccb0 0x55aebb3da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01fb3dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aebb3df880 con 0x55aebb3dccb0 2024-09-19T06:20:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01fb3dd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aebb2bf290 con 0x55aebb3d2410 2024-09-19T06:20:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01fb3dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55aebb3dfa00 con 0x55aebb3d1af0 2024-09-19T06:20:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01fb3dd740 10 client.0 _ll_register_callbacks cb 0x55aebb2f0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01f63e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3d1af0 0x55aebb3d9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01f5be2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3da140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01f63e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3d1af0 0x55aebb3d9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57108/0 (socket says 172.21.15.5:57108) 2024-09-19T06:20:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01f63e3640 1 -- 172.21.15.5:0/3063580609 learned_addr learned my addr 172.21.15.5:0/3063580609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01f6be4640 1 --2- 172.21.15.5:0/3063580609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3dccb0 0x55aebb3da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01f5be2640 1 -- 172.21.15.5:0/3063580609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3d1af0 msgr2=0x55aebb3d9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01f5be2640 1 --2- 172.21.15.5:0/3063580609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3d1af0 0x55aebb3d9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01f5be2640 1 -- 172.21.15.5:0/3063580609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3dccb0 msgr2=0x55aebb3da680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01f5be2640 1 --2- 172.21.15.5:0/3063580609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3dccb0 0x55aebb3da680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01f5be2640 1 -- 172.21.15.5:0/3063580609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aebb049210 con 0x55aebb3d2410 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01f5be2640 1 --2- 172.21.15.5:0/3063580609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3da140 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f01e000f500 tx=0x7f01e009c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=341098280aa92c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.307+0000 7f01d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01d9ffb640 1 -- 172.21.15.5:0/3063580609 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e00b5020 con 0x55aebb3d2410 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01d9ffb640 1 -- 172.21.15.5:0/3063580609 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f01e00a0170 con 0x55aebb3d2410 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01fb3dd740 1 -- 172.21.15.5:0/3063580609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ec004de0 con 0x55aebb3d2410 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01d9ffb640 1 -- 172.21.15.5:0/3063580609 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f01e00bc0b0 con 0x55aebb3d2410 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01fb3dd740 10 client.29303 fetch_fsmap learned FSMap version 58 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01fb3dd740 1 -- 172.21.15.5:0/3063580609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55aebb115620 con 0x55aebb3d2410 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01d9ffb640 1 -- 172.21.15.5:0/3063580609 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01e00bc250 con 0x55aebb3d2410 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01d9ffb640 1 -- 172.21.15.5:0/3063580609 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f01e000fb20 con 0x55aebb3d2410 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01fb3dd740 10 client.29303 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:20:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01fb3dd740 10 client.29303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01fb3dd740 20 client.29303 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/3063580609 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aebb3ecca0 0x55aebb3ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01fb3dd740 4 client.29303 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:20:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.308+0000 7f01fb3dd740 1 -- 172.21.15.5:0/3063580609 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f01ec004de0 con 0x55aebb3ecca0 2024-09-19T06:20:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.309+0000 7f01f63e3640 1 --2- 172.21.15.5:0/3063580609 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aebb3ecca0 0x55aebb3ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.309+0000 7f01f63e3640 1 --2- 172.21.15.5:0/3063580609 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aebb3ecca0 0x55aebb3ef0c0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167a673a2234dd79 server_cookie=c221a566d5609abf in_seq=0 out_seq=0 2024-09-19T06:20:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.309+0000 7f01d9ffb640 10 client.29303 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01d9ffb640 1 -- 172.21.15.5:0/3063580609 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f01ec004de0 con 0x55aebb3ecca0 2024-09-19T06:20:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01d9ffb640 10 client.29303 handle_command_reply: tid=0 2024-09-19T06:20:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01fb3dd740 1 client.29303 shutdown 2024-09-19T06:20:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01fb3dd740 1 -- 172.21.15.5:0/3063580609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 msgr2=0x55aebb3da140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/3063580609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3da140 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f01e000f500 tx=0x7f01e009c9c0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01fb3dd740 1 -- 172.21.15.5:0/3063580609 shutdown_connections 2024-09-19T06:20:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/3063580609 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aebb3ecca0 0x55aebb3ef0c0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/3063580609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3dccb0 0x55aebb3da680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/3063580609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3da140 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01d9ffb640 0 client.29303 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/3063580609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3d1af0 0x55aebb3d9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01fb3dd740 1 -- 172.21.15.5:0/3063580609 >> 172.21.15.5:0/3063580609 conn(0x55aebb3ce060 msgr2=0x55aebb3d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01fb3dd740 1 -- 172.21.15.5:0/3063580609 shutdown_connections 2024-09-19T06:20:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01fb3dd740 1 -- 172.21.15.5:0/3063580609 wait complete. 2024-09-19T06:20:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.310+0000 7f01fb3dd740 20 client.29303 trim_cache size 0 max 16384 2024-09-19T06:20:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.333+0000 7f01fb3dd740 1 Processor -- start 2024-09-19T06:20:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.333+0000 7f01fb3dd740 1 -- start start 2024-09-19T06:20:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.333+0000 7f01fb3dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 0x55aebb3da4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01fb3dd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01fb3dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3dccb0 0x55aebb3f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01fb3dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01d400e7d0 con 0x55aebb3d1af0 2024-09-19T06:20:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01fb3dd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f01d4010310 con 0x55aebb3d2410 2024-09-19T06:20:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01fb3dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01d400eea0 con 0x55aebb3dccb0 2024-09-19T06:20:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01f63e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 0x55aebb3da4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01f5be2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01f63e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 0x55aebb3da4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47878/0 (socket says 172.21.15.5:47878) 2024-09-19T06:20:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01f63e3640 1 -- 172.21.15.5:0/1106710030 learned_addr learned my addr 172.21.15.5:0/1106710030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01f6be4640 1 --2- 172.21.15.5:0/1106710030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3dccb0 0x55aebb3f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01f63e3640 1 -- 172.21.15.5:0/1106710030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3dccb0 msgr2=0x55aebb3f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01f63e3640 1 --2- 172.21.15.5:0/1106710030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3dccb0 0x55aebb3f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01f63e3640 1 -- 172.21.15.5:0/1106710030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 msgr2=0x55aebb3cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01f63e3640 1 --2- 172.21.15.5:0/1106710030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01f63e3640 1 -- 172.21.15.5:0/1106710030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aebb3d2240 con 0x55aebb3d1af0 2024-09-19T06:20:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01f5be2640 1 --2- 172.21.15.5:0/1106710030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.334+0000 7f01f63e3640 1 --2- 172.21.15.5:0/1106710030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 0x55aebb3da4b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f01ec02aa10 tx=0x7f01ec0043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d3716ffa2bc05f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.335+0000 7f01dbfff640 1 -- 172.21.15.5:0/1106710030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec061070 con 0x55aebb3d1af0 2024-09-19T06:20:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.335+0000 7f01dbfff640 1 -- 172.21.15.5:0/1106710030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01ec004930 con 0x55aebb3d1af0 2024-09-19T06:20:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.335+0000 7f01dbfff640 1 -- 172.21.15.5:0/1106710030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec040070 con 0x55aebb3d1af0 2024-09-19T06:20:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.336+0000 7f01fb3dd740 1 -- 172.21.15.5:0/1106710030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 msgr2=0x55aebb3da4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.336+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/1106710030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 0x55aebb3da4b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f01ec02aa10 tx=0x7f01ec0043f0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.336+0000 7f01fb3dd740 1 -- 172.21.15.5:0/1106710030 shutdown_connections 2024-09-19T06:20:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.336+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/1106710030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3dccb0 0x55aebb3f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.336+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/1106710030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.336+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/1106710030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 0x55aebb3da4b0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.336+0000 7f01fb3dd740 1 -- 172.21.15.5:0/1106710030 >> 172.21.15.5:0/1106710030 conn(0x55aebb3ce060 msgr2=0x55aebb3cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.336+0000 7f01fb3dd740 1 -- 172.21.15.5:0/1106710030 shutdown_connections 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.336+0000 7f01fb3dd740 1 -- 172.21.15.5:0/1106710030 wait complete. 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.336+0000 7f01fb3dd740 1 -- start start 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.336+0000 7f01fb3dd740 1 Processor -- start 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.336+0000 7f01fb3dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 0x55aebb3e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01fb3dd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01fb3dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3dccb0 0x55aebb3e14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01fb3dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01d400fc40 con 0x55aebb3d1af0 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01fb3dd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f01d400f570 con 0x55aebb3d2410 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01fb3dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01d400c7d0 con 0x55aebb3dccb0 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01f63e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 0x55aebb3e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01fb3dd740 10 client.0 _ll_register_callbacks cb 0x7f01e009bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:20:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01f63e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 0x55aebb3e0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47894/0 (socket says 172.21.15.5:47894) 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01f63e3640 1 -- 172.21.15.5:0/2183128179 learned_addr learned my addr 172.21.15.5:0/2183128179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01f5be2640 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01f6be4640 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3dccb0 0x55aebb3e14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01f63e3640 1 -- 172.21.15.5:0/2183128179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3dccb0 msgr2=0x55aebb3e14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01f63e3640 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3dccb0 0x55aebb3e14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01f63e3640 1 -- 172.21.15.5:0/2183128179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 msgr2=0x55aebb3e0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01f63e3640 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3e0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01f63e3640 1 -- 172.21.15.5:0/2183128179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aebb45f1b0 con 0x55aebb3d1af0 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01f5be2640 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3e0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01f63e3640 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 0x55aebb3e0a60 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0035f0 tx=0x7f01ec003620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d8eacec4dde214 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.337+0000 7f01d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.338+0000 7f01d9ffb640 1 -- 172.21.15.5:0/2183128179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec061070 con 0x55aebb3d1af0 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.338+0000 7f01d9ffb640 1 -- 172.21.15.5:0/2183128179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8550+0+0 (secure 0 0 0) 0x7f01ec0039e0 con 0x55aebb3d1af0 2024-09-19T06:20:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.338+0000 7f01fb3dd740 1 -- 172.21.15.5:0/2183128179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ec004930 con 0x55aebb3d1af0 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.338+0000 7f01d9ffb640 1 -- 172.21.15.5:0/2183128179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f01ec040030 con 0x55aebb3d1af0 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.338+0000 7f01fb3dd740 10 client.19926 fetch_fsmap learned FSMap version 58 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.338+0000 7f01fb3dd740 1 -- 172.21.15.5:0/2183128179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55aebb4604a0 con 0x55aebb3d1af0 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.338+0000 7f01d9ffb640 1 -- 172.21.15.5:0/2183128179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01ec004930 con 0x55aebb3d1af0 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.338+0000 7f01d9ffb640 1 -- 172.21.15.5:0/2183128179 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f01ec049a20 con 0x55aebb3d1af0 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.339+0000 7f01fb3dd740 10 client.19926 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.339+0000 7f01fb3dd740 10 client.19926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.339+0000 7f01fb3dd740 20 client.19926 populate_metadata read hostname 'smithi005' 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.339+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aebb3ecca0 0x55aebb462730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.339+0000 7f01fb3dd740 4 client.19926 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.339+0000 7f01fb3dd740 1 -- 172.21.15.5:0/2183128179 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aebb343a20 con 0x55aebb3ecca0 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.339+0000 7f01f5be2640 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aebb3ecca0 0x55aebb462730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:20:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.339+0000 7f01d9ffb640 10 client.19926 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.339+0000 7f01f5be2640 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aebb3ecca0 0x55aebb462730 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4906ab0b8576eda server_cookie=4156a02937009078 in_seq=0 out_seq=0 2024-09-19T06:20:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01d9ffb640 1 -- 172.21.15.5:0/2183128179 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aebb343a20 con 0x55aebb3ecca0 2024-09-19T06:20:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01d9ffb640 10 client.19926 handle_command_reply: tid=0 2024-09-19T06:20:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01fb3dd740 1 client.19926 shutdown 2024-09-19T06:20:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01fb3dd740 1 -- 172.21.15.5:0/2183128179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 msgr2=0x55aebb3e0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 0x55aebb3e0a60 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0035f0 tx=0x7f01ec003620 comp rx=0 tx=0).stop 2024-09-19T06:20:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01fb3dd740 1 -- 172.21.15.5:0/2183128179 shutdown_connections 2024-09-19T06:20:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aebb3ecca0 0x55aebb462730 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aebb3dccb0 0x55aebb3e14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aebb3d2410 0x55aebb3e0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01fb3dd740 1 --2- 172.21.15.5:0/2183128179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aebb3d1af0 0x55aebb3e0a60 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01fb3dd740 1 -- 172.21.15.5:0/2183128179 >> 172.21.15.5:0/2183128179 conn(0x55aebb3ce060 msgr2=0x55aebb3dbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01d9ffb640 0 client.19926 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:20:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01fb3dd740 1 -- 172.21.15.5:0/2183128179 shutdown_connections 2024-09-19T06:20:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01fb3dd740 1 -- 172.21.15.5:0/2183128179 wait complete. 2024-09-19T06:20:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.413+0000 7f01fb3dd740 20 client.19926 trim_cache size 0 max 16384 2024-09-19T06:20:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:20:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:20:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:20:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:20:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.414+0000 7f01f8647640 1 -- 172.21.15.5:0/2752512558 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f01dc0a3430 msgr2=0x7f01dc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.414+0000 7f01f8647640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f01dc0a3430 0x7f01dc0a58f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f01ec031440 tx=0x7f01ec005a20 comp rx=0 tx=0).stop 2024-09-19T06:20:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.414+0000 7f01f8647640 1 -- 172.21.15.5:0/2752512558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f0101340 msgr2=0x7f01f0196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:20:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.414+0000 7f01f8647640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f0101340 0x7f01f0196670 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f01e0007860 tx=0x7f01e0007d20 comp rx=0 tx=0).stop 2024-09-19T06:20:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.414+0000 7f01f8647640 1 -- 172.21.15.5:0/2752512558 shutdown_connections 2024-09-19T06:20:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.414+0000 7f01f8647640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f01dc0a3430 0x7f01dc0a58f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.414+0000 7f01f8647640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f01069b0 0x7f01f019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.414+0000 7f01f8647640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f0101340 0x7f01f0196670 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.414+0000 7f01f8647640 1 --2- 172.21.15.5:0/2752512558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f01009a0 0x7f01f0196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:20:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.414+0000 7f01f8647640 1 -- 172.21.15.5:0/2752512558 >> 172.21.15.5:0/2752512558 conn(0x7f01f00779a0 msgr2=0x7f01f00ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:20:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.414+0000 7f01f8647640 1 -- 172.21.15.5:0/2752512558 shutdown_connections 2024-09-19T06:20:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:20:53.415+0000 7f01f8647640 1 -- 172.21.15.5:0/2752512558 wait complete. 2024-09-19T06:20:53.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 42638 8.65 MB/sec execute 137 sec latency 132.040 ms 2024-09-19T06:20:54.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 42843 8.71 MB/sec execute 138 sec latency 112.522 ms 2024-09-19T06:20:54.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:54 smithi005 ceph-mon[30955]: pgmap v552: 97 pgs: 97 active+clean; 752 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 445 op/s 2024-09-19T06:20:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:54 smithi049 ceph-mon[34640]: pgmap v552: 97 pgs: 97 active+clean; 752 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 445 op/s 2024-09-19T06:20:55.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:54 smithi157 ceph-mon[35346]: pgmap v552: 97 pgs: 97 active+clean; 752 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 445 op/s 2024-09-19T06:20:55.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43049 8.73 MB/sec execute 139 sec latency 87.462 ms 2024-09-19T06:20:56.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43241 8.75 MB/sec execute 140 sec latency 87.124 ms 2024-09-19T06:20:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:56 smithi005 ceph-mon[30955]: pgmap v553: 97 pgs: 97 active+clean; 752 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-09-19T06:20:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:56 smithi049 ceph-mon[34640]: pgmap v553: 97 pgs: 97 active+clean; 752 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-09-19T06:20:57.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:56 smithi157 ceph-mon[35346]: pgmap v553: 97 pgs: 97 active+clean; 752 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-09-19T06:20:57.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43409 8.78 MB/sec execute 141 sec latency 60.094 ms 2024-09-19T06:20:58.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43596 8.83 MB/sec execute 142 sec latency 55.350 ms 2024-09-19T06:20:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:20:58 smithi005 ceph-mon[30955]: pgmap v554: 97 pgs: 97 active+clean; 767 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 397 op/s 2024-09-19T06:20:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:20:58 smithi049 ceph-mon[34640]: pgmap v554: 97 pgs: 97 active+clean; 767 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 397 op/s 2024-09-19T06:20:59.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:20:58 smithi157 ceph-mon[35346]: pgmap v554: 97 pgs: 97 active+clean; 767 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 397 op/s 2024-09-19T06:20:59.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43792 8.87 MB/sec execute 143 sec latency 49.950 ms 2024-09-19T06:21:00.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43908 8.85 MB/sec execute 144 sec latency 101.406 ms 2024-09-19T06:21:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:00 smithi005 ceph-mon[30955]: pgmap v555: 97 pgs: 97 active+clean; 767 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-09-19T06:21:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:00 smithi049 ceph-mon[34640]: pgmap v555: 97 pgs: 97 active+clean; 767 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-09-19T06:21:01.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:00 smithi157 ceph-mon[35346]: pgmap v555: 97 pgs: 97 active+clean; 767 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-09-19T06:21:01.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 44032 8.81 MB/sec execute 145 sec latency 66.667 ms 2024-09-19T06:21:02.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 44213 8.82 MB/sec execute 146 sec latency 67.462 ms 2024-09-19T06:21:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:02 smithi005 ceph-mon[30955]: pgmap v556: 97 pgs: 97 active+clean; 767 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-09-19T06:21:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:02 smithi049 ceph-mon[34640]: pgmap v556: 97 pgs: 97 active+clean; 767 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-09-19T06:21:03.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:02 smithi157 ceph-mon[35346]: pgmap v556: 97 pgs: 97 active+clean; 767 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-09-19T06:21:03.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 44366 8.80 MB/sec execute 147 sec latency 57.873 ms 2024-09-19T06:21:04.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 44503 8.77 MB/sec execute 148 sec latency 58.841 ms 2024-09-19T06:21:04.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:04 smithi005 ceph-mon[30955]: pgmap v557: 97 pgs: 97 active+clean; 786 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 403 op/s 2024-09-19T06:21:04.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:04 smithi049 ceph-mon[34640]: pgmap v557: 97 pgs: 97 active+clean; 786 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 403 op/s 2024-09-19T06:21:05.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:04 smithi157 ceph-mon[35346]: pgmap v557: 97 pgs: 97 active+clean; 786 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 403 op/s 2024-09-19T06:21:05.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 44663 8.76 MB/sec execute 149 sec latency 107.374 ms 2024-09-19T06:21:05.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:05 smithi005 ceph-mon[30955]: osdmap e108: 12 total, 12 up, 12 in 2024-09-19T06:21:05.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:05 smithi049 ceph-mon[34640]: osdmap e108: 12 total, 12 up, 12 in 2024-09-19T06:21:06.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:05 smithi157 ceph-mon[35346]: osdmap e108: 12 total, 12 up, 12 in 2024-09-19T06:21:06.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 44848 8.80 MB/sec execute 150 sec latency 71.383 ms 2024-09-19T06:21:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:06 smithi005 ceph-mon[30955]: pgmap v559: 97 pgs: 97 active+clean; 786 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 335 op/s 2024-09-19T06:21:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:06 smithi049 ceph-mon[34640]: pgmap v559: 97 pgs: 97 active+clean; 786 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 335 op/s 2024-09-19T06:21:07.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:06 smithi157 ceph-mon[35346]: pgmap v559: 97 pgs: 97 active+clean; 786 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 335 op/s 2024-09-19T06:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.142+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1762004486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40ffe20 msgr2=0x7f8ff4100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.142+0000 7f8ffbee6640 1 --2- 172.21.15.5:0/1762004486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40ffe20 0x7f8ff4100200 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009890 tx=0x7f8fec02a370 comp rx=0 tx=0).stop 2024-09-19T06:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.142+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1762004486 shutdown_connections 2024-09-19T06:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.142+0000 7f8ffbee6640 1 --2- 172.21.15.5:0/1762004486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff410a360 0x7f8ff4110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.142+0000 7f8ffbee6640 1 --2- 172.21.15.5:0/1762004486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff41007d0 0x7f8ff4109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.142+0000 7f8ffbee6640 1 --2- 172.21.15.5:0/1762004486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40ffe20 0x7f8ff4100200 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.142+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1762004486 >> 172.21.15.5:0/1762004486 conn(0x7f8ff40fba00 msgr2=0x7f8ff40fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.142+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1762004486 shutdown_connections 2024-09-19T06:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.142+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1762004486 wait complete. 2024-09-19T06:21:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ffbee6640 1 Processor -- start 2024-09-19T06:21:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ffbee6640 1 -- start start 2024-09-19T06:21:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ffbee6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40ffe20 0x7f8ff41107a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ffbee6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff410a360 0x7f8ff410e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ffbee6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff4110df0 0x7f8ff410eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ffbee6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4103af0 con 0x7f8ff4110df0 2024-09-19T06:21:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ffbee6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4103970 con 0x7f8ff410a360 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ffbee6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4103c70 con 0x7f8ff40ffe20 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ff9c82640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40ffe20 0x7f8ff41107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ff9c82640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40ffe20 0x7f8ff41107a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33574/0 (socket says 172.21.15.5:33574) 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ff9c82640 1 -- 172.21.15.5:0/1836802625 learned_addr learned my addr 172.21.15.5:0/1836802625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ff9481640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff410a360 0x7f8ff410e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.143+0000 7f8ffa483640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff4110df0 0x7f8ff410eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.144+0000 7f8ff9c82640 1 -- 172.21.15.5:0/1836802625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff410a360 msgr2=0x7f8ff410e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.144+0000 7f8ff9c82640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff410a360 0x7f8ff410e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.144+0000 7f8ff9c82640 1 -- 172.21.15.5:0/1836802625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff4110df0 msgr2=0x7f8ff410eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.144+0000 7f8ff9c82640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff4110df0 0x7f8ff410eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.144+0000 7f8ff9c82640 1 -- 172.21.15.5:0/1836802625 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff41ab190 con 0x7f8ff40ffe20 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.144+0000 7f8ff9c82640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40ffe20 0x7f8ff41107a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8fec000c00 tx=0x7f8fec030bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.144+0000 7f8ff2ffd640 1 -- 172.21.15.5:0/1836802625 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec040020 con 0x7f8ff40ffe20 2024-09-19T06:21:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.144+0000 7f8ff2ffd640 1 -- 172.21.15.5:0/1836802625 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fec038040 con 0x7f8ff40ffe20 2024-09-19T06:21:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.144+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1836802625 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff41ab360 con 0x7f8ff40ffe20 2024-09-19T06:21:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.144+0000 7f8ff2ffd640 1 -- 172.21.15.5:0/1836802625 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec037070 con 0x7f8ff40ffe20 2024-09-19T06:21:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.144+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1836802625 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff41ab820 con 0x7f8ff40ffe20 2024-09-19T06:21:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.149+0000 7f8ff2ffd640 1 -- 172.21.15.5:0/1836802625 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8fec043020 con 0x7f8ff40ffe20 2024-09-19T06:21:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.150+0000 7f8ffec77740 1 Processor -- start 2024-09-19T06:21:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.150+0000 7f8ffec77740 1 -- start start 2024-09-19T06:21:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ffec77740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cafaf0 0x55a085cafed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ffec77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cb0410 0x55a085cba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ffec77740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cbacb0 0x55a085cbd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ffec77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a085c142f0 con 0x55a085cb0410 2024-09-19T06:21:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ffec77740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a085bbc270 con 0x55a085cbacb0 2024-09-19T06:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ffec77740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a085c102b0 con 0x55a085cafaf0 2024-09-19T06:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ff2ffd640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8fdc0a33b0 0x7f8fdc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ff9c82640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cb0410 0x55a085cba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ff9481640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cbacb0 0x55a085cbd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ff9481640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cbacb0 0x55a085cbd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55524/0 (socket says 172.21.15.5:55524) 2024-09-19T06:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ff9c82640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cb0410 0x55a085cba770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49228/0 (socket says 172.21.15.5:49228) 2024-09-19T06:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ff9481640 1 -- 172.21.15.5:0/2329641336 learned_addr learned my addr 172.21.15.5:0/2329641336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ffa483640 1 --2- 172.21.15.5:0/2329641336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cafaf0 0x55a085cafed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ff2ffd640 1 -- 172.21.15.5:0/1836802625 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8594+0+0 (secure 0 0 0) 0x7f8fec0ef9e0 con 0x7f8ff40ffe20 2024-09-19T06:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ffa483640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8fdc0a33b0 0x7f8fdc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ff9481640 1 -- 172.21.15.5:0/2329641336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cafaf0 msgr2=0x55a085cafed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ff9481640 1 --2- 172.21.15.5:0/2329641336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cafaf0 0x55a085cafed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ff9481640 1 -- 172.21.15.5:0/2329641336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cb0410 msgr2=0x55a085cba770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ff9481640 1 --2- 172.21.15.5:0/2329641336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cb0410 0x55a085cba770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.151+0000 7f8ff9481640 1 -- 172.21.15.5:0/2329641336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a085bbb340 con 0x55a085cbacb0 2024-09-19T06:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.152+0000 7f8ff9481640 1 --2- 172.21.15.5:0/2329641336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cbacb0 0x55a085cbd0a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800e500 tx=0x7f8fe80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4df42b667ca0e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.152+0000 7f8ffa483640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8fdc0a33b0 0x7f8fdc0a5870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005f60 tx=0x7f8fe4005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.152+0000 7f8fdbfff640 1 -- 172.21.15.5:0/2329641336 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe801d070 con 0x55a085cbacb0 2024-09-19T06:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.152+0000 7f8fdbfff640 1 -- 172.21.15.5:0/2329641336 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fe8003e40 con 0x55a085cbacb0 2024-09-19T06:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.152+0000 7f8fdbfff640 1 -- 172.21.15.5:0/2329641336 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800dac0 con 0x55a085cbacb0 2024-09-19T06:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.153+0000 7f8ffec77740 1 -- 172.21.15.5:0/2329641336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cbacb0 msgr2=0x55a085cbd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.153+0000 7f8ffec77740 1 --2- 172.21.15.5:0/2329641336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cbacb0 0x55a085cbd0a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800e500 tx=0x7f8fe80095c0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.153+0000 7f8ffec77740 1 -- 172.21.15.5:0/2329641336 shutdown_connections 2024-09-19T06:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.153+0000 7f8ffec77740 1 --2- 172.21.15.5:0/2329641336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cbacb0 0x55a085cbd0a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.153+0000 7f8ffec77740 1 --2- 172.21.15.5:0/2329641336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cb0410 0x55a085cba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.153+0000 7f8ffec77740 1 --2- 172.21.15.5:0/2329641336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cafaf0 0x55a085cafed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.153+0000 7f8ffec77740 1 -- 172.21.15.5:0/2329641336 >> 172.21.15.5:0/2329641336 conn(0x55a085cac060 msgr2=0x55a085cac440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.153+0000 7f8ffec77740 1 -- 172.21.15.5:0/2329641336 shutdown_connections 2024-09-19T06:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.153+0000 7f8ffec77740 1 -- 172.21.15.5:0/2329641336 wait complete. 2024-09-19T06:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.153+0000 7f8ffec77740 1 -- start start 2024-09-19T06:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.153+0000 7f8ffec77740 1 Processor -- start 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ffec77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cafaf0 0x55a085cb7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ffec77740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cb0410 0x55a085cb8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ffec77740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cbacb0 0x55a085cb85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ffec77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a085cbd880 con 0x55a085cafaf0 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ffec77740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a085b9d290 con 0x55a085cb0410 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ffec77740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a085cbda00 con 0x55a085cbacb0 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ffec77740 10 client.0 _ll_register_callbacks cb 0x55a085bce140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ff9c82640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cafaf0 0x55a085cb7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ff9c82640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cafaf0 0x55a085cb7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49234/0 (socket says 172.21.15.5:49234) 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ff9c82640 1 -- 172.21.15.5:0/3037063112 learned_addr learned my addr 172.21.15.5:0/3037063112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ffa483640 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cbacb0 0x55a085cb85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ff9481640 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cb0410 0x55a085cb8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ff9c82640 1 -- 172.21.15.5:0/3037063112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cbacb0 msgr2=0x55a085cb85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ff9c82640 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cbacb0 0x55a085cb85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ff9c82640 1 -- 172.21.15.5:0/3037063112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cb0410 msgr2=0x55a085cb8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ff9c82640 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cb0410 0x55a085cb8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.154+0000 7f8ff9c82640 1 -- 172.21.15.5:0/3037063112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a085927210 con 0x55a085cafaf0 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.155+0000 7f8ff9c82640 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cafaf0 0x55a085cb7b40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0ed1f0 tx=0x7f8fec08aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f749bd3db72b3db7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.155+0000 7f8fd97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.155+0000 7f8fd97fa640 1 -- 172.21.15.5:0/3037063112 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0328f0 con 0x55a085cafaf0 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.155+0000 7f8fd97fa640 1 -- 172.21.15.5:0/3037063112 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8594+0+0 (secure 0 0 0) 0x7f8fec0c4500 con 0x55a085cafaf0 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.155+0000 7f8ffec77740 1 -- 172.21.15.5:0/3037063112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a085c21a20 con 0x55a085cafaf0 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.155+0000 7f8fd97fa640 1 -- 172.21.15.5:0/3037063112 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8fec0ed500 con 0x55a085cafaf0 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.155+0000 7f8ffec77740 10 client.19938 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.155+0000 7f8ffec77740 1 -- 172.21.15.5:0/3037063112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55a0859f3620 con 0x55a085cafaf0 2024-09-19T06:21:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.156+0000 7f8fd97fa640 1 -- 172.21.15.5:0/3037063112 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a085c21a20 con 0x55a085cafaf0 2024-09-19T06:21:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.157+0000 7f8fd97fa640 1 -- 172.21.15.5:0/3037063112 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f8fec08ebc0 con 0x55a085cafaf0 2024-09-19T06:21:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.157+0000 7f8ffec77740 10 client.19938 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.157+0000 7f8ffec77740 10 client.19938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.157+0000 7f8ffec77740 20 client.19938 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.157+0000 7f8ffec77740 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a085ccabe0 0x55a085ccd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.157+0000 7f8ffec77740 4 client.19938 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:21:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.157+0000 7f8ffec77740 1 -- 172.21.15.5:0/3037063112 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a085c1bd70 con 0x55a085ccabe0 2024-09-19T06:21:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.157+0000 7f8ff9481640 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a085ccabe0 0x55a085ccd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.158+0000 7f8ff9481640 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a085ccabe0 0x55a085ccd050 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b91c0e0add658fa server_cookie=918678bde0aa81bd in_seq=0 out_seq=0 2024-09-19T06:21:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.158+0000 7f8fd97fa640 10 client.19938 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.159+0000 7f8fd97fa640 1 -- 172.21.15.5:0/3037063112 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a085c1bd70 con 0x55a085ccabe0 2024-09-19T06:21:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.159+0000 7f8fd97fa640 10 client.19938 handle_command_reply: tid=0 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.159+0000 7f8ffec77740 1 client.19938 shutdown 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.159+0000 7f8ffec77740 1 -- 172.21.15.5:0/3037063112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cafaf0 msgr2=0x55a085cb7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.159+0000 7f8ffec77740 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cafaf0 0x55a085cb7b40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0ed1f0 tx=0x7f8fec08aa10 comp rx=0 tx=0).stop 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.159+0000 7f8ffec77740 1 -- 172.21.15.5:0/3037063112 shutdown_connections 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.159+0000 7f8ffec77740 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a085ccabe0 0x55a085ccd050 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.159+0000 7f8ffec77740 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cbacb0 0x55a085cb85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.159+0000 7f8fd97fa640 0 client.19938 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.159+0000 7f8ffec77740 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cb0410 0x55a085cb8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.159+0000 7f8ffec77740 1 --2- 172.21.15.5:0/3037063112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cafaf0 0x55a085cb7b40 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.160+0000 7f8ffec77740 1 -- 172.21.15.5:0/3037063112 >> 172.21.15.5:0/3037063112 conn(0x55a085cac060 msgr2=0x55a085cb1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.160+0000 7f8ffec77740 1 -- 172.21.15.5:0/3037063112 shutdown_connections 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.160+0000 7f8ffec77740 1 -- 172.21.15.5:0/3037063112 wait complete. 2024-09-19T06:21:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.160+0000 7f8ffec77740 20 client.19938 trim_cache size 0 max 16384 2024-09-19T06:21:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.184+0000 7f8ffec77740 1 Processor -- start 2024-09-19T06:21:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ffec77740 1 -- start start 2024-09-19T06:21:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ffec77740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cafaf0 0x55a085cb6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ffec77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cb0410 0x55a085cb83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ffec77740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cbacb0 0x55a085cd2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ffec77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41b3ee0 con 0x55a085cb0410 2024-09-19T06:21:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ffec77740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41b5a20 con 0x55a085cafaf0 2024-09-19T06:21:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ffec77740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41b45b0 con 0x55a085cbacb0 2024-09-19T06:21:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ff9c82640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cafaf0 0x55a085cb6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ff9481640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cb0410 0x55a085cb83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ff9c82640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cafaf0 0x55a085cb6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55542/0 (socket says 172.21.15.5:55542) 2024-09-19T06:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ff9c82640 1 -- 172.21.15.5:0/3802117733 learned_addr learned my addr 172.21.15.5:0/3802117733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ffa483640 1 --2- 172.21.15.5:0/3802117733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cbacb0 0x55a085cd2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ff9c82640 1 -- 172.21.15.5:0/3802117733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cbacb0 msgr2=0x55a085cd2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ff9c82640 1 --2- 172.21.15.5:0/3802117733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cbacb0 0x55a085cd2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ff9c82640 1 -- 172.21.15.5:0/3802117733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cb0410 msgr2=0x55a085cb83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ff9c82640 1 --2- 172.21.15.5:0/3802117733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cb0410 0x55a085cb83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.185+0000 7f8ff9c82640 1 -- 172.21.15.5:0/3802117733 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a085cabe90 con 0x55a085cafaf0 2024-09-19T06:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.186+0000 7f8ff9c82640 1 --2- 172.21.15.5:0/3802117733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cafaf0 0x55a085cb6a60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0b86f0 tx=0x7f8fec005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9512b0d9a7e8cfd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.186+0000 7f8fdb7fe640 1 -- 172.21.15.5:0/3802117733 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0bbaf0 con 0x55a085cafaf0 2024-09-19T06:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.186+0000 7f8fdb7fe640 1 -- 172.21.15.5:0/3802117733 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fec0328f0 con 0x55a085cafaf0 2024-09-19T06:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.186+0000 7f8fdb7fe640 1 -- 172.21.15.5:0/3802117733 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0c00d0 con 0x55a085cafaf0 2024-09-19T06:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.187+0000 7f8ffec77740 1 -- 172.21.15.5:0/3802117733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cafaf0 msgr2=0x55a085cb6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.187+0000 7f8ffec77740 1 --2- 172.21.15.5:0/3802117733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cafaf0 0x55a085cb6a60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0b86f0 tx=0x7f8fec005c90 comp rx=0 tx=0).stop 2024-09-19T06:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.187+0000 7f8ffec77740 1 -- 172.21.15.5:0/3802117733 shutdown_connections 2024-09-19T06:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.187+0000 7f8ffec77740 1 --2- 172.21.15.5:0/3802117733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cbacb0 0x55a085cd2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.187+0000 7f8ffec77740 1 --2- 172.21.15.5:0/3802117733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cb0410 0x55a085cb83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.187+0000 7f8ffec77740 1 --2- 172.21.15.5:0/3802117733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cafaf0 0x55a085cb6a60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.187+0000 7f8ffec77740 1 -- 172.21.15.5:0/3802117733 >> 172.21.15.5:0/3802117733 conn(0x55a085cac060 msgr2=0x55a085cb9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.187+0000 7f8ffec77740 1 -- 172.21.15.5:0/3802117733 shutdown_connections 2024-09-19T06:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.187+0000 7f8ffec77740 1 -- 172.21.15.5:0/3802117733 wait complete. 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.187+0000 7f8ffec77740 1 -- start start 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.187+0000 7f8ffec77740 1 Processor -- start 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ffec77740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cafaf0 0x55a085cc2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ffec77740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cb0410 0x55a085cc33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ffec77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cbacb0 0x55a085cb4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ffec77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41b5350 con 0x55a085cbacb0 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ffec77740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41b4c80 con 0x55a085cb0410 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ffec77740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41b1ee0 con 0x55a085cafaf0 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ffec77740 10 client.0 _ll_register_callbacks cb 0x7f8fec0097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ff9c82640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cafaf0 0x55a085cc2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ffa483640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cbacb0 0x55a085cb4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ff9c82640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cafaf0 0x55a085cc2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33596/0 (socket says 172.21.15.5:33596) 2024-09-19T06:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ff9c82640 1 -- 172.21.15.5:0/2910389621 learned_addr learned my addr 172.21.15.5:0/2910389621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ff9481640 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cb0410 0x55a085cc33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ff9c82640 1 -- 172.21.15.5:0/2910389621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cb0410 msgr2=0x55a085cc33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ff9c82640 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cb0410 0x55a085cc33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ff9c82640 1 -- 172.21.15.5:0/2910389621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cbacb0 msgr2=0x55a085cb4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.188+0000 7f8ff9c82640 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cbacb0 0x55a085cb4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.189+0000 7f8ff9c82640 1 -- 172.21.15.5:0/2910389621 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a085caccf0 con 0x55a085cafaf0 2024-09-19T06:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.189+0000 7f8ffa483640 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cbacb0 0x55a085cb4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.189+0000 7f8ff9c82640 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cafaf0 0x55a085cc2e80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0ba270 tx=0x7f8fec0efc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d30870fed2468fab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.189+0000 7f8fd97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.189+0000 7f8fd97fa640 1 -- 172.21.15.5:0/2910389621 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec08b070 con 0x55a085cafaf0 2024-09-19T06:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.189+0000 7f8fd97fa640 1 -- 172.21.15.5:0/2910389621 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8594+0+0 (secure 0 0 0) 0x7f8fec0bb370 con 0x55a085cafaf0 2024-09-19T06:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.189+0000 7f8ffec77740 1 -- 172.21.15.5:0/2910389621 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fec0328f0 con 0x55a085cafaf0 2024-09-19T06:21:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.189+0000 7f8fd97fa640 1 -- 172.21.15.5:0/2910389621 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8fec0bbaf0 con 0x55a085cafaf0 2024-09-19T06:21:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.189+0000 7f8ffec77740 10 client.29260 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.189+0000 7f8ffec77740 1 -- 172.21.15.5:0/2910389621 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55a085caccf0 con 0x55a085cafaf0 2024-09-19T06:21:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.190+0000 7f8fd97fa640 1 -- 172.21.15.5:0/2910389621 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fec0c00d0 con 0x55a085cafaf0 2024-09-19T06:21:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.190+0000 7f8fd97fa640 1 -- 172.21.15.5:0/2910389621 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f8fec0babe0 con 0x55a085cafaf0 2024-09-19T06:21:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.190+0000 7f8ffec77740 10 client.29260 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.190+0000 7f8ffec77740 10 client.29260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.190+0000 7f8ffec77740 20 client.29260 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.190+0000 7f8ffec77740 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a085ccabe0 0x55a085d37550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.190+0000 7f8ffec77740 4 client.29260 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:21:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.190+0000 7f8ffec77740 1 -- 172.21.15.5:0/2910389621 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8fec0328f0 con 0x55a085ccabe0 2024-09-19T06:21:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.190+0000 7f8ff9481640 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a085ccabe0 0x55a085d37550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.190+0000 7f8fd97fa640 10 client.29260 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.191+0000 7f8ff9481640 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a085ccabe0 0x55a085d37550 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc1a74678939fe44 server_cookie=44668b593ef41f7c in_seq=0 out_seq=0 2024-09-19T06:21:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.280+0000 7f8fd97fa640 1 -- 172.21.15.5:0/2910389621 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8fec0328f0 con 0x55a085ccabe0 2024-09-19T06:21:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.280+0000 7f8fd97fa640 10 client.29260 handle_command_reply: tid=0 2024-09-19T06:21:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.280+0000 7f8ffec77740 1 client.29260 shutdown 2024-09-19T06:21:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.281+0000 7f8ffec77740 1 -- 172.21.15.5:0/2910389621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cafaf0 msgr2=0x55a085cc2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.281+0000 7f8ffec77740 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cafaf0 0x55a085cc2e80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0ba270 tx=0x7f8fec0efc60 comp rx=0 tx=0).stop 2024-09-19T06:21:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.281+0000 7f8ffec77740 1 -- 172.21.15.5:0/2910389621 shutdown_connections 2024-09-19T06:21:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.281+0000 7f8ffec77740 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a085ccabe0 0x55a085d37550 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.281+0000 7f8ffec77740 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a085cbacb0 0x55a085cb4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.281+0000 7f8ffec77740 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a085cb0410 0x55a085cc33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.281+0000 7f8ffec77740 1 --2- 172.21.15.5:0/2910389621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a085cafaf0 0x55a085cc2e80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.281+0000 7f8fd97fa640 0 client.29260 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.281+0000 7f8ffec77740 1 -- 172.21.15.5:0/2910389621 >> 172.21.15.5:0/2910389621 conn(0x55a085cac060 msgr2=0x55a085caa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.281+0000 7f8ffec77740 1 -- 172.21.15.5:0/2910389621 shutdown_connections 2024-09-19T06:21:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.281+0000 7f8ffec77740 1 -- 172.21.15.5:0/2910389621 wait complete. 2024-09-19T06:21:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.281+0000 7f8ffec77740 20 client.29260 trim_cache size 0 max 16384 2024-09-19T06:21:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:21:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:21:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:21:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:21:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.282+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1836802625 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8fdc0a33b0 msgr2=0x7f8fdc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.282+0000 7f8ffbee6640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8fdc0a33b0 0x7f8fdc0a5870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005f60 tx=0x7f8fe4005f90 comp rx=0 tx=0).stop 2024-09-19T06:21:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.282+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1836802625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40ffe20 msgr2=0x7f8ff41107a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.282+0000 7f8ffbee6640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40ffe20 0x7f8ff41107a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8fec000c00 tx=0x7f8fec030bc0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.282+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1836802625 shutdown_connections 2024-09-19T06:21:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.282+0000 7f8ffbee6640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff4110df0 0x7f8ff410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.282+0000 7f8ffbee6640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff410a360 0x7f8ff410e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.282+0000 7f8ffbee6640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8fdc0a33b0 0x7f8fdc0a5870 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.282+0000 7f8ffbee6640 1 --2- 172.21.15.5:0/1836802625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40ffe20 0x7f8ff41107a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.282+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1836802625 >> 172.21.15.5:0/1836802625 conn(0x7f8ff40fba00 msgr2=0x7f8ff40fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.282+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1836802625 shutdown_connections 2024-09-19T06:21:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:07.282+0000 7f8ffbee6640 1 -- 172.21.15.5:0/1836802625 wait complete. 2024-09-19T06:21:07.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 45010 8.79 MB/sec execute 151 sec latency 141.803 ms 2024-09-19T06:21:07.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:07 smithi005 ceph-mon[30955]: pgmap v560: 97 pgs: 97 active+clean; 809 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.5 MiB/s wr, 328 op/s 2024-09-19T06:21:07.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:07 smithi049 ceph-mon[34640]: pgmap v560: 97 pgs: 97 active+clean; 809 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.5 MiB/s wr, 328 op/s 2024-09-19T06:21:08.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:07 smithi157 ceph-mon[35346]: pgmap v560: 97 pgs: 97 active+clean; 809 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.5 MiB/s wr, 328 op/s 2024-09-19T06:21:08.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 45157 8.75 MB/sec execute 152 sec latency 59.318 ms 2024-09-19T06:21:09.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:21:09.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 45326 8.75 MB/sec execute 153 sec latency 131.213 ms 2024-09-19T06:21:09.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:09 smithi157 ceph-mon[35346]: osdmap e109: 12 total, 12 up, 12 in 2024-09-19T06:21:09.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:09 smithi005 ceph-mon[30955]: osdmap e109: 12 total, 12 up, 12 in 2024-09-19T06:21:09.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:09 smithi049 ceph-mon[34640]: osdmap e109: 12 total, 12 up, 12 in 2024-09-19T06:21:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:21:10.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 45465 8.74 MB/sec execute 154 sec latency 163.194 ms 2024-09-19T06:21:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:10 smithi157 ceph-mon[35346]: pgmap v562: 97 pgs: 97 active+clean; 809 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.1 MiB/s wr, 410 op/s 2024-09-19T06:21:10.832 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:10 smithi005 ceph-mon[30955]: pgmap v562: 97 pgs: 97 active+clean; 809 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.1 MiB/s wr, 410 op/s 2024-09-19T06:21:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:10 smithi049 ceph-mon[34640]: pgmap v562: 97 pgs: 97 active+clean; 809 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.1 MiB/s wr, 410 op/s 2024-09-19T06:21:11.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.388+0000 7f576b5c7640 1 -- 172.21.15.5:0/2666677418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5764103ff0 msgr2=0x7f57641043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:11.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.388+0000 7f576b5c7640 1 --2- 172.21.15.5:0/2666677418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5764103ff0 0x7f57641043d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5760009900 tx=0x7f576002a410 comp rx=0 tx=0).stop 2024-09-19T06:21:11.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.388+0000 7f576b5c7640 1 -- 172.21.15.5:0/2666677418 shutdown_connections 2024-09-19T06:21:11.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.388+0000 7f576b5c7640 1 --2- 172.21.15.5:0/2666677418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5764109120 0x7f5764110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.388+0000 7f576b5c7640 1 --2- 172.21.15.5:0/2666677418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57641049a0 0x7f57641089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.388+0000 7f576b5c7640 1 --2- 172.21.15.5:0/2666677418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5764103ff0 0x7f57641043d0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.388+0000 7f576b5c7640 1 -- 172.21.15.5:0/2666677418 >> 172.21.15.5:0/2666677418 conn(0x7f57640ffa90 msgr2=0x7f5764101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:11.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.388+0000 7f576b5c7640 1 -- 172.21.15.5:0/2666677418 shutdown_connections 2024-09-19T06:21:11.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.388+0000 7f576b5c7640 1 -- 172.21.15.5:0/2666677418 wait complete. 2024-09-19T06:21:11.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.389+0000 7f576b5c7640 1 Processor -- start 2024-09-19T06:21:11.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.389+0000 7f576b5c7640 1 -- start start 2024-09-19T06:21:11.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f576b5c7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764103ff0 0x7f5764073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f576b5c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57641049a0 0x7f576406cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f576b5c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5764109120 0x7f576406d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f576b5c7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5764114020 con 0x7f57641049a0 2024-09-19T06:21:11.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f576b5c7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5764113ea0 con 0x7f5764103ff0 2024-09-19T06:21:11.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f576b5c7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57641141a0 con 0x7f5764109120 2024-09-19T06:21:11.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f5768b62640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57641049a0 0x7f576406cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f5769b64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5764109120 0x7f576406d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f5769b64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5764109120 0x7f576406d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33622/0 (socket says 172.21.15.5:33622) 2024-09-19T06:21:11.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f5769b64640 1 -- 172.21.15.5:0/1134782221 learned_addr learned my addr 172.21.15.5:0/1134782221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:11.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f5769363640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764103ff0 0x7f5764073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f5769b64640 1 -- 172.21.15.5:0/1134782221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764103ff0 msgr2=0x7f5764073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f5769b64640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764103ff0 0x7f5764073a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f5769b64640 1 -- 172.21.15.5:0/1134782221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57641049a0 msgr2=0x7f576406cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f5769b64640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57641049a0 0x7f576406cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.390+0000 7f5769b64640 1 -- 172.21.15.5:0/1134782221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576406d680 con 0x7f5764109120 2024-09-19T06:21:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.391+0000 7f5769b64640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5764109120 0x7f576406d040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f575c00eb40 tx=0x7f575c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.391+0000 7f575a7fc640 1 -- 172.21.15.5:0/1134782221 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c004b20 con 0x7f5764109120 2024-09-19T06:21:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.391+0000 7f575a7fc640 1 -- 172.21.15.5:0/1134782221 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f575c004cc0 con 0x7f5764109120 2024-09-19T06:21:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.391+0000 7f576b5c7640 1 -- 172.21.15.5:0/1134782221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57640715a0 con 0x7f5764109120 2024-09-19T06:21:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.391+0000 7f575a7fc640 1 -- 172.21.15.5:0/1134782221 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c010690 con 0x7f5764109120 2024-09-19T06:21:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.391+0000 7f576b5c7640 1 -- 172.21.15.5:0/1134782221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5764071ae0 con 0x7f5764109120 2024-09-19T06:21:11.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.395+0000 7f576d9a0740 1 Processor -- start 2024-09-19T06:21:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.396+0000 7f575a7fc640 1 -- 172.21.15.5:0/1134782221 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f575c005020 con 0x7f5764109120 2024-09-19T06:21:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.396+0000 7f576d9a0740 1 -- start start 2024-09-19T06:21:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.396+0000 7f576d9a0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 0x55ecf87da880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.396+0000 7f576d9a0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87dadc0 0x55ecf87e5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.396+0000 7f576d9a0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87e5680 0x55ecf87e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.396+0000 7f576d9a0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ecf87402e0 con 0x55ecf87dadc0 2024-09-19T06:21:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.396+0000 7f576d9a0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ecf86e7ce0 con 0x55ecf87da4a0 2024-09-19T06:21:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.396+0000 7f576d9a0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ecf873c2a0 con 0x55ecf87e5680 2024-09-19T06:21:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.396+0000 7f5768b62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 0x55ecf87da880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.396+0000 7f5768b62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 0x55ecf87da880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55584/0 (socket says 172.21.15.5:55584) 2024-09-19T06:21:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.396+0000 7f5768b62640 1 -- 172.21.15.5:0/2632459379 learned_addr learned my addr 172.21.15.5:0/2632459379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.397+0000 7f5769363640 1 --2- 172.21.15.5:0/2632459379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87dadc0 0x55ecf87e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.397+0000 7f5768b62640 1 --2- 172.21.15.5:0/2632459379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87e5680 0x55ecf87e7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.397+0000 7f5768b62640 1 -- 172.21.15.5:0/2632459379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87e5680 msgr2=0x55ecf87e7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.397+0000 7f5768b62640 1 --2- 172.21.15.5:0/2632459379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87e5680 0x55ecf87e7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.397+0000 7f5768b62640 1 -- 172.21.15.5:0/2632459379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87dadc0 msgr2=0x55ecf87e5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.397+0000 7f5768b62640 1 --2- 172.21.15.5:0/2632459379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87dadc0 0x55ecf87e5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.397+0000 7f5768b62640 1 -- 172.21.15.5:0/2632459379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecf86cc440 con 0x55ecf87da4a0 2024-09-19T06:21:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.397+0000 7f5768b62640 1 --2- 172.21.15.5:0/2632459379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 0x55ecf87da880 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5754009730 tx=0x7f5754017d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f9e8b03e4f8f5e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:11.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.398+0000 7f574f7fe640 1 -- 172.21.15.5:0/2632459379 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575400e9f0 con 0x55ecf87da4a0 2024-09-19T06:21:11.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.398+0000 7f574f7fe640 1 -- 172.21.15.5:0/2632459379 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f575400eb90 con 0x55ecf87da4a0 2024-09-19T06:21:11.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.398+0000 7f574f7fe640 1 -- 172.21.15.5:0/2632459379 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754003d20 con 0x55ecf87da4a0 2024-09-19T06:21:11.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.398+0000 7f575a7fc640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57500a34b0 0x7f57500a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.398+0000 7f5769363640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57500a34b0 0x7f57500a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.398+0000 7f575a7fc640 1 -- 172.21.15.5:0/1134782221 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f575c0d02e0 con 0x7f5764109120 2024-09-19T06:21:11.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.399+0000 7f576d9a0740 1 -- 172.21.15.5:0/2632459379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 msgr2=0x55ecf87da880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:11.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.399+0000 7f576d9a0740 1 --2- 172.21.15.5:0/2632459379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 0x55ecf87da880 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5754009730 tx=0x7f5754017d40 comp rx=0 tx=0).stop 2024-09-19T06:21:11.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.399+0000 7f576d9a0740 1 -- 172.21.15.5:0/2632459379 shutdown_connections 2024-09-19T06:21:11.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.399+0000 7f576d9a0740 1 --2- 172.21.15.5:0/2632459379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87e5680 0x55ecf87e7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.399+0000 7f576d9a0740 1 --2- 172.21.15.5:0/2632459379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87dadc0 0x55ecf87e5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.399+0000 7f576d9a0740 1 --2- 172.21.15.5:0/2632459379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 0x55ecf87da880 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.399+0000 7f576d9a0740 1 -- 172.21.15.5:0/2632459379 >> 172.21.15.5:0/2632459379 conn(0x55ecf87d6a30 msgr2=0x55ecf87d6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:11.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.399+0000 7f576d9a0740 1 -- 172.21.15.5:0/2632459379 shutdown_connections 2024-09-19T06:21:11.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.399+0000 7f576d9a0740 1 -- 172.21.15.5:0/2632459379 wait complete. 2024-09-19T06:21:11.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.399+0000 7f5769363640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57500a34b0 0x7f57500a5970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f576002aa00 tx=0x7f5760005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.399+0000 7f576d9a0740 1 -- start start 2024-09-19T06:21:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f576d9a0740 1 Processor -- start 2024-09-19T06:21:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f576d9a0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 0x55ecf87e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f576d9a0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87dadc0 0x55ecf87e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f576d9a0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87e5680 0x55ecf87e27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f576d9a0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ecf87e8250 con 0x55ecf87e5680 2024-09-19T06:21:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f576d9a0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ecf86fb050 con 0x55ecf87dadc0 2024-09-19T06:21:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f576d9a0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ecf87e83d0 con 0x55ecf87da4a0 2024-09-19T06:21:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f576d9a0740 10 client.0 _ll_register_callbacks cb 0x55ecf873b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f5769363640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 0x55ecf87e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f5768b62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87dadc0 0x55ecf87e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f5769363640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 0x55ecf87e4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33646/0 (socket says 172.21.15.5:33646) 2024-09-19T06:21:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f5769363640 1 -- 172.21.15.5:0/1933921239 learned_addr learned my addr 172.21.15.5:0/1933921239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.400+0000 7f5769b64640 1 --2- 172.21.15.5:0/1933921239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87e5680 0x55ecf87e27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.401+0000 7f5769363640 1 -- 172.21.15.5:0/1933921239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87dadc0 msgr2=0x55ecf87e2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:11.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.401+0000 7f5769363640 1 --2- 172.21.15.5:0/1933921239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87dadc0 0x55ecf87e2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.401+0000 7f5769363640 1 -- 172.21.15.5:0/1933921239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87e5680 msgr2=0x55ecf87e27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:11.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.401+0000 7f5769363640 1 --2- 172.21.15.5:0/1933921239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87e5680 0x55ecf87e27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.401+0000 7f5769363640 1 -- 172.21.15.5:0/1933921239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecf84b13c0 con 0x55ecf87da4a0 2024-09-19T06:21:11.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.401+0000 7f5769363640 1 --2- 172.21.15.5:0/1933921239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 0x55ecf87e4fb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5760005c40 tx=0x7f576002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61b735537f40d11d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:11.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.401+0000 7f574cff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:21:11.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.402+0000 7f574cff9640 1 -- 172.21.15.5:0/1933921239 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576004d020 con 0x55ecf87da4a0 2024-09-19T06:21:11.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.402+0000 7f576d9a0740 1 -- 172.21.15.5:0/1933921239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f575400eb90 con 0x55ecf87da4a0 2024-09-19T06:21:11.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.402+0000 7f574cff9640 1 -- 172.21.15.5:0/1933921239 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f57600055d0 con 0x55ecf87da4a0 2024-09-19T06:21:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.402+0000 7f574cff9640 1 -- 172.21.15.5:0/1933921239 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f5760058050 con 0x55ecf87da4a0 2024-09-19T06:21:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.402+0000 7f576d9a0740 10 client.29269 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.402+0000 7f576d9a0740 1 -- 172.21.15.5:0/1933921239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55ecf87f3c50 con 0x55ecf87da4a0 2024-09-19T06:21:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.402+0000 7f574cff9640 1 -- 172.21.15.5:0/1933921239 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f575400eb90 con 0x55ecf87da4a0 2024-09-19T06:21:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.402+0000 7f574cff9640 1 -- 172.21.15.5:0/1933921239 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f5760003a20 con 0x55ecf87da4a0 2024-09-19T06:21:11.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.403+0000 7f576d9a0740 10 client.29269 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:11.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.403+0000 7f576d9a0740 10 client.29269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:11.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.403+0000 7f576d9a0740 20 client.29269 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:11.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.403+0000 7f576d9a0740 1 --2- 172.21.15.5:0/1933921239 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ecf87f54e0 0x55ecf87f7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.403+0000 7f576d9a0740 4 client.29269 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:21:11.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.403+0000 7f576d9a0740 1 -- 172.21.15.5:0/1933921239 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecf874d850 con 0x55ecf87f54e0 2024-09-19T06:21:11.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.403+0000 7f5768b62640 1 --2- 172.21.15.5:0/1933921239 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ecf87f54e0 0x55ecf87f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.403+0000 7f5768b62640 1 --2- 172.21.15.5:0/1933921239 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ecf87f54e0 0x55ecf87f7970 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d7ceb1e92824b8 server_cookie=5249539dcefd60ad in_seq=0 out_seq=0 2024-09-19T06:21:11.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.403+0000 7f574cff9640 10 client.29269 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:11.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.404+0000 7f574cff9640 1 -- 172.21.15.5:0/1933921239 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ecf874d850 con 0x55ecf87f54e0 2024-09-19T06:21:11.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.404+0000 7f574cff9640 10 client.29269 handle_command_reply: tid=0 2024-09-19T06:21:11.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.404+0000 7f576d9a0740 1 client.29269 shutdown 2024-09-19T06:21:11.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.405+0000 7f576d9a0740 1 -- 172.21.15.5:0/1933921239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 msgr2=0x55ecf87e4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:11.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.405+0000 7f576d9a0740 1 --2- 172.21.15.5:0/1933921239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 0x55ecf87e4fb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5760005c40 tx=0x7f576002a410 comp rx=0 tx=0).stop 2024-09-19T06:21:11.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.405+0000 7f576d9a0740 1 -- 172.21.15.5:0/1933921239 shutdown_connections 2024-09-19T06:21:11.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.405+0000 7f576d9a0740 1 --2- 172.21.15.5:0/1933921239 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ecf87f54e0 0x55ecf87f7970 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.405+0000 7f576d9a0740 1 --2- 172.21.15.5:0/1933921239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87e5680 0x55ecf87e27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.405+0000 7f576d9a0740 1 --2- 172.21.15.5:0/1933921239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87dadc0 0x55ecf87e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.405+0000 7f574cff9640 0 client.29269 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:11.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.405+0000 7f576d9a0740 1 --2- 172.21.15.5:0/1933921239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 0x55ecf87e4fb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.405+0000 7f576d9a0740 1 -- 172.21.15.5:0/1933921239 >> 172.21.15.5:0/1933921239 conn(0x55ecf87d6a30 msgr2=0x55ecf87dc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:11.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.405+0000 7f576d9a0740 1 -- 172.21.15.5:0/1933921239 shutdown_connections 2024-09-19T06:21:11.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.405+0000 7f576d9a0740 1 -- 172.21.15.5:0/1933921239 wait complete. 2024-09-19T06:21:11.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.405+0000 7f576d9a0740 20 client.29269 trim_cache size 0 max 16384 2024-09-19T06:21:11.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.435+0000 7f576d9a0740 1 Processor -- start 2024-09-19T06:21:11.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.435+0000 7f576d9a0740 1 -- start start 2024-09-19T06:21:11.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.435+0000 7f576d9a0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 0x55ecf87d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.435+0000 7f576d9a0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 0x55ecf87fd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.435+0000 7f576d9a0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87dadc0 0x55ecf87fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.435+0000 7f576d9a0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57641b3e10 con 0x55ecf87d6a30 2024-09-19T06:21:11.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.435+0000 7f576d9a0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f57641b5950 con 0x55ecf87dadc0 2024-09-19T06:21:11.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.435+0000 7f576d9a0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57641b44e0 con 0x55ecf87da4a0 2024-09-19T06:21:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.435+0000 7f5769363640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 0x55ecf87d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.435+0000 7f5769363640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 0x55ecf87d5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49290/0 (socket says 172.21.15.5:49290) 2024-09-19T06:21:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.435+0000 7f5769363640 1 -- 172.21.15.5:0/2970404142 learned_addr learned my addr 172.21.15.5:0/2970404142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f5769b64640 1 --2- 172.21.15.5:0/2970404142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87dadc0 0x55ecf87fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f5768b62640 1 --2- 172.21.15.5:0/2970404142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 0x55ecf87fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f5769b64640 1 -- 172.21.15.5:0/2970404142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 msgr2=0x55ecf87fd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f5769b64640 1 --2- 172.21.15.5:0/2970404142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 0x55ecf87fd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f5769b64640 1 -- 172.21.15.5:0/2970404142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 msgr2=0x55ecf87d5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f5769b64640 1 --2- 172.21.15.5:0/2970404142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 0x55ecf87d5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f5769b64640 1 -- 172.21.15.5:0/2970404142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecf87d7ae0 con 0x55ecf87dadc0 2024-09-19T06:21:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f5769b64640 1 --2- 172.21.15.5:0/2970404142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87dadc0 0x55ecf87fd760 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f575c0cd980 tx=0x7f575c0988a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc54b98eeb75326b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:11.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f574effd640 1 -- 172.21.15.5:0/2970404142 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c0b4070 con 0x55ecf87dadc0 2024-09-19T06:21:11.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f574effd640 1 -- 172.21.15.5:0/2970404142 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f575c0bc040 con 0x55ecf87dadc0 2024-09-19T06:21:11.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f5769363640 1 --2- 172.21.15.5:0/2970404142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 0x55ecf87d5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:11.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f574effd640 1 -- 172.21.15.5:0/2970404142 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c0cdc30 con 0x55ecf87dadc0 2024-09-19T06:21:11.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.436+0000 7f5768b62640 1 --2- 172.21.15.5:0/2970404142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 0x55ecf87fd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:21:11.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.437+0000 7f576d9a0740 1 -- 172.21.15.5:0/2970404142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87dadc0 msgr2=0x55ecf87fd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:11.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.437+0000 7f576d9a0740 1 --2- 172.21.15.5:0/2970404142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87dadc0 0x55ecf87fd760 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f575c0cd980 tx=0x7f575c0988a0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.438+0000 7f576d9a0740 1 -- 172.21.15.5:0/2970404142 shutdown_connections 2024-09-19T06:21:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.438+0000 7f576d9a0740 1 --2- 172.21.15.5:0/2970404142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87dadc0 0x55ecf87fd760 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.438+0000 7f576d9a0740 1 --2- 172.21.15.5:0/2970404142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87da4a0 0x55ecf87fd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.438+0000 7f576d9a0740 1 --2- 172.21.15.5:0/2970404142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 0x55ecf87d5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.438+0000 7f576d9a0740 1 -- 172.21.15.5:0/2970404142 >> 172.21.15.5:0/2970404142 conn(0x55ecf87e2900 msgr2=0x55ecf87d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.438+0000 7f576d9a0740 1 -- 172.21.15.5:0/2970404142 shutdown_connections 2024-09-19T06:21:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.438+0000 7f576d9a0740 1 -- 172.21.15.5:0/2970404142 wait complete. 2024-09-19T06:21:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.438+0000 7f576d9a0740 1 -- start start 2024-09-19T06:21:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.439+0000 7f576d9a0740 1 Processor -- start 2024-09-19T06:21:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.439+0000 7f576d9a0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 0x55ecf87edd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f576d9a0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 0x55ecf87eb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f576d9a0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87dadc0 0x55ecf87eb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f576d9a0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57641b5280 con 0x55ecf87d6a30 2024-09-19T06:21:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f576d9a0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f57641b4bb0 con 0x55ecf87da4a0 2024-09-19T06:21:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f576d9a0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f576407a800 con 0x55ecf87dadc0 2024-09-19T06:21:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f576d9a0740 10 client.0 _ll_register_callbacks cb 0x7f5760040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f5769363640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 0x55ecf87edd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f5769363640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 0x55ecf87edd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49294/0 (socket says 172.21.15.5:49294) 2024-09-19T06:21:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f5769363640 1 -- 172.21.15.5:0/3509324 learned_addr learned my addr 172.21.15.5:0/3509324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f5769b64640 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87dadc0 0x55ecf87eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f5769363640 1 -- 172.21.15.5:0/3509324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87dadc0 msgr2=0x55ecf87eb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f5768b62640 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 0x55ecf87eb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f5769363640 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87dadc0 0x55ecf87eb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f5769363640 1 -- 172.21.15.5:0/3509324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 msgr2=0x55ecf87eb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f5769363640 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 0x55ecf87eb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f5769363640 1 -- 172.21.15.5:0/3509324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecf87ebe10 con 0x55ecf87d6a30 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.440+0000 7f5768b62640 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 0x55ecf87eb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.441+0000 7f5769363640 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 0x55ecf87edd30 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5760003e40 tx=0x7f5760032760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc07d3a0fb9e407 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.441+0000 7f574cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.441+0000 7f574cff9640 1 -- 172.21.15.5:0/3509324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576004d070 con 0x55ecf87d6a30 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.441+0000 7f574cff9640 1 -- 172.21.15.5:0/3509324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f57600037f0 con 0x55ecf87d6a30 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.441+0000 7f576d9a0740 1 -- 172.21.15.5:0/3509324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f575c0bc040 con 0x55ecf87d6a30 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.441+0000 7f574cff9640 1 -- 172.21.15.5:0/3509324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f576006d050 con 0x55ecf87d6a30 2024-09-19T06:21:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.441+0000 7f576d9a0740 10 client.19962 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:11.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.441+0000 7f576d9a0740 1 -- 172.21.15.5:0/3509324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55ecf8860150 con 0x55ecf87d6a30 2024-09-19T06:21:11.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.442+0000 7f574cff9640 1 -- 172.21.15.5:0/3509324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5760032d50 con 0x55ecf87d6a30 2024-09-19T06:21:11.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.442+0000 7f574cff9640 1 -- 172.21.15.5:0/3509324 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f57600049c0 con 0x55ecf87d6a30 2024-09-19T06:21:11.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.442+0000 7f576d9a0740 10 client.19962 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:11.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.442+0000 7f576d9a0740 10 client.19962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:11.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.442+0000 7f576d9a0740 20 client.19962 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:11.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.442+0000 7f576d9a0740 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ecf87e5680 0x55ecf88628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:11.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.442+0000 7f576d9a0740 4 client.19962 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:21:11.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.442+0000 7f576d9a0740 1 -- 172.21.15.5:0/3509324 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f575c0bc040 con 0x55ecf87e5680 2024-09-19T06:21:11.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.442+0000 7f5768b62640 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ecf87e5680 0x55ecf88628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:11.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.443+0000 7f5768b62640 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ecf87e5680 0x55ecf88628e0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c402a9c885aa0701 server_cookie=9e4754bc40fd61f in_seq=0 out_seq=0 2024-09-19T06:21:11.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.443+0000 7f574cff9640 10 client.19962 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.443+0000 7f574cff9640 1 -- 172.21.15.5:0/3509324 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f575c0bc040 con 0x55ecf87e5680 2024-09-19T06:21:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.443+0000 7f574cff9640 10 client.19962 handle_command_reply: tid=0 2024-09-19T06:21:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.443+0000 7f576d9a0740 1 client.19962 shutdown 2024-09-19T06:21:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.443+0000 7f576d9a0740 1 -- 172.21.15.5:0/3509324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 msgr2=0x55ecf87edd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.443+0000 7f576d9a0740 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 0x55ecf87edd30 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5760003e40 tx=0x7f5760032760 comp rx=0 tx=0).stop 2024-09-19T06:21:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.443+0000 7f576d9a0740 1 -- 172.21.15.5:0/3509324 shutdown_connections 2024-09-19T06:21:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.443+0000 7f576d9a0740 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ecf87e5680 0x55ecf88628e0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.444+0000 7f576d9a0740 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf87dadc0 0x55ecf87eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.444+0000 7f576d9a0740 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ecf87da4a0 0x55ecf87eb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.444+0000 7f576d9a0740 1 --2- 172.21.15.5:0/3509324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ecf87d6a30 0x55ecf87edd30 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.444+0000 7f576d9a0740 1 -- 172.21.15.5:0/3509324 >> 172.21.15.5:0/3509324 conn(0x55ecf87e2900 msgr2=0x55ecf87d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.444+0000 7f574cff9640 0 client.19962 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.444+0000 7f576d9a0740 1 -- 172.21.15.5:0/3509324 shutdown_connections 2024-09-19T06:21:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.444+0000 7f576d9a0740 1 -- 172.21.15.5:0/3509324 wait complete. 2024-09-19T06:21:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.444+0000 7f576d9a0740 20 client.19962 trim_cache size 0 max 16384 2024-09-19T06:21:11.452 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:21:11.452 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:21:11.453 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:21:11.453 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:21:11.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.445+0000 7f576b5c7640 1 -- 172.21.15.5:0/1134782221 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57500a34b0 msgr2=0x7f57500a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:11.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.445+0000 7f576b5c7640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57500a34b0 0x7f57500a5970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f576002aa00 tx=0x7f5760005a20 comp rx=0 tx=0).stop 2024-09-19T06:21:11.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.445+0000 7f576b5c7640 1 -- 172.21.15.5:0/1134782221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5764109120 msgr2=0x7f576406d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:11.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.445+0000 7f576b5c7640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5764109120 0x7f576406d040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f575c00eb40 tx=0x7f575c00c2d0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.445+0000 7f576b5c7640 1 -- 172.21.15.5:0/1134782221 shutdown_connections 2024-09-19T06:21:11.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.445+0000 7f576b5c7640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57500a34b0 0x7f57500a5970 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.445+0000 7f576b5c7640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5764109120 0x7f576406d040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.445+0000 7f576b5c7640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57641049a0 0x7f576406cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.445+0000 7f576b5c7640 1 --2- 172.21.15.5:0/1134782221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764103ff0 0x7f5764073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:11.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.445+0000 7f576b5c7640 1 -- 172.21.15.5:0/1134782221 >> 172.21.15.5:0/1134782221 conn(0x7f57640ffa90 msgr2=0x7f5764100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:11.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.445+0000 7f576b5c7640 1 -- 172.21.15.5:0/1134782221 shutdown_connections 2024-09-19T06:21:11.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:11.445+0000 7f576b5c7640 1 -- 172.21.15.5:0/1134782221 wait complete. 2024-09-19T06:21:11.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 45619 8.73 MB/sec execute 155 sec latency 311.218 ms 2024-09-19T06:21:12.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 45756 8.69 MB/sec execute 156 sec latency 72.890 ms 2024-09-19T06:21:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:12 smithi157 ceph-mon[35346]: pgmap v563: 97 pgs: 97 active+clean; 809 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-09-19T06:21:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:12 smithi005 ceph-mon[30955]: pgmap v563: 97 pgs: 97 active+clean; 809 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-09-19T06:21:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:12 smithi049 ceph-mon[34640]: pgmap v563: 97 pgs: 97 active+clean; 809 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-09-19T06:21:13.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 45904 8.66 MB/sec execute 157 sec latency 151.328 ms 2024-09-19T06:21:13.706 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:21:13.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:21:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:21:14.524 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:14 smithi005 ceph-mon[30955]: pgmap v564: 97 pgs: 97 active+clean; 814 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 418 op/s 2024-09-19T06:21:14.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 46084 8.68 MB/sec execute 158 sec latency 94.070 ms 2024-09-19T06:21:14.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.673+0000 7f36471c3640 1 -- 172.21.15.5:0/350364895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3640100e30 msgr2=0x7f3640104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:14.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.673+0000 7f36471c3640 1 --2- 172.21.15.5:0/350364895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3640100e30 0x7f3640104e80 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f362c00da00 tx=0x7f362c02c490 comp rx=0 tx=0).stop 2024-09-19T06:21:14.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.673+0000 7f36471c3640 1 -- 172.21.15.5:0/350364895 shutdown_connections 2024-09-19T06:21:14.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.673+0000 7f36471c3640 1 --2- 172.21.15.5:0/350364895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36401055b0 0x7f364010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:14.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.673+0000 7f36471c3640 1 --2- 172.21.15.5:0/350364895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3640100e30 0x7f3640104e80 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:14.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.673+0000 7f36471c3640 1 --2- 172.21.15.5:0/350364895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3640100480 0x7f3640100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:14.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.673+0000 7f36471c3640 1 -- 172.21.15.5:0/350364895 >> 172.21.15.5:0/350364895 conn(0x7f3640077220 msgr2=0x7f36400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:14.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.673+0000 7f36471c3640 1 -- 172.21.15.5:0/350364895 shutdown_connections 2024-09-19T06:21:14.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.673+0000 7f36471c3640 1 -- 172.21.15.5:0/350364895 wait complete. 2024-09-19T06:21:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.674+0000 7f36471c3640 1 Processor -- start 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.674+0000 7f36471c3640 1 -- start start 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.674+0000 7f36471c3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3640100480 0x7f364019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.674+0000 7f36471c3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3640100e30 0x7f364019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.674+0000 7f36471c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36401055b0 0x7f364019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f3644f5f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3640100480 0x7f364019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f3645760640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36401055b0 0x7f364019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f36471c3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f364010fc60 con 0x7f36401055b0 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f36471c3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f364010fae0 con 0x7f3640100480 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f3644f5f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3640100480 0x7f364019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55112/0 (socket says 172.21.15.5:55112) 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f36471c3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f364010fde0 con 0x7f3640100e30 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f3644f5f640 1 -- 172.21.15.5:0/1179902253 learned_addr learned my addr 172.21.15.5:0/1179902253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f363ffff640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3640100e30 0x7f364019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f363ffff640 1 -- 172.21.15.5:0/1179902253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3640100480 msgr2=0x7f364019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:14.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f363ffff640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3640100480 0x7f364019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:14.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f363ffff640 1 -- 172.21.15.5:0/1179902253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36401055b0 msgr2=0x7f364019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:14.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f363ffff640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36401055b0 0x7f364019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:14.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.675+0000 7f363ffff640 1 -- 172.21.15.5:0/1179902253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364019f550 con 0x7f3640100e30 2024-09-19T06:21:14.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.676+0000 7f363ffff640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3640100e30 0x7f364019aae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f362c031550 tx=0x7f362c033630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:14.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.676+0000 7f363dffb640 1 -- 172.21.15.5:0/1179902253 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c040020 con 0x7f3640100e30 2024-09-19T06:21:14.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.676+0000 7f363dffb640 1 -- 172.21.15.5:0/1179902253 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f362c004460 con 0x7f3640100e30 2024-09-19T06:21:14.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.676+0000 7f363dffb640 1 -- 172.21.15.5:0/1179902253 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c037070 con 0x7f3640100e30 2024-09-19T06:21:14.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.676+0000 7f36471c3640 1 -- 172.21.15.5:0/1179902253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364019f7e0 con 0x7f3640100e30 2024-09-19T06:21:14.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.676+0000 7f36471c3640 1 -- 172.21.15.5:0/1179902253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f364019fc70 con 0x7f3640100e30 2024-09-19T06:21:14.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.678+0000 7f36471c3640 1 -- 172.21.15.5:0/1179902253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3640100860 con 0x7f3640100e30 2024-09-19T06:21:14.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.680+0000 7f363dffb640 1 -- 172.21.15.5:0/1179902253 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f362c043020 con 0x7f3640100e30 2024-09-19T06:21:14.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.681+0000 7f363dffb640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36300a3430 0x7f36300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:14.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.681+0000 7f363dffb640 1 -- 172.21.15.5:0/1179902253 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f362c0f0510 con 0x7f3640100e30 2024-09-19T06:21:14.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.681+0000 7f3644f5f640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36300a3430 0x7f36300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:14.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.681+0000 7f3644f5f640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36300a3430 0x7f36300a58f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3634009c20 tx=0x7f3634005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:14.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:14.686+0000 7f363dffb640 1 -- 172.21.15.5:0/1179902253 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f362c04a030 con 0x7f3640100e30 2024-09-19T06:21:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:14 smithi157 ceph-mon[35346]: pgmap v564: 97 pgs: 97 active+clean; 814 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 418 op/s 2024-09-19T06:21:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:14 smithi049 ceph-mon[34640]: pgmap v564: 97 pgs: 97 active+clean; 814 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 418 op/s 2024-09-19T06:21:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.045+0000 7f36471c3640 1 -- 172.21.15.5:0/1179902253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f364019b900 con 0x7f3640100e30 2024-09-19T06:21:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.046+0000 7f363dffb640 1 -- 172.21.15.5:0/1179902253 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11959 (secure 0 0 0) 0x7f362c0bced0 con 0x7f3640100e30 2024-09-19T06:21:15.047 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:21:15.048 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":58,"btime":"2024-09-19T06:20:33:468419+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:20:32.963303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:21:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 58 2024-09-19T06:21:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.050+0000 7f36471c3640 1 -- 172.21.15.5:0/1179902253 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36300a3430 msgr2=0x7f36300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.050+0000 7f36471c3640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36300a3430 0x7f36300a58f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3634009c20 tx=0x7f3634005f90 comp rx=0 tx=0).stop 2024-09-19T06:21:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.050+0000 7f36471c3640 1 -- 172.21.15.5:0/1179902253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3640100e30 msgr2=0x7f364019aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.050+0000 7f36471c3640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3640100e30 0x7f364019aae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f362c031550 tx=0x7f362c033630 comp rx=0 tx=0).stop 2024-09-19T06:21:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.050+0000 7f36471c3640 1 -- 172.21.15.5:0/1179902253 shutdown_connections 2024-09-19T06:21:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.050+0000 7f36471c3640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36401055b0 0x7f364019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.051+0000 7f36471c3640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f36300a3430 0x7f36300a58f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.051+0000 7f36471c3640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3640100e30 0x7f364019aae0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.051+0000 7f36471c3640 1 --2- 172.21.15.5:0/1179902253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3640100480 0x7f364019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.051+0000 7f36471c3640 1 -- 172.21.15.5:0/1179902253 >> 172.21.15.5:0/1179902253 conn(0x7f3640077220 msgr2=0x7f36401038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.051+0000 7f36471c3640 1 -- 172.21.15.5:0/1179902253 shutdown_connections 2024-09-19T06:21:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:15.051+0000 7f36471c3640 1 -- 172.21.15.5:0/1179902253 wait complete. 2024-09-19T06:21:15.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 46290 8.70 MB/sec execute 159 sec latency 59.565 ms 2024-09-19T06:21:15.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:15 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1179902253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:21:15.712 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:21:15.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:15 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1179902253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:21:15.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:15 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1179902253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:21:16.089 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:21:16.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 46465 8.72 MB/sec execute 160 sec latency 89.650 ms 2024-09-19T06:21:16.576 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:16 smithi005 ceph-mon[30955]: pgmap v565: 97 pgs: 97 active+clean; 814 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 347 op/s 2024-09-19T06:21:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:16 smithi157 ceph-mon[35346]: pgmap v565: 97 pgs: 97 active+clean; 814 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 347 op/s 2024-09-19T06:21:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:16 smithi049 ceph-mon[34640]: pgmap v565: 97 pgs: 97 active+clean; 814 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 347 op/s 2024-09-19T06:21:17.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 46626 8.73 MB/sec execute 161 sec latency 157.210 ms 2024-09-19T06:21:17.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.668+0000 7f6990a63640 1 -- 172.21.15.5:0/2758496818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698c100e20 msgr2=0x7f698c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:17.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.668+0000 7f6990a63640 1 --2- 172.21.15.5:0/2758496818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698c100e20 0x7f698c101200 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f6984009900 tx=0x7f698402a480 comp rx=0 tx=0).stop 2024-09-19T06:21:17.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.668+0000 7f6990a63640 1 -- 172.21.15.5:0/2758496818 shutdown_connections 2024-09-19T06:21:17.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.668+0000 7f6990a63640 1 --2- 172.21.15.5:0/2758496818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f698c102ef0 0x7f698c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.668+0000 7f6990a63640 1 --2- 172.21.15.5:0/2758496818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c102570 0x7f698c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.668+0000 7f6990a63640 1 --2- 172.21.15.5:0/2758496818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698c100e20 0x7f698c101200 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.668+0000 7f6990a63640 1 -- 172.21.15.5:0/2758496818 >> 172.21.15.5:0/2758496818 conn(0x7f698c0fc7d0 msgr2=0x7f698c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:17.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.668+0000 7f6990a63640 1 -- 172.21.15.5:0/2758496818 shutdown_connections 2024-09-19T06:21:17.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.668+0000 7f6990a63640 1 -- 172.21.15.5:0/2758496818 wait complete. 2024-09-19T06:21:17.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f6990a63640 1 Processor -- start 2024-09-19T06:21:17.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f6990a63640 1 -- start start 2024-09-19T06:21:17.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f6990a63640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f698c100e20 0x7f698c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f6990a63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c102570 0x7f698c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f6990a63640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698c102ef0 0x7f698c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f6990a63640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f698c110e80 con 0x7f698c102ef0 2024-09-19T06:21:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f6990a63640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f698c110d00 con 0x7f698c102570 2024-09-19T06:21:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f6990a63640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f698c111000 con 0x7f698c100e20 2024-09-19T06:21:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698c102ef0 0x7f698c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f698a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f698c100e20 0x7f698c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698c102ef0 0x7f698c19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36458/0 (socket says 172.21.15.5:36458) 2024-09-19T06:21:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f698ad9d640 1 -- 172.21.15.5:0/1871746900 learned_addr learned my addr 172.21.15.5:0/1871746900 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.669+0000 7f6989d9b640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c102570 0x7f698c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f698ad9d640 1 -- 172.21.15.5:0/1871746900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f698c100e20 msgr2=0x7f698c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f698ad9d640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f698c100e20 0x7f698c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f698ad9d640 1 -- 172.21.15.5:0/1871746900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c102570 msgr2=0x7f698c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f698ad9d640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c102570 0x7f698c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f698ad9d640 1 -- 172.21.15.5:0/1871746900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c19d340 con 0x7f698c102ef0 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f6989d9b640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c102570 0x7f698c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f698a59c640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f698c100e20 0x7f698c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f698ad9d640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698c102ef0 0x7f698c19cc60 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f697c00eaa0 tx=0x7f697c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f69837fe640 1 -- 172.21.15.5:0/1871746900 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c00cdb0 con 0x7f698c102ef0 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f69837fe640 1 -- 172.21.15.5:0/1871746900 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f697c005020 con 0x7f698c102ef0 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f6990a63640 1 -- 172.21.15.5:0/1871746900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c19d630 con 0x7f698c102ef0 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f69837fe640 1 -- 172.21.15.5:0/1871746900 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c0116d0 con 0x7f698c102ef0 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.670+0000 7f6990a63640 1 -- 172.21.15.5:0/1871746900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c1a4f10 con 0x7f698c102ef0 2024-09-19T06:21:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.673+0000 7f69837fe640 1 -- 172.21.15.5:0/1871746900 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f697c0051d0 con 0x7f698c102ef0 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.673+0000 7f6992e3c740 1 Processor -- start 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.673+0000 7f6992e3c740 1 -- start start 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.673+0000 7f6992e3c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0deb40 0x55dc2c0dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.673+0000 7f6992e3c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.673+0000 7f6992e3c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0e8370 0x55dc2c0ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.673+0000 7f6992e3c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc2c0432f0 con 0x55dc2c0dd5d0 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.673+0000 7f6992e3c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc2bfeacf0 con 0x55dc2c0deb40 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.673+0000 7f6992e3c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dc2c03f2b0 con 0x55dc2c0e8370 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6989d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0deb40 0x55dc2c0dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f698a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6989d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0deb40 0x55dc2c0dd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55134/0 (socket says 172.21.15.5:55134) 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6989d9b640 1 -- 172.21.15.5:0/3988351446 learned_addr learned my addr 172.21.15.5:0/3988351446 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6989d9b640 1 --2- 172.21.15.5:0/3988351446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0e8370 0x55dc2c0ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6989d9b640 1 -- 172.21.15.5:0/3988351446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0e8370 msgr2=0x55dc2c0ea760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6989d9b640 1 --2- 172.21.15.5:0/3988351446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0e8370 0x55dc2c0ea760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6989d9b640 1 -- 172.21.15.5:0/3988351446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0dd5d0 msgr2=0x55dc2c0dda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6989d9b640 1 --2- 172.21.15.5:0/3988351446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0dda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6989d9b640 1 -- 172.21.15.5:0/3988351446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc2bfcf3a0 con 0x55dc2c0deb40 2024-09-19T06:21:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f69837fe640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69780a33b0 0x7f69780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6989d9b640 1 --2- 172.21.15.5:0/3988351446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0deb40 0x55dc2c0dd090 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f697400bfd0 tx=0x7f697400b500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5451849416a6f78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f698a59c640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69780a33b0 0x7f69780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f69837fe640 1 -- 172.21.15.5:0/1871746900 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f697c0cff10 con 0x7f698c102ef0 2024-09-19T06:21:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6980ff9640 1 -- 172.21.15.5:0/3988351446 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974004030 con 0x55dc2c0deb40 2024-09-19T06:21:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6980ff9640 1 -- 172.21.15.5:0/3988351446 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f697400e960 con 0x55dc2c0deb40 2024-09-19T06:21:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.674+0000 7f6980ff9640 1 -- 172.21.15.5:0/3988351446 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974003850 con 0x55dc2c0deb40 2024-09-19T06:21:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.675+0000 7f6992e3c740 1 -- 172.21.15.5:0/3988351446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0deb40 msgr2=0x55dc2c0dd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.675+0000 7f6992e3c740 1 --2- 172.21.15.5:0/3988351446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0deb40 0x55dc2c0dd090 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f697400bfd0 tx=0x7f697400b500 comp rx=0 tx=0).stop 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.675+0000 7f698a59c640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69780a33b0 0x7f69780a5870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6984000c00 tx=0x7f6984005e60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.675+0000 7f6992e3c740 1 -- 172.21.15.5:0/3988351446 shutdown_connections 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.675+0000 7f6992e3c740 1 --2- 172.21.15.5:0/3988351446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0e8370 0x55dc2c0ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.675+0000 7f6992e3c740 1 --2- 172.21.15.5:0/3988351446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.675+0000 7f6992e3c740 1 --2- 172.21.15.5:0/3988351446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0deb40 0x55dc2c0dd090 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.675+0000 7f6992e3c740 1 -- 172.21.15.5:0/3988351446 >> 172.21.15.5:0/3988351446 conn(0x55dc2c0a6310 msgr2=0x55dc2c0d9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.675+0000 7f6992e3c740 1 -- 172.21.15.5:0/3988351446 shutdown_connections 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.675+0000 7f6992e3c740 1 -- 172.21.15.5:0/3988351446 wait complete. 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f6992e3c740 1 -- start start 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f6992e3c740 1 Processor -- start 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f6992e3c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f6992e3c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0deb40 0x55dc2c0e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f6992e3c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0e8370 0x55dc2c0e5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f6992e3c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc2c0eb070 con 0x55dc2c0e8370 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f6992e3c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc2bffe060 con 0x55dc2c0dd5d0 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f6992e3c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dc2c0eb1f0 con 0x55dc2c0deb40 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f6992e3c740 10 client.0 _ll_register_callbacks cb 0x55dc2bd518c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0e8370 0x55dc2c0e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f698a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0e8370 0x55dc2c0e5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36476/0 (socket says 172.21.15.5:36476) 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f698ad9d640 1 -- 172.21.15.5:0/2774917194 learned_addr learned my addr 172.21.15.5:0/2774917194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.676+0000 7f6989d9b640 1 --2- 172.21.15.5:0/2774917194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0deb40 0x55dc2c0e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f698ad9d640 1 -- 172.21.15.5:0/2774917194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0deb40 msgr2=0x55dc2c0e4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f698ad9d640 1 --2- 172.21.15.5:0/2774917194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0deb40 0x55dc2c0e4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f698ad9d640 1 -- 172.21.15.5:0/2774917194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 msgr2=0x55dc2c0e7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f698ad9d640 1 --2- 172.21.15.5:0/2774917194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0e7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f698ad9d640 1 -- 172.21.15.5:0/2774917194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc2bdb43c0 con 0x55dc2c0e8370 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f698ad9d640 1 --2- 172.21.15.5:0/2774917194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0e8370 0x55dc2c0e5570 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f697c098630 tx=0x7f697c09c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64e3806b94d1fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f6971ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f6971ffb640 1 -- 172.21.15.5:0/2774917194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c01f030 con 0x55dc2c0e8370 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f6971ffb640 1 -- 172.21.15.5:0/2774917194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f697c0a0120 con 0x55dc2c0e8370 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f6992e3c740 1 -- 172.21.15.5:0/2774917194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f697400e960 con 0x55dc2c0e8370 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f6971ffb640 1 -- 172.21.15.5:0/2774917194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f697c0a4e40 con 0x55dc2c0e8370 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f6992e3c740 10 client.19980 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f6992e3c740 1 -- 172.21.15.5:0/2774917194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55dc2c0f6ac0 con 0x55dc2c0e8370 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.677+0000 7f6971ffb640 1 -- 172.21.15.5:0/2774917194 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f697400e960 con 0x55dc2c0e8370 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.678+0000 7f6971ffb640 1 -- 172.21.15.5:0/2774917194 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f697c00f040 con 0x55dc2c0e8370 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.678+0000 7f6992e3c740 10 client.19980 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.678+0000 7f6992e3c740 10 client.19980 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:21:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.678+0000 7f6992e3c740 20 client.19980 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.678+0000 7f6992e3c740 1 --2- 172.21.15.5:0/2774917194 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55dc2c0f7fd0 0x55dc2c0fa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.678+0000 7f6992e3c740 4 client.19980 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:21:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.678+0000 7f6992e3c740 1 -- 172.21.15.5:0/2774917194 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc2c050860 con 0x55dc2c0f7fd0 2024-09-19T06:21:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.678+0000 7f698a59c640 1 --2- 172.21.15.5:0/2774917194 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55dc2c0f7fd0 0x55dc2c0fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.678+0000 7f698a59c640 1 --2- 172.21.15.5:0/2774917194 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55dc2c0f7fd0 0x55dc2c0fa410 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d08b9b1db8e3013 server_cookie=7e2df825ff46f6a8 in_seq=0 out_seq=0 2024-09-19T06:21:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.678+0000 7f6971ffb640 10 client.19980 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:21:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.679+0000 7f6971ffb640 1 -- 172.21.15.5:0/2774917194 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc2c050860 con 0x55dc2c0f7fd0 2024-09-19T06:21:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.679+0000 7f6971ffb640 10 client.19980 handle_command_reply: tid=0 2024-09-19T06:21:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.679+0000 7f6992e3c740 1 client.19980 shutdown 2024-09-19T06:21:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.679+0000 7f6992e3c740 1 -- 172.21.15.5:0/2774917194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0e8370 msgr2=0x55dc2c0e5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.679+0000 7f6992e3c740 1 --2- 172.21.15.5:0/2774917194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0e8370 0x55dc2c0e5570 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f697c098630 tx=0x7f697c09c940 comp rx=0 tx=0).stop 2024-09-19T06:21:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.679+0000 7f6992e3c740 1 -- 172.21.15.5:0/2774917194 shutdown_connections 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.679+0000 7f6992e3c740 1 --2- 172.21.15.5:0/2774917194 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55dc2c0f7fd0 0x55dc2c0fa410 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.679+0000 7f6992e3c740 1 --2- 172.21.15.5:0/2774917194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0e8370 0x55dc2c0e5570 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.679+0000 7f6992e3c740 1 --2- 172.21.15.5:0/2774917194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0deb40 0x55dc2c0e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.679+0000 7f6971ffb640 0 client.19980 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.679+0000 7f6992e3c740 1 --2- 172.21.15.5:0/2774917194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0e7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.679+0000 7f6992e3c740 1 -- 172.21.15.5:0/2774917194 >> 172.21.15.5:0/2774917194 conn(0x55dc2c0a6310 msgr2=0x55dc2c0df960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.680+0000 7f6992e3c740 1 -- 172.21.15.5:0/2774917194 shutdown_connections 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.680+0000 7f6992e3c740 1 -- 172.21.15.5:0/2774917194 wait complete. 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.680+0000 7f6992e3c740 20 client.19980 trim_cache size 0 max 16384 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f6992e3c740 1 Processor -- start 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f6992e3c740 1 -- start start 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f6992e3c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f6992e3c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0deb40 0x55dc2c0ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f6992e3c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0e8370 0x55dc2c100220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f6992e3c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1ad830 con 0x55dc2c0deb40 2024-09-19T06:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f6992e3c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1af370 con 0x55dc2c0dd5d0 2024-09-19T06:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f6992e3c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1adf00 con 0x55dc2c0e8370 2024-09-19T06:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f6989d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0deb40 0x55dc2c0ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0e8370 0x55dc2c100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0e8370 0x55dc2c100220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47028/0 (socket says 172.21.15.5:47028) 2024-09-19T06:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f6989d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0deb40 0x55dc2c0ffce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36478/0 (socket says 172.21.15.5:36478) 2024-09-19T06:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f698ad9d640 1 -- 172.21.15.5:0/4087253222 learned_addr learned my addr 172.21.15.5:0/4087253222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.693+0000 7f698a59c640 1 --2- 172.21.15.5:0/4087253222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.694+0000 7f698ad9d640 1 -- 172.21.15.5:0/4087253222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 msgr2=0x55dc2c0f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.694+0000 7f698ad9d640 1 --2- 172.21.15.5:0/4087253222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0f4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.694+0000 7f698ad9d640 1 -- 172.21.15.5:0/4087253222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0deb40 msgr2=0x55dc2c0ffce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.694+0000 7f698ad9d640 1 --2- 172.21.15.5:0/4087253222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0deb40 0x55dc2c0ffce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.694+0000 7f698ad9d640 1 -- 172.21.15.5:0/4087253222 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc2c0e0ae0 con 0x55dc2c0e8370 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.694+0000 7f698ad9d640 1 --2- 172.21.15.5:0/4087253222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0e8370 0x55dc2c100220 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f697c0cd6f0 tx=0x7f697c0a0700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a3a6f39bbcf93cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.694+0000 7f6973fff640 1 -- 172.21.15.5:0/4087253222 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c01f030 con 0x55dc2c0e8370 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.694+0000 7f6973fff640 1 -- 172.21.15.5:0/4087253222 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f697c0bc0c0 con 0x55dc2c0e8370 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.694+0000 7f6973fff640 1 -- 172.21.15.5:0/4087253222 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c09b090 con 0x55dc2c0e8370 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.695+0000 7f6992e3c740 1 -- 172.21.15.5:0/4087253222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0e8370 msgr2=0x55dc2c100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.695+0000 7f6992e3c740 1 --2- 172.21.15.5:0/4087253222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0e8370 0x55dc2c100220 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f697c0cd6f0 tx=0x7f697c0a0700 comp rx=0 tx=0).stop 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.695+0000 7f6992e3c740 1 -- 172.21.15.5:0/4087253222 shutdown_connections 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.695+0000 7f6992e3c740 1 --2- 172.21.15.5:0/4087253222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0e8370 0x55dc2c100220 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.695+0000 7f6992e3c740 1 --2- 172.21.15.5:0/4087253222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0deb40 0x55dc2c0ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.695+0000 7f6992e3c740 1 --2- 172.21.15.5:0/4087253222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.695+0000 7f6992e3c740 1 -- 172.21.15.5:0/4087253222 >> 172.21.15.5:0/4087253222 conn(0x55dc2c0a6310 msgr2=0x55dc2c0e4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.695+0000 7f6992e3c740 1 -- 172.21.15.5:0/4087253222 shutdown_connections 2024-09-19T06:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.695+0000 7f6992e3c740 1 -- 172.21.15.5:0/4087253222 wait complete. 2024-09-19T06:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f6992e3c740 1 -- start start 2024-09-19T06:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f6992e3c740 1 Processor -- start 2024-09-19T06:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f6992e3c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f6992e3c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0deb40 0x55dc2c0e1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f6992e3c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0e8370 0x55dc2c0e2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f6992e3c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1aeca0 con 0x55dc2c0e8370 2024-09-19T06:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f6992e3c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1ae5d0 con 0x55dc2c0dd5d0 2024-09-19T06:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f6992e3c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1ab830 con 0x55dc2c0deb40 2024-09-19T06:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f6992e3c740 10 client.0 _ll_register_callbacks cb 0x7f697c005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f6989d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0deb40 0x55dc2c0e1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0e8370 0x55dc2c0e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f6989d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0deb40 0x55dc2c0e1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47036/0 (socket says 172.21.15.5:47036) 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.696+0000 7f6989d9b640 1 -- 172.21.15.5:0/3462414771 learned_addr learned my addr 172.21.15.5:0/3462414771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.697+0000 7f6989d9b640 1 -- 172.21.15.5:0/3462414771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 msgr2=0x55dc2c0f0c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.697+0000 7f6989d9b640 1 --2- 172.21.15.5:0/3462414771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0f0c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.697+0000 7f6989d9b640 1 -- 172.21.15.5:0/3462414771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0e8370 msgr2=0x55dc2c0e2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.697+0000 7f6989d9b640 1 --2- 172.21.15.5:0/3462414771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0e8370 0x55dc2c0e2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.697+0000 7f6989d9b640 1 -- 172.21.15.5:0/3462414771 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc2c0e2ba0 con 0x55dc2c0deb40 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.697+0000 7f6989d9b640 1 --2- 172.21.15.5:0/3462414771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0deb40 0x55dc2c0e1dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f697400b4d0 tx=0x7f697400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=603c837cc8c86272 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.697+0000 7f6971ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.697+0000 7f6971ffb640 1 -- 172.21.15.5:0/3462414771 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697400e960 con 0x55dc2c0deb40 2024-09-19T06:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.697+0000 7f6971ffb640 1 -- 172.21.15.5:0/3462414771 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f697400eb00 con 0x55dc2c0deb40 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.697+0000 7f6992e3c740 1 -- 172.21.15.5:0/3462414771 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f697c0bc0c0 con 0x55dc2c0deb40 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.697+0000 7f6971ffb640 1 -- 172.21.15.5:0/3462414771 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f697400ed00 con 0x55dc2c0deb40 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.697+0000 7f6992e3c740 10 client.29293 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.698+0000 7f6992e3c740 1 -- 172.21.15.5:0/3462414771 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55dc2c162d80 con 0x55dc2c0deb40 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.698+0000 7f6971ffb640 1 -- 172.21.15.5:0/3462414771 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f697c0bc0c0 con 0x55dc2c0deb40 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.698+0000 7f6971ffb640 1 -- 172.21.15.5:0/3462414771 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f69740218b0 con 0x55dc2c0deb40 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.698+0000 7f6992e3c740 10 client.29293 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.698+0000 7f6992e3c740 10 client.29293 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.698+0000 7f6992e3c740 20 client.29293 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.698+0000 7f6992e3c740 1 --2- 172.21.15.5:0/3462414771 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55dc2c1630c0 0x55dc2c1654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.698+0000 7f6992e3c740 4 client.29293 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.698+0000 7f6992e3c740 1 -- 172.21.15.5:0/3462414771 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc2c04ad70 con 0x55dc2c1630c0 2024-09-19T06:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.698+0000 7f698a59c640 1 --2- 172.21.15.5:0/3462414771 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55dc2c1630c0 0x55dc2c1654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f698a59c640 1 --2- 172.21.15.5:0/3462414771 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55dc2c1630c0 0x55dc2c1654b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=90962dc8a4d9354f server_cookie=4829fb6757c9b164 in_seq=0 out_seq=0 2024-09-19T06:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6971ffb640 10 client.29293 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6971ffb640 1 -- 172.21.15.5:0/3462414771 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc2c04ad70 con 0x55dc2c1630c0 2024-09-19T06:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6971ffb640 10 client.29293 handle_command_reply: tid=0 2024-09-19T06:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6992e3c740 1 client.29293 shutdown 2024-09-19T06:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6992e3c740 1 -- 172.21.15.5:0/3462414771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0deb40 msgr2=0x55dc2c0e1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6992e3c740 1 --2- 172.21.15.5:0/3462414771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0deb40 0x55dc2c0e1dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f697400b4d0 tx=0x7f697400bab0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6992e3c740 1 -- 172.21.15.5:0/3462414771 shutdown_connections 2024-09-19T06:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6992e3c740 1 --2- 172.21.15.5:0/3462414771 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55dc2c1630c0 0x55dc2c1654b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6992e3c740 1 --2- 172.21.15.5:0/3462414771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc2c0e8370 0x55dc2c0e2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6971ffb640 0 client.29293 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6992e3c740 1 --2- 172.21.15.5:0/3462414771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2c0deb40 0x55dc2c0e1dd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.808 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:21:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6992e3c740 1 --2- 172.21.15.5:0/3462414771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc2c0dd5d0 0x55dc2c0f0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6992e3c740 1 -- 172.21.15.5:0/3462414771 >> 172.21.15.5:0/3462414771 conn(0x55dc2c0a6310 msgr2=0x55dc2c0e38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6992e3c740 1 -- 172.21.15.5:0/3462414771 shutdown_connections 2024-09-19T06:21:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6992e3c740 1 -- 172.21.15.5:0/3462414771 wait complete. 2024-09-19T06:21:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.699+0000 7f6992e3c740 20 client.29293 trim_cache size 0 max 16384 2024-09-19T06:21:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.700+0000 7f6990a63640 1 -- 172.21.15.5:0/1871746900 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69780a33b0 msgr2=0x7f69780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.700+0000 7f6990a63640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69780a33b0 0x7f69780a5870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6984000c00 tx=0x7f6984005e60 comp rx=0 tx=0).stop 2024-09-19T06:21:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.700+0000 7f6990a63640 1 -- 172.21.15.5:0/1871746900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698c102ef0 msgr2=0x7f698c19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.700+0000 7f6990a63640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698c102ef0 0x7f698c19cc60 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f697c00eaa0 tx=0x7f697c00ef60 comp rx=0 tx=0).stop 2024-09-19T06:21:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.700+0000 7f6990a63640 1 -- 172.21.15.5:0/1871746900 shutdown_connections 2024-09-19T06:21:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.700+0000 7f6990a63640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698c102ef0 0x7f698c19cc60 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.700+0000 7f6990a63640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c102570 0x7f698c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.700+0000 7f6990a63640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f69780a33b0 0x7f69780a5870 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.700+0000 7f6990a63640 1 --2- 172.21.15.5:0/1871746900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f698c100e20 0x7f698c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.700+0000 7f6990a63640 1 -- 172.21.15.5:0/1871746900 >> 172.21.15.5:0/1871746900 conn(0x7f698c0fc7d0 msgr2=0x7f698c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.700+0000 7f6990a63640 1 -- 172.21.15.5:0/1871746900 shutdown_connections 2024-09-19T06:21:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:16.700+0000 7f6990a63640 1 -- 172.21.15.5:0/1871746900 wait complete. 2024-09-19T06:21:18.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 46795 8.78 MB/sec execute 162 sec latency 105.135 ms 2024-09-19T06:21:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:18 smithi157 ceph-mon[35346]: pgmap v566: 97 pgs: 97 active+clean; 828 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 320 op/s 2024-09-19T06:21:18.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:18 smithi005 ceph-mon[30955]: pgmap v566: 97 pgs: 97 active+clean; 828 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 320 op/s 2024-09-19T06:21:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:18 smithi049 ceph-mon[34640]: pgmap v566: 97 pgs: 97 active+clean; 828 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 320 op/s 2024-09-19T06:21:19.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 46989 8.82 MB/sec execute 163 sec latency 95.473 ms 2024-09-19T06:21:20.194 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:21:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.411+0000 7ff751dc4640 1 -- 172.21.15.5:0/2998992939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff74c101340 msgr2=0x7ff74c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.411+0000 7ff751dc4640 1 --2- 172.21.15.5:0/2998992939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff74c101340 0x7ff74c10d870 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7ff744009cd0 tx=0x7ff74402a8f0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.411+0000 7ff751dc4640 1 -- 172.21.15.5:0/2998992939 shutdown_connections 2024-09-19T06:21:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.411+0000 7ff751dc4640 1 --2- 172.21.15.5:0/2998992939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff74c101340 0x7ff74c10d870 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.411+0000 7ff751dc4640 1 --2- 172.21.15.5:0/2998992939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff74c1009a0 0x7ff74c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.411+0000 7ff751dc4640 1 --2- 172.21.15.5:0/2998992939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c1069b0 0x7ff74c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.411+0000 7ff751dc4640 1 -- 172.21.15.5:0/2998992939 >> 172.21.15.5:0/2998992939 conn(0x7ff74c0779a0 msgr2=0x7ff74c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.411+0000 7ff751dc4640 1 -- 172.21.15.5:0/2998992939 shutdown_connections 2024-09-19T06:21:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.411+0000 7ff751dc4640 1 -- 172.21.15.5:0/2998992939 wait complete. 2024-09-19T06:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.412+0000 7ff751dc4640 1 Processor -- start 2024-09-19T06:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.412+0000 7ff751dc4640 1 -- start start 2024-09-19T06:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.412+0000 7ff751dc4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff74c1009a0 0x7ff74c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.412+0000 7ff751dc4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c101340 0x7ff74c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.412+0000 7ff751dc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff74c1069b0 0x7ff74c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.412+0000 7ff751dc4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c110cf0 con 0x7ff74c1069b0 2024-09-19T06:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.412+0000 7ff751dc4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c110b70 con 0x7ff74c1009a0 2024-09-19T06:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.412+0000 7ff751dc4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c110e70 con 0x7ff74c101340 2024-09-19T06:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.413+0000 7ff74b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff74c1009a0 0x7ff74c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.413+0000 7ff74b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff74c1009a0 0x7ff74c1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55176/0 (socket says 172.21.15.5:55176) 2024-09-19T06:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.413+0000 7ff74b7fe640 1 -- 172.21.15.5:0/3006676181 learned_addr learned my addr 172.21.15.5:0/3006676181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.413+0000 7ff74bfff640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff74c1069b0 0x7ff74c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.413+0000 7ff74affd640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c101340 0x7ff74c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.413+0000 7ff74b7fe640 1 -- 172.21.15.5:0/3006676181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c101340 msgr2=0x7ff74c1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.413+0000 7ff74b7fe640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c101340 0x7ff74c1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.413+0000 7ff74b7fe640 1 -- 172.21.15.5:0/3006676181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff74c1069b0 msgr2=0x7ff74c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.413+0000 7ff74b7fe640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff74c1069b0 0x7ff74c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.413+0000 7ff74b7fe640 1 -- 172.21.15.5:0/3006676181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c19d2a0 con 0x7ff74c1009a0 2024-09-19T06:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.413+0000 7ff74b7fe640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff74c1009a0 0x7ff74c1982b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00e9f0 tx=0x7ff73c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.414+0000 7ff748ff9640 1 -- 172.21.15.5:0/3006676181 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00cd80 con 0x7ff74c1009a0 2024-09-19T06:21:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.414+0000 7ff748ff9640 1 -- 172.21.15.5:0/3006676181 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff73c0054d0 con 0x7ff74c1009a0 2024-09-19T06:21:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.414+0000 7ff748ff9640 1 -- 172.21.15.5:0/3006676181 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c010490 con 0x7ff74c1009a0 2024-09-19T06:21:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.414+0000 7ff751dc4640 1 -- 172.21.15.5:0/3006676181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74c19d590 con 0x7ff74c1009a0 2024-09-19T06:21:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.414+0000 7ff751dc4640 1 -- 172.21.15.5:0/3006676181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74c1a4e20 con 0x7ff74c1009a0 2024-09-19T06:21:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.417+0000 7ff754c3e740 1 Processor -- start 2024-09-19T06:21:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.418+0000 7ff754c3e740 1 -- start start 2024-09-19T06:21:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.418+0000 7ff748ff9640 1 -- 172.21.15.5:0/3006676181 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff73c005000 con 0x7ff74c1009a0 2024-09-19T06:21:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.418+0000 7ff754c3e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd7970 0x560bf2bd7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.418+0000 7ff754c3e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd8290 0x560bf2be6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.418+0000 7ff754c3e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2be70c0 0x560bf2be94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.418+0000 7ff754c3e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560bf2b3c2f0 con 0x560bf2be70c0 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.418+0000 7ff754c3e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560bf2ae4270 con 0x560bf2bd7970 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.418+0000 7ff754c3e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560bf2b382b0 con 0x560bf2bd8290 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.419+0000 7ff74bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd7970 0x560bf2bd7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.419+0000 7ff74bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd7970 0x560bf2bd7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55182/0 (socket says 172.21.15.5:55182) 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.419+0000 7ff74bfff640 1 -- 172.21.15.5:0/926453382 learned_addr learned my addr 172.21.15.5:0/926453382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.419+0000 7ff74affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2be70c0 0x560bf2be94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.419+0000 7ff74b7fe640 1 --2- 172.21.15.5:0/926453382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd8290 0x560bf2be6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.419+0000 7ff74bfff640 1 -- 172.21.15.5:0/926453382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd8290 msgr2=0x560bf2be6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.419+0000 7ff74bfff640 1 --2- 172.21.15.5:0/926453382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd8290 0x560bf2be6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.419+0000 7ff74bfff640 1 -- 172.21.15.5:0/926453382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2be70c0 msgr2=0x560bf2be94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.419+0000 7ff74bfff640 1 --2- 172.21.15.5:0/926453382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2be70c0 0x560bf2be94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.419+0000 7ff74bfff640 1 -- 172.21.15.5:0/926453382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bf2ae3340 con 0x560bf2bd7970 2024-09-19T06:21:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.419+0000 7ff74b7fe640 1 --2- 172.21.15.5:0/926453382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd8290 0x560bf2be6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:21:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.419+0000 7ff74bfff640 1 --2- 172.21.15.5:0/926453382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd7970 0x560bf2bd7d50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff744009cd0 tx=0x7ff7440093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=274ac57d1eacb8bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.420+0000 7ff731ffb640 1 -- 172.21.15.5:0/926453382 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744030660 con 0x560bf2bd7970 2024-09-19T06:21:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.420+0000 7ff731ffb640 1 -- 172.21.15.5:0/926453382 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff744036030 con 0x560bf2bd7970 2024-09-19T06:21:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.420+0000 7ff748ff9640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7340a33b0 0x7ff7340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.420+0000 7ff74b7fe640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7340a33b0 0x7ff7340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.420+0000 7ff748ff9640 1 -- 172.21.15.5:0/3006676181 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7ff73c0d00a0 con 0x7ff74c1009a0 2024-09-19T06:21:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.420+0000 7ff731ffb640 1 -- 172.21.15.5:0/926453382 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7440043c0 con 0x560bf2bd7970 2024-09-19T06:21:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.421+0000 7ff754c3e740 1 -- 172.21.15.5:0/926453382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd7970 msgr2=0x560bf2bd7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.421+0000 7ff74b7fe640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7340a33b0 0x7ff7340a5870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0cd880 tx=0x7ff73c097ee0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.421+0000 7ff754c3e740 1 --2- 172.21.15.5:0/926453382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd7970 0x560bf2bd7d50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff744009cd0 tx=0x7ff7440093e0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.421+0000 7ff754c3e740 1 -- 172.21.15.5:0/926453382 shutdown_connections 2024-09-19T06:21:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.421+0000 7ff754c3e740 1 --2- 172.21.15.5:0/926453382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2be70c0 0x560bf2be94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.421+0000 7ff754c3e740 1 --2- 172.21.15.5:0/926453382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd8290 0x560bf2be6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.421+0000 7ff754c3e740 1 --2- 172.21.15.5:0/926453382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd7970 0x560bf2bd7d50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.421+0000 7ff754c3e740 1 -- 172.21.15.5:0/926453382 >> 172.21.15.5:0/926453382 conn(0x560bf2bd4060 msgr2=0x560bf2bd4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.421+0000 7ff754c3e740 1 -- 172.21.15.5:0/926453382 shutdown_connections 2024-09-19T06:21:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.421+0000 7ff754c3e740 1 -- 172.21.15.5:0/926453382 wait complete. 2024-09-19T06:21:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.421+0000 7ff754c3e740 1 -- start start 2024-09-19T06:21:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.422+0000 7ff754c3e740 1 Processor -- start 2024-09-19T06:21:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.422+0000 7ff754c3e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd7970 0x560bf2be3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.422+0000 7ff754c3e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2bd8290 0x560bf2be4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.422+0000 7ff74bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd7970 0x560bf2be3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff74bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd7970 0x560bf2be3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47058/0 (socket says 172.21.15.5:47058) 2024-09-19T06:21:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff74bfff640 1 -- 172.21.15.5:0/1489446467 learned_addr learned my addr 172.21.15.5:0/1489446467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff754c3e740 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2be70c0 0x560bf2be4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff754c3e740 1 -- 172.21.15.5:0/1489446467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560bf2be9ce0 con 0x560bf2bd8290 2024-09-19T06:21:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff74affd640 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2bd8290 0x560bf2be4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff754c3e740 1 -- 172.21.15.5:0/1489446467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560bf2ac5290 con 0x560bf2be70c0 2024-09-19T06:21:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff754c3e740 1 -- 172.21.15.5:0/1489446467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560bf2be9e60 con 0x560bf2bd7970 2024-09-19T06:21:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff74affd640 1 -- 172.21.15.5:0/1489446467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd7970 msgr2=0x560bf2be3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff74affd640 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd7970 0x560bf2be3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff74affd640 1 -- 172.21.15.5:0/1489446467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2be70c0 msgr2=0x560bf2be4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff74bfff640 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2be70c0 0x560bf2be4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff754c3e740 10 client.0 _ll_register_callbacks cb 0x560bf2af6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff74affd640 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2be70c0 0x560bf2be4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff74affd640 1 -- 172.21.15.5:0/1489446467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bf284f210 con 0x560bf2bd8290 2024-09-19T06:21:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff74bfff640 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2be70c0 0x560bf2be4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:21:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.423+0000 7ff754c3e740 1 -- 172.21.15.5:0/1489446467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff744036030 con 0x560bf2bd8290 2024-09-19T06:21:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.424+0000 7ff74affd640 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2bd8290 0x560bf2be4500 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0ffef0 tx=0x7ff740012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55172c9174bd6df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.424+0000 7ff72affd640 10 client.19998 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:21:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.424+0000 7ff72affd640 1 -- 172.21.15.5:0/1489446467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740004320 con 0x560bf2bd8290 2024-09-19T06:21:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.424+0000 7ff72affd640 1 -- 172.21.15.5:0/1489446467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7ff7400119a0 con 0x560bf2bd8290 2024-09-19T06:21:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.424+0000 7ff72affd640 1 -- 172.21.15.5:0/1489446467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff740011be0 con 0x560bf2bd8290 2024-09-19T06:21:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.424+0000 7ff754c3e740 10 client.19998 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.424+0000 7ff754c3e740 1 -- 172.21.15.5:0/1489446467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=58}) -- 0x560bf291b620 con 0x560bf2bd8290 2024-09-19T06:21:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.425+0000 7ff72affd640 1 -- 172.21.15.5:0/1489446467 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7ff7400035f0 con 0x560bf2bd8290 2024-09-19T06:21:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.425+0000 7ff754c3e740 10 client.19998 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.425+0000 7ff754c3e740 10 client.19998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.425+0000 7ff754c3e740 20 client.19998 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.425+0000 7ff754c3e740 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560bf2bf6f30 0x560bf2bf9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.425+0000 7ff754c3e740 4 client.19998 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:21:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.425+0000 7ff754c3e740 1 -- 172.21.15.5:0/1489446467 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bf2b49a20 con 0x560bf2bf6f30 2024-09-19T06:21:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.425+0000 7ff74bfff640 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560bf2bf6f30 0x560bf2bf9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.426+0000 7ff74bfff640 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560bf2bf6f30 0x560bf2bf9320 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10260c4d234b4779 server_cookie=8244075bf77085c2 in_seq=0 out_seq=0 2024-09-19T06:21:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.426+0000 7ff72affd640 10 client.19998 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.426+0000 7ff72affd640 1 -- 172.21.15.5:0/1489446467 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bf2b49a20 con 0x560bf2bf6f30 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.426+0000 7ff72affd640 10 client.19998 handle_command_reply: tid=0 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.426+0000 7ff754c3e740 1 client.19998 shutdown 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.427+0000 7ff754c3e740 1 -- 172.21.15.5:0/1489446467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2bd8290 msgr2=0x560bf2be4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.427+0000 7ff754c3e740 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2bd8290 0x560bf2be4500 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0ffef0 tx=0x7ff740012b60 comp rx=0 tx=0).stop 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.427+0000 7ff754c3e740 1 -- 172.21.15.5:0/1489446467 shutdown_connections 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.427+0000 7ff754c3e740 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560bf2bf6f30 0x560bf2bf9320 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.427+0000 7ff754c3e740 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2be70c0 0x560bf2be4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.427+0000 7ff72affd640 0 client.19998 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.427+0000 7ff754c3e740 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2bd8290 0x560bf2be4500 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.427+0000 7ff754c3e740 1 --2- 172.21.15.5:0/1489446467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd7970 0x560bf2be3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.427+0000 7ff754c3e740 1 -- 172.21.15.5:0/1489446467 >> 172.21.15.5:0/1489446467 conn(0x560bf2bd4060 msgr2=0x560bf2bd9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.427+0000 7ff754c3e740 1 -- 172.21.15.5:0/1489446467 shutdown_connections 2024-09-19T06:21:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.427+0000 7ff754c3e740 1 -- 172.21.15.5:0/1489446467 wait complete. 2024-09-19T06:21:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.427+0000 7ff754c3e740 20 client.19998 trim_cache size 0 max 16384 2024-09-19T06:21:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.453+0000 7ff754c3e740 1 Processor -- start 2024-09-19T06:21:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.453+0000 7ff754c3e740 1 -- start start 2024-09-19T06:21:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff754c3e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd7970 0x560bf2be62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff754c3e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 0x560bf2be4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff754c3e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2be70c0 0x560bf2bfeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff754c3e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1ab550 con 0x560bf2be70c0 2024-09-19T06:21:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff754c3e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1ad090 con 0x560bf2bd8290 2024-09-19T06:21:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff754c3e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1abc20 con 0x560bf2bd7970 2024-09-19T06:21:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff74bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd7970 0x560bf2be62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff74affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 0x560bf2be4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff74bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd7970 0x560bf2be62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47070/0 (socket says 172.21.15.5:47070) 2024-09-19T06:21:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff74bfff640 1 -- 172.21.15.5:0/2696614747 learned_addr learned my addr 172.21.15.5:0/2696614747 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff74bfff640 1 --2- 172.21.15.5:0/2696614747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2be70c0 0x560bf2bfeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff74affd640 1 -- 172.21.15.5:0/2696614747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd7970 msgr2=0x560bf2be62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff74affd640 1 --2- 172.21.15.5:0/2696614747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd7970 0x560bf2be62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff74affd640 1 -- 172.21.15.5:0/2696614747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2be70c0 msgr2=0x560bf2bfeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff74affd640 1 --2- 172.21.15.5:0/2696614747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2be70c0 0x560bf2bfeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.454+0000 7ff74affd640 1 -- 172.21.15.5:0/2696614747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bf2bd80c0 con 0x560bf2bd8290 2024-09-19T06:21:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.455+0000 7ff74affd640 1 --2- 172.21.15.5:0/2696614747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 0x560bf2be4870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x560bf2be5570 tx=0x7ff74000e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bff37e8b1626c8e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.455+0000 7ff7317fa640 1 -- 172.21.15.5:0/2696614747 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7400119a0 con 0x560bf2bd8290 2024-09-19T06:21:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.455+0000 7ff7317fa640 1 -- 172.21.15.5:0/2696614747 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff744036030 con 0x560bf2bd8290 2024-09-19T06:21:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.455+0000 7ff7317fa640 1 -- 172.21.15.5:0/2696614747 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740003740 con 0x560bf2bd8290 2024-09-19T06:21:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.456+0000 7ff754c3e740 1 -- 172.21.15.5:0/2696614747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 msgr2=0x560bf2be4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.456+0000 7ff754c3e740 1 --2- 172.21.15.5:0/2696614747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 0x560bf2be4870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x560bf2be5570 tx=0x7ff74000e000 comp rx=0 tx=0).stop 2024-09-19T06:21:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.456+0000 7ff754c3e740 1 -- 172.21.15.5:0/2696614747 shutdown_connections 2024-09-19T06:21:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.456+0000 7ff754c3e740 1 --2- 172.21.15.5:0/2696614747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2be70c0 0x560bf2bfeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.456+0000 7ff754c3e740 1 --2- 172.21.15.5:0/2696614747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 0x560bf2be4870 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.456+0000 7ff754c3e740 1 --2- 172.21.15.5:0/2696614747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2bd7970 0x560bf2be62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.456+0000 7ff754c3e740 1 -- 172.21.15.5:0/2696614747 >> 172.21.15.5:0/2696614747 conn(0x560bf2bd4060 msgr2=0x560bf2bd31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.456+0000 7ff754c3e740 1 -- 172.21.15.5:0/2696614747 shutdown_connections 2024-09-19T06:21:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.457+0000 7ff754c3e740 1 -- 172.21.15.5:0/2696614747 wait complete. 2024-09-19T06:21:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.457+0000 7ff754c3e740 1 -- start start 2024-09-19T06:21:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.457+0000 7ff754c3e740 1 Processor -- start 2024-09-19T06:21:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff754c3e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2bd7970 0x560bf2be1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff754c3e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 0x560bf2be1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff754c3e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2be70c0 0x560bf2be1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff754c3e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1ac9c0 con 0x560bf2bd7970 2024-09-19T06:21:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff754c3e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1ac2f0 con 0x560bf2bd8290 2024-09-19T06:21:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff754c3e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1a9570 con 0x560bf2be70c0 2024-09-19T06:21:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff754c3e740 10 client.0 _ll_register_callbacks cb 0x7ff7400097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff74affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 0x560bf2be1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff74bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2bd7970 0x560bf2be1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff74affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 0x560bf2be1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55218/0 (socket says 172.21.15.5:55218) 2024-09-19T06:21:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff74bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2bd7970 0x560bf2be1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36552/0 (socket says 172.21.15.5:36552) 2024-09-19T06:21:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff74affd640 1 -- 172.21.15.5:0/449212957 learned_addr learned my addr 172.21.15.5:0/449212957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff74bfff640 1 --2- 172.21.15.5:0/449212957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2be70c0 0x560bf2be1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff74affd640 1 -- 172.21.15.5:0/449212957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2be70c0 msgr2=0x560bf2be1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff74affd640 1 --2- 172.21.15.5:0/449212957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2be70c0 0x560bf2be1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff74affd640 1 -- 172.21.15.5:0/449212957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2bd7970 msgr2=0x560bf2be1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff74affd640 1 --2- 172.21.15.5:0/449212957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2bd7970 0x560bf2be1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.458+0000 7ff74affd640 1 -- 172.21.15.5:0/449212957 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bf2bda710 con 0x560bf2bd8290 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.459+0000 7ff74affd640 1 --2- 172.21.15.5:0/449212957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 0x560bf2be1550 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff740012ba0 tx=0x7ff740009730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60fa54142164fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.459+0000 7ff72affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.459+0000 7ff72affd640 1 -- 172.21.15.5:0/449212957 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740005850 con 0x560bf2bd8290 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.459+0000 7ff72affd640 1 -- 172.21.15.5:0/449212957 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7ff740018070 con 0x560bf2bd8290 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.459+0000 7ff754c3e740 1 -- 172.21.15.5:0/449212957 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff744036030 con 0x560bf2bd8290 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.459+0000 7ff72affd640 1 -- 172.21.15.5:0/449212957 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff740011d80 con 0x560bf2bd8290 2024-09-19T06:21:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.459+0000 7ff754c3e740 10 client.29372 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.459+0000 7ff754c3e740 1 -- 172.21.15.5:0/449212957 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x560bf2bdb9e0 con 0x560bf2bd8290 2024-09-19T06:21:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.459+0000 7ff72affd640 1 -- 172.21.15.5:0/449212957 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff744036030 con 0x560bf2bd8290 2024-09-19T06:21:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.460+0000 7ff72affd640 1 -- 172.21.15.5:0/449212957 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7ff740031bd0 con 0x560bf2bd8290 2024-09-19T06:21:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.460+0000 7ff754c3e740 10 client.29372 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.460+0000 7ff754c3e740 10 client.29372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.460+0000 7ff754c3e740 20 client.29372 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.460+0000 7ff754c3e740 1 --2- 172.21.15.5:0/449212957 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560bf2bf6f30 0x560bf2bde120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.460+0000 7ff754c3e740 4 client.29372 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:21:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.460+0000 7ff754c3e740 1 -- 172.21.15.5:0/449212957 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bf2b43d70 con 0x560bf2bf6f30 2024-09-19T06:21:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.460+0000 7ff74bfff640 1 --2- 172.21.15.5:0/449212957 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560bf2bf6f30 0x560bf2bde120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.460+0000 7ff72affd640 10 client.29372 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.460+0000 7ff74bfff640 1 --2- 172.21.15.5:0/449212957 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560bf2bf6f30 0x560bf2bde120 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b9e1ce4e4bc37c server_cookie=947f3fb20a62e351 in_seq=0 out_seq=0 2024-09-19T06:21:20.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:20 smithi005 ceph-mon[30955]: pgmap v567: 97 pgs: 97 active+clean; 828 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 307 op/s 2024-09-19T06:21:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff72affd640 1 -- 172.21.15.5:0/449212957 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bf2b43d70 con 0x560bf2bf6f30 2024-09-19T06:21:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff72affd640 10 client.29372 handle_command_reply: tid=0 2024-09-19T06:21:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff754c3e740 1 client.29372 shutdown 2024-09-19T06:21:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff754c3e740 1 -- 172.21.15.5:0/449212957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 msgr2=0x560bf2be1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff754c3e740 1 --2- 172.21.15.5:0/449212957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 0x560bf2be1550 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff740012ba0 tx=0x7ff740009730 comp rx=0 tx=0).stop 2024-09-19T06:21:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff754c3e740 1 -- 172.21.15.5:0/449212957 shutdown_connections 2024-09-19T06:21:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff754c3e740 1 --2- 172.21.15.5:0/449212957 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560bf2bf6f30 0x560bf2bde120 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff754c3e740 1 --2- 172.21.15.5:0/449212957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf2be70c0 0x560bf2be1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff754c3e740 1 --2- 172.21.15.5:0/449212957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bf2bd8290 0x560bf2be1550 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff754c3e740 1 --2- 172.21.15.5:0/449212957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bf2bd7970 0x560bf2be1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff72affd640 0 client.29372 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff754c3e740 1 -- 172.21.15.5:0/449212957 >> 172.21.15.5:0/449212957 conn(0x560bf2bd4060 msgr2=0x560bf2bd94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.528+0000 7ff754c3e740 1 -- 172.21.15.5:0/449212957 shutdown_connections 2024-09-19T06:21:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.529+0000 7ff754c3e740 1 -- 172.21.15.5:0/449212957 wait complete. 2024-09-19T06:21:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.529+0000 7ff754c3e740 20 client.29372 trim_cache size 0 max 16384 2024-09-19T06:21:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:21:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:21:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:21:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:21:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.529+0000 7ff751dc4640 1 -- 172.21.15.5:0/3006676181 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7340a33b0 msgr2=0x7ff7340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.529+0000 7ff751dc4640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7340a33b0 0x7ff7340a5870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0cd880 tx=0x7ff73c097ee0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.530+0000 7ff751dc4640 1 -- 172.21.15.5:0/3006676181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff74c1009a0 msgr2=0x7ff74c1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.530+0000 7ff751dc4640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff74c1009a0 0x7ff74c1982b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00e9f0 tx=0x7ff73c00edf0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.530+0000 7ff751dc4640 1 -- 172.21.15.5:0/3006676181 shutdown_connections 2024-09-19T06:21:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.530+0000 7ff751dc4640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff74c1069b0 0x7ff74c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.530+0000 7ff751dc4640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7340a33b0 0x7ff7340a5870 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.530+0000 7ff751dc4640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c101340 0x7ff74c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.530+0000 7ff751dc4640 1 --2- 172.21.15.5:0/3006676181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff74c1009a0 0x7ff74c1982b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.530+0000 7ff751dc4640 1 -- 172.21.15.5:0/3006676181 >> 172.21.15.5:0/3006676181 conn(0x7ff74c0779a0 msgr2=0x7ff74c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.530+0000 7ff751dc4640 1 -- 172.21.15.5:0/3006676181 shutdown_connections 2024-09-19T06:21:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:20.530+0000 7ff751dc4640 1 -- 172.21.15.5:0/3006676181 wait complete. 2024-09-19T06:21:20.561 INFO:tasks.workunit.client.0.smithi005.stdout: 10 47167 8.83 MB/sec execute 164 sec latency 100.664 ms 2024-09-19T06:21:20.593 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:21:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:20 smithi157 ceph-mon[35346]: pgmap v567: 97 pgs: 97 active+clean; 828 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 307 op/s 2024-09-19T06:21:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:20 smithi049 ceph-mon[34640]: pgmap v567: 97 pgs: 97 active+clean; 828 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 307 op/s 2024-09-19T06:21:21.561 INFO:tasks.workunit.client.0.smithi005.stdout: 10 47300 8.79 MB/sec execute 165 sec latency 97.206 ms 2024-09-19T06:21:21.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.634+0000 7f60b257c640 1 -- 172.21.15.5:0/3984374895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ac106a80 msgr2=0x7f60ac106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:21.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.634+0000 7f60b257c640 1 --2- 172.21.15.5:0/3984374895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ac106a80 0x7f60ac106e60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009930 tx=0x7f60a002a420 comp rx=0 tx=0).stop 2024-09-19T06:21:21.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.634+0000 7f60b257c640 1 -- 172.21.15.5:0/3984374895 shutdown_connections 2024-09-19T06:21:21.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.634+0000 7f60b257c640 1 --2- 172.21.15.5:0/3984374895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60ac101420 0x7f60ac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.634+0000 7f60b257c640 1 --2- 172.21.15.5:0/3984374895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac100a80 0x7f60ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.634+0000 7f60b257c640 1 --2- 172.21.15.5:0/3984374895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ac106a80 0x7f60ac106e60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.634+0000 7f60b257c640 1 -- 172.21.15.5:0/3984374895 >> 172.21.15.5:0/3984374895 conn(0x7f60ac0fc7a0 msgr2=0x7f60ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.634+0000 7f60b257c640 1 -- 172.21.15.5:0/3984374895 shutdown_connections 2024-09-19T06:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.634+0000 7f60b257c640 1 -- 172.21.15.5:0/3984374895 wait complete. 2024-09-19T06:21:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.635+0000 7f60b257c640 1 Processor -- start 2024-09-19T06:21:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.635+0000 7f60b257c640 1 -- start start 2024-09-19T06:21:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.636+0000 7f60b257c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ac100a80 0x7f60ac1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.636+0000 7f60b257c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60ac101420 0x7f60ac198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.636+0000 7f60b257c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac106a80 0x7f60ac19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.636+0000 7f60abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ac100a80 0x7f60ac1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.636+0000 7f60b257c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac110d80 con 0x7f60ac106a80 2024-09-19T06:21:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.636+0000 7f60abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ac100a80 0x7f60ac1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47104/0 (socket says 172.21.15.5:47104) 2024-09-19T06:21:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.636+0000 7f60abfff640 1 -- 172.21.15.5:0/413587169 learned_addr learned my addr 172.21.15.5:0/413587169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.636+0000 7f60b257c640 1 -- 172.21.15.5:0/413587169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac110c00 con 0x7f60ac101420 2024-09-19T06:21:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.636+0000 7f60b257c640 1 -- 172.21.15.5:0/413587169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac110f00 con 0x7f60ac100a80 2024-09-19T06:21:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.636+0000 7f60ab7fe640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60ac101420 0x7f60ac198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.636+0000 7f60b0b19640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac106a80 0x7f60ac19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.637+0000 7f60abfff640 1 -- 172.21.15.5:0/413587169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60ac101420 msgr2=0x7f60ac198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.637+0000 7f60abfff640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60ac101420 0x7f60ac198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.637+0000 7f60abfff640 1 -- 172.21.15.5:0/413587169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac106a80 msgr2=0x7f60ac19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.637+0000 7f60abfff640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac106a80 0x7f60ac19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.637+0000 7f60abfff640 1 -- 172.21.15.5:0/413587169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac19d370 con 0x7f60ac100a80 2024-09-19T06:21:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.637+0000 7f60ab7fe640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60ac101420 0x7f60ac198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.637+0000 7f60b0b19640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac106a80 0x7f60ac19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.637+0000 7f60abfff640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ac100a80 0x7f60ac1983c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f60a0005ab0 tx=0x7f60a0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:21.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.637+0000 7f60a97fa640 1 -- 172.21.15.5:0/413587169 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a003f070 con 0x7f60ac100a80 2024-09-19T06:21:21.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.637+0000 7f60a97fa640 1 -- 172.21.15.5:0/413587169 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60a0031b50 con 0x7f60ac100a80 2024-09-19T06:21:21.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.637+0000 7f60a97fa640 1 -- 172.21.15.5:0/413587169 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0035030 con 0x7f60ac100a80 2024-09-19T06:21:21.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.637+0000 7f60b257c640 1 -- 172.21.15.5:0/413587169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac19d600 con 0x7f60ac100a80 2024-09-19T06:21:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.638+0000 7f60b257c640 1 -- 172.21.15.5:0/413587169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac19da90 con 0x7f60ac100a80 2024-09-19T06:21:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.641+0000 7f60a97fa640 1 -- 172.21.15.5:0/413587169 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f60a0042070 con 0x7f60ac100a80 2024-09-19T06:21:21.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.641+0000 7f60a97fa640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60940a3430 0x7f60940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.642+0000 7f60a97fa640 1 -- 172.21.15.5:0/413587169 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f60a00f0260 con 0x7f60ac100a80 2024-09-19T06:21:21.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.642+0000 7f60ab7fe640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60940a3430 0x7f60940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.642+0000 7f60ab7fe640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60940a3430 0x7f60940a58f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0ffee0 tx=0x7f609c005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.657+0000 7f60b4955740 1 Processor -- start 2024-09-19T06:21:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.657+0000 7f60b4955740 1 -- start start 2024-09-19T06:21:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b4955740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccedcec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b4955740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556ccedcd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b4955740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556ccede2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b4955740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cced332f0 con 0x556ccedcd120 2024-09-19T06:21:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b4955740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556ccecdacf0 con 0x556ccedcdb80 2024-09-19T06:21:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b4955740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556cced2f2b0 con 0x556ccedce840 2024-09-19T06:21:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b0b19640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556ccede2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556ccedcd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b0b19640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556ccede2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55246/0 (socket says 172.21.15.5:55246) 2024-09-19T06:21:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccedcec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b0b19640 1 -- 172.21.15.5:0/1569980726 learned_addr learned my addr 172.21.15.5:0/1569980726 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b0b19640 1 -- 172.21.15.5:0/1569980726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 msgr2=0x556ccedcec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b0b19640 1 --2- 172.21.15.5:0/1569980726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccedcec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b0b19640 1 -- 172.21.15.5:0/1569980726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 msgr2=0x556ccedcd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b0b19640 1 --2- 172.21.15.5:0/1569980726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556ccedcd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60b0b19640 1 -- 172.21.15.5:0/1569980726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ccecbf3a0 con 0x556ccedcdb80 2024-09-19T06:21:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.658+0000 7f60ab7fe640 1 --2- 172.21.15.5:0/1569980726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556ccedcd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.659+0000 7f60abfff640 1 --2- 172.21.15.5:0/1569980726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccedcec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.659+0000 7f60b0b19640 1 --2- 172.21.15.5:0/1569980726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556ccede2f60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f60a40117c0 tx=0x7f60a4011c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a8ba6b562aed6e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.659+0000 7f60927fc640 1 -- 172.21.15.5:0/1569980726 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4004d20 con 0x556ccedcdb80 2024-09-19T06:21:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.659+0000 7f60927fc640 1 -- 172.21.15.5:0/1569980726 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60a4003970 con 0x556ccedcdb80 2024-09-19T06:21:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.659+0000 7f60927fc640 1 -- 172.21.15.5:0/1569980726 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4004560 con 0x556ccedcdb80 2024-09-19T06:21:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.660+0000 7f60b4955740 1 -- 172.21.15.5:0/1569980726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 msgr2=0x556ccede2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.660+0000 7f60b4955740 1 --2- 172.21.15.5:0/1569980726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556ccede2f60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f60a40117c0 tx=0x7f60a4011c80 comp rx=0 tx=0).stop 2024-09-19T06:21:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.660+0000 7f60b4955740 1 -- 172.21.15.5:0/1569980726 shutdown_connections 2024-09-19T06:21:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.660+0000 7f60b4955740 1 --2- 172.21.15.5:0/1569980726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556ccede2f60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.660+0000 7f60b4955740 1 --2- 172.21.15.5:0/1569980726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556ccedcd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.660+0000 7f60b4955740 1 --2- 172.21.15.5:0/1569980726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccedcec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.660+0000 7f60b4955740 1 -- 172.21.15.5:0/1569980726 >> 172.21.15.5:0/1569980726 conn(0x556cced96310 msgr2=0x556ccedc9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.660+0000 7f60b4955740 1 -- 172.21.15.5:0/1569980726 shutdown_connections 2024-09-19T06:21:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.660+0000 7f60b4955740 1 -- 172.21.15.5:0/1569980726 wait complete. 2024-09-19T06:21:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.661+0000 7f60b4955740 1 -- start start 2024-09-19T06:21:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.661+0000 7f60b4955740 1 Processor -- start 2024-09-19T06:21:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.661+0000 7f60b4955740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556cceddce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.661+0000 7f60b4955740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556cceddd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.661+0000 7f60b4955740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccede2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.661+0000 7f60b4955740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ccedd4b20 con 0x556ccedcd120 2024-09-19T06:21:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.661+0000 7f60b4955740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556ccecee060 con 0x556ccedcdb80 2024-09-19T06:21:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.661+0000 7f60b4955740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556ccedd4ca0 con 0x556ccedce840 2024-09-19T06:21:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60abfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556cceddce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60abfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556cceddce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36590/0 (socket says 172.21.15.5:36590) 2024-09-19T06:21:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60b0b19640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccede2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60abfff640 1 -- 172.21.15.5:0/4289735253 learned_addr learned my addr 172.21.15.5:0/4289735253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60b4955740 10 client.0 _ll_register_callbacks cb 0x556ccea418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60ab7fe640 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556cceddd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60b0b19640 1 -- 172.21.15.5:0/4289735253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 msgr2=0x556cceddd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60b0b19640 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556cceddd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60b0b19640 1 -- 172.21.15.5:0/4289735253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 msgr2=0x556cceddce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60b0b19640 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556cceddce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60b0b19640 1 -- 172.21.15.5:0/4289735253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cceaa43c0 con 0x556ccedce840 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60abfff640 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556cceddce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f60b0b19640 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccede2b60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f60a40117c0 tx=0x7f60a400fec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c380c35367501b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f608b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.662+0000 7f608b7fe640 1 -- 172.21.15.5:0/4289735253 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4026070 con 0x556ccedce840 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.663+0000 7f608b7fe640 1 -- 172.21.15.5:0/4289735253 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f60a40053d0 con 0x556ccedce840 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.663+0000 7f60b4955740 1 -- 172.21.15.5:0/4289735253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a4003970 con 0x556ccedce840 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.663+0000 7f608b7fe640 1 -- 172.21.15.5:0/4289735253 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f60a4032050 con 0x556ccedce840 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.663+0000 7f60b4955740 10 client.29320 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.663+0000 7f60b4955740 1 -- 172.21.15.5:0/4289735253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556ccedf7ba0 con 0x556ccedce840 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.663+0000 7f608b7fe640 1 -- 172.21.15.5:0/4289735253 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60a4003970 con 0x556ccedce840 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.664+0000 7f608b7fe640 1 -- 172.21.15.5:0/4289735253 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f60a40199e0 con 0x556ccedce840 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.664+0000 7f60b4955740 10 client.29320 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.664+0000 7f60b4955740 10 client.29320 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:21:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.664+0000 7f60b4955740 20 client.29320 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.664+0000 7f60b4955740 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x556ccedf8f50 0x556ccedfb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.664+0000 7f60b4955740 4 client.29320 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:21:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.664+0000 7f60b4955740 1 -- 172.21.15.5:0/4289735253 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cced40860 con 0x556ccedf8f50 2024-09-19T06:21:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.664+0000 7f60abfff640 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x556ccedf8f50 0x556ccedfb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.665+0000 7f608b7fe640 10 client.29320 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:21:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.665+0000 7f60abfff640 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x556ccedf8f50 0x556ccedfb340 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=28a55e62df1a9e9c server_cookie=c87a0967661a0c8c in_seq=0 out_seq=0 2024-09-19T06:21:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f608b7fe640 1 -- 172.21.15.5:0/4289735253 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cced40860 con 0x556ccedf8f50 2024-09-19T06:21:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f608b7fe640 10 client.29320 handle_command_reply: tid=0 2024-09-19T06:21:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f60b4955740 1 client.29320 shutdown 2024-09-19T06:21:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f60b4955740 1 -- 172.21.15.5:0/4289735253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 msgr2=0x556ccede2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f60b4955740 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccede2b60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f60a40117c0 tx=0x7f60a400fec0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f60b4955740 1 -- 172.21.15.5:0/4289735253 shutdown_connections 2024-09-19T06:21:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f60b4955740 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x556ccedf8f50 0x556ccedfb340 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f60b4955740 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccede2b60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f60b4955740 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556cceddd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f60b4955740 1 --2- 172.21.15.5:0/4289735253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556cceddce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f60b4955740 1 -- 172.21.15.5:0/4289735253 >> 172.21.15.5:0/4289735253 conn(0x556cced96310 msgr2=0x556ccedcf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f608b7fe640 0 client.29320 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:21:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f60b4955740 1 -- 172.21.15.5:0/4289735253 shutdown_connections 2024-09-19T06:21:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.666+0000 7f60b4955740 1 -- 172.21.15.5:0/4289735253 wait complete. 2024-09-19T06:21:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.667+0000 7f60b4955740 20 client.29320 trim_cache size 0 max 16384 2024-09-19T06:21:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60b4955740 1 Processor -- start 2024-09-19T06:21:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60b4955740 1 -- start start 2024-09-19T06:21:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60b4955740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcd120 0x556ccede2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60b4955740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedcdb80 0x556ccedd4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60b4955740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedce840 0x556ccedd7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60b4955740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1ab690 con 0x556ccedce840 2024-09-19T06:21:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60b4955740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1ad1d0 con 0x556ccedcd120 2024-09-19T06:21:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60b4955740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1abd60 con 0x556ccedcdb80 2024-09-19T06:21:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcd120 0x556ccede2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcd120 0x556ccede2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55256/0 (socket says 172.21.15.5:55256) 2024-09-19T06:21:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60abfff640 1 -- 172.21.15.5:0/4095666885 learned_addr learned my addr 172.21.15.5:0/4095666885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60ab7fe640 1 --2- 172.21.15.5:0/4095666885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedcdb80 0x556ccedd4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.688+0000 7f60b0b19640 1 --2- 172.21.15.5:0/4095666885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedce840 0x556ccedd7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.689+0000 7f60b0b19640 1 -- 172.21.15.5:0/4095666885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedcdb80 msgr2=0x556ccedd4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.689+0000 7f60b0b19640 1 --2- 172.21.15.5:0/4095666885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedcdb80 0x556ccedd4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.689+0000 7f60b0b19640 1 -- 172.21.15.5:0/4095666885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcd120 msgr2=0x556ccede2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.689+0000 7f60b0b19640 1 --2- 172.21.15.5:0/4095666885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcd120 0x556ccede2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.689+0000 7f60b0b19640 1 -- 172.21.15.5:0/4095666885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ccedd90c0 con 0x556ccedce840 2024-09-19T06:21:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.689+0000 7f60abfff640 1 --2- 172.21.15.5:0/4095666885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcd120 0x556ccede2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.689+0000 7f60b0b19640 1 --2- 172.21.15.5:0/4095666885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedce840 0x556ccedd7040 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009b80 tx=0x7f60a4005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc386e1508a80373 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.689+0000 7f6091ffb640 1 -- 172.21.15.5:0/4095666885 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a403b070 con 0x556ccedce840 2024-09-19T06:21:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.689+0000 7f6091ffb640 1 -- 172.21.15.5:0/4095666885 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60a4003970 con 0x556ccedce840 2024-09-19T06:21:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.689+0000 7f6091ffb640 1 -- 172.21.15.5:0/4095666885 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a402f030 con 0x556ccedce840 2024-09-19T06:21:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.690+0000 7f60b4955740 1 -- 172.21.15.5:0/4095666885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedce840 msgr2=0x556ccedd7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.690+0000 7f60b4955740 1 --2- 172.21.15.5:0/4095666885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedce840 0x556ccedd7040 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009b80 tx=0x7f60a4005cf0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.690+0000 7f60b4955740 1 -- 172.21.15.5:0/4095666885 shutdown_connections 2024-09-19T06:21:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.690+0000 7f60b4955740 1 --2- 172.21.15.5:0/4095666885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedce840 0x556ccedd7040 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.690+0000 7f60b4955740 1 --2- 172.21.15.5:0/4095666885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedcdb80 0x556ccedd4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.690+0000 7f60b4955740 1 --2- 172.21.15.5:0/4095666885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcd120 0x556ccede2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.690+0000 7f60b4955740 1 -- 172.21.15.5:0/4095666885 >> 172.21.15.5:0/4095666885 conn(0x556cced96310 msgr2=0x556ccedd6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.691+0000 7f60b4955740 1 -- 172.21.15.5:0/4095666885 shutdown_connections 2024-09-19T06:21:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.691+0000 7f60b4955740 1 -- 172.21.15.5:0/4095666885 wait complete. 2024-09-19T06:21:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.691+0000 7f60b4955740 1 -- start start 2024-09-19T06:21:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.691+0000 7f60b4955740 1 Processor -- start 2024-09-19T06:21:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60b4955740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556ccedffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60b4955740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556ccee00030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60b4955740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccee005d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60b4955740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1acb00 con 0x556ccedcd120 2024-09-19T06:21:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60b4955740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1ac430 con 0x556ccedcdb80 2024-09-19T06:21:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60b4955740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1a9690 con 0x556ccedce840 2024-09-19T06:21:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60b4955740 10 client.0 _ll_register_callbacks cb 0x7f609c009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60ab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556ccee00030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60b0b19640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccee005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60ab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556ccee00030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55268/0 (socket says 172.21.15.5:55268) 2024-09-19T06:21:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60b0b19640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccee005d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47142/0 (socket says 172.21.15.5:47142) 2024-09-19T06:21:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60ab7fe640 1 -- 172.21.15.5:0/4149570975 learned_addr learned my addr 172.21.15.5:0/4149570975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60abfff640 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556ccedffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60ab7fe640 1 -- 172.21.15.5:0/4149570975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 msgr2=0x556ccee005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60ab7fe640 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccee005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60ab7fe640 1 -- 172.21.15.5:0/4149570975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 msgr2=0x556ccedffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60ab7fe640 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556ccedffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60ab7fe640 1 -- 172.21.15.5:0/4149570975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cceddd700 con 0x556ccedcdb80 2024-09-19T06:21:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.692+0000 7f60b0b19640 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccee005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.693+0000 7f60ab7fe640 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556ccee00030 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x556ccedcaac0 tx=0x7f609c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ee5c0f7802e73b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.693+0000 7f608b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:21:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.693+0000 7f608b7fe640 1 -- 172.21.15.5:0/4149570975 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c00f810 con 0x556ccedcdb80 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.693+0000 7f608b7fe640 1 -- 172.21.15.5:0/4149570975 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f609c00f9b0 con 0x556ccedcdb80 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.693+0000 7f60b4955740 1 -- 172.21.15.5:0/4149570975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a4003970 con 0x556ccedcdb80 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.693+0000 7f608b7fe640 1 -- 172.21.15.5:0/4149570975 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f609c00fbb0 con 0x556ccedcdb80 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.693+0000 7f60b4955740 10 client.29390 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.693+0000 7f60b4955740 1 -- 172.21.15.5:0/4149570975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556ccee0cbf0 con 0x556ccedcdb80 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.694+0000 7f608b7fe640 1 -- 172.21.15.5:0/4149570975 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60a4003970 con 0x556ccedcdb80 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.694+0000 7f608b7fe640 1 -- 172.21.15.5:0/4149570975 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f609c01a040 con 0x556ccedcdb80 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.694+0000 7f60b4955740 10 client.29390 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.694+0000 7f60b4955740 10 client.29390 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.694+0000 7f60b4955740 20 client.29390 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.694+0000 7f60b4955740 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x556ccee63950 0x556ccee65d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.694+0000 7f60b4955740 4 client.29390 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:21:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.694+0000 7f60b4955740 1 -- 172.21.15.5:0/4149570975 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cced3ad70 con 0x556ccee63950 2024-09-19T06:21:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.694+0000 7f60abfff640 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x556ccee63950 0x556ccee65d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f608b7fe640 10 client.29390 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:21:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f60abfff640 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x556ccee63950 0x556ccee65d40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c24648a21f7fbff server_cookie=b1d1973726f99e7c in_seq=0 out_seq=0 2024-09-19T06:21:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f608b7fe640 1 -- 172.21.15.5:0/4149570975 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cced3ad70 con 0x556ccee63950 2024-09-19T06:21:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f608b7fe640 10 client.29390 handle_command_reply: tid=0 2024-09-19T06:21:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f60b4955740 1 client.29390 shutdown 2024-09-19T06:21:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f60b4955740 1 -- 172.21.15.5:0/4149570975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 msgr2=0x556ccee00030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f60b4955740 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556ccee00030 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x556ccedcaac0 tx=0x7f609c004730 comp rx=0 tx=0).stop 2024-09-19T06:21:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f60b4955740 1 -- 172.21.15.5:0/4149570975 shutdown_connections 2024-09-19T06:21:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f60b4955740 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x556ccee63950 0x556ccee65d40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f608b7fe640 0 client.29390 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:21:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f60b4955740 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ccedce840 0x556ccee005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f60b4955740 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ccedcdb80 0x556ccee00030 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f60b4955740 1 --2- 172.21.15.5:0/4149570975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccedcd120 0x556ccedffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f60b4955740 1 -- 172.21.15.5:0/4149570975 >> 172.21.15.5:0/4149570975 conn(0x556cced96310 msgr2=0x556ccedd4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f60b4955740 1 -- 172.21.15.5:0/4149570975 shutdown_connections 2024-09-19T06:21:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.695+0000 7f60b4955740 1 -- 172.21.15.5:0/4149570975 wait complete. 2024-09-19T06:21:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b4955740 20 client.29390 trim_cache size 0 max 16384 2024-09-19T06:21:21.703 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:21:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b257c640 1 -- 172.21.15.5:0/413587169 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60940a3430 msgr2=0x7f60940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b257c640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60940a3430 0x7f60940a58f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0ffee0 tx=0x7f609c005f90 comp rx=0 tx=0).stop 2024-09-19T06:21:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b257c640 1 -- 172.21.15.5:0/413587169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ac100a80 msgr2=0x7f60ac1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b257c640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ac100a80 0x7f60ac1983c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f60a0005ab0 tx=0x7f60a0003b90 comp rx=0 tx=0).stop 2024-09-19T06:21:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b257c640 1 -- 172.21.15.5:0/413587169 shutdown_connections 2024-09-19T06:21:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b257c640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac106a80 0x7f60ac19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b257c640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60ac101420 0x7f60ac198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b257c640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f60940a3430 0x7f60940a58f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b257c640 1 --2- 172.21.15.5:0/413587169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ac100a80 0x7f60ac1983c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b257c640 1 -- 172.21.15.5:0/413587169 >> 172.21.15.5:0/413587169 conn(0x7f60ac0fc7a0 msgr2=0x7f60ac06a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b257c640 1 -- 172.21.15.5:0/413587169 shutdown_connections 2024-09-19T06:21:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:21.696+0000 7f60b257c640 1 -- 172.21.15.5:0/413587169 wait complete. 2024-09-19T06:21:22.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:21:22.561 INFO:tasks.workunit.client.0.smithi005.stdout: 10 47444 8.79 MB/sec execute 166 sec latency 55.028 ms 2024-09-19T06:21:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:22 smithi157 ceph-mon[35346]: pgmap v568: 97 pgs: 97 active+clean; 828 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 267 op/s 2024-09-19T06:21:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:22 smithi005 ceph-mon[30955]: pgmap v568: 97 pgs: 97 active+clean; 828 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 267 op/s 2024-09-19T06:21:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:21:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:22 smithi049 ceph-mon[34640]: pgmap v568: 97 pgs: 97 active+clean; 828 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 267 op/s 2024-09-19T06:21:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.556+0000 7f8afa0ea640 1 -- 172.21.15.5:0/3609047732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8af4100500 msgr2=0x7f8af410ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.556+0000 7f8afa0ea640 1 --2- 172.21.15.5:0/3609047732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8af4100500 0x7f8af410ea20 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009ca0 tx=0x7f8aec02a8b0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.556+0000 7f8afa0ea640 1 -- 172.21.15.5:0/3609047732 shutdown_connections 2024-09-19T06:21:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.556+0000 7f8afa0ea640 1 --2- 172.21.15.5:0/3609047732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8af4100500 0x7f8af410ea20 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.556+0000 7f8afa0ea640 1 --2- 172.21.15.5:0/3609047732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8af40ffb60 0x7f8af40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.556+0000 7f8afa0ea640 1 --2- 172.21.15.5:0/3609047732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af4106b80 0x7f8af4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.556+0000 7f8afa0ea640 1 -- 172.21.15.5:0/3609047732 >> 172.21.15.5:0/3609047732 conn(0x7f8af40fba90 msgr2=0x7f8af40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.557+0000 7f8afa0ea640 1 -- 172.21.15.5:0/3609047732 shutdown_connections 2024-09-19T06:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.557+0000 7f8afa0ea640 1 -- 172.21.15.5:0/3609047732 wait complete. 2024-09-19T06:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.558+0000 7f8afa0ea640 1 Processor -- start 2024-09-19T06:21:23.561 INFO:tasks.workunit.client.0.smithi005.stdout: 10 47591 8.79 MB/sec execute 167 sec latency 75.391 ms 2024-09-19T06:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.558+0000 7f8afa0ea640 1 -- start start 2024-09-19T06:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.559+0000 7f8afa0ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8af40ffb60 0x7f8af41015d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.559+0000 7f8afa0ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8af4100500 0x7f8af4101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.559+0000 7f8afa0ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af4106b80 0x7f8af4102050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.559+0000 7f8afa0ea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8af4111e50 con 0x7f8af4100500 2024-09-19T06:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.559+0000 7f8afa0ea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8af4111cd0 con 0x7f8af40ffb60 2024-09-19T06:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.559+0000 7f8afa0ea640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8af4111fd0 con 0x7f8af4106b80 2024-09-19T06:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.559+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af4106b80 0x7f8af4102050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.559+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af4106b80 0x7f8af4102050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47164/0 (socket says 172.21.15.5:47164) 2024-09-19T06:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.559+0000 7f8af3fff640 1 -- 172.21.15.5:0/2717487616 learned_addr learned my addr 172.21.15.5:0/2717487616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.559+0000 7f8af37fe640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8af40ffb60 0x7f8af41015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.560+0000 7f8af3fff640 1 -- 172.21.15.5:0/2717487616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8af40ffb60 msgr2=0x7f8af41015d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.560+0000 7f8af3fff640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8af40ffb60 0x7f8af41015d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.560+0000 7f8af3fff640 1 -- 172.21.15.5:0/2717487616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8af4100500 msgr2=0x7f8af4101b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.560+0000 7f8af3fff640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8af4100500 0x7f8af4101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.560+0000 7f8af3fff640 1 -- 172.21.15.5:0/2717487616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af41aacc0 con 0x7f8af4106b80 2024-09-19T06:21:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.560+0000 7f8af3fff640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af4106b80 0x7f8af4102050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009bf0 tx=0x7f8aec030d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.561+0000 7f8af0ff9640 1 -- 172.21.15.5:0/2717487616 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec003e00 con 0x7f8af4106b80 2024-09-19T06:21:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.561+0000 7f8afa0ea640 1 -- 172.21.15.5:0/2717487616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af41aaef0 con 0x7f8af4106b80 2024-09-19T06:21:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.561+0000 7f8afa0ea640 1 -- 172.21.15.5:0/2717487616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af41ab3a0 con 0x7f8af4106b80 2024-09-19T06:21:23.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.562+0000 7f8af0ff9640 1 -- 172.21.15.5:0/2717487616 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8aec040070 con 0x7f8af4106b80 2024-09-19T06:21:23.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.562+0000 7f8af0ff9640 1 -- 172.21.15.5:0/2717487616 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec045ab0 con 0x7f8af4106b80 2024-09-19T06:21:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.565+0000 7f8af0ff9640 1 -- 172.21.15.5:0/2717487616 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8aec031440 con 0x7f8af4106b80 2024-09-19T06:21:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.565+0000 7f8af0ff9640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8adc0a3540 0x7f8adc0a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.566+0000 7f8af0ff9640 1 -- 172.21.15.5:0/2717487616 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f8aec0f2070 con 0x7f8af4106b80 2024-09-19T06:21:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.566+0000 7f8af37fe640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8adc0a3540 0x7f8adc0a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.573+0000 7f8af37fe640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8adc0a3540 0x7f8adc0a5a00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009b50 tx=0x7f8ae4005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.575+0000 7f8afc4c3740 1 Processor -- start 2024-09-19T06:21:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.575+0000 7f8afc4c3740 1 -- start start 2024-09-19T06:21:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.575+0000 7f8afc4c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b59d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.575+0000 7f8afc4c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59d5d0 0x561b4b59da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.575+0000 7f8afc4c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b5b9380 0x561b4b5bb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.575+0000 7f8afc4c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561b4b5032f0 con 0x561b4b59d5d0 2024-09-19T06:21:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.575+0000 7f8afc4c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561b4b4aacf0 con 0x561b4b59eb40 2024-09-19T06:21:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.575+0000 7f8afc4c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561b4b4ff2b0 con 0x561b4b5b9380 2024-09-19T06:21:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.575+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b5b9380 0x561b4b5bb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.575+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b5b9380 0x561b4b5bb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47174/0 (socket says 172.21.15.5:47174) 2024-09-19T06:21:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.575+0000 7f8af3fff640 1 -- 172.21.15.5:0/474212713 learned_addr learned my addr 172.21.15.5:0/474212713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.575+0000 7f8af2ffd640 1 --2- 172.21.15.5:0/474212713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59d5d0 0x561b4b59da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.576+0000 7f8af37fe640 1 --2- 172.21.15.5:0/474212713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b59d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.576+0000 7f8af3fff640 1 -- 172.21.15.5:0/474212713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 msgr2=0x561b4b59d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.576+0000 7f8af3fff640 1 --2- 172.21.15.5:0/474212713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b59d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.576+0000 7f8af3fff640 1 -- 172.21.15.5:0/474212713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59d5d0 msgr2=0x561b4b59da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:23.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.576+0000 7f8af3fff640 1 --2- 172.21.15.5:0/474212713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59d5d0 0x561b4b59da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.576+0000 7f8af3fff640 1 -- 172.21.15.5:0/474212713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b4b48f3a0 con 0x561b4b5b9380 2024-09-19T06:21:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.576+0000 7f8af3fff640 1 --2- 172.21.15.5:0/474212713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b5b9380 0x561b4b5bb770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8aec005e30 tx=0x7f8aec045510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e262c797fcb4344a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.577+0000 7f8ad9ffb640 1 -- 172.21.15.5:0/474212713 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0d7070 con 0x561b4b5b9380 2024-09-19T06:21:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.577+0000 7f8ad9ffb640 1 -- 172.21.15.5:0/474212713 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8aec036030 con 0x561b4b5b9380 2024-09-19T06:21:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.577+0000 7f8ad9ffb640 1 -- 172.21.15.5:0/474212713 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0d7210 con 0x561b4b5b9380 2024-09-19T06:21:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.578+0000 7f8afc4c3740 1 -- 172.21.15.5:0/474212713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b5b9380 msgr2=0x561b4b5bb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.578+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/474212713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b5b9380 0x561b4b5bb770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8aec005e30 tx=0x7f8aec045510 comp rx=0 tx=0).stop 2024-09-19T06:21:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.578+0000 7f8afc4c3740 1 -- 172.21.15.5:0/474212713 shutdown_connections 2024-09-19T06:21:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.578+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/474212713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b5b9380 0x561b4b5bb770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.578+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/474212713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59d5d0 0x561b4b59da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.578+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/474212713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b59d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.578+0000 7f8afc4c3740 1 -- 172.21.15.5:0/474212713 >> 172.21.15.5:0/474212713 conn(0x561b4b566310 msgr2=0x561b4b599c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.578+0000 7f8afc4c3740 1 -- 172.21.15.5:0/474212713 shutdown_connections 2024-09-19T06:21:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.578+0000 7f8afc4c3740 1 -- 172.21.15.5:0/474212713 wait complete. 2024-09-19T06:21:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.578+0000 7f8afc4c3740 1 -- start start 2024-09-19T06:21:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.579+0000 7f8afc4c3740 1 Processor -- start 2024-09-19T06:21:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.579+0000 7f8afc4c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59d5d0 0x561b4b5b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.579+0000 7f8afc4c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b5b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.579+0000 7f8afc4c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b5b9380 0x561b4b5b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.579+0000 7f8afc4c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561b4b5bbe40 con 0x561b4b59d5d0 2024-09-19T06:21:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.579+0000 7f8afc4c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561b4b4be060 con 0x561b4b59eb40 2024-09-19T06:21:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.579+0000 7f8afc4c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561b4b5bbfc0 con 0x561b4b5b9380 2024-09-19T06:21:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.579+0000 7f8afc4c3740 10 client.0 _ll_register_callbacks cb 0x561b4b2118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.579+0000 7f8af37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59d5d0 0x561b4b5b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.579+0000 7f8af37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59d5d0 0x561b4b5b8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36656/0 (socket says 172.21.15.5:36656) 2024-09-19T06:21:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.580+0000 7f8af37fe640 1 -- 172.21.15.5:0/85342087 learned_addr learned my addr 172.21.15.5:0/85342087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.580+0000 7f8af2ffd640 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b5b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.580+0000 7f8af3fff640 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b5b9380 0x561b4b5b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.580+0000 7f8af2ffd640 1 -- 172.21.15.5:0/85342087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b5b9380 msgr2=0x561b4b5b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.580+0000 7f8af2ffd640 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b5b9380 0x561b4b5b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.580+0000 7f8af2ffd640 1 -- 172.21.15.5:0/85342087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59d5d0 msgr2=0x561b4b5b8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.580+0000 7f8af2ffd640 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59d5d0 0x561b4b5b8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.580+0000 7f8af2ffd640 1 -- 172.21.15.5:0/85342087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b4b2743c0 con 0x561b4b59eb40 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.580+0000 7f8af2ffd640 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b5b60c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8af4102d90 tx=0x7f8ae80118e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7393c6958757a683 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.580+0000 7f8ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.580+0000 7f8ad2ffd640 1 -- 172.21.15.5:0/85342087 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8012920 con 0x561b4b59eb40 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.580+0000 7f8ad2ffd640 1 -- 172.21.15.5:0/85342087 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f8ae8012ac0 con 0x561b4b59eb40 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8afc4c3740 1 -- 172.21.15.5:0/85342087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aec036030 con 0x561b4b59eb40 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8ad2ffd640 1 -- 172.21.15.5:0/85342087 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8ae8012cc0 con 0x561b4b59eb40 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8afc4c3740 10 client.29402 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8afc4c3740 1 -- 172.21.15.5:0/85342087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561b4b5c79a0 con 0x561b4b59eb40 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8ad2ffd640 1 -- 172.21.15.5:0/85342087 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ae8023930 con 0x561b4b59eb40 2024-09-19T06:21:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8ad2ffd640 1 -- 172.21.15.5:0/85342087 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f8ae800f640 con 0x561b4b59eb40 2024-09-19T06:21:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8afc4c3740 10 client.29402 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8afc4c3740 10 client.29402 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:21:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8afc4c3740 20 client.29402 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x561b4b5c8f20 0x561b4b5cb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8afc4c3740 4 client.29402 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:21:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8afc4c3740 1 -- 172.21.15.5:0/85342087 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8aec036030 con 0x561b4b5c8f20 2024-09-19T06:21:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.581+0000 7f8af37fe640 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x561b4b5c8f20 0x561b4b5cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.582+0000 7f8ad2ffd640 10 client.29402 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:21:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.582+0000 7f8af37fe640 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x561b4b5c8f20 0x561b4b5cb340 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=379e362489711066 server_cookie=fb7d1833358f2d7d in_seq=0 out_seq=0 2024-09-19T06:21:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8ad2ffd640 1 -- 172.21.15.5:0/85342087 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8aec036030 con 0x561b4b5c8f20 2024-09-19T06:21:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8ad2ffd640 10 client.29402 handle_command_reply: tid=0 2024-09-19T06:21:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8afc4c3740 1 client.29402 shutdown 2024-09-19T06:21:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8afc4c3740 1 -- 172.21.15.5:0/85342087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 msgr2=0x561b4b5b60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b5b60c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8af4102d90 tx=0x7f8ae80118e0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8afc4c3740 1 -- 172.21.15.5:0/85342087 shutdown_connections 2024-09-19T06:21:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x561b4b5c8f20 0x561b4b5cb340 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8ad2ffd640 0 client.29402 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:21:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b5b9380 0x561b4b5b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b5b60c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/85342087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59d5d0 0x561b4b5b8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8afc4c3740 1 -- 172.21.15.5:0/85342087 >> 172.21.15.5:0/85342087 conn(0x561b4b566310 msgr2=0x561b4b59f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8afc4c3740 1 -- 172.21.15.5:0/85342087 shutdown_connections 2024-09-19T06:21:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8afc4c3740 1 -- 172.21.15.5:0/85342087 wait complete. 2024-09-19T06:21:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.583+0000 7f8afc4c3740 20 client.29402 trim_cache size 0 max 16384 2024-09-19T06:21:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.604+0000 7f8afc4c3740 1 Processor -- start 2024-09-19T06:21:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.604+0000 7f8afc4c3740 1 -- start start 2024-09-19T06:21:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.604+0000 7f8afc4c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b59d5d0 0x561b4b599d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.604+0000 7f8afc4c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b5b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.604+0000 7f8afc4c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b5b9380 0x561b4b5d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.604+0000 7f8afc4c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae000e7e0 con 0x561b4b5b9380 2024-09-19T06:21:23.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.604+0000 7f8afc4c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae0010320 con 0x561b4b59eb40 2024-09-19T06:21:23.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.604+0000 7f8afc4c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae000eeb0 con 0x561b4b59d5d0 2024-09-19T06:21:23.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.605+0000 7f8af2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b5b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.605+0000 7f8af2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b5b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55318/0 (socket says 172.21.15.5:55318) 2024-09-19T06:21:23.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.605+0000 7f8af2ffd640 1 -- 172.21.15.5:0/3362867152 learned_addr learned my addr 172.21.15.5:0/3362867152 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:23.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.605+0000 7f8af37fe640 1 --2- 172.21.15.5:0/3362867152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b59d5d0 0x561b4b599d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.605+0000 7f8af3fff640 1 --2- 172.21.15.5:0/3362867152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b5b9380 0x561b4b5d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.605+0000 7f8af2ffd640 1 -- 172.21.15.5:0/3362867152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b59d5d0 msgr2=0x561b4b599d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:23.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.605+0000 7f8af2ffd640 1 --2- 172.21.15.5:0/3362867152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b59d5d0 0x561b4b599d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.605+0000 7f8af2ffd640 1 -- 172.21.15.5:0/3362867152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b5b9380 msgr2=0x561b4b5d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:23.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.605+0000 7f8af2ffd640 1 --2- 172.21.15.5:0/3362867152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b5b9380 0x561b4b5d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.605+0000 7f8af2ffd640 1 -- 172.21.15.5:0/3362867152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b4b5b6bc0 con 0x561b4b59eb40 2024-09-19T06:21:23.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.605+0000 7f8af2ffd640 1 --2- 172.21.15.5:0/3362867152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b5b50c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8026040 tx=0x7f8ae8011fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ff49791e643e8a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:23.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.606+0000 7f8ad97fa640 1 -- 172.21.15.5:0/3362867152 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8030020 con 0x561b4b59eb40 2024-09-19T06:21:23.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.606+0000 7f8ad97fa640 1 -- 172.21.15.5:0/3362867152 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ae800fe50 con 0x561b4b59eb40 2024-09-19T06:21:23.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.606+0000 7f8ad97fa640 1 -- 172.21.15.5:0/3362867152 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8004a00 con 0x561b4b59eb40 2024-09-19T06:21:23.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.607+0000 7f8afc4c3740 1 -- 172.21.15.5:0/3362867152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 msgr2=0x561b4b5b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:23.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.607+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/3362867152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b5b50c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8026040 tx=0x7f8ae8011fc0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.607+0000 7f8afc4c3740 1 -- 172.21.15.5:0/3362867152 shutdown_connections 2024-09-19T06:21:23.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.607+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/3362867152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b5b9380 0x561b4b5d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.607+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/3362867152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b59eb40 0x561b4b5b50c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.607+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/3362867152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b59d5d0 0x561b4b599d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.607+0000 7f8afc4c3740 1 -- 172.21.15.5:0/3362867152 >> 172.21.15.5:0/3362867152 conn(0x561b4b566310 msgr2=0x561b4b5b49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:23.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.607+0000 7f8afc4c3740 1 -- 172.21.15.5:0/3362867152 shutdown_connections 2024-09-19T06:21:23.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.607+0000 7f8afc4c3740 1 -- 172.21.15.5:0/3362867152 wait complete. 2024-09-19T06:21:23.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.607+0000 7f8afc4c3740 1 -- start start 2024-09-19T06:21:23.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8afc4c3740 1 Processor -- start 2024-09-19T06:21:23.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8afc4c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b59d5d0 0x561b4b5c1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8afc4c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59eb40 0x561b4b5b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8afc4c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b5b9380 0x561b4b5b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8afc4c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae000fc50 con 0x561b4b59eb40 2024-09-19T06:21:23.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8afc4c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae000f580 con 0x561b4b5b9380 2024-09-19T06:21:23.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8afc4c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae000c7e0 con 0x561b4b59d5d0 2024-09-19T06:21:23.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8afc4c3740 10 client.0 _ll_register_callbacks cb 0x7f8ae8009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:23.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8af37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b59d5d0 0x561b4b5c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8af37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b59d5d0 0x561b4b5c1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47192/0 (socket says 172.21.15.5:47192) 2024-09-19T06:21:23.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8af37fe640 1 -- 172.21.15.5:0/4002120304 learned_addr learned my addr 172.21.15.5:0/4002120304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8af3fff640 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b5b9380 0x561b4b5b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8af2ffd640 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59eb40 0x561b4b5b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8af37fe640 1 -- 172.21.15.5:0/4002120304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b5b9380 msgr2=0x561b4b5b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8af37fe640 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b5b9380 0x561b4b5b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8af37fe640 1 -- 172.21.15.5:0/4002120304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59eb40 msgr2=0x561b4b5b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8af37fe640 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59eb40 0x561b4b5b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8af37fe640 1 -- 172.21.15.5:0/4002120304 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b4b5b3920 con 0x561b4b59d5d0 2024-09-19T06:21:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.608+0000 7f8af2ffd640 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59eb40 0x561b4b5b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:21:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.609+0000 7f8af37fe640 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b59d5d0 0x561b4b5c1830 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x561b4b5cbb40 tx=0x7f8ae4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feb9858e10fc5e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.609+0000 7f8ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:21:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.609+0000 7f8ad2ffd640 1 -- 172.21.15.5:0/4002120304 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae402b7d0 con 0x561b4b59d5d0 2024-09-19T06:21:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.609+0000 7f8afc4c3740 1 -- 172.21.15.5:0/4002120304 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ae800fe50 con 0x561b4b59d5d0 2024-09-19T06:21:23.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.609+0000 7f8ad2ffd640 1 -- 172.21.15.5:0/4002120304 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f8ae402b970 con 0x561b4b59d5d0 2024-09-19T06:21:23.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.609+0000 7f8ad2ffd640 1 -- 172.21.15.5:0/4002120304 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8ae402bb70 con 0x561b4b59d5d0 2024-09-19T06:21:23.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.609+0000 7f8afc4c3740 10 client.29344 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:23.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.609+0000 7f8afc4c3740 1 -- 172.21.15.5:0/4002120304 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561b4b633ac0 con 0x561b4b59d5d0 2024-09-19T06:21:23.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.609+0000 7f8ad2ffd640 1 -- 172.21.15.5:0/4002120304 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ae800fe50 con 0x561b4b59d5d0 2024-09-19T06:21:23.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.610+0000 7f8ad2ffd640 1 -- 172.21.15.5:0/4002120304 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f8ae401f430 con 0x561b4b59d5d0 2024-09-19T06:21:23.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.610+0000 7f8afc4c3740 10 client.29344 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:23.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.610+0000 7f8afc4c3740 10 client.29344 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:21:23.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.610+0000 7f8afc4c3740 20 client.29344 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:23.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.610+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x561b4b5c8f20 0x561b4b6362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.610+0000 7f8afc4c3740 4 client.29344 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.610+0000 7f8afc4c3740 1 -- 172.21.15.5:0/4002120304 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b4b510860 con 0x561b4b5c8f20 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.610+0000 7f8af2ffd640 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x561b4b5c8f20 0x561b4b6362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.610+0000 7f8af2ffd640 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x561b4b5c8f20 0x561b4b6362e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e09b5af86c9b078 server_cookie=281586ef1a45fa80 in_seq=0 out_seq=0 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.610+0000 7f8ad2ffd640 10 client.29344 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8ad2ffd640 1 -- 172.21.15.5:0/4002120304 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b4b510860 con 0x561b4b5c8f20 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8ad2ffd640 10 client.29344 handle_command_reply: tid=0 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8afc4c3740 1 client.29344 shutdown 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8afc4c3740 1 -- 172.21.15.5:0/4002120304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b59d5d0 msgr2=0x561b4b5c1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b59d5d0 0x561b4b5c1830 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x561b4b5cbb40 tx=0x7f8ae4004730 comp rx=0 tx=0).stop 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8afc4c3740 1 -- 172.21.15.5:0/4002120304 shutdown_connections 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x561b4b5c8f20 0x561b4b6362e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8ad2ffd640 0 client.29344 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:21:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b4b5b9380 0x561b4b5b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b4b59eb40 0x561b4b5b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8afc4c3740 1 --2- 172.21.15.5:0/4002120304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b4b59d5d0 0x561b4b5c1830 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8afc4c3740 1 -- 172.21.15.5:0/4002120304 >> 172.21.15.5:0/4002120304 conn(0x561b4b566310 msgr2=0x561b4b59f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8afc4c3740 1 -- 172.21.15.5:0/4002120304 shutdown_connections 2024-09-19T06:21:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8afc4c3740 1 -- 172.21.15.5:0/4002120304 wait complete. 2024-09-19T06:21:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.611+0000 7f8afc4c3740 20 client.29344 trim_cache size 0 max 16384 2024-09-19T06:21:23.620 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:21:23.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.612+0000 7f8afa0ea640 1 -- 172.21.15.5:0/2717487616 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8adc0a3540 msgr2=0x7f8adc0a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:23.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.612+0000 7f8afa0ea640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8adc0a3540 0x7f8adc0a5a00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009b50 tx=0x7f8ae4005f90 comp rx=0 tx=0).stop 2024-09-19T06:21:23.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.612+0000 7f8afa0ea640 1 -- 172.21.15.5:0/2717487616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af4106b80 msgr2=0x7f8af4102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:23.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.612+0000 7f8afa0ea640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af4106b80 0x7f8af4102050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009bf0 tx=0x7f8aec030d90 comp rx=0 tx=0).stop 2024-09-19T06:21:23.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.612+0000 7f8afa0ea640 1 -- 172.21.15.5:0/2717487616 shutdown_connections 2024-09-19T06:21:23.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.612+0000 7f8afa0ea640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8adc0a3540 0x7f8adc0a5a00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.612+0000 7f8afa0ea640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af4106b80 0x7f8af4102050 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.612+0000 7f8afa0ea640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8af4100500 0x7f8af4101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.612+0000 7f8afa0ea640 1 --2- 172.21.15.5:0/2717487616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8af40ffb60 0x7f8af41015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:23.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.612+0000 7f8afa0ea640 1 -- 172.21.15.5:0/2717487616 >> 172.21.15.5:0/2717487616 conn(0x7f8af40fba90 msgr2=0x7f8af40fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:23.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.612+0000 7f8afa0ea640 1 -- 172.21.15.5:0/2717487616 shutdown_connections 2024-09-19T06:21:23.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:23.612+0000 7f8afa0ea640 1 -- 172.21.15.5:0/2717487616 wait complete. 2024-09-19T06:21:24.561 INFO:tasks.workunit.client.0.smithi005.stdout: 10 47735 8.78 MB/sec execute 168 sec latency 130.341 ms 2024-09-19T06:21:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:24 smithi157 ceph-mon[35346]: pgmap v569: 97 pgs: 97 active+clean; 842 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.3 MiB/s wr, 389 op/s 2024-09-19T06:21:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:24 smithi005 ceph-mon[30955]: pgmap v569: 97 pgs: 97 active+clean; 842 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.3 MiB/s wr, 389 op/s 2024-09-19T06:21:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:24 smithi049 ceph-mon[34640]: pgmap v569: 97 pgs: 97 active+clean; 842 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.3 MiB/s wr, 389 op/s 2024-09-19T06:21:25.561 INFO:tasks.workunit.client.0.smithi005.stdout: 10 47870 8.78 MB/sec execute 169 sec latency 127.013 ms 2024-09-19T06:21:26.205 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:21:26.561 INFO:tasks.workunit.client.0.smithi005.stdout: 10 48020 8.77 MB/sec execute 170 sec latency 136.222 ms 2024-09-19T06:21:26.623 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:21:26.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:26 smithi005 ceph-mon[30955]: pgmap v570: 97 pgs: 97 active+clean; 842 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 237 op/s 2024-09-19T06:21:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:26 smithi157 ceph-mon[35346]: pgmap v570: 97 pgs: 97 active+clean; 842 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 237 op/s 2024-09-19T06:21:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:26 smithi049 ceph-mon[34640]: pgmap v570: 97 pgs: 97 active+clean; 842 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 237 op/s 2024-09-19T06:21:27.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.391+0000 7fd9b66ff640 1 -- 172.21.15.5:0/948000533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b0102570 msgr2=0x7fd9b01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:27.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.391+0000 7fd9b66ff640 1 --2- 172.21.15.5:0/948000533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b0102570 0x7fd9b01029b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009930 tx=0x7fd9a402a4d0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.392+0000 7fd9b66ff640 1 -- 172.21.15.5:0/948000533 shutdown_connections 2024-09-19T06:21:27.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.392+0000 7fd9b66ff640 1 --2- 172.21.15.5:0/948000533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b0102ef0 0x7fd9b010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.392+0000 7fd9b66ff640 1 --2- 172.21.15.5:0/948000533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b0102570 0x7fd9b01029b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.392+0000 7fd9b66ff640 1 --2- 172.21.15.5:0/948000533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b0100e20 0x7fd9b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.392+0000 7fd9b66ff640 1 -- 172.21.15.5:0/948000533 >> 172.21.15.5:0/948000533 conn(0x7fd9b00fc7d0 msgr2=0x7fd9b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:27.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.392+0000 7fd9b66ff640 1 -- 172.21.15.5:0/948000533 shutdown_connections 2024-09-19T06:21:27.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.392+0000 7fd9b66ff640 1 -- 172.21.15.5:0/948000533 wait complete. 2024-09-19T06:21:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.392+0000 7fd9b66ff640 1 Processor -- start 2024-09-19T06:21:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.393+0000 7fd9b66ff640 1 -- start start 2024-09-19T06:21:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.393+0000 7fd9b66ff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b0100e20 0x7fd9b0196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.393+0000 7fd9b66ff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b0102570 0x7fd9b0196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.393+0000 7fd9b66ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b0102ef0 0x7fd9b019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.393+0000 7fd9b66ff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b010fd30 con 0x7fd9b0102ef0 2024-09-19T06:21:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.393+0000 7fd9b66ff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b010fbb0 con 0x7fd9b0100e20 2024-09-19T06:21:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.393+0000 7fd9b66ff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b010feb0 con 0x7fd9b0102570 2024-09-19T06:21:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.393+0000 7fd9b4c9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b0102ef0 0x7fd9b019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.393+0000 7fd9b4c9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b0102ef0 0x7fd9b019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39262/0 (socket says 172.21.15.5:39262) 2024-09-19T06:21:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.393+0000 7fd9b4c9c640 1 -- 172.21.15.5:0/1909737826 learned_addr learned my addr 172.21.15.5:0/1909737826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.393+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b0102570 0x7fd9b0196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.393+0000 7fd9affff640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b0100e20 0x7fd9b0196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.394+0000 7fd9af7fe640 1 -- 172.21.15.5:0/1909737826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b0100e20 msgr2=0x7fd9b0196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.394+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b0100e20 0x7fd9b0196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.394+0000 7fd9af7fe640 1 -- 172.21.15.5:0/1909737826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b0102ef0 msgr2=0x7fd9b019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.394+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b0102ef0 0x7fd9b019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.394+0000 7fd9af7fe640 1 -- 172.21.15.5:0/1909737826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b019b200 con 0x7fd9b0102570 2024-09-19T06:21:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.394+0000 7fd9b4c9c640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b0102ef0 0x7fd9b019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.394+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b0102570 0x7fd9b0196790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4031440 tx=0x7fd9a4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.394+0000 7fd9ad7fa640 1 -- 172.21.15.5:0/1909737826 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a403f070 con 0x7fd9b0102570 2024-09-19T06:21:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.394+0000 7fd9ad7fa640 1 -- 172.21.15.5:0/1909737826 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9a4003ca0 con 0x7fd9b0102570 2024-09-19T06:21:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.394+0000 7fd9ad7fa640 1 -- 172.21.15.5:0/1909737826 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4035070 con 0x7fd9b0102570 2024-09-19T06:21:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.394+0000 7fd9b66ff640 1 -- 172.21.15.5:0/1909737826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b019b490 con 0x7fd9b0102570 2024-09-19T06:21:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.395+0000 7fd9b66ff640 1 -- 172.21.15.5:0/1909737826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b01a2dd0 con 0x7fd9b0102570 2024-09-19T06:21:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.398+0000 7fd9b8ad8740 1 Processor -- start 2024-09-19T06:21:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.398+0000 7fd9b8ad8740 1 -- start start 2024-09-19T06:21:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9b8ad8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb8b40 0x55fd52cb7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9b8ad8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb75d0 0x55fd52cb7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9b8ad8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cc2370 0x55fd52cc4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9b8ad8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd52c1d2f0 con 0x55fd52cc2370 2024-09-19T06:21:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9b8ad8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fd52bc4cf0 con 0x55fd52cb75d0 2024-09-19T06:21:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9b8ad8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fd52c192b0 con 0x55fd52cb8b40 2024-09-19T06:21:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9affff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb75d0 0x55fd52cb7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9affff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb75d0 0x55fd52cb7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35204/0 (socket says 172.21.15.5:35204) 2024-09-19T06:21:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9affff640 1 -- 172.21.15.5:0/3515691020 learned_addr learned my addr 172.21.15.5:0/3515691020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9b4c9c640 1 --2- 172.21.15.5:0/3515691020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb8b40 0x55fd52cb7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/3515691020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cc2370 0x55fd52cc4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9affff640 1 -- 172.21.15.5:0/3515691020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb8b40 msgr2=0x55fd52cb7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9affff640 1 --2- 172.21.15.5:0/3515691020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb8b40 0x55fd52cb7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9affff640 1 -- 172.21.15.5:0/3515691020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cc2370 msgr2=0x55fd52cc4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.399+0000 7fd9ad7fa640 1 -- 172.21.15.5:0/1909737826 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd9a403b4f0 con 0x7fd9b0102570 2024-09-19T06:21:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.400+0000 7fd9affff640 1 --2- 172.21.15.5:0/3515691020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cc2370 0x55fd52cc4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.400+0000 7fd9affff640 1 -- 172.21.15.5:0/3515691020 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd52ba93a0 con 0x55fd52cb75d0 2024-09-19T06:21:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.400+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/3515691020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cc2370 0x55fd52cc4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.400+0000 7fd9affff640 1 --2- 172.21.15.5:0/3515691020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb75d0 0x55fd52cb7a10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000bd50 tx=0x7fd9a00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b9a940ab470da8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.400+0000 7fd9967fc640 1 -- 172.21.15.5:0/3515691020 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0005020 con 0x55fd52cb75d0 2024-09-19T06:21:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.400+0000 7fd9967fc640 1 -- 172.21.15.5:0/3515691020 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9a00047a0 con 0x55fd52cb75d0 2024-09-19T06:21:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.400+0000 7fd9967fc640 1 -- 172.21.15.5:0/3515691020 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00116a0 con 0x55fd52cb75d0 2024-09-19T06:21:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.401+0000 7fd9ad7fa640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd9980a78c0 0x7fd9980a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.401+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3515691020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb75d0 msgr2=0x55fd52cb7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.401+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3515691020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb75d0 0x55fd52cb7a10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000bd50 tx=0x7fd9a00107a0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.402+0000 7fd9b4c9c640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd9980a78c0 0x7fd9980a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.402+0000 7fd9ad7fa640 1 -- 172.21.15.5:0/1909737826 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7fd9a40f0160 con 0x7fd9b0102570 2024-09-19T06:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.402+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3515691020 shutdown_connections 2024-09-19T06:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.402+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3515691020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cc2370 0x55fd52cc4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.402+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3515691020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb75d0 0x55fd52cb7a10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.402+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3515691020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb8b40 0x55fd52cb7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.402+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3515691020 >> 172.21.15.5:0/3515691020 conn(0x55fd52c80310 msgr2=0x55fd52cb3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.402+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3515691020 shutdown_connections 2024-09-19T06:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.402+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3515691020 wait complete. 2024-09-19T06:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.402+0000 7fd9b4c9c640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd9980a78c0 0x7fd9980a9d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80099a0 tx=0x7fd9a800f040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.403+0000 7fd9b8ad8740 1 -- start start 2024-09-19T06:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.403+0000 7fd9b8ad8740 1 Processor -- start 2024-09-19T06:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.403+0000 7fd9b8ad8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb75d0 0x55fd52cc1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.403+0000 7fd9b8ad8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb8b40 0x55fd52cbefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.403+0000 7fd9b8ad8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cc2370 0x55fd52cbf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.403+0000 7fd9b8ad8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd52cc5070 con 0x55fd52cb75d0 2024-09-19T06:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.403+0000 7fd9b8ad8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fd52bd8060 con 0x55fd52cb8b40 2024-09-19T06:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.403+0000 7fd9b8ad8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fd52cc51f0 con 0x55fd52cc2370 2024-09-19T06:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9b4c9c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cc2370 0x55fd52cbf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9af7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb8b40 0x55fd52cbefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9affff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb75d0 0x55fd52cc1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9af7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb8b40 0x55fd52cbefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35208/0 (socket says 172.21.15.5:35208) 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9af7fe640 1 -- 172.21.15.5:0/3470436123 learned_addr learned my addr 172.21.15.5:0/3470436123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9b4c9c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cc2370 0x55fd52cbf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38746/0 (socket says 172.21.15.5:38746) 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9b8ad8740 10 client.0 _ll_register_callbacks cb 0x55fd5292b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9af7fe640 1 -- 172.21.15.5:0/3470436123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cc2370 msgr2=0x55fd52cbf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/3470436123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cc2370 0x55fd52cbf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9af7fe640 1 -- 172.21.15.5:0/3470436123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb75d0 msgr2=0x55fd52cc1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/3470436123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb75d0 0x55fd52cc1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9af7fe640 1 -- 172.21.15.5:0/3470436123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd5298e3c0 con 0x55fd52cb8b40 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/3470436123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb8b40 0x55fd52cbefe0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40b8e40 tx=0x7fd9a4005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=958821ffe2c487d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd98f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd98f7fe640 1 -- 172.21.15.5:0/3470436123 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4048030 con 0x55fd52cb8b40 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd98f7fe640 1 -- 172.21.15.5:0/3470436123 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7fd9a40c4ba0 con 0x55fd52cb8b40 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.404+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3470436123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a00047a0 con 0x55fd52cb8b40 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.405+0000 7fd98f7fe640 1 -- 172.21.15.5:0/3470436123 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fd9a40b8710 con 0x55fd52cb8b40 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.405+0000 7fd9b8ad8740 10 client.29420 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.405+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3470436123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55fd52cd0ad0 con 0x55fd52cb8b40 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.405+0000 7fd98f7fe640 1 -- 172.21.15.5:0/3470436123 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9a00047a0 con 0x55fd52cb8b40 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.405+0000 7fd98f7fe640 1 -- 172.21.15.5:0/3470436123 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fd9a404b060 con 0x55fd52cb8b40 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.405+0000 7fd9b8ad8740 10 client.29420 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.405+0000 7fd9b8ad8740 10 client.29420 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.405+0000 7fd9b8ad8740 20 client.29420 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.405+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3470436123 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55fd52cd1ec0 0x55fd52cd4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.405+0000 7fd9b8ad8740 4 client.29420 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.405+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3470436123 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd52c2a860 con 0x55fd52cd1ec0 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.405+0000 7fd9affff640 1 --2- 172.21.15.5:0/3470436123 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55fd52cd1ec0 0x55fd52cd4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.406+0000 7fd9affff640 1 --2- 172.21.15.5:0/3470436123 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55fd52cd1ec0 0x55fd52cd4310 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6a69a1f8d097426 server_cookie=2fee887ceacd264 in_seq=0 out_seq=0 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.406+0000 7fd98f7fe640 10 client.29420 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd98f7fe640 1 -- 172.21.15.5:0/3470436123 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd52c2a860 con 0x55fd52cd1ec0 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd98f7fe640 10 client.29420 handle_command_reply: tid=0 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd9b8ad8740 1 client.29420 shutdown 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3470436123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb8b40 msgr2=0x55fd52cbefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3470436123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb8b40 0x55fd52cbefe0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40b8e40 tx=0x7fd9a4005c90 comp rx=0 tx=0).stop 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3470436123 shutdown_connections 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3470436123 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55fd52cd1ec0 0x55fd52cd4310 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd98f7fe640 0 client.29420 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:21:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3470436123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cc2370 0x55fd52cbf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3470436123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cb8b40 0x55fd52cbefe0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3470436123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb75d0 0x55fd52cc1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3470436123 >> 172.21.15.5:0/3470436123 conn(0x55fd52c80310 msgr2=0x55fd52cb9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3470436123 shutdown_connections 2024-09-19T06:21:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3470436123 wait complete. 2024-09-19T06:21:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.407+0000 7fd9b8ad8740 20 client.29420 trim_cache size 0 max 16384 2024-09-19T06:21:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.427+0000 7fd9b8ad8740 1 Processor -- start 2024-09-19T06:21:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.427+0000 7fd9b8ad8740 1 -- start start 2024-09-19T06:21:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9b8ad8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb75d0 0x55fd52cce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9b8ad8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb8b40 0x55fd52cba120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9affff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb75d0 0x55fd52cce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9affff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb75d0 0x55fd52cce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38754/0 (socket says 172.21.15.5:38754) 2024-09-19T06:21:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9affff640 1 -- 172.21.15.5:0/3409043766 learned_addr learned my addr 172.21.15.5:0/3409043766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3409043766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cc2370 0x55fd52cba660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3409043766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01ab630 con 0x55fd52cb8b40 2024-09-19T06:21:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3409043766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01ad170 con 0x55fd52cc2370 2024-09-19T06:21:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/3409043766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb8b40 0x55fd52cba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3409043766 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01abd00 con 0x55fd52cb75d0 2024-09-19T06:21:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9b4c9c640 1 --2- 172.21.15.5:0/3409043766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cc2370 0x55fd52cba660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9affff640 1 -- 172.21.15.5:0/3409043766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cc2370 msgr2=0x55fd52cba660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9affff640 1 --2- 172.21.15.5:0/3409043766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cc2370 0x55fd52cba660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9affff640 1 -- 172.21.15.5:0/3409043766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb8b40 msgr2=0x55fd52cba120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9affff640 1 --2- 172.21.15.5:0/3409043766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb8b40 0x55fd52cba120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9affff640 1 -- 172.21.15.5:0/3409043766 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd52cc1950 con 0x55fd52cb75d0 2024-09-19T06:21:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.428+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/3409043766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb8b40 0x55fd52cba120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.429+0000 7fd9affff640 1 --2- 172.21.15.5:0/3409043766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb75d0 0x55fd52cce310 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0011010 tx=0x7fd9a0005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83de06cebbe7747 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.429+0000 7fd995ffb640 1 -- 172.21.15.5:0/3409043766 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0004220 con 0x55fd52cb75d0 2024-09-19T06:21:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.429+0000 7fd995ffb640 1 -- 172.21.15.5:0/3409043766 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9a00054e0 con 0x55fd52cb75d0 2024-09-19T06:21:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.429+0000 7fd995ffb640 1 -- 172.21.15.5:0/3409043766 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0004d10 con 0x55fd52cb75d0 2024-09-19T06:21:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.430+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3409043766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb75d0 msgr2=0x55fd52cce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.430+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3409043766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb75d0 0x55fd52cce310 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0011010 tx=0x7fd9a0005fb0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.430+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3409043766 shutdown_connections 2024-09-19T06:21:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.430+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3409043766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cc2370 0x55fd52cba660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.430+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3409043766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb8b40 0x55fd52cba120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.430+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/3409043766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb75d0 0x55fd52cce310 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.430+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3409043766 >> 172.21.15.5:0/3409043766 conn(0x55fd52c80310 msgr2=0x55fd52cbe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.430+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3409043766 shutdown_connections 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.430+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/3409043766 wait complete. 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.430+0000 7fd9b8ad8740 1 -- start start 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.431+0000 7fd9b8ad8740 1 Processor -- start 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.431+0000 7fd9b8ad8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb75d0 0x55fd52ccac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.431+0000 7fd9b8ad8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb8b40 0x55fd52cc5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.431+0000 7fd9b8ad8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cc2370 0x55fd52cc64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.431+0000 7fd9b8ad8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01acaa0 con 0x55fd52cb75d0 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.431+0000 7fd9b8ad8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01ac3d0 con 0x55fd52cc2370 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.431+0000 7fd9b8ad8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01a9630 con 0x55fd52cb8b40 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.431+0000 7fd9affff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb75d0 0x55fd52ccac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.432+0000 7fd9affff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb75d0 0x55fd52ccac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39308/0 (socket says 172.21.15.5:39308) 2024-09-19T06:21:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.432+0000 7fd9affff640 1 -- 172.21.15.5:0/2497476605 learned_addr learned my addr 172.21.15.5:0/2497476605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.432+0000 7fd9b8ad8740 10 client.0 _ll_register_callbacks cb 0x7fd9a40f3a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.431+0000 7fd9b4c9c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cc2370 0x55fd52cc64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.431+0000 7fd9af7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb8b40 0x55fd52cc5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.432+0000 7fd9affff640 1 -- 172.21.15.5:0/2497476605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb8b40 msgr2=0x55fd52cc5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.432+0000 7fd9affff640 1 --2- 172.21.15.5:0/2497476605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb8b40 0x55fd52cc5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.432+0000 7fd9affff640 1 -- 172.21.15.5:0/2497476605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cc2370 msgr2=0x55fd52cc64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.432+0000 7fd9affff640 1 --2- 172.21.15.5:0/2497476605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cc2370 0x55fd52cc64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.432+0000 7fd9affff640 1 -- 172.21.15.5:0/2497476605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd52cc6d10 con 0x55fd52cb75d0 2024-09-19T06:21:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.432+0000 7fd9affff640 1 --2- 172.21.15.5:0/2497476605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb75d0 0x55fd52ccac50 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000b5a0 tx=0x7fd9a0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e082aeba45336831 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.432+0000 7fd98f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:21:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.432+0000 7fd98f7fe640 1 -- 172.21.15.5:0/2497476605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0035060 con 0x55fd52cb75d0 2024-09-19T06:21:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd98f7fe640 1 -- 172.21.15.5:0/2497476605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7fd9a00035f0 con 0x55fd52cb75d0 2024-09-19T06:21:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/2497476605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a00054e0 con 0x55fd52cb75d0 2024-09-19T06:21:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd98f7fe640 1 -- 172.21.15.5:0/2497476605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fd9a0004430 con 0x55fd52cb75d0 2024-09-19T06:21:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd9b8ad8740 10 client.20052 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/2497476605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55fd52d3cc40 con 0x55fd52cb75d0 2024-09-19T06:21:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd98f7fe640 1 -- 172.21.15.5:0/2497476605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9a00054e0 con 0x55fd52cb75d0 2024-09-19T06:21:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd98f7fe640 1 -- 172.21.15.5:0/2497476605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fd9a0027930 con 0x55fd52cb75d0 2024-09-19T06:21:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd9b8ad8740 10 client.20052 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd9b8ad8740 10 client.20052 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:21:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd9b8ad8740 20 client.20052 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/2497476605 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55fd52d3d0b0 0x55fd52d3f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd9b8ad8740 4 client.20052 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:21:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.433+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/2497476605 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd52c24d70 con 0x55fd52d3d0b0 2024-09-19T06:21:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.434+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/2497476605 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55fd52d3d0b0 0x55fd52d3f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.434+0000 7fd98f7fe640 10 client.20052 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:21:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.434+0000 7fd9af7fe640 1 --2- 172.21.15.5:0/2497476605 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55fd52d3d0b0 0x55fd52d3f4a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4b117707a32b244 server_cookie=e14be271f4d3428a in_seq=0 out_seq=0 2024-09-19T06:21:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.434+0000 7fd98f7fe640 1 -- 172.21.15.5:0/2497476605 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd52c24d70 con 0x55fd52d3d0b0 2024-09-19T06:21:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd98f7fe640 10 client.20052 handle_command_reply: tid=0 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd9b8ad8740 1 client.20052 shutdown 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/2497476605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb75d0 msgr2=0x55fd52ccac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/2497476605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb75d0 0x55fd52ccac50 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000b5a0 tx=0x7fd9a0005c00 comp rx=0 tx=0).stop 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/2497476605 shutdown_connections 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/2497476605 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55fd52d3d0b0 0x55fd52d3f4a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/2497476605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd52cc2370 0x55fd52cc64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/2497476605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd52cb8b40 0x55fd52cc5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd9b8ad8740 1 --2- 172.21.15.5:0/2497476605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd52cb75d0 0x55fd52ccac50 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/2497476605 >> 172.21.15.5:0/2497476605 conn(0x55fd52c80310 msgr2=0x55fd52cc0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd98f7fe640 0 client.20052 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/2497476605 shutdown_connections 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd9b8ad8740 1 -- 172.21.15.5:0/2497476605 wait complete. 2024-09-19T06:21:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.435+0000 7fd9b8ad8740 20 client.20052 trim_cache size 0 max 16384 2024-09-19T06:21:27.442 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:21:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.436+0000 7fd9b66ff640 1 -- 172.21.15.5:0/1909737826 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd9980a78c0 msgr2=0x7fd9980a9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.436+0000 7fd9b66ff640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd9980a78c0 0x7fd9980a9d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80099a0 tx=0x7fd9a800f040 comp rx=0 tx=0).stop 2024-09-19T06:21:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.436+0000 7fd9b66ff640 1 -- 172.21.15.5:0/1909737826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b0102570 msgr2=0x7fd9b0196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.436+0000 7fd9b66ff640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b0102570 0x7fd9b0196790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4031440 tx=0x7fd9a4004270 comp rx=0 tx=0).stop 2024-09-19T06:21:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.436+0000 7fd9b66ff640 1 -- 172.21.15.5:0/1909737826 shutdown_connections 2024-09-19T06:21:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.436+0000 7fd9b66ff640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b0102ef0 0x7fd9b019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.436+0000 7fd9b66ff640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd9980a78c0 0x7fd9980a9d80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.436+0000 7fd9b66ff640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b0102570 0x7fd9b0196790 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.436+0000 7fd9b66ff640 1 --2- 172.21.15.5:0/1909737826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b0100e20 0x7fd9b0196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:27.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.436+0000 7fd9b66ff640 1 -- 172.21.15.5:0/1909737826 >> 172.21.15.5:0/1909737826 conn(0x7fd9b00fc7d0 msgr2=0x7fd9b0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:27.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.437+0000 7fd9b66ff640 1 -- 172.21.15.5:0/1909737826 shutdown_connections 2024-09-19T06:21:27.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:27.437+0000 7fd9b66ff640 1 -- 172.21.15.5:0/1909737826 wait complete. 2024-09-19T06:21:27.561 INFO:tasks.workunit.client.0.smithi005.stdout: 10 48190 8.77 MB/sec execute 171 sec latency 175.276 ms 2024-09-19T06:21:28.225 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:21:28.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:28 smithi005 ceph-mon[30955]: pgmap v571: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.4 MiB/s wr, 360 op/s 2024-09-19T06:21:28.561 INFO:tasks.workunit.client.0.smithi005.stdout: 10 48360 8.76 MB/sec execute 172 sec latency 58.498 ms 2024-09-19T06:21:28.604 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:21:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:28 smithi157 ceph-mon[35346]: pgmap v571: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.4 MiB/s wr, 360 op/s 2024-09-19T06:21:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:28 smithi049 ceph-mon[34640]: pgmap v571: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.4 MiB/s wr, 360 op/s 2024-09-19T06:21:29.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.207+0000 7f0acbce1640 1 -- 172.21.15.5:0/551532663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ac4106a90 msgr2=0x7f0ac4106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:29.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.207+0000 7f0acbce1640 1 --2- 172.21.15.5:0/551532663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ac4106a90 0x7f0ac4106e70 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009930 tx=0x7f0ac002a420 comp rx=0 tx=0).stop 2024-09-19T06:21:29.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.207+0000 7f0acbce1640 1 -- 172.21.15.5:0/551532663 shutdown_connections 2024-09-19T06:21:29.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.207+0000 7f0acbce1640 1 --2- 172.21.15.5:0/551532663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ac4101430 0x7f0ac410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.207+0000 7f0acbce1640 1 --2- 172.21.15.5:0/551532663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac4100a90 0x7f0ac4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.207+0000 7f0acbce1640 1 --2- 172.21.15.5:0/551532663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ac4106a90 0x7f0ac4106e70 secure :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009930 tx=0x7f0ac002a420 comp rx=0 tx=0).stop 2024-09-19T06:21:29.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.207+0000 7f0acbce1640 1 -- 172.21.15.5:0/551532663 >> 172.21.15.5:0/551532663 conn(0x7f0ac40fc7d0 msgr2=0x7f0ac40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:29.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.207+0000 7f0acbce1640 1 -- 172.21.15.5:0/551532663 shutdown_connections 2024-09-19T06:21:29.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.208+0000 7f0acbce1640 1 -- 172.21.15.5:0/551532663 wait complete. 2024-09-19T06:21:29.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.209+0000 7f0acbce1640 1 Processor -- start 2024-09-19T06:21:29.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.209+0000 7f0acbce1640 1 -- start start 2024-09-19T06:21:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0acbce1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ac4100a90 0x7f0ac4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0acbce1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac4101430 0x7f0ac4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0acbce1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ac419ce20 0x7f0ac419d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0acbce1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4110d40 con 0x7f0ac4100a90 2024-09-19T06:21:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0acbce1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4110bc0 con 0x7f0ac419ce20 2024-09-19T06:21:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0acbce1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4110ec0 con 0x7f0ac4101430 2024-09-19T06:21:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0ac9a7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ac4100a90 0x7f0ac4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0aca27e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ac419ce20 0x7f0ac419d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0ac9a7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ac4100a90 0x7f0ac4198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39330/0 (socket says 172.21.15.5:39330) 2024-09-19T06:21:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0aca27e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ac419ce20 0x7f0ac419d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35248/0 (socket says 172.21.15.5:35248) 2024-09-19T06:21:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0ac9a7d640 1 -- 172.21.15.5:0/2085521142 learned_addr learned my addr 172.21.15.5:0/2085521142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0ac927c640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac4101430 0x7f0ac4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0aca27e640 1 -- 172.21.15.5:0/2085521142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac4101430 msgr2=0x7f0ac4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0aca27e640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac4101430 0x7f0ac4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0aca27e640 1 -- 172.21.15.5:0/2085521142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ac4100a90 msgr2=0x7f0ac4198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0aca27e640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ac4100a90 0x7f0ac4198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0aca27e640 1 -- 172.21.15.5:0/2085521142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac419d8e0 con 0x7f0ac419ce20 2024-09-19T06:21:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.210+0000 7f0ac927c640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac4101430 0x7f0ac4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:21:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.211+0000 7f0aca27e640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ac419ce20 0x7f0ac419d2d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009fd0 tx=0x7f0abc00be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.211+0000 7f0abaffd640 1 -- 172.21.15.5:0/2085521142 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc004d20 con 0x7f0ac419ce20 2024-09-19T06:21:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.211+0000 7f0abaffd640 1 -- 172.21.15.5:0/2085521142 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0abc005020 con 0x7f0ac419ce20 2024-09-19T06:21:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.211+0000 7f0abaffd640 1 -- 172.21.15.5:0/2085521142 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc00f6e0 con 0x7f0ac419ce20 2024-09-19T06:21:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.211+0000 7f0acbce1640 1 -- 172.21.15.5:0/2085521142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac419dbd0 con 0x7f0ac419ce20 2024-09-19T06:21:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.211+0000 7f0acbce1640 1 -- 172.21.15.5:0/2085521142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac40fd0f0 con 0x7f0ac419ce20 2024-09-19T06:21:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.214+0000 7f0abaffd640 1 -- 172.21.15.5:0/2085521142 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0abc018070 con 0x7f0ac419ce20 2024-09-19T06:21:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.215+0000 7f0ace0ba740 1 Processor -- start 2024-09-19T06:21:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.215+0000 7f0ace0ba740 1 -- start start 2024-09-19T06:21:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.215+0000 7f0ace0ba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816db40 0x55b9b816c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.215+0000 7f0ace0ba740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b816ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.215+0000 7f0ace0ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b8177370 0x55b9b8179760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.215+0000 7f0ace0ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b9b80d22f0 con 0x55b9b8177370 2024-09-19T06:21:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.215+0000 7f0ace0ba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b9b8079cf0 con 0x55b9b816db40 2024-09-19T06:21:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.215+0000 7f0ace0ba740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b9b80ce2b0 con 0x55b9b816c5d0 2024-09-19T06:21:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.216+0000 7f0ac9a7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b816ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.216+0000 7f0ac9a7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b816ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38790/0 (socket says 172.21.15.5:38790) 2024-09-19T06:21:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.216+0000 7f0ac9a7d640 1 -- 172.21.15.5:0/3146885728 learned_addr learned my addr 172.21.15.5:0/3146885728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.216+0000 7f0ac927c640 1 --2- 172.21.15.5:0/3146885728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816db40 0x55b9b816c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.216+0000 7f0ac927c640 1 --2- 172.21.15.5:0/3146885728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b8177370 0x55b9b8179760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.216+0000 7f0ac9a7d640 1 -- 172.21.15.5:0/3146885728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816db40 msgr2=0x55b9b816c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.216+0000 7f0ac9a7d640 1 --2- 172.21.15.5:0/3146885728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816db40 0x55b9b816c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.216+0000 7f0ac9a7d640 1 -- 172.21.15.5:0/3146885728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b8177370 msgr2=0x55b9b8179760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.216+0000 7f0ac9a7d640 1 --2- 172.21.15.5:0/3146885728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b8177370 0x55b9b8179760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.216+0000 7f0ac9a7d640 1 -- 172.21.15.5:0/3146885728 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b805e3a0 con 0x55b9b816c5d0 2024-09-19T06:21:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.216+0000 7f0ac927c640 1 --2- 172.21.15.5:0/3146885728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b8177370 0x55b9b8179760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:21:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.216+0000 7f0abaffd640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0ab00a3430 0x7f0ab00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.217+0000 7f0ac927c640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0ab00a3430 0x7f0ab00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.217+0000 7f0ac9a7d640 1 --2- 172.21.15.5:0/3146885728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b816ca10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0033000 tx=0x7f0ac0005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ec2d01474dd295 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.217+0000 7f0abaffd640 1 -- 172.21.15.5:0/2085521142 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f0abc0cfd10 con 0x7f0ac419ce20 2024-09-19T06:21:29.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.217+0000 7f0aaffff640 1 -- 172.21.15.5:0/3146885728 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0035020 con 0x55b9b816c5d0 2024-09-19T06:21:29.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.217+0000 7f0aaffff640 1 -- 172.21.15.5:0/3146885728 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ac0040b70 con 0x55b9b816c5d0 2024-09-19T06:21:29.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.217+0000 7f0aaffff640 1 -- 172.21.15.5:0/3146885728 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00047a0 con 0x55b9b816c5d0 2024-09-19T06:21:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.218+0000 7f0ace0ba740 1 -- 172.21.15.5:0/3146885728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 msgr2=0x55b9b816ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.218+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/3146885728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b816ca10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0033000 tx=0x7f0ac0005d50 comp rx=0 tx=0).stop 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.218+0000 7f0ac927c640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0ab00a3430 0x7f0ab00a58f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55b9b816d070 tx=0x7f0ab4004730 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.218+0000 7f0ace0ba740 1 -- 172.21.15.5:0/3146885728 shutdown_connections 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.218+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/3146885728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b8177370 0x55b9b8179760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.218+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/3146885728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b816ca10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.218+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/3146885728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816db40 0x55b9b816c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.218+0000 7f0ace0ba740 1 -- 172.21.15.5:0/3146885728 >> 172.21.15.5:0/3146885728 conn(0x55b9b8135310 msgr2=0x55b9b8168c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.218+0000 7f0ace0ba740 1 -- 172.21.15.5:0/3146885728 shutdown_connections 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.218+0000 7f0ace0ba740 1 -- 172.21.15.5:0/3146885728 wait complete. 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.219+0000 7f0ace0ba740 1 -- start start 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.219+0000 7f0ace0ba740 1 Processor -- start 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ace0ba740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b8176d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ace0ba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816db40 0x55b9b8173ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ace0ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b8177370 0x55b9b81745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ace0ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b9b8179fb0 con 0x55b9b8177370 2024-09-19T06:21:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ace0ba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b9b808d060 con 0x55b9b816db40 2024-09-19T06:21:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ace0ba740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b9b817a130 con 0x55b9b816c5d0 2024-09-19T06:21:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ace0ba740 10 client.0 _ll_register_callbacks cb 0x55b9b7de08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ac927c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816db40 0x55b9b8173ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ac9a7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b8176d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ac9a7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b8176d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38800/0 (socket says 172.21.15.5:38800) 2024-09-19T06:21:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ac9a7d640 1 -- 172.21.15.5:0/2372396379 learned_addr learned my addr 172.21.15.5:0/2372396379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ac927c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816db40 0x55b9b8173ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35260/0 (socket says 172.21.15.5:35260) 2024-09-19T06:21:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0aca27e640 1 --2- 172.21.15.5:0/2372396379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b8177370 0x55b9b81745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ac927c640 1 -- 172.21.15.5:0/2372396379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 msgr2=0x55b9b8176d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:29.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ac927c640 1 --2- 172.21.15.5:0/2372396379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b8176d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ac927c640 1 -- 172.21.15.5:0/2372396379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b8177370 msgr2=0x55b9b81745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:29.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ac927c640 1 --2- 172.21.15.5:0/2372396379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b8177370 0x55b9b81745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.220+0000 7f0ac927c640 1 -- 172.21.15.5:0/2372396379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b7e433c0 con 0x55b9b816db40 2024-09-19T06:21:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.221+0000 7f0ac927c640 1 --2- 172.21.15.5:0/2372396379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816db40 0x55b9b8173ff0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4010c00 tx=0x7f0ab4009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0038bbb118accf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.221+0000 7f0aad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:21:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.221+0000 7f0aad7fa640 1 -- 172.21.15.5:0/2372396379 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400fcd0 con 0x55b9b816db40 2024-09-19T06:21:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.221+0000 7f0aad7fa640 1 -- 172.21.15.5:0/2372396379 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f0ab4029470 con 0x55b9b816db40 2024-09-19T06:21:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.221+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2372396379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ac0040b70 con 0x55b9b816db40 2024-09-19T06:21:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.221+0000 7f0aad7fa640 1 -- 172.21.15.5:0/2372396379 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f0ab400fe70 con 0x55b9b816db40 2024-09-19T06:21:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.221+0000 7f0ace0ba740 10 client.29435 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.221+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2372396379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55b9b8185a80 con 0x55b9b816db40 2024-09-19T06:21:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.222+0000 7f0aad7fa640 1 -- 172.21.15.5:0/2372396379 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ac0040b70 con 0x55b9b816db40 2024-09-19T06:21:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.222+0000 7f0aad7fa640 1 -- 172.21.15.5:0/2372396379 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f0ab401f7d0 con 0x55b9b816db40 2024-09-19T06:21:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.222+0000 7f0ace0ba740 10 client.29435 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.222+0000 7f0ace0ba740 10 client.29435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.222+0000 7f0ace0ba740 20 client.29435 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.222+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/2372396379 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b9b8186f90 0x55b9b81893b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.222+0000 7f0ace0ba740 4 client.29435 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.222+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2372396379 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9b80df860 con 0x55b9b8186f90 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.222+0000 7f0ac9a7d640 1 --2- 172.21.15.5:0/2372396379 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b9b8186f90 0x55b9b81893b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.222+0000 7f0aad7fa640 10 client.29435 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.222+0000 7f0ac9a7d640 1 --2- 172.21.15.5:0/2372396379 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b9b8186f90 0x55b9b81893b0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a462c61d0ff0d131 server_cookie=699df912de206d13 in_seq=0 out_seq=0 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.223+0000 7f0aad7fa640 1 -- 172.21.15.5:0/2372396379 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9b80df860 con 0x55b9b8186f90 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.223+0000 7f0aad7fa640 10 client.29435 handle_command_reply: tid=0 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.223+0000 7f0ace0ba740 1 client.29435 shutdown 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.223+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2372396379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816db40 msgr2=0x55b9b8173ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.223+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/2372396379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816db40 0x55b9b8173ff0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4010c00 tx=0x7f0ab4009530 comp rx=0 tx=0).stop 2024-09-19T06:21:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.224+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2372396379 shutdown_connections 2024-09-19T06:21:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.224+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/2372396379 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b9b8186f90 0x55b9b81893b0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.224+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/2372396379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b8177370 0x55b9b81745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.224+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/2372396379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816db40 0x55b9b8173ff0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.224+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/2372396379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b8176d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.224+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2372396379 >> 172.21.15.5:0/2372396379 conn(0x55b9b8135310 msgr2=0x55b9b816e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.224+0000 7f0aad7fa640 0 client.29435 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.224+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2372396379 shutdown_connections 2024-09-19T06:21:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.224+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2372396379 wait complete. 2024-09-19T06:21:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.225+0000 7f0ace0ba740 20 client.29435 trim_cache size 0 max 16384 2024-09-19T06:21:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.244+0000 7f0ace0ba740 1 Processor -- start 2024-09-19T06:21:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.244+0000 7f0ace0ba740 1 -- start start 2024-09-19T06:21:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.244+0000 7f0ace0ba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816c5d0 0x55b9b8168e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.244+0000 7f0ace0ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 0x55b9b8183250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ace0ba740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b8177370 0x55b9b81738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ace0ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41abc80 con 0x55b9b816db40 2024-09-19T06:21:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ace0ba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41ad7c0 con 0x55b9b816c5d0 2024-09-19T06:21:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ace0ba740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41ac350 con 0x55b9b8177370 2024-09-19T06:21:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ac927c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 0x55b9b8183250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ac927c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 0x55b9b8183250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39360/0 (socket says 172.21.15.5:39360) 2024-09-19T06:21:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ac927c640 1 -- 172.21.15.5:0/618822920 learned_addr learned my addr 172.21.15.5:0/618822920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ac9a7d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816c5d0 0x55b9b8168e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0aca27e640 1 --2- 172.21.15.5:0/618822920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b8177370 0x55b9b81738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ac927c640 1 -- 172.21.15.5:0/618822920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b8177370 msgr2=0x55b9b81738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ac927c640 1 --2- 172.21.15.5:0/618822920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b8177370 0x55b9b81738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ac927c640 1 -- 172.21.15.5:0/618822920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816c5d0 msgr2=0x55b9b8168e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ac927c640 1 --2- 172.21.15.5:0/618822920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816c5d0 0x55b9b8168e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ac927c640 1 -- 172.21.15.5:0/618822920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b8172fa0 con 0x55b9b816db40 2024-09-19T06:21:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.245+0000 7f0ac927c640 1 --2- 172.21.15.5:0/618822920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 0x55b9b8183250 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40092e0 tx=0x7f0ab400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d56e1005042a82b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.246+0000 7f0aaf7fe640 1 -- 172.21.15.5:0/618822920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab403e070 con 0x55b9b816db40 2024-09-19T06:21:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.246+0000 7f0aaf7fe640 1 -- 172.21.15.5:0/618822920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab4024070 con 0x55b9b816db40 2024-09-19T06:21:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.246+0000 7f0aaf7fe640 1 -- 172.21.15.5:0/618822920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4035040 con 0x55b9b816db40 2024-09-19T06:21:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.247+0000 7f0ace0ba740 1 -- 172.21.15.5:0/618822920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 msgr2=0x55b9b8183250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.247+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/618822920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 0x55b9b8183250 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40092e0 tx=0x7f0ab400f790 comp rx=0 tx=0).stop 2024-09-19T06:21:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.247+0000 7f0ace0ba740 1 -- 172.21.15.5:0/618822920 shutdown_connections 2024-09-19T06:21:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.247+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/618822920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b8177370 0x55b9b81738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.247+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/618822920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 0x55b9b8183250 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.247+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/618822920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b816c5d0 0x55b9b8168e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.247+0000 7f0ace0ba740 1 -- 172.21.15.5:0/618822920 >> 172.21.15.5:0/618822920 conn(0x55b9b8135310 msgr2=0x55b9b8168700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.247+0000 7f0ace0ba740 1 -- 172.21.15.5:0/618822920 shutdown_connections 2024-09-19T06:21:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.247+0000 7f0ace0ba740 1 -- 172.21.15.5:0/618822920 wait complete. 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.247+0000 7f0ace0ba740 1 -- start start 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.248+0000 7f0ace0ba740 1 Processor -- start 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.248+0000 7f0ace0ba740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b817f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.248+0000 7f0ace0ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 0x55b9b817ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.248+0000 7f0ace0ba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b8177370 0x55b9b817b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.248+0000 7f0ace0ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41ad0f0 con 0x55b9b816db40 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.248+0000 7f0ace0ba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41aca20 con 0x55b9b8177370 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.248+0000 7f0ace0ba740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41a9c80 con 0x55b9b816c5d0 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.248+0000 7f0ace0ba740 10 client.0 _ll_register_callbacks cb 0x7f0ab4009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.248+0000 7f0ac9a7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b817f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.248+0000 7f0ac9a7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b817f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38826/0 (socket says 172.21.15.5:38826) 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.248+0000 7f0ac9a7d640 1 -- 172.21.15.5:0/2346482267 learned_addr learned my addr 172.21.15.5:0/2346482267 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0ac927c640 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 0x55b9b817ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0aca27e640 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b8177370 0x55b9b817b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0ac9a7d640 1 -- 172.21.15.5:0/2346482267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b8177370 msgr2=0x55b9b817b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0ac9a7d640 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b8177370 0x55b9b817b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0ac9a7d640 1 -- 172.21.15.5:0/2346482267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 msgr2=0x55b9b817ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0ac9a7d640 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 0x55b9b817ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0ac9a7d640 1 -- 172.21.15.5:0/2346482267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b817b750 con 0x55b9b816c5d0 2024-09-19T06:21:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0ac927c640 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 0x55b9b817ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:21:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0ac9a7d640 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b817f8f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0004730 tx=0x7f0ac0004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f3796f4b852cc8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0aad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:21:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0aad7fa640 1 -- 172.21.15.5:0/2346482267 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0058070 con 0x55b9b816c5d0 2024-09-19T06:21:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0aad7fa640 1 -- 172.21.15.5:0/2346482267 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f0ac0005230 con 0x55b9b816c5d0 2024-09-19T06:21:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.249+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2346482267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab4024070 con 0x55b9b816c5d0 2024-09-19T06:21:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.250+0000 7f0aad7fa640 1 -- 172.21.15.5:0/2346482267 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f0ac0066050 con 0x55b9b816c5d0 2024-09-19T06:21:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.250+0000 7f0ace0ba740 10 client.29380 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.250+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2346482267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55b9b81f1a90 con 0x55b9b816c5d0 2024-09-19T06:21:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.250+0000 7f0aad7fa640 1 -- 172.21.15.5:0/2346482267 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab4024070 con 0x55b9b816c5d0 2024-09-19T06:21:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.250+0000 7f0aad7fa640 1 -- 172.21.15.5:0/2346482267 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f0ac0031760 con 0x55b9b816c5d0 2024-09-19T06:21:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.250+0000 7f0ace0ba740 10 client.29380 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.250+0000 7f0ace0ba740 10 client.29380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.250+0000 7f0ace0ba740 20 client.29380 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.250+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b9b81f1d80 0x55b9b81f4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.250+0000 7f0ace0ba740 4 client.29380 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.250+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2346482267 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9b80d9d70 con 0x55b9b81f1d80 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.251+0000 7f0ac927c640 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b9b81f1d80 0x55b9b81f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.251+0000 7f0aad7fa640 10 client.29380 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.251+0000 7f0ac927c640 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b9b81f1d80 0x55b9b81f4170 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27b66323cfb2cf6 server_cookie=586f1e80ecbc3599 in_seq=0 out_seq=0 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.251+0000 7f0aad7fa640 1 -- 172.21.15.5:0/2346482267 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9b80d9d70 con 0x55b9b81f1d80 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.251+0000 7f0aad7fa640 10 client.29380 handle_command_reply: tid=0 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.251+0000 7f0ace0ba740 1 client.29380 shutdown 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.251+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2346482267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 msgr2=0x55b9b817f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.251+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b817f8f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0004730 tx=0x7f0ac0004610 comp rx=0 tx=0).stop 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.252+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2346482267 shutdown_connections 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.252+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b9b81f1d80 0x55b9b81f4170 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.252+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9b8177370 0x55b9b817b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.252+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9b816db40 0x55b9b817ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.252+0000 7f0ace0ba740 1 --2- 172.21.15.5:0/2346482267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b9b816c5d0 0x55b9b817f8f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.252+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2346482267 >> 172.21.15.5:0/2346482267 conn(0x55b9b8135310 msgr2=0x55b9b8175850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.252+0000 7f0aad7fa640 0 client.29380 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.252+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2346482267 shutdown_connections 2024-09-19T06:21:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.252+0000 7f0ace0ba740 1 -- 172.21.15.5:0/2346482267 wait complete. 2024-09-19T06:21:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.252+0000 7f0ace0ba740 20 client.29380 trim_cache size 0 max 16384 2024-09-19T06:21:29.258 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:21:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.253+0000 7f0acbce1640 1 -- 172.21.15.5:0/2085521142 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0ab00a3430 msgr2=0x7f0ab00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.253+0000 7f0acbce1640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0ab00a3430 0x7f0ab00a58f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55b9b816d070 tx=0x7f0ab4004730 comp rx=0 tx=0).stop 2024-09-19T06:21:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.253+0000 7f0acbce1640 1 -- 172.21.15.5:0/2085521142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ac419ce20 msgr2=0x7f0ac419d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.253+0000 7f0acbce1640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ac419ce20 0x7f0ac419d2d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009fd0 tx=0x7f0abc00be80 comp rx=0 tx=0).stop 2024-09-19T06:21:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.253+0000 7f0acbce1640 1 -- 172.21.15.5:0/2085521142 shutdown_connections 2024-09-19T06:21:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.253+0000 7f0acbce1640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ac419ce20 0x7f0ac419d2d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.253+0000 7f0acbce1640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0ab00a3430 0x7f0ab00a58f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.253+0000 7f0acbce1640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac4101430 0x7f0ac4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.253+0000 7f0acbce1640 1 --2- 172.21.15.5:0/2085521142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ac4100a90 0x7f0ac4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.253+0000 7f0acbce1640 1 -- 172.21.15.5:0/2085521142 >> 172.21.15.5:0/2085521142 conn(0x7f0ac40fc7d0 msgr2=0x7f0ac406ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:29.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.253+0000 7f0acbce1640 1 -- 172.21.15.5:0/2085521142 shutdown_connections 2024-09-19T06:21:29.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:29.253+0000 7f0acbce1640 1 -- 172.21.15.5:0/2085521142 wait complete. 2024-09-19T06:21:29.561 INFO:tasks.workunit.client.0.smithi005.stdout: 10 48502 8.75 MB/sec execute 173 sec latency 84.641 ms 2024-09-19T06:21:30.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 48656 8.73 MB/sec execute 174 sec latency 140.033 ms 2024-09-19T06:21:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:30 smithi157 ceph-mon[35346]: pgmap v572: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 245 op/s 2024-09-19T06:21:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:30 smithi005 ceph-mon[30955]: pgmap v572: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 245 op/s 2024-09-19T06:21:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:30 smithi049 ceph-mon[34640]: pgmap v572: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 245 op/s 2024-09-19T06:21:31.065 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:21:31.066 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:21:31.442 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:21:31.561 INFO:tasks.workunit.client.0.smithi005.stdout: 10 48828 8.74 MB/sec execute 175 sec latency 58.978 ms 2024-09-19T06:21:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.544+0000 7f61e706e640 1 -- 172.21.15.5:0/320129335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e0102a50 msgr2=0x7f61e0102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.544+0000 7f61e706e640 1 --2- 172.21.15.5:0/320129335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e0102a50 0x7f61e0102e30 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009900 tx=0x7f61d402a410 comp rx=0 tx=0).stop 2024-09-19T06:21:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.544+0000 7f61e706e640 1 -- 172.21.15.5:0/320129335 shutdown_connections 2024-09-19T06:21:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.544+0000 7f61e706e640 1 --2- 172.21.15.5:0/320129335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0100c80 0x7f61e0110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.544+0000 7f61e706e640 1 --2- 172.21.15.5:0/320129335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0100100 0x7f61e0100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.544+0000 7f61e706e640 1 --2- 172.21.15.5:0/320129335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e0102a50 0x7f61e0102e30 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.544+0000 7f61e706e640 1 -- 172.21.15.5:0/320129335 >> 172.21.15.5:0/320129335 conn(0x7f61e0077230 msgr2=0x7f61e00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:32.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.544+0000 7f61e706e640 1 -- 172.21.15.5:0/320129335 shutdown_connections 2024-09-19T06:21:32.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.544+0000 7f61e706e640 1 -- 172.21.15.5:0/320129335 wait complete. 2024-09-19T06:21:32.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.545+0000 7f61e706e640 1 Processor -- start 2024-09-19T06:21:32.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.545+0000 7f61e706e640 1 -- start start 2024-09-19T06:21:32.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61e706e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0100100 0x7f61e019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61e706e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0100c80 0x7f61e019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61e706e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e0102a50 0x7f61e019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61e706e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0113fb0 con 0x7f61e0102a50 2024-09-19T06:21:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61e706e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0113e30 con 0x7f61e0100100 2024-09-19T06:21:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61e706e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0114130 con 0x7f61e0100c80 2024-09-19T06:21:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61dffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0100c80 0x7f61e019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61e560b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e0102a50 0x7f61e019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61dffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0100c80 0x7f61e019aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38836/0 (socket says 172.21.15.5:38836) 2024-09-19T06:21:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61dffff640 1 -- 172.21.15.5:0/2206581279 learned_addr learned my addr 172.21.15.5:0/2206581279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61e4e0a640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0100100 0x7f61e019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61dffff640 1 -- 172.21.15.5:0/2206581279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0100100 msgr2=0x7f61e019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61dffff640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0100100 0x7f61e019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61dffff640 1 -- 172.21.15.5:0/2206581279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e0102a50 msgr2=0x7f61e019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:32.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.546+0000 7f61dffff640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e0102a50 0x7f61e019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.547+0000 7f61dffff640 1 -- 172.21.15.5:0/2206581279 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e019f520 con 0x7f61e0100c80 2024-09-19T06:21:32.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.547+0000 7f61e4e0a640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0100100 0x7f61e019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:21:32.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.547+0000 7f61e560b640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e0102a50 0x7f61e019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:32.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.547+0000 7f61dffff640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0100c80 0x7f61e019aab0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00e9f0 tx=0x7f61cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:32.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.547+0000 7f61ddffb640 1 -- 172.21.15.5:0/2206581279 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00cd80 con 0x7f61e0100c80 2024-09-19T06:21:32.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.547+0000 7f61ddffb640 1 -- 172.21.15.5:0/2206581279 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61cc0108e0 con 0x7f61e0100c80 2024-09-19T06:21:32.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.547+0000 7f61e706e640 1 -- 172.21.15.5:0/2206581279 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e019f810 con 0x7f61e0100c80 2024-09-19T06:21:32.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.547+0000 7f61ddffb640 1 -- 172.21.15.5:0/2206581279 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc018920 con 0x7f61e0100c80 2024-09-19T06:21:32.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.548+0000 7f61e706e640 1 -- 172.21.15.5:0/2206581279 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e01a70f0 con 0x7f61e0100c80 2024-09-19T06:21:32.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.550+0000 7f61ddffb640 1 -- 172.21.15.5:0/2206581279 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f61cc010a90 con 0x7f61e0100c80 2024-09-19T06:21:32.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61e9447740 1 Processor -- start 2024-09-19T06:21:32.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61e9447740 1 -- start start 2024-09-19T06:21:32.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61e9447740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 0x5620dd78b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61e9447740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78bc90 0x5620dd796010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61e9447740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 0x5620dd798940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61e9447740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620dd6e8280 con 0x5620dd78bc90 2024-09-19T06:21:32.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61e9447740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5620dd68fb10 con 0x5620dd78b370 2024-09-19T06:21:32.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61e9447740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5620dd6e4240 con 0x5620dd796550 2024-09-19T06:21:32.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61e560b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 0x5620dd78b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61ddffb640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61d00a3430 0x7f61d00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61e560b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 0x5620dd78b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35298/0 (socket says 172.21.15.5:35298) 2024-09-19T06:21:32.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61e560b640 1 -- 172.21.15.5:0/316157669 learned_addr learned my addr 172.21.15.5:0/316157669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:32.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61e560b640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61d00a3430 0x7f61d00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.551+0000 7f61ddffb640 1 -- 172.21.15.5:0/2206581279 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f61cc0d0070 con 0x7f61e0100c80 2024-09-19T06:21:32.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.552+0000 7f61e560b640 1 -- 172.21.15.5:0/316157669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 msgr2=0x5620dd798940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:21:32.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.552+0000 7f61e560b640 1 --2- 172.21.15.5:0/316157669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 0x5620dd798940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.552+0000 7f61e560b640 1 -- 172.21.15.5:0/316157669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78bc90 msgr2=0x5620dd796010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:32.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.552+0000 7f61e560b640 1 --2- 172.21.15.5:0/316157669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78bc90 0x5620dd796010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.552+0000 7f61e560b640 1 -- 172.21.15.5:0/316157669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620dd69c1f0 con 0x5620dd78b370 2024-09-19T06:21:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.552+0000 7f61e560b640 1 --2- 172.21.15.5:0/316157669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 0x5620dd78b750 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f61d8012de0 tx=0x7f61d80147e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e183f67db74b323c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.552+0000 7f61caffd640 1 -- 172.21.15.5:0/316157669 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8024070 con 0x5620dd78b370 2024-09-19T06:21:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.552+0000 7f61e560b640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61d00a3430 0x7f61d00a58f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f61d8021000 tx=0x7f61d801f000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.552+0000 7f61caffd640 1 -- 172.21.15.5:0/316157669 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61d801fb30 con 0x5620dd78b370 2024-09-19T06:21:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.553+0000 7f61caffd640 1 -- 172.21.15.5:0/316157669 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80117c0 con 0x5620dd78b370 2024-09-19T06:21:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.553+0000 7f61e9447740 1 -- 172.21.15.5:0/316157669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 msgr2=0x5620dd78b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.553+0000 7f61e9447740 1 --2- 172.21.15.5:0/316157669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 0x5620dd78b750 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f61d8012de0 tx=0x7f61d80147e0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.554+0000 7f61e9447740 1 -- 172.21.15.5:0/316157669 shutdown_connections 2024-09-19T06:21:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.554+0000 7f61e9447740 1 --2- 172.21.15.5:0/316157669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 0x5620dd798940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.554+0000 7f61e9447740 1 --2- 172.21.15.5:0/316157669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78bc90 0x5620dd796010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.554+0000 7f61e9447740 1 --2- 172.21.15.5:0/316157669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 0x5620dd78b750 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.554+0000 7f61e9447740 1 -- 172.21.15.5:0/316157669 >> 172.21.15.5:0/316157669 conn(0x5620dd787900 msgr2=0x5620dd787ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:32.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.554+0000 7f61e9447740 1 -- 172.21.15.5:0/316157669 shutdown_connections 2024-09-19T06:21:32.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.554+0000 7f61e9447740 1 -- 172.21.15.5:0/316157669 wait complete. 2024-09-19T06:21:32.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.554+0000 7f61e9447740 1 -- start start 2024-09-19T06:21:32.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.554+0000 7f61e9447740 1 Processor -- start 2024-09-19T06:21:32.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.555+0000 7f61e9447740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 0x5620dd793450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.555+0000 7f61e4e0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 0x5620dd793450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.555+0000 7f61e4e0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 0x5620dd793450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39420/0 (socket says 172.21.15.5:39420) 2024-09-19T06:21:32.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.555+0000 7f61e4e0a640 1 -- 172.21.15.5:0/3850701139 learned_addr learned my addr 172.21.15.5:0/3850701139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:32.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.555+0000 7f61e9447740 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78bc90 0x5620dd793990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.555+0000 7f61e9447740 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 0x5620dd793ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.555+0000 7f61e9447740 1 -- 172.21.15.5:0/3850701139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620dd7991c0 con 0x5620dd78b370 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.555+0000 7f61e9447740 1 -- 172.21.15.5:0/3850701139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5620dd6a2f10 con 0x5620dd78bc90 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.555+0000 7f61e9447740 1 -- 172.21.15.5:0/3850701139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5620dd799340 con 0x5620dd796550 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.555+0000 7f61dffff640 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78bc90 0x5620dd793990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.556+0000 7f61e560b640 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 0x5620dd793ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.556+0000 7f61e4e0a640 1 -- 172.21.15.5:0/3850701139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 msgr2=0x5620dd793ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.556+0000 7f61e4e0a640 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 0x5620dd793ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.556+0000 7f61e4e0a640 1 -- 172.21.15.5:0/3850701139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78bc90 msgr2=0x5620dd793990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.556+0000 7f61e4e0a640 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78bc90 0x5620dd793990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.556+0000 7f61e4e0a640 1 -- 172.21.15.5:0/3850701139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620dd4596a0 con 0x5620dd78b370 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.556+0000 7f61dffff640 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78bc90 0x5620dd793990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:32.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.556+0000 7f61e4e0a640 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 0x5620dd793450 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009900 tx=0x7f61d4032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7990e06e4a8a2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:32.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.556+0000 7f61bffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:21:32.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.556+0000 7f61e9447740 10 client.0 _ll_register_callbacks cb 0x5620dd66ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:32.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.557+0000 7f61bffff640 1 -- 172.21.15.5:0/3850701139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4035030 con 0x5620dd78b370 2024-09-19T06:21:32.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.557+0000 7f61bffff640 1 -- 172.21.15.5:0/3850701139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f61d4004240 con 0x5620dd78b370 2024-09-19T06:21:32.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.557+0000 7f61e9447740 1 -- 172.21.15.5:0/3850701139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d801fb30 con 0x5620dd78b370 2024-09-19T06:21:32.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.557+0000 7f61bffff640 1 -- 172.21.15.5:0/3850701139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f61d4004440 con 0x5620dd78b370 2024-09-19T06:21:32.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.557+0000 7f61e9447740 10 client.20079 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:32.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.557+0000 7f61e9447740 1 -- 172.21.15.5:0/3850701139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=58}) -- 0x5620dd7a4b90 con 0x5620dd78b370 2024-09-19T06:21:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.558+0000 7f61bffff640 1 -- 172.21.15.5:0/3850701139 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f61d4005050 con 0x5620dd78b370 2024-09-19T06:21:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.558+0000 7f61e9447740 10 client.20079 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.558+0000 7f61e9447740 10 client.20079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.558+0000 7f61e9447740 20 client.20079 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.558+0000 7f61e9447740 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5620dd7a5f40 0x5620dd7a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.558+0000 7f61e9447740 4 client.20079 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:21:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.558+0000 7f61e9447740 1 -- 172.21.15.5:0/3850701139 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f61d801fb30 con 0x5620dd7a5f40 2024-09-19T06:21:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.558+0000 7f61dffff640 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5620dd7a5f40 0x5620dd7a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.559+0000 7f61dffff640 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5620dd7a5f40 0x5620dd7a83d0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c891715c6730869 server_cookie=d1281e2ffd0eb0e8 in_seq=0 out_seq=0 2024-09-19T06:21:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.559+0000 7f61bffff640 10 client.20079 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.560+0000 7f61bffff640 1 -- 172.21.15.5:0/3850701139 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f61d801fb30 con 0x5620dd7a5f40 2024-09-19T06:21:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.560+0000 7f61bffff640 10 client.20079 handle_command_reply: tid=0 2024-09-19T06:21:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.560+0000 7f61e9447740 1 client.20079 shutdown 2024-09-19T06:21:32.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 48998 8.75 MB/sec execute 176 sec latency 89.451 ms 2024-09-19T06:21:32.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.561+0000 7f61e9447740 1 -- 172.21.15.5:0/3850701139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 msgr2=0x5620dd793450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:32.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.561+0000 7f61e9447740 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 0x5620dd793450 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009900 tx=0x7f61d4032f90 comp rx=0 tx=0).stop 2024-09-19T06:21:32.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.561+0000 7f61e9447740 1 -- 172.21.15.5:0/3850701139 shutdown_connections 2024-09-19T06:21:32.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.561+0000 7f61e9447740 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5620dd7a5f40 0x5620dd7a83d0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.561+0000 7f61e9447740 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 0x5620dd793ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.561+0000 7f61e9447740 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78bc90 0x5620dd793990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.561+0000 7f61bffff640 0 client.20079 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:32.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.561+0000 7f61e9447740 1 --2- 172.21.15.5:0/3850701139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 0x5620dd793450 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.561+0000 7f61e9447740 1 -- 172.21.15.5:0/3850701139 >> 172.21.15.5:0/3850701139 conn(0x5620dd787900 msgr2=0x5620dd78d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:32.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.562+0000 7f61e9447740 1 -- 172.21.15.5:0/3850701139 shutdown_connections 2024-09-19T06:21:32.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.562+0000 7f61e9447740 1 -- 172.21.15.5:0/3850701139 wait complete. 2024-09-19T06:21:32.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.562+0000 7f61e9447740 20 client.20079 trim_cache size 0 max 16384 2024-09-19T06:21:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.597+0000 7f61e9447740 1 Processor -- start 2024-09-19T06:21:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.597+0000 7f61e9447740 1 -- start start 2024-09-19T06:21:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.597+0000 7f61e9447740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 0x5620dd78d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.597+0000 7f61e9447740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd78bc90 0x5620dd7adce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.597+0000 7f61e9447740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd796550 0x5620dd7ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.597+0000 7f61e9447740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ab610 con 0x5620dd796550 2024-09-19T06:21:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.597+0000 7f61e9447740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ad150 con 0x5620dd78b370 2024-09-19T06:21:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.597+0000 7f61e9447740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01abce0 con 0x5620dd78bc90 2024-09-19T06:21:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.598+0000 7f61e4e0a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 0x5620dd78d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.598+0000 7f61e560b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd796550 0x5620dd7ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.598+0000 7f61e4e0a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 0x5620dd78d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35324/0 (socket says 172.21.15.5:35324) 2024-09-19T06:21:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.598+0000 7f61e4e0a640 1 -- 172.21.15.5:0/1645045172 learned_addr learned my addr 172.21.15.5:0/1645045172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.598+0000 7f61dffff640 1 --2- 172.21.15.5:0/1645045172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd78bc90 0x5620dd7adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.598+0000 7f61dffff640 1 -- 172.21.15.5:0/1645045172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 msgr2=0x5620dd78d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.598+0000 7f61dffff640 1 --2- 172.21.15.5:0/1645045172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 0x5620dd78d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.598+0000 7f61dffff640 1 -- 172.21.15.5:0/1645045172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd796550 msgr2=0x5620dd7ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.598+0000 7f61dffff640 1 --2- 172.21.15.5:0/1645045172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd796550 0x5620dd7ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.598+0000 7f61dffff640 1 -- 172.21.15.5:0/1645045172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620dd792400 con 0x5620dd78bc90 2024-09-19T06:21:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.598+0000 7f61e560b640 1 --2- 172.21.15.5:0/1645045172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd796550 0x5620dd7ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.598+0000 7f61dffff640 1 --2- 172.21.15.5:0/1645045172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd78bc90 0x5620dd7adce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0cd7a0 tx=0x7f61cc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0995f88083bfe3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.599+0000 7f61ca7fc640 1 -- 172.21.15.5:0/1645045172 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0a06e0 con 0x5620dd78bc90 2024-09-19T06:21:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.599+0000 7f61ca7fc640 1 -- 172.21.15.5:0/1645045172 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61cc0a4fa0 con 0x5620dd78bc90 2024-09-19T06:21:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.599+0000 7f61ca7fc640 1 -- 172.21.15.5:0/1645045172 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0b0290 con 0x5620dd78bc90 2024-09-19T06:21:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.600+0000 7f61e9447740 1 -- 172.21.15.5:0/1645045172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd78bc90 msgr2=0x5620dd7adce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.600+0000 7f61e9447740 1 --2- 172.21.15.5:0/1645045172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd78bc90 0x5620dd7adce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0cd7a0 tx=0x7f61cc009310 comp rx=0 tx=0).stop 2024-09-19T06:21:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.600+0000 7f61e9447740 1 -- 172.21.15.5:0/1645045172 shutdown_connections 2024-09-19T06:21:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.600+0000 7f61e9447740 1 --2- 172.21.15.5:0/1645045172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd796550 0x5620dd7ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.600+0000 7f61e9447740 1 --2- 172.21.15.5:0/1645045172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd78bc90 0x5620dd7adce0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.600+0000 7f61e9447740 1 --2- 172.21.15.5:0/1645045172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78b370 0x5620dd78d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.600+0000 7f61e9447740 1 -- 172.21.15.5:0/1645045172 >> 172.21.15.5:0/1645045172 conn(0x5620dd787900 msgr2=0x5620dd78ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.600+0000 7f61e9447740 1 -- 172.21.15.5:0/1645045172 shutdown_connections 2024-09-19T06:21:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.600+0000 7f61e9447740 1 -- 172.21.15.5:0/1645045172 wait complete. 2024-09-19T06:21:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.600+0000 7f61e9447740 1 -- start start 2024-09-19T06:21:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.600+0000 7f61e9447740 1 Processor -- start 2024-09-19T06:21:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.601+0000 7f61e9447740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 0x5620dd79ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.601+0000 7f61e9447740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78bc90 0x5620dd79b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.601+0000 7f61e9447740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 0x5620dd808c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.601+0000 7f61e4e0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 0x5620dd79ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.601+0000 7f61e9447740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01aca80 con 0x5620dd78b370 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.601+0000 7f61e9447740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ac3b0 con 0x5620dd78bc90 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.601+0000 7f61e9447740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01a9610 con 0x5620dd796550 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.601+0000 7f61e4e0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 0x5620dd79ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39440/0 (socket says 172.21.15.5:39440) 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.601+0000 7f61e4e0a640 1 -- 172.21.15.5:0/937198903 learned_addr learned my addr 172.21.15.5:0/937198903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.601+0000 7f61dffff640 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78bc90 0x5620dd79b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.601+0000 7f61e9447740 10 client.0 _ll_register_callbacks cb 0x7f61cc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.601+0000 7f61e560b640 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 0x5620dd808c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61dffff640 1 -- 172.21.15.5:0/937198903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 msgr2=0x5620dd808c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61dffff640 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 0x5620dd808c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61dffff640 1 -- 172.21.15.5:0/937198903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 msgr2=0x5620dd79ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61dffff640 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 0x5620dd79ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61dffff640 1 -- 172.21.15.5:0/937198903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620dd809330 con 0x5620dd78bc90 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61e4e0a640 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 0x5620dd79ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61e560b640 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 0x5620dd808c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61dffff640 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78bc90 0x5620dd79b3c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0cdda0 tx=0x7f61cc010eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff3094d9203a2b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61bffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61bffff640 1 -- 172.21.15.5:0/937198903 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc09a630 con 0x5620dd78bc90 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61bffff640 1 -- 172.21.15.5:0/937198903 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f61cc0989a0 con 0x5620dd78bc90 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61e9447740 1 -- 172.21.15.5:0/937198903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61cc0a4fa0 con 0x5620dd78bc90 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61bffff640 1 -- 172.21.15.5:0/937198903 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f61cc010430 con 0x5620dd78bc90 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61e9447740 10 client.29459 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.602+0000 7f61e9447740 1 -- 172.21.15.5:0/937198903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5620dd811890 con 0x5620dd78bc90 2024-09-19T06:21:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.603+0000 7f61bffff640 1 -- 172.21.15.5:0/937198903 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61cc0a4fa0 con 0x5620dd78bc90 2024-09-19T06:21:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.603+0000 7f61bffff640 1 -- 172.21.15.5:0/937198903 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f61cc0b0290 con 0x5620dd78bc90 2024-09-19T06:21:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.603+0000 7f61e9447740 10 client.29459 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.603+0000 7f61e9447740 10 client.29459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.603+0000 7f61e9447740 20 client.29459 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.603+0000 7f61e9447740 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5620dd811fe0 0x5620dd814400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.603+0000 7f61e9447740 4 client.29459 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:21:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.603+0000 7f61e9447740 1 -- 172.21.15.5:0/937198903 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5620dd6f57f0 con 0x5620dd811fe0 2024-09-19T06:21:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.603+0000 7f61e4e0a640 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5620dd811fe0 0x5620dd814400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:32.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.604+0000 7f61e4e0a640 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5620dd811fe0 0x5620dd814400 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af69ccbd585ce866 server_cookie=d53a6f5b09e32c83 in_seq=0 out_seq=0 2024-09-19T06:21:32.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.604+0000 7f61bffff640 10 client.29459 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.605+0000 7f61bffff640 1 -- 172.21.15.5:0/937198903 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5620dd6f57f0 con 0x5620dd811fe0 2024-09-19T06:21:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.605+0000 7f61bffff640 10 client.29459 handle_command_reply: tid=0 2024-09-19T06:21:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.605+0000 7f61e9447740 1 client.29459 shutdown 2024-09-19T06:21:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.606+0000 7f61e9447740 1 -- 172.21.15.5:0/937198903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78bc90 msgr2=0x5620dd79b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.606+0000 7f61e9447740 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78bc90 0x5620dd79b3c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0cdda0 tx=0x7f61cc010eb0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.606+0000 7f61e9447740 1 -- 172.21.15.5:0/937198903 shutdown_connections 2024-09-19T06:21:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.606+0000 7f61e9447740 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5620dd811fe0 0x5620dd814400 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.606+0000 7f61bffff640 0 client.29459 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.606+0000 7f61e9447740 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5620dd796550 0x5620dd808c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.606+0000 7f61e9447740 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620dd78bc90 0x5620dd79b3c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.606+0000 7f61e9447740 1 --2- 172.21.15.5:0/937198903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620dd78b370 0x5620dd79ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.606+0000 7f61e9447740 1 -- 172.21.15.5:0/937198903 >> 172.21.15.5:0/937198903 conn(0x5620dd787900 msgr2=0x5620dd77e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:32.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.606+0000 7f61e9447740 1 -- 172.21.15.5:0/937198903 shutdown_connections 2024-09-19T06:21:32.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.606+0000 7f61e9447740 1 -- 172.21.15.5:0/937198903 wait complete. 2024-09-19T06:21:32.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.606+0000 7f61e9447740 20 client.29459 trim_cache size 0 max 16384 2024-09-19T06:21:32.611 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:21:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:21:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "c8f28885-dd81-436e-9bf6-c06a6e41a087", 2024-09-19T06:21:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:21:32.611 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:21:32.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.607+0000 7f61e706e640 1 -- 172.21.15.5:0/2206581279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61d00a3430 msgr2=0x7f61d00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:32.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.607+0000 7f61e706e640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61d00a3430 0x7f61d00a58f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f61d8021000 tx=0x7f61d801f000 comp rx=0 tx=0).stop 2024-09-19T06:21:32.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.607+0000 7f61e706e640 1 -- 172.21.15.5:0/2206581279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0100c80 msgr2=0x7f61e019aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:32.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.607+0000 7f61e706e640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0100c80 0x7f61e019aab0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00e9f0 tx=0x7f61cc00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.607+0000 7f61e706e640 1 -- 172.21.15.5:0/2206581279 shutdown_connections 2024-09-19T06:21:32.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.607+0000 7f61e706e640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e0102a50 0x7f61e019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.607+0000 7f61e706e640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f61d00a3430 0x7f61d00a58f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.607+0000 7f61e706e640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e0100c80 0x7f61e019aab0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.607+0000 7f61e706e640 1 --2- 172.21.15.5:0/2206581279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61e0100100 0x7f61e019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:32.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.607+0000 7f61e706e640 1 -- 172.21.15.5:0/2206581279 >> 172.21.15.5:0/2206581279 conn(0x7f61e0077230 msgr2=0x7f61e00fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:32.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.607+0000 7f61e706e640 1 -- 172.21.15.5:0/2206581279 shutdown_connections 2024-09-19T06:21:32.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:32.608+0000 7f61e706e640 1 -- 172.21.15.5:0/2206581279 wait complete. 2024-09-19T06:21:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:32 smithi157 ceph-mon[35346]: pgmap v573: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 245 op/s 2024-09-19T06:21:32.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:32 smithi005 ceph-mon[30955]: pgmap v573: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 245 op/s 2024-09-19T06:21:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:32 smithi049 ceph-mon[34640]: pgmap v573: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 245 op/s 2024-09-19T06:21:33.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:21:33.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 49134 8.71 MB/sec execute 177 sec latency 189.025 ms 2024-09-19T06:21:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.668+0000 7f8a11eaf640 1 -- 172.21.15.5:0/343261238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0c10c110 msgr2=0x7f8a0c10e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.668+0000 7f8a11eaf640 1 --2- 172.21.15.5:0/343261238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0c10c110 0x7f8a0c10e540 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009ca0 tx=0x7f8a0402a950 comp rx=0 tx=0).stop 2024-09-19T06:21:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.668+0000 7f8a11eaf640 1 -- 172.21.15.5:0/343261238 shutdown_connections 2024-09-19T06:21:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.668+0000 7f8a11eaf640 1 --2- 172.21.15.5:0/343261238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0c10c110 0x7f8a0c10e540 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.668+0000 7f8a11eaf640 1 --2- 172.21.15.5:0/343261238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0c103660 0x7f8a0c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.668+0000 7f8a11eaf640 1 --2- 172.21.15.5:0/343261238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a0c102d40 0x7f8a0c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.668+0000 7f8a11eaf640 1 -- 172.21.15.5:0/343261238 >> 172.21.15.5:0/343261238 conn(0x7f8a0c0779a0 msgr2=0x7f8a0c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.668+0000 7f8a11eaf640 1 -- 172.21.15.5:0/343261238 shutdown_connections 2024-09-19T06:21:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.668+0000 7f8a11eaf640 1 -- 172.21.15.5:0/343261238 wait complete. 2024-09-19T06:21:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.669+0000 7f8a11eaf640 1 Processor -- start 2024-09-19T06:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.669+0000 7f8a11eaf640 1 -- start start 2024-09-19T06:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a11eaf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a0c102d40 0x7f8a0c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a11eaf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0c103660 0x7f8a0c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a11eaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0c10c110 0x7f8a0c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a11eaf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c111da0 con 0x7f8a0c103660 2024-09-19T06:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a11eaf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c111c20 con 0x7f8a0c102d40 2024-09-19T06:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a11eaf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c111f20 con 0x7f8a0c10c110 2024-09-19T06:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a0c102d40 0x7f8a0c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a0c102d40 0x7f8a0c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35354/0 (socket says 172.21.15.5:35354) 2024-09-19T06:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a0b7fe640 1 -- 172.21.15.5:0/3249543906 learned_addr learned my addr 172.21.15.5:0/3249543906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a0affd640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0c103660 0x7f8a0c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a0bfff640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0c10c110 0x7f8a0c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a0bfff640 1 -- 172.21.15.5:0/3249543906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a0c102d40 msgr2=0x7f8a0c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a0bfff640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a0c102d40 0x7f8a0c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a0bfff640 1 -- 172.21.15.5:0/3249543906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0c103660 msgr2=0x7f8a0c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a0bfff640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0c103660 0x7f8a0c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.670+0000 7f8a0bfff640 1 -- 172.21.15.5:0/3249543906 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c1a6790 con 0x7f8a0c10c110 2024-09-19T06:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.671+0000 7f8a0bfff640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0c10c110 0x7f8a0c101410 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005e30 tx=0x7f8a04003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.671+0000 7f8a08ff9640 1 -- 172.21.15.5:0/3249543906 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04034040 con 0x7f8a0c10c110 2024-09-19T06:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.671+0000 7f8a08ff9640 1 -- 172.21.15.5:0/3249543906 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a0402fd20 con 0x7f8a0c10c110 2024-09-19T06:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.671+0000 7f8a08ff9640 1 -- 172.21.15.5:0/3249543906 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04004590 con 0x7f8a0c10c110 2024-09-19T06:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.671+0000 7f8a11eaf640 1 -- 172.21.15.5:0/3249543906 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c1a6a20 con 0x7f8a0c10c110 2024-09-19T06:21:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.671+0000 7f8a11eaf640 1 -- 172.21.15.5:0/3249543906 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c1a6e90 con 0x7f8a0c10c110 2024-09-19T06:21:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.675+0000 7f8a08ff9640 1 -- 172.21.15.5:0/3249543906 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8a04037020 con 0x7f8a0c10c110 2024-09-19T06:21:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.675+0000 7f8a14c3f740 1 Processor -- start 2024-09-19T06:21:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.675+0000 7f8a14c3f740 1 -- start start 2024-09-19T06:21:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.675+0000 7f8a14c3f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d893daf0 0x55d4d893ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.675+0000 7f8a14c3f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893e410 0x55d4d8948770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.675+0000 7f8a14c3f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d8948cb0 0x55d4d894b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.675+0000 7f8a14c3f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d4d88a22f0 con 0x55d4d8948cb0 2024-09-19T06:21:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.675+0000 7f8a14c3f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d4d884a270 con 0x55d4d893daf0 2024-09-19T06:21:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.675+0000 7f8a14c3f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d4d889e2b0 con 0x55d4d893e410 2024-09-19T06:21:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.675+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893e410 0x55d4d8948770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.675+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893e410 0x55d4d8948770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38876/0 (socket says 172.21.15.5:38876) 2024-09-19T06:21:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.675+0000 7f8a0b7fe640 1 -- 172.21.15.5:0/1401854014 learned_addr learned my addr 172.21.15.5:0/1401854014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f8a0affd640 1 --2- 172.21.15.5:0/1401854014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d8948cb0 0x55d4d894b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f8a0b7fe640 1 -- 172.21.15.5:0/1401854014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d893daf0 msgr2=0x55d4d893ded0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:21:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f8a0b7fe640 1 --2- 172.21.15.5:0/1401854014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d893daf0 0x55d4d893ded0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f8a0b7fe640 1 -- 172.21.15.5:0/1401854014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d8948cb0 msgr2=0x55d4d894b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f8a0b7fe640 1 --2- 172.21.15.5:0/1401854014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d8948cb0 0x55d4d894b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f8a0b7fe640 1 -- 172.21.15.5:0/1401854014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4d8849340 con 0x55d4d893e410 2024-09-19T06:21:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f8a08ff9640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f89f40a3430 0x7f89f40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f8a0affd640 1 --2- 172.21.15.5:0/1401854014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d8948cb0 0x55d4d894b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:21:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f8a08ff9640 1 -- 172.21.15.5:0/3249543906 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f8a040f08a0 con 0x7f8a0c10c110 2024-09-19T06:21:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f8a0b7fe640 1 --2- 172.21.15.5:0/1401854014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893e410 0x55d4d8948770 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009dc0 tx=0x7f89fc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaab249ae937a8a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f8a0affd640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f89f40a3430 0x7f89f40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f89f1ffb640 1 -- 172.21.15.5:0/1401854014 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc005220 con 0x55d4d893e410 2024-09-19T06:21:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f89f1ffb640 1 -- 172.21.15.5:0/1401854014 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89fc004c50 con 0x55d4d893e410 2024-09-19T06:21:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.676+0000 7f89f1ffb640 1 -- 172.21.15.5:0/1401854014 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc019900 con 0x55d4d893e410 2024-09-19T06:21:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.677+0000 7f8a14c3f740 1 -- 172.21.15.5:0/1401854014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893e410 msgr2=0x55d4d8948770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.677+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/1401854014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893e410 0x55d4d8948770 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009dc0 tx=0x7f89fc011810 comp rx=0 tx=0).stop 2024-09-19T06:21:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.677+0000 7f8a0affd640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f89f40a3430 0x7f89f40a58f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55d4d893b3e0 tx=0x7f8a00005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.677+0000 7f8a14c3f740 1 -- 172.21.15.5:0/1401854014 shutdown_connections 2024-09-19T06:21:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.677+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/1401854014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d8948cb0 0x55d4d894b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.677+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/1401854014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893e410 0x55d4d8948770 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.677+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/1401854014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d893daf0 0x55d4d893ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.677+0000 7f8a14c3f740 1 -- 172.21.15.5:0/1401854014 >> 172.21.15.5:0/1401854014 conn(0x55d4d893a060 msgr2=0x55d4d893a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.678+0000 7f8a14c3f740 1 -- 172.21.15.5:0/1401854014 shutdown_connections 2024-09-19T06:21:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.678+0000 7f8a14c3f740 1 -- 172.21.15.5:0/1401854014 wait complete. 2024-09-19T06:21:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.678+0000 7f8a14c3f740 1 -- start start 2024-09-19T06:21:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a14c3f740 1 Processor -- start 2024-09-19T06:21:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a14c3f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d893daf0 0x55d4d8945be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a14c3f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893e410 0x55d4d8946120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a14c3f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d8948cb0 0x55d4d8946660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a14c3f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d4d894b880 con 0x55d4d893e410 2024-09-19T06:21:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a14c3f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d4d882b290 con 0x55d4d893daf0 2024-09-19T06:21:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a14c3f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d4d894ba00 con 0x55d4d8948cb0 2024-09-19T06:21:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a14c3f740 10 client.0 _ll_register_callbacks cb 0x55d4d885c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d893daf0 0x55d4d8945be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d893daf0 0x55d4d8945be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35368/0 (socket says 172.21.15.5:35368) 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a0bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d8948cb0 0x55d4d8946660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a0b7fe640 1 -- 172.21.15.5:0/3022926781 learned_addr learned my addr 172.21.15.5:0/3022926781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.679+0000 7f8a0affd640 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893e410 0x55d4d8946120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.680+0000 7f8a0b7fe640 1 -- 172.21.15.5:0/3022926781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d8948cb0 msgr2=0x55d4d8946660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.680+0000 7f8a0b7fe640 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d8948cb0 0x55d4d8946660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.680+0000 7f8a0b7fe640 1 -- 172.21.15.5:0/3022926781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893e410 msgr2=0x55d4d8946120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.680+0000 7f8a0b7fe640 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893e410 0x55d4d8946120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.680+0000 7f8a0b7fe640 1 -- 172.21.15.5:0/3022926781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4d85b5210 con 0x55d4d893daf0 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.680+0000 7f8a0affd640 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893e410 0x55d4d8946120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.680+0000 7f8a0bfff640 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d8948cb0 0x55d4d8946660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.680+0000 7f8a0b7fe640 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d893daf0 0x55d4d8945be0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0042a0 tx=0x7f89fc0042d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb655799874a30c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.680+0000 7f89eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.680+0000 7f89eaffd640 1 -- 172.21.15.5:0/3022926781 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc027070 con 0x55d4d893daf0 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.680+0000 7f89eaffd640 1 -- 172.21.15.5:0/3022926781 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f89fc005000 con 0x55d4d893daf0 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.680+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3022926781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89fc004c50 con 0x55d4d893daf0 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.681+0000 7f89eaffd640 1 -- 172.21.15.5:0/3022926781 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f89fc031050 con 0x55d4d893daf0 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.681+0000 7f8a14c3f740 10 client.29468 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.681+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3022926781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d4d8681620 con 0x55d4d893daf0 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.681+0000 7f89eaffd640 1 -- 172.21.15.5:0/3022926781 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89fc004c50 con 0x55d4d893daf0 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.681+0000 7f89eaffd640 1 -- 172.21.15.5:0/3022926781 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f89fc019c90 con 0x55d4d893daf0 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.681+0000 7f8a14c3f740 10 client.29468 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.681+0000 7f8a14c3f740 10 client.29468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.681+0000 7f8a14c3f740 20 client.29468 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.681+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d4d89613c0 0x55d4d89637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.681+0000 7f8a14c3f740 4 client.29468 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:21:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.681+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3022926781 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4d88afa20 con 0x55d4d89613c0 2024-09-19T06:21:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.682+0000 7f8a0affd640 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d4d89613c0 0x55d4d89637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.682+0000 7f89eaffd640 10 client.29468 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.682+0000 7f8a0affd640 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d4d89613c0 0x55d4d89637b0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9901da442fd74c server_cookie=c01ab55e328adcb6 in_seq=0 out_seq=0 2024-09-19T06:21:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.683+0000 7f89eaffd640 1 -- 172.21.15.5:0/3022926781 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4d88afa20 con 0x55d4d89613c0 2024-09-19T06:21:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.683+0000 7f89eaffd640 10 client.29468 handle_command_reply: tid=0 2024-09-19T06:21:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.683+0000 7f8a14c3f740 1 client.29468 shutdown 2024-09-19T06:21:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.683+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3022926781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d893daf0 msgr2=0x55d4d8945be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.683+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d893daf0 0x55d4d8945be0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0042a0 tx=0x7f89fc0042d0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.683+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3022926781 shutdown_connections 2024-09-19T06:21:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.683+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d4d89613c0 0x55d4d89637b0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.683+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d8948cb0 0x55d4d8946660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.683+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893e410 0x55d4d8946120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.683+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/3022926781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d893daf0 0x55d4d8945be0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.684+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3022926781 >> 172.21.15.5:0/3022926781 conn(0x55d4d893a060 msgr2=0x55d4d893fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.684+0000 7f89eaffd640 0 client.29468 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.684+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3022926781 shutdown_connections 2024-09-19T06:21:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.684+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3022926781 wait complete. 2024-09-19T06:21:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.684+0000 7f8a14c3f740 20 client.29468 trim_cache size 0 max 16384 2024-09-19T06:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.705+0000 7f8a14c3f740 1 Processor -- start 2024-09-19T06:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.705+0000 7f8a14c3f740 1 -- start start 2024-09-19T06:21:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.705+0000 7f8a14c3f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893daf0 0x55d4d8968fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.705+0000 7f8a14c3f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893e410 0x55d4d89694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893daf0 0x55d4d8968fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893daf0 0x55d4d8968fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38902/0 (socket says 172.21.15.5:38902) 2024-09-19T06:21:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a14c3f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d8948cb0 0x55d4d893f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a14c3f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1ab5a0 con 0x55d4d893e410 2024-09-19T06:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a14c3f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1ad0e0 con 0x55d4d8948cb0 2024-09-19T06:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a14c3f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1abc70 con 0x55d4d893daf0 2024-09-19T06:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a0b7fe640 1 -- 172.21.15.5:0/1757262263 learned_addr learned my addr 172.21.15.5:0/1757262263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a0affd640 1 --2- 172.21.15.5:0/1757262263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893e410 0x55d4d89694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a0bfff640 1 --2- 172.21.15.5:0/1757262263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d8948cb0 0x55d4d893f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a0affd640 1 -- 172.21.15.5:0/1757262263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893daf0 msgr2=0x55d4d8968fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a0affd640 1 --2- 172.21.15.5:0/1757262263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893daf0 0x55d4d8968fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a0affd640 1 -- 172.21.15.5:0/1757262263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d8948cb0 msgr2=0x55d4d893f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a0affd640 1 --2- 172.21.15.5:0/1757262263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d8948cb0 0x55d4d893f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.706+0000 7f8a0affd640 1 -- 172.21.15.5:0/1757262263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4d893d870 con 0x55d4d893e410 2024-09-19T06:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.707+0000 7f8a0affd640 1 --2- 172.21.15.5:0/1757262263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893e410 0x55d4d89694f0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x55d4d8947220 tx=0x7f8a00016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7701669e209c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.707+0000 7f89f17fa640 1 -- 172.21.15.5:0/1757262263 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000044e0 con 0x55d4d893e410 2024-09-19T06:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.707+0000 7f89f17fa640 1 -- 172.21.15.5:0/1757262263 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a000282d0 con 0x55d4d893e410 2024-09-19T06:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.707+0000 7f89f17fa640 1 -- 172.21.15.5:0/1757262263 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0001f680 con 0x55d4d893e410 2024-09-19T06:21:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.708+0000 7f8a14c3f740 1 -- 172.21.15.5:0/1757262263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893e410 msgr2=0x55d4d89694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.708+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/1757262263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893e410 0x55d4d89694f0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x55d4d8947220 tx=0x7f8a00016000 comp rx=0 tx=0).stop 2024-09-19T06:21:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.708+0000 7f8a14c3f740 1 -- 172.21.15.5:0/1757262263 shutdown_connections 2024-09-19T06:21:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.708+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/1757262263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d8948cb0 0x55d4d893f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.708+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/1757262263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893e410 0x55d4d89694f0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.708+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/1757262263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893daf0 0x55d4d8968fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.708+0000 7f8a14c3f740 1 -- 172.21.15.5:0/1757262263 >> 172.21.15.5:0/1757262263 conn(0x55d4d893a060 msgr2=0x55d4d89633d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.708+0000 7f8a14c3f740 1 -- 172.21.15.5:0/1757262263 shutdown_connections 2024-09-19T06:21:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.709+0000 7f8a14c3f740 1 -- 172.21.15.5:0/1757262263 wait complete. 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.709+0000 7f8a14c3f740 1 -- start start 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.709+0000 7f8a14c3f740 1 Processor -- start 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a14c3f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893daf0 0x55d4d8959850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a14c3f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893e410 0x55d4d8959d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a14c3f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d8948cb0 0x55d4d8942590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893daf0 0x55d4d8959850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a14c3f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1aca10 con 0x55d4d893daf0 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a14c3f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1ac340 con 0x55d4d8948cb0 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893daf0 0x55d4d8959850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39466/0 (socket says 172.21.15.5:39466) 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a0b7fe640 1 -- 172.21.15.5:0/3992676753 learned_addr learned my addr 172.21.15.5:0/3992676753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3992676753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1a95a0 con 0x55d4d893e410 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a0affd640 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893e410 0x55d4d8959d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a0bfff640 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d8948cb0 0x55d4d8942590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a14c3f740 10 client.0 _ll_register_callbacks cb 0x7f89fc009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a0bfff640 1 -- 172.21.15.5:0/3992676753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893e410 msgr2=0x55d4d8959d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a0bfff640 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893e410 0x55d4d8959d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a0bfff640 1 -- 172.21.15.5:0/3992676753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893daf0 msgr2=0x55d4d8959850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a0bfff640 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893daf0 0x55d4d8959850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.710+0000 7f8a0bfff640 1 -- 172.21.15.5:0/3992676753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4d8942dc0 con 0x55d4d8948cb0 2024-09-19T06:21:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.711+0000 7f8a0bfff640 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d8948cb0 0x55d4d8942590 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8a040ee0e0 tx=0x7f8a040bc7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2557583a7d8f637 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.711+0000 7f89eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:21:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.711+0000 7f89eaffd640 1 -- 172.21.15.5:0/3992676753 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0402c040 con 0x55d4d8948cb0 2024-09-19T06:21:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.711+0000 7f89eaffd640 1 -- 172.21.15.5:0/3992676753 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f8a040bd0d0 con 0x55d4d8948cb0 2024-09-19T06:21:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.711+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3992676753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a000282d0 con 0x55d4d8948cb0 2024-09-19T06:21:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.711+0000 7f89eaffd640 1 -- 172.21.15.5:0/3992676753 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8a040bd370 con 0x55d4d8948cb0 2024-09-19T06:21:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.712+0000 7f8a14c3f740 10 client.29474 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.712+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3992676753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d4d89cbb30 con 0x55d4d8948cb0 2024-09-19T06:21:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.712+0000 7f89eaffd640 1 -- 172.21.15.5:0/3992676753 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a000282d0 con 0x55d4d8948cb0 2024-09-19T06:21:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.712+0000 7f89eaffd640 1 -- 172.21.15.5:0/3992676753 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f8a040bc0c0 con 0x55d4d8948cb0 2024-09-19T06:21:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.712+0000 7f8a14c3f740 10 client.29474 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.712+0000 7f8a14c3f740 10 client.29474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.712+0000 7f8a14c3f740 20 client.29474 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.712+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d4d89cc2a0 0x55d4d89ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.712+0000 7f8a14c3f740 4 client.29474 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:21:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.712+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3992676753 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d4d88a9d70 con 0x55d4d89cc2a0 2024-09-19T06:21:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.713+0000 7f8a0b7fe640 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d4d89cc2a0 0x55d4d89ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.713+0000 7f89eaffd640 10 client.29474 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.713+0000 7f8a0b7fe640 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d4d89cc2a0 0x55d4d89ce690 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce740cd7103ac44 server_cookie=636ef68c063e96b5 in_seq=0 out_seq=0 2024-09-19T06:21:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.785+0000 7f89eaffd640 1 -- 172.21.15.5:0/3992676753 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d4d88a9d70 con 0x55d4d89cc2a0 2024-09-19T06:21:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.785+0000 7f89eaffd640 10 client.29474 handle_command_reply: tid=0 2024-09-19T06:21:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.785+0000 7f8a14c3f740 1 client.29474 shutdown 2024-09-19T06:21:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3992676753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d8948cb0 msgr2=0x55d4d8942590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d8948cb0 0x55d4d8942590 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8a040ee0e0 tx=0x7f8a040bc7b0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3992676753 shutdown_connections 2024-09-19T06:21:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d4d89cc2a0 0x55d4d89ce690 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4d8948cb0 0x55d4d8942590 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4d893e410 0x55d4d8959d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f8a14c3f740 1 --2- 172.21.15.5:0/3992676753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4d893daf0 0x55d4d8959850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3992676753 >> 172.21.15.5:0/3992676753 conn(0x55d4d893a060 msgr2=0x55d4d893f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f89eaffd640 0 client.29474 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3992676753 shutdown_connections 2024-09-19T06:21:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f8a14c3f740 1 -- 172.21.15.5:0/3992676753 wait complete. 2024-09-19T06:21:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f8a14c3f740 20 client.29474 trim_cache size 0 max 16384 2024-09-19T06:21:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:21:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:21:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:21:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:21:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.786+0000 7f8a11eaf640 1 -- 172.21.15.5:0/3249543906 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f89f40a3430 msgr2=0x7f89f40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.787+0000 7f8a11eaf640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f89f40a3430 0x7f89f40a58f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55d4d893b3e0 tx=0x7f8a00005f90 comp rx=0 tx=0).stop 2024-09-19T06:21:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.787+0000 7f8a11eaf640 1 -- 172.21.15.5:0/3249543906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0c10c110 msgr2=0x7f8a0c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.787+0000 7f8a11eaf640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0c10c110 0x7f8a0c101410 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005e30 tx=0x7f8a04003c10 comp rx=0 tx=0).stop 2024-09-19T06:21:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.787+0000 7f8a11eaf640 1 -- 172.21.15.5:0/3249543906 shutdown_connections 2024-09-19T06:21:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.787+0000 7f8a11eaf640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f89f40a3430 0x7f89f40a58f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.787+0000 7f8a11eaf640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0c10c110 0x7f8a0c101410 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.787+0000 7f8a11eaf640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0c103660 0x7f8a0c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.787+0000 7f8a11eaf640 1 --2- 172.21.15.5:0/3249543906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a0c102d40 0x7f8a0c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.787+0000 7f8a11eaf640 1 -- 172.21.15.5:0/3249543906 >> 172.21.15.5:0/3249543906 conn(0x7f8a0c0779a0 msgr2=0x7f8a0c0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.787+0000 7f8a11eaf640 1 -- 172.21.15.5:0/3249543906 shutdown_connections 2024-09-19T06:21:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:33.787+0000 7f8a11eaf640 1 -- 172.21.15.5:0/3249543906 wait complete. 2024-09-19T06:21:33.793 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:21:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:21:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:33.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:33.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:21:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:33.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:33.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:33.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:21:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:33.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:33.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:33.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:33.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:33.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:33.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.342+0000 7f675865d640 1 -- 172.21.15.5:0/372917197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6750102790 msgr2=0x7f67501067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:34.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.342+0000 7f675865d640 1 --2- 172.21.15.5:0/372917197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6750102790 0x7f67501067e0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6740009900 tx=0x7f674002a410 comp rx=0 tx=0).stop 2024-09-19T06:21:34.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.342+0000 7f675865d640 1 -- 172.21.15.5:0/372917197 shutdown_connections 2024-09-19T06:21:34.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.342+0000 7f675865d640 1 --2- 172.21.15.5:0/372917197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6750106f10 0x7f675010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.342+0000 7f675865d640 1 --2- 172.21.15.5:0/372917197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6750102790 0x7f67501067e0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.342+0000 7f675865d640 1 --2- 172.21.15.5:0/372917197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6750101de0 0x7f67501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.342+0000 7f675865d640 1 -- 172.21.15.5:0/372917197 >> 172.21.15.5:0/372917197 conn(0x7f67500fd880 msgr2=0x7f67500ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:34.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.342+0000 7f675865d640 1 -- 172.21.15.5:0/372917197 shutdown_connections 2024-09-19T06:21:34.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.342+0000 7f675865d640 1 -- 172.21.15.5:0/372917197 wait complete. 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.343+0000 7f675865d640 1 Processor -- start 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.343+0000 7f675865d640 1 -- start start 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f675865d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6750101de0 0x7f6750198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f675865d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6750102790 0x7f6750198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f675865d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6750106f10 0x7f675019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f675865d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6750111e20 con 0x7f6750106f10 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f675865d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6750111ca0 con 0x7f6750102790 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f675865d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6750111fa0 con 0x7f6750101de0 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f6756bfa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6750106f10 0x7f675019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f6755bf8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6750102790 0x7f6750198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f67563f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6750101de0 0x7f6750198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f6755bf8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6750102790 0x7f6750198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35390/0 (socket says 172.21.15.5:35390) 2024-09-19T06:21:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f6755bf8640 1 -- 172.21.15.5:0/1037441196 learned_addr learned my addr 172.21.15.5:0/1037441196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f6755bf8640 1 -- 172.21.15.5:0/1037441196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6750101de0 msgr2=0x7f6750198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f6755bf8640 1 --2- 172.21.15.5:0/1037441196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6750101de0 0x7f6750198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f6755bf8640 1 -- 172.21.15.5:0/1037441196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6750106f10 msgr2=0x7f675019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f6755bf8640 1 --2- 172.21.15.5:0/1037441196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6750106f10 0x7f675019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.344+0000 7f6755bf8640 1 -- 172.21.15.5:0/1037441196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675019d2f0 con 0x7f6750102790 2024-09-19T06:21:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.345+0000 7f6755bf8640 1 --2- 172.21.15.5:0/1037441196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6750102790 0x7f6750198880 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6740005940 tx=0x7f67400040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.345+0000 7f67477fe640 1 -- 172.21.15.5:0/1037441196 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674003f030 con 0x7f6750102790 2024-09-19T06:21:34.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.345+0000 7f67477fe640 1 -- 172.21.15.5:0/1037441196 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67400044c0 con 0x7f6750102790 2024-09-19T06:21:34.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.345+0000 7f675865d640 1 -- 172.21.15.5:0/1037441196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675019d580 con 0x7f6750102790 2024-09-19T06:21:34.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.345+0000 7f67477fe640 1 -- 172.21.15.5:0/1037441196 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740035070 con 0x7f6750102790 2024-09-19T06:21:34.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.345+0000 7f675865d640 1 -- 172.21.15.5:0/1037441196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67501a4ec0 con 0x7f6750102790 2024-09-19T06:21:34.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.348+0000 7f67477fe640 1 -- 172.21.15.5:0/1037441196 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6740042030 con 0x7f6750102790 2024-09-19T06:21:34.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.348+0000 7f675aa36740 1 Processor -- start 2024-09-19T06:21:34.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f675aa36740 1 -- start start 2024-09-19T06:21:34.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f675aa36740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d44a0 0x55b92a7d4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f675aa36740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d4dc0 0x55b92a7df140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f675aa36740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7df680 0x55b92a7e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f675aa36740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b92a73a2e0 con 0x55b92a7df680 2024-09-19T06:21:34.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f675aa36740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b92a6e1ce0 con 0x55b92a7d44a0 2024-09-19T06:21:34.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f675aa36740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b92a7362a0 con 0x55b92a7d4dc0 2024-09-19T06:21:34.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f6756bfa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d44a0 0x55b92a7d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f6756bfa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d44a0 0x55b92a7d4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35402/0 (socket says 172.21.15.5:35402) 2024-09-19T06:21:34.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f6756bfa640 1 -- 172.21.15.5:0/3826562340 learned_addr learned my addr 172.21.15.5:0/3826562340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:34.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f67563f9640 1 --2- 172.21.15.5:0/3826562340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d4dc0 0x55b92a7df140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f6755bf8640 1 --2- 172.21.15.5:0/3826562340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7df680 0x55b92a7e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.349+0000 7f67477fe640 1 --2- 172.21.15.5:0/1037441196 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f673c0a33b0 0x7f673c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.350+0000 7f6756bfa640 1 -- 172.21.15.5:0/3826562340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d4dc0 msgr2=0x55b92a7df140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:34.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.350+0000 7f6756bfa640 1 --2- 172.21.15.5:0/3826562340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d4dc0 0x55b92a7df140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.350+0000 7f6756bfa640 1 -- 172.21.15.5:0/3826562340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7df680 msgr2=0x55b92a7e1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:34.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.350+0000 7f6756bfa640 1 --2- 172.21.15.5:0/3826562340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7df680 0x55b92a7e1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.350+0000 7f6756bfa640 1 -- 172.21.15.5:0/3826562340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b92a6c6440 con 0x55b92a7d44a0 2024-09-19T06:21:34.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.350+0000 7f6755bf8640 1 --2- 172.21.15.5:0/3826562340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7df680 0x55b92a7e1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:21:34.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.350+0000 7f6756bfa640 1 --2- 172.21.15.5:0/1037441196 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f673c0a33b0 0x7f673c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.350+0000 7f6756bfa640 1 --2- 172.21.15.5:0/3826562340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d44a0 0x55b92a7d4880 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6748011590 tx=0x7f6748009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18568caed3029234 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:34.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.350+0000 7f6744ff9640 1 -- 172.21.15.5:0/3826562340 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748017050 con 0x55b92a7d44a0 2024-09-19T06:21:34.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.350+0000 7f6744ff9640 1 -- 172.21.15.5:0/3826562340 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f674801bbf0 con 0x55b92a7d44a0 2024-09-19T06:21:34.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.350+0000 7f6744ff9640 1 -- 172.21.15.5:0/3826562340 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748026ac0 con 0x55b92a7d44a0 2024-09-19T06:21:34.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.351+0000 7f67477fe640 1 -- 172.21.15.5:0/1037441196 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f67400efce0 con 0x7f6750102790 2024-09-19T06:21:34.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.351+0000 7f6756bfa640 1 --2- 172.21.15.5:0/1037441196 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f673c0a33b0 0x7f673c0a5870 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6748023000 tx=0x7f6748024000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:34.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.351+0000 7f675aa36740 1 -- 172.21.15.5:0/3826562340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d44a0 msgr2=0x55b92a7d4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:34.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.351+0000 7f675aa36740 1 --2- 172.21.15.5:0/3826562340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d44a0 0x55b92a7d4880 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6748011590 tx=0x7f6748009f90 comp rx=0 tx=0).stop 2024-09-19T06:21:34.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.351+0000 7f675aa36740 1 -- 172.21.15.5:0/3826562340 shutdown_connections 2024-09-19T06:21:34.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.351+0000 7f675aa36740 1 --2- 172.21.15.5:0/3826562340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7df680 0x55b92a7e1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.351+0000 7f675aa36740 1 --2- 172.21.15.5:0/3826562340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d4dc0 0x55b92a7df140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.351+0000 7f675aa36740 1 --2- 172.21.15.5:0/3826562340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d44a0 0x55b92a7d4880 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.351+0000 7f675aa36740 1 -- 172.21.15.5:0/3826562340 >> 172.21.15.5:0/3826562340 conn(0x55b92a7d0a30 msgr2=0x55b92a7d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:34.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.352+0000 7f675aa36740 1 -- 172.21.15.5:0/3826562340 shutdown_connections 2024-09-19T06:21:34.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.352+0000 7f675aa36740 1 -- 172.21.15.5:0/3826562340 wait complete. 2024-09-19T06:21:34.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.352+0000 7f675aa36740 1 -- start start 2024-09-19T06:21:34.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.352+0000 7f675aa36740 1 Processor -- start 2024-09-19T06:21:34.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f675aa36740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d44a0 0x55b92a7def90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f67563f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d44a0 0x55b92a7def90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f67563f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d44a0 0x55b92a7def90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39494/0 (socket says 172.21.15.5:39494) 2024-09-19T06:21:34.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f675aa36740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d4dc0 0x55b92a7dc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f67563f9640 1 -- 172.21.15.5:0/3705257831 learned_addr learned my addr 172.21.15.5:0/3705257831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:34.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f675aa36740 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7df680 0x55b92a7dc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f675aa36740 1 -- 172.21.15.5:0/3705257831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b92a7e22f0 con 0x55b92a7d44a0 2024-09-19T06:21:34.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f675aa36740 1 -- 172.21.15.5:0/3705257831 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b92a6f5050 con 0x55b92a7df680 2024-09-19T06:21:34.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f675aa36740 1 -- 172.21.15.5:0/3705257831 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b92a7e2470 con 0x55b92a7d4dc0 2024-09-19T06:21:34.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f6755bf8640 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d4dc0 0x55b92a7dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f675aa36740 10 client.0 _ll_register_callbacks cb 0x55b92a735160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f67563f9640 1 -- 172.21.15.5:0/3705257831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d4dc0 msgr2=0x55b92a7dc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f67563f9640 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d4dc0 0x55b92a7dc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f67563f9640 1 -- 172.21.15.5:0/3705257831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7df680 msgr2=0x55b92a7dc780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f67563f9640 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7df680 0x55b92a7dc780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.353+0000 7f67563f9640 1 -- 172.21.15.5:0/3705257831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b92a4ab3c0 con 0x55b92a7d44a0 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.354+0000 7f67563f9640 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d44a0 0x55b92a7def90 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f674c01cae0 tx=0x7f674c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7fad21d081c9f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.354+0000 7f6739ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.354+0000 7f6739ffb640 1 -- 172.21.15.5:0/3705257831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c010840 con 0x55b92a7d44a0 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.354+0000 7f675aa36740 1 -- 172.21.15.5:0/3705257831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f674801bbf0 con 0x55b92a7d44a0 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.354+0000 7f6739ffb640 1 -- 172.21.15.5:0/3705257831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f674c0109e0 con 0x55b92a7d44a0 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.354+0000 7f6739ffb640 1 -- 172.21.15.5:0/3705257831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f674c010be0 con 0x55b92a7d44a0 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.354+0000 7f675aa36740 10 client.20112 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.354+0000 7f675aa36740 1 -- 172.21.15.5:0/3705257831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55b92a7edd00 con 0x55b92a7d44a0 2024-09-19T06:21:34.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.354+0000 7f6739ffb640 1 -- 172.21.15.5:0/3705257831 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f674801bbf0 con 0x55b92a7d44a0 2024-09-19T06:21:34.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.355+0000 7f6739ffb640 1 -- 172.21.15.5:0/3705257831 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f674c023070 con 0x55b92a7d44a0 2024-09-19T06:21:34.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.355+0000 7f675aa36740 10 client.20112 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:34.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.355+0000 7f675aa36740 10 client.20112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:34.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.355+0000 7f675aa36740 20 client.20112 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:34.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.355+0000 7f675aa36740 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b92a7ef1b0 0x55b92a7f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.355+0000 7f675aa36740 4 client.20112 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:21:34.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.355+0000 7f675aa36740 1 -- 172.21.15.5:0/3705257831 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b92a747850 con 0x55b92a7ef1b0 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.355+0000 7f6755bf8640 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b92a7ef1b0 0x55b92a7f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.355+0000 7f6755bf8640 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b92a7ef1b0 0x55b92a7f1600 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f25638622bf072b server_cookie=51b6249d53648871 in_seq=0 out_seq=0 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.355+0000 7f6739ffb640 10 client.20112 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.356+0000 7f6739ffb640 1 -- 172.21.15.5:0/3705257831 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b92a747850 con 0x55b92a7ef1b0 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.356+0000 7f6739ffb640 10 client.20112 handle_command_reply: tid=0 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.356+0000 7f675aa36740 1 client.20112 shutdown 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.356+0000 7f675aa36740 1 -- 172.21.15.5:0/3705257831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d44a0 msgr2=0x55b92a7def90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.356+0000 7f675aa36740 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d44a0 0x55b92a7def90 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f674c01cae0 tx=0x7f674c011440 comp rx=0 tx=0).stop 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.356+0000 7f675aa36740 1 -- 172.21.15.5:0/3705257831 shutdown_connections 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.356+0000 7f675aa36740 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b92a7ef1b0 0x55b92a7f1600 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.357+0000 7f675aa36740 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7df680 0x55b92a7dc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.357+0000 7f675aa36740 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d4dc0 0x55b92a7dc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.357+0000 7f675aa36740 1 --2- 172.21.15.5:0/3705257831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d44a0 0x55b92a7def90 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.357+0000 7f675aa36740 1 -- 172.21.15.5:0/3705257831 >> 172.21.15.5:0/3705257831 conn(0x55b92a7d0a30 msgr2=0x55b92a7d6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.357+0000 7f6739ffb640 0 client.20112 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.357+0000 7f675aa36740 1 -- 172.21.15.5:0/3705257831 shutdown_connections 2024-09-19T06:21:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.357+0000 7f675aa36740 1 -- 172.21.15.5:0/3705257831 wait complete. 2024-09-19T06:21:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.357+0000 7f675aa36740 20 client.20112 trim_cache size 0 max 16384 2024-09-19T06:21:34.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.393+0000 7f675aa36740 1 Processor -- start 2024-09-19T06:21:34.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.393+0000 7f675aa36740 1 -- start start 2024-09-19T06:21:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.393+0000 7f675aa36740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 0x55b92a7eb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.393+0000 7f675aa36740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d4dc0 0x55b92a7d77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.393+0000 7f675aa36740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7df680 0x55b92a7f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.393+0000 7f675aa36740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67501ab5c0 con 0x55b92a7df680 2024-09-19T06:21:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.393+0000 7f675aa36740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f67501ad100 con 0x55b92a7d4dc0 2024-09-19T06:21:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.393+0000 7f675aa36740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f67501abc90 con 0x55b92a7d44a0 2024-09-19T06:21:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.393+0000 7f67563f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 0x55b92a7eb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.393+0000 7f67563f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 0x55b92a7eb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38956/0 (socket says 172.21.15.5:38956) 2024-09-19T06:21:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.393+0000 7f67563f9640 1 -- 172.21.15.5:0/4073274902 learned_addr learned my addr 172.21.15.5:0/4073274902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.393+0000 7f6755bf8640 1 --2- 172.21.15.5:0/4073274902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d4dc0 0x55b92a7d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.394+0000 7f6756bfa640 1 --2- 172.21.15.5:0/4073274902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7df680 0x55b92a7f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.394+0000 7f67563f9640 1 -- 172.21.15.5:0/4073274902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d4dc0 msgr2=0x55b92a7d77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.394+0000 7f67563f9640 1 --2- 172.21.15.5:0/4073274902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d4dc0 0x55b92a7d77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.394+0000 7f67563f9640 1 -- 172.21.15.5:0/4073274902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7df680 msgr2=0x55b92a7f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.394+0000 7f67563f9640 1 --2- 172.21.15.5:0/4073274902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7df680 0x55b92a7f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.394+0000 7f67563f9640 1 -- 172.21.15.5:0/4073274902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b92a7d42d0 con 0x55b92a7d44a0 2024-09-19T06:21:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.394+0000 7f6756bfa640 1 --2- 172.21.15.5:0/4073274902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7df680 0x55b92a7f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:21:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.394+0000 7f6755bf8640 1 --2- 172.21.15.5:0/4073274902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d4dc0 0x55b92a7d77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:21:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.394+0000 7f67563f9640 1 --2- 172.21.15.5:0/4073274902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 0x55b92a7eb550 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f674c02d490 tx=0x7f674c009420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25701bdcb0aa8cf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.394+0000 7f673bfff640 1 -- 172.21.15.5:0/4073274902 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c010840 con 0x55b92a7d44a0 2024-09-19T06:21:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.394+0000 7f673bfff640 1 -- 172.21.15.5:0/4073274902 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f674c0109e0 con 0x55b92a7d44a0 2024-09-19T06:21:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.394+0000 7f673bfff640 1 -- 172.21.15.5:0/4073274902 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c02de40 con 0x55b92a7d44a0 2024-09-19T06:21:34.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.395+0000 7f675aa36740 1 -- 172.21.15.5:0/4073274902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 msgr2=0x55b92a7eb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:34.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.395+0000 7f675aa36740 1 --2- 172.21.15.5:0/4073274902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 0x55b92a7eb550 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f674c02d490 tx=0x7f674c009420 comp rx=0 tx=0).stop 2024-09-19T06:21:34.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.395+0000 7f675aa36740 1 -- 172.21.15.5:0/4073274902 shutdown_connections 2024-09-19T06:21:34.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.395+0000 7f675aa36740 1 --2- 172.21.15.5:0/4073274902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7df680 0x55b92a7f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.395+0000 7f675aa36740 1 --2- 172.21.15.5:0/4073274902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7d4dc0 0x55b92a7d77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.395+0000 7f675aa36740 1 --2- 172.21.15.5:0/4073274902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 0x55b92a7eb550 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.395+0000 7f675aa36740 1 -- 172.21.15.5:0/4073274902 >> 172.21.15.5:0/4073274902 conn(0x55b92a7d0a30 msgr2=0x55b92a7d70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.396+0000 7f675aa36740 1 -- 172.21.15.5:0/4073274902 shutdown_connections 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.396+0000 7f675aa36740 1 -- 172.21.15.5:0/4073274902 wait complete. 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.396+0000 7f675aa36740 1 -- start start 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.396+0000 7f675aa36740 1 Processor -- start 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.397+0000 7f675aa36740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 0x55b92a7e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.397+0000 7f675aa36740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d4dc0 0x55b92a7e2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.397+0000 7f675aa36740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7df680 0x55b92a7e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.397+0000 7f675aa36740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67501aca30 con 0x55b92a7d4dc0 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.397+0000 7f6755bf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d4dc0 0x55b92a7e2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.397+0000 7f6755bf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d4dc0 0x55b92a7e2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39518/0 (socket says 172.21.15.5:39518) 2024-09-19T06:21:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.397+0000 7f67563f9640 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 0x55b92a7e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.397+0000 7f6755bf8640 1 -- 172.21.15.5:0/865422702 learned_addr learned my addr 172.21.15.5:0/865422702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.397+0000 7f675aa36740 1 -- 172.21.15.5:0/865422702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f67501ac360 con 0x55b92a7df680 2024-09-19T06:21:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.397+0000 7f675aa36740 1 -- 172.21.15.5:0/865422702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f67501a95c0 con 0x55b92a7d44a0 2024-09-19T06:21:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.397+0000 7f6756bfa640 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7df680 0x55b92a7e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.398+0000 7f675aa36740 10 client.0 _ll_register_callbacks cb 0x7f674c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.398+0000 7f6756bfa640 1 -- 172.21.15.5:0/865422702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 msgr2=0x55b92a7e7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.398+0000 7f6756bfa640 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 0x55b92a7e7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.398+0000 7f6756bfa640 1 -- 172.21.15.5:0/865422702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d4dc0 msgr2=0x55b92a7e2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.398+0000 7f6756bfa640 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d4dc0 0x55b92a7e2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.398+0000 7f6756bfa640 1 -- 172.21.15.5:0/865422702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b92a7e38c0 con 0x55b92a7df680 2024-09-19T06:21:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.398+0000 7f6755bf8640 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d4dc0 0x55b92a7e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.398+0000 7f67563f9640 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 0x55b92a7e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:21:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.398+0000 7f6756bfa640 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7df680 0x55b92a7e3380 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55b92a7d0420 tx=0x7f6748026bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7adc9886e5d2f24b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.399+0000 7f6739ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:21:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.399+0000 7f6739ffb640 1 -- 172.21.15.5:0/865422702 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674801b740 con 0x55b92a7df680 2024-09-19T06:21:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.399+0000 7f6739ffb640 1 -- 172.21.15.5:0/865422702 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f674801b8e0 con 0x55b92a7df680 2024-09-19T06:21:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.399+0000 7f675aa36740 1 -- 172.21.15.5:0/865422702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f674c0109e0 con 0x55b92a7df680 2024-09-19T06:21:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.399+0000 7f6739ffb640 1 -- 172.21.15.5:0/865422702 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f674803c050 con 0x55b92a7df680 2024-09-19T06:21:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.399+0000 7f675aa36740 10 client.29489 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.399+0000 7f675aa36740 1 -- 172.21.15.5:0/865422702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55b92a859b80 con 0x55b92a7df680 2024-09-19T06:21:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.400+0000 7f6739ffb640 1 -- 172.21.15.5:0/865422702 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6748033070 con 0x55b92a7df680 2024-09-19T06:21:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.400+0000 7f6739ffb640 1 -- 172.21.15.5:0/865422702 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f674802e480 con 0x55b92a7df680 2024-09-19T06:21:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.400+0000 7f675aa36740 10 client.29489 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.400+0000 7f675aa36740 10 client.29489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.400+0000 7f675aa36740 20 client.29489 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.400+0000 7f675aa36740 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b92a85a130 0x55b92a85c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.400+0000 7f675aa36740 4 client.29489 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:21:34.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.400+0000 7f675aa36740 1 -- 172.21.15.5:0/865422702 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f674c0109e0 con 0x55b92a85a130 2024-09-19T06:21:34.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.400+0000 7f67563f9640 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b92a85a130 0x55b92a85c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:34.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f6739ffb640 10 client.29489 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:34.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f67563f9640 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b92a85a130 0x55b92a85c570 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6660be7a9971702b server_cookie=5fc291683ea7ea88 in_seq=0 out_seq=0 2024-09-19T06:21:34.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f6739ffb640 1 -- 172.21.15.5:0/865422702 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f674c0109e0 con 0x55b92a85a130 2024-09-19T06:21:34.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f6739ffb640 10 client.29489 handle_command_reply: tid=0 2024-09-19T06:21:34.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f675aa36740 1 client.29489 shutdown 2024-09-19T06:21:34.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f675aa36740 1 -- 172.21.15.5:0/865422702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7df680 msgr2=0x55b92a7e3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:34.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f675aa36740 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7df680 0x55b92a7e3380 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55b92a7d0420 tx=0x7f6748026bf0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f675aa36740 1 -- 172.21.15.5:0/865422702 shutdown_connections 2024-09-19T06:21:34.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f675aa36740 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b92a85a130 0x55b92a85c570 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f675aa36740 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b92a7df680 0x55b92a7e3380 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f6739ffb640 0 client.29489 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f675aa36740 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b92a7d4dc0 0x55b92a7e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f675aa36740 1 --2- 172.21.15.5:0/865422702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b92a7d44a0 0x55b92a7e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f675aa36740 1 -- 172.21.15.5:0/865422702 >> 172.21.15.5:0/865422702 conn(0x55b92a7d0a30 msgr2=0x55b92a7d6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.401+0000 7f675aa36740 1 -- 172.21.15.5:0/865422702 shutdown_connections 2024-09-19T06:21:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.402+0000 7f675aa36740 1 -- 172.21.15.5:0/865422702 wait complete. 2024-09-19T06:21:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.402+0000 7f675aa36740 20 client.29489 trim_cache size 0 max 16384 2024-09-19T06:21:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:21:34.408 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:21:34.408 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:21:34.408 INFO:teuthology.orchestra.run.smithi005.stdout: "c8f28885-dd81-436e-9bf6-c06a6e41a087": { 2024-09-19T06:21:34.408 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:21:34.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "c8f28885-dd81-436e-9bf6-c06a6e41a087", 2024-09-19T06:21:34.408 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:21:34.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:21:34.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:21:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:21:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.402+0000 7f675865d640 1 -- 172.21.15.5:0/1037441196 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f673c0a33b0 msgr2=0x7f673c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.402+0000 7f675865d640 1 --2- 172.21.15.5:0/1037441196 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f673c0a33b0 0x7f673c0a5870 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6748023000 tx=0x7f6748024000 comp rx=0 tx=0).stop 2024-09-19T06:21:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.402+0000 7f675865d640 1 -- 172.21.15.5:0/1037441196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6750102790 msgr2=0x7f6750198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.402+0000 7f675865d640 1 --2- 172.21.15.5:0/1037441196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6750102790 0x7f6750198880 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6740005940 tx=0x7f67400040c0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.402+0000 7f675865d640 1 -- 172.21.15.5:0/1037441196 shutdown_connections 2024-09-19T06:21:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.402+0000 7f675865d640 1 --2- 172.21.15.5:0/1037441196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6750106f10 0x7f675019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.402+0000 7f675865d640 1 --2- 172.21.15.5:0/1037441196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6750102790 0x7f6750198880 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.402+0000 7f675865d640 1 --2- 172.21.15.5:0/1037441196 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f673c0a33b0 0x7f673c0a5870 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.402+0000 7f675865d640 1 --2- 172.21.15.5:0/1037441196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6750101de0 0x7f6750198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.403+0000 7f675865d640 1 -- 172.21.15.5:0/1037441196 >> 172.21.15.5:0/1037441196 conn(0x7f67500fd880 msgr2=0x7f67500fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.403+0000 7f675865d640 1 -- 172.21.15.5:0/1037441196 shutdown_connections 2024-09-19T06:21:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:21:34.403+0000 7f675865d640 1 -- 172.21.15.5:0/1037441196 wait complete. 2024-09-19T06:21:34.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 49279 8.69 MB/sec execute 178 sec latency 93.618 ms 2024-09-19T06:21:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:34 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:34 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:34 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:34 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:34 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:34 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:21:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:34 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:34 smithi157 ceph-mon[35346]: pgmap v574: 97 pgs: 97 active+clean; 853 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 381 op/s 2024-09-19T06:21:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:34 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:34 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:34 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:34 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:34 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:34 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:21:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:34 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:34 smithi005 ceph-mon[30955]: pgmap v574: 97 pgs: 97 active+clean; 853 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 381 op/s 2024-09-19T06:21:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:34 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:34 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:34 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:34 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:34 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:34 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:21:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:34 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:21:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:34 smithi049 ceph-mon[34640]: pgmap v574: 97 pgs: 97 active+clean; 853 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 381 op/s 2024-09-19T06:21:35.266 INFO:tasks.cephfs.filesystem:scrub status for tag:c8f28885-dd81-436e-9bf6-c06a6e41a087 - {'path': '/', 'tag': 'c8f28885-dd81-436e-9bf6-c06a6e41a087', 'options': 'recursive,force'} 2024-09-19T06:21:35.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 49475 8.70 MB/sec execute 179 sec latency 57.034 ms 2024-09-19T06:21:36.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 49685 8.72 MB/sec execute 180 sec latency 197.184 ms 2024-09-19T06:21:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:36 smithi157 ceph-mon[35346]: pgmap v575: 97 pgs: 97 active+clean; 853 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2024-09-19T06:21:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:36 smithi005 ceph-mon[30955]: pgmap v575: 97 pgs: 97 active+clean; 853 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2024-09-19T06:21:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:36 smithi049 ceph-mon[34640]: pgmap v575: 97 pgs: 97 active+clean; 853 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2024-09-19T06:21:37.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 49849 8.73 MB/sec execute 181 sec latency 56.845 ms 2024-09-19T06:21:38.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 49982 8.70 MB/sec execute 182 sec latency 144.198 ms 2024-09-19T06:21:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:38 smithi157 ceph-mon[35346]: pgmap v576: 97 pgs: 97 active+clean; 859 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 465 op/s 2024-09-19T06:21:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:38 smithi005 ceph-mon[30955]: pgmap v576: 97 pgs: 97 active+clean; 859 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 465 op/s 2024-09-19T06:21:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:38 smithi049 ceph-mon[34640]: pgmap v576: 97 pgs: 97 active+clean; 859 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 465 op/s 2024-09-19T06:21:39.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 50193 8.76 MB/sec execute 183 sec latency 99.686 ms 2024-09-19T06:21:40.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 50402 8.79 MB/sec execute 184 sec latency 87.986 ms 2024-09-19T06:21:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:40 smithi157 ceph-mon[35346]: pgmap v577: 97 pgs: 97 active+clean; 859 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2024-09-19T06:21:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:40 smithi005 ceph-mon[30955]: pgmap v577: 97 pgs: 97 active+clean; 859 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2024-09-19T06:21:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:40 smithi049 ceph-mon[34640]: pgmap v577: 97 pgs: 97 active+clean; 859 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2024-09-19T06:21:41.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 50559 8.78 MB/sec execute 185 sec latency 84.361 ms 2024-09-19T06:21:42.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 50699 8.77 MB/sec execute 186 sec latency 50.801 ms 2024-09-19T06:21:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:42 smithi157 ceph-mon[35346]: pgmap v578: 97 pgs: 97 active+clean; 859 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2024-09-19T06:21:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:42 smithi005 ceph-mon[30955]: pgmap v578: 97 pgs: 97 active+clean; 859 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2024-09-19T06:21:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:42 smithi049 ceph-mon[34640]: pgmap v578: 97 pgs: 97 active+clean; 859 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2024-09-19T06:21:43.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 50846 8.77 MB/sec execute 187 sec latency 57.997 ms 2024-09-19T06:21:44.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51003 8.77 MB/sec execute 188 sec latency 71.504 ms 2024-09-19T06:21:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:44 smithi157 ceph-mon[35346]: pgmap v579: 97 pgs: 97 active+clean; 876 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 456 op/s 2024-09-19T06:21:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:21:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:21:44.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:21:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:44 smithi005 ceph-mon[30955]: pgmap v579: 97 pgs: 97 active+clean; 876 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 456 op/s 2024-09-19T06:21:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:21:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:21:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:21:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:44 smithi049 ceph-mon[34640]: pgmap v579: 97 pgs: 97 active+clean; 876 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 456 op/s 2024-09-19T06:21:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:21:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:21:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:21:45.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51192 8.81 MB/sec execute 189 sec latency 100.611 ms 2024-09-19T06:21:45.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:45 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:21:45.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:45 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:21:45.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:45 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:21:45.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:45 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:21:45.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:45 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:21:45.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:45 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:21:46.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51366 8.79 MB/sec execute 190 sec latency 59.926 ms 2024-09-19T06:21:46.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:46 smithi157 ceph-mon[35346]: pgmap v580: 97 pgs: 97 active+clean; 876 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 320 op/s 2024-09-19T06:21:46.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:46 smithi005 ceph-mon[30955]: pgmap v580: 97 pgs: 97 active+clean; 876 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 320 op/s 2024-09-19T06:21:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:46 smithi049 ceph-mon[34640]: pgmap v580: 97 pgs: 97 active+clean; 876 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 320 op/s 2024-09-19T06:21:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.918+0000 7f0db4178640 1 -- 172.21.15.5:0/305576715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac1009f0 msgr2=0x7f0dac100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.918+0000 7f0db4178640 1 --2- 172.21.15.5:0/305576715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac1009f0 0x7f0dac100e50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009930 tx=0x7f0da002a420 comp rx=0 tx=0).stop 2024-09-19T06:21:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.918+0000 7f0db4178640 1 -- 172.21.15.5:0/305576715 shutdown_connections 2024-09-19T06:21:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.918+0000 7f0db4178640 1 --2- 172.21.15.5:0/305576715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac101390 0x7f0dac10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.919+0000 7f0db4178640 1 --2- 172.21.15.5:0/305576715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac1009f0 0x7f0dac100e50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.919+0000 7f0db4178640 1 --2- 172.21.15.5:0/305576715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dac106a00 0x7f0dac106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.919+0000 7f0db4178640 1 -- 172.21.15.5:0/305576715 >> 172.21.15.5:0/305576715 conn(0x7f0dac0fc6f0 msgr2=0x7f0dac0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.919+0000 7f0db4178640 1 -- 172.21.15.5:0/305576715 shutdown_connections 2024-09-19T06:21:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.919+0000 7f0db4178640 1 -- 172.21.15.5:0/305576715 wait complete. 2024-09-19T06:21:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.919+0000 7f0db4178640 1 Processor -- start 2024-09-19T06:21:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.919+0000 7f0db4178640 1 -- start start 2024-09-19T06:21:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db4178640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac1009f0 0x7f0dac197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db4178640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac101390 0x7f0dac197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db4178640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dac106a00 0x7f0dac19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db4178640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac110cf0 con 0x7f0dac101390 2024-09-19T06:21:46.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db4178640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac110b70 con 0x7f0dac106a00 2024-09-19T06:21:46.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db4178640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac110e70 con 0x7f0dac1009f0 2024-09-19T06:21:46.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db1f14640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac1009f0 0x7f0dac197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db1713640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac101390 0x7f0dac197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db1f14640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac1009f0 0x7f0dac197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54644/0 (socket says 172.21.15.5:54644) 2024-09-19T06:21:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db1f14640 1 -- 172.21.15.5:0/3563197021 learned_addr learned my addr 172.21.15.5:0/3563197021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db1713640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac101390 0x7f0dac197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56870/0 (socket says 172.21.15.5:56870) 2024-09-19T06:21:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db2715640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dac106a00 0x7f0dac19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db1f14640 1 -- 172.21.15.5:0/3563197021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dac106a00 msgr2=0x7f0dac19bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db1f14640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dac106a00 0x7f0dac19bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db1f14640 1 -- 172.21.15.5:0/3563197021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac101390 msgr2=0x7f0dac197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db1f14640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac101390 0x7f0dac197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db1f14640 1 -- 172.21.15.5:0/3563197021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac19c210 con 0x7f0dac1009f0 2024-09-19T06:21:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.920+0000 7f0db1713640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac101390 0x7f0dac197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.921+0000 7f0db1f14640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac1009f0 0x7f0dac197220 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00eb90 tx=0x7f0d9c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.921+0000 7f0daaffd640 1 -- 172.21.15.5:0/3563197021 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c004d20 con 0x7f0dac1009f0 2024-09-19T06:21:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.921+0000 7f0db4178640 1 -- 172.21.15.5:0/3563197021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac19c500 con 0x7f0dac1009f0 2024-09-19T06:21:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.921+0000 7f0daaffd640 1 -- 172.21.15.5:0/3563197021 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d9c00ee90 con 0x7f0dac1009f0 2024-09-19T06:21:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.921+0000 7f0daaffd640 1 -- 172.21.15.5:0/3563197021 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c010670 con 0x7f0dac1009f0 2024-09-19T06:21:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.921+0000 7f0db4178640 1 -- 172.21.15.5:0/3563197021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac1a3d90 con 0x7f0dac1009f0 2024-09-19T06:21:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.924+0000 7f0db6ff1740 1 Processor -- start 2024-09-19T06:21:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0daaffd640 1 -- 172.21.15.5:0/3563197021 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0d9c005020 con 0x7f0dac1009f0 2024-09-19T06:21:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db6ff1740 1 -- start start 2024-09-19T06:21:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db6ff1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x5613098f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db6ff1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613098f8620 0x561309902980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db6ff1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561309902ec0 0x5613099052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db6ff1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56130985a2f0 con 0x5613098f7d00 2024-09-19T06:21:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db6ff1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561309802270 con 0x5613098f8620 2024-09-19T06:21:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db6ff1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5613098562b0 con 0x561309902ec0 2024-09-19T06:21:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db2715640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x5613098f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db1f14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613098f8620 0x561309902980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db2715640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x5613098f80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56884/0 (socket says 172.21.15.5:56884) 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db2715640 1 -- 172.21.15.5:0/3157173289 learned_addr learned my addr 172.21.15.5:0/3157173289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db1713640 1 --2- 172.21.15.5:0/3157173289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561309902ec0 0x5613099052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db1713640 1 -- 172.21.15.5:0/3157173289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613098f8620 msgr2=0x561309902980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db1713640 1 --2- 172.21.15.5:0/3157173289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613098f8620 0x561309902980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db1713640 1 -- 172.21.15.5:0/3157173289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 msgr2=0x5613098f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.925+0000 7f0db1713640 1 --2- 172.21.15.5:0/3157173289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x5613098f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.926+0000 7f0db1713640 1 -- 172.21.15.5:0/3157173289 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561309801340 con 0x561309902ec0 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.926+0000 7f0db1713640 1 --2- 172.21.15.5:0/3157173289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561309902ec0 0x5613099052b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0da0031440 tx=0x7f0da0032eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d3eeb1e66815470 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.926+0000 7f0d93fff640 1 -- 172.21.15.5:0/3157173289 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0035040 con 0x561309902ec0 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.926+0000 7f0d93fff640 1 -- 172.21.15.5:0/3157173289 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0da00317a0 con 0x561309902ec0 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.926+0000 7f0d93fff640 1 -- 172.21.15.5:0/3157173289 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0003dc0 con 0x561309902ec0 2024-09-19T06:21:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.926+0000 7f0daaffd640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0d940a33b0 0x7f0d940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.927+0000 7f0db2715640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0d940a33b0 0x7f0d940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.927+0000 7f0daaffd640 1 -- 172.21.15.5:0/3563197021 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f0d9c0d02d0 con 0x7f0dac1009f0 2024-09-19T06:21:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.927+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3157173289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561309902ec0 msgr2=0x5613099052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.927+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/3157173289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561309902ec0 0x5613099052b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0da0031440 tx=0x7f0da0032eb0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.927+0000 7f0db2715640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0d940a33b0 0x7f0d940a5870 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009ee0 tx=0x7f0da4005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.927+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3157173289 shutdown_connections 2024-09-19T06:21:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.927+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/3157173289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561309902ec0 0x5613099052b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.927+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/3157173289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613098f8620 0x561309902980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.927+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/3157173289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x5613098f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.927+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3157173289 >> 172.21.15.5:0/3157173289 conn(0x5613098f4270 msgr2=0x5613098f4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.927+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3157173289 shutdown_connections 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.928+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3157173289 wait complete. 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.928+0000 7f0db6ff1740 1 -- start start 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.928+0000 7f0db6ff1740 1 Processor -- start 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db6ff1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613098f7d00 0x5613098ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db6ff1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f8620 0x5613099002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db6ff1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561309902ec0 0x561309900820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db6ff1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561309905a90 con 0x5613098f8620 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db6ff1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5613097e3290 con 0x5613098f7d00 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db6ff1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561309905c10 con 0x561309902ec0 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db1f14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613098f7d00 0x5613098ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db6ff1740 10 client.0 _ll_register_callbacks cb 0x561309814140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db1713640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f8620 0x5613099002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db1713640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f8620 0x5613099002e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56890/0 (socket says 172.21.15.5:56890) 2024-09-19T06:21:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db1713640 1 -- 172.21.15.5:0/1608867067 learned_addr learned my addr 172.21.15.5:0/1608867067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db1f14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613098f7d00 0x5613098ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40064/0 (socket says 172.21.15.5:40064) 2024-09-19T06:21:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db2715640 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561309902ec0 0x561309900820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db1f14640 1 -- 172.21.15.5:0/1608867067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561309902ec0 msgr2=0x561309900820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db1f14640 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561309902ec0 0x561309900820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db1f14640 1 -- 172.21.15.5:0/1608867067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f8620 msgr2=0x5613099002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db1f14640 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f8620 0x5613099002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db1f14640 1 -- 172.21.15.5:0/1608867067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56130956d210 con 0x5613098f7d00 2024-09-19T06:21:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db1713640 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f8620 0x5613099002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0db1f14640 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613098f7d00 0x5613098ffda0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0cda00 tx=0x7f0d9c0d3950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc76648b2faf9365 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.929+0000 7f0d917fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.930+0000 7f0d917fa640 1 -- 172.21.15.5:0/1608867067 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c09c360 con 0x5613098f7d00 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.930+0000 7f0db6ff1740 1 -- 172.21.15.5:0/1608867067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da00317a0 con 0x5613098f7d00 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.930+0000 7f0d917fa640 1 -- 172.21.15.5:0/1608867067 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f0d9c005560 con 0x5613098f7d00 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.930+0000 7f0d917fa640 1 -- 172.21.15.5:0/1608867067 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f0d9c09bef0 con 0x5613098f7d00 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.930+0000 7f0db6ff1740 10 client.29501 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.930+0000 7f0db6ff1740 1 -- 172.21.15.5:0/1608867067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561309639620 con 0x5613098f7d00 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.930+0000 7f0d917fa640 1 -- 172.21.15.5:0/1608867067 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0da00317a0 con 0x5613098f7d00 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.930+0000 7f0d917fa640 1 -- 172.21.15.5:0/1608867067 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f0d9c098c00 con 0x5613098f7d00 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.931+0000 7f0db6ff1740 10 client.29501 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.931+0000 7f0db6ff1740 10 client.29501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.931+0000 7f0db6ff1740 20 client.29501 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.931+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5613098f2d00 0x5613098f3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.931+0000 7f0db6ff1740 4 client.29501 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:21:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.931+0000 7f0db6ff1740 1 -- 172.21.15.5:0/1608867067 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561309867a20 con 0x5613098f2d00 2024-09-19T06:21:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.931+0000 7f0db1713640 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5613098f2d00 0x5613098f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.931+0000 7f0d917fa640 10 client.29501 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.931+0000 7f0db1713640 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5613098f2d00 0x5613098f3160 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25862fbc0337aa22 server_cookie=c9280894a1f9d740 in_seq=0 out_seq=0 2024-09-19T06:21:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.932+0000 7f0d917fa640 1 -- 172.21.15.5:0/1608867067 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561309867a20 con 0x5613098f2d00 2024-09-19T06:21:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.932+0000 7f0d917fa640 10 client.29501 handle_command_reply: tid=0 2024-09-19T06:21:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.932+0000 7f0db6ff1740 1 client.29501 shutdown 2024-09-19T06:21:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.932+0000 7f0db6ff1740 1 -- 172.21.15.5:0/1608867067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613098f7d00 msgr2=0x5613098ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.932+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613098f7d00 0x5613098ffda0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0cda00 tx=0x7f0d9c0d3950 comp rx=0 tx=0).stop 2024-09-19T06:21:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.932+0000 7f0db6ff1740 1 -- 172.21.15.5:0/1608867067 shutdown_connections 2024-09-19T06:21:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.932+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5613098f2d00 0x5613098f3160 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.932+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561309902ec0 0x561309900820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.932+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f8620 0x5613099002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.932+0000 7f0d917fa640 0 client.29501 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:46.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.932+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/1608867067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613098f7d00 0x5613098ffda0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.932+0000 7f0db6ff1740 1 -- 172.21.15.5:0/1608867067 >> 172.21.15.5:0/1608867067 conn(0x5613098f4270 msgr2=0x5613098f11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:46.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.933+0000 7f0db6ff1740 1 -- 172.21.15.5:0/1608867067 shutdown_connections 2024-09-19T06:21:46.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.933+0000 7f0db6ff1740 1 -- 172.21.15.5:0/1608867067 wait complete. 2024-09-19T06:21:46.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.933+0000 7f0db6ff1740 20 client.29501 trim_cache size 0 max 16384 2024-09-19T06:21:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.960+0000 7f0db6ff1740 1 Processor -- start 2024-09-19T06:21:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.960+0000 7f0db6ff1740 1 -- start start 2024-09-19T06:21:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.960+0000 7f0db6ff1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x561309901d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db6ff1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613098f8620 0x561309916a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db6ff1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561309902ec0 0x561309916fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db6ff1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1ab5d0 con 0x5613098f7d00 2024-09-19T06:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db6ff1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1ad110 con 0x561309902ec0 2024-09-19T06:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db6ff1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1abca0 con 0x5613098f8620 2024-09-19T06:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db1f14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x561309901d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db1713640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613098f8620 0x561309916a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db1f14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x561309901d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56896/0 (socket says 172.21.15.5:56896) 2024-09-19T06:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db1f14640 1 -- 172.21.15.5:0/39304255 learned_addr learned my addr 172.21.15.5:0/39304255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db2715640 1 --2- 172.21.15.5:0/39304255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561309902ec0 0x561309916fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db1713640 1 -- 172.21.15.5:0/39304255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561309902ec0 msgr2=0x561309916fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db1713640 1 --2- 172.21.15.5:0/39304255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561309902ec0 0x561309916fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db1713640 1 -- 172.21.15.5:0/39304255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 msgr2=0x561309901d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db1713640 1 --2- 172.21.15.5:0/39304255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x561309901d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db1713640 1 -- 172.21.15.5:0/39304255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613098f40a0 con 0x5613098f8620 2024-09-19T06:21:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db1f14640 1 --2- 172.21.15.5:0/39304255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x561309901d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:21:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.961+0000 7f0db1713640 1 --2- 172.21.15.5:0/39304255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613098f8620 0x561309916a80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0da00057a0 tx=0x7f0da002a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b54225653e4486f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.962+0000 7f0d937fe640 1 -- 172.21.15.5:0/39304255 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0035040 con 0x5613098f8620 2024-09-19T06:21:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.962+0000 7f0d937fe640 1 -- 172.21.15.5:0/39304255 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0da0004030 con 0x5613098f8620 2024-09-19T06:21:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.962+0000 7f0d937fe640 1 -- 172.21.15.5:0/39304255 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da004ac70 con 0x5613098f8620 2024-09-19T06:21:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.962+0000 7f0db6ff1740 1 -- 172.21.15.5:0/39304255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613098f8620 msgr2=0x561309916a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.962+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/39304255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613098f8620 0x561309916a80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0da00057a0 tx=0x7f0da002a420 comp rx=0 tx=0).stop 2024-09-19T06:21:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.963+0000 7f0db6ff1740 1 -- 172.21.15.5:0/39304255 shutdown_connections 2024-09-19T06:21:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.963+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/39304255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561309902ec0 0x561309916fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.963+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/39304255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613098f8620 0x561309916a80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.963+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/39304255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x561309901d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.963+0000 7f0db6ff1740 1 -- 172.21.15.5:0/39304255 >> 172.21.15.5:0/39304255 conn(0x5613098f4270 msgr2=0x561309901600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.963+0000 7f0db6ff1740 1 -- 172.21.15.5:0/39304255 shutdown_connections 2024-09-19T06:21:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.963+0000 7f0db6ff1740 1 -- 172.21.15.5:0/39304255 wait complete. 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.963+0000 7f0db6ff1740 1 -- start start 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.963+0000 7f0db6ff1740 1 Processor -- start 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db6ff1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x5613099170f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db6ff1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613098f8620 0x561309914430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db6ff1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561309902ec0 0x561309914970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db6ff1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1aca40 con 0x5613098f7d00 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db6ff1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1ac370 con 0x561309902ec0 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db6ff1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1a95d0 con 0x5613098f8620 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db6ff1740 10 client.0 _ll_register_callbacks cb 0x7f0d9c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db1f14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x5613099170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db1f14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x5613099170f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56912/0 (socket says 172.21.15.5:56912) 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db1f14640 1 -- 172.21.15.5:0/3140615062 learned_addr learned my addr 172.21.15.5:0/3140615062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:21:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db1713640 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613098f8620 0x561309914430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db2715640 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561309902ec0 0x561309914970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db1f14640 1 -- 172.21.15.5:0/3140615062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613098f8620 msgr2=0x561309914430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db1f14640 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613098f8620 0x561309914430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db1f14640 1 -- 172.21.15.5:0/3140615062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561309902ec0 msgr2=0x561309914970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db1f14640 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561309902ec0 0x561309914970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db1f14640 1 -- 172.21.15.5:0/3140615062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613099151a0 con 0x5613098f7d00 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.964+0000 7f0db1f14640 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x5613099170f0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0ce000 tx=0x7f0d9c098230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b607d5a8a09bc41d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.965+0000 7f0d917fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.965+0000 7f0d917fa640 1 -- 172.21.15.5:0/3140615062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c09c0d0 con 0x5613098f7d00 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.965+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3140615062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da0004030 con 0x5613098f7d00 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.965+0000 7f0d917fa640 1 -- 172.21.15.5:0/3140615062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7f0d9c0d3a40 con 0x5613098f7d00 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.965+0000 7f0d917fa640 1 -- 172.21.15.5:0/3140615062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f0d9c07c020 con 0x5613098f7d00 2024-09-19T06:21:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.965+0000 7f0db6ff1740 10 client.20136 fetch_fsmap learned FSMap version 58 2024-09-19T06:21:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.965+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3140615062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56130997ba50 con 0x5613098f7d00 2024-09-19T06:21:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.965+0000 7f0d917fa640 1 -- 172.21.15.5:0/3140615062 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0da0004030 con 0x5613098f7d00 2024-09-19T06:21:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.965+0000 7f0d917fa640 1 -- 172.21.15.5:0/3140615062 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f0d9c0b0ab0 con 0x5613098f7d00 2024-09-19T06:21:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.965+0000 7f0db6ff1740 10 client.20136 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:21:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.965+0000 7f0db6ff1740 10 client.20136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:21:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.966+0000 7f0db6ff1740 20 client.20136 populate_metadata read hostname 'smithi005' 2024-09-19T06:21:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.966+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56130997c150 0x56130997e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:21:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.966+0000 7f0db6ff1740 4 client.20136 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:21:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.966+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3140615062 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561309861d70 con 0x56130997c150 2024-09-19T06:21:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.966+0000 7f0db1713640 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56130997c150 0x56130997e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:21:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.966+0000 7f0d917fa640 10 client.20136 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:46.966+0000 7f0db1713640 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56130997c150 0x56130997e570 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b23e80b3842339 server_cookie=766f996d57c723e2 in_seq=0 out_seq=0 2024-09-19T06:21:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.032+0000 7f0d917fa640 1 -- 172.21.15.5:0/3140615062 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561309861d70 con 0x56130997c150 2024-09-19T06:21:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.032+0000 7f0d917fa640 10 client.20136 handle_command_reply: tid=0 2024-09-19T06:21:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.032+0000 7f0db6ff1740 1 client.20136 shutdown 2024-09-19T06:21:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.033+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3140615062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 msgr2=0x5613099170f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.033+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x5613099170f0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0ce000 tx=0x7f0d9c098230 comp rx=0 tx=0).stop 2024-09-19T06:21:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.033+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3140615062 shutdown_connections 2024-09-19T06:21:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.033+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56130997c150 0x56130997e570 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.033+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561309902ec0 0x561309914970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.033+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613098f8620 0x561309914430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.033+0000 7f0db6ff1740 1 --2- 172.21.15.5:0/3140615062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613098f7d00 0x5613099170f0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.033+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3140615062 >> 172.21.15.5:0/3140615062 conn(0x5613098f4270 msgr2=0x56130990dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.033+0000 7f0d917fa640 0 client.20136 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:21:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.033+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3140615062 shutdown_connections 2024-09-19T06:21:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.033+0000 7f0db6ff1740 1 -- 172.21.15.5:0/3140615062 wait complete. 2024-09-19T06:21:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.033+0000 7f0db6ff1740 20 client.20136 trim_cache size 0 max 16384 2024-09-19T06:21:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:21:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:21:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:21:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:21:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.034+0000 7f0db4178640 1 -- 172.21.15.5:0/3563197021 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0d940a33b0 msgr2=0x7f0d940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.034+0000 7f0db4178640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0d940a33b0 0x7f0d940a5870 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009ee0 tx=0x7f0da4005f90 comp rx=0 tx=0).stop 2024-09-19T06:21:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.034+0000 7f0db4178640 1 -- 172.21.15.5:0/3563197021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac1009f0 msgr2=0x7f0dac197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:21:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.034+0000 7f0db4178640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac1009f0 0x7f0dac197220 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00eb90 tx=0x7f0d9c00c6a0 comp rx=0 tx=0).stop 2024-09-19T06:21:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.034+0000 7f0db4178640 1 -- 172.21.15.5:0/3563197021 shutdown_connections 2024-09-19T06:21:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.034+0000 7f0db4178640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dac106a00 0x7f0dac19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.034+0000 7f0db4178640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac101390 0x7f0dac197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.034+0000 7f0db4178640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0d940a33b0 0x7f0d940a5870 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.034+0000 7f0db4178640 1 --2- 172.21.15.5:0/3563197021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac1009f0 0x7f0dac197220 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:21:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.034+0000 7f0db4178640 1 -- 172.21.15.5:0/3563197021 >> 172.21.15.5:0/3563197021 conn(0x7f0dac0fc6f0 msgr2=0x7f0dac0fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:21:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.035+0000 7f0db4178640 1 -- 172.21.15.5:0/3563197021 shutdown_connections 2024-09-19T06:21:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:21:47.035+0000 7f0db4178640 1 -- 172.21.15.5:0/3563197021 wait complete. 2024-09-19T06:21:47.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51512 8.79 MB/sec execute 191 sec latency 79.468 ms 2024-09-19T06:21:48.562 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51660 8.78 MB/sec execute 192 sec latency 92.998 ms 2024-09-19T06:21:48.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:48 smithi157 ceph-mon[35346]: pgmap v581: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.3 MiB/s wr, 440 op/s 2024-09-19T06:21:48.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:48 smithi005 ceph-mon[30955]: pgmap v581: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.3 MiB/s wr, 440 op/s 2024-09-19T06:21:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:48 smithi049 ceph-mon[34640]: pgmap v581: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.3 MiB/s wr, 440 op/s 2024-09-19T06:21:49.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51827 8.78 MB/sec execute 193 sec latency 281.970 ms 2024-09-19T06:21:50.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51987 8.76 MB/sec execute 194 sec latency 133.232 ms 2024-09-19T06:21:50.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:50 smithi157 ceph-mon[35346]: pgmap v582: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 233 op/s 2024-09-19T06:21:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:50 smithi005 ceph-mon[30955]: pgmap v582: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 233 op/s 2024-09-19T06:21:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:50 smithi049 ceph-mon[34640]: pgmap v582: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 233 op/s 2024-09-19T06:21:51.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 52150 8.77 MB/sec execute 195 sec latency 60.470 ms 2024-09-19T06:21:51.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:51 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:21:51.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:51 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:21:51.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:51 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:21:52.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 52311 8.75 MB/sec execute 196 sec latency 126.691 ms 2024-09-19T06:21:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:52 smithi157 ceph-mon[35346]: pgmap v583: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 233 op/s 2024-09-19T06:21:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:21:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:21:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:21:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:52 smithi005 ceph-mon[30955]: pgmap v583: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 233 op/s 2024-09-19T06:21:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:21:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:21:52.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:21:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:52 smithi049 ceph-mon[34640]: pgmap v583: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 233 op/s 2024-09-19T06:21:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:21:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:21:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:21:53.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 52483 8.75 MB/sec execute 197 sec latency 68.204 ms 2024-09-19T06:21:54.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 52629 8.72 MB/sec execute 198 sec latency 60.328 ms 2024-09-19T06:21:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:54 smithi157 ceph-mon[35346]: pgmap v584: 97 pgs: 97 active+clean; 886 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.2 MiB/s wr, 383 op/s 2024-09-19T06:21:54.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:54 smithi005 ceph-mon[30955]: pgmap v584: 97 pgs: 97 active+clean; 886 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.2 MiB/s wr, 383 op/s 2024-09-19T06:21:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:54 smithi049 ceph-mon[34640]: pgmap v584: 97 pgs: 97 active+clean; 886 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.2 MiB/s wr, 383 op/s 2024-09-19T06:21:55.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 52832 8.76 MB/sec execute 199 sec latency 55.028 ms 2024-09-19T06:21:56.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53045 8.76 MB/sec execute 200 sec latency 56.882 ms 2024-09-19T06:21:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:56 smithi157 ceph-mon[35346]: pgmap v585: 97 pgs: 97 active+clean; 886 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-09-19T06:21:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:56 smithi005 ceph-mon[30955]: pgmap v585: 97 pgs: 97 active+clean; 886 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-09-19T06:21:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:56 smithi049 ceph-mon[34640]: pgmap v585: 97 pgs: 97 active+clean; 886 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-09-19T06:21:57.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53210 8.78 MB/sec execute 201 sec latency 54.154 ms 2024-09-19T06:21:58.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53336 8.76 MB/sec execute 202 sec latency 55.924 ms 2024-09-19T06:21:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:21:58 smithi157 ceph-mon[35346]: pgmap v586: 97 pgs: 97 active+clean; 898 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 399 op/s 2024-09-19T06:21:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:21:58 smithi005 ceph-mon[30955]: pgmap v586: 97 pgs: 97 active+clean; 898 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 399 op/s 2024-09-19T06:21:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:21:58 smithi049 ceph-mon[34640]: pgmap v586: 97 pgs: 97 active+clean; 898 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 399 op/s 2024-09-19T06:21:59.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53545 8.80 MB/sec execute 203 sec latency 63.584 ms 2024-09-19T06:22:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.257+0000 7ff9b25b7640 1 -- 172.21.15.5:0/2743417297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ac102d00 msgr2=0x7ff9ac1030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.257+0000 7ff9b25b7640 1 --2- 172.21.15.5:0/2743417297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ac102d00 0x7ff9ac1030e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009900 tx=0x7ff99c02a480 comp rx=0 tx=0).stop 2024-09-19T06:22:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.257+0000 7ff9b25b7640 1 -- 172.21.15.5:0/2743417297 shutdown_connections 2024-09-19T06:22:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.257+0000 7ff9b25b7640 1 --2- 172.21.15.5:0/2743417297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9ac10c0d0 0x7ff9ac10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.257+0000 7ff9b25b7640 1 --2- 172.21.15.5:0/2743417297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9ac103620 0x7ff9ac10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.257+0000 7ff9b25b7640 1 --2- 172.21.15.5:0/2743417297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ac102d00 0x7ff9ac1030e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.257+0000 7ff9b25b7640 1 -- 172.21.15.5:0/2743417297 >> 172.21.15.5:0/2743417297 conn(0x7ff9ac0fc6b0 msgr2=0x7ff9ac0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.257+0000 7ff9b25b7640 1 -- 172.21.15.5:0/2743417297 shutdown_connections 2024-09-19T06:22:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.257+0000 7ff9b25b7640 1 -- 172.21.15.5:0/2743417297 wait complete. 2024-09-19T06:22:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.258+0000 7ff9b25b7640 1 Processor -- start 2024-09-19T06:22:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.258+0000 7ff9b25b7640 1 -- start start 2024-09-19T06:22:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9b25b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ac102d00 0x7ff9ac198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9b25b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9ac103620 0x7ff9ac1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9b25b7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9ac10c0d0 0x7ff9ac19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9b25b7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac111d80 con 0x7ff9ac103620 2024-09-19T06:22:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9b25b7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac111c00 con 0x7ff9ac10c0d0 2024-09-19T06:22:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9b25b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac111f00 con 0x7ff9ac102d00 2024-09-19T06:22:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ac102d00 0x7ff9ac198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ac102d00 0x7ff9ac198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60140/0 (socket says 172.21.15.5:60140) 2024-09-19T06:22:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9abfff640 1 -- 172.21.15.5:0/699811037 learned_addr learned my addr 172.21.15.5:0/699811037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9ab7fe640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9ac103620 0x7ff9ac1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9ac10c0d0 0x7ff9ac19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9abfff640 1 -- 172.21.15.5:0/699811037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9ac10c0d0 msgr2=0x7ff9ac19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9abfff640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9ac10c0d0 0x7ff9ac19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9abfff640 1 -- 172.21.15.5:0/699811037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9ac103620 msgr2=0x7ff9ac1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9abfff640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9ac103620 0x7ff9ac1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.259+0000 7ff9abfff640 1 -- 172.21.15.5:0/699811037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac19d350 con 0x7ff9ac102d00 2024-09-19T06:22:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.260+0000 7ff9abfff640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ac102d00 0x7ff9ac198360 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff99c005940 tx=0x7ff99c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.260+0000 7ff9a97fa640 1 -- 172.21.15.5:0/699811037 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c03f030 con 0x7ff9ac102d00 2024-09-19T06:22:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.260+0000 7ff9a97fa640 1 -- 172.21.15.5:0/699811037 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff99c003cb0 con 0x7ff9ac102d00 2024-09-19T06:22:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.260+0000 7ff9b25b7640 1 -- 172.21.15.5:0/699811037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ac19d5e0 con 0x7ff9ac102d00 2024-09-19T06:22:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.260+0000 7ff9a97fa640 1 -- 172.21.15.5:0/699811037 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c035070 con 0x7ff9ac102d00 2024-09-19T06:22:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.260+0000 7ff9b25b7640 1 -- 172.21.15.5:0/699811037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ac1a4ed0 con 0x7ff9ac102d00 2024-09-19T06:22:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.263+0000 7ff9a97fa640 1 -- 172.21.15.5:0/699811037 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff99c042030 con 0x7ff9ac102d00 2024-09-19T06:22:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.263+0000 7ff9b542a740 1 Processor -- start 2024-09-19T06:22:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b542a740 1 -- start start 2024-09-19T06:22:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b542a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba2d9f0 0x56293ba2ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b542a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 0x56293ba3cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b542a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba3d0c0 0x56293ba3f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b542a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56293b9922f0 con 0x56293ba3d0c0 2024-09-19T06:22:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b542a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56293b93a270 con 0x56293ba2e310 2024-09-19T06:22:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b542a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56293b98e2b0 con 0x56293ba2d9f0 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 0x56293ba3cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b0b54640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba2d9f0 0x56293ba2ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 0x56293ba3cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56406/0 (socket says 172.21.15.5:56406) 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9abfff640 1 -- 172.21.15.5:0/3537239355 learned_addr learned my addr 172.21.15.5:0/3537239355 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9ab7fe640 1 --2- 172.21.15.5:0/3537239355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba3d0c0 0x56293ba3f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b0b54640 1 -- 172.21.15.5:0/3537239355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 msgr2=0x56293ba3cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/3537239355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 0x56293ba3cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b0b54640 1 -- 172.21.15.5:0/3537239355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba3d0c0 msgr2=0x56293ba3f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/3537239355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba3d0c0 0x56293ba3f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.264+0000 7ff9b0b54640 1 -- 172.21.15.5:0/3537239355 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56293b939340 con 0x56293ba2d9f0 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.265+0000 7ff9abfff640 1 --2- 172.21.15.5:0/3537239355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 0x56293ba3cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.265+0000 7ff9a97fa640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9940a33b0 0x7ff9940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.265+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/3537239355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba2d9f0 0x56293ba2ddd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009eb0 tx=0x7ff9a4010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3e9490fe1d5145c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.265+0000 7ff9abfff640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9940a33b0 0x7ff9940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.265+0000 7ff9927fc640 1 -- 172.21.15.5:0/3537239355 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4016070 con 0x56293ba2d9f0 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.265+0000 7ff9a97fa640 1 -- 172.21.15.5:0/699811037 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7ff99c0f0480 con 0x7ff9ac102d00 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.265+0000 7ff9927fc640 1 -- 172.21.15.5:0/3537239355 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4004d20 con 0x56293ba2d9f0 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.265+0000 7ff9927fc640 1 -- 172.21.15.5:0/3537239355 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40126b0 con 0x56293ba2d9f0 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.266+0000 7ff9b542a740 1 -- 172.21.15.5:0/3537239355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba2d9f0 msgr2=0x56293ba2ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.266+0000 7ff9abfff640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9940a33b0 0x7ff9940a5870 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0bc780 tx=0x7ff99c0f6060 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.266+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3537239355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba2d9f0 0x56293ba2ddd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009eb0 tx=0x7ff9a4010740 comp rx=0 tx=0).stop 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.266+0000 7ff9b542a740 1 -- 172.21.15.5:0/3537239355 shutdown_connections 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.266+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3537239355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba3d0c0 0x56293ba3f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.266+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3537239355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 0x56293ba3cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.266+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3537239355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba2d9f0 0x56293ba2ddd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.266+0000 7ff9b542a740 1 -- 172.21.15.5:0/3537239355 >> 172.21.15.5:0/3537239355 conn(0x56293ba2a060 msgr2=0x56293ba2a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.266+0000 7ff9b542a740 1 -- 172.21.15.5:0/3537239355 shutdown_connections 2024-09-19T06:22:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.266+0000 7ff9b542a740 1 -- 172.21.15.5:0/3537239355 wait complete. 2024-09-19T06:22:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.266+0000 7ff9b542a740 1 -- start start 2024-09-19T06:22:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9b542a740 1 Processor -- start 2024-09-19T06:22:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9b542a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2d9f0 0x56293ba3a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9b542a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2e310 0x56293ba3a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9b542a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba3d0c0 0x56293ba3ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9b542a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56293ba3fd20 con 0x56293ba2e310 2024-09-19T06:22:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9b542a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56293b91b290 con 0x56293ba2d9f0 2024-09-19T06:22:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9b542a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56293ba3fea0 con 0x56293ba3d0c0 2024-09-19T06:22:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9b542a740 10 client.0 _ll_register_callbacks cb 0x56293b94c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2e310 0x56293ba3a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2e310 0x56293ba3a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39498/0 (socket says 172.21.15.5:39498) 2024-09-19T06:22:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9ab7fe640 1 -- 172.21.15.5:0/486381998 learned_addr learned my addr 172.21.15.5:0/486381998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2d9f0 0x56293ba3a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.267+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba3d0c0 0x56293ba3ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.268+0000 7ff9ab7fe640 1 -- 172.21.15.5:0/486381998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba3d0c0 msgr2=0x56293ba3ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.268+0000 7ff9ab7fe640 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba3d0c0 0x56293ba3ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.268+0000 7ff9ab7fe640 1 -- 172.21.15.5:0/486381998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2d9f0 msgr2=0x56293ba3a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.268+0000 7ff9ab7fe640 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2d9f0 0x56293ba3a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.268+0000 7ff9ab7fe640 1 -- 172.21.15.5:0/486381998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56293b6a5210 con 0x56293ba2e310 2024-09-19T06:22:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.268+0000 7ff9ab7fe640 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2e310 0x56293ba3a5f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0012700 tx=0x7ff9a0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16808d92499b3c95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.268+0000 7ff98b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:22:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.268+0000 7ff98b7fe640 1 -- 172.21.15.5:0/486381998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000f8c0 con 0x56293ba2e310 2024-09-19T06:22:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.268+0000 7ff98b7fe640 1 -- 172.21.15.5:0/486381998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7ff9a000fa60 con 0x56293ba2e310 2024-09-19T06:22:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.268+0000 7ff9b542a740 1 -- 172.21.15.5:0/486381998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a4004d20 con 0x56293ba2e310 2024-09-19T06:22:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.268+0000 7ff98b7fe640 1 -- 172.21.15.5:0/486381998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff9a000fc60 con 0x56293ba2e310 2024-09-19T06:22:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.269+0000 7ff9b542a740 10 client.20148 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.269+0000 7ff9b542a740 1 -- 172.21.15.5:0/486381998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56293b771620 con 0x56293ba2e310 2024-09-19T06:22:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.269+0000 7ff98b7fe640 1 -- 172.21.15.5:0/486381998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4004d20 con 0x56293ba2e310 2024-09-19T06:22:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.269+0000 7ff98b7fe640 1 -- 172.21.15.5:0/486381998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7ff9a0010430 con 0x56293ba2e310 2024-09-19T06:22:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.269+0000 7ff9b542a740 10 client.20148 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.269+0000 7ff9b542a740 10 client.20148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.269+0000 7ff9b542a740 20 client.20148 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.269+0000 7ff9b542a740 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56293ba4d240 0x56293ba4f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.269+0000 7ff9b542a740 4 client.20148 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.269+0000 7ff9b542a740 1 -- 172.21.15.5:0/486381998 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56293b99fa20 con 0x56293ba4d240 2024-09-19T06:22:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.270+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56293ba4d240 0x56293ba4f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.270+0000 7ff98b7fe640 10 client.20148 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.270+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56293ba4d240 0x56293ba4f630 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aab58a323e4093e server_cookie=a83df67bd70d2bf6 in_seq=0 out_seq=0 2024-09-19T06:22:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff98b7fe640 1 -- 172.21.15.5:0/486381998 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56293b99fa20 con 0x56293ba4d240 2024-09-19T06:22:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff98b7fe640 10 client.20148 handle_command_reply: tid=0 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff9b542a740 1 client.20148 shutdown 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff9b542a740 1 -- 172.21.15.5:0/486381998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2e310 msgr2=0x56293ba3a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff9b542a740 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2e310 0x56293ba3a5f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0012700 tx=0x7ff9a0012bc0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff9b542a740 1 -- 172.21.15.5:0/486381998 shutdown_connections 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff9b542a740 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56293ba4d240 0x56293ba4f630 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff9b542a740 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba3d0c0 0x56293ba3ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff9b542a740 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2e310 0x56293ba3a5f0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff9b542a740 1 --2- 172.21.15.5:0/486381998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2d9f0 0x56293ba3a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff9b542a740 1 -- 172.21.15.5:0/486381998 >> 172.21.15.5:0/486381998 conn(0x56293ba2a060 msgr2=0x56293ba2fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff98b7fe640 0 client.20148 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff9b542a740 1 -- 172.21.15.5:0/486381998 shutdown_connections 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff9b542a740 1 -- 172.21.15.5:0/486381998 wait complete. 2024-09-19T06:22:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.271+0000 7ff9b542a740 20 client.20148 trim_cache size 0 max 16384 2024-09-19T06:22:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9b542a740 1 Processor -- start 2024-09-19T06:22:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9b542a740 1 -- start start 2024-09-19T06:22:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9b542a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2d9f0 0x56293ba2fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9b542a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba2e310 0x56293ba54f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9b542a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba3d0c0 0x56293ba55480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9b542a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1ab670 con 0x56293ba2d9f0 2024-09-19T06:22:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9b542a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1ad1b0 con 0x56293ba3d0c0 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9b542a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1abd40 con 0x56293ba2e310 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9b0b54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2d9f0 0x56293ba2fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9b0b54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2d9f0 0x56293ba2fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39500/0 (socket says 172.21.15.5:39500) 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9b0b54640 1 -- 172.21.15.5:0/3190698603 learned_addr learned my addr 172.21.15.5:0/3190698603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/3190698603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba3d0c0 0x56293ba55480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.298+0000 7ff9ab7fe640 1 --2- 172.21.15.5:0/3190698603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba2e310 0x56293ba54f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.299+0000 7ff9b0b54640 1 -- 172.21.15.5:0/3190698603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba2e310 msgr2=0x56293ba54f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.299+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/3190698603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba2e310 0x56293ba54f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.299+0000 7ff9b0b54640 1 -- 172.21.15.5:0/3190698603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba3d0c0 msgr2=0x56293ba55480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.299+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/3190698603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba3d0c0 0x56293ba55480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.299+0000 7ff9b0b54640 1 -- 172.21.15.5:0/3190698603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56293ba39040 con 0x56293ba2d9f0 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.299+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/3190698603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2d9f0 0x56293ba2fc40 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4011010 tx=0x7ff9a4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a373195825401049 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.299+0000 7ff991ffb640 1 -- 172.21.15.5:0/3190698603 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4016040 con 0x56293ba2d9f0 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.299+0000 7ff991ffb640 1 -- 172.21.15.5:0/3190698603 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4005660 con 0x56293ba2d9f0 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.299+0000 7ff991ffb640 1 -- 172.21.15.5:0/3190698603 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4023040 con 0x56293ba2d9f0 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.300+0000 7ff9b542a740 1 -- 172.21.15.5:0/3190698603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2d9f0 msgr2=0x56293ba2fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.300+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3190698603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2d9f0 0x56293ba2fc40 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4011010 tx=0x7ff9a4010740 comp rx=0 tx=0).stop 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.300+0000 7ff9b542a740 1 -- 172.21.15.5:0/3190698603 shutdown_connections 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.300+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3190698603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba3d0c0 0x56293ba55480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.300+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3190698603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba2e310 0x56293ba54f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.300+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3190698603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2d9f0 0x56293ba2fc40 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.300+0000 7ff9b542a740 1 -- 172.21.15.5:0/3190698603 >> 172.21.15.5:0/3190698603 conn(0x56293ba2a060 msgr2=0x56293ba2f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.300+0000 7ff9b542a740 1 -- 172.21.15.5:0/3190698603 shutdown_connections 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.300+0000 7ff9b542a740 1 -- 172.21.15.5:0/3190698603 wait complete. 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.300+0000 7ff9b542a740 1 -- start start 2024-09-19T06:22:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9b542a740 1 Processor -- start 2024-09-19T06:22:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9b542a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2d9f0 0x56293ba45a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9b542a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 0x56293ba40d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9b542a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba3d0c0 0x56293ba414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9b542a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1acae0 con 0x56293ba2d9f0 2024-09-19T06:22:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9b542a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1ac410 con 0x56293ba2e310 2024-09-19T06:22:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9b542a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1a9670 con 0x56293ba3d0c0 2024-09-19T06:22:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9ab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 0x56293ba40d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9b0b54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2d9f0 0x56293ba45a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9ab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 0x56293ba40d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56434/0 (socket says 172.21.15.5:56434) 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9ab7fe640 1 -- 172.21.15.5:0/3608291030 learned_addr learned my addr 172.21.15.5:0/3608291030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9b542a740 10 client.0 _ll_register_callbacks cb 0x7ff9a0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9ab7fe640 1 -- 172.21.15.5:0/3608291030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba3d0c0 msgr2=0x56293ba414b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9ab7fe640 1 --2- 172.21.15.5:0/3608291030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba3d0c0 0x56293ba414b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9ab7fe640 1 -- 172.21.15.5:0/3608291030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2d9f0 msgr2=0x56293ba45a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9ab7fe640 1 --2- 172.21.15.5:0/3608291030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2d9f0 0x56293ba45a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.301+0000 7ff9ab7fe640 1 -- 172.21.15.5:0/3608291030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56293ba2e140 con 0x56293ba2e310 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.302+0000 7ff9ab7fe640 1 --2- 172.21.15.5:0/3608291030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 0x56293ba40d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x56293ba3b6f0 tx=0x7ff9a0009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d17529ed0055ccd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.302+0000 7ff98b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.302+0000 7ff98b7fe640 1 -- 172.21.15.5:0/3608291030 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0029040 con 0x56293ba2e310 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.302+0000 7ff9b542a740 1 -- 172.21.15.5:0/3608291030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a4005660 con 0x56293ba2e310 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.302+0000 7ff98b7fe640 1 -- 172.21.15.5:0/3608291030 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8550+0+0 (secure 0 0 0) 0x7ff9a000f790 con 0x56293ba2e310 2024-09-19T06:22:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.302+0000 7ff98b7fe640 1 -- 172.21.15.5:0/3608291030 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff9a0035050 con 0x56293ba2e310 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.302+0000 7ff9b542a740 10 client.29525 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.302+0000 7ff9b542a740 1 -- 172.21.15.5:0/3608291030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56293bab7e80 con 0x56293ba2e310 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.303+0000 7ff98b7fe640 1 -- 172.21.15.5:0/3608291030 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4005660 con 0x56293ba2e310 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.303+0000 7ff98b7fe640 1 -- 172.21.15.5:0/3608291030 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7ff9a00053d0 con 0x56293ba2e310 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.303+0000 7ff9b542a740 10 client.29525 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.303+0000 7ff9b542a740 10 client.29525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.303+0000 7ff9b542a740 20 client.29525 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.303+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3608291030 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56293bab8540 0x56293baba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.303+0000 7ff9b542a740 4 client.29525 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.303+0000 7ff9b542a740 1 -- 172.21.15.5:0/3608291030 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56293b999d70 con 0x56293bab8540 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.303+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/3608291030 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56293bab8540 0x56293baba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.304+0000 7ff9b0b54640 1 --2- 172.21.15.5:0/3608291030 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56293bab8540 0x56293baba960 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe5932f8aae1fe03 server_cookie=26800baea6fe987 in_seq=0 out_seq=0 2024-09-19T06:22:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.304+0000 7ff98b7fe640 10 client.29525 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.521+0000 7ff98b7fe640 1 -- 172.21.15.5:0/3608291030 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56293b999d70 con 0x56293bab8540 2024-09-19T06:22:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.521+0000 7ff98b7fe640 10 client.29525 handle_command_reply: tid=0 2024-09-19T06:22:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.521+0000 7ff9b542a740 1 client.29525 shutdown 2024-09-19T06:22:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.521+0000 7ff9b542a740 1 -- 172.21.15.5:0/3608291030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 msgr2=0x56293ba40d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.521+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3608291030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 0x56293ba40d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x56293ba3b6f0 tx=0x7ff9a0009fd0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.522+0000 7ff9b542a740 1 -- 172.21.15.5:0/3608291030 shutdown_connections 2024-09-19T06:22:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.522+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3608291030 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56293bab8540 0x56293baba960 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.522+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3608291030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56293ba3d0c0 0x56293ba414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.522+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3608291030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56293ba2e310 0x56293ba40d20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.522+0000 7ff98b7fe640 0 client.29525 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.522+0000 7ff9b542a740 1 --2- 172.21.15.5:0/3608291030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56293ba2d9f0 0x56293ba45a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.522+0000 7ff9b542a740 1 -- 172.21.15.5:0/3608291030 >> 172.21.15.5:0/3608291030 conn(0x56293ba2a060 msgr2=0x56293ba3c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.522+0000 7ff9b542a740 1 -- 172.21.15.5:0/3608291030 shutdown_connections 2024-09-19T06:22:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.522+0000 7ff9b542a740 1 -- 172.21.15.5:0/3608291030 wait complete. 2024-09-19T06:22:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.522+0000 7ff9b542a740 20 client.29525 trim_cache size 0 max 16384 2024-09-19T06:22:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:22:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:22:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:22:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:22:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.523+0000 7ff9b25b7640 1 -- 172.21.15.5:0/699811037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9940a33b0 msgr2=0x7ff9940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.523+0000 7ff9b25b7640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9940a33b0 0x7ff9940a5870 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0bc780 tx=0x7ff99c0f6060 comp rx=0 tx=0).stop 2024-09-19T06:22:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.523+0000 7ff9b25b7640 1 -- 172.21.15.5:0/699811037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ac102d00 msgr2=0x7ff9ac198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.523+0000 7ff9b25b7640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ac102d00 0x7ff9ac198360 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff99c005940 tx=0x7ff99c0042f0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.523+0000 7ff9b25b7640 1 -- 172.21.15.5:0/699811037 shutdown_connections 2024-09-19T06:22:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.523+0000 7ff9b25b7640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9ac10c0d0 0x7ff9ac19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.523+0000 7ff9b25b7640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9ac103620 0x7ff9ac1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.523+0000 7ff9b25b7640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9940a33b0 0x7ff9940a5870 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.523+0000 7ff9b25b7640 1 --2- 172.21.15.5:0/699811037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ac102d00 0x7ff9ac198360 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.523+0000 7ff9b25b7640 1 -- 172.21.15.5:0/699811037 >> 172.21.15.5:0/699811037 conn(0x7ff9ac0fc6b0 msgr2=0x7ff9ac0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.523+0000 7ff9b25b7640 1 -- 172.21.15.5:0/699811037 shutdown_connections 2024-09-19T06:22:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:00.523+0000 7ff9b25b7640 1 -- 172.21.15.5:0/699811037 wait complete. 2024-09-19T06:22:00.564 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53721 8.81 MB/sec execute 204 sec latency 173.796 ms 2024-09-19T06:22:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:00 smithi157 ceph-mon[35346]: pgmap v587: 97 pgs: 97 active+clean; 898 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 279 op/s 2024-09-19T06:22:00.935 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:00 smithi049 ceph-mon[34640]: pgmap v587: 97 pgs: 97 active+clean; 898 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 279 op/s 2024-09-19T06:22:00.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:00 smithi005 ceph-mon[30955]: pgmap v587: 97 pgs: 97 active+clean; 898 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 279 op/s 2024-09-19T06:22:01.564 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53846 8.80 MB/sec execute 205 sec latency 191.931 ms 2024-09-19T06:22:02.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53973 8.77 MB/sec execute 206 sec latency 56.784 ms 2024-09-19T06:22:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:02 smithi157 ceph-mon[35346]: pgmap v588: 97 pgs: 97 active+clean; 898 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 279 op/s 2024-09-19T06:22:02.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:02 smithi005 ceph-mon[30955]: pgmap v588: 97 pgs: 97 active+clean; 898 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 279 op/s 2024-09-19T06:22:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:02 smithi049 ceph-mon[34640]: pgmap v588: 97 pgs: 97 active+clean; 898 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 279 op/s 2024-09-19T06:22:03.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 54156 8.78 MB/sec execute 207 sec latency 64.438 ms 2024-09-19T06:22:04.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 54319 8.78 MB/sec execute 208 sec latency 66.636 ms 2024-09-19T06:22:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:04 smithi157 ceph-mon[35346]: pgmap v589: 97 pgs: 97 active+clean; 913 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.6 MiB/s wr, 435 op/s 2024-09-19T06:22:04.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:04 smithi005 ceph-mon[30955]: pgmap v589: 97 pgs: 97 active+clean; 913 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.6 MiB/s wr, 435 op/s 2024-09-19T06:22:04.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:04 smithi049 ceph-mon[34640]: pgmap v589: 97 pgs: 97 active+clean; 913 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.6 MiB/s wr, 435 op/s 2024-09-19T06:22:05.268 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:22:05.566 INFO:tasks.workunit.client.0.smithi005.stdout: 10 54493 8.78 MB/sec execute 209 sec latency 78.696 ms 2024-09-19T06:22:05.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:05 smithi005 ceph-mon[30955]: osdmap e110: 12 total, 12 up, 12 in 2024-09-19T06:22:05.664 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:22:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:05 smithi157 ceph-mon[35346]: osdmap e110: 12 total, 12 up, 12 in 2024-09-19T06:22:05.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:05 smithi049 ceph-mon[34640]: osdmap e110: 12 total, 12 up, 12 in 2024-09-19T06:22:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.253+0000 7f985f2b7640 1 -- 172.21.15.5:0/3118586978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9858100e20 msgr2=0x7f9858101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.253+0000 7f985f2b7640 1 --2- 172.21.15.5:0/3118586978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9858100e20 0x7f9858101200 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f9854009860 tx=0x7f985402a3d0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.254+0000 7f985f2b7640 1 -- 172.21.15.5:0/3118586978 shutdown_connections 2024-09-19T06:22:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.254+0000 7f985f2b7640 1 --2- 172.21.15.5:0/3118586978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858102ef0 0x7f985810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.254+0000 7f985f2b7640 1 --2- 172.21.15.5:0/3118586978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858102570 0x7f98581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.254+0000 7f985f2b7640 1 --2- 172.21.15.5:0/3118586978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9858100e20 0x7f9858101200 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.254+0000 7f985f2b7640 1 -- 172.21.15.5:0/3118586978 >> 172.21.15.5:0/3118586978 conn(0x7f98580fc7d0 msgr2=0x7f98580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.254+0000 7f985f2b7640 1 -- 172.21.15.5:0/3118586978 shutdown_connections 2024-09-19T06:22:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.254+0000 7f985f2b7640 1 -- 172.21.15.5:0/3118586978 wait complete. 2024-09-19T06:22:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985f2b7640 1 Processor -- start 2024-09-19T06:22:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985f2b7640 1 -- start start 2024-09-19T06:22:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985f2b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858100e20 0x7f9858103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985f2b7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858102570 0x7f9858103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985f2b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9858102ef0 0x7f98581041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985f2b7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9858110e80 con 0x7f9858102ef0 2024-09-19T06:22:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985f2b7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9858110d00 con 0x7f9858102570 2024-09-19T06:22:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985f2b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9858111000 con 0x7f9858100e20 2024-09-19T06:22:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985d053640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858100e20 0x7f9858103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985c852640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858102570 0x7f9858103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985c852640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858102570 0x7f9858103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33478/0 (socket says 172.21.15.5:33478) 2024-09-19T06:22:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985c852640 1 -- 172.21.15.5:0/2609707701 learned_addr learned my addr 172.21.15.5:0/2609707701 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.255+0000 7f985d854640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9858102ef0 0x7f98581041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.256+0000 7f985c852640 1 -- 172.21.15.5:0/2609707701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858100e20 msgr2=0x7f9858103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.256+0000 7f985c852640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858100e20 0x7f9858103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.256+0000 7f985c852640 1 -- 172.21.15.5:0/2609707701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9858102ef0 msgr2=0x7f98581041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.256+0000 7f985c852640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9858102ef0 0x7f98581041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.256+0000 7f985c852640 1 -- 172.21.15.5:0/2609707701 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98581a6870 con 0x7f9858102570 2024-09-19T06:22:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.256+0000 7f985d854640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9858102ef0 0x7f98581041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.256+0000 7f985c852640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858102570 0x7f9858103c90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f984800c920 tx=0x7f984800cd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.256+0000 7f984e7fc640 1 -- 172.21.15.5:0/2609707701 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848007bb0 con 0x7f9858102570 2024-09-19T06:22:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.256+0000 7f984e7fc640 1 -- 172.21.15.5:0/2609707701 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9848007d50 con 0x7f9858102570 2024-09-19T06:22:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.256+0000 7f984e7fc640 1 -- 172.21.15.5:0/2609707701 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848018730 con 0x7f9858102570 2024-09-19T06:22:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.256+0000 7f985f2b7640 1 -- 172.21.15.5:0/2609707701 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98581a6b60 con 0x7f9858102570 2024-09-19T06:22:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.256+0000 7f985f2b7640 1 -- 172.21.15.5:0/2609707701 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98581a6ff0 con 0x7f9858102570 2024-09-19T06:22:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.259+0000 7f984e7fc640 1 -- 172.21.15.5:0/2609707701 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f984800f450 con 0x7f9858102570 2024-09-19T06:22:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.260+0000 7f9861690740 1 Processor -- start 2024-09-19T06:22:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.260+0000 7f9861690740 1 -- start start 2024-09-19T06:22:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.260+0000 7f984e7fc640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f98440a3400 0x7f98440a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985d854640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f98440a3400 0x7f98440a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f9861690740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 0x557d74185880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f9861690740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74185dc0 0x557d74190140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f9861690740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d74190680 0x557d74192a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f9861690740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d740eb2e0 con 0x557d741854a0 2024-09-19T06:22:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f9861690740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557d74092ce0 con 0x557d74190680 2024-09-19T06:22:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f9861690740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557d740e72a0 con 0x557d74185dc0 2024-09-19T06:22:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f984e7fc640 1 -- 172.21.15.5:0/2609707701 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8594+0+0 (secure 0 0 0) 0x7f98480cfdf0 con 0x7f9858102570 2024-09-19T06:22:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985d053640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 0x557d74185880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985d854640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d74190680 0x557d74192a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985d053640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 0x557d74185880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50922/0 (socket says 172.21.15.5:50922) 2024-09-19T06:22:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985d053640 1 -- 172.21.15.5:0/2242895473 learned_addr learned my addr 172.21.15.5:0/2242895473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985c852640 1 --2- 172.21.15.5:0/2242895473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74185dc0 0x557d74190140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985d053640 1 -- 172.21.15.5:0/2242895473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74185dc0 msgr2=0x557d74190140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985d053640 1 --2- 172.21.15.5:0/2242895473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74185dc0 0x557d74190140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985d053640 1 -- 172.21.15.5:0/2242895473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d74190680 msgr2=0x557d74192a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985d053640 1 --2- 172.21.15.5:0/2242895473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d74190680 0x557d74192a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985d053640 1 -- 172.21.15.5:0/2242895473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d74077440 con 0x557d741854a0 2024-09-19T06:22:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985d854640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f98440a3400 0x7f98440a58c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9850009430 tx=0x7f985000ff40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.261+0000 7f985c852640 1 --2- 172.21.15.5:0/2242895473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74185dc0 0x557d74190140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.262+0000 7f985d053640 1 --2- 172.21.15.5:0/2242895473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 0x557d74185880 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f9854034000 tx=0x7f9854030550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc0d337f1d47d5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.262+0000 7f98437fe640 1 -- 172.21.15.5:0/2242895473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98540309a0 con 0x557d741854a0 2024-09-19T06:22:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.262+0000 7f98437fe640 1 -- 172.21.15.5:0/2242895473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9854041070 con 0x557d741854a0 2024-09-19T06:22:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.262+0000 7f98437fe640 1 -- 172.21.15.5:0/2242895473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98540049b0 con 0x557d741854a0 2024-09-19T06:22:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.263+0000 7f9861690740 1 -- 172.21.15.5:0/2242895473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 msgr2=0x557d74185880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.263+0000 7f9861690740 1 --2- 172.21.15.5:0/2242895473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 0x557d74185880 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f9854034000 tx=0x7f9854030550 comp rx=0 tx=0).stop 2024-09-19T06:22:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.263+0000 7f9861690740 1 -- 172.21.15.5:0/2242895473 shutdown_connections 2024-09-19T06:22:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.263+0000 7f9861690740 1 --2- 172.21.15.5:0/2242895473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d74190680 0x557d74192a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.263+0000 7f9861690740 1 --2- 172.21.15.5:0/2242895473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74185dc0 0x557d74190140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.263+0000 7f9861690740 1 --2- 172.21.15.5:0/2242895473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 0x557d74185880 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.263+0000 7f9861690740 1 -- 172.21.15.5:0/2242895473 >> 172.21.15.5:0/2242895473 conn(0x557d74181a30 msgr2=0x557d74181e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.263+0000 7f9861690740 1 -- 172.21.15.5:0/2242895473 shutdown_connections 2024-09-19T06:22:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.263+0000 7f9861690740 1 -- 172.21.15.5:0/2242895473 wait complete. 2024-09-19T06:22:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.264+0000 7f9861690740 1 -- start start 2024-09-19T06:22:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.264+0000 7f9861690740 1 Processor -- start 2024-09-19T06:22:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.264+0000 7f9861690740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d741854a0 0x557d7418ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.264+0000 7f9861690740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74185dc0 0x557d7418d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.264+0000 7f9861690740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d74190680 0x557d7418d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.264+0000 7f9861690740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d74193250 con 0x557d74190680 2024-09-19T06:22:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.264+0000 7f9861690740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557d740a6050 con 0x557d741854a0 2024-09-19T06:22:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.264+0000 7f9861690740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557d741933d0 con 0x557d74185dc0 2024-09-19T06:22:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f985d053640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d741854a0 0x557d7418ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f985d053640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d741854a0 0x557d7418ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33488/0 (socket says 172.21.15.5:33488) 2024-09-19T06:22:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f985d053640 1 -- 172.21.15.5:0/2130979196 learned_addr learned my addr 172.21.15.5:0/2130979196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f9861690740 10 client.0 _ll_register_callbacks cb 0x557d740e6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f985d854640 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d74190680 0x557d7418d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f985c852640 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74185dc0 0x557d7418d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f985d053640 1 -- 172.21.15.5:0/2130979196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74185dc0 msgr2=0x557d7418d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f985d053640 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74185dc0 0x557d7418d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f985d053640 1 -- 172.21.15.5:0/2130979196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d74190680 msgr2=0x557d7418d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:06.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f985d053640 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d74190680 0x557d7418d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f985d053640 1 -- 172.21.15.5:0/2130979196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d73e5c3c0 con 0x557d741854a0 2024-09-19T06:22:06.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f9861690740 1 -- 172.21.15.5:0/2130979196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9854041070 con 0x557d741854a0 2024-09-19T06:22:06.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f985c852640 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74185dc0 0x557d7418d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:22:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f985d053640 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d741854a0 0x557d7418ff70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9854000c00 tx=0x7f9854005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf1621140f55dea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.265+0000 7f9840ff9640 10 client.29534 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:22:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f9840ff9640 1 -- 172.21.15.5:0/2130979196 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854055060 con 0x557d741854a0 2024-09-19T06:22:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f9840ff9640 1 -- 172.21.15.5:0/2130979196 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8594+0+0 (secure 0 0 0) 0x7f9854046470 con 0x557d741854a0 2024-09-19T06:22:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f9840ff9640 1 -- 172.21.15.5:0/2130979196 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9854046670 con 0x557d741854a0 2024-09-19T06:22:06.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f9861690740 10 client.29534 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:06.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f9861690740 1 -- 172.21.15.5:0/2130979196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=58}) -- 0x557d7419eae0 con 0x557d741854a0 2024-09-19T06:22:06.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f9840ff9640 1 -- 172.21.15.5:0/2130979196 <== mon.2 v2:172.21.15.49:3300/0 4 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f9854046810 con 0x557d741854a0 2024-09-19T06:22:06.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f9861690740 10 client.29534 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:06.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f9861690740 10 client.29534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:06.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f9861690740 20 client.29534 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:06.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f9861690740 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557d7419fe30 0x557d741a2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f9861690740 4 client.29534 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:06.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f9861690740 1 -- 172.21.15.5:0/2130979196 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d740f8850 con 0x557d7419fe30 2024-09-19T06:22:06.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.266+0000 7f985c852640 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557d7419fe30 0x557d741a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.267+0000 7f985c852640 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557d7419fe30 0x557d741a2220 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ddaad9a084cfd server_cookie=9224b6fb8841f8a6 in_seq=0 out_seq=0 2024-09-19T06:22:06.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.267+0000 7f9840ff9640 10 client.29534 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:06.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.268+0000 7f9840ff9640 1 -- 172.21.15.5:0/2130979196 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d740f8850 con 0x557d7419fe30 2024-09-19T06:22:06.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.268+0000 7f9840ff9640 10 client.29534 handle_command_reply: tid=0 2024-09-19T06:22:06.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.268+0000 7f9861690740 1 client.29534 shutdown 2024-09-19T06:22:06.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.268+0000 7f9861690740 1 -- 172.21.15.5:0/2130979196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d741854a0 msgr2=0x557d7418ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:06.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.268+0000 7f9861690740 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d741854a0 0x557d7418ff70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9854000c00 tx=0x7f9854005d30 comp rx=0 tx=0).stop 2024-09-19T06:22:06.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.268+0000 7f9861690740 1 -- 172.21.15.5:0/2130979196 shutdown_connections 2024-09-19T06:22:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.268+0000 7f9861690740 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557d7419fe30 0x557d741a2220 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.268+0000 7f9861690740 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d74190680 0x557d7418d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.268+0000 7f9861690740 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74185dc0 0x557d7418d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.268+0000 7f9861690740 1 --2- 172.21.15.5:0/2130979196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d741854a0 0x557d7418ff70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.268+0000 7f9840ff9640 0 client.29534 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.268+0000 7f9861690740 1 -- 172.21.15.5:0/2130979196 >> 172.21.15.5:0/2130979196 conn(0x557d74181a30 msgr2=0x557d74180b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.269+0000 7f9861690740 1 -- 172.21.15.5:0/2130979196 shutdown_connections 2024-09-19T06:22:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.269+0000 7f9861690740 1 -- 172.21.15.5:0/2130979196 wait complete. 2024-09-19T06:22:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.269+0000 7f9861690740 20 client.29534 trim_cache size 0 max 16384 2024-09-19T06:22:06.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.299+0000 7f9861690740 1 Processor -- start 2024-09-19T06:22:06.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.299+0000 7f9861690740 1 -- start start 2024-09-19T06:22:06.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.299+0000 7f9861690740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d741854a0 0x557d7418a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.299+0000 7f9861690740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d74185dc0 0x557d7418a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f9861690740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74190680 0x557d74180370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f9861690740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98581af9b0 con 0x557d74185dc0 2024-09-19T06:22:06.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f9861690740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f98581b14f0 con 0x557d741854a0 2024-09-19T06:22:06.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f9861690740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f98581b0080 con 0x557d74190680 2024-09-19T06:22:06.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f985d854640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74190680 0x557d74180370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f985d854640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74190680 0x557d74180370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50600/0 (socket says 172.21.15.5:50600) 2024-09-19T06:22:06.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f985d854640 1 -- 172.21.15.5:0/3163849472 learned_addr learned my addr 172.21.15.5:0/3163849472 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:06.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f985c852640 1 --2- 172.21.15.5:0/3163849472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d74185dc0 0x557d7418a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f985d053640 1 --2- 172.21.15.5:0/3163849472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d741854a0 0x557d7418a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f985d854640 1 -- 172.21.15.5:0/3163849472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d741854a0 msgr2=0x557d7418a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f985d854640 1 --2- 172.21.15.5:0/3163849472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d741854a0 0x557d7418a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f985d854640 1 -- 172.21.15.5:0/3163849472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d74185dc0 msgr2=0x557d7418a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f985d854640 1 --2- 172.21.15.5:0/3163849472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d74185dc0 0x557d7418a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.300+0000 7f985d854640 1 -- 172.21.15.5:0/3163849472 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d7418b770 con 0x557d74190680 2024-09-19T06:22:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.301+0000 7f985d854640 1 --2- 172.21.15.5:0/3163849472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74190680 0x557d74180370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f985000ed50 tx=0x7f985001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70e8bd0d3c0608d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.301+0000 7f9842ffd640 1 -- 172.21.15.5:0/3163849472 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850028420 con 0x557d74190680 2024-09-19T06:22:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.301+0000 7f9842ffd640 1 -- 172.21.15.5:0/3163849472 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f985002d070 con 0x557d74190680 2024-09-19T06:22:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.301+0000 7f9842ffd640 1 -- 172.21.15.5:0/3163849472 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985001d5c0 con 0x557d74190680 2024-09-19T06:22:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.302+0000 7f9861690740 1 -- 172.21.15.5:0/3163849472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74190680 msgr2=0x557d74180370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.302+0000 7f9861690740 1 --2- 172.21.15.5:0/3163849472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74190680 0x557d74180370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f985000ed50 tx=0x7f985001e000 comp rx=0 tx=0).stop 2024-09-19T06:22:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.302+0000 7f9861690740 1 -- 172.21.15.5:0/3163849472 shutdown_connections 2024-09-19T06:22:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.302+0000 7f9861690740 1 --2- 172.21.15.5:0/3163849472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74190680 0x557d74180370 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.302+0000 7f9861690740 1 --2- 172.21.15.5:0/3163849472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d74185dc0 0x557d7418a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.302+0000 7f9861690740 1 --2- 172.21.15.5:0/3163849472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d741854a0 0x557d7418a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.302+0000 7f9861690740 1 -- 172.21.15.5:0/3163849472 >> 172.21.15.5:0/3163849472 conn(0x557d74181a30 msgr2=0x557d7418c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.302+0000 7f9861690740 1 -- 172.21.15.5:0/3163849472 shutdown_connections 2024-09-19T06:22:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.302+0000 7f9861690740 1 -- 172.21.15.5:0/3163849472 wait complete. 2024-09-19T06:22:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.302+0000 7f9861690740 1 -- start start 2024-09-19T06:22:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.303+0000 7f9861690740 1 Processor -- start 2024-09-19T06:22:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.303+0000 7f9861690740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 0x557d741a4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.303+0000 7f9861690740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d74185dc0 0x557d741a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.303+0000 7f9861690740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74190680 0x557d742027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.303+0000 7f9861690740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98581b0e20 con 0x557d741854a0 2024-09-19T06:22:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.303+0000 7f9861690740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f98581b0750 con 0x557d74185dc0 2024-09-19T06:22:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.303+0000 7f9861690740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f98581ad9b0 con 0x557d74190680 2024-09-19T06:22:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.303+0000 7f9861690740 10 client.0 _ll_register_callbacks cb 0x7f9850009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.303+0000 7f985d053640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 0x557d741a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.303+0000 7f985d053640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 0x557d741a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50940/0 (socket says 172.21.15.5:50940) 2024-09-19T06:22:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.303+0000 7f985d053640 1 -- 172.21.15.5:0/859701687 learned_addr learned my addr 172.21.15.5:0/859701687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.304+0000 7f985c852640 1 --2- 172.21.15.5:0/859701687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d74185dc0 0x557d741a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.304+0000 7f985d053640 1 -- 172.21.15.5:0/859701687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74190680 msgr2=0x557d742027b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:22:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.304+0000 7f985d053640 1 --2- 172.21.15.5:0/859701687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74190680 0x557d742027b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.304+0000 7f985d053640 1 -- 172.21.15.5:0/859701687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d74185dc0 msgr2=0x557d741a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.304+0000 7f985d053640 1 --2- 172.21.15.5:0/859701687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d74185dc0 0x557d741a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.304+0000 7f985d053640 1 -- 172.21.15.5:0/859701687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d74202e20 con 0x557d741854a0 2024-09-19T06:22:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.304+0000 7f985d053640 1 --2- 172.21.15.5:0/859701687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 0x557d741a4930 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9854004570 tx=0x7f9854030550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaad66cd91bd9c2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.304+0000 7f9840ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:22:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.304+0000 7f9840ff9640 1 -- 172.21.15.5:0/859701687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854009450 con 0x557d741854a0 2024-09-19T06:22:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.305+0000 7f9861690740 1 -- 172.21.15.5:0/859701687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f985002d070 con 0x557d741854a0 2024-09-19T06:22:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.305+0000 7f9840ff9640 1 -- 172.21.15.5:0/859701687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8594+0+0 (secure 0 0 0) 0x7f9854003800 con 0x557d741854a0 2024-09-19T06:22:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.305+0000 7f9840ff9640 1 -- 172.21.15.5:0/859701687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9854005480 con 0x557d741854a0 2024-09-19T06:22:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.305+0000 7f9861690740 10 client.20172 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.305+0000 7f9861690740 1 -- 172.21.15.5:0/859701687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557d7420a1b0 con 0x557d741854a0 2024-09-19T06:22:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.305+0000 7f9840ff9640 1 -- 172.21.15.5:0/859701687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f985002d070 con 0x557d741854a0 2024-09-19T06:22:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.306+0000 7f9840ff9640 1 -- 172.21.15.5:0/859701687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f98540516b0 con 0x557d741854a0 2024-09-19T06:22:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.306+0000 7f9861690740 10 client.20172 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.306+0000 7f9861690740 10 client.20172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.306+0000 7f9861690740 20 client.20172 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.306+0000 7f9861690740 1 --2- 172.21.15.5:0/859701687 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557d7420a8c0 0x557d7420ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.306+0000 7f9861690740 4 client.20172 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:22:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.306+0000 7f9861690740 1 -- 172.21.15.5:0/859701687 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557d740f2d60 con 0x557d7420a8c0 2024-09-19T06:22:06.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.307+0000 7f985c852640 1 --2- 172.21.15.5:0/859701687 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557d7420a8c0 0x557d7420ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:06.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.307+0000 7f985c852640 1 --2- 172.21.15.5:0/859701687 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557d7420a8c0 0x557d7420ccb0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bda466ddf245cac server_cookie=aac8b13890b5e034 in_seq=0 out_seq=0 2024-09-19T06:22:06.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.307+0000 7f9840ff9640 10 client.20172 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:06.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.308+0000 7f9840ff9640 1 -- 172.21.15.5:0/859701687 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557d740f2d60 con 0x557d7420a8c0 2024-09-19T06:22:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.308+0000 7f9840ff9640 10 client.20172 handle_command_reply: tid=0 2024-09-19T06:22:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.308+0000 7f9861690740 1 client.20172 shutdown 2024-09-19T06:22:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.308+0000 7f9861690740 1 -- 172.21.15.5:0/859701687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 msgr2=0x557d741a4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.308+0000 7f9861690740 1 --2- 172.21.15.5:0/859701687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 0x557d741a4930 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9854004570 tx=0x7f9854030550 comp rx=0 tx=0).stop 2024-09-19T06:22:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.308+0000 7f9861690740 1 -- 172.21.15.5:0/859701687 shutdown_connections 2024-09-19T06:22:06.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.308+0000 7f9861690740 1 --2- 172.21.15.5:0/859701687 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557d7420a8c0 0x557d7420ccb0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.308+0000 7f9861690740 1 --2- 172.21.15.5:0/859701687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d74190680 0x557d742027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.308+0000 7f9840ff9640 0 client.20172 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:06.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.308+0000 7f9861690740 1 --2- 172.21.15.5:0/859701687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d74185dc0 0x557d741a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.308+0000 7f9861690740 1 --2- 172.21.15.5:0/859701687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d741854a0 0x557d741a4930 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.308+0000 7f9861690740 1 -- 172.21.15.5:0/859701687 >> 172.21.15.5:0/859701687 conn(0x557d74181a30 msgr2=0x557d7418ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.309+0000 7f9861690740 1 -- 172.21.15.5:0/859701687 shutdown_connections 2024-09-19T06:22:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.309+0000 7f9861690740 1 -- 172.21.15.5:0/859701687 wait complete. 2024-09-19T06:22:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.309+0000 7f9861690740 20 client.20172 trim_cache size 0 max 16384 2024-09-19T06:22:06.327 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:22:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:22:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:22:06.328 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:22:06.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.310+0000 7f985f2b7640 1 -- 172.21.15.5:0/2609707701 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f98440a3400 msgr2=0x7f98440a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:06.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.310+0000 7f985f2b7640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f98440a3400 0x7f98440a58c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9850009430 tx=0x7f985000ff40 comp rx=0 tx=0).stop 2024-09-19T06:22:06.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.310+0000 7f985f2b7640 1 -- 172.21.15.5:0/2609707701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858102570 msgr2=0x7f9858103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:06.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.310+0000 7f985f2b7640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858102570 0x7f9858103c90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f984800c920 tx=0x7f984800cd20 comp rx=0 tx=0).stop 2024-09-19T06:22:06.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.310+0000 7f985f2b7640 1 -- 172.21.15.5:0/2609707701 shutdown_connections 2024-09-19T06:22:06.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.310+0000 7f985f2b7640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9858102ef0 0x7f98581041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.310+0000 7f985f2b7640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858102570 0x7f9858103c90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.310+0000 7f985f2b7640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f98440a3400 0x7f98440a58c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.310+0000 7f985f2b7640 1 --2- 172.21.15.5:0/2609707701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858100e20 0x7f9858103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.310+0000 7f985f2b7640 1 -- 172.21.15.5:0/2609707701 >> 172.21.15.5:0/2609707701 conn(0x7f98580fc7d0 msgr2=0x7f985806a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.311+0000 7f985f2b7640 1 -- 172.21.15.5:0/2609707701 shutdown_connections 2024-09-19T06:22:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:06.311+0000 7f985f2b7640 1 -- 172.21.15.5:0/2609707701 wait complete. 2024-09-19T06:22:06.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 54648 8.78 MB/sec execute 210 sec latency 79.190 ms 2024-09-19T06:22:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:06 smithi157 ceph-mon[35346]: pgmap v591: 97 pgs: 97 active+clean; 913 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 343 op/s 2024-09-19T06:22:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:06 smithi005 ceph-mon[30955]: pgmap v591: 97 pgs: 97 active+clean; 913 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 343 op/s 2024-09-19T06:22:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:06 smithi049 ceph-mon[34640]: pgmap v591: 97 pgs: 97 active+clean; 913 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 343 op/s 2024-09-19T06:22:07.459 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:22:07.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:22:07.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 54809 8.80 MB/sec execute 211 sec latency 87.959 ms 2024-09-19T06:22:07.854 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:22:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.443+0000 7f75c0829640 1 -- 172.21.15.5:0/2746650686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc10e3c0 msgr2=0x7f75bc110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.443+0000 7f75c0829640 1 --2- 172.21.15.5:0/2746650686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc10e3c0 0x7f75bc110810 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009ca0 tx=0x7f75b402a950 comp rx=0 tx=0).stop 2024-09-19T06:22:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.443+0000 7f75c0829640 1 -- 172.21.15.5:0/2746650686 shutdown_connections 2024-09-19T06:22:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.443+0000 7f75c0829640 1 --2- 172.21.15.5:0/2746650686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc10e3c0 0x7f75bc110810 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.443+0000 7f75c0829640 1 --2- 172.21.15.5:0/2746650686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc074f40 0x7f75bc10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.443+0000 7f75c0829640 1 --2- 172.21.15.5:0/2746650686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75bc074620 0x7f75bc074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.443+0000 7f75c0829640 1 -- 172.21.15.5:0/2746650686 >> 172.21.15.5:0/2746650686 conn(0x7f75bc0fd8e0 msgr2=0x7f75bc0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.443+0000 7f75c0829640 1 -- 172.21.15.5:0/2746650686 shutdown_connections 2024-09-19T06:22:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.443+0000 7f75c0829640 1 -- 172.21.15.5:0/2746650686 wait complete. 2024-09-19T06:22:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.444+0000 7f75c0829640 1 Processor -- start 2024-09-19T06:22:08.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.444+0000 7f75c0829640 1 -- start start 2024-09-19T06:22:08.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75c0829640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc074620 0x7f75bc10dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:08.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75c0829640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc074f40 0x7f75bc10ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:08.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75c0829640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75bc10e3c0 0x7f75bc10b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:08.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75c0829640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc114020 con 0x7f75bc074f40 2024-09-19T06:22:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75c0829640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc113ea0 con 0x7f75bc10e3c0 2024-09-19T06:22:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75c0829640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1141a0 con 0x7f75bc074620 2024-09-19T06:22:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75b9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc074f40 0x7f75bc10ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75ba59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc074620 0x7f75bc10dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75b9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc074f40 0x7f75bc10ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50950/0 (socket says 172.21.15.5:50950) 2024-09-19T06:22:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75b9d9b640 1 -- 172.21.15.5:0/2415813658 learned_addr learned my addr 172.21.15.5:0/2415813658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75bad9d640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75bc10e3c0 0x7f75bc10b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75b9d9b640 1 -- 172.21.15.5:0/2415813658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc074620 msgr2=0x7f75bc10dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75b9d9b640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc074620 0x7f75bc10dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75b9d9b640 1 -- 172.21.15.5:0/2415813658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75bc10e3c0 msgr2=0x7f75bc10b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75b9d9b640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75bc10e3c0 0x7f75bc10b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75b9d9b640 1 -- 172.21.15.5:0/2415813658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc10bc00 con 0x7f75bc074f40 2024-09-19T06:22:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.445+0000 7f75bad9d640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75bc10e3c0 0x7f75bc10b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.446+0000 7f75b9d9b640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc074f40 0x7f75bc10ade0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f75b000e9f0 tx=0x7f75b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.446+0000 7f75ab7fe640 1 -- 172.21.15.5:0/2415813658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b000cd80 con 0x7f75bc074f40 2024-09-19T06:22:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.446+0000 7f75ab7fe640 1 -- 172.21.15.5:0/2415813658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75b00054d0 con 0x7f75bc074f40 2024-09-19T06:22:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.446+0000 7f75c0829640 1 -- 172.21.15.5:0/2415813658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75bc199550 con 0x7f75bc074f40 2024-09-19T06:22:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.447+0000 7f75c0829640 1 -- 172.21.15.5:0/2415813658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75bc199a30 con 0x7f75bc074f40 2024-09-19T06:22:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.447+0000 7f75ab7fe640 1 -- 172.21.15.5:0/2415813658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0010490 con 0x7f75bc074f40 2024-09-19T06:22:08.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.448+0000 7f75c0829640 1 -- 172.21.15.5:0/2415813658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75bc075cd0 con 0x7f75bc074f40 2024-09-19T06:22:08.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.450+0000 7f75ab7fe640 1 -- 172.21.15.5:0/2415813658 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f75b0010770 con 0x7f75bc074f40 2024-09-19T06:22:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.451+0000 7f75ab7fe640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f75a40a3580 0x7f75a40a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:08.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.451+0000 7f75ba59c640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f75a40a3580 0x7f75a40a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:08.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.451+0000 7f75ab7fe640 1 -- 172.21.15.5:0/2415813658 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8594+0+0 (secure 0 0 0) 0x7f75b00d0000 con 0x7f75bc074f40 2024-09-19T06:22:08.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.452+0000 7f75ba59c640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f75a40a3580 0x7f75a40a5a40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f75ac0095f0 tx=0x7f75ac005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.457+0000 7f75ab7fe640 1 -- 172.21.15.5:0/2415813658 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f75b009d860 con 0x7f75bc074f40 2024-09-19T06:22:08.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 54936 8.80 MB/sec execute 212 sec latency 134.245 ms 2024-09-19T06:22:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:08 smithi157 ceph-mon[35346]: pgmap v592: 97 pgs: 97 active+clean; 919 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 334 op/s 2024-09-19T06:22:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:08 smithi157 ceph-mon[35346]: osdmap e111: 12 total, 12 up, 12 in 2024-09-19T06:22:08.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.850+0000 7f75c0829640 1 -- 172.21.15.5:0/2415813658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75bc10c310 con 0x7f75bc074f40 2024-09-19T06:22:08.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:08 smithi005 ceph-mon[30955]: pgmap v592: 97 pgs: 97 active+clean; 919 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 334 op/s 2024-09-19T06:22:08.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:08 smithi005 ceph-mon[30955]: osdmap e111: 12 total, 12 up, 12 in 2024-09-19T06:22:08.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.851+0000 7f75ab7fe640 1 -- 172.21.15.5:0/2415813658 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11959 (secure 0 0 0) 0x7f75b009d160 con 0x7f75bc074f40 2024-09-19T06:22:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:22:08.853 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":58,"btime":"2024-09-19T06:20:33:468419+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:20:32.963303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:22:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 58 2024-09-19T06:22:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.854+0000 7f75c0829640 1 -- 172.21.15.5:0/2415813658 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f75a40a3580 msgr2=0x7f75a40a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.854+0000 7f75c0829640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f75a40a3580 0x7f75a40a5a40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f75ac0095f0 tx=0x7f75ac005f90 comp rx=0 tx=0).stop 2024-09-19T06:22:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.854+0000 7f75c0829640 1 -- 172.21.15.5:0/2415813658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc074f40 msgr2=0x7f75bc10ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.854+0000 7f75c0829640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc074f40 0x7f75bc10ade0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f75b000e9f0 tx=0x7f75b000edf0 comp rx=0 tx=0).stop 2024-09-19T06:22:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.854+0000 7f75c0829640 1 -- 172.21.15.5:0/2415813658 shutdown_connections 2024-09-19T06:22:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.854+0000 7f75c0829640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75bc10e3c0 0x7f75bc10b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.854+0000 7f75c0829640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc074f40 0x7f75bc10ade0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.854+0000 7f75c0829640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f75a40a3580 0x7f75a40a5a40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.854+0000 7f75c0829640 1 --2- 172.21.15.5:0/2415813658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc074620 0x7f75bc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.854+0000 7f75c0829640 1 -- 172.21.15.5:0/2415813658 >> 172.21.15.5:0/2415813658 conn(0x7f75bc0fd8e0 msgr2=0x7f75bc0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.855+0000 7f75c0829640 1 -- 172.21.15.5:0/2415813658 shutdown_connections 2024-09-19T06:22:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:08.855+0000 7f75c0829640 1 -- 172.21.15.5:0/2415813658 wait complete. 2024-09-19T06:22:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:08 smithi049 ceph-mon[34640]: pgmap v592: 97 pgs: 97 active+clean; 919 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 334 op/s 2024-09-19T06:22:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:08 smithi049 ceph-mon[34640]: osdmap e111: 12 total, 12 up, 12 in 2024-09-19T06:22:09.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 55055 8.77 MB/sec execute 213 sec latency 133.133 ms 2024-09-19T06:22:09.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:09 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2415813658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:22:09.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:09 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2415813658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:22:10.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:09 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2415813658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:22:10.328 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:22:10.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 55213 8.77 MB/sec execute 214 sec latency 321.912 ms 2024-09-19T06:22:10.688 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:22:10.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:10 smithi005 ceph-mon[30955]: pgmap v594: 97 pgs: 97 active+clean; 919 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 417 op/s 2024-09-19T06:22:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:10 smithi049 ceph-mon[34640]: pgmap v594: 97 pgs: 97 active+clean; 919 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 417 op/s 2024-09-19T06:22:11.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:10 smithi157 ceph-mon[35346]: pgmap v594: 97 pgs: 97 active+clean; 919 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 417 op/s 2024-09-19T06:22:11.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 55372 8.75 MB/sec execute 215 sec latency 95.482 ms 2024-09-19T06:22:12.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 55520 8.74 MB/sec execute 216 sec latency 165.003 ms 2024-09-19T06:22:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:12 smithi005 ceph-mon[30955]: pgmap v595: 97 pgs: 97 active+clean; 919 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 184 op/s 2024-09-19T06:22:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:12 smithi049 ceph-mon[34640]: pgmap v595: 97 pgs: 97 active+clean; 919 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 184 op/s 2024-09-19T06:22:13.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:12 smithi157 ceph-mon[35346]: pgmap v595: 97 pgs: 97 active+clean; 919 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 184 op/s 2024-09-19T06:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.544+0000 7f000ac76640 1 -- 172.21.15.5:0/3307919954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0004100b30 msgr2=0x7f0004100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.544+0000 7f000ac76640 1 --2- 172.21.15.5:0/3307919954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0004100b30 0x7f0004100f10 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7efff8009860 tx=0x7efff802a3b0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.545+0000 7f000ac76640 1 -- 172.21.15.5:0/3307919954 shutdown_connections 2024-09-19T06:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.545+0000 7f000ac76640 1 --2- 172.21.15.5:0/3307919954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0004105f40 0x7f00041063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.545+0000 7f000ac76640 1 --2- 172.21.15.5:0/3307919954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0004101450 0x7f0004105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.545+0000 7f000ac76640 1 --2- 172.21.15.5:0/3307919954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0004100b30 0x7f0004100f10 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.545+0000 7f000ac76640 1 -- 172.21.15.5:0/3307919954 >> 172.21.15.5:0/3307919954 conn(0x7f00040fc7a0 msgr2=0x7f00040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.545+0000 7f000ac76640 1 -- 172.21.15.5:0/3307919954 shutdown_connections 2024-09-19T06:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.545+0000 7f000ac76640 1 -- 172.21.15.5:0/3307919954 wait complete. 2024-09-19T06:22:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.546+0000 7f000ac76640 1 Processor -- start 2024-09-19T06:22:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.546+0000 7f000ac76640 1 -- start start 2024-09-19T06:22:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f000ac76640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0004100b30 0x7f00041a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f000ac76640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0004101450 0x7f00041a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f000ac76640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0004105f40 0x7f000419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f000ac76640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0004118480 con 0x7f0004100b30 2024-09-19T06:22:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f000ac76640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0004118300 con 0x7f0004105f40 2024-09-19T06:22:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f000ac76640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0004118600 con 0x7f0004101450 2024-09-19T06:22:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f0003fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0004101450 0x7f00041a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f0008a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0004100b30 0x7f00041a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f0003fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0004101450 0x7f00041a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50632/0 (socket says 172.21.15.5:50632) 2024-09-19T06:22:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f0008a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0004100b30 0x7f00041a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50970/0 (socket says 172.21.15.5:50970) 2024-09-19T06:22:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f0003fff640 1 -- 172.21.15.5:0/2418794258 learned_addr learned my addr 172.21.15.5:0/2418794258 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f0009213640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0004105f40 0x7f000419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f0009213640 1 -- 172.21.15.5:0/2418794258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0004101450 msgr2=0x7f00041a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.547+0000 7f0009213640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0004101450 0x7f00041a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.548+0000 7f0009213640 1 -- 172.21.15.5:0/2418794258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0004100b30 msgr2=0x7f00041a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.548+0000 7f0009213640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0004100b30 0x7f00041a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.548+0000 7f0009213640 1 -- 172.21.15.5:0/2418794258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000419efb0 con 0x7f0004105f40 2024-09-19T06:22:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.548+0000 7f0009213640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0004105f40 0x7f000419e810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efffc00be20 tx=0x7efffc00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.548+0000 7f0001ffb640 1 -- 172.21.15.5:0/2418794258 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc019070 con 0x7f0004105f40 2024-09-19T06:22:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.548+0000 7f0001ffb640 1 -- 172.21.15.5:0/2418794258 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efffc0148f0 con 0x7f0004105f40 2024-09-19T06:22:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.548+0000 7f0001ffb640 1 -- 172.21.15.5:0/2418794258 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc0127f0 con 0x7f0004105f40 2024-09-19T06:22:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.548+0000 7f000ac76640 1 -- 172.21.15.5:0/2418794258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000419f2a0 con 0x7f0004105f40 2024-09-19T06:22:13.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.549+0000 7f000ac76640 1 -- 172.21.15.5:0/2418794258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00041ab4f0 con 0x7f0004105f40 2024-09-19T06:22:13.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.552+0000 7f0001ffb640 1 -- 172.21.15.5:0/2418794258 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7efffc01c070 con 0x7f0004105f40 2024-09-19T06:22:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.553+0000 7f000d04f740 1 Processor -- start 2024-09-19T06:22:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.553+0000 7f000d04f740 1 -- start start 2024-09-19T06:22:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.553+0000 7f0001ffb640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efff40a3430 0x7efff40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f000d04f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 0x55cf74a6c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f000d04f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 0x55cf74a6ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f000d04f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a88380 0x55cf74a8a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f000d04f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cf749d22f0 con 0x55cf74a88380 2024-09-19T06:22:13.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f000d04f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cf74979cf0 con 0x55cf74a6c5d0 2024-09-19T06:22:13.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f0008a12640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 0x55cf74a6c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f000d04f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cf749ce2b0 con 0x55cf74a6db40 2024-09-19T06:22:13.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f0008a12640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 0x55cf74a6c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50636/0 (socket says 172.21.15.5:50636) 2024-09-19T06:22:13.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f0008a12640 1 -- 172.21.15.5:0/3914888578 learned_addr learned my addr 172.21.15.5:0/3914888578 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:13.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f0001ffb640 1 -- 172.21.15.5:0/2418794258 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7efffc0cf400 con 0x7f0004105f40 2024-09-19T06:22:13.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f0003fff640 1 --2- 172.21.15.5:0/3914888578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 0x55cf74a6ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f0008a12640 1 -- 172.21.15.5:0/3914888578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 msgr2=0x55cf74a6ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f0008a12640 1 --2- 172.21.15.5:0/3914888578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 0x55cf74a6ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f0008a12640 1 -- 172.21.15.5:0/3914888578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a88380 msgr2=0x55cf74a8a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f0008a12640 1 --2- 172.21.15.5:0/3914888578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a88380 0x55cf74a8a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f0008a12640 1 -- 172.21.15.5:0/3914888578 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7495e3a0 con 0x55cf74a6db40 2024-09-19T06:22:13.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.554+0000 7f0003fff640 1 --2- 172.21.15.5:0/3914888578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 0x55cf74a6ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:22:13.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.555+0000 7f0008a12640 1 --2- 172.21.15.5:0/3914888578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 0x55cf74a6c090 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efff80057c0 tx=0x7efff802acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20cafe295ac02ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:13.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.555+0000 7f0003fff640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efff40a3430 0x7efff40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.556+0000 7f0003fff640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efff40a3430 0x7efff40a58f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efff00094b0 tx=0x7efff0013040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:13.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.556+0000 7effeeffd640 1 -- 172.21.15.5:0/3914888578 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8041020 con 0x55cf74a6db40 2024-09-19T06:22:13.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.556+0000 7effeeffd640 1 -- 172.21.15.5:0/3914888578 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efff8004240 con 0x55cf74a6db40 2024-09-19T06:22:13.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.556+0000 7effeeffd640 1 -- 172.21.15.5:0/3914888578 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8037040 con 0x55cf74a6db40 2024-09-19T06:22:13.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.557+0000 7f000d04f740 1 -- 172.21.15.5:0/3914888578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 msgr2=0x55cf74a6c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.557+0000 7f000d04f740 1 --2- 172.21.15.5:0/3914888578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 0x55cf74a6c090 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efff80057c0 tx=0x7efff802acf0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.557+0000 7f000d04f740 1 -- 172.21.15.5:0/3914888578 shutdown_connections 2024-09-19T06:22:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.557+0000 7f000d04f740 1 --2- 172.21.15.5:0/3914888578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a88380 0x55cf74a8a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.557+0000 7f000d04f740 1 --2- 172.21.15.5:0/3914888578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 0x55cf74a6ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.557+0000 7f000d04f740 1 --2- 172.21.15.5:0/3914888578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 0x55cf74a6c090 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.557+0000 7f000d04f740 1 -- 172.21.15.5:0/3914888578 >> 172.21.15.5:0/3914888578 conn(0x55cf74a35310 msgr2=0x55cf74a68c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.557+0000 7f000d04f740 1 -- 172.21.15.5:0/3914888578 shutdown_connections 2024-09-19T06:22:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.557+0000 7f000d04f740 1 -- 172.21.15.5:0/3914888578 wait complete. 2024-09-19T06:22:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.558+0000 7f000d04f740 1 -- start start 2024-09-19T06:22:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.558+0000 7f000d04f740 1 Processor -- start 2024-09-19T06:22:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.558+0000 7f000d04f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 0x55cf74a87dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f000d04f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 0x55cf74a85070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f000d04f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a88380 0x55cf74a85800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f000d04f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cf74a8b020 con 0x55cf74a88380 2024-09-19T06:22:13.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f000d04f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cf7498d060 con 0x55cf74a6c5d0 2024-09-19T06:22:13.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f000d04f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cf74a8b1a0 con 0x55cf74a6db40 2024-09-19T06:22:13.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f000d04f740 10 client.0 _ll_register_callbacks cb 0x55cf746e08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:13.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f0008a12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 0x55cf74a87dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f0008a12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 0x55cf74a87dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33548/0 (socket says 172.21.15.5:33548) 2024-09-19T06:22:13.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f0008a12640 1 -- 172.21.15.5:0/55162265 learned_addr learned my addr 172.21.15.5:0/55162265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:13.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f0009213640 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a88380 0x55cf74a85800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f0008a12640 1 -- 172.21.15.5:0/55162265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 msgr2=0x55cf74a85070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f0003fff640 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 0x55cf74a85070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f0008a12640 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 0x55cf74a85070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f0008a12640 1 -- 172.21.15.5:0/55162265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a88380 msgr2=0x55cf74a85800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f0008a12640 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a88380 0x55cf74a85800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f0008a12640 1 -- 172.21.15.5:0/55162265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf747433c0 con 0x55cf74a6c5d0 2024-09-19T06:22:13.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f0009213640 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a88380 0x55cf74a85800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:22:13.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.559+0000 7f0003fff640 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 0x55cf74a85070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:22:13.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.560+0000 7f0008a12640 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 0x55cf74a87dc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efff802a9a0 tx=0x7efff8003b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b7599e79dc1997a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:13.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.560+0000 7effe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:22:13.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.560+0000 7effe3fff640 1 -- 172.21.15.5:0/55162265 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8041030 con 0x55cf74a6c5d0 2024-09-19T06:22:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.560+0000 7f000d04f740 1 -- 172.21.15.5:0/55162265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff8004240 con 0x55cf74a6c5d0 2024-09-19T06:22:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.560+0000 7effe3fff640 1 -- 172.21.15.5:0/55162265 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7efff8003d50 con 0x55cf74a6c5d0 2024-09-19T06:22:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.560+0000 7effe3fff640 1 -- 172.21.15.5:0/55162265 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efff8058050 con 0x55cf74a6c5d0 2024-09-19T06:22:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.560+0000 7f000d04f740 10 client.29558 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.560+0000 7f000d04f740 1 -- 172.21.15.5:0/55162265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55cf74a96b10 con 0x55cf74a6c5d0 2024-09-19T06:22:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.561+0000 7effe3fff640 1 -- 172.21.15.5:0/55162265 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efff8004240 con 0x55cf74a6c5d0 2024-09-19T06:22:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.561+0000 7effe3fff640 1 -- 172.21.15.5:0/55162265 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7efff80464e0 con 0x55cf74a6c5d0 2024-09-19T06:22:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.561+0000 7f000d04f740 10 client.29558 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.561+0000 7f000d04f740 10 client.29558 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:22:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.561+0000 7f000d04f740 20 client.29558 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.561+0000 7f000d04f740 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cf74a98090 0x55cf74a9a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.561+0000 7f000d04f740 4 client.29558 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:13.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.561+0000 7f000d04f740 1 -- 172.21.15.5:0/55162265 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf749df860 con 0x55cf74a98090 2024-09-19T06:22:13.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.561+0000 7f0003fff640 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cf74a98090 0x55cf74a9a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.562+0000 7effe3fff640 10 client.29558 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:22:13.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.562+0000 7f0003fff640 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cf74a98090 0x55cf74a9a4b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2f8a71e39685336 server_cookie=4f52788aa1870f7f in_seq=0 out_seq=0 2024-09-19T06:22:13.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7effe3fff640 1 -- 172.21.15.5:0/55162265 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf749df860 con 0x55cf74a98090 2024-09-19T06:22:13.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7effe3fff640 10 client.29558 handle_command_reply: tid=0 2024-09-19T06:22:13.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7f000d04f740 1 client.29558 shutdown 2024-09-19T06:22:13.569 INFO:tasks.workunit.client.0.smithi005.stdout: 10 55659 8.72 MB/sec execute 217 sec latency 111.755 ms 2024-09-19T06:22:13.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7f000d04f740 1 -- 172.21.15.5:0/55162265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 msgr2=0x55cf74a87dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7f000d04f740 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 0x55cf74a87dc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efff802a9a0 tx=0x7efff8003b30 comp rx=0 tx=0).stop 2024-09-19T06:22:13.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7f000d04f740 1 -- 172.21.15.5:0/55162265 shutdown_connections 2024-09-19T06:22:13.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7f000d04f740 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cf74a98090 0x55cf74a9a4b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7f000d04f740 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a88380 0x55cf74a85800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7effe3fff640 0 client.29558 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:22:13.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7f000d04f740 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6db40 0x55cf74a85070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7f000d04f740 1 --2- 172.21.15.5:0/55162265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6c5d0 0x55cf74a87dc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7f000d04f740 1 -- 172.21.15.5:0/55162265 >> 172.21.15.5:0/55162265 conn(0x55cf74a35310 msgr2=0x55cf74a6e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:13.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.563+0000 7f000d04f740 1 -- 172.21.15.5:0/55162265 shutdown_connections 2024-09-19T06:22:13.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.564+0000 7f000d04f740 1 -- 172.21.15.5:0/55162265 wait complete. 2024-09-19T06:22:13.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.564+0000 7f000d04f740 20 client.29558 trim_cache size 0 max 16384 2024-09-19T06:22:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f000d04f740 1 Processor -- start 2024-09-19T06:22:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f000d04f740 1 -- start start 2024-09-19T06:22:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f000d04f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6c5d0 0x55cf74a85040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f000d04f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6db40 0x55cf74a9fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f000d04f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a88380 0x55cf74aa0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f000d04f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00041ab720 con 0x55cf74a6db40 2024-09-19T06:22:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f000d04f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f00041ad260 con 0x55cf74a88380 2024-09-19T06:22:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f000d04f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f00041abdf0 con 0x55cf74a6c5d0 2024-09-19T06:22:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f0003fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6db40 0x55cf74a9fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f0003fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6db40 0x55cf74a9fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51002/0 (socket says 172.21.15.5:51002) 2024-09-19T06:22:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f0003fff640 1 -- 172.21.15.5:0/2567238581 learned_addr learned my addr 172.21.15.5:0/2567238581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f0008a12640 1 --2- 172.21.15.5:0/2567238581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6c5d0 0x55cf74a85040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.583+0000 7f0009213640 1 --2- 172.21.15.5:0/2567238581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a88380 0x55cf74aa0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.584+0000 7f0009213640 1 -- 172.21.15.5:0/2567238581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6c5d0 msgr2=0x55cf74a85040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.584+0000 7f0009213640 1 --2- 172.21.15.5:0/2567238581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6c5d0 0x55cf74a85040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.584+0000 7f0009213640 1 -- 172.21.15.5:0/2567238581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6db40 msgr2=0x55cf74a9fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.584+0000 7f0009213640 1 --2- 172.21.15.5:0/2567238581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6db40 0x55cf74a9fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.584+0000 7f0009213640 1 -- 172.21.15.5:0/2567238581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf74a84020 con 0x55cf74a88380 2024-09-19T06:22:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.584+0000 7f0003fff640 1 --2- 172.21.15.5:0/2567238581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6db40 0x55cf74a9fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.584+0000 7f0008a12640 1 --2- 172.21.15.5:0/2567238581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6c5d0 0x55cf74a85040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.584+0000 7f0009213640 1 --2- 172.21.15.5:0/2567238581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a88380 0x55cf74aa0300 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efffc0ccc40 tx=0x7efffc09b1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b256eb8c07c730a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.584+0000 7effee7fc640 1 -- 172.21.15.5:0/2567238581 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc09fb80 con 0x55cf74a88380 2024-09-19T06:22:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.584+0000 7effee7fc640 1 -- 172.21.15.5:0/2567238581 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efffc0a4440 con 0x55cf74a88380 2024-09-19T06:22:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.584+0000 7effee7fc640 1 -- 172.21.15.5:0/2567238581 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc0a3fb0 con 0x55cf74a88380 2024-09-19T06:22:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.585+0000 7f000d04f740 1 -- 172.21.15.5:0/2567238581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a88380 msgr2=0x55cf74aa0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.585+0000 7f000d04f740 1 --2- 172.21.15.5:0/2567238581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a88380 0x55cf74aa0300 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efffc0ccc40 tx=0x7efffc09b1e0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.586+0000 7f000d04f740 1 -- 172.21.15.5:0/2567238581 shutdown_connections 2024-09-19T06:22:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.586+0000 7f000d04f740 1 --2- 172.21.15.5:0/2567238581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a88380 0x55cf74aa0300 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.586+0000 7f000d04f740 1 --2- 172.21.15.5:0/2567238581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6db40 0x55cf74a9fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.586+0000 7f000d04f740 1 --2- 172.21.15.5:0/2567238581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a6c5d0 0x55cf74a85040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.586+0000 7f000d04f740 1 -- 172.21.15.5:0/2567238581 >> 172.21.15.5:0/2567238581 conn(0x55cf74a35310 msgr2=0x55cf74a84930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:13.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.586+0000 7f000d04f740 1 -- 172.21.15.5:0/2567238581 shutdown_connections 2024-09-19T06:22:13.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.586+0000 7f000d04f740 1 -- 172.21.15.5:0/2567238581 wait complete. 2024-09-19T06:22:13.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.586+0000 7f000d04f740 1 -- start start 2024-09-19T06:22:13.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.587+0000 7f000d04f740 1 Processor -- start 2024-09-19T06:22:13.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.587+0000 7f000d04f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6c5d0 0x55cf74a82080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.588+0000 7f000d04f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6db40 0x55cf74a825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.588+0000 7f0008a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6c5d0 0x55cf74a82080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.588+0000 7f0008a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6c5d0 0x55cf74a82080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51012/0 (socket says 172.21.15.5:51012) 2024-09-19T06:22:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.588+0000 7f0008a12640 1 -- 172.21.15.5:0/3895417388 learned_addr learned my addr 172.21.15.5:0/3895417388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.588+0000 7f000d04f740 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a88380 0x55cf74a82b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.588+0000 7f000d04f740 1 -- 172.21.15.5:0/3895417388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00041acb90 con 0x55cf74a6c5d0 2024-09-19T06:22:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.588+0000 7f0003fff640 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6db40 0x55cf74a825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.588+0000 7f000d04f740 1 -- 172.21.15.5:0/3895417388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f00041ac4c0 con 0x55cf74a6db40 2024-09-19T06:22:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.588+0000 7f000d04f740 1 -- 172.21.15.5:0/3895417388 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0004105920 con 0x55cf74a88380 2024-09-19T06:22:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.588+0000 7f0009213640 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a88380 0x55cf74a82b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.589+0000 7f000d04f740 10 client.0 _ll_register_callbacks cb 0x7efff80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.589+0000 7f0008a12640 1 -- 172.21.15.5:0/3895417388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a88380 msgr2=0x55cf74a82b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.589+0000 7f0008a12640 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a88380 0x55cf74a82b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.589+0000 7f0008a12640 1 -- 172.21.15.5:0/3895417388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6db40 msgr2=0x55cf74a825c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.589+0000 7f0008a12640 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6db40 0x55cf74a825c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.589+0000 7f0008a12640 1 -- 172.21.15.5:0/3895417388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf74b01d90 con 0x55cf74a6c5d0 2024-09-19T06:22:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.589+0000 7f0009213640 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a88380 0x55cf74a82b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:22:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.589+0000 7f0003fff640 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6db40 0x55cf74a825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.589+0000 7f0008a12640 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6c5d0 0x55cf74a82080 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7efff802a9a0 tx=0x7efff8046d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4804a73336e1c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.589+0000 7effe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:22:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.589+0000 7effe3fff640 1 -- 172.21.15.5:0/3895417388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8041070 con 0x55cf74a6c5d0 2024-09-19T06:22:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.590+0000 7effe3fff640 1 -- 172.21.15.5:0/3895417388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7efff8005040 con 0x55cf74a6c5d0 2024-09-19T06:22:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.590+0000 7f000d04f740 1 -- 172.21.15.5:0/3895417388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efffc0a4440 con 0x55cf74a6c5d0 2024-09-19T06:22:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.590+0000 7effe3fff640 1 -- 172.21.15.5:0/3895417388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efff8061050 con 0x55cf74a6c5d0 2024-09-19T06:22:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.590+0000 7f000d04f740 10 client.20193 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.590+0000 7f000d04f740 1 -- 172.21.15.5:0/3895417388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55cf74b03010 con 0x55cf74a6c5d0 2024-09-19T06:22:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.591+0000 7effe3fff640 1 -- 172.21.15.5:0/3895417388 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efffc0a4440 con 0x55cf74a6c5d0 2024-09-19T06:22:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.591+0000 7effe3fff640 1 -- 172.21.15.5:0/3895417388 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7efff80046c0 con 0x55cf74a6c5d0 2024-09-19T06:22:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.591+0000 7f000d04f740 10 client.20193 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.591+0000 7f000d04f740 10 client.20193 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:22:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.591+0000 7f000d04f740 20 client.20193 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.591+0000 7f000d04f740 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cf74b036b0 0x55cf74b05ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.591+0000 7f000d04f740 4 client.20193 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:22:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.591+0000 7f000d04f740 1 -- 172.21.15.5:0/3895417388 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf749d9d70 con 0x55cf74b036b0 2024-09-19T06:22:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.591+0000 7f0003fff640 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cf74b036b0 0x55cf74b05ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.592+0000 7f0003fff640 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cf74b036b0 0x55cf74b05ad0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e055bf47b9ae996a server_cookie=4b34c62c3eab2075 in_seq=0 out_seq=0 2024-09-19T06:22:13.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.592+0000 7effe3fff640 10 client.20193 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:22:13.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.592+0000 7effe3fff640 1 -- 172.21.15.5:0/3895417388 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf749d9d70 con 0x55cf74b036b0 2024-09-19T06:22:13.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.592+0000 7effe3fff640 10 client.20193 handle_command_reply: tid=0 2024-09-19T06:22:13.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.592+0000 7f000d04f740 1 client.20193 shutdown 2024-09-19T06:22:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.593+0000 7f000d04f740 1 -- 172.21.15.5:0/3895417388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6c5d0 msgr2=0x55cf74a82080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.593+0000 7f000d04f740 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6c5d0 0x55cf74a82080 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7efff802a9a0 tx=0x7efff8046d00 comp rx=0 tx=0).stop 2024-09-19T06:22:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.593+0000 7f000d04f740 1 -- 172.21.15.5:0/3895417388 shutdown_connections 2024-09-19T06:22:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.593+0000 7f000d04f740 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55cf74b036b0 0x55cf74b05ad0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.593+0000 7effe3fff640 0 client.20193 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:22:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.593+0000 7f000d04f740 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf74a88380 0x55cf74a82b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.593+0000 7f000d04f740 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cf74a6db40 0x55cf74a825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.593+0000 7f000d04f740 1 --2- 172.21.15.5:0/3895417388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf74a6c5d0 0x55cf74a82080 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.593+0000 7f000d04f740 1 -- 172.21.15.5:0/3895417388 >> 172.21.15.5:0/3895417388 conn(0x55cf74a35310 msgr2=0x55cf74a67310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:13.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.593+0000 7f000d04f740 1 -- 172.21.15.5:0/3895417388 shutdown_connections 2024-09-19T06:22:13.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.593+0000 7f000d04f740 1 -- 172.21.15.5:0/3895417388 wait complete. 2024-09-19T06:22:13.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.593+0000 7f000d04f740 20 client.20193 trim_cache size 0 max 16384 2024-09-19T06:22:13.602 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:22:13.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.594+0000 7f000ac76640 1 -- 172.21.15.5:0/2418794258 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efff40a3430 msgr2=0x7efff40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.594+0000 7f000ac76640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efff40a3430 0x7efff40a58f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efff00094b0 tx=0x7efff0013040 comp rx=0 tx=0).stop 2024-09-19T06:22:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.594+0000 7f000ac76640 1 -- 172.21.15.5:0/2418794258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0004105f40 msgr2=0x7f000419e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.594+0000 7f000ac76640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0004105f40 0x7f000419e810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efffc00be20 tx=0x7efffc00bf10 comp rx=0 tx=0).stop 2024-09-19T06:22:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.595+0000 7f000ac76640 1 -- 172.21.15.5:0/2418794258 shutdown_connections 2024-09-19T06:22:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.595+0000 7f000ac76640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0004105f40 0x7f000419e810 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.595+0000 7f000ac76640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efff40a3430 0x7efff40a58f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.595+0000 7f000ac76640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0004101450 0x7f00041a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.595+0000 7f000ac76640 1 --2- 172.21.15.5:0/2418794258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0004100b30 0x7f00041a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.595+0000 7f000ac76640 1 -- 172.21.15.5:0/2418794258 >> 172.21.15.5:0/2418794258 conn(0x7f00040fc7a0 msgr2=0x7f00041038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.595+0000 7f000ac76640 1 -- 172.21.15.5:0/2418794258 shutdown_connections 2024-09-19T06:22:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:13.595+0000 7f000ac76640 1 -- 172.21.15.5:0/2418794258 wait complete. 2024-09-19T06:22:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.655+0000 7fd0af911640 1 -- 172.21.15.5:0/1526281354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a810c110 msgr2=0x7fd0a810e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.655+0000 7fd0af911640 1 --2- 172.21.15.5:0/1526281354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a810c110 0x7fd0a810e540 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009ca0 tx=0x7fd0a002a950 comp rx=0 tx=0).stop 2024-09-19T06:22:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.655+0000 7fd0af911640 1 -- 172.21.15.5:0/1526281354 shutdown_connections 2024-09-19T06:22:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.655+0000 7fd0af911640 1 --2- 172.21.15.5:0/1526281354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a810c110 0x7fd0a810e540 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.655+0000 7fd0af911640 1 --2- 172.21.15.5:0/1526281354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8103660 0x7fd0a810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.655+0000 7fd0af911640 1 --2- 172.21.15.5:0/1526281354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0a8102d40 0x7fd0a8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.655+0000 7fd0af911640 1 -- 172.21.15.5:0/1526281354 >> 172.21.15.5:0/1526281354 conn(0x7fd0a80779a0 msgr2=0x7fd0a80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.655+0000 7fd0af911640 1 -- 172.21.15.5:0/1526281354 shutdown_connections 2024-09-19T06:22:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.655+0000 7fd0af911640 1 -- 172.21.15.5:0/1526281354 wait complete. 2024-09-19T06:22:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.656+0000 7fd0af911640 1 Processor -- start 2024-09-19T06:22:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.656+0000 7fd0af911640 1 -- start start 2024-09-19T06:22:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.656+0000 7fd0af911640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a8102d40 0x7fd0a819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.656+0000 7fd0af911640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8103660 0x7fd0a819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0af911640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0a810c110 0x7fd0a819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0af911640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8111d60 con 0x7fd0a8102d40 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0ad6ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a8102d40 0x7fd0a819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0af911640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8111be0 con 0x7fd0a810c110 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0af911640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8111ee0 con 0x7fd0a8103660 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0ad6ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a8102d40 0x7fd0a819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51030/0 (socket says 172.21.15.5:51030) 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/4209506714 learned_addr learned my addr 172.21.15.5:0/4209506714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0aceac640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8103660 0x7fd0a819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0adeae640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0a810c110 0x7fd0a819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/4209506714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8103660 msgr2=0x7fd0a819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0ad6ad640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8103660 0x7fd0a819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/4209506714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0a810c110 msgr2=0x7fd0a819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0ad6ad640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0a810c110 0x7fd0a819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/4209506714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a819f480 con 0x7fd0a8102d40 2024-09-19T06:22:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.657+0000 7fd0ad6ad640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a8102d40 0x7fd0a819a490 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400e9f0 tx=0x7fd0a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.658+0000 7fd09e7fc640 1 -- 172.21.15.5:0/4209506714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a400cd80 con 0x7fd0a8102d40 2024-09-19T06:22:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.658+0000 7fd0af911640 1 -- 172.21.15.5:0/4209506714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a819f770 con 0x7fd0a8102d40 2024-09-19T06:22:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.658+0000 7fd09e7fc640 1 -- 172.21.15.5:0/4209506714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a40108e0 con 0x7fd0a8102d40 2024-09-19T06:22:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.658+0000 7fd09e7fc640 1 -- 172.21.15.5:0/4209506714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4018920 con 0x7fd0a8102d40 2024-09-19T06:22:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.658+0000 7fd0af911640 1 -- 172.21.15.5:0/4209506714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a81a7000 con 0x7fd0a8102d40 2024-09-19T06:22:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.660+0000 7fd09e7fc640 1 -- 172.21.15.5:0/4209506714 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd0a4010a90 con 0x7fd0a8102d40 2024-09-19T06:22:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.661+0000 7fd09e7fc640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0940a33b0 0x7fd0940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.661+0000 7fd09e7fc640 1 -- 172.21.15.5:0/4209506714 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7fd0a40d0010 con 0x7fd0a8102d40 2024-09-19T06:22:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.661+0000 7fd0aceac640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0940a33b0 0x7fd0940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.661+0000 7fd0b26ab740 1 Processor -- start 2024-09-19T06:22:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0b26ab740 1 -- start start 2024-09-19T06:22:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0aceac640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0940a33b0 0x7fd0940a5870 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd0a81029b0 tx=0x7fd098005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0b26ab740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130aa10 0x55cdc130adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0b26ab740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130b330 0x55cdc131df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0b26ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 0x55cdc13208b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0b26ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cdc126f2f0 con 0x55cdc131e4c0 2024-09-19T06:22:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0b26ab740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cdc1217270 con 0x55cdc130aa10 2024-09-19T06:22:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0b26ab740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cdc126b2b0 con 0x55cdc130b330 2024-09-19T06:22:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0ad6ad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130aa10 0x55cdc130adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0aceac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130b330 0x55cdc131df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0ad6ad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130aa10 0x55cdc130adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33590/0 (socket says 172.21.15.5:33590) 2024-09-19T06:22:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/2296407485 learned_addr learned my addr 172.21.15.5:0/2296407485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0aceac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130b330 0x55cdc131df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50678/0 (socket says 172.21.15.5:50678) 2024-09-19T06:22:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.662+0000 7fd0adeae640 1 --2- 172.21.15.5:0/2296407485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 0x55cdc13208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.663+0000 7fd0aceac640 1 -- 172.21.15.5:0/2296407485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130aa10 msgr2=0x55cdc130adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.663+0000 7fd0aceac640 1 --2- 172.21.15.5:0/2296407485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130aa10 0x55cdc130adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.663+0000 7fd0aceac640 1 -- 172.21.15.5:0/2296407485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 msgr2=0x55cdc13208b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.663+0000 7fd0aceac640 1 --2- 172.21.15.5:0/2296407485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 0x55cdc13208b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.663+0000 7fd0aceac640 1 -- 172.21.15.5:0/2296407485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc1216340 con 0x55cdc130b330 2024-09-19T06:22:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.663+0000 7fd0aceac640 1 --2- 172.21.15.5:0/2296407485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130b330 0x55cdc131df80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd098011df0 tx=0x7fd098014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41a645fc8fe1ba6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.663+0000 7fd0937fe640 1 -- 172.21.15.5:0/2296407485 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098003d70 con 0x55cdc130b330 2024-09-19T06:22:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.663+0000 7fd0937fe640 1 -- 172.21.15.5:0/2296407485 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0980054d0 con 0x55cdc130b330 2024-09-19T06:22:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.663+0000 7fd0937fe640 1 -- 172.21.15.5:0/2296407485 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09801e420 con 0x55cdc130b330 2024-09-19T06:22:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.664+0000 7fd0b26ab740 1 -- 172.21.15.5:0/2296407485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130b330 msgr2=0x55cdc131df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.664+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/2296407485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130b330 0x55cdc131df80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd098011df0 tx=0x7fd098014000 comp rx=0 tx=0).stop 2024-09-19T06:22:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.664+0000 7fd0b26ab740 1 -- 172.21.15.5:0/2296407485 shutdown_connections 2024-09-19T06:22:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.664+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/2296407485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 0x55cdc13208b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.664+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/2296407485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130b330 0x55cdc131df80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.664+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/2296407485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130aa10 0x55cdc130adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.664+0000 7fd0b26ab740 1 -- 172.21.15.5:0/2296407485 >> 172.21.15.5:0/2296407485 conn(0x55cdc1307060 msgr2=0x55cdc1307440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.664+0000 7fd0b26ab740 1 -- 172.21.15.5:0/2296407485 shutdown_connections 2024-09-19T06:22:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.664+0000 7fd0b26ab740 1 -- 172.21.15.5:0/2296407485 wait complete. 2024-09-19T06:22:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.665+0000 7fd0b26ab740 1 -- start start 2024-09-19T06:22:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.665+0000 7fd0b26ab740 1 Processor -- start 2024-09-19T06:22:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.665+0000 7fd0b26ab740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 0x55cdc131b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0b26ab740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130b330 0x55cdc131b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0ad6ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 0x55cdc131b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0ad6ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 0x55cdc131b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50688/0 (socket says 172.21.15.5:50688) 2024-09-19T06:22:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/3067476511 learned_addr learned my addr 172.21.15.5:0/3067476511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 0x55cdc131bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3067476511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cdc1321090 con 0x55cdc131e4c0 2024-09-19T06:22:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3067476511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cdc11f8290 con 0x55cdc130b330 2024-09-19T06:22:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3067476511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cdc1321210 con 0x55cdc130aa10 2024-09-19T06:22:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0b26ab740 10 client.0 _ll_register_callbacks cb 0x55cdc1229140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0aceac640 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130b330 0x55cdc131b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/3067476511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130b330 msgr2=0x55cdc131b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0ad6ad640 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130b330 0x55cdc131b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/3067476511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 msgr2=0x55cdc131bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0ad6ad640 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 0x55cdc131bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/3067476511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc0f82210 con 0x55cdc130aa10 2024-09-19T06:22:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd0ad6ad640 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 0x55cdc131b340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40987e0 tx=0x7fd0a409bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a55f9ac3b9d77dd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.666+0000 7fd090ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:22:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd090ff9640 1 -- 172.21.15.5:0/3067476511 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a40b5060 con 0x55cdc130aa10 2024-09-19T06:22:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd090ff9640 1 -- 172.21.15.5:0/3067476511 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7fd0a4010430 con 0x55cdc130aa10 2024-09-19T06:22:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3067476511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0980054d0 con 0x55cdc130aa10 2024-09-19T06:22:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd090ff9640 1 -- 172.21.15.5:0/3067476511 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fd0a40a4f40 con 0x55cdc130aa10 2024-09-19T06:22:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd0b26ab740 10 client.29506 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3067476511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55cdc104e620 con 0x55cdc130aa10 2024-09-19T06:22:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd090ff9640 1 -- 172.21.15.5:0/3067476511 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0980054d0 con 0x55cdc130aa10 2024-09-19T06:22:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd090ff9640 1 -- 172.21.15.5:0/3067476511 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fd0a40a0110 con 0x55cdc130aa10 2024-09-19T06:22:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd0b26ab740 10 client.29506 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd0b26ab740 10 client.29506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd0b26ab740 20 client.29506 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cdc132e3d0 0x55cdc13307e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd0b26ab740 4 client.29506 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3067476511 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdc127ca20 con 0x55cdc132e3d0 2024-09-19T06:22:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.667+0000 7fd0aceac640 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cdc132e3d0 0x55cdc13307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.668+0000 7fd090ff9640 10 client.29506 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.668+0000 7fd0aceac640 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cdc132e3d0 0x55cdc13307e0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d84b21c6f990ff server_cookie=3e7df8c186b5f17b in_seq=0 out_seq=0 2024-09-19T06:22:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.669+0000 7fd090ff9640 1 -- 172.21.15.5:0/3067476511 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cdc127ca20 con 0x55cdc132e3d0 2024-09-19T06:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.669+0000 7fd090ff9640 10 client.29506 handle_command_reply: tid=0 2024-09-19T06:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.669+0000 7fd0b26ab740 1 client.29506 shutdown 2024-09-19T06:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.669+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3067476511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 msgr2=0x55cdc131b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.669+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 0x55cdc131b340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40987e0 tx=0x7fd0a409bdd0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.669+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3067476511 shutdown_connections 2024-09-19T06:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.670+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cdc132e3d0 0x55cdc13307e0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.670+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 0x55cdc131bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.670+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130b330 0x55cdc131b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.670+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3067476511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 0x55cdc131b340 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.670+0000 7fd090ff9640 0 client.29506 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.670+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3067476511 >> 172.21.15.5:0/3067476511 conn(0x55cdc1307060 msgr2=0x55cdc13061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.670+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3067476511 shutdown_connections 2024-09-19T06:22:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.670+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3067476511 wait complete. 2024-09-19T06:22:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.670+0000 7fd0b26ab740 20 client.29506 trim_cache size 0 max 16384 2024-09-19T06:22:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0b26ab740 1 Processor -- start 2024-09-19T06:22:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0b26ab740 1 -- start start 2024-09-19T06:22:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0b26ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc130aa10 0x55cdc13307c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0b26ab740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130b330 0x55cdc13360c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0b26ab740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc131e4c0 0x55cdc1336600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0b26ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81ab6a0 con 0x55cdc130aa10 2024-09-19T06:22:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0b26ab740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81ad1e0 con 0x55cdc131e4c0 2024-09-19T06:22:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0b26ab740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81abd70 con 0x55cdc130b330 2024-09-19T06:22:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0ad6ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc130aa10 0x55cdc13307c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0ad6ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc130aa10 0x55cdc13307c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51054/0 (socket says 172.21.15.5:51054) 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/2856510242 learned_addr learned my addr 172.21.15.5:0/2856510242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0aceac640 1 --2- 172.21.15.5:0/2856510242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130b330 0x55cdc13360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.699+0000 7fd0adeae640 1 --2- 172.21.15.5:0/2856510242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc131e4c0 0x55cdc1336600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.700+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/2856510242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130b330 msgr2=0x55cdc13360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.700+0000 7fd0ad6ad640 1 --2- 172.21.15.5:0/2856510242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130b330 0x55cdc13360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.700+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/2856510242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc131e4c0 msgr2=0x55cdc1336600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.700+0000 7fd0ad6ad640 1 --2- 172.21.15.5:0/2856510242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc131e4c0 0x55cdc1336600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.700+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/2856510242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc131a2d0 con 0x55cdc130aa10 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.700+0000 7fd0adeae640 1 --2- 172.21.15.5:0/2856510242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc131e4c0 0x55cdc1336600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.700+0000 7fd0ad6ad640 1 --2- 172.21.15.5:0/2856510242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc130aa10 0x55cdc13307c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fd0a409a630 tx=0x7fd0a409bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336193dac8de864f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.700+0000 7fd092ffd640 1 -- 172.21.15.5:0/2856510242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a40a0310 con 0x55cdc130aa10 2024-09-19T06:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.700+0000 7fd092ffd640 1 -- 172.21.15.5:0/2856510242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a4098210 con 0x55cdc130aa10 2024-09-19T06:22:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.700+0000 7fd092ffd640 1 -- 172.21.15.5:0/2856510242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a409b7d0 con 0x55cdc130aa10 2024-09-19T06:22:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.701+0000 7fd0b26ab740 1 -- 172.21.15.5:0/2856510242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc130aa10 msgr2=0x55cdc13307c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.701+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/2856510242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc130aa10 0x55cdc13307c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fd0a409a630 tx=0x7fd0a409bec0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.701+0000 7fd0b26ab740 1 -- 172.21.15.5:0/2856510242 shutdown_connections 2024-09-19T06:22:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.701+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/2856510242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc131e4c0 0x55cdc1336600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.701+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/2856510242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130b330 0x55cdc13360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.701+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/2856510242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc130aa10 0x55cdc13307c0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.701+0000 7fd0b26ab740 1 -- 172.21.15.5:0/2856510242 >> 172.21.15.5:0/2856510242 conn(0x55cdc1307060 msgr2=0x55cdc131d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.701+0000 7fd0b26ab740 1 -- 172.21.15.5:0/2856510242 shutdown_connections 2024-09-19T06:22:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.701+0000 7fd0b26ab740 1 -- 172.21.15.5:0/2856510242 wait complete. 2024-09-19T06:22:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.702+0000 7fd0b26ab740 1 -- start start 2024-09-19T06:22:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.702+0000 7fd0b26ab740 1 Processor -- start 2024-09-19T06:22:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.702+0000 7fd0b26ab740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 0x55cdc13223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.702+0000 7fd0b26ab740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130b330 0x55cdc1322930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.702+0000 7fd0b26ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 0x55cdc1322e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0ad6ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 0x55cdc13223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0ad6ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 0x55cdc13223f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50708/0 (socket says 172.21.15.5:50708) 2024-09-19T06:22:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/3061758710 learned_addr learned my addr 172.21.15.5:0/3061758710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0aceac640 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130b330 0x55cdc1322930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0adeae640 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 0x55cdc1322e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3061758710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81acb10 con 0x55cdc131e4c0 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3061758710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81ac440 con 0x55cdc130b330 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3061758710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81a96c0 con 0x55cdc130aa10 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0b26ab740 10 client.0 _ll_register_callbacks cb 0x7fd0a409bd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/3061758710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130b330 msgr2=0x55cdc1322930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0ad6ad640 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130b330 0x55cdc1322930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/3061758710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 msgr2=0x55cdc1322e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0ad6ad640 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 0x55cdc1322e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0ad6ad640 1 -- 172.21.15.5:0/3061758710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc1398290 con 0x55cdc130aa10 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd0ad6ad640 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 0x55cdc13223f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400c6a0 tx=0x7fd0a40a0800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce8cf496d5eb5d02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd090ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.703+0000 7fd090ff9640 1 -- 172.21.15.5:0/3061758710 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a40a0110 con 0x55cdc130aa10 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.704+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3061758710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a4098210 con 0x55cdc130aa10 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.704+0000 7fd090ff9640 1 -- 172.21.15.5:0/3061758710 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7fd0a40a02b0 con 0x55cdc130aa10 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.704+0000 7fd090ff9640 1 -- 172.21.15.5:0/3061758710 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fd0a40bc0b0 con 0x55cdc130aa10 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.704+0000 7fd0b26ab740 10 client.29512 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.704+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3061758710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55cdc1399520 con 0x55cdc130aa10 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.704+0000 7fd090ff9640 1 -- 172.21.15.5:0/3061758710 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a4098210 con 0x55cdc130aa10 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.704+0000 7fd090ff9640 1 -- 172.21.15.5:0/3061758710 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fd0a409b240 con 0x55cdc130aa10 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.704+0000 7fd0b26ab740 10 client.29512 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.704+0000 7fd0b26ab740 10 client.29512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.704+0000 7fd0b26ab740 20 client.29512 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.704+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cdc13999a0 0x55cdc139bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.705+0000 7fd0b26ab740 4 client.29512 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.705+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3061758710 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cdc1276d70 con 0x55cdc13999a0 2024-09-19T06:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.705+0000 7fd0aceac640 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cdc13999a0 0x55cdc139bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.705+0000 7fd0aceac640 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cdc13999a0 0x55cdc139bd90 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=195af6e160a11d58 server_cookie=3e880858af98b82b in_seq=0 out_seq=0 2024-09-19T06:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.705+0000 7fd090ff9640 10 client.29512 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.814+0000 7fd090ff9640 1 -- 172.21.15.5:0/3061758710 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cdc1276d70 con 0x55cdc13999a0 2024-09-19T06:22:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.814+0000 7fd090ff9640 10 client.29512 handle_command_reply: tid=0 2024-09-19T06:22:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.814+0000 7fd0b26ab740 1 client.29512 shutdown 2024-09-19T06:22:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.815+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3061758710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 msgr2=0x55cdc13223f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.815+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 0x55cdc13223f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400c6a0 tx=0x7fd0a40a0800 comp rx=0 tx=0).stop 2024-09-19T06:22:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.815+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3061758710 shutdown_connections 2024-09-19T06:22:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.815+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55cdc13999a0 0x55cdc139bd90 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.815+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdc131e4c0 0x55cdc1322e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.815+0000 7fd090ff9640 0 client.29512 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.815+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdc130b330 0x55cdc1322930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.815+0000 7fd0b26ab740 1 --2- 172.21.15.5:0/3061758710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cdc130aa10 0x55cdc13223f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.815+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3061758710 >> 172.21.15.5:0/3061758710 conn(0x55cdc1307060 msgr2=0x55cdc131cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.815+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3061758710 shutdown_connections 2024-09-19T06:22:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.815+0000 7fd0b26ab740 1 -- 172.21.15.5:0/3061758710 wait complete. 2024-09-19T06:22:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.815+0000 7fd0b26ab740 20 client.29512 trim_cache size 0 max 16384 2024-09-19T06:22:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:22:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:22:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:22:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:22:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.816+0000 7fd0af911640 1 -- 172.21.15.5:0/4209506714 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0940a33b0 msgr2=0x7fd0940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.816+0000 7fd0af911640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0940a33b0 0x7fd0940a5870 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd0a81029b0 tx=0x7fd098005f90 comp rx=0 tx=0).stop 2024-09-19T06:22:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.816+0000 7fd0af911640 1 -- 172.21.15.5:0/4209506714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a8102d40 msgr2=0x7fd0a819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.816+0000 7fd0af911640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a8102d40 0x7fd0a819a490 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400e9f0 tx=0x7fd0a400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.816+0000 7fd0af911640 1 -- 172.21.15.5:0/4209506714 shutdown_connections 2024-09-19T06:22:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.816+0000 7fd0af911640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0a810c110 0x7fd0a819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.816+0000 7fd0af911640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0940a33b0 0x7fd0940a5870 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.816+0000 7fd0af911640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8103660 0x7fd0a819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.816+0000 7fd0af911640 1 --2- 172.21.15.5:0/4209506714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a8102d40 0x7fd0a819a490 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.816+0000 7fd0af911640 1 -- 172.21.15.5:0/4209506714 >> 172.21.15.5:0/4209506714 conn(0x7fd0a80779a0 msgr2=0x7fd0a80fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.816+0000 7fd0af911640 1 -- 172.21.15.5:0/4209506714 shutdown_connections 2024-09-19T06:22:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:13.816+0000 7fd0af911640 1 -- 172.21.15.5:0/4209506714 wait complete. 2024-09-19T06:22:14.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:22:14.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 55799 8.71 MB/sec execute 218 sec latency 79.560 ms 2024-09-19T06:22:14.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:14 smithi005 ceph-mon[30955]: pgmap v596: 97 pgs: 97 active+clean; 932 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.6 MiB/s wr, 342 op/s 2024-09-19T06:22:14.733 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:22:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:14 smithi049 ceph-mon[34640]: pgmap v596: 97 pgs: 97 active+clean; 932 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.6 MiB/s wr, 342 op/s 2024-09-19T06:22:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:14 smithi157 ceph-mon[35346]: pgmap v596: 97 pgs: 97 active+clean; 932 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.6 MiB/s wr, 342 op/s 2024-09-19T06:22:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.329+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2490650620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c281082a0 msgr2=0x7f0c2810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.329+0000 7f0c2fb6c640 1 --2- 172.21.15.5:0/2490650620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c281082a0 0x7f0c2810ec40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009cd0 tx=0x7f0c2002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.329+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2490650620 shutdown_connections 2024-09-19T06:22:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.329+0000 7f0c2fb6c640 1 --2- 172.21.15.5:0/2490650620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c281082a0 0x7f0c2810ec40 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.329+0000 7f0c2fb6c640 1 --2- 172.21.15.5:0/2490650620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c281036e0 0x7f0c28107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.329+0000 7f0c2fb6c640 1 --2- 172.21.15.5:0/2490650620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c28102d30 0x7f0c28103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.329+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2490650620 >> 172.21.15.5:0/2490650620 conn(0x7f0c280fc7d0 msgr2=0x7f0c280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.329+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2490650620 shutdown_connections 2024-09-19T06:22:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.329+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2490650620 wait complete. 2024-09-19T06:22:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.330+0000 7f0c2fb6c640 1 Processor -- start 2024-09-19T06:22:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.330+0000 7f0c2fb6c640 1 -- start start 2024-09-19T06:22:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.330+0000 7f0c2fb6c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c28102d30 0x7f0c28100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.330+0000 7f0c2fb6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c281036e0 0x7f0c28100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.330+0000 7f0c2fb6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c281082a0 0x7f0c28101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.330+0000 7f0c2fb6c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28111f50 con 0x7f0c281082a0 2024-09-19T06:22:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.330+0000 7f0c2fb6c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28111dd0 con 0x7f0c28102d30 2024-09-19T06:22:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.330+0000 7f0c2fb6c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281120d0 con 0x7f0c281036e0 2024-09-19T06:22:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.331+0000 7f0c2d908640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c28102d30 0x7f0c28100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.331+0000 7f0c2d107640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c281036e0 0x7f0c28100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.331+0000 7f0c2d908640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c28102d30 0x7f0c28100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37314/0 (socket says 172.21.15.5:37314) 2024-09-19T06:22:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.331+0000 7f0c2d908640 1 -- 172.21.15.5:0/2004768976 learned_addr learned my addr 172.21.15.5:0/2004768976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.331+0000 7f0c2e109640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c281082a0 0x7f0c28101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.331+0000 7f0c2d107640 1 -- 172.21.15.5:0/2004768976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c28102d30 msgr2=0x7f0c28100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.331+0000 7f0c2d107640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c28102d30 0x7f0c28100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.331+0000 7f0c2d107640 1 -- 172.21.15.5:0/2004768976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c281082a0 msgr2=0x7f0c28101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:15.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.331+0000 7f0c2d107640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c281082a0 0x7f0c28101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.331+0000 7f0c2d107640 1 -- 172.21.15.5:0/2004768976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c281a6930 con 0x7f0c281036e0 2024-09-19T06:22:15.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.331+0000 7f0c2d908640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c28102d30 0x7f0c28100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:15.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.332+0000 7f0c2d107640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c281036e0 0x7f0c28100ed0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005fd0 tx=0x7f0c1800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.332+0000 7f0c1effd640 1 -- 172.21.15.5:0/2004768976 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18009c00 con 0x7f0c281036e0 2024-09-19T06:22:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.332+0000 7f0c1effd640 1 -- 172.21.15.5:0/2004768976 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c18009da0 con 0x7f0c281036e0 2024-09-19T06:22:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.332+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2004768976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c281a6c20 con 0x7f0c281036e0 2024-09-19T06:22:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.332+0000 7f0c1effd640 1 -- 172.21.15.5:0/2004768976 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18003e10 con 0x7f0c281036e0 2024-09-19T06:22:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.332+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2004768976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c281a71b0 con 0x7f0c281036e0 2024-09-19T06:22:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.335+0000 7f0c1effd640 1 -- 172.21.15.5:0/2004768976 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0c18003fb0 con 0x7f0c281036e0 2024-09-19T06:22:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.336+0000 7f0c31f45740 1 Processor -- start 2024-09-19T06:22:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.336+0000 7f0c31f45740 1 -- start start 2024-09-19T06:22:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.337+0000 7f0c31f45740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759d350 0x55661759b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.337+0000 7f0c31f45740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759bde0 0x55661759c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.337+0000 7f0c2e109640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759d350 0x55661759b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.337+0000 7f0c1effd640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0c140a3430 0x7f0c140a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.337+0000 7f0c2e109640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759d350 0x55661759b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44602/0 (socket says 172.21.15.5:44602) 2024-09-19T06:22:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.337+0000 7f0c2e109640 1 -- 172.21.15.5:0/2316282054 learned_addr learned my addr 172.21.15.5:0/2316282054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.337+0000 7f0c31f45740 1 --2- 172.21.15.5:0/2316282054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566175a6b80 0x5566175a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.337+0000 7f0c31f45740 1 -- 172.21.15.5:0/2316282054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5566174f92f0 con 0x55661759bde0 2024-09-19T06:22:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.337+0000 7f0c31f45740 1 -- 172.21.15.5:0/2316282054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5566174a0cf0 con 0x5566175a6b80 2024-09-19T06:22:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.337+0000 7f0c31f45740 1 -- 172.21.15.5:0/2316282054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5566174f52b0 con 0x55661759d350 2024-09-19T06:22:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.338+0000 7f0c1effd640 1 -- 172.21.15.5:0/2004768976 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f0c180cf440 con 0x7f0c281036e0 2024-09-19T06:22:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.338+0000 7f0c2e109640 1 -- 172.21.15.5:0/2316282054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566175a6b80 msgr2=0x5566175a8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.338+0000 7f0c2e109640 1 --2- 172.21.15.5:0/2316282054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566175a6b80 0x5566175a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.338+0000 7f0c2e109640 1 -- 172.21.15.5:0/2316282054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759bde0 msgr2=0x55661759c220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:22:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.338+0000 7f0c2d107640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0c140a3430 0x7f0c140a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.338+0000 7f0c2d908640 1 --2- 172.21.15.5:0/2316282054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759bde0 0x55661759c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.338+0000 7f0c2e109640 1 --2- 172.21.15.5:0/2316282054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759bde0 0x55661759c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.338+0000 7f0c2e109640 1 -- 172.21.15.5:0/2316282054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566174853a0 con 0x55661759d350 2024-09-19T06:22:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.338+0000 7f0c2d107640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0c140a3430 0x7f0c140a58f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0c180a3e40 tx=0x7f0c1800df10 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.338+0000 7f0c2e109640 1 --2- 172.21.15.5:0/2316282054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759d350 0x55661759b8a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005df0 tx=0x7f0c200093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=febc9541a2e7c283 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.339+0000 7f0c13fff640 1 -- 172.21.15.5:0/2316282054 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20031850 con 0x55661759d350 2024-09-19T06:22:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.339+0000 7f0c13fff640 1 -- 172.21.15.5:0/2316282054 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c20036040 con 0x55661759d350 2024-09-19T06:22:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.339+0000 7f0c13fff640 1 -- 172.21.15.5:0/2316282054 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20004430 con 0x55661759d350 2024-09-19T06:22:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.340+0000 7f0c31f45740 1 -- 172.21.15.5:0/2316282054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759d350 msgr2=0x55661759b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.340+0000 7f0c31f45740 1 --2- 172.21.15.5:0/2316282054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759d350 0x55661759b8a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005df0 tx=0x7f0c200093c0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.340+0000 7f0c31f45740 1 -- 172.21.15.5:0/2316282054 shutdown_connections 2024-09-19T06:22:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.340+0000 7f0c31f45740 1 --2- 172.21.15.5:0/2316282054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566175a6b80 0x5566175a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.340+0000 7f0c31f45740 1 --2- 172.21.15.5:0/2316282054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759bde0 0x55661759c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.340+0000 7f0c31f45740 1 --2- 172.21.15.5:0/2316282054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759d350 0x55661759b8a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.340+0000 7f0c31f45740 1 -- 172.21.15.5:0/2316282054 >> 172.21.15.5:0/2316282054 conn(0x55661755c310 msgr2=0x556617598470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.340+0000 7f0c31f45740 1 -- 172.21.15.5:0/2316282054 shutdown_connections 2024-09-19T06:22:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.340+0000 7f0c31f45740 1 -- 172.21.15.5:0/2316282054 wait complete. 2024-09-19T06:22:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.341+0000 7f0c31f45740 1 -- start start 2024-09-19T06:22:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.341+0000 7f0c31f45740 1 Processor -- start 2024-09-19T06:22:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.341+0000 7f0c31f45740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759bde0 0x5566175a6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.341+0000 7f0c31f45740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759d350 0x5566175a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c31f45740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566175a6b80 0x5566175a4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c31f45740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5566175a9870 con 0x5566175a6b80 2024-09-19T06:22:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c31f45740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5566174b4060 con 0x55661759d350 2024-09-19T06:22:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c2d908640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759d350 0x5566175a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c2d908640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759d350 0x5566175a38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37334/0 (socket says 172.21.15.5:37334) 2024-09-19T06:22:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c31f45740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5566175a99f0 con 0x55661759bde0 2024-09-19T06:22:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c2d908640 1 -- 172.21.15.5:0/372827690 learned_addr learned my addr 172.21.15.5:0/372827690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c31f45740 10 client.0 _ll_register_callbacks cb 0x5566172078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c2e109640 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759bde0 0x5566175a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c2d908640 1 -- 172.21.15.5:0/372827690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759bde0 msgr2=0x5566175a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c2d908640 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759bde0 0x5566175a6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c2d908640 1 -- 172.21.15.5:0/372827690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566175a6b80 msgr2=0x5566175a4050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c2e109640 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566175a6b80 0x5566175a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c2d908640 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566175a6b80 0x5566175a4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c2d908640 1 -- 172.21.15.5:0/372827690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55661726a3c0 con 0x55661759d350 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.342+0000 7f0c2e109640 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566175a6b80 0x5566175a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.343+0000 7f0c2d908640 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759d350 0x5566175a38c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0c24013d20 tx=0x7f0c240117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9e672fde982661 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.343+0000 7f0c117fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.343+0000 7f0c117fa640 1 -- 172.21.15.5:0/372827690 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24020060 con 0x55661759d350 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.343+0000 7f0c117fa640 1 -- 172.21.15.5:0/372827690 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f0c2401b630 con 0x55661759d350 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.343+0000 7f0c31f45740 1 -- 172.21.15.5:0/372827690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c20036040 con 0x55661759d350 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.343+0000 7f0c117fa640 1 -- 172.21.15.5:0/372827690 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f0c2401b830 con 0x55661759d350 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.343+0000 7f0c31f45740 10 client.29591 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.343+0000 7f0c31f45740 1 -- 172.21.15.5:0/372827690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5566175b5400 con 0x55661759d350 2024-09-19T06:22:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.344+0000 7f0c117fa640 1 -- 172.21.15.5:0/372827690 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c20036040 con 0x55661759d350 2024-09-19T06:22:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.344+0000 7f0c117fa640 1 -- 172.21.15.5:0/372827690 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f0c24023510 con 0x55661759d350 2024-09-19T06:22:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.344+0000 7f0c31f45740 10 client.29591 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.344+0000 7f0c31f45740 10 client.29591 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:22:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.344+0000 7f0c31f45740 20 client.29591 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.344+0000 7f0c31f45740 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5566175b6910 0x5566175b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.344+0000 7f0c31f45740 4 client.29591 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.344+0000 7f0c31f45740 1 -- 172.21.15.5:0/372827690 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556617506860 con 0x5566175b6910 2024-09-19T06:22:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.344+0000 7f0c2e109640 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5566175b6910 0x5566175b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.345+0000 7f0c2e109640 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5566175b6910 0x5566175b8d50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e8be0d2607adffdb server_cookie=4dce303300455e26 in_seq=0 out_seq=0 2024-09-19T06:22:15.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.345+0000 7f0c117fa640 10 client.29591 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:22:15.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.345+0000 7f0c117fa640 1 -- 172.21.15.5:0/372827690 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556617506860 con 0x5566175b6910 2024-09-19T06:22:15.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.345+0000 7f0c117fa640 10 client.29591 handle_command_reply: tid=0 2024-09-19T06:22:15.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.345+0000 7f0c31f45740 1 client.29591 shutdown 2024-09-19T06:22:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.345+0000 7f0c31f45740 1 -- 172.21.15.5:0/372827690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759d350 msgr2=0x5566175a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.346+0000 7f0c31f45740 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759d350 0x5566175a38c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0c24013d20 tx=0x7f0c240117b0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.346+0000 7f0c31f45740 1 -- 172.21.15.5:0/372827690 shutdown_connections 2024-09-19T06:22:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.346+0000 7f0c31f45740 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5566175b6910 0x5566175b8d50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.346+0000 7f0c31f45740 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566175a6b80 0x5566175a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.346+0000 7f0c31f45740 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759d350 0x5566175a38c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.346+0000 7f0c31f45740 1 --2- 172.21.15.5:0/372827690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759bde0 0x5566175a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.346+0000 7f0c31f45740 1 -- 172.21.15.5:0/372827690 >> 172.21.15.5:0/372827690 conn(0x55661755c310 msgr2=0x55661759e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.346+0000 7f0c117fa640 0 client.29591 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:22:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.346+0000 7f0c31f45740 1 -- 172.21.15.5:0/372827690 shutdown_connections 2024-09-19T06:22:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.346+0000 7f0c31f45740 1 -- 172.21.15.5:0/372827690 wait complete. 2024-09-19T06:22:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.346+0000 7f0c31f45740 20 client.29591 trim_cache size 0 max 16384 2024-09-19T06:22:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c31f45740 1 Processor -- start 2024-09-19T06:22:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c31f45740 1 -- start start 2024-09-19T06:22:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c31f45740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759bde0 0x5566175a2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c31f45740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759d350 0x55661759e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c31f45740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566175a6b80 0x55661759ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c31f45740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281ab5c0 con 0x55661759d350 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c31f45740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281ad100 con 0x55661759bde0 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c31f45740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281abc90 con 0x5566175a6b80 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c2d908640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759d350 0x55661759e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c2e109640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759bde0 0x5566175a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c2e109640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759bde0 0x5566175a2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37340/0 (socket says 172.21.15.5:37340) 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c2e109640 1 -- 172.21.15.5:0/3656498874 learned_addr learned my addr 172.21.15.5:0/3656498874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c2e109640 1 --2- 172.21.15.5:0/3656498874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566175a6b80 0x55661759ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c2e109640 1 -- 172.21.15.5:0/3656498874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566175a6b80 msgr2=0x55661759ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c2e109640 1 --2- 172.21.15.5:0/3656498874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566175a6b80 0x55661759ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c2e109640 1 -- 172.21.15.5:0/3656498874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759d350 msgr2=0x55661759e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c2e109640 1 --2- 172.21.15.5:0/3656498874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759d350 0x55661759e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.371+0000 7f0c2e109640 1 -- 172.21.15.5:0/3656498874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566175993d0 con 0x55661759bde0 2024-09-19T06:22:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.372+0000 7f0c2e109640 1 --2- 172.21.15.5:0/3656498874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759bde0 0x5566175a2770 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009ca0 tx=0x7f0c20000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afac5e907c3a08a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.372+0000 7f0c137fe640 1 -- 172.21.15.5:0/3656498874 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c200318f0 con 0x55661759bde0 2024-09-19T06:22:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.372+0000 7f0c137fe640 1 -- 172.21.15.5:0/3656498874 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c20036030 con 0x55661759bde0 2024-09-19T06:22:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.372+0000 7f0c137fe640 1 -- 172.21.15.5:0/3656498874 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20004540 con 0x55661759bde0 2024-09-19T06:22:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.373+0000 7f0c31f45740 1 -- 172.21.15.5:0/3656498874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759bde0 msgr2=0x5566175a2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.373+0000 7f0c31f45740 1 --2- 172.21.15.5:0/3656498874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759bde0 0x5566175a2770 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009ca0 tx=0x7f0c20000f40 comp rx=0 tx=0).stop 2024-09-19T06:22:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.373+0000 7f0c31f45740 1 -- 172.21.15.5:0/3656498874 shutdown_connections 2024-09-19T06:22:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.373+0000 7f0c31f45740 1 --2- 172.21.15.5:0/3656498874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566175a6b80 0x55661759ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.373+0000 7f0c31f45740 1 --2- 172.21.15.5:0/3656498874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759d350 0x55661759e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.373+0000 7f0c31f45740 1 --2- 172.21.15.5:0/3656498874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55661759bde0 0x5566175a2770 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.373+0000 7f0c31f45740 1 -- 172.21.15.5:0/3656498874 >> 172.21.15.5:0/3656498874 conn(0x55661755c310 msgr2=0x5566175a2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.373+0000 7f0c31f45740 1 -- 172.21.15.5:0/3656498874 shutdown_connections 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.373+0000 7f0c31f45740 1 -- 172.21.15.5:0/3656498874 wait complete. 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.374+0000 7f0c31f45740 1 -- start start 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.374+0000 7f0c31f45740 1 Processor -- start 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.374+0000 7f0c31f45740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759bde0 0x5566176198e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.374+0000 7f0c31f45740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759d350 0x556617619e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c31f45740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566175a6b80 0x5566175aa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c31f45740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281aca30 con 0x55661759bde0 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c31f45740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281ac360 con 0x5566175a6b80 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c31f45740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281a95c0 con 0x55661759d350 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c31f45740 10 client.0 _ll_register_callbacks cb 0x7f0c24009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c2e109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759bde0 0x5566176198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c2e109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759bde0 0x5566176198e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49266/0 (socket says 172.21.15.5:49266) 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c2e109640 1 -- 172.21.15.5:0/3672581164 learned_addr learned my addr 172.21.15.5:0/3672581164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c2d908640 1 --2- 172.21.15.5:0/3672581164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759d350 0x556617619e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c2e109640 1 -- 172.21.15.5:0/3672581164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759d350 msgr2=0x556617619e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c2e109640 1 --2- 172.21.15.5:0/3672581164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759d350 0x556617619e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c2e109640 1 -- 172.21.15.5:0/3672581164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566175a6b80 msgr2=0x5566175aa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c2e109640 1 --2- 172.21.15.5:0/3672581164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566175a6b80 0x5566175aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c2e109640 1 -- 172.21.15.5:0/3672581164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566175aa950 con 0x55661759bde0 2024-09-19T06:22:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c2e109640 1 --2- 172.21.15.5:0/3672581164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759bde0 0x5566176198e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009430 tx=0x7f0c20004090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6326760a5786ead6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.375+0000 7f0c117fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:22:15.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.376+0000 7f0c117fa640 1 -- 172.21.15.5:0/3672581164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20061020 con 0x55661759bde0 2024-09-19T06:22:15.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.376+0000 7f0c117fa640 1 -- 172.21.15.5:0/3672581164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f0c20003b70 con 0x55661759bde0 2024-09-19T06:22:15.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.376+0000 7f0c31f45740 1 -- 172.21.15.5:0/3672581164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c20036030 con 0x55661759bde0 2024-09-19T06:22:15.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.376+0000 7f0c117fa640 1 -- 172.21.15.5:0/3672581164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f0c2006c050 con 0x55661759bde0 2024-09-19T06:22:15.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.376+0000 7f0c31f45740 10 client.20220 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:15.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.376+0000 7f0c31f45740 1 -- 172.21.15.5:0/3672581164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556617621140 con 0x55661759bde0 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.376+0000 7f0c117fa640 1 -- 172.21.15.5:0/3672581164 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c20036030 con 0x55661759bde0 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.376+0000 7f0c117fa640 1 -- 172.21.15.5:0/3672581164 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f0c20030070 con 0x55661759bde0 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.377+0000 7f0c31f45740 10 client.20220 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.377+0000 7f0c31f45740 10 client.20220 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.377+0000 7f0c31f45740 20 client.20220 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.377+0000 7f0c31f45740 1 --2- 172.21.15.5:0/3672581164 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x556617621310 0x556617623700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.377+0000 7f0c31f45740 4 client.20220 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.377+0000 7f0c31f45740 1 -- 172.21.15.5:0/3672581164 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556617500d70 con 0x556617621310 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.377+0000 7f0c2d908640 1 --2- 172.21.15.5:0/3672581164 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x556617621310 0x556617623700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.377+0000 7f0c2d908640 1 --2- 172.21.15.5:0/3672581164 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x556617621310 0x556617623700 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88c57f9c19660eee server_cookie=8cd7c2ea9f1eb34c in_seq=0 out_seq=0 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.377+0000 7f0c117fa640 10 client.20220 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c117fa640 1 -- 172.21.15.5:0/3672581164 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556617500d70 con 0x556617621310 2024-09-19T06:22:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c117fa640 10 client.20220 handle_command_reply: tid=0 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c31f45740 1 client.20220 shutdown 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c31f45740 1 -- 172.21.15.5:0/3672581164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759bde0 msgr2=0x5566176198e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c31f45740 1 --2- 172.21.15.5:0/3672581164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759bde0 0x5566176198e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009430 tx=0x7f0c20004090 comp rx=0 tx=0).stop 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c31f45740 1 -- 172.21.15.5:0/3672581164 shutdown_connections 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c31f45740 1 --2- 172.21.15.5:0/3672581164 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x556617621310 0x556617623700 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c31f45740 1 --2- 172.21.15.5:0/3672581164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566175a6b80 0x5566175aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c31f45740 1 --2- 172.21.15.5:0/3672581164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55661759d350 0x556617619e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c31f45740 1 --2- 172.21.15.5:0/3672581164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55661759bde0 0x5566176198e0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c31f45740 1 -- 172.21.15.5:0/3672581164 >> 172.21.15.5:0/3672581164 conn(0x55661755c310 msgr2=0x5566175a2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c117fa640 0 client.20220 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c31f45740 1 -- 172.21.15.5:0/3672581164 shutdown_connections 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c31f45740 1 -- 172.21.15.5:0/3672581164 wait complete. 2024-09-19T06:22:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.378+0000 7f0c31f45740 20 client.20220 trim_cache size 0 max 16384 2024-09-19T06:22:15.385 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:22:15.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.379+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2004768976 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0c140a3430 msgr2=0x7f0c140a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:15.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.379+0000 7f0c2fb6c640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0c140a3430 0x7f0c140a58f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0c180a3e40 tx=0x7f0c1800df10 comp rx=0 tx=0).stop 2024-09-19T06:22:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.379+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2004768976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c281036e0 msgr2=0x7f0c28100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.379+0000 7f0c2fb6c640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c281036e0 0x7f0c28100ed0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005fd0 tx=0x7f0c1800dde0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.379+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2004768976 shutdown_connections 2024-09-19T06:22:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.379+0000 7f0c2fb6c640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c281082a0 0x7f0c28101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.379+0000 7f0c2fb6c640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f0c140a3430 0x7f0c140a58f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.379+0000 7f0c2fb6c640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c281036e0 0x7f0c28100ed0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.379+0000 7f0c2fb6c640 1 --2- 172.21.15.5:0/2004768976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c28102d30 0x7f0c28100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.379+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2004768976 >> 172.21.15.5:0/2004768976 conn(0x7f0c280fc7d0 msgr2=0x7f0c28105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.379+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2004768976 shutdown_connections 2024-09-19T06:22:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:15.379+0000 7f0c2fb6c640 1 -- 172.21.15.5:0/2004768976 wait complete. 2024-09-19T06:22:15.566 INFO:tasks.workunit.client.0.smithi005.stdout: 10 55959 8.71 MB/sec execute 219 sec latency 107.156 ms 2024-09-19T06:22:16.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:22:16.507 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:22:16.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 56149 8.74 MB/sec execute 220 sec latency 177.804 ms 2024-09-19T06:22:16.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:16 smithi005 ceph-mon[30955]: pgmap v597: 97 pgs: 97 active+clean; 932 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 286 op/s 2024-09-19T06:22:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:16 smithi049 ceph-mon[34640]: pgmap v597: 97 pgs: 97 active+clean; 932 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 286 op/s 2024-09-19T06:22:17.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:16 smithi157 ceph-mon[35346]: pgmap v597: 97 pgs: 97 active+clean; 932 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 286 op/s 2024-09-19T06:22:17.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.074+0000 7ff514357640 1 -- 172.21.15.5:0/413245290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c100a90 msgr2=0x7ff50c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:17.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.074+0000 7ff514357640 1 --2- 172.21.15.5:0/413245290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c100a90 0x7ff50c100ef0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009860 tx=0x7ff4fc02a340 comp rx=0 tx=0).stop 2024-09-19T06:22:17.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.074+0000 7ff514357640 1 -- 172.21.15.5:0/413245290 shutdown_connections 2024-09-19T06:22:17.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.074+0000 7ff514357640 1 --2- 172.21.15.5:0/413245290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50c101430 0x7ff50c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.074+0000 7ff514357640 1 --2- 172.21.15.5:0/413245290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c100a90 0x7ff50c100ef0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.074+0000 7ff514357640 1 --2- 172.21.15.5:0/413245290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff50c106a90 0x7ff50c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.074+0000 7ff514357640 1 -- 172.21.15.5:0/413245290 >> 172.21.15.5:0/413245290 conn(0x7ff50c0fc7d0 msgr2=0x7ff50c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:17.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.074+0000 7ff514357640 1 -- 172.21.15.5:0/413245290 shutdown_connections 2024-09-19T06:22:17.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.074+0000 7ff514357640 1 -- 172.21.15.5:0/413245290 wait complete. 2024-09-19T06:22:17.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.076+0000 7ff514357640 1 Processor -- start 2024-09-19T06:22:17.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.076+0000 7ff514357640 1 -- start start 2024-09-19T06:22:17.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.076+0000 7ff514357640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50c100a90 0x7ff50c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.077+0000 7ff514357640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff50c101430 0x7ff50c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.077+0000 7ff514357640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c106a90 0x7ff50c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.077+0000 7ff5120f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50c100a90 0x7ff50c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.077+0000 7ff5120f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50c100a90 0x7ff50c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44636/0 (socket says 172.21.15.5:44636) 2024-09-19T06:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.077+0000 7ff5120f3640 1 -- 172.21.15.5:0/4166068298 learned_addr learned my addr 172.21.15.5:0/4166068298 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.077+0000 7ff514357640 1 -- 172.21.15.5:0/4166068298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c110d40 con 0x7ff50c101430 2024-09-19T06:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.077+0000 7ff5128f4640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c106a90 0x7ff50c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.077+0000 7ff514357640 1 -- 172.21.15.5:0/4166068298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c110bc0 con 0x7ff50c106a90 2024-09-19T06:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.077+0000 7ff514357640 1 -- 172.21.15.5:0/4166068298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c110ec0 con 0x7ff50c100a90 2024-09-19T06:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.077+0000 7ff5118f2640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff50c101430 0x7ff50c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.078+0000 7ff5120f3640 1 -- 172.21.15.5:0/4166068298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c106a90 msgr2=0x7ff50c19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.078+0000 7ff5120f3640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c106a90 0x7ff50c19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.078+0000 7ff5120f3640 1 -- 172.21.15.5:0/4166068298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff50c101430 msgr2=0x7ff50c197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.078+0000 7ff5120f3640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff50c101430 0x7ff50c197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.078+0000 7ff5120f3640 1 -- 172.21.15.5:0/4166068298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c19c1f0 con 0x7ff50c100a90 2024-09-19T06:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.078+0000 7ff5128f4640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c106a90 0x7ff50c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.078+0000 7ff5118f2640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff50c101430 0x7ff50c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.078+0000 7ff5120f3640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50c100a90 0x7ff50c197240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff508012dd0 tx=0x7ff50800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.079+0000 7ff5037fe640 1 -- 172.21.15.5:0/4166068298 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff508016020 con 0x7ff50c100a90 2024-09-19T06:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.079+0000 7ff5037fe640 1 -- 172.21.15.5:0/4166068298 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5080122d0 con 0x7ff50c100a90 2024-09-19T06:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.079+0000 7ff5037fe640 1 -- 172.21.15.5:0/4166068298 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50800e610 con 0x7ff50c100a90 2024-09-19T06:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.079+0000 7ff514357640 1 -- 172.21.15.5:0/4166068298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c19c4e0 con 0x7ff50c100a90 2024-09-19T06:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.079+0000 7ff514357640 1 -- 172.21.15.5:0/4166068298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c06a790 con 0x7ff50c100a90 2024-09-19T06:22:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.082+0000 7ff5037fe640 1 -- 172.21.15.5:0/4166068298 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff508019020 con 0x7ff50c100a90 2024-09-19T06:22:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.083+0000 7ff5037fe640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4f80a3500 0x7ff4f80a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.083+0000 7ff516730740 1 Processor -- start 2024-09-19T06:22:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.083+0000 7ff5037fe640 1 -- 172.21.15.5:0/4166068298 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7ff5080cf770 con 0x7ff50c100a90 2024-09-19T06:22:17.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.083+0000 7ff516730740 1 -- start start 2024-09-19T06:22:17.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.083+0000 7ff5118f2640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4f80a3500 0x7ff4f80a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.083+0000 7ff516730740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db51ab40 0x55f1db519090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.083+0000 7ff516730740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db5195d0 0x55f1db519a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.083+0000 7ff516730740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db524370 0x55f1db526760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.083+0000 7ff516730740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f1db47f2f0 con 0x55f1db524370 2024-09-19T06:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.083+0000 7ff516730740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f1db426cf0 con 0x55f1db5195d0 2024-09-19T06:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.083+0000 7ff516730740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f1db47b2b0 con 0x55f1db51ab40 2024-09-19T06:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.084+0000 7ff5128f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db524370 0x55f1db526760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.084+0000 7ff5128f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db524370 0x55f1db526760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49288/0 (socket says 172.21.15.5:49288) 2024-09-19T06:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.084+0000 7ff5128f4640 1 -- 172.21.15.5:0/3133546419 learned_addr learned my addr 172.21.15.5:0/3133546419 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.084+0000 7ff5120f3640 1 --2- 172.21.15.5:0/3133546419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db51ab40 0x55f1db519090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.084+0000 7ff5128f4640 1 -- 172.21.15.5:0/3133546419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db51ab40 msgr2=0x55f1db519090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.084+0000 7ff5128f4640 1 --2- 172.21.15.5:0/3133546419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db51ab40 0x55f1db519090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.084+0000 7ff5128f4640 1 -- 172.21.15.5:0/3133546419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db5195d0 msgr2=0x55f1db519a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.084+0000 7ff5128f4640 1 --2- 172.21.15.5:0/3133546419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db5195d0 0x55f1db519a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.084+0000 7ff5128f4640 1 -- 172.21.15.5:0/3133546419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1db40b3a0 con 0x55f1db524370 2024-09-19T06:22:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.084+0000 7ff5118f2640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4f80a3500 0x7ff4f80a59c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc02a930 tx=0x7ff4fc041040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.084+0000 7ff5128f4640 1 --2- 172.21.15.5:0/3133546419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db524370 0x55f1db526760 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7ff504010750 tx=0x7ff504010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d617f2802665c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.085+0000 7ff500dea640 1 -- 172.21.15.5:0/3133546419 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50400ead0 con 0x55f1db524370 2024-09-19T06:22:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.085+0000 7ff500dea640 1 -- 172.21.15.5:0/3133546419 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff504010e60 con 0x55f1db524370 2024-09-19T06:22:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.085+0000 7ff500dea640 1 -- 172.21.15.5:0/3133546419 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504011620 con 0x55f1db524370 2024-09-19T06:22:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.086+0000 7ff516730740 1 -- 172.21.15.5:0/3133546419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db524370 msgr2=0x55f1db526760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.086+0000 7ff516730740 1 --2- 172.21.15.5:0/3133546419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db524370 0x55f1db526760 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7ff504010750 tx=0x7ff504010c10 comp rx=0 tx=0).stop 2024-09-19T06:22:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.086+0000 7ff516730740 1 -- 172.21.15.5:0/3133546419 shutdown_connections 2024-09-19T06:22:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.086+0000 7ff516730740 1 --2- 172.21.15.5:0/3133546419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db524370 0x55f1db526760 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.086+0000 7ff516730740 1 --2- 172.21.15.5:0/3133546419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db5195d0 0x55f1db519a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.086+0000 7ff516730740 1 --2- 172.21.15.5:0/3133546419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db51ab40 0x55f1db519090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.086+0000 7ff516730740 1 -- 172.21.15.5:0/3133546419 >> 172.21.15.5:0/3133546419 conn(0x55f1db4e2310 msgr2=0x55f1db515c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.086+0000 7ff516730740 1 -- 172.21.15.5:0/3133546419 shutdown_connections 2024-09-19T06:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.086+0000 7ff516730740 1 -- 172.21.15.5:0/3133546419 wait complete. 2024-09-19T06:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.087+0000 7ff516730740 1 -- start start 2024-09-19T06:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.087+0000 7ff516730740 1 Processor -- start 2024-09-19T06:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.088+0000 7ff516730740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db5195d0 0x55f1db523cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.088+0000 7ff516730740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db51ab40 0x55f1db520f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.088+0000 7ff516730740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db524370 0x55f1db5214d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.088+0000 7ff516730740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f1db527070 con 0x55f1db5195d0 2024-09-19T06:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.088+0000 7ff516730740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f1db43a060 con 0x55f1db51ab40 2024-09-19T06:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.088+0000 7ff516730740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f1db5271f0 con 0x55f1db524370 2024-09-19T06:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.088+0000 7ff516730740 10 client.0 _ll_register_callbacks cb 0x55f1db18d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.088+0000 7ff5120f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db5195d0 0x55f1db523cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.088+0000 7ff5128f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db524370 0x55f1db5214d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.088+0000 7ff5120f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db5195d0 0x55f1db523cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49290/0 (socket says 172.21.15.5:49290) 2024-09-19T06:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.088+0000 7ff5120f3640 1 -- 172.21.15.5:0/2641252513 learned_addr learned my addr 172.21.15.5:0/2641252513 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.088+0000 7ff5118f2640 1 --2- 172.21.15.5:0/2641252513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db51ab40 0x55f1db520f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.089+0000 7ff5120f3640 1 -- 172.21.15.5:0/2641252513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db524370 msgr2=0x55f1db5214d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.089+0000 7ff5120f3640 1 --2- 172.21.15.5:0/2641252513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db524370 0x55f1db5214d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.089+0000 7ff5120f3640 1 -- 172.21.15.5:0/2641252513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db51ab40 msgr2=0x55f1db520f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.089+0000 7ff5120f3640 1 --2- 172.21.15.5:0/2641252513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db51ab40 0x55f1db520f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.089+0000 7ff5120f3640 1 -- 172.21.15.5:0/2641252513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1db1f03c0 con 0x55f1db5195d0 2024-09-19T06:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.089+0000 7ff5120f3640 1 --2- 172.21.15.5:0/2641252513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db5195d0 0x55f1db523cc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7ff508004770 tx=0x7ff50809b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7edf62297bcf66e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.089+0000 7ff4f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.089+0000 7ff4f5ffb640 1 -- 172.21.15.5:0/2641252513 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5080b9060 con 0x55f1db5195d0 2024-09-19T06:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.089+0000 7ff516730740 1 -- 172.21.15.5:0/2641252513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff504010e60 con 0x55f1db5195d0 2024-09-19T06:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.089+0000 7ff4f5ffb640 1 -- 172.21.15.5:0/2641252513 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7ff508098040 con 0x55f1db5195d0 2024-09-19T06:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.090+0000 7ff4f5ffb640 1 -- 172.21.15.5:0/2641252513 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff5080b3030 con 0x55f1db5195d0 2024-09-19T06:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.090+0000 7ff516730740 10 client.20232 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.090+0000 7ff516730740 1 -- 172.21.15.5:0/2641252513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f1db532ad0 con 0x55f1db5195d0 2024-09-19T06:22:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.091+0000 7ff4f5ffb640 1 -- 172.21.15.5:0/2641252513 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff50809fd80 con 0x55f1db5195d0 2024-09-19T06:22:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.091+0000 7ff4f5ffb640 1 -- 172.21.15.5:0/2641252513 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7ff508097720 con 0x55f1db5195d0 2024-09-19T06:22:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.091+0000 7ff516730740 10 client.20232 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.091+0000 7ff516730740 10 client.20232 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:22:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.091+0000 7ff516730740 20 client.20232 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.091+0000 7ff516730740 1 --2- 172.21.15.5:0/2641252513 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f1db533ee0 0x55f1db5362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.091+0000 7ff516730740 4 client.20232 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.091+0000 7ff516730740 1 -- 172.21.15.5:0/2641252513 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff504010e60 con 0x55f1db533ee0 2024-09-19T06:22:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.091+0000 7ff5118f2640 1 --2- 172.21.15.5:0/2641252513 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f1db533ee0 0x55f1db5362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.092+0000 7ff5118f2640 1 --2- 172.21.15.5:0/2641252513 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f1db533ee0 0x55f1db5362d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f7c07c911d7c54a server_cookie=c4c24b85ccd27f67 in_seq=0 out_seq=0 2024-09-19T06:22:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.092+0000 7ff4f5ffb640 10 client.20232 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:22:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff4f5ffb640 1 -- 172.21.15.5:0/2641252513 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff504010e60 con 0x55f1db533ee0 2024-09-19T06:22:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff4f5ffb640 10 client.20232 handle_command_reply: tid=0 2024-09-19T06:22:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff516730740 1 client.20232 shutdown 2024-09-19T06:22:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff516730740 1 -- 172.21.15.5:0/2641252513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db5195d0 msgr2=0x55f1db523cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff516730740 1 --2- 172.21.15.5:0/2641252513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db5195d0 0x55f1db523cc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7ff508004770 tx=0x7ff50809b8a0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff516730740 1 -- 172.21.15.5:0/2641252513 shutdown_connections 2024-09-19T06:22:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff516730740 1 --2- 172.21.15.5:0/2641252513 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f1db533ee0 0x55f1db5362d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff516730740 1 --2- 172.21.15.5:0/2641252513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db524370 0x55f1db5214d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff516730740 1 --2- 172.21.15.5:0/2641252513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db51ab40 0x55f1db520f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff4f5ffb640 0 client.20232 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:22:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff516730740 1 --2- 172.21.15.5:0/2641252513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db5195d0 0x55f1db523cc0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff516730740 1 -- 172.21.15.5:0/2641252513 >> 172.21.15.5:0/2641252513 conn(0x55f1db4e2310 msgr2=0x55f1db51b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff516730740 1 -- 172.21.15.5:0/2641252513 shutdown_connections 2024-09-19T06:22:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.094+0000 7ff516730740 1 -- 172.21.15.5:0/2641252513 wait complete. 2024-09-19T06:22:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.095+0000 7ff516730740 20 client.20232 trim_cache size 0 max 16384 2024-09-19T06:22:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.116+0000 7ff516730740 1 Processor -- start 2024-09-19T06:22:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.116+0000 7ff516730740 1 -- start start 2024-09-19T06:22:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.116+0000 7ff516730740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db5195d0 0x55f1db520f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.116+0000 7ff516730740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db51ab40 0x55f1db530310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.116+0000 7ff516730740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db524370 0x55f1db53bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff516730740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1ab630 con 0x55f1db524370 2024-09-19T06:22:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff516730740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1ad170 con 0x55f1db51ab40 2024-09-19T06:22:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff516730740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1abd00 con 0x55f1db5195d0 2024-09-19T06:22:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff5120f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db5195d0 0x55f1db520f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff5118f2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db51ab40 0x55f1db530310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff5120f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db5195d0 0x55f1db520f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44672/0 (socket says 172.21.15.5:44672) 2024-09-19T06:22:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff5120f3640 1 -- 172.21.15.5:0/1754473146 learned_addr learned my addr 172.21.15.5:0/1754473146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff5128f4640 1 --2- 172.21.15.5:0/1754473146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db524370 0x55f1db53bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff5120f3640 1 -- 172.21.15.5:0/1754473146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db51ab40 msgr2=0x55f1db530310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff5120f3640 1 --2- 172.21.15.5:0/1754473146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db51ab40 0x55f1db530310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff5120f3640 1 -- 172.21.15.5:0/1754473146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db524370 msgr2=0x55f1db53bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff5120f3640 1 --2- 172.21.15.5:0/1754473146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db524370 0x55f1db53bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff5120f3640 1 -- 172.21.15.5:0/1754473146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1db516960 con 0x55f1db5195d0 2024-09-19T06:22:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.117+0000 7ff5120f3640 1 --2- 172.21.15.5:0/1754473146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db5195d0 0x55f1db520f40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff50800cba0 tx=0x7ff50809b510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90edd32897cf2369 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.118+0000 7ff4f7fff640 1 -- 172.21.15.5:0/1754473146 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5080b9040 con 0x55f1db5195d0 2024-09-19T06:22:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.118+0000 7ff4f7fff640 1 -- 172.21.15.5:0/1754473146 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5080a4640 con 0x55f1db5195d0 2024-09-19T06:22:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.118+0000 7ff4f7fff640 1 -- 172.21.15.5:0/1754473146 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50809aa00 con 0x55f1db5195d0 2024-09-19T06:22:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.118+0000 7ff516730740 1 -- 172.21.15.5:0/1754473146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db5195d0 msgr2=0x55f1db520f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.119+0000 7ff516730740 1 --2- 172.21.15.5:0/1754473146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db5195d0 0x55f1db520f40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff50800cba0 tx=0x7ff50809b510 comp rx=0 tx=0).stop 2024-09-19T06:22:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.119+0000 7ff516730740 1 -- 172.21.15.5:0/1754473146 shutdown_connections 2024-09-19T06:22:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.119+0000 7ff516730740 1 --2- 172.21.15.5:0/1754473146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db524370 0x55f1db53bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.119+0000 7ff516730740 1 --2- 172.21.15.5:0/1754473146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db51ab40 0x55f1db530310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.119+0000 7ff516730740 1 --2- 172.21.15.5:0/1754473146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db5195d0 0x55f1db520f40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.119+0000 7ff516730740 1 -- 172.21.15.5:0/1754473146 >> 172.21.15.5:0/1754473146 conn(0x55f1db4e2310 msgr2=0x55f1db520830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.119+0000 7ff516730740 1 -- 172.21.15.5:0/1754473146 shutdown_connections 2024-09-19T06:22:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.119+0000 7ff516730740 1 -- 172.21.15.5:0/1754473146 wait complete. 2024-09-19T06:22:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.119+0000 7ff516730740 1 -- start start 2024-09-19T06:22:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff516730740 1 Processor -- start 2024-09-19T06:22:17.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff516730740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db5195d0 0x55f1db5282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff516730740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db51ab40 0x55f1db528820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff5120f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db5195d0 0x55f1db5282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff516730740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db524370 0x55f1db528d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff5120f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db5195d0 0x55f1db5282e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37370/0 (socket says 172.21.15.5:37370) 2024-09-19T06:22:17.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff5118f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db51ab40 0x55f1db528820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff5118f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db51ab40 0x55f1db528820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49310/0 (socket says 172.21.15.5:49310) 2024-09-19T06:22:17.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff5120f3640 1 -- 172.21.15.5:0/3171030874 learned_addr learned my addr 172.21.15.5:0/3171030874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:17.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff516730740 1 -- 172.21.15.5:0/3171030874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1acaa0 con 0x55f1db51ab40 2024-09-19T06:22:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff516730740 1 -- 172.21.15.5:0/3171030874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1ac3d0 con 0x55f1db5195d0 2024-09-19T06:22:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff516730740 1 -- 172.21.15.5:0/3171030874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1a9630 con 0x55f1db524370 2024-09-19T06:22:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff516730740 10 client.0 _ll_register_callbacks cb 0x7ff508097c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.120+0000 7ff5128f4640 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db524370 0x55f1db528d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.121+0000 7ff5120f3640 1 -- 172.21.15.5:0/3171030874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db524370 msgr2=0x55f1db528d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.121+0000 7ff5120f3640 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db524370 0x55f1db528d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.121+0000 7ff5120f3640 1 -- 172.21.15.5:0/3171030874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db51ab40 msgr2=0x55f1db528820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.121+0000 7ff5120f3640 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db51ab40 0x55f1db528820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.121+0000 7ff5120f3640 1 -- 172.21.15.5:0/3171030874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1db59db80 con 0x55f1db5195d0 2024-09-19T06:22:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.121+0000 7ff5118f2640 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db51ab40 0x55f1db528820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.121+0000 7ff5120f3640 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db5195d0 0x55f1db5282e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff50809b960 tx=0x7ff508012690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22643b6e3b631e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.121+0000 7ff4f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:22:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.121+0000 7ff4f5ffb640 1 -- 172.21.15.5:0/3171030874 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5080b9070 con 0x55f1db5195d0 2024-09-19T06:22:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.121+0000 7ff4f5ffb640 1 -- 172.21.15.5:0/3171030874 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7ff50809ae20 con 0x55f1db5195d0 2024-09-19T06:22:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.121+0000 7ff516730740 1 -- 172.21.15.5:0/3171030874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5080a4640 con 0x55f1db5195d0 2024-09-19T06:22:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.122+0000 7ff4f5ffb640 1 -- 172.21.15.5:0/3171030874 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff50809b060 con 0x55f1db5195d0 2024-09-19T06:22:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.122+0000 7ff516730740 10 client.29609 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.122+0000 7ff516730740 1 -- 172.21.15.5:0/3171030874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f1db59ee60 con 0x55f1db5195d0 2024-09-19T06:22:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.122+0000 7ff4f5ffb640 1 -- 172.21.15.5:0/3171030874 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5080a0f10 con 0x55f1db5195d0 2024-09-19T06:22:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.122+0000 7ff4f5ffb640 1 -- 172.21.15.5:0/3171030874 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7ff50800f450 con 0x55f1db5195d0 2024-09-19T06:22:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.122+0000 7ff516730740 10 client.29609 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.122+0000 7ff516730740 10 client.29609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:22:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.122+0000 7ff516730740 20 client.29609 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.122+0000 7ff516730740 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f1db59f330 0x55f1db5a1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.122+0000 7ff516730740 4 client.29609 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:22:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.122+0000 7ff516730740 1 -- 172.21.15.5:0/3171030874 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff5080a4640 con 0x55f1db59f330 2024-09-19T06:22:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.123+0000 7ff5118f2640 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f1db59f330 0x55f1db5a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.123+0000 7ff4f5ffb640 10 client.29609 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:22:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.123+0000 7ff5118f2640 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f1db59f330 0x55f1db5a1720 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6eba4b19567cc57 server_cookie=d820a6832ea8dab2 in_seq=0 out_seq=0 2024-09-19T06:22:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff4f5ffb640 1 -- 172.21.15.5:0/3171030874 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff5080a4640 con 0x55f1db59f330 2024-09-19T06:22:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff4f5ffb640 10 client.29609 handle_command_reply: tid=0 2024-09-19T06:22:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff516730740 1 client.29609 shutdown 2024-09-19T06:22:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff516730740 1 -- 172.21.15.5:0/3171030874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db5195d0 msgr2=0x55f1db5282e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff516730740 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db5195d0 0x55f1db5282e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff50809b960 tx=0x7ff508012690 comp rx=0 tx=0).stop 2024-09-19T06:22:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff516730740 1 -- 172.21.15.5:0/3171030874 shutdown_connections 2024-09-19T06:22:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff516730740 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55f1db59f330 0x55f1db5a1720 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff516730740 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1db524370 0x55f1db528d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff516730740 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1db51ab40 0x55f1db528820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff516730740 1 --2- 172.21.15.5:0/3171030874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1db5195d0 0x55f1db5282e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff4f5ffb640 0 client.29609 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:22:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff516730740 1 -- 172.21.15.5:0/3171030874 >> 172.21.15.5:0/3171030874 conn(0x55f1db4e2310 msgr2=0x55f1db5227d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff516730740 1 -- 172.21.15.5:0/3171030874 shutdown_connections 2024-09-19T06:22:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff516730740 1 -- 172.21.15.5:0/3171030874 wait complete. 2024-09-19T06:22:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.124+0000 7ff516730740 20 client.29609 trim_cache size 0 max 16384 2024-09-19T06:22:17.133 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:22:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.125+0000 7ff514357640 1 -- 172.21.15.5:0/4166068298 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4f80a3500 msgr2=0x7ff4f80a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.125+0000 7ff514357640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4f80a3500 0x7ff4f80a59c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc02a930 tx=0x7ff4fc041040 comp rx=0 tx=0).stop 2024-09-19T06:22:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.125+0000 7ff514357640 1 -- 172.21.15.5:0/4166068298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50c100a90 msgr2=0x7ff50c197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.125+0000 7ff514357640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50c100a90 0x7ff50c197240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff508012dd0 tx=0x7ff50800c6a0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.125+0000 7ff514357640 1 -- 172.21.15.5:0/4166068298 shutdown_connections 2024-09-19T06:22:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.125+0000 7ff514357640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c106a90 0x7ff50c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.125+0000 7ff514357640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff50c101430 0x7ff50c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.125+0000 7ff514357640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff4f80a3500 0x7ff4f80a59c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.125+0000 7ff514357640 1 --2- 172.21.15.5:0/4166068298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50c100a90 0x7ff50c197240 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.125+0000 7ff514357640 1 -- 172.21.15.5:0/4166068298 >> 172.21.15.5:0/4166068298 conn(0x7ff50c0fc7d0 msgr2=0x7ff50c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.126+0000 7ff514357640 1 -- 172.21.15.5:0/4166068298 shutdown_connections 2024-09-19T06:22:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:17.126+0000 7ff514357640 1 -- 172.21.15.5:0/4166068298 wait complete. 2024-09-19T06:22:17.566 INFO:tasks.workunit.client.0.smithi005.stdout: 10 56316 8.75 MB/sec execute 221 sec latency 330.757 ms 2024-09-19T06:22:18.566 INFO:tasks.workunit.client.0.smithi005.stdout: 10 56507 8.75 MB/sec execute 222 sec latency 220.794 ms 2024-09-19T06:22:18.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:18 smithi005 ceph-mon[30955]: pgmap v598: 97 pgs: 97 active+clean; 938 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-09-19T06:22:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:18 smithi049 ceph-mon[34640]: pgmap v598: 97 pgs: 97 active+clean; 938 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-09-19T06:22:19.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:18 smithi157 ceph-mon[35346]: pgmap v598: 97 pgs: 97 active+clean; 938 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-09-19T06:22:19.566 INFO:tasks.workunit.client.0.smithi005.stdout: 10 56623 8.73 MB/sec execute 223 sec latency 110.264 ms 2024-09-19T06:22:20.087 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:22:20.463 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:22:20.566 INFO:tasks.workunit.client.0.smithi005.stdout: 10 56754 8.73 MB/sec execute 224 sec latency 101.422 ms 2024-09-19T06:22:20.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:20 smithi005 ceph-mon[30955]: pgmap v599: 97 pgs: 97 active+clean; 938 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 301 op/s 2024-09-19T06:22:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:20 smithi049 ceph-mon[34640]: pgmap v599: 97 pgs: 97 active+clean; 938 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 301 op/s 2024-09-19T06:22:21.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.031+0000 7f78e4184640 1 -- 172.21.15.5:0/381986992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78dc1049a0 msgr2=0x7f78dc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:21.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.031+0000 7f78e4184640 1 --2- 172.21.15.5:0/381986992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78dc1049a0 0x7f78dc1089f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009900 tx=0x7f78cc02a410 comp rx=0 tx=0).stop 2024-09-19T06:22:21.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.031+0000 7f78e4184640 1 -- 172.21.15.5:0/381986992 shutdown_connections 2024-09-19T06:22:21.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.031+0000 7f78e4184640 1 --2- 172.21.15.5:0/381986992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78dc109120 0x7f78dc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.031+0000 7f78e4184640 1 --2- 172.21.15.5:0/381986992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78dc1049a0 0x7f78dc1089f0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.031+0000 7f78e4184640 1 --2- 172.21.15.5:0/381986992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78dc103ff0 0x7f78dc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.031+0000 7f78e4184640 1 -- 172.21.15.5:0/381986992 >> 172.21.15.5:0/381986992 conn(0x7f78dc0ffab0 msgr2=0x7f78dc101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:21.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.031+0000 7f78e4184640 1 -- 172.21.15.5:0/381986992 shutdown_connections 2024-09-19T06:22:21.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.031+0000 7f78e4184640 1 -- 172.21.15.5:0/381986992 wait complete. 2024-09-19T06:22:21.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.032+0000 7f78e4184640 1 Processor -- start 2024-09-19T06:22:21.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.032+0000 7f78e4184640 1 -- start start 2024-09-19T06:22:21.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.032+0000 7f78e4184640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78dc103ff0 0x7f78dc19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.032+0000 7f78e4184640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78dc1049a0 0x7f78dc19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.032+0000 7f78e4184640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78dc109120 0x7f78dc19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.032+0000 7f78e4184640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc113fd0 con 0x7f78dc1049a0 2024-09-19T06:22:21.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.032+0000 7f78e4184640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc113e50 con 0x7f78dc103ff0 2024-09-19T06:22:21.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.032+0000 7f78e4184640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc114150 con 0x7f78dc109120 2024-09-19T06:22:21.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.033+0000 7f78e171f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78dc1049a0 0x7f78dc19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.033+0000 7f78e171f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78dc1049a0 0x7f78dc19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49338/0 (socket says 172.21.15.5:49338) 2024-09-19T06:22:21.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.033+0000 7f78e171f640 1 -- 172.21.15.5:0/3405902785 learned_addr learned my addr 172.21.15.5:0/3405902785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:21.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.033+0000 7f78e2721640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78dc109120 0x7f78dc19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.033+0000 7f78e171f640 1 -- 172.21.15.5:0/3405902785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78dc109120 msgr2=0x7f78dc19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.033+0000 7f78e171f640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78dc109120 0x7f78dc19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.033+0000 7f78e171f640 1 -- 172.21.15.5:0/3405902785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78dc103ff0 msgr2=0x7f78dc19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.033+0000 7f78e171f640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78dc103ff0 0x7f78dc19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.033+0000 7f78e171f640 1 -- 172.21.15.5:0/3405902785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc19f4e0 con 0x7f78dc1049a0 2024-09-19T06:22:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.033+0000 7f78e171f640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78dc1049a0 0x7f78dc19aa70 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f78cc005940 tx=0x7f78cc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.034+0000 7f78d2ffd640 1 -- 172.21.15.5:0/3405902785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc03f030 con 0x7f78dc1049a0 2024-09-19T06:22:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.034+0000 7f78d2ffd640 1 -- 172.21.15.5:0/3405902785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78cc0044c0 con 0x7f78dc1049a0 2024-09-19T06:22:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.034+0000 7f78d2ffd640 1 -- 172.21.15.5:0/3405902785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc035070 con 0x7f78dc1049a0 2024-09-19T06:22:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.034+0000 7f78e4184640 1 -- 172.21.15.5:0/3405902785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78dc19f770 con 0x7f78dc1049a0 2024-09-19T06:22:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.034+0000 7f78e4184640 1 -- 172.21.15.5:0/3405902785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78dc1a70b0 con 0x7f78dc1049a0 2024-09-19T06:22:21.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.036+0000 7f78d2ffd640 1 -- 172.21.15.5:0/3405902785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f78cc042030 con 0x7f78dc1049a0 2024-09-19T06:22:21.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.037+0000 7f78e655d740 1 Processor -- start 2024-09-19T06:22:21.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.037+0000 7f78e655d740 1 -- start start 2024-09-19T06:22:21.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.037+0000 7f78e655d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 0x563e8f35b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.037+0000 7f78e655d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f35b5d0 0x563e8f35ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.037+0000 7f78e655d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f366370 0x563e8f368760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.037+0000 7f78e655d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e8f2c12f0 con 0x563e8f366370 2024-09-19T06:22:21.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.037+0000 7f78e655d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563e8f268cf0 con 0x563e8f35cb40 2024-09-19T06:22:21.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.037+0000 7f78e655d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563e8f2bd2b0 con 0x563e8f35b5d0 2024-09-19T06:22:21.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.037+0000 7f78e2721640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 0x563e8f35b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78d2ffd640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f78c80a3430 0x7f78c80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78e2721640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 0x563e8f35b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37398/0 (socket says 172.21.15.5:37398) 2024-09-19T06:22:21.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78e2721640 1 -- 172.21.15.5:0/930998031 learned_addr learned my addr 172.21.15.5:0/930998031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:21.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78e171f640 1 --2- 172.21.15.5:0/930998031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f366370 0x563e8f368760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78e1f20640 1 --2- 172.21.15.5:0/930998031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f35b5d0 0x563e8f35ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78e2721640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f78c80a3430 0x7f78c80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78d2ffd640 1 -- 172.21.15.5:0/3405902785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f78cc0efe00 con 0x7f78dc1049a0 2024-09-19T06:22:21.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78e2721640 1 -- 172.21.15.5:0/930998031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f35b5d0 msgr2=0x563e8f35ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:21.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78e2721640 1 --2- 172.21.15.5:0/930998031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f35b5d0 0x563e8f35ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78e2721640 1 -- 172.21.15.5:0/930998031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f366370 msgr2=0x563e8f368760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:21.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78e2721640 1 --2- 172.21.15.5:0/930998031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f366370 0x563e8f368760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78e2721640 1 -- 172.21.15.5:0/930998031 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e8f24d3a0 con 0x563e8f35cb40 2024-09-19T06:22:21.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78e171f640 1 --2- 172.21.15.5:0/930998031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f366370 0x563e8f368760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:22:21.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.038+0000 7f78e2721640 1 --2- 172.21.15.5:0/930998031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 0x563e8f35b090 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f78d40129f0 tx=0x7f78d4012eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6326683f50afcd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:21.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.039+0000 7f78e2721640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f78c80a3430 0x7f78c80a58f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f78d400e720 tx=0x7f78d4020000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:21.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.039+0000 7f78c7fff640 1 -- 172.21.15.5:0/930998031 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d40120d0 con 0x563e8f35cb40 2024-09-19T06:22:21.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.039+0000 7f78c7fff640 1 -- 172.21.15.5:0/930998031 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78d4023020 con 0x563e8f35cb40 2024-09-19T06:22:21.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.039+0000 7f78c7fff640 1 -- 172.21.15.5:0/930998031 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4004030 con 0x563e8f35cb40 2024-09-19T06:22:21.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.040+0000 7f78e655d740 1 -- 172.21.15.5:0/930998031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 msgr2=0x563e8f35b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:21.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.040+0000 7f78e655d740 1 --2- 172.21.15.5:0/930998031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 0x563e8f35b090 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f78d40129f0 tx=0x7f78d4012eb0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.040+0000 7f78e655d740 1 -- 172.21.15.5:0/930998031 shutdown_connections 2024-09-19T06:22:21.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.040+0000 7f78e655d740 1 --2- 172.21.15.5:0/930998031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f366370 0x563e8f368760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.040+0000 7f78e655d740 1 --2- 172.21.15.5:0/930998031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f35b5d0 0x563e8f35ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.040+0000 7f78e655d740 1 --2- 172.21.15.5:0/930998031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 0x563e8f35b090 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.040+0000 7f78e655d740 1 -- 172.21.15.5:0/930998031 >> 172.21.15.5:0/930998031 conn(0x563e8f324310 msgr2=0x563e8f357c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:21.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.040+0000 7f78e655d740 1 -- 172.21.15.5:0/930998031 shutdown_connections 2024-09-19T06:22:21.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.040+0000 7f78e655d740 1 -- 172.21.15.5:0/930998031 wait complete. 2024-09-19T06:22:21.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.040+0000 7f78e655d740 1 -- start start 2024-09-19T06:22:21.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.041+0000 7f78e655d740 1 Processor -- start 2024-09-19T06:22:21.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.041+0000 7f78e655d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35b5d0 0x563e8f365db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.041+0000 7f78e655d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 0x563e8f363060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.041+0000 7f78e655d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f366370 0x563e8f3637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.041+0000 7f78e171f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 0x563e8f363060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e171f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 0x563e8f363060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49350/0 (socket says 172.21.15.5:49350) 2024-09-19T06:22:21.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e171f640 1 -- 172.21.15.5:0/1332821363 learned_addr learned my addr 172.21.15.5:0/1332821363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e655d740 1 -- 172.21.15.5:0/1332821363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e8f368fb0 con 0x563e8f35cb40 2024-09-19T06:22:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e655d740 1 -- 172.21.15.5:0/1332821363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563e8f27c060 con 0x563e8f35b5d0 2024-09-19T06:22:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e655d740 1 -- 172.21.15.5:0/1332821363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563e8f369130 con 0x563e8f366370 2024-09-19T06:22:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e2721640 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f366370 0x563e8f3637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e1f20640 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35b5d0 0x563e8f365db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e655d740 10 client.0 _ll_register_callbacks cb 0x563e8efcf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e2721640 1 -- 172.21.15.5:0/1332821363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35b5d0 msgr2=0x563e8f365db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e2721640 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35b5d0 0x563e8f365db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e2721640 1 -- 172.21.15.5:0/1332821363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 msgr2=0x563e8f363060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e2721640 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 0x563e8f363060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.042+0000 7f78e2721640 1 -- 172.21.15.5:0/1332821363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e8f0323c0 con 0x563e8f366370 2024-09-19T06:22:21.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.043+0000 7f78e2721640 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f366370 0x563e8f3637f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009b20 tx=0x7f78d40122e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9777518557374800 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.043+0000 7f78c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:22:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.043+0000 7f78c57fa640 1 -- 172.21.15.5:0/1332821363 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4004a40 con 0x563e8f366370 2024-09-19T06:22:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.043+0000 7f78e655d740 1 -- 172.21.15.5:0/1332821363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d4023020 con 0x563e8f366370 2024-09-19T06:22:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.043+0000 7f78c57fa640 1 -- 172.21.15.5:0/1332821363 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f78d4004be0 con 0x563e8f366370 2024-09-19T06:22:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.044+0000 7f78c57fa640 1 -- 172.21.15.5:0/1332821363 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f78d4004e00 con 0x563e8f366370 2024-09-19T06:22:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.044+0000 7f78e655d740 10 client.29557 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.044+0000 7f78e655d740 1 -- 172.21.15.5:0/1332821363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563e8f374a80 con 0x563e8f366370 2024-09-19T06:22:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.044+0000 7f78c57fa640 1 -- 172.21.15.5:0/1332821363 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78d4023020 con 0x563e8f366370 2024-09-19T06:22:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.044+0000 7f78c57fa640 1 -- 172.21.15.5:0/1332821363 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f78d401e070 con 0x563e8f366370 2024-09-19T06:22:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.044+0000 7f78e655d740 10 client.29557 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.044+0000 7f78e655d740 10 client.29557 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:22:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.044+0000 7f78e655d740 20 client.29557 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.045+0000 7f78e655d740 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x563e8f375e90 0x563e8f378280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.045+0000 7f78e655d740 4 client.29557 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.045+0000 7f78e655d740 1 -- 172.21.15.5:0/1332821363 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e8f2ce860 con 0x563e8f375e90 2024-09-19T06:22:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.045+0000 7f78e1f20640 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x563e8f375e90 0x563e8f378280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.045+0000 7f78c57fa640 10 client.29557 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:22:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.045+0000 7f78e1f20640 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x563e8f375e90 0x563e8f378280 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=464f4fdee8cd822e server_cookie=e2c81f6bf5f113fa in_seq=0 out_seq=0 2024-09-19T06:22:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.046+0000 7f78c57fa640 1 -- 172.21.15.5:0/1332821363 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e8f2ce860 con 0x563e8f375e90 2024-09-19T06:22:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.046+0000 7f78c57fa640 10 client.29557 handle_command_reply: tid=0 2024-09-19T06:22:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.046+0000 7f78e655d740 1 client.29557 shutdown 2024-09-19T06:22:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.046+0000 7f78e655d740 1 -- 172.21.15.5:0/1332821363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f366370 msgr2=0x563e8f3637f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.046+0000 7f78e655d740 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f366370 0x563e8f3637f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009b20 tx=0x7f78d40122e0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.047+0000 7f78e655d740 1 -- 172.21.15.5:0/1332821363 shutdown_connections 2024-09-19T06:22:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.047+0000 7f78e655d740 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x563e8f375e90 0x563e8f378280 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.047+0000 7f78e655d740 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f366370 0x563e8f3637f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.047+0000 7f78e655d740 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 0x563e8f363060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.047+0000 7f78e655d740 1 --2- 172.21.15.5:0/1332821363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35b5d0 0x563e8f365db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.047+0000 7f78c57fa640 0 client.29557 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:22:21.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.047+0000 7f78e655d740 1 -- 172.21.15.5:0/1332821363 >> 172.21.15.5:0/1332821363 conn(0x563e8f324310 msgr2=0x563e8f35d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:21.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.047+0000 7f78e655d740 1 -- 172.21.15.5:0/1332821363 shutdown_connections 2024-09-19T06:22:21.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.047+0000 7f78e655d740 1 -- 172.21.15.5:0/1332821363 wait complete. 2024-09-19T06:22:21.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.047+0000 7f78e655d740 20 client.29557 trim_cache size 0 max 16384 2024-09-19T06:22:21.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:20 smithi157 ceph-mon[35346]: pgmap v599: 97 pgs: 97 active+clean; 938 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 301 op/s 2024-09-19T06:22:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.064+0000 7f78e655d740 1 Processor -- start 2024-09-19T06:22:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.064+0000 7f78e655d740 1 -- start start 2024-09-19T06:22:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.064+0000 7f78e655d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35b5d0 0x563e8f362920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.064+0000 7f78e655d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 0x563e8f362150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.064+0000 7f78e655d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f366370 0x563e8f35e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.064+0000 7f78e655d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc1afb40 con 0x563e8f35cb40 2024-09-19T06:22:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.064+0000 7f78e655d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc1b1680 con 0x563e8f35b5d0 2024-09-19T06:22:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.064+0000 7f78e655d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc1b0210 con 0x563e8f366370 2024-09-19T06:22:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.064+0000 7f78e171f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 0x563e8f362150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.064+0000 7f78e171f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 0x563e8f362150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49362/0 (socket says 172.21.15.5:49362) 2024-09-19T06:22:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.065+0000 7f78e171f640 1 -- 172.21.15.5:0/324747583 learned_addr learned my addr 172.21.15.5:0/324747583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.065+0000 7f78e2721640 1 --2- 172.21.15.5:0/324747583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f366370 0x563e8f35e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.065+0000 7f78e1f20640 1 --2- 172.21.15.5:0/324747583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35b5d0 0x563e8f362920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.065+0000 7f78e171f640 1 -- 172.21.15.5:0/324747583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f366370 msgr2=0x563e8f35e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.065+0000 7f78e171f640 1 --2- 172.21.15.5:0/324747583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f366370 0x563e8f35e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.065+0000 7f78e171f640 1 -- 172.21.15.5:0/324747583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35b5d0 msgr2=0x563e8f362920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.065+0000 7f78e171f640 1 --2- 172.21.15.5:0/324747583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35b5d0 0x563e8f362920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.065+0000 7f78e171f640 1 -- 172.21.15.5:0/324747583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e8f368ec0 con 0x563e8f35cb40 2024-09-19T06:22:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.065+0000 7f78e171f640 1 --2- 172.21.15.5:0/324747583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 0x563e8f362150 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0ed5e0 tx=0x7f78cc0c0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b56b610463c768f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.065+0000 7f78c77fe640 1 -- 172.21.15.5:0/324747583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc049070 con 0x563e8f35cb40 2024-09-19T06:22:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.065+0000 7f78c77fe640 1 -- 172.21.15.5:0/324747583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78cc0bc0c0 con 0x563e8f35cb40 2024-09-19T06:22:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.065+0000 7f78c77fe640 1 -- 172.21.15.5:0/324747583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0b8730 con 0x563e8f35cb40 2024-09-19T06:22:21.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.066+0000 7f78e655d740 1 -- 172.21.15.5:0/324747583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 msgr2=0x563e8f362150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.066+0000 7f78e655d740 1 --2- 172.21.15.5:0/324747583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 0x563e8f362150 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0ed5e0 tx=0x7f78cc0c0040 comp rx=0 tx=0).stop 2024-09-19T06:22:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.067+0000 7f78e655d740 1 -- 172.21.15.5:0/324747583 shutdown_connections 2024-09-19T06:22:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.067+0000 7f78e655d740 1 --2- 172.21.15.5:0/324747583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f366370 0x563e8f35e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.067+0000 7f78e655d740 1 --2- 172.21.15.5:0/324747583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f35cb40 0x563e8f362150 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.067+0000 7f78e655d740 1 --2- 172.21.15.5:0/324747583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35b5d0 0x563e8f362920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.067+0000 7f78e655d740 1 -- 172.21.15.5:0/324747583 >> 172.21.15.5:0/324747583 conn(0x563e8f324310 msgr2=0x563e8f352a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.067+0000 7f78e655d740 1 -- 172.21.15.5:0/324747583 shutdown_connections 2024-09-19T06:22:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.067+0000 7f78e655d740 1 -- 172.21.15.5:0/324747583 wait complete. 2024-09-19T06:22:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.067+0000 7f78e655d740 1 -- start start 2024-09-19T06:22:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.067+0000 7f78e655d740 1 Processor -- start 2024-09-19T06:22:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.068+0000 7f78e655d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f35b5d0 0x563e8f369b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.068+0000 7f78e655d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 0x563e8f36a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.068+0000 7f78e655d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f366370 0x563e8f36e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.068+0000 7f78e655d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc1b0fb0 con 0x563e8f366370 2024-09-19T06:22:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.068+0000 7f78e655d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc1b08e0 con 0x563e8f35cb40 2024-09-19T06:22:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e655d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc1adb40 con 0x563e8f35b5d0 2024-09-19T06:22:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e1f20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f35b5d0 0x563e8f369b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e1f20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f35b5d0 0x563e8f369b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44730/0 (socket says 172.21.15.5:44730) 2024-09-19T06:22:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e1f20640 1 -- 172.21.15.5:0/1087268369 learned_addr learned my addr 172.21.15.5:0/1087268369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e655d740 10 client.0 _ll_register_callbacks cb 0x7f78d80093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e2721640 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f366370 0x563e8f36e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e171f640 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 0x563e8f36a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e2721640 1 -- 172.21.15.5:0/1087268369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f35b5d0 msgr2=0x563e8f369b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e2721640 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f35b5d0 0x563e8f369b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e2721640 1 -- 172.21.15.5:0/1087268369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 msgr2=0x563e8f36a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e2721640 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 0x563e8f36a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e2721640 1 -- 172.21.15.5:0/1087268369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e8f36e8d0 con 0x563e8f366370 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e171f640 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 0x563e8f36a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78e2721640 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f366370 0x563e8f36e1e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f78d4003a70 tx=0x7f78d4009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e216d12d07ba7a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.069+0000 7f78c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.070+0000 7f78c57fa640 1 -- 172.21.15.5:0/1087268369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4004030 con 0x563e8f366370 2024-09-19T06:22:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.070+0000 7f78e655d740 1 -- 172.21.15.5:0/1087268369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78cc0bc0c0 con 0x563e8f366370 2024-09-19T06:22:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.070+0000 7f78c57fa640 1 -- 172.21.15.5:0/1087268369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f78d40041d0 con 0x563e8f366370 2024-09-19T06:22:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.070+0000 7f78c57fa640 1 -- 172.21.15.5:0/1087268369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f78d4004410 con 0x563e8f366370 2024-09-19T06:22:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.070+0000 7f78e655d740 10 client.20256 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.070+0000 7f78e655d740 1 -- 172.21.15.5:0/1087268369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563e8f3e0510 con 0x563e8f366370 2024-09-19T06:22:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.070+0000 7f78c57fa640 1 -- 172.21.15.5:0/1087268369 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78cc0bc0c0 con 0x563e8f366370 2024-09-19T06:22:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.071+0000 7f78c57fa640 1 -- 172.21.15.5:0/1087268369 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f78d403f070 con 0x563e8f366370 2024-09-19T06:22:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.071+0000 7f78e655d740 10 client.20256 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.071+0000 7f78e655d740 10 client.20256 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:22:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.071+0000 7f78e655d740 20 client.20256 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.071+0000 7f78e655d740 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x563e8f3e0ac0 0x563e8f3e2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.071+0000 7f78e655d740 4 client.20256 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.071+0000 7f78e655d740 1 -- 172.21.15.5:0/1087268369 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e8f2c8d70 con 0x563e8f3e0ac0 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.071+0000 7f78e1f20640 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x563e8f3e0ac0 0x563e8f3e2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.072+0000 7f78c57fa640 10 client.20256 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.072+0000 7f78e1f20640 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x563e8f3e0ac0 0x563e8f3e2eb0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec2151594e46981 server_cookie=6640ed23481f263f in_seq=0 out_seq=0 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.073+0000 7f78c57fa640 1 -- 172.21.15.5:0/1087268369 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e8f2c8d70 con 0x563e8f3e0ac0 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.073+0000 7f78c57fa640 10 client.20256 handle_command_reply: tid=0 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.073+0000 7f78e655d740 1 client.20256 shutdown 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.073+0000 7f78e655d740 1 -- 172.21.15.5:0/1087268369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f366370 msgr2=0x563e8f36e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.073+0000 7f78e655d740 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f366370 0x563e8f36e1e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f78d4003a70 tx=0x7f78d4009ab0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.074+0000 7f78e655d740 1 -- 172.21.15.5:0/1087268369 shutdown_connections 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.074+0000 7f78e655d740 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x563e8f3e0ac0 0x563e8f3e2eb0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.074+0000 7f78e655d740 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e8f366370 0x563e8f36e1e0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.074+0000 7f78c57fa640 0 client.20256 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:22:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.074+0000 7f78e655d740 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e8f35cb40 0x563e8f36a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.074+0000 7f78e655d740 1 --2- 172.21.15.5:0/1087268369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e8f35b5d0 0x563e8f369b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.074+0000 7f78e655d740 1 -- 172.21.15.5:0/1087268369 >> 172.21.15.5:0/1087268369 conn(0x563e8f324310 msgr2=0x563e8f3650e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.074+0000 7f78e655d740 1 -- 172.21.15.5:0/1087268369 shutdown_connections 2024-09-19T06:22:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.074+0000 7f78e655d740 1 -- 172.21.15.5:0/1087268369 wait complete. 2024-09-19T06:22:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.074+0000 7f78e655d740 20 client.20256 trim_cache size 0 max 16384 2024-09-19T06:22:21.078 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:22:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.075+0000 7f78e4184640 1 -- 172.21.15.5:0/3405902785 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f78c80a3430 msgr2=0x7f78c80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.075+0000 7f78e4184640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f78c80a3430 0x7f78c80a58f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f78d400e720 tx=0x7f78d4020000 comp rx=0 tx=0).stop 2024-09-19T06:22:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.075+0000 7f78e4184640 1 -- 172.21.15.5:0/3405902785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78dc1049a0 msgr2=0x7f78dc19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.075+0000 7f78e4184640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78dc1049a0 0x7f78dc19aa70 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f78cc005940 tx=0x7f78cc0040c0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.075+0000 7f78e4184640 1 -- 172.21.15.5:0/3405902785 shutdown_connections 2024-09-19T06:22:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.075+0000 7f78e4184640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f78c80a3430 0x7f78c80a58f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.075+0000 7f78e4184640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78dc109120 0x7f78dc19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.075+0000 7f78e4184640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78dc1049a0 0x7f78dc19aa70 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.075+0000 7f78e4184640 1 --2- 172.21.15.5:0/3405902785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78dc103ff0 0x7f78dc19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.075+0000 7f78e4184640 1 -- 172.21.15.5:0/3405902785 >> 172.21.15.5:0/3405902785 conn(0x7f78dc0ffab0 msgr2=0x7f78dc100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.075+0000 7f78e4184640 1 -- 172.21.15.5:0/3405902785 shutdown_connections 2024-09-19T06:22:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:21.075+0000 7f78e4184640 1 -- 172.21.15.5:0/3405902785 wait complete. 2024-09-19T06:22:21.566 INFO:tasks.workunit.client.0.smithi005.stdout: 10 56967 8.77 MB/sec execute 225 sec latency 73.902 ms 2024-09-19T06:22:21.858 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:22:22.234 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:22:22.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57159 8.78 MB/sec execute 226 sec latency 77.206 ms 2024-09-19T06:22:22.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:22 smithi005 ceph-mon[30955]: pgmap v600: 97 pgs: 97 active+clean; 938 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-09-19T06:22:22.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.802+0000 7f870f8b3640 1 -- 172.21.15.5:0/1829565966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87081082a0 msgr2=0x7f870810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:22.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.802+0000 7f870f8b3640 1 --2- 172.21.15.5:0/1829565966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87081082a0 0x7f870810ec40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8700009ca0 tx=0x7f870002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.802+0000 7f870f8b3640 1 -- 172.21.15.5:0/1829565966 shutdown_connections 2024-09-19T06:22:22.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.802+0000 7f870f8b3640 1 --2- 172.21.15.5:0/1829565966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87081082a0 0x7f870810ec40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.802+0000 7f870f8b3640 1 --2- 172.21.15.5:0/1829565966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87081036e0 0x7f8708107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.802+0000 7f870f8b3640 1 --2- 172.21.15.5:0/1829565966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8708102d30 0x7f8708103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.802+0000 7f870f8b3640 1 -- 172.21.15.5:0/1829565966 >> 172.21.15.5:0/1829565966 conn(0x7f87080fc7d0 msgr2=0x7f87080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:22.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.803+0000 7f870f8b3640 1 -- 172.21.15.5:0/1829565966 shutdown_connections 2024-09-19T06:22:22.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.803+0000 7f870f8b3640 1 -- 172.21.15.5:0/1829565966 wait complete. 2024-09-19T06:22:22.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.804+0000 7f870f8b3640 1 Processor -- start 2024-09-19T06:22:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.804+0000 7f870f8b3640 1 -- start start 2024-09-19T06:22:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.804+0000 7f870f8b3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8708102d30 0x7f8708198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.804+0000 7f870f8b3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87081036e0 0x7f8708198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.804+0000 7f870f8b3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87081082a0 0x7f870819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.804+0000 7f870f8b3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8708111f80 con 0x7f87081036e0 2024-09-19T06:22:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.804+0000 7f870f8b3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8708111e00 con 0x7f8708102d30 2024-09-19T06:22:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.804+0000 7f870f8b3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8708112100 con 0x7f87081082a0 2024-09-19T06:22:22.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.805+0000 7f870de50640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87081082a0 0x7f870819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.805+0000 7f870de50640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87081082a0 0x7f870819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44744/0 (socket says 172.21.15.5:44744) 2024-09-19T06:22:22.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.805+0000 7f870de50640 1 -- 172.21.15.5:0/1894976890 learned_addr learned my addr 172.21.15.5:0/1894976890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:22.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.805+0000 7f870ce4e640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87081036e0 0x7f8708198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.805+0000 7f870d64f640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8708102d30 0x7f8708198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.805+0000 7f870ce4e640 1 -- 172.21.15.5:0/1894976890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87081082a0 msgr2=0x7f870819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:22.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.805+0000 7f870ce4e640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87081082a0 0x7f870819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.805+0000 7f870ce4e640 1 -- 172.21.15.5:0/1894976890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8708102d30 msgr2=0x7f8708198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.805+0000 7f870ce4e640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8708102d30 0x7f8708198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.805+0000 7f870ce4e640 1 -- 172.21.15.5:0/1894976890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870819d4b0 con 0x7f87081036e0 2024-09-19T06:22:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.806+0000 7f870ce4e640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87081036e0 0x7f8708198a40 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f87080697d0 tx=0x7f86f800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:22.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.806+0000 7f86fe7fc640 1 -- 172.21.15.5:0/1894976890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f800cc10 con 0x7f87081036e0 2024-09-19T06:22:22.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.806+0000 7f870f8b3640 1 -- 172.21.15.5:0/1894976890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870819d7a0 con 0x7f87081036e0 2024-09-19T06:22:22.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.807+0000 7f870f8b3640 1 -- 172.21.15.5:0/1894976890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87081008c0 con 0x7f87081036e0 2024-09-19T06:22:22.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.807+0000 7f86fe7fc640 1 -- 172.21.15.5:0/1894976890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f800cdb0 con 0x7f87081036e0 2024-09-19T06:22:22.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.807+0000 7f86fe7fc640 1 -- 172.21.15.5:0/1894976890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f80186f0 con 0x7f87081036e0 2024-09-19T06:22:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.811+0000 7f8711c8c740 1 Processor -- start 2024-09-19T06:22:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.811+0000 7f8711c8c740 1 -- start start 2024-09-19T06:22:22.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f8711c8c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 0x558f56320090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f8711c8c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f563205d0 0x558f56320a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f8711c8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f5632d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f8711c8c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558f562862f0 con 0x558f563205d0 2024-09-19T06:22:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f8711c8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558f5622dcf0 con 0x558f5632b370 2024-09-19T06:22:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f8711c8c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558f562822b0 con 0x558f56321b40 2024-09-19T06:22:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f870d64f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f563205d0 0x558f56320a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f870de50640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 0x558f56320090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f870de50640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 0x558f56320090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44754/0 (socket says 172.21.15.5:44754) 2024-09-19T06:22:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f870d64f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f563205d0 0x558f56320a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49404/0 (socket says 172.21.15.5:49404) 2024-09-19T06:22:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f870de50640 1 -- 172.21.15.5:0/3465904909 learned_addr learned my addr 172.21.15.5:0/3465904909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:22.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f86fe7fc640 1 -- 172.21.15.5:0/1894976890 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f86f8018890 con 0x7f87081036e0 2024-09-19T06:22:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f870d64f640 1 -- 172.21.15.5:0/3465904909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 msgr2=0x558f56320090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f870d64f640 1 --2- 172.21.15.5:0/3465904909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 0x558f56320090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f870d64f640 1 -- 172.21.15.5:0/3465904909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 msgr2=0x558f5632d760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:22:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f870d64f640 1 --2- 172.21.15.5:0/3465904909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f5632d760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.812+0000 7f870d64f640 1 -- 172.21.15.5:0/3465904909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f562123a0 con 0x558f563205d0 2024-09-19T06:22:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.813+0000 7f870d64f640 1 --2- 172.21.15.5:0/3465904909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f563205d0 0x558f56320a10 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f8704010d70 tx=0x7f8704005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b206f729080992 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.813+0000 7f86f37fe640 1 -- 172.21.15.5:0/3465904909 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704004050 con 0x558f563205d0 2024-09-19T06:22:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.813+0000 7f86f37fe640 1 -- 172.21.15.5:0/3465904909 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87040054d0 con 0x558f563205d0 2024-09-19T06:22:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.813+0000 7f86fe7fc640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f86f40a3430 0x7f86f40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.813+0000 7f86f37fe640 1 -- 172.21.15.5:0/3465904909 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704004b60 con 0x558f563205d0 2024-09-19T06:22:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.814+0000 7f86fe7fc640 1 -- 172.21.15.5:0/1894976890 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f86f80cfe30 con 0x7f87081036e0 2024-09-19T06:22:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.814+0000 7f870de50640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f86f40a3430 0x7f86f40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.814+0000 7f8711c8c740 1 -- 172.21.15.5:0/3465904909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f563205d0 msgr2=0x558f56320a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.814+0000 7f8711c8c740 1 --2- 172.21.15.5:0/3465904909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f563205d0 0x558f56320a10 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f8704010d70 tx=0x7f8704005f90 comp rx=0 tx=0).stop 2024-09-19T06:22:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.815+0000 7f8711c8c740 1 -- 172.21.15.5:0/3465904909 shutdown_connections 2024-09-19T06:22:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.815+0000 7f8711c8c740 1 --2- 172.21.15.5:0/3465904909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f5632d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.815+0000 7f8711c8c740 1 --2- 172.21.15.5:0/3465904909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f563205d0 0x558f56320a10 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.815+0000 7f8711c8c740 1 --2- 172.21.15.5:0/3465904909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 0x558f56320090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.815+0000 7f8711c8c740 1 -- 172.21.15.5:0/3465904909 >> 172.21.15.5:0/3465904909 conn(0x558f562e9310 msgr2=0x558f5631cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.815+0000 7f870de50640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f86f40a3430 0x7f86f40a58f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8700009430 tx=0x7f8700044040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.815+0000 7f8711c8c740 1 -- 172.21.15.5:0/3465904909 shutdown_connections 2024-09-19T06:22:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.815+0000 7f8711c8c740 1 -- 172.21.15.5:0/3465904909 wait complete. 2024-09-19T06:22:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.816+0000 7f8711c8c740 1 -- start start 2024-09-19T06:22:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.816+0000 7f8711c8c740 1 Processor -- start 2024-09-19T06:22:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.816+0000 7f8711c8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f563205d0 0x558f5632ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f8711c8c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f56321b40 0x558f56328000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f8711c8c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f5632b370 0x558f563285d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f8711c8c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558f5632e070 con 0x558f56321b40 2024-09-19T06:22:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f8711c8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558f56241060 con 0x558f563205d0 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f8711c8c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558f5632e1f0 con 0x558f5632b370 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f870d64f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f563205d0 0x558f5632ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f870d64f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f563205d0 0x558f5632ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37468/0 (socket says 172.21.15.5:37468) 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f870d64f640 1 -- 172.21.15.5:0/3804986905 learned_addr learned my addr 172.21.15.5:0/3804986905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f870de50640 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f5632b370 0x558f563285d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f870ce4e640 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f56321b40 0x558f56328000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f870d64f640 1 -- 172.21.15.5:0/3804986905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f5632b370 msgr2=0x558f563285d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f870d64f640 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f5632b370 0x558f563285d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f870d64f640 1 -- 172.21.15.5:0/3804986905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f56321b40 msgr2=0x558f56328000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f870d64f640 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f56321b40 0x558f56328000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f870d64f640 1 -- 172.21.15.5:0/3804986905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f55ff73c0 con 0x558f563205d0 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f8711c8c740 10 client.0 _ll_register_callbacks cb 0x558f55f948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f8711c8c740 1 -- 172.21.15.5:0/3804986905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87040054d0 con 0x558f563205d0 2024-09-19T06:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.817+0000 7f870ce4e640 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f56321b40 0x558f56328000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.818+0000 7f870d64f640 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f563205d0 0x558f5632ad50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8704004610 tx=0x7f8704010660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bda7ff6dca5c737 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.818+0000 7f86f0ff9640 10 client.29630 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.818+0000 7f86f0ff9640 1 -- 172.21.15.5:0/3804986905 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704028020 con 0x558f563205d0 2024-09-19T06:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.818+0000 7f86f0ff9640 1 -- 172.21.15.5:0/3804986905 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f8704003800 con 0x558f563205d0 2024-09-19T06:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.818+0000 7f86f0ff9640 1 -- 172.21.15.5:0/3804986905 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8704003a00 con 0x558f563205d0 2024-09-19T06:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.818+0000 7f8711c8c740 10 client.29630 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.818+0000 7f8711c8c740 1 -- 172.21.15.5:0/3804986905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=58}) -- 0x558f56339a60 con 0x558f563205d0 2024-09-19T06:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.819+0000 7f86f0ff9640 1 -- 172.21.15.5:0/3804986905 <== mon.2 v2:172.21.15.49:3300/0 4 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f8704003ba0 con 0x558f563205d0 2024-09-19T06:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.819+0000 7f8711c8c740 10 client.29630 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.819+0000 7f8711c8c740 10 client.29630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.819+0000 7f8711c8c740 20 client.29630 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.819+0000 7f8711c8c740 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558f5633adb0 0x558f5633d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.819+0000 7f8711c8c740 4 client.29630 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.819+0000 7f8711c8c740 1 -- 172.21.15.5:0/3804986905 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f56293860 con 0x558f5633adb0 2024-09-19T06:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.819+0000 7f870ce4e640 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558f5633adb0 0x558f5633d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f870ce4e640 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558f5633adb0 0x558f5633d1a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=229994db979200bd server_cookie=4e814d77dc0623d0 in_seq=0 out_seq=0 2024-09-19T06:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f86f0ff9640 10 client.29630 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f86f0ff9640 1 -- 172.21.15.5:0/3804986905 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f56293860 con 0x558f5633adb0 2024-09-19T06:22:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f86f0ff9640 10 client.29630 handle_command_reply: tid=0 2024-09-19T06:22:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f8711c8c740 1 client.29630 shutdown 2024-09-19T06:22:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f8711c8c740 1 -- 172.21.15.5:0/3804986905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f563205d0 msgr2=0x558f5632ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f8711c8c740 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f563205d0 0x558f5632ad50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8704004610 tx=0x7f8704010660 comp rx=0 tx=0).stop 2024-09-19T06:22:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f8711c8c740 1 -- 172.21.15.5:0/3804986905 shutdown_connections 2024-09-19T06:22:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f8711c8c740 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558f5633adb0 0x558f5633d1a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f8711c8c740 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f5632b370 0x558f563285d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f8711c8c740 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f56321b40 0x558f56328000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f86f0ff9640 0 client.29630 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f8711c8c740 1 --2- 172.21.15.5:0/3804986905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f563205d0 0x558f5632ad50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.820+0000 7f8711c8c740 1 -- 172.21.15.5:0/3804986905 >> 172.21.15.5:0/3804986905 conn(0x558f562e9310 msgr2=0x558f56322990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.821+0000 7f8711c8c740 1 -- 172.21.15.5:0/3804986905 shutdown_connections 2024-09-19T06:22:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.821+0000 7f8711c8c740 1 -- 172.21.15.5:0/3804986905 wait complete. 2024-09-19T06:22:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.821+0000 7f8711c8c740 20 client.29630 trim_cache size 0 max 16384 2024-09-19T06:22:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.843+0000 7f8711c8c740 1 Processor -- start 2024-09-19T06:22:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f8711c8c740 1 -- start start 2024-09-19T06:22:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f8711c8c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f563205d0 0x558f563373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f8711c8c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f56321b40 0x558f56326fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f8711c8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f5632c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f8711c8c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec00e880 con 0x558f56321b40 2024-09-19T06:22:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f8711c8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec0103c0 con 0x558f5632b370 2024-09-19T06:22:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f8711c8c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec00ef50 con 0x558f563205d0 2024-09-19T06:22:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f870de50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f5632c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f870ce4e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f56321b40 0x558f56326fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f870de50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f5632c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37474/0 (socket says 172.21.15.5:37474) 2024-09-19T06:22:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f870de50640 1 -- 172.21.15.5:0/4270805415 learned_addr learned my addr 172.21.15.5:0/4270805415 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f870d64f640 1 --2- 172.21.15.5:0/4270805415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f563205d0 0x558f563373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f870de50640 1 -- 172.21.15.5:0/4270805415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f563205d0 msgr2=0x558f563373a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f870de50640 1 --2- 172.21.15.5:0/4270805415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f563205d0 0x558f563373a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f870de50640 1 -- 172.21.15.5:0/4270805415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f56321b40 msgr2=0x558f56326fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.844+0000 7f870de50640 1 --2- 172.21.15.5:0/4270805415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f56321b40 0x558f56326fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.845+0000 7f870de50640 1 -- 172.21.15.5:0/4270805415 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f5633d510 con 0x558f5632b370 2024-09-19T06:22:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.845+0000 7f870de50640 1 --2- 172.21.15.5:0/4270805415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f5632c760 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8700031870 tx=0x7f870004c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a9e7ca08b96e752 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.845+0000 7f86f2ffd640 1 -- 172.21.15.5:0/4270805415 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700004040 con 0x558f5632b370 2024-09-19T06:22:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.845+0000 7f86f2ffd640 1 -- 172.21.15.5:0/4270805415 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8700036040 con 0x558f5632b370 2024-09-19T06:22:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.845+0000 7f86f2ffd640 1 -- 172.21.15.5:0/4270805415 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700030c70 con 0x558f5632b370 2024-09-19T06:22:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.846+0000 7f8711c8c740 1 -- 172.21.15.5:0/4270805415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 msgr2=0x558f5632c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.846+0000 7f8711c8c740 1 --2- 172.21.15.5:0/4270805415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f5632c760 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8700031870 tx=0x7f870004c000 comp rx=0 tx=0).stop 2024-09-19T06:22:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.846+0000 7f8711c8c740 1 -- 172.21.15.5:0/4270805415 shutdown_connections 2024-09-19T06:22:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.846+0000 7f8711c8c740 1 --2- 172.21.15.5:0/4270805415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f5632c760 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.846+0000 7f8711c8c740 1 --2- 172.21.15.5:0/4270805415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f56321b40 0x558f56326fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.846+0000 7f8711c8c740 1 --2- 172.21.15.5:0/4270805415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f563205d0 0x558f563373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.846+0000 7f8711c8c740 1 -- 172.21.15.5:0/4270805415 >> 172.21.15.5:0/4270805415 conn(0x558f562e9310 msgr2=0x558f563268d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.846+0000 7f8711c8c740 1 -- 172.21.15.5:0/4270805415 shutdown_connections 2024-09-19T06:22:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.846+0000 7f8711c8c740 1 -- 172.21.15.5:0/4270805415 wait complete. 2024-09-19T06:22:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.847+0000 7f8711c8c740 1 -- start start 2024-09-19T06:22:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.847+0000 7f8711c8c740 1 Processor -- start 2024-09-19T06:22:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f8711c8c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f563205d0 0x558f56342480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f8711c8c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 0x558f563429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f8711c8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f56342f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f8711c8c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec00fcf0 con 0x558f563205d0 2024-09-19T06:22:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f8711c8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec00f620 con 0x558f5632b370 2024-09-19T06:22:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f8711c8c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec00c880 con 0x558f56321b40 2024-09-19T06:22:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f870ce4e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 0x558f563429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f870ce4e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 0x558f563429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44778/0 (socket says 172.21.15.5:44778) 2024-09-19T06:22:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f8711c8c740 10 client.0 _ll_register_callbacks cb 0x7f8704005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f870ce4e640 1 -- 172.21.15.5:0/2790199219 learned_addr learned my addr 172.21.15.5:0/2790199219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f870d64f640 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f563205d0 0x558f56342480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f870de50640 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f56342f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f870ce4e640 1 -- 172.21.15.5:0/2790199219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 msgr2=0x558f56342f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f870ce4e640 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f56342f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f870ce4e640 1 -- 172.21.15.5:0/2790199219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f563205d0 msgr2=0x558f56342480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f870ce4e640 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f563205d0 0x558f56342480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f870ce4e640 1 -- 172.21.15.5:0/2790199219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f56331bd0 con 0x558f56321b40 2024-09-19T06:22:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.848+0000 7f870de50640 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f56342f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.849+0000 7f870ce4e640 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 0x558f563429c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86f8098600 tx=0x7f86f8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2b2ba97dd29208d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.849+0000 7f86f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:22:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.849+0000 7f86f0ff9640 1 -- 172.21.15.5:0/2790199219 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f80b4020 con 0x558f56321b40 2024-09-19T06:22:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.849+0000 7f8711c8c740 1 -- 172.21.15.5:0/2790199219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8700036040 con 0x558f56321b40 2024-09-19T06:22:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.849+0000 7f86f0ff9640 1 -- 172.21.15.5:0/2790199219 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f86f80cd7e0 con 0x558f56321b40 2024-09-19T06:22:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.849+0000 7f86f0ff9640 1 -- 172.21.15.5:0/2790199219 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f86f807f050 con 0x558f56321b40 2024-09-19T06:22:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.849+0000 7f8711c8c740 10 client.29581 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.849+0000 7f8711c8c740 1 -- 172.21.15.5:0/2790199219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x558f563a5af0 con 0x558f56321b40 2024-09-19T06:22:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.850+0000 7f86f0ff9640 1 -- 172.21.15.5:0/2790199219 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8700036040 con 0x558f56321b40 2024-09-19T06:22:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.850+0000 7f86f0ff9640 1 -- 172.21.15.5:0/2790199219 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f86f80b6070 con 0x558f56321b40 2024-09-19T06:22:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.850+0000 7f8711c8c740 10 client.29581 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.850+0000 7f8711c8c740 10 client.29581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.850+0000 7f8711c8c740 20 client.29581 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.850+0000 7f8711c8c740 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558f563a6220 0x558f563a8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.850+0000 7f8711c8c740 4 client.29581 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:22:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.850+0000 7f8711c8c740 1 -- 172.21.15.5:0/2790199219 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f5628dd70 con 0x558f563a6220 2024-09-19T06:22:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.850+0000 7f870d64f640 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558f563a6220 0x558f563a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:22.857 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:22 smithi157 ceph-mon[35346]: pgmap v600: 97 pgs: 97 active+clean; 938 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-09-19T06:22:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f870d64f640 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558f563a6220 0x558f563a8640 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c56e85d9a0745860 server_cookie=89304560db77094e in_seq=0 out_seq=0 2024-09-19T06:22:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f86f0ff9640 10 client.29581 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f86f0ff9640 1 -- 172.21.15.5:0/2790199219 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f5628dd70 con 0x558f563a6220 2024-09-19T06:22:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f86f0ff9640 10 client.29581 handle_command_reply: tid=0 2024-09-19T06:22:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f8711c8c740 1 client.29581 shutdown 2024-09-19T06:22:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f8711c8c740 1 -- 172.21.15.5:0/2790199219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 msgr2=0x558f563429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f8711c8c740 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 0x558f563429c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86f8098600 tx=0x7f86f8000bc0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f8711c8c740 1 -- 172.21.15.5:0/2790199219 shutdown_connections 2024-09-19T06:22:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f8711c8c740 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558f563a6220 0x558f563a8640 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f8711c8c740 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f5632b370 0x558f56342f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f8711c8c740 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f56321b40 0x558f563429c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f8711c8c740 1 --2- 172.21.15.5:0/2790199219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f563205d0 0x558f56342480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.851+0000 7f8711c8c740 1 -- 172.21.15.5:0/2790199219 >> 172.21.15.5:0/2790199219 conn(0x558f562e9310 msgr2=0x558f563270f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.852+0000 7f86f0ff9640 0 client.29581 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.852+0000 7f8711c8c740 1 -- 172.21.15.5:0/2790199219 shutdown_connections 2024-09-19T06:22:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.852+0000 7f8711c8c740 1 -- 172.21.15.5:0/2790199219 wait complete. 2024-09-19T06:22:22.860 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:22:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.852+0000 7f8711c8c740 20 client.29581 trim_cache size 0 max 16384 2024-09-19T06:22:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.853+0000 7f870f8b3640 1 -- 172.21.15.5:0/1894976890 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f86f40a3430 msgr2=0x7f86f40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.853+0000 7f870f8b3640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f86f40a3430 0x7f86f40a58f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8700009430 tx=0x7f8700044040 comp rx=0 tx=0).stop 2024-09-19T06:22:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.853+0000 7f870f8b3640 1 -- 172.21.15.5:0/1894976890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87081036e0 msgr2=0x7f8708198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.853+0000 7f870f8b3640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87081036e0 0x7f8708198a40 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f87080697d0 tx=0x7f86f800ef10 comp rx=0 tx=0).stop 2024-09-19T06:22:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.853+0000 7f870f8b3640 1 -- 172.21.15.5:0/1894976890 shutdown_connections 2024-09-19T06:22:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.853+0000 7f870f8b3640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f86f40a3430 0x7f86f40a58f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.853+0000 7f870f8b3640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87081082a0 0x7f870819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.853+0000 7f870f8b3640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87081036e0 0x7f8708198a40 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.853+0000 7f870f8b3640 1 --2- 172.21.15.5:0/1894976890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8708102d30 0x7f8708198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.853+0000 7f870f8b3640 1 -- 172.21.15.5:0/1894976890 >> 172.21.15.5:0/1894976890 conn(0x7f87080fc7d0 msgr2=0x7f8708105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.853+0000 7f870f8b3640 1 -- 172.21.15.5:0/1894976890 shutdown_connections 2024-09-19T06:22:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:22.853+0000 7f870f8b3640 1 -- 172.21.15.5:0/1894976890 wait complete. 2024-09-19T06:22:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:22 smithi049 ceph-mon[34640]: pgmap v600: 97 pgs: 97 active+clean; 938 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-09-19T06:22:23.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57273 8.76 MB/sec execute 227 sec latency 86.148 ms 2024-09-19T06:22:24.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57422 8.77 MB/sec execute 228 sec latency 149.473 ms 2024-09-19T06:22:24.784 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:22:24.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:22:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:24 smithi005 ceph-mon[30955]: pgmap v601: 97 pgs: 97 active+clean; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 376 op/s 2024-09-19T06:22:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:24 smithi049 ceph-mon[34640]: pgmap v601: 97 pgs: 97 active+clean; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 376 op/s 2024-09-19T06:22:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:24 smithi157 ceph-mon[35346]: pgmap v601: 97 pgs: 97 active+clean; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 376 op/s 2024-09-19T06:22:25.157 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:22:25.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57594 8.76 MB/sec execute 229 sec latency 172.633 ms 2024-09-19T06:22:25.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.687+0000 7f4c2bac7640 1 -- 172.21.15.5:0/3106420696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c241082a0 msgr2=0x7f4c2410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:25.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.687+0000 7f4c2bac7640 1 --2- 172.21.15.5:0/3106420696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c241082a0 0x7f4c2410ec40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009be0 tx=0x7f4c1c02a6f0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.687+0000 7f4c2bac7640 1 -- 172.21.15.5:0/3106420696 shutdown_connections 2024-09-19T06:22:25.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.687+0000 7f4c2bac7640 1 --2- 172.21.15.5:0/3106420696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c241082a0 0x7f4c2410ec40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.687+0000 7f4c2bac7640 1 --2- 172.21.15.5:0/3106420696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c241036e0 0x7f4c24107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.687+0000 7f4c2bac7640 1 --2- 172.21.15.5:0/3106420696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c24102d30 0x7f4c24103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.687+0000 7f4c2bac7640 1 -- 172.21.15.5:0/3106420696 >> 172.21.15.5:0/3106420696 conn(0x7f4c240fc7d0 msgr2=0x7f4c240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:25.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.687+0000 7f4c2bac7640 1 -- 172.21.15.5:0/3106420696 shutdown_connections 2024-09-19T06:22:25.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.687+0000 7f4c2bac7640 1 -- 172.21.15.5:0/3106420696 wait complete. 2024-09-19T06:22:25.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.690+0000 7f4c2bac7640 1 Processor -- start 2024-09-19T06:22:25.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.690+0000 7f4c2bac7640 1 -- start start 2024-09-19T06:22:25.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.690+0000 7f4c2bac7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c24102d30 0x7f4c2406c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.690+0000 7f4c2bac7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c241036e0 0x7f4c2406c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.691+0000 7f4c2bac7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c241082a0 0x7f4c2406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.691+0000 7f4c29863640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c24102d30 0x7f4c2406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.691+0000 7f4c29863640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c24102d30 0x7f4c2406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:58996/0 (socket says 172.21.15.5:58996) 2024-09-19T06:22:25.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.691+0000 7f4c29062640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c241036e0 0x7f4c2406c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.691+0000 7f4c29062640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c241036e0 0x7f4c2406c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:32832/0 (socket says 172.21.15.5:32832) 2024-09-19T06:22:25.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.691+0000 7f4c29863640 1 -- 172.21.15.5:0/2687156007 learned_addr learned my addr 172.21.15.5:0/2687156007 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:25.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.691+0000 7f4c2bac7640 1 -- 172.21.15.5:0/2687156007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24101bc0 con 0x7f4c241082a0 2024-09-19T06:22:25.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.691+0000 7f4c2bac7640 1 -- 172.21.15.5:0/2687156007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24101a40 con 0x7f4c241036e0 2024-09-19T06:22:25.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.691+0000 7f4c2bac7640 1 -- 172.21.15.5:0/2687156007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24101d40 con 0x7f4c24102d30 2024-09-19T06:22:25.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.691+0000 7f4c2a064640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c241082a0 0x7f4c2406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.692+0000 7f4c29062640 1 -- 172.21.15.5:0/2687156007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c24102d30 msgr2=0x7f4c2406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:25.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.692+0000 7f4c29062640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c24102d30 0x7f4c2406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.692+0000 7f4c29062640 1 -- 172.21.15.5:0/2687156007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c241082a0 msgr2=0x7f4c2406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:25.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.692+0000 7f4c29062640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c241082a0 0x7f4c2406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.692+0000 7f4c29062640 1 -- 172.21.15.5:0/2687156007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c241aef90 con 0x7f4c241036e0 2024-09-19T06:22:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.692+0000 7f4c2a064640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c241082a0 0x7f4c2406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.692+0000 7f4c29062640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c241036e0 0x7f4c2406c6b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4c240697d0 tx=0x7f4c1400db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.692+0000 7f4c1affd640 1 -- 172.21.15.5:0/2687156007 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14017070 con 0x7f4c241036e0 2024-09-19T06:22:25.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.692+0000 7f4c2bac7640 1 -- 172.21.15.5:0/2687156007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c241af280 con 0x7f4c241036e0 2024-09-19T06:22:25.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.692+0000 7f4c1affd640 1 -- 172.21.15.5:0/2687156007 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c14009820 con 0x7f4c241036e0 2024-09-19T06:22:25.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.693+0000 7f4c2bac7640 1 -- 172.21.15.5:0/2687156007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c241af750 con 0x7f4c241036e0 2024-09-19T06:22:25.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.693+0000 7f4c1affd640 1 -- 172.21.15.5:0/2687156007 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14004b20 con 0x7f4c241036e0 2024-09-19T06:22:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.697+0000 7f4c1affd640 1 -- 172.21.15.5:0/2687156007 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4c14004cc0 con 0x7f4c241036e0 2024-09-19T06:22:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.697+0000 7f4c2dea0740 1 Processor -- start 2024-09-19T06:22:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.697+0000 7f4c2dea0740 1 -- start start 2024-09-19T06:22:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.697+0000 7f4c2dea0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb88eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.697+0000 7f4c2dea0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb88fa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.697+0000 7f4c2dea0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb88fd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.697+0000 7f4c2dea0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eeb8855280 con 0x55eeb88efb60 2024-09-19T06:22:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.697+0000 7f4c2dea0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eeb87fcb10 con 0x55eeb88fad40 2024-09-19T06:22:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.697+0000 7f4c2dea0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eeb8851240 con 0x55eeb88f0480 2024-09-19T06:22:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.699+0000 7f4c1affd640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4c100a3480 0x7f4c100a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.699+0000 7f4c29062640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb88fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.699+0000 7f4c29062640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb88fd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:32842/0 (socket says 172.21.15.5:32842) 2024-09-19T06:22:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.699+0000 7f4c1affd640 1 -- 172.21.15.5:0/2687156007 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f4c140cf1b0 con 0x7f4c241036e0 2024-09-19T06:22:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.699+0000 7f4c29062640 1 -- 172.21.15.5:0/477393637 learned_addr learned my addr 172.21.15.5:0/477393637 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.700+0000 7f4c29863640 1 --2- 172.21.15.5:0/477393637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb88fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.700+0000 7f4c2a064640 1 --2- 172.21.15.5:0/477393637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb88eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.701+0000 7f4c2a064640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4c100a3480 0x7f4c100a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.701+0000 7f4c29062640 1 -- 172.21.15.5:0/477393637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 msgr2=0x55eeb88fa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.701+0000 7f4c29062640 1 --2- 172.21.15.5:0/477393637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb88fa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.701+0000 7f4c29062640 1 -- 172.21.15.5:0/477393637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 msgr2=0x55eeb88eff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.701+0000 7f4c29062640 1 --2- 172.21.15.5:0/477393637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb88eff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.701+0000 7f4c29062640 1 -- 172.21.15.5:0/477393637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeb88091f0 con 0x55eeb88fad40 2024-09-19T06:22:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.701+0000 7f4c29062640 1 --2- 172.21.15.5:0/477393637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb88fd130 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009f70 tx=0x7f4c14090950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9073742355d01c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.702+0000 7f4c0ffff640 1 -- 172.21.15.5:0/477393637 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14016170 con 0x55eeb88fad40 2024-09-19T06:22:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.702+0000 7f4c0ffff640 1 -- 172.21.15.5:0/477393637 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c14009cd0 con 0x55eeb88fad40 2024-09-19T06:22:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.702+0000 7f4c0ffff640 1 -- 172.21.15.5:0/477393637 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c140ccb00 con 0x55eeb88fad40 2024-09-19T06:22:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.703+0000 7f4c2dea0740 1 -- 172.21.15.5:0/477393637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 msgr2=0x55eeb88fd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.703+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/477393637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb88fd130 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009f70 tx=0x7f4c14090950 comp rx=0 tx=0).stop 2024-09-19T06:22:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.703+0000 7f4c2dea0740 1 -- 172.21.15.5:0/477393637 shutdown_connections 2024-09-19T06:22:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.703+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/477393637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb88fd130 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.703+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/477393637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb88fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.703+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/477393637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb88eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.703+0000 7f4c2dea0740 1 -- 172.21.15.5:0/477393637 >> 172.21.15.5:0/477393637 conn(0x55eeb88ec0f0 msgr2=0x55eeb88ec4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.703+0000 7f4c2a064640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4c100a3480 0x7f4c100a5940 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c031040 tx=0x7f4c1c02a6f0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.704+0000 7f4c2dea0740 1 -- 172.21.15.5:0/477393637 shutdown_connections 2024-09-19T06:22:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.704+0000 7f4c2dea0740 1 -- 172.21.15.5:0/477393637 wait complete. 2024-09-19T06:22:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.704+0000 7f4c2dea0740 1 -- start start 2024-09-19T06:22:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.704+0000 7f4c2dea0740 1 Processor -- start 2024-09-19T06:22:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.705+0000 7f4c2dea0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb88f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.705+0000 7f4c2dea0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb88f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.705+0000 7f4c2dea0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb88f8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.705+0000 7f4c2dea0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eeb88fd910 con 0x55eeb88efb60 2024-09-19T06:22:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.705+0000 7f4c2dea0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eeb880ff10 con 0x55eeb88fad40 2024-09-19T06:22:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.705+0000 7f4c2dea0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eeb88fda90 con 0x55eeb88f0480 2024-09-19T06:22:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.705+0000 7f4c2dea0740 10 client.0 _ll_register_callbacks cb 0x55eeb87d8a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.705+0000 7f4c29062640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb88f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.705+0000 7f4c29062640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb88f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59008/0 (socket says 172.21.15.5:59008) 2024-09-19T06:22:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.705+0000 7f4c29062640 1 -- 172.21.15.5:0/833294731 learned_addr learned my addr 172.21.15.5:0/833294731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.705+0000 7f4c2a064640 1 --2- 172.21.15.5:0/833294731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb88f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.706+0000 7f4c29062640 1 -- 172.21.15.5:0/833294731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 msgr2=0x55eeb88f8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.706+0000 7f4c29062640 1 --2- 172.21.15.5:0/833294731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb88f8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.706+0000 7f4c29062640 1 -- 172.21.15.5:0/833294731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 msgr2=0x55eeb88f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.706+0000 7f4c29062640 1 --2- 172.21.15.5:0/833294731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb88f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.706+0000 7f4c29062640 1 -- 172.21.15.5:0/833294731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeb85c66a0 con 0x55eeb88f0480 2024-09-19T06:22:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.706+0000 7f4c29062640 1 --2- 172.21.15.5:0/833294731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb88f81f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4c1409b450 tx=0x7f4c1409f7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a6f2302430db60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.706+0000 7f4c0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:22:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.706+0000 7f4c0d7fa640 1 -- 172.21.15.5:0/833294731 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c140b3020 con 0x55eeb88f0480 2024-09-19T06:22:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.706+0000 7f4c0d7fa640 1 -- 172.21.15.5:0/833294731 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f4c140ccb00 con 0x55eeb88f0480 2024-09-19T06:22:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.706+0000 7f4c2dea0740 1 -- 172.21.15.5:0/833294731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eeb88627f0 con 0x55eeb88f0480 2024-09-19T06:22:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.707+0000 7f4c0d7fa640 1 -- 172.21.15.5:0/833294731 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4c14020040 con 0x55eeb88f0480 2024-09-19T06:22:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.707+0000 7f4c2dea0740 10 client.29593 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.707+0000 7f4c2dea0740 1 -- 172.21.15.5:0/833294731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55eeb88fd910 con 0x55eeb88f0480 2024-09-19T06:22:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.707+0000 7f4c0d7fa640 1 -- 172.21.15.5:0/833294731 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55eeb88627f0 con 0x55eeb88f0480 2024-09-19T06:22:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.707+0000 7f4c0d7fa640 1 -- 172.21.15.5:0/833294731 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f4c14023060 con 0x55eeb88f0480 2024-09-19T06:22:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.707+0000 7f4c2dea0740 10 client.29593 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.707+0000 7f4c2dea0740 10 client.29593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.707+0000 7f4c2dea0740 20 client.29593 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.708+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/833294731 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb8912f30 0x55eeb8915320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.708+0000 7f4c2dea0740 4 client.29593 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.708+0000 7f4c2dea0740 1 -- 172.21.15.5:0/833294731 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eeb885cd00 con 0x55eeb8912f30 2024-09-19T06:22:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.708+0000 7f4c29863640 1 --2- 172.21.15.5:0/833294731 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb8912f30 0x55eeb8915320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.708+0000 7f4c29863640 1 --2- 172.21.15.5:0/833294731 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb8912f30 0x55eeb8915320 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc417308e8cba9e5 server_cookie=eb0ae87a0621647f in_seq=0 out_seq=0 2024-09-19T06:22:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.708+0000 7f4c0d7fa640 10 client.29593 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.709+0000 7f4c0d7fa640 1 -- 172.21.15.5:0/833294731 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eeb885cd00 con 0x55eeb8912f30 2024-09-19T06:22:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.709+0000 7f4c0d7fa640 10 client.29593 handle_command_reply: tid=0 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.709+0000 7f4c2dea0740 1 client.29593 shutdown 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.710+0000 7f4c2dea0740 1 -- 172.21.15.5:0/833294731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 msgr2=0x55eeb88f81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.710+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/833294731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb88f81f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4c1409b450 tx=0x7f4c1409f7e0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.710+0000 7f4c2dea0740 1 -- 172.21.15.5:0/833294731 shutdown_connections 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.710+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/833294731 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb8912f30 0x55eeb8915320 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.710+0000 7f4c0d7fa640 0 client.29593 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.710+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/833294731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb88f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.710+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/833294731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb88f81f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.710+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/833294731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb88f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.710+0000 7f4c2dea0740 1 -- 172.21.15.5:0/833294731 >> 172.21.15.5:0/833294731 conn(0x55eeb88ec0f0 msgr2=0x55eeb88f1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.710+0000 7f4c2dea0740 1 -- 172.21.15.5:0/833294731 shutdown_connections 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.710+0000 7f4c2dea0740 1 -- 172.21.15.5:0/833294731 wait complete. 2024-09-19T06:22:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.710+0000 7f4c2dea0740 20 client.29593 trim_cache size 0 max 16384 2024-09-19T06:22:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c2dea0740 1 Processor -- start 2024-09-19T06:22:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c2dea0740 1 -- start start 2024-09-19T06:22:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c2dea0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb891aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c2dea0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88f0480 0x55eeb891af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c2dea0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88fad40 0x55eeb88f9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c2dea0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241b3f40 con 0x55eeb88efb60 2024-09-19T06:22:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c2dea0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241b5a80 con 0x55eeb88f0480 2024-09-19T06:22:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c2dea0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241b4610 con 0x55eeb88fad40 2024-09-19T06:22:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c29863640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb891aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c29863640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb891aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59336/0 (socket says 172.21.15.5:59336) 2024-09-19T06:22:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c29863640 1 -- 172.21.15.5:0/1885081239 learned_addr learned my addr 172.21.15.5:0/1885081239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c29062640 1 --2- 172.21.15.5:0/1885081239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88f0480 0x55eeb891af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.738+0000 7f4c2a064640 1 --2- 172.21.15.5:0/1885081239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88fad40 0x55eeb88f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.739+0000 7f4c29062640 1 -- 172.21.15.5:0/1885081239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88fad40 msgr2=0x55eeb88f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.739+0000 7f4c29062640 1 --2- 172.21.15.5:0/1885081239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88fad40 0x55eeb88f9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.739+0000 7f4c29062640 1 -- 172.21.15.5:0/1885081239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 msgr2=0x55eeb891aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.739+0000 7f4c29062640 1 --2- 172.21.15.5:0/1885081239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb891aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.739+0000 7f4c29062640 1 -- 172.21.15.5:0/1885081239 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeb88fa580 con 0x55eeb88f0480 2024-09-19T06:22:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.739+0000 7f4c29863640 1 --2- 172.21.15.5:0/1885081239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb891aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.739+0000 7f4c29062640 1 --2- 172.21.15.5:0/1885081239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88f0480 0x55eeb891af50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4c140ccfd0 tx=0x7f4c14005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebbc5c9687d78af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.739+0000 7f4c0f7fe640 1 -- 172.21.15.5:0/1885081239 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c140b3070 con 0x55eeb88f0480 2024-09-19T06:22:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.739+0000 7f4c0f7fe640 1 -- 172.21.15.5:0/1885081239 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c140a4080 con 0x55eeb88f0480 2024-09-19T06:22:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.739+0000 7f4c0f7fe640 1 -- 172.21.15.5:0/1885081239 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1409a380 con 0x55eeb88f0480 2024-09-19T06:22:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.740+0000 7f4c2dea0740 1 -- 172.21.15.5:0/1885081239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88f0480 msgr2=0x55eeb891af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.740+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/1885081239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88f0480 0x55eeb891af50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4c140ccfd0 tx=0x7f4c14005850 comp rx=0 tx=0).stop 2024-09-19T06:22:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.740+0000 7f4c2dea0740 1 -- 172.21.15.5:0/1885081239 shutdown_connections 2024-09-19T06:22:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.740+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/1885081239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88fad40 0x55eeb88f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.740+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/1885081239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88f0480 0x55eeb891af50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.741+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/1885081239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb891aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.741+0000 7f4c2dea0740 1 -- 172.21.15.5:0/1885081239 >> 172.21.15.5:0/1885081239 conn(0x55eeb88ec0f0 msgr2=0x55eeb88f6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.741+0000 7f4c2dea0740 1 -- 172.21.15.5:0/1885081239 shutdown_connections 2024-09-19T06:22:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.741+0000 7f4c2dea0740 1 -- 172.21.15.5:0/1885081239 wait complete. 2024-09-19T06:22:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.741+0000 7f4c2dea0740 1 -- start start 2024-09-19T06:22:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.742+0000 7f4c2dea0740 1 Processor -- start 2024-09-19T06:22:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.743+0000 7f4c2dea0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb890bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.743+0000 7f4c2dea0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb8906fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.743+0000 7f4c29863640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb890bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.743+0000 7f4c29863640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb890bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59338/0 (socket says 172.21.15.5:59338) 2024-09-19T06:22:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.743+0000 7f4c29863640 1 -- 172.21.15.5:0/3059682099 learned_addr learned my addr 172.21.15.5:0/3059682099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.743+0000 7f4c29062640 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb8906fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.743+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb8907510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.744+0000 7f4c2dea0740 1 -- 172.21.15.5:0/3059682099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241b53b0 con 0x55eeb88efb60 2024-09-19T06:22:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.744+0000 7f4c2a064640 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb8907510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.744+0000 7f4c2dea0740 1 -- 172.21.15.5:0/3059682099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241b4ce0 con 0x55eeb88fad40 2024-09-19T06:22:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.744+0000 7f4c2dea0740 1 -- 172.21.15.5:0/3059682099 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241b1f60 con 0x55eeb88f0480 2024-09-19T06:22:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.744+0000 7f4c2dea0740 10 client.0 _ll_register_callbacks cb 0x7f4c20005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.744+0000 7f4c29062640 1 -- 172.21.15.5:0/3059682099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 msgr2=0x55eeb8907510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.744+0000 7f4c29062640 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb8907510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.744+0000 7f4c29062640 1 -- 172.21.15.5:0/3059682099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 msgr2=0x55eeb890bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.744+0000 7f4c29062640 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb890bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.744+0000 7f4c29062640 1 -- 172.21.15.5:0/3059682099 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeb8907d40 con 0x55eeb88f0480 2024-09-19T06:22:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.744+0000 7f4c29863640 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb890bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.745+0000 7f4c29062640 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb8906fd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55eeb88fd1b0 tx=0x7f4c14000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b05e3c3b3613d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.745+0000 7f4c0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:22:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.745+0000 7f4c0d7fa640 1 -- 172.21.15.5:0/3059682099 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c140b3070 con 0x55eeb88f0480 2024-09-19T06:22:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.745+0000 7f4c0d7fa640 1 -- 172.21.15.5:0/3059682099 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f4c1409a380 con 0x55eeb88f0480 2024-09-19T06:22:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.745+0000 7f4c2dea0740 1 -- 172.21.15.5:0/3059682099 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c140a4080 con 0x55eeb88f0480 2024-09-19T06:22:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.745+0000 7f4c0d7fa640 1 -- 172.21.15.5:0/3059682099 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4c1409a5c0 con 0x55eeb88f0480 2024-09-19T06:22:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.745+0000 7f4c2dea0740 10 client.29599 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.745+0000 7f4c2dea0740 1 -- 172.21.15.5:0/3059682099 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55eeb8907d40 con 0x55eeb88f0480 2024-09-19T06:22:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.746+0000 7f4c0d7fa640 1 -- 172.21.15.5:0/3059682099 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c140bb0b0 con 0x55eeb88f0480 2024-09-19T06:22:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.746+0000 7f4c0d7fa640 1 -- 172.21.15.5:0/3059682099 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f4c14004020 con 0x55eeb88f0480 2024-09-19T06:22:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.746+0000 7f4c2dea0740 10 client.29599 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.746+0000 7f4c2dea0740 10 client.29599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.746+0000 7f4c2dea0740 20 client.29599 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.746+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb897dca0 0x55eeb8980090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.746+0000 7f4c2dea0740 4 client.29599 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:22:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.746+0000 7f4c2dea0740 1 -- 172.21.15.5:0/3059682099 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f4c140a4080 con 0x55eeb897dca0 2024-09-19T06:22:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.746+0000 7f4c29863640 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb897dca0 0x55eeb8980090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.746+0000 7f4c0d7fa640 10 client.29599 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.747+0000 7f4c29863640 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb897dca0 0x55eeb8980090 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c792272b7a73a4a server_cookie=c532b36b727b3a57 in_seq=0 out_seq=0 2024-09-19T06:22:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c0d7fa640 1 -- 172.21.15.5:0/3059682099 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f4c140a4080 con 0x55eeb897dca0 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c0d7fa640 10 client.29599 handle_command_reply: tid=0 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c2dea0740 1 client.29599 shutdown 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c2dea0740 1 -- 172.21.15.5:0/3059682099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 msgr2=0x55eeb8906fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb8906fd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55eeb88fd1b0 tx=0x7f4c14000bc0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c2dea0740 1 -- 172.21.15.5:0/3059682099 shutdown_connections 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55eeb897dca0 0x55eeb8980090 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eeb88fad40 0x55eeb8907510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb88f0480 0x55eeb8906fd0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c0d7fa640 0 client.29599 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c2dea0740 1 --2- 172.21.15.5:0/3059682099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eeb88efb60 0x55eeb890bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c2dea0740 1 -- 172.21.15.5:0/3059682099 >> 172.21.15.5:0/3059682099 conn(0x55eeb88ec0f0 msgr2=0x55eeb88f15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c2dea0740 1 -- 172.21.15.5:0/3059682099 shutdown_connections 2024-09-19T06:22:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.748+0000 7f4c2dea0740 1 -- 172.21.15.5:0/3059682099 wait complete. 2024-09-19T06:22:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.749+0000 7f4c2dea0740 20 client.29599 trim_cache size 0 max 16384 2024-09-19T06:22:25.754 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:22:25.754 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:22:25.754 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "d802d51e-d3d3-4db0-8f88-f5323564ddc7", 2024-09-19T06:22:25.754 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:22:25.754 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:22:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.750+0000 7f4c2bac7640 1 -- 172.21.15.5:0/2687156007 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4c100a3480 msgr2=0x7f4c100a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.750+0000 7f4c2bac7640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4c100a3480 0x7f4c100a5940 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c031040 tx=0x7f4c1c02a6f0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.750+0000 7f4c2bac7640 1 -- 172.21.15.5:0/2687156007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c241036e0 msgr2=0x7f4c2406c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.750+0000 7f4c2bac7640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c241036e0 0x7f4c2406c6b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4c240697d0 tx=0x7f4c1400db30 comp rx=0 tx=0).stop 2024-09-19T06:22:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.750+0000 7f4c2bac7640 1 -- 172.21.15.5:0/2687156007 shutdown_connections 2024-09-19T06:22:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.750+0000 7f4c2bac7640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c241082a0 0x7f4c2406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.750+0000 7f4c2bac7640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c241036e0 0x7f4c2406c6b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.750+0000 7f4c2bac7640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4c100a3480 0x7f4c100a5940 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.750+0000 7f4c2bac7640 1 --2- 172.21.15.5:0/2687156007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c24102d30 0x7f4c2406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.750+0000 7f4c2bac7640 1 -- 172.21.15.5:0/2687156007 >> 172.21.15.5:0/2687156007 conn(0x7f4c240fc7d0 msgr2=0x7f4c24107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.750+0000 7f4c2bac7640 1 -- 172.21.15.5:0/2687156007 shutdown_connections 2024-09-19T06:22:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:25.750+0000 7f4c2bac7640 1 -- 172.21.15.5:0/2687156007 wait complete. 2024-09-19T06:22:26.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57782 8.77 MB/sec execute 230 sec latency 74.733 ms 2024-09-19T06:22:26.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:22:26.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: pgmap v602: 97 pgs: 97 active+clean; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 260 op/s 2024-09-19T06:22:26.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:22:26.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:26.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:22:26.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:22:26.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:22:26.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:26.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:26.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:22:26.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:22:26.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:26.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:26.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:26 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: pgmap v602: 97 pgs: 97 active+clean; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 260 op/s 2024-09-19T06:22:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:22:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:22:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:22:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:22:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:22:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:22:26.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:26.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:26.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:26 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.949+0000 7fac880c0640 1 -- 172.21.15.5:0/1162299751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac80101340 msgr2=0x7fac8010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.949+0000 7fac880c0640 1 --2- 172.21.15.5:0/1162299751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac80101340 0x7fac8010d870 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fac78009c10 tx=0x7fac7802a6e0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.949+0000 7fac880c0640 1 -- 172.21.15.5:0/1162299751 shutdown_connections 2024-09-19T06:22:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.949+0000 7fac880c0640 1 --2- 172.21.15.5:0/1162299751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac80101340 0x7fac8010d870 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.949+0000 7fac880c0640 1 --2- 172.21.15.5:0/1162299751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac801009a0 0x7fac80100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.949+0000 7fac880c0640 1 --2- 172.21.15.5:0/1162299751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac801069b0 0x7fac80106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.949+0000 7fac880c0640 1 -- 172.21.15.5:0/1162299751 >> 172.21.15.5:0/1162299751 conn(0x7fac800779a0 msgr2=0x7fac800fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.949+0000 7fac880c0640 1 -- 172.21.15.5:0/1162299751 shutdown_connections 2024-09-19T06:22:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.949+0000 7fac880c0640 1 -- 172.21.15.5:0/1162299751 wait complete. 2024-09-19T06:22:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.950+0000 7fac880c0640 1 Processor -- start 2024-09-19T06:22:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.950+0000 7fac880c0640 1 -- start start 2024-09-19T06:22:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.950+0000 7fac880c0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac801009a0 0x7fac80198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.950+0000 7fac880c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac80101340 0x7fac801988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.950+0000 7fac880c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac801069b0 0x7fac8019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.950+0000 7fac85e5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac801009a0 0x7fac80198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.950+0000 7fac85e5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac801009a0 0x7fac80198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:32898/0 (socket says 172.21.15.5:32898) 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.950+0000 7fac85e5c640 1 -- 172.21.15.5:0/4136871380 learned_addr learned my addr 172.21.15.5:0/4136871380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.950+0000 7fac880c0640 1 -- 172.21.15.5:0/4136871380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac80110ca0 con 0x7fac80101340 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.950+0000 7fac880c0640 1 -- 172.21.15.5:0/4136871380 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac80110b20 con 0x7fac801009a0 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.950+0000 7fac880c0640 1 -- 172.21.15.5:0/4136871380 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac80110e20 con 0x7fac801069b0 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.950+0000 7fac8665d640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac801069b0 0x7fac8019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.951+0000 7fac85e5c640 1 -- 172.21.15.5:0/4136871380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac801069b0 msgr2=0x7fac8019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.951+0000 7fac85e5c640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac801069b0 0x7fac8019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.951+0000 7fac85e5c640 1 -- 172.21.15.5:0/4136871380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac80101340 msgr2=0x7fac801988b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:22:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.951+0000 7fac8565b640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac80101340 0x7fac801988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.951+0000 7fac85e5c640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac80101340 0x7fac801988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.951+0000 7fac85e5c640 1 -- 172.21.15.5:0/4136871380 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8019d360 con 0x7fac801009a0 2024-09-19T06:22:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.951+0000 7fac8565b640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac80101340 0x7fac801988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:22:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.951+0000 7fac85e5c640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac801009a0 0x7fac80198370 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00e9c0 tx=0x7fac7c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.951+0000 7fac76ffd640 1 -- 172.21.15.5:0/4136871380 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c00cde0 con 0x7fac801009a0 2024-09-19T06:22:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.952+0000 7fac880c0640 1 -- 172.21.15.5:0/4136871380 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8019d650 con 0x7fac801009a0 2024-09-19T06:22:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.952+0000 7fac76ffd640 1 -- 172.21.15.5:0/4136871380 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac7c0035f0 con 0x7fac801009a0 2024-09-19T06:22:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.952+0000 7fac76ffd640 1 -- 172.21.15.5:0/4136871380 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c010690 con 0x7fac801009a0 2024-09-19T06:22:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.952+0000 7fac880c0640 1 -- 172.21.15.5:0/4136871380 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac801a4ee0 con 0x7fac801009a0 2024-09-19T06:22:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.955+0000 7fac76ffd640 1 -- 172.21.15.5:0/4136871380 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fac7c0037a0 con 0x7fac801009a0 2024-09-19T06:22:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.956+0000 7fac8ae54740 1 Processor -- start 2024-09-19T06:22:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.956+0000 7fac76ffd640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fac6c0a33b0 0x7fac6c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.956+0000 7fac8ae54740 1 -- start start 2024-09-19T06:22:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.956+0000 7fac8665d640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fac6c0a33b0 0x7fac6c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.956+0000 7fac76ffd640 1 -- 172.21.15.5:0/4136871380 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7fac7c0d0180 con 0x7fac801009a0 2024-09-19T06:22:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac8ae54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e490af0 0x56347e490ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac8ae54740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e49b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac8ae54740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e49bcb0 0x56347e49e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac8ae54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56347e3f52f0 con 0x56347e490af0 2024-09-19T06:22:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac8ae54740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56347e39d270 con 0x56347e491410 2024-09-19T06:22:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac8ae54740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56347e3f12b0 con 0x56347e49bcb0 2024-09-19T06:22:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac85e5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e490af0 0x56347e490ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac8565b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e49b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac85e5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e490af0 0x56347e490ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59354/0 (socket says 172.21.15.5:59354) 2024-09-19T06:22:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac85e5c640 1 -- 172.21.15.5:0/2724482258 learned_addr learned my addr 172.21.15.5:0/2724482258 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac8665d640 1 --2- 172.21.15.5:0/2724482258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e49bcb0 0x56347e49e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac8665d640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fac6c0a33b0 0x7fac6c0a5870 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fac7802ac00 tx=0x7fac78030530 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac85e5c640 1 -- 172.21.15.5:0/2724482258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e49bcb0 msgr2=0x56347e49e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac85e5c640 1 --2- 172.21.15.5:0/2724482258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e49bcb0 0x56347e49e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac85e5c640 1 -- 172.21.15.5:0/2724482258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 msgr2=0x56347e49b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac85e5c640 1 --2- 172.21.15.5:0/2724482258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e49b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac85e5c640 1 -- 172.21.15.5:0/2724482258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347e39c340 con 0x56347e490af0 2024-09-19T06:22:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.957+0000 7fac8565b640 1 --2- 172.21.15.5:0/2724482258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e49b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.958+0000 7fac85e5c640 1 --2- 172.21.15.5:0/2724482258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e490af0 0x56347e490ed0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009230 tx=0x7fac7c003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bdbd7b2ecf6bd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.958+0000 7fac6bfff640 1 -- 172.21.15.5:0/2724482258 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0b4020 con 0x56347e490af0 2024-09-19T06:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.958+0000 7fac6bfff640 1 -- 172.21.15.5:0/2724482258 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac7c09cae0 con 0x56347e490af0 2024-09-19T06:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.958+0000 7fac6bfff640 1 -- 172.21.15.5:0/2724482258 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c098a50 con 0x56347e490af0 2024-09-19T06:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.959+0000 7fac8ae54740 1 -- 172.21.15.5:0/2724482258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e490af0 msgr2=0x56347e490ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.959+0000 7fac8ae54740 1 --2- 172.21.15.5:0/2724482258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e490af0 0x56347e490ed0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009230 tx=0x7fac7c003b10 comp rx=0 tx=0).stop 2024-09-19T06:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.959+0000 7fac8ae54740 1 -- 172.21.15.5:0/2724482258 shutdown_connections 2024-09-19T06:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.959+0000 7fac8ae54740 1 --2- 172.21.15.5:0/2724482258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e49bcb0 0x56347e49e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.959+0000 7fac8ae54740 1 --2- 172.21.15.5:0/2724482258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e49b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.959+0000 7fac8ae54740 1 --2- 172.21.15.5:0/2724482258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e490af0 0x56347e490ed0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.959+0000 7fac8ae54740 1 -- 172.21.15.5:0/2724482258 >> 172.21.15.5:0/2724482258 conn(0x56347e48d060 msgr2=0x56347e48d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.959+0000 7fac8ae54740 1 -- 172.21.15.5:0/2724482258 shutdown_connections 2024-09-19T06:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.959+0000 7fac8ae54740 1 -- 172.21.15.5:0/2724482258 wait complete. 2024-09-19T06:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.959+0000 7fac8ae54740 1 -- start start 2024-09-19T06:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.960+0000 7fac8ae54740 1 Processor -- start 2024-09-19T06:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.960+0000 7fac8ae54740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e498bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.960+0000 7fac8ae54740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e499100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.960+0000 7fac8ae54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e499640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.960+0000 7fac8ae54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56347e49e880 con 0x56347e49bcb0 2024-09-19T06:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.960+0000 7fac8ae54740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56347e37e290 con 0x56347e491410 2024-09-19T06:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.960+0000 7fac8ae54740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56347e49ea00 con 0x56347e490af0 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.960+0000 7fac8665d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e499640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.960+0000 7fac85e5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e498bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac8665d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e499640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59360/0 (socket says 172.21.15.5:59360) 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac8665d640 1 -- 172.21.15.5:0/2226214292 learned_addr learned my addr 172.21.15.5:0/2226214292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac8ae54740 10 client.0 _ll_register_callbacks cb 0x56347e3af140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac8565b640 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e499100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac85e5c640 1 -- 172.21.15.5:0/2226214292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 msgr2=0x56347e499100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac85e5c640 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e499100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac85e5c640 1 -- 172.21.15.5:0/2226214292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 msgr2=0x56347e499640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac85e5c640 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e499640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac85e5c640 1 -- 172.21.15.5:0/2226214292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347e108210 con 0x56347e490af0 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac8665d640 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e499640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac85e5c640 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e498bc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00c8e0 tx=0x7fac7c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f32bbe802d245c91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac697fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.961+0000 7fac697fa640 1 -- 172.21.15.5:0/2226214292 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0b9070 con 0x56347e490af0 2024-09-19T06:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.962+0000 7fac697fa640 1 -- 172.21.15.5:0/2226214292 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7fac7c0a4bc0 con 0x56347e490af0 2024-09-19T06:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.962+0000 7fac8ae54740 1 -- 172.21.15.5:0/2226214292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac7c09cae0 con 0x56347e490af0 2024-09-19T06:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.962+0000 7fac697fa640 1 -- 172.21.15.5:0/2226214292 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fac7c0b4070 con 0x56347e490af0 2024-09-19T06:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.962+0000 7fac8ae54740 10 client.29611 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.962+0000 7fac8ae54740 1 -- 172.21.15.5:0/2226214292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56347e1d4620 con 0x56347e490af0 2024-09-19T06:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.962+0000 7fac697fa640 1 -- 172.21.15.5:0/2226214292 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac7c09cae0 con 0x56347e490af0 2024-09-19T06:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.962+0000 7fac697fa640 1 -- 172.21.15.5:0/2226214292 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fac7c0981c0 con 0x56347e490af0 2024-09-19T06:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.962+0000 7fac8ae54740 10 client.29611 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.962+0000 7fac8ae54740 10 client.29611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.962+0000 7fac8ae54740 20 client.29611 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.962+0000 7fac8ae54740 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56347e4abd90 0x56347e4ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.963+0000 7fac8ae54740 4 client.29611 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.963+0000 7fac8ae54740 1 -- 172.21.15.5:0/2226214292 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56347e402a20 con 0x56347e4abd90 2024-09-19T06:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.963+0000 7fac8565b640 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56347e4abd90 0x56347e4ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.963+0000 7fac8565b640 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56347e4abd90 0x56347e4ae210 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62ec994baad7a00a server_cookie=2f62fb8d5957985 in_seq=0 out_seq=0 2024-09-19T06:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.963+0000 7fac697fa640 10 client.29611 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac697fa640 1 -- 172.21.15.5:0/2226214292 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56347e402a20 con 0x56347e4abd90 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac697fa640 10 client.29611 handle_command_reply: tid=0 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac8ae54740 1 client.29611 shutdown 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac8ae54740 1 -- 172.21.15.5:0/2226214292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 msgr2=0x56347e498bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac8ae54740 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e498bc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00c8e0 tx=0x7fac7c0046c0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac8ae54740 1 -- 172.21.15.5:0/2226214292 shutdown_connections 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac8ae54740 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56347e4abd90 0x56347e4ae210 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac8ae54740 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e499640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac8ae54740 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e499100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac697fa640 0 client.29611 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac8ae54740 1 --2- 172.21.15.5:0/2226214292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e498bc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac8ae54740 1 -- 172.21.15.5:0/2226214292 >> 172.21.15.5:0/2226214292 conn(0x56347e48d060 msgr2=0x56347e492dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.964+0000 7fac8ae54740 1 -- 172.21.15.5:0/2226214292 shutdown_connections 2024-09-19T06:22:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.965+0000 7fac8ae54740 1 -- 172.21.15.5:0/2226214292 wait complete. 2024-09-19T06:22:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.965+0000 7fac8ae54740 20 client.29611 trim_cache size 0 max 16384 2024-09-19T06:22:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.985+0000 7fac8ae54740 1 Processor -- start 2024-09-19T06:22:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.985+0000 7fac8ae54740 1 -- start start 2024-09-19T06:22:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.985+0000 7fac8ae54740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e493560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.985+0000 7fac8ae54740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e499470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.985+0000 7fac8ae54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e4b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.985+0000 7fac8ae54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac801ab6b0 con 0x56347e49bcb0 2024-09-19T06:22:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.985+0000 7fac8ae54740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac801ad1f0 con 0x56347e491410 2024-09-19T06:22:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.985+0000 7fac8ae54740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac801abd80 con 0x56347e490af0 2024-09-19T06:22:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.985+0000 7fac85e5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e493560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.985+0000 7fac85e5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e493560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59080/0 (socket says 172.21.15.5:59080) 2024-09-19T06:22:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.985+0000 7fac85e5c640 1 -- 172.21.15.5:0/198879597 learned_addr learned my addr 172.21.15.5:0/198879597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.986+0000 7fac8565b640 1 --2- 172.21.15.5:0/198879597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e499470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.986+0000 7fac8665d640 1 --2- 172.21.15.5:0/198879597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e4b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.986+0000 7fac85e5c640 1 -- 172.21.15.5:0/198879597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 msgr2=0x56347e499470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.986+0000 7fac85e5c640 1 --2- 172.21.15.5:0/198879597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e499470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.986+0000 7fac85e5c640 1 -- 172.21.15.5:0/198879597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 msgr2=0x56347e4b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.986+0000 7fac85e5c640 1 --2- 172.21.15.5:0/198879597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e4b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.986+0000 7fac85e5c640 1 -- 172.21.15.5:0/198879597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347e490920 con 0x56347e490af0 2024-09-19T06:22:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.986+0000 7fac85e5c640 1 --2- 172.21.15.5:0/198879597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e493560 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fac7c07c040 tx=0x7fac7c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0ca1ea2a81d58b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.986+0000 7fac6b7fe640 1 -- 172.21.15.5:0/198879597 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0b9030 con 0x56347e490af0 2024-09-19T06:22:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.987+0000 7fac6b7fe640 1 -- 172.21.15.5:0/198879597 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac7c09cae0 con 0x56347e490af0 2024-09-19T06:22:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.987+0000 7fac6b7fe640 1 -- 172.21.15.5:0/198879597 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0a0790 con 0x56347e490af0 2024-09-19T06:22:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.987+0000 7fac8ae54740 1 -- 172.21.15.5:0/198879597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 msgr2=0x56347e493560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.987+0000 7fac8ae54740 1 --2- 172.21.15.5:0/198879597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e493560 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fac7c07c040 tx=0x7fac7c004730 comp rx=0 tx=0).stop 2024-09-19T06:22:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.988+0000 7fac8ae54740 1 -- 172.21.15.5:0/198879597 shutdown_connections 2024-09-19T06:22:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.988+0000 7fac8ae54740 1 --2- 172.21.15.5:0/198879597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e4b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.988+0000 7fac8ae54740 1 --2- 172.21.15.5:0/198879597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e499470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.988+0000 7fac8ae54740 1 --2- 172.21.15.5:0/198879597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e493560 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.988+0000 7fac8ae54740 1 -- 172.21.15.5:0/198879597 >> 172.21.15.5:0/198879597 conn(0x56347e48d060 msgr2=0x56347e49ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.988+0000 7fac8ae54740 1 -- 172.21.15.5:0/198879597 shutdown_connections 2024-09-19T06:22:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.988+0000 7fac8ae54740 1 -- 172.21.15.5:0/198879597 wait complete. 2024-09-19T06:22:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.988+0000 7fac8ae54740 1 -- start start 2024-09-19T06:22:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.988+0000 7fac8ae54740 1 Processor -- start 2024-09-19T06:22:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8ae54740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e4a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8ae54740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e4a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8ae54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e4951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8ae54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac801acb20 con 0x56347e49bcb0 2024-09-19T06:22:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8ae54740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac801ac450 con 0x56347e491410 2024-09-19T06:22:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8ae54740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac801a96b0 con 0x56347e490af0 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8ae54740 10 client.0 _ll_register_callbacks cb 0x7fac700127c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8665d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e4951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8565b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e4a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac85e5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e4a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8565b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e4a41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:32938/0 (socket says 172.21.15.5:32938) 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8565b640 1 -- 172.21.15.5:0/935382529 learned_addr learned my addr 172.21.15.5:0/935382529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8565b640 1 -- 172.21.15.5:0/935382529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 msgr2=0x56347e4a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8565b640 1 --2- 172.21.15.5:0/935382529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e4a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8565b640 1 -- 172.21.15.5:0/935382529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 msgr2=0x56347e4951c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8565b640 1 --2- 172.21.15.5:0/935382529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e4951c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8565b640 1 -- 172.21.15.5:0/935382529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347e495700 con 0x56347e491410 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.989+0000 7fac8665d640 1 --2- 172.21.15.5:0/935382529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e4951c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.990+0000 7fac8565b640 1 --2- 172.21.15.5:0/935382529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e4a41a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fac70013eb0 tx=0x7fac70013b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=504e5a19cec2b27d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.990+0000 7fac697fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:22:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.990+0000 7fac697fa640 1 -- 172.21.15.5:0/935382529 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70005040 con 0x56347e491410 2024-09-19T06:22:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.990+0000 7fac697fa640 1 -- 172.21.15.5:0/935382529 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7fac700051e0 con 0x56347e491410 2024-09-19T06:22:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.990+0000 7fac8ae54740 1 -- 172.21.15.5:0/935382529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac7c09cae0 con 0x56347e491410 2024-09-19T06:22:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.990+0000 7fac697fa640 1 -- 172.21.15.5:0/935382529 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fac700053e0 con 0x56347e491410 2024-09-19T06:22:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.990+0000 7fac8ae54740 10 client.29672 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.990+0000 7fac8ae54740 1 -- 172.21.15.5:0/935382529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56347e51ead0 con 0x56347e491410 2024-09-19T06:22:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.991+0000 7fac697fa640 1 -- 172.21.15.5:0/935382529 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac7c09cae0 con 0x56347e491410 2024-09-19T06:22:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.991+0000 7fac697fa640 1 -- 172.21.15.5:0/935382529 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fac70009530 con 0x56347e491410 2024-09-19T06:22:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.991+0000 7fac8ae54740 10 client.29672 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.991+0000 7fac8ae54740 10 client.29672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.991+0000 7fac8ae54740 20 client.29672 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.991+0000 7fac8ae54740 1 --2- 172.21.15.5:0/935382529 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56347e4abd90 0x56347e521290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.991+0000 7fac8ae54740 4 client.29672 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:22:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.991+0000 7fac8ae54740 1 -- 172.21.15.5:0/935382529 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56347e3fcd70 con 0x56347e4abd90 2024-09-19T06:22:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.991+0000 7fac85e5c640 1 --2- 172.21.15.5:0/935382529 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56347e4abd90 0x56347e521290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.992+0000 7fac697fa640 10 client.29672 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:26.992+0000 7fac85e5c640 1 --2- 172.21.15.5:0/935382529 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56347e4abd90 0x56347e521290 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=575492ef85a467e8 server_cookie=2a3aa392b6a7ee76 in_seq=0 out_seq=0 2024-09-19T06:22:27.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: pgmap v602: 97 pgs: 97 active+clean; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 260 op/s 2024-09-19T06:22:27.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:22:27.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:27.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:22:27.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:22:27.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:22:27.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:27.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:27.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:22:27.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:22:27.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:27.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:27.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:26 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:22:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac697fa640 1 -- 172.21.15.5:0/935382529 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56347e3fcd70 con 0x56347e4abd90 2024-09-19T06:22:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac697fa640 10 client.29672 handle_command_reply: tid=0 2024-09-19T06:22:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac8ae54740 1 client.29672 shutdown 2024-09-19T06:22:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac8ae54740 1 -- 172.21.15.5:0/935382529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 msgr2=0x56347e4a41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac8ae54740 1 --2- 172.21.15.5:0/935382529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e4a41a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fac70013eb0 tx=0x7fac70013b90 comp rx=0 tx=0).stop 2024-09-19T06:22:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac8ae54740 1 -- 172.21.15.5:0/935382529 shutdown_connections 2024-09-19T06:22:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac8ae54740 1 --2- 172.21.15.5:0/935382529 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56347e4abd90 0x56347e521290 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac8ae54740 1 --2- 172.21.15.5:0/935382529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347e49bcb0 0x56347e4951c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac8ae54740 1 --2- 172.21.15.5:0/935382529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56347e491410 0x56347e4a41a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac8ae54740 1 --2- 172.21.15.5:0/935382529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56347e490af0 0x56347e4a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac8ae54740 1 -- 172.21.15.5:0/935382529 >> 172.21.15.5:0/935382529 conn(0x56347e48d060 msgr2=0x56347e4a6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac697fa640 0 client.29672 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac8ae54740 1 -- 172.21.15.5:0/935382529 shutdown_connections 2024-09-19T06:22:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac8ae54740 1 -- 172.21.15.5:0/935382529 wait complete. 2024-09-19T06:22:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.105+0000 7fac8ae54740 20 client.29672 trim_cache size 0 max 16384 2024-09-19T06:22:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:22:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:22:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:22:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:22:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.106+0000 7fac880c0640 1 -- 172.21.15.5:0/4136871380 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fac6c0a33b0 msgr2=0x7fac6c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.106+0000 7fac880c0640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fac6c0a33b0 0x7fac6c0a5870 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fac7802ac00 tx=0x7fac78030530 comp rx=0 tx=0).stop 2024-09-19T06:22:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.106+0000 7fac880c0640 1 -- 172.21.15.5:0/4136871380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac801009a0 msgr2=0x7fac80198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.106+0000 7fac880c0640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac801009a0 0x7fac80198370 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00e9c0 tx=0x7fac7c00ee80 comp rx=0 tx=0).stop 2024-09-19T06:22:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.106+0000 7fac880c0640 1 -- 172.21.15.5:0/4136871380 shutdown_connections 2024-09-19T06:22:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.106+0000 7fac880c0640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fac6c0a33b0 0x7fac6c0a5870 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.106+0000 7fac880c0640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac801069b0 0x7fac8019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.106+0000 7fac880c0640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac80101340 0x7fac801988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.106+0000 7fac880c0640 1 --2- 172.21.15.5:0/4136871380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac801009a0 0x7fac80198370 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.106+0000 7fac880c0640 1 -- 172.21.15.5:0/4136871380 >> 172.21.15.5:0/4136871380 conn(0x7fac800779a0 msgr2=0x7fac80069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.107+0000 7fac880c0640 1 -- 172.21.15.5:0/4136871380 shutdown_connections 2024-09-19T06:22:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:27.107+0000 7fac880c0640 1 -- 172.21.15.5:0/4136871380 wait complete. 2024-09-19T06:22:27.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57891 8.75 MB/sec execute 231 sec latency 93.995 ms 2024-09-19T06:22:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.649+0000 7f0507ec2640 1 -- 172.21.15.5:0/3968677495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f050010c2f0 msgr2=0x7f050010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.649+0000 7f0507ec2640 1 --2- 172.21.15.5:0/3968677495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f050010c2f0 0x7f050010e790 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009be0 tx=0x7f04f802a6c0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.650+0000 7f0507ec2640 1 -- 172.21.15.5:0/3968677495 shutdown_connections 2024-09-19T06:22:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.650+0000 7f0507ec2640 1 --2- 172.21.15.5:0/3968677495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f050010c2f0 0x7f050010e790 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.650+0000 7f0507ec2640 1 --2- 172.21.15.5:0/3968677495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05001037a0 0x7f050010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.650+0000 7f0507ec2640 1 --2- 172.21.15.5:0/3968677495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0500102df0 0x7f05001031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.650+0000 7f0507ec2640 1 -- 172.21.15.5:0/3968677495 >> 172.21.15.5:0/3968677495 conn(0x7f05000fc7d0 msgr2=0x7f05000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.650+0000 7f0507ec2640 1 -- 172.21.15.5:0/3968677495 shutdown_connections 2024-09-19T06:22:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.650+0000 7f0507ec2640 1 -- 172.21.15.5:0/3968677495 wait complete. 2024-09-19T06:22:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.650+0000 7f0507ec2640 1 Processor -- start 2024-09-19T06:22:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f0507ec2640 1 -- start start 2024-09-19T06:22:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f0507ec2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0500102df0 0x7f05001a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f0507ec2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05001037a0 0x7f05001a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f0507ec2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050010c2f0 0x7f050010adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f0507ec2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0500111fa0 con 0x7f050010c2f0 2024-09-19T06:22:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f0507ec2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0500111e20 con 0x7f05001037a0 2024-09-19T06:22:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f0507ec2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0500112120 con 0x7f0500102df0 2024-09-19T06:22:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f0505c5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0500102df0 0x7f05001a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f0505c5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0500102df0 0x7f05001a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59112/0 (socket says 172.21.15.5:59112) 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f0505c5e640 1 -- 172.21.15.5:0/446606201 learned_addr learned my addr 172.21.15.5:0/446606201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f050545d640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05001037a0 0x7f05001a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f050645f640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050010c2f0 0x7f050010adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f0505c5e640 1 -- 172.21.15.5:0/446606201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05001037a0 msgr2=0x7f05001a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.651+0000 7f0505c5e640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05001037a0 0x7f05001a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.652+0000 7f0505c5e640 1 -- 172.21.15.5:0/446606201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050010c2f0 msgr2=0x7f050010adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.652+0000 7f0505c5e640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050010c2f0 0x7f050010adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.652+0000 7f0505c5e640 1 -- 172.21.15.5:0/446606201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050010b4f0 con 0x7f0500102df0 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.652+0000 7f050645f640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050010c2f0 0x7f050010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.652+0000 7f0505c5e640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0500102df0 0x7f05001a4740 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00e9c0 tx=0x7f04fc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.652+0000 7f04f6ffd640 1 -- 172.21.15.5:0/446606201 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc00cde0 con 0x7f0500102df0 2024-09-19T06:22:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.652+0000 7f04f6ffd640 1 -- 172.21.15.5:0/446606201 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04fc0035f0 con 0x7f0500102df0 2024-09-19T06:22:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.652+0000 7f0507ec2640 1 -- 172.21.15.5:0/446606201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050010b7e0 con 0x7f0500102df0 2024-09-19T06:22:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.652+0000 7f04f6ffd640 1 -- 172.21.15.5:0/446606201 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc010690 con 0x7f0500102df0 2024-09-19T06:22:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.652+0000 7f0507ec2640 1 -- 172.21.15.5:0/446606201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05001ab610 con 0x7f0500102df0 2024-09-19T06:22:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.655+0000 7f050a29b740 1 Processor -- start 2024-09-19T06:22:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.656+0000 7f050a29b740 1 -- start start 2024-09-19T06:22:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.656+0000 7f04f6ffd640 1 -- 172.21.15.5:0/446606201 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f04fc0037a0 con 0x7f0500102df0 2024-09-19T06:22:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.656+0000 7f050a29b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 0x55991a58e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.656+0000 7f050a29b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58edc0 0x55991a599140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.656+0000 7f050a29b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a599680 0x55991a59ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.656+0000 7f050a29b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55991a4f42e0 con 0x55991a58e4a0 2024-09-19T06:22:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.656+0000 7f050a29b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55991a49bce0 con 0x55991a58edc0 2024-09-19T06:22:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.656+0000 7f050a29b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55991a4f02a0 con 0x55991a599680 2024-09-19T06:22:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.657+0000 7f050645f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 0x55991a58e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.657+0000 7f0505c5e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58edc0 0x55991a599140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.657+0000 7f050645f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 0x55991a58e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59406/0 (socket says 172.21.15.5:59406) 2024-09-19T06:22:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.657+0000 7f050645f640 1 -- 172.21.15.5:0/503549156 learned_addr learned my addr 172.21.15.5:0/503549156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.657+0000 7f050545d640 1 --2- 172.21.15.5:0/503549156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a599680 0x55991a59ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f050545d640 1 -- 172.21.15.5:0/503549156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58edc0 msgr2=0x55991a599140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f050545d640 1 --2- 172.21.15.5:0/503549156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58edc0 0x55991a599140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f050545d640 1 -- 172.21.15.5:0/503549156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 msgr2=0x55991a58e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f050545d640 1 --2- 172.21.15.5:0/503549156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 0x55991a58e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f050545d640 1 -- 172.21.15.5:0/503549156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55991a480440 con 0x55991a599680 2024-09-19T06:22:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f04f6ffd640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f04ec0a3430 0x7f04ec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f050645f640 1 --2- 172.21.15.5:0/503549156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 0x55991a58e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f0505c5e640 1 --2- 172.21.15.5:0/503549156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58edc0 0x55991a599140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:22:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f04f6ffd640 1 -- 172.21.15.5:0/446606201 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f04fc0d00a0 con 0x7f0500102df0 2024-09-19T06:22:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f050545d640 1 --2- 172.21.15.5:0/503549156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a599680 0x55991a59ba70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f05001a00f0 tx=0x7f04f000af50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51ce7dab50007096 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f050645f640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f04ec0a3430 0x7f04ec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f04ebfff640 1 -- 172.21.15.5:0/503549156 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0004020 con 0x55991a599680 2024-09-19T06:22:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f04ebfff640 1 -- 172.21.15.5:0/503549156 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04f00041c0 con 0x55991a599680 2024-09-19T06:22:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.658+0000 7f04ebfff640 1 -- 172.21.15.5:0/503549156 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f00186a0 con 0x55991a599680 2024-09-19T06:22:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.659+0000 7f050a29b740 1 -- 172.21.15.5:0/503549156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a599680 msgr2=0x55991a59ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.659+0000 7f050645f640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f04ec0a3430 0x7f04ec0a58f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009370 tx=0x7f04f8005e50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.659+0000 7f050a29b740 1 --2- 172.21.15.5:0/503549156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a599680 0x55991a59ba70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f05001a00f0 tx=0x7f04f000af50 comp rx=0 tx=0).stop 2024-09-19T06:22:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.659+0000 7f050a29b740 1 -- 172.21.15.5:0/503549156 shutdown_connections 2024-09-19T06:22:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.659+0000 7f050a29b740 1 --2- 172.21.15.5:0/503549156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a599680 0x55991a59ba70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.659+0000 7f050a29b740 1 --2- 172.21.15.5:0/503549156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58edc0 0x55991a599140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.659+0000 7f050a29b740 1 --2- 172.21.15.5:0/503549156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 0x55991a58e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.659+0000 7f050a29b740 1 -- 172.21.15.5:0/503549156 >> 172.21.15.5:0/503549156 conn(0x55991a58aa30 msgr2=0x55991a58ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.660+0000 7f050a29b740 1 -- 172.21.15.5:0/503549156 shutdown_connections 2024-09-19T06:22:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.660+0000 7f050a29b740 1 -- 172.21.15.5:0/503549156 wait complete. 2024-09-19T06:22:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.660+0000 7f050a29b740 1 -- start start 2024-09-19T06:22:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.660+0000 7f050a29b740 1 Processor -- start 2024-09-19T06:22:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.661+0000 7f050a29b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 0x55991a598f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.661+0000 7f050a29b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58edc0 0x55991a596240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.661+0000 7f050a29b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a599680 0x55991a596780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.661+0000 7f050a29b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55991a59c250 con 0x55991a58e4a0 2024-09-19T06:22:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.661+0000 7f050a29b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55991a4af050 con 0x55991a58edc0 2024-09-19T06:22:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.661+0000 7f050a29b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55991a59c3d0 con 0x55991a599680 2024-09-19T06:22:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.661+0000 7f050a29b740 10 client.0 _ll_register_callbacks cb 0x55991a4ef160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.661+0000 7f0505c5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 0x55991a598f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.661+0000 7f050545d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58edc0 0x55991a596240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.661+0000 7f0505c5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 0x55991a598f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59408/0 (socket says 172.21.15.5:59408) 2024-09-19T06:22:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.661+0000 7f0505c5e640 1 -- 172.21.15.5:0/2037443380 learned_addr learned my addr 172.21.15.5:0/2037443380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.661+0000 7f050645f640 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a599680 0x55991a596780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.662+0000 7f050545d640 1 -- 172.21.15.5:0/2037443380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a599680 msgr2=0x55991a596780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.662+0000 7f050545d640 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a599680 0x55991a596780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.662+0000 7f050545d640 1 -- 172.21.15.5:0/2037443380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 msgr2=0x55991a598f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.662+0000 7f050545d640 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 0x55991a598f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.662+0000 7f050545d640 1 -- 172.21.15.5:0/2037443380 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55991a2653c0 con 0x55991a58edc0 2024-09-19T06:22:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.662+0000 7f050645f640 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a599680 0x55991a596780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:22:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.662+0000 7f050545d640 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58edc0 0x55991a596240 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f04f000a4d0 tx=0x7f04f0005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a0869e2d49f2791 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.662+0000 7f04e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:22:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.662+0000 7f04e97fa640 1 -- 172.21.15.5:0/2037443380 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0027020 con 0x55991a58edc0 2024-09-19T06:22:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.662+0000 7f04e97fa640 1 -- 172.21.15.5:0/2037443380 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f04f000fd60 con 0x55991a58edc0 2024-09-19T06:22:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.663+0000 7f050a29b740 1 -- 172.21.15.5:0/2037443380 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04f00041c0 con 0x55991a58edc0 2024-09-19T06:22:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.663+0000 7f04e97fa640 1 -- 172.21.15.5:0/2037443380 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f04f00035f0 con 0x55991a58edc0 2024-09-19T06:22:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.663+0000 7f050a29b740 10 client.29681 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.663+0000 7f050a29b740 1 -- 172.21.15.5:0/2037443380 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55991a5a7b70 con 0x55991a58edc0 2024-09-19T06:22:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.663+0000 7f04e97fa640 1 -- 172.21.15.5:0/2037443380 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04f00041c0 con 0x55991a58edc0 2024-09-19T06:22:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.663+0000 7f04e97fa640 1 -- 172.21.15.5:0/2037443380 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f04f0004b70 con 0x55991a58edc0 2024-09-19T06:22:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.663+0000 7f050a29b740 10 client.29681 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.663+0000 7f050a29b740 10 client.29681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.663+0000 7f050a29b740 20 client.29681 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.663+0000 7f050a29b740 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55991a5a8fd0 0x55991a5ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.663+0000 7f050a29b740 4 client.29681 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.663+0000 7f050a29b740 1 -- 172.21.15.5:0/2037443380 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55991a501850 con 0x55991a5a8fd0 2024-09-19T06:22:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.664+0000 7f0505c5e640 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55991a5a8fd0 0x55991a5ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.664+0000 7f0505c5e640 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55991a5a8fd0 0x55991a5ab3c0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d9f9488794ee21 server_cookie=ff4d591e60fc439 in_seq=0 out_seq=0 2024-09-19T06:22:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.664+0000 7f04e97fa640 10 client.29681 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:27.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.665+0000 7f04e97fa640 1 -- 172.21.15.5:0/2037443380 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55991a501850 con 0x55991a5a8fd0 2024-09-19T06:22:27.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.665+0000 7f04e97fa640 10 client.29681 handle_command_reply: tid=0 2024-09-19T06:22:27.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.665+0000 7f050a29b740 1 client.29681 shutdown 2024-09-19T06:22:27.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.665+0000 7f050a29b740 1 -- 172.21.15.5:0/2037443380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58edc0 msgr2=0x55991a596240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:27.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.665+0000 7f050a29b740 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58edc0 0x55991a596240 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f04f000a4d0 tx=0x7f04f0005770 comp rx=0 tx=0).stop 2024-09-19T06:22:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.665+0000 7f050a29b740 1 -- 172.21.15.5:0/2037443380 shutdown_connections 2024-09-19T06:22:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.665+0000 7f050a29b740 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55991a5a8fd0 0x55991a5ab3c0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.665+0000 7f04e97fa640 0 client.29681 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.665+0000 7f050a29b740 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a599680 0x55991a596780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.666+0000 7f050a29b740 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58edc0 0x55991a596240 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.666+0000 7f050a29b740 1 --2- 172.21.15.5:0/2037443380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58e4a0 0x55991a598f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.666+0000 7f050a29b740 1 -- 172.21.15.5:0/2037443380 >> 172.21.15.5:0/2037443380 conn(0x55991a58aa30 msgr2=0x55991a590760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.666+0000 7f050a29b740 1 -- 172.21.15.5:0/2037443380 shutdown_connections 2024-09-19T06:22:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.666+0000 7f050a29b740 1 -- 172.21.15.5:0/2037443380 wait complete. 2024-09-19T06:22:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.666+0000 7f050a29b740 20 client.29681 trim_cache size 0 max 16384 2024-09-19T06:22:27.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.699+0000 7f050a29b740 1 Processor -- start 2024-09-19T06:22:27.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.699+0000 7f050a29b740 1 -- start start 2024-09-19T06:22:27.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.699+0000 7f050a29b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58aa30 0x55991a5a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.699+0000 7f050a29b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 0x55991a5b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.699+0000 7f050a29b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a58edc0 0x55991a5b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.699+0000 7f050545d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 0x55991a5b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.699+0000 7f0505c5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58aa30 0x55991a5a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.699+0000 7f050545d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 0x55991a5b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:32998/0 (socket says 172.21.15.5:32998) 2024-09-19T06:22:27.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.699+0000 7f050545d640 1 -- 172.21.15.5:0/2287941077 learned_addr learned my addr 172.21.15.5:0/2287941077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:27.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.700+0000 7f050645f640 1 --2- 172.21.15.5:0/2287941077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a58edc0 0x55991a5b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.699+0000 7f050a29b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05001b3f00 con 0x55991a58aa30 2024-09-19T06:22:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.700+0000 7f050a29b740 1 -- 172.21.15.5:0/2287941077 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f05001b5a40 con 0x55991a58e4a0 2024-09-19T06:22:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.700+0000 7f050a29b740 1 -- 172.21.15.5:0/2287941077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05001b45d0 con 0x55991a58edc0 2024-09-19T06:22:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.700+0000 7f050545d640 1 -- 172.21.15.5:0/2287941077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a58edc0 msgr2=0x55991a5b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.700+0000 7f050545d640 1 --2- 172.21.15.5:0/2287941077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a58edc0 0x55991a5b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.700+0000 7f050545d640 1 -- 172.21.15.5:0/2287941077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58aa30 msgr2=0x55991a5a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.700+0000 7f050545d640 1 --2- 172.21.15.5:0/2287941077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58aa30 0x55991a5a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.700+0000 7f050545d640 1 -- 172.21.15.5:0/2287941077 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55991a58e2d0 con 0x55991a58e4a0 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.700+0000 7f0505c5e640 1 --2- 172.21.15.5:0/2287941077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58aa30 0x55991a5a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.700+0000 7f050545d640 1 --2- 172.21.15.5:0/2287941077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 0x55991a5b0990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f04f0005e60 tx=0x7f04f00053e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bd466593bc5c092 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.701+0000 7f04eb7fe640 1 -- 172.21.15.5:0/2287941077 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f003b070 con 0x55991a58e4a0 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.701+0000 7f04eb7fe640 1 -- 172.21.15.5:0/2287941077 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04f000a700 con 0x55991a58e4a0 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.701+0000 7f04eb7fe640 1 -- 172.21.15.5:0/2287941077 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0027040 con 0x55991a58e4a0 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.702+0000 7f050a29b740 1 -- 172.21.15.5:0/2287941077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 msgr2=0x55991a5b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.702+0000 7f050a29b740 1 --2- 172.21.15.5:0/2287941077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 0x55991a5b0990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f04f0005e60 tx=0x7f04f00053e0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.702+0000 7f050a29b740 1 -- 172.21.15.5:0/2287941077 shutdown_connections 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.702+0000 7f050a29b740 1 --2- 172.21.15.5:0/2287941077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a58edc0 0x55991a5b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.702+0000 7f050a29b740 1 --2- 172.21.15.5:0/2287941077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 0x55991a5b0990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.702+0000 7f050a29b740 1 --2- 172.21.15.5:0/2287941077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58aa30 0x55991a5a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.702+0000 7f050a29b740 1 -- 172.21.15.5:0/2287941077 >> 172.21.15.5:0/2287941077 conn(0x55991a5968e0 msgr2=0x55991a591900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.702+0000 7f050a29b740 1 -- 172.21.15.5:0/2287941077 shutdown_connections 2024-09-19T06:22:27.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.702+0000 7f050a29b740 1 -- 172.21.15.5:0/2287941077 wait complete. 2024-09-19T06:22:27.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.703+0000 7f050a29b740 1 -- start start 2024-09-19T06:22:27.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.703+0000 7f050a29b740 1 Processor -- start 2024-09-19T06:22:27.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.703+0000 7f050a29b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58aa30 0x55991a5a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.703+0000 7f050a29b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 0x55991a59ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.703+0000 7f050a29b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a58edc0 0x55991a59d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.703+0000 7f050a29b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05001b5370 con 0x55991a58aa30 2024-09-19T06:22:27.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.703+0000 7f050a29b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f05001b4ca0 con 0x55991a58e4a0 2024-09-19T06:22:27.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.703+0000 7f050a29b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05001b1f00 con 0x55991a58edc0 2024-09-19T06:22:27.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.703+0000 7f050a29b740 10 client.0 _ll_register_callbacks cb 0x7f04f8034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:27.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.704+0000 7f050545d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 0x55991a59ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.704+0000 7f050545d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 0x55991a59ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33012/0 (socket says 172.21.15.5:33012) 2024-09-19T06:22:27.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.704+0000 7f050645f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a58edc0 0x55991a59d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.704+0000 7f050545d640 1 -- 172.21.15.5:0/774380610 learned_addr learned my addr 172.21.15.5:0/774380610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:27.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.704+0000 7f0505c5e640 1 --2- 172.21.15.5:0/774380610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58aa30 0x55991a5a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.704+0000 7f050545d640 1 -- 172.21.15.5:0/774380610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a58edc0 msgr2=0x55991a59d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:27.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.704+0000 7f050545d640 1 --2- 172.21.15.5:0/774380610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a58edc0 0x55991a59d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.704+0000 7f050545d640 1 -- 172.21.15.5:0/774380610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58aa30 msgr2=0x55991a5a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:27.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.704+0000 7f050545d640 1 --2- 172.21.15.5:0/774380610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58aa30 0x55991a5a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.704+0000 7f050545d640 1 -- 172.21.15.5:0/774380610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55991a59d950 con 0x55991a58e4a0 2024-09-19T06:22:27.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.704+0000 7f050545d640 1 --2- 172.21.15.5:0/774380610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 0x55991a59ce50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f04f0004610 tx=0x7f04f000afd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=214c8e38cce8ab49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:27.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.705+0000 7f04e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:22:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.705+0000 7f04e97fa640 1 -- 172.21.15.5:0/774380610 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f003b070 con 0x55991a58e4a0 2024-09-19T06:22:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.705+0000 7f04e97fa640 1 -- 172.21.15.5:0/774380610 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f04f0004660 con 0x55991a58e4a0 2024-09-19T06:22:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.705+0000 7f050a29b740 1 -- 172.21.15.5:0/774380610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04f000a700 con 0x55991a58e4a0 2024-09-19T06:22:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.705+0000 7f04e97fa640 1 -- 172.21.15.5:0/774380610 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f04f0027040 con 0x55991a58e4a0 2024-09-19T06:22:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.705+0000 7f050a29b740 10 client.29687 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.705+0000 7f050a29b740 1 -- 172.21.15.5:0/774380610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55991a6136e0 con 0x55991a58e4a0 2024-09-19T06:22:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.705+0000 7f04e97fa640 1 -- 172.21.15.5:0/774380610 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04f000a700 con 0x55991a58e4a0 2024-09-19T06:22:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.705+0000 7f04e97fa640 1 -- 172.21.15.5:0/774380610 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f04f0003a20 con 0x55991a58e4a0 2024-09-19T06:22:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.705+0000 7f050a29b740 10 client.29687 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:27.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.705+0000 7f050a29b740 10 client.29687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:27.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.705+0000 7f050a29b740 20 client.29687 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:27.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.706+0000 7f050a29b740 1 --2- 172.21.15.5:0/774380610 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55991a599680 0x55991a615de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:27.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.706+0000 7f050a29b740 4 client.29687 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:22:27.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.706+0000 7f050a29b740 1 -- 172.21.15.5:0/774380610 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55991a4fbd60 con 0x55991a599680 2024-09-19T06:22:27.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.706+0000 7f0505c5e640 1 --2- 172.21.15.5:0/774380610 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55991a599680 0x55991a615de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.706+0000 7f04e97fa640 10 client.29687 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.706+0000 7f0505c5e640 1 --2- 172.21.15.5:0/774380610 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55991a599680 0x55991a615de0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7da2b49c96c6e5 server_cookie=edcf7cea6ae9e15b in_seq=0 out_seq=0 2024-09-19T06:22:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f04e97fa640 1 -- 172.21.15.5:0/774380610 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55991a4fbd60 con 0x55991a599680 2024-09-19T06:22:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f04e97fa640 10 client.29687 handle_command_reply: tid=0 2024-09-19T06:22:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f050a29b740 1 client.29687 shutdown 2024-09-19T06:22:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f050a29b740 1 -- 172.21.15.5:0/774380610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 msgr2=0x55991a59ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f050a29b740 1 --2- 172.21.15.5:0/774380610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 0x55991a59ce50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f04f0004610 tx=0x7f04f000afd0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f050a29b740 1 -- 172.21.15.5:0/774380610 shutdown_connections 2024-09-19T06:22:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f050a29b740 1 --2- 172.21.15.5:0/774380610 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55991a599680 0x55991a615de0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f050a29b740 1 --2- 172.21.15.5:0/774380610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55991a58edc0 0x55991a59d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f050a29b740 1 --2- 172.21.15.5:0/774380610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55991a58e4a0 0x55991a59ce50 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f04e97fa640 0 client.29687 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:27.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f050a29b740 1 --2- 172.21.15.5:0/774380610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55991a58aa30 0x55991a5a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f050a29b740 1 -- 172.21.15.5:0/774380610 >> 172.21.15.5:0/774380610 conn(0x55991a5968e0 msgr2=0x55991a589350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:27.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f050a29b740 1 -- 172.21.15.5:0/774380610 shutdown_connections 2024-09-19T06:22:27.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.707+0000 7f050a29b740 1 -- 172.21.15.5:0/774380610 wait complete. 2024-09-19T06:22:27.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.708+0000 7f050a29b740 20 client.29687 trim_cache size 0 max 16384 2024-09-19T06:22:27.713 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:22:27.713 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:22:27.713 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:22:27.713 INFO:teuthology.orchestra.run.smithi005.stdout: "d802d51e-d3d3-4db0-8f88-f5323564ddc7": { 2024-09-19T06:22:27.713 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:22:27.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d802d51e-d3d3-4db0-8f88-f5323564ddc7", 2024-09-19T06:22:27.714 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:22:27.714 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:22:27.714 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:22:27.714 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:22:27.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.708+0000 7f0507ec2640 1 -- 172.21.15.5:0/446606201 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f04ec0a3430 msgr2=0x7f04ec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:27.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.708+0000 7f0507ec2640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f04ec0a3430 0x7f04ec0a58f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009370 tx=0x7f04f8005e50 comp rx=0 tx=0).stop 2024-09-19T06:22:27.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.708+0000 7f0507ec2640 1 -- 172.21.15.5:0/446606201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0500102df0 msgr2=0x7f05001a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:27.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.708+0000 7f0507ec2640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0500102df0 0x7f05001a4740 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00e9c0 tx=0x7f04fc00ee80 comp rx=0 tx=0).stop 2024-09-19T06:22:27.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.708+0000 7f0507ec2640 1 -- 172.21.15.5:0/446606201 shutdown_connections 2024-09-19T06:22:27.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.708+0000 7f0507ec2640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050010c2f0 0x7f050010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.708+0000 7f0507ec2640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05001037a0 0x7f05001a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.708+0000 7f0507ec2640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f04ec0a3430 0x7f04ec0a58f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.708+0000 7f0507ec2640 1 --2- 172.21.15.5:0/446606201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0500102df0 0x7f05001a4740 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:27.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.708+0000 7f0507ec2640 1 -- 172.21.15.5:0/446606201 >> 172.21.15.5:0/446606201 conn(0x7f05000fc7d0 msgr2=0x7f0500101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:27.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.709+0000 7f0507ec2640 1 -- 172.21.15.5:0/446606201 shutdown_connections 2024-09-19T06:22:27.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:22:27.709+0000 7f0507ec2640 1 -- 172.21.15.5:0/446606201 wait complete. 2024-09-19T06:22:27.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:22:27.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:22:27.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:27.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:22:27.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:27.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:22:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:22:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:22:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:28.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:22:28.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:22:28.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:28.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:22:28.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:28.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:22:28.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 58016 8.73 MB/sec execute 232 sec latency 130.840 ms 2024-09-19T06:22:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:28 smithi005 ceph-mon[30955]: pgmap v603: 97 pgs: 97 active+clean; 971 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.1 MiB/s wr, 440 op/s 2024-09-19T06:22:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:28 smithi049 ceph-mon[34640]: pgmap v603: 97 pgs: 97 active+clean; 971 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.1 MiB/s wr, 440 op/s 2024-09-19T06:22:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:28 smithi157 ceph-mon[35346]: pgmap v603: 97 pgs: 97 active+clean; 971 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.1 MiB/s wr, 440 op/s 2024-09-19T06:22:29.452 INFO:tasks.cephfs.filesystem:scrub status for tag:d802d51e-d3d3-4db0-8f88-f5323564ddc7 - {'path': '/', 'tag': 'd802d51e-d3d3-4db0-8f88-f5323564ddc7', 'options': 'recursive,force'} 2024-09-19T06:22:29.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 58192 8.74 MB/sec execute 233 sec latency 325.323 ms 2024-09-19T06:22:30.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 58356 8.74 MB/sec execute 234 sec latency 93.945 ms 2024-09-19T06:22:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:30 smithi005 ceph-mon[30955]: pgmap v604: 97 pgs: 97 active+clean; 971 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 294 op/s 2024-09-19T06:22:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:30 smithi049 ceph-mon[34640]: pgmap v604: 97 pgs: 97 active+clean; 971 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 294 op/s 2024-09-19T06:22:31.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:30 smithi157 ceph-mon[35346]: pgmap v604: 97 pgs: 97 active+clean; 971 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 294 op/s 2024-09-19T06:22:31.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 58511 8.73 MB/sec execute 235 sec latency 84.528 ms 2024-09-19T06:22:32.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 58680 8.72 MB/sec execute 236 sec latency 62.045 ms 2024-09-19T06:22:32.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:32 smithi005 ceph-mon[30955]: pgmap v605: 97 pgs: 97 active+clean; 971 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 294 op/s 2024-09-19T06:22:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:32 smithi049 ceph-mon[34640]: pgmap v605: 97 pgs: 97 active+clean; 971 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 294 op/s 2024-09-19T06:22:33.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:32 smithi157 ceph-mon[35346]: pgmap v605: 97 pgs: 97 active+clean; 971 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 294 op/s 2024-09-19T06:22:33.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 58868 8.73 MB/sec execute 237 sec latency 62.056 ms 2024-09-19T06:22:34.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 58999 8.70 MB/sec execute 238 sec latency 106.359 ms 2024-09-19T06:22:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:34 smithi005 ceph-mon[30955]: pgmap v606: 97 pgs: 97 active+clean; 965 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 445 op/s 2024-09-19T06:22:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:34 smithi049 ceph-mon[34640]: pgmap v606: 97 pgs: 97 active+clean; 965 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 445 op/s 2024-09-19T06:22:35.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:34 smithi157 ceph-mon[35346]: pgmap v606: 97 pgs: 97 active+clean; 965 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 445 op/s 2024-09-19T06:22:35.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 59160 8.70 MB/sec execute 239 sec latency 65.088 ms 2024-09-19T06:22:36.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 59349 8.71 MB/sec execute 240 sec latency 62.938 ms 2024-09-19T06:22:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:36 smithi005 ceph-mon[30955]: pgmap v607: 97 pgs: 97 active+clean; 965 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 330 op/s 2024-09-19T06:22:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:36 smithi049 ceph-mon[34640]: pgmap v607: 97 pgs: 97 active+clean; 965 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 330 op/s 2024-09-19T06:22:37.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:36 smithi157 ceph-mon[35346]: pgmap v607: 97 pgs: 97 active+clean; 965 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 330 op/s 2024-09-19T06:22:37.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 59548 8.73 MB/sec execute 241 sec latency 83.548 ms 2024-09-19T06:22:38.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 59757 8.74 MB/sec execute 242 sec latency 60.726 ms 2024-09-19T06:22:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:38 smithi005 ceph-mon[30955]: pgmap v608: 97 pgs: 97 active+clean; 975 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 449 op/s 2024-09-19T06:22:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:38 smithi049 ceph-mon[34640]: pgmap v608: 97 pgs: 97 active+clean; 975 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 449 op/s 2024-09-19T06:22:39.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:38 smithi157 ceph-mon[35346]: pgmap v608: 97 pgs: 97 active+clean; 975 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 449 op/s 2024-09-19T06:22:39.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 59906 8.74 MB/sec execute 243 sec latency 69.333 ms 2024-09-19T06:22:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:22:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:22:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:22:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:22:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:22:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:22:40.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:22:40.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:22:40.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:22:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.240+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/3974454386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c78114ab0 msgr2=0x7f6c78116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.240+0000 7f6c7e2e4640 1 --2- 172.21.15.5:0/3974454386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c78114ab0 0x7f6c78116ea0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009ca0 tx=0x7f6c7002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.241+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/3974454386 shutdown_connections 2024-09-19T06:22:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.241+0000 7f6c7e2e4640 1 --2- 172.21.15.5:0/3974454386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c78114ab0 0x7f6c78116ea0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.241+0000 7f6c7e2e4640 1 --2- 172.21.15.5:0/3974454386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c78109b10 0x7f6c78114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.241+0000 7f6c7e2e4640 1 --2- 172.21.15.5:0/3974454386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c781091f0 0x7f6c781095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.241+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/3974454386 >> 172.21.15.5:0/3974454386 conn(0x7f6c7806cdf0 msgr2=0x7f6c7806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.241+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/3974454386 shutdown_connections 2024-09-19T06:22:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.241+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/3974454386 wait complete. 2024-09-19T06:22:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c7e2e4640 1 Processor -- start 2024-09-19T06:22:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c7e2e4640 1 -- start start 2024-09-19T06:22:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c7e2e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781091f0 0x7f6c7810a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c7e2e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c78109b10 0x7f6c7810a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c7e2e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c78114ab0 0x7f6c7810ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c7e2e4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7811a590 con 0x7f6c78109b10 2024-09-19T06:22:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c7e2e4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7811a410 con 0x7f6c781091f0 2024-09-19T06:22:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c7e2e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7811a710 con 0x7f6c78114ab0 2024-09-19T06:22:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c77fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781091f0 0x7f6c7810a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c77fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781091f0 0x7f6c7810a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36456/0 (socket says 172.21.15.5:36456) 2024-09-19T06:22:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c77fff640 1 -- 172.21.15.5:0/1799914236 learned_addr learned my addr 172.21.15.5:0/1799914236 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c777fe640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c78109b10 0x7f6c7810a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.242+0000 7f6c7c881640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c78114ab0 0x7f6c7810ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.243+0000 7f6c77fff640 1 -- 172.21.15.5:0/1799914236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c78114ab0 msgr2=0x7f6c7810ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.243+0000 7f6c77fff640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c78114ab0 0x7f6c7810ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.243+0000 7f6c77fff640 1 -- 172.21.15.5:0/1799914236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c78109b10 msgr2=0x7f6c7810a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.243+0000 7f6c77fff640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c78109b10 0x7f6c7810a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.243+0000 7f6c77fff640 1 -- 172.21.15.5:0/1799914236 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c781aef60 con 0x7f6c781091f0 2024-09-19T06:22:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.243+0000 7f6c777fe640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c78109b10 0x7f6c7810a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.243+0000 7f6c77fff640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781091f0 0x7f6c7810a3d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e9f0 tx=0x7f6c6c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.243+0000 7f6c757fa640 1 -- 172.21.15.5:0/1799914236 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c00cd80 con 0x7f6c781091f0 2024-09-19T06:22:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.243+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/1799914236 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c781af250 con 0x7f6c781091f0 2024-09-19T06:22:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.243+0000 7f6c757fa640 1 -- 172.21.15.5:0/1799914236 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c6c0108e0 con 0x7f6c781091f0 2024-09-19T06:22:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.243+0000 7f6c757fa640 1 -- 172.21.15.5:0/1799914236 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c018920 con 0x7f6c781091f0 2024-09-19T06:22:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.244+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/1799914236 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c781af6e0 con 0x7f6c781091f0 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.246+0000 7f6c757fa640 1 -- 172.21.15.5:0/1799914236 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6c6c010430 con 0x7f6c781091f0 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.246+0000 7f6c81085740 1 Processor -- start 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c81085740 1 -- start start 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c81085740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224244af0 0x557224244ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c81085740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 0x55722424f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c81085740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x5572242520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c81085740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5572241a92f0 con 0x557224245410 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c81085740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557224151270 con 0x55722424fcb0 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c81085740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5572241a52b0 con 0x557224244af0 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c7c881640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224244af0 0x557224244ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c7c881640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224244af0 0x557224244ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:51152/0 (socket says 172.21.15.5:51152) 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c7c881640 1 -- 172.21.15.5:0/480908569 learned_addr learned my addr 172.21.15.5:0/480908569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c777fe640 1 --2- 172.21.15.5:0/480908569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x5572242520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c77fff640 1 --2- 172.21.15.5:0/480908569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 0x55722424f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c7c881640 1 -- 172.21.15.5:0/480908569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 msgr2=0x5572242520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c7c881640 1 --2- 172.21.15.5:0/480908569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x5572242520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c7c881640 1 -- 172.21.15.5:0/480908569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 msgr2=0x55722424f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.247+0000 7f6c757fa640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6c600a3430 0x7f6c600a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.248+0000 7f6c7c881640 1 --2- 172.21.15.5:0/480908569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 0x55722424f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.248+0000 7f6c7c881640 1 -- 172.21.15.5:0/480908569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557224150340 con 0x557224244af0 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.248+0000 7f6c777fe640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6c600a3430 0x7f6c600a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.248+0000 7f6c77fff640 1 --2- 172.21.15.5:0/480908569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 0x55722424f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.248+0000 7f6c7c881640 1 --2- 172.21.15.5:0/480908569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224244af0 0x557224244ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6c7002aeb0 tx=0x7f6c70005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e441020b4188f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.248+0000 7f6c757fa640 1 -- 172.21.15.5:0/1799914236 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f6c6c0d00c0 con 0x7f6c781091f0 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.248+0000 7f6c777fe640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6c600a3430 0x7f6c600a58f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x5572242423e0 tx=0x7f6c68005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.248+0000 7f6c5e7fc640 1 -- 172.21.15.5:0/480908569 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70031d40 con 0x557224244af0 2024-09-19T06:22:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.248+0000 7f6c5e7fc640 1 -- 172.21.15.5:0/480908569 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c70036030 con 0x557224244af0 2024-09-19T06:22:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.248+0000 7f6c5e7fc640 1 -- 172.21.15.5:0/480908569 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70004330 con 0x557224244af0 2024-09-19T06:22:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.249+0000 7f6c81085740 1 -- 172.21.15.5:0/480908569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224244af0 msgr2=0x557224244ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.249+0000 7f6c81085740 1 --2- 172.21.15.5:0/480908569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224244af0 0x557224244ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6c7002aeb0 tx=0x7f6c70005f30 comp rx=0 tx=0).stop 2024-09-19T06:22:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.249+0000 7f6c81085740 1 -- 172.21.15.5:0/480908569 shutdown_connections 2024-09-19T06:22:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.249+0000 7f6c81085740 1 --2- 172.21.15.5:0/480908569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x5572242520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.249+0000 7f6c81085740 1 --2- 172.21.15.5:0/480908569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 0x55722424f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.249+0000 7f6c81085740 1 --2- 172.21.15.5:0/480908569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224244af0 0x557224244ed0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.249+0000 7f6c81085740 1 -- 172.21.15.5:0/480908569 >> 172.21.15.5:0/480908569 conn(0x557224241060 msgr2=0x557224241440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.249+0000 7f6c81085740 1 -- 172.21.15.5:0/480908569 shutdown_connections 2024-09-19T06:22:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.249+0000 7f6c81085740 1 -- 172.21.15.5:0/480908569 wait complete. 2024-09-19T06:22:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.249+0000 7f6c81085740 1 -- start start 2024-09-19T06:22:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c81085740 1 Processor -- start 2024-09-19T06:22:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c81085740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 0x55722424cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c81085740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557224245410 0x55722424d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c81085740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55722424fcb0 0x55722424d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c81085740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557224252700 con 0x557224244af0 2024-09-19T06:22:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c81085740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557224132290 con 0x557224245410 2024-09-19T06:22:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c81085740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557224252880 con 0x55722424fcb0 2024-09-19T06:22:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 0x55722424cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c81085740 10 client.0 _ll_register_callbacks cb 0x557224163140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 0x55722424cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43418/0 (socket says 172.21.15.5:43418) 2024-09-19T06:22:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c77fff640 1 -- 172.21.15.5:0/1485984466 learned_addr learned my addr 172.21.15.5:0/1485984466 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c777fe640 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557224245410 0x55722424d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c7c881640 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55722424fcb0 0x55722424d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c77fff640 1 -- 172.21.15.5:0/1485984466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55722424fcb0 msgr2=0x55722424d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c77fff640 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55722424fcb0 0x55722424d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c77fff640 1 -- 172.21.15.5:0/1485984466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557224245410 msgr2=0x55722424d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c77fff640 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557224245410 0x55722424d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c77fff640 1 -- 172.21.15.5:0/1485984466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557223ebc210 con 0x557224244af0 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.250+0000 7f6c777fe640 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557224245410 0x55722424d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.251+0000 7f6c77fff640 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 0x55722424cb10 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e9c0 tx=0x7f6c6c09bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b0484221f25a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.251+0000 7f6c577fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.251+0000 7f6c577fe640 1 -- 172.21.15.5:0/1485984466 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c09c3d0 con 0x557224244af0 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.251+0000 7f6c577fe640 1 -- 172.21.15.5:0/1485984466 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f6c6c0d20d0 con 0x557224244af0 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.251+0000 7f6c81085740 1 -- 172.21.15.5:0/1485984466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c70036030 con 0x557224244af0 2024-09-19T06:22:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.251+0000 7f6c577fe640 1 -- 172.21.15.5:0/1485984466 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6c6c0d22f0 con 0x557224244af0 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.251+0000 7f6c81085740 10 client.20328 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.251+0000 7f6c81085740 1 -- 172.21.15.5:0/1485984466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557223f88620 con 0x557224244af0 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.252+0000 7f6c577fe640 1 -- 172.21.15.5:0/1485984466 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c6c0a06d0 con 0x557224244af0 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.252+0000 7f6c577fe640 1 -- 172.21.15.5:0/1485984466 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f6c6c0bc220 con 0x557224244af0 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.253+0000 7f6c81085740 10 client.20328 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.253+0000 7f6c81085740 10 client.20328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.253+0000 7f6c81085740 20 client.20328 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.253+0000 7f6c81085740 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55722425f960 0x557224261d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.253+0000 7f6c81085740 4 client.20328 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.253+0000 7f6c81085740 1 -- 172.21.15.5:0/1485984466 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6c70036030 con 0x55722425f960 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.253+0000 7f6c777fe640 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55722425f960 0x557224261d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.253+0000 7f6c777fe640 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55722425f960 0x557224261d50 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf58026e78b2a49d server_cookie=c01304e5c1043059 in_seq=0 out_seq=0 2024-09-19T06:22:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.253+0000 7f6c577fe640 10 client.20328 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c577fe640 1 -- 172.21.15.5:0/1485984466 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6c70036030 con 0x55722425f960 2024-09-19T06:22:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c577fe640 10 client.20328 handle_command_reply: tid=0 2024-09-19T06:22:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c81085740 1 client.20328 shutdown 2024-09-19T06:22:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c81085740 1 -- 172.21.15.5:0/1485984466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 msgr2=0x55722424cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c81085740 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 0x55722424cb10 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e9c0 tx=0x7f6c6c09bdb0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c81085740 1 -- 172.21.15.5:0/1485984466 shutdown_connections 2024-09-19T06:22:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c81085740 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55722425f960 0x557224261d50 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c81085740 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55722424fcb0 0x55722424d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c81085740 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557224245410 0x55722424d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c81085740 1 --2- 172.21.15.5:0/1485984466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 0x55722424cb10 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c81085740 1 -- 172.21.15.5:0/1485984466 >> 172.21.15.5:0/1485984466 conn(0x557224241060 msgr2=0x557224246dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c577fe640 0 client.20328 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c81085740 1 -- 172.21.15.5:0/1485984466 shutdown_connections 2024-09-19T06:22:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.254+0000 7f6c81085740 1 -- 172.21.15.5:0/1485984466 wait complete. 2024-09-19T06:22:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.255+0000 7f6c81085740 20 client.20328 trim_cache size 0 max 16384 2024-09-19T06:22:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.279+0000 7f6c81085740 1 Processor -- start 2024-09-19T06:22:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.279+0000 7f6c81085740 1 -- start start 2024-09-19T06:22:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.279+0000 7f6c81085740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224244af0 0x55722424d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.279+0000 7f6c81085740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 0x55722424ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c81085740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x557224267620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c81085740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781b8320 con 0x557224245410 2024-09-19T06:22:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c81085740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781b9e60 con 0x55722424fcb0 2024-09-19T06:22:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c81085740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781b89f0 con 0x557224244af0 2024-09-19T06:22:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 0x55722424ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 0x55722424ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43428/0 (socket says 172.21.15.5:43428) 2024-09-19T06:22:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c777fe640 1 -- 172.21.15.5:0/1401643642 learned_addr learned my addr 172.21.15.5:0/1401643642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c7c881640 1 --2- 172.21.15.5:0/1401643642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x557224267620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c77fff640 1 --2- 172.21.15.5:0/1401643642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224244af0 0x55722424d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c777fe640 1 -- 172.21.15.5:0/1401643642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224244af0 msgr2=0x55722424d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c777fe640 1 --2- 172.21.15.5:0/1401643642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224244af0 0x55722424d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c777fe640 1 -- 172.21.15.5:0/1401643642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 msgr2=0x557224267620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c777fe640 1 --2- 172.21.15.5:0/1401643642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x557224267620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.280+0000 7f6c777fe640 1 -- 172.21.15.5:0/1401643642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557224240e90 con 0x557224245410 2024-09-19T06:22:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.281+0000 7f6c777fe640 1 --2- 172.21.15.5:0/1401643642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 0x55722424ea80 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x55722424c380 tx=0x7f6c6801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3a04ae973059e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.281+0000 7f6c5dffb640 1 -- 172.21.15.5:0/1401643642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6801d920 con 0x557224245410 2024-09-19T06:22:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.282+0000 7f6c5dffb640 1 -- 172.21.15.5:0/1401643642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c68005020 con 0x557224245410 2024-09-19T06:22:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.282+0000 7f6c5dffb640 1 -- 172.21.15.5:0/1401643642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68004b10 con 0x557224245410 2024-09-19T06:22:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.283+0000 7f6c81085740 1 -- 172.21.15.5:0/1401643642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 msgr2=0x55722424ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.283+0000 7f6c81085740 1 --2- 172.21.15.5:0/1401643642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 0x55722424ea80 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x55722424c380 tx=0x7f6c6801b000 comp rx=0 tx=0).stop 2024-09-19T06:22:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.283+0000 7f6c81085740 1 -- 172.21.15.5:0/1401643642 shutdown_connections 2024-09-19T06:22:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.283+0000 7f6c81085740 1 --2- 172.21.15.5:0/1401643642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x557224267620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.283+0000 7f6c81085740 1 --2- 172.21.15.5:0/1401643642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224245410 0x55722424ea80 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.283+0000 7f6c81085740 1 --2- 172.21.15.5:0/1401643642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224244af0 0x55722424d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.283+0000 7f6c81085740 1 -- 172.21.15.5:0/1401643642 >> 172.21.15.5:0/1401643642 conn(0x557224241060 msgr2=0x55722424e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.283+0000 7f6c81085740 1 -- 172.21.15.5:0/1401643642 shutdown_connections 2024-09-19T06:22:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.283+0000 7f6c81085740 1 -- 172.21.15.5:0/1401643642 wait complete. 2024-09-19T06:22:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.284+0000 7f6c81085740 1 -- start start 2024-09-19T06:22:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.284+0000 7f6c81085740 1 Processor -- start 2024-09-19T06:22:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.284+0000 7f6c81085740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 0x5572242581e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.284+0000 7f6c81085740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224245410 0x557224249760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.284+0000 7f6c81085740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x557224249ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c81085740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781b9790 con 0x557224244af0 2024-09-19T06:22:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c81085740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781b90c0 con 0x55722424fcb0 2024-09-19T06:22:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c81085740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781b6320 con 0x557224245410 2024-09-19T06:22:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 0x5572242581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 0x5572242581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43436/0 (socket says 172.21.15.5:43436) 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c77fff640 1 -- 172.21.15.5:0/2967714986 learned_addr learned my addr 172.21.15.5:0/2967714986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c777fe640 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224245410 0x557224249760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c7c881640 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x557224249ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c81085740 10 client.0 _ll_register_callbacks cb 0x7f6c6c09bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c77fff640 1 -- 172.21.15.5:0/2967714986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224245410 msgr2=0x557224249760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c77fff640 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224245410 0x557224249760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c77fff640 1 -- 172.21.15.5:0/2967714986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 msgr2=0x557224249ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c77fff640 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x557224249ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c77fff640 1 -- 172.21.15.5:0/2967714986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55722424a4d0 con 0x557224244af0 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c777fe640 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224245410 0x557224249760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c7c881640 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x557224249ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:22:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c77fff640 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 0x5572242581e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c09c0d0 tx=0x7f6c6c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2690caf5411c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c577fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c577fe640 1 -- 172.21.15.5:0/2967714986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c09c1a0 con 0x557224244af0 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c577fe640 1 -- 172.21.15.5:0/2967714986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f6c6c09b820 con 0x557224244af0 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.285+0000 7f6c81085740 1 -- 172.21.15.5:0/2967714986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c68005020 con 0x557224244af0 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.286+0000 7f6c577fe640 1 -- 172.21.15.5:0/2967714986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6c6c09c3d0 con 0x557224244af0 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.286+0000 7f6c81085740 10 client.20334 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.286+0000 7f6c81085740 1 -- 172.21.15.5:0/2967714986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5572242ca5f0 con 0x557224244af0 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.286+0000 7f6c577fe640 1 -- 172.21.15.5:0/2967714986 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c68005020 con 0x557224244af0 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.286+0000 7f6c577fe640 1 -- 172.21.15.5:0/2967714986 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f6c6c0856d0 con 0x557224244af0 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.286+0000 7f6c81085740 10 client.20334 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.286+0000 7f6c81085740 10 client.20334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.286+0000 7f6c81085740 20 client.20334 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.286+0000 7f6c81085740 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5572242cac90 0x5572242cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.286+0000 7f6c81085740 4 client.20334 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:22:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.286+0000 7f6c81085740 1 -- 172.21.15.5:0/2967714986 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5572241b6a20 con 0x5572242cac90 2024-09-19T06:22:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.286+0000 7f6c777fe640 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5572242cac90 0x5572242cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.287+0000 7f6c777fe640 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5572242cac90 0x5572242cd0b0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c17c4a0a8b29ce server_cookie=836355d44fa49d98 in_seq=0 out_seq=0 2024-09-19T06:22:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.287+0000 7f6c577fe640 10 client.20334 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.378+0000 7f6c577fe640 1 -- 172.21.15.5:0/2967714986 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5572241b6a20 con 0x5572242cac90 2024-09-19T06:22:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.378+0000 7f6c577fe640 10 client.20334 handle_command_reply: tid=0 2024-09-19T06:22:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.378+0000 7f6c81085740 1 client.20334 shutdown 2024-09-19T06:22:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.378+0000 7f6c81085740 1 -- 172.21.15.5:0/2967714986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 msgr2=0x5572242581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.378+0000 7f6c81085740 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 0x5572242581e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c09c0d0 tx=0x7f6c6c009700 comp rx=0 tx=0).stop 2024-09-19T06:22:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.378+0000 7f6c81085740 1 -- 172.21.15.5:0/2967714986 shutdown_connections 2024-09-19T06:22:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.378+0000 7f6c81085740 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5572242cac90 0x5572242cd0b0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.378+0000 7f6c81085740 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55722424fcb0 0x557224249ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.378+0000 7f6c81085740 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557224245410 0x557224249760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.378+0000 7f6c81085740 1 --2- 172.21.15.5:0/2967714986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557224244af0 0x5572242581e0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.378+0000 7f6c577fe640 0 client.20334 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.378+0000 7f6c81085740 1 -- 172.21.15.5:0/2967714986 >> 172.21.15.5:0/2967714986 conn(0x557224241060 msgr2=0x557224246d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.379+0000 7f6c81085740 1 -- 172.21.15.5:0/2967714986 shutdown_connections 2024-09-19T06:22:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.379+0000 7f6c81085740 1 -- 172.21.15.5:0/2967714986 wait complete. 2024-09-19T06:22:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.379+0000 7f6c81085740 20 client.20334 trim_cache size 0 max 16384 2024-09-19T06:22:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:22:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:22:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:22:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:22:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.380+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/1799914236 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6c600a3430 msgr2=0x7f6c600a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.380+0000 7f6c7e2e4640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6c600a3430 0x7f6c600a58f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x5572242423e0 tx=0x7f6c68005f90 comp rx=0 tx=0).stop 2024-09-19T06:22:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.380+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/1799914236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781091f0 msgr2=0x7f6c7810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.380+0000 7f6c7e2e4640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781091f0 0x7f6c7810a3d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e9f0 tx=0x7f6c6c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.380+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/1799914236 shutdown_connections 2024-09-19T06:22:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.380+0000 7f6c7e2e4640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f6c600a3430 0x7f6c600a58f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.380+0000 7f6c7e2e4640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c78114ab0 0x7f6c7810ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.380+0000 7f6c7e2e4640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c78109b10 0x7f6c7810a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.380+0000 7f6c7e2e4640 1 --2- 172.21.15.5:0/1799914236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781091f0 0x7f6c7810a3d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.380+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/1799914236 >> 172.21.15.5:0/1799914236 conn(0x7f6c7806cdf0 msgr2=0x7f6c78070880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.380+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/1799914236 shutdown_connections 2024-09-19T06:22:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:40.380+0000 7f6c7e2e4640 1 -- 172.21.15.5:0/1799914236 wait complete. 2024-09-19T06:22:40.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60016 8.73 MB/sec execute 244 sec latency 105.333 ms 2024-09-19T06:22:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:40 smithi005 ceph-mon[30955]: pgmap v609: 97 pgs: 97 active+clean; 975 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 269 op/s 2024-09-19T06:22:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:40 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:22:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:40 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:22:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:40 smithi049 ceph-mon[34640]: pgmap v609: 97 pgs: 97 active+clean; 975 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 269 op/s 2024-09-19T06:22:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:40 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:22:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:40 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:22:41.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:40 smithi157 ceph-mon[35346]: pgmap v609: 97 pgs: 97 active+clean; 975 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 269 op/s 2024-09-19T06:22:41.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:40 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:22:41.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:40 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:22:41.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60195 8.75 MB/sec execute 245 sec latency 85.166 ms 2024-09-19T06:22:41.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:41 smithi005 ceph-mon[30955]: pgmap v610: 97 pgs: 97 active+clean; 975 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 269 op/s 2024-09-19T06:22:41.908 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:41 smithi049 ceph-mon[34640]: pgmap v610: 97 pgs: 97 active+clean; 975 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 269 op/s 2024-09-19T06:22:42.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:41 smithi157 ceph-mon[35346]: pgmap v610: 97 pgs: 97 active+clean; 975 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 269 op/s 2024-09-19T06:22:42.582 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60417 8.78 MB/sec execute 246 sec latency 55.097 ms 2024-09-19T06:22:43.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60575 8.77 MB/sec execute 247 sec latency 61.253 ms 2024-09-19T06:22:44.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:43 smithi157 ceph-mon[35346]: pgmap v611: 97 pgs: 97 active+clean; 988 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 406 op/s 2024-09-19T06:22:44.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:43 smithi005 ceph-mon[30955]: pgmap v611: 97 pgs: 97 active+clean; 988 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 406 op/s 2024-09-19T06:22:44.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:43 smithi049 ceph-mon[34640]: pgmap v611: 97 pgs: 97 active+clean; 988 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 406 op/s 2024-09-19T06:22:44.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60706 8.76 MB/sec execute 248 sec latency 68.147 ms 2024-09-19T06:22:45.582 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60855 8.76 MB/sec execute 249 sec latency 66.939 ms 2024-09-19T06:22:46.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:45 smithi157 ceph-mon[35346]: pgmap v612: 97 pgs: 97 active+clean; 988 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 255 op/s 2024-09-19T06:22:46.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:45 smithi005 ceph-mon[30955]: pgmap v612: 97 pgs: 97 active+clean; 988 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 255 op/s 2024-09-19T06:22:46.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:45 smithi049 ceph-mon[34640]: pgmap v612: 97 pgs: 97 active+clean; 988 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 255 op/s 2024-09-19T06:22:46.581 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61022 8.77 MB/sec execute 250 sec latency 68.314 ms 2024-09-19T06:22:47.582 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61189 8.77 MB/sec execute 251 sec latency 52.443 ms 2024-09-19T06:22:48.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:47 smithi157 ceph-mon[35346]: pgmap v613: 97 pgs: 97 active+clean; 1008 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.3 MiB/s wr, 374 op/s 2024-09-19T06:22:48.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:47 smithi005 ceph-mon[30955]: pgmap v613: 97 pgs: 97 active+clean; 1008 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.3 MiB/s wr, 374 op/s 2024-09-19T06:22:48.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:47 smithi049 ceph-mon[34640]: pgmap v613: 97 pgs: 97 active+clean; 1008 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.3 MiB/s wr, 374 op/s 2024-09-19T06:22:48.582 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61367 8.78 MB/sec execute 252 sec latency 61.276 ms 2024-09-19T06:22:49.582 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61511 8.77 MB/sec execute 253 sec latency 55.012 ms 2024-09-19T06:22:50.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:49 smithi157 ceph-mon[35346]: pgmap v614: 97 pgs: 97 active+clean; 1008 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-19T06:22:50.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:49 smithi005 ceph-mon[30955]: pgmap v614: 97 pgs: 97 active+clean; 1008 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-19T06:22:50.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:49 smithi049 ceph-mon[34640]: pgmap v614: 97 pgs: 97 active+clean; 1008 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-19T06:22:50.582 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61674 8.77 MB/sec execute 254 sec latency 91.702 ms 2024-09-19T06:22:51.582 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61829 8.76 MB/sec execute 255 sec latency 156.897 ms 2024-09-19T06:22:52.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:51 smithi157 ceph-mon[35346]: pgmap v615: 97 pgs: 97 active+clean; 1008 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-19T06:22:52.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:51 smithi005 ceph-mon[30955]: pgmap v615: 97 pgs: 97 active+clean; 1008 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-19T06:22:52.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:51 smithi049 ceph-mon[34640]: pgmap v615: 97 pgs: 97 active+clean; 1008 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-19T06:22:52.582 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61990 8.74 MB/sec execute 256 sec latency 57.054 ms 2024-09-19T06:22:53.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:22:53.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:22:53.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:22:53.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:22:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:22:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:22:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:22:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:22:53.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:22:53.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:22:53.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:22:53.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:22:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.512+0000 7f73bf02c640 1 -- 172.21.15.5:0/3113698863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b81011b0 msgr2=0x7f73b810de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.512+0000 7f73bf02c640 1 --2- 172.21.15.5:0/3113698863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b81011b0 0x7f73b810de70 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009900 tx=0x7f73a402a3f0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.512+0000 7f73bf02c640 1 -- 172.21.15.5:0/3113698863 shutdown_connections 2024-09-19T06:22:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.512+0000 7f73bf02c640 1 --2- 172.21.15.5:0/3113698863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73b810e3b0 0x7f73b81107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.512+0000 7f73bf02c640 1 --2- 172.21.15.5:0/3113698863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b81011b0 0x7f73b810de70 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.512+0000 7f73bf02c640 1 --2- 172.21.15.5:0/3113698863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b8100800 0x7f73b8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.512+0000 7f73bf02c640 1 -- 172.21.15.5:0/3113698863 >> 172.21.15.5:0/3113698863 conn(0x7f73b80779a0 msgr2=0x7f73b80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.512+0000 7f73bf02c640 1 -- 172.21.15.5:0/3113698863 shutdown_connections 2024-09-19T06:22:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.512+0000 7f73bf02c640 1 -- 172.21.15.5:0/3113698863 wait complete. 2024-09-19T06:22:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73bf02c640 1 Processor -- start 2024-09-19T06:22:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73bf02c640 1 -- start start 2024-09-19T06:22:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73bf02c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b8100800 0x7f73b819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73bf02c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73b81011b0 0x7f73b819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73bf02c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810e3b0 0x7f73b819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73bf02c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8113f60 con 0x7f73b810e3b0 2024-09-19T06:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73bf02c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8113de0 con 0x7f73b81011b0 2024-09-19T06:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73bf02c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81140e0 con 0x7f73b8100800 2024-09-19T06:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73bcdc8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b8100800 0x7f73b819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73b81011b0 0x7f73b819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73bd5c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810e3b0 0x7f73b819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73bcdc8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b8100800 0x7f73b819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:49862/0 (socket says 172.21.15.5:49862) 2024-09-19T06:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73b81011b0 0x7f73b819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35272/0 (socket says 172.21.15.5:35272) 2024-09-19T06:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.513+0000 7f73bcdc8640 1 -- 172.21.15.5:0/2805013874 learned_addr learned my addr 172.21.15.5:0/2805013874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.514+0000 7f73bd5c9640 1 -- 172.21.15.5:0/2805013874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b8100800 msgr2=0x7f73b819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.514+0000 7f73bd5c9640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b8100800 0x7f73b819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.514+0000 7f73bd5c9640 1 -- 172.21.15.5:0/2805013874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73b81011b0 msgr2=0x7f73b819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.514+0000 7f73bd5c9640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73b81011b0 0x7f73b819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.514+0000 7f73bd5c9640 1 -- 172.21.15.5:0/2805013874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b819f580 con 0x7f73b810e3b0 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.514+0000 7f73b7fff640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73b81011b0 0x7f73b819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.514+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b8100800 0x7f73b819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.514+0000 7f73bd5c9640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810e3b0 0x7f73b819ee50 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f73b81029e0 tx=0x7f73b000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.514+0000 7f73b5ffb640 1 -- 172.21.15.5:0/2805013874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0003b70 con 0x7f73b810e3b0 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.514+0000 7f73b5ffb640 1 -- 172.21.15.5:0/2805013874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73b000ce40 con 0x7f73b810e3b0 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.514+0000 7f73bf02c640 1 -- 172.21.15.5:0/2805013874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b819f870 con 0x7f73b810e3b0 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.514+0000 7f73b5ffb640 1 -- 172.21.15.5:0/2805013874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b00106b0 con 0x7f73b810e3b0 2024-09-19T06:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.515+0000 7f73bf02c640 1 -- 172.21.15.5:0/2805013874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b81a7100 con 0x7f73b810e3b0 2024-09-19T06:22:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.518+0000 7f73b5ffb640 1 -- 172.21.15.5:0/2805013874 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f73b0004020 con 0x7f73b810e3b0 2024-09-19T06:22:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.518+0000 7f73c1dbf740 1 Processor -- start 2024-09-19T06:22:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73c1dbf740 1 -- start start 2024-09-19T06:22:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73c1dbf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f3bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73c1dbf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f46770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73c1dbf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f46cb0 0x5611a4f490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73c1dbf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5611a4ea02f0 con 0x5611a4f3c410 2024-09-19T06:22:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73c1dbf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5611a4e48270 con 0x5611a4f46cb0 2024-09-19T06:22:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73c1dbf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5611a4e9c2b0 con 0x5611a4f3baf0 2024-09-19T06:22:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f3bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73bcdc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f46770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73b5ffb640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73a80a33b0 0x7f73a80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f3bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:49864/0 (socket says 172.21.15.5:49864) 2024-09-19T06:22:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73b7fff640 1 -- 172.21.15.5:0/2982125856 learned_addr learned my addr 172.21.15.5:0/2982125856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73b7fff640 1 --2- 172.21.15.5:0/2982125856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f46cb0 0x5611a4f490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73bd5c9640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73a80a33b0 0x7f73a80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73bcdc8640 1 -- 172.21.15.5:0/2982125856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 msgr2=0x5611a4f3bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/2982125856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f3bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73bcdc8640 1 -- 172.21.15.5:0/2982125856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f46cb0 msgr2=0x5611a4f490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/2982125856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f46cb0 0x5611a4f490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73bcdc8640 1 -- 172.21.15.5:0/2982125856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a4e47340 con 0x5611a4f3c410 2024-09-19T06:22:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.519+0000 7f73b5ffb640 1 -- 172.21.15.5:0/2805013874 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f73b00d0270 con 0x7f73b810e3b0 2024-09-19T06:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.520+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/2982125856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f46770 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0058c0 tx=0x7f73ac0109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a4e064be7f56c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.520+0000 7f73bd5c9640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73a80a33b0 0x7f73a80a5870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f73b00d0620 tx=0x7f73b0009b50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.520+0000 7f73a2ffd640 1 -- 172.21.15.5:0/2982125856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac011820 con 0x5611a4f3c410 2024-09-19T06:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.520+0000 7f73a2ffd640 1 -- 172.21.15.5:0/2982125856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73ac00d340 con 0x5611a4f3c410 2024-09-19T06:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.520+0000 7f73a2ffd640 1 -- 172.21.15.5:0/2982125856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac019590 con 0x5611a4f3c410 2024-09-19T06:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.521+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2982125856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 msgr2=0x5611a4f46770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.521+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/2982125856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f46770 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0058c0 tx=0x7f73ac0109e0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.521+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2982125856 shutdown_connections 2024-09-19T06:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.521+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/2982125856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f46cb0 0x5611a4f490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.521+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/2982125856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f46770 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.521+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/2982125856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f3bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.521+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2982125856 >> 172.21.15.5:0/2982125856 conn(0x5611a4f38060 msgr2=0x5611a4f38440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.521+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2982125856 shutdown_connections 2024-09-19T06:22:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.521+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2982125856 wait complete. 2024-09-19T06:22:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.521+0000 7f73c1dbf740 1 -- start start 2024-09-19T06:22:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73c1dbf740 1 Processor -- start 2024-09-19T06:22:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73c1dbf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3baf0 0x5611a4f43bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73c1dbf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73c1dbf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f46cb0 0x5611a4f44630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73c1dbf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5611a4f49880 con 0x5611a4f3c410 2024-09-19T06:22:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73c1dbf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5611a4e29290 con 0x5611a4f3baf0 2024-09-19T06:22:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73c1dbf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5611a4f49a00 con 0x5611a4f46cb0 2024-09-19T06:22:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73c1dbf740 10 client.0 _ll_register_callbacks cb 0x5611a4e5a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73bcdc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3baf0 0x5611a4f43bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73bcdc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48574/0 (socket says 172.21.15.5:48574) 2024-09-19T06:22:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3baf0 0x5611a4f43bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35304/0 (socket says 172.21.15.5:35304) 2024-09-19T06:22:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73bcdc8640 1 -- 172.21.15.5:0/2830853005 learned_addr learned my addr 172.21.15.5:0/2830853005 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.522+0000 7f73b7fff640 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f46cb0 0x5611a4f44630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.523+0000 7f73bcdc8640 1 -- 172.21.15.5:0/2830853005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f46cb0 msgr2=0x5611a4f44630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.523+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f46cb0 0x5611a4f44630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.523+0000 7f73bcdc8640 1 -- 172.21.15.5:0/2830853005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3baf0 msgr2=0x5611a4f43bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.523+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3baf0 0x5611a4f43bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.523+0000 7f73bcdc8640 1 -- 172.21.15.5:0/2830853005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a4bb3210 con 0x5611a4f3c410 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.523+0000 7f73b7fff640 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3baf0 0x5611a4f43bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.523+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f440f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00dd00 tx=0x7f73ac00d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eccb06e8575e976 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.523+0000 7f7397fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.523+0000 7f7397fff640 1 -- 172.21.15.5:0/2830853005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac00de60 con 0x5611a4f3c410 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.523+0000 7f7397fff640 1 -- 172.21.15.5:0/2830853005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f73ac004260 con 0x5611a4f3c410 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.523+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2830853005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73ac00d340 con 0x5611a4f3c410 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.524+0000 7f7397fff640 1 -- 172.21.15.5:0/2830853005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f73ac00bd20 con 0x5611a4f3c410 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.524+0000 7f73c1dbf740 10 client.20346 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.524+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2830853005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5611a4c7f620 con 0x5611a4f3c410 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.524+0000 7f7397fff640 1 -- 172.21.15.5:0/2830853005 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73ac00d340 con 0x5611a4f3c410 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.524+0000 7f7397fff640 1 -- 172.21.15.5:0/2830853005 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f73ac0047a0 con 0x5611a4f3c410 2024-09-19T06:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.524+0000 7f73c1dbf740 10 client.20346 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.524+0000 7f73c1dbf740 10 client.20346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.524+0000 7f73c1dbf740 20 client.20346 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.524+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5611a4f5f3d0 0x5611a4f617c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.524+0000 7f73c1dbf740 4 client.20346 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.524+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2830853005 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611a4eada20 con 0x5611a4f5f3d0 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.525+0000 7f73b7fff640 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5611a4f5f3d0 0x5611a4f617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.525+0000 7f73b7fff640 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5611a4f5f3d0 0x5611a4f617c0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c05cbff31122bc2 server_cookie=f5d1d353edcee091 in_seq=0 out_seq=0 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.525+0000 7f7397fff640 10 client.20346 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.526+0000 7f7397fff640 1 -- 172.21.15.5:0/2830853005 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611a4eada20 con 0x5611a4f5f3d0 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.526+0000 7f7397fff640 10 client.20346 handle_command_reply: tid=0 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.526+0000 7f73c1dbf740 1 client.20346 shutdown 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.526+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2830853005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 msgr2=0x5611a4f440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.526+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f440f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00dd00 tx=0x7f73ac00d720 comp rx=0 tx=0).stop 2024-09-19T06:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.526+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2830853005 shutdown_connections 2024-09-19T06:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.526+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5611a4f5f3d0 0x5611a4f617c0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.526+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f46cb0 0x5611a4f44630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.526+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f440f0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.526+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/2830853005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3baf0 0x5611a4f43bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.526+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2830853005 >> 172.21.15.5:0/2830853005 conn(0x5611a4f38060 msgr2=0x5611a4f3dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.526+0000 7f7397fff640 0 client.20346 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.527+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2830853005 shutdown_connections 2024-09-19T06:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.527+0000 7f73c1dbf740 1 -- 172.21.15.5:0/2830853005 wait complete. 2024-09-19T06:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.527+0000 7f73c1dbf740 20 client.20346 trim_cache size 0 max 16384 2024-09-19T06:22:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.548+0000 7f73c1dbf740 1 Processor -- start 2024-09-19T06:22:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.548+0000 7f73c1dbf740 1 -- start start 2024-09-19T06:22:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.548+0000 7f73c1dbf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f3e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.548+0000 7f73c1dbf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3c410 0x5611a4f613e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.548+0000 7f73c1dbf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f46cb0 0x5611a4f44460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.548+0000 7f73c1dbf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81ab6e0 con 0x5611a4f46cb0 2024-09-19T06:22:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.548+0000 7f73c1dbf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81ad220 con 0x5611a4f3c410 2024-09-19T06:22:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.548+0000 7f73c1dbf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81abdb0 con 0x5611a4f3baf0 2024-09-19T06:22:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.549+0000 7f73bcdc8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3c410 0x5611a4f613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.549+0000 7f73bcdc8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3c410 0x5611a4f613e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35318/0 (socket says 172.21.15.5:35318) 2024-09-19T06:22:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.549+0000 7f73bcdc8640 1 -- 172.21.15.5:0/3437221125 learned_addr learned my addr 172.21.15.5:0/3437221125 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.549+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f46cb0 0x5611a4f44460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.549+0000 7f73b7fff640 1 --2- 172.21.15.5:0/3437221125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f3e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.549+0000 7f73bcdc8640 1 -- 172.21.15.5:0/3437221125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 msgr2=0x5611a4f3e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.549+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/3437221125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f3e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.549+0000 7f73bcdc8640 1 -- 172.21.15.5:0/3437221125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f46cb0 msgr2=0x5611a4f44460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.549+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/3437221125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f46cb0 0x5611a4f44460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.549+0000 7f73bcdc8640 1 -- 172.21.15.5:0/3437221125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a4f49760 con 0x5611a4f3c410 2024-09-19T06:22:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.549+0000 7f73b7fff640 1 --2- 172.21.15.5:0/3437221125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f46cb0 0x5611a4f44460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.549+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/3437221125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3c410 0x5611a4f613e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f73ac004db0 tx=0x7f73ac00b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d81b594e769e6a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.550+0000 7f73a27fc640 1 -- 172.21.15.5:0/3437221125 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac011620 con 0x5611a4f3c410 2024-09-19T06:22:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.550+0000 7f73a27fc640 1 -- 172.21.15.5:0/3437221125 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73ac00bc70 con 0x5611a4f3c410 2024-09-19T06:22:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.550+0000 7f73a27fc640 1 -- 172.21.15.5:0/3437221125 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac0045e0 con 0x5611a4f3c410 2024-09-19T06:22:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.551+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3437221125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3c410 msgr2=0x5611a4f613e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.551+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/3437221125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3c410 0x5611a4f613e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f73ac004db0 tx=0x7f73ac00b700 comp rx=0 tx=0).stop 2024-09-19T06:22:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.551+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3437221125 shutdown_connections 2024-09-19T06:22:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.551+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/3437221125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f46cb0 0x5611a4f44460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.551+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/3437221125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f3c410 0x5611a4f613e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.551+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/3437221125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f3e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.551+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3437221125 >> 172.21.15.5:0/3437221125 conn(0x5611a4f38060 msgr2=0x5611a4f45c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.551+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3437221125 shutdown_connections 2024-09-19T06:22:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.551+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3437221125 wait complete. 2024-09-19T06:22:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.551+0000 7f73c1dbf740 1 -- start start 2024-09-19T06:22:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73c1dbf740 1 Processor -- start 2024-09-19T06:22:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73c1dbf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f4f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73c1dbf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f4a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73c1dbf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f46cb0 0x5611a4f4a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73c1dbf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81acb50 con 0x5611a4f3c410 2024-09-19T06:22:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73c1dbf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81ac480 con 0x5611a4f46cb0 2024-09-19T06:22:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73c1dbf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81a9700 con 0x5611a4f3baf0 2024-09-19T06:22:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73c1dbf740 10 client.0 _ll_register_callbacks cb 0x7f73a4031520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:22:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73bcdc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f4a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73bcdc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f4a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48584/0 (socket says 172.21.15.5:48584) 2024-09-19T06:22:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f4f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73bcdc8640 1 -- 172.21.15.5:0/3411875407 learned_addr learned my addr 172.21.15.5:0/3411875407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:22:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.552+0000 7f73b7fff640 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f46cb0 0x5611a4f4a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.553+0000 7f73bcdc8640 1 -- 172.21.15.5:0/3411875407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 msgr2=0x5611a4f4f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.553+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f4f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.553+0000 7f73bcdc8640 1 -- 172.21.15.5:0/3411875407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f46cb0 msgr2=0x5611a4f4a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:22:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.553+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f46cb0 0x5611a4f4a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.553+0000 7f73bcdc8640 1 -- 172.21.15.5:0/3411875407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a4f4af80 con 0x5611a4f3c410 2024-09-19T06:22:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.553+0000 7f73b7fff640 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f4f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:22:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.553+0000 7f73bcdc8640 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f4a480 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f73ac010fe0 tx=0x7f73ac004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5726e5e01ea07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:22:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.553+0000 7f7397fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:22:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.553+0000 7f7397fff640 1 -- 172.21.15.5:0/3411875407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac005880 con 0x5611a4f3c410 2024-09-19T06:22:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.553+0000 7f7397fff640 1 -- 172.21.15.5:0/3411875407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7f73ac011620 con 0x5611a4f3c410 2024-09-19T06:22:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.553+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3411875407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73ac00bc70 con 0x5611a4f3c410 2024-09-19T06:22:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.553+0000 7f7397fff640 1 -- 172.21.15.5:0/3411875407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f73ac00d340 con 0x5611a4f3c410 2024-09-19T06:22:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.554+0000 7f73c1dbf740 10 client.20352 fetch_fsmap learned FSMap version 58 2024-09-19T06:22:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.554+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3411875407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5611a4fc9ce0 con 0x5611a4f3c410 2024-09-19T06:22:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.554+0000 7f7397fff640 1 -- 172.21.15.5:0/3411875407 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73ac00bc70 con 0x5611a4f3c410 2024-09-19T06:22:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.554+0000 7f7397fff640 1 -- 172.21.15.5:0/3411875407 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f73ac005190 con 0x5611a4f3c410 2024-09-19T06:22:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.554+0000 7f73c1dbf740 10 client.20352 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:22:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.554+0000 7f73c1dbf740 10 client.20352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:22:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.554+0000 7f73c1dbf740 20 client.20352 populate_metadata read hostname 'smithi005' 2024-09-19T06:22:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.554+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5611a4fca3d0 0x5611a4fcc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:22:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.554+0000 7f73c1dbf740 4 client.20352 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:22:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.554+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3411875407 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611a4ea7d70 con 0x5611a4fca3d0 2024-09-19T06:22:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.554+0000 7f73b7fff640 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5611a4fca3d0 0x5611a4fcc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:22:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.555+0000 7f73b7fff640 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5611a4fca3d0 0x5611a4fcc7f0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f907cdd27a9807ed server_cookie=7794e54d4262b3cf in_seq=0 out_seq=0 2024-09-19T06:22:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.555+0000 7f7397fff640 10 client.20352 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:53.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 62147 8.72 MB/sec execute 257 sec latency 70.639 ms 2024-09-19T06:22:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.663+0000 7f7397fff640 1 -- 172.21.15.5:0/3411875407 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611a4ea7d70 con 0x5611a4fca3d0 2024-09-19T06:22:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.663+0000 7f7397fff640 10 client.20352 handle_command_reply: tid=0 2024-09-19T06:22:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.663+0000 7f73c1dbf740 1 client.20352 shutdown 2024-09-19T06:22:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.663+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3411875407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 msgr2=0x5611a4f4a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.663+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f4a480 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f73ac010fe0 tx=0x7f73ac004980 comp rx=0 tx=0).stop 2024-09-19T06:22:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.663+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3411875407 shutdown_connections 2024-09-19T06:22:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.663+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5611a4fca3d0 0x5611a4fcc7f0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.663+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611a4f46cb0 0x5611a4f4a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.663+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a4f3c410 0x5611a4f4a480 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.663+0000 7f7397fff640 0 client.20352 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:22:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.663+0000 7f73c1dbf740 1 --2- 172.21.15.5:0/3411875407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5611a4f3baf0 0x5611a4f4f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.663+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3411875407 >> 172.21.15.5:0/3411875407 conn(0x5611a4f38060 msgr2=0x5611a4f369a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.664+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3411875407 shutdown_connections 2024-09-19T06:22:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.664+0000 7f73c1dbf740 1 -- 172.21.15.5:0/3411875407 wait complete. 2024-09-19T06:22:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.664+0000 7f73c1dbf740 20 client.20352 trim_cache size 0 max 16384 2024-09-19T06:22:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:22:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:22:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:22:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:22:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.665+0000 7f73bf02c640 1 -- 172.21.15.5:0/2805013874 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73a80a33b0 msgr2=0x7f73a80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.665+0000 7f73bf02c640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73a80a33b0 0x7f73a80a5870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f73b00d0620 tx=0x7f73b0009b50 comp rx=0 tx=0).stop 2024-09-19T06:22:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.665+0000 7f73bf02c640 1 -- 172.21.15.5:0/2805013874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810e3b0 msgr2=0x7f73b819ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:22:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.665+0000 7f73bf02c640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810e3b0 0x7f73b819ee50 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f73b81029e0 tx=0x7f73b000c6a0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.666+0000 7f73bf02c640 1 -- 172.21.15.5:0/2805013874 shutdown_connections 2024-09-19T06:22:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.666+0000 7f73bf02c640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810e3b0 0x7f73b819ee50 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.666+0000 7f73bf02c640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73b81011b0 0x7f73b819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.666+0000 7f73bf02c640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73a80a33b0 0x7f73a80a5870 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.666+0000 7f73bf02c640 1 --2- 172.21.15.5:0/2805013874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b8100800 0x7f73b819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:22:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.666+0000 7f73bf02c640 1 -- 172.21.15.5:0/2805013874 >> 172.21.15.5:0/2805013874 conn(0x7f73b80779a0 msgr2=0x7f73b80ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:22:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.666+0000 7f73bf02c640 1 -- 172.21.15.5:0/2805013874 shutdown_connections 2024-09-19T06:22:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:22:53.666+0000 7f73bf02c640 1 -- 172.21.15.5:0/2805013874 wait complete. 2024-09-19T06:22:54.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:53 smithi157 ceph-mon[35346]: pgmap v616: 97 pgs: 97 active+clean; 1007 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.4 MiB/s wr, 399 op/s 2024-09-19T06:22:54.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:53 smithi005 ceph-mon[30955]: pgmap v616: 97 pgs: 97 active+clean; 1007 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.4 MiB/s wr, 399 op/s 2024-09-19T06:22:54.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:53 smithi049 ceph-mon[34640]: pgmap v616: 97 pgs: 97 active+clean; 1007 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.4 MiB/s wr, 399 op/s 2024-09-19T06:22:54.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 62316 8.72 MB/sec execute 258 sec latency 87.722 ms 2024-09-19T06:22:55.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 62499 8.72 MB/sec execute 259 sec latency 55.696 ms 2024-09-19T06:22:56.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:55 smithi157 ceph-mon[35346]: pgmap v617: 97 pgs: 97 active+clean; 1007 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 263 op/s 2024-09-19T06:22:56.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:55 smithi005 ceph-mon[30955]: pgmap v617: 97 pgs: 97 active+clean; 1007 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 263 op/s 2024-09-19T06:22:56.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:55 smithi049 ceph-mon[34640]: pgmap v617: 97 pgs: 97 active+clean; 1007 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 263 op/s 2024-09-19T06:22:56.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 62667 8.72 MB/sec execute 260 sec latency 111.991 ms 2024-09-19T06:22:57.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 62861 8.74 MB/sec execute 261 sec latency 54.616 ms 2024-09-19T06:22:58.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:57 smithi157 ceph-mon[35346]: pgmap v618: 97 pgs: 97 active+clean; 1012 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 415 op/s 2024-09-19T06:22:58.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:57 smithi005 ceph-mon[30955]: pgmap v618: 97 pgs: 97 active+clean; 1012 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 415 op/s 2024-09-19T06:22:58.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:57 smithi049 ceph-mon[34640]: pgmap v618: 97 pgs: 97 active+clean; 1012 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 415 op/s 2024-09-19T06:22:58.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63070 8.74 MB/sec execute 262 sec latency 57.918 ms 2024-09-19T06:22:59.454 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:22:59.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63241 8.74 MB/sec execute 263 sec latency 64.948 ms 2024-09-19T06:22:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:23:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:22:59 smithi157 ceph-mon[35346]: pgmap v619: 97 pgs: 97 active+clean; 1012 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 295 op/s 2024-09-19T06:23:00.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:22:59 smithi005 ceph-mon[30955]: pgmap v619: 97 pgs: 97 active+clean; 1012 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 295 op/s 2024-09-19T06:23:00.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:22:59 smithi049 ceph-mon[34640]: pgmap v619: 97 pgs: 97 active+clean; 1012 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 295 op/s 2024-09-19T06:23:00.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63367 8.75 MB/sec execute 264 sec latency 171.888 ms 2024-09-19T06:23:01.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63484 8.74 MB/sec execute 265 sec latency 160.474 ms 2024-09-19T06:23:02.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:01 smithi005 ceph-mon[30955]: pgmap v620: 97 pgs: 97 active+clean; 1012 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 295 op/s 2024-09-19T06:23:02.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:01 smithi157 ceph-mon[35346]: pgmap v620: 97 pgs: 97 active+clean; 1012 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 295 op/s 2024-09-19T06:23:02.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:01 smithi049 ceph-mon[34640]: pgmap v620: 97 pgs: 97 active+clean; 1012 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 295 op/s 2024-09-19T06:23:02.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63684 8.78 MB/sec execute 266 sec latency 136.124 ms 2024-09-19T06:23:03.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63845 8.77 MB/sec execute 267 sec latency 150.814 ms 2024-09-19T06:23:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.262+0000 7fd14a923640 1 -- 172.21.15.5:0/2644611107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14410c180 msgr2=0x7fd14410e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.262+0000 7fd14a923640 1 --2- 172.21.15.5:0/2644611107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14410c180 0x7fd14410e620 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009ca0 tx=0x7fd13c02a950 comp rx=0 tx=0).stop 2024-09-19T06:23:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.262+0000 7fd14a923640 1 -- 172.21.15.5:0/2644611107 shutdown_connections 2024-09-19T06:23:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.262+0000 7fd14a923640 1 --2- 172.21.15.5:0/2644611107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14410c180 0x7fd14410e620 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.262+0000 7fd14a923640 1 --2- 172.21.15.5:0/2644611107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd144103670 0x7fd14410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.262+0000 7fd14a923640 1 --2- 172.21.15.5:0/2644611107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd144102d50 0x7fd144103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.262+0000 7fd14a923640 1 -- 172.21.15.5:0/2644611107 >> 172.21.15.5:0/2644611107 conn(0x7fd1440fc7d0 msgr2=0x7fd1440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.262+0000 7fd14a923640 1 -- 172.21.15.5:0/2644611107 shutdown_connections 2024-09-19T06:23:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.262+0000 7fd14a923640 1 -- 172.21.15.5:0/2644611107 wait complete. 2024-09-19T06:23:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.263+0000 7fd14a923640 1 Processor -- start 2024-09-19T06:23:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.263+0000 7fd14a923640 1 -- start start 2024-09-19T06:23:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.264+0000 7fd14a923640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd144102d50 0x7fd144103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.264+0000 7fd14a923640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd144103670 0x7fd1441040e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.264+0000 7fd14a923640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd14410c180 0x7fd144104620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.264+0000 7fd14a923640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd144111df0 con 0x7fd144102d50 2024-09-19T06:23:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.264+0000 7fd14a923640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd144111c70 con 0x7fd14410c180 2024-09-19T06:23:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.264+0000 7fd14a923640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd144111f70 con 0x7fd144103670 2024-09-19T06:23:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.264+0000 7fd143fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd144102d50 0x7fd144103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.264+0000 7fd143fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd144102d50 0x7fd144103ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59578/0 (socket says 172.21.15.5:59578) 2024-09-19T06:23:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.264+0000 7fd143fff640 1 -- 172.21.15.5:0/1064321553 learned_addr learned my addr 172.21.15.5:0/1064321553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.264+0000 7fd148ec0640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd14410c180 0x7fd144104620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.265+0000 7fd143fff640 1 -- 172.21.15.5:0/1064321553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd144103670 msgr2=0x7fd1441040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.265+0000 7fd143fff640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd144103670 0x7fd1441040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.265+0000 7fd143fff640 1 -- 172.21.15.5:0/1064321553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd14410c180 msgr2=0x7fd144104620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.265+0000 7fd143fff640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd14410c180 0x7fd144104620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.265+0000 7fd143fff640 1 -- 172.21.15.5:0/1064321553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1441aabd0 con 0x7fd144102d50 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.265+0000 7fd148ec0640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd14410c180 0x7fd144104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.265+0000 7fd143fff640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd144102d50 0x7fd144103ba0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd13400ea90 tx=0x7fd13400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.265+0000 7fd1417fa640 1 -- 172.21.15.5:0/1064321553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13400ce40 con 0x7fd144102d50 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.265+0000 7fd1417fa640 1 -- 172.21.15.5:0/1064321553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1340035f0 con 0x7fd144102d50 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.265+0000 7fd1417fa640 1 -- 172.21.15.5:0/1064321553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134010690 con 0x7fd144102d50 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.265+0000 7fd14a923640 1 -- 172.21.15.5:0/1064321553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1441aaec0 con 0x7fd144102d50 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.265+0000 7fd14a923640 1 -- 172.21.15.5:0/1064321553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1441ab2c0 con 0x7fd144102d50 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.268+0000 7fd1417fa640 1 -- 172.21.15.5:0/1064321553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd1340037a0 con 0x7fd144102d50 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.269+0000 7fd14ccfc740 1 Processor -- start 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.269+0000 7fd14ccfc740 1 -- start start 2024-09-19T06:23:04.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.269+0000 7fd14ccfc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1b99880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.269+0000 7fd14ccfc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1b99dc0 0x562bb1ba4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.269+0000 7fd14ccfc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba4680 0x562bb1ba6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.269+0000 7fd14ccfc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562bb1aff2e0 con 0x562bb1ba4680 2024-09-19T06:23:04.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.269+0000 7fd14ccfc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562bb1aa6ce0 con 0x562bb1b99dc0 2024-09-19T06:23:04.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.269+0000 7fd14ccfc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562bb1afb2a0 con 0x562bb1b994a0 2024-09-19T06:23:04.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.269+0000 7fd1417fa640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd12c0a3430 0x7fd12c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.269+0000 7fd143fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1b99dc0 0x562bb1ba4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba4680 0x562bb1ba6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba4680 0x562bb1ba6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59584/0 (socket says 172.21.15.5:59584) 2024-09-19T06:23:04.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd143fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1b99dc0 0x562bb1ba4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59234/0 (socket says 172.21.15.5:59234) 2024-09-19T06:23:04.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd1437fe640 1 -- 172.21.15.5:0/2351926396 learned_addr learned my addr 172.21.15.5:0/2351926396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:04.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd1417fa640 1 -- 172.21.15.5:0/1064321553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7fd1340d00c0 con 0x7fd144102d50 2024-09-19T06:23:04.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd148ec0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1b99880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd143fff640 1 -- 172.21.15.5:0/2351926396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 msgr2=0x562bb1b99880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:04.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd143fff640 1 --2- 172.21.15.5:0/2351926396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1b99880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd143fff640 1 -- 172.21.15.5:0/2351926396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba4680 msgr2=0x562bb1ba6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:04.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd143fff640 1 --2- 172.21.15.5:0/2351926396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba4680 0x562bb1ba6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd143fff640 1 -- 172.21.15.5:0/2351926396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb1a8b440 con 0x562bb1b99dc0 2024-09-19T06:23:04.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd148ec0640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd12c0a3430 0x7fd12c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd1437fe640 1 --2- 172.21.15.5:0/2351926396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba4680 0x562bb1ba6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.270+0000 7fd143fff640 1 --2- 172.21.15.5:0/2351926396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1b99dc0 0x562bb1ba4140 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd1340cd7f0 tx=0x7fd13406f1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d31cd450c715d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.271+0000 7fd12a7fc640 1 -- 172.21.15.5:0/2351926396 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1340671d0 con 0x562bb1b99dc0 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.271+0000 7fd12a7fc640 1 -- 172.21.15.5:0/2351926396 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd13409ca80 con 0x562bb1b99dc0 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.271+0000 7fd12a7fc640 1 -- 172.21.15.5:0/2351926396 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1340cdc00 con 0x562bb1b99dc0 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.271+0000 7fd148ec0640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd12c0a3430 0x7fd12c0a58f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009430 tx=0x7fd13c02a950 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.271+0000 7fd14ccfc740 1 -- 172.21.15.5:0/2351926396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1b99dc0 msgr2=0x562bb1ba4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.271+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/2351926396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1b99dc0 0x562bb1ba4140 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd1340cd7f0 tx=0x7fd13406f1b0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.272+0000 7fd14ccfc740 1 -- 172.21.15.5:0/2351926396 shutdown_connections 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.272+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/2351926396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba4680 0x562bb1ba6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.272+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/2351926396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1b99dc0 0x562bb1ba4140 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.272+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/2351926396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1b99880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.272+0000 7fd14ccfc740 1 -- 172.21.15.5:0/2351926396 >> 172.21.15.5:0/2351926396 conn(0x562bb1b95a30 msgr2=0x562bb1b95e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.272+0000 7fd14ccfc740 1 -- 172.21.15.5:0/2351926396 shutdown_connections 2024-09-19T06:23:04.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.272+0000 7fd14ccfc740 1 -- 172.21.15.5:0/2351926396 wait complete. 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.272+0000 7fd14ccfc740 1 -- start start 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.272+0000 7fd14ccfc740 1 Processor -- start 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd14ccfc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1ba3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd14ccfc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1b99dc0 0x562bb1ba1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd14ccfc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1ba1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd14ccfc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562bb1ba7250 con 0x562bb1b99dc0 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd14ccfc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562bb1aba050 con 0x562bb1ba4680 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd14ccfc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562bb1ba73d0 con 0x562bb1b994a0 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd14ccfc740 10 client.0 _ll_register_callbacks cb 0x562bb1afa160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1b99dc0 0x562bb1ba1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1b99dc0 0x562bb1ba1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59590/0 (socket says 172.21.15.5:59590) 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd1437fe640 1 -- 172.21.15.5:0/1920931597 learned_addr learned my addr 172.21.15.5:0/1920931597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:04.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd143fff640 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1ba3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd148ec0640 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1ba1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd1437fe640 1 -- 172.21.15.5:0/1920931597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 msgr2=0x562bb1ba3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd1437fe640 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1ba3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd1437fe640 1 -- 172.21.15.5:0/1920931597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 msgr2=0x562bb1ba1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd1437fe640 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1ba1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd1437fe640 1 -- 172.21.15.5:0/1920931597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb18703c0 con 0x562bb1b99dc0 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.273+0000 7fd148ec0640 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1ba1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.274+0000 7fd1437fe640 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1b99dc0 0x562bb1ba1200 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd138004620 tx=0x7fd138011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d7c16782563780 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.274+0000 7fd1237fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.274+0000 7fd1237fe640 1 -- 172.21.15.5:0/1920931597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13801e060 con 0x562bb1b99dc0 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.274+0000 7fd1237fe640 1 -- 172.21.15.5:0/1920931597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7fd13800fd30 con 0x562bb1b99dc0 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.274+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1920931597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd13409ca80 con 0x562bb1b99dc0 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.274+0000 7fd1237fe640 1 -- 172.21.15.5:0/1920931597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fd138022430 con 0x562bb1b99dc0 2024-09-19T06:23:04.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.274+0000 7fd14ccfc740 10 client.20364 fetch_fsmap learned FSMap version 58 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.274+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1920931597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562bb1bb2c00 con 0x562bb1b99dc0 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.275+0000 7fd1237fe640 1 -- 172.21.15.5:0/1920931597 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd13409ca80 con 0x562bb1b99dc0 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.275+0000 7fd1237fe640 1 -- 172.21.15.5:0/1920931597 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fd1380214e0 con 0x562bb1b99dc0 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.275+0000 7fd14ccfc740 10 client.20364 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.275+0000 7fd14ccfc740 10 client.20364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.275+0000 7fd14ccfc740 20 client.20364 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.275+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bb1bb40c0 0x562bb1bb6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.275+0000 7fd14ccfc740 4 client.20364 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.275+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1920931597 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bb1b0c850 con 0x562bb1bb40c0 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.275+0000 7fd143fff640 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bb1bb40c0 0x562bb1bb6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.276+0000 7fd143fff640 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bb1bb40c0 0x562bb1bb6500 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=838e14d0bbbb0841 server_cookie=6dd36523ebde614e in_seq=0 out_seq=0 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.276+0000 7fd1237fe640 10 client.20364 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:04.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.277+0000 7fd1237fe640 1 -- 172.21.15.5:0/1920931597 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562bb1b0c850 con 0x562bb1bb40c0 2024-09-19T06:23:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.277+0000 7fd1237fe640 10 client.20364 handle_command_reply: tid=0 2024-09-19T06:23:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.277+0000 7fd14ccfc740 1 client.20364 shutdown 2024-09-19T06:23:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.277+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1920931597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1b99dc0 msgr2=0x562bb1ba1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.277+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1b99dc0 0x562bb1ba1200 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd138004620 tx=0x7fd138011b50 comp rx=0 tx=0).stop 2024-09-19T06:23:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.277+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1920931597 shutdown_connections 2024-09-19T06:23:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.277+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bb1bb40c0 0x562bb1bb6500 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.277+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1ba1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.277+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1b99dc0 0x562bb1ba1200 secure :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd138004620 tx=0x7fd138011b50 comp rx=0 tx=0).stop 2024-09-19T06:23:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.277+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/1920931597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1ba3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.277+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1920931597 >> 172.21.15.5:0/1920931597 conn(0x562bb1b95a30 msgr2=0x562bb1b9b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.277+0000 7fd1237fe640 0 client.20364 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.278+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1920931597 shutdown_connections 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.278+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1920931597 wait complete. 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.278+0000 7fd14ccfc740 20 client.20364 trim_cache size 0 max 16384 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.311+0000 7fd14ccfc740 1 Processor -- start 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.311+0000 7fd14ccfc740 1 -- start start 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd14ccfc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1bb04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd14ccfc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1b9c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd14ccfc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 0x562bb1bbbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd14ccfc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1441b3fb0 con 0x562bb1ba3170 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd14ccfc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1441b5af0 con 0x562bb1ba4680 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd14ccfc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1441b4680 con 0x562bb1b994a0 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd148ec0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 0x562bb1bbbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd148ec0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 0x562bb1bbbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59596/0 (socket says 172.21.15.5:59596) 2024-09-19T06:23:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1b9c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd148ec0640 1 -- 172.21.15.5:0/1476030405 learned_addr learned my addr 172.21.15.5:0/1476030405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd143fff640 1 --2- 172.21.15.5:0/1476030405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1bb04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd1437fe640 1 -- 172.21.15.5:0/1476030405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 msgr2=0x562bb1bb04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd1437fe640 1 --2- 172.21.15.5:0/1476030405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1bb04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd1437fe640 1 -- 172.21.15.5:0/1476030405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 msgr2=0x562bb1bbbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd1437fe640 1 --2- 172.21.15.5:0/1476030405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 0x562bb1bbbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd1437fe640 1 -- 172.21.15.5:0/1476030405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb1ba01b0 con 0x562bb1ba4680 2024-09-19T06:23:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.312+0000 7fd148ec0640 1 --2- 172.21.15.5:0/1476030405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 0x562bb1bbbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.313+0000 7fd1437fe640 1 --2- 172.21.15.5:0/1476030405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1b9c900 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x562bb1ba22f0 tx=0x7fd138009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b214b6d413a29e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.313+0000 7fd129ffb640 1 -- 172.21.15.5:0/1476030405 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138021b70 con 0x562bb1ba4680 2024-09-19T06:23:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.313+0000 7fd129ffb640 1 -- 172.21.15.5:0/1476030405 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd138021d10 con 0x562bb1ba4680 2024-09-19T06:23:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.313+0000 7fd129ffb640 1 -- 172.21.15.5:0/1476030405 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138005480 con 0x562bb1ba4680 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.314+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1476030405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 msgr2=0x562bb1b9c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.314+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/1476030405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1b9c900 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x562bb1ba22f0 tx=0x7fd138009d00 comp rx=0 tx=0).stop 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.314+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1476030405 shutdown_connections 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.314+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/1476030405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 0x562bb1bbbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.314+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/1476030405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1b9c900 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.314+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/1476030405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1bb04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.314+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1476030405 >> 172.21.15.5:0/1476030405 conn(0x562bb1b95a30 msgr2=0x562bb1ba2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.314+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1476030405 shutdown_connections 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.314+0000 7fd14ccfc740 1 -- 172.21.15.5:0/1476030405 wait complete. 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.315+0000 7fd14ccfc740 1 -- start start 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.315+0000 7fd14ccfc740 1 Processor -- start 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.315+0000 7fd14ccfc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1ba89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.315+0000 7fd14ccfc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 0x562bb1ba8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.315+0000 7fd14ccfc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1c1dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.315+0000 7fd14ccfc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1441b5420 con 0x562bb1ba3170 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.315+0000 7fd14ccfc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1441b4d50 con 0x562bb1ba4680 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.315+0000 7fd14ccfc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1441b1fb0 con 0x562bb1b994a0 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd14ccfc740 10 client.0 _ll_register_callbacks cb 0x7fd138009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd143fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1ba89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 0x562bb1ba8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd143fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1ba89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:35616/0 (socket says 172.21.15.5:35616) 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd143fff640 1 -- 172.21.15.5:0/3018525675 learned_addr learned my addr 172.21.15.5:0/3018525675 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd148ec0640 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1c1dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd143fff640 1 -- 172.21.15.5:0/3018525675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 msgr2=0x562bb1c1dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd143fff640 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1c1dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd143fff640 1 -- 172.21.15.5:0/3018525675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 msgr2=0x562bb1ba8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd143fff640 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 0x562bb1ba8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd143fff640 1 -- 172.21.15.5:0/3018525675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb1ba42c0 con 0x562bb1b994a0 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd148ec0640 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1c1dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd1437fe640 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 0x562bb1ba8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd143fff640 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1ba89e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd134098d50 tx=0x7fd134003c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c965db13d68fad35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd1237fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.316+0000 7fd1237fe640 1 -- 172.21.15.5:0/3018525675 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13409ca80 con 0x562bb1b994a0 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd14ccfc740 1 -- 172.21.15.5:0/3018525675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd138021d10 con 0x562bb1b994a0 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd1237fe640 1 -- 172.21.15.5:0/3018525675 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8550+0+0 (secure 0 0 0) 0x7fd1340cda70 con 0x562bb1b994a0 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd1237fe640 1 -- 172.21.15.5:0/3018525675 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fd1340a0730 con 0x562bb1b994a0 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd14ccfc740 10 client.29683 fetch_fsmap learned FSMap version 58 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd14ccfc740 1 -- 172.21.15.5:0/3018525675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562bb1c1f390 con 0x562bb1b994a0 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd1237fe640 1 -- 172.21.15.5:0/3018525675 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd138021d10 con 0x562bb1b994a0 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd1237fe640 1 -- 172.21.15.5:0/3018525675 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fd1340a4b60 con 0x562bb1b994a0 2024-09-19T06:23:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd14ccfc740 10 client.29683 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:23:04.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd14ccfc740 10 client.29683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:04.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd14ccfc740 20 client.29683 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:04.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bb1c1f680 0x562bb1c21a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:04.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd14ccfc740 4 client.29683 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:23:04.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.317+0000 7fd14ccfc740 1 -- 172.21.15.5:0/3018525675 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562bb1b06d60 con 0x562bb1c1f680 2024-09-19T06:23:04.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.318+0000 7fd1437fe640 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bb1c1f680 0x562bb1c21a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:04.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.318+0000 7fd1437fe640 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bb1c1f680 0x562bb1c21a70 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7596e12032d275c2 server_cookie=71a8400573670eeb in_seq=0 out_seq=0 2024-09-19T06:23:04.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.318+0000 7fd1237fe640 10 client.29683 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:04.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.318+0000 7fd1237fe640 1 -- 172.21.15.5:0/3018525675 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562bb1b06d60 con 0x562bb1c1f680 2024-09-19T06:23:04.097 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:23:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:23:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:23:04.098 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:23:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.318+0000 7fd1237fe640 10 client.29683 handle_command_reply: tid=0 2024-09-19T06:23:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.318+0000 7fd14ccfc740 1 client.29683 shutdown 2024-09-19T06:23:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.319+0000 7fd14ccfc740 1 -- 172.21.15.5:0/3018525675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 msgr2=0x562bb1ba89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.319+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1ba89e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd134098d50 tx=0x7fd134003c90 comp rx=0 tx=0).stop 2024-09-19T06:23:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.319+0000 7fd14ccfc740 1 -- 172.21.15.5:0/3018525675 shutdown_connections 2024-09-19T06:23:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.319+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x562bb1c1f680 0x562bb1c21a70 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.319+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bb1ba4680 0x562bb1c1dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.319+0000 7fd1237fe640 0 client.29683 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.319+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb1ba3170 0x562bb1ba8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.319+0000 7fd14ccfc740 1 --2- 172.21.15.5:0/3018525675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bb1b994a0 0x562bb1ba89e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.319+0000 7fd14ccfc740 1 -- 172.21.15.5:0/3018525675 >> 172.21.15.5:0/3018525675 conn(0x562bb1b95a30 msgr2=0x562bb1b94b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.319+0000 7fd14ccfc740 1 -- 172.21.15.5:0/3018525675 shutdown_connections 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.319+0000 7fd14ccfc740 1 -- 172.21.15.5:0/3018525675 wait complete. 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.319+0000 7fd14ccfc740 20 client.29683 trim_cache size 0 max 16384 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.320+0000 7fd14a923640 1 -- 172.21.15.5:0/1064321553 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd12c0a3430 msgr2=0x7fd12c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.320+0000 7fd14a923640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd12c0a3430 0x7fd12c0a58f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009430 tx=0x7fd13c02a950 comp rx=0 tx=0).stop 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.320+0000 7fd14a923640 1 -- 172.21.15.5:0/1064321553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd144102d50 msgr2=0x7fd144103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.320+0000 7fd14a923640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd144102d50 0x7fd144103ba0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd13400ea90 tx=0x7fd13400ef50 comp rx=0 tx=0).stop 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.320+0000 7fd14a923640 1 -- 172.21.15.5:0/1064321553 shutdown_connections 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.320+0000 7fd14a923640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd14410c180 0x7fd144104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.320+0000 7fd14a923640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd12c0a3430 0x7fd12c0a58f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.320+0000 7fd14a923640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd144103670 0x7fd1441040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.320+0000 7fd14a923640 1 --2- 172.21.15.5:0/1064321553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd144102d50 0x7fd144103ba0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.320+0000 7fd14a923640 1 -- 172.21.15.5:0/1064321553 >> 172.21.15.5:0/1064321553 conn(0x7fd1440fc7d0 msgr2=0x7fd144101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.320+0000 7fd14a923640 1 -- 172.21.15.5:0/1064321553 shutdown_connections 2024-09-19T06:23:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:03.320+0000 7fd14a923640 1 -- 172.21.15.5:0/1064321553 wait complete. 2024-09-19T06:23:04.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:03 smithi157 ceph-mon[35346]: pgmap v621: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.9 MiB/s wr, 421 op/s 2024-09-19T06:23:04.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:03 smithi005 ceph-mon[30955]: pgmap v621: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.9 MiB/s wr, 421 op/s 2024-09-19T06:23:04.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:03 smithi049 ceph-mon[34640]: pgmap v621: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.9 MiB/s wr, 421 op/s 2024-09-19T06:23:04.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63983 8.77 MB/sec execute 268 sec latency 71.462 ms 2024-09-19T06:23:05.156 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:23:05.156 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:23:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:23:05.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 64102 8.76 MB/sec execute 269 sec latency 134.026 ms 2024-09-19T06:23:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.090+0000 7f63ef31b640 1 -- 172.21.15.5:0/2761446701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63e8101de0 msgr2=0x7f63e81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.090+0000 7f63ef31b640 1 --2- 172.21.15.5:0/2761446701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63e8101de0 0x7f63e81021c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009900 tx=0x7f63e402a3f0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.090+0000 7f63ef31b640 1 -- 172.21.15.5:0/2761446701 shutdown_connections 2024-09-19T06:23:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.090+0000 7f63ef31b640 1 --2- 172.21.15.5:0/2761446701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63e8106f10 0x7f63e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.090+0000 7f63ef31b640 1 --2- 172.21.15.5:0/2761446701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e8102790 0x7f63e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.090+0000 7f63ef31b640 1 --2- 172.21.15.5:0/2761446701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63e8101de0 0x7f63e81021c0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.090+0000 7f63ef31b640 1 -- 172.21.15.5:0/2761446701 >> 172.21.15.5:0/2761446701 conn(0x7f63e80fd8e0 msgr2=0x7f63e80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.090+0000 7f63ef31b640 1 -- 172.21.15.5:0/2761446701 shutdown_connections 2024-09-19T06:23:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.090+0000 7f63ef31b640 1 -- 172.21.15.5:0/2761446701 wait complete. 2024-09-19T06:23:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.091+0000 7f63ef31b640 1 Processor -- start 2024-09-19T06:23:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.091+0000 7f63ef31b640 1 -- start start 2024-09-19T06:23:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ef31b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63e8101de0 0x7f63e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ef31b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63e8102790 0x7f63e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ef31b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e8106f10 0x7f63e8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ef31b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63e8111e20 con 0x7f63e8101de0 2024-09-19T06:23:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ef31b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63e8111ca0 con 0x7f63e8106f10 2024-09-19T06:23:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ef31b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f63e8111fa0 con 0x7f63e8102790 2024-09-19T06:23:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ed8b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e8106f10 0x7f63e8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ed8b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e8106f10 0x7f63e8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55820/0 (socket says 172.21.15.5:55820) 2024-09-19T06:23:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ed8b8640 1 -- 172.21.15.5:0/1490304586 learned_addr learned my addr 172.21.15.5:0/1490304586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ec8b6640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63e8102790 0x7f63e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ed8b8640 1 -- 172.21.15.5:0/1490304586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63e8102790 msgr2=0x7f63e8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ed8b8640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63e8102790 0x7f63e8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ed8b8640 1 -- 172.21.15.5:0/1490304586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63e8101de0 msgr2=0x7f63e80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.092+0000 7f63ed8b8640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63e8101de0 0x7f63e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.093+0000 7f63ed8b8640 1 -- 172.21.15.5:0/1490304586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e81aad90 con 0x7f63e8106f10 2024-09-19T06:23:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.093+0000 7f63ed8b8640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e8106f10 0x7f63e8074f70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f63e000eaa0 tx=0x7f63e000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.093+0000 7f63da7fc640 1 -- 172.21.15.5:0/1490304586 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e000cdb0 con 0x7f63e8106f10 2024-09-19T06:23:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.093+0000 7f63da7fc640 1 -- 172.21.15.5:0/1490304586 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63e0005020 con 0x7f63e8106f10 2024-09-19T06:23:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.093+0000 7f63da7fc640 1 -- 172.21.15.5:0/1490304586 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e00116d0 con 0x7f63e8106f10 2024-09-19T06:23:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.094+0000 7f63ef31b640 1 -- 172.21.15.5:0/1490304586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e81aafc0 con 0x7f63e8106f10 2024-09-19T06:23:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.094+0000 7f63ef31b640 1 -- 172.21.15.5:0/1490304586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e81ab450 con 0x7f63e8106f10 2024-09-19T06:23:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.095+0000 7f63ef31b640 1 -- 172.21.15.5:0/1490304586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e81032c0 con 0x7f63e8106f10 2024-09-19T06:23:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.096+0000 7f63da7fc640 1 -- 172.21.15.5:0/1490304586 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f63e00051d0 con 0x7f63e8106f10 2024-09-19T06:23:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.097+0000 7f63da7fc640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63d40a3430 0x7f63d40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.097+0000 7f63da7fc640 1 -- 172.21.15.5:0/1490304586 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8594+0+0 (secure 0 0 0) 0x7f63e00cfeb0 con 0x7f63e8106f10 2024-09-19T06:23:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.097+0000 7f63ed0b7640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63d40a3430 0x7f63d40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.098+0000 7f63ed0b7640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63d40a3430 0x7f63d40a58f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f63e40096c0 tx=0x7f63e4031400 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.101+0000 7f63da7fc640 1 -- 172.21.15.5:0/1490304586 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f63e0016030 con 0x7f63e8106f10 2024-09-19T06:23:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:05 smithi157 ceph-mon[35346]: pgmap v622: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 278 op/s 2024-09-19T06:23:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:05 smithi157 ceph-mon[35346]: osdmap e112: 12 total, 12 up, 12 in 2024-09-19T06:23:06.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:05 smithi005 ceph-mon[30955]: pgmap v622: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 278 op/s 2024-09-19T06:23:06.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:05 smithi005 ceph-mon[30955]: osdmap e112: 12 total, 12 up, 12 in 2024-09-19T06:23:06.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:05 smithi049 ceph-mon[34640]: pgmap v622: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 278 op/s 2024-09-19T06:23:06.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:05 smithi049 ceph-mon[34640]: osdmap e112: 12 total, 12 up, 12 in 2024-09-19T06:23:06.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.458+0000 7f63ef31b640 1 -- 172.21.15.5:0/1490304586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63e810a120 con 0x7f63e8106f10 2024-09-19T06:23:06.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.458+0000 7f63da7fc640 1 -- 172.21.15.5:0/1490304586 <== mon.2 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11959 (secure 0 0 0) 0x7f63e009c840 con 0x7f63e8106f10 2024-09-19T06:23:06.461 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:23:06.461 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":58,"btime":"2024-09-19T06:20:33:468419+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:20:32.963303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:23:06.462 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 58 2024-09-19T06:23:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.461+0000 7f63ef31b640 1 -- 172.21.15.5:0/1490304586 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63d40a3430 msgr2=0x7f63d40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.461+0000 7f63ef31b640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63d40a3430 0x7f63d40a58f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f63e40096c0 tx=0x7f63e4031400 comp rx=0 tx=0).stop 2024-09-19T06:23:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.461+0000 7f63ef31b640 1 -- 172.21.15.5:0/1490304586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e8106f10 msgr2=0x7f63e8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.461+0000 7f63ef31b640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e8106f10 0x7f63e8074f70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f63e000eaa0 tx=0x7f63e000ef60 comp rx=0 tx=0).stop 2024-09-19T06:23:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.461+0000 7f63ef31b640 1 -- 172.21.15.5:0/1490304586 shutdown_connections 2024-09-19T06:23:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.461+0000 7f63ef31b640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e8106f10 0x7f63e8074f70 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.461+0000 7f63ef31b640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f63d40a3430 0x7f63d40a58f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.461+0000 7f63ef31b640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63e8102790 0x7f63e8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.461+0000 7f63ef31b640 1 --2- 172.21.15.5:0/1490304586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63e8101de0 0x7f63e80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.461+0000 7f63ef31b640 1 -- 172.21.15.5:0/1490304586 >> 172.21.15.5:0/1490304586 conn(0x7f63e80fd8e0 msgr2=0x7f63e80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.462+0000 7f63ef31b640 1 -- 172.21.15.5:0/1490304586 shutdown_connections 2024-09-19T06:23:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:06.462+0000 7f63ef31b640 1 -- 172.21.15.5:0/1490304586 wait complete. 2024-09-19T06:23:06.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 64237 8.76 MB/sec execute 270 sec latency 76.739 ms 2024-09-19T06:23:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.784+0000 7f96e9ca5640 1 -- 172.21.15.5:0/2749670067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e4101ae0 msgr2=0x7f96e4101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.784+0000 7f96e9ca5640 1 --2- 172.21.15.5:0/2749670067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e4101ae0 0x7f96e4101f40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009900 tx=0x7f96d802a410 comp rx=0 tx=0).stop 2024-09-19T06:23:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.784+0000 7f96e9ca5640 1 -- 172.21.15.5:0/2749670067 shutdown_connections 2024-09-19T06:23:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.784+0000 7f96e9ca5640 1 --2- 172.21.15.5:0/2749670067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4102480 0x7f96e410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.784+0000 7f96e9ca5640 1 --2- 172.21.15.5:0/2749670067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e4101ae0 0x7f96e4101f40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.784+0000 7f96e9ca5640 1 --2- 172.21.15.5:0/2749670067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96e4107b80 0x7f96e4107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.784+0000 7f96e9ca5640 1 -- 172.21.15.5:0/2749670067 >> 172.21.15.5:0/2749670067 conn(0x7f96e40fd7e0 msgr2=0x7f96e40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.784+0000 7f96e9ca5640 1 -- 172.21.15.5:0/2749670067 shutdown_connections 2024-09-19T06:23:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.784+0000 7f96e9ca5640 1 -- 172.21.15.5:0/2749670067 wait complete. 2024-09-19T06:23:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.784+0000 7f96e9ca5640 1 Processor -- start 2024-09-19T06:23:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.785+0000 7f96e9ca5640 1 -- start start 2024-09-19T06:23:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.785+0000 7f96e9ca5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96e4101ae0 0x7f96e410bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.785+0000 7f96e9ca5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4102480 0x7f96e410c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.785+0000 7f96e9ca5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e4107b80 0x7f96e410c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.785+0000 7f96e9ca5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96e4076120 con 0x7f96e4101ae0 2024-09-19T06:23:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.785+0000 7f96e9ca5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f96e4075fa0 con 0x7f96e4107b80 2024-09-19T06:23:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.785+0000 7f96e9ca5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96e40762a0 con 0x7f96e4102480 2024-09-19T06:23:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.785+0000 7f96e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96e4101ae0 0x7f96e410bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.785+0000 7f96e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96e4101ae0 0x7f96e410bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57474/0 (socket says 172.21.15.5:57474) 2024-09-19T06:23:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.785+0000 7f96e37fe640 1 -- 172.21.15.5:0/1227273536 learned_addr learned my addr 172.21.15.5:0/1227273536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.785+0000 7f96e3fff640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e4107b80 0x7f96e410c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.785+0000 7f96e2ffd640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4102480 0x7f96e410c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e3fff640 1 -- 172.21.15.5:0/1227273536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4102480 msgr2=0x7f96e410c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e3fff640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4102480 0x7f96e410c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e3fff640 1 -- 172.21.15.5:0/1227273536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96e4101ae0 msgr2=0x7f96e410bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e3fff640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96e4101ae0 0x7f96e410bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e3fff640 1 -- 172.21.15.5:0/1227273536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e41aac20 con 0x7f96e4107b80 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e2ffd640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4102480 0x7f96e410c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e37fe640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96e4101ae0 0x7f96e410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e3fff640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e4107b80 0x7f96e410c770 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00da10 tx=0x7f96dc00ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e0ff9640 1 -- 172.21.15.5:0/1227273536 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc00bd70 con 0x7f96e4107b80 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e0ff9640 1 -- 172.21.15.5:0/1227273536 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96dc003e20 con 0x7f96e4107b80 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e0ff9640 1 -- 172.21.15.5:0/1227273536 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0106c0 con 0x7f96e4107b80 2024-09-19T06:23:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e9ca5640 1 -- 172.21.15.5:0/1227273536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e41aaf10 con 0x7f96e4107b80 2024-09-19T06:23:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.786+0000 7f96e9ca5640 1 -- 172.21.15.5:0/1227273536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e41ab3e0 con 0x7f96e4107b80 2024-09-19T06:23:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.789+0000 7f96eca46740 1 Processor -- start 2024-09-19T06:23:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.790+0000 7f96e0ff9640 1 -- 172.21.15.5:0/1227273536 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f96dc003970 con 0x7f96e4107b80 2024-09-19T06:23:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.790+0000 7f96eca46740 1 -- start start 2024-09-19T06:23:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.790+0000 7f96eca46740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 0x561f115f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.790+0000 7f96eca46740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 0x561f115fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.790+0000 7f96eca46740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 0x561f116010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.790+0000 7f96eca46740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561f115582f0 con 0x561f115f3af0 2024-09-19T06:23:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.790+0000 7f96eca46740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561f11500270 con 0x561f115f4410 2024-09-19T06:23:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.790+0000 7f96eca46740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561f115542b0 con 0x561f115fecb0 2024-09-19T06:23:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.790+0000 7f96e37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 0x561f115fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.790+0000 7f96e2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 0x561f115f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.790+0000 7f96e37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 0x561f115fe770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55842/0 (socket says 172.21.15.5:55842) 2024-09-19T06:23:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.790+0000 7f96e37fe640 1 -- 172.21.15.5:0/1467542272 learned_addr learned my addr 172.21.15.5:0/1467542272 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.791+0000 7f96e37fe640 1 -- 172.21.15.5:0/1467542272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 msgr2=0x561f116010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.791+0000 7f96e37fe640 1 --2- 172.21.15.5:0/1467542272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 0x561f116010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.791+0000 7f96e37fe640 1 -- 172.21.15.5:0/1467542272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 msgr2=0x561f115f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.791+0000 7f96e37fe640 1 --2- 172.21.15.5:0/1467542272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 0x561f115f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.791+0000 7f96e37fe640 1 -- 172.21.15.5:0/1467542272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f114ff340 con 0x561f115f4410 2024-09-19T06:23:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.791+0000 7f96e2ffd640 1 --2- 172.21.15.5:0/1467542272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 0x561f115f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.791+0000 7f96e0ff9640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f96cc0a3430 0x7f96cc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.791+0000 7f96e37fe640 1 --2- 172.21.15.5:0/1467542272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 0x561f115fe770 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f96d40115c0 tx=0x7f96d4011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a3d3ed2ccdcfeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.791+0000 7f96e2ffd640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f96cc0a3430 0x7f96cc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.791+0000 7f96e0ff9640 1 -- 172.21.15.5:0/1227273536 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8594+0+0 (secure 0 0 0) 0x7f96dc0d0050 con 0x7f96e4107b80 2024-09-19T06:23:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.792+0000 7f96c9ffb640 1 -- 172.21.15.5:0/1467542272 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d400f660 con 0x561f115f4410 2024-09-19T06:23:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.792+0000 7f96c9ffb640 1 -- 172.21.15.5:0/1467542272 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96d400fcb0 con 0x561f115f4410 2024-09-19T06:23:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.792+0000 7f96c9ffb640 1 -- 172.21.15.5:0/1467542272 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4019d40 con 0x561f115f4410 2024-09-19T06:23:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.793+0000 7f96eca46740 1 -- 172.21.15.5:0/1467542272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 msgr2=0x561f115fe770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.793+0000 7f96eca46740 1 --2- 172.21.15.5:0/1467542272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 0x561f115fe770 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f96d40115c0 tx=0x7f96d4011a80 comp rx=0 tx=0).stop 2024-09-19T06:23:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.793+0000 7f96e2ffd640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f96cc0a3430 0x7f96cc0a58f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f96d8000c00 tx=0x7f96d8005b90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.793+0000 7f96eca46740 1 -- 172.21.15.5:0/1467542272 shutdown_connections 2024-09-19T06:23:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.793+0000 7f96eca46740 1 --2- 172.21.15.5:0/1467542272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 0x561f116010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.793+0000 7f96eca46740 1 --2- 172.21.15.5:0/1467542272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 0x561f115fe770 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.793+0000 7f96eca46740 1 --2- 172.21.15.5:0/1467542272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 0x561f115f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.793+0000 7f96eca46740 1 -- 172.21.15.5:0/1467542272 >> 172.21.15.5:0/1467542272 conn(0x561f115f0060 msgr2=0x561f115f0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.793+0000 7f96eca46740 1 -- 172.21.15.5:0/1467542272 shutdown_connections 2024-09-19T06:23:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.793+0000 7f96eca46740 1 -- 172.21.15.5:0/1467542272 wait complete. 2024-09-19T06:23:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.793+0000 7f96eca46740 1 -- start start 2024-09-19T06:23:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96eca46740 1 Processor -- start 2024-09-19T06:23:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96eca46740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 0x561f115fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96eca46740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 0x561f115fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96eca46740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 0x561f115fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96eca46740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561f11601880 con 0x561f115f3af0 2024-09-19T06:23:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96eca46740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561f114e1290 con 0x561f115f4410 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96eca46740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561f11601a00 con 0x561f115fecb0 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 0x561f115fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96eca46740 10 client.0 _ll_register_callbacks cb 0x561f11512140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 0x561f115fbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57490/0 (socket says 172.21.15.5:57490) 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96e37fe640 1 -- 172.21.15.5:0/3775663769 learned_addr learned my addr 172.21.15.5:0/3775663769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96e3fff640 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 0x561f115fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.794+0000 7f96e2ffd640 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 0x561f115fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.795+0000 7f96e2ffd640 1 -- 172.21.15.5:0/3775663769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 msgr2=0x561f115fc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.795+0000 7f96e2ffd640 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 0x561f115fc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.795+0000 7f96e2ffd640 1 -- 172.21.15.5:0/3775663769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 msgr2=0x561f115fbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.795+0000 7f96e2ffd640 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 0x561f115fbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.795+0000 7f96e2ffd640 1 -- 172.21.15.5:0/3775663769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f1126b210 con 0x561f115f4410 2024-09-19T06:23:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.795+0000 7f96e37fe640 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 0x561f115fbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.795+0000 7f96e2ffd640 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 0x561f115fc0c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f96d8005940 tx=0x7f96d80058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1952fc7ac930482f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.795+0000 7f96c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.795+0000 7f96c2ffd640 1 -- 172.21.15.5:0/3775663769 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d804e020 con 0x561f115f4410 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.795+0000 7f96c2ffd640 1 -- 172.21.15.5:0/3775663769 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8594+0+0 (secure 0 0 0) 0x7f96d8004240 con 0x561f115f4410 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.795+0000 7f96eca46740 1 -- 172.21.15.5:0/3775663769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d400fcb0 con 0x561f115f4410 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.796+0000 7f96c2ffd640 1 -- 172.21.15.5:0/3775663769 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f96d8059050 con 0x561f115f4410 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.796+0000 7f96eca46740 10 client.29756 fetch_fsmap learned FSMap version 58 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.796+0000 7f96eca46740 1 -- 172.21.15.5:0/3775663769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561f11337620 con 0x561f115f4410 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.796+0000 7f96c2ffd640 1 -- 172.21.15.5:0/3775663769 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96d400fcb0 con 0x561f115f4410 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.796+0000 7f96c2ffd640 1 -- 172.21.15.5:0/3775663769 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f96d803a040 con 0x561f115f4410 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.796+0000 7f96eca46740 10 client.29756 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.796+0000 7f96eca46740 10 client.29756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.796+0000 7f96eca46740 20 client.29756 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.796+0000 7f96eca46740 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561f1160ebf0 0x561f11611040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.796+0000 7f96eca46740 4 client.29756 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.796+0000 7f96eca46740 1 -- 172.21.15.5:0/3775663769 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f11565a20 con 0x561f1160ebf0 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.796+0000 7f96e37fe640 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561f1160ebf0 0x561f11611040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.797+0000 7f96c2ffd640 10 client.29756 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.797+0000 7f96e37fe640 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561f1160ebf0 0x561f11611040 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2392b584db4bed96 server_cookie=5b4bc36c28e8671c in_seq=0 out_seq=0 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.798+0000 7f96c2ffd640 1 -- 172.21.15.5:0/3775663769 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f11565a20 con 0x561f1160ebf0 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.798+0000 7f96c2ffd640 10 client.29756 handle_command_reply: tid=0 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.798+0000 7f96eca46740 1 client.29756 shutdown 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.798+0000 7f96eca46740 1 -- 172.21.15.5:0/3775663769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 msgr2=0x561f115fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.798+0000 7f96eca46740 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 0x561f115fc0c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f96d8005940 tx=0x7f96d80058d0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.798+0000 7f96eca46740 1 -- 172.21.15.5:0/3775663769 shutdown_connections 2024-09-19T06:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.798+0000 7f96eca46740 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561f1160ebf0 0x561f11611040 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.798+0000 7f96eca46740 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 0x561f115fc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.798+0000 7f96eca46740 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f4410 0x561f115fc0c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.798+0000 7f96c2ffd640 0 client.29756 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.798+0000 7f96eca46740 1 --2- 172.21.15.5:0/3775663769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f3af0 0x561f115fbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.798+0000 7f96eca46740 1 -- 172.21.15.5:0/3775663769 >> 172.21.15.5:0/3775663769 conn(0x561f115f0060 msgr2=0x561f115f5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.799+0000 7f96eca46740 1 -- 172.21.15.5:0/3775663769 shutdown_connections 2024-09-19T06:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.799+0000 7f96eca46740 1 -- 172.21.15.5:0/3775663769 wait complete. 2024-09-19T06:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.799+0000 7f96eca46740 20 client.29756 trim_cache size 0 max 16384 2024-09-19T06:23:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96eca46740 1 Processor -- start 2024-09-19T06:23:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96eca46740 1 -- start start 2024-09-19T06:23:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96eca46740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 0x561f115fc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96eca46740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f4410 0x561f11616950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96eca46740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 0x561f11616e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96eca46740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41b3f80 con 0x561f115f4410 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96eca46740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41b5ac0 con 0x561f115f3af0 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96eca46740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41b4650 con 0x561f115fecb0 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96e2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f4410 0x561f11616950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96e37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 0x561f115fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96e2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f4410 0x561f11616950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57496/0 (socket says 172.21.15.5:57496) 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96e2ffd640 1 -- 172.21.15.5:0/2880374034 learned_addr learned my addr 172.21.15.5:0/2880374034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.824+0000 7f96e3fff640 1 --2- 172.21.15.5:0/2880374034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 0x561f11616e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.825+0000 7f96e37fe640 1 -- 172.21.15.5:0/2880374034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 msgr2=0x561f11616e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.825+0000 7f96e37fe640 1 --2- 172.21.15.5:0/2880374034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 0x561f11616e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.825+0000 7f96e37fe640 1 -- 172.21.15.5:0/2880374034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f4410 msgr2=0x561f11616950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.825+0000 7f96e37fe640 1 --2- 172.21.15.5:0/2880374034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f4410 0x561f11616950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.825+0000 7f96e37fe640 1 -- 172.21.15.5:0/2880374034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f115efe90 con 0x561f115f3af0 2024-09-19T06:23:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.825+0000 7f96e2ffd640 1 --2- 172.21.15.5:0/2880374034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f4410 0x561f11616950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.825+0000 7f96e37fe640 1 --2- 172.21.15.5:0/2880374034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 0x561f115fc430 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009ee0 tx=0x7f96d4019990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4583852865294376 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.825+0000 7f96c97fa640 1 -- 172.21.15.5:0/2880374034 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4019750 con 0x561f115f3af0 2024-09-19T06:23:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.825+0000 7f96c97fa640 1 -- 172.21.15.5:0/2880374034 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96d40054e0 con 0x561f115f3af0 2024-09-19T06:23:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.825+0000 7f96c97fa640 1 -- 172.21.15.5:0/2880374034 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d400fb00 con 0x561f115f3af0 2024-09-19T06:23:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.826+0000 7f96eca46740 1 -- 172.21.15.5:0/2880374034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 msgr2=0x561f115fc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.826+0000 7f96eca46740 1 --2- 172.21.15.5:0/2880374034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 0x561f115fc430 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009ee0 tx=0x7f96d4019990 comp rx=0 tx=0).stop 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.826+0000 7f96eca46740 1 -- 172.21.15.5:0/2880374034 shutdown_connections 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.826+0000 7f96eca46740 1 --2- 172.21.15.5:0/2880374034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115fecb0 0x561f11616e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.826+0000 7f96eca46740 1 --2- 172.21.15.5:0/2880374034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115f4410 0x561f11616950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.826+0000 7f96eca46740 1 --2- 172.21.15.5:0/2880374034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 0x561f115fc430 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.826+0000 7f96eca46740 1 -- 172.21.15.5:0/2880374034 >> 172.21.15.5:0/2880374034 conn(0x561f115f0060 msgr2=0x561f115fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.826+0000 7f96eca46740 1 -- 172.21.15.5:0/2880374034 shutdown_connections 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.826+0000 7f96eca46740 1 -- 172.21.15.5:0/2880374034 wait complete. 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.827+0000 7f96eca46740 1 -- start start 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.827+0000 7f96eca46740 1 Processor -- start 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.827+0000 7f96eca46740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 0x561f11607010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.827+0000 7f96eca46740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115f4410 0x561f11607550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.827+0000 7f96eca46740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115fecb0 0x561f115f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 0x561f11607010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 0x561f11607010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55868/0 (socket says 172.21.15.5:55868) 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115fecb0 0x561f115f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115f4410 0x561f11607550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115fecb0 0x561f115f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57506/0 (socket says 172.21.15.5:57506) 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115f4410 0x561f11607550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56128/0 (socket says 172.21.15.5:56128) 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96eca46740 1 -- 172.21.15.5:0/749867479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41b53f0 con 0x561f115fecb0 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e37fe640 1 -- 172.21.15.5:0/749867479 learned_addr learned my addr 172.21.15.5:0/749867479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96eca46740 1 -- 172.21.15.5:0/749867479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41b4d20 con 0x561f115f3af0 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96eca46740 1 -- 172.21.15.5:0/749867479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41b1f80 con 0x561f115f4410 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96eca46740 10 client.0 _ll_register_callbacks cb 0x7f96d8041050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e37fe640 1 -- 172.21.15.5:0/749867479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115f4410 msgr2=0x561f11607550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e37fe640 1 --2- 172.21.15.5:0/749867479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115f4410 0x561f11607550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e37fe640 1 -- 172.21.15.5:0/749867479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115fecb0 msgr2=0x561f115f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e37fe640 1 --2- 172.21.15.5:0/749867479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115fecb0 0x561f115f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e37fe640 1 -- 172.21.15.5:0/749867479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f115f8f90 con 0x561f115f3af0 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e3fff640 1 --2- 172.21.15.5:0/749867479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115fecb0 0x561f115f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96e37fe640 1 --2- 172.21.15.5:0/749867479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 0x561f11607010 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f96d400f510 tx=0x7f96d4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c850e47b4c505a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.828+0000 7f96c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.829+0000 7f96c2ffd640 1 -- 172.21.15.5:0/749867479 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4035060 con 0x561f115f3af0 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.829+0000 7f96c2ffd640 1 -- 172.21.15.5:0/749867479 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8594+0+0 (secure 0 0 0) 0x7f96d4005030 con 0x561f115f3af0 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.829+0000 7f96eca46740 1 -- 172.21.15.5:0/749867479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d40054e0 con 0x561f115f3af0 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.829+0000 7f96c2ffd640 1 -- 172.21.15.5:0/749867479 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f96d4019410 con 0x561f115f3af0 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.829+0000 7f96eca46740 10 client.29762 fetch_fsmap learned FSMap version 58 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.829+0000 7f96eca46740 1 -- 172.21.15.5:0/749867479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561f116795c0 con 0x561f115f3af0 2024-09-19T06:23:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.829+0000 7f96c2ffd640 1 -- 172.21.15.5:0/749867479 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96d4005230 con 0x561f115f3af0 2024-09-19T06:23:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.829+0000 7f96c2ffd640 1 -- 172.21.15.5:0/749867479 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f96d402d430 con 0x561f115f3af0 2024-09-19T06:23:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.830+0000 7f96eca46740 10 client.29762 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:23:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.830+0000 7f96eca46740 10 client.29762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.830+0000 7f96eca46740 20 client.29762 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.830+0000 7f96eca46740 1 --2- 172.21.15.5:0/749867479 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561f1160ebf0 0x561f1167b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.830+0000 7f96eca46740 4 client.29762 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:23:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.830+0000 7f96eca46740 1 -- 172.21.15.5:0/749867479 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f96d40054e0 con 0x561f1160ebf0 2024-09-19T06:23:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.830+0000 7f96e2ffd640 1 --2- 172.21.15.5:0/749867479 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561f1160ebf0 0x561f1167b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.830+0000 7f96e2ffd640 1 --2- 172.21.15.5:0/749867479 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561f1160ebf0 0x561f1167b9f0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef027a67db5e615d server_cookie=f3803ad0812d7dad in_seq=0 out_seq=0 2024-09-19T06:23:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.830+0000 7f96c2ffd640 10 client.29762 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96c2ffd640 1 -- 172.21.15.5:0/749867479 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f96d40054e0 con 0x561f1160ebf0 2024-09-19T06:23:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96c2ffd640 10 client.29762 handle_command_reply: tid=0 2024-09-19T06:23:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96eca46740 1 client.29762 shutdown 2024-09-19T06:23:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96eca46740 1 -- 172.21.15.5:0/749867479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 msgr2=0x561f11607010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96eca46740 1 --2- 172.21.15.5:0/749867479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 0x561f11607010 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f96d400f510 tx=0x7f96d4000bc0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96eca46740 1 -- 172.21.15.5:0/749867479 shutdown_connections 2024-09-19T06:23:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96eca46740 1 --2- 172.21.15.5:0/749867479 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561f1160ebf0 0x561f1167b9f0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96eca46740 1 --2- 172.21.15.5:0/749867479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f115fecb0 0x561f115f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96eca46740 1 --2- 172.21.15.5:0/749867479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f115f4410 0x561f11607550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96eca46740 1 --2- 172.21.15.5:0/749867479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f115f3af0 0x561f11607010 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96c2ffd640 0 client.29762 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96eca46740 1 -- 172.21.15.5:0/749867479 >> 172.21.15.5:0/749867479 conn(0x561f115f0060 msgr2=0x561f115ee9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96eca46740 1 -- 172.21.15.5:0/749867479 shutdown_connections 2024-09-19T06:23:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96eca46740 1 -- 172.21.15.5:0/749867479 wait complete. 2024-09-19T06:23:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.927+0000 7f96eca46740 20 client.29762 trim_cache size 0 max 16384 2024-09-19T06:23:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:23:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:23:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:23:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:23:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.928+0000 7f96e9ca5640 1 -- 172.21.15.5:0/1227273536 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f96cc0a3430 msgr2=0x7f96cc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.928+0000 7f96e9ca5640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f96cc0a3430 0x7f96cc0a58f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f96d8000c00 tx=0x7f96d8005b90 comp rx=0 tx=0).stop 2024-09-19T06:23:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.928+0000 7f96e9ca5640 1 -- 172.21.15.5:0/1227273536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e4107b80 msgr2=0x7f96e410c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.928+0000 7f96e9ca5640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e4107b80 0x7f96e410c770 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00da10 tx=0x7f96dc00ded0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.928+0000 7f96e9ca5640 1 -- 172.21.15.5:0/1227273536 shutdown_connections 2024-09-19T06:23:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.928+0000 7f96e9ca5640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e4107b80 0x7f96e410c770 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.928+0000 7f96e9ca5640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f96cc0a3430 0x7f96cc0a58f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.928+0000 7f96e9ca5640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4102480 0x7f96e410c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.928+0000 7f96e9ca5640 1 --2- 172.21.15.5:0/1227273536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96e4101ae0 0x7f96e410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.928+0000 7f96e9ca5640 1 -- 172.21.15.5:0/1227273536 >> 172.21.15.5:0/1227273536 conn(0x7f96e40fd7e0 msgr2=0x7f96e40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.928+0000 7f96e9ca5640 1 -- 172.21.15.5:0/1227273536 shutdown_connections 2024-09-19T06:23:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:06.929+0000 7f96e9ca5640 1 -- 172.21.15.5:0/1227273536 wait complete. 2024-09-19T06:23:07.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:06 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1490304586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:23:07.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:06 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1490304586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:23:07.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:06 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1490304586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:23:07.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 64394 8.76 MB/sec execute 271 sec latency 111.057 ms 2024-09-19T06:23:07.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:23:08.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:07 smithi005 ceph-mon[30955]: pgmap v624: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 267 op/s 2024-09-19T06:23:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:23:08.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:07 smithi157 ceph-mon[35346]: pgmap v624: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 267 op/s 2024-09-19T06:23:08.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:07 smithi049 ceph-mon[34640]: pgmap v624: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 267 op/s 2024-09-19T06:23:08.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 64548 8.76 MB/sec execute 272 sec latency 137.775 ms 2024-09-19T06:23:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.750+0000 7f1437a9b640 1 -- 172.21.15.5:0/1884669504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430102790 msgr2=0x7f14301067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.750+0000 7f1437a9b640 1 --2- 172.21.15.5:0/1884669504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430102790 0x7f14301067e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f142002a410 comp rx=0 tx=0).stop 2024-09-19T06:23:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.750+0000 7f1437a9b640 1 -- 172.21.15.5:0/1884669504 shutdown_connections 2024-09-19T06:23:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.750+0000 7f1437a9b640 1 --2- 172.21.15.5:0/1884669504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1430106f10 0x7f143010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.750+0000 7f1437a9b640 1 --2- 172.21.15.5:0/1884669504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430102790 0x7f14301067e0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.750+0000 7f1437a9b640 1 --2- 172.21.15.5:0/1884669504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1430101de0 0x7f14301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.750+0000 7f1437a9b640 1 -- 172.21.15.5:0/1884669504 >> 172.21.15.5:0/1884669504 conn(0x7f14300fd8e0 msgr2=0x7f14300ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.750+0000 7f1437a9b640 1 -- 172.21.15.5:0/1884669504 shutdown_connections 2024-09-19T06:23:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.750+0000 7f1437a9b640 1 -- 172.21.15.5:0/1884669504 wait complete. 2024-09-19T06:23:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.751+0000 7f1437a9b640 1 Processor -- start 2024-09-19T06:23:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.751+0000 7f1437a9b640 1 -- start start 2024-09-19T06:23:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.751+0000 7f1437a9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1430101de0 0x7f1430198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.751+0000 7f1437a9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1430102790 0x7f1430198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.751+0000 7f1437a9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430106f10 0x7f143019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.751+0000 7f1437a9b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1430111e70 con 0x7f1430106f10 2024-09-19T06:23:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.751+0000 7f1436038640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430106f10 0x7f143019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.751+0000 7f1436038640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430106f10 0x7f143019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57528/0 (socket says 172.21.15.5:57528) 2024-09-19T06:23:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.751+0000 7f1436038640 1 -- 172.21.15.5:0/33304762 learned_addr learned my addr 172.21.15.5:0/33304762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.751+0000 7f1437a9b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1430111cf0 con 0x7f1430102790 2024-09-19T06:23:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.751+0000 7f1437a9b640 1 -- 172.21.15.5:0/33304762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1430111ff0 con 0x7f1430101de0 2024-09-19T06:23:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.751+0000 7f1435036640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1430102790 0x7f1430198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.752+0000 7f1436038640 1 -- 172.21.15.5:0/33304762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1430101de0 msgr2=0x7f1430198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.752+0000 7f1436038640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1430101de0 0x7f1430198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.752+0000 7f1436038640 1 -- 172.21.15.5:0/33304762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1430102790 msgr2=0x7f1430198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.752+0000 7f1436038640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1430102790 0x7f1430198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.752+0000 7f1436038640 1 -- 172.21.15.5:0/33304762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143019d3c0 con 0x7f1430106f10 2024-09-19T06:23:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.752+0000 7f1436038640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430106f10 0x7f143019cce0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f142800eb10 tx=0x7f142800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.752+0000 7f1426ffd640 1 -- 172.21.15.5:0/33304762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428004b20 con 0x7f1430106f10 2024-09-19T06:23:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.752+0000 7f1426ffd640 1 -- 172.21.15.5:0/33304762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1428004cc0 con 0x7f1430106f10 2024-09-19T06:23:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.752+0000 7f1437a9b640 1 -- 172.21.15.5:0/33304762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143019d6b0 con 0x7f1430106f10 2024-09-19T06:23:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.752+0000 7f1437a9b640 1 -- 172.21.15.5:0/33304762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14301a4f90 con 0x7f1430106f10 2024-09-19T06:23:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.753+0000 7f1426ffd640 1 -- 172.21.15.5:0/33304762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428010690 con 0x7f1430106f10 2024-09-19T06:23:08.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.756+0000 7f1426ffd640 1 -- 172.21.15.5:0/33304762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1428005020 con 0x7f1430106f10 2024-09-19T06:23:08.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.756+0000 7f1439e74740 1 Processor -- start 2024-09-19T06:23:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.756+0000 7f1439e74740 1 -- start start 2024-09-19T06:23:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1439e74740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f7155d0 0x56055f7159b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1439e74740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f7106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1439e74740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f710c00 0x56055f71c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1439e74740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56055f66c2f0 con 0x56055f710280 2024-09-19T06:23:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1439e74740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56055f613cf0 con 0x56055f710c00 2024-09-19T06:23:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1439e74740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56055f6682b0 con 0x56055f7155d0 2024-09-19T06:23:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1435036640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f7155d0 0x56055f7159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1435036640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f7155d0 0x56055f7159b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56156/0 (socket says 172.21.15.5:56156) 2024-09-19T06:23:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1435036640 1 -- 172.21.15.5:0/1661368149 learned_addr learned my addr 172.21.15.5:0/1661368149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:08.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1435837640 1 --2- 172.21.15.5:0/1661368149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f7106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1435036640 1 --2- 172.21.15.5:0/1661368149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f710c00 0x56055f71c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1435036640 1 -- 172.21.15.5:0/1661368149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f710c00 msgr2=0x56055f71c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1435036640 1 --2- 172.21.15.5:0/1661368149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f710c00 0x56055f71c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1435036640 1 -- 172.21.15.5:0/1661368149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 msgr2=0x56055f7106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1426ffd640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f141c0a3430 0x7f141c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1435036640 1 --2- 172.21.15.5:0/1661368149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f7106c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1435036640 1 -- 172.21.15.5:0/1661368149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055f5f83a0 con 0x56055f7155d0 2024-09-19T06:23:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.757+0000 7f1435837640 1 --2- 172.21.15.5:0/1661368149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f7106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.758+0000 7f1426ffd640 1 -- 172.21.15.5:0/33304762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f14280d01f0 con 0x7f1430106f10 2024-09-19T06:23:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.758+0000 7f1435036640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f141c0a3430 0x7f141c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.758+0000 7f1435036640 1 --2- 172.21.15.5:0/1661368149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f7155d0 0x56055f7159b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f1420005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c7321bb438b21f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.758+0000 7f141bfff640 1 -- 172.21.15.5:0/1661368149 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420035030 con 0x56055f7155d0 2024-09-19T06:23:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.758+0000 7f141bfff640 1 -- 172.21.15.5:0/1661368149 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14200314a0 con 0x56055f7155d0 2024-09-19T06:23:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.759+0000 7f141bfff640 1 -- 172.21.15.5:0/1661368149 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142003d650 con 0x56055f7155d0 2024-09-19T06:23:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.759+0000 7f1439e74740 1 -- 172.21.15.5:0/1661368149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f7155d0 msgr2=0x56055f7159b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.759+0000 7f1435036640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f141c0a3430 0x7f141c0a58f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1420004200 tx=0x7f1420049000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.759+0000 7f1439e74740 1 --2- 172.21.15.5:0/1661368149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f7155d0 0x56055f7159b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f1420005780 comp rx=0 tx=0).stop 2024-09-19T06:23:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.759+0000 7f1439e74740 1 -- 172.21.15.5:0/1661368149 shutdown_connections 2024-09-19T06:23:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.759+0000 7f1439e74740 1 --2- 172.21.15.5:0/1661368149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f710c00 0x56055f71c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.759+0000 7f1439e74740 1 --2- 172.21.15.5:0/1661368149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f7106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.759+0000 7f1439e74740 1 --2- 172.21.15.5:0/1661368149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f7155d0 0x56055f7159b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.759+0000 7f1439e74740 1 -- 172.21.15.5:0/1661368149 >> 172.21.15.5:0/1661368149 conn(0x56055f6cf310 msgr2=0x56055f7026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.759+0000 7f1439e74740 1 -- 172.21.15.5:0/1661368149 shutdown_connections 2024-09-19T06:23:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.760+0000 7f1439e74740 1 -- 172.21.15.5:0/1661368149 wait complete. 2024-09-19T06:23:08.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.760+0000 7f1439e74740 1 -- start start 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.760+0000 7f1439e74740 1 Processor -- start 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.761+0000 7f1439e74740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710280 0x56055f71c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.761+0000 7f1439e74740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710c00 0x56055f719300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.761+0000 7f1439e74740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f7155d0 0x56055f719840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.761+0000 7f1439e74740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56055f71c990 con 0x56055f710c00 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.761+0000 7f1439e74740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56055f627060 con 0x56055f7155d0 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.761+0000 7f1439e74740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56055f71cb10 con 0x56055f710280 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.761+0000 7f1439e74740 10 client.0 _ll_register_callbacks cb 0x56055f37a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.761+0000 7f1435036640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710c00 0x56055f719300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.761+0000 7f1435036640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710c00 0x56055f719300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57548/0 (socket says 172.21.15.5:57548) 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.761+0000 7f1435036640 1 -- 172.21.15.5:0/162759016 learned_addr learned my addr 172.21.15.5:0/162759016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.761+0000 7f1436038640 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f7155d0 0x56055f719840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.762+0000 7f1435837640 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710280 0x56055f71c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.762+0000 7f1435036640 1 -- 172.21.15.5:0/162759016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710280 msgr2=0x56055f71c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.762+0000 7f1435036640 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710280 0x56055f71c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.762+0000 7f1435036640 1 -- 172.21.15.5:0/162759016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f7155d0 msgr2=0x56055f719840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.762+0000 7f1435036640 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f7155d0 0x56055f719840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.762+0000 7f1435036640 1 -- 172.21.15.5:0/162759016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055f3dd3c0 con 0x56055f710c00 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.762+0000 7f1435837640 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710280 0x56055f71c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.762+0000 7f1435036640 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710c00 0x56055f719300 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f1420003c80 tx=0x7f1420003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d0039b3dfc1627 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.762+0000 7f14197fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.762+0000 7f14197fa640 1 -- 172.21.15.5:0/162759016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420035070 con 0x56055f710c00 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.762+0000 7f14197fa640 1 -- 172.21.15.5:0/162759016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f1420031860 con 0x56055f710c00 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.762+0000 7f1439e74740 1 -- 172.21.15.5:0/162759016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14200314a0 con 0x56055f710c00 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.763+0000 7f14197fa640 1 -- 172.21.15.5:0/162759016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1420064050 con 0x56055f710c00 2024-09-19T06:23:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.763+0000 7f1439e74740 10 client.20403 fetch_fsmap learned FSMap version 58 2024-09-19T06:23:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.763+0000 7f1439e74740 1 -- 172.21.15.5:0/162759016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56055f71c990 con 0x56055f710c00 2024-09-19T06:23:08.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.763+0000 7f14197fa640 1 -- 172.21.15.5:0/162759016 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14200314a0 con 0x56055f710c00 2024-09-19T06:23:08.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.763+0000 7f14197fa640 1 -- 172.21.15.5:0/162759016 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f142005f030 con 0x56055f710c00 2024-09-19T06:23:08.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.764+0000 7f1439e74740 10 client.20403 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:23:08.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.764+0000 7f1439e74740 10 client.20403 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:23:08.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.764+0000 7f1439e74740 20 client.20403 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:08.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.764+0000 7f1439e74740 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56055f729750 0x56055f72bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.764+0000 7f1439e74740 4 client.20403 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.764+0000 7f1439e74740 1 -- 172.21.15.5:0/162759016 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56055f679860 con 0x56055f729750 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.764+0000 7f1435837640 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56055f729750 0x56055f72bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.765+0000 7f1435837640 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56055f729750 0x56055f72bbb0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=64c525b8834c0071 server_cookie=589864a0178c57b9 in_seq=0 out_seq=0 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.765+0000 7f14197fa640 10 client.20403 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.765+0000 7f14197fa640 1 -- 172.21.15.5:0/162759016 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56055f679860 con 0x56055f729750 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.765+0000 7f14197fa640 10 client.20403 handle_command_reply: tid=0 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.765+0000 7f1439e74740 1 client.20403 shutdown 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.766+0000 7f1439e74740 1 -- 172.21.15.5:0/162759016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710c00 msgr2=0x56055f719300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.766+0000 7f1439e74740 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710c00 0x56055f719300 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f1420003c80 tx=0x7f1420003ed0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.766+0000 7f1439e74740 1 -- 172.21.15.5:0/162759016 shutdown_connections 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.766+0000 7f1439e74740 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56055f729750 0x56055f72bbb0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.766+0000 7f1439e74740 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f7155d0 0x56055f719840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.766+0000 7f1439e74740 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710c00 0x56055f719300 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.766+0000 7f1439e74740 1 --2- 172.21.15.5:0/162759016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710280 0x56055f71c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.766+0000 7f1439e74740 1 -- 172.21.15.5:0/162759016 >> 172.21.15.5:0/162759016 conn(0x56055f6cf310 msgr2=0x56055f70fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.766+0000 7f14197fa640 0 client.20403 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:23:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.766+0000 7f1439e74740 1 -- 172.21.15.5:0/162759016 shutdown_connections 2024-09-19T06:23:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.766+0000 7f1439e74740 1 -- 172.21.15.5:0/162759016 wait complete. 2024-09-19T06:23:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.766+0000 7f1439e74740 20 client.20403 trim_cache size 0 max 16384 2024-09-19T06:23:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.786+0000 7f1439e74740 1 Processor -- start 2024-09-19T06:23:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.786+0000 7f1439e74740 1 -- start start 2024-09-19T06:23:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.786+0000 7f1439e74740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f714d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.786+0000 7f1439e74740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f710c00 0x56055f731480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.786+0000 7f1439e74740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f7155d0 0x56055f7319c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.786+0000 7f1439e74740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f141400e7e0 con 0x56055f710280 2024-09-19T06:23:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.786+0000 7f1439e74740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1414010320 con 0x56055f710c00 2024-09-19T06:23:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.786+0000 7f1439e74740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f141400eeb0 con 0x56055f7155d0 2024-09-19T06:23:08.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.786+0000 7f1435837640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f714d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.786+0000 7f1435837640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f714d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57558/0 (socket says 172.21.15.5:57558) 2024-09-19T06:23:08.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.786+0000 7f1435837640 1 -- 172.21.15.5:0/4084422385 learned_addr learned my addr 172.21.15.5:0/4084422385 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:08.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.787+0000 7f1435036640 1 --2- 172.21.15.5:0/4084422385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f710c00 0x56055f731480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.787+0000 7f1436038640 1 --2- 172.21.15.5:0/4084422385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f7155d0 0x56055f7319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.787+0000 7f1435036640 1 -- 172.21.15.5:0/4084422385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f7155d0 msgr2=0x56055f7319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.787+0000 7f1435036640 1 --2- 172.21.15.5:0/4084422385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f7155d0 0x56055f7319c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.787+0000 7f1435036640 1 -- 172.21.15.5:0/4084422385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 msgr2=0x56055f714d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.787+0000 7f1435036640 1 --2- 172.21.15.5:0/4084422385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f714d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.787+0000 7f1435036640 1 -- 172.21.15.5:0/4084422385 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055f706580 con 0x56055f710c00 2024-09-19T06:23:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.787+0000 7f1435036640 1 --2- 172.21.15.5:0/4084422385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f710c00 0x56055f731480 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f142002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8590280b6053484 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.787+0000 7f141b7fe640 1 -- 172.21.15.5:0/4084422385 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142006c020 con 0x56055f710c00 2024-09-19T06:23:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.788+0000 7f141b7fe640 1 -- 172.21.15.5:0/4084422385 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14200314a0 con 0x56055f710c00 2024-09-19T06:23:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.788+0000 7f141b7fe640 1 -- 172.21.15.5:0/4084422385 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420000e60 con 0x56055f710c00 2024-09-19T06:23:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.788+0000 7f1439e74740 1 -- 172.21.15.5:0/4084422385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f710c00 msgr2=0x56055f731480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.788+0000 7f1439e74740 1 --2- 172.21.15.5:0/4084422385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f710c00 0x56055f731480 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f142002a410 comp rx=0 tx=0).stop 2024-09-19T06:23:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.789+0000 7f1439e74740 1 -- 172.21.15.5:0/4084422385 shutdown_connections 2024-09-19T06:23:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.789+0000 7f1439e74740 1 --2- 172.21.15.5:0/4084422385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f7155d0 0x56055f7319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.789+0000 7f1439e74740 1 --2- 172.21.15.5:0/4084422385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f710c00 0x56055f731480 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.789+0000 7f1439e74740 1 --2- 172.21.15.5:0/4084422385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f714d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.789+0000 7f1439e74740 1 -- 172.21.15.5:0/4084422385 >> 172.21.15.5:0/4084422385 conn(0x56055f6cf310 msgr2=0x56055f714660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.789+0000 7f1439e74740 1 -- 172.21.15.5:0/4084422385 shutdown_connections 2024-09-19T06:23:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.789+0000 7f1439e74740 1 -- 172.21.15.5:0/4084422385 wait complete. 2024-09-19T06:23:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.789+0000 7f1439e74740 1 -- start start 2024-09-19T06:23:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.789+0000 7f1439e74740 1 Processor -- start 2024-09-19T06:23:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1439e74740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f721e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1439e74740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710c00 0x56055f722380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1439e74740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f7155d0 0x56055f730630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1439e74740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f141400fc50 con 0x56055f710280 2024-09-19T06:23:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1439e74740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f141400f580 con 0x56055f7155d0 2024-09-19T06:23:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1439e74740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f141400c7e0 con 0x56055f710c00 2024-09-19T06:23:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1435036640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710c00 0x56055f722380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1435036640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710c00 0x56055f722380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56174/0 (socket says 172.21.15.5:56174) 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1435036640 1 -- 172.21.15.5:0/1856394852 learned_addr learned my addr 172.21.15.5:0/1856394852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1435837640 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f721e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1439e74740 10 client.0 _ll_register_callbacks cb 0x7f142c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1436038640 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f7155d0 0x56055f730630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1435036640 1 -- 172.21.15.5:0/1856394852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f7155d0 msgr2=0x56055f730630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1435036640 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f7155d0 0x56055f730630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1435036640 1 -- 172.21.15.5:0/1856394852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 msgr2=0x56055f721e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1435036640 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f721e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.790+0000 7f1435036640 1 -- 172.21.15.5:0/1856394852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055f730c70 con 0x56055f710c00 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.791+0000 7f1435036640 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710c00 0x56055f722380 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1420003c50 tx=0x7f142003efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83a648f7a56a10a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.791+0000 7f14197fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.791+0000 7f14197fa640 1 -- 172.21.15.5:0/1856394852 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142006c040 con 0x56055f710c00 2024-09-19T06:23:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.791+0000 7f14197fa640 1 -- 172.21.15.5:0/1856394852 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f1420031bd0 con 0x56055f710c00 2024-09-19T06:23:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.791+0000 7f1439e74740 1 -- 172.21.15.5:0/1856394852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14200314a0 con 0x56055f710c00 2024-09-19T06:23:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.791+0000 7f14197fa640 1 -- 172.21.15.5:0/1856394852 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1420080050 con 0x56055f710c00 2024-09-19T06:23:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.791+0000 7f1439e74740 10 client.29719 fetch_fsmap learned FSMap version 58 2024-09-19T06:23:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.791+0000 7f1439e74740 1 -- 172.21.15.5:0/1856394852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56055f793d50 con 0x56055f710c00 2024-09-19T06:23:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.792+0000 7f14197fa640 1 -- 172.21.15.5:0/1856394852 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1420009520 con 0x56055f710c00 2024-09-19T06:23:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.792+0000 7f14197fa640 1 -- 172.21.15.5:0/1856394852 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f142005f070 con 0x56055f710c00 2024-09-19T06:23:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.792+0000 7f1439e74740 10 client.29719 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:23:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.792+0000 7f1439e74740 10 client.29719 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:23:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.792+0000 7f1439e74740 20 client.29719 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.792+0000 7f1439e74740 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56055f794380 0x56055f7967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.792+0000 7f1439e74740 4 client.29719 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:23:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.792+0000 7f1439e74740 1 -- 172.21.15.5:0/1856394852 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f14200314a0 con 0x56055f794380 2024-09-19T06:23:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.792+0000 7f1435837640 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56055f794380 0x56055f7967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f14197fa640 10 client.29719 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:23:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f1435837640 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56055f794380 0x56055f7967a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=932cb17969bedc21 server_cookie=e788470da9c43208 in_seq=0 out_seq=0 2024-09-19T06:23:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f14197fa640 1 -- 172.21.15.5:0/1856394852 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f14200314a0 con 0x56055f794380 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f14197fa640 10 client.29719 handle_command_reply: tid=0 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f1439e74740 1 client.29719 shutdown 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f1439e74740 1 -- 172.21.15.5:0/1856394852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710c00 msgr2=0x56055f722380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f1439e74740 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710c00 0x56055f722380 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1420003c50 tx=0x7f142003efd0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f1439e74740 1 -- 172.21.15.5:0/1856394852 shutdown_connections 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f1439e74740 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x56055f794380 0x56055f7967a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f1439e74740 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56055f7155d0 0x56055f730630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f14197fa640 0 client.29719 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f1439e74740 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055f710c00 0x56055f722380 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f1439e74740 1 --2- 172.21.15.5:0/1856394852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56055f710280 0x56055f721e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.793+0000 7f1439e74740 1 -- 172.21.15.5:0/1856394852 >> 172.21.15.5:0/1856394852 conn(0x56055f6cf310 msgr2=0x56055f7132a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.794+0000 7f1439e74740 1 -- 172.21.15.5:0/1856394852 shutdown_connections 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.794+0000 7f1439e74740 1 -- 172.21.15.5:0/1856394852 wait complete. 2024-09-19T06:23:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.794+0000 7f1439e74740 20 client.29719 trim_cache size 0 max 16384 2024-09-19T06:23:08.800 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:23:08.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.794+0000 7f1437a9b640 1 -- 172.21.15.5:0/33304762 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f141c0a3430 msgr2=0x7f141c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.794+0000 7f1437a9b640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f141c0a3430 0x7f141c0a58f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1420004200 tx=0x7f1420049000 comp rx=0 tx=0).stop 2024-09-19T06:23:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.794+0000 7f1437a9b640 1 -- 172.21.15.5:0/33304762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430106f10 msgr2=0x7f143019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.794+0000 7f1437a9b640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430106f10 0x7f143019cce0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f142800eb10 tx=0x7f142800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.795+0000 7f1437a9b640 1 -- 172.21.15.5:0/33304762 shutdown_connections 2024-09-19T06:23:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.795+0000 7f1437a9b640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430106f10 0x7f143019cce0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.795+0000 7f1437a9b640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1430102790 0x7f1430198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.795+0000 7f1437a9b640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f141c0a3430 0x7f141c0a58f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.795+0000 7f1437a9b640 1 --2- 172.21.15.5:0/33304762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1430101de0 0x7f1430198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.795+0000 7f1437a9b640 1 -- 172.21.15.5:0/33304762 >> 172.21.15.5:0/33304762 conn(0x7f14300fd8e0 msgr2=0x7f14300fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.795+0000 7f1437a9b640 1 -- 172.21.15.5:0/33304762 shutdown_connections 2024-09-19T06:23:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:08.795+0000 7f1437a9b640 1 -- 172.21.15.5:0/33304762 wait complete. 2024-09-19T06:23:09.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:23:09.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 64715 8.76 MB/sec execute 273 sec latency 71.380 ms 2024-09-19T06:23:09.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:09 smithi005 ceph-mon[30955]: osdmap e113: 12 total, 12 up, 12 in 2024-09-19T06:23:09.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:09 smithi157 ceph-mon[35346]: osdmap e113: 12 total, 12 up, 12 in 2024-09-19T06:23:09.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:09 smithi049 ceph-mon[34640]: osdmap e113: 12 total, 12 up, 12 in 2024-09-19T06:23:09.906 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:23:10.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 64843 8.75 MB/sec execute 274 sec latency 205.027 ms 2024-09-19T06:23:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:10 smithi157 ceph-mon[35346]: pgmap v626: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.3 MiB/s wr, 334 op/s 2024-09-19T06:23:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:10 smithi005 ceph-mon[30955]: pgmap v626: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.3 MiB/s wr, 334 op/s 2024-09-19T06:23:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:10 smithi049 ceph-mon[34640]: pgmap v626: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.3 MiB/s wr, 334 op/s 2024-09-19T06:23:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.174+0000 7f25247bb640 1 -- 172.21.15.5:0/3098373531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251c106f10 msgr2=0x7f251c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.174+0000 7f25247bb640 1 --2- 172.21.15.5:0/3098373531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251c106f10 0x7f251c10ea90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2514009860 tx=0x7f251402a340 comp rx=0 tx=0).stop 2024-09-19T06:23:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.174+0000 7f25247bb640 1 -- 172.21.15.5:0/3098373531 shutdown_connections 2024-09-19T06:23:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.174+0000 7f25247bb640 1 --2- 172.21.15.5:0/3098373531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251c106f10 0x7f251c10ea90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.174+0000 7f25247bb640 1 --2- 172.21.15.5:0/3098373531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f251c102790 0x7f251c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.174+0000 7f25247bb640 1 --2- 172.21.15.5:0/3098373531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c101de0 0x7f251c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.174+0000 7f25247bb640 1 -- 172.21.15.5:0/3098373531 >> 172.21.15.5:0/3098373531 conn(0x7f251c0fd880 msgr2=0x7f251c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.175+0000 7f25247bb640 1 -- 172.21.15.5:0/3098373531 shutdown_connections 2024-09-19T06:23:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.175+0000 7f25247bb640 1 -- 172.21.15.5:0/3098373531 wait complete. 2024-09-19T06:23:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f25247bb640 1 Processor -- start 2024-09-19T06:23:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f25247bb640 1 -- start start 2024-09-19T06:23:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f25247bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f251c101de0 0x7f251c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f25247bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251c102790 0x7f251c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f25247bb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c106f10 0x7f251c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f25247bb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f251c111e20 con 0x7f251c101de0 2024-09-19T06:23:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f25247bb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f251c111ca0 con 0x7f251c106f10 2024-09-19T06:23:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f25247bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f251c111fa0 con 0x7f251c102790 2024-09-19T06:23:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f2522557640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f251c101de0 0x7f251c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f2522557640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f251c101de0 0x7f251c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57570/0 (socket says 172.21.15.5:57570) 2024-09-19T06:23:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f2522557640 1 -- 172.21.15.5:0/1847177879 learned_addr learned my addr 172.21.15.5:0/1847177879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f2521d56640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251c102790 0x7f251c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.176+0000 7f2522d58640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c106f10 0x7f251c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.177+0000 7f2522557640 1 -- 172.21.15.5:0/1847177879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251c102790 msgr2=0x7f251c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.177+0000 7f2522557640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251c102790 0x7f251c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.177+0000 7f2522557640 1 -- 172.21.15.5:0/1847177879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c106f10 msgr2=0x7f251c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.177+0000 7f2522557640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c106f10 0x7f251c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.177+0000 7f2522557640 1 -- 172.21.15.5:0/1847177879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c1a6820 con 0x7f251c101de0 2024-09-19T06:23:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.177+0000 7f2521d56640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251c102790 0x7f251c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.177+0000 7f2522557640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f251c101de0 0x7f251c0744f0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f251800ebc0 tx=0x7f251800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.177+0000 7f25137fe640 1 -- 172.21.15.5:0/1847177879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518010630 con 0x7f251c101de0 2024-09-19T06:23:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.177+0000 7f25247bb640 1 -- 172.21.15.5:0/1847177879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251c1a6b10 con 0x7f251c101de0 2024-09-19T06:23:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.177+0000 7f25137fe640 1 -- 172.21.15.5:0/1847177879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2518010c80 con 0x7f251c101de0 2024-09-19T06:23:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.177+0000 7f25137fe640 1 -- 172.21.15.5:0/1847177879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518018bd0 con 0x7f251c101de0 2024-09-19T06:23:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.177+0000 7f25247bb640 1 -- 172.21.15.5:0/1847177879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251c1a70a0 con 0x7f251c101de0 2024-09-19T06:23:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.180+0000 7f25137fe640 1 -- 172.21.15.5:0/1847177879 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f25180107d0 con 0x7f251c101de0 2024-09-19T06:23:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.181+0000 7f2526b94740 1 Processor -- start 2024-09-19T06:23:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.181+0000 7f2526b94740 1 -- start start 2024-09-19T06:23:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2526b94740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d5d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2526b94740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d5dde0 0x55ea09d5e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2526b94740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d68b80 0x55ea09d6af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2526b94740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea09cbb2f0 con 0x55ea09d5dde0 2024-09-19T06:23:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2526b94740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ea09c62cf0 con 0x55ea09d68b80 2024-09-19T06:23:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2526b94740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ea09cb72b0 con 0x55ea09d5f350 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2522d58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d5d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2522557640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d5dde0 0x55ea09d5e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2522d58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d5d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56202/0 (socket says 172.21.15.5:56202) 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2522d58640 1 -- 172.21.15.5:0/995535580 learned_addr learned my addr 172.21.15.5:0/995535580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2521d56640 1 --2- 172.21.15.5:0/995535580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d68b80 0x55ea09d6af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2522d58640 1 -- 172.21.15.5:0/995535580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d68b80 msgr2=0x55ea09d6af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2522d58640 1 --2- 172.21.15.5:0/995535580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d68b80 0x55ea09d6af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2522d58640 1 -- 172.21.15.5:0/995535580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d5dde0 msgr2=0x55ea09d5e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2522d58640 1 --2- 172.21.15.5:0/995535580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d5dde0 0x55ea09d5e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f2522d58640 1 -- 172.21.15.5:0/995535580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea09c473a0 con 0x55ea09d5f350 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.182+0000 7f25137fe640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25080a3430 0x7f25080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.183+0000 7f2522d58640 1 --2- 172.21.15.5:0/995535580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d5d8a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2514035040 tx=0x7f2514005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a5ff3abb96dd06d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.183+0000 7f2522557640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25080a3430 0x7f25080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.183+0000 7f25137fe640 1 -- 172.21.15.5:0/1847177879 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f25180d0310 con 0x7f251c101de0 2024-09-19T06:23:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.183+0000 7f2510ff9640 1 -- 172.21.15.5:0/995535580 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514037040 con 0x55ea09d5f350 2024-09-19T06:23:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.183+0000 7f2510ff9640 1 -- 172.21.15.5:0/995535580 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2514038040 con 0x55ea09d5f350 2024-09-19T06:23:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.183+0000 7f2510ff9640 1 -- 172.21.15.5:0/995535580 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514003850 con 0x55ea09d5f350 2024-09-19T06:23:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.184+0000 7f2526b94740 1 -- 172.21.15.5:0/995535580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 msgr2=0x55ea09d5d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.184+0000 7f2526b94740 1 --2- 172.21.15.5:0/995535580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d5d8a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2514035040 tx=0x7f2514005d90 comp rx=0 tx=0).stop 2024-09-19T06:23:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.184+0000 7f2522557640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25080a3430 0x7f25080a58f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f25180cdaf0 tx=0x7f25180d3e60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.184+0000 7f2526b94740 1 -- 172.21.15.5:0/995535580 shutdown_connections 2024-09-19T06:23:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.184+0000 7f2526b94740 1 --2- 172.21.15.5:0/995535580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d68b80 0x55ea09d6af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.184+0000 7f2526b94740 1 --2- 172.21.15.5:0/995535580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d5dde0 0x55ea09d5e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.184+0000 7f2526b94740 1 --2- 172.21.15.5:0/995535580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d5d8a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.184+0000 7f2526b94740 1 -- 172.21.15.5:0/995535580 >> 172.21.15.5:0/995535580 conn(0x55ea09d1e310 msgr2=0x55ea09d5a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.184+0000 7f2526b94740 1 -- 172.21.15.5:0/995535580 shutdown_connections 2024-09-19T06:23:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.184+0000 7f2526b94740 1 -- 172.21.15.5:0/995535580 wait complete. 2024-09-19T06:23:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.184+0000 7f2526b94740 1 -- start start 2024-09-19T06:23:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.184+0000 7f2526b94740 1 Processor -- start 2024-09-19T06:23:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2526b94740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5dde0 0x55ea09d68550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2526b94740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d65800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2526b94740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d65df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2526b94740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea09d6b820 con 0x55ea09d68b80 2024-09-19T06:23:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2526b94740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ea09c76060 con 0x55ea09d5dde0 2024-09-19T06:23:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2526b94740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ea09d6b9a0 con 0x55ea09d5f350 2024-09-19T06:23:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2526b94740 10 client.0 _ll_register_callbacks cb 0x55ea099c98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2522d58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d65df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2522d58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d65df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57576/0 (socket says 172.21.15.5:57576) 2024-09-19T06:23:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2522d58640 1 -- 172.21.15.5:0/319733146 learned_addr learned my addr 172.21.15.5:0/319733146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2521d56640 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d65800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2522d58640 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5dde0 0x55ea09d68550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2522d58640 1 -- 172.21.15.5:0/319733146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 msgr2=0x55ea09d65800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2522d58640 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d65800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2522d58640 1 -- 172.21.15.5:0/319733146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5dde0 msgr2=0x55ea09d68550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2522d58640 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5dde0 0x55ea09d68550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2522d58640 1 -- 172.21.15.5:0/319733146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea09a2c3c0 con 0x55ea09d68b80 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.185+0000 7f2521d56640 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d65800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.186+0000 7f2522d58640 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d65df0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f25140057c0 tx=0x7f2514005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8faff1789170e28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.186+0000 7f2505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.186+0000 7f2505ffb640 1 -- 172.21.15.5:0/319733146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251404e020 con 0x55ea09d68b80 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.186+0000 7f2505ffb640 1 -- 172.21.15.5:0/319733146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f2514005230 con 0x55ea09d68b80 2024-09-19T06:23:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.186+0000 7f2526b94740 1 -- 172.21.15.5:0/319733146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2514038040 con 0x55ea09d68b80 2024-09-19T06:23:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.186+0000 7f2505ffb640 1 -- 172.21.15.5:0/319733146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2514059050 con 0x55ea09d68b80 2024-09-19T06:23:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.186+0000 7f2526b94740 10 client.20421 fetch_fsmap learned FSMap version 58 2024-09-19T06:23:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.186+0000 7f2526b94740 1 -- 172.21.15.5:0/319733146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55ea09d772c0 con 0x55ea09d68b80 2024-09-19T06:23:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.186+0000 7f2505ffb640 1 -- 172.21.15.5:0/319733146 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2514005430 con 0x55ea09d68b80 2024-09-19T06:23:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.187+0000 7f2505ffb640 1 -- 172.21.15.5:0/319733146 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f251404a450 con 0x55ea09d68b80 2024-09-19T06:23:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.187+0000 7f2526b94740 10 client.20421 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:23:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.187+0000 7f2526b94740 10 client.20421 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:23:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.187+0000 7f2526b94740 20 client.20421 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.187+0000 7f2526b94740 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ea09d78610 0x55ea09d7aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.187+0000 7f2526b94740 4 client.20421 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.187+0000 7f2526b94740 1 -- 172.21.15.5:0/319733146 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2514038040 con 0x55ea09d78610 2024-09-19T06:23:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.187+0000 7f2521d56640 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ea09d78610 0x55ea09d7aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.187+0000 7f2505ffb640 10 client.20421 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:23:11.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.187+0000 7f2521d56640 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ea09d78610 0x55ea09d7aa00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93f1aba36eb2ab0f server_cookie=9c6a1095fd27f958 in_seq=0 out_seq=0 2024-09-19T06:23:11.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.188+0000 7f2505ffb640 1 -- 172.21.15.5:0/319733146 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2514038040 con 0x55ea09d78610 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.188+0000 7f2505ffb640 10 client.20421 handle_command_reply: tid=0 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.188+0000 7f2526b94740 1 client.20421 shutdown 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.188+0000 7f2526b94740 1 -- 172.21.15.5:0/319733146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 msgr2=0x55ea09d65df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.188+0000 7f2526b94740 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d65df0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f25140057c0 tx=0x7f2514005f90 comp rx=0 tx=0).stop 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.188+0000 7f2526b94740 1 -- 172.21.15.5:0/319733146 shutdown_connections 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.188+0000 7f2526b94740 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ea09d78610 0x55ea09d7aa00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.188+0000 7f2526b94740 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d65df0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.188+0000 7f2526b94740 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d65800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.188+0000 7f2526b94740 1 --2- 172.21.15.5:0/319733146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5dde0 0x55ea09d68550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.188+0000 7f2526b94740 1 -- 172.21.15.5:0/319733146 >> 172.21.15.5:0/319733146 conn(0x55ea09d1e310 msgr2=0x55ea09d60140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.189+0000 7f2505ffb640 0 client.20421 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.189+0000 7f2526b94740 1 -- 172.21.15.5:0/319733146 shutdown_connections 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.189+0000 7f2526b94740 1 -- 172.21.15.5:0/319733146 wait complete. 2024-09-19T06:23:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.189+0000 7f2526b94740 20 client.20421 trim_cache size 0 max 16384 2024-09-19T06:23:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.208+0000 7f2526b94740 1 Processor -- start 2024-09-19T06:23:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.208+0000 7f2526b94740 1 -- start start 2024-09-19T06:23:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2526b94740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5dde0 0x55ea09d67770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2526b94740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5f350 0x55ea09d60930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2526b94740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d60e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2526b94740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1ab6f0 con 0x55ea09d68b80 2024-09-19T06:23:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2526b94740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1ad230 con 0x55ea09d5f350 2024-09-19T06:23:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2526b94740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1abdc0 con 0x55ea09d5dde0 2024-09-19T06:23:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2522d58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5dde0 0x55ea09d67770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2522d58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5dde0 0x55ea09d67770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56218/0 (socket says 172.21.15.5:56218) 2024-09-19T06:23:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2521d56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5f350 0x55ea09d60930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2522d58640 1 -- 172.21.15.5:0/2670843418 learned_addr learned my addr 172.21.15.5:0/2670843418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2522d58640 1 --2- 172.21.15.5:0/2670843418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d60e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2521d56640 1 -- 172.21.15.5:0/2670843418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5dde0 msgr2=0x55ea09d67770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2521d56640 1 --2- 172.21.15.5:0/2670843418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5dde0 0x55ea09d67770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2521d56640 1 -- 172.21.15.5:0/2670843418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 msgr2=0x55ea09d60e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2521d56640 1 --2- 172.21.15.5:0/2670843418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d60e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2521d56640 1 -- 172.21.15.5:0/2670843418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea09d688c0 con 0x55ea09d5f350 2024-09-19T06:23:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.209+0000 7f2522d58640 1 --2- 172.21.15.5:0/2670843418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d60e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:23:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.210+0000 7f2521d56640 1 --2- 172.21.15.5:0/2670843418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5f350 0x55ea09d60930 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f250c0096c0 tx=0x7f250c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb491a312b8c062c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.210+0000 7f2507fff640 1 -- 172.21.15.5:0/2670843418 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c00fb50 con 0x55ea09d5f350 2024-09-19T06:23:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.210+0000 7f2507fff640 1 -- 172.21.15.5:0/2670843418 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f250c0054d0 con 0x55ea09d5f350 2024-09-19T06:23:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.210+0000 7f2507fff640 1 -- 172.21.15.5:0/2670843418 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c004d70 con 0x55ea09d5f350 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.211+0000 7f2526b94740 1 -- 172.21.15.5:0/2670843418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5f350 msgr2=0x55ea09d60930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.211+0000 7f2526b94740 1 --2- 172.21.15.5:0/2670843418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5f350 0x55ea09d60930 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f250c0096c0 tx=0x7f250c009260 comp rx=0 tx=0).stop 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.211+0000 7f2526b94740 1 -- 172.21.15.5:0/2670843418 shutdown_connections 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.211+0000 7f2526b94740 1 --2- 172.21.15.5:0/2670843418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d60e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.211+0000 7f2526b94740 1 --2- 172.21.15.5:0/2670843418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5f350 0x55ea09d60930 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.211+0000 7f2526b94740 1 --2- 172.21.15.5:0/2670843418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5dde0 0x55ea09d67770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.211+0000 7f2526b94740 1 -- 172.21.15.5:0/2670843418 >> 172.21.15.5:0/2670843418 conn(0x55ea09d1e310 msgr2=0x55ea09d67060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.211+0000 7f2526b94740 1 -- 172.21.15.5:0/2670843418 shutdown_connections 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.211+0000 7f2526b94740 1 -- 172.21.15.5:0/2670843418 wait complete. 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.211+0000 7f2526b94740 1 -- start start 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2526b94740 1 Processor -- start 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2526b94740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5dde0 0x55ea09d7db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2526b94740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d7ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2526b94740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d7b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2526b94740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1acb60 con 0x55ea09d68b80 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2521d56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d7ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2526b94740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1ac490 con 0x55ea09d5dde0 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2526b94740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1a9710 con 0x55ea09d5f350 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2521d56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d7ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56232/0 (socket says 172.21.15.5:56232) 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2521d56640 1 -- 172.21.15.5:0/645723973 learned_addr learned my addr 172.21.15.5:0/645723973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2522d58640 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5dde0 0x55ea09d7db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2526b94740 10 client.0 _ll_register_callbacks cb 0x7f25180d3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.212+0000 7f2522d58640 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d7b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.213+0000 7f2521d56640 1 -- 172.21.15.5:0/645723973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5dde0 msgr2=0x55ea09d7db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.213+0000 7f2521d56640 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5dde0 0x55ea09d7db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.213+0000 7f2521d56640 1 -- 172.21.15.5:0/645723973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 msgr2=0x55ea09d7b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.213+0000 7f2521d56640 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d7b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.213+0000 7f2521d56640 1 -- 172.21.15.5:0/645723973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea09d7b970 con 0x55ea09d5f350 2024-09-19T06:23:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.213+0000 7f2521d56640 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d7ae70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f250c0096c0 tx=0x7f250c0052d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33556cf38d749c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.213+0000 7f2505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:23:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.213+0000 7f2505ffb640 1 -- 172.21.15.5:0/645723973 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c037030 con 0x55ea09d5f350 2024-09-19T06:23:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.213+0000 7f2505ffb640 1 -- 172.21.15.5:0/645723973 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f250c00fb60 con 0x55ea09d5f350 2024-09-19T06:23:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.214+0000 7f2526b94740 1 -- 172.21.15.5:0/645723973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f250c0054d0 con 0x55ea09d5f350 2024-09-19T06:23:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.214+0000 7f2505ffb640 1 -- 172.21.15.5:0/645723973 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f250c00fda0 con 0x55ea09d5f350 2024-09-19T06:23:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.214+0000 7f2526b94740 10 client.29737 fetch_fsmap learned FSMap version 58 2024-09-19T06:23:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.214+0000 7f2526b94740 1 -- 172.21.15.5:0/645723973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55ea09de2c60 con 0x55ea09d5f350 2024-09-19T06:23:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.214+0000 7f2505ffb640 1 -- 172.21.15.5:0/645723973 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f250c0054d0 con 0x55ea09d5f350 2024-09-19T06:23:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.214+0000 7f2505ffb640 1 -- 172.21.15.5:0/645723973 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f250c004c00 con 0x55ea09d5f350 2024-09-19T06:23:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.214+0000 7f2526b94740 10 client.29737 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:23:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.214+0000 7f2526b94740 10 client.29737 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.214+0000 7f2526b94740 20 client.29737 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.214+0000 7f2526b94740 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ea09de3400 0x55ea09de5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.214+0000 7f2526b94740 4 client.29737 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.214+0000 7f2526b94740 1 -- 172.21.15.5:0/645723973 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea09cc8860 con 0x55ea09de3400 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2522d58640 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ea09de3400 0x55ea09de5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2522d58640 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ea09de3400 0x55ea09de5820 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd64af0444adbadb server_cookie=7cf9b36966fea16f in_seq=0 out_seq=0 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2505ffb640 10 client.29737 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2505ffb640 1 -- 172.21.15.5:0/645723973 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea09cc8860 con 0x55ea09de3400 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2505ffb640 10 client.29737 handle_command_reply: tid=0 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2526b94740 1 client.29737 shutdown 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2526b94740 1 -- 172.21.15.5:0/645723973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 msgr2=0x55ea09d7ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2526b94740 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d7ae70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f250c0096c0 tx=0x7f250c0052d0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2526b94740 1 -- 172.21.15.5:0/645723973 shutdown_connections 2024-09-19T06:23:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2526b94740 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x55ea09de3400 0x55ea09de5820 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2526b94740 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea09d68b80 0x55ea09d7b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2526b94740 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea09d5f350 0x55ea09d7ae70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2526b94740 1 --2- 172.21.15.5:0/645723973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea09d5dde0 0x55ea09d7db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2526b94740 1 -- 172.21.15.5:0/645723973 >> 172.21.15.5:0/645723973 conn(0x55ea09d1e310 msgr2=0x55ea09d67880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.215+0000 7f2505ffb640 0 client.29737 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:23:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.216+0000 7f2526b94740 1 -- 172.21.15.5:0/645723973 shutdown_connections 2024-09-19T06:23:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.216+0000 7f2526b94740 1 -- 172.21.15.5:0/645723973 wait complete. 2024-09-19T06:23:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.216+0000 7f2526b94740 20 client.29737 trim_cache size 0 max 16384 2024-09-19T06:23:11.221 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:23:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.216+0000 7f25247bb640 1 -- 172.21.15.5:0/1847177879 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25080a3430 msgr2=0x7f25080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.216+0000 7f25247bb640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25080a3430 0x7f25080a58f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f25180cdaf0 tx=0x7f25180d3e60 comp rx=0 tx=0).stop 2024-09-19T06:23:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.216+0000 7f25247bb640 1 -- 172.21.15.5:0/1847177879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f251c101de0 msgr2=0x7f251c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.216+0000 7f25247bb640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f251c101de0 0x7f251c0744f0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f251800ebc0 tx=0x7f251800c6a0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.216+0000 7f25247bb640 1 -- 172.21.15.5:0/1847177879 shutdown_connections 2024-09-19T06:23:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.216+0000 7f25247bb640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c106f10 0x7f251c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.216+0000 7f25247bb640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25080a3430 0x7f25080a58f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.216+0000 7f25247bb640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251c102790 0x7f251c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.217+0000 7f25247bb640 1 --2- 172.21.15.5:0/1847177879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f251c101de0 0x7f251c0744f0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.217+0000 7f25247bb640 1 -- 172.21.15.5:0/1847177879 >> 172.21.15.5:0/1847177879 conn(0x7f251c0fd880 msgr2=0x7f251c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.217+0000 7f25247bb640 1 -- 172.21.15.5:0/1847177879 shutdown_connections 2024-09-19T06:23:11.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:11.217+0000 7f25247bb640 1 -- 172.21.15.5:0/1847177879 wait complete. 2024-09-19T06:23:11.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 64979 8.75 MB/sec execute 275 sec latency 348.895 ms 2024-09-19T06:23:11.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:23:12.335 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:23:12.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65127 8.74 MB/sec execute 276 sec latency 64.071 ms 2024-09-19T06:23:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:12 smithi157 ceph-mon[35346]: pgmap v627: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 145 op/s 2024-09-19T06:23:12.841 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:12 smithi005 ceph-mon[30955]: pgmap v627: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 145 op/s 2024-09-19T06:23:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:12 smithi049 ceph-mon[34640]: pgmap v627: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 145 op/s 2024-09-19T06:23:13.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.166+0000 7efcee918640 1 -- 172.21.15.5:0/4163998771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efce8102ef0 msgr2=0x7efce810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:13.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.166+0000 7efcee918640 1 --2- 172.21.15.5:0/4163998771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efce8102ef0 0x7efce810da40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009930 tx=0x7efcd802a4d0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.166+0000 7efcee918640 1 -- 172.21.15.5:0/4163998771 shutdown_connections 2024-09-19T06:23:13.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.166+0000 7efcee918640 1 --2- 172.21.15.5:0/4163998771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efce8102ef0 0x7efce810da40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.166+0000 7efcee918640 1 --2- 172.21.15.5:0/4163998771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce8102570 0x7efce81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.166+0000 7efcee918640 1 --2- 172.21.15.5:0/4163998771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce8100e20 0x7efce8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.166+0000 7efcee918640 1 -- 172.21.15.5:0/4163998771 >> 172.21.15.5:0/4163998771 conn(0x7efce80fc7d0 msgr2=0x7efce80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:13.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.166+0000 7efcee918640 1 -- 172.21.15.5:0/4163998771 shutdown_connections 2024-09-19T06:23:13.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.166+0000 7efcee918640 1 -- 172.21.15.5:0/4163998771 wait complete. 2024-09-19T06:23:13.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.167+0000 7efcee918640 1 Processor -- start 2024-09-19T06:23:13.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.167+0000 7efcee918640 1 -- start start 2024-09-19T06:23:13.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efcee918640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce8100e20 0x7efce819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efcee918640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efce8102570 0x7efce819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efcee918640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce8102ef0 0x7efce819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efcee918640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efce8110e80 con 0x7efce8100e20 2024-09-19T06:23:13.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efcee918640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efce8110d00 con 0x7efce8102570 2024-09-19T06:23:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efcee918640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efce8111000 con 0x7efce8102ef0 2024-09-19T06:23:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efce77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efce8102570 0x7efce819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efce77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efce8102570 0x7efce819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55988/0 (socket says 172.21.15.5:55988) 2024-09-19T06:23:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efce77fe640 1 -- 172.21.15.5:0/3585264413 learned_addr learned my addr 172.21.15.5:0/3585264413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efceceb5640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce8102ef0 0x7efce819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efce7fff640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce8100e20 0x7efce819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efce77fe640 1 -- 172.21.15.5:0/3585264413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce8102ef0 msgr2=0x7efce819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efce77fe640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce8102ef0 0x7efce819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efce77fe640 1 -- 172.21.15.5:0/3585264413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce8100e20 msgr2=0x7efce819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efce77fe640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce8100e20 0x7efce819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efce77fe640 1 -- 172.21.15.5:0/3585264413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce819f5e0 con 0x7efce8102570 2024-09-19T06:23:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efceceb5640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce8102ef0 0x7efce819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efce7fff640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce8100e20 0x7efce819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.168+0000 7efce77fe640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efce8102570 0x7efce819ab70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00ea60 tx=0x7efcdc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.169+0000 7efce57fa640 1 -- 172.21.15.5:0/3585264413 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00cde0 con 0x7efce8102570 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.169+0000 7efce57fa640 1 -- 172.21.15.5:0/3585264413 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcdc0035f0 con 0x7efce8102570 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.169+0000 7efce57fa640 1 -- 172.21.15.5:0/3585264413 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc010690 con 0x7efce8102570 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.169+0000 7efcee918640 1 -- 172.21.15.5:0/3585264413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce819f8d0 con 0x7efce8102570 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.169+0000 7efcee918640 1 -- 172.21.15.5:0/3585264413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce81a71b0 con 0x7efce8102570 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.172+0000 7efce57fa640 1 -- 172.21.15.5:0/3585264413 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7efcdc0037a0 con 0x7efce8102570 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.172+0000 7efcf0cf1740 1 Processor -- start 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.172+0000 7efcf0cf1740 1 -- start start 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efcf0cf1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x564940883090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efcf0cf1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649408835d0 0x564940883a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efcf0cf1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56494088e370 0x564940890760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efcf0cf1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5649407e92f0 con 0x5649408835d0 2024-09-19T06:23:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efcf0cf1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564940790cf0 con 0x564940884b40 2024-09-19T06:23:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efcf0cf1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5649407e52b0 con 0x56494088e370 2024-09-19T06:23:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efce77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56494088e370 0x564940890760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efce7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649408835d0 0x564940883a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efce77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56494088e370 0x564940890760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56270/0 (socket says 172.21.15.5:56270) 2024-09-19T06:23:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efce77fe640 1 -- 172.21.15.5:0/3908142352 learned_addr learned my addr 172.21.15.5:0/3908142352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efceceb5640 1 --2- 172.21.15.5:0/3908142352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x564940883090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efce77fe640 1 -- 172.21.15.5:0/3908142352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 msgr2=0x564940883090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efce77fe640 1 --2- 172.21.15.5:0/3908142352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x564940883090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efce77fe640 1 -- 172.21.15.5:0/3908142352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649408835d0 msgr2=0x564940883a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efce77fe640 1 --2- 172.21.15.5:0/3908142352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649408835d0 0x564940883a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.173+0000 7efce77fe640 1 -- 172.21.15.5:0/3908142352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649407753a0 con 0x56494088e370 2024-09-19T06:23:13.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.174+0000 7efce57fa640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efcd00a3430 0x7efcd00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.174+0000 7efceceb5640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efcd00a3430 0x7efcd00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.174+0000 7efce57fa640 1 -- 172.21.15.5:0/3585264413 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7efcdc0d0120 con 0x7efce8102570 2024-09-19T06:23:13.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.174+0000 7efce77fe640 1 --2- 172.21.15.5:0/3908142352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56494088e370 0x564940890760 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0cd730 tx=0x7efcdc0d8220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b835ce0cc806a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:13.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.174+0000 7efcce7fc640 1 -- 172.21.15.5:0/3908142352 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc098620 con 0x56494088e370 2024-09-19T06:23:13.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.174+0000 7efcce7fc640 1 -- 172.21.15.5:0/3908142352 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcdc003ad0 con 0x56494088e370 2024-09-19T06:23:13.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.174+0000 7efcce7fc640 1 -- 172.21.15.5:0/3908142352 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc09bce0 con 0x56494088e370 2024-09-19T06:23:13.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.175+0000 7efcf0cf1740 1 -- 172.21.15.5:0/3908142352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56494088e370 msgr2=0x564940890760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:13.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.175+0000 7efceceb5640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efcd00a3430 0x7efcd00a58f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efcd8006010 tx=0x7efcd8005b50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:13.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.175+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/3908142352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56494088e370 0x564940890760 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0cd730 tx=0x7efcdc0d8220 comp rx=0 tx=0).stop 2024-09-19T06:23:13.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.175+0000 7efcf0cf1740 1 -- 172.21.15.5:0/3908142352 shutdown_connections 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.175+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/3908142352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56494088e370 0x564940890760 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.175+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/3908142352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649408835d0 0x564940883a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.175+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/3908142352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x564940883090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.175+0000 7efcf0cf1740 1 -- 172.21.15.5:0/3908142352 >> 172.21.15.5:0/3908142352 conn(0x56494084c310 msgr2=0x56494087fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.176+0000 7efcf0cf1740 1 -- 172.21.15.5:0/3908142352 shutdown_connections 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.176+0000 7efcf0cf1740 1 -- 172.21.15.5:0/3908142352 wait complete. 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.176+0000 7efcf0cf1740 1 -- start start 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.176+0000 7efcf0cf1740 1 Processor -- start 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efcf0cf1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649408835d0 0x56494088dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efcf0cf1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x56494088b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efcf0cf1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56494088e370 0x56494088b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efcf0cf1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564940891070 con 0x5649408835d0 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efce7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649408835d0 0x56494088dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efcf0cf1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5649407a4060 con 0x564940884b40 2024-09-19T06:23:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efcf0cf1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5649408911f0 con 0x56494088e370 2024-09-19T06:23:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efceceb5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56494088e370 0x56494088b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efce7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649408835d0 0x56494088dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57612/0 (socket says 172.21.15.5:57612) 2024-09-19T06:23:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efce7fff640 1 -- 172.21.15.5:0/1097281985 learned_addr learned my addr 172.21.15.5:0/1097281985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efce77fe640 1 --2- 172.21.15.5:0/1097281985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x56494088b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efcf0cf1740 10 client.0 _ll_register_callbacks cb 0x5649404f78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efce7fff640 1 -- 172.21.15.5:0/1097281985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56494088e370 msgr2=0x56494088b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efce7fff640 1 --2- 172.21.15.5:0/1097281985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56494088e370 0x56494088b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efce7fff640 1 -- 172.21.15.5:0/1097281985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 msgr2=0x56494088b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efce7fff640 1 --2- 172.21.15.5:0/1097281985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x56494088b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.177+0000 7efce7fff640 1 -- 172.21.15.5:0/1097281985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56494055a3c0 con 0x5649408835d0 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.178+0000 7efce7fff640 1 --2- 172.21.15.5:0/1097281985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649408835d0 0x56494088dd60 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7efce0012700 tx=0x7efce0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903d45658c3b5f56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.178+0000 7efcc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.178+0000 7efcc77fe640 1 -- 172.21.15.5:0/1097281985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0010dc0 con 0x5649408835d0 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.178+0000 7efcc77fe640 1 -- 172.21.15.5:0/1097281985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7efce0004b20 con 0x5649408835d0 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.178+0000 7efcf0cf1740 1 -- 172.21.15.5:0/1097281985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5649407f6860 con 0x5649408835d0 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.178+0000 7efcc77fe640 1 -- 172.21.15.5:0/1097281985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efce0004d20 con 0x5649408835d0 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.178+0000 7efcf0cf1740 10 client.20436 fetch_fsmap learned FSMap version 58 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.178+0000 7efcf0cf1740 1 -- 172.21.15.5:0/1097281985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56494089cbb0 con 0x5649408835d0 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.179+0000 7efcc77fe640 1 -- 172.21.15.5:0/1097281985 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5649407f6860 con 0x5649408835d0 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.179+0000 7efcc77fe640 1 -- 172.21.15.5:0/1097281985 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7efce0019070 con 0x5649408835d0 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.179+0000 7efcf0cf1740 10 client.20436 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.179+0000 7efcf0cf1740 10 client.20436 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:23:13.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.179+0000 7efcf0cf1740 20 client.20436 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.179+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/1097281985 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56494089e070 0x5649408a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.179+0000 7efcf0cf1740 4 client.20436 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.179+0000 7efcf0cf1740 1 -- 172.21.15.5:0/1097281985 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649407f0d70 con 0x56494089e070 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.179+0000 7efce77fe640 1 --2- 172.21.15.5:0/1097281985 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56494089e070 0x5649408a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.180+0000 7efce77fe640 1 --2- 172.21.15.5:0/1097281985 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56494089e070 0x5649408a04e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fedbb40698a57695 server_cookie=2ce5efa324185f78 in_seq=0 out_seq=0 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.180+0000 7efcc77fe640 10 client.20436 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcc77fe640 1 -- 172.21.15.5:0/1097281985 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649407f0d70 con 0x56494089e070 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcc77fe640 10 client.20436 handle_command_reply: tid=0 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcf0cf1740 1 client.20436 shutdown 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcf0cf1740 1 -- 172.21.15.5:0/1097281985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649408835d0 msgr2=0x56494088dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/1097281985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649408835d0 0x56494088dd60 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7efce0012700 tx=0x7efce0012bc0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcf0cf1740 1 -- 172.21.15.5:0/1097281985 shutdown_connections 2024-09-19T06:23:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/1097281985 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56494089e070 0x5649408a04e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/1097281985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56494088e370 0x56494088b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcc77fe640 0 client.20436 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:23:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/1097281985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x56494088b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/1097281985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649408835d0 0x56494088dd60 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcf0cf1740 1 -- 172.21.15.5:0/1097281985 >> 172.21.15.5:0/1097281985 conn(0x56494084c310 msgr2=0x564940885930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcf0cf1740 1 -- 172.21.15.5:0/1097281985 shutdown_connections 2024-09-19T06:23:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.181+0000 7efcf0cf1740 1 -- 172.21.15.5:0/1097281985 wait complete. 2024-09-19T06:23:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.182+0000 7efcf0cf1740 20 client.20436 trim_cache size 0 max 16384 2024-09-19T06:23:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efcf0cf1740 1 Processor -- start 2024-09-19T06:23:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efcf0cf1740 1 -- start start 2024-09-19T06:23:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efcf0cf1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649408835d0 0x56494088a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efcf0cf1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564940884b40 0x56494089a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efcf0cf1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56494088e370 0x56494087ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.460 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:23:13.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efcf0cf1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efce81afb50 con 0x564940884b40 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efcf0cf1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efce81b1690 con 0x56494088e370 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efcf0cf1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efce81b0220 con 0x5649408835d0 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efce77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564940884b40 0x56494089a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efce77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564940884b40 0x56494089a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57628/0 (socket says 172.21.15.5:57628) 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efce77fe640 1 -- 172.21.15.5:0/3184544166 learned_addr learned my addr 172.21.15.5:0/3184544166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efce7fff640 1 --2- 172.21.15.5:0/3184544166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649408835d0 0x56494088a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efceceb5640 1 --2- 172.21.15.5:0/3184544166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56494088e370 0x56494087ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efce77fe640 1 -- 172.21.15.5:0/3184544166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649408835d0 msgr2=0x56494088a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efce77fe640 1 --2- 172.21.15.5:0/3184544166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649408835d0 0x56494088a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efce77fe640 1 -- 172.21.15.5:0/3184544166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56494088e370 msgr2=0x56494087ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efce77fe640 1 --2- 172.21.15.5:0/3184544166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56494088e370 0x56494087ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.203+0000 7efce77fe640 1 -- 172.21.15.5:0/3184544166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564940889fc0 con 0x564940884b40 2024-09-19T06:23:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.204+0000 7efce77fe640 1 --2- 172.21.15.5:0/3184544166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564940884b40 0x56494089a390 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0d8720 tx=0x7efcdc00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110040f80b55db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.204+0000 7efccdffb640 1 -- 172.21.15.5:0/3184544166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc09bec0 con 0x564940884b40 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.204+0000 7efccdffb640 1 -- 172.21.15.5:0/3184544166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcdc09ca80 con 0x564940884b40 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.204+0000 7efccdffb640 1 -- 172.21.15.5:0/3184544166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc09c150 con 0x564940884b40 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.205+0000 7efcf0cf1740 1 -- 172.21.15.5:0/3184544166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564940884b40 msgr2=0x56494089a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.205+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/3184544166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564940884b40 0x56494089a390 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0d8720 tx=0x7efcdc00ef90 comp rx=0 tx=0).stop 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.205+0000 7efcf0cf1740 1 -- 172.21.15.5:0/3184544166 shutdown_connections 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.205+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/3184544166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56494088e370 0x56494087ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.205+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/3184544166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564940884b40 0x56494089a390 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.205+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/3184544166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649408835d0 0x56494088a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.205+0000 7efcf0cf1740 1 -- 172.21.15.5:0/3184544166 >> 172.21.15.5:0/3184544166 conn(0x56494084c310 msgr2=0x56494087e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.205+0000 7efcf0cf1740 1 -- 172.21.15.5:0/3184544166 shutdown_connections 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.205+0000 7efcf0cf1740 1 -- 172.21.15.5:0/3184544166 wait complete. 2024-09-19T06:23:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.206+0000 7efcf0cf1740 1 -- start start 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.206+0000 7efcf0cf1740 1 Processor -- start 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.206+0000 7efcf0cf1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649408835d0 0x564940888190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.206+0000 7efcf0cf1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x5649408886d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.206+0000 7efcf0cf1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56494088e370 0x564940888c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.206+0000 7efcf0cf1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efce81b0fc0 con 0x56494088e370 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.206+0000 7efcf0cf1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efce81b08f0 con 0x564940884b40 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.206+0000 7efcf0cf1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efce81adb50 con 0x5649408835d0 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.206+0000 7efcf0cf1740 10 client.0 _ll_register_callbacks cb 0x7efce00099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.206+0000 7efce7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649408835d0 0x564940888190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.207+0000 7efce7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649408835d0 0x564940888190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56312/0 (socket says 172.21.15.5:56312) 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.207+0000 7efceceb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56494088e370 0x564940888c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.207+0000 7efce7fff640 1 -- 172.21.15.5:0/607920065 learned_addr learned my addr 172.21.15.5:0/607920065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:13.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.207+0000 7efce77fe640 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x5649408886d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.207+0000 7efce77fe640 1 -- 172.21.15.5:0/607920065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649408835d0 msgr2=0x564940888190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:13.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.207+0000 7efce77fe640 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649408835d0 0x564940888190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.207+0000 7efce77fe640 1 -- 172.21.15.5:0/607920065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56494088e370 msgr2=0x564940888c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:13.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.207+0000 7efce77fe640 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56494088e370 0x564940888c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.207+0000 7efce77fe640 1 -- 172.21.15.5:0/607920065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56494088b970 con 0x564940884b40 2024-09-19T06:23:13.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.207+0000 7efceceb5640 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56494088e370 0x564940888c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:13.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.208+0000 7efce77fe640 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x5649408886d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x5649408a0750 tx=0x7efcdc09c150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10945bf435f0c330 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:13.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.208+0000 7efcc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:23:13.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.208+0000 7efcc77fe640 1 -- 172.21.15.5:0/607920065 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc098620 con 0x564940884b40 2024-09-19T06:23:13.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.208+0000 7efcc77fe640 1 -- 172.21.15.5:0/607920065 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7efcdc0b5230 con 0x564940884b40 2024-09-19T06:23:13.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.208+0000 7efcf0cf1740 1 -- 172.21.15.5:0/607920065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcdc09ca80 con 0x564940884b40 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.208+0000 7efcc77fe640 1 -- 172.21.15.5:0/607920065 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efcdc0a0730 con 0x564940884b40 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.208+0000 7efcf0cf1740 10 client.29816 fetch_fsmap learned FSMap version 58 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.208+0000 7efcf0cf1740 1 -- 172.21.15.5:0/607920065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x564940911850 con 0x564940884b40 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.209+0000 7efcc77fe640 1 -- 172.21.15.5:0/607920065 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcdc09ca80 con 0x564940884b40 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.209+0000 7efcc77fe640 1 -- 172.21.15.5:0/607920065 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7efcdc016330 con 0x564940884b40 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.209+0000 7efcf0cf1740 10 client.29816 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.209+0000 7efcf0cf1740 10 client.29816 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.209+0000 7efcf0cf1740 20 client.29816 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.209+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56494089e070 0x564940913a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.209+0000 7efcf0cf1740 4 client.29816 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.209+0000 7efcf0cf1740 1 -- 172.21.15.5:0/607920065 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56494076d170 con 0x56494089e070 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.209+0000 7efce7fff640 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56494089e070 0x564940913a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.210+0000 7efcc77fe640 10 client.29816 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.210+0000 7efce7fff640 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56494089e070 0x564940913a30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6da5b19f7c0628 server_cookie=716184f82d3d143b in_seq=0 out_seq=0 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.210+0000 7efcc77fe640 1 -- 172.21.15.5:0/607920065 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56494076d170 con 0x56494089e070 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.210+0000 7efcc77fe640 10 client.29816 handle_command_reply: tid=0 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.210+0000 7efcf0cf1740 1 client.29816 shutdown 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.210+0000 7efcf0cf1740 1 -- 172.21.15.5:0/607920065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 msgr2=0x5649408886d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.210+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x5649408886d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x5649408a0750 tx=0x7efcdc09c150 comp rx=0 tx=0).stop 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcf0cf1740 1 -- 172.21.15.5:0/607920065 shutdown_connections 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56494089e070 0x564940913a30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56494088e370 0x564940888c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcc77fe640 0 client.29816 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564940884b40 0x5649408886d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcf0cf1740 1 --2- 172.21.15.5:0/607920065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649408835d0 0x564940888190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcf0cf1740 1 -- 172.21.15.5:0/607920065 >> 172.21.15.5:0/607920065 conn(0x56494084c310 msgr2=0x564940885110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcf0cf1740 1 -- 172.21.15.5:0/607920065 shutdown_connections 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcf0cf1740 1 -- 172.21.15.5:0/607920065 wait complete. 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcf0cf1740 20 client.29816 trim_cache size 0 max 16384 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcee918640 1 -- 172.21.15.5:0/3585264413 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efcd00a3430 msgr2=0x7efcd00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcee918640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efcd00a3430 0x7efcd00a58f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efcd8006010 tx=0x7efcd8005b50 comp rx=0 tx=0).stop 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcee918640 1 -- 172.21.15.5:0/3585264413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efce8102570 msgr2=0x7efce819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.211+0000 7efcee918640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efce8102570 0x7efce819ab70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00ea60 tx=0x7efcdc00ef20 comp rx=0 tx=0).stop 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.212+0000 7efcee918640 1 -- 172.21.15.5:0/3585264413 shutdown_connections 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.212+0000 7efcee918640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7efcd00a3430 0x7efcd00a58f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.212+0000 7efcee918640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce8102ef0 0x7efce819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.212+0000 7efcee918640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efce8102570 0x7efce819ab70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.212+0000 7efcee918640 1 --2- 172.21.15.5:0/3585264413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce8100e20 0x7efce819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.212+0000 7efcee918640 1 -- 172.21.15.5:0/3585264413 >> 172.21.15.5:0/3585264413 conn(0x7efce80fc7d0 msgr2=0x7efce806a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.212+0000 7efcee918640 1 -- 172.21.15.5:0/3585264413 shutdown_connections 2024-09-19T06:23:13.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:13.212+0000 7efcee918640 1 -- 172.21.15.5:0/3585264413 wait complete. 2024-09-19T06:23:13.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65275 8.72 MB/sec execute 277 sec latency 105.902 ms 2024-09-19T06:23:14.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:23:14.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65437 8.72 MB/sec execute 278 sec latency 84.597 ms 2024-09-19T06:23:14.626 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:23:14.763 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:14 smithi005 ceph-mon[30955]: pgmap v628: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.6 MiB/s wr, 382 op/s 2024-09-19T06:23:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:14 smithi157 ceph-mon[35346]: pgmap v628: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.6 MiB/s wr, 382 op/s 2024-09-19T06:23:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:14 smithi049 ceph-mon[34640]: pgmap v628: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.6 MiB/s wr, 382 op/s 2024-09-19T06:23:15.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.204+0000 7f7b0e50b640 1 -- 172.21.15.5:0/2727878461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b081081f0 msgr2=0x7f7b08108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:15.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.204+0000 7f7b0e50b640 1 --2- 172.21.15.5:0/2727878461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b081081f0 0x7f7b08108610 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009900 tx=0x7f7afc02a410 comp rx=0 tx=0).stop 2024-09-19T06:23:15.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.205+0000 7f7b0e50b640 1 -- 172.21.15.5:0/2727878461 shutdown_connections 2024-09-19T06:23:15.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.205+0000 7f7b0e50b640 1 --2- 172.21.15.5:0/2727878461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b08108b50 0x7f7b08115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.205+0000 7f7b0e50b640 1 --2- 172.21.15.5:0/2727878461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b081081f0 0x7f7b08108610 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.205+0000 7f7b0e50b640 1 --2- 172.21.15.5:0/2727878461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0810e230 0x7f7b0810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.205+0000 7f7b0e50b640 1 -- 172.21.15.5:0/2727878461 >> 172.21.15.5:0/2727878461 conn(0x7f7b08077bb0 msgr2=0x7f7b08079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:15.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.205+0000 7f7b0e50b640 1 -- 172.21.15.5:0/2727878461 shutdown_connections 2024-09-19T06:23:15.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.205+0000 7f7b0e50b640 1 -- 172.21.15.5:0/2727878461 wait complete. 2024-09-19T06:23:15.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.205+0000 7f7b0e50b640 1 Processor -- start 2024-09-19T06:23:15.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.205+0000 7f7b0e50b640 1 -- start start 2024-09-19T06:23:15.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.206+0000 7f7b0e50b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b081081f0 0x7f7b081a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.206+0000 7f7b0e50b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b08108b50 0x7f7b081a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.206+0000 7f7b0e50b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0810e230 0x7f7b0819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.206+0000 7f7b0e50b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b08118450 con 0x7f7b081081f0 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.206+0000 7f7b0e50b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081182d0 con 0x7f7b08108b50 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.206+0000 7f7b0e50b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081185d0 con 0x7f7b0810e230 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.206+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b081081f0 0x7f7b081a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.206+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b08108b50 0x7f7b081a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.206+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b081081f0 0x7f7b081a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34274/0 (socket says 172.21.15.5:34274) 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.206+0000 7f7b07fff640 1 -- 172.21.15.5:0/1882791609 learned_addr learned my addr 172.21.15.5:0/1882791609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.206+0000 7f7b0caa8640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0810e230 0x7f7b0819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b077fe640 1 -- 172.21.15.5:0/1882791609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0810e230 msgr2=0x7f7b0819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b077fe640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0810e230 0x7f7b0819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b077fe640 1 -- 172.21.15.5:0/1882791609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b081081f0 msgr2=0x7f7b081a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b077fe640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b081081f0 0x7f7b081a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b077fe640 1 -- 172.21.15.5:0/1882791609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0819ef60 con 0x7f7b08108b50 2024-09-19T06:23:15.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b07fff640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b081081f0 0x7f7b081a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:15.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b0caa8640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0810e230 0x7f7b0819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:15.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b077fe640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b08108b50 0x7f7b081a4b80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7afc02a930 tx=0x7f7afc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:15.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b057fa640 1 -- 172.21.15.5:0/1882791609 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc03f070 con 0x7f7b08108b50 2024-09-19T06:23:15.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b057fa640 1 -- 172.21.15.5:0/1882791609 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7afc03bb10 con 0x7f7b08108b50 2024-09-19T06:23:15.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b057fa640 1 -- 172.21.15.5:0/1882791609 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc035030 con 0x7f7b08108b50 2024-09-19T06:23:15.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b0e50b640 1 -- 172.21.15.5:0/1882791609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0819f1f0 con 0x7f7b08108b50 2024-09-19T06:23:15.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.207+0000 7f7b0e50b640 1 -- 172.21.15.5:0/1882791609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b081ab4a0 con 0x7f7b08108b50 2024-09-19T06:23:15.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.210+0000 7f7b057fa640 1 -- 172.21.15.5:0/1882791609 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7afc042070 con 0x7f7b08108b50 2024-09-19T06:23:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.210+0000 7f7b108e4740 1 Processor -- start 2024-09-19T06:23:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b108e4740 1 -- start start 2024-09-19T06:23:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b108e4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 0x55f4bb05f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b108e4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb05f5d0 0x55f4bb05fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b108e4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb06a370 0x55f4bb06c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b108e4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4bafc52f0 con 0x55f4bb060b40 2024-09-19T06:23:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b108e4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f4baf6ccf0 con 0x55f4bb06a370 2024-09-19T06:23:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b108e4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f4bafc12b0 con 0x55f4bb05f5d0 2024-09-19T06:23:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b057fa640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7af00a33b0 0x7f7af00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b0caa8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 0x55f4bb05f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b0caa8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 0x55f4bb05f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34288/0 (socket says 172.21.15.5:34288) 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b0caa8640 1 -- 172.21.15.5:0/308494504 learned_addr learned my addr 172.21.15.5:0/308494504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b07fff640 1 --2- 172.21.15.5:0/308494504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb05f5d0 0x55f4bb05fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b077fe640 1 --2- 172.21.15.5:0/308494504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb06a370 0x55f4bb06c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b0caa8640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7af00a33b0 0x7f7af00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.211+0000 7f7b057fa640 1 -- 172.21.15.5:0/1882791609 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f7afc0f0350 con 0x7f7b08108b50 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.212+0000 7f7b07fff640 1 -- 172.21.15.5:0/308494504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb06a370 msgr2=0x55f4bb06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.212+0000 7f7b07fff640 1 --2- 172.21.15.5:0/308494504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb06a370 0x55f4bb06c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.212+0000 7f7b07fff640 1 -- 172.21.15.5:0/308494504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 msgr2=0x55f4bb05f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.212+0000 7f7b07fff640 1 --2- 172.21.15.5:0/308494504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 0x55f4bb05f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.212+0000 7f7b07fff640 1 -- 172.21.15.5:0/308494504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4baf513a0 con 0x55f4bb05f5d0 2024-09-19T06:23:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.212+0000 7f7b0caa8640 1 --2- 172.21.15.5:0/308494504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 0x55f4bb05f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.212+0000 7f7b0caa8640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7af00a33b0 0x7f7af00a5870 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7b00005c40 tx=0x7f7b00004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.212+0000 7f7b07fff640 1 --2- 172.21.15.5:0/308494504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb05f5d0 0x55f4bb05fa10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7af8010480 tx=0x7f7af800bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2b1db5f8aadfc2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.212+0000 7f7aee7fc640 1 -- 172.21.15.5:0/308494504 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8011820 con 0x55f4bb05f5d0 2024-09-19T06:23:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.212+0000 7f7aee7fc640 1 -- 172.21.15.5:0/308494504 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7af801c480 con 0x55f4bb05f5d0 2024-09-19T06:23:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.212+0000 7f7aee7fc640 1 -- 172.21.15.5:0/308494504 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af801a5c0 con 0x55f4bb05f5d0 2024-09-19T06:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.213+0000 7f7b108e4740 1 -- 172.21.15.5:0/308494504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb05f5d0 msgr2=0x55f4bb05fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.213+0000 7f7b108e4740 1 --2- 172.21.15.5:0/308494504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb05f5d0 0x55f4bb05fa10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7af8010480 tx=0x7f7af800bf90 comp rx=0 tx=0).stop 2024-09-19T06:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.213+0000 7f7b108e4740 1 -- 172.21.15.5:0/308494504 shutdown_connections 2024-09-19T06:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.213+0000 7f7b108e4740 1 --2- 172.21.15.5:0/308494504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb06a370 0x55f4bb06c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.213+0000 7f7b108e4740 1 --2- 172.21.15.5:0/308494504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb05f5d0 0x55f4bb05fa10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.213+0000 7f7b108e4740 1 --2- 172.21.15.5:0/308494504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 0x55f4bb05f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.213+0000 7f7b108e4740 1 -- 172.21.15.5:0/308494504 >> 172.21.15.5:0/308494504 conn(0x55f4bb028310 msgr2=0x55f4bb05bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.214+0000 7f7b108e4740 1 -- 172.21.15.5:0/308494504 shutdown_connections 2024-09-19T06:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.214+0000 7f7b108e4740 1 -- 172.21.15.5:0/308494504 wait complete. 2024-09-19T06:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.214+0000 7f7b108e4740 1 -- start start 2024-09-19T06:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.214+0000 7f7b108e4740 1 Processor -- start 2024-09-19T06:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.215+0000 7f7b108e4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 0x55f4bb069d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.215+0000 7f7b108e4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb060b40 0x55f4bb066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.215+0000 7f7b108e4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb06a370 0x55f4bb067530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.215+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 0x55f4bb069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.215+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 0x55f4bb069d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34292/0 (socket says 172.21.15.5:34292) 2024-09-19T06:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.215+0000 7f7b108e4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb06d110 con 0x55f4bb05f5d0 2024-09-19T06:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.215+0000 7f7b108e4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f4baf80060 con 0x55f4bb060b40 2024-09-19T06:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.215+0000 7f7b108e4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb06d290 con 0x55f4bb06a370 2024-09-19T06:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.215+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb060b40 0x55f4bb066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.215+0000 7f7b108e4740 10 client.0 _ll_register_callbacks cb 0x55f4bacd38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.215+0000 7f7b07fff640 1 -- 172.21.15.5:0/1561098862 learned_addr learned my addr 172.21.15.5:0/1561098862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.215+0000 7f7b0caa8640 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb06a370 0x55f4bb067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.216+0000 7f7b07fff640 1 -- 172.21.15.5:0/1561098862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb06a370 msgr2=0x55f4bb067530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.216+0000 7f7b07fff640 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb06a370 0x55f4bb067530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.216+0000 7f7b07fff640 1 -- 172.21.15.5:0/1561098862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb060b40 msgr2=0x55f4bb066fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.216+0000 7f7b07fff640 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb060b40 0x55f4bb066fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.216+0000 7f7b07fff640 1 -- 172.21.15.5:0/1561098862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4bad363c0 con 0x55f4bb05f5d0 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.216+0000 7f7b077fe640 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb060b40 0x55f4bb066fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.216+0000 7f7b07fff640 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 0x55f4bb069d00 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f7af801c840 tx=0x7f7af801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39babf23bfcb6aec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.216+0000 7f7ae77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.216+0000 7f7ae77fe640 1 -- 172.21.15.5:0/1561098862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af802b060 con 0x55f4bb05f5d0 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.216+0000 7f7b108e4740 1 -- 172.21.15.5:0/1561098862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7af801c480 con 0x55f4bb05f5d0 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.217+0000 7f7ae77fe640 1 -- 172.21.15.5:0/1561098862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f7af8011830 con 0x55f4bb05f5d0 2024-09-19T06:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.217+0000 7f7ae77fe640 1 -- 172.21.15.5:0/1561098862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f7af801c990 con 0x55f4bb05f5d0 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.217+0000 7f7b108e4740 10 client.20454 fetch_fsmap learned FSMap version 59 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.217+0000 7f7b108e4740 1 -- 172.21.15.5:0/1561098862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55f4bb078b40 con 0x55f4bb05f5d0 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.217+0000 7f7ae77fe640 1 -- 172.21.15.5:0/1561098862 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7af801c480 con 0x55f4bb05f5d0 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.217+0000 7f7ae77fe640 1 -- 172.21.15.5:0/1561098862 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7f7af802e790 con 0x55f4bb05f5d0 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.217+0000 7f7b108e4740 10 client.20454 fetch_fsmap finished waiting for FSMap version 59 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.217+0000 7f7b108e4740 10 client.20454 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.218+0000 7f7b108e4740 20 client.20454 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.218+0000 7f7b108e4740 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4bb079ef0 0x55f4bb07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.218+0000 7f7b108e4740 4 client.20454 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.218+0000 7f7b108e4740 1 -- 172.21.15.5:0/1561098862 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4bafd2860 con 0x55f4bb079ef0 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.218+0000 7f7b077fe640 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4bb079ef0 0x55f4bb07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.218+0000 7f7ae77fe640 10 client.20454 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:23:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.218+0000 7f7b077fe640 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4bb079ef0 0x55f4bb07c2e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b598786b83be17 server_cookie=af702388989be01a in_seq=0 out_seq=0 2024-09-19T06:23:15.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.219+0000 7f7ae77fe640 1 -- 172.21.15.5:0/1561098862 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4bafd2860 con 0x55f4bb079ef0 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.219+0000 7f7ae77fe640 10 client.20454 handle_command_reply: tid=0 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.219+0000 7f7b108e4740 1 client.20454 shutdown 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.220+0000 7f7b108e4740 1 -- 172.21.15.5:0/1561098862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 msgr2=0x55f4bb069d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.220+0000 7f7b108e4740 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 0x55f4bb069d00 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f7af801c840 tx=0x7f7af801c870 comp rx=0 tx=0).stop 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.220+0000 7f7b108e4740 1 -- 172.21.15.5:0/1561098862 shutdown_connections 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.220+0000 7f7b108e4740 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4bb079ef0 0x55f4bb07c2e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.220+0000 7f7b108e4740 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb06a370 0x55f4bb067530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.220+0000 7f7b108e4740 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb060b40 0x55f4bb066fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.220+0000 7f7b108e4740 1 --2- 172.21.15.5:0/1561098862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 0x55f4bb069d00 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.220+0000 7f7b108e4740 1 -- 172.21.15.5:0/1561098862 >> 172.21.15.5:0/1561098862 conn(0x55f4bb028310 msgr2=0x55f4bb061960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.220+0000 7f7ae77fe640 0 client.20454 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.220+0000 7f7b108e4740 1 -- 172.21.15.5:0/1561098862 shutdown_connections 2024-09-19T06:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.220+0000 7f7b108e4740 1 -- 172.21.15.5:0/1561098862 wait complete. 2024-09-19T06:23:15.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.220+0000 7f7b108e4740 20 client.20454 trim_cache size 0 max 16384 2024-09-19T06:23:15.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.238+0000 7f7b108e4740 1 Processor -- start 2024-09-19T06:23:15.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.238+0000 7f7b108e4740 1 -- start start 2024-09-19T06:23:15.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b108e4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb05f5d0 0x55f4bb0763b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b108e4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 0x55f4bb06c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b108e4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb06a370 0x55f4bb06b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b108e4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b3e30 con 0x55f4bb060b40 2024-09-19T06:23:15.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b108e4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b59a0 con 0x55f4bb06a370 2024-09-19T06:23:15.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b108e4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b4500 con 0x55f4bb05f5d0 2024-09-19T06:23:15.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb05f5d0 0x55f4bb0763b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 0x55f4bb06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb05f5d0 0x55f4bb0763b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34866/0 (socket says 172.21.15.5:34866) 2024-09-19T06:23:15.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b07fff640 1 -- 172.21.15.5:0/618799924 learned_addr learned my addr 172.21.15.5:0/618799924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:15.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b077fe640 1 -- 172.21.15.5:0/618799924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb05f5d0 msgr2=0x55f4bb0763b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:15.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b077fe640 1 --2- 172.21.15.5:0/618799924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb05f5d0 0x55f4bb0763b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b077fe640 1 -- 172.21.15.5:0/618799924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb06a370 msgr2=0x55f4bb06b760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:23:15.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b077fe640 1 --2- 172.21.15.5:0/618799924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb06a370 0x55f4bb06b760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.239+0000 7f7b077fe640 1 -- 172.21.15.5:0/618799924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4bb05c960 con 0x55f4bb060b40 2024-09-19T06:23:15.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.240+0000 7f7b077fe640 1 --2- 172.21.15.5:0/618799924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 0x55f4bb06c370 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0eda20 tx=0x7f7afc0b8290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bc2b7d28abf74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:15.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.240+0000 7f7aedffb640 1 -- 172.21.15.5:0/618799924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0d5020 con 0x55f4bb060b40 2024-09-19T06:23:15.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.240+0000 7f7aedffb640 1 -- 172.21.15.5:0/618799924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7afc0c5220 con 0x55f4bb060b40 2024-09-19T06:23:15.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.240+0000 7f7aedffb640 1 -- 172.21.15.5:0/618799924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0ef040 con 0x55f4bb060b40 2024-09-19T06:23:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.241+0000 7f7b108e4740 1 -- 172.21.15.5:0/618799924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 msgr2=0x55f4bb06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.241+0000 7f7b108e4740 1 --2- 172.21.15.5:0/618799924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 0x55f4bb06c370 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0eda20 tx=0x7f7afc0b8290 comp rx=0 tx=0).stop 2024-09-19T06:23:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.241+0000 7f7b108e4740 1 -- 172.21.15.5:0/618799924 shutdown_connections 2024-09-19T06:23:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.241+0000 7f7b108e4740 1 --2- 172.21.15.5:0/618799924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb06a370 0x55f4bb06b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.241+0000 7f7b108e4740 1 --2- 172.21.15.5:0/618799924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb060b40 0x55f4bb06c370 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.241+0000 7f7b108e4740 1 --2- 172.21.15.5:0/618799924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb05f5d0 0x55f4bb0763b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.241+0000 7f7b108e4740 1 -- 172.21.15.5:0/618799924 >> 172.21.15.5:0/618799924 conn(0x55f4bb028310 msgr2=0x55f4bb065f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.241+0000 7f7b108e4740 1 -- 172.21.15.5:0/618799924 shutdown_connections 2024-09-19T06:23:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.241+0000 7f7b108e4740 1 -- 172.21.15.5:0/618799924 wait complete. 2024-09-19T06:23:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.241+0000 7f7b108e4740 1 -- start start 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b108e4740 1 Processor -- start 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b108e4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 0x55f4bb0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b108e4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb060b40 0x55f4bb072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b108e4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb06a370 0x55f4bb06b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b108e4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b52d0 con 0x55f4bb05f5d0 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b108e4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b4bd0 con 0x55f4bb060b40 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b108e4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b1e30 con 0x55f4bb06a370 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b108e4740 10 client.0 _ll_register_callbacks cb 0x7f7b00009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 0x55f4bb0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 0x55f4bb0728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34316/0 (socket says 172.21.15.5:34316) 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b07fff640 1 -- 172.21.15.5:0/413770187 learned_addr learned my addr 172.21.15.5:0/413770187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b077fe640 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb060b40 0x55f4bb072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.242+0000 7f7b0caa8640 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb06a370 0x55f4bb06b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.243+0000 7f7b07fff640 1 -- 172.21.15.5:0/413770187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb06a370 msgr2=0x55f4bb06b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.243+0000 7f7b07fff640 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb06a370 0x55f4bb06b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.243+0000 7f7b07fff640 1 -- 172.21.15.5:0/413770187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb060b40 msgr2=0x55f4bb072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.243+0000 7f7b07fff640 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb060b40 0x55f4bb072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.243+0000 7f7b07fff640 1 -- 172.21.15.5:0/413770187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4bb06bb60 con 0x55f4bb05f5d0 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.243+0000 7f7b07fff640 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 0x55f4bb0728a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f7af802ebe0 tx=0x7f7af801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4df7eca06b7256 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.243+0000 7f7ae77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.243+0000 7f7ae77fe640 1 -- 172.21.15.5:0/413770187 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af802b040 con 0x55f4bb05f5d0 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.243+0000 7f7ae77fe640 1 -- 172.21.15.5:0/413770187 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f7af802edd0 con 0x55f4bb05f5d0 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.243+0000 7f7b108e4740 1 -- 172.21.15.5:0/413770187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7afc0c5220 con 0x55f4bb05f5d0 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.244+0000 7f7ae77fe640 1 -- 172.21.15.5:0/413770187 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f7af8010dd0 con 0x55f4bb05f5d0 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.244+0000 7f7b108e4740 10 client.20460 fetch_fsmap learned FSMap version 59 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.244+0000 7f7b108e4740 1 -- 172.21.15.5:0/413770187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55f4bb072220 con 0x55f4bb05f5d0 2024-09-19T06:23:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.244+0000 7f7ae77fe640 1 -- 172.21.15.5:0/413770187 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7af80054f0 con 0x55f4bb05f5d0 2024-09-19T06:23:15.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.244+0000 7f7ae77fe640 1 -- 172.21.15.5:0/413770187 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7f7af8004660 con 0x55f4bb05f5d0 2024-09-19T06:23:15.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.244+0000 7f7b108e4740 10 client.20460 fetch_fsmap finished waiting for FSMap version 59 2024-09-19T06:23:15.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.244+0000 7f7b108e4740 10 client.20460 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:23:15.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.244+0000 7f7b108e4740 20 client.20460 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:15.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.244+0000 7f7b108e4740 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4bb0e4c20 0x55f4bb0e7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:15.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.244+0000 7f7b108e4740 4 client.20460 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:23:15.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.244+0000 7f7b108e4740 1 -- 172.21.15.5:0/413770187 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7afc0c5220 con 0x55f4bb0e4c20 2024-09-19T06:23:15.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.245+0000 7f7b077fe640 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4bb0e4c20 0x55f4bb0e7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.245+0000 7f7b077fe640 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4bb0e4c20 0x55f4bb0e7040 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6716e2b482d66c server_cookie=ebefe1f673fc38e7 in_seq=0 out_seq=0 2024-09-19T06:23:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.245+0000 7f7ae77fe640 10 client.20460 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:23:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7ae77fe640 1 -- 172.21.15.5:0/413770187 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7afc0c5220 con 0x55f4bb0e4c20 2024-09-19T06:23:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7ae77fe640 10 client.20460 handle_command_reply: tid=0 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7b108e4740 1 client.20460 shutdown 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7b108e4740 1 -- 172.21.15.5:0/413770187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 msgr2=0x55f4bb0728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7b108e4740 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 0x55f4bb0728a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f7af802ebe0 tx=0x7f7af801c480 comp rx=0 tx=0).stop 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7b108e4740 1 -- 172.21.15.5:0/413770187 shutdown_connections 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7b108e4740 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55f4bb0e4c20 0x55f4bb0e7040 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7b108e4740 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4bb06a370 0x55f4bb06b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7b108e4740 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4bb060b40 0x55f4bb072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7ae77fe640 0 client.20460 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7b108e4740 1 --2- 172.21.15.5:0/413770187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4bb05f5d0 0x55f4bb0728a0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7b108e4740 1 -- 172.21.15.5:0/413770187 >> 172.21.15.5:0/413770187 conn(0x55f4bb028310 msgr2=0x55f4bb065750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7b108e4740 1 -- 172.21.15.5:0/413770187 shutdown_connections 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7b108e4740 1 -- 172.21.15.5:0/413770187 wait complete. 2024-09-19T06:23:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.246+0000 7f7b108e4740 20 client.20460 trim_cache size 0 max 16384 2024-09-19T06:23:15.252 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:23:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.247+0000 7f7b0e50b640 1 -- 172.21.15.5:0/1882791609 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7af00a33b0 msgr2=0x7f7af00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.247+0000 7f7b0e50b640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7af00a33b0 0x7f7af00a5870 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7b00005c40 tx=0x7f7b00004ab0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.247+0000 7f7b0e50b640 1 -- 172.21.15.5:0/1882791609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b08108b50 msgr2=0x7f7b081a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.247+0000 7f7b0e50b640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b08108b50 0x7f7b081a4b80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7afc02a930 tx=0x7f7afc003b90 comp rx=0 tx=0).stop 2024-09-19T06:23:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.247+0000 7f7b0e50b640 1 -- 172.21.15.5:0/1882791609 shutdown_connections 2024-09-19T06:23:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.247+0000 7f7b0e50b640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7af00a33b0 0x7f7af00a5870 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.247+0000 7f7b0e50b640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0810e230 0x7f7b0819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.247+0000 7f7b0e50b640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b08108b50 0x7f7b081a4b80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.247+0000 7f7b0e50b640 1 --2- 172.21.15.5:0/1882791609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b081081f0 0x7f7b081a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.247+0000 7f7b0e50b640 1 -- 172.21.15.5:0/1882791609 >> 172.21.15.5:0/1882791609 conn(0x7f7b08077bb0 msgr2=0x7f7b080794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.248+0000 7f7b0e50b640 1 -- 172.21.15.5:0/1882791609 shutdown_connections 2024-09-19T06:23:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:15.248+0000 7f7b0e50b640 1 -- 172.21.15.5:0/1882791609 wait complete. 2024-09-19T06:23:15.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65597 8.71 MB/sec execute 279 sec latency 151.023 ms 2024-09-19T06:23:15.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:15 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:23:15.822 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:15 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:23:15.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:15 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:23:16.150 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:23:16.536 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:23:16.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65745 8.70 MB/sec execute 280 sec latency 95.562 ms 2024-09-19T06:23:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:16 smithi157 ceph-mon[35346]: pgmap v629: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.1 MiB/s wr, 308 op/s 2024-09-19T06:23:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:16 smithi005 ceph-mon[30955]: pgmap v629: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.1 MiB/s wr, 308 op/s 2024-09-19T06:23:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:16 smithi049 ceph-mon[34640]: pgmap v629: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.1 MiB/s wr, 308 op/s 2024-09-19T06:23:17.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65893 8.70 MB/sec execute 281 sec latency 84.396 ms 2024-09-19T06:23:18.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 66027 8.69 MB/sec execute 282 sec latency 118.121 ms 2024-09-19T06:23:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:18 smithi157 ceph-mon[35346]: pgmap v630: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 350 op/s 2024-09-19T06:23:18.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:18 smithi005 ceph-mon[30955]: pgmap v630: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 350 op/s 2024-09-19T06:23:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:18 smithi049 ceph-mon[34640]: pgmap v630: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 350 op/s 2024-09-19T06:23:19.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 66194 8.69 MB/sec execute 283 sec latency 167.757 ms 2024-09-19T06:23:19.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.804+0000 7fbd3be49640 1 -- 172.21.15.5:0/3312963181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd34102570 msgr2=0x7fbd341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:19.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.804+0000 7fbd3be49640 1 --2- 172.21.15.5:0/3312963181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd34102570 0x7fbd341029b0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009900 tx=0x7fbd2402e890 comp rx=0 tx=0).stop 2024-09-19T06:23:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.804+0000 7fbd3be49640 1 -- 172.21.15.5:0/3312963181 shutdown_connections 2024-09-19T06:23:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.804+0000 7fbd3be49640 1 --2- 172.21.15.5:0/3312963181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd34102ef0 0x7fbd3410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.804+0000 7fbd3be49640 1 --2- 172.21.15.5:0/3312963181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd34102570 0x7fbd341029b0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.804+0000 7fbd3be49640 1 --2- 172.21.15.5:0/3312963181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd34100e20 0x7fbd34101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.804+0000 7fbd3be49640 1 -- 172.21.15.5:0/3312963181 >> 172.21.15.5:0/3312963181 conn(0x7fbd340fc7d0 msgr2=0x7fbd340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.804+0000 7fbd3be49640 1 -- 172.21.15.5:0/3312963181 shutdown_connections 2024-09-19T06:23:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.804+0000 7fbd3be49640 1 -- 172.21.15.5:0/3312963181 wait complete. 2024-09-19T06:23:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.805+0000 7fbd3be49640 1 Processor -- start 2024-09-19T06:23:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.805+0000 7fbd3be49640 1 -- start start 2024-09-19T06:23:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.805+0000 7fbd3be49640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd34100e20 0x7fbd34196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.805+0000 7fbd3be49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd34102570 0x7fbd34196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.805+0000 7fbd3be49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd34102ef0 0x7fbd3419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.805+0000 7fbd3be49640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3410fd30 con 0x7fbd34100e20 2024-09-19T06:23:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.805+0000 7fbd3be49640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3410fbb0 con 0x7fbd34102ef0 2024-09-19T06:23:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.805+0000 7fbd3be49640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3410feb0 con 0x7fbd34102570 2024-09-19T06:23:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.805+0000 7fbd39be5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd34100e20 0x7fbd34196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.805+0000 7fbd39be5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd34100e20 0x7fbd34196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34320/0 (socket says 172.21.15.5:34320) 2024-09-19T06:23:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.805+0000 7fbd39be5640 1 -- 172.21.15.5:0/3147582575 learned_addr learned my addr 172.21.15.5:0/3147582575 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.805+0000 7fbd393e4640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd34102570 0x7fbd34196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.806+0000 7fbd3a3e6640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd34102ef0 0x7fbd3419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.806+0000 7fbd39be5640 1 -- 172.21.15.5:0/3147582575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd34102570 msgr2=0x7fbd34196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.806+0000 7fbd39be5640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd34102570 0x7fbd34196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.806+0000 7fbd39be5640 1 -- 172.21.15.5:0/3147582575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd34102ef0 msgr2=0x7fbd3419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.806+0000 7fbd39be5640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd34102ef0 0x7fbd3419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.806+0000 7fbd39be5640 1 -- 172.21.15.5:0/3147582575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3419b1b0 con 0x7fbd34100e20 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.806+0000 7fbd39be5640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd34100e20 0x7fbd34196200 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009990 tx=0x7fbd30009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.807+0000 7fbd2affd640 1 -- 172.21.15.5:0/3147582575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3001b070 con 0x7fbd34100e20 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.807+0000 7fbd2affd640 1 -- 172.21.15.5:0/3147582575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd30016410 con 0x7fbd34100e20 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.807+0000 7fbd2affd640 1 -- 172.21.15.5:0/3147582575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30015610 con 0x7fbd34100e20 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.807+0000 7fbd3be49640 1 -- 172.21.15.5:0/3147582575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3419b4a0 con 0x7fbd34100e20 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.807+0000 7fbd3be49640 1 -- 172.21.15.5:0/3147582575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3419b900 con 0x7fbd34100e20 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.810+0000 7fbd2affd640 1 -- 172.21.15.5:0/3147582575 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbd3001e070 con 0x7fbd34100e20 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.812+0000 7fbd2affd640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbd200a3500 0x7fbd200a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.812+0000 7fbd2affd640 1 -- 172.21.15.5:0/3147582575 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7fbd300cf000 con 0x7fbd34100e20 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.812+0000 7fbd393e4640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbd200a3500 0x7fbd200a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.812+0000 7fbd3e222740 1 Processor -- start 2024-09-19T06:23:19.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.812+0000 7fbd3e222740 1 -- start start 2024-09-19T06:23:19.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd3e222740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319aab40 0x564c319a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd3e222740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319a95d0 0x564c319a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd3e222740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319b4370 0x564c319b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd3e222740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564c3190f2f0 con 0x564c319b4370 2024-09-19T06:23:19.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd3e222740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564c318b6cf0 con 0x564c319aab40 2024-09-19T06:23:19.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd3e222740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564c3190b2b0 con 0x564c319a95d0 2024-09-19T06:23:19.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd393e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319a95d0 0x564c319a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd393e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319a95d0 0x564c319a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34896/0 (socket says 172.21.15.5:34896) 2024-09-19T06:23:19.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd393e4640 1 -- 172.21.15.5:0/873705437 learned_addr learned my addr 172.21.15.5:0/873705437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:19.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd393e4640 1 -- 172.21.15.5:0/873705437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319aab40 msgr2=0x564c319a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd393e4640 1 --2- 172.21.15.5:0/873705437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319aab40 0x564c319a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd393e4640 1 -- 172.21.15.5:0/873705437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319b4370 msgr2=0x564c319b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd393e4640 1 --2- 172.21.15.5:0/873705437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319b4370 0x564c319b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.813+0000 7fbd393e4640 1 -- 172.21.15.5:0/873705437 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c3189b3a0 con 0x564c319a95d0 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.814+0000 7fbd393e4640 1 --2- 172.21.15.5:0/873705437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319a95d0 0x564c319a9a10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbd240096c0 tx=0x7fbd24046000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50b892d3487e0509 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.814+0000 7fbd1ffff640 1 -- 172.21.15.5:0/873705437 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24035d30 con 0x564c319a95d0 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.814+0000 7fbd393e4640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbd200a3500 0x7fbd200a59c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbd24000c00 tx=0x7fbd24010370 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.815+0000 7fbd1ffff640 1 -- 172.21.15.5:0/873705437 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd24003aa0 con 0x564c319a95d0 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.815+0000 7fbd1ffff640 1 -- 172.21.15.5:0/873705437 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2404a830 con 0x564c319a95d0 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.815+0000 7fbd3e222740 1 -- 172.21.15.5:0/873705437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319a95d0 msgr2=0x564c319a9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.815+0000 7fbd3e222740 1 --2- 172.21.15.5:0/873705437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319a95d0 0x564c319a9a10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbd240096c0 tx=0x7fbd24046000 comp rx=0 tx=0).stop 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.816+0000 7fbd3e222740 1 -- 172.21.15.5:0/873705437 shutdown_connections 2024-09-19T06:23:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.816+0000 7fbd3e222740 1 --2- 172.21.15.5:0/873705437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319b4370 0x564c319b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.816+0000 7fbd3e222740 1 --2- 172.21.15.5:0/873705437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319a95d0 0x564c319a9a10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.816+0000 7fbd3e222740 1 --2- 172.21.15.5:0/873705437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319aab40 0x564c319a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.816+0000 7fbd3e222740 1 -- 172.21.15.5:0/873705437 >> 172.21.15.5:0/873705437 conn(0x564c31972310 msgr2=0x564c319a5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.816+0000 7fbd3e222740 1 -- 172.21.15.5:0/873705437 shutdown_connections 2024-09-19T06:23:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.816+0000 7fbd3e222740 1 -- 172.21.15.5:0/873705437 wait complete. 2024-09-19T06:23:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.817+0000 7fbd3e222740 1 -- start start 2024-09-19T06:23:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.817+0000 7fbd3e222740 1 Processor -- start 2024-09-19T06:23:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.818+0000 7fbd3e222740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.818+0000 7fbd3e222740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 0x564c319b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.818+0000 7fbd3e222740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319b4370 0x564c319b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.818+0000 7fbd3e222740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564c319c8070 con 0x564c319aab40 2024-09-19T06:23:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.818+0000 7fbd393e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 0x564c319b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.818+0000 7fbd3e222740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564c318ca060 con 0x564c319a95d0 2024-09-19T06:23:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.818+0000 7fbd3e222740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564c319c81f0 con 0x564c319b4370 2024-09-19T06:23:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.818+0000 7fbd3e222740 10 client.0 _ll_register_callbacks cb 0x564c3161d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.818+0000 7fbd393e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 0x564c319b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34346/0 (socket says 172.21.15.5:34346) 2024-09-19T06:23:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.818+0000 7fbd393e4640 1 -- 172.21.15.5:0/3638493082 learned_addr learned my addr 172.21.15.5:0/3638493082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.818+0000 7fbd39be5640 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.818+0000 7fbd3a3e6640 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319b4370 0x564c319b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.819+0000 7fbd39be5640 1 -- 172.21.15.5:0/3638493082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319b4370 msgr2=0x564c319b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.819+0000 7fbd39be5640 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319b4370 0x564c319b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.819+0000 7fbd39be5640 1 -- 172.21.15.5:0/3638493082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 msgr2=0x564c319b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.819+0000 7fbd39be5640 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 0x564c319b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.819+0000 7fbd39be5640 1 -- 172.21.15.5:0/3638493082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c316803c0 con 0x564c319a95d0 2024-09-19T06:23:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.819+0000 7fbd39be5640 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319b3e00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbd300cc730 tx=0x7fbd3009af70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bff640db0061e40c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.819+0000 7fbd1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:23:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.819+0000 7fbd1d7fa640 1 -- 172.21.15.5:0/3638493082 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3009f670 con 0x564c319a95d0 2024-09-19T06:23:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd1d7fa640 1 -- 172.21.15.5:0/3638493082 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7fbd3009f170 con 0x564c319a95d0 2024-09-19T06:23:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd3e222740 1 -- 172.21.15.5:0/3638493082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd24003aa0 con 0x564c319a95d0 2024-09-19T06:23:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd1d7fa640 1 -- 172.21.15.5:0/3638493082 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fbd300a3f30 con 0x564c319a95d0 2024-09-19T06:23:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd3e222740 10 client.29840 fetch_fsmap learned FSMap version 59 2024-09-19T06:23:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd3e222740 1 -- 172.21.15.5:0/3638493082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x564c319d3c40 con 0x564c319a95d0 2024-09-19T06:23:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd1d7fa640 1 -- 172.21.15.5:0/3638493082 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd300b4260 con 0x564c319a95d0 2024-09-19T06:23:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd1d7fa640 1 -- 172.21.15.5:0/3638493082 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7fbd300970c0 con 0x564c319a95d0 2024-09-19T06:23:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd3e222740 10 client.29840 fetch_fsmap finished waiting for FSMap version 59 2024-09-19T06:23:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd3e222740 10 client.29840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd3e222740 20 client.29840 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd3e222740 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564c319d5420 0x564c319d7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd3e222740 4 client.29840 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.820+0000 7fbd3e222740 1 -- 172.21.15.5:0/3638493082 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbd24003aa0 con 0x564c319d5420 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.821+0000 7fbd393e4640 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564c319d5420 0x564c319d7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.821+0000 7fbd1d7fa640 10 client.29840 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.821+0000 7fbd393e4640 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564c319d5420 0x564c319d7880 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b8abf5a9642092 server_cookie=ca76ee0ae233d059 in_seq=0 out_seq=0 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.822+0000 7fbd1d7fa640 1 -- 172.21.15.5:0/3638493082 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fbd24003aa0 con 0x564c319d5420 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.822+0000 7fbd1d7fa640 10 client.29840 handle_command_reply: tid=0 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.822+0000 7fbd3e222740 1 client.29840 shutdown 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.822+0000 7fbd3e222740 1 -- 172.21.15.5:0/3638493082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 msgr2=0x564c319b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.823+0000 7fbd3e222740 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319b3e00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbd300cc730 tx=0x7fbd3009af70 comp rx=0 tx=0).stop 2024-09-19T06:23:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.823+0000 7fbd3e222740 1 -- 172.21.15.5:0/3638493082 shutdown_connections 2024-09-19T06:23:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.823+0000 7fbd3e222740 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564c319d5420 0x564c319d7880 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.823+0000 7fbd3e222740 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319b4370 0x564c319b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.823+0000 7fbd3e222740 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 0x564c319b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.823+0000 7fbd3e222740 1 --2- 172.21.15.5:0/3638493082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319b3e00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.823+0000 7fbd3e222740 1 -- 172.21.15.5:0/3638493082 >> 172.21.15.5:0/3638493082 conn(0x564c31972310 msgr2=0x564c319ab960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.823+0000 7fbd1d7fa640 0 client.29840 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.824+0000 7fbd3e222740 1 -- 172.21.15.5:0/3638493082 shutdown_connections 2024-09-19T06:23:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.824+0000 7fbd3e222740 1 -- 172.21.15.5:0/3638493082 wait complete. 2024-09-19T06:23:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.824+0000 7fbd3e222740 20 client.29840 trim_cache size 0 max 16384 2024-09-19T06:23:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd3e222740 1 Processor -- start 2024-09-19T06:23:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd3e222740 1 -- start start 2024-09-19T06:23:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd3e222740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd3e222740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 0x564c319b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd3e222740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319b4370 0x564c319dd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd3e222740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341ab6b0 con 0x564c319aab40 2024-09-19T06:23:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd3e222740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341ad1a0 con 0x564c319a95d0 2024-09-19T06:23:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd3e222740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341abd00 con 0x564c319b4370 2024-09-19T06:23:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd393e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 0x564c319b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd393e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 0x564c319b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34358/0 (socket says 172.21.15.5:34358) 2024-09-19T06:23:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd393e4640 1 -- 172.21.15.5:0/2710539880 learned_addr learned my addr 172.21.15.5:0/2710539880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd39be5640 1 --2- 172.21.15.5:0/2710539880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.843+0000 7fbd3a3e6640 1 --2- 172.21.15.5:0/2710539880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319b4370 0x564c319dd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.844+0000 7fbd393e4640 1 -- 172.21.15.5:0/2710539880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319b4370 msgr2=0x564c319dd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.844+0000 7fbd393e4640 1 --2- 172.21.15.5:0/2710539880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319b4370 0x564c319dd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.844+0000 7fbd393e4640 1 -- 172.21.15.5:0/2710539880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 msgr2=0x564c319ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.844+0000 7fbd393e4640 1 --2- 172.21.15.5:0/2710539880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.844+0000 7fbd393e4640 1 -- 172.21.15.5:0/2710539880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c319d7f00 con 0x564c319aab40 2024-09-19T06:23:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.844+0000 7fbd393e4640 1 --2- 172.21.15.5:0/2710539880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 0x564c319b0090 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fbd24035bb0 tx=0x7fbd24009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c17f383a0b12f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.844+0000 7fbd1f7fe640 1 -- 172.21.15.5:0/2710539880 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240035f0 con 0x564c319aab40 2024-09-19T06:23:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.844+0000 7fbd1f7fe640 1 -- 172.21.15.5:0/2710539880 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd24003c40 con 0x564c319aab40 2024-09-19T06:23:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.844+0000 7fbd1f7fe640 1 -- 172.21.15.5:0/2710539880 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2404a920 con 0x564c319aab40 2024-09-19T06:23:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.845+0000 7fbd3e222740 1 -- 172.21.15.5:0/2710539880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 msgr2=0x564c319b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.845+0000 7fbd3e222740 1 --2- 172.21.15.5:0/2710539880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 0x564c319b0090 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fbd24035bb0 tx=0x7fbd24009680 comp rx=0 tx=0).stop 2024-09-19T06:23:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.846+0000 7fbd3e222740 1 -- 172.21.15.5:0/2710539880 shutdown_connections 2024-09-19T06:23:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.846+0000 7fbd3e222740 1 --2- 172.21.15.5:0/2710539880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319b4370 0x564c319dd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.846+0000 7fbd3e222740 1 --2- 172.21.15.5:0/2710539880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319aab40 0x564c319b0090 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.846+0000 7fbd3e222740 1 --2- 172.21.15.5:0/2710539880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.846+0000 7fbd3e222740 1 -- 172.21.15.5:0/2710539880 >> 172.21.15.5:0/2710539880 conn(0x564c31972310 msgr2=0x564c319af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.846+0000 7fbd3e222740 1 -- 172.21.15.5:0/2710539880 shutdown_connections 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.846+0000 7fbd3e222740 1 -- 172.21.15.5:0/2710539880 wait complete. 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.846+0000 7fbd3e222740 1 -- start start 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.847+0000 7fbd3e222740 1 Processor -- start 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.847+0000 7fbd3e222740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd3e222740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319aab40 0x564c319c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd3e222740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319b4370 0x564c31a38120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd3e222740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341acad0 con 0x564c319b4370 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd3e222740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341ac3d0 con 0x564c319a95d0 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd3e222740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341a9510 con 0x564c319aab40 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd3e222740 10 client.0 _ll_register_callbacks cb 0x7fbd30005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd39be5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd39be5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43030/0 (socket says 172.21.15.5:43030) 2024-09-19T06:23:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd39be5640 1 -- 172.21.15.5:0/82094353 learned_addr learned my addr 172.21.15.5:0/82094353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd393e4640 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319aab40 0x564c319c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd3a3e6640 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319b4370 0x564c31a38120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd39be5640 1 -- 172.21.15.5:0/82094353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319aab40 msgr2=0x564c319c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd39be5640 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319aab40 0x564c319c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd39be5640 1 -- 172.21.15.5:0/82094353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319b4370 msgr2=0x564c31a38120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd39be5640 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319b4370 0x564c31a38120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd39be5640 1 -- 172.21.15.5:0/82094353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c31a38760 con 0x564c319a95d0 2024-09-19T06:23:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd3a3e6640 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319b4370 0x564c31a38120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:23:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd39be5640 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319c8c40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000c640 tx=0x7fbd3009f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86f033ab6be20808 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.848+0000 7fbd1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:23:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.849+0000 7fbd1d7fa640 1 -- 172.21.15.5:0/82094353 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3009b140 con 0x564c319a95d0 2024-09-19T06:23:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.849+0000 7fbd1d7fa640 1 -- 172.21.15.5:0/82094353 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7fbd300bb0c0 con 0x564c319a95d0 2024-09-19T06:23:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.849+0000 7fbd3e222740 1 -- 172.21.15.5:0/82094353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd24003c40 con 0x564c319a95d0 2024-09-19T06:23:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.849+0000 7fbd1d7fa640 1 -- 172.21.15.5:0/82094353 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fbd300b4260 con 0x564c319a95d0 2024-09-19T06:23:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.849+0000 7fbd3e222740 10 client.29846 fetch_fsmap learned FSMap version 59 2024-09-19T06:23:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.849+0000 7fbd3e222740 1 -- 172.21.15.5:0/82094353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x564c31a3fab0 con 0x564c319a95d0 2024-09-19T06:23:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.849+0000 7fbd1d7fa640 1 -- 172.21.15.5:0/82094353 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd300a5060 con 0x564c319a95d0 2024-09-19T06:23:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.849+0000 7fbd1d7fa640 1 -- 172.21.15.5:0/82094353 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7fbd300a0890 con 0x564c319a95d0 2024-09-19T06:23:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.849+0000 7fbd3e222740 10 client.29846 fetch_fsmap finished waiting for FSMap version 59 2024-09-19T06:23:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.850+0000 7fbd3e222740 10 client.29846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.850+0000 7fbd3e222740 20 client.29846 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.850+0000 7fbd3e222740 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564c31a401c0 0x564c31a425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.850+0000 7fbd3e222740 4 client.29846 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.850+0000 7fbd3e222740 1 -- 172.21.15.5:0/82094353 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbd24003c40 con 0x564c31a401c0 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.850+0000 7fbd393e4640 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564c31a401c0 0x564c31a425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.850+0000 7fbd1d7fa640 10 client.29846 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.850+0000 7fbd393e4640 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564c31a401c0 0x564c31a425b0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a40a4f6f31a3788a server_cookie=cd2e4585fcf70db0 in_seq=0 out_seq=0 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.850+0000 7fbd1d7fa640 1 -- 172.21.15.5:0/82094353 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbd24003c40 con 0x564c31a401c0 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.850+0000 7fbd1d7fa640 10 client.29846 handle_command_reply: tid=0 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.850+0000 7fbd3e222740 1 client.29846 shutdown 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.851+0000 7fbd3e222740 1 -- 172.21.15.5:0/82094353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 msgr2=0x564c319c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.851+0000 7fbd3e222740 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319c8c40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000c640 tx=0x7fbd3009f420 comp rx=0 tx=0).stop 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.851+0000 7fbd3e222740 1 -- 172.21.15.5:0/82094353 shutdown_connections 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.851+0000 7fbd3e222740 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564c31a401c0 0x564c31a425b0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.851+0000 7fbd3e222740 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c319b4370 0x564c31a38120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.851+0000 7fbd3e222740 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c319aab40 0x564c319c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.851+0000 7fbd1d7fa640 0 client.29846 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.851+0000 7fbd3e222740 1 --2- 172.21.15.5:0/82094353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c319a95d0 0x564c319c8c40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.851+0000 7fbd3e222740 1 -- 172.21.15.5:0/82094353 >> 172.21.15.5:0/82094353 conn(0x564c31972310 msgr2=0x564c319b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.851+0000 7fbd3e222740 1 -- 172.21.15.5:0/82094353 shutdown_connections 2024-09-19T06:23:19.655 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:23:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.851+0000 7fbd3e222740 1 -- 172.21.15.5:0/82094353 wait complete. 2024-09-19T06:23:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.851+0000 7fbd3e222740 20 client.29846 trim_cache size 0 max 16384 2024-09-19T06:23:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.852+0000 7fbd3be49640 1 -- 172.21.15.5:0/3147582575 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbd200a3500 msgr2=0x7fbd200a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.852+0000 7fbd3be49640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbd200a3500 0x7fbd200a59c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbd24000c00 tx=0x7fbd24010370 comp rx=0 tx=0).stop 2024-09-19T06:23:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.852+0000 7fbd3be49640 1 -- 172.21.15.5:0/3147582575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd34100e20 msgr2=0x7fbd34196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.852+0000 7fbd3be49640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd34100e20 0x7fbd34196200 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009990 tx=0x7fbd30009e50 comp rx=0 tx=0).stop 2024-09-19T06:23:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.852+0000 7fbd3be49640 1 -- 172.21.15.5:0/3147582575 shutdown_connections 2024-09-19T06:23:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.852+0000 7fbd3be49640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd34102ef0 0x7fbd3419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.852+0000 7fbd3be49640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbd200a3500 0x7fbd200a59c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.852+0000 7fbd3be49640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd34102570 0x7fbd34196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.852+0000 7fbd3be49640 1 --2- 172.21.15.5:0/3147582575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd34100e20 0x7fbd34196200 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.852+0000 7fbd3be49640 1 -- 172.21.15.5:0/3147582575 >> 172.21.15.5:0/3147582575 conn(0x7fbd340fc7d0 msgr2=0x7fbd340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.852+0000 7fbd3be49640 1 -- 172.21.15.5:0/3147582575 shutdown_connections 2024-09-19T06:23:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:18.852+0000 7fbd3be49640 1 -- 172.21.15.5:0/3147582575 wait complete. 2024-09-19T06:23:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.060+0000 7ff5ad45e640 1 -- 172.21.15.5:0/2500995491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5a8102460 msgr2=0x7ff5a810e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.060+0000 7ff5ad45e640 1 --2- 172.21.15.5:0/2500995491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5a8102460 0x7ff5a810e990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009ca0 tx=0x7ff5a002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.060+0000 7ff5ad45e640 1 -- 172.21.15.5:0/2500995491 shutdown_connections 2024-09-19T06:23:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.060+0000 7ff5ad45e640 1 --2- 172.21.15.5:0/2500995491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5a8102460 0x7ff5a810e990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.060+0000 7ff5ad45e640 1 --2- 172.21.15.5:0/2500995491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8101ac0 0x7ff5a8101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.060+0000 7ff5ad45e640 1 --2- 172.21.15.5:0/2500995491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a8107ad0 0x7ff5a8107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.060+0000 7ff5ad45e640 1 -- 172.21.15.5:0/2500995491 >> 172.21.15.5:0/2500995491 conn(0x7ff5a80fd7a0 msgr2=0x7ff5a80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.061+0000 7ff5ad45e640 1 -- 172.21.15.5:0/2500995491 shutdown_connections 2024-09-19T06:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.061+0000 7ff5ad45e640 1 -- 172.21.15.5:0/2500995491 wait complete. 2024-09-19T06:23:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.061+0000 7ff5ad45e640 1 Processor -- start 2024-09-19T06:23:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.061+0000 7ff5ad45e640 1 -- start start 2024-09-19T06:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5ad45e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8101ac0 0x7ff5a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5ad45e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a8102460 0x7ff5a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5ad45e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5a8107ad0 0x7ff5a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5ad45e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a8111e10 con 0x7ff5a8101ac0 2024-09-19T06:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5ad45e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a8111c90 con 0x7ff5a8107ad0 2024-09-19T06:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5ad45e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a8111f90 con 0x7ff5a8102460 2024-09-19T06:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5a67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a8102460 0x7ff5a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8101ac0 0x7ff5a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5a77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5a8107ad0 0x7ff5a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5a77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5a8107ad0 0x7ff5a8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43036/0 (socket says 172.21.15.5:43036) 2024-09-19T06:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5a77fe640 1 -- 172.21.15.5:0/492476202 learned_addr learned my addr 172.21.15.5:0/492476202 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8101ac0 0x7ff5a80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34388/0 (socket says 172.21.15.5:34388) 2024-09-19T06:23:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5a67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a8102460 0x7ff5a8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34950/0 (socket says 172.21.15.5:34950) 2024-09-19T06:23:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5a77fe640 1 -- 172.21.15.5:0/492476202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a8102460 msgr2=0x7ff5a8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5a77fe640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a8102460 0x7ff5a8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5a77fe640 1 -- 172.21.15.5:0/492476202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8101ac0 msgr2=0x7ff5a80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5a77fe640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8101ac0 0x7ff5a80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.062+0000 7ff5a77fe640 1 -- 172.21.15.5:0/492476202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a81a67b0 con 0x7ff5a8107ad0 2024-09-19T06:23:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.063+0000 7ff5a77fe640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5a8107ad0 0x7ff5a8074f70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0031040 tx=0x7ff5a0032f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.063+0000 7ff58ffff640 1 -- 172.21.15.5:0/492476202 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0003b70 con 0x7ff5a8107ad0 2024-09-19T06:23:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.063+0000 7ff58ffff640 1 -- 172.21.15.5:0/492476202 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5a0003d10 con 0x7ff5a8107ad0 2024-09-19T06:23:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.063+0000 7ff5ad45e640 1 -- 172.21.15.5:0/492476202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a81a6a40 con 0x7ff5a8107ad0 2024-09-19T06:23:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.063+0000 7ff58ffff640 1 -- 172.21.15.5:0/492476202 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0030ab0 con 0x7ff5a8107ad0 2024-09-19T06:23:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.063+0000 7ff5ad45e640 1 -- 172.21.15.5:0/492476202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a81a6f20 con 0x7ff5a8107ad0 2024-09-19T06:23:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.066+0000 7ff58ffff640 1 -- 172.21.15.5:0/492476202 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff5a0004040 con 0x7ff5a8107ad0 2024-09-19T06:23:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.066+0000 7ff58ffff640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff5900a3430 0x7ff5900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.067+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff5900a3430 0x7ff5900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.067+0000 7ff58ffff640 1 -- 172.21.15.5:0/492476202 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7ff5a00f0f60 con 0x7ff5a8107ad0 2024-09-19T06:23:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.067+0000 7ff5b01f7740 1 Processor -- start 2024-09-19T06:23:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.067+0000 7ff5b01f7740 1 -- start start 2024-09-19T06:23:20.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.067+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff5900a3430 0x7ff5900a58f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff598009b50 tx=0x7ff598005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.067+0000 7ff5b01f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 0x5602644f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5b01f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644f3410 0x5602644fd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5b01f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644fdcb0 0x5602645000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 0x5602644f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 0x5602644f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34398/0 (socket says 172.21.15.5:34398) 2024-09-19T06:23:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a67fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644f3410 0x5602644fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a6ffd640 1 -- 172.21.15.5:0/3605897883 learned_addr learned my addr 172.21.15.5:0/3605897883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5b01f7740 1 -- 172.21.15.5:0/3605897883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5602644572f0 con 0x5602644f2af0 2024-09-19T06:23:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5b01f7740 1 -- 172.21.15.5:0/3605897883 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5602643ff270 con 0x5602644f3410 2024-09-19T06:23:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5b01f7740 1 -- 172.21.15.5:0/3605897883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5602644532b0 con 0x5602644fdcb0 2024-09-19T06:23:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a77fe640 1 --2- 172.21.15.5:0/3605897883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644fdcb0 0x5602645000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a6ffd640 1 -- 172.21.15.5:0/3605897883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644fdcb0 msgr2=0x5602645000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/3605897883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644fdcb0 0x5602645000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a6ffd640 1 -- 172.21.15.5:0/3605897883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644f3410 msgr2=0x5602644fd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/3605897883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644f3410 0x5602644fd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a6ffd640 1 -- 172.21.15.5:0/3605897883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602643fe340 con 0x5602644f2af0 2024-09-19T06:23:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a77fe640 1 --2- 172.21.15.5:0/3605897883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644fdcb0 0x5602645000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:23:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a67fc640 1 --2- 172.21.15.5:0/3605897883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644f3410 0x5602644fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.068+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/3605897883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 0x5602644f2ed0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7ff598009670 tx=0x7ff598014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be9088e30bf7e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.069+0000 7ff58d7fa640 1 -- 172.21.15.5:0/3605897883 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59801e420 con 0x5602644f2af0 2024-09-19T06:23:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.069+0000 7ff58d7fa640 1 -- 172.21.15.5:0/3605897883 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff59801ea70 con 0x5602644f2af0 2024-09-19T06:23:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.069+0000 7ff58d7fa640 1 -- 172.21.15.5:0/3605897883 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59801db00 con 0x5602644f2af0 2024-09-19T06:23:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.070+0000 7ff5b01f7740 1 -- 172.21.15.5:0/3605897883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 msgr2=0x5602644f2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.070+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/3605897883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 0x5602644f2ed0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7ff598009670 tx=0x7ff598014000 comp rx=0 tx=0).stop 2024-09-19T06:23:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.070+0000 7ff5b01f7740 1 -- 172.21.15.5:0/3605897883 shutdown_connections 2024-09-19T06:23:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.070+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/3605897883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644fdcb0 0x5602645000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.070+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/3605897883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644f3410 0x5602644fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.070+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/3605897883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 0x5602644f2ed0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.070+0000 7ff5b01f7740 1 -- 172.21.15.5:0/3605897883 >> 172.21.15.5:0/3605897883 conn(0x5602644ef060 msgr2=0x5602644ef440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.070+0000 7ff5b01f7740 1 -- 172.21.15.5:0/3605897883 shutdown_connections 2024-09-19T06:23:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.070+0000 7ff5b01f7740 1 -- 172.21.15.5:0/3605897883 wait complete. 2024-09-19T06:23:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.070+0000 7ff5b01f7740 1 -- start start 2024-09-19T06:23:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.071+0000 7ff5b01f7740 1 Processor -- start 2024-09-19T06:23:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.071+0000 7ff5b01f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 0x5602644fac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.071+0000 7ff5b01f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644f3410 0x5602644fb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.071+0000 7ff5a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 0x5602644fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.071+0000 7ff5a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 0x5602644fac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34402/0 (socket says 172.21.15.5:34402) 2024-09-19T06:23:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.071+0000 7ff5a6ffd640 1 -- 172.21.15.5:0/2904521886 learned_addr learned my addr 172.21.15.5:0/2904521886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.071+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644fdcb0 0x5602644fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.072+0000 7ff5a67fc640 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644f3410 0x5602644fb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.072+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2904521886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560264500880 con 0x5602644f2af0 2024-09-19T06:23:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.072+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2904521886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5602643e0290 con 0x5602644f3410 2024-09-19T06:23:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.072+0000 7ff5a77fe640 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644fdcb0 0x5602644fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.072+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2904521886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560264500a00 con 0x5602644fdcb0 2024-09-19T06:23:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.072+0000 7ff5b01f7740 10 client.0 _ll_register_callbacks cb 0x560264411140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.072+0000 7ff5a67fc640 1 -- 172.21.15.5:0/2904521886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644fdcb0 msgr2=0x5602644fb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.072+0000 7ff5a67fc640 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644fdcb0 0x5602644fb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.072+0000 7ff5a67fc640 1 -- 172.21.15.5:0/2904521886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 msgr2=0x5602644fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.072+0000 7ff5a67fc640 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 0x5602644fac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.072+0000 7ff5a67fc640 1 -- 172.21.15.5:0/2904521886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026416a210 con 0x5602644f3410 2024-09-19T06:23:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.072+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 0x5602644fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.073+0000 7ff5a67fc640 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644f3410 0x5602644fb170 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8075f80 tx=0x7ff59c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d99189fe5f0a72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.073+0000 7ff58aff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:23:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.073+0000 7ff58aff5640 1 -- 172.21.15.5:0/2904521886 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c004030 con 0x5602644f3410 2024-09-19T06:23:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.073+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2904521886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff59801ea70 con 0x5602644f3410 2024-09-19T06:23:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.073+0000 7ff58aff5640 1 -- 172.21.15.5:0/2904521886 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7ff59c0041d0 con 0x5602644f3410 2024-09-19T06:23:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.073+0000 7ff58aff5640 1 -- 172.21.15.5:0/2904521886 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff59c0043d0 con 0x5602644f3410 2024-09-19T06:23:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.073+0000 7ff5b01f7740 10 client.29858 fetch_fsmap learned FSMap version 59 2024-09-19T06:23:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.073+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2904521886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x560264236620 con 0x5602644f3410 2024-09-19T06:23:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.073+0000 7ff58aff5640 1 -- 172.21.15.5:0/2904521886 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff59801ea70 con 0x5602644f3410 2024-09-19T06:23:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.073+0000 7ff58aff5640 1 -- 172.21.15.5:0/2904521886 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7ff59c003b20 con 0x5602644f3410 2024-09-19T06:23:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.073+0000 7ff5b01f7740 10 client.29858 fetch_fsmap finished waiting for FSMap version 59 2024-09-19T06:23:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.074+0000 7ff5b01f7740 10 client.29858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.074+0000 7ff5b01f7740 20 client.29858 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.074+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56026450dcb0 0x5602645100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.074+0000 7ff5b01f7740 4 client.29858 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.074+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2904521886 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560264464a20 con 0x56026450dcb0 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.074+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56026450dcb0 0x5602645100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.074+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56026450dcb0 0x5602645100e0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca6096d5ed58708 server_cookie=74587832671655af in_seq=0 out_seq=0 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.074+0000 7ff58aff5640 10 client.29858 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff58aff5640 1 -- 172.21.15.5:0/2904521886 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560264464a20 con 0x56026450dcb0 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff58aff5640 10 client.29858 handle_command_reply: tid=0 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff5b01f7740 1 client.29858 shutdown 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2904521886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644f3410 msgr2=0x5602644fb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644f3410 0x5602644fb170 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8075f80 tx=0x7ff59c012b60 comp rx=0 tx=0).stop 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2904521886 shutdown_connections 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56026450dcb0 0x5602645100e0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff58aff5640 0 client.29858 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644fdcb0 0x5602644fb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644f3410 0x5602644fb170 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/2904521886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f2af0 0x5602644fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2904521886 >> 172.21.15.5:0/2904521886 conn(0x5602644ef060 msgr2=0x5602644f4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2904521886 shutdown_connections 2024-09-19T06:23:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2904521886 wait complete. 2024-09-19T06:23:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.075+0000 7ff5b01f7740 20 client.29858 trim_cache size 0 max 16384 2024-09-19T06:23:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.102+0000 7ff5b01f7740 1 Processor -- start 2024-09-19T06:23:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.102+0000 7ff5b01f7740 1 -- start start 2024-09-19T06:23:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.102+0000 7ff5b01f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644f2af0 0x5602644f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.102+0000 7ff5b01f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 0x5602644fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.102+0000 7ff5b01f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644fdcb0 0x5602644fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.102+0000 7ff5b01f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81af9a0 con 0x5602644f3410 2024-09-19T06:23:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.102+0000 7ff5b01f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b1510 con 0x5602644fdcb0 2024-09-19T06:23:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.102+0000 7ff5b01f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b0070 con 0x5602644f2af0 2024-09-19T06:23:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.102+0000 7ff5a67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 0x5602644fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.102+0000 7ff5a67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 0x5602644fb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34414/0 (socket says 172.21.15.5:34414) 2024-09-19T06:23:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.102+0000 7ff5a67fc640 1 -- 172.21.15.5:0/2489955185 learned_addr learned my addr 172.21.15.5:0/2489955185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.102+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/2489955185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644f2af0 0x5602644f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.103+0000 7ff5a77fe640 1 --2- 172.21.15.5:0/2489955185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644fdcb0 0x5602644fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.103+0000 7ff5a67fc640 1 -- 172.21.15.5:0/2489955185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644f2af0 msgr2=0x5602644f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.103+0000 7ff5a67fc640 1 --2- 172.21.15.5:0/2489955185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644f2af0 0x5602644f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.103+0000 7ff5a67fc640 1 -- 172.21.15.5:0/2489955185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644fdcb0 msgr2=0x5602644fcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.103+0000 7ff5a67fc640 1 --2- 172.21.15.5:0/2489955185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644fdcb0 0x5602644fcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.103+0000 7ff5a67fc640 1 -- 172.21.15.5:0/2489955185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602644f9160 con 0x5602644f3410 2024-09-19T06:23:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.103+0000 7ff5a67fc640 1 --2- 172.21.15.5:0/2489955185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 0x5602644fb4e0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7ff59c000c00 tx=0x7ff59c005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431ee74db74cd059 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.103+0000 7ff58cff9640 1 -- 172.21.15.5:0/2489955185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c028070 con 0x5602644f3410 2024-09-19T06:23:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.103+0000 7ff58cff9640 1 -- 172.21.15.5:0/2489955185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff59c00fc40 con 0x5602644f3410 2024-09-19T06:23:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.103+0000 7ff58cff9640 1 -- 172.21.15.5:0/2489955185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c003940 con 0x5602644f3410 2024-09-19T06:23:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.104+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2489955185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 msgr2=0x5602644fb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.104+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/2489955185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 0x5602644fb4e0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7ff59c000c00 tx=0x7ff59c005580 comp rx=0 tx=0).stop 2024-09-19T06:23:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.104+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2489955185 shutdown_connections 2024-09-19T06:23:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.104+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/2489955185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644fdcb0 0x5602644fcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.104+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/2489955185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 0x5602644fb4e0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.104+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/2489955185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644f2af0 0x5602644f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.104+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2489955185 >> 172.21.15.5:0/2489955185 conn(0x5602644ef060 msgr2=0x5602644fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.104+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2489955185 shutdown_connections 2024-09-19T06:23:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.104+0000 7ff5b01f7740 1 -- 172.21.15.5:0/2489955185 wait complete. 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.105+0000 7ff5b01f7740 1 -- start start 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.105+0000 7ff5b01f7740 1 Processor -- start 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.105+0000 7ff5b01f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644f2af0 0x560264506120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.105+0000 7ff5b01f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 0x560264501440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.105+0000 7ff5b01f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644fdcb0 0x560264501980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.105+0000 7ff5b01f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b0e40 con 0x5602644f3410 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.105+0000 7ff5b01f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b0740 con 0x5602644fdcb0 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.105+0000 7ff5b01f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81ad9a0 con 0x5602644f2af0 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.105+0000 7ff5b01f7740 10 client.0 _ll_register_callbacks cb 0x7ff59c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.105+0000 7ff5a77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644fdcb0 0x560264501980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.105+0000 7ff5a6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644f2af0 0x560264506120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.105+0000 7ff5a67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 0x560264501440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff5a77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644fdcb0 0x560264501980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43090/0 (socket says 172.21.15.5:43090) 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff5a77fe640 1 -- 172.21.15.5:0/1173175911 learned_addr learned my addr 172.21.15.5:0/1173175911 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff5a6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644f2af0 0x560264506120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34992/0 (socket says 172.21.15.5:34992) 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff5a67fc640 1 -- 172.21.15.5:0/1173175911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644f2af0 msgr2=0x560264506120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff5a67fc640 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644f2af0 0x560264506120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff5a67fc640 1 -- 172.21.15.5:0/1173175911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644fdcb0 msgr2=0x560264501980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff5a67fc640 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644fdcb0 0x560264501980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff5a67fc640 1 -- 172.21.15.5:0/1173175911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560264501ec0 con 0x5602644f3410 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff5a77fe640 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644fdcb0 0x560264501980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644f2af0 0x560264506120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff5a67fc640 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 0x560264501440 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x5602644f0000 tx=0x7ff59c00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb79aa4a73532d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff58aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff58aff5640 1 -- 172.21.15.5:0/1173175911 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c031020 con 0x5602644f3410 2024-09-19T06:23:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff5b01f7740 1 -- 172.21.15.5:0/1173175911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff59c00fc40 con 0x5602644f3410 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.106+0000 7ff58aff5640 1 -- 172.21.15.5:0/1173175911 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7ff59c003870 con 0x5602644f3410 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.107+0000 7ff58aff5640 1 -- 172.21.15.5:0/1173175911 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff59c03c050 con 0x5602644f3410 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.107+0000 7ff5b01f7740 10 client.20493 fetch_fsmap learned FSMap version 59 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.107+0000 7ff5b01f7740 1 -- 172.21.15.5:0/1173175911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x560264578720 con 0x5602644f3410 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.107+0000 7ff58aff5640 1 -- 172.21.15.5:0/1173175911 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff59c00fc40 con 0x5602644f3410 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.107+0000 7ff58aff5640 1 -- 172.21.15.5:0/1173175911 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7ff59c010cd0 con 0x5602644f3410 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.107+0000 7ff5b01f7740 10 client.20493 fetch_fsmap finished waiting for FSMap version 59 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.107+0000 7ff5b01f7740 10 client.20493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.107+0000 7ff5b01f7740 20 client.20493 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.107+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56026450dcb0 0x56026457ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.107+0000 7ff5b01f7740 4 client.20493 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.107+0000 7ff5b01f7740 1 -- 172.21.15.5:0/1173175911 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56026445ed70 con 0x56026450dcb0 2024-09-19T06:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.107+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56026450dcb0 0x56026457ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.108+0000 7ff5a6ffd640 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56026450dcb0 0x56026457ae30 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85d53a8a4c99deb6 server_cookie=f58dcea60bc07bff in_seq=0 out_seq=0 2024-09-19T06:23:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.108+0000 7ff58aff5640 10 client.20493 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.174+0000 7ff58aff5640 1 -- 172.21.15.5:0/1173175911 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56026445ed70 con 0x56026450dcb0 2024-09-19T06:23:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.174+0000 7ff58aff5640 10 client.20493 handle_command_reply: tid=0 2024-09-19T06:23:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.174+0000 7ff5b01f7740 1 client.20493 shutdown 2024-09-19T06:23:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.174+0000 7ff5b01f7740 1 -- 172.21.15.5:0/1173175911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 msgr2=0x560264501440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.174+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 0x560264501440 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x5602644f0000 tx=0x7ff59c00fa40 comp rx=0 tx=0).stop 2024-09-19T06:23:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.174+0000 7ff5b01f7740 1 -- 172.21.15.5:0/1173175911 shutdown_connections 2024-09-19T06:23:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.174+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56026450dcb0 0x56026457ae30 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.174+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602644fdcb0 0x560264501980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.174+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602644f3410 0x560264501440 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.174+0000 7ff58aff5640 0 client.20493 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.174+0000 7ff5b01f7740 1 --2- 172.21.15.5:0/1173175911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602644f2af0 0x560264506120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.174+0000 7ff5b01f7740 1 -- 172.21.15.5:0/1173175911 >> 172.21.15.5:0/1173175911 conn(0x5602644ef060 msgr2=0x5602644ee1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5b01f7740 1 -- 172.21.15.5:0/1173175911 shutdown_connections 2024-09-19T06:23:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5b01f7740 1 -- 172.21.15.5:0/1173175911 wait complete. 2024-09-19T06:23:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5b01f7740 20 client.20493 trim_cache size 0 max 16384 2024-09-19T06:23:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:23:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:23:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:23:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:23:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5ad45e640 1 -- 172.21.15.5:0/492476202 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff5900a3430 msgr2=0x7ff5900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5ad45e640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff5900a3430 0x7ff5900a58f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff598009b50 tx=0x7ff598005f90 comp rx=0 tx=0).stop 2024-09-19T06:23:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5ad45e640 1 -- 172.21.15.5:0/492476202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5a8107ad0 msgr2=0x7ff5a8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5ad45e640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5a8107ad0 0x7ff5a8074f70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0031040 tx=0x7ff5a0032f70 comp rx=0 tx=0).stop 2024-09-19T06:23:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5ad45e640 1 -- 172.21.15.5:0/492476202 shutdown_connections 2024-09-19T06:23:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5ad45e640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5a8107ad0 0x7ff5a8074f70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5ad45e640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff5900a3430 0x7ff5900a58f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5ad45e640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a8102460 0x7ff5a8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5ad45e640 1 --2- 172.21.15.5:0/492476202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8101ac0 0x7ff5a80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5ad45e640 1 -- 172.21.15.5:0/492476202 >> 172.21.15.5:0/492476202 conn(0x7ff5a80fd7a0 msgr2=0x7ff5a80fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5ad45e640 1 -- 172.21.15.5:0/492476202 shutdown_connections 2024-09-19T06:23:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:20.175+0000 7ff5ad45e640 1 -- 172.21.15.5:0/492476202 wait complete. 2024-09-19T06:23:20.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 66383 8.70 MB/sec execute 284 sec latency 152.969 ms 2024-09-19T06:23:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:20 smithi157 ceph-mon[35346]: pgmap v631: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 336 op/s 2024-09-19T06:23:20.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:20 smithi005 ceph-mon[30955]: pgmap v631: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 336 op/s 2024-09-19T06:23:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:20 smithi049 ceph-mon[34640]: pgmap v631: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 336 op/s 2024-09-19T06:23:21.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 66540 8.70 MB/sec execute 285 sec latency 142.016 ms 2024-09-19T06:23:22.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 66670 8.68 MB/sec execute 286 sec latency 134.632 ms 2024-09-19T06:23:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:22 smithi157 ceph-mon[35346]: pgmap v632: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2024-09-19T06:23:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:22 smithi005 ceph-mon[30955]: pgmap v632: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2024-09-19T06:23:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:22 smithi049 ceph-mon[34640]: pgmap v632: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2024-09-19T06:23:23.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 66807 8.69 MB/sec execute 287 sec latency 88.309 ms 2024-09-19T06:23:24.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 66990 8.72 MB/sec execute 288 sec latency 100.798 ms 2024-09-19T06:23:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:24 smithi157 ceph-mon[35346]: pgmap v633: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 428 op/s 2024-09-19T06:23:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:24 smithi005 ceph-mon[30955]: pgmap v633: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 428 op/s 2024-09-19T06:23:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:24 smithi049 ceph-mon[34640]: pgmap v633: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 428 op/s 2024-09-19T06:23:25.222 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:23:25.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:23:25.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 67202 8.74 MB/sec execute 289 sec latency 120.684 ms 2024-09-19T06:23:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:23:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.195+0000 7fddf0f00640 1 -- 172.21.15.5:0/2371562247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddec101de0 msgr2=0x7fddec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.195+0000 7fddf0f00640 1 --2- 172.21.15.5:0/2371562247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddec101de0 0x7fddec1021c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fdddc009900 tx=0x7fdddc02a410 comp rx=0 tx=0).stop 2024-09-19T06:23:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.196+0000 7fddf0f00640 1 -- 172.21.15.5:0/2371562247 shutdown_connections 2024-09-19T06:23:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.196+0000 7fddf0f00640 1 --2- 172.21.15.5:0/2371562247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddec106f10 0x7fddec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.196+0000 7fddf0f00640 1 --2- 172.21.15.5:0/2371562247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec102790 0x7fddec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.196+0000 7fddf0f00640 1 --2- 172.21.15.5:0/2371562247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddec101de0 0x7fddec1021c0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.196+0000 7fddf0f00640 1 -- 172.21.15.5:0/2371562247 >> 172.21.15.5:0/2371562247 conn(0x7fddec0fd8e0 msgr2=0x7fddec0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.196+0000 7fddf0f00640 1 -- 172.21.15.5:0/2371562247 shutdown_connections 2024-09-19T06:23:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.196+0000 7fddf0f00640 1 -- 172.21.15.5:0/2371562247 wait complete. 2024-09-19T06:23:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.197+0000 7fddf0f00640 1 Processor -- start 2024-09-19T06:23:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.197+0000 7fddf0f00640 1 -- start start 2024-09-19T06:23:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.197+0000 7fddf0f00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec101de0 0x7fddec1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.197+0000 7fddf0f00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddec102790 0x7fddec198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.197+0000 7fddf0f00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddec106f10 0x7fddec19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.197+0000 7fddf0f00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fddec111de0 con 0x7fddec102790 2024-09-19T06:23:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.197+0000 7fddf0f00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fddec111c60 con 0x7fddec101de0 2024-09-19T06:23:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.197+0000 7fddf0f00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fddec111f60 con 0x7fddec106f10 2024-09-19T06:23:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.197+0000 7fddea59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec101de0 0x7fddec1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.197+0000 7fddead9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddec106f10 0x7fddec19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.198+0000 7fddea59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec101de0 0x7fddec1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59452/0 (socket says 172.21.15.5:59452) 2024-09-19T06:23:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.198+0000 7fddead9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddec106f10 0x7fddec19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:37454/0 (socket says 172.21.15.5:37454) 2024-09-19T06:23:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.198+0000 7fddea59c640 1 -- 172.21.15.5:0/2660305604 learned_addr learned my addr 172.21.15.5:0/2660305604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.198+0000 7fdde9d9b640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddec102790 0x7fddec198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.198+0000 7fddea59c640 1 -- 172.21.15.5:0/2660305604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddec106f10 msgr2=0x7fddec19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.198+0000 7fddea59c640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddec106f10 0x7fddec19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.198+0000 7fddea59c640 1 -- 172.21.15.5:0/2660305604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddec102790 msgr2=0x7fddec198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.198+0000 7fddea59c640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddec102790 0x7fddec198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.198+0000 7fddea59c640 1 -- 172.21.15.5:0/2660305604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddec19d390 con 0x7fddec101de0 2024-09-19T06:23:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.198+0000 7fddea59c640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec101de0 0x7fddec1983e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdddc005940 tx=0x7fdddc0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.199+0000 7fdddb7fe640 1 -- 172.21.15.5:0/2660305604 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc03f030 con 0x7fddec101de0 2024-09-19T06:23:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.199+0000 7fdddb7fe640 1 -- 172.21.15.5:0/2660305604 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdddc0044c0 con 0x7fddec101de0 2024-09-19T06:23:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.199+0000 7fdddb7fe640 1 -- 172.21.15.5:0/2660305604 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc035070 con 0x7fddec101de0 2024-09-19T06:23:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.199+0000 7fddf0f00640 1 -- 172.21.15.5:0/2660305604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddec19d620 con 0x7fddec101de0 2024-09-19T06:23:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.199+0000 7fddf0f00640 1 -- 172.21.15.5:0/2660305604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddec19dab0 con 0x7fddec101de0 2024-09-19T06:23:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.202+0000 7fddf32d9740 1 Processor -- start 2024-09-19T06:23:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.202+0000 7fddf32d9740 1 -- start start 2024-09-19T06:23:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fdddb7fe640 1 -- 172.21.15.5:0/2660305604 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fdddc042030 con 0x7fddec101de0 2024-09-19T06:23:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fddf32d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469cb60 0x56322469cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fddf32d9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469d480 0x5632246a7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fddf32d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632246a7d40 0x5632246aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fddf32d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563224602280 con 0x5632246a7d40 2024-09-19T06:23:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fddf32d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5632245a9b10 con 0x56322469cb60 2024-09-19T06:23:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fddf32d9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5632245fe240 con 0x56322469d480 2024-09-19T06:23:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fddead9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469cb60 0x56322469cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fddead9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469cb60 0x56322469cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59468/0 (socket says 172.21.15.5:59468) 2024-09-19T06:23:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fddead9d640 1 -- 172.21.15.5:0/4082632245 learned_addr learned my addr 172.21.15.5:0/4082632245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fdde9d9b640 1 --2- 172.21.15.5:0/4082632245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632246a7d40 0x5632246aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fddea59c640 1 --2- 172.21.15.5:0/4082632245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469d480 0x5632246a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fdde9d9b640 1 -- 172.21.15.5:0/4082632245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469d480 msgr2=0x5632246a7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fdde9d9b640 1 --2- 172.21.15.5:0/4082632245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469d480 0x5632246a7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fdde9d9b640 1 -- 172.21.15.5:0/4082632245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469cb60 msgr2=0x56322469cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.203+0000 7fdde9d9b640 1 --2- 172.21.15.5:0/4082632245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469cb60 0x56322469cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.204+0000 7fdde9d9b640 1 -- 172.21.15.5:0/4082632245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632245b61f0 con 0x5632246a7d40 2024-09-19T06:23:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.204+0000 7fddea59c640 1 --2- 172.21.15.5:0/4082632245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469d480 0x5632246a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:23:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.204+0000 7fddead9d640 1 --2- 172.21.15.5:0/4082632245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469cb60 0x56322469cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.204+0000 7fdde9d9b640 1 --2- 172.21.15.5:0/4082632245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632246a7d40 0x5632246aa130 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009bf0 tx=0x7fdde0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9831b99570590a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.204+0000 7fddd8ff9640 1 -- 172.21.15.5:0/4082632245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde001e070 con 0x5632246a7d40 2024-09-19T06:23:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.204+0000 7fddd8ff9640 1 -- 172.21.15.5:0/4082632245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdde0009da0 con 0x5632246a7d40 2024-09-19T06:23:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.204+0000 7fdddb7fe640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fddd40a3430 0x7fddd40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.205+0000 7fddd8ff9640 1 -- 172.21.15.5:0/4082632245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0018730 con 0x5632246a7d40 2024-09-19T06:23:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.205+0000 7fdddb7fe640 1 -- 172.21.15.5:0/2660305604 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7fdddc0effc0 con 0x7fddec101de0 2024-09-19T06:23:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.205+0000 7fddead9d640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fddd40a3430 0x7fddd40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.206+0000 7fddf32d9740 1 -- 172.21.15.5:0/4082632245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632246a7d40 msgr2=0x5632246aa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.206+0000 7fddf32d9740 1 --2- 172.21.15.5:0/4082632245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632246a7d40 0x5632246aa130 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009bf0 tx=0x7fdde0011810 comp rx=0 tx=0).stop 2024-09-19T06:23:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.206+0000 7fddead9d640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fddd40a3430 0x7fddd40a58f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009fd0 tx=0x7fdde4009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.206+0000 7fddf32d9740 1 -- 172.21.15.5:0/4082632245 shutdown_connections 2024-09-19T06:23:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.206+0000 7fddf32d9740 1 --2- 172.21.15.5:0/4082632245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632246a7d40 0x5632246aa130 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.206+0000 7fddf32d9740 1 --2- 172.21.15.5:0/4082632245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469d480 0x5632246a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.206+0000 7fddf32d9740 1 --2- 172.21.15.5:0/4082632245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469cb60 0x56322469cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.206+0000 7fddf32d9740 1 -- 172.21.15.5:0/4082632245 >> 172.21.15.5:0/4082632245 conn(0x5632246990f0 msgr2=0x5632246994d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.206+0000 7fddf32d9740 1 -- 172.21.15.5:0/4082632245 shutdown_connections 2024-09-19T06:23:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.206+0000 7fddf32d9740 1 -- 172.21.15.5:0/4082632245 wait complete. 2024-09-19T06:23:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.206+0000 7fddf32d9740 1 -- start start 2024-09-19T06:23:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddf32d9740 1 Processor -- start 2024-09-19T06:23:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddf32d9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 0x5632246a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddf32d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322469d480 0x5632246a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddf32d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632246a7d40 0x5632246a5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddf32d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5632246aa960 con 0x56322469d480 2024-09-19T06:23:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddf32d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5632245bcf10 con 0x5632246a7d40 2024-09-19T06:23:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddf32d9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5632246aaae0 con 0x56322469cb60 2024-09-19T06:23:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddea59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 0x5632246a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddea59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 0x5632246a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:37486/0 (socket says 172.21.15.5:37486) 2024-09-19T06:23:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddea59c640 1 -- 172.21.15.5:0/3859826292 learned_addr learned my addr 172.21.15.5:0/3859826292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddf32d9740 10 client.0 _ll_register_callbacks cb 0x563224585a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fdde9d9b640 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322469d480 0x5632246a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddead9d640 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632246a7d40 0x5632246a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddea59c640 1 -- 172.21.15.5:0/3859826292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632246a7d40 msgr2=0x5632246a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddea59c640 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632246a7d40 0x5632246a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.207+0000 7fddea59c640 1 -- 172.21.15.5:0/3859826292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322469d480 msgr2=0x5632246a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.208+0000 7fddea59c640 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322469d480 0x5632246a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.208+0000 7fddea59c640 1 -- 172.21.15.5:0/3859826292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632243736a0 con 0x56322469cb60 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.208+0000 7fdde9d9b640 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322469d480 0x5632246a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.208+0000 7fddea59c640 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 0x5632246a4ba0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0bc030 tx=0x7fdddc0b8690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0c2189e6ab80269 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.208+0000 7fddd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.208+0000 7fddd1ffb640 1 -- 172.21.15.5:0/3859826292 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc049050 con 0x56322469cb60 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.208+0000 7fddd1ffb640 1 -- 172.21.15.5:0/3859826292 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7fdddc0f20d0 con 0x56322469cb60 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.208+0000 7fddf32d9740 1 -- 172.21.15.5:0/3859826292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde0009da0 con 0x56322469cb60 2024-09-19T06:23:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.209+0000 7fddd1ffb640 1 -- 172.21.15.5:0/3859826292 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fdddc0f22d0 con 0x56322469cb60 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.209+0000 7fddf32d9740 10 client.29821 fetch_fsmap learned FSMap version 59 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.209+0000 7fddf32d9740 1 -- 172.21.15.5:0/3859826292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5632246aa960 con 0x56322469cb60 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.209+0000 7fddd1ffb640 1 -- 172.21.15.5:0/3859826292 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdde0009da0 con 0x56322469cb60 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.209+0000 7fddd1ffb640 1 -- 172.21.15.5:0/3859826292 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7fdddc0bc590 con 0x56322469cb60 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.209+0000 7fddf32d9740 10 client.29821 fetch_fsmap finished waiting for FSMap version 59 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.209+0000 7fddf32d9740 10 client.29821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.209+0000 7fddf32d9740 20 client.29821 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.209+0000 7fddf32d9740 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5632246b7820 0x5632246b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.209+0000 7fddf32d9740 4 client.29821 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.209+0000 7fddf32d9740 1 -- 172.21.15.5:0/3859826292 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56322460f7f0 con 0x5632246b7820 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.209+0000 7fdde9d9b640 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5632246b7820 0x5632246b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.210+0000 7fdde9d9b640 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5632246b7820 0x5632246b9c90 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=859c7758b04f1dad server_cookie=f232a0ae6adf8883 in_seq=0 out_seq=0 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.210+0000 7fddd1ffb640 10 client.29821 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddd1ffb640 1 -- 172.21.15.5:0/3859826292 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56322460f7f0 con 0x5632246b7820 2024-09-19T06:23:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddd1ffb640 10 client.29821 handle_command_reply: tid=0 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddf32d9740 1 client.29821 shutdown 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddf32d9740 1 -- 172.21.15.5:0/3859826292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 msgr2=0x5632246a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddf32d9740 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 0x5632246a4ba0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0bc030 tx=0x7fdddc0b8690 comp rx=0 tx=0).stop 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddf32d9740 1 -- 172.21.15.5:0/3859826292 shutdown_connections 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddf32d9740 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5632246b7820 0x5632246b9c90 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddf32d9740 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632246a7d40 0x5632246a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddf32d9740 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322469d480 0x5632246a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddf32d9740 1 --2- 172.21.15.5:0/3859826292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 0x5632246a4ba0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddf32d9740 1 -- 172.21.15.5:0/3859826292 >> 172.21.15.5:0/3859826292 conn(0x5632246990f0 msgr2=0x563224698270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddd1ffb640 0 client.29821 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddf32d9740 1 -- 172.21.15.5:0/3859826292 shutdown_connections 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddf32d9740 1 -- 172.21.15.5:0/3859826292 wait complete. 2024-09-19T06:23:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.211+0000 7fddf32d9740 20 client.29821 trim_cache size 0 max 16384 2024-09-19T06:23:26.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.246+0000 7fddf32d9740 1 Processor -- start 2024-09-19T06:23:26.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.246+0000 7fddf32d9740 1 -- start start 2024-09-19T06:23:26.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.246+0000 7fddf32d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322469cb60 0x5632246a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.246+0000 7fddf32d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469d480 0x5632246bf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.246+0000 7fddf32d9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632246a7d40 0x5632246bfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fddf32d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fddec0722d0 con 0x56322469cb60 2024-09-19T06:23:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fddea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322469cb60 0x5632246a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fddea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322469cb60 0x5632246a3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36722/0 (socket says 172.21.15.5:36722) 2024-09-19T06:23:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fddea59c640 1 -- 172.21.15.5:0/467204193 learned_addr learned my addr 172.21.15.5:0/467204193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fdde9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469d480 0x5632246bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fddf32d9740 1 -- 172.21.15.5:0/467204193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fddec073e40 con 0x56322469d480 2024-09-19T06:23:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fddf32d9740 1 -- 172.21.15.5:0/467204193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fddec0729a0 con 0x5632246a7d40 2024-09-19T06:23:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fddead9d640 1 --2- 172.21.15.5:0/467204193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632246a7d40 0x5632246bfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fddead9d640 1 -- 172.21.15.5:0/467204193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469d480 msgr2=0x5632246bf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fddead9d640 1 --2- 172.21.15.5:0/467204193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469d480 0x5632246bf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fddead9d640 1 -- 172.21.15.5:0/467204193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322469cb60 msgr2=0x5632246a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fddead9d640 1 --2- 172.21.15.5:0/467204193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322469cb60 0x5632246a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.247+0000 7fddead9d640 1 -- 172.21.15.5:0/467204193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56322469d2b0 con 0x5632246a7d40 2024-09-19T06:23:26.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.248+0000 7fddead9d640 1 --2- 172.21.15.5:0/467204193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632246a7d40 0x5632246bfaa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009e70 tx=0x7fdde4016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a98ed5309a642ca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:26.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.248+0000 7fddd3fff640 1 -- 172.21.15.5:0/467204193 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4004b20 con 0x5632246a7d40 2024-09-19T06:23:26.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.248+0000 7fddd3fff640 1 -- 172.21.15.5:0/467204193 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdde4004cc0 con 0x5632246a7d40 2024-09-19T06:23:26.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.248+0000 7fddd3fff640 1 -- 172.21.15.5:0/467204193 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde401c700 con 0x5632246a7d40 2024-09-19T06:23:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.249+0000 7fddf32d9740 1 -- 172.21.15.5:0/467204193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632246a7d40 msgr2=0x5632246bfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.249+0000 7fddf32d9740 1 --2- 172.21.15.5:0/467204193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632246a7d40 0x5632246bfaa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009e70 tx=0x7fdde4016000 comp rx=0 tx=0).stop 2024-09-19T06:23:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.249+0000 7fddf32d9740 1 -- 172.21.15.5:0/467204193 shutdown_connections 2024-09-19T06:23:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.249+0000 7fddf32d9740 1 --2- 172.21.15.5:0/467204193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632246a7d40 0x5632246bfaa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.249+0000 7fddf32d9740 1 --2- 172.21.15.5:0/467204193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469d480 0x5632246bf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.249+0000 7fddf32d9740 1 --2- 172.21.15.5:0/467204193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56322469cb60 0x5632246a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.249+0000 7fddf32d9740 1 -- 172.21.15.5:0/467204193 >> 172.21.15.5:0/467204193 conn(0x5632246990f0 msgr2=0x5632246a3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.249+0000 7fddf32d9740 1 -- 172.21.15.5:0/467204193 shutdown_connections 2024-09-19T06:23:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.249+0000 7fddf32d9740 1 -- 172.21.15.5:0/467204193 wait complete. 2024-09-19T06:23:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.250+0000 7fddf32d9740 1 -- start start 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.250+0000 7fddf32d9740 1 Processor -- start 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.250+0000 7fddf32d9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 0x5632246b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.250+0000 7fddf32d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469d480 0x5632246ab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.250+0000 7fddf32d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632246a7d40 0x5632246ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.250+0000 7fddea59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 0x5632246b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddf32d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fddec073770 con 0x5632246a7d40 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddf32d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fddec073070 con 0x56322469d480 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddf32d9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fddec0702d0 con 0x56322469cb60 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddea59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 0x5632246b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:37492/0 (socket says 172.21.15.5:37492) 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddea59c640 1 -- 172.21.15.5:0/3110749595 learned_addr learned my addr 172.21.15.5:0/3110749595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddf32d9740 10 client.0 _ll_register_callbacks cb 0x7fdddc0b7eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fdde9d9b640 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469d480 0x5632246ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddead9d640 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632246a7d40 0x5632246ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddea59c640 1 -- 172.21.15.5:0/3110749595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469d480 msgr2=0x5632246ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddea59c640 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469d480 0x5632246ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddea59c640 1 -- 172.21.15.5:0/3110749595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632246a7d40 msgr2=0x5632246ac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddea59c640 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632246a7d40 0x5632246ac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddea59c640 1 -- 172.21.15.5:0/3110749595 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563224729d40 con 0x56322469cb60 2024-09-19T06:23:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddea59c640 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 0x5632246b05d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0b8790 tx=0x7fdddc0c4e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=504ccf2d8439849e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:23:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddd1ffb640 1 -- 172.21.15.5:0/3110749595 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc049040 con 0x56322469cb60 2024-09-19T06:23:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddd1ffb640 1 -- 172.21.15.5:0/3110749595 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7fdddc0f3620 con 0x56322469cb60 2024-09-19T06:23:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.251+0000 7fddf32d9740 1 -- 172.21.15.5:0/3110749595 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde4004cc0 con 0x56322469cb60 2024-09-19T06:23:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.252+0000 7fddd1ffb640 1 -- 172.21.15.5:0/3110749595 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fdddc0d5250 con 0x56322469cb60 2024-09-19T06:23:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.252+0000 7fddf32d9740 10 client.29827 fetch_fsmap learned FSMap version 59 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.252+0000 7fddf32d9740 1 -- 172.21.15.5:0/3110749595 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x563224729d40 con 0x56322469cb60 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.252+0000 7fddd1ffb640 1 -- 172.21.15.5:0/3110749595 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdddc0bc330 con 0x56322469cb60 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.252+0000 7fddd1ffb640 1 -- 172.21.15.5:0/3110749595 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7fdddc0b8080 con 0x56322469cb60 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.252+0000 7fddf32d9740 10 client.29827 fetch_fsmap finished waiting for FSMap version 59 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.252+0000 7fddf32d9740 10 client.29827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.252+0000 7fddf32d9740 20 client.29827 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.252+0000 7fddf32d9740 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56322472b290 0x56322472d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.252+0000 7fddf32d9740 4 client.29827 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.252+0000 7fddf32d9740 1 -- 172.21.15.5:0/3110749595 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fdde4004cc0 con 0x56322472b290 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.253+0000 7fdde9d9b640 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56322472b290 0x56322472d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.253+0000 7fddd1ffb640 10 client.29827 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.253+0000 7fdde9d9b640 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56322472b290 0x56322472d680 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f419a42faf9ea1 server_cookie=e890ac85c21defc7 in_seq=0 out_seq=0 2024-09-19T06:23:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.254+0000 7fddd1ffb640 1 -- 172.21.15.5:0/3110749595 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fdde4004cc0 con 0x56322472b290 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.254+0000 7fddd1ffb640 10 client.29827 handle_command_reply: tid=0 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.254+0000 7fddf32d9740 1 client.29827 shutdown 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.254+0000 7fddf32d9740 1 -- 172.21.15.5:0/3110749595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 msgr2=0x5632246b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.254+0000 7fddf32d9740 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 0x5632246b05d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0b8790 tx=0x7fdddc0c4e00 comp rx=0 tx=0).stop 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.254+0000 7fddf32d9740 1 -- 172.21.15.5:0/3110749595 shutdown_connections 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.254+0000 7fddf32d9740 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56322472b290 0x56322472d680 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.254+0000 7fddf32d9740 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632246a7d40 0x5632246ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.254+0000 7fddf32d9740 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56322469d480 0x5632246ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.254+0000 7fddf32d9740 1 --2- 172.21.15.5:0/3110749595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56322469cb60 0x5632246b05d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.254+0000 7fddf32d9740 1 -- 172.21.15.5:0/3110749595 >> 172.21.15.5:0/3110749595 conn(0x5632246990f0 msgr2=0x5632246a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.255+0000 7fddd1ffb640 0 client.29827 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.255+0000 7fddf32d9740 1 -- 172.21.15.5:0/3110749595 shutdown_connections 2024-09-19T06:23:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.255+0000 7fddf32d9740 1 -- 172.21.15.5:0/3110749595 wait complete. 2024-09-19T06:23:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.255+0000 7fddf32d9740 20 client.29827 trim_cache size 0 max 16384 2024-09-19T06:23:26.260 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:23:26.260 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:23:26.260 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "9ea736b3-2c94-4757-aa77-2a38cc438dd5", 2024-09-19T06:23:26.260 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:23:26.260 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:23:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.256+0000 7fddf0f00640 1 -- 172.21.15.5:0/2660305604 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fddd40a3430 msgr2=0x7fddd40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.256+0000 7fddf0f00640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fddd40a3430 0x7fddd40a58f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009fd0 tx=0x7fdde4009500 comp rx=0 tx=0).stop 2024-09-19T06:23:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.256+0000 7fddf0f00640 1 -- 172.21.15.5:0/2660305604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec101de0 msgr2=0x7fddec1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.256+0000 7fddf0f00640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec101de0 0x7fddec1983e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdddc005940 tx=0x7fdddc0040c0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.256+0000 7fddf0f00640 1 -- 172.21.15.5:0/2660305604 shutdown_connections 2024-09-19T06:23:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.256+0000 7fddf0f00640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fddd40a3430 0x7fddd40a58f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.256+0000 7fddf0f00640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddec106f10 0x7fddec19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.256+0000 7fddf0f00640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddec102790 0x7fddec198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.256+0000 7fddf0f00640 1 --2- 172.21.15.5:0/2660305604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec101de0 0x7fddec1983e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:26.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.256+0000 7fddf0f00640 1 -- 172.21.15.5:0/2660305604 >> 172.21.15.5:0/2660305604 conn(0x7fddec0fd8e0 msgr2=0x7fddec0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:26.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.256+0000 7fddf0f00640 1 -- 172.21.15.5:0/2660305604 shutdown_connections 2024-09-19T06:23:26.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:26.256+0000 7fddf0f00640 1 -- 172.21.15.5:0/2660305604 wait complete. 2024-09-19T06:23:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:26 smithi157 ceph-mon[35346]: pgmap v634: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 270 op/s 2024-09-19T06:23:26.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 67348 8.74 MB/sec execute 290 sec latency 104.215 ms 2024-09-19T06:23:26.784 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:26 smithi005 ceph-mon[30955]: pgmap v634: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 270 op/s 2024-09-19T06:23:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:26 smithi049 ceph-mon[34640]: pgmap v634: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 270 op/s 2024-09-19T06:23:27.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:23:27.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 67478 8.73 MB/sec execute 291 sec latency 68.744 ms 2024-09-19T06:23:27.608 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:23:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:23:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:23:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:23:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:23:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:23:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:23:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:23:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:23:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:23:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:23:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:23:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:23:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:23:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:23:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:23:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:23:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:23:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:23:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:23:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:23:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:23:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:23:28.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 67595 8.72 MB/sec execute 292 sec latency 184.726 ms 2024-09-19T06:23:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:28 smithi157 ceph-mon[35346]: pgmap v635: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.7 MiB/s wr, 432 op/s 2024-09-19T06:23:28.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:28 smithi005 ceph-mon[30955]: pgmap v635: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.7 MiB/s wr, 432 op/s 2024-09-19T06:23:28.895 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:28 smithi049 ceph-mon[34640]: pgmap v635: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.7 MiB/s wr, 432 op/s 2024-09-19T06:23:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.981+0000 7f1597155640 1 -- 172.21.15.5:0/3254467711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1590100100 msgr2=0x7f1590100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.981+0000 7f1597155640 1 --2- 172.21.15.5:0/3254467711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1590100100 0x7f1590100560 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1580009900 tx=0x7f158002a410 comp rx=0 tx=0).stop 2024-09-19T06:23:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.981+0000 7f1597155640 1 -- 172.21.15.5:0/3254467711 shutdown_connections 2024-09-19T06:23:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.981+0000 7f1597155640 1 --2- 172.21.15.5:0/3254467711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1590100aa0 0x7f159010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.981+0000 7f1597155640 1 --2- 172.21.15.5:0/3254467711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1590100100 0x7f1590100560 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.981+0000 7f1597155640 1 --2- 172.21.15.5:0/3254467711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1590106b80 0x7f1590106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.981+0000 7f1597155640 1 -- 172.21.15.5:0/3254467711 >> 172.21.15.5:0/3254467711 conn(0x7f1590077230 msgr2=0x7f15900fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.981+0000 7f1597155640 1 -- 172.21.15.5:0/3254467711 shutdown_connections 2024-09-19T06:23:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.981+0000 7f1597155640 1 -- 172.21.15.5:0/3254467711 wait complete. 2024-09-19T06:23:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.982+0000 7f1597155640 1 Processor -- start 2024-09-19T06:23:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.982+0000 7f1597155640 1 -- start start 2024-09-19T06:23:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.982+0000 7f1597155640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1590100100 0x7f15901012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.982+0000 7f1597155640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1590100aa0 0x7f1590101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.982+0000 7f1597155640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1590106b80 0x7f1590101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.982+0000 7f1597155640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1590111f50 con 0x7f1590106b80 2024-09-19T06:23:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.982+0000 7f1597155640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1590111dd0 con 0x7f1590100100 2024-09-19T06:23:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.982+0000 7f1597155640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15901120d0 con 0x7f1590100aa0 2024-09-19T06:23:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.982+0000 7f1594ef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1590100100 0x7f15901012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.982+0000 7f1594ef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1590100100 0x7f15901012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59498/0 (socket says 172.21.15.5:59498) 2024-09-19T06:23:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.982+0000 7f1594ef1640 1 -- 172.21.15.5:0/2836967625 learned_addr learned my addr 172.21.15.5:0/2836967625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.982+0000 7f15956f2640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1590106b80 0x7f1590101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.983+0000 7f158ffff640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1590100aa0 0x7f1590101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.983+0000 7f1594ef1640 1 -- 172.21.15.5:0/2836967625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1590100aa0 msgr2=0x7f1590101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.983+0000 7f1594ef1640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1590100aa0 0x7f1590101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.983+0000 7f1594ef1640 1 -- 172.21.15.5:0/2836967625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1590106b80 msgr2=0x7f1590101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.983+0000 7f1594ef1640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1590106b80 0x7f1590101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.983+0000 7f1594ef1640 1 -- 172.21.15.5:0/2836967625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15901aadc0 con 0x7f1590100100 2024-09-19T06:23:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.983+0000 7f1594ef1640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1590100100 0x7f15901012e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f158400e9f0 tx=0x7f158400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.983+0000 7f158dffb640 1 -- 172.21.15.5:0/2836967625 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158400cd80 con 0x7f1590100100 2024-09-19T06:23:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.983+0000 7f158dffb640 1 -- 172.21.15.5:0/2836967625 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15840108e0 con 0x7f1590100100 2024-09-19T06:23:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.983+0000 7f1597155640 1 -- 172.21.15.5:0/2836967625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15901aaff0 con 0x7f1590100100 2024-09-19T06:23:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.983+0000 7f158dffb640 1 -- 172.21.15.5:0/2836967625 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584018920 con 0x7f1590100100 2024-09-19T06:23:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.984+0000 7f1597155640 1 -- 172.21.15.5:0/2836967625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15901ab3f0 con 0x7f1590100100 2024-09-19T06:23:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.986+0000 7f158dffb640 1 -- 172.21.15.5:0/2836967625 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1584010a90 con 0x7f1590100100 2024-09-19T06:23:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.987+0000 7f158dffb640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15780a33b0 0x7f15780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.987+0000 7f159952e740 1 Processor -- start 2024-09-19T06:23:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.987+0000 7f158dffb640 1 -- 172.21.15.5:0/2836967625 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f15840cff60 con 0x7f1590100100 2024-09-19T06:23:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.987+0000 7f159952e740 1 -- start start 2024-09-19T06:23:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.987+0000 7f158ffff640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15780a33b0 0x7f15780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.988+0000 7f159952e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7360 0x558bbe6f7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.988+0000 7f159952e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7c80 0x558bbe713150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.988+0000 7f159952e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe715a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.988+0000 7f159952e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558bbe65d2e0 con 0x558bbe6f7360 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.988+0000 7f159952e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558bbe604ce0 con 0x558bbe6f7c80 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.988+0000 7f159952e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558bbe6592a0 con 0x558bbe713690 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.988+0000 7f1594ef1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7360 0x558bbe6f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.988+0000 7f1594ef1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7360 0x558bbe6f7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36762/0 (socket says 172.21.15.5:36762) 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.988+0000 7f1594ef1640 1 -- 172.21.15.5:0/3310078945 learned_addr learned my addr 172.21.15.5:0/3310078945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.988+0000 7f158ffff640 1 --2- 172.21.15.5:0/3310078945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7c80 0x558bbe713150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.988+0000 7f15956f2640 1 --2- 172.21.15.5:0/3310078945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe715a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.989+0000 7f1594ef1640 1 -- 172.21.15.5:0/3310078945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 msgr2=0x558bbe715a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.989+0000 7f1594ef1640 1 --2- 172.21.15.5:0/3310078945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe715a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.989+0000 7f1594ef1640 1 -- 172.21.15.5:0/3310078945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7c80 msgr2=0x558bbe713150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.989+0000 7f1594ef1640 1 --2- 172.21.15.5:0/3310078945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7c80 0x558bbe713150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.989+0000 7f1594ef1640 1 -- 172.21.15.5:0/3310078945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bbe5e9440 con 0x558bbe6f7360 2024-09-19T06:23:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.989+0000 7f158ffff640 1 --2- 172.21.15.5:0/3310078945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7c80 0x558bbe713150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:23:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.989+0000 7f1594ef1640 1 --2- 172.21.15.5:0/3310078945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7360 0x558bbe6f7740 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f1584098ca0 tx=0x7f1584010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf6e89338631480 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.989+0000 7f158ffff640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15780a33b0 0x7f15780a5870 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1580005ab0 tx=0x7f1580004310 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.989+0000 7f1576ffd640 1 -- 172.21.15.5:0/3310078945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840bae60 con 0x558bbe6f7360 2024-09-19T06:23:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.989+0000 7f1576ffd640 1 -- 172.21.15.5:0/3310078945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15840a4e90 con 0x558bbe6f7360 2024-09-19T06:23:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.989+0000 7f1576ffd640 1 -- 172.21.15.5:0/3310078945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840a0120 con 0x558bbe6f7360 2024-09-19T06:23:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.990+0000 7f159952e740 1 -- 172.21.15.5:0/3310078945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7360 msgr2=0x558bbe6f7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.990+0000 7f159952e740 1 --2- 172.21.15.5:0/3310078945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7360 0x558bbe6f7740 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f1584098ca0 tx=0x7f1584010ea0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.991+0000 7f159952e740 1 -- 172.21.15.5:0/3310078945 shutdown_connections 2024-09-19T06:23:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.991+0000 7f159952e740 1 --2- 172.21.15.5:0/3310078945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe715a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.991+0000 7f159952e740 1 --2- 172.21.15.5:0/3310078945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7c80 0x558bbe713150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.991+0000 7f159952e740 1 --2- 172.21.15.5:0/3310078945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7360 0x558bbe6f7740 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.991+0000 7f159952e740 1 -- 172.21.15.5:0/3310078945 >> 172.21.15.5:0/3310078945 conn(0x558bbe6f3a30 msgr2=0x558bbe6f3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.991+0000 7f159952e740 1 -- 172.21.15.5:0/3310078945 shutdown_connections 2024-09-19T06:23:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.991+0000 7f159952e740 1 -- 172.21.15.5:0/3310078945 wait complete. 2024-09-19T06:23:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.992+0000 7f159952e740 1 -- start start 2024-09-19T06:23:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.992+0000 7f159952e740 1 Processor -- start 2024-09-19T06:23:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.993+0000 7f159952e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe713080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.993+0000 7f159952e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7c80 0x558bbe710330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.993+0000 7f1594ef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe713080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.993+0000 7f1594ef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe713080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59510/0 (socket says 172.21.15.5:59510) 2024-09-19T06:23:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.993+0000 7f158ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7c80 0x558bbe710330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.993+0000 7f158ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7c80 0x558bbe710330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36776/0 (socket says 172.21.15.5:36776) 2024-09-19T06:23:28.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.993+0000 7f1594ef1640 1 -- 172.21.15.5:0/3089119622 learned_addr learned my addr 172.21.15.5:0/3089119622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:28.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.993+0000 7f159952e740 1 --2- 172.21.15.5:0/3089119622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe710920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:28.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.993+0000 7f159952e740 1 -- 172.21.15.5:0/3089119622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558bbe716200 con 0x558bbe6f7c80 2024-09-19T06:23:28.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.993+0000 7f159952e740 1 -- 172.21.15.5:0/3089119622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558bbe618050 con 0x558bbe6f7360 2024-09-19T06:23:28.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.993+0000 7f159952e740 1 -- 172.21.15.5:0/3089119622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558bbe716380 con 0x558bbe713690 2024-09-19T06:23:28.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.993+0000 7f159952e740 10 client.0 _ll_register_callbacks cb 0x558bbe658160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.994+0000 7f158ffff640 1 -- 172.21.15.5:0/3089119622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 msgr2=0x558bbe710920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:23:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.994+0000 7f158ffff640 1 --2- 172.21.15.5:0/3089119622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe710920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.994+0000 7f158ffff640 1 -- 172.21.15.5:0/3089119622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 msgr2=0x558bbe713080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.994+0000 7f158ffff640 1 --2- 172.21.15.5:0/3089119622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe713080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.994+0000 7f158ffff640 1 -- 172.21.15.5:0/3089119622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bbe3ce3c0 con 0x558bbe6f7c80 2024-09-19T06:23:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.994+0000 7f158ffff640 1 --2- 172.21.15.5:0/3089119622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7c80 0x558bbe710330 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f158002aa00 tx=0x7f158002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7b10c1732941bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.994+0000 7f156ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:23:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.995+0000 7f156ffff640 1 -- 172.21.15.5:0/3089119622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158004d070 con 0x558bbe6f7c80 2024-09-19T06:23:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.995+0000 7f159952e740 1 -- 172.21.15.5:0/3089119622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15840a4e90 con 0x558bbe6f7c80 2024-09-19T06:23:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.995+0000 7f156ffff640 1 -- 172.21.15.5:0/3089119622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f1580004b80 con 0x558bbe6f7c80 2024-09-19T06:23:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.995+0000 7f156ffff640 1 -- 172.21.15.5:0/3089119622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f1580059050 con 0x558bbe6f7c80 2024-09-19T06:23:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.995+0000 7f159952e740 10 client.20523 fetch_fsmap learned FSMap version 59 2024-09-19T06:23:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.995+0000 7f159952e740 1 -- 172.21.15.5:0/3089119622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x558bbe721c60 con 0x558bbe6f7c80 2024-09-19T06:23:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.995+0000 7f156ffff640 1 -- 172.21.15.5:0/3089119622 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15840a4e90 con 0x558bbe6f7c80 2024-09-19T06:23:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.996+0000 7f156ffff640 1 -- 172.21.15.5:0/3089119622 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7f1580048040 con 0x558bbe6f7c80 2024-09-19T06:23:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.996+0000 7f159952e740 10 client.20523 fetch_fsmap finished waiting for FSMap version 59 2024-09-19T06:23:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.996+0000 7f159952e740 10 client.20523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.996+0000 7f159952e740 20 client.20523 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.996+0000 7f159952e740 1 --2- 172.21.15.5:0/3089119622 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558bbe723130 0x558bbe7255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.996+0000 7f159952e740 4 client.20523 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.996+0000 7f159952e740 1 -- 172.21.15.5:0/3089119622 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bbe66a850 con 0x558bbe723130 2024-09-19T06:23:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.996+0000 7f1594ef1640 1 --2- 172.21.15.5:0/3089119622 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558bbe723130 0x558bbe7255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.997+0000 7f1594ef1640 1 --2- 172.21.15.5:0/3089119622 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558bbe723130 0x558bbe7255b0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc34b92a021252f server_cookie=3da6e01e36ed26db in_seq=0 out_seq=0 2024-09-19T06:23:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.997+0000 7f156ffff640 10 client.20523 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f156ffff640 1 -- 172.21.15.5:0/3089119622 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558bbe66a850 con 0x558bbe723130 2024-09-19T06:23:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f156ffff640 10 client.20523 handle_command_reply: tid=0 2024-09-19T06:23:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f159952e740 1 client.20523 shutdown 2024-09-19T06:23:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f159952e740 1 -- 172.21.15.5:0/3089119622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7c80 msgr2=0x558bbe710330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f159952e740 1 --2- 172.21.15.5:0/3089119622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7c80 0x558bbe710330 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f158002aa00 tx=0x7f158002a410 comp rx=0 tx=0).stop 2024-09-19T06:23:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f159952e740 1 -- 172.21.15.5:0/3089119622 shutdown_connections 2024-09-19T06:23:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f159952e740 1 --2- 172.21.15.5:0/3089119622 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558bbe723130 0x558bbe7255b0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f156ffff640 0 client.20523 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f159952e740 1 --2- 172.21.15.5:0/3089119622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe710920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f159952e740 1 --2- 172.21.15.5:0/3089119622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe6f7c80 0x558bbe710330 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f159952e740 1 --2- 172.21.15.5:0/3089119622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe713080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f159952e740 1 -- 172.21.15.5:0/3089119622 >> 172.21.15.5:0/3089119622 conn(0x558bbe6f3a30 msgr2=0x558bbe6f9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f159952e740 1 -- 172.21.15.5:0/3089119622 shutdown_connections 2024-09-19T06:23:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f159952e740 1 -- 172.21.15.5:0/3089119622 wait complete. 2024-09-19T06:23:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:28.998+0000 7f159952e740 20 client.20523 trim_cache size 0 max 16384 2024-09-19T06:23:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.027+0000 7f159952e740 1 Processor -- start 2024-09-19T06:23:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f159952e740 1 -- start start 2024-09-19T06:23:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f159952e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe71f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f159952e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe70b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f159952e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe72ae80 0x558bbe72b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f159952e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15901ab710 con 0x558bbe72ae80 2024-09-19T06:23:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f159952e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15901ad210 con 0x558bbe6f7360 2024-09-19T06:23:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f159952e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15901abd70 con 0x558bbe713690 2024-09-19T06:23:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f1594ef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe71f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f158ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe70b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f15956f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe72ae80 0x558bbe72b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f1594ef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe71f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59518/0 (socket says 172.21.15.5:59518) 2024-09-19T06:23:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f1594ef1640 1 -- 172.21.15.5:0/4179495869 learned_addr learned my addr 172.21.15.5:0/4179495869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f1594ef1640 1 -- 172.21.15.5:0/4179495869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 msgr2=0x558bbe70b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.028+0000 7f1594ef1640 1 --2- 172.21.15.5:0/4179495869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe70b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.029+0000 7f1594ef1640 1 -- 172.21.15.5:0/4179495869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe72ae80 msgr2=0x558bbe72b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.029+0000 7f1594ef1640 1 --2- 172.21.15.5:0/4179495869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe72ae80 0x558bbe72b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.029+0000 7f1594ef1640 1 -- 172.21.15.5:0/4179495869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bbe6f84b0 con 0x558bbe6f7360 2024-09-19T06:23:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.029+0000 7f15956f2640 1 --2- 172.21.15.5:0/4179495869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe72ae80 0x558bbe72b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.029+0000 7f1594ef1640 1 --2- 172.21.15.5:0/4179495869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe71f4a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f158400e9c0 tx=0x7f15840a0750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db3955e1b588eab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.029+0000 7f15767fc640 1 -- 172.21.15.5:0/4179495869 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840bae60 con 0x558bbe6f7360 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.029+0000 7f15767fc640 1 -- 172.21.15.5:0/4179495869 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15840cd910 con 0x558bbe6f7360 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.029+0000 7f15767fc640 1 -- 172.21.15.5:0/4179495869 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840a0380 con 0x558bbe6f7360 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.030+0000 7f159952e740 1 -- 172.21.15.5:0/4179495869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 msgr2=0x558bbe71f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.030+0000 7f159952e740 1 --2- 172.21.15.5:0/4179495869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe71f4a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f158400e9c0 tx=0x7f15840a0750 comp rx=0 tx=0).stop 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.030+0000 7f159952e740 1 -- 172.21.15.5:0/4179495869 shutdown_connections 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.030+0000 7f159952e740 1 --2- 172.21.15.5:0/4179495869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe72ae80 0x558bbe72b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.030+0000 7f159952e740 1 --2- 172.21.15.5:0/4179495869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe70b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.030+0000 7f159952e740 1 --2- 172.21.15.5:0/4179495869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe71f4a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.030+0000 7f159952e740 1 -- 172.21.15.5:0/4179495869 >> 172.21.15.5:0/4179495869 conn(0x558bbe6f3a30 msgr2=0x558bbe7123b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.030+0000 7f159952e740 1 -- 172.21.15.5:0/4179495869 shutdown_connections 2024-09-19T06:23:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.030+0000 7f159952e740 1 -- 172.21.15.5:0/4179495869 wait complete. 2024-09-19T06:23:29.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.030+0000 7f159952e740 1 -- start start 2024-09-19T06:23:29.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f159952e740 1 Processor -- start 2024-09-19T06:23:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f159952e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe717640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f159952e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe717b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f159952e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe72ae80 0x558bbe7180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f159952e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15901acb40 con 0x558bbe72ae80 2024-09-19T06:23:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f159952e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15901ac440 con 0x558bbe6f7360 2024-09-19T06:23:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f159952e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15901a2d20 con 0x558bbe713690 2024-09-19T06:23:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f1594ef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe717640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f15956f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe72ae80 0x558bbe7180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f158ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe717b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f1594ef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe717640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59520/0 (socket says 172.21.15.5:59520) 2024-09-19T06:23:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f1594ef1640 1 -- 172.21.15.5:0/1329013700 learned_addr learned my addr 172.21.15.5:0/1329013700 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.031+0000 7f159952e740 10 client.0 _ll_register_callbacks cb 0x7f15880093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.032+0000 7f1594ef1640 1 -- 172.21.15.5:0/1329013700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 msgr2=0x558bbe717b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.032+0000 7f1594ef1640 1 --2- 172.21.15.5:0/1329013700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe717b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.032+0000 7f1594ef1640 1 -- 172.21.15.5:0/1329013700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe72ae80 msgr2=0x558bbe7180c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.032+0000 7f1594ef1640 1 --2- 172.21.15.5:0/1329013700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe72ae80 0x558bbe7180c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.032+0000 7f1594ef1640 1 -- 172.21.15.5:0/1329013700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bbe78d050 con 0x558bbe6f7360 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.032+0000 7f1594ef1640 1 --2- 172.21.15.5:0/1329013700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe717640 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f15840a4960 tx=0x7f158400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd5e25948a753d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.032+0000 7f156ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.032+0000 7f156ffff640 1 -- 172.21.15.5:0/1329013700 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840bae60 con 0x558bbe6f7360 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.032+0000 7f156ffff640 1 -- 172.21.15.5:0/1329013700 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f1584010430 con 0x558bbe6f7360 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.032+0000 7f159952e740 1 -- 172.21.15.5:0/1329013700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15840cd910 con 0x558bbe6f7360 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.033+0000 7f156ffff640 1 -- 172.21.15.5:0/1329013700 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f15840a4e90 con 0x558bbe6f7360 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.033+0000 7f159952e740 10 client.29897 fetch_fsmap learned FSMap version 59 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.033+0000 7f159952e740 1 -- 172.21.15.5:0/1329013700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x558bbe78e260 con 0x558bbe6f7360 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.033+0000 7f156ffff640 1 -- 172.21.15.5:0/1329013700 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15840cd910 con 0x558bbe6f7360 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.033+0000 7f156ffff640 1 -- 172.21.15.5:0/1329013700 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7f15840aa020 con 0x558bbe6f7360 2024-09-19T06:23:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.033+0000 7f159952e740 10 client.29897 fetch_fsmap finished waiting for FSMap version 59 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.033+0000 7f159952e740 10 client.29897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.033+0000 7f159952e740 20 client.29897 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.033+0000 7f159952e740 1 --2- 172.21.15.5:0/1329013700 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558bbe78e430 0x558bbe790820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.033+0000 7f159952e740 4 client.29897 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.033+0000 7f159952e740 1 -- 172.21.15.5:0/1329013700 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558bbe664d60 con 0x558bbe78e430 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.033+0000 7f158ffff640 1 --2- 172.21.15.5:0/1329013700 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558bbe78e430 0x558bbe790820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.034+0000 7f156ffff640 10 client.29897 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.034+0000 7f158ffff640 1 --2- 172.21.15.5:0/1329013700 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558bbe78e430 0x558bbe790820 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82288ec3e0c9a010 server_cookie=bdd46dcf139bbc64 in_seq=0 out_seq=0 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.034+0000 7f156ffff640 1 -- 172.21.15.5:0/1329013700 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558bbe664d60 con 0x558bbe78e430 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.034+0000 7f156ffff640 10 client.29897 handle_command_reply: tid=0 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.034+0000 7f159952e740 1 client.29897 shutdown 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.034+0000 7f159952e740 1 -- 172.21.15.5:0/1329013700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 msgr2=0x558bbe717640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.034+0000 7f159952e740 1 --2- 172.21.15.5:0/1329013700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe717640 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f15840a4960 tx=0x7f158400cf90 comp rx=0 tx=0).stop 2024-09-19T06:23:29.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.034+0000 7f159952e740 1 -- 172.21.15.5:0/1329013700 shutdown_connections 2024-09-19T06:23:29.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.034+0000 7f159952e740 1 --2- 172.21.15.5:0/1329013700 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558bbe78e430 0x558bbe790820 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.034+0000 7f159952e740 1 --2- 172.21.15.5:0/1329013700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bbe72ae80 0x558bbe7180c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.034+0000 7f156ffff640 0 client.29897 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:29.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.035+0000 7f159952e740 1 --2- 172.21.15.5:0/1329013700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558bbe713690 0x558bbe717b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.035+0000 7f159952e740 1 --2- 172.21.15.5:0/1329013700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bbe6f7360 0x558bbe717640 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.035+0000 7f159952e740 1 -- 172.21.15.5:0/1329013700 >> 172.21.15.5:0/1329013700 conn(0x558bbe6f3a30 msgr2=0x558bbe7123b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:29.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.035+0000 7f159952e740 1 -- 172.21.15.5:0/1329013700 shutdown_connections 2024-09-19T06:23:29.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.035+0000 7f159952e740 1 -- 172.21.15.5:0/1329013700 wait complete. 2024-09-19T06:23:29.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.035+0000 7f159952e740 20 client.29897 trim_cache size 0 max 16384 2024-09-19T06:23:29.041 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:23:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:23:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:23:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "9ea736b3-2c94-4757-aa77-2a38cc438dd5": { 2024-09-19T06:23:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:23:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "9ea736b3-2c94-4757-aa77-2a38cc438dd5", 2024-09-19T06:23:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:23:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:23:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:23:29.041 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:23:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.036+0000 7f1597155640 1 -- 172.21.15.5:0/2836967625 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15780a33b0 msgr2=0x7f15780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.036+0000 7f1597155640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15780a33b0 0x7f15780a5870 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1580005ab0 tx=0x7f1580004310 comp rx=0 tx=0).stop 2024-09-19T06:23:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.036+0000 7f1597155640 1 -- 172.21.15.5:0/2836967625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1590100100 msgr2=0x7f15901012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.036+0000 7f1597155640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1590100100 0x7f15901012e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f158400e9f0 tx=0x7f158400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.036+0000 7f1597155640 1 -- 172.21.15.5:0/2836967625 shutdown_connections 2024-09-19T06:23:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.036+0000 7f1597155640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1590106b80 0x7f1590101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.036+0000 7f1597155640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f15780a33b0 0x7f15780a5870 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.036+0000 7f1597155640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1590100aa0 0x7f1590101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.036+0000 7f1597155640 1 --2- 172.21.15.5:0/2836967625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1590100100 0x7f15901012e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.036+0000 7f1597155640 1 -- 172.21.15.5:0/2836967625 >> 172.21.15.5:0/2836967625 conn(0x7f1590077230 msgr2=0x7f15900fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.036+0000 7f1597155640 1 -- 172.21.15.5:0/2836967625 shutdown_connections 2024-09-19T06:23:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:23:29.036+0000 7f1597155640 1 -- 172.21.15.5:0/2836967625 wait complete. 2024-09-19T06:23:29.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 67769 8.72 MB/sec execute 293 sec latency 68.557 ms 2024-09-19T06:23:30.006 INFO:tasks.cephfs.filesystem:scrub status for tag:9ea736b3-2c94-4757-aa77-2a38cc438dd5 - {'path': '/', 'tag': '9ea736b3-2c94-4757-aa77-2a38cc438dd5', 'options': 'recursive,force'} 2024-09-19T06:23:30.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 67912 8.71 MB/sec execute 294 sec latency 70.363 ms 2024-09-19T06:23:30.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:30 smithi157 ceph-mon[35346]: pgmap v636: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-09-19T06:23:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:30 smithi005 ceph-mon[30955]: pgmap v636: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-09-19T06:23:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:30 smithi049 ceph-mon[34640]: pgmap v636: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-09-19T06:23:31.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 68053 8.70 MB/sec execute 295 sec latency 85.424 ms 2024-09-19T06:23:32.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 68223 8.72 MB/sec execute 296 sec latency 101.700 ms 2024-09-19T06:23:32.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:32 smithi157 ceph-mon[35346]: pgmap v637: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-09-19T06:23:32.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:32 smithi005 ceph-mon[30955]: pgmap v637: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-09-19T06:23:32.895 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:32 smithi049 ceph-mon[34640]: pgmap v637: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-09-19T06:23:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.298+0000 7f873fbe4640 1 -- 172.21.15.5:0/1864568756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8738101370 msgr2=0x7f873810d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.298+0000 7f873fbe4640 1 --2- 172.21.15.5:0/1864568756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8738101370 0x7f873810d8a0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8730009cd0 tx=0x7f873002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.299+0000 7f873fbe4640 1 -- 172.21.15.5:0/1864568756 shutdown_connections 2024-09-19T06:23:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.299+0000 7f873fbe4640 1 --2- 172.21.15.5:0/1864568756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8738101370 0x7f873810d8a0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.299+0000 7f873fbe4640 1 --2- 172.21.15.5:0/1864568756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87381009d0 0x7f8738100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.299+0000 7f873fbe4640 1 --2- 172.21.15.5:0/1864568756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87381069e0 0x7f8738106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.299+0000 7f873fbe4640 1 -- 172.21.15.5:0/1864568756 >> 172.21.15.5:0/1864568756 conn(0x7f87380fc6b0 msgr2=0x7f87380fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.299+0000 7f873fbe4640 1 -- 172.21.15.5:0/1864568756 shutdown_connections 2024-09-19T06:23:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.299+0000 7f873fbe4640 1 -- 172.21.15.5:0/1864568756 wait complete. 2024-09-19T06:23:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.299+0000 7f873fbe4640 1 Processor -- start 2024-09-19T06:23:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873fbe4640 1 -- start start 2024-09-19T06:23:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873fbe4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87381009d0 0x7f8738198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873fbe4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738101370 0x7f8738198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873fbe4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87381069e0 0x7f873819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873fbe4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8738110cd0 con 0x7f87381009d0 2024-09-19T06:23:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873fbe4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8738110b50 con 0x7f87381069e0 2024-09-19T06:23:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873fbe4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8738110e50 con 0x7f8738101370 2024-09-19T06:23:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873d980640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87381009d0 0x7f8738198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873d17f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738101370 0x7f8738198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873e181640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87381069e0 0x7f873819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873d980640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87381009d0 0x7f8738198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36820/0 (socket says 172.21.15.5:36820) 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873d980640 1 -- 172.21.15.5:0/2932080396 learned_addr learned my addr 172.21.15.5:0/2932080396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873d17f640 1 -- 172.21.15.5:0/2932080396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87381069e0 msgr2=0x7f873819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873d17f640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87381069e0 0x7f873819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873d17f640 1 -- 172.21.15.5:0/2932080396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87381009d0 msgr2=0x7f8738198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873d17f640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87381009d0 0x7f8738198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873d17f640 1 -- 172.21.15.5:0/2932080396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873819d310 con 0x7f8738101370 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.300+0000 7f873e181640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87381069e0 0x7f873819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.301+0000 7f873d17f640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738101370 0x7f8738198860 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f872800e9f0 tx=0x7f872800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.301+0000 7f872effd640 1 -- 172.21.15.5:0/2932080396 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872800ccc0 con 0x7f8738101370 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.301+0000 7f873fbe4640 1 -- 172.21.15.5:0/2932080396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873819d600 con 0x7f8738101370 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.301+0000 7f872effd640 1 -- 172.21.15.5:0/2932080396 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87280108e0 con 0x7f8738101370 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.301+0000 7f872effd640 1 -- 172.21.15.5:0/2932080396 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728018880 con 0x7f8738101370 2024-09-19T06:23:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.301+0000 7f873fbe4640 1 -- 172.21.15.5:0/2932080396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87381a4e90 con 0x7f8738101370 2024-09-19T06:23:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.304+0000 7f872effd640 1 -- 172.21.15.5:0/2932080396 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8728010a90 con 0x7f8738101370 2024-09-19T06:23:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.305+0000 7f8742a56740 1 Processor -- start 2024-09-19T06:23:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.305+0000 7f8742a56740 1 -- start start 2024-09-19T06:23:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.305+0000 7f872effd640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f87240a33b0 0x7f87240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.305+0000 7f8742a56740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520473af0 0x558520473ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.305+0000 7f8742a56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520474410 0x55852047e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.305+0000 7f8742a56740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 0x5585204810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f8742a56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585203d82f0 con 0x558520474410 2024-09-19T06:23:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f8742a56740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558520380270 con 0x55852047ecb0 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f8742a56740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5585203d42b0 con 0x558520473af0 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f872effd640 1 -- 172.21.15.5:0/2932080396 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f87280d0090 con 0x7f8738101370 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873d980640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520473af0 0x558520473ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873d980640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520473af0 0x558520473ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:37552/0 (socket says 172.21.15.5:37552) 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873d980640 1 -- 172.21.15.5:0/4280301560 learned_addr learned my addr 172.21.15.5:0/4280301560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873d17f640 1 --2- 172.21.15.5:0/4280301560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520474410 0x55852047e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873e181640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f87240a33b0 0x7f87240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873d980640 1 -- 172.21.15.5:0/4280301560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 msgr2=0x5585204810a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873d980640 1 --2- 172.21.15.5:0/4280301560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 0x5585204810a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873d980640 1 -- 172.21.15.5:0/4280301560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520474410 msgr2=0x55852047e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873d980640 1 --2- 172.21.15.5:0/4280301560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520474410 0x55852047e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873d980640 1 -- 172.21.15.5:0/4280301560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852037f340 con 0x558520473af0 2024-09-19T06:23:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873d17f640 1 --2- 172.21.15.5:0/4280301560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520474410 0x55852047e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873d980640 1 --2- 172.21.15.5:0/4280301560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520473af0 0x558520473ed0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8734009da0 tx=0x7f8734011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3b8d8742a3546c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.306+0000 7f873e181640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f87240a33b0 0x7f87240a5870 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8730004af0 tx=0x7f8730004a60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.307+0000 7f8723fff640 1 -- 172.21.15.5:0/4280301560 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873401e070 con 0x558520473af0 2024-09-19T06:23:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.307+0000 7f8723fff640 1 -- 172.21.15.5:0/4280301560 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8734019ae0 con 0x558520473af0 2024-09-19T06:23:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.307+0000 7f8723fff640 1 -- 172.21.15.5:0/4280301560 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873400fa50 con 0x558520473af0 2024-09-19T06:23:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.307+0000 7f8742a56740 1 -- 172.21.15.5:0/4280301560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520473af0 msgr2=0x558520473ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.307+0000 7f8742a56740 1 --2- 172.21.15.5:0/4280301560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520473af0 0x558520473ed0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8734009da0 tx=0x7f8734011810 comp rx=0 tx=0).stop 2024-09-19T06:23:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.308+0000 7f8742a56740 1 -- 172.21.15.5:0/4280301560 shutdown_connections 2024-09-19T06:23:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.308+0000 7f8742a56740 1 --2- 172.21.15.5:0/4280301560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 0x5585204810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.308+0000 7f8742a56740 1 --2- 172.21.15.5:0/4280301560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520474410 0x55852047e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.308+0000 7f8742a56740 1 --2- 172.21.15.5:0/4280301560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520473af0 0x558520473ed0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.308+0000 7f8742a56740 1 -- 172.21.15.5:0/4280301560 >> 172.21.15.5:0/4280301560 conn(0x558520470060 msgr2=0x558520470440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.308+0000 7f8742a56740 1 -- 172.21.15.5:0/4280301560 shutdown_connections 2024-09-19T06:23:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.308+0000 7f8742a56740 1 -- 172.21.15.5:0/4280301560 wait complete. 2024-09-19T06:23:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.308+0000 7f8742a56740 1 -- start start 2024-09-19T06:23:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.308+0000 7f8742a56740 1 Processor -- start 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f8742a56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x55852047bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f8742a56740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520474410 0x55852047c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f8742a56740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 0x55852047c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f8742a56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558520481880 con 0x558520473af0 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f873d980640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x55852047bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f8742a56740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558520361290 con 0x55852047ecb0 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f873d980640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x55852047bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36842/0 (socket says 172.21.15.5:36842) 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f8742a56740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558520481a00 con 0x558520474410 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f873d980640 1 -- 172.21.15.5:0/2820251746 learned_addr learned my addr 172.21.15.5:0/2820251746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f8742a56740 10 client.0 _ll_register_callbacks cb 0x558520392140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f873d17f640 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520474410 0x55852047c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f873e181640 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 0x55852047c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f873d980640 1 -- 172.21.15.5:0/2820251746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520474410 msgr2=0x55852047c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f873d980640 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520474410 0x55852047c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f873d980640 1 -- 172.21.15.5:0/2820251746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 msgr2=0x55852047c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f873d980640 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 0x55852047c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.309+0000 7f873d980640 1 -- 172.21.15.5:0/2820251746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585200eb210 con 0x558520473af0 2024-09-19T06:23:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.310+0000 7f873d980640 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x55852047bb30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f873400f5f0 tx=0x7f8734009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a711314e1738f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.310+0000 7f87217fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:23:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.310+0000 7f87217fa640 1 -- 172.21.15.5:0/2820251746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873401e070 con 0x558520473af0 2024-09-19T06:23:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.310+0000 7f87217fa640 1 -- 172.21.15.5:0/2820251746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f87340198c0 con 0x558520473af0 2024-09-19T06:23:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.310+0000 7f8742a56740 1 -- 172.21.15.5:0/2820251746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8734019ae0 con 0x558520473af0 2024-09-19T06:23:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.310+0000 7f87217fa640 1 -- 172.21.15.5:0/2820251746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f8734005030 con 0x558520473af0 2024-09-19T06:23:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.311+0000 7f8742a56740 10 client.20541 fetch_fsmap learned FSMap version 59 2024-09-19T06:23:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.311+0000 7f8742a56740 1 -- 172.21.15.5:0/2820251746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5585201b7620 con 0x558520473af0 2024-09-19T06:23:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.311+0000 7f87217fa640 1 -- 172.21.15.5:0/2820251746 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8734019ae0 con 0x558520473af0 2024-09-19T06:23:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.311+0000 7f87217fa640 1 -- 172.21.15.5:0/2820251746 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7f87340046c0 con 0x558520473af0 2024-09-19T06:23:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.311+0000 7f8742a56740 10 client.20541 fetch_fsmap finished waiting for FSMap version 59 2024-09-19T06:23:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.311+0000 7f8742a56740 10 client.20541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.311+0000 7f8742a56740 20 client.20541 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.311+0000 7f8742a56740 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55852048eb20 0x558520490f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.311+0000 7f8742a56740 4 client.20541 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.311+0000 7f8742a56740 1 -- 172.21.15.5:0/2820251746 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585203e5a20 con 0x55852048eb20 2024-09-19T06:23:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.311+0000 7f873d17f640 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55852048eb20 0x558520490f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.312+0000 7f87217fa640 10 client.20541 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.312+0000 7f873d17f640 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55852048eb20 0x558520490f10 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b22ada276e0809dd server_cookie=19000bb7c002e67c in_seq=0 out_seq=0 2024-09-19T06:23:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.312+0000 7f87217fa640 1 -- 172.21.15.5:0/2820251746 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585203e5a20 con 0x55852048eb20 2024-09-19T06:23:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.312+0000 7f87217fa640 10 client.20541 handle_command_reply: tid=0 2024-09-19T06:23:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f8742a56740 1 client.20541 shutdown 2024-09-19T06:23:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f8742a56740 1 -- 172.21.15.5:0/2820251746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 msgr2=0x55852047bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f8742a56740 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x55852047bb30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f873400f5f0 tx=0x7f8734009a60 comp rx=0 tx=0).stop 2024-09-19T06:23:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f8742a56740 1 -- 172.21.15.5:0/2820251746 shutdown_connections 2024-09-19T06:23:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f8742a56740 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55852048eb20 0x558520490f10 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f8742a56740 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 0x55852047c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f8742a56740 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520474410 0x55852047c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f8742a56740 1 --2- 172.21.15.5:0/2820251746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x55852047bb30 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f8742a56740 1 -- 172.21.15.5:0/2820251746 >> 172.21.15.5:0/2820251746 conn(0x558520470060 msgr2=0x558520475dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f87217fa640 0 client.20541 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f8742a56740 1 -- 172.21.15.5:0/2820251746 shutdown_connections 2024-09-19T06:23:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f8742a56740 1 -- 172.21.15.5:0/2820251746 wait complete. 2024-09-19T06:23:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.313+0000 7f8742a56740 20 client.20541 trim_cache size 0 max 16384 2024-09-19T06:23:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.341+0000 7f8742a56740 1 Processor -- start 2024-09-19T06:23:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.341+0000 7f8742a56740 1 -- start start 2024-09-19T06:23:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.341+0000 7f8742a56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x55852047daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.341+0000 7f8742a56740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520474410 0x55852047c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.341+0000 7f8742a56740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 0x55852047c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.341+0000 7f8742a56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f87381afb00 con 0x558520473af0 2024-09-19T06:23:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.341+0000 7f8742a56740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f87381b1670 con 0x55852047ecb0 2024-09-19T06:23:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.341+0000 7f8742a56740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87381b01d0 con 0x558520474410 2024-09-19T06:23:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.341+0000 7f873d980640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x55852047daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.341+0000 7f873d980640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x55852047daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36854/0 (socket says 172.21.15.5:36854) 2024-09-19T06:23:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f873d980640 1 -- 172.21.15.5:0/2019964027 learned_addr learned my addr 172.21.15.5:0/2019964027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f873d17f640 1 --2- 172.21.15.5:0/2019964027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520474410 0x55852047c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f873e181640 1 --2- 172.21.15.5:0/2019964027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 0x55852047c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f873d980640 1 -- 172.21.15.5:0/2019964027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520474410 msgr2=0x55852047c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f873d980640 1 --2- 172.21.15.5:0/2019964027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520474410 0x55852047c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f873d980640 1 -- 172.21.15.5:0/2019964027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 msgr2=0x55852047c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f873d980640 1 --2- 172.21.15.5:0/2019964027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 0x55852047c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f873d980640 1 -- 172.21.15.5:0/2019964027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852046fe90 con 0x558520473af0 2024-09-19T06:23:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f873e181640 1 --2- 172.21.15.5:0/2019964027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 0x55852047c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f873d17f640 1 --2- 172.21.15.5:0/2019964027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520474410 0x55852047c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f873d980640 1 --2- 172.21.15.5:0/2019964027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x55852047daa0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8734009cf0 tx=0x7f8734011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161831e4c4883bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f87237fe640 1 -- 172.21.15.5:0/2019964027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873401e040 con 0x558520473af0 2024-09-19T06:23:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f87237fe640 1 -- 172.21.15.5:0/2019964027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8734028070 con 0x558520473af0 2024-09-19T06:23:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.342+0000 7f87237fe640 1 -- 172.21.15.5:0/2019964027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734019920 con 0x558520473af0 2024-09-19T06:23:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.343+0000 7f8742a56740 1 -- 172.21.15.5:0/2019964027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 msgr2=0x55852047daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.343+0000 7f8742a56740 1 --2- 172.21.15.5:0/2019964027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x55852047daa0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8734009cf0 tx=0x7f8734011d30 comp rx=0 tx=0).stop 2024-09-19T06:23:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.343+0000 7f8742a56740 1 -- 172.21.15.5:0/2019964027 shutdown_connections 2024-09-19T06:23:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.343+0000 7f8742a56740 1 --2- 172.21.15.5:0/2019964027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852047ecb0 0x55852047c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.343+0000 7f8742a56740 1 --2- 172.21.15.5:0/2019964027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558520474410 0x55852047c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.343+0000 7f8742a56740 1 --2- 172.21.15.5:0/2019964027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x55852047daa0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.343+0000 7f8742a56740 1 -- 172.21.15.5:0/2019964027 >> 172.21.15.5:0/2019964027 conn(0x558520470060 msgr2=0x55852047d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.343+0000 7f8742a56740 1 -- 172.21.15.5:0/2019964027 shutdown_connections 2024-09-19T06:23:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.343+0000 7f8742a56740 1 -- 172.21.15.5:0/2019964027 wait complete. 2024-09-19T06:23:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.344+0000 7f8742a56740 1 -- start start 2024-09-19T06:23:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.344+0000 7f8742a56740 1 Processor -- start 2024-09-19T06:23:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.344+0000 7f8742a56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x558520486c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.344+0000 7f8742a56740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558520474410 0x5585204871d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.344+0000 7f8742a56740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55852047ecb0 0x558520477dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f8742a56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f87381b0fa0 con 0x558520473af0 2024-09-19T06:23:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f8742a56740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f87381b08a0 con 0x558520474410 2024-09-19T06:23:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f8742a56740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87381adb00 con 0x55852047ecb0 2024-09-19T06:23:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f873d980640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x558520486c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f8742a56740 10 client.0 _ll_register_callbacks cb 0x7f8734009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f873d980640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x558520486c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36862/0 (socket says 172.21.15.5:36862) 2024-09-19T06:23:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f873d980640 1 -- 172.21.15.5:0/4053715932 learned_addr learned my addr 172.21.15.5:0/4053715932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f873d17f640 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558520474410 0x5585204871d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f873e181640 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55852047ecb0 0x558520477dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f873d980640 1 -- 172.21.15.5:0/4053715932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55852047ecb0 msgr2=0x558520477dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f873d980640 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55852047ecb0 0x558520477dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f873d980640 1 -- 172.21.15.5:0/4053715932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558520474410 msgr2=0x5585204871d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f873d980640 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558520474410 0x5585204871d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f873d980640 1 -- 172.21.15.5:0/4053715932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558520473920 con 0x558520473af0 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f873d17f640 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558520474410 0x5585204871d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f873d980640 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x558520486c90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f8734006010 tx=0x7f87340048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86dad06c73ea1abc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.345+0000 7f87217fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.346+0000 7f87217fa640 1 -- 172.21.15.5:0/4053715932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873403c020 con 0x558520473af0 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.346+0000 7f87217fa640 1 -- 172.21.15.5:0/4053715932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f87340196c0 con 0x558520473af0 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.346+0000 7f8742a56740 1 -- 172.21.15.5:0/4053715932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8734028070 con 0x558520473af0 2024-09-19T06:23:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.346+0000 7f87217fa640 1 -- 172.21.15.5:0/4053715932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f8734047050 con 0x558520473af0 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.346+0000 7f8742a56740 10 client.20547 fetch_fsmap learned FSMap version 59 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.346+0000 7f8742a56740 1 -- 172.21.15.5:0/4053715932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5585204f8a00 con 0x558520473af0 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.346+0000 7f87217fa640 1 -- 172.21.15.5:0/4053715932 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8734028070 con 0x558520473af0 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.346+0000 7f87217fa640 1 -- 172.21.15.5:0/4053715932 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7f873400fac0 con 0x558520473af0 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.346+0000 7f8742a56740 10 client.20547 fetch_fsmap finished waiting for FSMap version 59 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.347+0000 7f8742a56740 10 client.20547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.347+0000 7f8742a56740 20 client.20547 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.347+0000 7f8742a56740 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5585204f90d0 0x5585204fb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.347+0000 7f8742a56740 4 client.20547 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.347+0000 7f8742a56740 1 -- 172.21.15.5:0/4053715932 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585203dfd70 con 0x5585204f90d0 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.347+0000 7f873d17f640 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5585204f90d0 0x5585204fb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.347+0000 7f873d17f640 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5585204f90d0 0x5585204fb4f0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c96df9a98096332 server_cookie=9cb14dfd862bb00c in_seq=0 out_seq=0 2024-09-19T06:23:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.347+0000 7f87217fa640 10 client.20547 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.500+0000 7f87217fa640 1 -- 172.21.15.5:0/4053715932 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585203dfd70 con 0x5585204f90d0 2024-09-19T06:23:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.500+0000 7f87217fa640 10 client.20547 handle_command_reply: tid=0 2024-09-19T06:23:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.500+0000 7f8742a56740 1 client.20547 shutdown 2024-09-19T06:23:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.501+0000 7f8742a56740 1 -- 172.21.15.5:0/4053715932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 msgr2=0x558520486c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.501+0000 7f8742a56740 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x558520486c90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f8734006010 tx=0x7f87340048b0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.501+0000 7f8742a56740 1 -- 172.21.15.5:0/4053715932 shutdown_connections 2024-09-19T06:23:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.501+0000 7f8742a56740 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5585204f90d0 0x5585204fb4f0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.501+0000 7f8742a56740 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55852047ecb0 0x558520477dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.501+0000 7f8742a56740 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558520474410 0x5585204871d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.501+0000 7f8742a56740 1 --2- 172.21.15.5:0/4053715932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558520473af0 0x558520486c90 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.501+0000 7f8742a56740 1 -- 172.21.15.5:0/4053715932 >> 172.21.15.5:0/4053715932 conn(0x558520470060 msgr2=0x558520479fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.501+0000 7f87217fa640 0 client.20547 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.501+0000 7f8742a56740 1 -- 172.21.15.5:0/4053715932 shutdown_connections 2024-09-19T06:23:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.501+0000 7f8742a56740 1 -- 172.21.15.5:0/4053715932 wait complete. 2024-09-19T06:23:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.501+0000 7f8742a56740 20 client.20547 trim_cache size 0 max 16384 2024-09-19T06:23:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:23:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:23:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:23:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:23:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.502+0000 7f873fbe4640 1 -- 172.21.15.5:0/2932080396 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f87240a33b0 msgr2=0x7f87240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.502+0000 7f873fbe4640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f87240a33b0 0x7f87240a5870 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8730004af0 tx=0x7f8730004a60 comp rx=0 tx=0).stop 2024-09-19T06:23:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.502+0000 7f873fbe4640 1 -- 172.21.15.5:0/2932080396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738101370 msgr2=0x7f8738198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.502+0000 7f873fbe4640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738101370 0x7f8738198860 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f872800e9f0 tx=0x7f872800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.502+0000 7f873fbe4640 1 -- 172.21.15.5:0/2932080396 shutdown_connections 2024-09-19T06:23:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.502+0000 7f873fbe4640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87381069e0 0x7f873819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.502+0000 7f873fbe4640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f87240a33b0 0x7f87240a5870 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.502+0000 7f873fbe4640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738101370 0x7f8738198860 secure :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f872800e9f0 tx=0x7f872800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.502+0000 7f873fbe4640 1 --2- 172.21.15.5:0/2932080396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87381009d0 0x7f8738198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.502+0000 7f873fbe4640 1 -- 172.21.15.5:0/2932080396 >> 172.21.15.5:0/2932080396 conn(0x7f87380fc6b0 msgr2=0x7f873806a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.502+0000 7f873fbe4640 1 -- 172.21.15.5:0/2932080396 shutdown_connections 2024-09-19T06:23:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:33.502+0000 7f873fbe4640 1 -- 172.21.15.5:0/2932080396 wait complete. 2024-09-19T06:23:33.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 68389 8.71 MB/sec execute 297 sec latency 86.689 ms 2024-09-19T06:23:34.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 68590 8.72 MB/sec execute 298 sec latency 51.648 ms 2024-09-19T06:23:34.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:34 smithi157 ceph-mon[35346]: pgmap v638: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.5 MiB/s wr, 415 op/s 2024-09-19T06:23:34.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:34 smithi005 ceph-mon[30955]: pgmap v638: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.5 MiB/s wr, 415 op/s 2024-09-19T06:23:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:34 smithi049 ceph-mon[34640]: pgmap v638: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.5 MiB/s wr, 415 op/s 2024-09-19T06:23:35.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 68713 8.70 MB/sec execute 299 sec latency 130.142 ms 2024-09-19T06:23:36.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 68861 8.69 MB/sec execute 300 sec latency 67.209 ms 2024-09-19T06:23:36.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:36 smithi157 ceph-mon[35346]: pgmap v639: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 278 op/s 2024-09-19T06:23:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:36 smithi005 ceph-mon[30955]: pgmap v639: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 278 op/s 2024-09-19T06:23:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:36 smithi049 ceph-mon[34640]: pgmap v639: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 278 op/s 2024-09-19T06:23:37.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69038 8.71 MB/sec execute 301 sec latency 139.935 ms 2024-09-19T06:23:37.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:37 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:23:37.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:37 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:23:37.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:37 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:23:38.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69176 8.70 MB/sec execute 302 sec latency 239.401 ms 2024-09-19T06:23:38.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:38 smithi157 ceph-mon[35346]: pgmap v640: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.5 MiB/s wr, 439 op/s 2024-09-19T06:23:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:38 smithi005 ceph-mon[30955]: pgmap v640: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.5 MiB/s wr, 439 op/s 2024-09-19T06:23:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:38 smithi049 ceph-mon[34640]: pgmap v640: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.5 MiB/s wr, 439 op/s 2024-09-19T06:23:39.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69334 8.69 MB/sec execute 303 sec latency 131.468 ms 2024-09-19T06:23:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:23:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:23:39.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:23:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:23:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:23:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:23:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:23:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:23:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:23:40.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69497 8.68 MB/sec execute 304 sec latency 52.172 ms 2024-09-19T06:23:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:40 smithi157 ceph-mon[35346]: pgmap v641: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-09-19T06:23:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:40 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:23:40.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:40 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:23:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:40 smithi005 ceph-mon[30955]: pgmap v641: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-09-19T06:23:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:40 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:23:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:40 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:23:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:40 smithi049 ceph-mon[34640]: pgmap v641: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-09-19T06:23:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:40 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:23:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:40 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:23:41.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69695 8.69 MB/sec execute 305 sec latency 86.801 ms 2024-09-19T06:23:42.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69905 8.69 MB/sec execute 306 sec latency 54.259 ms 2024-09-19T06:23:42.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:42 smithi157 ceph-mon[35346]: pgmap v642: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-09-19T06:23:42.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:42 smithi005 ceph-mon[30955]: pgmap v642: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-09-19T06:23:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:42 smithi049 ceph-mon[34640]: pgmap v642: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-09-19T06:23:43.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 70056 8.69 MB/sec execute 307 sec latency 64.747 ms 2024-09-19T06:23:44.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 70230 8.71 MB/sec execute 308 sec latency 59.392 ms 2024-09-19T06:23:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:44 smithi005 ceph-mon[30955]: pgmap v643: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 400 op/s 2024-09-19T06:23:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:44 smithi049 ceph-mon[34640]: pgmap v643: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 400 op/s 2024-09-19T06:23:45.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:44 smithi157 ceph-mon[35346]: pgmap v643: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 400 op/s 2024-09-19T06:23:45.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 70446 8.74 MB/sec execute 309 sec latency 128.114 ms 2024-09-19T06:23:46.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 70666 8.76 MB/sec execute 310 sec latency 73.183 ms 2024-09-19T06:23:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.636+0000 7fe343423640 1 -- 172.21.15.5:0/3966250123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe33c101150 msgr2=0x7fe33c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.636+0000 7fe343423640 1 --2- 172.21.15.5:0/3966250123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe33c101150 0x7fe33c10dd30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe330009900 tx=0x7fe33002a400 comp rx=0 tx=0).stop 2024-09-19T06:23:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.636+0000 7fe343423640 1 -- 172.21.15.5:0/3966250123 shutdown_connections 2024-09-19T06:23:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.636+0000 7fe343423640 1 --2- 172.21.15.5:0/3966250123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe33c10e350 0x7fe33c1107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.636+0000 7fe343423640 1 --2- 172.21.15.5:0/3966250123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe33c101150 0x7fe33c10dd30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.636+0000 7fe343423640 1 --2- 172.21.15.5:0/3966250123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe33c100830 0x7fe33c100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.636+0000 7fe343423640 1 -- 172.21.15.5:0/3966250123 >> 172.21.15.5:0/3966250123 conn(0x7fe33c075f50 msgr2=0x7fe33c076360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.637+0000 7fe343423640 1 -- 172.21.15.5:0/3966250123 shutdown_connections 2024-09-19T06:23:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.637+0000 7fe343423640 1 -- 172.21.15.5:0/3966250123 wait complete. 2024-09-19T06:23:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.638+0000 7fe343423640 1 Processor -- start 2024-09-19T06:23:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.638+0000 7fe343423640 1 -- start start 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.638+0000 7fe343423640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe33c100830 0x7fe33c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.638+0000 7fe343423640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe33c101150 0x7fe33c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.638+0000 7fe343423640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe33c10e350 0x7fe33c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.638+0000 7fe343423640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c113fd0 con 0x7fe33c101150 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.638+0000 7fe343423640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c113e50 con 0x7fe33c10e350 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.638+0000 7fe343423640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c114150 con 0x7fe33c100830 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.638+0000 7fe3411bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe33c100830 0x7fe33c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.638+0000 7fe3411bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe33c100830 0x7fe33c19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41052/0 (socket says 172.21.15.5:41052) 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.638+0000 7fe3411bf640 1 -- 172.21.15.5:0/1731447166 learned_addr learned my addr 172.21.15.5:0/1731447166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.638+0000 7fe3409be640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe33c101150 0x7fe33c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.639+0000 7fe3419c0640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe33c10e350 0x7fe33c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.639+0000 7fe3411bf640 1 -- 172.21.15.5:0/1731447166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe33c10e350 msgr2=0x7fe33c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.639+0000 7fe3411bf640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe33c10e350 0x7fe33c19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.639+0000 7fe3411bf640 1 -- 172.21.15.5:0/1731447166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe33c101150 msgr2=0x7fe33c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.639+0000 7fe3411bf640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe33c101150 0x7fe33c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.639+0000 7fe3411bf640 1 -- 172.21.15.5:0/1731447166 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c19f530 con 0x7fe33c100830 2024-09-19T06:23:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.639+0000 7fe3411bf640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe33c100830 0x7fe33c19a540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe33800e9f0 tx=0x7fe33800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.639+0000 7fe32e7fc640 1 -- 172.21.15.5:0/1731447166 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33800cd80 con 0x7fe33c100830 2024-09-19T06:23:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.639+0000 7fe32e7fc640 1 -- 172.21.15.5:0/1731447166 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3380108e0 con 0x7fe33c100830 2024-09-19T06:23:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.639+0000 7fe343423640 1 -- 172.21.15.5:0/1731447166 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c19f820 con 0x7fe33c100830 2024-09-19T06:23:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.639+0000 7fe32e7fc640 1 -- 172.21.15.5:0/1731447166 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338018920 con 0x7fe33c100830 2024-09-19T06:23:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.639+0000 7fe343423640 1 -- 172.21.15.5:0/1731447166 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c1a70b0 con 0x7fe33c100830 2024-09-19T06:23:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3461bc740 1 Processor -- start 2024-09-19T06:23:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe32e7fc640 1 -- 172.21.15.5:0/1731447166 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe338010a90 con 0x7fe33c100830 2024-09-19T06:23:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3461bc740 1 -- start start 2024-09-19T06:23:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3461bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a29095aaf0 0x55a29095aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3461bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29095b410 0x55a290965770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3461bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a290965cb0 0x55a2909680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3461bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a2908bf2f0 con 0x55a290965cb0 2024-09-19T06:23:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3461bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a290867270 con 0x55a29095b410 2024-09-19T06:23:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3461bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a2908bb2b0 con 0x55a29095aaf0 2024-09-19T06:23:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3411bf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29095b410 0x55a290965770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3419c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a29095aaf0 0x55a29095aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3419c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a29095aaf0 0x55a29095aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41058/0 (socket says 172.21.15.5:41058) 2024-09-19T06:23:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3411bf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29095b410 0x55a290965770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36844/0 (socket says 172.21.15.5:36844) 2024-09-19T06:23:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3419c0640 1 -- 172.21.15.5:0/3185254621 learned_addr learned my addr 172.21.15.5:0/3185254621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.643+0000 7fe3409be640 1 --2- 172.21.15.5:0/3185254621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a290965cb0 0x55a2909680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.644+0000 7fe3419c0640 1 -- 172.21.15.5:0/3185254621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29095b410 msgr2=0x55a290965770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.644+0000 7fe3419c0640 1 --2- 172.21.15.5:0/3185254621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29095b410 0x55a290965770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.644+0000 7fe3419c0640 1 -- 172.21.15.5:0/3185254621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a290965cb0 msgr2=0x55a2909680a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.644+0000 7fe3419c0640 1 --2- 172.21.15.5:0/3185254621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a290965cb0 0x55a2909680a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.644+0000 7fe3419c0640 1 -- 172.21.15.5:0/3185254621 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a290866340 con 0x55a29095aaf0 2024-09-19T06:23:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.644+0000 7fe3409be640 1 --2- 172.21.15.5:0/3185254621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a290965cb0 0x55a2909680a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.644+0000 7fe32e7fc640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe3280a34b0 0x7fe3280a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.644+0000 7fe3411bf640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe3280a34b0 0x7fe3280a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.644+0000 7fe3419c0640 1 --2- 172.21.15.5:0/3185254621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a29095aaf0 0x55a29095aed0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe334009f40 tx=0x7fe334010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdac3bf5767bcbce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.644+0000 7fe32e7fc640 1 -- 172.21.15.5:0/1731447166 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7fe3380d0080 con 0x7fe33c100830 2024-09-19T06:23:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.645+0000 7fe3277fe640 1 -- 172.21.15.5:0/3185254621 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334003b70 con 0x55a29095aaf0 2024-09-19T06:23:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.645+0000 7fe3277fe640 1 -- 172.21.15.5:0/3185254621 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe334003d10 con 0x55a29095aaf0 2024-09-19T06:23:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.645+0000 7fe3277fe640 1 -- 172.21.15.5:0/3185254621 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334011960 con 0x55a29095aaf0 2024-09-19T06:23:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.646+0000 7fe3461bc740 1 -- 172.21.15.5:0/3185254621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a29095aaf0 msgr2=0x55a29095aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.646+0000 7fe3411bf640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe3280a34b0 0x7fe3280a5970 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55a290957ec0 tx=0x7fe3380d37e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.646+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3185254621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a29095aaf0 0x55a29095aed0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe334009f40 tx=0x7fe334010860 comp rx=0 tx=0).stop 2024-09-19T06:23:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.646+0000 7fe3461bc740 1 -- 172.21.15.5:0/3185254621 shutdown_connections 2024-09-19T06:23:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.646+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3185254621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a290965cb0 0x55a2909680a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.646+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3185254621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29095b410 0x55a290965770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.646+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3185254621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a29095aaf0 0x55a29095aed0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.646+0000 7fe3461bc740 1 -- 172.21.15.5:0/3185254621 >> 172.21.15.5:0/3185254621 conn(0x55a290957060 msgr2=0x55a290957440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.646+0000 7fe3461bc740 1 -- 172.21.15.5:0/3185254621 shutdown_connections 2024-09-19T06:23:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.646+0000 7fe3461bc740 1 -- 172.21.15.5:0/3185254621 wait complete. 2024-09-19T06:23:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.646+0000 7fe3461bc740 1 -- start start 2024-09-19T06:23:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.646+0000 7fe3461bc740 1 Processor -- start 2024-09-19T06:23:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3461bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29095aaf0 0x55a290962c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3461bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 0x55a290963150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3461bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a290965cb0 0x55a290963690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3461bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a290968880 con 0x55a29095b410 2024-09-19T06:23:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3461bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a290848290 con 0x55a29095aaf0 2024-09-19T06:23:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3461bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a290968a00 con 0x55a290965cb0 2024-09-19T06:23:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3461bc740 10 client.0 _ll_register_callbacks cb 0x55a290879140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3409be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 0x55a290963150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3419c0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29095aaf0 0x55a290962c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3409be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 0x55a290963150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56112/0 (socket says 172.21.15.5:56112) 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3409be640 1 -- 172.21.15.5:0/3522499211 learned_addr learned my addr 172.21.15.5:0/3522499211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3419c0640 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a290965cb0 0x55a290963690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3419c0640 1 -- 172.21.15.5:0/3522499211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29095aaf0 msgr2=0x55a290962c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3419c0640 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29095aaf0 0x55a290962c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3419c0640 1 -- 172.21.15.5:0/3522499211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 msgr2=0x55a290963150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3419c0640 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 0x55a290963150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3419c0640 1 -- 172.21.15.5:0/3522499211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2905d2210 con 0x55a290965cb0 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.647+0000 7fe3409be640 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 0x55a290963150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.648+0000 7fe3419c0640 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a290965cb0 0x55a290963690 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe334009f10 tx=0x7fe334005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a9729d46a47c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.648+0000 7fe324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.648+0000 7fe324ff9640 1 -- 172.21.15.5:0/3522499211 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334024040 con 0x55a290965cb0 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.648+0000 7fe3461bc740 1 -- 172.21.15.5:0/3522499211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe334003d10 con 0x55a290965cb0 2024-09-19T06:23:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.648+0000 7fe324ff9640 1 -- 172.21.15.5:0/3522499211 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7fe3340045f0 con 0x55a290965cb0 2024-09-19T06:23:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.648+0000 7fe324ff9640 1 -- 172.21.15.5:0/3522499211 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fe3340116a0 con 0x55a290965cb0 2024-09-19T06:23:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.648+0000 7fe3461bc740 10 client.29872 fetch_fsmap learned FSMap version 60 2024-09-19T06:23:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.648+0000 7fe3461bc740 1 -- 172.21.15.5:0/3522499211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55a29069e620 con 0x55a290965cb0 2024-09-19T06:23:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.648+0000 7fe324ff9640 1 -- 172.21.15.5:0/3522499211 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe334003d10 con 0x55a290965cb0 2024-09-19T06:23:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.649+0000 7fe324ff9640 1 -- 172.21.15.5:0/3522499211 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fe33402b810 con 0x55a290965cb0 2024-09-19T06:23:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.649+0000 7fe3461bc740 10 client.29872 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:23:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.649+0000 7fe3461bc740 10 client.29872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.649+0000 7fe3461bc740 20 client.29872 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.649+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a29097e3f0 0x55a2909807e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.649+0000 7fe3461bc740 4 client.29872 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.649+0000 7fe3461bc740 1 -- 172.21.15.5:0/3522499211 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2908cca20 con 0x55a29097e3f0 2024-09-19T06:23:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.649+0000 7fe3409be640 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a29097e3f0 0x55a2909807e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe324ff9640 10 client.29872 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe3409be640 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a29097e3f0 0x55a2909807e0 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b8f2430171aaa5 server_cookie=bd09e6fab612880 in_seq=0 out_seq=0 2024-09-19T06:23:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe324ff9640 1 -- 172.21.15.5:0/3522499211 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2908cca20 con 0x55a29097e3f0 2024-09-19T06:23:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe324ff9640 10 client.29872 handle_command_reply: tid=0 2024-09-19T06:23:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe3461bc740 1 client.29872 shutdown 2024-09-19T06:23:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe3461bc740 1 -- 172.21.15.5:0/3522499211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a290965cb0 msgr2=0x55a290963690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a290965cb0 0x55a290963690 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe334009f10 tx=0x7fe334005fd0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe3461bc740 1 -- 172.21.15.5:0/3522499211 shutdown_connections 2024-09-19T06:23:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a29097e3f0 0x55a2909807e0 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a290965cb0 0x55a290963690 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 0x55a290963150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe324ff9640 0 client.29872 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.650+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3522499211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29095aaf0 0x55a290962c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.651+0000 7fe3461bc740 1 -- 172.21.15.5:0/3522499211 >> 172.21.15.5:0/3522499211 conn(0x55a290957060 msgr2=0x55a2909561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.651+0000 7fe3461bc740 1 -- 172.21.15.5:0/3522499211 shutdown_connections 2024-09-19T06:23:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.651+0000 7fe3461bc740 1 -- 172.21.15.5:0/3522499211 wait complete. 2024-09-19T06:23:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.651+0000 7fe3461bc740 20 client.29872 trim_cache size 0 max 16384 2024-09-19T06:23:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.672+0000 7fe3461bc740 1 Processor -- start 2024-09-19T06:23:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.672+0000 7fe3461bc740 1 -- start start 2024-09-19T06:23:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3461bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a29095aaf0 0x55a290957440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3461bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 0x55a290985f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3461bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a290965cb0 0x55a2909864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3461bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1ab710 con 0x55a29095b410 2024-09-19T06:23:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3461bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1ad2b0 con 0x55a290965cb0 2024-09-19T06:23:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3461bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1abde0 con 0x55a29095aaf0 2024-09-19T06:23:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3409be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 0x55a290985f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3409be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 0x55a290985f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56120/0 (socket says 172.21.15.5:56120) 2024-09-19T06:23:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3419c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a29095aaf0 0x55a290957440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3409be640 1 -- 172.21.15.5:0/1606735857 learned_addr learned my addr 172.21.15.5:0/1606735857 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3419c0640 1 --2- 172.21.15.5:0/1606735857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a290965cb0 0x55a2909864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3409be640 1 -- 172.21.15.5:0/1606735857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a29095aaf0 msgr2=0x55a290957440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3409be640 1 --2- 172.21.15.5:0/1606735857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a29095aaf0 0x55a290957440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3409be640 1 -- 172.21.15.5:0/1606735857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a290965cb0 msgr2=0x55a2909864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3409be640 1 --2- 172.21.15.5:0/1606735857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a290965cb0 0x55a2909864b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.673+0000 7fe3409be640 1 -- 172.21.15.5:0/1606735857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a290961be0 con 0x55a29095b410 2024-09-19T06:23:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.674+0000 7fe3409be640 1 --2- 172.21.15.5:0/1606735857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 0x55a290985f70 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fe330009230 tx=0x7fe33002a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff9307541d484ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.674+0000 7fe326ffd640 1 -- 172.21.15.5:0/1606735857 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330035040 con 0x55a29095b410 2024-09-19T06:23:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.674+0000 7fe326ffd640 1 -- 172.21.15.5:0/1606735857 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe33004b070 con 0x55a29095b410 2024-09-19T06:23:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.674+0000 7fe326ffd640 1 -- 172.21.15.5:0/1606735857 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3300035f0 con 0x55a29095b410 2024-09-19T06:23:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.675+0000 7fe3461bc740 1 -- 172.21.15.5:0/1606735857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 msgr2=0x55a290985f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.675+0000 7fe3461bc740 1 --2- 172.21.15.5:0/1606735857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 0x55a290985f70 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fe330009230 tx=0x7fe33002a9a0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.675+0000 7fe3461bc740 1 -- 172.21.15.5:0/1606735857 shutdown_connections 2024-09-19T06:23:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.675+0000 7fe3461bc740 1 --2- 172.21.15.5:0/1606735857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a290965cb0 0x55a2909864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.675+0000 7fe3461bc740 1 --2- 172.21.15.5:0/1606735857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095b410 0x55a290985f70 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.675+0000 7fe3461bc740 1 --2- 172.21.15.5:0/1606735857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a29095aaf0 0x55a290957440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.675+0000 7fe3461bc740 1 -- 172.21.15.5:0/1606735857 >> 172.21.15.5:0/1606735857 conn(0x55a290957060 msgr2=0x55a290964c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.675+0000 7fe3461bc740 1 -- 172.21.15.5:0/1606735857 shutdown_connections 2024-09-19T06:23:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.675+0000 7fe3461bc740 1 -- 172.21.15.5:0/1606735857 wait complete. 2024-09-19T06:23:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.675+0000 7fe3461bc740 1 -- start start 2024-09-19T06:23:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.675+0000 7fe3461bc740 1 Processor -- start 2024-09-19T06:23:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3461bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095aaf0 0x55a29095fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3461bc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a290965cb0 0x55a290960280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3461bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2909607c0 0x55a2909e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3461bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1acbe0 con 0x55a29095aaf0 2024-09-19T06:23:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3461bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1ac4e0 con 0x55a2909607c0 2024-09-19T06:23:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3461bc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1a9730 con 0x55a290965cb0 2024-09-19T06:23:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3461bc740 10 client.0 _ll_register_callbacks cb 0x7fe338005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3419c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095aaf0 0x55a29095fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3409be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a290965cb0 0x55a290960280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3409be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a290965cb0 0x55a290960280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41094/0 (socket says 172.21.15.5:41094) 2024-09-19T06:23:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3409be640 1 -- 172.21.15.5:0/3530424179 learned_addr learned my addr 172.21.15.5:0/3530424179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3419c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095aaf0 0x55a29095fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56136/0 (socket says 172.21.15.5:56136) 2024-09-19T06:23:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3419c0640 1 --2- 172.21.15.5:0/3530424179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2909607c0 0x55a2909e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3419c0640 1 -- 172.21.15.5:0/3530424179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a290965cb0 msgr2=0x55a290960280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3419c0640 1 --2- 172.21.15.5:0/3530424179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a290965cb0 0x55a290960280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3419c0640 1 -- 172.21.15.5:0/3530424179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2909607c0 msgr2=0x55a2909e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3419c0640 1 --2- 172.21.15.5:0/3530424179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2909607c0 0x55a2909e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.676+0000 7fe3419c0640 1 -- 172.21.15.5:0/3530424179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2909e84b0 con 0x55a29095aaf0 2024-09-19T06:23:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.677+0000 7fe3419c0640 1 --2- 172.21.15.5:0/3530424179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095aaf0 0x55a29095fd40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fe334009ee0 tx=0x7fe334005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e4e5161766dc94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.677+0000 7fe324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:23:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.677+0000 7fe324ff9640 1 -- 172.21.15.5:0/3530424179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334037070 con 0x55a29095aaf0 2024-09-19T06:23:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.677+0000 7fe324ff9640 1 -- 172.21.15.5:0/3530424179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7fe33402b640 con 0x55a29095aaf0 2024-09-19T06:23:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.677+0000 7fe3461bc740 1 -- 172.21.15.5:0/3530424179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe33004b070 con 0x55a29095aaf0 2024-09-19T06:23:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.677+0000 7fe324ff9640 1 -- 172.21.15.5:0/3530424179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fe33402b840 con 0x55a29095aaf0 2024-09-19T06:23:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.677+0000 7fe3461bc740 10 client.20565 fetch_fsmap learned FSMap version 60 2024-09-19T06:23:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.677+0000 7fe3461bc740 1 -- 172.21.15.5:0/3530424179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55a2909e9760 con 0x55a29095aaf0 2024-09-19T06:23:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.677+0000 7fe324ff9640 1 -- 172.21.15.5:0/3530424179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe33402be50 con 0x55a29095aaf0 2024-09-19T06:23:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.678+0000 7fe324ff9640 1 -- 172.21.15.5:0/3530424179 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fe33403d070 con 0x55a29095aaf0 2024-09-19T06:23:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.678+0000 7fe3461bc740 10 client.20565 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:23:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.678+0000 7fe3461bc740 10 client.20565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.678+0000 7fe3461bc740 20 client.20565 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.678+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3530424179 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a2909e9dd0 0x55a2909ec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:46.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.678+0000 7fe3461bc740 4 client.20565 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:23:46.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.678+0000 7fe3461bc740 1 -- 172.21.15.5:0/3530424179 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe33004b070 con 0x55a2909e9dd0 2024-09-19T06:23:46.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.678+0000 7fe3409be640 1 --2- 172.21.15.5:0/3530424179 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a2909e9dd0 0x55a2909ec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:46.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.679+0000 7fe3409be640 1 --2- 172.21.15.5:0/3530424179 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a2909e9dd0 0x55a2909ec1c0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7154b4da1b099182 server_cookie=7946f7f75f6918b0 in_seq=0 out_seq=0 2024-09-19T06:23:46.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.679+0000 7fe324ff9640 10 client.20565 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe324ff9640 1 -- 172.21.15.5:0/3530424179 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe33004b070 con 0x55a2909e9dd0 2024-09-19T06:23:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe324ff9640 10 client.20565 handle_command_reply: tid=0 2024-09-19T06:23:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe3461bc740 1 client.20565 shutdown 2024-09-19T06:23:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe3461bc740 1 -- 172.21.15.5:0/3530424179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095aaf0 msgr2=0x55a29095fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3530424179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095aaf0 0x55a29095fd40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fe334009ee0 tx=0x7fe334005fd0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe3461bc740 1 -- 172.21.15.5:0/3530424179 shutdown_connections 2024-09-19T06:23:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3530424179 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a2909e9dd0 0x55a2909ec1c0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3530424179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2909607c0 0x55a2909e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3530424179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a290965cb0 0x55a290960280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe3461bc740 1 --2- 172.21.15.5:0/3530424179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a29095aaf0 0x55a29095fd40 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe324ff9640 0 client.20565 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe3461bc740 1 -- 172.21.15.5:0/3530424179 >> 172.21.15.5:0/3530424179 conn(0x55a290957060 msgr2=0x55a29095c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe3461bc740 1 -- 172.21.15.5:0/3530424179 shutdown_connections 2024-09-19T06:23:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.795+0000 7fe3461bc740 1 -- 172.21.15.5:0/3530424179 wait complete. 2024-09-19T06:23:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.796+0000 7fe3461bc740 20 client.20565 trim_cache size 0 max 16384 2024-09-19T06:23:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:23:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:23:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:23:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:23:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.796+0000 7fe343423640 1 -- 172.21.15.5:0/1731447166 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe3280a34b0 msgr2=0x7fe3280a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.796+0000 7fe343423640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe3280a34b0 0x7fe3280a5970 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55a290957ec0 tx=0x7fe3380d37e0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.796+0000 7fe343423640 1 -- 172.21.15.5:0/1731447166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe33c100830 msgr2=0x7fe33c19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.796+0000 7fe343423640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe33c100830 0x7fe33c19a540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe33800e9f0 tx=0x7fe33800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.797+0000 7fe343423640 1 -- 172.21.15.5:0/1731447166 shutdown_connections 2024-09-19T06:23:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.797+0000 7fe343423640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe33c10e350 0x7fe33c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.797+0000 7fe343423640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe33c101150 0x7fe33c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.797+0000 7fe343423640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe3280a34b0 0x7fe3280a5970 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.797+0000 7fe343423640 1 --2- 172.21.15.5:0/1731447166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe33c100830 0x7fe33c19a540 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.797+0000 7fe343423640 1 -- 172.21.15.5:0/1731447166 >> 172.21.15.5:0/1731447166 conn(0x7fe33c075f50 msgr2=0x7fe33c10e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.797+0000 7fe343423640 1 -- 172.21.15.5:0/1731447166 shutdown_connections 2024-09-19T06:23:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:46.797+0000 7fe343423640 1 -- 172.21.15.5:0/1731447166 wait complete. 2024-09-19T06:23:46.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:46 smithi005 ceph-mon[30955]: pgmap v644: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 283 op/s 2024-09-19T06:23:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:46 smithi049 ceph-mon[34640]: pgmap v644: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 283 op/s 2024-09-19T06:23:47.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:46 smithi157 ceph-mon[35346]: pgmap v644: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 283 op/s 2024-09-19T06:23:47.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 70767 8.74 MB/sec execute 311 sec latency 85.884 ms 2024-09-19T06:23:48.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 70905 8.73 MB/sec execute 312 sec latency 56.019 ms 2024-09-19T06:23:48.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:48 smithi005 ceph-mon[30955]: pgmap v645: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 429 op/s 2024-09-19T06:23:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:48 smithi049 ceph-mon[34640]: pgmap v645: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 429 op/s 2024-09-19T06:23:49.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:48 smithi157 ceph-mon[35346]: pgmap v645: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 429 op/s 2024-09-19T06:23:49.588 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71078 8.74 MB/sec execute 313 sec latency 69.640 ms 2024-09-19T06:23:50.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71234 8.73 MB/sec execute 314 sec latency 85.566 ms 2024-09-19T06:23:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:50 smithi005 ceph-mon[30955]: pgmap v646: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-09-19T06:23:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:50 smithi049 ceph-mon[34640]: pgmap v646: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-09-19T06:23:51.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:50 smithi157 ceph-mon[35346]: pgmap v646: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-09-19T06:23:51.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71390 8.73 MB/sec execute 315 sec latency 79.850 ms 2024-09-19T06:23:52.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71545 8.71 MB/sec execute 316 sec latency 101.778 ms 2024-09-19T06:23:52.853 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:52 smithi157 ceph-mon[35346]: pgmap v647: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-09-19T06:23:52.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:52 smithi005 ceph-mon[30955]: pgmap v647: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-09-19T06:23:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:52 smithi049 ceph-mon[34640]: pgmap v647: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-09-19T06:23:53.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71722 8.72 MB/sec execute 317 sec latency 61.436 ms 2024-09-19T06:23:53.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:53 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:23:53.881 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:53 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:23:53.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:53 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:23:54.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71877 8.72 MB/sec execute 318 sec latency 78.180 ms 2024-09-19T06:23:54.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:54 smithi005 ceph-mon[30955]: pgmap v648: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.6 MiB/s wr, 389 op/s 2024-09-19T06:23:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:54 smithi049 ceph-mon[34640]: pgmap v648: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.6 MiB/s wr, 389 op/s 2024-09-19T06:23:55.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:54 smithi157 ceph-mon[35346]: pgmap v648: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.6 MiB/s wr, 389 op/s 2024-09-19T06:23:55.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72017 8.70 MB/sec execute 319 sec latency 76.190 ms 2024-09-19T06:23:56.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:56.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:56.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:56.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:56.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: pgmap v649: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 266 op/s 2024-09-19T06:23:56.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T06:23:56.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: Unable to set osd_memory_target on smithi157 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-19T06:23:56.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-19T06:23:56.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:55 smithi005 ceph-mon[30955]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T06:23:56.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: pgmap v649: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 266 op/s 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: Unable to set osd_memory_target on smithi157 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.147 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-19T06:23:56.148 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:55 smithi049 ceph-mon[34640]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T06:23:56.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:56.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:56.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:56.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:56.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: pgmap v649: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 266 op/s 2024-09-19T06:23:56.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T06:23:56.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: Unable to set osd_memory_target on smithi157 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-19T06:23:56.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:56.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-19T06:23:56.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:55 smithi157 ceph-mon[35346]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T06:23:56.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72204 8.71 MB/sec execute 320 sec latency 56.479 ms 2024-09-19T06:23:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:57 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:57 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:57 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:57 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:57 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:57 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:57 smithi005 ceph-mon[30955]: Adjusting osd_memory_target on smithi049 to 176.0M 2024-09-19T06:23:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:57 smithi005 ceph-mon[30955]: Unable to set osd_memory_target on smithi049 to 184553062: error parsing value: Value '184553062' is below minimum 939524096 2024-09-19T06:23:57.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:57 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:23:57.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:57 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:23:57.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:57 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:57.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:57 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:57.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:57 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:57.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:57 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:57.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:57 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:57.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:57 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:57.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:57 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:57.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:57 smithi049 ceph-mon[34640]: Adjusting osd_memory_target on smithi049 to 176.0M 2024-09-19T06:23:57.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:57 smithi049 ceph-mon[34640]: Unable to set osd_memory_target on smithi049 to 184553062: error parsing value: Value '184553062' is below minimum 939524096 2024-09-19T06:23:57.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:57 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:23:57.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:57 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:23:57.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:57 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:57.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:57 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:57.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:57 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:57.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:57 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:57.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:57 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:57.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:57 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:57.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:57 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:23:57.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:57 smithi157 ceph-mon[35346]: Adjusting osd_memory_target on smithi049 to 176.0M 2024-09-19T06:23:57.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:57 smithi157 ceph-mon[35346]: Unable to set osd_memory_target on smithi049 to 184553062: error parsing value: Value '184553062' is below minimum 939524096 2024-09-19T06:23:57.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:57 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:23:57.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:57 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:23:57.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:57 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:23:57.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72364 8.70 MB/sec execute 321 sec latency 70.729 ms 2024-09-19T06:23:58.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72540 8.71 MB/sec execute 322 sec latency 65.141 ms 2024-09-19T06:23:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:23:58 smithi157 ceph-mon[35346]: pgmap v650: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.8 MiB/s wr, 393 op/s 2024-09-19T06:23:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:23:58 smithi005 ceph-mon[30955]: pgmap v650: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.8 MiB/s wr, 393 op/s 2024-09-19T06:23:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:23:58 smithi049 ceph-mon[34640]: pgmap v650: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.8 MiB/s wr, 393 op/s 2024-09-19T06:23:59.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72714 8.71 MB/sec execute 323 sec latency 60.070 ms 2024-09-19T06:23:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.925+0000 7faa76986640 1 -- 172.21.15.5:0/576995902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa701069b0 msgr2=0x7faa70106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.925+0000 7faa76986640 1 --2- 172.21.15.5:0/576995902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa701069b0 0x7faa70106d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faa68009900 tx=0x7faa6802a410 comp rx=0 tx=0).stop 2024-09-19T06:23:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.925+0000 7faa76986640 1 -- 172.21.15.5:0/576995902 shutdown_connections 2024-09-19T06:23:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.925+0000 7faa76986640 1 --2- 172.21.15.5:0/576995902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa70101340 0x7faa7010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.925+0000 7faa76986640 1 --2- 172.21.15.5:0/576995902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa701009a0 0x7faa70100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.925+0000 7faa76986640 1 --2- 172.21.15.5:0/576995902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa701069b0 0x7faa70106d90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.925+0000 7faa76986640 1 -- 172.21.15.5:0/576995902 >> 172.21.15.5:0/576995902 conn(0x7faa700779a0 msgr2=0x7faa700fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.926+0000 7faa76986640 1 -- 172.21.15.5:0/576995902 shutdown_connections 2024-09-19T06:23:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.926+0000 7faa76986640 1 -- 172.21.15.5:0/576995902 wait complete. 2024-09-19T06:23:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.926+0000 7faa76986640 1 Processor -- start 2024-09-19T06:23:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.927+0000 7faa76986640 1 -- start start 2024-09-19T06:23:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.927+0000 7faa76986640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa701009a0 0x7faa701960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.927+0000 7faa76986640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa70101340 0x7faa701965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.927+0000 7faa76986640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa701069b0 0x7faa7019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.927+0000 7faa76986640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faa7010fba0 con 0x7faa701069b0 2024-09-19T06:23:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.927+0000 7faa76986640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faa7010fa20 con 0x7faa70101340 2024-09-19T06:23:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.927+0000 7faa76986640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faa7010fd20 con 0x7faa701009a0 2024-09-19T06:23:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.927+0000 7faa6f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa70101340 0x7faa701965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.927+0000 7faa74f23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa701069b0 0x7faa7019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.927+0000 7faa6f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa70101340 0x7faa701965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56084/0 (socket says 172.21.15.5:56084) 2024-09-19T06:23:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.927+0000 7faa6f7fe640 1 -- 172.21.15.5:0/232565228 learned_addr learned my addr 172.21.15.5:0/232565228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.927+0000 7faa6ffff640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa701009a0 0x7faa701960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.928+0000 7faa6f7fe640 1 -- 172.21.15.5:0/232565228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa701009a0 msgr2=0x7faa701960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.928+0000 7faa6f7fe640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa701009a0 0x7faa701960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.928+0000 7faa6f7fe640 1 -- 172.21.15.5:0/232565228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa701069b0 msgr2=0x7faa7019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.928+0000 7faa6f7fe640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa701069b0 0x7faa7019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.928+0000 7faa6f7fe640 1 -- 172.21.15.5:0/232565228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7019b0a0 con 0x7faa70101340 2024-09-19T06:23:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.928+0000 7faa6f7fe640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa70101340 0x7faa701965f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7faa58009960 tx=0x7faa58009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.928+0000 7faa6d7fa640 1 -- 172.21.15.5:0/232565228 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58007c70 con 0x7faa70101340 2024-09-19T06:23:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.928+0000 7faa6d7fa640 1 -- 172.21.15.5:0/232565228 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa5801c070 con 0x7faa70101340 2024-09-19T06:23:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.928+0000 7faa76986640 1 -- 172.21.15.5:0/232565228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7019b390 con 0x7faa70101340 2024-09-19T06:23:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.928+0000 7faa6d7fa640 1 -- 172.21.15.5:0/232565228 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa580165b0 con 0x7faa70101340 2024-09-19T06:23:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.928+0000 7faa76986640 1 -- 172.21.15.5:0/232565228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa701a2c20 con 0x7faa70101340 2024-09-19T06:23:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.932+0000 7faa6d7fa640 1 -- 172.21.15.5:0/232565228 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7faa580179c0 con 0x7faa70101340 2024-09-19T06:23:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.932+0000 7faa797f8740 1 Processor -- start 2024-09-19T06:23:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.932+0000 7faa797f8740 1 -- start start 2024-09-19T06:23:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa797f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b91cc0 0x561b48b920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa797f8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8d660 0x561b48b8dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa797f8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8e000 0x561b48b999c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa797f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561b48ae82f0 con 0x561b48b91cc0 2024-09-19T06:23:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa797f8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561b48a90270 con 0x561b48b8d660 2024-09-19T06:23:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa797f8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561b48ae42b0 con 0x561b48b8e000 2024-09-19T06:23:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa74f23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b91cc0 0x561b48b920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa74f23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b91cc0 0x561b48b920a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58886/0 (socket says 172.21.15.5:58886) 2024-09-19T06:23:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa74f23640 1 -- 172.21.15.5:0/153215334 learned_addr learned my addr 172.21.15.5:0/153215334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa6ffff640 1 --2- 172.21.15.5:0/153215334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8d660 0x561b48b8dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa74f23640 1 -- 172.21.15.5:0/153215334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8e000 msgr2=0x561b48b999c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:23:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa74f23640 1 --2- 172.21.15.5:0/153215334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8e000 0x561b48b999c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa74f23640 1 -- 172.21.15.5:0/153215334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8d660 msgr2=0x561b48b8dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa74f23640 1 --2- 172.21.15.5:0/153215334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8d660 0x561b48b8dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa74f23640 1 -- 172.21.15.5:0/153215334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b48a8f340 con 0x561b48b91cc0 2024-09-19T06:23:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.933+0000 7faa6d7fa640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7faa5c0a3430 0x7faa5c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.934+0000 7faa6ffff640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7faa5c0a3430 0x7faa5c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.934+0000 7faa6d7fa640 1 -- 172.21.15.5:0/232565228 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7faa5800a250 con 0x7faa70101340 2024-09-19T06:23:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.934+0000 7faa74f23640 1 --2- 172.21.15.5:0/153215334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b91cc0 0x561b48b920a0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7faa60009f50 tx=0x7faa600118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8376b3c1bf828b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.934+0000 7faa667fc640 1 -- 172.21.15.5:0/153215334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60004d20 con 0x561b48b91cc0 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.934+0000 7faa667fc640 1 -- 172.21.15.5:0/153215334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa60004050 con 0x561b48b91cc0 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.934+0000 7faa667fc640 1 -- 172.21.15.5:0/153215334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60005280 con 0x561b48b91cc0 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.935+0000 7faa797f8740 1 -- 172.21.15.5:0/153215334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b91cc0 msgr2=0x561b48b920a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.935+0000 7faa6ffff640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7faa5c0a3430 0x7faa5c0a58f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faa68009230 tx=0x7faa680058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.935+0000 7faa797f8740 1 --2- 172.21.15.5:0/153215334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b91cc0 0x561b48b920a0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7faa60009f50 tx=0x7faa600118d0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.935+0000 7faa797f8740 1 -- 172.21.15.5:0/153215334 shutdown_connections 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.935+0000 7faa797f8740 1 --2- 172.21.15.5:0/153215334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8e000 0x561b48b999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.935+0000 7faa797f8740 1 --2- 172.21.15.5:0/153215334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8d660 0x561b48b8dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.935+0000 7faa797f8740 1 --2- 172.21.15.5:0/153215334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b91cc0 0x561b48b920a0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.935+0000 7faa797f8740 1 -- 172.21.15.5:0/153215334 >> 172.21.15.5:0/153215334 conn(0x561b48b7fb10 msgr2=0x561b48b7fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.935+0000 7faa797f8740 1 -- 172.21.15.5:0/153215334 shutdown_connections 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.935+0000 7faa797f8740 1 -- 172.21.15.5:0/153215334 wait complete. 2024-09-19T06:23:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.936+0000 7faa797f8740 1 -- start start 2024-09-19T06:23:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.936+0000 7faa797f8740 1 Processor -- start 2024-09-19T06:23:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa797f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8d660 0x561b48b96e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa797f8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8e000 0x561b48b97360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa797f8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b91cc0 0x561b48b978a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa797f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561b48b9a0b0 con 0x561b48b8d660 2024-09-19T06:23:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa797f8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561b48a71290 con 0x561b48b8e000 2024-09-19T06:23:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa797f8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561b48b9a230 con 0x561b48b91cc0 2024-09-19T06:23:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8d660 0x561b48b96e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa6f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8e000 0x561b48b97360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8d660 0x561b48b96e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58894/0 (socket says 172.21.15.5:58894) 2024-09-19T06:23:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa6ffff640 1 -- 172.21.15.5:0/3069736969 learned_addr learned my addr 172.21.15.5:0/3069736969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa6f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8e000 0x561b48b97360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56102/0 (socket says 172.21.15.5:56102) 2024-09-19T06:23:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa74f23640 1 --2- 172.21.15.5:0/3069736969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b91cc0 0x561b48b978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa797f8740 10 client.0 _ll_register_callbacks cb 0x561b48aa2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa6f7fe640 1 -- 172.21.15.5:0/3069736969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b91cc0 msgr2=0x561b48b978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa6f7fe640 1 --2- 172.21.15.5:0/3069736969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b91cc0 0x561b48b978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa6f7fe640 1 -- 172.21.15.5:0/3069736969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8d660 msgr2=0x561b48b96e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa6f7fe640 1 --2- 172.21.15.5:0/3069736969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8d660 0x561b48b96e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.937+0000 7faa6f7fe640 1 -- 172.21.15.5:0/3069736969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b487fb210 con 0x561b48b8e000 2024-09-19T06:23:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.938+0000 7faa6f7fe640 1 --2- 172.21.15.5:0/3069736969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8e000 0x561b48b97360 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faa580cf0f0 tx=0x7faa5809d6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a3b64edf9d3608b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.938+0000 7faa4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:23:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.938+0000 7faa4f7fe640 1 -- 172.21.15.5:0/3069736969 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa580a68f0 con 0x561b48b8e000 2024-09-19T06:23:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.938+0000 7faa797f8740 1 -- 172.21.15.5:0/3069736969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa60004050 con 0x561b48b8e000 2024-09-19T06:23:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.938+0000 7faa4f7fe640 1 -- 172.21.15.5:0/3069736969 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7faa5809d0d0 con 0x561b48b8e000 2024-09-19T06:23:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.938+0000 7faa4f7fe640 1 -- 172.21.15.5:0/3069736969 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7faa580bb050 con 0x561b48b8e000 2024-09-19T06:23:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.938+0000 7faa797f8740 10 client.29942 fetch_fsmap learned FSMap version 60 2024-09-19T06:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.938+0000 7faa797f8740 1 -- 172.21.15.5:0/3069736969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x561b488c7620 con 0x561b48b8e000 2024-09-19T06:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.938+0000 7faa4f7fe640 1 -- 172.21.15.5:0/3069736969 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa60004050 con 0x561b48b8e000 2024-09-19T06:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.939+0000 7faa4f7fe640 1 -- 172.21.15.5:0/3069736969 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7faa580999d0 con 0x561b48b8e000 2024-09-19T06:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.939+0000 7faa797f8740 10 client.29942 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.939+0000 7faa797f8740 10 client.29942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.939+0000 7faa797f8740 20 client.29942 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.939+0000 7faa797f8740 1 --2- 172.21.15.5:0/3069736969 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561b48ba7450 0x561b48ba9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.939+0000 7faa797f8740 4 client.29942 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:23:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.940+0000 7faa797f8740 1 -- 172.21.15.5:0/3069736969 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b48af5a20 con 0x561b48ba7450 2024-09-19T06:23:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.940+0000 7faa6ffff640 1 --2- 172.21.15.5:0/3069736969 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561b48ba7450 0x561b48ba9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.940+0000 7faa4f7fe640 10 client.29942 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.940+0000 7faa6ffff640 1 --2- 172.21.15.5:0/3069736969 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561b48ba7450 0x561b48ba9840 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed87536f68d3f283 server_cookie=f0c471b39efb817d in_seq=0 out_seq=0 2024-09-19T06:23:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.941+0000 7faa4f7fe640 1 -- 172.21.15.5:0/3069736969 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b48af5a20 con 0x561b48ba7450 2024-09-19T06:23:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.941+0000 7faa4f7fe640 10 client.29942 handle_command_reply: tid=0 2024-09-19T06:23:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.941+0000 7faa797f8740 1 client.29942 shutdown 2024-09-19T06:23:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.942+0000 7faa797f8740 1 -- 172.21.15.5:0/3069736969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8e000 msgr2=0x561b48b97360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.942+0000 7faa797f8740 1 --2- 172.21.15.5:0/3069736969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8e000 0x561b48b97360 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faa580cf0f0 tx=0x7faa5809d6a0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.942+0000 7faa797f8740 1 -- 172.21.15.5:0/3069736969 shutdown_connections 2024-09-19T06:23:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.942+0000 7faa797f8740 1 --2- 172.21.15.5:0/3069736969 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561b48ba7450 0x561b48ba9840 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.942+0000 7faa797f8740 1 --2- 172.21.15.5:0/3069736969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b91cc0 0x561b48b978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.942+0000 7faa797f8740 1 --2- 172.21.15.5:0/3069736969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8e000 0x561b48b97360 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.942+0000 7faa797f8740 1 --2- 172.21.15.5:0/3069736969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8d660 0x561b48b96e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.942+0000 7faa797f8740 1 -- 172.21.15.5:0/3069736969 >> 172.21.15.5:0/3069736969 conn(0x561b48b7fb10 msgr2=0x561b48b8ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.942+0000 7faa4f7fe640 0 client.29942 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.942+0000 7faa797f8740 1 -- 172.21.15.5:0/3069736969 shutdown_connections 2024-09-19T06:23:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.942+0000 7faa797f8740 1 -- 172.21.15.5:0/3069736969 wait complete. 2024-09-19T06:23:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.942+0000 7faa797f8740 20 client.29942 trim_cache size 0 max 16384 2024-09-19T06:23:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.965+0000 7faa797f8740 1 Processor -- start 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.965+0000 7faa797f8740 1 -- start start 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.965+0000 7faa797f8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8d660 0x561b48b8cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa797f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8e000 0x561b48baf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa797f8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b91cc0 0x561b48baf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa797f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faa701ab640 con 0x561b48b8e000 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa797f8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faa701ad1e0 con 0x561b48b91cc0 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa797f8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faa701abd10 con 0x561b48b8d660 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8e000 0x561b48baf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8e000 0x561b48baf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58900/0 (socket says 172.21.15.5:58900) 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa6f7fe640 1 -- 172.21.15.5:0/2632841746 learned_addr learned my addr 172.21.15.5:0/2632841746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa74f23640 1 --2- 172.21.15.5:0/2632841746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b91cc0 0x561b48baf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa6ffff640 1 --2- 172.21.15.5:0/2632841746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8d660 0x561b48b8cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa6f7fe640 1 -- 172.21.15.5:0/2632841746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8d660 msgr2=0x561b48b8cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa6f7fe640 1 --2- 172.21.15.5:0/2632841746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8d660 0x561b48b8cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa6f7fe640 1 -- 172.21.15.5:0/2632841746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b91cc0 msgr2=0x561b48baf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa6f7fe640 1 --2- 172.21.15.5:0/2632841746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b91cc0 0x561b48baf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa6f7fe640 1 -- 172.21.15.5:0/2632841746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b48b92410 con 0x561b48b8e000 2024-09-19T06:23:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.966+0000 7faa6f7fe640 1 --2- 172.21.15.5:0/2632841746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8e000 0x561b48baf0a0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7faa580cf1a0 tx=0x7faa5809d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecffbc0b5f4a52d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.967+0000 7faa65ffb640 1 -- 172.21.15.5:0/2632841746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5809d7c0 con 0x561b48b8e000 2024-09-19T06:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.967+0000 7faa65ffb640 1 -- 172.21.15.5:0/2632841746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa580a68f0 con 0x561b48b8e000 2024-09-19T06:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.967+0000 7faa65ffb640 1 -- 172.21.15.5:0/2632841746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa580999d0 con 0x561b48b8e000 2024-09-19T06:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.968+0000 7faa797f8740 1 -- 172.21.15.5:0/2632841746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8e000 msgr2=0x561b48baf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.968+0000 7faa797f8740 1 --2- 172.21.15.5:0/2632841746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8e000 0x561b48baf0a0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7faa580cf1a0 tx=0x7faa5809d710 comp rx=0 tx=0).stop 2024-09-19T06:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.968+0000 7faa797f8740 1 -- 172.21.15.5:0/2632841746 shutdown_connections 2024-09-19T06:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.968+0000 7faa797f8740 1 --2- 172.21.15.5:0/2632841746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b91cc0 0x561b48baf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.968+0000 7faa797f8740 1 --2- 172.21.15.5:0/2632841746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b8e000 0x561b48baf0a0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.968+0000 7faa797f8740 1 --2- 172.21.15.5:0/2632841746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8d660 0x561b48b8cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.968+0000 7faa797f8740 1 -- 172.21.15.5:0/2632841746 >> 172.21.15.5:0/2632841746 conn(0x561b48b7fb10 msgr2=0x561b48b8c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.968+0000 7faa797f8740 1 -- 172.21.15.5:0/2632841746 shutdown_connections 2024-09-19T06:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.968+0000 7faa797f8740 1 -- 172.21.15.5:0/2632841746 wait complete. 2024-09-19T06:23:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.968+0000 7faa797f8740 1 -- start start 2024-09-19T06:23:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.969+0000 7faa797f8740 1 Processor -- start 2024-09-19T06:23:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.969+0000 7faa797f8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8d660 0x561b48b9b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.969+0000 7faa797f8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8e000 0x561b48b9ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.969+0000 7faa797f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b91cc0 0x561b48c11070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.969+0000 7faa797f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faa701acb10 con 0x561b48b91cc0 2024-09-19T06:23:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.969+0000 7faa797f8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faa701ac410 con 0x561b48b8d660 2024-09-19T06:23:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.969+0000 7faa797f8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faa701a9640 con 0x561b48b8e000 2024-09-19T06:23:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.969+0000 7faa797f8740 10 client.0 _ll_register_callbacks cb 0x7faa5800a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:23:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.969+0000 7faa6ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8d660 0x561b48b9b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.969+0000 7faa6ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8d660 0x561b48b9b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56130/0 (socket says 172.21.15.5:56130) 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.969+0000 7faa6ffff640 1 -- 172.21.15.5:0/3094060096 learned_addr learned my addr 172.21.15.5:0/3094060096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.969+0000 7faa74f23640 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b91cc0 0x561b48c11070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa6f7fe640 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8e000 0x561b48b9ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa6ffff640 1 -- 172.21.15.5:0/3094060096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8e000 msgr2=0x561b48b9ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa6ffff640 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8e000 0x561b48b9ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa6ffff640 1 -- 172.21.15.5:0/3094060096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b91cc0 msgr2=0x561b48c11070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa6ffff640 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b91cc0 0x561b48c11070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa6ffff640 1 -- 172.21.15.5:0/3094060096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b48b9c300 con 0x561b48b8d660 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa6ffff640 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8d660 0x561b48b9b530 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faa6802a930 tx=0x7faa68009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=851cddcbbfd99368 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa4f7fe640 1 -- 172.21.15.5:0/3094060096 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68035040 con 0x561b48b8d660 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa797f8740 1 -- 172.21.15.5:0/3094060096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa580a68f0 con 0x561b48b8d660 2024-09-19T06:23:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa4f7fe640 1 -- 172.21.15.5:0/3094060096 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7faa680035f0 con 0x561b48b8d660 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa4f7fe640 1 -- 172.21.15.5:0/3094060096 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7faa680037f0 con 0x561b48b8d660 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa797f8740 10 client.29948 fetch_fsmap learned FSMap version 60 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.970+0000 7faa797f8740 1 -- 172.21.15.5:0/3094060096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x561b48c126f0 con 0x561b48b8d660 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.971+0000 7faa4f7fe640 1 -- 172.21.15.5:0/3094060096 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa580a68f0 con 0x561b48b8d660 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.971+0000 7faa4f7fe640 1 -- 172.21.15.5:0/3094060096 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7faa68053860 con 0x561b48b8d660 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.971+0000 7faa797f8740 10 client.29948 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.971+0000 7faa797f8740 10 client.29948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.971+0000 7faa797f8740 20 client.29948 populate_metadata read hostname 'smithi005' 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.971+0000 7faa797f8740 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561b48c12e70 0x561b48c15290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.971+0000 7faa797f8740 4 client.29948 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.971+0000 7faa797f8740 1 -- 172.21.15.5:0/3094060096 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b48aefd70 con 0x561b48c12e70 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.971+0000 7faa6f7fe640 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561b48c12e70 0x561b48c15290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:23:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.972+0000 7faa4f7fe640 10 client.29948 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:23:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:23:59.972+0000 7faa6f7fe640 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561b48c12e70 0x561b48c15290 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70790427f01d672d server_cookie=f41885109b31b4c9 in_seq=0 out_seq=0 2024-09-19T06:24:00.006 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:24:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.066+0000 7faa4f7fe640 1 -- 172.21.15.5:0/3094060096 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561b48aefd70 con 0x561b48c12e70 2024-09-19T06:24:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.066+0000 7faa4f7fe640 10 client.29948 handle_command_reply: tid=0 2024-09-19T06:24:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.066+0000 7faa797f8740 1 client.29948 shutdown 2024-09-19T06:24:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.066+0000 7faa797f8740 1 -- 172.21.15.5:0/3094060096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8d660 msgr2=0x561b48b9b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.066+0000 7faa797f8740 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8d660 0x561b48b9b530 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faa6802a930 tx=0x7faa68009260 comp rx=0 tx=0).stop 2024-09-19T06:24:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.066+0000 7faa797f8740 1 -- 172.21.15.5:0/3094060096 shutdown_connections 2024-09-19T06:24:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.066+0000 7faa797f8740 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561b48c12e70 0x561b48c15290 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.066+0000 7faa797f8740 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561b48b91cc0 0x561b48c11070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.066+0000 7faa797f8740 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b48b8e000 0x561b48b9ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.066+0000 7faa797f8740 1 --2- 172.21.15.5:0/3094060096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561b48b8d660 0x561b48b9b530 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.067+0000 7faa797f8740 1 -- 172.21.15.5:0/3094060096 >> 172.21.15.5:0/3094060096 conn(0x561b48b7fb10 msgr2=0x561b48b8ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.067+0000 7faa4f7fe640 0 client.29948 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.067+0000 7faa797f8740 1 -- 172.21.15.5:0/3094060096 shutdown_connections 2024-09-19T06:24:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.067+0000 7faa797f8740 1 -- 172.21.15.5:0/3094060096 wait complete. 2024-09-19T06:24:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.067+0000 7faa797f8740 20 client.29948 trim_cache size 0 max 16384 2024-09-19T06:24:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:24:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:24:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:24:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:24:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.067+0000 7faa76986640 1 -- 172.21.15.5:0/232565228 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7faa5c0a3430 msgr2=0x7faa5c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.067+0000 7faa76986640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7faa5c0a3430 0x7faa5c0a58f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faa68009230 tx=0x7faa680058d0 comp rx=0 tx=0).stop 2024-09-19T06:24:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.068+0000 7faa76986640 1 -- 172.21.15.5:0/232565228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa70101340 msgr2=0x7faa701965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.068+0000 7faa76986640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa70101340 0x7faa701965f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7faa58009960 tx=0x7faa58009e20 comp rx=0 tx=0).stop 2024-09-19T06:24:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.068+0000 7faa76986640 1 -- 172.21.15.5:0/232565228 shutdown_connections 2024-09-19T06:24:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.068+0000 7faa76986640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa701069b0 0x7faa7019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.068+0000 7faa76986640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa70101340 0x7faa701965f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.068+0000 7faa76986640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7faa5c0a3430 0x7faa5c0a58f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.068+0000 7faa76986640 1 --2- 172.21.15.5:0/232565228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa701009a0 0x7faa701960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.068+0000 7faa76986640 1 -- 172.21.15.5:0/232565228 >> 172.21.15.5:0/232565228 conn(0x7faa700779a0 msgr2=0x7faa700ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.068+0000 7faa76986640 1 -- 172.21.15.5:0/232565228 shutdown_connections 2024-09-19T06:24:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:00.068+0000 7faa76986640 1 -- 172.21.15.5:0/232565228 wait complete. 2024-09-19T06:24:00.387 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:24:00.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72856 8.70 MB/sec execute 324 sec latency 101.008 ms 2024-09-19T06:24:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:00 smithi157 ceph-mon[35346]: pgmap v651: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 247 op/s 2024-09-19T06:24:00.833 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:00 smithi005 ceph-mon[30955]: pgmap v651: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 247 op/s 2024-09-19T06:24:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:00 smithi049 ceph-mon[34640]: pgmap v651: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 247 op/s 2024-09-19T06:24:01.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72991 8.71 MB/sec execute 325 sec latency 163.700 ms 2024-09-19T06:24:01.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.928+0000 7f6484816640 1 -- 172.21.15.5:0/358354430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6480100a80 msgr2=0x7f6480100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:01.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.928+0000 7f6484816640 1 --2- 172.21.15.5:0/358354430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6480100a80 0x7f6480100ee0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f6474009930 tx=0x7f647402a420 comp rx=0 tx=0).stop 2024-09-19T06:24:01.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.928+0000 7f6484816640 1 -- 172.21.15.5:0/358354430 shutdown_connections 2024-09-19T06:24:01.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.928+0000 7f6484816640 1 --2- 172.21.15.5:0/358354430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6480101420 0x7f648010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.928+0000 7f6484816640 1 --2- 172.21.15.5:0/358354430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6480100a80 0x7f6480100ee0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.928+0000 7f6484816640 1 --2- 172.21.15.5:0/358354430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6480106a80 0x7f6480106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.928+0000 7f6484816640 1 -- 172.21.15.5:0/358354430 >> 172.21.15.5:0/358354430 conn(0x7f64800fc7a0 msgr2=0x7f64800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.928+0000 7f6484816640 1 -- 172.21.15.5:0/358354430 shutdown_connections 2024-09-19T06:24:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.928+0000 7f6484816640 1 -- 172.21.15.5:0/358354430 wait complete. 2024-09-19T06:24:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f6484816640 1 Processor -- start 2024-09-19T06:24:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f6484816640 1 -- start start 2024-09-19T06:24:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f6484816640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6480100a80 0x7f64801982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f6484816640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6480101420 0x7f6480198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f6484816640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6480106a80 0x7f648019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f6484816640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6480110cf0 con 0x7f6480101420 2024-09-19T06:24:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f6484816640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6480110b70 con 0x7f6480100a80 2024-09-19T06:24:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f6484816640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6480110e70 con 0x7f6480106a80 2024-09-19T06:24:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f647dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6480101420 0x7f6480198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f647dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6480101420 0x7f6480198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58918/0 (socket says 172.21.15.5:58918) 2024-09-19T06:24:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f647dd9b640 1 -- 172.21.15.5:0/4260852940 learned_addr learned my addr 172.21.15.5:0/4260852940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f647e59c640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6480100a80 0x7f64801982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f647dd9b640 1 -- 172.21.15.5:0/4260852940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6480106a80 msgr2=0x7f648019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f647dd9b640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6480106a80 0x7f648019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f647dd9b640 1 -- 172.21.15.5:0/4260852940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6480100a80 msgr2=0x7f64801982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f647dd9b640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6480100a80 0x7f64801982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.929+0000 7f647dd9b640 1 -- 172.21.15.5:0/4260852940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648019d2a0 con 0x7f6480101420 2024-09-19T06:24:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.930+0000 7f647dd9b640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6480101420 0x7f6480198830 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f6474005940 tx=0x7f64740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:01.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.930+0000 7f646f7fe640 1 -- 172.21.15.5:0/4260852940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647403f030 con 0x7f6480101420 2024-09-19T06:24:01.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.930+0000 7f646f7fe640 1 -- 172.21.15.5:0/4260852940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64740318a0 con 0x7f6480101420 2024-09-19T06:24:01.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.930+0000 7f6484816640 1 -- 172.21.15.5:0/4260852940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648019d530 con 0x7f6480101420 2024-09-19T06:24:01.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.930+0000 7f646f7fe640 1 -- 172.21.15.5:0/4260852940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6474035070 con 0x7f6480101420 2024-09-19T06:24:01.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.930+0000 7f6484816640 1 -- 172.21.15.5:0/4260852940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64801a4e70 con 0x7f6480101420 2024-09-19T06:24:01.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f646f7fe640 1 -- 172.21.15.5:0/4260852940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6474042030 con 0x7f6480101420 2024-09-19T06:24:01.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f6486bef740 1 Processor -- start 2024-09-19T06:24:01.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f6486bef740 1 -- start start 2024-09-19T06:24:01.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f6486bef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 0x55c35268e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f6486bef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 0x55c352699140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f6486bef740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c352699680 0x55c35269ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f6486bef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c3525f42e0 con 0x55c35268e4a0 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f6486bef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c35259bce0 con 0x55c35268edc0 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f6486bef740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c3525f02a0 con 0x55c352699680 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f647ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 0x55c35268e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f647ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 0x55c35268e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58924/0 (socket says 172.21.15.5:58924) 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f647ed9d640 1 -- 172.21.15.5:0/3782287312 learned_addr learned my addr 172.21.15.5:0/3782287312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f647e59c640 1 --2- 172.21.15.5:0/3782287312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 0x55c352699140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f647dd9b640 1 --2- 172.21.15.5:0/3782287312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c352699680 0x55c35269ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f646f7fe640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f64680a33b0 0x7f64680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f647e59c640 1 -- 172.21.15.5:0/3782287312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c352699680 msgr2=0x55c35269ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f647e59c640 1 --2- 172.21.15.5:0/3782287312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c352699680 0x55c35269ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f647e59c640 1 -- 172.21.15.5:0/3782287312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 msgr2=0x55c35268e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f647e59c640 1 --2- 172.21.15.5:0/3782287312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 0x55c35268e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f647e59c640 1 -- 172.21.15.5:0/3782287312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c352580440 con 0x55c35268edc0 2024-09-19T06:24:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f647ed9d640 1 --2- 172.21.15.5:0/3782287312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 0x55c35268e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.933+0000 7f647dd9b640 1 --2- 172.21.15.5:0/3782287312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c352699680 0x55c35269ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:24:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f647ed9d640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f64680a33b0 0x7f64680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f646f7fe640 1 -- 172.21.15.5:0/4260852940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f64740f05e0 con 0x7f6480101420 2024-09-19T06:24:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f647e59c640 1 --2- 172.21.15.5:0/3782287312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 0x55c352699140 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f647000cb20 tx=0x7f647000e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14f20ec6eb8871a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f646cff9640 1 -- 172.21.15.5:0/3782287312 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470014070 con 0x55c35268edc0 2024-09-19T06:24:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f647ed9d640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f64680a33b0 0x7f64680a5870 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f64780049d0 tx=0x7f6478004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f646cff9640 1 -- 172.21.15.5:0/3782287312 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6470005020 con 0x55c35268edc0 2024-09-19T06:24:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f646cff9640 1 -- 172.21.15.5:0/3782287312 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470004b20 con 0x55c35268edc0 2024-09-19T06:24:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 -- 172.21.15.5:0/3782287312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 msgr2=0x55c352699140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 --2- 172.21.15.5:0/3782287312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 0x55c352699140 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f647000cb20 tx=0x7f647000e350 comp rx=0 tx=0).stop 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 -- 172.21.15.5:0/3782287312 shutdown_connections 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 --2- 172.21.15.5:0/3782287312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c352699680 0x55c35269ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 --2- 172.21.15.5:0/3782287312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 0x55c352699140 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 --2- 172.21.15.5:0/3782287312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 0x55c35268e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 -- 172.21.15.5:0/3782287312 >> 172.21.15.5:0/3782287312 conn(0x55c35268aa30 msgr2=0x55c35268ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 -- 172.21.15.5:0/3782287312 shutdown_connections 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 -- 172.21.15.5:0/3782287312 wait complete. 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 -- start start 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 Processor -- start 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268e4a0 0x55c352698f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268edc0 0x55c352696230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c352699680 0x55c352696770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c35269c250 con 0x55c35268edc0 2024-09-19T06:24:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c3525af050 con 0x55c352699680 2024-09-19T06:24:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c35269c3d0 con 0x55c35268e4a0 2024-09-19T06:24:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.935+0000 7f6486bef740 10 client.0 _ll_register_callbacks cb 0x55c3525ef160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f647dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268edc0 0x55c352696230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f647dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268edc0 0x55c352696230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58940/0 (socket says 172.21.15.5:58940) 2024-09-19T06:24:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f647dd9b640 1 -- 172.21.15.5:0/3235689897 learned_addr learned my addr 172.21.15.5:0/3235689897 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f647ed9d640 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c352699680 0x55c352696770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f647e59c640 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268e4a0 0x55c352698f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f647dd9b640 1 -- 172.21.15.5:0/3235689897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268e4a0 msgr2=0x55c352698f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f647dd9b640 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268e4a0 0x55c352698f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f647dd9b640 1 -- 172.21.15.5:0/3235689897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c352699680 msgr2=0x55c352696770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f647dd9b640 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c352699680 0x55c352696770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f647dd9b640 1 -- 172.21.15.5:0/3235689897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3523653c0 con 0x55c35268edc0 2024-09-19T06:24:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f647dd9b640 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268edc0 0x55c352696230 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f64740ee310 tx=0x7f64740bc200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2120064276763a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f6465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:24:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f6465ffb640 1 -- 172.21.15.5:0/3235689897 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6474049070 con 0x55c35268edc0 2024-09-19T06:24:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f6465ffb640 1 -- 172.21.15.5:0/3235689897 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f64740bcb00 con 0x55c35268edc0 2024-09-19T06:24:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f6486bef740 1 -- 172.21.15.5:0/3235689897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6470005020 con 0x55c35268edc0 2024-09-19T06:24:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f6465ffb640 1 -- 172.21.15.5:0/3235689897 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f647403b460 con 0x55c35268edc0 2024-09-19T06:24:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f6486bef740 10 client.20595 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.936+0000 7f6486bef740 1 -- 172.21.15.5:0/3235689897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55c3526a7bc0 con 0x55c35268edc0 2024-09-19T06:24:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.937+0000 7f6465ffb640 1 -- 172.21.15.5:0/3235689897 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6470005020 con 0x55c35268edc0 2024-09-19T06:24:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.937+0000 7f6465ffb640 1 -- 172.21.15.5:0/3235689897 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f64740c5080 con 0x55c35268edc0 2024-09-19T06:24:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.937+0000 7f6486bef740 10 client.20595 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.937+0000 7f6486bef740 10 client.20595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.937+0000 7f6486bef740 20 client.20595 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.937+0000 7f6486bef740 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c3526a8fb0 0x55c3526ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.937+0000 7f6486bef740 4 client.20595 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.937+0000 7f6486bef740 1 -- 172.21.15.5:0/3235689897 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c352601850 con 0x55c3526a8fb0 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.937+0000 7f647e59c640 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c3526a8fb0 0x55c3526ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.937+0000 7f6465ffb640 10 client.20595 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.937+0000 7f647e59c640 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c3526a8fb0 0x55c3526ab3c0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cf4813c1ccfc0c1 server_cookie=bd5e401f7811770a in_seq=0 out_seq=0 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6465ffb640 1 -- 172.21.15.5:0/3235689897 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c352601850 con 0x55c3526a8fb0 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6465ffb640 10 client.20595 handle_command_reply: tid=0 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6486bef740 1 client.20595 shutdown 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6486bef740 1 -- 172.21.15.5:0/3235689897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268edc0 msgr2=0x55c352696230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6486bef740 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268edc0 0x55c352696230 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f64740ee310 tx=0x7f64740bc200 comp rx=0 tx=0).stop 2024-09-19T06:24:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6486bef740 1 -- 172.21.15.5:0/3235689897 shutdown_connections 2024-09-19T06:24:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6486bef740 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c3526a8fb0 0x55c3526ab3c0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6486bef740 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c352699680 0x55c352696770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6486bef740 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268edc0 0x55c352696230 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6486bef740 1 --2- 172.21.15.5:0/3235689897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268e4a0 0x55c352698f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6465ffb640 0 client.20595 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6486bef740 1 -- 172.21.15.5:0/3235689897 >> 172.21.15.5:0/3235689897 conn(0x55c35268aa30 msgr2=0x55c352690760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6486bef740 1 -- 172.21.15.5:0/3235689897 shutdown_connections 2024-09-19T06:24:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6486bef740 1 -- 172.21.15.5:0/3235689897 wait complete. 2024-09-19T06:24:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.938+0000 7f6486bef740 20 client.20595 trim_cache size 0 max 16384 2024-09-19T06:24:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.960+0000 7f6486bef740 1 Processor -- start 2024-09-19T06:24:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.960+0000 7f6486bef740 1 -- start start 2024-09-19T06:24:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f6486bef740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268aa30 0x55c3526a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f6486bef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268e4a0 0x55c3526917a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f6486bef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268edc0 0x55c3526b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f6486bef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64801ad790 con 0x55c35268edc0 2024-09-19T06:24:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f6486bef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f64801af330 con 0x55c35268e4a0 2024-09-19T06:24:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f6486bef740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f64801ade60 con 0x55c35268aa30 2024-09-19T06:24:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f647e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268aa30 0x55c3526a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f647e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268aa30 0x55c3526a54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36524/0 (socket says 172.21.15.5:36524) 2024-09-19T06:24:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f647e59c640 1 -- 172.21.15.5:0/3987150473 learned_addr learned my addr 172.21.15.5:0/3987150473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f647dd9b640 1 --2- 172.21.15.5:0/3987150473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268e4a0 0x55c3526917a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f647ed9d640 1 --2- 172.21.15.5:0/3987150473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268edc0 0x55c3526b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f647dd9b640 1 -- 172.21.15.5:0/3987150473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268aa30 msgr2=0x55c3526a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f647dd9b640 1 --2- 172.21.15.5:0/3987150473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268aa30 0x55c3526a54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f647dd9b640 1 -- 172.21.15.5:0/3987150473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268edc0 msgr2=0x55c3526b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.961+0000 7f647dd9b640 1 --2- 172.21.15.5:0/3987150473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268edc0 0x55c3526b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.962+0000 7f647dd9b640 1 -- 172.21.15.5:0/3987150473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3526aba40 con 0x55c35268e4a0 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.962+0000 7f647dd9b640 1 --2- 172.21.15.5:0/3987150473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268e4a0 0x55c3526917a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f64740bab50 tx=0x7f64740bc330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=272cb9d43605dec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.962+0000 7f6467fff640 1 -- 172.21.15.5:0/3987150473 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6474049070 con 0x55c35268e4a0 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.962+0000 7f6467fff640 1 -- 172.21.15.5:0/3987150473 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64740bc8a0 con 0x55c35268e4a0 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.962+0000 7f6467fff640 1 -- 172.21.15.5:0/3987150473 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64740c5080 con 0x55c35268e4a0 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.964+0000 7f6486bef740 1 -- 172.21.15.5:0/3987150473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268e4a0 msgr2=0x55c3526917a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.964+0000 7f6486bef740 1 --2- 172.21.15.5:0/3987150473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268e4a0 0x55c3526917a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f64740bab50 tx=0x7f64740bc330 comp rx=0 tx=0).stop 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.964+0000 7f6486bef740 1 -- 172.21.15.5:0/3987150473 shutdown_connections 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.964+0000 7f6486bef740 1 --2- 172.21.15.5:0/3987150473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268edc0 0x55c3526b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.964+0000 7f6486bef740 1 --2- 172.21.15.5:0/3987150473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268e4a0 0x55c3526917a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.964+0000 7f6486bef740 1 --2- 172.21.15.5:0/3987150473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268aa30 0x55c3526a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.964+0000 7f6486bef740 1 -- 172.21.15.5:0/3987150473 >> 172.21.15.5:0/3987150473 conn(0x55c3526968d0 msgr2=0x55c352691090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.964+0000 7f6486bef740 1 -- 172.21.15.5:0/3987150473 shutdown_connections 2024-09-19T06:24:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.964+0000 7f6486bef740 1 -- 172.21.15.5:0/3987150473 wait complete. 2024-09-19T06:24:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.964+0000 7f6486bef740 1 -- start start 2024-09-19T06:24:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f6486bef740 1 Processor -- start 2024-09-19T06:24:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f6486bef740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268aa30 0x55c3526a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f6486bef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 0x55c35269d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f6486bef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 0x55c35269d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f6486bef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64801aec60 con 0x55c35268e4a0 2024-09-19T06:24:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f6486bef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f64801ae560 con 0x55c35268edc0 2024-09-19T06:24:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f6486bef740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f64801ab790 con 0x55c35268aa30 2024-09-19T06:24:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f6486bef740 10 client.0 _ll_register_callbacks cb 0x7f6474009230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f647dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 0x55c35269d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f647ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 0x55c35269d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f647dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 0x55c35269d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58952/0 (socket says 172.21.15.5:58952) 2024-09-19T06:24:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f647ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 0x55c35269d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56192/0 (socket says 172.21.15.5:56192) 2024-09-19T06:24:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.965+0000 7f647dd9b640 1 -- 172.21.15.5:0/1126200918 learned_addr learned my addr 172.21.15.5:0/1126200918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.966+0000 7f647e59c640 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268aa30 0x55c3526a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.966+0000 7f647ed9d640 1 -- 172.21.15.5:0/1126200918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268aa30 msgr2=0x55c3526a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.966+0000 7f647ed9d640 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268aa30 0x55c3526a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.966+0000 7f647ed9d640 1 -- 172.21.15.5:0/1126200918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 msgr2=0x55c35269d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.966+0000 7f647ed9d640 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 0x55c35269d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.966+0000 7f647ed9d640 1 -- 172.21.15.5:0/1126200918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c35268a860 con 0x55c35268edc0 2024-09-19T06:24:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.966+0000 7f647dd9b640 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 0x55c35269d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.966+0000 7f647ed9d640 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 0x55c35269d5e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55c352686630 tx=0x7f6478009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1d4c63ea8f5fcc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.966+0000 7f6465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.966+0000 7f6465ffb640 1 -- 172.21.15.5:0/1126200918 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647802d060 con 0x55c35268edc0 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.967+0000 7f6486bef740 1 -- 172.21.15.5:0/1126200918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64740bc8a0 con 0x55c35268edc0 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.967+0000 7f6465ffb640 1 -- 172.21.15.5:0/1126200918 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8550+0+0 (secure 0 0 0) 0x7f647801d470 con 0x55c35268edc0 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.967+0000 7f6465ffb640 1 -- 172.21.15.5:0/1126200918 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f6478012650 con 0x55c35268edc0 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.967+0000 7f6486bef740 10 client.29966 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.967+0000 7f6486bef740 1 -- 172.21.15.5:0/1126200918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55c352713bb0 con 0x55c35268edc0 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.967+0000 7f6465ffb640 1 -- 172.21.15.5:0/1126200918 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64740bc8a0 con 0x55c35268edc0 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.967+0000 7f6465ffb640 1 -- 172.21.15.5:0/1126200918 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f6478027030 con 0x55c35268edc0 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.967+0000 7f6486bef740 10 client.29966 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.967+0000 7f6486bef740 10 client.29966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.968+0000 7f6486bef740 20 client.29966 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.968+0000 7f6486bef740 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c352699680 0x55c352716320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.968+0000 7f6486bef740 4 client.29966 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.968+0000 7f6486bef740 1 -- 172.21.15.5:0/1126200918 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c3525fbd60 con 0x55c352699680 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.968+0000 7f647e59c640 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c352699680 0x55c352716320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.968+0000 7f6465ffb640 10 client.29966 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.968+0000 7f647e59c640 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c352699680 0x55c352716320 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9258ce353f335da0 server_cookie=507324a82d0be9ae in_seq=0 out_seq=0 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.969+0000 7f6465ffb640 1 -- 172.21.15.5:0/1126200918 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c3525fbd60 con 0x55c352699680 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.969+0000 7f6465ffb640 10 client.29966 handle_command_reply: tid=0 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.969+0000 7f6486bef740 1 client.29966 shutdown 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.969+0000 7f6486bef740 1 -- 172.21.15.5:0/1126200918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 msgr2=0x55c35269d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.969+0000 7f6486bef740 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 0x55c35269d5e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55c352686630 tx=0x7f6478009500 comp rx=0 tx=0).stop 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.969+0000 7f6486bef740 1 -- 172.21.15.5:0/1126200918 shutdown_connections 2024-09-19T06:24:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.969+0000 7f6486bef740 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c352699680 0x55c352716320 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.969+0000 7f6486bef740 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c35268edc0 0x55c35269d5e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.969+0000 7f6486bef740 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35268e4a0 0x55c35269d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.969+0000 7f6465ffb640 0 client.29966 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.969+0000 7f6486bef740 1 --2- 172.21.15.5:0/1126200918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c35268aa30 0x55c3526a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.969+0000 7f6486bef740 1 -- 172.21.15.5:0/1126200918 >> 172.21.15.5:0/1126200918 conn(0x55c3526968d0 msgr2=0x55c352697a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:01.916 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:24:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:24:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:24:01.916 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:24:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.970+0000 7f6486bef740 1 -- 172.21.15.5:0/1126200918 shutdown_connections 2024-09-19T06:24:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.970+0000 7f6486bef740 1 -- 172.21.15.5:0/1126200918 wait complete. 2024-09-19T06:24:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.970+0000 7f6486bef740 20 client.29966 trim_cache size 0 max 16384 2024-09-19T06:24:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.971+0000 7f6484816640 1 -- 172.21.15.5:0/4260852940 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f64680a33b0 msgr2=0x7f64680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.971+0000 7f6484816640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f64680a33b0 0x7f64680a5870 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f64780049d0 tx=0x7f6478004a40 comp rx=0 tx=0).stop 2024-09-19T06:24:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.971+0000 7f6484816640 1 -- 172.21.15.5:0/4260852940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6480101420 msgr2=0x7f6480198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.971+0000 7f6484816640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6480101420 0x7f6480198830 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f6474005940 tx=0x7f64740042f0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.971+0000 7f6484816640 1 -- 172.21.15.5:0/4260852940 shutdown_connections 2024-09-19T06:24:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.971+0000 7f6484816640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f64680a33b0 0x7f64680a5870 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.971+0000 7f6484816640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6480106a80 0x7f648019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.971+0000 7f6484816640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6480101420 0x7f6480198830 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.971+0000 7f6484816640 1 --2- 172.21.15.5:0/4260852940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6480100a80 0x7f64801982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.971+0000 7f6484816640 1 -- 172.21.15.5:0/4260852940 >> 172.21.15.5:0/4260852940 conn(0x7f64800fc7a0 msgr2=0x7f6480104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.971+0000 7f6484816640 1 -- 172.21.15.5:0/4260852940 shutdown_connections 2024-09-19T06:24:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:00.971+0000 7f6484816640 1 -- 172.21.15.5:0/4260852940 wait complete. 2024-09-19T06:24:02.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 73177 8.71 MB/sec execute 326 sec latency 105.506 ms 2024-09-19T06:24:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:02 smithi005 ceph-mon[30955]: pgmap v652: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 247 op/s 2024-09-19T06:24:02.895 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:02 smithi049 ceph-mon[34640]: pgmap v652: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 247 op/s 2024-09-19T06:24:03.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:02 smithi157 ceph-mon[35346]: pgmap v652: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 247 op/s 2024-09-19T06:24:03.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 73296 8.70 MB/sec execute 327 sec latency 199.282 ms 2024-09-19T06:24:04.389 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:24:04.389 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:24:04.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 73422 8.70 MB/sec execute 328 sec latency 312.939 ms 2024-09-19T06:24:04.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:04 smithi005 ceph-mon[30955]: pgmap v653: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 402 op/s 2024-09-19T06:24:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:24:04.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:04 smithi049 ceph-mon[34640]: pgmap v653: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 402 op/s 2024-09-19T06:24:05.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:04 smithi157 ceph-mon[35346]: pgmap v653: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 402 op/s 2024-09-19T06:24:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.377+0000 7f941d1b2640 1 -- 172.21.15.5:0/2167574027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9418100100 msgr2=0x7f9418100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.377+0000 7f941d1b2640 1 --2- 172.21.15.5:0/2167574027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9418100100 0x7f9418100560 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f940c009900 tx=0x7f940c02a3f0 comp rx=0 tx=0).stop 2024-09-19T06:24:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.377+0000 7f941d1b2640 1 -- 172.21.15.5:0/2167574027 shutdown_connections 2024-09-19T06:24:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.377+0000 7f941d1b2640 1 --2- 172.21.15.5:0/2167574027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9418100aa0 0x7f941810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:05.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.377+0000 7f941d1b2640 1 --2- 172.21.15.5:0/2167574027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9418100100 0x7f9418100560 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.377+0000 7f941d1b2640 1 --2- 172.21.15.5:0/2167574027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9418106b80 0x7f9418106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.377+0000 7f941d1b2640 1 -- 172.21.15.5:0/2167574027 >> 172.21.15.5:0/2167574027 conn(0x7f9418077230 msgr2=0x7f94180fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.377+0000 7f941d1b2640 1 -- 172.21.15.5:0/2167574027 shutdown_connections 2024-09-19T06:24:05.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.377+0000 7f941d1b2640 1 -- 172.21.15.5:0/2167574027 wait complete. 2024-09-19T06:24:05.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.378+0000 7f941d1b2640 1 Processor -- start 2024-09-19T06:24:05.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.378+0000 7f941d1b2640 1 -- start start 2024-09-19T06:24:05.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f941d1b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9418100100 0x7f94181012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:05.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f941d1b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9418100aa0 0x7f9418101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f941d1b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9418106b80 0x7f9418101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f941d1b2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9418111ec0 con 0x7f9418106b80 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f941d1b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9418111d40 con 0x7f9418100aa0 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f941d1b2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9418112040 con 0x7f9418100100 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f9416d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9418100100 0x7f94181012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f9416d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9418100100 0x7f94181012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34942/0 (socket says 172.21.15.5:34942) 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f9416d9d640 1 -- 172.21.15.5:0/3214498338 learned_addr learned my addr 172.21.15.5:0/3214498338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f941659c640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9418100aa0 0x7f9418101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f9416d9d640 1 -- 172.21.15.5:0/3214498338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9418100aa0 msgr2=0x7f9418101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f9416d9d640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9418100aa0 0x7f9418101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f9416d9d640 1 -- 172.21.15.5:0/3214498338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9418106b80 msgr2=0x7f9418101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f9416d9d640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9418106b80 0x7f9418101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:05.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.379+0000 7f9416d9d640 1 -- 172.21.15.5:0/3214498338 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94181a68a0 con 0x7f9418100100 2024-09-19T06:24:05.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.380+0000 7f9416d9d640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9418100100 0x7f94181012e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9f0 tx=0x7f940800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:05.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.380+0000 7f941659c640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9418100aa0 0x7f9418101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:24:05.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.380+0000 7f93fffff640 1 -- 172.21.15.5:0/3214498338 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940800cd80 con 0x7f9418100100 2024-09-19T06:24:05.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.380+0000 7f93fffff640 1 -- 172.21.15.5:0/3214498338 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94080108e0 con 0x7f9418100100 2024-09-19T06:24:05.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.380+0000 7f93fffff640 1 -- 172.21.15.5:0/3214498338 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94080188e0 con 0x7f9418100100 2024-09-19T06:24:05.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.381+0000 7f941d1b2640 1 -- 172.21.15.5:0/3214498338 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94181a6b90 con 0x7f9418100100 2024-09-19T06:24:05.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.381+0000 7f941d1b2640 1 -- 172.21.15.5:0/3214498338 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94181a7050 con 0x7f9418100100 2024-09-19T06:24:05.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.382+0000 7f941d1b2640 1 -- 172.21.15.5:0/3214498338 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9418103420 con 0x7f9418100100 2024-09-19T06:24:05.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.384+0000 7f93fffff640 1 -- 172.21.15.5:0/3214498338 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9408010a90 con 0x7f9418100100 2024-09-19T06:24:05.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.386+0000 7f93fffff640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f94000a34b0 0x7f94000a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.386+0000 7f941659c640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f94000a34b0 0x7f94000a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.387+0000 7f93fffff640 1 -- 172.21.15.5:0/3214498338 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8594+0+0 (secure 0 0 0) 0x7f94080d04a0 con 0x7f9418100100 2024-09-19T06:24:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.387+0000 7f941659c640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f94000a34b0 0x7f94000a5970 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f940c031440 tx=0x7f940c005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:05.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.390+0000 7f93fffff640 1 -- 172.21.15.5:0/3214498338 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f940809cdd0 con 0x7f9418100100 2024-09-19T06:24:05.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 73545 8.69 MB/sec execute 329 sec latency 116.008 ms 2024-09-19T06:24:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.773+0000 7f941d1b2640 1 -- 172.21.15.5:0/3214498338 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9418100560 con 0x7f9418100100 2024-09-19T06:24:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.774+0000 7f93fffff640 1 -- 172.21.15.5:0/3214498338 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11961 (secure 0 0 0) 0x7f9418100560 con 0x7f9418100100 2024-09-19T06:24:05.776 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:24:05.776 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":60,"btime":"2024-09-19T06:23:36:530049+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:23:35.567762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:24:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 60 2024-09-19T06:24:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.780+0000 7f941d1b2640 1 -- 172.21.15.5:0/3214498338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f94000a34b0 msgr2=0x7f94000a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.780+0000 7f941d1b2640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f94000a34b0 0x7f94000a5970 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f940c031440 tx=0x7f940c005a20 comp rx=0 tx=0).stop 2024-09-19T06:24:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.780+0000 7f941d1b2640 1 -- 172.21.15.5:0/3214498338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9418100100 msgr2=0x7f94181012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.780+0000 7f941d1b2640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9418100100 0x7f94181012e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9f0 tx=0x7f940800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:24:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.780+0000 7f941d1b2640 1 -- 172.21.15.5:0/3214498338 shutdown_connections 2024-09-19T06:24:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.780+0000 7f941d1b2640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9418106b80 0x7f9418101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.780+0000 7f941d1b2640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9418100aa0 0x7f9418101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.780+0000 7f941d1b2640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f94000a34b0 0x7f94000a5970 secure :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f940c031440 tx=0x7f940c005a20 comp rx=0 tx=0).stop 2024-09-19T06:24:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.780+0000 7f941d1b2640 1 --2- 172.21.15.5:0/3214498338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9418100100 0x7f94181012e0 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9f0 tx=0x7f940800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:24:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.780+0000 7f941d1b2640 1 -- 172.21.15.5:0/3214498338 >> 172.21.15.5:0/3214498338 conn(0x7f9418077230 msgr2=0x7f9418105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:05.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.781+0000 7f941d1b2640 1 -- 172.21.15.5:0/3214498338 shutdown_connections 2024-09-19T06:24:05.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:05.781+0000 7f941d1b2640 1 -- 172.21.15.5:0/3214498338 wait complete. 2024-09-19T06:24:05.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:05 smithi005 ceph-mon[30955]: osdmap e114: 12 total, 12 up, 12 in 2024-09-19T06:24:05.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:05 smithi049 ceph-mon[34640]: osdmap e114: 12 total, 12 up, 12 in 2024-09-19T06:24:06.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:05 smithi157 ceph-mon[35346]: osdmap e114: 12 total, 12 up, 12 in 2024-09-19T06:24:06.472 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:24:06.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 73730 8.71 MB/sec execute 330 sec latency 73.269 ms 2024-09-19T06:24:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:24:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:06 smithi005 ceph-mon[30955]: pgmap v655: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 338 op/s 2024-09-19T06:24:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:06 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3214498338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:24:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:06 smithi049 ceph-mon[34640]: pgmap v655: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 338 op/s 2024-09-19T06:24:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:06 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3214498338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:24:07.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:06 smithi157 ceph-mon[35346]: pgmap v655: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 338 op/s 2024-09-19T06:24:07.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:06 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3214498338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:24:07.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.416+0000 7fab3a470640 1 -- 172.21.15.5:0/2489085679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab34106f10 msgr2=0x7fab3410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:07.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.416+0000 7fab3a470640 1 --2- 172.21.15.5:0/2489085679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab34106f10 0x7fab3410ea90 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009ca0 tx=0x7fab2c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.416+0000 7fab3a470640 1 -- 172.21.15.5:0/2489085679 shutdown_connections 2024-09-19T06:24:07.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.416+0000 7fab3a470640 1 --2- 172.21.15.5:0/2489085679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab34106f10 0x7fab3410ea90 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.416+0000 7fab3a470640 1 --2- 172.21.15.5:0/2489085679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34102790 0x7fab341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.416+0000 7fab3a470640 1 --2- 172.21.15.5:0/2489085679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab34101de0 0x7fab341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.416+0000 7fab3a470640 1 -- 172.21.15.5:0/2489085679 >> 172.21.15.5:0/2489085679 conn(0x7fab340fd8e0 msgr2=0x7fab340ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:07.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.416+0000 7fab3a470640 1 -- 172.21.15.5:0/2489085679 shutdown_connections 2024-09-19T06:24:07.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.416+0000 7fab3a470640 1 -- 172.21.15.5:0/2489085679 wait complete. 2024-09-19T06:24:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.417+0000 7fab3a470640 1 Processor -- start 2024-09-19T06:24:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.417+0000 7fab3a470640 1 -- start start 2024-09-19T06:24:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.418+0000 7fab3a470640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34101de0 0x7fab34198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.418+0000 7fab3a470640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab34102790 0x7fab34198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.418+0000 7fab3a470640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab34106f10 0x7fab3419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.418+0000 7fab3a470640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111e20 con 0x7fab34106f10 2024-09-19T06:24:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.418+0000 7fab3a470640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111ca0 con 0x7fab34102790 2024-09-19T06:24:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.418+0000 7fab3a470640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111fa0 con 0x7fab34101de0 2024-09-19T06:24:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.418+0000 7fab33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34101de0 0x7fab34198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.418+0000 7fab33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34101de0 0x7fab34198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34962/0 (socket says 172.21.15.5:34962) 2024-09-19T06:24:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.418+0000 7fab38a0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab34106f10 0x7fab3419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.418+0000 7fab33fff640 1 -- 172.21.15.5:0/301033212 learned_addr learned my addr 172.21.15.5:0/301033212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:07.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.418+0000 7fab337fe640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab34102790 0x7fab34198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.419+0000 7fab337fe640 1 -- 172.21.15.5:0/301033212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34101de0 msgr2=0x7fab34198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:07.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.419+0000 7fab337fe640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34101de0 0x7fab34198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.419+0000 7fab337fe640 1 -- 172.21.15.5:0/301033212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab34106f10 msgr2=0x7fab3419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:07.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.419+0000 7fab337fe640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab34106f10 0x7fab3419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.419+0000 7fab337fe640 1 -- 172.21.15.5:0/301033212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3419d2b0 con 0x7fab34102790 2024-09-19T06:24:07.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.419+0000 7fab38a0d640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab34106f10 0x7fab3419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:07.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.419+0000 7fab33fff640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34101de0 0x7fab34198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:07.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.419+0000 7fab337fe640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab34102790 0x7fab34198840 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fab2800e9f0 tx=0x7fab2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:07.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.419+0000 7fab317fa640 1 -- 172.21.15.5:0/301033212 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2800ccc0 con 0x7fab34102790 2024-09-19T06:24:07.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.420+0000 7fab317fa640 1 -- 172.21.15.5:0/301033212 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab28005020 con 0x7fab34102790 2024-09-19T06:24:07.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.420+0000 7fab317fa640 1 -- 172.21.15.5:0/301033212 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2800ccc0 con 0x7fab34102790 2024-09-19T06:24:07.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.420+0000 7fab3a470640 1 -- 172.21.15.5:0/301033212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3419d5a0 con 0x7fab34102790 2024-09-19T06:24:07.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.420+0000 7fab3a470640 1 -- 172.21.15.5:0/301033212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a4e80 con 0x7fab34102790 2024-09-19T06:24:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.423+0000 7fab317fa640 1 -- 172.21.15.5:0/301033212 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fab280051d0 con 0x7fab34102790 2024-09-19T06:24:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.424+0000 7fab3c849740 1 Processor -- start 2024-09-19T06:24:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.424+0000 7fab3c849740 1 -- start start 2024-09-19T06:24:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.424+0000 7fab3c849740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592be45d0 0x555592be49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.424+0000 7fab3c849740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 0x555592bdf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.424+0000 7fab3c849740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 0x555592beb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.424+0000 7fab3c849740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555592b3b2f0 con 0x555592bdf280 2024-09-19T06:24:07.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.424+0000 7fab3c849740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555592ae2cf0 con 0x555592bdfc00 2024-09-19T06:24:07.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.424+0000 7fab3c849740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555592b372b0 con 0x555592be45d0 2024-09-19T06:24:07.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab317fa640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fab1c0a3630 0x7fab1c0a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 0x555592beb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab38a0d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592be45d0 0x555592be49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 0x555592beb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:53920/0 (socket says 172.21.15.5:53920) 2024-09-19T06:24:07.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab337fe640 1 -- 172.21.15.5:0/4088301175 learned_addr learned my addr 172.21.15.5:0/4088301175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:07.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab38a0d640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fab1c0a3630 0x7fab1c0a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab317fa640 1 -- 172.21.15.5:0/301033212 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8594+0+0 (secure 0 0 0) 0x7fab280cf890 con 0x7fab34102790 2024-09-19T06:24:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab33fff640 1 --2- 172.21.15.5:0/4088301175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 0x555592bdf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab38a0d640 1 -- 172.21.15.5:0/4088301175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 msgr2=0x555592beb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab38a0d640 1 --2- 172.21.15.5:0/4088301175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 0x555592beb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab38a0d640 1 -- 172.21.15.5:0/4088301175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 msgr2=0x555592bdf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab38a0d640 1 --2- 172.21.15.5:0/4088301175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 0x555592bdf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab38a0d640 1 -- 172.21.15.5:0/4088301175 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555592ac73a0 con 0x555592be45d0 2024-09-19T06:24:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab33fff640 1 --2- 172.21.15.5:0/4088301175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 0x555592bdf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:24:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.425+0000 7fab38a0d640 1 --2- 172.21.15.5:0/4088301175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592be45d0 0x555592be49b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fab2c031040 tx=0x7fab2c032f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf23d2c8353aaac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.426+0000 7fab38a0d640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fab1c0a3630 0x7fab1c0a5af0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fab2c005fe0 tx=0x7fab2c03b000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.426+0000 7fab1a7fc640 1 -- 172.21.15.5:0/4088301175 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c04e070 con 0x555592be45d0 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.426+0000 7fab1a7fc640 1 -- 172.21.15.5:0/4088301175 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab2c036040 con 0x555592be45d0 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.426+0000 7fab1a7fc640 1 -- 172.21.15.5:0/4088301175 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c032050 con 0x555592be45d0 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.426+0000 7fab3c849740 1 -- 172.21.15.5:0/4088301175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592be45d0 msgr2=0x555592be49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.427+0000 7fab3c849740 1 --2- 172.21.15.5:0/4088301175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592be45d0 0x555592be49b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fab2c031040 tx=0x7fab2c032f60 comp rx=0 tx=0).stop 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.427+0000 7fab3c849740 1 -- 172.21.15.5:0/4088301175 shutdown_connections 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.427+0000 7fab3c849740 1 --2- 172.21.15.5:0/4088301175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 0x555592beb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.427+0000 7fab3c849740 1 --2- 172.21.15.5:0/4088301175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 0x555592bdf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.427+0000 7fab3c849740 1 --2- 172.21.15.5:0/4088301175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592be45d0 0x555592be49b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.427+0000 7fab3c849740 1 -- 172.21.15.5:0/4088301175 >> 172.21.15.5:0/4088301175 conn(0x555592b9e310 msgr2=0x555592bd16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.427+0000 7fab3c849740 1 -- 172.21.15.5:0/4088301175 shutdown_connections 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.427+0000 7fab3c849740 1 -- 172.21.15.5:0/4088301175 wait complete. 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.427+0000 7fab3c849740 1 -- start start 2024-09-19T06:24:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.427+0000 7fab3c849740 1 Processor -- start 2024-09-19T06:24:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.428+0000 7fab3c849740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 0x555592beb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.428+0000 7fab3c849740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 0x555592be84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.428+0000 7fab33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 0x555592beb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.428+0000 7fab33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 0x555592beb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47848/0 (socket says 172.21.15.5:47848) 2024-09-19T06:24:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.428+0000 7fab337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 0x555592be84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.428+0000 7fab337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 0x555592be84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:53930/0 (socket says 172.21.15.5:53930) 2024-09-19T06:24:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.428+0000 7fab3c849740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592be45d0 0x555592be89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.428+0000 7fab33fff640 1 -- 172.21.15.5:0/2645974934 learned_addr learned my addr 172.21.15.5:0/2645974934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.428+0000 7fab3c849740 1 -- 172.21.15.5:0/2645974934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555592be5100 con 0x555592bdf280 2024-09-19T06:24:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.428+0000 7fab3c849740 1 -- 172.21.15.5:0/2645974934 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555592af6060 con 0x555592bdfc00 2024-09-19T06:24:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.428+0000 7fab3c849740 1 -- 172.21.15.5:0/2645974934 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555592beb870 con 0x555592be45d0 2024-09-19T06:24:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.428+0000 7fab38a0d640 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592be45d0 0x555592be89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.429+0000 7fab3c849740 10 client.0 _ll_register_callbacks cb 0x5555928498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.429+0000 7fab33fff640 1 -- 172.21.15.5:0/2645974934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592be45d0 msgr2=0x555592be89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.429+0000 7fab33fff640 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592be45d0 0x555592be89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.429+0000 7fab33fff640 1 -- 172.21.15.5:0/2645974934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 msgr2=0x555592be84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.429+0000 7fab33fff640 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 0x555592be84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.429+0000 7fab33fff640 1 -- 172.21.15.5:0/2645974934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555928ac3c0 con 0x555592bdf280 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.429+0000 7fab337fe640 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 0x555592be84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.429+0000 7fab33fff640 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 0x555592beb1a0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fab24013d30 tx=0x7fab240117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29584e6723a60c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.429+0000 7fab137fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.429+0000 7fab137fe640 1 -- 172.21.15.5:0/2645974934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24020060 con 0x555592bdf280 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.430+0000 7fab137fe640 1 -- 172.21.15.5:0/2645974934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8594+0+0 (secure 0 0 0) 0x7fab2401b630 con 0x555592bdf280 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.430+0000 7fab3c849740 1 -- 172.21.15.5:0/2645974934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab2c036040 con 0x555592bdf280 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.430+0000 7fab137fe640 1 -- 172.21.15.5:0/2645974934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fab2401b830 con 0x555592bdf280 2024-09-19T06:24:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.430+0000 7fab3c849740 10 client.20613 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.430+0000 7fab3c849740 1 -- 172.21.15.5:0/2645974934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x555592bf7010 con 0x555592bdf280 2024-09-19T06:24:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.430+0000 7fab137fe640 1 -- 172.21.15.5:0/2645974934 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab2c036040 con 0x555592bdf280 2024-09-19T06:24:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.431+0000 7fab137fe640 1 -- 172.21.15.5:0/2645974934 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fab2400fba0 con 0x555592bdf280 2024-09-19T06:24:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.431+0000 7fab3c849740 10 client.20613 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.431+0000 7fab3c849740 10 client.20613 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:24:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.431+0000 7fab3c849740 20 client.20613 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.431+0000 7fab3c849740 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555592bf8420 0x555592bfa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.431+0000 7fab3c849740 4 client.20613 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:24:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.431+0000 7fab3c849740 1 -- 172.21.15.5:0/2645974934 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555592b48860 con 0x555592bf8420 2024-09-19T06:24:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.431+0000 7fab337fe640 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555592bf8420 0x555592bfa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.432+0000 7fab337fe640 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555592bf8420 0x555592bfa890 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea64756e2924fe5c server_cookie=6fcc75e4716fbc79 in_seq=0 out_seq=0 2024-09-19T06:24:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.432+0000 7fab137fe640 10 client.20613 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:24:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.434+0000 7fab137fe640 1 -- 172.21.15.5:0/2645974934 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555592b48860 con 0x555592bf8420 2024-09-19T06:24:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.434+0000 7fab137fe640 10 client.20613 handle_command_reply: tid=0 2024-09-19T06:24:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.434+0000 7fab3c849740 1 client.20613 shutdown 2024-09-19T06:24:07.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.435+0000 7fab3c849740 1 -- 172.21.15.5:0/2645974934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 msgr2=0x555592beb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:07.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.435+0000 7fab3c849740 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 0x555592beb1a0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fab24013d30 tx=0x7fab240117b0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.435+0000 7fab3c849740 1 -- 172.21.15.5:0/2645974934 shutdown_connections 2024-09-19T06:24:07.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.435+0000 7fab3c849740 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555592bf8420 0x555592bfa890 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.435+0000 7fab3c849740 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592be45d0 0x555592be89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.435+0000 7fab3c849740 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592bdfc00 0x555592be84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.435+0000 7fab3c849740 1 --2- 172.21.15.5:0/2645974934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592bdf280 0x555592beb1a0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.435+0000 7fab3c849740 1 -- 172.21.15.5:0/2645974934 >> 172.21.15.5:0/2645974934 conn(0x555592b9e310 msgr2=0x555592bdea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:07.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.435+0000 7fab137fe640 0 client.20613 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:24:07.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.435+0000 7fab3c849740 1 -- 172.21.15.5:0/2645974934 shutdown_connections 2024-09-19T06:24:07.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.435+0000 7fab3c849740 1 -- 172.21.15.5:0/2645974934 wait complete. 2024-09-19T06:24:07.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.435+0000 7fab3c849740 20 client.20613 trim_cache size 0 max 16384 2024-09-19T06:24:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.454+0000 7fab3c849740 1 Processor -- start 2024-09-19T06:24:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.454+0000 7fab3c849740 1 -- start start 2024-09-19T06:24:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.454+0000 7fab3c849740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 0x555592be3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.454+0000 7fab3c849740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592be45d0 0x555592bf4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.454+0000 7fab3c849740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592be88c0 0x555592be22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.454+0000 7fab3c849740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab341ab6b0 con 0x555592be88c0 2024-09-19T06:24:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.454+0000 7fab3c849740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab341ad250 con 0x555592be45d0 2024-09-19T06:24:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.454+0000 7fab3c849740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab341abd80 con 0x555592bdfc00 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.454+0000 7fab38a0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592be88c0 0x555592be22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 0x555592be3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab38a0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592be88c0 0x555592be22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47854/0 (socket says 172.21.15.5:47854) 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab38a0d640 1 -- 172.21.15.5:0/3126349507 learned_addr learned my addr 172.21.15.5:0/3126349507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab337fe640 1 --2- 172.21.15.5:0/3126349507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592be45d0 0x555592bf4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab337fe640 1 -- 172.21.15.5:0/3126349507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 msgr2=0x555592be3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab337fe640 1 --2- 172.21.15.5:0/3126349507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 0x555592be3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab337fe640 1 -- 172.21.15.5:0/3126349507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592be88c0 msgr2=0x555592be22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab337fe640 1 --2- 172.21.15.5:0/3126349507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592be88c0 0x555592be22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab337fe640 1 -- 172.21.15.5:0/3126349507 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555592be69f0 con 0x555592be45d0 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab33fff640 1 --2- 172.21.15.5:0/3126349507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 0x555592be3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab38a0d640 1 --2- 172.21.15.5:0/3126349507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592be88c0 0x555592be22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab337fe640 1 --2- 172.21.15.5:0/3126349507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592be45d0 0x555592bf4950 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fab28097f20 tx=0x7fab280b8040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=143e010994747fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.455+0000 7fab19ffb640 1 -- 172.21.15.5:0/3126349507 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2809c1c0 con 0x555592be45d0 2024-09-19T06:24:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.456+0000 7fab19ffb640 1 -- 172.21.15.5:0/3126349507 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab280b80b0 con 0x555592be45d0 2024-09-19T06:24:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.456+0000 7fab19ffb640 1 -- 172.21.15.5:0/3126349507 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28099dc0 con 0x555592be45d0 2024-09-19T06:24:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.456+0000 7fab3c849740 1 -- 172.21.15.5:0/3126349507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592be45d0 msgr2=0x555592bf4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.456+0000 7fab3c849740 1 --2- 172.21.15.5:0/3126349507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592be45d0 0x555592bf4950 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fab28097f20 tx=0x7fab280b8040 comp rx=0 tx=0).stop 2024-09-19T06:24:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.457+0000 7fab3c849740 1 -- 172.21.15.5:0/3126349507 shutdown_connections 2024-09-19T06:24:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.457+0000 7fab3c849740 1 --2- 172.21.15.5:0/3126349507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592be88c0 0x555592be22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.457+0000 7fab3c849740 1 --2- 172.21.15.5:0/3126349507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592be45d0 0x555592bf4950 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.457+0000 7fab3c849740 1 --2- 172.21.15.5:0/3126349507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 0x555592be3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.457+0000 7fab3c849740 1 -- 172.21.15.5:0/3126349507 >> 172.21.15.5:0/3126349507 conn(0x555592b9e310 msgr2=0x555592be7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.457+0000 7fab3c849740 1 -- 172.21.15.5:0/3126349507 shutdown_connections 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.457+0000 7fab3c849740 1 -- 172.21.15.5:0/3126349507 wait complete. 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.457+0000 7fab3c849740 1 -- start start 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.457+0000 7fab3c849740 1 Processor -- start 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.458+0000 7fab3c849740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 0x555592bffe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.458+0000 7fab3c849740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592be45d0 0x555592c00360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.458+0000 7fab3c849740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592be88c0 0x555592c008a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.458+0000 7fab3c849740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab341acb80 con 0x555592be45d0 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.458+0000 7fab33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 0x555592bffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.458+0000 7fab3c849740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab341ac480 con 0x555592be88c0 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.458+0000 7fab38a0d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592be88c0 0x555592c008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.458+0000 7fab33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 0x555592bffe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34992/0 (socket says 172.21.15.5:34992) 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.458+0000 7fab33fff640 1 -- 172.21.15.5:0/2714853825 learned_addr learned my addr 172.21.15.5:0/2714853825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.458+0000 7fab3c849740 1 -- 172.21.15.5:0/2714853825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab341a96b0 con 0x555592bdfc00 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.458+0000 7fab337fe640 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592be45d0 0x555592c00360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.458+0000 7fab3c849740 10 client.0 _ll_register_callbacks cb 0x7fab24009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.459+0000 7fab38a0d640 1 -- 172.21.15.5:0/2714853825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 msgr2=0x555592bffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.459+0000 7fab38a0d640 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 0x555592bffe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.459+0000 7fab38a0d640 1 -- 172.21.15.5:0/2714853825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592be45d0 msgr2=0x555592c00360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.459+0000 7fab38a0d640 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592be45d0 0x555592c00360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.459+0000 7fab38a0d640 1 -- 172.21.15.5:0/2714853825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555592c62180 con 0x555592be88c0 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.459+0000 7fab33fff640 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 0x555592bffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.459+0000 7fab38a0d640 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592be88c0 0x555592c008a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fab2c006010 tx=0x7fab2c031f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd308f13a943b919 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.459+0000 7fab137fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:24:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.459+0000 7fab137fe640 1 -- 172.21.15.5:0/2714853825 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c04e040 con 0x555592be88c0 2024-09-19T06:24:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.459+0000 7fab137fe640 1 -- 172.21.15.5:0/2714853825 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8594+0+0 (secure 0 0 0) 0x7fab2c004510 con 0x555592be88c0 2024-09-19T06:24:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.459+0000 7fab3c849740 1 -- 172.21.15.5:0/2714853825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab280b80b0 con 0x555592be88c0 2024-09-19T06:24:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.460+0000 7fab137fe640 1 -- 172.21.15.5:0/2714853825 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fab2c004710 con 0x555592be88c0 2024-09-19T06:24:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.460+0000 7fab3c849740 10 client.29990 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.460+0000 7fab3c849740 1 -- 172.21.15.5:0/2714853825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x555592c633e0 con 0x555592be88c0 2024-09-19T06:24:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.460+0000 7fab137fe640 1 -- 172.21.15.5:0/2714853825 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab280b80b0 con 0x555592be88c0 2024-09-19T06:24:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.460+0000 7fab137fe640 1 -- 172.21.15.5:0/2714853825 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fab2c030c70 con 0x555592be88c0 2024-09-19T06:24:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.460+0000 7fab3c849740 10 client.29990 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.460+0000 7fab3c849740 10 client.29990 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:24:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.460+0000 7fab3c849740 20 client.29990 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.460+0000 7fab3c849740 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555592c638d0 0x555592c65d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.460+0000 7fab3c849740 4 client.29990 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:24:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.460+0000 7fab3c849740 1 -- 172.21.15.5:0/2714853825 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555592b42d70 con 0x555592c638d0 2024-09-19T06:24:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.461+0000 7fab33fff640 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555592c638d0 0x555592c65d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.461+0000 7fab137fe640 10 client.29990 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:24:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.461+0000 7fab33fff640 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555592c638d0 0x555592c65d30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=153acd6bba1dfca6 server_cookie=a021ea80a4d03c17 in_seq=0 out_seq=0 2024-09-19T06:24:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab137fe640 1 -- 172.21.15.5:0/2714853825 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555592b42d70 con 0x555592c638d0 2024-09-19T06:24:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab137fe640 10 client.29990 handle_command_reply: tid=0 2024-09-19T06:24:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab3c849740 1 client.29990 shutdown 2024-09-19T06:24:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab3c849740 1 -- 172.21.15.5:0/2714853825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592be88c0 msgr2=0x555592c008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab3c849740 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592be88c0 0x555592c008a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fab2c006010 tx=0x7fab2c031f80 comp rx=0 tx=0).stop 2024-09-19T06:24:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab3c849740 1 -- 172.21.15.5:0/2714853825 shutdown_connections 2024-09-19T06:24:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab3c849740 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555592c638d0 0x555592c65d30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab3c849740 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555592be88c0 0x555592c008a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab137fe640 0 client.29990 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:24:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab3c849740 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555592be45d0 0x555592c00360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab3c849740 1 --2- 172.21.15.5:0/2714853825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555592bdfc00 0x555592bffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab3c849740 1 -- 172.21.15.5:0/2714853825 >> 172.21.15.5:0/2714853825 conn(0x555592b9e310 msgr2=0x555592be22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab3c849740 1 -- 172.21.15.5:0/2714853825 shutdown_connections 2024-09-19T06:24:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.462+0000 7fab3c849740 1 -- 172.21.15.5:0/2714853825 wait complete. 2024-09-19T06:24:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.463+0000 7fab3c849740 20 client.29990 trim_cache size 0 max 16384 2024-09-19T06:24:07.468 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:24:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.463+0000 7fab3a470640 1 -- 172.21.15.5:0/301033212 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fab1c0a3630 msgr2=0x7fab1c0a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.463+0000 7fab3a470640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fab1c0a3630 0x7fab1c0a5af0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fab2c005fe0 tx=0x7fab2c03b000 comp rx=0 tx=0).stop 2024-09-19T06:24:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.463+0000 7fab3a470640 1 -- 172.21.15.5:0/301033212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab34102790 msgr2=0x7fab34198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.463+0000 7fab3a470640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab34102790 0x7fab34198840 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fab2800e9f0 tx=0x7fab2800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.463+0000 7fab3a470640 1 -- 172.21.15.5:0/301033212 shutdown_connections 2024-09-19T06:24:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.463+0000 7fab3a470640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab34106f10 0x7fab3419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.463+0000 7fab3a470640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab34102790 0x7fab34198840 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.463+0000 7fab3a470640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fab1c0a3630 0x7fab1c0a5af0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.463+0000 7fab3a470640 1 --2- 172.21.15.5:0/301033212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34101de0 0x7fab34198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.463+0000 7fab3a470640 1 -- 172.21.15.5:0/301033212 >> 172.21.15.5:0/301033212 conn(0x7fab340fd8e0 msgr2=0x7fab340fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.464+0000 7fab3a470640 1 -- 172.21.15.5:0/301033212 shutdown_connections 2024-09-19T06:24:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:07.464+0000 7fab3a470640 1 -- 172.21.15.5:0/301033212 wait complete. 2024-09-19T06:24:07.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 73892 8.72 MB/sec execute 331 sec latency 77.337 ms 2024-09-19T06:24:08.218 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:24:08.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74066 8.73 MB/sec execute 332 sec latency 70.201 ms 2024-09-19T06:24:08.602 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:24:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:08 smithi005 ceph-mon[30955]: pgmap v656: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 316 op/s 2024-09-19T06:24:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:08 smithi005 ceph-mon[30955]: osdmap e115: 12 total, 12 up, 12 in 2024-09-19T06:24:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:08 smithi049 ceph-mon[34640]: pgmap v656: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 316 op/s 2024-09-19T06:24:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:08 smithi049 ceph-mon[34640]: osdmap e115: 12 total, 12 up, 12 in 2024-09-19T06:24:09.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:08 smithi157 ceph-mon[35346]: pgmap v656: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 316 op/s 2024-09-19T06:24:09.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:08 smithi157 ceph-mon[35346]: osdmap e115: 12 total, 12 up, 12 in 2024-09-19T06:24:09.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.183+0000 7f2cc369f640 1 -- 172.21.15.5:0/2401993819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc100e20 msgr2=0x7f2cbc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:09.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.183+0000 7f2cc369f640 1 --2- 172.21.15.5:0/2401993819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc100e20 0x7f2cbc101200 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009900 tx=0x7f2cb802a480 comp rx=0 tx=0).stop 2024-09-19T06:24:09.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.183+0000 7f2cc369f640 1 -- 172.21.15.5:0/2401993819 shutdown_connections 2024-09-19T06:24:09.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.183+0000 7f2cc369f640 1 --2- 172.21.15.5:0/2401993819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc102ef0 0x7f2cbc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.183+0000 7f2cc369f640 1 --2- 172.21.15.5:0/2401993819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc102570 0x7f2cbc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.184+0000 7f2cc369f640 1 --2- 172.21.15.5:0/2401993819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc100e20 0x7f2cbc101200 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.184+0000 7f2cc369f640 1 -- 172.21.15.5:0/2401993819 >> 172.21.15.5:0/2401993819 conn(0x7f2cbc0fc7d0 msgr2=0x7f2cbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:09.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.184+0000 7f2cc369f640 1 -- 172.21.15.5:0/2401993819 shutdown_connections 2024-09-19T06:24:09.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.184+0000 7f2cc369f640 1 -- 172.21.15.5:0/2401993819 wait complete. 2024-09-19T06:24:09.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.185+0000 7f2cc369f640 1 Processor -- start 2024-09-19T06:24:09.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.185+0000 7f2cc369f640 1 -- start start 2024-09-19T06:24:09.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.185+0000 7f2cc369f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc100e20 0x7f2cbc197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.185+0000 7f2cc369f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc102570 0x7f2cbc197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.185+0000 7f2cc369f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc102ef0 0x7f2cbc197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.185+0000 7f2cc369f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc110e80 con 0x7f2cbc102ef0 2024-09-19T06:24:09.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.185+0000 7f2cc369f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc110d00 con 0x7f2cbc100e20 2024-09-19T06:24:09.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.185+0000 7f2cc369f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc111000 con 0x7f2cbc102570 2024-09-19T06:24:09.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.185+0000 7f2cc0c3a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc102570 0x7f2cbc197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.185+0000 7f2cc143b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc100e20 0x7f2cbc197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.185+0000 7f2cc0c3a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc102570 0x7f2cbc197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:35016/0 (socket says 172.21.15.5:35016) 2024-09-19T06:24:09.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.185+0000 7f2cc0c3a640 1 -- 172.21.15.5:0/2586246310 learned_addr learned my addr 172.21.15.5:0/2586246310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:09.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.186+0000 7f2cc1c3c640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc102ef0 0x7f2cbc197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.186+0000 7f2cc0c3a640 1 -- 172.21.15.5:0/2586246310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc100e20 msgr2=0x7f2cbc197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:09.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.186+0000 7f2cc0c3a640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc100e20 0x7f2cbc197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.186+0000 7f2cc0c3a640 1 -- 172.21.15.5:0/2586246310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc102ef0 msgr2=0x7f2cbc197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:09.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.186+0000 7f2cc0c3a640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc102ef0 0x7f2cbc197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.186+0000 7f2cc0c3a640 1 -- 172.21.15.5:0/2586246310 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc1a6970 con 0x7f2cbc102570 2024-09-19T06:24:09.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.186+0000 7f2cc0c3a640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc102570 0x7f2cbc197640 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00ea60 tx=0x7f2cac00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:09.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.186+0000 7f2cb27fc640 1 -- 172.21.15.5:0/2586246310 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00cde0 con 0x7f2cbc102570 2024-09-19T06:24:09.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.187+0000 7f2cb27fc640 1 -- 172.21.15.5:0/2586246310 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cac0035f0 con 0x7f2cbc102570 2024-09-19T06:24:09.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.187+0000 7f2cc369f640 1 -- 172.21.15.5:0/2586246310 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc1a6c60 con 0x7f2cbc102570 2024-09-19T06:24:09.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.187+0000 7f2cc369f640 1 -- 172.21.15.5:0/2586246310 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc1a71a0 con 0x7f2cbc102570 2024-09-19T06:24:09.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.187+0000 7f2cb27fc640 1 -- 172.21.15.5:0/2586246310 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac010690 con 0x7f2cbc102570 2024-09-19T06:24:09.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.190+0000 7f2cc5a78740 1 Processor -- start 2024-09-19T06:24:09.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.190+0000 7f2cc5a78740 1 -- start start 2024-09-19T06:24:09.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.190+0000 7f2cc5a78740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 0x561bdc6ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.190+0000 7f2cc5a78740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6ec5d0 0x561bdc6eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.190+0000 7f2cc5a78740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6f7370 0x561bdc6f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.190+0000 7f2cc5a78740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561bdc6522f0 con 0x561bdc6ec5d0 2024-09-19T06:24:09.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.190+0000 7f2cc5a78740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561bdc5f9cf0 con 0x561bdc6edb40 2024-09-19T06:24:09.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.190+0000 7f2cc5a78740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561bdc64e2b0 con 0x561bdc6f7370 2024-09-19T06:24:09.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.190+0000 7f2cb27fc640 1 -- 172.21.15.5:0/2586246310 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2cac004000 con 0x7f2cbc102570 2024-09-19T06:24:09.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.191+0000 7f2cc1c3c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 0x561bdc6ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.191+0000 7f2cc1c3c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 0x561bdc6ec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:53978/0 (socket says 172.21.15.5:53978) 2024-09-19T06:24:09.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.191+0000 7f2cc1c3c640 1 -- 172.21.15.5:0/11026640 learned_addr learned my addr 172.21.15.5:0/11026640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:09.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.191+0000 7f2cc143b640 1 --2- 172.21.15.5:0/11026640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6ec5d0 0x561bdc6eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.191+0000 7f2cc0c3a640 1 --2- 172.21.15.5:0/11026640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6f7370 0x561bdc6f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.191+0000 7f2cb27fc640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2ca80a3430 0x7f2ca80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.192+0000 7f2cb27fc640 1 -- 172.21.15.5:0/2586246310 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f2cac0cfe60 con 0x7f2cbc102570 2024-09-19T06:24:09.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.192+0000 7f2cc1c3c640 1 -- 172.21.15.5:0/11026640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6f7370 msgr2=0x561bdc6f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:09.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.192+0000 7f2cc1c3c640 1 --2- 172.21.15.5:0/11026640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6f7370 0x561bdc6f9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.192+0000 7f2cc1c3c640 1 -- 172.21.15.5:0/11026640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6ec5d0 msgr2=0x561bdc6eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:09.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.192+0000 7f2cc1c3c640 1 --2- 172.21.15.5:0/11026640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6ec5d0 0x561bdc6eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.192+0000 7f2cc1c3c640 1 -- 172.21.15.5:0/11026640 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bdc5de3a0 con 0x561bdc6edb40 2024-09-19T06:24:09.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.192+0000 7f2cc0c3a640 1 --2- 172.21.15.5:0/11026640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6f7370 0x561bdc6f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:24:09.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.192+0000 7f2cc1c3c640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2ca80a3430 0x7f2ca80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.192+0000 7f2cc1c3c640 1 --2- 172.21.15.5:0/11026640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 0x561bdc6ec090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4011590 tx=0x7f2cb4011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e666282c6019df2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:09.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.193+0000 7f2cc1c3c640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2ca80a3430 0x7f2ca80a58f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4022000 tx=0x7f2cb4023000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:09.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.193+0000 7f2ca77fe640 1 -- 172.21.15.5:0/11026640 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4017050 con 0x561bdc6edb40 2024-09-19T06:24:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.193+0000 7f2ca77fe640 1 -- 172.21.15.5:0/11026640 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cb401bb90 con 0x561bdc6edb40 2024-09-19T06:24:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.193+0000 7f2ca77fe640 1 -- 172.21.15.5:0/11026640 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb400e3b0 con 0x561bdc6edb40 2024-09-19T06:24:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.194+0000 7f2cc5a78740 1 -- 172.21.15.5:0/11026640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 msgr2=0x561bdc6ec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.194+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/11026640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 0x561bdc6ec090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4011590 tx=0x7f2cb4011a50 comp rx=0 tx=0).stop 2024-09-19T06:24:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.194+0000 7f2cc5a78740 1 -- 172.21.15.5:0/11026640 shutdown_connections 2024-09-19T06:24:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.194+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/11026640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6f7370 0x561bdc6f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.194+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/11026640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6ec5d0 0x561bdc6eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.194+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/11026640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 0x561bdc6ec090 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.194+0000 7f2cc5a78740 1 -- 172.21.15.5:0/11026640 >> 172.21.15.5:0/11026640 conn(0x561bdc6b5310 msgr2=0x561bdc6e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.194+0000 7f2cc5a78740 1 -- 172.21.15.5:0/11026640 shutdown_connections 2024-09-19T06:24:09.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.195+0000 7f2cc5a78740 1 -- 172.21.15.5:0/11026640 wait complete. 2024-09-19T06:24:09.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.195+0000 7f2cc5a78740 1 -- start start 2024-09-19T06:24:09.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.195+0000 7f2cc5a78740 1 Processor -- start 2024-09-19T06:24:09.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc5a78740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6ec5d0 0x561bdc6f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc5a78740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 0x561bdc6f4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc5a78740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6f7370 0x561bdc6f4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc0c3a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 0x561bdc6f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc5a78740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561bdc6fa070 con 0x561bdc6f7370 2024-09-19T06:24:09.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc5a78740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561bdc60d060 con 0x561bdc6edb40 2024-09-19T06:24:09.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc5a78740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561bdc6fa1f0 con 0x561bdc6ec5d0 2024-09-19T06:24:09.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc0c3a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 0x561bdc6f4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:53980/0 (socket says 172.21.15.5:53980) 2024-09-19T06:24:09.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc0c3a640 1 -- 172.21.15.5:0/2917824682 learned_addr learned my addr 172.21.15.5:0/2917824682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:09.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc1c3c640 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6f7370 0x561bdc6f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc5a78740 10 client.0 _ll_register_callbacks cb 0x561bdc3608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:09.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc0c3a640 1 -- 172.21.15.5:0/2917824682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6ec5d0 msgr2=0x561bdc6f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:09.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc143b640 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6ec5d0 0x561bdc6f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc0c3a640 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6ec5d0 0x561bdc6f6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc0c3a640 1 -- 172.21.15.5:0/2917824682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6f7370 msgr2=0x561bdc6f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:09.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc0c3a640 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6f7370 0x561bdc6f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.196+0000 7f2cc0c3a640 1 -- 172.21.15.5:0/2917824682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bdc3c33c0 con 0x561bdc6edb40 2024-09-19T06:24:09.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.197+0000 7f2cc1c3c640 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6f7370 0x561bdc6f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:09.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.197+0000 7f2cc0c3a640 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 0x561bdc6f4020 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00ea30 tx=0x7f2cac09bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17bbc3f7f18ee7af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:09.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.197+0000 7f2ca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:24:09.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.197+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/2917824682 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac09c0c0 con 0x561bdc6edb40 2024-09-19T06:24:09.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.197+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/2917824682 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f2cac0a0120 con 0x561bdc6edb40 2024-09-19T06:24:09.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.197+0000 7f2cc5a78740 1 -- 172.21.15.5:0/2917824682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cb401bb90 con 0x561bdc6edb40 2024-09-19T06:24:09.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.198+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/2917824682 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2cac0a0470 con 0x561bdc6edb40 2024-09-19T06:24:09.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.198+0000 7f2cc5a78740 10 client.30002 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:09.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.198+0000 7f2cc5a78740 1 -- 172.21.15.5:0/2917824682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x561bdc705b10 con 0x561bdc6edb40 2024-09-19T06:24:09.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.198+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/2917824682 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cb401bb90 con 0x561bdc6edb40 2024-09-19T06:24:09.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.198+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/2917824682 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f2cac0cd7b0 con 0x561bdc6edb40 2024-09-19T06:24:09.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.198+0000 7f2cc5a78740 10 client.30002 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:09.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.198+0000 7f2cc5a78740 10 client.30002 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:24:09.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.198+0000 7f2cc5a78740 20 client.30002 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:09.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.198+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561bdc707560 0x561bdc7099e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.198+0000 7f2cc5a78740 4 client.30002 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:24:09.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.198+0000 7f2cc5a78740 1 -- 172.21.15.5:0/2917824682 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bdc65f860 con 0x561bdc707560 2024-09-19T06:24:09.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.199+0000 7f2cc143b640 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561bdc707560 0x561bdc7099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.200+0000 7f2ca4ff9640 10 client.30002 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:24:09.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.200+0000 7f2cc143b640 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561bdc707560 0x561bdc7099e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef64d0ea4a102afb server_cookie=14d842deb0bbc87f in_seq=0 out_seq=0 2024-09-19T06:24:09.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.201+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/2917824682 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561bdc65f860 con 0x561bdc707560 2024-09-19T06:24:09.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.201+0000 7f2ca4ff9640 10 client.30002 handle_command_reply: tid=0 2024-09-19T06:24:09.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.201+0000 7f2cc5a78740 1 client.30002 shutdown 2024-09-19T06:24:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.201+0000 7f2cc5a78740 1 -- 172.21.15.5:0/2917824682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 msgr2=0x561bdc6f4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.201+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 0x561bdc6f4020 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00ea30 tx=0x7f2cac09bcc0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.201+0000 7f2cc5a78740 1 -- 172.21.15.5:0/2917824682 shutdown_connections 2024-09-19T06:24:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.201+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561bdc707560 0x561bdc7099e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.201+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6f7370 0x561bdc6f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.201+0000 7f2ca4ff9640 0 client.30002 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:24:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.201+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6edb40 0x561bdc6f4020 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.201+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/2917824682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6ec5d0 0x561bdc6f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.202+0000 7f2cc5a78740 1 -- 172.21.15.5:0/2917824682 >> 172.21.15.5:0/2917824682 conn(0x561bdc6b5310 msgr2=0x561bdc6ee930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.202+0000 7f2cc5a78740 1 -- 172.21.15.5:0/2917824682 shutdown_connections 2024-09-19T06:24:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.202+0000 7f2cc5a78740 1 -- 172.21.15.5:0/2917824682 wait complete. 2024-09-19T06:24:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.202+0000 7f2cc5a78740 20 client.30002 trim_cache size 0 max 16384 2024-09-19T06:24:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc5a78740 1 Processor -- start 2024-09-19T06:24:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc5a78740 1 -- start start 2024-09-19T06:24:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc5a78740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6ec5d0 0x561bdc703370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc5a78740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6edb40 0x561bdc6e8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc5a78740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6f7370 0x561bdc6e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc5a78740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca000e740 con 0x561bdc6edb40 2024-09-19T06:24:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc5a78740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca00102e0 con 0x561bdc6ec5d0 2024-09-19T06:24:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc5a78740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca000ee10 con 0x561bdc6f7370 2024-09-19T06:24:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc143b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6ec5d0 0x561bdc703370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc143b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6ec5d0 0x561bdc703370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:53996/0 (socket says 172.21.15.5:53996) 2024-09-19T06:24:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc143b640 1 -- 172.21.15.5:0/3617252398 learned_addr learned my addr 172.21.15.5:0/3617252398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc0c3a640 1 --2- 172.21.15.5:0/3617252398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6edb40 0x561bdc6e8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.224+0000 7f2cc1c3c640 1 --2- 172.21.15.5:0/3617252398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6f7370 0x561bdc6e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.225+0000 7f2cc143b640 1 -- 172.21.15.5:0/3617252398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6f7370 msgr2=0x561bdc6e8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.225+0000 7f2cc143b640 1 --2- 172.21.15.5:0/3617252398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6f7370 0x561bdc6e8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.225+0000 7f2cc143b640 1 -- 172.21.15.5:0/3617252398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6edb40 msgr2=0x561bdc6e8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.225+0000 7f2cc143b640 1 --2- 172.21.15.5:0/3617252398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6edb40 0x561bdc6e8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.225+0000 7f2cc143b640 1 -- 172.21.15.5:0/3617252398 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bdc6f9ad0 con 0x561bdc6ec5d0 2024-09-19T06:24:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.225+0000 7f2cc143b640 1 --2- 172.21.15.5:0/3617252398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6ec5d0 0x561bdc703370 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2cb802aa40 tx=0x7f2cb8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa93d788960b3bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.225+0000 7f2ca6ffd640 1 -- 172.21.15.5:0/3617252398 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8035040 con 0x561bdc6ec5d0 2024-09-19T06:24:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.225+0000 7f2ca6ffd640 1 -- 172.21.15.5:0/3617252398 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cb804a070 con 0x561bdc6ec5d0 2024-09-19T06:24:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.225+0000 7f2ca6ffd640 1 -- 172.21.15.5:0/3617252398 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8004ce0 con 0x561bdc6ec5d0 2024-09-19T06:24:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.226+0000 7f2cc5a78740 1 -- 172.21.15.5:0/3617252398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6ec5d0 msgr2=0x561bdc703370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.226+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/3617252398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6ec5d0 0x561bdc703370 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2cb802aa40 tx=0x7f2cb8005e80 comp rx=0 tx=0).stop 2024-09-19T06:24:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.226+0000 7f2cc5a78740 1 -- 172.21.15.5:0/3617252398 shutdown_connections 2024-09-19T06:24:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.226+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/3617252398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6f7370 0x561bdc6e8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.226+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/3617252398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6edb40 0x561bdc6e8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.226+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/3617252398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6ec5d0 0x561bdc703370 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.226+0000 7f2cc5a78740 1 -- 172.21.15.5:0/3617252398 >> 172.21.15.5:0/3617252398 conn(0x561bdc6b5310 msgr2=0x561bdc6e3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.226+0000 7f2cc5a78740 1 -- 172.21.15.5:0/3617252398 shutdown_connections 2024-09-19T06:24:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.227+0000 7f2cc5a78740 1 -- 172.21.15.5:0/3617252398 wait complete. 2024-09-19T06:24:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.227+0000 7f2cc5a78740 1 -- start start 2024-09-19T06:24:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.227+0000 7f2cc5a78740 1 Processor -- start 2024-09-19T06:24:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc5a78740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6ec5d0 0x561bdc6ff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc5a78740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6edb40 0x561bdc6ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc5a78740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6f7370 0x561bdc6f0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc5a78740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca000fc10 con 0x561bdc6ec5d0 2024-09-19T06:24:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc5a78740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca000f510 con 0x561bdc6f7370 2024-09-19T06:24:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc5a78740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca000c740 con 0x561bdc6edb40 2024-09-19T06:24:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc5a78740 10 client.0 _ll_register_callbacks cb 0x7f2cac004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc143b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6ec5d0 0x561bdc6ff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc143b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6ec5d0 0x561bdc6ff6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47908/0 (socket says 172.21.15.5:47908) 2024-09-19T06:24:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc143b640 1 -- 172.21.15.5:0/1247105239 learned_addr learned my addr 172.21.15.5:0/1247105239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc0c3a640 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6edb40 0x561bdc6ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc1c3c640 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6f7370 0x561bdc6f0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc143b640 1 -- 172.21.15.5:0/1247105239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6edb40 msgr2=0x561bdc6ffc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc143b640 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6edb40 0x561bdc6ffc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc143b640 1 -- 172.21.15.5:0/1247105239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6f7370 msgr2=0x561bdc6f0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc143b640 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6f7370 0x561bdc6f0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.228+0000 7f2cc143b640 1 -- 172.21.15.5:0/1247105239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bdc6f70e0 con 0x561bdc6ec5d0 2024-09-19T06:24:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.229+0000 7f2cc143b640 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6ec5d0 0x561bdc6ff6c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8006010 tx=0x7f2cb8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434482b280b06ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.229+0000 7f2ca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:24:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.229+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1247105239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8035070 con 0x561bdc6ec5d0 2024-09-19T06:24:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.229+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1247105239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f2cb8004a60 con 0x561bdc6ec5d0 2024-09-19T06:24:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.229+0000 7f2cc5a78740 1 -- 172.21.15.5:0/1247105239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cb804a070 con 0x561bdc6ec5d0 2024-09-19T06:24:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.229+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1247105239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2cb8004c60 con 0x561bdc6ec5d0 2024-09-19T06:24:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.229+0000 7f2cc5a78740 10 client.20637 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.229+0000 7f2cc5a78740 1 -- 172.21.15.5:0/1247105239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x561bdc771b20 con 0x561bdc6ec5d0 2024-09-19T06:24:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.229+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1247105239 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cb8053070 con 0x561bdc6ec5d0 2024-09-19T06:24:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.230+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1247105239 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f2cb80037f0 con 0x561bdc6ec5d0 2024-09-19T06:24:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.230+0000 7f2cc5a78740 10 client.20637 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.230+0000 7f2cc5a78740 10 client.20637 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:24:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.230+0000 7f2cc5a78740 20 client.20637 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.230+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561bdc707560 0x561bdc774310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.230+0000 7f2cc5a78740 4 client.20637 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:24:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.230+0000 7f2cc5a78740 1 -- 172.21.15.5:0/1247105239 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2cb804a070 con 0x561bdc707560 2024-09-19T06:24:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.230+0000 7f2cc0c3a640 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561bdc707560 0x561bdc774310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.231+0000 7f2cc0c3a640 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561bdc707560 0x561bdc774310 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d875e505e649b442 server_cookie=25c374ef6f5705c4 in_seq=0 out_seq=0 2024-09-19T06:24:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.231+0000 7f2ca4ff9640 10 client.20637 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:24:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.231+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1247105239 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2cb804a070 con 0x561bdc707560 2024-09-19T06:24:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.232+0000 7f2ca4ff9640 10 client.20637 handle_command_reply: tid=0 2024-09-19T06:24:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.232+0000 7f2cc5a78740 1 client.20637 shutdown 2024-09-19T06:24:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.232+0000 7f2cc5a78740 1 -- 172.21.15.5:0/1247105239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6ec5d0 msgr2=0x561bdc6ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.232+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6ec5d0 0x561bdc6ff6c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8006010 tx=0x7f2cb8009520 comp rx=0 tx=0).stop 2024-09-19T06:24:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.232+0000 7f2cc5a78740 1 -- 172.21.15.5:0/1247105239 shutdown_connections 2024-09-19T06:24:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.232+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561bdc707560 0x561bdc774310 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.232+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561bdc6f7370 0x561bdc6f0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.232+0000 7f2ca4ff9640 0 client.20637 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:24:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.232+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bdc6edb40 0x561bdc6ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.232+0000 7f2cc5a78740 1 --2- 172.21.15.5:0/1247105239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bdc6ec5d0 0x561bdc6ff6c0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.232+0000 7f2cc5a78740 1 -- 172.21.15.5:0/1247105239 >> 172.21.15.5:0/1247105239 conn(0x561bdc6b5310 msgr2=0x561bdc6e8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.233+0000 7f2cc5a78740 1 -- 172.21.15.5:0/1247105239 shutdown_connections 2024-09-19T06:24:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.233+0000 7f2cc5a78740 1 -- 172.21.15.5:0/1247105239 wait complete. 2024-09-19T06:24:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.233+0000 7f2cc5a78740 20 client.20637 trim_cache size 0 max 16384 2024-09-19T06:24:09.238 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:24:09.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.234+0000 7f2cc369f640 1 -- 172.21.15.5:0/2586246310 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2ca80a3430 msgr2=0x7f2ca80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.234+0000 7f2cc369f640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2ca80a3430 0x7f2ca80a58f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4022000 tx=0x7f2cb4023000 comp rx=0 tx=0).stop 2024-09-19T06:24:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.234+0000 7f2cc369f640 1 -- 172.21.15.5:0/2586246310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc102570 msgr2=0x7f2cbc197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.234+0000 7f2cc369f640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc102570 0x7f2cbc197640 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00ea60 tx=0x7f2cac00ef20 comp rx=0 tx=0).stop 2024-09-19T06:24:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.234+0000 7f2cc369f640 1 -- 172.21.15.5:0/2586246310 shutdown_connections 2024-09-19T06:24:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.234+0000 7f2cc369f640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc102ef0 0x7f2cbc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.234+0000 7f2cc369f640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2ca80a3430 0x7f2ca80a58f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.234+0000 7f2cc369f640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc102570 0x7f2cbc197640 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.234+0000 7f2cc369f640 1 --2- 172.21.15.5:0/2586246310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc100e20 0x7f2cbc197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.234+0000 7f2cc369f640 1 -- 172.21.15.5:0/2586246310 >> 172.21.15.5:0/2586246310 conn(0x7f2cbc0fc7d0 msgr2=0x7f2cbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.234+0000 7f2cc369f640 1 -- 172.21.15.5:0/2586246310 shutdown_connections 2024-09-19T06:24:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:09.234+0000 7f2cc369f640 1 -- 172.21.15.5:0/2586246310 wait complete. 2024-09-19T06:24:09.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74190 8.73 MB/sec execute 333 sec latency 96.669 ms 2024-09-19T06:24:10.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74287 8.71 MB/sec execute 334 sec latency 137.944 ms 2024-09-19T06:24:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:10 smithi005 ceph-mon[30955]: pgmap v658: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 396 op/s 2024-09-19T06:24:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:10 smithi049 ceph-mon[34640]: pgmap v658: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 396 op/s 2024-09-19T06:24:11.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:10 smithi157 ceph-mon[35346]: pgmap v658: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 396 op/s 2024-09-19T06:24:11.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74456 8.71 MB/sec execute 335 sec latency 107.625 ms 2024-09-19T06:24:12.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74565 8.69 MB/sec execute 336 sec latency 254.095 ms 2024-09-19T06:24:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:12 smithi005 ceph-mon[30955]: pgmap v659: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 163 op/s 2024-09-19T06:24:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:12 smithi049 ceph-mon[34640]: pgmap v659: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 163 op/s 2024-09-19T06:24:13.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:12 smithi157 ceph-mon[35346]: pgmap v659: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 163 op/s 2024-09-19T06:24:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.204+0000 7f47905f1640 1 -- 172.21.15.5:0/1898455369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4788101ad0 msgr2=0x7f4788101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.204+0000 7f47905f1640 1 --2- 172.21.15.5:0/1898455369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4788101ad0 0x7f4788101f30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f4778009930 tx=0x7f477802a440 comp rx=0 tx=0).stop 2024-09-19T06:24:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.205+0000 7f47905f1640 1 -- 172.21.15.5:0/1898455369 shutdown_connections 2024-09-19T06:24:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.205+0000 7f47905f1640 1 --2- 172.21.15.5:0/1898455369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4788102470 0x7f478810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.205+0000 7f47905f1640 1 --2- 172.21.15.5:0/1898455369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4788101ad0 0x7f4788101f30 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.205+0000 7f47905f1640 1 --2- 172.21.15.5:0/1898455369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4788107b70 0x7f4788107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.205+0000 7f47905f1640 1 -- 172.21.15.5:0/1898455369 >> 172.21.15.5:0/1898455369 conn(0x7f47880fd7d0 msgr2=0x7f47880ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.205+0000 7f47905f1640 1 -- 172.21.15.5:0/1898455369 shutdown_connections 2024-09-19T06:24:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.206+0000 7f47905f1640 1 -- 172.21.15.5:0/1898455369 wait complete. 2024-09-19T06:24:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.207+0000 7f47905f1640 1 Processor -- start 2024-09-19T06:24:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.207+0000 7f47905f1640 1 -- start start 2024-09-19T06:24:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.207+0000 7f47905f1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4788101ad0 0x7f478819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.207+0000 7f47905f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4788102470 0x7f478819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.207+0000 7f47905f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4788107b70 0x7f478819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.207+0000 7f47905f1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4788074b40 con 0x7f4788102470 2024-09-19T06:24:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.207+0000 7f47905f1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47880749c0 con 0x7f4788101ad0 2024-09-19T06:24:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.207+0000 7f47905f1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4788074cc0 con 0x7f4788107b70 2024-09-19T06:24:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.208+0000 7f478e38d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4788101ad0 0x7f478819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.208+0000 7f478e38d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4788101ad0 0x7f478819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54024/0 (socket says 172.21.15.5:54024) 2024-09-19T06:24:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.208+0000 7f478e38d640 1 -- 172.21.15.5:0/2694931039 learned_addr learned my addr 172.21.15.5:0/2694931039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.208+0000 7f478eb8e640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4788107b70 0x7f478819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.208+0000 7f478db8c640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4788102470 0x7f478819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.208+0000 7f478e38d640 1 -- 172.21.15.5:0/2694931039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4788107b70 msgr2=0x7f478819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.208+0000 7f478e38d640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4788107b70 0x7f478819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.208+0000 7f478e38d640 1 -- 172.21.15.5:0/2694931039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4788102470 msgr2=0x7f478819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.208+0000 7f478e38d640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4788102470 0x7f478819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.208+0000 7f478e38d640 1 -- 172.21.15.5:0/2694931039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478819f5e0 con 0x7f4788101ad0 2024-09-19T06:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.208+0000 7f478db8c640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4788102470 0x7f478819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.208+0000 7f478e38d640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4788101ad0 0x7f478819aa00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f478400b450 tx=0x7f478400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.209+0000 7f477f7fe640 1 -- 172.21.15.5:0/2694931039 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784015020 con 0x7f4788101ad0 2024-09-19T06:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.209+0000 7f47905f1640 1 -- 172.21.15.5:0/2694931039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478819f8d0 con 0x7f4788101ad0 2024-09-19T06:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.209+0000 7f47905f1640 1 -- 172.21.15.5:0/2694931039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47881a7160 con 0x7f4788101ad0 2024-09-19T06:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.209+0000 7f477f7fe640 1 -- 172.21.15.5:0/2694931039 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4784011d40 con 0x7f4788101ad0 2024-09-19T06:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.209+0000 7f477f7fe640 1 -- 172.21.15.5:0/2694931039 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47840103e0 con 0x7f4788101ad0 2024-09-19T06:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.211+0000 7f477f7fe640 1 -- 172.21.15.5:0/2694931039 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4784018020 con 0x7f4788101ad0 2024-09-19T06:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.212+0000 7f477f7fe640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f47740a3430 0x7f47740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.212+0000 7f478db8c640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f47740a3430 0x7f47740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.213+0000 7f477f7fe640 1 -- 172.21.15.5:0/2694931039 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f47840d06c0 con 0x7f4788101ad0 2024-09-19T06:24:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.213+0000 7f4793464740 1 Processor -- start 2024-09-19T06:24:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.213+0000 7f4793464740 1 -- start start 2024-09-19T06:24:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.213+0000 7f4793464740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 0x56532ae566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.213+0000 7f478db8c640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f47740a3430 0x7f47740a58f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4778005ab0 tx=0x7f4778005fd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.214+0000 7f4793464740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae56c20 0x56532ae60f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.214+0000 7f478e38d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 0x56532ae566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.214+0000 7f478e38d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 0x56532ae566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54032/0 (socket says 172.21.15.5:54032) 2024-09-19T06:24:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.214+0000 7f4793464740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae614c0 0x56532ae638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.214+0000 7f478db8c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae56c20 0x56532ae60f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.214+0000 7f478db8c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae56c20 0x56532ae60f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:35072/0 (socket says 172.21.15.5:35072) 2024-09-19T06:24:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.214+0000 7f478e38d640 1 -- 172.21.15.5:0/1426687356 learned_addr learned my addr 172.21.15.5:0/1426687356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.214+0000 7f4793464740 1 -- 172.21.15.5:0/1426687356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56532adb22f0 con 0x56532ae614c0 2024-09-19T06:24:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.214+0000 7f4793464740 1 -- 172.21.15.5:0/1426687356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56532ad5a270 con 0x56532ae56300 2024-09-19T06:24:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.214+0000 7f4793464740 1 -- 172.21.15.5:0/1426687356 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56532adae2b0 con 0x56532ae56c20 2024-09-19T06:24:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.214+0000 7f478eb8e640 1 --2- 172.21.15.5:0/1426687356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae614c0 0x56532ae638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.215+0000 7f478e38d640 1 -- 172.21.15.5:0/1426687356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae56c20 msgr2=0x56532ae60f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.215+0000 7f478e38d640 1 --2- 172.21.15.5:0/1426687356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae56c20 0x56532ae60f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.215+0000 7f478e38d640 1 -- 172.21.15.5:0/1426687356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae614c0 msgr2=0x56532ae638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.215+0000 7f478e38d640 1 --2- 172.21.15.5:0/1426687356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae614c0 0x56532ae638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.215+0000 7f478e38d640 1 -- 172.21.15.5:0/1426687356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532ad59340 con 0x56532ae56300 2024-09-19T06:24:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.215+0000 7f478eb8e640 1 --2- 172.21.15.5:0/1426687356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae614c0 0x56532ae638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.215+0000 7f478e38d640 1 --2- 172.21.15.5:0/1426687356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 0x56532ae566e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4784006e80 tx=0x7f478409c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f18748e80b6fe3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.215+0000 7f477cbfe640 1 -- 172.21.15.5:0/1426687356 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47840a56e0 con 0x56532ae56300 2024-09-19T06:24:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.215+0000 7f477cbfe640 1 -- 172.21.15.5:0/1426687356 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f478409ad70 con 0x56532ae56300 2024-09-19T06:24:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.216+0000 7f477cbfe640 1 -- 172.21.15.5:0/1426687356 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478409bec0 con 0x56532ae56300 2024-09-19T06:24:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.217+0000 7f4793464740 1 -- 172.21.15.5:0/1426687356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 msgr2=0x56532ae566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.217+0000 7f4793464740 1 --2- 172.21.15.5:0/1426687356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 0x56532ae566e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4784006e80 tx=0x7f478409c6e0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.217+0000 7f4793464740 1 -- 172.21.15.5:0/1426687356 shutdown_connections 2024-09-19T06:24:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.217+0000 7f4793464740 1 --2- 172.21.15.5:0/1426687356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae614c0 0x56532ae638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.217+0000 7f4793464740 1 --2- 172.21.15.5:0/1426687356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae56c20 0x56532ae60f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.217+0000 7f4793464740 1 --2- 172.21.15.5:0/1426687356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 0x56532ae566e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.217+0000 7f4793464740 1 -- 172.21.15.5:0/1426687356 >> 172.21.15.5:0/1426687356 conn(0x56532ae52870 msgr2=0x56532ae52c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.217+0000 7f4793464740 1 -- 172.21.15.5:0/1426687356 shutdown_connections 2024-09-19T06:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.217+0000 7f4793464740 1 -- 172.21.15.5:0/1426687356 wait complete. 2024-09-19T06:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.217+0000 7f4793464740 1 -- start start 2024-09-19T06:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f4793464740 1 Processor -- start 2024-09-19T06:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f4793464740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae56300 0x56532ae5e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f4793464740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56c20 0x56532ae5e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f4793464740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae614c0 0x56532ae5eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f4793464740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56532ae64090 con 0x56532ae614c0 2024-09-19T06:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f4793464740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56532ad3b290 con 0x56532ae56c20 2024-09-19T06:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f4793464740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56532ae64210 con 0x56532ae56300 2024-09-19T06:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f4793464740 10 client.0 _ll_register_callbacks cb 0x56532ad6c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f478db8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56c20 0x56532ae5e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f478db8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56c20 0x56532ae5e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54044/0 (socket says 172.21.15.5:54044) 2024-09-19T06:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f478db8c640 1 -- 172.21.15.5:0/3749306423 learned_addr learned my addr 172.21.15.5:0/3749306423 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f478e38d640 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae56300 0x56532ae5e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f478eb8e640 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae614c0 0x56532ae5eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f478db8c640 1 -- 172.21.15.5:0/3749306423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae56300 msgr2=0x56532ae5e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f478db8c640 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae56300 0x56532ae5e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.218+0000 7f478db8c640 1 -- 172.21.15.5:0/3749306423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae614c0 msgr2=0x56532ae5eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.219+0000 7f478db8c640 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae614c0 0x56532ae5eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.219+0000 7f478db8c640 1 -- 172.21.15.5:0/3749306423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56532aac5210 con 0x56532ae56c20 2024-09-19T06:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.219+0000 7f478db8c640 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56c20 0x56532ae5e980 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4778009930 tx=0x7f4778031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8b56ad41a1e1786 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.219+0000 7f4771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.219+0000 7f4771ffb640 1 -- 172.21.15.5:0/3749306423 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477804d070 con 0x56532ae56c20 2024-09-19T06:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.219+0000 7f4793464740 1 -- 172.21.15.5:0/3749306423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f478409ad70 con 0x56532ae56c20 2024-09-19T06:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.219+0000 7f4771ffb640 1 -- 172.21.15.5:0/3749306423 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f4778004d90 con 0x56532ae56c20 2024-09-19T06:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.219+0000 7f4771ffb640 1 -- 172.21.15.5:0/3749306423 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f4778059050 con 0x56532ae56c20 2024-09-19T06:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.219+0000 7f4793464740 10 client.30020 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.219+0000 7f4793464740 1 -- 172.21.15.5:0/3749306423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x56532ab91620 con 0x56532ae56c20 2024-09-19T06:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.220+0000 7f4771ffb640 1 -- 172.21.15.5:0/3749306423 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f478409ad70 con 0x56532ae56c20 2024-09-19T06:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.220+0000 7f4771ffb640 1 -- 172.21.15.5:0/3749306423 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f47780326f0 con 0x56532ae56c20 2024-09-19T06:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.220+0000 7f4793464740 10 client.30020 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.220+0000 7f4793464740 10 client.30020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.220+0000 7f4793464740 20 client.30020 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.220+0000 7f4793464740 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56532ae71540 0x56532ae73960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.220+0000 7f4793464740 4 client.30020 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.220+0000 7f4793464740 1 -- 172.21.15.5:0/3749306423 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56532adbfa20 con 0x56532ae71540 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.220+0000 7f478e38d640 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56532ae71540 0x56532ae73960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.221+0000 7f4771ffb640 10 client.30020 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.221+0000 7f478e38d640 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56532ae71540 0x56532ae73960 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d963bf5961c38bd9 server_cookie=808c18d75937828c in_seq=0 out_seq=0 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4771ffb640 1 -- 172.21.15.5:0/3749306423 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56532adbfa20 con 0x56532ae71540 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4771ffb640 10 client.30020 handle_command_reply: tid=0 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4793464740 1 client.30020 shutdown 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4793464740 1 -- 172.21.15.5:0/3749306423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56c20 msgr2=0x56532ae5e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4793464740 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56c20 0x56532ae5e980 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4778009930 tx=0x7f4778031400 comp rx=0 tx=0).stop 2024-09-19T06:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4793464740 1 -- 172.21.15.5:0/3749306423 shutdown_connections 2024-09-19T06:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4793464740 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56532ae71540 0x56532ae73960 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4793464740 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae614c0 0x56532ae5eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4793464740 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56c20 0x56532ae5e980 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4793464740 1 --2- 172.21.15.5:0/3749306423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae56300 0x56532ae5e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4771ffb640 0 client.30020 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4793464740 1 -- 172.21.15.5:0/3749306423 >> 172.21.15.5:0/3749306423 conn(0x56532ae52870 msgr2=0x56532ae58580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4793464740 1 -- 172.21.15.5:0/3749306423 shutdown_connections 2024-09-19T06:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.222+0000 7f4793464740 1 -- 172.21.15.5:0/3749306423 wait complete. 2024-09-19T06:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.223+0000 7f4793464740 20 client.30020 trim_cache size 0 max 16384 2024-09-19T06:24:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.245+0000 7f4793464740 1 Processor -- start 2024-09-19T06:24:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.245+0000 7f4793464740 1 -- start start 2024-09-19T06:24:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.245+0000 7f4793464740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56300 0x56532ae79280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.245+0000 7f4793464740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56c20 0x56532ae797c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.245+0000 7f4793464740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae614c0 0x56532ae5fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.245+0000 7f4793464740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47881ab720 con 0x56532ae56300 2024-09-19T06:24:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.245+0000 7f4793464740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47881ad2c0 con 0x56532ae56c20 2024-09-19T06:24:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.245+0000 7f4793464740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47881abdf0 con 0x56532ae614c0 2024-09-19T06:24:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.246+0000 7f478e38d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56300 0x56532ae79280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.246+0000 7f478e38d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56300 0x56532ae79280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47964/0 (socket says 172.21.15.5:47964) 2024-09-19T06:24:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.246+0000 7f478eb8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae614c0 0x56532ae5fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.246+0000 7f478e38d640 1 -- 172.21.15.5:0/912128021 learned_addr learned my addr 172.21.15.5:0/912128021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.246+0000 7f478db8c640 1 --2- 172.21.15.5:0/912128021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56c20 0x56532ae797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.246+0000 7f478eb8e640 1 -- 172.21.15.5:0/912128021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56c20 msgr2=0x56532ae797c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.246+0000 7f478eb8e640 1 --2- 172.21.15.5:0/912128021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56c20 0x56532ae797c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.246+0000 7f478eb8e640 1 -- 172.21.15.5:0/912128021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56300 msgr2=0x56532ae79280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.246+0000 7f478eb8e640 1 --2- 172.21.15.5:0/912128021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56300 0x56532ae79280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.246+0000 7f478eb8e640 1 -- 172.21.15.5:0/912128021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532ae6dea0 con 0x56532ae614c0 2024-09-19T06:24:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.246+0000 7f478e38d640 1 --2- 172.21.15.5:0/912128021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56300 0x56532ae79280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.247+0000 7f478eb8e640 1 --2- 172.21.15.5:0/912128021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae614c0 0x56532ae5fca0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x56532ae58ea0 tx=0x7f4780011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edfa25024b42c077 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.247+0000 7f4773fff640 1 -- 172.21.15.5:0/912128021 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780021070 con 0x56532ae614c0 2024-09-19T06:24:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.247+0000 7f4773fff640 1 -- 172.21.15.5:0/912128021 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4780003e20 con 0x56532ae614c0 2024-09-19T06:24:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.247+0000 7f4773fff640 1 -- 172.21.15.5:0/912128021 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780004440 con 0x56532ae614c0 2024-09-19T06:24:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.248+0000 7f4793464740 1 -- 172.21.15.5:0/912128021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae614c0 msgr2=0x56532ae5fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.248+0000 7f4793464740 1 --2- 172.21.15.5:0/912128021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae614c0 0x56532ae5fca0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x56532ae58ea0 tx=0x7f4780011bf0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.248+0000 7f4793464740 1 -- 172.21.15.5:0/912128021 shutdown_connections 2024-09-19T06:24:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.248+0000 7f4793464740 1 --2- 172.21.15.5:0/912128021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae614c0 0x56532ae5fca0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.248+0000 7f4793464740 1 --2- 172.21.15.5:0/912128021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56c20 0x56532ae797c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.248+0000 7f4793464740 1 --2- 172.21.15.5:0/912128021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56300 0x56532ae79280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.248+0000 7f4793464740 1 -- 172.21.15.5:0/912128021 >> 172.21.15.5:0/912128021 conn(0x56532ae52870 msgr2=0x56532ae5d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.248+0000 7f4793464740 1 -- 172.21.15.5:0/912128021 shutdown_connections 2024-09-19T06:24:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.248+0000 7f4793464740 1 -- 172.21.15.5:0/912128021 wait complete. 2024-09-19T06:24:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.248+0000 7f4793464740 1 -- start start 2024-09-19T06:24:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.248+0000 7f4793464740 1 Processor -- start 2024-09-19T06:24:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.249+0000 7f4793464740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 0x56532ae69970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.249+0000 7f4793464740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56c20 0x56532ae5af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.249+0000 7f4793464740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae614c0 0x56532ae5b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.249+0000 7f4793464740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47881acbf0 con 0x56532ae56c20 2024-09-19T06:24:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.249+0000 7f4793464740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47881ac4f0 con 0x56532ae56300 2024-09-19T06:24:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.249+0000 7f4793464740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47881a9760 con 0x56532ae614c0 2024-09-19T06:24:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.249+0000 7f4793464740 10 client.0 _ll_register_callbacks cb 0x7f4778009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.249+0000 7f478db8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56c20 0x56532ae5af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.249+0000 7f478db8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56c20 0x56532ae5af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47974/0 (socket says 172.21.15.5:47974) 2024-09-19T06:24:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.249+0000 7f478db8c640 1 -- 172.21.15.5:0/838324230 learned_addr learned my addr 172.21.15.5:0/838324230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.249+0000 7f478e38d640 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 0x56532ae69970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.249+0000 7f478eb8e640 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae614c0 0x56532ae5b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.250+0000 7f478db8c640 1 -- 172.21.15.5:0/838324230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae614c0 msgr2=0x56532ae5b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.250+0000 7f478db8c640 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae614c0 0x56532ae5b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.250+0000 7f478db8c640 1 -- 172.21.15.5:0/838324230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 msgr2=0x56532ae69970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.250+0000 7f478db8c640 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 0x56532ae69970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.250+0000 7f478db8c640 1 -- 172.21.15.5:0/838324230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56532ae5ba70 con 0x56532ae56c20 2024-09-19T06:24:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.250+0000 7f478e38d640 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 0x56532ae69970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:24:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.250+0000 7f478db8c640 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56c20 0x56532ae5af70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f4778005dd0 tx=0x7f47780326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bb090089ae90c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.250+0000 7f4771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:24:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.250+0000 7f4771ffb640 1 -- 172.21.15.5:0/838324230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778061020 con 0x56532ae56c20 2024-09-19T06:24:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.250+0000 7f4771ffb640 1 -- 172.21.15.5:0/838324230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f4778004ea0 con 0x56532ae56c20 2024-09-19T06:24:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.250+0000 7f4793464740 1 -- 172.21.15.5:0/838324230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4780003e20 con 0x56532ae56c20 2024-09-19T06:24:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.251+0000 7f4771ffb640 1 -- 172.21.15.5:0/838324230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f477806c050 con 0x56532ae56c20 2024-09-19T06:24:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.251+0000 7f4793464740 10 client.20652 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.251+0000 7f4793464740 1 -- 172.21.15.5:0/838324230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x56532aedc060 con 0x56532ae56c20 2024-09-19T06:24:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.251+0000 7f4771ffb640 1 -- 172.21.15.5:0/838324230 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4780003e20 con 0x56532ae56c20 2024-09-19T06:24:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.252+0000 7f4771ffb640 1 -- 172.21.15.5:0/838324230 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f477803bb40 con 0x56532ae56c20 2024-09-19T06:24:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.252+0000 7f4793464740 10 client.20652 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.252+0000 7f4793464740 10 client.20652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.252+0000 7f4793464740 20 client.20652 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.252+0000 7f4793464740 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56532aedc890 0x56532aedecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.252+0000 7f4793464740 4 client.20652 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:24:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.252+0000 7f4793464740 1 -- 172.21.15.5:0/838324230 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56532adb9d70 con 0x56532aedc890 2024-09-19T06:24:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.252+0000 7f478e38d640 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56532aedc890 0x56532aedecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.252+0000 7f478e38d640 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56532aedc890 0x56532aedecb0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c339546cc4c9459a server_cookie=2cc6e7bbba149fed in_seq=0 out_seq=0 2024-09-19T06:24:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.252+0000 7f4771ffb640 10 client.20652 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.346+0000 7f4771ffb640 1 -- 172.21.15.5:0/838324230 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56532adb9d70 con 0x56532aedc890 2024-09-19T06:24:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.346+0000 7f4771ffb640 10 client.20652 handle_command_reply: tid=0 2024-09-19T06:24:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.346+0000 7f4793464740 1 client.20652 shutdown 2024-09-19T06:24:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.347+0000 7f4793464740 1 -- 172.21.15.5:0/838324230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56c20 msgr2=0x56532ae5af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.347+0000 7f4793464740 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56c20 0x56532ae5af70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f4778005dd0 tx=0x7f47780326e0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.347+0000 7f4793464740 1 -- 172.21.15.5:0/838324230 shutdown_connections 2024-09-19T06:24:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.347+0000 7f4793464740 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56532aedc890 0x56532aedecb0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.347+0000 7f4793464740 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56532ae614c0 0x56532ae5b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.347+0000 7f4793464740 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532ae56c20 0x56532ae5af70 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.347+0000 7f4771ffb640 0 client.20652 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.347+0000 7f4793464740 1 --2- 172.21.15.5:0/838324230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56532ae56300 0x56532ae69970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.347+0000 7f4793464740 1 -- 172.21.15.5:0/838324230 >> 172.21.15.5:0/838324230 conn(0x56532ae52870 msgr2=0x56532ae489a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.348+0000 7f4793464740 1 -- 172.21.15.5:0/838324230 shutdown_connections 2024-09-19T06:24:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.348+0000 7f4793464740 1 -- 172.21.15.5:0/838324230 wait complete. 2024-09-19T06:24:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.348+0000 7f4793464740 20 client.20652 trim_cache size 0 max 16384 2024-09-19T06:24:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:24:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:24:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:24:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:24:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.349+0000 7f47905f1640 1 -- 172.21.15.5:0/2694931039 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f47740a3430 msgr2=0x7f47740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.349+0000 7f47905f1640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f47740a3430 0x7f47740a58f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4778005ab0 tx=0x7f4778005fd0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.349+0000 7f47905f1640 1 -- 172.21.15.5:0/2694931039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4788101ad0 msgr2=0x7f478819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.349+0000 7f47905f1640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4788101ad0 0x7f478819aa00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f478400b450 tx=0x7f478400b910 comp rx=0 tx=0).stop 2024-09-19T06:24:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.349+0000 7f47905f1640 1 -- 172.21.15.5:0/2694931039 shutdown_connections 2024-09-19T06:24:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.349+0000 7f47905f1640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f47740a3430 0x7f47740a58f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.349+0000 7f47905f1640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4788107b70 0x7f478819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.349+0000 7f47905f1640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4788102470 0x7f478819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.349+0000 7f47905f1640 1 --2- 172.21.15.5:0/2694931039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4788101ad0 0x7f478819aa00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.349+0000 7f47905f1640 1 -- 172.21.15.5:0/2694931039 >> 172.21.15.5:0/2694931039 conn(0x7f47880fd7d0 msgr2=0x7f4788075550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.349+0000 7f47905f1640 1 -- 172.21.15.5:0/2694931039 shutdown_connections 2024-09-19T06:24:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:13.349+0000 7f47905f1640 1 -- 172.21.15.5:0/2694931039 wait complete. 2024-09-19T06:24:13.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:24:13.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74690 8.68 MB/sec execute 337 sec latency 112.975 ms 2024-09-19T06:24:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:24:14.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74833 8.66 MB/sec execute 338 sec latency 77.522 ms 2024-09-19T06:24:14.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:14 smithi005 ceph-mon[30955]: pgmap v660: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.0 MiB/s wr, 308 op/s 2024-09-19T06:24:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:14 smithi049 ceph-mon[34640]: pgmap v660: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.0 MiB/s wr, 308 op/s 2024-09-19T06:24:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:14 smithi157 ceph-mon[35346]: pgmap v660: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.0 MiB/s wr, 308 op/s 2024-09-19T06:24:15.589 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74980 8.67 MB/sec execute 339 sec latency 96.743 ms 2024-09-19T06:24:16.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75115 8.67 MB/sec execute 340 sec latency 122.523 ms 2024-09-19T06:24:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:16 smithi005 ceph-mon[30955]: pgmap v661: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.8 MiB/s wr, 257 op/s 2024-09-19T06:24:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:16 smithi049 ceph-mon[34640]: pgmap v661: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.8 MiB/s wr, 257 op/s 2024-09-19T06:24:17.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:16 smithi157 ceph-mon[35346]: pgmap v661: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.8 MiB/s wr, 257 op/s 2024-09-19T06:24:17.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.495+0000 7f8cc76f8640 1 -- 172.21.15.5:0/257724710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc0102570 msgr2=0x7f8cc01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:17.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.495+0000 7f8cc76f8640 1 --2- 172.21.15.5:0/257724710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc0102570 0x7f8cc01029b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000fb40 tx=0x7f8cb002e650 comp rx=0 tx=0).stop 2024-09-19T06:24:17.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.496+0000 7f8cc76f8640 1 -- 172.21.15.5:0/257724710 shutdown_connections 2024-09-19T06:24:17.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.496+0000 7f8cc76f8640 1 --2- 172.21.15.5:0/257724710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cc0102ef0 0x7f8cc010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.496+0000 7f8cc76f8640 1 --2- 172.21.15.5:0/257724710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc0102570 0x7f8cc01029b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.496+0000 7f8cc76f8640 1 --2- 172.21.15.5:0/257724710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cc0100e20 0x7f8cc0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.496+0000 7f8cc76f8640 1 -- 172.21.15.5:0/257724710 >> 172.21.15.5:0/257724710 conn(0x7f8cc00fc7d0 msgr2=0x7f8cc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:17.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.496+0000 7f8cc76f8640 1 -- 172.21.15.5:0/257724710 shutdown_connections 2024-09-19T06:24:17.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.496+0000 7f8cc76f8640 1 -- 172.21.15.5:0/257724710 wait complete. 2024-09-19T06:24:17.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc76f8640 1 Processor -- start 2024-09-19T06:24:17.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc76f8640 1 -- start start 2024-09-19T06:24:17.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc76f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc0100e20 0x7f8cc0196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc76f8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cc0102570 0x7f8cc0196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc76f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cc0102ef0 0x7f8cc019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc76f8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc010fd30 con 0x7f8cc0102570 2024-09-19T06:24:17.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc76f8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc010fbb0 con 0x7f8cc0102ef0 2024-09-19T06:24:17.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc76f8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc010feb0 con 0x7f8cc0100e20 2024-09-19T06:24:17.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc5c95640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cc0102ef0 0x7f8cc019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc4c93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cc0102570 0x7f8cc0196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc5c95640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cc0102ef0 0x7f8cc019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33782/0 (socket says 172.21.15.5:33782) 2024-09-19T06:24:17.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc5c95640 1 -- 172.21.15.5:0/1678400604 learned_addr learned my addr 172.21.15.5:0/1678400604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:17.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.497+0000 7f8cc5494640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc0100e20 0x7f8cc0196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.498+0000 7f8cc5c95640 1 -- 172.21.15.5:0/1678400604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc0100e20 msgr2=0x7f8cc0196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:17.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.498+0000 7f8cc5c95640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc0100e20 0x7f8cc0196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.498+0000 7f8cc5c95640 1 -- 172.21.15.5:0/1678400604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cc0102570 msgr2=0x7f8cc0196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.498+0000 7f8cc5c95640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cc0102570 0x7f8cc0196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.498+0000 7f8cc5c95640 1 -- 172.21.15.5:0/1678400604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc019b200 con 0x7f8cc0102ef0 2024-09-19T06:24:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.498+0000 7f8cc5c95640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cc0102ef0 0x7f8cc019ab20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800eb10 tx=0x7f8cb800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.498+0000 7f8cb67fc640 1 -- 172.21.15.5:0/1678400604 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8004b20 con 0x7f8cc0102ef0 2024-09-19T06:24:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.498+0000 7f8cc76f8640 1 -- 172.21.15.5:0/1678400604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc019b4f0 con 0x7f8cc0102ef0 2024-09-19T06:24:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.498+0000 7f8cb67fc640 1 -- 172.21.15.5:0/1678400604 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cb8004cc0 con 0x7f8cc0102ef0 2024-09-19T06:24:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.498+0000 7f8cb67fc640 1 -- 172.21.15.5:0/1678400604 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8010690 con 0x7f8cc0102ef0 2024-09-19T06:24:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.498+0000 7f8cc76f8640 1 -- 172.21.15.5:0/1678400604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc01a2dd0 con 0x7f8cc0102ef0 2024-09-19T06:24:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.501+0000 7f8cb67fc640 1 -- 172.21.15.5:0/1678400604 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8cb8005020 con 0x7f8cc0102ef0 2024-09-19T06:24:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.502+0000 7f8cc9ad1740 1 Processor -- start 2024-09-19T06:24:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.502+0000 7f8cc9ad1740 1 -- start start 2024-09-19T06:24:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.502+0000 7f8cc9ad1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e85090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cb67fc640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8cac0a34b0 0x7f8cac0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc9ad1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e85a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc9ad1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e92760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc9ad1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558cb0deb2f0 con 0x558cb0e86b40 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc9ad1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558cb0d92cf0 con 0x558cb0e90370 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc9ad1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558cb0de72b0 con 0x558cb0e855d0 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc5494640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8cac0a34b0 0x7f8cac0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cb67fc640 1 -- 172.21.15.5:0/1678400604 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f8cb80d0340 con 0x7f8cc0102ef0 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc4c93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e85090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc4c93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e85090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55506/0 (socket says 172.21.15.5:55506) 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc4c93640 1 -- 172.21.15.5:0/1748894741 learned_addr learned my addr 172.21.15.5:0/1748894741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc5c95640 1 --2- 172.21.15.5:0/1748894741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e92760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc4c93640 1 --2- 172.21.15.5:0/1748894741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e85a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc5c95640 1 -- 172.21.15.5:0/1748894741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 msgr2=0x558cb0e85a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:17.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc5c95640 1 --2- 172.21.15.5:0/1748894741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e85a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc5c95640 1 -- 172.21.15.5:0/1748894741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 msgr2=0x558cb0e85090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:17.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc5c95640 1 --2- 172.21.15.5:0/1748894741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e85090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc5c95640 1 -- 172.21.15.5:0/1748894741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cb0d773a0 con 0x558cb0e90370 2024-09-19T06:24:17.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.503+0000 7f8cc4c93640 1 --2- 172.21.15.5:0/1748894741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e85a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:24:17.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.504+0000 7f8cc5c95640 1 --2- 172.21.15.5:0/1748894741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e92760 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80cda10 tx=0x7f8cb809c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81d5966653b5fc34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:17.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.504+0000 7f8cc5494640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8cac0a34b0 0x7f8cac0a5970 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00bb10 tx=0x7f8cbc00d040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:17.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.504+0000 7f8cab7fe640 1 -- 172.21.15.5:0/1748894741 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800be30 con 0x558cb0e90370 2024-09-19T06:24:17.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.504+0000 7f8cab7fe640 1 -- 172.21.15.5:0/1748894741 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cb809cca0 con 0x558cb0e90370 2024-09-19T06:24:17.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.504+0000 7f8cab7fe640 1 -- 172.21.15.5:0/1748894741 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb809a8a0 con 0x558cb0e90370 2024-09-19T06:24:17.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.505+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/1748894741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 msgr2=0x558cb0e92760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.505+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/1748894741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e92760 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80cda10 tx=0x7f8cb809c370 comp rx=0 tx=0).stop 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.505+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/1748894741 shutdown_connections 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.505+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/1748894741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e92760 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.505+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/1748894741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e85a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.505+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/1748894741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e85090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.505+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/1748894741 >> 172.21.15.5:0/1748894741 conn(0x558cb0e4e310 msgr2=0x558cb0e81c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.505+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/1748894741 shutdown_connections 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.505+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/1748894741 wait complete. 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.506+0000 7f8cc9ad1740 1 -- start start 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.506+0000 7f8cc9ad1740 1 Processor -- start 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.506+0000 7f8cc9ad1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e8fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.506+0000 7f8cc9ad1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e8d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.506+0000 7f8cc9ad1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e8d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.506+0000 7f8cc9ad1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558cb0e93070 con 0x558cb0e86b40 2024-09-19T06:24:17.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.506+0000 7f8cc9ad1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558cb0da6060 con 0x558cb0e90370 2024-09-19T06:24:17.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.506+0000 7f8cc9ad1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558cb0e931f0 con 0x558cb0e855d0 2024-09-19T06:24:17.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.506+0000 7f8cc9ad1740 10 client.0 _ll_register_callbacks cb 0x558cb0af98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:17.114 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:24:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.506+0000 7f8cc5494640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e8fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.506+0000 7f8cc5c95640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e8d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.506+0000 7f8cc5c95640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e8d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33798/0 (socket says 172.21.15.5:33798) 2024-09-19T06:24:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8cc5c95640 1 -- 172.21.15.5:0/3752284317 learned_addr learned my addr 172.21.15.5:0/3752284317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8cc4c93640 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8cc5c95640 1 -- 172.21.15.5:0/3752284317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 msgr2=0x558cb0e8fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8cc5c95640 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e8fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8cc5c95640 1 -- 172.21.15.5:0/3752284317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 msgr2=0x558cb0e8d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8cc5c95640 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e8d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8cc5c95640 1 -- 172.21.15.5:0/3752284317 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cb0b5c3c0 con 0x558cb0e90370 2024-09-19T06:24:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8cc4c93640 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e8d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:24:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8cc5c95640 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e8d810 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8004f90 tx=0x7f8cb80a0950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdda063972390fa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8ca8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:24:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8ca8ff9640 1 -- 172.21.15.5:0/3752284317 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800bdf0 con 0x558cb0e90370 2024-09-19T06:24:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/3752284317 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb809cca0 con 0x558cb0e90370 2024-09-19T06:24:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.507+0000 7f8ca8ff9640 1 -- 172.21.15.5:0/3752284317 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f8cb80d0040 con 0x558cb0e90370 2024-09-19T06:24:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.508+0000 7f8ca8ff9640 1 -- 172.21.15.5:0/3752284317 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8cb809c5a0 con 0x558cb0e90370 2024-09-19T06:24:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.508+0000 7f8cc9ad1740 10 client.30038 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.508+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/3752284317 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x558cb0ea73a0 con 0x558cb0e90370 2024-09-19T06:24:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.508+0000 7f8ca8ff9640 1 -- 172.21.15.5:0/3752284317 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cb809cca0 con 0x558cb0e90370 2024-09-19T06:24:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.508+0000 7f8ca8ff9640 1 -- 172.21.15.5:0/3752284317 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f8cb807f070 con 0x558cb0e90370 2024-09-19T06:24:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.508+0000 7f8cc9ad1740 10 client.30038 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.508+0000 7f8cc9ad1740 10 client.30038 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:24:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.508+0000 7f8cc9ad1740 20 client.30038 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.508+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x558cb0ea8850 0x558cb0eaac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.508+0000 7f8cc9ad1740 4 client.30038 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:24:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.508+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/3752284317 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cb0df8860 con 0x558cb0ea8850 2024-09-19T06:24:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.508+0000 7f8cc5494640 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x558cb0ea8850 0x558cb0eaac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.509+0000 7f8cc5494640 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x558cb0ea8850 0x558cb0eaac60 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac2844a2dec5fde6 server_cookie=83eef137e883736a in_seq=0 out_seq=0 2024-09-19T06:24:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.509+0000 7f8ca8ff9640 10 client.30038 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:24:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.509+0000 7f8ca8ff9640 1 -- 172.21.15.5:0/3752284317 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cb0df8860 con 0x558cb0ea8850 2024-09-19T06:24:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.509+0000 7f8ca8ff9640 10 client.30038 handle_command_reply: tid=0 2024-09-19T06:24:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8cc9ad1740 1 client.30038 shutdown 2024-09-19T06:24:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/3752284317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 msgr2=0x558cb0e8d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e8d810 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8004f90 tx=0x7f8cb80a0950 comp rx=0 tx=0).stop 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/3752284317 shutdown_connections 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x558cb0ea8850 0x558cb0eaac60 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e8d810 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e8d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8ca8ff9640 0 client.30038 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/3752284317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e8fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/3752284317 >> 172.21.15.5:0/3752284317 conn(0x558cb0e4e310 msgr2=0x558cb0e879c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/3752284317 shutdown_connections 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/3752284317 wait complete. 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.510+0000 7f8cc9ad1740 20 client.30038 trim_cache size 0 max 16384 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc9ad1740 1 Processor -- start 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc9ad1740 1 -- start start 2024-09-19T06:24:17.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc9ad1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e855d0 0x558cb0e88120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc9ad1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e86b40 0x558cb0e88660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc9ad1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0eb0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc9ad1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01ab5a0 con 0x558cb0e855d0 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc9ad1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01ad140 con 0x558cb0e90370 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc9ad1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01abc70 con 0x558cb0e86b40 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc5494640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e855d0 0x558cb0e88120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc5494640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e855d0 0x558cb0e88120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55528/0 (socket says 172.21.15.5:55528) 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc5494640 1 -- 172.21.15.5:0/53510867 learned_addr learned my addr 172.21.15.5:0/53510867 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc5c95640 1 --2- 172.21.15.5:0/53510867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0eb0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.535+0000 7f8cc4c93640 1 --2- 172.21.15.5:0/53510867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e86b40 0x558cb0e88660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.536+0000 7f8cc5494640 1 -- 172.21.15.5:0/53510867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e86b40 msgr2=0x558cb0e88660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.536+0000 7f8cc5494640 1 --2- 172.21.15.5:0/53510867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e86b40 0x558cb0e88660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.536+0000 7f8cc5494640 1 -- 172.21.15.5:0/53510867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 msgr2=0x558cb0eb0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.536+0000 7f8cc5494640 1 --2- 172.21.15.5:0/53510867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0eb0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.536+0000 7f8cc5494640 1 -- 172.21.15.5:0/53510867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cb0e80310 con 0x558cb0e855d0 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.536+0000 7f8cc5494640 1 --2- 172.21.15.5:0/53510867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e855d0 0x558cb0e88120 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x558cb0eab4a0 tx=0x7f8cbc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f87df9f04b65923 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.536+0000 7f8caaffd640 1 -- 172.21.15.5:0/53510867 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc019c00 con 0x558cb0e855d0 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.536+0000 7f8caaffd640 1 -- 172.21.15.5:0/53510867 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cbc033070 con 0x558cb0e855d0 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.536+0000 7f8caaffd640 1 -- 172.21.15.5:0/53510867 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc026670 con 0x558cb0e855d0 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.537+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/53510867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e855d0 msgr2=0x558cb0e88120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.537+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/53510867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e855d0 0x558cb0e88120 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x558cb0eab4a0 tx=0x7f8cbc014000 comp rx=0 tx=0).stop 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.538+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/53510867 shutdown_connections 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.538+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/53510867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0eb0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.538+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/53510867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e86b40 0x558cb0e88660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.538+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/53510867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e855d0 0x558cb0e88120 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.538+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/53510867 >> 172.21.15.5:0/53510867 conn(0x558cb0e4e310 msgr2=0x558cb0e80d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.538+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/53510867 shutdown_connections 2024-09-19T06:24:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.538+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/53510867 wait complete. 2024-09-19T06:24:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.538+0000 7f8cc9ad1740 1 -- start start 2024-09-19T06:24:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.538+0000 7f8cc9ad1740 1 Processor -- start 2024-09-19T06:24:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.539+0000 7f8cc9ad1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e98830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.539+0000 7f8cc9ad1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e98d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.539+0000 7f8cc9ad1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e93db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.539+0000 7f8cc9ad1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01aca70 con 0x558cb0e86b40 2024-09-19T06:24:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.539+0000 7f8cc5494640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e98830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.539+0000 7f8cc5494640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e98830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36352/0 (socket says 172.21.15.5:36352) 2024-09-19T06:24:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.539+0000 7f8cc5494640 1 -- 172.21.15.5:0/835929330 learned_addr learned my addr 172.21.15.5:0/835929330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.540+0000 7f8cc4c93640 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e98d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.540+0000 7f8cc5c95640 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e93db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.540+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/835929330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01ac370 con 0x558cb0e90370 2024-09-19T06:24:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.540+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/835929330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01a95a0 con 0x558cb0e855d0 2024-09-19T06:24:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.540+0000 7f8cc9ad1740 10 client.0 _ll_register_callbacks cb 0x7f8cbc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.540+0000 7f8cc5494640 1 -- 172.21.15.5:0/835929330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 msgr2=0x558cb0e93db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.540+0000 7f8cc5494640 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e93db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.540+0000 7f8cc5494640 1 -- 172.21.15.5:0/835929330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 msgr2=0x558cb0e98d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.540+0000 7f8cc5494640 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e98d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.540+0000 7f8cc5494640 1 -- 172.21.15.5:0/835929330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cb0e82dd0 con 0x558cb0e855d0 2024-09-19T06:24:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.540+0000 7f8cc5494640 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e98830 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00b610 tx=0x7f8cbc009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5cb013a0e74c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.540+0000 7f8ca8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:24:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.541+0000 7f8ca8ff9640 1 -- 172.21.15.5:0/835929330 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc03e060 con 0x558cb0e855d0 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.541+0000 7f8ca8ff9640 1 -- 172.21.15.5:0/835929330 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f8cbc016840 con 0x558cb0e855d0 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.541+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/835929330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cbc033070 con 0x558cb0e855d0 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.541+0000 7f8ca8ff9640 1 -- 172.21.15.5:0/835929330 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8cbc016a40 con 0x558cb0e855d0 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.541+0000 7f8cc9ad1740 10 client.29974 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.541+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/835929330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x558cb0f12f50 con 0x558cb0e855d0 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.542+0000 7f8ca8ff9640 1 -- 172.21.15.5:0/835929330 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cbc033070 con 0x558cb0e855d0 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.542+0000 7f8ca8ff9640 1 -- 172.21.15.5:0/835929330 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f8cbc026410 con 0x558cb0e855d0 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.542+0000 7f8cc9ad1740 10 client.29974 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.542+0000 7f8cc9ad1740 10 client.29974 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.542+0000 7f8cc9ad1740 20 client.29974 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.542+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x558cb0f13570 0x558cb0f15960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.542+0000 7f8cc9ad1740 4 client.29974 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:24:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.542+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/835929330 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558cb0df2d70 con 0x558cb0f13570 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.542+0000 7f8cc4c93640 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x558cb0f13570 0x558cb0f15960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.543+0000 7f8ca8ff9640 10 client.29974 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.543+0000 7f8cc4c93640 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x558cb0f13570 0x558cb0f15960 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=95fd8b9519b9fd77 server_cookie=2c5cdbba5748a437 in_seq=0 out_seq=0 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.544+0000 7f8ca8ff9640 1 -- 172.21.15.5:0/835929330 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558cb0df2d70 con 0x558cb0f13570 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.544+0000 7f8ca8ff9640 10 client.29974 handle_command_reply: tid=0 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.544+0000 7f8cc9ad1740 1 client.29974 shutdown 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.544+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/835929330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 msgr2=0x558cb0e98830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.544+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e98830 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00b610 tx=0x7f8cbc009ba0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.545+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/835929330 shutdown_connections 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.545+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x558cb0f13570 0x558cb0f15960 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.545+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cb0e90370 0x558cb0e93db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.545+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cb0e86b40 0x558cb0e98d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.545+0000 7f8cc9ad1740 1 --2- 172.21.15.5:0/835929330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cb0e855d0 0x558cb0e98830 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.545+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/835929330 >> 172.21.15.5:0/835929330 conn(0x558cb0e4e310 msgr2=0x558cb0e8b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.545+0000 7f8ca8ff9640 0 client.29974 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:24:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.545+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/835929330 shutdown_connections 2024-09-19T06:24:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.545+0000 7f8cc9ad1740 1 -- 172.21.15.5:0/835929330 wait complete. 2024-09-19T06:24:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.545+0000 7f8cc9ad1740 20 client.29974 trim_cache size 0 max 16384 2024-09-19T06:24:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.546+0000 7f8cc76f8640 1 -- 172.21.15.5:0/1678400604 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8cac0a34b0 msgr2=0x7f8cac0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.546+0000 7f8cc76f8640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8cac0a34b0 0x7f8cac0a5970 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00bb10 tx=0x7f8cbc00d040 comp rx=0 tx=0).stop 2024-09-19T06:24:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.546+0000 7f8cc76f8640 1 -- 172.21.15.5:0/1678400604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cc0102ef0 msgr2=0x7f8cc019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.546+0000 7f8cc76f8640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cc0102ef0 0x7f8cc019ab20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800eb10 tx=0x7f8cb800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.546+0000 7f8cc76f8640 1 -- 172.21.15.5:0/1678400604 shutdown_connections 2024-09-19T06:24:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.546+0000 7f8cc76f8640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cc0102ef0 0x7f8cc019ab20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.546+0000 7f8cc76f8640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cc0102570 0x7f8cc0196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.546+0000 7f8cc76f8640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8cac0a34b0 0x7f8cac0a5970 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.546+0000 7f8cc76f8640 1 --2- 172.21.15.5:0/1678400604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc0100e20 0x7f8cc0196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.546+0000 7f8cc76f8640 1 -- 172.21.15.5:0/1678400604 >> 172.21.15.5:0/1678400604 conn(0x7f8cc00fc7d0 msgr2=0x7f8cc01048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.546+0000 7f8cc76f8640 1 -- 172.21.15.5:0/1678400604 shutdown_connections 2024-09-19T06:24:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:16.546+0000 7f8cc76f8640 1 -- 172.21.15.5:0/1678400604 wait complete. 2024-09-19T06:24:17.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75257 8.67 MB/sec execute 341 sec latency 89.420 ms 2024-09-19T06:24:18.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75371 8.65 MB/sec execute 342 sec latency 154.125 ms 2024-09-19T06:24:18.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:18 smithi005 ceph-mon[30955]: pgmap v662: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 290 op/s 2024-09-19T06:24:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:18 smithi049 ceph-mon[34640]: pgmap v662: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 290 op/s 2024-09-19T06:24:19.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:18 smithi157 ceph-mon[35346]: pgmap v662: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 290 op/s 2024-09-19T06:24:19.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75529 8.65 MB/sec execute 343 sec latency 112.475 ms 2024-09-19T06:24:20.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75670 8.64 MB/sec execute 344 sec latency 99.576 ms 2024-09-19T06:24:20.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:20 smithi005 ceph-mon[30955]: pgmap v663: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 279 op/s 2024-09-19T06:24:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:20 smithi049 ceph-mon[34640]: pgmap v663: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 279 op/s 2024-09-19T06:24:21.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:20 smithi157 ceph-mon[35346]: pgmap v663: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 279 op/s 2024-09-19T06:24:21.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75848 8.64 MB/sec execute 345 sec latency 93.227 ms 2024-09-19T06:24:21.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:24:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:24:22.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75974 8.63 MB/sec execute 346 sec latency 69.718 ms 2024-09-19T06:24:22.853 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:22 smithi157 ceph-mon[35346]: pgmap v664: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-19T06:24:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:22 smithi005 ceph-mon[30955]: pgmap v664: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-19T06:24:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:22 smithi049 ceph-mon[34640]: pgmap v664: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-19T06:24:23.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 76069 8.61 MB/sec execute 347 sec latency 142.350 ms 2024-09-19T06:24:24.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 76221 8.62 MB/sec execute 348 sec latency 117.344 ms 2024-09-19T06:24:24.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:24 smithi005 ceph-mon[30955]: pgmap v665: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 354 op/s 2024-09-19T06:24:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:24 smithi049 ceph-mon[34640]: pgmap v665: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 354 op/s 2024-09-19T06:24:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:24 smithi157 ceph-mon[35346]: pgmap v665: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 354 op/s 2024-09-19T06:24:25.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 76377 8.62 MB/sec execute 349 sec latency 101.739 ms 2024-09-19T06:24:25.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.015+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2308569281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ff0100e20 msgr2=0x7f2ff0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:25.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.015+0000 7f2ff5ff9640 1 --2- 172.21.15.5:0/2308569281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ff0100e20 0x7f2ff0101200 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009930 tx=0x7f2fe002a4f0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.015+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2308569281 shutdown_connections 2024-09-19T06:24:25.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.015+0000 7f2ff5ff9640 1 --2- 172.21.15.5:0/2308569281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ff0102ef0 0x7f2ff010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.015+0000 7f2ff5ff9640 1 --2- 172.21.15.5:0/2308569281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff0102570 0x7f2ff01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.015+0000 7f2ff5ff9640 1 --2- 172.21.15.5:0/2308569281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ff0100e20 0x7f2ff0101200 secure :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009930 tx=0x7f2fe002a4f0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.015+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2308569281 >> 172.21.15.5:0/2308569281 conn(0x7f2ff00fc7d0 msgr2=0x7f2ff00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:25.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.015+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2308569281 shutdown_connections 2024-09-19T06:24:25.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.015+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2308569281 wait complete. 2024-09-19T06:24:25.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.016+0000 7f2ff5ff9640 1 Processor -- start 2024-09-19T06:24:25.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.016+0000 7f2ff5ff9640 1 -- start start 2024-09-19T06:24:25.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.017+0000 7f2ff5ff9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ff0102570 0x7f2ff010d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.017+0000 7f2ff5ff9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff0102ef0 0x7f2ff0109970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.017+0000 7f2ff5ff9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ff0109eb0 0x7f2ff010a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.017+0000 7f2ff5ff9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff010fdd0 con 0x7f2ff0109eb0 2024-09-19T06:24:25.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.017+0000 7f2ff5ff9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff010fc50 con 0x7f2ff0102570 2024-09-19T06:24:25.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.017+0000 7f2ff5ff9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff010ff50 con 0x7f2ff0102ef0 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.017+0000 7f2fef7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ff0102570 0x7f2ff010d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.017+0000 7f2fef7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ff0102570 0x7f2ff010d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56486/0 (socket says 172.21.15.5:56486) 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.017+0000 7f2fef7fe640 1 -- 172.21.15.5:0/2341452854 learned_addr learned my addr 172.21.15.5:0/2341452854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.017+0000 7f2feeffd640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff0102ef0 0x7f2ff0109970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.017+0000 7f2feffff640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ff0109eb0 0x7f2ff010a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2fef7fe640 1 -- 172.21.15.5:0/2341452854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff0102ef0 msgr2=0x7f2ff0109970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2fef7fe640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff0102ef0 0x7f2ff0109970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2fef7fe640 1 -- 172.21.15.5:0/2341452854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ff0109eb0 msgr2=0x7f2ff010a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2fef7fe640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ff0109eb0 0x7f2ff010a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2fef7fe640 1 -- 172.21.15.5:0/2341452854 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff01ab380 con 0x7f2ff0102570 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2feeffd640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff0102ef0 0x7f2ff0109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2feffff640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ff0109eb0 0x7f2ff010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:24:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2fef7fe640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ff0102570 0x7f2ff010d8d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009900 tx=0x7f2fe0031ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:25.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2fecff9640 1 -- 172.21.15.5:0/2341452854 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0040030 con 0x7f2ff0102570 2024-09-19T06:24:25.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2fecff9640 1 -- 172.21.15.5:0/2341452854 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fe003c8f0 con 0x7f2ff0102570 2024-09-19T06:24:25.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2341452854 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff01ab5b0 con 0x7f2ff0102570 2024-09-19T06:24:25.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2fecff9640 1 -- 172.21.15.5:0/2341452854 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00447e0 con 0x7f2ff0102570 2024-09-19T06:24:25.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.018+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2341452854 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff01abae0 con 0x7f2ff0102570 2024-09-19T06:24:25.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.021+0000 7f2ff83d2740 1 Processor -- start 2024-09-19T06:24:25.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.022+0000 7f2ff83d2740 1 -- start start 2024-09-19T06:24:25.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.022+0000 7f2fecff9640 1 -- 172.21.15.5:0/2341452854 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2fe0043030 con 0x7f2ff0102570 2024-09-19T06:24:25.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.022+0000 7f2ff83d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb75e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.022+0000 7f2ff83d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb75ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.022+0000 7f2feffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb75e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.022+0000 7f2ff83d2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb76b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.022+0000 7f2ff83d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609cb6c42f0 con 0x5609cb75fb40 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.022+0000 7f2feffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb75e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55238/0 (socket says 172.21.15.5:55238) 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.022+0000 7f2feffff640 1 -- 172.21.15.5:0/1548824644 learned_addr learned my addr 172.21.15.5:0/1548824644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.023+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1548824644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5609cb66bcf0 con 0x5609cb769370 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.023+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1548824644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5609cb6c02b0 con 0x5609cb75e5d0 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.023+0000 7f2fef7fe640 1 --2- 172.21.15.5:0/1548824644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb75ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.023+0000 7f2feeffd640 1 --2- 172.21.15.5:0/1548824644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb76b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.023+0000 7f2feffff640 1 -- 172.21.15.5:0/1548824644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 msgr2=0x5609cb75ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.023+0000 7f2feffff640 1 --2- 172.21.15.5:0/1548824644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb75ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.023+0000 7f2feffff640 1 -- 172.21.15.5:0/1548824644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 msgr2=0x5609cb76b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.023+0000 7f2feffff640 1 --2- 172.21.15.5:0/1548824644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb76b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.023+0000 7f2feffff640 1 -- 172.21.15.5:0/1548824644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609cb6503a0 con 0x5609cb75fb40 2024-09-19T06:24:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.024+0000 7f2feffff640 1 --2- 172.21.15.5:0/1548824644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb75e090 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009fd0 tx=0x7f2fe80108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb1c1548152eeab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:25.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.024+0000 7f2fecff9640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2fd80a3460 0x7f2fd80a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.025+0000 7f2fd5ffb640 1 -- 172.21.15.5:0/1548824644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8011760 con 0x5609cb75fb40 2024-09-19T06:24:25.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.025+0000 7f2fd5ffb640 1 -- 172.21.15.5:0/1548824644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fe8011db0 con 0x5609cb75fb40 2024-09-19T06:24:25.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.025+0000 7f2fef7fe640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2fd80a3460 0x7f2fd80a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.025+0000 7f2fecff9640 1 -- 172.21.15.5:0/2341452854 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f2fe00f0380 con 0x7f2ff0102570 2024-09-19T06:24:25.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.025+0000 7f2fd5ffb640 1 -- 172.21.15.5:0/1548824644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800e3c0 con 0x5609cb75fb40 2024-09-19T06:24:25.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.026+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1548824644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 msgr2=0x5609cb75e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:25.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.026+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/1548824644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb75e090 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009fd0 tx=0x7f2fe80108e0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.026+0000 7f2fef7fe640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2fd80a3460 0x7f2fd80a5920 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x5609cb75bac0 tx=0x7f2fe00f5040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:25.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.026+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1548824644 shutdown_connections 2024-09-19T06:24:25.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.026+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/1548824644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb76b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.026+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/1548824644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb75ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.026+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/1548824644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb75e090 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.027+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1548824644 >> 172.21.15.5:0/1548824644 conn(0x5609cb727310 msgr2=0x5609cb75ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.027+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1548824644 shutdown_connections 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.027+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1548824644 wait complete. 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.028+0000 7f2ff83d2740 1 -- start start 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.028+0000 7f2ff83d2740 1 Processor -- start 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.029+0000 7f2ff83d2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb75e5d0 0x5609cb768d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.029+0000 7f2ff83d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75fb40 0x5609cb765fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.029+0000 7f2ff83d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb769370 0x5609cb766570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.029+0000 7f2ff83d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609cb76c0c0 con 0x5609cb769370 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.029+0000 7f2ff83d2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5609cb67f060 con 0x5609cb75e5d0 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.029+0000 7f2feffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb75e5d0 0x5609cb768d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.029+0000 7f2feffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb75e5d0 0x5609cb768d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56500/0 (socket says 172.21.15.5:56500) 2024-09-19T06:24:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.029+0000 7f2feffff640 1 -- 172.21.15.5:0/3526204655 learned_addr learned my addr 172.21.15.5:0/3526204655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.029+0000 7f2ff83d2740 1 -- 172.21.15.5:0/3526204655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5609cb76c240 con 0x5609cb75fb40 2024-09-19T06:24:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.029+0000 7f2feffff640 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb769370 0x5609cb766570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.029+0000 7f2ff83d2740 10 client.0 _ll_register_callbacks cb 0x5609cb3d28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.029+0000 7f2feeffd640 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75fb40 0x5609cb765fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2feffff640 1 -- 172.21.15.5:0/3526204655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75fb40 msgr2=0x5609cb765fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2feffff640 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75fb40 0x5609cb765fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2feffff640 1 -- 172.21.15.5:0/3526204655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb769370 msgr2=0x5609cb766570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2feffff640 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb769370 0x5609cb766570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2feffff640 1 -- 172.21.15.5:0/3526204655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609cb4353c0 con 0x5609cb75e5d0 2024-09-19T06:24:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2feffff640 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb75e5d0 0x5609cb768d20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2fe801bbc0 tx=0x7f2fe801bac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c73446acd54a102f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2fceffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:24:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2fceffd640 1 -- 172.21.15.5:0/3526204655 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8024040 con 0x5609cb75e5d0 2024-09-19T06:24:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2fceffd640 1 -- 172.21.15.5:0/3526204655 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f2fe8011ba0 con 0x5609cb75e5d0 2024-09-19T06:24:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2ff83d2740 1 -- 172.21.15.5:0/3526204655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fe8011db0 con 0x5609cb75e5d0 2024-09-19T06:24:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2fceffd640 1 -- 172.21.15.5:0/3526204655 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2fe8030050 con 0x5609cb75e5d0 2024-09-19T06:24:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2ff83d2740 10 client.30056 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.030+0000 7f2ff83d2740 1 -- 172.21.15.5:0/3526204655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5609cb777b70 con 0x5609cb75e5d0 2024-09-19T06:24:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.031+0000 7f2fceffd640 1 -- 172.21.15.5:0/3526204655 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fe8011db0 con 0x5609cb75e5d0 2024-09-19T06:24:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.031+0000 7f2fceffd640 1 -- 172.21.15.5:0/3526204655 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f2fe80055c0 con 0x5609cb75e5d0 2024-09-19T06:24:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.031+0000 7f2ff83d2740 10 client.30056 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.031+0000 7f2ff83d2740 10 client.30056 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:24:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.031+0000 7f2ff83d2740 20 client.30056 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.031+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5609cb778f20 0x5609cb77b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.031+0000 7f2ff83d2740 4 client.30056 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:24:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.031+0000 7f2ff83d2740 1 -- 172.21.15.5:0/3526204655 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609cb6d1860 con 0x5609cb778f20 2024-09-19T06:24:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.031+0000 7f2feeffd640 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5609cb778f20 0x5609cb77b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.032+0000 7f2fceffd640 10 client.30056 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:24:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.032+0000 7f2feeffd640 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5609cb778f20 0x5609cb77b310 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33a84e9d0ee1c8ab server_cookie=fec5f1db6fff136b in_seq=0 out_seq=0 2024-09-19T06:24:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.032+0000 7f2fceffd640 1 -- 172.21.15.5:0/3526204655 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609cb6d1860 con 0x5609cb778f20 2024-09-19T06:24:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.032+0000 7f2fceffd640 10 client.30056 handle_command_reply: tid=0 2024-09-19T06:24:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.032+0000 7f2ff83d2740 1 client.30056 shutdown 2024-09-19T06:24:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.033+0000 7f2ff83d2740 1 -- 172.21.15.5:0/3526204655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb75e5d0 msgr2=0x5609cb768d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.033+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb75e5d0 0x5609cb768d20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2fe801bbc0 tx=0x7f2fe801bac0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.033+0000 7f2ff83d2740 1 -- 172.21.15.5:0/3526204655 shutdown_connections 2024-09-19T06:24:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.033+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5609cb778f20 0x5609cb77b310 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.033+0000 7f2fceffd640 0 client.30056 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.033+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb769370 0x5609cb766570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.033+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75fb40 0x5609cb765fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.033+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/3526204655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb75e5d0 0x5609cb768d20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.033+0000 7f2ff83d2740 1 -- 172.21.15.5:0/3526204655 >> 172.21.15.5:0/3526204655 conn(0x5609cb727310 msgr2=0x5609cb760930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.033+0000 7f2ff83d2740 1 -- 172.21.15.5:0/3526204655 shutdown_connections 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.033+0000 7f2ff83d2740 1 -- 172.21.15.5:0/3526204655 wait complete. 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.033+0000 7f2ff83d2740 20 client.30056 trim_cache size 0 max 16384 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2ff83d2740 1 Processor -- start 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2ff83d2740 1 -- start start 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2ff83d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb765890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2ff83d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb767f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2ff83d2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb761120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2ff83d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff01b4460 con 0x5609cb75fb40 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2ff83d2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff01b6000 con 0x5609cb769370 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2ff83d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff01b4b30 con 0x5609cb75e5d0 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb767f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2feffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb765890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb767f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55258/0 (socket says 172.21.15.5:55258) 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2feeffd640 1 -- 172.21.15.5:0/810064829 learned_addr learned my addr 172.21.15.5:0/810064829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2feffff640 1 --2- 172.21.15.5:0/810064829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb765890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41746/0 (socket says 172.21.15.5:41746) 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2feffff640 1 --2- 172.21.15.5:0/810064829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb761120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2feeffd640 1 -- 172.21.15.5:0/810064829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 msgr2=0x5609cb765890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2feeffd640 1 --2- 172.21.15.5:0/810064829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb765890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2feeffd640 1 -- 172.21.15.5:0/810064829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 msgr2=0x5609cb761120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.057+0000 7f2feeffd640 1 --2- 172.21.15.5:0/810064829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb761120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.058+0000 7f2feeffd640 1 -- 172.21.15.5:0/810064829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609cb75f760 con 0x5609cb75fb40 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.058+0000 7f2feeffd640 1 --2- 172.21.15.5:0/810064829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb767f40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400ba90 tx=0x7f2fe4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9738ccafc760338 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.058+0000 7f2fd57fa640 1 -- 172.21.15.5:0/810064829 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4016ac0 con 0x5609cb75fb40 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.058+0000 7f2fd57fa640 1 -- 172.21.15.5:0/810064829 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fe4016c60 con 0x5609cb75fb40 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.058+0000 7f2fd57fa640 1 -- 172.21.15.5:0/810064829 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40143b0 con 0x5609cb75fb40 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.059+0000 7f2ff83d2740 1 -- 172.21.15.5:0/810064829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 msgr2=0x5609cb767f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.059+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/810064829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb767f40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400ba90 tx=0x7f2fe4008040 comp rx=0 tx=0).stop 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.059+0000 7f2ff83d2740 1 -- 172.21.15.5:0/810064829 shutdown_connections 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.059+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/810064829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb761120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.059+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/810064829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb767f40 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.059+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/810064829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb765890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.059+0000 7f2ff83d2740 1 -- 172.21.15.5:0/810064829 >> 172.21.15.5:0/810064829 conn(0x5609cb727310 msgr2=0x5609cb767830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.059+0000 7f2ff83d2740 1 -- 172.21.15.5:0/810064829 shutdown_connections 2024-09-19T06:24:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.059+0000 7f2ff83d2740 1 -- 172.21.15.5:0/810064829 wait complete. 2024-09-19T06:24:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.059+0000 7f2ff83d2740 1 -- start start 2024-09-19T06:24:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2ff83d2740 1 Processor -- start 2024-09-19T06:24:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2ff83d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb771c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2ff83d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb76cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2ff83d2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb76d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2ff83d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff01b5930 con 0x5609cb75fb40 2024-09-19T06:24:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2ff83d2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff01b5230 con 0x5609cb769370 2024-09-19T06:24:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2ff83d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff01b2460 con 0x5609cb75e5d0 2024-09-19T06:24:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2ff83d2740 10 client.0 _ll_register_callbacks cb 0x7f2fe8009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb76cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2feffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb76d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb76cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55264/0 (socket says 172.21.15.5:55264) 2024-09-19T06:24:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2feeffd640 1 -- 172.21.15.5:0/1249357808 learned_addr learned my addr 172.21.15.5:0/1249357808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2feffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb76d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56514/0 (socket says 172.21.15.5:56514) 2024-09-19T06:24:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.060+0000 7f2feffff640 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb771c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.061+0000 7f2feeffd640 1 -- 172.21.15.5:0/1249357808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 msgr2=0x5609cb771c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.061+0000 7f2feeffd640 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb771c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.061+0000 7f2feeffd640 1 -- 172.21.15.5:0/1249357808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 msgr2=0x5609cb76d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.061+0000 7f2feeffd640 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb76d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.061+0000 7f2feeffd640 1 -- 172.21.15.5:0/1249357808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609cb76dd20 con 0x5609cb75fb40 2024-09-19T06:24:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.061+0000 7f2feffff640 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb76d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.061+0000 7f2feeffd640 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb76cfb0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4008010 tx=0x7f2fe4016ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccb10d27c66ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.061+0000 7f2fceffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:24:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.061+0000 7f2fceffd640 1 -- 172.21.15.5:0/1249357808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4029070 con 0x5609cb75fb40 2024-09-19T06:24:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.061+0000 7f2fceffd640 1 -- 172.21.15.5:0/1249357808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f2fe4006040 con 0x5609cb75fb40 2024-09-19T06:24:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.061+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1249357808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fe4016c60 con 0x5609cb75fb40 2024-09-19T06:24:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.062+0000 7f2fceffd640 1 -- 172.21.15.5:0/1249357808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2fe4016ac0 con 0x5609cb75fb40 2024-09-19T06:24:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.062+0000 7f2ff83d2740 10 client.20685 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.062+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1249357808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5609cb7e3a40 con 0x5609cb75fb40 2024-09-19T06:24:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.062+0000 7f2fceffd640 1 -- 172.21.15.5:0/1249357808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fe40066f0 con 0x5609cb75fb40 2024-09-19T06:24:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.062+0000 7f2fceffd640 1 -- 172.21.15.5:0/1249357808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f2fe401c660 con 0x5609cb75fb40 2024-09-19T06:24:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.062+0000 7f2ff83d2740 10 client.20685 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.062+0000 7f2ff83d2740 10 client.20685 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:24:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.062+0000 7f2ff83d2740 20 client.20685 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.062+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5609cb7e4190 0x5609cb7e65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.062+0000 7f2ff83d2740 4 client.20685 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:24:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.062+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1249357808 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2fe4016c60 con 0x5609cb7e4190 2024-09-19T06:24:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.062+0000 7f2feffff640 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5609cb7e4190 0x5609cb7e65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.063+0000 7f2feffff640 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5609cb7e4190 0x5609cb7e65b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5df0731a32693c1e server_cookie=7ca640291ae5909a in_seq=0 out_seq=0 2024-09-19T06:24:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.063+0000 7f2fceffd640 10 client.20685 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:24:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.063+0000 7f2fceffd640 1 -- 172.21.15.5:0/1249357808 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2fe4016c60 con 0x5609cb7e4190 2024-09-19T06:24:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.063+0000 7f2fceffd640 10 client.20685 handle_command_reply: tid=0 2024-09-19T06:24:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.063+0000 7f2ff83d2740 1 client.20685 shutdown 2024-09-19T06:24:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.063+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1249357808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 msgr2=0x5609cb76cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:25.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.063+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb76cfb0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4008010 tx=0x7f2fe4016ea0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.064+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1249357808 shutdown_connections 2024-09-19T06:24:25.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.064+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5609cb7e4190 0x5609cb7e65b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.064+0000 7f2fceffd640 0 client.20685 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:24:25.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.064+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609cb769370 0x5609cb76d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.064+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609cb75fb40 0x5609cb76cfb0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.064+0000 7f2ff83d2740 1 --2- 172.21.15.5:0/1249357808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609cb75e5d0 0x5609cb771c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.064+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1249357808 >> 172.21.15.5:0/1249357808 conn(0x5609cb727310 msgr2=0x5609cb7594e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:25.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.064+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1249357808 shutdown_connections 2024-09-19T06:24:25.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.064+0000 7f2ff83d2740 1 -- 172.21.15.5:0/1249357808 wait complete. 2024-09-19T06:24:25.611 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:24:25.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.064+0000 7f2ff83d2740 20 client.20685 trim_cache size 0 max 16384 2024-09-19T06:24:25.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.065+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2341452854 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2fd80a3460 msgr2=0x7f2fd80a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:25.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.065+0000 7f2ff5ff9640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2fd80a3460 0x7f2fd80a5920 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x5609cb75bac0 tx=0x7f2fe00f5040 comp rx=0 tx=0).stop 2024-09-19T06:24:25.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.065+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2341452854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ff0102570 msgr2=0x7f2ff010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:25.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.065+0000 7f2ff5ff9640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ff0102570 0x7f2ff010d8d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009900 tx=0x7f2fe0031ba0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.065+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2341452854 shutdown_connections 2024-09-19T06:24:25.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.065+0000 7f2ff5ff9640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ff0109eb0 0x7f2ff010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.065+0000 7f2ff5ff9640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2fd80a3460 0x7f2fd80a5920 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.065+0000 7f2ff5ff9640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff0102ef0 0x7f2ff0109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.065+0000 7f2ff5ff9640 1 --2- 172.21.15.5:0/2341452854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ff0102570 0x7f2ff010d8d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:25.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.065+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2341452854 >> 172.21.15.5:0/2341452854 conn(0x7f2ff00fc7d0 msgr2=0x7f2ff0104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:25.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.065+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2341452854 shutdown_connections 2024-09-19T06:24:25.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:25.065+0000 7f2ff5ff9640 1 -- 172.21.15.5:0/2341452854 wait complete. 2024-09-19T06:24:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.486+0000 7f35a36af640 1 -- 172.21.15.5:0/3988933233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c1069e0 msgr2=0x7f359c106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.486+0000 7f35a36af640 1 --2- 172.21.15.5:0/3988933233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c1069e0 0x7f359c106dc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3598009860 tx=0x7f359802a340 comp rx=0 tx=0).stop 2024-09-19T06:24:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.487+0000 7f35a36af640 1 -- 172.21.15.5:0/3988933233 shutdown_connections 2024-09-19T06:24:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.487+0000 7f35a36af640 1 --2- 172.21.15.5:0/3988933233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c101370 0x7f359c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.487+0000 7f35a36af640 1 --2- 172.21.15.5:0/3988933233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f359c1009d0 0x7f359c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.487+0000 7f35a36af640 1 --2- 172.21.15.5:0/3988933233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c1069e0 0x7f359c106dc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.487+0000 7f35a36af640 1 -- 172.21.15.5:0/3988933233 >> 172.21.15.5:0/3988933233 conn(0x7f359c0fc690 msgr2=0x7f359c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.487+0000 7f35a36af640 1 -- 172.21.15.5:0/3988933233 shutdown_connections 2024-09-19T06:24:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.488+0000 7f35a36af640 1 -- 172.21.15.5:0/3988933233 wait complete. 2024-09-19T06:24:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.489+0000 7f35a36af640 1 Processor -- start 2024-09-19T06:24:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.489+0000 7f35a36af640 1 -- start start 2024-09-19T06:24:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.489+0000 7f35a36af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c1009d0 0x7f359c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.489+0000 7f35a36af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c101370 0x7f359c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.489+0000 7f35a36af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f359c1069e0 0x7f359c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a36af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f359c110cd0 con 0x7f359c1009d0 2024-09-19T06:24:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a36af640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f359c110b50 con 0x7f359c101370 2024-09-19T06:24:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a36af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f359c110e50 con 0x7f359c1069e0 2024-09-19T06:24:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a0c4a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c101370 0x7f359c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a0c4a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c101370 0x7f359c198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56532/0 (socket says 172.21.15.5:56532) 2024-09-19T06:24:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a0c4a640 1 -- 172.21.15.5:0/958214279 learned_addr learned my addr 172.21.15.5:0/958214279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a144b640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c1009d0 0x7f359c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a1c4c640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f359c1069e0 0x7f359c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a0c4a640 1 -- 172.21.15.5:0/958214279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f359c1069e0 msgr2=0x7f359c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f359c1069e0 0x7f359c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a0c4a640 1 -- 172.21.15.5:0/958214279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c1009d0 msgr2=0x7f359c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c1009d0 0x7f359c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a0c4a640 1 -- 172.21.15.5:0/958214279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c19d320 con 0x7f359c101370 2024-09-19T06:24:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c101370 0x7f359c198870 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f358c00c890 tx=0x7f358c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.490+0000 7f35927fc640 1 -- 172.21.15.5:0/958214279 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c004220 con 0x7f359c101370 2024-09-19T06:24:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.491+0000 7f35a36af640 1 -- 172.21.15.5:0/958214279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c19d610 con 0x7f359c101370 2024-09-19T06:24:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.491+0000 7f35927fc640 1 -- 172.21.15.5:0/958214279 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f358c0043c0 con 0x7f359c101370 2024-09-19T06:24:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.491+0000 7f35927fc640 1 -- 172.21.15.5:0/958214279 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c004a00 con 0x7f359c101370 2024-09-19T06:24:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.491+0000 7f35a36af640 1 -- 172.21.15.5:0/958214279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c1a4ea0 con 0x7f359c101370 2024-09-19T06:24:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.494+0000 7f35927fc640 1 -- 172.21.15.5:0/958214279 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f358c004ba0 con 0x7f359c101370 2024-09-19T06:24:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.494+0000 7f35a643f740 1 Processor -- start 2024-09-19T06:24:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.494+0000 7f35a643f740 1 -- start start 2024-09-19T06:24:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.494+0000 7f35a643f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74b3af0 0x558dd74b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.494+0000 7f35a643f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 0x558dd74be770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.494+0000 7f35a643f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.494+0000 7f35a643f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558dd74182f0 con 0x558dd74b3af0 2024-09-19T06:24:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.494+0000 7f35a643f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558dd73c0270 con 0x558dd74b4410 2024-09-19T06:24:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.494+0000 7f35a643f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558dd74142b0 con 0x558dd74becb0 2024-09-19T06:24:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.495+0000 7f35a144b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 0x558dd74be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.495+0000 7f35a1c4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74b3af0 0x558dd74b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.495+0000 7f35a144b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 0x558dd74be770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56536/0 (socket says 172.21.15.5:56536) 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.495+0000 7f35a144b640 1 -- 172.21.15.5:0/2604445787 learned_addr learned my addr 172.21.15.5:0/2604445787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.495+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/2604445787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.495+0000 7f35a144b640 1 -- 172.21.15.5:0/2604445787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 msgr2=0x558dd74c10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.495+0000 7f35a144b640 1 --2- 172.21.15.5:0/2604445787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74c10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.495+0000 7f35a144b640 1 -- 172.21.15.5:0/2604445787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74b3af0 msgr2=0x558dd74b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.495+0000 7f35a144b640 1 --2- 172.21.15.5:0/2604445787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74b3af0 0x558dd74b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.495+0000 7f35a144b640 1 -- 172.21.15.5:0/2604445787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dd73bf340 con 0x558dd74b4410 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.495+0000 7f35a144b640 1 --2- 172.21.15.5:0/2604445787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 0x558dd74be770 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3598009860 tx=0x7f3598009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e418846e7ca1e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.496+0000 7f35877fe640 1 -- 172.21.15.5:0/2604445787 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598043020 con 0x558dd74b4410 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.496+0000 7f35877fe640 1 -- 172.21.15.5:0/2604445787 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3598038040 con 0x558dd74b4410 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.496+0000 7f35877fe640 1 -- 172.21.15.5:0/2604445787 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598037040 con 0x558dd74b4410 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.496+0000 7f35927fc640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35880a34b0 0x7f35880a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.496+0000 7f35a1c4c640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35880a34b0 0x7f35880a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.496+0000 7f35a643f740 1 -- 172.21.15.5:0/2604445787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 msgr2=0x558dd74be770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.496+0000 7f35a643f740 1 --2- 172.21.15.5:0/2604445787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 0x558dd74be770 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3598009860 tx=0x7f3598009490 comp rx=0 tx=0).stop 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.496+0000 7f35a643f740 1 -- 172.21.15.5:0/2604445787 shutdown_connections 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.496+0000 7f35a643f740 1 --2- 172.21.15.5:0/2604445787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.496+0000 7f35a643f740 1 --2- 172.21.15.5:0/2604445787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 0x558dd74be770 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.496+0000 7f35a643f740 1 --2- 172.21.15.5:0/2604445787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74b3af0 0x558dd74b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.496+0000 7f35a643f740 1 -- 172.21.15.5:0/2604445787 >> 172.21.15.5:0/2604445787 conn(0x558dd74b0060 msgr2=0x558dd74b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.497+0000 7f35927fc640 1 -- 172.21.15.5:0/958214279 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f358c0cf9e0 con 0x7f359c101370 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.497+0000 7f35a643f740 1 -- 172.21.15.5:0/2604445787 shutdown_connections 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.497+0000 7f35a643f740 1 -- 172.21.15.5:0/2604445787 wait complete. 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.497+0000 7f35a643f740 1 -- start start 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.497+0000 7f35a1c4c640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35880a34b0 0x7f35880a5970 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3594005fd0 tx=0x7f35940095c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.497+0000 7f35a643f740 1 Processor -- start 2024-09-19T06:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a643f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74b3af0 0x558dd74bbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a643f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 0x558dd74bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a643f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74becb0 0x558dd74bc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a643f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558dd74c1880 con 0x558dd74becb0 2024-09-19T06:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a643f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558dd73a1290 con 0x558dd74b4410 2024-09-19T06:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a643f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558dd74c1a00 con 0x558dd74b3af0 2024-09-19T06:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a643f740 10 client.0 _ll_register_callbacks cb 0x558dd73d2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a1c4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74becb0 0x558dd74bc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a0c4a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 0x558dd74bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a1c4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74becb0 0x558dd74bc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55314/0 (socket says 172.21.15.5:55314) 2024-09-19T06:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a0c4a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 0x558dd74bc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56546/0 (socket says 172.21.15.5:56546) 2024-09-19T06:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a1c4c640 1 -- 172.21.15.5:0/3777028426 learned_addr learned my addr 172.21.15.5:0/3777028426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a144b640 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74b3af0 0x558dd74bbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a0c4a640 1 -- 172.21.15.5:0/3777028426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74b3af0 msgr2=0x558dd74bbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74b3af0 0x558dd74bbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a0c4a640 1 -- 172.21.15.5:0/3777028426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74becb0 msgr2=0x558dd74bc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74becb0 0x558dd74bc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a0c4a640 1 -- 172.21.15.5:0/3777028426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dd712b210 con 0x558dd74b4410 2024-09-19T06:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.498+0000 7f35a144b640 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74b3af0 0x558dd74bbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.499+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 0x558dd74bc0a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f358c0cd0b0 tx=0x7f358c0057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346c531d95d8d819 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.499+0000 7f3584ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.499+0000 7f3584ff9640 1 -- 172.21.15.5:0/3777028426 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c01d040 con 0x558dd74b4410 2024-09-19T06:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.499+0000 7f3584ff9640 1 -- 172.21.15.5:0/3777028426 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f358c0982b0 con 0x558dd74b4410 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.499+0000 7f35a643f740 1 -- 172.21.15.5:0/3777028426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3598038040 con 0x558dd74b4410 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.499+0000 7f3584ff9640 1 -- 172.21.15.5:0/3777028426 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f358c09b780 con 0x558dd74b4410 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.499+0000 7f35a643f740 10 client.30074 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.499+0000 7f35a643f740 1 -- 172.21.15.5:0/3777028426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x558dd71f7620 con 0x558dd74b4410 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.499+0000 7f3584ff9640 1 -- 172.21.15.5:0/3777028426 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3598038040 con 0x558dd74b4410 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.500+0000 7f3584ff9640 1 -- 172.21.15.5:0/3777028426 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f358c097990 con 0x558dd74b4410 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.500+0000 7f35a643f740 10 client.30074 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.500+0000 7f35a643f740 10 client.30074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.500+0000 7f35a643f740 20 client.30074 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.500+0000 7f35a643f740 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd74d7820 0x558dd74d9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.500+0000 7f35a643f740 4 client.30074 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.500+0000 7f35a643f740 1 -- 172.21.15.5:0/3777028426 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dd7425a20 con 0x558dd74d7820 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.500+0000 7f35a144b640 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd74d7820 0x558dd74d9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.501+0000 7f35a144b640 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd74d7820 0x558dd74d9ce0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63246e124cdcb524 server_cookie=7198fca89f84fef9 in_seq=0 out_seq=0 2024-09-19T06:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.501+0000 7f3584ff9640 10 client.30074 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f3584ff9640 1 -- 172.21.15.5:0/3777028426 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558dd7425a20 con 0x558dd74d7820 2024-09-19T06:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f3584ff9640 10 client.30074 handle_command_reply: tid=0 2024-09-19T06:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f35a643f740 1 client.30074 shutdown 2024-09-19T06:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f35a643f740 1 -- 172.21.15.5:0/3777028426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 msgr2=0x558dd74bc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f35a643f740 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 0x558dd74bc0a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f358c0cd0b0 tx=0x7f358c0057e0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f35a643f740 1 -- 172.21.15.5:0/3777028426 shutdown_connections 2024-09-19T06:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f35a643f740 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd74d7820 0x558dd74d9ce0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f35a643f740 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74becb0 0x558dd74bc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f35a643f740 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b4410 0x558dd74bc0a0 secure :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f358c0cd0b0 tx=0x7f358c0057e0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f35a643f740 1 --2- 172.21.15.5:0/3777028426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74b3af0 0x558dd74bbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f35a643f740 1 -- 172.21.15.5:0/3777028426 >> 172.21.15.5:0/3777028426 conn(0x558dd74b0060 msgr2=0x558dd74b5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f3584ff9640 0 client.30074 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.502+0000 7f35a643f740 1 -- 172.21.15.5:0/3777028426 shutdown_connections 2024-09-19T06:24:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.503+0000 7f35a643f740 1 -- 172.21.15.5:0/3777028426 wait complete. 2024-09-19T06:24:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.503+0000 7f35a643f740 20 client.30074 trim_cache size 0 max 16384 2024-09-19T06:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.529+0000 7f35a643f740 1 Processor -- start 2024-09-19T06:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.529+0000 7f35a643f740 1 -- start start 2024-09-19T06:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.529+0000 7f35a643f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74b3af0 0x558dd74b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.529+0000 7f35a643f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.529+0000 7f35a643f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74dfb30 0x558dd74bd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.529+0000 7f35a643f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1ab660 con 0x558dd74b3af0 2024-09-19T06:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.529+0000 7f35a0c4a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.529+0000 7f35a0c4a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74df5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41794/0 (socket says 172.21.15.5:41794) 2024-09-19T06:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.529+0000 7f35a0c4a640 1 -- 172.21.15.5:0/2507915621 learned_addr learned my addr 172.21.15.5:0/2507915621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.529+0000 7f35a643f740 1 -- 172.21.15.5:0/2507915621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1ad200 con 0x558dd74dfb30 2024-09-19T06:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.529+0000 7f35a643f740 1 -- 172.21.15.5:0/2507915621 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1abd30 con 0x558dd74becb0 2024-09-19T06:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.529+0000 7f35a1c4c640 1 --2- 172.21.15.5:0/2507915621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74dfb30 0x558dd74bd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.530+0000 7f35a0c4a640 1 -- 172.21.15.5:0/2507915621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74dfb30 msgr2=0x558dd74bd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.530+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/2507915621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74dfb30 0x558dd74bd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.530+0000 7f35a0c4a640 1 -- 172.21.15.5:0/2507915621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74b3af0 msgr2=0x558dd74b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.530+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/2507915621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74b3af0 0x558dd74b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.530+0000 7f35a0c4a640 1 -- 172.21.15.5:0/2507915621 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dd74b4240 con 0x558dd74becb0 2024-09-19T06:24:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.530+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/2507915621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74df5f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f358c0cd0e0 tx=0x7f358c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c79a69aab6b82804 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.530+0000 7f3586ffd640 1 -- 172.21.15.5:0/2507915621 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c09b5a0 con 0x558dd74becb0 2024-09-19T06:24:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.530+0000 7f3586ffd640 1 -- 172.21.15.5:0/2507915621 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f358c09c340 con 0x558dd74becb0 2024-09-19T06:24:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.530+0000 7f3586ffd640 1 -- 172.21.15.5:0/2507915621 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c0982b0 con 0x558dd74becb0 2024-09-19T06:24:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.531+0000 7f35a643f740 1 -- 172.21.15.5:0/2507915621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 msgr2=0x558dd74df5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.531+0000 7f35a643f740 1 --2- 172.21.15.5:0/2507915621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74df5f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f358c0cd0e0 tx=0x7f358c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.531+0000 7f35a643f740 1 -- 172.21.15.5:0/2507915621 shutdown_connections 2024-09-19T06:24:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.531+0000 7f35a643f740 1 --2- 172.21.15.5:0/2507915621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74dfb30 0x558dd74bd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.531+0000 7f35a643f740 1 --2- 172.21.15.5:0/2507915621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74df5f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.531+0000 7f35a643f740 1 --2- 172.21.15.5:0/2507915621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74b3af0 0x558dd74b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.531+0000 7f35a643f740 1 -- 172.21.15.5:0/2507915621 >> 172.21.15.5:0/2507915621 conn(0x558dd74b0060 msgr2=0x558dd74b6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.531+0000 7f35a643f740 1 -- 172.21.15.5:0/2507915621 shutdown_connections 2024-09-19T06:24:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.531+0000 7f35a643f740 1 -- 172.21.15.5:0/2507915621 wait complete. 2024-09-19T06:24:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.532+0000 7f35a643f740 1 -- start start 2024-09-19T06:24:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.532+0000 7f35a643f740 1 Processor -- start 2024-09-19T06:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a643f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b3af0 0x558dd74cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a643f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a643f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74dfb30 0x558dd74cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a643f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1acb30 con 0x558dd74dfb30 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a144b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b3af0 0x558dd74cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a144b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b3af0 0x558dd74cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56550/0 (socket says 172.21.15.5:56550) 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a643f740 1 -- 172.21.15.5:0/713933513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1ac430 con 0x558dd74b3af0 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a144b640 1 -- 172.21.15.5:0/713933513 learned_addr learned my addr 172.21.15.5:0/713933513 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a643f740 1 -- 172.21.15.5:0/713933513 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1a9680 con 0x558dd74becb0 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a643f740 10 client.0 _ll_register_callbacks cb 0x7f358c005380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a1c4c640 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74dfb30 0x558dd74cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a0c4a640 1 -- 172.21.15.5:0/713933513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b3af0 msgr2=0x558dd74cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b3af0 0x558dd74cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a0c4a640 1 -- 172.21.15.5:0/713933513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74dfb30 msgr2=0x558dd74cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74dfb30 0x558dd74cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a0c4a640 1 -- 172.21.15.5:0/713933513 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dd74cb840 con 0x558dd74becb0 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.533+0000 7f35a1c4c640 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74dfb30 0x558dd74cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.534+0000 7f35a0c4a640 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74cad40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f358c0980a0 tx=0x7f358c09b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6737ba1f5a5eb19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.534+0000 7f3584ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.534+0000 7f3584ff9640 1 -- 172.21.15.5:0/713933513 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c01d030 con 0x558dd74becb0 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.534+0000 7f3584ff9640 1 -- 172.21.15.5:0/713933513 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f358c0862d0 con 0x558dd74becb0 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.534+0000 7f35a643f740 1 -- 172.21.15.5:0/713933513 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f358c09c340 con 0x558dd74becb0 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.534+0000 7f3584ff9640 1 -- 172.21.15.5:0/713933513 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f358c09b5a0 con 0x558dd74becb0 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.534+0000 7f35a643f740 10 client.30001 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.534+0000 7f35a643f740 1 -- 172.21.15.5:0/713933513 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x558dd7542380 con 0x558dd74becb0 2024-09-19T06:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.535+0000 7f3584ff9640 1 -- 172.21.15.5:0/713933513 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f358c09c340 con 0x558dd74becb0 2024-09-19T06:24:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.535+0000 7f3584ff9640 1 -- 172.21.15.5:0/713933513 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f358c09ab00 con 0x558dd74becb0 2024-09-19T06:24:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.535+0000 7f35a643f740 10 client.30001 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.535+0000 7f35a643f740 10 client.30001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.535+0000 7f35a643f740 20 client.30001 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.535+0000 7f35a643f740 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd75427c0 0x558dd7544be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.535+0000 7f35a643f740 4 client.30001 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:24:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.535+0000 7f35a643f740 1 -- 172.21.15.5:0/713933513 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558dd741fd70 con 0x558dd75427c0 2024-09-19T06:24:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.535+0000 7f35a144b640 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd75427c0 0x558dd7544be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.535+0000 7f3584ff9640 10 client.30001 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.536+0000 7f35a144b640 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd75427c0 0x558dd7544be0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a263bc3d7a2a960a server_cookie=6883d09029d5ad13 in_seq=0 out_seq=0 2024-09-19T06:24:26.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 76501 8.63 MB/sec execute 350 sec latency 311.004 ms 2024-09-19T06:24:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.649+0000 7f3584ff9640 1 -- 172.21.15.5:0/713933513 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558dd741fd70 con 0x558dd75427c0 2024-09-19T06:24:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.649+0000 7f3584ff9640 10 client.30001 handle_command_reply: tid=0 2024-09-19T06:24:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.649+0000 7f35a643f740 1 client.30001 shutdown 2024-09-19T06:24:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.649+0000 7f35a643f740 1 -- 172.21.15.5:0/713933513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 msgr2=0x558dd74cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.649+0000 7f35a643f740 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74cad40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f358c0980a0 tx=0x7f358c09b2c0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.650+0000 7f35a643f740 1 -- 172.21.15.5:0/713933513 shutdown_connections 2024-09-19T06:24:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.650+0000 7f35a643f740 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558dd75427c0 0x558dd7544be0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.650+0000 7f35a643f740 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dd74dfb30 0x558dd74cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.650+0000 7f35a643f740 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dd74becb0 0x558dd74cad40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.650+0000 7f35a643f740 1 --2- 172.21.15.5:0/713933513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dd74b3af0 0x558dd74cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.650+0000 7f35a643f740 1 -- 172.21.15.5:0/713933513 >> 172.21.15.5:0/713933513 conn(0x558dd74b0060 msgr2=0x558dd74bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.650+0000 7f3584ff9640 0 client.30001 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.650+0000 7f35a643f740 1 -- 172.21.15.5:0/713933513 shutdown_connections 2024-09-19T06:24:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.650+0000 7f35a643f740 1 -- 172.21.15.5:0/713933513 wait complete. 2024-09-19T06:24:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.650+0000 7f35a643f740 20 client.30001 trim_cache size 0 max 16384 2024-09-19T06:24:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:24:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:24:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:24:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:24:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.651+0000 7f35a36af640 1 -- 172.21.15.5:0/958214279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35880a34b0 msgr2=0x7f35880a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.651+0000 7f35a36af640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35880a34b0 0x7f35880a5970 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3594005fd0 tx=0x7f35940095c0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.651+0000 7f35a36af640 1 -- 172.21.15.5:0/958214279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c101370 msgr2=0x7f359c198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.651+0000 7f35a36af640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c101370 0x7f359c198870 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f358c00c890 tx=0x7f358c00cd50 comp rx=0 tx=0).stop 2024-09-19T06:24:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.651+0000 7f35a36af640 1 -- 172.21.15.5:0/958214279 shutdown_connections 2024-09-19T06:24:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.651+0000 7f35a36af640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f35880a34b0 0x7f35880a5970 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.651+0000 7f35a36af640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f359c1069e0 0x7f359c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.651+0000 7f35a36af640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c101370 0x7f359c198870 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.651+0000 7f35a36af640 1 --2- 172.21.15.5:0/958214279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c1009d0 0x7f359c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.651+0000 7f35a36af640 1 -- 172.21.15.5:0/958214279 >> 172.21.15.5:0/958214279 conn(0x7f359c0fc690 msgr2=0x7f359c0fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.651+0000 7f35a36af640 1 -- 172.21.15.5:0/958214279 shutdown_connections 2024-09-19T06:24:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:26.651+0000 7f35a36af640 1 -- 172.21.15.5:0/958214279 wait complete. 2024-09-19T06:24:26.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:26 smithi005 ceph-mon[30955]: pgmap v666: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 248 op/s 2024-09-19T06:24:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:26 smithi049 ceph-mon[34640]: pgmap v666: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 248 op/s 2024-09-19T06:24:27.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:26 smithi157 ceph-mon[35346]: pgmap v666: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 248 op/s 2024-09-19T06:24:27.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 76700 8.64 MB/sec execute 351 sec latency 119.474 ms 2024-09-19T06:24:28.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:24:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:24:28.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 76851 8.64 MB/sec execute 352 sec latency 57.468 ms 2024-09-19T06:24:28.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:28 smithi005 ceph-mon[30955]: pgmap v667: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.9 MiB/s wr, 377 op/s 2024-09-19T06:24:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:28 smithi049 ceph-mon[34640]: pgmap v667: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.9 MiB/s wr, 377 op/s 2024-09-19T06:24:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:28 smithi157 ceph-mon[35346]: pgmap v667: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.9 MiB/s wr, 377 op/s 2024-09-19T06:24:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.150+0000 7f4997ff7640 1 -- 172.21.15.5:0/3268246827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4990102da0 msgr2=0x7f4990103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.150+0000 7f4997ff7640 1 --2- 172.21.15.5:0/3268246827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4990102da0 0x7f4990103180 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f498c009900 tx=0x7f498c02a480 comp rx=0 tx=0).stop 2024-09-19T06:24:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.150+0000 7f4997ff7640 1 -- 172.21.15.5:0/3268246827 shutdown_connections 2024-09-19T06:24:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.150+0000 7f4997ff7640 1 --2- 172.21.15.5:0/3268246827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499010c180 0x7f499010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.150+0000 7f4997ff7640 1 --2- 172.21.15.5:0/3268246827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49901036c0 0x7f499010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.150+0000 7f4997ff7640 1 --2- 172.21.15.5:0/3268246827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4990102da0 0x7f4990103180 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.150+0000 7f4997ff7640 1 -- 172.21.15.5:0/3268246827 >> 172.21.15.5:0/3268246827 conn(0x7f49900fc7a0 msgr2=0x7f49900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.151+0000 7f4997ff7640 1 -- 172.21.15.5:0/3268246827 shutdown_connections 2024-09-19T06:24:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.151+0000 7f4997ff7640 1 -- 172.21.15.5:0/3268246827 wait complete. 2024-09-19T06:24:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.152+0000 7f4997ff7640 1 Processor -- start 2024-09-19T06:24:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.152+0000 7f4997ff7640 1 -- start start 2024-09-19T06:24:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.152+0000 7f4997ff7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4990102da0 0x7f499019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.152+0000 7f4997ff7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49901036c0 0x7f499019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.152+0000 7f4997ff7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f499010c180 0x7f499019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.152+0000 7f4997ff7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4990111e60 con 0x7f49901036c0 2024-09-19T06:24:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.152+0000 7f4997ff7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4990111ce0 con 0x7f4990102da0 2024-09-19T06:24:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4997ff7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4990111fe0 con 0x7f499010c180 2024-09-19T06:24:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4995d93640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4990102da0 0x7f499019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4996594640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f499010c180 0x7f499019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4995592640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49901036c0 0x7f499019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4996594640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f499010c180 0x7f499019ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41824/0 (socket says 172.21.15.5:41824) 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4996594640 1 -- 172.21.15.5:0/3138362792 learned_addr learned my addr 172.21.15.5:0/3138362792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4996594640 1 -- 172.21.15.5:0/3138362792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4990102da0 msgr2=0x7f499019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4996594640 1 --2- 172.21.15.5:0/3138362792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4990102da0 0x7f499019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4996594640 1 -- 172.21.15.5:0/3138362792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49901036c0 msgr2=0x7f499019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4996594640 1 --2- 172.21.15.5:0/3138362792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49901036c0 0x7f499019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4996594640 1 -- 172.21.15.5:0/3138362792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499019f520 con 0x7f499010c180 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4996594640 1 --2- 172.21.15.5:0/3138362792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f499010c180 0x7f499019ee40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f498800eb10 tx=0x7f498800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4986ffd640 1 -- 172.21.15.5:0/3138362792 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4988004d20 con 0x7f499010c180 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.153+0000 7f4986ffd640 1 -- 172.21.15.5:0/3138362792 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4988005020 con 0x7f499010c180 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.154+0000 7f4986ffd640 1 -- 172.21.15.5:0/3138362792 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498800ce60 con 0x7f499010c180 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.154+0000 7f4997ff7640 1 -- 172.21.15.5:0/3138362792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499019f810 con 0x7f499010c180 2024-09-19T06:24:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.154+0000 7f4997ff7640 1 -- 172.21.15.5:0/3138362792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4990103b00 con 0x7f499010c180 2024-09-19T06:24:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.157+0000 7f4986ffd640 1 -- 172.21.15.5:0/3138362792 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f49880051d0 con 0x7f499010c180 2024-09-19T06:24:29.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.158+0000 7f499a3d0740 1 Processor -- start 2024-09-19T06:24:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.158+0000 7f499a3d0740 1 -- start start 2024-09-19T06:24:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.158+0000 7f4986ffd640 1 --2- 172.21.15.5:0/3138362792 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f497c0a33b0 0x7f497c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.158+0000 7f499a3d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b08834b40 0x563b08833090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.158+0000 7f499a3d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b088335d0 0x563b08833a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.158+0000 7f499a3d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b0883e370 0x563b08840760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.158+0000 7f499a3d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563b087992f0 con 0x563b0883e370 2024-09-19T06:24:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.158+0000 7f499a3d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563b08740cf0 con 0x563b08834b40 2024-09-19T06:24:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.158+0000 7f499a3d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563b087952b0 con 0x563b088335d0 2024-09-19T06:24:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.159+0000 7f4986ffd640 1 -- 172.21.15.5:0/3138362792 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f49880d8580 con 0x7f499010c180 2024-09-19T06:24:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.159+0000 7f4995d93640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b08834b40 0x563b08833090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.159+0000 7f4995592640 1 --2- 172.21.15.5:0/3138362792 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f497c0a33b0 0x7f497c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.159+0000 7f4995d93640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b08834b40 0x563b08833090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56568/0 (socket says 172.21.15.5:56568) 2024-09-19T06:24:29.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.159+0000 7f4995d93640 1 -- 172.21.15.5:0/2904152315 learned_addr learned my addr 172.21.15.5:0/2904152315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:29.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.159+0000 7f4996594640 1 --2- 172.21.15.5:0/2904152315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b0883e370 0x563b08840760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.159+0000 7f4995592640 1 --2- 172.21.15.5:0/2904152315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b088335d0 0x563b08833a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.159+0000 7f4995592640 1 -- 172.21.15.5:0/2904152315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b08834b40 msgr2=0x563b08833090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:29.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.160+0000 7f4995592640 1 --2- 172.21.15.5:0/2904152315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b08834b40 0x563b08833090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.160+0000 7f4995592640 1 -- 172.21.15.5:0/2904152315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b0883e370 msgr2=0x563b08840760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:29.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.160+0000 7f4995592640 1 --2- 172.21.15.5:0/2904152315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b0883e370 0x563b08840760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.160+0000 7f4995592640 1 -- 172.21.15.5:0/2904152315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b087253a0 con 0x563b088335d0 2024-09-19T06:24:29.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.160+0000 7f4995592640 1 --2- 172.21.15.5:0/3138362792 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f497c0a33b0 0x7f497c0a5870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f49900fec70 tx=0x7f49800098b0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:29.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.160+0000 7f4995592640 1 --2- 172.21.15.5:0/2904152315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b088335d0 0x563b08833a10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4980013dd0 tx=0x7f4980020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b10a779b897e34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:29.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.160+0000 7f497bfff640 1 -- 172.21.15.5:0/2904152315 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980013120 con 0x563b088335d0 2024-09-19T06:24:29.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.160+0000 7f497bfff640 1 -- 172.21.15.5:0/2904152315 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4980024030 con 0x563b088335d0 2024-09-19T06:24:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.160+0000 7f497bfff640 1 -- 172.21.15.5:0/2904152315 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980010aa0 con 0x563b088335d0 2024-09-19T06:24:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.161+0000 7f499a3d0740 1 -- 172.21.15.5:0/2904152315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b088335d0 msgr2=0x563b08833a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.161+0000 7f499a3d0740 1 --2- 172.21.15.5:0/2904152315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b088335d0 0x563b08833a10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4980013dd0 tx=0x7f4980020000 comp rx=0 tx=0).stop 2024-09-19T06:24:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.161+0000 7f499a3d0740 1 -- 172.21.15.5:0/2904152315 shutdown_connections 2024-09-19T06:24:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.161+0000 7f499a3d0740 1 --2- 172.21.15.5:0/2904152315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b0883e370 0x563b08840760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.161+0000 7f499a3d0740 1 --2- 172.21.15.5:0/2904152315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b088335d0 0x563b08833a10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.161+0000 7f499a3d0740 1 --2- 172.21.15.5:0/2904152315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b08834b40 0x563b08833090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.161+0000 7f499a3d0740 1 -- 172.21.15.5:0/2904152315 >> 172.21.15.5:0/2904152315 conn(0x563b087fc310 msgr2=0x563b0882fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.161+0000 7f499a3d0740 1 -- 172.21.15.5:0/2904152315 shutdown_connections 2024-09-19T06:24:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.161+0000 7f499a3d0740 1 -- 172.21.15.5:0/2904152315 wait complete. 2024-09-19T06:24:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.162+0000 7f499a3d0740 1 -- start start 2024-09-19T06:24:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.162+0000 7f499a3d0740 1 Processor -- start 2024-09-19T06:24:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.162+0000 7f499a3d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b088335d0 0x563b0883dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.162+0000 7f499a3d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b08834b40 0x563b0883b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f499a3d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b0883e370 0x563b0883b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f499a3d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563b08841010 con 0x563b08834b40 2024-09-19T06:24:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f499a3d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563b08754060 con 0x563b0883e370 2024-09-19T06:24:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f499a3d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563b08841190 con 0x563b088335d0 2024-09-19T06:24:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f499a3d0740 10 client.0 _ll_register_callbacks cb 0x563b084a78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f4996594640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b0883e370 0x563b0883b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f4995592640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b08834b40 0x563b0883b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f4996594640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b0883e370 0x563b0883b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56570/0 (socket says 172.21.15.5:56570) 2024-09-19T06:24:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f4996594640 1 -- 172.21.15.5:0/788916955 learned_addr learned my addr 172.21.15.5:0/788916955 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f4995d93640 1 --2- 172.21.15.5:0/788916955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b088335d0 0x563b0883dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f4996594640 1 -- 172.21.15.5:0/788916955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b088335d0 msgr2=0x563b0883dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f4996594640 1 --2- 172.21.15.5:0/788916955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b088335d0 0x563b0883dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f4996594640 1 -- 172.21.15.5:0/788916955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b08834b40 msgr2=0x563b0883b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f4996594640 1 --2- 172.21.15.5:0/788916955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b08834b40 0x563b0883b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f4996594640 1 -- 172.21.15.5:0/788916955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b0850a3c0 con 0x563b0883e370 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f4996594640 1 --2- 172.21.15.5:0/788916955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b0883e370 0x563b0883b7e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4988005ba0 tx=0x7f49880a4420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77afbda406d9806b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.163+0000 7f49797fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f49797fa640 1 -- 172.21.15.5:0/788916955 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498802c380 con 0x563b0883e370 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f49797fa640 1 -- 172.21.15.5:0/788916955 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f49880d5f30 con 0x563b0883e370 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f499a3d0740 1 -- 172.21.15.5:0/788916955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4980024030 con 0x563b0883e370 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f49797fa640 1 -- 172.21.15.5:0/788916955 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f49880d6130 con 0x563b0883e370 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f499a3d0740 10 client.30092 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f499a3d0740 1 -- 172.21.15.5:0/788916955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x563b0884cb40 con 0x563b0883e370 2024-09-19T06:24:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f49797fa640 1 -- 172.21.15.5:0/788916955 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4980024030 con 0x563b0883e370 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f49797fa640 1 -- 172.21.15.5:0/788916955 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f49880ad130 con 0x563b0883e370 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f499a3d0740 10 client.30092 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f499a3d0740 10 client.30092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f499a3d0740 20 client.30092 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f499a3d0740 1 --2- 172.21.15.5:0/788916955 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563b0884e0f0 0x563b08850500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f499a3d0740 4 client.30092 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.164+0000 7f499a3d0740 1 -- 172.21.15.5:0/788916955 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b087a6860 con 0x563b0884e0f0 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.165+0000 7f4995d93640 1 --2- 172.21.15.5:0/788916955 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563b0884e0f0 0x563b08850500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.165+0000 7f4995d93640 1 --2- 172.21.15.5:0/788916955 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563b0884e0f0 0x563b08850500 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87db3fba773cbdf server_cookie=7f90902fd99408f8 in_seq=0 out_seq=0 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.165+0000 7f49797fa640 10 client.30092 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.166+0000 7f49797fa640 1 -- 172.21.15.5:0/788916955 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b087a6860 con 0x563b0884e0f0 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.166+0000 7f49797fa640 10 client.30092 handle_command_reply: tid=0 2024-09-19T06:24:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.166+0000 7f499a3d0740 1 client.30092 shutdown 2024-09-19T06:24:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.166+0000 7f499a3d0740 1 -- 172.21.15.5:0/788916955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b0883e370 msgr2=0x563b0883b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.166+0000 7f499a3d0740 1 --2- 172.21.15.5:0/788916955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b0883e370 0x563b0883b7e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4988005ba0 tx=0x7f49880a4420 comp rx=0 tx=0).stop 2024-09-19T06:24:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.167+0000 7f499a3d0740 1 -- 172.21.15.5:0/788916955 shutdown_connections 2024-09-19T06:24:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.167+0000 7f499a3d0740 1 --2- 172.21.15.5:0/788916955 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563b0884e0f0 0x563b08850500 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.167+0000 7f499a3d0740 1 --2- 172.21.15.5:0/788916955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b0883e370 0x563b0883b7e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.167+0000 7f499a3d0740 1 --2- 172.21.15.5:0/788916955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b08834b40 0x563b0883b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.167+0000 7f499a3d0740 1 --2- 172.21.15.5:0/788916955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b088335d0 0x563b0883dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.167+0000 7f499a3d0740 1 -- 172.21.15.5:0/788916955 >> 172.21.15.5:0/788916955 conn(0x563b087fc310 msgr2=0x563b08835960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.167+0000 7f49797fa640 0 client.30092 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.167+0000 7f499a3d0740 1 -- 172.21.15.5:0/788916955 shutdown_connections 2024-09-19T06:24:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.167+0000 7f499a3d0740 1 -- 172.21.15.5:0/788916955 wait complete. 2024-09-19T06:24:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.167+0000 7f499a3d0740 20 client.30092 trim_cache size 0 max 16384 2024-09-19T06:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f499a3d0740 1 Processor -- start 2024-09-19T06:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f499a3d0740 1 -- start start 2024-09-19T06:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f499a3d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b088335d0 0x563b0882fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f499a3d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b08834b40 0x563b08836120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f499a3d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b0883e370 0x563b08836660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f499a3d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49901ab5e0 con 0x563b0883e370 2024-09-19T06:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f499a3d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49901ad180 con 0x563b088335d0 2024-09-19T06:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f499a3d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f49901abcb0 con 0x563b08834b40 2024-09-19T06:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f4995d93640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b088335d0 0x563b0882fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f4996594640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b0883e370 0x563b08836660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f4995d93640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b088335d0 0x563b0882fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56576/0 (socket says 172.21.15.5:56576) 2024-09-19T06:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f4995d93640 1 -- 172.21.15.5:0/3600195128 learned_addr learned my addr 172.21.15.5:0/3600195128 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.188+0000 7f4995592640 1 --2- 172.21.15.5:0/3600195128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b08834b40 0x563b08836120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.189+0000 7f4995d93640 1 -- 172.21.15.5:0/3600195128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b08834b40 msgr2=0x563b08836120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.189+0000 7f4995d93640 1 --2- 172.21.15.5:0/3600195128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b08834b40 0x563b08836120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.189+0000 7f4995d93640 1 -- 172.21.15.5:0/3600195128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b0883e370 msgr2=0x563b08836660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.189+0000 7f4995d93640 1 --2- 172.21.15.5:0/3600195128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b0883e370 0x563b08836660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.189+0000 7f4995d93640 1 -- 172.21.15.5:0/3600195128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b0884a300 con 0x563b088335d0 2024-09-19T06:24:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.189+0000 7f4995592640 1 --2- 172.21.15.5:0/3600195128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b08834b40 0x563b08836120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.189+0000 7f4995d93640 1 --2- 172.21.15.5:0/3600195128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b088335d0 0x563b0882fe10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f498c009230 tx=0x7f498c005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5efb6256182f137 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.189+0000 7f497b7fe640 1 -- 172.21.15.5:0/3600195128 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c035040 con 0x563b088335d0 2024-09-19T06:24:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.189+0000 7f497b7fe640 1 -- 172.21.15.5:0/3600195128 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f498c04b070 con 0x563b088335d0 2024-09-19T06:24:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.189+0000 7f497b7fe640 1 -- 172.21.15.5:0/3600195128 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c0035f0 con 0x563b088335d0 2024-09-19T06:24:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.190+0000 7f499a3d0740 1 -- 172.21.15.5:0/3600195128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b088335d0 msgr2=0x563b0882fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.190+0000 7f499a3d0740 1 --2- 172.21.15.5:0/3600195128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b088335d0 0x563b0882fe10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f498c009230 tx=0x7f498c005a20 comp rx=0 tx=0).stop 2024-09-19T06:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.190+0000 7f499a3d0740 1 -- 172.21.15.5:0/3600195128 shutdown_connections 2024-09-19T06:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.190+0000 7f499a3d0740 1 --2- 172.21.15.5:0/3600195128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b0883e370 0x563b08836660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.190+0000 7f499a3d0740 1 --2- 172.21.15.5:0/3600195128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b08834b40 0x563b08836120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.190+0000 7f499a3d0740 1 --2- 172.21.15.5:0/3600195128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b088335d0 0x563b0882fe10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.190+0000 7f499a3d0740 1 -- 172.21.15.5:0/3600195128 >> 172.21.15.5:0/3600195128 conn(0x563b087fc310 msgr2=0x563b0882f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.190+0000 7f499a3d0740 1 -- 172.21.15.5:0/3600195128 shutdown_connections 2024-09-19T06:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.190+0000 7f499a3d0740 1 -- 172.21.15.5:0/3600195128 wait complete. 2024-09-19T06:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.191+0000 7f499a3d0740 1 -- start start 2024-09-19T06:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.191+0000 7f499a3d0740 1 Processor -- start 2024-09-19T06:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f499a3d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b088335d0 0x563b088466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f499a3d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b08834b40 0x563b08846c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f499a3d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b0883e370 0x563b08837920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f4995d93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b088335d0 0x563b088466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f4995d93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b088335d0 0x563b088466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55394/0 (socket says 172.21.15.5:55394) 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f4995d93640 1 -- 172.21.15.5:0/3455999233 learned_addr learned my addr 172.21.15.5:0/3455999233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f499a3d0740 1 -- 172.21.15.5:0/3455999233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49901acab0 con 0x563b088335d0 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f499a3d0740 1 -- 172.21.15.5:0/3455999233 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49901ac3b0 con 0x563b08834b40 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f499a3d0740 1 -- 172.21.15.5:0/3455999233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f49901a9620 con 0x563b0883e370 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f4995592640 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b08834b40 0x563b08846c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f499a3d0740 10 client.0 _ll_register_callbacks cb 0x7f4988004f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f4996594640 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b0883e370 0x563b08837920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f4995d93640 1 -- 172.21.15.5:0/3455999233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b0883e370 msgr2=0x563b08837920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f4995d93640 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b0883e370 0x563b08837920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f4995d93640 1 -- 172.21.15.5:0/3455999233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b08834b40 msgr2=0x563b08846c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f4995d93640 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b08834b40 0x563b08846c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f4995d93640 1 -- 172.21.15.5:0/3455999233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b08838150 con 0x563b088335d0 2024-09-19T06:24:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.192+0000 7f4996594640 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b0883e370 0x563b08837920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.193+0000 7f4995592640 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b08834b40 0x563b08846c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.193+0000 7f4995d93640 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b088335d0 0x563b088466e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f498c005ab0 tx=0x7f498c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb91e34e7f92f9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.193+0000 7f49797fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:24:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.193+0000 7f49797fa640 1 -- 172.21.15.5:0/3455999233 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c035050 con 0x563b088335d0 2024-09-19T06:24:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.193+0000 7f49797fa640 1 -- 172.21.15.5:0/3455999233 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f498c004970 con 0x563b088335d0 2024-09-19T06:24:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.193+0000 7f499a3d0740 1 -- 172.21.15.5:0/3455999233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498c04b070 con 0x563b088335d0 2024-09-19T06:24:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.193+0000 7f49797fa640 1 -- 172.21.15.5:0/3455999233 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f498c005040 con 0x563b088335d0 2024-09-19T06:24:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.193+0000 7f499a3d0740 10 client.20718 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.193+0000 7f499a3d0740 1 -- 172.21.15.5:0/3455999233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x563b088b88e0 con 0x563b088335d0 2024-09-19T06:24:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.194+0000 7f49797fa640 1 -- 172.21.15.5:0/3455999233 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f498c04b070 con 0x563b088335d0 2024-09-19T06:24:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.194+0000 7f49797fa640 1 -- 172.21.15.5:0/3455999233 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f498c0314a0 con 0x563b088335d0 2024-09-19T06:24:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.194+0000 7f499a3d0740 10 client.20718 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.194+0000 7f499a3d0740 10 client.20718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.194+0000 7f499a3d0740 20 client.20718 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.194+0000 7f499a3d0740 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563b088b9020 0x563b088bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.194+0000 7f499a3d0740 4 client.20718 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:24:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.194+0000 7f499a3d0740 1 -- 172.21.15.5:0/3455999233 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b087a0d70 con 0x563b088b9020 2024-09-19T06:24:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.194+0000 7f4995592640 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563b088b9020 0x563b088bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f49797fa640 10 client.20718 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f4995592640 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563b088b9020 0x563b088bb440 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74068a5362ae75e5 server_cookie=a6e921e08bbc64cb in_seq=0 out_seq=0 2024-09-19T06:24:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f49797fa640 1 -- 172.21.15.5:0/3455999233 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b087a0d70 con 0x563b088b9020 2024-09-19T06:24:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f49797fa640 10 client.20718 handle_command_reply: tid=0 2024-09-19T06:24:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f499a3d0740 1 client.20718 shutdown 2024-09-19T06:24:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f499a3d0740 1 -- 172.21.15.5:0/3455999233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b088335d0 msgr2=0x563b088466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f499a3d0740 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b088335d0 0x563b088466e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f498c005ab0 tx=0x7f498c02a480 comp rx=0 tx=0).stop 2024-09-19T06:24:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f499a3d0740 1 -- 172.21.15.5:0/3455999233 shutdown_connections 2024-09-19T06:24:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f499a3d0740 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x563b088b9020 0x563b088bb440 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f499a3d0740 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b0883e370 0x563b08837920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f499a3d0740 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b08834b40 0x563b08846c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f499a3d0740 1 --2- 172.21.15.5:0/3455999233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b088335d0 0x563b088466e0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f499a3d0740 1 -- 172.21.15.5:0/3455999233 >> 172.21.15.5:0/3455999233 conn(0x563b087fc310 msgr2=0x563b0882e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:29.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f49797fa640 0 client.20718 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:29.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.195+0000 7f499a3d0740 1 -- 172.21.15.5:0/3455999233 shutdown_connections 2024-09-19T06:24:29.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.196+0000 7f499a3d0740 1 -- 172.21.15.5:0/3455999233 wait complete. 2024-09-19T06:24:29.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.196+0000 7f499a3d0740 20 client.20718 trim_cache size 0 max 16384 2024-09-19T06:24:29.201 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:24:29.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.196+0000 7f4997ff7640 1 -- 172.21.15.5:0/3138362792 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f497c0a33b0 msgr2=0x7f497c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:29.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.196+0000 7f4997ff7640 1 --2- 172.21.15.5:0/3138362792 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f497c0a33b0 0x7f497c0a5870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f49900fec70 tx=0x7f49800098b0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.196+0000 7f4997ff7640 1 -- 172.21.15.5:0/3138362792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f499010c180 msgr2=0x7f499019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:29.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.196+0000 7f4997ff7640 1 --2- 172.21.15.5:0/3138362792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f499010c180 0x7f499019ee40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f498800eb10 tx=0x7f498800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.196+0000 7f4997ff7640 1 -- 172.21.15.5:0/3138362792 shutdown_connections 2024-09-19T06:24:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.196+0000 7f4997ff7640 1 --2- 172.21.15.5:0/3138362792 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f497c0a33b0 0x7f497c0a5870 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.196+0000 7f4997ff7640 1 --2- 172.21.15.5:0/3138362792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f499010c180 0x7f499019ee40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.196+0000 7f4997ff7640 1 --2- 172.21.15.5:0/3138362792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49901036c0 0x7f499019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.196+0000 7f4997ff7640 1 --2- 172.21.15.5:0/3138362792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4990102da0 0x7f499019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.196+0000 7f4997ff7640 1 -- 172.21.15.5:0/3138362792 >> 172.21.15.5:0/3138362792 conn(0x7f49900fc7a0 msgr2=0x7f4990101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.197+0000 7f4997ff7640 1 -- 172.21.15.5:0/3138362792 shutdown_connections 2024-09-19T06:24:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:29.197+0000 7f4997ff7640 1 -- 172.21.15.5:0/3138362792 wait complete. 2024-09-19T06:24:29.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 76958 8.64 MB/sec execute 353 sec latency 67.824 ms 2024-09-19T06:24:30.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77120 8.65 MB/sec execute 354 sec latency 62.461 ms 2024-09-19T06:24:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:30 smithi005 ceph-mon[30955]: pgmap v668: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-09-19T06:24:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:30 smithi049 ceph-mon[34640]: pgmap v668: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-09-19T06:24:31.042 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:24:31.043 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:24:31.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:30 smithi157 ceph-mon[35346]: pgmap v668: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-09-19T06:24:31.424 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:24:31.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77299 8.66 MB/sec execute 355 sec latency 72.611 ms 2024-09-19T06:24:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.263+0000 7f266d3cd640 1 -- 172.21.15.5:0/2208741475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2668100e20 msgr2=0x7f2668101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.263+0000 7f266d3cd640 1 --2- 172.21.15.5:0/2208741475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2668100e20 0x7f2668101200 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f2658009860 tx=0x7f265802a3d0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.263+0000 7f266d3cd640 1 -- 172.21.15.5:0/2208741475 shutdown_connections 2024-09-19T06:24:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.263+0000 7f266d3cd640 1 --2- 172.21.15.5:0/2208741475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2668102ef0 0x7f266810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.263+0000 7f266d3cd640 1 --2- 172.21.15.5:0/2208741475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2668102570 0x7f26681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.263+0000 7f266d3cd640 1 --2- 172.21.15.5:0/2208741475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2668100e20 0x7f2668101200 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.263+0000 7f266d3cd640 1 -- 172.21.15.5:0/2208741475 >> 172.21.15.5:0/2208741475 conn(0x7f26680fc7d0 msgr2=0x7f26680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.263+0000 7f266d3cd640 1 -- 172.21.15.5:0/2208741475 shutdown_connections 2024-09-19T06:24:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.264+0000 7f266d3cd640 1 -- 172.21.15.5:0/2208741475 wait complete. 2024-09-19T06:24:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.264+0000 7f266d3cd640 1 Processor -- start 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.264+0000 7f266d3cd640 1 -- start start 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f266d3cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2668100e20 0x7f2668103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f266d3cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2668102570 0x7f2668103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f266d3cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2668102ef0 0x7f26681041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f266d3cd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2668110e30 con 0x7f2668102ef0 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f266d3cd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2668110cb0 con 0x7f2668102570 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f266d3cd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2668110fb0 con 0x7f2668100e20 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f2666ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2668100e20 0x7f2668103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f26667fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2668102570 0x7f2668103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f26677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2668102ef0 0x7f26681041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f26677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2668102ef0 0x7f26681041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55410/0 (socket says 172.21.15.5:55410) 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f26667fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2668102570 0x7f2668103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56610/0 (socket says 172.21.15.5:56610) 2024-09-19T06:24:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f26677fe640 1 -- 172.21.15.5:0/1135649695 learned_addr learned my addr 172.21.15.5:0/1135649695 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:32.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f2666ffd640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2668100e20 0x7f2668103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41860/0 (socket says 172.21.15.5:41860) 2024-09-19T06:24:32.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f26677fe640 1 -- 172.21.15.5:0/1135649695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2668100e20 msgr2=0x7f2668103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f26677fe640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2668100e20 0x7f2668103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f26677fe640 1 -- 172.21.15.5:0/1135649695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2668102570 msgr2=0x7f2668103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f26677fe640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2668102570 0x7f2668103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f26677fe640 1 -- 172.21.15.5:0/1135649695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26681a6920 con 0x7f2668102ef0 2024-09-19T06:24:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.265+0000 7f26667fc640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2668102570 0x7f2668103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.266+0000 7f26677fe640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2668102ef0 0x7f26681041d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f2660009fd0 tx=0x7f266000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.266+0000 7f264ffff640 1 -- 172.21.15.5:0/1135649695 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660004b20 con 0x7f2668102ef0 2024-09-19T06:24:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.266+0000 7f264ffff640 1 -- 172.21.15.5:0/1135649695 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2660003e20 con 0x7f2668102ef0 2024-09-19T06:24:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.266+0000 7f264ffff640 1 -- 172.21.15.5:0/1135649695 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660004820 con 0x7f2668102ef0 2024-09-19T06:24:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.266+0000 7f266d3cd640 1 -- 172.21.15.5:0/1135649695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26681a6c10 con 0x7f2668102ef0 2024-09-19T06:24:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.266+0000 7f266d3cd640 1 -- 172.21.15.5:0/1135649695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26681a7100 con 0x7f2668102ef0 2024-09-19T06:24:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.269+0000 7f264ffff640 1 -- 172.21.15.5:0/1135649695 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f266001d020 con 0x7f2668102ef0 2024-09-19T06:24:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.270+0000 7f266f7a6740 1 Processor -- start 2024-09-19T06:24:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.270+0000 7f266f7a6740 1 -- start start 2024-09-19T06:24:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.270+0000 7f266f7a6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f71db60 0x55f22f71df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.270+0000 7f266f7a6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f71e480 0x55f22f728800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.270+0000 7f266f7a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f728d40 0x55f22f72b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f266f7a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f22f683280 con 0x55f22f728d40 2024-09-19T06:24:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f266f7a6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f22f62ab10 con 0x55f22f71e480 2024-09-19T06:24:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f266f7a6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f22f67f240 con 0x55f22f71db60 2024-09-19T06:24:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f2666ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f71e480 0x55f22f728800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f264ffff640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26500a3430 0x7f26500a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f2666ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f71e480 0x55f22f728800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56616/0 (socket says 172.21.15.5:56616) 2024-09-19T06:24:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f2666ffd640 1 -- 172.21.15.5:0/2459234326 learned_addr learned my addr 172.21.15.5:0/2459234326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f26667fc640 1 --2- 172.21.15.5:0/2459234326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f728d40 0x55f22f72b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f264ffff640 1 -- 172.21.15.5:0/1135649695 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f26600cfc70 con 0x7f2668102ef0 2024-09-19T06:24:32.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f26667fc640 1 --2- 172.21.15.5:0/2459234326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f71db60 0x55f22f71df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f26677fe640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26500a3430 0x7f26500a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f2666ffd640 1 -- 172.21.15.5:0/2459234326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f71db60 msgr2=0x55f22f71df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:32.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f2666ffd640 1 --2- 172.21.15.5:0/2459234326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f71db60 0x55f22f71df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f2666ffd640 1 -- 172.21.15.5:0/2459234326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f728d40 msgr2=0x55f22f72b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:32.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.271+0000 7f2666ffd640 1 --2- 172.21.15.5:0/2459234326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f728d40 0x55f22f72b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.272+0000 7f2666ffd640 1 -- 172.21.15.5:0/2459234326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f22f6371f0 con 0x55f22f71e480 2024-09-19T06:24:32.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.272+0000 7f2666ffd640 1 --2- 172.21.15.5:0/2459234326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f71e480 0x55f22f728800 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2658005b60 tx=0x7f26580303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dc57e06ca94b546 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.272+0000 7f26677fe640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26500a3430 0x7f26500a58f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2660098390 tx=0x7f2660004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.272+0000 7f264d7fa640 1 -- 172.21.15.5:0/2459234326 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580309a0 con 0x55f22f71e480 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.272+0000 7f264d7fa640 1 -- 172.21.15.5:0/2459234326 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26580054f0 con 0x55f22f71e480 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.272+0000 7f264d7fa640 1 -- 172.21.15.5:0/2459234326 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658004bb0 con 0x55f22f71e480 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.273+0000 7f266f7a6740 1 -- 172.21.15.5:0/2459234326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f71e480 msgr2=0x55f22f728800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.273+0000 7f266f7a6740 1 --2- 172.21.15.5:0/2459234326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f71e480 0x55f22f728800 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2658005b60 tx=0x7f26580303a0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.273+0000 7f266f7a6740 1 -- 172.21.15.5:0/2459234326 shutdown_connections 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.273+0000 7f266f7a6740 1 --2- 172.21.15.5:0/2459234326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f728d40 0x55f22f72b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.273+0000 7f266f7a6740 1 --2- 172.21.15.5:0/2459234326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f71e480 0x55f22f728800 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.273+0000 7f266f7a6740 1 --2- 172.21.15.5:0/2459234326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f71db60 0x55f22f71df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.273+0000 7f266f7a6740 1 -- 172.21.15.5:0/2459234326 >> 172.21.15.5:0/2459234326 conn(0x55f22f71a0f0 msgr2=0x55f22f71a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.273+0000 7f266f7a6740 1 -- 172.21.15.5:0/2459234326 shutdown_connections 2024-09-19T06:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.273+0000 7f266f7a6740 1 -- 172.21.15.5:0/2459234326 wait complete. 2024-09-19T06:24:32.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.274+0000 7f266f7a6740 1 -- start start 2024-09-19T06:24:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.274+0000 7f266f7a6740 1 Processor -- start 2024-09-19T06:24:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.274+0000 7f266f7a6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f71db60 0x55f22f7287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.274+0000 7f266f7a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71e480 0x55f22f725b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.274+0000 7f266f7a6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f728d40 0x55f22f7262d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.274+0000 7f266f7a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f22f72b960 con 0x55f22f71e480 2024-09-19T06:24:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.274+0000 7f266f7a6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f22f63df10 con 0x55f22f728d40 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.274+0000 7f266f7a6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f22f72bae0 con 0x55f22f71db60 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.274+0000 7f266f7a6740 10 client.0 _ll_register_callbacks cb 0x55f22f606a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.275+0000 7f2666ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71e480 0x55f22f725b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.275+0000 7f2666ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71e480 0x55f22f725b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55426/0 (socket says 172.21.15.5:55426) 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.275+0000 7f2666ffd640 1 -- 172.21.15.5:0/1539065994 learned_addr learned my addr 172.21.15.5:0/1539065994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.275+0000 7f2666ffd640 1 -- 172.21.15.5:0/1539065994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f71db60 msgr2=0x55f22f7287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.275+0000 7f2666ffd640 1 --2- 172.21.15.5:0/1539065994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f71db60 0x55f22f7287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.275+0000 7f2666ffd640 1 -- 172.21.15.5:0/1539065994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f728d40 msgr2=0x55f22f7262d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.275+0000 7f2666ffd640 1 --2- 172.21.15.5:0/1539065994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f728d40 0x55f22f7262d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.275+0000 7f2666ffd640 1 -- 172.21.15.5:0/1539065994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f22f3f46a0 con 0x55f22f71e480 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.275+0000 7f2666ffd640 1 --2- 172.21.15.5:0/1539065994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71e480 0x55f22f725b40 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f26580308b0 tx=0x7f265802a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679beb80bc96872a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.275+0000 7f264aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:24:32.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.275+0000 7f264aff5640 1 -- 172.21.15.5:0/1539065994 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658009450 con 0x55f22f71e480 2024-09-19T06:24:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.276+0000 7f264aff5640 1 -- 172.21.15.5:0/1539065994 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f2658004040 con 0x55f22f71e480 2024-09-19T06:24:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.276+0000 7f266f7a6740 1 -- 172.21.15.5:0/1539065994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26580054f0 con 0x55f22f71e480 2024-09-19T06:24:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.276+0000 7f264aff5640 1 -- 172.21.15.5:0/1539065994 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2658004240 con 0x55f22f71e480 2024-09-19T06:24:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.276+0000 7f266f7a6740 10 client.20730 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.276+0000 7f266f7a6740 1 -- 172.21.15.5:0/1539065994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55f22f72b960 con 0x55f22f71e480 2024-09-19T06:24:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.276+0000 7f264aff5640 1 -- 172.21.15.5:0/1539065994 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26580054f0 con 0x55f22f71e480 2024-09-19T06:24:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.277+0000 7f264aff5640 1 -- 172.21.15.5:0/1539065994 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f2658044020 con 0x55f22f71e480 2024-09-19T06:24:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.277+0000 7f266f7a6740 10 client.20730 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.277+0000 7f266f7a6740 10 client.20730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.277+0000 7f266f7a6740 20 client.20730 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.277+0000 7f266f7a6740 1 --2- 172.21.15.5:0/1539065994 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f22f738640 0x55f22f73aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.277+0000 7f266f7a6740 4 client.20730 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:24:32.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.277+0000 7f266f7a6740 1 -- 172.21.15.5:0/1539065994 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f22f6907f0 con 0x55f22f738640 2024-09-19T06:24:32.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.277+0000 7f26667fc640 1 --2- 172.21.15.5:0/1539065994 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f22f738640 0x55f22f73aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.278+0000 7f26667fc640 1 --2- 172.21.15.5:0/1539065994 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f22f738640 0x55f22f73aa30 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f3cc51ea07039a8 server_cookie=86d91ccb1480898b in_seq=0 out_seq=0 2024-09-19T06:24:32.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.278+0000 7f264aff5640 10 client.20730 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:32.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.278+0000 7f264aff5640 1 -- 172.21.15.5:0/1539065994 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f22f6907f0 con 0x55f22f738640 2024-09-19T06:24:32.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.278+0000 7f264aff5640 10 client.20730 handle_command_reply: tid=0 2024-09-19T06:24:32.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.278+0000 7f266f7a6740 1 client.20730 shutdown 2024-09-19T06:24:32.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.279+0000 7f266f7a6740 1 -- 172.21.15.5:0/1539065994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71e480 msgr2=0x55f22f725b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:32.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.279+0000 7f266f7a6740 1 --2- 172.21.15.5:0/1539065994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71e480 0x55f22f725b40 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f26580308b0 tx=0x7f265802a3d0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.279+0000 7f266f7a6740 1 -- 172.21.15.5:0/1539065994 shutdown_connections 2024-09-19T06:24:32.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.279+0000 7f266f7a6740 1 --2- 172.21.15.5:0/1539065994 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f22f738640 0x55f22f73aa30 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.279+0000 7f266f7a6740 1 --2- 172.21.15.5:0/1539065994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f728d40 0x55f22f7262d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.279+0000 7f266f7a6740 1 --2- 172.21.15.5:0/1539065994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71e480 0x55f22f725b40 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.279+0000 7f266f7a6740 1 --2- 172.21.15.5:0/1539065994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f71db60 0x55f22f7287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.279+0000 7f264aff5640 0 client.20730 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:32.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.279+0000 7f266f7a6740 1 -- 172.21.15.5:0/1539065994 >> 172.21.15.5:0/1539065994 conn(0x55f22f71a0f0 msgr2=0x55f22f719270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:32.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.279+0000 7f266f7a6740 1 -- 172.21.15.5:0/1539065994 shutdown_connections 2024-09-19T06:24:32.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.279+0000 7f266f7a6740 1 -- 172.21.15.5:0/1539065994 wait complete. 2024-09-19T06:24:32.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.279+0000 7f266f7a6740 20 client.20730 trim_cache size 0 max 16384 2024-09-19T06:24:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.309+0000 7f266f7a6740 1 Processor -- start 2024-09-19T06:24:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.309+0000 7f266f7a6740 1 -- start start 2024-09-19T06:24:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.309+0000 7f266f7a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 0x55f22f724b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.309+0000 7f266f7a6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f728d40 0x55f22f740290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.309+0000 7f266f7a6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f7407d0 0x55f22f722da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.309+0000 7f266f7a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26681af920 con 0x55f22f71db60 2024-09-19T06:24:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.309+0000 7f266f7a6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f26681b14c0 con 0x55f22f7407d0 2024-09-19T06:24:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.309+0000 7f266f7a6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26681afff0 con 0x55f22f728d40 2024-09-19T06:24:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f2666ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f728d40 0x55f22f740290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f2666ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f728d40 0x55f22f740290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41890/0 (socket says 172.21.15.5:41890) 2024-09-19T06:24:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f2666ffd640 1 -- 172.21.15.5:0/3644955276 learned_addr learned my addr 172.21.15.5:0/3644955276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f26667fc640 1 --2- 172.21.15.5:0/3644955276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f7407d0 0x55f22f722da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f26667fc640 1 --2- 172.21.15.5:0/3644955276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 0x55f22f724b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f2666ffd640 1 -- 172.21.15.5:0/3644955276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f7407d0 msgr2=0x55f22f722da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f2666ffd640 1 --2- 172.21.15.5:0/3644955276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f7407d0 0x55f22f722da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f2666ffd640 1 -- 172.21.15.5:0/3644955276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 msgr2=0x55f22f724b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f2666ffd640 1 --2- 172.21.15.5:0/3644955276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 0x55f22f724b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f2666ffd640 1 -- 172.21.15.5:0/3644955276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f22f71e2b0 con 0x55f22f728d40 2024-09-19T06:24:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f26667fc640 1 --2- 172.21.15.5:0/3644955276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 0x55f22f724b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:24:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f2666ffd640 1 --2- 172.21.15.5:0/3644955276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f728d40 0x55f22f740290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2658003840 tx=0x7f2658005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c806e3cb58adc96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.310+0000 7f264cff9640 1 -- 172.21.15.5:0/3644955276 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658061070 con 0x55f22f728d40 2024-09-19T06:24:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.311+0000 7f264cff9640 1 -- 172.21.15.5:0/3644955276 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f265804f070 con 0x55f22f728d40 2024-09-19T06:24:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.311+0000 7f264cff9640 1 -- 172.21.15.5:0/3644955276 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658058030 con 0x55f22f728d40 2024-09-19T06:24:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.312+0000 7f266f7a6740 1 -- 172.21.15.5:0/3644955276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f728d40 msgr2=0x55f22f740290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.312+0000 7f266f7a6740 1 --2- 172.21.15.5:0/3644955276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f728d40 0x55f22f740290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2658003840 tx=0x7f2658005dc0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.312+0000 7f266f7a6740 1 -- 172.21.15.5:0/3644955276 shutdown_connections 2024-09-19T06:24:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.312+0000 7f266f7a6740 1 --2- 172.21.15.5:0/3644955276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f7407d0 0x55f22f722da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.312+0000 7f266f7a6740 1 --2- 172.21.15.5:0/3644955276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f728d40 0x55f22f740290 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.312+0000 7f266f7a6740 1 --2- 172.21.15.5:0/3644955276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 0x55f22f724b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.312+0000 7f266f7a6740 1 -- 172.21.15.5:0/3644955276 >> 172.21.15.5:0/3644955276 conn(0x55f22f71a0f0 msgr2=0x55f22f724410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.312+0000 7f266f7a6740 1 -- 172.21.15.5:0/3644955276 shutdown_connections 2024-09-19T06:24:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.312+0000 7f266f7a6740 1 -- 172.21.15.5:0/3644955276 wait complete. 2024-09-19T06:24:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.312+0000 7f266f7a6740 1 -- start start 2024-09-19T06:24:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.312+0000 7f266f7a6740 1 Processor -- start 2024-09-19T06:24:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f266f7a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 0x55f22f72d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f266f7a6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f728d40 0x55f22f72d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f266f7a6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f7407d0 0x55f22f7a2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f266f7a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26681b0df0 con 0x55f22f71db60 2024-09-19T06:24:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f266f7a6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f26681b06f0 con 0x55f22f728d40 2024-09-19T06:24:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f266f7a6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26681ad920 con 0x55f22f7407d0 2024-09-19T06:24:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f266f7a6740 10 client.0 _ll_register_callbacks cb 0x7f265c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f26667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 0x55f22f72d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f26667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 0x55f22f72d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55440/0 (socket says 172.21.15.5:55440) 2024-09-19T06:24:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f26667fc640 1 -- 172.21.15.5:0/3620754024 learned_addr learned my addr 172.21.15.5:0/3620754024 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f2666ffd640 1 --2- 172.21.15.5:0/3620754024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f728d40 0x55f22f72d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f26667fc640 1 -- 172.21.15.5:0/3620754024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f7407d0 msgr2=0x55f22f7a2260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:24:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f26667fc640 1 --2- 172.21.15.5:0/3620754024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f7407d0 0x55f22f7a2260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f26667fc640 1 -- 172.21.15.5:0/3620754024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f728d40 msgr2=0x55f22f72d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f26667fc640 1 --2- 172.21.15.5:0/3620754024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f728d40 0x55f22f72d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.313+0000 7f26667fc640 1 -- 172.21.15.5:0/3620754024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f22f7a27a0 con 0x55f22f71db60 2024-09-19T06:24:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.314+0000 7f26667fc640 1 --2- 172.21.15.5:0/3620754024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 0x55f22f72d1e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x55f22f71b470 tx=0x7f265c003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09bea5435fe70cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.314+0000 7f264aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:24:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.314+0000 7f264aff5640 1 -- 172.21.15.5:0/3620754024 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c003ea0 con 0x55f22f71db60 2024-09-19T06:24:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.314+0000 7f264aff5640 1 -- 172.21.15.5:0/3620754024 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f265c01fe80 con 0x55f22f71db60 2024-09-19T06:24:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.314+0000 7f266f7a6740 1 -- 172.21.15.5:0/3620754024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f265804f070 con 0x55f22f71db60 2024-09-19T06:24:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.315+0000 7f264aff5640 1 -- 172.21.15.5:0/3620754024 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f265c011970 con 0x55f22f71db60 2024-09-19T06:24:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.315+0000 7f266f7a6740 10 client.20733 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.315+0000 7f266f7a6740 1 -- 172.21.15.5:0/3620754024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55f22f7a3a80 con 0x55f22f71db60 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.315+0000 7f264aff5640 1 -- 172.21.15.5:0/3620754024 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f265804f070 con 0x55f22f71db60 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.315+0000 7f264aff5640 1 -- 172.21.15.5:0/3620754024 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f265c031420 con 0x55f22f71db60 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.315+0000 7f266f7a6740 10 client.20733 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.315+0000 7f266f7a6740 10 client.20733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.315+0000 7f266f7a6740 20 client.20733 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.315+0000 7f266f7a6740 1 --2- 172.21.15.5:0/3620754024 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f22f7a4160 0x55f22f7a6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.315+0000 7f266f7a6740 4 client.20733 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.315+0000 7f266f7a6740 1 -- 172.21.15.5:0/3620754024 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f22f68ad00 con 0x55f22f7a4160 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.316+0000 7f2666ffd640 1 --2- 172.21.15.5:0/3620754024 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f22f7a4160 0x55f22f7a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.316+0000 7f264aff5640 10 client.20733 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.316+0000 7f2666ffd640 1 --2- 172.21.15.5:0/3620754024 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f22f7a4160 0x55f22f7a6580 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cc98c38d5060f6f server_cookie=dae5ff1ef80c0cfc in_seq=0 out_seq=0 2024-09-19T06:24:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f264aff5640 1 -- 172.21.15.5:0/3620754024 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f22f68ad00 con 0x55f22f7a4160 2024-09-19T06:24:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f264aff5640 10 client.20733 handle_command_reply: tid=0 2024-09-19T06:24:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f266f7a6740 1 client.20733 shutdown 2024-09-19T06:24:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f266f7a6740 1 -- 172.21.15.5:0/3620754024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 msgr2=0x55f22f72d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f266f7a6740 1 --2- 172.21.15.5:0/3620754024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 0x55f22f72d1e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x55f22f71b470 tx=0x7f265c003cb0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f266f7a6740 1 -- 172.21.15.5:0/3620754024 shutdown_connections 2024-09-19T06:24:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f266f7a6740 1 --2- 172.21.15.5:0/3620754024 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f22f7a4160 0x55f22f7a6580 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f264aff5640 0 client.20733 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f266f7a6740 1 --2- 172.21.15.5:0/3620754024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f22f7407d0 0x55f22f7a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f266f7a6740 1 --2- 172.21.15.5:0/3620754024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f22f728d40 0x55f22f72d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f266f7a6740 1 --2- 172.21.15.5:0/3620754024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f22f71db60 0x55f22f72d1e0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f266f7a6740 1 -- 172.21.15.5:0/3620754024 >> 172.21.15.5:0/3620754024 conn(0x55f22f71a0f0 msgr2=0x55f22f724c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.317+0000 7f266f7a6740 1 -- 172.21.15.5:0/3620754024 shutdown_connections 2024-09-19T06:24:32.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.318+0000 7f266f7a6740 1 -- 172.21.15.5:0/3620754024 wait complete. 2024-09-19T06:24:32.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.318+0000 7f266f7a6740 20 client.20733 trim_cache size 0 max 16384 2024-09-19T06:24:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:24:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:24:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "d7930ae0-abfb-4051-9e40-8f42ef279c61", 2024-09-19T06:24:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:24:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:24:32.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.319+0000 7f266d3cd640 1 -- 172.21.15.5:0/1135649695 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26500a3430 msgr2=0x7f26500a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:32.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.319+0000 7f266d3cd640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26500a3430 0x7f26500a58f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2660098390 tx=0x7f2660004a40 comp rx=0 tx=0).stop 2024-09-19T06:24:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.319+0000 7f266d3cd640 1 -- 172.21.15.5:0/1135649695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2668102ef0 msgr2=0x7f26681041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.319+0000 7f266d3cd640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2668102ef0 0x7f26681041d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f2660009fd0 tx=0x7f266000c800 comp rx=0 tx=0).stop 2024-09-19T06:24:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.319+0000 7f266d3cd640 1 -- 172.21.15.5:0/1135649695 shutdown_connections 2024-09-19T06:24:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.319+0000 7f266d3cd640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2668102ef0 0x7f26681041d0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.319+0000 7f266d3cd640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2668102570 0x7f2668103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.319+0000 7f266d3cd640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f26500a3430 0x7f26500a58f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.319+0000 7f266d3cd640 1 --2- 172.21.15.5:0/1135649695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2668100e20 0x7f2668103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.319+0000 7f266d3cd640 1 -- 172.21.15.5:0/1135649695 >> 172.21.15.5:0/1135649695 conn(0x7f26680fc7d0 msgr2=0x7f266806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.319+0000 7f266d3cd640 1 -- 172.21.15.5:0/1135649695 shutdown_connections 2024-09-19T06:24:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:32.319+0000 7f266d3cd640 1 -- 172.21.15.5:0/1135649695 wait complete. 2024-09-19T06:24:32.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77431 8.65 MB/sec execute 356 sec latency 283.628 ms 2024-09-19T06:24:32.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:32 smithi005 ceph-mon[30955]: pgmap v669: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-09-19T06:24:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:32 smithi049 ceph-mon[34640]: pgmap v669: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-09-19T06:24:33.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:32 smithi157 ceph-mon[35346]: pgmap v669: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-09-19T06:24:33.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77535 8.64 MB/sec execute 357 sec latency 302.984 ms 2024-09-19T06:24:33.610 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:24:33.610 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.611 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:24:33.611 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:24:33.611 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:24:33.611 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:24:33.611 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:24:33.611 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.611 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:24:33.611 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:24:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:24:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:24:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:24:33.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:24:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:24:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:24:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:24:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:24:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:24:33.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:24:33.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:33 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:24:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:24:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:24:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:24:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:24:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:24:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:33.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:33 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:24:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:34.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:34.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:24:34.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:34.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:24:34.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:34.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:24:34.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:34.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:33 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:24:34.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77627 8.62 MB/sec execute 358 sec latency 106.335 ms 2024-09-19T06:24:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:34 smithi005 ceph-mon[30955]: pgmap v670: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 348 op/s 2024-09-19T06:24:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:34 smithi049 ceph-mon[34640]: pgmap v670: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 348 op/s 2024-09-19T06:24:35.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:34 smithi157 ceph-mon[35346]: pgmap v670: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 348 op/s 2024-09-19T06:24:35.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77752 8.63 MB/sec execute 359 sec latency 182.083 ms 2024-09-19T06:24:35.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:35 smithi005 ceph-mon[30955]: pgmap v671: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 236 op/s 2024-09-19T06:24:35.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:35 smithi049 ceph-mon[34640]: pgmap v671: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 236 op/s 2024-09-19T06:24:36.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:35 smithi157 ceph-mon[35346]: pgmap v671: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 236 op/s 2024-09-19T06:24:36.386 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:24:36.591 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77884 8.61 MB/sec execute 360 sec latency 146.459 ms 2024-09-19T06:24:36.824 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:24:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.373+0000 7fbbd118d640 1 -- 172.21.15.5:0/3932934716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc102d60 msgr2=0x7fbbcc103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.373+0000 7fbbd118d640 1 --2- 172.21.15.5:0/3932934716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc102d60 0x7fbbcc103140 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009900 tx=0x7fbbbc02a480 comp rx=0 tx=0).stop 2024-09-19T06:24:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.373+0000 7fbbd118d640 1 -- 172.21.15.5:0/3932934716 shutdown_connections 2024-09-19T06:24:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.373+0000 7fbbd118d640 1 --2- 172.21.15.5:0/3932934716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbcc10c190 0x7fbbcc10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.373+0000 7fbbd118d640 1 --2- 172.21.15.5:0/3932934716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbcc103680 0x7fbbcc10bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.373+0000 7fbbd118d640 1 --2- 172.21.15.5:0/3932934716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc102d60 0x7fbbcc103140 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.373+0000 7fbbd118d640 1 -- 172.21.15.5:0/3932934716 >> 172.21.15.5:0/3932934716 conn(0x7fbbcc0fc7a0 msgr2=0x7fbbcc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.373+0000 7fbbd118d640 1 -- 172.21.15.5:0/3932934716 shutdown_connections 2024-09-19T06:24:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.373+0000 7fbbd118d640 1 -- 172.21.15.5:0/3932934716 wait complete. 2024-09-19T06:24:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.374+0000 7fbbd118d640 1 Processor -- start 2024-09-19T06:24:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.374+0000 7fbbd118d640 1 -- start start 2024-09-19T06:24:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.374+0000 7fbbd118d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbcc102d60 0x7fbbcc1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.374+0000 7fbbd118d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc103680 0x7fbbcc1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbd118d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbcc10c190 0x7fbbcc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbd118d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc111e80 con 0x7fbbcc103680 2024-09-19T06:24:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbd118d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc111d00 con 0x7fbbcc102d60 2024-09-19T06:24:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbd118d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc112000 con 0x7fbbcc10c190 2024-09-19T06:24:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbcad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbcc102d60 0x7fbbcc1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbcb59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbcc10c190 0x7fbbcc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbcb59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbcc10c190 0x7fbbcc19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50148/0 (socket says 172.21.15.5:50148) 2024-09-19T06:24:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbcb59e640 1 -- 172.21.15.5:0/2599247487 learned_addr learned my addr 172.21.15.5:0/2599247487 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbca59c640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc103680 0x7fbbcc1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbcb59e640 1 -- 172.21.15.5:0/2599247487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbcc102d60 msgr2=0x7fbbcc1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbcb59e640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbcc102d60 0x7fbbcc1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbcb59e640 1 -- 172.21.15.5:0/2599247487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc103680 msgr2=0x7fbbcc1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbcb59e640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc103680 0x7fbbcc1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbcb59e640 1 -- 172.21.15.5:0/2599247487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc19d360 con 0x7fbbcc10c190 2024-09-19T06:24:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbcad9d640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbcc102d60 0x7fbbcc1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:24:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.375+0000 7fbbcb59e640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbcc10c190 0x7fbbcc19cc80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400efd0 tx=0x7fbbc400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:37.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.376+0000 7fbbb3fff640 1 -- 172.21.15.5:0/2599247487 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4003b70 con 0x7fbbcc10c190 2024-09-19T06:24:37.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.376+0000 7fbbd118d640 1 -- 172.21.15.5:0/2599247487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbcc19d650 con 0x7fbbcc10c190 2024-09-19T06:24:37.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.376+0000 7fbbb3fff640 1 -- 172.21.15.5:0/2599247487 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbc4003d10 con 0x7fbbcc10c190 2024-09-19T06:24:37.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.376+0000 7fbbb3fff640 1 -- 172.21.15.5:0/2599247487 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4010980 con 0x7fbbcc10c190 2024-09-19T06:24:37.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.376+0000 7fbbd118d640 1 -- 172.21.15.5:0/2599247487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbcc1008f0 con 0x7fbbcc10c190 2024-09-19T06:24:37.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.379+0000 7fbbb3fff640 1 -- 172.21.15.5:0/2599247487 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbbc4004020 con 0x7fbbcc10c190 2024-09-19T06:24:37.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.379+0000 7fbbd3566740 1 Processor -- start 2024-09-19T06:24:37.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.379+0000 7fbbd3566740 1 -- start start 2024-09-19T06:24:37.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.380+0000 7fbbb3fff640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbbb40a3430 0x7fbbb40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.380+0000 7fbbd3566740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a064a0 0x55aec6a06880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.380+0000 7fbbd3566740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a06dc0 0x55aec6a11140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.380+0000 7fbbd3566740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a11680 0x55aec6a13a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.380+0000 7fbbd3566740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aec696c2e0 con 0x55aec6a06dc0 2024-09-19T06:24:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.380+0000 7fbbd3566740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aec6913ce0 con 0x55aec6a064a0 2024-09-19T06:24:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.380+0000 7fbbd3566740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55aec69682a0 con 0x55aec6a11680 2024-09-19T06:24:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.380+0000 7fbbb3fff640 1 -- 172.21.15.5:0/2599247487 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7fbbc40d0160 con 0x7fbbcc10c190 2024-09-19T06:24:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.380+0000 7fbbcb59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a11680 0x55aec6a13a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.380+0000 7fbbca59c640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbbb40a3430 0x7fbbb40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.380+0000 7fbbcb59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a11680 0x55aec6a13a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50150/0 (socket says 172.21.15.5:50150) 2024-09-19T06:24:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.380+0000 7fbbcb59e640 1 -- 172.21.15.5:0/157255714 learned_addr learned my addr 172.21.15.5:0/157255714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.381+0000 7fbbcb59e640 1 -- 172.21.15.5:0/157255714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a064a0 msgr2=0x55aec6a06880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:24:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.381+0000 7fbbcb59e640 1 --2- 172.21.15.5:0/157255714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a064a0 0x55aec6a06880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.381+0000 7fbbcb59e640 1 -- 172.21.15.5:0/157255714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a06dc0 msgr2=0x55aec6a11140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.381+0000 7fbbcb59e640 1 --2- 172.21.15.5:0/157255714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a06dc0 0x55aec6a11140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.381+0000 7fbbcb59e640 1 -- 172.21.15.5:0/157255714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec68f8440 con 0x55aec6a11680 2024-09-19T06:24:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.381+0000 7fbbca59c640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbbb40a3430 0x7fbbb40a58f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55aec6a03890 tx=0x7fbbc00096a0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.381+0000 7fbbcb59e640 1 --2- 172.21.15.5:0/157255714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a11680 0x55aec6a13a70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4098d90 tx=0x7fbbc409bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca115879c94d1f61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.381+0000 7fbbb17fa640 1 -- 172.21.15.5:0/157255714 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc409cac0 con 0x55aec6a11680 2024-09-19T06:24:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.381+0000 7fbbb17fa640 1 -- 172.21.15.5:0/157255714 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbc409c3c0 con 0x55aec6a11680 2024-09-19T06:24:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.381+0000 7fbbb17fa640 1 -- 172.21.15.5:0/157255714 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4098a30 con 0x55aec6a11680 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.382+0000 7fbbd3566740 1 -- 172.21.15.5:0/157255714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a11680 msgr2=0x55aec6a13a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.382+0000 7fbbd3566740 1 --2- 172.21.15.5:0/157255714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a11680 0x55aec6a13a70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4098d90 tx=0x7fbbc409bfc0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.382+0000 7fbbd3566740 1 -- 172.21.15.5:0/157255714 shutdown_connections 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.382+0000 7fbbd3566740 1 --2- 172.21.15.5:0/157255714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a11680 0x55aec6a13a70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.382+0000 7fbbd3566740 1 --2- 172.21.15.5:0/157255714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a06dc0 0x55aec6a11140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.382+0000 7fbbd3566740 1 --2- 172.21.15.5:0/157255714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a064a0 0x55aec6a06880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.382+0000 7fbbd3566740 1 -- 172.21.15.5:0/157255714 >> 172.21.15.5:0/157255714 conn(0x55aec6a02a30 msgr2=0x55aec6a02e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.383+0000 7fbbd3566740 1 -- 172.21.15.5:0/157255714 shutdown_connections 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.383+0000 7fbbd3566740 1 -- 172.21.15.5:0/157255714 wait complete. 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.383+0000 7fbbd3566740 1 -- start start 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.383+0000 7fbbd3566740 1 Processor -- start 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbd3566740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a064a0 0x55aec6a11050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbd3566740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a06dc0 0x55aec6a0e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbd3566740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 0x55aec6a0e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbd3566740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aec6a14250 con 0x55aec6a11680 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbd3566740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aec6927050 con 0x55aec6a064a0 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbd3566740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55aec6a143d0 con 0x55aec6a06dc0 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbcad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a064a0 0x55aec6a11050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbcb59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 0x55aec6a0e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbcad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a064a0 0x55aec6a11050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37458/0 (socket says 172.21.15.5:37458) 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbcad9d640 1 -- 172.21.15.5:0/1954957583 learned_addr learned my addr 172.21.15.5:0/1954957583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbcb59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 0x55aec6a0e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35334/0 (socket says 172.21.15.5:35334) 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbd3566740 10 client.0 _ll_register_callbacks cb 0x55aec6967160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbca59c640 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a06dc0 0x55aec6a0e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbcad9d640 1 -- 172.21.15.5:0/1954957583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a06dc0 msgr2=0x55aec6a0e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbcad9d640 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a06dc0 0x55aec6a0e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbcad9d640 1 -- 172.21.15.5:0/1954957583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 msgr2=0x55aec6a0e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbcad9d640 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 0x55aec6a0e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbcad9d640 1 -- 172.21.15.5:0/1954957583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aec66dd3c0 con 0x55aec6a064a0 2024-09-19T06:24:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.384+0000 7fbbcb59e640 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 0x55aec6a0e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.385+0000 7fbbcad9d640 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a064a0 0x55aec6a11050 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009230 tx=0x7fbbbc005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cdb9fac3a32d336 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.385+0000 7fbbaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:24:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.385+0000 7fbbaeff5640 1 -- 172.21.15.5:0/1954957583 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc035030 con 0x55aec6a064a0 2024-09-19T06:24:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.385+0000 7fbbd3566740 1 -- 172.21.15.5:0/1954957583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc409c3c0 con 0x55aec6a064a0 2024-09-19T06:24:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.385+0000 7fbbaeff5640 1 -- 172.21.15.5:0/1954957583 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7fbbbc0046c0 con 0x55aec6a064a0 2024-09-19T06:24:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.385+0000 7fbbaeff5640 1 -- 172.21.15.5:0/1954957583 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fbbbc005d50 con 0x55aec6a064a0 2024-09-19T06:24:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.385+0000 7fbbd3566740 10 client.30119 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.385+0000 7fbbd3566740 1 -- 172.21.15.5:0/1954957583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55aec6a1fcb0 con 0x55aec6a064a0 2024-09-19T06:24:37.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.385+0000 7fbbaeff5640 1 -- 172.21.15.5:0/1954957583 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbbc004d70 con 0x55aec6a064a0 2024-09-19T06:24:37.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.386+0000 7fbbaeff5640 1 -- 172.21.15.5:0/1954957583 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fbbbc003d20 con 0x55aec6a064a0 2024-09-19T06:24:37.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.386+0000 7fbbd3566740 10 client.30119 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:37.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.386+0000 7fbbd3566740 10 client.30119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.386+0000 7fbbd3566740 20 client.30119 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.386+0000 7fbbd3566740 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aec6a210c0 0x55aec6a23500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.386+0000 7fbbd3566740 4 client.30119 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.386+0000 7fbbd3566740 1 -- 172.21.15.5:0/1954957583 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbbc409c3c0 con 0x55aec6a210c0 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.386+0000 7fbbca59c640 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aec6a210c0 0x55aec6a23500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.386+0000 7fbbaeff5640 10 client.30119 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.386+0000 7fbbca59c640 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aec6a210c0 0x55aec6a23500 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4099e86f6eab47b server_cookie=5c84114853067df4 in_seq=0 out_seq=0 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.387+0000 7fbbaeff5640 1 -- 172.21.15.5:0/1954957583 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fbbc409c3c0 con 0x55aec6a210c0 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.387+0000 7fbbaeff5640 10 client.30119 handle_command_reply: tid=0 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.387+0000 7fbbd3566740 1 client.30119 shutdown 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.388+0000 7fbbd3566740 1 -- 172.21.15.5:0/1954957583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a064a0 msgr2=0x55aec6a11050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.388+0000 7fbbd3566740 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a064a0 0x55aec6a11050 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009230 tx=0x7fbbbc005270 comp rx=0 tx=0).stop 2024-09-19T06:24:37.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.388+0000 7fbbd3566740 1 -- 172.21.15.5:0/1954957583 shutdown_connections 2024-09-19T06:24:37.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.388+0000 7fbbd3566740 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aec6a210c0 0x55aec6a23500 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.388+0000 7fbbd3566740 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 0x55aec6a0e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.388+0000 7fbbaeff5640 0 client.30119 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:37.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.388+0000 7fbbd3566740 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a06dc0 0x55aec6a0e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.388+0000 7fbbd3566740 1 --2- 172.21.15.5:0/1954957583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a064a0 0x55aec6a11050 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.388+0000 7fbbd3566740 1 -- 172.21.15.5:0/1954957583 >> 172.21.15.5:0/1954957583 conn(0x55aec6a02a30 msgr2=0x55aec6a08710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:37.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.388+0000 7fbbd3566740 1 -- 172.21.15.5:0/1954957583 shutdown_connections 2024-09-19T06:24:37.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.388+0000 7fbbd3566740 1 -- 172.21.15.5:0/1954957583 wait complete. 2024-09-19T06:24:37.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.388+0000 7fbbd3566740 20 client.30119 trim_cache size 0 max 16384 2024-09-19T06:24:37.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.411+0000 7fbbd3566740 1 Processor -- start 2024-09-19T06:24:37.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.411+0000 7fbbd3566740 1 -- start start 2024-09-19T06:24:37.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.411+0000 7fbbd3566740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a064a0 0x55aec6a1d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.411+0000 7fbbd3566740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a06dc0 0x55aec6a0dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.411+0000 7fbbd3566740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 0x55aec6a01a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.411+0000 7fbbcad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a064a0 0x55aec6a1d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.411+0000 7fbbcad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a064a0 0x55aec6a1d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50172/0 (socket says 172.21.15.5:50172) 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.411+0000 7fbbca59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a06dc0 0x55aec6a0dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbca59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a06dc0 0x55aec6a0dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37466/0 (socket says 172.21.15.5:37466) 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbcad9d640 1 -- 172.21.15.5:0/1866437683 learned_addr learned my addr 172.21.15.5:0/1866437683 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbd3566740 1 -- 172.21.15.5:0/1866437683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1afa90 con 0x55aec6a11680 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbd3566740 1 -- 172.21.15.5:0/1866437683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1b1630 con 0x55aec6a06dc0 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbd3566740 1 -- 172.21.15.5:0/1866437683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1b0160 con 0x55aec6a064a0 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbcb59e640 1 --2- 172.21.15.5:0/1866437683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 0x55aec6a01a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbcad9d640 1 -- 172.21.15.5:0/1866437683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a06dc0 msgr2=0x55aec6a0dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbcad9d640 1 --2- 172.21.15.5:0/1866437683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a06dc0 0x55aec6a0dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbcad9d640 1 -- 172.21.15.5:0/1866437683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 msgr2=0x55aec6a01a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbcad9d640 1 --2- 172.21.15.5:0/1866437683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 0x55aec6a01a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbcad9d640 1 -- 172.21.15.5:0/1866437683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec6a0ec60 con 0x55aec6a064a0 2024-09-19T06:24:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbcb59e640 1 --2- 172.21.15.5:0/1866437683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 0x55aec6a01a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:24:37.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.412+0000 7fbbcad9d640 1 --2- 172.21.15.5:0/1866437683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a064a0 0x55aec6a1d4b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc031440 tx=0x7fbbbc001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39fb5d4363633ec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:37.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.413+0000 7fbbb0ff9640 1 -- 172.21.15.5:0/1866437683 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc035070 con 0x55aec6a064a0 2024-09-19T06:24:37.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.413+0000 7fbbb0ff9640 1 -- 172.21.15.5:0/1866437683 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbbc031770 con 0x55aec6a064a0 2024-09-19T06:24:37.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.413+0000 7fbbb0ff9640 1 -- 172.21.15.5:0/1866437683 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc0038e0 con 0x55aec6a064a0 2024-09-19T06:24:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.414+0000 7fbbd3566740 1 -- 172.21.15.5:0/1866437683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a064a0 msgr2=0x55aec6a1d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.414+0000 7fbbd3566740 1 --2- 172.21.15.5:0/1866437683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a064a0 0x55aec6a1d4b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc031440 tx=0x7fbbbc001e70 comp rx=0 tx=0).stop 2024-09-19T06:24:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.414+0000 7fbbd3566740 1 -- 172.21.15.5:0/1866437683 shutdown_connections 2024-09-19T06:24:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.414+0000 7fbbd3566740 1 --2- 172.21.15.5:0/1866437683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a11680 0x55aec6a01a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.414+0000 7fbbd3566740 1 --2- 172.21.15.5:0/1866437683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a06dc0 0x55aec6a0dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.414+0000 7fbbd3566740 1 --2- 172.21.15.5:0/1866437683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a064a0 0x55aec6a1d4b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.414+0000 7fbbd3566740 1 -- 172.21.15.5:0/1866437683 >> 172.21.15.5:0/1866437683 conn(0x55aec6a02a30 msgr2=0x55aec6a01350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.414+0000 7fbbd3566740 1 -- 172.21.15.5:0/1866437683 shutdown_connections 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.414+0000 7fbbd3566740 1 -- 172.21.15.5:0/1866437683 wait complete. 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.414+0000 7fbbd3566740 1 -- start start 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbd3566740 1 Processor -- start 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbd3566740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a064a0 0x55aec6a19e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbd3566740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a06dc0 0x55aec6a15170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbd3566740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a11680 0x55aec6a15710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbd3566740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1b0f60 con 0x55aec6a064a0 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbd3566740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1b0860 con 0x55aec6a11680 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbd3566740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1ada90 con 0x55aec6a06dc0 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbd3566740 10 client.0 _ll_register_callbacks cb 0x7fbbc40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbcad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a064a0 0x55aec6a19e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbcad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a064a0 0x55aec6a19e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35346/0 (socket says 172.21.15.5:35346) 2024-09-19T06:24:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbcad9d640 1 -- 172.21.15.5:0/519029155 learned_addr learned my addr 172.21.15.5:0/519029155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbca59c640 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a06dc0 0x55aec6a15170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.415+0000 7fbbcb59e640 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a11680 0x55aec6a15710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbca59c640 1 -- 172.21.15.5:0/519029155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a11680 msgr2=0x55aec6a15710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbca59c640 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a11680 0x55aec6a15710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbca59c640 1 -- 172.21.15.5:0/519029155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a064a0 msgr2=0x55aec6a19e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbca59c640 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a064a0 0x55aec6a19e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbca59c640 1 -- 172.21.15.5:0/519029155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aec6a15f40 con 0x55aec6a06dc0 2024-09-19T06:24:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbcad9d640 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a064a0 0x55aec6a19e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbca59c640 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a06dc0 0x55aec6a15170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55aec6a0db90 tx=0x7fbbc0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e0352af20477ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:24:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbaeff5640 1 -- 172.21.15.5:0/519029155 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0034060 con 0x55aec6a06dc0 2024-09-19T06:24:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbaeff5640 1 -- 172.21.15.5:0/519029155 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7fbbc001e630 con 0x55aec6a06dc0 2024-09-19T06:24:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbd3566740 1 -- 172.21.15.5:0/519029155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbbc031770 con 0x55aec6a06dc0 2024-09-19T06:24:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbaeff5640 1 -- 172.21.15.5:0/519029155 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fbbc0018050 con 0x55aec6a06dc0 2024-09-19T06:24:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbd3566740 10 client.30049 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.416+0000 7fbbd3566740 1 -- 172.21.15.5:0/519029155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55aec6a8bee0 con 0x55aec6a06dc0 2024-09-19T06:24:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.417+0000 7fbbaeff5640 1 -- 172.21.15.5:0/519029155 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbc0004e90 con 0x55aec6a06dc0 2024-09-19T06:24:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.417+0000 7fbbaeff5640 1 -- 172.21.15.5:0/519029155 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fbbc002e030 con 0x55aec6a06dc0 2024-09-19T06:24:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.417+0000 7fbbd3566740 10 client.30049 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.417+0000 7fbbd3566740 10 client.30049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.417+0000 7fbbd3566740 20 client.30049 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.417+0000 7fbbd3566740 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aec6a210c0 0x55aec6a8e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.417+0000 7fbbd3566740 4 client.30049 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:24:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.417+0000 7fbbd3566740 1 -- 172.21.15.5:0/519029155 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fbbbc031770 con 0x55aec6a210c0 2024-09-19T06:24:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.417+0000 7fbbcad9d640 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aec6a210c0 0x55aec6a8e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbcad9d640 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aec6a210c0 0x55aec6a8e6d0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=141a49cd435dd42d server_cookie=f9ba04d66042fbcc in_seq=0 out_seq=0 2024-09-19T06:24:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbaeff5640 10 client.30049 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbaeff5640 1 -- 172.21.15.5:0/519029155 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fbbbc031770 con 0x55aec6a210c0 2024-09-19T06:24:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbaeff5640 10 client.30049 handle_command_reply: tid=0 2024-09-19T06:24:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbd3566740 1 client.30049 shutdown 2024-09-19T06:24:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbd3566740 1 -- 172.21.15.5:0/519029155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a06dc0 msgr2=0x55aec6a15170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbd3566740 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a06dc0 0x55aec6a15170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55aec6a0db90 tx=0x7fbbc0005f90 comp rx=0 tx=0).stop 2024-09-19T06:24:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbd3566740 1 -- 172.21.15.5:0/519029155 shutdown_connections 2024-09-19T06:24:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbd3566740 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aec6a210c0 0x55aec6a8e6d0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbd3566740 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aec6a11680 0x55aec6a15710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbd3566740 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aec6a06dc0 0x55aec6a15170 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbd3566740 1 --2- 172.21.15.5:0/519029155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aec6a064a0 0x55aec6a19e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbd3566740 1 -- 172.21.15.5:0/519029155 >> 172.21.15.5:0/519029155 conn(0x55aec6a02a30 msgr2=0x55aec6a08710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:37.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.418+0000 7fbbaeff5640 0 client.30049 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:37.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.419+0000 7fbbd3566740 1 -- 172.21.15.5:0/519029155 shutdown_connections 2024-09-19T06:24:37.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.419+0000 7fbbd3566740 1 -- 172.21.15.5:0/519029155 wait complete. 2024-09-19T06:24:37.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.419+0000 7fbbd3566740 20 client.30049 trim_cache size 0 max 16384 2024-09-19T06:24:37.424 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:24:37.424 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:24:37.424 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:24:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: "d7930ae0-abfb-4051-9e40-8f42ef279c61": { 2024-09-19T06:24:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:24:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d7930ae0-abfb-4051-9e40-8f42ef279c61", 2024-09-19T06:24:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:24:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:24:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:24:37.425 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:24:37.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.419+0000 7fbbd118d640 1 -- 172.21.15.5:0/2599247487 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbbb40a3430 msgr2=0x7fbbb40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:37.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.419+0000 7fbbd118d640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbbb40a3430 0x7fbbb40a58f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55aec6a03890 tx=0x7fbbc00096a0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.419+0000 7fbbd118d640 1 -- 172.21.15.5:0/2599247487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbcc10c190 msgr2=0x7fbbcc19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:37.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.419+0000 7fbbd118d640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbcc10c190 0x7fbbcc19cc80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400efd0 tx=0x7fbbc400c6a0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.420+0000 7fbbd118d640 1 -- 172.21.15.5:0/2599247487 shutdown_connections 2024-09-19T06:24:37.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.420+0000 7fbbd118d640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbbb40a3430 0x7fbbb40a58f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.420+0000 7fbbd118d640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbcc10c190 0x7fbbcc19cc80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.420+0000 7fbbd118d640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc103680 0x7fbbcc1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.420+0000 7fbbd118d640 1 --2- 172.21.15.5:0/2599247487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbcc102d60 0x7fbbcc1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:37.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.420+0000 7fbbd118d640 1 -- 172.21.15.5:0/2599247487 >> 172.21.15.5:0/2599247487 conn(0x7fbbcc0fc7a0 msgr2=0x7fbbcc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:37.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.420+0000 7fbbd118d640 1 -- 172.21.15.5:0/2599247487 shutdown_connections 2024-09-19T06:24:37.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:24:37.420+0000 7fbbd118d640 1 -- 172.21.15.5:0/2599247487 wait complete. 2024-09-19T06:24:37.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 78035 8.61 MB/sec execute 361 sec latency 95.009 ms 2024-09-19T06:24:38.183 INFO:tasks.cephfs.filesystem:scrub status for tag:d7930ae0-abfb-4051-9e40-8f42ef279c61 - {'path': '/', 'tag': 'd7930ae0-abfb-4051-9e40-8f42ef279c61', 'options': 'recursive,force'} 2024-09-19T06:24:38.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:37 smithi157 ceph-mon[35346]: pgmap v672: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.2 MiB/s wr, 396 op/s 2024-09-19T06:24:38.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:37 smithi005 ceph-mon[30955]: pgmap v672: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.2 MiB/s wr, 396 op/s 2024-09-19T06:24:38.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:37 smithi049 ceph-mon[34640]: pgmap v672: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.2 MiB/s wr, 396 op/s 2024-09-19T06:24:38.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 78194 8.61 MB/sec execute 362 sec latency 78.243 ms 2024-09-19T06:24:39.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 78349 8.61 MB/sec execute 363 sec latency 70.018 ms 2024-09-19T06:24:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.797+0000 7f576ed6f640 1 -- 172.21.15.5:0/475811351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57681009d0 msgr2=0x7f5768100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.798+0000 7f576ed6f640 1 --2- 172.21.15.5:0/475811351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57681009d0 0x7f5768100e30 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5758009930 tx=0x7f575802a420 comp rx=0 tx=0).stop 2024-09-19T06:24:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.798+0000 7f576ed6f640 1 -- 172.21.15.5:0/475811351 shutdown_connections 2024-09-19T06:24:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.798+0000 7f576ed6f640 1 --2- 172.21.15.5:0/475811351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5768101370 0x7f576810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.798+0000 7f576ed6f640 1 --2- 172.21.15.5:0/475811351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57681009d0 0x7f5768100e30 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.798+0000 7f576ed6f640 1 --2- 172.21.15.5:0/475811351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57681069e0 0x7f5768106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.798+0000 7f576ed6f640 1 -- 172.21.15.5:0/475811351 >> 172.21.15.5:0/475811351 conn(0x7f57680fc6b0 msgr2=0x7f57680fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.798+0000 7f576ed6f640 1 -- 172.21.15.5:0/475811351 shutdown_connections 2024-09-19T06:24:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.798+0000 7f576ed6f640 1 -- 172.21.15.5:0/475811351 wait complete. 2024-09-19T06:24:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.799+0000 7f576ed6f640 1 Processor -- start 2024-09-19T06:24:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.799+0000 7f576ed6f640 1 -- start start 2024-09-19T06:24:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576ed6f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57681009d0 0x7f5768197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576ed6f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5768101370 0x7f57681977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576ed6f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57681069e0 0x7f576819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576ed6f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5768110d20 con 0x7f57681069e0 2024-09-19T06:24:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576ed6f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5768110ba0 con 0x7f5768101370 2024-09-19T06:24:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576ed6f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5768110ea0 con 0x7f57681009d0 2024-09-19T06:24:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576cb0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57681009d0 0x7f5768197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576cb0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57681009d0 0x7f5768197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50204/0 (socket says 172.21.15.5:50204) 2024-09-19T06:24:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576cb0b640 1 -- 172.21.15.5:0/3479687644 learned_addr learned my addr 172.21.15.5:0/3479687644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f575ffff640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5768101370 0x7f57681977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576d30c640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57681069e0 0x7f576819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576cb0b640 1 -- 172.21.15.5:0/3479687644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5768101370 msgr2=0x7f57681977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576cb0b640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5768101370 0x7f57681977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576cb0b640 1 -- 172.21.15.5:0/3479687644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57681069e0 msgr2=0x7f576819bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576cb0b640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57681069e0 0x7f576819bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.800+0000 7f576cb0b640 1 -- 172.21.15.5:0/3479687644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576819c280 con 0x7f57681009d0 2024-09-19T06:24:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.801+0000 7f576cb0b640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57681009d0 0x7f5768197290 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5764009fa0 tx=0x7f576400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.801+0000 7f575dffb640 1 -- 172.21.15.5:0/3479687644 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576400ce10 con 0x7f57681009d0 2024-09-19T06:24:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.801+0000 7f575dffb640 1 -- 172.21.15.5:0/3479687644 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57640035f0 con 0x7f57681009d0 2024-09-19T06:24:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.801+0000 7f575dffb640 1 -- 172.21.15.5:0/3479687644 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764011810 con 0x7f57681009d0 2024-09-19T06:24:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.801+0000 7f576ed6f640 1 -- 172.21.15.5:0/3479687644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576819c570 con 0x7f57681009d0 2024-09-19T06:24:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.801+0000 7f576ed6f640 1 -- 172.21.15.5:0/3479687644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576806a230 con 0x7f57681009d0 2024-09-19T06:24:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.805+0000 7f5771be2740 1 Processor -- start 2024-09-19T06:24:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.805+0000 7f5771be2740 1 -- start start 2024-09-19T06:24:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f5771be2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c5554af0 0x5571c5554ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f5771be2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c5555410 0x5571c555f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f5771be2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555fcb0 0x5571c55620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f5771be2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571c54b92f0 con 0x5571c5555410 2024-09-19T06:24:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f5771be2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5571c5461270 con 0x5571c555fcb0 2024-09-19T06:24:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f5771be2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5571c54b52b0 con 0x5571c5554af0 2024-09-19T06:24:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f576d30c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c5554af0 0x5571c5554ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f575ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555fcb0 0x5571c55620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f575ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555fcb0 0x5571c55620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37508/0 (socket says 172.21.15.5:37508) 2024-09-19T06:24:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f575ffff640 1 -- 172.21.15.5:0/446965807 learned_addr learned my addr 172.21.15.5:0/446965807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f575ffff640 1 -- 172.21.15.5:0/446965807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c5554af0 msgr2=0x5571c5554ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f575ffff640 1 --2- 172.21.15.5:0/446965807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c5554af0 0x5571c5554ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f575ffff640 1 -- 172.21.15.5:0/446965807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c5555410 msgr2=0x5571c555f770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:24:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f575ffff640 1 --2- 172.21.15.5:0/446965807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c5555410 0x5571c555f770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.806+0000 7f575ffff640 1 -- 172.21.15.5:0/446965807 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c5460340 con 0x5571c555fcb0 2024-09-19T06:24:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.807+0000 7f575ffff640 1 --2- 172.21.15.5:0/446965807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555fcb0 0x5571c55620a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5758006010 tx=0x7f5758004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7706dd0ceb1b4f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.807+0000 7f574effd640 1 -- 172.21.15.5:0/446965807 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758035040 con 0x5571c555fcb0 2024-09-19T06:24:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.807+0000 7f574effd640 1 -- 172.21.15.5:0/446965807 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57580054f0 con 0x5571c555fcb0 2024-09-19T06:24:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.807+0000 7f574effd640 1 -- 172.21.15.5:0/446965807 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57580035f0 con 0x5571c555fcb0 2024-09-19T06:24:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.808+0000 7f5771be2740 1 -- 172.21.15.5:0/446965807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555fcb0 msgr2=0x5571c55620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.808+0000 7f5771be2740 1 --2- 172.21.15.5:0/446965807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555fcb0 0x5571c55620a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5758006010 tx=0x7f5758004040 comp rx=0 tx=0).stop 2024-09-19T06:24:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.809+0000 7f5771be2740 1 -- 172.21.15.5:0/446965807 shutdown_connections 2024-09-19T06:24:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.809+0000 7f5771be2740 1 --2- 172.21.15.5:0/446965807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555fcb0 0x5571c55620a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.809+0000 7f5771be2740 1 --2- 172.21.15.5:0/446965807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c5555410 0x5571c555f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.809+0000 7f5771be2740 1 --2- 172.21.15.5:0/446965807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c5554af0 0x5571c5554ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.809+0000 7f5771be2740 1 -- 172.21.15.5:0/446965807 >> 172.21.15.5:0/446965807 conn(0x5571c5551060 msgr2=0x5571c5551440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.809+0000 7f5771be2740 1 -- 172.21.15.5:0/446965807 shutdown_connections 2024-09-19T06:24:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.809+0000 7f575dffb640 1 -- 172.21.15.5:0/3479687644 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f57640037a0 con 0x7f57681009d0 2024-09-19T06:24:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.810+0000 7f5771be2740 1 -- 172.21.15.5:0/446965807 wait complete. 2024-09-19T06:24:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.810+0000 7f5771be2740 1 -- start start 2024-09-19T06:24:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.811+0000 7f5771be2740 1 Processor -- start 2024-09-19T06:24:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f5771be2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c5554af0 0x5571c555cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f5771be2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c555d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f5771be2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555d7b0 0x5571c5574540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f5771be2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571c5562880 con 0x5571c555fcb0 2024-09-19T06:24:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f5771be2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5571c5442290 con 0x5571c555d7b0 2024-09-19T06:24:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f5771be2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5571c5562a00 con 0x5571c5554af0 2024-09-19T06:24:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f575ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555d7b0 0x5571c5574540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f575ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555d7b0 0x5571c5574540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37520/0 (socket says 172.21.15.5:37520) 2024-09-19T06:24:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f575ffff640 1 -- 172.21.15.5:0/1735059711 learned_addr learned my addr 172.21.15.5:0/1735059711 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f5771be2740 10 client.0 _ll_register_callbacks cb 0x5571c5473140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f576cb0b640 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c555d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f575ffff640 1 -- 172.21.15.5:0/1735059711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c5554af0 msgr2=0x5571c555cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f576d30c640 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c5554af0 0x5571c555cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f575dffb640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57500a3360 0x7f57500a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f575ffff640 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c5554af0 0x5571c555cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f575ffff640 1 -- 172.21.15.5:0/1735059711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 msgr2=0x5571c555d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f575ffff640 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c555d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.812+0000 7f575ffff640 1 -- 172.21.15.5:0/1735059711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c51cc210 con 0x5571c555d7b0 2024-09-19T06:24:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.813+0000 7f576d30c640 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c5554af0 0x5571c555cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:24:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.813+0000 7f575dffb640 1 -- 172.21.15.5:0/3479687644 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f57640cfdf0 con 0x7f57681009d0 2024-09-19T06:24:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.813+0000 7f575ffff640 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555d7b0 0x5571c5574540 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5758009900 tx=0x7f5758004520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1008c73b207d1a27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.813+0000 7f576cb0b640 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c555d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:24:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.813+0000 7f5747fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:24:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.813+0000 7f576d30c640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57500a3360 0x7f57500a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.813+0000 7f5747fff640 1 -- 172.21.15.5:0/1735059711 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758052060 con 0x5571c555d7b0 2024-09-19T06:24:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.814+0000 7f5771be2740 1 -- 172.21.15.5:0/1735059711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57580054f0 con 0x5571c555d7b0 2024-09-19T06:24:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.814+0000 7f576d30c640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57500a3360 0x7f57500a5820 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5760009fd0 tx=0x7f5760014040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.814+0000 7f5747fff640 1 -- 172.21.15.5:0/1735059711 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f5758004cc0 con 0x5571c555d7b0 2024-09-19T06:24:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.814+0000 7f5747fff640 1 -- 172.21.15.5:0/1735059711 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f5758035050 con 0x5571c555d7b0 2024-09-19T06:24:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.815+0000 7f5771be2740 10 client.30137 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.815+0000 7f5771be2740 1 -- 172.21.15.5:0/1735059711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5571c5298620 con 0x5571c555d7b0 2024-09-19T06:24:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.815+0000 7f5747fff640 1 -- 172.21.15.5:0/1735059711 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57580054f0 con 0x5571c555d7b0 2024-09-19T06:24:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.815+0000 7f5747fff640 1 -- 172.21.15.5:0/1735059711 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f5758003a80 con 0x5571c555d7b0 2024-09-19T06:24:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.816+0000 7f5771be2740 10 client.30137 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.816+0000 7f5771be2740 10 client.30137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.816+0000 7f5771be2740 20 client.30137 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.816+0000 7f5771be2740 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5571c55789f0 0x5571c557ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.816+0000 7f576cb0b640 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5571c55789f0 0x5571c557ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.816+0000 7f5771be2740 4 client.30137 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:24:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.816+0000 7f5771be2740 1 -- 172.21.15.5:0/1735059711 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571c54c6a20 con 0x5571c55789f0 2024-09-19T06:24:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.819+0000 7f576cb0b640 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5571c55789f0 0x5571c557ae00 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c1ebf881ce7fe85 server_cookie=dfe7cb47d60a10f9 in_seq=0 out_seq=0 2024-09-19T06:24:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.819+0000 7f5747fff640 10 client.30137 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.820+0000 7f5747fff640 1 -- 172.21.15.5:0/1735059711 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571c54c6a20 con 0x5571c55789f0 2024-09-19T06:24:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.820+0000 7f5747fff640 10 client.30137 handle_command_reply: tid=0 2024-09-19T06:24:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.820+0000 7f5771be2740 1 client.30137 shutdown 2024-09-19T06:24:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.821+0000 7f5771be2740 1 -- 172.21.15.5:0/1735059711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555d7b0 msgr2=0x5571c5574540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.821+0000 7f5771be2740 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555d7b0 0x5571c5574540 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5758009900 tx=0x7f5758004520 comp rx=0 tx=0).stop 2024-09-19T06:24:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.821+0000 7f5771be2740 1 -- 172.21.15.5:0/1735059711 shutdown_connections 2024-09-19T06:24:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.821+0000 7f5771be2740 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5571c55789f0 0x5571c557ae00 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.821+0000 7f5771be2740 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c555d7b0 0x5571c5574540 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.821+0000 7f5771be2740 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c555d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.821+0000 7f5771be2740 1 --2- 172.21.15.5:0/1735059711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c5554af0 0x5571c555cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.821+0000 7f5771be2740 1 -- 172.21.15.5:0/1735059711 >> 172.21.15.5:0/1735059711 conn(0x5571c5551060 msgr2=0x5571c5556dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.821+0000 7f5747fff640 0 client.30137 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.821+0000 7f5771be2740 1 -- 172.21.15.5:0/1735059711 shutdown_connections 2024-09-19T06:24:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.821+0000 7f5771be2740 1 -- 172.21.15.5:0/1735059711 wait complete. 2024-09-19T06:24:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.822+0000 7f5771be2740 20 client.30137 trim_cache size 0 max 16384 2024-09-19T06:24:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f5771be2740 1 Processor -- start 2024-09-19T06:24:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f5771be2740 1 -- start start 2024-09-19T06:24:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f5771be2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 0x5571c5574530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f5771be2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c555d7b0 0x5571c5559fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f5771be2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c555a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f5771be2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57681ab670 con 0x5571c555fcb0 2024-09-19T06:24:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f5771be2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f57681ad210 con 0x5571c5554af0 2024-09-19T06:24:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f5771be2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57681abd40 con 0x5571c555d7b0 2024-09-19T06:24:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f576d30c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c555a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f576cb0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 0x5571c5574530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f576cb0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 0x5571c5574530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37528/0 (socket says 172.21.15.5:37528) 2024-09-19T06:24:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f575ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c555d7b0 0x5571c5559fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.847+0000 7f576cb0b640 1 -- 172.21.15.5:0/4099744372 learned_addr learned my addr 172.21.15.5:0/4099744372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.848+0000 7f576cb0b640 1 -- 172.21.15.5:0/4099744372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c555d7b0 msgr2=0x5571c5559fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.848+0000 7f576cb0b640 1 --2- 172.21.15.5:0/4099744372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c555d7b0 0x5571c5559fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.848+0000 7f576cb0b640 1 -- 172.21.15.5:0/4099744372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 msgr2=0x5571c555a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.848+0000 7f576cb0b640 1 --2- 172.21.15.5:0/4099744372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c555a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.848+0000 7f576cb0b640 1 -- 172.21.15.5:0/4099744372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c5555240 con 0x5571c5554af0 2024-09-19T06:24:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.848+0000 7f576d30c640 1 --2- 172.21.15.5:0/4099744372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c555a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.848+0000 7f576cb0b640 1 --2- 172.21.15.5:0/4099744372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 0x5571c5574530 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5764000c00 tx=0x7f57640d21e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b39162104d378fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.848+0000 7f574e7fc640 1 -- 172.21.15.5:0/4099744372 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576409c160 con 0x5571c5554af0 2024-09-19T06:24:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.848+0000 7f574e7fc640 1 -- 172.21.15.5:0/4099744372 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57640a0510 con 0x5571c5554af0 2024-09-19T06:24:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.848+0000 7f574e7fc640 1 -- 172.21.15.5:0/4099744372 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57640bc0c0 con 0x5571c5554af0 2024-09-19T06:24:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.849+0000 7f5771be2740 1 -- 172.21.15.5:0/4099744372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 msgr2=0x5571c5574530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.849+0000 7f5771be2740 1 --2- 172.21.15.5:0/4099744372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 0x5571c5574530 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5764000c00 tx=0x7f57640d21e0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.849+0000 7f5771be2740 1 -- 172.21.15.5:0/4099744372 shutdown_connections 2024-09-19T06:24:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.850+0000 7f5771be2740 1 --2- 172.21.15.5:0/4099744372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c555a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.850+0000 7f5771be2740 1 --2- 172.21.15.5:0/4099744372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c555d7b0 0x5571c5559fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.850+0000 7f5771be2740 1 --2- 172.21.15.5:0/4099744372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 0x5571c5574530 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.850+0000 7f5771be2740 1 -- 172.21.15.5:0/4099744372 >> 172.21.15.5:0/4099744372 conn(0x5571c5551060 msgr2=0x5571c555edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.850+0000 7f5771be2740 1 -- 172.21.15.5:0/4099744372 shutdown_connections 2024-09-19T06:24:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.850+0000 7f5771be2740 1 -- 172.21.15.5:0/4099744372 wait complete. 2024-09-19T06:24:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.850+0000 7f5771be2740 1 -- start start 2024-09-19T06:24:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.850+0000 7f5771be2740 1 Processor -- start 2024-09-19T06:24:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f5771be2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 0x5571c557dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f5771be2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c555d7b0 0x5571c557e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f5771be2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c55db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f5771be2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57681acb40 con 0x5571c555fcb0 2024-09-19T06:24:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f5771be2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f57681ac440 con 0x5571c5554af0 2024-09-19T06:24:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f5771be2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57681a9670 con 0x5571c555d7b0 2024-09-19T06:24:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f5771be2740 10 client.0 _ll_register_callbacks cb 0x7f5760009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f576cb0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 0x5571c557dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f575ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c555d7b0 0x5571c557e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f575ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c555d7b0 0x5571c557e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50238/0 (socket says 172.21.15.5:50238) 2024-09-19T06:24:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f575ffff640 1 -- 172.21.15.5:0/2362044905 learned_addr learned my addr 172.21.15.5:0/2362044905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f576cb0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 0x5571c557dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37532/0 (socket says 172.21.15.5:37532) 2024-09-19T06:24:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.851+0000 7f576d30c640 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c55db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.852+0000 7f575ffff640 1 -- 172.21.15.5:0/2362044905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 msgr2=0x5571c557dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.852+0000 7f575ffff640 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 0x5571c557dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.852+0000 7f575ffff640 1 -- 172.21.15.5:0/2362044905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 msgr2=0x5571c55db5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.852+0000 7f575ffff640 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c55db5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.852+0000 7f575ffff640 1 -- 172.21.15.5:0/2362044905 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c55dbc70 con 0x5571c555d7b0 2024-09-19T06:24:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.852+0000 7f576d30c640 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c55db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.852+0000 7f575ffff640 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c555d7b0 0x5571c557e160 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5758009930 tx=0x7f5758032e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4444d035f2711e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.852+0000 7f5747fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:24:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.852+0000 7f5747fff640 1 -- 172.21.15.5:0/2362044905 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758052070 con 0x5571c555d7b0 2024-09-19T06:24:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.852+0000 7f5771be2740 1 -- 172.21.15.5:0/2362044905 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57640a0510 con 0x5571c555d7b0 2024-09-19T06:24:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.852+0000 7f5747fff640 1 -- 172.21.15.5:0/2362044905 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f5758003800 con 0x5571c555d7b0 2024-09-19T06:24:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.853+0000 7f5747fff640 1 -- 172.21.15.5:0/2362044905 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f5758003a00 con 0x5571c555d7b0 2024-09-19T06:24:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.853+0000 7f5771be2740 10 client.30064 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.853+0000 7f5771be2740 1 -- 172.21.15.5:0/2362044905 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5571c55e3030 con 0x5571c555d7b0 2024-09-19T06:24:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.853+0000 7f5747fff640 1 -- 172.21.15.5:0/2362044905 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57640a0510 con 0x5571c555d7b0 2024-09-19T06:24:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.853+0000 7f5747fff640 1 -- 172.21.15.5:0/2362044905 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f575803c440 con 0x5571c555d7b0 2024-09-19T06:24:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.853+0000 7f5771be2740 10 client.30064 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.853+0000 7f5771be2740 10 client.30064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.853+0000 7f5771be2740 20 client.30064 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.853+0000 7f5771be2740 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5571c55e34c0 0x5571c55e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.853+0000 7f5771be2740 4 client.30064 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:24:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.853+0000 7f5771be2740 1 -- 172.21.15.5:0/2362044905 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5571c54c0d70 con 0x5571c55e34c0 2024-09-19T06:24:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.853+0000 7f576cb0b640 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5571c55e34c0 0x5571c55e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.854+0000 7f576cb0b640 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5571c55e34c0 0x5571c55e58e0 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fef86d8cd3f6ebb server_cookie=e54d3478ca2e91ac in_seq=0 out_seq=0 2024-09-19T06:24:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.854+0000 7f5747fff640 10 client.30064 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.928+0000 7f5747fff640 1 -- 172.21.15.5:0/2362044905 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5571c54c0d70 con 0x5571c55e34c0 2024-09-19T06:24:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.928+0000 7f5747fff640 10 client.30064 handle_command_reply: tid=0 2024-09-19T06:24:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.928+0000 7f5771be2740 1 client.30064 shutdown 2024-09-19T06:24:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.928+0000 7f5771be2740 1 -- 172.21.15.5:0/2362044905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c555d7b0 msgr2=0x5571c557e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.928+0000 7f5771be2740 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c555d7b0 0x5571c557e160 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5758009930 tx=0x7f5758032e30 comp rx=0 tx=0).stop 2024-09-19T06:24:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.928+0000 7f5771be2740 1 -- 172.21.15.5:0/2362044905 shutdown_connections 2024-09-19T06:24:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.928+0000 7f5771be2740 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5571c55e34c0 0x5571c55e58e0 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.929+0000 7f5771be2740 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c555fcb0 0x5571c55db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.929+0000 7f5747fff640 0 client.30064 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.929+0000 7f5771be2740 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571c555d7b0 0x5571c557e160 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.929+0000 7f5771be2740 1 --2- 172.21.15.5:0/2362044905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571c5554af0 0x5571c557dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.929+0000 7f5771be2740 1 -- 172.21.15.5:0/2362044905 >> 172.21.15.5:0/2362044905 conn(0x5571c5551060 msgr2=0x5571c554f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.929+0000 7f5771be2740 1 -- 172.21.15.5:0/2362044905 shutdown_connections 2024-09-19T06:24:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.929+0000 7f5771be2740 1 -- 172.21.15.5:0/2362044905 wait complete. 2024-09-19T06:24:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.929+0000 7f5771be2740 20 client.30064 trim_cache size 0 max 16384 2024-09-19T06:24:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:24:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:24:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:24:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:24:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.930+0000 7f576ed6f640 1 -- 172.21.15.5:0/3479687644 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57500a3360 msgr2=0x7f57500a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.930+0000 7f576ed6f640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57500a3360 0x7f57500a5820 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5760009fd0 tx=0x7f5760014040 comp rx=0 tx=0).stop 2024-09-19T06:24:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.930+0000 7f576ed6f640 1 -- 172.21.15.5:0/3479687644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57681009d0 msgr2=0x7f5768197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.930+0000 7f576ed6f640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57681009d0 0x7f5768197290 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5764009fa0 tx=0x7f576400eef0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.930+0000 7f576ed6f640 1 -- 172.21.15.5:0/3479687644 shutdown_connections 2024-09-19T06:24:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.930+0000 7f576ed6f640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57681069e0 0x7f576819bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.930+0000 7f576ed6f640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5768101370 0x7f57681977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.930+0000 7f576ed6f640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f57500a3360 0x7f57500a5820 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.930+0000 7f576ed6f640 1 --2- 172.21.15.5:0/3479687644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57681009d0 0x7f5768197290 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.930+0000 7f576ed6f640 1 -- 172.21.15.5:0/3479687644 >> 172.21.15.5:0/3479687644 conn(0x7f57680fc6b0 msgr2=0x7f57680fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.930+0000 7f576ed6f640 1 -- 172.21.15.5:0/3479687644 shutdown_connections 2024-09-19T06:24:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:39.930+0000 7f576ed6f640 1 -- 172.21.15.5:0/3479687644 wait complete. 2024-09-19T06:24:40.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:39 smithi157 ceph-mon[35346]: pgmap v673: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 267 op/s 2024-09-19T06:24:40.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:39 smithi005 ceph-mon[30955]: pgmap v673: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 267 op/s 2024-09-19T06:24:40.400 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:39 smithi049 ceph-mon[34640]: pgmap v673: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 267 op/s 2024-09-19T06:24:40.591 INFO:tasks.workunit.client.0.smithi005.stdout: 10 78508 8.61 MB/sec execute 364 sec latency 134.063 ms 2024-09-19T06:24:41.591 INFO:tasks.workunit.client.0.smithi005.stdout: 10 78648 8.59 MB/sec execute 365 sec latency 136.525 ms 2024-09-19T06:24:42.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:41 smithi157 ceph-mon[35346]: pgmap v674: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 267 op/s 2024-09-19T06:24:42.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:41 smithi005 ceph-mon[30955]: pgmap v674: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 267 op/s 2024-09-19T06:24:42.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:41 smithi049 ceph-mon[34640]: pgmap v674: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 267 op/s 2024-09-19T06:24:42.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 78801 8.57 MB/sec execute 366 sec latency 64.221 ms 2024-09-19T06:24:43.591 INFO:tasks.workunit.client.0.smithi005.stdout: 10 78982 8.58 MB/sec execute 367 sec latency 107.485 ms 2024-09-19T06:24:44.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:43 smithi157 ceph-mon[35346]: pgmap v675: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.0 MiB/s wr, 411 op/s 2024-09-19T06:24:44.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:43 smithi005 ceph-mon[30955]: pgmap v675: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.0 MiB/s wr, 411 op/s 2024-09-19T06:24:44.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:43 smithi049 ceph-mon[34640]: pgmap v675: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.0 MiB/s wr, 411 op/s 2024-09-19T06:24:44.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 79142 8.57 MB/sec execute 368 sec latency 226.688 ms 2024-09-19T06:24:45.212 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:24:45.212 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:24:45.212 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:24:45.212 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:24:45.212 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:24:45.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:24:45.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:24:45.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:24:45.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:24:45.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:24:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:24:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:24:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:24:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:24:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:24:45.591 INFO:tasks.workunit.client.0.smithi005.stdout: 10 79313 8.58 MB/sec execute 369 sec latency 69.302 ms 2024-09-19T06:24:46.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:45 smithi157 ceph-mon[35346]: pgmap v676: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 304 op/s 2024-09-19T06:24:46.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:45 smithi005 ceph-mon[30955]: pgmap v676: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 304 op/s 2024-09-19T06:24:46.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:45 smithi049 ceph-mon[34640]: pgmap v676: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 304 op/s 2024-09-19T06:24:46.591 INFO:tasks.workunit.client.0.smithi005.stdout: 10 79474 8.58 MB/sec execute 370 sec latency 60.321 ms 2024-09-19T06:24:47.591 INFO:tasks.workunit.client.0.smithi005.stdout: 10 79642 8.59 MB/sec execute 371 sec latency 70.476 ms 2024-09-19T06:24:48.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:47 smithi157 ceph-mon[35346]: pgmap v677: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 424 op/s 2024-09-19T06:24:48.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:47 smithi005 ceph-mon[30955]: pgmap v677: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 424 op/s 2024-09-19T06:24:48.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:47 smithi049 ceph-mon[34640]: pgmap v677: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 424 op/s 2024-09-19T06:24:48.591 INFO:tasks.workunit.client.0.smithi005.stdout: 10 79835 8.60 MB/sec execute 372 sec latency 85.406 ms 2024-09-19T06:24:49.591 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80042 8.61 MB/sec execute 373 sec latency 63.565 ms 2024-09-19T06:24:50.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:49 smithi157 ceph-mon[35346]: pgmap v678: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-09-19T06:24:50.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:49 smithi005 ceph-mon[30955]: pgmap v678: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-09-19T06:24:50.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:49 smithi049 ceph-mon[34640]: pgmap v678: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-09-19T06:24:50.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80159 8.60 MB/sec execute 374 sec latency 74.933 ms 2024-09-19T06:24:51.591 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80292 8.60 MB/sec execute 375 sec latency 79.785 ms 2024-09-19T06:24:52.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:52 smithi157 ceph-mon[35346]: pgmap v679: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-09-19T06:24:52.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:52 smithi005 ceph-mon[30955]: pgmap v679: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-09-19T06:24:52.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:52 smithi049 ceph-mon[34640]: pgmap v679: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-09-19T06:24:52.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80467 8.62 MB/sec execute 376 sec latency 86.830 ms 2024-09-19T06:24:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.073+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3831121519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34c8108d30 msgr2=0x7f34c810cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.073+0000 7f34cdf4b640 1 --2- 172.21.15.5:0/3831121519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34c8108d30 0x7f34c810cd70 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009860 tx=0x7f34b802a360 comp rx=0 tx=0).stop 2024-09-19T06:24:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.074+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3831121519 shutdown_connections 2024-09-19T06:24:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.074+0000 7f34cdf4b640 1 --2- 172.21.15.5:0/3831121519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34c810d4d0 0x7f34c8115020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.074+0000 7f34cdf4b640 1 --2- 172.21.15.5:0/3831121519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34c8108d30 0x7f34c810cd70 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.074+0000 7f34cdf4b640 1 --2- 172.21.15.5:0/3831121519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8108380 0x7f34c8108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.074+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3831121519 >> 172.21.15.5:0/3831121519 conn(0x7f34c8077ae0 msgr2=0x7f34c8079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.074+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3831121519 shutdown_connections 2024-09-19T06:24:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.074+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3831121519 wait complete. 2024-09-19T06:24:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.074+0000 7f34cdf4b640 1 Processor -- start 2024-09-19T06:24:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34cdf4b640 1 -- start start 2024-09-19T06:24:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34cdf4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8108380 0x7f34c81a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34cdf4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34c810d4d0 0x7f34c81a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34cdf4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34c819e750 0x7f34c819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34cdf4b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34c8118380 con 0x7f34c819e750 2024-09-19T06:24:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34cdf4b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f34c8118200 con 0x7f34c810d4d0 2024-09-19T06:24:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34cdf4b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34c8118500 con 0x7f34c8108380 2024-09-19T06:24:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34c819e750 0x7f34c819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34c819e750 0x7f34c819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49584/0 (socket says 172.21.15.5:49584) 2024-09-19T06:24:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34c7fff640 1 -- 172.21.15.5:0/3098996847 learned_addr learned my addr 172.21.15.5:0/3098996847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34c77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8108380 0x7f34c81a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34beffd640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34c810d4d0 0x7f34c81a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34c7fff640 1 -- 172.21.15.5:0/3098996847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8108380 msgr2=0x7f34c81a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34c7fff640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8108380 0x7f34c81a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.075+0000 7f34c7fff640 1 -- 172.21.15.5:0/3098996847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34c810d4d0 msgr2=0x7f34c81a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.076+0000 7f34c7fff640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34c810d4d0 0x7f34c81a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.076+0000 7f34c7fff640 1 -- 172.21.15.5:0/3098996847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c819f350 con 0x7f34c819e750 2024-09-19T06:24:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.076+0000 7f34beffd640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34c810d4d0 0x7f34c81a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.076+0000 7f34c7fff640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34c819e750 0x7f34c819ebd0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f34c000e6f0 tx=0x7f34c000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.076+0000 7f34c57fa640 1 -- 172.21.15.5:0/3098996847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c000cab0 con 0x7f34c819e750 2024-09-19T06:24:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.076+0000 7f34c57fa640 1 -- 172.21.15.5:0/3098996847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34c00054d0 con 0x7f34c819e750 2024-09-19T06:24:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.076+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3098996847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c81ab430 con 0x7f34c819e750 2024-09-19T06:24:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.076+0000 7f34c57fa640 1 -- 172.21.15.5:0/3098996847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0010430 con 0x7f34c819e750 2024-09-19T06:24:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.077+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3098996847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c81ab7e0 con 0x7f34c819e750 2024-09-19T06:24:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.080+0000 7f34c57fa640 1 -- 172.21.15.5:0/3098996847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f34c0004ee0 con 0x7f34c819e750 2024-09-19T06:24:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.080+0000 7f34d0dc1740 1 Processor -- start 2024-09-19T06:24:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.081+0000 7f34d0dc1740 1 -- start start 2024-09-19T06:24:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.081+0000 7f34d0dc1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5caf0 0x557666c5ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.081+0000 7f34d0dc1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 0x557666c67770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.081+0000 7f34c77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 0x557666c67770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.081+0000 7f34c77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 0x557666c67770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43462/0 (socket says 172.21.15.5:43462) 2024-09-19T06:24:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.081+0000 7f34beffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5caf0 0x557666c5ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.081+0000 7f34d0dc1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c67cb0 0x557666c6a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.081+0000 7f34d0dc1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557666bc12f0 con 0x557666c67cb0 2024-09-19T06:24:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.081+0000 7f34d0dc1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557666b69270 con 0x557666c5d410 2024-09-19T06:24:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.081+0000 7f34d0dc1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557666bbd2b0 con 0x557666c5caf0 2024-09-19T06:24:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.081+0000 7f34c77fe640 1 -- 172.21.15.5:0/794373145 learned_addr learned my addr 172.21.15.5:0/794373145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.082+0000 7f34c57fa640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f34a80a3430 0x7f34a80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.082+0000 7f34c57fa640 1 -- 172.21.15.5:0/3098996847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f34c00cfd70 con 0x7f34c819e750 2024-09-19T06:24:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.082+0000 7f34beffd640 1 --2- 172.21.15.5:0/794373145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c67cb0 0x557666c6a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.082+0000 7f34c77fe640 1 -- 172.21.15.5:0/794373145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5caf0 msgr2=0x557666c5ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.082+0000 7f34c77fe640 1 --2- 172.21.15.5:0/794373145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5caf0 0x557666c5ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.082+0000 7f34c77fe640 1 -- 172.21.15.5:0/794373145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c67cb0 msgr2=0x557666c6a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.082+0000 7f34c77fe640 1 --2- 172.21.15.5:0/794373145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c67cb0 0x557666c6a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.082+0000 7f34c77fe640 1 -- 172.21.15.5:0/794373145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557666b68340 con 0x557666c5d410 2024-09-19T06:24:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.083+0000 7f34beffd640 1 --2- 172.21.15.5:0/794373145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c67cb0 0x557666c6a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:24:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.083+0000 7f34c77fe640 1 --2- 172.21.15.5:0/794373145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 0x557666c67770 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f34b00114f0 tx=0x7f34b00119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=994a341e48211519 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.083+0000 7f34bdffb640 1 -- 172.21.15.5:0/794373145 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b001e070 con 0x557666c5d410 2024-09-19T06:24:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.083+0000 7f34bdffb640 1 -- 172.21.15.5:0/794373145 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34b000fad0 con 0x557666c5d410 2024-09-19T06:24:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.083+0000 7f34bdffb640 1 -- 172.21.15.5:0/794373145 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0019b10 con 0x557666c5d410 2024-09-19T06:24:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.084+0000 7f34c7fff640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f34a80a3430 0x7f34a80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.084+0000 7f34d0dc1740 1 -- 172.21.15.5:0/794373145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 msgr2=0x557666c67770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.084+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/794373145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 0x557666c67770 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f34b00114f0 tx=0x7f34b00119b0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.085+0000 7f34d0dc1740 1 -- 172.21.15.5:0/794373145 shutdown_connections 2024-09-19T06:24:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.085+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/794373145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c67cb0 0x557666c6a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.085+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/794373145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 0x557666c67770 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.085+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/794373145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5caf0 0x557666c5ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.085+0000 7f34d0dc1740 1 -- 172.21.15.5:0/794373145 >> 172.21.15.5:0/794373145 conn(0x557666c59060 msgr2=0x557666c59440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.085+0000 7f34c7fff640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f34a80a3430 0x7f34a80a58f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f34c009bd30 tx=0x7f34c000ccd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.085+0000 7f34d0dc1740 1 -- 172.21.15.5:0/794373145 shutdown_connections 2024-09-19T06:24:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.085+0000 7f34d0dc1740 1 -- 172.21.15.5:0/794373145 wait complete. 2024-09-19T06:24:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.085+0000 7f34d0dc1740 1 -- start start 2024-09-19T06:24:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.086+0000 7f34d0dc1740 1 Processor -- start 2024-09-19T06:24:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.086+0000 7f34d0dc1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5caf0 0x557666c64bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.086+0000 7f34d0dc1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5d410 0x557666c65100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.086+0000 7f34beffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5caf0 0x557666c64bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.086+0000 7f34beffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5caf0 0x557666c64bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40562/0 (socket says 172.21.15.5:40562) 2024-09-19T06:24:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.086+0000 7f34beffd640 1 -- 172.21.15.5:0/1368322711 learned_addr learned my addr 172.21.15.5:0/1368322711 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.086+0000 7f34c77fe640 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5d410 0x557666c65100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.086+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c67cb0 0x557666c65640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.087+0000 7f34d0dc1740 1 -- 172.21.15.5:0/1368322711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557666c6a880 con 0x557666c5d410 2024-09-19T06:24:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.087+0000 7f34d0dc1740 1 -- 172.21.15.5:0/1368322711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557666b4a290 con 0x557666c67cb0 2024-09-19T06:24:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.087+0000 7f34d0dc1740 1 -- 172.21.15.5:0/1368322711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557666c6aa00 con 0x557666c5caf0 2024-09-19T06:24:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.087+0000 7f34d0dc1740 10 client.0 _ll_register_callbacks cb 0x557666b7b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.087+0000 7f34beffd640 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c67cb0 0x557666c65640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.087+0000 7f34c77fe640 1 -- 172.21.15.5:0/1368322711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5caf0 msgr2=0x557666c64bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.087+0000 7f34c77fe640 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5caf0 0x557666c64bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.087+0000 7f34c77fe640 1 -- 172.21.15.5:0/1368322711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c67cb0 msgr2=0x557666c65640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.087+0000 7f34c77fe640 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c67cb0 0x557666c65640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.087+0000 7f34c77fe640 1 -- 172.21.15.5:0/1368322711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576668d4210 con 0x557666c5d410 2024-09-19T06:24:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.087+0000 7f34c77fe640 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5d410 0x557666c65100 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f34b0011eb0 tx=0x7f34b0009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ef1178f50dace2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.087+0000 7f34b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:24:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.088+0000 7f34b6ffd640 1 -- 172.21.15.5:0/1368322711 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b001e040 con 0x557666c5d410 2024-09-19T06:24:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.088+0000 7f34b6ffd640 1 -- 172.21.15.5:0/1368322711 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f34b000f720 con 0x557666c5d410 2024-09-19T06:24:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.088+0000 7f34d0dc1740 1 -- 172.21.15.5:0/1368322711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b000fad0 con 0x557666c5d410 2024-09-19T06:24:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.088+0000 7f34b6ffd640 1 -- 172.21.15.5:0/1368322711 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f34b0004a40 con 0x557666c5d410 2024-09-19T06:24:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.088+0000 7f34d0dc1740 10 client.20769 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.088+0000 7f34d0dc1740 1 -- 172.21.15.5:0/1368322711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5576669a0620 con 0x557666c5d410 2024-09-19T06:24:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.089+0000 7f34b6ffd640 1 -- 172.21.15.5:0/1368322711 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34b000fad0 con 0x557666c5d410 2024-09-19T06:24:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.089+0000 7f34b6ffd640 1 -- 172.21.15.5:0/1368322711 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f34b00050d0 con 0x557666c5d410 2024-09-19T06:24:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.089+0000 7f34d0dc1740 10 client.20769 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.089+0000 7f34d0dc1740 10 client.20769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.089+0000 7f34d0dc1740 20 client.20769 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.089+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557666c77b30 0x557666c79f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.089+0000 7f34d0dc1740 4 client.20769 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:24:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.089+0000 7f34d0dc1740 1 -- 172.21.15.5:0/1368322711 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557666bcea20 con 0x557666c77b30 2024-09-19T06:24:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.090+0000 7f34beffd640 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557666c77b30 0x557666c79f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.090+0000 7f34beffd640 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557666c77b30 0x557666c79f20 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e3ed0dde30b6171 server_cookie=6002ce28304fba4a in_seq=0 out_seq=0 2024-09-19T06:24:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.090+0000 7f34b6ffd640 10 client.20769 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34b6ffd640 1 -- 172.21.15.5:0/1368322711 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557666bcea20 con 0x557666c77b30 2024-09-19T06:24:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34b6ffd640 10 client.20769 handle_command_reply: tid=0 2024-09-19T06:24:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34d0dc1740 1 client.20769 shutdown 2024-09-19T06:24:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34d0dc1740 1 -- 172.21.15.5:0/1368322711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5d410 msgr2=0x557666c65100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5d410 0x557666c65100 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f34b0011eb0 tx=0x7f34b0009d50 comp rx=0 tx=0).stop 2024-09-19T06:24:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34d0dc1740 1 -- 172.21.15.5:0/1368322711 shutdown_connections 2024-09-19T06:24:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557666c77b30 0x557666c79f20 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c67cb0 0x557666c65640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5d410 0x557666c65100 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/1368322711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5caf0 0x557666c64bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34b6ffd640 0 client.20769 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34d0dc1740 1 -- 172.21.15.5:0/1368322711 >> 172.21.15.5:0/1368322711 conn(0x557666c59060 msgr2=0x557666c581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34d0dc1740 1 -- 172.21.15.5:0/1368322711 shutdown_connections 2024-09-19T06:24:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34d0dc1740 1 -- 172.21.15.5:0/1368322711 wait complete. 2024-09-19T06:24:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.091+0000 7f34d0dc1740 20 client.20769 trim_cache size 0 max 16384 2024-09-19T06:24:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.111+0000 7f34d0dc1740 1 Processor -- start 2024-09-19T06:24:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.111+0000 7f34d0dc1740 1 -- start start 2024-09-19T06:24:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.111+0000 7f34d0dc1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5caf0 0x557666c65470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.111+0000 7f34d0dc1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 0x557666c7f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.111+0000 7f34d0dc1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c67cb0 0x557666c7fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.111+0000 7f34d0dc1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81b3e30 con 0x557666c5caf0 2024-09-19T06:24:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.111+0000 7f34d0dc1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81b59d0 con 0x557666c5d410 2024-09-19T06:24:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.111+0000 7f34d0dc1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81b4500 con 0x557666c67cb0 2024-09-19T06:24:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.111+0000 7f34beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5caf0 0x557666c65470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.111+0000 7f34beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5caf0 0x557666c65470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49612/0 (socket says 172.21.15.5:49612) 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.111+0000 7f34beffd640 1 -- 172.21.15.5:0/2483201954 learned_addr learned my addr 172.21.15.5:0/2483201954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.111+0000 7f34c77fe640 1 --2- 172.21.15.5:0/2483201954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 0x557666c7f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.112+0000 7f34beffd640 1 --2- 172.21.15.5:0/2483201954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c67cb0 0x557666c7fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.112+0000 7f34beffd640 1 -- 172.21.15.5:0/2483201954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c67cb0 msgr2=0x557666c7fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.112+0000 7f34beffd640 1 --2- 172.21.15.5:0/2483201954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c67cb0 0x557666c7fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.112+0000 7f34beffd640 1 -- 172.21.15.5:0/2483201954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 msgr2=0x557666c7f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.112+0000 7f34beffd640 1 --2- 172.21.15.5:0/2483201954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 0x557666c7f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.112+0000 7f34beffd640 1 -- 172.21.15.5:0/2483201954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557666c6a410 con 0x557666c5caf0 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.112+0000 7f34c77fe640 1 --2- 172.21.15.5:0/2483201954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 0x557666c7f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.112+0000 7f34beffd640 1 --2- 172.21.15.5:0/2483201954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5caf0 0x557666c65470 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f34b802a880 tx=0x7f34b8030340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d4b8b288042a00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.112+0000 7f34bd7fa640 1 -- 172.21.15.5:0/2483201954 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8037040 con 0x557666c5caf0 2024-09-19T06:24:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.112+0000 7f34bd7fa640 1 -- 172.21.15.5:0/2483201954 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34b8030900 con 0x557666c5caf0 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.112+0000 7f34bd7fa640 1 -- 172.21.15.5:0/2483201954 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8032a50 con 0x557666c5caf0 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.113+0000 7f34d0dc1740 1 -- 172.21.15.5:0/2483201954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5caf0 msgr2=0x557666c65470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.113+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/2483201954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5caf0 0x557666c65470 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f34b802a880 tx=0x7f34b8030340 comp rx=0 tx=0).stop 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.113+0000 7f34d0dc1740 1 -- 172.21.15.5:0/2483201954 shutdown_connections 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.113+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/2483201954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c67cb0 0x557666c7fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.113+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/2483201954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5d410 0x557666c7f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.113+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/2483201954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c5caf0 0x557666c65470 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.113+0000 7f34d0dc1740 1 -- 172.21.15.5:0/2483201954 >> 172.21.15.5:0/2483201954 conn(0x557666c59060 msgr2=0x557666c66c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.113+0000 7f34d0dc1740 1 -- 172.21.15.5:0/2483201954 shutdown_connections 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.113+0000 7f34d0dc1740 1 -- 172.21.15.5:0/2483201954 wait complete. 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.114+0000 7f34d0dc1740 1 -- start start 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.114+0000 7f34d0dc1740 1 Processor -- start 2024-09-19T06:24:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.114+0000 7f34d0dc1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5caf0 0x557666c704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.114+0000 7f34d0dc1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5d410 0x557666c61a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.115+0000 7f34beffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5caf0 0x557666c704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.115+0000 7f34d0dc1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c67cb0 0x557666c62070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.115+0000 7f34d0dc1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81b5300 con 0x557666c67cb0 2024-09-19T06:24:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.115+0000 7f34d0dc1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81b4c00 con 0x557666c5caf0 2024-09-19T06:24:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.115+0000 7f34beffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5caf0 0x557666c704c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43492/0 (socket says 172.21.15.5:43492) 2024-09-19T06:24:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.115+0000 7f34beffd640 1 -- 172.21.15.5:0/233304562 learned_addr learned my addr 172.21.15.5:0/233304562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:24:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.115+0000 7f34d0dc1740 1 -- 172.21.15.5:0/233304562 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81b1e30 con 0x557666c5d410 2024-09-19T06:24:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.115+0000 7f34c77fe640 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5d410 0x557666c61a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.115+0000 7f34d0dc1740 10 client.0 _ll_register_callbacks cb 0x7f34b001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:24:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.115+0000 7f34beffd640 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c67cb0 0x557666c62070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.116+0000 7f34beffd640 1 -- 172.21.15.5:0/233304562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5d410 msgr2=0x557666c61a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.116+0000 7f34beffd640 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5d410 0x557666c61a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.116+0000 7f34beffd640 1 -- 172.21.15.5:0/233304562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c67cb0 msgr2=0x557666c62070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:24:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.116+0000 7f34beffd640 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c67cb0 0x557666c62070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.116+0000 7f34beffd640 1 -- 172.21.15.5:0/233304562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557666ce1660 con 0x557666c5caf0 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.116+0000 7f34beffd640 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5caf0 0x557666c704c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f34b802ab50 tx=0x7f34b802aae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91e6232870864468 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.116+0000 7f34b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.116+0000 7f34b6ffd640 1 -- 172.21.15.5:0/233304562 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8056060 con 0x557666c5caf0 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.116+0000 7f34b6ffd640 1 -- 172.21.15.5:0/233304562 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8550+0+0 (secure 0 0 0) 0x7f34b80329f0 con 0x557666c5caf0 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.116+0000 7f34d0dc1740 1 -- 172.21.15.5:0/233304562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b8030900 con 0x557666c5caf0 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.117+0000 7f34b6ffd640 1 -- 172.21.15.5:0/233304562 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f34b8037040 con 0x557666c5caf0 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.117+0000 7f34d0dc1740 10 client.30161 fetch_fsmap learned FSMap version 60 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.117+0000 7f34d0dc1740 1 -- 172.21.15.5:0/233304562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x557666ce2760 con 0x557666c5caf0 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.118+0000 7f34b6ffd640 1 -- 172.21.15.5:0/233304562 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34b805a930 con 0x557666c5caf0 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.118+0000 7f34b6ffd640 1 -- 172.21.15.5:0/233304562 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f34b803d040 con 0x557666c5caf0 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.118+0000 7f34d0dc1740 10 client.30161 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:24:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.118+0000 7f34d0dc1740 10 client.30161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:24:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.118+0000 7f34d0dc1740 20 client.30161 populate_metadata read hostname 'smithi005' 2024-09-19T06:24:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.118+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557666ce2dd0 0x557666ce51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:24:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.118+0000 7f34d0dc1740 4 client.30161 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:24:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.118+0000 7f34d0dc1740 1 -- 172.21.15.5:0/233304562 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f34b8030900 con 0x557666ce2dd0 2024-09-19T06:24:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.118+0000 7f34c77fe640 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557666ce2dd0 0x557666ce51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:24:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.119+0000 7f34c77fe640 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557666ce2dd0 0x557666ce51f0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf7a282c3c1d66f server_cookie=5dd2b04d6f2005bd in_seq=0 out_seq=0 2024-09-19T06:24:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.119+0000 7f34b6ffd640 10 client.30161 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.256+0000 7f34b6ffd640 1 -- 172.21.15.5:0/233304562 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f34b8030900 con 0x557666ce2dd0 2024-09-19T06:24:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.256+0000 7f34b6ffd640 10 client.30161 handle_command_reply: tid=0 2024-09-19T06:24:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.256+0000 7f34d0dc1740 1 client.30161 shutdown 2024-09-19T06:24:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.257+0000 7f34d0dc1740 1 -- 172.21.15.5:0/233304562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5caf0 msgr2=0x557666c704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.257+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5caf0 0x557666c704c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f34b802ab50 tx=0x7f34b802aae0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.257+0000 7f34d0dc1740 1 -- 172.21.15.5:0/233304562 shutdown_connections 2024-09-19T06:24:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.257+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557666ce2dd0 0x557666ce51f0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.257+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666c67cb0 0x557666c62070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.257+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557666c5d410 0x557666c61a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.257+0000 7f34d0dc1740 1 --2- 172.21.15.5:0/233304562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666c5caf0 0x557666c704c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.257+0000 7f34d0dc1740 1 -- 172.21.15.5:0/233304562 >> 172.21.15.5:0/233304562 conn(0x557666c59060 msgr2=0x557666c579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.257+0000 7f34b6ffd640 0 client.30161 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:24:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.257+0000 7f34d0dc1740 1 -- 172.21.15.5:0/233304562 shutdown_connections 2024-09-19T06:24:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.257+0000 7f34d0dc1740 1 -- 172.21.15.5:0/233304562 wait complete. 2024-09-19T06:24:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.258+0000 7f34d0dc1740 20 client.30161 trim_cache size 0 max 16384 2024-09-19T06:24:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:24:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:24:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:24:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:24:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.260+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3098996847 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f34a80a3430 msgr2=0x7f34a80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.260+0000 7f34cdf4b640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f34a80a3430 0x7f34a80a58f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f34c009bd30 tx=0x7f34c000ccd0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.260+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3098996847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34c819e750 msgr2=0x7f34c819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:24:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.260+0000 7f34cdf4b640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34c819e750 0x7f34c819ebd0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f34c000e6f0 tx=0x7f34c000ebb0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.260+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3098996847 shutdown_connections 2024-09-19T06:24:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.260+0000 7f34cdf4b640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34c819e750 0x7f34c819ebd0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.261+0000 7f34cdf4b640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34c810d4d0 0x7f34c81a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.261+0000 7f34cdf4b640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f34a80a3430 0x7f34a80a58f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.261+0000 7f34cdf4b640 1 --2- 172.21.15.5:0/3098996847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8108380 0x7f34c81a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:24:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.261+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3098996847 >> 172.21.15.5:0/3098996847 conn(0x7f34c8077ae0 msgr2=0x7f34c8078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:24:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.261+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3098996847 shutdown_connections 2024-09-19T06:24:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:24:53.261+0000 7f34cdf4b640 1 -- 172.21.15.5:0/3098996847 wait complete. 2024-09-19T06:24:53.591 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80618 8.63 MB/sec execute 377 sec latency 101.840 ms 2024-09-19T06:24:54.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:54 smithi157 ceph-mon[35346]: pgmap v680: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 376 op/s 2024-09-19T06:24:54.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:54 smithi005 ceph-mon[30955]: pgmap v680: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 376 op/s 2024-09-19T06:24:54.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:54 smithi049 ceph-mon[34640]: pgmap v680: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 376 op/s 2024-09-19T06:24:54.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80819 8.64 MB/sec execute 378 sec latency 64.904 ms 2024-09-19T06:24:55.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80935 8.63 MB/sec execute 379 sec latency 53.488 ms 2024-09-19T06:24:56.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:56 smithi157 ceph-mon[35346]: pgmap v681: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 231 op/s 2024-09-19T06:24:56.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:56 smithi005 ceph-mon[30955]: pgmap v681: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 231 op/s 2024-09-19T06:24:56.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:56 smithi049 ceph-mon[34640]: pgmap v681: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 231 op/s 2024-09-19T06:24:56.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 81039 8.60 MB/sec execute 380 sec latency 85.263 ms 2024-09-19T06:24:57.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:57 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:24:57.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:57 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:24:57.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:57 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:24:57.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 81208 8.61 MB/sec execute 381 sec latency 75.084 ms 2024-09-19T06:24:58.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:58 smithi157 ceph-mon[35346]: pgmap v682: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.6 MiB/s wr, 367 op/s 2024-09-19T06:24:58.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:58 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:24:58.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:58 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:24:58.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:24:58 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:24:58.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:58 smithi005 ceph-mon[30955]: pgmap v682: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.6 MiB/s wr, 367 op/s 2024-09-19T06:24:58.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:58 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:24:58.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:58 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:24:58.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:24:58 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:24:58.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:58 smithi049 ceph-mon[34640]: pgmap v682: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.6 MiB/s wr, 367 op/s 2024-09-19T06:24:58.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:58 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:24:58.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:58 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:24:58.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:24:58 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:24:58.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 81374 8.61 MB/sec execute 382 sec latency 102.355 ms 2024-09-19T06:24:59.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 81535 8.61 MB/sec execute 383 sec latency 162.388 ms 2024-09-19T06:25:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:00 smithi157 ceph-mon[35346]: pgmap v683: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 247 op/s 2024-09-19T06:25:00.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:00 smithi005 ceph-mon[30955]: pgmap v683: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 247 op/s 2024-09-19T06:25:00.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:00 smithi049 ceph-mon[34640]: pgmap v683: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 247 op/s 2024-09-19T06:25:00.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 81706 8.61 MB/sec execute 384 sec latency 81.331 ms 2024-09-19T06:25:01.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 81836 8.60 MB/sec execute 385 sec latency 92.527 ms 2024-09-19T06:25:02.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:02 smithi157 ceph-mon[35346]: pgmap v684: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 247 op/s 2024-09-19T06:25:02.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:02 smithi005 ceph-mon[30955]: pgmap v684: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 247 op/s 2024-09-19T06:25:02.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:02 smithi049 ceph-mon[34640]: pgmap v684: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 247 op/s 2024-09-19T06:25:02.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 81963 8.59 MB/sec execute 386 sec latency 129.395 ms 2024-09-19T06:25:03.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 82147 8.59 MB/sec execute 387 sec latency 64.187 ms 2024-09-19T06:25:04.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:04 smithi157 ceph-mon[35346]: pgmap v685: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.0 MiB/s wr, 402 op/s 2024-09-19T06:25:04.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:04 smithi005 ceph-mon[30955]: pgmap v685: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.0 MiB/s wr, 402 op/s 2024-09-19T06:25:04.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:04 smithi049 ceph-mon[34640]: pgmap v685: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.0 MiB/s wr, 402 op/s 2024-09-19T06:25:04.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 82310 8.57 MB/sec execute 388 sec latency 134.694 ms 2024-09-19T06:25:05.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 82471 8.57 MB/sec execute 389 sec latency 108.246 ms 2024-09-19T06:25:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:06 smithi157 ceph-mon[35346]: pgmap v686: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 290 op/s 2024-09-19T06:25:06.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:06 smithi157 ceph-mon[35346]: osdmap e116: 12 total, 12 up, 12 in 2024-09-19T06:25:06.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:06 smithi005 ceph-mon[30955]: pgmap v686: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 290 op/s 2024-09-19T06:25:06.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:06 smithi005 ceph-mon[30955]: osdmap e116: 12 total, 12 up, 12 in 2024-09-19T06:25:06.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:06 smithi049 ceph-mon[34640]: pgmap v686: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 290 op/s 2024-09-19T06:25:06.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:06 smithi049 ceph-mon[34640]: osdmap e116: 12 total, 12 up, 12 in 2024-09-19T06:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.397+0000 7f13dc21e640 1 -- 172.21.15.5:0/4016611123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d4101ac0 msgr2=0x7f13d4101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.397+0000 7f13dc21e640 1 --2- 172.21.15.5:0/4016611123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d4101ac0 0x7f13d4101f20 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009930 tx=0x7f13c402a420 comp rx=0 tx=0).stop 2024-09-19T06:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.397+0000 7f13dc21e640 1 -- 172.21.15.5:0/4016611123 shutdown_connections 2024-09-19T06:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.397+0000 7f13dc21e640 1 --2- 172.21.15.5:0/4016611123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d4102460 0x7f13d410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.397+0000 7f13dc21e640 1 --2- 172.21.15.5:0/4016611123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d4101ac0 0x7f13d4101f20 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.397+0000 7f13dc21e640 1 --2- 172.21.15.5:0/4016611123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13d4107ad0 0x7f13d4107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.397+0000 7f13dc21e640 1 -- 172.21.15.5:0/4016611123 >> 172.21.15.5:0/4016611123 conn(0x7f13d40fd7a0 msgr2=0x7f13d40ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.397+0000 7f13dc21e640 1 -- 172.21.15.5:0/4016611123 shutdown_connections 2024-09-19T06:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.397+0000 7f13dc21e640 1 -- 172.21.15.5:0/4016611123 wait complete. 2024-09-19T06:25:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.398+0000 7f13dc21e640 1 Processor -- start 2024-09-19T06:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.398+0000 7f13dc21e640 1 -- start start 2024-09-19T06:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.398+0000 7f13dc21e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d4101ac0 0x7f13d41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.398+0000 7f13dc21e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d4102460 0x7f13d41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.398+0000 7f13dc21e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13d4107ad0 0x7f13d419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.398+0000 7f13dc21e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4111e10 con 0x7f13d4102460 2024-09-19T06:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.398+0000 7f13dc21e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4111c90 con 0x7f13d4107ad0 2024-09-19T06:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.398+0000 7f13dc21e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4111f90 con 0x7f13d4101ac0 2024-09-19T06:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13d9fba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d4101ac0 0x7f13d41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13d97b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d4102460 0x7f13d41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13d9fba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d4101ac0 0x7f13d41983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47194/0 (socket says 172.21.15.5:47194) 2024-09-19T06:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13d9fba640 1 -- 172.21.15.5:0/1512146204 learned_addr learned my addr 172.21.15.5:0/1512146204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13da7bb640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13d4107ad0 0x7f13d419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13d9fba640 1 -- 172.21.15.5:0/1512146204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13d4107ad0 msgr2=0x7f13d419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13d9fba640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13d4107ad0 0x7f13d419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13d9fba640 1 -- 172.21.15.5:0/1512146204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d4102460 msgr2=0x7f13d41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13d9fba640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d4102460 0x7f13d41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13d9fba640 1 -- 172.21.15.5:0/1512146204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d419d390 con 0x7f13d4101ac0 2024-09-19T06:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13d9fba640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d4101ac0 0x7f13d41983a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f13d000e9f0 tx=0x7f13d000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13caffd640 1 -- 172.21.15.5:0/1512146204 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d000ccc0 con 0x7f13d4101ac0 2024-09-19T06:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13caffd640 1 -- 172.21.15.5:0/1512146204 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13d00108e0 con 0x7f13d4101ac0 2024-09-19T06:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13caffd640 1 -- 172.21.15.5:0/1512146204 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00187c0 con 0x7f13d4101ac0 2024-09-19T06:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.399+0000 7f13dc21e640 1 -- 172.21.15.5:0/1512146204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d419d680 con 0x7f13d4101ac0 2024-09-19T06:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.400+0000 7f13dc21e640 1 -- 172.21.15.5:0/1512146204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d41a4f10 con 0x7f13d4101ac0 2024-09-19T06:25:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.402+0000 7f13caffd640 1 -- 172.21.15.5:0/1512146204 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f13d0010a90 con 0x7f13d4101ac0 2024-09-19T06:25:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.403+0000 7f13df08c740 1 Processor -- start 2024-09-19T06:25:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.404+0000 7f13df08c740 1 -- start start 2024-09-19T06:25:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.404+0000 7f13caffd640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f13c00a33b0 0x7f13c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.404+0000 7f13df08c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 0x55f76cb96d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.404+0000 7f13caffd640 1 -- 172.21.15.5:0/1512146204 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8594+0+0 (secure 0 0 0) 0x7f13d00d0070 con 0x7f13d4101ac0 2024-09-19T06:25:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.404+0000 7f13df08c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb972d0 0x55f76cba5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.404+0000 7f13da7bb640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f13c00a33b0 0x7f13c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.404+0000 7f13df08c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cba84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.404+0000 7f13df08c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f76cafb2f0 con 0x55f76cba60c0 2024-09-19T06:25:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.404+0000 7f13df08c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f76caa3270 con 0x55f76cb972d0 2024-09-19T06:25:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.404+0000 7f13df08c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f76caf72b0 con 0x55f76cb969b0 2024-09-19T06:25:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.404+0000 7f13d97b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb972d0 0x55f76cba5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.404+0000 7f13d9fba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 0x55f76cb96d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.405+0000 7f13d9fba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 0x55f76cb96d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47196/0 (socket says 172.21.15.5:47196) 2024-09-19T06:25:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.405+0000 7f13d97b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb972d0 0x55f76cba5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54940/0 (socket says 172.21.15.5:54940) 2024-09-19T06:25:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.405+0000 7f13d9fba640 1 -- 172.21.15.5:0/2864992811 learned_addr learned my addr 172.21.15.5:0/2864992811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.405+0000 7f13da7bb640 1 --2- 172.21.15.5:0/2864992811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cba84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.405+0000 7f13d9fba640 1 -- 172.21.15.5:0/2864992811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb972d0 msgr2=0x55f76cba5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.405+0000 7f13d9fba640 1 --2- 172.21.15.5:0/2864992811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb972d0 0x55f76cba5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.405+0000 7f13d9fba640 1 -- 172.21.15.5:0/2864992811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 msgr2=0x55f76cba84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.405+0000 7f13d9fba640 1 --2- 172.21.15.5:0/2864992811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cba84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.405+0000 7f13d9fba640 1 -- 172.21.15.5:0/2864992811 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f76caa2340 con 0x55f76cb969b0 2024-09-19T06:25:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.405+0000 7f13da7bb640 1 --2- 172.21.15.5:0/2864992811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cba84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.405+0000 7f13da7bb640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f13c00a33b0 0x7f13c00a5870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55f76cb94150 tx=0x7f13cc0098f0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.406+0000 7f13d9fba640 1 --2- 172.21.15.5:0/2864992811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 0x55f76cb96d90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f13d0098760 tx=0x7f13d009bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5cc4a98f73d9914 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.406+0000 7f13bffff640 1 -- 172.21.15.5:0/2864992811 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00a4f70 con 0x55f76cb969b0 2024-09-19T06:25:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.406+0000 7f13bffff640 1 -- 172.21.15.5:0/2864992811 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13d00cd9f0 con 0x55f76cb969b0 2024-09-19T06:25:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.406+0000 7f13bffff640 1 -- 172.21.15.5:0/2864992811 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00bc0c0 con 0x55f76cb969b0 2024-09-19T06:25:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.407+0000 7f13df08c740 1 -- 172.21.15.5:0/2864992811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 msgr2=0x55f76cb96d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.407+0000 7f13df08c740 1 --2- 172.21.15.5:0/2864992811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 0x55f76cb96d90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f13d0098760 tx=0x7f13d009bf70 comp rx=0 tx=0).stop 2024-09-19T06:25:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.407+0000 7f13df08c740 1 -- 172.21.15.5:0/2864992811 shutdown_connections 2024-09-19T06:25:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.407+0000 7f13df08c740 1 --2- 172.21.15.5:0/2864992811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cba84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.407+0000 7f13df08c740 1 --2- 172.21.15.5:0/2864992811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb972d0 0x55f76cba5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.407+0000 7f13df08c740 1 --2- 172.21.15.5:0/2864992811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 0x55f76cb96d90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.407+0000 7f13df08c740 1 -- 172.21.15.5:0/2864992811 >> 172.21.15.5:0/2864992811 conn(0x55f76cb93060 msgr2=0x55f76cb93440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.407+0000 7f13df08c740 1 -- 172.21.15.5:0/2864992811 shutdown_connections 2024-09-19T06:25:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.407+0000 7f13df08c740 1 -- 172.21.15.5:0/2864992811 wait complete. 2024-09-19T06:25:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.408+0000 7f13df08c740 1 -- start start 2024-09-19T06:25:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.408+0000 7f13df08c740 1 Processor -- start 2024-09-19T06:25:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.409+0000 7f13df08c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 0x55f76cba3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.409+0000 7f13df08c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb972d0 0x55f76cba3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.409+0000 7f13df08c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cba3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.409+0000 7f13df08c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f76cba8c30 con 0x55f76cba60c0 2024-09-19T06:25:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.409+0000 7f13d9fba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 0x55f76cba3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.409+0000 7f13da7bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cba3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.409+0000 7f13d9fba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 0x55f76cba3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54956/0 (socket says 172.21.15.5:54956) 2024-09-19T06:25:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.409+0000 7f13d9fba640 1 -- 172.21.15.5:0/3667964583 learned_addr learned my addr 172.21.15.5:0/3667964583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.409+0000 7f13d97b9640 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb972d0 0x55f76cba3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13df08c740 1 -- 172.21.15.5:0/3667964583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f76ca84290 con 0x55f76cb969b0 2024-09-19T06:25:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13df08c740 1 -- 172.21.15.5:0/3667964583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f76cba8db0 con 0x55f76cb972d0 2024-09-19T06:25:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13df08c740 10 client.0 _ll_register_callbacks cb 0x55f76cab5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13d9fba640 1 -- 172.21.15.5:0/3667964583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb972d0 msgr2=0x55f76cba3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13d9fba640 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb972d0 0x55f76cba3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13d9fba640 1 -- 172.21.15.5:0/3667964583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 msgr2=0x55f76cba3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13d9fba640 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cba3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13d9fba640 1 -- 172.21.15.5:0/3667964583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f76c80e210 con 0x55f76cb969b0 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13df08c740 1 -- 172.21.15.5:0/3667964583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13d00cd9f0 con 0x55f76cb969b0 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13da7bb640 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cba3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13d9fba640 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 0x55f76cba3020 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f13d00cd820 tx=0x7f13d0009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=472d437f3ca0cf51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13bd7fa640 10 client.30173 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.410+0000 7f13bd7fa640 1 -- 172.21.15.5:0/3667964583 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00ba060 con 0x55f76cb969b0 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.411+0000 7f13bd7fa640 1 -- 172.21.15.5:0/3667964583 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8594+0+0 (secure 0 0 0) 0x7f13d00b5260 con 0x55f76cb969b0 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.411+0000 7f13bd7fa640 1 -- 172.21.15.5:0/3667964583 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f13d00a4f70 con 0x55f76cb969b0 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.411+0000 7f13df08c740 10 client.30173 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.411+0000 7f13df08c740 1 -- 172.21.15.5:0/3667964583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=60}) -- 0x55f76c8da620 con 0x55f76cb969b0 2024-09-19T06:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.411+0000 7f13bd7fa640 1 -- 172.21.15.5:0/3667964583 <== mon.2 v2:172.21.15.49:3300/0 4 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f13d00bc080 con 0x55f76cb969b0 2024-09-19T06:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.411+0000 7f13df08c740 10 client.30173 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.411+0000 7f13df08c740 10 client.30173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.411+0000 7f13df08c740 20 client.30173 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.411+0000 7f13df08c740 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f76cbb5db0 0x55f76cbb81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.411+0000 7f13df08c740 4 client.30173 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.411+0000 7f13df08c740 1 -- 172.21.15.5:0/3667964583 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f76cb08a20 con 0x55f76cbb5db0 2024-09-19T06:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.411+0000 7f13d97b9640 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f76cbb5db0 0x55f76cbb81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.412+0000 7f13bd7fa640 10 client.30173 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.412+0000 7f13d97b9640 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f76cbb5db0 0x55f76cbb81a0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15fd4a20f826a8bf server_cookie=903a3baefd622d12 in_seq=0 out_seq=0 2024-09-19T06:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13bd7fa640 1 -- 172.21.15.5:0/3667964583 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f76cb08a20 con 0x55f76cbb5db0 2024-09-19T06:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13bd7fa640 10 client.30173 handle_command_reply: tid=0 2024-09-19T06:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13df08c740 1 client.30173 shutdown 2024-09-19T06:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13df08c740 1 -- 172.21.15.5:0/3667964583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 msgr2=0x55f76cba3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13df08c740 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 0x55f76cba3020 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f13d00cd820 tx=0x7f13d0009700 comp rx=0 tx=0).stop 2024-09-19T06:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13df08c740 1 -- 172.21.15.5:0/3667964583 shutdown_connections 2024-09-19T06:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13df08c740 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f76cbb5db0 0x55f76cbb81a0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13df08c740 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cba3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13df08c740 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb972d0 0x55f76cba3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13df08c740 1 --2- 172.21.15.5:0/3667964583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 0x55f76cba3020 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13bd7fa640 0 client.30173 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13df08c740 1 -- 172.21.15.5:0/3667964583 >> 172.21.15.5:0/3667964583 conn(0x55f76cb93060 msgr2=0x55f76cb921c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13df08c740 1 -- 172.21.15.5:0/3667964583 shutdown_connections 2024-09-19T06:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13df08c740 1 -- 172.21.15.5:0/3667964583 wait complete. 2024-09-19T06:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.413+0000 7f13df08c740 20 client.30173 trim_cache size 0 max 16384 2024-09-19T06:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13df08c740 1 Processor -- start 2024-09-19T06:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13df08c740 1 -- start start 2024-09-19T06:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13df08c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 0x55f76cbb1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13df08c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb972d0 0x55f76cbbb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13df08c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cbbb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13df08c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41ab700 con 0x55f76cba60c0 2024-09-19T06:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13df08c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41ad2a0 con 0x55f76cb969b0 2024-09-19T06:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13df08c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41abdd0 con 0x55f76cb972d0 2024-09-19T06:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13d9fba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 0x55f76cbb1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13d97b9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb972d0 0x55f76cbbb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13d97b9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb972d0 0x55f76cbbb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47208/0 (socket says 172.21.15.5:47208) 2024-09-19T06:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13d9fba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 0x55f76cbb1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54970/0 (socket says 172.21.15.5:54970) 2024-09-19T06:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13da7bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cbbb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.434+0000 7f13d97b9640 1 -- 172.21.15.5:0/3109557489 learned_addr learned my addr 172.21.15.5:0/3109557489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.435+0000 7f13d97b9640 1 -- 172.21.15.5:0/3109557489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 msgr2=0x55f76cbb1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.435+0000 7f13d97b9640 1 --2- 172.21.15.5:0/3109557489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 0x55f76cbb1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.435+0000 7f13d97b9640 1 -- 172.21.15.5:0/3109557489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 msgr2=0x55f76cbbb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.435+0000 7f13d97b9640 1 --2- 172.21.15.5:0/3109557489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cbbb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.435+0000 7f13d97b9640 1 -- 172.21.15.5:0/3109557489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f76cb8e2d0 con 0x55f76cb972d0 2024-09-19T06:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.435+0000 7f13d9fba640 1 --2- 172.21.15.5:0/3109557489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 0x55f76cbb1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.435+0000 7f13d97b9640 1 --2- 172.21.15.5:0/3109557489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb972d0 0x55f76cbbb1c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f13c402a9e0 tx=0x7f13c4005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a9c06d3e89acf01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.435+0000 7f13bf7fe640 1 -- 172.21.15.5:0/3109557489 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4035040 con 0x55f76cb972d0 2024-09-19T06:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.435+0000 7f13bf7fe640 1 -- 172.21.15.5:0/3109557489 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13c404b070 con 0x55f76cb972d0 2024-09-19T06:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.435+0000 7f13bf7fe640 1 -- 172.21.15.5:0/3109557489 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c40035f0 con 0x55f76cb972d0 2024-09-19T06:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.436+0000 7f13df08c740 1 -- 172.21.15.5:0/3109557489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb972d0 msgr2=0x55f76cbbb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.436+0000 7f13df08c740 1 --2- 172.21.15.5:0/3109557489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb972d0 0x55f76cbbb1c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f13c402a9e0 tx=0x7f13c4005a20 comp rx=0 tx=0).stop 2024-09-19T06:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.437+0000 7f13df08c740 1 -- 172.21.15.5:0/3109557489 shutdown_connections 2024-09-19T06:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.437+0000 7f13df08c740 1 --2- 172.21.15.5:0/3109557489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cbbb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.437+0000 7f13df08c740 1 --2- 172.21.15.5:0/3109557489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb972d0 0x55f76cbbb1c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.437+0000 7f13df08c740 1 --2- 172.21.15.5:0/3109557489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb969b0 0x55f76cbb1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.437+0000 7f13df08c740 1 -- 172.21.15.5:0/3109557489 >> 172.21.15.5:0/3109557489 conn(0x55f76cb93060 msgr2=0x55f76cbb1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.437+0000 7f13df08c740 1 -- 172.21.15.5:0/3109557489 shutdown_connections 2024-09-19T06:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.437+0000 7f13df08c740 1 -- 172.21.15.5:0/3109557489 wait complete. 2024-09-19T06:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.437+0000 7f13df08c740 1 -- start start 2024-09-19T06:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13df08c740 1 Processor -- start 2024-09-19T06:25:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13df08c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 0x55f76cbbbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13df08c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb972d0 0x55f76cbb8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13df08c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cbb95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13df08c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41acbd0 con 0x55f76cba60c0 2024-09-19T06:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13df08c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41ac4d0 con 0x55f76cb972d0 2024-09-19T06:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13df08c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41a9700 con 0x55f76cb969b0 2024-09-19T06:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13df08c740 10 client.0 _ll_register_callbacks cb 0x7f13cc0137e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13d9fba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 0x55f76cbbbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13d9fba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 0x55f76cbbbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47212/0 (socket says 172.21.15.5:47212) 2024-09-19T06:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13da7bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cbb95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13d97b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb972d0 0x55f76cbb8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.438+0000 7f13d9fba640 1 -- 172.21.15.5:0/557231467 learned_addr learned my addr 172.21.15.5:0/557231467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.439+0000 7f13d97b9640 1 -- 172.21.15.5:0/557231467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 msgr2=0x55f76cbbbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.439+0000 7f13d97b9640 1 --2- 172.21.15.5:0/557231467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 0x55f76cbbbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.439+0000 7f13d97b9640 1 -- 172.21.15.5:0/557231467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 msgr2=0x55f76cbb95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.439+0000 7f13d97b9640 1 --2- 172.21.15.5:0/557231467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cbb95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.439+0000 7f13d97b9640 1 -- 172.21.15.5:0/557231467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f76cc1f6f0 con 0x55f76cb972d0 2024-09-19T06:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.439+0000 7f13d97b9640 1 --2- 172.21.15.5:0/557231467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb972d0 0x55f76cbb8e20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x55f76cba5e20 tx=0x7f13c4031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f6915e5daa384 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.439+0000 7f13bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.439+0000 7f13bd7fa640 1 -- 172.21.15.5:0/557231467 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4053070 con 0x55f76cb972d0 2024-09-19T06:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.439+0000 7f13df08c740 1 -- 172.21.15.5:0/557231467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c404b070 con 0x55f76cb972d0 2024-09-19T06:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.439+0000 7f13bd7fa640 1 -- 172.21.15.5:0/557231467 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8594+0+0 (secure 0 0 0) 0x7f13c40037f0 con 0x55f76cb972d0 2024-09-19T06:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.440+0000 7f13bd7fa640 1 -- 172.21.15.5:0/557231467 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f13c4035050 con 0x55f76cb972d0 2024-09-19T06:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.440+0000 7f13df08c740 10 client.30179 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.440+0000 7f13df08c740 1 -- 172.21.15.5:0/557231467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55f76cc208c0 con 0x55f76cb972d0 2024-09-19T06:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.440+0000 7f13bd7fa640 1 -- 172.21.15.5:0/557231467 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13c404b070 con 0x55f76cb972d0 2024-09-19T06:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.440+0000 7f13bd7fa640 1 -- 172.21.15.5:0/557231467 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f13c40052d0 con 0x55f76cb972d0 2024-09-19T06:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.440+0000 7f13df08c740 10 client.30179 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.440+0000 7f13df08c740 10 client.30179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.440+0000 7f13df08c740 20 client.30179 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.440+0000 7f13df08c740 1 --2- 172.21.15.5:0/557231467 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f76cc20f90 0x55f76cc23380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.440+0000 7f13df08c740 4 client.30179 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.440+0000 7f13df08c740 1 -- 172.21.15.5:0/557231467 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f76cb02d70 con 0x55f76cc20f90 2024-09-19T06:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.441+0000 7f13d9fba640 1 --2- 172.21.15.5:0/557231467 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f76cc20f90 0x55f76cc23380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.441+0000 7f13bd7fa640 10 client.30179 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.441+0000 7f13d9fba640 1 --2- 172.21.15.5:0/557231467 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f76cc20f90 0x55f76cc23380 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a09fd4f8385409e server_cookie=ab0cab13ed376c6d in_seq=0 out_seq=0 2024-09-19T06:25:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.569+0000 7f13bd7fa640 1 -- 172.21.15.5:0/557231467 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f76cb02d70 con 0x55f76cc20f90 2024-09-19T06:25:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.569+0000 7f13bd7fa640 10 client.30179 handle_command_reply: tid=0 2024-09-19T06:25:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.569+0000 7f13df08c740 1 client.30179 shutdown 2024-09-19T06:25:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.569+0000 7f13df08c740 1 -- 172.21.15.5:0/557231467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb972d0 msgr2=0x55f76cbb8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.569+0000 7f13df08c740 1 --2- 172.21.15.5:0/557231467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb972d0 0x55f76cbb8e20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x55f76cba5e20 tx=0x7f13c4031400 comp rx=0 tx=0).stop 2024-09-19T06:25:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.570+0000 7f13df08c740 1 -- 172.21.15.5:0/557231467 shutdown_connections 2024-09-19T06:25:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.570+0000 7f13df08c740 1 --2- 172.21.15.5:0/557231467 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55f76cc20f90 0x55f76cc23380 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.570+0000 7f13df08c740 1 --2- 172.21.15.5:0/557231467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f76cba60c0 0x55f76cbb95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.570+0000 7f13bd7fa640 0 client.30179 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.570+0000 7f13df08c740 1 --2- 172.21.15.5:0/557231467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f76cb972d0 0x55f76cbb8e20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.570+0000 7f13df08c740 1 --2- 172.21.15.5:0/557231467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f76cb969b0 0x55f76cbbbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.570+0000 7f13df08c740 1 -- 172.21.15.5:0/557231467 >> 172.21.15.5:0/557231467 conn(0x55f76cb93060 msgr2=0x55f76cbb1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.570+0000 7f13df08c740 1 -- 172.21.15.5:0/557231467 shutdown_connections 2024-09-19T06:25:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.570+0000 7f13df08c740 1 -- 172.21.15.5:0/557231467 wait complete. 2024-09-19T06:25:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.570+0000 7f13df08c740 20 client.30179 trim_cache size 0 max 16384 2024-09-19T06:25:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:25:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:25:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:25:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:25:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.571+0000 7f13dc21e640 1 -- 172.21.15.5:0/1512146204 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f13c00a33b0 msgr2=0x7f13c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.571+0000 7f13dc21e640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f13c00a33b0 0x7f13c00a5870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55f76cb94150 tx=0x7f13cc0098f0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.572+0000 7f13dc21e640 1 -- 172.21.15.5:0/1512146204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d4101ac0 msgr2=0x7f13d41983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.572+0000 7f13dc21e640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d4101ac0 0x7f13d41983a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f13d000e9f0 tx=0x7f13d000edf0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.572+0000 7f13dc21e640 1 -- 172.21.15.5:0/1512146204 shutdown_connections 2024-09-19T06:25:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.572+0000 7f13dc21e640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13d4107ad0 0x7f13d419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.572+0000 7f13dc21e640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d4102460 0x7f13d41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.572+0000 7f13dc21e640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f13c00a33b0 0x7f13c00a5870 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.572+0000 7f13dc21e640 1 --2- 172.21.15.5:0/1512146204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d4101ac0 0x7f13d41983a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.572+0000 7f13dc21e640 1 -- 172.21.15.5:0/1512146204 >> 172.21.15.5:0/1512146204 conn(0x7f13d40fd7a0 msgr2=0x7f13d40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.572+0000 7f13dc21e640 1 -- 172.21.15.5:0/1512146204 shutdown_connections 2024-09-19T06:25:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:06.572+0000 7f13dc21e640 1 -- 172.21.15.5:0/1512146204 wait complete. 2024-09-19T06:25:06.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 82625 8.56 MB/sec execute 390 sec latency 140.323 ms 2024-09-19T06:25:07.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 82798 8.57 MB/sec execute 391 sec latency 134.374 ms 2024-09-19T06:25:08.184 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:25:08.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:08 smithi157 ceph-mon[35346]: pgmap v688: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 362 op/s 2024-09-19T06:25:08.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:08 smithi005 ceph-mon[30955]: pgmap v688: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 362 op/s 2024-09-19T06:25:08.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:08 smithi049 ceph-mon[34640]: pgmap v688: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 362 op/s 2024-09-19T06:25:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:25:08.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 83007 8.59 MB/sec execute 392 sec latency 75.778 ms 2024-09-19T06:25:09.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.159+0000 7f2d5a219640 1 -- 172.21.15.5:0/947028253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d54100e20 msgr2=0x7f2d54101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:09.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.159+0000 7f2d5a219640 1 --2- 172.21.15.5:0/947028253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d54100e20 0x7f2d54101200 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009900 tx=0x7f2d4402a480 comp rx=0 tx=0).stop 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.159+0000 7f2d5a219640 1 -- 172.21.15.5:0/947028253 shutdown_connections 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.159+0000 7f2d5a219640 1 --2- 172.21.15.5:0/947028253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d54102ef0 0x7f2d5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.159+0000 7f2d5a219640 1 --2- 172.21.15.5:0/947028253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d54102570 0x7f2d541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.159+0000 7f2d5a219640 1 --2- 172.21.15.5:0/947028253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d54100e20 0x7f2d54101200 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.159+0000 7f2d5a219640 1 -- 172.21.15.5:0/947028253 >> 172.21.15.5:0/947028253 conn(0x7f2d540fc7d0 msgr2=0x7f2d540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.159+0000 7f2d5a219640 1 -- 172.21.15.5:0/947028253 shutdown_connections 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.159+0000 7f2d5a219640 1 -- 172.21.15.5:0/947028253 wait complete. 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.160+0000 7f2d5a219640 1 Processor -- start 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.160+0000 7f2d5a219640 1 -- start start 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.160+0000 7f2d5a219640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d54100e20 0x7f2d541961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.160+0000 7f2d5a219640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d54102570 0x7f2d541966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d5a219640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d54102ef0 0x7f2d5419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d5a219640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5410fd70 con 0x7f2d54102570 2024-09-19T06:25:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d5a219640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5410fbf0 con 0x7f2d54100e20 2024-09-19T06:25:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d5a219640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5410fef0 con 0x7f2d54102ef0 2024-09-19T06:25:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d53fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d54102ef0 0x7f2d5419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d54102570 0x7f2d541966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d53fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d54102ef0 0x7f2d5419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47234/0 (socket says 172.21.15.5:47234) 2024-09-19T06:25:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d54102570 0x7f2d541966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47306/0 (socket says 172.21.15.5:47306) 2024-09-19T06:25:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d53fff640 1 -- 172.21.15.5:0/1556182186 learned_addr learned my addr 172.21.15.5:0/1556182186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d52ffd640 1 -- 172.21.15.5:0/1556182186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d54102ef0 msgr2=0x7f2d5419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/1556182186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d54102ef0 0x7f2d5419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d52ffd640 1 -- 172.21.15.5:0/1556182186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d54100e20 msgr2=0x7f2d541961a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:25:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/1556182186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d54100e20 0x7f2d541961a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.161+0000 7f2d52ffd640 1 -- 172.21.15.5:0/1556182186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5419b150 con 0x7f2d54102570 2024-09-19T06:25:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.162+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/1556182186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d54102570 0x7f2d541966e0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800e9f0 tx=0x7f2d4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.162+0000 7f2d50ff9640 1 -- 172.21.15.5:0/1556182186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4800ccc0 con 0x7f2d54102570 2024-09-19T06:25:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.162+0000 7f2d50ff9640 1 -- 172.21.15.5:0/1556182186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d480108e0 con 0x7f2d54102570 2024-09-19T06:25:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.162+0000 7f2d5a219640 1 -- 172.21.15.5:0/1556182186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5419b440 con 0x7f2d54102570 2024-09-19T06:25:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.162+0000 7f2d50ff9640 1 -- 172.21.15.5:0/1556182186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d480188e0 con 0x7f2d54102570 2024-09-19T06:25:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.162+0000 7f2d5a219640 1 -- 172.21.15.5:0/1556182186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d541a2d20 con 0x7f2d54102570 2024-09-19T06:25:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.165+0000 7f2d50ff9640 1 -- 172.21.15.5:0/1556182186 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2d48010a90 con 0x7f2d54102570 2024-09-19T06:25:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.166+0000 7f2d50ff9640 1 --2- 172.21.15.5:0/1556182186 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d3c0a3430 0x7f2d3c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.166+0000 7f2d537fe640 1 --2- 172.21.15.5:0/1556182186 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d3c0a3430 0x7f2d3c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.166+0000 7f2d50ff9640 1 -- 172.21.15.5:0/1556182186 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f2d480d0080 con 0x7f2d54102570 2024-09-19T06:25:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.166+0000 7f2d5c5f2740 1 Processor -- start 2024-09-19T06:25:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d5c5f2740 1 -- start start 2024-09-19T06:25:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d5c5f2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 0x55d29f205720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d5c5f2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205c60 0x55d29f214550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d5c5f2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f214a90 0x55d29f216e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d5c5f2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d29f16b2e0 con 0x55d29f205340 2024-09-19T06:25:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d5c5f2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d29f112ce0 con 0x55d29f205c60 2024-09-19T06:25:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d5c5f2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d29f1672a0 con 0x55d29f214a90 2024-09-19T06:25:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d537fe640 1 --2- 172.21.15.5:0/1556182186 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d3c0a3430 0x7f2d3c0a58f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2d440096c0 tx=0x7f2d44005fd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 0x55d29f205720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 0x55d29f205720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47314/0 (socket says 172.21.15.5:47314) 2024-09-19T06:25:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d537fe640 1 -- 172.21.15.5:0/1582344478 learned_addr learned my addr 172.21.15.5:0/1582344478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/1582344478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205c60 0x55d29f214550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d53fff640 1 --2- 172.21.15.5:0/1582344478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f214a90 0x55d29f216e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d52ffd640 1 -- 172.21.15.5:0/1582344478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f214a90 msgr2=0x55d29f216e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/1582344478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f214a90 0x55d29f216e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d52ffd640 1 -- 172.21.15.5:0/1582344478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 msgr2=0x55d29f205720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/1582344478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 0x55d29f205720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d52ffd640 1 -- 172.21.15.5:0/1582344478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d29f0f7440 con 0x55d29f205c60 2024-09-19T06:25:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.167+0000 7f2d537fe640 1 --2- 172.21.15.5:0/1582344478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 0x55d29f205720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.168+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/1582344478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205c60 0x55d29f214550 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2d480cd7b0 tx=0x7f2d48010f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad510f31943777c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.168+0000 7f2d39ffb640 1 -- 172.21.15.5:0/1582344478 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d480a06f0 con 0x55d29f205c60 2024-09-19T06:25:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.168+0000 7f2d39ffb640 1 -- 172.21.15.5:0/1582344478 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d480a4fb0 con 0x55d29f205c60 2024-09-19T06:25:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.168+0000 7f2d39ffb640 1 -- 172.21.15.5:0/1582344478 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4809ba80 con 0x55d29f205c60 2024-09-19T06:25:09.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.169+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1582344478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205c60 msgr2=0x55d29f214550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:09.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.169+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/1582344478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205c60 0x55d29f214550 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2d480cd7b0 tx=0x7f2d48010f40 comp rx=0 tx=0).stop 2024-09-19T06:25:09.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.169+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1582344478 shutdown_connections 2024-09-19T06:25:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.169+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/1582344478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f214a90 0x55d29f216e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.169+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/1582344478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205c60 0x55d29f214550 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.169+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/1582344478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 0x55d29f205720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.169+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1582344478 >> 172.21.15.5:0/1582344478 conn(0x55d29f201a30 msgr2=0x55d29f201e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.169+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1582344478 shutdown_connections 2024-09-19T06:25:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.169+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1582344478 wait complete. 2024-09-19T06:25:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.169+0000 7f2d5c5f2740 1 -- start start 2024-09-19T06:25:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.170+0000 7f2d5c5f2740 1 Processor -- start 2024-09-19T06:25:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.170+0000 7f2d5c5f2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205340 0x55d29f214040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.170+0000 7f2d5c5f2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205c60 0x55d29f20d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.170+0000 7f2d5c5f2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f214a90 0x55d29f20d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.170+0000 7f2d5c5f2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d29f217660 con 0x55d29f205c60 2024-09-19T06:25:09.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.170+0000 7f2d5c5f2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d29f126050 con 0x55d29f205340 2024-09-19T06:25:09.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.170+0000 7f2d5c5f2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d29f2177e0 con 0x55d29f214a90 2024-09-19T06:25:09.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.170+0000 7f2d5c5f2740 10 client.0 _ll_register_callbacks cb 0x55d29f166160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:09.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.170+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205c60 0x55d29f20d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.170+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205c60 0x55d29f20d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47330/0 (socket says 172.21.15.5:47330) 2024-09-19T06:25:09.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.170+0000 7f2d52ffd640 1 -- 172.21.15.5:0/3412544227 learned_addr learned my addr 172.21.15.5:0/3412544227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:09.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.171+0000 7f2d53fff640 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f214a90 0x55d29f20d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.171+0000 7f2d52ffd640 1 -- 172.21.15.5:0/3412544227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f214a90 msgr2=0x55d29f20d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:09.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.171+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f214a90 0x55d29f20d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.171+0000 7f2d52ffd640 1 -- 172.21.15.5:0/3412544227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205340 msgr2=0x55d29f214040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:09.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.171+0000 7f2d537fe640 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205340 0x55d29f214040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.171+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205340 0x55d29f214040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.171+0000 7f2d52ffd640 1 -- 172.21.15.5:0/3412544227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d29eedc3c0 con 0x55d29f205c60 2024-09-19T06:25:09.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.171+0000 7f2d537fe640 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205340 0x55d29f214040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:25:09.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.171+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205c60 0x55d29f20d2d0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f2d480cddb0 tx=0x7f2d48010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97b7422ea20da47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:09.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.171+0000 7f2d32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:25:09.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.171+0000 7f2d32ffd640 1 -- 172.21.15.5:0/3412544227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d480a06f0 con 0x55d29f205c60 2024-09-19T06:25:09.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.171+0000 7f2d32ffd640 1 -- 172.21.15.5:0/3412544227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f2d480b5250 con 0x55d29f205c60 2024-09-19T06:25:09.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.172+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/3412544227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d480a4fb0 con 0x55d29f205c60 2024-09-19T06:25:09.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.172+0000 7f2d32ffd640 1 -- 172.21.15.5:0/3412544227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2d4809b9a0 con 0x55d29f205c60 2024-09-19T06:25:09.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.172+0000 7f2d5c5f2740 10 client.20805 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:09.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.172+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/3412544227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55d29f2274d0 con 0x55d29f205c60 2024-09-19T06:25:09.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.172+0000 7f2d32ffd640 1 -- 172.21.15.5:0/3412544227 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d480a4fb0 con 0x55d29f205c60 2024-09-19T06:25:09.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.172+0000 7f2d32ffd640 1 -- 172.21.15.5:0/3412544227 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f2d480a0120 con 0x55d29f205c60 2024-09-19T06:25:09.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.172+0000 7f2d5c5f2740 10 client.20805 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:09.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.172+0000 7f2d5c5f2740 10 client.20805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:09.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.173+0000 7f2d5c5f2740 20 client.20805 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:09.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.173+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d29f228820 0x55d29f22ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.173+0000 7f2d5c5f2740 4 client.20805 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.173+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/3412544227 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d29f178850 con 0x55d29f228820 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.173+0000 7f2d537fe640 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d29f228820 0x55d29f22ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.173+0000 7f2d537fe640 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d29f228820 0x55d29f22ac10 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfba8f99636be0ba server_cookie=10181ac1fca9543d in_seq=0 out_seq=0 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.173+0000 7f2d32ffd640 10 client.20805 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.174+0000 7f2d32ffd640 1 -- 172.21.15.5:0/3412544227 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d29f178850 con 0x55d29f228820 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.174+0000 7f2d32ffd640 10 client.20805 handle_command_reply: tid=0 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.174+0000 7f2d5c5f2740 1 client.20805 shutdown 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.174+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/3412544227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205c60 msgr2=0x55d29f20d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.174+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205c60 0x55d29f20d2d0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f2d480cddb0 tx=0x7f2d48010660 comp rx=0 tx=0).stop 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.174+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/3412544227 shutdown_connections 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.174+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d29f228820 0x55d29f22ac10 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.174+0000 7f2d32ffd640 0 client.20805 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:09.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.174+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f214a90 0x55d29f20d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.174+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205c60 0x55d29f20d2d0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.174+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/3412544227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205340 0x55d29f214040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.174+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/3412544227 >> 172.21.15.5:0/3412544227 conn(0x55d29f201a30 msgr2=0x55d29f207790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:09.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.175+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/3412544227 shutdown_connections 2024-09-19T06:25:09.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.175+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/3412544227 wait complete. 2024-09-19T06:25:09.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.175+0000 7f2d5c5f2740 20 client.20805 trim_cache size 0 max 16384 2024-09-19T06:25:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.203+0000 7f2d5c5f2740 1 Processor -- start 2024-09-19T06:25:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.203+0000 7f2d5c5f2740 1 -- start start 2024-09-19T06:25:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.203+0000 7f2d5c5f2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205340 0x55d29f2301b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.203+0000 7f2d5c5f2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 0x55d29f2306f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d5c5f2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f214a90 0x55d29f20a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d537fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205340 0x55d29f2301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d5c5f2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541ab5a0 con 0x55d29f214a90 2024-09-19T06:25:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d5c5f2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541ad140 con 0x55d29f205340 2024-09-19T06:25:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d5c5f2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541abc70 con 0x55d29f205c60 2024-09-19T06:25:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d537fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205340 0x55d29f2301b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55020/0 (socket says 172.21.15.5:55020) 2024-09-19T06:25:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d537fe640 1 -- 172.21.15.5:0/4235760417 learned_addr learned my addr 172.21.15.5:0/4235760417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/4235760417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 0x55d29f2306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d53fff640 1 --2- 172.21.15.5:0/4235760417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f214a90 0x55d29f20a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d52ffd640 1 -- 172.21.15.5:0/4235760417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205340 msgr2=0x55d29f2301b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/4235760417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205340 0x55d29f2301b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d52ffd640 1 -- 172.21.15.5:0/4235760417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f214a90 msgr2=0x55d29f20a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/4235760417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f214a90 0x55d29f20a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d52ffd640 1 -- 172.21.15.5:0/4235760417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d29f2143b0 con 0x55d29f205c60 2024-09-19T06:25:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.204+0000 7f2d53fff640 1 --2- 172.21.15.5:0/4235760417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f214a90 0x55d29f20a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:25:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.205+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/4235760417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 0x55d29f2306f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2d4809c040 tx=0x7f2d480cd860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e6a9ad1380b0a72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.205+0000 7f2d397fa640 1 -- 172.21.15.5:0/4235760417 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d480a1880 con 0x55d29f205c60 2024-09-19T06:25:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.205+0000 7f2d397fa640 1 -- 172.21.15.5:0/4235760417 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d480a1a90 con 0x55d29f205c60 2024-09-19T06:25:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.205+0000 7f2d397fa640 1 -- 172.21.15.5:0/4235760417 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d480a0120 con 0x55d29f205c60 2024-09-19T06:25:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.206+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/4235760417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 msgr2=0x55d29f2306f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.206+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/4235760417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 0x55d29f2306f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2d4809c040 tx=0x7f2d480cd860 comp rx=0 tx=0).stop 2024-09-19T06:25:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.206+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/4235760417 shutdown_connections 2024-09-19T06:25:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.206+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/4235760417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f214a90 0x55d29f20a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.206+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/4235760417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 0x55d29f2306f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.206+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/4235760417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f205340 0x55d29f2301b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.206+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/4235760417 >> 172.21.15.5:0/4235760417 conn(0x55d29f201a30 msgr2=0x55d29f211360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.206+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/4235760417 shutdown_connections 2024-09-19T06:25:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.206+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/4235760417 wait complete. 2024-09-19T06:25:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.207+0000 7f2d5c5f2740 1 -- start start 2024-09-19T06:25:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.207+0000 7f2d5c5f2740 1 Processor -- start 2024-09-19T06:25:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.207+0000 7f2d5c5f2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 0x55d29f222cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.207+0000 7f2d5c5f2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 0x55d29f2231f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.207+0000 7f2d5c5f2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f214a90 0x55d29f28a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 0x55d29f2231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 0x55d29f222cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 0x55d29f2231f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47276/0 (socket says 172.21.15.5:47276) 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d52ffd640 1 -- 172.21.15.5:0/1813481838 learned_addr learned my addr 172.21.15.5:0/1813481838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 0x55d29f222cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47348/0 (socket says 172.21.15.5:47348) 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1813481838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541aca70 con 0x55d29f205340 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1813481838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541ac370 con 0x55d29f214a90 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1813481838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541a95a0 con 0x55d29f205c60 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d5c5f2740 10 client.0 _ll_register_callbacks cb 0x7f2d4c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d53fff640 1 --2- 172.21.15.5:0/1813481838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f214a90 0x55d29f28a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d52ffd640 1 -- 172.21.15.5:0/1813481838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f214a90 msgr2=0x55d29f28a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/1813481838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f214a90 0x55d29f28a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d52ffd640 1 -- 172.21.15.5:0/1813481838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 msgr2=0x55d29f222cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/1813481838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 0x55d29f222cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d52ffd640 1 -- 172.21.15.5:0/1813481838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d29f28a780 con 0x55d29f205c60 2024-09-19T06:25:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d52ffd640 1 --2- 172.21.15.5:0/1813481838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 0x55d29f2231f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2d480987a0 tx=0x7f2d4800c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=913e27b8ca4d8701 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.208+0000 7f2d32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:25:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.209+0000 7f2d32ffd640 1 -- 172.21.15.5:0/1813481838 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48010740 con 0x55d29f205c60 2024-09-19T06:25:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.209+0000 7f2d32ffd640 1 -- 172.21.15.5:0/1813481838 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f2d480850b0 con 0x55d29f205c60 2024-09-19T06:25:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.209+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1813481838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d29f172d60 con 0x55d29f205c60 2024-09-19T06:25:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.209+0000 7f2d32ffd640 1 -- 172.21.15.5:0/1813481838 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2d4809c110 con 0x55d29f205c60 2024-09-19T06:25:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.209+0000 7f2d5c5f2740 10 client.30118 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.209+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1813481838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55d29f28a780 con 0x55d29f205c60 2024-09-19T06:25:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.209+0000 7f2d32ffd640 1 -- 172.21.15.5:0/1813481838 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d480a06f0 con 0x55d29f205c60 2024-09-19T06:25:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.210+0000 7f2d32ffd640 1 -- 172.21.15.5:0/1813481838 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f2d480bc2b0 con 0x55d29f205c60 2024-09-19T06:25:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.210+0000 7f2d5c5f2740 10 client.30118 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.210+0000 7f2d5c5f2740 10 client.30118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.210+0000 7f2d5c5f2740 20 client.30118 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.210+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/1813481838 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d29f228820 0x55d29f294fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.210+0000 7f2d5c5f2740 4 client.30118 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:25:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.210+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1813481838 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d29f172d60 con 0x55d29f228820 2024-09-19T06:25:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.210+0000 7f2d537fe640 1 --2- 172.21.15.5:0/1813481838 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d29f228820 0x55d29f294fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.211+0000 7f2d537fe640 1 --2- 172.21.15.5:0/1813481838 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d29f228820 0x55d29f294fd0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebef9c27d02f1e10 server_cookie=b39c091cc3c73ecf in_seq=0 out_seq=0 2024-09-19T06:25:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.211+0000 7f2d32ffd640 10 client.30118 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.212+0000 7f2d32ffd640 1 -- 172.21.15.5:0/1813481838 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d29f172d60 con 0x55d29f228820 2024-09-19T06:25:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.212+0000 7f2d32ffd640 10 client.30118 handle_command_reply: tid=0 2024-09-19T06:25:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.212+0000 7f2d5c5f2740 1 client.30118 shutdown 2024-09-19T06:25:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.212+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1813481838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 msgr2=0x55d29f2231f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.212+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/1813481838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 0x55d29f2231f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2d480987a0 tx=0x7f2d4800c7d0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.212+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1813481838 shutdown_connections 2024-09-19T06:25:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.212+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/1813481838 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55d29f228820 0x55d29f294fd0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.212+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/1813481838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d29f214a90 0x55d29f28a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.212+0000 7f2d32ffd640 0 client.30118 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.212+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/1813481838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d29f205c60 0x55d29f2231f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.212+0000 7f2d5c5f2740 1 --2- 172.21.15.5:0/1813481838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d29f205340 0x55d29f222cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.212+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1813481838 >> 172.21.15.5:0/1813481838 conn(0x55d29f201a30 msgr2=0x55d29f20cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.213+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1813481838 shutdown_connections 2024-09-19T06:25:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.213+0000 7f2d5c5f2740 1 -- 172.21.15.5:0/1813481838 wait complete. 2024-09-19T06:25:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.213+0000 7f2d5c5f2740 20 client.30118 trim_cache size 0 max 16384 2024-09-19T06:25:09.218 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:25:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:25:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:25:09.219 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:25:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.214+0000 7f2d5a219640 1 -- 172.21.15.5:0/1556182186 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d3c0a3430 msgr2=0x7f2d3c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.214+0000 7f2d5a219640 1 --2- 172.21.15.5:0/1556182186 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d3c0a3430 0x7f2d3c0a58f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2d440096c0 tx=0x7f2d44005fd0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.214+0000 7f2d5a219640 1 -- 172.21.15.5:0/1556182186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d54102570 msgr2=0x7f2d541966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.214+0000 7f2d5a219640 1 --2- 172.21.15.5:0/1556182186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d54102570 0x7f2d541966e0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800e9f0 tx=0x7f2d4800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.214+0000 7f2d5a219640 1 -- 172.21.15.5:0/1556182186 shutdown_connections 2024-09-19T06:25:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.214+0000 7f2d5a219640 1 --2- 172.21.15.5:0/1556182186 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d3c0a3430 0x7f2d3c0a58f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.214+0000 7f2d5a219640 1 --2- 172.21.15.5:0/1556182186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d54102ef0 0x7f2d5419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.214+0000 7f2d5a219640 1 --2- 172.21.15.5:0/1556182186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d54102570 0x7f2d541966e0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.214+0000 7f2d5a219640 1 --2- 172.21.15.5:0/1556182186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d54100e20 0x7f2d541961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.214+0000 7f2d5a219640 1 -- 172.21.15.5:0/1556182186 >> 172.21.15.5:0/1556182186 conn(0x7f2d540fc7d0 msgr2=0x7f2d540fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.214+0000 7f2d5a219640 1 -- 172.21.15.5:0/1556182186 shutdown_connections 2024-09-19T06:25:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:09.214+0000 7f2d5a219640 1 -- 172.21.15.5:0/1556182186 wait complete. 2024-09-19T06:25:09.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 83239 8.61 MB/sec execute 393 sec latency 186.349 ms 2024-09-19T06:25:09.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:09 smithi157 ceph-mon[35346]: osdmap e117: 12 total, 12 up, 12 in 2024-09-19T06:25:09.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:09 smithi005 ceph-mon[30955]: osdmap e117: 12 total, 12 up, 12 in 2024-09-19T06:25:09.895 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:09 smithi049 ceph-mon[34640]: osdmap e117: 12 total, 12 up, 12 in 2024-09-19T06:25:10.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 83375 8.61 MB/sec execute 394 sec latency 103.606 ms 2024-09-19T06:25:10.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:10 smithi157 ceph-mon[35346]: pgmap v690: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 453 op/s 2024-09-19T06:25:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:10 smithi005 ceph-mon[30955]: pgmap v690: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 453 op/s 2024-09-19T06:25:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:10 smithi049 ceph-mon[34640]: pgmap v690: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 453 op/s 2024-09-19T06:25:11.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 83457 8.59 MB/sec execute 395 sec latency 107.717 ms 2024-09-19T06:25:12.480 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:25:12.481 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:25:12.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 83611 8.60 MB/sec execute 396 sec latency 102.572 ms 2024-09-19T06:25:12.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:12 smithi157 ceph-mon[35346]: pgmap v691: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 221 op/s 2024-09-19T06:25:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:12 smithi005 ceph-mon[30955]: pgmap v691: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 221 op/s 2024-09-19T06:25:12.881 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:25:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:12 smithi049 ceph-mon[34640]: pgmap v691: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 221 op/s 2024-09-19T06:25:13.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 83754 8.61 MB/sec execute 397 sec latency 126.325 ms 2024-09-19T06:25:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.825+0000 7f6189a72640 1 -- 172.21.15.5:0/235651226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6184108b50 msgr2=0x7f6184115070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.825+0000 7f6189a72640 1 --2- 172.21.15.5:0/235651226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6184108b50 0x7f6184115070 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f617c009cd0 tx=0x7f617c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:25:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.825+0000 7f6189a72640 1 -- 172.21.15.5:0/235651226 shutdown_connections 2024-09-19T06:25:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.825+0000 7f6189a72640 1 --2- 172.21.15.5:0/235651226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6184108b50 0x7f6184115070 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.825+0000 7f6189a72640 1 --2- 172.21.15.5:0/235651226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61841081f0 0x7f6184108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:13.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.825+0000 7f6189a72640 1 --2- 172.21.15.5:0/235651226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f618410e230 0x7f618410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:13.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.825+0000 7f6189a72640 1 -- 172.21.15.5:0/235651226 >> 172.21.15.5:0/235651226 conn(0x7f6184077bb0 msgr2=0x7f6184079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:13.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.825+0000 7f6189a72640 1 -- 172.21.15.5:0/235651226 shutdown_connections 2024-09-19T06:25:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.825+0000 7f6189a72640 1 -- 172.21.15.5:0/235651226 wait complete. 2024-09-19T06:25:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.826+0000 7f6189a72640 1 Processor -- start 2024-09-19T06:25:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.826+0000 7f6189a72640 1 -- start start 2024-09-19T06:25:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6189a72640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61841081f0 0x7f6184073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6189a72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184108b50 0x7f618406dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6189a72640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f618410e230 0x7f618406e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6189a72640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f618406c4f0 con 0x7f6184108b50 2024-09-19T06:25:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6189a72640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f618406c370 con 0x7f618410e230 2024-09-19T06:25:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6189a72640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f618406c670 con 0x7f61841081f0 2024-09-19T06:25:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6182ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184108b50 0x7f618406dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6182ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184108b50 0x7f618406dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47376/0 (socket says 172.21.15.5:47376) 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6182ffd640 1 -- 172.21.15.5:0/1652183691 learned_addr learned my addr 172.21.15.5:0/1652183691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f61837fe640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61841081f0 0x7f6184073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6183fff640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f618410e230 0x7f618406e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6182ffd640 1 -- 172.21.15.5:0/1652183691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61841081f0 msgr2=0x7f6184073b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6182ffd640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61841081f0 0x7f6184073b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6182ffd640 1 -- 172.21.15.5:0/1652183691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f618410e230 msgr2=0x7f618406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6182ffd640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f618410e230 0x7f618406e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.827+0000 7f6182ffd640 1 -- 172.21.15.5:0/1652183691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618406e800 con 0x7f6184108b50 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.828+0000 7f6182ffd640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184108b50 0x7f618406dbc0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f617800ea60 tx=0x7f617800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.828+0000 7f6180ff9640 1 -- 172.21.15.5:0/1652183691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617800cde0 con 0x7f6184108b50 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.828+0000 7f6189a72640 1 -- 172.21.15.5:0/1652183691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61841b3860 con 0x7f6184108b50 2024-09-19T06:25:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.828+0000 7f6189a72640 1 -- 172.21.15.5:0/1652183691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61841b3d10 con 0x7f6184108b50 2024-09-19T06:25:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.828+0000 7f6180ff9640 1 -- 172.21.15.5:0/1652183691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61780035f0 con 0x7f6184108b50 2024-09-19T06:25:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.829+0000 7f6180ff9640 1 -- 172.21.15.5:0/1652183691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178010690 con 0x7f6184108b50 2024-09-19T06:25:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.830+0000 7f6189a72640 1 -- 172.21.15.5:0/1652183691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61700052c0 con 0x7f6184108b50 2024-09-19T06:25:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.831+0000 7f6180ff9640 1 -- 172.21.15.5:0/1652183691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6178003fe0 con 0x7f6184108b50 2024-09-19T06:25:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.832+0000 7f6180ff9640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f616c0a3430 0x7f616c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.832+0000 7f61837fe640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f616c0a3430 0x7f616c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.833+0000 7f6180ff9640 1 -- 172.21.15.5:0/1652183691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f61780cffa0 con 0x7f6184108b50 2024-09-19T06:25:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.833+0000 7f61837fe640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f616c0a3430 0x7f616c0a58f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6174009ae0 tx=0x7f6174005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:13.839+0000 7f6180ff9640 1 -- 172.21.15.5:0/1652183691 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f617809c960 con 0x7f6184108b50 2024-09-19T06:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.220+0000 7f6189a72640 1 -- 172.21.15.5:0/1652183691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6170005880 con 0x7f6184108b50 2024-09-19T06:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.221+0000 7f6180ff9640 1 -- 172.21.15.5:0/1652183691 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11961 (secure 0 0 0) 0x7f617809c260 con 0x7f6184108b50 2024-09-19T06:25:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:25:14.223 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":60,"btime":"2024-09-19T06:23:36:530049+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:23:35.567762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:25:14.224 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 60 2024-09-19T06:25:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.225+0000 7f6189a72640 1 -- 172.21.15.5:0/1652183691 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f616c0a3430 msgr2=0x7f616c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.225+0000 7f6189a72640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f616c0a3430 0x7f616c0a58f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6174009ae0 tx=0x7f6174005f90 comp rx=0 tx=0).stop 2024-09-19T06:25:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.225+0000 7f6189a72640 1 -- 172.21.15.5:0/1652183691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184108b50 msgr2=0x7f618406dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.225+0000 7f6189a72640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184108b50 0x7f618406dbc0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f617800ea60 tx=0x7f617800ef20 comp rx=0 tx=0).stop 2024-09-19T06:25:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.225+0000 7f6189a72640 1 -- 172.21.15.5:0/1652183691 shutdown_connections 2024-09-19T06:25:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.225+0000 7f6189a72640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f618410e230 0x7f618406e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.225+0000 7f6189a72640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184108b50 0x7f618406dbc0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.225+0000 7f6189a72640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f616c0a3430 0x7f616c0a58f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.225+0000 7f6189a72640 1 --2- 172.21.15.5:0/1652183691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61841081f0 0x7f6184073b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.225+0000 7f6189a72640 1 -- 172.21.15.5:0/1652183691 >> 172.21.15.5:0/1652183691 conn(0x7f6184077bb0 msgr2=0x7f61840795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.226+0000 7f6189a72640 1 -- 172.21.15.5:0/1652183691 shutdown_connections 2024-09-19T06:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:14.226+0000 7f6189a72640 1 -- 172.21.15.5:0/1652183691 wait complete. 2024-09-19T06:25:14.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 83908 8.61 MB/sec execute 398 sec latency 77.422 ms 2024-09-19T06:25:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:14 smithi157 ceph-mon[35346]: pgmap v692: 97 pgs: 97 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.7 MiB/s wr, 381 op/s 2024-09-19T06:25:14.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:14 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1652183691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:25:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:14 smithi005 ceph-mon[30955]: pgmap v692: 97 pgs: 97 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.7 MiB/s wr, 381 op/s 2024-09-19T06:25:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:14 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1652183691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:25:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:14 smithi049 ceph-mon[34640]: pgmap v692: 97 pgs: 97 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.7 MiB/s wr, 381 op/s 2024-09-19T06:25:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:14 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1652183691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:25:14.980 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:25:15.363 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:25:15.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84043 8.61 MB/sec execute 399 sec latency 70.205 ms 2024-09-19T06:25:15.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.918+0000 7f9cd2a32640 1 -- 172.21.15.5:0/3017206925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc102570 msgr2=0x7f9ccc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:15.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.918+0000 7f9cd2a32640 1 --2- 172.21.15.5:0/3017206925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc102570 0x7f9ccc1029b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400d910 tx=0x7f9cb402a210 comp rx=0 tx=0).stop 2024-09-19T06:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.918+0000 7f9cd2a32640 1 -- 172.21.15.5:0/3017206925 shutdown_connections 2024-09-19T06:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.918+0000 7f9cd2a32640 1 --2- 172.21.15.5:0/3017206925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ccc102ef0 0x7f9ccc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.918+0000 7f9cd2a32640 1 --2- 172.21.15.5:0/3017206925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc102570 0x7f9ccc1029b0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.918+0000 7f9cd2a32640 1 --2- 172.21.15.5:0/3017206925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ccc100e20 0x7f9ccc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.918+0000 7f9cd2a32640 1 -- 172.21.15.5:0/3017206925 >> 172.21.15.5:0/3017206925 conn(0x7f9ccc0fc7d0 msgr2=0x7f9ccc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.918+0000 7f9cd2a32640 1 -- 172.21.15.5:0/3017206925 shutdown_connections 2024-09-19T06:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.918+0000 7f9cd2a32640 1 -- 172.21.15.5:0/3017206925 wait complete. 2024-09-19T06:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9cd2a32640 1 Processor -- start 2024-09-19T06:25:15.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9cd2a32640 1 -- start start 2024-09-19T06:25:15.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9cd2a32640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ccc100e20 0x7f9ccc1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9cd2a32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ccc102570 0x7f9ccc1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9cd2a32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc102ef0 0x7f9ccc19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9cd2a32640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc10fd30 con 0x7f9ccc102ef0 2024-09-19T06:25:15.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9cd2a32640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc10fbb0 con 0x7f9ccc102570 2024-09-19T06:25:15.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9cd2a32640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc10feb0 con 0x7f9ccc100e20 2024-09-19T06:25:15.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9ccbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ccc100e20 0x7f9ccc1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9cd0fcf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc102ef0 0x7f9ccc19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9cd0fcf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc102ef0 0x7f9ccc19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35776/0 (socket says 172.21.15.5:35776) 2024-09-19T06:25:15.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9cd0fcf640 1 -- 172.21.15.5:0/820450238 learned_addr learned my addr 172.21.15.5:0/820450238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:15.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.919+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ccc102570 0x7f9ccc1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.920+0000 7f9cd0fcf640 1 -- 172.21.15.5:0/820450238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ccc100e20 msgr2=0x7f9ccc1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:15.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.920+0000 7f9cd0fcf640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ccc100e20 0x7f9ccc1a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.920+0000 7f9cd0fcf640 1 -- 172.21.15.5:0/820450238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ccc102570 msgr2=0x7f9ccc1a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:15.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.920+0000 7f9cd0fcf640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ccc102570 0x7f9ccc1a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.920+0000 7f9cd0fcf640 1 -- 172.21.15.5:0/820450238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc19efd0 con 0x7f9ccc102ef0 2024-09-19T06:25:15.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.920+0000 7f9cd0fcf640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc102ef0 0x7f9ccc19e830 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400dfb0 tx=0x7f9cc400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:15.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.921+0000 7f9cc9ffb640 1 -- 172.21.15.5:0/820450238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4015070 con 0x7f9ccc102ef0 2024-09-19T06:25:15.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.921+0000 7f9cc9ffb640 1 -- 172.21.15.5:0/820450238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cc4013070 con 0x7f9ccc102ef0 2024-09-19T06:25:15.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.921+0000 7f9cd2a32640 1 -- 172.21.15.5:0/820450238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc19f2c0 con 0x7f9ccc102ef0 2024-09-19T06:25:15.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.921+0000 7f9cc9ffb640 1 -- 172.21.15.5:0/820450238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4019670 con 0x7f9ccc102ef0 2024-09-19T06:25:15.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.921+0000 7f9cd2a32640 1 -- 172.21.15.5:0/820450238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc1ab510 con 0x7f9ccc102ef0 2024-09-19T06:25:15.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.924+0000 7f9cc9ffb640 1 -- 172.21.15.5:0/820450238 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9cc4004020 con 0x7f9ccc102ef0 2024-09-19T06:25:15.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.924+0000 7f9cd4e0b740 1 Processor -- start 2024-09-19T06:25:15.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.924+0000 7f9cd4e0b740 1 -- start start 2024-09-19T06:25:15.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cd4e0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4ecb40 0x557dbe4eb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cd4e0b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4eb5d0 0x557dbe4eba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cd4e0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4f6370 0x557dbe4f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cd4e0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557dbe4512f0 con 0x557dbe4f6370 2024-09-19T06:25:15.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cd4e0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557dbe3f8cf0 con 0x557dbe4ecb40 2024-09-19T06:25:15.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cd4e0b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557dbe44d2b0 con 0x557dbe4eb5d0 2024-09-19T06:25:15.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cc3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4ecb40 0x557dbe4eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cc3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4ecb40 0x557dbe4eb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40232/0 (socket says 172.21.15.5:40232) 2024-09-19T06:25:15.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cc3fff640 1 -- 172.21.15.5:0/3220375356 learned_addr learned my addr 172.21.15.5:0/3220375356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:15.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/3220375356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4f6370 0x557dbe4f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9ccbfff640 1 --2- 172.21.15.5:0/3220375356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4eb5d0 0x557dbe4eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cc3fff640 1 -- 172.21.15.5:0/3220375356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4eb5d0 msgr2=0x557dbe4eba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:15.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/3220375356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4eb5d0 0x557dbe4eba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cc3fff640 1 -- 172.21.15.5:0/3220375356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4f6370 msgr2=0x557dbe4f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:15.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/3220375356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4f6370 0x557dbe4f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.925+0000 7f9cc3fff640 1 -- 172.21.15.5:0/3220375356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbe3dd3a0 con 0x557dbe4ecb40 2024-09-19T06:25:15.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.926+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/3220375356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4ecb40 0x557dbe4eb090 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9cb402a730 tx=0x7f9cb4005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4a1b49f362f3f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:15.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.926+0000 7f9cc9ffb640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9cb80a3430 0x7f9cb80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.926+0000 7f9ccbfff640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9cb80a3430 0x7f9cb80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.926+0000 7f9cc27fc640 1 -- 172.21.15.5:0/3220375356 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb400d3d0 con 0x557dbe4ecb40 2024-09-19T06:25:15.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.926+0000 7f9cc27fc640 1 -- 172.21.15.5:0/3220375356 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cb402e070 con 0x557dbe4ecb40 2024-09-19T06:25:15.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.926+0000 7f9cc27fc640 1 -- 172.21.15.5:0/3220375356 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40046c0 con 0x557dbe4ecb40 2024-09-19T06:25:15.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.926+0000 7f9cc9ffb640 1 -- 172.21.15.5:0/820450238 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f9cc40cfb30 con 0x7f9ccc102ef0 2024-09-19T06:25:15.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.927+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/3220375356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4ecb40 msgr2=0x557dbe4eb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:15.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.927+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/3220375356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4ecb40 0x557dbe4eb090 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9cb402a730 tx=0x7f9cb4005b50 comp rx=0 tx=0).stop 2024-09-19T06:25:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.927+0000 7f9ccbfff640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9cb80a3430 0x7f9cb80a58f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x557dbe4e8ac0 tx=0x7f9cbc005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.927+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/3220375356 shutdown_connections 2024-09-19T06:25:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.927+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/3220375356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4f6370 0x557dbe4f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.927+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/3220375356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4eb5d0 0x557dbe4eba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.927+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/3220375356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4ecb40 0x557dbe4eb090 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.927+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/3220375356 >> 172.21.15.5:0/3220375356 conn(0x557dbe4b4310 msgr2=0x557dbe4e7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.927+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/3220375356 shutdown_connections 2024-09-19T06:25:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.927+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/3220375356 wait complete. 2024-09-19T06:25:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.928+0000 7f9cd4e0b740 1 -- start start 2024-09-19T06:25:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.928+0000 7f9cd4e0b740 1 Processor -- start 2024-09-19T06:25:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.928+0000 7f9cd4e0b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4eb5d0 0x557dbe4f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.928+0000 7f9cd4e0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4ecb40 0x557dbe4f3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.928+0000 7f9cd4e0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4f6370 0x557dbe4f3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.928+0000 7f9cd4e0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557dbe4f9010 con 0x557dbe4ecb40 2024-09-19T06:25:15.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.928+0000 7f9cd4e0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557dbe40c060 con 0x557dbe4f6370 2024-09-19T06:25:15.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.928+0000 7f9cd4e0b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557dbe4f9190 con 0x557dbe4eb5d0 2024-09-19T06:25:15.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.928+0000 7f9cd4e0b740 10 client.0 _ll_register_callbacks cb 0x557dbe15f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:15.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.928+0000 7f9cc3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4ecb40 0x557dbe4f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.928+0000 7f9cc3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4ecb40 0x557dbe4f3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35798/0 (socket says 172.21.15.5:35798) 2024-09-19T06:25:15.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.928+0000 7f9cc3fff640 1 -- 172.21.15.5:0/890093865 learned_addr learned my addr 172.21.15.5:0/890093865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:15.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9ccbfff640 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4eb5d0 0x557dbe4f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9cd0fcf640 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4f6370 0x557dbe4f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9cc3fff640 1 -- 172.21.15.5:0/890093865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4eb5d0 msgr2=0x557dbe4f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:15.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4eb5d0 0x557dbe4f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9cc3fff640 1 -- 172.21.15.5:0/890093865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4f6370 msgr2=0x557dbe4f3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4f6370 0x557dbe4f3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9cc3fff640 1 -- 172.21.15.5:0/890093865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbe1c23c0 con 0x557dbe4ecb40 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9cd0fcf640 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4f6370 0x557dbe4f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4ecb40 0x557dbe4f3080 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400d690 tx=0x7f9cb400d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124aa2fc7afaa6e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9cab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9cab7fe640 1 -- 172.21.15.5:0/890093865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb404e060 con 0x557dbe4ecb40 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9cab7fe640 1 -- 172.21.15.5:0/890093865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f9cb4045040 con 0x557dbe4ecb40 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.929+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/890093865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb402e070 con 0x557dbe4ecb40 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.930+0000 7f9cab7fe640 1 -- 172.21.15.5:0/890093865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f9cb40055a0 con 0x557dbe4ecb40 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.930+0000 7f9cd4e0b740 10 client.20829 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.930+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/890093865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x557dbe504b10 con 0x557dbe4ecb40 2024-09-19T06:25:15.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.930+0000 7f9cab7fe640 1 -- 172.21.15.5:0/890093865 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cb402e070 con 0x557dbe4ecb40 2024-09-19T06:25:15.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.930+0000 7f9cab7fe640 1 -- 172.21.15.5:0/890093865 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f9cb400c580 con 0x557dbe4ecb40 2024-09-19T06:25:15.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.931+0000 7f9cd4e0b740 10 client.20829 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:15.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.931+0000 7f9cd4e0b740 10 client.20829 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:25:15.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.931+0000 7f9cd4e0b740 20 client.20829 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:15.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.931+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557dbe505e60 0x557dbe508250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.931+0000 7f9cd4e0b740 4 client.20829 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:15.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.931+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/890093865 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dbe45e860 con 0x557dbe505e60 2024-09-19T06:25:15.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.931+0000 7f9ccbfff640 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557dbe505e60 0x557dbe508250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.931+0000 7f9ccbfff640 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557dbe505e60 0x557dbe508250 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da589d44761af3c2 server_cookie=540788eca53ff6a7 in_seq=0 out_seq=0 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.931+0000 7f9cab7fe640 10 client.20829 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.932+0000 7f9cab7fe640 1 -- 172.21.15.5:0/890093865 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dbe45e860 con 0x557dbe505e60 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.932+0000 7f9cab7fe640 10 client.20829 handle_command_reply: tid=0 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.932+0000 7f9cd4e0b740 1 client.20829 shutdown 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.932+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/890093865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4ecb40 msgr2=0x557dbe4f3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.932+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4ecb40 0x557dbe4f3080 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400d690 tx=0x7f9cb400d2a0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.933+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/890093865 shutdown_connections 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.933+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557dbe505e60 0x557dbe508250 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.933+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4f6370 0x557dbe4f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.933+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4ecb40 0x557dbe4f3080 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.933+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/890093865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4eb5d0 0x557dbe4f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.933+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/890093865 >> 172.21.15.5:0/890093865 conn(0x557dbe4b4310 msgr2=0x557dbe4ed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:15.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.933+0000 7f9cab7fe640 0 client.20829 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:25:15.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.933+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/890093865 shutdown_connections 2024-09-19T06:25:15.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.933+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/890093865 wait complete. 2024-09-19T06:25:15.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.933+0000 7f9cd4e0b740 20 client.20829 trim_cache size 0 max 16384 2024-09-19T06:25:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9cd4e0b740 1 Processor -- start 2024-09-19T06:25:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9cd4e0b740 1 -- start start 2024-09-19T06:25:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9cd4e0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4eb5d0 0x557dbe4f2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9cd4e0b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4ecb40 0x557dbe50ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9cd4e0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4f6370 0x557dbe50b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9cd4e0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b3c90 con 0x557dbe4eb5d0 2024-09-19T06:25:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9cd4e0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b5830 con 0x557dbe4f6370 2024-09-19T06:25:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9cd4e0b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b4360 con 0x557dbe4ecb40 2024-09-19T06:25:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9ccbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4eb5d0 0x557dbe4f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9ccbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4eb5d0 0x557dbe4f2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35802/0 (socket says 172.21.15.5:35802) 2024-09-19T06:25:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9ccbfff640 1 -- 172.21.15.5:0/1866422704 learned_addr learned my addr 172.21.15.5:0/1866422704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9cd0fcf640 1 --2- 172.21.15.5:0/1866422704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4f6370 0x557dbe50b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/1866422704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4ecb40 0x557dbe50ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9ccbfff640 1 -- 172.21.15.5:0/1866422704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4ecb40 msgr2=0x557dbe50ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.953+0000 7f9ccbfff640 1 --2- 172.21.15.5:0/1866422704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4ecb40 0x557dbe50ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.954+0000 7f9ccbfff640 1 -- 172.21.15.5:0/1866422704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4f6370 msgr2=0x557dbe50b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.954+0000 7f9ccbfff640 1 --2- 172.21.15.5:0/1866422704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4f6370 0x557dbe50b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.954+0000 7f9ccbfff640 1 -- 172.21.15.5:0/1866422704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbe4f15d0 con 0x557dbe4eb5d0 2024-09-19T06:25:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.954+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/1866422704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4ecb40 0x557dbe50ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:25:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.954+0000 7f9cd0fcf640 1 --2- 172.21.15.5:0/1866422704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4f6370 0x557dbe50b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.954+0000 7f9ccbfff640 1 --2- 172.21.15.5:0/1866422704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4eb5d0 0x557dbe4f2060 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0041e0 tx=0x7f9cbc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b38c2c9ca30a50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.954+0000 7f9cc1ffb640 1 -- 172.21.15.5:0/1866422704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc004460 con 0x557dbe4eb5d0 2024-09-19T06:25:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.954+0000 7f9cc1ffb640 1 -- 172.21.15.5:0/1866422704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cbc0054d0 con 0x557dbe4eb5d0 2024-09-19T06:25:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.954+0000 7f9cc1ffb640 1 -- 172.21.15.5:0/1866422704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc02b6d0 con 0x557dbe4eb5d0 2024-09-19T06:25:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.955+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/1866422704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4eb5d0 msgr2=0x557dbe4f2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.955+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/1866422704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4eb5d0 0x557dbe4f2060 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0041e0 tx=0x7f9cbc01b000 comp rx=0 tx=0).stop 2024-09-19T06:25:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.955+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/1866422704 shutdown_connections 2024-09-19T06:25:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.955+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/1866422704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4f6370 0x557dbe50b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.955+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/1866422704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4ecb40 0x557dbe50ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.955+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/1866422704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4eb5d0 0x557dbe4f2060 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.955+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/1866422704 >> 172.21.15.5:0/1866422704 conn(0x557dbe4b4310 msgr2=0x557dbe4f1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.955+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/1866422704 shutdown_connections 2024-09-19T06:25:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.955+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/1866422704 wait complete. 2024-09-19T06:25:15.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.956+0000 7f9cd4e0b740 1 -- start start 2024-09-19T06:25:15.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.956+0000 7f9cd4e0b740 1 Processor -- start 2024-09-19T06:25:15.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.956+0000 7f9cd4e0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4eb5d0 0x557dbe50b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.956+0000 7f9cd4e0b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4ecb40 0x557dbe508640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.956+0000 7f9cd4e0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4f6370 0x557dbe508b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.956+0000 7f9cd4e0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b5160 con 0x557dbe4f6370 2024-09-19T06:25:15.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9cd4e0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b4a60 con 0x557dbe4eb5d0 2024-09-19T06:25:15.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9cd4e0b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b1c90 con 0x557dbe4ecb40 2024-09-19T06:25:15.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9ccbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4eb5d0 0x557dbe50b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9ccbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4eb5d0 0x557dbe50b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40266/0 (socket says 172.21.15.5:40266) 2024-09-19T06:25:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9ccbfff640 1 -- 172.21.15.5:0/2880267705 learned_addr learned my addr 172.21.15.5:0/2880267705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9cd4e0b740 10 client.0 _ll_register_callbacks cb 0x7f9cc40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4ecb40 0x557dbe508640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9cd0fcf640 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4f6370 0x557dbe508b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9ccbfff640 1 -- 172.21.15.5:0/2880267705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4ecb40 msgr2=0x557dbe508640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9ccbfff640 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4ecb40 0x557dbe508640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9ccbfff640 1 -- 172.21.15.5:0/2880267705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4f6370 msgr2=0x557dbe508b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9ccbfff640 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4f6370 0x557dbe508b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9ccbfff640 1 -- 172.21.15.5:0/2880267705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbe5090c0 con 0x557dbe4eb5d0 2024-09-19T06:25:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4ecb40 0x557dbe508640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9ccbfff640 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4eb5d0 0x557dbe50b2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc004230 tx=0x7f9cbc004ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d957322071de8e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9cab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.957+0000 7f9cab7fe640 1 -- 172.21.15.5:0/2880267705 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc004460 con 0x557dbe4eb5d0 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.958+0000 7f9cab7fe640 1 -- 172.21.15.5:0/2880267705 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f9cbc004d40 con 0x557dbe4eb5d0 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.958+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/2880267705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cbc0054d0 con 0x557dbe4eb5d0 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.958+0000 7f9cab7fe640 1 -- 172.21.15.5:0/2880267705 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f9cbc0048c0 con 0x557dbe4eb5d0 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.958+0000 7f9cd4e0b740 10 client.30215 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.958+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/2880267705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x557dbe570490 con 0x557dbe4eb5d0 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.958+0000 7f9cab7fe640 1 -- 172.21.15.5:0/2880267705 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cbc0054d0 con 0x557dbe4eb5d0 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.958+0000 7f9cab7fe640 1 -- 172.21.15.5:0/2880267705 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f9cbc02b9b0 con 0x557dbe4eb5d0 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.958+0000 7f9cd4e0b740 10 client.30215 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.958+0000 7f9cd4e0b740 10 client.30215 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.959+0000 7f9cd4e0b740 20 client.30215 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.959+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557dbe570af0 0x557dbe572f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.959+0000 7f9cd4e0b740 4 client.30215 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.959+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/2880267705 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557dbe458d70 con 0x557dbe570af0 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.959+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557dbe570af0 0x557dbe572f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.959+0000 7f9cc3fff640 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557dbe570af0 0x557dbe572f10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5b0a160ffa35f49b server_cookie=8b212ab23071c7b5 in_seq=0 out_seq=0 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.959+0000 7f9cab7fe640 10 client.30215 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.959+0000 7f9cab7fe640 1 -- 172.21.15.5:0/2880267705 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557dbe458d70 con 0x557dbe570af0 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.959+0000 7f9cab7fe640 10 client.30215 handle_command_reply: tid=0 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.959+0000 7f9cd4e0b740 1 client.30215 shutdown 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.960+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/2880267705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4eb5d0 msgr2=0x557dbe50b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.960+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4eb5d0 0x557dbe50b2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc004230 tx=0x7f9cbc004ba0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.960+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/2880267705 shutdown_connections 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.960+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x557dbe570af0 0x557dbe572f10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.960+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dbe4f6370 0x557dbe508b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.960+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557dbe4ecb40 0x557dbe508640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.960+0000 7f9cab7fe640 0 client.30215 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:25:15.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.960+0000 7f9cd4e0b740 1 --2- 172.21.15.5:0/2880267705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557dbe4eb5d0 0x557dbe50b2f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.960+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/2880267705 >> 172.21.15.5:0/2880267705 conn(0x557dbe4b4310 msgr2=0x557dbe4f2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:15.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.960+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/2880267705 shutdown_connections 2024-09-19T06:25:15.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.960+0000 7f9cd4e0b740 1 -- 172.21.15.5:0/2880267705 wait complete. 2024-09-19T06:25:15.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.960+0000 7f9cd4e0b740 20 client.30215 trim_cache size 0 max 16384 2024-09-19T06:25:15.966 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:25:15.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.961+0000 7f9cd2a32640 1 -- 172.21.15.5:0/820450238 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9cb80a3430 msgr2=0x7f9cb80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:15.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.961+0000 7f9cd2a32640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9cb80a3430 0x7f9cb80a58f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x557dbe4e8ac0 tx=0x7f9cbc005f90 comp rx=0 tx=0).stop 2024-09-19T06:25:15.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.961+0000 7f9cd2a32640 1 -- 172.21.15.5:0/820450238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc102ef0 msgr2=0x7f9ccc19e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:15.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.961+0000 7f9cd2a32640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc102ef0 0x7f9ccc19e830 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400dfb0 tx=0x7f9cc400b630 comp rx=0 tx=0).stop 2024-09-19T06:25:15.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.961+0000 7f9cd2a32640 1 -- 172.21.15.5:0/820450238 shutdown_connections 2024-09-19T06:25:15.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.961+0000 7f9cd2a32640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc102ef0 0x7f9ccc19e830 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.961+0000 7f9cd2a32640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ccc102570 0x7f9ccc1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.961+0000 7f9cd2a32640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9cb80a3430 0x7f9cb80a58f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.961+0000 7f9cd2a32640 1 --2- 172.21.15.5:0/820450238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ccc100e20 0x7f9ccc1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:15.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.961+0000 7f9cd2a32640 1 -- 172.21.15.5:0/820450238 >> 172.21.15.5:0/820450238 conn(0x7f9ccc0fc7d0 msgr2=0x7f9ccc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:15.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.961+0000 7f9cd2a32640 1 -- 172.21.15.5:0/820450238 shutdown_connections 2024-09-19T06:25:15.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:15.961+0000 7f9cd2a32640 1 -- 172.21.15.5:0/820450238 wait complete. 2024-09-19T06:25:16.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84179 8.60 MB/sec execute 400 sec latency 94.589 ms 2024-09-19T06:25:16.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:16 smithi157 ceph-mon[35346]: pgmap v693: 97 pgs: 97 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 308 op/s 2024-09-19T06:25:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:16 smithi005 ceph-mon[30955]: pgmap v693: 97 pgs: 97 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 308 op/s 2024-09-19T06:25:16.895 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:16 smithi049 ceph-mon[34640]: pgmap v693: 97 pgs: 97 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 308 op/s 2024-09-19T06:25:17.072 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:25:17.434 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:25:17.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84344 8.60 MB/sec execute 401 sec latency 56.277 ms 2024-09-19T06:25:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.021+0000 7f829d436640 1 -- 172.21.15.5:0/4206451181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8298102ef0 msgr2=0x7f829810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.021+0000 7f829d436640 1 --2- 172.21.15.5:0/4206451181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8298102ef0 0x7f829810da40 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f8290009ca0 tx=0x7f829002a970 comp rx=0 tx=0).stop 2024-09-19T06:25:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.021+0000 7f829d436640 1 -- 172.21.15.5:0/4206451181 shutdown_connections 2024-09-19T06:25:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.021+0000 7f829d436640 1 --2- 172.21.15.5:0/4206451181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8298102ef0 0x7f829810da40 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.021+0000 7f829d436640 1 --2- 172.21.15.5:0/4206451181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8298102570 0x7f82981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.021+0000 7f829d436640 1 --2- 172.21.15.5:0/4206451181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8298100e20 0x7f8298101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.021+0000 7f829d436640 1 -- 172.21.15.5:0/4206451181 >> 172.21.15.5:0/4206451181 conn(0x7f82980fc7d0 msgr2=0x7f82980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.021+0000 7f829d436640 1 -- 172.21.15.5:0/4206451181 shutdown_connections 2024-09-19T06:25:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.021+0000 7f829d436640 1 -- 172.21.15.5:0/4206451181 wait complete. 2024-09-19T06:25:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.022+0000 7f829d436640 1 Processor -- start 2024-09-19T06:25:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.022+0000 7f829d436640 1 -- start start 2024-09-19T06:25:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.023+0000 7f829d436640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8298100e20 0x7f829819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.023+0000 7f829d436640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8298102570 0x7f829819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.023+0000 7f829d436640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8298102ef0 0x7f829819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.023+0000 7f829d436640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8298110e70 con 0x7f8298102570 2024-09-19T06:25:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.023+0000 7f829d436640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8298110cf0 con 0x7f8298102ef0 2024-09-19T06:25:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.023+0000 7f829d436640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8298110ff0 con 0x7f8298100e20 2024-09-19T06:25:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.023+0000 7f82977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8298102ef0 0x7f829819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.023+0000 7f82977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8298102ef0 0x7f829819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40292/0 (socket says 172.21.15.5:40292) 2024-09-19T06:25:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.023+0000 7f82977fe640 1 -- 172.21.15.5:0/3447837594 learned_addr learned my addr 172.21.15.5:0/3447837594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.023+0000 7f82967fc640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8298102570 0x7f829819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.023+0000 7f8296ffd640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8298100e20 0x7f829819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.023+0000 7f82977fe640 1 -- 172.21.15.5:0/3447837594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8298100e20 msgr2=0x7f829819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.024+0000 7f82977fe640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8298100e20 0x7f829819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.024+0000 7f82977fe640 1 -- 172.21.15.5:0/3447837594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8298102570 msgr2=0x7f829819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.024+0000 7f82977fe640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8298102570 0x7f829819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.024+0000 7f82977fe640 1 -- 172.21.15.5:0/3447837594 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829819f5f0 con 0x7f8298102ef0 2024-09-19T06:25:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.024+0000 7f82977fe640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8298102ef0 0x7f829819ef10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8290006010 tx=0x7f8290003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.024+0000 7f827ffff640 1 -- 172.21.15.5:0/3447837594 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290047070 con 0x7f8298102ef0 2024-09-19T06:25:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.024+0000 7f827ffff640 1 -- 172.21.15.5:0/3447837594 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f829002f3c0 con 0x7f8298102ef0 2024-09-19T06:25:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.024+0000 7f827ffff640 1 -- 172.21.15.5:0/3447837594 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290034070 con 0x7f8298102ef0 2024-09-19T06:25:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.024+0000 7f829d436640 1 -- 172.21.15.5:0/3447837594 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829819f880 con 0x7f8298102ef0 2024-09-19T06:25:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.024+0000 7f829d436640 1 -- 172.21.15.5:0/3447837594 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82981a71c0 con 0x7f8298102ef0 2024-09-19T06:25:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.027+0000 7f827ffff640 1 -- 172.21.15.5:0/3447837594 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8290037020 con 0x7f8298102ef0 2024-09-19T06:25:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.027+0000 7f829f80f740 1 Processor -- start 2024-09-19T06:25:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.028+0000 7f829f80f740 1 -- start start 2024-09-19T06:25:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.028+0000 7f829f80f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c81b40 0x5574a8c80090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.028+0000 7f829f80f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c805d0 0x5574a8c80a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.028+0000 7f829f80f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c93b80 0x5574a8c95f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.028+0000 7f829f80f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574a8be62f0 con 0x5574a8c81b40 2024-09-19T06:25:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.028+0000 7f829f80f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5574a8b8dcf0 con 0x5574a8c805d0 2024-09-19T06:25:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.028+0000 7f829f80f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5574a8be22b0 con 0x5574a8c93b80 2024-09-19T06:25:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.028+0000 7f8296ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c805d0 0x5574a8c80a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.028+0000 7f82967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c81b40 0x5574a8c80090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.028+0000 7f8296ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c805d0 0x5574a8c80a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40294/0 (socket says 172.21.15.5:40294) 2024-09-19T06:25:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.028+0000 7f8296ffd640 1 -- 172.21.15.5:0/1103753969 learned_addr learned my addr 172.21.15.5:0/1103753969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.028+0000 7f82967fc640 1 --2- 172.21.15.5:0/1103753969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c93b80 0x5574a8c95f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.029+0000 7f82967fc640 1 -- 172.21.15.5:0/1103753969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c93b80 msgr2=0x5574a8c95f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.029+0000 7f82967fc640 1 --2- 172.21.15.5:0/1103753969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c93b80 0x5574a8c95f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.029+0000 7f82967fc640 1 -- 172.21.15.5:0/1103753969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c805d0 msgr2=0x5574a8c80a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.029+0000 7f82967fc640 1 --2- 172.21.15.5:0/1103753969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c805d0 0x5574a8c80a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.029+0000 7f82967fc640 1 -- 172.21.15.5:0/1103753969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a8b723a0 con 0x5574a8c81b40 2024-09-19T06:25:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.029+0000 7f8296ffd640 1 --2- 172.21.15.5:0/1103753969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c805d0 0x5574a8c80a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:25:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.029+0000 7f827ffff640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f82800a33b0 0x7f82800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.029+0000 7f827ffff640 1 -- 172.21.15.5:0/3447837594 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f82900f0760 con 0x7f8298102ef0 2024-09-19T06:25:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.029+0000 7f82967fc640 1 --2- 172.21.15.5:0/1103753969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c81b40 0x5574a8c80090 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f8298104bf0 tx=0x7f828c009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a592c675f8181c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.029+0000 7f8296ffd640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f82800a33b0 0x7f82800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.029+0000 7f827d7fa640 1 -- 172.21.15.5:0/1103753969 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c0155e0 con 0x5574a8c81b40 2024-09-19T06:25:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.030+0000 7f827d7fa640 1 -- 172.21.15.5:0/1103753969 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f828c01d070 con 0x5574a8c81b40 2024-09-19T06:25:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.030+0000 7f827d7fa640 1 -- 172.21.15.5:0/1103753969 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c0188e0 con 0x5574a8c81b40 2024-09-19T06:25:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.030+0000 7f8296ffd640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f82800a33b0 0x7f82800a5870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5574a8c7dac0 tx=0x7f8288005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.030+0000 7f829f80f740 1 -- 172.21.15.5:0/1103753969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c81b40 msgr2=0x5574a8c80090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.030+0000 7f829f80f740 1 --2- 172.21.15.5:0/1103753969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c81b40 0x5574a8c80090 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f8298104bf0 tx=0x7f828c009490 comp rx=0 tx=0).stop 2024-09-19T06:25:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.030+0000 7f829f80f740 1 -- 172.21.15.5:0/1103753969 shutdown_connections 2024-09-19T06:25:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.030+0000 7f829f80f740 1 --2- 172.21.15.5:0/1103753969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c93b80 0x5574a8c95f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.030+0000 7f829f80f740 1 --2- 172.21.15.5:0/1103753969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c805d0 0x5574a8c80a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.030+0000 7f829f80f740 1 --2- 172.21.15.5:0/1103753969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c81b40 0x5574a8c80090 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.031+0000 7f829f80f740 1 -- 172.21.15.5:0/1103753969 >> 172.21.15.5:0/1103753969 conn(0x5574a8c49310 msgr2=0x5574a8c7cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.031+0000 7f829f80f740 1 -- 172.21.15.5:0/1103753969 shutdown_connections 2024-09-19T06:25:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.031+0000 7f829f80f740 1 -- 172.21.15.5:0/1103753969 wait complete. 2024-09-19T06:25:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.031+0000 7f829f80f740 1 -- start start 2024-09-19T06:25:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.031+0000 7f829f80f740 1 Processor -- start 2024-09-19T06:25:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f829f80f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 0x5574a8c93570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f829f80f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c81b40 0x5574a8c90820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f829f80f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c93b80 0x5574a8c90e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f8296ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 0x5574a8c93570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f8296ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 0x5574a8c93570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41424/0 (socket says 172.21.15.5:41424) 2024-09-19T06:25:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f8296ffd640 1 -- 172.21.15.5:0/1748563862 learned_addr learned my addr 172.21.15.5:0/1748563862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f829f80f740 1 -- 172.21.15.5:0/1748563862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574a8c96820 con 0x5574a8c93b80 2024-09-19T06:25:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f829f80f740 1 -- 172.21.15.5:0/1748563862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5574a8ba1060 con 0x5574a8c81b40 2024-09-19T06:25:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f829f80f740 1 -- 172.21.15.5:0/1748563862 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5574a8c969a0 con 0x5574a8c805d0 2024-09-19T06:25:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f829f80f740 10 client.0 _ll_register_callbacks cb 0x5574a88f48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f8296ffd640 1 -- 172.21.15.5:0/1748563862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c81b40 msgr2=0x5574a8c90820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f8296ffd640 1 --2- 172.21.15.5:0/1748563862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c81b40 0x5574a8c90820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f8296ffd640 1 -- 172.21.15.5:0/1748563862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c93b80 msgr2=0x5574a8c90e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:25:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f8296ffd640 1 --2- 172.21.15.5:0/1748563862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c93b80 0x5574a8c90e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.032+0000 7f8296ffd640 1 -- 172.21.15.5:0/1748563862 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a89573c0 con 0x5574a8c805d0 2024-09-19T06:25:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.033+0000 7f829f80f740 1 -- 172.21.15.5:0/1748563862 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f828c01d070 con 0x5574a8c805d0 2024-09-19T06:25:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.033+0000 7f8296ffd640 1 --2- 172.21.15.5:0/1748563862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 0x5574a8c93570 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8288009590 tx=0x7f82880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d1e1b1c12942dbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.033+0000 7f827aff5640 10 client.30151 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:25:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.033+0000 7f827aff5640 1 -- 172.21.15.5:0/1748563862 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288011b20 con 0x5574a8c805d0 2024-09-19T06:25:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.033+0000 7f827aff5640 1 -- 172.21.15.5:0/1748563862 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f8288011cc0 con 0x5574a8c805d0 2024-09-19T06:25:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.034+0000 7f827aff5640 1 -- 172.21.15.5:0/1748563862 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8288020420 con 0x5574a8c805d0 2024-09-19T06:25:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.034+0000 7f829f80f740 10 client.30151 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.034+0000 7f829f80f740 1 -- 172.21.15.5:0/1748563862 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=60}) -- 0x5574a8ca2290 con 0x5574a8c805d0 2024-09-19T06:25:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.034+0000 7f827aff5640 1 -- 172.21.15.5:0/1748563862 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f82880205c0 con 0x5574a8c805d0 2024-09-19T06:25:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.034+0000 7f829f80f740 10 client.30151 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.035+0000 7f829f80f740 10 client.30151 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:25:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.035+0000 7f829f80f740 20 client.30151 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.035+0000 7f829f80f740 1 --2- 172.21.15.5:0/1748563862 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5574a8ca3690 0x5574a8ca5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.035+0000 7f829f80f740 4 client.30151 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.035+0000 7f829f80f740 1 -- 172.21.15.5:0/1748563862 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574a8bf3860 con 0x5574a8ca3690 2024-09-19T06:25:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.035+0000 7f82967fc640 1 --2- 172.21.15.5:0/1748563862 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5574a8ca3690 0x5574a8ca5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.037+0000 7f827aff5640 10 client.30151 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:25:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.037+0000 7f82967fc640 1 --2- 172.21.15.5:0/1748563862 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5574a8ca3690 0x5574a8ca5b00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f8f5a3efaa0c26c server_cookie=b9b3fdfb4901e8e0 in_seq=0 out_seq=0 2024-09-19T06:25:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.038+0000 7f827aff5640 1 -- 172.21.15.5:0/1748563862 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574a8bf3860 con 0x5574a8ca3690 2024-09-19T06:25:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.038+0000 7f827aff5640 10 client.30151 handle_command_reply: tid=0 2024-09-19T06:25:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.038+0000 7f829f80f740 1 client.30151 shutdown 2024-09-19T06:25:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.038+0000 7f829f80f740 1 -- 172.21.15.5:0/1748563862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 msgr2=0x5574a8c93570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.038+0000 7f829f80f740 1 --2- 172.21.15.5:0/1748563862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 0x5574a8c93570 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8288009590 tx=0x7f82880046c0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.038+0000 7f829f80f740 1 -- 172.21.15.5:0/1748563862 shutdown_connections 2024-09-19T06:25:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.038+0000 7f829f80f740 1 --2- 172.21.15.5:0/1748563862 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5574a8ca3690 0x5574a8ca5b00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.038+0000 7f829f80f740 1 --2- 172.21.15.5:0/1748563862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c93b80 0x5574a8c90e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.038+0000 7f829f80f740 1 --2- 172.21.15.5:0/1748563862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c81b40 0x5574a8c90820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.039+0000 7f829f80f740 1 --2- 172.21.15.5:0/1748563862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 0x5574a8c93570 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.039+0000 7f827aff5640 0 client.30151 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:25:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.039+0000 7f829f80f740 1 -- 172.21.15.5:0/1748563862 >> 172.21.15.5:0/1748563862 conn(0x5574a8c49310 msgr2=0x5574a8c82930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.039+0000 7f829f80f740 1 -- 172.21.15.5:0/1748563862 shutdown_connections 2024-09-19T06:25:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.039+0000 7f829f80f740 1 -- 172.21.15.5:0/1748563862 wait complete. 2024-09-19T06:25:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.039+0000 7f829f80f740 20 client.30151 trim_cache size 0 max 16384 2024-09-19T06:25:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.061+0000 7f829f80f740 1 Processor -- start 2024-09-19T06:25:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.061+0000 7f829f80f740 1 -- start start 2024-09-19T06:25:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.061+0000 7f829f80f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c805d0 0x5574a8c907f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.061+0000 7f829f80f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c81b40 0x5574a8c9fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.061+0000 7f829f80f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c93b80 0x5574a8cab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.061+0000 7f829f80f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82981ab730 con 0x5574a8c805d0 2024-09-19T06:25:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.061+0000 7f829f80f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82981ad2d0 con 0x5574a8c93b80 2024-09-19T06:25:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.061+0000 7f829f80f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f82981abe00 con 0x5574a8c81b40 2024-09-19T06:25:18.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.061+0000 7f8296ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c805d0 0x5574a8c907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.061+0000 7f8296ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c805d0 0x5574a8c907f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35840/0 (socket says 172.21.15.5:35840) 2024-09-19T06:25:18.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.061+0000 7f8296ffd640 1 -- 172.21.15.5:0/1865779166 learned_addr learned my addr 172.21.15.5:0/1865779166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:18.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.061+0000 7f82977fe640 1 --2- 172.21.15.5:0/1865779166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c93b80 0x5574a8cab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.062+0000 7f8296ffd640 1 -- 172.21.15.5:0/1865779166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c81b40 msgr2=0x5574a8c9fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:18.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.062+0000 7f8296ffd640 1 --2- 172.21.15.5:0/1865779166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c81b40 0x5574a8c9fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.062+0000 7f8296ffd640 1 -- 172.21.15.5:0/1865779166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c93b80 msgr2=0x5574a8cab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:18.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.062+0000 7f8296ffd640 1 --2- 172.21.15.5:0/1865779166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c93b80 0x5574a8cab3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.062+0000 7f8296ffd640 1 -- 172.21.15.5:0/1865779166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a8c8ed70 con 0x5574a8c805d0 2024-09-19T06:25:18.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.062+0000 7f8296ffd640 1 --2- 172.21.15.5:0/1865779166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c805d0 0x5574a8c907f0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f8288020a50 tx=0x7f8288021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6900ebb2d2e22d0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:18.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.063+0000 7f827cff9640 1 -- 172.21.15.5:0/1865779166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288020e50 con 0x5574a8c805d0 2024-09-19T06:25:18.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.063+0000 7f827cff9640 1 -- 172.21.15.5:0/1865779166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f828c01d070 con 0x5574a8c805d0 2024-09-19T06:25:18.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.063+0000 7f827cff9640 1 -- 172.21.15.5:0/1865779166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828803e460 con 0x5574a8c805d0 2024-09-19T06:25:18.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.063+0000 7f829f80f740 1 -- 172.21.15.5:0/1865779166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c805d0 msgr2=0x5574a8c907f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:18.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.063+0000 7f829f80f740 1 --2- 172.21.15.5:0/1865779166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c805d0 0x5574a8c907f0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f8288020a50 tx=0x7f8288021000 comp rx=0 tx=0).stop 2024-09-19T06:25:18.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.064+0000 7f829f80f740 1 -- 172.21.15.5:0/1865779166 shutdown_connections 2024-09-19T06:25:18.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.064+0000 7f829f80f740 1 --2- 172.21.15.5:0/1865779166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c93b80 0x5574a8cab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.064+0000 7f829f80f740 1 --2- 172.21.15.5:0/1865779166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c81b40 0x5574a8c9fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.064+0000 7f829f80f740 1 --2- 172.21.15.5:0/1865779166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c805d0 0x5574a8c907f0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.064+0000 7f829f80f740 1 -- 172.21.15.5:0/1865779166 >> 172.21.15.5:0/1865779166 conn(0x5574a8c49310 msgr2=0x5574a8c900e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:18.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.064+0000 7f829f80f740 1 -- 172.21.15.5:0/1865779166 shutdown_connections 2024-09-19T06:25:18.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.064+0000 7f829f80f740 1 -- 172.21.15.5:0/1865779166 wait complete. 2024-09-19T06:25:18.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.064+0000 7f829f80f740 1 -- start start 2024-09-19T06:25:18.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.064+0000 7f829f80f740 1 Processor -- start 2024-09-19T06:25:18.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.065+0000 7f829f80f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 0x5574a8c979f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.065+0000 7f829f80f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c81b40 0x5574a8c97f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.065+0000 7f8296ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 0x5574a8c979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.065+0000 7f829f80f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c93b80 0x5574a8c98470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.065+0000 7f8296ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 0x5574a8c979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41444/0 (socket says 172.21.15.5:41444) 2024-09-19T06:25:18.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.065+0000 7f8296ffd640 1 -- 172.21.15.5:0/719423869 learned_addr learned my addr 172.21.15.5:0/719423869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:18.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.065+0000 7f82967fc640 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c81b40 0x5574a8c97f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.065+0000 7f829f80f740 1 -- 172.21.15.5:0/719423869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82981acc00 con 0x5574a8c81b40 2024-09-19T06:25:18.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.065+0000 7f829f80f740 1 -- 172.21.15.5:0/719423869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82981ac500 con 0x5574a8c93b80 2024-09-19T06:25:18.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.065+0000 7f829f80f740 1 -- 172.21.15.5:0/719423869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f82981a9770 con 0x5574a8c805d0 2024-09-19T06:25:18.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.065+0000 7f82977fe640 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c93b80 0x5574a8c98470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.065+0000 7f829f80f740 10 client.0 _ll_register_callbacks cb 0x7f8288009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:18.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.066+0000 7f8296ffd640 1 -- 172.21.15.5:0/719423869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c93b80 msgr2=0x5574a8c98470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:18.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.066+0000 7f8296ffd640 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c93b80 0x5574a8c98470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.066+0000 7f8296ffd640 1 -- 172.21.15.5:0/719423869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c81b40 msgr2=0x5574a8c97f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:18.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.066+0000 7f8296ffd640 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c81b40 0x5574a8c97f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.066+0000 7f8296ffd640 1 -- 172.21.15.5:0/719423869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a8d0d420 con 0x5574a8c805d0 2024-09-19T06:25:18.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.066+0000 7f82977fe640 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c93b80 0x5574a8c98470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.066+0000 7f8296ffd640 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 0x5574a8c979f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8288011e30 tx=0x7f8288011ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c47141493d289417 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.066+0000 7f827aff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.066+0000 7f827aff5640 1 -- 172.21.15.5:0/719423869 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288020e50 con 0x5574a8c805d0 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.066+0000 7f827aff5640 1 -- 172.21.15.5:0/719423869 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f828803a070 con 0x5574a8c805d0 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.066+0000 7f829f80f740 1 -- 172.21.15.5:0/719423869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f828c01d070 con 0x5574a8c805d0 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.067+0000 7f827aff5640 1 -- 172.21.15.5:0/719423869 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f82880043e0 con 0x5574a8c805d0 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.067+0000 7f829f80f740 10 client.30154 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.067+0000 7f829f80f740 1 -- 172.21.15.5:0/719423869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5574a8d0e6c0 con 0x5574a8c805d0 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.067+0000 7f827aff5640 1 -- 172.21.15.5:0/719423869 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f828c01d070 con 0x5574a8c805d0 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.067+0000 7f827aff5640 1 -- 172.21.15.5:0/719423869 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f8288004a90 con 0x5574a8c805d0 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.067+0000 7f829f80f740 10 client.30154 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.067+0000 7f829f80f740 10 client.30154 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.067+0000 7f829f80f740 20 client.30154 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:18.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.067+0000 7f829f80f740 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5574a8ca3690 0x5574a8d10cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:18.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.067+0000 7f829f80f740 4 client.30154 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:25:18.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.067+0000 7f829f80f740 1 -- 172.21.15.5:0/719423869 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574a8bedd70 con 0x5574a8ca3690 2024-09-19T06:25:18.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.067+0000 7f82967fc640 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5574a8ca3690 0x5574a8d10cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:18.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.068+0000 7f82967fc640 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5574a8ca3690 0x5574a8d10cd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15a0c24783e3171 server_cookie=c27c08bab0554d64 in_seq=0 out_seq=0 2024-09-19T06:25:18.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.068+0000 7f827aff5640 10 client.30154 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:25:18.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.068+0000 7f827aff5640 1 -- 172.21.15.5:0/719423869 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574a8bedd70 con 0x5574a8ca3690 2024-09-19T06:25:18.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.068+0000 7f827aff5640 10 client.30154 handle_command_reply: tid=0 2024-09-19T06:25:18.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.068+0000 7f829f80f740 1 client.30154 shutdown 2024-09-19T06:25:18.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.069+0000 7f829f80f740 1 -- 172.21.15.5:0/719423869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 msgr2=0x5574a8c979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:18.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.069+0000 7f829f80f740 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 0x5574a8c979f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8288011e30 tx=0x7f8288011ad0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.069+0000 7f829f80f740 1 -- 172.21.15.5:0/719423869 shutdown_connections 2024-09-19T06:25:18.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.069+0000 7f829f80f740 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5574a8ca3690 0x5574a8d10cd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.069+0000 7f829f80f740 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574a8c93b80 0x5574a8c98470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.069+0000 7f829f80f740 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a8c81b40 0x5574a8c97f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.069+0000 7f827aff5640 0 client.30154 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:25:18.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.069+0000 7f829f80f740 1 --2- 172.21.15.5:0/719423869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a8c805d0 0x5574a8c979f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.069+0000 7f829f80f740 1 -- 172.21.15.5:0/719423869 >> 172.21.15.5:0/719423869 conn(0x5574a8c49310 msgr2=0x5574a8c7b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:18.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.069+0000 7f829f80f740 1 -- 172.21.15.5:0/719423869 shutdown_connections 2024-09-19T06:25:18.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.069+0000 7f829f80f740 1 -- 172.21.15.5:0/719423869 wait complete. 2024-09-19T06:25:18.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.069+0000 7f829f80f740 20 client.30154 trim_cache size 0 max 16384 2024-09-19T06:25:18.076 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.070+0000 7f829d436640 1 -- 172.21.15.5:0/3447837594 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f82800a33b0 msgr2=0x7f82800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.070+0000 7f829d436640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f82800a33b0 0x7f82800a5870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5574a8c7dac0 tx=0x7f8288005f90 comp rx=0 tx=0).stop 2024-09-19T06:25:18.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.070+0000 7f829d436640 1 -- 172.21.15.5:0/3447837594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8298102ef0 msgr2=0x7f829819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:18.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.070+0000 7f829d436640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8298102ef0 0x7f829819ef10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8290006010 tx=0x7f8290003ba0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.070+0000 7f829d436640 1 -- 172.21.15.5:0/3447837594 shutdown_connections 2024-09-19T06:25:18.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.070+0000 7f829d436640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8298102ef0 0x7f829819ef10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.070+0000 7f829d436640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8298102570 0x7f829819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.070+0000 7f829d436640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f82800a33b0 0x7f82800a5870 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.070+0000 7f829d436640 1 --2- 172.21.15.5:0/3447837594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8298100e20 0x7f829819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:18.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.070+0000 7f829d436640 1 -- 172.21.15.5:0/3447837594 >> 172.21.15.5:0/3447837594 conn(0x7f82980fc7d0 msgr2=0x7f829806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:18.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.070+0000 7f829d436640 1 -- 172.21.15.5:0/3447837594 shutdown_connections 2024-09-19T06:25:18.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:18.070+0000 7f829d436640 1 -- 172.21.15.5:0/3447837594 wait complete. 2024-09-19T06:25:18.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84489 8.59 MB/sec execute 402 sec latency 105.708 ms 2024-09-19T06:25:18.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:18 smithi157 ceph-mon[35346]: pgmap v694: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.5 MiB/s wr, 251 op/s 2024-09-19T06:25:18.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:18 smithi005 ceph-mon[30955]: pgmap v694: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.5 MiB/s wr, 251 op/s 2024-09-19T06:25:18.859 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:25:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:18 smithi049 ceph-mon[34640]: pgmap v694: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.5 MiB/s wr, 251 op/s 2024-09-19T06:25:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:25:19.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84659 8.60 MB/sec execute 403 sec latency 80.319 ms 2024-09-19T06:25:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.709+0000 7fb2b35e4640 1 -- 172.21.15.5:0/772907950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac112980 msgr2=0x7fb2ac114d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.709+0000 7fb2b35e4640 1 --2- 172.21.15.5:0/772907950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac112980 0x7fb2ac114d70 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009ca0 tx=0x7fb2a402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.710+0000 7fb2b35e4640 1 -- 172.21.15.5:0/772907950 shutdown_connections 2024-09-19T06:25:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.710+0000 7fb2b35e4640 1 --2- 172.21.15.5:0/772907950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac112980 0x7fb2ac114d70 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.710+0000 7fb2b35e4640 1 --2- 172.21.15.5:0/772907950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2ac077040 0x7fb2ac112440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.710+0000 7fb2b35e4640 1 --2- 172.21.15.5:0/772907950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2ac076720 0x7fb2ac076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.710+0000 7fb2b35e4640 1 -- 172.21.15.5:0/772907950 >> 172.21.15.5:0/772907950 conn(0x7fb2ac0ff980 msgr2=0x7fb2ac101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.710+0000 7fb2b35e4640 1 -- 172.21.15.5:0/772907950 shutdown_connections 2024-09-19T06:25:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.710+0000 7fb2b35e4640 1 -- 172.21.15.5:0/772907950 wait complete. 2024-09-19T06:25:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.710+0000 7fb2b35e4640 1 Processor -- start 2024-09-19T06:25:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b35e4640 1 -- start start 2024-09-19T06:25:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b35e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2ac076720 0x7fb2ac1a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b35e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac077040 0x7fb2ac1a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b35e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2ac112980 0x7fb2ac19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b35e4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac1183c0 con 0x7fb2ac077040 2024-09-19T06:25:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b35e4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac118240 con 0x7fb2ac076720 2024-09-19T06:25:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b35e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac118540 con 0x7fb2ac112980 2024-09-19T06:25:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b0b7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac077040 0x7fb2ac1a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b0b7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac077040 0x7fb2ac1a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35876/0 (socket says 172.21.15.5:35876) 2024-09-19T06:25:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b0b7f640 1 -- 172.21.15.5:0/793342602 learned_addr learned my addr 172.21.15.5:0/793342602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b1380640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2ac076720 0x7fb2ac1a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b1b81640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2ac112980 0x7fb2ac19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b0b7f640 1 -- 172.21.15.5:0/793342602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2ac112980 msgr2=0x7fb2ac19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2ac112980 0x7fb2ac19e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b0b7f640 1 -- 172.21.15.5:0/793342602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2ac076720 msgr2=0x7fb2ac1a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.711+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2ac076720 0x7fb2ac1a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.712+0000 7fb2b0b7f640 1 -- 172.21.15.5:0/793342602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ac19ef80 con 0x7fb2ac077040 2024-09-19T06:25:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.712+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac077040 0x7fb2ac1a4b80 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800eb30 tx=0x7fb2a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.712+0000 7fb2a27fc640 1 -- 172.21.15.5:0/793342602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8010430 con 0x7fb2ac077040 2024-09-19T06:25:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.712+0000 7fb2b35e4640 1 -- 172.21.15.5:0/793342602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ac19f270 con 0x7fb2ac077040 2024-09-19T06:25:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.712+0000 7fb2a27fc640 1 -- 172.21.15.5:0/793342602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2a8010a80 con 0x7fb2ac077040 2024-09-19T06:25:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.712+0000 7fb2a27fc640 1 -- 172.21.15.5:0/793342602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8018af0 con 0x7fb2ac077040 2024-09-19T06:25:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.712+0000 7fb2b35e4640 1 -- 172.21.15.5:0/793342602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ac1ab490 con 0x7fb2ac077040 2024-09-19T06:25:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.716+0000 7fb2a27fc640 1 -- 172.21.15.5:0/793342602 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb2a8010c30 con 0x7fb2ac077040 2024-09-19T06:25:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.716+0000 7fb2b645c740 1 Processor -- start 2024-09-19T06:25:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.716+0000 7fb2b645c740 1 -- start start 2024-09-19T06:25:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.717+0000 7fb2b645c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e48250ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.717+0000 7fb2b645c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e48251410 0x560e4825b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.717+0000 7fb2b645c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4825bcb0 0x560e4825e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.717+0000 7fb2b645c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560e481b52f0 con 0x560e4825bcb0 2024-09-19T06:25:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.717+0000 7fb2b645c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560e4815d270 con 0x560e48250af0 2024-09-19T06:25:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.717+0000 7fb2b645c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560e481b12b0 con 0x560e48251410 2024-09-19T06:25:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.717+0000 7fb2b1b81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e48250ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.717+0000 7fb2b1b81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e48250ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40358/0 (socket says 172.21.15.5:40358) 2024-09-19T06:25:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.717+0000 7fb2b1b81640 1 -- 172.21.15.5:0/1225893182 learned_addr learned my addr 172.21.15.5:0/1225893182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.717+0000 7fb2b1380640 1 --2- 172.21.15.5:0/1225893182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e48251410 0x560e4825b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.717+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/1225893182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4825bcb0 0x560e4825e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.717+0000 7fb2a27fc640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb2980a3430 0x7fb2980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.718+0000 7fb2b1b81640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb2980a3430 0x7fb2980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.718+0000 7fb2a27fc640 1 -- 172.21.15.5:0/793342602 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7fb2a80d0270 con 0x7fb2ac077040 2024-09-19T06:25:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.718+0000 7fb2b1b81640 1 -- 172.21.15.5:0/1225893182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e48251410 msgr2=0x560e4825b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.718+0000 7fb2b1b81640 1 --2- 172.21.15.5:0/1225893182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e48251410 0x560e4825b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.718+0000 7fb2b1b81640 1 -- 172.21.15.5:0/1225893182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4825bcb0 msgr2=0x560e4825e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.718+0000 7fb2b1b81640 1 --2- 172.21.15.5:0/1225893182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4825bcb0 0x560e4825e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.718+0000 7fb2b1b81640 1 -- 172.21.15.5:0/1225893182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e4815c340 con 0x560e48250af0 2024-09-19T06:25:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.718+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/1225893182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4825bcb0 0x560e4825e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.719+0000 7fb2b1b81640 1 --2- 172.21.15.5:0/1225893182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e48250ed0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4004ad0 tx=0x7fb2a4031c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb7da99e205f4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.719+0000 7fb2b1b81640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb2980a3430 0x7fb2980a58f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4045000 tx=0x7fb2a4047000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.719+0000 7fb2977fe640 1 -- 172.21.15.5:0/1225893182 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a404c070 con 0x560e48250af0 2024-09-19T06:25:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.719+0000 7fb2977fe640 1 -- 172.21.15.5:0/1225893182 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2a4036070 con 0x560e48250af0 2024-09-19T06:25:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.719+0000 7fb2977fe640 1 -- 172.21.15.5:0/1225893182 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4030820 con 0x560e48250af0 2024-09-19T06:25:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.720+0000 7fb2b645c740 1 -- 172.21.15.5:0/1225893182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 msgr2=0x560e48250ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.720+0000 7fb2b645c740 1 --2- 172.21.15.5:0/1225893182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e48250ed0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4004ad0 tx=0x7fb2a4031c90 comp rx=0 tx=0).stop 2024-09-19T06:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.720+0000 7fb2b645c740 1 -- 172.21.15.5:0/1225893182 shutdown_connections 2024-09-19T06:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.720+0000 7fb2b645c740 1 --2- 172.21.15.5:0/1225893182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4825bcb0 0x560e4825e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.720+0000 7fb2b645c740 1 --2- 172.21.15.5:0/1225893182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e48251410 0x560e4825b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.720+0000 7fb2b645c740 1 --2- 172.21.15.5:0/1225893182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e48250ed0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.720+0000 7fb2b645c740 1 -- 172.21.15.5:0/1225893182 >> 172.21.15.5:0/1225893182 conn(0x560e4824d060 msgr2=0x560e4824d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.720+0000 7fb2b645c740 1 -- 172.21.15.5:0/1225893182 shutdown_connections 2024-09-19T06:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.720+0000 7fb2b645c740 1 -- 172.21.15.5:0/1225893182 wait complete. 2024-09-19T06:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.721+0000 7fb2b645c740 1 -- start start 2024-09-19T06:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.721+0000 7fb2b645c740 1 Processor -- start 2024-09-19T06:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b645c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e48258c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b645c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e48251410 0x560e48259160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b645c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4825bcb0 0x560e482596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b1380640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e48258c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b0b7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e48251410 0x560e48259160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b0b7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e48251410 0x560e48259160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41478/0 (socket says 172.21.15.5:41478) 2024-09-19T06:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b1b81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4825bcb0 0x560e482596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b1b81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4825bcb0 0x560e482596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35904/0 (socket says 172.21.15.5:35904) 2024-09-19T06:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b645c740 1 -- 172.21.15.5:0/3210463459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560e4825e700 con 0x560e4825bcb0 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b645c740 1 -- 172.21.15.5:0/3210463459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560e4813e290 con 0x560e48250af0 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b645c740 1 -- 172.21.15.5:0/3210463459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560e4825e880 con 0x560e48251410 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b0b7f640 1 -- 172.21.15.5:0/3210463459 learned_addr learned my addr 172.21.15.5:0/3210463459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.722+0000 7fb2b1380640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e48258c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40372/0 (socket says 172.21.15.5:40372) 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.723+0000 7fb2b0b7f640 1 -- 172.21.15.5:0/3210463459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 msgr2=0x560e48258c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.723+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/3210463459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e48258c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.723+0000 7fb2b0b7f640 1 -- 172.21.15.5:0/3210463459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4825bcb0 msgr2=0x560e482596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.723+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/3210463459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4825bcb0 0x560e482596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.723+0000 7fb2b0b7f640 1 -- 172.21.15.5:0/3210463459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e47ec8210 con 0x560e48251410 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.723+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/3210463459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e48251410 0x560e48259160 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8098a40 tx=0x7fb2a80d38d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=867579a37b36293a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.723+0000 7fb294ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.723+0000 7fb2b645c740 10 client.0 _ll_register_callbacks cb 0x560e4816f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.724+0000 7fb294ff9640 1 -- 172.21.15.5:0/3210463459 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80bc040 con 0x560e48251410 2024-09-19T06:25:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.724+0000 7fb294ff9640 1 -- 172.21.15.5:0/3210463459 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7fb2a80105d0 con 0x560e48251410 2024-09-19T06:25:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.724+0000 7fb2b645c740 1 -- 172.21.15.5:0/3210463459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2a4036070 con 0x560e48251410 2024-09-19T06:25:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.724+0000 7fb294ff9640 1 -- 172.21.15.5:0/3210463459 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fb2a809c5e0 con 0x560e48251410 2024-09-19T06:25:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.724+0000 7fb2b645c740 10 client.30166 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.725+0000 7fb2b645c740 1 -- 172.21.15.5:0/3210463459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=60}) -- 0x560e47f94620 con 0x560e48251410 2024-09-19T06:25:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.725+0000 7fb294ff9640 1 -- 172.21.15.5:0/3210463459 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fb2a80b52e0 con 0x560e48251410 2024-09-19T06:25:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.725+0000 7fb2b645c740 10 client.30166 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.726+0000 7fb2b645c740 10 client.30166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.726+0000 7fb2b645c740 20 client.30166 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.726+0000 7fb2b645c740 1 --2- 172.21.15.5:0/3210463459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e4826b7e0 0x560e4826dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.726+0000 7fb2b645c740 4 client.30166 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.726+0000 7fb2b645c740 1 -- 172.21.15.5:0/3210463459 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb2a4036070 con 0x560e4826b7e0 2024-09-19T06:25:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.726+0000 7fb2b1380640 1 --2- 172.21.15.5:0/3210463459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e4826b7e0 0x560e4826dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.727+0000 7fb294ff9640 10 client.30166 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.727+0000 7fb2b1380640 1 --2- 172.21.15.5:0/3210463459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e4826b7e0 0x560e4826dc70 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e81d15567a572ebb server_cookie=22c4e1d64c7a70d2 in_seq=0 out_seq=0 2024-09-19T06:25:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.728+0000 7fb294ff9640 1 -- 172.21.15.5:0/3210463459 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fb2a4036070 con 0x560e4826b7e0 2024-09-19T06:25:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.728+0000 7fb294ff9640 10 client.30166 handle_command_reply: tid=0 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.728+0000 7fb2b645c740 1 client.30166 shutdown 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.728+0000 7fb2b645c740 1 -- 172.21.15.5:0/3210463459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e48251410 msgr2=0x560e48259160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.728+0000 7fb2b645c740 1 --2- 172.21.15.5:0/3210463459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e48251410 0x560e48259160 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8098a40 tx=0x7fb2a80d38d0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.728+0000 7fb2b645c740 1 -- 172.21.15.5:0/3210463459 shutdown_connections 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.728+0000 7fb2b645c740 1 --2- 172.21.15.5:0/3210463459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e4826b7e0 0x560e4826dc70 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.728+0000 7fb2b645c740 1 --2- 172.21.15.5:0/3210463459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4825bcb0 0x560e482596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.728+0000 7fb2b645c740 1 --2- 172.21.15.5:0/3210463459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e48251410 0x560e48259160 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.728+0000 7fb2b645c740 1 --2- 172.21.15.5:0/3210463459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e48258c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.728+0000 7fb294ff9640 0 client.30166 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.728+0000 7fb2b645c740 1 -- 172.21.15.5:0/3210463459 >> 172.21.15.5:0/3210463459 conn(0x560e4824d060 msgr2=0x560e4824c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.729+0000 7fb2b645c740 1 -- 172.21.15.5:0/3210463459 shutdown_connections 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.729+0000 7fb2b645c740 1 -- 172.21.15.5:0/3210463459 wait complete. 2024-09-19T06:25:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.729+0000 7fb2b645c740 20 client.30166 trim_cache size 0 max 16384 2024-09-19T06:25:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b645c740 1 Processor -- start 2024-09-19T06:25:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b645c740 1 -- start start 2024-09-19T06:25:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b645c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e482594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b645c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e48251410 0x560e48273580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b645c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4825bcb0 0x560e48273ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b645c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac1b3e00 con 0x560e48251410 2024-09-19T06:25:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b645c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac1b59a0 con 0x560e48250af0 2024-09-19T06:25:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b645c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac1b44d0 con 0x560e4825bcb0 2024-09-19T06:25:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b1380640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e482594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b1380640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e482594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40380/0 (socket says 172.21.15.5:40380) 2024-09-19T06:25:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b1380640 1 -- 172.21.15.5:0/4292811262 learned_addr learned my addr 172.21.15.5:0/4292811262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/4292811262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e48251410 0x560e48273580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b1380640 1 -- 172.21.15.5:0/4292811262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4825bcb0 msgr2=0x560e48273ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b1b81640 1 --2- 172.21.15.5:0/4292811262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4825bcb0 0x560e48273ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b1380640 1 --2- 172.21.15.5:0/4292811262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4825bcb0 0x560e48273ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b1380640 1 -- 172.21.15.5:0/4292811262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e48251410 msgr2=0x560e48273580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.753+0000 7fb2b1380640 1 --2- 172.21.15.5:0/4292811262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e48251410 0x560e48273580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.754+0000 7fb2b1380640 1 -- 172.21.15.5:0/4292811262 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e48252000 con 0x560e48250af0 2024-09-19T06:25:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.754+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/4292811262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e48251410 0x560e48273580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.754+0000 7fb2b1b81640 1 --2- 172.21.15.5:0/4292811262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4825bcb0 0x560e48273ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:25:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.754+0000 7fb2b1380640 1 --2- 172.21.15.5:0/4292811262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e482594d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x560e4826e4f0 tx=0x7fb29c00a2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde3d29eb8a5b30e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.754+0000 7fb296ffd640 1 -- 172.21.15.5:0/4292811262 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c005220 con 0x560e48250af0 2024-09-19T06:25:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.754+0000 7fb296ffd640 1 -- 172.21.15.5:0/4292811262 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb29c004c50 con 0x560e48250af0 2024-09-19T06:25:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.754+0000 7fb296ffd640 1 -- 172.21.15.5:0/4292811262 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c01d8b0 con 0x560e48250af0 2024-09-19T06:25:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.755+0000 7fb2b645c740 1 -- 172.21.15.5:0/4292811262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 msgr2=0x560e482594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.755+0000 7fb2b645c740 1 --2- 172.21.15.5:0/4292811262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e482594d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x560e4826e4f0 tx=0x7fb29c00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.755+0000 7fb2b645c740 1 -- 172.21.15.5:0/4292811262 shutdown_connections 2024-09-19T06:25:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.755+0000 7fb2b645c740 1 --2- 172.21.15.5:0/4292811262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4825bcb0 0x560e48273ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.755+0000 7fb2b645c740 1 --2- 172.21.15.5:0/4292811262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e48251410 0x560e48273580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.755+0000 7fb2b645c740 1 --2- 172.21.15.5:0/4292811262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48250af0 0x560e482594d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.755+0000 7fb2b645c740 1 -- 172.21.15.5:0/4292811262 >> 172.21.15.5:0/4292811262 conn(0x560e4824d060 msgr2=0x560e4825aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.755+0000 7fb2b645c740 1 -- 172.21.15.5:0/4292811262 shutdown_connections 2024-09-19T06:25:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.755+0000 7fb2b645c740 1 -- 172.21.15.5:0/4292811262 wait complete. 2024-09-19T06:25:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b645c740 1 -- start start 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b645c740 1 Processor -- start 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b645c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e48250af0 0x560e48263b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b645c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48251410 0x560e48264050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b645c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4825bcb0 0x560e48260ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b645c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac1b52d0 con 0x560e48250af0 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b645c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac1b4bd0 con 0x560e48251410 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b645c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac1b1e00 con 0x560e4825bcb0 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b0b7f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48251410 0x560e48264050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b0b7f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48251410 0x560e48264050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40394/0 (socket says 172.21.15.5:40394) 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b0b7f640 1 -- 172.21.15.5:0/3080175856 learned_addr learned my addr 172.21.15.5:0/3080175856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b645c740 10 client.0 _ll_register_callbacks cb 0x7fb2a8018530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b1380640 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e48250af0 0x560e48263b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.756+0000 7fb2b1b81640 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4825bcb0 0x560e48260ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.757+0000 7fb2b0b7f640 1 -- 172.21.15.5:0/3080175856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4825bcb0 msgr2=0x560e48260ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.757+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4825bcb0 0x560e48260ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.757+0000 7fb2b0b7f640 1 -- 172.21.15.5:0/3080175856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e48250af0 msgr2=0x560e48263b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.757+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e48250af0 0x560e48263b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.757+0000 7fb2b0b7f640 1 -- 172.21.15.5:0/3080175856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e48264790 con 0x560e48251410 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.757+0000 7fb2b0b7f640 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48251410 0x560e48264050 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8098fb0 tx=0x7fb2a80d2780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70efe218e60be251 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.757+0000 7fb294ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.757+0000 7fb294ff9640 1 -- 172.21.15.5:0/3080175856 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80bc040 con 0x560e48251410 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.757+0000 7fb294ff9640 1 -- 172.21.15.5:0/3080175856 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7fb2a80105d0 con 0x560e48251410 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.757+0000 7fb2b645c740 1 -- 172.21.15.5:0/3080175856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb29c004c50 con 0x560e48251410 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.758+0000 7fb294ff9640 1 -- 172.21.15.5:0/3080175856 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fb2a80107d0 con 0x560e48251410 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.758+0000 7fb2b645c740 10 client.30245 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.758+0000 7fb2b645c740 1 -- 172.21.15.5:0/3080175856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x560e482d6b20 con 0x560e48251410 2024-09-19T06:25:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.758+0000 7fb294ff9640 1 -- 172.21.15.5:0/3080175856 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb29c004c50 con 0x560e48251410 2024-09-19T06:25:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.758+0000 7fb294ff9640 1 -- 172.21.15.5:0/3080175856 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fb2a809b4a0 con 0x560e48251410 2024-09-19T06:25:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.758+0000 7fb2b645c740 10 client.30245 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.758+0000 7fb2b645c740 10 client.30245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.758+0000 7fb2b645c740 20 client.30245 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.758+0000 7fb2b645c740 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e482d72f0 0x560e482d9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.758+0000 7fb2b645c740 4 client.30245 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:25:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.758+0000 7fb2b645c740 1 -- 172.21.15.5:0/3080175856 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e481c2a20 con 0x560e482d72f0 2024-09-19T06:25:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.758+0000 7fb2b1380640 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e482d72f0 0x560e482d9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.759+0000 7fb2b1380640 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e482d72f0 0x560e482d9710 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c8df25f642412a server_cookie=6b96710b9557466d in_seq=0 out_seq=0 2024-09-19T06:25:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.759+0000 7fb294ff9640 10 client.30245 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.866+0000 7fb294ff9640 1 -- 172.21.15.5:0/3080175856 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e481c2a20 con 0x560e482d72f0 2024-09-19T06:25:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.866+0000 7fb294ff9640 10 client.30245 handle_command_reply: tid=0 2024-09-19T06:25:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.866+0000 7fb2b645c740 1 client.30245 shutdown 2024-09-19T06:25:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.866+0000 7fb2b645c740 1 -- 172.21.15.5:0/3080175856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48251410 msgr2=0x560e48264050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.866+0000 7fb2b645c740 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48251410 0x560e48264050 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8098fb0 tx=0x7fb2a80d2780 comp rx=0 tx=0).stop 2024-09-19T06:25:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.867+0000 7fb2b645c740 1 -- 172.21.15.5:0/3080175856 shutdown_connections 2024-09-19T06:25:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.867+0000 7fb2b645c740 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x560e482d72f0 0x560e482d9710 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.867+0000 7fb294ff9640 0 client.30245 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.867+0000 7fb2b645c740 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4825bcb0 0x560e48260ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.867+0000 7fb2b645c740 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e48251410 0x560e48264050 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.867+0000 7fb2b645c740 1 --2- 172.21.15.5:0/3080175856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e48250af0 0x560e48263b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.867+0000 7fb2b645c740 1 -- 172.21.15.5:0/3080175856 >> 172.21.15.5:0/3080175856 conn(0x560e4824d060 msgr2=0x560e482525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.867+0000 7fb2b645c740 1 -- 172.21.15.5:0/3080175856 shutdown_connections 2024-09-19T06:25:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.867+0000 7fb2b645c740 1 -- 172.21.15.5:0/3080175856 wait complete. 2024-09-19T06:25:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.868+0000 7fb2b645c740 20 client.30245 trim_cache size 0 max 16384 2024-09-19T06:25:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:25:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:25:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:25:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:25:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.868+0000 7fb2b35e4640 1 -- 172.21.15.5:0/793342602 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb2980a3430 msgr2=0x7fb2980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.869+0000 7fb2b35e4640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb2980a3430 0x7fb2980a58f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4045000 tx=0x7fb2a4047000 comp rx=0 tx=0).stop 2024-09-19T06:25:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.869+0000 7fb2b35e4640 1 -- 172.21.15.5:0/793342602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac077040 msgr2=0x7fb2ac1a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.869+0000 7fb2b35e4640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac077040 0x7fb2ac1a4b80 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800eb30 tx=0x7fb2a800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.869+0000 7fb2b35e4640 1 -- 172.21.15.5:0/793342602 shutdown_connections 2024-09-19T06:25:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.869+0000 7fb2b35e4640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb2980a3430 0x7fb2980a58f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.869+0000 7fb2b35e4640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2ac112980 0x7fb2ac19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.869+0000 7fb2b35e4640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac077040 0x7fb2ac1a4b80 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.869+0000 7fb2b35e4640 1 --2- 172.21.15.5:0/793342602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2ac076720 0x7fb2ac1a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.869+0000 7fb2b35e4640 1 -- 172.21.15.5:0/793342602 >> 172.21.15.5:0/793342602 conn(0x7fb2ac0ff980 msgr2=0x7fb2ac101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.870+0000 7fb2b35e4640 1 -- 172.21.15.5:0/793342602 shutdown_connections 2024-09-19T06:25:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:19.870+0000 7fb2b35e4640 1 -- 172.21.15.5:0/793342602 wait complete. 2024-09-19T06:25:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.842+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/3089083109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdec102570 msgr2=0x7fcdec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.842+0000 7fcdf1ebb640 1 --2- 172.21.15.5:0/3089083109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdec102570 0x7fcdec1029b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009890 tx=0x7fcde002a3f0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.843+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/3089083109 shutdown_connections 2024-09-19T06:25:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.843+0000 7fcdf1ebb640 1 --2- 172.21.15.5:0/3089083109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec102ef0 0x7fcdec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.843+0000 7fcdf1ebb640 1 --2- 172.21.15.5:0/3089083109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdec102570 0x7fcdec1029b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.843+0000 7fcdf1ebb640 1 --2- 172.21.15.5:0/3089083109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdec100e20 0x7fcdec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.843+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/3089083109 >> 172.21.15.5:0/3089083109 conn(0x7fcdec0fc7d0 msgr2=0x7fcdec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.843+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/3089083109 shutdown_connections 2024-09-19T06:25:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.843+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/3089083109 wait complete. 2024-09-19T06:25:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.843+0000 7fcdf1ebb640 1 Processor -- start 2024-09-19T06:25:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdf1ebb640 1 -- start start 2024-09-19T06:25:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdf1ebb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec100e20 0x7fcdec073af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdf1ebb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdec102570 0x7fcdec06db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdf1ebb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdec102ef0 0x7fcdec06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdf1ebb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec06c4f0 con 0x7fcdec100e20 2024-09-19T06:25:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdf1ebb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec06c370 con 0x7fcdec102570 2024-09-19T06:25:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdf1ebb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec06c670 con 0x7fcdec102ef0 2024-09-19T06:25:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdeb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec100e20 0x7fcdec073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdec102570 0x7fcdec06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdec102570 0x7fcdec06db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40416/0 (socket says 172.21.15.5:40416) 2024-09-19T06:25:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdeb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec100e20 0x7fcdec073af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35948/0 (socket says 172.21.15.5:35948) 2024-09-19T06:25:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdeaffd640 1 -- 172.21.15.5:0/707698981 learned_addr learned my addr 172.21.15.5:0/707698981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.844+0000 7fcdebfff640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdec102ef0 0x7fcdec06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.845+0000 7fcdeaffd640 1 -- 172.21.15.5:0/707698981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdec102ef0 msgr2=0x7fcdec06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.845+0000 7fcdeaffd640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdec102ef0 0x7fcdec06e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.845+0000 7fcdeaffd640 1 -- 172.21.15.5:0/707698981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec100e20 msgr2=0x7fcdec073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.845+0000 7fcdeaffd640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec100e20 0x7fcdec073af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.845+0000 7fcdeaffd640 1 -- 172.21.15.5:0/707698981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec06e7b0 con 0x7fcdec102570 2024-09-19T06:25:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.845+0000 7fcdebfff640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdec102ef0 0x7fcdec06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:25:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.845+0000 7fcdeaffd640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdec102570 0x7fcdec06db70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcde0000c00 tx=0x7fcde0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.845+0000 7fcde8ff9640 1 -- 172.21.15.5:0/707698981 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde0040020 con 0x7fcdec102570 2024-09-19T06:25:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.845+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/707698981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec1ab200 con 0x7fcdec102570 2024-09-19T06:25:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.846+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/707698981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec1ab5a0 con 0x7fcdec102570 2024-09-19T06:25:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.846+0000 7fcde8ff9640 1 -- 172.21.15.5:0/707698981 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcde0003aa0 con 0x7fcdec102570 2024-09-19T06:25:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.846+0000 7fcde8ff9640 1 -- 172.21.15.5:0/707698981 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde0037040 con 0x7fcdec102570 2024-09-19T06:25:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.849+0000 7fcdf4294740 1 Processor -- start 2024-09-19T06:25:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.849+0000 7fcdf4294740 1 -- start start 2024-09-19T06:25:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.849+0000 7fcdf4294740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 0x55ffd677a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.849+0000 7fcdf4294740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd677aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.849+0000 7fcdf4294740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd6785370 0x55ffd6787760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.849+0000 7fcdf4294740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffd66e02f0 con 0x55ffd677bb40 2024-09-19T06:25:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.849+0000 7fcdf4294740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ffd6687cf0 con 0x55ffd677a5d0 2024-09-19T06:25:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.849+0000 7fcdf4294740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ffd66dc2b0 con 0x55ffd6785370 2024-09-19T06:25:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.850+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 0x55ffd677a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.850+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 0x55ffd677a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35952/0 (socket says 172.21.15.5:35952) 2024-09-19T06:25:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.850+0000 7fcdebfff640 1 -- 172.21.15.5:0/3801579973 learned_addr learned my addr 172.21.15.5:0/3801579973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.850+0000 7fcdeb7fe640 1 --2- 172.21.15.5:0/3801579973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd677aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.850+0000 7fcde8ff9640 1 -- 172.21.15.5:0/707698981 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcde0043020 con 0x7fcdec102570 2024-09-19T06:25:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.850+0000 7fcdeb7fe640 1 -- 172.21.15.5:0/3801579973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd6785370 msgr2=0x55ffd6787760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.850+0000 7fcdeb7fe640 1 --2- 172.21.15.5:0/3801579973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd6785370 0x55ffd6787760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.850+0000 7fcdeb7fe640 1 -- 172.21.15.5:0/3801579973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 msgr2=0x55ffd677a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.850+0000 7fcdeb7fe640 1 --2- 172.21.15.5:0/3801579973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 0x55ffd677a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.850+0000 7fcdeb7fe640 1 -- 172.21.15.5:0/3801579973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd666c3a0 con 0x55ffd677a5d0 2024-09-19T06:25:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.850+0000 7fcdeb7fe640 1 --2- 172.21.15.5:0/3801579973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd677aa10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0099a0 tx=0x7fcddc009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c53d616e6c9d1040 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.851+0000 7fcdd1ffb640 1 -- 172.21.15.5:0/3801579973 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc017070 con 0x55ffd677a5d0 2024-09-19T06:25:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.851+0000 7fcdd1ffb640 1 -- 172.21.15.5:0/3801579973 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcddc013470 con 0x55ffd677a5d0 2024-09-19T06:25:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.851+0000 7fcdd1ffb640 1 -- 172.21.15.5:0/3801579973 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc012630 con 0x55ffd677a5d0 2024-09-19T06:25:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.851+0000 7fcde8ff9640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcdd40a3480 0x7fcdd40a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.851+0000 7fcdf4294740 1 -- 172.21.15.5:0/3801579973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 msgr2=0x55ffd677aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.851+0000 7fcdf4294740 1 --2- 172.21.15.5:0/3801579973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd677aa10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0099a0 tx=0x7fcddc009e60 comp rx=0 tx=0).stop 2024-09-19T06:25:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.852+0000 7fcdebfff640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcdd40a3480 0x7fcdd40a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.852+0000 7fcdf4294740 1 -- 172.21.15.5:0/3801579973 shutdown_connections 2024-09-19T06:25:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.852+0000 7fcdf4294740 1 --2- 172.21.15.5:0/3801579973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd6785370 0x55ffd6787760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.852+0000 7fcdf4294740 1 --2- 172.21.15.5:0/3801579973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd677aa10 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.852+0000 7fcdf4294740 1 --2- 172.21.15.5:0/3801579973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 0x55ffd677a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.852+0000 7fcdf4294740 1 -- 172.21.15.5:0/3801579973 >> 172.21.15.5:0/3801579973 conn(0x55ffd6743310 msgr2=0x55ffd6776c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.852+0000 7fcdf4294740 1 -- 172.21.15.5:0/3801579973 shutdown_connections 2024-09-19T06:25:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.852+0000 7fcdf4294740 1 -- 172.21.15.5:0/3801579973 wait complete. 2024-09-19T06:25:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.853+0000 7fcdebfff640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcdd40a3480 0x7fcdd40a5940 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcde4005bd0 tx=0x7fcde401a8d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.853+0000 7fcde8ff9640 1 -- 172.21.15.5:0/707698981 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7fcde00efb10 con 0x7fcdec102570 2024-09-19T06:25:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.853+0000 7fcdf4294740 1 -- start start 2024-09-19T06:25:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.853+0000 7fcdf4294740 1 Processor -- start 2024-09-19T06:25:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdf4294740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd6784d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdf4294740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd677bb40 0x55ffd6781fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdf4294740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 0x55ffd6782530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdeb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd6784d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdeb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd6784d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40428/0 (socket says 172.21.15.5:40428) 2024-09-19T06:25:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd677bb40 0x55ffd6781fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd677bb40 0x55ffd6781fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41522/0 (socket says 172.21.15.5:41522) 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 0x55ffd6782530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 0x55ffd6782530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35968/0 (socket says 172.21.15.5:35968) 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdeb7fe640 1 -- 172.21.15.5:0/1750669016 learned_addr learned my addr 172.21.15.5:0/1750669016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdf4294740 1 -- 172.21.15.5:0/1750669016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffd6788070 con 0x55ffd6785370 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdf4294740 1 -- 172.21.15.5:0/1750669016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ffd669b060 con 0x55ffd677a5d0 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdf4294740 1 -- 172.21.15.5:0/1750669016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ffd67881f0 con 0x55ffd677bb40 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.854+0000 7fcdf4294740 10 client.0 _ll_register_callbacks cb 0x55ffd63ee8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.855+0000 7fcdeb7fe640 1 -- 172.21.15.5:0/1750669016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd677bb40 msgr2=0x55ffd6781fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.855+0000 7fcdeb7fe640 1 --2- 172.21.15.5:0/1750669016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd677bb40 0x55ffd6781fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.855+0000 7fcdeb7fe640 1 -- 172.21.15.5:0/1750669016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 msgr2=0x55ffd6782530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.855+0000 7fcdeb7fe640 1 --2- 172.21.15.5:0/1750669016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 0x55ffd6782530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.855+0000 7fcdeb7fe640 1 -- 172.21.15.5:0/1750669016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd64513c0 con 0x55ffd677a5d0 2024-09-19T06:25:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.855+0000 7fcdebfff640 1 --2- 172.21.15.5:0/1750669016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 0x55ffd6782530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.855+0000 7fcdeb7fe640 1 --2- 172.21.15.5:0/1750669016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd6784d00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006aa0 tx=0x7fcddc006ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d57a8d6f6c8cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.855+0000 7fcdcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.855+0000 7fcdcaffd640 1 -- 172.21.15.5:0/1750669016 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc017050 con 0x55ffd677a5d0 2024-09-19T06:25:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.855+0000 7fcdcaffd640 1 -- 172.21.15.5:0/1750669016 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7fcddc01c450 con 0x55ffd677a5d0 2024-09-19T06:25:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.855+0000 7fcdf4294740 1 -- 172.21.15.5:0/1750669016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcddc013470 con 0x55ffd677a5d0 2024-09-19T06:25:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.856+0000 7fcdcaffd640 1 -- 172.21.15.5:0/1750669016 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fcddc01c650 con 0x55ffd677a5d0 2024-09-19T06:25:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.856+0000 7fcdf4294740 10 client.30254 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.856+0000 7fcdf4294740 1 -- 172.21.15.5:0/1750669016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55ffd6793ab0 con 0x55ffd677a5d0 2024-09-19T06:25:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.856+0000 7fcdcaffd640 1 -- 172.21.15.5:0/1750669016 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcddc013470 con 0x55ffd677a5d0 2024-09-19T06:25:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.856+0000 7fcdcaffd640 1 -- 172.21.15.5:0/1750669016 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fcddc0267c0 con 0x55ffd677a5d0 2024-09-19T06:25:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.856+0000 7fcdf4294740 10 client.30254 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.856+0000 7fcdf4294740 10 client.30254 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:25:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.856+0000 7fcdf4294740 20 client.30254 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.856+0000 7fcdf4294740 1 --2- 172.21.15.5:0/1750669016 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ffd6794e60 0x55ffd6797250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.856+0000 7fcdf4294740 4 client.30254 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.856+0000 7fcdf4294740 1 -- 172.21.15.5:0/1750669016 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffd66ed860 con 0x55ffd6794e60 2024-09-19T06:25:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.856+0000 7fcdeaffd640 1 --2- 172.21.15.5:0/1750669016 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ffd6794e60 0x55ffd6797250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.857+0000 7fcdeaffd640 1 --2- 172.21.15.5:0/1750669016 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ffd6794e60 0x55ffd6797250 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f14beb7f99781b00 server_cookie=1cc0ad97e8875515 in_seq=0 out_seq=0 2024-09-19T06:25:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.857+0000 7fcdcaffd640 10 client.30254 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:25:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdcaffd640 1 -- 172.21.15.5:0/1750669016 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffd66ed860 con 0x55ffd6794e60 2024-09-19T06:25:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdcaffd640 10 client.30254 handle_command_reply: tid=0 2024-09-19T06:25:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdf4294740 1 client.30254 shutdown 2024-09-19T06:25:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdf4294740 1 -- 172.21.15.5:0/1750669016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 msgr2=0x55ffd6784d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdf4294740 1 --2- 172.21.15.5:0/1750669016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd6784d00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006aa0 tx=0x7fcddc006ec0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdf4294740 1 -- 172.21.15.5:0/1750669016 shutdown_connections 2024-09-19T06:25:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdf4294740 1 --2- 172.21.15.5:0/1750669016 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ffd6794e60 0x55ffd6797250 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdf4294740 1 --2- 172.21.15.5:0/1750669016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 0x55ffd6782530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdf4294740 1 --2- 172.21.15.5:0/1750669016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd677bb40 0x55ffd6781fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdf4294740 1 --2- 172.21.15.5:0/1750669016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd6784d00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdf4294740 1 -- 172.21.15.5:0/1750669016 >> 172.21.15.5:0/1750669016 conn(0x55ffd6743310 msgr2=0x55ffd677c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdcaffd640 0 client.30254 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdf4294740 1 -- 172.21.15.5:0/1750669016 shutdown_connections 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdf4294740 1 -- 172.21.15.5:0/1750669016 wait complete. 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.858+0000 7fcdf4294740 20 client.30254 trim_cache size 0 max 16384 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.881+0000 7fcdf4294740 1 Processor -- start 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.881+0000 7fcdf4294740 1 -- start start 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.882+0000 7fcdf4294740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd677a5d0 0x55ffd6776e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.882+0000 7fcdf4294740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677bb40 0x55ffd6791340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.882+0000 7fcdf4294740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 0x55ffd679c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.882+0000 7fcdf4294740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1b3f80 con 0x55ffd6785370 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.882+0000 7fcdf4294740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1b5b20 con 0x55ffd677bb40 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.882+0000 7fcdf4294740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1b4650 con 0x55ffd677a5d0 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.882+0000 7fcdeb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd677a5d0 0x55ffd6776e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.882+0000 7fcdeb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd677a5d0 0x55ffd6776e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41536/0 (socket says 172.21.15.5:41536) 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.882+0000 7fcdeb7fe640 1 -- 172.21.15.5:0/4187009648 learned_addr learned my addr 172.21.15.5:0/4187009648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.882+0000 7fcdebfff640 1 --2- 172.21.15.5:0/4187009648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 0x55ffd679c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.882+0000 7fcdeaffd640 1 --2- 172.21.15.5:0/4187009648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677bb40 0x55ffd6791340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.883+0000 7fcdeaffd640 1 -- 172.21.15.5:0/4187009648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd677a5d0 msgr2=0x55ffd6776e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.883+0000 7fcdeaffd640 1 --2- 172.21.15.5:0/4187009648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd677a5d0 0x55ffd6776e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.883+0000 7fcdeaffd640 1 -- 172.21.15.5:0/4187009648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 msgr2=0x55ffd679c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.883+0000 7fcdeaffd640 1 --2- 172.21.15.5:0/4187009648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 0x55ffd679c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.883+0000 7fcdeaffd640 1 -- 172.21.15.5:0/4187009648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd6784920 con 0x55ffd677bb40 2024-09-19T06:25:19.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.883+0000 7fcdebfff640 1 --2- 172.21.15.5:0/4187009648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 0x55ffd679c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:19.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.883+0000 7fcdeaffd640 1 --2- 172.21.15.5:0/4187009648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677bb40 0x55ffd6791340 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcde00ed2f0 tx=0x7fcde00095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c529b7dc57281452 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:19.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.883+0000 7fcdd17fa640 1 -- 172.21.15.5:0/4187009648 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00c0230 con 0x55ffd677bb40 2024-09-19T06:25:19.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.883+0000 7fcdd17fa640 1 -- 172.21.15.5:0/4187009648 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcde00d8070 con 0x55ffd677bb40 2024-09-19T06:25:19.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.883+0000 7fcdd17fa640 1 -- 172.21.15.5:0/4187009648 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00ed460 con 0x55ffd677bb40 2024-09-19T06:25:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.884+0000 7fcdf4294740 1 -- 172.21.15.5:0/4187009648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677bb40 msgr2=0x55ffd6791340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.884+0000 7fcdf4294740 1 --2- 172.21.15.5:0/4187009648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677bb40 0x55ffd6791340 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcde00ed2f0 tx=0x7fcde00095b0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.884+0000 7fcdf4294740 1 -- 172.21.15.5:0/4187009648 shutdown_connections 2024-09-19T06:25:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.884+0000 7fcdf4294740 1 --2- 172.21.15.5:0/4187009648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd6785370 0x55ffd679c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.884+0000 7fcdf4294740 1 --2- 172.21.15.5:0/4187009648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677bb40 0x55ffd6791340 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.884+0000 7fcdf4294740 1 --2- 172.21.15.5:0/4187009648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd677a5d0 0x55ffd6776e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.884+0000 7fcdf4294740 1 -- 172.21.15.5:0/4187009648 >> 172.21.15.5:0/4187009648 conn(0x55ffd6743310 msgr2=0x55ffd6776700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.884+0000 7fcdf4294740 1 -- 172.21.15.5:0/4187009648 shutdown_connections 2024-09-19T06:25:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.884+0000 7fcdf4294740 1 -- 172.21.15.5:0/4187009648 wait complete. 2024-09-19T06:25:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.884+0000 7fcdf4294740 1 -- start start 2024-09-19T06:25:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdf4294740 1 Processor -- start 2024-09-19T06:25:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdf4294740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd6788470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdf4294740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 0x55ffd67889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdf4294740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd6785370 0x55ffd6788ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdf4294740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1b5450 con 0x55ffd677bb40 2024-09-19T06:25:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdf4294740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1b4d50 con 0x55ffd677a5d0 2024-09-19T06:25:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdf4294740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1b1f80 con 0x55ffd6785370 2024-09-19T06:25:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdf4294740 10 client.0 _ll_register_callbacks cb 0x7fcde401a770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdeb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd6788470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 0x55ffd67889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd6785370 0x55ffd6788ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdeb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd6788470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40448/0 (socket says 172.21.15.5:40448) 2024-09-19T06:25:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 0x55ffd67889b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35978/0 (socket says 172.21.15.5:35978) 2024-09-19T06:25:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.885+0000 7fcdeb7fe640 1 -- 172.21.15.5:0/189690188 learned_addr learned my addr 172.21.15.5:0/189690188 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.886+0000 7fcdeb7fe640 1 -- 172.21.15.5:0/189690188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd6785370 msgr2=0x55ffd6788ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.886+0000 7fcdeb7fe640 1 --2- 172.21.15.5:0/189690188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd6785370 0x55ffd6788ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.886+0000 7fcdeb7fe640 1 -- 172.21.15.5:0/189690188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 msgr2=0x55ffd67889b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.886+0000 7fcdeb7fe640 1 --2- 172.21.15.5:0/189690188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 0x55ffd67889b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.886+0000 7fcdeb7fe640 1 -- 172.21.15.5:0/189690188 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd67fd9e0 con 0x55ffd677a5d0 2024-09-19T06:25:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.886+0000 7fcdeaffd640 1 --2- 172.21.15.5:0/189690188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 0x55ffd67889b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.886+0000 7fcdeb7fe640 1 --2- 172.21.15.5:0/189690188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd6788470 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcddc026c60 tx=0x7fcddc013930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=562abf2c48306759 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.886+0000 7fcdcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.886+0000 7fcdcaffd640 1 -- 172.21.15.5:0/189690188 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc033030 con 0x55ffd677a5d0 2024-09-19T06:25:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.886+0000 7fcdf4294740 1 -- 172.21.15.5:0/189690188 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde00d8070 con 0x55ffd677a5d0 2024-09-19T06:25:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.886+0000 7fcdcaffd640 1 -- 172.21.15.5:0/189690188 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7fcddc01c450 con 0x55ffd677a5d0 2024-09-19T06:25:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.886+0000 7fcdcaffd640 1 -- 172.21.15.5:0/189690188 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fcddc017040 con 0x55ffd677a5d0 2024-09-19T06:25:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.887+0000 7fcdf4294740 10 client.30260 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.887+0000 7fcdf4294740 1 -- 172.21.15.5:0/189690188 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55ffd67fecc0 con 0x55ffd677a5d0 2024-09-19T06:25:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.887+0000 7fcdcaffd640 1 -- 172.21.15.5:0/189690188 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcde00d8070 con 0x55ffd677a5d0 2024-09-19T06:25:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.887+0000 7fcdcaffd640 1 -- 172.21.15.5:0/189690188 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fcddc01ccb0 con 0x55ffd677a5d0 2024-09-19T06:25:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.887+0000 7fcdf4294740 10 client.30260 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.887+0000 7fcdf4294740 10 client.30260 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:25:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.887+0000 7fcdf4294740 20 client.30260 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.887+0000 7fcdf4294740 1 --2- 172.21.15.5:0/189690188 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ffd67ff380 0x55ffd68017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.887+0000 7fcdf4294740 4 client.30260 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:25:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.887+0000 7fcdf4294740 1 -- 172.21.15.5:0/189690188 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffd66e7d70 con 0x55ffd67ff380 2024-09-19T06:25:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.887+0000 7fcdeaffd640 1 --2- 172.21.15.5:0/189690188 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ffd67ff380 0x55ffd68017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdcaffd640 10 client.30260 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:25:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdeaffd640 1 --2- 172.21.15.5:0/189690188 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ffd67ff380 0x55ffd68017a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=659ef0f2353f2c6f server_cookie=b1b2e0bd66b8b2be in_seq=0 out_seq=0 2024-09-19T06:25:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdcaffd640 1 -- 172.21.15.5:0/189690188 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffd66e7d70 con 0x55ffd67ff380 2024-09-19T06:25:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdcaffd640 10 client.30260 handle_command_reply: tid=0 2024-09-19T06:25:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdf4294740 1 client.30260 shutdown 2024-09-19T06:25:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdf4294740 1 -- 172.21.15.5:0/189690188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 msgr2=0x55ffd6788470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdf4294740 1 --2- 172.21.15.5:0/189690188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd6788470 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcddc026c60 tx=0x7fcddc013930 comp rx=0 tx=0).stop 2024-09-19T06:25:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdf4294740 1 -- 172.21.15.5:0/189690188 shutdown_connections 2024-09-19T06:25:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdf4294740 1 --2- 172.21.15.5:0/189690188 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55ffd67ff380 0x55ffd68017a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdf4294740 1 --2- 172.21.15.5:0/189690188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd6785370 0x55ffd6788ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdcaffd640 0 client.30260 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:25:19.945 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:25:19.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdf4294740 1 --2- 172.21.15.5:0/189690188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffd677bb40 0x55ffd67889b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdf4294740 1 --2- 172.21.15.5:0/189690188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd677a5d0 0x55ffd6788470 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdf4294740 1 -- 172.21.15.5:0/189690188 >> 172.21.15.5:0/189690188 conn(0x55ffd6743310 msgr2=0x55ffd6780880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:19.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.888+0000 7fcdf4294740 1 -- 172.21.15.5:0/189690188 shutdown_connections 2024-09-19T06:25:19.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf4294740 1 -- 172.21.15.5:0/189690188 wait complete. 2024-09-19T06:25:19.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf4294740 20 client.30260 trim_cache size 0 max 16384 2024-09-19T06:25:19.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/707698981 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcdd40a3480 msgr2=0x7fcdd40a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf1ebb640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcdd40a3480 0x7fcdd40a5940 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcde4005bd0 tx=0x7fcde401a8d0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/707698981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdec102570 msgr2=0x7fcdec06db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:19.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf1ebb640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdec102570 0x7fcdec06db70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcde0000c00 tx=0x7fcde0004270 comp rx=0 tx=0).stop 2024-09-19T06:25:19.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/707698981 shutdown_connections 2024-09-19T06:25:19.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf1ebb640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fcdd40a3480 0x7fcdd40a5940 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf1ebb640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdec102ef0 0x7fcdec06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf1ebb640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdec102570 0x7fcdec06db70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf1ebb640 1 --2- 172.21.15.5:0/707698981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec100e20 0x7fcdec073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:19.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/707698981 >> 172.21.15.5:0/707698981 conn(0x7fcdec0fc7d0 msgr2=0x7fcdec104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:19.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.889+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/707698981 shutdown_connections 2024-09-19T06:25:19.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:19.890+0000 7fcdf1ebb640 1 -- 172.21.15.5:0/707698981 wait complete. 2024-09-19T06:25:20.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84829 8.60 MB/sec execute 404 sec latency 94.244 ms 2024-09-19T06:25:20.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:20 smithi005 ceph-mon[30955]: pgmap v695: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 241 op/s 2024-09-19T06:25:20.707 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:25:20.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:20 smithi157 ceph-mon[35346]: pgmap v695: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 241 op/s 2024-09-19T06:25:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:20 smithi049 ceph-mon[34640]: pgmap v695: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 241 op/s 2024-09-19T06:25:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:25:21.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84970 8.60 MB/sec execute 405 sec latency 102.929 ms 2024-09-19T06:25:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.663+0000 7fde13373640 1 -- 172.21.15.5:0/23690887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c1082a0 msgr2=0x7fde0c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.663+0000 7fde13373640 1 --2- 172.21.15.5:0/23690887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c1082a0 0x7fde0c10ec40 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fde04009cd0 tx=0x7fde0402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.663+0000 7fde13373640 1 -- 172.21.15.5:0/23690887 shutdown_connections 2024-09-19T06:25:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.663+0000 7fde13373640 1 --2- 172.21.15.5:0/23690887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c1082a0 0x7fde0c10ec40 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.663+0000 7fde13373640 1 --2- 172.21.15.5:0/23690887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde0c1036e0 0x7fde0c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.663+0000 7fde13373640 1 --2- 172.21.15.5:0/23690887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde0c102d30 0x7fde0c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.663+0000 7fde13373640 1 -- 172.21.15.5:0/23690887 >> 172.21.15.5:0/23690887 conn(0x7fde0c0fc7d0 msgr2=0x7fde0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.663+0000 7fde13373640 1 -- 172.21.15.5:0/23690887 shutdown_connections 2024-09-19T06:25:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.663+0000 7fde13373640 1 -- 172.21.15.5:0/23690887 wait complete. 2024-09-19T06:25:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde13373640 1 Processor -- start 2024-09-19T06:25:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde13373640 1 -- start start 2024-09-19T06:25:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde13373640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde0c102d30 0x7fde0c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde13373640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde0c1036e0 0x7fde0c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde13373640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c1082a0 0x7fde0c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde13373640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c111f50 con 0x7fde0c1082a0 2024-09-19T06:25:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde13373640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c111dd0 con 0x7fde0c102d30 2024-09-19T06:25:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde13373640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1120d0 con 0x7fde0c1036e0 2024-09-19T06:25:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde11910640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c1082a0 0x7fde0c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde1110f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde0c102d30 0x7fde0c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde1090e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde0c1036e0 0x7fde0c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde11910640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c1082a0 0x7fde0c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35996/0 (socket says 172.21.15.5:35996) 2024-09-19T06:25:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde1110f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde0c102d30 0x7fde0c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40474/0 (socket says 172.21.15.5:40474) 2024-09-19T06:25:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.664+0000 7fde11910640 1 -- 172.21.15.5:0/3182252802 learned_addr learned my addr 172.21.15.5:0/3182252802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.665+0000 7fde11910640 1 -- 172.21.15.5:0/3182252802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde0c1036e0 msgr2=0x7fde0c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.665+0000 7fde11910640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde0c1036e0 0x7fde0c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.665+0000 7fde11910640 1 -- 172.21.15.5:0/3182252802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde0c102d30 msgr2=0x7fde0c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.665+0000 7fde11910640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde0c102d30 0x7fde0c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.665+0000 7fde11910640 1 -- 172.21.15.5:0/3182252802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c19f620 con 0x7fde0c1082a0 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.665+0000 7fde1090e640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde0c1036e0 0x7fde0c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.665+0000 7fde11910640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c1082a0 0x7fde0c19ef40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fde04005cc0 tx=0x7fde04003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.665+0000 7fde027fc640 1 -- 172.21.15.5:0/3182252802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04004450 con 0x7fde0c1082a0 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.665+0000 7fde027fc640 1 -- 172.21.15.5:0/3182252802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde04031c90 con 0x7fde0c1082a0 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.665+0000 7fde027fc640 1 -- 172.21.15.5:0/3182252802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0403b400 con 0x7fde0c1082a0 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.665+0000 7fde13373640 1 -- 172.21.15.5:0/3182252802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c19f8b0 con 0x7fde0c1082a0 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.665+0000 7fde13373640 1 -- 172.21.15.5:0/3182252802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c1a71f0 con 0x7fde0c1082a0 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.667+0000 7fde027fc640 1 -- 172.21.15.5:0/3182252802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fde04004680 con 0x7fde0c1082a0 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.668+0000 7fde1574c740 1 Processor -- start 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.668+0000 7fde1574c740 1 -- start start 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.668+0000 7fde1574c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700017b40 0x561700016090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.668+0000 7fde1574c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617000165d0 0x561700016a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.668+0000 7fde1574c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x561700023760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.668+0000 7fde1574c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5616fff7c2f0 con 0x561700021370 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.668+0000 7fde1574c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5616fff23cf0 con 0x5617000165d0 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.668+0000 7fde1574c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5616fff782b0 con 0x561700017b40 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde1090e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x561700023760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde1090e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x561700023760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35998/0 (socket says 172.21.15.5:35998) 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde027fc640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fddf80a3430 0x7fddf80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde1090e640 1 -- 172.21.15.5:0/1224191217 learned_addr learned my addr 172.21.15.5:0/1224191217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde1090e640 1 --2- 172.21.15.5:0/1224191217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700017b40 0x561700016090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde1110f640 1 --2- 172.21.15.5:0/1224191217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617000165d0 0x561700016a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde11910640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fddf80a3430 0x7fddf80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde1090e640 1 -- 172.21.15.5:0/1224191217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700017b40 msgr2=0x561700016090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde1090e640 1 --2- 172.21.15.5:0/1224191217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700017b40 0x561700016090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde1090e640 1 -- 172.21.15.5:0/1224191217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617000165d0 msgr2=0x561700016a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde1090e640 1 --2- 172.21.15.5:0/1224191217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617000165d0 0x561700016a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde1090e640 1 -- 172.21.15.5:0/1224191217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616fff083a0 con 0x561700021370 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde027fc640 1 -- 172.21.15.5:0/3182252802 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7fde0402e030 con 0x7fde0c1082a0 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde1090e640 1 --2- 172.21.15.5:0/1224191217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x561700023760 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fddfc012a90 tx=0x7fddfc00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0be8235d3c9a64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fde11910640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fddf80a3430 0x7fddf80a58f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fde040b9520 tx=0x7fde0408ee40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fddf77fe640 1 -- 172.21.15.5:0/1224191217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc004020 con 0x561700021370 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.669+0000 7fddf77fe640 1 -- 172.21.15.5:0/1224191217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddfc0054d0 con 0x561700021370 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.670+0000 7fddf77fe640 1 -- 172.21.15.5:0/1224191217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc010ce0 con 0x561700021370 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.670+0000 7fde1574c740 1 -- 172.21.15.5:0/1224191217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 msgr2=0x561700023760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.670+0000 7fde1574c740 1 --2- 172.21.15.5:0/1224191217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x561700023760 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fddfc012a90 tx=0x7fddfc00ba90 comp rx=0 tx=0).stop 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.670+0000 7fde1574c740 1 -- 172.21.15.5:0/1224191217 shutdown_connections 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.670+0000 7fde1574c740 1 --2- 172.21.15.5:0/1224191217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x561700023760 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.670+0000 7fde1574c740 1 --2- 172.21.15.5:0/1224191217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617000165d0 0x561700016a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.670+0000 7fde1574c740 1 --2- 172.21.15.5:0/1224191217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700017b40 0x561700016090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.670+0000 7fde1574c740 1 -- 172.21.15.5:0/1224191217 >> 172.21.15.5:0/1224191217 conn(0x5616fffdf310 msgr2=0x561700012c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.670+0000 7fde1574c740 1 -- 172.21.15.5:0/1224191217 shutdown_connections 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.670+0000 7fde1574c740 1 -- 172.21.15.5:0/1224191217 wait complete. 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.670+0000 7fde1574c740 1 -- start start 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1574c740 1 Processor -- start 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1574c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617000165d0 0x561700020d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1574c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561700017b40 0x56170001dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1574c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x56170001e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1574c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617000240c0 con 0x561700021370 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1574c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5616fff37060 con 0x561700017b40 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1574c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561700024240 con 0x5617000165d0 2024-09-19T06:25:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1574c740 10 client.0 _ll_register_callbacks cb 0x5616ffc8a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1090e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x56170001e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1090e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x56170001e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36000/0 (socket says 172.21.15.5:36000) 2024-09-19T06:25:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1090e640 1 -- 172.21.15.5:0/797017789 learned_addr learned my addr 172.21.15.5:0/797017789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1090e640 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617000165d0 0x561700020d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.671+0000 7fde1110f640 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561700017b40 0x56170001dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fde1090e640 1 -- 172.21.15.5:0/797017789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617000165d0 msgr2=0x561700020d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fde1090e640 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617000165d0 0x561700020d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fde1090e640 1 -- 172.21.15.5:0/797017789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561700017b40 msgr2=0x56170001dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fde1090e640 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561700017b40 0x56170001dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fde1090e640 1 -- 172.21.15.5:0/797017789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616ffced3c0 con 0x561700021370 2024-09-19T06:25:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fde1090e640 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x56170001e5e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00b450 tx=0x7fddfc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ab85550ad0548e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fddf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fddf4ff9640 1 -- 172.21.15.5:0/797017789 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc010ac0 con 0x561700021370 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fddf4ff9640 1 -- 172.21.15.5:0/797017789 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7fddfc010c60 con 0x561700021370 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fde1574c740 1 -- 172.21.15.5:0/797017789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddfc0054d0 con 0x561700021370 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fddf4ff9640 1 -- 172.21.15.5:0/797017789 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fddfc033050 con 0x561700021370 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fde1574c740 10 client.20898 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fde1574c740 1 -- 172.21.15.5:0/797017789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x56170002fb30 con 0x561700021370 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fddf4ff9640 1 -- 172.21.15.5:0/797017789 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddfc0054d0 con 0x561700021370 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fddf4ff9640 1 -- 172.21.15.5:0/797017789 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fddfc01f470 con 0x561700021370 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fde1574c740 10 client.20898 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fde1574c740 10 client.20898 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.672+0000 7fde1574c740 20 client.20898 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.673+0000 7fde1574c740 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x561700030e80 0x561700033270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.673+0000 7fde1574c740 4 client.20898 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.673+0000 7fde1574c740 1 -- 172.21.15.5:0/797017789 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616fff89860 con 0x561700030e80 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.673+0000 7fde1110f640 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x561700030e80 0x561700033270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.673+0000 7fddf4ff9640 10 client.20898 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.673+0000 7fde1110f640 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x561700030e80 0x561700033270 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bde64f0fbf130f6 server_cookie=5f20c6c5c5322f69 in_seq=0 out_seq=0 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fddf4ff9640 1 -- 172.21.15.5:0/797017789 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616fff89860 con 0x561700030e80 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fddf4ff9640 10 client.20898 handle_command_reply: tid=0 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fde1574c740 1 client.20898 shutdown 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fde1574c740 1 -- 172.21.15.5:0/797017789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 msgr2=0x56170001e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fde1574c740 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x56170001e5e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00b450 tx=0x7fddfc014040 comp rx=0 tx=0).stop 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fde1574c740 1 -- 172.21.15.5:0/797017789 shutdown_connections 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fde1574c740 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x561700030e80 0x561700033270 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fde1574c740 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x56170001e5e0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fde1574c740 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561700017b40 0x56170001dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fde1574c740 1 --2- 172.21.15.5:0/797017789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617000165d0 0x561700020d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fddf4ff9640 0 client.20898 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fde1574c740 1 -- 172.21.15.5:0/797017789 >> 172.21.15.5:0/797017789 conn(0x5616fffdf310 msgr2=0x561700018960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fde1574c740 1 -- 172.21.15.5:0/797017789 shutdown_connections 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fde1574c740 1 -- 172.21.15.5:0/797017789 wait complete. 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.674+0000 7fde1574c740 20 client.20898 trim_cache size 0 max 16384 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.687+0000 7fde1574c740 1 Processor -- start 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.687+0000 7fde1574c740 1 -- start start 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.687+0000 7fde1574c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617000165d0 0x561700012e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.687+0000 7fde1574c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700017b40 0x56170002d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.687+0000 7fde1574c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x5617000190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.687+0000 7fde1574c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1afa60 con 0x561700021370 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.687+0000 7fde1574c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1b1600 con 0x5617000165d0 2024-09-19T06:25:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.687+0000 7fde1574c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1b0130 con 0x561700017b40 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fde1090e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x5617000190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fde1090e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x5617000190f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36008/0 (socket says 172.21.15.5:36008) 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fde1090e640 1 -- 172.21.15.5:0/1543796096 learned_addr learned my addr 172.21.15.5:0/1543796096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fde1090e640 1 --2- 172.21.15.5:0/1543796096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617000165d0 0x561700012e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fde1110f640 1 --2- 172.21.15.5:0/1543796096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700017b40 0x56170002d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fde1090e640 1 -- 172.21.15.5:0/1543796096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700017b40 msgr2=0x56170002d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fde1090e640 1 --2- 172.21.15.5:0/1543796096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700017b40 0x56170002d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fde1090e640 1 -- 172.21.15.5:0/1543796096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617000165d0 msgr2=0x561700012e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fde1090e640 1 --2- 172.21.15.5:0/1543796096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617000165d0 0x561700012e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fde1090e640 1 -- 172.21.15.5:0/1543796096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561700033830 con 0x561700021370 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fde1110f640 1 --2- 172.21.15.5:0/1543796096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700017b40 0x56170002d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fde1090e640 1 --2- 172.21.15.5:0/1543796096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x5617000190f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0058d0 tx=0x7fddfc012f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf3e4f8e2abf94d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fddf6ffd640 1 -- 172.21.15.5:0/1543796096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc024030 con 0x561700021370 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fddf6ffd640 1 -- 172.21.15.5:0/1543796096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddfc00bd80 con 0x561700021370 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.688+0000 7fddf6ffd640 1 -- 172.21.15.5:0/1543796096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc005620 con 0x561700021370 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.689+0000 7fde1574c740 1 -- 172.21.15.5:0/1543796096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 msgr2=0x5617000190f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.689+0000 7fde1574c740 1 --2- 172.21.15.5:0/1543796096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x5617000190f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0058d0 tx=0x7fddfc012f50 comp rx=0 tx=0).stop 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.689+0000 7fde1574c740 1 -- 172.21.15.5:0/1543796096 shutdown_connections 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.689+0000 7fde1574c740 1 --2- 172.21.15.5:0/1543796096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700021370 0x5617000190f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.689+0000 7fde1574c740 1 --2- 172.21.15.5:0/1543796096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700017b40 0x56170002d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.689+0000 7fde1574c740 1 --2- 172.21.15.5:0/1543796096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617000165d0 0x561700012e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.689+0000 7fde1574c740 1 -- 172.21.15.5:0/1543796096 >> 172.21.15.5:0/1543796096 conn(0x5616fffdf310 msgr2=0x561700012700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.689+0000 7fde1574c740 1 -- 172.21.15.5:0/1543796096 shutdown_connections 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.689+0000 7fde1574c740 1 -- 172.21.15.5:0/1543796096 wait complete. 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.689+0000 7fde1574c740 1 -- start start 2024-09-19T06:25:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.689+0000 7fde1574c740 1 Processor -- start 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1574c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617000165d0 0x5617000296b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1574c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561700017b40 0x561700029bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1574c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700021370 0x56170001a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1574c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1b0f30 con 0x5617000165d0 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1574c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1b0830 con 0x561700017b40 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1574c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1ada60 con 0x561700021370 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1574c740 10 client.0 _ll_register_callbacks cb 0x7fde0800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1090e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617000165d0 0x5617000296b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1090e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617000165d0 0x5617000296b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36014/0 (socket says 172.21.15.5:36014) 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1090e640 1 -- 172.21.15.5:0/546960830 learned_addr learned my addr 172.21.15.5:0/546960830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1110f640 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561700017b40 0x561700029bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1090e640 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700021370 0x56170001a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1090e640 1 -- 172.21.15.5:0/546960830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700021370 msgr2=0x56170001a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1090e640 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700021370 0x56170001a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1090e640 1 -- 172.21.15.5:0/546960830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561700017b40 msgr2=0x561700029bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1090e640 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561700017b40 0x561700029bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1090e640 1 -- 172.21.15.5:0/546960830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56170001af10 con 0x5617000165d0 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.690+0000 7fde1110f640 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561700017b40 0x561700029bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fde1090e640 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617000165d0 0x5617000296b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fddfc01f9b0 tx=0x7fddfc01f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57f33310dfaad1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fddf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fddf4ff9640 1 -- 172.21.15.5:0/546960830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc024070 con 0x5617000165d0 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fddf4ff9640 1 -- 172.21.15.5:0/546960830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7fddfc004020 con 0x5617000165d0 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fde1574c740 1 -- 172.21.15.5:0/546960830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5616fff83d70 con 0x5617000165d0 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fddf4ff9640 1 -- 172.21.15.5:0/546960830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fddfc004220 con 0x5617000165d0 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fde1574c740 10 client.20904 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fde1574c740 1 -- 172.21.15.5:0/546960830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x56170009b3c0 con 0x5617000165d0 2024-09-19T06:25:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fddf4ff9640 1 -- 172.21.15.5:0/546960830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5616fff83d70 con 0x5617000165d0 2024-09-19T06:25:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fddf4ff9640 1 -- 172.21.15.5:0/546960830 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fddfc0038b0 con 0x5617000165d0 2024-09-19T06:25:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fde1574c740 10 client.20904 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fde1574c740 10 client.20904 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:25:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fde1574c740 20 client.20904 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fde1574c740 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56170009bab0 0x56170009ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fde1574c740 4 client.20904 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:25:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.691+0000 7fde1574c740 1 -- 172.21.15.5:0/546960830 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616fff00170 con 0x56170009bab0 2024-09-19T06:25:22.369 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:25:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1110f640 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56170009bab0 0x56170009ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fddf4ff9640 10 client.20904 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:25:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1110f640 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56170009bab0 0x56170009ded0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e76953b5ca4e630 server_cookie=c92ff5e9b2b6f0da in_seq=0 out_seq=0 2024-09-19T06:25:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fddf4ff9640 1 -- 172.21.15.5:0/546960830 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616fff00170 con 0x56170009bab0 2024-09-19T06:25:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fddf4ff9640 10 client.20904 handle_command_reply: tid=0 2024-09-19T06:25:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1574c740 1 client.20904 shutdown 2024-09-19T06:25:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1574c740 1 -- 172.21.15.5:0/546960830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617000165d0 msgr2=0x5617000296b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1574c740 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617000165d0 0x5617000296b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fddfc01f9b0 tx=0x7fddfc01f660 comp rx=0 tx=0).stop 2024-09-19T06:25:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1574c740 1 -- 172.21.15.5:0/546960830 shutdown_connections 2024-09-19T06:25:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1574c740 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56170009bab0 0x56170009ded0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1574c740 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561700021370 0x56170001a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1574c740 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561700017b40 0x561700029bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1574c740 1 --2- 172.21.15.5:0/546960830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617000165d0 0x5617000296b0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fddf4ff9640 0 client.20904 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1574c740 1 -- 172.21.15.5:0/546960830 >> 172.21.15.5:0/546960830 conn(0x5616fffdf310 msgr2=0x56170001f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1574c740 1 -- 172.21.15.5:0/546960830 shutdown_connections 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1574c740 1 -- 172.21.15.5:0/546960830 wait complete. 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.692+0000 7fde1574c740 20 client.20904 trim_cache size 0 max 16384 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.693+0000 7fde13373640 1 -- 172.21.15.5:0/3182252802 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fddf80a3430 msgr2=0x7fddf80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.693+0000 7fde13373640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fddf80a3430 0x7fddf80a58f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fde040b9520 tx=0x7fde0408ee40 comp rx=0 tx=0).stop 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.693+0000 7fde13373640 1 -- 172.21.15.5:0/3182252802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c1082a0 msgr2=0x7fde0c19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.693+0000 7fde13373640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c1082a0 0x7fde0c19ef40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fde04005cc0 tx=0x7fde04003c20 comp rx=0 tx=0).stop 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.693+0000 7fde13373640 1 -- 172.21.15.5:0/3182252802 shutdown_connections 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.693+0000 7fde13373640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c1082a0 0x7fde0c19ef40 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.693+0000 7fde13373640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fddf80a3430 0x7fddf80a58f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.693+0000 7fde13373640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde0c1036e0 0x7fde0c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.693+0000 7fde13373640 1 --2- 172.21.15.5:0/3182252802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde0c102d30 0x7fde0c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.693+0000 7fde13373640 1 -- 172.21.15.5:0/3182252802 >> 172.21.15.5:0/3182252802 conn(0x7fde0c0fc7d0 msgr2=0x7fde0c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.693+0000 7fde13373640 1 -- 172.21.15.5:0/3182252802 shutdown_connections 2024-09-19T06:25:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:21.693+0000 7fde13373640 1 -- 172.21.15.5:0/3182252802 wait complete. 2024-09-19T06:25:22.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85104 8.61 MB/sec execute 406 sec latency 153.191 ms 2024-09-19T06:25:22.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:22 smithi005 ceph-mon[30955]: pgmap v696: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 209 op/s 2024-09-19T06:25:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:22 smithi157 ceph-mon[35346]: pgmap v696: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 209 op/s 2024-09-19T06:25:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:22 smithi049 ceph-mon[34640]: pgmap v696: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 209 op/s 2024-09-19T06:25:23.127 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:25:23.510 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:25:23.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85245 8.60 MB/sec execute 407 sec latency 62.973 ms 2024-09-19T06:25:24.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.073+0000 7f91e24a1640 1 -- 172.21.15.5:0/581260639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91dc1082a0 msgr2=0x7f91dc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:24.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.073+0000 7f91e24a1640 1 --2- 172.21.15.5:0/581260639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91dc1082a0 0x7f91dc10ec40 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009cd0 tx=0x7f91d402a8f0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.073+0000 7f91e24a1640 1 -- 172.21.15.5:0/581260639 shutdown_connections 2024-09-19T06:25:24.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.073+0000 7f91e24a1640 1 --2- 172.21.15.5:0/581260639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91dc1082a0 0x7f91dc10ec40 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.073+0000 7f91e24a1640 1 --2- 172.21.15.5:0/581260639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91dc1036e0 0x7f91dc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.073+0000 7f91e24a1640 1 --2- 172.21.15.5:0/581260639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc102d30 0x7f91dc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.073+0000 7f91e24a1640 1 -- 172.21.15.5:0/581260639 >> 172.21.15.5:0/581260639 conn(0x7f91dc0fc7d0 msgr2=0x7f91dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.073+0000 7f91e24a1640 1 -- 172.21.15.5:0/581260639 shutdown_connections 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.073+0000 7f91e24a1640 1 -- 172.21.15.5:0/581260639 wait complete. 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.074+0000 7f91e24a1640 1 Processor -- start 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.074+0000 7f91e24a1640 1 -- start start 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91e24a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91dc102d30 0x7f91dc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91e24a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91dc1036e0 0x7f91dc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91e24a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc1082a0 0x7f91dc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91e24a1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc111f80 con 0x7f91dc102d30 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91e24a1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc111e00 con 0x7f91dc1036e0 2024-09-19T06:25:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91e24a1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc112100 con 0x7f91dc1082a0 2024-09-19T06:25:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91db7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91dc1036e0 0x7f91dc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91db7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91dc1036e0 0x7f91dc100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40520/0 (socket says 172.21.15.5:40520) 2024-09-19T06:25:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91db7fe640 1 -- 172.21.15.5:0/2312423426 learned_addr learned my addr 172.21.15.5:0/2312423426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91e0a3e640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc1082a0 0x7f91dc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91dbfff640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91dc102d30 0x7f91dc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91db7fe640 1 -- 172.21.15.5:0/2312423426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc1082a0 msgr2=0x7f91dc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91db7fe640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc1082a0 0x7f91dc101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91db7fe640 1 -- 172.21.15.5:0/2312423426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91dc102d30 msgr2=0x7f91dc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91db7fe640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91dc102d30 0x7f91dc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91db7fe640 1 -- 172.21.15.5:0/2312423426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc1a67a0 con 0x7f91dc1036e0 2024-09-19T06:25:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91dbfff640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91dc102d30 0x7f91dc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:25:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.075+0000 7f91e0a3e640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc1082a0 0x7f91dc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:25:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.076+0000 7f91db7fe640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91dc1036e0 0x7f91dc100ed0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f91d000e970 tx=0x7f91d000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.076+0000 7f91d97fa640 1 -- 172.21.15.5:0/2312423426 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d000cd80 con 0x7f91dc1036e0 2024-09-19T06:25:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.076+0000 7f91e24a1640 1 -- 172.21.15.5:0/2312423426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc1a6a90 con 0x7f91dc1036e0 2024-09-19T06:25:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.076+0000 7f91d97fa640 1 -- 172.21.15.5:0/2312423426 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91d00035f0 con 0x7f91dc1036e0 2024-09-19T06:25:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.076+0000 7f91e24a1640 1 -- 172.21.15.5:0/2312423426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc1a7020 con 0x7f91dc1036e0 2024-09-19T06:25:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.076+0000 7f91d97fa640 1 -- 172.21.15.5:0/2312423426 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0010690 con 0x7f91dc1036e0 2024-09-19T06:25:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.079+0000 7f91d97fa640 1 -- 172.21.15.5:0/2312423426 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f91d0003f80 con 0x7f91dc1036e0 2024-09-19T06:25:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.079+0000 7f91e487a740 1 Processor -- start 2024-09-19T06:25:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.079+0000 7f91e487a740 1 -- start start 2024-09-19T06:25:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.080+0000 7f91e487a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8deb40 0x55fd1d8dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.080+0000 7f91e487a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.080+0000 7f91e487a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.080+0000 7f91e487a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd1d8432f0 con 0x55fd1d8deb40 2024-09-19T06:25:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.080+0000 7f91e487a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fd1d7eacf0 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.080+0000 7f91e487a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fd1d83f2b0 con 0x55fd1d8e8370 2024-09-19T06:25:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.080+0000 7f91dbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.080+0000 7f91e0a3e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8deb40 0x55fd1d8dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.080+0000 7f91dbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8dda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40526/0 (socket says 172.21.15.5:40526) 2024-09-19T06:25:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.080+0000 7f91dbfff640 1 -- 172.21.15.5:0/3304484504 learned_addr learned my addr 172.21.15.5:0/3304484504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.080+0000 7f91db7fe640 1 --2- 172.21.15.5:0/3304484504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.080+0000 7f91d97fa640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91c40a3430 0x7f91c40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.081+0000 7f91e0a3e640 1 -- 172.21.15.5:0/3304484504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 msgr2=0x55fd1d8ea760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:24.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.081+0000 7f91e0a3e640 1 --2- 172.21.15.5:0/3304484504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8ea760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.081+0000 7f91e0a3e640 1 -- 172.21.15.5:0/3304484504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 msgr2=0x55fd1d8dda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:24.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.081+0000 7f91e0a3e640 1 --2- 172.21.15.5:0/3304484504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8dda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.081+0000 7f91e0a3e640 1 -- 172.21.15.5:0/3304484504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd1d7cf3a0 con 0x55fd1d8deb40 2024-09-19T06:25:24.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.081+0000 7f91e0a3e640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91c40a3430 0x7f91c40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.081+0000 7f91d97fa640 1 -- 172.21.15.5:0/2312423426 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f91d00cff60 con 0x7f91dc1036e0 2024-09-19T06:25:24.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.081+0000 7f91e0a3e640 1 --2- 172.21.15.5:0/3304484504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8deb40 0x55fd1d8dd090 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009bf0 tx=0x7f91d4009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f6b2ea76e112d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:24.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.081+0000 7f91e0a3e640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91c40a3430 0x7f91c40a58f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f91d402ae10 tx=0x7f91d4049000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:24.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.081+0000 7f91c27fc640 1 -- 172.21.15.5:0/3304484504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4030c30 con 0x55fd1d8deb40 2024-09-19T06:25:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.081+0000 7f91c27fc640 1 -- 172.21.15.5:0/3304484504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91d4036040 con 0x55fd1d8deb40 2024-09-19T06:25:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.081+0000 7f91c27fc640 1 -- 172.21.15.5:0/3304484504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4004750 con 0x55fd1d8deb40 2024-09-19T06:25:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.082+0000 7f91e487a740 1 -- 172.21.15.5:0/3304484504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8deb40 msgr2=0x55fd1d8dd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.082+0000 7f91e487a740 1 --2- 172.21.15.5:0/3304484504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8deb40 0x55fd1d8dd090 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009bf0 tx=0x7f91d4009880 comp rx=0 tx=0).stop 2024-09-19T06:25:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.082+0000 7f91e487a740 1 -- 172.21.15.5:0/3304484504 shutdown_connections 2024-09-19T06:25:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.082+0000 7f91e487a740 1 --2- 172.21.15.5:0/3304484504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.082+0000 7f91e487a740 1 --2- 172.21.15.5:0/3304484504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.082+0000 7f91e487a740 1 --2- 172.21.15.5:0/3304484504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8deb40 0x55fd1d8dd090 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.082+0000 7f91e487a740 1 -- 172.21.15.5:0/3304484504 >> 172.21.15.5:0/3304484504 conn(0x55fd1d8a6310 msgr2=0x55fd1d8d9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.083+0000 7f91e487a740 1 -- 172.21.15.5:0/3304484504 shutdown_connections 2024-09-19T06:25:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.083+0000 7f91e487a740 1 -- 172.21.15.5:0/3304484504 wait complete. 2024-09-19T06:25:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.083+0000 7f91e487a740 1 -- start start 2024-09-19T06:25:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.083+0000 7f91e487a740 1 Processor -- start 2024-09-19T06:25:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91e487a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91e487a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8deb40 0x55fd1d8e5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91e487a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91e487a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd1d8eb070 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91e487a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fd1d7fe060 con 0x55fd1d8deb40 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91e487a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fd1d8eb1f0 con 0x55fd1d8e8370 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91e487a740 10 client.0 _ll_register_callbacks cb 0x55fd1d5518c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91dbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91e0a3e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91db7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8deb40 0x55fd1d8e5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91dbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8e7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36056/0 (socket says 172.21.15.5:36056) 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91e0a3e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8e55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41638/0 (socket says 172.21.15.5:41638) 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.084+0000 7f91dbfff640 1 -- 172.21.15.5:0/1155414043 learned_addr learned my addr 172.21.15.5:0/1155414043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.085+0000 7f91db7fe640 1 -- 172.21.15.5:0/1155414043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 msgr2=0x55fd1d8e55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.085+0000 7f91db7fe640 1 --2- 172.21.15.5:0/1155414043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8e55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.085+0000 7f91db7fe640 1 -- 172.21.15.5:0/1155414043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 msgr2=0x55fd1d8e7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.085+0000 7f91db7fe640 1 --2- 172.21.15.5:0/1155414043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8e7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.085+0000 7f91db7fe640 1 -- 172.21.15.5:0/1155414043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd1d5b43c0 con 0x55fd1d8deb40 2024-09-19T06:25:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.085+0000 7f91dbfff640 1 --2- 172.21.15.5:0/1155414043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.085+0000 7f91db7fe640 1 --2- 172.21.15.5:0/1155414043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8deb40 0x55fd1d8e5000 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f91d00cd630 tx=0x7f91d009c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44a7d150b89fffca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.085+0000 7f91bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.085+0000 7f91bb7fe640 1 -- 172.21.15.5:0/1155414043 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d00b5060 con 0x55fd1d8deb40 2024-09-19T06:25:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.085+0000 7f91bb7fe640 1 -- 172.21.15.5:0/1155414043 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f91d00a0170 con 0x55fd1d8deb40 2024-09-19T06:25:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.086+0000 7f91e487a740 1 -- 172.21.15.5:0/1155414043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91d4036040 con 0x55fd1d8deb40 2024-09-19T06:25:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.086+0000 7f91bb7fe640 1 -- 172.21.15.5:0/1155414043 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f91d00cdac0 con 0x55fd1d8deb40 2024-09-19T06:25:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.086+0000 7f91e487a740 10 client.30290 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.086+0000 7f91e487a740 1 -- 172.21.15.5:0/1155414043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55fd1d8f6b50 con 0x55fd1d8deb40 2024-09-19T06:25:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.086+0000 7f91bb7fe640 1 -- 172.21.15.5:0/1155414043 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91d4036040 con 0x55fd1d8deb40 2024-09-19T06:25:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.086+0000 7f91bb7fe640 1 -- 172.21.15.5:0/1155414043 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f91d009c4b0 con 0x55fd1d8deb40 2024-09-19T06:25:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.086+0000 7f91e487a740 10 client.30290 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.086+0000 7f91e487a740 10 client.30290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.086+0000 7f91e487a740 20 client.30290 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.086+0000 7f91e487a740 1 --2- 172.21.15.5:0/1155414043 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fd1d8f8100 0x55fd1d8fa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.087+0000 7f91e487a740 4 client.30290 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.087+0000 7f91e487a740 1 -- 172.21.15.5:0/1155414043 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd1d850860 con 0x55fd1d8f8100 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.087+0000 7f91dbfff640 1 --2- 172.21.15.5:0/1155414043 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fd1d8f8100 0x55fd1d8fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.087+0000 7f91dbfff640 1 --2- 172.21.15.5:0/1155414043 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fd1d8f8100 0x55fd1d8fa540 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a7374689a2787f server_cookie=68c87403dd5148e2 in_seq=0 out_seq=0 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.087+0000 7f91bb7fe640 10 client.30290 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.088+0000 7f91bb7fe640 1 -- 172.21.15.5:0/1155414043 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd1d850860 con 0x55fd1d8f8100 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.088+0000 7f91bb7fe640 10 client.30290 handle_command_reply: tid=0 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.088+0000 7f91e487a740 1 client.30290 shutdown 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.089+0000 7f91e487a740 1 -- 172.21.15.5:0/1155414043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8deb40 msgr2=0x55fd1d8e5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.089+0000 7f91e487a740 1 --2- 172.21.15.5:0/1155414043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8deb40 0x55fd1d8e5000 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f91d00cd630 tx=0x7f91d009c8c0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.089+0000 7f91e487a740 1 -- 172.21.15.5:0/1155414043 shutdown_connections 2024-09-19T06:25:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.089+0000 7f91e487a740 1 --2- 172.21.15.5:0/1155414043 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fd1d8f8100 0x55fd1d8fa540 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.089+0000 7f91bb7fe640 0 client.30290 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.089+0000 7f91e487a740 1 --2- 172.21.15.5:0/1155414043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.089+0000 7f91e487a740 1 --2- 172.21.15.5:0/1155414043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8deb40 0x55fd1d8e5000 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.089+0000 7f91e487a740 1 --2- 172.21.15.5:0/1155414043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.089+0000 7f91e487a740 1 -- 172.21.15.5:0/1155414043 >> 172.21.15.5:0/1155414043 conn(0x55fd1d8a6310 msgr2=0x55fd1d8df930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.089+0000 7f91e487a740 1 -- 172.21.15.5:0/1155414043 shutdown_connections 2024-09-19T06:25:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.089+0000 7f91e487a740 1 -- 172.21.15.5:0/1155414043 wait complete. 2024-09-19T06:25:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.089+0000 7f91e487a740 20 client.30290 trim_cache size 0 max 16384 2024-09-19T06:25:24.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.110+0000 7f91e487a740 1 Processor -- start 2024-09-19T06:25:24.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.110+0000 7f91e487a740 1 -- start start 2024-09-19T06:25:24.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91e487a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91e487a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8deb40 0x55fd1d8f4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91e487a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91e487a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91c800e7d0 con 0x55fd1d8deb40 2024-09-19T06:25:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91e487a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f91c8010370 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91e487a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91c800eea0 con 0x55fd1d8e8370 2024-09-19T06:25:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91dbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91dbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8e48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40542/0 (socket says 172.21.15.5:40542) 2024-09-19T06:25:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91dbfff640 1 -- 172.21.15.5:0/198640326 learned_addr learned my addr 172.21.15.5:0/198640326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91db7fe640 1 --2- 172.21.15.5:0/198640326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8deb40 0x55fd1d8f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91e0a3e640 1 --2- 172.21.15.5:0/198640326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91dbfff640 1 -- 172.21.15.5:0/198640326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 msgr2=0x55fd1d8ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91dbfff640 1 --2- 172.21.15.5:0/198640326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91dbfff640 1 -- 172.21.15.5:0/198640326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8deb40 msgr2=0x55fd1d8f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91dbfff640 1 --2- 172.21.15.5:0/198640326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8deb40 0x55fd1d8f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.111+0000 7f91dbfff640 1 -- 172.21.15.5:0/198640326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd1d8eaad0 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.112+0000 7f91e0a3e640 1 --2- 172.21.15.5:0/198640326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:25:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.112+0000 7f91dbfff640 1 --2- 172.21.15.5:0/198640326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8e48c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f91cc012b20 tx=0x7f91cc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=840e7560124060c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.112+0000 7f91c1ffb640 1 -- 172.21.15.5:0/198640326 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc02b070 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.112+0000 7f91c1ffb640 1 -- 172.21.15.5:0/198640326 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91cc026400 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.112+0000 7f91c1ffb640 1 -- 172.21.15.5:0/198640326 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc01f660 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.113+0000 7f91e487a740 1 -- 172.21.15.5:0/198640326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 msgr2=0x55fd1d8e48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.113+0000 7f91e487a740 1 --2- 172.21.15.5:0/198640326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8e48c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f91cc012b20 tx=0x7f91cc005f90 comp rx=0 tx=0).stop 2024-09-19T06:25:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.113+0000 7f91e487a740 1 -- 172.21.15.5:0/198640326 shutdown_connections 2024-09-19T06:25:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.113+0000 7f91e487a740 1 --2- 172.21.15.5:0/198640326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8e8370 0x55fd1d8ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.113+0000 7f91e487a740 1 --2- 172.21.15.5:0/198640326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8deb40 0x55fd1d8f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.113+0000 7f91e487a740 1 --2- 172.21.15.5:0/198640326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8e48c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.113+0000 7f91e487a740 1 -- 172.21.15.5:0/198640326 >> 172.21.15.5:0/198640326 conn(0x55fd1d8a6310 msgr2=0x55fd1d8e4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.114+0000 7f91e487a740 1 -- 172.21.15.5:0/198640326 shutdown_connections 2024-09-19T06:25:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.114+0000 7f91e487a740 1 -- 172.21.15.5:0/198640326 wait complete. 2024-09-19T06:25:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.114+0000 7f91e487a740 1 -- start start 2024-09-19T06:25:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.114+0000 7f91e487a740 1 Processor -- start 2024-09-19T06:25:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91e487a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8eb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91e487a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8deb40 0x55fd1d8eba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91e487a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8e8370 0x55fd1d960dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91e487a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91c800fca0 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91e487a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f91c800f5a0 con 0x55fd1d8e8370 2024-09-19T06:25:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91e487a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91c800c7d0 con 0x55fd1d8deb40 2024-09-19T06:25:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91db7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8deb40 0x55fd1d8eba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91e0a3e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8e8370 0x55fd1d960dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91e487a740 10 client.0 _ll_register_callbacks cb 0x7f91d0097e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91db7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8deb40 0x55fd1d8eba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41662/0 (socket says 172.21.15.5:41662) 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91e0a3e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8e8370 0x55fd1d960dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40558/0 (socket says 172.21.15.5:40558) 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91db7fe640 1 -- 172.21.15.5:0/405671385 learned_addr learned my addr 172.21.15.5:0/405671385 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.115+0000 7f91dbfff640 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8eb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.116+0000 7f91dbfff640 1 -- 172.21.15.5:0/405671385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8deb40 msgr2=0x55fd1d8eba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.116+0000 7f91dbfff640 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8deb40 0x55fd1d8eba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.116+0000 7f91dbfff640 1 -- 172.21.15.5:0/405671385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8e8370 msgr2=0x55fd1d960dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.116+0000 7f91dbfff640 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8e8370 0x55fd1d960dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.116+0000 7f91dbfff640 1 -- 172.21.15.5:0/405671385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd1d8ec360 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.116+0000 7f91db7fe640 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8deb40 0x55fd1d8eba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.116+0000 7f91dbfff640 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8eb540 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f91cc028000 tx=0x7f91cc012500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a81accac4e99996 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.116+0000 7f91bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:25:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.116+0000 7f91bb7fe640 1 -- 172.21.15.5:0/405671385 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc02b040 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.116+0000 7f91bb7fe640 1 -- 172.21.15.5:0/405671385 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f91cc0267b0 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.116+0000 7f91e487a740 1 -- 172.21.15.5:0/405671385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91cc026400 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.117+0000 7f91bb7fe640 1 -- 172.21.15.5:0/405671385 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f91cc011920 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.117+0000 7f91e487a740 10 client.20922 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.117+0000 7f91e487a740 1 -- 172.21.15.5:0/405671385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55fd1d962420 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.117+0000 7f91bb7fe640 1 -- 172.21.15.5:0/405671385 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91cc026400 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.117+0000 7f91bb7fe640 1 -- 172.21.15.5:0/405671385 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f91cc004790 con 0x55fd1d8dd5d0 2024-09-19T06:25:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.117+0000 7f91e487a740 10 client.20922 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.117+0000 7f91e487a740 10 client.20922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.117+0000 7f91e487a740 20 client.20922 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.117+0000 7f91e487a740 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fd1d8f8100 0x55fd1d964a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.117+0000 7f91e487a740 4 client.20922 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:25:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.117+0000 7f91e487a740 1 -- 172.21.15.5:0/405671385 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd1d84ad70 con 0x55fd1d8f8100 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.118+0000 7f91db7fe640 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fd1d8f8100 0x55fd1d964a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.118+0000 7f91db7fe640 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fd1d8f8100 0x55fd1d964a50 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c583ab50f8bd5381 server_cookie=2fb0ea9b513217ab in_seq=0 out_seq=0 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.118+0000 7f91bb7fe640 10 client.20922 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.118+0000 7f91bb7fe640 1 -- 172.21.15.5:0/405671385 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd1d84ad70 con 0x55fd1d8f8100 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.118+0000 7f91bb7fe640 10 client.20922 handle_command_reply: tid=0 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.118+0000 7f91e487a740 1 client.20922 shutdown 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.119+0000 7f91e487a740 1 -- 172.21.15.5:0/405671385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 msgr2=0x55fd1d8eb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.119+0000 7f91e487a740 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8eb540 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f91cc028000 tx=0x7f91cc012500 comp rx=0 tx=0).stop 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.119+0000 7f91e487a740 1 -- 172.21.15.5:0/405671385 shutdown_connections 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.119+0000 7f91e487a740 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fd1d8f8100 0x55fd1d964a50 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.119+0000 7f91e487a740 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd1d8e8370 0x55fd1d960dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.119+0000 7f91bb7fe640 0 client.20922 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.119+0000 7f91e487a740 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd1d8deb40 0x55fd1d8eba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.119+0000 7f91e487a740 1 --2- 172.21.15.5:0/405671385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd1d8dd5d0 0x55fd1d8eb540 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.119+0000 7f91e487a740 1 -- 172.21.15.5:0/405671385 >> 172.21.15.5:0/405671385 conn(0x55fd1d8a6310 msgr2=0x55fd1d8e40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:24.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.119+0000 7f91e487a740 1 -- 172.21.15.5:0/405671385 shutdown_connections 2024-09-19T06:25:24.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.119+0000 7f91e487a740 1 -- 172.21.15.5:0/405671385 wait complete. 2024-09-19T06:25:24.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.119+0000 7f91e487a740 20 client.20922 trim_cache size 0 max 16384 2024-09-19T06:25:24.124 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:25:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.120+0000 7f91e24a1640 1 -- 172.21.15.5:0/2312423426 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91c40a3430 msgr2=0x7f91c40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.120+0000 7f91e24a1640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91c40a3430 0x7f91c40a58f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f91d402ae10 tx=0x7f91d4049000 comp rx=0 tx=0).stop 2024-09-19T06:25:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.120+0000 7f91e24a1640 1 -- 172.21.15.5:0/2312423426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91dc1036e0 msgr2=0x7f91dc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.120+0000 7f91e24a1640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91dc1036e0 0x7f91dc100ed0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f91d000e970 tx=0x7f91d000ee30 comp rx=0 tx=0).stop 2024-09-19T06:25:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.120+0000 7f91e24a1640 1 -- 172.21.15.5:0/2312423426 shutdown_connections 2024-09-19T06:25:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.120+0000 7f91e24a1640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f91c40a3430 0x7f91c40a58f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.120+0000 7f91e24a1640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc1082a0 0x7f91dc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.120+0000 7f91e24a1640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91dc1036e0 0x7f91dc100ed0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.120+0000 7f91e24a1640 1 --2- 172.21.15.5:0/2312423426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91dc102d30 0x7f91dc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.121+0000 7f91e24a1640 1 -- 172.21.15.5:0/2312423426 >> 172.21.15.5:0/2312423426 conn(0x7f91dc0fc7d0 msgr2=0x7f91dc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.121+0000 7f91e24a1640 1 -- 172.21.15.5:0/2312423426 shutdown_connections 2024-09-19T06:25:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:24.121+0000 7f91e24a1640 1 -- 172.21.15.5:0/2312423426 wait complete. 2024-09-19T06:25:24.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85381 8.58 MB/sec execute 408 sec latency 288.260 ms 2024-09-19T06:25:24.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:24 smithi005 ceph-mon[30955]: pgmap v697: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.2 MiB/s wr, 368 op/s 2024-09-19T06:25:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:24 smithi157 ceph-mon[35346]: pgmap v697: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.2 MiB/s wr, 368 op/s 2024-09-19T06:25:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:24 smithi049 ceph-mon[34640]: pgmap v697: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.2 MiB/s wr, 368 op/s 2024-09-19T06:25:25.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85547 8.56 MB/sec execute 409 sec latency 130.764 ms 2024-09-19T06:25:25.989 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:25:25.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:25:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:25:26.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85705 8.55 MB/sec execute 410 sec latency 82.245 ms 2024-09-19T06:25:26.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:26 smithi157 ceph-mon[35346]: pgmap v698: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 260 op/s 2024-09-19T06:25:26.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:26 smithi005 ceph-mon[30955]: pgmap v698: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 260 op/s 2024-09-19T06:25:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:26 smithi049 ceph-mon[34640]: pgmap v698: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 260 op/s 2024-09-19T06:25:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.948+0000 7ffa04d22640 1 -- 172.21.15.5:0/3749435877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa00102ef0 msgr2=0x7ffa0010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.948+0000 7ffa04d22640 1 --2- 172.21.15.5:0/3749435877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa00102ef0 0x7ffa0010da40 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009ca0 tx=0x7ff9f802a950 comp rx=0 tx=0).stop 2024-09-19T06:25:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.948+0000 7ffa04d22640 1 -- 172.21.15.5:0/3749435877 shutdown_connections 2024-09-19T06:25:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.948+0000 7ffa04d22640 1 --2- 172.21.15.5:0/3749435877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa00102ef0 0x7ffa0010da40 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.948+0000 7ffa04d22640 1 --2- 172.21.15.5:0/3749435877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa00102570 0x7ffa001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.948+0000 7ffa04d22640 1 --2- 172.21.15.5:0/3749435877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa00100e20 0x7ffa00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.948+0000 7ffa04d22640 1 -- 172.21.15.5:0/3749435877 >> 172.21.15.5:0/3749435877 conn(0x7ffa000fc7d0 msgr2=0x7ffa000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.948+0000 7ffa04d22640 1 -- 172.21.15.5:0/3749435877 shutdown_connections 2024-09-19T06:25:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.948+0000 7ffa04d22640 1 -- 172.21.15.5:0/3749435877 wait complete. 2024-09-19T06:25:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.949+0000 7ffa04d22640 1 Processor -- start 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.949+0000 7ffa04d22640 1 -- start start 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.949+0000 7ffa04d22640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa00100e20 0x7ffa00197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.949+0000 7ffa04d22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa00102570 0x7ffa00197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.949+0000 7ffa04d22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa00102ef0 0x7ffa00197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.949+0000 7ffa04d22640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa00110e80 con 0x7ffa00102ef0 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.949+0000 7ffa04d22640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa00110d00 con 0x7ffa00102570 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.949+0000 7ffa04d22640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa00111000 con 0x7ffa00100e20 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.949+0000 7ff9fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa00102ef0 0x7ffa00197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.949+0000 7ff9fdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa00102570 0x7ffa00197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ff9fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa00102ef0 0x7ffa00197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39512/0 (socket says 172.21.15.5:39512) 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ff9fe59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa00100e20 0x7ffa00197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ff9fed9d640 1 -- 172.21.15.5:0/3486194918 learned_addr learned my addr 172.21.15.5:0/3486194918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/3486194918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa00100e20 msgr2=0x7ffa00197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/3486194918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa00100e20 0x7ffa00197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/3486194918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa00102ef0 msgr2=0x7ffa00197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/3486194918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa00102ef0 0x7ffa00197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/3486194918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa001a6890 con 0x7ffa00102570 2024-09-19T06:25:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/3486194918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa00102570 0x7ffa00197560 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009f10 tx=0x7ff9f400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ff9ef7fe640 1 -- 172.21.15.5:0/3486194918 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4024070 con 0x7ffa00102570 2024-09-19T06:25:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ffa04d22640 1 -- 172.21.15.5:0/3486194918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa00199290 con 0x7ffa00102570 2024-09-19T06:25:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ff9ef7fe640 1 -- 172.21.15.5:0/3486194918 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9f400cd60 con 0x7ffa00102570 2024-09-19T06:25:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.950+0000 7ff9ef7fe640 1 -- 172.21.15.5:0/3486194918 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f401d340 con 0x7ffa00102570 2024-09-19T06:25:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.951+0000 7ffa04d22640 1 -- 172.21.15.5:0/3486194918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa00199820 con 0x7ffa00102570 2024-09-19T06:25:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.953+0000 7ff9ef7fe640 1 -- 172.21.15.5:0/3486194918 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff9f4027070 con 0x7ffa00102570 2024-09-19T06:25:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.954+0000 7ffa070fb740 1 Processor -- start 2024-09-19T06:25:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.954+0000 7ffa070fb740 1 -- start start 2024-09-19T06:25:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.954+0000 7ff9ef7fe640 1 --2- 172.21.15.5:0/3486194918 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9e80a33b0 0x7ff9e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ffa070fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 0x561e07cf3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ffa070fb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07cfe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ff9fed9d640 1 --2- 172.21.15.5:0/3486194918 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9e80a33b0 0x7ff9e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ff9ef7fe640 1 -- 172.21.15.5:0/3486194918 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7ff9f40d80c0 con 0x7ffa00102570 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ffa070fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cfed40 0x561e07d01130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ffa070fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e07c59280 con 0x561e07cf3b60 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ffa070fb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561e07c00b10 con 0x561e07cf4480 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ffa070fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561e07c55240 con 0x561e07cfed40 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ff9fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 0x561e07cf3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ff9fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 0x561e07cf3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39528/0 (socket says 172.21.15.5:39528) 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ff9fe59c640 1 -- 172.21.15.5:0/1334557221 learned_addr learned my addr 172.21.15.5:0/1334557221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/1334557221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07cfe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ff9fed9d640 1 --2- 172.21.15.5:0/1334557221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cfed40 0x561e07d01130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ff9fe59c640 1 -- 172.21.15.5:0/1334557221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cfed40 msgr2=0x561e07d01130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ff9fe59c640 1 --2- 172.21.15.5:0/1334557221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cfed40 0x561e07d01130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ff9fe59c640 1 -- 172.21.15.5:0/1334557221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 msgr2=0x561e07cfe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ff9fe59c640 1 --2- 172.21.15.5:0/1334557221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07cfe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.955+0000 7ff9fe59c640 1 -- 172.21.15.5:0/1334557221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e07c0d1f0 con 0x561e07cf3b60 2024-09-19T06:25:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.956+0000 7ff9fe59c640 1 --2- 172.21.15.5:0/1334557221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 0x561e07cf3f40 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000bd50 tx=0x7ff9f00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcf7d654b279d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.956+0000 7ff9fed9d640 1 --2- 172.21.15.5:0/3486194918 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9e80a33b0 0x7ff9e80a5870 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff9f802f860 tx=0x7ff9f802f490 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.956+0000 7ff9ecff9640 1 -- 172.21.15.5:0/1334557221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f001c680 con 0x561e07cf3b60 2024-09-19T06:25:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.956+0000 7ff9ecff9640 1 -- 172.21.15.5:0/1334557221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9f000be40 con 0x561e07cf3b60 2024-09-19T06:25:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.956+0000 7ff9ecff9640 1 -- 172.21.15.5:0/1334557221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0011aa0 con 0x561e07cf3b60 2024-09-19T06:25:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.957+0000 7ffa070fb740 1 -- 172.21.15.5:0/1334557221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 msgr2=0x561e07cf3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.957+0000 7ffa070fb740 1 --2- 172.21.15.5:0/1334557221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 0x561e07cf3f40 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000bd50 tx=0x7ff9f00107a0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.957+0000 7ffa070fb740 1 -- 172.21.15.5:0/1334557221 shutdown_connections 2024-09-19T06:25:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.958+0000 7ffa070fb740 1 --2- 172.21.15.5:0/1334557221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cfed40 0x561e07d01130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.958+0000 7ffa070fb740 1 --2- 172.21.15.5:0/1334557221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07cfe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.958+0000 7ffa070fb740 1 --2- 172.21.15.5:0/1334557221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 0x561e07cf3f40 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.958+0000 7ffa070fb740 1 -- 172.21.15.5:0/1334557221 >> 172.21.15.5:0/1334557221 conn(0x561e07cf00f0 msgr2=0x561e07cf04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.958+0000 7ffa070fb740 1 -- 172.21.15.5:0/1334557221 shutdown_connections 2024-09-19T06:25:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.959+0000 7ffa070fb740 1 -- 172.21.15.5:0/1334557221 wait complete. 2024-09-19T06:25:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.959+0000 7ffa070fb740 1 -- start start 2024-09-19T06:25:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.959+0000 7ffa070fb740 1 Processor -- start 2024-09-19T06:25:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.960+0000 7ffa070fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 0x561e07cfbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.960+0000 7ffa070fb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07cfc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.960+0000 7ffa070fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cfed40 0x561e07cfc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.960+0000 7ffa070fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e07d019b0 con 0x561e07cf3b60 2024-09-19T06:25:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.960+0000 7ffa070fb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561e07c13f10 con 0x561e07cf4480 2024-09-19T06:25:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ff9fdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07cfc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ff9fdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07cfc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55344/0 (socket says 172.21.15.5:55344) 2024-09-19T06:25:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/3726093111 learned_addr learned my addr 172.21.15.5:0/3726093111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ff9fed9d640 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cfed40 0x561e07cfc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ff9fe59c640 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 0x561e07cfbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ffa070fb740 1 -- 172.21.15.5:0/3726093111 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561e07d01b30 con 0x561e07cfed40 2024-09-19T06:25:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ffa070fb740 10 client.0 _ll_register_callbacks cb 0x561e07bdca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/3726093111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cfed40 msgr2=0x561e07cfc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cfed40 0x561e07cfc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/3726093111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 msgr2=0x561e07cfbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 0x561e07cfbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/3726093111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e079ca6a0 con 0x561e07cf4480 2024-09-19T06:25:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.961+0000 7ff9fed9d640 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cfed40 0x561e07cfc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.962+0000 7ff9fe59c640 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 0x561e07cfbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.962+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07cfc130 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40a4240 tx=0x7ff9f40a4440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d0e81a612c737 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.962+0000 7ff9e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.962+0000 7ff9e5ffb640 1 -- 172.21.15.5:0/3726093111 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40bc020 con 0x561e07cf4480 2024-09-19T06:25:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.962+0000 7ff9e5ffb640 1 -- 172.21.15.5:0/3726093111 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7ff9f40acc70 con 0x561e07cf4480 2024-09-19T06:25:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.962+0000 7ffa070fb740 1 -- 172.21.15.5:0/3726093111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561e07c667f0 con 0x561e07cf4480 2024-09-19T06:25:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.962+0000 7ff9e5ffb640 1 -- 172.21.15.5:0/3726093111 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7ff9f4085050 con 0x561e07cf4480 2024-09-19T06:25:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.962+0000 7ffa070fb740 10 client.30308 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.962+0000 7ffa070fb740 1 -- 172.21.15.5:0/3726093111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x561e07d0d580 con 0x561e07cf4480 2024-09-19T06:25:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.963+0000 7ff9e5ffb640 1 -- 172.21.15.5:0/3726093111 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561e07c667f0 con 0x561e07cf4480 2024-09-19T06:25:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.963+0000 7ff9e5ffb640 1 -- 172.21.15.5:0/3726093111 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7ff9f40a0290 con 0x561e07cf4480 2024-09-19T06:25:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.963+0000 7ffa070fb740 10 client.30308 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.963+0000 7ffa070fb740 10 client.30308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.963+0000 7ffa070fb740 20 client.30308 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.963+0000 7ffa070fb740 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e07d0ee00 0x561e07d112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.963+0000 7ffa070fb740 4 client.30308 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.963+0000 7ffa070fb740 1 -- 172.21.15.5:0/3726093111 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e07c60d00 con 0x561e07d0ee00 2024-09-19T06:25:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.963+0000 7ff9fe59c640 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e07d0ee00 0x561e07d112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.964+0000 7ff9fe59c640 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e07d0ee00 0x561e07d112c0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e93ae6b69259ba server_cookie=d480a3ed4115559e in_seq=0 out_seq=0 2024-09-19T06:25:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.964+0000 7ff9e5ffb640 10 client.30308 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ff9e5ffb640 1 -- 172.21.15.5:0/3726093111 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e07c60d00 con 0x561e07d0ee00 2024-09-19T06:25:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ff9e5ffb640 10 client.30308 handle_command_reply: tid=0 2024-09-19T06:25:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ffa070fb740 1 client.30308 shutdown 2024-09-19T06:25:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ffa070fb740 1 -- 172.21.15.5:0/3726093111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 msgr2=0x561e07cfc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ffa070fb740 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07cfc130 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40a4240 tx=0x7ff9f40a4440 comp rx=0 tx=0).stop 2024-09-19T06:25:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ffa070fb740 1 -- 172.21.15.5:0/3726093111 shutdown_connections 2024-09-19T06:25:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ffa070fb740 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e07d0ee00 0x561e07d112c0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ffa070fb740 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cfed40 0x561e07cfc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ffa070fb740 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07cfc130 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ffa070fb740 1 --2- 172.21.15.5:0/3726093111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cf3b60 0x561e07cfbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ffa070fb740 1 -- 172.21.15.5:0/3726093111 >> 172.21.15.5:0/3726093111 conn(0x561e07cf00f0 msgr2=0x561e07cf5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ff9e5ffb640 0 client.30308 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ffa070fb740 1 -- 172.21.15.5:0/3726093111 shutdown_connections 2024-09-19T06:25:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ffa070fb740 1 -- 172.21.15.5:0/3726093111 wait complete. 2024-09-19T06:25:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.965+0000 7ffa070fb740 20 client.30308 trim_cache size 0 max 16384 2024-09-19T06:25:26.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.994+0000 7ffa070fb740 1 Processor -- start 2024-09-19T06:25:26.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.994+0000 7ffa070fb740 1 -- start start 2024-09-19T06:25:26.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.994+0000 7ffa070fb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf3b60 0x561e07cfabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.994+0000 7ffa070fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf4480 0x561e07d16bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.994+0000 7ffa070fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cfed40 0x561e07d17110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:26.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.994+0000 7ffa070fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001ab6a0 con 0x561e07cfed40 2024-09-19T06:25:26.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.994+0000 7ffa070fb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001ad240 con 0x561e07cf3b60 2024-09-19T06:25:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.994+0000 7ffa070fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001abd70 con 0x561e07cf4480 2024-09-19T06:25:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.995+0000 7ff9fdd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf4480 0x561e07d16bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.995+0000 7ff9fdd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf4480 0x561e07d16bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50516/0 (socket says 172.21.15.5:50516) 2024-09-19T06:25:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.995+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/1333600512 learned_addr learned my addr 172.21.15.5:0/1333600512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.995+0000 7ff9fe59c640 1 --2- 172.21.15.5:0/1333600512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf3b60 0x561e07cfabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.995+0000 7ff9fed9d640 1 --2- 172.21.15.5:0/1333600512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cfed40 0x561e07d17110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.995+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/1333600512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf3b60 msgr2=0x561e07cfabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.995+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/1333600512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf3b60 0x561e07cfabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.995+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/1333600512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cfed40 msgr2=0x561e07d17110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.995+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/1333600512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cfed40 0x561e07d17110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.995+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/1333600512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e07ceff20 con 0x561e07cf4480 2024-09-19T06:25:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.996+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/1333600512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf4480 0x561e07d16bd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40a0600 tx=0x7ff9f40096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b9412489e44206 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.996+0000 7ff9e7fff640 1 -- 172.21.15.5:0/1333600512 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40bc070 con 0x561e07cf4480 2024-09-19T06:25:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.996+0000 7ff9e7fff640 1 -- 172.21.15.5:0/1333600512 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9f40ad100 con 0x561e07cf4480 2024-09-19T06:25:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.996+0000 7ff9e7fff640 1 -- 172.21.15.5:0/1333600512 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40a3590 con 0x561e07cf4480 2024-09-19T06:25:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.997+0000 7ffa070fb740 1 -- 172.21.15.5:0/1333600512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf4480 msgr2=0x561e07d16bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.997+0000 7ffa070fb740 1 --2- 172.21.15.5:0/1333600512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf4480 0x561e07d16bd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40a0600 tx=0x7ff9f40096d0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.997+0000 7ffa070fb740 1 -- 172.21.15.5:0/1333600512 shutdown_connections 2024-09-19T06:25:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.997+0000 7ffa070fb740 1 --2- 172.21.15.5:0/1333600512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cfed40 0x561e07d17110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.997+0000 7ffa070fb740 1 --2- 172.21.15.5:0/1333600512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf4480 0x561e07d16bd0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.997+0000 7ffa070fb740 1 --2- 172.21.15.5:0/1333600512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf3b60 0x561e07cfabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.997+0000 7ffa070fb740 1 -- 172.21.15.5:0/1333600512 >> 172.21.15.5:0/1333600512 conn(0x561e07cf00f0 msgr2=0x561e07cfa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.997+0000 7ffa070fb740 1 -- 172.21.15.5:0/1333600512 shutdown_connections 2024-09-19T06:25:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.997+0000 7ffa070fb740 1 -- 172.21.15.5:0/1333600512 wait complete. 2024-09-19T06:25:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.998+0000 7ffa070fb740 1 -- start start 2024-09-19T06:25:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.998+0000 7ffa070fb740 1 Processor -- start 2024-09-19T06:25:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.998+0000 7ffa070fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf3b60 0x561e07d01d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.998+0000 7ffa070fb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07d02280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.998+0000 7ffa070fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cfed40 0x561e07d027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ff9fe59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf3b60 0x561e07d01d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ffa070fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001acb70 con 0x561e07cfed40 2024-09-19T06:25:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ffa070fb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001ac470 con 0x561e07cf4480 2024-09-19T06:25:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ffa070fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001a96a0 con 0x561e07cf3b60 2024-09-19T06:25:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ff9fe59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf3b60 0x561e07d01d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50518/0 (socket says 172.21.15.5:50518) 2024-09-19T06:25:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ffa070fb740 10 client.0 _ll_register_callbacks cb 0x7ff9f40a0850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ff9fdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07d02280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:27.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ff9fe59c640 1 -- 172.21.15.5:0/33477859 learned_addr learned my addr 172.21.15.5:0/33477859 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/33477859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf3b60 msgr2=0x561e07d01d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/33477859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf3b60 0x561e07d01d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/33477859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cfed40 msgr2=0x561e07d027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/33477859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cfed40 0x561e07d027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:26.999+0000 7ff9fdd9b640 1 -- 172.21.15.5:0/33477859 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e07d77cf0 con 0x561e07cf4480 2024-09-19T06:25:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ff9fdd9b640 1 --2- 172.21.15.5:0/33477859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07d02280 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40a4240 tx=0x7ff9f40a0540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16aee2e90cab2b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ff9e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ff9e5ffb640 1 -- 172.21.15.5:0/33477859 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40bc070 con 0x561e07cf4480 2024-09-19T06:25:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ff9e5ffb640 1 -- 172.21.15.5:0/33477859 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7ff9f40bd250 con 0x561e07cf4480 2024-09-19T06:25:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ffa070fb740 1 -- 172.21.15.5:0/33477859 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f40ad100 con 0x561e07cf4480 2024-09-19T06:25:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ff9e5ffb640 1 -- 172.21.15.5:0/33477859 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7ff9f40a3ea0 con 0x561e07cf4480 2024-09-19T06:25:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ffa070fb740 10 client.30314 fetch_fsmap learned FSMap version 60 2024-09-19T06:25:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ffa070fb740 1 -- 172.21.15.5:0/33477859 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x561e07d78ff0 con 0x561e07cf4480 2024-09-19T06:25:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ff9e5ffb640 1 -- 172.21.15.5:0/33477859 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9f40ad100 con 0x561e07cf4480 2024-09-19T06:25:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ff9e5ffb640 1 -- 172.21.15.5:0/33477859 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7ff9f40a99d0 con 0x561e07cf4480 2024-09-19T06:25:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ffa070fb740 10 client.30314 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T06:25:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ffa070fb740 10 client.30314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.000+0000 7ffa070fb740 20 client.30314 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.001+0000 7ffa070fb740 1 --2- 172.21.15.5:0/33477859 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e07d794c0 0x561e07d7b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:27.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.001+0000 7ffa070fb740 4 client.30314 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:25:27.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.001+0000 7ffa070fb740 1 -- 172.21.15.5:0/33477859 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561e07bdd110 con 0x561e07d794c0 2024-09-19T06:25:27.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.001+0000 7ff9fe59c640 1 --2- 172.21.15.5:0/33477859 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e07d794c0 0x561e07d7b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.001+0000 7ff9e5ffb640 10 client.30314 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.001+0000 7ff9fe59c640 1 --2- 172.21.15.5:0/33477859 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e07d794c0 0x561e07d7b8b0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea2bd2b01d906b0 server_cookie=121cd0647af7fec8 in_seq=0 out_seq=0 2024-09-19T06:25:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.002+0000 7ff9e5ffb640 1 -- 172.21.15.5:0/33477859 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561e07bdd110 con 0x561e07d794c0 2024-09-19T06:25:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.002+0000 7ff9e5ffb640 10 client.30314 handle_command_reply: tid=0 2024-09-19T06:25:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.002+0000 7ffa070fb740 1 client.30314 shutdown 2024-09-19T06:25:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.002+0000 7ffa070fb740 1 -- 172.21.15.5:0/33477859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 msgr2=0x561e07d02280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.002+0000 7ffa070fb740 1 --2- 172.21.15.5:0/33477859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07d02280 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40a4240 tx=0x7ff9f40a0540 comp rx=0 tx=0).stop 2024-09-19T06:25:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.002+0000 7ffa070fb740 1 -- 172.21.15.5:0/33477859 shutdown_connections 2024-09-19T06:25:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.002+0000 7ffa070fb740 1 --2- 172.21.15.5:0/33477859 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x561e07d794c0 0x561e07d7b8b0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.002+0000 7ffa070fb740 1 --2- 172.21.15.5:0/33477859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e07cfed40 0x561e07d027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.002+0000 7ffa070fb740 1 --2- 172.21.15.5:0/33477859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561e07cf4480 0x561e07d02280 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.002+0000 7ff9e5ffb640 0 client.30314 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.002+0000 7ffa070fb740 1 --2- 172.21.15.5:0/33477859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561e07cf3b60 0x561e07d01d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.003+0000 7ffa070fb740 1 -- 172.21.15.5:0/33477859 >> 172.21.15.5:0/33477859 conn(0x561e07cf00f0 msgr2=0x561e07cface0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.003+0000 7ffa070fb740 1 -- 172.21.15.5:0/33477859 shutdown_connections 2024-09-19T06:25:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.003+0000 7ffa070fb740 1 -- 172.21.15.5:0/33477859 wait complete. 2024-09-19T06:25:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.003+0000 7ffa070fb740 20 client.30314 trim_cache size 0 max 16384 2024-09-19T06:25:27.010 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:25:27.010 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:25:27.010 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "6d3753c4-801b-477e-893a-7482f6d07be1", 2024-09-19T06:25:27.010 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:25:27.011 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:25:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.004+0000 7ffa04d22640 1 -- 172.21.15.5:0/3486194918 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9e80a33b0 msgr2=0x7ff9e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.004+0000 7ffa04d22640 1 --2- 172.21.15.5:0/3486194918 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9e80a33b0 0x7ff9e80a5870 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff9f802f860 tx=0x7ff9f802f490 comp rx=0 tx=0).stop 2024-09-19T06:25:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.004+0000 7ffa04d22640 1 -- 172.21.15.5:0/3486194918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa00102570 msgr2=0x7ffa00197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.004+0000 7ffa04d22640 1 --2- 172.21.15.5:0/3486194918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa00102570 0x7ffa00197560 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009f10 tx=0x7ff9f400ed10 comp rx=0 tx=0).stop 2024-09-19T06:25:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.004+0000 7ffa04d22640 1 -- 172.21.15.5:0/3486194918 shutdown_connections 2024-09-19T06:25:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.004+0000 7ffa04d22640 1 --2- 172.21.15.5:0/3486194918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa00102ef0 0x7ffa00197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.004+0000 7ffa04d22640 1 --2- 172.21.15.5:0/3486194918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa00102570 0x7ffa00197560 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.004+0000 7ffa04d22640 1 --2- 172.21.15.5:0/3486194918 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9e80a33b0 0x7ff9e80a5870 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.004+0000 7ffa04d22640 1 --2- 172.21.15.5:0/3486194918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa00100e20 0x7ffa00197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.004+0000 7ffa04d22640 1 -- 172.21.15.5:0/3486194918 >> 172.21.15.5:0/3486194918 conn(0x7ffa000fc7d0 msgr2=0x7ffa000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.004+0000 7ffa04d22640 1 -- 172.21.15.5:0/3486194918 shutdown_connections 2024-09-19T06:25:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:27.004+0000 7ffa04d22640 1 -- 172.21.15.5:0/3486194918 wait complete. 2024-09-19T06:25:27.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85853 8.55 MB/sec execute 411 sec latency 86.353 ms 2024-09-19T06:25:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:25:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:25:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:25:27.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:25:27.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:27 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:25:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:25:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:25:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:25:27.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:25:27.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:27 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:25:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:25:27.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:25:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:25:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:25:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:25:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:25:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:25:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:25:27.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:27.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:25:27.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:27 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:28.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85997 8.54 MB/sec execute 412 sec latency 140.318 ms 2024-09-19T06:25:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:28 smithi157 ceph-mon[35346]: pgmap v699: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.2 MiB/s wr, 380 op/s 2024-09-19T06:25:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:25:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:25:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:25:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:28.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:28 smithi005 ceph-mon[30955]: pgmap v699: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.2 MiB/s wr, 380 op/s 2024-09-19T06:25:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:25:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:25:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:25:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:28.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:28 smithi049 ceph-mon[34640]: pgmap v699: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.2 MiB/s wr, 380 op/s 2024-09-19T06:25:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:25:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:25:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:25:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:25:29.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86161 8.56 MB/sec execute 413 sec latency 135.135 ms 2024-09-19T06:25:30.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86367 8.58 MB/sec execute 414 sec latency 69.450 ms 2024-09-19T06:25:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:30 smithi005 ceph-mon[30955]: pgmap v700: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-09-19T06:25:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:30 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:25:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:30 smithi049 ceph-mon[34640]: pgmap v700: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-09-19T06:25:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:30 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:25:31.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:30 smithi157 ceph-mon[35346]: pgmap v700: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-09-19T06:25:31.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:30 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:25:31.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86557 8.59 MB/sec execute 415 sec latency 201.815 ms 2024-09-19T06:25:31.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:25:32.178 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:25:32.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86717 8.58 MB/sec execute 416 sec latency 85.101 ms 2024-09-19T06:25:32.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:32 smithi005 ceph-mon[30955]: pgmap v701: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-09-19T06:25:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:32 smithi049 ceph-mon[34640]: pgmap v701: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-09-19T06:25:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.006+0000 7f681c768640 1 -- 172.21.15.5:0/1064220005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68141009d0 msgr2=0x7f6814100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.006+0000 7f681c768640 1 --2- 172.21.15.5:0/1064220005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68141009d0 0x7f6814100e30 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f680800d940 tx=0x7f680802c3b0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.006+0000 7f681c768640 1 -- 172.21.15.5:0/1064220005 shutdown_connections 2024-09-19T06:25:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.006+0000 7f681c768640 1 --2- 172.21.15.5:0/1064220005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6814101370 0x7f681410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.006+0000 7f681c768640 1 --2- 172.21.15.5:0/1064220005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68141009d0 0x7f6814100e30 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.006+0000 7f681c768640 1 --2- 172.21.15.5:0/1064220005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68141069e0 0x7f6814106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.006+0000 7f681c768640 1 -- 172.21.15.5:0/1064220005 >> 172.21.15.5:0/1064220005 conn(0x7f68140fc6b0 msgr2=0x7f68140fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.006+0000 7f681c768640 1 -- 172.21.15.5:0/1064220005 shutdown_connections 2024-09-19T06:25:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.006+0000 7f681c768640 1 -- 172.21.15.5:0/1064220005 wait complete. 2024-09-19T06:25:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f681c768640 1 Processor -- start 2024-09-19T06:25:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f681c768640 1 -- start start 2024-09-19T06:25:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f681c768640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68141009d0 0x7f6814196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f681c768640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6814101370 0x7f68141966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f681c768640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68141069e0 0x7f681419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f681c768640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f681410fbd0 con 0x7f68141009d0 2024-09-19T06:25:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f681c768640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f681410fa50 con 0x7f68141069e0 2024-09-19T06:25:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f681c768640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f681410fd50 con 0x7f6814101370 2024-09-19T06:25:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f681a504640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68141009d0 0x7f6814196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f681a504640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68141009d0 0x7f6814196180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39572/0 (socket says 172.21.15.5:39572) 2024-09-19T06:25:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f681a504640 1 -- 172.21.15.5:0/2141213954 learned_addr learned my addr 172.21.15.5:0/2141213954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f6819d03640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6814101370 0x7f68141966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.007+0000 7f681ad05640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68141069e0 0x7f681419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.008+0000 7f681a504640 1 -- 172.21.15.5:0/2141213954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6814101370 msgr2=0x7f68141966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.008+0000 7f681a504640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6814101370 0x7f68141966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.008+0000 7f681a504640 1 -- 172.21.15.5:0/2141213954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68141069e0 msgr2=0x7f681419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.008+0000 7f681a504640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68141069e0 0x7f681419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.008+0000 7f681a504640 1 -- 172.21.15.5:0/2141213954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681419b170 con 0x7f68141009d0 2024-09-19T06:25:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.008+0000 7f6819d03640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6814101370 0x7f68141966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.008+0000 7f681a504640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68141009d0 0x7f6814196180 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6804009d50 tx=0x7f680400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.008+0000 7f68137fe640 1 -- 172.21.15.5:0/2141213954 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804003b70 con 0x7f68141009d0 2024-09-19T06:25:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.008+0000 7f681c768640 1 -- 172.21.15.5:0/2141213954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681419b460 con 0x7f68141009d0 2024-09-19T06:25:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.008+0000 7f68137fe640 1 -- 172.21.15.5:0/2141213954 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6804003d10 con 0x7f68141009d0 2024-09-19T06:25:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.009+0000 7f68137fe640 1 -- 172.21.15.5:0/2141213954 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804010790 con 0x7f68141009d0 2024-09-19T06:25:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.009+0000 7f681c768640 1 -- 172.21.15.5:0/2141213954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68141a2cf0 con 0x7f68141009d0 2024-09-19T06:25:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.012+0000 7f68137fe640 1 -- 172.21.15.5:0/2141213954 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6804004000 con 0x7f68141009d0 2024-09-19T06:25:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.012+0000 7f68137fe640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67fc0a33b0 0x7f67fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.012+0000 7f68137fe640 1 -- 172.21.15.5:0/2141213954 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f68040d03b0 con 0x7f68141009d0 2024-09-19T06:25:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.012+0000 7f6819d03640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67fc0a33b0 0x7f67fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.013+0000 7f681f5d9740 1 Processor -- start 2024-09-19T06:25:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.013+0000 7f681f5d9740 1 -- start start 2024-09-19T06:25:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.013+0000 7f681f5d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a2af0 0x55dd758a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.013+0000 7f681f5d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758a3410 0x55dd758ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.013+0000 7f681f5d9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758adcb0 0x55dd758b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.013+0000 7f6819d03640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67fc0a33b0 0x7f67fc0a5870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f680800d910 tx=0x7f680800d440 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.013+0000 7f681f5d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dd758072f0 con 0x55dd758a2af0 2024-09-19T06:25:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.013+0000 7f681f5d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dd757af270 con 0x55dd758a3410 2024-09-19T06:25:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.013+0000 7f681f5d9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dd758032b0 con 0x55dd758adcb0 2024-09-19T06:25:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.013+0000 7f681a504640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a2af0 0x55dd758a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f681a504640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a2af0 0x55dd758a2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39586/0 (socket says 172.21.15.5:39586) 2024-09-19T06:25:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f6819d03640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758a3410 0x55dd758ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f681a504640 1 -- 172.21.15.5:0/1268417138 learned_addr learned my addr 172.21.15.5:0/1268417138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f681ad05640 1 --2- 172.21.15.5:0/1268417138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758adcb0 0x55dd758b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f681a504640 1 -- 172.21.15.5:0/1268417138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758adcb0 msgr2=0x55dd758b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f681a504640 1 --2- 172.21.15.5:0/1268417138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758adcb0 0x55dd758b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f681a504640 1 -- 172.21.15.5:0/1268417138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758a3410 msgr2=0x55dd758ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f681a504640 1 --2- 172.21.15.5:0/1268417138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758a3410 0x55dd758ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f681a504640 1 -- 172.21.15.5:0/1268417138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd757ae340 con 0x55dd758a2af0 2024-09-19T06:25:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f681a504640 1 --2- 172.21.15.5:0/1268417138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a2af0 0x55dd758a2ed0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6804099040 tx=0x7f6804009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169495948fb575 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f6810ff9640 1 -- 172.21.15.5:0/1268417138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68040b4020 con 0x55dd758a2af0 2024-09-19T06:25:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f6810ff9640 1 -- 172.21.15.5:0/1268417138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f680409c670 con 0x55dd758a2af0 2024-09-19T06:25:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.014+0000 7f6810ff9640 1 -- 172.21.15.5:0/1268417138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68040b0b90 con 0x55dd758a2af0 2024-09-19T06:25:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.015+0000 7f681f5d9740 1 -- 172.21.15.5:0/1268417138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a2af0 msgr2=0x55dd758a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.015+0000 7f681f5d9740 1 --2- 172.21.15.5:0/1268417138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a2af0 0x55dd758a2ed0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6804099040 tx=0x7f6804009f90 comp rx=0 tx=0).stop 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.016+0000 7f681f5d9740 1 -- 172.21.15.5:0/1268417138 shutdown_connections 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.016+0000 7f681f5d9740 1 --2- 172.21.15.5:0/1268417138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758adcb0 0x55dd758b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.016+0000 7f681f5d9740 1 --2- 172.21.15.5:0/1268417138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758a3410 0x55dd758ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.016+0000 7f681f5d9740 1 --2- 172.21.15.5:0/1268417138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a2af0 0x55dd758a2ed0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.016+0000 7f681f5d9740 1 -- 172.21.15.5:0/1268417138 >> 172.21.15.5:0/1268417138 conn(0x55dd7589f060 msgr2=0x55dd7589f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.016+0000 7f681f5d9740 1 -- 172.21.15.5:0/1268417138 shutdown_connections 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.016+0000 7f681f5d9740 1 -- 172.21.15.5:0/1268417138 wait complete. 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.016+0000 7f681f5d9740 1 -- start start 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.017+0000 7f681f5d9740 1 Processor -- start 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.017+0000 7f681f5d9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.017+0000 7f681f5d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a3410 0x55dd758ab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.017+0000 7f681f5d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758adcb0 0x55dd758ab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.017+0000 7f681f5d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dd758b0880 con 0x55dd758a3410 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.017+0000 7f681a504640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.017+0000 7f681a504640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758aabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50530/0 (socket says 172.21.15.5:50530) 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.017+0000 7f6819d03640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a3410 0x55dd758ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.017+0000 7f681a504640 1 -- 172.21.15.5:0/4181053315 learned_addr learned my addr 172.21.15.5:0/4181053315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.017+0000 7f681f5d9740 1 -- 172.21.15.5:0/4181053315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dd75790290 con 0x55dd758adcb0 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.017+0000 7f681f5d9740 1 -- 172.21.15.5:0/4181053315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dd758b0a00 con 0x55dd758a2af0 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.017+0000 7f681ad05640 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758adcb0 0x55dd758ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.018+0000 7f681f5d9740 10 client.0 _ll_register_callbacks cb 0x55dd757c1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.018+0000 7f681a504640 1 -- 172.21.15.5:0/4181053315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758adcb0 msgr2=0x55dd758ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.018+0000 7f681a504640 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758adcb0 0x55dd758ab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.018+0000 7f681a504640 1 -- 172.21.15.5:0/4181053315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a3410 msgr2=0x55dd758ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.018+0000 7f681a504640 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a3410 0x55dd758ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.018+0000 7f681a504640 1 -- 172.21.15.5:0/4181053315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd7551a210 con 0x55dd758a2af0 2024-09-19T06:25:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.018+0000 7f6819d03640 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a3410 0x55dd758ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.018+0000 7f681a504640 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758aabd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f680401f000 tx=0x7f6804009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fde070ef3cb6bbc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.018+0000 7f67f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:25:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.018+0000 7f67f9ffb640 1 -- 172.21.15.5:0/4181053315 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68040b4070 con 0x55dd758a2af0 2024-09-19T06:25:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.019+0000 7f681f5d9740 1 -- 172.21.15.5:0/4181053315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f680409c670 con 0x55dd758a2af0 2024-09-19T06:25:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.019+0000 7f67f9ffb640 1 -- 172.21.15.5:0/4181053315 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f68040b0da0 con 0x55dd758a2af0 2024-09-19T06:25:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.019+0000 7f67f9ffb640 1 -- 172.21.15.5:0/4181053315 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f68040a0a20 con 0x55dd758a2af0 2024-09-19T06:25:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.019+0000 7f681f5d9740 10 client.30247 fetch_fsmap learned FSMap version 61 2024-09-19T06:25:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.019+0000 7f681f5d9740 1 -- 172.21.15.5:0/4181053315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55dd755e6620 con 0x55dd758a2af0 2024-09-19T06:25:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.020+0000 7f67f9ffb640 1 -- 172.21.15.5:0/4181053315 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68040ba070 con 0x55dd758a2af0 2024-09-19T06:25:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.020+0000 7f67f9ffb640 1 -- 172.21.15.5:0/4181053315 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 61) ==== 8366+0+0 (secure 0 0 0) 0x7f680407d020 con 0x55dd758a2af0 2024-09-19T06:25:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.020+0000 7f681f5d9740 10 client.30247 fetch_fsmap finished waiting for FSMap version 61 2024-09-19T06:25:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.020+0000 7f681f5d9740 10 client.30247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.020+0000 7f681f5d9740 20 client.30247 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.020+0000 7f681f5d9740 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dd758bdbd0 0x55dd758c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.020+0000 7f681f5d9740 4 client.30247 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.020+0000 7f681f5d9740 1 -- 172.21.15.5:0/4181053315 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f680409c670 con 0x55dd758bdbd0 2024-09-19T06:25:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.021+0000 7f6819d03640 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dd758bdbd0 0x55dd758c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.021+0000 7f67f9ffb640 10 client.30247 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.021+0000 7f6819d03640 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dd758bdbd0 0x55dd758c0040 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e7e2909dfb6ef server_cookie=9c531b8b3720c4b8 in_seq=0 out_seq=0 2024-09-19T06:25:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.022+0000 7f67f9ffb640 1 -- 172.21.15.5:0/4181053315 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f680409c670 con 0x55dd758bdbd0 2024-09-19T06:25:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.022+0000 7f67f9ffb640 10 client.30247 handle_command_reply: tid=0 2024-09-19T06:25:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.022+0000 7f681f5d9740 1 client.30247 shutdown 2024-09-19T06:25:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.022+0000 7f681f5d9740 1 -- 172.21.15.5:0/4181053315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 msgr2=0x55dd758aabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.022+0000 7f681f5d9740 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758aabd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f680401f000 tx=0x7f6804009200 comp rx=0 tx=0).stop 2024-09-19T06:25:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.022+0000 7f681f5d9740 1 -- 172.21.15.5:0/4181053315 shutdown_connections 2024-09-19T06:25:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.022+0000 7f681f5d9740 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dd758bdbd0 0x55dd758c0040 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.022+0000 7f681f5d9740 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758adcb0 0x55dd758ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.022+0000 7f681f5d9740 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a3410 0x55dd758ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.022+0000 7f681f5d9740 1 --2- 172.21.15.5:0/4181053315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758aabd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.022+0000 7f681f5d9740 1 -- 172.21.15.5:0/4181053315 >> 172.21.15.5:0/4181053315 conn(0x55dd7589f060 msgr2=0x55dd758a4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.022+0000 7f67f9ffb640 0 client.30247 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.023+0000 7f681f5d9740 1 -- 172.21.15.5:0/4181053315 shutdown_connections 2024-09-19T06:25:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.023+0000 7f681f5d9740 1 -- 172.21.15.5:0/4181053315 wait complete. 2024-09-19T06:25:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.023+0000 7f681f5d9740 20 client.30247 trim_cache size 0 max 16384 2024-09-19T06:25:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.042+0000 7f681f5d9740 1 Processor -- start 2024-09-19T06:25:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.042+0000 7f681f5d9740 1 -- start start 2024-09-19T06:25:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.042+0000 7f681f5d9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758ab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.042+0000 7f681f5d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758a3410 0x55dd758ab640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.042+0000 7f681f5d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758adcb0 0x55dd758c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.042+0000 7f681f5d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68141ab730 con 0x55dd758adcb0 2024-09-19T06:25:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.042+0000 7f681f5d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f68141ad2a0 con 0x55dd758a3410 2024-09-19T06:25:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.042+0000 7f681f5d9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f68141abe00 con 0x55dd758a2af0 2024-09-19T06:25:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.042+0000 7f681a504640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f681a504640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758ab100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50546/0 (socket says 172.21.15.5:50546) 2024-09-19T06:25:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f6819d03640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758a3410 0x55dd758ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f681a504640 1 -- 172.21.15.5:0/3236774176 learned_addr learned my addr 172.21.15.5:0/3236774176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f681ad05640 1 --2- 172.21.15.5:0/3236774176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758adcb0 0x55dd758c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f681a504640 1 -- 172.21.15.5:0/3236774176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758a3410 msgr2=0x55dd758ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f681a504640 1 --2- 172.21.15.5:0/3236774176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758a3410 0x55dd758ab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f681a504640 1 -- 172.21.15.5:0/3236774176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758adcb0 msgr2=0x55dd758c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f681a504640 1 --2- 172.21.15.5:0/3236774176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758adcb0 0x55dd758c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f681a504640 1 -- 172.21.15.5:0/3236774176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd758a3240 con 0x55dd758a2af0 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f681ad05640 1 --2- 172.21.15.5:0/3236774176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758adcb0 0x55dd758c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f6819d03640 1 --2- 172.21.15.5:0/3236774176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758a3410 0x55dd758ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f681a504640 1 --2- 172.21.15.5:0/3236774176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758ab100 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6804005fa0 tx=0x7f6804009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e59659184d237ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f67fbfff640 1 -- 172.21.15.5:0/3236774176 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68040b4030 con 0x55dd758a2af0 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f67fbfff640 1 -- 172.21.15.5:0/3236774176 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68040ba070 con 0x55dd758a2af0 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.043+0000 7f67fbfff640 1 -- 172.21.15.5:0/3236774176 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68040983c0 con 0x55dd758a2af0 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.044+0000 7f681f5d9740 1 -- 172.21.15.5:0/3236774176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 msgr2=0x55dd758ab100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.044+0000 7f681f5d9740 1 --2- 172.21.15.5:0/3236774176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758ab100 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6804005fa0 tx=0x7f6804009f90 comp rx=0 tx=0).stop 2024-09-19T06:25:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.044+0000 7f681f5d9740 1 -- 172.21.15.5:0/3236774176 shutdown_connections 2024-09-19T06:25:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.044+0000 7f681f5d9740 1 --2- 172.21.15.5:0/3236774176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758adcb0 0x55dd758c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.044+0000 7f681f5d9740 1 --2- 172.21.15.5:0/3236774176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758a3410 0x55dd758ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.044+0000 7f681f5d9740 1 --2- 172.21.15.5:0/3236774176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758ab100 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.044+0000 7f681f5d9740 1 -- 172.21.15.5:0/3236774176 >> 172.21.15.5:0/3236774176 conn(0x55dd7589f060 msgr2=0x55dd758acc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.044+0000 7f681f5d9740 1 -- 172.21.15.5:0/3236774176 shutdown_connections 2024-09-19T06:25:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.044+0000 7f681f5d9740 1 -- 172.21.15.5:0/3236774176 wait complete. 2024-09-19T06:25:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681f5d9740 1 -- start start 2024-09-19T06:25:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681f5d9740 1 Processor -- start 2024-09-19T06:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681f5d9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681f5d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a3410 0x55dd758b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681f5d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758adcb0 0x55dd758b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681f5d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68141acbd0 con 0x55dd758a3410 2024-09-19T06:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681f5d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f68141ac500 con 0x55dd758adcb0 2024-09-19T06:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681f5d9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f68141a9730 con 0x55dd758a2af0 2024-09-19T06:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681f5d9740 10 client.0 _ll_register_callbacks cb 0x7f68040093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f6819d03640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a3410 0x55dd758b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681a504640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681ad05640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758adcb0 0x55dd758b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681a504640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50562/0 (socket says 172.21.15.5:50562) 2024-09-19T06:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681a504640 1 -- 172.21.15.5:0/3999102510 learned_addr learned my addr 172.21.15.5:0/3999102510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.045+0000 7f681a504640 1 -- 172.21.15.5:0/3999102510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758adcb0 msgr2=0x55dd758b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f681a504640 1 --2- 172.21.15.5:0/3999102510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758adcb0 0x55dd758b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f681a504640 1 -- 172.21.15.5:0/3999102510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a3410 msgr2=0x55dd758b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f681a504640 1 --2- 172.21.15.5:0/3999102510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a3410 0x55dd758b65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f681a504640 1 -- 172.21.15.5:0/3999102510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd758b1e10 con 0x55dd758a2af0 2024-09-19T06:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f6819d03640 1 --2- 172.21.15.5:0/3999102510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a3410 0x55dd758b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f681a504640 1 --2- 172.21.15.5:0/3999102510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758b6070 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f680409c370 tx=0x7f680409bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae7491e3945c5666 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f67f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f67f9ffb640 1 -- 172.21.15.5:0/3999102510 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680409c670 con 0x55dd758a2af0 2024-09-19T06:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f681f5d9740 1 -- 172.21.15.5:0/3999102510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68040ba070 con 0x55dd758a2af0 2024-09-19T06:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f67f9ffb640 1 -- 172.21.15.5:0/3999102510 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f680409a970 con 0x55dd758a2af0 2024-09-19T06:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f67f9ffb640 1 -- 172.21.15.5:0/3999102510 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f68040ad050 con 0x55dd758a2af0 2024-09-19T06:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f681f5d9740 10 client.30253 fetch_fsmap learned FSMap version 61 2024-09-19T06:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f681f5d9740 1 -- 172.21.15.5:0/3999102510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55dd758b1e10 con 0x55dd758a2af0 2024-09-19T06:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.046+0000 7f67f9ffb640 1 -- 172.21.15.5:0/3999102510 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68040ba070 con 0x55dd758a2af0 2024-09-19T06:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.047+0000 7f67f9ffb640 1 -- 172.21.15.5:0/3999102510 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 61) ==== 8366+0+0 (secure 0 0 0) 0x7f680407d070 con 0x55dd758a2af0 2024-09-19T06:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.047+0000 7f681f5d9740 10 client.30253 fetch_fsmap finished waiting for FSMap version 61 2024-09-19T06:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.047+0000 7f681f5d9740 10 client.30253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.047+0000 7f681f5d9740 20 client.30253 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.047+0000 7f681f5d9740 1 --2- 172.21.15.5:0/3999102510 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dd759287d0 0x55dd7592abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.047+0000 7f681f5d9740 4 client.30253 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.047+0000 7f681f5d9740 1 -- 172.21.15.5:0/3999102510 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd75814a20 con 0x55dd759287d0 2024-09-19T06:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.047+0000 7f6819d03640 1 --2- 172.21.15.5:0/3999102510 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dd759287d0 0x55dd7592abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.047+0000 7f6819d03640 1 --2- 172.21.15.5:0/3999102510 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dd759287d0 0x55dd7592abc0 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cfd23fe4a98b6fd server_cookie=224e16cc603da174 in_seq=0 out_seq=0 2024-09-19T06:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.047+0000 7f67f9ffb640 10 client.30253 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:33.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:32 smithi157 ceph-mon[35346]: pgmap v701: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-09-19T06:25:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.145+0000 7f67f9ffb640 1 -- 172.21.15.5:0/3999102510 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd75814a20 con 0x55dd759287d0 2024-09-19T06:25:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.145+0000 7f67f9ffb640 10 client.30253 handle_command_reply: tid=0 2024-09-19T06:25:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.145+0000 7f681f5d9740 1 client.30253 shutdown 2024-09-19T06:25:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.145+0000 7f681f5d9740 1 -- 172.21.15.5:0/3999102510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 msgr2=0x55dd758b6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.146+0000 7f681f5d9740 1 --2- 172.21.15.5:0/3999102510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758b6070 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f680409c370 tx=0x7f680409bcf0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.146+0000 7f681f5d9740 1 -- 172.21.15.5:0/3999102510 shutdown_connections 2024-09-19T06:25:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.146+0000 7f681f5d9740 1 --2- 172.21.15.5:0/3999102510 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55dd759287d0 0x55dd7592abc0 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.146+0000 7f681f5d9740 1 --2- 172.21.15.5:0/3999102510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dd758adcb0 0x55dd758b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.146+0000 7f67f9ffb640 0 client.30253 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.146+0000 7f681f5d9740 1 --2- 172.21.15.5:0/3999102510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd758a3410 0x55dd758b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.146+0000 7f681f5d9740 1 --2- 172.21.15.5:0/3999102510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd758a2af0 0x55dd758b6070 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.146+0000 7f681f5d9740 1 -- 172.21.15.5:0/3999102510 >> 172.21.15.5:0/3999102510 conn(0x55dd7589f060 msgr2=0x55dd758a4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.146+0000 7f681f5d9740 1 -- 172.21.15.5:0/3999102510 shutdown_connections 2024-09-19T06:25:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.146+0000 7f681f5d9740 1 -- 172.21.15.5:0/3999102510 wait complete. 2024-09-19T06:25:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.146+0000 7f681f5d9740 20 client.30253 trim_cache size 0 max 16384 2024-09-19T06:25:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:25:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:25:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:25:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:25:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.146+0000 7f681c768640 1 -- 172.21.15.5:0/2141213954 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67fc0a33b0 msgr2=0x7f67fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.147+0000 7f681c768640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67fc0a33b0 0x7f67fc0a5870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f680800d910 tx=0x7f680800d440 comp rx=0 tx=0).stop 2024-09-19T06:25:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.147+0000 7f681c768640 1 -- 172.21.15.5:0/2141213954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68141009d0 msgr2=0x7f6814196180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.147+0000 7f681c768640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68141009d0 0x7f6814196180 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6804009d50 tx=0x7f680400c6a0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.147+0000 7f681c768640 1 -- 172.21.15.5:0/2141213954 shutdown_connections 2024-09-19T06:25:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.147+0000 7f681c768640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68141069e0 0x7f681419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.147+0000 7f681c768640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f67fc0a33b0 0x7f67fc0a5870 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.147+0000 7f681c768640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6814101370 0x7f68141966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.147+0000 7f681c768640 1 --2- 172.21.15.5:0/2141213954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68141009d0 0x7f6814196180 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.147+0000 7f681c768640 1 -- 172.21.15.5:0/2141213954 >> 172.21.15.5:0/2141213954 conn(0x7f68140fc6b0 msgr2=0x7f68140fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.147+0000 7f681c768640 1 -- 172.21.15.5:0/2141213954 shutdown_connections 2024-09-19T06:25:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:33.147+0000 7f681c768640 1 -- 172.21.15.5:0/2141213954 wait complete. 2024-09-19T06:25:33.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86835 8.58 MB/sec execute 417 sec latency 113.268 ms 2024-09-19T06:25:34.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86968 8.58 MB/sec execute 418 sec latency 107.609 ms 2024-09-19T06:25:34.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:34 smithi005 ceph-mon[30955]: pgmap v702: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.1 MiB/s wr, 452 op/s 2024-09-19T06:25:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:34 smithi049 ceph-mon[34640]: pgmap v702: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.1 MiB/s wr, 452 op/s 2024-09-19T06:25:35.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:34 smithi157 ceph-mon[35346]: pgmap v702: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.1 MiB/s wr, 452 op/s 2024-09-19T06:25:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.506+0000 7f32bb761640 1 -- 172.21.15.5:0/1715682261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4106a90 msgr2=0x7f32b4106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.506+0000 7f32bb761640 1 --2- 172.21.15.5:0/1715682261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4106a90 0x7f32b4106e70 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009930 tx=0x7f32b002a440 comp rx=0 tx=0).stop 2024-09-19T06:25:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.506+0000 7f32bb761640 1 -- 172.21.15.5:0/1715682261 shutdown_connections 2024-09-19T06:25:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.506+0000 7f32bb761640 1 --2- 172.21.15.5:0/1715682261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32b4101430 0x7f32b410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.506+0000 7f32bb761640 1 --2- 172.21.15.5:0/1715682261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b4100a90 0x7f32b4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.506+0000 7f32bb761640 1 --2- 172.21.15.5:0/1715682261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4106a90 0x7f32b4106e70 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.506+0000 7f32bb761640 1 -- 172.21.15.5:0/1715682261 >> 172.21.15.5:0/1715682261 conn(0x7f32b40fc7d0 msgr2=0x7f32b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.506+0000 7f32bb761640 1 -- 172.21.15.5:0/1715682261 shutdown_connections 2024-09-19T06:25:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.506+0000 7f32bb761640 1 -- 172.21.15.5:0/1715682261 wait complete. 2024-09-19T06:25:35.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.507+0000 7f32bb761640 1 Processor -- start 2024-09-19T06:25:35.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.507+0000 7f32bb761640 1 -- start start 2024-09-19T06:25:35.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.507+0000 7f32bb761640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b4100a90 0x7f32b4196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32bb761640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32b4101430 0x7f32b41966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32bb761640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4106a90 0x7f32b419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32bb761640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32b410fc90 con 0x7f32b4106a90 2024-09-19T06:25:35.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32bb761640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32b410fb10 con 0x7f32b4101430 2024-09-19T06:25:35.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32bb761640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f32b410fe10 con 0x7f32b4100a90 2024-09-19T06:25:35.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32b94fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b4100a90 0x7f32b4196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32b94fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b4100a90 0x7f32b4196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55872/0 (socket says 172.21.15.5:55872) 2024-09-19T06:25:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32b94fd640 1 -- 172.21.15.5:0/2830320829 learned_addr learned my addr 172.21.15.5:0/2830320829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32b8cfc640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32b4101430 0x7f32b41966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4106a90 0x7f32b419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32b94fd640 1 -- 172.21.15.5:0/2830320829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32b4101430 msgr2=0x7f32b41966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32b94fd640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32b4101430 0x7f32b41966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32b94fd640 1 -- 172.21.15.5:0/2830320829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4106a90 msgr2=0x7f32b419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32b94fd640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4106a90 0x7f32b419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32b94fd640 1 -- 172.21.15.5:0/2830320829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b419b140 con 0x7f32b4100a90 2024-09-19T06:25:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.508+0000 7f32b94fd640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b4100a90 0x7f32b4196190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f32b0005ab0 tx=0x7f32b00045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.509+0000 7f32aa7fc640 1 -- 172.21.15.5:0/2830320829 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0050070 con 0x7f32b4100a90 2024-09-19T06:25:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.509+0000 7f32aa7fc640 1 -- 172.21.15.5:0/2830320829 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32b0031b50 con 0x7f32b4100a90 2024-09-19T06:25:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.509+0000 7f32bb761640 1 -- 172.21.15.5:0/2830320829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b419b3d0 con 0x7f32b4100a90 2024-09-19T06:25:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.509+0000 7f32aa7fc640 1 -- 172.21.15.5:0/2830320829 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0035030 con 0x7f32b4100a90 2024-09-19T06:25:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.509+0000 7f32bb761640 1 -- 172.21.15.5:0/2830320829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b419b7f0 con 0x7f32b4100a90 2024-09-19T06:25:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32bdb3a740 1 Processor -- start 2024-09-19T06:25:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32bdb3a740 1 -- start start 2024-09-19T06:25:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32bdb3a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb714880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32bdb3a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb71f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32bdb3a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb721a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32bdb3a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5592cb67a2e0 con 0x5592cb71f680 2024-09-19T06:25:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32bdb3a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5592cb621ce0 con 0x5592cb714dc0 2024-09-19T06:25:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32bdb3a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5592cb6762a0 con 0x5592cb7144a0 2024-09-19T06:25:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32b9cfe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb714880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32b9cfe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb714880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55888/0 (socket says 172.21.15.5:55888) 2024-09-19T06:25:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32b9cfe640 1 -- 172.21.15.5:0/795690749 learned_addr learned my addr 172.21.15.5:0/795690749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32b8cfc640 1 --2- 172.21.15.5:0/795690749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb721a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32b9cfe640 1 -- 172.21.15.5:0/795690749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 msgr2=0x5592cb71f140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:25:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/795690749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb71f140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32b9cfe640 1 -- 172.21.15.5:0/795690749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 msgr2=0x5592cb721a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/795690749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb721a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.512+0000 7f32b9cfe640 1 -- 172.21.15.5:0/795690749 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592cb606440 con 0x5592cb7144a0 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.513+0000 7f32aa7fc640 1 -- 172.21.15.5:0/2830320829 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f32b0053070 con 0x7f32b4100a90 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.513+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/795690749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb714880 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009aa0 tx=0x7f32ac009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e2f5d6c36b2cec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.513+0000 7f329f7fe640 1 -- 172.21.15.5:0/795690749 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00ecf0 con 0x5592cb7144a0 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.513+0000 7f329f7fe640 1 -- 172.21.15.5:0/795690749 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32ac00ee90 con 0x5592cb7144a0 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.513+0000 7f329f7fe640 1 -- 172.21.15.5:0/795690749 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac004030 con 0x5592cb7144a0 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.514+0000 7f32bdb3a740 1 -- 172.21.15.5:0/795690749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 msgr2=0x5592cb714880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.514+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/795690749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb714880 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009aa0 tx=0x7f32ac009500 comp rx=0 tx=0).stop 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.514+0000 7f32bdb3a740 1 -- 172.21.15.5:0/795690749 shutdown_connections 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.514+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/795690749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb721a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.514+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/795690749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb71f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.514+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/795690749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb714880 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.514+0000 7f32bdb3a740 1 -- 172.21.15.5:0/795690749 >> 172.21.15.5:0/795690749 conn(0x5592cb710a30 msgr2=0x5592cb710e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.515+0000 7f32bdb3a740 1 -- 172.21.15.5:0/795690749 shutdown_connections 2024-09-19T06:25:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.515+0000 7f32bdb3a740 1 -- 172.21.15.5:0/795690749 wait complete. 2024-09-19T06:25:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.516+0000 7f32aa7fc640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f32a00a3340 0x7f32a00a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.516+0000 7f32bdb3a740 1 -- start start 2024-09-19T06:25:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.516+0000 7f32bdb3a740 1 Processor -- start 2024-09-19T06:25:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.516+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f32a00a3340 0x7f32a00a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.517+0000 7f32aa7fc640 1 -- 172.21.15.5:0/2830320829 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f32b0101730 con 0x7f32b4100a90 2024-09-19T06:25:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f32a00a3340 0x7f32a00a5800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009270 tx=0x7f32ac000f40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32bdb3a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb7144a0 0x5592cb71eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32bdb3a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb71c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32bdb3a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb71f680 0x5592cb71c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32bdb3a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5592cb722250 con 0x5592cb7144a0 2024-09-19T06:25:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32bdb3a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5592cb635050 con 0x5592cb714dc0 2024-09-19T06:25:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32bdb3a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5592cb7223d0 con 0x5592cb71f680 2024-09-19T06:25:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32bdb3a740 10 client.0 _ll_register_callbacks cb 0x5592cb675160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32b9cfe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb71f680 0x5592cb71c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32b8cfc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb71c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32b9cfe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb71f680 0x5592cb71c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55900/0 (socket says 172.21.15.5:55900) 2024-09-19T06:25:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32b8cfc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb71c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:45810/0 (socket says 172.21.15.5:45810) 2024-09-19T06:25:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32b9cfe640 1 -- 172.21.15.5:0/402176019 learned_addr learned my addr 172.21.15.5:0/402176019 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.518+0000 7f32b94fd640 1 --2- 172.21.15.5:0/402176019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb7144a0 0x5592cb71eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.519+0000 7f32b9cfe640 1 -- 172.21.15.5:0/402176019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 msgr2=0x5592cb71c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.519+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/402176019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb71c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.519+0000 7f32b9cfe640 1 -- 172.21.15.5:0/402176019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb7144a0 msgr2=0x5592cb71eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.519+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/402176019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb7144a0 0x5592cb71eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.519+0000 7f32b9cfe640 1 -- 172.21.15.5:0/402176019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592cb3eb3c0 con 0x5592cb71f680 2024-09-19T06:25:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.519+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/402176019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb71f680 0x5592cb71c800 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f32ac01b000 tx=0x7f32ac005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c52927c98d44f779 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.519+0000 7f329cff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:25:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.519+0000 7f329cff9640 1 -- 172.21.15.5:0/402176019 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac0047c0 con 0x5592cb71f680 2024-09-19T06:25:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.519+0000 7f329cff9640 1 -- 172.21.15.5:0/402176019 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f32ac01f450 con 0x5592cb71f680 2024-09-19T06:25:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.519+0000 7f32bdb3a740 1 -- 172.21.15.5:0/402176019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32ac00ee90 con 0x5592cb71f680 2024-09-19T06:25:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.520+0000 7f329cff9640 1 -- 172.21.15.5:0/402176019 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f32ac01f650 con 0x5592cb71f680 2024-09-19T06:25:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.520+0000 7f32bdb3a740 10 client.30265 fetch_fsmap learned FSMap version 61 2024-09-19T06:25:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.520+0000 7f32bdb3a740 1 -- 172.21.15.5:0/402176019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5592cb72dc20 con 0x5592cb71f680 2024-09-19T06:25:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.520+0000 7f329cff9640 1 -- 172.21.15.5:0/402176019 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32ac00ee90 con 0x5592cb71f680 2024-09-19T06:25:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.520+0000 7f329cff9640 1 -- 172.21.15.5:0/402176019 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 61) ==== 8366+0+0 (secure 0 0 0) 0x7f32ac026070 con 0x5592cb71f680 2024-09-19T06:25:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.520+0000 7f32bdb3a740 10 client.30265 fetch_fsmap finished waiting for FSMap version 61 2024-09-19T06:25:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.520+0000 7f32bdb3a740 10 client.30265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.520+0000 7f32bdb3a740 20 client.30265 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.520+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/402176019 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5592cb72f030 0x5592cb731420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.520+0000 7f32bdb3a740 4 client.30265 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.520+0000 7f32bdb3a740 1 -- 172.21.15.5:0/402176019 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592cb687850 con 0x5592cb72f030 2024-09-19T06:25:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.521+0000 7f32b94fd640 1 --2- 172.21.15.5:0/402176019 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5592cb72f030 0x5592cb731420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.521+0000 7f32b94fd640 1 --2- 172.21.15.5:0/402176019 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5592cb72f030 0x5592cb731420 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25a59b7da070d4b1 server_cookie=91f9dad20d3517fb in_seq=0 out_seq=0 2024-09-19T06:25:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.521+0000 7f329cff9640 10 client.30265 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f329cff9640 1 -- 172.21.15.5:0/402176019 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592cb687850 con 0x5592cb72f030 2024-09-19T06:25:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f329cff9640 10 client.30265 handle_command_reply: tid=0 2024-09-19T06:25:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f32bdb3a740 1 client.30265 shutdown 2024-09-19T06:25:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f32bdb3a740 1 -- 172.21.15.5:0/402176019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb71f680 msgr2=0x5592cb71c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/402176019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb71f680 0x5592cb71c800 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f32ac01b000 tx=0x7f32ac005bd0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f32bdb3a740 1 -- 172.21.15.5:0/402176019 shutdown_connections 2024-09-19T06:25:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/402176019 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5592cb72f030 0x5592cb731420 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/402176019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb71f680 0x5592cb71c800 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/402176019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb71c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/402176019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb7144a0 0x5592cb71eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f32bdb3a740 1 -- 172.21.15.5:0/402176019 >> 172.21.15.5:0/402176019 conn(0x5592cb710a30 msgr2=0x5592cb716710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f329cff9640 0 client.30265 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.522+0000 7f32bdb3a740 1 -- 172.21.15.5:0/402176019 shutdown_connections 2024-09-19T06:25:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.523+0000 7f32bdb3a740 1 -- 172.21.15.5:0/402176019 wait complete. 2024-09-19T06:25:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.523+0000 7f32bdb3a740 20 client.30265 trim_cache size 0 max 16384 2024-09-19T06:25:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.553+0000 7f32bdb3a740 1 Processor -- start 2024-09-19T06:25:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.553+0000 7f32bdb3a740 1 -- start start 2024-09-19T06:25:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.553+0000 7f32bdb3a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb72b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.553+0000 7f32bdb3a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb736c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.553+0000 7f32bdb3a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb737160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.553+0000 7f32bdb3a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32b41ab7a0 con 0x5592cb71f680 2024-09-19T06:25:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.553+0000 7f32bdb3a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32b41ad310 con 0x5592cb714dc0 2024-09-19T06:25:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.553+0000 7f32bdb3a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f32b41abe70 con 0x5592cb7144a0 2024-09-19T06:25:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.554+0000 7f32b8cfc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb736c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.554+0000 7f32b94fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb72b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.554+0000 7f32b94fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb72b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55906/0 (socket says 172.21.15.5:55906) 2024-09-19T06:25:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.554+0000 7f32b94fd640 1 -- 172.21.15.5:0/1139282189 learned_addr learned my addr 172.21.15.5:0/1139282189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.554+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/1139282189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb737160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.554+0000 7f32b94fd640 1 -- 172.21.15.5:0/1139282189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 msgr2=0x5592cb736c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.554+0000 7f32b94fd640 1 --2- 172.21.15.5:0/1139282189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb736c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.554+0000 7f32b94fd640 1 -- 172.21.15.5:0/1139282189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 msgr2=0x5592cb737160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.554+0000 7f32b94fd640 1 --2- 172.21.15.5:0/1139282189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb737160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.554+0000 7f32b94fd640 1 -- 172.21.15.5:0/1139282189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592cb71a850 con 0x5592cb7144a0 2024-09-19T06:25:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.554+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/1139282189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb737160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:25:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.555+0000 7f32b94fd640 1 --2- 172.21.15.5:0/1139282189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb72b4b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f32b0004650 tx=0x7f32b002a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30071ec7870bf42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.555+0000 7f329effd640 1 -- 172.21.15.5:0/1139282189 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b005a040 con 0x5592cb7144a0 2024-09-19T06:25:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.555+0000 7f329effd640 1 -- 172.21.15.5:0/1139282189 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32b0031d00 con 0x5592cb7144a0 2024-09-19T06:25:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.555+0000 7f329effd640 1 -- 172.21.15.5:0/1139282189 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b004c470 con 0x5592cb7144a0 2024-09-19T06:25:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.556+0000 7f32bdb3a740 1 -- 172.21.15.5:0/1139282189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 msgr2=0x5592cb72b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.556+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/1139282189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb72b4b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f32b0004650 tx=0x7f32b002a440 comp rx=0 tx=0).stop 2024-09-19T06:25:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.556+0000 7f32bdb3a740 1 -- 172.21.15.5:0/1139282189 shutdown_connections 2024-09-19T06:25:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.556+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/1139282189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb737160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.556+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/1139282189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb736c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.556+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/1139282189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb72b4b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.556+0000 7f32bdb3a740 1 -- 172.21.15.5:0/1139282189 >> 172.21.15.5:0/1139282189 conn(0x5592cb710a30 msgr2=0x5592cb717900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:35.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.557+0000 7f32bdb3a740 1 -- 172.21.15.5:0/1139282189 shutdown_connections 2024-09-19T06:25:35.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.557+0000 7f32bdb3a740 1 -- 172.21.15.5:0/1139282189 wait complete. 2024-09-19T06:25:35.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.557+0000 7f32bdb3a740 1 -- start start 2024-09-19T06:25:35.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.557+0000 7f32bdb3a740 1 Processor -- start 2024-09-19T06:25:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.558+0000 7f32bdb3a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb727ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.558+0000 7f32bdb3a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb723010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.558+0000 7f32bdb3a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb723550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.558+0000 7f32bdb3a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32b41acc40 con 0x5592cb71f680 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.558+0000 7f32bdb3a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32b41ac570 con 0x5592cb714dc0 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.558+0000 7f32bdb3a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f32b41a97a0 con 0x5592cb7144a0 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f32b8cfc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb723010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f32bdb3a740 10 client.0 _ll_register_callbacks cb 0x7f32a4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f32b94fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb727ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f32b8cfc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb723010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:45820/0 (socket says 172.21.15.5:45820) 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f32b8cfc640 1 -- 172.21.15.5:0/4105845348 learned_addr learned my addr 172.21.15.5:0/4105845348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb723550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f32b8cfc640 1 -- 172.21.15.5:0/4105845348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 msgr2=0x5592cb727ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f32b8cfc640 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb727ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f32b8cfc640 1 -- 172.21.15.5:0/4105845348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 msgr2=0x5592cb723550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f32b8cfc640 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb723550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f32b8cfc640 1 -- 172.21.15.5:0/4105845348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592cb723d60 con 0x5592cb714dc0 2024-09-19T06:25:35.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f32b8cfc640 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb723010 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x5592cb711db0 tx=0x7f32a40039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=632848b046d8c0d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:35.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.559+0000 7f329cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:35.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.560+0000 7f32b9cfe640 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb723550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:25:35.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.560+0000 7f329cff9640 1 -- 172.21.15.5:0/4105845348 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4005220 con 0x5592cb714dc0 2024-09-19T06:25:35.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.560+0000 7f329cff9640 1 -- 172.21.15.5:0/4105845348 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f32a40053c0 con 0x5592cb714dc0 2024-09-19T06:25:35.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.560+0000 7f32bdb3a740 1 -- 172.21.15.5:0/4105845348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b0031d00 con 0x5592cb714dc0 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.560+0000 7f329cff9640 1 -- 172.21.15.5:0/4105845348 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f32a40055c0 con 0x5592cb714dc0 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.560+0000 7f32bdb3a740 10 client.30347 fetch_fsmap learned FSMap version 61 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.560+0000 7f32bdb3a740 1 -- 172.21.15.5:0/4105845348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5592cb799c00 con 0x5592cb714dc0 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.560+0000 7f329cff9640 1 -- 172.21.15.5:0/4105845348 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32a4004c50 con 0x5592cb714dc0 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.561+0000 7f329cff9640 1 -- 172.21.15.5:0/4105845348 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 61) ==== 8366+0+0 (secure 0 0 0) 0x7f32a4020030 con 0x5592cb714dc0 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.561+0000 7f32bdb3a740 10 client.30347 fetch_fsmap finished waiting for FSMap version 61 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.561+0000 7f32bdb3a740 10 client.30347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.561+0000 7f32bdb3a740 20 client.30347 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.561+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5592cb79a3b0 0x5592cb79c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.561+0000 7f32bdb3a740 4 client.30347 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.561+0000 7f32bdb3a740 1 -- 172.21.15.5:0/4105845348 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f32b0031d00 con 0x5592cb79a3b0 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.561+0000 7f32b94fd640 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5592cb79a3b0 0x5592cb79c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.562+0000 7f32b94fd640 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5592cb79a3b0 0x5592cb79c7d0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52423558a871100e server_cookie=88bd4d6be558fd06 in_seq=0 out_seq=0 2024-09-19T06:25:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.562+0000 7f329cff9640 10 client.30347 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.562+0000 7f329cff9640 1 -- 172.21.15.5:0/4105845348 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f32b0031d00 con 0x5592cb79a3b0 2024-09-19T06:25:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.562+0000 7f329cff9640 10 client.30347 handle_command_reply: tid=0 2024-09-19T06:25:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.562+0000 7f32bdb3a740 1 client.30347 shutdown 2024-09-19T06:25:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.562+0000 7f32bdb3a740 1 -- 172.21.15.5:0/4105845348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 msgr2=0x5592cb723010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.562+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb723010 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x5592cb711db0 tx=0x7f32a40039c0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.563+0000 7f32bdb3a740 1 -- 172.21.15.5:0/4105845348 shutdown_connections 2024-09-19T06:25:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.563+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5592cb79a3b0 0x5592cb79c7d0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.563+0000 7f329cff9640 0 client.30347 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.563+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592cb71f680 0x5592cb723550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.563+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592cb714dc0 0x5592cb723010 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.563+0000 7f32bdb3a740 1 --2- 172.21.15.5:0/4105845348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592cb7144a0 0x5592cb727ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.563+0000 7f32bdb3a740 1 -- 172.21.15.5:0/4105845348 >> 172.21.15.5:0/4105845348 conn(0x5592cb710a30 msgr2=0x5592cb70f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:35.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.563+0000 7f32bdb3a740 1 -- 172.21.15.5:0/4105845348 shutdown_connections 2024-09-19T06:25:35.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.563+0000 7f32bdb3a740 1 -- 172.21.15.5:0/4105845348 wait complete. 2024-09-19T06:25:35.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.563+0000 7f32bdb3a740 20 client.30347 trim_cache size 0 max 16384 2024-09-19T06:25:35.568 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:25:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:25:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:25:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "6d3753c4-801b-477e-893a-7482f6d07be1": { 2024-09-19T06:25:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:25:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "6d3753c4-801b-477e-893a-7482f6d07be1", 2024-09-19T06:25:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:25:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:25:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:25:35.569 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:25:35.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.564+0000 7f32bb761640 1 -- 172.21.15.5:0/2830320829 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f32a00a3340 msgr2=0x7f32a00a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:35.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.564+0000 7f32bb761640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f32a00a3340 0x7f32a00a5800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009270 tx=0x7f32ac000f40 comp rx=0 tx=0).stop 2024-09-19T06:25:35.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.564+0000 7f32bb761640 1 -- 172.21.15.5:0/2830320829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b4100a90 msgr2=0x7f32b4196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:35.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.564+0000 7f32bb761640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b4100a90 0x7f32b4196190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f32b0005ab0 tx=0x7f32b00045c0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.564+0000 7f32bb761640 1 -- 172.21.15.5:0/2830320829 shutdown_connections 2024-09-19T06:25:35.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.564+0000 7f32bb761640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4106a90 0x7f32b419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.564+0000 7f32bb761640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32b4101430 0x7f32b41966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.564+0000 7f32bb761640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f32a00a3340 0x7f32a00a5800 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.564+0000 7f32bb761640 1 --2- 172.21.15.5:0/2830320829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b4100a90 0x7f32b4196190 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:35.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.564+0000 7f32bb761640 1 -- 172.21.15.5:0/2830320829 >> 172.21.15.5:0/2830320829 conn(0x7f32b40fc7d0 msgr2=0x7f32b40fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:35.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.564+0000 7f32bb761640 1 -- 172.21.15.5:0/2830320829 shutdown_connections 2024-09-19T06:25:35.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:25:35.564+0000 7f32bb761640 1 -- 172.21.15.5:0/2830320829 wait complete. 2024-09-19T06:25:35.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87122 8.58 MB/sec execute 419 sec latency 407.031 ms 2024-09-19T06:25:36.335 INFO:tasks.cephfs.filesystem:scrub status for tag:6d3753c4-801b-477e-893a-7482f6d07be1 - {'path': '/', 'tag': '6d3753c4-801b-477e-893a-7482f6d07be1', 'options': 'recursive,force'} 2024-09-19T06:25:36.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87266 8.59 MB/sec execute 420 sec latency 77.617 ms 2024-09-19T06:25:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:36 smithi005 ceph-mon[30955]: pgmap v703: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2024-09-19T06:25:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:36 smithi049 ceph-mon[34640]: pgmap v703: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2024-09-19T06:25:37.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:36 smithi157 ceph-mon[35346]: pgmap v703: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2024-09-19T06:25:37.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87393 8.58 MB/sec execute 421 sec latency 145.430 ms 2024-09-19T06:25:38.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87569 8.59 MB/sec execute 422 sec latency 59.206 ms 2024-09-19T06:25:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:38 smithi005 ceph-mon[30955]: pgmap v704: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.0 MiB/s wr, 423 op/s 2024-09-19T06:25:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:38 smithi049 ceph-mon[34640]: pgmap v704: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.0 MiB/s wr, 423 op/s 2024-09-19T06:25:39.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:38 smithi157 ceph-mon[35346]: pgmap v704: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.0 MiB/s wr, 423 op/s 2024-09-19T06:25:39.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87746 8.58 MB/sec execute 423 sec latency 64.511 ms 2024-09-19T06:25:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:25:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:25:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:25:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:25:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:25:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:25:40.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:25:40.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:25:40.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:25:40.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87899 8.57 MB/sec execute 424 sec latency 93.747 ms 2024-09-19T06:25:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:40 smithi005 ceph-mon[30955]: pgmap v705: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.5 MiB/s wr, 304 op/s 2024-09-19T06:25:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:40 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:25:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:40 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:25:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:40 smithi049 ceph-mon[34640]: pgmap v705: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.5 MiB/s wr, 304 op/s 2024-09-19T06:25:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:40 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:25:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:40 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:25:41.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:40 smithi157 ceph-mon[35346]: pgmap v705: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.5 MiB/s wr, 304 op/s 2024-09-19T06:25:41.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:40 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:25:41.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:40 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:25:41.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 88022 8.56 MB/sec execute 425 sec latency 97.702 ms 2024-09-19T06:25:42.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 88212 8.58 MB/sec execute 426 sec latency 61.340 ms 2024-09-19T06:25:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:42 smithi005 ceph-mon[30955]: pgmap v706: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.5 MiB/s wr, 304 op/s 2024-09-19T06:25:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:42 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:25:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:42 smithi049 ceph-mon[34640]: pgmap v706: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.5 MiB/s wr, 304 op/s 2024-09-19T06:25:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:42 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:25:43.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:42 smithi157 ceph-mon[35346]: pgmap v706: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.5 MiB/s wr, 304 op/s 2024-09-19T06:25:43.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:42 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:25:43.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 88388 8.59 MB/sec execute 427 sec latency 62.030 ms 2024-09-19T06:25:44.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 88521 8.58 MB/sec execute 428 sec latency 69.506 ms 2024-09-19T06:25:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:44 smithi005 ceph-mon[30955]: pgmap v707: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.0 MiB/s wr, 426 op/s 2024-09-19T06:25:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:44 smithi049 ceph-mon[34640]: pgmap v707: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.0 MiB/s wr, 426 op/s 2024-09-19T06:25:45.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:44 smithi157 ceph-mon[35346]: pgmap v707: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.0 MiB/s wr, 426 op/s 2024-09-19T06:25:45.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 88698 8.59 MB/sec execute 429 sec latency 81.154 ms 2024-09-19T06:25:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.275+0000 7f732539d640 1 -- 172.21.15.5:0/2738159274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7320101910 msgr2=0x7f7320101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.275+0000 7f732539d640 1 --2- 172.21.15.5:0/2738159274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7320101910 0x7f7320101d70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7314009930 tx=0x7f731402a420 comp rx=0 tx=0).stop 2024-09-19T06:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.275+0000 7f732539d640 1 -- 172.21.15.5:0/2738159274 shutdown_connections 2024-09-19T06:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.275+0000 7f732539d640 1 --2- 172.21.15.5:0/2738159274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73201022b0 0x7f732010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.275+0000 7f732539d640 1 --2- 172.21.15.5:0/2738159274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7320101910 0x7f7320101d70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.275+0000 7f732539d640 1 --2- 172.21.15.5:0/2738159274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7320107920 0x7f7320107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.275+0000 7f732539d640 1 -- 172.21.15.5:0/2738159274 >> 172.21.15.5:0/2738159274 conn(0x7f73200fd650 msgr2=0x7f73200ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.275+0000 7f732539d640 1 -- 172.21.15.5:0/2738159274 shutdown_connections 2024-09-19T06:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.275+0000 7f732539d640 1 -- 172.21.15.5:0/2738159274 wait complete. 2024-09-19T06:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.275+0000 7f732539d640 1 Processor -- start 2024-09-19T06:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.276+0000 7f732539d640 1 -- start start 2024-09-19T06:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.276+0000 7f732539d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7320101910 0x7f73201981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.276+0000 7f732539d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73201022b0 0x7f7320198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.276+0000 7f732539d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7320107920 0x7f732019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.276+0000 7f731ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7320101910 0x7f73201981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.276+0000 7f731ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7320101910 0x7f73201981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53658/0 (socket says 172.21.15.5:53658) 2024-09-19T06:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.276+0000 7f731f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73201022b0 0x7f7320198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.276+0000 7f731f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73201022b0 0x7f7320198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35072/0 (socket says 172.21.15.5:35072) 2024-09-19T06:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.276+0000 7f732539d640 1 -- 172.21.15.5:0/301570544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7320111c10 con 0x7f7320107920 2024-09-19T06:25:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.276+0000 7f731ffff640 1 -- 172.21.15.5:0/301570544 learned_addr learned my addr 172.21.15.5:0/301570544 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.276+0000 7f732539d640 1 -- 172.21.15.5:0/301570544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7320111a90 con 0x7f73201022b0 2024-09-19T06:25:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.276+0000 7f732539d640 1 -- 172.21.15.5:0/301570544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7320111d90 con 0x7f7320101910 2024-09-19T06:25:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.277+0000 7f7324b9c640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7320107920 0x7f732019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.277+0000 7f731ffff640 1 -- 172.21.15.5:0/301570544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73201022b0 msgr2=0x7f7320198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.277+0000 7f731ffff640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73201022b0 0x7f7320198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.277+0000 7f731ffff640 1 -- 172.21.15.5:0/301570544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7320107920 msgr2=0x7f732019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.277+0000 7f731ffff640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7320107920 0x7f732019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.277+0000 7f731ffff640 1 -- 172.21.15.5:0/301570544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732019d1b0 con 0x7f7320101910 2024-09-19T06:25:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.277+0000 7f731ffff640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7320101910 0x7f73201981c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f731000e9f0 tx=0x7f731000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.277+0000 7f731d7fa640 1 -- 172.21.15.5:0/301570544 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731000cd80 con 0x7f7320101910 2024-09-19T06:25:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.277+0000 7f731d7fa640 1 -- 172.21.15.5:0/301570544 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73100108e0 con 0x7f7320101910 2024-09-19T06:25:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.277+0000 7f731d7fa640 1 -- 172.21.15.5:0/301570544 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310018920 con 0x7f7320101910 2024-09-19T06:25:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.277+0000 7f732539d640 1 -- 172.21.15.5:0/301570544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732019d4a0 con 0x7f7320101910 2024-09-19T06:25:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.278+0000 7f732539d640 1 -- 172.21.15.5:0/301570544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73201a4d30 con 0x7f7320101910 2024-09-19T06:25:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.281+0000 7f731d7fa640 1 -- 172.21.15.5:0/301570544 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7310010a90 con 0x7f7320101910 2024-09-19T06:25:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.281+0000 7f732946e740 1 Processor -- start 2024-09-19T06:25:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.281+0000 7f732946e740 1 -- start start 2024-09-19T06:25:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f732946e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c83f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f732946e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7c92d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f732946e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7c95680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f732946e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bdb7be82f0 con 0x55bdb7c83b60 2024-09-19T06:25:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f732946e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bdb7b90270 con 0x55bdb7c93290 2024-09-19T06:25:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f732946e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bdb7be42b0 con 0x55bdb7c84480 2024-09-19T06:25:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f7324b9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c83f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f731f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7c95680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f731f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7c95680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35082/0 (socket says 172.21.15.5:35082) 2024-09-19T06:25:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f7324b9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c83f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48470/0 (socket says 172.21.15.5:48470) 2024-09-19T06:25:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f731f7fe640 1 -- 172.21.15.5:0/1361818277 learned_addr learned my addr 172.21.15.5:0/1361818277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f731ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7c92d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f731f7fe640 1 -- 172.21.15.5:0/1361818277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 msgr2=0x55bdb7c92d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f731f7fe640 1 --2- 172.21.15.5:0/1361818277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7c92d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f731f7fe640 1 -- 172.21.15.5:0/1361818277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 msgr2=0x55bdb7c83f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f731f7fe640 1 --2- 172.21.15.5:0/1361818277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c83f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f731f7fe640 1 -- 172.21.15.5:0/1361818277 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb7b8f340 con 0x55bdb7c93290 2024-09-19T06:25:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.282+0000 7f7324b9c640 1 --2- 172.21.15.5:0/1361818277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c83f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.283+0000 7f731f7fe640 1 --2- 172.21.15.5:0/1361818277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7c95680 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f731402a940 tx=0x7f7314005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e597443ac73772 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.283+0000 7f731d7fa640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73080a3420 0x7f73080a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.283+0000 7f7324b9c640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73080a3420 0x7f73080a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.283+0000 7f73067fc640 1 -- 172.21.15.5:0/1361818277 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314040070 con 0x55bdb7c93290 2024-09-19T06:25:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.283+0000 7f731d7fa640 1 -- 172.21.15.5:0/301570544 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f73100cffd0 con 0x7f7320101910 2024-09-19T06:25:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.283+0000 7f73067fc640 1 -- 172.21.15.5:0/1361818277 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7314004030 con 0x55bdb7c93290 2024-09-19T06:25:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.284+0000 7f73067fc640 1 -- 172.21.15.5:0/1361818277 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314035040 con 0x55bdb7c93290 2024-09-19T06:25:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.284+0000 7f732946e740 1 -- 172.21.15.5:0/1361818277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 msgr2=0x55bdb7c95680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.284+0000 7f732946e740 1 --2- 172.21.15.5:0/1361818277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7c95680 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f731402a940 tx=0x7f7314005be0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.284+0000 7f7324b9c640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73080a3420 0x7f73080a58e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7318005fd0 tx=0x7f7318009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.284+0000 7f732946e740 1 -- 172.21.15.5:0/1361818277 shutdown_connections 2024-09-19T06:25:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.284+0000 7f732946e740 1 --2- 172.21.15.5:0/1361818277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7c95680 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.284+0000 7f732946e740 1 --2- 172.21.15.5:0/1361818277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7c92d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.284+0000 7f732946e740 1 --2- 172.21.15.5:0/1361818277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c83f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.284+0000 7f732946e740 1 -- 172.21.15.5:0/1361818277 >> 172.21.15.5:0/1361818277 conn(0x55bdb7c80230 msgr2=0x55bdb7c80610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.285+0000 7f732946e740 1 -- 172.21.15.5:0/1361818277 shutdown_connections 2024-09-19T06:25:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.285+0000 7f732946e740 1 -- 172.21.15.5:0/1361818277 wait complete. 2024-09-19T06:25:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.285+0000 7f732946e740 1 -- start start 2024-09-19T06:25:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.285+0000 7f732946e740 1 Processor -- start 2024-09-19T06:25:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f732946e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c90170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f732946e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7c906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f732946e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7c90bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f732946e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bdb7c95eb0 con 0x55bdb7c83b60 2024-09-19T06:25:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f732946e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bdb7b71290 con 0x55bdb7c93290 2024-09-19T06:25:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f732946e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bdb7c96030 con 0x55bdb7c84480 2024-09-19T06:25:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f732946e740 10 client.0 _ll_register_callbacks cb 0x55bdb7ba2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f7324b9c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7c90bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f731ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c90170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f731ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c90170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48478/0 (socket says 172.21.15.5:48478) 2024-09-19T06:25:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f7324b9c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7c90bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35088/0 (socket says 172.21.15.5:35088) 2024-09-19T06:25:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f731ffff640 1 -- 172.21.15.5:0/2974684684 learned_addr learned my addr 172.21.15.5:0/2974684684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f731f7fe640 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7c906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f7324b9c640 1 -- 172.21.15.5:0/2974684684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 msgr2=0x55bdb7c906b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f7324b9c640 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7c906b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f7324b9c640 1 -- 172.21.15.5:0/2974684684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 msgr2=0x55bdb7c90170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f7324b9c640 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c90170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f7324b9c640 1 -- 172.21.15.5:0/2974684684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb78fb210 con 0x55bdb7c93290 2024-09-19T06:25:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.286+0000 7f731ffff640 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c90170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.287+0000 7f7324b9c640 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7c90bf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7318011fd0 tx=0x7f7318004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c991df24e95dcef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.287+0000 7f72ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.287+0000 7f72ff7fe640 1 -- 172.21.15.5:0/2974684684 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731801fbc0 con 0x55bdb7c93290 2024-09-19T06:25:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.287+0000 7f732946e740 1 -- 172.21.15.5:0/2974684684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7314004030 con 0x55bdb7c93290 2024-09-19T06:25:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.287+0000 7f72ff7fe640 1 -- 172.21.15.5:0/2974684684 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f7318017030 con 0x55bdb7c93290 2024-09-19T06:25:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.287+0000 7f72ff7fe640 1 -- 172.21.15.5:0/2974684684 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f731801fd60 con 0x55bdb7c93290 2024-09-19T06:25:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.287+0000 7f732946e740 10 client.30359 fetch_fsmap learned FSMap version 62 2024-09-19T06:25:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.287+0000 7f732946e740 1 -- 172.21.15.5:0/2974684684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55bdb79c7620 con 0x55bdb7c93290 2024-09-19T06:25:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.287+0000 7f72ff7fe640 1 -- 172.21.15.5:0/2974684684 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7314004030 con 0x55bdb7c93290 2024-09-19T06:25:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.288+0000 7f72ff7fe640 1 -- 172.21.15.5:0/2974684684 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f7318004610 con 0x55bdb7c93290 2024-09-19T06:25:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.288+0000 7f732946e740 10 client.30359 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:25:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.288+0000 7f732946e740 10 client.30359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.288+0000 7f732946e740 20 client.30359 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.288+0000 7f732946e740 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdb7ca3390 0x55bdb7ca57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.288+0000 7f732946e740 4 client.30359 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.288+0000 7f732946e740 1 -- 172.21.15.5:0/2974684684 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdb7bf5a20 con 0x55bdb7ca3390 2024-09-19T06:25:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.288+0000 7f731ffff640 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdb7ca3390 0x55bdb7ca57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.288+0000 7f731ffff640 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdb7ca3390 0x55bdb7ca57a0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8471000e1be91a0 server_cookie=aa843edd766840a0 in_seq=0 out_seq=0 2024-09-19T06:25:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.288+0000 7f72ff7fe640 10 client.30359 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f72ff7fe640 1 -- 172.21.15.5:0/2974684684 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdb7bf5a20 con 0x55bdb7ca3390 2024-09-19T06:25:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f72ff7fe640 10 client.30359 handle_command_reply: tid=0 2024-09-19T06:25:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f732946e740 1 client.30359 shutdown 2024-09-19T06:25:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f732946e740 1 -- 172.21.15.5:0/2974684684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 msgr2=0x55bdb7c90bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f732946e740 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7c90bf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7318011fd0 tx=0x7f7318004ab0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f732946e740 1 -- 172.21.15.5:0/2974684684 shutdown_connections 2024-09-19T06:25:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f732946e740 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdb7ca3390 0x55bdb7ca57a0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f732946e740 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7c90bf0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f732946e740 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7c906b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f72ff7fe640 0 client.30359 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f732946e740 1 --2- 172.21.15.5:0/2974684684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c90170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f732946e740 1 -- 172.21.15.5:0/2974684684 >> 172.21.15.5:0/2974684684 conn(0x55bdb7c80230 msgr2=0x55bdb7c85f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.289+0000 7f732946e740 1 -- 172.21.15.5:0/2974684684 shutdown_connections 2024-09-19T06:25:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.290+0000 7f732946e740 1 -- 172.21.15.5:0/2974684684 wait complete. 2024-09-19T06:25:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.290+0000 7f732946e740 20 client.30359 trim_cache size 0 max 16384 2024-09-19T06:25:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.315+0000 7f732946e740 1 Processor -- start 2024-09-19T06:25:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.315+0000 7f732946e740 1 -- start start 2024-09-19T06:25:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.316+0000 7f732946e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c8ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.316+0000 7f731ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c8ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.316+0000 7f731ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c8ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48486/0 (socket says 172.21.15.5:48486) 2024-09-19T06:25:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.316+0000 7f731ffff640 1 -- 172.21.15.5:0/1703412692 learned_addr learned my addr 172.21.15.5:0/1703412692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.316+0000 7f732946e740 1 --2- 172.21.15.5:0/1703412692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7cab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.316+0000 7f732946e740 1 --2- 172.21.15.5:0/1703412692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7cab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.316+0000 7f732946e740 1 -- 172.21.15.5:0/1703412692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73201ab4f0 con 0x55bdb7c83b60 2024-09-19T06:25:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.316+0000 7f732946e740 1 -- 172.21.15.5:0/1703412692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73201ad030 con 0x55bdb7c93290 2024-09-19T06:25:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.316+0000 7f732946e740 1 -- 172.21.15.5:0/1703412692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f73201abbc0 con 0x55bdb7c84480 2024-09-19T06:25:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.316+0000 7f731f7fe640 1 --2- 172.21.15.5:0/1703412692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7cab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.316+0000 7f7324b9c640 1 --2- 172.21.15.5:0/1703412692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7cab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.317+0000 7f731ffff640 1 -- 172.21.15.5:0/1703412692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 msgr2=0x55bdb7cab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.317+0000 7f731ffff640 1 --2- 172.21.15.5:0/1703412692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7cab0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.317+0000 7f731ffff640 1 -- 172.21.15.5:0/1703412692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 msgr2=0x55bdb7cab5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.317+0000 7f731ffff640 1 --2- 172.21.15.5:0/1703412692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7cab5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.317+0000 7f731ffff640 1 -- 172.21.15.5:0/1703412692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb7c8f100 con 0x55bdb7c83b60 2024-09-19T06:25:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.317+0000 7f731f7fe640 1 --2- 172.21.15.5:0/1703412692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7cab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.317+0000 7f731ffff640 1 --2- 172.21.15.5:0/1703412692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c8ea20 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f73100cdd00 tx=0x7f73100d24e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8242dab9dd427747 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.317+0000 7f7305ffb640 1 -- 172.21.15.5:0/1703412692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731009bbf0 con 0x55bdb7c83b60 2024-09-19T06:25:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.317+0000 7f7305ffb640 1 -- 172.21.15.5:0/1703412692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73100cde50 con 0x55bdb7c83b60 2024-09-19T06:25:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.317+0000 7f7305ffb640 1 -- 172.21.15.5:0/1703412692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73100a4f00 con 0x55bdb7c83b60 2024-09-19T06:25:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.318+0000 7f732946e740 1 -- 172.21.15.5:0/1703412692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 msgr2=0x55bdb7c8ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.318+0000 7f732946e740 1 --2- 172.21.15.5:0/1703412692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c8ea20 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f73100cdd00 tx=0x7f73100d24e0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.318+0000 7f732946e740 1 -- 172.21.15.5:0/1703412692 shutdown_connections 2024-09-19T06:25:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.318+0000 7f732946e740 1 --2- 172.21.15.5:0/1703412692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c93290 0x55bdb7cab5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.318+0000 7f732946e740 1 --2- 172.21.15.5:0/1703412692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c84480 0x55bdb7cab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.318+0000 7f732946e740 1 --2- 172.21.15.5:0/1703412692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c83b60 0x55bdb7c8ea20 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.318+0000 7f732946e740 1 -- 172.21.15.5:0/1703412692 >> 172.21.15.5:0/1703412692 conn(0x55bdb7c80230 msgr2=0x55bdb7c7f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.318+0000 7f732946e740 1 -- 172.21.15.5:0/1703412692 shutdown_connections 2024-09-19T06:25:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.318+0000 7f732946e740 1 -- 172.21.15.5:0/1703412692 wait complete. 2024-09-19T06:25:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.319+0000 7f732946e740 1 -- start start 2024-09-19T06:25:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.319+0000 7f732946e740 1 Processor -- start 2024-09-19T06:25:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.319+0000 7f732946e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c83b60 0x55bdb7c8a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.319+0000 7f732946e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c84480 0x55bdb7c9b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.319+0000 7f732946e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c93290 0x55bdb7c9b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.319+0000 7f732946e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73201ac960 con 0x55bdb7c84480 2024-09-19T06:25:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.319+0000 7f732946e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73201ac290 con 0x55bdb7c83b60 2024-09-19T06:25:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.319+0000 7f732946e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f73201a94f0 con 0x55bdb7c93290 2024-09-19T06:25:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.319+0000 7f732946e740 10 client.0 _ll_register_callbacks cb 0x7f7318011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f731ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c83b60 0x55bdb7c8a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f731ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c83b60 0x55bdb7c8a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35104/0 (socket says 172.21.15.5:35104) 2024-09-19T06:25:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f731ffff640 1 -- 172.21.15.5:0/2199306323 learned_addr learned my addr 172.21.15.5:0/2199306323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f731f7fe640 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c84480 0x55bdb7c9b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f7324b9c640 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c93290 0x55bdb7c9b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f731ffff640 1 -- 172.21.15.5:0/2199306323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c93290 msgr2=0x55bdb7c9b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f731ffff640 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c93290 0x55bdb7c9b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f731ffff640 1 -- 172.21.15.5:0/2199306323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c84480 msgr2=0x55bdb7c9b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f731ffff640 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c84480 0x55bdb7c9b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f731ffff640 1 -- 172.21.15.5:0/2199306323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb7c842b0 con 0x55bdb7c83b60 2024-09-19T06:25:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f731f7fe640 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c84480 0x55bdb7c9b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f731ffff640 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c83b60 0x55bdb7c8a5b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f731009c220 tx=0x7f73100d3aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b736f488150a244 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.320+0000 7f72ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f72ff7fe640 1 -- 172.21.15.5:0/2199306323 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310010de0 con 0x55bdb7c83b60 2024-09-19T06:25:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f72ff7fe640 1 -- 172.21.15.5:0/2199306323 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7f7310098900 con 0x55bdb7c83b60 2024-09-19T06:25:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f732946e740 1 -- 172.21.15.5:0/2199306323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73100cde50 con 0x55bdb7c83b60 2024-09-19T06:25:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f72ff7fe640 1 -- 172.21.15.5:0/2199306323 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f731009c290 con 0x55bdb7c83b60 2024-09-19T06:25:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f732946e740 10 client.30365 fetch_fsmap learned FSMap version 62 2024-09-19T06:25:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f732946e740 1 -- 172.21.15.5:0/2199306323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55bdb7d12870 con 0x55bdb7c83b60 2024-09-19T06:25:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f72ff7fe640 1 -- 172.21.15.5:0/2199306323 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73100cde50 con 0x55bdb7c83b60 2024-09-19T06:25:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f72ff7fe640 1 -- 172.21.15.5:0/2199306323 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f73100bc0e0 con 0x55bdb7c83b60 2024-09-19T06:25:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f732946e740 10 client.30365 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:25:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f732946e740 10 client.30365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f732946e740 20 client.30365 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f732946e740 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdb7d12ed0 0x55bdb7d152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f732946e740 4 client.30365 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:25:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.321+0000 7f732946e740 1 -- 172.21.15.5:0/2199306323 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bdb7befd70 con 0x55bdb7d12ed0 2024-09-19T06:25:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.322+0000 7f731f7fe640 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdb7d12ed0 0x55bdb7d152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.322+0000 7f72ff7fe640 10 client.30365 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.322+0000 7f731f7fe640 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdb7d12ed0 0x55bdb7d152c0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62480150c17e3fd3 server_cookie=2a1d1d809283f724 in_seq=0 out_seq=0 2024-09-19T06:25:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.427+0000 7f72ff7fe640 1 -- 172.21.15.5:0/2199306323 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bdb7befd70 con 0x55bdb7d12ed0 2024-09-19T06:25:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.427+0000 7f72ff7fe640 10 client.30365 handle_command_reply: tid=0 2024-09-19T06:25:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.427+0000 7f732946e740 1 client.30365 shutdown 2024-09-19T06:25:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.428+0000 7f732946e740 1 -- 172.21.15.5:0/2199306323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c83b60 msgr2=0x55bdb7c8a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.428+0000 7f732946e740 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c83b60 0x55bdb7c8a5b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f731009c220 tx=0x7f73100d3aa0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.429+0000 7f732946e740 1 -- 172.21.15.5:0/2199306323 shutdown_connections 2024-09-19T06:25:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.429+0000 7f732946e740 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55bdb7d12ed0 0x55bdb7d152c0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.429+0000 7f732946e740 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdb7c93290 0x55bdb7c9b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.429+0000 7f732946e740 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb7c84480 0x55bdb7c9b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.429+0000 7f732946e740 1 --2- 172.21.15.5:0/2199306323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdb7c83b60 0x55bdb7c8a5b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.429+0000 7f732946e740 1 -- 172.21.15.5:0/2199306323 >> 172.21.15.5:0/2199306323 conn(0x55bdb7c80230 msgr2=0x55bdb7c919d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.429+0000 7f72ff7fe640 0 client.30365 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.429+0000 7f732946e740 1 -- 172.21.15.5:0/2199306323 shutdown_connections 2024-09-19T06:25:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.429+0000 7f732946e740 1 -- 172.21.15.5:0/2199306323 wait complete. 2024-09-19T06:25:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.429+0000 7f732946e740 20 client.30365 trim_cache size 0 max 16384 2024-09-19T06:25:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:25:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:25:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:25:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:25:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.430+0000 7f732539d640 1 -- 172.21.15.5:0/301570544 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73080a3420 msgr2=0x7f73080a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.430+0000 7f732539d640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73080a3420 0x7f73080a58e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7318005fd0 tx=0x7f7318009500 comp rx=0 tx=0).stop 2024-09-19T06:25:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.430+0000 7f732539d640 1 -- 172.21.15.5:0/301570544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7320101910 msgr2=0x7f73201981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.430+0000 7f732539d640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7320101910 0x7f73201981c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f731000e9f0 tx=0x7f731000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.431+0000 7f732539d640 1 -- 172.21.15.5:0/301570544 shutdown_connections 2024-09-19T06:25:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.431+0000 7f732539d640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7320107920 0x7f732019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.431+0000 7f732539d640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73201022b0 0x7f7320198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.431+0000 7f732539d640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f73080a3420 0x7f73080a58e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.431+0000 7f732539d640 1 --2- 172.21.15.5:0/301570544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7320101910 0x7f73201981c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.431+0000 7f732539d640 1 -- 172.21.15.5:0/301570544 >> 172.21.15.5:0/301570544 conn(0x7f73200fd650 msgr2=0x7f73200fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.431+0000 7f732539d640 1 -- 172.21.15.5:0/301570544 shutdown_connections 2024-09-19T06:25:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:46.431+0000 7f732539d640 1 -- 172.21.15.5:0/301570544 wait complete. 2024-09-19T06:25:46.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 88835 8.57 MB/sec execute 430 sec latency 114.432 ms 2024-09-19T06:25:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:46 smithi005 ceph-mon[30955]: pgmap v708: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2024-09-19T06:25:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:46 smithi049 ceph-mon[34640]: pgmap v708: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2024-09-19T06:25:47.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:46 smithi157 ceph-mon[35346]: pgmap v708: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2024-09-19T06:25:47.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 88980 8.56 MB/sec execute 431 sec latency 74.614 ms 2024-09-19T06:25:48.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 89130 8.55 MB/sec execute 432 sec latency 85.627 ms 2024-09-19T06:25:48.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:48 smithi005 ceph-mon[30955]: pgmap v709: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.5 MiB/s wr, 401 op/s 2024-09-19T06:25:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:48 smithi049 ceph-mon[34640]: pgmap v709: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.5 MiB/s wr, 401 op/s 2024-09-19T06:25:49.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:48 smithi157 ceph-mon[35346]: pgmap v709: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.5 MiB/s wr, 401 op/s 2024-09-19T06:25:49.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 89293 8.55 MB/sec execute 433 sec latency 79.302 ms 2024-09-19T06:25:50.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 89474 8.55 MB/sec execute 434 sec latency 68.043 ms 2024-09-19T06:25:50.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:50 smithi005 ceph-mon[30955]: pgmap v710: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 271 op/s 2024-09-19T06:25:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:50 smithi049 ceph-mon[34640]: pgmap v710: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 271 op/s 2024-09-19T06:25:51.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:50 smithi157 ceph-mon[35346]: pgmap v710: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 271 op/s 2024-09-19T06:25:51.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 89717 8.58 MB/sec execute 435 sec latency 77.187 ms 2024-09-19T06:25:52.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 89900 8.58 MB/sec execute 436 sec latency 81.228 ms 2024-09-19T06:25:52.853 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:52 smithi157 ceph-mon[35346]: pgmap v711: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 271 op/s 2024-09-19T06:25:52.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:52 smithi005 ceph-mon[30955]: pgmap v711: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 271 op/s 2024-09-19T06:25:52.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:52 smithi049 ceph-mon[34640]: pgmap v711: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 271 op/s 2024-09-19T06:25:53.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90027 8.58 MB/sec execute 437 sec latency 309.868 ms 2024-09-19T06:25:54.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90210 8.60 MB/sec execute 438 sec latency 60.676 ms 2024-09-19T06:25:54.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:54 smithi005 ceph-mon[30955]: pgmap v712: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.4 MiB/s wr, 394 op/s 2024-09-19T06:25:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:54 smithi049 ceph-mon[34640]: pgmap v712: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.4 MiB/s wr, 394 op/s 2024-09-19T06:25:55.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:54 smithi157 ceph-mon[35346]: pgmap v712: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.4 MiB/s wr, 394 op/s 2024-09-19T06:25:55.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90404 8.60 MB/sec execute 439 sec latency 62.898 ms 2024-09-19T06:25:56.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90553 8.60 MB/sec execute 440 sec latency 60.209 ms 2024-09-19T06:25:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:56 smithi005 ceph-mon[30955]: pgmap v713: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 272 op/s 2024-09-19T06:25:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:56 smithi049 ceph-mon[34640]: pgmap v713: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 272 op/s 2024-09-19T06:25:57.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:56 smithi157 ceph-mon[35346]: pgmap v713: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 272 op/s 2024-09-19T06:25:57.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90675 8.59 MB/sec execute 441 sec latency 107.264 ms 2024-09-19T06:25:57.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:57 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:25:57.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:57 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:25:58.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:57 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:25:58.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90808 8.58 MB/sec execute 442 sec latency 129.749 ms 2024-09-19T06:25:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:58 smithi005 ceph-mon[30955]: pgmap v714: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.7 MiB/s wr, 391 op/s 2024-09-19T06:25:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:58 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:58 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:58 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:58 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:58 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:58 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:58 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:25:58.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:58 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:25:58.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:25:58 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:58 smithi049 ceph-mon[34640]: pgmap v714: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.7 MiB/s wr, 391 op/s 2024-09-19T06:25:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:58 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:58 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:58 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:58 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:58 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:58 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:58 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:25:58.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:58 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:25:58.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:25:58 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:59.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:58 smithi157 ceph-mon[35346]: pgmap v714: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.7 MiB/s wr, 391 op/s 2024-09-19T06:25:59.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:58 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:59.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:58 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:59.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:58 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:58 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:58 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:58 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:58 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:25:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:58 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:25:59.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:25:58 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:25:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.565+0000 7fd9bca00640 1 -- 172.21.15.5:0/365768734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b8100850 msgr2=0x7fd9b8100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.565+0000 7fd9bca00640 1 --2- 172.21.15.5:0/365768734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b8100850 0x7fd9b8100cb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009930 tx=0x7fd9ac02a420 comp rx=0 tx=0).stop 2024-09-19T06:25:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.565+0000 7fd9bca00640 1 -- 172.21.15.5:0/365768734 shutdown_connections 2024-09-19T06:25:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.565+0000 7fd9bca00640 1 --2- 172.21.15.5:0/365768734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b81011f0 0x7fd9b810d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.565+0000 7fd9bca00640 1 --2- 172.21.15.5:0/365768734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b8100850 0x7fd9b8100cb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.565+0000 7fd9bca00640 1 --2- 172.21.15.5:0/365768734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b8106860 0x7fd9b8106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.565+0000 7fd9bca00640 1 -- 172.21.15.5:0/365768734 >> 172.21.15.5:0/365768734 conn(0x7fd9b80fc550 msgr2=0x7fd9b80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.565+0000 7fd9bca00640 1 -- 172.21.15.5:0/365768734 shutdown_connections 2024-09-19T06:25:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.565+0000 7fd9bca00640 1 -- 172.21.15.5:0/365768734 wait complete. 2024-09-19T06:25:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.566+0000 7fd9bca00640 1 Processor -- start 2024-09-19T06:25:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.567+0000 7fd9bca00640 1 -- start start 2024-09-19T06:25:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.567+0000 7fd9bca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b8100850 0x7fd9b8195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.567+0000 7fd9bca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b81011f0 0x7fd9b81964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.567+0000 7fd9bca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b8106860 0x7fd9b819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.567+0000 7fd9bca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b810fa90 con 0x7fd9b8106860 2024-09-19T06:25:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.567+0000 7fd9bca00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b810f910 con 0x7fd9b8100850 2024-09-19T06:25:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.567+0000 7fd9bca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b810fc10 con 0x7fd9b81011f0 2024-09-19T06:25:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.567+0000 7fd9b759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b8106860 0x7fd9b819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.567+0000 7fd9b6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b8100850 0x7fd9b8195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.567+0000 7fd9b759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b8106860 0x7fd9b819a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50114/0 (socket says 172.21.15.5:50114) 2024-09-19T06:25:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.567+0000 7fd9b759e640 1 -- 172.21.15.5:0/745894269 learned_addr learned my addr 172.21.15.5:0/745894269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.568+0000 7fd9b659c640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b81011f0 0x7fd9b81964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.568+0000 7fd9b6d9d640 1 -- 172.21.15.5:0/745894269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b81011f0 msgr2=0x7fd9b81964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.568+0000 7fd9b6d9d640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b81011f0 0x7fd9b81964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.568+0000 7fd9b6d9d640 1 -- 172.21.15.5:0/745894269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b8106860 msgr2=0x7fd9b819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.568+0000 7fd9b6d9d640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b8106860 0x7fd9b819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.568+0000 7fd9b6d9d640 1 -- 172.21.15.5:0/745894269 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b819af90 con 0x7fd9b8100850 2024-09-19T06:25:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.568+0000 7fd9b6d9d640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b8100850 0x7fd9b8195fa0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8010a20 tx=0x7fd9a8010ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.568+0000 7fd99ffff640 1 -- 172.21.15.5:0/745894269 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a801a030 con 0x7fd9b8100850 2024-09-19T06:25:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.568+0000 7fd9bca00640 1 -- 172.21.15.5:0/745894269 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b819b280 con 0x7fd9b8100850 2024-09-19T06:25:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.568+0000 7fd99ffff640 1 -- 172.21.15.5:0/745894269 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9a8004020 con 0x7fd9b8100850 2024-09-19T06:25:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.569+0000 7fd99ffff640 1 -- 172.21.15.5:0/745894269 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8003ba0 con 0x7fd9b8100850 2024-09-19T06:25:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.569+0000 7fd9bca00640 1 -- 172.21.15.5:0/745894269 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b81a2b10 con 0x7fd9b8100850 2024-09-19T06:25:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9c0075740 1 Processor -- start 2024-09-19T06:25:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9c0075740 1 -- start start 2024-09-19T06:25:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd99ffff640 1 -- 172.21.15.5:0/745894269 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd9a801d070 con 0x7fd9b8100850 2024-09-19T06:25:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9c0075740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3ccc0 0x564256d3d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9c0075740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3d5e0 0x564256d47940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9c0075740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d47e80 0x564256d4a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9c0075740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564256ca12f0 con 0x564256d3d5e0 2024-09-19T06:25:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9c0075740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564256c49270 con 0x564256d47e80 2024-09-19T06:25:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9c0075740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564256c9d2b0 con 0x564256d3ccc0 2024-09-19T06:25:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9b6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3d5e0 0x564256d47940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9b759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3ccc0 0x564256d3d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9b759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3ccc0 0x564256d3d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36146/0 (socket says 172.21.15.5:36146) 2024-09-19T06:25:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9b6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3d5e0 0x564256d47940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50128/0 (socket says 172.21.15.5:50128) 2024-09-19T06:25:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9b759e640 1 -- 172.21.15.5:0/2020105350 learned_addr learned my addr 172.21.15.5:0/2020105350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.572+0000 7fd9b659c640 1 --2- 172.21.15.5:0/2020105350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d47e80 0x564256d4a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd9b759e640 1 -- 172.21.15.5:0/2020105350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d47e80 msgr2=0x564256d4a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd9b759e640 1 --2- 172.21.15.5:0/2020105350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d47e80 0x564256d4a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd9b759e640 1 -- 172.21.15.5:0/2020105350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3d5e0 msgr2=0x564256d47940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd9b759e640 1 --2- 172.21.15.5:0/2020105350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3d5e0 0x564256d47940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd9b759e640 1 -- 172.21.15.5:0/2020105350 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564256c48340 con 0x564256d3ccc0 2024-09-19T06:25:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd9b659c640 1 --2- 172.21.15.5:0/2020105350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d47e80 0x564256d4a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:25:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd9b759e640 1 --2- 172.21.15.5:0/2020105350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3ccc0 0x564256d3d0a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0011570 tx=0x7fd9b0011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c27f8eb8feeb4479 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd99ffff640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd9a00a33a0 0x7fd9a00a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd99ffff640 1 -- 172.21.15.5:0/745894269 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7fd9a80d1870 con 0x7fd9b8100850 2024-09-19T06:25:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd9b6d9d640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd9a00a33a0 0x7fd9a00a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd99d7fa640 1 -- 172.21.15.5:0/2020105350 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b001e070 con 0x564256d3ccc0 2024-09-19T06:25:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd99d7fa640 1 -- 172.21.15.5:0/2020105350 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9b000fa70 con 0x564256d3ccc0 2024-09-19T06:25:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.573+0000 7fd99d7fa640 1 -- 172.21.15.5:0/2020105350 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0019c00 con 0x564256d3ccc0 2024-09-19T06:25:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.574+0000 7fd9c0075740 1 -- 172.21.15.5:0/2020105350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3ccc0 msgr2=0x564256d3d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.574+0000 7fd9b6d9d640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd9a00a33a0 0x7fd9a00a5860 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80cf050 tx=0x7fd9a80d7550 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.574+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2020105350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3ccc0 0x564256d3d0a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0011570 tx=0x7fd9b0011a30 comp rx=0 tx=0).stop 2024-09-19T06:25:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.575+0000 7fd9c0075740 1 -- 172.21.15.5:0/2020105350 shutdown_connections 2024-09-19T06:25:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.575+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2020105350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d47e80 0x564256d4a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.575+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2020105350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3d5e0 0x564256d47940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.575+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2020105350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3ccc0 0x564256d3d0a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.575+0000 7fd9c0075740 1 -- 172.21.15.5:0/2020105350 >> 172.21.15.5:0/2020105350 conn(0x564256d39230 msgr2=0x564256d39610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.575+0000 7fd9c0075740 1 -- 172.21.15.5:0/2020105350 shutdown_connections 2024-09-19T06:25:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.575+0000 7fd9c0075740 1 -- 172.21.15.5:0/2020105350 wait complete. 2024-09-19T06:25:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.575+0000 7fd9c0075740 1 -- start start 2024-09-19T06:25:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.575+0000 7fd9c0075740 1 Processor -- start 2024-09-19T06:25:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9c0075740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3ccc0 0x564256d44e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9c0075740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3d5e0 0x564256d45360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9c0075740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d47e80 0x564256d458a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9c0075740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564256d4aa50 con 0x564256d3ccc0 2024-09-19T06:25:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9c0075740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564256c2a290 con 0x564256d3d5e0 2024-09-19T06:25:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9c0075740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564256d4abd0 con 0x564256d47e80 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9b759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3ccc0 0x564256d44e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9b759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3ccc0 0x564256d44e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50136/0 (socket says 172.21.15.5:50136) 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9b759e640 1 -- 172.21.15.5:0/2885036310 learned_addr learned my addr 172.21.15.5:0/2885036310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9b659c640 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3d5e0 0x564256d45360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9b759e640 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d47e80 0x564256d458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9c0075740 10 client.0 _ll_register_callbacks cb 0x564256c5b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9b759e640 1 -- 172.21.15.5:0/2885036310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d47e80 msgr2=0x564256d458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9b759e640 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d47e80 0x564256d458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9b759e640 1 -- 172.21.15.5:0/2885036310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3d5e0 msgr2=0x564256d45360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9b759e640 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3d5e0 0x564256d45360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9b759e640 1 -- 172.21.15.5:0/2885036310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642569b4210 con 0x564256d3ccc0 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.576+0000 7fd9b659c640 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3d5e0 0x564256d45360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.577+0000 7fd9b759e640 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3ccc0 0x564256d44e20 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0019740 tx=0x7fd9b00197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e16ff8af7a769 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.577+0000 7fd99aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.577+0000 7fd99aff5640 1 -- 172.21.15.5:0/2885036310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b001e070 con 0x564256d3ccc0 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.577+0000 7fd9c0075740 1 -- 172.21.15.5:0/2885036310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9b000fa70 con 0x564256d3ccc0 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.577+0000 7fd99aff5640 1 -- 172.21.15.5:0/2885036310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7fd9b0019b50 con 0x564256d3ccc0 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.577+0000 7fd99aff5640 1 -- 172.21.15.5:0/2885036310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fd9b0019d50 con 0x564256d3ccc0 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.577+0000 7fd9c0075740 10 client.21000 fetch_fsmap learned FSMap version 62 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.577+0000 7fd9c0075740 1 -- 172.21.15.5:0/2885036310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x564256a80620 con 0x564256d3ccc0 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.577+0000 7fd99aff5640 1 -- 172.21.15.5:0/2885036310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9b000fa70 con 0x564256d3ccc0 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.578+0000 7fd99aff5640 1 -- 172.21.15.5:0/2885036310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fd9b0004330 con 0x564256d3ccc0 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.578+0000 7fd9c0075740 10 client.21000 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.578+0000 7fd9c0075740 10 client.21000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.578+0000 7fd9c0075740 20 client.21000 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.578+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564256d60980 0x564256d62d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.578+0000 7fd9c0075740 4 client.21000 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:25:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.578+0000 7fd9c0075740 1 -- 172.21.15.5:0/2885036310 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564256caea20 con 0x564256d60980 2024-09-19T06:25:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.578+0000 7fd9b659c640 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564256d60980 0x564256d62d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.579+0000 7fd9b659c640 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564256d60980 0x564256d62d70 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a07a59d87f5a73e2 server_cookie=9e5c964f27b3200 in_seq=0 out_seq=0 2024-09-19T06:25:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.579+0000 7fd99aff5640 10 client.21000 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.579+0000 7fd99aff5640 1 -- 172.21.15.5:0/2885036310 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564256caea20 con 0x564256d60980 2024-09-19T06:25:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.579+0000 7fd99aff5640 10 client.21000 handle_command_reply: tid=0 2024-09-19T06:25:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.579+0000 7fd9c0075740 1 client.21000 shutdown 2024-09-19T06:25:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.580+0000 7fd9c0075740 1 -- 172.21.15.5:0/2885036310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3ccc0 msgr2=0x564256d44e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.580+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3ccc0 0x564256d44e20 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0019740 tx=0x7fd9b00197a0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.580+0000 7fd9c0075740 1 -- 172.21.15.5:0/2885036310 shutdown_connections 2024-09-19T06:25:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.580+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564256d60980 0x564256d62d70 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.580+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d47e80 0x564256d458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.580+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3d5e0 0x564256d45360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.580+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2885036310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d3ccc0 0x564256d44e20 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.580+0000 7fd9c0075740 1 -- 172.21.15.5:0/2885036310 >> 172.21.15.5:0/2885036310 conn(0x564256d39230 msgr2=0x564256d3ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.580+0000 7fd99aff5640 0 client.21000 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.580+0000 7fd9c0075740 1 -- 172.21.15.5:0/2885036310 shutdown_connections 2024-09-19T06:25:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.580+0000 7fd9c0075740 1 -- 172.21.15.5:0/2885036310 wait complete. 2024-09-19T06:25:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.580+0000 7fd9c0075740 20 client.21000 trim_cache size 0 max 16384 2024-09-19T06:25:59.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90991 8.59 MB/sec execute 443 sec latency 89.165 ms 2024-09-19T06:25:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9c0075740 1 Processor -- start 2024-09-19T06:25:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9c0075740 1 -- start start 2024-09-19T06:25:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9c0075740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3ccc0 0x564256d68660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9c0075740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3d5e0 0x564256d68ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9c0075740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d47e80 0x564256d46680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9c0075740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81ab560 con 0x564256d47e80 2024-09-19T06:25:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9c0075740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81ad0a0 con 0x564256d3ccc0 2024-09-19T06:25:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9c0075740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81abc30 con 0x564256d3d5e0 2024-09-19T06:25:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9b659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3d5e0 0x564256d68ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9b759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3ccc0 0x564256d68660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9b659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3d5e0 0x564256d68ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36168/0 (socket says 172.21.15.5:36168) 2024-09-19T06:25:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9b759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d47e80 0x564256d46680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.608+0000 7fd9b659c640 1 -- 172.21.15.5:0/2392719683 learned_addr learned my addr 172.21.15.5:0/2392719683 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.609+0000 7fd9b659c640 1 -- 172.21.15.5:0/2392719683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3ccc0 msgr2=0x564256d68660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.609+0000 7fd9b659c640 1 --2- 172.21.15.5:0/2392719683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3ccc0 0x564256d68660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.609+0000 7fd9b659c640 1 -- 172.21.15.5:0/2392719683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d47e80 msgr2=0x564256d46680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.609+0000 7fd9b659c640 1 --2- 172.21.15.5:0/2392719683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d47e80 0x564256d46680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.609+0000 7fd9b659c640 1 -- 172.21.15.5:0/2392719683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564256d3d410 con 0x564256d3d5e0 2024-09-19T06:25:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.609+0000 7fd9b659c640 1 --2- 172.21.15.5:0/2392719683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3d5e0 0x564256d68ba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac02aa10 tx=0x7fd9ac005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c6a54f0c0355678 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.609+0000 7fd99cff9640 1 -- 172.21.15.5:0/2392719683 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac035040 con 0x564256d3d5e0 2024-09-19T06:25:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.609+0000 7fd99cff9640 1 -- 172.21.15.5:0/2392719683 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9ac04a070 con 0x564256d3d5e0 2024-09-19T06:25:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.609+0000 7fd99cff9640 1 -- 172.21.15.5:0/2392719683 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac045380 con 0x564256d3d5e0 2024-09-19T06:25:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.610+0000 7fd9c0075740 1 -- 172.21.15.5:0/2392719683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3d5e0 msgr2=0x564256d68ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.610+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2392719683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3d5e0 0x564256d68ba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac02aa10 tx=0x7fd9ac005a20 comp rx=0 tx=0).stop 2024-09-19T06:25:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.610+0000 7fd9c0075740 1 -- 172.21.15.5:0/2392719683 shutdown_connections 2024-09-19T06:25:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.610+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2392719683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d47e80 0x564256d46680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.610+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2392719683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3d5e0 0x564256d68ba0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.610+0000 7fd9c0075740 1 --2- 172.21.15.5:0/2392719683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3ccc0 0x564256d68660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.610+0000 7fd9c0075740 1 -- 172.21.15.5:0/2392719683 >> 172.21.15.5:0/2392719683 conn(0x564256d39230 msgr2=0x564256d40100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.610+0000 7fd9c0075740 1 -- 172.21.15.5:0/2392719683 shutdown_connections 2024-09-19T06:25:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.610+0000 7fd9c0075740 1 -- 172.21.15.5:0/2392719683 wait complete. 2024-09-19T06:25:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9c0075740 1 -- start start 2024-09-19T06:25:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9c0075740 1 Processor -- start 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9c0075740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3ccc0 0x564256d427b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9c0075740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3d5e0 0x564256d586f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9c0075740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d47e80 0x564256d58c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9c0075740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81ac9d0 con 0x564256d47e80 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9c0075740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81ac300 con 0x564256d3d5e0 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9c0075740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81a9560 con 0x564256d3ccc0 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9c0075740 10 client.0 _ll_register_callbacks cb 0x7fd9b001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9b759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3ccc0 0x564256d427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9b759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d47e80 0x564256d58c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9b659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3d5e0 0x564256d586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9b759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d47e80 0x564256d58c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50162/0 (socket says 172.21.15.5:50162) 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.611+0000 7fd9b759e640 1 -- 172.21.15.5:0/3664413520 learned_addr learned my addr 172.21.15.5:0/3664413520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:25:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.612+0000 7fd9b659c640 1 -- 172.21.15.5:0/3664413520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3ccc0 msgr2=0x564256d427b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.612+0000 7fd9b659c640 1 --2- 172.21.15.5:0/3664413520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3ccc0 0x564256d427b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.612+0000 7fd9b659c640 1 -- 172.21.15.5:0/3664413520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d47e80 msgr2=0x564256d58c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:25:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.612+0000 7fd9b659c640 1 --2- 172.21.15.5:0/3664413520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d47e80 0x564256d58c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.612+0000 7fd9b659c640 1 -- 172.21.15.5:0/3664413520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564256d59320 con 0x564256d3d5e0 2024-09-19T06:25:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.612+0000 7fd9b759e640 1 --2- 172.21.15.5:0/3664413520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d47e80 0x564256d58c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:25:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.612+0000 7fd9b659c640 1 --2- 172.21.15.5:0/3664413520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3d5e0 0x564256d586f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x564256d34210 tx=0x7fd9ac005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d85cb485fcdd77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:25:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.612+0000 7fd99aff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.612+0000 7fd99aff5640 1 -- 172.21.15.5:0/3664413520 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac052070 con 0x564256d3d5e0 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.612+0000 7fd99aff5640 1 -- 172.21.15.5:0/3664413520 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8550+0+0 (secure 0 0 0) 0x7fd9ac0048d0 con 0x564256d3d5e0 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.612+0000 7fd9c0075740 1 -- 172.21.15.5:0/3664413520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ac04a070 con 0x564256d3d5e0 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.612+0000 7fd99aff5640 1 -- 172.21.15.5:0/3664413520 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fd9ac035050 con 0x564256d3d5e0 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.613+0000 7fd9c0075740 10 client.30383 fetch_fsmap learned FSMap version 62 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.613+0000 7fd9c0075740 1 -- 172.21.15.5:0/3664413520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x564256dca6f0 con 0x564256d3d5e0 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.613+0000 7fd99aff5640 1 -- 172.21.15.5:0/3664413520 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9ac04a070 con 0x564256d3d5e0 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.613+0000 7fd99aff5640 1 -- 172.21.15.5:0/3664413520 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fd9ac003970 con 0x564256d3d5e0 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.613+0000 7fd9c0075740 10 client.30383 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.613+0000 7fd9c0075740 10 client.30383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.613+0000 7fd9c0075740 20 client.30383 populate_metadata read hostname 'smithi005' 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.613+0000 7fd9c0075740 1 --2- 172.21.15.5:0/3664413520 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564256dcadb0 0x564256dcd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:25:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.613+0000 7fd9c0075740 4 client.30383 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:25:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.613+0000 7fd9c0075740 1 -- 172.21.15.5:0/3664413520 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564256ca8d70 con 0x564256dcadb0 2024-09-19T06:25:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.613+0000 7fd9b759e640 1 --2- 172.21.15.5:0/3664413520 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564256dcadb0 0x564256dcd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:25:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.614+0000 7fd9b759e640 1 --2- 172.21.15.5:0/3664413520 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564256dcadb0 0x564256dcd1d0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0e8e2d6cf6859ce server_cookie=5620f3f6425f07b3 in_seq=0 out_seq=0 2024-09-19T06:25:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.614+0000 7fd99aff5640 10 client.30383 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd99aff5640 1 -- 172.21.15.5:0/3664413520 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564256ca8d70 con 0x564256dcadb0 2024-09-19T06:25:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd99aff5640 10 client.30383 handle_command_reply: tid=0 2024-09-19T06:25:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd9c0075740 1 client.30383 shutdown 2024-09-19T06:25:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd9c0075740 1 -- 172.21.15.5:0/3664413520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3d5e0 msgr2=0x564256d586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd9c0075740 1 --2- 172.21.15.5:0/3664413520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3d5e0 0x564256d586f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x564256d34210 tx=0x7fd9ac005e70 comp rx=0 tx=0).stop 2024-09-19T06:25:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd9c0075740 1 -- 172.21.15.5:0/3664413520 shutdown_connections 2024-09-19T06:25:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd9c0075740 1 --2- 172.21.15.5:0/3664413520 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564256dcadb0 0x564256dcd1d0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd9c0075740 1 --2- 172.21.15.5:0/3664413520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564256d47e80 0x564256d58c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd9c0075740 1 --2- 172.21.15.5:0/3664413520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564256d3d5e0 0x564256d586f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd99aff5640 0 client.30383 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:25:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd9c0075740 1 --2- 172.21.15.5:0/3664413520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256d3ccc0 0x564256d427b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd9c0075740 1 -- 172.21.15.5:0/3664413520 >> 172.21.15.5:0/3664413520 conn(0x564256d39230 msgr2=0x564256d46680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd9c0075740 1 -- 172.21.15.5:0/3664413520 shutdown_connections 2024-09-19T06:25:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd9c0075740 1 -- 172.21.15.5:0/3664413520 wait complete. 2024-09-19T06:25:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.696+0000 7fd9c0075740 20 client.30383 trim_cache size 0 max 16384 2024-09-19T06:25:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:25:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:25:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:25:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:25:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.697+0000 7fd9bca00640 1 -- 172.21.15.5:0/745894269 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd9a00a33a0 msgr2=0x7fd9a00a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.697+0000 7fd9bca00640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd9a00a33a0 0x7fd9a00a5860 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80cf050 tx=0x7fd9a80d7550 comp rx=0 tx=0).stop 2024-09-19T06:25:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.697+0000 7fd9bca00640 1 -- 172.21.15.5:0/745894269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b8100850 msgr2=0x7fd9b8195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:25:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.697+0000 7fd9bca00640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b8100850 0x7fd9b8195fa0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8010a20 tx=0x7fd9a8010ee0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.697+0000 7fd9bca00640 1 -- 172.21.15.5:0/745894269 shutdown_connections 2024-09-19T06:25:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.697+0000 7fd9bca00640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9b8106860 0x7fd9b819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.697+0000 7fd9bca00640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd9a00a33a0 0x7fd9a00a5860 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.697+0000 7fd9bca00640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b81011f0 0x7fd9b81964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.697+0000 7fd9bca00640 1 --2- 172.21.15.5:0/745894269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b8100850 0x7fd9b8195fa0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:25:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.697+0000 7fd9bca00640 1 -- 172.21.15.5:0/745894269 >> 172.21.15.5:0/745894269 conn(0x7fd9b80fc550 msgr2=0x7fd9b80fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:25:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.698+0000 7fd9bca00640 1 -- 172.21.15.5:0/745894269 shutdown_connections 2024-09-19T06:25:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:25:59.698+0000 7fd9bca00640 1 -- 172.21.15.5:0/745894269 wait complete. 2024-09-19T06:26:00.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 91180 8.59 MB/sec execute 444 sec latency 85.731 ms 2024-09-19T06:26:00.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:00 smithi005 ceph-mon[30955]: pgmap v715: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 242 op/s 2024-09-19T06:26:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:00 smithi049 ceph-mon[34640]: pgmap v715: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 242 op/s 2024-09-19T06:26:01.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:00 smithi157 ceph-mon[35346]: pgmap v715: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 242 op/s 2024-09-19T06:26:01.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 91318 8.59 MB/sec execute 445 sec latency 89.432 ms 2024-09-19T06:26:02.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 91453 8.58 MB/sec execute 446 sec latency 64.862 ms 2024-09-19T06:26:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:02 smithi005 ceph-mon[30955]: pgmap v716: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 242 op/s 2024-09-19T06:26:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:02 smithi049 ceph-mon[34640]: pgmap v716: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 242 op/s 2024-09-19T06:26:03.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:02 smithi157 ceph-mon[35346]: pgmap v716: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 242 op/s 2024-09-19T06:26:03.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 91640 8.59 MB/sec execute 447 sec latency 74.583 ms 2024-09-19T06:26:04.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 91820 8.60 MB/sec execute 448 sec latency 65.867 ms 2024-09-19T06:26:04.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:04 smithi005 ceph-mon[30955]: pgmap v717: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.4 MiB/s wr, 411 op/s 2024-09-19T06:26:04.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:04 smithi049 ceph-mon[34640]: pgmap v717: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.4 MiB/s wr, 411 op/s 2024-09-19T06:26:05.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:04 smithi157 ceph-mon[35346]: pgmap v717: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.4 MiB/s wr, 411 op/s 2024-09-19T06:26:05.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 91952 8.59 MB/sec execute 449 sec latency 128.110 ms 2024-09-19T06:26:05.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:05 smithi005 ceph-mon[30955]: osdmap e118: 12 total, 12 up, 12 in 2024-09-19T06:26:05.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:05 smithi049 ceph-mon[34640]: osdmap e118: 12 total, 12 up, 12 in 2024-09-19T06:26:06.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:05 smithi157 ceph-mon[35346]: osdmap e118: 12 total, 12 up, 12 in 2024-09-19T06:26:06.337 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:26:06.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 92100 8.58 MB/sec execute 450 sec latency 124.691 ms 2024-09-19T06:26:06.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:06 smithi005 ceph-mon[30955]: pgmap v719: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.9 MiB/s wr, 345 op/s 2024-09-19T06:26:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:26:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:06 smithi049 ceph-mon[34640]: pgmap v719: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.9 MiB/s wr, 345 op/s 2024-09-19T06:26:07.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:06 smithi157 ceph-mon[35346]: pgmap v719: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.9 MiB/s wr, 345 op/s 2024-09-19T06:26:07.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 92226 8.56 MB/sec execute 451 sec latency 91.322 ms 2024-09-19T06:26:08.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 92371 8.56 MB/sec execute 452 sec latency 107.167 ms 2024-09-19T06:26:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:08 smithi005 ceph-mon[30955]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 351 op/s 2024-09-19T06:26:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:08 smithi005 ceph-mon[30955]: osdmap e119: 12 total, 12 up, 12 in 2024-09-19T06:26:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:08 smithi049 ceph-mon[34640]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 351 op/s 2024-09-19T06:26:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:08 smithi049 ceph-mon[34640]: osdmap e119: 12 total, 12 up, 12 in 2024-09-19T06:26:09.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:08 smithi157 ceph-mon[35346]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 351 op/s 2024-09-19T06:26:09.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:08 smithi157 ceph-mon[35346]: osdmap e119: 12 total, 12 up, 12 in 2024-09-19T06:26:09.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 92519 8.56 MB/sec execute 453 sec latency 102.546 ms 2024-09-19T06:26:10.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:09 smithi157 ceph-mon[35346]: pgmap v722: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.6 MiB/s wr, 439 op/s 2024-09-19T06:26:10.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:09 smithi005 ceph-mon[30955]: pgmap v722: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.6 MiB/s wr, 439 op/s 2024-09-19T06:26:10.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:09 smithi049 ceph-mon[34640]: pgmap v722: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.6 MiB/s wr, 439 op/s 2024-09-19T06:26:10.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 92685 8.57 MB/sec execute 454 sec latency 119.320 ms 2024-09-19T06:26:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.393+0000 7f7a48da2640 1 -- 172.21.15.5:0/3031304161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a44101df0 msgr2=0x7f7a441021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.393+0000 7f7a48da2640 1 --2- 172.21.15.5:0/3031304161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a44101df0 0x7f7a441021d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009860 tx=0x7f7a3402a340 comp rx=0 tx=0).stop 2024-09-19T06:26:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.393+0000 7f7a48da2640 1 -- 172.21.15.5:0/3031304161 shutdown_connections 2024-09-19T06:26:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.393+0000 7f7a48da2640 1 --2- 172.21.15.5:0/3031304161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a44106f20 0x7f7a4410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.393+0000 7f7a48da2640 1 --2- 172.21.15.5:0/3031304161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a441027a0 0x7f7a441067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.393+0000 7f7a48da2640 1 --2- 172.21.15.5:0/3031304161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a44101df0 0x7f7a441021d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.393+0000 7f7a48da2640 1 -- 172.21.15.5:0/3031304161 >> 172.21.15.5:0/3031304161 conn(0x7f7a440fd8f0 msgr2=0x7f7a440ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.393+0000 7f7a48da2640 1 -- 172.21.15.5:0/3031304161 shutdown_connections 2024-09-19T06:26:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.393+0000 7f7a48da2640 1 -- 172.21.15.5:0/3031304161 wait complete. 2024-09-19T06:26:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.394+0000 7f7a48da2640 1 Processor -- start 2024-09-19T06:26:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.394+0000 7f7a48da2640 1 -- start start 2024-09-19T06:26:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.394+0000 7f7a48da2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a44101df0 0x7f7a4419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.394+0000 7f7a48da2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a441027a0 0x7f7a4419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.394+0000 7f7a48da2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a44106f20 0x7f7a4419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a48da2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7a44111df0 con 0x7f7a44106f20 2024-09-19T06:26:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a48da2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7a44111c70 con 0x7f7a441027a0 2024-09-19T06:26:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a48da2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7a44111f70 con 0x7f7a44101df0 2024-09-19T06:26:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a41d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a441027a0 0x7f7a4419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a4259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a44101df0 0x7f7a4419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a41d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a441027a0 0x7f7a4419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36164/0 (socket says 172.21.15.5:36164) 2024-09-19T06:26:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a41d9b640 1 -- 172.21.15.5:0/3623871478 learned_addr learned my addr 172.21.15.5:0/3623871478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a42d9d640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a44106f20 0x7f7a4419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a41d9b640 1 -- 172.21.15.5:0/3623871478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a44101df0 msgr2=0x7f7a4419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a41d9b640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a44101df0 0x7f7a4419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a41d9b640 1 -- 172.21.15.5:0/3623871478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a44106f20 msgr2=0x7f7a4419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a41d9b640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a44106f20 0x7f7a4419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.395+0000 7f7a41d9b640 1 -- 172.21.15.5:0/3623871478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4419f540 con 0x7f7a441027a0 2024-09-19T06:26:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.396+0000 7f7a41d9b640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a441027a0 0x7f7a4419aad0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800e9c0 tx=0x7f7a3800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.396+0000 7f7a337fe640 1 -- 172.21.15.5:0/3623871478 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800cde0 con 0x7f7a441027a0 2024-09-19T06:26:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.396+0000 7f7a337fe640 1 -- 172.21.15.5:0/3623871478 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a380035f0 con 0x7f7a441027a0 2024-09-19T06:26:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.396+0000 7f7a337fe640 1 -- 172.21.15.5:0/3623871478 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38010690 con 0x7f7a441027a0 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.396+0000 7f7a48da2640 1 -- 172.21.15.5:0/3623871478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4419f830 con 0x7f7a441027a0 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.396+0000 7f7a48da2640 1 -- 172.21.15.5:0/3623871478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a441a7110 con 0x7f7a441027a0 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.399+0000 7f7a337fe640 1 -- 172.21.15.5:0/3623871478 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7a380037a0 con 0x7f7a441027a0 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.399+0000 7f7a4b17b740 1 Processor -- start 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.399+0000 7f7a4b17b740 1 -- start start 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a4b17b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57e4a0 0x558eed57e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a4b17b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57edc0 0x558eed589140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a4b17b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed58ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a4b17b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558eed4e42e0 con 0x558eed57edc0 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a4b17b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558eed48bce0 con 0x558eed57e4a0 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a4b17b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558eed4e02a0 con 0x558eed589680 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57edc0 0x558eed589140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a42d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57e4a0 0x558eed57e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a42d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57e4a0 0x558eed57e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36176/0 (socket says 172.21.15.5:36176) 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a42d9d640 1 -- 172.21.15.5:0/233781264 learned_addr learned my addr 172.21.15.5:0/233781264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a41d9b640 1 --2- 172.21.15.5:0/233781264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed58ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a42d9d640 1 -- 172.21.15.5:0/233781264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 msgr2=0x558eed58ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a42d9d640 1 --2- 172.21.15.5:0/233781264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed58ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a42d9d640 1 -- 172.21.15.5:0/233781264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57edc0 msgr2=0x558eed589140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a42d9d640 1 --2- 172.21.15.5:0/233781264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57edc0 0x558eed589140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a42d9d640 1 -- 172.21.15.5:0/233781264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eed470440 con 0x558eed57e4a0 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.400+0000 7f7a4259c640 1 --2- 172.21.15.5:0/233781264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57edc0 0x558eed589140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.401+0000 7f7a42d9d640 1 --2- 172.21.15.5:0/233781264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57e4a0 0x558eed57e880 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009b60 tx=0x7f7a3c00e780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b534374c1971ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.401+0000 7f7a30ff9640 1 -- 172.21.15.5:0/233781264 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c009c50 con 0x558eed57e4a0 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.401+0000 7f7a30ff9640 1 -- 172.21.15.5:0/233781264 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a3c003e20 con 0x558eed57e4a0 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.401+0000 7f7a337fe640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7a2c0a3430 0x7f7a2c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.401+0000 7f7a30ff9640 1 -- 172.21.15.5:0/233781264 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c0203c0 con 0x558eed57e4a0 2024-09-19T06:26:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.401+0000 7f7a4259c640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7a2c0a3430 0x7f7a2c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.401+0000 7f7a337fe640 1 -- 172.21.15.5:0/3623871478 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f7a380cfff0 con 0x7f7a441027a0 2024-09-19T06:26:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.402+0000 7f7a4259c640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7a2c0a3430 0x7f7a2c0a58f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7a3402a930 tx=0x7f7a340058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.402+0000 7f7a4b17b740 1 -- 172.21.15.5:0/233781264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57e4a0 msgr2=0x558eed57e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.402+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/233781264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57e4a0 0x558eed57e880 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009b60 tx=0x7f7a3c00e780 comp rx=0 tx=0).stop 2024-09-19T06:26:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.402+0000 7f7a4b17b740 1 -- 172.21.15.5:0/233781264 shutdown_connections 2024-09-19T06:26:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.402+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/233781264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed58ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.402+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/233781264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57edc0 0x558eed589140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.402+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/233781264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57e4a0 0x558eed57e880 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.402+0000 7f7a4b17b740 1 -- 172.21.15.5:0/233781264 >> 172.21.15.5:0/233781264 conn(0x558eed57aa30 msgr2=0x558eed57ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.402+0000 7f7a4b17b740 1 -- 172.21.15.5:0/233781264 shutdown_connections 2024-09-19T06:26:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.402+0000 7f7a4b17b740 1 -- 172.21.15.5:0/233781264 wait complete. 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.402+0000 7f7a4b17b740 1 -- start start 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a4b17b740 1 Processor -- start 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a4b17b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 0x558eed588f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a4b17b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 0x558eed5861c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a4b17b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed586700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 0x558eed588f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a4b17b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558eed58c2f0 con 0x558eed57e4a0 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a4b17b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558eed49f050 con 0x558eed57edc0 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a41d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 0x558eed5861c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a4b17b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558eed58c470 con 0x558eed589680 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 0x558eed588f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52020/0 (socket says 172.21.15.5:52020) 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a4259c640 1 -- 172.21.15.5:0/1566390923 learned_addr learned my addr 172.21.15.5:0/1566390923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a42d9d640 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed586700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.403+0000 7f7a4b17b740 10 client.0 _ll_register_callbacks cb 0x558eed4df160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.404+0000 7f7a4259c640 1 -- 172.21.15.5:0/1566390923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 msgr2=0x558eed586700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.404+0000 7f7a4259c640 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed586700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.404+0000 7f7a4259c640 1 -- 172.21.15.5:0/1566390923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 msgr2=0x558eed5861c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.404+0000 7f7a4259c640 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 0x558eed5861c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.404+0000 7f7a4259c640 1 -- 172.21.15.5:0/1566390923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eed2553c0 con 0x558eed57e4a0 2024-09-19T06:26:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.404+0000 7f7a41d9b640 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 0x558eed5861c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.404+0000 7f7a4259c640 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 0x558eed588f10 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009230 tx=0x7f7a3402a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9414f6d1885ef07f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.404+0000 7f7a29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:26:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.404+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1566390923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34037040 con 0x558eed57e4a0 2024-09-19T06:26:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.404+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1566390923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f7a340037f0 con 0x558eed57e4a0 2024-09-19T06:26:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.404+0000 7f7a4b17b740 1 -- 172.21.15.5:0/1566390923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a3c003e20 con 0x558eed57e4a0 2024-09-19T06:26:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.405+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1566390923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f7a340044d0 con 0x558eed57e4a0 2024-09-19T06:26:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.405+0000 7f7a4b17b740 10 client.21018 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.405+0000 7f7a4b17b740 1 -- 172.21.15.5:0/1566390923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x558eed597bc0 con 0x558eed57e4a0 2024-09-19T06:26:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.405+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1566390923 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a3c003e20 con 0x558eed57e4a0 2024-09-19T06:26:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.405+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1566390923 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f7a34032960 con 0x558eed57e4a0 2024-09-19T06:26:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.405+0000 7f7a4b17b740 10 client.21018 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.405+0000 7f7a4b17b740 10 client.21018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.405+0000 7f7a4b17b740 20 client.21018 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.405+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558eed599080 0x558eed59b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.405+0000 7f7a4b17b740 4 client.21018 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:26:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.405+0000 7f7a4b17b740 1 -- 172.21.15.5:0/1566390923 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558eed4f1850 con 0x558eed599080 2024-09-19T06:26:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.405+0000 7f7a41d9b640 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558eed599080 0x558eed59b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.406+0000 7f7a41d9b640 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558eed599080 0x558eed59b4d0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eadebba0de696298 server_cookie=7d919206c9cf9412 in_seq=0 out_seq=0 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.406+0000 7f7a29ffb640 10 client.21018 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1566390923 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558eed4f1850 con 0x558eed599080 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a29ffb640 10 client.21018 handle_command_reply: tid=0 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a4b17b740 1 client.21018 shutdown 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a4b17b740 1 -- 172.21.15.5:0/1566390923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 msgr2=0x558eed588f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 0x558eed588f10 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009230 tx=0x7f7a3402a340 comp rx=0 tx=0).stop 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a4b17b740 1 -- 172.21.15.5:0/1566390923 shutdown_connections 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558eed599080 0x558eed59b4d0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed586700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 0x558eed5861c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/1566390923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 0x558eed588f10 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a4b17b740 1 -- 172.21.15.5:0/1566390923 >> 172.21.15.5:0/1566390923 conn(0x558eed57aa30 msgr2=0x558eed579b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a29ffb640 0 client.21018 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a4b17b740 1 -- 172.21.15.5:0/1566390923 shutdown_connections 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.407+0000 7f7a4b17b740 1 -- 172.21.15.5:0/1566390923 wait complete. 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.408+0000 7f7a4b17b740 20 client.21018 trim_cache size 0 max 16384 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.435+0000 7f7a4b17b740 1 Processor -- start 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.435+0000 7f7a4b17b740 1 -- start start 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.435+0000 7f7a4b17b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57e4a0 0x558eed595550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.435+0000 7f7a4b17b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed57edc0 0x558eed5a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.435+0000 7f7a4b17b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed589680 0x558eed5a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.435+0000 7f7a4b17b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441ab7f0 con 0x558eed589680 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.435+0000 7f7a4b17b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441ad330 con 0x558eed57e4a0 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.435+0000 7f7a4b17b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441abec0 con 0x558eed57edc0 2024-09-19T06:26:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.436+0000 7f7a42d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed589680 0x558eed5a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.436+0000 7f7a41d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed57edc0 0x558eed5a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.436+0000 7f7a41d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed57edc0 0x558eed5a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50212/0 (socket says 172.21.15.5:50212) 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.436+0000 7f7a41d9b640 1 -- 172.21.15.5:0/2341555376 learned_addr learned my addr 172.21.15.5:0/2341555376 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.436+0000 7f7a4259c640 1 --2- 172.21.15.5:0/2341555376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57e4a0 0x558eed595550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.436+0000 7f7a42d9d640 1 -- 172.21.15.5:0/2341555376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed57edc0 msgr2=0x558eed5a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.436+0000 7f7a42d9d640 1 --2- 172.21.15.5:0/2341555376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed57edc0 0x558eed5a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.436+0000 7f7a42d9d640 1 -- 172.21.15.5:0/2341555376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57e4a0 msgr2=0x558eed595550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.436+0000 7f7a42d9d640 1 --2- 172.21.15.5:0/2341555376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57e4a0 0x558eed595550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.436+0000 7f7a42d9d640 1 -- 172.21.15.5:0/2341555376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eed59bb50 con 0x558eed589680 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.436+0000 7f7a42d9d640 1 --2- 172.21.15.5:0/2341555376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed589680 0x558eed5a12e0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009b30 tx=0x7f7a3c009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57bf92846593b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.437+0000 7f7a2bfff640 1 -- 172.21.15.5:0/2341555376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c026040 con 0x558eed589680 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.437+0000 7f7a2bfff640 1 -- 172.21.15.5:0/2341555376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a3c003e20 con 0x558eed589680 2024-09-19T06:26:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.437+0000 7f7a2bfff640 1 -- 172.21.15.5:0/2341555376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c004bf0 con 0x558eed589680 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.438+0000 7f7a4b17b740 1 -- 172.21.15.5:0/2341555376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed589680 msgr2=0x558eed5a12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.438+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/2341555376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed589680 0x558eed5a12e0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009b30 tx=0x7f7a3c009d20 comp rx=0 tx=0).stop 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.438+0000 7f7a4b17b740 1 -- 172.21.15.5:0/2341555376 shutdown_connections 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.438+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/2341555376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed589680 0x558eed5a12e0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.438+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/2341555376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed57edc0 0x558eed5a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.438+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/2341555376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57e4a0 0x558eed595550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.438+0000 7f7a4b17b740 1 -- 172.21.15.5:0/2341555376 >> 172.21.15.5:0/2341555376 conn(0x558eed57aa30 msgr2=0x558eed5852b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.438+0000 7f7a4b17b740 1 -- 172.21.15.5:0/2341555376 shutdown_connections 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.438+0000 7f7a4b17b740 1 -- 172.21.15.5:0/2341555376 wait complete. 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.438+0000 7f7a4b17b740 1 -- start start 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.438+0000 7f7a4b17b740 1 Processor -- start 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4b17b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 0x558eed591b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4b17b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 0x558eed58cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4b17b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed58d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4b17b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441acc60 con 0x558eed57e4a0 2024-09-19T06:26:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4b17b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441ac590 con 0x558eed57edc0 2024-09-19T06:26:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4b17b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441a9810 con 0x558eed589680 2024-09-19T06:26:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4b17b740 10 client.0 _ll_register_callbacks cb 0x7f7a38005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 0x558eed591b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a41d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 0x558eed58cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a42d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed58d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a41d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 0x558eed58cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36196/0 (socket says 172.21.15.5:36196) 2024-09-19T06:26:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 0x558eed591b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52052/0 (socket says 172.21.15.5:52052) 2024-09-19T06:26:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a41d9b640 1 -- 172.21.15.5:0/688980303 learned_addr learned my addr 172.21.15.5:0/688980303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4259c640 1 -- 172.21.15.5:0/688980303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 msgr2=0x558eed58d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4259c640 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed58d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4259c640 1 -- 172.21.15.5:0/688980303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 msgr2=0x558eed58cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4259c640 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 0x558eed58cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a4259c640 1 -- 172.21.15.5:0/688980303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eed58d970 con 0x558eed57e4a0 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a42d9d640 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed58d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.439+0000 7f7a41d9b640 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 0x558eed58cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.440+0000 7f7a4259c640 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 0x558eed591b20 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7a34032d80 tx=0x7f7a340044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9942109b1bd1a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.440+0000 7f7a29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.440+0000 7f7a29ffb640 1 -- 172.21.15.5:0/688980303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34060020 con 0x558eed57e4a0 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.440+0000 7f7a29ffb640 1 -- 172.21.15.5:0/688980303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f7a34032850 con 0x558eed57e4a0 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.440+0000 7f7a4b17b740 1 -- 172.21.15.5:0/688980303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a3c003e20 con 0x558eed57e4a0 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.440+0000 7f7a29ffb640 1 -- 172.21.15.5:0/688980303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f7a34032a70 con 0x558eed57e4a0 2024-09-19T06:26:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.440+0000 7f7a4b17b740 10 client.21024 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.441+0000 7f7a4b17b740 1 -- 172.21.15.5:0/688980303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x558eed603a90 con 0x558eed57e4a0 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.441+0000 7f7a29ffb640 1 -- 172.21.15.5:0/688980303 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a3c003e20 con 0x558eed57e4a0 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.441+0000 7f7a29ffb640 1 -- 172.21.15.5:0/688980303 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f7a34003930 con 0x558eed57e4a0 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.441+0000 7f7a4b17b740 10 client.21024 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.441+0000 7f7a4b17b740 10 client.21024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.441+0000 7f7a4b17b740 20 client.21024 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.441+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558eed604200 0x558eed606620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.441+0000 7f7a4b17b740 4 client.21024 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.441+0000 7f7a4b17b740 1 -- 172.21.15.5:0/688980303 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558eed4ebd60 con 0x558eed604200 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.441+0000 7f7a41d9b640 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558eed604200 0x558eed606620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a29ffb640 10 client.21024 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a41d9b640 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558eed604200 0x558eed606620 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c8c00f7c4f84328 server_cookie=d22f093dd356ce02 in_seq=0 out_seq=0 2024-09-19T06:26:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a29ffb640 1 -- 172.21.15.5:0/688980303 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558eed4ebd60 con 0x558eed604200 2024-09-19T06:26:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a29ffb640 10 client.21024 handle_command_reply: tid=0 2024-09-19T06:26:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a4b17b740 1 client.21024 shutdown 2024-09-19T06:26:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a4b17b740 1 -- 172.21.15.5:0/688980303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 msgr2=0x558eed591b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 0x558eed591b20 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7a34032d80 tx=0x7f7a340044d0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a4b17b740 1 -- 172.21.15.5:0/688980303 shutdown_connections 2024-09-19T06:26:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558eed604200 0x558eed606620 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558eed589680 0x558eed58d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eed57edc0 0x558eed58cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a29ffb640 0 client.21024 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a4b17b740 1 --2- 172.21.15.5:0/688980303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eed57e4a0 0x558eed591b20 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.442+0000 7f7a4b17b740 1 -- 172.21.15.5:0/688980303 >> 172.21.15.5:0/688980303 conn(0x558eed57aa30 msgr2=0x558eed580fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.443+0000 7f7a4b17b740 1 -- 172.21.15.5:0/688980303 shutdown_connections 2024-09-19T06:26:10.969 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:26:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:26:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:26:10.969 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:26:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.443+0000 7f7a4b17b740 1 -- 172.21.15.5:0/688980303 wait complete. 2024-09-19T06:26:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.443+0000 7f7a4b17b740 20 client.21024 trim_cache size 0 max 16384 2024-09-19T06:26:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.443+0000 7f7a48da2640 1 -- 172.21.15.5:0/3623871478 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7a2c0a3430 msgr2=0x7f7a2c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.443+0000 7f7a48da2640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7a2c0a3430 0x7f7a2c0a58f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7a3402a930 tx=0x7f7a340058d0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.443+0000 7f7a48da2640 1 -- 172.21.15.5:0/3623871478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a441027a0 msgr2=0x7f7a4419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.443+0000 7f7a48da2640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a441027a0 0x7f7a4419aad0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800e9c0 tx=0x7f7a3800ee80 comp rx=0 tx=0).stop 2024-09-19T06:26:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.444+0000 7f7a48da2640 1 -- 172.21.15.5:0/3623871478 shutdown_connections 2024-09-19T06:26:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.444+0000 7f7a48da2640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a44106f20 0x7f7a4419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.444+0000 7f7a48da2640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a441027a0 0x7f7a4419aad0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.444+0000 7f7a48da2640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7a2c0a3430 0x7f7a2c0a58f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.444+0000 7f7a48da2640 1 --2- 172.21.15.5:0/3623871478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a44101df0 0x7f7a4419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.444+0000 7f7a48da2640 1 -- 172.21.15.5:0/3623871478 >> 172.21.15.5:0/3623871478 conn(0x7f7a440fd8f0 msgr2=0x7f7a440fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.444+0000 7f7a48da2640 1 -- 172.21.15.5:0/3623871478 shutdown_connections 2024-09-19T06:26:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:10.444+0000 7f7a48da2640 1 -- 172.21.15.5:0/3623871478 wait complete. 2024-09-19T06:26:11.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 92832 8.57 MB/sec execute 455 sec latency 96.582 ms 2024-09-19T06:26:12.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:12 smithi157 ceph-mon[35346]: pgmap v723: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 186 op/s 2024-09-19T06:26:12.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:12 smithi005 ceph-mon[30955]: pgmap v723: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 186 op/s 2024-09-19T06:26:12.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:12 smithi049 ceph-mon[34640]: pgmap v723: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 186 op/s 2024-09-19T06:26:12.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 93005 8.58 MB/sec execute 456 sec latency 118.957 ms 2024-09-19T06:26:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.821+0000 7fc16a54b640 1 -- 172.21.15.5:0/1718525958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc164102460 msgr2=0x7fc16410e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.821+0000 7fc16a54b640 1 --2- 172.21.15.5:0/1718525958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc164102460 0x7fc16410e990 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009cd0 tx=0x7fc15c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.822+0000 7fc16a54b640 1 -- 172.21.15.5:0/1718525958 shutdown_connections 2024-09-19T06:26:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.822+0000 7fc16a54b640 1 --2- 172.21.15.5:0/1718525958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc164102460 0x7fc16410e990 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.822+0000 7fc16a54b640 1 --2- 172.21.15.5:0/1718525958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc164101ac0 0x7fc164101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.822+0000 7fc16a54b640 1 --2- 172.21.15.5:0/1718525958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164107ad0 0x7fc164107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.822+0000 7fc16a54b640 1 -- 172.21.15.5:0/1718525958 >> 172.21.15.5:0/1718525958 conn(0x7fc1640fd7a0 msgr2=0x7fc1640ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.822+0000 7fc16a54b640 1 -- 172.21.15.5:0/1718525958 shutdown_connections 2024-09-19T06:26:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.822+0000 7fc16a54b640 1 -- 172.21.15.5:0/1718525958 wait complete. 2024-09-19T06:26:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.823+0000 7fc16a54b640 1 Processor -- start 2024-09-19T06:26:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.823+0000 7fc16a54b640 1 -- start start 2024-09-19T06:26:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.823+0000 7fc16a54b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164101ac0 0x7fc1640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.823+0000 7fc16a54b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc164102460 0x7fc164074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.823+0000 7fc16a54b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc164107ad0 0x7fc164074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.823+0000 7fc16a54b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc164111e10 con 0x7fc164107ad0 2024-09-19T06:26:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.823+0000 7fc16a54b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc164111c90 con 0x7fc164102460 2024-09-19T06:26:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.823+0000 7fc16a54b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc164111f90 con 0x7fc164101ac0 2024-09-19T06:26:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.824+0000 7fc163fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164101ac0 0x7fc1640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.824+0000 7fc163fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164101ac0 0x7fc1640744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50248/0 (socket says 172.21.15.5:50248) 2024-09-19T06:26:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.824+0000 7fc163fff640 1 -- 172.21.15.5:0/1468680176 learned_addr learned my addr 172.21.15.5:0/1468680176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.824+0000 7fc168ae8640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc164107ad0 0x7fc164074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.824+0000 7fc1637fe640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc164102460 0x7fc164074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.824+0000 7fc163fff640 1 -- 172.21.15.5:0/1468680176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc164102460 msgr2=0x7fc164074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.824+0000 7fc163fff640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc164102460 0x7fc164074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.824+0000 7fc163fff640 1 -- 172.21.15.5:0/1468680176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc164107ad0 msgr2=0x7fc164074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.824+0000 7fc163fff640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc164107ad0 0x7fc164074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.824+0000 7fc163fff640 1 -- 172.21.15.5:0/1468680176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1641a6800 con 0x7fc164101ac0 2024-09-19T06:26:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.824+0000 7fc168ae8640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc164107ad0 0x7fc164074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.824+0000 7fc163fff640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164101ac0 0x7fc1640744f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc15400e9f0 tx=0x7fc15400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.825+0000 7fc1617fa640 1 -- 172.21.15.5:0/1468680176 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15400cd80 con 0x7fc164101ac0 2024-09-19T06:26:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.825+0000 7fc1617fa640 1 -- 172.21.15.5:0/1468680176 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1540054d0 con 0x7fc164101ac0 2024-09-19T06:26:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.825+0000 7fc1617fa640 1 -- 172.21.15.5:0/1468680176 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154010490 con 0x7fc164101ac0 2024-09-19T06:26:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.825+0000 7fc16a54b640 1 -- 172.21.15.5:0/1468680176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1641980b0 con 0x7fc164101ac0 2024-09-19T06:26:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.825+0000 7fc16a54b640 1 -- 172.21.15.5:0/1468680176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc164198600 con 0x7fc164101ac0 2024-09-19T06:26:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.828+0000 7fc1617fa640 1 -- 172.21.15.5:0/1468680176 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc154005020 con 0x7fc164101ac0 2024-09-19T06:26:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.828+0000 7fc16cf22740 1 Processor -- start 2024-09-19T06:26:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc16cf22740 1 -- start start 2024-09-19T06:26:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc1617fa640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc14c0a3430 0x7fc14c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc16cf22740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d1aaf0 0x55ab69d1aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc16cf22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1b410 0x55ab69d25770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc16cf22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d25cb0 0x55ab69d280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc16cf22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab69c7f2f0 con 0x55ab69d25cb0 2024-09-19T06:26:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc16cf22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ab69c27270 con 0x55ab69d1b410 2024-09-19T06:26:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc16cf22740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ab69c7b2b0 con 0x55ab69d1aaf0 2024-09-19T06:26:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc168ae8640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc14c0a3430 0x7fc14c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc168ae8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d25cb0 0x55ab69d280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc1617fa640 1 -- 172.21.15.5:0/1468680176 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fc1540cfff0 con 0x7fc164101ac0 2024-09-19T06:26:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc168ae8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d25cb0 0x55ab69d280a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52068/0 (socket says 172.21.15.5:52068) 2024-09-19T06:26:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc168ae8640 1 -- 172.21.15.5:0/1549573494 learned_addr learned my addr 172.21.15.5:0/1549573494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc1637fe640 1 --2- 172.21.15.5:0/1549573494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1b410 0x55ab69d25770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.829+0000 7fc163fff640 1 --2- 172.21.15.5:0/1549573494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d1aaf0 0x55ab69d1aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.830+0000 7fc168ae8640 1 -- 172.21.15.5:0/1549573494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d1aaf0 msgr2=0x55ab69d1aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.830+0000 7fc168ae8640 1 --2- 172.21.15.5:0/1549573494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d1aaf0 0x55ab69d1aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.830+0000 7fc168ae8640 1 -- 172.21.15.5:0/1549573494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1b410 msgr2=0x55ab69d25770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.830+0000 7fc168ae8640 1 --2- 172.21.15.5:0/1549573494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1b410 0x55ab69d25770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.830+0000 7fc168ae8640 1 -- 172.21.15.5:0/1549573494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab69c26340 con 0x55ab69d25cb0 2024-09-19T06:26:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.830+0000 7fc1637fe640 1 --2- 172.21.15.5:0/1549573494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1b410 0x55ab69d25770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.830+0000 7fc168ae8640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc14c0a3430 0x7fc14c0a58f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc15c02aed0 tx=0x7fc15c009930 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.830+0000 7fc168ae8640 1 --2- 172.21.15.5:0/1549573494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d25cb0 0x55ab69d280a0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc15c033040 tx=0x7fc15c050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769260e4ade6f311 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.830+0000 7fc14a7fc640 1 -- 172.21.15.5:0/1549573494 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c040c40 con 0x55ab69d25cb0 2024-09-19T06:26:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.830+0000 7fc14a7fc640 1 -- 172.21.15.5:0/1549573494 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc15c054070 con 0x55ab69d25cb0 2024-09-19T06:26:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.830+0000 7fc14a7fc640 1 -- 172.21.15.5:0/1549573494 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c030ed0 con 0x55ab69d25cb0 2024-09-19T06:26:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.831+0000 7fc16cf22740 1 -- 172.21.15.5:0/1549573494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d25cb0 msgr2=0x55ab69d280a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.831+0000 7fc16cf22740 1 --2- 172.21.15.5:0/1549573494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d25cb0 0x55ab69d280a0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc15c033040 tx=0x7fc15c050000 comp rx=0 tx=0).stop 2024-09-19T06:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.831+0000 7fc16cf22740 1 -- 172.21.15.5:0/1549573494 shutdown_connections 2024-09-19T06:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.831+0000 7fc16cf22740 1 --2- 172.21.15.5:0/1549573494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d25cb0 0x55ab69d280a0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.831+0000 7fc16cf22740 1 --2- 172.21.15.5:0/1549573494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1b410 0x55ab69d25770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.831+0000 7fc16cf22740 1 --2- 172.21.15.5:0/1549573494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d1aaf0 0x55ab69d1aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.831+0000 7fc16cf22740 1 -- 172.21.15.5:0/1549573494 >> 172.21.15.5:0/1549573494 conn(0x55ab69d17060 msgr2=0x55ab69d17440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.831+0000 7fc16cf22740 1 -- 172.21.15.5:0/1549573494 shutdown_connections 2024-09-19T06:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.831+0000 7fc16cf22740 1 -- 172.21.15.5:0/1549573494 wait complete. 2024-09-19T06:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.832+0000 7fc16cf22740 1 -- start start 2024-09-19T06:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.832+0000 7fc16cf22740 1 Processor -- start 2024-09-19T06:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.832+0000 7fc16cf22740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d1aaf0 0x55ab69d22ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.832+0000 7fc16cf22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.832+0000 7fc16cf22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d25cb0 0x55ab69d23720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.832+0000 7fc1637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc163fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d1aaf0 0x55ab69d22ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc163fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d1aaf0 0x55ab69d22ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50270/0 (socket says 172.21.15.5:50270) 2024-09-19T06:26:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc1637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d231e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52074/0 (socket says 172.21.15.5:52074) 2024-09-19T06:26:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc163fff640 1 -- 172.21.15.5:0/4224531696 learned_addr learned my addr 172.21.15.5:0/4224531696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc16cf22740 1 -- 172.21.15.5:0/4224531696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab69d28880 con 0x55ab69d1b410 2024-09-19T06:26:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc16cf22740 1 -- 172.21.15.5:0/4224531696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ab69c08290 con 0x55ab69d25cb0 2024-09-19T06:26:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc16cf22740 1 -- 172.21.15.5:0/4224531696 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ab69d28a00 con 0x55ab69d1aaf0 2024-09-19T06:26:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc16cf22740 10 client.0 _ll_register_callbacks cb 0x55ab69c39140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc168ae8640 1 --2- 172.21.15.5:0/4224531696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d25cb0 0x55ab69d23720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc163fff640 1 -- 172.21.15.5:0/4224531696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d25cb0 msgr2=0x55ab69d23720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc163fff640 1 --2- 172.21.15.5:0/4224531696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d25cb0 0x55ab69d23720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc163fff640 1 -- 172.21.15.5:0/4224531696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 msgr2=0x55ab69d231e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc163fff640 1 --2- 172.21.15.5:0/4224531696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d231e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc163fff640 1 -- 172.21.15.5:0/4224531696 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab69992210 con 0x55ab69d1aaf0 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc163fff640 1 --2- 172.21.15.5:0/4224531696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d1aaf0 0x55ab69d22ca0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc15409c220 tx=0x7fc154064fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5847bf10bb844f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc1437fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc1437fe640 1 -- 172.21.15.5:0/4224531696 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15409c990 con 0x55ab69d1aaf0 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc16cf22740 1 -- 172.21.15.5:0/4224531696 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc15c054070 con 0x55ab69d1aaf0 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.833+0000 7fc1437fe640 1 -- 172.21.15.5:0/4224531696 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fc1540046c0 con 0x55ab69d1aaf0 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.834+0000 7fc1437fe640 1 -- 172.21.15.5:0/4224531696 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc15409c290 con 0x55ab69d1aaf0 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.834+0000 7fc16cf22740 10 client.30334 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.834+0000 7fc16cf22740 1 -- 172.21.15.5:0/4224531696 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55ab69a5e620 con 0x55ab69d1aaf0 2024-09-19T06:26:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.834+0000 7fc1437fe640 1 -- 172.21.15.5:0/4224531696 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc15c054070 con 0x55ab69d1aaf0 2024-09-19T06:26:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.834+0000 7fc1437fe640 1 -- 172.21.15.5:0/4224531696 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc1540cd980 con 0x55ab69d1aaf0 2024-09-19T06:26:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.834+0000 7fc16cf22740 10 client.30334 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.834+0000 7fc16cf22740 10 client.30334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.835+0000 7fc16cf22740 20 client.30334 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.835+0000 7fc16cf22740 1 --2- 172.21.15.5:0/4224531696 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab69d35f70 0x55ab69d38360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.835+0000 7fc16cf22740 4 client.30334 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:26:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.835+0000 7fc16cf22740 1 -- 172.21.15.5:0/4224531696 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab69c8ca20 con 0x55ab69d35f70 2024-09-19T06:26:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.835+0000 7fc1637fe640 1 --2- 172.21.15.5:0/4224531696 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab69d35f70 0x55ab69d38360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.836+0000 7fc1637fe640 1 --2- 172.21.15.5:0/4224531696 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab69d35f70 0x55ab69d38360 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d73515605f5eb50 server_cookie=61cdbfce0ea1881d in_seq=0 out_seq=0 2024-09-19T06:26:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.837+0000 7fc1437fe640 10 client.30334 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.837+0000 7fc1437fe640 1 -- 172.21.15.5:0/4224531696 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab69c8ca20 con 0x55ab69d35f70 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.837+0000 7fc1437fe640 10 client.30334 handle_command_reply: tid=0 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.837+0000 7fc16cf22740 1 client.30334 shutdown 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.837+0000 7fc16cf22740 1 -- 172.21.15.5:0/4224531696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d1aaf0 msgr2=0x55ab69d22ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.837+0000 7fc16cf22740 1 --2- 172.21.15.5:0/4224531696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d1aaf0 0x55ab69d22ca0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc15409c220 tx=0x7fc154064fb0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.838+0000 7fc16cf22740 1 -- 172.21.15.5:0/4224531696 shutdown_connections 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.838+0000 7fc16cf22740 1 --2- 172.21.15.5:0/4224531696 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab69d35f70 0x55ab69d38360 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.838+0000 7fc1437fe640 0 client.30334 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.838+0000 7fc16cf22740 1 --2- 172.21.15.5:0/4224531696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d25cb0 0x55ab69d23720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.838+0000 7fc16cf22740 1 --2- 172.21.15.5:0/4224531696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.838+0000 7fc16cf22740 1 --2- 172.21.15.5:0/4224531696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d1aaf0 0x55ab69d22ca0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.838+0000 7fc16cf22740 1 -- 172.21.15.5:0/4224531696 >> 172.21.15.5:0/4224531696 conn(0x55ab69d17060 msgr2=0x55ab69d1cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.838+0000 7fc16cf22740 1 -- 172.21.15.5:0/4224531696 shutdown_connections 2024-09-19T06:26:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.838+0000 7fc16cf22740 1 -- 172.21.15.5:0/4224531696 wait complete. 2024-09-19T06:26:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.838+0000 7fc16cf22740 20 client.30334 trim_cache size 0 max 16384 2024-09-19T06:26:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.866+0000 7fc16cf22740 1 Processor -- start 2024-09-19T06:26:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.866+0000 7fc16cf22740 1 -- start start 2024-09-19T06:26:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.867+0000 7fc16cf22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 0x55ab69d21bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.867+0000 7fc16cf22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d23550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.867+0000 7fc16cf22740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d25cb0 0x55ab69d3da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.867+0000 7fc16cf22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1641ab590 con 0x55ab69d1b410 2024-09-19T06:26:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.867+0000 7fc1637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d23550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.867+0000 7fc163fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 0x55ab69d21bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.867+0000 7fc163fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 0x55ab69d21bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36258/0 (socket says 172.21.15.5:36258) 2024-09-19T06:26:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.867+0000 7fc1637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d23550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52084/0 (socket says 172.21.15.5:52084) 2024-09-19T06:26:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.867+0000 7fc163fff640 1 -- 172.21.15.5:0/1666292670 learned_addr learned my addr 172.21.15.5:0/1666292670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.867+0000 7fc168ae8640 1 --2- 172.21.15.5:0/1666292670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d25cb0 0x55ab69d3da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.867+0000 7fc16cf22740 1 -- 172.21.15.5:0/1666292670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1641ad0d0 con 0x55ab69d1aaf0 2024-09-19T06:26:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.867+0000 7fc16cf22740 1 -- 172.21.15.5:0/1666292670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1641abc60 con 0x55ab69d25cb0 2024-09-19T06:26:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.868+0000 7fc163fff640 1 -- 172.21.15.5:0/1666292670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d25cb0 msgr2=0x55ab69d3da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.868+0000 7fc163fff640 1 --2- 172.21.15.5:0/1666292670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d25cb0 0x55ab69d3da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.868+0000 7fc163fff640 1 -- 172.21.15.5:0/1666292670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 msgr2=0x55ab69d23550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.868+0000 7fc163fff640 1 --2- 172.21.15.5:0/1666292670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d23550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.868+0000 7fc163fff640 1 -- 172.21.15.5:0/1666292670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab69d17cf0 con 0x55ab69d1aaf0 2024-09-19T06:26:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.868+0000 7fc1637fe640 1 --2- 172.21.15.5:0/1666292670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d23550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.868+0000 7fc163fff640 1 --2- 172.21.15.5:0/1666292670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 0x55ab69d21bc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc154098c60 tx=0x7fc154009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f289e0de29f06517 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.868+0000 7fc149ffb640 1 -- 172.21.15.5:0/1666292670 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15409c990 con 0x55ab69d1aaf0 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.868+0000 7fc149ffb640 1 -- 172.21.15.5:0/1666292670 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1540cde50 con 0x55ab69d1aaf0 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.868+0000 7fc149ffb640 1 -- 172.21.15.5:0/1666292670 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154065630 con 0x55ab69d1aaf0 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.869+0000 7fc16cf22740 1 -- 172.21.15.5:0/1666292670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 msgr2=0x55ab69d21bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.869+0000 7fc16cf22740 1 --2- 172.21.15.5:0/1666292670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 0x55ab69d21bc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc154098c60 tx=0x7fc154009310 comp rx=0 tx=0).stop 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.869+0000 7fc16cf22740 1 -- 172.21.15.5:0/1666292670 shutdown_connections 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.869+0000 7fc16cf22740 1 --2- 172.21.15.5:0/1666292670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d25cb0 0x55ab69d3da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.869+0000 7fc16cf22740 1 --2- 172.21.15.5:0/1666292670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d23550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.869+0000 7fc16cf22740 1 --2- 172.21.15.5:0/1666292670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 0x55ab69d21bc0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.869+0000 7fc16cf22740 1 -- 172.21.15.5:0/1666292670 >> 172.21.15.5:0/1666292670 conn(0x55ab69d17060 msgr2=0x55ab69d24d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.869+0000 7fc16cf22740 1 -- 172.21.15.5:0/1666292670 shutdown_connections 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.869+0000 7fc16cf22740 1 -- 172.21.15.5:0/1666292670 wait complete. 2024-09-19T06:26:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.870+0000 7fc16cf22740 1 -- start start 2024-09-19T06:26:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.870+0000 7fc16cf22740 1 Processor -- start 2024-09-19T06:26:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.870+0000 7fc16cf22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 0x55ab69d298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.870+0000 7fc16cf22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d29e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.870+0000 7fc16cf22740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d25cb0 0x55ab69d2a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.870+0000 7fc16cf22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1641aca00 con 0x55ab69d1b410 2024-09-19T06:26:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.870+0000 7fc16cf22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1641ac330 con 0x55ab69d1aaf0 2024-09-19T06:26:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.870+0000 7fc16cf22740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1641a9590 con 0x55ab69d25cb0 2024-09-19T06:26:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.870+0000 7fc16cf22740 10 client.0 _ll_register_callbacks cb 0x7fc154004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.871+0000 7fc168ae8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d25cb0 0x55ab69d2a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.871+0000 7fc163fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 0x55ab69d298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.871+0000 7fc168ae8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d25cb0 0x55ab69d2a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50280/0 (socket says 172.21.15.5:50280) 2024-09-19T06:26:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.871+0000 7fc163fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 0x55ab69d298f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36274/0 (socket says 172.21.15.5:36274) 2024-09-19T06:26:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.871+0000 7fc168ae8640 1 -- 172.21.15.5:0/2738818047 learned_addr learned my addr 172.21.15.5:0/2738818047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.871+0000 7fc1637fe640 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d29e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.871+0000 7fc163fff640 1 -- 172.21.15.5:0/2738818047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d25cb0 msgr2=0x55ab69d2a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.871+0000 7fc163fff640 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d25cb0 0x55ab69d2a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.871+0000 7fc163fff640 1 -- 172.21.15.5:0/2738818047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 msgr2=0x55ab69d29e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.871+0000 7fc163fff640 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d29e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.871+0000 7fc163fff640 1 -- 172.21.15.5:0/2738818047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab69d23a90 con 0x55ab69d1aaf0 2024-09-19T06:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.872+0000 7fc1637fe640 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d29e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.872+0000 7fc168ae8640 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d25cb0 0x55ab69d2a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.872+0000 7fc163fff640 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 0x55ab69d298f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc15409be70 tx=0x7fc154009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=badd41e8bcb21538 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.872+0000 7fc1437fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.872+0000 7fc1437fe640 1 -- 172.21.15.5:0/2738818047 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15409c990 con 0x55ab69d1aaf0 2024-09-19T06:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.872+0000 7fc1437fe640 1 -- 172.21.15.5:0/2738818047 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fc1540046c0 con 0x55ab69d1aaf0 2024-09-19T06:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.872+0000 7fc16cf22740 1 -- 172.21.15.5:0/2738818047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1540cde50 con 0x55ab69d1aaf0 2024-09-19T06:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.872+0000 7fc1437fe640 1 -- 172.21.15.5:0/2738818047 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc1540a0640 con 0x55ab69d1aaf0 2024-09-19T06:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.872+0000 7fc16cf22740 10 client.30419 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.872+0000 7fc16cf22740 1 -- 172.21.15.5:0/2738818047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55ab69da0a20 con 0x55ab69d1aaf0 2024-09-19T06:26:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.873+0000 7fc1437fe640 1 -- 172.21.15.5:0/2738818047 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1540cde50 con 0x55ab69d1aaf0 2024-09-19T06:26:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.873+0000 7fc1437fe640 1 -- 172.21.15.5:0/2738818047 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc1540a9070 con 0x55ab69d1aaf0 2024-09-19T06:26:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.873+0000 7fc16cf22740 10 client.30419 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.873+0000 7fc16cf22740 10 client.30419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.873+0000 7fc16cf22740 20 client.30419 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.873+0000 7fc16cf22740 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab69d35f70 0x55ab69da2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.873+0000 7fc16cf22740 4 client.30419 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:26:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.873+0000 7fc16cf22740 1 -- 172.21.15.5:0/2738818047 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab69c86d70 con 0x55ab69d35f70 2024-09-19T06:26:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.873+0000 7fc1637fe640 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab69d35f70 0x55ab69da2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.874+0000 7fc1437fe640 10 client.30419 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.874+0000 7fc1637fe640 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab69d35f70 0x55ab69da2eb0 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67207819ecd9b78b server_cookie=bc39d5ce81234774 in_seq=0 out_seq=0 2024-09-19T06:26:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.967+0000 7fc1437fe640 1 -- 172.21.15.5:0/2738818047 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab69c86d70 con 0x55ab69d35f70 2024-09-19T06:26:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.967+0000 7fc1437fe640 10 client.30419 handle_command_reply: tid=0 2024-09-19T06:26:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc16cf22740 1 client.30419 shutdown 2024-09-19T06:26:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc16cf22740 1 -- 172.21.15.5:0/2738818047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 msgr2=0x55ab69d298f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc16cf22740 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 0x55ab69d298f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc15409be70 tx=0x7fc154009720 comp rx=0 tx=0).stop 2024-09-19T06:26:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc16cf22740 1 -- 172.21.15.5:0/2738818047 shutdown_connections 2024-09-19T06:26:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc16cf22740 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ab69d35f70 0x55ab69da2eb0 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc16cf22740 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab69d25cb0 0x55ab69d2a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc16cf22740 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab69d1b410 0x55ab69d29e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc1437fe640 0 client.30419 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc16cf22740 1 --2- 172.21.15.5:0/2738818047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab69d1aaf0 0x55ab69d298f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc16cf22740 1 -- 172.21.15.5:0/2738818047 >> 172.21.15.5:0/2738818047 conn(0x55ab69d17060 msgr2=0x55ab69d1c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc16cf22740 1 -- 172.21.15.5:0/2738818047 shutdown_connections 2024-09-19T06:26:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc16cf22740 1 -- 172.21.15.5:0/2738818047 wait complete. 2024-09-19T06:26:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.968+0000 7fc16cf22740 20 client.30419 trim_cache size 0 max 16384 2024-09-19T06:26:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:26:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:26:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:26:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:26:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.969+0000 7fc16a54b640 1 -- 172.21.15.5:0/1468680176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc14c0a3430 msgr2=0x7fc14c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.969+0000 7fc16a54b640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc14c0a3430 0x7fc14c0a58f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc15c02aed0 tx=0x7fc15c009930 comp rx=0 tx=0).stop 2024-09-19T06:26:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.969+0000 7fc16a54b640 1 -- 172.21.15.5:0/1468680176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164101ac0 msgr2=0x7fc1640744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.969+0000 7fc16a54b640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164101ac0 0x7fc1640744f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc15400e9f0 tx=0x7fc15400edf0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.969+0000 7fc16a54b640 1 -- 172.21.15.5:0/1468680176 shutdown_connections 2024-09-19T06:26:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.969+0000 7fc16a54b640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc164107ad0 0x7fc164074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.969+0000 7fc16a54b640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc164102460 0x7fc164074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.969+0000 7fc16a54b640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc14c0a3430 0x7fc14c0a58f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.969+0000 7fc16a54b640 1 --2- 172.21.15.5:0/1468680176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164101ac0 0x7fc1640744f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.969+0000 7fc16a54b640 1 -- 172.21.15.5:0/1468680176 >> 172.21.15.5:0/1468680176 conn(0x7fc1640fd7a0 msgr2=0x7fc1640fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.969+0000 7fc16a54b640 1 -- 172.21.15.5:0/1468680176 shutdown_connections 2024-09-19T06:26:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:12.969+0000 7fc16a54b640 1 -- 172.21.15.5:0/1468680176 wait complete. 2024-09-19T06:26:13.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 93155 8.58 MB/sec execute 457 sec latency 109.796 ms 2024-09-19T06:26:14.063 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:26:14.064 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:26:14.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:14 smithi157 ceph-mon[35346]: pgmap v724: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.7 MiB/s wr, 349 op/s 2024-09-19T06:26:14.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:14 smithi005 ceph-mon[30955]: pgmap v724: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.7 MiB/s wr, 349 op/s 2024-09-19T06:26:14.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:14 smithi049 ceph-mon[34640]: pgmap v724: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.7 MiB/s wr, 349 op/s 2024-09-19T06:26:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:26:14.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 93307 8.58 MB/sec execute 458 sec latency 115.705 ms 2024-09-19T06:26:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.124+0000 7f1ac40a1640 1 -- 172.21.15.5:0/209216831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc102570 msgr2=0x7f1abc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.124+0000 7f1ac40a1640 1 --2- 172.21.15.5:0/209216831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc102570 0x7f1abc1029b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009930 tx=0x7f1aac02a4d0 comp rx=0 tx=0).stop 2024-09-19T06:26:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.124+0000 7f1ac40a1640 1 -- 172.21.15.5:0/209216831 shutdown_connections 2024-09-19T06:26:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.124+0000 7f1ac40a1640 1 --2- 172.21.15.5:0/209216831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1abc102ef0 0x7f1abc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.124+0000 7f1ac40a1640 1 --2- 172.21.15.5:0/209216831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc102570 0x7f1abc1029b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.124+0000 7f1ac40a1640 1 --2- 172.21.15.5:0/209216831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc100e20 0x7f1abc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.124+0000 7f1ac40a1640 1 -- 172.21.15.5:0/209216831 >> 172.21.15.5:0/209216831 conn(0x7f1abc0fc7d0 msgr2=0x7f1abc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.125+0000 7f1ac40a1640 1 -- 172.21.15.5:0/209216831 shutdown_connections 2024-09-19T06:26:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.125+0000 7f1ac40a1640 1 -- 172.21.15.5:0/209216831 wait complete. 2024-09-19T06:26:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.126+0000 7f1ac40a1640 1 Processor -- start 2024-09-19T06:26:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.126+0000 7f1ac40a1640 1 -- start start 2024-09-19T06:26:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.126+0000 7f1ac40a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc100e20 0x7f1abc198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.126+0000 7f1ac40a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc102570 0x7f1abc1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.126+0000 7f1ac40a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1abc102ef0 0x7f1abc19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.126+0000 7f1ac40a1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc110e00 con 0x7f1abc102570 2024-09-19T06:26:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.126+0000 7f1ac40a1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc110c80 con 0x7f1abc102ef0 2024-09-19T06:26:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.126+0000 7f1ac40a1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc110f80 con 0x7f1abc100e20 2024-09-19T06:26:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.126+0000 7f1ac163c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc102570 0x7f1abc1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.127+0000 7f1ac163c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc102570 0x7f1abc1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38770/0 (socket says 172.21.15.5:38770) 2024-09-19T06:26:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.127+0000 7f1ac163c640 1 -- 172.21.15.5:0/595053865 learned_addr learned my addr 172.21.15.5:0/595053865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.127+0000 7f1ac1e3d640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc100e20 0x7f1abc198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.127+0000 7f1ac263e640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1abc102ef0 0x7f1abc19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.127+0000 7f1ac163c640 1 -- 172.21.15.5:0/595053865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc100e20 msgr2=0x7f1abc198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.127+0000 7f1ac163c640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc100e20 0x7f1abc198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.127+0000 7f1ac163c640 1 -- 172.21.15.5:0/595053865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1abc102ef0 msgr2=0x7f1abc19cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.127+0000 7f1ac163c640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1abc102ef0 0x7f1abc19cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.127+0000 7f1ac163c640 1 -- 172.21.15.5:0/595053865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc19d220 con 0x7f1abc102570 2024-09-19T06:26:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.127+0000 7f1ac163c640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc102570 0x7f1abc1987b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009900 tx=0x7f1aac030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.128+0000 7f1ab2ffd640 1 -- 172.21.15.5:0/595053865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac03f070 con 0x7f1abc102570 2024-09-19T06:26:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.128+0000 7f1ab2ffd640 1 -- 172.21.15.5:0/595053865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1aac004460 con 0x7f1abc102570 2024-09-19T06:26:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.128+0000 7f1ab2ffd640 1 -- 172.21.15.5:0/595053865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac035030 con 0x7f1abc102570 2024-09-19T06:26:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.128+0000 7f1ac40a1640 1 -- 172.21.15.5:0/595053865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc19d4b0 con 0x7f1abc102570 2024-09-19T06:26:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.128+0000 7f1ac40a1640 1 -- 172.21.15.5:0/595053865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc107a10 con 0x7f1abc102570 2024-09-19T06:26:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.129+0000 7f1ac40a1640 1 -- 172.21.15.5:0/595053865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1abc1021b0 con 0x7f1abc102570 2024-09-19T06:26:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.134+0000 7f1ab2ffd640 1 -- 172.21.15.5:0/595053865 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1aac042070 con 0x7f1abc102570 2024-09-19T06:26:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.135+0000 7f1ab2ffd640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1aa80a34b0 0x7f1aa80a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.136+0000 7f1ab2ffd640 1 -- 172.21.15.5:0/595053865 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f1aac0f0a10 con 0x7f1abc102570 2024-09-19T06:26:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.137+0000 7f1ac1e3d640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1aa80a34b0 0x7f1aa80a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.138+0000 7f1ac1e3d640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1aa80a34b0 0x7f1aa80a5970 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009b60 tx=0x7f1ab8005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.139+0000 7f1ab2ffd640 1 -- 172.21.15.5:0/595053865 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1aac049030 con 0x7f1abc102570 2024-09-19T06:26:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.527+0000 7f1ac40a1640 1 -- 172.21.15.5:0/595053865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1abc1995d0 con 0x7f1abc102570 2024-09-19T06:26:15.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.528+0000 7f1ab2ffd640 1 -- 172.21.15.5:0/595053865 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+11959 (secure 0 0 0) 0x7f1abc1995d0 con 0x7f1abc102570 2024-09-19T06:26:15.530 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:26:15.530 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":62,"btime":"2024-09-19T06:25:41:566205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":62,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:25:40.570592+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:26:15.530 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 62 2024-09-19T06:26:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.531+0000 7f1ac40a1640 1 -- 172.21.15.5:0/595053865 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1aa80a34b0 msgr2=0x7f1aa80a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.531+0000 7f1ac40a1640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1aa80a34b0 0x7f1aa80a5970 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009b60 tx=0x7f1ab8005f90 comp rx=0 tx=0).stop 2024-09-19T06:26:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.531+0000 7f1ac40a1640 1 -- 172.21.15.5:0/595053865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc102570 msgr2=0x7f1abc1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.531+0000 7f1ac40a1640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc102570 0x7f1abc1987b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009900 tx=0x7f1aac030650 comp rx=0 tx=0).stop 2024-09-19T06:26:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.531+0000 7f1ac40a1640 1 -- 172.21.15.5:0/595053865 shutdown_connections 2024-09-19T06:26:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.531+0000 7f1ac40a1640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1abc102ef0 0x7f1abc19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.531+0000 7f1ac40a1640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc102570 0x7f1abc1987b0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.531+0000 7f1ac40a1640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1aa80a34b0 0x7f1aa80a5970 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.531+0000 7f1ac40a1640 1 --2- 172.21.15.5:0/595053865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc100e20 0x7f1abc198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.531+0000 7f1ac40a1640 1 -- 172.21.15.5:0/595053865 >> 172.21.15.5:0/595053865 conn(0x7f1abc0fc7d0 msgr2=0x7f1abc069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.531+0000 7f1ac40a1640 1 -- 172.21.15.5:0/595053865 shutdown_connections 2024-09-19T06:26:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:15.531+0000 7f1ac40a1640 1 -- 172.21.15.5:0/595053865 wait complete. 2024-09-19T06:26:15.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 93422 8.57 MB/sec execute 459 sec latency 74.780 ms 2024-09-19T06:26:16.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:16 smithi157 ceph-mon[35346]: pgmap v725: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 291 op/s 2024-09-19T06:26:16.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:16 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/595053865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:26:16.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:16 smithi005 ceph-mon[30955]: pgmap v725: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 291 op/s 2024-09-19T06:26:16.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:16 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/595053865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:26:16.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:16 smithi049 ceph-mon[34640]: pgmap v725: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 291 op/s 2024-09-19T06:26:16.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:16 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/595053865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:26:16.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 93555 8.57 MB/sec execute 460 sec latency 112.320 ms 2024-09-19T06:26:17.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 93698 8.57 MB/sec execute 461 sec latency 158.362 ms 2024-09-19T06:26:17.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:26:18.067 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:26:18.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:18 smithi157 ceph-mon[35346]: pgmap v726: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 293 op/s 2024-09-19T06:26:18.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:18 smithi005 ceph-mon[30955]: pgmap v726: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 293 op/s 2024-09-19T06:26:18.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:18 smithi049 ceph-mon[34640]: pgmap v726: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 293 op/s 2024-09-19T06:26:18.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 93837 8.57 MB/sec execute 462 sec latency 86.480 ms 2024-09-19T06:26:18.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.634+0000 7fa803cad640 1 -- 172.21.15.5:0/884265970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc102790 msgr2=0x7fa7fc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:18.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.634+0000 7fa803cad640 1 --2- 172.21.15.5:0/884265970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc102790 0x7fa7fc1067e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009860 tx=0x7fa7ec02a340 comp rx=0 tx=0).stop 2024-09-19T06:26:18.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.634+0000 7fa803cad640 1 -- 172.21.15.5:0/884265970 shutdown_connections 2024-09-19T06:26:18.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.634+0000 7fa803cad640 1 --2- 172.21.15.5:0/884265970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7fc106f10 0x7fa7fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.634+0000 7fa803cad640 1 --2- 172.21.15.5:0/884265970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc102790 0x7fa7fc1067e0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.634+0000 7fa803cad640 1 --2- 172.21.15.5:0/884265970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc101de0 0x7fa7fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.634+0000 7fa803cad640 1 -- 172.21.15.5:0/884265970 >> 172.21.15.5:0/884265970 conn(0x7fa7fc0fd8c0 msgr2=0x7fa7fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:18.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.634+0000 7fa803cad640 1 -- 172.21.15.5:0/884265970 shutdown_connections 2024-09-19T06:26:18.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.634+0000 7fa803cad640 1 -- 172.21.15.5:0/884265970 wait complete. 2024-09-19T06:26:18.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.635+0000 7fa803cad640 1 Processor -- start 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.635+0000 7fa803cad640 1 -- start start 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.635+0000 7fa803cad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc101de0 0x7fa7fc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.635+0000 7fa803cad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc102790 0x7fa7fc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.635+0000 7fa803cad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7fc106f10 0x7fa7fc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.635+0000 7fa803cad640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc111de0 con 0x7fa7fc101de0 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.635+0000 7fa803cad640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc111c60 con 0x7fa7fc106f10 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.635+0000 7fa803cad640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc111f60 con 0x7fa7fc102790 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.636+0000 7fa801248640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc102790 0x7fa7fc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.636+0000 7fa801a49640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc101de0 0x7fa7fc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.636+0000 7fa801248640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc102790 0x7fa7fc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50914/0 (socket says 172.21.15.5:50914) 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.636+0000 7fa801248640 1 -- 172.21.15.5:0/1265896676 learned_addr learned my addr 172.21.15.5:0/1265896676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.636+0000 7fa80224a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7fc106f10 0x7fa7fc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.636+0000 7fa801a49640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc101de0 0x7fa7fc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38786/0 (socket says 172.21.15.5:38786) 2024-09-19T06:26:18.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.636+0000 7fa801248640 1 -- 172.21.15.5:0/1265896676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7fc106f10 msgr2=0x7fa7fc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:18.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.636+0000 7fa801248640 1 --2- 172.21.15.5:0/1265896676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7fc106f10 0x7fa7fc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.636+0000 7fa801248640 1 -- 172.21.15.5:0/1265896676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc101de0 msgr2=0x7fa7fc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:18.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.636+0000 7fa801248640 1 --2- 172.21.15.5:0/1265896676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc101de0 0x7fa7fc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.636+0000 7fa801248640 1 -- 172.21.15.5:0/1265896676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc1a6800 con 0x7fa7fc102790 2024-09-19T06:26:18.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.636+0000 7fa801248640 1 --2- 172.21.15.5:0/1265896676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc102790 0x7fa7fc074a30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0303f0 tx=0x7fa7ec030e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:18.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.637+0000 7fa7f2ffd640 1 -- 172.21.15.5:0/1265896676 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec037050 con 0x7fa7fc102790 2024-09-19T06:26:18.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.637+0000 7fa7f2ffd640 1 -- 172.21.15.5:0/1265896676 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7ec043070 con 0x7fa7fc102790 2024-09-19T06:26:18.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.637+0000 7fa7f2ffd640 1 -- 172.21.15.5:0/1265896676 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec003ae0 con 0x7fa7fc102790 2024-09-19T06:26:18.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.637+0000 7fa803cad640 1 -- 172.21.15.5:0/1265896676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc1a6a90 con 0x7fa7fc102790 2024-09-19T06:26:18.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.637+0000 7fa803cad640 1 -- 172.21.15.5:0/1265896676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc1a6f20 con 0x7fa7fc102790 2024-09-19T06:26:18.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.640+0000 7fa7f2ffd640 1 -- 172.21.15.5:0/1265896676 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa7ec003c80 con 0x7fa7fc102790 2024-09-19T06:26:18.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.641+0000 7fa806086740 1 Processor -- start 2024-09-19T06:26:18.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa806086740 1 -- start start 2024-09-19T06:26:18.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa7f2ffd640 1 --2- 172.21.15.5:0/1265896676 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa7e80a3430 0x7fa7e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa806086740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa7f2ffd640 1 -- 172.21.15.5:0/1265896676 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fa7ec0efe50 con 0x7fa7fc102790 2024-09-19T06:26:18.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa806086740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a6b5d0 0x55aab7a6ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa806086740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a7eb80 0x55aab7a80f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa806086740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aab79d12f0 con 0x55aab7a6b5d0 2024-09-19T06:26:18.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa806086740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aab7978cf0 con 0x55aab7a7eb80 2024-09-19T06:26:18.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa806086740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55aab79cd2b0 con 0x55aab7a6cb40 2024-09-19T06:26:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa801a49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa801a49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a6b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50916/0 (socket says 172.21.15.5:50916) 2024-09-19T06:26:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa801a49640 1 -- 172.21.15.5:0/2775835527 learned_addr learned my addr 172.21.15.5:0/2775835527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.642+0000 7fa80224a640 1 --2- 172.21.15.5:0/1265896676 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa7e80a3430 0x7fa7e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.643+0000 7fa801a49640 1 -- 172.21.15.5:0/2775835527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a7eb80 msgr2=0x55aab7a80f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:26:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.643+0000 7fa801a49640 1 --2- 172.21.15.5:0/2775835527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a7eb80 0x55aab7a80f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.643+0000 7fa801a49640 1 -- 172.21.15.5:0/2775835527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a6b5d0 msgr2=0x55aab7a6ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.643+0000 7fa801a49640 1 --2- 172.21.15.5:0/2775835527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a6b5d0 0x55aab7a6ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.643+0000 7fa801a49640 1 -- 172.21.15.5:0/2775835527 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aab795d3a0 con 0x55aab7a6cb40 2024-09-19T06:26:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.643+0000 7fa801a49640 1 --2- 172.21.15.5:0/2775835527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a6b090 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009cd0 tx=0x7fa7f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5c9a71abcd069d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.643+0000 7fa80224a640 1 --2- 172.21.15.5:0/1265896676 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa7e80a3430 0x7fa7e80a58f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4005fd0 tx=0x7fa7f4009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:18.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.643+0000 7fa7e7fff640 1 -- 172.21.15.5:0/2775835527 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f801e070 con 0x55aab7a6cb40 2024-09-19T06:26:18.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.643+0000 7fa7e7fff640 1 -- 172.21.15.5:0/2775835527 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7f8019ae0 con 0x55aab7a6cb40 2024-09-19T06:26:18.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.643+0000 7fa7e7fff640 1 -- 172.21.15.5:0/2775835527 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f800e9b0 con 0x55aab7a6cb40 2024-09-19T06:26:18.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.645+0000 7fa806086740 1 -- 172.21.15.5:0/2775835527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 msgr2=0x55aab7a6b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:18.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.645+0000 7fa806086740 1 --2- 172.21.15.5:0/2775835527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a6b090 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009cd0 tx=0x7fa7f8011810 comp rx=0 tx=0).stop 2024-09-19T06:26:18.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.645+0000 7fa806086740 1 -- 172.21.15.5:0/2775835527 shutdown_connections 2024-09-19T06:26:18.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.645+0000 7fa806086740 1 --2- 172.21.15.5:0/2775835527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a7eb80 0x55aab7a80f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.645+0000 7fa806086740 1 --2- 172.21.15.5:0/2775835527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a6b5d0 0x55aab7a6ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.645+0000 7fa806086740 1 --2- 172.21.15.5:0/2775835527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a6b090 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.645+0000 7fa806086740 1 -- 172.21.15.5:0/2775835527 >> 172.21.15.5:0/2775835527 conn(0x55aab7a34310 msgr2=0x55aab7a67c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:18.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.645+0000 7fa806086740 1 -- 172.21.15.5:0/2775835527 shutdown_connections 2024-09-19T06:26:18.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.645+0000 7fa806086740 1 -- 172.21.15.5:0/2775835527 wait complete. 2024-09-19T06:26:18.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.646+0000 7fa806086740 1 -- start start 2024-09-19T06:26:18.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.646+0000 7fa806086740 1 Processor -- start 2024-09-19T06:26:18.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.646+0000 7fa806086740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 0x55aab7a7e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.646+0000 7fa806086740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a7b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.646+0000 7fa806086740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a7eb80 0x55aab7a7c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.646+0000 7fa806086740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aab7a81880 con 0x55aab7a7eb80 2024-09-19T06:26:18.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.646+0000 7fa806086740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aab798c060 con 0x55aab7a6b5d0 2024-09-19T06:26:18.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.646+0000 7fa806086740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55aab7a81a00 con 0x55aab7a6cb40 2024-09-19T06:26:18.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa801a49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 0x55aab7a7e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa801a49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 0x55aab7a7e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48124/0 (socket says 172.21.15.5:48124) 2024-09-19T06:26:18.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa801a49640 1 -- 172.21.15.5:0/353923325 learned_addr learned my addr 172.21.15.5:0/353923325 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:18.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa806086740 10 client.0 _ll_register_callbacks cb 0x55aab76df8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:18.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa80224a640 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a7eb80 0x55aab7a7c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa801248640 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a7b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa801a49640 1 -- 172.21.15.5:0/353923325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 msgr2=0x55aab7a7b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:18.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa801a49640 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a7b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa801a49640 1 -- 172.21.15.5:0/353923325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a7eb80 msgr2=0x55aab7a7c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:18.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa801a49640 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a7eb80 0x55aab7a7c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa801a49640 1 -- 172.21.15.5:0/353923325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aab77423c0 con 0x55aab7a6b5d0 2024-09-19T06:26:18.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa801a49640 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 0x55aab7a7e620 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800e590 tx=0x7fa7f8005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55069a98f506590a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.647+0000 7fa7e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:26:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.648+0000 7fa7e57fa640 1 -- 172.21.15.5:0/353923325 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f801e050 con 0x55aab7a6b5d0 2024-09-19T06:26:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.648+0000 7fa806086740 1 -- 172.21.15.5:0/353923325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f8019ae0 con 0x55aab7a6b5d0 2024-09-19T06:26:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.648+0000 7fa7e57fa640 1 -- 172.21.15.5:0/353923325 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fa7f800e710 con 0x55aab7a6b5d0 2024-09-19T06:26:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.648+0000 7fa7e57fa640 1 -- 172.21.15.5:0/353923325 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fa7f8033050 con 0x55aab7a6b5d0 2024-09-19T06:26:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.648+0000 7fa806086740 10 client.30434 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.648+0000 7fa806086740 1 -- 172.21.15.5:0/353923325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55aab7a8d3b0 con 0x55aab7a6b5d0 2024-09-19T06:26:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.648+0000 7fa7e57fa640 1 -- 172.21.15.5:0/353923325 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7f8019ae0 con 0x55aab7a6b5d0 2024-09-19T06:26:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.648+0000 7fa7e57fa640 1 -- 172.21.15.5:0/353923325 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fa7f8004850 con 0x55aab7a6b5d0 2024-09-19T06:26:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.649+0000 7fa806086740 10 client.30434 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.649+0000 7fa806086740 10 client.30434 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:26:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.649+0000 7fa806086740 20 client.30434 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.649+0000 7fa806086740 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55aab7a8e700 0x55aab7a90af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.649+0000 7fa806086740 4 client.30434 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:26:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.649+0000 7fa806086740 1 -- 172.21.15.5:0/353923325 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aab79de860 con 0x55aab7a8e700 2024-09-19T06:26:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.649+0000 7fa801248640 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55aab7a8e700 0x55aab7a90af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.650+0000 7fa7e57fa640 10 client.30434 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:26:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.650+0000 7fa801248640 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55aab7a8e700 0x55aab7a90af0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d9403c04ab4b57c8 server_cookie=b8ae2b288270fdd5 in_seq=0 out_seq=0 2024-09-19T06:26:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa7e57fa640 1 -- 172.21.15.5:0/353923325 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aab79de860 con 0x55aab7a8e700 2024-09-19T06:26:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa7e57fa640 10 client.30434 handle_command_reply: tid=0 2024-09-19T06:26:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa806086740 1 client.30434 shutdown 2024-09-19T06:26:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa806086740 1 -- 172.21.15.5:0/353923325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 msgr2=0x55aab7a7e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa806086740 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 0x55aab7a7e620 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800e590 tx=0x7fa7f8005270 comp rx=0 tx=0).stop 2024-09-19T06:26:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa806086740 1 -- 172.21.15.5:0/353923325 shutdown_connections 2024-09-19T06:26:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa806086740 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55aab7a8e700 0x55aab7a90af0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa806086740 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a7eb80 0x55aab7a7c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa806086740 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a7b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa7e57fa640 0 client.30434 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:26:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa806086740 1 --2- 172.21.15.5:0/353923325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 0x55aab7a7e620 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa806086740 1 -- 172.21.15.5:0/353923325 >> 172.21.15.5:0/353923325 conn(0x55aab7a34310 msgr2=0x55aab7a6d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:18.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa806086740 1 -- 172.21.15.5:0/353923325 shutdown_connections 2024-09-19T06:26:18.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.651+0000 7fa806086740 1 -- 172.21.15.5:0/353923325 wait complete. 2024-09-19T06:26:18.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.652+0000 7fa806086740 20 client.30434 trim_cache size 0 max 16384 2024-09-19T06:26:18.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.674+0000 7fa806086740 1 Processor -- start 2024-09-19T06:26:18.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.674+0000 7fa806086740 1 -- start start 2024-09-19T06:26:18.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.674+0000 7fa806086740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a6b5d0 0x55aab7a7a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa806086740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6cb40 0x55aab7a961e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa806086740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a7eb80 0x55aab7a96720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa806086740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1ab650 con 0x55aab7a6b5d0 2024-09-19T06:26:18.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa806086740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1ad190 con 0x55aab7a6cb40 2024-09-19T06:26:18.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa806086740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1abd20 con 0x55aab7a7eb80 2024-09-19T06:26:18.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa801a49640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a6b5d0 0x55aab7a7a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa801248640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6cb40 0x55aab7a961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa801a49640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a6b5d0 0x55aab7a7a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38820/0 (socket says 172.21.15.5:38820) 2024-09-19T06:26:18.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa801a49640 1 -- 172.21.15.5:0/3762705941 learned_addr learned my addr 172.21.15.5:0/3762705941 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:18.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa801248640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6cb40 0x55aab7a961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48130/0 (socket says 172.21.15.5:48130) 2024-09-19T06:26:18.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa80224a640 1 --2- 172.21.15.5:0/3762705941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a7eb80 0x55aab7a96720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa801248640 1 -- 172.21.15.5:0/3762705941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a7eb80 msgr2=0x55aab7a96720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:18.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa801248640 1 --2- 172.21.15.5:0/3762705941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a7eb80 0x55aab7a96720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa801248640 1 -- 172.21.15.5:0/3762705941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a6b5d0 msgr2=0x55aab7a7a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:18.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa801248640 1 --2- 172.21.15.5:0/3762705941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a6b5d0 0x55aab7a7a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa801248640 1 -- 172.21.15.5:0/3762705941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aab7a79e20 con 0x55aab7a6cb40 2024-09-19T06:26:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.675+0000 7fa801248640 1 --2- 172.21.15.5:0/3762705941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6cb40 0x55aab7a961e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ed690 tx=0x7fa7ec0dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d1d1b474f60df47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.676+0000 7fa7e77fe640 1 -- 172.21.15.5:0/3762705941 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0dae60 con 0x55aab7a6cb40 2024-09-19T06:26:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.676+0000 7fa7e77fe640 1 -- 172.21.15.5:0/3762705941 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7ec038040 con 0x55aab7a6cb40 2024-09-19T06:26:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.676+0000 7fa7e77fe640 1 -- 172.21.15.5:0/3762705941 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0ba2c0 con 0x55aab7a6cb40 2024-09-19T06:26:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.677+0000 7fa806086740 1 -- 172.21.15.5:0/3762705941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6cb40 msgr2=0x55aab7a961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.677+0000 7fa806086740 1 --2- 172.21.15.5:0/3762705941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6cb40 0x55aab7a961e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ed690 tx=0x7fa7ec0dc040 comp rx=0 tx=0).stop 2024-09-19T06:26:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.677+0000 7fa806086740 1 -- 172.21.15.5:0/3762705941 shutdown_connections 2024-09-19T06:26:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.677+0000 7fa806086740 1 --2- 172.21.15.5:0/3762705941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a7eb80 0x55aab7a96720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.677+0000 7fa806086740 1 --2- 172.21.15.5:0/3762705941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6cb40 0x55aab7a961e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.677+0000 7fa806086740 1 --2- 172.21.15.5:0/3762705941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a6b5d0 0x55aab7a7a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.677+0000 7fa806086740 1 -- 172.21.15.5:0/3762705941 >> 172.21.15.5:0/3762705941 conn(0x55aab7a34310 msgr2=0x55aab7a7a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.677+0000 7fa806086740 1 -- 172.21.15.5:0/3762705941 shutdown_connections 2024-09-19T06:26:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.677+0000 7fa806086740 1 -- 172.21.15.5:0/3762705941 wait complete. 2024-09-19T06:26:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.678+0000 7fa806086740 1 -- start start 2024-09-19T06:26:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.678+0000 7fa806086740 1 Processor -- start 2024-09-19T06:26:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa806086740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 0x55aab7a87510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa806086740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa806086740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a7eb80 0x55aab7a82ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa801a49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 0x55aab7a87510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa801a49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 0x55aab7a87510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48142/0 (socket says 172.21.15.5:48142) 2024-09-19T06:26:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa801248640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa801248640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a828e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50930/0 (socket says 172.21.15.5:50930) 2024-09-19T06:26:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa80224a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a7eb80 0x55aab7a82ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa80224a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a7eb80 0x55aab7a82ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38834/0 (socket says 172.21.15.5:38834) 2024-09-19T06:26:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa801a49640 1 -- 172.21.15.5:0/3452412221 learned_addr learned my addr 172.21.15.5:0/3452412221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa806086740 1 -- 172.21.15.5:0/3452412221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1acac0 con 0x55aab7a7eb80 2024-09-19T06:26:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa806086740 1 -- 172.21.15.5:0/3452412221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1ac3f0 con 0x55aab7a6b5d0 2024-09-19T06:26:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.679+0000 7fa806086740 1 -- 172.21.15.5:0/3452412221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1a9650 con 0x55aab7a6cb40 2024-09-19T06:26:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa801248640 1 -- 172.21.15.5:0/3452412221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 msgr2=0x55aab7a87510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa801248640 1 --2- 172.21.15.5:0/3452412221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 0x55aab7a87510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa806086740 10 client.0 _ll_register_callbacks cb 0x7fa7f801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa801248640 1 -- 172.21.15.5:0/3452412221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a7eb80 msgr2=0x55aab7a82ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa801248640 1 --2- 172.21.15.5:0/3452412221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a7eb80 0x55aab7a82ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa801248640 1 -- 172.21.15.5:0/3452412221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aab7a83720 con 0x55aab7a6cb40 2024-09-19T06:26:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa806086740 1 -- 172.21.15.5:0/3452412221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7ec038040 con 0x55aab7a6cb40 2024-09-19T06:26:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa80224a640 1 --2- 172.21.15.5:0/3452412221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a7eb80 0x55aab7a82ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa801248640 1 --2- 172.21.15.5:0/3452412221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a828e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec030a40 tx=0x7fa7ec0ed6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72ccbae2f2d1ec9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa7e57fa640 10 client.30364 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa7e57fa640 1 -- 172.21.15.5:0/3452412221 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0d5200 con 0x55aab7a6cb40 2024-09-19T06:26:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa7e57fa640 1 -- 172.21.15.5:0/3452412221 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fa7ec0ba2c0 con 0x55aab7a6cb40 2024-09-19T06:26:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa7e57fa640 1 -- 172.21.15.5:0/3452412221 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fa7ec0bc920 con 0x55aab7a6cb40 2024-09-19T06:26:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa806086740 10 client.30364 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:18.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.680+0000 7fa806086740 1 -- 172.21.15.5:0/3452412221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=62}) -- 0x55aab7af91c0 con 0x55aab7a6cb40 2024-09-19T06:26:18.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.681+0000 7fa7e57fa640 1 -- 172.21.15.5:0/3452412221 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fa7ec0ed800 con 0x55aab7a6cb40 2024-09-19T06:26:18.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.681+0000 7fa806086740 10 client.30364 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:18.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.681+0000 7fa806086740 10 client.30364 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:26:18.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.681+0000 7fa806086740 20 client.30364 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:18.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.681+0000 7fa806086740 1 --2- 172.21.15.5:0/3452412221 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55aab7af9780 0x55aab7afbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:18.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.681+0000 7fa806086740 4 client.30364 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:26:18.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.681+0000 7fa806086740 1 -- 172.21.15.5:0/3452412221 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aab79d8d70 con 0x55aab7af9780 2024-09-19T06:26:18.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.681+0000 7fa801a49640 1 --2- 172.21.15.5:0/3452412221 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55aab7af9780 0x55aab7afbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:18.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.681+0000 7fa801a49640 1 --2- 172.21.15.5:0/3452412221 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55aab7af9780 0x55aab7afbba0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c3ffe5599a5871e8 server_cookie=28ffa17ea989176b in_seq=0 out_seq=0 2024-09-19T06:26:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.681+0000 7fa7e57fa640 10 client.30364 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:26:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa7e57fa640 1 -- 172.21.15.5:0/3452412221 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aab79d8d70 con 0x55aab7af9780 2024-09-19T06:26:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa7e57fa640 10 client.30364 handle_command_reply: tid=0 2024-09-19T06:26:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa806086740 1 client.30364 shutdown 2024-09-19T06:26:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa806086740 1 -- 172.21.15.5:0/3452412221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 msgr2=0x55aab7a828e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa806086740 1 --2- 172.21.15.5:0/3452412221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a828e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec030a40 tx=0x7fa7ec0ed6b0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa806086740 1 -- 172.21.15.5:0/3452412221 shutdown_connections 2024-09-19T06:26:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa806086740 1 --2- 172.21.15.5:0/3452412221 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55aab7af9780 0x55aab7afbba0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa806086740 1 --2- 172.21.15.5:0/3452412221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aab7a7eb80 0x55aab7a82ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa806086740 1 --2- 172.21.15.5:0/3452412221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aab7a6cb40 0x55aab7a828e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa7e57fa640 0 client.30364 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:26:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa806086740 1 --2- 172.21.15.5:0/3452412221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aab7a6b5d0 0x55aab7a87510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa806086740 1 -- 172.21.15.5:0/3452412221 >> 172.21.15.5:0/3452412221 conn(0x55aab7a34310 msgr2=0x55aab7a6d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa806086740 1 -- 172.21.15.5:0/3452412221 shutdown_connections 2024-09-19T06:26:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.682+0000 7fa806086740 1 -- 172.21.15.5:0/3452412221 wait complete. 2024-09-19T06:26:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.683+0000 7fa806086740 20 client.30364 trim_cache size 0 max 16384 2024-09-19T06:26:18.691 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:26:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.683+0000 7fa803cad640 1 -- 172.21.15.5:0/1265896676 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa7e80a3430 msgr2=0x7fa7e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.683+0000 7fa803cad640 1 --2- 172.21.15.5:0/1265896676 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa7e80a3430 0x7fa7e80a58f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4005fd0 tx=0x7fa7f4009500 comp rx=0 tx=0).stop 2024-09-19T06:26:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.683+0000 7fa803cad640 1 -- 172.21.15.5:0/1265896676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc102790 msgr2=0x7fa7fc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.683+0000 7fa803cad640 1 --2- 172.21.15.5:0/1265896676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc102790 0x7fa7fc074a30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0303f0 tx=0x7fa7ec030e00 comp rx=0 tx=0).stop 2024-09-19T06:26:18.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.684+0000 7fa803cad640 1 -- 172.21.15.5:0/1265896676 shutdown_connections 2024-09-19T06:26:18.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.684+0000 7fa803cad640 1 --2- 172.21.15.5:0/1265896676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7fc106f10 0x7fa7fc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.684+0000 7fa803cad640 1 --2- 172.21.15.5:0/1265896676 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa7e80a3430 0x7fa7e80a58f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.684+0000 7fa803cad640 1 --2- 172.21.15.5:0/1265896676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc102790 0x7fa7fc074a30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.684+0000 7fa803cad640 1 --2- 172.21.15.5:0/1265896676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc101de0 0x7fa7fc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:18.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.684+0000 7fa803cad640 1 -- 172.21.15.5:0/1265896676 >> 172.21.15.5:0/1265896676 conn(0x7fa7fc0fd8c0 msgr2=0x7fa7fc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:18.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.684+0000 7fa803cad640 1 -- 172.21.15.5:0/1265896676 shutdown_connections 2024-09-19T06:26:18.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:18.684+0000 7fa803cad640 1 -- 172.21.15.5:0/1265896676 wait complete. 2024-09-19T06:26:19.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:26:19.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 94000 8.57 MB/sec execute 463 sec latency 63.098 ms 2024-09-19T06:26:19.816 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:26:20.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:20 smithi005 ceph-mon[30955]: pgmap v727: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 281 op/s 2024-09-19T06:26:20.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:20 smithi157 ceph-mon[35346]: pgmap v727: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 281 op/s 2024-09-19T06:26:20.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.376+0000 7f1e90bfe640 1 -- 172.21.15.5:0/3691894193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e8c1027a0 msgr2=0x7f1e8c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:20.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.376+0000 7f1e90bfe640 1 --2- 172.21.15.5:0/3691894193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e8c1027a0 0x7f1e8c1067f0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009900 tx=0x7f1e8002a410 comp rx=0 tx=0).stop 2024-09-19T06:26:20.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.376+0000 7f1e90bfe640 1 -- 172.21.15.5:0/3691894193 shutdown_connections 2024-09-19T06:26:20.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.376+0000 7f1e90bfe640 1 --2- 172.21.15.5:0/3691894193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e8c106f20 0x7f1e8c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.376+0000 7f1e90bfe640 1 --2- 172.21.15.5:0/3691894193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e8c1027a0 0x7f1e8c1067f0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.376+0000 7f1e90bfe640 1 --2- 172.21.15.5:0/3691894193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c101df0 0x7f1e8c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.376+0000 7f1e90bfe640 1 -- 172.21.15.5:0/3691894193 >> 172.21.15.5:0/3691894193 conn(0x7f1e8c0fd8f0 msgr2=0x7f1e8c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:20.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.376+0000 7f1e90bfe640 1 -- 172.21.15.5:0/3691894193 shutdown_connections 2024-09-19T06:26:20.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.376+0000 7f1e90bfe640 1 -- 172.21.15.5:0/3691894193 wait complete. 2024-09-19T06:26:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.377+0000 7f1e90bfe640 1 Processor -- start 2024-09-19T06:26:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.377+0000 7f1e90bfe640 1 -- start start 2024-09-19T06:26:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e90bfe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c101df0 0x7f1e8c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e90bfe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e8c1027a0 0x7f1e8c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e90bfe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e8c106f20 0x7f1e8c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e90bfe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c111df0 con 0x7f1e8c106f20 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e90bfe640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c111c70 con 0x7f1e8c1027a0 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e90bfe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c111f70 con 0x7f1e8c101df0 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e8ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e8c106f20 0x7f1e8c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e8ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e8c106f20 0x7f1e8c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38858/0 (socket says 172.21.15.5:38858) 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e8ad9d640 1 -- 172.21.15.5:0/2026642199 learned_addr learned my addr 172.21.15.5:0/2026642199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e8c1027a0 0x7f1e8c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c101df0 0x7f1e8c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e8ad9d640 1 -- 172.21.15.5:0/2026642199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c101df0 msgr2=0x7f1e8c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e8ad9d640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c101df0 0x7f1e8c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e8ad9d640 1 -- 172.21.15.5:0/2026642199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e8c1027a0 msgr2=0x7f1e8c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e8ad9d640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e8c1027a0 0x7f1e8c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.378+0000 7f1e8ad9d640 1 -- 172.21.15.5:0/2026642199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c1a6830 con 0x7f1e8c106f20 2024-09-19T06:26:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.379+0000 7f1e8ad9d640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e8c106f20 0x7f1e8c074f70 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400eb10 tx=0x7f1e8400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:20.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.379+0000 7f1e7b7fe640 1 -- 172.21.15.5:0/2026642199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84004b20 con 0x7f1e8c106f20 2024-09-19T06:26:20.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.379+0000 7f1e7b7fe640 1 -- 172.21.15.5:0/2026642199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e84004cc0 con 0x7f1e8c106f20 2024-09-19T06:26:20.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.379+0000 7f1e90bfe640 1 -- 172.21.15.5:0/2026642199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c1a6b20 con 0x7f1e8c106f20 2024-09-19T06:26:20.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.379+0000 7f1e7b7fe640 1 -- 172.21.15.5:0/2026642199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84010690 con 0x7f1e8c106f20 2024-09-19T06:26:20.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.380+0000 7f1e90bfe640 1 -- 172.21.15.5:0/2026642199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c1a7030 con 0x7f1e8c106f20 2024-09-19T06:26:20.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.383+0000 7f1e7b7fe640 1 -- 172.21.15.5:0/2026642199 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1e84005020 con 0x7f1e8c106f20 2024-09-19T06:26:20.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.383+0000 7f1e92fd7740 1 Processor -- start 2024-09-19T06:26:20.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e92fd7740 1 -- start start 2024-09-19T06:26:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e92fd7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 0x559db9a1e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e92fd7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 0x559db9a1ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e92fd7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a29370 0x559db9a2b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e92fd7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559db99842f0 con 0x559db9a1fb40 2024-09-19T06:26:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e92fd7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559db992bcf0 con 0x559db9a29370 2024-09-19T06:26:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e92fd7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559db99802b0 con 0x559db9a1e5d0 2024-09-19T06:26:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e8a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 0x559db9a1ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e89d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 0x559db9a1e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e7b7fe640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1e740a3430 0x7f1e740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e8a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 0x559db9a1ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50956/0 (socket says 172.21.15.5:50956) 2024-09-19T06:26:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e89d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 0x559db9a1e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38860/0 (socket says 172.21.15.5:38860) 2024-09-19T06:26:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.384+0000 7f1e8a59c640 1 -- 172.21.15.5:0/2110224143 learned_addr learned my addr 172.21.15.5:0/2110224143 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.385+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/2110224143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a29370 0x559db9a2b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.385+0000 7f1e8ad9d640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1e740a3430 0x7f1e740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.385+0000 7f1e7b7fe640 1 -- 172.21.15.5:0/2026642199 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f1e840d0280 con 0x7f1e8c106f20 2024-09-19T06:26:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.385+0000 7f1e8a59c640 1 -- 172.21.15.5:0/2110224143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a29370 msgr2=0x559db9a2b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.385+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/2110224143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a29370 0x559db9a2b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.385+0000 7f1e8a59c640 1 -- 172.21.15.5:0/2110224143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 msgr2=0x559db9a1e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.385+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/2110224143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 0x559db9a1e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.385+0000 7f1e8a59c640 1 -- 172.21.15.5:0/2110224143 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db99103a0 con 0x559db9a1e5d0 2024-09-19T06:26:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.385+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/2110224143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 0x559db9a1e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:26:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.385+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/2110224143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 0x559db9a1ea10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009ca0 tx=0x7f1e7c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc150ca523912675 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.385+0000 7f1e8ad9d640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1e740a3430 0x7f1e740a58f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1e840989a0 tx=0x7f1e84005b40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:20.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.386+0000 7f1e78ff9640 1 -- 172.21.15.5:0/2110224143 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c012450 con 0x559db9a1e5d0 2024-09-19T06:26:20.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.386+0000 7f1e78ff9640 1 -- 172.21.15.5:0/2110224143 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e7c012aa0 con 0x559db9a1e5d0 2024-09-19T06:26:20.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.386+0000 7f1e78ff9640 1 -- 172.21.15.5:0/2110224143 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c00faf0 con 0x559db9a1e5d0 2024-09-19T06:26:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.387+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2110224143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 msgr2=0x559db9a1ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.387+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/2110224143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 0x559db9a1ea10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009ca0 tx=0x7f1e7c0117b0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.387+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2110224143 shutdown_connections 2024-09-19T06:26:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.387+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/2110224143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a29370 0x559db9a2b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.387+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/2110224143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 0x559db9a1ea10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.387+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/2110224143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 0x559db9a1e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.387+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2110224143 >> 172.21.15.5:0/2110224143 conn(0x559db99e7310 msgr2=0x559db9a1ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.387+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2110224143 shutdown_connections 2024-09-19T06:26:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.387+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2110224143 wait complete. 2024-09-19T06:26:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.388+0000 7f1e92fd7740 1 -- start start 2024-09-19T06:26:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.388+0000 7f1e92fd7740 1 Processor -- start 2024-09-19T06:26:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.388+0000 7f1e92fd7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 0x559db9a28cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.388+0000 7f1e92fd7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 0x559db9a25f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.388+0000 7f1e92fd7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a29370 0x559db9a264e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.388+0000 7f1e92fd7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559db9a34880 con 0x559db9a1fb40 2024-09-19T06:26:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.388+0000 7f1e92fd7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559db993f060 con 0x559db9a29370 2024-09-19T06:26:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.388+0000 7f1e92fd7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559db9a34a00 con 0x559db9a1e5d0 2024-09-19T06:26:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e8a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 0x559db9a25f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e92fd7740 10 client.0 _ll_register_callbacks cb 0x559db96928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e8a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 0x559db9a25f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38874/0 (socket says 172.21.15.5:38874) 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e8a59c640 1 -- 172.21.15.5:0/4121449873 learned_addr learned my addr 172.21.15.5:0/4121449873 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 0x559db9a28cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a29370 0x559db9a264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e89d9b640 1 -- 172.21.15.5:0/4121449873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a29370 msgr2=0x559db9a264e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a29370 0x559db9a264e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e89d9b640 1 -- 172.21.15.5:0/4121449873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 msgr2=0x559db9a25f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 0x559db9a25f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e89d9b640 1 -- 172.21.15.5:0/4121449873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db96f53c0 con 0x559db9a1e5d0 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 0x559db9a25f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.389+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 0x559db9a28cd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009230 tx=0x7f1e80009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc79c8134c989ea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.390+0000 7f1e71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.390+0000 7f1e71ffb640 1 -- 172.21.15.5:0/4121449873 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80035040 con 0x559db9a1e5d0 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.390+0000 7f1e71ffb640 1 -- 172.21.15.5:0/4121449873 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f1e80004450 con 0x559db9a1e5d0 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.390+0000 7f1e92fd7740 1 -- 172.21.15.5:0/4121449873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e7c012aa0 con 0x559db9a1e5d0 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.390+0000 7f1e71ffb640 1 -- 172.21.15.5:0/4121449873 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f1e80052050 con 0x559db9a1e5d0 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.390+0000 7f1e92fd7740 10 client.30376 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.390+0000 7f1e92fd7740 1 -- 172.21.15.5:0/4121449873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x559db9a40270 con 0x559db9a1e5d0 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.390+0000 7f1e71ffb640 1 -- 172.21.15.5:0/4121449873 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e7c012aa0 con 0x559db9a1e5d0 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.391+0000 7f1e71ffb640 1 -- 172.21.15.5:0/4121449873 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f1e80004b60 con 0x559db9a1e5d0 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.391+0000 7f1e92fd7740 10 client.30376 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.391+0000 7f1e92fd7740 10 client.30376 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.391+0000 7f1e92fd7740 20 client.30376 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.391+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x559db9a41870 0x559db9a43c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.391+0000 7f1e92fd7740 4 client.30376 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:26:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.391+0000 7f1e92fd7740 1 -- 172.21.15.5:0/4121449873 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559db9991860 con 0x559db9a41870 2024-09-19T06:26:20.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.391+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x559db9a41870 0x559db9a43c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.392+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x559db9a41870 0x559db9a43c60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c61e8a2ce5b1412 server_cookie=54dcaed67543ab26 in_seq=0 out_seq=0 2024-09-19T06:26:20.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.392+0000 7f1e71ffb640 10 client.30376 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e71ffb640 1 -- 172.21.15.5:0/4121449873 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559db9991860 con 0x559db9a41870 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e71ffb640 10 client.30376 handle_command_reply: tid=0 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e92fd7740 1 client.30376 shutdown 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e92fd7740 1 -- 172.21.15.5:0/4121449873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 msgr2=0x559db9a28cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 0x559db9a28cd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009230 tx=0x7f1e80009620 comp rx=0 tx=0).stop 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e92fd7740 1 -- 172.21.15.5:0/4121449873 shutdown_connections 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x559db9a41870 0x559db9a43c60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a29370 0x559db9a264e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1fb40 0x559db9a25f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/4121449873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a1e5d0 0x559db9a28cd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e92fd7740 1 -- 172.21.15.5:0/4121449873 >> 172.21.15.5:0/4121449873 conn(0x559db99e7310 msgr2=0x559db9a20960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e71ffb640 0 client.30376 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:26:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e92fd7740 1 -- 172.21.15.5:0/4121449873 shutdown_connections 2024-09-19T06:26:20.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e92fd7740 1 -- 172.21.15.5:0/4121449873 wait complete. 2024-09-19T06:26:20.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.393+0000 7f1e92fd7740 20 client.30376 trim_cache size 0 max 16384 2024-09-19T06:26:20.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:20 smithi049 ceph-mon[34640]: pgmap v727: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 281 op/s 2024-09-19T06:26:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.414+0000 7f1e92fd7740 1 Processor -- start 2024-09-19T06:26:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.414+0000 7f1e92fd7740 1 -- start start 2024-09-19T06:26:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.414+0000 7f1e92fd7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 0x559db9a3db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e92fd7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a1fb40 0x559db9a2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e92fd7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a29370 0x559db9a2aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e92fd7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1ab650 con 0x559db9a1e5d0 2024-09-19T06:26:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e92fd7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1ad190 con 0x559db9a1fb40 2024-09-19T06:26:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e92fd7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1abd20 con 0x559db9a29370 2024-09-19T06:26:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e89d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 0x559db9a3db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e89d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 0x559db9a3db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38876/0 (socket says 172.21.15.5:38876) 2024-09-19T06:26:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e89d9b640 1 -- 172.21.15.5:0/282565713 learned_addr learned my addr 172.21.15.5:0/282565713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:20.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/282565713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a29370 0x559db9a2aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e89d9b640 1 -- 172.21.15.5:0/282565713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a29370 msgr2=0x559db9a2aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/282565713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a29370 0x559db9a2aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e89d9b640 1 -- 172.21.15.5:0/282565713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a1fb40 msgr2=0x559db9a2a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/282565713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a1fb40 0x559db9a2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/282565713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a1fb40 0x559db9a2a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e89d9b640 1 -- 172.21.15.5:0/282565713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db9a1b960 con 0x559db9a1e5d0 2024-09-19T06:26:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/282565713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a1fb40 0x559db9a2a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:26:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.415+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/282565713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 0x559db9a3db20 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009850 tx=0x7f1e80000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5a10f5a6ba3eb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.416+0000 7f1e73fff640 1 -- 172.21.15.5:0/282565713 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80041070 con 0x559db9a1e5d0 2024-09-19T06:26:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.416+0000 7f1e73fff640 1 -- 172.21.15.5:0/282565713 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e80004450 con 0x559db9a1e5d0 2024-09-19T06:26:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.416+0000 7f1e73fff640 1 -- 172.21.15.5:0/282565713 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80035040 con 0x559db9a1e5d0 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.416+0000 7f1e92fd7740 1 -- 172.21.15.5:0/282565713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 msgr2=0x559db9a3db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.416+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/282565713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 0x559db9a3db20 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009850 tx=0x7f1e80000bc0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.417+0000 7f1e92fd7740 1 -- 172.21.15.5:0/282565713 shutdown_connections 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.417+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/282565713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a29370 0x559db9a2aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.417+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/282565713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a1fb40 0x559db9a2a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.417+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/282565713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 0x559db9a3db20 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.417+0000 7f1e92fd7740 1 -- 172.21.15.5:0/282565713 >> 172.21.15.5:0/282565713 conn(0x559db99e7310 msgr2=0x559db9a43740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.417+0000 7f1e92fd7740 1 -- 172.21.15.5:0/282565713 shutdown_connections 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.417+0000 7f1e92fd7740 1 -- 172.21.15.5:0/282565713 wait complete. 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.417+0000 7f1e92fd7740 1 -- start start 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.417+0000 7f1e92fd7740 1 Processor -- start 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.417+0000 7f1e92fd7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 0x559db9a49ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.417+0000 7f1e92fd7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a1fb40 0x559db9a48390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.417+0000 7f1e92fd7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a29370 0x559db9a488d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e92fd7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1acac0 con 0x559db9a1e5d0 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e92fd7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1ac3f0 con 0x559db9a1fb40 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e92fd7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1a9670 con 0x559db9a29370 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e89d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 0x559db9a49ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e92fd7740 10 client.0 _ll_register_callbacks cb 0x7f1e80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e89d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 0x559db9a49ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38880/0 (socket says 172.21.15.5:38880) 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e89d9b640 1 -- 172.21.15.5:0/2268990574 learned_addr learned my addr 172.21.15.5:0/2268990574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a1fb40 0x559db9a48390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a29370 0x559db9a488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e8a59c640 1 -- 172.21.15.5:0/2268990574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a29370 msgr2=0x559db9a488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a29370 0x559db9a488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e8a59c640 1 -- 172.21.15.5:0/2268990574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 msgr2=0x559db9a49ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:20.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 0x559db9a49ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e8a59c640 1 -- 172.21.15.5:0/2268990574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db9a49100 con 0x559db9a1fb40 2024-09-19T06:26:20.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e8a59c640 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a1fb40 0x559db9a48390 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c01c040 tx=0x7f1e7c009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=536afd10dc6e3614 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:20.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.418+0000 7f1e71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:26:20.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e71ffb640 1 -- 172.21.15.5:0/2268990574 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c004020 con 0x559db9a1fb40 2024-09-19T06:26:20.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2268990574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e80004450 con 0x559db9a1fb40 2024-09-19T06:26:20.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e71ffb640 1 -- 172.21.15.5:0/2268990574 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f1e7c020040 con 0x559db9a1fb40 2024-09-19T06:26:20.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e71ffb640 1 -- 172.21.15.5:0/2268990574 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f1e7c0041c0 con 0x559db9a1fb40 2024-09-19T06:26:20.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e92fd7740 10 client.30455 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:20.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2268990574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x559db9a39bf0 con 0x559db9a1fb40 2024-09-19T06:26:20.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e71ffb640 1 -- 172.21.15.5:0/2268990574 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e80004450 con 0x559db9a1fb40 2024-09-19T06:26:20.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e71ffb640 1 -- 172.21.15.5:0/2268990574 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f1e7c012880 con 0x559db9a1fb40 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e92fd7740 10 client.30455 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e92fd7740 10 client.30455 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e92fd7740 20 client.30455 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x559db9a3a230 0x559db9aae380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e92fd7740 4 client.30455 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.419+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2268990574 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559db998bd70 con 0x559db9a3a230 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x559db9a3a230 0x559db9aae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e71ffb640 10 client.30455 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e89d9b640 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x559db9a3a230 0x559db9aae380 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=85aa618ad32c662f server_cookie=a16c74bca918e6c2 in_seq=0 out_seq=0 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e71ffb640 1 -- 172.21.15.5:0/2268990574 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559db998bd70 con 0x559db9a3a230 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e71ffb640 10 client.30455 handle_command_reply: tid=0 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e92fd7740 1 client.30455 shutdown 2024-09-19T06:26:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2268990574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a1fb40 msgr2=0x559db9a48390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a1fb40 0x559db9a48390 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c01c040 tx=0x7f1e7c009880 comp rx=0 tx=0).stop 2024-09-19T06:26:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2268990574 shutdown_connections 2024-09-19T06:26:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x559db9a3a230 0x559db9aae380 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559db9a29370 0x559db9a488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559db9a1fb40 0x559db9a48390 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e71ffb640 0 client.30455 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:26:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e92fd7740 1 --2- 172.21.15.5:0/2268990574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559db9a1e5d0 0x559db9a49ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.420+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2268990574 >> 172.21.15.5:0/2268990574 conn(0x559db99e7310 msgr2=0x559db9a24810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.421+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2268990574 shutdown_connections 2024-09-19T06:26:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.421+0000 7f1e92fd7740 1 -- 172.21.15.5:0/2268990574 wait complete. 2024-09-19T06:26:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.421+0000 7f1e92fd7740 20 client.30455 trim_cache size 0 max 16384 2024-09-19T06:26:20.428 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:26:20.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.421+0000 7f1e90bfe640 1 -- 172.21.15.5:0/2026642199 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1e740a3430 msgr2=0x7f1e740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:20.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.421+0000 7f1e90bfe640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1e740a3430 0x7f1e740a58f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1e840989a0 tx=0x7f1e84005b40 comp rx=0 tx=0).stop 2024-09-19T06:26:20.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.421+0000 7f1e90bfe640 1 -- 172.21.15.5:0/2026642199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e8c106f20 msgr2=0x7f1e8c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:20.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.421+0000 7f1e90bfe640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e8c106f20 0x7f1e8c074f70 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400eb10 tx=0x7f1e8400c2d0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.422+0000 7f1e90bfe640 1 -- 172.21.15.5:0/2026642199 shutdown_connections 2024-09-19T06:26:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.422+0000 7f1e90bfe640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e8c106f20 0x7f1e8c074f70 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.422+0000 7f1e90bfe640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e8c1027a0 0x7f1e8c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.422+0000 7f1e90bfe640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1e740a3430 0x7f1e740a58f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.422+0000 7f1e90bfe640 1 --2- 172.21.15.5:0/2026642199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c101df0 0x7f1e8c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.422+0000 7f1e90bfe640 1 -- 172.21.15.5:0/2026642199 >> 172.21.15.5:0/2026642199 conn(0x7f1e8c0fd8f0 msgr2=0x7f1e8c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.422+0000 7f1e90bfe640 1 -- 172.21.15.5:0/2026642199 shutdown_connections 2024-09-19T06:26:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:20.422+0000 7f1e90bfe640 1 -- 172.21.15.5:0/2026642199 wait complete. 2024-09-19T06:26:20.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 94150 8.57 MB/sec execute 464 sec latency 76.823 ms 2024-09-19T06:26:21.560 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:26:21.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 94283 8.56 MB/sec execute 465 sec latency 198.281 ms 2024-09-19T06:26:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:26:22.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:22 smithi157 ceph-mon[35346]: pgmap v728: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 244 op/s 2024-09-19T06:26:22.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:22 smithi005 ceph-mon[30955]: pgmap v728: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 244 op/s 2024-09-19T06:26:22.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:22 smithi049 ceph-mon[34640]: pgmap v728: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 244 op/s 2024-09-19T06:26:22.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 94439 8.56 MB/sec execute 466 sec latency 202.989 ms 2024-09-19T06:26:23.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 94632 8.57 MB/sec execute 467 sec latency 119.297 ms 2024-09-19T06:26:24.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:24 smithi157 ceph-mon[35346]: pgmap v729: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 354 op/s 2024-09-19T06:26:24.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:24 smithi005 ceph-mon[30955]: pgmap v729: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 354 op/s 2024-09-19T06:26:24.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:24 smithi049 ceph-mon[34640]: pgmap v729: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 354 op/s 2024-09-19T06:26:24.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 94742 8.56 MB/sec execute 468 sec latency 130.938 ms 2024-09-19T06:26:25.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.794+0000 7f908faf7640 1 -- 172.21.15.5:0/3394811006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9088100e20 msgr2=0x7f9088101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:25.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.794+0000 7f908faf7640 1 --2- 172.21.15.5:0/3394811006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9088100e20 0x7f9088101200 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9084009900 tx=0x7f908402a480 comp rx=0 tx=0).stop 2024-09-19T06:26:25.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.794+0000 7f908faf7640 1 -- 172.21.15.5:0/3394811006 shutdown_connections 2024-09-19T06:26:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.794+0000 7f908faf7640 1 --2- 172.21.15.5:0/3394811006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088102ef0 0x7f908810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.794+0000 7f908faf7640 1 --2- 172.21.15.5:0/3394811006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088102570 0x7f90881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.794+0000 7f908faf7640 1 --2- 172.21.15.5:0/3394811006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9088100e20 0x7f9088101200 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.794+0000 7f908faf7640 1 -- 172.21.15.5:0/3394811006 >> 172.21.15.5:0/3394811006 conn(0x7f90880fc7d0 msgr2=0x7f90880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.794+0000 7f908faf7640 1 -- 172.21.15.5:0/3394811006 shutdown_connections 2024-09-19T06:26:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.794+0000 7f908faf7640 1 -- 172.21.15.5:0/3394811006 wait complete. 2024-09-19T06:26:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908faf7640 1 Processor -- start 2024-09-19T06:26:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908faf7640 1 -- start start 2024-09-19T06:26:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908faf7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088100e20 0x7f9088197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908faf7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088102570 0x7f9088197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908faf7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9088102ef0 0x7f908819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908faf7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9088110e30 con 0x7f9088102ef0 2024-09-19T06:26:25.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908faf7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9088110cb0 con 0x7f9088102570 2024-09-19T06:26:25.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908faf7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9088110fb0 con 0x7f9088100e20 2024-09-19T06:26:25.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908e094640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9088102ef0 0x7f908819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908d893640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088100e20 0x7f9088197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908e094640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9088102ef0 0x7f908819bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54356/0 (socket says 172.21.15.5:54356) 2024-09-19T06:26:25.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908d092640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088102570 0x7f9088197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908e094640 1 -- 172.21.15.5:0/2056115941 learned_addr learned my addr 172.21.15.5:0/2056115941 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:25.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908e094640 1 -- 172.21.15.5:0/2056115941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088100e20 msgr2=0x7f9088197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:25.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908e094640 1 --2- 172.21.15.5:0/2056115941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088100e20 0x7f9088197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908e094640 1 -- 172.21.15.5:0/2056115941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088102570 msgr2=0x7f9088197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:25.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908e094640 1 --2- 172.21.15.5:0/2056115941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088102570 0x7f9088197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.795+0000 7f908e094640 1 -- 172.21.15.5:0/2056115941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908819c2e0 con 0x7f9088102ef0 2024-09-19T06:26:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.796+0000 7f908e094640 1 --2- 172.21.15.5:0/2056115941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9088102ef0 0x7f908819bc00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f908000eb10 tx=0x7f908000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.796+0000 7f907effd640 1 -- 172.21.15.5:0/2056115941 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080004b20 con 0x7f9088102ef0 2024-09-19T06:26:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.796+0000 7f907effd640 1 -- 172.21.15.5:0/2056115941 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9080004cc0 con 0x7f9088102ef0 2024-09-19T06:26:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.796+0000 7f908faf7640 1 -- 172.21.15.5:0/2056115941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908819c5d0 con 0x7f9088102ef0 2024-09-19T06:26:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.796+0000 7f907effd640 1 -- 172.21.15.5:0/2056115941 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080010690 con 0x7f9088102ef0 2024-09-19T06:26:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.796+0000 7f908faf7640 1 -- 172.21.15.5:0/2056115941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90881a3eb0 con 0x7f9088102ef0 2024-09-19T06:26:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.798+0000 7f907effd640 1 -- 172.21.15.5:0/2056115941 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9080005020 con 0x7f9088102ef0 2024-09-19T06:26:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f9091ed0740 1 Processor -- start 2024-09-19T06:26:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f9091ed0740 1 -- start start 2024-09-19T06:26:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f9091ed0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c4b40 0x55cd408c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f9091ed0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 0x55cd408c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f9091ed0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408d0580 0x55cd408d2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f9091ed0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cd408292f0 con 0x55cd408c35d0 2024-09-19T06:26:25.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f9091ed0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cd407d0cf0 con 0x55cd408c4b40 2024-09-19T06:26:25.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f9091ed0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cd408252b0 con 0x55cd408d0580 2024-09-19T06:26:25.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f908d893640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 0x55cd408c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f908d893640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 0x55cd408c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54358/0 (socket says 172.21.15.5:54358) 2024-09-19T06:26:25.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f907effd640 1 --2- 172.21.15.5:0/2056115941 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90740a3430 0x7f90740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f908d893640 1 -- 172.21.15.5:0/1729433999 learned_addr learned my addr 172.21.15.5:0/1729433999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:25.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f908e094640 1 --2- 172.21.15.5:0/2056115941 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90740a3430 0x7f90740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f908d893640 1 -- 172.21.15.5:0/1729433999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408d0580 msgr2=0x55cd408d2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:25.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f908d893640 1 --2- 172.21.15.5:0/1729433999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408d0580 0x55cd408d2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.799+0000 7f908d092640 1 --2- 172.21.15.5:0/1729433999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c4b40 0x55cd408c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f908d893640 1 -- 172.21.15.5:0/1729433999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c4b40 msgr2=0x55cd408c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:25.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f908d893640 1 --2- 172.21.15.5:0/1729433999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c4b40 0x55cd408c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f907effd640 1 -- 172.21.15.5:0/2056115941 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f90800d0380 con 0x7f9088102ef0 2024-09-19T06:26:25.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f908d893640 1 -- 172.21.15.5:0/1729433999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd407b53a0 con 0x55cd408c35d0 2024-09-19T06:26:25.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f908d092640 1 --2- 172.21.15.5:0/1729433999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c4b40 0x55cd408c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:26:25.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f908d893640 1 --2- 172.21.15.5:0/1729433999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 0x55cd408c3a10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f9084009900 tx=0x7f9084005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4505fcd1041b54d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:25.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f908e094640 1 --2- 172.21.15.5:0/2056115941 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90740a3430 0x7f90740a58f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f90800049d0 tx=0x7f9080007850 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:25.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f9073fff640 1 -- 172.21.15.5:0/1729433999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084040070 con 0x55cd408c35d0 2024-09-19T06:26:25.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f9073fff640 1 -- 172.21.15.5:0/1729433999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9084004030 con 0x55cd408c35d0 2024-09-19T06:26:25.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.800+0000 7f9073fff640 1 -- 172.21.15.5:0/1729433999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084035040 con 0x55cd408c35d0 2024-09-19T06:26:25.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.801+0000 7f9091ed0740 1 -- 172.21.15.5:0/1729433999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 msgr2=0x55cd408c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:25.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.801+0000 7f9091ed0740 1 --2- 172.21.15.5:0/1729433999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 0x55cd408c3a10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f9084009900 tx=0x7f9084005bd0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.801+0000 7f9091ed0740 1 -- 172.21.15.5:0/1729433999 shutdown_connections 2024-09-19T06:26:25.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.801+0000 7f9091ed0740 1 --2- 172.21.15.5:0/1729433999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408d0580 0x55cd408d2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.801+0000 7f9091ed0740 1 --2- 172.21.15.5:0/1729433999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 0x55cd408c3a10 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.801+0000 7f9091ed0740 1 --2- 172.21.15.5:0/1729433999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c4b40 0x55cd408c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.801+0000 7f9091ed0740 1 -- 172.21.15.5:0/1729433999 >> 172.21.15.5:0/1729433999 conn(0x55cd4088c310 msgr2=0x55cd408bfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:25.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.801+0000 7f9091ed0740 1 -- 172.21.15.5:0/1729433999 shutdown_connections 2024-09-19T06:26:25.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.801+0000 7f9091ed0740 1 -- 172.21.15.5:0/1729433999 wait complete. 2024-09-19T06:26:25.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f9091ed0740 1 -- start start 2024-09-19T06:26:25.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f9091ed0740 1 Processor -- start 2024-09-19T06:26:25.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f9091ed0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c35d0 0x55cd408cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f9091ed0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c4b40 0x55cd408cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f9091ed0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 0x55cd408cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f9091ed0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cd408d3220 con 0x55cd408d0580 2024-09-19T06:26:25.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f9091ed0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cd407e4060 con 0x55cd408c4b40 2024-09-19T06:26:25.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f908d092640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 0x55cd408cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f9091ed0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cd408d33a0 con 0x55cd408c35d0 2024-09-19T06:26:25.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f908d092640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 0x55cd408cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54370/0 (socket says 172.21.15.5:54370) 2024-09-19T06:26:25.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f908d092640 1 -- 172.21.15.5:0/3164073529 learned_addr learned my addr 172.21.15.5:0/3164073529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:25.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f908d893640 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c4b40 0x55cd408cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f908d092640 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c35d0 0x55cd408cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.802+0000 7f9091ed0740 10 client.0 _ll_register_callbacks cb 0x55cd405378c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:25.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f908d092640 1 -- 172.21.15.5:0/3164073529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c35d0 msgr2=0x55cd408cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:25.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f908d092640 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c35d0 0x55cd408cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f908d092640 1 -- 172.21.15.5:0/3164073529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c4b40 msgr2=0x55cd408cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:25.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f908d092640 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c4b40 0x55cd408cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f908d092640 1 -- 172.21.15.5:0/3164073529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd4059a3c0 con 0x55cd408d0580 2024-09-19T06:26:25.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f9091ed0740 1 -- 172.21.15.5:0/3164073529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9084004030 con 0x55cd408d0580 2024-09-19T06:26:25.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f908d092640 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 0x55cd408cd850 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f9078009560 tx=0x7f9078005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef0c395a5aeacc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:25.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f90717fa640 10 client.21093 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:26:25.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f90717fa640 1 -- 172.21.15.5:0/3164073529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907800fe60 con 0x55cd408d0580 2024-09-19T06:26:25.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f90717fa640 1 -- 172.21.15.5:0/3164073529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f9078005020 con 0x55cd408d0580 2024-09-19T06:26:25.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f90717fa640 1 -- 172.21.15.5:0/3164073529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f9078005220 con 0x55cd408d0580 2024-09-19T06:26:25.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f9091ed0740 10 client.21093 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:25.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f9091ed0740 1 -- 172.21.15.5:0/3164073529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=62}) -- 0x55cd408deb40 con 0x55cd408d0580 2024-09-19T06:26:25.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.803+0000 7f90717fa640 1 -- 172.21.15.5:0/3164073529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f9078028020 con 0x55cd408d0580 2024-09-19T06:26:25.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.804+0000 7f9091ed0740 10 client.21093 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:25.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.804+0000 7f9091ed0740 10 client.21093 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:26:25.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.804+0000 7f9091ed0740 20 client.21093 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:25.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.804+0000 7f9091ed0740 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd408dfef0 0x55cd408e22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.804+0000 7f9091ed0740 4 client.21093 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:26:25.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.804+0000 7f9091ed0740 1 -- 172.21.15.5:0/3164073529 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd40836860 con 0x55cd408dfef0 2024-09-19T06:26:25.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.804+0000 7f908d893640 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd408dfef0 0x55cd408e22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.804+0000 7f908d893640 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd408dfef0 0x55cd408e22e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6083461913074b7 server_cookie=903a2c45f57afdb8 in_seq=0 out_seq=0 2024-09-19T06:26:25.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.804+0000 7f90717fa640 10 client.21093 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:26:25.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.805+0000 7f90717fa640 1 -- 172.21.15.5:0/3164073529 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd40836860 con 0x55cd408dfef0 2024-09-19T06:26:25.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.805+0000 7f90717fa640 10 client.21093 handle_command_reply: tid=0 2024-09-19T06:26:25.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.805+0000 7f9091ed0740 1 client.21093 shutdown 2024-09-19T06:26:25.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.805+0000 7f9091ed0740 1 -- 172.21.15.5:0/3164073529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 msgr2=0x55cd408cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:25.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.805+0000 7f9091ed0740 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 0x55cd408cd850 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f9078009560 tx=0x7f9078005780 comp rx=0 tx=0).stop 2024-09-19T06:26:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.805+0000 7f9091ed0740 1 -- 172.21.15.5:0/3164073529 shutdown_connections 2024-09-19T06:26:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.805+0000 7f9091ed0740 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd408dfef0 0x55cd408e22e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.805+0000 7f9091ed0740 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 0x55cd408cd850 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.805+0000 7f90717fa640 0 client.21093 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:26:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.805+0000 7f9091ed0740 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c4b40 0x55cd408cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.805+0000 7f9091ed0740 1 --2- 172.21.15.5:0/3164073529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c35d0 0x55cd408cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.805+0000 7f9091ed0740 1 -- 172.21.15.5:0/3164073529 >> 172.21.15.5:0/3164073529 conn(0x55cd4088c310 msgr2=0x55cd408c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.806+0000 7f9091ed0740 1 -- 172.21.15.5:0/3164073529 shutdown_connections 2024-09-19T06:26:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.806+0000 7f9091ed0740 1 -- 172.21.15.5:0/3164073529 wait complete. 2024-09-19T06:26:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.806+0000 7f9091ed0740 20 client.21093 trim_cache size 0 max 16384 2024-09-19T06:26:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.829+0000 7f9091ed0740 1 Processor -- start 2024-09-19T06:26:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.829+0000 7f9091ed0740 1 -- start start 2024-09-19T06:26:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f9091ed0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 0x55cd408dc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f9091ed0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c4b40 0x55cd408baad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f9091ed0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408d0580 0x55cd408cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f9091ed0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90881ab610 con 0x55cd408c35d0 2024-09-19T06:26:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f9091ed0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90881ad150 con 0x55cd408d0580 2024-09-19T06:26:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f9091ed0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90881abce0 con 0x55cd408c4b40 2024-09-19T06:26:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f908d092640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 0x55cd408dc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f908d092640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 0x55cd408dc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54372/0 (socket says 172.21.15.5:54372) 2024-09-19T06:26:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f908d893640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c4b40 0x55cd408baad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f908d092640 1 -- 172.21.15.5:0/2759371399 learned_addr learned my addr 172.21.15.5:0/2759371399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:25.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f908d092640 1 --2- 172.21.15.5:0/2759371399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408d0580 0x55cd408cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f908d092640 1 -- 172.21.15.5:0/2759371399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c4b40 msgr2=0x55cd408baad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f908d092640 1 --2- 172.21.15.5:0/2759371399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c4b40 0x55cd408baad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f908d092640 1 -- 172.21.15.5:0/2759371399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408d0580 msgr2=0x55cd408cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f908d092640 1 --2- 172.21.15.5:0/2759371399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408d0580 0x55cd408cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f908d092640 1 -- 172.21.15.5:0/2759371399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd408cc1f0 con 0x55cd408c35d0 2024-09-19T06:26:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.830+0000 7f908d092640 1 --2- 172.21.15.5:0/2759371399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 0x55cd408dc4c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f90780056c0 tx=0x7f9078014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f44987f03e200c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.831+0000 7f90737fe640 1 -- 172.21.15.5:0/2759371399 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907800fe60 con 0x55cd408c35d0 2024-09-19T06:26:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.831+0000 7f90737fe640 1 -- 172.21.15.5:0/2759371399 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9084004030 con 0x55cd408c35d0 2024-09-19T06:26:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.831+0000 7f90737fe640 1 -- 172.21.15.5:0/2759371399 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907802b6b0 con 0x55cd408c35d0 2024-09-19T06:26:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.832+0000 7f9091ed0740 1 -- 172.21.15.5:0/2759371399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 msgr2=0x55cd408dc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:25.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.832+0000 7f9091ed0740 1 --2- 172.21.15.5:0/2759371399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 0x55cd408dc4c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f90780056c0 tx=0x7f9078014000 comp rx=0 tx=0).stop 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.832+0000 7f9091ed0740 1 -- 172.21.15.5:0/2759371399 shutdown_connections 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.832+0000 7f9091ed0740 1 --2- 172.21.15.5:0/2759371399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408d0580 0x55cd408cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.832+0000 7f9091ed0740 1 --2- 172.21.15.5:0/2759371399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c4b40 0x55cd408baad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.832+0000 7f9091ed0740 1 --2- 172.21.15.5:0/2759371399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408c35d0 0x55cd408dc4c0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.832+0000 7f9091ed0740 1 -- 172.21.15.5:0/2759371399 >> 172.21.15.5:0/2759371399 conn(0x55cd4088c310 msgr2=0x55cd408ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.832+0000 7f9091ed0740 1 -- 172.21.15.5:0/2759371399 shutdown_connections 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.832+0000 7f9091ed0740 1 -- 172.21.15.5:0/2759371399 wait complete. 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.832+0000 7f9091ed0740 1 -- start start 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f9091ed0740 1 Processor -- start 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f9091ed0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c35d0 0x55cd408d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f9091ed0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c4b40 0x55cd408d8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f9091ed0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 0x55cd408c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f9091ed0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90881aca80 con 0x55cd408d0580 2024-09-19T06:26:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f9091ed0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90881ac3b0 con 0x55cd408c35d0 2024-09-19T06:26:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f9091ed0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90881a9610 con 0x55cd408c4b40 2024-09-19T06:26:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f9091ed0740 10 client.0 _ll_register_callbacks cb 0x7f9078009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f908d092640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 0x55cd408c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f908d893640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c4b40 0x55cd408d8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f908d092640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 0x55cd408c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54388/0 (socket says 172.21.15.5:54388) 2024-09-19T06:26:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f908d092640 1 -- 172.21.15.5:0/3571425849 learned_addr learned my addr 172.21.15.5:0/3571425849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f908d893640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c4b40 0x55cd408d8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36248/0 (socket says 172.21.15.5:36248) 2024-09-19T06:26:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.833+0000 7f908d092640 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c35d0 0x55cd408d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f908d893640 1 -- 172.21.15.5:0/3571425849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c35d0 msgr2=0x55cd408d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f908d893640 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c35d0 0x55cd408d89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f908d893640 1 -- 172.21.15.5:0/3571425849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 msgr2=0x55cd408c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:25.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f908d893640 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 0x55cd408c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f908d893640 1 -- 172.21.15.5:0/3571425849 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd408ca520 con 0x55cd408c4b40 2024-09-19T06:26:25.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f908d092640 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 0x55cd408c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:25.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f908d893640 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c4b40 0x55cd408d8ee0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9084005940 tx=0x7f9084031ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3742e237331c5f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:25.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f90717fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:25.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f90717fa640 1 -- 172.21.15.5:0/3571425849 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084057060 con 0x55cd408c4b40 2024-09-19T06:26:25.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f90717fa640 1 -- 172.21.15.5:0/3571425849 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f9084031630 con 0x55cd408c4b40 2024-09-19T06:26:25.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f9091ed0740 1 -- 172.21.15.5:0/3571425849 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9084004030 con 0x55cd408c4b40 2024-09-19T06:26:25.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f90717fa640 1 -- 172.21.15.5:0/3571425849 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f9084035070 con 0x55cd408c4b40 2024-09-19T06:26:25.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f9091ed0740 10 client.30397 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:25.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.834+0000 7f9091ed0740 1 -- 172.21.15.5:0/3571425849 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55cd408c6fb0 con 0x55cd408c4b40 2024-09-19T06:26:25.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.835+0000 7f90717fa640 1 -- 172.21.15.5:0/3571425849 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9084031830 con 0x55cd408c4b40 2024-09-19T06:26:25.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.835+0000 7f90717fa640 1 -- 172.21.15.5:0/3571425849 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f9084004b80 con 0x55cd408c4b40 2024-09-19T06:26:25.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.835+0000 7f9091ed0740 10 client.30397 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:25.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.835+0000 7f9091ed0740 10 client.30397 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:26:25.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.835+0000 7f9091ed0740 20 client.30397 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:25.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.835+0000 7f9091ed0740 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd408dfef0 0x55cd408c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:25.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.835+0000 7f9091ed0740 4 client.30397 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:26:25.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.835+0000 7f9091ed0740 1 -- 172.21.15.5:0/3571425849 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9084004030 con 0x55cd408dfef0 2024-09-19T06:26:25.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.835+0000 7f908d092640 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd408dfef0 0x55cd408c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:25.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.836+0000 7f90717fa640 10 client.30397 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:26:25.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.836+0000 7f908d092640 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd408dfef0 0x55cd408c7750 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0d61911acb9f623 server_cookie=bc96036dc721166e in_seq=0 out_seq=0 2024-09-19T06:26:25.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.836+0000 7f90717fa640 1 -- 172.21.15.5:0/3571425849 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9084004030 con 0x55cd408dfef0 2024-09-19T06:26:25.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.836+0000 7f90717fa640 10 client.30397 handle_command_reply: tid=0 2024-09-19T06:26:25.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.836+0000 7f9091ed0740 1 client.30397 shutdown 2024-09-19T06:26:25.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.836+0000 7f9091ed0740 1 -- 172.21.15.5:0/3571425849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c4b40 msgr2=0x55cd408d8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:25.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.836+0000 7f9091ed0740 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c4b40 0x55cd408d8ee0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9084005940 tx=0x7f9084031ba0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.837+0000 7f9091ed0740 1 -- 172.21.15.5:0/3571425849 shutdown_connections 2024-09-19T06:26:25.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.837+0000 7f9091ed0740 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55cd408dfef0 0x55cd408c7750 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.837+0000 7f9091ed0740 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd408d0580 0x55cd408c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.837+0000 7f90717fa640 0 client.30397 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:26:25.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.837+0000 7f9091ed0740 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd408c4b40 0x55cd408d8ee0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.837+0000 7f9091ed0740 1 --2- 172.21.15.5:0/3571425849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd408c35d0 0x55cd408d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.837+0000 7f9091ed0740 1 -- 172.21.15.5:0/3571425849 >> 172.21.15.5:0/3571425849 conn(0x55cd4088c310 msgr2=0x55cd408bed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:25.446 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:26:25.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.837+0000 7f9091ed0740 1 -- 172.21.15.5:0/3571425849 shutdown_connections 2024-09-19T06:26:25.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.837+0000 7f9091ed0740 1 -- 172.21.15.5:0/3571425849 wait complete. 2024-09-19T06:26:25.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.837+0000 7f9091ed0740 20 client.30397 trim_cache size 0 max 16384 2024-09-19T06:26:25.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.838+0000 7f908faf7640 1 -- 172.21.15.5:0/2056115941 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90740a3430 msgr2=0x7f90740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:25.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.838+0000 7f908faf7640 1 --2- 172.21.15.5:0/2056115941 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90740a3430 0x7f90740a58f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f90800049d0 tx=0x7f9080007850 comp rx=0 tx=0).stop 2024-09-19T06:26:25.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.838+0000 7f908faf7640 1 -- 172.21.15.5:0/2056115941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9088102ef0 msgr2=0x7f908819bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:25.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.838+0000 7f908faf7640 1 --2- 172.21.15.5:0/2056115941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9088102ef0 0x7f908819bc00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f908000eb10 tx=0x7f908000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.838+0000 7f908faf7640 1 -- 172.21.15.5:0/2056115941 shutdown_connections 2024-09-19T06:26:25.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.838+0000 7f908faf7640 1 --2- 172.21.15.5:0/2056115941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9088102ef0 0x7f908819bc00 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.838+0000 7f908faf7640 1 --2- 172.21.15.5:0/2056115941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088102570 0x7f9088197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.838+0000 7f908faf7640 1 --2- 172.21.15.5:0/2056115941 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f90740a3430 0x7f90740a58f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.838+0000 7f908faf7640 1 --2- 172.21.15.5:0/2056115941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088100e20 0x7f9088197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:25.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.838+0000 7f908faf7640 1 -- 172.21.15.5:0/2056115941 >> 172.21.15.5:0/2056115941 conn(0x7f90880fc7d0 msgr2=0x7f90880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:25.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.838+0000 7f908faf7640 1 -- 172.21.15.5:0/2056115941 shutdown_connections 2024-09-19T06:26:25.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:24.838+0000 7f908faf7640 1 -- 172.21.15.5:0/2056115941 wait complete. 2024-09-19T06:26:25.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 94874 8.56 MB/sec execute 469 sec latency 86.252 ms 2024-09-19T06:26:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.105+0000 7fc49e559640 1 -- 172.21.15.5:0/2649110458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4981048f0 msgr2=0x7fc498108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.105+0000 7fc49e559640 1 --2- 172.21.15.5:0/2649110458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4981048f0 0x7fc498108930 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc480009900 tx=0x7fc48002a420 comp rx=0 tx=0).stop 2024-09-19T06:26:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.105+0000 7fc49e559640 1 -- 172.21.15.5:0/2649110458 shutdown_connections 2024-09-19T06:26:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.105+0000 7fc49e559640 1 --2- 172.21.15.5:0/2649110458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc498109090 0x7fc498110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.105+0000 7fc49e559640 1 --2- 172.21.15.5:0/2649110458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4981048f0 0x7fc498108930 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.105+0000 7fc49e559640 1 --2- 172.21.15.5:0/2649110458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc498103f40 0x7fc498104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.105+0000 7fc49e559640 1 -- 172.21.15.5:0/2649110458 >> 172.21.15.5:0/2649110458 conn(0x7fc4980ff970 msgr2=0x7fc498101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.105+0000 7fc49e559640 1 -- 172.21.15.5:0/2649110458 shutdown_connections 2024-09-19T06:26:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.106+0000 7fc49e559640 1 -- 172.21.15.5:0/2649110458 wait complete. 2024-09-19T06:26:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.106+0000 7fc49e559640 1 Processor -- start 2024-09-19T06:26:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.107+0000 7fc49e559640 1 -- start start 2024-09-19T06:26:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.107+0000 7fc49e559640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc498103f40 0x7fc49819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.107+0000 7fc49e559640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc498109090 0x7fc49819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.107+0000 7fc49e559640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49819ee80 0x7fc49819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.107+0000 7fc49e559640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc498114030 con 0x7fc498109090 2024-09-19T06:26:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.107+0000 7fc49e559640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc498113eb0 con 0x7fc498103f40 2024-09-19T06:26:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.107+0000 7fc49e559640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981141b0 con 0x7fc49819ee80 2024-09-19T06:26:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.107+0000 7fc49caf6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49819ee80 0x7fc49819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.107+0000 7fc48ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc498109090 0x7fc49819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.107+0000 7fc48ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc498109090 0x7fc49819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54410/0 (socket says 172.21.15.5:54410) 2024-09-19T06:26:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.107+0000 7fc48ffff640 1 -- 172.21.15.5:0/680403948 learned_addr learned my addr 172.21.15.5:0/680403948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.108+0000 7fc497fff640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc498103f40 0x7fc49819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.108+0000 7fc48ffff640 1 -- 172.21.15.5:0/680403948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49819ee80 msgr2=0x7fc49819f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.108+0000 7fc48ffff640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49819ee80 0x7fc49819f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.108+0000 7fc48ffff640 1 -- 172.21.15.5:0/680403948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc498103f40 msgr2=0x7fc49819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.108+0000 7fc48ffff640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc498103f40 0x7fc49819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.108+0000 7fc48ffff640 1 -- 172.21.15.5:0/680403948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49819f970 con 0x7fc498109090 2024-09-19T06:26:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.108+0000 7fc48ffff640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc498109090 0x7fc49819ab00 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fc480031440 tx=0x7fc480003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.108+0000 7fc495ffb640 1 -- 172.21.15.5:0/680403948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48003f070 con 0x7fc498109090 2024-09-19T06:26:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.108+0000 7fc495ffb640 1 -- 172.21.15.5:0/680403948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc480031b50 con 0x7fc498109090 2024-09-19T06:26:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.108+0000 7fc49e559640 1 -- 172.21.15.5:0/680403948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49819fc00 con 0x7fc498109090 2024-09-19T06:26:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.108+0000 7fc495ffb640 1 -- 172.21.15.5:0/680403948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480035030 con 0x7fc498109090 2024-09-19T06:26:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.109+0000 7fc49e559640 1 -- 172.21.15.5:0/680403948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4981a73c0 con 0x7fc498109090 2024-09-19T06:26:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc4a13d1740 1 Processor -- start 2024-09-19T06:26:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc4a13d1740 1 -- start start 2024-09-19T06:26:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc495ffb640 1 -- 172.21.15.5:0/680403948 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc480042070 con 0x7fc498109090 2024-09-19T06:26:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc4a13d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0246af0 0x55e7a0246ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc4a13d1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0247410 0x55e7a0251770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc4a13d1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0251cb0 0x55e7a02540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc4a13d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e7a01ab2f0 con 0x55e7a0246af0 2024-09-19T06:26:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc4a13d1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e7a0153270 con 0x55e7a0251cb0 2024-09-19T06:26:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc4a13d1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e7a01a72b0 con 0x55e7a0247410 2024-09-19T06:26:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc49caf6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0246af0 0x55e7a0246ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc497fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0247410 0x55e7a0251770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc497fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0247410 0x55e7a0251770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36274/0 (socket says 172.21.15.5:36274) 2024-09-19T06:26:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc49caf6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0246af0 0x55e7a0246ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54420/0 (socket says 172.21.15.5:54420) 2024-09-19T06:26:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc497fff640 1 -- 172.21.15.5:0/3212914442 learned_addr learned my addr 172.21.15.5:0/3212914442 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.112+0000 7fc48ffff640 1 --2- 172.21.15.5:0/3212914442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0251cb0 0x55e7a02540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.113+0000 7fc497fff640 1 -- 172.21.15.5:0/3212914442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0251cb0 msgr2=0x55e7a02540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.113+0000 7fc497fff640 1 --2- 172.21.15.5:0/3212914442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0251cb0 0x55e7a02540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.113+0000 7fc497fff640 1 -- 172.21.15.5:0/3212914442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0246af0 msgr2=0x55e7a0246ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.113+0000 7fc497fff640 1 --2- 172.21.15.5:0/3212914442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0246af0 0x55e7a0246ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.113+0000 7fc497fff640 1 -- 172.21.15.5:0/3212914442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7a0152340 con 0x55e7a0247410 2024-09-19T06:26:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.113+0000 7fc495ffb640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc4840a3400 0x7fc4840a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.113+0000 7fc497fff640 1 --2- 172.21.15.5:0/3212914442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0247410 0x55e7a0251770 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc48800b820 tx=0x7fc48800bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=925e553014de918e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.113+0000 7fc495ffb640 1 -- 172.21.15.5:0/680403948 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fc4800f0550 con 0x7fc498109090 2024-09-19T06:26:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.113+0000 7fc48e7fc640 1 -- 172.21.15.5:0/3212914442 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48800d950 con 0x55e7a0247410 2024-09-19T06:26:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.113+0000 7fc49caf6640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc4840a3400 0x7fc4840a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.113+0000 7fc48e7fc640 1 -- 172.21.15.5:0/3212914442 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc488019070 con 0x55e7a0247410 2024-09-19T06:26:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.114+0000 7fc48e7fc640 1 -- 172.21.15.5:0/3212914442 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4880123e0 con 0x55e7a0247410 2024-09-19T06:26:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.114+0000 7fc49caf6640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc4840a3400 0x7fc4840a58c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc490005fd0 tx=0x7fc490009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.114+0000 7fc4a13d1740 1 -- 172.21.15.5:0/3212914442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0247410 msgr2=0x55e7a0251770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.114+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/3212914442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0247410 0x55e7a0251770 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc48800b820 tx=0x7fc48800bce0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.115+0000 7fc4a13d1740 1 -- 172.21.15.5:0/3212914442 shutdown_connections 2024-09-19T06:26:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.115+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/3212914442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0251cb0 0x55e7a02540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.115+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/3212914442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0247410 0x55e7a0251770 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.115+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/3212914442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0246af0 0x55e7a0246ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.115+0000 7fc4a13d1740 1 -- 172.21.15.5:0/3212914442 >> 172.21.15.5:0/3212914442 conn(0x55e7a0243060 msgr2=0x55e7a0243440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.115+0000 7fc4a13d1740 1 -- 172.21.15.5:0/3212914442 shutdown_connections 2024-09-19T06:26:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.115+0000 7fc4a13d1740 1 -- 172.21.15.5:0/3212914442 wait complete. 2024-09-19T06:26:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.115+0000 7fc4a13d1740 1 -- start start 2024-09-19T06:26:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.116+0000 7fc4a13d1740 1 Processor -- start 2024-09-19T06:26:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc4a13d1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 0x55e7a024eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc4a13d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a024f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc497fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 0x55e7a024eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc497fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 0x55e7a024eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46554/0 (socket says 172.21.15.5:46554) 2024-09-19T06:26:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc48ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a024f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc48ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a024f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54424/0 (socket says 172.21.15.5:54424) 2024-09-19T06:26:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc497fff640 1 -- 172.21.15.5:0/4006494603 learned_addr learned my addr 172.21.15.5:0/4006494603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0251cb0 0x55e7a024f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc4a13d1740 1 -- 172.21.15.5:0/4006494603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e7a02548d0 con 0x55e7a0247410 2024-09-19T06:26:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc4a13d1740 1 -- 172.21.15.5:0/4006494603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e7a0134290 con 0x55e7a0246af0 2024-09-19T06:26:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc4a13d1740 1 -- 172.21.15.5:0/4006494603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e7a0254a50 con 0x55e7a0251cb0 2024-09-19T06:26:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc4a13d1740 10 client.0 _ll_register_callbacks cb 0x55e7a0165140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc49caf6640 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0251cb0 0x55e7a024f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc497fff640 1 -- 172.21.15.5:0/4006494603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0251cb0 msgr2=0x55e7a024f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc497fff640 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0251cb0 0x55e7a024f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc497fff640 1 -- 172.21.15.5:0/4006494603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 msgr2=0x55e7a024f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc497fff640 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a024f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.117+0000 7fc497fff640 1 -- 172.21.15.5:0/4006494603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e79febe210 con 0x55e7a0246af0 2024-09-19T06:26:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.118+0000 7fc497fff640 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 0x55e7a024eaf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc488014690 tx=0x7fc4880146c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f81e475b375e516 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.118+0000 7fc4777fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:26:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.118+0000 7fc4777fe640 1 -- 172.21.15.5:0/4006494603 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488023070 con 0x55e7a0246af0 2024-09-19T06:26:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.118+0000 7fc4777fe640 1 -- 172.21.15.5:0/4006494603 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fc48800d950 con 0x55e7a0246af0 2024-09-19T06:26:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.118+0000 7fc4a13d1740 1 -- 172.21.15.5:0/4006494603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc488019070 con 0x55e7a0246af0 2024-09-19T06:26:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.118+0000 7fc4777fe640 1 -- 172.21.15.5:0/4006494603 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc48802f050 con 0x55e7a0246af0 2024-09-19T06:26:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.118+0000 7fc4a13d1740 10 client.30479 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.118+0000 7fc4a13d1740 1 -- 172.21.15.5:0/4006494603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55e79ff8a620 con 0x55e7a0246af0 2024-09-19T06:26:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.119+0000 7fc4777fe640 1 -- 172.21.15.5:0/4006494603 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc488019070 con 0x55e7a0246af0 2024-09-19T06:26:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.119+0000 7fc4777fe640 1 -- 172.21.15.5:0/4006494603 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc488013070 con 0x55e7a0246af0 2024-09-19T06:26:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.119+0000 7fc4a13d1740 10 client.30479 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.119+0000 7fc4a13d1740 10 client.30479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.119+0000 7fc4a13d1740 20 client.30479 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.119+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e7a0261aa0 0x55e7a0263e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.119+0000 7fc4a13d1740 4 client.30479 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:26:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.119+0000 7fc4a13d1740 1 -- 172.21.15.5:0/4006494603 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7a01b8a20 con 0x55e7a0261aa0 2024-09-19T06:26:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.119+0000 7fc48ffff640 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e7a0261aa0 0x55e7a0263e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.120+0000 7fc48ffff640 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e7a0261aa0 0x55e7a0263e90 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6205ea595ca2f584 server_cookie=62a6a997cd2b8b in_seq=0 out_seq=0 2024-09-19T06:26:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.120+0000 7fc4777fe640 10 client.30479 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.120+0000 7fc4777fe640 1 -- 172.21.15.5:0/4006494603 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7a01b8a20 con 0x55e7a0261aa0 2024-09-19T06:26:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.121+0000 7fc4777fe640 10 client.30479 handle_command_reply: tid=0 2024-09-19T06:26:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.121+0000 7fc4a13d1740 1 client.30479 shutdown 2024-09-19T06:26:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.121+0000 7fc4a13d1740 1 -- 172.21.15.5:0/4006494603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 msgr2=0x55e7a024eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.121+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 0x55e7a024eaf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc488014690 tx=0x7fc4880146c0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.122+0000 7fc4a13d1740 1 -- 172.21.15.5:0/4006494603 shutdown_connections 2024-09-19T06:26:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.122+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e7a0261aa0 0x55e7a0263e90 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.122+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0251cb0 0x55e7a024f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.122+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a024f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.122+0000 7fc4777fe640 0 client.30479 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.122+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/4006494603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 0x55e7a024eaf0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.122+0000 7fc4a13d1740 1 -- 172.21.15.5:0/4006494603 >> 172.21.15.5:0/4006494603 conn(0x55e7a0243060 msgr2=0x55e7a0248dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.122+0000 7fc4a13d1740 1 -- 172.21.15.5:0/4006494603 shutdown_connections 2024-09-19T06:26:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.122+0000 7fc4a13d1740 1 -- 172.21.15.5:0/4006494603 wait complete. 2024-09-19T06:26:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.122+0000 7fc4a13d1740 20 client.30479 trim_cache size 0 max 16384 2024-09-19T06:26:26.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:26 smithi005 ceph-mon[30955]: pgmap v730: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 235 op/s 2024-09-19T06:26:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.148+0000 7fc4a13d1740 1 Processor -- start 2024-09-19T06:26:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.148+0000 7fc4a13d1740 1 -- start start 2024-09-19T06:26:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.148+0000 7fc4a13d1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 0x55e7a0249560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.148+0000 7fc4a13d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a024f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.148+0000 7fc4a13d1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0251cb0 0x55e7a02694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc4a13d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981af9f0 con 0x55e7a0247410 2024-09-19T06:26:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc4a13d1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981b1530 con 0x55e7a0246af0 2024-09-19T06:26:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc4a13d1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981b00c0 con 0x55e7a0251cb0 2024-09-19T06:26:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc48ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a024f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc497fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 0x55e7a0249560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc497fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 0x55e7a0249560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46560/0 (socket says 172.21.15.5:46560) 2024-09-19T06:26:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc497fff640 1 -- 172.21.15.5:0/2048417703 learned_addr learned my addr 172.21.15.5:0/2048417703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc49caf6640 1 --2- 172.21.15.5:0/2048417703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0251cb0 0x55e7a02694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc497fff640 1 -- 172.21.15.5:0/2048417703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0251cb0 msgr2=0x55e7a02694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc497fff640 1 --2- 172.21.15.5:0/2048417703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0251cb0 0x55e7a02694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc497fff640 1 -- 172.21.15.5:0/2048417703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 msgr2=0x55e7a024f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc497fff640 1 --2- 172.21.15.5:0/2048417703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a024f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc497fff640 1 -- 172.21.15.5:0/2048417703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7a0242e90 con 0x55e7a0246af0 2024-09-19T06:26:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc497fff640 1 --2- 172.21.15.5:0/2048417703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 0x55e7a0249560 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc48801fdf0 tx=0x7fc488014b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d43106626c84555 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.149+0000 7fc48dffb640 1 -- 172.21.15.5:0/2048417703 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488023070 con 0x55e7a0246af0 2024-09-19T06:26:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.150+0000 7fc48dffb640 1 -- 172.21.15.5:0/2048417703 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc488019070 con 0x55e7a0246af0 2024-09-19T06:26:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.150+0000 7fc48dffb640 1 -- 172.21.15.5:0/2048417703 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488004030 con 0x55e7a0246af0 2024-09-19T06:26:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.151+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2048417703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 msgr2=0x55e7a0249560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.151+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/2048417703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 0x55e7a0249560 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc48801fdf0 tx=0x7fc488014b70 comp rx=0 tx=0).stop 2024-09-19T06:26:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.151+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2048417703 shutdown_connections 2024-09-19T06:26:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.151+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/2048417703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0251cb0 0x55e7a02694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.151+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/2048417703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a024f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.151+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/2048417703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0246af0 0x55e7a0249560 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.151+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2048417703 >> 172.21.15.5:0/2048417703 conn(0x55e7a0243060 msgr2=0x55e7a0250b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.151+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2048417703 shutdown_connections 2024-09-19T06:26:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.151+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2048417703 wait complete. 2024-09-19T06:26:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.151+0000 7fc4a13d1740 1 -- start start 2024-09-19T06:26:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.152+0000 7fc4a13d1740 1 Processor -- start 2024-09-19T06:26:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.152+0000 7fc4a13d1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0246af0 0x55e7a025a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.152+0000 7fc4a13d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a0255580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.152+0000 7fc4a13d1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0251cb0 0x55e7a0255ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.152+0000 7fc4a13d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981b0e60 con 0x55e7a0247410 2024-09-19T06:26:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.152+0000 7fc4a13d1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981b0790 con 0x55e7a0251cb0 2024-09-19T06:26:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.152+0000 7fc4a13d1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981ad9f0 con 0x55e7a0246af0 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.152+0000 7fc497fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0246af0 0x55e7a025a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.152+0000 7fc4a13d1740 10 client.0 _ll_register_callbacks cb 0x7fc490009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc497fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0246af0 0x55e7a025a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36298/0 (socket says 172.21.15.5:36298) 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc497fff640 1 -- 172.21.15.5:0/2477479604 learned_addr learned my addr 172.21.15.5:0/2477479604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc48ffff640 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a0255580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc49caf6640 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0251cb0 0x55e7a0255ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc497fff640 1 -- 172.21.15.5:0/2477479604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0251cb0 msgr2=0x55e7a0255ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc497fff640 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0251cb0 0x55e7a0255ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc497fff640 1 -- 172.21.15.5:0/2477479604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 msgr2=0x55e7a0255580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc497fff640 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a0255580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc497fff640 1 -- 172.21.15.5:0/2477479604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7a023e2d0 con 0x55e7a0246af0 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc497fff640 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0246af0 0x55e7a025a260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc488001ef0 tx=0x7fc488001f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bb8300623d34e98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc4777fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc4777fe640 1 -- 172.21.15.5:0/2477479604 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488023040 con 0x55e7a0246af0 2024-09-19T06:26:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc4777fe640 1 -- 172.21.15.5:0/2477479604 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fc48800db50 con 0x55e7a0246af0 2024-09-19T06:26:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.153+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2477479604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc488019070 con 0x55e7a0246af0 2024-09-19T06:26:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.154+0000 7fc4777fe640 1 -- 172.21.15.5:0/2477479604 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc488006c50 con 0x55e7a0246af0 2024-09-19T06:26:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.154+0000 7fc4a13d1740 10 client.30415 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.154+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2477479604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55e7a02cc0f0 con 0x55e7a0246af0 2024-09-19T06:26:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.154+0000 7fc4777fe640 1 -- 172.21.15.5:0/2477479604 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc488019070 con 0x55e7a0246af0 2024-09-19T06:26:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.154+0000 7fc4777fe640 1 -- 172.21.15.5:0/2477479604 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc4880036b0 con 0x55e7a0246af0 2024-09-19T06:26:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.154+0000 7fc4a13d1740 10 client.30415 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.154+0000 7fc4a13d1740 10 client.30415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.154+0000 7fc4a13d1740 20 client.30415 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.154+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e7a0261aa0 0x55e7a02ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.154+0000 7fc4a13d1740 4 client.30415 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:26:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.154+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2477479604 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7a01b2d70 con 0x55e7a0261aa0 2024-09-19T06:26:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.154+0000 7fc48ffff640 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e7a0261aa0 0x55e7a02ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.155+0000 7fc48ffff640 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e7a0261aa0 0x55e7a02ce7b0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b203d74f6c4caf45 server_cookie=ff6a08a11714fb50 in_seq=0 out_seq=0 2024-09-19T06:26:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.155+0000 7fc4777fe640 10 client.30415 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.243+0000 7fc4777fe640 1 -- 172.21.15.5:0/2477479604 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7a01b2d70 con 0x55e7a0261aa0 2024-09-19T06:26:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.243+0000 7fc4777fe640 10 client.30415 handle_command_reply: tid=0 2024-09-19T06:26:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.243+0000 7fc4a13d1740 1 client.30415 shutdown 2024-09-19T06:26:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.243+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2477479604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0246af0 msgr2=0x55e7a025a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.243+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0246af0 0x55e7a025a260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc488001ef0 tx=0x7fc488001f20 comp rx=0 tx=0).stop 2024-09-19T06:26:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.245+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2477479604 shutdown_connections 2024-09-19T06:26:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.245+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55e7a0261aa0 0x55e7a02ce7b0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.245+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7a0251cb0 0x55e7a0255ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.245+0000 7fc4777fe640 0 client.30415 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.245+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7a0247410 0x55e7a0255580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.245+0000 7fc4a13d1740 1 --2- 172.21.15.5:0/2477479604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7a0246af0 0x55e7a025a260 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.245+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2477479604 >> 172.21.15.5:0/2477479604 conn(0x55e7a0243060 msgr2=0x55e7a02419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.245+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2477479604 shutdown_connections 2024-09-19T06:26:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.245+0000 7fc4a13d1740 1 -- 172.21.15.5:0/2477479604 wait complete. 2024-09-19T06:26:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.246+0000 7fc4a13d1740 20 client.30415 trim_cache size 0 max 16384 2024-09-19T06:26:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:26:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:26:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:26:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:26:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.247+0000 7fc49e559640 1 -- 172.21.15.5:0/680403948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc4840a3400 msgr2=0x7fc4840a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.247+0000 7fc49e559640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc4840a3400 0x7fc4840a58c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc490005fd0 tx=0x7fc490009500 comp rx=0 tx=0).stop 2024-09-19T06:26:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.247+0000 7fc49e559640 1 -- 172.21.15.5:0/680403948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc498109090 msgr2=0x7fc49819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.247+0000 7fc49e559640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc498109090 0x7fc49819ab00 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fc480031440 tx=0x7fc480003c00 comp rx=0 tx=0).stop 2024-09-19T06:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.248+0000 7fc49e559640 1 -- 172.21.15.5:0/680403948 shutdown_connections 2024-09-19T06:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.248+0000 7fc49e559640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc4840a3400 0x7fc4840a58c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.248+0000 7fc49e559640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49819ee80 0x7fc49819f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.248+0000 7fc49e559640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc498109090 0x7fc49819ab00 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.248+0000 7fc49e559640 1 --2- 172.21.15.5:0/680403948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc498103f40 0x7fc49819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.248+0000 7fc49e559640 1 -- 172.21.15.5:0/680403948 >> 172.21.15.5:0/680403948 conn(0x7fc4980ff970 msgr2=0x7fc498100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.248+0000 7fc49e559640 1 -- 172.21.15.5:0/680403948 shutdown_connections 2024-09-19T06:26:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:26.248+0000 7fc49e559640 1 -- 172.21.15.5:0/680403948 wait complete. 2024-09-19T06:26:26.286 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:26:26.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:26 smithi157 ceph-mon[35346]: pgmap v730: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 235 op/s 2024-09-19T06:26:26.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:26 smithi049 ceph-mon[34640]: pgmap v730: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 235 op/s 2024-09-19T06:26:26.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 95026 8.56 MB/sec execute 470 sec latency 82.863 ms 2024-09-19T06:26:26.708 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:26:27.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.259+0000 7f77767cd640 1 -- 172.21.15.5:0/2609382878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7770102ef0 msgr2=0x7f777010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:27.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.259+0000 7f77767cd640 1 --2- 172.21.15.5:0/2609382878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7770102ef0 0x7f777010da40 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f7768009ca0 tx=0x7f776802a950 comp rx=0 tx=0).stop 2024-09-19T06:26:27.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.259+0000 7f77767cd640 1 -- 172.21.15.5:0/2609382878 shutdown_connections 2024-09-19T06:26:27.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.259+0000 7f77767cd640 1 --2- 172.21.15.5:0/2609382878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7770102ef0 0x7f777010da40 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.259+0000 7f77767cd640 1 --2- 172.21.15.5:0/2609382878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7770102570 0x7f77701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.259+0000 7f77767cd640 1 --2- 172.21.15.5:0/2609382878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7770100e20 0x7f7770101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.259+0000 7f77767cd640 1 -- 172.21.15.5:0/2609382878 >> 172.21.15.5:0/2609382878 conn(0x7f77700fc7d0 msgr2=0x7f77700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.259+0000 7f77767cd640 1 -- 172.21.15.5:0/2609382878 shutdown_connections 2024-09-19T06:26:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.259+0000 7f77767cd640 1 -- 172.21.15.5:0/2609382878 wait complete. 2024-09-19T06:26:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.260+0000 7f77767cd640 1 Processor -- start 2024-09-19T06:26:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.260+0000 7f77767cd640 1 -- start start 2024-09-19T06:26:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.261+0000 7f77767cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7770100e20 0x7f77701a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.261+0000 7f77767cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7770102570 0x7f77701a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.261+0000 7f77767cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7770102ef0 0x7f777010cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.261+0000 7f77767cd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7770111f80 con 0x7f7770102570 2024-09-19T06:26:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.261+0000 7f77767cd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7770111e00 con 0x7f7770100e20 2024-09-19T06:26:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.261+0000 7f77767cd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7770112100 con 0x7f7770102ef0 2024-09-19T06:26:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.261+0000 7f776ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7770100e20 0x7f77701a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.261+0000 7f776ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7770100e20 0x7f77701a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46590/0 (socket says 172.21.15.5:46590) 2024-09-19T06:26:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.261+0000 7f776ffff640 1 -- 172.21.15.5:0/2564372962 learned_addr learned my addr 172.21.15.5:0/2564372962 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.261+0000 7f7774d6a640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7770102ef0 0x7f777010cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.261+0000 7f776f7fe640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7770102570 0x7f77701a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.262+0000 7f776ffff640 1 -- 172.21.15.5:0/2564372962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7770102ef0 msgr2=0x7f777010cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.262+0000 7f776ffff640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7770102ef0 0x7f777010cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.262+0000 7f776ffff640 1 -- 172.21.15.5:0/2564372962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7770102570 msgr2=0x7f77701a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.262+0000 7f776ffff640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7770102570 0x7f77701a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.262+0000 7f776ffff640 1 -- 172.21.15.5:0/2564372962 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777010d2b0 con 0x7f7770100e20 2024-09-19T06:26:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.262+0000 7f776ffff640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7770100e20 0x7f77701a4630 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f776000e9f0 tx=0x7f776000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.262+0000 7f776d7fa640 1 -- 172.21.15.5:0/2564372962 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776000cd80 con 0x7f7770100e20 2024-09-19T06:26:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.263+0000 7f77767cd640 1 -- 172.21.15.5:0/2564372962 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777010d5a0 con 0x7f7770100e20 2024-09-19T06:26:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.263+0000 7f776d7fa640 1 -- 172.21.15.5:0/2564372962 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77600108e0 con 0x7f7770100e20 2024-09-19T06:26:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.263+0000 7f776d7fa640 1 -- 172.21.15.5:0/2564372962 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760018920 con 0x7f7770100e20 2024-09-19T06:26:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.263+0000 7f77767cd640 1 -- 172.21.15.5:0/2564372962 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77701ab490 con 0x7f7770100e20 2024-09-19T06:26:27.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f776d7fa640 1 -- 172.21.15.5:0/2564372962 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7760010a90 con 0x7f7770100e20 2024-09-19T06:26:27.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f7778ba6740 1 Processor -- start 2024-09-19T06:26:27.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f7778ba6740 1 -- start start 2024-09-19T06:26:27.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f7778ba6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 0x5617991c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f7778ba6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 0x5617991c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f7778ba6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991d4370 0x5617991d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f7778ba6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56179912f2f0 con 0x5617991cab40 2024-09-19T06:26:27.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f7778ba6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5617990d6cf0 con 0x5617991c95d0 2024-09-19T06:26:27.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f7778ba6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56179912b2b0 con 0x5617991d4370 2024-09-19T06:26:27.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f7774d6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 0x5617991c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f776f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991d4370 0x5617991d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f776f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991d4370 0x5617991d6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36316/0 (socket says 172.21.15.5:36316) 2024-09-19T06:26:27.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.266+0000 7f776f7fe640 1 -- 172.21.15.5:0/3135071921 learned_addr learned my addr 172.21.15.5:0/3135071921 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:27.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.267+0000 7f7774d6a640 1 -- 172.21.15.5:0/3135071921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991d4370 msgr2=0x5617991d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:27.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.267+0000 7f7774d6a640 1 --2- 172.21.15.5:0/3135071921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991d4370 0x5617991d6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.267+0000 7f7774d6a640 1 -- 172.21.15.5:0/3135071921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 msgr2=0x5617991c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:27.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.267+0000 7f776d7fa640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f77580a3430 0x7f77580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.267+0000 7f776d7fa640 1 -- 172.21.15.5:0/2564372962 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f77600cff70 con 0x7f7770100e20 2024-09-19T06:26:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.267+0000 7f776ffff640 1 --2- 172.21.15.5:0/3135071921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 0x5617991c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.267+0000 7f7774d6a640 1 --2- 172.21.15.5:0/3135071921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 0x5617991c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.267+0000 7f7774d6a640 1 -- 172.21.15.5:0/3135071921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617990bb3a0 con 0x5617991cab40 2024-09-19T06:26:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.267+0000 7f776f7fe640 1 --2- 172.21.15.5:0/3135071921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991d4370 0x5617991d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:26:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.267+0000 7f776ffff640 1 --2- 172.21.15.5:0/3135071921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 0x5617991c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:26:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.268+0000 7f7774d6a640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f77580a3430 0x7f77580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.268+0000 7f7774d6a640 1 --2- 172.21.15.5:0/3135071921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 0x5617991c9090 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f7768005e30 tx=0x7f7768005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c692b1467723fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.268+0000 7f77567fc640 1 -- 172.21.15.5:0/3135071921 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768034040 con 0x5617991cab40 2024-09-19T06:26:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.268+0000 7f77567fc640 1 -- 172.21.15.5:0/3135071921 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f776802fe30 con 0x5617991cab40 2024-09-19T06:26:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.268+0000 7f77567fc640 1 -- 172.21.15.5:0/3135071921 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680047b0 con 0x5617991cab40 2024-09-19T06:26:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.269+0000 7f7778ba6740 1 -- 172.21.15.5:0/3135071921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 msgr2=0x5617991c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.269+0000 7f7774d6a640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f77580a3430 0x7f77580a58f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7768004af0 tx=0x7f776804d000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.269+0000 7f7778ba6740 1 --2- 172.21.15.5:0/3135071921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 0x5617991c9090 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f7768005e30 tx=0x7f7768005f80 comp rx=0 tx=0).stop 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.269+0000 7f7778ba6740 1 -- 172.21.15.5:0/3135071921 shutdown_connections 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.269+0000 7f7778ba6740 1 --2- 172.21.15.5:0/3135071921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991d4370 0x5617991d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.269+0000 7f7778ba6740 1 --2- 172.21.15.5:0/3135071921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 0x5617991c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.269+0000 7f7778ba6740 1 --2- 172.21.15.5:0/3135071921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 0x5617991c9090 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.269+0000 7f7778ba6740 1 -- 172.21.15.5:0/3135071921 >> 172.21.15.5:0/3135071921 conn(0x561799192310 msgr2=0x5617991c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.270+0000 7f7778ba6740 1 -- 172.21.15.5:0/3135071921 shutdown_connections 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.270+0000 7f7778ba6740 1 -- 172.21.15.5:0/3135071921 wait complete. 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.270+0000 7f7778ba6740 1 -- start start 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f7778ba6740 1 Processor -- start 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f7778ba6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991c95d0 0x5617991d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f7778ba6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991cab40 0x5617991d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f7778ba6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991d4370 0x5617991d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f7778ba6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617991d7010 con 0x5617991c95d0 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f7778ba6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5617990ea060 con 0x5617991d4370 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f7778ba6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5617991d7190 con 0x5617991cab40 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f7778ba6740 10 client.0 _ll_register_callbacks cb 0x561798e3d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f776f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991cab40 0x5617991d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f776f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991cab40 0x5617991d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36322/0 (socket says 172.21.15.5:36322) 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f776f7fe640 1 -- 172.21.15.5:0/4138124077 learned_addr learned my addr 172.21.15.5:0/4138124077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f7774d6a640 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991d4370 0x5617991d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f776f7fe640 1 -- 172.21.15.5:0/4138124077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991d4370 msgr2=0x5617991d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f776f7fe640 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991d4370 0x5617991d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.271+0000 7f776f7fe640 1 -- 172.21.15.5:0/4138124077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991c95d0 msgr2=0x5617991d3d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:26:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.272+0000 7f776ffff640 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991c95d0 0x5617991d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.272+0000 7f776f7fe640 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991c95d0 0x5617991d3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.272+0000 7f776f7fe640 1 -- 172.21.15.5:0/4138124077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561798ea03c0 con 0x5617991cab40 2024-09-19T06:26:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.272+0000 7f776ffff640 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991c95d0 0x5617991d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:26:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.272+0000 7f776f7fe640 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991cab40 0x5617991d1000 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7764012700 tx=0x7f7764012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=537d06cc1c9fb734 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.272+0000 7f774f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.272+0000 7f774f7fe640 1 -- 172.21.15.5:0/4138124077 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764004030 con 0x5617991cab40 2024-09-19T06:26:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.272+0000 7f7778ba6740 1 -- 172.21.15.5:0/4138124077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f776802fe30 con 0x5617991cab40 2024-09-19T06:26:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.272+0000 7f774f7fe640 1 -- 172.21.15.5:0/4138124077 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f77640041d0 con 0x5617991cab40 2024-09-19T06:26:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.273+0000 7f774f7fe640 1 -- 172.21.15.5:0/4138124077 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f77640043d0 con 0x5617991cab40 2024-09-19T06:26:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.273+0000 7f7778ba6740 10 client.30427 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.273+0000 7f7778ba6740 1 -- 172.21.15.5:0/4138124077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x5617991e2ab0 con 0x5617991cab40 2024-09-19T06:26:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.273+0000 7f774f7fe640 1 -- 172.21.15.5:0/4138124077 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f776802fe30 con 0x5617991cab40 2024-09-19T06:26:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.273+0000 7f774f7fe640 1 -- 172.21.15.5:0/4138124077 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f7764003b40 con 0x5617991cab40 2024-09-19T06:26:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.273+0000 7f7778ba6740 10 client.30427 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.273+0000 7f7778ba6740 10 client.30427 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:26:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.273+0000 7f7778ba6740 20 client.30427 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.273+0000 7f7778ba6740 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5617991e3f30 0x5617991e6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.273+0000 7f7778ba6740 4 client.30427 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:26:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.273+0000 7f7778ba6740 1 -- 172.21.15.5:0/4138124077 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56179913c860 con 0x5617991e3f30 2024-09-19T06:26:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.273+0000 7f776ffff640 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5617991e3f30 0x5617991e6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.274+0000 7f774f7fe640 10 client.30427 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:26:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.274+0000 7f776ffff640 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5617991e3f30 0x5617991e6370 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6dcf7bdaea943cf server_cookie=9865dfd8af012798 in_seq=0 out_seq=0 2024-09-19T06:26:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.275+0000 7f774f7fe640 1 -- 172.21.15.5:0/4138124077 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56179913c860 con 0x5617991e3f30 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.275+0000 7f774f7fe640 10 client.30427 handle_command_reply: tid=0 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.275+0000 7f7778ba6740 1 client.30427 shutdown 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.275+0000 7f7778ba6740 1 -- 172.21.15.5:0/4138124077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991cab40 msgr2=0x5617991d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.275+0000 7f7778ba6740 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991cab40 0x5617991d1000 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7764012700 tx=0x7f7764012bc0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.275+0000 7f7778ba6740 1 -- 172.21.15.5:0/4138124077 shutdown_connections 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.275+0000 7f7778ba6740 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5617991e3f30 0x5617991e6370 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.276+0000 7f7778ba6740 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991d4370 0x5617991d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.276+0000 7f7778ba6740 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991cab40 0x5617991d1000 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.276+0000 7f7778ba6740 1 --2- 172.21.15.5:0/4138124077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991c95d0 0x5617991d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.276+0000 7f7778ba6740 1 -- 172.21.15.5:0/4138124077 >> 172.21.15.5:0/4138124077 conn(0x561799192310 msgr2=0x5617991cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.276+0000 7f774f7fe640 0 client.30427 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.276+0000 7f7778ba6740 1 -- 172.21.15.5:0/4138124077 shutdown_connections 2024-09-19T06:26:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.276+0000 7f7778ba6740 1 -- 172.21.15.5:0/4138124077 wait complete. 2024-09-19T06:26:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.276+0000 7f7778ba6740 20 client.30427 trim_cache size 0 max 16384 2024-09-19T06:26:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.296+0000 7f7778ba6740 1 Processor -- start 2024-09-19T06:26:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.296+0000 7f7778ba6740 1 -- start start 2024-09-19T06:26:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.297+0000 7f7778ba6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991c95d0 0x5617991c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.297+0000 7f7778ba6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 0x5617991e02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.297+0000 7f7778ba6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991d4370 0x5617991d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.297+0000 7f7778ba6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77701b3e20 con 0x5617991cab40 2024-09-19T06:26:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.297+0000 7f7778ba6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77701b5960 con 0x5617991d4370 2024-09-19T06:26:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.297+0000 7f7778ba6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77701b44f0 con 0x5617991c95d0 2024-09-19T06:26:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.298+0000 7f776ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991c95d0 0x5617991c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.298+0000 7f776f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 0x5617991e02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.298+0000 7f776f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 0x5617991e02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54486/0 (socket says 172.21.15.5:54486) 2024-09-19T06:26:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.298+0000 7f776f7fe640 1 -- 172.21.15.5:0/1246751833 learned_addr learned my addr 172.21.15.5:0/1246751833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.298+0000 7f776f7fe640 1 -- 172.21.15.5:0/1246751833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991c95d0 msgr2=0x5617991c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.298+0000 7f776f7fe640 1 --2- 172.21.15.5:0/1246751833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991c95d0 0x5617991c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.298+0000 7f776f7fe640 1 -- 172.21.15.5:0/1246751833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991d4370 msgr2=0x5617991d08c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:26:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.298+0000 7f776f7fe640 1 --2- 172.21.15.5:0/1246751833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991d4370 0x5617991d08c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.298+0000 7f776f7fe640 1 -- 172.21.15.5:0/1246751833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617991cffb0 con 0x5617991cab40 2024-09-19T06:26:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.298+0000 7f776f7fe640 1 --2- 172.21.15.5:0/1246751833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 0x5617991e02b0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f7764009cb0 tx=0x7f7764012360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3d6327e77ce4b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.299+0000 7f7755ffb640 1 -- 172.21.15.5:0/1246751833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764028040 con 0x5617991cab40 2024-09-19T06:26:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.299+0000 7f7755ffb640 1 -- 172.21.15.5:0/1246751833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f776400f790 con 0x5617991cab40 2024-09-19T06:26:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.299+0000 7f7755ffb640 1 -- 172.21.15.5:0/1246751833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776400fd10 con 0x5617991cab40 2024-09-19T06:26:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.300+0000 7f7778ba6740 1 -- 172.21.15.5:0/1246751833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 msgr2=0x5617991e02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.300+0000 7f7778ba6740 1 --2- 172.21.15.5:0/1246751833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 0x5617991e02b0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f7764009cb0 tx=0x7f7764012360 comp rx=0 tx=0).stop 2024-09-19T06:26:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.300+0000 7f7778ba6740 1 -- 172.21.15.5:0/1246751833 shutdown_connections 2024-09-19T06:26:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.300+0000 7f7778ba6740 1 --2- 172.21.15.5:0/1246751833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991d4370 0x5617991d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.300+0000 7f7778ba6740 1 --2- 172.21.15.5:0/1246751833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991cab40 0x5617991e02b0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.300+0000 7f7778ba6740 1 --2- 172.21.15.5:0/1246751833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991c95d0 0x5617991c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.300+0000 7f7778ba6740 1 -- 172.21.15.5:0/1246751833 >> 172.21.15.5:0/1246751833 conn(0x561799192310 msgr2=0x5617991c5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.300+0000 7f7778ba6740 1 -- 172.21.15.5:0/1246751833 shutdown_connections 2024-09-19T06:26:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.301+0000 7f7778ba6740 1 -- 172.21.15.5:0/1246751833 wait complete. 2024-09-19T06:26:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.301+0000 7f7778ba6740 1 -- start start 2024-09-19T06:26:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.302+0000 7f7778ba6740 1 Processor -- start 2024-09-19T06:26:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.302+0000 7f7778ba6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 0x5617991dcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.302+0000 7f7778ba6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991cab40 0x5617991cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.302+0000 7f7778ba6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991d4370 0x5617991ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.302+0000 7f7778ba6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77701b5290 con 0x5617991d4370 2024-09-19T06:26:27.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.302+0000 7f7778ba6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77701b4bc0 con 0x5617991c95d0 2024-09-19T06:26:27.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.302+0000 7f7778ba6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77701b1e20 con 0x5617991cab40 2024-09-19T06:26:27.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.302+0000 7f776ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 0x5617991dcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.302+0000 7f7778ba6740 10 client.0 _ll_register_callbacks cb 0x7f7764009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:27.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.302+0000 7f776ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 0x5617991dcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46628/0 (socket says 172.21.15.5:46628) 2024-09-19T06:26:27.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.302+0000 7f776ffff640 1 -- 172.21.15.5:0/3684240001 learned_addr learned my addr 172.21.15.5:0/3684240001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:27.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.302+0000 7f776f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991cab40 0x5617991cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.303+0000 7f7774d6a640 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991d4370 0x5617991ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.303+0000 7f776ffff640 1 -- 172.21.15.5:0/3684240001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991cab40 msgr2=0x5617991cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.303+0000 7f776ffff640 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991cab40 0x5617991cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.303+0000 7f776ffff640 1 -- 172.21.15.5:0/3684240001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991d4370 msgr2=0x5617991ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.303+0000 7f776ffff640 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991d4370 0x5617991ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.303+0000 7f776ffff640 1 -- 172.21.15.5:0/3684240001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617991d40c0 con 0x5617991c95d0 2024-09-19T06:26:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.303+0000 7f7774d6a640 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991d4370 0x5617991ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:26:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.303+0000 7f776ffff640 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 0x5617991dcce0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7760000c00 tx=0x7f77600bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca56d7d2edaa6e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.303+0000 7f774f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:26:27.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.303+0000 7f774f7fe640 1 -- 172.21.15.5:0/3684240001 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776009c2e0 con 0x5617991c95d0 2024-09-19T06:26:27.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.303+0000 7f774f7fe640 1 -- 172.21.15.5:0/3684240001 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f776009a5e0 con 0x5617991c95d0 2024-09-19T06:26:27.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.303+0000 7f7778ba6740 1 -- 172.21.15.5:0/3684240001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f776400f790 con 0x5617991c95d0 2024-09-19T06:26:27.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.304+0000 7f774f7fe640 1 -- 172.21.15.5:0/3684240001 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f77600b5120 con 0x5617991c95d0 2024-09-19T06:26:27.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.304+0000 7f7778ba6740 10 client.30497 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:27.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.304+0000 7f7778ba6740 1 -- 172.21.15.5:0/3684240001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x56179924eb30 con 0x5617991c95d0 2024-09-19T06:26:27.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.304+0000 7f774f7fe640 1 -- 172.21.15.5:0/3684240001 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f776400f790 con 0x5617991c95d0 2024-09-19T06:26:27.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.304+0000 7f774f7fe640 1 -- 172.21.15.5:0/3684240001 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f77600d20b0 con 0x5617991c95d0 2024-09-19T06:26:27.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.304+0000 7f7778ba6740 10 client.30497 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.304+0000 7f7778ba6740 10 client.30497 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:26:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.304+0000 7f7778ba6740 20 client.30497 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.304+0000 7f7778ba6740 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56179924f200 0x5617992515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.304+0000 7f7778ba6740 4 client.30497 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:26:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.304+0000 7f7778ba6740 1 -- 172.21.15.5:0/3684240001 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561799136d70 con 0x56179924f200 2024-09-19T06:26:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.305+0000 7f776f7fe640 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56179924f200 0x5617992515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.305+0000 7f774f7fe640 10 client.30497 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:26:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.306+0000 7f776f7fe640 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56179924f200 0x5617992515f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd744a3e459e4117 server_cookie=a897f3f67dc9351 in_seq=0 out_seq=0 2024-09-19T06:26:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.306+0000 7f774f7fe640 1 -- 172.21.15.5:0/3684240001 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561799136d70 con 0x56179924f200 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.306+0000 7f774f7fe640 10 client.30497 handle_command_reply: tid=0 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.306+0000 7f7778ba6740 1 client.30497 shutdown 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.306+0000 7f7778ba6740 1 -- 172.21.15.5:0/3684240001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 msgr2=0x5617991dcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.306+0000 7f7778ba6740 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 0x5617991dcce0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7760000c00 tx=0x7f77600bc040 comp rx=0 tx=0).stop 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.306+0000 7f7778ba6740 1 -- 172.21.15.5:0/3684240001 shutdown_connections 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.306+0000 7f7778ba6740 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x56179924f200 0x5617992515f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.307+0000 7f7778ba6740 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617991d4370 0x5617991ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.307+0000 7f774f7fe640 0 client.30497 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.307+0000 7f7778ba6740 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617991cab40 0x5617991cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.307+0000 7f7778ba6740 1 --2- 172.21.15.5:0/3684240001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617991c95d0 0x5617991dcce0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.307+0000 7f7778ba6740 1 -- 172.21.15.5:0/3684240001 >> 172.21.15.5:0/3684240001 conn(0x561799192310 msgr2=0x5617991d2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.307+0000 7f7778ba6740 1 -- 172.21.15.5:0/3684240001 shutdown_connections 2024-09-19T06:26:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.307+0000 7f7778ba6740 1 -- 172.21.15.5:0/3684240001 wait complete. 2024-09-19T06:26:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.307+0000 7f7778ba6740 20 client.30497 trim_cache size 0 max 16384 2024-09-19T06:26:27.312 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:26:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.308+0000 7f77767cd640 1 -- 172.21.15.5:0/2564372962 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f77580a3430 msgr2=0x7f77580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.308+0000 7f77767cd640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f77580a3430 0x7f77580a58f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7768004af0 tx=0x7f776804d000 comp rx=0 tx=0).stop 2024-09-19T06:26:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.308+0000 7f77767cd640 1 -- 172.21.15.5:0/2564372962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7770100e20 msgr2=0x7f77701a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.308+0000 7f77767cd640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7770100e20 0x7f77701a4630 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f776000e9f0 tx=0x7f776000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.308+0000 7f77767cd640 1 -- 172.21.15.5:0/2564372962 shutdown_connections 2024-09-19T06:26:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.308+0000 7f77767cd640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f77580a3430 0x7f77580a58f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.308+0000 7f77767cd640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7770102ef0 0x7f777010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.308+0000 7f77767cd640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7770102570 0x7f77701a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.308+0000 7f77767cd640 1 --2- 172.21.15.5:0/2564372962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7770100e20 0x7f77701a4630 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.308+0000 7f77767cd640 1 -- 172.21.15.5:0/2564372962 >> 172.21.15.5:0/2564372962 conn(0x7f77700fc7d0 msgr2=0x7f77701067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.308+0000 7f77767cd640 1 -- 172.21.15.5:0/2564372962 shutdown_connections 2024-09-19T06:26:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:27.308+0000 7f77767cd640 1 -- 172.21.15.5:0/2564372962 wait complete. 2024-09-19T06:26:27.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 95178 8.57 MB/sec execute 471 sec latency 70.544 ms 2024-09-19T06:26:28.128 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:28 smithi005 ceph-mon[30955]: pgmap v731: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.4 MiB/s wr, 381 op/s 2024-09-19T06:26:28.210 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:26:28.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:28 smithi157 ceph-mon[35346]: pgmap v731: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.4 MiB/s wr, 381 op/s 2024-09-19T06:26:28.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:28 smithi049 ceph-mon[34640]: pgmap v731: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.4 MiB/s wr, 381 op/s 2024-09-19T06:26:28.588 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:26:28.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 95300 8.55 MB/sec execute 472 sec latency 89.272 ms 2024-09-19T06:26:29.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.175+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/2004017460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f58100e20 msgr2=0x7f8f58101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:29.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.175+0000 7f8f5f9c9640 1 --2- 172.21.15.5:0/2004017460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f58100e20 0x7f8f58101200 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009900 tx=0x7f8f5402a480 comp rx=0 tx=0).stop 2024-09-19T06:26:29.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.175+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/2004017460 shutdown_connections 2024-09-19T06:26:29.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.175+0000 7f8f5f9c9640 1 --2- 172.21.15.5:0/2004017460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f58102ef0 0x7f8f5810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.175+0000 7f8f5f9c9640 1 --2- 172.21.15.5:0/2004017460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f58102570 0x7f8f581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.175+0000 7f8f5f9c9640 1 --2- 172.21.15.5:0/2004017460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f58100e20 0x7f8f58101200 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.175+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/2004017460 >> 172.21.15.5:0/2004017460 conn(0x7f8f580fc7d0 msgr2=0x7f8f580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:29.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.175+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/2004017460 shutdown_connections 2024-09-19T06:26:29.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.175+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/2004017460 wait complete. 2024-09-19T06:26:29.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.176+0000 7f8f5f9c9640 1 Processor -- start 2024-09-19T06:26:29.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.176+0000 7f8f5f9c9640 1 -- start start 2024-09-19T06:26:29.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.176+0000 7f8f5f9c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f58100e20 0x7f8f58198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5f9c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f58102570 0x7f8f581989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5f9c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f58102ef0 0x7f8f5819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5f9c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f58111f30 con 0x7f8f58102570 2024-09-19T06:26:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5f9c9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f58111db0 con 0x7f8f58102ef0 2024-09-19T06:26:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5f9c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581120b0 con 0x7f8f58100e20 2024-09-19T06:26:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5cf64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f58102570 0x7f8f581989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5d765640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f58100e20 0x7f8f58198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5cf64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f58102570 0x7f8f581989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54528/0 (socket says 172.21.15.5:54528) 2024-09-19T06:26:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5cf64640 1 -- 172.21.15.5:0/507050037 learned_addr learned my addr 172.21.15.5:0/507050037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5d765640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f58100e20 0x7f8f58198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36356/0 (socket says 172.21.15.5:36356) 2024-09-19T06:26:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5df66640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f58102ef0 0x7f8f5819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5df66640 1 -- 172.21.15.5:0/507050037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f58100e20 msgr2=0x7f8f58198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5df66640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f58100e20 0x7f8f58198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5df66640 1 -- 172.21.15.5:0/507050037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f58102570 msgr2=0x7f8f581989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5df66640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f58102570 0x7f8f581989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5df66640 1 -- 172.21.15.5:0/507050037 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5819d420 con 0x7f8f58102ef0 2024-09-19T06:26:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5d765640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f58100e20 0x7f8f58198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.177+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f58102570 0x7f8f581989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.178+0000 7f8f5df66640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f58102ef0 0x7f8f5819cd40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009fd0 tx=0x7f8f5000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.178+0000 7f8f4e7fc640 1 -- 172.21.15.5:0/507050037 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5000ce20 con 0x7f8f58102ef0 2024-09-19T06:26:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.178+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/507050037 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5819d710 con 0x7f8f58102ef0 2024-09-19T06:26:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.178+0000 7f8f4e7fc640 1 -- 172.21.15.5:0/507050037 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f50005020 con 0x7f8f58102ef0 2024-09-19T06:26:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.178+0000 7f8f4e7fc640 1 -- 172.21.15.5:0/507050037 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50010690 con 0x7f8f58102ef0 2024-09-19T06:26:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.178+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/507050037 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5810df80 con 0x7f8f58102ef0 2024-09-19T06:26:29.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.181+0000 7f8f4e7fc640 1 -- 172.21.15.5:0/507050037 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8f500051d0 con 0x7f8f58102ef0 2024-09-19T06:26:29.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.181+0000 7f8f61da2740 1 Processor -- start 2024-09-19T06:26:29.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.181+0000 7f8f61da2740 1 -- start start 2024-09-19T06:26:29.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f61da2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae458a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f61da2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae46220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f61da2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae50b80 0x55893ae52f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f61da2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55893ada32f0 con 0x55893ae50b80 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f61da2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55893ad4acf0 con 0x55893ae45de0 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f61da2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55893ad9f2b0 con 0x55893ae47350 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f5d765640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae46220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f5cf64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f5d765640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae46220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46660/0 (socket says 172.21.15.5:46660) 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f5d765640 1 -- 172.21.15.5:0/2345391431 learned_addr learned my addr 172.21.15.5:0/2345391431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/2345391431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae50b80 0x55893ae52f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f5d765640 1 -- 172.21.15.5:0/2345391431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 msgr2=0x55893ae458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f5d765640 1 --2- 172.21.15.5:0/2345391431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f5d765640 1 -- 172.21.15.5:0/2345391431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae50b80 msgr2=0x55893ae52f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f5d765640 1 --2- 172.21.15.5:0/2345391431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae50b80 0x55893ae52f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f5d765640 1 -- 172.21.15.5:0/2345391431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55893ad2f3a0 con 0x55893ae45de0 2024-09-19T06:26:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.182+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/2345391431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae50b80 0x55893ae52f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:26:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.183+0000 7f8f4e7fc640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8f440a33b0 0x7f8f440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.183+0000 7f8f5d765640 1 --2- 172.21.15.5:0/2345391431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae46220 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009230 tx=0x7f8f54005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d590fd6aedb0fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.183+0000 7f8f4e7fc640 1 -- 172.21.15.5:0/507050037 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f8f500d0470 con 0x7f8f58102ef0 2024-09-19T06:26:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.183+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8f440a33b0 0x7f8f440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.183+0000 7f8f437fe640 1 -- 172.21.15.5:0/2345391431 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54035040 con 0x55893ae45de0 2024-09-19T06:26:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.183+0000 7f8f437fe640 1 -- 172.21.15.5:0/2345391431 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f54004770 con 0x55893ae45de0 2024-09-19T06:26:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.183+0000 7f8f437fe640 1 -- 172.21.15.5:0/2345391431 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54003950 con 0x55893ae45de0 2024-09-19T06:26:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.184+0000 7f8f61da2740 1 -- 172.21.15.5:0/2345391431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 msgr2=0x55893ae46220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.184+0000 7f8f61da2740 1 --2- 172.21.15.5:0/2345391431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae46220 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009230 tx=0x7f8f54005c00 comp rx=0 tx=0).stop 2024-09-19T06:26:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.184+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8f440a33b0 0x7f8f440a5870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55893ae46880 tx=0x7f8f48011f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.184+0000 7f8f61da2740 1 -- 172.21.15.5:0/2345391431 shutdown_connections 2024-09-19T06:26:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.184+0000 7f8f61da2740 1 --2- 172.21.15.5:0/2345391431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae50b80 0x55893ae52f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.184+0000 7f8f61da2740 1 --2- 172.21.15.5:0/2345391431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae46220 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.184+0000 7f8f61da2740 1 --2- 172.21.15.5:0/2345391431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.184+0000 7f8f61da2740 1 -- 172.21.15.5:0/2345391431 >> 172.21.15.5:0/2345391431 conn(0x55893ae06310 msgr2=0x55893ae42470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.184+0000 7f8f61da2740 1 -- 172.21.15.5:0/2345391431 shutdown_connections 2024-09-19T06:26:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.184+0000 7f8f61da2740 1 -- 172.21.15.5:0/2345391431 wait complete. 2024-09-19T06:26:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.185+0000 7f8f61da2740 1 -- start start 2024-09-19T06:26:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.185+0000 7f8f61da2740 1 Processor -- start 2024-09-19T06:26:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.185+0000 7f8f61da2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.185+0000 7f8f61da2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae4d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f61da2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae50b80 0x55893ae4e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f61da2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55893ae53820 con 0x55893ae50b80 2024-09-19T06:26:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f61da2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55893ad5e060 con 0x55893ae45de0 2024-09-19T06:26:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f61da2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55893ae539a0 con 0x55893ae47350 2024-09-19T06:26:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f5cf64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae4d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f5cf64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae4d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36368/0 (socket says 172.21.15.5:36368) 2024-09-19T06:26:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f5cf64640 1 -- 172.21.15.5:0/1251052030 learned_addr learned my addr 172.21.15.5:0/1251052030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f61da2740 10 client.0 _ll_register_callbacks cb 0x55893aab18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f5d765640 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f5df66640 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae50b80 0x55893ae4e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f5cf64640 1 -- 172.21.15.5:0/1251052030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 msgr2=0x55893ae505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f5cf64640 1 -- 172.21.15.5:0/1251052030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae50b80 msgr2=0x55893ae4e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae50b80 0x55893ae4e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f5cf64640 1 -- 172.21.15.5:0/1251052030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55893ab143c0 con 0x55893ae47350 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f5df66640 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae50b80 0x55893ae4e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae4d890 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8f48003d60 tx=0x7f8f48004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a46082086672641c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.186+0000 7f8f40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.187+0000 7f8f40ff9640 1 -- 172.21.15.5:0/1251052030 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4801fe60 con 0x55893ae47350 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.187+0000 7f8f40ff9640 1 -- 172.21.15.5:0/1251052030 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f8f48019040 con 0x55893ae47350 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.187+0000 7f8f61da2740 1 -- 172.21.15.5:0/1251052030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f54004770 con 0x55893ae47350 2024-09-19T06:26:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.187+0000 7f8f40ff9640 1 -- 172.21.15.5:0/1251052030 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f8f48003d90 con 0x55893ae47350 2024-09-19T06:26:29.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.187+0000 7f8f61da2740 10 client.30445 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:29.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.187+0000 7f8f61da2740 1 -- 172.21.15.5:0/1251052030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55893ae5f3a0 con 0x55893ae47350 2024-09-19T06:26:29.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.187+0000 7f8f40ff9640 1 -- 172.21.15.5:0/1251052030 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f54004770 con 0x55893ae47350 2024-09-19T06:26:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.187+0000 7f8f40ff9640 1 -- 172.21.15.5:0/1251052030 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f8f4801d070 con 0x55893ae47350 2024-09-19T06:26:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.188+0000 7f8f61da2740 10 client.30445 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.188+0000 7f8f61da2740 10 client.30445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.188+0000 7f8f61da2740 20 client.30445 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.188+0000 7f8f61da2740 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55893ae60900 0x55893ae62d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.188+0000 7f8f61da2740 4 client.30445 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:26:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.188+0000 7f8f61da2740 1 -- 172.21.15.5:0/1251052030 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55893adb0860 con 0x55893ae60900 2024-09-19T06:26:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.188+0000 7f8f5d765640 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55893ae60900 0x55893ae62d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.188+0000 7f8f40ff9640 10 client.30445 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.188+0000 7f8f5d765640 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55893ae60900 0x55893ae62d20 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d44dad071e1ba82 server_cookie=f10928cd5c8d4dd in_seq=0 out_seq=0 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.189+0000 7f8f40ff9640 1 -- 172.21.15.5:0/1251052030 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55893adb0860 con 0x55893ae60900 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.189+0000 7f8f40ff9640 10 client.30445 handle_command_reply: tid=0 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.189+0000 7f8f61da2740 1 client.30445 shutdown 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.189+0000 7f8f61da2740 1 -- 172.21.15.5:0/1251052030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 msgr2=0x55893ae4d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.189+0000 7f8f61da2740 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae4d890 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8f48003d60 tx=0x7f8f48004730 comp rx=0 tx=0).stop 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.189+0000 7f8f61da2740 1 -- 172.21.15.5:0/1251052030 shutdown_connections 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.189+0000 7f8f61da2740 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55893ae60900 0x55893ae62d20 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.190+0000 7f8f40ff9640 0 client.30445 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.190+0000 7f8f61da2740 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae50b80 0x55893ae4e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.190+0000 7f8f61da2740 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae4d890 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.190+0000 7f8f61da2740 1 --2- 172.21.15.5:0/1251052030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.190+0000 7f8f61da2740 1 -- 172.21.15.5:0/1251052030 >> 172.21.15.5:0/1251052030 conn(0x55893ae06310 msgr2=0x55893ae48170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.190+0000 7f8f61da2740 1 -- 172.21.15.5:0/1251052030 shutdown_connections 2024-09-19T06:26:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.190+0000 7f8f61da2740 1 -- 172.21.15.5:0/1251052030 wait complete. 2024-09-19T06:26:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.190+0000 7f8f61da2740 20 client.30445 trim_cache size 0 max 16384 2024-09-19T06:26:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.210+0000 7f8f61da2740 1 Processor -- start 2024-09-19T06:26:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.210+0000 7f8f61da2740 1 -- start start 2024-09-19T06:26:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f61da2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f61da2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae47350 0x55893ae41f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f61da2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae50b80 0x55893ae42450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f61da2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581ab640 con 0x55893ae47350 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f61da2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581ad180 con 0x55893ae45de0 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f61da2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581abd10 con 0x55893ae50b80 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f5d765640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f5cf64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae47350 0x55893ae41f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f5d765640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46670/0 (socket says 172.21.15.5:46670) 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f5d765640 1 -- 172.21.15.5:0/2027405838 learned_addr learned my addr 172.21.15.5:0/2027405838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f5df66640 1 --2- 172.21.15.5:0/2027405838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae50b80 0x55893ae42450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f5cf64640 1 -- 172.21.15.5:0/2027405838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae50b80 msgr2=0x55893ae42450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/2027405838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae50b80 0x55893ae42450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f5cf64640 1 -- 172.21.15.5:0/2027405838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 msgr2=0x55893ae505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/2027405838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.211+0000 7f8f5cf64640 1 -- 172.21.15.5:0/2027405838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55893ae39320 con 0x55893ae47350 2024-09-19T06:26:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.212+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/2027405838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae47350 0x55893ae41f10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f8f4801d660 tx=0x7f8f480043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a69bb493d6929ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:29.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.212+0000 7f8f42ffd640 1 -- 172.21.15.5:0/2027405838 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48003920 con 0x55893ae47350 2024-09-19T06:26:29.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.212+0000 7f8f42ffd640 1 -- 172.21.15.5:0/2027405838 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f4801db90 con 0x55893ae47350 2024-09-19T06:26:29.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.212+0000 7f8f42ffd640 1 -- 172.21.15.5:0/2027405838 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48033ba0 con 0x55893ae47350 2024-09-19T06:26:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.213+0000 7f8f61da2740 1 -- 172.21.15.5:0/2027405838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae47350 msgr2=0x55893ae41f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.213+0000 7f8f61da2740 1 --2- 172.21.15.5:0/2027405838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae47350 0x55893ae41f10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f8f4801d660 tx=0x7f8f480043f0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.213+0000 7f8f61da2740 1 -- 172.21.15.5:0/2027405838 shutdown_connections 2024-09-19T06:26:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.213+0000 7f8f61da2740 1 --2- 172.21.15.5:0/2027405838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae50b80 0x55893ae42450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.213+0000 7f8f61da2740 1 --2- 172.21.15.5:0/2027405838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae47350 0x55893ae41f10 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.213+0000 7f8f61da2740 1 --2- 172.21.15.5:0/2027405838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae45de0 0x55893ae505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.213+0000 7f8f61da2740 1 -- 172.21.15.5:0/2027405838 >> 172.21.15.5:0/2027405838 conn(0x55893ae06310 msgr2=0x55893ae34a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.213+0000 7f8f61da2740 1 -- 172.21.15.5:0/2027405838 shutdown_connections 2024-09-19T06:26:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.213+0000 7f8f61da2740 1 -- 172.21.15.5:0/2027405838 wait complete. 2024-09-19T06:26:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.213+0000 7f8f61da2740 1 -- start start 2024-09-19T06:26:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.213+0000 7f8f61da2740 1 Processor -- start 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.214+0000 7f8f61da2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae45de0 0x55893ae58fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.214+0000 7f8f61da2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.214+0000 7f8f61da2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae50b80 0x55893ae4a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.214+0000 7f8f61da2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581acab0 con 0x55893ae45de0 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.214+0000 7f8f61da2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581ac3e0 con 0x55893ae50b80 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.214+0000 7f8f61da2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581a9640 con 0x55893ae47350 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.214+0000 7f8f61da2740 10 client.0 _ll_register_callbacks cb 0x7f8f480041e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.214+0000 7f8f5cf64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.214+0000 7f8f5cf64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36398/0 (socket says 172.21.15.5:36398) 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.214+0000 7f8f5cf64640 1 -- 172.21.15.5:0/1902730895 learned_addr learned my addr 172.21.15.5:0/1902730895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.214+0000 7f8f5d765640 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae45de0 0x55893ae58fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.214+0000 7f8f5df66640 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae50b80 0x55893ae4a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f5cf64640 1 -- 172.21.15.5:0/1902730895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae50b80 msgr2=0x55893ae4a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae50b80 0x55893ae4a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f5cf64640 1 -- 172.21.15.5:0/1902730895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae45de0 msgr2=0x55893ae58fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae45de0 0x55893ae58fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f5cf64640 1 -- 172.21.15.5:0/1902730895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55893ae4ac30 con 0x55893ae47350 2024-09-19T06:26:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f5cf64640 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae594e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f4801d4f0 tx=0x7f8f48005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9db74bfe62e1a063 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f40ff9640 1 -- 172.21.15.5:0/1902730895 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48003920 con 0x55893ae47350 2024-09-19T06:26:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f40ff9640 1 -- 172.21.15.5:0/1902730895 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f8f4801d8a0 con 0x55893ae47350 2024-09-19T06:26:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f61da2740 1 -- 172.21.15.5:0/1902730895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f4801db90 con 0x55893ae47350 2024-09-19T06:26:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f40ff9640 1 -- 172.21.15.5:0/1902730895 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f8f4804c050 con 0x55893ae47350 2024-09-19T06:26:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f61da2740 10 client.30451 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.215+0000 7f8f61da2740 1 -- 172.21.15.5:0/1902730895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55893aecb1c0 con 0x55893ae47350 2024-09-19T06:26:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.216+0000 7f8f40ff9640 1 -- 172.21.15.5:0/1902730895 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f4801db90 con 0x55893ae47350 2024-09-19T06:26:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.216+0000 7f8f40ff9640 1 -- 172.21.15.5:0/1902730895 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f8f4803b030 con 0x55893ae47350 2024-09-19T06:26:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.216+0000 7f8f61da2740 10 client.30451 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.216+0000 7f8f61da2740 10 client.30451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.216+0000 7f8f61da2740 20 client.30451 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.216+0000 7f8f61da2740 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55893ae60900 0x55893aecd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.216+0000 7f8f61da2740 4 client.30451 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.216+0000 7f8f61da2740 1 -- 172.21.15.5:0/1902730895 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55893adaad70 con 0x55893ae60900 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.216+0000 7f8f5d765640 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55893ae60900 0x55893aecd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f40ff9640 10 client.30451 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f5d765640 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55893ae60900 0x55893aecd7e0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc99cc447824bb2 server_cookie=60bd7e4cee69d67 in_seq=0 out_seq=0 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f40ff9640 1 -- 172.21.15.5:0/1902730895 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55893adaad70 con 0x55893ae60900 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f40ff9640 10 client.30451 handle_command_reply: tid=0 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f61da2740 1 client.30451 shutdown 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f61da2740 1 -- 172.21.15.5:0/1902730895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 msgr2=0x55893ae594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f61da2740 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae594e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f4801d4f0 tx=0x7f8f48005f90 comp rx=0 tx=0).stop 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f61da2740 1 -- 172.21.15.5:0/1902730895 shutdown_connections 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f61da2740 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55893ae60900 0x55893aecd7e0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f61da2740 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55893ae50b80 0x55893ae4a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f61da2740 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55893ae47350 0x55893ae594e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f61da2740 1 --2- 172.21.15.5:0/1902730895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55893ae45de0 0x55893ae58fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f40ff9640 0 client.30451 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.217+0000 7f8f61da2740 1 -- 172.21.15.5:0/1902730895 >> 172.21.15.5:0/1902730895 conn(0x55893ae06310 msgr2=0x55893ae41f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.218+0000 7f8f61da2740 1 -- 172.21.15.5:0/1902730895 shutdown_connections 2024-09-19T06:26:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.218+0000 7f8f61da2740 1 -- 172.21.15.5:0/1902730895 wait complete. 2024-09-19T06:26:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.218+0000 7f8f61da2740 20 client.30451 trim_cache size 0 max 16384 2024-09-19T06:26:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:26:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.218+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/507050037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8f440a33b0 msgr2=0x7f8f440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.218+0000 7f8f5f9c9640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8f440a33b0 0x7f8f440a5870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55893ae46880 tx=0x7f8f48011f90 comp rx=0 tx=0).stop 2024-09-19T06:26:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.218+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/507050037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f58102ef0 msgr2=0x7f8f5819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.218+0000 7f8f5f9c9640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f58102ef0 0x7f8f5819cd40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009fd0 tx=0x7f8f5000ef10 comp rx=0 tx=0).stop 2024-09-19T06:26:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.219+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/507050037 shutdown_connections 2024-09-19T06:26:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.219+0000 7f8f5f9c9640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f58102ef0 0x7f8f5819cd40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.219+0000 7f8f5f9c9640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f58102570 0x7f8f581989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.219+0000 7f8f5f9c9640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8f440a33b0 0x7f8f440a5870 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.219+0000 7f8f5f9c9640 1 --2- 172.21.15.5:0/507050037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f58100e20 0x7f8f58198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.219+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/507050037 >> 172.21.15.5:0/507050037 conn(0x7f8f580fc7d0 msgr2=0x7f8f581048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.219+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/507050037 shutdown_connections 2024-09-19T06:26:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:29.219+0000 7f8f5f9c9640 1 -- 172.21.15.5:0/507050037 wait complete. 2024-09-19T06:26:29.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 95425 8.54 MB/sec execute 473 sec latency 202.798 ms 2024-09-19T06:26:30.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:30 smithi005 ceph-mon[30955]: pgmap v732: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-19T06:26:30.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:30 smithi157 ceph-mon[35346]: pgmap v732: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-19T06:26:30.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:30 smithi049 ceph-mon[34640]: pgmap v732: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-19T06:26:30.596 INFO:tasks.workunit.client.0.smithi005.stdout: 10 95579 8.54 MB/sec execute 474 sec latency 114.503 ms 2024-09-19T06:26:31.276 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:26:31.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:26:31.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 95746 8.53 MB/sec execute 475 sec latency 156.857 ms 2024-09-19T06:26:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:26:32.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:32 smithi157 ceph-mon[35346]: pgmap v733: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-19T06:26:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:32 smithi005 ceph-mon[30955]: pgmap v733: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-19T06:26:32.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:32 smithi049 ceph-mon[34640]: pgmap v733: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-19T06:26:32.596 INFO:tasks.workunit.client.0.smithi005.stdout: 10 95946 8.53 MB/sec execute 476 sec latency 58.547 ms 2024-09-19T06:26:33.596 INFO:tasks.workunit.client.0.smithi005.stdout: 10 96078 8.52 MB/sec execute 477 sec latency 116.773 ms 2024-09-19T06:26:33.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.768+0000 7f9b9b61e640 1 -- 172.21.15.5:0/2104481059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94100e20 msgr2=0x7f9b94101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:33.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.768+0000 7f9b9b61e640 1 --2- 172.21.15.5:0/2104481059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94100e20 0x7f9b94101200 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009900 tx=0x7f9b9002a480 comp rx=0 tx=0).stop 2024-09-19T06:26:33.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.769+0000 7f9b9b61e640 1 -- 172.21.15.5:0/2104481059 shutdown_connections 2024-09-19T06:26:33.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.769+0000 7f9b9b61e640 1 --2- 172.21.15.5:0/2104481059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94102ef0 0x7f9b9410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.769+0000 7f9b9b61e640 1 --2- 172.21.15.5:0/2104481059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b94102570 0x7f9b941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.769+0000 7f9b9b61e640 1 --2- 172.21.15.5:0/2104481059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94100e20 0x7f9b94101200 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.769+0000 7f9b9b61e640 1 -- 172.21.15.5:0/2104481059 >> 172.21.15.5:0/2104481059 conn(0x7f9b940fc7d0 msgr2=0x7f9b940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:33.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.769+0000 7f9b9b61e640 1 -- 172.21.15.5:0/2104481059 shutdown_connections 2024-09-19T06:26:33.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.769+0000 7f9b9b61e640 1 -- 172.21.15.5:0/2104481059 wait complete. 2024-09-19T06:26:33.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.770+0000 7f9b9b61e640 1 Processor -- start 2024-09-19T06:26:33.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.770+0000 7f9b9b61e640 1 -- start start 2024-09-19T06:26:33.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.770+0000 7f9b9b61e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b94100e20 0x7f9b94103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.770+0000 7f9b9b61e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94102570 0x7f9b94103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.770+0000 7f9b9b61e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94102ef0 0x7f9b941041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.770+0000 7f9b9b61e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94110e30 con 0x7f9b94102570 2024-09-19T06:26:33.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.770+0000 7f9b9b61e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94110cb0 con 0x7f9b94100e20 2024-09-19T06:26:33.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.770+0000 7f9b9b61e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94110fb0 con 0x7f9b94102ef0 2024-09-19T06:26:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.770+0000 7f9b98bb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94102570 0x7f9b94103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.771+0000 7f9b99bbb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94102ef0 0x7f9b941041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.771+0000 7f9b993ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b94100e20 0x7f9b94103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.771+0000 7f9b99bbb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94102ef0 0x7f9b941041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36410/0 (socket says 172.21.15.5:36410) 2024-09-19T06:26:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.771+0000 7f9b98bb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94102570 0x7f9b94103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54594/0 (socket says 172.21.15.5:54594) 2024-09-19T06:26:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.771+0000 7f9b99bbb640 1 -- 172.21.15.5:0/130062820 learned_addr learned my addr 172.21.15.5:0/130062820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.771+0000 7f9b993ba640 1 -- 172.21.15.5:0/130062820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94102ef0 msgr2=0x7f9b941041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.771+0000 7f9b993ba640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94102ef0 0x7f9b941041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.771+0000 7f9b993ba640 1 -- 172.21.15.5:0/130062820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94102570 msgr2=0x7f9b94103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.771+0000 7f9b993ba640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94102570 0x7f9b94103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.771+0000 7f9b993ba640 1 -- 172.21.15.5:0/130062820 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b941a6860 con 0x7f9b94100e20 2024-09-19T06:26:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.771+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94102ef0 0x7f9b941041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.771+0000 7f9b993ba640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b94100e20 0x7f9b94103750 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9b90005940 tx=0x7f9b90004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.772+0000 7f9b8a7fc640 1 -- 172.21.15.5:0/130062820 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9003f030 con 0x7f9b94100e20 2024-09-19T06:26:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.772+0000 7f9b9b61e640 1 -- 172.21.15.5:0/130062820 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b941a6af0 con 0x7f9b94100e20 2024-09-19T06:26:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.772+0000 7f9b9b61e640 1 -- 172.21.15.5:0/130062820 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b941a7020 con 0x7f9b94100e20 2024-09-19T06:26:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.772+0000 7f9b8a7fc640 1 -- 172.21.15.5:0/130062820 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b90004440 con 0x7f9b94100e20 2024-09-19T06:26:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.772+0000 7f9b8a7fc640 1 -- 172.21.15.5:0/130062820 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b90035070 con 0x7f9b94100e20 2024-09-19T06:26:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.775+0000 7f9b9d9f7740 1 Processor -- start 2024-09-19T06:26:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.775+0000 7f9b9d9f7740 1 -- start start 2024-09-19T06:26:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.775+0000 7f9b8a7fc640 1 -- 172.21.15.5:0/130062820 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9b9004c020 con 0x7f9b94100e20 2024-09-19T06:26:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.775+0000 7f9b9d9f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0db60 0x5632edf0df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.775+0000 7f9b9d9f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0e480 0x5632edf18800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.775+0000 7f9b9d9f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 0x5632edf1b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.775+0000 7f9b9d9f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5632ede73280 con 0x5632edf0db60 2024-09-19T06:26:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.775+0000 7f9b9d9f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5632ede1ab10 con 0x5632edf18d40 2024-09-19T06:26:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.775+0000 7f9b9d9f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5632ede6f240 con 0x5632edf0e480 2024-09-19T06:26:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.776+0000 7f9b99bbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0db60 0x5632edf0df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.776+0000 7f9b99bbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0db60 0x5632edf0df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54608/0 (socket says 172.21.15.5:54608) 2024-09-19T06:26:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.776+0000 7f9b99bbb640 1 -- 172.21.15.5:0/1927264291 learned_addr learned my addr 172.21.15.5:0/1927264291 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.776+0000 7f9b98bb9640 1 --2- 172.21.15.5:0/1927264291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 0x5632edf1b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.776+0000 7f9b99bbb640 1 -- 172.21.15.5:0/1927264291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0e480 msgr2=0x5632edf18800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.776+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/1927264291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0e480 0x5632edf18800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.776+0000 7f9b99bbb640 1 -- 172.21.15.5:0/1927264291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 msgr2=0x5632edf1b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.776+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/1927264291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 0x5632edf1b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.776+0000 7f9b99bbb640 1 -- 172.21.15.5:0/1927264291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ede271f0 con 0x5632edf0db60 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.776+0000 7f9b98bb9640 1 --2- 172.21.15.5:0/1927264291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 0x5632edf1b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.776+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/1927264291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0db60 0x5632edf0df40 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c011850 tx=0x7f9b8c011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ca5caec3edc79f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.777+0000 7f9b7f7fe640 1 -- 172.21.15.5:0/1927264291 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c011330 con 0x5632edf0db60 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.777+0000 7f9b8a7fc640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9b800a3430 0x7f9b800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.777+0000 7f9b7f7fe640 1 -- 172.21.15.5:0/1927264291 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b8c00fe50 con 0x5632edf0db60 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.777+0000 7f9b7f7fe640 1 -- 172.21.15.5:0/1927264291 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0193e0 con 0x5632edf0db60 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.777+0000 7f9b993ba640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9b800a3430 0x7f9b800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.777+0000 7f9b8a7fc640 1 -- 172.21.15.5:0/130062820 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f9b900efcd0 con 0x7f9b94100e20 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.778+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1927264291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0db60 msgr2=0x5632edf0df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.778+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1927264291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0db60 0x5632edf0df40 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c011850 tx=0x7f9b8c011d10 comp rx=0 tx=0).stop 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.778+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1927264291 shutdown_connections 2024-09-19T06:26:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.778+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1927264291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 0x5632edf1b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.778+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1927264291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0e480 0x5632edf18800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.778+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1927264291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0db60 0x5632edf0df40 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.778+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1927264291 >> 172.21.15.5:0/1927264291 conn(0x5632edf0a0f0 msgr2=0x5632edf0a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.778+0000 7f9b993ba640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9b800a3430 0x7f9b800a58f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x5632edf0af50 tx=0x7f9b9004ae30 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.778+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1927264291 shutdown_connections 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.778+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1927264291 wait complete. 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.778+0000 7f9b9d9f7740 1 -- start start 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.778+0000 7f9b9d9f7740 1 Processor -- start 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.779+0000 7f9b9d9f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf0db60 0x5632edf15b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.779+0000 7f9b9d9f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 0x5632edf160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.779+0000 7f9b9d9f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf18d40 0x5632edf165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.779+0000 7f9b9d9f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5632edf1b910 con 0x5632edf0e480 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.779+0000 7f9b9d9f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5632ede2df10 con 0x5632edf0db60 2024-09-19T06:26:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.779+0000 7f9b9d9f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5632edf1ba90 con 0x5632edf18d40 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.779+0000 7f9b9d9f7740 10 client.0 _ll_register_callbacks cb 0x5632eddf6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.779+0000 7f9b98bb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 0x5632edf160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.779+0000 7f9b98bb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 0x5632edf160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54610/0 (socket says 172.21.15.5:54610) 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.779+0000 7f9b98bb9640 1 -- 172.21.15.5:0/2243748117 learned_addr learned my addr 172.21.15.5:0/2243748117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.779+0000 7f9b99bbb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf18d40 0x5632edf165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.779+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/2243748117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf0db60 0x5632edf15b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.780+0000 7f9b98bb9640 1 -- 172.21.15.5:0/2243748117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf18d40 msgr2=0x5632edf165e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.780+0000 7f9b98bb9640 1 --2- 172.21.15.5:0/2243748117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf18d40 0x5632edf165e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.780+0000 7f9b98bb9640 1 -- 172.21.15.5:0/2243748117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf0db60 msgr2=0x5632edf15b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.780+0000 7f9b98bb9640 1 --2- 172.21.15.5:0/2243748117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf0db60 0x5632edf15b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.780+0000 7f9b98bb9640 1 -- 172.21.15.5:0/2243748117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632edbe46a0 con 0x5632edf0e480 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.780+0000 7f9b98bb9640 1 --2- 172.21.15.5:0/2243748117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 0x5632edf160a0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f9b84012700 tx=0x7f9b84012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231c3fb51ac4b0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:33.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.780+0000 7f9b7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:26:33.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.780+0000 7f9b7cff9640 1 -- 172.21.15.5:0/2243748117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84004030 con 0x5632edf0e480 2024-09-19T06:26:33.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.780+0000 7f9b7cff9640 1 -- 172.21.15.5:0/2243748117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f9b840041d0 con 0x5632edf0e480 2024-09-19T06:26:33.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.780+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/2243748117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b8c00fe50 con 0x5632edf0e480 2024-09-19T06:26:33.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.780+0000 7f9b7cff9640 1 -- 172.21.15.5:0/2243748117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f9b84004410 con 0x5632edf0e480 2024-09-19T06:26:33.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.781+0000 7f9b9d9f7740 10 client.21159 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:33.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.781+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/2243748117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x5632edf27580 con 0x5632edf0e480 2024-09-19T06:26:33.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.781+0000 7f9b7cff9640 1 -- 172.21.15.5:0/2243748117 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b8400fd70 con 0x5632edf0e480 2024-09-19T06:26:33.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.781+0000 7f9b7cff9640 1 -- 172.21.15.5:0/2243748117 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f9b84010430 con 0x5632edf0e480 2024-09-19T06:26:33.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.781+0000 7f9b9d9f7740 10 client.21159 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:33.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.781+0000 7f9b9d9f7740 10 client.21159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:33.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.781+0000 7f9b9d9f7740 20 client.21159 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:33.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.781+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/2243748117 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5632edf28a40 0x5632edf2ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.781+0000 7f9b9d9f7740 4 client.21159 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:26:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.781+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/2243748117 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9b8c00fe50 con 0x5632edf28a40 2024-09-19T06:26:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.781+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/2243748117 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5632edf28a40 0x5632edf2ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.782+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/2243748117 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5632edf28a40 0x5632edf2ae50 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce6757a08bc2508 server_cookie=ce5c6c1af20c74f8 in_seq=0 out_seq=0 2024-09-19T06:26:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.782+0000 7f9b7cff9640 10 client.21159 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.782+0000 7f9b7cff9640 1 -- 172.21.15.5:0/2243748117 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9b8c00fe50 con 0x5632edf28a40 2024-09-19T06:26:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.782+0000 7f9b7cff9640 10 client.21159 handle_command_reply: tid=0 2024-09-19T06:26:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.782+0000 7f9b9d9f7740 1 client.21159 shutdown 2024-09-19T06:26:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.783+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/2243748117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 msgr2=0x5632edf160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.783+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/2243748117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 0x5632edf160a0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f9b84012700 tx=0x7f9b84012bc0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.783+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/2243748117 shutdown_connections 2024-09-19T06:26:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.783+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/2243748117 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5632edf28a40 0x5632edf2ae50 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.783+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/2243748117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf18d40 0x5632edf165e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.783+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/2243748117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 0x5632edf160a0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.783+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/2243748117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf0db60 0x5632edf15b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.783+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/2243748117 >> 172.21.15.5:0/2243748117 conn(0x5632edf0a0f0 msgr2=0x5632edf0fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.783+0000 7f9b7cff9640 0 client.21159 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.783+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/2243748117 shutdown_connections 2024-09-19T06:26:33.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.783+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/2243748117 wait complete. 2024-09-19T06:26:33.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.783+0000 7f9b9d9f7740 20 client.21159 trim_cache size 0 max 16384 2024-09-19T06:26:33.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.810+0000 7f9b9d9f7740 1 Processor -- start 2024-09-19T06:26:33.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.810+0000 7f9b9d9f7740 1 -- start start 2024-09-19T06:26:33.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.810+0000 7f9b9d9f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0db60 0x5632edf105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.810+0000 7f9b9d9f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 0x5632edf179c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.810+0000 7f9b9d9f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 0x5632edf30760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.810+0000 7f9b9d9f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7800e810 con 0x5632edf0e480 2024-09-19T06:26:33.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.810+0000 7f9b9d9f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9b78010350 con 0x5632edf18d40 2024-09-19T06:26:33.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.810+0000 7f9b9d9f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7800eee0 con 0x5632edf0db60 2024-09-19T06:26:33.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.810+0000 7f9b98bb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 0x5632edf179c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b99bbb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 0x5632edf30760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b99bbb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 0x5632edf30760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46724/0 (socket says 172.21.15.5:46724) 2024-09-19T06:26:33.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b98bb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 0x5632edf179c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54620/0 (socket says 172.21.15.5:54620) 2024-09-19T06:26:33.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b99bbb640 1 -- 172.21.15.5:0/1894954674 learned_addr learned my addr 172.21.15.5:0/1894954674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:33.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/1894954674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0db60 0x5632edf105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b99bbb640 1 -- 172.21.15.5:0/1894954674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0db60 msgr2=0x5632edf105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:33.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/1894954674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0db60 0x5632edf105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b99bbb640 1 -- 172.21.15.5:0/1894954674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 msgr2=0x5632edf179c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:33.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/1894954674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 0x5632edf179c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b99bbb640 1 -- 172.21.15.5:0/1894954674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632edf0e2b0 con 0x5632edf18d40 2024-09-19T06:26:33.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b98bb9640 1 --2- 172.21.15.5:0/1894954674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 0x5632edf179c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-19T06:26:33.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/1894954674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 0x5632edf30760 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c011fd0 tx=0x7f9b8c010a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7983c2340c7883eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:33.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b7effd640 1 -- 172.21.15.5:0/1894954674 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0112a0 con 0x5632edf18d40 2024-09-19T06:26:33.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.811+0000 7f9b7effd640 1 -- 172.21.15.5:0/1894954674 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b8c00f9a0 con 0x5632edf18d40 2024-09-19T06:26:33.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.812+0000 7f9b7effd640 1 -- 172.21.15.5:0/1894954674 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c01a5c0 con 0x5632edf18d40 2024-09-19T06:26:33.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.812+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1894954674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 msgr2=0x5632edf30760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:33.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.812+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1894954674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 0x5632edf30760 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c011fd0 tx=0x7f9b8c010a80 comp rx=0 tx=0).stop 2024-09-19T06:26:33.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.813+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1894954674 shutdown_connections 2024-09-19T06:26:33.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.813+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1894954674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf18d40 0x5632edf30760 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.813+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1894954674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf0e480 0x5632edf179c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.813+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1894954674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0db60 0x5632edf105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.813+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1894954674 >> 172.21.15.5:0/1894954674 conn(0x5632edf0a0f0 msgr2=0x5632edf172b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:33.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.813+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1894954674 shutdown_connections 2024-09-19T06:26:33.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.813+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1894954674 wait complete. 2024-09-19T06:26:33.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.813+0000 7f9b9d9f7740 1 -- start start 2024-09-19T06:26:33.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.813+0000 7f9b9d9f7740 1 Processor -- start 2024-09-19T06:26:33.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b9d9f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf0db60 0x5632edf20d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b9d9f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0e480 0x5632edf212b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b9d9f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf18d40 0x5632edf1dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b9d9f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7800fc80 con 0x5632edf18d40 2024-09-19T06:26:33.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b9d9f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7800f5b0 con 0x5632edf0db60 2024-09-19T06:26:33.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b9d9f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7800c810 con 0x5632edf0e480 2024-09-19T06:26:33.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b9d9f7740 10 client.0 _ll_register_callbacks cb 0x7f9b84009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:33.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b99bbb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf0db60 0x5632edf20d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b99bbb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf0db60 0x5632edf20d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46730/0 (socket says 172.21.15.5:46730) 2024-09-19T06:26:33.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b99bbb640 1 -- 172.21.15.5:0/1510791049 learned_addr learned my addr 172.21.15.5:0/1510791049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:33.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b98bb9640 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0e480 0x5632edf212b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf18d40 0x5632edf1dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b99bbb640 1 -- 172.21.15.5:0/1510791049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0e480 msgr2=0x5632edf212b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:33.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0e480 0x5632edf212b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b99bbb640 1 -- 172.21.15.5:0/1510791049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf18d40 msgr2=0x5632edf1dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:33.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf18d40 0x5632edf1dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b99bbb640 1 -- 172.21.15.5:0/1510791049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632edf13b00 con 0x5632edf0db60 2024-09-19T06:26:33.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b99bbb640 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf0db60 0x5632edf20d70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c010e30 tx=0x7f9b8c01c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=796ab93db5a60ab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:33.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.814+0000 7f9b7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:26:33.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.815+0000 7f9b7cff9640 1 -- 172.21.15.5:0/1510791049 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00fd10 con 0x5632edf0db60 2024-09-19T06:26:33.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.815+0000 7f9b7cff9640 1 -- 172.21.15.5:0/1510791049 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7f9b8c03f040 con 0x5632edf0db60 2024-09-19T06:26:33.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.815+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1510791049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b8c00f9a0 con 0x5632edf0db60 2024-09-19T06:26:33.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.815+0000 7f9b7cff9640 1 -- 172.21.15.5:0/1510791049 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f9b8c010430 con 0x5632edf0db60 2024-09-19T06:26:33.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.815+0000 7f9b9d9f7740 10 client.30533 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:33.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.815+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1510791049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x5632edf93c60 con 0x5632edf0db60 2024-09-19T06:26:33.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.815+0000 7f9b7cff9640 1 -- 172.21.15.5:0/1510791049 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b8c00f9a0 con 0x5632edf0db60 2024-09-19T06:26:33.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.816+0000 7f9b7cff9640 1 -- 172.21.15.5:0/1510791049 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f9b8c042020 con 0x5632edf0db60 2024-09-19T06:26:33.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.816+0000 7f9b9d9f7740 10 client.30533 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:33.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.816+0000 7f9b9d9f7740 10 client.30533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:33.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.816+0000 7f9b9d9f7740 20 client.30533 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:33.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.816+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5632edf94130 0x5632edf96520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:33.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.816+0000 7f9b9d9f7740 4 client.30533 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:26:33.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.816+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1510791049 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5632ede807f0 con 0x5632edf94130 2024-09-19T06:26:33.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.816+0000 7f9b98bb9640 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5632edf94130 0x5632edf96520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:33.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.817+0000 7f9b7cff9640 10 client.30533 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:33.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.817+0000 7f9b98bb9640 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5632edf94130 0x5632edf96520 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa46d17075a7d7df server_cookie=3c4b08eb5341d7b8 in_seq=0 out_seq=0 2024-09-19T06:26:33.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.818+0000 7f9b7cff9640 1 -- 172.21.15.5:0/1510791049 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5632ede807f0 con 0x5632edf94130 2024-09-19T06:26:33.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.818+0000 7f9b7cff9640 10 client.30533 handle_command_reply: tid=0 2024-09-19T06:26:33.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.818+0000 7f9b9d9f7740 1 client.30533 shutdown 2024-09-19T06:26:33.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.818+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1510791049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf0db60 msgr2=0x5632edf20d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:33.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.818+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf0db60 0x5632edf20d70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c010e30 tx=0x7f9b8c01c040 comp rx=0 tx=0).stop 2024-09-19T06:26:33.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.818+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1510791049 shutdown_connections 2024-09-19T06:26:33.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.818+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5632edf94130 0x5632edf96520 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.818+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632edf18d40 0x5632edf1dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.818+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632edf0e480 0x5632edf212b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.818+0000 7f9b9d9f7740 1 --2- 172.21.15.5:0/1510791049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632edf0db60 0x5632edf20d70 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.818+0000 7f9b7cff9640 0 client.30533 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:33.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.818+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1510791049 >> 172.21.15.5:0/1510791049 conn(0x5632edf0a0f0 msgr2=0x5632edf09270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:33.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.819+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1510791049 shutdown_connections 2024-09-19T06:26:33.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.819+0000 7f9b9d9f7740 1 -- 172.21.15.5:0/1510791049 wait complete. 2024-09-19T06:26:33.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.819+0000 7f9b9d9f7740 20 client.30533 trim_cache size 0 max 16384 2024-09-19T06:26:33.837 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:26:33.838 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:26:33.838 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "0701e706-c399-4ccb-b81c-b15c39c33412", 2024-09-19T06:26:33.838 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:26:33.838 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:26:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.820+0000 7f9b9b61e640 1 -- 172.21.15.5:0/130062820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9b800a3430 msgr2=0x7f9b800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:33.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.820+0000 7f9b9b61e640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9b800a3430 0x7f9b800a58f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x5632edf0af50 tx=0x7f9b9004ae30 comp rx=0 tx=0).stop 2024-09-19T06:26:33.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.820+0000 7f9b9b61e640 1 -- 172.21.15.5:0/130062820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b94100e20 msgr2=0x7f9b94103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:33.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.820+0000 7f9b9b61e640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b94100e20 0x7f9b94103750 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9b90005940 tx=0x7f9b90004040 comp rx=0 tx=0).stop 2024-09-19T06:26:33.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.820+0000 7f9b9b61e640 1 -- 172.21.15.5:0/130062820 shutdown_connections 2024-09-19T06:26:33.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.820+0000 7f9b9b61e640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9b800a3430 0x7f9b800a58f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.820+0000 7f9b9b61e640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94102ef0 0x7f9b941041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.820+0000 7f9b9b61e640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94102570 0x7f9b94103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.820+0000 7f9b9b61e640 1 --2- 172.21.15.5:0/130062820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b94100e20 0x7f9b94103750 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:33.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.820+0000 7f9b9b61e640 1 -- 172.21.15.5:0/130062820 >> 172.21.15.5:0/130062820 conn(0x7f9b940fc7d0 msgr2=0x7f9b9406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:33.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.820+0000 7f9b9b61e640 1 -- 172.21.15.5:0/130062820 shutdown_connections 2024-09-19T06:26:33.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:33.820+0000 7f9b9b61e640 1 -- 172.21.15.5:0/130062820 wait complete. 2024-09-19T06:26:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:34 smithi157 ceph-mon[35346]: pgmap v734: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.2 MiB/s wr, 365 op/s 2024-09-19T06:26:34.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:34 smithi005 ceph-mon[30955]: pgmap v734: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.2 MiB/s wr, 365 op/s 2024-09-19T06:26:34.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:34 smithi049 ceph-mon[34640]: pgmap v734: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.2 MiB/s wr, 365 op/s 2024-09-19T06:26:34.582 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:26:34.595 INFO:tasks.workunit.client.0.smithi005.stdout: 10 96227 8.53 MB/sec execute 478 sec latency 93.976 ms 2024-09-19T06:26:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:26:35.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:26:35.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:26:35.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:26:35.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:26:35.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:26:35.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:26:35.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:26:35.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:26:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:26:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:26:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:26:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:35 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:26:35.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:26:35.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:26:35.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:26:35.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:26:35.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:26:35.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:26:35.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:26:35.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:26:35.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:26:35.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:26:35.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:35 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:26:35.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:26:35.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:26:35.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:26:35.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:26:35.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:26:35.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:26:35.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:26:35.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:26:35.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:26:35.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:26:35.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.398 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:35 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:26:35.596 INFO:tasks.workunit.client.0.smithi005.stdout: 10 96431 8.56 MB/sec execute 479 sec latency 79.839 ms 2024-09-19T06:26:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.895+0000 7ff3776c7640 1 -- 172.21.15.5:0/3783802176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370102ef0 msgr2=0x7ff37010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.895+0000 7ff3776c7640 1 --2- 172.21.15.5:0/3783802176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370102ef0 0x7ff37010da40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff368009c10 tx=0x7ff36802a770 comp rx=0 tx=0).stop 2024-09-19T06:26:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.895+0000 7ff3776c7640 1 -- 172.21.15.5:0/3783802176 shutdown_connections 2024-09-19T06:26:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.895+0000 7ff3776c7640 1 --2- 172.21.15.5:0/3783802176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370102ef0 0x7ff37010da40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.895+0000 7ff3776c7640 1 --2- 172.21.15.5:0/3783802176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff370102570 0x7ff3701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.895+0000 7ff3776c7640 1 --2- 172.21.15.5:0/3783802176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff370100e20 0x7ff370101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.895+0000 7ff3776c7640 1 -- 172.21.15.5:0/3783802176 >> 172.21.15.5:0/3783802176 conn(0x7ff3700fc7d0 msgr2=0x7ff3700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.895+0000 7ff3776c7640 1 -- 172.21.15.5:0/3783802176 shutdown_connections 2024-09-19T06:26:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.895+0000 7ff3776c7640 1 -- 172.21.15.5:0/3783802176 wait complete. 2024-09-19T06:26:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.896+0000 7ff3776c7640 1 Processor -- start 2024-09-19T06:26:35.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.896+0000 7ff3776c7640 1 -- start start 2024-09-19T06:26:35.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.896+0000 7ff3776c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff370100e20 0x7ff3701a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.896+0000 7ff3776c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff370102570 0x7ff3701a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.896+0000 7ff3776c7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370102ef0 0x7ff37019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.896+0000 7ff3776c7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff370110e30 con 0x7ff370100e20 2024-09-19T06:26:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.896+0000 7ff3776c7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff370110cb0 con 0x7ff370102ef0 2024-09-19T06:26:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.896+0000 7ff3776c7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff370110fb0 con 0x7ff370102570 2024-09-19T06:26:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.897+0000 7ff375463640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff370100e20 0x7ff3701a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.897+0000 7ff375463640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff370100e20 0x7ff3701a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60820/0 (socket says 172.21.15.5:60820) 2024-09-19T06:26:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.897+0000 7ff375463640 1 -- 172.21.15.5:0/2210857119 learned_addr learned my addr 172.21.15.5:0/2210857119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.897+0000 7ff374c62640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff370102570 0x7ff3701a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.897+0000 7ff375c64640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370102ef0 0x7ff37019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.897+0000 7ff375463640 1 -- 172.21.15.5:0/2210857119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff370102570 msgr2=0x7ff3701a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.897+0000 7ff375463640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff370102570 0x7ff3701a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.897+0000 7ff375463640 1 -- 172.21.15.5:0/2210857119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370102ef0 msgr2=0x7ff37019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.897+0000 7ff375463640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370102ef0 0x7ff37019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.897+0000 7ff375463640 1 -- 172.21.15.5:0/2210857119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37019f230 con 0x7ff370100e20 2024-09-19T06:26:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.897+0000 7ff375463640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff370100e20 0x7ff3701a46b0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00e9f0 tx=0x7ff36c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.898+0000 7ff3667fc640 1 -- 172.21.15.5:0/2210857119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00cd80 con 0x7ff370100e20 2024-09-19T06:26:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.898+0000 7ff3776c7640 1 -- 172.21.15.5:0/2210857119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37019f520 con 0x7ff370100e20 2024-09-19T06:26:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.898+0000 7ff3776c7640 1 -- 172.21.15.5:0/2210857119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3701ab510 con 0x7ff370100e20 2024-09-19T06:26:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.898+0000 7ff3667fc640 1 -- 172.21.15.5:0/2210857119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff36c0108e0 con 0x7ff370100e20 2024-09-19T06:26:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.898+0000 7ff3667fc640 1 -- 172.21.15.5:0/2210857119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c018920 con 0x7ff370100e20 2024-09-19T06:26:35.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.900+0000 7ff3667fc640 1 -- 172.21.15.5:0/2210857119 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff36c010430 con 0x7ff370100e20 2024-09-19T06:26:35.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.901+0000 7ff379aa0740 1 Processor -- start 2024-09-19T06:26:35.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.901+0000 7ff379aa0740 1 -- start start 2024-09-19T06:26:35.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.902+0000 7ff379aa0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedec4a0 0x5576cedec880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.902+0000 7ff379aa0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedecdc0 0x5576cedf7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.902+0000 7ff379aa0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf7680 0x5576cedf9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.902+0000 7ff379aa0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5576ced522e0 con 0x5576cedecdc0 2024-09-19T06:26:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.902+0000 7ff379aa0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5576cecf9ce0 con 0x5576cedec4a0 2024-09-19T06:26:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.902+0000 7ff379aa0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5576ced4e2a0 con 0x5576cedf7680 2024-09-19T06:26:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.902+0000 7ff374c62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf7680 0x5576cedf9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.902+0000 7ff375463640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedecdc0 0x5576cedf7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.902+0000 7ff375463640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedecdc0 0x5576cedf7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60834/0 (socket says 172.21.15.5:60834) 2024-09-19T06:26:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.902+0000 7ff375463640 1 -- 172.21.15.5:0/1533536059 learned_addr learned my addr 172.21.15.5:0/1533536059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.902+0000 7ff374c62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf7680 0x5576cedf9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:35458/0 (socket says 172.21.15.5:35458) 2024-09-19T06:26:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.903+0000 7ff375c64640 1 --2- 172.21.15.5:0/1533536059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedec4a0 0x5576cedec880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.903+0000 7ff3667fc640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff35c0a3430 0x7ff35c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.903+0000 7ff375463640 1 -- 172.21.15.5:0/1533536059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf7680 msgr2=0x5576cedf9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.903+0000 7ff375463640 1 --2- 172.21.15.5:0/1533536059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf7680 0x5576cedf9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.903+0000 7ff375463640 1 -- 172.21.15.5:0/1533536059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedec4a0 msgr2=0x5576cedec880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:35.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.903+0000 7ff375463640 1 --2- 172.21.15.5:0/1533536059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedec4a0 0x5576cedec880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.903+0000 7ff375463640 1 -- 172.21.15.5:0/1533536059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576cecde440 con 0x5576cedecdc0 2024-09-19T06:26:35.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.903+0000 7ff375463640 1 --2- 172.21.15.5:0/1533536059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedecdc0 0x5576cedf7140 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0cd590 tx=0x7ff36c0b0310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4272ed6704b59f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:35.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.903+0000 7ff375c64640 1 --2- 172.21.15.5:0/1533536059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedec4a0 0x5576cedec880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:26:35.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.904+0000 7ff35b7fe640 1 -- 172.21.15.5:0/1533536059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c09c030 con 0x5576cedecdc0 2024-09-19T06:26:35.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.904+0000 7ff3667fc640 1 -- 172.21.15.5:0/2210857119 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7ff36c0cfef0 con 0x7ff370100e20 2024-09-19T06:26:35.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.904+0000 7ff35b7fe640 1 -- 172.21.15.5:0/1533536059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff36c010720 con 0x5576cedecdc0 2024-09-19T06:26:35.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.904+0000 7ff35b7fe640 1 -- 172.21.15.5:0/1533536059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c018ac0 con 0x5576cedecdc0 2024-09-19T06:26:35.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.904+0000 7ff375c64640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff35c0a3430 0x7ff35c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.905+0000 7ff375c64640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff35c0a3430 0x7ff35c0a58f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff368004af0 tx=0x7ff368009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:35.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.905+0000 7ff379aa0740 1 -- 172.21.15.5:0/1533536059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedecdc0 msgr2=0x5576cedf7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:35.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.905+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1533536059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedecdc0 0x5576cedf7140 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0cd590 tx=0x7ff36c0b0310 comp rx=0 tx=0).stop 2024-09-19T06:26:35.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.905+0000 7ff379aa0740 1 -- 172.21.15.5:0/1533536059 shutdown_connections 2024-09-19T06:26:35.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.905+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1533536059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf7680 0x5576cedf9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.905+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1533536059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedecdc0 0x5576cedf7140 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.905+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1533536059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedec4a0 0x5576cedec880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.905+0000 7ff379aa0740 1 -- 172.21.15.5:0/1533536059 >> 172.21.15.5:0/1533536059 conn(0x5576cede8a30 msgr2=0x5576cede8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:35.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.905+0000 7ff379aa0740 1 -- 172.21.15.5:0/1533536059 shutdown_connections 2024-09-19T06:26:35.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.905+0000 7ff379aa0740 1 -- 172.21.15.5:0/1533536059 wait complete. 2024-09-19T06:26:35.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.906+0000 7ff379aa0740 1 -- start start 2024-09-19T06:26:35.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.906+0000 7ff379aa0740 1 Processor -- start 2024-09-19T06:26:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.907+0000 7ff379aa0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedec4a0 0x5576cedf70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.907+0000 7ff375463640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedec4a0 0x5576cedf70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.907+0000 7ff375463640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedec4a0 0x5576cedf70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60848/0 (socket says 172.21.15.5:60848) 2024-09-19T06:26:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.907+0000 7ff375463640 1 -- 172.21.15.5:0/1388859164 learned_addr learned my addr 172.21.15.5:0/1388859164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.907+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf7680 0x5576cedf4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.907+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf4a50 0x5576cee03750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.907+0000 7ff374c62640 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf7680 0x5576cedf4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.908+0000 7ff379aa0740 1 -- 172.21.15.5:0/1388859164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5576cedfa250 con 0x5576cedec4a0 2024-09-19T06:26:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.908+0000 7ff379aa0740 1 -- 172.21.15.5:0/1388859164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5576ced0d050 con 0x5576cedf4a50 2024-09-19T06:26:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.908+0000 7ff379aa0740 1 -- 172.21.15.5:0/1388859164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5576cedfa3d0 con 0x5576cedf7680 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.908+0000 7ff375c64640 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf4a50 0x5576cee03750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.908+0000 7ff379aa0740 10 client.0 _ll_register_callbacks cb 0x5576ced4d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.908+0000 7ff374c62640 1 -- 172.21.15.5:0/1388859164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf4a50 msgr2=0x5576cee03750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.908+0000 7ff374c62640 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf4a50 0x5576cee03750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.908+0000 7ff374c62640 1 -- 172.21.15.5:0/1388859164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedec4a0 msgr2=0x5576cedf70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.908+0000 7ff374c62640 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedec4a0 0x5576cedf70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.908+0000 7ff374c62640 1 -- 172.21.15.5:0/1388859164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576ceac33c0 con 0x5576cedf7680 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.908+0000 7ff375c64640 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf4a50 0x5576cee03750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.909+0000 7ff374c62640 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf7680 0x5576cedf4360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff37019fcc0 tx=0x7ff360012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d876f1de702a3458 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.909+0000 7ff358ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.909+0000 7ff358ff9640 1 -- 172.21.15.5:0/1388859164 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360004020 con 0x5576cedf7680 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.909+0000 7ff358ff9640 1 -- 172.21.15.5:0/1388859164 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7ff3600041c0 con 0x5576cedf7680 2024-09-19T06:26:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.909+0000 7ff379aa0740 1 -- 172.21.15.5:0/1388859164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5576ced5f850 con 0x5576cedf7680 2024-09-19T06:26:35.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.909+0000 7ff358ff9640 1 -- 172.21.15.5:0/1388859164 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7ff3600043c0 con 0x5576cedf7680 2024-09-19T06:26:35.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.909+0000 7ff379aa0740 10 client.30478 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:35.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.909+0000 7ff379aa0740 1 -- 172.21.15.5:0/1388859164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x5576cee06140 con 0x5576cedf7680 2024-09-19T06:26:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.910+0000 7ff358ff9640 1 -- 172.21.15.5:0/1388859164 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5576ced5f850 con 0x5576cedf7680 2024-09-19T06:26:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.910+0000 7ff358ff9640 1 -- 172.21.15.5:0/1388859164 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7ff360003cc0 con 0x5576cedf7680 2024-09-19T06:26:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.910+0000 7ff379aa0740 10 client.30478 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.910+0000 7ff379aa0740 10 client.30478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.910+0000 7ff379aa0740 20 client.30478 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.910+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5576cee07550 0x5576cee09990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.910+0000 7ff379aa0740 4 client.30478 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:26:35.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.910+0000 7ff379aa0740 1 -- 172.21.15.5:0/1388859164 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576ced59d60 con 0x5576cee07550 2024-09-19T06:26:35.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.910+0000 7ff375463640 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5576cee07550 0x5576cee09990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.911+0000 7ff358ff9640 10 client.30478 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:35.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.911+0000 7ff375463640 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5576cee07550 0x5576cee09990 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f796e8cdcd4d0f server_cookie=437c1dbc051a1ed in_seq=0 out_seq=0 2024-09-19T06:26:35.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff358ff9640 1 -- 172.21.15.5:0/1388859164 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576ced59d60 con 0x5576cee07550 2024-09-19T06:26:35.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff358ff9640 10 client.30478 handle_command_reply: tid=0 2024-09-19T06:26:35.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff379aa0740 1 client.30478 shutdown 2024-09-19T06:26:35.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff379aa0740 1 -- 172.21.15.5:0/1388859164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf7680 msgr2=0x5576cedf4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:35.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf7680 0x5576cedf4360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff37019fcc0 tx=0x7ff360012b60 comp rx=0 tx=0).stop 2024-09-19T06:26:35.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff379aa0740 1 -- 172.21.15.5:0/1388859164 shutdown_connections 2024-09-19T06:26:35.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5576cee07550 0x5576cee09990 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf4a50 0x5576cee03750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf7680 0x5576cedf4360 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1388859164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedec4a0 0x5576cedf70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff358ff9640 0 client.30478 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:35.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff379aa0740 1 -- 172.21.15.5:0/1388859164 >> 172.21.15.5:0/1388859164 conn(0x5576cede8a30 msgr2=0x5576cede7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:35.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff379aa0740 1 -- 172.21.15.5:0/1388859164 shutdown_connections 2024-09-19T06:26:35.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.912+0000 7ff379aa0740 1 -- 172.21.15.5:0/1388859164 wait complete. 2024-09-19T06:26:35.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.913+0000 7ff379aa0740 20 client.30478 trim_cache size 0 max 16384 2024-09-19T06:26:35.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.945+0000 7ff379aa0740 1 Processor -- start 2024-09-19T06:26:35.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.945+0000 7ff379aa0740 1 -- start start 2024-09-19T06:26:35.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff379aa0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedec4a0 0x5576cee09560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff379aa0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf4a50 0x5576cee0f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff379aa0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedf7680 0x5576cee0f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff379aa0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff35400e8f0 con 0x5576cedf7680 2024-09-19T06:26:35.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff379aa0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff354010430 con 0x5576cedf4a50 2024-09-19T06:26:35.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff379aa0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff35400efc0 con 0x5576cedec4a0 2024-09-19T06:26:35.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff375c64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedf7680 0x5576cee0f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff374c62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf4a50 0x5576cee0f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff375463640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedec4a0 0x5576cee09560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff375c64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedf7680 0x5576cee0f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60858/0 (socket says 172.21.15.5:60858) 2024-09-19T06:26:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff374c62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf4a50 0x5576cee0f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44974/0 (socket says 172.21.15.5:44974) 2024-09-19T06:26:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff375463640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedec4a0 0x5576cee09560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:35464/0 (socket says 172.21.15.5:35464) 2024-09-19T06:26:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff375c64640 1 -- 172.21.15.5:0/590290231 learned_addr learned my addr 172.21.15.5:0/590290231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff375c64640 1 -- 172.21.15.5:0/590290231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedec4a0 msgr2=0x5576cee09560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff375c64640 1 --2- 172.21.15.5:0/590290231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedec4a0 0x5576cee09560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff375c64640 1 -- 172.21.15.5:0/590290231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf4a50 msgr2=0x5576cee0f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff375c64640 1 --2- 172.21.15.5:0/590290231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf4a50 0x5576cee0f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff375c64640 1 -- 172.21.15.5:0/590290231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576cedf3310 con 0x5576cedf7680 2024-09-19T06:26:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.946+0000 7ff375463640 1 --2- 172.21.15.5:0/590290231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedec4a0 0x5576cee09560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:26:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.947+0000 7ff375c64640 1 --2- 172.21.15.5:0/590290231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedf7680 0x5576cee0f7a0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff368009c10 tx=0x7ff36803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa81359767d23d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.947+0000 7ff35affd640 1 -- 172.21.15.5:0/590290231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368050070 con 0x5576cedf7680 2024-09-19T06:26:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.947+0000 7ff35affd640 1 -- 172.21.15.5:0/590290231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff368044030 con 0x5576cedf7680 2024-09-19T06:26:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.947+0000 7ff35affd640 1 -- 172.21.15.5:0/590290231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368036040 con 0x5576cedf7680 2024-09-19T06:26:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.948+0000 7ff379aa0740 1 -- 172.21.15.5:0/590290231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedf7680 msgr2=0x5576cee0f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.948+0000 7ff379aa0740 1 --2- 172.21.15.5:0/590290231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedf7680 0x5576cee0f7a0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff368009c10 tx=0x7ff36803e000 comp rx=0 tx=0).stop 2024-09-19T06:26:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.948+0000 7ff379aa0740 1 -- 172.21.15.5:0/590290231 shutdown_connections 2024-09-19T06:26:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.948+0000 7ff379aa0740 1 --2- 172.21.15.5:0/590290231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedf7680 0x5576cee0f7a0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.948+0000 7ff379aa0740 1 --2- 172.21.15.5:0/590290231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf4a50 0x5576cee0f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.948+0000 7ff379aa0740 1 --2- 172.21.15.5:0/590290231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedec4a0 0x5576cee09560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.948+0000 7ff379aa0740 1 -- 172.21.15.5:0/590290231 >> 172.21.15.5:0/590290231 conn(0x5576cede8a30 msgr2=0x5576cedf63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.948+0000 7ff379aa0740 1 -- 172.21.15.5:0/590290231 shutdown_connections 2024-09-19T06:26:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.949+0000 7ff379aa0740 1 -- 172.21.15.5:0/590290231 wait complete. 2024-09-19T06:26:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.949+0000 7ff379aa0740 1 -- start start 2024-09-19T06:26:35.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.949+0000 7ff379aa0740 1 Processor -- start 2024-09-19T06:26:35.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff379aa0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedec4a0 0x5576cedffa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff379aa0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf4a50 0x5576cedfff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff379aa0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf7680 0x5576cedfce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff379aa0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff35400fd60 con 0x5576cedec4a0 2024-09-19T06:26:35.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff379aa0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff35400f690 con 0x5576cedf7680 2024-09-19T06:26:35.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff379aa0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff35400c8f0 con 0x5576cedf4a50 2024-09-19T06:26:35.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff379aa0740 10 client.0 _ll_register_callbacks cb 0x7ff360009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:35.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff374c62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf4a50 0x5576cedfff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff374c62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf4a50 0x5576cedfff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:35468/0 (socket says 172.21.15.5:35468) 2024-09-19T06:26:35.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff374c62640 1 -- 172.21.15.5:0/1347078643 learned_addr learned my addr 172.21.15.5:0/1347078643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff375c64640 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf7680 0x5576cedfce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff374c62640 1 -- 172.21.15.5:0/1347078643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf7680 msgr2=0x5576cedfce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff374c62640 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf7680 0x5576cedfce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.950+0000 7ff374c62640 1 -- 172.21.15.5:0/1347078643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedec4a0 msgr2=0x5576cedffa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.951+0000 7ff375463640 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedec4a0 0x5576cedffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.951+0000 7ff374c62640 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedec4a0 0x5576cedffa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.951+0000 7ff374c62640 1 -- 172.21.15.5:0/1347078643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576cedfd650 con 0x5576cedf4a50 2024-09-19T06:26:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.951+0000 7ff375463640 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedec4a0 0x5576cedffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:26:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.951+0000 7ff374c62640 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf4a50 0x5576cedfff40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff360006010 tx=0x7ff360009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5887bf4a0e218acc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.951+0000 7ff358ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.951+0000 7ff358ff9640 1 -- 172.21.15.5:0/1347078643 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36001cd30 con 0x5576cedf4a50 2024-09-19T06:26:35.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.951+0000 7ff358ff9640 1 -- 172.21.15.5:0/1347078643 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7ff36000f8f0 con 0x5576cedf4a50 2024-09-19T06:26:35.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.952+0000 7ff379aa0740 1 -- 172.21.15.5:0/1347078643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff368044030 con 0x5576cedf4a50 2024-09-19T06:26:35.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.952+0000 7ff358ff9640 1 -- 172.21.15.5:0/1347078643 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7ff36000fb50 con 0x5576cedf4a50 2024-09-19T06:26:35.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.952+0000 7ff379aa0740 10 client.30484 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:35.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.952+0000 7ff379aa0740 1 -- 172.21.15.5:0/1347078643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x5576cee71e80 con 0x5576cedf4a50 2024-09-19T06:26:35.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.952+0000 7ff358ff9640 1 -- 172.21.15.5:0/1347078643 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff368044030 con 0x5576cedf4a50 2024-09-19T06:26:35.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.952+0000 7ff358ff9640 1 -- 172.21.15.5:0/1347078643 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7ff360037ae0 con 0x5576cedf4a50 2024-09-19T06:26:35.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.952+0000 7ff379aa0740 10 client.30484 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:35.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.952+0000 7ff379aa0740 10 client.30484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:35.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.952+0000 7ff379aa0740 20 client.30484 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:35.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.952+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5576cee72620 0x5576cee74a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:35.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.953+0000 7ff379aa0740 4 client.30484 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:26:35.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.953+0000 7ff379aa0740 1 -- 172.21.15.5:0/1347078643 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5576cecd6210 con 0x5576cee72620 2024-09-19T06:26:35.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.953+0000 7ff375463640 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5576cee72620 0x5576cee74a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:35.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.953+0000 7ff375463640 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5576cee72620 0x5576cee74a40 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78dfb52b6a96bd server_cookie=e0f702436d92a942 in_seq=0 out_seq=0 2024-09-19T06:26:35.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.953+0000 7ff358ff9640 10 client.30484 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:35.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.954+0000 7ff358ff9640 1 -- 172.21.15.5:0/1347078643 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5576cecd6210 con 0x5576cee72620 2024-09-19T06:26:35.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.954+0000 7ff358ff9640 10 client.30484 handle_command_reply: tid=0 2024-09-19T06:26:35.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.955+0000 7ff379aa0740 1 client.30484 shutdown 2024-09-19T06:26:35.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.955+0000 7ff379aa0740 1 -- 172.21.15.5:0/1347078643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf4a50 msgr2=0x5576cedfff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:35.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.955+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf4a50 0x5576cedfff40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff360006010 tx=0x7ff360009ea0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.955+0000 7ff379aa0740 1 -- 172.21.15.5:0/1347078643 shutdown_connections 2024-09-19T06:26:35.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.955+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5576cee72620 0x5576cee74a40 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.955+0000 7ff358ff9640 0 client.30484 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:35.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.955+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576cedf7680 0x5576cedfce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.955+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576cedf4a50 0x5576cedfff40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.955+0000 7ff379aa0740 1 --2- 172.21.15.5:0/1347078643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576cedec4a0 0x5576cedffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.955+0000 7ff379aa0740 1 -- 172.21.15.5:0/1347078643 >> 172.21.15.5:0/1347078643 conn(0x5576cede8a30 msgr2=0x5576cede7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:35.974 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:26:35.974 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:26:35.974 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:26:35.974 INFO:teuthology.orchestra.run.smithi005.stdout: "0701e706-c399-4ccb-b81c-b15c39c33412": { 2024-09-19T06:26:35.975 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:26:35.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0701e706-c399-4ccb-b81c-b15c39c33412", 2024-09-19T06:26:35.975 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:26:35.975 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:26:35.975 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:26:35.976 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:26:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.955+0000 7ff379aa0740 1 -- 172.21.15.5:0/1347078643 shutdown_connections 2024-09-19T06:26:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.955+0000 7ff379aa0740 1 -- 172.21.15.5:0/1347078643 wait complete. 2024-09-19T06:26:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.956+0000 7ff379aa0740 20 client.30484 trim_cache size 0 max 16384 2024-09-19T06:26:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.956+0000 7ff3776c7640 1 -- 172.21.15.5:0/2210857119 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff35c0a3430 msgr2=0x7ff35c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.956+0000 7ff3776c7640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff35c0a3430 0x7ff35c0a58f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff368004af0 tx=0x7ff368009500 comp rx=0 tx=0).stop 2024-09-19T06:26:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.957+0000 7ff3776c7640 1 -- 172.21.15.5:0/2210857119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff370100e20 msgr2=0x7ff3701a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.957+0000 7ff3776c7640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff370100e20 0x7ff3701a46b0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00e9f0 tx=0x7ff36c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.957+0000 7ff3776c7640 1 -- 172.21.15.5:0/2210857119 shutdown_connections 2024-09-19T06:26:35.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.957+0000 7ff3776c7640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370102ef0 0x7ff37019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.957+0000 7ff3776c7640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff35c0a3430 0x7ff35c0a58f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.957+0000 7ff3776c7640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff370102570 0x7ff3701a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.957+0000 7ff3776c7640 1 --2- 172.21.15.5:0/2210857119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff370100e20 0x7ff3701a46b0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:35.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.957+0000 7ff3776c7640 1 -- 172.21.15.5:0/2210857119 >> 172.21.15.5:0/2210857119 conn(0x7ff3700fc7d0 msgr2=0x7ff37006a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:35.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.957+0000 7ff3776c7640 1 -- 172.21.15.5:0/2210857119 shutdown_connections 2024-09-19T06:26:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:26:35.957+0000 7ff3776c7640 1 -- 172.21.15.5:0/2210857119 wait complete. 2024-09-19T06:26:36.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:36 smithi005 ceph-mon[30955]: pgmap v735: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 255 op/s 2024-09-19T06:26:36.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:36 smithi049 ceph-mon[34640]: pgmap v735: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 255 op/s 2024-09-19T06:26:36.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:36 smithi157 ceph-mon[35346]: pgmap v735: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 255 op/s 2024-09-19T06:26:36.596 INFO:tasks.workunit.client.0.smithi005.stdout: 10 96570 8.56 MB/sec execute 480 sec latency 137.622 ms 2024-09-19T06:26:37.597 INFO:tasks.workunit.client.0.smithi005.stdout: 10 96723 8.56 MB/sec execute 481 sec latency 132.556 ms 2024-09-19T06:26:38.340 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:38 smithi005 ceph-mon[30955]: pgmap v736: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.3 MiB/s wr, 427 op/s 2024-09-19T06:26:38.377 INFO:tasks.cephfs.filesystem:scrub status for tag:0701e706-c399-4ccb-b81c-b15c39c33412 - {'path': '/', 'tag': '0701e706-c399-4ccb-b81c-b15c39c33412', 'options': 'recursive,force'} 2024-09-19T06:26:38.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:38 smithi049 ceph-mon[34640]: pgmap v736: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.3 MiB/s wr, 427 op/s 2024-09-19T06:26:38.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:38 smithi157 ceph-mon[35346]: pgmap v736: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.3 MiB/s wr, 427 op/s 2024-09-19T06:26:38.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 96857 8.55 MB/sec execute 482 sec latency 99.403 ms 2024-09-19T06:26:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.379+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1732881614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec0101760 msgr2=0x7fcec0101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.379+0000 7fcec7bd6640 1 --2- 172.21.15.5:0/1732881614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec0101760 0x7fcec0101b40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009900 tx=0x7fcebc02a3f0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.379+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1732881614 shutdown_connections 2024-09-19T06:26:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.379+0000 7fcec7bd6640 1 --2- 172.21.15.5:0/1732881614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec0105da0 0x7fcec010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.379+0000 7fcec7bd6640 1 --2- 172.21.15.5:0/1732881614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec0102110 0x7fcec0105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.379+0000 7fcec7bd6640 1 --2- 172.21.15.5:0/1732881614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec0101760 0x7fcec0101b40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.379+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1732881614 >> 172.21.15.5:0/1732881614 conn(0x7fcec00fba00 msgr2=0x7fcec00fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.380+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1732881614 shutdown_connections 2024-09-19T06:26:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.380+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1732881614 wait complete. 2024-09-19T06:26:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.380+0000 7fcec7bd6640 1 Processor -- start 2024-09-19T06:26:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.380+0000 7fcec7bd6640 1 -- start start 2024-09-19T06:26:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec7bd6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec0101760 0x7fcec01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec7bd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec0105da0 0x7fcec01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec7bd6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec019cc60 0x7fcec019d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec7bd6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcec0111db0 con 0x7fcec0105da0 2024-09-19T06:26:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec7bd6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcec0111c30 con 0x7fcec019cc60 2024-09-19T06:26:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec7bd6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcec0111f30 con 0x7fcec0101760 2024-09-19T06:26:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec5972640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec0101760 0x7fcec01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec5171640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec0105da0 0x7fcec01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec6173640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec019cc60 0x7fcec019d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec5972640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec0101760 0x7fcec01983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:35484/0 (socket says 172.21.15.5:35484) 2024-09-19T06:26:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec5972640 1 -- 172.21.15.5:0/1109634609 learned_addr learned my addr 172.21.15.5:0/1109634609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec5972640 1 -- 172.21.15.5:0/1109634609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec019cc60 msgr2=0x7fcec019d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec5972640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec019cc60 0x7fcec019d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec5972640 1 -- 172.21.15.5:0/1109634609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec0105da0 msgr2=0x7fcec01988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec5972640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec0105da0 0x7fcec01988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec5972640 1 -- 172.21.15.5:0/1109634609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec019d750 con 0x7fcec0101760 2024-09-19T06:26:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.381+0000 7fcec6173640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec019cc60 0x7fcec019d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.382+0000 7fcec5972640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec0101760 0x7fcec01983a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcebc005940 tx=0x7fcebc0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.382+0000 7fceb2ffd640 1 -- 172.21.15.5:0/1109634609 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc03f030 con 0x7fcec0101760 2024-09-19T06:26:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.382+0000 7fceb2ffd640 1 -- 172.21.15.5:0/1109634609 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcebc0044c0 con 0x7fcec0101760 2024-09-19T06:26:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.382+0000 7fceb2ffd640 1 -- 172.21.15.5:0/1109634609 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc035070 con 0x7fcec0101760 2024-09-19T06:26:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.382+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1109634609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec019d9e0 con 0x7fcec0101760 2024-09-19T06:26:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.382+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1109634609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec01a51a0 con 0x7fcec0101760 2024-09-19T06:26:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.385+0000 7fceb2ffd640 1 -- 172.21.15.5:0/1109634609 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcebc042030 con 0x7fcec0101760 2024-09-19T06:26:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.386+0000 7fceb2ffd640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fceac0a33b0 0x7fceac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.386+0000 7fcecaa50740 1 Processor -- start 2024-09-19T06:26:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.386+0000 7fcec5171640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fceac0a33b0 0x7fceac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.386+0000 7fceb2ffd640 1 -- 172.21.15.5:0/1109634609 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fcebc0efd00 con 0x7fcec0101760 2024-09-19T06:26:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.386+0000 7fcecaa50740 1 -- start start 2024-09-19T06:26:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcecaa50740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef8faf0 0x56209ef8fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcec5972640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef8faf0 0x56209ef8fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcec5972640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef8faf0 0x56209ef8fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60902/0 (socket says 172.21.15.5:60902) 2024-09-19T06:26:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcecaa50740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef90410 0x56209ef9a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcec5972640 1 -- 172.21.15.5:0/616652486 learned_addr learned my addr 172.21.15.5:0/616652486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcec5171640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fceac0a33b0 0x7fceac0a5870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcec00fdff0 tx=0x7fceb4005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcecaa50740 1 --2- 172.21.15.5:0/616652486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef9acb0 0x56209ef9d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcecaa50740 1 -- 172.21.15.5:0/616652486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56209eef42f0 con 0x56209ef8faf0 2024-09-19T06:26:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcecaa50740 1 -- 172.21.15.5:0/616652486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56209ee9c270 con 0x56209ef90410 2024-09-19T06:26:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcecaa50740 1 -- 172.21.15.5:0/616652486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56209eef02b0 con 0x56209ef9acb0 2024-09-19T06:26:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcec5171640 1 --2- 172.21.15.5:0/616652486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef90410 0x56209ef9a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcec6173640 1 --2- 172.21.15.5:0/616652486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef9acb0 0x56209ef9d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcec5972640 1 -- 172.21.15.5:0/616652486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef9acb0 msgr2=0x56209ef9d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcec5972640 1 --2- 172.21.15.5:0/616652486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef9acb0 0x56209ef9d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcec5972640 1 -- 172.21.15.5:0/616652486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef90410 msgr2=0x56209ef9a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcec5972640 1 --2- 172.21.15.5:0/616652486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef90410 0x56209ef9a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.387+0000 7fcec5972640 1 -- 172.21.15.5:0/616652486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209ee9b340 con 0x56209ef8faf0 2024-09-19T06:26:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.388+0000 7fcec5972640 1 --2- 172.21.15.5:0/616652486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef8faf0 0x56209ef8fed0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0c4700 tx=0x7fcebc0bc6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b342311d3046ce72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.388+0000 7fceabfff640 1 -- 172.21.15.5:0/616652486 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc049070 con 0x56209ef8faf0 2024-09-19T06:26:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.388+0000 7fceabfff640 1 -- 172.21.15.5:0/616652486 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcebc0c4c30 con 0x56209ef8faf0 2024-09-19T06:26:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.388+0000 7fceabfff640 1 -- 172.21.15.5:0/616652486 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0c49e0 con 0x56209ef8faf0 2024-09-19T06:26:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.389+0000 7fcecaa50740 1 -- 172.21.15.5:0/616652486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef8faf0 msgr2=0x56209ef8fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.389+0000 7fcecaa50740 1 --2- 172.21.15.5:0/616652486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef8faf0 0x56209ef8fed0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0c4700 tx=0x7fcebc0bc6c0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.389+0000 7fcecaa50740 1 -- 172.21.15.5:0/616652486 shutdown_connections 2024-09-19T06:26:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.389+0000 7fcecaa50740 1 --2- 172.21.15.5:0/616652486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef9acb0 0x56209ef9d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.389+0000 7fcecaa50740 1 --2- 172.21.15.5:0/616652486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef90410 0x56209ef9a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.389+0000 7fcecaa50740 1 --2- 172.21.15.5:0/616652486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef8faf0 0x56209ef8fed0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.389+0000 7fcecaa50740 1 -- 172.21.15.5:0/616652486 >> 172.21.15.5:0/616652486 conn(0x56209ef8c060 msgr2=0x56209ef8c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.389+0000 7fcecaa50740 1 -- 172.21.15.5:0/616652486 shutdown_connections 2024-09-19T06:26:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.389+0000 7fcecaa50740 1 -- 172.21.15.5:0/616652486 wait complete. 2024-09-19T06:26:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.390+0000 7fcecaa50740 1 -- start start 2024-09-19T06:26:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.390+0000 7fcecaa50740 1 Processor -- start 2024-09-19T06:26:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.390+0000 7fcecaa50740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef8faf0 0x56209ef97be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.390+0000 7fcecaa50740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef90410 0x56209ef98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.390+0000 7fcecaa50740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef9acb0 0x56209ef98660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.390+0000 7fcecaa50740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56209ef9d640 con 0x56209ef9acb0 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.390+0000 7fcecaa50740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56209ee7d290 con 0x56209ef8faf0 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.390+0000 7fcecaa50740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56209ef9d7c0 con 0x56209ef90410 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.390+0000 7fcecaa50740 10 client.0 _ll_register_callbacks cb 0x56209eeae140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.390+0000 7fcec5171640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef90410 0x56209ef98120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcec6173640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef9acb0 0x56209ef98660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcec5972640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef8faf0 0x56209ef97be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcec5171640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef90410 0x56209ef98120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:35500/0 (socket says 172.21.15.5:35500) 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcec5171640 1 -- 172.21.15.5:0/1692053275 learned_addr learned my addr 172.21.15.5:0/1692053275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcec5171640 1 -- 172.21.15.5:0/1692053275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef8faf0 msgr2=0x56209ef97be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcec5171640 1 --2- 172.21.15.5:0/1692053275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef8faf0 0x56209ef97be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcec5171640 1 -- 172.21.15.5:0/1692053275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef9acb0 msgr2=0x56209ef98660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcec5171640 1 --2- 172.21.15.5:0/1692053275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef9acb0 0x56209ef98660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcec5171640 1 -- 172.21.15.5:0/1692053275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209ec07210 con 0x56209ef90410 2024-09-19T06:26:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcec6173640 1 --2- 172.21.15.5:0/1692053275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef9acb0 0x56209ef98660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcec5171640 1 --2- 172.21.15.5:0/1692053275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef90410 0x56209ef98120 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fceb4020c20 tx=0x7fceb4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46e44e67521eae16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcea97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcea97fa640 1 -- 172.21.15.5:0/1692053275 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4003cf0 con 0x56209ef90410 2024-09-19T06:26:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcea97fa640 1 -- 172.21.15.5:0/1692053275 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fceb40200b0 con 0x56209ef90410 2024-09-19T06:26:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.391+0000 7fcecaa50740 1 -- 172.21.15.5:0/1692053275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcebc0c4c30 con 0x56209ef90410 2024-09-19T06:26:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.392+0000 7fcea97fa640 1 -- 172.21.15.5:0/1692053275 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fceb4003e90 con 0x56209ef90410 2024-09-19T06:26:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.392+0000 7fcecaa50740 10 client.30496 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.392+0000 7fcecaa50740 1 -- 172.21.15.5:0/1692053275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x56209ecd3620 con 0x56209ef90410 2024-09-19T06:26:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.392+0000 7fcea97fa640 1 -- 172.21.15.5:0/1692053275 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcebc0c4c30 con 0x56209ef90410 2024-09-19T06:26:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.392+0000 7fcea97fa640 1 -- 172.21.15.5:0/1692053275 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fceb4029bb0 con 0x56209ef90410 2024-09-19T06:26:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.392+0000 7fcecaa50740 10 client.30496 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.392+0000 7fcecaa50740 10 client.30496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.392+0000 7fcecaa50740 20 client.30496 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.392+0000 7fcecaa50740 1 --2- 172.21.15.5:0/1692053275 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56209efaaa60 0x56209eface50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.392+0000 7fcecaa50740 4 client.30496 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:26:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.392+0000 7fcecaa50740 1 -- 172.21.15.5:0/1692053275 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56209ef01a20 con 0x56209efaaa60 2024-09-19T06:26:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.392+0000 7fcec5972640 1 --2- 172.21.15.5:0/1692053275 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56209efaaa60 0x56209eface50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.393+0000 7fcea97fa640 10 client.30496 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.393+0000 7fcec5972640 1 --2- 172.21.15.5:0/1692053275 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56209efaaa60 0x56209eface50 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b5e876e68a1613a server_cookie=62b93817d13feacd in_seq=0 out_seq=0 2024-09-19T06:26:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.393+0000 7fcea97fa640 1 -- 172.21.15.5:0/1692053275 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56209ef01a20 con 0x56209efaaa60 2024-09-19T06:26:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.393+0000 7fcea97fa640 10 client.30496 handle_command_reply: tid=0 2024-09-19T06:26:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.393+0000 7fcecaa50740 1 client.30496 shutdown 2024-09-19T06:26:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.394+0000 7fcecaa50740 1 -- 172.21.15.5:0/1692053275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef90410 msgr2=0x56209ef98120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.394+0000 7fcecaa50740 1 --2- 172.21.15.5:0/1692053275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef90410 0x56209ef98120 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fceb4020c20 tx=0x7fceb4004730 comp rx=0 tx=0).stop 2024-09-19T06:26:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.394+0000 7fcecaa50740 1 -- 172.21.15.5:0/1692053275 shutdown_connections 2024-09-19T06:26:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.394+0000 7fcecaa50740 1 --2- 172.21.15.5:0/1692053275 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56209efaaa60 0x56209eface50 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.394+0000 7fcecaa50740 1 --2- 172.21.15.5:0/1692053275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef9acb0 0x56209ef98660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.394+0000 7fcecaa50740 1 --2- 172.21.15.5:0/1692053275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef90410 0x56209ef98120 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.394+0000 7fcecaa50740 1 --2- 172.21.15.5:0/1692053275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef8faf0 0x56209ef97be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.394+0000 7fcecaa50740 1 -- 172.21.15.5:0/1692053275 >> 172.21.15.5:0/1692053275 conn(0x56209ef8c060 msgr2=0x56209ef91dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.394+0000 7fcea97fa640 0 client.30496 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.394+0000 7fcecaa50740 1 -- 172.21.15.5:0/1692053275 shutdown_connections 2024-09-19T06:26:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.394+0000 7fcecaa50740 1 -- 172.21.15.5:0/1692053275 wait complete. 2024-09-19T06:26:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.394+0000 7fcecaa50740 20 client.30496 trim_cache size 0 max 16384 2024-09-19T06:26:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.423+0000 7fcecaa50740 1 Processor -- start 2024-09-19T06:26:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.423+0000 7fcecaa50740 1 -- start start 2024-09-19T06:26:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.423+0000 7fcecaa50740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef8faf0 0x56209ef92ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.423+0000 7fcecaa50740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef90410 0x56209ef98490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.423+0000 7fcecaa50740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef9acb0 0x56209efb2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.423+0000 7fcecaa50740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01ab700 con 0x56209ef90410 2024-09-19T06:26:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.423+0000 7fcecaa50740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01ad240 con 0x56209ef8faf0 2024-09-19T06:26:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.423+0000 7fcecaa50740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01abdd0 con 0x56209ef9acb0 2024-09-19T06:26:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.424+0000 7fcec5171640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef90410 0x56209ef98490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.424+0000 7fcec5171640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef90410 0x56209ef98490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60920/0 (socket says 172.21.15.5:60920) 2024-09-19T06:26:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.424+0000 7fcec5171640 1 -- 172.21.15.5:0/3256284252 learned_addr learned my addr 172.21.15.5:0/3256284252 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.424+0000 7fcec6173640 1 --2- 172.21.15.5:0/3256284252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef9acb0 0x56209efb2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.424+0000 7fcec5972640 1 --2- 172.21.15.5:0/3256284252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef8faf0 0x56209ef92ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.424+0000 7fcec6173640 1 -- 172.21.15.5:0/3256284252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef8faf0 msgr2=0x56209ef92ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.424+0000 7fcec6173640 1 --2- 172.21.15.5:0/3256284252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef8faf0 0x56209ef92ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.424+0000 7fcec6173640 1 -- 172.21.15.5:0/3256284252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef90410 msgr2=0x56209ef98490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.424+0000 7fcec6173640 1 --2- 172.21.15.5:0/3256284252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef90410 0x56209ef98490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.424+0000 7fcec6173640 1 -- 172.21.15.5:0/3256284252 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209ef872d0 con 0x56209ef9acb0 2024-09-19T06:26:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.424+0000 7fcec5171640 1 --2- 172.21.15.5:0/3256284252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef90410 0x56209ef98490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.424+0000 7fcec6173640 1 --2- 172.21.15.5:0/3256284252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef9acb0 0x56209efb2720 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fceb8005fd0 tx=0x7fceb8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41ca424d84665eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.425+0000 7fceab7fe640 1 -- 172.21.15.5:0/3256284252 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8022070 con 0x56209ef9acb0 2024-09-19T06:26:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.425+0000 7fceab7fe640 1 -- 172.21.15.5:0/3256284252 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceb8003e20 con 0x56209ef9acb0 2024-09-19T06:26:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.425+0000 7fceab7fe640 1 -- 172.21.15.5:0/3256284252 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8004660 con 0x56209ef9acb0 2024-09-19T06:26:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.426+0000 7fcecaa50740 1 -- 172.21.15.5:0/3256284252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef9acb0 msgr2=0x56209efb2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.426+0000 7fcecaa50740 1 --2- 172.21.15.5:0/3256284252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef9acb0 0x56209efb2720 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fceb8005fd0 tx=0x7fceb8009500 comp rx=0 tx=0).stop 2024-09-19T06:26:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.426+0000 7fcecaa50740 1 -- 172.21.15.5:0/3256284252 shutdown_connections 2024-09-19T06:26:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.426+0000 7fcecaa50740 1 --2- 172.21.15.5:0/3256284252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef9acb0 0x56209efb2720 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.426+0000 7fcecaa50740 1 --2- 172.21.15.5:0/3256284252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef90410 0x56209ef98490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.426+0000 7fcecaa50740 1 --2- 172.21.15.5:0/3256284252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef8faf0 0x56209ef92ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.426+0000 7fcecaa50740 1 -- 172.21.15.5:0/3256284252 >> 172.21.15.5:0/3256284252 conn(0x56209ef8c060 msgr2=0x56209ef99c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.426+0000 7fcecaa50740 1 -- 172.21.15.5:0/3256284252 shutdown_connections 2024-09-19T06:26:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.426+0000 7fcecaa50740 1 -- 172.21.15.5:0/3256284252 wait complete. 2024-09-19T06:26:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.426+0000 7fcecaa50740 1 -- start start 2024-09-19T06:26:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.426+0000 7fcecaa50740 1 Processor -- start 2024-09-19T06:26:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.427+0000 7fcecaa50740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef8faf0 0x56209efa2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.427+0000 7fcecaa50740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef90410 0x56209efa3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.427+0000 7fcecaa50740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef9acb0 0x56209ef94520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.427+0000 7fcecaa50740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01acb70 con 0x56209ef9acb0 2024-09-19T06:26:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.427+0000 7fcecaa50740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01ac4a0 con 0x56209ef90410 2024-09-19T06:26:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.427+0000 7fcecaa50740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a9700 con 0x56209ef8faf0 2024-09-19T06:26:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.427+0000 7fcecaa50740 10 client.0 _ll_register_callbacks cb 0x7fceb4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.427+0000 7fcec5972640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef8faf0 0x56209efa2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.427+0000 7fcec5972640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef8faf0 0x56209efa2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:35528/0 (socket says 172.21.15.5:35528) 2024-09-19T06:26:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.427+0000 7fcec5171640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef90410 0x56209efa3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.427+0000 7fcec5972640 1 -- 172.21.15.5:0/4036973186 learned_addr learned my addr 172.21.15.5:0/4036973186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.427+0000 7fcec6173640 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef9acb0 0x56209ef94520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcec5972640 1 -- 172.21.15.5:0/4036973186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef90410 msgr2=0x56209efa3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcec5972640 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef90410 0x56209efa3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcec5972640 1 -- 172.21.15.5:0/4036973186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef9acb0 msgr2=0x56209ef94520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcec5972640 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef9acb0 0x56209ef94520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcec5972640 1 -- 172.21.15.5:0/4036973186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209ef94d50 con 0x56209ef8faf0 2024-09-19T06:26:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcec5171640 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef90410 0x56209efa3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcec5972640 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef8faf0 0x56209efa2ce0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0ed4e0 tx=0x7fcebc005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c4175feca4e07c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcea97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcea97fa640 1 -- 172.21.15.5:0/4036973186 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc049040 con 0x56209ef8faf0 2024-09-19T06:26:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcea97fa640 1 -- 172.21.15.5:0/4036973186 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fcebc0ba2c0 con 0x56209ef8faf0 2024-09-19T06:26:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcecaa50740 1 -- 172.21.15.5:0/4036973186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb8003e20 con 0x56209ef8faf0 2024-09-19T06:26:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcea97fa640 1 -- 172.21.15.5:0/4036973186 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fcebc0c4c30 con 0x56209ef8faf0 2024-09-19T06:26:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcecaa50740 10 client.30502 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.428+0000 7fcecaa50740 1 -- 172.21.15.5:0/4036973186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x56209f0152c0 con 0x56209ef8faf0 2024-09-19T06:26:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.429+0000 7fcea97fa640 1 -- 172.21.15.5:0/4036973186 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceb8003e20 con 0x56209ef8faf0 2024-09-19T06:26:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.429+0000 7fcea97fa640 1 -- 172.21.15.5:0/4036973186 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fcebc0da070 con 0x56209ef8faf0 2024-09-19T06:26:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.429+0000 7fcecaa50740 10 client.30502 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.429+0000 7fcecaa50740 10 client.30502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.429+0000 7fcecaa50740 20 client.30502 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.429+0000 7fcecaa50740 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56209f015740 0x56209f017b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.429+0000 7fcecaa50740 4 client.30502 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:26:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.429+0000 7fcecaa50740 1 -- 172.21.15.5:0/4036973186 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56209eefbd70 con 0x56209f015740 2024-09-19T06:26:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.429+0000 7fcec5171640 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56209f015740 0x56209f017b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.430+0000 7fcea97fa640 10 client.30502 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.430+0000 7fcec5171640 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56209f015740 0x56209f017b30 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68b045b676cdd62 server_cookie=a2bbd16da6784644 in_seq=0 out_seq=0 2024-09-19T06:26:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.537+0000 7fcea97fa640 1 -- 172.21.15.5:0/4036973186 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56209eefbd70 con 0x56209f015740 2024-09-19T06:26:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.537+0000 7fcea97fa640 10 client.30502 handle_command_reply: tid=0 2024-09-19T06:26:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.537+0000 7fcecaa50740 1 client.30502 shutdown 2024-09-19T06:26:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.537+0000 7fcecaa50740 1 -- 172.21.15.5:0/4036973186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef8faf0 msgr2=0x56209efa2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.537+0000 7fcecaa50740 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef8faf0 0x56209efa2ce0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0ed4e0 tx=0x7fcebc005bd0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.538+0000 7fcecaa50740 1 -- 172.21.15.5:0/4036973186 shutdown_connections 2024-09-19T06:26:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.538+0000 7fcecaa50740 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56209f015740 0x56209f017b30 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.538+0000 7fcecaa50740 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56209ef9acb0 0x56209ef94520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.538+0000 7fcea97fa640 0 client.30502 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.538+0000 7fcecaa50740 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56209ef90410 0x56209efa3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.538+0000 7fcecaa50740 1 --2- 172.21.15.5:0/4036973186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56209ef8faf0 0x56209efa2ce0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.538+0000 7fcecaa50740 1 -- 172.21.15.5:0/4036973186 >> 172.21.15.5:0/4036973186 conn(0x56209ef8c060 msgr2=0x56209ef8a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.538+0000 7fcecaa50740 1 -- 172.21.15.5:0/4036973186 shutdown_connections 2024-09-19T06:26:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.538+0000 7fcecaa50740 1 -- 172.21.15.5:0/4036973186 wait complete. 2024-09-19T06:26:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.538+0000 7fcecaa50740 20 client.30502 trim_cache size 0 max 16384 2024-09-19T06:26:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:26:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:26:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:26:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:26:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.539+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1109634609 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fceac0a33b0 msgr2=0x7fceac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.539+0000 7fcec7bd6640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fceac0a33b0 0x7fceac0a5870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcec00fdff0 tx=0x7fceb4005f90 comp rx=0 tx=0).stop 2024-09-19T06:26:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.539+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1109634609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec0101760 msgr2=0x7fcec01983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.539+0000 7fcec7bd6640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec0101760 0x7fcec01983a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcebc005940 tx=0x7fcebc0040c0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.539+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1109634609 shutdown_connections 2024-09-19T06:26:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.539+0000 7fcec7bd6640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec019cc60 0x7fcec019d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.539+0000 7fcec7bd6640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec0105da0 0x7fcec01988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.539+0000 7fcec7bd6640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fceac0a33b0 0x7fceac0a5870 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.539+0000 7fcec7bd6640 1 --2- 172.21.15.5:0/1109634609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec0101760 0x7fcec01983a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.539+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1109634609 >> 172.21.15.5:0/1109634609 conn(0x7fcec00fba00 msgr2=0x7fcec00fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.539+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1109634609 shutdown_connections 2024-09-19T06:26:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:39.539+0000 7fcec7bd6640 1 -- 172.21.15.5:0/1109634609 wait complete. 2024-09-19T06:26:39.597 INFO:tasks.workunit.client.0.smithi005.stdout: 10 97011 8.55 MB/sec execute 483 sec latency 85.988 ms 2024-09-19T06:26:40.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:40 smithi005 ceph-mon[30955]: pgmap v737: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 281 op/s 2024-09-19T06:26:40.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:40 smithi049 ceph-mon[34640]: pgmap v737: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 281 op/s 2024-09-19T06:26:40.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:40 smithi157 ceph-mon[35346]: pgmap v737: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 281 op/s 2024-09-19T06:26:40.597 INFO:tasks.workunit.client.0.smithi005.stdout: 10 97167 8.54 MB/sec execute 484 sec latency 64.091 ms 2024-09-19T06:26:41.597 INFO:tasks.workunit.client.0.smithi005.stdout: 10 97309 8.54 MB/sec execute 485 sec latency 57.882 ms 2024-09-19T06:26:42.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:42 smithi005 ceph-mon[30955]: pgmap v738: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 281 op/s 2024-09-19T06:26:42.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:42 smithi049 ceph-mon[34640]: pgmap v738: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 281 op/s 2024-09-19T06:26:42.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:42 smithi157 ceph-mon[35346]: pgmap v738: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 281 op/s 2024-09-19T06:26:42.597 INFO:tasks.workunit.client.0.smithi005.stdout: 10 97453 8.54 MB/sec execute 486 sec latency 128.184 ms 2024-09-19T06:26:43.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 97615 8.54 MB/sec execute 487 sec latency 125.211 ms 2024-09-19T06:26:44.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:44 smithi005 ceph-mon[30955]: pgmap v739: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 419 op/s 2024-09-19T06:26:44.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:44 smithi049 ceph-mon[34640]: pgmap v739: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 419 op/s 2024-09-19T06:26:44.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:44 smithi157 ceph-mon[35346]: pgmap v739: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 419 op/s 2024-09-19T06:26:44.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 97821 8.55 MB/sec execute 488 sec latency 115.632 ms 2024-09-19T06:26:45.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:45 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:26:45.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:45 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:26:45.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:45 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:26:45.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:45 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:26:45.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:45 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:26:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:45 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:26:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:45 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:26:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:45 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:26:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:45 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:26:45.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:45 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:26:45.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:45 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:26:45.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:45 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:26:45.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:45 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:26:45.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:45 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:26:45.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:45 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:26:45.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 98030 8.56 MB/sec execute 489 sec latency 58.372 ms 2024-09-19T06:26:46.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:46 smithi005 ceph-mon[30955]: pgmap v740: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 310 op/s 2024-09-19T06:26:46.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:46 smithi049 ceph-mon[34640]: pgmap v740: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 310 op/s 2024-09-19T06:26:46.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:46 smithi157 ceph-mon[35346]: pgmap v740: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 310 op/s 2024-09-19T06:26:46.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 98170 8.55 MB/sec execute 490 sec latency 63.924 ms 2024-09-19T06:26:47.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 98336 8.57 MB/sec execute 491 sec latency 53.722 ms 2024-09-19T06:26:48.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:48 smithi005 ceph-mon[30955]: pgmap v741: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.2 MiB/s wr, 443 op/s 2024-09-19T06:26:48.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:48 smithi049 ceph-mon[34640]: pgmap v741: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.2 MiB/s wr, 443 op/s 2024-09-19T06:26:48.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:48 smithi157 ceph-mon[35346]: pgmap v741: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.2 MiB/s wr, 443 op/s 2024-09-19T06:26:48.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 98507 8.57 MB/sec execute 492 sec latency 59.670 ms 2024-09-19T06:26:49.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 98661 8.56 MB/sec execute 493 sec latency 97.033 ms 2024-09-19T06:26:50.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:50 smithi005 ceph-mon[30955]: pgmap v742: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 271 op/s 2024-09-19T06:26:50.395 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:50 smithi049 ceph-mon[34640]: pgmap v742: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 271 op/s 2024-09-19T06:26:50.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:50 smithi157 ceph-mon[35346]: pgmap v742: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 271 op/s 2024-09-19T06:26:50.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 98813 8.55 MB/sec execute 494 sec latency 157.441 ms 2024-09-19T06:26:51.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 98972 8.55 MB/sec execute 495 sec latency 101.142 ms 2024-09-19T06:26:52.353 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:52 smithi157 ceph-mon[35346]: pgmap v743: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 271 op/s 2024-09-19T06:26:52.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:52 smithi005 ceph-mon[30955]: pgmap v743: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 271 op/s 2024-09-19T06:26:52.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:52 smithi049 ceph-mon[34640]: pgmap v743: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 271 op/s 2024-09-19T06:26:52.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 99145 8.54 MB/sec execute 496 sec latency 56.112 ms 2024-09-19T06:26:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.659+0000 7fc200477640 1 -- 172.21.15.5:0/3128618979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f8102c80 msgr2=0x7fc1f8103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.659+0000 7fc200477640 1 --2- 172.21.15.5:0/3128618979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f8102c80 0x7fc1f8103060 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009900 tx=0x7fc1f402a480 comp rx=0 tx=0).stop 2024-09-19T06:26:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.659+0000 7fc200477640 1 -- 172.21.15.5:0/3128618979 shutdown_connections 2024-09-19T06:26:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.659+0000 7fc200477640 1 --2- 172.21.15.5:0/3128618979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f810c0a0 0x7fc1f810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.659+0000 7fc200477640 1 --2- 172.21.15.5:0/3128618979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f81035a0 0x7fc1f810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.659+0000 7fc200477640 1 --2- 172.21.15.5:0/3128618979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f8102c80 0x7fc1f8103060 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.659+0000 7fc200477640 1 -- 172.21.15.5:0/3128618979 >> 172.21.15.5:0/3128618979 conn(0x7fc1f80779a0 msgr2=0x7fc1f80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.659+0000 7fc200477640 1 -- 172.21.15.5:0/3128618979 shutdown_connections 2024-09-19T06:26:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.659+0000 7fc200477640 1 -- 172.21.15.5:0/3128618979 wait complete. 2024-09-19T06:26:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc200477640 1 Processor -- start 2024-09-19T06:26:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc200477640 1 -- start start 2024-09-19T06:26:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc200477640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f8102c80 0x7fc1f81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc200477640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f81035a0 0x7fc1f8198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc200477640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f810c0a0 0x7fc1f819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc200477640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f8111da0 con 0x7fc1f81035a0 2024-09-19T06:26:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc200477640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f8111c20 con 0x7fc1f810c0a0 2024-09-19T06:26:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc200477640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f8111f20 con 0x7fc1f8102c80 2024-09-19T06:26:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc1fe213640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f8102c80 0x7fc1f81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc1fda12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f81035a0 0x7fc1f8198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc1fe213640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f8102c80 0x7fc1f81982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46760/0 (socket says 172.21.15.5:46760) 2024-09-19T06:26:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc1fe213640 1 -- 172.21.15.5:0/1353708430 learned_addr learned my addr 172.21.15.5:0/1353708430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.660+0000 7fc1fea14640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f810c0a0 0x7fc1f819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.661+0000 7fc1fe213640 1 -- 172.21.15.5:0/1353708430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f810c0a0 msgr2=0x7fc1f819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.661+0000 7fc1fe213640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f810c0a0 0x7fc1f819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.661+0000 7fc1fe213640 1 -- 172.21.15.5:0/1353708430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f81035a0 msgr2=0x7fc1f8198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.661+0000 7fc1fe213640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f81035a0 0x7fc1f8198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.661+0000 7fc1fe213640 1 -- 172.21.15.5:0/1353708430 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f819d2e0 con 0x7fc1f8102c80 2024-09-19T06:26:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.661+0000 7fc1fda12640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f81035a0 0x7fc1f8198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.661+0000 7fc1fe213640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f8102c80 0x7fc1f81982f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005ab0 tx=0x7fc1f4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.661+0000 7fc1ef7fe640 1 -- 172.21.15.5:0/1353708430 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f403f070 con 0x7fc1f8102c80 2024-09-19T06:26:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.661+0000 7fc1ef7fe640 1 -- 172.21.15.5:0/1353708430 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1f4031b60 con 0x7fc1f8102c80 2024-09-19T06:26:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.661+0000 7fc200477640 1 -- 172.21.15.5:0/1353708430 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f819d570 con 0x7fc1f8102c80 2024-09-19T06:26:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.661+0000 7fc1ef7fe640 1 -- 172.21.15.5:0/1353708430 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4035030 con 0x7fc1f8102c80 2024-09-19T06:26:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.661+0000 7fc200477640 1 -- 172.21.15.5:0/1353708430 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f81a4e60 con 0x7fc1f8102c80 2024-09-19T06:26:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.664+0000 7fc203211740 1 Processor -- start 2024-09-19T06:26:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc1ef7fe640 1 -- 172.21.15.5:0/1353708430 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc1f4042070 con 0x7fc1f8102c80 2024-09-19T06:26:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc203211740 1 -- start start 2024-09-19T06:26:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc203211740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a042bc0 0x55ca8a042fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc203211740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03e680 0x55ca8a03eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc203211740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a03f020 0x55ca8a04aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc203211740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca89f992f0 con 0x55ca8a03f020 2024-09-19T06:26:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc203211740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ca89f41270 con 0x55ca8a042bc0 2024-09-19T06:26:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc203211740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ca89f952b0 con 0x55ca8a03e680 2024-09-19T06:26:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc1fea14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a042bc0 0x55ca8a042fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc1fea14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a042bc0 0x55ca8a042fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58634/0 (socket says 172.21.15.5:58634) 2024-09-19T06:26:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc1fea14640 1 -- 172.21.15.5:0/3161336224 learned_addr learned my addr 172.21.15.5:0/3161336224 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc1fda12640 1 --2- 172.21.15.5:0/3161336224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a03f020 0x55ca8a04aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.665+0000 7fc1fe213640 1 --2- 172.21.15.5:0/3161336224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03e680 0x55ca8a03eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.666+0000 7fc1fea14640 1 -- 172.21.15.5:0/3161336224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03e680 msgr2=0x55ca8a03eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.666+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3161336224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03e680 0x55ca8a03eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.666+0000 7fc1fea14640 1 -- 172.21.15.5:0/3161336224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a03f020 msgr2=0x55ca8a04aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.666+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3161336224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a03f020 0x55ca8a04aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.666+0000 7fc1fea14640 1 -- 172.21.15.5:0/3161336224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca89f40340 con 0x55ca8a042bc0 2024-09-19T06:26:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.666+0000 7fc1ef7fe640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1e40a33b0 0x7fc1e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.666+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3161336224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a042bc0 0x55ca8a042fa0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009f50 tx=0x7fc1f00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=221f47715e9a1ff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.666+0000 7fc1ef7fe640 1 -- 172.21.15.5:0/1353708430 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fc1f40f0490 con 0x7fc1f8102c80 2024-09-19T06:26:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.666+0000 7fc1ecbfe640 1 -- 172.21.15.5:0/3161336224 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0004d20 con 0x55ca8a042bc0 2024-09-19T06:26:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.666+0000 7fc1fe213640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1e40a33b0 0x7fc1e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.666+0000 7fc1ecbfe640 1 -- 172.21.15.5:0/3161336224 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1f0004050 con 0x55ca8a042bc0 2024-09-19T06:26:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.666+0000 7fc1ecbfe640 1 -- 172.21.15.5:0/3161336224 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0005280 con 0x55ca8a042bc0 2024-09-19T06:26:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.667+0000 7fc203211740 1 -- 172.21.15.5:0/3161336224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a042bc0 msgr2=0x55ca8a042fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.667+0000 7fc1fe213640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1e40a33b0 0x7fc1e40a5870 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40b8c00 tx=0x7fc1f4009260 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.667+0000 7fc203211740 1 --2- 172.21.15.5:0/3161336224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a042bc0 0x55ca8a042fa0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009f50 tx=0x7fc1f00118d0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.667+0000 7fc203211740 1 -- 172.21.15.5:0/3161336224 shutdown_connections 2024-09-19T06:26:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.667+0000 7fc203211740 1 --2- 172.21.15.5:0/3161336224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a03f020 0x55ca8a04aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.667+0000 7fc203211740 1 --2- 172.21.15.5:0/3161336224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03e680 0x55ca8a03eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.667+0000 7fc203211740 1 --2- 172.21.15.5:0/3161336224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a042bc0 0x55ca8a042fa0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.667+0000 7fc203211740 1 -- 172.21.15.5:0/3161336224 >> 172.21.15.5:0/3161336224 conn(0x55ca8a030b10 msgr2=0x55ca8a030ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.668+0000 7fc203211740 1 -- 172.21.15.5:0/3161336224 shutdown_connections 2024-09-19T06:26:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.668+0000 7fc203211740 1 -- 172.21.15.5:0/3161336224 wait complete. 2024-09-19T06:26:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.668+0000 7fc203211740 1 -- start start 2024-09-19T06:26:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.668+0000 7fc203211740 1 Processor -- start 2024-09-19T06:26:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.668+0000 7fc203211740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 0x55ca8a047eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.668+0000 7fc203211740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a0483f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.668+0000 7fc203211740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a042bc0 0x55ca8a048930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.668+0000 7fc203211740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca8a04b0c0 con 0x55ca8a042bc0 2024-09-19T06:26:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.668+0000 7fc203211740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ca89f22290 con 0x55ca8a03e680 2024-09-19T06:26:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.668+0000 7fc203211740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ca8a04b240 con 0x55ca8a03f020 2024-09-19T06:26:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc203211740 10 client.0 _ll_register_callbacks cb 0x55ca89f53140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1fea14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 0x55ca8a047eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1fda12640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a0483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1fea14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 0x55ca8a047eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58644/0 (socket says 172.21.15.5:58644) 2024-09-19T06:26:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1fda12640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a0483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46780/0 (socket says 172.21.15.5:46780) 2024-09-19T06:26:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1fea14640 1 -- 172.21.15.5:0/3797330702 learned_addr learned my addr 172.21.15.5:0/3797330702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3797330702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a042bc0 0x55ca8a048930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1fda12640 1 -- 172.21.15.5:0/3797330702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 msgr2=0x55ca8a047eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1fda12640 1 --2- 172.21.15.5:0/3797330702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 0x55ca8a047eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1fda12640 1 -- 172.21.15.5:0/3797330702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a042bc0 msgr2=0x55ca8a048930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1fda12640 1 --2- 172.21.15.5:0/3797330702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a042bc0 0x55ca8a048930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1fda12640 1 -- 172.21.15.5:0/3797330702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca89cac210 con 0x55ca8a03f020 2024-09-19T06:26:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1fda12640 1 --2- 172.21.15.5:0/3797330702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a0483f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8012cf0 tx=0x7fc1e800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7000af0339c2daff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.669+0000 7fc1e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.670+0000 7fc1e1ffb640 1 -- 172.21.15.5:0/3797330702 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8012110 con 0x55ca8a03f020 2024-09-19T06:26:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.670+0000 7fc1e1ffb640 1 -- 172.21.15.5:0/3797330702 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fc1e8004050 con 0x55ca8a03f020 2024-09-19T06:26:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.670+0000 7fc203211740 1 -- 172.21.15.5:0/3797330702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f0004050 con 0x55ca8a03f020 2024-09-19T06:26:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.670+0000 7fc1e1ffb640 1 -- 172.21.15.5:0/3797330702 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc1e80122b0 con 0x55ca8a03f020 2024-09-19T06:26:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.670+0000 7fc203211740 10 client.30514 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.670+0000 7fc203211740 1 -- 172.21.15.5:0/3797330702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55ca89d78620 con 0x55ca8a03f020 2024-09-19T06:26:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.670+0000 7fc1e1ffb640 1 -- 172.21.15.5:0/3797330702 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1f0004050 con 0x55ca8a03f020 2024-09-19T06:26:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.671+0000 7fc1e1ffb640 1 -- 172.21.15.5:0/3797330702 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc1e8010430 con 0x55ca8a03f020 2024-09-19T06:26:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.671+0000 7fc203211740 10 client.30514 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.671+0000 7fc203211740 10 client.30514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.671+0000 7fc203211740 20 client.30514 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.671+0000 7fc203211740 1 --2- 172.21.15.5:0/3797330702 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca8a0584b0 0x55ca8a05a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.671+0000 7fc203211740 4 client.30514 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:26:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.671+0000 7fc203211740 1 -- 172.21.15.5:0/3797330702 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca89fa6a20 con 0x55ca8a0584b0 2024-09-19T06:26:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.671+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3797330702 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca8a0584b0 0x55ca8a05a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.672+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3797330702 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca8a0584b0 0x55ca8a05a8a0 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=719a5d49545d5085 server_cookie=2505ac901af23ae6 in_seq=0 out_seq=0 2024-09-19T06:26:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.672+0000 7fc1e1ffb640 10 client.30514 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.672+0000 7fc1e1ffb640 1 -- 172.21.15.5:0/3797330702 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca89fa6a20 con 0x55ca8a0584b0 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.672+0000 7fc1e1ffb640 10 client.30514 handle_command_reply: tid=0 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.672+0000 7fc203211740 1 client.30514 shutdown 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.673+0000 7fc203211740 1 -- 172.21.15.5:0/3797330702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 msgr2=0x55ca8a0483f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.673+0000 7fc203211740 1 --2- 172.21.15.5:0/3797330702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a0483f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8012cf0 tx=0x7fc1e800f790 comp rx=0 tx=0).stop 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.673+0000 7fc203211740 1 -- 172.21.15.5:0/3797330702 shutdown_connections 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.673+0000 7fc203211740 1 --2- 172.21.15.5:0/3797330702 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca8a0584b0 0x55ca8a05a8a0 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.673+0000 7fc203211740 1 --2- 172.21.15.5:0/3797330702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a042bc0 0x55ca8a048930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.673+0000 7fc203211740 1 --2- 172.21.15.5:0/3797330702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a0483f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.673+0000 7fc203211740 1 --2- 172.21.15.5:0/3797330702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 0x55ca8a047eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.673+0000 7fc203211740 1 -- 172.21.15.5:0/3797330702 >> 172.21.15.5:0/3797330702 conn(0x55ca8a030b10 msgr2=0x55ca8a0441b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.673+0000 7fc1e1ffb640 0 client.30514 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.673+0000 7fc203211740 1 -- 172.21.15.5:0/3797330702 shutdown_connections 2024-09-19T06:26:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.673+0000 7fc203211740 1 -- 172.21.15.5:0/3797330702 wait complete. 2024-09-19T06:26:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.673+0000 7fc203211740 20 client.30514 trim_cache size 0 max 16384 2024-09-19T06:26:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc203211740 1 Processor -- start 2024-09-19T06:26:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc203211740 1 -- start start 2024-09-19T06:26:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc203211740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 0x55ca8a0459d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc203211740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a049710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc203211740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a042bc0 0x55ca8a049c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc203211740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81ab5c0 con 0x55ca8a042bc0 2024-09-19T06:26:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc203211740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81ad100 con 0x55ca8a03e680 2024-09-19T06:26:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc203211740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81abc90 con 0x55ca8a03f020 2024-09-19T06:26:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc1fea14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 0x55ca8a0459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc1fea14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 0x55ca8a0459d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58648/0 (socket says 172.21.15.5:58648) 2024-09-19T06:26:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc1fea14640 1 -- 172.21.15.5:0/3110307483 learned_addr learned my addr 172.21.15.5:0/3110307483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc1fda12640 1 --2- 172.21.15.5:0/3110307483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a049710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3110307483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a042bc0 0x55ca8a049c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.694+0000 7fc1fea14640 1 -- 172.21.15.5:0/3110307483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 msgr2=0x55ca8a049710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.695+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3110307483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a049710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.695+0000 7fc1fea14640 1 -- 172.21.15.5:0/3110307483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a042bc0 msgr2=0x55ca8a049c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.695+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3110307483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a042bc0 0x55ca8a049c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.695+0000 7fc1fea14640 1 -- 172.21.15.5:0/3110307483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8a04aa00 con 0x55ca8a03e680 2024-09-19T06:26:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.695+0000 7fc1fda12640 1 --2- 172.21.15.5:0/3110307483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a049710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.695+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3110307483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 0x55ca8a0459d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0012fd0 tx=0x7fc1f00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5725aacaf74a9fab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.695+0000 7fc1e3fff640 1 -- 172.21.15.5:0/3110307483 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0004480 con 0x55ca8a03e680 2024-09-19T06:26:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.695+0000 7fc1e3fff640 1 -- 172.21.15.5:0/3110307483 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1f0004820 con 0x55ca8a03e680 2024-09-19T06:26:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.695+0000 7fc1e3fff640 1 -- 172.21.15.5:0/3110307483 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0005190 con 0x55ca8a03e680 2024-09-19T06:26:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.696+0000 7fc203211740 1 -- 172.21.15.5:0/3110307483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 msgr2=0x55ca8a0459d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.696+0000 7fc203211740 1 --2- 172.21.15.5:0/3110307483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 0x55ca8a0459d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0012fd0 tx=0x7fc1f00118d0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.696+0000 7fc203211740 1 -- 172.21.15.5:0/3110307483 shutdown_connections 2024-09-19T06:26:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.696+0000 7fc203211740 1 --2- 172.21.15.5:0/3110307483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a042bc0 0x55ca8a049c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.696+0000 7fc203211740 1 --2- 172.21.15.5:0/3110307483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a049710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.696+0000 7fc203211740 1 --2- 172.21.15.5:0/3110307483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a03e680 0x55ca8a0459d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.696+0000 7fc203211740 1 -- 172.21.15.5:0/3110307483 >> 172.21.15.5:0/3110307483 conn(0x55ca8a030b10 msgr2=0x55ca8a041020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.696+0000 7fc203211740 1 -- 172.21.15.5:0/3110307483 shutdown_connections 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.696+0000 7fc203211740 1 -- 172.21.15.5:0/3110307483 wait complete. 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.696+0000 7fc203211740 1 -- start start 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc203211740 1 Processor -- start 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc203211740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a03e680 0x55ca8a0506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc203211740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a050c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc203211740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a042bc0 0x55ca8a05f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc203211740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81aca30 con 0x55ca8a03e680 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc203211740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81ac360 con 0x55ca8a042bc0 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc203211740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81a95c0 con 0x55ca8a03f020 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc203211740 10 client.0 _ll_register_callbacks cb 0x7fc1e8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc1fda12640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a050c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc1fea14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a03e680 0x55ca8a0506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc1fda12640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a050c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46802/0 (socket says 172.21.15.5:46802) 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc1fda12640 1 -- 172.21.15.5:0/3855627669 learned_addr learned my addr 172.21.15.5:0/3855627669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a042bc0 0x55ca8a05f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc1fda12640 1 -- 172.21.15.5:0/3855627669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a042bc0 msgr2=0x55ca8a05f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc1fda12640 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a042bc0 0x55ca8a05f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc1fda12640 1 -- 172.21.15.5:0/3855627669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a03e680 msgr2=0x55ca8a0506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.697+0000 7fc1fda12640 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a03e680 0x55ca8a0506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.698+0000 7fc1fda12640 1 -- 172.21.15.5:0/3855627669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8a05f7f0 con 0x55ca8a03f020 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.698+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a03e680 0x55ca8a0506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.698+0000 7fc1fda12640 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a050c00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009cb0 tx=0x7fc1e8010ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b79d912a34d77f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.698+0000 7fc1e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.698+0000 7fc1e1ffb640 1 -- 172.21.15.5:0/3855627669 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8029040 con 0x55ca8a03f020 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.698+0000 7fc1e1ffb640 1 -- 172.21.15.5:0/3855627669 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8550+0+0 (secure 0 0 0) 0x7fc1e8010430 con 0x55ca8a03f020 2024-09-19T06:26:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.698+0000 7fc203211740 1 -- 172.21.15.5:0/3855627669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f0004820 con 0x55ca8a03f020 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.698+0000 7fc1e1ffb640 1 -- 172.21.15.5:0/3855627669 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc1e803a050 con 0x55ca8a03f020 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.698+0000 7fc203211740 10 client.30520 fetch_fsmap learned FSMap version 62 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.699+0000 7fc203211740 1 -- 172.21.15.5:0/3855627669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55ca8a060b30 con 0x55ca8a03f020 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.699+0000 7fc1e1ffb640 1 -- 172.21.15.5:0/3855627669 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1e8010630 con 0x55ca8a03f020 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.699+0000 7fc1e1ffb640 1 -- 172.21.15.5:0/3855627669 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc1e8035030 con 0x55ca8a03f020 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.699+0000 7fc203211740 10 client.30520 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.699+0000 7fc203211740 10 client.30520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.699+0000 7fc203211740 20 client.30520 populate_metadata read hostname 'smithi005' 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.699+0000 7fc203211740 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca8a0c44c0 0x55ca8a0c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.699+0000 7fc203211740 4 client.30520 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.699+0000 7fc203211740 1 -- 172.21.15.5:0/3855627669 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc1f0004820 con 0x55ca8a0c44c0 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.700+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca8a0c44c0 0x55ca8a0c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:26:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.700+0000 7fc1e1ffb640 10 client.30520 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.700+0000 7fc1fea14640 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca8a0c44c0 0x55ca8a0c68e0 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ceaeb563256c0d3 server_cookie=3ab7ad8d76e852ee in_seq=0 out_seq=0 2024-09-19T06:26:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.813+0000 7fc1e1ffb640 1 -- 172.21.15.5:0/3855627669 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc1f0004820 con 0x55ca8a0c44c0 2024-09-19T06:26:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc1e1ffb640 10 client.30520 handle_command_reply: tid=0 2024-09-19T06:26:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc203211740 1 client.30520 shutdown 2024-09-19T06:26:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc203211740 1 -- 172.21.15.5:0/3855627669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 msgr2=0x55ca8a050c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc203211740 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a050c00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009cb0 tx=0x7fc1e8010ac0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc203211740 1 -- 172.21.15.5:0/3855627669 shutdown_connections 2024-09-19T06:26:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc203211740 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55ca8a0c44c0 0x55ca8a0c68e0 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc1e1ffb640 0 client.30520 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:26:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc203211740 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ca8a042bc0 0x55ca8a05f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc203211740 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca8a03f020 0x55ca8a050c00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc203211740 1 --2- 172.21.15.5:0/3855627669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca8a03e680 0x55ca8a0506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc203211740 1 -- 172.21.15.5:0/3855627669 >> 172.21.15.5:0/3855627669 conn(0x55ca8a030b10 msgr2=0x55ca8a043830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc203211740 1 -- 172.21.15.5:0/3855627669 shutdown_connections 2024-09-19T06:26:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc203211740 1 -- 172.21.15.5:0/3855627669 wait complete. 2024-09-19T06:26:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.814+0000 7fc203211740 20 client.30520 trim_cache size 0 max 16384 2024-09-19T06:26:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:26:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:26:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:26:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:26:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.815+0000 7fc200477640 1 -- 172.21.15.5:0/1353708430 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1e40a33b0 msgr2=0x7fc1e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.815+0000 7fc200477640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1e40a33b0 0x7fc1e40a5870 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40b8c00 tx=0x7fc1f4009260 comp rx=0 tx=0).stop 2024-09-19T06:26:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.815+0000 7fc200477640 1 -- 172.21.15.5:0/1353708430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f8102c80 msgr2=0x7fc1f81982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:26:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.815+0000 7fc200477640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f8102c80 0x7fc1f81982f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005ab0 tx=0x7fc1f4003c10 comp rx=0 tx=0).stop 2024-09-19T06:26:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.815+0000 7fc200477640 1 -- 172.21.15.5:0/1353708430 shutdown_connections 2024-09-19T06:26:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.815+0000 7fc200477640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f810c0a0 0x7fc1f819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.815+0000 7fc200477640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f81035a0 0x7fc1f8198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.815+0000 7fc200477640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1e40a33b0 0x7fc1e40a5870 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.815+0000 7fc200477640 1 --2- 172.21.15.5:0/1353708430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f8102c80 0x7fc1f81982f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:26:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.815+0000 7fc200477640 1 -- 172.21.15.5:0/1353708430 >> 172.21.15.5:0/1353708430 conn(0x7fc1f80779a0 msgr2=0x7fc1f80fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:26:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.815+0000 7fc200477640 1 -- 172.21.15.5:0/1353708430 shutdown_connections 2024-09-19T06:26:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:26:52.815+0000 7fc200477640 1 -- 172.21.15.5:0/1353708430 wait complete. 2024-09-19T06:26:53.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 99308 8.54 MB/sec execute 497 sec latency 130.918 ms 2024-09-19T06:26:54.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 99462 8.54 MB/sec execute 498 sec latency 73.998 ms 2024-09-19T06:26:54.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:54 smithi157 ceph-mon[35346]: pgmap v744: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.9 MiB/s wr, 394 op/s 2024-09-19T06:26:54.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:54 smithi005 ceph-mon[30955]: pgmap v744: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.9 MiB/s wr, 394 op/s 2024-09-19T06:26:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:54 smithi049 ceph-mon[34640]: pgmap v744: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.9 MiB/s wr, 394 op/s 2024-09-19T06:26:55.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 99676 8.56 MB/sec execute 499 sec latency 57.099 ms 2024-09-19T06:26:56.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 99902 8.57 MB/sec execute 500 sec latency 54.725 ms 2024-09-19T06:26:56.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:56 smithi157 ceph-mon[35346]: pgmap v745: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 255 op/s 2024-09-19T06:26:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:56 smithi005 ceph-mon[30955]: pgmap v745: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 255 op/s 2024-09-19T06:26:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:56 smithi049 ceph-mon[34640]: pgmap v745: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 255 op/s 2024-09-19T06:26:57.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 100066 8.57 MB/sec execute 501 sec latency 69.722 ms 2024-09-19T06:26:58.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 100192 8.57 MB/sec execute 502 sec latency 86.538 ms 2024-09-19T06:26:58.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:58 smithi157 ceph-mon[35346]: pgmap v746: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.6 MiB/s wr, 424 op/s 2024-09-19T06:26:58.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:58 smithi005 ceph-mon[30955]: pgmap v746: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.6 MiB/s wr, 424 op/s 2024-09-19T06:26:58.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:58 smithi049 ceph-mon[34640]: pgmap v746: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.6 MiB/s wr, 424 op/s 2024-09-19T06:26:59.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 100336 8.56 MB/sec execute 503 sec latency 107.288 ms 2024-09-19T06:26:59.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:26:59 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:26:59.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:26:59 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:26:59.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:26:59 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:27:00.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 100508 8.57 MB/sec execute 504 sec latency 87.012 ms 2024-09-19T06:27:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:00 smithi157 ceph-mon[35346]: pgmap v747: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 292 op/s 2024-09-19T06:27:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:00 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:27:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:00 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:27:00.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:00 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:27:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:00 smithi005 ceph-mon[30955]: pgmap v747: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 292 op/s 2024-09-19T06:27:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:00 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:27:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:00 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:27:00.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:00 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:27:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:00 smithi049 ceph-mon[34640]: pgmap v747: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 292 op/s 2024-09-19T06:27:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:00 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:27:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:00 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:27:00.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:00 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:27:01.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 100662 8.56 MB/sec execute 505 sec latency 124.570 ms 2024-09-19T06:27:02.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 100815 8.56 MB/sec execute 506 sec latency 175.524 ms 2024-09-19T06:27:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:02 smithi005 ceph-mon[30955]: pgmap v748: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 292 op/s 2024-09-19T06:27:02.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:02 smithi049 ceph-mon[34640]: pgmap v748: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 292 op/s 2024-09-19T06:27:03.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:02 smithi157 ceph-mon[35346]: pgmap v748: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 292 op/s 2024-09-19T06:27:03.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 100978 8.56 MB/sec execute 507 sec latency 81.180 ms 2024-09-19T06:27:04.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 101121 8.56 MB/sec execute 508 sec latency 111.601 ms 2024-09-19T06:27:04.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:04 smithi005 ceph-mon[30955]: pgmap v749: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.0 MiB/s wr, 422 op/s 2024-09-19T06:27:04.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:04 smithi049 ceph-mon[34640]: pgmap v749: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.0 MiB/s wr, 422 op/s 2024-09-19T06:27:05.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:04 smithi157 ceph-mon[35346]: pgmap v749: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.0 MiB/s wr, 422 op/s 2024-09-19T06:27:05.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 101266 8.55 MB/sec execute 509 sec latency 145.012 ms 2024-09-19T06:27:05.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:05 smithi005 ceph-mon[30955]: osdmap e120: 12 total, 12 up, 12 in 2024-09-19T06:27:05.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:05 smithi049 ceph-mon[34640]: osdmap e120: 12 total, 12 up, 12 in 2024-09-19T06:27:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.954+0000 7ffaa1331640 1 -- 172.21.15.5:0/1822939521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c102460 msgr2=0x7ffa9c10e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.954+0000 7ffaa1331640 1 --2- 172.21.15.5:0/1822939521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c102460 0x7ffa9c10e990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009be0 tx=0x7ffa9402a6d0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.954+0000 7ffaa1331640 1 -- 172.21.15.5:0/1822939521 shutdown_connections 2024-09-19T06:27:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.954+0000 7ffaa1331640 1 --2- 172.21.15.5:0/1822939521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c102460 0x7ffa9c10e990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.954+0000 7ffaa1331640 1 --2- 172.21.15.5:0/1822939521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9c101ac0 0x7ffa9c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.954+0000 7ffaa1331640 1 --2- 172.21.15.5:0/1822939521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9c107ad0 0x7ffa9c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.954+0000 7ffaa1331640 1 -- 172.21.15.5:0/1822939521 >> 172.21.15.5:0/1822939521 conn(0x7ffa9c0fd7a0 msgr2=0x7ffa9c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.956+0000 7ffaa1331640 1 -- 172.21.15.5:0/1822939521 shutdown_connections 2024-09-19T06:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.956+0000 7ffaa1331640 1 -- 172.21.15.5:0/1822939521 wait complete. 2024-09-19T06:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.957+0000 7ffaa1331640 1 Processor -- start 2024-09-19T06:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.957+0000 7ffaa1331640 1 -- start start 2024-09-19T06:27:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffaa1331640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9c101ac0 0x7ffa9c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffaa1331640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9c102460 0x7ffa9c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffaa1331640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c107ad0 0x7ffa9c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffaa1331640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c111dc0 con 0x7ffa9c102460 2024-09-19T06:27:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffaa1331640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c111c40 con 0x7ffa9c107ad0 2024-09-19T06:27:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffaa1331640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c111f40 con 0x7ffa9c101ac0 2024-09-19T06:27:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9c102460 0x7ffa9c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffa9affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9c101ac0 0x7ffa9c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffa9affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9c101ac0 0x7ffa9c1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:49920/0 (socket says 172.21.15.5:49920) 2024-09-19T06:27:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9c102460 0x7ffa9c198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37476/0 (socket says 172.21.15.5:37476) 2024-09-19T06:27:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffa93fff640 1 -- 172.21.15.5:0/1280016000 learned_addr learned my addr 172.21.15.5:0/1280016000 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffa9b7fe640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c107ad0 0x7ffa9c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffa93fff640 1 -- 172.21.15.5:0/1280016000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9c101ac0 msgr2=0x7ffa9c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffa93fff640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9c101ac0 0x7ffa9c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffa93fff640 1 -- 172.21.15.5:0/1280016000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c107ad0 msgr2=0x7ffa9c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffa93fff640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c107ad0 0x7ffa9c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffa93fff640 1 -- 172.21.15.5:0/1280016000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9c19d2c0 con 0x7ffa9c102460 2024-09-19T06:27:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.958+0000 7ffa9b7fe640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c107ad0 0x7ffa9c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:27:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.959+0000 7ffa93fff640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9c102460 0x7ffa9c198810 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00e9f0 tx=0x7ffa8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.959+0000 7ffa98ff9640 1 -- 172.21.15.5:0/1280016000 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00ccc0 con 0x7ffa9c102460 2024-09-19T06:27:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.959+0000 7ffa98ff9640 1 -- 172.21.15.5:0/1280016000 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa8c0108e0 con 0x7ffa9c102460 2024-09-19T06:27:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.959+0000 7ffa98ff9640 1 -- 172.21.15.5:0/1280016000 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c0187c0 con 0x7ffa9c102460 2024-09-19T06:27:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.959+0000 7ffaa1331640 1 -- 172.21.15.5:0/1280016000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9c19d5b0 con 0x7ffa9c102460 2024-09-19T06:27:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.959+0000 7ffaa1331640 1 -- 172.21.15.5:0/1280016000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa9c074420 con 0x7ffa9c102460 2024-09-19T06:27:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.962+0000 7ffa98ff9640 1 -- 172.21.15.5:0/1280016000 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ffa8c010a90 con 0x7ffa9c102460 2024-09-19T06:27:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.962+0000 7ffaa40d0740 1 Processor -- start 2024-09-19T06:27:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.962+0000 7ffaa40d0740 1 -- start start 2024-09-19T06:27:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.963+0000 7ffa98ff9640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ffa7c0a3430 0x7ffa7c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.963+0000 7ffaa40d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a84af0 0x558c84a84ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.963+0000 7ffaa40d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a85410 0x558c84a8f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.963+0000 7ffa9b7fe640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ffa7c0a3430 0x7ffa7c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.963+0000 7ffa98ff9640 1 -- 172.21.15.5:0/1280016000 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8594+0+0 (secure 0 0 0) 0x7ffa8c0d0180 con 0x7ffa9c102460 2024-09-19T06:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.964+0000 7ffaa40d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.964+0000 7ffa9affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a84af0 0x558c84a84ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.964+0000 7ffa9affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a84af0 0x558c84a84ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58486/0 (socket says 172.21.15.5:58486) 2024-09-19T06:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.964+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a85410 0x558c84a8f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.964+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a85410 0x558c84a8f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:49932/0 (socket says 172.21.15.5:49932) 2024-09-19T06:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.964+0000 7ffa9affd640 1 -- 172.21.15.5:0/2828770767 learned_addr learned my addr 172.21.15.5:0/2828770767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.964+0000 7ffaa40d0740 1 -- 172.21.15.5:0/2828770767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558c849e92f0 con 0x558c84a8fcb0 2024-09-19T06:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.964+0000 7ffaa40d0740 1 -- 172.21.15.5:0/2828770767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558c84991270 con 0x558c84a84af0 2024-09-19T06:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.964+0000 7ffaa40d0740 1 -- 172.21.15.5:0/2828770767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558c849e52b0 con 0x558c84a85410 2024-09-19T06:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.964+0000 7ffa9b7fe640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ffa7c0a3430 0x7ffa7c0a58f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffa94004af0 tx=0x7ffa94004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.964+0000 7ffa9b7fe640 1 --2- 172.21.15.5:0/2828770767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.965+0000 7ffa93fff640 1 -- 172.21.15.5:0/2828770767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a84af0 msgr2=0x558c84a84ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.965+0000 7ffa93fff640 1 --2- 172.21.15.5:0/2828770767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a84af0 0x558c84a84ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.965+0000 7ffa93fff640 1 -- 172.21.15.5:0/2828770767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 msgr2=0x558c84a920a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.965+0000 7ffa93fff640 1 --2- 172.21.15.5:0/2828770767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a920a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.965+0000 7ffa93fff640 1 -- 172.21.15.5:0/2828770767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c84990340 con 0x558c84a85410 2024-09-19T06:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.965+0000 7ffa93fff640 1 --2- 172.21.15.5:0/2828770767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a85410 0x558c84a8f770 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0cd880 tx=0x7ffa8c09cb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9976df6530afeccd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.965+0000 7ffa917fa640 1 -- 172.21.15.5:0/2828770767 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c0a5080 con 0x558c84a85410 2024-09-19T06:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.965+0000 7ffa917fa640 1 -- 172.21.15.5:0/2828770767 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa8c0cdb00 con 0x558c84a85410 2024-09-19T06:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.965+0000 7ffa917fa640 1 -- 172.21.15.5:0/2828770767 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c098160 con 0x558c84a85410 2024-09-19T06:27:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.966+0000 7ffaa40d0740 1 -- 172.21.15.5:0/2828770767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a85410 msgr2=0x558c84a8f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.966+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/2828770767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a85410 0x558c84a8f770 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0cd880 tx=0x7ffa8c09cb10 comp rx=0 tx=0).stop 2024-09-19T06:27:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.966+0000 7ffaa40d0740 1 -- 172.21.15.5:0/2828770767 shutdown_connections 2024-09-19T06:27:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.966+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/2828770767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.966+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/2828770767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a85410 0x558c84a8f770 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.966+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/2828770767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a84af0 0x558c84a84ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.966+0000 7ffaa40d0740 1 -- 172.21.15.5:0/2828770767 >> 172.21.15.5:0/2828770767 conn(0x558c84a81060 msgr2=0x558c84a81440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.966+0000 7ffaa40d0740 1 -- 172.21.15.5:0/2828770767 shutdown_connections 2024-09-19T06:27:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.966+0000 7ffaa40d0740 1 -- 172.21.15.5:0/2828770767 wait complete. 2024-09-19T06:27:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffaa40d0740 1 -- start start 2024-09-19T06:27:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffaa40d0740 1 Processor -- start 2024-09-19T06:27:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffaa40d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a8cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffaa40d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a8d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffaa40d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a8d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffaa40d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558c84a92820 con 0x558c84a8fcb0 2024-09-19T06:27:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffaa40d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558c84972290 con 0x558c84a85410 2024-09-19T06:27:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffaa40d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558c84a929a0 con 0x558c84a84af0 2024-09-19T06:27:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffaa40d0740 10 client.0 _ll_register_callbacks cb 0x558c849a3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffa9affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a8cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffa9affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a8cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:49948/0 (socket says 172.21.15.5:49948) 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffa9affd640 1 -- 172.21.15.5:0/1701891483 learned_addr learned my addr 172.21.15.5:0/1701891483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffa93fff640 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a8d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.967+0000 7ffa9b7fe640 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a8d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.968+0000 7ffa93fff640 1 -- 172.21.15.5:0/1701891483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 msgr2=0x558c84a8cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.968+0000 7ffa93fff640 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a8cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.968+0000 7ffa93fff640 1 -- 172.21.15.5:0/1701891483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 msgr2=0x558c84a8d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.968+0000 7ffa93fff640 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a8d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.968+0000 7ffa93fff640 1 -- 172.21.15.5:0/1701891483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c846fc210 con 0x558c84a85410 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.968+0000 7ffa9b7fe640 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a8d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.968+0000 7ffa93fff640 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a8d170 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0cdfd0 tx=0x7ffa8c09bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92e2cc2284342dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.968+0000 7ffa8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.968+0000 7ffa8a7fc640 1 -- 172.21.15.5:0/1701891483 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c0b4020 con 0x558c84a85410 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.968+0000 7ffa8a7fc640 1 -- 172.21.15.5:0/1701891483 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 8594+0+0 (secure 0 0 0) 0x7ffa8c098370 con 0x558c84a85410 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.968+0000 7ffaa40d0740 1 -- 172.21.15.5:0/1701891483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa8c0cdb00 con 0x558c84a85410 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.969+0000 7ffa8a7fc640 1 -- 172.21.15.5:0/1701891483 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7ffa8c098c10 con 0x558c84a85410 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.969+0000 7ffaa40d0740 10 client.30596 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.969+0000 7ffaa40d0740 1 -- 172.21.15.5:0/1701891483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x558c847c8620 con 0x558c84a85410 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.969+0000 7ffa8a7fc640 1 -- 172.21.15.5:0/1701891483 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa8c0a5080 con 0x558c84a85410 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.969+0000 7ffa8a7fc640 1 -- 172.21.15.5:0/1701891483 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7ffa8c0a1950 con 0x558c84a85410 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.969+0000 7ffaa40d0740 10 client.30596 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.969+0000 7ffaa40d0740 10 client.30596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.969+0000 7ffaa40d0740 20 client.30596 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.969+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558c84aa82d0 0x558c84aaa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.969+0000 7ffaa40d0740 4 client.30596 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.969+0000 7ffaa40d0740 1 -- 172.21.15.5:0/1701891483 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffa8c0cdb00 con 0x558c84aa82d0 2024-09-19T06:27:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.969+0000 7ffa9affd640 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558c84aa82d0 0x558c84aaa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.970+0000 7ffa8a7fc640 10 client.30596 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.970+0000 7ffa9affd640 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558c84aa82d0 0x558c84aaa6c0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ea04a5d206521a9 server_cookie=db895fd341c25f51 in_seq=0 out_seq=0 2024-09-19T06:27:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.970+0000 7ffa8a7fc640 1 -- 172.21.15.5:0/1701891483 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ffa8c0cdb00 con 0x558c84aa82d0 2024-09-19T06:27:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.970+0000 7ffa8a7fc640 10 client.30596 handle_command_reply: tid=0 2024-09-19T06:27:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.970+0000 7ffaa40d0740 1 client.30596 shutdown 2024-09-19T06:27:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.971+0000 7ffaa40d0740 1 -- 172.21.15.5:0/1701891483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 msgr2=0x558c84a8d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.971+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a8d170 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0cdfd0 tx=0x7ffa8c09bc70 comp rx=0 tx=0).stop 2024-09-19T06:27:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.971+0000 7ffaa40d0740 1 -- 172.21.15.5:0/1701891483 shutdown_connections 2024-09-19T06:27:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.971+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558c84aa82d0 0x558c84aaa6c0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.971+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a8d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.971+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a8d170 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.971+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/1701891483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a8cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.971+0000 7ffaa40d0740 1 -- 172.21.15.5:0/1701891483 >> 172.21.15.5:0/1701891483 conn(0x558c84a81060 msgr2=0x558c84a86d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.971+0000 7ffa8a7fc640 0 client.30596 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.971+0000 7ffaa40d0740 1 -- 172.21.15.5:0/1701891483 shutdown_connections 2024-09-19T06:27:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.971+0000 7ffaa40d0740 1 -- 172.21.15.5:0/1701891483 wait complete. 2024-09-19T06:27:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.971+0000 7ffaa40d0740 20 client.30596 trim_cache size 0 max 16384 2024-09-19T06:27:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.999+0000 7ffaa40d0740 1 Processor -- start 2024-09-19T06:27:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:05.999+0000 7ffaa40d0740 1 -- start start 2024-09-19T06:27:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.000+0000 7ffaa40d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a87560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.000+0000 7ffaa40d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a8e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.000+0000 7ffaa40d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a8d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.000+0000 7ffaa40d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1ab6d0 con 0x558c84a8fcb0 2024-09-19T06:27:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.000+0000 7ffaa40d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1ad210 con 0x558c84a85410 2024-09-19T06:27:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.000+0000 7ffaa40d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1abda0 con 0x558c84a84af0 2024-09-19T06:27:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.000+0000 7ffa9affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a87560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.000+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a8e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.000+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a8e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58504/0 (socket says 172.21.15.5:58504) 2024-09-19T06:27:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.000+0000 7ffa93fff640 1 -- 172.21.15.5:0/3382654622 learned_addr learned my addr 172.21.15.5:0/3382654622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.000+0000 7ffa9b7fe640 1 --2- 172.21.15.5:0/3382654622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a8d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.001+0000 7ffa93fff640 1 -- 172.21.15.5:0/3382654622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 msgr2=0x558c84a87560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.001+0000 7ffa93fff640 1 --2- 172.21.15.5:0/3382654622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a87560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.001+0000 7ffa93fff640 1 -- 172.21.15.5:0/3382654622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 msgr2=0x558c84a8d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.001+0000 7ffa93fff640 1 --2- 172.21.15.5:0/3382654622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a8d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.001+0000 7ffa93fff640 1 -- 172.21.15.5:0/3382654622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c84a819a0 con 0x558c84a85410 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.001+0000 7ffa93fff640 1 --2- 172.21.15.5:0/3382654622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a8e490 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c010f90 tx=0x7ffa8c0a0940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5aa4893665fe0cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.002+0000 7ffa90ff9640 1 -- 172.21.15.5:0/3382654622 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c09b860 con 0x558c84a85410 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.002+0000 7ffa90ff9640 1 -- 172.21.15.5:0/3382654622 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa8c0a5080 con 0x558c84a85410 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.002+0000 7ffa90ff9640 1 -- 172.21.15.5:0/3382654622 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c0bc360 con 0x558c84a85410 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.003+0000 7ffaa40d0740 1 -- 172.21.15.5:0/3382654622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 msgr2=0x558c84a8e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.003+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/3382654622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a8e490 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c010f90 tx=0x7ffa8c0a0940 comp rx=0 tx=0).stop 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.003+0000 7ffaa40d0740 1 -- 172.21.15.5:0/3382654622 shutdown_connections 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.003+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/3382654622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a8d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.003+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/3382654622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a8e490 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.003+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/3382654622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a87560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.003+0000 7ffaa40d0740 1 -- 172.21.15.5:0/3382654622 >> 172.21.15.5:0/3382654622 conn(0x558c84a81060 msgr2=0x558c84a8bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.003+0000 7ffaa40d0740 1 -- 172.21.15.5:0/3382654622 shutdown_connections 2024-09-19T06:27:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.003+0000 7ffaa40d0740 1 -- 172.21.15.5:0/3382654622 wait complete. 2024-09-19T06:27:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.004+0000 7ffaa40d0740 1 -- start start 2024-09-19T06:27:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.004+0000 7ffaa40d0740 1 Processor -- start 2024-09-19T06:27:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.005+0000 7ffaa40d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a97ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.005+0000 7ffaa40d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a98410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.005+0000 7ffaa40d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.005+0000 7ffa9affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a97ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.005+0000 7ffa9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.005+0000 7ffa9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a894a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37510/0 (socket says 172.21.15.5:37510) 2024-09-19T06:27:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.005+0000 7ffa9b7fe640 1 -- 172.21.15.5:0/518812899 learned_addr learned my addr 172.21.15.5:0/518812899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.005+0000 7ffaa40d0740 1 -- 172.21.15.5:0/518812899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1acb40 con 0x558c84a8fcb0 2024-09-19T06:27:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.005+0000 7ffaa40d0740 1 -- 172.21.15.5:0/518812899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1ac470 con 0x558c84a85410 2024-09-19T06:27:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.005+0000 7ffaa40d0740 1 -- 172.21.15.5:0/518812899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1a96d0 con 0x558c84a84af0 2024-09-19T06:27:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.005+0000 7ffaa40d0740 10 client.0 _ll_register_callbacks cb 0x7ffa94034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.005+0000 7ffa93fff640 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a98410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.006+0000 7ffa9affd640 1 -- 172.21.15.5:0/518812899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 msgr2=0x558c84a98410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.006+0000 7ffa9affd640 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a98410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.006+0000 7ffa9affd640 1 -- 172.21.15.5:0/518812899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 msgr2=0x558c84a894a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.006+0000 7ffa9affd640 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a894a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.006+0000 7ffa9affd640 1 -- 172.21.15.5:0/518812899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c84a89cd0 con 0x558c84a84af0 2024-09-19T06:27:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.006+0000 7ffa93fff640 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a98410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:27:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.006+0000 7ffa9b7fe640 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.007+0000 7ffa9affd640 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a97ed0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ffa84005fd0 tx=0x7ffa840045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8689b3335a833b49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.007+0000 7ffa8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:27:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.007+0000 7ffa8a7fc640 1 -- 172.21.15.5:0/518812899 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8401d640 con 0x558c84a84af0 2024-09-19T06:27:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.007+0000 7ffa8a7fc640 1 -- 172.21.15.5:0/518812899 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 8594+0+0 (secure 0 0 0) 0x7ffa8401d7e0 con 0x558c84a84af0 2024-09-19T06:27:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.007+0000 7ffaa40d0740 1 -- 172.21.15.5:0/518812899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa8c0a5080 con 0x558c84a84af0 2024-09-19T06:27:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.007+0000 7ffa8a7fc640 1 -- 172.21.15.5:0/518812899 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7ffa8401d9e0 con 0x558c84a84af0 2024-09-19T06:27:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.007+0000 7ffaa40d0740 10 client.30538 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.007+0000 7ffaa40d0740 1 -- 172.21.15.5:0/518812899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x558c84b127e0 con 0x558c84a84af0 2024-09-19T06:27:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.008+0000 7ffa8a7fc640 1 -- 172.21.15.5:0/518812899 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa8c0a5080 con 0x558c84a84af0 2024-09-19T06:27:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.008+0000 7ffa8a7fc640 1 -- 172.21.15.5:0/518812899 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7ffa84030c90 con 0x558c84a84af0 2024-09-19T06:27:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.008+0000 7ffaa40d0740 10 client.30538 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.008+0000 7ffaa40d0740 10 client.30538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.008+0000 7ffaa40d0740 20 client.30538 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.008+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558c84b12f10 0x558c84b15330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.008+0000 7ffaa40d0740 4 client.30538 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:27:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.008+0000 7ffaa40d0740 1 -- 172.21.15.5:0/518812899 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c849f6a20 con 0x558c84b12f10 2024-09-19T06:27:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.008+0000 7ffa93fff640 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558c84b12f10 0x558c84b15330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.009+0000 7ffa8a7fc640 10 client.30538 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.009+0000 7ffa93fff640 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558c84b12f10 0x558c84b15330 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c93cfaf26029f43 server_cookie=7d7d43405fa13b1c in_seq=0 out_seq=0 2024-09-19T06:27:06.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:05 smithi157 ceph-mon[35346]: osdmap e120: 12 total, 12 up, 12 in 2024-09-19T06:27:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.110+0000 7ffa8a7fc640 1 -- 172.21.15.5:0/518812899 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c849f6a20 con 0x558c84b12f10 2024-09-19T06:27:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.110+0000 7ffa8a7fc640 10 client.30538 handle_command_reply: tid=0 2024-09-19T06:27:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.110+0000 7ffaa40d0740 1 client.30538 shutdown 2024-09-19T06:27:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.110+0000 7ffaa40d0740 1 -- 172.21.15.5:0/518812899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 msgr2=0x558c84a97ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.110+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a97ed0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ffa84005fd0 tx=0x7ffa840045a0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.110+0000 7ffaa40d0740 1 -- 172.21.15.5:0/518812899 shutdown_connections 2024-09-19T06:27:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.110+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558c84b12f10 0x558c84b15330 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.110+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c84a8fcb0 0x558c84a894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.110+0000 7ffa8a7fc640 0 client.30538 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.110+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c84a85410 0x558c84a98410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.110+0000 7ffaa40d0740 1 --2- 172.21.15.5:0/518812899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c84a84af0 0x558c84a97ed0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.110+0000 7ffaa40d0740 1 -- 172.21.15.5:0/518812899 >> 172.21.15.5:0/518812899 conn(0x558c84a81060 msgr2=0x558c84a9aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.111+0000 7ffaa40d0740 1 -- 172.21.15.5:0/518812899 shutdown_connections 2024-09-19T06:27:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.111+0000 7ffaa40d0740 1 -- 172.21.15.5:0/518812899 wait complete. 2024-09-19T06:27:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.111+0000 7ffaa40d0740 20 client.30538 trim_cache size 0 max 16384 2024-09-19T06:27:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:27:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:27:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:27:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:27:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.111+0000 7ffaa1331640 1 -- 172.21.15.5:0/1280016000 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ffa7c0a3430 msgr2=0x7ffa7c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.111+0000 7ffaa1331640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ffa7c0a3430 0x7ffa7c0a58f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffa94004af0 tx=0x7ffa94004a40 comp rx=0 tx=0).stop 2024-09-19T06:27:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.111+0000 7ffaa1331640 1 -- 172.21.15.5:0/1280016000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9c102460 msgr2=0x7ffa9c198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.111+0000 7ffaa1331640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9c102460 0x7ffa9c198810 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00e9f0 tx=0x7ffa8c00edf0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.111+0000 7ffaa1331640 1 -- 172.21.15.5:0/1280016000 shutdown_connections 2024-09-19T06:27:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.112+0000 7ffaa1331640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c107ad0 0x7ffa9c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.112+0000 7ffaa1331640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9c102460 0x7ffa9c198810 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.112+0000 7ffaa1331640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ffa7c0a3430 0x7ffa7c0a58f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.112+0000 7ffaa1331640 1 --2- 172.21.15.5:0/1280016000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9c101ac0 0x7ffa9c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.112+0000 7ffaa1331640 1 -- 172.21.15.5:0/1280016000 >> 172.21.15.5:0/1280016000 conn(0x7ffa9c0fd7a0 msgr2=0x7ffa9c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.112+0000 7ffaa1331640 1 -- 172.21.15.5:0/1280016000 shutdown_connections 2024-09-19T06:27:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:06.112+0000 7ffaa1331640 1 -- 172.21.15.5:0/1280016000 wait complete. 2024-09-19T06:27:06.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 101463 8.56 MB/sec execute 510 sec latency 102.523 ms 2024-09-19T06:27:06.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:06 smithi005 ceph-mon[30955]: pgmap v751: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.5 MiB/s wr, 359 op/s 2024-09-19T06:27:06.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:06 smithi049 ceph-mon[34640]: pgmap v751: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.5 MiB/s wr, 359 op/s 2024-09-19T06:27:07.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:06 smithi157 ceph-mon[35346]: pgmap v751: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.5 MiB/s wr, 359 op/s 2024-09-19T06:27:07.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 101623 8.56 MB/sec execute 511 sec latency 123.203 ms 2024-09-19T06:27:08.379 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:27:08.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 101789 8.57 MB/sec execute 512 sec latency 289.561 ms 2024-09-19T06:27:08.765 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:27:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:08 smithi005 ceph-mon[30955]: pgmap v752: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.5 MiB/s wr, 328 op/s 2024-09-19T06:27:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:08 smithi005 ceph-mon[30955]: osdmap e121: 12 total, 12 up, 12 in 2024-09-19T06:27:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:08 smithi049 ceph-mon[34640]: pgmap v752: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.5 MiB/s wr, 328 op/s 2024-09-19T06:27:08.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:08 smithi049 ceph-mon[34640]: osdmap e121: 12 total, 12 up, 12 in 2024-09-19T06:27:09.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:08 smithi157 ceph-mon[35346]: pgmap v752: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.5 MiB/s wr, 328 op/s 2024-09-19T06:27:09.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:08 smithi157 ceph-mon[35346]: osdmap e121: 12 total, 12 up, 12 in 2024-09-19T06:27:09.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.375+0000 7f7f282e6640 1 -- 172.21.15.5:0/1323339180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f20102d30 msgr2=0x7f7f20103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:09.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.375+0000 7f7f282e6640 1 --2- 172.21.15.5:0/1323339180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f20102d30 0x7f7f20103110 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009900 tx=0x7f7f1c02a480 comp rx=0 tx=0).stop 2024-09-19T06:27:09.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.375+0000 7f7f282e6640 1 -- 172.21.15.5:0/1323339180 shutdown_connections 2024-09-19T06:27:09.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.375+0000 7f7f282e6640 1 --2- 172.21.15.5:0/1323339180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f2010c160 0x7f7f2010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.375+0000 7f7f282e6640 1 --2- 172.21.15.5:0/1323339180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f20103650 0x7f7f2010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.375+0000 7f7f282e6640 1 --2- 172.21.15.5:0/1323339180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f20102d30 0x7f7f20103110 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.375+0000 7f7f282e6640 1 -- 172.21.15.5:0/1323339180 >> 172.21.15.5:0/1323339180 conn(0x7f7f200fc790 msgr2=0x7f7f200febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:09.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.375+0000 7f7f282e6640 1 -- 172.21.15.5:0/1323339180 shutdown_connections 2024-09-19T06:27:09.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.375+0000 7f7f282e6640 1 -- 172.21.15.5:0/1323339180 wait complete. 2024-09-19T06:27:09.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.375+0000 7f7f282e6640 1 Processor -- start 2024-09-19T06:27:09.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f282e6640 1 -- start start 2024-09-19T06:27:09.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f282e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f20102d30 0x7f7f2019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f282e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f20103650 0x7f7f2019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f282e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f2010c160 0x7f7f2019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f282e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20111e10 con 0x7f7f20102d30 2024-09-19T06:27:09.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f282e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20111c90 con 0x7f7f20103650 2024-09-19T06:27:09.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f282e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20111f90 con 0x7f7f2010c160 2024-09-19T06:27:09.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f26082640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f20102d30 0x7f7f2019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f26082640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f20102d30 0x7f7f2019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37528/0 (socket says 172.21.15.5:37528) 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f25881640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f20103650 0x7f7f2019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f26082640 1 -- 172.21.15.5:0/3600821504 learned_addr learned my addr 172.21.15.5:0/3600821504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f26883640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f2010c160 0x7f7f2019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f25881640 1 -- 172.21.15.5:0/3600821504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f2010c160 msgr2=0x7f7f2019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f25881640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f2010c160 0x7f7f2019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f25881640 1 -- 172.21.15.5:0/3600821504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f20102d30 msgr2=0x7f7f2019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f25881640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f20102d30 0x7f7f2019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.376+0000 7f7f25881640 1 -- 172.21.15.5:0/3600821504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2019f520 con 0x7f7f20103650 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.377+0000 7f7f25881640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f20103650 0x7f7f2019aab0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000e7a0 tx=0x7f7f1000ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.377+0000 7f7f177fe640 1 -- 172.21.15.5:0/3600821504 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1000ccc0 con 0x7f7f20103650 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.377+0000 7f7f282e6640 1 -- 172.21.15.5:0/3600821504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2019f810 con 0x7f7f20103650 2024-09-19T06:27:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.377+0000 7f7f177fe640 1 -- 172.21.15.5:0/3600821504 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f10009530 con 0x7f7f20103650 2024-09-19T06:27:09.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.377+0000 7f7f177fe640 1 -- 172.21.15.5:0/3600821504 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10018710 con 0x7f7f20103650 2024-09-19T06:27:09.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.377+0000 7f7f282e6640 1 -- 172.21.15.5:0/3600821504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f201a70f0 con 0x7f7f20103650 2024-09-19T06:27:09.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f2a6bf740 1 Processor -- start 2024-09-19T06:27:09.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f2a6bf740 1 -- start start 2024-09-19T06:27:09.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f177fe640 1 -- 172.21.15.5:0/3600821504 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7f10010430 con 0x7f7f20103650 2024-09-19T06:27:09.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f2a6bf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 0x55fde56f9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f2a6bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 0x55fde5704140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f2a6bf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde5704680 0x55fde5706a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f2a6bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fde565f2e0 con 0x55fde56f9dc0 2024-09-19T06:27:09.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f2a6bf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fde5606ce0 con 0x55fde5704680 2024-09-19T06:27:09.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f2a6bf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fde565b2a0 con 0x55fde56f94a0 2024-09-19T06:27:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f26082640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 0x55fde5704140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f26082640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 0x55fde5704140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37530/0 (socket says 172.21.15.5:37530) 2024-09-19T06:27:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f26082640 1 -- 172.21.15.5:0/1130327039 learned_addr learned my addr 172.21.15.5:0/1130327039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f25881640 1 --2- 172.21.15.5:0/1130327039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde5704680 0x55fde5706a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.380+0000 7f7f26883640 1 --2- 172.21.15.5:0/1130327039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 0x55fde56f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.381+0000 7f7f26883640 1 -- 172.21.15.5:0/1130327039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde5704680 msgr2=0x55fde5706a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.381+0000 7f7f26883640 1 --2- 172.21.15.5:0/1130327039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde5704680 0x55fde5706a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.381+0000 7f7f26883640 1 -- 172.21.15.5:0/1130327039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 msgr2=0x55fde5704140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.381+0000 7f7f26883640 1 --2- 172.21.15.5:0/1130327039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 0x55fde5704140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.381+0000 7f7f26883640 1 -- 172.21.15.5:0/1130327039 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fde55eb440 con 0x55fde56f94a0 2024-09-19T06:27:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.381+0000 7f7f26082640 1 --2- 172.21.15.5:0/1130327039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 0x55fde5704140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.381+0000 7f7f26883640 1 --2- 172.21.15.5:0/1130327039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 0x55fde56f9880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009f00 tx=0x7f7f18011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=821f9e2c4b168dbc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:09.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.381+0000 7f7f177fe640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7f0c0a3430 0x7f7f0c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.381+0000 7f7f14dea640 1 -- 172.21.15.5:0/1130327039 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18004d20 con 0x55fde56f94a0 2024-09-19T06:27:09.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.382+0000 7f7f177fe640 1 -- 172.21.15.5:0/3600821504 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f7f100cfee0 con 0x7f7f20103650 2024-09-19T06:27:09.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.382+0000 7f7f14dea640 1 -- 172.21.15.5:0/1130327039 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f18004050 con 0x55fde56f94a0 2024-09-19T06:27:09.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.382+0000 7f7f26082640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7f0c0a3430 0x7f7f0c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.382+0000 7f7f14dea640 1 -- 172.21.15.5:0/1130327039 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18005280 con 0x55fde56f94a0 2024-09-19T06:27:09.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.383+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/1130327039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 msgr2=0x55fde56f9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:09.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.383+0000 7f7f26082640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7f0c0a3430 0x7f7f0c0a58f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55fde56f6890 tx=0x7f7f1c0058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:09.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.383+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/1130327039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 0x55fde56f9880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009f00 tx=0x7f7f18011810 comp rx=0 tx=0).stop 2024-09-19T06:27:09.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.383+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/1130327039 shutdown_connections 2024-09-19T06:27:09.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.383+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/1130327039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde5704680 0x55fde5706a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.383+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/1130327039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 0x55fde5704140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.383+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/1130327039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 0x55fde56f9880 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.383+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/1130327039 >> 172.21.15.5:0/1130327039 conn(0x55fde56f5a30 msgr2=0x55fde56f5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:09.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.383+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/1130327039 shutdown_connections 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.383+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/1130327039 wait complete. 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.383+0000 7f7f2a6bf740 1 -- start start 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.383+0000 7f7f2a6bf740 1 Processor -- start 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f2a6bf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f94a0 0x55fde5703f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f2a6bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 0x55fde57011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f2a6bf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde5704680 0x55fde5701710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f2a6bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fde57072f0 con 0x55fde56f9dc0 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f2a6bf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fde561a050 con 0x55fde56f94a0 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f2a6bf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fde5707470 con 0x55fde5704680 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f2a6bf740 10 client.0 _ll_register_callbacks cb 0x55fde565a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f25881640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 0x55fde57011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f25881640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 0x55fde57011d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37544/0 (socket says 172.21.15.5:37544) 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f25881640 1 -- 172.21.15.5:0/3723194554 learned_addr learned my addr 172.21.15.5:0/3723194554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:09.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f26883640 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde5704680 0x55fde5701710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f26082640 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f94a0 0x55fde5703f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f26883640 1 -- 172.21.15.5:0/3723194554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f94a0 msgr2=0x55fde5703f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f26883640 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f94a0 0x55fde5703f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f26883640 1 -- 172.21.15.5:0/3723194554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 msgr2=0x55fde57011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f26883640 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 0x55fde57011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f26883640 1 -- 172.21.15.5:0/3723194554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fde53d03c0 con 0x55fde5704680 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f25881640 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 0x55fde57011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f26883640 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde5704680 0x55fde5701710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7f1801b040 tx=0x7f7f18004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5902ea9838e53b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.384+0000 7f7f09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f09ffb640 1 -- 172.21.15.5:0/3723194554 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18028020 con 0x55fde5704680 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f09ffb640 1 -- 172.21.15.5:0/3723194554 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f7f18004c50 con 0x55fde5704680 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3723194554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f18004050 con 0x55fde5704680 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f09ffb640 1 -- 172.21.15.5:0/3723194554 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f7f18033050 con 0x55fde5704680 2024-09-19T06:27:09.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f2a6bf740 10 client.30550 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:09.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3723194554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55fde5712c60 con 0x55fde5704680 2024-09-19T06:27:09.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f09ffb640 1 -- 172.21.15.5:0/3723194554 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f18004050 con 0x55fde5704680 2024-09-19T06:27:09.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f09ffb640 1 -- 172.21.15.5:0/3723194554 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f7f18023450 con 0x55fde5704680 2024-09-19T06:27:09.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f2a6bf740 10 client.30550 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:09.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f2a6bf740 10 client.30550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:09.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f2a6bf740 20 client.30550 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:09.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fde57140c0 0x55fde57164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f2a6bf740 4 client.30550 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:09.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.385+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3723194554 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fde566c850 con 0x55fde57140c0 2024-09-19T06:27:09.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.431+0000 7f7f26082640 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fde57140c0 0x55fde57164b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.431+0000 7f7f26082640 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fde57140c0 0x55fde57164b0 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddf93c4eb3eb5004 server_cookie=f35f569722e2186c in_seq=0 out_seq=0 2024-09-19T06:27:09.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.431+0000 7f7f09ffb640 10 client.30550 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:09.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.434+0000 7f7f09ffb640 1 -- 172.21.15.5:0/3723194554 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fde566c850 con 0x55fde57140c0 2024-09-19T06:27:09.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.434+0000 7f7f09ffb640 10 client.30550 handle_command_reply: tid=0 2024-09-19T06:27:09.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.434+0000 7f7f2a6bf740 1 client.30550 shutdown 2024-09-19T06:27:09.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.434+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3723194554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde5704680 msgr2=0x55fde5701710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:09.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.434+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde5704680 0x55fde5701710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7f1801b040 tx=0x7f7f18004430 comp rx=0 tx=0).stop 2024-09-19T06:27:09.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.434+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3723194554 shutdown_connections 2024-09-19T06:27:09.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.434+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fde57140c0 0x55fde57164b0 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.434+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde5704680 0x55fde5701710 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.434+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f9dc0 0x55fde57011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.434+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/3723194554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f94a0 0x55fde5703f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.434+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3723194554 >> 172.21.15.5:0/3723194554 conn(0x55fde56f5a30 msgr2=0x55fde56f4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:09.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.434+0000 7f7f09ffb640 0 client.30550 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:09.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.435+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3723194554 shutdown_connections 2024-09-19T06:27:09.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.435+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3723194554 wait complete. 2024-09-19T06:27:09.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.435+0000 7f7f2a6bf740 20 client.30550 trim_cache size 0 max 16384 2024-09-19T06:27:09.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f2a6bf740 1 Processor -- start 2024-09-19T06:27:09.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f2a6bf740 1 -- start start 2024-09-19T06:27:09.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f2a6bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 0x55fde57160d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f2a6bf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 0x55fde5703140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f2a6bf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f9dc0 0x55fde571bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f2a6bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201afa50 con 0x55fde56f5a30 2024-09-19T06:27:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f2a6bf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201b1590 con 0x55fde56f9dc0 2024-09-19T06:27:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f2a6bf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201b0120 con 0x55fde56f94a0 2024-09-19T06:27:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f26082640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 0x55fde57160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f25881640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 0x55fde5703140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f25881640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 0x55fde5703140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50022/0 (socket says 172.21.15.5:50022) 2024-09-19T06:27:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f26082640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 0x55fde57160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37546/0 (socket says 172.21.15.5:37546) 2024-09-19T06:27:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f25881640 1 -- 172.21.15.5:0/3275171250 learned_addr learned my addr 172.21.15.5:0/3275171250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.461+0000 7f7f26883640 1 --2- 172.21.15.5:0/3275171250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f9dc0 0x55fde571bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.462+0000 7f7f25881640 1 -- 172.21.15.5:0/3275171250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f9dc0 msgr2=0x55fde571bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.462+0000 7f7f25881640 1 --2- 172.21.15.5:0/3275171250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f9dc0 0x55fde571bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.462+0000 7f7f25881640 1 -- 172.21.15.5:0/3275171250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 msgr2=0x55fde57160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:09.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.462+0000 7f7f25881640 1 --2- 172.21.15.5:0/3275171250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 0x55fde57160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.462+0000 7f7f25881640 1 -- 172.21.15.5:0/3275171250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fde56f6ae0 con 0x55fde56f94a0 2024-09-19T06:27:09.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.462+0000 7f7f25881640 1 --2- 172.21.15.5:0/3275171250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 0x55fde5703140 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000e770 tx=0x7f7f100d3a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c60eeef1e49a01f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:09.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.462+0000 7f7f0bfff640 1 -- 172.21.15.5:0/3275171250 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1009c020 con 0x55fde56f94a0 2024-09-19T06:27:09.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.462+0000 7f7f0bfff640 1 -- 172.21.15.5:0/3275171250 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f1009c250 con 0x55fde56f94a0 2024-09-19T06:27:09.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.462+0000 7f7f0bfff640 1 -- 172.21.15.5:0/3275171250 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1009bbb0 con 0x55fde56f94a0 2024-09-19T06:27:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.463+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3275171250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 msgr2=0x55fde5703140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.463+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/3275171250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 0x55fde5703140 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000e770 tx=0x7f7f100d3a30 comp rx=0 tx=0).stop 2024-09-19T06:27:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.463+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3275171250 shutdown_connections 2024-09-19T06:27:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.463+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/3275171250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f9dc0 0x55fde571bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.463+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/3275171250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f94a0 0x55fde5703140 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.463+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/3275171250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 0x55fde57160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.463+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3275171250 >> 172.21.15.5:0/3275171250 conn(0x55fde5701870 msgr2=0x55fde5702a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.463+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3275171250 shutdown_connections 2024-09-19T06:27:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.463+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/3275171250 wait complete. 2024-09-19T06:27:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.463+0000 7f7f2a6bf740 1 -- start start 2024-09-19T06:27:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.463+0000 7f7f2a6bf740 1 Processor -- start 2024-09-19T06:27:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f2a6bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 0x55fde570cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f2a6bf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f94a0 0x55fde570cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f2a6bf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f9dc0 0x55fde570a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f2a6bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201b0ec0 con 0x55fde56f5a30 2024-09-19T06:27:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f2a6bf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201b07f0 con 0x55fde56f94a0 2024-09-19T06:27:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f26082640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 0x55fde570cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f2a6bf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201ada50 con 0x55fde56f9dc0 2024-09-19T06:27:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f25881640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f94a0 0x55fde570cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f26082640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 0x55fde570cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37558/0 (socket says 172.21.15.5:37558) 2024-09-19T06:27:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f26883640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f9dc0 0x55fde570a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f26082640 1 -- 172.21.15.5:0/2863333917 learned_addr learned my addr 172.21.15.5:0/2863333917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f2a6bf740 10 client.0 _ll_register_callbacks cb 0x7f7f180093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f26082640 1 -- 172.21.15.5:0/2863333917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f9dc0 msgr2=0x55fde570a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f26082640 1 --2- 172.21.15.5:0/2863333917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f9dc0 0x55fde570a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f26082640 1 -- 172.21.15.5:0/2863333917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f94a0 msgr2=0x55fde570cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f26082640 1 --2- 172.21.15.5:0/2863333917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f94a0 0x55fde570cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.464+0000 7f7f26082640 1 -- 172.21.15.5:0/2863333917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fde56f66c0 con 0x55fde56f5a30 2024-09-19T06:27:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.465+0000 7f7f26082640 1 --2- 172.21.15.5:0/2863333917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 0x55fde570cab0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006010 tx=0x7f7f1c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d582b320197492bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.465+0000 7f7f09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:27:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.465+0000 7f7f09ffb640 1 -- 172.21.15.5:0/2863333917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c035040 con 0x55fde56f5a30 2024-09-19T06:27:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.465+0000 7f7f09ffb640 1 -- 172.21.15.5:0/2863333917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f7f1c0037f0 con 0x55fde56f5a30 2024-09-19T06:27:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.465+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/2863333917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f1009c250 con 0x55fde56f5a30 2024-09-19T06:27:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.465+0000 7f7f09ffb640 1 -- 172.21.15.5:0/2863333917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f7f1c0039f0 con 0x55fde56f5a30 2024-09-19T06:27:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.465+0000 7f7f2a6bf740 10 client.21249 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.465+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/2863333917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55fde577e680 con 0x55fde56f5a30 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.465+0000 7f7f09ffb640 1 -- 172.21.15.5:0/2863333917 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f1009c250 con 0x55fde56f5a30 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.466+0000 7f7f09ffb640 1 -- 172.21.15.5:0/2863333917 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f7f1c054b20 con 0x55fde56f5a30 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.466+0000 7f7f2a6bf740 10 client.21249 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.466+0000 7f7f2a6bf740 10 client.21249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.466+0000 7f7f2a6bf740 20 client.21249 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.466+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/2863333917 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fde5704680 0x55fde5780de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.466+0000 7f7f2a6bf740 4 client.21249 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.466+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/2863333917 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fde5666d60 con 0x55fde5704680 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.466+0000 7f7f25881640 1 --2- 172.21.15.5:0/2863333917 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fde5704680 0x55fde5780de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.466+0000 7f7f09ffb640 10 client.21249 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.466+0000 7f7f25881640 1 --2- 172.21.15.5:0/2863333917 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fde5704680 0x55fde5780de0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae4d790f075a6b51 server_cookie=2bc5cc905edbc378 in_seq=0 out_seq=0 2024-09-19T06:27:09.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f09ffb640 1 -- 172.21.15.5:0/2863333917 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fde5666d60 con 0x55fde5704680 2024-09-19T06:27:09.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f09ffb640 10 client.21249 handle_command_reply: tid=0 2024-09-19T06:27:09.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f2a6bf740 1 client.21249 shutdown 2024-09-19T06:27:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/2863333917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 msgr2=0x55fde570cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/2863333917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 0x55fde570cab0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006010 tx=0x7f7f1c009260 comp rx=0 tx=0).stop 2024-09-19T06:27:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/2863333917 shutdown_connections 2024-09-19T06:27:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/2863333917 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55fde5704680 0x55fde5780de0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/2863333917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fde56f9dc0 0x55fde570a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/2863333917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fde56f94a0 0x55fde570cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f2a6bf740 1 --2- 172.21.15.5:0/2863333917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fde56f5a30 0x55fde570cab0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/2863333917 >> 172.21.15.5:0/2863333917 conn(0x55fde5701870 msgr2=0x55fde5703250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f09ffb640 0 client.21249 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/2863333917 shutdown_connections 2024-09-19T06:27:09.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f2a6bf740 1 -- 172.21.15.5:0/2863333917 wait complete. 2024-09-19T06:27:09.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.467+0000 7f7f2a6bf740 20 client.21249 trim_cache size 0 max 16384 2024-09-19T06:27:09.474 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:27:09.474 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:27:09.474 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:27:09.474 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:27:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.468+0000 7f7f282e6640 1 -- 172.21.15.5:0/3600821504 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7f0c0a3430 msgr2=0x7f7f0c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.468+0000 7f7f282e6640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7f0c0a3430 0x7f7f0c0a58f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55fde56f6890 tx=0x7f7f1c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.468+0000 7f7f282e6640 1 -- 172.21.15.5:0/3600821504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f20103650 msgr2=0x7f7f2019aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.468+0000 7f7f282e6640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f20103650 0x7f7f2019aab0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000e7a0 tx=0x7f7f1000ec60 comp rx=0 tx=0).stop 2024-09-19T06:27:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.468+0000 7f7f282e6640 1 -- 172.21.15.5:0/3600821504 shutdown_connections 2024-09-19T06:27:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.468+0000 7f7f282e6640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7f0c0a3430 0x7f7f0c0a58f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.468+0000 7f7f282e6640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f2010c160 0x7f7f2019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.468+0000 7f7f282e6640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f20103650 0x7f7f2019aab0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.468+0000 7f7f282e6640 1 --2- 172.21.15.5:0/3600821504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f20102d30 0x7f7f2019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.468+0000 7f7f282e6640 1 -- 172.21.15.5:0/3600821504 >> 172.21.15.5:0/3600821504 conn(0x7f7f200fc790 msgr2=0x7f7f201008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:09.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.468+0000 7f7f282e6640 1 -- 172.21.15.5:0/3600821504 shutdown_connections 2024-09-19T06:27:09.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:09.468+0000 7f7f282e6640 1 -- 172.21.15.5:0/3600821504 wait complete. 2024-09-19T06:27:09.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 101928 8.57 MB/sec execute 513 sec latency 143.357 ms 2024-09-19T06:27:10.276 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:27:10.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:27:10.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:10 smithi005 ceph-mon[30955]: pgmap v754: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.1 MiB/s wr, 410 op/s 2024-09-19T06:27:10.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 102095 8.57 MB/sec execute 514 sec latency 76.621 ms 2024-09-19T06:27:10.669 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:27:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:10 smithi049 ceph-mon[34640]: pgmap v754: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.1 MiB/s wr, 410 op/s 2024-09-19T06:27:11.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:10 smithi157 ceph-mon[35346]: pgmap v754: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.1 MiB/s wr, 410 op/s 2024-09-19T06:27:11.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 102245 8.57 MB/sec execute 515 sec latency 94.553 ms 2024-09-19T06:27:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.637+0000 7fad64334640 1 -- 172.21.15.5:0/3848079952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c100e20 msgr2=0x7fad5c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.637+0000 7fad64334640 1 --2- 172.21.15.5:0/3848079952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c100e20 0x7fad5c101200 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fad58009900 tx=0x7fad5802a480 comp rx=0 tx=0).stop 2024-09-19T06:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.637+0000 7fad64334640 1 -- 172.21.15.5:0/3848079952 shutdown_connections 2024-09-19T06:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.637+0000 7fad64334640 1 --2- 172.21.15.5:0/3848079952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad5c102ef0 0x7fad5c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.637+0000 7fad64334640 1 --2- 172.21.15.5:0/3848079952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad5c102570 0x7fad5c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.637+0000 7fad64334640 1 --2- 172.21.15.5:0/3848079952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c100e20 0x7fad5c101200 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.637+0000 7fad64334640 1 -- 172.21.15.5:0/3848079952 >> 172.21.15.5:0/3848079952 conn(0x7fad5c0fc7d0 msgr2=0x7fad5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.637+0000 7fad64334640 1 -- 172.21.15.5:0/3848079952 shutdown_connections 2024-09-19T06:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.637+0000 7fad64334640 1 -- 172.21.15.5:0/3848079952 wait complete. 2024-09-19T06:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.638+0000 7fad64334640 1 Processor -- start 2024-09-19T06:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.638+0000 7fad64334640 1 -- start start 2024-09-19T06:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad64334640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad5c100e20 0x7fad5c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad64334640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102570 0x7fad5c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad64334640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad5c102ef0 0x7fad5c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad64334640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c110e30 con 0x7fad5c102570 2024-09-19T06:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad64334640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c110cb0 con 0x7fad5c100e20 2024-09-19T06:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad64334640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c110fb0 con 0x7fad5c102ef0 2024-09-19T06:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad618cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102570 0x7fad5c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad618cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102570 0x7fad5c198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37582/0 (socket says 172.21.15.5:37582) 2024-09-19T06:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad618cf640 1 -- 172.21.15.5:0/874358220 learned_addr learned my addr 172.21.15.5:0/874358220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad628d1640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad5c102ef0 0x7fad5c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad620d0640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad5c100e20 0x7fad5c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad618cf640 1 -- 172.21.15.5:0/874358220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad5c102ef0 msgr2=0x7fad5c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad618cf640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad5c102ef0 0x7fad5c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad618cf640 1 -- 172.21.15.5:0/874358220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad5c100e20 msgr2=0x7fad5c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad618cf640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad5c100e20 0x7fad5c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.639+0000 7fad618cf640 1 -- 172.21.15.5:0/874358220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c19d400 con 0x7fad5c102570 2024-09-19T06:27:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.640+0000 7fad618cf640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102570 0x7fad5c198990 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00e9f0 tx=0x7fad4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.640+0000 7fad537fe640 1 -- 172.21.15.5:0/874358220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c00cd80 con 0x7fad5c102570 2024-09-19T06:27:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.640+0000 7fad537fe640 1 -- 172.21.15.5:0/874358220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad4c0108e0 con 0x7fad5c102570 2024-09-19T06:27:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.640+0000 7fad64334640 1 -- 172.21.15.5:0/874358220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5c19d6f0 con 0x7fad5c102570 2024-09-19T06:27:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.640+0000 7fad537fe640 1 -- 172.21.15.5:0/874358220 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c018920 con 0x7fad5c102570 2024-09-19T06:27:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.640+0000 7fad64334640 1 -- 172.21.15.5:0/874358220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5c1a4fd0 con 0x7fad5c102570 2024-09-19T06:27:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.641+0000 7fad64334640 1 -- 172.21.15.5:0/874358220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad5c101200 con 0x7fad5c102570 2024-09-19T06:27:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.643+0000 7fad537fe640 1 -- 172.21.15.5:0/874358220 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fad4c010a90 con 0x7fad5c102570 2024-09-19T06:27:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.644+0000 7fad537fe640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fad480a33b0 0x7fad480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.645+0000 7fad620d0640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fad480a33b0 0x7fad480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.645+0000 7fad537fe640 1 -- 172.21.15.5:0/874358220 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fad4c0d00c0 con 0x7fad5c102570 2024-09-19T06:27:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.645+0000 7fad620d0640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fad480a33b0 0x7fad480a5870 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fad58009230 tx=0x7fad580058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:11.650+0000 7fad537fe640 1 -- 172.21.15.5:0/874358220 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fad4c09ca80 con 0x7fad5c102570 2024-09-19T06:27:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.024+0000 7fad64334640 1 -- 172.21.15.5:0/874358220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad5c1a51a0 con 0x7fad5c102570 2024-09-19T06:27:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.025+0000 7fad537fe640 1 -- 172.21.15.5:0/874358220 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+11959 (secure 0 0 0) 0x7fad4c09c380 con 0x7fad5c102570 2024-09-19T06:27:12.027 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:27:12.027 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":62,"btime":"2024-09-19T06:25:41:566205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":62,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:25:40.570592+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:27:12.027 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 62 2024-09-19T06:27:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.028+0000 7fad64334640 1 -- 172.21.15.5:0/874358220 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fad480a33b0 msgr2=0x7fad480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.028+0000 7fad64334640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fad480a33b0 0x7fad480a5870 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fad58009230 tx=0x7fad580058d0 comp rx=0 tx=0).stop 2024-09-19T06:27:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.028+0000 7fad64334640 1 -- 172.21.15.5:0/874358220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102570 msgr2=0x7fad5c198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.028+0000 7fad64334640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102570 0x7fad5c198990 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00e9f0 tx=0x7fad4c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:27:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.028+0000 7fad64334640 1 -- 172.21.15.5:0/874358220 shutdown_connections 2024-09-19T06:27:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.028+0000 7fad64334640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fad480a33b0 0x7fad480a5870 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.028+0000 7fad64334640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad5c102ef0 0x7fad5c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.028+0000 7fad64334640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102570 0x7fad5c198990 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.028+0000 7fad64334640 1 --2- 172.21.15.5:0/874358220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad5c100e20 0x7fad5c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.028+0000 7fad64334640 1 -- 172.21.15.5:0/874358220 >> 172.21.15.5:0/874358220 conn(0x7fad5c0fc7d0 msgr2=0x7fad5c105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.028+0000 7fad64334640 1 -- 172.21.15.5:0/874358220 shutdown_connections 2024-09-19T06:27:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:12.028+0000 7fad64334640 1 -- 172.21.15.5:0/874358220 wait complete. 2024-09-19T06:27:12.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 102392 8.56 MB/sec execute 516 sec latency 66.773 ms 2024-09-19T06:27:12.805 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:12 smithi005 ceph-mon[30955]: pgmap v755: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 214 op/s 2024-09-19T06:27:12.805 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:12 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/874358220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:27:12.846 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:27:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:12 smithi049 ceph-mon[34640]: pgmap v755: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 214 op/s 2024-09-19T06:27:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:12 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/874358220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:27:13.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:12 smithi157 ceph-mon[35346]: pgmap v755: 97 pgs: 97 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 214 op/s 2024-09-19T06:27:13.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:12 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/874358220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:27:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:27:13.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 102505 8.56 MB/sec execute 517 sec latency 168.477 ms 2024-09-19T06:27:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.894+0000 7f54e40ea640 1 -- 172.21.15.5:0/2625832447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54dc102570 msgr2=0x7f54dc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.894+0000 7f54e40ea640 1 --2- 172.21.15.5:0/2625832447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54dc102570 0x7f54dc1029b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f54cc009c10 tx=0x7f54cc02a770 comp rx=0 tx=0).stop 2024-09-19T06:27:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.894+0000 7f54e40ea640 1 -- 172.21.15.5:0/2625832447 shutdown_connections 2024-09-19T06:27:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.894+0000 7f54e40ea640 1 --2- 172.21.15.5:0/2625832447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54dc102ef0 0x7f54dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.894+0000 7f54e40ea640 1 --2- 172.21.15.5:0/2625832447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54dc102570 0x7f54dc1029b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.894+0000 7f54e40ea640 1 --2- 172.21.15.5:0/2625832447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54dc100e20 0x7f54dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.894+0000 7f54e40ea640 1 -- 172.21.15.5:0/2625832447 >> 172.21.15.5:0/2625832447 conn(0x7f54dc0fc7d0 msgr2=0x7f54dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.895+0000 7f54e40ea640 1 -- 172.21.15.5:0/2625832447 shutdown_connections 2024-09-19T06:27:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.895+0000 7f54e40ea640 1 -- 172.21.15.5:0/2625832447 wait complete. 2024-09-19T06:27:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.895+0000 7f54e40ea640 1 Processor -- start 2024-09-19T06:27:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.896+0000 7f54e40ea640 1 -- start start 2024-09-19T06:27:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.896+0000 7f54e40ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54dc100e20 0x7f54dc10d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.896+0000 7f54e40ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54dc102570 0x7f54dc10aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.896+0000 7f54e40ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54dc102ef0 0x7f54dc10afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.896+0000 7f54e40ea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc114100 con 0x7f54dc100e20 2024-09-19T06:27:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.896+0000 7f54e40ea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc113f80 con 0x7f54dc102570 2024-09-19T06:27:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.896+0000 7f54e40ea640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc114280 con 0x7f54dc102ef0 2024-09-19T06:27:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.896+0000 7f54e1e86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54dc100e20 0x7f54dc10d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.896+0000 7f54e1685640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54dc102570 0x7f54dc10aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.896+0000 7f54e1e86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54dc100e20 0x7f54dc10d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37606/0 (socket says 172.21.15.5:37606) 2024-09-19T06:27:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.896+0000 7f54e1e86640 1 -- 172.21.15.5:0/1790891768 learned_addr learned my addr 172.21.15.5:0/1790891768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.896+0000 7f54e2687640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54dc102ef0 0x7f54dc10afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.897+0000 7f54e1e86640 1 -- 172.21.15.5:0/1790891768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54dc102ef0 msgr2=0x7f54dc10afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.897+0000 7f54e1e86640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54dc102ef0 0x7f54dc10afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.897+0000 7f54e1e86640 1 -- 172.21.15.5:0/1790891768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54dc102570 msgr2=0x7f54dc10aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.897+0000 7f54e1e86640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54dc102570 0x7f54dc10aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.897+0000 7f54e1e86640 1 -- 172.21.15.5:0/1790891768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54dc1aae40 con 0x7f54dc100e20 2024-09-19T06:27:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.897+0000 7f54e1e86640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54dc100e20 0x7f54dc10d960 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f54d800e9c0 tx=0x7f54d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.897+0000 7f54e1685640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54dc102570 0x7f54dc10aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:27:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.897+0000 7f54d2ffd640 1 -- 172.21.15.5:0/1790891768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d800cde0 con 0x7f54dc100e20 2024-09-19T06:27:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.897+0000 7f54e40ea640 1 -- 172.21.15.5:0/1790891768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54dc1ab010 con 0x7f54dc100e20 2024-09-19T06:27:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.897+0000 7f54e40ea640 1 -- 172.21.15.5:0/1790891768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54dc1ab540 con 0x7f54dc100e20 2024-09-19T06:27:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.898+0000 7f54d2ffd640 1 -- 172.21.15.5:0/1790891768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54d80035f0 con 0x7f54dc100e20 2024-09-19T06:27:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.898+0000 7f54d2ffd640 1 -- 172.21.15.5:0/1790891768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8010690 con 0x7f54dc100e20 2024-09-19T06:27:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.901+0000 7f54e64c3740 1 Processor -- start 2024-09-19T06:27:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.901+0000 7f54e64c3740 1 -- start start 2024-09-19T06:27:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54e64c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc8273090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54e64c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc8273a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54e64c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc827e370 0x55edc8280760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54e64c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55edc81d92f0 con 0x55edc82735d0 2024-09-19T06:27:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54e64c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55edc8180cf0 con 0x55edc827e370 2024-09-19T06:27:13.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54e64c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55edc81d52b0 con 0x55edc8274b40 2024-09-19T06:27:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54e1685640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc827e370 0x55edc8280760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54e2687640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc8273090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54e2687640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc8273090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50084/0 (socket says 172.21.15.5:50084) 2024-09-19T06:27:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54e2687640 1 -- 172.21.15.5:0/848274778 learned_addr learned my addr 172.21.15.5:0/848274778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54e1685640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc827e370 0x55edc8280760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58602/0 (socket says 172.21.15.5:58602) 2024-09-19T06:27:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54d2ffd640 1 -- 172.21.15.5:0/1790891768 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f54d8004020 con 0x7f54dc100e20 2024-09-19T06:27:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.902+0000 7f54e1e86640 1 --2- 172.21.15.5:0/848274778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc8273a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.903+0000 7f54e2687640 1 -- 172.21.15.5:0/848274778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc827e370 msgr2=0x55edc8280760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.903+0000 7f54e2687640 1 --2- 172.21.15.5:0/848274778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc827e370 0x55edc8280760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.903+0000 7f54e2687640 1 -- 172.21.15.5:0/848274778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 msgr2=0x55edc8273a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.903+0000 7f54e2687640 1 --2- 172.21.15.5:0/848274778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc8273a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.903+0000 7f54e2687640 1 -- 172.21.15.5:0/848274778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edc81653a0 con 0x55edc8274b40 2024-09-19T06:27:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.903+0000 7f54e1e86640 1 --2- 172.21.15.5:0/848274778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc8273a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:27:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.903+0000 7f54e2687640 1 --2- 172.21.15.5:0/848274778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc8273090 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009ca0 tx=0x7f54d4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2a4f4832cd10f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.903+0000 7f54c3fff640 1 -- 172.21.15.5:0/848274778 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d401e070 con 0x55edc8274b40 2024-09-19T06:27:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.903+0000 7f54c3fff640 1 -- 172.21.15.5:0/848274778 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54d4009e50 con 0x55edc8274b40 2024-09-19T06:27:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.903+0000 7f54c3fff640 1 -- 172.21.15.5:0/848274778 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d400ea10 con 0x55edc8274b40 2024-09-19T06:27:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.904+0000 7f54d2ffd640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f54c40a3430 0x7f54c40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.904+0000 7f54d2ffd640 1 -- 172.21.15.5:0/1790891768 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f54d80cffb0 con 0x7f54dc100e20 2024-09-19T06:27:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.904+0000 7f54e1e86640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f54c40a3430 0x7f54c40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.908+0000 7f54e64c3740 1 -- 172.21.15.5:0/848274778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 msgr2=0x55edc8273090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.908+0000 7f54e64c3740 1 --2- 172.21.15.5:0/848274778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc8273090 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009ca0 tx=0x7f54d4011810 comp rx=0 tx=0).stop 2024-09-19T06:27:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.908+0000 7f54e64c3740 1 -- 172.21.15.5:0/848274778 shutdown_connections 2024-09-19T06:27:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.908+0000 7f54e64c3740 1 --2- 172.21.15.5:0/848274778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc827e370 0x55edc8280760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.908+0000 7f54e64c3740 1 --2- 172.21.15.5:0/848274778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc8273a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.908+0000 7f54e64c3740 1 --2- 172.21.15.5:0/848274778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc8273090 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.908+0000 7f54e64c3740 1 -- 172.21.15.5:0/848274778 >> 172.21.15.5:0/848274778 conn(0x55edc823c310 msgr2=0x55edc826fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.908+0000 7f54e1e86640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f54c40a3430 0x7f54c40a58f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f54d80cd790 tx=0x7f54d8020000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.908+0000 7f54e64c3740 1 -- 172.21.15.5:0/848274778 shutdown_connections 2024-09-19T06:27:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.909+0000 7f54e64c3740 1 -- 172.21.15.5:0/848274778 wait complete. 2024-09-19T06:27:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.909+0000 7f54e64c3740 1 -- start start 2024-09-19T06:27:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.909+0000 7f54e64c3740 1 Processor -- start 2024-09-19T06:27:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.910+0000 7f54e64c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc827dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.910+0000 7f54e64c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc827afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.910+0000 7f54e64c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc827e370 0x55edc827b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.910+0000 7f54e64c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55edc8281070 con 0x55edc82735d0 2024-09-19T06:27:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.910+0000 7f54e64c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55edc8194060 con 0x55edc827e370 2024-09-19T06:27:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.910+0000 7f54e64c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55edc82811f0 con 0x55edc8274b40 2024-09-19T06:27:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.910+0000 7f54e64c3740 10 client.0 _ll_register_callbacks cb 0x55edc7ee78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.910+0000 7f54e2687640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc827dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.910+0000 7f54e2687640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc827dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37616/0 (socket says 172.21.15.5:37616) 2024-09-19T06:27:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.910+0000 7f54e2687640 1 -- 172.21.15.5:0/922128713 learned_addr learned my addr 172.21.15.5:0/922128713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.911+0000 7f54e2687640 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc827e370 0x55edc827b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.911+0000 7f54e1685640 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc827afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.911+0000 7f54e2687640 1 -- 172.21.15.5:0/922128713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 msgr2=0x55edc827afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.911+0000 7f54e2687640 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc827afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.911+0000 7f54e2687640 1 -- 172.21.15.5:0/922128713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 msgr2=0x55edc827dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.911+0000 7f54e2687640 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc827dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.911+0000 7f54e2687640 1 -- 172.21.15.5:0/922128713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edc7f4a3c0 con 0x55edc827e370 2024-09-19T06:27:13.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.912+0000 7f54e2687640 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc827e370 0x55edc827b5d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f54d40057c0 tx=0x7f54d4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6198e6bb7680a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:13.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.912+0000 7f54c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:27:13.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.912+0000 7f54c17fa640 1 -- 172.21.15.5:0/922128713 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d401e070 con 0x55edc827e370 2024-09-19T06:27:13.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.912+0000 7f54e64c3740 1 -- 172.21.15.5:0/922128713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d4009e50 con 0x55edc827e370 2024-09-19T06:27:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.912+0000 7f54c17fa640 1 -- 172.21.15.5:0/922128713 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f54d4005030 con 0x55edc827e370 2024-09-19T06:27:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.912+0000 7f54c17fa640 1 -- 172.21.15.5:0/922128713 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f54d4005230 con 0x55edc827e370 2024-09-19T06:27:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.912+0000 7f54e64c3740 10 client.30632 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.912+0000 7f54e64c3740 1 -- 172.21.15.5:0/922128713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55edc828ca70 con 0x55edc827e370 2024-09-19T06:27:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.913+0000 7f54c17fa640 1 -- 172.21.15.5:0/922128713 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54d4009e50 con 0x55edc827e370 2024-09-19T06:27:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.913+0000 7f54c17fa640 1 -- 172.21.15.5:0/922128713 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f54d4025070 con 0x55edc827e370 2024-09-19T06:27:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.913+0000 7f54e64c3740 10 client.30632 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.913+0000 7f54e64c3740 10 client.30632 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:27:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.913+0000 7f54e64c3740 20 client.30632 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.913+0000 7f54e64c3740 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55edc828dee0 0x55edc82902d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.913+0000 7f54e64c3740 4 client.30632 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.913+0000 7f54e64c3740 1 -- 172.21.15.5:0/922128713 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edc81e6860 con 0x55edc828dee0 2024-09-19T06:27:13.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.913+0000 7f54e1685640 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55edc828dee0 0x55edc82902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.915+0000 7f54c17fa640 10 client.30632 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:27:13.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.915+0000 7f54e1685640 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55edc828dee0 0x55edc82902d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da3589ba78f7afba server_cookie=22509d082ba922cf in_seq=0 out_seq=0 2024-09-19T06:27:13.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.915+0000 7f54c17fa640 1 -- 172.21.15.5:0/922128713 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edc81e6860 con 0x55edc828dee0 2024-09-19T06:27:13.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.915+0000 7f54c17fa640 10 client.30632 handle_command_reply: tid=0 2024-09-19T06:27:13.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.916+0000 7f54e64c3740 1 client.30632 shutdown 2024-09-19T06:27:13.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.916+0000 7f54e64c3740 1 -- 172.21.15.5:0/922128713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc827e370 msgr2=0x55edc827b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:13.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.916+0000 7f54e64c3740 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc827e370 0x55edc827b5d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f54d40057c0 tx=0x7f54d4005f90 comp rx=0 tx=0).stop 2024-09-19T06:27:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.916+0000 7f54e64c3740 1 -- 172.21.15.5:0/922128713 shutdown_connections 2024-09-19T06:27:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.916+0000 7f54e64c3740 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55edc828dee0 0x55edc82902d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.916+0000 7f54e64c3740 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc827e370 0x55edc827b5d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.916+0000 7f54e64c3740 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc827afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.916+0000 7f54e64c3740 1 --2- 172.21.15.5:0/922128713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc827dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.916+0000 7f54e64c3740 1 -- 172.21.15.5:0/922128713 >> 172.21.15.5:0/922128713 conn(0x55edc823c310 msgr2=0x55edc8275930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.916+0000 7f54c17fa640 0 client.30632 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:27:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.917+0000 7f54e64c3740 1 -- 172.21.15.5:0/922128713 shutdown_connections 2024-09-19T06:27:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.917+0000 7f54e64c3740 1 -- 172.21.15.5:0/922128713 wait complete. 2024-09-19T06:27:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.917+0000 7f54e64c3740 20 client.30632 trim_cache size 0 max 16384 2024-09-19T06:27:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.943+0000 7f54e64c3740 1 Processor -- start 2024-09-19T06:27:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.943+0000 7f54e64c3740 1 -- start start 2024-09-19T06:27:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e64c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc82735d0 0x55edc828a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e64c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc8295bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e64c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc827e370 0x55edc8296100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e64c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54c800e7e0 con 0x55edc827e370 2024-09-19T06:27:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e64c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f54c8010320 con 0x55edc82735d0 2024-09-19T06:27:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e64c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54c800eeb0 con 0x55edc8274b40 2024-09-19T06:27:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e2687640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc82735d0 0x55edc828a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e1685640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc8295bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e2687640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc82735d0 0x55edc828a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58622/0 (socket says 172.21.15.5:58622) 2024-09-19T06:27:13.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e1685640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc8295bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50106/0 (socket says 172.21.15.5:50106) 2024-09-19T06:27:13.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e2687640 1 -- 172.21.15.5:0/1579695641 learned_addr learned my addr 172.21.15.5:0/1579695641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:13.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e2687640 1 --2- 172.21.15.5:0/1579695641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc827e370 0x55edc8296100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e1685640 1 -- 172.21.15.5:0/1579695641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc82735d0 msgr2=0x55edc828a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:13.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e1685640 1 --2- 172.21.15.5:0/1579695641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc82735d0 0x55edc828a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e1685640 1 -- 172.21.15.5:0/1579695641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc827e370 msgr2=0x55edc8296100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:13.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e1685640 1 --2- 172.21.15.5:0/1579695641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc827e370 0x55edc8296100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.944+0000 7f54e1685640 1 -- 172.21.15.5:0/1579695641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edc8270960 con 0x55edc8274b40 2024-09-19T06:27:13.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.945+0000 7f54e1685640 1 --2- 172.21.15.5:0/1579695641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc8295bc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f54cc004af0 tx=0x7f54cc005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d65163759978468 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:13.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.945+0000 7f54c37fe640 1 -- 172.21.15.5:0/1579695641 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc036040 con 0x55edc8274b40 2024-09-19T06:27:13.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.945+0000 7f54c37fe640 1 -- 172.21.15.5:0/1579695641 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54cc049020 con 0x55edc8274b40 2024-09-19T06:27:13.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.945+0000 7f54c37fe640 1 -- 172.21.15.5:0/1579695641 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc030df0 con 0x55edc8274b40 2024-09-19T06:27:13.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.946+0000 7f54e64c3740 1 -- 172.21.15.5:0/1579695641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 msgr2=0x55edc8295bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:13.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.946+0000 7f54e64c3740 1 --2- 172.21.15.5:0/1579695641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc8295bc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f54cc004af0 tx=0x7f54cc005fd0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.946+0000 7f54e64c3740 1 -- 172.21.15.5:0/1579695641 shutdown_connections 2024-09-19T06:27:13.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.946+0000 7f54e64c3740 1 --2- 172.21.15.5:0/1579695641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc827e370 0x55edc8296100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.946+0000 7f54e64c3740 1 --2- 172.21.15.5:0/1579695641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc8274b40 0x55edc8295bc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.946+0000 7f54e64c3740 1 --2- 172.21.15.5:0/1579695641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc82735d0 0x55edc828a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.946+0000 7f54e64c3740 1 -- 172.21.15.5:0/1579695641 >> 172.21.15.5:0/1579695641 conn(0x55edc823c310 msgr2=0x55edc82798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:13.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.946+0000 7f54e64c3740 1 -- 172.21.15.5:0/1579695641 shutdown_connections 2024-09-19T06:27:13.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.946+0000 7f54e64c3740 1 -- 172.21.15.5:0/1579695641 wait complete. 2024-09-19T06:27:13.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.947+0000 7f54e64c3740 1 -- start start 2024-09-19T06:27:13.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.947+0000 7f54e64c3740 1 Processor -- start 2024-09-19T06:27:13.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.948+0000 7f54e64c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc8286c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.948+0000 7f54e64c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc8274b40 0x55edc8281f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.948+0000 7f54e2687640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc8286c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.948+0000 7f54e2687640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc8286c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37636/0 (socket says 172.21.15.5:37636) 2024-09-19T06:27:13.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.948+0000 7f54e64c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc827e370 0x55edc82824b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.948+0000 7f54e64c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54c800fc50 con 0x55edc82735d0 2024-09-19T06:27:13.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.948+0000 7f54e64c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f54c800f580 con 0x55edc8274b40 2024-09-19T06:27:13.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.948+0000 7f54e64c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54c800c7e0 con 0x55edc827e370 2024-09-19T06:27:13.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.948+0000 7f54e64c3740 10 client.0 _ll_register_callbacks cb 0x7f54cc034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:13.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.949+0000 7f54e2687640 1 -- 172.21.15.5:0/1070858379 learned_addr learned my addr 172.21.15.5:0/1070858379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.949+0000 7f54e1685640 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc8274b40 0x55edc8281f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.949+0000 7f54e2687640 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc827e370 0x55edc82824b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.949+0000 7f54e1685640 1 -- 172.21.15.5:0/1070858379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc827e370 msgr2=0x55edc82824b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.949+0000 7f54e1685640 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc827e370 0x55edc82824b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.949+0000 7f54e1685640 1 -- 172.21.15.5:0/1070858379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 msgr2=0x55edc8286c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.949+0000 7f54e1685640 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc8286c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.949+0000 7f54e1685640 1 -- 172.21.15.5:0/1070858379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edc8282cb0 con 0x55edc8274b40 2024-09-19T06:27:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.949+0000 7f54e1685640 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc8274b40 0x55edc8281f70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f54cc02ad30 tx=0x7f54cc030b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c929044e23128760 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.950+0000 7f54c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:27:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.950+0000 7f54c17fa640 1 -- 172.21.15.5:0/1070858379 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc036040 con 0x55edc8274b40 2024-09-19T06:27:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.950+0000 7f54e64c3740 1 -- 172.21.15.5:0/1070858379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54cc049020 con 0x55edc8274b40 2024-09-19T06:27:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.950+0000 7f54c17fa640 1 -- 172.21.15.5:0/1070858379 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f54cc030e00 con 0x55edc8274b40 2024-09-19T06:27:13.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.950+0000 7f54c17fa640 1 -- 172.21.15.5:0/1070858379 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f54cc003970 con 0x55edc8274b40 2024-09-19T06:27:13.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.950+0000 7f54e64c3740 10 client.30638 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:13.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.950+0000 7f54e64c3740 1 -- 172.21.15.5:0/1070858379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55edc82f8bc0 con 0x55edc8274b40 2024-09-19T06:27:13.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.950+0000 7f54c17fa640 1 -- 172.21.15.5:0/1070858379 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54cc037040 con 0x55edc8274b40 2024-09-19T06:27:13.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.951+0000 7f54c17fa640 1 -- 172.21.15.5:0/1070858379 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f54cc03c040 con 0x55edc8274b40 2024-09-19T06:27:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.951+0000 7f54e64c3740 10 client.30638 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.951+0000 7f54e64c3740 10 client.30638 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:27:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.951+0000 7f54e64c3740 20 client.30638 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.951+0000 7f54e64c3740 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55edc82f92e0 0x55edc82fb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.951+0000 7f54e64c3740 4 client.30638 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:27:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.951+0000 7f54e64c3740 1 -- 172.21.15.5:0/1070858379 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f54cc049020 con 0x55edc82f92e0 2024-09-19T06:27:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.952+0000 7f54e2687640 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55edc82f92e0 0x55edc82fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.952+0000 7f54e2687640 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55edc82f92e0 0x55edc82fb700 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d84db262c8c1a7fb server_cookie=fb33dbf726910fda in_seq=0 out_seq=0 2024-09-19T06:27:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.952+0000 7f54c17fa640 10 client.30638 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:27:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.953+0000 7f54c17fa640 1 -- 172.21.15.5:0/1070858379 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f54cc049020 con 0x55edc82f92e0 2024-09-19T06:27:13.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.953+0000 7f54c17fa640 10 client.30638 handle_command_reply: tid=0 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.953+0000 7f54e64c3740 1 client.30638 shutdown 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.953+0000 7f54e64c3740 1 -- 172.21.15.5:0/1070858379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc8274b40 msgr2=0x55edc8281f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.953+0000 7f54e64c3740 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc8274b40 0x55edc8281f70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f54cc02ad30 tx=0x7f54cc030b50 comp rx=0 tx=0).stop 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.953+0000 7f54e64c3740 1 -- 172.21.15.5:0/1070858379 shutdown_connections 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.953+0000 7f54e64c3740 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55edc82f92e0 0x55edc82fb700 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.953+0000 7f54e64c3740 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edc827e370 0x55edc82824b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.953+0000 7f54e64c3740 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55edc8274b40 0x55edc8281f70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.953+0000 7f54e64c3740 1 --2- 172.21.15.5:0/1070858379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edc82735d0 0x55edc8286c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.953+0000 7f54e64c3740 1 -- 172.21.15.5:0/1070858379 >> 172.21.15.5:0/1070858379 conn(0x55edc823c310 msgr2=0x55edc827c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.953+0000 7f54c17fa640 0 client.30638 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.954+0000 7f54e64c3740 1 -- 172.21.15.5:0/1070858379 shutdown_connections 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.954+0000 7f54e64c3740 1 -- 172.21.15.5:0/1070858379 wait complete. 2024-09-19T06:27:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.954+0000 7f54e64c3740 20 client.30638 trim_cache size 0 max 16384 2024-09-19T06:27:13.958 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:27:13.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.955+0000 7f54e40ea640 1 -- 172.21.15.5:0/1790891768 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f54c40a3430 msgr2=0x7f54c40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:13.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.955+0000 7f54e40ea640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f54c40a3430 0x7f54c40a58f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f54d80cd790 tx=0x7f54d8020000 comp rx=0 tx=0).stop 2024-09-19T06:27:13.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.955+0000 7f54e40ea640 1 -- 172.21.15.5:0/1790891768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54dc100e20 msgr2=0x7f54dc10d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:13.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.955+0000 7f54e40ea640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54dc100e20 0x7f54dc10d960 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f54d800e9c0 tx=0x7f54d800ee80 comp rx=0 tx=0).stop 2024-09-19T06:27:13.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.955+0000 7f54e40ea640 1 -- 172.21.15.5:0/1790891768 shutdown_connections 2024-09-19T06:27:13.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.955+0000 7f54e40ea640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f54c40a3430 0x7f54c40a58f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.955+0000 7f54e40ea640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54dc102ef0 0x7f54dc10afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.955+0000 7f54e40ea640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54dc102570 0x7f54dc10aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.955+0000 7f54e40ea640 1 --2- 172.21.15.5:0/1790891768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54dc100e20 0x7f54dc10d960 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:13.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.955+0000 7f54e40ea640 1 -- 172.21.15.5:0/1790891768 >> 172.21.15.5:0/1790891768 conn(0x7f54dc0fc7d0 msgr2=0x7f54dc104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:13.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.955+0000 7f54e40ea640 1 -- 172.21.15.5:0/1790891768 shutdown_connections 2024-09-19T06:27:13.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:13.955+0000 7f54e40ea640 1 -- 172.21.15.5:0/1790891768 wait complete. 2024-09-19T06:27:14.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 102631 8.55 MB/sec execute 518 sec latency 102.102 ms 2024-09-19T06:27:14.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:14 smithi005 ceph-mon[30955]: pgmap v756: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.8 MiB/s wr, 371 op/s 2024-09-19T06:27:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:14 smithi049 ceph-mon[34640]: pgmap v756: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.8 MiB/s wr, 371 op/s 2024-09-19T06:27:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:14 smithi157 ceph-mon[35346]: pgmap v756: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.8 MiB/s wr, 371 op/s 2024-09-19T06:27:15.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:27:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:27:15.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 102796 8.55 MB/sec execute 519 sec latency 276.928 ms 2024-09-19T06:27:16.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.148+0000 7f7eb9d9b640 1 -- 172.21.15.5:0/2871533996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac031830 con 0x7f7ebc100e20 2024-09-19T06:27:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.149+0000 7f7ec1018640 1 -- 172.21.15.5:0/2871533996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc100e20 msgr2=0x7f7ebc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.149+0000 7f7ec1018640 1 --2- 172.21.15.5:0/2871533996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc100e20 0x7f7ebc101200 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009900 tx=0x7f7eac02a480 comp rx=0 tx=0).stop 2024-09-19T06:27:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.149+0000 7f7ec1018640 1 -- 172.21.15.5:0/2871533996 shutdown_connections 2024-09-19T06:27:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.149+0000 7f7ec1018640 1 --2- 172.21.15.5:0/2871533996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ebc102ef0 0x7f7ebc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.149+0000 7f7ec1018640 1 --2- 172.21.15.5:0/2871533996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ebc102570 0x7f7ebc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.149+0000 7f7ec1018640 1 --2- 172.21.15.5:0/2871533996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc100e20 0x7f7ebc101200 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.149+0000 7f7ec1018640 1 -- 172.21.15.5:0/2871533996 >> 172.21.15.5:0/2871533996 conn(0x7f7ebc0fc7d0 msgr2=0x7f7ebc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.149+0000 7f7ec1018640 1 -- 172.21.15.5:0/2871533996 shutdown_connections 2024-09-19T06:27:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.149+0000 7f7ec1018640 1 -- 172.21.15.5:0/2871533996 wait complete. 2024-09-19T06:27:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.150+0000 7f7ec1018640 1 Processor -- start 2024-09-19T06:27:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.151+0000 7f7ec1018640 1 -- start start 2024-09-19T06:27:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.151+0000 7f7ec1018640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc100e20 0x7f7ebc19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.151+0000 7f7ec1018640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ebc102570 0x7f7ebc1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.151+0000 7f7ec1018640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ebc102ef0 0x7f7ebc196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.151+0000 7f7ec1018640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc110e80 con 0x7f7ebc100e20 2024-09-19T06:27:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.151+0000 7f7ec1018640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc110d00 con 0x7f7ebc102ef0 2024-09-19T06:27:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.151+0000 7f7ec1018640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc111000 con 0x7f7ebc102570 2024-09-19T06:27:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.151+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ebc102570 0x7f7ebc1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.151+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ebc102570 0x7f7ebc1962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56244/0 (socket says 172.21.15.5:56244) 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.151+0000 7f7eba59c640 1 -- 172.21.15.5:0/2261253111 learned_addr learned my addr 172.21.15.5:0/2261253111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.152+0000 7f7ebad9d640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc100e20 0x7f7ebc19e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.152+0000 7f7ebb59e640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ebc102ef0 0x7f7ebc196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.152+0000 7f7ebad9d640 1 -- 172.21.15.5:0/2261253111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ebc102570 msgr2=0x7f7ebc1962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.152+0000 7f7ebad9d640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ebc102570 0x7f7ebc1962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.152+0000 7f7ebad9d640 1 -- 172.21.15.5:0/2261253111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ebc102ef0 msgr2=0x7f7ebc196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.152+0000 7f7ebad9d640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ebc102ef0 0x7f7ebc196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.152+0000 7f7ebad9d640 1 -- 172.21.15.5:0/2261253111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc196d40 con 0x7f7ebc100e20 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.153+0000 7f7ebad9d640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc100e20 0x7f7ebc19e0e0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f7eac006010 tx=0x7f7eac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.153+0000 7f7ea3fff640 1 -- 172.21.15.5:0/2261253111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac03f030 con 0x7f7ebc100e20 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.154+0000 7f7ec1018640 1 -- 172.21.15.5:0/2261253111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc196fd0 con 0x7f7ebc100e20 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.154+0000 7f7ec1018640 1 -- 172.21.15.5:0/2261253111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc1b3d30 con 0x7f7ebc100e20 2024-09-19T06:27:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.157+0000 7f7ec33f1740 1 Processor -- start 2024-09-19T06:27:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.158+0000 7f7ea3fff640 1 -- 172.21.15.5:0/2261253111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eac004440 con 0x7f7ebc100e20 2024-09-19T06:27:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.158+0000 7f7ea3fff640 1 -- 172.21.15.5:0/2261253111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac035070 con 0x7f7ebc100e20 2024-09-19T06:27:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.158+0000 7f7ea3fff640 1 -- 172.21.15.5:0/2261253111 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7eac042030 con 0x7f7ebc100e20 2024-09-19T06:27:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.159+0000 7f7ea3fff640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7ea40a34e0 0x7f7ea40a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.159+0000 7f7ea3fff640 1 -- 172.21.15.5:0/2261253111 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f7eac0f0b30 con 0x7f7ebc100e20 2024-09-19T06:27:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.160+0000 7f7ec33f1740 1 -- start start 2024-09-19T06:27:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.160+0000 7f7ebb59e640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7ea40a34e0 0x7f7ea40a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.161+0000 7f7ec33f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e6b40 0x5603362e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.161+0000 7f7ec33f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e55d0 0x5603362e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.161+0000 7f7ec33f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x5603362f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.161+0000 7f7ec33f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56033624b2f0 con 0x5603362e6b40 2024-09-19T06:27:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.161+0000 7f7ec33f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5603361f2cf0 con 0x5603362e55d0 2024-09-19T06:27:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.161+0000 7f7ec33f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5603362472b0 con 0x5603362f0370 2024-09-19T06:27:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.161+0000 7f7ebad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e6b40 0x5603362e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.161+0000 7f7ebb59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x5603362f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.161+0000 7f7ebad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e6b40 0x5603362e5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44692/0 (socket says 172.21.15.5:44692) 2024-09-19T06:27:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.161+0000 7f7ebad9d640 1 -- 172.21.15.5:0/4173783968 learned_addr learned my addr 172.21.15.5:0/4173783968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.161+0000 7f7ebb59e640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7ea40a34e0 0x7f7ea40a59a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4005fd0 tx=0x7f7eb4009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.162+0000 7f7eba59c640 1 --2- 172.21.15.5:0/4173783968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e55d0 0x5603362e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.162+0000 7f7ebb59e640 1 -- 172.21.15.5:0/4173783968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e55d0 msgr2=0x5603362e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.162+0000 7f7ebb59e640 1 --2- 172.21.15.5:0/4173783968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e55d0 0x5603362e5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.162+0000 7f7ebb59e640 1 -- 172.21.15.5:0/4173783968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e6b40 msgr2=0x5603362e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.162+0000 7f7ebb59e640 1 --2- 172.21.15.5:0/4173783968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e6b40 0x5603362e5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.162+0000 7f7ebb59e640 1 -- 172.21.15.5:0/4173783968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603361d73a0 con 0x5603362f0370 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.162+0000 7f7eba59c640 1 --2- 172.21.15.5:0/4173783968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e55d0 0x5603362e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.162+0000 7f7ebad9d640 1 --2- 172.21.15.5:0/4173783968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e6b40 0x5603362e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.162+0000 7f7ebb59e640 1 --2- 172.21.15.5:0/4173783968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x5603362f2760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4011ca0 tx=0x7f7eb4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f87572aceba57e00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.162+0000 7f7ea17fa640 1 -- 172.21.15.5:0/4173783968 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4004b20 con 0x5603362f0370 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.163+0000 7f7ea17fa640 1 -- 172.21.15.5:0/4173783968 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eb4004cc0 con 0x5603362f0370 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.163+0000 7f7ea17fa640 1 -- 172.21.15.5:0/4173783968 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4004b20 con 0x5603362f0370 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.164+0000 7f7ec33f1740 1 -- 172.21.15.5:0/4173783968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 msgr2=0x5603362f2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.164+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/4173783968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x5603362f2760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4011ca0 tx=0x7f7eb4014000 comp rx=0 tx=0).stop 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.164+0000 7f7ec33f1740 1 -- 172.21.15.5:0/4173783968 shutdown_connections 2024-09-19T06:27:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.164+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/4173783968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x5603362f2760 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.164+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/4173783968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e55d0 0x5603362e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.164+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/4173783968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e6b40 0x5603362e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.164+0000 7f7ec33f1740 1 -- 172.21.15.5:0/4173783968 >> 172.21.15.5:0/4173783968 conn(0x5603362ae310 msgr2=0x5603362e1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.164+0000 7f7ec33f1740 1 -- 172.21.15.5:0/4173783968 shutdown_connections 2024-09-19T06:27:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.164+0000 7f7ec33f1740 1 -- 172.21.15.5:0/4173783968 wait complete. 2024-09-19T06:27:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.165+0000 7f7ec33f1740 1 -- start start 2024-09-19T06:27:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.165+0000 7f7ec33f1740 1 Processor -- start 2024-09-19T06:27:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.165+0000 7f7ec33f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 0x5603362efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.165+0000 7f7ebad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 0x5603362efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ebad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 0x5603362efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44706/0 (socket says 172.21.15.5:44706) 2024-09-19T06:27:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ebad9d640 1 -- 172.21.15.5:0/1387893528 learned_addr learned my addr 172.21.15.5:0/1387893528 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x5603362ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x5603362ed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1387893528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5603362f3070 con 0x5603362e55d0 2024-09-19T06:27:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1387893528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560336206060 con 0x5603362e6b40 2024-09-19T06:27:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1387893528 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5603362f31f0 con 0x5603362f0370 2024-09-19T06:27:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ec33f1740 10 client.0 _ll_register_callbacks cb 0x560335f598c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7eba59c640 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x5603362ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ebad9d640 1 -- 172.21.15.5:0/1387893528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 msgr2=0x5603362ed450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ebad9d640 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x5603362ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ebad9d640 1 -- 172.21.15.5:0/1387893528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 msgr2=0x5603362ecf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ebad9d640 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x5603362ecf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.166+0000 7f7ebad9d640 1 -- 172.21.15.5:0/1387893528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560335fbc3c0 con 0x5603362e55d0 2024-09-19T06:27:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.167+0000 7f7ebad9d640 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 0x5603362efc60 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0bce50 tx=0x7f7eac0c1280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c2a6d867a81ab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.167+0000 7f7e9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:27:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.167+0000 7f7e9eff5640 1 -- 172.21.15.5:0/1387893528 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac049040 con 0x5603362e55d0 2024-09-19T06:27:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.167+0000 7f7e9eff5640 1 -- 172.21.15.5:0/1387893528 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f7eac0bd320 con 0x5603362e55d0 2024-09-19T06:27:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.167+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1387893528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb4004cc0 con 0x5603362e55d0 2024-09-19T06:27:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.167+0000 7f7e9eff5640 1 -- 172.21.15.5:0/1387893528 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f7eac0c55f0 con 0x5603362e55d0 2024-09-19T06:27:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.167+0000 7f7ec33f1740 10 client.21282 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.167+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1387893528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x5603362fea20 con 0x5603362e55d0 2024-09-19T06:27:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.168+0000 7f7e9eff5640 1 -- 172.21.15.5:0/1387893528 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eac0c5790 con 0x5603362e55d0 2024-09-19T06:27:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.168+0000 7f7e9eff5640 1 -- 172.21.15.5:0/1387893528 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f7eac04c060 con 0x5603362e55d0 2024-09-19T06:27:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.168+0000 7f7ec33f1740 10 client.21282 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.168+0000 7f7ec33f1740 10 client.21282 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.168+0000 7f7ec33f1740 20 client.21282 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.168+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603362ffe40 0x5603363022a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.168+0000 7f7ec33f1740 4 client.21282 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.168+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1387893528 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7eb4004cc0 con 0x5603362ffe40 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.168+0000 7f7eba59c640 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603362ffe40 0x5603363022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.169+0000 7f7e9eff5640 10 client.21282 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.169+0000 7f7eba59c640 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603362ffe40 0x5603363022a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=75c4b35cb2a630ee server_cookie=1161b639067e2af5 in_seq=0 out_seq=0 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.170+0000 7f7e9eff5640 1 -- 172.21.15.5:0/1387893528 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f7eb4004cc0 con 0x5603362ffe40 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7e9eff5640 10 client.21282 handle_command_reply: tid=0 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7ec33f1740 1 client.21282 shutdown 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1387893528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 msgr2=0x5603362efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 0x5603362efc60 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0bce50 tx=0x7f7eac0c1280 comp rx=0 tx=0).stop 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1387893528 shutdown_connections 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603362ffe40 0x5603363022a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x5603362ed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7e9eff5640 0 client.21282 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x5603362ecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/1387893528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 0x5603362efc60 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1387893528 >> 172.21.15.5:0/1387893528 conn(0x5603362ae310 msgr2=0x5603362e7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1387893528 shutdown_connections 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1387893528 wait complete. 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.171+0000 7f7ec33f1740 20 client.21282 trim_cache size 0 max 16384 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.189+0000 7f7ec33f1740 1 Processor -- start 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.189+0000 7f7ec33f1740 1 -- start start 2024-09-19T06:27:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7ec33f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362e55d0 0x5603362fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7ec33f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x560336307bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7ec33f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362f0370 0x5603363080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x560336307bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7ebad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362e55d0 0x5603362fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x560336307bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48942/0 (socket says 172.21.15.5:48942) 2024-09-19T06:27:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7ebad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362e55d0 0x5603362fc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56264/0 (socket says 172.21.15.5:56264) 2024-09-19T06:27:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7eba59c640 1 -- 172.21.15.5:0/1901877511 learned_addr learned my addr 172.21.15.5:0/1901877511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1901877511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea800e8a0 con 0x5603362f0370 2024-09-19T06:27:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1901877511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea80103e0 con 0x5603362e6b40 2024-09-19T06:27:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1901877511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea800ef70 con 0x5603362e55d0 2024-09-19T06:27:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7ebb59e640 1 --2- 172.21.15.5:0/1901877511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362f0370 0x5603363080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7eba59c640 1 -- 172.21.15.5:0/1901877511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362e55d0 msgr2=0x5603362fc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7eba59c640 1 --2- 172.21.15.5:0/1901877511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362e55d0 0x5603362fc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7eba59c640 1 -- 172.21.15.5:0/1901877511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362f0370 msgr2=0x5603363080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7eba59c640 1 --2- 172.21.15.5:0/1901877511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362f0370 0x5603363080f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7eba59c640 1 -- 172.21.15.5:0/1901877511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603362e5d50 con 0x5603362e6b40 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.190+0000 7f7ebad9d640 1 --2- 172.21.15.5:0/1901877511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362e55d0 0x5603362fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.191+0000 7f7eba59c640 1 --2- 172.21.15.5:0/1901877511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x560336307bb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc197870 tx=0x7f7eb0009420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8271cb586ff520c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.191+0000 7f7ea0ff9640 1 -- 172.21.15.5:0/1901877511 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0023070 con 0x5603362e6b40 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.191+0000 7f7ea0ff9640 1 -- 172.21.15.5:0/1901877511 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eb001e470 con 0x5603362e6b40 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.191+0000 7f7ea0ff9640 1 -- 172.21.15.5:0/1901877511 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb00142d0 con 0x5603362e6b40 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.192+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1901877511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 msgr2=0x560336307bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.192+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/1901877511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x560336307bb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc197870 tx=0x7f7eb0009420 comp rx=0 tx=0).stop 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.192+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1901877511 shutdown_connections 2024-09-19T06:27:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.192+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/1901877511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362f0370 0x5603363080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.192+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/1901877511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x560336307bb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.192+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/1901877511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362e55d0 0x5603362fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.192+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1901877511 >> 172.21.15.5:0/1901877511 conn(0x5603362ae310 msgr2=0x5603362eced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.192+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1901877511 shutdown_connections 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.192+0000 7f7ec33f1740 1 -- 172.21.15.5:0/1901877511 wait complete. 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.192+0000 7f7ec33f1740 1 -- start start 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.193+0000 7f7ec33f1740 1 Processor -- start 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.193+0000 7f7ec33f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 0x5603362f4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.193+0000 7f7ec33f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x5603362f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.193+0000 7f7ebad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 0x5603362f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.193+0000 7f7ebad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 0x5603362f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44732/0 (socket says 172.21.15.5:44732) 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.193+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x5603362f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.193+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x5603362f4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48946/0 (socket says 172.21.15.5:48946) 2024-09-19T06:27:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.193+0000 7f7ec33f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x560336369b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.194+0000 7f7ec33f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea800fd10 con 0x5603362e55d0 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.194+0000 7f7ec33f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea800f640 con 0x5603362e6b40 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.194+0000 7f7ec33f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea800c8a0 con 0x5603362f0370 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.194+0000 7f7ec33f1740 10 client.0 _ll_register_callbacks cb 0x7f7eac0bcf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.194+0000 7f7ebb59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x560336369b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.194+0000 7f7ebb59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x560336369b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56272/0 (socket says 172.21.15.5:56272) 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.194+0000 7f7ebb59e640 1 -- 172.21.15.5:0/350006136 learned_addr learned my addr 172.21.15.5:0/350006136 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.194+0000 7f7ebb59e640 1 -- 172.21.15.5:0/350006136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 msgr2=0x5603362f4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.194+0000 7f7ebb59e640 1 --2- 172.21.15.5:0/350006136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x5603362f4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.194+0000 7f7ebb59e640 1 -- 172.21.15.5:0/350006136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 msgr2=0x5603362f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.194+0000 7f7ebb59e640 1 --2- 172.21.15.5:0/350006136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 0x5603362f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.194+0000 7f7ebb59e640 1 -- 172.21.15.5:0/350006136 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56033636a0c0 con 0x5603362f0370 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.195+0000 7f7ebb59e640 1 --2- 172.21.15.5:0/350006136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x560336369b80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x5603362e6070 tx=0x7f7eb4026b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96023df9025eba64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.195+0000 7f7e9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.195+0000 7f7e9eff5640 1 -- 172.21.15.5:0/350006136 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4036060 con 0x5603362f0370 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.195+0000 7f7e9eff5640 1 -- 172.21.15.5:0/350006136 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f7eb4026cc0 con 0x5603362f0370 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.195+0000 7f7ec33f1740 1 -- 172.21.15.5:0/350006136 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb001e470 con 0x5603362f0370 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.195+0000 7f7e9eff5640 1 -- 172.21.15.5:0/350006136 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f7eb4004b20 con 0x5603362f0370 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.195+0000 7f7ec33f1740 10 client.30592 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.195+0000 7f7ec33f1740 1 -- 172.21.15.5:0/350006136 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x56033636b240 con 0x5603362f0370 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.195+0000 7f7e9eff5640 1 -- 172.21.15.5:0/350006136 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eb40318d0 con 0x5603362f0370 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.196+0000 7f7e9eff5640 1 -- 172.21.15.5:0/350006136 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f7eb401e3f0 con 0x5603362f0370 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.196+0000 7f7ec33f1740 10 client.30592 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.196+0000 7f7ec33f1740 10 client.30592 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.196+0000 7f7ec33f1740 20 client.30592 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.196+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/350006136 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603362ffe40 0x56033636d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.196+0000 7f7ec33f1740 4 client.30592 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:27:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.196+0000 7f7ec33f1740 1 -- 172.21.15.5:0/350006136 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7eb001e470 con 0x5603362ffe40 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.196+0000 7f7ebad9d640 1 --2- 172.21.15.5:0/350006136 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603362ffe40 0x56033636d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.197+0000 7f7e9eff5640 10 client.30592 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.197+0000 7f7ebad9d640 1 --2- 172.21.15.5:0/350006136 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603362ffe40 0x56033636d740 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ba80c61fcec2589 server_cookie=c27ed28d9e6a9d65 in_seq=0 out_seq=0 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.197+0000 7f7e9eff5640 1 -- 172.21.15.5:0/350006136 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7eb001e470 con 0x5603362ffe40 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.197+0000 7f7e9eff5640 10 client.30592 handle_command_reply: tid=0 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.197+0000 7f7ec33f1740 1 client.30592 shutdown 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.197+0000 7f7ec33f1740 1 -- 172.21.15.5:0/350006136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 msgr2=0x560336369b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.197+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/350006136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x560336369b80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x5603362e6070 tx=0x7f7eb4026b50 comp rx=0 tx=0).stop 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.197+0000 7f7ec33f1740 1 -- 172.21.15.5:0/350006136 shutdown_connections 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.198+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/350006136 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x5603362ffe40 0x56033636d740 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.198+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/350006136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5603362f0370 0x560336369b80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.198+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/350006136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5603362e6b40 0x5603362f4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.198+0000 7f7ec33f1740 1 --2- 172.21.15.5:0/350006136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603362e55d0 0x5603362f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.198+0000 7f7ec33f1740 1 -- 172.21.15.5:0/350006136 >> 172.21.15.5:0/350006136 conn(0x5603362ae310 msgr2=0x5603362e0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:16.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.198+0000 7f7e9eff5640 0 client.30592 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:27:16.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.198+0000 7f7ec33f1740 1 -- 172.21.15.5:0/350006136 shutdown_connections 2024-09-19T06:27:16.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.198+0000 7f7ec33f1740 1 -- 172.21.15.5:0/350006136 wait complete. 2024-09-19T06:27:16.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.198+0000 7f7ec33f1740 20 client.30592 trim_cache size 0 max 16384 2024-09-19T06:27:16.217 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:27:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.199+0000 7f7ec1018640 1 -- 172.21.15.5:0/2261253111 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7ea40a34e0 msgr2=0x7f7ea40a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.199+0000 7f7ec1018640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7ea40a34e0 0x7f7ea40a59a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4005fd0 tx=0x7f7eb4009500 comp rx=0 tx=0).stop 2024-09-19T06:27:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.199+0000 7f7ec1018640 1 -- 172.21.15.5:0/2261253111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc100e20 msgr2=0x7f7ebc19e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.199+0000 7f7ec1018640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc100e20 0x7f7ebc19e0e0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f7eac006010 tx=0x7f7eac004040 comp rx=0 tx=0).stop 2024-09-19T06:27:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.199+0000 7f7ec1018640 1 -- 172.21.15.5:0/2261253111 shutdown_connections 2024-09-19T06:27:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.199+0000 7f7ec1018640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ebc102ef0 0x7f7ebc196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.199+0000 7f7ec1018640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f7ea40a34e0 0x7f7ea40a59a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.199+0000 7f7ec1018640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ebc102570 0x7f7ebc1962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.199+0000 7f7ec1018640 1 --2- 172.21.15.5:0/2261253111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc100e20 0x7f7ebc19e0e0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.199+0000 7f7ec1018640 1 -- 172.21.15.5:0/2261253111 >> 172.21.15.5:0/2261253111 conn(0x7f7ebc0fc7d0 msgr2=0x7f7ebc069ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.199+0000 7f7ec1018640 1 -- 172.21.15.5:0/2261253111 shutdown_connections 2024-09-19T06:27:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:16.199+0000 7f7ec1018640 1 -- 172.21.15.5:0/2261253111 wait complete. 2024-09-19T06:27:16.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 102969 8.55 MB/sec execute 520 sec latency 92.699 ms 2024-09-19T06:27:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:16 smithi005 ceph-mon[30955]: pgmap v757: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.6 MiB/s wr, 312 op/s 2024-09-19T06:27:16.895 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:16 smithi049 ceph-mon[34640]: pgmap v757: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.6 MiB/s wr, 312 op/s 2024-09-19T06:27:17.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:16 smithi157 ceph-mon[35346]: pgmap v757: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.6 MiB/s wr, 312 op/s 2024-09-19T06:27:17.462 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:27:17.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 103152 8.56 MB/sec execute 521 sec latency 59.955 ms 2024-09-19T06:27:17.844 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:27:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.431+0000 7f254b9ee640 1 -- 172.21.15.5:0/2520877994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254410d530 msgr2=0x7f25441150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.431+0000 7f254b9ee640 1 --2- 172.21.15.5:0/2520877994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254410d530 0x7f25441150b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f253c009ca0 tx=0x7f253c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.431+0000 7f254b9ee640 1 -- 172.21.15.5:0/2520877994 shutdown_connections 2024-09-19T06:27:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.431+0000 7f254b9ee640 1 --2- 172.21.15.5:0/2520877994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254410d530 0x7f25441150b0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.431+0000 7f254b9ee640 1 --2- 172.21.15.5:0/2520877994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2544108e00 0x7f254410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.431+0000 7f254b9ee640 1 --2- 172.21.15.5:0/2520877994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544108450 0x7f2544108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.431+0000 7f254b9ee640 1 -- 172.21.15.5:0/2520877994 >> 172.21.15.5:0/2520877994 conn(0x7f2544077ba0 msgr2=0x7f2544079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.431+0000 7f254b9ee640 1 -- 172.21.15.5:0/2520877994 shutdown_connections 2024-09-19T06:27:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.431+0000 7f254b9ee640 1 -- 172.21.15.5:0/2520877994 wait complete. 2024-09-19T06:27:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.432+0000 7f254b9ee640 1 Processor -- start 2024-09-19T06:27:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.432+0000 7f254b9ee640 1 -- start start 2024-09-19T06:27:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.432+0000 7f254b9ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544108450 0x7f2544082b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.432+0000 7f254b9ee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2544108e00 0x7f254407ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.432+0000 7f254b9ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254410d530 0x7f254407b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.432+0000 7f254b9ee640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2544118460 con 0x7f254410d530 2024-09-19T06:27:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.432+0000 7f254b9ee640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25441182e0 con 0x7f2544108450 2024-09-19T06:27:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.432+0000 7f254b9ee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f25441185e0 con 0x7f2544108e00 2024-09-19T06:27:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.433+0000 7f2549f8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254410d530 0x7f254407b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.433+0000 7f2549f8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254410d530 0x7f254407b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44748/0 (socket says 172.21.15.5:44748) 2024-09-19T06:27:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.433+0000 7f254978a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544108450 0x7f2544082b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.433+0000 7f2549f8b640 1 -- 172.21.15.5:0/3299950519 learned_addr learned my addr 172.21.15.5:0/3299950519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.433+0000 7f2549f8b640 1 -- 172.21.15.5:0/3299950519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2544108e00 msgr2=0x7f254407ab10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:27:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.433+0000 7f2549f8b640 1 --2- 172.21.15.5:0/3299950519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2544108e00 0x7f254407ab10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.433+0000 7f2549f8b640 1 -- 172.21.15.5:0/3299950519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544108450 msgr2=0x7f2544082b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.433+0000 7f2549f8b640 1 --2- 172.21.15.5:0/3299950519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544108450 0x7f2544082b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.433+0000 7f2549f8b640 1 -- 172.21.15.5:0/3299950519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254407b870 con 0x7f254410d530 2024-09-19T06:27:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.433+0000 7f2549f8b640 1 --2- 172.21.15.5:0/3299950519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254410d530 0x7f254407b050 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f253c031040 tx=0x7f253c032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.434+0000 7f253a7fc640 1 -- 172.21.15.5:0/3299950519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c003b70 con 0x7f254410d530 2024-09-19T06:27:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.434+0000 7f254b9ee640 1 -- 172.21.15.5:0/3299950519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25441b3690 con 0x7f254410d530 2024-09-19T06:27:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.434+0000 7f254b9ee640 1 -- 172.21.15.5:0/3299950519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25441b3c20 con 0x7f254410d530 2024-09-19T06:27:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.434+0000 7f253a7fc640 1 -- 172.21.15.5:0/3299950519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f253c003d10 con 0x7f254410d530 2024-09-19T06:27:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.434+0000 7f253a7fc640 1 -- 172.21.15.5:0/3299950519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c030a20 con 0x7f254410d530 2024-09-19T06:27:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.437+0000 7f254ddc7740 1 Processor -- start 2024-09-19T06:27:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.437+0000 7f253a7fc640 1 -- 172.21.15.5:0/3299950519 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f253c030c60 con 0x7f254410d530 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.438+0000 7f254ddc7740 1 -- start start 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.438+0000 7f254ddc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e846d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.438+0000 7f254ddc7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 0x55d1e846da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.439+0000 7f254978a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 0x55d1e846da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.439+0000 7f254978a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 0x55d1e846da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44762/0 (socket says 172.21.15.5:44762) 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.439+0000 7f254ddc7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e8478370 0x55d1e847a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.439+0000 7f254ddc7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1e83d32f0 con 0x55d1e846d5d0 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.439+0000 7f254ddc7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d1e837acf0 con 0x55d1e8478370 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.439+0000 7f254ddc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d1e83cf2b0 con 0x55d1e846eb40 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.439+0000 7f254978a640 1 -- 172.21.15.5:0/1709040184 learned_addr learned my addr 172.21.15.5:0/1709040184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.439+0000 7f2548f89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e846d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.440+0000 7f254978a640 1 -- 172.21.15.5:0/1709040184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 msgr2=0x55d1e846d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.440+0000 7f254978a640 1 --2- 172.21.15.5:0/1709040184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e846d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.440+0000 7f254978a640 1 -- 172.21.15.5:0/1709040184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e8478370 msgr2=0x55d1e847a760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:27:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.440+0000 7f254978a640 1 --2- 172.21.15.5:0/1709040184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e8478370 0x55d1e847a760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.440+0000 7f254978a640 1 -- 172.21.15.5:0/1709040184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e835f3a0 con 0x55d1e846d5d0 2024-09-19T06:27:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.440+0000 7f2548f89640 1 --2- 172.21.15.5:0/1709040184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e846d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:27:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.440+0000 7f254978a640 1 --2- 172.21.15.5:0/1709040184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 0x55d1e846da10 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2540009cf0 tx=0x7f2540011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71ba1e871da85e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.441+0000 7f253a7fc640 1 --2- 172.21.15.5:0/3299950519 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25300a3430 0x7f25300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.441+0000 7f252f7fe640 1 -- 172.21.15.5:0/1709040184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540004230 con 0x55d1e846d5d0 2024-09-19T06:27:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.441+0000 7f252f7fe640 1 -- 172.21.15.5:0/1709040184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25400043d0 con 0x55d1e846d5d0 2024-09-19T06:27:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.441+0000 7f252f7fe640 1 -- 172.21.15.5:0/1709040184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540019890 con 0x55d1e846d5d0 2024-09-19T06:27:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.441+0000 7f2548f89640 1 --2- 172.21.15.5:0/3299950519 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25300a3430 0x7f25300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.441+0000 7f253a7fc640 1 -- 172.21.15.5:0/3299950519 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f253c0f0fb0 con 0x7f254410d530 2024-09-19T06:27:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.442+0000 7f254ddc7740 1 -- 172.21.15.5:0/1709040184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 msgr2=0x55d1e846da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.442+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1709040184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 0x55d1e846da10 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2540009cf0 tx=0x7f2540011810 comp rx=0 tx=0).stop 2024-09-19T06:27:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.442+0000 7f254ddc7740 1 -- 172.21.15.5:0/1709040184 shutdown_connections 2024-09-19T06:27:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.442+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1709040184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e8478370 0x55d1e847a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.442+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1709040184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 0x55d1e846da10 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.442+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1709040184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e846d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.442+0000 7f254ddc7740 1 -- 172.21.15.5:0/1709040184 >> 172.21.15.5:0/1709040184 conn(0x55d1e8436310 msgr2=0x55d1e8469c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.442+0000 7f2548f89640 1 --2- 172.21.15.5:0/3299950519 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25300a3430 0x7f25300a58f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55d1e846e070 tx=0x7f2534005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.442+0000 7f254ddc7740 1 -- 172.21.15.5:0/1709040184 shutdown_connections 2024-09-19T06:27:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.442+0000 7f254ddc7740 1 -- 172.21.15.5:0/1709040184 wait complete. 2024-09-19T06:27:18.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.443+0000 7f254ddc7740 1 -- start start 2024-09-19T06:27:18.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.443+0000 7f254ddc7740 1 Processor -- start 2024-09-19T06:27:18.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.443+0000 7f254ddc7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e846d5d0 0x55d1e8477d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.443+0000 7f254ddc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e8475000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.443+0000 7f254ddc7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e8478370 0x55d1e84755d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f254978a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e846d5d0 0x55d1e8477d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f254ddc7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1e847b070 con 0x55d1e8478370 2024-09-19T06:27:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f254ddc7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d1e838e060 con 0x55d1e846d5d0 2024-09-19T06:27:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f254ddc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d1e847b1f0 con 0x55d1e846eb40 2024-09-19T06:27:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f254978a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e846d5d0 0x55d1e8477d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48988/0 (socket says 172.21.15.5:48988) 2024-09-19T06:27:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f254978a640 1 -- 172.21.15.5:0/1259159040 learned_addr learned my addr 172.21.15.5:0/1259159040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f2548f89640 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e8475000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f254ddc7740 10 client.0 _ll_register_callbacks cb 0x55d1e80e18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f2549f8b640 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e8478370 0x55d1e84755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f2548f89640 1 -- 172.21.15.5:0/1259159040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e846d5d0 msgr2=0x55d1e8477d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f2548f89640 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e846d5d0 0x55d1e8477d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f2548f89640 1 -- 172.21.15.5:0/1259159040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e8478370 msgr2=0x55d1e84755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f2548f89640 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e8478370 0x55d1e84755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f2548f89640 1 -- 172.21.15.5:0/1259159040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e81443c0 con 0x55d1e846eb40 2024-09-19T06:27:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f2548f89640 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e8475000 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f254407c360 tx=0x7f2534004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc421565dc10c37e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f252cff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:27:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.444+0000 7f252cff9640 1 -- 172.21.15.5:0/1259159040 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534023070 con 0x55d1e846eb40 2024-09-19T06:27:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.445+0000 7f254ddc7740 1 -- 172.21.15.5:0/1259159040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25400043d0 con 0x55d1e846eb40 2024-09-19T06:27:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.445+0000 7f252cff9640 1 -- 172.21.15.5:0/1259159040 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f25340035f0 con 0x55d1e846eb40 2024-09-19T06:27:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.445+0000 7f252cff9640 1 -- 172.21.15.5:0/1259159040 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f253402f050 con 0x55d1e846eb40 2024-09-19T06:27:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.445+0000 7f254ddc7740 10 client.30601 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.445+0000 7f254ddc7740 1 -- 172.21.15.5:0/1259159040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55d1e8486c40 con 0x55d1e846eb40 2024-09-19T06:27:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.445+0000 7f252cff9640 1 -- 172.21.15.5:0/1259159040 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2534003ca0 con 0x55d1e846eb40 2024-09-19T06:27:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.445+0000 7f252cff9640 1 -- 172.21.15.5:0/1259159040 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f253401fab0 con 0x55d1e846eb40 2024-09-19T06:27:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.446+0000 7f254ddc7740 10 client.30601 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.446+0000 7f254ddc7740 10 client.30601 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:27:18.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.446+0000 7f254ddc7740 20 client.30601 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:18.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.446+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d1e84880f0 0x55d1e848a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.446+0000 7f254ddc7740 4 client.30601 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:18.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.446+0000 7f254ddc7740 1 -- 172.21.15.5:0/1259159040 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f25400043d0 con 0x55d1e84880f0 2024-09-19T06:27:18.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.446+0000 7f254978a640 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d1e84880f0 0x55d1e848a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.447+0000 7f254978a640 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d1e84880f0 0x55d1e848a520 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ba999a948a98c1e server_cookie=aa76a3557be2b342 in_seq=0 out_seq=0 2024-09-19T06:27:18.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.447+0000 7f252cff9640 10 client.30601 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:27:18.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.447+0000 7f252cff9640 1 -- 172.21.15.5:0/1259159040 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f25400043d0 con 0x55d1e84880f0 2024-09-19T06:27:18.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.447+0000 7f252cff9640 10 client.30601 handle_command_reply: tid=0 2024-09-19T06:27:18.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.447+0000 7f254ddc7740 1 client.30601 shutdown 2024-09-19T06:27:18.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.448+0000 7f254ddc7740 1 -- 172.21.15.5:0/1259159040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 msgr2=0x55d1e8475000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:18.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.448+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e8475000 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f254407c360 tx=0x7f2534004730 comp rx=0 tx=0).stop 2024-09-19T06:27:18.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.448+0000 7f254ddc7740 1 -- 172.21.15.5:0/1259159040 shutdown_connections 2024-09-19T06:27:18.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.448+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d1e84880f0 0x55d1e848a520 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.448+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e8478370 0x55d1e84755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.448+0000 7f252cff9640 0 client.30601 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:27:18.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.448+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e8475000 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.448+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1259159040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e846d5d0 0x55d1e8477d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.448+0000 7f254ddc7740 1 -- 172.21.15.5:0/1259159040 >> 172.21.15.5:0/1259159040 conn(0x55d1e8436310 msgr2=0x55d1e846f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:18.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.448+0000 7f254ddc7740 1 -- 172.21.15.5:0/1259159040 shutdown_connections 2024-09-19T06:27:18.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.448+0000 7f254ddc7740 1 -- 172.21.15.5:0/1259159040 wait complete. 2024-09-19T06:27:18.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.448+0000 7f254ddc7740 20 client.30601 trim_cache size 0 max 16384 2024-09-19T06:27:18.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.469+0000 7f254ddc7740 1 Processor -- start 2024-09-19T06:27:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254ddc7740 1 -- start start 2024-09-19T06:27:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254ddc7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 0x55d1e84843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254ddc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e8473fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254ddc7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e8478370 0x55d1e848fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254ddc7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f252800e7e0 con 0x55d1e846d5d0 2024-09-19T06:27:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254ddc7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2528010320 con 0x55d1e8478370 2024-09-19T06:27:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254ddc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f252800eeb0 con 0x55d1e846eb40 2024-09-19T06:27:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254978a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 0x55d1e84843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254978a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 0x55d1e84843a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44780/0 (socket says 172.21.15.5:44780) 2024-09-19T06:27:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254978a640 1 -- 172.21.15.5:0/3345641035 learned_addr learned my addr 172.21.15.5:0/3345641035 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f2549f8b640 1 --2- 172.21.15.5:0/3345641035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e8478370 0x55d1e848fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f2548f89640 1 --2- 172.21.15.5:0/3345641035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e8473fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254978a640 1 -- 172.21.15.5:0/3345641035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 msgr2=0x55d1e8473fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:18.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254978a640 1 --2- 172.21.15.5:0/3345641035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e8473fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254978a640 1 -- 172.21.15.5:0/3345641035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e8478370 msgr2=0x55d1e848fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:18.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.470+0000 7f254978a640 1 --2- 172.21.15.5:0/3345641035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e8478370 0x55d1e848fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.471+0000 7f254978a640 1 -- 172.21.15.5:0/3345641035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e84694f0 con 0x55d1e846d5d0 2024-09-19T06:27:18.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.471+0000 7f254978a640 1 --2- 172.21.15.5:0/3345641035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 0x55d1e84843a0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2540014040 tx=0x7f2540005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9731b31823402d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:18.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.471+0000 7f252effd640 1 -- 172.21.15.5:0/3345641035 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540031070 con 0x55d1e846d5d0 2024-09-19T06:27:18.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.471+0000 7f252effd640 1 -- 172.21.15.5:0/3345641035 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2540005040 con 0x55d1e846d5d0 2024-09-19T06:27:18.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.471+0000 7f252effd640 1 -- 172.21.15.5:0/3345641035 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25400038e0 con 0x55d1e846d5d0 2024-09-19T06:27:18.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.472+0000 7f254ddc7740 1 -- 172.21.15.5:0/3345641035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 msgr2=0x55d1e84843a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:18.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.472+0000 7f254ddc7740 1 --2- 172.21.15.5:0/3345641035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 0x55d1e84843a0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2540014040 tx=0x7f2540005850 comp rx=0 tx=0).stop 2024-09-19T06:27:18.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.472+0000 7f254ddc7740 1 -- 172.21.15.5:0/3345641035 shutdown_connections 2024-09-19T06:27:18.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.472+0000 7f254ddc7740 1 --2- 172.21.15.5:0/3345641035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e8478370 0x55d1e848fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.472+0000 7f254ddc7740 1 --2- 172.21.15.5:0/3345641035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e8473fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.472+0000 7f254ddc7740 1 --2- 172.21.15.5:0/3345641035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e846d5d0 0x55d1e84843a0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.472+0000 7f254ddc7740 1 -- 172.21.15.5:0/3345641035 >> 172.21.15.5:0/3345641035 conn(0x55d1e8436310 msgr2=0x55d1e84738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.473+0000 7f254ddc7740 1 -- 172.21.15.5:0/3345641035 shutdown_connections 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.473+0000 7f254ddc7740 1 -- 172.21.15.5:0/3345641035 wait complete. 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.473+0000 7f254ddc7740 1 -- start start 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.473+0000 7f254ddc7740 1 Processor -- start 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f254ddc7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e846d5d0 0x55d1e8480dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f254ddc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e847c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f254ddc7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e8478370 0x55d1e847c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f254ddc7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f252800fc50 con 0x55d1e8478370 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f254ddc7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f252800f580 con 0x55d1e846d5d0 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f254ddc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f252800c7e0 con 0x55d1e846eb40 2024-09-19T06:27:18.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f254ddc7740 10 client.0 _ll_register_callbacks cb 0x7f25340098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f2548f89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e847c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f2548f89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e847c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56304/0 (socket says 172.21.15.5:56304) 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f2548f89640 1 -- 172.21.15.5:0/1431228070 learned_addr learned my addr 172.21.15.5:0/1431228070 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f2549f8b640 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e8478370 0x55d1e847c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f254978a640 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e846d5d0 0x55d1e8480dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f2548f89640 1 -- 172.21.15.5:0/1431228070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e846d5d0 msgr2=0x55d1e8480dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f2548f89640 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e846d5d0 0x55d1e8480dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f2548f89640 1 -- 172.21.15.5:0/1431228070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e8478370 msgr2=0x55d1e847c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f2548f89640 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e8478370 0x55d1e847c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.474+0000 7f2548f89640 1 -- 172.21.15.5:0/1431228070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e84f1dc0 con 0x55d1e846eb40 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.475+0000 7f2548f89640 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e847c0f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55d1e8474890 tx=0x7f2534009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d93f7bd748bc8220 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.475+0000 7f252cff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:27:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.475+0000 7f252cff9640 1 -- 172.21.15.5:0/1431228070 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534023040 con 0x55d1e846eb40 2024-09-19T06:27:18.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.475+0000 7f252cff9640 1 -- 172.21.15.5:0/1431228070 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f253401f6e0 con 0x55d1e846eb40 2024-09-19T06:27:18.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.475+0000 7f254ddc7740 1 -- 172.21.15.5:0/1431228070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2540005040 con 0x55d1e846eb40 2024-09-19T06:27:18.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.475+0000 7f252cff9640 1 -- 172.21.15.5:0/1431228070 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f253401f8e0 con 0x55d1e846eb40 2024-09-19T06:27:18.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.475+0000 7f254ddc7740 10 client.30607 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:18.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.475+0000 7f254ddc7740 1 -- 172.21.15.5:0/1431228070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55d1e84f2f50 con 0x55d1e846eb40 2024-09-19T06:27:18.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.476+0000 7f252cff9640 1 -- 172.21.15.5:0/1431228070 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2540005040 con 0x55d1e846eb40 2024-09-19T06:27:18.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.476+0000 7f252cff9640 1 -- 172.21.15.5:0/1431228070 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f253401b480 con 0x55d1e846eb40 2024-09-19T06:27:18.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.476+0000 7f254ddc7740 10 client.30607 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:18.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.476+0000 7f254ddc7740 10 client.30607 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:27:18.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.476+0000 7f254ddc7740 20 client.30607 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:18.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.476+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d1e84880f0 0x55d1e84f5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:18.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.476+0000 7f254ddc7740 4 client.30607 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:27:18.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.476+0000 7f254ddc7740 1 -- 172.21.15.5:0/1431228070 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1e83e0860 con 0x55d1e84880f0 2024-09-19T06:27:18.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.476+0000 7f254978a640 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d1e84880f0 0x55d1e84f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:18.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f252cff9640 10 client.30607 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:27:18.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f254978a640 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d1e84880f0 0x55d1e84f5740 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd70b2b0ee0709ac server_cookie=4a18af2d0cc7fc9c in_seq=0 out_seq=0 2024-09-19T06:27:18.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f252cff9640 1 -- 172.21.15.5:0/1431228070 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1e83e0860 con 0x55d1e84880f0 2024-09-19T06:27:18.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f252cff9640 10 client.30607 handle_command_reply: tid=0 2024-09-19T06:27:18.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f254ddc7740 1 client.30607 shutdown 2024-09-19T06:27:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f254ddc7740 1 -- 172.21.15.5:0/1431228070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 msgr2=0x55d1e847c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e847c0f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55d1e8474890 tx=0x7f2534009a10 comp rx=0 tx=0).stop 2024-09-19T06:27:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f254ddc7740 1 -- 172.21.15.5:0/1431228070 shutdown_connections 2024-09-19T06:27:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x55d1e84880f0 0x55d1e84f5740 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1e8478370 0x55d1e847c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1e846eb40 0x55d1e847c0f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f254ddc7740 1 --2- 172.21.15.5:0/1431228070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d1e846d5d0 0x55d1e8480dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f254ddc7740 1 -- 172.21.15.5:0/1431228070 >> 172.21.15.5:0/1431228070 conn(0x55d1e8436310 msgr2=0x55d1e84740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.477+0000 7f252cff9640 0 client.30607 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:27:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.478+0000 7f254ddc7740 1 -- 172.21.15.5:0/1431228070 shutdown_connections 2024-09-19T06:27:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.478+0000 7f254ddc7740 1 -- 172.21.15.5:0/1431228070 wait complete. 2024-09-19T06:27:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.478+0000 7f254ddc7740 20 client.30607 trim_cache size 0 max 16384 2024-09-19T06:27:18.486 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:27:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.479+0000 7f254b9ee640 1 -- 172.21.15.5:0/3299950519 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25300a3430 msgr2=0x7f25300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.479+0000 7f254b9ee640 1 --2- 172.21.15.5:0/3299950519 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25300a3430 0x7f25300a58f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55d1e846e070 tx=0x7f2534005f90 comp rx=0 tx=0).stop 2024-09-19T06:27:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.479+0000 7f254b9ee640 1 -- 172.21.15.5:0/3299950519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254410d530 msgr2=0x7f254407b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.479+0000 7f254b9ee640 1 --2- 172.21.15.5:0/3299950519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254410d530 0x7f254407b050 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f253c031040 tx=0x7f253c032ed0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.479+0000 7f254b9ee640 1 -- 172.21.15.5:0/3299950519 shutdown_connections 2024-09-19T06:27:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.479+0000 7f254b9ee640 1 --2- 172.21.15.5:0/3299950519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254410d530 0x7f254407b050 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.479+0000 7f254b9ee640 1 --2- 172.21.15.5:0/3299950519 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25300a3430 0x7f25300a58f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.479+0000 7f254b9ee640 1 --2- 172.21.15.5:0/3299950519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2544108e00 0x7f254407ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.479+0000 7f254b9ee640 1 --2- 172.21.15.5:0/3299950519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544108450 0x7f2544082b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.479+0000 7f254b9ee640 1 -- 172.21.15.5:0/3299950519 >> 172.21.15.5:0/3299950519 conn(0x7f2544077ba0 msgr2=0x7f2544078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.479+0000 7f254b9ee640 1 -- 172.21.15.5:0/3299950519 shutdown_connections 2024-09-19T06:27:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:18.479+0000 7f254b9ee640 1 -- 172.21.15.5:0/3299950519 wait complete. 2024-09-19T06:27:18.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 103294 8.56 MB/sec execute 522 sec latency 108.313 ms 2024-09-19T06:27:18.881 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:18 smithi005 ceph-mon[30955]: pgmap v758: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 280 op/s 2024-09-19T06:27:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:18 smithi049 ceph-mon[34640]: pgmap v758: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 280 op/s 2024-09-19T06:27:19.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:18 smithi157 ceph-mon[35346]: pgmap v758: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 280 op/s 2024-09-19T06:27:19.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.251+0000 7fd10bdf8640 1 -- 172.21.15.5:0/4098202345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1041069b0 msgr2=0x7fd104106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.251+0000 7fd10bdf8640 1 --2- 172.21.15.5:0/4098202345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1041069b0 0x7fd104106d90 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd100009930 tx=0x7fd10002a420 comp rx=0 tx=0).stop 2024-09-19T06:27:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.251+0000 7fd10bdf8640 1 -- 172.21.15.5:0/4098202345 shutdown_connections 2024-09-19T06:27:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.251+0000 7fd10bdf8640 1 --2- 172.21.15.5:0/4098202345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd104101340 0x7fd10410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.251+0000 7fd10bdf8640 1 --2- 172.21.15.5:0/4098202345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1041009a0 0x7fd104100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.251+0000 7fd10bdf8640 1 --2- 172.21.15.5:0/4098202345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1041069b0 0x7fd104106d90 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.251+0000 7fd10bdf8640 1 -- 172.21.15.5:0/4098202345 >> 172.21.15.5:0/4098202345 conn(0x7fd1040779a0 msgr2=0x7fd1040fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.251+0000 7fd10bdf8640 1 -- 172.21.15.5:0/4098202345 shutdown_connections 2024-09-19T06:27:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.251+0000 7fd10bdf8640 1 -- 172.21.15.5:0/4098202345 wait complete. 2024-09-19T06:27:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd10bdf8640 1 Processor -- start 2024-09-19T06:27:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd10bdf8640 1 -- start start 2024-09-19T06:27:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd10bdf8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1041009a0 0x7fd10419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd10bdf8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd104101340 0x7fd10419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd10bdf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1041069b0 0x7fd10419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd10bdf8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd10410fba0 con 0x7fd1041069b0 2024-09-19T06:27:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd10bdf8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd10410fa20 con 0x7fd1041009a0 2024-09-19T06:27:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd10bdf8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd10410fd20 con 0x7fd104101340 2024-09-19T06:27:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd109b94640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1041009a0 0x7fd10419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd109b94640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1041009a0 0x7fd10419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49032/0 (socket says 172.21.15.5:49032) 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd109b94640 1 -- 172.21.15.5:0/2201017129 learned_addr learned my addr 172.21.15.5:0/2201017129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd109393640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd104101340 0x7fd10419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.252+0000 7fd10a395640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1041069b0 0x7fd10419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.253+0000 7fd109b94640 1 -- 172.21.15.5:0/2201017129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd104101340 msgr2=0x7fd10419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.253+0000 7fd109b94640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd104101340 0x7fd10419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.253+0000 7fd109b94640 1 -- 172.21.15.5:0/2201017129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1041069b0 msgr2=0x7fd10419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.253+0000 7fd109b94640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1041069b0 0x7fd10419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.253+0000 7fd109b94640 1 -- 172.21.15.5:0/2201017129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10419f510 con 0x7fd1041009a0 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.253+0000 7fd109b94640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1041009a0 0x7fd10419a520 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd100005940 tx=0x7fd1000040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.253+0000 7fd0faffd640 1 -- 172.21.15.5:0/2201017129 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10003f030 con 0x7fd1041009a0 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.253+0000 7fd0faffd640 1 -- 172.21.15.5:0/2201017129 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1000044c0 con 0x7fd1041009a0 2024-09-19T06:27:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.253+0000 7fd10bdf8640 1 -- 172.21.15.5:0/2201017129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10419f7a0 con 0x7fd1041009a0 2024-09-19T06:27:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.253+0000 7fd0faffd640 1 -- 172.21.15.5:0/2201017129 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100035070 con 0x7fd1041009a0 2024-09-19T06:27:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.254+0000 7fd10bdf8640 1 -- 172.21.15.5:0/2201017129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1041a7090 con 0x7fd1041009a0 2024-09-19T06:27:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.257+0000 7fd0faffd640 1 -- 172.21.15.5:0/2201017129 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd100042030 con 0x7fd1041009a0 2024-09-19T06:27:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.257+0000 7fd10ec66740 1 Processor -- start 2024-09-19T06:27:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.258+0000 7fd10ec66740 1 -- start start 2024-09-19T06:27:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.258+0000 7fd10ec66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.258+0000 7fd10ec66740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4c8330 0x56417c4daf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.258+0000 7fd10ec66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4db4c0 0x56417c4dd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.258+0000 7fd10ec66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56417c42c2f0 con 0x56417c4c7a10 2024-09-19T06:27:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.258+0000 7fd10ec66740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56417c3d4270 con 0x56417c4c8330 2024-09-19T06:27:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.258+0000 7fd10ec66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56417c4282b0 con 0x56417c4db4c0 2024-09-19T06:27:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.258+0000 7fd109393640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4db4c0 0x56417c4dd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.258+0000 7fd109393640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4db4c0 0x56417c4dd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56342/0 (socket says 172.21.15.5:56342) 2024-09-19T06:27:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.258+0000 7fd109393640 1 -- 172.21.15.5:0/2182956329 learned_addr learned my addr 172.21.15.5:0/2182956329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.258+0000 7fd10a395640 1 --2- 172.21.15.5:0/2182956329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.258+0000 7fd0faffd640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0f00a3430 0x7fd0f00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.259+0000 7fd109b94640 1 --2- 172.21.15.5:0/2182956329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4c8330 0x56417c4daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.259+0000 7fd10a395640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0f00a3430 0x7fd0f00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.259+0000 7fd109393640 1 -- 172.21.15.5:0/2182956329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4c8330 msgr2=0x56417c4daf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.259+0000 7fd109393640 1 --2- 172.21.15.5:0/2182956329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4c8330 0x56417c4daf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.259+0000 7fd109393640 1 -- 172.21.15.5:0/2182956329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 msgr2=0x56417c4c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.259+0000 7fd109393640 1 --2- 172.21.15.5:0/2182956329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.259+0000 7fd109393640 1 -- 172.21.15.5:0/2182956329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56417c3d3340 con 0x56417c4db4c0 2024-09-19T06:27:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.259+0000 7fd0faffd640 1 -- 172.21.15.5:0/2201017129 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fd1000effe0 con 0x7fd1041009a0 2024-09-19T06:27:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.259+0000 7fd109393640 1 --2- 172.21.15.5:0/2182956329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4db4c0 0x56417c4dd8b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009c90 tx=0x7fd0f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1420a3e2854f2dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.259+0000 7fd10a395640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0f00a3430 0x7fd0f00a58f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009fd0 tx=0x7fd0fc004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.259+0000 7fd0effff640 1 -- 172.21.15.5:0/2182956329 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f401e070 con 0x56417c4db4c0 2024-09-19T06:27:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.260+0000 7fd0effff640 1 -- 172.21.15.5:0/2182956329 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0f4019ae0 con 0x56417c4db4c0 2024-09-19T06:27:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.260+0000 7fd0effff640 1 -- 172.21.15.5:0/2182956329 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400ea40 con 0x56417c4db4c0 2024-09-19T06:27:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.260+0000 7fd10ec66740 1 -- 172.21.15.5:0/2182956329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4db4c0 msgr2=0x56417c4dd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.260+0000 7fd10ec66740 1 --2- 172.21.15.5:0/2182956329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4db4c0 0x56417c4dd8b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009c90 tx=0x7fd0f4011810 comp rx=0 tx=0).stop 2024-09-19T06:27:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.261+0000 7fd10ec66740 1 -- 172.21.15.5:0/2182956329 shutdown_connections 2024-09-19T06:27:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.261+0000 7fd10ec66740 1 --2- 172.21.15.5:0/2182956329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4db4c0 0x56417c4dd8b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.261+0000 7fd10ec66740 1 --2- 172.21.15.5:0/2182956329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4c8330 0x56417c4daf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.261+0000 7fd10ec66740 1 --2- 172.21.15.5:0/2182956329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.261+0000 7fd10ec66740 1 -- 172.21.15.5:0/2182956329 >> 172.21.15.5:0/2182956329 conn(0x56417c4c4060 msgr2=0x56417c4c4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.261+0000 7fd10ec66740 1 -- 172.21.15.5:0/2182956329 shutdown_connections 2024-09-19T06:27:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.261+0000 7fd10ec66740 1 -- 172.21.15.5:0/2182956329 wait complete. 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.261+0000 7fd10ec66740 1 -- start start 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.261+0000 7fd10ec66740 1 Processor -- start 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd10ec66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4d83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd10ec66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 0x56417c4d8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd10ec66740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4d8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd10ec66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56417c4de090 con 0x56417c4c7a10 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd10ec66740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56417c3b5290 con 0x56417c4db4c0 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd10ec66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56417c4de210 con 0x56417c4c8330 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd10ec66740 10 client.0 _ll_register_callbacks cb 0x56417c3e6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd10a395640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4d8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd109b94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4d83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd10a395640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4d8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49050/0 (socket says 172.21.15.5:49050) 2024-09-19T06:27:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd10a395640 1 -- 172.21.15.5:0/3818440727 learned_addr learned my addr 172.21.15.5:0/3818440727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd109393640 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 0x56417c4d8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd109b94640 1 -- 172.21.15.5:0/3818440727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 msgr2=0x56417c4d8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd109b94640 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 0x56417c4d8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd109b94640 1 -- 172.21.15.5:0/3818440727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 msgr2=0x56417c4d8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd109b94640 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4d8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.262+0000 7fd109b94640 1 -- 172.21.15.5:0/3818440727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56417c13f210 con 0x56417c4c7a10 2024-09-19T06:27:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.263+0000 7fd109393640 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 0x56417c4d8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.263+0000 7fd10a395640 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4d8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.263+0000 7fd109b94640 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4d83d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd1000ed6b0 tx=0x7fd100009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1071e93d0edb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.263+0000 7fd0ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:27:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.263+0000 7fd0ed7fa640 1 -- 172.21.15.5:0/3818440727 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100049030 con 0x56417c4c7a10 2024-09-19T06:27:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.263+0000 7fd0ed7fa640 1 -- 172.21.15.5:0/3818440727 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fd1000b8960 con 0x56417c4c7a10 2024-09-19T06:27:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.263+0000 7fd10ec66740 1 -- 172.21.15.5:0/3818440727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f4019ae0 con 0x56417c4c7a10 2024-09-19T06:27:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.263+0000 7fd0ed7fa640 1 -- 172.21.15.5:0/3818440727 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fd1000b6e60 con 0x56417c4c7a10 2024-09-19T06:27:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.263+0000 7fd10ec66740 10 client.21318 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.263+0000 7fd10ec66740 1 -- 172.21.15.5:0/3818440727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x56417c20b620 con 0x56417c4c7a10 2024-09-19T06:27:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.264+0000 7fd0ed7fa640 1 -- 172.21.15.5:0/3818440727 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0f4019ae0 con 0x56417c4c7a10 2024-09-19T06:27:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.264+0000 7fd0ed7fa640 1 -- 172.21.15.5:0/3818440727 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fd1000d4070 con 0x56417c4c7a10 2024-09-19T06:27:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.264+0000 7fd10ec66740 10 client.21318 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.264+0000 7fd10ec66740 10 client.21318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.264+0000 7fd10ec66740 20 client.21318 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.264+0000 7fd10ec66740 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56417c4eb400 0x56417c4ed850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.264+0000 7fd10ec66740 4 client.21318 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.264+0000 7fd10ec66740 1 -- 172.21.15.5:0/3818440727 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56417c439a20 con 0x56417c4eb400 2024-09-19T06:27:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.264+0000 7fd109393640 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56417c4eb400 0x56417c4ed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.265+0000 7fd109393640 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56417c4eb400 0x56417c4ed850 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0ebd4a0722f731 server_cookie=ad31d2a40f338e40 in_seq=0 out_seq=0 2024-09-19T06:27:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.265+0000 7fd0ed7fa640 10 client.21318 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.265+0000 7fd0ed7fa640 1 -- 172.21.15.5:0/3818440727 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56417c439a20 con 0x56417c4eb400 2024-09-19T06:27:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.265+0000 7fd0ed7fa640 10 client.21318 handle_command_reply: tid=0 2024-09-19T06:27:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd10ec66740 1 client.21318 shutdown 2024-09-19T06:27:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd10ec66740 1 -- 172.21.15.5:0/3818440727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 msgr2=0x56417c4d83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd10ec66740 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4d83d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd1000ed6b0 tx=0x7fd100009680 comp rx=0 tx=0).stop 2024-09-19T06:27:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd10ec66740 1 -- 172.21.15.5:0/3818440727 shutdown_connections 2024-09-19T06:27:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd10ec66740 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56417c4eb400 0x56417c4ed850 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd10ec66740 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4d8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd10ec66740 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 0x56417c4d8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd10ec66740 1 --2- 172.21.15.5:0/3818440727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4d83d0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd0ed7fa640 0 client.21318 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd10ec66740 1 -- 172.21.15.5:0/3818440727 >> 172.21.15.5:0/3818440727 conn(0x56417c4c4060 msgr2=0x56417c4c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd10ec66740 1 -- 172.21.15.5:0/3818440727 shutdown_connections 2024-09-19T06:27:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd10ec66740 1 -- 172.21.15.5:0/3818440727 wait complete. 2024-09-19T06:27:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.266+0000 7fd10ec66740 20 client.21318 trim_cache size 0 max 16384 2024-09-19T06:27:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.288+0000 7fd10ec66740 1 Processor -- start 2024-09-19T06:27:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd10ec66740 1 -- start start 2024-09-19T06:27:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd10ec66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd10ec66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 0x56417c4f3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd10ec66740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd10ec66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041ab6f0 con 0x56417c4c7a10 2024-09-19T06:27:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd10ec66740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041ad230 con 0x56417c4db4c0 2024-09-19T06:27:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd10ec66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041abdc0 con 0x56417c4c8330 2024-09-19T06:27:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd109b94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd109393640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 0x56417c4f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd109b94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44830/0 (socket says 172.21.15.5:44830) 2024-09-19T06:27:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd109393640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 0x56417c4f3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56360/0 (socket says 172.21.15.5:56360) 2024-09-19T06:27:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd109b94640 1 -- 172.21.15.5:0/2735550529 learned_addr learned my addr 172.21.15.5:0/2735550529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd10a395640 1 --2- 172.21.15.5:0/2735550529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd109b94640 1 -- 172.21.15.5:0/2735550529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 msgr2=0x56417c4f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd109b94640 1 --2- 172.21.15.5:0/2735550529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 0x56417c4f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd109b94640 1 -- 172.21.15.5:0/2735550529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 msgr2=0x56417c4f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd109b94640 1 --2- 172.21.15.5:0/2735550529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd109b94640 1 -- 172.21.15.5:0/2735550529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56417c4c8160 con 0x56417c4c7a10 2024-09-19T06:27:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.289+0000 7fd109393640 1 --2- 172.21.15.5:0/2735550529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 0x56417c4f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.290+0000 7fd109b94640 1 --2- 172.21.15.5:0/2735550529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4c30b0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fd1000c4980 tx=0x7fd100003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2176c6f674260abb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.290+0000 7fd0ef7fe640 1 -- 172.21.15.5:0/2735550529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100049030 con 0x56417c4c7a10 2024-09-19T06:27:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.290+0000 7fd0ef7fe640 1 -- 172.21.15.5:0/2735550529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1000c4eb0 con 0x56417c4c7a10 2024-09-19T06:27:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.290+0000 7fd0ef7fe640 1 -- 172.21.15.5:0/2735550529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1000ed930 con 0x56417c4c7a10 2024-09-19T06:27:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.291+0000 7fd10ec66740 1 -- 172.21.15.5:0/2735550529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 msgr2=0x56417c4c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.291+0000 7fd10ec66740 1 --2- 172.21.15.5:0/2735550529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4c30b0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fd1000c4980 tx=0x7fd100003f90 comp rx=0 tx=0).stop 2024-09-19T06:27:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.291+0000 7fd10ec66740 1 -- 172.21.15.5:0/2735550529 shutdown_connections 2024-09-19T06:27:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.291+0000 7fd10ec66740 1 --2- 172.21.15.5:0/2735550529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.291+0000 7fd10ec66740 1 --2- 172.21.15.5:0/2735550529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4c8330 0x56417c4f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.291+0000 7fd10ec66740 1 --2- 172.21.15.5:0/2735550529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c7a10 0x56417c4c30b0 secure :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fd1000c4980 tx=0x7fd100003f90 comp rx=0 tx=0).stop 2024-09-19T06:27:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.291+0000 7fd10ec66740 1 -- 172.21.15.5:0/2735550529 >> 172.21.15.5:0/2735550529 conn(0x56417c4c4060 msgr2=0x56417c4c29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.291+0000 7fd10ec66740 1 -- 172.21.15.5:0/2735550529 shutdown_connections 2024-09-19T06:27:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.291+0000 7fd10ec66740 1 -- 172.21.15.5:0/2735550529 wait complete. 2024-09-19T06:27:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.292+0000 7fd10ec66740 1 -- start start 2024-09-19T06:27:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.292+0000 7fd10ec66740 1 Processor -- start 2024-09-19T06:27:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.292+0000 7fd10ec66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c8330 0x56417c4df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.292+0000 7fd10ec66740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.292+0000 7fd10ec66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4eb400 0x56417c4dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.292+0000 7fd10ec66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041acb60 con 0x56417c4c8330 2024-09-19T06:27:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.292+0000 7fd10ec66740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041ac490 con 0x56417c4db4c0 2024-09-19T06:27:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.292+0000 7fd10ec66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041a96f0 con 0x56417c4eb400 2024-09-19T06:27:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.292+0000 7fd10ec66740 10 client.0 _ll_register_callbacks cb 0x7fd100009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd109b94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c8330 0x56417c4df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd10a395640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4eb400 0x56417c4dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd10a395640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4eb400 0x56417c4dfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56362/0 (socket says 172.21.15.5:56362) 2024-09-19T06:27:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd10a395640 1 -- 172.21.15.5:0/670458687 learned_addr learned my addr 172.21.15.5:0/670458687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd109393640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd109b94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c8330 0x56417c4df2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44834/0 (socket says 172.21.15.5:44834) 2024-09-19T06:27:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd10a395640 1 -- 172.21.15.5:0/670458687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 msgr2=0x56417c4df7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd10a395640 1 --2- 172.21.15.5:0/670458687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4df7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd10a395640 1 -- 172.21.15.5:0/670458687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c8330 msgr2=0x56417c4df2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd10a395640 1 --2- 172.21.15.5:0/670458687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c8330 0x56417c4df2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd10a395640 1 -- 172.21.15.5:0/670458687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56417c555290 con 0x56417c4eb400 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd109b94640 1 --2- 172.21.15.5:0/670458687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c8330 0x56417c4df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd10a395640 1 --2- 172.21.15.5:0/670458687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4eb400 0x56417c4dfd30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc010010 tx=0x7fd0fc009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa62dad0768fc4bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.293+0000 7fd0ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd0ed7fa640 1 -- 172.21.15.5:0/670458687 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc004b20 con 0x56417c4eb400 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd10ec66740 1 -- 172.21.15.5:0/670458687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1000c4eb0 con 0x56417c4eb400 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd0ed7fa640 1 -- 172.21.15.5:0/670458687 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fd0fc004cc0 con 0x56417c4eb400 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd0ed7fa640 1 -- 172.21.15.5:0/670458687 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fd0fc010640 con 0x56417c4eb400 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd10ec66740 10 client.30625 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd10ec66740 1 -- 172.21.15.5:0/670458687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x56417c556590 con 0x56417c4eb400 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd0ed7fa640 1 -- 172.21.15.5:0/670458687 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1000c4eb0 con 0x56417c4eb400 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd0ed7fa640 1 -- 172.21.15.5:0/670458687 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fd0fc02b070 con 0x56417c4eb400 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd10ec66740 10 client.30625 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd10ec66740 10 client.30625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd10ec66740 20 client.30625 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd10ec66740 1 --2- 172.21.15.5:0/670458687 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56417c556d10 0x56417c559130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd10ec66740 4 client.30625 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:27:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.294+0000 7fd10ec66740 1 -- 172.21.15.5:0/670458687 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56417c433d70 con 0x56417c556d10 2024-09-19T06:27:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.295+0000 7fd109b94640 1 --2- 172.21.15.5:0/670458687 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56417c556d10 0x56417c559130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.295+0000 7fd109b94640 1 --2- 172.21.15.5:0/670458687 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56417c556d10 0x56417c559130 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff578d3a1c979a77 server_cookie=21ba5581999ea1d4 in_seq=0 out_seq=0 2024-09-19T06:27:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.295+0000 7fd0ed7fa640 10 client.30625 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.398+0000 7fd0ed7fa640 1 -- 172.21.15.5:0/670458687 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56417c433d70 con 0x56417c556d10 2024-09-19T06:27:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.398+0000 7fd0ed7fa640 10 client.30625 handle_command_reply: tid=0 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.398+0000 7fd10ec66740 1 client.30625 shutdown 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.398+0000 7fd10ec66740 1 -- 172.21.15.5:0/670458687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4eb400 msgr2=0x56417c4dfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.398+0000 7fd10ec66740 1 --2- 172.21.15.5:0/670458687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4eb400 0x56417c4dfd30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc010010 tx=0x7fd0fc009500 comp rx=0 tx=0).stop 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.399+0000 7fd10ec66740 1 -- 172.21.15.5:0/670458687 shutdown_connections 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.399+0000 7fd10ec66740 1 --2- 172.21.15.5:0/670458687 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56417c556d10 0x56417c559130 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.399+0000 7fd10ec66740 1 --2- 172.21.15.5:0/670458687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417c4eb400 0x56417c4dfd30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.399+0000 7fd0ed7fa640 0 client.30625 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.399+0000 7fd10ec66740 1 --2- 172.21.15.5:0/670458687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56417c4db4c0 0x56417c4df7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.399+0000 7fd10ec66740 1 --2- 172.21.15.5:0/670458687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56417c4c8330 0x56417c4df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.399+0000 7fd10ec66740 1 -- 172.21.15.5:0/670458687 >> 172.21.15.5:0/670458687 conn(0x56417c4c4060 msgr2=0x56417c4c31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.399+0000 7fd10ec66740 1 -- 172.21.15.5:0/670458687 shutdown_connections 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.399+0000 7fd10ec66740 1 -- 172.21.15.5:0/670458687 wait complete. 2024-09-19T06:27:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.399+0000 7fd10ec66740 20 client.30625 trim_cache size 0 max 16384 2024-09-19T06:27:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:27:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:27:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:27:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:27:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.400+0000 7fd10bdf8640 1 -- 172.21.15.5:0/2201017129 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0f00a3430 msgr2=0x7fd0f00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.400+0000 7fd10bdf8640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0f00a3430 0x7fd0f00a58f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009fd0 tx=0x7fd0fc004ab0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.400+0000 7fd10bdf8640 1 -- 172.21.15.5:0/2201017129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1041009a0 msgr2=0x7fd10419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.400+0000 7fd10bdf8640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1041009a0 0x7fd10419a520 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd100005940 tx=0x7fd1000040c0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.400+0000 7fd10bdf8640 1 -- 172.21.15.5:0/2201017129 shutdown_connections 2024-09-19T06:27:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.400+0000 7fd10bdf8640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1041069b0 0x7fd10419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.400+0000 7fd10bdf8640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd0f00a3430 0x7fd0f00a58f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.400+0000 7fd10bdf8640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd104101340 0x7fd10419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.400+0000 7fd10bdf8640 1 --2- 172.21.15.5:0/2201017129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1041009a0 0x7fd10419a520 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.400+0000 7fd10bdf8640 1 -- 172.21.15.5:0/2201017129 >> 172.21.15.5:0/2201017129 conn(0x7fd1040779a0 msgr2=0x7fd104103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.400+0000 7fd10bdf8640 1 -- 172.21.15.5:0/2201017129 shutdown_connections 2024-09-19T06:27:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:19.400+0000 7fd10bdf8640 1 -- 172.21.15.5:0/2201017129 wait complete. 2024-09-19T06:27:19.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 103405 8.56 MB/sec execute 523 sec latency 115.777 ms 2024-09-19T06:27:19.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:27:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:27:20.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 103534 8.55 MB/sec execute 524 sec latency 94.604 ms 2024-09-19T06:27:20.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:20 smithi005 ceph-mon[30955]: pgmap v759: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.9 MiB/s wr, 268 op/s 2024-09-19T06:27:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:20 smithi049 ceph-mon[34640]: pgmap v759: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.9 MiB/s wr, 268 op/s 2024-09-19T06:27:21.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:20 smithi157 ceph-mon[35346]: pgmap v759: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.9 MiB/s wr, 268 op/s 2024-09-19T06:27:21.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 103703 8.56 MB/sec execute 525 sec latency 118.685 ms 2024-09-19T06:27:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.998+0000 7f4b57a85640 1 -- 172.21.15.5:0/1894161354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b50102d30 msgr2=0x7f4b50103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.998+0000 7f4b57a85640 1 --2- 172.21.15.5:0/1894161354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b50102d30 0x7f4b50103110 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009930 tx=0x7f4b4c02a420 comp rx=0 tx=0).stop 2024-09-19T06:27:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.998+0000 7f4b57a85640 1 -- 172.21.15.5:0/1894161354 shutdown_connections 2024-09-19T06:27:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.998+0000 7f4b57a85640 1 --2- 172.21.15.5:0/1894161354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b501082a0 0x7f4b5010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.998+0000 7f4b57a85640 1 --2- 172.21.15.5:0/1894161354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b501036e0 0x7f4b50107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.998+0000 7f4b57a85640 1 --2- 172.21.15.5:0/1894161354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b50102d30 0x7f4b50103110 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.998+0000 7f4b57a85640 1 -- 172.21.15.5:0/1894161354 >> 172.21.15.5:0/1894161354 conn(0x7f4b500fc7d0 msgr2=0x7f4b500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.999+0000 7f4b57a85640 1 -- 172.21.15.5:0/1894161354 shutdown_connections 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.999+0000 7f4b57a85640 1 -- 172.21.15.5:0/1894161354 wait complete. 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.999+0000 7f4b57a85640 1 Processor -- start 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.999+0000 7f4b57a85640 1 -- start start 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.999+0000 7f4b57a85640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b50102d30 0x7f4b5019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.999+0000 7f4b57a85640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b501036e0 0x7f4b5019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.999+0000 7f4b57a85640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b501082a0 0x7f4b5019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.999+0000 7f4b57a85640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50111f50 con 0x7f4b501082a0 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.999+0000 7f4b57a85640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50111dd0 con 0x7f4b501036e0 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:20.999+0000 7f4b57a85640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501120d0 con 0x7f4b50102d30 2024-09-19T06:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b56022640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b501082a0 0x7f4b5019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b56022640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b501082a0 0x7f4b5019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44852/0 (socket says 172.21.15.5:44852) 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b56022640 1 -- 172.21.15.5:0/1260294766 learned_addr learned my addr 172.21.15.5:0/1260294766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b55821640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b50102d30 0x7f4b5019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b55020640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b501036e0 0x7f4b5019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b56022640 1 -- 172.21.15.5:0/1260294766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b50102d30 msgr2=0x7f4b5019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b56022640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b50102d30 0x7f4b5019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b56022640 1 -- 172.21.15.5:0/1260294766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b501036e0 msgr2=0x7f4b5019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b56022640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b501036e0 0x7f4b5019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b56022640 1 -- 172.21.15.5:0/1260294766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5019f6a0 con 0x7f4b501082a0 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b55821640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b50102d30 0x7f4b5019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b56022640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b501082a0 0x7f4b5019efc0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800eb10 tx=0x7f4b4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b46ffd640 1 -- 172.21.15.5:0/1260294766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48004b20 con 0x7f4b501082a0 2024-09-19T06:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b46ffd640 1 -- 172.21.15.5:0/1260294766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b48004cc0 con 0x7f4b501082a0 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b57a85640 1 -- 172.21.15.5:0/1260294766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5019f990 con 0x7f4b501082a0 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b46ffd640 1 -- 172.21.15.5:0/1260294766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48010690 con 0x7f4b501082a0 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.000+0000 7f4b57a85640 1 -- 172.21.15.5:0/1260294766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b501a7270 con 0x7f4b501082a0 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.003+0000 7f4b46ffd640 1 -- 172.21.15.5:0/1260294766 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4b48005020 con 0x7f4b501082a0 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.003+0000 7f4b59e5e740 1 Processor -- start 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.003+0000 7f4b59e5e740 1 -- start start 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.003+0000 7f4b59e5e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce1a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.003+0000 7f4b59e5e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce1a5d0 0x55d99ce1aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b46ffd640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4b3c0a33b0 0x7f4b3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b59e5e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 0x55d99ce27760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b59e5e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d99cd802f0 con 0x55d99ce25370 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b59e5e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d99cd27cf0 con 0x55d99ce1bb40 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b59e5e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d99cd7c2b0 con 0x55d99ce1a5d0 2024-09-19T06:27:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b56022640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 0x55d99ce27760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b55020640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4b3c0a33b0 0x7f4b3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b56022640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 0x55d99ce27760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44856/0 (socket says 172.21.15.5:44856) 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b56022640 1 -- 172.21.15.5:0/433040079 learned_addr learned my addr 172.21.15.5:0/433040079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b55821640 1 --2- 172.21.15.5:0/433040079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce1a5d0 0x55d99ce1aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b55020640 1 --2- 172.21.15.5:0/433040079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce1a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b46ffd640 1 -- 172.21.15.5:0/1260294766 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f4b480d0300 con 0x7f4b501082a0 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b56022640 1 -- 172.21.15.5:0/433040079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce1a5d0 msgr2=0x55d99ce1aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b56022640 1 --2- 172.21.15.5:0/433040079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce1a5d0 0x55d99ce1aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b56022640 1 -- 172.21.15.5:0/433040079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 msgr2=0x55d99ce1a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b56022640 1 --2- 172.21.15.5:0/433040079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce1a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b56022640 1 -- 172.21.15.5:0/433040079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99cd0c3a0 con 0x55d99ce25370 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b56022640 1 --2- 172.21.15.5:0/433040079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 0x55d99ce27760 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4b480049d0 tx=0x7f4b48009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5269b371eabc3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b55020640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4b3c0a33b0 0x7f4b3c0a5870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4b5010f0a0 tx=0x7f4b4000aa50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:21.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b3bfff640 1 -- 172.21.15.5:0/433040079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4809bf20 con 0x55d99ce25370 2024-09-19T06:27:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b3bfff640 1 -- 172.21.15.5:0/433040079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b480a60f0 con 0x55d99ce25370 2024-09-19T06:27:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.004+0000 7f4b3bfff640 1 -- 172.21.15.5:0/433040079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4809ccc0 con 0x55d99ce25370 2024-09-19T06:27:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.005+0000 7f4b59e5e740 1 -- 172.21.15.5:0/433040079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 msgr2=0x55d99ce27760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.005+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/433040079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 0x55d99ce27760 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4b480049d0 tx=0x7f4b48009a70 comp rx=0 tx=0).stop 2024-09-19T06:27:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.005+0000 7f4b59e5e740 1 -- 172.21.15.5:0/433040079 shutdown_connections 2024-09-19T06:27:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.005+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/433040079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 0x55d99ce27760 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.005+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/433040079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce1a5d0 0x55d99ce1aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.005+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/433040079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce1a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.005+0000 7f4b59e5e740 1 -- 172.21.15.5:0/433040079 >> 172.21.15.5:0/433040079 conn(0x55d99cde3310 msgr2=0x55d99ce16c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.005+0000 7f4b59e5e740 1 -- 172.21.15.5:0/433040079 shutdown_connections 2024-09-19T06:27:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.005+0000 7f4b59e5e740 1 -- 172.21.15.5:0/433040079 wait complete. 2024-09-19T06:27:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b59e5e740 1 -- start start 2024-09-19T06:27:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b59e5e740 1 Processor -- start 2024-09-19T06:27:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b59e5e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 0x55d99ce24d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b59e5e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce22010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b59e5e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce25370 0x55d99ce22600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b59e5e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d99ce28070 con 0x55d99ce1a5d0 2024-09-19T06:27:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b59e5e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d99cd3b060 con 0x55d99ce1bb40 2024-09-19T06:27:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b59e5e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d99ce281f0 con 0x55d99ce25370 2024-09-19T06:27:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b59e5e740 10 client.0 _ll_register_callbacks cb 0x55d99ca8e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b55821640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 0x55d99ce24d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b55821640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 0x55d99ce24d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44866/0 (socket says 172.21.15.5:44866) 2024-09-19T06:27:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b55821640 1 -- 172.21.15.5:0/3546791370 learned_addr learned my addr 172.21.15.5:0/3546791370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b56022640 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce25370 0x55d99ce22600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.006+0000 7f4b55020640 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce22010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b55821640 1 -- 172.21.15.5:0/3546791370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce25370 msgr2=0x55d99ce22600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b55821640 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce25370 0x55d99ce22600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b55821640 1 -- 172.21.15.5:0/3546791370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 msgr2=0x55d99ce22010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b55821640 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce22010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b55821640 1 -- 172.21.15.5:0/3546791370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99caf13c0 con 0x55d99ce1a5d0 2024-09-19T06:27:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b56022640 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce25370 0x55d99ce22600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b55821640 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 0x55d99ce24d60 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c02a940 tx=0x7f4b4c0317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35411f65b82207c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b397fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:27:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b397fa640 1 -- 172.21.15.5:0/3546791370 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c035040 con 0x55d99ce1a5d0 2024-09-19T06:27:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b397fa640 1 -- 172.21.15.5:0/3546791370 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f4b4c004240 con 0x55d99ce1a5d0 2024-09-19T06:27:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b59e5e740 1 -- 172.21.15.5:0/3546791370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b480a60f0 con 0x55d99ce1a5d0 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b397fa640 1 -- 172.21.15.5:0/3546791370 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f4b4c004440 con 0x55d99ce1a5d0 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b59e5e740 10 client.21336 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b59e5e740 1 -- 172.21.15.5:0/3546791370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55d99ce33b10 con 0x55d99ce1a5d0 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.007+0000 7f4b397fa640 1 -- 172.21.15.5:0/3546791370 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b480a60f0 con 0x55d99ce1a5d0 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.008+0000 7f4b397fa640 1 -- 172.21.15.5:0/3546791370 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f4b4c004b10 con 0x55d99ce1a5d0 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.008+0000 7f4b59e5e740 10 client.21336 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.008+0000 7f4b59e5e740 10 client.21336 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.008+0000 7f4b59e5e740 20 client.21336 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.008+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55d99ce34fd0 0x55d99ce37400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.008+0000 7f4b59e5e740 4 client.21336 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.008+0000 7f4b59e5e740 1 -- 172.21.15.5:0/3546791370 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d99cd8d860 con 0x55d99ce34fd0 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.008+0000 7f4b55020640 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55d99ce34fd0 0x55d99ce37400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.008+0000 7f4b397fa640 10 client.21336 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:27:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.008+0000 7f4b55020640 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55d99ce34fd0 0x55d99ce37400 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=606cd77a91bba410 server_cookie=cec21b78182dbd42 in_seq=0 out_seq=0 2024-09-19T06:27:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b397fa640 1 -- 172.21.15.5:0/3546791370 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d99cd8d860 con 0x55d99ce34fd0 2024-09-19T06:27:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b397fa640 10 client.21336 handle_command_reply: tid=0 2024-09-19T06:27:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b59e5e740 1 client.21336 shutdown 2024-09-19T06:27:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b59e5e740 1 -- 172.21.15.5:0/3546791370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 msgr2=0x55d99ce24d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 0x55d99ce24d60 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c02a940 tx=0x7f4b4c0317f0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b59e5e740 1 -- 172.21.15.5:0/3546791370 shutdown_connections 2024-09-19T06:27:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55d99ce34fd0 0x55d99ce37400 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b397fa640 0 client.21336 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:27:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce25370 0x55d99ce22600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce22010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/3546791370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 0x55d99ce24d60 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b59e5e740 1 -- 172.21.15.5:0/3546791370 >> 172.21.15.5:0/3546791370 conn(0x55d99cde3310 msgr2=0x55d99ce1c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b59e5e740 1 -- 172.21.15.5:0/3546791370 shutdown_connections 2024-09-19T06:27:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b59e5e740 1 -- 172.21.15.5:0/3546791370 wait complete. 2024-09-19T06:27:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.009+0000 7f4b59e5e740 20 client.21336 trim_cache size 0 max 16384 2024-09-19T06:27:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b59e5e740 1 Processor -- start 2024-09-19T06:27:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b59e5e740 1 -- start start 2024-09-19T06:27:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b59e5e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1a5d0 0x55d99ce31310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b59e5e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce1bb40 0x55d99ce218d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b59e5e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 0x55d99ce26760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b59e5e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501afae0 con 0x55d99ce25370 2024-09-19T06:27:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b59e5e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501b1620 con 0x55d99ce1a5d0 2024-09-19T06:27:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b59e5e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501b01b0 con 0x55d99ce1bb40 2024-09-19T06:27:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b56022640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 0x55d99ce26760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b55821640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1a5d0 0x55d99ce31310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b55821640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1a5d0 0x55d99ce31310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49102/0 (socket says 172.21.15.5:49102) 2024-09-19T06:27:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b55821640 1 -- 172.21.15.5:0/1713108167 learned_addr learned my addr 172.21.15.5:0/1713108167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b56022640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 0x55d99ce26760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44882/0 (socket says 172.21.15.5:44882) 2024-09-19T06:27:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b55020640 1 --2- 172.21.15.5:0/1713108167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce1bb40 0x55d99ce218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b55821640 1 -- 172.21.15.5:0/1713108167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce1bb40 msgr2=0x55d99ce218d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b55821640 1 --2- 172.21.15.5:0/1713108167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce1bb40 0x55d99ce218d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b55821640 1 -- 172.21.15.5:0/1713108167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 msgr2=0x55d99ce26760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b55821640 1 --2- 172.21.15.5:0/1713108167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 0x55d99ce26760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b55821640 1 -- 172.21.15.5:0/1713108167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99ce27f50 con 0x55d99ce1a5d0 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b56022640 1 --2- 172.21.15.5:0/1713108167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 0x55d99ce26760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.024+0000 7f4b55821640 1 --2- 172.21.15.5:0/1713108167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1a5d0 0x55d99ce31310 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c033000 tx=0x7f4b4c02a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=222e10a6f3d6a9fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.025+0000 7f4b3b7fe640 1 -- 172.21.15.5:0/1713108167 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c052070 con 0x55d99ce1a5d0 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.025+0000 7f4b3b7fe640 1 -- 172.21.15.5:0/1713108167 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b4c005000 con 0x55d99ce1a5d0 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.025+0000 7f4b3b7fe640 1 -- 172.21.15.5:0/1713108167 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c035040 con 0x55d99ce1a5d0 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.025+0000 7f4b59e5e740 1 -- 172.21.15.5:0/1713108167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1a5d0 msgr2=0x55d99ce31310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.025+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/1713108167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1a5d0 0x55d99ce31310 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c033000 tx=0x7f4b4c02a420 comp rx=0 tx=0).stop 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.025+0000 7f4b59e5e740 1 -- 172.21.15.5:0/1713108167 shutdown_connections 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.025+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/1713108167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce25370 0x55d99ce26760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.025+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/1713108167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce1bb40 0x55d99ce218d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.025+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/1713108167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1a5d0 0x55d99ce31310 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.025+0000 7f4b59e5e740 1 -- 172.21.15.5:0/1713108167 >> 172.21.15.5:0/1713108167 conn(0x55d99cde3310 msgr2=0x55d99ce208e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.025+0000 7f4b59e5e740 1 -- 172.21.15.5:0/1713108167 shutdown_connections 2024-09-19T06:27:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b59e5e740 1 -- 172.21.15.5:0/1713108167 wait complete. 2024-09-19T06:27:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b59e5e740 1 -- start start 2024-09-19T06:27:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b59e5e740 1 Processor -- start 2024-09-19T06:27:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b59e5e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 0x55d99ce1fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b59e5e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce2d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b59e5e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce25370 0x55d99ce2d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b59e5e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501b0f50 con 0x55d99ce1a5d0 2024-09-19T06:27:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b59e5e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501b0880 con 0x55d99ce1bb40 2024-09-19T06:27:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b59e5e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501adae0 con 0x55d99ce25370 2024-09-19T06:27:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b55821640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 0x55d99ce1fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b59e5e740 10 client.0 _ll_register_callbacks cb 0x7f4b4c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b55821640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 0x55d99ce1fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44898/0 (socket says 172.21.15.5:44898) 2024-09-19T06:27:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.026+0000 7f4b55821640 1 -- 172.21.15.5:0/2034409742 learned_addr learned my addr 172.21.15.5:0/2034409742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.027+0000 7f4b55020640 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce2d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.027+0000 7f4b56022640 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce25370 0x55d99ce2d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.027+0000 7f4b55821640 1 -- 172.21.15.5:0/2034409742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce25370 msgr2=0x55d99ce2d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.027+0000 7f4b55821640 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce25370 0x55d99ce2d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.027+0000 7f4b55821640 1 -- 172.21.15.5:0/2034409742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 msgr2=0x55d99ce2d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.027+0000 7f4b55821640 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce2d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.027+0000 7f4b55821640 1 -- 172.21.15.5:0/2034409742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99ce3c6c0 con 0x55d99ce1a5d0 2024-09-19T06:27:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.027+0000 7f4b55020640 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce2d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.027+0000 7f4b55821640 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 0x55d99ce1fad0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c004f30 tx=0x7f4b4c0053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b948e141da5fdc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.027+0000 7f4b397fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:27:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.027+0000 7f4b397fa640 1 -- 172.21.15.5:0/2034409742 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c035070 con 0x55d99ce1a5d0 2024-09-19T06:27:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.028+0000 7f4b397fa640 1 -- 172.21.15.5:0/2034409742 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f4b4c0046c0 con 0x55d99ce1a5d0 2024-09-19T06:27:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.028+0000 7f4b59e5e740 1 -- 172.21.15.5:0/2034409742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b4c005000 con 0x55d99ce1a5d0 2024-09-19T06:27:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.028+0000 7f4b397fa640 1 -- 172.21.15.5:0/2034409742 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f4b4c065050 con 0x55d99ce1a5d0 2024-09-19T06:27:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.028+0000 7f4b59e5e740 10 client.21342 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.028+0000 7f4b59e5e740 1 -- 172.21.15.5:0/2034409742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55d99ce9fd90 con 0x55d99ce1a5d0 2024-09-19T06:27:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.028+0000 7f4b397fa640 1 -- 172.21.15.5:0/2034409742 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b4c005000 con 0x55d99ce1a5d0 2024-09-19T06:27:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.028+0000 7f4b397fa640 1 -- 172.21.15.5:0/2034409742 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f4b4c004010 con 0x55d99ce1a5d0 2024-09-19T06:27:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.029+0000 7f4b59e5e740 10 client.21342 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.029+0000 7f4b59e5e740 10 client.21342 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:27:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.029+0000 7f4b59e5e740 20 client.21342 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.029+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55d99ce34fd0 0x55d99cea23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.029+0000 7f4b59e5e740 4 client.21342 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:27:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.029+0000 7f4b59e5e740 1 -- 172.21.15.5:0/2034409742 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d99cd87d70 con 0x55d99ce34fd0 2024-09-19T06:27:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.029+0000 7f4b55020640 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55d99ce34fd0 0x55d99cea23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.030+0000 7f4b397fa640 10 client.21342 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:27:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.030+0000 7f4b55020640 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55d99ce34fd0 0x55d99cea23a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=333abbaddd05858 server_cookie=f968e4a0c555108 in_seq=0 out_seq=0 2024-09-19T06:27:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.031+0000 7f4b397fa640 1 -- 172.21.15.5:0/2034409742 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d99cd87d70 con 0x55d99ce34fd0 2024-09-19T06:27:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.031+0000 7f4b397fa640 10 client.21342 handle_command_reply: tid=0 2024-09-19T06:27:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.031+0000 7f4b59e5e740 1 client.21342 shutdown 2024-09-19T06:27:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.031+0000 7f4b59e5e740 1 -- 172.21.15.5:0/2034409742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 msgr2=0x55d99ce1fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.031+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 0x55d99ce1fad0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c004f30 tx=0x7f4b4c0053e0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.032+0000 7f4b59e5e740 1 -- 172.21.15.5:0/2034409742 shutdown_connections 2024-09-19T06:27:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.032+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x55d99ce34fd0 0x55d99cea23a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.032+0000 7f4b397fa640 0 client.21342 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:27:21.624 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:27:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.032+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d99ce25370 0x55d99ce2d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.032+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d99ce1bb40 0x55d99ce2d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.032+0000 7f4b59e5e740 1 --2- 172.21.15.5:0/2034409742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99ce1a5d0 0x55d99ce1fad0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.032+0000 7f4b59e5e740 1 -- 172.21.15.5:0/2034409742 >> 172.21.15.5:0/2034409742 conn(0x55d99cde3310 msgr2=0x55d99ce154e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.032+0000 7f4b59e5e740 1 -- 172.21.15.5:0/2034409742 shutdown_connections 2024-09-19T06:27:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.032+0000 7f4b59e5e740 1 -- 172.21.15.5:0/2034409742 wait complete. 2024-09-19T06:27:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.032+0000 7f4b59e5e740 20 client.21342 trim_cache size 0 max 16384 2024-09-19T06:27:21.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.033+0000 7f4b57a85640 1 -- 172.21.15.5:0/1260294766 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4b3c0a33b0 msgr2=0x7f4b3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:21.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.033+0000 7f4b57a85640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4b3c0a33b0 0x7f4b3c0a5870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4b5010f0a0 tx=0x7f4b4000aa50 comp rx=0 tx=0).stop 2024-09-19T06:27:21.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.033+0000 7f4b57a85640 1 -- 172.21.15.5:0/1260294766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b501082a0 msgr2=0x7f4b5019efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:21.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.033+0000 7f4b57a85640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b501082a0 0x7f4b5019efc0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800eb10 tx=0x7f4b4800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.033+0000 7f4b57a85640 1 -- 172.21.15.5:0/1260294766 shutdown_connections 2024-09-19T06:27:21.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.033+0000 7f4b57a85640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b501082a0 0x7f4b5019efc0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.033+0000 7f4b57a85640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b501036e0 0x7f4b5019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.033+0000 7f4b57a85640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f4b3c0a33b0 0x7f4b3c0a5870 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.033+0000 7f4b57a85640 1 --2- 172.21.15.5:0/1260294766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b50102d30 0x7f4b5019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:21.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.033+0000 7f4b57a85640 1 -- 172.21.15.5:0/1260294766 >> 172.21.15.5:0/1260294766 conn(0x7f4b500fc7d0 msgr2=0x7f4b5010cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:21.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.033+0000 7f4b57a85640 1 -- 172.21.15.5:0/1260294766 shutdown_connections 2024-09-19T06:27:21.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:21.033+0000 7f4b57a85640 1 -- 172.21.15.5:0/1260294766 wait complete. 2024-09-19T06:27:22.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 103851 8.55 MB/sec execute 526 sec latency 100.602 ms 2024-09-19T06:27:22.853 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:22 smithi157 ceph-mon[35346]: pgmap v760: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 233 op/s 2024-09-19T06:27:22.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:22 smithi005 ceph-mon[30955]: pgmap v760: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 233 op/s 2024-09-19T06:27:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:22 smithi049 ceph-mon[34640]: pgmap v760: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 233 op/s 2024-09-19T06:27:23.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 103992 8.54 MB/sec execute 527 sec latency 102.443 ms 2024-09-19T06:27:24.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 104125 8.54 MB/sec execute 528 sec latency 135.641 ms 2024-09-19T06:27:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:24 smithi005 ceph-mon[30955]: pgmap v761: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.5 MiB/s wr, 382 op/s 2024-09-19T06:27:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:24 smithi049 ceph-mon[34640]: pgmap v761: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.5 MiB/s wr, 382 op/s 2024-09-19T06:27:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:24 smithi157 ceph-mon[35346]: pgmap v761: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.5 MiB/s wr, 382 op/s 2024-09-19T06:27:25.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 104223 8.53 MB/sec execute 529 sec latency 147.399 ms 2024-09-19T06:27:26.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 104340 8.52 MB/sec execute 530 sec latency 143.002 ms 2024-09-19T06:27:26.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:26 smithi005 ceph-mon[30955]: pgmap v762: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 265 op/s 2024-09-19T06:27:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:26 smithi049 ceph-mon[34640]: pgmap v762: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 265 op/s 2024-09-19T06:27:27.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:26 smithi157 ceph-mon[35346]: pgmap v762: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 265 op/s 2024-09-19T06:27:27.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 104483 8.53 MB/sec execute 531 sec latency 175.664 ms 2024-09-19T06:27:28.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:27:28.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 104659 8.52 MB/sec execute 532 sec latency 112.512 ms 2024-09-19T06:27:28.661 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:27:28.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:28 smithi005 ceph-mon[30955]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 363 op/s 2024-09-19T06:27:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:28 smithi049 ceph-mon[34640]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 363 op/s 2024-09-19T06:27:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:28 smithi157 ceph-mon[35346]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 363 op/s 2024-09-19T06:27:29.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 104833 8.53 MB/sec execute 533 sec latency 105.242 ms 2024-09-19T06:27:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.025+0000 7f5941936640 1 -- 172.21.15.5:0/39273429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c1082a0 msgr2=0x7f593c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.025+0000 7f5941936640 1 --2- 172.21.15.5:0/39273429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c1082a0 0x7f593c10ec40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f5934009c10 tx=0x7f593402a6f0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.025+0000 7f5941936640 1 -- 172.21.15.5:0/39273429 shutdown_connections 2024-09-19T06:27:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.025+0000 7f5941936640 1 --2- 172.21.15.5:0/39273429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c1082a0 0x7f593c10ec40 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.025+0000 7f5941936640 1 --2- 172.21.15.5:0/39273429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f593c1036e0 0x7f593c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.025+0000 7f5941936640 1 --2- 172.21.15.5:0/39273429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f593c102d30 0x7f593c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.025+0000 7f5941936640 1 -- 172.21.15.5:0/39273429 >> 172.21.15.5:0/39273429 conn(0x7f593c0fc7d0 msgr2=0x7f593c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.025+0000 7f5941936640 1 -- 172.21.15.5:0/39273429 shutdown_connections 2024-09-19T06:27:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.025+0000 7f5941936640 1 -- 172.21.15.5:0/39273429 wait complete. 2024-09-19T06:27:30.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.026+0000 7f5941936640 1 Processor -- start 2024-09-19T06:27:30.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.026+0000 7f5941936640 1 -- start start 2024-09-19T06:27:30.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f5941936640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c102d30 0x7f593c1023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f5941936640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f593c1036e0 0x7f593c100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f5941936640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f593c1082a0 0x7f593c100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f5941936640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f593c111f50 con 0x7f593c102d30 2024-09-19T06:27:30.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f5941936640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f593c111dd0 con 0x7f593c1082a0 2024-09-19T06:27:30.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f5941936640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1120d0 con 0x7f593c1036e0 2024-09-19T06:27:30.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f593a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f593c1036e0 0x7f593c100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f593a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f593c1036e0 0x7f593c100940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60792/0 (socket says 172.21.15.5:60792) 2024-09-19T06:27:30.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f593a7fc640 1 -- 172.21.15.5:0/3139702952 learned_addr learned my addr 172.21.15.5:0/3139702952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:30.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f593affd640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c102d30 0x7f593c1023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f593b7fe640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f593c1082a0 0x7f593c100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f593a7fc640 1 -- 172.21.15.5:0/3139702952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f593c1082a0 msgr2=0x7f593c100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:30.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f593a7fc640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f593c1082a0 0x7f593c100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f593a7fc640 1 -- 172.21.15.5:0/3139702952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c102d30 msgr2=0x7f593c1023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:30.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f593a7fc640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c102d30 0x7f593c1023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.027+0000 7f593a7fc640 1 -- 172.21.15.5:0/3139702952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c101600 con 0x7f593c1036e0 2024-09-19T06:27:30.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.028+0000 7f593a7fc640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f593c1036e0 0x7f593c100940 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f59300058c0 tx=0x7f593000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:30.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.028+0000 7f5940934640 1 -- 172.21.15.5:0/3139702952 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593000cd30 con 0x7f593c1036e0 2024-09-19T06:27:30.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.028+0000 7f5940934640 1 -- 172.21.15.5:0/3139702952 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59300035f0 con 0x7f593c1036e0 2024-09-19T06:27:30.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.028+0000 7f5941936640 1 -- 172.21.15.5:0/3139702952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c1af450 con 0x7f593c1036e0 2024-09-19T06:27:30.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.028+0000 7f5940934640 1 -- 172.21.15.5:0/3139702952 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59300044e0 con 0x7f593c1036e0 2024-09-19T06:27:30.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.028+0000 7f5941936640 1 -- 172.21.15.5:0/3139702952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c1af930 con 0x7f593c1036e0 2024-09-19T06:27:30.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.032+0000 7f5940934640 1 -- 172.21.15.5:0/3139702952 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5930004000 con 0x7f593c1036e0 2024-09-19T06:27:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.033+0000 7f5943d0f740 1 Processor -- start 2024-09-19T06:27:30.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.033+0000 7f5943d0f740 1 -- start start 2024-09-19T06:27:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.033+0000 7f5943d0f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896ad8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.033+0000 7f5943d0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ad85d0 0x55b896ad8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.033+0000 7f5943d0f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ae3370 0x55b896ae5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.033+0000 7f5943d0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b896a3e2f0 con 0x55b896ad85d0 2024-09-19T06:27:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.033+0000 7f5943d0f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b8969e5cf0 con 0x55b896ae3370 2024-09-19T06:27:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.033+0000 7f5943d0f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b896a3a2b0 con 0x55b896ad9b40 2024-09-19T06:27:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.034+0000 7f593b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896ad8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.034+0000 7f593b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896ad8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60806/0 (socket says 172.21.15.5:60806) 2024-09-19T06:27:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.034+0000 7f593b7fe640 1 -- 172.21.15.5:0/2767266304 learned_addr learned my addr 172.21.15.5:0/2767266304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.034+0000 7f593a7fc640 1 --2- 172.21.15.5:0/2767266304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ae3370 0x55b896ae5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.034+0000 7f593affd640 1 --2- 172.21.15.5:0/2767266304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ad85d0 0x55b896ad8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.034+0000 7f593b7fe640 1 -- 172.21.15.5:0/2767266304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ae3370 msgr2=0x55b896ae5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:30.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.034+0000 7f593b7fe640 1 --2- 172.21.15.5:0/2767266304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ae3370 0x55b896ae5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.034+0000 7f593b7fe640 1 -- 172.21.15.5:0/2767266304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ad85d0 msgr2=0x55b896ad8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:30.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.034+0000 7f593b7fe640 1 --2- 172.21.15.5:0/2767266304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ad85d0 0x55b896ad8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.034+0000 7f593b7fe640 1 -- 172.21.15.5:0/2767266304 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8969ca3a0 con 0x55b896ad9b40 2024-09-19T06:27:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.035+0000 7f593b7fe640 1 --2- 172.21.15.5:0/2767266304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896ad8090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5934033000 tx=0x7f59340314c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d60675ed2c3de3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.035+0000 7f5940934640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f59240a3430 0x7f59240a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.035+0000 7f5921ffb640 1 -- 172.21.15.5:0/2767266304 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934041070 con 0x55b896ad9b40 2024-09-19T06:27:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.035+0000 7f593affd640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f59240a3430 0x7f59240a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.035+0000 7f5921ffb640 1 -- 172.21.15.5:0/2767266304 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5934003e20 con 0x55b896ad9b40 2024-09-19T06:27:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.035+0000 7f5921ffb640 1 -- 172.21.15.5:0/2767266304 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934036030 con 0x55b896ad9b40 2024-09-19T06:27:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.035+0000 7f5940934640 1 -- 172.21.15.5:0/3139702952 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f59300d00c0 con 0x7f593c1036e0 2024-09-19T06:27:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.036+0000 7f5943d0f740 1 -- 172.21.15.5:0/2767266304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 msgr2=0x55b896ad8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.036+0000 7f5943d0f740 1 --2- 172.21.15.5:0/2767266304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896ad8090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5934033000 tx=0x7f59340314c0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.036+0000 7f593affd640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f59240a3430 0x7f59240a58f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f592c005fd0 tx=0x7f592c00b650 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.036+0000 7f5943d0f740 1 -- 172.21.15.5:0/2767266304 shutdown_connections 2024-09-19T06:27:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.036+0000 7f5943d0f740 1 --2- 172.21.15.5:0/2767266304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ae3370 0x55b896ae5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.036+0000 7f5943d0f740 1 --2- 172.21.15.5:0/2767266304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ad85d0 0x55b896ad8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.036+0000 7f5943d0f740 1 --2- 172.21.15.5:0/2767266304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896ad8090 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.036+0000 7f5943d0f740 1 -- 172.21.15.5:0/2767266304 >> 172.21.15.5:0/2767266304 conn(0x55b896aa1310 msgr2=0x55b896ad4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.037+0000 7f5943d0f740 1 -- 172.21.15.5:0/2767266304 shutdown_connections 2024-09-19T06:27:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.037+0000 7f5943d0f740 1 -- 172.21.15.5:0/2767266304 wait complete. 2024-09-19T06:27:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.037+0000 7f5943d0f740 1 -- start start 2024-09-19T06:27:30.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.038+0000 7f5943d0f740 1 Processor -- start 2024-09-19T06:27:30.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.038+0000 7f5943d0f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 0x55b896ae2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.039+0000 7f5943d0f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896adff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.039+0000 7f5943d0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ae3370 0x55b896ae04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.039+0000 7f593affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 0x55b896ae2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.039+0000 7f593affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 0x55b896ae2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58164/0 (socket says 172.21.15.5:58164) 2024-09-19T06:27:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.039+0000 7f593affd640 1 -- 172.21.15.5:0/4175012104 learned_addr learned my addr 172.21.15.5:0/4175012104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.039+0000 7f593b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ae3370 0x55b896ae04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.039+0000 7f5943d0f740 1 -- 172.21.15.5:0/4175012104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b896ae6010 con 0x55b896ae3370 2024-09-19T06:27:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.039+0000 7f593a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896adff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.039+0000 7f5943d0f740 1 -- 172.21.15.5:0/4175012104 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b8969f9060 con 0x55b896ad85d0 2024-09-19T06:27:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.039+0000 7f5943d0f740 1 -- 172.21.15.5:0/4175012104 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b896ae6190 con 0x55b896ad9b40 2024-09-19T06:27:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.039+0000 7f5943d0f740 10 client.0 _ll_register_callbacks cb 0x55b89674c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.040+0000 7f593affd640 1 -- 172.21.15.5:0/4175012104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 msgr2=0x55b896adff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.040+0000 7f593affd640 1 --2- 172.21.15.5:0/4175012104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896adff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.040+0000 7f593affd640 1 -- 172.21.15.5:0/4175012104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ae3370 msgr2=0x55b896ae04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.040+0000 7f593affd640 1 --2- 172.21.15.5:0/4175012104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ae3370 0x55b896ae04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.040+0000 7f593affd640 1 -- 172.21.15.5:0/4175012104 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8967af3c0 con 0x55b896ad85d0 2024-09-19T06:27:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.041+0000 7f593affd640 1 --2- 172.21.15.5:0/4175012104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 0x55b896ae2ce0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f592c0126d0 tx=0x7f592c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebbbee514f4a1a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.041+0000 7f591affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:27:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.041+0000 7f591affd640 1 -- 172.21.15.5:0/4175012104 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c01f8f0 con 0x55b896ad85d0 2024-09-19T06:27:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.041+0000 7f591affd640 1 -- 172.21.15.5:0/4175012104 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f592c01fa90 con 0x55b896ad85d0 2024-09-19T06:27:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.041+0000 7f5943d0f740 1 -- 172.21.15.5:0/4175012104 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5934003e20 con 0x55b896ad85d0 2024-09-19T06:27:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.041+0000 7f591affd640 1 -- 172.21.15.5:0/4175012104 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f592c01fc90 con 0x55b896ad85d0 2024-09-19T06:27:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.041+0000 7f5943d0f740 10 client.30716 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.041+0000 7f5943d0f740 1 -- 172.21.15.5:0/4175012104 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55b896afa2c0 con 0x55b896ad85d0 2024-09-19T06:27:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.042+0000 7f591affd640 1 -- 172.21.15.5:0/4175012104 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5934003e20 con 0x55b896ad85d0 2024-09-19T06:27:30.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.042+0000 7f591affd640 1 -- 172.21.15.5:0/4175012104 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f592c02f3d0 con 0x55b896ad85d0 2024-09-19T06:27:30.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.042+0000 7f5943d0f740 10 client.30716 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:30.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.042+0000 7f5943d0f740 10 client.30716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:30.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.042+0000 7f5943d0f740 20 client.30716 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:30.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.042+0000 7f5943d0f740 1 --2- 172.21.15.5:0/4175012104 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b896afb780 0x55b896afdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.042+0000 7f5943d0f740 4 client.30716 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:30.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.042+0000 7f5943d0f740 1 -- 172.21.15.5:0/4175012104 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b896a4b860 con 0x55b896afb780 2024-09-19T06:27:30.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.043+0000 7f593a7fc640 1 --2- 172.21.15.5:0/4175012104 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b896afb780 0x55b896afdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.043+0000 7f593a7fc640 1 --2- 172.21.15.5:0/4175012104 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b896afb780 0x55b896afdb70 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ad60d20aabd2a6 server_cookie=b22bcfacebf252f5 in_seq=0 out_seq=0 2024-09-19T06:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.043+0000 7f591affd640 10 client.30716 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.044+0000 7f591affd640 1 -- 172.21.15.5:0/4175012104 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b896a4b860 con 0x55b896afb780 2024-09-19T06:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.044+0000 7f591affd640 10 client.30716 handle_command_reply: tid=0 2024-09-19T06:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.044+0000 7f5943d0f740 1 client.30716 shutdown 2024-09-19T06:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.044+0000 7f5943d0f740 1 -- 172.21.15.5:0/4175012104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 msgr2=0x55b896ae2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.044+0000 7f5943d0f740 1 --2- 172.21.15.5:0/4175012104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 0x55b896ae2ce0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f592c0126d0 tx=0x7f592c0046c0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.044+0000 7f5943d0f740 1 -- 172.21.15.5:0/4175012104 shutdown_connections 2024-09-19T06:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.044+0000 7f5943d0f740 1 --2- 172.21.15.5:0/4175012104 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b896afb780 0x55b896afdb70 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.044+0000 7f5943d0f740 1 --2- 172.21.15.5:0/4175012104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ae3370 0x55b896ae04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.044+0000 7f5943d0f740 1 --2- 172.21.15.5:0/4175012104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896adff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.044+0000 7f5943d0f740 1 --2- 172.21.15.5:0/4175012104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 0x55b896ae2ce0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.045+0000 7f5943d0f740 1 -- 172.21.15.5:0/4175012104 >> 172.21.15.5:0/4175012104 conn(0x55b896aa1310 msgr2=0x55b896ada960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.045+0000 7f591affd640 0 client.30716 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.045+0000 7f5943d0f740 1 -- 172.21.15.5:0/4175012104 shutdown_connections 2024-09-19T06:27:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.045+0000 7f5943d0f740 1 -- 172.21.15.5:0/4175012104 wait complete. 2024-09-19T06:27:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.045+0000 7f5943d0f740 20 client.30716 trim_cache size 0 max 16384 2024-09-19T06:27:30.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.065+0000 7f5943d0f740 1 Processor -- start 2024-09-19T06:27:30.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.065+0000 7f5943d0f740 1 -- start start 2024-09-19T06:27:30.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f5943d0f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 0x55b896acfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f5943d0f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896af7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f5943d0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ae3370 0x55b896b03420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f593affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 0x55b896acfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f593affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 0x55b896acfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58176/0 (socket says 172.21.15.5:58176) 2024-09-19T06:27:30.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f593affd640 1 -- 172.21.15.5:0/1598302514 learned_addr learned my addr 172.21.15.5:0/1598302514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:30.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f5943d0f740 1 -- 172.21.15.5:0/1598302514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1b3fd0 con 0x55b896ae3370 2024-09-19T06:27:30.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f5943d0f740 1 -- 172.21.15.5:0/1598302514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1b5b10 con 0x55b896ad85d0 2024-09-19T06:27:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f5943d0f740 1 -- 172.21.15.5:0/1598302514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1b46a0 con 0x55b896ad9b40 2024-09-19T06:27:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f593a7fc640 1 --2- 172.21.15.5:0/1598302514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896af7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f593b7fe640 1 --2- 172.21.15.5:0/1598302514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ae3370 0x55b896b03420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f593affd640 1 -- 172.21.15.5:0/1598302514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 msgr2=0x55b896af7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f593affd640 1 --2- 172.21.15.5:0/1598302514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896af7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f593affd640 1 -- 172.21.15.5:0/1598302514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ae3370 msgr2=0x55b896b03420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f593affd640 1 --2- 172.21.15.5:0/1598302514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ae3370 0x55b896b03420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.066+0000 7f593affd640 1 -- 172.21.15.5:0/1598302514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b896ad5dd0 con 0x55b896ad85d0 2024-09-19T06:27:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.067+0000 7f593affd640 1 --2- 172.21.15.5:0/1598302514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 0x55b896acfc60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f592c02f7d0 tx=0x7f592c0059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36c986d6f66dd63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.067+0000 7f59217fa640 1 -- 172.21.15.5:0/1598302514 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c034070 con 0x55b896ad85d0 2024-09-19T06:27:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.067+0000 7f59217fa640 1 -- 172.21.15.5:0/1598302514 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f592c035040 con 0x55b896ad85d0 2024-09-19T06:27:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.067+0000 7f59217fa640 1 -- 172.21.15.5:0/1598302514 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c02fcb0 con 0x55b896ad85d0 2024-09-19T06:27:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.069+0000 7f5943d0f740 1 -- 172.21.15.5:0/1598302514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 msgr2=0x55b896acfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.069+0000 7f5943d0f740 1 --2- 172.21.15.5:0/1598302514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 0x55b896acfc60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f592c02f7d0 tx=0x7f592c0059b0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.069+0000 7f5943d0f740 1 -- 172.21.15.5:0/1598302514 shutdown_connections 2024-09-19T06:27:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.069+0000 7f5943d0f740 1 --2- 172.21.15.5:0/1598302514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ae3370 0x55b896b03420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.069+0000 7f5943d0f740 1 --2- 172.21.15.5:0/1598302514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896af7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.069+0000 7f5943d0f740 1 --2- 172.21.15.5:0/1598302514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ad85d0 0x55b896acfc60 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.069+0000 7f5943d0f740 1 -- 172.21.15.5:0/1598302514 >> 172.21.15.5:0/1598302514 conn(0x55b896aa1310 msgr2=0x55b896adb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.069+0000 7f5943d0f740 1 -- 172.21.15.5:0/1598302514 shutdown_connections 2024-09-19T06:27:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.070+0000 7f5943d0f740 1 -- 172.21.15.5:0/1598302514 wait complete. 2024-09-19T06:27:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.070+0000 7f5943d0f740 1 -- start start 2024-09-19T06:27:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.071+0000 7f5943d0f740 1 Processor -- start 2024-09-19T06:27:30.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.071+0000 7f5943d0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ad85d0 0x55b896af3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.071+0000 7f5943d0f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896af41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.071+0000 7f5943d0f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ae3370 0x55b896af0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.071+0000 7f5943d0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1b5440 con 0x55b896ad85d0 2024-09-19T06:27:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.071+0000 7f5943d0f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1b4d70 con 0x55b896ae3370 2024-09-19T06:27:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.071+0000 7f5943d0f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1b1fd0 con 0x55b896ad9b40 2024-09-19T06:27:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.071+0000 7f593affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ad85d0 0x55b896af3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.071+0000 7f593affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ad85d0 0x55b896af3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49908/0 (socket says 172.21.15.5:49908) 2024-09-19T06:27:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.071+0000 7f5943d0f740 10 client.0 _ll_register_callbacks cb 0x7f592c00b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.071+0000 7f593affd640 1 -- 172.21.15.5:0/3317628078 learned_addr learned my addr 172.21.15.5:0/3317628078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.072+0000 7f593b7fe640 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ae3370 0x55b896af0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.072+0000 7f593a7fc640 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896af41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.072+0000 7f593affd640 1 -- 172.21.15.5:0/3317628078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 msgr2=0x55b896af41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.072+0000 7f593affd640 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896af41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.072+0000 7f593affd640 1 -- 172.21.15.5:0/3317628078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ae3370 msgr2=0x55b896af0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.072+0000 7f593affd640 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ae3370 0x55b896af0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.072+0000 7f593affd640 1 -- 172.21.15.5:0/3317628078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b896adb120 con 0x55b896ad85d0 2024-09-19T06:27:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.072+0000 7f593affd640 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ad85d0 0x55b896af3cb0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f592c02f870 tx=0x7f592c01f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9a1cc2ed4779ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.072+0000 7f591affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:27:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.073+0000 7f591affd640 1 -- 172.21.15.5:0/3317628078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c034070 con 0x55b896ad85d0 2024-09-19T06:27:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.073+0000 7f5943d0f740 1 -- 172.21.15.5:0/3317628078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f592c035040 con 0x55b896ad85d0 2024-09-19T06:27:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.073+0000 7f591affd640 1 -- 172.21.15.5:0/3317628078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f592c02fb40 con 0x55b896ad85d0 2024-09-19T06:27:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.073+0000 7f591affd640 1 -- 172.21.15.5:0/3317628078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f592c03d040 con 0x55b896ad85d0 2024-09-19T06:27:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.073+0000 7f5943d0f740 10 client.21360 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.073+0000 7f5943d0f740 1 -- 172.21.15.5:0/3317628078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55b896ad5960 con 0x55b896ad85d0 2024-09-19T06:27:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.073+0000 7f591affd640 1 -- 172.21.15.5:0/3317628078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f592c035040 con 0x55b896ad85d0 2024-09-19T06:27:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.074+0000 7f591affd640 1 -- 172.21.15.5:0/3317628078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f592c004900 con 0x55b896ad85d0 2024-09-19T06:27:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.074+0000 7f5943d0f740 10 client.21360 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.074+0000 7f5943d0f740 10 client.21360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.074+0000 7f5943d0f740 20 client.21360 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.074+0000 7f5943d0f740 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b896b66eb0 0x55b896b692a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:30.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.074+0000 7f5943d0f740 4 client.21360 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:27:30.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.074+0000 7f5943d0f740 1 -- 172.21.15.5:0/3317628078 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b896a45d70 con 0x55b896b66eb0 2024-09-19T06:27:30.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.074+0000 7f593a7fc640 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b896b66eb0 0x55b896b692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:30.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f591affd640 10 client.21360 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:30.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f593a7fc640 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b896b66eb0 0x55b896b692a0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea073d8f5acfc93 server_cookie=d4d58003c63c77f6 in_seq=0 out_seq=0 2024-09-19T06:27:30.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f591affd640 1 -- 172.21.15.5:0/3317628078 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b896a45d70 con 0x55b896b66eb0 2024-09-19T06:27:30.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f591affd640 10 client.21360 handle_command_reply: tid=0 2024-09-19T06:27:30.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f5943d0f740 1 client.21360 shutdown 2024-09-19T06:27:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f5943d0f740 1 -- 172.21.15.5:0/3317628078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ad85d0 msgr2=0x55b896af3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f5943d0f740 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ad85d0 0x55b896af3cb0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f592c02f870 tx=0x7f592c01f920 comp rx=0 tx=0).stop 2024-09-19T06:27:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f5943d0f740 1 -- 172.21.15.5:0/3317628078 shutdown_connections 2024-09-19T06:27:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f5943d0f740 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b896b66eb0 0x55b896b692a0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f591affd640 0 client.21360 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f5943d0f740 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b896ae3370 0x55b896af0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f5943d0f740 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b896ad9b40 0x55b896af41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f5943d0f740 1 --2- 172.21.15.5:0/3317628078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b896ad85d0 0x55b896af3cb0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.075+0000 7f5943d0f740 1 -- 172.21.15.5:0/3317628078 >> 172.21.15.5:0/3317628078 conn(0x55b896aa1310 msgr2=0x55b896ae17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.076+0000 7f5943d0f740 1 -- 172.21.15.5:0/3317628078 shutdown_connections 2024-09-19T06:27:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.076+0000 7f5943d0f740 1 -- 172.21.15.5:0/3317628078 wait complete. 2024-09-19T06:27:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.076+0000 7f5943d0f740 20 client.21360 trim_cache size 0 max 16384 2024-09-19T06:27:30.082 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:27:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.077+0000 7f5941936640 1 -- 172.21.15.5:0/3139702952 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f59240a3430 msgr2=0x7f59240a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.077+0000 7f5941936640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f59240a3430 0x7f59240a58f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f592c005fd0 tx=0x7f592c00b650 comp rx=0 tx=0).stop 2024-09-19T06:27:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.077+0000 7f5941936640 1 -- 172.21.15.5:0/3139702952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f593c1036e0 msgr2=0x7f593c100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.077+0000 7f5941936640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f593c1036e0 0x7f593c100940 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f59300058c0 tx=0x7f593000ee50 comp rx=0 tx=0).stop 2024-09-19T06:27:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.077+0000 7f5941936640 1 -- 172.21.15.5:0/3139702952 shutdown_connections 2024-09-19T06:27:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.077+0000 7f5941936640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f593c1082a0 0x7f593c100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.077+0000 7f5941936640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f59240a3430 0x7f59240a58f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.077+0000 7f5941936640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f593c1036e0 0x7f593c100940 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.077+0000 7f5941936640 1 --2- 172.21.15.5:0/3139702952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c102d30 0x7f593c1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.077+0000 7f5941936640 1 -- 172.21.15.5:0/3139702952 >> 172.21.15.5:0/3139702952 conn(0x7f593c0fc7d0 msgr2=0x7f593c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.077+0000 7f5941936640 1 -- 172.21.15.5:0/3139702952 shutdown_connections 2024-09-19T06:27:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:30.078+0000 7f5941936640 1 -- 172.21.15.5:0/3139702952 wait complete. 2024-09-19T06:27:30.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 104972 8.53 MB/sec execute 534 sec latency 76.729 ms 2024-09-19T06:27:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:30 smithi005 ceph-mon[30955]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 247 op/s 2024-09-19T06:27:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:30 smithi049 ceph-mon[34640]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 247 op/s 2024-09-19T06:27:31.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:30 smithi157 ceph-mon[35346]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 247 op/s 2024-09-19T06:27:31.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 105136 8.53 MB/sec execute 535 sec latency 242.608 ms 2024-09-19T06:27:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.527+0000 7fc5fe48c640 1 -- 172.21.15.5:0/1280524628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f81011c0 msgr2=0x7fc5f810df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.527+0000 7fc5fe48c640 1 --2- 172.21.15.5:0/1280524628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f81011c0 0x7fc5f810df10 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009900 tx=0x7fc5ec02a480 comp rx=0 tx=0).stop 2024-09-19T06:27:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.528+0000 7fc5fe48c640 1 -- 172.21.15.5:0/1280524628 shutdown_connections 2024-09-19T06:27:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.528+0000 7fc5fe48c640 1 --2- 172.21.15.5:0/1280524628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f810e450 0x7fc5f8110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.528+0000 7fc5fe48c640 1 --2- 172.21.15.5:0/1280524628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f81011c0 0x7fc5f810df10 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.528+0000 7fc5fe48c640 1 --2- 172.21.15.5:0/1280524628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5f81008a0 0x7fc5f8100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.528+0000 7fc5fe48c640 1 -- 172.21.15.5:0/1280524628 >> 172.21.15.5:0/1280524628 conn(0x7fc5f80779a0 msgr2=0x7fc5f80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.528+0000 7fc5fe48c640 1 -- 172.21.15.5:0/1280524628 shutdown_connections 2024-09-19T06:27:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.528+0000 7fc5fe48c640 1 -- 172.21.15.5:0/1280524628 wait complete. 2024-09-19T06:27:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.529+0000 7fc5fe48c640 1 Processor -- start 2024-09-19T06:27:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.529+0000 7fc5fe48c640 1 -- start start 2024-09-19T06:27:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.529+0000 7fc5fe48c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5f81008a0 0x7fc5f819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.529+0000 7fc5fe48c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f81011c0 0x7fc5f819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.529+0000 7fc5fe48c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f810e450 0x7fc5f819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.529+0000 7fc5fe48c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f8113fe0 con 0x7fc5f810e450 2024-09-19T06:27:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.529+0000 7fc5fe48c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f8113e60 con 0x7fc5f81011c0 2024-09-19T06:27:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.529+0000 7fc5fe48c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f8114160 con 0x7fc5f81008a0 2024-09-19T06:27:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f81011c0 0x7fc5f819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f81011c0 0x7fc5f819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58196/0 (socket says 172.21.15.5:58196) 2024-09-19T06:27:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f77fe640 1 -- 172.21.15.5:0/731408149 learned_addr learned my addr 172.21.15.5:0/731408149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5f81008a0 0x7fc5f819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5fca29640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f810e450 0x7fc5f819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f77fe640 1 -- 172.21.15.5:0/731408149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5f81008a0 msgr2=0x7fc5f819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f77fe640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5f81008a0 0x7fc5f819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f77fe640 1 -- 172.21.15.5:0/731408149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f810e450 msgr2=0x7fc5f819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f77fe640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f810e450 0x7fc5f819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f77fe640 1 -- 172.21.15.5:0/731408149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f819f5a0 con 0x7fc5f81011c0 2024-09-19T06:27:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f77fe640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f81011c0 0x7fc5f819aaf0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec031440 tx=0x7fc5ec003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f57fa640 1 -- 172.21.15.5:0/731408149 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec03f070 con 0x7fc5f81011c0 2024-09-19T06:27:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f57fa640 1 -- 172.21.15.5:0/731408149 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5ec031b50 con 0x7fc5f81011c0 2024-09-19T06:27:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5fe48c640 1 -- 172.21.15.5:0/731408149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f819f830 con 0x7fc5f81011c0 2024-09-19T06:27:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.530+0000 7fc5f57fa640 1 -- 172.21.15.5:0/731408149 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec035030 con 0x7fc5f81011c0 2024-09-19T06:27:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.531+0000 7fc5fe48c640 1 -- 172.21.15.5:0/731408149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f81a7120 con 0x7fc5f81011c0 2024-09-19T06:27:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.534+0000 7fc5f57fa640 1 -- 172.21.15.5:0/731408149 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc5ec042070 con 0x7fc5f81011c0 2024-09-19T06:27:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.534+0000 7fc601225740 1 Processor -- start 2024-09-19T06:27:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.534+0000 7fc601225740 1 -- start start 2024-09-19T06:27:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc601225740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bafaf0 0x55b767bafed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc601225740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bb0410 0x55b767bba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc601225740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bbd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc5f57fa640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc5e00a3400 0x7fc5e00a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc601225740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b767b142f0 con 0x55b767bbacb0 2024-09-19T06:27:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc601225740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b767abc270 con 0x55b767bafaf0 2024-09-19T06:27:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc601225740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b767b102b0 con 0x55b767bb0410 2024-09-19T06:27:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc5f77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bbd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc5f7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bb0410 0x55b767bba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc5f7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bb0410 0x55b767bba770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60850/0 (socket says 172.21.15.5:60850) 2024-09-19T06:27:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc5f7fff640 1 -- 172.21.15.5:0/3338897832 learned_addr learned my addr 172.21.15.5:0/3338897832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc5fca29640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc5e00a3400 0x7fc5e00a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc5f7fff640 1 -- 172.21.15.5:0/3338897832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bafaf0 msgr2=0x55b767bafed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc5f7fff640 1 --2- 172.21.15.5:0/3338897832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bafaf0 0x55b767bafed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.535+0000 7fc5f7fff640 1 -- 172.21.15.5:0/3338897832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 msgr2=0x55b767bbd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.536+0000 7fc5f7fff640 1 --2- 172.21.15.5:0/3338897832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bbd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.536+0000 7fc5f57fa640 1 -- 172.21.15.5:0/731408149 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fc5ec0f0040 con 0x7fc5f81011c0 2024-09-19T06:27:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.536+0000 7fc5f7fff640 1 -- 172.21.15.5:0/3338897832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b767abb340 con 0x55b767bb0410 2024-09-19T06:27:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.536+0000 7fc5fca29640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc5e00a3400 0x7fc5e00a58c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8074c90 tx=0x7fc5f0009b70 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.536+0000 7fc5f77fe640 1 --2- 172.21.15.5:0/3338897832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bbd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.536+0000 7fc5f7fff640 1 --2- 172.21.15.5:0/3338897832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bb0410 0x55b767bba770 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009cf0 tx=0x7fc5e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13484d5d127d4950 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.536+0000 7fc5de7fc640 1 -- 172.21.15.5:0/3338897832 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8004230 con 0x55b767bb0410 2024-09-19T06:27:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.537+0000 7fc5de7fc640 1 -- 172.21.15.5:0/3338897832 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5e80043d0 con 0x55b767bb0410 2024-09-19T06:27:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.537+0000 7fc5de7fc640 1 -- 172.21.15.5:0/3338897832 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8019850 con 0x55b767bb0410 2024-09-19T06:27:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.538+0000 7fc601225740 1 -- 172.21.15.5:0/3338897832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bb0410 msgr2=0x55b767bba770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.538+0000 7fc601225740 1 --2- 172.21.15.5:0/3338897832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bb0410 0x55b767bba770 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009cf0 tx=0x7fc5e8011810 comp rx=0 tx=0).stop 2024-09-19T06:27:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.538+0000 7fc601225740 1 -- 172.21.15.5:0/3338897832 shutdown_connections 2024-09-19T06:27:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.538+0000 7fc601225740 1 --2- 172.21.15.5:0/3338897832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bbd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.538+0000 7fc601225740 1 --2- 172.21.15.5:0/3338897832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bb0410 0x55b767bba770 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.538+0000 7fc601225740 1 --2- 172.21.15.5:0/3338897832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bafaf0 0x55b767bafed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.538+0000 7fc601225740 1 -- 172.21.15.5:0/3338897832 >> 172.21.15.5:0/3338897832 conn(0x55b767bac060 msgr2=0x55b767bac440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.538+0000 7fc601225740 1 -- 172.21.15.5:0/3338897832 shutdown_connections 2024-09-19T06:27:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.538+0000 7fc601225740 1 -- 172.21.15.5:0/3338897832 wait complete. 2024-09-19T06:27:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.538+0000 7fc601225740 1 -- start start 2024-09-19T06:27:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.539+0000 7fc601225740 1 Processor -- start 2024-09-19T06:27:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.539+0000 7fc601225740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bb7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.539+0000 7fc601225740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bb80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.539+0000 7fc601225740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bb8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.539+0000 7fc601225740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b767bbd880 con 0x55b767bbacb0 2024-09-19T06:27:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.539+0000 7fc601225740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b767a9d290 con 0x55b767bb0410 2024-09-19T06:27:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.539+0000 7fc601225740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b767bbda00 con 0x55b767bafaf0 2024-09-19T06:27:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.539+0000 7fc5fca29640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bb8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.539+0000 7fc5f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bb80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.539+0000 7fc601225740 10 client.0 _ll_register_callbacks cb 0x55b767ace140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.539+0000 7fc5f7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bb7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.539+0000 7fc5fca29640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bb8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49948/0 (socket says 172.21.15.5:49948) 2024-09-19T06:27:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc5fca29640 1 -- 172.21.15.5:0/2318885808 learned_addr learned my addr 172.21.15.5:0/2318885808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc5f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bb80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58228/0 (socket says 172.21.15.5:58228) 2024-09-19T06:27:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc5fca29640 1 -- 172.21.15.5:0/2318885808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 msgr2=0x55b767bb7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc5fca29640 1 --2- 172.21.15.5:0/2318885808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bb7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc5fca29640 1 -- 172.21.15.5:0/2318885808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 msgr2=0x55b767bb80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc5fca29640 1 --2- 172.21.15.5:0/2318885808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bb80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc5fca29640 1 -- 172.21.15.5:0/2318885808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b767827210 con 0x55b767bbacb0 2024-09-19T06:27:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc5f77fe640 1 --2- 172.21.15.5:0/2318885808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bb80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc5fca29640 1 --2- 172.21.15.5:0/2318885808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bb8620 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0003ea0 tx=0x7fc5f0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469de2793488b102 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc5d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:27:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc5d77fe640 1 -- 172.21.15.5:0/2318885808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f001e570 con 0x55b767bbacb0 2024-09-19T06:27:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc5d77fe640 1 -- 172.21.15.5:0/2318885808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fc5f001e710 con 0x55b767bbacb0 2024-09-19T06:27:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.540+0000 7fc601225740 1 -- 172.21.15.5:0/2318885808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5e80043d0 con 0x55b767bbacb0 2024-09-19T06:27:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.541+0000 7fc5d77fe640 1 -- 172.21.15.5:0/2318885808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc5f001e910 con 0x55b767bbacb0 2024-09-19T06:27:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.541+0000 7fc601225740 10 client.21372 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.541+0000 7fc601225740 1 -- 172.21.15.5:0/2318885808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55b7678f3620 con 0x55b767bbacb0 2024-09-19T06:27:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.541+0000 7fc5d77fe640 1 -- 172.21.15.5:0/2318885808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5e80043d0 con 0x55b767bbacb0 2024-09-19T06:27:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.541+0000 7fc5d77fe640 1 -- 172.21.15.5:0/2318885808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc5f001b070 con 0x55b767bbacb0 2024-09-19T06:27:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.541+0000 7fc601225740 10 client.21372 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.541+0000 7fc601225740 10 client.21372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.541+0000 7fc601225740 20 client.21372 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.541+0000 7fc601225740 1 --2- 172.21.15.5:0/2318885808 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b767bcac60 0x55b767bcd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.542+0000 7fc601225740 4 client.21372 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.542+0000 7fc601225740 1 -- 172.21.15.5:0/2318885808 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b767b21a20 con 0x55b767bcac60 2024-09-19T06:27:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.542+0000 7fc5f7fff640 1 --2- 172.21.15.5:0/2318885808 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b767bcac60 0x55b767bcd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.542+0000 7fc5d77fe640 10 client.21372 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.542+0000 7fc5f7fff640 1 --2- 172.21.15.5:0/2318885808 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b767bcac60 0x55b767bcd090 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c7e949e62e1682 server_cookie=e2401c46b503fa17 in_seq=0 out_seq=0 2024-09-19T06:27:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.543+0000 7fc5d77fe640 1 -- 172.21.15.5:0/2318885808 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b767b21a20 con 0x55b767bcac60 2024-09-19T06:27:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.543+0000 7fc5d77fe640 10 client.21372 handle_command_reply: tid=0 2024-09-19T06:27:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.543+0000 7fc601225740 1 client.21372 shutdown 2024-09-19T06:27:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.544+0000 7fc601225740 1 -- 172.21.15.5:0/2318885808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 msgr2=0x55b767bb8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.544+0000 7fc601225740 1 --2- 172.21.15.5:0/2318885808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bb8620 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0003ea0 tx=0x7fc5f0004ab0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.544+0000 7fc601225740 1 -- 172.21.15.5:0/2318885808 shutdown_connections 2024-09-19T06:27:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.544+0000 7fc601225740 1 --2- 172.21.15.5:0/2318885808 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b767bcac60 0x55b767bcd090 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.544+0000 7fc601225740 1 --2- 172.21.15.5:0/2318885808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bb8620 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.544+0000 7fc5d77fe640 0 client.21372 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.544+0000 7fc601225740 1 --2- 172.21.15.5:0/2318885808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bb80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.544+0000 7fc601225740 1 --2- 172.21.15.5:0/2318885808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bb7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.544+0000 7fc601225740 1 -- 172.21.15.5:0/2318885808 >> 172.21.15.5:0/2318885808 conn(0x55b767bac060 msgr2=0x55b767bb1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.544+0000 7fc601225740 1 -- 172.21.15.5:0/2318885808 shutdown_connections 2024-09-19T06:27:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.544+0000 7fc601225740 1 -- 172.21.15.5:0/2318885808 wait complete. 2024-09-19T06:27:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.544+0000 7fc601225740 20 client.21372 trim_cache size 0 max 16384 2024-09-19T06:27:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.566+0000 7fc601225740 1 Processor -- start 2024-09-19T06:27:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.566+0000 7fc601225740 1 -- start start 2024-09-19T06:27:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.566+0000 7fc601225740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bb8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.566+0000 7fc601225740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bb9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.566+0000 7fc601225740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bd29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.566+0000 7fc601225740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81afa70 con 0x55b767bbacb0 2024-09-19T06:27:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.566+0000 7fc601225740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81b15b0 con 0x55b767bb0410 2024-09-19T06:27:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.566+0000 7fc601225740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81b0140 con 0x55b767bafaf0 2024-09-19T06:27:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5fca29640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bd29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5f7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bb8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bb9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5f7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bb8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60878/0 (socket says 172.21.15.5:60878) 2024-09-19T06:27:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5f7fff640 1 -- 172.21.15.5:0/3813225337 learned_addr learned my addr 172.21.15.5:0/3813225337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5f7fff640 1 -- 172.21.15.5:0/3813225337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 msgr2=0x55b767bb9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5f7fff640 1 --2- 172.21.15.5:0/3813225337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bb9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5f7fff640 1 -- 172.21.15.5:0/3813225337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 msgr2=0x55b767bd29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5f7fff640 1 --2- 172.21.15.5:0/3813225337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bd29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5f7fff640 1 -- 172.21.15.5:0/3813225337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b767baf920 con 0x55b767bafaf0 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5fca29640 1 --2- 172.21.15.5:0/3813225337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bd29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5f77fe640 1 --2- 172.21.15.5:0/3813225337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bb9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.567+0000 7fc5f7fff640 1 --2- 172.21.15.5:0/3813225337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bb8450 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55b767bacec0 tx=0x7fc5e8003d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a1ff19525dc14a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.568+0000 7fc5ddffb640 1 -- 172.21.15.5:0/3813225337 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e802f020 con 0x55b767bafaf0 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.568+0000 7fc5ddffb640 1 -- 172.21.15.5:0/3813225337 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5e8003aa0 con 0x55b767bafaf0 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.568+0000 7fc5ddffb640 1 -- 172.21.15.5:0/3813225337 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e802b530 con 0x55b767bafaf0 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.569+0000 7fc601225740 1 -- 172.21.15.5:0/3813225337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 msgr2=0x55b767bb8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.569+0000 7fc601225740 1 --2- 172.21.15.5:0/3813225337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bb8450 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55b767bacec0 tx=0x7fc5e8003d20 comp rx=0 tx=0).stop 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.569+0000 7fc601225740 1 -- 172.21.15.5:0/3813225337 shutdown_connections 2024-09-19T06:27:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.569+0000 7fc601225740 1 --2- 172.21.15.5:0/3813225337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bd29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.569+0000 7fc601225740 1 --2- 172.21.15.5:0/3813225337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bb9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.569+0000 7fc601225740 1 --2- 172.21.15.5:0/3813225337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bb8450 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.569+0000 7fc601225740 1 -- 172.21.15.5:0/3813225337 >> 172.21.15.5:0/3813225337 conn(0x55b767bac060 msgr2=0x55b767bb9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.569+0000 7fc601225740 1 -- 172.21.15.5:0/3813225337 shutdown_connections 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.569+0000 7fc601225740 1 -- 172.21.15.5:0/3813225337 wait complete. 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.569+0000 7fc601225740 1 -- start start 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.569+0000 7fc601225740 1 Processor -- start 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc601225740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bc3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc601225740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bbe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc601225740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bbe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc601225740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81b0ee0 con 0x55b767bbacb0 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc601225740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81b0810 con 0x55b767bb0410 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc601225740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81ada70 con 0x55b767bafaf0 2024-09-19T06:27:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc5f7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bc3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc5f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bbe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc5f7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bc3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60884/0 (socket says 172.21.15.5:60884) 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc5f7fff640 1 -- 172.21.15.5:0/2212705165 learned_addr learned my addr 172.21.15.5:0/2212705165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc601225740 10 client.0 _ll_register_callbacks cb 0x7fc5f0009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc5fca29640 1 --2- 172.21.15.5:0/2212705165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bbe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc5f77fe640 1 -- 172.21.15.5:0/2212705165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 msgr2=0x55b767bc3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc5f77fe640 1 --2- 172.21.15.5:0/2212705165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bc3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc5f77fe640 1 -- 172.21.15.5:0/2212705165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 msgr2=0x55b767bbe9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc5f77fe640 1 --2- 172.21.15.5:0/2212705165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bbe9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.570+0000 7fc5f77fe640 1 -- 172.21.15.5:0/2212705165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b767bbef80 con 0x55b767bb0410 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.571+0000 7fc5f77fe640 1 --2- 172.21.15.5:0/2212705165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bbe480 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0ed770 tx=0x7fc5ec009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a912d38581c4eeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.571+0000 7fc5d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:27:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.571+0000 7fc5d77fe640 1 -- 172.21.15.5:0/2212705165 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec049040 con 0x55b767bb0410 2024-09-19T06:27:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.571+0000 7fc5d77fe640 1 -- 172.21.15.5:0/2212705165 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fc5ec0d5260 con 0x55b767bb0410 2024-09-19T06:27:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.571+0000 7fc601225740 1 -- 172.21.15.5:0/2212705165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5e8003aa0 con 0x55b767bb0410 2024-09-19T06:27:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.571+0000 7fc5d77fe640 1 -- 172.21.15.5:0/2212705165 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc5ec031d00 con 0x55b767bb0410 2024-09-19T06:27:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.571+0000 7fc601225740 10 client.30737 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.571+0000 7fc601225740 1 -- 172.21.15.5:0/2212705165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55b767c357a0 con 0x55b767bb0410 2024-09-19T06:27:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.571+0000 7fc5d77fe640 1 -- 172.21.15.5:0/2212705165 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5e8003aa0 con 0x55b767bb0410 2024-09-19T06:27:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.571+0000 7fc5d77fe640 1 -- 172.21.15.5:0/2212705165 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc5ec0c4ae0 con 0x55b767bb0410 2024-09-19T06:27:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.572+0000 7fc601225740 10 client.30737 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.572+0000 7fc601225740 10 client.30737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.572+0000 7fc601225740 20 client.30737 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.572+0000 7fc601225740 1 --2- 172.21.15.5:0/2212705165 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b767bcac60 0x55b767c37f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.572+0000 7fc601225740 4 client.30737 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:27:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.572+0000 7fc601225740 1 -- 172.21.15.5:0/2212705165 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b767b1bd70 con 0x55b767bcac60 2024-09-19T06:27:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.572+0000 7fc5f7fff640 1 --2- 172.21.15.5:0/2212705165 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b767bcac60 0x55b767c37f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.572+0000 7fc5d77fe640 10 client.30737 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.572+0000 7fc5f7fff640 1 --2- 172.21.15.5:0/2212705165 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b767bcac60 0x55b767c37f10 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f0003ee583add0 server_cookie=a3cad400235f97 in_seq=0 out_seq=0 2024-09-19T06:27:32.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 105306 8.54 MB/sec execute 536 sec latency 60.903 ms 2024-09-19T06:27:32.603 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:27:32.604 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:27:32.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.658+0000 7fc5d77fe640 1 -- 172.21.15.5:0/2212705165 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b767b1bd70 con 0x55b767bcac60 2024-09-19T06:27:32.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.658+0000 7fc5d77fe640 10 client.30737 handle_command_reply: tid=0 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.658+0000 7fc601225740 1 client.30737 shutdown 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.658+0000 7fc601225740 1 -- 172.21.15.5:0/2212705165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 msgr2=0x55b767bbe480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.658+0000 7fc601225740 1 --2- 172.21.15.5:0/2212705165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bbe480 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0ed770 tx=0x7fc5ec009260 comp rx=0 tx=0).stop 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.658+0000 7fc601225740 1 -- 172.21.15.5:0/2212705165 shutdown_connections 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.658+0000 7fc601225740 1 --2- 172.21.15.5:0/2212705165 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55b767bcac60 0x55b767c37f10 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.659+0000 7fc601225740 1 --2- 172.21.15.5:0/2212705165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b767bbacb0 0x55b767bbe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.659+0000 7fc601225740 1 --2- 172.21.15.5:0/2212705165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b767bb0410 0x55b767bbe480 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.659+0000 7fc601225740 1 --2- 172.21.15.5:0/2212705165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b767bafaf0 0x55b767bc3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.659+0000 7fc601225740 1 -- 172.21.15.5:0/2212705165 >> 172.21.15.5:0/2212705165 conn(0x55b767bac060 msgr2=0x55b767bb9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.659+0000 7fc5d77fe640 0 client.30737 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.659+0000 7fc601225740 1 -- 172.21.15.5:0/2212705165 shutdown_connections 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.659+0000 7fc601225740 1 -- 172.21.15.5:0/2212705165 wait complete. 2024-09-19T06:27:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.659+0000 7fc601225740 20 client.30737 trim_cache size 0 max 16384 2024-09-19T06:27:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:27:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:27:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:27:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:27:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.660+0000 7fc5fe48c640 1 -- 172.21.15.5:0/731408149 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc5e00a3400 msgr2=0x7fc5e00a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.660+0000 7fc5fe48c640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc5e00a3400 0x7fc5e00a58c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8074c90 tx=0x7fc5f0009b70 comp rx=0 tx=0).stop 2024-09-19T06:27:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.660+0000 7fc5fe48c640 1 -- 172.21.15.5:0/731408149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f81011c0 msgr2=0x7fc5f819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.660+0000 7fc5fe48c640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f81011c0 0x7fc5f819aaf0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec031440 tx=0x7fc5ec003b90 comp rx=0 tx=0).stop 2024-09-19T06:27:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.660+0000 7fc5fe48c640 1 -- 172.21.15.5:0/731408149 shutdown_connections 2024-09-19T06:27:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.660+0000 7fc5fe48c640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f810e450 0x7fc5f819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.660+0000 7fc5fe48c640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f81011c0 0x7fc5f819aaf0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.660+0000 7fc5fe48c640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc5e00a3400 0x7fc5e00a58c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.660+0000 7fc5fe48c640 1 --2- 172.21.15.5:0/731408149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5f81008a0 0x7fc5f819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.660+0000 7fc5fe48c640 1 -- 172.21.15.5:0/731408149 >> 172.21.15.5:0/731408149 conn(0x7fc5f80779a0 msgr2=0x7fc5f80ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.660+0000 7fc5fe48c640 1 -- 172.21.15.5:0/731408149 shutdown_connections 2024-09-19T06:27:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:32.660+0000 7fc5fe48c640 1 -- 172.21.15.5:0/731408149 wait complete. 2024-09-19T06:27:32.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:32 smithi005 ceph-mon[30955]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 247 op/s 2024-09-19T06:27:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:32 smithi049 ceph-mon[34640]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 247 op/s 2024-09-19T06:27:33.010 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:27:33.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:32 smithi157 ceph-mon[35346]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 247 op/s 2024-09-19T06:27:33.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.587+0000 7fe5dba3a640 1 -- 172.21.15.5:0/1839435678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d4100e20 msgr2=0x7fe5d4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:33.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.587+0000 7fe5dba3a640 1 --2- 172.21.15.5:0/1839435678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d4100e20 0x7fe5d4101200 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009900 tx=0x7fe5d002a480 comp rx=0 tx=0).stop 2024-09-19T06:27:33.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.587+0000 7fe5dba3a640 1 -- 172.21.15.5:0/1839435678 shutdown_connections 2024-09-19T06:27:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.587+0000 7fe5dba3a640 1 --2- 172.21.15.5:0/1839435678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4102ef0 0x7fe5d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.587+0000 7fe5dba3a640 1 --2- 172.21.15.5:0/1839435678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d4102570 0x7fe5d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.587+0000 7fe5dba3a640 1 --2- 172.21.15.5:0/1839435678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d4100e20 0x7fe5d4101200 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.587+0000 7fe5dba3a640 1 -- 172.21.15.5:0/1839435678 >> 172.21.15.5:0/1839435678 conn(0x7fe5d40fc7d0 msgr2=0x7fe5d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.588+0000 7fe5dba3a640 1 -- 172.21.15.5:0/1839435678 shutdown_connections 2024-09-19T06:27:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.588+0000 7fe5dba3a640 1 -- 172.21.15.5:0/1839435678 wait complete. 2024-09-19T06:27:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.588+0000 7fe5dba3a640 1 Processor -- start 2024-09-19T06:27:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5dba3a640 1 -- start start 2024-09-19T06:27:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5dba3a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4100e20 0x7fe5d419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5dba3a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d4102570 0x7fe5d419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5dba3a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d4102ef0 0x7fe5d419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5dba3a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d406a540 con 0x7fe5d4102ef0 2024-09-19T06:27:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5dba3a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d406a3c0 con 0x7fe5d4100e20 2024-09-19T06:27:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5dba3a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d406a6c0 con 0x7fe5d4102570 2024-09-19T06:27:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5d97d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4100e20 0x7fe5d419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5d97d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4100e20 0x7fe5d419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58278/0 (socket says 172.21.15.5:58278) 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5d97d6640 1 -- 172.21.15.5:0/4005944830 learned_addr learned my addr 172.21.15.5:0/4005944830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d4102ef0 0x7fe5d419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5d8fd5640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d4102570 0x7fe5d419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5d97d6640 1 -- 172.21.15.5:0/4005944830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d4102570 msgr2=0x7fe5d419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5d97d6640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d4102570 0x7fe5d419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5d97d6640 1 -- 172.21.15.5:0/4005944830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d4102ef0 msgr2=0x7fe5d419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5d97d6640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d4102ef0 0x7fe5d419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.589+0000 7fe5d97d6640 1 -- 172.21.15.5:0/4005944830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d419f5a0 con 0x7fe5d4100e20 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.590+0000 7fe5d97d6640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4100e20 0x7fe5d419a5f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0006010 tx=0x7fe5d00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.590+0000 7fe5ca7fc640 1 -- 172.21.15.5:0/4005944830 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d003f030 con 0x7fe5d4100e20 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.590+0000 7fe5dba3a640 1 -- 172.21.15.5:0/4005944830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d419f830 con 0x7fe5d4100e20 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.590+0000 7fe5dba3a640 1 -- 172.21.15.5:0/4005944830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d41a7170 con 0x7fe5d4100e20 2024-09-19T06:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.590+0000 7fe5ca7fc640 1 -- 172.21.15.5:0/4005944830 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5d00044c0 con 0x7fe5d4100e20 2024-09-19T06:27:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.590+0000 7fe5ca7fc640 1 -- 172.21.15.5:0/4005944830 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0035070 con 0x7fe5d4100e20 2024-09-19T06:27:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.593+0000 7fe5ca7fc640 1 -- 172.21.15.5:0/4005944830 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe5d0042030 con 0x7fe5d4100e20 2024-09-19T06:27:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.594+0000 7fe5dde13740 1 Processor -- start 2024-09-19T06:27:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.594+0000 7fe5ca7fc640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe5c00a3430 0x7fe5c00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.594+0000 7fe5ca7fc640 1 -- 172.21.15.5:0/4005944830 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fe5d00f0010 con 0x7fe5d4100e20 2024-09-19T06:27:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.594+0000 7fe5dde13740 1 -- start start 2024-09-19T06:27:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.594+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe5c00a3430 0x7fe5c00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5dde13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 0x557febf2df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5dde13740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2e480 0x557febf38800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5dde13740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf3b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5dde13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557febe93280 con 0x557febf2db60 2024-09-19T06:27:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5dde13740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557febe3ab10 con 0x557febf2e480 2024-09-19T06:27:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5dde13740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557febe8f240 con 0x557febf38d40 2024-09-19T06:27:33.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5d97d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 0x557febf2df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5d8fd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2e480 0x557febf38800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5d97d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 0x557febf2df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49990/0 (socket says 172.21.15.5:49990) 2024-09-19T06:27:33.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5d8fd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2e480 0x557febf38800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58288/0 (socket says 172.21.15.5:58288) 2024-09-19T06:27:33.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5d97d6640 1 -- 172.21.15.5:0/4241565381 learned_addr learned my addr 172.21.15.5:0/4241565381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:33.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe5c00a3430 0x7fe5c00a58f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc005fd0 tx=0x7fe5cc009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:33.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/4241565381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf3b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5d8fd5640 1 -- 172.21.15.5:0/4241565381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 msgr2=0x557febf3b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:33.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5d8fd5640 1 --2- 172.21.15.5:0/4241565381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf3b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5d8fd5640 1 -- 172.21.15.5:0/4241565381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 msgr2=0x557febf2df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.595+0000 7fe5d8fd5640 1 --2- 172.21.15.5:0/4241565381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 0x557febf2df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.596+0000 7fe5d8fd5640 1 -- 172.21.15.5:0/4241565381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557febe471f0 con 0x557febf2e480 2024-09-19T06:27:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.596+0000 7fe5d97d6640 1 --2- 172.21.15.5:0/4241565381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 0x557febf2df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:27:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.596+0000 7fe5d8fd5640 1 --2- 172.21.15.5:0/4241565381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2e480 0x557febf38800 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009b40 tx=0x7fe5c40113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d6d67055d58ee1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.596+0000 7fe5bf7fe640 1 -- 172.21.15.5:0/4241565381 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4011e10 con 0x557febf2e480 2024-09-19T06:27:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.596+0000 7fe5bf7fe640 1 -- 172.21.15.5:0/4241565381 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5c4012900 con 0x557febf2e480 2024-09-19T06:27:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.596+0000 7fe5bf7fe640 1 -- 172.21.15.5:0/4241565381 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400fa60 con 0x557febf2e480 2024-09-19T06:27:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.597+0000 7fe5dde13740 1 -- 172.21.15.5:0/4241565381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2e480 msgr2=0x557febf38800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.597+0000 7fe5dde13740 1 --2- 172.21.15.5:0/4241565381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2e480 0x557febf38800 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009b40 tx=0x7fe5c40113e0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.597+0000 7fe5dde13740 1 -- 172.21.15.5:0/4241565381 shutdown_connections 2024-09-19T06:27:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.597+0000 7fe5dde13740 1 --2- 172.21.15.5:0/4241565381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf3b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.597+0000 7fe5dde13740 1 --2- 172.21.15.5:0/4241565381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2e480 0x557febf38800 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.597+0000 7fe5dde13740 1 --2- 172.21.15.5:0/4241565381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 0x557febf2df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.597+0000 7fe5dde13740 1 -- 172.21.15.5:0/4241565381 >> 172.21.15.5:0/4241565381 conn(0x557febf2a0f0 msgr2=0x557febf2a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.597+0000 7fe5dde13740 1 -- 172.21.15.5:0/4241565381 shutdown_connections 2024-09-19T06:27:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.597+0000 7fe5dde13740 1 -- 172.21.15.5:0/4241565381 wait complete. 2024-09-19T06:27:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5dde13740 1 -- start start 2024-09-19T06:27:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5dde13740 1 Processor -- start 2024-09-19T06:27:33.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 105455 8.54 MB/sec execute 537 sec latency 107.352 ms 2024-09-19T06:27:33.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5dde13740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf2db60 0x557febf35c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5dde13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2e480 0x557febf36140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5dde13740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf38d40 0x557febf36680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5dde13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557febf3b910 con 0x557febf2e480 2024-09-19T06:27:33.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5dde13740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557febe4df10 con 0x557febf38d40 2024-09-19T06:27:33.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5dde13740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557febf3ba90 con 0x557febf2db60 2024-09-19T06:27:33.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5dde13740 10 client.0 _ll_register_callbacks cb 0x557febe16a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:33.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5d9fd7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf38d40 0x557febf36680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5d97d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf2db60 0x557febf35c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5d9fd7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf38d40 0x557febf36680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58302/0 (socket says 172.21.15.5:58302) 2024-09-19T06:27:33.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5d97d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf2db60 0x557febf35c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60912/0 (socket says 172.21.15.5:60912) 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.598+0000 7fe5d9fd7640 1 -- 172.21.15.5:0/69936279 learned_addr learned my addr 172.21.15.5:0/69936279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5d8fd5640 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2e480 0x557febf36140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5d9fd7640 1 -- 172.21.15.5:0/69936279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf2db60 msgr2=0x557febf35c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf2db60 0x557febf35c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5d9fd7640 1 -- 172.21.15.5:0/69936279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2e480 msgr2=0x557febf36140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2e480 0x557febf36140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5d9fd7640 1 -- 172.21.15.5:0/69936279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557febc046a0 con 0x557febf38d40 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5d8fd5640 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2e480 0x557febf36140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5d97d6640 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf2db60 0x557febf35c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf38d40 0x557febf36680 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00ed40 tx=0x7fe5cc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a0c3167a50b511 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5bcff9640 1 -- 172.21.15.5:0/69936279 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc00ee30 con 0x557febf38d40 2024-09-19T06:27:33.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5dde13740 1 -- 172.21.15.5:0/69936279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5c4012900 con 0x557febf38d40 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.599+0000 7fe5bcff9640 1 -- 172.21.15.5:0/69936279 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fe5cc004b20 con 0x557febf38d40 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.600+0000 7fe5bcff9640 1 -- 172.21.15.5:0/69936279 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fe5cc004d20 con 0x557febf38d40 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.600+0000 7fe5dde13740 10 client.30749 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.600+0000 7fe5dde13740 1 -- 172.21.15.5:0/69936279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x557febf4fd90 con 0x557febf38d40 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.600+0000 7fe5bcff9640 1 -- 172.21.15.5:0/69936279 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5c4012900 con 0x557febf38d40 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.600+0000 7fe5bcff9640 1 -- 172.21.15.5:0/69936279 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fe5cc026070 con 0x557febf38d40 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.600+0000 7fe5dde13740 10 client.30749 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.600+0000 7fe5dde13740 10 client.30749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.600+0000 7fe5dde13740 20 client.30749 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.600+0000 7fe5dde13740 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557febf512f0 0x557febf53770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.600+0000 7fe5dde13740 4 client.30749 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.600+0000 7fe5dde13740 1 -- 172.21.15.5:0/69936279 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557febea07f0 con 0x557febf512f0 2024-09-19T06:27:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.600+0000 7fe5d97d6640 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557febf512f0 0x557febf53770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.601+0000 7fe5d97d6640 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557febf512f0 0x557febf53770 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fe66666bf01c827 server_cookie=580f74895d1dbc1 in_seq=0 out_seq=0 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.601+0000 7fe5bcff9640 10 client.30749 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5bcff9640 1 -- 172.21.15.5:0/69936279 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557febea07f0 con 0x557febf512f0 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5bcff9640 10 client.30749 handle_command_reply: tid=0 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5dde13740 1 client.30749 shutdown 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5dde13740 1 -- 172.21.15.5:0/69936279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf38d40 msgr2=0x557febf36680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5dde13740 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf38d40 0x557febf36680 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00ed40 tx=0x7fe5cc004ab0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5dde13740 1 -- 172.21.15.5:0/69936279 shutdown_connections 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5dde13740 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557febf512f0 0x557febf53770 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5dde13740 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf38d40 0x557febf36680 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5dde13740 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2e480 0x557febf36140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5dde13740 1 --2- 172.21.15.5:0/69936279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf2db60 0x557febf35c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5dde13740 1 -- 172.21.15.5:0/69936279 >> 172.21.15.5:0/69936279 conn(0x557febf2a0f0 msgr2=0x557febf29270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:33.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5bcff9640 0 client.30749 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:33.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5dde13740 1 -- 172.21.15.5:0/69936279 shutdown_connections 2024-09-19T06:27:33.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5dde13740 1 -- 172.21.15.5:0/69936279 wait complete. 2024-09-19T06:27:33.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.602+0000 7fe5dde13740 20 client.30749 trim_cache size 0 max 16384 2024-09-19T06:27:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5dde13740 1 Processor -- start 2024-09-19T06:27:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5dde13740 1 -- start start 2024-09-19T06:27:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5dde13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 0x557febf53760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5dde13740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2e480 0x557febf59040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5dde13740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf59580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5dde13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41ab5b0 con 0x557febf2db60 2024-09-19T06:27:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5dde13740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41ad0f0 con 0x557febf2e480 2024-09-19T06:27:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5dde13740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41abc80 con 0x557febf38d40 2024-09-19T06:27:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5d97d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 0x557febf53760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5d97d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 0x557febf53760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50022/0 (socket says 172.21.15.5:50022) 2024-09-19T06:27:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5d97d6640 1 -- 172.21.15.5:0/3750364594 learned_addr learned my addr 172.21.15.5:0/3750364594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/3750364594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf59580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.629+0000 7fe5d8fd5640 1 --2- 172.21.15.5:0/3750364594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2e480 0x557febf59040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.630+0000 7fe5d9fd7640 1 -- 172.21.15.5:0/3750364594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2e480 msgr2=0x557febf59040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.630+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/3750364594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2e480 0x557febf59040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.630+0000 7fe5d9fd7640 1 -- 172.21.15.5:0/3750364594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 msgr2=0x557febf53760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.630+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/3750364594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 0x557febf53760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.630+0000 7fe5d9fd7640 1 -- 172.21.15.5:0/3750364594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557febf29f20 con 0x557febf38d40 2024-09-19T06:27:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.630+0000 7fe5d97d6640 1 --2- 172.21.15.5:0/3750364594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 0x557febf53760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.630+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/3750364594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf59580 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0054c0 tx=0x7fe5cc00e460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf68a30bdfe7748 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.630+0000 7fe5beffd640 1 -- 172.21.15.5:0/3750364594 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc004b20 con 0x557febf38d40 2024-09-19T06:27:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.630+0000 7fe5beffd640 1 -- 172.21.15.5:0/3750364594 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5cc003e20 con 0x557febf38d40 2024-09-19T06:27:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.630+0000 7fe5beffd640 1 -- 172.21.15.5:0/3750364594 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0042a0 con 0x557febf38d40 2024-09-19T06:27:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.631+0000 7fe5dde13740 1 -- 172.21.15.5:0/3750364594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 msgr2=0x557febf59580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.631+0000 7fe5dde13740 1 --2- 172.21.15.5:0/3750364594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf59580 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0054c0 tx=0x7fe5cc00e460 comp rx=0 tx=0).stop 2024-09-19T06:27:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.631+0000 7fe5dde13740 1 -- 172.21.15.5:0/3750364594 shutdown_connections 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.631+0000 7fe5dde13740 1 --2- 172.21.15.5:0/3750364594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf59580 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.631+0000 7fe5dde13740 1 --2- 172.21.15.5:0/3750364594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2e480 0x557febf59040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.631+0000 7fe5dde13740 1 --2- 172.21.15.5:0/3750364594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2db60 0x557febf53760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.631+0000 7fe5dde13740 1 -- 172.21.15.5:0/3750364594 >> 172.21.15.5:0/3750364594 conn(0x557febf2a0f0 msgr2=0x557febf34cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.631+0000 7fe5dde13740 1 -- 172.21.15.5:0/3750364594 shutdown_connections 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.631+0000 7fe5dde13740 1 -- 172.21.15.5:0/3750364594 wait complete. 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.631+0000 7fe5dde13740 1 -- start start 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5dde13740 1 Processor -- start 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5dde13740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2db60 0x557febf49870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5dde13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2e480 0x557febf49db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5dde13740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf44c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5dde13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41aca20 con 0x557febf2e480 2024-09-19T06:27:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5d97d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2db60 0x557febf49870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5dde13740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41ac350 con 0x557febf2db60 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5dde13740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41a95f0 con 0x557febf38d40 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5d97d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2db60 0x557febf49870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58314/0 (socket says 172.21.15.5:58314) 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5d97d6640 1 -- 172.21.15.5:0/137729177 learned_addr learned my addr 172.21.15.5:0/137729177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5dde13740 10 client.0 _ll_register_callbacks cb 0x7fe5cc00e4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5d8fd5640 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2e480 0x557febf49db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.632+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf44c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.633+0000 7fe5d9fd7640 1 -- 172.21.15.5:0/137729177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2db60 msgr2=0x557febf49870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.633+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2db60 0x557febf49870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.633+0000 7fe5d9fd7640 1 -- 172.21.15.5:0/137729177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2e480 msgr2=0x557febf49db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.633+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2e480 0x557febf49db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.633+0000 7fe5d9fd7640 1 -- 172.21.15.5:0/137729177 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557febf309d0 con 0x557febf38d40 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.633+0000 7fe5d9fd7640 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf44c00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x557febf29b40 tx=0x7fe5cc00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21a929b54b3e5325 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.633+0000 7fe5bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.633+0000 7fe5bcff9640 1 -- 172.21.15.5:0/137729177 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc01a5b0 con 0x557febf38d40 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.634+0000 7fe5dde13740 1 -- 172.21.15.5:0/137729177 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5cc003e20 con 0x557febf38d40 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.634+0000 7fe5bcff9640 1 -- 172.21.15.5:0/137729177 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fe5cc01a750 con 0x557febf38d40 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.634+0000 7fe5bcff9640 1 -- 172.21.15.5:0/137729177 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fe5cc01a950 con 0x557febf38d40 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.634+0000 7fe5dde13740 10 client.30691 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.634+0000 7fe5dde13740 1 -- 172.21.15.5:0/137729177 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x557febfbba10 con 0x557febf38d40 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.634+0000 7fe5bcff9640 1 -- 172.21.15.5:0/137729177 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5cc003e20 con 0x557febf38d40 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.635+0000 7fe5bcff9640 1 -- 172.21.15.5:0/137729177 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fe5cc036070 con 0x557febf38d40 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.635+0000 7fe5dde13740 10 client.30691 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.635+0000 7fe5dde13740 10 client.30691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.635+0000 7fe5dde13740 20 client.30691 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.635+0000 7fe5dde13740 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557febfbbfc0 0x557febfbe3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.635+0000 7fe5dde13740 4 client.30691 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:27:33.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.635+0000 7fe5dde13740 1 -- 172.21.15.5:0/137729177 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557febe9ad00 con 0x557febfbbfc0 2024-09-19T06:27:33.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.635+0000 7fe5d97d6640 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557febfbbfc0 0x557febfbe3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:33.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.635+0000 7fe5bcff9640 10 client.30691 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:33.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.635+0000 7fe5d97d6640 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557febfbbfc0 0x557febfbe3b0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e8b1a89e2cba10e server_cookie=f9ce4820448d4d45 in_seq=0 out_seq=0 2024-09-19T06:27:33.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.636+0000 7fe5bcff9640 1 -- 172.21.15.5:0/137729177 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557febe9ad00 con 0x557febfbbfc0 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.636+0000 7fe5bcff9640 10 client.30691 handle_command_reply: tid=0 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.636+0000 7fe5dde13740 1 client.30691 shutdown 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.637+0000 7fe5dde13740 1 -- 172.21.15.5:0/137729177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 msgr2=0x557febf44c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.637+0000 7fe5dde13740 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf44c00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x557febf29b40 tx=0x7fe5cc00ee30 comp rx=0 tx=0).stop 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.637+0000 7fe5dde13740 1 -- 172.21.15.5:0/137729177 shutdown_connections 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.637+0000 7fe5dde13740 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557febfbbfc0 0x557febfbe3b0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.637+0000 7fe5dde13740 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557febf38d40 0x557febf44c00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.637+0000 7fe5dde13740 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557febf2e480 0x557febf49db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.637+0000 7fe5dde13740 1 --2- 172.21.15.5:0/137729177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557febf2db60 0x557febf49870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.637+0000 7fe5dde13740 1 -- 172.21.15.5:0/137729177 >> 172.21.15.5:0/137729177 conn(0x557febf2a0f0 msgr2=0x557febf28a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.637+0000 7fe5bcff9640 0 client.30691 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.637+0000 7fe5dde13740 1 -- 172.21.15.5:0/137729177 shutdown_connections 2024-09-19T06:27:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.637+0000 7fe5dde13740 1 -- 172.21.15.5:0/137729177 wait complete. 2024-09-19T06:27:33.642 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:27:33.642 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:27:33.642 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "49657249-7c60-436c-89e9-7f9d274f170d", 2024-09-19T06:27:33.642 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:27:33.642 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:27:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.637+0000 7fe5dde13740 20 client.30691 trim_cache size 0 max 16384 2024-09-19T06:27:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.638+0000 7fe5dba3a640 1 -- 172.21.15.5:0/4005944830 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe5c00a3430 msgr2=0x7fe5c00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.638+0000 7fe5dba3a640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe5c00a3430 0x7fe5c00a58f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc005fd0 tx=0x7fe5cc009500 comp rx=0 tx=0).stop 2024-09-19T06:27:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.638+0000 7fe5dba3a640 1 -- 172.21.15.5:0/4005944830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4100e20 msgr2=0x7fe5d419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.638+0000 7fe5dba3a640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4100e20 0x7fe5d419a5f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0006010 tx=0x7fe5d00040c0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.638+0000 7fe5dba3a640 1 -- 172.21.15.5:0/4005944830 shutdown_connections 2024-09-19T06:27:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.638+0000 7fe5dba3a640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d4102ef0 0x7fe5d419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.638+0000 7fe5dba3a640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fe5c00a3430 0x7fe5c00a58f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.638+0000 7fe5dba3a640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d4102570 0x7fe5d419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.638+0000 7fe5dba3a640 1 --2- 172.21.15.5:0/4005944830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4100e20 0x7fe5d419a5f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:33.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.638+0000 7fe5dba3a640 1 -- 172.21.15.5:0/4005944830 >> 172.21.15.5:0/4005944830 conn(0x7fe5d40fc7d0 msgr2=0x7fe5d4110df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:33.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.638+0000 7fe5dba3a640 1 -- 172.21.15.5:0/4005944830 shutdown_connections 2024-09-19T06:27:33.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:33.639+0000 7fe5dba3a640 1 -- 172.21.15.5:0/4005944830 wait complete. 2024-09-19T06:27:34.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 105605 8.53 MB/sec execute 538 sec latency 80.934 ms 2024-09-19T06:27:34.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:27:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.3 MiB/s wr, 367 op/s 2024-09-19T06:27:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:27:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:27:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:27:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:27:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:27:34.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:27:34.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:34.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:27:34.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:34.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:34.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:34.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:34 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.3 MiB/s wr, 367 op/s 2024-09-19T06:27:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:27:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:27:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:27:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:27:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:27:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:27:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:27:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:34.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:34.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:34.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:34 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:27:35.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.3 MiB/s wr, 367 op/s 2024-09-19T06:27:35.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:27:35.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:35.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:27:35.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:27:35.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:27:35.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:27:35.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:27:35.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:35.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:27:35.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:35.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:35.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:35.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:34 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:35.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 105756 8.53 MB/sec execute 539 sec latency 457.096 ms 2024-09-19T06:27:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.599+0000 7fc165950640 1 -- 172.21.15.5:0/1629472444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc160102d30 msgr2=0x7fc160103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.599+0000 7fc165950640 1 --2- 172.21.15.5:0/1629472444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc160102d30 0x7fc160103110 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc150009930 tx=0x7fc15002a420 comp rx=0 tx=0).stop 2024-09-19T06:27:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.599+0000 7fc165950640 1 -- 172.21.15.5:0/1629472444 shutdown_connections 2024-09-19T06:27:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.599+0000 7fc165950640 1 --2- 172.21.15.5:0/1629472444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1601082a0 0x7fc16010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.599+0000 7fc165950640 1 --2- 172.21.15.5:0/1629472444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1601036e0 0x7fc160107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.599+0000 7fc165950640 1 --2- 172.21.15.5:0/1629472444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc160102d30 0x7fc160103110 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.599+0000 7fc165950640 1 -- 172.21.15.5:0/1629472444 >> 172.21.15.5:0/1629472444 conn(0x7fc1600fc7d0 msgr2=0x7fc1600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.599+0000 7fc165950640 1 -- 172.21.15.5:0/1629472444 shutdown_connections 2024-09-19T06:27:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.599+0000 7fc165950640 1 -- 172.21.15.5:0/1629472444 wait complete. 2024-09-19T06:27:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.600+0000 7fc165950640 1 Processor -- start 2024-09-19T06:27:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.601+0000 7fc165950640 1 -- start start 2024-09-19T06:27:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.601+0000 7fc165950640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc160102d30 0x7fc160198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.601+0000 7fc165950640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1601036e0 0x7fc160198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.601+0000 7fc15effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc160102d30 0x7fc160198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc165950640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1601082a0 0x7fc16019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc165950640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc160111f50 con 0x7fc160102d30 2024-09-19T06:27:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc165950640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc160111dd0 con 0x7fc1601082a0 2024-09-19T06:27:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc165950640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601120d0 con 0x7fc1601036e0 2024-09-19T06:27:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc15effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc160102d30 0x7fc160198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48282/0 (socket says 172.21.15.5:48282) 2024-09-19T06:27:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc15effd640 1 -- 172.21.15.5:0/3222288520 learned_addr learned my addr 172.21.15.5:0/3222288520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc15e7fc640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1601036e0 0x7fc160198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc15f7fe640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1601082a0 0x7fc16019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc15f7fe640 1 -- 172.21.15.5:0/3222288520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1601036e0 msgr2=0x7fc160198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc15f7fe640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1601036e0 0x7fc160198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc15f7fe640 1 -- 172.21.15.5:0/3222288520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc160102d30 msgr2=0x7fc160198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc15f7fe640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc160102d30 0x7fc160198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc15f7fe640 1 -- 172.21.15.5:0/3222288520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16019d4c0 con 0x7fc1601082a0 2024-09-19T06:27:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.602+0000 7fc15effd640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc160102d30 0x7fc160198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.603+0000 7fc15f7fe640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1601082a0 0x7fc16019cde0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc158009fd0 tx=0x7fc15800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.603+0000 7fc16494e640 1 -- 172.21.15.5:0/3222288520 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15800ce20 con 0x7fc1601082a0 2024-09-19T06:27:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.603+0000 7fc16494e640 1 -- 172.21.15.5:0/3222288520 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc158005020 con 0x7fc1601082a0 2024-09-19T06:27:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.603+0000 7fc16494e640 1 -- 172.21.15.5:0/3222288520 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158010690 con 0x7fc1601082a0 2024-09-19T06:27:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.603+0000 7fc165950640 1 -- 172.21.15.5:0/3222288520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16019d7b0 con 0x7fc1601082a0 2024-09-19T06:27:35.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.603+0000 7fc165950640 1 -- 172.21.15.5:0/3222288520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1601008c0 con 0x7fc1601082a0 2024-09-19T06:27:35.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.606+0000 7fc16494e640 1 -- 172.21.15.5:0/3222288520 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc1580051d0 con 0x7fc1601082a0 2024-09-19T06:27:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.606+0000 7fc167d29740 1 Processor -- start 2024-09-19T06:27:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.606+0000 7fc167d29740 1 -- start start 2024-09-19T06:27:35.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.607+0000 7fc167d29740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eec44a0 0x55960eec4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.607+0000 7fc167d29740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 0x55960eecf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.607+0000 7fc167d29740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eecf680 0x55960eed1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.607+0000 7fc15effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 0x55960eecf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.607+0000 7fc167d29740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55960ee2a2e0 con 0x55960eecf680 2024-09-19T06:27:35.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.607+0000 7fc167d29740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55960edd1ce0 con 0x55960eec4dc0 2024-09-19T06:27:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.607+0000 7fc167d29740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55960ee262a0 con 0x55960eec44a0 2024-09-19T06:27:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.607+0000 7fc15effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 0x55960eecf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44704/0 (socket says 172.21.15.5:44704) 2024-09-19T06:27:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.607+0000 7fc15effd640 1 -- 172.21.15.5:0/1207294899 learned_addr learned my addr 172.21.15.5:0/1207294899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.607+0000 7fc16494e640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1480a34b0 0x7fc1480a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.607+0000 7fc15e7fc640 1 --2- 172.21.15.5:0/1207294899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eecf680 0x55960eed1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.608+0000 7fc15effd640 1 -- 172.21.15.5:0/1207294899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eec44a0 msgr2=0x55960eec4880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:27:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.608+0000 7fc15effd640 1 --2- 172.21.15.5:0/1207294899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eec44a0 0x55960eec4880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.608+0000 7fc15effd640 1 -- 172.21.15.5:0/1207294899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eecf680 msgr2=0x55960eed1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.608+0000 7fc15effd640 1 --2- 172.21.15.5:0/1207294899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eecf680 0x55960eed1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.608+0000 7fc15effd640 1 -- 172.21.15.5:0/1207294899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960edb6440 con 0x55960eec4dc0 2024-09-19T06:27:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.608+0000 7fc15e7fc640 1 --2- 172.21.15.5:0/1207294899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eecf680 0x55960eed1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:27:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.608+0000 7fc16494e640 1 -- 172.21.15.5:0/3222288520 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fc1580d0360 con 0x7fc1601082a0 2024-09-19T06:27:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.608+0000 7fc15effd640 1 --2- 172.21.15.5:0/1207294899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 0x55960eecf140 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc150009230 tx=0x7fc150032eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98f453633d4e6db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.608+0000 7fc145ffb640 1 -- 172.21.15.5:0/1207294899 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150035040 con 0x55960eec4dc0 2024-09-19T06:27:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.608+0000 7fc145ffb640 1 -- 172.21.15.5:0/1207294899 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1500317a0 con 0x55960eec4dc0 2024-09-19T06:27:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.608+0000 7fc145ffb640 1 -- 172.21.15.5:0/1207294899 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150004a00 con 0x55960eec4dc0 2024-09-19T06:27:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.609+0000 7fc15f7fe640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1480a34b0 0x7fc1480a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.609+0000 7fc167d29740 1 -- 172.21.15.5:0/1207294899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 msgr2=0x55960eecf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:35.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.609+0000 7fc15f7fe640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1480a34b0 0x7fc1480a5970 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc1580a4d00 tx=0x7fc15800c890 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:35.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.609+0000 7fc167d29740 1 --2- 172.21.15.5:0/1207294899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 0x55960eecf140 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc150009230 tx=0x7fc150032eb0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.609+0000 7fc167d29740 1 -- 172.21.15.5:0/1207294899 shutdown_connections 2024-09-19T06:27:35.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.610+0000 7fc167d29740 1 --2- 172.21.15.5:0/1207294899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eecf680 0x55960eed1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.610+0000 7fc167d29740 1 --2- 172.21.15.5:0/1207294899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 0x55960eecf140 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.610+0000 7fc167d29740 1 --2- 172.21.15.5:0/1207294899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eec44a0 0x55960eec4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.610+0000 7fc167d29740 1 -- 172.21.15.5:0/1207294899 >> 172.21.15.5:0/1207294899 conn(0x55960eec0a30 msgr2=0x55960eec0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:35.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.610+0000 7fc167d29740 1 -- 172.21.15.5:0/1207294899 shutdown_connections 2024-09-19T06:27:35.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.610+0000 7fc167d29740 1 -- 172.21.15.5:0/1207294899 wait complete. 2024-09-19T06:27:35.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.610+0000 7fc167d29740 1 -- start start 2024-09-19T06:27:35.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.611+0000 7fc167d29740 1 Processor -- start 2024-09-19T06:27:35.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.611+0000 7fc167d29740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec44a0 0x55960eecf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc167d29740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 0x55960eecc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc15e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec44a0 0x55960eecf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc167d29740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eecf680 0x55960eecc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc167d29740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55960eed2250 con 0x55960eec44a0 2024-09-19T06:27:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc15effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 0x55960eecc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc15effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 0x55960eecc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44708/0 (socket says 172.21.15.5:44708) 2024-09-19T06:27:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc15effd640 1 -- 172.21.15.5:0/3360469584 learned_addr learned my addr 172.21.15.5:0/3360469584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc15e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec44a0 0x55960eecf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48304/0 (socket says 172.21.15.5:48304) 2024-09-19T06:27:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc167d29740 1 -- 172.21.15.5:0/3360469584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55960ede5050 con 0x55960eec4dc0 2024-09-19T06:27:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc167d29740 1 -- 172.21.15.5:0/3360469584 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55960eed23d0 con 0x55960eecf680 2024-09-19T06:27:35.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc167d29740 10 client.0 _ll_register_callbacks cb 0x55960ee25160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:35.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc15effd640 1 -- 172.21.15.5:0/3360469584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eecf680 msgr2=0x55960eecc8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:27:35.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc15effd640 1 --2- 172.21.15.5:0/3360469584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eecf680 0x55960eecc8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc15effd640 1 -- 172.21.15.5:0/3360469584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec44a0 msgr2=0x55960eecf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:35.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc15effd640 1 --2- 172.21.15.5:0/3360469584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec44a0 0x55960eecf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.612+0000 7fc15effd640 1 -- 172.21.15.5:0/3360469584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55960eb9b3c0 con 0x55960eec4dc0 2024-09-19T06:27:35.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.613+0000 7fc15effd640 1 --2- 172.21.15.5:0/3360469584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 0x55960eecc310 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc150031440 tx=0x7fc150004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f652384246921986 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:35.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.613+0000 7fc13effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:27:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.615+0000 7fc13effd640 1 -- 172.21.15.5:0/3360469584 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15004e020 con 0x55960eec4dc0 2024-09-19T06:27:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.615+0000 7fc13effd640 1 -- 172.21.15.5:0/3360469584 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fc1500035f0 con 0x55960eec4dc0 2024-09-19T06:27:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.615+0000 7fc167d29740 1 -- 172.21.15.5:0/3360469584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1500317a0 con 0x55960eec4dc0 2024-09-19T06:27:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.615+0000 7fc13effd640 1 -- 172.21.15.5:0/3360469584 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc150059050 con 0x55960eec4dc0 2024-09-19T06:27:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.615+0000 7fc167d29740 10 client.30767 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.615+0000 7fc167d29740 1 -- 172.21.15.5:0/3360469584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55960eee63f0 con 0x55960eec4dc0 2024-09-19T06:27:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.615+0000 7fc13effd640 1 -- 172.21.15.5:0/3360469584 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1500317a0 con 0x55960eec4dc0 2024-09-19T06:27:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.616+0000 7fc13effd640 1 -- 172.21.15.5:0/3360469584 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc1500467c0 con 0x55960eec4dc0 2024-09-19T06:27:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.616+0000 7fc167d29740 10 client.30767 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.616+0000 7fc167d29740 10 client.30767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.616+0000 7fc167d29740 20 client.30767 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.616+0000 7fc167d29740 1 --2- 172.21.15.5:0/3360469584 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55960eee7740 0x55960eee9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.616+0000 7fc167d29740 4 client.30767 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.616+0000 7fc167d29740 1 -- 172.21.15.5:0/3360469584 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55960ee37850 con 0x55960eee7740 2024-09-19T06:27:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.616+0000 7fc15e7fc640 1 --2- 172.21.15.5:0/3360469584 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55960eee7740 0x55960eee9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.616+0000 7fc13effd640 10 client.30767 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.616+0000 7fc15e7fc640 1 --2- 172.21.15.5:0/3360469584 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55960eee7740 0x55960eee9b30 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=655c39b3fe317a3f server_cookie=8a9078427b4f2466 in_seq=0 out_seq=0 2024-09-19T06:27:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.617+0000 7fc13effd640 1 -- 172.21.15.5:0/3360469584 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55960ee37850 con 0x55960eee7740 2024-09-19T06:27:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.617+0000 7fc13effd640 10 client.30767 handle_command_reply: tid=0 2024-09-19T06:27:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.617+0000 7fc167d29740 1 client.30767 shutdown 2024-09-19T06:27:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.618+0000 7fc167d29740 1 -- 172.21.15.5:0/3360469584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 msgr2=0x55960eecc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.618+0000 7fc167d29740 1 --2- 172.21.15.5:0/3360469584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 0x55960eecc310 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc150031440 tx=0x7fc150004270 comp rx=0 tx=0).stop 2024-09-19T06:27:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.618+0000 7fc167d29740 1 -- 172.21.15.5:0/3360469584 shutdown_connections 2024-09-19T06:27:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.618+0000 7fc167d29740 1 --2- 172.21.15.5:0/3360469584 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55960eee7740 0x55960eee9b30 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.618+0000 7fc167d29740 1 --2- 172.21.15.5:0/3360469584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eecf680 0x55960eecc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.618+0000 7fc167d29740 1 --2- 172.21.15.5:0/3360469584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec4dc0 0x55960eecc310 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.618+0000 7fc167d29740 1 --2- 172.21.15.5:0/3360469584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec44a0 0x55960eecf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.618+0000 7fc167d29740 1 -- 172.21.15.5:0/3360469584 >> 172.21.15.5:0/3360469584 conn(0x55960eec0a30 msgr2=0x55960eec6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.618+0000 7fc13effd640 0 client.30767 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.618+0000 7fc167d29740 1 -- 172.21.15.5:0/3360469584 shutdown_connections 2024-09-19T06:27:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.618+0000 7fc167d29740 1 -- 172.21.15.5:0/3360469584 wait complete. 2024-09-19T06:27:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.618+0000 7fc167d29740 20 client.30767 trim_cache size 0 max 16384 2024-09-19T06:27:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.642+0000 7fc167d29740 1 Processor -- start 2024-09-19T06:27:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.642+0000 7fc167d29740 1 -- start start 2024-09-19T06:27:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.642+0000 7fc167d29740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec44a0 0x55960eec95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.642+0000 7fc167d29740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec4dc0 0x55960eec9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.642+0000 7fc167d29740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eecf680 0x55960eecbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.642+0000 7fc167d29740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601ab680 con 0x55960eec4dc0 2024-09-19T06:27:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.642+0000 7fc167d29740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601ad1c0 con 0x55960eec44a0 2024-09-19T06:27:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.642+0000 7fc167d29740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601abd50 con 0x55960eecf680 2024-09-19T06:27:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.643+0000 7fc15e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eecf680 0x55960eecbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.643+0000 7fc15effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec4dc0 0x55960eec9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.643+0000 7fc15e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eecf680 0x55960eecbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34922/0 (socket says 172.21.15.5:34922) 2024-09-19T06:27:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.643+0000 7fc15e7fc640 1 -- 172.21.15.5:0/3602759039 learned_addr learned my addr 172.21.15.5:0/3602759039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.643+0000 7fc15e7fc640 1 --2- 172.21.15.5:0/3602759039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec44a0 0x55960eec95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.643+0000 7fc15e7fc640 1 -- 172.21.15.5:0/3602759039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec44a0 msgr2=0x55960eec95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.643+0000 7fc15e7fc640 1 --2- 172.21.15.5:0/3602759039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec44a0 0x55960eec95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.643+0000 7fc15e7fc640 1 -- 172.21.15.5:0/3602759039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec4dc0 msgr2=0x55960eec9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.643+0000 7fc15e7fc640 1 --2- 172.21.15.5:0/3602759039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec4dc0 0x55960eec9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.643+0000 7fc15e7fc640 1 -- 172.21.15.5:0/3602759039 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960eeca880 con 0x55960eecf680 2024-09-19T06:27:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.643+0000 7fc15effd640 1 --2- 172.21.15.5:0/3602759039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec4dc0 0x55960eec9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.643+0000 7fc15e7fc640 1 --2- 172.21.15.5:0/3602759039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eecf680 0x55960eecbbd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc154012810 tx=0x7fc154005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11d1a58c6e51140b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.644+0000 7fc1457fa640 1 -- 172.21.15.5:0/3602759039 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154009a10 con 0x55960eecf680 2024-09-19T06:27:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.644+0000 7fc1457fa640 1 -- 172.21.15.5:0/3602759039 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc154009bb0 con 0x55960eecf680 2024-09-19T06:27:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.644+0000 7fc1457fa640 1 -- 172.21.15.5:0/3602759039 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15401d580 con 0x55960eecf680 2024-09-19T06:27:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.644+0000 7fc167d29740 1 -- 172.21.15.5:0/3602759039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eecf680 msgr2=0x55960eecbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.644+0000 7fc167d29740 1 --2- 172.21.15.5:0/3602759039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eecf680 0x55960eecbbd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc154012810 tx=0x7fc154005f90 comp rx=0 tx=0).stop 2024-09-19T06:27:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.645+0000 7fc167d29740 1 -- 172.21.15.5:0/3602759039 shutdown_connections 2024-09-19T06:27:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.645+0000 7fc167d29740 1 --2- 172.21.15.5:0/3602759039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eecf680 0x55960eecbbd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.645+0000 7fc167d29740 1 --2- 172.21.15.5:0/3602759039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec4dc0 0x55960eec9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.645+0000 7fc167d29740 1 --2- 172.21.15.5:0/3602759039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eec44a0 0x55960eec95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.645+0000 7fc167d29740 1 -- 172.21.15.5:0/3602759039 >> 172.21.15.5:0/3602759039 conn(0x55960eec0a30 msgr2=0x55960eecb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.645+0000 7fc167d29740 1 -- 172.21.15.5:0/3602759039 shutdown_connections 2024-09-19T06:27:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.645+0000 7fc167d29740 1 -- 172.21.15.5:0/3602759039 wait complete. 2024-09-19T06:27:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.645+0000 7fc167d29740 1 -- start start 2024-09-19T06:27:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.645+0000 7fc167d29740 1 Processor -- start 2024-09-19T06:27:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.646+0000 7fc167d29740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec44a0 0x55960ef4a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.646+0000 7fc167d29740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eec4dc0 0x55960ef4ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.646+0000 7fc167d29740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eecf680 0x55960eeec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.646+0000 7fc167d29740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601acaf0 con 0x55960eec44a0 2024-09-19T06:27:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.646+0000 7fc167d29740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601ac420 con 0x55960eecf680 2024-09-19T06:27:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.646+0000 7fc167d29740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601a9680 con 0x55960eec4dc0 2024-09-19T06:27:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.646+0000 7fc167d29740 10 client.0 _ll_register_callbacks cb 0x7fc150040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.646+0000 7fc15e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec44a0 0x55960ef4a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.646+0000 7fc15e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eecf680 0x55960eeec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.646+0000 7fc15e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eecf680 0x55960eeec190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44726/0 (socket says 172.21.15.5:44726) 2024-09-19T06:27:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.646+0000 7fc15e7fc640 1 -- 172.21.15.5:0/355249689 learned_addr learned my addr 172.21.15.5:0/355249689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.646+0000 7fc15effd640 1 --2- 172.21.15.5:0/355249689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eec4dc0 0x55960ef4ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc15e7fc640 1 -- 172.21.15.5:0/355249689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eec4dc0 msgr2=0x55960ef4ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc15e7fc640 1 --2- 172.21.15.5:0/355249689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eec4dc0 0x55960ef4ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc15e7fc640 1 -- 172.21.15.5:0/355249689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec44a0 msgr2=0x55960ef4a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc15e7fc640 1 --2- 172.21.15.5:0/355249689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec44a0 0x55960ef4a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc15e7fc640 1 -- 172.21.15.5:0/355249689 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55960eeec8d0 con 0x55960eecf680 2024-09-19T06:27:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc15e7fc640 1 --2- 172.21.15.5:0/355249689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eecf680 0x55960eeec190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc154012560 tx=0x7fc1540045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb96a5e0f0727e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc13effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:27:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc13effd640 1 -- 172.21.15.5:0/355249689 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154009df0 con 0x55960eecf680 2024-09-19T06:27:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc167d29740 1 -- 172.21.15.5:0/355249689 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc154009bb0 con 0x55960eecf680 2024-09-19T06:27:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc13effd640 1 -- 172.21.15.5:0/355249689 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7fc1540035f0 con 0x55960eecf680 2024-09-19T06:27:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc13effd640 1 -- 172.21.15.5:0/355249689 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc1540037f0 con 0x55960eecf680 2024-09-19T06:27:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc167d29740 10 client.30773 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.647+0000 7fc167d29740 1 -- 172.21.15.5:0/355249689 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55960ef51f50 con 0x55960eecf680 2024-09-19T06:27:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.648+0000 7fc13effd640 1 -- 172.21.15.5:0/355249689 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc154009bb0 con 0x55960eecf680 2024-09-19T06:27:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.648+0000 7fc13effd640 1 -- 172.21.15.5:0/355249689 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc154028020 con 0x55960eecf680 2024-09-19T06:27:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.648+0000 7fc167d29740 10 client.30773 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.648+0000 7fc167d29740 10 client.30773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.648+0000 7fc167d29740 20 client.30773 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.648+0000 7fc167d29740 1 --2- 172.21.15.5:0/355249689 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55960ef52620 0x55960ef54a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.648+0000 7fc167d29740 4 client.30773 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.648+0000 7fc167d29740 1 -- 172.21.15.5:0/355249689 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55960ee31d60 con 0x55960ef52620 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.648+0000 7fc15effd640 1 --2- 172.21.15.5:0/355249689 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55960ef52620 0x55960ef54a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.648+0000 7fc13effd640 10 client.30773 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.648+0000 7fc15effd640 1 --2- 172.21.15.5:0/355249689 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55960ef52620 0x55960ef54a40 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee320138711910f server_cookie=e8022b2d2cbc4220 in_seq=0 out_seq=0 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc13effd640 1 -- 172.21.15.5:0/355249689 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55960ee31d60 con 0x55960ef52620 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc13effd640 10 client.30773 handle_command_reply: tid=0 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc167d29740 1 client.30773 shutdown 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc167d29740 1 -- 172.21.15.5:0/355249689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eecf680 msgr2=0x55960eeec190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc167d29740 1 --2- 172.21.15.5:0/355249689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eecf680 0x55960eeec190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc154012560 tx=0x7fc1540045e0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc167d29740 1 -- 172.21.15.5:0/355249689 shutdown_connections 2024-09-19T06:27:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc167d29740 1 --2- 172.21.15.5:0/355249689 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55960ef52620 0x55960ef54a40 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc167d29740 1 --2- 172.21.15.5:0/355249689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55960eecf680 0x55960eeec190 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc167d29740 1 --2- 172.21.15.5:0/355249689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55960eec4dc0 0x55960ef4ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc13effd640 0 client.30773 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc167d29740 1 --2- 172.21.15.5:0/355249689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55960eec44a0 0x55960ef4a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc167d29740 1 -- 172.21.15.5:0/355249689 >> 172.21.15.5:0/355249689 conn(0x55960eec0a30 msgr2=0x55960eebf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc167d29740 1 -- 172.21.15.5:0/355249689 shutdown_connections 2024-09-19T06:27:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc167d29740 1 -- 172.21.15.5:0/355249689 wait complete. 2024-09-19T06:27:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.650+0000 7fc167d29740 20 client.30773 trim_cache size 0 max 16384 2024-09-19T06:27:35.656 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:27:35.656 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:27:35.656 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:27:35.656 INFO:teuthology.orchestra.run.smithi005.stdout: "49657249-7c60-436c-89e9-7f9d274f170d": { 2024-09-19T06:27:35.656 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:27:35.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "49657249-7c60-436c-89e9-7f9d274f170d", 2024-09-19T06:27:35.656 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:27:35.656 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:27:35.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:27:35.657 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:27:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.651+0000 7fc165950640 1 -- 172.21.15.5:0/3222288520 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1480a34b0 msgr2=0x7fc1480a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.651+0000 7fc165950640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1480a34b0 0x7fc1480a5970 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc1580a4d00 tx=0x7fc15800c890 comp rx=0 tx=0).stop 2024-09-19T06:27:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.651+0000 7fc165950640 1 -- 172.21.15.5:0/3222288520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1601082a0 msgr2=0x7fc16019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.651+0000 7fc165950640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1601082a0 0x7fc16019cde0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc158009fd0 tx=0x7fc15800ef10 comp rx=0 tx=0).stop 2024-09-19T06:27:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.651+0000 7fc165950640 1 -- 172.21.15.5:0/3222288520 shutdown_connections 2024-09-19T06:27:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.651+0000 7fc165950640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1601082a0 0x7fc16019cde0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.651+0000 7fc165950640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc1480a34b0 0x7fc1480a5970 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.651+0000 7fc165950640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1601036e0 0x7fc160198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.651+0000 7fc165950640 1 --2- 172.21.15.5:0/3222288520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc160102d30 0x7fc160198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.651+0000 7fc165950640 1 -- 172.21.15.5:0/3222288520 >> 172.21.15.5:0/3222288520 conn(0x7fc1600fc7d0 msgr2=0x7fc160105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.651+0000 7fc165950640 1 -- 172.21.15.5:0/3222288520 shutdown_connections 2024-09-19T06:27:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:27:35.651+0000 7fc165950640 1 -- 172.21.15.5:0/3222288520 wait complete. 2024-09-19T06:27:35.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:35 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:35.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:35 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:36.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:35 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:27:36.397 INFO:tasks.cephfs.filesystem:scrub status for tag:49657249-7c60-436c-89e9-7f9d274f170d - {'path': '/', 'tag': '49657249-7c60-436c-89e9-7f9d274f170d', 'options': 'recursive,force'} 2024-09-19T06:27:36.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 105894 8.53 MB/sec execute 540 sec latency 713.999 ms 2024-09-19T06:27:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:36 smithi005 ceph-mon[30955]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 218 op/s 2024-09-19T06:27:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:36 smithi049 ceph-mon[34640]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 218 op/s 2024-09-19T06:27:37.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:36 smithi157 ceph-mon[35346]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 218 op/s 2024-09-19T06:27:37.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 106076 8.53 MB/sec execute 541 sec latency 111.769 ms 2024-09-19T06:27:38.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 106254 8.54 MB/sec execute 542 sec latency 52.714 ms 2024-09-19T06:27:38.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:38 smithi005 ceph-mon[30955]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.3 MiB/s wr, 411 op/s 2024-09-19T06:27:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:38 smithi049 ceph-mon[34640]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.3 MiB/s wr, 411 op/s 2024-09-19T06:27:39.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:38 smithi157 ceph-mon[35346]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.3 MiB/s wr, 411 op/s 2024-09-19T06:27:39.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 106426 8.55 MB/sec execute 543 sec latency 114.173 ms 2024-09-19T06:27:40.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 106580 8.54 MB/sec execute 544 sec latency 58.425 ms 2024-09-19T06:27:40.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:40 smithi005 ceph-mon[30955]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 312 op/s 2024-09-19T06:27:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:40 smithi049 ceph-mon[34640]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 312 op/s 2024-09-19T06:27:41.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:40 smithi157 ceph-mon[35346]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 312 op/s 2024-09-19T06:27:41.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 106719 8.54 MB/sec execute 545 sec latency 130.219 ms 2024-09-19T06:27:42.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 106880 8.54 MB/sec execute 546 sec latency 64.778 ms 2024-09-19T06:27:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:42 smithi005 ceph-mon[30955]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 312 op/s 2024-09-19T06:27:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:42 smithi049 ceph-mon[34640]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 312 op/s 2024-09-19T06:27:43.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:42 smithi157 ceph-mon[35346]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 312 op/s 2024-09-19T06:27:43.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107043 8.54 MB/sec execute 547 sec latency 93.555 ms 2024-09-19T06:27:44.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107178 8.53 MB/sec execute 548 sec latency 61.823 ms 2024-09-19T06:27:44.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:44 smithi005 ceph-mon[30955]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.1 MiB/s wr, 435 op/s 2024-09-19T06:27:44.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:27:44.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:27:44.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:44 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:27:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:44 smithi049 ceph-mon[34640]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.1 MiB/s wr, 435 op/s 2024-09-19T06:27:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:27:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:27:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:44 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:27:45.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:44 smithi157 ceph-mon[35346]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.1 MiB/s wr, 435 op/s 2024-09-19T06:27:45.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:27:45.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:27:45.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:44 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:27:45.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107319 8.53 MB/sec execute 549 sec latency 74.882 ms 2024-09-19T06:27:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.799+0000 7f319d94b640 1 -- 172.21.15.5:0/1546603072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31981069b0 msgr2=0x7f3198106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.799+0000 7f319d94b640 1 --2- 172.21.15.5:0/1546603072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31981069b0 0x7f3198106d90 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f3188009930 tx=0x7f318802a420 comp rx=0 tx=0).stop 2024-09-19T06:27:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.800+0000 7f319d94b640 1 -- 172.21.15.5:0/1546603072 shutdown_connections 2024-09-19T06:27:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.800+0000 7f319d94b640 1 --2- 172.21.15.5:0/1546603072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3198101340 0x7f319810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.800+0000 7f319d94b640 1 --2- 172.21.15.5:0/1546603072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31981009a0 0x7f3198100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.800+0000 7f319d94b640 1 --2- 172.21.15.5:0/1546603072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31981069b0 0x7f3198106d90 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.800+0000 7f319d94b640 1 -- 172.21.15.5:0/1546603072 >> 172.21.15.5:0/1546603072 conn(0x7f31980779a0 msgr2=0x7f31980fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.800+0000 7f319d94b640 1 -- 172.21.15.5:0/1546603072 shutdown_connections 2024-09-19T06:27:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.800+0000 7f319d94b640 1 -- 172.21.15.5:0/1546603072 wait complete. 2024-09-19T06:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.800+0000 7f319d94b640 1 Processor -- start 2024-09-19T06:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.801+0000 7f319d94b640 1 -- start start 2024-09-19T06:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.801+0000 7f319d94b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31981009a0 0x7f31981078d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.801+0000 7f319d94b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198101340 0x7f3198107e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.801+0000 7f319d94b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31981069b0 0x7f3198108350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.801+0000 7f319d94b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3198110c70 con 0x7f3198101340 2024-09-19T06:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.801+0000 7f319d94b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3198110af0 con 0x7f31981069b0 2024-09-19T06:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.801+0000 7f319d94b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3198110df0 con 0x7f31981009a0 2024-09-19T06:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.801+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31981009a0 0x7f31981078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.801+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31981009a0 0x7f31981078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57620/0 (socket says 172.21.15.5:57620) 2024-09-19T06:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.801+0000 7f3196ffd640 1 -- 172.21.15.5:0/2656761369 learned_addr learned my addr 172.21.15.5:0/2656761369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.801+0000 7f31967fc640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198101340 0x7f3198107e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.802+0000 7f31977fe640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31981069b0 0x7f3198108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.802+0000 7f3196ffd640 1 -- 172.21.15.5:0/2656761369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31981069b0 msgr2=0x7f3198108350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.802+0000 7f3196ffd640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31981069b0 0x7f3198108350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.802+0000 7f3196ffd640 1 -- 172.21.15.5:0/2656761369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198101340 msgr2=0x7f3198107e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.802+0000 7f3196ffd640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198101340 0x7f3198107e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.802+0000 7f3196ffd640 1 -- 172.21.15.5:0/2656761369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31981aab70 con 0x7f31981009a0 2024-09-19T06:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.802+0000 7f31967fc640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198101340 0x7f3198107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.802+0000 7f3196ffd640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31981009a0 0x7f31981078d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3188005940 tx=0x7f31880042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.802+0000 7f319c949640 1 -- 172.21.15.5:0/2656761369 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318803f030 con 0x7f31981009a0 2024-09-19T06:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.802+0000 7f319c949640 1 -- 172.21.15.5:0/2656761369 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31880318a0 con 0x7f31981009a0 2024-09-19T06:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.802+0000 7f319c949640 1 -- 172.21.15.5:0/2656761369 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188035070 con 0x7f31981009a0 2024-09-19T06:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.802+0000 7f319d94b640 1 -- 172.21.15.5:0/2656761369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31981aae00 con 0x7f31981009a0 2024-09-19T06:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.803+0000 7f319d94b640 1 -- 172.21.15.5:0/2656761369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31981ab1b0 con 0x7f31981009a0 2024-09-19T06:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.805+0000 7f31a07c1740 1 Processor -- start 2024-09-19T06:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.805+0000 7f31a07c1740 1 -- start start 2024-09-19T06:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.806+0000 7f31a07c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a2af0 0x56433d9a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.806+0000 7f31a07c1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a3410 0x56433d9ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.806+0000 7f31a07c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 0x56433d9b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.806+0000 7f31a07c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56433d9072f0 con 0x56433d9adcb0 2024-09-19T06:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.806+0000 7f31a07c1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56433d8af270 con 0x56433d9a3410 2024-09-19T06:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.806+0000 7f31a07c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56433d9032b0 con 0x56433d9a2af0 2024-09-19T06:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.806+0000 7f31967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 0x56433d9b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.806+0000 7f31977fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a2af0 0x56433d9a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.806+0000 7f31967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 0x56433d9b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48126/0 (socket says 172.21.15.5:48126) 2024-09-19T06:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.806+0000 7f31967fc640 1 -- 172.21.15.5:0/2077205722 learned_addr learned my addr 172.21.15.5:0/2077205722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.806+0000 7f319c949640 1 -- 172.21.15.5:0/2656761369 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3188042030 con 0x7f31981009a0 2024-09-19T06:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.807+0000 7f31967fc640 1 -- 172.21.15.5:0/2077205722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a2af0 msgr2=0x56433d9a2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.807+0000 7f31967fc640 1 --2- 172.21.15.5:0/2077205722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a2af0 0x56433d9a2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.807+0000 7f31967fc640 1 -- 172.21.15.5:0/2077205722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a3410 msgr2=0x56433d9ad770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.807+0000 7f31967fc640 1 --2- 172.21.15.5:0/2077205722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a3410 0x56433d9ad770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.807+0000 7f31967fc640 1 -- 172.21.15.5:0/2077205722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56433d8ae340 con 0x56433d9adcb0 2024-09-19T06:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.807+0000 7f31967fc640 1 --2- 172.21.15.5:0/2077205722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 0x56433d9b00a0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f318c010480 tx=0x7f318c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3b0767c9dab89f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.807+0000 7f317dffb640 1 -- 172.21.15.5:0/2077205722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c011760 con 0x56433d9adcb0 2024-09-19T06:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.807+0000 7f317dffb640 1 -- 172.21.15.5:0/2077205722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f318c011db0 con 0x56433d9adcb0 2024-09-19T06:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.807+0000 7f317dffb640 1 -- 172.21.15.5:0/2077205722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c01a380 con 0x56433d9adcb0 2024-09-19T06:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.807+0000 7f319c949640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f31800a34b0 0x7f31800a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.807+0000 7f31977fe640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f31800a34b0 0x7f31800a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.808+0000 7f319c949640 1 -- 172.21.15.5:0/2656761369 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f31880f0080 con 0x7f31981009a0 2024-09-19T06:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.809+0000 7f31a07c1740 1 -- 172.21.15.5:0/2077205722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 msgr2=0x56433d9b00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.809+0000 7f31977fe640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f31800a34b0 0x7f31800a5970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3190005fd0 tx=0x7f3190009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.809+0000 7f31a07c1740 1 --2- 172.21.15.5:0/2077205722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 0x56433d9b00a0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f318c010480 tx=0x7f318c00bf90 comp rx=0 tx=0).stop 2024-09-19T06:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.810+0000 7f31a07c1740 1 -- 172.21.15.5:0/2077205722 shutdown_connections 2024-09-19T06:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.810+0000 7f31a07c1740 1 --2- 172.21.15.5:0/2077205722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 0x56433d9b00a0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.810+0000 7f31a07c1740 1 --2- 172.21.15.5:0/2077205722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a3410 0x56433d9ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.810+0000 7f31a07c1740 1 --2- 172.21.15.5:0/2077205722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a2af0 0x56433d9a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.810+0000 7f31a07c1740 1 -- 172.21.15.5:0/2077205722 >> 172.21.15.5:0/2077205722 conn(0x56433d99f060 msgr2=0x56433d99f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.810+0000 7f31a07c1740 1 -- 172.21.15.5:0/2077205722 shutdown_connections 2024-09-19T06:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.810+0000 7f31a07c1740 1 -- 172.21.15.5:0/2077205722 wait complete. 2024-09-19T06:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.810+0000 7f31a07c1740 1 -- start start 2024-09-19T06:27:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f31a07c1740 1 Processor -- start 2024-09-19T06:27:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f31a07c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9a2af0 0x56433d9aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f31a07c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a3410 0x56433d9ab180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f31a07c1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9adcb0 0x56433d9ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f31a07c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56433d9b0880 con 0x56433d9a2af0 2024-09-19T06:27:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f31a07c1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56433d890290 con 0x56433d9adcb0 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f31a07c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56433d9b0a00 con 0x56433d9a3410 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9a2af0 0x56433d9aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f31a07c1740 10 client.0 _ll_register_callbacks cb 0x56433d8c1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9a2af0 0x56433d9aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48130/0 (socket says 172.21.15.5:48130) 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f3196ffd640 1 -- 172.21.15.5:0/399970484 learned_addr learned my addr 172.21.15.5:0/399970484 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f31967fc640 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a3410 0x56433d9ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.811+0000 7f31977fe640 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9adcb0 0x56433d9ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.812+0000 7f3196ffd640 1 -- 172.21.15.5:0/399970484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a3410 msgr2=0x56433d9ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.812+0000 7f3196ffd640 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a3410 0x56433d9ab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.812+0000 7f3196ffd640 1 -- 172.21.15.5:0/399970484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9adcb0 msgr2=0x56433d9ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.812+0000 7f3196ffd640 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9adcb0 0x56433d9ab6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.812+0000 7f3196ffd640 1 -- 172.21.15.5:0/399970484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56433d61a210 con 0x56433d9a2af0 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.812+0000 7f31967fc640 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a3410 0x56433d9ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.812+0000 7f3196ffd640 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9a2af0 0x56433d9aac40 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f3188031440 tx=0x7f31880bbf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cdadfdcb80623d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.812+0000 7f3176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:27:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.812+0000 7f3176ffd640 1 -- 172.21.15.5:0/399970484 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188049050 con 0x56433d9a2af0 2024-09-19T06:27:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.812+0000 7f3176ffd640 1 -- 172.21.15.5:0/399970484 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f31880c4ac0 con 0x56433d9a2af0 2024-09-19T06:27:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.812+0000 7f31a07c1740 1 -- 172.21.15.5:0/399970484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f318c011db0 con 0x56433d9a2af0 2024-09-19T06:27:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.813+0000 7f3176ffd640 1 -- 172.21.15.5:0/399970484 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f31880edbd0 con 0x56433d9a2af0 2024-09-19T06:27:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.813+0000 7f31a07c1740 10 client.21420 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.813+0000 7f31a07c1740 1 -- 172.21.15.5:0/399970484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x56433d6e6620 con 0x56433d9a2af0 2024-09-19T06:27:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.813+0000 7f3176ffd640 1 -- 172.21.15.5:0/399970484 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f318c011db0 con 0x56433d9a2af0 2024-09-19T06:27:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.813+0000 7f3176ffd640 1 -- 172.21.15.5:0/399970484 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f31880d50d0 con 0x56433d9a2af0 2024-09-19T06:27:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.813+0000 7f31a07c1740 10 client.21420 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.813+0000 7f31a07c1740 10 client.21420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.813+0000 7f31a07c1740 20 client.21420 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.813+0000 7f31a07c1740 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56433d9bdc40 0x56433d9c0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.813+0000 7f31a07c1740 4 client.21420 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.813+0000 7f31a07c1740 1 -- 172.21.15.5:0/399970484 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56433d914a20 con 0x56433d9bdc40 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.814+0000 7f31967fc640 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56433d9bdc40 0x56433d9c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.814+0000 7f31967fc640 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56433d9bdc40 0x56433d9c0030 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad559cd2ea3f155 server_cookie=1f44989aa3ebcf88 in_seq=0 out_seq=0 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.814+0000 7f3176ffd640 10 client.21420 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.815+0000 7f3176ffd640 1 -- 172.21.15.5:0/399970484 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56433d914a20 con 0x56433d9bdc40 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.815+0000 7f3176ffd640 10 client.21420 handle_command_reply: tid=0 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.815+0000 7f31a07c1740 1 client.21420 shutdown 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.815+0000 7f31a07c1740 1 -- 172.21.15.5:0/399970484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9a2af0 msgr2=0x56433d9aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.815+0000 7f31a07c1740 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9a2af0 0x56433d9aac40 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f3188031440 tx=0x7f31880bbf90 comp rx=0 tx=0).stop 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.815+0000 7f31a07c1740 1 -- 172.21.15.5:0/399970484 shutdown_connections 2024-09-19T06:27:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.815+0000 7f31a07c1740 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56433d9bdc40 0x56433d9c0030 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.816+0000 7f31a07c1740 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9adcb0 0x56433d9ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.816+0000 7f31a07c1740 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a3410 0x56433d9ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.816+0000 7f31a07c1740 1 --2- 172.21.15.5:0/399970484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9a2af0 0x56433d9aac40 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.816+0000 7f31a07c1740 1 -- 172.21.15.5:0/399970484 >> 172.21.15.5:0/399970484 conn(0x56433d99f060 msgr2=0x56433d9a4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.816+0000 7f3176ffd640 0 client.21420 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.816+0000 7f31a07c1740 1 -- 172.21.15.5:0/399970484 shutdown_connections 2024-09-19T06:27:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.816+0000 7f31a07c1740 1 -- 172.21.15.5:0/399970484 wait complete. 2024-09-19T06:27:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.816+0000 7f31a07c1740 20 client.21420 trim_cache size 0 max 16384 2024-09-19T06:27:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.846+0000 7f31a07c1740 1 Processor -- start 2024-09-19T06:27:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.846+0000 7f31a07c1740 1 -- start start 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.846+0000 7f31a07c1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 0x56433d9ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.846+0000 7f31a07c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a3410 0x56433d9acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.847+0000 7f31a07c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 0x56433d9c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.847+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 0x56433d9ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.847+0000 7f31967fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a3410 0x56433d9acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.847+0000 7f31967fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a3410 0x56433d9acbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57648/0 (socket says 172.21.15.5:57648) 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.847+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 0x56433d9ab4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:52108/0 (socket says 172.21.15.5:52108) 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.847+0000 7f31977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 0x56433d9c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.847+0000 7f31977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 0x56433d9c58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48138/0 (socket says 172.21.15.5:48138) 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.847+0000 7f31967fc640 1 -- 172.21.15.5:0/4027112019 learned_addr learned my addr 172.21.15.5:0/4027112019 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.847+0000 7f31a07c1740 1 -- 172.21.15.5:0/4027112019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31981b3e40 con 0x56433d9adcb0 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.847+0000 7f31a07c1740 1 -- 172.21.15.5:0/4027112019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f31981b5980 con 0x56433d9a2af0 2024-09-19T06:27:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.847+0000 7f31a07c1740 1 -- 172.21.15.5:0/4027112019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f31981b4510 con 0x56433d9a3410 2024-09-19T06:27:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.848+0000 7f3196ffd640 1 -- 172.21.15.5:0/4027112019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a3410 msgr2=0x56433d9acbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.848+0000 7f3196ffd640 1 --2- 172.21.15.5:0/4027112019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a3410 0x56433d9acbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.848+0000 7f3196ffd640 1 -- 172.21.15.5:0/4027112019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 msgr2=0x56433d9c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.848+0000 7f3196ffd640 1 --2- 172.21.15.5:0/4027112019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 0x56433d9c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.848+0000 7f3196ffd640 1 -- 172.21.15.5:0/4027112019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56433d99a2d0 con 0x56433d9a2af0 2024-09-19T06:27:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.848+0000 7f3196ffd640 1 --2- 172.21.15.5:0/4027112019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 0x56433d9ab4f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3188009900 tx=0x7f31880bc000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ce4d1dc518adb4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.848+0000 7f317d7fa640 1 -- 172.21.15.5:0/4027112019 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188003e20 con 0x56433d9a2af0 2024-09-19T06:27:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.848+0000 7f317d7fa640 1 -- 172.21.15.5:0/4027112019 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31880bc390 con 0x56433d9a2af0 2024-09-19T06:27:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.848+0000 7f317d7fa640 1 -- 172.21.15.5:0/4027112019 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31880c4ce0 con 0x56433d9a2af0 2024-09-19T06:27:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.849+0000 7f31a07c1740 1 -- 172.21.15.5:0/4027112019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 msgr2=0x56433d9ab4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.849+0000 7f31a07c1740 1 --2- 172.21.15.5:0/4027112019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 0x56433d9ab4f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3188009900 tx=0x7f31880bc000 comp rx=0 tx=0).stop 2024-09-19T06:27:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.849+0000 7f31a07c1740 1 -- 172.21.15.5:0/4027112019 shutdown_connections 2024-09-19T06:27:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.849+0000 7f31a07c1740 1 --2- 172.21.15.5:0/4027112019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9adcb0 0x56433d9c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.849+0000 7f31a07c1740 1 --2- 172.21.15.5:0/4027112019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9a3410 0x56433d9acbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.849+0000 7f31a07c1740 1 --2- 172.21.15.5:0/4027112019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 0x56433d9ab4f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.849+0000 7f31a07c1740 1 -- 172.21.15.5:0/4027112019 >> 172.21.15.5:0/4027112019 conn(0x56433d99f060 msgr2=0x56433d9ac4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.849+0000 7f31a07c1740 1 -- 172.21.15.5:0/4027112019 shutdown_connections 2024-09-19T06:27:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.849+0000 7f31a07c1740 1 -- 172.21.15.5:0/4027112019 wait complete. 2024-09-19T06:27:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.849+0000 7f31a07c1740 1 -- start start 2024-09-19T06:27:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.850+0000 7f31a07c1740 1 Processor -- start 2024-09-19T06:27:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.850+0000 7f31a07c1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 0x56433d9b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.850+0000 7f31a07c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9a3410 0x56433d9b6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.850+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 0x56433d9b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.850+0000 7f31a07c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9adcb0 0x56433d9a7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.850+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 0x56433d9b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:52114/0 (socket says 172.21.15.5:52114) 2024-09-19T06:27:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f31967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9a3410 0x56433d9b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f31a07c1740 1 -- 172.21.15.5:0/3508742396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31981b52b0 con 0x56433d9a3410 2024-09-19T06:27:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f3196ffd640 1 -- 172.21.15.5:0/3508742396 learned_addr learned my addr 172.21.15.5:0/3508742396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f31a07c1740 1 -- 172.21.15.5:0/3508742396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f31981b4be0 con 0x56433d9a2af0 2024-09-19T06:27:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f31a07c1740 1 -- 172.21.15.5:0/3508742396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f31981b1e40 con 0x56433d9adcb0 2024-09-19T06:27:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f31977fe640 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9adcb0 0x56433d9a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f31a07c1740 10 client.0 _ll_register_callbacks cb 0x7f31880bc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f3196ffd640 1 -- 172.21.15.5:0/3508742396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9adcb0 msgr2=0x56433d9a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f3196ffd640 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9adcb0 0x56433d9a7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f3196ffd640 1 -- 172.21.15.5:0/3508742396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9a3410 msgr2=0x56433d9b6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f3196ffd640 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9a3410 0x56433d9b6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f3196ffd640 1 -- 172.21.15.5:0/3508742396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56433d9a77a0 con 0x56433d9a2af0 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f31977fe640 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9adcb0 0x56433d9a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f3196ffd640 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 0x56433d9b5cf0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f31880c0310 tx=0x7f31880bbf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70eae164d53d9244 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f3176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.851+0000 7f3176ffd640 1 -- 172.21.15.5:0/3508742396 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31880edbe0 con 0x56433d9a2af0 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f3176ffd640 1 -- 172.21.15.5:0/3508742396 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f31880c4cc0 con 0x56433d9a2af0 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f31a07c1740 1 -- 172.21.15.5:0/3508742396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31880bc390 con 0x56433d9a2af0 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f3176ffd640 1 -- 172.21.15.5:0/3508742396 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f3188003e20 con 0x56433d9a2af0 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f31a07c1740 10 client.30788 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f31a07c1740 1 -- 172.21.15.5:0/3508742396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x56433d9a77a0 con 0x56433d9a2af0 2024-09-19T06:27:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f3176ffd640 1 -- 172.21.15.5:0/3508742396 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31880c4f50 con 0x56433d9a2af0 2024-09-19T06:27:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f3176ffd640 1 -- 172.21.15.5:0/3508742396 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f318803bab0 con 0x56433d9a2af0 2024-09-19T06:27:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f31a07c1740 10 client.30788 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f31a07c1740 10 client.30788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f31a07c1740 20 client.30788 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f31a07c1740 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56433da28360 0x56433da2a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f31a07c1740 4 client.30788 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:27:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f31a07c1740 1 -- 172.21.15.5:0/3508742396 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f31880bc390 con 0x56433da28360 2024-09-19T06:27:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.852+0000 7f31967fc640 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56433da28360 0x56433da2a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.853+0000 7f3176ffd640 10 client.30788 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.853+0000 7f31967fc640 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56433da28360 0x56433da2a750 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21012539206192a1 server_cookie=1676a68eb587ab08 in_seq=0 out_seq=0 2024-09-19T06:27:45.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:45 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:27:45.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:45 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:27:45.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:45 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:27:45.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:45 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:27:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.965+0000 7f3176ffd640 1 -- 172.21.15.5:0/3508742396 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f31880bc390 con 0x56433da28360 2024-09-19T06:27:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.965+0000 7f3176ffd640 10 client.30788 handle_command_reply: tid=0 2024-09-19T06:27:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.965+0000 7f31a07c1740 1 client.30788 shutdown 2024-09-19T06:27:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.965+0000 7f31a07c1740 1 -- 172.21.15.5:0/3508742396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 msgr2=0x56433d9b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.965+0000 7f31a07c1740 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 0x56433d9b5cf0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f31880c0310 tx=0x7f31880bbf90 comp rx=0 tx=0).stop 2024-09-19T06:27:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.966+0000 7f31a07c1740 1 -- 172.21.15.5:0/3508742396 shutdown_connections 2024-09-19T06:27:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.966+0000 7f31a07c1740 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x56433da28360 0x56433da2a750 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.966+0000 7f31a07c1740 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56433d9adcb0 0x56433d9a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.966+0000 7f31a07c1740 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56433d9a3410 0x56433d9b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.966+0000 7f31a07c1740 1 --2- 172.21.15.5:0/3508742396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56433d9a2af0 0x56433d9b5cf0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.966+0000 7f3176ffd640 0 client.30788 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.966+0000 7f31a07c1740 1 -- 172.21.15.5:0/3508742396 >> 172.21.15.5:0/3508742396 conn(0x56433d99f060 msgr2=0x56433d9a4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.966+0000 7f31a07c1740 1 -- 172.21.15.5:0/3508742396 shutdown_connections 2024-09-19T06:27:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.966+0000 7f31a07c1740 1 -- 172.21.15.5:0/3508742396 wait complete. 2024-09-19T06:27:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.966+0000 7f31a07c1740 20 client.30788 trim_cache size 0 max 16384 2024-09-19T06:27:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:27:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:27:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:27:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:27:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.967+0000 7f319d94b640 1 -- 172.21.15.5:0/2656761369 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f31800a34b0 msgr2=0x7f31800a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.967+0000 7f319d94b640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f31800a34b0 0x7f31800a5970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3190005fd0 tx=0x7f3190009500 comp rx=0 tx=0).stop 2024-09-19T06:27:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.967+0000 7f319d94b640 1 -- 172.21.15.5:0/2656761369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31981009a0 msgr2=0x7f31981078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.967+0000 7f319d94b640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31981009a0 0x7f31981078d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3188005940 tx=0x7f31880042f0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.967+0000 7f319d94b640 1 -- 172.21.15.5:0/2656761369 shutdown_connections 2024-09-19T06:27:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.967+0000 7f319d94b640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31981069b0 0x7f3198108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.967+0000 7f319d94b640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198101340 0x7f3198107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.967+0000 7f319d94b640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f31800a34b0 0x7f31800a5970 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.967+0000 7f319d94b640 1 --2- 172.21.15.5:0/2656761369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31981009a0 0x7f31981078d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.967+0000 7f319d94b640 1 -- 172.21.15.5:0/2656761369 >> 172.21.15.5:0/2656761369 conn(0x7f31980779a0 msgr2=0x7f3198105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.967+0000 7f319d94b640 1 -- 172.21.15.5:0/2656761369 shutdown_connections 2024-09-19T06:27:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:45.967+0000 7f319d94b640 1 -- 172.21.15.5:0/2656761369 wait complete. 2024-09-19T06:27:46.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:45 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:27:46.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:45 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:27:46.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107449 8.52 MB/sec execute 550 sec latency 108.486 ms 2024-09-19T06:27:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:46 smithi049 ceph-mon[34640]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 316 op/s 2024-09-19T06:27:47.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:46 smithi157 ceph-mon[35346]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 316 op/s 2024-09-19T06:27:47.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:46 smithi005 ceph-mon[30955]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 316 op/s 2024-09-19T06:27:47.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107596 8.52 MB/sec execute 551 sec latency 91.794 ms 2024-09-19T06:27:47.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:47 smithi049 ceph-mon[34640]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.9 MiB/s wr, 459 op/s 2024-09-19T06:27:48.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:47 smithi157 ceph-mon[35346]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.9 MiB/s wr, 459 op/s 2024-09-19T06:27:48.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:47 smithi005 ceph-mon[30955]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.9 MiB/s wr, 459 op/s 2024-09-19T06:27:48.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107773 8.52 MB/sec execute 552 sec latency 122.231 ms 2024-09-19T06:27:49.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107979 8.53 MB/sec execute 553 sec latency 71.021 ms 2024-09-19T06:27:50.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:50 smithi157 ceph-mon[35346]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 266 op/s 2024-09-19T06:27:50.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:50 smithi005 ceph-mon[30955]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 266 op/s 2024-09-19T06:27:50.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:50 smithi049 ceph-mon[34640]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 266 op/s 2024-09-19T06:27:50.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 108175 8.53 MB/sec execute 554 sec latency 61.640 ms 2024-09-19T06:27:51.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 108320 8.53 MB/sec execute 555 sec latency 54.888 ms 2024-09-19T06:27:52.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:52 smithi157 ceph-mon[35346]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 266 op/s 2024-09-19T06:27:52.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:52 smithi005 ceph-mon[30955]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 266 op/s 2024-09-19T06:27:52.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:52 smithi049 ceph-mon[34640]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 266 op/s 2024-09-19T06:27:52.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 108463 8.53 MB/sec execute 556 sec latency 59.316 ms 2024-09-19T06:27:53.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 108647 8.54 MB/sec execute 557 sec latency 106.243 ms 2024-09-19T06:27:54.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:54 smithi157 ceph-mon[35346]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.1 MiB/s wr, 393 op/s 2024-09-19T06:27:54.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:54 smithi005 ceph-mon[30955]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.1 MiB/s wr, 393 op/s 2024-09-19T06:27:54.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:54 smithi049 ceph-mon[34640]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.1 MiB/s wr, 393 op/s 2024-09-19T06:27:54.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 108836 8.54 MB/sec execute 558 sec latency 82.270 ms 2024-09-19T06:27:55.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 108983 8.53 MB/sec execute 559 sec latency 116.889 ms 2024-09-19T06:27:56.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:56 smithi157 ceph-mon[35346]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 270 op/s 2024-09-19T06:27:56.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:56 smithi005 ceph-mon[30955]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 270 op/s 2024-09-19T06:27:56.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:56 smithi049 ceph-mon[34640]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 270 op/s 2024-09-19T06:27:56.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 109150 8.54 MB/sec execute 560 sec latency 58.990 ms 2024-09-19T06:27:57.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 109297 8.53 MB/sec execute 561 sec latency 60.965 ms 2024-09-19T06:27:58.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:27:58 smithi157 ceph-mon[35346]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.8 MiB/s wr, 384 op/s 2024-09-19T06:27:58.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:27:58 smithi005 ceph-mon[30955]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.8 MiB/s wr, 384 op/s 2024-09-19T06:27:58.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:27:58 smithi049 ceph-mon[34640]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.8 MiB/s wr, 384 op/s 2024-09-19T06:27:58.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 109455 8.54 MB/sec execute 562 sec latency 114.058 ms 2024-09-19T06:27:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.102+0000 7f3876149640 1 -- 172.21.15.5:0/945230244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38701022d0 msgr2=0x7f3870106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.102+0000 7f3876149640 1 --2- 172.21.15.5:0/945230244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38701022d0 0x7f3870106700 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3864009900 tx=0x7f386402a3f0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.102+0000 7f3876149640 1 -- 172.21.15.5:0/945230244 shutdown_connections 2024-09-19T06:27:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.102+0000 7f3876149640 1 --2- 172.21.15.5:0/945230244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3870106e60 0x7f387010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.102+0000 7f3876149640 1 --2- 172.21.15.5:0/945230244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38701022d0 0x7f3870106700 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.102+0000 7f3876149640 1 --2- 172.21.15.5:0/945230244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3870101920 0x7f3870101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.102+0000 7f3876149640 1 -- 172.21.15.5:0/945230244 >> 172.21.15.5:0/945230244 conn(0x7f3870075eb0 msgr2=0x7f38700762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.102+0000 7f3876149640 1 -- 172.21.15.5:0/945230244 shutdown_connections 2024-09-19T06:27:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.102+0000 7f3876149640 1 -- 172.21.15.5:0/945230244 wait complete. 2024-09-19T06:27:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f3876149640 1 Processor -- start 2024-09-19T06:27:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f3876149640 1 -- start start 2024-09-19T06:27:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f3876149640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3870101920 0x7f3870198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f3876149640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3870106e60 0x7f3870198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f3876149640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f387019cbd0 0x7f387019d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f3876149640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3870100a50 con 0x7f387019cbd0 2024-09-19T06:27:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f3876149640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f38701008d0 con 0x7f3870101920 2024-09-19T06:27:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f3876149640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3870100bd0 con 0x7f3870106e60 2024-09-19T06:27:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f386ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f387019cbd0 0x7f387019d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f386ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f387019cbd0 0x7f387019d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37762/0 (socket says 172.21.15.5:37762) 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f386f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3870101920 0x7f3870198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f386effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3870106e60 0x7f3870198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.103+0000 7f386ffff640 1 -- 172.21.15.5:0/1716470608 learned_addr learned my addr 172.21.15.5:0/1716470608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.104+0000 7f386f7fe640 1 -- 172.21.15.5:0/1716470608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3870106e60 msgr2=0x7f3870198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.104+0000 7f386f7fe640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3870106e60 0x7f3870198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.104+0000 7f386f7fe640 1 -- 172.21.15.5:0/1716470608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f387019cbd0 msgr2=0x7f387019d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.104+0000 7f386f7fe640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f387019cbd0 0x7f387019d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.104+0000 7f386f7fe640 1 -- 172.21.15.5:0/1716470608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387019d6c0 con 0x7f3870101920 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.104+0000 7f386ffff640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f387019cbd0 0x7f387019d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.104+0000 7f386f7fe640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3870101920 0x7f3870198310 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f386000e9f0 tx=0x7f386000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.104+0000 7f386cff9640 1 -- 172.21.15.5:0/1716470608 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386000ccc0 con 0x7f3870101920 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.104+0000 7f386cff9640 1 -- 172.21.15.5:0/1716470608 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38600108e0 con 0x7f3870101920 2024-09-19T06:27:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.104+0000 7f386cff9640 1 -- 172.21.15.5:0/1716470608 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860018880 con 0x7f3870101920 2024-09-19T06:27:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.104+0000 7f3876149640 1 -- 172.21.15.5:0/1716470608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387019d9b0 con 0x7f3870101920 2024-09-19T06:27:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.104+0000 7f3876149640 1 -- 172.21.15.5:0/1716470608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3870110a50 con 0x7f3870101920 2024-09-19T06:27:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.107+0000 7f3878fbe740 1 Processor -- start 2024-09-19T06:27:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.107+0000 7f3878fbe740 1 -- start start 2024-09-19T06:27:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f386cff9640 1 -- 172.21.15.5:0/1716470608 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3860010a90 con 0x7f3870101920 2024-09-19T06:27:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f3878fbe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa81621af0 0x55aa81621ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f3878fbe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81622410 0x55aa8162c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f3878fbe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa8162ccb0 0x55aa8162f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f3878fbe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aa815862f0 con 0x55aa8162ccb0 2024-09-19T06:27:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f3878fbe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aa8152e270 con 0x55aa81622410 2024-09-19T06:27:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f3878fbe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55aa815822b0 con 0x55aa81621af0 2024-09-19T06:27:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f386f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81622410 0x55aa8162c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f386effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa8162ccb0 0x55aa8162f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f386ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa81621af0 0x55aa81621ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f386ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa81621af0 0x55aa81621ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:51388/0 (socket says 172.21.15.5:51388) 2024-09-19T06:27:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f386f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81622410 0x55aa8162c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35866/0 (socket says 172.21.15.5:35866) 2024-09-19T06:27:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.108+0000 7f386ffff640 1 -- 172.21.15.5:0/838120541 learned_addr learned my addr 172.21.15.5:0/838120541 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.109+0000 7f386effd640 1 -- 172.21.15.5:0/838120541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa81621af0 msgr2=0x55aa81621ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.109+0000 7f386effd640 1 --2- 172.21.15.5:0/838120541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa81621af0 0x55aa81621ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.109+0000 7f386effd640 1 -- 172.21.15.5:0/838120541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81622410 msgr2=0x55aa8162c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.109+0000 7f386effd640 1 --2- 172.21.15.5:0/838120541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81622410 0x55aa8162c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.109+0000 7f386effd640 1 -- 172.21.15.5:0/838120541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa8152d340 con 0x55aa8162ccb0 2024-09-19T06:27:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.109+0000 7f386effd640 1 --2- 172.21.15.5:0/838120541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa8162ccb0 0x55aa8162f0a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f3864006010 tx=0x7f3864005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8610b854cf2adafc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.109+0000 7f386cff9640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f38580a7840 0x7f38580a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.109+0000 7f3855ffb640 1 -- 172.21.15.5:0/838120541 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864040070 con 0x55aa8162ccb0 2024-09-19T06:27:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.109+0000 7f386cff9640 1 -- 172.21.15.5:0/1716470608 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f38600d0010 con 0x7f3870101920 2024-09-19T06:27:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.109+0000 7f3855ffb640 1 -- 172.21.15.5:0/838120541 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3864004030 con 0x55aa8162ccb0 2024-09-19T06:27:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.109+0000 7f386ffff640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f38580a7840 0x7f38580a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.110+0000 7f3855ffb640 1 -- 172.21.15.5:0/838120541 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864035040 con 0x55aa8162ccb0 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.110+0000 7f3878fbe740 1 -- 172.21.15.5:0/838120541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa8162ccb0 msgr2=0x55aa8162f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.110+0000 7f386ffff640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f38580a7840 0x7f38580a9d00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f38680116f0 tx=0x7f3868009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.110+0000 7f3878fbe740 1 --2- 172.21.15.5:0/838120541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa8162ccb0 0x55aa8162f0a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f3864006010 tx=0x7f3864005e40 comp rx=0 tx=0).stop 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.110+0000 7f3878fbe740 1 -- 172.21.15.5:0/838120541 shutdown_connections 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.110+0000 7f3878fbe740 1 --2- 172.21.15.5:0/838120541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa8162ccb0 0x55aa8162f0a0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.110+0000 7f3878fbe740 1 --2- 172.21.15.5:0/838120541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81622410 0x55aa8162c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.110+0000 7f3878fbe740 1 --2- 172.21.15.5:0/838120541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa81621af0 0x55aa81621ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.110+0000 7f3878fbe740 1 -- 172.21.15.5:0/838120541 >> 172.21.15.5:0/838120541 conn(0x55aa8161e060 msgr2=0x55aa8161e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.111+0000 7f3878fbe740 1 -- 172.21.15.5:0/838120541 shutdown_connections 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.111+0000 7f3878fbe740 1 -- 172.21.15.5:0/838120541 wait complete. 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.111+0000 7f3878fbe740 1 -- start start 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.111+0000 7f3878fbe740 1 Processor -- start 2024-09-19T06:27:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f3878fbe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 0x55aa81629ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f3878fbe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa8162a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f3878fbe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa8162ccb0 0x55aa8162a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f3878fbe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aa8162f920 con 0x55aa81622410 2024-09-19T06:27:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f3878fbe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aa8150f290 con 0x55aa81621af0 2024-09-19T06:27:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f3878fbe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55aa8162faa0 con 0x55aa8162ccb0 2024-09-19T06:27:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 0x55aa81629ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa8162a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f3878fbe740 10 client.0 _ll_register_callbacks cb 0x55aa81540140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa8162a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37774/0 (socket says 172.21.15.5:37774) 2024-09-19T06:27:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 0x55aa81629ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35868/0 (socket says 172.21.15.5:35868) 2024-09-19T06:27:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386effd640 1 -- 172.21.15.5:0/1808408971 learned_addr learned my addr 172.21.15.5:0/1808408971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386ffff640 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa8162ccb0 0x55aa8162a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386effd640 1 -- 172.21.15.5:0/1808408971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa8162ccb0 msgr2=0x55aa8162a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386effd640 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa8162ccb0 0x55aa8162a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386effd640 1 -- 172.21.15.5:0/1808408971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 msgr2=0x55aa81629ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386effd640 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 0x55aa81629ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386effd640 1 -- 172.21.15.5:0/1808408971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa81299210 con 0x55aa81622410 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386f7fe640 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 0x55aa81629ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.112+0000 7f386ffff640 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa8162ccb0 0x55aa8162a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.113+0000 7f386effd640 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa8162a000 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f3864032eb0 tx=0x7f3864009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccd1f896a703218 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.113+0000 7f384effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.113+0000 7f384effd640 1 -- 172.21.15.5:0/1808408971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864040070 con 0x55aa81622410 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.113+0000 7f384effd640 1 -- 172.21.15.5:0/1808408971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f38640314a0 con 0x55aa81622410 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.113+0000 7f3878fbe740 1 -- 172.21.15.5:0/1808408971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3864004030 con 0x55aa81622410 2024-09-19T06:27:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.113+0000 7f384effd640 1 -- 172.21.15.5:0/1808408971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f3864056050 con 0x55aa81622410 2024-09-19T06:27:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.113+0000 7f3878fbe740 10 client.21438 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.113+0000 7f3878fbe740 1 -- 172.21.15.5:0/1808408971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55aa81365620 con 0x55aa81622410 2024-09-19T06:27:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.113+0000 7f384effd640 1 -- 172.21.15.5:0/1808408971 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3864004030 con 0x55aa81622410 2024-09-19T06:27:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.114+0000 7f384effd640 1 -- 172.21.15.5:0/1808408971 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f3864004ac0 con 0x55aa81622410 2024-09-19T06:27:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.114+0000 7f3878fbe740 10 client.21438 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.114+0000 7f3878fbe740 10 client.21438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.114+0000 7f3878fbe740 20 client.21438 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.114+0000 7f3878fbe740 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aa8163caa0 0x55aa8163ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.114+0000 7f3878fbe740 4 client.21438 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.114+0000 7f3878fbe740 1 -- 172.21.15.5:0/1808408971 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa81593a20 con 0x55aa8163caa0 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.114+0000 7f386f7fe640 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aa8163caa0 0x55aa8163ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.115+0000 7f384effd640 10 client.21438 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.115+0000 7f386f7fe640 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aa8163caa0 0x55aa8163ee90 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4662c53363776889 server_cookie=44c06b79569f4918 in_seq=0 out_seq=0 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.115+0000 7f384effd640 1 -- 172.21.15.5:0/1808408971 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa81593a20 con 0x55aa8163caa0 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.115+0000 7f384effd640 10 client.21438 handle_command_reply: tid=0 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.115+0000 7f3878fbe740 1 client.21438 shutdown 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.116+0000 7f3878fbe740 1 -- 172.21.15.5:0/1808408971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 msgr2=0x55aa8162a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.116+0000 7f3878fbe740 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa8162a000 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f3864032eb0 tx=0x7f3864009520 comp rx=0 tx=0).stop 2024-09-19T06:27:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.116+0000 7f3878fbe740 1 -- 172.21.15.5:0/1808408971 shutdown_connections 2024-09-19T06:27:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.116+0000 7f3878fbe740 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aa8163caa0 0x55aa8163ee90 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.116+0000 7f3878fbe740 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa8162ccb0 0x55aa8162a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.116+0000 7f384effd640 0 client.21438 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.116+0000 7f3878fbe740 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa8162a000 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.116+0000 7f3878fbe740 1 --2- 172.21.15.5:0/1808408971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 0x55aa81629ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.116+0000 7f3878fbe740 1 -- 172.21.15.5:0/1808408971 >> 172.21.15.5:0/1808408971 conn(0x55aa8161e060 msgr2=0x55aa81623d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.116+0000 7f3878fbe740 1 -- 172.21.15.5:0/1808408971 shutdown_connections 2024-09-19T06:27:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.116+0000 7f3878fbe740 1 -- 172.21.15.5:0/1808408971 wait complete. 2024-09-19T06:27:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.116+0000 7f3878fbe740 20 client.21438 trim_cache size 0 max 16384 2024-09-19T06:27:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.140+0000 7f3878fbe740 1 Processor -- start 2024-09-19T06:27:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.140+0000 7f3878fbe740 1 -- start start 2024-09-19T06:27:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.140+0000 7f3878fbe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa81621af0 0x55aa8162a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.140+0000 7f3878fbe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa81626d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.140+0000 7f3878fbe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa8162ccb0 0x55aa81627240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f3878fbe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38701ab600 con 0x55aa81622410 2024-09-19T06:27:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f3878fbe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f38701ad140 con 0x55aa8162ccb0 2024-09-19T06:27:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f3878fbe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38701abcd0 con 0x55aa81621af0 2024-09-19T06:27:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f386effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa81626d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f386effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa81626d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37776/0 (socket says 172.21.15.5:37776) 2024-09-19T06:27:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f386ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa8162ccb0 0x55aa81627240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f386effd640 1 -- 172.21.15.5:0/3283679903 learned_addr learned my addr 172.21.15.5:0/3283679903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f386f7fe640 1 --2- 172.21.15.5:0/3283679903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa81621af0 0x55aa8162a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f386ffff640 1 -- 172.21.15.5:0/3283679903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa81621af0 msgr2=0x55aa8162a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f386ffff640 1 --2- 172.21.15.5:0/3283679903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa81621af0 0x55aa8162a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f386ffff640 1 -- 172.21.15.5:0/3283679903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 msgr2=0x55aa81626d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f386ffff640 1 --2- 172.21.15.5:0/3283679903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa81626d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f386ffff640 1 -- 172.21.15.5:0/3283679903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa8161de90 con 0x55aa8162ccb0 2024-09-19T06:27:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.141+0000 7f386ffff640 1 --2- 172.21.15.5:0/3283679903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa8162ccb0 0x55aa81627240 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3868011f80 tx=0x7f3868016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd8d317dab26ea1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.142+0000 7f38557fa640 1 -- 172.21.15.5:0/3283679903 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386801c4d0 con 0x55aa8162ccb0 2024-09-19T06:27:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.142+0000 7f38557fa640 1 -- 172.21.15.5:0/3283679903 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f386802c070 con 0x55aa8162ccb0 2024-09-19T06:27:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.142+0000 7f38557fa640 1 -- 172.21.15.5:0/3283679903 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868027900 con 0x55aa8162ccb0 2024-09-19T06:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.143+0000 7f3878fbe740 1 -- 172.21.15.5:0/3283679903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa8162ccb0 msgr2=0x55aa81627240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.143+0000 7f3878fbe740 1 --2- 172.21.15.5:0/3283679903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa8162ccb0 0x55aa81627240 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3868011f80 tx=0x7f3868016000 comp rx=0 tx=0).stop 2024-09-19T06:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.143+0000 7f3878fbe740 1 -- 172.21.15.5:0/3283679903 shutdown_connections 2024-09-19T06:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.143+0000 7f3878fbe740 1 --2- 172.21.15.5:0/3283679903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa8162ccb0 0x55aa81627240 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.143+0000 7f3878fbe740 1 --2- 172.21.15.5:0/3283679903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa81626d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.143+0000 7f3878fbe740 1 --2- 172.21.15.5:0/3283679903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa81621af0 0x55aa8162a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.143+0000 7f3878fbe740 1 -- 172.21.15.5:0/3283679903 >> 172.21.15.5:0/3283679903 conn(0x55aa8161e060 msgr2=0x55aa81628b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.143+0000 7f3878fbe740 1 -- 172.21.15.5:0/3283679903 shutdown_connections 2024-09-19T06:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.143+0000 7f3878fbe740 1 -- 172.21.15.5:0/3283679903 wait complete. 2024-09-19T06:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f3878fbe740 1 -- start start 2024-09-19T06:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f3878fbe740 1 Processor -- start 2024-09-19T06:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f3878fbe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 0x55aa8163f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f3878fbe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa8163fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f3878fbe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa8162ccb0 0x55aa8169f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f3878fbe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38701aca70 con 0x55aa81622410 2024-09-19T06:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f3878fbe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f38701ac3a0 con 0x55aa81621af0 2024-09-19T06:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f3878fbe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38701a9600 con 0x55aa8162ccb0 2024-09-19T06:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f3878fbe740 10 client.0 _ll_register_callbacks cb 0x7f38600d3a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f386f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 0x55aa8163f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f386effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa8163fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f386f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 0x55aa8163f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:35878/0 (socket says 172.21.15.5:35878) 2024-09-19T06:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f386effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa8163fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37784/0 (socket says 172.21.15.5:37784) 2024-09-19T06:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.144+0000 7f386f7fe640 1 -- 172.21.15.5:0/3870695831 learned_addr learned my addr 172.21.15.5:0/3870695831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.145+0000 7f386ffff640 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa8162ccb0 0x55aa8169f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.145+0000 7f386f7fe640 1 -- 172.21.15.5:0/3870695831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa8162ccb0 msgr2=0x55aa8169f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.145+0000 7f386f7fe640 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa8162ccb0 0x55aa8169f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.145+0000 7f386f7fe640 1 -- 172.21.15.5:0/3870695831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 msgr2=0x55aa8163fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.145+0000 7f386f7fe640 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa8163fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.145+0000 7f386f7fe640 1 -- 172.21.15.5:0/3870695831 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa8169fac0 con 0x55aa81621af0 2024-09-19T06:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.145+0000 7f386effd640 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa8163fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.145+0000 7f386f7fe640 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 0x55aa8163f530 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3860098d50 tx=0x7f38600bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e248cbd2295b1af3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.145+0000 7f384effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.145+0000 7f384effd640 1 -- 172.21.15.5:0/3870695831 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860010e00 con 0x55aa81621af0 2024-09-19T06:27:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.145+0000 7f384effd640 1 -- 172.21.15.5:0/3870695831 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8550+0+0 (secure 0 0 0) 0x7f38600d3c90 con 0x55aa81621af0 2024-09-19T06:27:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.145+0000 7f3878fbe740 1 -- 172.21.15.5:0/3870695831 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f386802c070 con 0x55aa81621af0 2024-09-19T06:27:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.146+0000 7f384effd640 1 -- 172.21.15.5:0/3870695831 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f386009c380 con 0x55aa81621af0 2024-09-19T06:27:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.146+0000 7f3878fbe740 10 client.30806 fetch_fsmap learned FSMap version 62 2024-09-19T06:27:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.146+0000 7f3878fbe740 1 -- 172.21.15.5:0/3870695831 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55aa816a6e80 con 0x55aa81621af0 2024-09-19T06:27:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.146+0000 7f384effd640 1 -- 172.21.15.5:0/3870695831 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f386802c070 con 0x55aa81621af0 2024-09-19T06:27:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.146+0000 7f384effd640 1 -- 172.21.15.5:0/3870695831 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f38600d20b0 con 0x55aa81621af0 2024-09-19T06:27:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.146+0000 7f3878fbe740 10 client.30806 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:27:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.146+0000 7f3878fbe740 10 client.30806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:27:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.146+0000 7f3878fbe740 20 client.30806 populate_metadata read hostname 'smithi005' 2024-09-19T06:27:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.146+0000 7f3878fbe740 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aa8163caa0 0x55aa816a9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:27:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.146+0000 7f3878fbe740 4 client.30806 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:27:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.146+0000 7f3878fbe740 1 -- 172.21.15.5:0/3870695831 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa8158dd70 con 0x55aa8163caa0 2024-09-19T06:27:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.146+0000 7f386effd640 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aa8163caa0 0x55aa816a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:27:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.147+0000 7f384effd640 10 client.30806 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.147+0000 7f386effd640 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aa8163caa0 0x55aa816a9510 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=904796255688f6e1 server_cookie=b831b7caaf6ecbcf in_seq=0 out_seq=0 2024-09-19T06:27:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f384effd640 1 -- 172.21.15.5:0/3870695831 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa8158dd70 con 0x55aa8163caa0 2024-09-19T06:27:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f384effd640 10 client.30806 handle_command_reply: tid=0 2024-09-19T06:27:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f3878fbe740 1 client.30806 shutdown 2024-09-19T06:27:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f3878fbe740 1 -- 172.21.15.5:0/3870695831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 msgr2=0x55aa8163f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f3878fbe740 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 0x55aa8163f530 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3860098d50 tx=0x7f38600bc040 comp rx=0 tx=0).stop 2024-09-19T06:27:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f3878fbe740 1 -- 172.21.15.5:0/3870695831 shutdown_connections 2024-09-19T06:27:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f3878fbe740 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55aa8163caa0 0x55aa816a9510 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f3878fbe740 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa8162ccb0 0x55aa8169f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f3878fbe740 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa81622410 0x55aa8163fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f3878fbe740 1 --2- 172.21.15.5:0/3870695831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa81621af0 0x55aa8163f530 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f3878fbe740 1 -- 172.21.15.5:0/3870695831 >> 172.21.15.5:0/3870695831 conn(0x55aa8161e060 msgr2=0x55aa81637d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f384effd640 0 client.30806 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:27:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.262+0000 7f3878fbe740 1 -- 172.21.15.5:0/3870695831 shutdown_connections 2024-09-19T06:27:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.263+0000 7f3878fbe740 1 -- 172.21.15.5:0/3870695831 wait complete. 2024-09-19T06:27:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.263+0000 7f3878fbe740 20 client.30806 trim_cache size 0 max 16384 2024-09-19T06:27:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:27:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:27:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:27:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:27:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.264+0000 7f3876149640 1 -- 172.21.15.5:0/1716470608 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f38580a7840 msgr2=0x7f38580a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.264+0000 7f3876149640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f38580a7840 0x7f38580a9d00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f38680116f0 tx=0x7f3868009500 comp rx=0 tx=0).stop 2024-09-19T06:27:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.264+0000 7f3876149640 1 -- 172.21.15.5:0/1716470608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3870101920 msgr2=0x7f3870198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:27:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.264+0000 7f3876149640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3870101920 0x7f3870198310 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f386000e9f0 tx=0x7f386000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.264+0000 7f3876149640 1 -- 172.21.15.5:0/1716470608 shutdown_connections 2024-09-19T06:27:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.264+0000 7f3876149640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f387019cbd0 0x7f387019d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.264+0000 7f3876149640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f38580a7840 0x7f38580a9d00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.264+0000 7f3876149640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3870106e60 0x7f3870198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.264+0000 7f3876149640 1 --2- 172.21.15.5:0/1716470608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3870101920 0x7f3870198310 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:27:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.265+0000 7f3876149640 1 -- 172.21.15.5:0/1716470608 >> 172.21.15.5:0/1716470608 conn(0x7f3870075eb0 msgr2=0x7f38700ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:27:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.265+0000 7f3876149640 1 -- 172.21.15.5:0/1716470608 shutdown_connections 2024-09-19T06:27:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:27:59.265+0000 7f3876149640 1 -- 172.21.15.5:0/1716470608 wait complete. 2024-09-19T06:27:59.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 109617 8.54 MB/sec execute 563 sec latency 91.198 ms 2024-09-19T06:28:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:00 smithi157 ceph-mon[35346]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 241 op/s 2024-09-19T06:28:00.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:00 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:28:00.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:00 smithi005 ceph-mon[30955]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 241 op/s 2024-09-19T06:28:00.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:00 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:28:00.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:00 smithi049 ceph-mon[34640]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 241 op/s 2024-09-19T06:28:00.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:00 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:28:00.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 109788 8.54 MB/sec execute 564 sec latency 101.212 ms 2024-09-19T06:28:01.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:01 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:28:01.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:01 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:28:01.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:01 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:28:01.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:01 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:28:01.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:01 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:28:01.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:01 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:28:01.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:01 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:28:01.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:01 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:28:01.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:01 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:28:01.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 109950 8.55 MB/sec execute 565 sec latency 94.003 ms 2024-09-19T06:28:02.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110114 8.55 MB/sec execute 566 sec latency 139.091 ms 2024-09-19T06:28:02.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:02 smithi005 ceph-mon[30955]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 241 op/s 2024-09-19T06:28:02.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:02 smithi049 ceph-mon[34640]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 241 op/s 2024-09-19T06:28:02.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:02 smithi157 ceph-mon[35346]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 241 op/s 2024-09-19T06:28:03.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110270 8.55 MB/sec execute 567 sec latency 66.604 ms 2024-09-19T06:28:04.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110406 8.55 MB/sec execute 568 sec latency 136.097 ms 2024-09-19T06:28:04.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:04 smithi005 ceph-mon[30955]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.6 MiB/s wr, 405 op/s 2024-09-19T06:28:04.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:04 smithi049 ceph-mon[34640]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.6 MiB/s wr, 405 op/s 2024-09-19T06:28:04.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:04 smithi157 ceph-mon[35346]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.6 MiB/s wr, 405 op/s 2024-09-19T06:28:05.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110560 8.55 MB/sec execute 569 sec latency 82.538 ms 2024-09-19T06:28:05.628 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:05 smithi005 ceph-mon[30955]: osdmap e122: 12 total, 12 up, 12 in 2024-09-19T06:28:05.645 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:05 smithi049 ceph-mon[34640]: osdmap e122: 12 total, 12 up, 12 in 2024-09-19T06:28:05.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:05 smithi157 ceph-mon[35346]: osdmap e122: 12 total, 12 up, 12 in 2024-09-19T06:28:06.398 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:28:06.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110697 8.54 MB/sec execute 570 sec latency 103.977 ms 2024-09-19T06:28:06.628 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:06 smithi005 ceph-mon[30955]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.2 MiB/s wr, 334 op/s 2024-09-19T06:28:06.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:06 smithi049 ceph-mon[34640]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.2 MiB/s wr, 334 op/s 2024-09-19T06:28:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:28:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:06 smithi157 ceph-mon[35346]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.2 MiB/s wr, 334 op/s 2024-09-19T06:28:07.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.372+0000 7fd47a878640 1 -- 172.21.15.5:0/705023893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd474101de0 msgr2=0x7fd4741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:07.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.372+0000 7fd47a878640 1 --2- 172.21.15.5:0/705023893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd474101de0 0x7fd4741021c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd464009900 tx=0x7fd46402a410 comp rx=0 tx=0).stop 2024-09-19T06:28:07.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.372+0000 7fd47a878640 1 -- 172.21.15.5:0/705023893 shutdown_connections 2024-09-19T06:28:07.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.372+0000 7fd47a878640 1 --2- 172.21.15.5:0/705023893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd474106f10 0x7fd47410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.372+0000 7fd47a878640 1 --2- 172.21.15.5:0/705023893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd474102790 0x7fd4741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.372+0000 7fd47a878640 1 --2- 172.21.15.5:0/705023893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd474101de0 0x7fd4741021c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.372+0000 7fd47a878640 1 -- 172.21.15.5:0/705023893 >> 172.21.15.5:0/705023893 conn(0x7fd4740fd8e0 msgr2=0x7fd4740ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.372+0000 7fd47a878640 1 -- 172.21.15.5:0/705023893 shutdown_connections 2024-09-19T06:28:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.373+0000 7fd47a878640 1 -- 172.21.15.5:0/705023893 wait complete. 2024-09-19T06:28:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.373+0000 7fd47a878640 1 Processor -- start 2024-09-19T06:28:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.373+0000 7fd47a878640 1 -- start start 2024-09-19T06:28:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.374+0000 7fd47a878640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd474101de0 0x7fd4740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.374+0000 7fd47a878640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd474102790 0x7fd474074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.374+0000 7fd47a878640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd474106f10 0x7fd474074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.374+0000 7fd47a878640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd474111e70 con 0x7fd474102790 2024-09-19T06:28:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.374+0000 7fd47a878640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd474111cf0 con 0x7fd474106f10 2024-09-19T06:28:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.374+0000 7fd4737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd474102790 0x7fd474074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.374+0000 7fd4737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd474102790 0x7fd474074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42302/0 (socket says 172.21.15.5:42302) 2024-09-19T06:28:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.374+0000 7fd4737fe640 1 -- 172.21.15.5:0/530400804 learned_addr learned my addr 172.21.15.5:0/530400804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.374+0000 7fd478e15640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd474106f10 0x7fd474074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.375+0000 7fd47a878640 1 -- 172.21.15.5:0/530400804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd474111ff0 con 0x7fd474101de0 2024-09-19T06:28:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.375+0000 7fd473fff640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd474101de0 0x7fd4740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.375+0000 7fd478e15640 1 -- 172.21.15.5:0/530400804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd474101de0 msgr2=0x7fd4740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.375+0000 7fd478e15640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd474101de0 0x7fd4740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.375+0000 7fd478e15640 1 -- 172.21.15.5:0/530400804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd474102790 msgr2=0x7fd474074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.375+0000 7fd478e15640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd474102790 0x7fd474074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.375+0000 7fd478e15640 1 -- 172.21.15.5:0/530400804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4741a68d0 con 0x7fd474106f10 2024-09-19T06:28:07.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.376+0000 7fd478e15640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd474106f10 0x7fd474074f70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00ec50 tx=0x7fd46c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:07.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.376+0000 7fd4717fa640 1 -- 172.21.15.5:0/530400804 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c003b70 con 0x7fd474106f10 2024-09-19T06:28:07.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.376+0000 7fd4717fa640 1 -- 172.21.15.5:0/530400804 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd46c003d10 con 0x7fd474106f10 2024-09-19T06:28:07.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.376+0000 7fd47a878640 1 -- 172.21.15.5:0/530400804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4741a6bc0 con 0x7fd474106f10 2024-09-19T06:28:07.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.376+0000 7fd4717fa640 1 -- 172.21.15.5:0/530400804 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c003b70 con 0x7fd474106f10 2024-09-19T06:28:07.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.376+0000 7fd47a878640 1 -- 172.21.15.5:0/530400804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4741a70b0 con 0x7fd474106f10 2024-09-19T06:28:07.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.379+0000 7fd4717fa640 1 -- 172.21.15.5:0/530400804 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd46c003fc0 con 0x7fd474106f10 2024-09-19T06:28:07.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.380+0000 7fd47cc51740 1 Processor -- start 2024-09-19T06:28:07.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.380+0000 7fd47cc51740 1 -- start start 2024-09-19T06:28:07.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.380+0000 7fd47cc51740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d5f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.380+0000 7fd47cc51740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d6a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.380+0000 7fd47cc51740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d6ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.380+0000 7fd47cc51740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55efc8cc52e0 con 0x55efc8d5f4a0 2024-09-19T06:28:07.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.380+0000 7fd47cc51740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55efc8c6cce0 con 0x55efc8d6a680 2024-09-19T06:28:07.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.380+0000 7fd47cc51740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55efc8cc12a0 con 0x55efc8d5fdc0 2024-09-19T06:28:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.380+0000 7fd4737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d5f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.380+0000 7fd4737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d5f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42312/0 (socket says 172.21.15.5:42312) 2024-09-19T06:28:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.380+0000 7fd4737fe640 1 -- 172.21.15.5:0/1971236871 learned_addr learned my addr 172.21.15.5:0/1971236871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.380+0000 7fd473fff640 1 --2- 172.21.15.5:0/1971236871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d6a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.381+0000 7fd4737fe640 1 --2- 172.21.15.5:0/1971236871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d6ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.381+0000 7fd4737fe640 1 -- 172.21.15.5:0/1971236871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 msgr2=0x55efc8d6a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.381+0000 7fd4737fe640 1 --2- 172.21.15.5:0/1971236871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d6a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.381+0000 7fd4737fe640 1 -- 172.21.15.5:0/1971236871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 msgr2=0x55efc8d6ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.381+0000 7fd4737fe640 1 --2- 172.21.15.5:0/1971236871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d6ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.381+0000 7fd4737fe640 1 -- 172.21.15.5:0/1971236871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc8c51440 con 0x55efc8d5f4a0 2024-09-19T06:28:07.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.381+0000 7fd4717fa640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd45c0a35b0 0x7fd45c0a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.381+0000 7fd4737fe640 1 --2- 172.21.15.5:0/1971236871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d5f880 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fd468009620 tx=0x7fd4680099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7d2646a6c57f87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:07.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.381+0000 7fd4717fa640 1 -- 172.21.15.5:0/530400804 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8594+0+0 (secure 0 0 0) 0x7fd46c0cffa0 con 0x7fd474106f10 2024-09-19T06:28:07.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.382+0000 7fd473fff640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd45c0a35b0 0x7fd45c0a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.382+0000 7fd45a7fc640 1 -- 172.21.15.5:0/1971236871 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46801a070 con 0x55efc8d5f4a0 2024-09-19T06:28:07.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.382+0000 7fd45a7fc640 1 -- 172.21.15.5:0/1971236871 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd46800fd70 con 0x55efc8d5f4a0 2024-09-19T06:28:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.382+0000 7fd45a7fc640 1 -- 172.21.15.5:0/1971236871 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468003b40 con 0x55efc8d5f4a0 2024-09-19T06:28:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.383+0000 7fd47cc51740 1 -- 172.21.15.5:0/1971236871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 msgr2=0x55efc8d5f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.383+0000 7fd473fff640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd45c0a35b0 0x7fd45c0a5a70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd464000c00 tx=0x7fd464005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.383+0000 7fd47cc51740 1 --2- 172.21.15.5:0/1971236871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d5f880 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fd468009620 tx=0x7fd4680099f0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.383+0000 7fd47cc51740 1 -- 172.21.15.5:0/1971236871 shutdown_connections 2024-09-19T06:28:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.383+0000 7fd47cc51740 1 --2- 172.21.15.5:0/1971236871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d6ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.383+0000 7fd47cc51740 1 --2- 172.21.15.5:0/1971236871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d6a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.383+0000 7fd47cc51740 1 --2- 172.21.15.5:0/1971236871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d5f880 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.383+0000 7fd47cc51740 1 -- 172.21.15.5:0/1971236871 >> 172.21.15.5:0/1971236871 conn(0x55efc8d5ba30 msgr2=0x55efc8d5be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.383+0000 7fd47cc51740 1 -- 172.21.15.5:0/1971236871 shutdown_connections 2024-09-19T06:28:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.383+0000 7fd47cc51740 1 -- 172.21.15.5:0/1971236871 wait complete. 2024-09-19T06:28:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.383+0000 7fd47cc51740 1 -- start start 2024-09-19T06:28:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.384+0000 7fd47cc51740 1 Processor -- start 2024-09-19T06:28:07.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.384+0000 7fd47cc51740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5f4a0 0x55efc8d69f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.384+0000 7fd47cc51740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5fdc0 0x55efc8d67210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.384+0000 7fd473fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5f4a0 0x55efc8d69f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.384+0000 7fd473fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5f4a0 0x55efc8d69f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46068/0 (socket says 172.21.15.5:46068) 2024-09-19T06:28:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.384+0000 7fd47cc51740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d67750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd47cc51740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55efc8d6d2a0 con 0x55efc8d5fdc0 2024-09-19T06:28:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd47cc51740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55efc8c80050 con 0x55efc8d6a680 2024-09-19T06:28:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd47cc51740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55efc8d6d420 con 0x55efc8d5f4a0 2024-09-19T06:28:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd4737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5fdc0 0x55efc8d67210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd47cc51740 10 client.0 _ll_register_callbacks cb 0x55efc8cc0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd473fff640 1 -- 172.21.15.5:0/1600835031 learned_addr learned my addr 172.21.15.5:0/1600835031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd478e15640 1 --2- 172.21.15.5:0/1600835031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d67750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd473fff640 1 -- 172.21.15.5:0/1600835031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 msgr2=0x55efc8d67750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd473fff640 1 --2- 172.21.15.5:0/1600835031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d67750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd473fff640 1 -- 172.21.15.5:0/1600835031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5fdc0 msgr2=0x55efc8d67210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd473fff640 1 --2- 172.21.15.5:0/1600835031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5fdc0 0x55efc8d67210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd473fff640 1 -- 172.21.15.5:0/1600835031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc8a363c0 con 0x55efc8d5f4a0 2024-09-19T06:28:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd473fff640 1 --2- 172.21.15.5:0/1600835031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5f4a0 0x55efc8d69f60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd464009900 tx=0x7fd464009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c9ef7b77eb8dd54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.385+0000 7fd4537fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:28:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.386+0000 7fd4537fe640 1 -- 172.21.15.5:0/1600835031 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46404c070 con 0x55efc8d5f4a0 2024-09-19T06:28:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.386+0000 7fd4537fe640 1 -- 172.21.15.5:0/1600835031 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 8594+0+0 (secure 0 0 0) 0x7fd464004ad0 con 0x55efc8d5f4a0 2024-09-19T06:28:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.386+0000 7fd47cc51740 1 -- 172.21.15.5:0/1600835031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd46800fd70 con 0x55efc8d5f4a0 2024-09-19T06:28:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.386+0000 7fd4537fe640 1 -- 172.21.15.5:0/1600835031 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fd464058050 con 0x55efc8d5f4a0 2024-09-19T06:28:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.386+0000 7fd47cc51740 10 client.30751 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.386+0000 7fd47cc51740 1 -- 172.21.15.5:0/1600835031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55efc8d81450 con 0x55efc8d5f4a0 2024-09-19T06:28:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.386+0000 7fd4537fe640 1 -- 172.21.15.5:0/1600835031 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd46800fd70 con 0x55efc8d5f4a0 2024-09-19T06:28:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.387+0000 7fd4537fe640 1 -- 172.21.15.5:0/1600835031 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fd4640324b0 con 0x55efc8d5f4a0 2024-09-19T06:28:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.387+0000 7fd47cc51740 10 client.30751 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.387+0000 7fd47cc51740 10 client.30751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.387+0000 7fd47cc51740 20 client.30751 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.387+0000 7fd47cc51740 1 --2- 172.21.15.5:0/1600835031 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55efc8d828b0 0x55efc8d84d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.387+0000 7fd47cc51740 4 client.30751 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:28:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.387+0000 7fd47cc51740 1 -- 172.21.15.5:0/1600835031 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efc8cd2850 con 0x55efc8d828b0 2024-09-19T06:28:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.387+0000 7fd4737fe640 1 --2- 172.21.15.5:0/1600835031 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55efc8d828b0 0x55efc8d84d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.388+0000 7fd4537fe640 10 client.30751 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.388+0000 7fd4737fe640 1 --2- 172.21.15.5:0/1600835031 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55efc8d828b0 0x55efc8d84d20 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70dfd42baa475319 server_cookie=99ca7c1e920de59c in_seq=0 out_seq=0 2024-09-19T06:28:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.388+0000 7fd4537fe640 1 -- 172.21.15.5:0/1600835031 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55efc8cd2850 con 0x55efc8d828b0 2024-09-19T06:28:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.388+0000 7fd4537fe640 10 client.30751 handle_command_reply: tid=0 2024-09-19T06:28:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.388+0000 7fd47cc51740 1 client.30751 shutdown 2024-09-19T06:28:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.389+0000 7fd47cc51740 1 -- 172.21.15.5:0/1600835031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5f4a0 msgr2=0x55efc8d69f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.389+0000 7fd47cc51740 1 --2- 172.21.15.5:0/1600835031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5f4a0 0x55efc8d69f60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd464009900 tx=0x7fd464009260 comp rx=0 tx=0).stop 2024-09-19T06:28:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.389+0000 7fd47cc51740 1 -- 172.21.15.5:0/1600835031 shutdown_connections 2024-09-19T06:28:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.389+0000 7fd47cc51740 1 --2- 172.21.15.5:0/1600835031 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55efc8d828b0 0x55efc8d84d20 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.389+0000 7fd47cc51740 1 --2- 172.21.15.5:0/1600835031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d67750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.389+0000 7fd47cc51740 1 --2- 172.21.15.5:0/1600835031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5fdc0 0x55efc8d67210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.389+0000 7fd47cc51740 1 --2- 172.21.15.5:0/1600835031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5f4a0 0x55efc8d69f60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.389+0000 7fd47cc51740 1 -- 172.21.15.5:0/1600835031 >> 172.21.15.5:0/1600835031 conn(0x55efc8d5ba30 msgr2=0x55efc8d5ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.389+0000 7fd4537fe640 0 client.30751 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.389+0000 7fd47cc51740 1 -- 172.21.15.5:0/1600835031 shutdown_connections 2024-09-19T06:28:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.390+0000 7fd47cc51740 1 -- 172.21.15.5:0/1600835031 wait complete. 2024-09-19T06:28:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.390+0000 7fd47cc51740 20 client.30751 trim_cache size 0 max 16384 2024-09-19T06:28:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.422+0000 7fd47cc51740 1 Processor -- start 2024-09-19T06:28:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.423+0000 7fd47cc51740 1 -- start start 2024-09-19T06:28:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.423+0000 7fd47cc51740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d8a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.423+0000 7fd47cc51740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d8ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.423+0000 7fd47cc51740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d64470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.423+0000 7fd473fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d8a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.423+0000 7fd47cc51740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741ab6e0 con 0x55efc8d5f4a0 2024-09-19T06:28:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.423+0000 7fd473fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d8a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42326/0 (socket says 172.21.15.5:42326) 2024-09-19T06:28:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.423+0000 7fd473fff640 1 -- 172.21.15.5:0/3868426573 learned_addr learned my addr 172.21.15.5:0/3868426573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.423+0000 7fd47cc51740 1 -- 172.21.15.5:0/3868426573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741ad220 con 0x55efc8d6a680 2024-09-19T06:28:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.423+0000 7fd47cc51740 1 -- 172.21.15.5:0/3868426573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741abdb0 con 0x55efc8d5fdc0 2024-09-19T06:28:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.423+0000 7fd4737fe640 1 --2- 172.21.15.5:0/3868426573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d8ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.423+0000 7fd478e15640 1 --2- 172.21.15.5:0/3868426573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d64470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.424+0000 7fd4737fe640 1 -- 172.21.15.5:0/3868426573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 msgr2=0x55efc8d64470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.424+0000 7fd4737fe640 1 --2- 172.21.15.5:0/3868426573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d64470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.424+0000 7fd4737fe640 1 -- 172.21.15.5:0/3868426573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 msgr2=0x55efc8d8a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.424+0000 7fd4737fe640 1 --2- 172.21.15.5:0/3868426573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d8a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.424+0000 7fd4737fe640 1 -- 172.21.15.5:0/3868426573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc8d6a2d0 con 0x55efc8d5fdc0 2024-09-19T06:28:07.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.424+0000 7fd4737fe640 1 --2- 172.21.15.5:0/3868426573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d8ab30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55efc8d66aa0 tx=0x7fd468011660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a357f1c0b82417e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:07.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.424+0000 7fd459ffb640 1 -- 172.21.15.5:0/3868426573 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46801a070 con 0x55efc8d5fdc0 2024-09-19T06:28:07.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.424+0000 7fd459ffb640 1 -- 172.21.15.5:0/3868426573 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd468011300 con 0x55efc8d5fdc0 2024-09-19T06:28:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.424+0000 7fd459ffb640 1 -- 172.21.15.5:0/3868426573 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46801f6b0 con 0x55efc8d5fdc0 2024-09-19T06:28:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.425+0000 7fd47cc51740 1 -- 172.21.15.5:0/3868426573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 msgr2=0x55efc8d8ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.425+0000 7fd47cc51740 1 --2- 172.21.15.5:0/3868426573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d8ab30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55efc8d66aa0 tx=0x7fd468011660 comp rx=0 tx=0).stop 2024-09-19T06:28:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.425+0000 7fd47cc51740 1 -- 172.21.15.5:0/3868426573 shutdown_connections 2024-09-19T06:28:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.425+0000 7fd47cc51740 1 --2- 172.21.15.5:0/3868426573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d64470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.425+0000 7fd47cc51740 1 --2- 172.21.15.5:0/3868426573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d8ab30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.425+0000 7fd47cc51740 1 --2- 172.21.15.5:0/3868426573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d8a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.425+0000 7fd47cc51740 1 -- 172.21.15.5:0/3868426573 >> 172.21.15.5:0/3868426573 conn(0x55efc8d5ba30 msgr2=0x55efc8d627e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.425+0000 7fd47cc51740 1 -- 172.21.15.5:0/3868426573 shutdown_connections 2024-09-19T06:28:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.425+0000 7fd47cc51740 1 -- 172.21.15.5:0/3868426573 wait complete. 2024-09-19T06:28:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.426+0000 7fd47cc51740 1 -- start start 2024-09-19T06:28:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.426+0000 7fd47cc51740 1 Processor -- start 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.426+0000 7fd47cc51740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d7b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd47cc51740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d76870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd47cc51740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d76db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd47cc51740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741acb50 con 0x55efc8d5f4a0 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd47cc51740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741ac480 con 0x55efc8d6a680 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd47cc51740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741a9700 con 0x55efc8d5fdc0 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd473fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d7b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd4737fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d76870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd47cc51740 10 client.0 _ll_register_callbacks cb 0x7fd468005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd4737fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d76870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46088/0 (socket says 172.21.15.5:46088) 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd4737fe640 1 -- 172.21.15.5:0/3796396083 learned_addr learned my addr 172.21.15.5:0/3796396083 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd478e15640 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d76db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd478e15640 1 -- 172.21.15.5:0/3796396083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 msgr2=0x55efc8d76870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd478e15640 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d76870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd478e15640 1 -- 172.21.15.5:0/3796396083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 msgr2=0x55efc8d7b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd478e15640 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d7b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd478e15640 1 -- 172.21.15.5:0/3796396083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc8d77450 con 0x55efc8d6a680 2024-09-19T06:28:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd473fff640 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d7b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:28:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd478e15640 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d76db0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0cd640 tx=0x7fd46c09bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ab652e0e520fbd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd4537fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:28:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.427+0000 7fd4537fe640 1 -- 172.21.15.5:0/3796396083 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0a0580 con 0x55efc8d6a680 2024-09-19T06:28:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd47cc51740 1 -- 172.21.15.5:0/3796396083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd468011300 con 0x55efc8d6a680 2024-09-19T06:28:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd4537fe640 1 -- 172.21.15.5:0/3796396083 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 8594+0+0 (secure 0 0 0) 0x7fd46c0cd8c0 con 0x55efc8d6a680 2024-09-19T06:28:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd4537fe640 1 -- 172.21.15.5:0/3796396083 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fd46c0a4e40 con 0x55efc8d6a680 2024-09-19T06:28:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd47cc51740 10 client.30824 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd47cc51740 1 -- 172.21.15.5:0/3796396083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55efc8ded510 con 0x55efc8d6a680 2024-09-19T06:28:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd4537fe640 1 -- 172.21.15.5:0/3796396083 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd468011300 con 0x55efc8d6a680 2024-09-19T06:28:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd4537fe640 1 -- 172.21.15.5:0/3796396083 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fd46c0a00c0 con 0x55efc8d6a680 2024-09-19T06:28:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd47cc51740 10 client.30824 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd47cc51740 10 client.30824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd47cc51740 20 client.30824 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd47cc51740 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55efc8dedbe0 0x55efc8deffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd47cc51740 4 client.30824 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:28:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.428+0000 7fd47cc51740 1 -- 172.21.15.5:0/3796396083 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55efc8d67ae0 con 0x55efc8dedbe0 2024-09-19T06:28:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.429+0000 7fd473fff640 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55efc8dedbe0 0x55efc8deffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.429+0000 7fd473fff640 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55efc8dedbe0 0x55efc8deffd0 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682ccc5a898714c3 server_cookie=988c9d120d080539 in_seq=0 out_seq=0 2024-09-19T06:28:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.429+0000 7fd4537fe640 10 client.30824 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.430+0000 7fd4537fe640 1 -- 172.21.15.5:0/3796396083 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55efc8d67ae0 con 0x55efc8dedbe0 2024-09-19T06:28:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.430+0000 7fd4537fe640 10 client.30824 handle_command_reply: tid=0 2024-09-19T06:28:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.430+0000 7fd47cc51740 1 client.30824 shutdown 2024-09-19T06:28:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.430+0000 7fd47cc51740 1 -- 172.21.15.5:0/3796396083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 msgr2=0x55efc8d76db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.430+0000 7fd47cc51740 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d76db0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0cd640 tx=0x7fd46c09bc60 comp rx=0 tx=0).stop 2024-09-19T06:28:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.430+0000 7fd47cc51740 1 -- 172.21.15.5:0/3796396083 shutdown_connections 2024-09-19T06:28:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.430+0000 7fd47cc51740 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55efc8dedbe0 0x55efc8deffd0 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.430+0000 7fd47cc51740 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efc8d6a680 0x55efc8d76db0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.430+0000 7fd4537fe640 0 client.30824 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.430+0000 7fd47cc51740 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55efc8d5fdc0 0x55efc8d76870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.430+0000 7fd47cc51740 1 --2- 172.21.15.5:0/3796396083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efc8d5f4a0 0x55efc8d7b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.430+0000 7fd47cc51740 1 -- 172.21.15.5:0/3796396083 >> 172.21.15.5:0/3796396083 conn(0x55efc8d5ba30 msgr2=0x55efc8d60f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.431+0000 7fd47cc51740 1 -- 172.21.15.5:0/3796396083 shutdown_connections 2024-09-19T06:28:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.431+0000 7fd47cc51740 1 -- 172.21.15.5:0/3796396083 wait complete. 2024-09-19T06:28:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.431+0000 7fd47cc51740 20 client.30824 trim_cache size 0 max 16384 2024-09-19T06:28:07.436 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:28:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:28:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:28:07.436 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:28:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.432+0000 7fd47a878640 1 -- 172.21.15.5:0/530400804 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd45c0a35b0 msgr2=0x7fd45c0a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.432+0000 7fd47a878640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd45c0a35b0 0x7fd45c0a5a70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd464000c00 tx=0x7fd464005a20 comp rx=0 tx=0).stop 2024-09-19T06:28:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.432+0000 7fd47a878640 1 -- 172.21.15.5:0/530400804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd474106f10 msgr2=0x7fd474074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.432+0000 7fd47a878640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd474106f10 0x7fd474074f70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00ec50 tx=0x7fd46c00c6a0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.432+0000 7fd47a878640 1 -- 172.21.15.5:0/530400804 shutdown_connections 2024-09-19T06:28:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.432+0000 7fd47a878640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd474106f10 0x7fd474074f70 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.432+0000 7fd47a878640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd474102790 0x7fd474074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.432+0000 7fd47a878640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd45c0a35b0 0x7fd45c0a5a70 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.432+0000 7fd47a878640 1 --2- 172.21.15.5:0/530400804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd474101de0 0x7fd4740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.432+0000 7fd47a878640 1 -- 172.21.15.5:0/530400804 >> 172.21.15.5:0/530400804 conn(0x7fd4740fd8e0 msgr2=0x7fd4740fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.432+0000 7fd47a878640 1 -- 172.21.15.5:0/530400804 shutdown_connections 2024-09-19T06:28:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:07.432+0000 7fd47a878640 1 -- 172.21.15.5:0/530400804 wait complete. 2024-09-19T06:28:07.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110834 8.53 MB/sec execute 571 sec latency 113.553 ms 2024-09-19T06:28:08.467 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:28:08.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:28:08.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110981 8.53 MB/sec execute 572 sec latency 76.291 ms 2024-09-19T06:28:08.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:08 smithi005 ceph-mon[30955]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.1 MiB/s wr, 344 op/s 2024-09-19T06:28:08.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:08 smithi049 ceph-mon[34640]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.1 MiB/s wr, 344 op/s 2024-09-19T06:28:08.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:08 smithi157 ceph-mon[35346]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.1 MiB/s wr, 344 op/s 2024-09-19T06:28:08.897 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:28:09.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.480+0000 7f9a89985640 1 -- 172.21.15.5:0/3589152957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a84103ff0 msgr2=0x7f9a841043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:09.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.480+0000 7f9a89985640 1 --2- 172.21.15.5:0/3589152957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a84103ff0 0x7f9a841043d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009860 tx=0x7f9a7402a340 comp rx=0 tx=0).stop 2024-09-19T06:28:09.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.480+0000 7f9a89985640 1 -- 172.21.15.5:0/3589152957 shutdown_connections 2024-09-19T06:28:09.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.480+0000 7f9a89985640 1 --2- 172.21.15.5:0/3589152957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a84109120 0x7f9a84110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:09.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.480+0000 7f9a89985640 1 --2- 172.21.15.5:0/3589152957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a841049a0 0x7f9a841089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:09.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.480+0000 7f9a89985640 1 --2- 172.21.15.5:0/3589152957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a84103ff0 0x7f9a841043d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:09.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.480+0000 7f9a89985640 1 -- 172.21.15.5:0/3589152957 >> 172.21.15.5:0/3589152957 conn(0x7f9a840ffab0 msgr2=0x7f9a84101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:09.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.480+0000 7f9a89985640 1 -- 172.21.15.5:0/3589152957 shutdown_connections 2024-09-19T06:28:09.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.480+0000 7f9a89985640 1 -- 172.21.15.5:0/3589152957 wait complete. 2024-09-19T06:28:09.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.481+0000 7f9a89985640 1 Processor -- start 2024-09-19T06:28:09.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.481+0000 7f9a89985640 1 -- start start 2024-09-19T06:28:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.481+0000 7f9a89985640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a84103ff0 0x7f9a8419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.481+0000 7f9a89985640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a841049a0 0x7f9a8419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.482+0000 7f9a89985640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a84109120 0x7f9a8419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.482+0000 7f9a89985640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84113fd0 con 0x7f9a841049a0 2024-09-19T06:28:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.482+0000 7f9a7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a841049a0 0x7f9a8419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.482+0000 7f9a7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a841049a0 0x7f9a8419aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42358/0 (socket says 172.21.15.5:42358) 2024-09-19T06:28:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.482+0000 7f9a7bfff640 1 -- 172.21.15.5:0/3048437299 learned_addr learned my addr 172.21.15.5:0/3048437299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.482+0000 7f9a82ffd640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a84103ff0 0x7f9a8419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.482+0000 7f9a837fe640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a84109120 0x7f9a8419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.482+0000 7f9a89985640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84113e50 con 0x7f9a84103ff0 2024-09-19T06:28:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.482+0000 7f9a89985640 1 -- 172.21.15.5:0/3048437299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84114150 con 0x7f9a84109120 2024-09-19T06:28:09.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.483+0000 7f9a7bfff640 1 -- 172.21.15.5:0/3048437299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a84109120 msgr2=0x7f9a8419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:09.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.483+0000 7f9a7bfff640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a84109120 0x7f9a8419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:09.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.483+0000 7f9a7bfff640 1 -- 172.21.15.5:0/3048437299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a84103ff0 msgr2=0x7f9a8419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:09.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.483+0000 7f9a7bfff640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a84103ff0 0x7f9a8419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:09.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.483+0000 7f9a7bfff640 1 -- 172.21.15.5:0/3048437299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8419f4c0 con 0x7f9a841049a0 2024-09-19T06:28:09.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.483+0000 7f9a82ffd640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a84103ff0 0x7f9a8419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:28:09.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.483+0000 7f9a7bfff640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a841049a0 0x7f9a8419aa50 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00e9c0 tx=0x7f9a6c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:09.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.484+0000 7f9a88983640 1 -- 172.21.15.5:0/3048437299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00cde0 con 0x7f9a841049a0 2024-09-19T06:28:09.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.484+0000 7f9a88983640 1 -- 172.21.15.5:0/3048437299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a6c0035f0 con 0x7f9a841049a0 2024-09-19T06:28:09.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.484+0000 7f9a89985640 1 -- 172.21.15.5:0/3048437299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8419f7b0 con 0x7f9a841049a0 2024-09-19T06:28:09.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.484+0000 7f9a88983640 1 -- 172.21.15.5:0/3048437299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c010690 con 0x7f9a841049a0 2024-09-19T06:28:09.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.484+0000 7f9a89985640 1 -- 172.21.15.5:0/3048437299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a841a7090 con 0x7f9a841049a0 2024-09-19T06:28:09.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.485+0000 7f9a89985640 1 -- 172.21.15.5:0/3048437299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a841043d0 con 0x7f9a841049a0 2024-09-19T06:28:09.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.488+0000 7f9a88983640 1 -- 172.21.15.5:0/3048437299 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9a6c0037a0 con 0x7f9a841049a0 2024-09-19T06:28:09.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.489+0000 7f9a88983640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9a700a3430 0x7f9a700a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:09.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.489+0000 7f9a88983640 1 -- 172.21.15.5:0/3048437299 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f9a6c0d0040 con 0x7f9a841049a0 2024-09-19T06:28:09.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.489+0000 7f9a82ffd640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9a700a3430 0x7f9a700a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:09.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.490+0000 7f9a82ffd640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9a700a3430 0x7f9a700a58f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9a74000c00 tx=0x7f9a740058b0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:09.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.498+0000 7f9a88983640 1 -- 172.21.15.5:0/3048437299 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9a6c009510 con 0x7f9a841049a0 2024-09-19T06:28:09.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 111104 8.52 MB/sec execute 573 sec latency 100.000 ms 2024-09-19T06:28:09.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.862+0000 7f9a89985640 1 -- 172.21.15.5:0/3048437299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a8410c360 con 0x7f9a841049a0 2024-09-19T06:28:09.864 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:09 smithi005 ceph-mon[30955]: osdmap e123: 12 total, 12 up, 12 in 2024-09-19T06:28:09.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.863+0000 7f9a88983640 1 -- 172.21.15.5:0/3048437299 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+11959 (secure 0 0 0) 0x7f9a6c09ca00 con 0x7f9a841049a0 2024-09-19T06:28:09.866 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:28:09.866 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":62,"btime":"2024-09-19T06:25:41:566205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":62,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:25:40.570592+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:28:09.866 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 62 2024-09-19T06:28:09.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.867+0000 7f9a89985640 1 -- 172.21.15.5:0/3048437299 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9a700a3430 msgr2=0x7f9a700a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:09.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.867+0000 7f9a89985640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9a700a3430 0x7f9a700a58f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9a74000c00 tx=0x7f9a740058b0 comp rx=0 tx=0).stop 2024-09-19T06:28:09.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.867+0000 7f9a89985640 1 -- 172.21.15.5:0/3048437299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a841049a0 msgr2=0x7f9a8419aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:09.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.867+0000 7f9a89985640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a841049a0 0x7f9a8419aa50 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00e9c0 tx=0x7f9a6c00ee80 comp rx=0 tx=0).stop 2024-09-19T06:28:09.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.867+0000 7f9a89985640 1 -- 172.21.15.5:0/3048437299 shutdown_connections 2024-09-19T06:28:09.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.867+0000 7f9a89985640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9a700a3430 0x7f9a700a58f0 secure :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9a74000c00 tx=0x7f9a740058b0 comp rx=0 tx=0).stop 2024-09-19T06:28:09.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.867+0000 7f9a89985640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a84109120 0x7f9a8419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:09.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.867+0000 7f9a89985640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a841049a0 0x7f9a8419aa50 secure :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00e9c0 tx=0x7f9a6c00ee80 comp rx=0 tx=0).stop 2024-09-19T06:28:09.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.867+0000 7f9a89985640 1 --2- 172.21.15.5:0/3048437299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a84103ff0 0x7f9a8419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:09.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.867+0000 7f9a89985640 1 -- 172.21.15.5:0/3048437299 >> 172.21.15.5:0/3048437299 conn(0x7f9a840ffab0 msgr2=0x7f9a84100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:09.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.869+0000 7f9a89985640 1 -- 172.21.15.5:0/3048437299 shutdown_connections 2024-09-19T06:28:09.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:09.869+0000 7f9a89985640 1 -- 172.21.15.5:0/3048437299 wait complete. 2024-09-19T06:28:09.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:09 smithi049 ceph-mon[34640]: osdmap e123: 12 total, 12 up, 12 in 2024-09-19T06:28:10.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:09 smithi157 ceph-mon[35346]: osdmap e123: 12 total, 12 up, 12 in 2024-09-19T06:28:10.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 111266 8.52 MB/sec execute 574 sec latency 97.907 ms 2024-09-19T06:28:10.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:10 smithi005 ceph-mon[30955]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.9 MiB/s wr, 430 op/s 2024-09-19T06:28:10.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:10 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3048437299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:28:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:10 smithi049 ceph-mon[34640]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.9 MiB/s wr, 430 op/s 2024-09-19T06:28:10.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:10 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3048437299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:28:11.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:10 smithi157 ceph-mon[35346]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.9 MiB/s wr, 430 op/s 2024-09-19T06:28:11.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:10 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3048437299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:28:11.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 111425 8.52 MB/sec execute 575 sec latency 141.065 ms 2024-09-19T06:28:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.399+0000 7fc41e3bd640 1 -- 172.21.15.5:0/3255030664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4181009a0 msgr2=0x7fc418100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.399+0000 7fc41e3bd640 1 --2- 172.21.15.5:0/3255030664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4181009a0 0x7fc418100e00 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009930 tx=0x7fc40c02a420 comp rx=0 tx=0).stop 2024-09-19T06:28:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.399+0000 7fc41e3bd640 1 -- 172.21.15.5:0/3255030664 shutdown_connections 2024-09-19T06:28:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.399+0000 7fc41e3bd640 1 --2- 172.21.15.5:0/3255030664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc418101340 0x7fc41810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.399+0000 7fc41e3bd640 1 --2- 172.21.15.5:0/3255030664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4181009a0 0x7fc418100e00 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.399+0000 7fc41e3bd640 1 --2- 172.21.15.5:0/3255030664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4181069b0 0x7fc418106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.399+0000 7fc41e3bd640 1 -- 172.21.15.5:0/3255030664 >> 172.21.15.5:0/3255030664 conn(0x7fc4180779a0 msgr2=0x7fc4180fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.399+0000 7fc41e3bd640 1 -- 172.21.15.5:0/3255030664 shutdown_connections 2024-09-19T06:28:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.399+0000 7fc41e3bd640 1 -- 172.21.15.5:0/3255030664 wait complete. 2024-09-19T06:28:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.400+0000 7fc41e3bd640 1 Processor -- start 2024-09-19T06:28:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.400+0000 7fc41e3bd640 1 -- start start 2024-09-19T06:28:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc41e3bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc418101340 0x7fc418198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc41e3bd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4181069b0 0x7fc418198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc41e3bd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc41819cde0 0x7fc41819d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc41e3bd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc418110cb0 con 0x7fc418101340 2024-09-19T06:28:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc41e3bd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc418110b30 con 0x7fc41819cde0 2024-09-19T06:28:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc41e3bd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc418110e30 con 0x7fc4181069b0 2024-09-19T06:28:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc417fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc418101340 0x7fc418198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc4177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4181069b0 0x7fc418198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc4177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4181069b0 0x7fc418198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46130/0 (socket says 172.21.15.5:46130) 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc4177fe640 1 -- 172.21.15.5:0/2871605884 learned_addr learned my addr 172.21.15.5:0/2871605884 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc417fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc418101340 0x7fc418198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42388/0 (socket says 172.21.15.5:42388) 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc41c95a640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc41819cde0 0x7fc41819d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc4177fe640 1 -- 172.21.15.5:0/2871605884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc41819cde0 msgr2=0x7fc41819d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc4177fe640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc41819cde0 0x7fc41819d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc4177fe640 1 -- 172.21.15.5:0/2871605884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc418101340 msgr2=0x7fc418198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc4177fe640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc418101340 0x7fc418198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc4177fe640 1 -- 172.21.15.5:0/2871605884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41819d8d0 con 0x7fc4181069b0 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.401+0000 7fc417fff640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc418101340 0x7fc418198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.402+0000 7fc4177fe640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4181069b0 0x7fc418198a60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009900 tx=0x7fc40c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.402+0000 7fc4157fa640 1 -- 172.21.15.5:0/2871605884 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c03f070 con 0x7fc4181069b0 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.402+0000 7fc4157fa640 1 -- 172.21.15.5:0/2871605884 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc40c031b60 con 0x7fc4181069b0 2024-09-19T06:28:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.402+0000 7fc41e3bd640 1 -- 172.21.15.5:0/2871605884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41819db60 con 0x7fc4181069b0 2024-09-19T06:28:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.402+0000 7fc4157fa640 1 -- 172.21.15.5:0/2871605884 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c035070 con 0x7fc4181069b0 2024-09-19T06:28:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.402+0000 7fc41e3bd640 1 -- 172.21.15.5:0/2871605884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4181a52d0 con 0x7fc4181069b0 2024-09-19T06:28:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.405+0000 7fc421234740 1 Processor -- start 2024-09-19T06:28:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.405+0000 7fc4157fa640 1 -- 172.21.15.5:0/2871605884 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc40c03b460 con 0x7fc4181069b0 2024-09-19T06:28:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.405+0000 7fc421234740 1 -- start start 2024-09-19T06:28:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc421234740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b10af0 0x558740b10ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc421234740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b11410 0x558740b1b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc421234740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b1bcb0 0x558740b1e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc421234740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558740a752f0 con 0x558740b1bcb0 2024-09-19T06:28:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc421234740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558740a1d270 con 0x558740b10af0 2024-09-19T06:28:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc421234740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558740a712b0 con 0x558740b11410 2024-09-19T06:28:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc4177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b1bcb0 0x558740b1e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc417fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b11410 0x558740b1b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc417fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b11410 0x558740b1b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46136/0 (socket says 172.21.15.5:46136) 2024-09-19T06:28:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc417fff640 1 -- 172.21.15.5:0/1795351648 learned_addr learned my addr 172.21.15.5:0/1795351648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc41c95a640 1 --2- 172.21.15.5:0/1795351648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b10af0 0x558740b10ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc417fff640 1 -- 172.21.15.5:0/1795351648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b10af0 msgr2=0x558740b10ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc417fff640 1 --2- 172.21.15.5:0/1795351648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b10af0 0x558740b10ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc417fff640 1 -- 172.21.15.5:0/1795351648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b1bcb0 msgr2=0x558740b1e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc417fff640 1 --2- 172.21.15.5:0/1795351648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b1bcb0 0x558740b1e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc417fff640 1 -- 172.21.15.5:0/1795351648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558740a1c340 con 0x558740b11410 2024-09-19T06:28:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.406+0000 7fc4157fa640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc4000a33b0 0x7fc4000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.407+0000 7fc417fff640 1 --2- 172.21.15.5:0/1795351648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b11410 0x558740b1b770 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc4080114c0 tx=0x7fc408011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a33865410a38576 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.407+0000 7fc4157fa640 1 -- 172.21.15.5:0/2871605884 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fc40c0efa50 con 0x7fc4181069b0 2024-09-19T06:28:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.407+0000 7fc41c95a640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc4000a33b0 0x7fc4000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.407+0000 7fc3fe7fc640 1 -- 172.21.15.5:0/1795351648 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40800f5c0 con 0x558740b11410 2024-09-19T06:28:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.407+0000 7fc3fe7fc640 1 -- 172.21.15.5:0/1795351648 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc40800fc10 con 0x558740b11410 2024-09-19T06:28:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.408+0000 7fc3fe7fc640 1 -- 172.21.15.5:0/1795351648 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408019d10 con 0x558740b11410 2024-09-19T06:28:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.408+0000 7fc41c95a640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc4000a33b0 0x7fc4000a5870 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc410005fd0 tx=0x7fc410009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.408+0000 7fc421234740 1 -- 172.21.15.5:0/1795351648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b11410 msgr2=0x558740b1b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.408+0000 7fc421234740 1 --2- 172.21.15.5:0/1795351648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b11410 0x558740b1b770 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc4080114c0 tx=0x7fc408011980 comp rx=0 tx=0).stop 2024-09-19T06:28:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.408+0000 7fc421234740 1 -- 172.21.15.5:0/1795351648 shutdown_connections 2024-09-19T06:28:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.408+0000 7fc421234740 1 --2- 172.21.15.5:0/1795351648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b1bcb0 0x558740b1e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.408+0000 7fc421234740 1 --2- 172.21.15.5:0/1795351648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b11410 0x558740b1b770 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.408+0000 7fc421234740 1 --2- 172.21.15.5:0/1795351648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b10af0 0x558740b10ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.408+0000 7fc421234740 1 -- 172.21.15.5:0/1795351648 >> 172.21.15.5:0/1795351648 conn(0x558740b0d060 msgr2=0x558740b0d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.408+0000 7fc421234740 1 -- 172.21.15.5:0/1795351648 shutdown_connections 2024-09-19T06:28:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.409+0000 7fc421234740 1 -- 172.21.15.5:0/1795351648 wait complete. 2024-09-19T06:28:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.409+0000 7fc421234740 1 -- start start 2024-09-19T06:28:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.409+0000 7fc421234740 1 Processor -- start 2024-09-19T06:28:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc421234740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 0x558740b18b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc421234740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b11410 0x558740b19060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc421234740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b195a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc417fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 0x558740b18b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc421234740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558740b22d30 con 0x558740b10af0 2024-09-19T06:28:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc417fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 0x558740b18b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42408/0 (socket says 172.21.15.5:42408) 2024-09-19T06:28:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc417fff640 1 -- 172.21.15.5:0/1899171537 learned_addr learned my addr 172.21.15.5:0/1899171537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc421234740 1 -- 172.21.15.5:0/1899171537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5587409fe290 con 0x558740b11410 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc421234740 1 -- 172.21.15.5:0/1899171537 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558740b22eb0 con 0x558740b1bcb0 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc4177fe640 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b11410 0x558740b19060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc41c95a640 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc421234740 10 client.0 _ll_register_callbacks cb 0x558740a2f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc417fff640 1 -- 172.21.15.5:0/1899171537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 msgr2=0x558740b195a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc417fff640 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b195a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc417fff640 1 -- 172.21.15.5:0/1899171537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b11410 msgr2=0x558740b19060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc417fff640 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b11410 0x558740b19060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.410+0000 7fc417fff640 1 -- 172.21.15.5:0/1899171537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558740788210 con 0x558740b10af0 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.411+0000 7fc417fff640 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 0x558740b18b20 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fc4080198f0 tx=0x7fc408005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbee91e0879f6e11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.411+0000 7fc3f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.411+0000 7fc3f77fe640 1 -- 172.21.15.5:0/1899171537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408027060 con 0x558740b10af0 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.411+0000 7fc3f77fe640 1 -- 172.21.15.5:0/1899171537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fc40800f7d0 con 0x558740b10af0 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.411+0000 7fc421234740 1 -- 172.21.15.5:0/1899171537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc40800fc10 con 0x558740b10af0 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.411+0000 7fc3f77fe640 1 -- 172.21.15.5:0/1899171537 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc408009cc0 con 0x558740b10af0 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.411+0000 7fc421234740 10 client.21480 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.411+0000 7fc421234740 1 -- 172.21.15.5:0/1899171537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x558740854620 con 0x558740b10af0 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.412+0000 7fc3f77fe640 1 -- 172.21.15.5:0/1899171537 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc40800fc10 con 0x558740b10af0 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.412+0000 7fc3f77fe640 1 -- 172.21.15.5:0/1899171537 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc408004940 con 0x558740b10af0 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.412+0000 7fc421234740 10 client.21480 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.412+0000 7fc421234740 10 client.21480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.412+0000 7fc421234740 20 client.21480 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.412+0000 7fc421234740 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558740b30020 0x558740b32410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.412+0000 7fc421234740 4 client.21480 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.412+0000 7fc421234740 1 -- 172.21.15.5:0/1899171537 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558740a82a20 con 0x558740b30020 2024-09-19T06:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.412+0000 7fc4177fe640 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558740b30020 0x558740b32410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.413+0000 7fc3f77fe640 10 client.21480 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.413+0000 7fc4177fe640 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558740b30020 0x558740b32410 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add810d35113d726 server_cookie=541597645ab78d66 in_seq=0 out_seq=0 2024-09-19T06:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc3f77fe640 1 -- 172.21.15.5:0/1899171537 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558740a82a20 con 0x558740b30020 2024-09-19T06:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc3f77fe640 10 client.21480 handle_command_reply: tid=0 2024-09-19T06:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc421234740 1 client.21480 shutdown 2024-09-19T06:28:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc421234740 1 -- 172.21.15.5:0/1899171537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 msgr2=0x558740b18b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc421234740 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 0x558740b18b20 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fc4080198f0 tx=0x7fc408005d90 comp rx=0 tx=0).stop 2024-09-19T06:28:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc421234740 1 -- 172.21.15.5:0/1899171537 shutdown_connections 2024-09-19T06:28:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc421234740 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558740b30020 0x558740b32410 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc421234740 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b195a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc421234740 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b11410 0x558740b19060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc421234740 1 --2- 172.21.15.5:0/1899171537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 0x558740b18b20 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc421234740 1 -- 172.21.15.5:0/1899171537 >> 172.21.15.5:0/1899171537 conn(0x558740b0d060 msgr2=0x558740b12d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc3f77fe640 0 client.21480 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc421234740 1 -- 172.21.15.5:0/1899171537 shutdown_connections 2024-09-19T06:28:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.414+0000 7fc421234740 1 -- 172.21.15.5:0/1899171537 wait complete. 2024-09-19T06:28:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.415+0000 7fc421234740 20 client.21480 trim_cache size 0 max 16384 2024-09-19T06:28:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.436+0000 7fc421234740 1 Processor -- start 2024-09-19T06:28:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.436+0000 7fc421234740 1 -- start start 2024-09-19T06:28:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.436+0000 7fc421234740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b10af0 0x558740b13560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.436+0000 7fc421234740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b11410 0x558740b1aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.436+0000 7fc421234740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b37b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.436+0000 7fc421234740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4181abad0 con 0x558740b11410 2024-09-19T06:28:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.436+0000 7fc421234740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4181ad610 con 0x558740b10af0 2024-09-19T06:28:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.436+0000 7fc421234740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4181ac1a0 con 0x558740b1bcb0 2024-09-19T06:28:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.436+0000 7fc417fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b10af0 0x558740b13560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.436+0000 7fc417fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b10af0 0x558740b13560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:34378/0 (socket says 172.21.15.5:34378) 2024-09-19T06:28:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.436+0000 7fc417fff640 1 -- 172.21.15.5:0/3027828815 learned_addr learned my addr 172.21.15.5:0/3027828815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.436+0000 7fc4177fe640 1 --2- 172.21.15.5:0/3027828815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b11410 0x558740b1aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.437+0000 7fc41c95a640 1 --2- 172.21.15.5:0/3027828815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b37b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.437+0000 7fc417fff640 1 -- 172.21.15.5:0/3027828815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 msgr2=0x558740b37b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.437+0000 7fc417fff640 1 --2- 172.21.15.5:0/3027828815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b37b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.437+0000 7fc417fff640 1 -- 172.21.15.5:0/3027828815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b11410 msgr2=0x558740b1aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.437+0000 7fc417fff640 1 --2- 172.21.15.5:0/3027828815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b11410 0x558740b1aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.437+0000 7fc417fff640 1 -- 172.21.15.5:0/3027828815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558740b11240 con 0x558740b10af0 2024-09-19T06:28:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.437+0000 7fc41c95a640 1 --2- 172.21.15.5:0/3027828815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b37b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:28:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.437+0000 7fc417fff640 1 --2- 172.21.15.5:0/3027828815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b10af0 0x558740b13560 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc408004d60 tx=0x7fc40800ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca785ef21f9c8f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.437+0000 7fc3fdffb640 1 -- 172.21.15.5:0/3027828815 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408022440 con 0x558740b10af0 2024-09-19T06:28:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.437+0000 7fc3fdffb640 1 -- 172.21.15.5:0/3027828815 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc408015070 con 0x558740b10af0 2024-09-19T06:28:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.437+0000 7fc3fdffb640 1 -- 172.21.15.5:0/3027828815 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408022e00 con 0x558740b10af0 2024-09-19T06:28:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.438+0000 7fc421234740 1 -- 172.21.15.5:0/3027828815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b10af0 msgr2=0x558740b13560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.438+0000 7fc421234740 1 --2- 172.21.15.5:0/3027828815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b10af0 0x558740b13560 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc408004d60 tx=0x7fc40800ffd0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.438+0000 7fc421234740 1 -- 172.21.15.5:0/3027828815 shutdown_connections 2024-09-19T06:28:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.438+0000 7fc421234740 1 --2- 172.21.15.5:0/3027828815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b37b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.439+0000 7fc421234740 1 --2- 172.21.15.5:0/3027828815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b11410 0x558740b1aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.439+0000 7fc421234740 1 --2- 172.21.15.5:0/3027828815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b10af0 0x558740b13560 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.439+0000 7fc421234740 1 -- 172.21.15.5:0/3027828815 >> 172.21.15.5:0/3027828815 conn(0x558740b0d060 msgr2=0x558740b1a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.439+0000 7fc421234740 1 -- 172.21.15.5:0/3027828815 shutdown_connections 2024-09-19T06:28:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.439+0000 7fc421234740 1 -- 172.21.15.5:0/3027828815 wait complete. 2024-09-19T06:28:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.439+0000 7fc421234740 1 -- start start 2024-09-19T06:28:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.439+0000 7fc421234740 1 Processor -- start 2024-09-19T06:28:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc421234740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 0x558740b23730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc421234740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b11410 0x558740b23c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc421234740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b98af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc421234740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4181acf40 con 0x558740b10af0 2024-09-19T06:28:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc421234740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4181ac870 con 0x558740b11410 2024-09-19T06:28:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc421234740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4181a9ad0 con 0x558740b1bcb0 2024-09-19T06:28:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc421234740 10 client.0 _ll_register_callbacks cb 0x7fc40c0f3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc417fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 0x558740b23730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc4177fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b11410 0x558740b23c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc417fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 0x558740b23730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42426/0 (socket says 172.21.15.5:42426) 2024-09-19T06:28:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc417fff640 1 -- 172.21.15.5:0/2142806668 learned_addr learned my addr 172.21.15.5:0/2142806668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc41c95a640 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b98af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc417fff640 1 -- 172.21.15.5:0/2142806668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 msgr2=0x558740b98af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc417fff640 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b98af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc417fff640 1 -- 172.21.15.5:0/2142806668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b11410 msgr2=0x558740b23c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.440+0000 7fc417fff640 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b11410 0x558740b23c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.441+0000 7fc417fff640 1 -- 172.21.15.5:0/2142806668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558740b99030 con 0x558740b10af0 2024-09-19T06:28:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.441+0000 7fc4177fe640 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b11410 0x558740b23c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.441+0000 7fc417fff640 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 0x558740b23730 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fc4080059f0 tx=0x7fc408022510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207f2daa7921d5c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.441+0000 7fc3f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:28:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.441+0000 7fc3f77fe640 1 -- 172.21.15.5:0/2142806668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408005c50 con 0x558740b10af0 2024-09-19T06:28:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.441+0000 7fc3f77fe640 1 -- 172.21.15.5:0/2142806668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fc408022610 con 0x558740b10af0 2024-09-19T06:28:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.441+0000 7fc421234740 1 -- 172.21.15.5:0/2142806668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc408015070 con 0x558740b10af0 2024-09-19T06:28:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.441+0000 7fc3f77fe640 1 -- 172.21.15.5:0/2142806668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc408022810 con 0x558740b10af0 2024-09-19T06:28:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.441+0000 7fc421234740 10 client.21486 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.441+0000 7fc421234740 1 -- 172.21.15.5:0/2142806668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x558740b9a270 con 0x558740b10af0 2024-09-19T06:28:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.442+0000 7fc3f77fe640 1 -- 172.21.15.5:0/2142806668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc408015070 con 0x558740b10af0 2024-09-19T06:28:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.442+0000 7fc3f77fe640 1 -- 172.21.15.5:0/2142806668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc408005520 con 0x558740b10af0 2024-09-19T06:28:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.442+0000 7fc421234740 10 client.21486 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.442+0000 7fc421234740 10 client.21486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.442+0000 7fc421234740 20 client.21486 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.442+0000 7fc421234740 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558740b9a850 0x558740b9cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.442+0000 7fc421234740 4 client.21486 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:28:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.442+0000 7fc421234740 1 -- 172.21.15.5:0/2142806668 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558740a7cd70 con 0x558740b9a850 2024-09-19T06:28:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.442+0000 7fc4177fe640 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558740b9a850 0x558740b9cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.443+0000 7fc3f77fe640 10 client.21486 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.443+0000 7fc4177fe640 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558740b9a850 0x558740b9cc70 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36fa5ac780d30fdc server_cookie=71d48dd6ac078082 in_seq=0 out_seq=0 2024-09-19T06:28:12.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 111594 8.52 MB/sec execute 576 sec latency 154.483 ms 2024-09-19T06:28:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.610+0000 7fc3f77fe640 1 -- 172.21.15.5:0/2142806668 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558740a7cd70 con 0x558740b9a850 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.610+0000 7fc3f77fe640 10 client.21486 handle_command_reply: tid=0 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.610+0000 7fc421234740 1 client.21486 shutdown 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.610+0000 7fc421234740 1 -- 172.21.15.5:0/2142806668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 msgr2=0x558740b23730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.610+0000 7fc421234740 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 0x558740b23730 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fc4080059f0 tx=0x7fc408022510 comp rx=0 tx=0).stop 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.610+0000 7fc421234740 1 -- 172.21.15.5:0/2142806668 shutdown_connections 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.610+0000 7fc421234740 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x558740b9a850 0x558740b9cc70 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.610+0000 7fc421234740 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558740b1bcb0 0x558740b98af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.610+0000 7fc421234740 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558740b11410 0x558740b23c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.610+0000 7fc421234740 1 --2- 172.21.15.5:0/2142806668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558740b10af0 0x558740b23730 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.610+0000 7fc421234740 1 -- 172.21.15.5:0/2142806668 >> 172.21.15.5:0/2142806668 conn(0x558740b0d060 msgr2=0x558740b1a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.610+0000 7fc3f77fe640 0 client.21486 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.611+0000 7fc421234740 1 -- 172.21.15.5:0/2142806668 shutdown_connections 2024-09-19T06:28:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.611+0000 7fc421234740 1 -- 172.21.15.5:0/2142806668 wait complete. 2024-09-19T06:28:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.611+0000 7fc421234740 20 client.21486 trim_cache size 0 max 16384 2024-09-19T06:28:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:28:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:28:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:28:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:28:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.611+0000 7fc41e3bd640 1 -- 172.21.15.5:0/2871605884 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc4000a33b0 msgr2=0x7fc4000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.611+0000 7fc41e3bd640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc4000a33b0 0x7fc4000a5870 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc410005fd0 tx=0x7fc410009500 comp rx=0 tx=0).stop 2024-09-19T06:28:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.611+0000 7fc41e3bd640 1 -- 172.21.15.5:0/2871605884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4181069b0 msgr2=0x7fc418198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.611+0000 7fc41e3bd640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4181069b0 0x7fc418198a60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009900 tx=0x7fc40c003c10 comp rx=0 tx=0).stop 2024-09-19T06:28:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.612+0000 7fc41e3bd640 1 -- 172.21.15.5:0/2871605884 shutdown_connections 2024-09-19T06:28:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.612+0000 7fc41e3bd640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc41819cde0 0x7fc41819d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.612+0000 7fc41e3bd640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc4000a33b0 0x7fc4000a5870 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.612+0000 7fc41e3bd640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4181069b0 0x7fc418198a60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.612+0000 7fc41e3bd640 1 --2- 172.21.15.5:0/2871605884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc418101340 0x7fc418198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.612+0000 7fc41e3bd640 1 -- 172.21.15.5:0/2871605884 >> 172.21.15.5:0/2871605884 conn(0x7fc4180779a0 msgr2=0x7fc4180ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.612+0000 7fc41e3bd640 1 -- 172.21.15.5:0/2871605884 shutdown_connections 2024-09-19T06:28:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:12.612+0000 7fc41e3bd640 1 -- 172.21.15.5:0/2871605884 wait complete. 2024-09-19T06:28:12.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:12 smithi005 ceph-mon[30955]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 185 op/s 2024-09-19T06:28:12.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:12 smithi049 ceph-mon[34640]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 185 op/s 2024-09-19T06:28:13.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:12 smithi157 ceph-mon[35346]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 185 op/s 2024-09-19T06:28:13.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 111729 8.53 MB/sec execute 577 sec latency 150.506 ms 2024-09-19T06:28:14.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 111854 8.52 MB/sec execute 578 sec latency 129.665 ms 2024-09-19T06:28:14.838 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:14 smithi005 ceph-mon[30955]: pgmap v788: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.3 MiB/s wr, 345 op/s 2024-09-19T06:28:14.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:28:14.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:14 smithi049 ceph-mon[34640]: pgmap v788: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.3 MiB/s wr, 345 op/s 2024-09-19T06:28:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:14 smithi157 ceph-mon[35346]: pgmap v788: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.3 MiB/s wr, 345 op/s 2024-09-19T06:28:15.231 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:28:15.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 112038 8.53 MB/sec execute 579 sec latency 81.189 ms 2024-09-19T06:28:16.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 112166 8.52 MB/sec execute 580 sec latency 118.217 ms 2024-09-19T06:28:16.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:16 smithi005 ceph-mon[30955]: pgmap v789: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.3 MiB/s wr, 298 op/s 2024-09-19T06:28:16.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:16 smithi049 ceph-mon[34640]: pgmap v789: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.3 MiB/s wr, 298 op/s 2024-09-19T06:28:17.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:16 smithi157 ceph-mon[35346]: pgmap v789: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.3 MiB/s wr, 298 op/s 2024-09-19T06:28:17.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.252+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1631531224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708100a90 msgr2=0x7fc708100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:17.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.252+0000 7fc70ffb7640 1 --2- 172.21.15.5:0/1631531224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708100a90 0x7fc708100ef0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009860 tx=0x7fc6f802a340 comp rx=0 tx=0).stop 2024-09-19T06:28:17.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.252+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1631531224 shutdown_connections 2024-09-19T06:28:17.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.252+0000 7fc70ffb7640 1 --2- 172.21.15.5:0/1631531224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc708101430 0x7fc70810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.252+0000 7fc70ffb7640 1 --2- 172.21.15.5:0/1631531224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708100a90 0x7fc708100ef0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.252+0000 7fc70ffb7640 1 --2- 172.21.15.5:0/1631531224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc708106a90 0x7fc708106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.252+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1631531224 >> 172.21.15.5:0/1631531224 conn(0x7fc7080fc7d0 msgr2=0x7fc7080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:17.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.253+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1631531224 shutdown_connections 2024-09-19T06:28:17.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.253+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1631531224 wait complete. 2024-09-19T06:28:17.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.253+0000 7fc70ffb7640 1 Processor -- start 2024-09-19T06:28:17.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.254+0000 7fc70ffb7640 1 -- start start 2024-09-19T06:28:17.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.254+0000 7fc70ffb7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708100a90 0x7fc7081972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.254+0000 7fc70ffb7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc708101430 0x7fc708197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.254+0000 7fc70ffb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc708106a90 0x7fc70819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.254+0000 7fc70ffb7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc708110d40 con 0x7fc708106a90 2024-09-19T06:28:17.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.254+0000 7fc70ffb7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc708110bc0 con 0x7fc708100a90 2024-09-19T06:28:17.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.254+0000 7fc70ffb7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc708110ec0 con 0x7fc708101430 2024-09-19T06:28:17.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.254+0000 7fc70dd53640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708100a90 0x7fc7081972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.254+0000 7fc70d552640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc708101430 0x7fc708197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.254+0000 7fc70d552640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc708101430 0x7fc708197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45552/0 (socket says 172.21.15.5:45552) 2024-09-19T06:28:17.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.254+0000 7fc70d552640 1 -- 172.21.15.5:0/1440030133 learned_addr learned my addr 172.21.15.5:0/1440030133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:17.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.254+0000 7fc70e554640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc708106a90 0x7fc70819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.255+0000 7fc70d552640 1 -- 172.21.15.5:0/1440030133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708100a90 msgr2=0x7fc7081972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.255+0000 7fc70d552640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708100a90 0x7fc7081972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.255+0000 7fc70d552640 1 -- 172.21.15.5:0/1440030133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc708106a90 msgr2=0x7fc70819bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.255+0000 7fc70d552640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc708106a90 0x7fc70819bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.255+0000 7fc70d552640 1 -- 172.21.15.5:0/1440030133 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70819c2b0 con 0x7fc708101430 2024-09-19T06:28:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.255+0000 7fc70d552640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc708101430 0x7fc708197820 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009980 tx=0x7fc6f8030df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.255+0000 7fc6feffd640 1 -- 172.21.15.5:0/1440030133 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8037050 con 0x7fc708101430 2024-09-19T06:28:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.255+0000 7fc6feffd640 1 -- 172.21.15.5:0/1440030133 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6f8043070 con 0x7fc708101430 2024-09-19T06:28:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.255+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1440030133 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70819c540 con 0x7fc708101430 2024-09-19T06:28:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.255+0000 7fc6feffd640 1 -- 172.21.15.5:0/1440030133 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8003c50 con 0x7fc708101430 2024-09-19T06:28:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.255+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1440030133 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70806a790 con 0x7fc708101430 2024-09-19T06:28:17.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.259+0000 7fc712390740 1 Processor -- start 2024-09-19T06:28:17.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.259+0000 7fc6feffd640 1 -- 172.21.15.5:0/1440030133 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc6f8004040 con 0x7fc708101430 2024-09-19T06:28:17.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.259+0000 7fc712390740 1 -- start start 2024-09-19T06:28:17.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.260+0000 7fc712390740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 0x555cfed01090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.260+0000 7fc712390740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed01a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.260+0000 7fc712390740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed0c370 0x555cfed0e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.260+0000 7fc712390740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555cfec672f0 con 0x555cfed015d0 2024-09-19T06:28:17.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.260+0000 7fc712390740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555cfec0ecf0 con 0x555cfed02b40 2024-09-19T06:28:17.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.260+0000 7fc712390740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555cfec632b0 con 0x555cfed0c370 2024-09-19T06:28:17.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.260+0000 7fc70e554640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 0x555cfed01090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.260+0000 7fc70e554640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 0x555cfed01090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48066/0 (socket says 172.21.15.5:48066) 2024-09-19T06:28:17.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.260+0000 7fc70dd53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed01a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.260+0000 7fc70e554640 1 -- 172.21.15.5:0/3344233476 learned_addr learned my addr 172.21.15.5:0/3344233476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:17.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.260+0000 7fc70d552640 1 --2- 172.21.15.5:0/3344233476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed0c370 0x555cfed0e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.260+0000 7fc70e554640 1 -- 172.21.15.5:0/3344233476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed0c370 msgr2=0x555cfed0e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:17.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.261+0000 7fc70e554640 1 --2- 172.21.15.5:0/3344233476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed0c370 0x555cfed0e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.261+0000 7fc70e554640 1 -- 172.21.15.5:0/3344233476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 msgr2=0x555cfed01a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.261+0000 7fc70e554640 1 --2- 172.21.15.5:0/3344233476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed01a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.261+0000 7fc70e554640 1 -- 172.21.15.5:0/3344233476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cfebf33a0 con 0x555cfed02b40 2024-09-19T06:28:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.261+0000 7fc70dd53640 1 --2- 172.21.15.5:0/3344233476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed01a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.261+0000 7fc70e554640 1 --2- 172.21.15.5:0/3344233476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 0x555cfed01090 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc70000e450 tx=0x7fc7000095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60078b719e8b0e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.261+0000 7fc6feffd640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc6f40a3430 0x7fc6f40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.261+0000 7fc6f3fff640 1 -- 172.21.15.5:0/3344233476 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70000d910 con 0x555cfed02b40 2024-09-19T06:28:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.261+0000 7fc6f3fff640 1 -- 172.21.15.5:0/3344233476 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc70001c070 con 0x555cfed02b40 2024-09-19T06:28:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.261+0000 7fc6f3fff640 1 -- 172.21.15.5:0/3344233476 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700017db0 con 0x555cfed02b40 2024-09-19T06:28:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.261+0000 7fc70dd53640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc6f40a3430 0x7fc6f40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.261+0000 7fc6feffd640 1 -- 172.21.15.5:0/1440030133 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fc6f80efeb0 con 0x7fc708101430 2024-09-19T06:28:17.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.262+0000 7fc712390740 1 -- 172.21.15.5:0/3344233476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 msgr2=0x555cfed01090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:17.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.262+0000 7fc712390740 1 --2- 172.21.15.5:0/3344233476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 0x555cfed01090 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc70000e450 tx=0x7fc7000095c0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.262+0000 7fc70dd53640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc6f40a3430 0x7fc6f40a58f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x555cfecfeac0 tx=0x7fc704005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:17.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.262+0000 7fc712390740 1 -- 172.21.15.5:0/3344233476 shutdown_connections 2024-09-19T06:28:17.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.262+0000 7fc712390740 1 --2- 172.21.15.5:0/3344233476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed0c370 0x555cfed0e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.262+0000 7fc712390740 1 --2- 172.21.15.5:0/3344233476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed01a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.262+0000 7fc712390740 1 --2- 172.21.15.5:0/3344233476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 0x555cfed01090 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.262+0000 7fc712390740 1 -- 172.21.15.5:0/3344233476 >> 172.21.15.5:0/3344233476 conn(0x555cfecca310 msgr2=0x555cfecfdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:17.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.262+0000 7fc712390740 1 -- 172.21.15.5:0/3344233476 shutdown_connections 2024-09-19T06:28:17.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.262+0000 7fc712390740 1 -- 172.21.15.5:0/3344233476 wait complete. 2024-09-19T06:28:17.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.263+0000 7fc712390740 1 -- start start 2024-09-19T06:28:17.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.263+0000 7fc712390740 1 Processor -- start 2024-09-19T06:28:17.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.263+0000 7fc712390740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed0bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.263+0000 7fc712390740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 0x555cfed09000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.263+0000 7fc712390740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed0c370 0x555cfed095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.263+0000 7fc712390740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555cfed0f070 con 0x555cfed015d0 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.263+0000 7fc712390740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555cfec22060 con 0x555cfed02b40 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.263+0000 7fc712390740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555cfed0f1f0 con 0x555cfed0c370 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.263+0000 7fc712390740 10 client.0 _ll_register_callbacks cb 0x555cfe9758c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.263+0000 7fc70e554640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed0c370 0x555cfed095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.263+0000 7fc70dd53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed0bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc70e554640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed0c370 0x555cfed095d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45568/0 (socket says 172.21.15.5:45568) 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc70e554640 1 -- 172.21.15.5:0/1815743605 learned_addr learned my addr 172.21.15.5:0/1815743605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc70d552640 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 0x555cfed09000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc70e554640 1 -- 172.21.15.5:0/1815743605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 msgr2=0x555cfed09000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc70e554640 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 0x555cfed09000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc70e554640 1 -- 172.21.15.5:0/1815743605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 msgr2=0x555cfed0bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc70e554640 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed0bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc70e554640 1 -- 172.21.15.5:0/1815743605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cfe9d83c0 con 0x555cfed0c370 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc70d552640 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 0x555cfed09000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc70dd53640 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed0bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc70e554640 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed0c370 0x555cfed095d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc700000f80 tx=0x7fc700017dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c35857c2ac753e96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc6f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.264+0000 7fc6f17fa640 1 -- 172.21.15.5:0/1815743605 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700004a40 con 0x555cfed0c370 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.265+0000 7fc6f17fa640 1 -- 172.21.15.5:0/1815743605 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fc700004be0 con 0x555cfed0c370 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.265+0000 7fc712390740 1 -- 172.21.15.5:0/1815743605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70001c070 con 0x555cfed0c370 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.265+0000 7fc6f17fa640 1 -- 172.21.15.5:0/1815743605 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc700004de0 con 0x555cfed0c370 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.265+0000 7fc712390740 10 client.30793 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.265+0000 7fc712390740 1 -- 172.21.15.5:0/1815743605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x555cfed1aaf0 con 0x555cfed0c370 2024-09-19T06:28:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.265+0000 7fc6f17fa640 1 -- 172.21.15.5:0/1815743605 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc70001c070 con 0x555cfed0c370 2024-09-19T06:28:17.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.265+0000 7fc6f17fa640 1 -- 172.21.15.5:0/1815743605 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc70000b040 con 0x555cfed0c370 2024-09-19T06:28:17.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.265+0000 7fc712390740 10 client.30793 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:17.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.265+0000 7fc712390740 10 client.30793 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:28:17.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.265+0000 7fc712390740 20 client.30793 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:17.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.265+0000 7fc712390740 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555cfed1bf50 0x555cfed1e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.266+0000 7fc712390740 4 client.30793 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:28:17.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.266+0000 7fc712390740 1 -- 172.21.15.5:0/1815743605 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cfec74860 con 0x555cfed1bf50 2024-09-19T06:28:17.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.266+0000 7fc70dd53640 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555cfed1bf50 0x555cfed1e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.266+0000 7fc6f17fa640 10 client.30793 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:28:17.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.266+0000 7fc70dd53640 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555cfed1bf50 0x555cfed1e340 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=11ef1009844fd34b server_cookie=666dd1cd4e1c3d91 in_seq=0 out_seq=0 2024-09-19T06:28:17.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.267+0000 7fc6f17fa640 1 -- 172.21.15.5:0/1815743605 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cfec74860 con 0x555cfed1bf50 2024-09-19T06:28:17.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.267+0000 7fc6f17fa640 10 client.30793 handle_command_reply: tid=0 2024-09-19T06:28:17.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.268+0000 7fc712390740 1 client.30793 shutdown 2024-09-19T06:28:17.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.268+0000 7fc712390740 1 -- 172.21.15.5:0/1815743605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed0c370 msgr2=0x555cfed095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:17.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.268+0000 7fc712390740 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed0c370 0x555cfed095d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc700000f80 tx=0x7fc700017dc0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.268+0000 7fc712390740 1 -- 172.21.15.5:0/1815743605 shutdown_connections 2024-09-19T06:28:17.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.268+0000 7fc712390740 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555cfed1bf50 0x555cfed1e340 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.268+0000 7fc712390740 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed0c370 0x555cfed095d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.268+0000 7fc6f17fa640 0 client.30793 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:28:17.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.268+0000 7fc712390740 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed02b40 0x555cfed09000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.268+0000 7fc712390740 1 --2- 172.21.15.5:0/1815743605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed0bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.268+0000 7fc712390740 1 -- 172.21.15.5:0/1815743605 >> 172.21.15.5:0/1815743605 conn(0x555cfecca310 msgr2=0x555cfed03930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:17.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.269+0000 7fc712390740 1 -- 172.21.15.5:0/1815743605 shutdown_connections 2024-09-19T06:28:17.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.269+0000 7fc712390740 1 -- 172.21.15.5:0/1815743605 wait complete. 2024-09-19T06:28:17.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.269+0000 7fc712390740 20 client.30793 trim_cache size 0 max 16384 2024-09-19T06:28:17.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.284+0000 7fc712390740 1 Processor -- start 2024-09-19T06:28:17.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.284+0000 7fc712390740 1 -- start start 2024-09-19T06:28:17.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc712390740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed18310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc712390740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed02b40 0x555cfed238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc712390740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 0x555cfed23e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc712390740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081ab6d0 con 0x555cfed015d0 2024-09-19T06:28:17.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc712390740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081ad210 con 0x555cfed0c370 2024-09-19T06:28:17.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc712390740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081abda0 con 0x555cfed02b40 2024-09-19T06:28:17.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc70e554640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 0x555cfed23e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc70e554640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 0x555cfed23e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48076/0 (socket says 172.21.15.5:48076) 2024-09-19T06:28:17.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc70e554640 1 -- 172.21.15.5:0/361605124 learned_addr learned my addr 172.21.15.5:0/361605124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:17.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc70dd53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed18310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc70d552640 1 --2- 172.21.15.5:0/361605124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed02b40 0x555cfed238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc70e554640 1 -- 172.21.15.5:0/361605124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed02b40 msgr2=0x555cfed238e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:17.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc70e554640 1 --2- 172.21.15.5:0/361605124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed02b40 0x555cfed238e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc70e554640 1 -- 172.21.15.5:0/361605124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 msgr2=0x555cfed18310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:17.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc70e554640 1 --2- 172.21.15.5:0/361605124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed18310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.285+0000 7fc70e554640 1 -- 172.21.15.5:0/361605124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cfed0c0c0 con 0x555cfed0c370 2024-09-19T06:28:17.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.286+0000 7fc70e554640 1 --2- 172.21.15.5:0/361605124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 0x555cfed23e20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc700006010 tx=0x7fc700005610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6337401473c4b171 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:17.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.286+0000 7fc6f37fe640 1 -- 172.21.15.5:0/361605124 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700038030 con 0x555cfed0c370 2024-09-19T06:28:17.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.286+0000 7fc6f37fe640 1 -- 172.21.15.5:0/361605124 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc70001c040 con 0x555cfed0c370 2024-09-19T06:28:17.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.286+0000 7fc6f37fe640 1 -- 172.21.15.5:0/361605124 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700003bd0 con 0x555cfed0c370 2024-09-19T06:28:17.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.287+0000 7fc712390740 1 -- 172.21.15.5:0/361605124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 msgr2=0x555cfed23e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:17.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.287+0000 7fc712390740 1 --2- 172.21.15.5:0/361605124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 0x555cfed23e20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc700006010 tx=0x7fc700005610 comp rx=0 tx=0).stop 2024-09-19T06:28:17.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.287+0000 7fc712390740 1 -- 172.21.15.5:0/361605124 shutdown_connections 2024-09-19T06:28:17.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.287+0000 7fc712390740 1 --2- 172.21.15.5:0/361605124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 0x555cfed23e20 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.287+0000 7fc712390740 1 --2- 172.21.15.5:0/361605124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed02b40 0x555cfed238e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.287+0000 7fc712390740 1 --2- 172.21.15.5:0/361605124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed18310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.287+0000 7fc712390740 1 -- 172.21.15.5:0/361605124 >> 172.21.15.5:0/361605124 conn(0x555cfecca310 msgr2=0x555cfed080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:17.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.287+0000 7fc712390740 1 -- 172.21.15.5:0/361605124 shutdown_connections 2024-09-19T06:28:17.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.287+0000 7fc712390740 1 -- 172.21.15.5:0/361605124 wait complete. 2024-09-19T06:28:17.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.287+0000 7fc712390740 1 -- start start 2024-09-19T06:28:17.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc712390740 1 Processor -- start 2024-09-19T06:28:17.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc712390740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed147a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc712390740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed02b40 0x555cfed14ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc712390740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 0x555cfed0fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc712390740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081acb40 con 0x555cfed015d0 2024-09-19T06:28:17.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc712390740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081ac470 con 0x555cfed0c370 2024-09-19T06:28:17.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc712390740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081a96d0 con 0x555cfed02b40 2024-09-19T06:28:17.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc712390740 10 client.0 _ll_register_callbacks cb 0x7fc704009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:17.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc70e554640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 0x555cfed0fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc70e554640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 0x555cfed0fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48088/0 (socket says 172.21.15.5:48088) 2024-09-19T06:28:17.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc70e554640 1 -- 172.21.15.5:0/4065863239 learned_addr learned my addr 172.21.15.5:0/4065863239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:17.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc70d552640 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed02b40 0x555cfed14ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc70dd53640 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed147a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc70e554640 1 -- 172.21.15.5:0/4065863239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed02b40 msgr2=0x555cfed14ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:17.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc70e554640 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed02b40 0x555cfed14ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc70e554640 1 -- 172.21.15.5:0/4065863239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 msgr2=0x555cfed147a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc70e554640 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed147a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.288+0000 7fc70e554640 1 -- 172.21.15.5:0/4065863239 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cfed10400 con 0x555cfed0c370 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.289+0000 7fc70e554640 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 0x555cfed0fbd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc700038000 tx=0x7fc700009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4000d720ee9055f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.289+0000 7fc6f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.289+0000 7fc6f17fa640 1 -- 172.21.15.5:0/4065863239 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700005ab0 con 0x555cfed0c370 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.289+0000 7fc712390740 1 -- 172.21.15.5:0/4065863239 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70001c040 con 0x555cfed0c370 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.289+0000 7fc6f17fa640 1 -- 172.21.15.5:0/4065863239 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fc700003970 con 0x555cfed0c370 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.289+0000 7fc6f17fa640 1 -- 172.21.15.5:0/4065863239 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc700017630 con 0x555cfed0c370 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.289+0000 7fc712390740 10 client.30863 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.289+0000 7fc712390740 1 -- 172.21.15.5:0/4065863239 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x555cfed86490 con 0x555cfed0c370 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.289+0000 7fc6f17fa640 1 -- 172.21.15.5:0/4065863239 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc70001c040 con 0x555cfed0c370 2024-09-19T06:28:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.290+0000 7fc6f17fa640 1 -- 172.21.15.5:0/4065863239 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fc70000b070 con 0x555cfed0c370 2024-09-19T06:28:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.290+0000 7fc712390740 10 client.30863 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.290+0000 7fc712390740 10 client.30863 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:28:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.290+0000 7fc712390740 20 client.30863 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.290+0000 7fc712390740 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555cfed1bf50 0x555cfed88920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.290+0000 7fc712390740 4 client.30863 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:28:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.290+0000 7fc712390740 1 -- 172.21.15.5:0/4065863239 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555cfec6ed70 con 0x555cfed1bf50 2024-09-19T06:28:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.290+0000 7fc70dd53640 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555cfed1bf50 0x555cfed88920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:17.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.291+0000 7fc70dd53640 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555cfed1bf50 0x555cfed88920 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=12964d2d48f0ec5f server_cookie=dacde50d9bbdd0fa in_seq=0 out_seq=0 2024-09-19T06:28:17.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.291+0000 7fc6f17fa640 10 client.30863 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:28:17.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.291+0000 7fc6f17fa640 1 -- 172.21.15.5:0/4065863239 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cfec6ed70 con 0x555cfed1bf50 2024-09-19T06:28:17.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.291+0000 7fc6f17fa640 10 client.30863 handle_command_reply: tid=0 2024-09-19T06:28:17.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc712390740 1 client.30863 shutdown 2024-09-19T06:28:17.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc712390740 1 -- 172.21.15.5:0/4065863239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 msgr2=0x555cfed0fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:17.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc712390740 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 0x555cfed0fbd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc700038000 tx=0x7fc700009900 comp rx=0 tx=0).stop 2024-09-19T06:28:17.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc712390740 1 -- 172.21.15.5:0/4065863239 shutdown_connections 2024-09-19T06:28:17.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc712390740 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x555cfed1bf50 0x555cfed88920 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc712390740 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cfed0c370 0x555cfed0fbd0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc712390740 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cfed02b40 0x555cfed14ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc712390740 1 --2- 172.21.15.5:0/4065863239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cfed015d0 0x555cfed147a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc712390740 1 -- 172.21.15.5:0/4065863239 >> 172.21.15.5:0/4065863239 conn(0x555cfecca310 msgr2=0x555cfed0a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:17.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc6f17fa640 0 client.30863 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:28:17.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc712390740 1 -- 172.21.15.5:0/4065863239 shutdown_connections 2024-09-19T06:28:17.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc712390740 1 -- 172.21.15.5:0/4065863239 wait complete. 2024-09-19T06:28:17.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.292+0000 7fc712390740 20 client.30863 trim_cache size 0 max 16384 2024-09-19T06:28:17.300 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:28:17.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.293+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1440030133 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc6f40a3430 msgr2=0x7fc6f40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:17.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.293+0000 7fc70ffb7640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc6f40a3430 0x7fc6f40a58f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x555cfecfeac0 tx=0x7fc704005f90 comp rx=0 tx=0).stop 2024-09-19T06:28:17.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.293+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1440030133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc708101430 msgr2=0x7fc708197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:17.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.293+0000 7fc70ffb7640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc708101430 0x7fc708197820 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009980 tx=0x7fc6f8030df0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.293+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1440030133 shutdown_connections 2024-09-19T06:28:17.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.293+0000 7fc70ffb7640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc708106a90 0x7fc70819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.293+0000 7fc70ffb7640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fc6f40a3430 0x7fc6f40a58f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.293+0000 7fc70ffb7640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc708101430 0x7fc708197820 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.293+0000 7fc70ffb7640 1 --2- 172.21.15.5:0/1440030133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708100a90 0x7fc7081972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:17.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.293+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1440030133 >> 172.21.15.5:0/1440030133 conn(0x7fc7080fc7d0 msgr2=0x7fc7080fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:17.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.294+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1440030133 shutdown_connections 2024-09-19T06:28:17.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:17.294+0000 7fc70ffb7640 1 -- 172.21.15.5:0/1440030133 wait complete. 2024-09-19T06:28:17.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 112306 8.52 MB/sec execute 581 sec latency 139.378 ms 2024-09-19T06:28:18.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:28:18.507 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:28:18.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 112465 8.52 MB/sec execute 582 sec latency 66.833 ms 2024-09-19T06:28:18.829 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:18 smithi005 ceph-mon[30955]: pgmap v790: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.6 MiB/s wr, 308 op/s 2024-09-19T06:28:18.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:18 smithi049 ceph-mon[34640]: pgmap v790: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.6 MiB/s wr, 308 op/s 2024-09-19T06:28:19.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:18 smithi157 ceph-mon[35346]: pgmap v790: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.6 MiB/s wr, 308 op/s 2024-09-19T06:28:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.073+0000 7fbb99b17640 1 -- 172.21.15.5:0/2103434673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb94101de0 msgr2=0x7fbb941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.073+0000 7fbb99b17640 1 --2- 172.21.15.5:0/2103434673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb94101de0 0x7fbb941021c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009900 tx=0x7fbb8402a3f0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.073+0000 7fbb99b17640 1 -- 172.21.15.5:0/2103434673 shutdown_connections 2024-09-19T06:28:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.073+0000 7fbb99b17640 1 --2- 172.21.15.5:0/2103434673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb94106f10 0x7fbb9410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.073+0000 7fbb99b17640 1 --2- 172.21.15.5:0/2103434673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb94102790 0x7fbb941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.073+0000 7fbb99b17640 1 --2- 172.21.15.5:0/2103434673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb94101de0 0x7fbb941021c0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.073+0000 7fbb99b17640 1 -- 172.21.15.5:0/2103434673 >> 172.21.15.5:0/2103434673 conn(0x7fbb940fd8e0 msgr2=0x7fbb940ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.074+0000 7fbb99b17640 1 -- 172.21.15.5:0/2103434673 shutdown_connections 2024-09-19T06:28:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.074+0000 7fbb99b17640 1 -- 172.21.15.5:0/2103434673 wait complete. 2024-09-19T06:28:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.075+0000 7fbb99b17640 1 Processor -- start 2024-09-19T06:28:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.075+0000 7fbb99b17640 1 -- start start 2024-09-19T06:28:19.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.076+0000 7fbb99b17640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb94101de0 0x7fbb94195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.076+0000 7fbb99b17640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb94102790 0x7fbb941964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.077+0000 7fbb99b17640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb94106f10 0x7fbb9419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.077+0000 7fbb92ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb94102790 0x7fbb941964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.077+0000 7fbb92ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb94102790 0x7fbb941964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45604/0 (socket says 172.21.15.5:45604) 2024-09-19T06:28:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.077+0000 7fbb92ffd640 1 -- 172.21.15.5:0/3310460702 learned_addr learned my addr 172.21.15.5:0/3310460702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.077+0000 7fbb93fff640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb94106f10 0x7fbb9419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.078+0000 7fbb99b17640 1 -- 172.21.15.5:0/3310460702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9410fc60 con 0x7fbb94101de0 2024-09-19T06:28:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.078+0000 7fbb99b17640 1 -- 172.21.15.5:0/3310460702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9410fae0 con 0x7fbb94106f10 2024-09-19T06:28:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.078+0000 7fbb937fe640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb94101de0 0x7fbb94195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.078+0000 7fbb99b17640 1 -- 172.21.15.5:0/3310460702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9410fde0 con 0x7fbb94102790 2024-09-19T06:28:19.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.079+0000 7fbb937fe640 1 -- 172.21.15.5:0/3310460702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb94102790 msgr2=0x7fbb941964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:19.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.079+0000 7fbb937fe640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb94102790 0x7fbb941964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.079+0000 7fbb937fe640 1 -- 172.21.15.5:0/3310460702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb94106f10 msgr2=0x7fbb9419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:19.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.079+0000 7fbb937fe640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb94106f10 0x7fbb9419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.079+0000 7fbb937fe640 1 -- 172.21.15.5:0/3310460702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9419af30 con 0x7fbb94101de0 2024-09-19T06:28:19.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.080+0000 7fbb937fe640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb94101de0 0x7fbb94195f80 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fbb8402a910 tx=0x7fbb84030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:19.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.080+0000 7fbb90ff9640 1 -- 172.21.15.5:0/3310460702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8403f070 con 0x7fbb94101de0 2024-09-19T06:28:19.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.080+0000 7fbb99b17640 1 -- 172.21.15.5:0/3310460702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9419b1c0 con 0x7fbb94101de0 2024-09-19T06:28:19.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.080+0000 7fbb99b17640 1 -- 172.21.15.5:0/3310460702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941a2b00 con 0x7fbb94101de0 2024-09-19T06:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.081+0000 7fbb90ff9640 1 -- 172.21.15.5:0/3310460702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb84004460 con 0x7fbb94101de0 2024-09-19T06:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.081+0000 7fbb90ff9640 1 -- 172.21.15.5:0/3310460702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84049060 con 0x7fbb94101de0 2024-09-19T06:28:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.084+0000 7fbb9bef0740 1 Processor -- start 2024-09-19T06:28:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.084+0000 7fbb9bef0740 1 -- start start 2024-09-19T06:28:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb9bef0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab9b40 0x561a9eab8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb9bef0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab85d0 0x561a9eab8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb9bef0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eac5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb9bef0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a9ea1e2f0 con 0x561a9eac3370 2024-09-19T06:28:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb9bef0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561a9e9c5cf0 con 0x561a9eab85d0 2024-09-19T06:28:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb9bef0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561a9ea1a2b0 con 0x561a9eab9b40 2024-09-19T06:28:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb90ff9640 1 -- 172.21.15.5:0/3310460702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbb84042070 con 0x7fbb94101de0 2024-09-19T06:28:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab9b40 0x561a9eab8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab9b40 0x561a9eab8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45614/0 (socket says 172.21.15.5:45614) 2024-09-19T06:28:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb93fff640 1 -- 172.21.15.5:0/1947017067 learned_addr learned my addr 172.21.15.5:0/1947017067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/1947017067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eac5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb937fe640 1 --2- 172.21.15.5:0/1947017067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab85d0 0x561a9eab8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb92ffd640 1 -- 172.21.15.5:0/1947017067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab9b40 msgr2=0x561a9eab8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/1947017067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab9b40 0x561a9eab8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb92ffd640 1 -- 172.21.15.5:0/1947017067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab85d0 msgr2=0x561a9eab8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/1947017067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab85d0 0x561a9eab8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.085+0000 7fbb92ffd640 1 -- 172.21.15.5:0/1947017067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a9e9aa3a0 con 0x561a9eac3370 2024-09-19T06:28:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.086+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/1947017067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eac5760 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009c00 tx=0x7fbb880117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3c501edc7411b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.086+0000 7fbb79ffb640 1 -- 172.21.15.5:0/1947017067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88005020 con 0x561a9eac3370 2024-09-19T06:28:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.086+0000 7fbb79ffb640 1 -- 172.21.15.5:0/1947017067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb880047a0 con 0x561a9eac3370 2024-09-19T06:28:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.086+0000 7fbb79ffb640 1 -- 172.21.15.5:0/1947017067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb880126b0 con 0x561a9eac3370 2024-09-19T06:28:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.086+0000 7fbb90ff9640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbb7c0a3430 0x7fbb7c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.087+0000 7fbb93fff640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbb7c0a3430 0x7fbb7c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.087+0000 7fbb90ff9640 1 -- 172.21.15.5:0/3310460702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fbb840f05e0 con 0x7fbb94101de0 2024-09-19T06:28:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.087+0000 7fbb9bef0740 1 -- 172.21.15.5:0/1947017067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 msgr2=0x561a9eac5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.087+0000 7fbb93fff640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbb7c0a3430 0x7fbb7c0a58f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0099a0 tx=0x7fbb8c011040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.087+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/1947017067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eac5760 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009c00 tx=0x7fbb880117b0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.087+0000 7fbb9bef0740 1 -- 172.21.15.5:0/1947017067 shutdown_connections 2024-09-19T06:28:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.087+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/1947017067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eac5760 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.087+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/1947017067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab85d0 0x561a9eab8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.087+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/1947017067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab9b40 0x561a9eab8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.088+0000 7fbb9bef0740 1 -- 172.21.15.5:0/1947017067 >> 172.21.15.5:0/1947017067 conn(0x561a9ea81310 msgr2=0x561a9eab4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.088+0000 7fbb9bef0740 1 -- 172.21.15.5:0/1947017067 shutdown_connections 2024-09-19T06:28:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.088+0000 7fbb9bef0740 1 -- 172.21.15.5:0/1947017067 wait complete. 2024-09-19T06:28:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.088+0000 7fbb9bef0740 1 -- start start 2024-09-19T06:28:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.088+0000 7fbb9bef0740 1 Processor -- start 2024-09-19T06:28:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.089+0000 7fbb9bef0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9eac2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.089+0000 7fbb9bef0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eab9b40 0x561a9eabffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.089+0000 7fbb9bef0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eac3370 0x561a9eac05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.089+0000 7fbb9bef0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a9eac6070 con 0x561a9eab9b40 2024-09-19T06:28:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.089+0000 7fbb9bef0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561a9e9d9060 con 0x561a9eac3370 2024-09-19T06:28:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.089+0000 7fbb9bef0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561a9eac61f0 con 0x561a9eab85d0 2024-09-19T06:28:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.089+0000 7fbb9bef0740 10 client.0 _ll_register_callbacks cb 0x561a9e72c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.089+0000 7fbb92ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eab9b40 0x561a9eabffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.089+0000 7fbb92ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eab9b40 0x561a9eabffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42014/0 (socket says 172.21.15.5:42014) 2024-09-19T06:28:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.089+0000 7fbb92ffd640 1 -- 172.21.15.5:0/2067914302 learned_addr learned my addr 172.21.15.5:0/2067914302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.089+0000 7fbb93fff640 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eac3370 0x561a9eac05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.089+0000 7fbb937fe640 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9eac2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb92ffd640 1 -- 172.21.15.5:0/2067914302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 msgr2=0x561a9eac2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9eac2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb92ffd640 1 -- 172.21.15.5:0/2067914302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eac3370 msgr2=0x561a9eac05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eac3370 0x561a9eac05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb92ffd640 1 -- 172.21.15.5:0/2067914302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a9e78f3c0 con 0x561a9eab9b40 2024-09-19T06:28:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb937fe640 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9eac2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:28:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eab9b40 0x561a9eabffe0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fbb88004730 tx=0x7fbb88004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bcb9ecafb2c1fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:28:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb72ffd640 1 -- 172.21.15.5:0/2067914302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8801f040 con 0x561a9eab9b40 2024-09-19T06:28:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb72ffd640 1 -- 172.21.15.5:0/2067914302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fbb88005200 con 0x561a9eab9b40 2024-09-19T06:28:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2067914302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb880047a0 con 0x561a9eab9b40 2024-09-19T06:28:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb72ffd640 1 -- 172.21.15.5:0/2067914302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fbb88032050 con 0x561a9eab9b40 2024-09-19T06:28:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb9bef0740 10 client.21516 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.090+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2067914302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x561a9eada2e0 con 0x561a9eab9b40 2024-09-19T06:28:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.091+0000 7fbb72ffd640 1 -- 172.21.15.5:0/2067914302 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb880047a0 con 0x561a9eab9b40 2024-09-19T06:28:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.091+0000 7fbb72ffd640 1 -- 172.21.15.5:0/2067914302 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fbb88024410 con 0x561a9eab9b40 2024-09-19T06:28:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.091+0000 7fbb9bef0740 10 client.21516 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.091+0000 7fbb9bef0740 10 client.21516 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:28:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.091+0000 7fbb9bef0740 20 client.21516 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.091+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561a9eadb6e0 0x561a9eaddad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.091+0000 7fbb9bef0740 4 client.21516 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:28:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.091+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2067914302 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a9ea2b860 con 0x561a9eadb6e0 2024-09-19T06:28:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.091+0000 7fbb937fe640 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561a9eadb6e0 0x561a9eaddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.092+0000 7fbb72ffd640 10 client.21516 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:28:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.092+0000 7fbb937fe640 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561a9eadb6e0 0x561a9eaddad0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e41b91a10b8137a8 server_cookie=9caaa20d9a426e4b in_seq=0 out_seq=0 2024-09-19T06:28:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.093+0000 7fbb72ffd640 1 -- 172.21.15.5:0/2067914302 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a9ea2b860 con 0x561a9eadb6e0 2024-09-19T06:28:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.093+0000 7fbb72ffd640 10 client.21516 handle_command_reply: tid=0 2024-09-19T06:28:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.093+0000 7fbb9bef0740 1 client.21516 shutdown 2024-09-19T06:28:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.094+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2067914302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eab9b40 msgr2=0x561a9eabffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.094+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eab9b40 0x561a9eabffe0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fbb88004730 tx=0x7fbb88004ea0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.094+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2067914302 shutdown_connections 2024-09-19T06:28:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.094+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561a9eadb6e0 0x561a9eaddad0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.094+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eac3370 0x561a9eac05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.094+0000 7fbb72ffd640 0 client.21516 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:28:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.094+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eab9b40 0x561a9eabffe0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.094+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/2067914302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9eac2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.094+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2067914302 >> 172.21.15.5:0/2067914302 conn(0x561a9ea81310 msgr2=0x561a9eaba960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.094+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2067914302 shutdown_connections 2024-09-19T06:28:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.095+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2067914302 wait complete. 2024-09-19T06:28:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.095+0000 7fbb9bef0740 20 client.21516 trim_cache size 0 max 16384 2024-09-19T06:28:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.115+0000 7fbb9bef0740 1 Processor -- start 2024-09-19T06:28:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb9bef0740 1 -- start start 2024-09-19T06:28:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb9bef0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9eac1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb9bef0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 0x561a9eabb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb9bef0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eabb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb9bef0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ab590 con 0x561a9eac3370 2024-09-19T06:28:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb9bef0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ad0d0 con 0x561a9eab9b40 2024-09-19T06:28:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb9bef0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941abc60 con 0x561a9eab85d0 2024-09-19T06:28:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eabb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb92ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 0x561a9eabb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb92ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 0x561a9eabb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48128/0 (socket says 172.21.15.5:48128) 2024-09-19T06:28:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb92ffd640 1 -- 172.21.15.5:0/2362978086 learned_addr learned my addr 172.21.15.5:0/2362978086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eabb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42028/0 (socket says 172.21.15.5:42028) 2024-09-19T06:28:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.116+0000 7fbb937fe640 1 --2- 172.21.15.5:0/2362978086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9eac1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.117+0000 7fbb92ffd640 1 -- 172.21.15.5:0/2362978086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 msgr2=0x561a9eac1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.117+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/2362978086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9eac1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.117+0000 7fbb92ffd640 1 -- 172.21.15.5:0/2362978086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 msgr2=0x561a9eabb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.117+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/2362978086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eabb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.117+0000 7fbb92ffd640 1 -- 172.21.15.5:0/2362978086 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a9eabe550 con 0x561a9eab9b40 2024-09-19T06:28:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.117+0000 7fbb93fff640 1 --2- 172.21.15.5:0/2362978086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eabb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.117+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/2362978086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 0x561a9eabb120 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbb88024860 tx=0x7fbb88004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b18e32b4170f8759 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.117+0000 7fbb797fa640 1 -- 172.21.15.5:0/2362978086 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88004760 con 0x561a9eab9b40 2024-09-19T06:28:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.117+0000 7fbb797fa640 1 -- 172.21.15.5:0/2362978086 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb88029070 con 0x561a9eab9b40 2024-09-19T06:28:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.117+0000 7fbb797fa640 1 -- 172.21.15.5:0/2362978086 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8802e450 con 0x561a9eab9b40 2024-09-19T06:28:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.118+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2362978086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 msgr2=0x561a9eabb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.118+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/2362978086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 0x561a9eabb120 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbb88024860 tx=0x7fbb88004dc0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.118+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2362978086 shutdown_connections 2024-09-19T06:28:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.118+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/2362978086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eabb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.118+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/2362978086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 0x561a9eabb120 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.118+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/2362978086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9eac1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.118+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2362978086 >> 172.21.15.5:0/2362978086 conn(0x561a9ea81310 msgr2=0x561a9eac1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.118+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2362978086 shutdown_connections 2024-09-19T06:28:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.118+0000 7fbb9bef0740 1 -- 172.21.15.5:0/2362978086 wait complete. 2024-09-19T06:28:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.119+0000 7fbb9bef0740 1 -- start start 2024-09-19T06:28:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.119+0000 7fbb9bef0740 1 Processor -- start 2024-09-19T06:28:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb9bef0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9ead45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb9bef0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 0x561a9eabceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb9bef0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eabd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb9bef0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941aca00 con 0x561a9eac3370 2024-09-19T06:28:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb9bef0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ac330 con 0x561a9eab9b40 2024-09-19T06:28:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb92ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 0x561a9eabceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb9bef0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941a9590 con 0x561a9eab85d0 2024-09-19T06:28:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb92ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 0x561a9eabceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48138/0 (socket says 172.21.15.5:48138) 2024-09-19T06:28:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb92ffd640 1 -- 172.21.15.5:0/4192331879 learned_addr learned my addr 172.21.15.5:0/4192331879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb9bef0740 10 client.0 _ll_register_callbacks cb 0x7fbb8c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb937fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9ead45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb93fff640 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eabd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb92ffd640 1 -- 172.21.15.5:0/4192331879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 msgr2=0x561a9ead45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9ead45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb92ffd640 1 -- 172.21.15.5:0/4192331879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 msgr2=0x561a9eabd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.120+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eabd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.121+0000 7fbb92ffd640 1 -- 172.21.15.5:0/4192331879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a9eabdcf0 con 0x561a9eab9b40 2024-09-19T06:28:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.121+0000 7fbb93fff640 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eabd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:28:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.121+0000 7fbb92ffd640 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 0x561a9eabceb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbb88024900 tx=0x7fbb88004ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=578ecfdf63ac2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.121+0000 7fbb72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:28:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.121+0000 7fbb72ffd640 1 -- 172.21.15.5:0/4192331879 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88005000 con 0x561a9eab9b40 2024-09-19T06:28:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.121+0000 7fbb72ffd640 1 -- 172.21.15.5:0/4192331879 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fbb880051a0 con 0x561a9eab9b40 2024-09-19T06:28:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.121+0000 7fbb9bef0740 1 -- 172.21.15.5:0/4192331879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb88029070 con 0x561a9eab9b40 2024-09-19T06:28:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.121+0000 7fbb72ffd640 1 -- 172.21.15.5:0/4192331879 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fbb88045050 con 0x561a9eab9b40 2024-09-19T06:28:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.121+0000 7fbb9bef0740 10 client.30881 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.121+0000 7fbb9bef0740 1 -- 172.21.15.5:0/4192331879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x561a9eb46210 con 0x561a9eab9b40 2024-09-19T06:28:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.122+0000 7fbb72ffd640 1 -- 172.21.15.5:0/4192331879 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb88029070 con 0x561a9eab9b40 2024-09-19T06:28:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.122+0000 7fbb72ffd640 1 -- 172.21.15.5:0/4192331879 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fbb88012450 con 0x561a9eab9b40 2024-09-19T06:28:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.122+0000 7fbb9bef0740 10 client.30881 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.122+0000 7fbb9bef0740 10 client.30881 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:28:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.122+0000 7fbb9bef0740 20 client.30881 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.122+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561a9eadb6e0 0x561a9eb48990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.122+0000 7fbb9bef0740 4 client.30881 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:28:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.122+0000 7fbb9bef0740 1 -- 172.21.15.5:0/4192331879 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a9ea25d70 con 0x561a9eadb6e0 2024-09-19T06:28:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.122+0000 7fbb937fe640 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561a9eadb6e0 0x561a9eb48990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.123+0000 7fbb72ffd640 10 client.30881 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:28:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.123+0000 7fbb937fe640 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561a9eadb6e0 0x561a9eb48990 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8df4167bf01250ff server_cookie=126a6be547f9f562 in_seq=0 out_seq=0 2024-09-19T06:28:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.123+0000 7fbb72ffd640 1 -- 172.21.15.5:0/4192331879 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a9ea25d70 con 0x561a9eadb6e0 2024-09-19T06:28:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.123+0000 7fbb72ffd640 10 client.30881 handle_command_reply: tid=0 2024-09-19T06:28:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.123+0000 7fbb9bef0740 1 client.30881 shutdown 2024-09-19T06:28:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.123+0000 7fbb9bef0740 1 -- 172.21.15.5:0/4192331879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 msgr2=0x561a9eabceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.123+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 0x561a9eabceb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbb88024900 tx=0x7fbb88004ea0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.123+0000 7fbb9bef0740 1 -- 172.21.15.5:0/4192331879 shutdown_connections 2024-09-19T06:28:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.123+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x561a9eadb6e0 0x561a9eb48990 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.123+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a9eac3370 0x561a9eabd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.123+0000 7fbb72ffd640 0 client.30881 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:28:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.123+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a9eab9b40 0x561a9eabceb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.124+0000 7fbb9bef0740 1 --2- 172.21.15.5:0/4192331879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9eab85d0 0x561a9ead45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.124+0000 7fbb9bef0740 1 -- 172.21.15.5:0/4192331879 >> 172.21.15.5:0/4192331879 conn(0x561a9ea81310 msgr2=0x561a9eac2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.124+0000 7fbb9bef0740 1 -- 172.21.15.5:0/4192331879 shutdown_connections 2024-09-19T06:28:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.124+0000 7fbb9bef0740 1 -- 172.21.15.5:0/4192331879 wait complete. 2024-09-19T06:28:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.124+0000 7fbb9bef0740 20 client.30881 trim_cache size 0 max 16384 2024-09-19T06:28:19.132 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:28:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.124+0000 7fbb99b17640 1 -- 172.21.15.5:0/3310460702 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbb7c0a3430 msgr2=0x7fbb7c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.124+0000 7fbb99b17640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbb7c0a3430 0x7fbb7c0a58f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0099a0 tx=0x7fbb8c011040 comp rx=0 tx=0).stop 2024-09-19T06:28:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.125+0000 7fbb99b17640 1 -- 172.21.15.5:0/3310460702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb94101de0 msgr2=0x7fbb94195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.125+0000 7fbb99b17640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb94101de0 0x7fbb94195f80 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fbb8402a910 tx=0x7fbb84030650 comp rx=0 tx=0).stop 2024-09-19T06:28:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.125+0000 7fbb99b17640 1 -- 172.21.15.5:0/3310460702 shutdown_connections 2024-09-19T06:28:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.125+0000 7fbb99b17640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb94106f10 0x7fbb9419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.125+0000 7fbb99b17640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbb7c0a3430 0x7fbb7c0a58f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.125+0000 7fbb99b17640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb94102790 0x7fbb941964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.125+0000 7fbb99b17640 1 --2- 172.21.15.5:0/3310460702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb94101de0 0x7fbb94195f80 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.125+0000 7fbb99b17640 1 -- 172.21.15.5:0/3310460702 >> 172.21.15.5:0/3310460702 conn(0x7fbb940fd8e0 msgr2=0x7fbb940fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.125+0000 7fbb99b17640 1 -- 172.21.15.5:0/3310460702 shutdown_connections 2024-09-19T06:28:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:19.125+0000 7fbb99b17640 1 -- 172.21.15.5:0/3310460702 wait complete. 2024-09-19T06:28:19.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 112628 8.52 MB/sec execute 583 sec latency 75.053 ms 2024-09-19T06:28:19.988 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:28:20.371 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:28:20.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 112772 8.52 MB/sec execute 584 sec latency 62.381 ms 2024-09-19T06:28:20.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:20 smithi005 ceph-mon[30955]: pgmap v791: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 295 op/s 2024-09-19T06:28:20.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:20 smithi049 ceph-mon[34640]: pgmap v791: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 295 op/s 2024-09-19T06:28:20.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.961+0000 7fac3a4db640 1 -- 172.21.15.5:0/3350453113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac34101de0 msgr2=0x7fac341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:20.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.961+0000 7fac3a4db640 1 --2- 172.21.15.5:0/3350453113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac34101de0 0x7fac341021c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fac24009860 tx=0x7fac2402a340 comp rx=0 tx=0).stop 2024-09-19T06:28:20.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.961+0000 7fac3a4db640 1 -- 172.21.15.5:0/3350453113 shutdown_connections 2024-09-19T06:28:20.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.961+0000 7fac3a4db640 1 --2- 172.21.15.5:0/3350453113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac34106f10 0x7fac3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.961+0000 7fac3a4db640 1 --2- 172.21.15.5:0/3350453113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac34102790 0x7fac341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.961+0000 7fac3a4db640 1 --2- 172.21.15.5:0/3350453113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac34101de0 0x7fac341021c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.961+0000 7fac3a4db640 1 -- 172.21.15.5:0/3350453113 >> 172.21.15.5:0/3350453113 conn(0x7fac340fd880 msgr2=0x7fac340ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:20.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.961+0000 7fac3a4db640 1 -- 172.21.15.5:0/3350453113 shutdown_connections 2024-09-19T06:28:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.961+0000 7fac3a4db640 1 -- 172.21.15.5:0/3350453113 wait complete. 2024-09-19T06:28:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.962+0000 7fac3a4db640 1 Processor -- start 2024-09-19T06:28:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.962+0000 7fac3a4db640 1 -- start start 2024-09-19T06:28:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.962+0000 7fac3a4db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac34101de0 0x7fac341983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.962+0000 7fac3a4db640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac34102790 0x7fac34198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.962+0000 7fac3a4db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac34106f10 0x7fac3419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac3a4db640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac34111de0 con 0x7fac34101de0 2024-09-19T06:28:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac3a4db640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac34111c60 con 0x7fac34106f10 2024-09-19T06:28:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac3a4db640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac34111f60 con 0x7fac34102790 2024-09-19T06:28:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac38a78640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac34106f10 0x7fac3419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac38a78640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac34106f10 0x7fac3419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48154/0 (socket says 172.21.15.5:48154) 2024-09-19T06:28:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac38a78640 1 -- 172.21.15.5:0/3465259592 learned_addr learned my addr 172.21.15.5:0/3465259592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac33fff640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac34101de0 0x7fac341983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac337fe640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac34102790 0x7fac34198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac38a78640 1 -- 172.21.15.5:0/3465259592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac34102790 msgr2=0x7fac34198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac38a78640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac34102790 0x7fac34198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac38a78640 1 -- 172.21.15.5:0/3465259592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac34101de0 msgr2=0x7fac341983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac38a78640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac34101de0 0x7fac341983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac38a78640 1 -- 172.21.15.5:0/3465259592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3419d370 con 0x7fac34106f10 2024-09-19T06:28:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.963+0000 7fac38a78640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac34106f10 0x7fac3419cc90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009fd0 tx=0x7fac2c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.964+0000 7fac317fa640 1 -- 172.21.15.5:0/3465259592 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c017070 con 0x7fac34106f10 2024-09-19T06:28:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.964+0000 7fac317fa640 1 -- 172.21.15.5:0/3465259592 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac2c003e20 con 0x7fac34106f10 2024-09-19T06:28:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.964+0000 7fac317fa640 1 -- 172.21.15.5:0/3465259592 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c010460 con 0x7fac34106f10 2024-09-19T06:28:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.964+0000 7fac3a4db640 1 -- 172.21.15.5:0/3465259592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3419d660 con 0x7fac34106f10 2024-09-19T06:28:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.964+0000 7fac3a4db640 1 -- 172.21.15.5:0/3465259592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac341a4f40 con 0x7fac34106f10 2024-09-19T06:28:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.968+0000 7fac3c8b4740 1 Processor -- start 2024-09-19T06:28:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.968+0000 7fac317fa640 1 -- 172.21.15.5:0/3465259592 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fac2c003970 con 0x7fac34106f10 2024-09-19T06:28:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.969+0000 7fac3c8b4740 1 -- start start 2024-09-19T06:28:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.969+0000 7fac3c8b4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72779b40 0x556a72778090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.969+0000 7fac3c8b4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a727785d0 0x556a72778a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.969+0000 7fac3c8b4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72787780 0x556a72789b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.969+0000 7fac3c8b4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556a726de2f0 con 0x556a72787780 2024-09-19T06:28:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.969+0000 7fac3c8b4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556a72685cf0 con 0x556a72779b40 2024-09-19T06:28:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.969+0000 7fac3c8b4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556a726da2b0 con 0x556a727785d0 2024-09-19T06:28:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.969+0000 7fac337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72779b40 0x556a72778090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.969+0000 7fac337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72779b40 0x556a72778090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48158/0 (socket says 172.21.15.5:48158) 2024-09-19T06:28:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.969+0000 7fac337fe640 1 -- 172.21.15.5:0/454880556 learned_addr learned my addr 172.21.15.5:0/454880556 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.969+0000 7fac33fff640 1 --2- 172.21.15.5:0/454880556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a727785d0 0x556a72778a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.970+0000 7fac337fe640 1 --2- 172.21.15.5:0/454880556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72787780 0x556a72789b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.970+0000 7fac337fe640 1 -- 172.21.15.5:0/454880556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a727785d0 msgr2=0x556a72778a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.970+0000 7fac337fe640 1 --2- 172.21.15.5:0/454880556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a727785d0 0x556a72778a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.970+0000 7fac337fe640 1 -- 172.21.15.5:0/454880556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72787780 msgr2=0x556a72789b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.970+0000 7fac337fe640 1 --2- 172.21.15.5:0/454880556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72787780 0x556a72789b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.970+0000 7fac337fe640 1 -- 172.21.15.5:0/454880556 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7266a3a0 con 0x556a72779b40 2024-09-19T06:28:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.970+0000 7fac317fa640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fac1c0a3430 0x7fac1c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.970+0000 7fac317fa640 1 -- 172.21.15.5:0/3465259592 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fac2c0cfa80 con 0x7fac34106f10 2024-09-19T06:28:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.970+0000 7fac33fff640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fac1c0a3430 0x7fac1c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.970+0000 7fac337fe640 1 --2- 172.21.15.5:0/454880556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72779b40 0x556a72778090 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fac28009f60 tx=0x7fac28009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c54632bcf244bc7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.970+0000 7fac1a7fc640 1 -- 172.21.15.5:0/454880556 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28004250 con 0x556a72779b40 2024-09-19T06:28:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.971+0000 7fac1a7fc640 1 -- 172.21.15.5:0/454880556 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac280043f0 con 0x556a72779b40 2024-09-19T06:28:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.971+0000 7fac1a7fc640 1 -- 172.21.15.5:0/454880556 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28010a60 con 0x556a72779b40 2024-09-19T06:28:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.972+0000 7fac3c8b4740 1 -- 172.21.15.5:0/454880556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72779b40 msgr2=0x556a72778090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.972+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/454880556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72779b40 0x556a72778090 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fac28009f60 tx=0x7fac28009790 comp rx=0 tx=0).stop 2024-09-19T06:28:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.972+0000 7fac33fff640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fac1c0a3430 0x7fac1c0a58f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fac2402a930 tx=0x7fac240058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.972+0000 7fac3c8b4740 1 -- 172.21.15.5:0/454880556 shutdown_connections 2024-09-19T06:28:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.972+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/454880556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72787780 0x556a72789b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.972+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/454880556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a727785d0 0x556a72778a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.972+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/454880556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72779b40 0x556a72778090 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.972+0000 7fac3c8b4740 1 -- 172.21.15.5:0/454880556 >> 172.21.15.5:0/454880556 conn(0x556a72741310 msgr2=0x556a72774c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.972+0000 7fac3c8b4740 1 -- 172.21.15.5:0/454880556 shutdown_connections 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.972+0000 7fac3c8b4740 1 -- 172.21.15.5:0/454880556 wait complete. 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.972+0000 7fac3c8b4740 1 -- start start 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.973+0000 7fac3c8b4740 1 Processor -- start 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.973+0000 7fac3c8b4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a727785d0 0x556a72787160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.973+0000 7fac3c8b4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72779b40 0x556a72784410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.973+0000 7fac3c8b4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72787780 0x556a727849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.973+0000 7fac337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72779b40 0x556a72784410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.973+0000 7fac3c8b4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556a7278a520 con 0x556a72779b40 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.973+0000 7fac33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a727785d0 0x556a72787160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.973+0000 7fac337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72779b40 0x556a72784410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42074/0 (socket says 172.21.15.5:42074) 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac337fe640 1 -- 172.21.15.5:0/1739074978 learned_addr learned my addr 172.21.15.5:0/1739074978 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1739074978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556a72699060 con 0x556a72787780 2024-09-19T06:28:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1739074978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556a7278a6a0 con 0x556a727785d0 2024-09-19T06:28:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac38a78640 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72787780 0x556a727849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac3c8b4740 10 client.0 _ll_register_callbacks cb 0x556a723ec8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac337fe640 1 -- 172.21.15.5:0/1739074978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a727785d0 msgr2=0x556a72787160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac337fe640 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a727785d0 0x556a72787160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac337fe640 1 -- 172.21.15.5:0/1739074978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72787780 msgr2=0x556a727849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac337fe640 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72787780 0x556a727849e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac337fe640 1 -- 172.21.15.5:0/1739074978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7244f3c0 con 0x556a72779b40 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac38a78640 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72787780 0x556a727849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac337fe640 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72779b40 0x556a72784410 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fac28005570 tx=0x7fac280055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cec72385ad795b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.974+0000 7fac137fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.975+0000 7fac137fe640 1 -- 172.21.15.5:0/1739074978 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2802e060 con 0x556a72779b40 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.975+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1739074978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac280043f0 con 0x556a72779b40 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.975+0000 7fac137fe640 1 -- 172.21.15.5:0/1739074978 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fac28003800 con 0x556a72779b40 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.975+0000 7fac137fe640 1 -- 172.21.15.5:0/1739074978 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fac28003a00 con 0x556a72779b40 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.975+0000 7fac3c8b4740 10 client.21534 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.975+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1739074978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x556a72795f60 con 0x556a72779b40 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.975+0000 7fac137fe640 1 -- 172.21.15.5:0/1739074978 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac280043f0 con 0x556a72779b40 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.976+0000 7fac137fe640 1 -- 172.21.15.5:0/1739074978 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fac2801c020 con 0x556a72779b40 2024-09-19T06:28:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.976+0000 7fac3c8b4740 10 client.21534 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.976+0000 7fac3c8b4740 10 client.21534 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:28:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.976+0000 7fac3c8b4740 20 client.21534 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.976+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556a72797670 0x556a72799ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.976+0000 7fac3c8b4740 4 client.21534 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:28:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.976+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1739074978 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a726eb860 con 0x556a72797670 2024-09-19T06:28:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.976+0000 7fac33fff640 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556a72797670 0x556a72799ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.977+0000 7fac33fff640 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556a72797670 0x556a72799ab0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f55f71e944e537d3 server_cookie=3735e9928c1c37c7 in_seq=0 out_seq=0 2024-09-19T06:28:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.977+0000 7fac137fe640 10 client.21534 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:28:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.978+0000 7fac137fe640 1 -- 172.21.15.5:0/1739074978 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a726eb860 con 0x556a72797670 2024-09-19T06:28:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.978+0000 7fac137fe640 10 client.21534 handle_command_reply: tid=0 2024-09-19T06:28:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.978+0000 7fac3c8b4740 1 client.21534 shutdown 2024-09-19T06:28:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.978+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1739074978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72779b40 msgr2=0x556a72784410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.978+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72779b40 0x556a72784410 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fac28005570 tx=0x7fac280055a0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.978+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1739074978 shutdown_connections 2024-09-19T06:28:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.978+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556a72797670 0x556a72799ab0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.978+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72787780 0x556a727849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.978+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72779b40 0x556a72784410 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.978+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/1739074978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a727785d0 0x556a72787160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.978+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1739074978 >> 172.21.15.5:0/1739074978 conn(0x556a72741310 msgr2=0x556a7277a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.978+0000 7fac137fe640 0 client.21534 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:28:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.980+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1739074978 shutdown_connections 2024-09-19T06:28:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.980+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1739074978 wait complete. 2024-09-19T06:28:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:20.981+0000 7fac3c8b4740 20 client.21534 trim_cache size 0 max 16384 2024-09-19T06:28:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac3c8b4740 1 Processor -- start 2024-09-19T06:28:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac3c8b4740 1 -- start start 2024-09-19T06:28:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac3c8b4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a72741310 0x556a72773c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac3c8b4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a727785d0 0x556a72781670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac3c8b4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72779b40 0x556a72781bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac3c8b4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac341ab6e0 con 0x556a72779b40 2024-09-19T06:28:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac3c8b4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac341ad220 con 0x556a727785d0 2024-09-19T06:28:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac3c8b4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac341abdb0 con 0x556a72741310 2024-09-19T06:28:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a727785d0 0x556a72781670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac38a78640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72779b40 0x556a72781bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a727785d0 0x556a72781670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48186/0 (socket says 172.21.15.5:48186) 2024-09-19T06:28:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac337fe640 1 -- 172.21.15.5:0/1634990494 learned_addr learned my addr 172.21.15.5:0/1634990494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.000+0000 7fac33fff640 1 --2- 172.21.15.5:0/1634990494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a72741310 0x556a72773c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.001+0000 7fac337fe640 1 -- 172.21.15.5:0/1634990494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a72741310 msgr2=0x556a72773c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.001+0000 7fac337fe640 1 --2- 172.21.15.5:0/1634990494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a72741310 0x556a72773c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.001+0000 7fac337fe640 1 -- 172.21.15.5:0/1634990494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72779b40 msgr2=0x556a72781bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.001+0000 7fac337fe640 1 --2- 172.21.15.5:0/1634990494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72779b40 0x556a72781bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.001+0000 7fac337fe640 1 -- 172.21.15.5:0/1634990494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a72775960 con 0x556a727785d0 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.001+0000 7fac337fe640 1 --2- 172.21.15.5:0/1634990494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a727785d0 0x556a72781670 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fac28010420 tx=0x7fac280106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4a3fd8ea229fd2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.001+0000 7fac19ffb640 1 -- 172.21.15.5:0/1634990494 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2802e070 con 0x556a727785d0 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.001+0000 7fac19ffb640 1 -- 172.21.15.5:0/1634990494 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac28005610 con 0x556a727785d0 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.001+0000 7fac19ffb640 1 -- 172.21.15.5:0/1634990494 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28009600 con 0x556a727785d0 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.002+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1634990494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a727785d0 msgr2=0x556a72781670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.002+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/1634990494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a727785d0 0x556a72781670 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fac28010420 tx=0x7fac280106b0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.002+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1634990494 shutdown_connections 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.002+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/1634990494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a72779b40 0x556a72781bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.002+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/1634990494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a727785d0 0x556a72781670 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.002+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/1634990494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a72741310 0x556a72773c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.002+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1634990494 >> 172.21.15.5:0/1634990494 conn(0x556a727781f0 msgr2=0x556a72773500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:21.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.002+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1634990494 shutdown_connections 2024-09-19T06:28:21.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.002+0000 7fac3c8b4740 1 -- 172.21.15.5:0/1634990494 wait complete. 2024-09-19T06:28:21.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.003+0000 7fac3c8b4740 1 -- start start 2024-09-19T06:28:21.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.003+0000 7fac3c8b4740 1 Processor -- start 2024-09-19T06:28:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.003+0000 7fac3c8b4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72741310 0x556a7279c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.003+0000 7fac3c8b4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a727785d0 0x556a7279ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac3c8b4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a72779b40 0x556a727fa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac3c8b4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac341acb50 con 0x556a727785d0 2024-09-19T06:28:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac3c8b4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac341ac480 con 0x556a72741310 2024-09-19T06:28:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac3c8b4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac341a96e0 con 0x556a72779b40 2024-09-19T06:28:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac33fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72741310 0x556a7279c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac33fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72741310 0x556a7279c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48200/0 (socket says 172.21.15.5:48200) 2024-09-19T06:28:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac33fff640 1 -- 172.21.15.5:0/183043183 learned_addr learned my addr 172.21.15.5:0/183043183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a727785d0 0x556a7279ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac3c8b4740 10 client.0 _ll_register_callbacks cb 0x7fac28005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac38a78640 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a72779b40 0x556a727fa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac33fff640 1 -- 172.21.15.5:0/183043183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a72779b40 msgr2=0x556a727fa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac33fff640 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a72779b40 0x556a727fa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac33fff640 1 -- 172.21.15.5:0/183043183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a727785d0 msgr2=0x556a7279ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac33fff640 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a727785d0 0x556a7279ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac33fff640 1 -- 172.21.15.5:0/183043183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7277b120 con 0x556a72741310 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac38a78640 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a72779b40 0x556a727fa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac33fff640 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72741310 0x556a7279c970 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fac24009860 tx=0x7fac2402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d47100dfbc204b27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.004+0000 7fac137fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.005+0000 7fac137fe640 1 -- 172.21.15.5:0/183043183 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2405a060 con 0x556a72741310 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.005+0000 7fac137fe640 1 -- 172.21.15.5:0/183043183 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fac240037f0 con 0x556a72741310 2024-09-19T06:28:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.005+0000 7fac3c8b4740 1 -- 172.21.15.5:0/183043183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556a726e5d70 con 0x556a72741310 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.005+0000 7fac137fe640 1 -- 172.21.15.5:0/183043183 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fac24037040 con 0x556a72741310 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.005+0000 7fac3c8b4740 10 client.30899 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.005+0000 7fac3c8b4740 1 -- 172.21.15.5:0/183043183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x556a72801b60 con 0x556a72741310 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.005+0000 7fac137fe640 1 -- 172.21.15.5:0/183043183 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556a726e5d70 con 0x556a72741310 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.005+0000 7fac137fe640 1 -- 172.21.15.5:0/183043183 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fac2404a040 con 0x556a72741310 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.006+0000 7fac3c8b4740 10 client.30899 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.006+0000 7fac3c8b4740 10 client.30899 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.006+0000 7fac3c8b4740 20 client.30899 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.006+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556a72787780 0x556a728042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.006+0000 7fac3c8b4740 4 client.30899 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.006+0000 7fac3c8b4740 1 -- 172.21.15.5:0/183043183 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a72662170 con 0x556a72787780 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.006+0000 7fac337fe640 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556a72787780 0x556a728042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac337fe640 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556a72787780 0x556a728042c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2509675c849cdf87 server_cookie=f7ca40d11fa57fb8 in_seq=0 out_seq=0 2024-09-19T06:28:21.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac137fe640 10 client.30899 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:28:21.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac137fe640 1 -- 172.21.15.5:0/183043183 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a72662170 con 0x556a72787780 2024-09-19T06:28:21.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac137fe640 10 client.30899 handle_command_reply: tid=0 2024-09-19T06:28:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac3c8b4740 1 client.30899 shutdown 2024-09-19T06:28:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac3c8b4740 1 -- 172.21.15.5:0/183043183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72741310 msgr2=0x556a7279c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72741310 0x556a7279c970 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fac24009860 tx=0x7fac2402a340 comp rx=0 tx=0).stop 2024-09-19T06:28:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac3c8b4740 1 -- 172.21.15.5:0/183043183 shutdown_connections 2024-09-19T06:28:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x556a72787780 0x556a728042c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a72779b40 0x556a727fa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a727785d0 0x556a7279ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac137fe640 0 client.30899 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:28:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac3c8b4740 1 --2- 172.21.15.5:0/183043183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a72741310 0x556a7279c970 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac3c8b4740 1 -- 172.21.15.5:0/183043183 >> 172.21.15.5:0/183043183 conn(0x556a727781f0 msgr2=0x556a7277a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac3c8b4740 1 -- 172.21.15.5:0/183043183 shutdown_connections 2024-09-19T06:28:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.007+0000 7fac3c8b4740 1 -- 172.21.15.5:0/183043183 wait complete. 2024-09-19T06:28:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.008+0000 7fac3c8b4740 20 client.30899 trim_cache size 0 max 16384 2024-09-19T06:28:21.013 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:28:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.008+0000 7fac3a4db640 1 -- 172.21.15.5:0/3465259592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fac1c0a3430 msgr2=0x7fac1c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.008+0000 7fac3a4db640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fac1c0a3430 0x7fac1c0a58f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fac2402a930 tx=0x7fac240058d0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.008+0000 7fac3a4db640 1 -- 172.21.15.5:0/3465259592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac34106f10 msgr2=0x7fac3419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.008+0000 7fac3a4db640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac34106f10 0x7fac3419cc90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009fd0 tx=0x7fac2c00b260 comp rx=0 tx=0).stop 2024-09-19T06:28:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.008+0000 7fac3a4db640 1 -- 172.21.15.5:0/3465259592 shutdown_connections 2024-09-19T06:28:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.008+0000 7fac3a4db640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac34106f10 0x7fac3419cc90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.008+0000 7fac3a4db640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fac1c0a3430 0x7fac1c0a58f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.008+0000 7fac3a4db640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac34102790 0x7fac34198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.008+0000 7fac3a4db640 1 --2- 172.21.15.5:0/3465259592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac34101de0 0x7fac341983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.008+0000 7fac3a4db640 1 -- 172.21.15.5:0/3465259592 >> 172.21.15.5:0/3465259592 conn(0x7fac340fd880 msgr2=0x7fac340fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.009+0000 7fac3a4db640 1 -- 172.21.15.5:0/3465259592 shutdown_connections 2024-09-19T06:28:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:21.009+0000 7fac3a4db640 1 -- 172.21.15.5:0/3465259592 wait complete. 2024-09-19T06:28:21.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:20 smithi157 ceph-mon[35346]: pgmap v791: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 295 op/s 2024-09-19T06:28:21.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 112940 8.52 MB/sec execute 585 sec latency 104.677 ms 2024-09-19T06:28:21.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:28:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:28:22.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113146 8.53 MB/sec execute 586 sec latency 134.639 ms 2024-09-19T06:28:22.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:22 smithi005 ceph-mon[30955]: pgmap v792: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-09-19T06:28:22.853 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:22 smithi157 ceph-mon[35346]: pgmap v792: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-09-19T06:28:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.871+0000 7fa932ffd640 1 -- 172.21.15.5:0/1318118142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924035020 con 0x7fa934101df0 2024-09-19T06:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.872+0000 7fa939cfb640 1 -- 172.21.15.5:0/1318118142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa934101df0 msgr2=0x7fa9341021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.872+0000 7fa939cfb640 1 --2- 172.21.15.5:0/1318118142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa934101df0 0x7fa9341021d0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fa924009900 tx=0x7fa92402a410 comp rx=0 tx=0).stop 2024-09-19T06:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.872+0000 7fa939cfb640 1 -- 172.21.15.5:0/1318118142 shutdown_connections 2024-09-19T06:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.872+0000 7fa939cfb640 1 --2- 172.21.15.5:0/1318118142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa934106f20 0x7fa93410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.872+0000 7fa939cfb640 1 --2- 172.21.15.5:0/1318118142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9341027a0 0x7fa9341067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.872+0000 7fa939cfb640 1 --2- 172.21.15.5:0/1318118142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa934101df0 0x7fa9341021d0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.872+0000 7fa939cfb640 1 -- 172.21.15.5:0/1318118142 >> 172.21.15.5:0/1318118142 conn(0x7fa9340fd8f0 msgr2=0x7fa9340ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.872+0000 7fa939cfb640 1 -- 172.21.15.5:0/1318118142 shutdown_connections 2024-09-19T06:28:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.873+0000 7fa939cfb640 1 -- 172.21.15.5:0/1318118142 wait complete. 2024-09-19T06:28:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.873+0000 7fa939cfb640 1 Processor -- start 2024-09-19T06:28:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.873+0000 7fa939cfb640 1 -- start start 2024-09-19T06:28:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa939cfb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa934101df0 0x7fa934198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa939cfb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9341027a0 0x7fa9341988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa939cfb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa934106f20 0x7fa93419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa939cfb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa934111e80 con 0x7fa9341027a0 2024-09-19T06:28:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa939cfb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa934111d00 con 0x7fa934106f20 2024-09-19T06:28:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa939cfb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa934112000 con 0x7fa934101df0 2024-09-19T06:28:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa92bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9341027a0 0x7fa9341988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa92bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9341027a0 0x7fa9341988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42106/0 (socket says 172.21.15.5:42106) 2024-09-19T06:28:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa92bfff640 1 -- 172.21.15.5:0/1811180468 learned_addr learned my addr 172.21.15.5:0/1811180468 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa933fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa934106f20 0x7fa93419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa9337fe640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa934101df0 0x7fa934198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa92bfff640 1 -- 172.21.15.5:0/1811180468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa934101df0 msgr2=0x7fa934198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa92bfff640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa934101df0 0x7fa934198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa92bfff640 1 -- 172.21.15.5:0/1811180468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa934106f20 msgr2=0x7fa93419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa92bfff640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa934106f20 0x7fa93419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa92bfff640 1 -- 172.21.15.5:0/1811180468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93419d340 con 0x7fa9341027a0 2024-09-19T06:28:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa933fff640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa934106f20 0x7fa93419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa9337fe640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa934101df0 0x7fa934198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:28:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.874+0000 7fa92bfff640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9341027a0 0x7fa9341988d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00e9f0 tx=0x7fa91c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:22.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.875+0000 7fa9317fa640 1 -- 172.21.15.5:0/1811180468 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00cd80 con 0x7fa9341027a0 2024-09-19T06:28:22.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.875+0000 7fa9317fa640 1 -- 172.21.15.5:0/1811180468 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa91c0054d0 con 0x7fa9341027a0 2024-09-19T06:28:22.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.875+0000 7fa9317fa640 1 -- 172.21.15.5:0/1811180468 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c010490 con 0x7fa9341027a0 2024-09-19T06:28:22.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.875+0000 7fa939cfb640 1 -- 172.21.15.5:0/1811180468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93419d630 con 0x7fa9341027a0 2024-09-19T06:28:22.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.875+0000 7fa939cfb640 1 -- 172.21.15.5:0/1811180468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa934074420 con 0x7fa9341027a0 2024-09-19T06:28:22.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.878+0000 7fa9317fa640 1 -- 172.21.15.5:0/1811180468 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa91c005020 con 0x7fa9341027a0 2024-09-19T06:28:22.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.878+0000 7fa93c0d4740 1 Processor -- start 2024-09-19T06:28:22.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.878+0000 7fa93c0d4740 1 -- start start 2024-09-19T06:28:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.878+0000 7fa9317fa640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa9200a34b0 0x7fa9200a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa93c0d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x56105088b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa93c0d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088b5d0 0x56105088ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa93c0d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x561050898760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa93c0d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5610507f12f0 con 0x56105088b5d0 2024-09-19T06:28:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa93c0d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561050798cf0 con 0x561050896370 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa93c0d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5610507ed2b0 con 0x56105088cb40 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa9317fa640 1 -- 172.21.15.5:0/1811180468 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fa91c0d0160 con 0x7fa9341027a0 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x56105088b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa92bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088b5d0 0x56105088ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x56105088b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45742/0 (socket says 172.21.15.5:45742) 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa9337fe640 1 -- 172.21.15.5:0/3609321931 learned_addr learned my addr 172.21.15.5:0/3609321931 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa933fff640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa9200a34b0 0x7fa9200a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa933fff640 1 --2- 172.21.15.5:0/3609321931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x561050898760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa92bfff640 1 -- 172.21.15.5:0/3609321931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 msgr2=0x56105088b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa92bfff640 1 --2- 172.21.15.5:0/3609321931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x56105088b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa92bfff640 1 -- 172.21.15.5:0/3609321931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 msgr2=0x561050898760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa92bfff640 1 --2- 172.21.15.5:0/3609321931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x561050898760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.879+0000 7fa92bfff640 1 -- 172.21.15.5:0/3609321931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105077d3a0 con 0x56105088b5d0 2024-09-19T06:28:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.880+0000 7fa92bfff640 1 --2- 172.21.15.5:0/3609321931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088b5d0 0x56105088ba10 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0cd830 tx=0x7fa91c09bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3065daba9c5f5e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.880+0000 7fa933fff640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa9200a34b0 0x7fa9200a5970 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x56105088c070 tx=0x7fa92c009f60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.880+0000 7fa929ffb640 1 -- 172.21.15.5:0/3609321931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0b5070 con 0x56105088b5d0 2024-09-19T06:28:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.880+0000 7fa929ffb640 1 -- 172.21.15.5:0/3609321931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa91c09c3c0 con 0x56105088b5d0 2024-09-19T06:28:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.880+0000 7fa929ffb640 1 -- 172.21.15.5:0/3609321931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0b5210 con 0x56105088b5d0 2024-09-19T06:28:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.881+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3609321931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088b5d0 msgr2=0x56105088ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.881+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/3609321931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088b5d0 0x56105088ba10 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0cd830 tx=0x7fa91c09bde0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.881+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3609321931 shutdown_connections 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.881+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/3609321931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x561050898760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.881+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/3609321931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088b5d0 0x56105088ba10 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.881+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/3609321931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x56105088b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.881+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3609321931 >> 172.21.15.5:0/3609321931 conn(0x561050854310 msgr2=0x561050887c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.881+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3609321931 shutdown_connections 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.881+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3609321931 wait complete. 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.881+0000 7fa93c0d4740 1 -- start start 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa93c0d4740 1 Processor -- start 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa93c0d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105088b5d0 0x561050895d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa93c0d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x561050893010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa93c0d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561050896370 0x561050893600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa93c0d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561050899070 con 0x561050896370 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa93c0d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5610507ac060 con 0x56105088b5d0 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa93c0d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5610508991f0 con 0x56105088cb40 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa93c0d4740 10 client.0 _ll_register_callbacks cb 0x5610504ff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa92bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x561050893010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa92bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x561050893010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45750/0 (socket says 172.21.15.5:45750) 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa92bfff640 1 -- 172.21.15.5:0/2769431918 learned_addr learned my addr 172.21.15.5:0/2769431918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa933fff640 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561050896370 0x561050893600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.882+0000 7fa9337fe640 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105088b5d0 0x561050895d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa92bfff640 1 -- 172.21.15.5:0/2769431918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105088b5d0 msgr2=0x561050895d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa92bfff640 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105088b5d0 0x561050895d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa92bfff640 1 -- 172.21.15.5:0/2769431918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561050896370 msgr2=0x561050893600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa92bfff640 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561050896370 0x561050893600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa92bfff640 1 -- 172.21.15.5:0/2769431918 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610505623c0 con 0x56105088cb40 2024-09-19T06:28:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa9337fe640 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105088b5d0 0x561050895d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:28:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa92bfff640 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x561050893010 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa91c09c0c0 tx=0x7fa91c00cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2154267eb206818d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:28:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa912ffd640 1 -- 172.21.15.5:0/2769431918 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0b9020 con 0x56105088cb40 2024-09-19T06:28:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2769431918 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa91c09c3c0 con 0x56105088cb40 2024-09-19T06:28:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa912ffd640 1 -- 172.21.15.5:0/2769431918 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fa91c0b51e0 con 0x56105088cb40 2024-09-19T06:28:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa912ffd640 1 -- 172.21.15.5:0/2769431918 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fa91c0b5040 con 0x56105088cb40 2024-09-19T06:28:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa93c0d4740 10 client.30841 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.883+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2769431918 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x561050899070 con 0x56105088cb40 2024-09-19T06:28:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.884+0000 7fa912ffd640 1 -- 172.21.15.5:0/2769431918 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa91c09c3c0 con 0x56105088cb40 2024-09-19T06:28:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.884+0000 7fa912ffd640 1 -- 172.21.15.5:0/2769431918 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fa91c098110 con 0x56105088cb40 2024-09-19T06:28:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.884+0000 7fa93c0d4740 10 client.30841 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.884+0000 7fa93c0d4740 10 client.30841 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:28:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.884+0000 7fa93c0d4740 20 client.30841 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.884+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5610508a6070 0x5610508a8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.884+0000 7fa93c0d4740 4 client.30841 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:28:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.884+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2769431918 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610507fe860 con 0x5610508a6070 2024-09-19T06:28:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.884+0000 7fa9337fe640 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5610508a6070 0x5610508a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.885+0000 7fa9337fe640 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5610508a6070 0x5610508a8480 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22f2561a1c1cd8a9 server_cookie=91417425fb5d6586 in_seq=0 out_seq=0 2024-09-19T06:28:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.885+0000 7fa912ffd640 10 client.30841 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:28:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa912ffd640 1 -- 172.21.15.5:0/2769431918 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610507fe860 con 0x5610508a6070 2024-09-19T06:28:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa912ffd640 10 client.30841 handle_command_reply: tid=0 2024-09-19T06:28:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa93c0d4740 1 client.30841 shutdown 2024-09-19T06:28:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2769431918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 msgr2=0x561050893010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x561050893010 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa91c09c0c0 tx=0x7fa91c00cf20 comp rx=0 tx=0).stop 2024-09-19T06:28:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2769431918 shutdown_connections 2024-09-19T06:28:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x5610508a6070 0x5610508a8480 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561050896370 0x561050893600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x561050893010 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/2769431918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56105088b5d0 0x561050895d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2769431918 >> 172.21.15.5:0/2769431918 conn(0x561050854310 msgr2=0x56105088d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa912ffd640 0 client.30841 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:28:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.886+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2769431918 shutdown_connections 2024-09-19T06:28:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.887+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2769431918 wait complete. 2024-09-19T06:28:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.887+0000 7fa93c0d4740 20 client.30841 trim_cache size 0 max 16384 2024-09-19T06:28:22.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:22 smithi049 ceph-mon[34640]: pgmap v792: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-09-19T06:28:22.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa93c0d4740 1 Processor -- start 2024-09-19T06:28:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa93c0d4740 1 -- start start 2024-09-19T06:28:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa93c0d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088b5d0 0x561050887e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa93c0d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088cb40 0x5610508a2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa93c0d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x56105088e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa93c0d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341ab720 con 0x56105088cb40 2024-09-19T06:28:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa93c0d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341ad260 con 0x561050896370 2024-09-19T06:28:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa93c0d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341abdf0 con 0x56105088b5d0 2024-09-19T06:28:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa933fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x56105088e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa933fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x56105088e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48266/0 (socket says 172.21.15.5:48266) 2024-09-19T06:28:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa933fff640 1 -- 172.21.15.5:0/2196609951 learned_addr learned my addr 172.21.15.5:0/2196609951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa92bfff640 1 --2- 172.21.15.5:0/2196609951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088cb40 0x5610508a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.911+0000 7fa9337fe640 1 --2- 172.21.15.5:0/2196609951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088b5d0 0x561050887e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.912+0000 7fa933fff640 1 -- 172.21.15.5:0/2196609951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088b5d0 msgr2=0x561050887e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:22.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.912+0000 7fa933fff640 1 --2- 172.21.15.5:0/2196609951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088b5d0 0x561050887e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.912+0000 7fa933fff640 1 -- 172.21.15.5:0/2196609951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088cb40 msgr2=0x5610508a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:22.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.912+0000 7fa933fff640 1 --2- 172.21.15.5:0/2196609951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088cb40 0x5610508a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.912+0000 7fa933fff640 1 -- 172.21.15.5:0/2196609951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561050891560 con 0x561050896370 2024-09-19T06:28:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.912+0000 7fa933fff640 1 --2- 172.21.15.5:0/2196609951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x56105088e120 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa92c01be60 tx=0x7fa92c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14978634fd7a4118 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.912+0000 7fa9297fa640 1 -- 172.21.15.5:0/2196609951 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c036070 con 0x561050896370 2024-09-19T06:28:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.912+0000 7fa9297fa640 1 -- 172.21.15.5:0/2196609951 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa92c01b730 con 0x561050896370 2024-09-19T06:28:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.912+0000 7fa9297fa640 1 -- 172.21.15.5:0/2196609951 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c031650 con 0x561050896370 2024-09-19T06:28:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.913+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2196609951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 msgr2=0x56105088e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.913+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/2196609951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x56105088e120 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa92c01be60 tx=0x7fa92c009500 comp rx=0 tx=0).stop 2024-09-19T06:28:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.914+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2196609951 shutdown_connections 2024-09-19T06:28:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.914+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/2196609951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x56105088e120 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.914+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/2196609951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088cb40 0x5610508a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.914+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/2196609951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088b5d0 0x561050887e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.914+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2196609951 >> 172.21.15.5:0/2196609951 conn(0x561050854310 msgr2=0x561050887700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.914+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2196609951 shutdown_connections 2024-09-19T06:28:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.914+0000 7fa93c0d4740 1 -- 172.21.15.5:0/2196609951 wait complete. 2024-09-19T06:28:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.914+0000 7fa93c0d4740 1 -- start start 2024-09-19T06:28:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.914+0000 7fa93c0d4740 1 Processor -- start 2024-09-19T06:28:22.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa93c0d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088b5d0 0x56105089e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa93c0d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x56105089eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa93c0d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x56105088f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa93c0d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341acb90 con 0x56105088b5d0 2024-09-19T06:28:22.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa93c0d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341ac4c0 con 0x561050896370 2024-09-19T06:28:22.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa93c0d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341a9720 con 0x56105088cb40 2024-09-19T06:28:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa93c0d4740 10 client.0 _ll_register_callbacks cb 0x7fa91c09bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa933fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x56105088f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa933fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x56105088f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:48276/0 (socket says 172.21.15.5:48276) 2024-09-19T06:28:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa933fff640 1 -- 172.21.15.5:0/3881810944 learned_addr learned my addr 172.21.15.5:0/3881810944 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa92bfff640 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x56105089eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa933fff640 1 -- 172.21.15.5:0/3881810944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 msgr2=0x56105089eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa933fff640 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x56105089eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa933fff640 1 -- 172.21.15.5:0/3881810944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088b5d0 msgr2=0x56105089e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa933fff640 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088b5d0 0x56105089e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.915+0000 7fa933fff640 1 -- 172.21.15.5:0/3881810944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105088ff30 con 0x561050896370 2024-09-19T06:28:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.916+0000 7fa92bfff640 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x56105089eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.916+0000 7fa933fff640 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x56105088f840 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009f30 tx=0x7fa92c01b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e5316fe68dfa1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.916+0000 7fa912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:28:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.916+0000 7fa912ffd640 1 -- 172.21.15.5:0/3881810944 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c03f060 con 0x561050896370 2024-09-19T06:28:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.916+0000 7fa912ffd640 1 -- 172.21.15.5:0/3881810944 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fa92c03a430 con 0x561050896370 2024-09-19T06:28:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.916+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3881810944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa92c01b730 con 0x561050896370 2024-09-19T06:28:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.916+0000 7fa912ffd640 1 -- 172.21.15.5:0/3881810944 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fa92c036050 con 0x561050896370 2024-09-19T06:28:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.916+0000 7fa93c0d4740 10 client.30917 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.916+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3881810944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x561050910720 con 0x561050896370 2024-09-19T06:28:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.917+0000 7fa912ffd640 1 -- 172.21.15.5:0/3881810944 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa92c004e50 con 0x561050896370 2024-09-19T06:28:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.917+0000 7fa912ffd640 1 -- 172.21.15.5:0/3881810944 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7fa92c030070 con 0x561050896370 2024-09-19T06:28:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.917+0000 7fa93c0d4740 10 client.30917 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:22.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.917+0000 7fa93c0d4740 10 client.30917 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:28:22.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.917+0000 7fa93c0d4740 20 client.30917 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:22.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.917+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x561050910d20 0x561050913140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.917+0000 7fa93c0d4740 4 client.30917 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:28:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.917+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3881810944 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa92c01b730 con 0x561050910d20 2024-09-19T06:28:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.917+0000 7fa9337fe640 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x561050910d20 0x561050913140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.918+0000 7fa9337fe640 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x561050910d20 0x561050913140 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=233265d5f6ae9943 server_cookie=61f36004da7dbb84 in_seq=0 out_seq=0 2024-09-19T06:28:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.918+0000 7fa912ffd640 10 client.30917 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:28:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.918+0000 7fa912ffd640 1 -- 172.21.15.5:0/3881810944 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa92c01b730 con 0x561050910d20 2024-09-19T06:28:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.918+0000 7fa912ffd640 10 client.30917 handle_command_reply: tid=0 2024-09-19T06:28:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.918+0000 7fa93c0d4740 1 client.30917 shutdown 2024-09-19T06:28:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.918+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3881810944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 msgr2=0x56105088f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.918+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x56105088f840 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009f30 tx=0x7fa92c01b9a0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.919+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3881810944 shutdown_connections 2024-09-19T06:28:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.919+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x561050910d20 0x561050913140 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.919+0000 7fa912ffd640 0 client.30917 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:28:22.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.919+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561050896370 0x56105088f840 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.919+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105088cb40 0x56105089eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.919+0000 7fa93c0d4740 1 --2- 172.21.15.5:0/3881810944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105088b5d0 0x56105089e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.919+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3881810944 >> 172.21.15.5:0/3881810944 conn(0x561050854310 msgr2=0x5610508864e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:22.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.919+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3881810944 shutdown_connections 2024-09-19T06:28:22.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.919+0000 7fa93c0d4740 1 -- 172.21.15.5:0/3881810944 wait complete. 2024-09-19T06:28:22.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.919+0000 7fa93c0d4740 20 client.30917 trim_cache size 0 max 16384 2024-09-19T06:28:22.923 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:28:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.920+0000 7fa939cfb640 1 -- 172.21.15.5:0/1811180468 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa9200a34b0 msgr2=0x7fa9200a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.920+0000 7fa939cfb640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa9200a34b0 0x7fa9200a5970 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x56105088c070 tx=0x7fa92c009f60 comp rx=0 tx=0).stop 2024-09-19T06:28:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.920+0000 7fa939cfb640 1 -- 172.21.15.5:0/1811180468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9341027a0 msgr2=0x7fa9341988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.920+0000 7fa939cfb640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9341027a0 0x7fa9341988d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00e9f0 tx=0x7fa91c00edf0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.920+0000 7fa939cfb640 1 -- 172.21.15.5:0/1811180468 shutdown_connections 2024-09-19T06:28:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.920+0000 7fa939cfb640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa934106f20 0x7fa93419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.920+0000 7fa939cfb640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9341027a0 0x7fa9341988d0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.920+0000 7fa939cfb640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa9200a34b0 0x7fa9200a5970 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.920+0000 7fa939cfb640 1 --2- 172.21.15.5:0/1811180468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa934101df0 0x7fa934198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.920+0000 7fa939cfb640 1 -- 172.21.15.5:0/1811180468 >> 172.21.15.5:0/1811180468 conn(0x7fa9340fd8f0 msgr2=0x7fa9340fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.920+0000 7fa939cfb640 1 -- 172.21.15.5:0/1811180468 shutdown_connections 2024-09-19T06:28:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:22.920+0000 7fa939cfb640 1 -- 172.21.15.5:0/1811180468 wait complete. 2024-09-19T06:28:23.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113306 8.53 MB/sec execute 587 sec latency 143.255 ms 2024-09-19T06:28:23.619 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:28:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:28:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.548+0000 7f2a3df17640 1 -- 172.21.15.5:0/2165180102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a38102ef0 msgr2=0x7f2a3810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.548+0000 7f2a3df17640 1 --2- 172.21.15.5:0/2165180102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a38102ef0 0x7f2a3810da40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009890 tx=0x7f2a3002a410 comp rx=0 tx=0).stop 2024-09-19T06:28:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.548+0000 7f2a3df17640 1 -- 172.21.15.5:0/2165180102 shutdown_connections 2024-09-19T06:28:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.548+0000 7f2a3df17640 1 --2- 172.21.15.5:0/2165180102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a38102ef0 0x7f2a3810da40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.548+0000 7f2a3df17640 1 --2- 172.21.15.5:0/2165180102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a38102570 0x7f2a381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.548+0000 7f2a3df17640 1 --2- 172.21.15.5:0/2165180102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a38100e20 0x7f2a38101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.548+0000 7f2a3df17640 1 -- 172.21.15.5:0/2165180102 >> 172.21.15.5:0/2165180102 conn(0x7f2a380fc7d0 msgr2=0x7f2a380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.548+0000 7f2a3df17640 1 -- 172.21.15.5:0/2165180102 shutdown_connections 2024-09-19T06:28:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.548+0000 7f2a3df17640 1 -- 172.21.15.5:0/2165180102 wait complete. 2024-09-19T06:28:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.549+0000 7f2a3df17640 1 Processor -- start 2024-09-19T06:28:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.549+0000 7f2a3df17640 1 -- start start 2024-09-19T06:28:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a3df17640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a38100e20 0x7f2a3819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a3df17640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a38102570 0x7f2a3819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a3df17640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a38102ef0 0x7f2a3819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a3df17640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a38111f30 con 0x7f2a38100e20 2024-09-19T06:28:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a3df17640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2a38111db0 con 0x7f2a38102570 2024-09-19T06:28:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a3df17640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a381120b0 con 0x7f2a38102ef0 2024-09-19T06:28:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a38100e20 0x7f2a3819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a37fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a38102ef0 0x7f2a3819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a38100e20 0x7f2a3819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42232/0 (socket says 172.21.15.5:42232) 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a377fe640 1 -- 172.21.15.5:0/2679855750 learned_addr learned my addr 172.21.15.5:0/2679855750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a36ffd640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a38102570 0x7f2a3819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a377fe640 1 -- 172.21.15.5:0/2679855750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a38102ef0 msgr2=0x7f2a3819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a377fe640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a38102ef0 0x7f2a3819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a377fe640 1 -- 172.21.15.5:0/2679855750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a38102570 msgr2=0x7f2a3819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a377fe640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a38102570 0x7f2a3819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.550+0000 7f2a377fe640 1 -- 172.21.15.5:0/2679855750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3819f640 con 0x7f2a38100e20 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.551+0000 7f2a377fe640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a38100e20 0x7f2a3819a690 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800efb0 tx=0x7f2a2800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.551+0000 7f2a34ff9640 1 -- 172.21.15.5:0/2679855750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2801b070 con 0x7f2a38100e20 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.551+0000 7f2a34ff9640 1 -- 172.21.15.5:0/2679855750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a2800ba20 con 0x7f2a38100e20 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.551+0000 7f2a3df17640 1 -- 172.21.15.5:0/2679855750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3819f930 con 0x7f2a38100e20 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.551+0000 7f2a3df17640 1 -- 172.21.15.5:0/2679855750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a381a7210 con 0x7f2a38100e20 2024-09-19T06:28:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.552+0000 7f2a34ff9640 1 -- 172.21.15.5:0/2679855750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28016b30 con 0x7f2a38100e20 2024-09-19T06:28:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.554+0000 7f2a34ff9640 1 -- 172.21.15.5:0/2679855750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2a2801e070 con 0x7f2a38100e20 2024-09-19T06:28:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.555+0000 7f2a402f0740 1 Processor -- start 2024-09-19T06:28:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.555+0000 7f2a402f0740 1 -- start start 2024-09-19T06:28:24.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.555+0000 7f2a402f0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335eb40 0x55749335d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.555+0000 7f2a402f0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335d5d0 0x55749335da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.555+0000 7f2a402f0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557493368370 0x55749336a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.555+0000 7f2a402f0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574932c32f0 con 0x55749335eb40 2024-09-19T06:28:24.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.555+0000 7f2a402f0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55749326acf0 con 0x55749335d5d0 2024-09-19T06:28:24.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.555+0000 7f2a402f0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5574932bf2b0 con 0x557493368370 2024-09-19T06:28:24.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.555+0000 7f2a37fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335eb40 0x55749335d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.555+0000 7f2a37fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335eb40 0x55749335d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42248/0 (socket says 172.21.15.5:42248) 2024-09-19T06:28:24.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.555+0000 7f2a37fff640 1 -- 172.21.15.5:0/127936852 learned_addr learned my addr 172.21.15.5:0/127936852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:24.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.555+0000 7f2a36ffd640 1 --2- 172.21.15.5:0/127936852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557493368370 0x55749336a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.556+0000 7f2a377fe640 1 --2- 172.21.15.5:0/127936852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335d5d0 0x55749335da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.556+0000 7f2a37fff640 1 -- 172.21.15.5:0/127936852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557493368370 msgr2=0x55749336a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.556+0000 7f2a37fff640 1 --2- 172.21.15.5:0/127936852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557493368370 0x55749336a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.556+0000 7f2a37fff640 1 -- 172.21.15.5:0/127936852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335d5d0 msgr2=0x55749335da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.556+0000 7f2a37fff640 1 --2- 172.21.15.5:0/127936852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335d5d0 0x55749335da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.556+0000 7f2a36ffd640 1 --2- 172.21.15.5:0/127936852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557493368370 0x55749336a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:28:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.556+0000 7f2a37fff640 1 -- 172.21.15.5:0/127936852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55749324f3a0 con 0x55749335eb40 2024-09-19T06:28:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.556+0000 7f2a34ff9640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2a200a3430 0x7f2a200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.556+0000 7f2a37fff640 1 --2- 172.21.15.5:0/127936852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335eb40 0x55749335d090 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2a300303d0 tx=0x7f2a30030400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ad39c790a52cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.557+0000 7f2a34ff9640 1 -- 172.21.15.5:0/2679855750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f2a280cfc20 con 0x7f2a38100e20 2024-09-19T06:28:24.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.557+0000 7f2a37fff640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2a200a3430 0x7f2a200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.557+0000 7f2a1dffb640 1 -- 172.21.15.5:0/127936852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30030970 con 0x55749335eb40 2024-09-19T06:28:24.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.557+0000 7f2a1dffb640 1 -- 172.21.15.5:0/127936852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a30041070 con 0x55749335eb40 2024-09-19T06:28:24.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.557+0000 7f2a1dffb640 1 -- 172.21.15.5:0/127936852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30047480 con 0x55749335eb40 2024-09-19T06:28:24.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.558+0000 7f2a402f0740 1 -- 172.21.15.5:0/127936852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335eb40 msgr2=0x55749335d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:24.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.558+0000 7f2a37fff640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2a200a3430 0x7f2a200a58f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2a30046000 tx=0x7f2a30047760 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:24.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.558+0000 7f2a402f0740 1 --2- 172.21.15.5:0/127936852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335eb40 0x55749335d090 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2a300303d0 tx=0x7f2a30030400 comp rx=0 tx=0).stop 2024-09-19T06:28:24.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.558+0000 7f2a402f0740 1 -- 172.21.15.5:0/127936852 shutdown_connections 2024-09-19T06:28:24.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.558+0000 7f2a402f0740 1 --2- 172.21.15.5:0/127936852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557493368370 0x55749336a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.558+0000 7f2a402f0740 1 --2- 172.21.15.5:0/127936852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335d5d0 0x55749335da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.558+0000 7f2a402f0740 1 --2- 172.21.15.5:0/127936852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335eb40 0x55749335d090 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.558+0000 7f2a402f0740 1 -- 172.21.15.5:0/127936852 >> 172.21.15.5:0/127936852 conn(0x557493326310 msgr2=0x557493359c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.558+0000 7f2a402f0740 1 -- 172.21.15.5:0/127936852 shutdown_connections 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.558+0000 7f2a402f0740 1 -- 172.21.15.5:0/127936852 wait complete. 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.559+0000 7f2a402f0740 1 -- start start 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.559+0000 7f2a402f0740 1 Processor -- start 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.560+0000 7f2a402f0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335d5d0 0x557493367d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.560+0000 7f2a402f0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55749335eb40 0x557493364fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.560+0000 7f2a402f0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557493368370 0x557493365560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.560+0000 7f2a402f0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55749336b0b0 con 0x557493368370 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.560+0000 7f2a377fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335d5d0 0x557493367d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.560+0000 7f2a402f0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55749327e060 con 0x55749335d5d0 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.560+0000 7f2a402f0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55749336b230 con 0x55749335eb40 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.560+0000 7f2a377fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335d5d0 0x557493367d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43326/0 (socket says 172.21.15.5:43326) 2024-09-19T06:28:24.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.560+0000 7f2a377fe640 1 -- 172.21.15.5:0/3168892124 learned_addr learned my addr 172.21.15.5:0/3168892124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.560+0000 7f2a37fff640 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557493368370 0x557493365560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.560+0000 7f2a36ffd640 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55749335eb40 0x557493364fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.560+0000 7f2a402f0740 10 client.0 _ll_register_callbacks cb 0x557492fd18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a377fe640 1 -- 172.21.15.5:0/3168892124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55749335eb40 msgr2=0x557493364fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a377fe640 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55749335eb40 0x557493364fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a377fe640 1 -- 172.21.15.5:0/3168892124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557493368370 msgr2=0x557493365560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a377fe640 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557493368370 0x557493365560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a377fe640 1 -- 172.21.15.5:0/3168892124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574930343c0 con 0x55749335d5d0 2024-09-19T06:28:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a377fe640 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335d5d0 0x557493367d10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2a2809bf20 tx=0x7f2a2809b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7504e167531a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:28:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a16ffd640 1 -- 172.21.15.5:0/3168892124 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2800bbd0 con 0x55749335d5d0 2024-09-19T06:28:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a16ffd640 1 -- 172.21.15.5:0/3168892124 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f2a2800b570 con 0x55749335d5d0 2024-09-19T06:28:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a402f0740 1 -- 172.21.15.5:0/3168892124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a30041070 con 0x55749335d5d0 2024-09-19T06:28:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a16ffd640 1 -- 172.21.15.5:0/3168892124 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f2a2800bd70 con 0x55749335d5d0 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a402f0740 10 client.30929 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.561+0000 7f2a402f0740 1 -- 172.21.15.5:0/3168892124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55749337f2c0 con 0x55749335d5d0 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.562+0000 7f2a16ffd640 1 -- 172.21.15.5:0/3168892124 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a2800b770 con 0x55749335d5d0 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.562+0000 7f2a16ffd640 1 -- 172.21.15.5:0/3168892124 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f2a280d5070 con 0x55749335d5d0 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.562+0000 7f2a402f0740 10 client.30929 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.562+0000 7f2a402f0740 10 client.30929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.562+0000 7f2a402f0740 20 client.30929 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.562+0000 7f2a402f0740 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557493380c90 0x557493383110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.562+0000 7f2a402f0740 4 client.30929 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.562+0000 7f2a402f0740 1 -- 172.21.15.5:0/3168892124 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2a30041070 con 0x557493380c90 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.562+0000 7f2a36ffd640 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557493380c90 0x557493383110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.563+0000 7f2a16ffd640 10 client.30929 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.563+0000 7f2a36ffd640 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557493380c90 0x557493383110 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1f99d9f8decc87 server_cookie=289bcaec75e05ad1 in_seq=0 out_seq=0 2024-09-19T06:28:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.563+0000 7f2a16ffd640 1 -- 172.21.15.5:0/3168892124 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2a30041070 con 0x557493380c90 2024-09-19T06:28:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.563+0000 7f2a16ffd640 10 client.30929 handle_command_reply: tid=0 2024-09-19T06:28:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a402f0740 1 client.30929 shutdown 2024-09-19T06:28:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a402f0740 1 -- 172.21.15.5:0/3168892124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335d5d0 msgr2=0x557493367d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a402f0740 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335d5d0 0x557493367d10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2a2809bf20 tx=0x7f2a2809b9b0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a402f0740 1 -- 172.21.15.5:0/3168892124 shutdown_connections 2024-09-19T06:28:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a402f0740 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x557493380c90 0x557493383110 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a402f0740 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557493368370 0x557493365560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a402f0740 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55749335eb40 0x557493364fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a402f0740 1 --2- 172.21.15.5:0/3168892124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335d5d0 0x557493367d10 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a402f0740 1 -- 172.21.15.5:0/3168892124 >> 172.21.15.5:0/3168892124 conn(0x557493326310 msgr2=0x55749335f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a16ffd640 0 client.30929 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a402f0740 1 -- 172.21.15.5:0/3168892124 shutdown_connections 2024-09-19T06:28:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a402f0740 1 -- 172.21.15.5:0/3168892124 wait complete. 2024-09-19T06:28:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.564+0000 7f2a402f0740 20 client.30929 trim_cache size 0 max 16384 2024-09-19T06:28:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a402f0740 1 Processor -- start 2024-09-19T06:28:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a402f0740 1 -- start start 2024-09-19T06:28:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a402f0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 0x55749337cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a402f0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335eb40 0x557493358bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a402f0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557493368370 0x557493388a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a402f0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a381ab670 con 0x55749335d5d0 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a402f0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2a381ad1b0 con 0x55749335eb40 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a402f0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a381abd40 con 0x557493368370 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 0x55749337cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 0x55749337cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42268/0 (socket says 172.21.15.5:42268) 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a377fe640 1 -- 172.21.15.5:0/1503714388 learned_addr learned my addr 172.21.15.5:0/1503714388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a37fff640 1 --2- 172.21.15.5:0/1503714388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557493368370 0x557493388a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.584+0000 7f2a36ffd640 1 --2- 172.21.15.5:0/1503714388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335eb40 0x557493358bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.585+0000 7f2a377fe640 1 -- 172.21.15.5:0/1503714388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557493368370 msgr2=0x557493388a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.585+0000 7f2a377fe640 1 --2- 172.21.15.5:0/1503714388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557493368370 0x557493388a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.585+0000 7f2a377fe640 1 -- 172.21.15.5:0/1503714388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335eb40 msgr2=0x557493358bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.585+0000 7f2a377fe640 1 --2- 172.21.15.5:0/1503714388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335eb40 0x557493358bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.585+0000 7f2a377fe640 1 -- 172.21.15.5:0/1503714388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557493359ed0 con 0x55749335d5d0 2024-09-19T06:28:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.585+0000 7f2a37fff640 1 --2- 172.21.15.5:0/1503714388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557493368370 0x557493388a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:28:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.585+0000 7f2a377fe640 1 --2- 172.21.15.5:0/1503714388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 0x55749337cbe0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f2a2809bc90 tx=0x7f2a2809ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92a6cf61c09226a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.585+0000 7f2a1d7fa640 1 -- 172.21.15.5:0/1503714388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a280d2070 con 0x55749335d5d0 2024-09-19T06:28:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.585+0000 7f2a1d7fa640 1 -- 172.21.15.5:0/1503714388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a280d3280 con 0x55749335d5d0 2024-09-19T06:28:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.585+0000 7f2a1d7fa640 1 -- 172.21.15.5:0/1503714388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2800e0b0 con 0x55749335d5d0 2024-09-19T06:28:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.586+0000 7f2a402f0740 1 -- 172.21.15.5:0/1503714388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 msgr2=0x55749337cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.586+0000 7f2a402f0740 1 --2- 172.21.15.5:0/1503714388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 0x55749337cbe0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f2a2809bc90 tx=0x7f2a2809ba20 comp rx=0 tx=0).stop 2024-09-19T06:28:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.586+0000 7f2a402f0740 1 -- 172.21.15.5:0/1503714388 shutdown_connections 2024-09-19T06:28:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.586+0000 7f2a402f0740 1 --2- 172.21.15.5:0/1503714388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557493368370 0x557493388a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.586+0000 7f2a402f0740 1 --2- 172.21.15.5:0/1503714388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55749335eb40 0x557493358bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.586+0000 7f2a402f0740 1 --2- 172.21.15.5:0/1503714388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 0x55749337cbe0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.586+0000 7f2a402f0740 1 -- 172.21.15.5:0/1503714388 >> 172.21.15.5:0/1503714388 conn(0x557493326310 msgr2=0x5574933584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.587+0000 7f2a402f0740 1 -- 172.21.15.5:0/1503714388 shutdown_connections 2024-09-19T06:28:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.587+0000 7f2a402f0740 1 -- 172.21.15.5:0/1503714388 wait complete. 2024-09-19T06:28:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.587+0000 7f2a402f0740 1 -- start start 2024-09-19T06:28:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.587+0000 7f2a402f0740 1 Processor -- start 2024-09-19T06:28:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a402f0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 0x557493361fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a402f0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55749335eb40 0x5574933624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a402f0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557493368370 0x557493362a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a402f0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a381acae0 con 0x55749335d5d0 2024-09-19T06:28:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a402f0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2a381ac410 con 0x557493368370 2024-09-19T06:28:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a402f0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a381a9670 con 0x55749335eb40 2024-09-19T06:28:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a402f0740 10 client.0 _ll_register_callbacks cb 0x7f2a28009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a36ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55749335eb40 0x5574933624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a36ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55749335eb40 0x5574933624e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55528/0 (socket says 172.21.15.5:55528) 2024-09-19T06:28:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a36ffd640 1 -- 172.21.15.5:0/2314974459 learned_addr learned my addr 172.21.15.5:0/2314974459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a37fff640 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557493368370 0x557493362a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a377fe640 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 0x557493361fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a37fff640 1 -- 172.21.15.5:0/2314974459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55749335eb40 msgr2=0x5574933624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a37fff640 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55749335eb40 0x5574933624e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a37fff640 1 -- 172.21.15.5:0/2314974459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 msgr2=0x557493361fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a37fff640 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 0x557493361fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a37fff640 1 -- 172.21.15.5:0/2314974459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574933ea9f0 con 0x557493368370 2024-09-19T06:28:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a36ffd640 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55749335eb40 0x5574933624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.588+0000 7f2a377fe640 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 0x557493361fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.589+0000 7f2a37fff640 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557493368370 0x557493362a20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2a300303a0 tx=0x7f2a30043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2161022a4d7481 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.589+0000 7f2a16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:28:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.589+0000 7f2a16ffd640 1 -- 172.21.15.5:0/2314974459 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30064060 con 0x557493368370 2024-09-19T06:28:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.589+0000 7f2a16ffd640 1 -- 172.21.15.5:0/2314974459 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f2a30047480 con 0x557493368370 2024-09-19T06:28:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.589+0000 7f2a402f0740 1 -- 172.21.15.5:0/2314974459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a280d3280 con 0x557493368370 2024-09-19T06:28:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.589+0000 7f2a16ffd640 1 -- 172.21.15.5:0/2314974459 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f2a30030970 con 0x557493368370 2024-09-19T06:28:24.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.589+0000 7f2a402f0740 10 client.30935 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:24.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.589+0000 7f2a402f0740 1 -- 172.21.15.5:0/2314974459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x5574933ebcb0 con 0x557493368370 2024-09-19T06:28:24.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.589+0000 7f2a16ffd640 1 -- 172.21.15.5:0/2314974459 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a280d3280 con 0x557493368370 2024-09-19T06:28:24.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.590+0000 7f2a16ffd640 1 -- 172.21.15.5:0/2314974459 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f2a3005e070 con 0x557493368370 2024-09-19T06:28:24.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.590+0000 7f2a402f0740 10 client.30935 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:24.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.590+0000 7f2a402f0740 10 client.30935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:24.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.590+0000 7f2a402f0740 20 client.30935 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:24.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.590+0000 7f2a402f0740 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5574933ebfe0 0x5574933ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:24.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.590+0000 7f2a402f0740 4 client.30935 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:28:24.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.590+0000 7f2a402f0740 1 -- 172.21.15.5:0/2314974459 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574932d0860 con 0x5574933ebfe0 2024-09-19T06:28:24.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.590+0000 7f2a377fe640 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5574933ebfe0 0x5574933ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.592+0000 7f2a377fe640 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5574933ebfe0 0x5574933ee3d0 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce613d476a9d4ff server_cookie=16dc331549cb1dd1 in_seq=0 out_seq=0 2024-09-19T06:28:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.592+0000 7f2a16ffd640 10 client.30935 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.592+0000 7f2a16ffd640 1 -- 172.21.15.5:0/2314974459 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574932d0860 con 0x5574933ebfe0 2024-09-19T06:28:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.592+0000 7f2a16ffd640 10 client.30935 handle_command_reply: tid=0 2024-09-19T06:28:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.592+0000 7f2a402f0740 1 client.30935 shutdown 2024-09-19T06:28:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.593+0000 7f2a402f0740 1 -- 172.21.15.5:0/2314974459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557493368370 msgr2=0x557493362a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.593+0000 7f2a402f0740 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557493368370 0x557493362a20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2a300303a0 tx=0x7f2a30043000 comp rx=0 tx=0).stop 2024-09-19T06:28:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.593+0000 7f2a402f0740 1 -- 172.21.15.5:0/2314974459 shutdown_connections 2024-09-19T06:28:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.594+0000 7f2a402f0740 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5574933ebfe0 0x5574933ee3d0 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.594+0000 7f2a402f0740 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557493368370 0x557493362a20 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.594+0000 7f2a16ffd640 0 client.30935 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.594+0000 7f2a402f0740 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55749335eb40 0x5574933624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.594+0000 7f2a402f0740 1 --2- 172.21.15.5:0/2314974459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749335d5d0 0x557493361fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.594+0000 7f2a402f0740 1 -- 172.21.15.5:0/2314974459 >> 172.21.15.5:0/2314974459 conn(0x557493326310 msgr2=0x55749335f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.594+0000 7f2a402f0740 1 -- 172.21.15.5:0/2314974459 shutdown_connections 2024-09-19T06:28:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.594+0000 7f2a402f0740 1 -- 172.21.15.5:0/2314974459 wait complete. 2024-09-19T06:28:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.594+0000 7f2a402f0740 20 client.30935 trim_cache size 0 max 16384 2024-09-19T06:28:24.600 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:28:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.595+0000 7f2a3df17640 1 -- 172.21.15.5:0/2679855750 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2a200a3430 msgr2=0x7f2a200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.595+0000 7f2a3df17640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2a200a3430 0x7f2a200a58f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2a30046000 tx=0x7f2a30047760 comp rx=0 tx=0).stop 2024-09-19T06:28:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.595+0000 7f2a3df17640 1 -- 172.21.15.5:0/2679855750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a38100e20 msgr2=0x7f2a3819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.595+0000 7f2a3df17640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a38100e20 0x7f2a3819a690 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800efb0 tx=0x7f2a2800c800 comp rx=0 tx=0).stop 2024-09-19T06:28:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.596+0000 7f2a3df17640 1 -- 172.21.15.5:0/2679855750 shutdown_connections 2024-09-19T06:28:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.596+0000 7f2a3df17640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2a200a3430 0x7f2a200a58f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.596+0000 7f2a3df17640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a38102ef0 0x7f2a3819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.596+0000 7f2a3df17640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a38102570 0x7f2a3819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.596+0000 7f2a3df17640 1 --2- 172.21.15.5:0/2679855750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a38100e20 0x7f2a3819a690 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.596+0000 7f2a3df17640 1 -- 172.21.15.5:0/2679855750 >> 172.21.15.5:0/2679855750 conn(0x7f2a380fc7d0 msgr2=0x7f2a38104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.596+0000 7f2a3df17640 1 -- 172.21.15.5:0/2679855750 shutdown_connections 2024-09-19T06:28:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:24.596+0000 7f2a3df17640 1 -- 172.21.15.5:0/2679855750 wait complete. 2024-09-19T06:28:24.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113449 8.53 MB/sec execute 588 sec latency 69.226 ms 2024-09-19T06:28:24.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:24 smithi005 ceph-mon[30955]: pgmap v793: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.5 MiB/s wr, 366 op/s 2024-09-19T06:28:24.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:24 smithi049 ceph-mon[34640]: pgmap v793: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.5 MiB/s wr, 366 op/s 2024-09-19T06:28:25.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:24 smithi157 ceph-mon[35346]: pgmap v793: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.5 MiB/s wr, 366 op/s 2024-09-19T06:28:25.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113600 8.54 MB/sec execute 589 sec latency 67.482 ms 2024-09-19T06:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.759+0000 7f8e4539f640 1 -- 172.21.15.5:0/2833552020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e401022c0 msgr2=0x7f8e4010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.759+0000 7f8e4539f640 1 --2- 172.21.15.5:0/2833552020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e401022c0 0x7f8e4010e7f0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009cd0 tx=0x7f8e3802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.759+0000 7f8e4539f640 1 -- 172.21.15.5:0/2833552020 shutdown_connections 2024-09-19T06:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.759+0000 7f8e4539f640 1 --2- 172.21.15.5:0/2833552020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e401022c0 0x7f8e4010e7f0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.759+0000 7f8e4539f640 1 --2- 172.21.15.5:0/2833552020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e40101920 0x7f8e40101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.759+0000 7f8e4539f640 1 --2- 172.21.15.5:0/2833552020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e40107930 0x7f8e40107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.759+0000 7f8e4539f640 1 -- 172.21.15.5:0/2833552020 >> 172.21.15.5:0/2833552020 conn(0x7f8e400fd600 msgr2=0x7f8e400ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.759+0000 7f8e4539f640 1 -- 172.21.15.5:0/2833552020 shutdown_connections 2024-09-19T06:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.759+0000 7f8e4539f640 1 -- 172.21.15.5:0/2833552020 wait complete. 2024-09-19T06:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.760+0000 7f8e4539f640 1 Processor -- start 2024-09-19T06:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.761+0000 7f8e4539f640 1 -- start start 2024-09-19T06:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.761+0000 7f8e4539f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e40101920 0x7f8e4019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.761+0000 7f8e4539f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e401022c0 0x7f8e4019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.761+0000 7f8e4539f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e40107930 0x7f8e4019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.761+0000 7f8e4539f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e40111c20 con 0x7f8e40101920 2024-09-19T06:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.761+0000 7f8e4539f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8e40111aa0 con 0x7f8e40107930 2024-09-19T06:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.761+0000 7f8e3f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e401022c0 0x7f8e4019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.761+0000 7f8e3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e40101920 0x7f8e4019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.761+0000 7f8e3f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e401022c0 0x7f8e4019a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55554/0 (socket says 172.21.15.5:55554) 2024-09-19T06:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.761+0000 7f8e3f7fe640 1 -- 172.21.15.5:0/3638712195 learned_addr learned my addr 172.21.15.5:0/3638712195 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.761+0000 7f8e4539f640 1 -- 172.21.15.5:0/3638712195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8e40111da0 con 0x7f8e401022c0 2024-09-19T06:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.762+0000 7f8e44b9e640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e40107930 0x7f8e4019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.762+0000 7f8e3f7fe640 1 -- 172.21.15.5:0/3638712195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e40107930 msgr2=0x7f8e4019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.762+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e40107930 0x7f8e4019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.762+0000 7f8e3f7fe640 1 -- 172.21.15.5:0/3638712195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e40101920 msgr2=0x7f8e4019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.762+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e40101920 0x7f8e4019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.762+0000 7f8e3f7fe640 1 -- 172.21.15.5:0/3638712195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4019f340 con 0x7f8e401022c0 2024-09-19T06:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.762+0000 7f8e3ffff640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e40101920 0x7f8e4019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.762+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e401022c0 0x7f8e4019a890 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e9f0 tx=0x7f8e3400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.762+0000 7f8e3d7fa640 1 -- 172.21.15.5:0/3638712195 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400cd80 con 0x7f8e401022c0 2024-09-19T06:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.762+0000 7f8e3d7fa640 1 -- 172.21.15.5:0/3638712195 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e340054d0 con 0x7f8e401022c0 2024-09-19T06:28:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.762+0000 7f8e4539f640 1 -- 172.21.15.5:0/3638712195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4019f630 con 0x7f8e401022c0 2024-09-19T06:28:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.762+0000 7f8e3d7fa640 1 -- 172.21.15.5:0/3638712195 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34010490 con 0x7f8e401022c0 2024-09-19T06:28:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.763+0000 7f8e4539f640 1 -- 172.21.15.5:0/3638712195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e401a6ec0 con 0x7f8e401022c0 2024-09-19T06:28:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.765+0000 7f8e3d7fa640 1 -- 172.21.15.5:0/3638712195 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8e34005020 con 0x7f8e401022c0 2024-09-19T06:28:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.766+0000 7f8e49474740 1 Processor -- start 2024-09-19T06:28:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.766+0000 7f8e49474740 1 -- start start 2024-09-19T06:28:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.766+0000 7f8e49474740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 0x5593476e50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.766+0000 7f8e49474740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 0x5593476ef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.766+0000 7f8e49474740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476efe80 0x5593476f2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.766+0000 7f8e49474740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593476492f0 con 0x5593476e55e0 2024-09-19T06:28:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.766+0000 7f8e49474740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5593475f1270 con 0x5593476efe80 2024-09-19T06:28:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.766+0000 7f8e49474740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5593476452b0 con 0x5593476e4cc0 2024-09-19T06:28:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e44b9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 0x5593476e50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e3f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476efe80 0x5593476f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 0x5593476ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e44b9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 0x5593476e50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55566/0 (socket says 172.21.15.5:55566) 2024-09-19T06:28:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e44b9e640 1 -- 172.21.15.5:0/469945437 learned_addr learned my addr 172.21.15.5:0/469945437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e3ffff640 1 --2- 172.21.15.5:0/469945437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 0x5593476ef940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42292/0 (socket says 172.21.15.5:42292) 2024-09-19T06:28:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e3d7fa640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8e280a3420 0x7f8e280a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e3f7fe640 1 -- 172.21.15.5:0/469945437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 msgr2=0x5593476e50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/469945437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 0x5593476e50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e3f7fe640 1 -- 172.21.15.5:0/469945437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 msgr2=0x5593476ef940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/469945437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 0x5593476ef940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e44b9e640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8e280a3420 0x7f8e280a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e3d7fa640 1 -- 172.21.15.5:0/3638712195 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f8e340d0100 con 0x7f8e401022c0 2024-09-19T06:28:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e3f7fe640 1 -- 172.21.15.5:0/469945437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593475f0340 con 0x5593476efe80 2024-09-19T06:28:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e3ffff640 1 --2- 172.21.15.5:0/469945437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 0x5593476ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.767+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/469945437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476efe80 0x5593476f2270 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8e340cd7d0 tx=0x7f8e3401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e952c1e64b287ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.768+0000 7f8e267fc640 1 -- 172.21.15.5:0/469945437 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e340cd030 con 0x5593476efe80 2024-09-19T06:28:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.768+0000 7f8e267fc640 1 -- 172.21.15.5:0/469945437 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e34005310 con 0x5593476efe80 2024-09-19T06:28:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.768+0000 7f8e267fc640 1 -- 172.21.15.5:0/469945437 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3409ca60 con 0x5593476efe80 2024-09-19T06:28:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.768+0000 7f8e49474740 1 -- 172.21.15.5:0/469945437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476efe80 msgr2=0x5593476f2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.768+0000 7f8e44b9e640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8e280a3420 0x7f8e280a58e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009430 tx=0x7f8e38009800 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.768+0000 7f8e49474740 1 --2- 172.21.15.5:0/469945437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476efe80 0x5593476f2270 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8e340cd7d0 tx=0x7f8e3401d000 comp rx=0 tx=0).stop 2024-09-19T06:28:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.769+0000 7f8e49474740 1 -- 172.21.15.5:0/469945437 shutdown_connections 2024-09-19T06:28:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.769+0000 7f8e49474740 1 --2- 172.21.15.5:0/469945437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476efe80 0x5593476f2270 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.769+0000 7f8e49474740 1 --2- 172.21.15.5:0/469945437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 0x5593476ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.769+0000 7f8e49474740 1 --2- 172.21.15.5:0/469945437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 0x5593476e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.769+0000 7f8e49474740 1 -- 172.21.15.5:0/469945437 >> 172.21.15.5:0/469945437 conn(0x5593476e1230 msgr2=0x5593476e1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.769+0000 7f8e49474740 1 -- 172.21.15.5:0/469945437 shutdown_connections 2024-09-19T06:28:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.769+0000 7f8e49474740 1 -- 172.21.15.5:0/469945437 wait complete. 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.769+0000 7f8e49474740 1 -- start start 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.769+0000 7f8e49474740 1 Processor -- start 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e49474740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 0x5593476ecd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e49474740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476e55e0 0x5593476ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e49474740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476efe80 0x5593476ed7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e49474740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593476f2a50 con 0x5593476efe80 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e49474740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5593475d2290 con 0x5593476e55e0 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e49474740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5593476f2bd0 con 0x5593476e4cc0 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e49474740 10 client.0 _ll_register_callbacks cb 0x559347603140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e3ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 0x5593476ecd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e3f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476e55e0 0x5593476ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e3ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 0x5593476ecd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55582/0 (socket says 172.21.15.5:55582) 2024-09-19T06:28:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e3ffff640 1 -- 172.21.15.5:0/1776527420 learned_addr learned my addr 172.21.15.5:0/1776527420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e44b9e640 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476efe80 0x5593476ed7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e3f7fe640 1 -- 172.21.15.5:0/1776527420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 msgr2=0x5593476ecd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 0x5593476ecd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e3f7fe640 1 -- 172.21.15.5:0/1776527420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476efe80 msgr2=0x5593476ed7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476efe80 0x5593476ed7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e3f7fe640 1 -- 172.21.15.5:0/1776527420 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55934735c210 con 0x5593476e55e0 2024-09-19T06:28:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e3ffff640 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 0x5593476ecd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e44b9e640 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476efe80 0x5593476ed7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476e55e0 0x5593476ed2b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x5593476e5c00 tx=0x7f8e34007e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4265a4143458f47c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:28:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.770+0000 7f8e1f7fe640 1 -- 172.21.15.5:0/1776527420 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e340cd040 con 0x5593476e55e0 2024-09-19T06:28:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e1f7fe640 1 -- 172.21.15.5:0/1776527420 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f8e340cda50 con 0x5593476e55e0 2024-09-19T06:28:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e49474740 1 -- 172.21.15.5:0/1776527420 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559347656a20 con 0x5593476e55e0 2024-09-19T06:28:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e1f7fe640 1 -- 172.21.15.5:0/1776527420 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f8e3409bcc0 con 0x5593476e55e0 2024-09-19T06:28:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e49474740 10 client.30947 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e49474740 1 -- 172.21.15.5:0/1776527420 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x559347428620 con 0x5593476e55e0 2024-09-19T06:28:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e1f7fe640 1 -- 172.21.15.5:0/1776527420 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559347656a20 con 0x5593476e55e0 2024-09-19T06:28:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e1f7fe640 1 -- 172.21.15.5:0/1776527420 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f8e3409c5c0 con 0x5593476e55e0 2024-09-19T06:28:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e49474740 10 client.30947 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e49474740 10 client.30947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e49474740 20 client.30947 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e49474740 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593476ffc60 0x559347702050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e49474740 4 client.30947 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:28:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.771+0000 7f8e49474740 1 -- 172.21.15.5:0/1776527420 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559347650d70 con 0x5593476ffc60 2024-09-19T06:28:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.772+0000 7f8e3ffff640 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593476ffc60 0x559347702050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.772+0000 7f8e3ffff640 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593476ffc60 0x559347702050 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66bc30f3ff74b428 server_cookie=28fa09a54b36e606 in_seq=0 out_seq=0 2024-09-19T06:28:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.772+0000 7f8e1f7fe640 10 client.30947 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e1f7fe640 1 -- 172.21.15.5:0/1776527420 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559347650d70 con 0x5593476ffc60 2024-09-19T06:28:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e1f7fe640 10 client.30947 handle_command_reply: tid=0 2024-09-19T06:28:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e49474740 1 client.30947 shutdown 2024-09-19T06:28:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e49474740 1 -- 172.21.15.5:0/1776527420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476e55e0 msgr2=0x5593476ed2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e49474740 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476e55e0 0x5593476ed2b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x5593476e5c00 tx=0x7f8e34007e90 comp rx=0 tx=0).stop 2024-09-19T06:28:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e49474740 1 -- 172.21.15.5:0/1776527420 shutdown_connections 2024-09-19T06:28:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e49474740 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593476ffc60 0x559347702050 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e1f7fe640 0 client.30947 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e49474740 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476efe80 0x5593476ed7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e49474740 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476e55e0 0x5593476ed2b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e49474740 1 --2- 172.21.15.5:0/1776527420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e4cc0 0x5593476ecd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e49474740 1 -- 172.21.15.5:0/1776527420 >> 172.21.15.5:0/1776527420 conn(0x5593476e1230 msgr2=0x5593476e6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e49474740 1 -- 172.21.15.5:0/1776527420 shutdown_connections 2024-09-19T06:28:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e49474740 1 -- 172.21.15.5:0/1776527420 wait complete. 2024-09-19T06:28:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.773+0000 7f8e49474740 20 client.30947 trim_cache size 0 max 16384 2024-09-19T06:28:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.798+0000 7f8e49474740 1 Processor -- start 2024-09-19T06:28:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.798+0000 7f8e49474740 1 -- start start 2024-09-19T06:28:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.798+0000 7f8e49474740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e4cc0 0x5593476ed620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.798+0000 7f8e49474740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e55e0 0x5593476e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.798+0000 7f8e49474740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476efe80 0x5593476ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e49474740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e401af760 con 0x5593476e4cc0 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e49474740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8e401b12a0 con 0x5593476efe80 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e49474740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8e401afe30 con 0x5593476e55e0 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e4cc0 0x5593476ed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e4cc0 0x5593476ed620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42310/0 (socket says 172.21.15.5:42310) 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e3ffff640 1 -- 172.21.15.5:0/142164431 learned_addr learned my addr 172.21.15.5:0/142164431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/142164431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e55e0 0x5593476e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e44b9e640 1 --2- 172.21.15.5:0/142164431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476efe80 0x5593476ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e3ffff640 1 -- 172.21.15.5:0/142164431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e55e0 msgr2=0x5593476e9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e3ffff640 1 --2- 172.21.15.5:0/142164431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e55e0 0x5593476e9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e3ffff640 1 -- 172.21.15.5:0/142164431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476efe80 msgr2=0x5593476ea4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e3ffff640 1 --2- 172.21.15.5:0/142164431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476efe80 0x5593476ea4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e3ffff640 1 -- 172.21.15.5:0/142164431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593476df7b0 con 0x5593476e4cc0 2024-09-19T06:28:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.799+0000 7f8e3ffff640 1 --2- 172.21.15.5:0/142164431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e4cc0 0x5593476ed620 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f8e30005d00 tx=0x7f8e30005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db004aac746518a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.800+0000 7f8e25ffb640 1 -- 172.21.15.5:0/142164431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30004030 con 0x5593476e4cc0 2024-09-19T06:28:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.800+0000 7f8e25ffb640 1 -- 172.21.15.5:0/142164431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e3000f8b0 con 0x5593476e4cc0 2024-09-19T06:28:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.800+0000 7f8e25ffb640 1 -- 172.21.15.5:0/142164431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30005520 con 0x5593476e4cc0 2024-09-19T06:28:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.801+0000 7f8e49474740 1 -- 172.21.15.5:0/142164431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e4cc0 msgr2=0x5593476ed620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.801+0000 7f8e49474740 1 --2- 172.21.15.5:0/142164431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e4cc0 0x5593476ed620 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f8e30005d00 tx=0x7f8e30005f90 comp rx=0 tx=0).stop 2024-09-19T06:28:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.801+0000 7f8e49474740 1 -- 172.21.15.5:0/142164431 shutdown_connections 2024-09-19T06:28:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.801+0000 7f8e49474740 1 --2- 172.21.15.5:0/142164431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476efe80 0x5593476ea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.801+0000 7f8e49474740 1 --2- 172.21.15.5:0/142164431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476e55e0 0x5593476e9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.801+0000 7f8e49474740 1 --2- 172.21.15.5:0/142164431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e4cc0 0x5593476ed620 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.801+0000 7f8e49474740 1 -- 172.21.15.5:0/142164431 >> 172.21.15.5:0/142164431 conn(0x5593476e1230 msgr2=0x5593476ebe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.801+0000 7f8e49474740 1 -- 172.21.15.5:0/142164431 shutdown_connections 2024-09-19T06:28:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.801+0000 7f8e49474740 1 -- 172.21.15.5:0/142164431 wait complete. 2024-09-19T06:28:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.801+0000 7f8e49474740 1 -- start start 2024-09-19T06:28:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.802+0000 7f8e49474740 1 Processor -- start 2024-09-19T06:28:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.802+0000 7f8e49474740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476e4cc0 0x559347705930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.802+0000 7f8e49474740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 0x559347701cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.802+0000 7f8e3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 0x559347701cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.802+0000 7f8e3ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476e4cc0 0x559347705930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.802+0000 7f8e3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 0x559347701cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42326/0 (socket says 172.21.15.5:42326) 2024-09-19T06:28:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.802+0000 7f8e3f7fe640 1 -- 172.21.15.5:0/4155001291 learned_addr learned my addr 172.21.15.5:0/4155001291 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.802+0000 7f8e49474740 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476efe80 0x559347702440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e49474740 1 -- 172.21.15.5:0/4155001291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e401b0bd0 con 0x5593476e55e0 2024-09-19T06:28:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e49474740 1 -- 172.21.15.5:0/4155001291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8e401b0500 con 0x5593476e4cc0 2024-09-19T06:28:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e49474740 1 -- 172.21.15.5:0/4155001291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8e401ad760 con 0x5593476efe80 2024-09-19T06:28:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e44b9e640 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476efe80 0x559347702440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e49474740 10 client.0 _ll_register_callbacks cb 0x7f8e30009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e3ffff640 1 -- 172.21.15.5:0/4155001291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476efe80 msgr2=0x559347702440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e3ffff640 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476efe80 0x559347702440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e3ffff640 1 -- 172.21.15.5:0/4155001291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 msgr2=0x559347701cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e3ffff640 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 0x559347701cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e3ffff640 1 -- 172.21.15.5:0/4155001291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559347768590 con 0x5593476e4cc0 2024-09-19T06:28:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 0x559347701cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e49474740 1 -- 172.21.15.5:0/4155001291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e3000f8b0 con 0x5593476e4cc0 2024-09-19T06:28:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e3ffff640 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476e4cc0 0x559347705930 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8e30005d00 tx=0x7f8e300127d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b59159e3dd066f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.803+0000 7f8e1f7fe640 10 client.30953 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:28:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.804+0000 7f8e1f7fe640 1 -- 172.21.15.5:0/4155001291 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e300052a0 con 0x5593476e4cc0 2024-09-19T06:28:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.804+0000 7f8e1f7fe640 1 -- 172.21.15.5:0/4155001291 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f8e30005440 con 0x5593476e4cc0 2024-09-19T06:28:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.804+0000 7f8e1f7fe640 1 -- 172.21.15.5:0/4155001291 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f8e30005680 con 0x5593476e4cc0 2024-09-19T06:28:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.804+0000 7f8e49474740 10 client.30953 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.804+0000 7f8e49474740 1 -- 172.21.15.5:0/4155001291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=62}) -- 0x559347769600 con 0x5593476e4cc0 2024-09-19T06:28:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.804+0000 7f8e1f7fe640 1 -- 172.21.15.5:0/4155001291 <== mon.2 v2:172.21.15.49:3300/0 4 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f8e3003d020 con 0x5593476e4cc0 2024-09-19T06:28:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.804+0000 7f8e49474740 10 client.30953 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.804+0000 7f8e49474740 10 client.30953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.804+0000 7f8e49474740 20 client.30953 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.804+0000 7f8e49474740 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593476ffc60 0x55934776bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.804+0000 7f8e49474740 4 client.30953 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:28:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.804+0000 7f8e49474740 1 -- 172.21.15.5:0/4155001291 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593475cd900 con 0x5593476ffc60 2024-09-19T06:28:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.805+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593476ffc60 0x55934776bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.805+0000 7f8e1f7fe640 10 client.30953 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.805+0000 7f8e3f7fe640 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593476ffc60 0x55934776bbf0 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bb66b79dbbadd58 server_cookie=ff0e09ba690d172b in_seq=0 out_seq=0 2024-09-19T06:28:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e1f7fe640 1 -- 172.21.15.5:0/4155001291 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5593475cd900 con 0x5593476ffc60 2024-09-19T06:28:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e1f7fe640 10 client.30953 handle_command_reply: tid=0 2024-09-19T06:28:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e49474740 1 client.30953 shutdown 2024-09-19T06:28:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e49474740 1 -- 172.21.15.5:0/4155001291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476e4cc0 msgr2=0x559347705930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e49474740 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476e4cc0 0x559347705930 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8e30005d00 tx=0x7f8e300127d0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e49474740 1 -- 172.21.15.5:0/4155001291 shutdown_connections 2024-09-19T06:28:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e49474740 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x5593476ffc60 0x55934776bbf0 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e49474740 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5593476efe80 0x559347702440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e49474740 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593476e55e0 0x559347701cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e49474740 1 --2- 172.21.15.5:0/4155001291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593476e4cc0 0x559347705930 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e49474740 1 -- 172.21.15.5:0/4155001291 >> 172.21.15.5:0/4155001291 conn(0x5593476e1230 msgr2=0x5593476fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e1f7fe640 0 client.30953 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e49474740 1 -- 172.21.15.5:0/4155001291 shutdown_connections 2024-09-19T06:28:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.902+0000 7f8e49474740 1 -- 172.21.15.5:0/4155001291 wait complete. 2024-09-19T06:28:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.903+0000 7f8e49474740 20 client.30953 trim_cache size 0 max 16384 2024-09-19T06:28:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:28:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:28:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:28:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:28:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.904+0000 7f8e4539f640 1 -- 172.21.15.5:0/3638712195 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8e280a3420 msgr2=0x7f8e280a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.904+0000 7f8e4539f640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8e280a3420 0x7f8e280a58e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009430 tx=0x7f8e38009800 comp rx=0 tx=0).stop 2024-09-19T06:28:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.904+0000 7f8e4539f640 1 -- 172.21.15.5:0/3638712195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e401022c0 msgr2=0x7f8e4019a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.904+0000 7f8e4539f640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e401022c0 0x7f8e4019a890 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e9f0 tx=0x7f8e3400edf0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.904+0000 7f8e4539f640 1 -- 172.21.15.5:0/3638712195 shutdown_connections 2024-09-19T06:28:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.904+0000 7f8e4539f640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e40107930 0x7f8e4019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.904+0000 7f8e4539f640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f8e280a3420 0x7f8e280a58e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.904+0000 7f8e4539f640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e401022c0 0x7f8e4019a890 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.904+0000 7f8e4539f640 1 --2- 172.21.15.5:0/3638712195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e40101920 0x7f8e4019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.904+0000 7f8e4539f640 1 -- 172.21.15.5:0/3638712195 >> 172.21.15.5:0/3638712195 conn(0x7f8e400fd600 msgr2=0x7f8e4010cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.904+0000 7f8e4539f640 1 -- 172.21.15.5:0/3638712195 shutdown_connections 2024-09-19T06:28:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:25.904+0000 7f8e4539f640 1 -- 172.21.15.5:0/3638712195 wait complete. 2024-09-19T06:28:26.399 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:28:26.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:28:26.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113756 8.54 MB/sec execute 590 sec latency 82.629 ms 2024-09-19T06:28:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:28:26.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:26 smithi005 ceph-mon[30955]: pgmap v794: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 241 op/s 2024-09-19T06:28:26.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:26 smithi049 ceph-mon[34640]: pgmap v794: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 241 op/s 2024-09-19T06:28:27.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:26 smithi157 ceph-mon[35346]: pgmap v794: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 241 op/s 2024-09-19T06:28:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.426+0000 7ff78bb57640 1 -- 172.21.15.5:0/3182863612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff784106f10 msgr2=0x7ff78410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.426+0000 7ff78bb57640 1 --2- 172.21.15.5:0/3182863612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff784106f10 0x7ff78410ea90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009be0 tx=0x7ff77c02a6c0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.426+0000 7ff78bb57640 1 -- 172.21.15.5:0/3182863612 shutdown_connections 2024-09-19T06:28:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.426+0000 7ff78bb57640 1 --2- 172.21.15.5:0/3182863612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff784106f10 0x7ff78410ea90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.426+0000 7ff78bb57640 1 --2- 172.21.15.5:0/3182863612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff784102790 0x7ff7841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.427+0000 7ff78bb57640 1 --2- 172.21.15.5:0/3182863612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff784101de0 0x7ff7841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.427+0000 7ff78bb57640 1 -- 172.21.15.5:0/3182863612 >> 172.21.15.5:0/3182863612 conn(0x7ff7840fd8e0 msgr2=0x7ff7840ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.427+0000 7ff78bb57640 1 -- 172.21.15.5:0/3182863612 shutdown_connections 2024-09-19T06:28:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.427+0000 7ff78bb57640 1 -- 172.21.15.5:0/3182863612 wait complete. 2024-09-19T06:28:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.427+0000 7ff78bb57640 1 Processor -- start 2024-09-19T06:28:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff78bb57640 1 -- start start 2024-09-19T06:28:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff78bb57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff784101de0 0x7ff7841982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff78bb57640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff784102790 0x7ff784198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff78bb57640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff784106f10 0x7ff78419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff78bb57640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff784111e20 con 0x7ff784101de0 2024-09-19T06:28:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff78bb57640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff784111ca0 con 0x7ff784102790 2024-09-19T06:28:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff78bb57640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff784111fa0 con 0x7ff784106f10 2024-09-19T06:28:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff7898f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff784101de0 0x7ff7841982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff7898f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff784101de0 0x7ff7841982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42348/0 (socket says 172.21.15.5:42348) 2024-09-19T06:28:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff78a0f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff784106f10 0x7ff78419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff7898f3640 1 -- 172.21.15.5:0/1104827289 learned_addr learned my addr 172.21.15.5:0/1104827289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff7890f2640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff784102790 0x7ff784198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff78a0f4640 1 -- 172.21.15.5:0/1104827289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff784102790 msgr2=0x7ff784198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.428+0000 7ff78a0f4640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff784102790 0x7ff784198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.429+0000 7ff78a0f4640 1 -- 172.21.15.5:0/1104827289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff784101de0 msgr2=0x7ff7841982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.429+0000 7ff78a0f4640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff784101de0 0x7ff7841982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.429+0000 7ff78a0f4640 1 -- 172.21.15.5:0/1104827289 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78419d290 con 0x7ff784106f10 2024-09-19T06:28:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.429+0000 7ff7898f3640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff784101de0 0x7ff7841982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.429+0000 7ff78a0f4640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff784106f10 0x7ff78419cbb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff77c02abe0 tx=0x7ff77c003c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.429+0000 7ff77affd640 1 -- 172.21.15.5:0/1104827289 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c041020 con 0x7ff784106f10 2024-09-19T06:28:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.429+0000 7ff78bb57640 1 -- 172.21.15.5:0/1104827289 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78419d520 con 0x7ff784106f10 2024-09-19T06:28:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.429+0000 7ff78bb57640 1 -- 172.21.15.5:0/1104827289 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78419d9d0 con 0x7ff784106f10 2024-09-19T06:28:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.429+0000 7ff77affd640 1 -- 172.21.15.5:0/1104827289 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff77c037040 con 0x7ff784106f10 2024-09-19T06:28:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.429+0000 7ff77affd640 1 -- 172.21.15.5:0/1104827289 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c036070 con 0x7ff784106f10 2024-09-19T06:28:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.432+0000 7ff77affd640 1 -- 172.21.15.5:0/1104827289 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff77c044020 con 0x7ff784106f10 2024-09-19T06:28:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.432+0000 7ff78df30740 1 Processor -- start 2024-09-19T06:28:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.433+0000 7ff78df30740 1 -- start start 2024-09-19T06:28:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.433+0000 7ff78df30740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e02df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.433+0000 7ff78df30740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e02e480 0x55657e038800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.433+0000 7ff78df30740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e038d40 0x55657e03b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.433+0000 7ff78df30740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55657df93280 con 0x55657e02e480 2024-09-19T06:28:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.433+0000 7ff78df30740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55657df3ab10 con 0x55657e038d40 2024-09-19T06:28:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.433+0000 7ff78df30740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55657df8f240 con 0x55657e02db60 2024-09-19T06:28:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.433+0000 7ff7890f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e02df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.433+0000 7ff7898f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e02e480 0x55657e038800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.433+0000 7ff7890f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e02df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55624/0 (socket says 172.21.15.5:55624) 2024-09-19T06:28:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.433+0000 7ff7890f2640 1 -- 172.21.15.5:0/4281082886 learned_addr learned my addr 172.21.15.5:0/4281082886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.433+0000 7ff7890f2640 1 --2- 172.21.15.5:0/4281082886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e038d40 0x55657e03b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.434+0000 7ff77affd640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7700a33b0 0x7ff7700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.434+0000 7ff7890f2640 1 -- 172.21.15.5:0/4281082886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e038d40 msgr2=0x55657e03b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.434+0000 7ff7890f2640 1 --2- 172.21.15.5:0/4281082886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e038d40 0x55657e03b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.434+0000 7ff7890f2640 1 -- 172.21.15.5:0/4281082886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e02e480 msgr2=0x55657e038800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.434+0000 7ff7890f2640 1 --2- 172.21.15.5:0/4281082886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e02e480 0x55657e038800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.434+0000 7ff7890f2640 1 -- 172.21.15.5:0/4281082886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55657df471f0 con 0x55657e02db60 2024-09-19T06:28:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.434+0000 7ff77affd640 1 -- 172.21.15.5:0/1104827289 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7ff77c0f0630 con 0x7ff784106f10 2024-09-19T06:28:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.434+0000 7ff78a0f4640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7700a33b0 0x7ff7700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.434+0000 7ff7890f2640 1 --2- 172.21.15.5:0/4281082886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e02df40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff774009fd0 tx=0x7ff774004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd14fbec6534c97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.434+0000 7ff76ffff640 1 -- 172.21.15.5:0/4281082886 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774004050 con 0x55657e02db60 2024-09-19T06:28:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.434+0000 7ff76ffff640 1 -- 172.21.15.5:0/4281082886 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7740054d0 con 0x55657e02db60 2024-09-19T06:28:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.434+0000 7ff76ffff640 1 -- 172.21.15.5:0/4281082886 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774004c60 con 0x55657e02db60 2024-09-19T06:28:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.435+0000 7ff78df30740 1 -- 172.21.15.5:0/4281082886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 msgr2=0x55657e02df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.435+0000 7ff78df30740 1 --2- 172.21.15.5:0/4281082886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e02df40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff774009fd0 tx=0x7ff774004730 comp rx=0 tx=0).stop 2024-09-19T06:28:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.435+0000 7ff78a0f4640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7700a33b0 0x7ff7700a5870 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff77c006010 tx=0x7ff77c02a6c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.435+0000 7ff78df30740 1 -- 172.21.15.5:0/4281082886 shutdown_connections 2024-09-19T06:28:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.435+0000 7ff78df30740 1 --2- 172.21.15.5:0/4281082886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e038d40 0x55657e03b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.435+0000 7ff78df30740 1 --2- 172.21.15.5:0/4281082886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e02e480 0x55657e038800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.435+0000 7ff78df30740 1 --2- 172.21.15.5:0/4281082886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e02df40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.435+0000 7ff78df30740 1 -- 172.21.15.5:0/4281082886 >> 172.21.15.5:0/4281082886 conn(0x55657e02a0f0 msgr2=0x55657e02a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.435+0000 7ff78df30740 1 -- 172.21.15.5:0/4281082886 shutdown_connections 2024-09-19T06:28:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.435+0000 7ff78df30740 1 -- 172.21.15.5:0/4281082886 wait complete. 2024-09-19T06:28:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.436+0000 7ff78df30740 1 -- start start 2024-09-19T06:28:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.436+0000 7ff78df30740 1 Processor -- start 2024-09-19T06:28:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.436+0000 7ff78df30740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e034540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.436+0000 7ff78df30740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e02e480 0x55657e030880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.436+0000 7ff78df30740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e038d40 0x55657e030e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.436+0000 7ff78df30740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55657e03b910 con 0x55657e038d40 2024-09-19T06:28:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.436+0000 7ff78df30740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55657df4df10 con 0x55657e02e480 2024-09-19T06:28:27.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.436+0000 7ff78df30740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55657e03ba90 con 0x55657e02db60 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.436+0000 7ff78df30740 10 client.0 _ll_register_callbacks cb 0x55657df16a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.436+0000 7ff7890f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e034540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff7898f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e02e480 0x55657e030880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff7890f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e034540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55632/0 (socket says 172.21.15.5:55632) 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff7890f2640 1 -- 172.21.15.5:0/328257351 learned_addr learned my addr 172.21.15.5:0/328257351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff7890f2640 1 --2- 172.21.15.5:0/328257351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e038d40 0x55657e030e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff7890f2640 1 -- 172.21.15.5:0/328257351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e02e480 msgr2=0x55657e030880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff7890f2640 1 --2- 172.21.15.5:0/328257351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e02e480 0x55657e030880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff7890f2640 1 -- 172.21.15.5:0/328257351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e038d40 msgr2=0x55657e030e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff7890f2640 1 --2- 172.21.15.5:0/328257351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e038d40 0x55657e030e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff7890f2640 1 -- 172.21.15.5:0/328257351 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55657dd046a0 con 0x55657e02db60 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff7890f2640 1 --2- 172.21.15.5:0/328257351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e034540 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff774005bc0 tx=0x7ff774004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=675a2230ff90f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff76d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff76d7fa640 1 -- 172.21.15.5:0/328257351 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774011110 con 0x55657e02db60 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff76d7fa640 1 -- 172.21.15.5:0/328257351 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7ff7740112b0 con 0x55657e02db60 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.437+0000 7ff78df30740 1 -- 172.21.15.5:0/328257351 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7740054d0 con 0x55657e02db60 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.438+0000 7ff76d7fa640 1 -- 172.21.15.5:0/328257351 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7ff774034050 con 0x55657e02db60 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.438+0000 7ff78df30740 10 client.30892 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.438+0000 7ff78df30740 1 -- 172.21.15.5:0/328257351 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55657e04fd30 con 0x55657e02db60 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.438+0000 7ff76d7fa640 1 -- 172.21.15.5:0/328257351 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7740054d0 con 0x55657e02db60 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.438+0000 7ff76d7fa640 1 -- 172.21.15.5:0/328257351 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7ff77401d020 con 0x55657e02db60 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.438+0000 7ff78df30740 10 client.30892 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.438+0000 7ff78df30740 10 client.30892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.438+0000 7ff78df30740 20 client.30892 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.438+0000 7ff78df30740 1 --2- 172.21.15.5:0/328257351 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55657e051080 0x55657e053470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.438+0000 7ff78df30740 4 client.30892 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:28:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.438+0000 7ff78df30740 1 -- 172.21.15.5:0/328257351 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55657dfa07f0 con 0x55657e051080 2024-09-19T06:28:27.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.439+0000 7ff7898f3640 1 --2- 172.21.15.5:0/328257351 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55657e051080 0x55657e053470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.439+0000 7ff7898f3640 1 --2- 172.21.15.5:0/328257351 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55657e051080 0x55657e053470 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6d5a3de2cb344d server_cookie=5a6bc98bcb4c49d1 in_seq=0 out_seq=0 2024-09-19T06:28:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.439+0000 7ff76d7fa640 10 client.30892 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.440+0000 7ff76d7fa640 1 -- 172.21.15.5:0/328257351 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55657dfa07f0 con 0x55657e051080 2024-09-19T06:28:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.440+0000 7ff76d7fa640 10 client.30892 handle_command_reply: tid=0 2024-09-19T06:28:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.440+0000 7ff78df30740 1 client.30892 shutdown 2024-09-19T06:28:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.440+0000 7ff78df30740 1 -- 172.21.15.5:0/328257351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 msgr2=0x55657e034540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.440+0000 7ff78df30740 1 --2- 172.21.15.5:0/328257351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e034540 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff774005bc0 tx=0x7ff774004840 comp rx=0 tx=0).stop 2024-09-19T06:28:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.440+0000 7ff78df30740 1 -- 172.21.15.5:0/328257351 shutdown_connections 2024-09-19T06:28:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.440+0000 7ff78df30740 1 --2- 172.21.15.5:0/328257351 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55657e051080 0x55657e053470 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.441+0000 7ff78df30740 1 --2- 172.21.15.5:0/328257351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e038d40 0x55657e030e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.441+0000 7ff76d7fa640 0 client.30892 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:27.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.441+0000 7ff78df30740 1 --2- 172.21.15.5:0/328257351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e02e480 0x55657e030880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.441+0000 7ff78df30740 1 --2- 172.21.15.5:0/328257351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e034540 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.441+0000 7ff78df30740 1 -- 172.21.15.5:0/328257351 >> 172.21.15.5:0/328257351 conn(0x55657e02a0f0 msgr2=0x55657e029270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:27.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.441+0000 7ff78df30740 1 -- 172.21.15.5:0/328257351 shutdown_connections 2024-09-19T06:28:27.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.441+0000 7ff78df30740 1 -- 172.21.15.5:0/328257351 wait complete. 2024-09-19T06:28:27.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.441+0000 7ff78df30740 20 client.30892 trim_cache size 0 max 16384 2024-09-19T06:28:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.471+0000 7ff78df30740 1 Processor -- start 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.471+0000 7ff78df30740 1 -- start start 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.471+0000 7ff78df30740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e03c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.471+0000 7ff78df30740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e02e480 0x55657e058d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.471+0000 7ff78df30740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e038d40 0x55657e059250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.471+0000 7ff78df30740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff76800e7e0 con 0x55657e02e480 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.471+0000 7ff78df30740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff768010320 con 0x55657e038d40 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.471+0000 7ff78df30740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff76800eeb0 con 0x55657e02db60 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.472+0000 7ff7890f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e03c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.472+0000 7ff7898f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e02e480 0x55657e058d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.472+0000 7ff7890f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e03c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55642/0 (socket says 172.21.15.5:55642) 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.472+0000 7ff7890f2640 1 -- 172.21.15.5:0/1427636849 learned_addr learned my addr 172.21.15.5:0/1427636849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.472+0000 7ff7890f2640 1 --2- 172.21.15.5:0/1427636849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e038d40 0x55657e059250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.472+0000 7ff7890f2640 1 -- 172.21.15.5:0/1427636849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e038d40 msgr2=0x55657e059250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:27.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.472+0000 7ff7890f2640 1 --2- 172.21.15.5:0/1427636849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e038d40 0x55657e059250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.472+0000 7ff7890f2640 1 -- 172.21.15.5:0/1427636849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e02e480 msgr2=0x55657e058d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:27.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.472+0000 7ff7890f2640 1 --2- 172.21.15.5:0/1427636849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e02e480 0x55657e058d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.472+0000 7ff7890f2640 1 -- 172.21.15.5:0/1427636849 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55657e031170 con 0x55657e02db60 2024-09-19T06:28:27.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.472+0000 7ff7890f2640 1 --2- 172.21.15.5:0/1427636849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e03c8f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff77400fee0 tx=0x7ff774005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb248509a8a61494 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.472+0000 7ff76f7fe640 1 -- 172.21.15.5:0/1427636849 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7740111d0 con 0x55657e02db60 2024-09-19T06:28:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.473+0000 7ff76f7fe640 1 -- 172.21.15.5:0/1427636849 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff774011370 con 0x55657e02db60 2024-09-19T06:28:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.473+0000 7ff76f7fe640 1 -- 172.21.15.5:0/1427636849 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774004320 con 0x55657e02db60 2024-09-19T06:28:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.474+0000 7ff78df30740 1 -- 172.21.15.5:0/1427636849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 msgr2=0x55657e03c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.474+0000 7ff78df30740 1 --2- 172.21.15.5:0/1427636849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e03c8f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff77400fee0 tx=0x7ff774005f90 comp rx=0 tx=0).stop 2024-09-19T06:28:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.474+0000 7ff78df30740 1 -- 172.21.15.5:0/1427636849 shutdown_connections 2024-09-19T06:28:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.474+0000 7ff78df30740 1 --2- 172.21.15.5:0/1427636849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e038d40 0x55657e059250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.474+0000 7ff78df30740 1 --2- 172.21.15.5:0/1427636849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e02e480 0x55657e058d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.474+0000 7ff78df30740 1 --2- 172.21.15.5:0/1427636849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e03c8f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.474+0000 7ff78df30740 1 -- 172.21.15.5:0/1427636849 >> 172.21.15.5:0/1427636849 conn(0x55657e02a0f0 msgr2=0x55657e02a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.474+0000 7ff78df30740 1 -- 172.21.15.5:0/1427636849 shutdown_connections 2024-09-19T06:28:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.474+0000 7ff78df30740 1 -- 172.21.15.5:0/1427636849 wait complete. 2024-09-19T06:28:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.474+0000 7ff78df30740 1 -- start start 2024-09-19T06:28:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.474+0000 7ff78df30740 1 Processor -- start 2024-09-19T06:28:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff78df30740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e04ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff78df30740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e02e480 0x55657e048170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff78df30740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e038d40 0x55657e0486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff78df30740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff76800fc50 con 0x55657e038d40 2024-09-19T06:28:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff78df30740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff76800f580 con 0x55657e02e480 2024-09-19T06:28:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff78df30740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff76800c7e0 con 0x55657e02db60 2024-09-19T06:28:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff78df30740 10 client.0 _ll_register_callbacks cb 0x7ff780009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff7890f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e038d40 0x55657e0486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff7898f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e02e480 0x55657e048170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff7890f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e038d40 0x55657e0486b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42392/0 (socket says 172.21.15.5:42392) 2024-09-19T06:28:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff7898f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e02e480 0x55657e048170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43436/0 (socket says 172.21.15.5:43436) 2024-09-19T06:28:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff7890f2640 1 -- 172.21.15.5:0/1305280648 learned_addr learned my addr 172.21.15.5:0/1305280648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.475+0000 7ff7890f2640 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e04ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.476+0000 7ff7898f3640 1 -- 172.21.15.5:0/1305280648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 msgr2=0x55657e04ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.476+0000 7ff7898f3640 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e04ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.476+0000 7ff7898f3640 1 -- 172.21.15.5:0/1305280648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e038d40 msgr2=0x55657e0486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.476+0000 7ff7898f3640 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e038d40 0x55657e0486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.476+0000 7ff7898f3640 1 -- 172.21.15.5:0/1305280648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55657e0348b0 con 0x55657e02e480 2024-09-19T06:28:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.476+0000 7ff7890f2640 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e038d40 0x55657e0486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:28:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.476+0000 7ff7898f3640 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e02e480 0x55657e048170 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x55657e053c80 tx=0x7ff780012f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a0b8abd8f55c1b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.476+0000 7ff76d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:28:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.476+0000 7ff76d7fa640 1 -- 172.21.15.5:0/1305280648 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780020c90 con 0x55657e02e480 2024-09-19T06:28:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.476+0000 7ff78df30740 1 -- 172.21.15.5:0/1305280648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff774011370 con 0x55657e02e480 2024-09-19T06:28:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.476+0000 7ff76d7fa640 1 -- 172.21.15.5:0/1305280648 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7ff780005020 con 0x55657e02e480 2024-09-19T06:28:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.477+0000 7ff76d7fa640 1 -- 172.21.15.5:0/1305280648 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7ff780020e30 con 0x55657e02e480 2024-09-19T06:28:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.477+0000 7ff78df30740 10 client.30971 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:27.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.477+0000 7ff78df30740 1 -- 172.21.15.5:0/1305280648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55657e0bbaf0 con 0x55657e02e480 2024-09-19T06:28:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.477+0000 7ff76d7fa640 1 -- 172.21.15.5:0/1305280648 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff774011370 con 0x55657e02e480 2024-09-19T06:28:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.477+0000 7ff76d7fa640 1 -- 172.21.15.5:0/1305280648 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7ff780010430 con 0x55657e02e480 2024-09-19T06:28:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.477+0000 7ff78df30740 10 client.30971 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.477+0000 7ff78df30740 10 client.30971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.477+0000 7ff78df30740 20 client.30971 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.477+0000 7ff78df30740 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55657e051080 0x55657e0be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.477+0000 7ff78df30740 4 client.30971 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:28:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.477+0000 7ff78df30740 1 -- 172.21.15.5:0/1305280648 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55657df9ad00 con 0x55657e051080 2024-09-19T06:28:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.478+0000 7ff7890f2640 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55657e051080 0x55657e0be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.478+0000 7ff7890f2640 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55657e051080 0x55657e0be300 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cac0ad3b345aa484 server_cookie=1cc5ce6009c926ea in_seq=0 out_seq=0 2024-09-19T06:28:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.478+0000 7ff76d7fa640 10 client.30971 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.479+0000 7ff76d7fa640 1 -- 172.21.15.5:0/1305280648 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55657df9ad00 con 0x55657e051080 2024-09-19T06:28:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.479+0000 7ff76d7fa640 10 client.30971 handle_command_reply: tid=0 2024-09-19T06:28:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.479+0000 7ff78df30740 1 client.30971 shutdown 2024-09-19T06:28:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.480+0000 7ff78df30740 1 -- 172.21.15.5:0/1305280648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e02e480 msgr2=0x55657e048170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.480+0000 7ff78df30740 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e02e480 0x55657e048170 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x55657e053c80 tx=0x7ff780012f40 comp rx=0 tx=0).stop 2024-09-19T06:28:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.480+0000 7ff78df30740 1 -- 172.21.15.5:0/1305280648 shutdown_connections 2024-09-19T06:28:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.480+0000 7ff78df30740 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55657e051080 0x55657e0be300 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.480+0000 7ff78df30740 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55657e038d40 0x55657e0486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.480+0000 7ff76d7fa640 0 client.30971 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.480+0000 7ff78df30740 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55657e02e480 0x55657e048170 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.480+0000 7ff78df30740 1 --2- 172.21.15.5:0/1305280648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55657e02db60 0x55657e04ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.480+0000 7ff78df30740 1 -- 172.21.15.5:0/1305280648 >> 172.21.15.5:0/1305280648 conn(0x55657e02a0f0 msgr2=0x55657e02f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:27.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.480+0000 7ff78df30740 1 -- 172.21.15.5:0/1305280648 shutdown_connections 2024-09-19T06:28:27.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.480+0000 7ff78df30740 1 -- 172.21.15.5:0/1305280648 wait complete. 2024-09-19T06:28:27.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.480+0000 7ff78df30740 20 client.30971 trim_cache size 0 max 16384 2024-09-19T06:28:27.486 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:28:27.486 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-19T06:28:27.486 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "d4d7a6e8-42ca-44d2-b1ca-069ae3bd85d3", 2024-09-19T06:28:27.486 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-19T06:28:27.486 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:28:27.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.481+0000 7ff78bb57640 1 -- 172.21.15.5:0/1104827289 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7700a33b0 msgr2=0x7ff7700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:27.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.481+0000 7ff78bb57640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7700a33b0 0x7ff7700a5870 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff77c006010 tx=0x7ff77c02a6c0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.481+0000 7ff78bb57640 1 -- 172.21.15.5:0/1104827289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff784106f10 msgr2=0x7ff78419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:27.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.481+0000 7ff78bb57640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff784106f10 0x7ff78419cbb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff77c02abe0 tx=0x7ff77c003c70 comp rx=0 tx=0).stop 2024-09-19T06:28:27.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.481+0000 7ff78bb57640 1 -- 172.21.15.5:0/1104827289 shutdown_connections 2024-09-19T06:28:27.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.481+0000 7ff78bb57640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff7700a33b0 0x7ff7700a5870 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.481+0000 7ff78bb57640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff784106f10 0x7ff78419cbb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.481+0000 7ff78bb57640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff784102790 0x7ff784198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.481+0000 7ff78bb57640 1 --2- 172.21.15.5:0/1104827289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff784101de0 0x7ff7841982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:27.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.481+0000 7ff78bb57640 1 -- 172.21.15.5:0/1104827289 >> 172.21.15.5:0/1104827289 conn(0x7ff7840fd8e0 msgr2=0x7ff7840fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:27.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.482+0000 7ff78bb57640 1 -- 172.21.15.5:0/1104827289 shutdown_connections 2024-09-19T06:28:27.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:27.482+0000 7ff78bb57640 1 -- 172.21.15.5:0/1104827289 wait complete. 2024-09-19T06:28:27.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113914 8.54 MB/sec execute 591 sec latency 94.411 ms 2024-09-19T06:28:28.337 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:28:28.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 114057 8.53 MB/sec execute 592 sec latency 183.615 ms 2024-09-19T06:28:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:28:28.848 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: pgmap v795: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.0 MiB/s wr, 384 op/s 2024-09-19T06:28:28.848 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub queued for path: / 2024-09-19T06:28:28.848 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.848 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:28:28.848 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:28:28.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:28:28.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:28:28.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: active paths [/] 2024-09-19T06:28:28.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:28 smithi005 ceph-mon[30955]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: pgmap v795: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.0 MiB/s wr, 384 op/s 2024-09-19T06:28:28.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub queued for path: / 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.897 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:28:28.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:28:28.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:28:28.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:28:28.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: active paths [/] 2024-09-19T06:28:28.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:28.898 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:28 smithi049 ceph-mon[34640]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: pgmap v795: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.0 MiB/s wr, 384 op/s 2024-09-19T06:28:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub queued for path: / 2024-09-19T06:28:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:28:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:28:29.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:28:29.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:28:29.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: active paths [/] 2024-09-19T06:28:29.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:28 smithi157 ceph-mon[35346]: scrub summary: idle+waiting paths [/] 2024-09-19T06:28:29.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.279+0000 7f25f7a06640 1 -- 172.21.15.5:0/2497076862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25f010c170 msgr2=0x7f25f010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:29.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.279+0000 7f25f7a06640 1 --2- 172.21.15.5:0/2497076862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25f010c170 0x7f25f010e610 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009ca0 tx=0x7f25e802a950 comp rx=0 tx=0).stop 2024-09-19T06:28:29.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.279+0000 7f25f7a06640 1 -- 172.21.15.5:0/2497076862 shutdown_connections 2024-09-19T06:28:29.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.279+0000 7f25f7a06640 1 --2- 172.21.15.5:0/2497076862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25f010c170 0x7f25f010e610 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.279+0000 7f25f7a06640 1 --2- 172.21.15.5:0/2497076862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f0103660 0x7f25f010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.279+0000 7f25f7a06640 1 --2- 172.21.15.5:0/2497076862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25f0102d40 0x7f25f0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.279+0000 7f25f7a06640 1 -- 172.21.15.5:0/2497076862 >> 172.21.15.5:0/2497076862 conn(0x7f25f00fc7a0 msgr2=0x7f25f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:29.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.280+0000 7f25f7a06640 1 -- 172.21.15.5:0/2497076862 shutdown_connections 2024-09-19T06:28:29.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.280+0000 7f25f7a06640 1 -- 172.21.15.5:0/2497076862 wait complete. 2024-09-19T06:28:29.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.280+0000 7f25f7a06640 1 Processor -- start 2024-09-19T06:28:29.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.281+0000 7f25f7a06640 1 -- start start 2024-09-19T06:28:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.281+0000 7f25f7a06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25f0102d40 0x7f25f0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.281+0000 7f25f7a06640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25f0103660 0x7f25f0198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.281+0000 7f25f7a06640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f010c170 0x7f25f019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.281+0000 7f25f57a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25f0102d40 0x7f25f0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.281+0000 7f25f7a06640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25f0111e70 con 0x7f25f0102d40 2024-09-19T06:28:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.281+0000 7f25f7a06640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25f0111cf0 con 0x7f25f0103660 2024-09-19T06:28:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.281+0000 7f25f7a06640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f25f0111ff0 con 0x7f25f010c170 2024-09-19T06:28:29.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.281+0000 7f25f57a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25f0102d40 0x7f25f0198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42420/0 (socket says 172.21.15.5:42420) 2024-09-19T06:28:29.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.281+0000 7f25f57a2640 1 -- 172.21.15.5:0/646863529 learned_addr learned my addr 172.21.15.5:0/646863529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:29.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.281+0000 7f25f4fa1640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25f0103660 0x7f25f0198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.282+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f010c170 0x7f25f019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.282+0000 7f25f4fa1640 1 -- 172.21.15.5:0/646863529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f010c170 msgr2=0x7f25f019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:29.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.282+0000 7f25f4fa1640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f010c170 0x7f25f019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.282+0000 7f25f4fa1640 1 -- 172.21.15.5:0/646863529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25f0102d40 msgr2=0x7f25f0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:29.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.282+0000 7f25f4fa1640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25f0102d40 0x7f25f0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.282+0000 7f25f4fa1640 1 -- 172.21.15.5:0/646863529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f019d300 con 0x7f25f0103660 2024-09-19T06:28:29.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.282+0000 7f25f57a2640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25f0102d40 0x7f25f0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:29.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.282+0000 7f25f4fa1640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25f0103660 0x7f25f0198890 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e7a0 tx=0x7f25e000ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:29.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.282+0000 7f25e67fc640 1 -- 172.21.15.5:0/646863529 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e000ccc0 con 0x7f25f0103660 2024-09-19T06:28:29.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.283+0000 7f25e67fc640 1 -- 172.21.15.5:0/646863529 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25e0009530 con 0x7f25f0103660 2024-09-19T06:28:29.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.283+0000 7f25f7a06640 1 -- 172.21.15.5:0/646863529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f019d5f0 con 0x7f25f0103660 2024-09-19T06:28:29.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.283+0000 7f25e67fc640 1 -- 172.21.15.5:0/646863529 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0018710 con 0x7f25f0103660 2024-09-19T06:28:29.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.283+0000 7f25f7a06640 1 -- 172.21.15.5:0/646863529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f01008d0 con 0x7f25f0103660 2024-09-19T06:28:29.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.286+0000 7f25f9ddf740 1 Processor -- start 2024-09-19T06:28:29.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.286+0000 7f25f9ddf740 1 -- start start 2024-09-19T06:28:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.286+0000 7f25f9ddf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e24a0 0x55a4b67e2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.286+0000 7f25f9ddf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.286+0000 7f25f9ddf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67ed680 0x55a4b67efa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.286+0000 7f25f9ddf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a4b67482e0 con 0x55a4b67e2dc0 2024-09-19T06:28:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.286+0000 7f25f9ddf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4b66efce0 con 0x55a4b67ed680 2024-09-19T06:28:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.286+0000 7f25f9ddf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a4b67442a0 con 0x55a4b67e24a0 2024-09-19T06:28:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.286+0000 7f25f57a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.286+0000 7f25f57a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ed140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42424/0 (socket says 172.21.15.5:42424) 2024-09-19T06:28:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.286+0000 7f25f57a2640 1 -- 172.21.15.5:0/264801497 learned_addr learned my addr 172.21.15.5:0/264801497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.286+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/264801497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e24a0 0x55a4b67e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.287+0000 7f25e67fc640 1 -- 172.21.15.5:0/646863529 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f25e0010430 con 0x7f25f0103660 2024-09-19T06:28:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.287+0000 7f25f4fa1640 1 --2- 172.21.15.5:0/264801497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67ed680 0x55a4b67efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.287+0000 7f25f57a2640 1 -- 172.21.15.5:0/264801497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e24a0 msgr2=0x55a4b67e2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.287+0000 7f25f57a2640 1 --2- 172.21.15.5:0/264801497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e24a0 0x55a4b67e2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.287+0000 7f25f57a2640 1 -- 172.21.15.5:0/264801497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67ed680 msgr2=0x55a4b67efa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.287+0000 7f25f57a2640 1 --2- 172.21.15.5:0/264801497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67ed680 0x55a4b67efa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.287+0000 7f25f57a2640 1 -- 172.21.15.5:0/264801497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4b66d4440 con 0x55a4b67e2dc0 2024-09-19T06:28:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.287+0000 7f25f4fa1640 1 --2- 172.21.15.5:0/264801497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67ed680 0x55a4b67efa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:28:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.287+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/264801497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e24a0 0x55a4b67e2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.287+0000 7f25f57a2640 1 --2- 172.21.15.5:0/264801497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ed140 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f25ec011650 tx=0x7f25ec009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a54df54cb04518 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.288+0000 7f25e67fc640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25dc0a3430 0x7f25dc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.288+0000 7f25db7fe640 1 -- 172.21.15.5:0/264801497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec01e070 con 0x55a4b67e2dc0 2024-09-19T06:28:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.288+0000 7f25db7fe640 1 -- 172.21.15.5:0/264801497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25ec00fb70 con 0x55a4b67e2dc0 2024-09-19T06:28:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.288+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25dc0a3430 0x7f25dc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.288+0000 7f25db7fe640 1 -- 172.21.15.5:0/264801497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec019c60 con 0x55a4b67e2dc0 2024-09-19T06:28:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.288+0000 7f25e67fc640 1 -- 172.21.15.5:0/646863529 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f25e00cfea0 con 0x7f25f0103660 2024-09-19T06:28:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.289+0000 7f25f9ddf740 1 -- 172.21.15.5:0/264801497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 msgr2=0x55a4b67ed140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.289+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25dc0a3430 0x7f25dc0a58f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f25e8006010 tx=0x7f25e8005d80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.289+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/264801497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ed140 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f25ec011650 tx=0x7f25ec009f90 comp rx=0 tx=0).stop 2024-09-19T06:28:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.289+0000 7f25f9ddf740 1 -- 172.21.15.5:0/264801497 shutdown_connections 2024-09-19T06:28:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.289+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/264801497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67ed680 0x55a4b67efa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.289+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/264801497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ed140 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.289+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/264801497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e24a0 0x55a4b67e2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.289+0000 7f25f9ddf740 1 -- 172.21.15.5:0/264801497 >> 172.21.15.5:0/264801497 conn(0x55a4b67dea30 msgr2=0x55a4b67dee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.289+0000 7f25f9ddf740 1 -- 172.21.15.5:0/264801497 shutdown_connections 2024-09-19T06:28:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.289+0000 7f25f9ddf740 1 -- 172.21.15.5:0/264801497 wait complete. 2024-09-19T06:28:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.289+0000 7f25f9ddf740 1 -- start start 2024-09-19T06:28:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.290+0000 7f25f9ddf740 1 Processor -- start 2024-09-19T06:28:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.290+0000 7f25f9ddf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e24a0 0x55a4b67ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.290+0000 7f25f9ddf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.290+0000 7f25f9ddf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67ed680 0x55a4b67ea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.290+0000 7f25f4fa1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.290+0000 7f25f4fa1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ea210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43466/0 (socket says 172.21.15.5:43466) 2024-09-19T06:28:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.290+0000 7f25f9ddf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a4b67f02f0 con 0x55a4b67ed680 2024-09-19T06:28:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.290+0000 7f25f4fa1640 1 -- 172.21.15.5:0/1364818065 learned_addr learned my addr 172.21.15.5:0/1364818065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.290+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1364818065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4b6703050 con 0x55a4b67e2dc0 2024-09-19T06:28:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.290+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67ed680 0x55a4b67ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.291+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1364818065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a4b67f0470 con 0x55a4b67e24a0 2024-09-19T06:28:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.291+0000 7f25f57a2640 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e24a0 0x55a4b67ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.291+0000 7f25f9ddf740 10 client.0 _ll_register_callbacks cb 0x55a4b6743160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.291+0000 7f25f5fa3640 1 -- 172.21.15.5:0/1364818065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e24a0 msgr2=0x55a4b67ecf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.291+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e24a0 0x55a4b67ecf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.291+0000 7f25f5fa3640 1 -- 172.21.15.5:0/1364818065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67e2dc0 msgr2=0x55a4b67ea210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.291+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ea210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.291+0000 7f25f5fa3640 1 -- 172.21.15.5:0/1364818065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4b64b93c0 con 0x55a4b67ed680 2024-09-19T06:28:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.291+0000 7f25f57a2640 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e24a0 0x55a4b67ecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.291+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67ed680 0x55a4b67ea750 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f25e8005e30 tx=0x7f25e80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229279b4729ef0ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.291+0000 7f25d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:28:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25d8ff9640 1 -- 172.21.15.5:0/1364818065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8056060 con 0x55a4b67ed680 2024-09-19T06:28:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25d8ff9640 1 -- 172.21.15.5:0/1364818065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f25e8004250 con 0x55a4b67ed680 2024-09-19T06:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1364818065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25ec00fb70 con 0x55a4b67ed680 2024-09-19T06:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25d8ff9640 1 -- 172.21.15.5:0/1364818065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f25e8034040 con 0x55a4b67ed680 2024-09-19T06:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25f9ddf740 10 client.21621 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1364818065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55a4b67fbc00 con 0x55a4b67ed680 2024-09-19T06:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25d8ff9640 1 -- 172.21.15.5:0/1364818065 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25ec00fb70 con 0x55a4b67ed680 2024-09-19T06:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25d8ff9640 1 -- 172.21.15.5:0/1364818065 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f25e8030880 con 0x55a4b67ed680 2024-09-19T06:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25f9ddf740 10 client.21621 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25f9ddf740 10 client.21621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25f9ddf740 20 client.21621 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4b67fd4a0 0x55a4b67ff910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25f9ddf740 4 client.21621 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:28:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.292+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1364818065 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4b6755850 con 0x55a4b67fd4a0 2024-09-19T06:28:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.293+0000 7f25f57a2640 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4b67fd4a0 0x55a4b67ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.293+0000 7f25d8ff9640 10 client.21621 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.293+0000 7f25f57a2640 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4b67fd4a0 0x55a4b67ff910 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=151ea95b4b871e44 server_cookie=74c68b19da94438b in_seq=0 out_seq=0 2024-09-19T06:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.294+0000 7f25d8ff9640 1 -- 172.21.15.5:0/1364818065 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4b6755850 con 0x55a4b67fd4a0 2024-09-19T06:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.294+0000 7f25d8ff9640 10 client.21621 handle_command_reply: tid=0 2024-09-19T06:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.294+0000 7f25f9ddf740 1 client.21621 shutdown 2024-09-19T06:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.295+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1364818065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67ed680 msgr2=0x55a4b67ea750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.295+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67ed680 0x55a4b67ea750 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f25e8005e30 tx=0x7f25e80095c0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.295+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1364818065 shutdown_connections 2024-09-19T06:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.295+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4b67fd4a0 0x55a4b67ff910 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.295+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67ed680 0x55a4b67ea750 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.295+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.295+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/1364818065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e24a0 0x55a4b67ecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.295+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1364818065 >> 172.21.15.5:0/1364818065 conn(0x55a4b67dea30 msgr2=0x55a4b67e4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.295+0000 7f25d8ff9640 0 client.21621 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.295+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1364818065 shutdown_connections 2024-09-19T06:28:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.295+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1364818065 wait complete. 2024-09-19T06:28:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.295+0000 7f25f9ddf740 20 client.21621 trim_cache size 0 max 16384 2024-09-19T06:28:29.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.331+0000 7f25f9ddf740 1 Processor -- start 2024-09-19T06:28:29.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.331+0000 7f25f9ddf740 1 -- start start 2024-09-19T06:28:29.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f9ddf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67dea30 0x55a4b67f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f9ddf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e24a0 0x55a4b67e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f9ddf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ec180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f9ddf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01afa00 con 0x55a4b67e24a0 2024-09-19T06:28:29.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f9ddf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01b1540 con 0x55a4b67dea30 2024-09-19T06:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f4fa1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e24a0 0x55a4b67e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f4fa1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e24a0 0x55a4b67e8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42448/0 (socket says 172.21.15.5:42448) 2024-09-19T06:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f4fa1640 1 -- 172.21.15.5:0/1020292763 learned_addr learned my addr 172.21.15.5:0/1020292763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f9ddf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01b00d0 con 0x55a4b67e2dc0 2024-09-19T06:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f57a2640 1 --2- 172.21.15.5:0/1020292763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67dea30 0x55a4b67f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/1020292763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f4fa1640 1 -- 172.21.15.5:0/1020292763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e2dc0 msgr2=0x55a4b67ec180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f4fa1640 1 --2- 172.21.15.5:0/1020292763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ec180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f4fa1640 1 -- 172.21.15.5:0/1020292763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67dea30 msgr2=0x55a4b67f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f4fa1640 1 --2- 172.21.15.5:0/1020292763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67dea30 0x55a4b67f9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.332+0000 7f25f4fa1640 1 -- 172.21.15.5:0/1020292763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4b67e91c0 con 0x55a4b67e24a0 2024-09-19T06:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.333+0000 7f25f4fa1640 1 --2- 172.21.15.5:0/1020292763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e24a0 0x55a4b67e8ae0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f25e0000c00 tx=0x7f25e0010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4306afe3cc19d07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.333+0000 7f25daffd640 1 -- 172.21.15.5:0/1020292763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0010a70 con 0x55a4b67e24a0 2024-09-19T06:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.333+0000 7f25daffd640 1 -- 172.21.15.5:0/1020292763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25e00a0170 con 0x55a4b67e24a0 2024-09-19T06:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.333+0000 7f25daffd640 1 -- 172.21.15.5:0/1020292763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e00cd7f0 con 0x55a4b67e24a0 2024-09-19T06:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.334+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1020292763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e24a0 msgr2=0x55a4b67e8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.334+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/1020292763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e24a0 0x55a4b67e8ae0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f25e0000c00 tx=0x7f25e0010d40 comp rx=0 tx=0).stop 2024-09-19T06:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.334+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1020292763 shutdown_connections 2024-09-19T06:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.334+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/1020292763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67e2dc0 0x55a4b67ec180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.334+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/1020292763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e24a0 0x55a4b67e8ae0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.334+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/1020292763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67dea30 0x55a4b67f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.334+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1020292763 >> 172.21.15.5:0/1020292763 conn(0x55a4b67ea8b0 msgr2=0x55a4b67eba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.334+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1020292763 shutdown_connections 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.334+0000 7f25f9ddf740 1 -- 172.21.15.5:0/1020292763 wait complete. 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.334+0000 7f25f9ddf740 1 -- start start 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.335+0000 7f25f9ddf740 1 Processor -- start 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.335+0000 7f25f9ddf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67dea30 0x55a4b67f1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.335+0000 7f25f9ddf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67e24a0 0x55a4b67f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.335+0000 7f25f9ddf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 0x55a4b67f2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.335+0000 7f25f5fa3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 0x55a4b67f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.335+0000 7f25f5fa3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 0x55a4b67f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42452/0 (socket says 172.21.15.5:42452) 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.335+0000 7f25f57a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67dea30 0x55a4b67f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.335+0000 7f25f57a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67dea30 0x55a4b67f1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55694/0 (socket says 172.21.15.5:55694) 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.336+0000 7f25f5fa3640 1 -- 172.21.15.5:0/2655144029 learned_addr learned my addr 172.21.15.5:0/2655144029 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.336+0000 7f25f9ddf740 1 -- 172.21.15.5:0/2655144029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01b0e70 con 0x55a4b67e2dc0 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.336+0000 7f25f9ddf740 1 -- 172.21.15.5:0/2655144029 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01b07a0 con 0x55a4b67e24a0 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.336+0000 7f25f9ddf740 1 -- 172.21.15.5:0/2655144029 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01ada00 con 0x55a4b67dea30 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.336+0000 7f25f4fa1640 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67e24a0 0x55a4b67f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.336+0000 7f25f9ddf740 10 client.0 _ll_register_callbacks cb 0x7f25e8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.336+0000 7f25f5fa3640 1 -- 172.21.15.5:0/2655144029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67dea30 msgr2=0x55a4b67f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.336+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67dea30 0x55a4b67f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.336+0000 7f25f5fa3640 1 -- 172.21.15.5:0/2655144029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67e24a0 msgr2=0x55a4b67f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.336+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67e24a0 0x55a4b67f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.336+0000 7f25f5fa3640 1 -- 172.21.15.5:0/2655144029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4b67ed2d0 con 0x55a4b67e2dc0 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.336+0000 7f25f57a2640 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67dea30 0x55a4b67f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.337+0000 7f25f5fa3640 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 0x55a4b67f2180 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f25e8005d50 tx=0x7f25e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70be86c61643259 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.337+0000 7f25d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.337+0000 7f25d8ff9640 1 -- 172.21.15.5:0/2655144029 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8056070 con 0x55a4b67e2dc0 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.337+0000 7f25d8ff9640 1 -- 172.21.15.5:0/2655144029 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f25e8030440 con 0x55a4b67e2dc0 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.337+0000 7f25f9ddf740 1 -- 172.21.15.5:0/2655144029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25e00a0170 con 0x55a4b67e2dc0 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.337+0000 7f25d8ff9640 1 -- 172.21.15.5:0/2655144029 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f25e80098e0 con 0x55a4b67e2dc0 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.337+0000 7f25f9ddf740 10 client.21627 fetch_fsmap learned FSMap version 62 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.337+0000 7f25f9ddf740 1 -- 172.21.15.5:0/2655144029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55a4b6868530 con 0x55a4b67e2dc0 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.338+0000 7f25d8ff9640 1 -- 172.21.15.5:0/2655144029 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25e8003d10 con 0x55a4b67e2dc0 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.338+0000 7f25d8ff9640 1 -- 172.21.15.5:0/2655144029 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) ==== 8362+0+0 (secure 0 0 0) 0x7f25e8052980 con 0x55a4b67e2dc0 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.338+0000 7f25f9ddf740 10 client.21627 fetch_fsmap finished waiting for FSMap version 62 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.338+0000 7f25f9ddf740 10 client.21627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.338+0000 7f25f9ddf740 20 client.21627 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.338+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4b67ed680 0x55a4b686acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.338+0000 7f25f9ddf740 4 client.21627 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.338+0000 7f25f9ddf740 1 -- 172.21.15.5:0/2655144029 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f25e00a0170 con 0x55a4b67ed680 2024-09-19T06:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.338+0000 7f25f57a2640 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4b67ed680 0x55a4b686acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25d8ff9640 10 client.21627 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25f57a2640 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4b67ed680 0x55a4b686acb0 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c4a5e0b9bd9d7f server_cookie=1b5f0b098c1f6af9 in_seq=0 out_seq=0 2024-09-19T06:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25d8ff9640 1 -- 172.21.15.5:0/2655144029 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f25e00a0170 con 0x55a4b67ed680 2024-09-19T06:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25d8ff9640 10 client.21627 handle_command_reply: tid=0 2024-09-19T06:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25f9ddf740 1 client.21627 shutdown 2024-09-19T06:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25f9ddf740 1 -- 172.21.15.5:0/2655144029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 msgr2=0x55a4b67f2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 0x55a4b67f2180 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f25e8005d50 tx=0x7f25e8009bd0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25f9ddf740 1 -- 172.21.15.5:0/2655144029 shutdown_connections 2024-09-19T06:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55a4b67ed680 0x55a4b686acb0 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25d8ff9640 0 client.21627 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4b67e2dc0 0x55a4b67f2180 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b67e24a0 0x55a4b67f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25f9ddf740 1 --2- 172.21.15.5:0/2655144029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a4b67dea30 0x55a4b67f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.339+0000 7f25f9ddf740 1 -- 172.21.15.5:0/2655144029 >> 172.21.15.5:0/2655144029 conn(0x55a4b67ea8b0 msgr2=0x55a4b67eba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.340+0000 7f25f9ddf740 1 -- 172.21.15.5:0/2655144029 shutdown_connections 2024-09-19T06:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.340+0000 7f25f9ddf740 1 -- 172.21.15.5:0/2655144029 wait complete. 2024-09-19T06:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.340+0000 7f25f9ddf740 20 client.21627 trim_cache size 0 max 16384 2024-09-19T06:28:29.353 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:28:29.354 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:28:29.354 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-19T06:28:29.354 INFO:teuthology.orchestra.run.smithi005.stdout: "d4d7a6e8-42ca-44d2-b1ca-069ae3bd85d3": { 2024-09-19T06:28:29.354 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-19T06:28:29.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d4d7a6e8-42ca-44d2-b1ca-069ae3bd85d3", 2024-09-19T06:28:29.354 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-19T06:28:29.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:28:29.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:28:29.354 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:28:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.340+0000 7f25f7a06640 1 -- 172.21.15.5:0/646863529 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25dc0a3430 msgr2=0x7f25dc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.341+0000 7f25f7a06640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25dc0a3430 0x7f25dc0a58f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f25e8006010 tx=0x7f25e8005d80 comp rx=0 tx=0).stop 2024-09-19T06:28:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.341+0000 7f25f7a06640 1 -- 172.21.15.5:0/646863529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25f0103660 msgr2=0x7f25f0198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.341+0000 7f25f7a06640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25f0103660 0x7f25f0198890 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e7a0 tx=0x7f25e000ec60 comp rx=0 tx=0).stop 2024-09-19T06:28:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.341+0000 7f25f7a06640 1 -- 172.21.15.5:0/646863529 shutdown_connections 2024-09-19T06:28:29.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.341+0000 7f25f7a06640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f25dc0a3430 0x7f25dc0a58f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.341+0000 7f25f7a06640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f010c170 0x7f25f019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.341+0000 7f25f7a06640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25f0103660 0x7f25f0198890 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.341+0000 7f25f7a06640 1 --2- 172.21.15.5:0/646863529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25f0102d40 0x7f25f0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:29.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.341+0000 7f25f7a06640 1 -- 172.21.15.5:0/646863529 >> 172.21.15.5:0/646863529 conn(0x7f25f00fc7a0 msgr2=0x7f25f00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:29.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.341+0000 7f25f7a06640 1 -- 172.21.15.5:0/646863529 shutdown_connections 2024-09-19T06:28:29.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:29.341+0000 7f25f7a06640 1 -- 172.21.15.5:0/646863529 wait complete. 2024-09-19T06:28:29.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 114185 8.52 MB/sec execute 593 sec latency 298.621 ms 2024-09-19T06:28:30.178 INFO:tasks.cephfs.filesystem:scrub status for tag:d4d7a6e8-42ca-44d2-b1ca-069ae3bd85d3 - {'path': '/', 'tag': 'd4d7a6e8-42ca-44d2-b1ca-069ae3bd85d3', 'options': 'recursive,force'} 2024-09-19T06:28:30.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 114358 8.52 MB/sec execute 594 sec latency 58.351 ms 2024-09-19T06:28:30.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:30 smithi005 ceph-mon[30955]: pgmap v796: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 252 op/s 2024-09-19T06:28:30.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:30 smithi049 ceph-mon[34640]: pgmap v796: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 252 op/s 2024-09-19T06:28:31.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:30 smithi157 ceph-mon[35346]: pgmap v796: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 252 op/s 2024-09-19T06:28:31.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 114518 8.52 MB/sec execute 595 sec latency 64.548 ms 2024-09-19T06:28:32.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 114692 8.53 MB/sec execute 596 sec latency 86.885 ms 2024-09-19T06:28:32.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:32 smithi005 ceph-mon[30955]: pgmap v797: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 252 op/s 2024-09-19T06:28:32.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:32 smithi049 ceph-mon[34640]: pgmap v797: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 252 op/s 2024-09-19T06:28:33.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:32 smithi157 ceph-mon[35346]: pgmap v797: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 252 op/s 2024-09-19T06:28:33.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 114886 8.53 MB/sec execute 597 sec latency 64.690 ms 2024-09-19T06:28:34.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 115096 8.54 MB/sec execute 598 sec latency 63.212 ms 2024-09-19T06:28:34.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:34 smithi005 ceph-mon[30955]: pgmap v798: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.7 MiB/s wr, 437 op/s 2024-09-19T06:28:34.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:34 smithi049 ceph-mon[34640]: pgmap v798: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.7 MiB/s wr, 437 op/s 2024-09-19T06:28:35.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:34 smithi157 ceph-mon[35346]: pgmap v798: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.7 MiB/s wr, 437 op/s 2024-09-19T06:28:35.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 115220 8.53 MB/sec execute 599 sec latency 228.811 ms 2024-09-19T06:28:36.602 INFO:tasks.workunit.client.0.smithi005.stdout: 10 cleanup 600 sec 2024-09-19T06:28:36.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:36 smithi005 ceph-mon[30955]: pgmap v799: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.8 MiB/s wr, 328 op/s 2024-09-19T06:28:36.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:36 smithi049 ceph-mon[34640]: pgmap v799: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.8 MiB/s wr, 328 op/s 2024-09-19T06:28:37.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:36 smithi157 ceph-mon[35346]: pgmap v799: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.8 MiB/s wr, 328 op/s 2024-09-19T06:28:37.602 INFO:tasks.workunit.client.0.smithi005.stdout: 4 cleanup 601 sec 2024-09-19T06:28:38.361 INFO:tasks.workunit.client.0.smithi005.stdout: 0 cleanup 602 sec 2024-09-19T06:28:38.361 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-19T06:28:38.361 INFO:tasks.workunit.client.0.smithi005.stdout: Operation Count AvgLat MaxLat 2024-09-19T06:28:38.361 INFO:tasks.workunit.client.0.smithi005.stdout: ---------------------------------------- 2024-09-19T06:28:38.361 INFO:tasks.workunit.client.0.smithi005.stdout: NTCreateX 164896 10.525 348.890 2024-09-19T06:28:38.361 INFO:tasks.workunit.client.0.smithi005.stdout: Close 121018 0.197 132.518 2024-09-19T06:28:38.361 INFO:tasks.workunit.client.0.smithi005.stdout: Rename 6967 25.053 713.993 2024-09-19T06:28:38.361 INFO:tasks.workunit.client.0.smithi005.stdout: Unlink 33343 8.670 278.405 2024-09-19T06:28:38.361 INFO:tasks.workunit.client.0.smithi005.stdout: Qpathinfo 149383 7.198 407.025 2024-09-19T06:28:38.361 INFO:tasks.workunit.client.0.smithi005.stdout: Qfileinfo 25999 0.371 42.917 2024-09-19T06:28:38.362 INFO:tasks.workunit.client.0.smithi005.stdout: Qfsinfo 27325 0.989 92.849 2024-09-19T06:28:38.362 INFO:tasks.workunit.client.0.smithi005.stdout: Sfileinfo 13456 3.124 123.160 2024-09-19T06:28:38.362 INFO:tasks.workunit.client.0.smithi005.stdout: Find 57701 42.682 373.776 2024-09-19T06:28:38.362 INFO:tasks.workunit.client.0.smithi005.stdout: WriteX 81287 0.681 316.885 2024-09-19T06:28:38.362 INFO:tasks.workunit.client.0.smithi005.stdout: ReadX 257577 0.008 17.384 2024-09-19T06:28:38.362 INFO:tasks.workunit.client.0.smithi005.stdout: LockX 532 2.015 17.260 2024-09-19T06:28:38.362 INFO:tasks.workunit.client.0.smithi005.stdout: UnlockX 532 1.957 66.114 2024-09-19T06:28:38.362 INFO:tasks.workunit.client.0.smithi005.stdout: Flush 11577 8.437 321.905 2024-09-19T06:28:38.362 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-19T06:28:38.362 INFO:tasks.workunit.client.0.smithi005.stdout:Throughput 8.527 MB/sec 10 clients 10 procs max_latency=713.999 ms 2024-09-19T06:28:38.362 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T06:28:38.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T06:28:38.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:38 smithi005 ceph-mon[30955]: pgmap v800: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.5 MiB/s wr, 457 op/s 2024-09-19T06:28:38.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:38 smithi049 ceph-mon[34640]: pgmap v800: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.5 MiB/s wr, 457 op/s 2024-09-19T06:28:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.006+0000 7f1c296e6640 1 -- 172.21.15.5:0/2393616012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c241069b0 msgr2=0x7f1c24106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.006+0000 7f1c296e6640 1 --2- 172.21.15.5:0/2393616012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c241069b0 0x7f1c24106d90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009890 tx=0x7f1c1402a380 comp rx=0 tx=0).stop 2024-09-19T06:28:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.006+0000 7f1c296e6640 1 -- 172.21.15.5:0/2393616012 shutdown_connections 2024-09-19T06:28:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.006+0000 7f1c296e6640 1 --2- 172.21.15.5:0/2393616012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c24101340 0x7f1c2410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.006+0000 7f1c296e6640 1 --2- 172.21.15.5:0/2393616012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c241009a0 0x7f1c24100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.006+0000 7f1c296e6640 1 --2- 172.21.15.5:0/2393616012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c241069b0 0x7f1c24106d90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.006+0000 7f1c296e6640 1 -- 172.21.15.5:0/2393616012 >> 172.21.15.5:0/2393616012 conn(0x7f1c240779a0 msgr2=0x7f1c240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.006+0000 7f1c296e6640 1 -- 172.21.15.5:0/2393616012 shutdown_connections 2024-09-19T06:28:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.006+0000 7f1c296e6640 1 -- 172.21.15.5:0/2393616012 wait complete. 2024-09-19T06:28:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.007+0000 7f1c296e6640 1 Processor -- start 2024-09-19T06:28:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.007+0000 7f1c296e6640 1 -- start start 2024-09-19T06:28:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.007+0000 7f1c296e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c241009a0 0x7f1c24198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.008+0000 7f1c296e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c24101340 0x7f1c24198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.008+0000 7f1c296e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c241069b0 0x7f1c2419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.008+0000 7f1c296e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c24110ca0 con 0x7f1c24101340 2024-09-19T06:28:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.008+0000 7f1c296e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c24110b20 con 0x7f1c241009a0 2024-09-19T06:28:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.008+0000 7f1c296e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c24110e20 con 0x7f1c241069b0 2024-09-19T06:28:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.008+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c241009a0 0x7f1c24198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.008+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c241009a0 0x7f1c24198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46376/0 (socket says 172.21.15.5:46376) 2024-09-19T06:28:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.008+0000 7f1c22ffd640 1 -- 172.21.15.5:0/3536458853 learned_addr learned my addr 172.21.15.5:0/3536458853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.008+0000 7f1c237fe640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c241069b0 0x7f1c2419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.008+0000 7f1c227fc640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c24101340 0x7f1c24198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.009+0000 7f1c237fe640 1 -- 172.21.15.5:0/3536458853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c241009a0 msgr2=0x7f1c24198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.009+0000 7f1c237fe640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c241009a0 0x7f1c24198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.009+0000 7f1c237fe640 1 -- 172.21.15.5:0/3536458853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c24101340 msgr2=0x7f1c24198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.009+0000 7f1c237fe640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c24101340 0x7f1c24198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.009+0000 7f1c237fe640 1 -- 172.21.15.5:0/3536458853 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2419d330 con 0x7f1c241069b0 2024-09-19T06:28:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.009+0000 7f1c237fe640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c241069b0 0x7f1c2419cc00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009fd0 tx=0x7f1c1c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.009+0000 7f1c0bfff640 1 -- 172.21.15.5:0/3536458853 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00ce20 con 0x7f1c241069b0 2024-09-19T06:28:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.009+0000 7f1c0bfff640 1 -- 172.21.15.5:0/3536458853 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c1c0137d0 con 0x7f1c241069b0 2024-09-19T06:28:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.009+0000 7f1c0bfff640 1 -- 172.21.15.5:0/3536458853 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c005540 con 0x7f1c241069b0 2024-09-19T06:28:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.009+0000 7f1c296e6640 1 -- 172.21.15.5:0/3536458853 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2419d620 con 0x7f1c241069b0 2024-09-19T06:28:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.010+0000 7f1c296e6640 1 -- 172.21.15.5:0/3536458853 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c241024a0 con 0x7f1c241069b0 2024-09-19T06:28:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.013+0000 7f1c0bfff640 1 -- 172.21.15.5:0/3536458853 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1c1c013980 con 0x7f1c241069b0 2024-09-19T06:28:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.013+0000 7f1c2c47e740 1 Processor -- start 2024-09-19T06:28:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.013+0000 7f1c2c47e740 1 -- start start 2024-09-19T06:28:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.014+0000 7f1c2c47e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f9278ca10 0x564f9278cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.014+0000 7f1c2c47e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f9279ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.014+0000 7f1c2c47e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f927a04c0 0x564f927a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.014+0000 7f1c2c47e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f926f12f0 con 0x564f927a04c0 2024-09-19T06:28:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.014+0000 7f1c2c47e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564f92699270 con 0x564f9278d330 2024-09-19T06:28:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.014+0000 7f1c2c47e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564f926ed2b0 con 0x564f9278ca10 2024-09-19T06:28:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.014+0000 7f1c227fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f9278ca10 0x564f9278cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.014+0000 7f1c227fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f9278ca10 0x564f9278cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47338/0 (socket says 172.21.15.5:47338) 2024-09-19T06:28:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.014+0000 7f1c227fc640 1 -- 172.21.15.5:0/1408816707 learned_addr learned my addr 172.21.15.5:0/1408816707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.014+0000 7f1c227fc640 1 --2- 172.21.15.5:0/1408816707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f927a04c0 0x564f927a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.014+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/1408816707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f9279ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.015+0000 7f1c227fc640 1 -- 172.21.15.5:0/1408816707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 msgr2=0x564f9279ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.015+0000 7f1c227fc640 1 --2- 172.21.15.5:0/1408816707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f9279ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.015+0000 7f1c227fc640 1 -- 172.21.15.5:0/1408816707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f927a04c0 msgr2=0x564f927a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.015+0000 7f1c227fc640 1 --2- 172.21.15.5:0/1408816707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f927a04c0 0x564f927a28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.015+0000 7f1c227fc640 1 -- 172.21.15.5:0/1408816707 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f92698340 con 0x564f9278ca10 2024-09-19T06:28:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.015+0000 7f1c0bfff640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1c0c0a3580 0x7f1c0c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.015+0000 7f1c227fc640 1 --2- 172.21.15.5:0/1408816707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f9278ca10 0x564f9278cdf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1c18012770 tx=0x7f1c180096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df638516ffe2b347 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.015+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1c0c0a3580 0x7f1c0c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.015+0000 7f1c0bfff640 1 -- 172.21.15.5:0/3536458853 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f1c1c0d42e0 con 0x7f1c241069b0 2024-09-19T06:28:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.015+0000 7f1c097fa640 1 -- 172.21.15.5:0/1408816707 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18012cf0 con 0x564f9278ca10 2024-09-19T06:28:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.015+0000 7f1c097fa640 1 -- 172.21.15.5:0/1408816707 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c18010c80 con 0x564f9278ca10 2024-09-19T06:28:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.015+0000 7f1c097fa640 1 -- 172.21.15.5:0/1408816707 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1801eb10 con 0x564f9278ca10 2024-09-19T06:28:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.016+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1408816707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f9278ca10 msgr2=0x564f9278cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.016+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1408816707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f9278ca10 0x564f9278cdf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1c18012770 tx=0x7f1c180096e0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.016+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1c0c0a3580 0x7f1c0c0a5a40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1c14005da0 tx=0x7f1c14005d30 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.016+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1408816707 shutdown_connections 2024-09-19T06:28:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.016+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1408816707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f927a04c0 0x564f927a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.016+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1408816707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f9279ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.016+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1408816707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f9278ca10 0x564f9278cdf0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.016+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1408816707 >> 172.21.15.5:0/1408816707 conn(0x564f92789060 msgr2=0x564f92789440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.016+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1408816707 shutdown_connections 2024-09-19T06:28:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.016+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1408816707 wait complete. 2024-09-19T06:28:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c2c47e740 1 -- start start 2024-09-19T06:28:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c2c47e740 1 Processor -- start 2024-09-19T06:28:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c2c47e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 0x564f9279d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c2c47e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f9279d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c2c47e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f9279de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c2c47e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f927a3090 con 0x564f9278ca10 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c2c47e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564f9267a290 con 0x564f9278d330 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c2c47e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564f927a3210 con 0x564f927a04c0 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c2c47e740 10 client.0 _ll_register_callbacks cb 0x564f926ab140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 0x564f9279d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 0x564f9279d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47072/0 (socket says 172.21.15.5:47072) 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c22ffd640 1 -- 172.21.15.5:0/1743533436 learned_addr learned my addr 172.21.15.5:0/1743533436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c237fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f9279de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.017+0000 7f1c227fc640 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f9279d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c237fe640 1 -- 172.21.15.5:0/1743533436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 msgr2=0x564f9279d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c237fe640 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f9279d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c237fe640 1 -- 172.21.15.5:0/1743533436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 msgr2=0x564f9279d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c237fe640 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 0x564f9279d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c237fe640 1 -- 172.21.15.5:0/1743533436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f92404210 con 0x564f927a04c0 2024-09-19T06:28:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 0x564f9279d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c237fe640 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f9279de80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0d1ac0 tx=0x7f1c1c0a0650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a575e8a32a44b56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c06ff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:28:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c06ff5640 1 -- 172.21.15.5:0/1743533436 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0d4040 con 0x564f927a04c0 2024-09-19T06:28:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c06ff5640 1 -- 172.21.15.5:0/1743533436 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f1c1c0a8e30 con 0x564f927a04c0 2024-09-19T06:28:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1743533436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c18010c80 con 0x564f927a04c0 2024-09-19T06:28:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c06ff5640 1 -- 172.21.15.5:0/1743533436 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f1c1c0d1e30 con 0x564f927a04c0 2024-09-19T06:28:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c2c47e740 10 client.30928 fetch_fsmap learned FSMap version 63 2024-09-19T06:28:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.018+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1743533436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x564f924d0620 con 0x564f927a04c0 2024-09-19T06:28:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.019+0000 7f1c06ff5640 1 -- 172.21.15.5:0/1743533436 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c18010c80 con 0x564f927a04c0 2024-09-19T06:28:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.019+0000 7f1c06ff5640 1 -- 172.21.15.5:0/1743533436 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 63) ==== 8366+0+0 (secure 0 0 0) 0x7f1c1c09c3a0 con 0x564f927a04c0 2024-09-19T06:28:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.019+0000 7f1c2c47e740 10 client.30928 fetch_fsmap finished waiting for FSMap version 63 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.019+0000 7f1c2c47e740 10 client.30928 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.019+0000 7f1c2c47e740 20 client.30928 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.019+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564f927b0500 0x564f927b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.019+0000 7f1c2c47e740 4 client.30928 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.019+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1743533436 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f926fea20 con 0x564f927b0500 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.019+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564f927b0500 0x564f927b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.019+0000 7f1c06ff5640 10 client.30928 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.020+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564f927b0500 0x564f927b28f0 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84dd961d4be08f36 server_cookie=c7cfa293023994f0 in_seq=0 out_seq=0 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c06ff5640 1 -- 172.21.15.5:0/1743533436 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f926fea20 con 0x564f927b0500 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c06ff5640 10 client.30928 handle_command_reply: tid=0 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c2c47e740 1 client.30928 shutdown 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1743533436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 msgr2=0x564f9279de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f9279de80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0d1ac0 tx=0x7f1c1c0a0650 comp rx=0 tx=0).stop 2024-09-19T06:28:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1743533436 shutdown_connections 2024-09-19T06:28:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564f927b0500 0x564f927b28f0 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f9279de80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f9279d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c06ff5640 0 client.30928 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1743533436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 0x564f9279d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1743533436 >> 172.21.15.5:0/1743533436 conn(0x564f92789060 msgr2=0x564f9278ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1743533436 shutdown_connections 2024-09-19T06:28:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1743533436 wait complete. 2024-09-19T06:28:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.021+0000 7f1c2c47e740 20 client.30928 trim_cache size 0 max 16384 2024-09-19T06:28:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c2c47e740 1 Processor -- start 2024-09-19T06:28:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c2c47e740 1 -- start start 2024-09-19T06:28:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c2c47e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278ca10 0x564f9279f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c2c47e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278d330 0x564f927b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c2c47e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f927b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c2c47e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241ab670 con 0x564f9278d330 2024-09-19T06:28:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c2c47e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241ad1b0 con 0x564f9278ca10 2024-09-19T06:28:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c2c47e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241abd40 con 0x564f927a04c0 2024-09-19T06:28:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c227fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278d330 0x564f927b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c227fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278d330 0x564f927b81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47082/0 (socket says 172.21.15.5:47082) 2024-09-19T06:28:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c227fc640 1 -- 172.21.15.5:0/3959367482 learned_addr learned my addr 172.21.15.5:0/3959367482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/3959367482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278ca10 0x564f9279f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.037+0000 7f1c237fe640 1 --2- 172.21.15.5:0/3959367482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f927b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.038+0000 7f1c22ffd640 1 -- 172.21.15.5:0/3959367482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 msgr2=0x564f927b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.038+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/3959367482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f927b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.038+0000 7f1c22ffd640 1 -- 172.21.15.5:0/3959367482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278d330 msgr2=0x564f927b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.038+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/3959367482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278d330 0x564f927b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.038+0000 7f1c22ffd640 1 -- 172.21.15.5:0/3959367482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f9278d160 con 0x564f9278ca10 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.038+0000 7f1c227fc640 1 --2- 172.21.15.5:0/3959367482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278d330 0x564f927b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.038+0000 7f1c237fe640 1 --2- 172.21.15.5:0/3959367482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f927b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.038+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/3959367482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278ca10 0x564f9279f370 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x564f927b3120 tx=0x7f1c1403e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ecf8cb63d0081ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.038+0000 7f1c08ff9640 1 -- 172.21.15.5:0/3959367482 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1405c070 con 0x564f9278ca10 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.038+0000 7f1c08ff9640 1 -- 172.21.15.5:0/3959367482 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c14038040 con 0x564f9278ca10 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.038+0000 7f1c08ff9640 1 -- 172.21.15.5:0/3959367482 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14037040 con 0x564f9278ca10 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.039+0000 7f1c2c47e740 1 -- 172.21.15.5:0/3959367482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278ca10 msgr2=0x564f9279f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.039+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/3959367482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278ca10 0x564f9279f370 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x564f927b3120 tx=0x7f1c1403e000 comp rx=0 tx=0).stop 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.039+0000 7f1c2c47e740 1 -- 172.21.15.5:0/3959367482 shutdown_connections 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.039+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/3959367482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f927b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.039+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/3959367482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278d330 0x564f927b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.039+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/3959367482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278ca10 0x564f9279f370 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.039+0000 7f1c2c47e740 1 -- 172.21.15.5:0/3959367482 >> 172.21.15.5:0/3959367482 conn(0x564f92789060 msgr2=0x564f9279ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.039+0000 7f1c2c47e740 1 -- 172.21.15.5:0/3959367482 shutdown_connections 2024-09-19T06:28:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.039+0000 7f1c2c47e740 1 -- 172.21.15.5:0/3959367482 wait complete. 2024-09-19T06:28:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.039+0000 7f1c2c47e740 1 -- start start 2024-09-19T06:28:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c2c47e740 1 Processor -- start 2024-09-19T06:28:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c2c47e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 0x564f927a8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c2c47e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f92799ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c2c47e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f9279a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c2c47e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241acae0 con 0x564f9278ca10 2024-09-19T06:28:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c2c47e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241ac410 con 0x564f9278d330 2024-09-19T06:28:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c2c47e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241a9690 con 0x564f927a04c0 2024-09-19T06:28:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c2c47e740 10 client.0 _ll_register_callbacks cb 0x7f1c1c09c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c227fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f92799ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 0x564f927a8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c227fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f92799ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46412/0 (socket says 172.21.15.5:46412) 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 0x564f927a8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47096/0 (socket says 172.21.15.5:47096) 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c227fc640 1 -- 172.21.15.5:0/1365606690 learned_addr learned my addr 172.21.15.5:0/1365606690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c237fe640 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f9279a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c22ffd640 1 -- 172.21.15.5:0/1365606690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 msgr2=0x564f9279a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f9279a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c22ffd640 1 -- 172.21.15.5:0/1365606690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 msgr2=0x564f92799ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f92799ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c22ffd640 1 -- 172.21.15.5:0/1365606690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f9279ac20 con 0x564f9278ca10 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c227fc640 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f92799ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.040+0000 7f1c237fe640 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f9279a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.041+0000 7f1c22ffd640 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 0x564f927a8ad0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x564f927b3120 tx=0x7f1c14057570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a83ad7c2bd748c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.041+0000 7f1c06ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.041+0000 7f1c06ff5640 1 -- 172.21.15.5:0/1365606690 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1405c050 con 0x564f9278ca10 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.041+0000 7f1c06ff5640 1 -- 172.21.15.5:0/1365606690 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f1c14057cd0 con 0x564f9278ca10 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.041+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1365606690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c14038040 con 0x564f9278ca10 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.041+0000 7f1c06ff5640 1 -- 172.21.15.5:0/1365606690 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f1c140035f0 con 0x564f9278ca10 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.041+0000 7f1c2c47e740 10 client.21645 fetch_fsmap learned FSMap version 63 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.041+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1365606690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x564f9281b0a0 con 0x564f9278ca10 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.042+0000 7f1c06ff5640 1 -- 172.21.15.5:0/1365606690 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c14038040 con 0x564f9278ca10 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.042+0000 7f1c06ff5640 1 -- 172.21.15.5:0/1365606690 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 63) ==== 8366+0+0 (secure 0 0 0) 0x7f1c1404a040 con 0x564f9278ca10 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.042+0000 7f1c2c47e740 10 client.21645 fetch_fsmap finished waiting for FSMap version 63 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.042+0000 7f1c2c47e740 10 client.21645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.042+0000 7f1c2c47e740 20 client.21645 populate_metadata read hostname 'smithi005' 2024-09-19T06:28:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.042+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564f9281b730 0x564f9281db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.042+0000 7f1c2c47e740 4 client.21645 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:28:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.042+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1365606690 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f926f8d70 con 0x564f9281b730 2024-09-19T06:28:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.042+0000 7f1c227fc640 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564f9281b730 0x564f9281db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.043+0000 7f1c06ff5640 10 client.21645 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.043+0000 7f1c227fc640 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564f9281b730 0x564f9281db50 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f32c481bf66eb0 server_cookie=f1e1656fc2136c9f in_seq=0 out_seq=0 2024-09-19T06:28:39.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:38 smithi157 ceph-mon[35346]: pgmap v800: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.5 MiB/s wr, 457 op/s 2024-09-19T06:28:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c06ff5640 1 -- 172.21.15.5:0/1365606690 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f926f8d70 con 0x564f9281b730 2024-09-19T06:28:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c06ff5640 10 client.21645 handle_command_reply: tid=0 2024-09-19T06:28:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c2c47e740 1 client.21645 shutdown 2024-09-19T06:28:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1365606690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 msgr2=0x564f927a8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 0x564f927a8ad0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x564f927b3120 tx=0x7f1c14057570 comp rx=0 tx=0).stop 2024-09-19T06:28:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1365606690 shutdown_connections 2024-09-19T06:28:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x564f9281b730 0x564f9281db50 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f927a04c0 0x564f9279a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c06ff5640 0 client.21645 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f9278d330 0x564f92799ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c2c47e740 1 --2- 172.21.15.5:0/1365606690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f9278ca10 0x564f927a8ad0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1365606690 >> 172.21.15.5:0/1365606690 conn(0x564f92789060 msgr2=0x564f9279f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1365606690 shutdown_connections 2024-09-19T06:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c2c47e740 1 -- 172.21.15.5:0/1365606690 wait complete. 2024-09-19T06:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.150+0000 7f1c2c47e740 20 client.21645 trim_cache size 0 max 16384 2024-09-19T06:28:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-19T06:28:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-19T06:28:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-19T06:28:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-19T06:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.151+0000 7f1c296e6640 1 -- 172.21.15.5:0/3536458853 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1c0c0a3580 msgr2=0x7f1c0c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.151+0000 7f1c296e6640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1c0c0a3580 0x7f1c0c0a5a40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1c14005da0 tx=0x7f1c14005d30 comp rx=0 tx=0).stop 2024-09-19T06:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.151+0000 7f1c296e6640 1 -- 172.21.15.5:0/3536458853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c241069b0 msgr2=0x7f1c2419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.151+0000 7f1c296e6640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c241069b0 0x7f1c2419cc00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009fd0 tx=0x7f1c1c00ef10 comp rx=0 tx=0).stop 2024-09-19T06:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.151+0000 7f1c296e6640 1 -- 172.21.15.5:0/3536458853 shutdown_connections 2024-09-19T06:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.151+0000 7f1c296e6640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1c0c0a3580 0x7f1c0c0a5a40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.151+0000 7f1c296e6640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c241069b0 0x7f1c2419cc00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.151+0000 7f1c296e6640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c24101340 0x7f1c24198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.151+0000 7f1c296e6640 1 --2- 172.21.15.5:0/3536458853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c241009a0 0x7f1c24198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.151+0000 7f1c296e6640 1 -- 172.21.15.5:0/3536458853 >> 172.21.15.5:0/3536458853 conn(0x7f1c240779a0 msgr2=0x7f1c24069bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.151+0000 7f1c296e6640 1 -- 172.21.15.5:0/3536458853 shutdown_connections 2024-09-19T06:28:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-19T06:28:39.151+0000 7f1c296e6640 1 -- 172.21.15.5:0/3536458853 wait complete. 2024-09-19T06:28:39.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:39 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:28:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:28:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:28:39.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:39 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:28:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:39 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:28:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:28:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:28:39.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:39 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:28:39.934 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-19T06:28:39.934 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T06:28:40.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:39 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:28:40.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:28:40.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:28:40.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:39 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:28:40.382 DEBUG:teuthology.parallel:result is None 2024-09-19T06:28:40.382 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T06:28:40.421 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T06:28:40.422 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-19T06:28:40.435 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi005.front.sepia.ceph.com 2024-09-19T06:28:40.471 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-19T06:28:40.482 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-19T06:28:40.483 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T06:28:40.483 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-19T06:28:40.483 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-19T06:28:40.609 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-19T06:28:40.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:40 smithi005 ceph-mon[30955]: pgmap v801: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 314 op/s 2024-09-19T06:28:40.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:40 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:28:40.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:40 smithi005 ceph-mon[30955]: scrub summary: idle 2024-09-19T06:28:40.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:40 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:28:40.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T06:28:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.796+0000 7fa0f8b98640 1 -- 172.21.15.5:0/2823526602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0f4101370 msgr2=0x7fa0f410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:40.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.796+0000 7fa0f8b98640 1 --2- 172.21.15.5:0/2823526602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0f4101370 0x7fa0f410d8a0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009cd0 tx=0x7fa0ec02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:28:40.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.796+0000 7fa0f8b98640 1 -- 172.21.15.5:0/2823526602 shutdown_connections 2024-09-19T06:28:40.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.796+0000 7fa0f8b98640 1 --2- 172.21.15.5:0/2823526602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0f4101370 0x7fa0f410d8a0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:40.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.796+0000 7fa0f8b98640 1 --2- 172.21.15.5:0/2823526602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f41009d0 0x7fa0f4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:40.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.796+0000 7fa0f8b98640 1 --2- 172.21.15.5:0/2823526602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0f41069e0 0x7fa0f4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:40.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.796+0000 7fa0f8b98640 1 -- 172.21.15.5:0/2823526602 >> 172.21.15.5:0/2823526602 conn(0x7fa0f40fc6b0 msgr2=0x7fa0f40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:40.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.797+0000 7fa0f8b98640 1 -- 172.21.15.5:0/2823526602 shutdown_connections 2024-09-19T06:28:40.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.797+0000 7fa0f8b98640 1 -- 172.21.15.5:0/2823526602 wait complete. 2024-09-19T06:28:40.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.797+0000 7fa0f8b98640 1 Processor -- start 2024-09-19T06:28:40.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.797+0000 7fa0f8b98640 1 -- start start 2024-09-19T06:28:40.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.797+0000 7fa0f8b98640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f41009d0 0x7fa0f4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.797+0000 7fa0f8b98640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0f4101370 0x7fa0f41996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.797+0000 7fa0f8b98640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0f41069e0 0x7fa0f41a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.797+0000 7fa0f8b98640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f4110cd0 con 0x7fa0f41069e0 2024-09-19T06:28:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.797+0000 7fa0f8b98640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f4110b50 con 0x7fa0f41009d0 2024-09-19T06:28:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.797+0000 7fa0f8b98640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f4110e50 con 0x7fa0f4101370 2024-09-19T06:28:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0f41069e0 0x7fa0f41a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0f41069e0 0x7fa0f41a41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47110/0 (socket says 172.21.15.5:47110) 2024-09-19T06:28:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f2d9d640 1 -- 172.21.15.5:0/3357894310 learned_addr learned my addr 172.21.15.5:0/3357894310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f1d9b640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0f4101370 0x7fa0f41996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f259c640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f41009d0 0x7fa0f4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f2d9d640 1 -- 172.21.15.5:0/3357894310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0f4101370 msgr2=0x7fa0f41996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f2d9d640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0f4101370 0x7fa0f41996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:40.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f2d9d640 1 -- 172.21.15.5:0/3357894310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f41009d0 msgr2=0x7fa0f4199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:40.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f2d9d640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f41009d0 0x7fa0f4199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:40.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f2d9d640 1 -- 172.21.15.5:0/3357894310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f41a4720 con 0x7fa0f41069e0 2024-09-19T06:28:40.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f259c640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f41009d0 0x7fa0f4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f1d9b640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0f4101370 0x7fa0f41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:28:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f2d9d640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0f41069e0 0x7fa0f41a41e0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec033040 tx=0x7fa0ec009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0e37fe640 1 -- 172.21.15.5:0/3357894310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec004040 con 0x7fa0f41069e0 2024-09-19T06:28:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0e37fe640 1 -- 172.21.15.5:0/3357894310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0ec004690 con 0x7fa0f41069e0 2024-09-19T06:28:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f8b98640 1 -- 172.21.15.5:0/3357894310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f41a48f0 con 0x7fa0f41069e0 2024-09-19T06:28:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0e37fe640 1 -- 172.21.15.5:0/3357894310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec030da0 con 0x7fa0f41069e0 2024-09-19T06:28:40.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.798+0000 7fa0f8b98640 1 -- 172.21.15.5:0/3357894310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f41a4db0 con 0x7fa0f41069e0 2024-09-19T06:28:40.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.799+0000 7fa0f8b98640 1 -- 172.21.15.5:0/3357894310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f4100e30 con 0x7fa0f41069e0 2024-09-19T06:28:40.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.801+0000 7fa0e37fe640 1 -- 172.21.15.5:0/3357894310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa0ec0041e0 con 0x7fa0f41069e0 2024-09-19T06:28:40.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.802+0000 7fa0e37fe640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa0dc0a33b0 0x7fa0dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:40.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.802+0000 7fa0e37fe640 1 -- 172.21.15.5:0/3357894310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fa0ec02e030 con 0x7fa0f41069e0 2024-09-19T06:28:40.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.802+0000 7fa0f259c640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa0dc0a33b0 0x7fa0dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:40.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.803+0000 7fa0f259c640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa0dc0a33b0 0x7fa0dc0a5870 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009f40 tx=0x7fa0e4005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:40.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:40.806+0000 7fa0e37fe640 1 -- 172.21.15.5:0/3357894310 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa0ec0bd880 con 0x7fa0f41069e0 2024-09-19T06:28:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:40 smithi049 ceph-mon[34640]: pgmap v801: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 314 op/s 2024-09-19T06:28:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:40 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:28:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:40 smithi049 ceph-mon[34640]: scrub summary: idle 2024-09-19T06:28:40.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:40 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:28:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.006+0000 7fa0f8b98640 1 -- 172.21.15.5:0/3357894310 --> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa0f4062630 con 0x7fa0dc0a33b0 2024-09-19T06:28:41.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.016+0000 7fa0e37fe640 1 -- 172.21.15.5:0/3357894310 <== mgr.15240 v2:172.21.15.5:6800/2837801016 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fa0f4062630 con 0x7fa0dc0a33b0 2024-09-19T06:28:41.018 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T06:03:30", "first": "2024-09-19T06:04:00", "last": "2024-09-19T06:28:00", "last_pruned": "2024-09-19T06:28:00", "created_count": 25, "pruned_count": 19, "active": true} 2024-09-19T06:28:41.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.018+0000 7fa0f8b98640 1 -- 172.21.15.5:0/3357894310 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa0dc0a33b0 msgr2=0x7fa0dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:41.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.018+0000 7fa0f8b98640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa0dc0a33b0 0x7fa0dc0a5870 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009f40 tx=0x7fa0e4005f90 comp rx=0 tx=0).stop 2024-09-19T06:28:41.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.018+0000 7fa0f8b98640 1 -- 172.21.15.5:0/3357894310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0f41069e0 msgr2=0x7fa0f41a41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:41.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.018+0000 7fa0f8b98640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0f41069e0 0x7fa0f41a41e0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec033040 tx=0x7fa0ec009360 comp rx=0 tx=0).stop 2024-09-19T06:28:41.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.018+0000 7fa0f8b98640 1 -- 172.21.15.5:0/3357894310 shutdown_connections 2024-09-19T06:28:41.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.018+0000 7fa0f8b98640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0f41069e0 0x7fa0f41a41e0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.018+0000 7fa0f8b98640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fa0dc0a33b0 0x7fa0dc0a5870 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.018+0000 7fa0f8b98640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0f4101370 0x7fa0f41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.018+0000 7fa0f8b98640 1 --2- 172.21.15.5:0/3357894310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f41009d0 0x7fa0f4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.019+0000 7fa0f8b98640 1 -- 172.21.15.5:0/3357894310 >> 172.21.15.5:0/3357894310 conn(0x7fa0f40fc6b0 msgr2=0x7fa0f40fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:41.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.019+0000 7fa0f8b98640 1 -- 172.21.15.5:0/3357894310 shutdown_connections 2024-09-19T06:28:41.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.019+0000 7fa0f8b98640 1 -- 172.21.15.5:0/3357894310 wait complete. 2024-09-19T06:28:41.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T06:28:41.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:40 smithi157 ceph-mon[35346]: pgmap v801: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 314 op/s 2024-09-19T06:28:41.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:40 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:28:41.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:40 smithi157 ceph-mon[35346]: scrub summary: idle 2024-09-19T06:28:41.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:40 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:28:41.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.136+0000 7fd4f92c9640 1 -- 172.21.15.5:0/2470701600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f41035c0 msgr2=0x7fd4f410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.136+0000 7fd4f92c9640 1 --2- 172.21.15.5:0/2470701600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f41035c0 0x7fd4f410baa0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009900 tx=0x7fd4dc02a480 comp rx=0 tx=0).stop 2024-09-19T06:28:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.136+0000 7fd4f92c9640 1 -- 172.21.15.5:0/2470701600 shutdown_connections 2024-09-19T06:28:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.136+0000 7fd4f92c9640 1 --2- 172.21.15.5:0/2470701600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4f410c0f0 0x7fd4f410e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.136+0000 7fd4f92c9640 1 --2- 172.21.15.5:0/2470701600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f41035c0 0x7fd4f410baa0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.136+0000 7fd4f92c9640 1 --2- 172.21.15.5:0/2470701600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4f4102ca0 0x7fd4f4103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.136+0000 7fd4f92c9640 1 -- 172.21.15.5:0/2470701600 >> 172.21.15.5:0/2470701600 conn(0x7fd4f40fc6b0 msgr2=0x7fd4f40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.136+0000 7fd4f92c9640 1 -- 172.21.15.5:0/2470701600 shutdown_connections 2024-09-19T06:28:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.136+0000 7fd4f92c9640 1 -- 172.21.15.5:0/2470701600 wait complete. 2024-09-19T06:28:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.136+0000 7fd4f92c9640 1 Processor -- start 2024-09-19T06:28:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f92c9640 1 -- start start 2024-09-19T06:28:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f92c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4f4102ca0 0x7fd4f4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f92c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f41035c0 0x7fd4f4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f92c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4f410c0f0 0x7fd4f419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f92c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4111dd0 con 0x7fd4f41035c0 2024-09-19T06:28:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f92c9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4111c50 con 0x7fd4f4102ca0 2024-09-19T06:28:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f92c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4111f50 con 0x7fd4f410c0f0 2024-09-19T06:28:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f41035c0 0x7fd4f4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f41035c0 0x7fd4f4198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47130/0 (socket says 172.21.15.5:47130) 2024-09-19T06:28:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f27fc640 1 -- 172.21.15.5:0/9821436 learned_addr learned my addr 172.21.15.5:0/9821436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:28:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f2ffd640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4f4102ca0 0x7fd4f4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f37fe640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4f410c0f0 0x7fd4f419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f27fc640 1 -- 172.21.15.5:0/9821436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4f410c0f0 msgr2=0x7fd4f419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f27fc640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4f410c0f0 0x7fd4f419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f27fc640 1 -- 172.21.15.5:0/9821436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4f4102ca0 msgr2=0x7fd4f4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f27fc640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4f4102ca0 0x7fd4f4198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f27fc640 1 -- 172.21.15.5:0/9821436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f419d310 con 0x7fd4f41035c0 2024-09-19T06:28:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.137+0000 7fd4f37fe640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4f410c0f0 0x7fd4f419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:28:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.138+0000 7fd4f27fc640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f41035c0 0x7fd4f4198860 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005ab0 tx=0x7fd4dc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.138+0000 7fd4ebfff640 1 -- 172.21.15.5:0/9821436 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc03f070 con 0x7fd4f41035c0 2024-09-19T06:28:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.138+0000 7fd4ebfff640 1 -- 172.21.15.5:0/9821436 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4dc031b60 con 0x7fd4f41035c0 2024-09-19T06:28:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.138+0000 7fd4ebfff640 1 -- 172.21.15.5:0/9821436 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc035030 con 0x7fd4f41035c0 2024-09-19T06:28:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.138+0000 7fd4f92c9640 1 -- 172.21.15.5:0/9821436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f419d5a0 con 0x7fd4f41035c0 2024-09-19T06:28:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.138+0000 7fd4f92c9640 1 -- 172.21.15.5:0/9821436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f41a4e90 con 0x7fd4f41035c0 2024-09-19T06:28:41.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.139+0000 7fd4f92c9640 1 -- 172.21.15.5:0/9821436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f4103080 con 0x7fd4f41035c0 2024-09-19T06:28:41.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.140+0000 7fd4ebfff640 1 -- 172.21.15.5:0/9821436 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd4dc042070 con 0x7fd4f41035c0 2024-09-19T06:28:41.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.141+0000 7fd4ebfff640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd4e00a33b0 0x7fd4e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:28:41.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.141+0000 7fd4f2ffd640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd4e00a33b0 0x7fd4e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:28:41.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.141+0000 7fd4ebfff640 1 -- 172.21.15.5:0/9821436 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fd4dc0f0550 con 0x7fd4f41035c0 2024-09-19T06:28:41.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.141+0000 7fd4f2ffd640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd4e00a33b0 0x7fd4e00a5870 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009b60 tx=0x7fd4e4005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:28:41.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.145+0000 7fd4ebfff640 1 -- 172.21.15.5:0/9821436 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd4dc0bceb0 con 0x7fd4f41035c0 2024-09-19T06:28:41.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.349+0000 7fd4f92c9640 1 -- 172.21.15.5:0/9821436 --> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd4f4107170 con 0x7fd4e00a33b0 2024-09-19T06:28:41.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.359+0000 7fd4ebfff640 1 -- 172.21.15.5:0/9821436 <== mgr.15240 v2:172.21.15.5:6800/2837801016 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd4f4107170 con 0x7fd4e00a33b0 2024-09-19T06:28:41.360 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2024-09-19T06:28:41.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.361+0000 7fd4f92c9640 1 -- 172.21.15.5:0/9821436 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd4e00a33b0 msgr2=0x7fd4e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:41.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.361+0000 7fd4f92c9640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd4e00a33b0 0x7fd4e00a5870 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009b60 tx=0x7fd4e4005f90 comp rx=0 tx=0).stop 2024-09-19T06:28:41.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.361+0000 7fd4f92c9640 1 -- 172.21.15.5:0/9821436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f41035c0 msgr2=0x7fd4f4198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:28:41.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.361+0000 7fd4f92c9640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f41035c0 0x7fd4f4198860 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005ab0 tx=0x7fd4dc003c10 comp rx=0 tx=0).stop 2024-09-19T06:28:41.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.361+0000 7fd4f92c9640 1 -- 172.21.15.5:0/9821436 shutdown_connections 2024-09-19T06:28:41.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.361+0000 7fd4f92c9640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd4e00a33b0 0x7fd4e00a5870 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.361+0000 7fd4f92c9640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4f410c0f0 0x7fd4f419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.361+0000 7fd4f92c9640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f41035c0 0x7fd4f4198860 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.361+0000 7fd4f92c9640 1 --2- 172.21.15.5:0/9821436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4f4102ca0 0x7fd4f4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:28:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.361+0000 7fd4f92c9640 1 -- 172.21.15.5:0/9821436 >> 172.21.15.5:0/9821436 conn(0x7fd4f40fc6b0 msgr2=0x7fd4f40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:28:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.361+0000 7fd4f92c9640 1 -- 172.21.15.5:0/9821436 shutdown_connections 2024-09-19T06:28:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:28:41.361+0000 7fd4f92c9640 1 -- 172.21.15.5:0/9821436 wait complete. 2024-09-19T06:28:41.374 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T06:28:41.384 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T06:28:41.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:41 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:28:41.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:41 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:28:42.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:41 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:28:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:42 smithi005 ceph-mon[30955]: pgmap v802: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 314 op/s 2024-09-19T06:28:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:42 smithi005 ceph-mon[30955]: from='client.21651 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:28:42.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:42 smithi005 ceph-mon[30955]: from='client.21657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:28:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:42 smithi049 ceph-mon[34640]: pgmap v802: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 314 op/s 2024-09-19T06:28:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:42 smithi049 ceph-mon[34640]: from='client.21651 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:28:42.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:42 smithi049 ceph-mon[34640]: from='client.21657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:28:43.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:42 smithi157 ceph-mon[35346]: pgmap v802: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 314 op/s 2024-09-19T06:28:43.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:42 smithi157 ceph-mon[35346]: from='client.21651 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:28:43.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:42 smithi157 ceph-mon[35346]: from='client.21657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:28:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:44 smithi005 ceph-mon[30955]: pgmap v803: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.0 MiB/s wr, 481 op/s 2024-09-19T06:28:44.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:44 smithi005 ceph-mon[30955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:28:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:44 smithi049 ceph-mon[34640]: pgmap v803: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.0 MiB/s wr, 481 op/s 2024-09-19T06:28:44.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:44 smithi049 ceph-mon[34640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:28:45.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:44 smithi157 ceph-mon[35346]: pgmap v803: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.0 MiB/s wr, 481 op/s 2024-09-19T06:28:45.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:44 smithi157 ceph-mon[35346]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:28:46.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:46 smithi005 ceph-mon[30955]: pgmap v804: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 297 op/s 2024-09-19T06:28:46.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:46 smithi049 ceph-mon[34640]: pgmap v804: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 297 op/s 2024-09-19T06:28:47.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:46 smithi157 ceph-mon[35346]: pgmap v804: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 297 op/s 2024-09-19T06:28:48.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:48 smithi049 ceph-mon[34640]: pgmap v805: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.3 MiB/s wr, 306 op/s 2024-09-19T06:28:49.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:48 smithi157 ceph-mon[35346]: pgmap v805: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.3 MiB/s wr, 306 op/s 2024-09-19T06:28:49.124 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:48 smithi005 ceph-mon[30955]: pgmap v805: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.3 MiB/s wr, 306 op/s 2024-09-19T06:28:50.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:50 smithi049 ceph-mon[34640]: pgmap v806: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-19T06:28:51.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:50 smithi157 ceph-mon[35346]: pgmap v806: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-19T06:28:51.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:50 smithi005 ceph-mon[30955]: pgmap v806: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-19T06:28:52.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:51 smithi157 ceph-mon[35346]: pgmap v807: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-19T06:28:52.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:51 smithi005 ceph-mon[30955]: pgmap v807: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-19T06:28:52.146 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:51 smithi049 ceph-mon[34640]: pgmap v807: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-19T06:28:54.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:54 smithi005 ceph-mon[30955]: pgmap v808: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-09-19T06:28:54.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:54 smithi049 ceph-mon[34640]: pgmap v808: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-09-19T06:28:54.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:54 smithi157 ceph-mon[35346]: pgmap v808: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-09-19T06:28:56.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:56 smithi005 ceph-mon[30955]: pgmap v809: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-09-19T06:28:56.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:56 smithi049 ceph-mon[34640]: pgmap v809: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-09-19T06:28:56.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:56 smithi157 ceph-mon[35346]: pgmap v809: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-09-19T06:28:58.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:28:58 smithi005 ceph-mon[30955]: pgmap v810: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-19T06:28:58.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:28:58 smithi049 ceph-mon[34640]: pgmap v810: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-19T06:28:58.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:28:58 smithi157 ceph-mon[35346]: pgmap v810: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-19T06:29:00.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:29:00.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:00 smithi005 ceph-mon[30955]: pgmap v811: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-19T06:29:00.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:00 smithi049 ceph-mon[34640]: pgmap v811: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-19T06:29:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:00.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:00 smithi157 ceph-mon[35346]: pgmap v811: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-19T06:29:01.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.963+0000 7f5e01c81640 1 -- 172.21.15.5:0/3583336445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dfc103650 msgr2=0x7f5dfc10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:01.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.963+0000 7f5e01c81640 1 --2- 172.21.15.5:0/3583336445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dfc103650 0x7f5dfc10bb40 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009900 tx=0x7f5df002a480 comp rx=0 tx=0).stop 2024-09-19T06:29:01.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.963+0000 7f5e01c81640 1 -- 172.21.15.5:0/3583336445 shutdown_connections 2024-09-19T06:29:01.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.963+0000 7f5e01c81640 1 --2- 172.21.15.5:0/3583336445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dfc10c160 0x7f5dfc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.963+0000 7f5e01c81640 1 --2- 172.21.15.5:0/3583336445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dfc103650 0x7f5dfc10bb40 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.963+0000 7f5e01c81640 1 --2- 172.21.15.5:0/3583336445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dfc102d30 0x7f5dfc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.963+0000 7f5e01c81640 1 -- 172.21.15.5:0/3583336445 >> 172.21.15.5:0/3583336445 conn(0x7f5dfc0fc7d0 msgr2=0x7f5dfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:01.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.964+0000 7f5e01c81640 1 -- 172.21.15.5:0/3583336445 shutdown_connections 2024-09-19T06:29:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.964+0000 7f5e01c81640 1 -- 172.21.15.5:0/3583336445 wait complete. 2024-09-19T06:29:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.964+0000 7f5e01c81640 1 Processor -- start 2024-09-19T06:29:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.964+0000 7f5e01c81640 1 -- start start 2024-09-19T06:29:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5e01c81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dfc102d30 0x7f5dfc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5e01c81640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dfc103650 0x7f5dfc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5e01c81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dfc10c160 0x7f5dfc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5e01c81640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc111e10 con 0x7f5dfc102d30 2024-09-19T06:29:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5e01c81640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc111c90 con 0x7f5dfc10c160 2024-09-19T06:29:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5e01c81640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc111f90 con 0x7f5dfc103650 2024-09-19T06:29:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5dfb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dfc102d30 0x7f5dfc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5dfb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dfc102d30 0x7f5dfc198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39630/0 (socket says 172.21.15.5:39630) 2024-09-19T06:29:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5dfb7fe640 1 -- 172.21.15.5:0/2703371722 learned_addr learned my addr 172.21.15.5:0/2703371722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5dfaffd640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dfc103650 0x7f5dfc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5dfbfff640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dfc10c160 0x7f5dfc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5dfaffd640 1 -- 172.21.15.5:0/2703371722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dfc10c160 msgr2=0x7f5dfc19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5dfaffd640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dfc10c160 0x7f5dfc19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5dfaffd640 1 -- 172.21.15.5:0/2703371722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dfc102d30 msgr2=0x7f5dfc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5dfaffd640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dfc102d30 0x7f5dfc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5dfaffd640 1 -- 172.21.15.5:0/2703371722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc19d320 con 0x7f5dfc103650 2024-09-19T06:29:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.965+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dfc102d30 0x7f5dfc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:29:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.966+0000 7f5dfaffd640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dfc103650 0x7f5dfc1988b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5df002a9a0 tx=0x7f5df0031970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.966+0000 7f5df8ff9640 1 -- 172.21.15.5:0/2703371722 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df003f030 con 0x7f5dfc103650 2024-09-19T06:29:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.966+0000 7f5df8ff9640 1 -- 172.21.15.5:0/2703371722 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5df0004410 con 0x7f5dfc103650 2024-09-19T06:29:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.966+0000 7f5df8ff9640 1 -- 172.21.15.5:0/2703371722 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0035070 con 0x7f5dfc103650 2024-09-19T06:29:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.966+0000 7f5e01c81640 1 -- 172.21.15.5:0/2703371722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dfc19d5b0 con 0x7f5dfc103650 2024-09-19T06:29:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.966+0000 7f5e01c81640 1 -- 172.21.15.5:0/2703371722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dfc19da40 con 0x7f5dfc103650 2024-09-19T06:29:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.969+0000 7f5df8ff9640 1 -- 172.21.15.5:0/2703371722 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5df0042030 con 0x7f5dfc103650 2024-09-19T06:29:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.969+0000 7f5e0405a740 1 Processor -- start 2024-09-19T06:29:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.969+0000 7f5e0405a740 1 -- start start 2024-09-19T06:29:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.969+0000 7f5e0405a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5b2380 0x55634a5b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.969+0000 7f5e0405a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5c1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5e0405a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5c1a90 0x55634a5c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5e0405a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55634a5182e0 con 0x55634a5b2ca0 2024-09-19T06:29:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5e0405a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55634a4bfce0 con 0x55634a5c1a90 2024-09-19T06:29:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5e0405a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55634a5142a0 con 0x55634a5b2380 2024-09-19T06:29:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5dfb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5c1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5dfb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5c1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39646/0 (socket says 172.21.15.5:39646) 2024-09-19T06:29:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5dfb7fe640 1 -- 172.21.15.5:0/4122230329 learned_addr learned my addr 172.21.15.5:0/4122230329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5dfaffd640 1 --2- 172.21.15.5:0/4122230329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5c1a90 0x55634a5c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5dfbfff640 1 --2- 172.21.15.5:0/4122230329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5b2380 0x55634a5b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5df8ff9640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5de40a3430 0x7f5de40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5dfb7fe640 1 -- 172.21.15.5:0/4122230329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5b2380 msgr2=0x55634a5b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/4122230329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5b2380 0x55634a5b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5dfb7fe640 1 -- 172.21.15.5:0/4122230329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5c1a90 msgr2=0x55634a5c3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/4122230329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5c1a90 0x55634a5c3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5dfb7fe640 1 -- 172.21.15.5:0/4122230329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55634a4a4440 con 0x55634a5b2ca0 2024-09-19T06:29:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/4122230329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5c1550 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f5dec011650 tx=0x7f5dec009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbce249f065764ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.970+0000 7f5df8ff9640 1 -- 172.21.15.5:0/2703371722 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f5df00eff30 con 0x7f5dfc103650 2024-09-19T06:29:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.971+0000 7f5dfbfff640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5de40a3430 0x7f5de40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.971+0000 7f5de1ffb640 1 -- 172.21.15.5:0/4122230329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec01e070 con 0x55634a5b2ca0 2024-09-19T06:29:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.971+0000 7f5de1ffb640 1 -- 172.21.15.5:0/4122230329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dec00fb70 con 0x55634a5b2ca0 2024-09-19T06:29:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.971+0000 7f5de1ffb640 1 -- 172.21.15.5:0/4122230329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec019c00 con 0x55634a5b2ca0 2024-09-19T06:29:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.971+0000 7f5e0405a740 1 -- 172.21.15.5:0/4122230329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 msgr2=0x55634a5c1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.971+0000 7f5dfbfff640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5de40a3430 0x7f5de40a58f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5df40094d0 tx=0x7f5df4004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.971+0000 7f5e0405a740 1 --2- 172.21.15.5:0/4122230329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5c1550 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f5dec011650 tx=0x7f5dec009f90 comp rx=0 tx=0).stop 2024-09-19T06:29:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.972+0000 7f5e0405a740 1 -- 172.21.15.5:0/4122230329 shutdown_connections 2024-09-19T06:29:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.972+0000 7f5e0405a740 1 --2- 172.21.15.5:0/4122230329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5c1a90 0x55634a5c3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.972+0000 7f5e0405a740 1 --2- 172.21.15.5:0/4122230329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5c1550 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.972+0000 7f5e0405a740 1 --2- 172.21.15.5:0/4122230329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5b2380 0x55634a5b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.972+0000 7f5e0405a740 1 -- 172.21.15.5:0/4122230329 >> 172.21.15.5:0/4122230329 conn(0x55634a5aea30 msgr2=0x55634a5aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.972+0000 7f5e0405a740 1 -- 172.21.15.5:0/4122230329 shutdown_connections 2024-09-19T06:29:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.972+0000 7f5e0405a740 1 -- 172.21.15.5:0/4122230329 wait complete. 2024-09-19T06:29:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.972+0000 7f5e0405a740 1 -- start start 2024-09-19T06:29:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.972+0000 7f5e0405a740 1 Processor -- start 2024-09-19T06:29:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5e0405a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2380 0x55634a5c1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5e0405a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5b2ca0 0x55634a5be6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5e0405a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5c1a90 0x55634a5bec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5e0405a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55634a5c4660 con 0x55634a5b2380 2024-09-19T06:29:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5e0405a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55634a4d3050 con 0x55634a5c1a90 2024-09-19T06:29:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5e0405a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55634a5c47e0 con 0x55634a5b2ca0 2024-09-19T06:29:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5e0405a740 10 client.0 _ll_register_callbacks cb 0x55634a513160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:29:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5dfb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2380 0x55634a5c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5dfb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2380 0x55634a5c1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39652/0 (socket says 172.21.15.5:39652) 2024-09-19T06:29:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5dfb7fe640 1 -- 172.21.15.5:0/1324880530 learned_addr learned my addr 172.21.15.5:0/1324880530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5dfaffd640 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5b2ca0 0x55634a5be6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5dfbfff640 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5c1a90 0x55634a5bec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5dfb7fe640 1 -- 172.21.15.5:0/1324880530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5b2ca0 msgr2=0x55634a5be6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5b2ca0 0x55634a5be6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5dfb7fe640 1 -- 172.21.15.5:0/1324880530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5c1a90 msgr2=0x55634a5bec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5c1a90 0x55634a5bec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5dfb7fe640 1 -- 172.21.15.5:0/1324880530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55634a2893c0 con 0x55634a5b2380 2024-09-19T06:29:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5dfbfff640 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5c1a90 0x55634a5bec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2380 0x55634a5c1410 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f5dec019860 tx=0x7f5dec009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3925804eb598e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.973+0000 7f5ddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:29:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.974+0000 7f5ddaffd640 1 -- 172.21.15.5:0/1324880530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec01e040 con 0x55634a5b2380 2024-09-19T06:29:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.974+0000 7f5ddaffd640 1 -- 172.21.15.5:0/1324880530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f5dec00f950 con 0x55634a5b2380 2024-09-19T06:29:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.974+0000 7f5e0405a740 1 -- 172.21.15.5:0/1324880530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dec00fb70 con 0x55634a5b2380 2024-09-19T06:29:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.974+0000 7f5ddaffd640 1 -- 172.21.15.5:0/1324880530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f5dec004040 con 0x55634a5b2380 2024-09-19T06:29:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.974+0000 7f5e0405a740 10 client.21669 fetch_fsmap learned FSMap version 66 2024-09-19T06:29:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.974+0000 7f5e0405a740 1 -- 172.21.15.5:0/1324880530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55634a5d0090 con 0x55634a5b2380 2024-09-19T06:29:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.974+0000 7f5ddaffd640 1 -- 172.21.15.5:0/1324880530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dec00fb70 con 0x55634a5b2380 2024-09-19T06:29:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.974+0000 7f5ddaffd640 1 -- 172.21.15.5:0/1324880530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 8394+0+0 (secure 0 0 0) 0x7f5dec003880 con 0x55634a5b2380 2024-09-19T06:29:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.974+0000 7f5e0405a740 10 client.21669 fetch_fsmap finished waiting for FSMap version 66 2024-09-19T06:29:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.974+0000 7f5e0405a740 10 client.21669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:29:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.975+0000 7f5e0405a740 20 client.21669 populate_metadata read hostname 'smithi005' 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.975+0000 7f5e0405a740 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55634a5d1490 0x55634a5d3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.975+0000 7f5e0405a740 4 client.21669 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.975+0000 7f5e0405a740 1 -- 172.21.15.5:0/1324880530 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55634a525850 con 0x55634a5d1490 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.975+0000 7f5dfaffd640 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55634a5d1490 0x55634a5d3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.975+0000 7f5dfaffd640 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55634a5d1490 0x55634a5d3880 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c747e08862afdc36 server_cookie=45fc95907b8d7ba8 in_seq=0 out_seq=0 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.975+0000 7f5ddaffd640 10 client.21669 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5ddaffd640 1 -- 172.21.15.5:0/1324880530 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55634a525850 con 0x55634a5d1490 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5ddaffd640 10 client.21669 handle_command_reply: tid=0 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5e0405a740 1 client.21669 shutdown 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5e0405a740 1 -- 172.21.15.5:0/1324880530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2380 msgr2=0x55634a5c1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5e0405a740 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2380 0x55634a5c1410 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f5dec019860 tx=0x7f5dec009ef0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5e0405a740 1 -- 172.21.15.5:0/1324880530 shutdown_connections 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5e0405a740 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55634a5d1490 0x55634a5d3880 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5e0405a740 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5c1a90 0x55634a5bec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5e0405a740 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5b2ca0 0x55634a5be6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5e0405a740 1 --2- 172.21.15.5:0/1324880530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2380 0x55634a5c1410 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5ddaffd640 0 client.21669 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5e0405a740 1 -- 172.21.15.5:0/1324880530 >> 172.21.15.5:0/1324880530 conn(0x55634a5aea30 msgr2=0x55634a5b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5e0405a740 1 -- 172.21.15.5:0/1324880530 shutdown_connections 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5e0405a740 1 -- 172.21.15.5:0/1324880530 wait complete. 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.976+0000 7f5e0405a740 20 client.21669 trim_cache size 0 max 16384 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5e0405a740 1 Processor -- start 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5e0405a740 1 -- start start 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5e0405a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5b2380 0x55634a5cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5e0405a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5d9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5e0405a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5c1a90 0x55634a5d9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5e0405a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc072330 con 0x55634a5b2ca0 2024-09-19T06:29:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5e0405a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc073f30 con 0x55634a5b2380 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5e0405a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc072a30 con 0x55634a5c1a90 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5dfaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5dfaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5d9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39658/0 (socket says 172.21.15.5:39658) 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5dfaffd640 1 -- 172.21.15.5:0/348111127 learned_addr learned my addr 172.21.15.5:0/348111127 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/348111127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5b2380 0x55634a5cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.994+0000 7f5dfbfff640 1 --2- 172.21.15.5:0/348111127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5c1a90 0x55634a5d9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.995+0000 7f5dfb7fe640 1 -- 172.21.15.5:0/348111127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5c1a90 msgr2=0x55634a5d9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.995+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/348111127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5c1a90 0x55634a5d9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.995+0000 7f5dfb7fe640 1 -- 172.21.15.5:0/348111127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 msgr2=0x55634a5d9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.995+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/348111127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5d9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.995+0000 7f5dfb7fe640 1 -- 172.21.15.5:0/348111127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55634a5ae4d0 con 0x55634a5b2380 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.995+0000 7f5dfaffd640 1 --2- 172.21.15.5:0/348111127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5d9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:29:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.995+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/348111127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5b2380 0x55634a5cd900 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5dec003c20 tx=0x7f5dec005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a43ca72c8344b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.995+0000 7f5de17fa640 1 -- 172.21.15.5:0/348111127 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec015070 con 0x55634a5b2380 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.995+0000 7f5de17fa640 1 -- 172.21.15.5:0/348111127 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dec038070 con 0x55634a5b2380 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.995+0000 7f5de17fa640 1 -- 172.21.15.5:0/348111127 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec005250 con 0x55634a5b2380 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.996+0000 7f5e0405a740 1 -- 172.21.15.5:0/348111127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5b2380 msgr2=0x55634a5cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.996+0000 7f5e0405a740 1 --2- 172.21.15.5:0/348111127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5b2380 0x55634a5cd900 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5dec003c20 tx=0x7f5dec005d30 comp rx=0 tx=0).stop 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.996+0000 7f5e0405a740 1 -- 172.21.15.5:0/348111127 shutdown_connections 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.996+0000 7f5e0405a740 1 --2- 172.21.15.5:0/348111127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5c1a90 0x55634a5d9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.996+0000 7f5e0405a740 1 --2- 172.21.15.5:0/348111127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5d9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.996+0000 7f5e0405a740 1 --2- 172.21.15.5:0/348111127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5b2380 0x55634a5cd900 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.996+0000 7f5e0405a740 1 -- 172.21.15.5:0/348111127 >> 172.21.15.5:0/348111127 conn(0x55634a5aea30 msgr2=0x55634a5bcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.996+0000 7f5e0405a740 1 -- 172.21.15.5:0/348111127 shutdown_connections 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.996+0000 7f5e0405a740 1 -- 172.21.15.5:0/348111127 wait complete. 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.996+0000 7f5e0405a740 1 -- start start 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5e0405a740 1 Processor -- start 2024-09-19T06:29:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5e0405a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5b2380 0x55634a5c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5e0405a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5e0405a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5c1a90 0x55634a5c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5e0405a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc073860 con 0x55634a5b2ca0 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5e0405a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc073160 con 0x55634a5b2380 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5e0405a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc070330 con 0x55634a5c1a90 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5e0405a740 10 client.0 _ll_register_callbacks cb 0x7f5dec01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5dfaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5dfaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39662/0 (socket says 172.21.15.5:39662) 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5dfb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5b2380 0x55634a5c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5dfaffd640 1 -- 172.21.15.5:0/1643470294 learned_addr learned my addr 172.21.15.5:0/1643470294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5dfbfff640 1 --2- 172.21.15.5:0/1643470294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5c1a90 0x55634a5c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5dfaffd640 1 -- 172.21.15.5:0/1643470294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5c1a90 msgr2=0x55634a5c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5dfaffd640 1 --2- 172.21.15.5:0/1643470294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5c1a90 0x55634a5c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5dfaffd640 1 -- 172.21.15.5:0/1643470294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5b2380 msgr2=0x55634a5c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5dfaffd640 1 --2- 172.21.15.5:0/1643470294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5b2380 0x55634a5c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.997+0000 7f5dfaffd640 1 -- 172.21.15.5:0/1643470294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55634a5c5bd0 con 0x55634a5b2ca0 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5dfaffd640 1 --2- 172.21.15.5:0/1643470294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5ca360 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5df00bc120 tx=0x7f5df00f2380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253f8ececa739cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5ddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5ddaffd640 1 -- 172.21.15.5:0/1643470294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00d5030 con 0x55634a5b2ca0 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5ddaffd640 1 -- 172.21.15.5:0/1643470294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f5df00c01b0 con 0x55634a5b2ca0 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5e0405a740 1 -- 172.21.15.5:0/1643470294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dec038070 con 0x55634a5b2ca0 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5ddaffd640 1 -- 172.21.15.5:0/1643470294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f5df0049040 con 0x55634a5b2ca0 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5e0405a740 10 client.21675 fetch_fsmap learned FSMap version 66 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5e0405a740 1 -- 172.21.15.5:0/1643470294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55634a63bcc0 con 0x55634a5b2ca0 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5ddaffd640 1 -- 172.21.15.5:0/1643470294 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dec038070 con 0x55634a5b2ca0 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5ddaffd640 1 -- 172.21.15.5:0/1643470294 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 8394+0+0 (secure 0 0 0) 0x7f5dec00fd20 con 0x55634a5b2ca0 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5e0405a740 10 client.21675 fetch_fsmap finished waiting for FSMap version 66 2024-09-19T06:29:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5e0405a740 10 client.21675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:29:01.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5e0405a740 20 client.21675 populate_metadata read hostname 'smithi005' 2024-09-19T06:29:01.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5e0405a740 1 --2- 172.21.15.5:0/1643470294 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55634a63c450 0x55634a63e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:01.022 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:29:01.022 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-19T06:29:01.022 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-19T06:29:01.022 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:29:01.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5e0405a740 4 client.21675 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:29:01.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5e0405a740 1 -- 172.21.15.5:0/1643470294 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55634a51fd60 con 0x55634a63c450 2024-09-19T06:29:01.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.998+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/1643470294 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55634a63c450 0x55634a63e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5ddaffd640 10 client.21675 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5dfb7fe640 1 --2- 172.21.15.5:0/1643470294 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55634a63c450 0x55634a63e870 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831f4073e5060ded server_cookie=583494f9e7705f57 in_seq=0 out_seq=0 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5ddaffd640 1 -- 172.21.15.5:0/1643470294 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55634a51fd60 con 0x55634a63c450 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5ddaffd640 10 client.21675 handle_command_reply: tid=0 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5e0405a740 1 client.21675 shutdown 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5e0405a740 1 -- 172.21.15.5:0/1643470294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 msgr2=0x55634a5ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5e0405a740 1 --2- 172.21.15.5:0/1643470294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5ca360 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5df00bc120 tx=0x7f5df00f2380 comp rx=0 tx=0).stop 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5e0405a740 1 -- 172.21.15.5:0/1643470294 shutdown_connections 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5e0405a740 1 --2- 172.21.15.5:0/1643470294 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55634a63c450 0x55634a63e870 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5e0405a740 1 --2- 172.21.15.5:0/1643470294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55634a5c1a90 0x55634a5c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5e0405a740 1 --2- 172.21.15.5:0/1643470294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55634a5b2ca0 0x55634a5ca360 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5ddaffd640 0 client.21675 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:29:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5e0405a740 1 --2- 172.21.15.5:0/1643470294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55634a5b2380 0x55634a5c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5e0405a740 1 -- 172.21.15.5:0/1643470294 >> 172.21.15.5:0/1643470294 conn(0x55634a5aea30 msgr2=0x55634a5ad350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5e0405a740 1 -- 172.21.15.5:0/1643470294 shutdown_connections 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:00.999+0000 7f5e0405a740 1 -- 172.21.15.5:0/1643470294 wait complete. 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e0405a740 20 client.21675 trim_cache size 0 max 16384 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e01c81640 1 -- 172.21.15.5:0/2703371722 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5de40a3430 msgr2=0x7f5de40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e01c81640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5de40a3430 0x7f5de40a58f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5df40094d0 tx=0x7f5df4004a40 comp rx=0 tx=0).stop 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e01c81640 1 -- 172.21.15.5:0/2703371722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dfc103650 msgr2=0x7f5dfc1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e01c81640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dfc103650 0x7f5dfc1988b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5df002a9a0 tx=0x7f5df0031970 comp rx=0 tx=0).stop 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e01c81640 1 -- 172.21.15.5:0/2703371722 shutdown_connections 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e01c81640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dfc10c160 0x7f5dfc19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e01c81640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f5de40a3430 0x7f5de40a58f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e01c81640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dfc103650 0x7f5dfc1988b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e01c81640 1 --2- 172.21.15.5:0/2703371722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dfc102d30 0x7f5dfc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e01c81640 1 -- 172.21.15.5:0/2703371722 >> 172.21.15.5:0/2703371722 conn(0x7f5dfc0fc7d0 msgr2=0x7f5dfc0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e01c81640 1 -- 172.21.15.5:0/2703371722 shutdown_connections 2024-09-19T06:29:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:01.000+0000 7f5e01c81640 1 -- 172.21.15.5:0/2703371722 wait complete. 2024-09-19T06:29:01.128 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:01 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:29:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:01 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:29:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:01 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:29:01.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:01 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:29:01.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:01 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:29:01.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:01 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:29:01.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:01 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:29:01.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:01 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:29:01.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:01 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:29:01.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:01 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:29:01.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:01 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:29:01.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:01 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' 2024-09-19T06:29:01.867 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:29:01.867 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:29:02.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:02 smithi005 ceph-mon[30955]: pgmap v812: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-19T06:29:02.134 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:02.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:02 smithi049 ceph-mon[34640]: pgmap v812: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-19T06:29:02.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:02 smithi157 ceph-mon[35346]: pgmap v812: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-19T06:29:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.614+0000 7fbd07b3e640 1 -- 172.21.15.5:0/688588624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00102d90 msgr2=0x7fbd00103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.614+0000 7fbd07b3e640 1 --2- 172.21.15.5:0/688588624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00102d90 0x7fbd00103170 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009cd0 tx=0x7fbcf002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.614+0000 7fbd07b3e640 1 -- 172.21.15.5:0/688588624 shutdown_connections 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.614+0000 7fbd07b3e640 1 --2- 172.21.15.5:0/688588624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd00108290 0x7fbd0010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.614+0000 7fbd07b3e640 1 --2- 172.21.15.5:0/688588624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd00103740 0x7fbd00107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.614+0000 7fbd07b3e640 1 --2- 172.21.15.5:0/688588624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00102d90 0x7fbd00103170 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.614+0000 7fbd07b3e640 1 -- 172.21.15.5:0/688588624 >> 172.21.15.5:0/688588624 conn(0x7fbd000fc7d0 msgr2=0x7fbd000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.614+0000 7fbd07b3e640 1 -- 172.21.15.5:0/688588624 shutdown_connections 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.614+0000 7fbd07b3e640 1 -- 172.21.15.5:0/688588624 wait complete. 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd07b3e640 1 Processor -- start 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd07b3e640 1 -- start start 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd07b3e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd00102d90 0x7fbd00198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd07b3e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00103740 0x7fbd00198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd07b3e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd00108290 0x7fbd0019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd07b3e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd00111f40 con 0x7fbd00103740 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd07b3e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbd00111dc0 con 0x7fbd00108290 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd07b3e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd001120c0 con 0x7fbd00102d90 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd050d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00103740 0x7fbd00198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd050d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00103740 0x7fbd00198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39684/0 (socket says 172.21.15.5:39684) 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd050d9640 1 -- 172.21.15.5:0/3834753064 learned_addr learned my addr 172.21.15.5:0/3834753064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd060db640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd00108290 0x7fbd0019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd058da640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd00102d90 0x7fbd00198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd050d9640 1 -- 172.21.15.5:0/3834753064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd00102d90 msgr2=0x7fbd00198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd050d9640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd00102d90 0x7fbd00198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd050d9640 1 -- 172.21.15.5:0/3834753064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd00108290 msgr2=0x7fbd0019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd050d9640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd00108290 0x7fbd0019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.615+0000 7fbd050d9640 1 -- 172.21.15.5:0/3834753064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0019d4b0 con 0x7fbd00103740 2024-09-19T06:29:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.616+0000 7fbd050d9640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00103740 0x7fbd00198a40 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400d740 tx=0x7fbcf400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.616+0000 7fbcfeffd640 1 -- 172.21.15.5:0/3834753064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4017070 con 0x7fbd00103740 2024-09-19T06:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.616+0000 7fbd07b3e640 1 -- 172.21.15.5:0/3834753064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0019d7a0 con 0x7fbd00103740 2024-09-19T06:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.616+0000 7fbcfeffd640 1 -- 172.21.15.5:0/3834753064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcf400de50 con 0x7fbd00103740 2024-09-19T06:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.616+0000 7fbcfeffd640 1 -- 172.21.15.5:0/3834753064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4004d60 con 0x7fbd00103740 2024-09-19T06:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.616+0000 7fbd07b3e640 1 -- 172.21.15.5:0/3834753064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd001a5080 con 0x7fbd00103740 2024-09-19T06:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.617+0000 7fbd07b3e640 1 -- 172.21.15.5:0/3834753064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd0006a7f0 con 0x7fbd00103740 2024-09-19T06:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.618+0000 7fbcfeffd640 1 -- 172.21.15.5:0/3834753064 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbcf4004f00 con 0x7fbd00103740 2024-09-19T06:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.619+0000 7fbcfeffd640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbce80a33b0 0x7fbce80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.619+0000 7fbd058da640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbce80a33b0 0x7fbce80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.619+0000 7fbcfeffd640 1 -- 172.21.15.5:0/3834753064 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fbcf40cfae0 con 0x7fbd00103740 2024-09-19T06:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.620+0000 7fbd058da640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbce80a33b0 0x7fbce80a5870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0000f80 tx=0x7fbcf00093f0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.623+0000 7fbcfeffd640 1 -- 172.21.15.5:0/3834753064 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbcf409c4a0 con 0x7fbd00103740 2024-09-19T06:29:02.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.836+0000 7fbd07b3e640 1 -- 172.21.15.5:0/3834753064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd0010a2a0 con 0x7fbd00103740 2024-09-19T06:29:02.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.837+0000 7fbcfeffd640 1 -- 172.21.15.5:0/3834753064 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11970 (secure 0 0 0) 0x7fbcf409bda0 con 0x7fbd00103740 2024-09-19T06:29:02.839 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:29:02.839 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":66,"btime":"2024-09-19T06:28:43:618012+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:28:43.170395+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:29:02.840 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 66 2024-09-19T06:29:02.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.839+0000 7fbd07b3e640 1 -- 172.21.15.5:0/3834753064 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbce80a33b0 msgr2=0x7fbce80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:02.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.839+0000 7fbd07b3e640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbce80a33b0 0x7fbce80a5870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0000f80 tx=0x7fbcf00093f0 comp rx=0 tx=0).stop 2024-09-19T06:29:02.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.839+0000 7fbd07b3e640 1 -- 172.21.15.5:0/3834753064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00103740 msgr2=0x7fbd00198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:02.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.839+0000 7fbd07b3e640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00103740 0x7fbd00198a40 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400d740 tx=0x7fbcf400dc00 comp rx=0 tx=0).stop 2024-09-19T06:29:02.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.839+0000 7fbd07b3e640 1 -- 172.21.15.5:0/3834753064 shutdown_connections 2024-09-19T06:29:02.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.839+0000 7fbd07b3e640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd00108290 0x7fbd0019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:02.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.839+0000 7fbd07b3e640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00103740 0x7fbd00198a40 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:02.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.839+0000 7fbd07b3e640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fbce80a33b0 0x7fbce80a5870 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:02.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.839+0000 7fbd07b3e640 1 --2- 172.21.15.5:0/3834753064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd00102d90 0x7fbd00198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:02.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.839+0000 7fbd07b3e640 1 -- 172.21.15.5:0/3834753064 >> 172.21.15.5:0/3834753064 conn(0x7fbd000fc7d0 msgr2=0x7fbd00105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:02.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.839+0000 7fbd07b3e640 1 -- 172.21.15.5:0/3834753064 shutdown_connections 2024-09-19T06:29:02.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:02.839+0000 7fbd07b3e640 1 -- 172.21.15.5:0/3834753064 wait complete. 2024-09-19T06:29:03.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:03 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3834753064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:29:03.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:03 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3834753064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:29:03.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:29:03.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:03 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3834753064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:29:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:04.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:04 smithi005 ceph-mon[30955]: pgmap v813: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 69 KiB/s wr, 37 op/s 2024-09-19T06:29:04.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.226+0000 7fb294d36640 1 -- 172.21.15.5:0/3266360614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb290101de0 msgr2=0x7fb2901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:04.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.226+0000 7fb294d36640 1 --2- 172.21.15.5:0/3266360614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb290101de0 0x7fb2901021c0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fb280009900 tx=0x7fb28002a410 comp rx=0 tx=0).stop 2024-09-19T06:29:04.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.226+0000 7fb294d36640 1 -- 172.21.15.5:0/3266360614 shutdown_connections 2024-09-19T06:29:04.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.226+0000 7fb294d36640 1 --2- 172.21.15.5:0/3266360614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb290106f10 0x7fb29010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.226+0000 7fb294d36640 1 --2- 172.21.15.5:0/3266360614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb290102790 0x7fb2901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.226+0000 7fb294d36640 1 --2- 172.21.15.5:0/3266360614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb290101de0 0x7fb2901021c0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.226+0000 7fb294d36640 1 -- 172.21.15.5:0/3266360614 >> 172.21.15.5:0/3266360614 conn(0x7fb2900fd8c0 msgr2=0x7fb2900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.226+0000 7fb294d36640 1 -- 172.21.15.5:0/3266360614 shutdown_connections 2024-09-19T06:29:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.226+0000 7fb294d36640 1 -- 172.21.15.5:0/3266360614 wait complete. 2024-09-19T06:29:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.227+0000 7fb294d36640 1 Processor -- start 2024-09-19T06:29:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.227+0000 7fb294d36640 1 -- start start 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.227+0000 7fb294d36640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb290101de0 0x7fb290198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.227+0000 7fb294d36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb290102790 0x7fb2901988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.227+0000 7fb294d36640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb290106f10 0x7fb29019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.227+0000 7fb294d36640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb290111de0 con 0x7fb290102790 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.227+0000 7fb294d36640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb290111c60 con 0x7fb290101de0 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.227+0000 7fb294d36640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb290111f60 con 0x7fb290106f10 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb290102790 0x7fb2901988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb290102790 0x7fb2901988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39694/0 (socket says 172.21.15.5:39694) 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb287fff640 1 -- 172.21.15.5:0/3988734643 learned_addr learned my addr 172.21.15.5:0/3988734643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb28e59c640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb290101de0 0x7fb290198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb28ed9d640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb290106f10 0x7fb29019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb287fff640 1 -- 172.21.15.5:0/3988734643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb290106f10 msgr2=0x7fb29019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb287fff640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb290106f10 0x7fb29019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb287fff640 1 -- 172.21.15.5:0/3988734643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb290101de0 msgr2=0x7fb290198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:04.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb287fff640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb290101de0 0x7fb290198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb287fff640 1 -- 172.21.15.5:0/3988734643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29019d330 con 0x7fb290102790 2024-09-19T06:29:04.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb287fff640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb290102790 0x7fb2901988c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fb27800d950 tx=0x7fb27800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:04.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb2877fe640 1 -- 172.21.15.5:0/3988734643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278017030 con 0x7fb290102790 2024-09-19T06:29:04.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb2877fe640 1 -- 172.21.15.5:0/3988734643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb278004020 con 0x7fb290102790 2024-09-19T06:29:04.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb294d36640 1 -- 172.21.15.5:0/3988734643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29019d620 con 0x7fb290102790 2024-09-19T06:29:04.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb2877fe640 1 -- 172.21.15.5:0/3988734643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2780047a0 con 0x7fb290102790 2024-09-19T06:29:04.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.228+0000 7fb294d36640 1 -- 172.21.15.5:0/3988734643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2901a4f00 con 0x7fb290102790 2024-09-19T06:29:04.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.231+0000 7fb2877fe640 1 -- 172.21.15.5:0/3988734643 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb2780041d0 con 0x7fb290102790 2024-09-19T06:29:04.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb29710f740 1 Processor -- start 2024-09-19T06:29:04.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb29710f740 1 -- start start 2024-09-19T06:29:04.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb29710f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e241090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb29710f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e2415d0 0x55a23e241a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb29710f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e24c370 0x55a23e24e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb29710f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a23e1a72f0 con 0x55a23e242b40 2024-09-19T06:29:04.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb29710f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a23e14ecf0 con 0x55a23e24c370 2024-09-19T06:29:04.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb29710f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a23e1a32b0 con 0x55a23e2415d0 2024-09-19T06:29:04.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb28ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e241090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb28ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e241090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39698/0 (socket says 172.21.15.5:39698) 2024-09-19T06:29:04.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb28ed9d640 1 -- 172.21.15.5:0/844233785 learned_addr learned my addr 172.21.15.5:0/844233785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:04.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb28e59c640 1 --2- 172.21.15.5:0/844233785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e2415d0 0x55a23e241a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.232+0000 7fb287fff640 1 --2- 172.21.15.5:0/844233785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e24c370 0x55a23e24e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb2877fe640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb27c0a33b0 0x7fb27c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb28ed9d640 1 -- 172.21.15.5:0/844233785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e2415d0 msgr2=0x55a23e241a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb28ed9d640 1 --2- 172.21.15.5:0/844233785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e2415d0 0x55a23e241a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb28ed9d640 1 -- 172.21.15.5:0/844233785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e24c370 msgr2=0x55a23e24e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb28ed9d640 1 --2- 172.21.15.5:0/844233785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e24c370 0x55a23e24e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb28ed9d640 1 -- 172.21.15.5:0/844233785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a23e1333a0 con 0x55a23e242b40 2024-09-19T06:29:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb287fff640 1 --2- 172.21.15.5:0/844233785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e24c370 0x55a23e24e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb2877fe640 1 -- 172.21.15.5:0/3988734643 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fb2780cf790 con 0x7fb290102790 2024-09-19T06:29:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb28ed9d640 1 --2- 172.21.15.5:0/844233785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e241090 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fb288009e60 tx=0x7fb288010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd98cc1d3eddd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb28ed9d640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb27c0a33b0 0x7fb27c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb284ff9640 1 -- 172.21.15.5:0/844233785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288021070 con 0x55a23e242b40 2024-09-19T06:29:04.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb284ff9640 1 -- 172.21.15.5:0/844233785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb28801b400 con 0x55a23e242b40 2024-09-19T06:29:04.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.233+0000 7fb284ff9640 1 -- 172.21.15.5:0/844233785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288016070 con 0x55a23e242b40 2024-09-19T06:29:04.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.234+0000 7fb29710f740 1 -- 172.21.15.5:0/844233785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 msgr2=0x55a23e241090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:04.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.234+0000 7fb28ed9d640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb27c0a33b0 0x7fb27c0a5870 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb28801e000 tx=0x7fb2880117b0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:04.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.234+0000 7fb29710f740 1 --2- 172.21.15.5:0/844233785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e241090 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fb288009e60 tx=0x7fb288010740 comp rx=0 tx=0).stop 2024-09-19T06:29:04.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.234+0000 7fb29710f740 1 -- 172.21.15.5:0/844233785 shutdown_connections 2024-09-19T06:29:04.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.234+0000 7fb29710f740 1 --2- 172.21.15.5:0/844233785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e24c370 0x55a23e24e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.234+0000 7fb29710f740 1 --2- 172.21.15.5:0/844233785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e2415d0 0x55a23e241a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.234+0000 7fb29710f740 1 --2- 172.21.15.5:0/844233785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e241090 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.234+0000 7fb29710f740 1 -- 172.21.15.5:0/844233785 >> 172.21.15.5:0/844233785 conn(0x55a23e20a310 msgr2=0x55a23e23dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:04.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.234+0000 7fb29710f740 1 -- 172.21.15.5:0/844233785 shutdown_connections 2024-09-19T06:29:04.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.234+0000 7fb29710f740 1 -- 172.21.15.5:0/844233785 wait complete. 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.234+0000 7fb29710f740 1 -- start start 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb29710f740 1 Processor -- start 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb29710f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e2415d0 0x55a23e24bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb29710f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e248fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb29710f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e24c370 0x55a23e249570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb29710f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a23e24f0c0 con 0x55a23e242b40 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb29710f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a23e162060 con 0x55a23e24c370 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb29710f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a23e24f240 con 0x55a23e2415d0 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb29710f740 10 client.0 _ll_register_callbacks cb 0x55a23deb58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e248fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e248fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39714/0 (socket says 172.21.15.5:39714) 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb287fff640 1 -- 172.21.15.5:0/2888002163 learned_addr learned my addr 172.21.15.5:0/2888002163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb28ed9d640 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e24c370 0x55a23e249570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb28e59c640 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e2415d0 0x55a23e24bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb287fff640 1 -- 172.21.15.5:0/2888002163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e2415d0 msgr2=0x55a23e24bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb287fff640 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e2415d0 0x55a23e24bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb287fff640 1 -- 172.21.15.5:0/2888002163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e24c370 msgr2=0x55a23e249570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb287fff640 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e24c370 0x55a23e249570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb287fff640 1 -- 172.21.15.5:0/2888002163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a23df183c0 con 0x55a23e242b40 2024-09-19T06:29:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb287fff640 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e248fd0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb278009610 tx=0x7fb27809b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86990543b5ef659e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.235+0000 7fb275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:29:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb275ffb640 1 -- 172.21.15.5:0/2888002163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27809ba50 con 0x55a23e242b40 2024-09-19T06:29:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb275ffb640 1 -- 172.21.15.5:0/2888002163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fb2780980c0 con 0x55a23e242b40 2024-09-19T06:29:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb29710f740 1 -- 172.21.15.5:0/2888002163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb28801b400 con 0x55a23e242b40 2024-09-19T06:29:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb275ffb640 1 -- 172.21.15.5:0/2888002163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fb27809fe00 con 0x55a23e242b40 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb29710f740 10 client.21693 fetch_fsmap learned FSMap version 66 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb29710f740 1 -- 172.21.15.5:0/2888002163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55a23e25ab60 con 0x55a23e242b40 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb275ffb640 1 -- 172.21.15.5:0/2888002163 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb28801b400 con 0x55a23e242b40 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb275ffb640 1 -- 172.21.15.5:0/2888002163 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 8394+0+0 (secure 0 0 0) 0x7fb2780ba2d0 con 0x55a23e242b40 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb29710f740 10 client.21693 fetch_fsmap finished waiting for FSMap version 66 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb29710f740 10 client.21693 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb29710f740 20 client.21693 populate_metadata read hostname 'smithi005' 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb29710f740 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55a23e25c020 0x55a23e25e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb29710f740 4 client.21693 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.236+0000 7fb29710f740 1 -- 172.21.15.5:0/2888002163 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a23e1b4860 con 0x55a23e25c020 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.237+0000 7fb28e59c640 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55a23e25c020 0x55a23e25e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.237+0000 7fb28e59c640 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55a23e25c020 0x55a23e25e430 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ff3e16528d62c4f2 server_cookie=d45956b9193726c8 in_seq=0 out_seq=0 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.237+0000 7fb275ffb640 10 client.21693 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:29:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb275ffb640 1 -- 172.21.15.5:0/2888002163 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a23e1b4860 con 0x55a23e25c020 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb275ffb640 10 client.21693 handle_command_reply: tid=0 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb29710f740 1 client.21693 shutdown 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb29710f740 1 -- 172.21.15.5:0/2888002163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 msgr2=0x55a23e248fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb29710f740 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e248fd0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb278009610 tx=0x7fb27809b590 comp rx=0 tx=0).stop 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb29710f740 1 -- 172.21.15.5:0/2888002163 shutdown_connections 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb29710f740 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55a23e25c020 0x55a23e25e430 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb29710f740 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e24c370 0x55a23e249570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb29710f740 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e248fd0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb29710f740 1 --2- 172.21.15.5:0/2888002163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e2415d0 0x55a23e24bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb29710f740 1 -- 172.21.15.5:0/2888002163 >> 172.21.15.5:0/2888002163 conn(0x55a23e20a310 msgr2=0x55a23e243960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb275ffb640 0 client.21693 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb29710f740 1 -- 172.21.15.5:0/2888002163 shutdown_connections 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb29710f740 1 -- 172.21.15.5:0/2888002163 wait complete. 2024-09-19T06:29:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.238+0000 7fb29710f740 20 client.21693 trim_cache size 0 max 16384 2024-09-19T06:29:04.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb29710f740 1 Processor -- start 2024-09-19T06:29:04.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb29710f740 1 -- start start 2024-09-19T06:29:04.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb29710f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e2415d0 0x55a23e258360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb29710f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e244120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb29710f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e24c370 0x55a23e244660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb29710f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901ad850 con 0x55a23e242b40 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb29710f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901af450 con 0x55a23e2415d0 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb29710f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901adf50 con 0x55a23e24c370 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e244120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e244120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39728/0 (socket says 172.21.15.5:39728) 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb287fff640 1 -- 172.21.15.5:0/3306025387 learned_addr learned my addr 172.21.15.5:0/3306025387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb28ed9d640 1 --2- 172.21.15.5:0/3306025387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e24c370 0x55a23e244660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb28e59c640 1 --2- 172.21.15.5:0/3306025387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e2415d0 0x55a23e258360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb287fff640 1 -- 172.21.15.5:0/3306025387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e24c370 msgr2=0x55a23e244660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.251+0000 7fb287fff640 1 --2- 172.21.15.5:0/3306025387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e24c370 0x55a23e244660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb287fff640 1 -- 172.21.15.5:0/3306025387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e2415d0 msgr2=0x55a23e258360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb287fff640 1 --2- 172.21.15.5:0/3306025387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e2415d0 0x55a23e258360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb287fff640 1 -- 172.21.15.5:0/3306025387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a23e242780 con 0x55a23e242b40 2024-09-19T06:29:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb28ed9d640 1 --2- 172.21.15.5:0/3306025387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e24c370 0x55a23e244660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:04.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb28e59c640 1 --2- 172.21.15.5:0/3306025387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e2415d0 0x55a23e258360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:04.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb287fff640 1 --2- 172.21.15.5:0/3306025387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e244120 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fb2780cd4c0 tx=0x7fb27809b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72d43d59ccea54c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:04.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb277fff640 1 -- 172.21.15.5:0/3306025387 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27809ba50 con 0x55a23e242b40 2024-09-19T06:29:04.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb277fff640 1 -- 172.21.15.5:0/3306025387 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2780a46c0 con 0x55a23e242b40 2024-09-19T06:29:04.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb277fff640 1 -- 172.21.15.5:0/3306025387 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2780980c0 con 0x55a23e242b40 2024-09-19T06:29:04.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb29710f740 1 -- 172.21.15.5:0/3306025387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 msgr2=0x55a23e244120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:04.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb29710f740 1 --2- 172.21.15.5:0/3306025387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e244120 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fb2780cd4c0 tx=0x7fb27809b6c0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb29710f740 1 -- 172.21.15.5:0/3306025387 shutdown_connections 2024-09-19T06:29:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb29710f740 1 --2- 172.21.15.5:0/3306025387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e24c370 0x55a23e244660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb29710f740 1 --2- 172.21.15.5:0/3306025387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e242b40 0x55a23e244120 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb29710f740 1 --2- 172.21.15.5:0/3306025387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e2415d0 0x55a23e258360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.252+0000 7fb29710f740 1 -- 172.21.15.5:0/3306025387 >> 172.21.15.5:0/3306025387 conn(0x55a23e20a310 msgr2=0x55a23e2480c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb29710f740 1 -- 172.21.15.5:0/3306025387 shutdown_connections 2024-09-19T06:29:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb29710f740 1 -- 172.21.15.5:0/3306025387 wait complete. 2024-09-19T06:29:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb29710f740 1 -- start start 2024-09-19T06:29:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb29710f740 1 Processor -- start 2024-09-19T06:29:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb29710f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e2415d0 0x55a23e254c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb29710f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e242b40 0x55a23e245d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb29710f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e24c370 0x55a23e246240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb29710f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901aed80 con 0x55a23e24c370 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb29710f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901ae680 con 0x55a23e2415d0 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb29710f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901ab850 con 0x55a23e242b40 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb29710f740 10 client.0 _ll_register_callbacks cb 0x7fb278005d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb28ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e24c370 0x55a23e246240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb28ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e24c370 0x55a23e246240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39732/0 (socket says 172.21.15.5:39732) 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb28ed9d640 1 -- 172.21.15.5:0/944586065 learned_addr learned my addr 172.21.15.5:0/944586065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb28e59c640 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e2415d0 0x55a23e254c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.253+0000 7fb287fff640 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e242b40 0x55a23e245d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb28ed9d640 1 -- 172.21.15.5:0/944586065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e242b40 msgr2=0x55a23e245d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb28ed9d640 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e242b40 0x55a23e245d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb28ed9d640 1 -- 172.21.15.5:0/944586065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e2415d0 msgr2=0x55a23e254c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb28ed9d640 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e2415d0 0x55a23e254c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb28ed9d640 1 -- 172.21.15.5:0/944586065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a23e246a70 con 0x55a23e24c370 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb28ed9d640 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e24c370 0x55a23e246240 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb288009e60 tx=0x7fb288016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414e3f54eb07d582 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb275ffb640 1 -- 172.21.15.5:0/944586065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288021040 con 0x55a23e24c370 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb275ffb640 1 -- 172.21.15.5:0/944586065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fb288012ae0 con 0x55a23e24c370 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb29710f740 1 -- 172.21.15.5:0/944586065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2780a46c0 con 0x55a23e24c370 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb275ffb640 1 -- 172.21.15.5:0/944586065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fb288040050 con 0x55a23e24c370 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb29710f740 10 client.21699 fetch_fsmap learned FSMap version 66 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb29710f740 1 -- 172.21.15.5:0/944586065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55a23e2c6ca0 con 0x55a23e24c370 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.254+0000 7fb275ffb640 1 -- 172.21.15.5:0/944586065 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2780a46c0 con 0x55a23e24c370 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.255+0000 7fb275ffb640 1 -- 172.21.15.5:0/944586065 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 8394+0+0 (secure 0 0 0) 0x7fb28800f040 con 0x55a23e24c370 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.255+0000 7fb29710f740 10 client.21699 fetch_fsmap finished waiting for FSMap version 66 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.255+0000 7fb29710f740 10 client.21699 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.255+0000 7fb29710f740 20 client.21699 populate_metadata read hostname 'smithi005' 2024-09-19T06:29:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.255+0000 7fb29710f740 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55a23e2c7140 0x55a23e2c9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.255+0000 7fb29710f740 4 client.21699 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.255+0000 7fb29710f740 1 -- 172.21.15.5:0/944586065 --> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a23e1aed70 con 0x55a23e2c7140 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.255+0000 7fb28e59c640 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55a23e2c7140 0x55a23e2c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.255+0000 7fb28e59c640 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55a23e2c7140 0x55a23e2c9560 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=737f1231e90c972b server_cookie=a847e6926fc857a7 in_seq=0 out_seq=0 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.255+0000 7fb275ffb640 10 client.21699 ms_handle_connect on v2:172.21.15.49:6836/2858664643 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb275ffb640 1 -- 172.21.15.5:0/944586065 <== mds.4 v2:172.21.15.49:6836/2858664643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a23e1aed70 con 0x55a23e2c7140 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb275ffb640 10 client.21699 handle_command_reply: tid=0 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb29710f740 1 client.21699 shutdown 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb29710f740 1 -- 172.21.15.5:0/944586065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e24c370 msgr2=0x55a23e246240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb29710f740 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e24c370 0x55a23e246240 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb288009e60 tx=0x7fb288016000 comp rx=0 tx=0).stop 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb29710f740 1 -- 172.21.15.5:0/944586065 shutdown_connections 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb29710f740 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.49:6836/2858664643,v1:172.21.15.49:6837/2858664643] conn(0x55a23e2c7140 0x55a23e2c9560 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb29710f740 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a23e24c370 0x55a23e246240 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb29710f740 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a23e242b40 0x55a23e245d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb29710f740 1 --2- 172.21.15.5:0/944586065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a23e2415d0 0x55a23e254c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb29710f740 1 -- 172.21.15.5:0/944586065 >> 172.21.15.5:0/944586065 conn(0x55a23e20a310 msgr2=0x55a23e24a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb275ffb640 0 client.21699 ms_handle_reset on v2:172.21.15.49:6836/2858664643 2024-09-19T06:29:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb29710f740 1 -- 172.21.15.5:0/944586065 shutdown_connections 2024-09-19T06:29:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb29710f740 1 -- 172.21.15.5:0/944586065 wait complete. 2024-09-19T06:29:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb29710f740 20 client.21699 trim_cache size 0 max 16384 2024-09-19T06:29:04.264 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:29:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb294d36640 1 -- 172.21.15.5:0/3988734643 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb27c0a33b0 msgr2=0x7fb27c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.256+0000 7fb294d36640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb27c0a33b0 0x7fb27c0a5870 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb28801e000 tx=0x7fb2880117b0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.257+0000 7fb294d36640 1 -- 172.21.15.5:0/3988734643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb290102790 msgr2=0x7fb2901988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.257+0000 7fb294d36640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb290102790 0x7fb2901988c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fb27800d950 tx=0x7fb27800de10 comp rx=0 tx=0).stop 2024-09-19T06:29:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.257+0000 7fb294d36640 1 -- 172.21.15.5:0/3988734643 shutdown_connections 2024-09-19T06:29:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.257+0000 7fb294d36640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fb27c0a33b0 0x7fb27c0a5870 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.257+0000 7fb294d36640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb290106f10 0x7fb29019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.257+0000 7fb294d36640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb290102790 0x7fb2901988c0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.257+0000 7fb294d36640 1 --2- 172.21.15.5:0/3988734643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb290101de0 0x7fb290198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.257+0000 7fb294d36640 1 -- 172.21.15.5:0/3988734643 >> 172.21.15.5:0/3988734643 conn(0x7fb2900fd8c0 msgr2=0x7fb2900fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.257+0000 7fb294d36640 1 -- 172.21.15.5:0/3988734643 shutdown_connections 2024-09-19T06:29:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:04.257+0000 7fb294d36640 1 -- 172.21.15.5:0/3988734643 wait complete. 2024-09-19T06:29:04.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:04 smithi049 ceph-mon[34640]: pgmap v813: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 69 KiB/s wr, 37 op/s 2024-09-19T06:29:04.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:04 smithi157 ceph-mon[35346]: pgmap v813: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 69 KiB/s wr, 37 op/s 2024-09-19T06:29:05.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:29:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.883+0000 7f51b4175640 1 -- 172.21.15.5:0/2260682454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac103660 msgr2=0x7f51ac10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.883+0000 7f51b4175640 1 --2- 172.21.15.5:0/2260682454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac103660 0x7f51ac10bb50 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f519c009900 tx=0x7f519c02a480 comp rx=0 tx=0).stop 2024-09-19T06:29:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.883+0000 7f51b4175640 1 -- 172.21.15.5:0/2260682454 shutdown_connections 2024-09-19T06:29:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.883+0000 7f51b4175640 1 --2- 172.21.15.5:0/2260682454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51ac10c170 0x7f51ac10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.883+0000 7f51b4175640 1 --2- 172.21.15.5:0/2260682454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac103660 0x7f51ac10bb50 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.883+0000 7f51b4175640 1 --2- 172.21.15.5:0/2260682454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51ac102d40 0x7f51ac103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.883+0000 7f51b4175640 1 -- 172.21.15.5:0/2260682454 >> 172.21.15.5:0/2260682454 conn(0x7f51ac0fc7a0 msgr2=0x7f51ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.883+0000 7f51b4175640 1 -- 172.21.15.5:0/2260682454 shutdown_connections 2024-09-19T06:29:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.883+0000 7f51b4175640 1 -- 172.21.15.5:0/2260682454 wait complete. 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b4175640 1 Processor -- start 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b4175640 1 -- start start 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b4175640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac102d40 0x7f51ac198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b4175640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51ac103660 0x7f51ac198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b4175640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51ac10c170 0x7f51ac19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b4175640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac111e20 con 0x7f51ac102d40 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b4175640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac111ca0 con 0x7f51ac103660 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b4175640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac111fa0 con 0x7f51ac10c170 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b1f11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac102d40 0x7f51ac198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b1f11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac102d40 0x7f51ac198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39116/0 (socket says 172.21.15.5:39116) 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b1f11640 1 -- 172.21.15.5:0/1602218498 learned_addr learned my addr 172.21.15.5:0/1602218498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b2712640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51ac10c170 0x7f51ac19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b1710640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51ac103660 0x7f51ac198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.884+0000 7f51b1f11640 1 -- 172.21.15.5:0/1602218498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51ac10c170 msgr2=0x7f51ac19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:05.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.885+0000 7f51b1f11640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51ac10c170 0x7f51ac19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.885+0000 7f51b1f11640 1 -- 172.21.15.5:0/1602218498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51ac103660 msgr2=0x7f51ac198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:05.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.885+0000 7f51b1f11640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51ac103660 0x7f51ac198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.885+0000 7f51b1f11640 1 -- 172.21.15.5:0/1602218498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ac19d2b0 con 0x7f51ac102d40 2024-09-19T06:29:05.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.885+0000 7f51b1710640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51ac103660 0x7f51ac198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:05.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.885+0000 7f51b1f11640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac102d40 0x7f51ac198300 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f51a800c920 tx=0x7f51a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:05.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.885+0000 7f51a2ffd640 1 -- 172.21.15.5:0/1602218498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8007c70 con 0x7f51ac102d40 2024-09-19T06:29:05.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.885+0000 7f51a2ffd640 1 -- 172.21.15.5:0/1602218498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51a8007e10 con 0x7f51ac102d40 2024-09-19T06:29:05.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.885+0000 7f51b4175640 1 -- 172.21.15.5:0/1602218498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ac19d5a0 con 0x7f51ac102d40 2024-09-19T06:29:05.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.885+0000 7f51a2ffd640 1 -- 172.21.15.5:0/1602218498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8018820 con 0x7f51ac102d40 2024-09-19T06:29:05.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.885+0000 7f51b4175640 1 -- 172.21.15.5:0/1602218498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ac1a4e80 con 0x7f51ac102d40 2024-09-19T06:29:05.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.887+0000 7f51a2ffd640 1 -- 172.21.15.5:0/1602218498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f51a80189c0 con 0x7f51ac102d40 2024-09-19T06:29:05.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b654e740 1 Processor -- start 2024-09-19T06:29:05.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b654e740 1 -- start start 2024-09-19T06:29:05.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b654e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d86e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b654e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d86e5d0 0x563b3d86ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b654e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d879370 0x563b3d87b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b654e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563b3d7d42f0 con 0x563b3d879370 2024-09-19T06:29:05.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b654e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563b3d77bcf0 con 0x563b3d86e5d0 2024-09-19T06:29:05.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b654e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563b3d7d02b0 con 0x563b3d86fb40 2024-09-19T06:29:05.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b1710640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d879370 0x563b3d87b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b1710640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d879370 0x563b3d87b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39120/0 (socket says 172.21.15.5:39120) 2024-09-19T06:29:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b1710640 1 -- 172.21.15.5:0/2282551912 learned_addr learned my addr 172.21.15.5:0/2282551912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51a2ffd640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f51980a33b0 0x7f51980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b1f11640 1 --2- 172.21.15.5:0/2282551912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d86e5d0 0x563b3d86ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51b2712640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f51980a33b0 0x7f51980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.888+0000 7f51a2ffd640 1 -- 172.21.15.5:0/1602218498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f51a80cfe70 con 0x7f51ac102d40 2024-09-19T06:29:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.889+0000 7f51b2712640 1 --2- 172.21.15.5:0/2282551912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d86e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.889+0000 7f51b1710640 1 -- 172.21.15.5:0/2282551912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 msgr2=0x563b3d86e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.889+0000 7f51b1710640 1 --2- 172.21.15.5:0/2282551912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d86e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.889+0000 7f51b1710640 1 -- 172.21.15.5:0/2282551912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d86e5d0 msgr2=0x563b3d86ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.889+0000 7f51b1710640 1 --2- 172.21.15.5:0/2282551912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d86e5d0 0x563b3d86ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.889+0000 7f51b1710640 1 -- 172.21.15.5:0/2282551912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b3d7603a0 con 0x563b3d879370 2024-09-19T06:29:05.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.889+0000 7f51b2712640 1 --2- 172.21.15.5:0/2282551912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d86e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:29:05.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.889+0000 7f51b1710640 1 --2- 172.21.15.5:0/2282551912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d879370 0x563b3d87b760 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f519c02a9a0 tx=0x7f519c032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e79380916b8d939 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:05.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.889+0000 7f51b2712640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f51980a33b0 0x7f51980a5870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f51a40098b0 tx=0x7f51a4009840 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:05.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.889+0000 7f5197fff640 1 -- 172.21.15.5:0/2282551912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c03f070 con 0x563b3d879370 2024-09-19T06:29:05.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.889+0000 7f5197fff640 1 -- 172.21.15.5:0/2282551912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f519c0316a0 con 0x563b3d879370 2024-09-19T06:29:05.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.889+0000 7f5197fff640 1 -- 172.21.15.5:0/2282551912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c035070 con 0x563b3d879370 2024-09-19T06:29:05.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.890+0000 7f51b654e740 1 -- 172.21.15.5:0/2282551912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d879370 msgr2=0x563b3d87b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:05.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.890+0000 7f51b654e740 1 --2- 172.21.15.5:0/2282551912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d879370 0x563b3d87b760 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f519c02a9a0 tx=0x7f519c032da0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.890+0000 7f51b654e740 1 -- 172.21.15.5:0/2282551912 shutdown_connections 2024-09-19T06:29:05.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.890+0000 7f51b654e740 1 --2- 172.21.15.5:0/2282551912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d879370 0x563b3d87b760 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.890+0000 7f51b654e740 1 --2- 172.21.15.5:0/2282551912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d86e5d0 0x563b3d86ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.890+0000 7f51b654e740 1 --2- 172.21.15.5:0/2282551912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d86e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.890+0000 7f51b654e740 1 -- 172.21.15.5:0/2282551912 >> 172.21.15.5:0/2282551912 conn(0x563b3d837310 msgr2=0x563b3d86ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.890+0000 7f51b654e740 1 -- 172.21.15.5:0/2282551912 shutdown_connections 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.890+0000 7f51b654e740 1 -- 172.21.15.5:0/2282551912 wait complete. 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.890+0000 7f51b654e740 1 -- start start 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.890+0000 7f51b654e740 1 Processor -- start 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b654e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d878d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b654e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d875fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b654e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d8765d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b654e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563b3d87c010 con 0x563b3d86e5d0 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b654e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563b3d78f060 con 0x563b3d879370 2024-09-19T06:29:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b654e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563b3d87c190 con 0x563b3d86fb40 2024-09-19T06:29:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b654e740 10 client.0 _ll_register_callbacks cb 0x563b3d4e28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:29:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b1f11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d878d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b1f11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d878d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39134/0 (socket says 172.21.15.5:39134) 2024-09-19T06:29:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b1f11640 1 -- 172.21.15.5:0/2034447551 learned_addr learned my addr 172.21.15.5:0/2034447551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b2712640 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d8765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b1710640 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d875fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b1f11640 1 -- 172.21.15.5:0/2034447551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 msgr2=0x563b3d875fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b1f11640 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d875fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b1f11640 1 -- 172.21.15.5:0/2034447551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 msgr2=0x563b3d8765d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b1f11640 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d8765d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b1f11640 1 -- 172.21.15.5:0/2034447551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b3d5453c0 con 0x563b3d86e5d0 2024-09-19T06:29:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b2712640 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d8765d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51b1f11640 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d878d30 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f51a800fda0 tx=0x7f51a809bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31253dcbc363f6ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.891+0000 7f51957fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:29:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51957fa640 1 -- 172.21.15.5:0/2034447551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a80b4020 con 0x563b3d86e5d0 2024-09-19T06:29:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51957fa640 1 -- 172.21.15.5:0/2034447551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f51a80987a0 con 0x563b3d86e5d0 2024-09-19T06:29:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51b654e740 1 -- 172.21.15.5:0/2034447551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f519c0316a0 con 0x563b3d86e5d0 2024-09-19T06:29:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51957fa640 1 -- 172.21.15.5:0/2034447551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f51a809c130 con 0x563b3d86e5d0 2024-09-19T06:29:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51b654e740 10 client.21711 fetch_fsmap learned FSMap version 66 2024-09-19T06:29:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51b654e740 1 -- 172.21.15.5:0/2034447551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x563b3d8879e0 con 0x563b3d86e5d0 2024-09-19T06:29:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51957fa640 1 -- 172.21.15.5:0/2034447551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f519c0316a0 con 0x563b3d86e5d0 2024-09-19T06:29:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51957fa640 1 -- 172.21.15.5:0/2034447551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 8394+0+0 (secure 0 0 0) 0x7f51a80a4910 con 0x563b3d86e5d0 2024-09-19T06:29:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51b654e740 10 client.21711 fetch_fsmap finished waiting for FSMap version 66 2024-09-19T06:29:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51b654e740 10 client.21711 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:29:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51b654e740 20 client.21711 populate_metadata read hostname 'smithi005' 2024-09-19T06:29:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51b654e740 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563b3d888ea0 0x563b3d88b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51b654e740 4 client.21711 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:29:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51b654e740 1 -- 172.21.15.5:0/2034447551 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b3d7e1860 con 0x563b3d888ea0 2024-09-19T06:29:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.892+0000 7f51b1710640 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563b3d888ea0 0x563b3d88b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.893+0000 7f51b1710640 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563b3d888ea0 0x563b3d88b2d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d68cf84627cf4472 server_cookie=c083be6b108921f in_seq=0 out_seq=0 2024-09-19T06:29:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.893+0000 7f51957fa640 10 client.21711 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:29:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51957fa640 1 -- 172.21.15.5:0/2034447551 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b3d7e1860 con 0x563b3d888ea0 2024-09-19T06:29:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51957fa640 10 client.21711 handle_command_reply: tid=0 2024-09-19T06:29:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51b654e740 1 client.21711 shutdown 2024-09-19T06:29:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51b654e740 1 -- 172.21.15.5:0/2034447551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 msgr2=0x563b3d878d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51b654e740 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d878d30 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f51a800fda0 tx=0x7f51a809bb50 comp rx=0 tx=0).stop 2024-09-19T06:29:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51b654e740 1 -- 172.21.15.5:0/2034447551 shutdown_connections 2024-09-19T06:29:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51b654e740 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563b3d888ea0 0x563b3d88b2d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51b654e740 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d8765d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51b654e740 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d875fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51957fa640 0 client.21711 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:29:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51b654e740 1 --2- 172.21.15.5:0/2034447551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d878d30 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51b654e740 1 -- 172.21.15.5:0/2034447551 >> 172.21.15.5:0/2034447551 conn(0x563b3d837310 msgr2=0x563b3d870960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51b654e740 1 -- 172.21.15.5:0/2034447551 shutdown_connections 2024-09-19T06:29:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51b654e740 1 -- 172.21.15.5:0/2034447551 wait complete. 2024-09-19T06:29:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.894+0000 7f51b654e740 20 client.21711 trim_cache size 0 max 16384 2024-09-19T06:29:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b654e740 1 Processor -- start 2024-09-19T06:29:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b654e740 1 -- start start 2024-09-19T06:29:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b654e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d869bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b654e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d871120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b654e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d871660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b654e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac1ad6f0 con 0x563b3d86e5d0 2024-09-19T06:29:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b654e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac1af2f0 con 0x563b3d879370 2024-09-19T06:29:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b654e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac1addf0 con 0x563b3d86fb40 2024-09-19T06:29:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b1f11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d869bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b1f11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d869bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39138/0 (socket says 172.21.15.5:39138) 2024-09-19T06:29:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b1f11640 1 -- 172.21.15.5:0/3601013515 learned_addr learned my addr 172.21.15.5:0/3601013515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b2712640 1 --2- 172.21.15.5:0/3601013515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d871660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b1710640 1 --2- 172.21.15.5:0/3601013515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d871120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b1f11640 1 -- 172.21.15.5:0/3601013515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 msgr2=0x563b3d871120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b1f11640 1 --2- 172.21.15.5:0/3601013515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d871120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b1f11640 1 -- 172.21.15.5:0/3601013515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 msgr2=0x563b3d871660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b1f11640 1 --2- 172.21.15.5:0/3601013515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d871660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b1f11640 1 -- 172.21.15.5:0/3601013515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b3d88b950 con 0x563b3d86e5d0 2024-09-19T06:29:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b1710640 1 --2- 172.21.15.5:0/3601013515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d871120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:29:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51b1f11640 1 --2- 172.21.15.5:0/3601013515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d869bf0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f51a809bde0 tx=0x7f51a800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dd7c5dd0ac768b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.908+0000 7f51977fe640 1 -- 172.21.15.5:0/3601013515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a80b4070 con 0x563b3d86e5d0 2024-09-19T06:29:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.909+0000 7f51977fe640 1 -- 172.21.15.5:0/3601013515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51a80a04e0 con 0x563b3d86e5d0 2024-09-19T06:29:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.909+0000 7f51977fe640 1 -- 172.21.15.5:0/3601013515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8097e80 con 0x563b3d86e5d0 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.909+0000 7f51b654e740 1 -- 172.21.15.5:0/3601013515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 msgr2=0x563b3d869bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.909+0000 7f51b654e740 1 --2- 172.21.15.5:0/3601013515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d869bf0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f51a809bde0 tx=0x7f51a800a560 comp rx=0 tx=0).stop 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.909+0000 7f51b654e740 1 -- 172.21.15.5:0/3601013515 shutdown_connections 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.909+0000 7f51b654e740 1 --2- 172.21.15.5:0/3601013515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d871660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.909+0000 7f51b654e740 1 --2- 172.21.15.5:0/3601013515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d871120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.909+0000 7f51b654e740 1 --2- 172.21.15.5:0/3601013515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d869bf0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.909+0000 7f51b654e740 1 -- 172.21.15.5:0/3601013515 >> 172.21.15.5:0/3601013515 conn(0x563b3d837310 msgr2=0x563b3d8694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.909+0000 7f51b654e740 1 -- 172.21.15.5:0/3601013515 shutdown_connections 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.909+0000 7f51b654e740 1 -- 172.21.15.5:0/3601013515 wait complete. 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b654e740 1 -- start start 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b654e740 1 Processor -- start 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b654e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d881bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b654e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d87cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b654e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d87d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b654e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac1aec20 con 0x563b3d86e5d0 2024-09-19T06:29:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b654e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac1ae520 con 0x563b3d879370 2024-09-19T06:29:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b654e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac1ab6f0 con 0x563b3d86fb40 2024-09-19T06:29:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b654e740 10 client.0 _ll_register_callbacks cb 0x7f51a809bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:29:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b1f11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d881bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b1f11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d881bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39140/0 (socket says 172.21.15.5:39140) 2024-09-19T06:29:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b1f11640 1 -- 172.21.15.5:0/2020796849 learned_addr learned my addr 172.21.15.5:0/2020796849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b1710640 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d87cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b2712640 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d87d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b1f11640 1 -- 172.21.15.5:0/2020796849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 msgr2=0x563b3d87cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b1f11640 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d87cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b1f11640 1 -- 172.21.15.5:0/2020796849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 msgr2=0x563b3d87d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.910+0000 7f51b1f11640 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d87d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b1f11640 1 -- 172.21.15.5:0/2020796849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b3d87dcb0 con 0x563b3d86e5d0 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b1710640 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d87cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b2712640 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d87d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b1f11640 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d881bf0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f51a809bcb0 tx=0x7f51a800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcc780bd28c4d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51957fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51957fa640 1 -- 172.21.15.5:0/2020796849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a80b4050 con 0x563b3d86e5d0 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51957fa640 1 -- 172.21.15.5:0/2020796849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7f51a809b580 con 0x563b3d86e5d0 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b654e740 1 -- 172.21.15.5:0/2020796849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51a80a04e0 con 0x563b3d86e5d0 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51957fa640 1 -- 172.21.15.5:0/2020796849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f51a809b780 con 0x563b3d86e5d0 2024-09-19T06:29:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b654e740 10 client.21717 fetch_fsmap learned FSMap version 66 2024-09-19T06:29:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b654e740 1 -- 172.21.15.5:0/2020796849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x563b3d8f3b70 con 0x563b3d86e5d0 2024-09-19T06:29:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51957fa640 1 -- 172.21.15.5:0/2020796849 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51a80a04e0 con 0x563b3d86e5d0 2024-09-19T06:29:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51957fa640 1 -- 172.21.15.5:0/2020796849 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 8394+0+0 (secure 0 0 0) 0x7f51a8081070 con 0x563b3d86e5d0 2024-09-19T06:29:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b654e740 10 client.21717 fetch_fsmap finished waiting for FSMap version 66 2024-09-19T06:29:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b654e740 10 client.21717 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T06:29:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b654e740 20 client.21717 populate_metadata read hostname 'smithi005' 2024-09-19T06:29:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b654e740 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563b3d8f4040 0x563b3d8f6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b654e740 4 client.21717 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:29:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.911+0000 7f51b654e740 1 -- 172.21.15.5:0/2020796849 --> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b3d7dbd70 con 0x563b3d8f4040 2024-09-19T06:29:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.912+0000 7f51b1710640 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563b3d8f4040 0x563b3d8f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.912+0000 7f51b1710640 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563b3d8f4040 0x563b3d8f6430 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=44674b7b217d0f51 server_cookie=58a09884cc7986f4 in_seq=0 out_seq=0 2024-09-19T06:29:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.912+0000 7f51957fa640 10 client.21717 ms_handle_connect on v2:172.21.15.157:6834/2070926870 2024-09-19T06:29:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.912+0000 7f51957fa640 1 -- 172.21.15.5:0/2020796849 <== mds.3 v2:172.21.15.157:6834/2070926870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b3d7dbd70 con 0x563b3d8f4040 2024-09-19T06:29:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.912+0000 7f51957fa640 10 client.21717 handle_command_reply: tid=0 2024-09-19T06:29:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.912+0000 7f51b654e740 1 client.21717 shutdown 2024-09-19T06:29:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.912+0000 7f51b654e740 1 -- 172.21.15.5:0/2020796849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 msgr2=0x563b3d881bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.912+0000 7f51b654e740 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d881bf0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f51a809bcb0 tx=0x7f51a800cf10 comp rx=0 tx=0).stop 2024-09-19T06:29:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b654e740 1 -- 172.21.15.5:0/2020796849 shutdown_connections 2024-09-19T06:29:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b654e740 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.157:6834/2070926870,v1:172.21.15.157:6835/2070926870] conn(0x563b3d8f4040 0x563b3d8f6430 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b654e740 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b3d879370 0x563b3d87d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b654e740 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563b3d86fb40 0x563b3d87cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51957fa640 0 client.21717 ms_handle_reset on v2:172.21.15.157:6834/2070926870 2024-09-19T06:29:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b654e740 1 --2- 172.21.15.5:0/2020796849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b3d86e5d0 0x563b3d881bf0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b654e740 1 -- 172.21.15.5:0/2020796849 >> 172.21.15.5:0/2020796849 conn(0x563b3d837310 msgr2=0x563b3d869d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b654e740 1 -- 172.21.15.5:0/2020796849 shutdown_connections 2024-09-19T06:29:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b654e740 1 -- 172.21.15.5:0/2020796849 wait complete. 2024-09-19T06:29:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b654e740 20 client.21717 trim_cache size 0 max 16384 2024-09-19T06:29:05.923 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:29:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b4175640 1 -- 172.21.15.5:0/1602218498 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f51980a33b0 msgr2=0x7f51980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b4175640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f51980a33b0 0x7f51980a5870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f51a40098b0 tx=0x7f51a4009840 comp rx=0 tx=0).stop 2024-09-19T06:29:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b4175640 1 -- 172.21.15.5:0/1602218498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac102d40 msgr2=0x7f51ac198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b4175640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac102d40 0x7f51ac198300 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f51a800c920 tx=0x7f51a800cde0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b4175640 1 -- 172.21.15.5:0/1602218498 shutdown_connections 2024-09-19T06:29:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b4175640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f51980a33b0 0x7f51980a5870 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b4175640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51ac10c170 0x7f51ac19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b4175640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51ac103660 0x7f51ac198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b4175640 1 --2- 172.21.15.5:0/1602218498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac102d40 0x7f51ac198300 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:05.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.913+0000 7f51b4175640 1 -- 172.21.15.5:0/1602218498 >> 172.21.15.5:0/1602218498 conn(0x7f51ac0fc7a0 msgr2=0x7f51ac0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:05.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.914+0000 7f51b4175640 1 -- 172.21.15.5:0/1602218498 shutdown_connections 2024-09-19T06:29:05.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:05.914+0000 7f51b4175640 1 -- 172.21.15.5:0/1602218498 wait complete. 2024-09-19T06:29:06.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:06 smithi005 ceph-mon[30955]: pgmap v814: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 27 op/s 2024-09-19T06:29:06.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:06 smithi049 ceph-mon[34640]: pgmap v814: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 27 op/s 2024-09-19T06:29:06.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:06 smithi157 ceph-mon[35346]: pgmap v814: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 27 op/s 2024-09-19T06:29:06.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:29:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.592+0000 7fba7762a640 1 -- 172.21.15.5:0/1512745972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7010c180 msgr2=0x7fba7010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.592+0000 7fba7762a640 1 --2- 172.21.15.5:0/1512745972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7010c180 0x7fba7010e600 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fba68009ca0 tx=0x7fba6802a950 comp rx=0 tx=0).stop 2024-09-19T06:29:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.592+0000 7fba7762a640 1 -- 172.21.15.5:0/1512745972 shutdown_connections 2024-09-19T06:29:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.592+0000 7fba7762a640 1 --2- 172.21.15.5:0/1512745972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7010c180 0x7fba7010e600 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.592+0000 7fba7762a640 1 --2- 172.21.15.5:0/1512745972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba701036c0 0x7fba7010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.592+0000 7fba7762a640 1 --2- 172.21.15.5:0/1512745972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba70102da0 0x7fba70103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.592+0000 7fba7762a640 1 -- 172.21.15.5:0/1512745972 >> 172.21.15.5:0/1512745972 conn(0x7fba700fc7a0 msgr2=0x7fba700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.592+0000 7fba7762a640 1 -- 172.21.15.5:0/1512745972 shutdown_connections 2024-09-19T06:29:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.592+0000 7fba7762a640 1 -- 172.21.15.5:0/1512745972 wait complete. 2024-09-19T06:29:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.593+0000 7fba7762a640 1 Processor -- start 2024-09-19T06:29:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.593+0000 7fba7762a640 1 -- start start 2024-09-19T06:29:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.593+0000 7fba7762a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba70102da0 0x7fba70100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.593+0000 7fba7762a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba701036c0 0x7fba70100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.593+0000 7fba7762a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7010c180 0x7fba70101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.593+0000 7fba7762a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba70111dd0 con 0x7fba70102da0 2024-09-19T06:29:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.593+0000 7fba7762a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fba70111c50 con 0x7fba701036c0 2024-09-19T06:29:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.593+0000 7fba7762a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba70111f50 con 0x7fba7010c180 2024-09-19T06:29:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.593+0000 7fba753c6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba70102da0 0x7fba70100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.593+0000 7fba753c6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba70102da0 0x7fba70100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39166/0 (socket says 172.21.15.5:39166) 2024-09-19T06:29:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.593+0000 7fba753c6640 1 -- 172.21.15.5:0/3901319350 learned_addr learned my addr 172.21.15.5:0/3901319350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba75bc7640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7010c180 0x7fba70101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba74bc5640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba701036c0 0x7fba70100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba753c6640 1 -- 172.21.15.5:0/3901319350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7010c180 msgr2=0x7fba70101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba753c6640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7010c180 0x7fba70101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba753c6640 1 -- 172.21.15.5:0/3901319350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba701036c0 msgr2=0x7fba70100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba753c6640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba701036c0 0x7fba70100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba753c6640 1 -- 172.21.15.5:0/3901319350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba701a66b0 con 0x7fba70102da0 2024-09-19T06:29:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba75bc7640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7010c180 0x7fba70101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba753c6640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba70102da0 0x7fba70100a00 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c920 tx=0x7fba6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba667fc640 1 -- 172.21.15.5:0/3901319350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c007c70 con 0x7fba70102da0 2024-09-19T06:29:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba7762a640 1 -- 172.21.15.5:0/3901319350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba701a69a0 con 0x7fba70102da0 2024-09-19T06:29:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba667fc640 1 -- 172.21.15.5:0/3901319350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba6c007e10 con 0x7fba70102da0 2024-09-19T06:29:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba667fc640 1 -- 172.21.15.5:0/3901319350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c018820 con 0x7fba70102da0 2024-09-19T06:29:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.594+0000 7fba7762a640 1 -- 172.21.15.5:0/3901319350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba701a6ee0 con 0x7fba70102da0 2024-09-19T06:29:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.596+0000 7fba667fc640 1 -- 172.21.15.5:0/3901319350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fba6c0189c0 con 0x7fba70102da0 2024-09-19T06:29:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.597+0000 7fba79a03740 1 Processor -- start 2024-09-19T06:29:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.597+0000 7fba79a03740 1 -- start start 2024-09-19T06:29:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.597+0000 7fba79a03740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.597+0000 7fba79a03740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682a75d0 0x5623682a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.597+0000 7fba79a03740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682b2370 0x5623682b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.597+0000 7fba79a03740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56236820d2f0 con 0x5623682a8b40 2024-09-19T06:29:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.597+0000 7fba79a03740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5623681b4cf0 con 0x5623682a75d0 2024-09-19T06:29:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.597+0000 7fba79a03740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5623682092b0 con 0x5623682b2370 2024-09-19T06:29:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.597+0000 7fba667fc640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fba5c0a33b0 0x7fba5c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.597+0000 7fba75bc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.597+0000 7fba75bc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682a7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39174/0 (socket says 172.21.15.5:39174) 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba75bc7640 1 -- 172.21.15.5:0/383706153 learned_addr learned my addr 172.21.15.5:0/383706153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba75bc7640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fba5c0a33b0 0x7fba5c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba667fc640 1 -- 172.21.15.5:0/3901319350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fba6c0cffa0 con 0x7fba70102da0 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba74bc5640 1 --2- 172.21.15.5:0/383706153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682b2370 0x5623682b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba753c6640 1 --2- 172.21.15.5:0/383706153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682a75d0 0x5623682a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba75bc7640 1 -- 172.21.15.5:0/383706153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682b2370 msgr2=0x5623682b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba75bc7640 1 --2- 172.21.15.5:0/383706153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682b2370 0x5623682b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba75bc7640 1 -- 172.21.15.5:0/383706153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682a75d0 msgr2=0x5623682a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba75bc7640 1 --2- 172.21.15.5:0/383706153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682a75d0 0x5623682a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba75bc7640 1 -- 172.21.15.5:0/383706153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623681993a0 con 0x5623682a8b40 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba75bc7640 1 --2- 172.21.15.5:0/383706153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682a7090 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fba6802f650 tx=0x7fba6802f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2602760fbf20b944 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba75bc7640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fba5c0a33b0 0x7fba5c0a5870 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fba6802f6f0 tx=0x7fba68042000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba5b7fe640 1 -- 172.21.15.5:0/383706153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6804c070 con 0x5623682a8b40 2024-09-19T06:29:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba5b7fe640 1 -- 172.21.15.5:0/383706153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba680047b0 con 0x5623682a8b40 2024-09-19T06:29:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.598+0000 7fba5b7fe640 1 -- 172.21.15.5:0/383706153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68034040 con 0x5623682a8b40 2024-09-19T06:29:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.599+0000 7fba79a03740 1 -- 172.21.15.5:0/383706153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 msgr2=0x5623682a7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.599+0000 7fba79a03740 1 --2- 172.21.15.5:0/383706153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682a7090 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fba6802f650 tx=0x7fba6802f680 comp rx=0 tx=0).stop 2024-09-19T06:29:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.599+0000 7fba79a03740 1 -- 172.21.15.5:0/383706153 shutdown_connections 2024-09-19T06:29:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.599+0000 7fba79a03740 1 --2- 172.21.15.5:0/383706153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682b2370 0x5623682b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.599+0000 7fba79a03740 1 --2- 172.21.15.5:0/383706153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682a75d0 0x5623682a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.599+0000 7fba79a03740 1 --2- 172.21.15.5:0/383706153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682a7090 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.599+0000 7fba79a03740 1 -- 172.21.15.5:0/383706153 >> 172.21.15.5:0/383706153 conn(0x562368270310 msgr2=0x5623682a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.599+0000 7fba79a03740 1 -- 172.21.15.5:0/383706153 shutdown_connections 2024-09-19T06:29:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.599+0000 7fba79a03740 1 -- 172.21.15.5:0/383706153 wait complete. 2024-09-19T06:29:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba79a03740 1 -- start start 2024-09-19T06:29:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba79a03740 1 Processor -- start 2024-09-19T06:29:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba79a03740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 0x5623682b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba79a03740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba79a03740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba79a03740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5623682b5010 con 0x5623682a8b40 2024-09-19T06:29:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba79a03740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5623681c8060 con 0x5623682b2370 2024-09-19T06:29:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba79a03740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5623682b5190 con 0x5623682a75d0 2024-09-19T06:29:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba79a03740 10 client.0 _ll_register_callbacks cb 0x562367f1b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:29:07.671 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:29:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba74bc5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba74bc5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39182/0 (socket says 172.21.15.5:39182) 2024-09-19T06:29:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba74bc5640 1 -- 172.21.15.5:0/1243594668 learned_addr learned my addr 172.21.15.5:0/1243594668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba753c6640 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 0x5623682b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.600+0000 7fba75bc7640 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba74bc5640 1 -- 172.21.15.5:0/1243594668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 msgr2=0x5623682b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba74bc5640 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 0x5623682b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba74bc5640 1 -- 172.21.15.5:0/1243594668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 msgr2=0x5623682af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba74bc5640 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba74bc5640 1 -- 172.21.15.5:0/1243594668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562367f7e3c0 con 0x5623682a8b40 2024-09-19T06:29:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba75bc7640 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:29:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba74bc5640 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682af040 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fba6000cd60 tx=0x7fba600039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2803029b9b34ffca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:29:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba58ff9640 1 -- 172.21.15.5:0/1243594668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60011910 con 0x5623682a8b40 2024-09-19T06:29:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba58ff9640 1 -- 172.21.15.5:0/1243594668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fba60011ab0 con 0x5623682a8b40 2024-09-19T06:29:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba79a03740 1 -- 172.21.15.5:0/1243594668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba680047b0 con 0x5623682a8b40 2024-09-19T06:29:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba58ff9640 1 -- 172.21.15.5:0/1243594668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fba60011cb0 con 0x5623682a8b40 2024-09-19T06:29:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba79a03740 10 client.21729 fetch_fsmap learned FSMap version 66 2024-09-19T06:29:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba79a03740 1 -- 172.21.15.5:0/1243594668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5623682c0b40 con 0x5623682a8b40 2024-09-19T06:29:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.601+0000 7fba58ff9640 1 -- 172.21.15.5:0/1243594668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba680047b0 con 0x5623682a8b40 2024-09-19T06:29:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.602+0000 7fba58ff9640 1 -- 172.21.15.5:0/1243594668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 8394+0+0 (secure 0 0 0) 0x7fba60018070 con 0x5623682a8b40 2024-09-19T06:29:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.602+0000 7fba79a03740 10 client.21729 fetch_fsmap finished waiting for FSMap version 66 2024-09-19T06:29:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.602+0000 7fba79a03740 10 client.21729 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:29:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.602+0000 7fba79a03740 20 client.21729 populate_metadata read hostname 'smithi005' 2024-09-19T06:29:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.602+0000 7fba79a03740 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5623682c20f0 0x5623682c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.602+0000 7fba79a03740 4 client.21729 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:29:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.602+0000 7fba79a03740 1 -- 172.21.15.5:0/1243594668 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56236821a860 con 0x5623682c20f0 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.602+0000 7fba753c6640 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5623682c20f0 0x5623682c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.602+0000 7fba753c6640 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5623682c20f0 0x5623682c4520 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16c22eae720522f0 server_cookie=7291d635bec74b60 in_seq=0 out_seq=0 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.602+0000 7fba58ff9640 10 client.21729 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.603+0000 7fba58ff9640 1 -- 172.21.15.5:0/1243594668 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56236821a860 con 0x5623682c20f0 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.603+0000 7fba58ff9640 10 client.21729 handle_command_reply: tid=0 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.603+0000 7fba79a03740 1 client.21729 shutdown 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.603+0000 7fba79a03740 1 -- 172.21.15.5:0/1243594668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 msgr2=0x5623682af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.603+0000 7fba79a03740 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682af040 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fba6000cd60 tx=0x7fba600039c0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.603+0000 7fba79a03740 1 -- 172.21.15.5:0/1243594668 shutdown_connections 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.603+0000 7fba79a03740 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x5623682c20f0 0x5623682c4520 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.603+0000 7fba79a03740 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.603+0000 7fba79a03740 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682af040 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.603+0000 7fba79a03740 1 --2- 172.21.15.5:0/1243594668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 0x5623682b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.603+0000 7fba58ff9640 0 client.21729 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:29:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.603+0000 7fba79a03740 1 -- 172.21.15.5:0/1243594668 >> 172.21.15.5:0/1243594668 conn(0x562368270310 msgr2=0x5623682a9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.604+0000 7fba79a03740 1 -- 172.21.15.5:0/1243594668 shutdown_connections 2024-09-19T06:29:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.604+0000 7fba79a03740 1 -- 172.21.15.5:0/1243594668 wait complete. 2024-09-19T06:29:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.604+0000 7fba79a03740 20 client.21729 trim_cache size 0 max 16384 2024-09-19T06:29:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba79a03740 1 Processor -- start 2024-09-19T06:29:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba79a03740 1 -- start start 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba79a03740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 0x5623682be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba79a03740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba79a03740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682aa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba79a03740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba701af9c0 con 0x5623682a8b40 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba79a03740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fba701b15c0 con 0x5623682b2370 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba79a03740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba701b00c0 con 0x5623682a75d0 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba74bc5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba74bc5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39198/0 (socket says 172.21.15.5:39198) 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba74bc5640 1 -- 172.21.15.5:0/3013330569 learned_addr learned my addr 172.21.15.5:0/3013330569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba753c6640 1 --2- 172.21.15.5:0/3013330569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 0x5623682be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba75bc7640 1 --2- 172.21.15.5:0/3013330569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba74bc5640 1 -- 172.21.15.5:0/3013330569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 msgr2=0x5623682be2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba74bc5640 1 --2- 172.21.15.5:0/3013330569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 0x5623682be2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba74bc5640 1 -- 172.21.15.5:0/3013330569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 msgr2=0x5623682aa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba74bc5640 1 --2- 172.21.15.5:0/3013330569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682aa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba74bc5640 1 -- 172.21.15.5:0/3013330569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623682a2310 con 0x5623682a8b40 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba75bc7640 1 --2- 172.21.15.5:0/3013330569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682aa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.617+0000 7fba74bc5640 1 --2- 172.21.15.5:0/3013330569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682ae900 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fba60004260 tx=0x7fba600229a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89011c59abd65672 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.618+0000 7fba5affd640 1 -- 172.21.15.5:0/3013330569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60022c90 con 0x5623682a8b40 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.618+0000 7fba5affd640 1 -- 172.21.15.5:0/3013330569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba60022e30 con 0x5623682a8b40 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.618+0000 7fba5affd640 1 -- 172.21.15.5:0/3013330569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba600055b0 con 0x5623682a8b40 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.618+0000 7fba79a03740 1 -- 172.21.15.5:0/3013330569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 msgr2=0x5623682ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.618+0000 7fba79a03740 1 --2- 172.21.15.5:0/3013330569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682ae900 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fba60004260 tx=0x7fba600229a0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.618+0000 7fba79a03740 1 -- 172.21.15.5:0/3013330569 shutdown_connections 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.618+0000 7fba79a03740 1 --2- 172.21.15.5:0/3013330569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682aa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.618+0000 7fba79a03740 1 --2- 172.21.15.5:0/3013330569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682ae900 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.618+0000 7fba79a03740 1 --2- 172.21.15.5:0/3013330569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 0x5623682be2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.618+0000 7fba79a03740 1 -- 172.21.15.5:0/3013330569 >> 172.21.15.5:0/3013330569 conn(0x562368270310 msgr2=0x5623682ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.618+0000 7fba79a03740 1 -- 172.21.15.5:0/3013330569 shutdown_connections 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.618+0000 7fba79a03740 1 -- 172.21.15.5:0/3013330569 wait complete. 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba79a03740 1 -- start start 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba79a03740 1 Processor -- start 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba79a03740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 0x5623682ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba79a03740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba79a03740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682aba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba79a03740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba701b0ef0 con 0x5623682a8b40 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba79a03740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fba701b07f0 con 0x5623682b2370 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba79a03740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba701ad9c0 con 0x5623682a75d0 2024-09-19T06:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba79a03740 10 client.0 _ll_register_callbacks cb 0x7fba60004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba74bc5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba74bc5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682bac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39204/0 (socket says 172.21.15.5:39204) 2024-09-19T06:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba74bc5640 1 -- 172.21.15.5:0/863554586 learned_addr learned my addr 172.21.15.5:0/863554586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.619+0000 7fba753c6640 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 0x5623682ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba75bc7640 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba74bc5640 1 -- 172.21.15.5:0/863554586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 msgr2=0x5623682ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba74bc5640 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 0x5623682ba6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba74bc5640 1 -- 172.21.15.5:0/863554586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 msgr2=0x5623682aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba74bc5640 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682aba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba74bc5640 1 -- 172.21.15.5:0/863554586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623682ac240 con 0x5623682a8b40 2024-09-19T06:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba74bc5640 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682bac10 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fba6000cd60 tx=0x7fba6000c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f32ef14b5114d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba58ff9640 1 -- 172.21.15.5:0/863554586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60039060 con 0x5623682a8b40 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba79a03740 1 -- 172.21.15.5:0/863554586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba60022e30 con 0x5623682a8b40 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba58ff9640 1 -- 172.21.15.5:0/863554586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8550+0+0 (secure 0 0 0) 0x7fba60011920 con 0x5623682a8b40 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba58ff9640 1 -- 172.21.15.5:0/863554586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fba60022740 con 0x5623682a8b40 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba79a03740 10 client.21735 fetch_fsmap learned FSMap version 66 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba79a03740 1 -- 172.21.15.5:0/863554586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56236832c9a0 con 0x5623682a8b40 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba58ff9640 1 -- 172.21.15.5:0/863554586 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba60022e30 con 0x5623682a8b40 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.620+0000 7fba58ff9640 1 -- 172.21.15.5:0/863554586 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 8394+0+0 (secure 0 0 0) 0x7fba60029070 con 0x5623682a8b40 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.621+0000 7fba79a03740 10 client.21735 fetch_fsmap finished waiting for FSMap version 66 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.621+0000 7fba79a03740 10 client.21735 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.621+0000 7fba79a03740 20 client.21735 populate_metadata read hostname 'smithi005' 2024-09-19T06:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.621+0000 7fba79a03740 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56236832d030 0x56236832f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.621+0000 7fba79a03740 4 client.21735 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.621+0000 7fba79a03740 1 -- 172.21.15.5:0/863554586 --> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562368214d70 con 0x56236832d030 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.621+0000 7fba753c6640 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56236832d030 0x56236832f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.621+0000 7fba753c6640 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56236832d030 0x56236832f450 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f70d365b2cba5a server_cookie=4281d3c3a8e58bd4 in_seq=0 out_seq=0 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.621+0000 7fba58ff9640 10 client.21735 ms_handle_connect on v2:172.21.15.157:6836/3264499329 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.621+0000 7fba58ff9640 1 -- 172.21.15.5:0/863554586 <== mds.2 v2:172.21.15.157:6836/3264499329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562368214d70 con 0x56236832d030 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.621+0000 7fba58ff9640 10 client.21735 handle_command_reply: tid=0 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.621+0000 7fba79a03740 1 client.21735 shutdown 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba79a03740 1 -- 172.21.15.5:0/863554586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 msgr2=0x5623682bac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba79a03740 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682bac10 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fba6000cd60 tx=0x7fba6000c3b0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba79a03740 1 -- 172.21.15.5:0/863554586 shutdown_connections 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba79a03740 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.157:6836/3264499329,v1:172.21.15.157:6837/3264499329] conn(0x56236832d030 0x56236832f450 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba79a03740 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623682b2370 0x5623682aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba79a03740 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623682a8b40 0x5623682bac10 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba79a03740 1 --2- 172.21.15.5:0/863554586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623682a75d0 0x5623682ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba79a03740 1 -- 172.21.15.5:0/863554586 >> 172.21.15.5:0/863554586 conn(0x562368270310 msgr2=0x5623682ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba58ff9640 0 client.21735 ms_handle_reset on v2:172.21.15.157:6836/3264499329 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba79a03740 1 -- 172.21.15.5:0/863554586 shutdown_connections 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba79a03740 1 -- 172.21.15.5:0/863554586 wait complete. 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba79a03740 20 client.21735 trim_cache size 0 max 16384 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba7762a640 1 -- 172.21.15.5:0/3901319350 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fba5c0a33b0 msgr2=0x7fba5c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba7762a640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fba5c0a33b0 0x7fba5c0a5870 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fba6802f6f0 tx=0x7fba68042000 comp rx=0 tx=0).stop 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba7762a640 1 -- 172.21.15.5:0/3901319350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba70102da0 msgr2=0x7fba70100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba7762a640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba70102da0 0x7fba70100a00 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c920 tx=0x7fba6c00cde0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba7762a640 1 -- 172.21.15.5:0/3901319350 shutdown_connections 2024-09-19T06:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba7762a640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fba5c0a33b0 0x7fba5c0a5870 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba7762a640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7010c180 0x7fba70101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba7762a640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba701036c0 0x7fba70100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba7762a640 1 --2- 172.21.15.5:0/3901319350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba70102da0 0x7fba70100a00 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:07.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba7762a640 1 -- 172.21.15.5:0/3901319350 >> 172.21.15.5:0/3901319350 conn(0x7fba700fc7a0 msgr2=0x7fba700fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:07.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.622+0000 7fba7762a640 1 -- 172.21.15.5:0/3901319350 shutdown_connections 2024-09-19T06:29:07.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:07.623+0000 7fba7762a640 1 -- 172.21.15.5:0/3901319350 wait complete. 2024-09-19T06:29:08.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:08 smithi005 ceph-mon[30955]: pgmap v815: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 32 op/s 2024-09-19T06:29:08.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:08 smithi049 ceph-mon[34640]: pgmap v815: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 32 op/s 2024-09-19T06:29:08.457 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:29:08.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:08 smithi157 ceph-mon[35346]: pgmap v815: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 32 op/s 2024-09-19T06:29:08.718 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.310+0000 7f1b9b231640 1 -- 172.21.15.5:0/549400012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b94109100 msgr2=0x7f1b94110c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.310+0000 7f1b9b231640 1 --2- 172.21.15.5:0/549400012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b94109100 0x7f1b94110c80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009ca0 tx=0x7f1b8c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.311+0000 7f1b9b231640 1 -- 172.21.15.5:0/549400012 shutdown_connections 2024-09-19T06:29:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.311+0000 7f1b9b231640 1 --2- 172.21.15.5:0/549400012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b94109100 0x7f1b94110c80 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.311+0000 7f1b9b231640 1 --2- 172.21.15.5:0/549400012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b94104980 0x7f1b941089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.311+0000 7f1b9b231640 1 --2- 172.21.15.5:0/549400012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b94103fd0 0x7f1b941043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.311+0000 7f1b9b231640 1 -- 172.21.15.5:0/549400012 >> 172.21.15.5:0/549400012 conn(0x7f1b940ffa30 msgr2=0x7f1b94101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.311+0000 7f1b9b231640 1 -- 172.21.15.5:0/549400012 shutdown_connections 2024-09-19T06:29:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.311+0000 7f1b9b231640 1 -- 172.21.15.5:0/549400012 wait complete. 2024-09-19T06:29:09.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.312+0000 7f1b9b231640 1 Processor -- start 2024-09-19T06:29:09.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.312+0000 7f1b9b231640 1 -- start start 2024-09-19T06:29:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.312+0000 7f1b9b231640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b94103fd0 0x7f1b9419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.312+0000 7f1b9b231640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b94104980 0x7f1b9419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.312+0000 7f1b9b231640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b94109100 0x7f1b9419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b9b231640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b94113fc0 con 0x7f1b94109100 2024-09-19T06:29:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b9b231640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b94113e40 con 0x7f1b94103fd0 2024-09-19T06:29:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b9b231640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1b94114140 con 0x7f1b94104980 2024-09-19T06:29:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b98fcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b94103fd0 0x7f1b9419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b98fcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b94103fd0 0x7f1b9419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33498/0 (socket says 172.21.15.5:33498) 2024-09-19T06:29:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b98fcd640 1 -- 172.21.15.5:0/3896392521 learned_addr learned my addr 172.21.15.5:0/3896392521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b93fff640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b94104980 0x7f1b9419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b997ce640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b94109100 0x7f1b9419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b98fcd640 1 -- 172.21.15.5:0/3896392521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b94104980 msgr2=0x7f1b9419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b94104980 0x7f1b9419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b98fcd640 1 -- 172.21.15.5:0/3896392521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b94109100 msgr2=0x7f1b9419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b94109100 0x7f1b9419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b98fcd640 1 -- 172.21.15.5:0/3896392521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9419f470 con 0x7f1b94103fd0 2024-09-19T06:29:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.313+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b94103fd0 0x7f1b9419a4c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000d950 tx=0x7f1b8000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.314+0000 7f1b91ffb640 1 -- 172.21.15.5:0/3896392521 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80017070 con 0x7f1b94103fd0 2024-09-19T06:29:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.314+0000 7f1b9b231640 1 -- 172.21.15.5:0/3896392521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9419f760 con 0x7f1b94103fd0 2024-09-19T06:29:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.314+0000 7f1b91ffb640 1 -- 172.21.15.5:0/3896392521 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b80009c30 con 0x7f1b94103fd0 2024-09-19T06:29:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.314+0000 7f1b91ffb640 1 -- 172.21.15.5:0/3896392521 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8000f620 con 0x7f1b94103fd0 2024-09-19T06:29:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.314+0000 7f1b9b231640 1 -- 172.21.15.5:0/3896392521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b941a7040 con 0x7f1b94103fd0 2024-09-19T06:29:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.317+0000 7f1b91ffb640 1 -- 172.21.15.5:0/3896392521 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1b800109a0 con 0x7f1b94103fd0 2024-09-19T06:29:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.318+0000 7f1b9d60a740 1 Processor -- start 2024-09-19T06:29:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.318+0000 7f1b9d60a740 1 -- start start 2024-09-19T06:29:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.318+0000 7f1b9d60a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c6b40 0x557c724c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.318+0000 7f1b9d60a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c55d0 0x557c724c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.318+0000 7f1b9d60a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.318+0000 7f1b9d60a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557c7242b2f0 con 0x557c724d0370 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.318+0000 7f1b9d60a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557c723d2cf0 con 0x557c724c6b40 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.318+0000 7f1b9d60a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557c724272b0 con 0x557c724c55d0 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.318+0000 7f1b91ffb640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1b840a33b0 0x7f1b840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.318+0000 7f1b997ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c6b40 0x557c724c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.319+0000 7f1b91ffb640 1 -- 172.21.15.5:0/3896392521 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8594+0+0 (secure 0 0 0) 0x7f1b800cfa70 con 0x7f1b94103fd0 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.319+0000 7f1b997ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c6b40 0x557c724c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33504/0 (socket says 172.21.15.5:33504) 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.319+0000 7f1b997ce640 1 -- 172.21.15.5:0/4112476659 learned_addr learned my addr 172.21.15.5:0/4112476659 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.319+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/4112476659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c55d0 0x557c724c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.319+0000 7f1b997ce640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1b840a33b0 0x7f1b840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.319+0000 7f1b98fcd640 1 -- 172.21.15.5:0/4112476659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c6b40 msgr2=0x557c724c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.319+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/4112476659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c6b40 0x557c724c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.319+0000 7f1b98fcd640 1 -- 172.21.15.5:0/4112476659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 msgr2=0x557c724d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.319+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/4112476659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.319+0000 7f1b98fcd640 1 -- 172.21.15.5:0/4112476659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c723b73a0 con 0x557c724c55d0 2024-09-19T06:29:09.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.320+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/4112476659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c55d0 0x557c724c5a10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1b800cd170 tx=0x7f1b8009bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a5e90d383727456 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.320+0000 7f1b997ce640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1b840a33b0 0x7f1b840a5870 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009a60 tx=0x7f1b8c02a8c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.320+0000 7f1b7effd640 1 -- 172.21.15.5:0/4112476659 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80020070 con 0x557c724c55d0 2024-09-19T06:29:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.320+0000 7f1b7effd640 1 -- 172.21.15.5:0/4112476659 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b800a00b0 con 0x557c724c55d0 2024-09-19T06:29:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.320+0000 7f1b7effd640 1 -- 172.21.15.5:0/4112476659 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80092b00 con 0x557c724c55d0 2024-09-19T06:29:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.321+0000 7f1b9d60a740 1 -- 172.21.15.5:0/4112476659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c55d0 msgr2=0x557c724c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.321+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/4112476659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c55d0 0x557c724c5a10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1b800cd170 tx=0x7f1b8009bc00 comp rx=0 tx=0).stop 2024-09-19T06:29:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.321+0000 7f1b9d60a740 1 -- 172.21.15.5:0/4112476659 shutdown_connections 2024-09-19T06:29:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.321+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/4112476659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.321+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/4112476659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c55d0 0x557c724c5a10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.321+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/4112476659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c6b40 0x557c724c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.321+0000 7f1b9d60a740 1 -- 172.21.15.5:0/4112476659 >> 172.21.15.5:0/4112476659 conn(0x557c7248e310 msgr2=0x557c724c1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.321+0000 7f1b9d60a740 1 -- 172.21.15.5:0/4112476659 shutdown_connections 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.321+0000 7f1b9d60a740 1 -- 172.21.15.5:0/4112476659 wait complete. 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.322+0000 7f1b9d60a740 1 -- start start 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.322+0000 7f1b9d60a740 1 Processor -- start 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.322+0000 7f1b9d60a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 0x557c724cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.322+0000 7f1b9d60a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c6b40 0x557c724cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.322+0000 7f1b9d60a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.322+0000 7f1b9d60a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557c724d2fb0 con 0x557c724d0370 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.322+0000 7f1b9d60a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557c723e6060 con 0x557c724c55d0 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.322+0000 7f1b9d60a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557c724d3130 con 0x557c724c6b40 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.322+0000 7f1b9d60a740 10 client.0 _ll_register_callbacks cb 0x557c721398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.322+0000 7f1b98fcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 0x557c724cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.322+0000 7f1b98fcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 0x557c724cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33518/0 (socket says 172.21.15.5:33518) 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.322+0000 7f1b98fcd640 1 -- 172.21.15.5:0/3980497608 learned_addr learned my addr 172.21.15.5:0/3980497608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.323+0000 7f1b93fff640 1 --2- 172.21.15.5:0/3980497608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c6b40 0x557c724cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.323+0000 7f1b98fcd640 1 -- 172.21.15.5:0/3980497608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c6b40 msgr2=0x557c724cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:09.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.323+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/3980497608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c6b40 0x557c724cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.323+0000 7f1b98fcd640 1 -- 172.21.15.5:0/3980497608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 msgr2=0x557c724cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:09.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.323+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/3980497608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.323+0000 7f1b98fcd640 1 -- 172.21.15.5:0/3980497608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c7219c3c0 con 0x557c724c55d0 2024-09-19T06:29:09.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.323+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/3980497608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 0x557c724cfd70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000c00 tx=0x7f1b80092720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bf3c6cb9333a25c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:09.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.323+0000 7f1b73fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:29:09.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.323+0000 7f1b73fff640 1 -- 172.21.15.5:0/3980497608 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80096070 con 0x557c724c55d0 2024-09-19T06:29:09.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.323+0000 7f1b73fff640 1 -- 172.21.15.5:0/3980497608 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8594+0+0 (secure 0 0 0) 0x7f1b800cd3f0 con 0x557c724c55d0 2024-09-19T06:29:09.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.323+0000 7f1b9d60a740 1 -- 172.21.15.5:0/3980497608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b800a00b0 con 0x557c724c55d0 2024-09-19T06:29:09.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.324+0000 7f1b73fff640 1 -- 172.21.15.5:0/3980497608 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f1b800b9050 con 0x557c724c55d0 2024-09-19T06:29:09.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.324+0000 7f1b9d60a740 10 client.31088 fetch_fsmap learned FSMap version 66 2024-09-19T06:29:09.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.324+0000 7f1b9d60a740 1 -- 172.21.15.5:0/3980497608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x557c724e7260 con 0x557c724c55d0 2024-09-19T06:29:09.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.324+0000 7f1b73fff640 1 -- 172.21.15.5:0/3980497608 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b800a4970 con 0x557c724c55d0 2024-09-19T06:29:09.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.324+0000 7f1b73fff640 1 -- 172.21.15.5:0/3980497608 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 66) ==== 8394+0+0 (secure 0 0 0) 0x7f1b80092c90 con 0x557c724c55d0 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.324+0000 7f1b9d60a740 10 client.31088 fetch_fsmap finished waiting for FSMap version 66 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.324+0000 7f1b9d60a740 10 client.31088 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.324+0000 7f1b9d60a740 20 client.31088 populate_metadata read hostname 'smithi005' 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.324+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/3980497608 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557c724e85b0 0x557c724ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.324+0000 7f1b9d60a740 4 client.31088 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.324+0000 7f1b9d60a740 1 -- 172.21.15.5:0/3980497608 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1b800a00b0 con 0x557c724e85b0 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.324+0000 7f1b93fff640 1 --2- 172.21.15.5:0/3980497608 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557c724e85b0 0x557c724ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.325+0000 7f1b73fff640 10 client.31088 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.325+0000 7f1b93fff640 1 --2- 172.21.15.5:0/3980497608 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557c724e85b0 0x557c724ea9a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77ab7e6f594a719 server_cookie=9ad5302859470f92 in_seq=0 out_seq=0 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.326+0000 7f1b73fff640 1 -- 172.21.15.5:0/3980497608 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f1b800a00b0 con 0x557c724e85b0 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.326+0000 7f1b73fff640 10 client.31088 handle_command_reply: tid=0 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.326+0000 7f1b9d60a740 1 client.31088 shutdown 2024-09-19T06:29:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.326+0000 7f1b9d60a740 1 -- 172.21.15.5:0/3980497608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 msgr2=0x557c724cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.326+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/3980497608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 0x557c724cfd70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000c00 tx=0x7f1b80092720 comp rx=0 tx=0).stop 2024-09-19T06:29:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.327+0000 7f1b9d60a740 1 -- 172.21.15.5:0/3980497608 shutdown_connections 2024-09-19T06:29:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.327+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/3980497608 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557c724e85b0 0x557c724ea9a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.327+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/3980497608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.327+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/3980497608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c6b40 0x557c724cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.327+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/3980497608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 0x557c724cfd70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.327+0000 7f1b9d60a740 1 -- 172.21.15.5:0/3980497608 >> 172.21.15.5:0/3980497608 conn(0x557c7248e310 msgr2=0x557c724c7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.327+0000 7f1b73fff640 0 client.31088 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:29:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.327+0000 7f1b9d60a740 1 -- 172.21.15.5:0/3980497608 shutdown_connections 2024-09-19T06:29:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.327+0000 7f1b9d60a740 1 -- 172.21.15.5:0/3980497608 wait complete. 2024-09-19T06:29:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.327+0000 7f1b9d60a740 20 client.31088 trim_cache size 0 max 16384 2024-09-19T06:29:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b9d60a740 1 Processor -- start 2024-09-19T06:29:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b9d60a740 1 -- start start 2024-09-19T06:29:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b9d60a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 0x557c724e4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b9d60a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c6b40 0x557c724c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b9d60a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b9d60a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941ab640 con 0x557c724d0370 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b9d60a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941ad240 con 0x557c724c55d0 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b9d60a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941abd40 con 0x557c724c6b40 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b98fcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 0x557c724e4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b93fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c6b40 0x557c724c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b98fcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 0x557c724e4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33528/0 (socket says 172.21.15.5:33528) 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b98fcd640 1 -- 172.21.15.5:0/1169426169 learned_addr learned my addr 172.21.15.5:0/1169426169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b997ce640 1 --2- 172.21.15.5:0/1169426169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b93fff640 1 -- 172.21.15.5:0/1169426169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 msgr2=0x557c724e4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b93fff640 1 --2- 172.21.15.5:0/1169426169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 0x557c724e4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b93fff640 1 -- 172.21.15.5:0/1169426169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 msgr2=0x557c724c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.345+0000 7f1b93fff640 1 --2- 172.21.15.5:0/1169426169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.346+0000 7f1b93fff640 1 -- 172.21.15.5:0/1169426169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c724cb570 con 0x557c724c6b40 2024-09-19T06:29:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.346+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/1169426169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 0x557c724e4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.346+0000 7f1b93fff640 1 --2- 172.21.15.5:0/1169426169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c6b40 0x557c724c8120 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x557c724ce660 tx=0x7f1b88014840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c862f0dc8dddd346 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.346+0000 7f1b7e7fc640 1 -- 172.21.15.5:0/1169426169 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88012450 con 0x557c724c6b40 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.346+0000 7f1b7e7fc640 1 -- 172.21.15.5:0/1169426169 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b88012aa0 con 0x557c724c6b40 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.346+0000 7f1b7e7fc640 1 -- 172.21.15.5:0/1169426169 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8801d830 con 0x557c724c6b40 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.347+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1169426169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c6b40 msgr2=0x557c724c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.347+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/1169426169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c6b40 0x557c724c8120 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x557c724ce660 tx=0x7f1b88014840 comp rx=0 tx=0).stop 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.347+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1169426169 shutdown_connections 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.347+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/1169426169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.347+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/1169426169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c6b40 0x557c724c8120 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.347+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/1169426169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c55d0 0x557c724e4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.347+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1169426169 >> 172.21.15.5:0/1169426169 conn(0x557c7248e310 msgr2=0x557c724cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.347+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1169426169 shutdown_connections 2024-09-19T06:29:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.347+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1169426169 wait complete. 2024-09-19T06:29:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.347+0000 7f1b9d60a740 1 -- start start 2024-09-19T06:29:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.348+0000 7f1b9d60a740 1 Processor -- start 2024-09-19T06:29:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.348+0000 7f1b9d60a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c55d0 0x557c724e0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.348+0000 7f1b9d60a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c6b40 0x557c724e1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.348+0000 7f1b9d60a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.348+0000 7f1b9d60a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941acb70 con 0x557c724d0370 2024-09-19T06:29:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.348+0000 7f1b9d60a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941ac470 con 0x557c724c6b40 2024-09-19T06:29:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.348+0000 7f1b98fcd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c55d0 0x557c724e0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.348+0000 7f1b93fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c6b40 0x557c724e1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.348+0000 7f1b98fcd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c55d0 0x557c724e0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34948/0 (socket says 172.21.15.5:34948) 2024-09-19T06:29:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.348+0000 7f1b98fcd640 1 -- 172.21.15.5:0/1453667001 learned_addr learned my addr 172.21.15.5:0/1453667001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.348+0000 7f1b9d60a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941a9660 con 0x557c724c55d0 2024-09-19T06:29:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.349+0000 7f1b997ce640 1 --2- 172.21.15.5:0/1453667001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.349+0000 7f1b9d60a740 10 client.0 _ll_register_callbacks cb 0x7f1b88005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:29:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.349+0000 7f1b93fff640 1 -- 172.21.15.5:0/1453667001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c55d0 msgr2=0x557c724e0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.349+0000 7f1b93fff640 1 --2- 172.21.15.5:0/1453667001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c55d0 0x557c724e0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.349+0000 7f1b93fff640 1 -- 172.21.15.5:0/1453667001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 msgr2=0x557c724c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.349+0000 7f1b93fff640 1 --2- 172.21.15.5:0/1453667001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.349+0000 7f1b93fff640 1 -- 172.21.15.5:0/1453667001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c724ca100 con 0x557c724c6b40 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.349+0000 7f1b93fff640 1 --2- 172.21.15.5:0/1453667001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c6b40 0x557c724e1370 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x557c724c6110 tx=0x7f1b88014340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c99596fc5183f81b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.349+0000 7f1b73fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.349+0000 7f1b73fff640 1 -- 172.21.15.5:0/1453667001 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88012450 con 0x557c724c6b40 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.349+0000 7f1b73fff640 1 -- 172.21.15.5:0/1453667001 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8594+0+0 (secure 0 0 0) 0x7f1b880125f0 con 0x557c724c6b40 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.349+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1453667001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b88012aa0 con 0x557c724c6b40 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.350+0000 7f1b73fff640 1 -- 172.21.15.5:0/1453667001 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f1b880127f0 con 0x557c724c6b40 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.350+0000 7f1b9d60a740 10 client.31094 fetch_fsmap learned FSMap version 66 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.350+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1453667001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x557c72552aa0 con 0x557c724c6b40 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.350+0000 7f1b73fff640 1 -- 172.21.15.5:0/1453667001 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b88012aa0 con 0x557c724c6b40 2024-09-19T06:29:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.350+0000 7f1b73fff640 1 -- 172.21.15.5:0/1453667001 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 66) ==== 8394+0+0 (secure 0 0 0) 0x7f1b88004bf0 con 0x557c724c6b40 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.350+0000 7f1b9d60a740 10 client.31094 fetch_fsmap finished waiting for FSMap version 66 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.350+0000 7f1b9d60a740 10 client.31094 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.350+0000 7f1b9d60a740 20 client.31094 populate_metadata read hostname 'smithi005' 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.350+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/1453667001 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557c72553120 0x557c72555540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.350+0000 7f1b9d60a740 4 client.31094 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.350+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1453667001 --> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c72438860 con 0x557c72553120 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.350+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/1453667001 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557c72553120 0x557c72555540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b73fff640 10 client.31094 ms_handle_connect on v2:172.21.15.157:6838/4294596962 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b98fcd640 1 --2- 172.21.15.5:0/1453667001 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557c72553120 0x557c72555540 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b83b9abdc181430 server_cookie=df857746e8a1454a in_seq=0 out_seq=0 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b73fff640 1 -- 172.21.15.5:0/1453667001 <== mds.1 v2:172.21.15.157:6838/4294596962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c72438860 con 0x557c72553120 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b73fff640 10 client.31094 handle_command_reply: tid=0 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b9d60a740 1 client.31094 shutdown 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1453667001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c6b40 msgr2=0x557c724e1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/1453667001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c6b40 0x557c724e1370 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x557c724c6110 tx=0x7f1b88014340 comp rx=0 tx=0).stop 2024-09-19T06:29:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1453667001 shutdown_connections 2024-09-19T06:29:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/1453667001 >> [v2:172.21.15.157:6838/4294596962,v1:172.21.15.157:6839/4294596962] conn(0x557c72553120 0x557c72555540 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b73fff640 0 client.31094 ms_handle_reset on v2:172.21.15.157:6838/4294596962 2024-09-19T06:29:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/1453667001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c724d0370 0x557c724c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/1453667001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c724c6b40 0x557c724e1370 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b9d60a740 1 --2- 172.21.15.5:0/1453667001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c724c55d0 0x557c724e0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1453667001 >> 172.21.15.5:0/1453667001 conn(0x557c7248e310 msgr2=0x557c724c04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1453667001 shutdown_connections 2024-09-19T06:29:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b9d60a740 1 -- 172.21.15.5:0/1453667001 wait complete. 2024-09-19T06:29:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.351+0000 7f1b9d60a740 20 client.31094 trim_cache size 0 max 16384 2024-09-19T06:29:09.359 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:29:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.352+0000 7f1b9b231640 1 -- 172.21.15.5:0/3896392521 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1b840a33b0 msgr2=0x7f1b840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.352+0000 7f1b9b231640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1b840a33b0 0x7f1b840a5870 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009a60 tx=0x7f1b8c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.352+0000 7f1b9b231640 1 -- 172.21.15.5:0/3896392521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b94103fd0 msgr2=0x7f1b9419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.352+0000 7f1b9b231640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b94103fd0 0x7f1b9419a4c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000d950 tx=0x7f1b8000de10 comp rx=0 tx=0).stop 2024-09-19T06:29:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.352+0000 7f1b9b231640 1 -- 172.21.15.5:0/3896392521 shutdown_connections 2024-09-19T06:29:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.352+0000 7f1b9b231640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b94109100 0x7f1b9419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.352+0000 7f1b9b231640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f1b840a33b0 0x7f1b840a5870 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.352+0000 7f1b9b231640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b94104980 0x7f1b9419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.352+0000 7f1b9b231640 1 --2- 172.21.15.5:0/3896392521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b94103fd0 0x7f1b9419a4c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.352+0000 7f1b9b231640 1 -- 172.21.15.5:0/3896392521 >> 172.21.15.5:0/3896392521 conn(0x7f1b940ffa30 msgr2=0x7f1b94100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.352+0000 7f1b9b231640 1 -- 172.21.15.5:0/3896392521 shutdown_connections 2024-09-19T06:29:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:09.352+0000 7f1b9b231640 1 -- 172.21.15.5:0/3896392521 wait complete. 2024-09-19T06:29:10.147 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:29:10.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:10 smithi005 ceph-mon[30955]: pgmap v816: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 27 op/s 2024-09-19T06:29:10.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:10 smithi005 ceph-mon[30955]: osdmap e124: 12 total, 12 up, 12 in 2024-09-19T06:29:10.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:10 smithi049 ceph-mon[34640]: pgmap v816: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 27 op/s 2024-09-19T06:29:10.397 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:10 smithi049 ceph-mon[34640]: osdmap e124: 12 total, 12 up, 12 in 2024-09-19T06:29:10.410 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:10.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:10 smithi157 ceph-mon[35346]: pgmap v816: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 27 op/s 2024-09-19T06:29:10.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:10 smithi157 ceph-mon[35346]: osdmap e124: 12 total, 12 up, 12 in 2024-09-19T06:29:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.878+0000 7f20961aa640 1 -- 172.21.15.5:0/2584796080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2090100a80 msgr2=0x7f2090100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.878+0000 7f20961aa640 1 --2- 172.21.15.5:0/2584796080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2090100a80 0x7f2090100ee0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f2084009930 tx=0x7f208402a420 comp rx=0 tx=0).stop 2024-09-19T06:29:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.879+0000 7f20961aa640 1 -- 172.21.15.5:0/2584796080 shutdown_connections 2024-09-19T06:29:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.879+0000 7f20961aa640 1 --2- 172.21.15.5:0/2584796080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2090101420 0x7f209010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.879+0000 7f20961aa640 1 --2- 172.21.15.5:0/2584796080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2090100a80 0x7f2090100ee0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.879+0000 7f20961aa640 1 --2- 172.21.15.5:0/2584796080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2090106a80 0x7f2090106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.879+0000 7f20961aa640 1 -- 172.21.15.5:0/2584796080 >> 172.21.15.5:0/2584796080 conn(0x7f20900fc7a0 msgr2=0x7f20900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.879+0000 7f20961aa640 1 -- 172.21.15.5:0/2584796080 shutdown_connections 2024-09-19T06:29:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.879+0000 7f20961aa640 1 -- 172.21.15.5:0/2584796080 wait complete. 2024-09-19T06:29:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.879+0000 7f20961aa640 1 Processor -- start 2024-09-19T06:29:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.879+0000 7f20961aa640 1 -- start start 2024-09-19T06:29:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f20961aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2090100a80 0x7f209019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f20961aa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2090101420 0x7f209019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f20961aa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2090106a80 0x7f209019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f20961aa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2090110d30 con 0x7f2090100a80 2024-09-19T06:29:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f20961aa640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2090110bb0 con 0x7f2090101420 2024-09-19T06:29:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f20961aa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2090110eb0 con 0x7f2090106a80 2024-09-19T06:29:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2090100a80 0x7f209019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2090100a80 0x7f209019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39316/0 (socket says 172.21.15.5:39316) 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208f7fe640 1 -- 172.21.15.5:0/466224365 learned_addr learned my addr 172.21.15.5:0/466224365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208effd640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2090101420 0x7f209019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208ffff640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2090106a80 0x7f209019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208f7fe640 1 -- 172.21.15.5:0/466224365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2090106a80 msgr2=0x7f209019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208f7fe640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2090106a80 0x7f209019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208f7fe640 1 -- 172.21.15.5:0/466224365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2090101420 msgr2=0x7f209019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208f7fe640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2090101420 0x7f209019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208f7fe640 1 -- 172.21.15.5:0/466224365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209019f510 con 0x7f2090100a80 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208ffff640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2090106a80 0x7f209019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208f7fe640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2090100a80 0x7f209019a560 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f208000d950 tx=0x7f208000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208cff9640 1 -- 172.21.15.5:0/466224365 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080017070 con 0x7f2090100a80 2024-09-19T06:29:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f20961aa640 1 -- 172.21.15.5:0/466224365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209019f800 con 0x7f2090100a80 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208cff9640 1 -- 172.21.15.5:0/466224365 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2080009c30 con 0x7f2090100a80 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f208cff9640 1 -- 172.21.15.5:0/466224365 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208000f620 con 0x7f2090100a80 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.880+0000 7f20961aa640 1 -- 172.21.15.5:0/466224365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20901a70e0 con 0x7f2090100a80 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.883+0000 7f208cff9640 1 -- 172.21.15.5:0/466224365 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f20800109a0 con 0x7f2090100a80 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.883+0000 7f2098583740 1 Processor -- start 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.883+0000 7f2098583740 1 -- start start 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.883+0000 7f208cff9640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20780a33b0 0x7f20780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208ffff640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20780a33b0 0x7f20780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f2098583740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8d8b40 0x55c30b8d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208cff9640 1 -- 172.21.15.5:0/466224365 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8594+0+0 (secure 0 0 0) 0x7f20800cf950 con 0x7f2090100a80 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f2098583740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d75d0 0x55c30b8d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f2098583740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8e2370 0x55c30b8e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f2098583740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c30b83d2f0 con 0x55c30b8d8b40 2024-09-19T06:29:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f2098583740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c30b7e4cf0 con 0x55c30b8d75d0 2024-09-19T06:29:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f2098583740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c30b8392b0 con 0x55c30b8e2370 2024-09-19T06:29:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8d8b40 0x55c30b8d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8d8b40 0x55c30b8d7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39320/0 (socket says 172.21.15.5:39320) 2024-09-19T06:29:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208f7fe640 1 -- 172.21.15.5:0/4186896382 learned_addr learned my addr 172.21.15.5:0/4186896382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208effd640 1 --2- 172.21.15.5:0/4186896382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d75d0 0x55c30b8d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208f7fe640 1 -- 172.21.15.5:0/4186896382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8e2370 msgr2=0x55c30b8e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208f7fe640 1 --2- 172.21.15.5:0/4186896382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8e2370 0x55c30b8e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208f7fe640 1 -- 172.21.15.5:0/4186896382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d75d0 msgr2=0x55c30b8d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208f7fe640 1 --2- 172.21.15.5:0/4186896382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d75d0 0x55c30b8d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208f7fe640 1 -- 172.21.15.5:0/4186896382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c30b7c93a0 con 0x55c30b8d8b40 2024-09-19T06:29:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208ffff640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20780a33b0 0x7f20780a5870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2088005fd0 tx=0x7f208800d710 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.884+0000 7f208f7fe640 1 --2- 172.21.15.5:0/4186896382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8d8b40 0x55c30b8d7090 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f20800a4320 tx=0x7f208009b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91201f63e35b9f50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.885+0000 7f2075ffb640 1 -- 172.21.15.5:0/4186896382 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080020040 con 0x55c30b8d8b40 2024-09-19T06:29:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.885+0000 7f2075ffb640 1 -- 172.21.15.5:0/4186896382 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f208009bbe0 con 0x55c30b8d8b40 2024-09-19T06:29:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.885+0000 7f2075ffb640 1 -- 172.21.15.5:0/4186896382 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20800860f0 con 0x55c30b8d8b40 2024-09-19T06:29:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.885+0000 7f2098583740 1 -- 172.21.15.5:0/4186896382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8d8b40 msgr2=0x55c30b8d7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.885+0000 7f2098583740 1 --2- 172.21.15.5:0/4186896382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8d8b40 0x55c30b8d7090 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f20800a4320 tx=0x7f208009b850 comp rx=0 tx=0).stop 2024-09-19T06:29:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.885+0000 7f2098583740 1 -- 172.21.15.5:0/4186896382 shutdown_connections 2024-09-19T06:29:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.885+0000 7f2098583740 1 --2- 172.21.15.5:0/4186896382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8e2370 0x55c30b8e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.885+0000 7f2098583740 1 --2- 172.21.15.5:0/4186896382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d75d0 0x55c30b8d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.885+0000 7f2098583740 1 --2- 172.21.15.5:0/4186896382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8d8b40 0x55c30b8d7090 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.885+0000 7f2098583740 1 -- 172.21.15.5:0/4186896382 >> 172.21.15.5:0/4186896382 conn(0x55c30b8a0310 msgr2=0x55c30b8d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.885+0000 7f2098583740 1 -- 172.21.15.5:0/4186896382 shutdown_connections 2024-09-19T06:29:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.885+0000 7f2098583740 1 -- 172.21.15.5:0/4186896382 wait complete. 2024-09-19T06:29:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f2098583740 1 -- start start 2024-09-19T06:29:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f2098583740 1 Processor -- start 2024-09-19T06:29:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f2098583740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d75d0 0x55c30b8e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f2098583740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d8b40 0x55c30b8df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f2098583740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b8df650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f2098583740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c30b8e4fb0 con 0x55c30b8e2370 2024-09-19T06:29:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f2098583740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c30b7f8060 con 0x55c30b8d8b40 2024-09-19T06:29:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f2098583740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c30b8e5130 con 0x55c30b8d75d0 2024-09-19T06:29:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f2098583740 10 client.0 _ll_register_callbacks cb 0x55c30b54b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:29:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f208ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b8df650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f208ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b8df650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39322/0 (socket says 172.21.15.5:39322) 2024-09-19T06:29:10.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d75d0 0x55c30b8e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f208ffff640 1 -- 172.21.15.5:0/921467337 learned_addr learned my addr 172.21.15.5:0/921467337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:10.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.886+0000 7f208effd640 1 --2- 172.21.15.5:0/921467337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d8b40 0x55c30b8df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f208ffff640 1 -- 172.21.15.5:0/921467337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d75d0 msgr2=0x55c30b8e1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:10.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f208ffff640 1 --2- 172.21.15.5:0/921467337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d75d0 0x55c30b8e1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f208ffff640 1 -- 172.21.15.5:0/921467337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d8b40 msgr2=0x55c30b8df040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:10.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f208ffff640 1 --2- 172.21.15.5:0/921467337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d8b40 0x55c30b8df040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f208ffff640 1 -- 172.21.15.5:0/921467337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c30b5ae3c0 con 0x55c30b8e2370 2024-09-19T06:29:10.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f208ffff640 1 --2- 172.21.15.5:0/921467337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b8df650 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f2088011dc0 tx=0x7f2088004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4af542852c28845 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f206effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-19T06:29:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f206effd640 1 -- 172.21.15.5:0/921467337 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088021070 con 0x55c30b8e2370 2024-09-19T06:29:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f206effd640 1 -- 172.21.15.5:0/921467337 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8594+0+0 (secure 0 0 0) 0x7f2088008d90 con 0x55c30b8e2370 2024-09-19T06:29:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f2098583740 1 -- 172.21.15.5:0/921467337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f208009bbe0 con 0x55c30b8e2370 2024-09-19T06:29:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f206effd640 1 -- 172.21.15.5:0/921467337 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f208802d050 con 0x55c30b8e2370 2024-09-19T06:29:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f2098583740 10 client.21759 fetch_fsmap learned FSMap version 66 2024-09-19T06:29:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f2098583740 1 -- 172.21.15.5:0/921467337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55c30b8f0ae0 con 0x55c30b8e2370 2024-09-19T06:29:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.887+0000 7f206effd640 1 -- 172.21.15.5:0/921467337 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f208009bbe0 con 0x55c30b8e2370 2024-09-19T06:29:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.888+0000 7f206effd640 1 -- 172.21.15.5:0/921467337 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 8394+0+0 (secure 0 0 0) 0x7f208801c440 con 0x55c30b8e2370 2024-09-19T06:29:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.888+0000 7f2098583740 10 client.21759 fetch_fsmap finished waiting for FSMap version 66 2024-09-19T06:29:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.888+0000 7f2098583740 10 client.21759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:29:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.888+0000 7f2098583740 20 client.21759 populate_metadata read hostname 'smithi005' 2024-09-19T06:29:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.888+0000 7f2098583740 1 --2- 172.21.15.5:0/921467337 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c30b8f1f40 0x55c30b8f4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.888+0000 7f2098583740 4 client.21759 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:29:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.888+0000 7f2098583740 1 -- 172.21.15.5:0/921467337 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c30b84a860 con 0x55c30b8f1f40 2024-09-19T06:29:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.888+0000 7f208f7fe640 1 --2- 172.21.15.5:0/921467337 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c30b8f1f40 0x55c30b8f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.888+0000 7f208f7fe640 1 --2- 172.21.15.5:0/921467337 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c30b8f1f40 0x55c30b8f4370 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e149014b7adcc2c server_cookie=1b759ba19ea340a6 in_seq=0 out_seq=0 2024-09-19T06:29:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.888+0000 7f206effd640 10 client.21759 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:29:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f206effd640 1 -- 172.21.15.5:0/921467337 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c30b84a860 con 0x55c30b8f1f40 2024-09-19T06:29:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f206effd640 10 client.21759 handle_command_reply: tid=0 2024-09-19T06:29:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f2098583740 1 client.21759 shutdown 2024-09-19T06:29:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f2098583740 1 -- 172.21.15.5:0/921467337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 msgr2=0x55c30b8df650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f2098583740 1 --2- 172.21.15.5:0/921467337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b8df650 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f2088011dc0 tx=0x7f2088004a40 comp rx=0 tx=0).stop 2024-09-19T06:29:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f2098583740 1 -- 172.21.15.5:0/921467337 shutdown_connections 2024-09-19T06:29:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f2098583740 1 --2- 172.21.15.5:0/921467337 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c30b8f1f40 0x55c30b8f4370 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f2098583740 1 --2- 172.21.15.5:0/921467337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b8df650 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f206effd640 0 client.21759 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:29:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f2098583740 1 --2- 172.21.15.5:0/921467337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d8b40 0x55c30b8df040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f2098583740 1 --2- 172.21.15.5:0/921467337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d75d0 0x55c30b8e1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f2098583740 1 -- 172.21.15.5:0/921467337 >> 172.21.15.5:0/921467337 conn(0x55c30b8a0310 msgr2=0x55c30b8d9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f2098583740 1 -- 172.21.15.5:0/921467337 shutdown_connections 2024-09-19T06:29:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.889+0000 7f2098583740 1 -- 172.21.15.5:0/921467337 wait complete. 2024-09-19T06:29:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.890+0000 7f2098583740 20 client.21759 trim_cache size 0 max 16384 2024-09-19T06:29:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.903+0000 7f2098583740 1 Processor -- start 2024-09-19T06:29:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.903+0000 7f2098583740 1 -- start start 2024-09-19T06:29:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.903+0000 7f2098583740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d75d0 0x55c30b8e0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.903+0000 7f2098583740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d8b40 0x55c30b8da180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.903+0000 7f2098583740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b8da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.903+0000 7f2098583740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20901afb80 con 0x55c30b8e2370 2024-09-19T06:29:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.903+0000 7f2098583740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b1780 con 0x55c30b8d75d0 2024-09-19T06:29:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.903+0000 7f2098583740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b0280 con 0x55c30b8d8b40 2024-09-19T06:29:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f208ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b8da6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f208ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b8da6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39326/0 (socket says 172.21.15.5:39326) 2024-09-19T06:29:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f208ffff640 1 -- 172.21.15.5:0/1095283333 learned_addr learned my addr 172.21.15.5:0/1095283333 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f208effd640 1 --2- 172.21.15.5:0/1095283333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d8b40 0x55c30b8da180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f208f7fe640 1 --2- 172.21.15.5:0/1095283333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d75d0 0x55c30b8e0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f208ffff640 1 -- 172.21.15.5:0/1095283333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d8b40 msgr2=0x55c30b8da180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f208ffff640 1 --2- 172.21.15.5:0/1095283333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d8b40 0x55c30b8da180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f208ffff640 1 -- 172.21.15.5:0/1095283333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d75d0 msgr2=0x55c30b8e0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f208ffff640 1 --2- 172.21.15.5:0/1095283333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d75d0 0x55c30b8e0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f208ffff640 1 -- 172.21.15.5:0/1095283333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c30b8d4960 con 0x55c30b8e2370 2024-09-19T06:29:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f208ffff640 1 --2- 172.21.15.5:0/1095283333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b8da6c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f208800de10 tx=0x7f2088011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcc1c097b5b6299 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f20757fa640 1 -- 172.21.15.5:0/1095283333 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088035070 con 0x55c30b8e2370 2024-09-19T06:29:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f20757fa640 1 -- 172.21.15.5:0/1095283333 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2088008e50 con 0x55c30b8e2370 2024-09-19T06:29:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.904+0000 7f20757fa640 1 -- 172.21.15.5:0/1095283333 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088021040 con 0x55c30b8e2370 2024-09-19T06:29:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.905+0000 7f2098583740 1 -- 172.21.15.5:0/1095283333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 msgr2=0x55c30b8da6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:10.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.905+0000 7f2098583740 1 --2- 172.21.15.5:0/1095283333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b8da6c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f208800de10 tx=0x7f2088011b20 comp rx=0 tx=0).stop 2024-09-19T06:29:10.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.905+0000 7f2098583740 1 -- 172.21.15.5:0/1095283333 shutdown_connections 2024-09-19T06:29:10.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.905+0000 7f2098583740 1 --2- 172.21.15.5:0/1095283333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b8da6c0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.905+0000 7f2098583740 1 --2- 172.21.15.5:0/1095283333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d8b40 0x55c30b8da180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.905+0000 7f2098583740 1 --2- 172.21.15.5:0/1095283333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d75d0 0x55c30b8e0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.905+0000 7f2098583740 1 -- 172.21.15.5:0/1095283333 >> 172.21.15.5:0/1095283333 conn(0x55c30b8a0310 msgr2=0x55c30b8e08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:10.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.905+0000 7f2098583740 1 -- 172.21.15.5:0/1095283333 shutdown_connections 2024-09-19T06:29:10.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.905+0000 7f2098583740 1 -- 172.21.15.5:0/1095283333 wait complete. 2024-09-19T06:29:10.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f2098583740 1 -- start start 2024-09-19T06:29:10.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f2098583740 1 Processor -- start 2024-09-19T06:29:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f2098583740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d75d0 0x55c30b8e67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f2098583740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d8b40 0x55c30b8e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f2098583740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b95bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f2098583740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b10b0 con 0x55c30b8e2370 2024-09-19T06:29:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f2098583740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b09b0 con 0x55c30b8d8b40 2024-09-19T06:29:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f2098583740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20901adb80 con 0x55c30b8d75d0 2024-09-19T06:29:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f2098583740 10 client.0 _ll_register_callbacks cb 0x7f2088015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:29:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d75d0 0x55c30b8e67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f208ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b95bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d75d0 0x55c30b8e67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34992/0 (socket says 172.21.15.5:34992) 2024-09-19T06:29:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f208f7fe640 1 -- 172.21.15.5:0/101493679 learned_addr learned my addr 172.21.15.5:0/101493679 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.906+0000 7f208effd640 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d8b40 0x55c30b8e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f208f7fe640 1 -- 172.21.15.5:0/101493679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d8b40 msgr2=0x55c30b8e6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f208f7fe640 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d8b40 0x55c30b8e6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f208f7fe640 1 -- 172.21.15.5:0/101493679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 msgr2=0x55c30b95bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:29:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f208f7fe640 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b95bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f208f7fe640 1 -- 172.21.15.5:0/101493679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c30b8d34f0 con 0x55c30b8d75d0 2024-09-19T06:29:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f208ffff640 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b95bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:29:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f208f7fe640 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d75d0 0x55c30b8e67b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f208009b8e0 tx=0x7f208009c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbe7b13c60adfeec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f206effd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T06:29:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f206effd640 1 -- 172.21.15.5:0/101493679 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080020040 con 0x55c30b8d75d0 2024-09-19T06:29:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f206effd640 1 -- 172.21.15.5:0/101493679 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8594+0+0 (secure 0 0 0) 0x7f2080098250 con 0x55c30b8d75d0 2024-09-19T06:29:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f2098583740 1 -- 172.21.15.5:0/101493679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2088008e50 con 0x55c30b8d75d0 2024-09-19T06:29:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f206effd640 1 -- 172.21.15.5:0/101493679 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f20800a4850 con 0x55c30b8d75d0 2024-09-19T06:29:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f2098583740 10 client.31036 fetch_fsmap learned FSMap version 66 2024-09-19T06:29:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.907+0000 7f2098583740 1 -- 172.21.15.5:0/101493679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55c30b95d1e0 con 0x55c30b8d75d0 2024-09-19T06:29:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.908+0000 7f206effd640 1 -- 172.21.15.5:0/101493679 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2088008e50 con 0x55c30b8d75d0 2024-09-19T06:29:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.908+0000 7f206effd640 1 -- 172.21.15.5:0/101493679 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 66) ==== 8394+0+0 (secure 0 0 0) 0x7f20800ba2b0 con 0x55c30b8d75d0 2024-09-19T06:29:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.908+0000 7f2098583740 10 client.31036 fetch_fsmap finished waiting for FSMap version 66 2024-09-19T06:29:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.908+0000 7f2098583740 10 client.31036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:29:10.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.908+0000 7f2098583740 20 client.31036 populate_metadata read hostname 'smithi005' 2024-09-19T06:29:10.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.908+0000 7f2098583740 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c30b95d540 0x55c30b95f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:10.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.908+0000 7f2098583740 4 client.31036 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:29:10.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.908+0000 7f2098583740 1 -- 172.21.15.5:0/101493679 --> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c30b844d70 con 0x55c30b95d540 2024-09-19T06:29:10.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.908+0000 7f208effd640 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c30b95d540 0x55c30b95f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.908+0000 7f208effd640 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c30b95d540 0x55c30b95f930 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c97b8f8af82efe57 server_cookie=112b36b1137ab56d in_seq=0 out_seq=0 2024-09-19T06:29:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.908+0000 7f206effd640 10 client.31036 ms_handle_connect on v2:172.21.15.49:6838/2359039211 2024-09-19T06:29:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f206effd640 1 -- 172.21.15.5:0/101493679 <== mds.0 v2:172.21.15.49:6838/2359039211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c30b844d70 con 0x55c30b95d540 2024-09-19T06:29:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f206effd640 10 client.31036 handle_command_reply: tid=0 2024-09-19T06:29:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f2098583740 1 client.31036 shutdown 2024-09-19T06:29:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f2098583740 1 -- 172.21.15.5:0/101493679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d75d0 msgr2=0x55c30b8e67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f2098583740 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d75d0 0x55c30b8e67b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f208009b8e0 tx=0x7f208009c420 comp rx=0 tx=0).stop 2024-09-19T06:29:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f2098583740 1 -- 172.21.15.5:0/101493679 shutdown_connections 2024-09-19T06:29:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f2098583740 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.49:6838/2359039211,v1:172.21.15.49:6839/2359039211] conn(0x55c30b95d540 0x55c30b95f930 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f2098583740 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c30b8e2370 0x55c30b95bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f2098583740 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c30b8d8b40 0x55c30b8e6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f2098583740 1 --2- 172.21.15.5:0/101493679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c30b8d75d0 0x55c30b8e67b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f206effd640 0 client.31036 ms_handle_reset on v2:172.21.15.49:6838/2359039211 2024-09-19T06:29:10.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f2098583740 1 -- 172.21.15.5:0/101493679 >> 172.21.15.5:0/101493679 conn(0x55c30b8a0310 msgr2=0x55c30b8d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:10.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f2098583740 1 -- 172.21.15.5:0/101493679 shutdown_connections 2024-09-19T06:29:10.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f2098583740 1 -- 172.21.15.5:0/101493679 wait complete. 2024-09-19T06:29:10.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.909+0000 7f2098583740 20 client.31036 trim_cache size 0 max 16384 2024-09-19T06:29:10.931 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-19T06:29:10.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.910+0000 7f20961aa640 1 -- 172.21.15.5:0/466224365 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20780a33b0 msgr2=0x7f20780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:10.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.910+0000 7f20961aa640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20780a33b0 0x7f20780a5870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2088005fd0 tx=0x7f208800d710 comp rx=0 tx=0).stop 2024-09-19T06:29:10.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.910+0000 7f20961aa640 1 -- 172.21.15.5:0/466224365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2090100a80 msgr2=0x7f209019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:10.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.910+0000 7f20961aa640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2090100a80 0x7f209019a560 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f208000d950 tx=0x7f208000de10 comp rx=0 tx=0).stop 2024-09-19T06:29:10.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.910+0000 7f20961aa640 1 -- 172.21.15.5:0/466224365 shutdown_connections 2024-09-19T06:29:10.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.910+0000 7f20961aa640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f20780a33b0 0x7f20780a5870 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.910+0000 7f20961aa640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2090106a80 0x7f209019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.910+0000 7f20961aa640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2090101420 0x7f209019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.910+0000 7f20961aa640 1 --2- 172.21.15.5:0/466224365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2090100a80 0x7f209019a560 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:10.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.910+0000 7f20961aa640 1 -- 172.21.15.5:0/466224365 >> 172.21.15.5:0/466224365 conn(0x7f20900fc7a0 msgr2=0x7f20900fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:10.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.910+0000 7f20961aa640 1 -- 172.21.15.5:0/466224365 shutdown_connections 2024-09-19T06:29:10.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:10.910+0000 7f20961aa640 1 -- 172.21.15.5:0/466224365 wait complete. 2024-09-19T06:29:11.598 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T06:29:11.599 INFO:tasks.fwd_scrub:done joining 2024-09-19T06:29:11.600 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T06:29:11.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T06:29:11.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.730+0000 7f9c1f111640 1 -- 172.21.15.5:0/2214574832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c18101380 msgr2=0x7f9c1810d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.730+0000 7f9c1f111640 1 --2- 172.21.15.5:0/2214574832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c18101380 0x7f9c1810d8b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009cd0 tx=0x7f9c1002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:29:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.730+0000 7f9c1f111640 1 -- 172.21.15.5:0/2214574832 shutdown_connections 2024-09-19T06:29:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.730+0000 7f9c1f111640 1 --2- 172.21.15.5:0/2214574832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c18101380 0x7f9c1810d8b0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.730+0000 7f9c1f111640 1 --2- 172.21.15.5:0/2214574832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c181009e0 0x7f9c18100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.730+0000 7f9c1f111640 1 --2- 172.21.15.5:0/2214574832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c181069f0 0x7f9c18106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.730+0000 7f9c1f111640 1 -- 172.21.15.5:0/2214574832 >> 172.21.15.5:0/2214574832 conn(0x7f9c180fc6c0 msgr2=0x7f9c180feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.730+0000 7f9c1f111640 1 -- 172.21.15.5:0/2214574832 shutdown_connections 2024-09-19T06:29:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.730+0000 7f9c1f111640 1 -- 172.21.15.5:0/2214574832 wait complete. 2024-09-19T06:29:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.730+0000 7f9c1f111640 1 Processor -- start 2024-09-19T06:29:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1f111640 1 -- start start 2024-09-19T06:29:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1f111640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c181009e0 0x7f9c18198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1f111640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c18101380 0x7f9c18198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1f111640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181069f0 0x7f9c1819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1f111640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c18110ce0 con 0x7f9c181069f0 2024-09-19T06:29:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1f111640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9c18110b60 con 0x7f9c181009e0 2024-09-19T06:29:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1f111640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9c18110e60 con 0x7f9c18101380 2024-09-19T06:29:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1d6ae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181069f0 0x7f9c1819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1d6ae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181069f0 0x7f9c1819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39348/0 (socket says 172.21.15.5:39348) 2024-09-19T06:29:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1d6ae640 1 -- 172.21.15.5:0/363213198 learned_addr learned my addr 172.21.15.5:0/363213198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1cead640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c181009e0 0x7f9c18198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c17fff640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c18101380 0x7f9c18198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1d6ae640 1 -- 172.21.15.5:0/363213198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c18101380 msgr2=0x7f9c18198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:11.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1d6ae640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c18101380 0x7f9c18198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:11.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1d6ae640 1 -- 172.21.15.5:0/363213198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c181009e0 msgr2=0x7f9c18198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:11.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1d6ae640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c181009e0 0x7f9c18198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:11.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1d6ae640 1 -- 172.21.15.5:0/363213198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1819d330 con 0x7f9c181069f0 2024-09-19T06:29:11.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.731+0000 7f9c1d6ae640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181069f0 0x7f9c1819cc00 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005cc0 tx=0x7f9c10003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:11.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.732+0000 7f9c15ffb640 1 -- 172.21.15.5:0/363213198 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10004450 con 0x7f9c181069f0 2024-09-19T06:29:11.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.732+0000 7f9c15ffb640 1 -- 172.21.15.5:0/363213198 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c10031c90 con 0x7f9c181069f0 2024-09-19T06:29:11.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.732+0000 7f9c1f111640 1 -- 172.21.15.5:0/363213198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1819d5c0 con 0x7f9c181069f0 2024-09-19T06:29:11.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.732+0000 7f9c15ffb640 1 -- 172.21.15.5:0/363213198 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1003b400 con 0x7f9c181069f0 2024-09-19T06:29:11.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.732+0000 7f9c1f111640 1 -- 172.21.15.5:0/363213198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c181a4eb0 con 0x7f9c181069f0 2024-09-19T06:29:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.733+0000 7f9c1f111640 1 -- 172.21.15.5:0/363213198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c18100e40 con 0x7f9c181069f0 2024-09-19T06:29:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.734+0000 7f9c15ffb640 1 -- 172.21.15.5:0/363213198 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9c10004680 con 0x7f9c181069f0 2024-09-19T06:29:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.735+0000 7f9c15ffb640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9c000a33c0 0x7f9c000a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.735+0000 7f9c1cead640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9c000a33c0 0x7f9c000a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.735+0000 7f9c15ffb640 1 -- 172.21.15.5:0/363213198 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8594+0+0 (secure 0 0 0) 0x7f9c1002e030 con 0x7f9c181069f0 2024-09-19T06:29:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.735+0000 7f9c1cead640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9c000a33c0 0x7f9c000a5880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9c08005fd0 tx=0x7f9c0800b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:11.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.739+0000 7f9c15ffb640 1 -- 172.21.15.5:0/363213198 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9c100f6020 con 0x7f9c181069f0 2024-09-19T06:29:11.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.918+0000 7f9c1f111640 1 -- 172.21.15.5:0/363213198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c18109fc0 con 0x7f9c181069f0 2024-09-19T06:29:11.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.919+0000 7f9c15ffb640 1 -- 172.21.15.5:0/363213198 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+23234 (secure 0 0 0) 0x7f9c100bd8f0 con 0x7f9c181069f0 2024-09-19T06:29:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:29:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":124,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","created":"2024-09-19T05:53:51.454068+0000","modified":"2024-09-19T06:29:09.077983+0000","last_up_change":"2024-09-19T06:00:38.837916+0000","last_in_change":"2024-09-19T06:00:15.609776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:56:55.213931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:01:25.648588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:01:27.760408+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":124,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f3c10d60-d354-407f-9cdf-099344599844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6803","nonce":3442981810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6805","nonce":3442981810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6809","nonce":3442981810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6807","nonce":3442981810}]},"public_addr":"172.21.15.5:6803/3442981810","cluster_addr":"172.21.15.5:6805/3442981810","heartbeat_back_addr":"172.21.15.5:6809/3442981810","heartbeat_front_addr":"172.21.15.5:6807/3442981810","state":["exists","up"]},{"osd":1,"uuid":"5f40f636-7761-45ac-87f3-910a2fa2feb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448265014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6803","nonce":1448265014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6807","nonce":1448265014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6805","nonce":1448265014}]},"public_addr":"172.21.15.49:6801/1448265014","cluster_addr":"172.21.15.49:6803/1448265014","heartbeat_back_addr":"172.21.15.49:6807/1448265014","heartbeat_front_addr":"172.21.15.49:6805/1448265014","state":["exists","up"]},{"osd":2,"uuid":"ad00940c-4fdd-4f5f-9eaa-b1f38bcac526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6801","nonce":2319626622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6803","nonce":2319626622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6807","nonce":2319626622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6805","nonce":2319626622}]},"public_addr":"172.21.15.157:6801/2319626622","cluster_addr":"172.21.15.157:6803/2319626622","heartbeat_back_addr":"172.21.15.157:6807/2319626622","heartbeat_front_addr":"172.21.15.157:6805/2319626622","state":["exists","up"]},{"osd":3,"uuid":"63fec329-1b62-4c5e-b8ca-084a84d4d762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6811","nonce":1452491753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6813","nonce":1452491753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6817","nonce":1452491753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6815","nonce":1452491753}]},"public_addr":"172.21.15.5:6811/1452491753","cluster_addr":"172.21.15.5:6813/1452491753","heartbeat_back_addr":"172.21.15.5:6817/1452491753","heartbeat_front_addr":"172.21.15.5:6815/1452491753","state":["exists","up"]},{"osd":4,"uuid":"ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6809","nonce":3114952580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6811","nonce":3114952580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6815","nonce":3114952580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6813","nonce":3114952580}]},"public_addr":"172.21.15.49:6809/3114952580","cluster_addr":"172.21.15.49:6811/3114952580","heartbeat_back_addr":"172.21.15.49:6815/3114952580","heartbeat_front_addr":"172.21.15.49:6813/3114952580","state":["exists","up"]},{"osd":5,"uuid":"21a7a7d3-ce42-4688-bfec-714be635eaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6809","nonce":3219063971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6811","nonce":3219063971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6815","nonce":3219063971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6813","nonce":3219063971}]},"public_addr":"172.21.15.157:6809/3219063971","cluster_addr":"172.21.15.157:6811/3219063971","heartbeat_back_addr":"172.21.15.157:6815/3219063971","heartbeat_front_addr":"172.21.15.157:6813/3219063971","state":["exists","up"]},{"osd":6,"uuid":"cbef0e08-d6b0-4515-a2ee-2c81b048583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6819","nonce":2957795194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6821","nonce":2957795194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6825","nonce":2957795194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6823","nonce":2957795194}]},"public_addr":"172.21.15.5:6819/2957795194","cluster_addr":"172.21.15.5:6821/2957795194","heartbeat_back_addr":"172.21.15.5:6825/2957795194","heartbeat_front_addr":"172.21.15.5:6823/2957795194","state":["exists","up"]},{"osd":7,"uuid":"d261ac55-532d-4aea-9b3e-ed2b4eeb1da6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6817","nonce":2959731400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6819","nonce":2959731400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6823","nonce":2959731400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6821","nonce":2959731400}]},"public_addr":"172.21.15.49:6817/2959731400","cluster_addr":"172.21.15.49:6819/2959731400","heartbeat_back_addr":"172.21.15.49:6823/2959731400","heartbeat_front_addr":"172.21.15.49:6821/2959731400","state":["exists","up"]},{"osd":8,"uuid":"15d0eaf7-d604-4546-b393-4d6d39018f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6817","nonce":290026859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6819","nonce":290026859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6823","nonce":290026859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6821","nonce":290026859}]},"public_addr":"172.21.15.157:6817/290026859","cluster_addr":"172.21.15.157:6819/290026859","heartbeat_back_addr":"172.21.15.157:6823/290026859","heartbeat_front_addr":"172.21.15.157:6821/290026859","state":["exists","up"]},{"osd":9,"uuid":"30c08537-f593-4299-bfa1-fa7ba8a95169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6827","nonce":4122448237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6829","nonce":4122448237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6833","nonce":4122448237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6831","nonce":4122448237}]},"public_addr":"172.21.15.5:6827/4122448237","cluster_addr":"172.21.15.5:6829/4122448237","heartbeat_back_addr":"172.21.15.5:6833/4122448237","heartbeat_front_addr":"172.21.15.5:6831/4122448237","state":["exists","up"]},{"osd":10,"uuid":"68821e98-bd81-49ec-8cd6-51fb9880b553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6825","nonce":2551748002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6827","nonce":2551748002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6831","nonce":2551748002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6829","nonce":2551748002}]},"public_addr":"172.21.15.49:6825/2551748002","cluster_addr":"172.21.15.49:6827/2551748002","heartbeat_back_addr":"172.21.15.49:6831/2551748002","heartbeat_front_addr":"172.21.15.49:6829/2551748002","state":["exists","up"]},{"osd":11,"uuid":"7e443b46-2697-4d0b-8452-868463a1ee1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6825","nonce":1996582544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6827","nonce":1996582544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6831","nonce":1996582544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6829","nonce":1996582544}]},"public_addr":"172.21.15.157:6825/1996582544","cluster_addr":"172.21.15.157:6827/1996582544","heartbeat_back_addr":"172.21.15.157:6831/1996582544","heartbeat_front_addr":"172.21.15.157:6829/1996582544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:01.453592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:27.194705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:51.525385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:17.676749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:43.075555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:07.258154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:32.112583+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:57.306534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:21.320950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:47.164610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:12.548337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:37.262580+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3703679969":"2024-09-20T06:03:20.622034+0000","172.21.15.5:6801/709970884":"2024-09-20T06:03:20.622034+0000","172.21.15.5:0/1367598148":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2930274429":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6800/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2197596976":"2024-09-20T06:03:20.622034+0000","172.21.15.5:6801/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/1335130331":"2024-09-20T06:03:20.622034+0000","172.21.15.5:6800/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/1639646508":"2024-09-20T06:03:20.622034+0000","172.21.15.5:0/1340147787":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/850678758":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6801/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/2593868621":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/4102275559":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6800/709970884":"2024-09-20T06:03:20.622034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:29:11.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.921+0000 7f9c1f111640 1 -- 172.21.15.5:0/363213198 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9c000a33c0 msgr2=0x7f9c000a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:11.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.921+0000 7f9c1f111640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9c000a33c0 0x7f9c000a5880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9c08005fd0 tx=0x7f9c0800b390 comp rx=0 tx=0).stop 2024-09-19T06:29:11.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.921+0000 7f9c1f111640 1 -- 172.21.15.5:0/363213198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181069f0 msgr2=0x7f9c1819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:11.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.921+0000 7f9c1f111640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181069f0 0x7f9c1819cc00 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005cc0 tx=0x7f9c10003c20 comp rx=0 tx=0).stop 2024-09-19T06:29:11.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.921+0000 7f9c1f111640 1 -- 172.21.15.5:0/363213198 shutdown_connections 2024-09-19T06:29:11.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.921+0000 7f9c1f111640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181069f0 0x7f9c1819cc00 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:11.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.921+0000 7f9c1f111640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f9c000a33c0 0x7f9c000a5880 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:11.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.921+0000 7f9c1f111640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c18101380 0x7f9c18198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:11.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.921+0000 7f9c1f111640 1 --2- 172.21.15.5:0/363213198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c181009e0 0x7f9c18198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:11.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.921+0000 7f9c1f111640 1 -- 172.21.15.5:0/363213198 >> 172.21.15.5:0/363213198 conn(0x7f9c180fc6c0 msgr2=0x7f9c1806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:11.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.921+0000 7f9c1f111640 1 -- 172.21.15.5:0/363213198 shutdown_connections 2024-09-19T06:29:11.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:11.921+0000 7f9c1f111640 1 -- 172.21.15.5:0/363213198 wait complete. 2024-09-19T06:29:11.934 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T05:56:55.213931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T06:01:25.648588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T06:01:27.760408+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 124, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T06:29:11.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T06:29:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.041+0000 7f957ed31640 1 -- 172.21.15.5:0/1080632450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578102be0 msgr2=0x7f9578102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.041+0000 7f957ed31640 1 --2- 172.21.15.5:0/1080632450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578102be0 0x7f9578102fc0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9568009900 tx=0x7f956802a410 comp rx=0 tx=0).stop 2024-09-19T06:29:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.041+0000 7f957ed31640 1 -- 172.21.15.5:0/1080632450 shutdown_connections 2024-09-19T06:29:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.041+0000 7f957ed31640 1 --2- 172.21.15.5:0/1080632450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f957810c0e0 0x7f957810e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.041+0000 7f957ed31640 1 --2- 172.21.15.5:0/1080632450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9578103590 0x7f957810ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.041+0000 7f957ed31640 1 --2- 172.21.15.5:0/1080632450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578102be0 0x7f9578102fc0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.041+0000 7f957ed31640 1 -- 172.21.15.5:0/1080632450 >> 172.21.15.5:0/1080632450 conn(0x7f95780fc6a0 msgr2=0x7f95780feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.041+0000 7f957ed31640 1 -- 172.21.15.5:0/1080632450 shutdown_connections 2024-09-19T06:29:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.041+0000 7f957ed31640 1 -- 172.21.15.5:0/1080632450 wait complete. 2024-09-19T06:29:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957ed31640 1 Processor -- start 2024-09-19T06:29:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957ed31640 1 -- start start 2024-09-19T06:29:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957ed31640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9578102be0 0x7f95781982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957ed31640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9578103590 0x7f9578198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957ed31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810c0e0 0x7f957819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957ed31640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9578111d40 con 0x7f957810c0e0 2024-09-19T06:29:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957ed31640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9578111bc0 con 0x7f9578103590 2024-09-19T06:29:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957ed31640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9578111ec0 con 0x7f9578102be0 2024-09-19T06:29:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957d2ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810c0e0 0x7f957819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957d2ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810c0e0 0x7f957819cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39364/0 (socket says 172.21.15.5:39364) 2024-09-19T06:29:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957d2ce640 1 -- 172.21.15.5:0/2401896037 learned_addr learned my addr 172.21.15.5:0/2401896037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f9577fff640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9578103590 0x7f9578198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957cacd640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9578102be0 0x7f95781982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957d2ce640 1 -- 172.21.15.5:0/2401896037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9578102be0 msgr2=0x7f95781982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957d2ce640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9578102be0 0x7f95781982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957d2ce640 1 -- 172.21.15.5:0/2401896037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9578103590 msgr2=0x7f9578198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957d2ce640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9578103590 0x7f9578198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.042+0000 7f957d2ce640 1 -- 172.21.15.5:0/2401896037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957819d2c0 con 0x7f957810c0e0 2024-09-19T06:29:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.043+0000 7f957d2ce640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810c0e0 0x7f957819cb90 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9570009fd0 tx=0x7f957000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.043+0000 7f9575ffb640 1 -- 172.21.15.5:0/2401896037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570017070 con 0x7f957810c0e0 2024-09-19T06:29:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.043+0000 7f957ed31640 1 -- 172.21.15.5:0/2401896037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957819d5b0 con 0x7f957810c0e0 2024-09-19T06:29:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.043+0000 7f9575ffb640 1 -- 172.21.15.5:0/2401896037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9570003e20 con 0x7f957810c0e0 2024-09-19T06:29:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.043+0000 7f9575ffb640 1 -- 172.21.15.5:0/2401896037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570010460 con 0x7f957810c0e0 2024-09-19T06:29:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.043+0000 7f957ed31640 1 -- 172.21.15.5:0/2401896037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95781a4e40 con 0x7f957810c0e0 2024-09-19T06:29:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.044+0000 7f957ed31640 1 -- 172.21.15.5:0/2401896037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9578102fc0 con 0x7f957810c0e0 2024-09-19T06:29:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.045+0000 7f9575ffb640 1 -- 172.21.15.5:0/2401896037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9570003970 con 0x7f957810c0e0 2024-09-19T06:29:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.046+0000 7f9575ffb640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f95640a33c0 0x7f95640a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.046+0000 7f9575ffb640 1 -- 172.21.15.5:0/2401896037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8594+0+0 (secure 0 0 0) 0x7f95700cfbd0 con 0x7f957810c0e0 2024-09-19T06:29:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.046+0000 7f957cacd640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f95640a33c0 0x7f95640a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.046+0000 7f957cacd640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f95640a33c0 0x7f95640a5880 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9568031440 tx=0x7f95680095f0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.050+0000 7f9575ffb640 1 -- 172.21.15.5:0/2401896037 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f957009c500 con 0x7f957810c0e0 2024-09-19T06:29:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.229+0000 7f957ed31640 1 -- 172.21.15.5:0/2401896037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9578107190 con 0x7f957810c0e0 2024-09-19T06:29:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.230+0000 7f9575ffb640 1 -- 172.21.15.5:0/2401896037 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) ==== 93+0+10 (secure 0 0 0) 0x7f957009be00 con 0x7f957810c0e0 2024-09-19T06:29:12.232 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-19T06:29:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.232+0000 7f957ed31640 1 -- 172.21.15.5:0/2401896037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f95640a33c0 msgr2=0x7f95640a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.232+0000 7f957ed31640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f95640a33c0 0x7f95640a5880 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9568031440 tx=0x7f95680095f0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.232+0000 7f957ed31640 1 -- 172.21.15.5:0/2401896037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810c0e0 msgr2=0x7f957819cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.232+0000 7f957ed31640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810c0e0 0x7f957819cb90 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9570009fd0 tx=0x7f957000b260 comp rx=0 tx=0).stop 2024-09-19T06:29:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.232+0000 7f957ed31640 1 -- 172.21.15.5:0/2401896037 shutdown_connections 2024-09-19T06:29:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.232+0000 7f957ed31640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810c0e0 0x7f957819cb90 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.232+0000 7f957ed31640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9578103590 0x7f9578198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.232+0000 7f957ed31640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f95640a33c0 0x7f95640a5880 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.232+0000 7f957ed31640 1 --2- 172.21.15.5:0/2401896037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9578102be0 0x7f95781982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.232+0000 7f957ed31640 1 -- 172.21.15.5:0/2401896037 >> 172.21.15.5:0/2401896037 conn(0x7f95780fc6a0 msgr2=0x7f95780fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.232+0000 7f957ed31640 1 -- 172.21.15.5:0/2401896037 shutdown_connections 2024-09-19T06:29:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.232+0000 7f957ed31640 1 -- 172.21.15.5:0/2401896037 wait complete. 2024-09-19T06:29:12.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:12 smithi005 ceph-mon[30955]: pgmap v818: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-09-19T06:29:12.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:12 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/363213198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:29:12.245 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T06:29:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.352+0000 7f5618911640 1 -- 172.21.15.5:0/1368978016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5614100e00 msgr2=0x7f5614101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.352+0000 7f5618911640 1 --2- 172.21.15.5:0/1368978016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5614100e00 0x7f5614101260 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009930 tx=0x7f55fc02a420 comp rx=0 tx=0).stop 2024-09-19T06:29:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.352+0000 7f5618911640 1 -- 172.21.15.5:0/1368978016 shutdown_connections 2024-09-19T06:29:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.352+0000 7f5618911640 1 --2- 172.21.15.5:0/1368978016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56141017a0 0x7f561410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.352+0000 7f5618911640 1 --2- 172.21.15.5:0/1368978016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5614100e00 0x7f5614101260 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.352+0000 7f5618911640 1 --2- 172.21.15.5:0/1368978016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56141069b0 0x7f5614106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.352+0000 7f5618911640 1 -- 172.21.15.5:0/1368978016 >> 172.21.15.5:0/1368978016 conn(0x7f56140766f0 msgr2=0x7f5614076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.352+0000 7f5618911640 1 -- 172.21.15.5:0/1368978016 shutdown_connections 2024-09-19T06:29:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.352+0000 7f5618911640 1 -- 172.21.15.5:0/1368978016 wait complete. 2024-09-19T06:29:12.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.353+0000 7f5618911640 1 Processor -- start 2024-09-19T06:29:12.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.353+0000 7f5618911640 1 -- start start 2024-09-19T06:29:12.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.353+0000 7f5618911640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614100e00 0x7f5614102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.353+0000 7f5618911640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56141017a0 0x7f5614102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.353+0000 7f5618911640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56141069b0 0x7f5614102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.353+0000 7f5618911640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5614110ca0 con 0x7f56141017a0 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.353+0000 7f5618911640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5614110b20 con 0x7f5614100e00 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.353+0000 7f5618911640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5614110e20 con 0x7f56141069b0 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f560bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56141017a0 0x7f5614102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f560bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56141017a0 0x7f5614102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39382/0 (socket says 172.21.15.5:39382) 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f560bfff640 1 -- 172.21.15.5:0/1172394669 learned_addr learned my addr 172.21.15.5:0/1172394669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f561259c640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614100e00 0x7f5614102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f5612d9d640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56141069b0 0x7f5614102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f560bfff640 1 -- 172.21.15.5:0/1172394669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56141069b0 msgr2=0x7f5614102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f560bfff640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56141069b0 0x7f5614102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f560bfff640 1 -- 172.21.15.5:0/1172394669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614100e00 msgr2=0x7f5614102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f560bfff640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614100e00 0x7f5614102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f560bfff640 1 -- 172.21.15.5:0/1172394669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56141a68d0 con 0x7f56141017a0 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f5612d9d640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56141069b0 0x7f5614102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f561259c640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614100e00 0x7f5614102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:12.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f560bfff640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56141017a0 0x7f5614102ab0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f55fc004770 tx=0x7f55fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:12.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.354+0000 7f560b7fe640 1 -- 172.21.15.5:0/1172394669 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc03f070 con 0x7f56141017a0 2024-09-19T06:29:12.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.355+0000 7f5618911640 1 -- 172.21.15.5:0/1172394669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56141a6b60 con 0x7f56141017a0 2024-09-19T06:29:12.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.355+0000 7f560b7fe640 1 -- 172.21.15.5:0/1172394669 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55fc03b460 con 0x7f56141017a0 2024-09-19T06:29:12.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.355+0000 7f560b7fe640 1 -- 172.21.15.5:0/1172394669 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc035070 con 0x7f56141017a0 2024-09-19T06:29:12.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.355+0000 7f5618911640 1 -- 172.21.15.5:0/1172394669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56141a70a0 con 0x7f56141017a0 2024-09-19T06:29:12.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.355+0000 7f5618911640 1 -- 172.21.15.5:0/1172394669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5614101260 con 0x7f56141017a0 2024-09-19T06:29:12.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.357+0000 7f560b7fe640 1 -- 172.21.15.5:0/1172394669 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f55fc03b610 con 0x7f56141017a0 2024-09-19T06:29:12.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.358+0000 7f560b7fe640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f56000a33c0 0x7f56000a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.358+0000 7f561259c640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f56000a33c0 0x7f56000a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.358+0000 7f560b7fe640 1 -- 172.21.15.5:0/1172394669 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8594+0+0 (secure 0 0 0) 0x7f55fc0f0480 con 0x7f56141017a0 2024-09-19T06:29:12.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.358+0000 7f561259c640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f56000a33c0 0x7f56000a5880 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f56040058c0 tx=0x7f5604005850 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.365+0000 7f560b7fe640 1 -- 172.21.15.5:0/1172394669 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f55fc0bcdb0 con 0x7f56141017a0 2024-09-19T06:29:12.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:12 smithi049 ceph-mon[34640]: pgmap v818: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-09-19T06:29:12.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:12 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/363213198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:29:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.544+0000 7f5618911640 1 -- 172.21.15.5:0/1172394669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5614109f40 con 0x7f56141017a0 2024-09-19T06:29:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.544+0000 7f560b7fe640 1 -- 172.21.15.5:0/1172394669 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) ==== 104+0+11 (secure 0 0 0) 0x7f55fc0bc6b0 con 0x7f56141017a0 2024-09-19T06:29:12.546 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-19T06:29:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.546+0000 7f5618911640 1 -- 172.21.15.5:0/1172394669 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f56000a33c0 msgr2=0x7f56000a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.546+0000 7f5618911640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f56000a33c0 0x7f56000a5880 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f56040058c0 tx=0x7f5604005850 comp rx=0 tx=0).stop 2024-09-19T06:29:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.546+0000 7f5618911640 1 -- 172.21.15.5:0/1172394669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56141017a0 msgr2=0x7f5614102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.546+0000 7f5618911640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56141017a0 0x7f5614102ab0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f55fc004770 tx=0x7f55fc0042f0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.546+0000 7f5618911640 1 -- 172.21.15.5:0/1172394669 shutdown_connections 2024-09-19T06:29:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.546+0000 7f5618911640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f56000a33c0 0x7f56000a5880 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.546+0000 7f5618911640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56141069b0 0x7f5614102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.546+0000 7f5618911640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56141017a0 0x7f5614102ab0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.546+0000 7f5618911640 1 --2- 172.21.15.5:0/1172394669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614100e00 0x7f5614102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.546+0000 7f5618911640 1 -- 172.21.15.5:0/1172394669 >> 172.21.15.5:0/1172394669 conn(0x7f56140766f0 msgr2=0x7f5614069880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.546+0000 7f5618911640 1 -- 172.21.15.5:0/1172394669 shutdown_connections 2024-09-19T06:29:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.546+0000 7f5618911640 1 -- 172.21.15.5:0/1172394669 wait complete. 2024-09-19T06:29:12.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:12 smithi157 ceph-mon[35346]: pgmap v818: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-09-19T06:29:12.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:12 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/363213198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:29:12.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T06:29:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.665+0000 7f2d22bf8640 1 -- 172.21.15.5:0/91584424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d1c100e00 msgr2=0x7f2d1c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.665+0000 7f2d22bf8640 1 --2- 172.21.15.5:0/91584424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d1c100e00 0x7f2d1c101260 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009cd0 tx=0x7f2d0c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.665+0000 7f2d22bf8640 1 -- 172.21.15.5:0/91584424 shutdown_connections 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.665+0000 7f2d22bf8640 1 --2- 172.21.15.5:0/91584424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d1c1017a0 0x7f2d1c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.665+0000 7f2d22bf8640 1 --2- 172.21.15.5:0/91584424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d1c100e00 0x7f2d1c101260 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.665+0000 7f2d22bf8640 1 --2- 172.21.15.5:0/91584424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d1c1069b0 0x7f2d1c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.665+0000 7f2d22bf8640 1 -- 172.21.15.5:0/91584424 >> 172.21.15.5:0/91584424 conn(0x7f2d1c0766f0 msgr2=0x7f2d1c076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.665+0000 7f2d22bf8640 1 -- 172.21.15.5:0/91584424 shutdown_connections 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.665+0000 7f2d22bf8640 1 -- 172.21.15.5:0/91584424 wait complete. 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d22bf8640 1 Processor -- start 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d22bf8640 1 -- start start 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d22bf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d1c100e00 0x7f2d1c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d22bf8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d1c1017a0 0x7f2d1c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d22bf8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d1c1069b0 0x7f2d1c199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d22bf8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c110ca0 con 0x7f2d1c100e00 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d22bf8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c110b20 con 0x7f2d1c1069b0 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d22bf8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c110e20 con 0x7f2d1c1017a0 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d20994640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d1c100e00 0x7f2d1c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d20994640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d1c100e00 0x7f2d1c1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39396/0 (socket says 172.21.15.5:39396) 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d20994640 1 -- 172.21.15.5:0/3492006035 learned_addr learned my addr 172.21.15.5:0/3492006035 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d1bfff640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d1c1017a0 0x7f2d1c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d21195640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d1c1069b0 0x7f2d1c199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d20994640 1 -- 172.21.15.5:0/3492006035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d1c1017a0 msgr2=0x7f2d1c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d20994640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d1c1017a0 0x7f2d1c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d20994640 1 -- 172.21.15.5:0/3492006035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d1c1069b0 msgr2=0x7f2d1c199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d20994640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d1c1069b0 0x7f2d1c199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d20994640 1 -- 172.21.15.5:0/3492006035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c1a45d0 con 0x7f2d1c100e00 2024-09-19T06:29:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d1bfff640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d1c1017a0 0x7f2d1c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.666+0000 7f2d20994640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d1c100e00 0x7f2d1c1991f0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400d770 tx=0x7f2d0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.667+0000 7f2d19ffb640 1 -- 172.21.15.5:0/3492006035 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04017070 con 0x7f2d1c100e00 2024-09-19T06:29:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.667+0000 7f2d19ffb640 1 -- 172.21.15.5:0/3492006035 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d040054d0 con 0x7f2d1c100e00 2024-09-19T06:29:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.667+0000 7f2d22bf8640 1 -- 172.21.15.5:0/3492006035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c1a48c0 con 0x7f2d1c100e00 2024-09-19T06:29:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.667+0000 7f2d19ffb640 1 -- 172.21.15.5:0/3492006035 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04004cf0 con 0x7f2d1c100e00 2024-09-19T06:29:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.667+0000 7f2d22bf8640 1 -- 172.21.15.5:0/3492006035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c1a4dc0 con 0x7f2d1c100e00 2024-09-19T06:29:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.668+0000 7f2d22bf8640 1 -- 172.21.15.5:0/3492006035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1c101260 con 0x7f2d1c100e00 2024-09-19T06:29:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.669+0000 7f2d19ffb640 1 -- 172.21.15.5:0/3492006035 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2d04004e90 con 0x7f2d1c100e00 2024-09-19T06:29:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.670+0000 7f2d19ffb640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d080a33c0 0x7f2d080a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.670+0000 7f2d19ffb640 1 -- 172.21.15.5:0/3492006035 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8594+0+0 (secure 0 0 0) 0x7f2d040cf850 con 0x7f2d1c100e00 2024-09-19T06:29:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.670+0000 7f2d1bfff640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d080a33c0 0x7f2d080a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.670+0000 7f2d1bfff640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d080a33c0 0x7f2d080a5880 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c069fa0 tx=0x7f2d0c0093f0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:12.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.674+0000 7f2d19ffb640 1 -- 172.21.15.5:0/3492006035 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2d0409c180 con 0x7f2d1c100e00 2024-09-19T06:29:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.852+0000 7f2d22bf8640 1 -- 172.21.15.5:0/3492006035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2d1c108f30 con 0x7f2d1c100e00 2024-09-19T06:29:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.853+0000 7f2d19ffb640 1 -- 172.21.15.5:0/3492006035 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) ==== 100+0+11 (secure 0 0 0) 0x7f2d0409ba80 con 0x7f2d1c100e00 2024-09-19T06:29:12.854 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-09-19T06:29:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.855+0000 7f2d22bf8640 1 -- 172.21.15.5:0/3492006035 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d080a33c0 msgr2=0x7f2d080a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.855+0000 7f2d22bf8640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d080a33c0 0x7f2d080a5880 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c069fa0 tx=0x7f2d0c0093f0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.855+0000 7f2d22bf8640 1 -- 172.21.15.5:0/3492006035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d1c100e00 msgr2=0x7f2d1c1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.855+0000 7f2d22bf8640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d1c100e00 0x7f2d1c1991f0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400d770 tx=0x7f2d0400dc30 comp rx=0 tx=0).stop 2024-09-19T06:29:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.855+0000 7f2d22bf8640 1 -- 172.21.15.5:0/3492006035 shutdown_connections 2024-09-19T06:29:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.855+0000 7f2d22bf8640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d1c1069b0 0x7f2d1c199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.855+0000 7f2d22bf8640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f2d080a33c0 0x7f2d080a5880 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.855+0000 7f2d22bf8640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d1c1017a0 0x7f2d1c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.855+0000 7f2d22bf8640 1 --2- 172.21.15.5:0/3492006035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d1c100e00 0x7f2d1c1991f0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.855+0000 7f2d22bf8640 1 -- 172.21.15.5:0/3492006035 >> 172.21.15.5:0/3492006035 conn(0x7f2d1c0766f0 msgr2=0x7f2d1c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.855+0000 7f2d22bf8640 1 -- 172.21.15.5:0/3492006035 shutdown_connections 2024-09-19T06:29:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.855+0000 7f2d22bf8640 1 -- 172.21.15.5:0/3492006035 wait complete. 2024-09-19T06:29:12.867 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T06:29:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.971+0000 7f3c3ac53640 1 -- 172.21.15.5:0/1840657129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c341009c0 msgr2=0x7f3c34100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.971+0000 7f3c3ac53640 1 --2- 172.21.15.5:0/1840657129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c341009c0 0x7f3c34100e20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009930 tx=0x7f3c2002a420 comp rx=0 tx=0).stop 2024-09-19T06:29:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.971+0000 7f3c3ac53640 1 -- 172.21.15.5:0/1840657129 shutdown_connections 2024-09-19T06:29:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.971+0000 7f3c3ac53640 1 --2- 172.21.15.5:0/1840657129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c34101360 0x7f3c3410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.971+0000 7f3c3ac53640 1 --2- 172.21.15.5:0/1840657129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c341009c0 0x7f3c34100e20 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.971+0000 7f3c3ac53640 1 --2- 172.21.15.5:0/1840657129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c341069d0 0x7f3c34106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.971+0000 7f3c3ac53640 1 -- 172.21.15.5:0/1840657129 >> 172.21.15.5:0/1840657129 conn(0x7f3c34077a10 msgr2=0x7f3c340fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.971+0000 7f3c3ac53640 1 -- 172.21.15.5:0/1840657129 shutdown_connections 2024-09-19T06:29:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.971+0000 7f3c3ac53640 1 -- 172.21.15.5:0/1840657129 wait complete. 2024-09-19T06:29:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.971+0000 7f3c3ac53640 1 Processor -- start 2024-09-19T06:29:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.971+0000 7f3c3ac53640 1 -- start start 2024-09-19T06:29:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c3ac53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c341009c0 0x7f3c34198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c3ac53640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c34101360 0x7f3c34198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c3ac53640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c341069d0 0x7f3c3419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c3ac53640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34110d10 con 0x7f3c341009c0 2024-09-19T06:29:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c3ac53640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34110b90 con 0x7f3c34101360 2024-09-19T06:29:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c3ac53640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34110e90 con 0x7f3c341069d0 2024-09-19T06:29:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c389ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c341009c0 0x7f3c34198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c389ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c341009c0 0x7f3c34198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39424/0 (socket says 172.21.15.5:39424) 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c389ef640 1 -- 172.21.15.5:0/3167533883 learned_addr learned my addr 172.21.15.5:0/3167533883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c391f0640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c341069d0 0x7f3c3419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c33fff640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c34101360 0x7f3c34198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c389ef640 1 -- 172.21.15.5:0/3167533883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c341069d0 msgr2=0x7f3c3419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c389ef640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c341069d0 0x7f3c3419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c389ef640 1 -- 172.21.15.5:0/3167533883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c34101360 msgr2=0x7f3c34198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c389ef640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c34101360 0x7f3c34198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c389ef640 1 -- 172.21.15.5:0/3167533883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3419d340 con 0x7f3c341009c0 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c389ef640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c341009c0 0x7f3c34198350 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800d950 tx=0x7f3c2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.972+0000 7f3c31ffb640 1 -- 172.21.15.5:0/3167533883 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28017070 con 0x7f3c341009c0 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.973+0000 7f3c31ffb640 1 -- 172.21.15.5:0/3167533883 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c28009c30 con 0x7f3c341009c0 2024-09-19T06:29:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.973+0000 7f3c31ffb640 1 -- 172.21.15.5:0/3167533883 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800f620 con 0x7f3c341009c0 2024-09-19T06:29:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.973+0000 7f3c3ac53640 1 -- 172.21.15.5:0/3167533883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3419d630 con 0x7f3c341009c0 2024-09-19T06:29:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.973+0000 7f3c3ac53640 1 -- 172.21.15.5:0/3167533883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c341a4ec0 con 0x7f3c341009c0 2024-09-19T06:29:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.974+0000 7f3c3ac53640 1 -- 172.21.15.5:0/3167533883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c34100e20 con 0x7f3c341009c0 2024-09-19T06:29:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.976+0000 7f3c31ffb640 1 -- 172.21.15.5:0/3167533883 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3c280109a0 con 0x7f3c341009c0 2024-09-19T06:29:12.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.976+0000 7f3c31ffb640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3c240a33c0 0x7f3c240a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:12.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.977+0000 7f3c31ffb640 1 -- 172.21.15.5:0/3167533883 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8594+0+0 (secure 0 0 0) 0x7f3c280cf890 con 0x7f3c341009c0 2024-09-19T06:29:12.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.977+0000 7f3c33fff640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3c240a33c0 0x7f3c240a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:12.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.977+0000 7f3c33fff640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3c240a33c0 0x7f3c240a5880 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3c20031440 tx=0x7f3c20005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:12.981+0000 7f3c31ffb640 1 -- 172.21.15.5:0/3167533883 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c28009de0 con 0x7f3c341009c0 2024-09-19T06:29:13.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:13 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2401896037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:29:13.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:13 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1172394669' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:29:13.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:13 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3492006035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:29:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.206+0000 7f3c3ac53640 1 -- 172.21.15.5:0/3167533883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f3c34109f40 con 0x7f3c341009c0 2024-09-19T06:29:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.214+0000 7f3c31ffb640 1 -- 172.21.15.5:0/3167533883 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+742860 (secure 0 0 0) 0x7f3c2809bbd0 con 0x7f3c341009c0 2024-09-19T06:29:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:29:13.226 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:29:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 26, 2024-09-19T06:29:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "active_gid": 15240, 2024-09-19T06:29:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "active_name": "x", 2024-09-19T06:29:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addrs": { 2024-09-19T06:29:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2837801016 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2837801016 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addr": "172.21.15.5:6801/2837801016", 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "active_change": "2024-09-19T06:03:20.622292+0000", 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "available": true, 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "standbys": [ 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24832, 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "z", 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T06:29:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-19T06:29:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-19T06:29:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-19T06:29:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-19T06:29:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-19T06:29:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-19T06:29:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-19T06:29:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-19T06:29:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-19T06:29:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-19T06:29:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T06:29:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-19T06:29:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-19T06:29:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-19T06:29:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:13.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:13.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-19T06:29:13.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-19T06:29:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-19T06:29:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-19T06:29:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-19T06:29:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-19T06:29:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-19T06:29:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T06:29:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T06:29:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-19T06:29:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-19T06:29:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-19T06:29:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-19T06:29:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-19T06:29:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-19T06:29:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T06:29:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-19T06:29:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-19T06:29:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-19T06:29:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T06:29:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-19T06:29:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-19T06:29:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-19T06:29:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-19T06:29:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T06:29:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T06:29:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-19T06:29:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-19T06:29:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-19T06:29:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-19T06:29:13.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-19T06:29:13.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.309 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.310 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.310 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.310 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.310 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-19T06:29:13.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-19T06:29:13.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.310 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.311 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.314 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-19T06:29:13.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.318 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-19T06:29:13.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-19T06:29:13.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.321 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.324 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-19T06:29:13.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-19T06:29:13.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T06:29:13.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T06:29:13.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T06:29:13.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T06:29:13.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.329 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.332 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.332 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.332 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T06:29:13.332 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.332 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-19T06:29:13.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-19T06:29:13.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-19T06:29:13.337 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.337 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.337 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.337 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.337 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-19T06:29:13.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-19T06:29:13.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.344 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.351 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.351 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.351 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T06:29:13.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T06:29:13.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.352 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T06:29:13.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T06:29:13.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T06:29:13.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.356 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.362 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-19T06:29:13.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-19T06:29:13.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.362 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.364 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.365 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.365 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T06:29:13.365 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T06:29:13.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.365 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.365 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.366 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.370 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.375 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T06:29:13.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T06:29:13.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.378 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.381 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T06:29:13.381 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T06:29:13.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-19T06:29:13.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.384 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T06:29:13.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T06:29:13.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-19T06:29:13.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:13.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.392 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.392 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.392 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.392 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.392 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-19T06:29:13.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-19T06:29:13.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-19T06:29:13.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-19T06:29:13.394 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.394 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.394 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.394 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.395 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.396 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.397 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.399 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.400 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.400 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.400 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.400 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.400 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.401 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.405 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.407 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.407 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-19T06:29:13.407 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-19T06:29:13.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.407 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.407 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-19T06:29:13.407 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-19T06:29:13.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-19T06:29:13.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-19T06:29:13.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.409 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-19T06:29:13.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-19T06:29:13.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.412 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.415 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.416 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.416 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.417 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-19T06:29:13.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.421 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-19T06:29:13.421 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.421 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.421 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T06:29:13.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.421 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.422 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.424 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-19T06:29:13.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-19T06:29:13.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.424 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.424 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-19T06:29:13.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.424 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T06:29:13.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.428 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.432 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-19T06:29:13.435 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.435 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.435 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T06:29:13.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.435 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-19T06:29:13.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-19T06:29:13.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-19T06:29:13.436 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.439 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.442 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.443 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.445 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.446 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.447 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.450 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.451 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.454 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.456 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T06:29:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-19T06:29:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-19T06:29:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-19T06:29:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-19T06:29:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-19T06:29:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-19T06:29:13.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.547 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.548 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-19T06:29:13.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-19T06:29:13.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-19T06:29:13.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.549 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-19T06:29:13.550 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-19T06:29:13.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.552 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.552 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-19T06:29:13.552 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.552 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.552 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.553 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.557 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-19T06:29:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-19T06:29:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-19T06:29:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-19T06:29:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-19T06:29:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-19T06:29:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-19T06:29:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-19T06:29:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-19T06:29:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-19T06:29:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-19T06:29:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-19T06:29:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-19T06:29:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-19T06:29:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-19T06:29:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-19T06:29:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-19T06:29:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-19T06:29:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-19T06:29:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-19T06:29:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-19T06:29:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-19T06:29:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-19T06:29:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-19T06:29:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T06:29:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-19T06:29:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-19T06:29:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-19T06:29:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-19T06:29:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-19T06:29:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-19T06:29:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-19T06:29:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T06:29:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-19T06:29:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-19T06:29:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-19T06:29:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-19T06:29:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T06:29:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-19T06:29:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-19T06:29:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T06:29:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-19T06:29:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-19T06:29:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T06:29:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-19T06:29:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-19T06:29:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T06:29:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-19T06:29:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-19T06:29:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T06:29:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-19T06:29:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-19T06:29:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-19T06:29:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-19T06:29:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.645 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.645 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.645 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.645 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.645 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.645 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-19T06:29:13.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-19T06:29:13.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T06:29:13.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.646 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.646 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-19T06:29:13.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-19T06:29:13.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.647 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.647 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.648 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:13.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:13.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-19T06:29:13.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-19T06:29:13.648 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.648 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.649 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.649 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-19T06:29:13.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-19T06:29:13.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.649 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-19T06:29:13.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.650 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.650 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.650 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.650 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-19T06:29:13.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-19T06:29:13.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.650 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.651 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.652 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.652 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.652 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.652 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.653 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.653 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.653 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.653 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.653 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.653 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.653 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.653 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.654 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.654 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.654 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.654 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.654 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.654 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.655 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.656 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.656 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.656 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.656 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.656 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.656 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.656 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.656 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.657 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.658 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.658 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.658 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.658 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.658 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-19T06:29:13.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-19T06:29:13.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-19T06:29:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-19T06:29:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.660 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.660 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.660 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.660 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.660 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.661 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-19T06:29:13.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-19T06:29:13.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.661 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T06:29:13.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-19T06:29:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.663 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.663 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.663 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.663 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.663 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.663 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.663 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.663 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.663 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.664 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.664 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.664 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.664 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.665 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.665 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.666 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.666 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.667 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.667 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.667 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.667 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.667 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.667 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.667 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.668 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.668 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-19T06:29:13.669 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.669 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.670 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.670 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.670 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.670 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.671 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.671 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.671 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.671 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.671 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.671 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.671 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.671 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.671 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.671 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.672 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.672 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.672 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.672 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.672 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.672 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.673 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.673 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.674 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.674 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.674 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.674 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.674 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.674 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.674 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.674 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.674 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.675 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.675 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.675 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.675 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.676 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-19T06:29:13.676 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-19T06:29:13.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.676 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-19T06:29:13.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.677 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.677 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.677 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T06:29:13.677 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.677 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.677 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.677 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-19T06:29:13.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-19T06:29:13.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.677 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.678 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.678 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.678 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.678 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-19T06:29:13.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.678 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-19T06:29:13.679 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-19T06:29:13.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:13.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T06:29:13.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.679 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.680 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-19T06:29:13.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T06:29:13.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.680 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.680 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.680 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.680 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.680 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.680 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T06:29:13.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.681 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.681 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.681 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.681 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.681 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-19T06:29:13.681 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.681 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.681 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.681 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-19T06:29:13.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-19T06:29:13.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:13.682 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-19T06:29:13.682 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.682 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.682 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.682 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.682 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-19T06:29:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-19T06:29:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.684 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.684 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.684 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.684 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:13.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:13.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.685 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:13.685 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.685 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.685 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.685 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.686 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.686 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.686 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.686 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.688 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.688 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.688 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.688 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.689 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.689 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.689 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.689 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.689 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.690 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.690 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.690 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.690 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.690 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.690 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.691 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.691 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.691 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.692 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-19T06:29:13.692 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-19T06:29:13.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.692 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.692 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.693 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.693 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-19T06:29:13.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-19T06:29:13.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.693 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.693 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-19T06:29:13.693 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.694 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.694 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.694 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.694 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-19T06:29:13.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-19T06:29:13.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T06:29:13.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.695 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.695 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.696 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-19T06:29:13.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.696 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24890, 2024-09-19T06:29:13.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "y", 2024-09-19T06:29:13.696 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-09-19T06:29:13.696 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-19T06:29:13.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-19T06:29:13.696 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.696 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T06:29:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.698 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.698 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.699 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.699 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.699 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.699 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.699 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.699 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.699 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.699 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.700 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.701 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.701 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.701 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.701 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.701 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.702 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.702 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.702 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.702 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.702 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.702 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.702 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.702 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.703 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.703 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.703 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.703 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.703 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.704 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-19T06:29:13.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-19T06:29:13.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.704 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.704 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-19T06:29:13.705 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.705 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.705 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.705 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-19T06:29:13.705 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-19T06:29:13.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.706 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-19T06:29:13.706 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.706 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.706 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.706 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-19T06:29:13.706 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.706 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-19T06:29:13.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-19T06:29:13.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-19T06:29:13.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.708 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-19T06:29:13.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-19T06:29:13.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.708 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.709 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-19T06:29:13.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.709 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-19T06:29:13.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-19T06:29:13.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.710 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-19T06:29:13.710 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.710 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.710 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-19T06:29:13.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.710 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-19T06:29:13.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-19T06:29:13.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-19T06:29:13.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.711 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.712 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-19T06:29:13.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-19T06:29:13.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.713 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T06:29:13.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.713 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-19T06:29:13.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-19T06:29:13.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.713 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-19T06:29:13.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.714 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.714 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-19T06:29:13.715 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.715 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.715 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.715 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-19T06:29:13.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-19T06:29:13.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.716 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T06:29:13.716 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.716 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-19T06:29:13.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-19T06:29:13.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-19T06:29:13.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T06:29:13.717 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T06:29:13.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.718 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-19T06:29:13.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-19T06:29:13.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:13.718 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.718 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:13.718 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-19T06:29:13.718 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-19T06:29:13.718 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T06:29:13.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T06:29:13.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.719 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.719 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-19T06:29:13.719 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-19T06:29:13.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:13.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-19T06:29:13.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-19T06:29:13.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-19T06:29:13.720 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T06:29:13.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.720 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-19T06:29:13.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-19T06:29:13.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T06:29:13.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T06:29:13.721 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T06:29:13.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.721 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.721 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-19T06:29:13.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-19T06:29:13.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:13.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-19T06:29:13.722 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-19T06:29:13.722 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-19T06:29:13.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.722 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-19T06:29:13.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T06:29:13.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.722 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.723 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-19T06:29:13.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-19T06:29:13.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-19T06:29:13.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.723 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T06:29:13.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T06:29:13.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.724 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-19T06:29:13.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-19T06:29:13.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:13.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:13.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-19T06:29:13.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-19T06:29:13.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T06:29:13.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T06:29:13.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.725 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.725 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.726 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.727 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.729 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.729 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.729 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.729 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.729 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.729 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.729 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-19T06:29:13.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-19T06:29:13.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:13.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:13.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T06:29:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-19T06:29:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-19T06:29:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-19T06:29:13.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.733 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-19T06:29:13.733 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-19T06:29:13.733 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-19T06:29:13.733 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-19T06:29:13.733 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-19T06:29:13.733 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-19T06:29:13.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.734 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-19T06:29:13.734 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-19T06:29:13.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.735 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.735 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T06:29:13.735 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.735 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:13.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:13.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.736 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:13.736 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T06:29:13.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.736 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-19T06:29:13.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-19T06:29:13.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.737 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:13.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-19T06:29:13.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.737 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.737 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T06:29:13.737 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T06:29:13.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.737 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.738 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-19T06:29:13.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-19T06:29:13.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:13.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.738 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.738 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T06:29:13.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-19T06:29:13.739 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.739 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.739 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.740 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-19T06:29:13.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-19T06:29:13.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:13.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-19T06:29:13.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T06:29:13.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.741 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-19T06:29:13.741 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-19T06:29:13.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-19T06:29:13.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.742 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T06:29:13.742 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.742 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-19T06:29:13.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-19T06:29:13.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-19T06:29:13.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T06:29:13.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.743 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-19T06:29:13.743 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-19T06:29:13.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.744 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.744 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T06:29:13.744 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T06:29:13.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.745 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-19T06:29:13.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-19T06:29:13.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:13.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T06:29:13.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.746 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-19T06:29:13.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T06:29:13.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:13.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.746 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-19T06:29:13.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T06:29:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-19T06:29:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-19T06:29:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-19T06:29:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-19T06:29:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-19T06:29:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T06:29:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-19T06:29:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-19T06:29:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T06:29:13.750 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.750 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.750 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-19T06:29:13.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-19T06:29:13.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.751 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T06:29:13.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.751 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.751 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-19T06:29:13.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-19T06:29:13.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.752 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.752 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.752 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T06:29:13.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.752 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-19T06:29:13.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-19T06:29:13.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T06:29:13.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.753 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-19T06:29:13.753 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.754 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-19T06:29:13.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-19T06:29:13.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.754 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.754 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T06:29:13.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.754 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.755 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-19T06:29:13.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.756 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.756 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-19T06:29:13.756 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-19T06:29:13.756 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.756 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.757 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T06:29:13.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.757 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.757 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.757 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-19T06:29:13.757 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.757 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.757 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-19T06:29:13.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-19T06:29:13.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.758 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.758 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T06:29:13.758 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.758 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-19T06:29:13.758 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.758 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-19T06:29:13.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-19T06:29:13.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.759 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.759 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T06:29:13.759 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.759 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.759 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-19T06:29:13.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.760 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-19T06:29:13.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-19T06:29:13.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.760 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T06:29:13.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-19T06:29:13.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.762 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-19T06:29:13.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-19T06:29:13.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.762 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.762 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T06:29:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-19T06:29:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-19T06:29:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-19T06:29:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T06:29:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-19T06:29:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-19T06:29:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.767 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.767 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-19T06:29:13.767 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-19T06:29:13.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.767 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.767 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T06:29:13.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.767 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.768 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.768 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-19T06:29:13.768 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.768 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-19T06:29:13.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-19T06:29:13.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T06:29:13.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-19T06:29:13.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.769 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-19T06:29:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-19T06:29:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T06:29:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-19T06:29:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.771 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-19T06:29:13.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-19T06:29:13.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.771 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.771 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T06:29:13.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-19T06:29:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-19T06:29:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-19T06:29:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T06:29:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.773 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-19T06:29:13.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-19T06:29:13.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.774 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:13.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.774 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T06:29:13.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.775 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.775 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-19T06:29:13.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T06:29:13.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.775 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.775 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-19T06:29:13.775 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.775 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.775 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.776 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T06:29:13.776 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.776 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-19T06:29:13.776 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-19T06:29:13.776 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.776 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.778 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-19T06:29:13.778 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-19T06:29:13.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.778 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.778 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-19T06:29:13.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-19T06:29:13.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.779 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.779 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.779 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.779 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.779 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T06:29:13.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.780 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.781 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-19T06:29:13.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-19T06:29:13.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.781 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T06:29:13.781 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.781 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.781 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.781 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T06:29:13.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.782 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-19T06:29:13.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-19T06:29:13.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.782 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.782 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:13.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-19T06:29:13.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.783 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.783 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-19T06:29:13.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-19T06:29:13.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.784 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.784 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T06:29:13.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.785 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-19T06:29:13.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-19T06:29:13.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.785 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.785 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T06:29:13.785 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-19T06:29:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-19T06:29:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T06:29:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T06:29:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.791 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.791 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.791 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.791 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.791 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.791 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.791 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.791 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.791 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.792 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.792 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.792 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.792 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.792 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.793 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.793 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.793 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.793 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T06:29:13.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T06:29:13.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.794 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.794 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.794 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T06:29:13.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.795 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T06:29:13.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T06:29:13.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.795 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-19T06:29:13.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T06:29:13.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.796 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-19T06:29:13.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-19T06:29:13.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.796 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.796 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.796 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-19T06:29:13.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-19T06:29:13.797 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-19T06:29:13.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.798 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.798 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.798 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T06:29:13.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.798 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-19T06:29:13.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-19T06:29:13.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.799 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.799 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-19T06:29:13.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.800 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-19T06:29:13.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-19T06:29:13.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-19T06:29:13.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.800 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-19T06:29:13.801 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-19T06:29:13.801 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T06:29:13.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.801 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.801 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-19T06:29:13.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-19T06:29:13.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-19T06:29:13.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.802 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T06:29:13.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.802 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-19T06:29:13.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-19T06:29:13.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T06:29:13.803 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.803 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T06:29:13.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.804 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-19T06:29:13.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-19T06:29:13.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.804 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.804 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.805 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T06:29:13.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.805 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-19T06:29:13.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-19T06:29:13.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.806 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.806 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T06:29:13.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.806 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.806 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-19T06:29:13.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-19T06:29:13.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.807 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T06:29:13.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.808 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.808 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-19T06:29:13.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-19T06:29:13.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.809 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T06:29:13.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.809 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-19T06:29:13.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-19T06:29:13.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T06:29:13.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.810 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-19T06:29:13.810 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-19T06:29:13.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-19T06:29:13.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.811 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-19T06:29:13.811 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.811 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-19T06:29:13.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-19T06:29:13.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.812 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.812 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T06:29:13.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.813 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-19T06:29:13.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-19T06:29:13.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.813 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T06:29:13.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.814 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-19T06:29:13.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-19T06:29:13.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.814 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.814 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.814 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.815 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.815 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T06:29:13.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.815 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-19T06:29:13.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-19T06:29:13.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T06:29:13.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.816 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.816 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-19T06:29:13.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-19T06:29:13.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.817 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.817 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.817 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.817 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T06:29:13.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.818 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-19T06:29:13.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-19T06:29:13.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.818 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T06:29:13.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.819 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-19T06:29:13.819 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.820 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.820 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.820 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.821 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.821 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.821 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.821 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.821 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.821 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.821 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.821 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.822 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.822 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.822 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.822 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.823 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.824 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.824 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.824 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.824 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.824 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.824 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.824 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.824 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.824 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.826 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-19T06:29:13.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-19T06:29:13.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-19T06:29:13.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T06:29:13.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.827 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-19T06:29:13.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-19T06:29:13.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-19T06:29:13.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.828 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.828 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.828 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T06:29:13.828 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.829 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-19T06:29:13.829 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.829 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.829 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.829 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T06:29:13.829 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T06:29:13.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.830 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.831 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T06:29:13.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T06:29:13.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.832 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T06:29:13.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T06:29:13.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.833 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.833 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.833 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.833 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.833 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-19T06:29:13.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T06:29:13.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.834 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.834 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.834 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.835 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T06:29:13.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T06:29:13.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.835 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.836 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T06:29:13.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T06:29:13.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.836 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.836 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.836 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.837 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.837 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T06:29:13.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T06:29:13.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.838 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.838 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.838 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T06:29:13.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T06:29:13.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.839 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.839 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.839 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.839 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.839 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T06:29:13.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T06:29:13.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.840 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.840 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.840 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.840 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.841 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T06:29:13.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T06:29:13.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.841 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.841 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.842 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T06:29:13.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T06:29:13.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.842 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.843 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.843 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.843 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.843 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T06:29:13.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T06:29:13.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.844 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.844 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.844 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T06:29:13.844 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T06:29:13.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.845 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.845 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.845 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.845 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.846 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T06:29:13.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T06:29:13.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.846 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.846 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.846 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.846 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.847 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.847 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T06:29:13.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T06:29:13.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-19T06:29:13.847 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.848 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.848 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T06:29:13.848 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T06:29:13.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.848 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.849 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.849 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-19T06:29:13.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-19T06:29:13.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.850 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.850 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.851 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T06:29:13.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T06:29:13.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.851 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.851 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-19T06:29:13.851 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.851 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.851 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.851 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.852 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.852 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T06:29:13.852 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T06:29:13.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.852 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.853 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.853 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T06:29:13.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T06:29:13.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.854 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.854 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T06:29:13.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T06:29:13.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.855 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.855 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.855 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.855 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.856 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T06:29:13.856 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T06:29:13.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.856 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.857 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T06:29:13.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T06:29:13.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.857 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.858 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T06:29:13.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T06:29:13.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.859 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.859 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T06:29:13.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T06:29:13.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.860 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.860 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.860 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.860 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.860 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.860 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.861 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.861 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T06:29:13.861 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T06:29:13.861 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.861 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.862 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.862 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.862 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.862 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T06:29:13.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T06:29:13.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.863 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.863 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.863 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T06:29:13.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T06:29:13.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.864 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.864 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.864 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.864 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.865 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T06:29:13.865 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T06:29:13.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T06:29:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T06:29:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.867 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T06:29:13.867 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T06:29:13.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.867 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.868 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.868 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.868 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.868 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.868 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.868 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T06:29:13.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T06:29:13.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.869 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.869 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.869 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.869 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.870 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.870 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T06:29:13.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T06:29:13.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T06:29:13.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.870 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.871 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T06:29:13.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T06:29:13.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.871 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.871 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.872 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.872 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.872 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.872 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T06:29:13.872 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T06:29:13.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.872 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.873 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-19T06:29:13.873 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.873 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.873 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.873 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.873 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T06:29:13.874 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T06:29:13.874 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.874 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-19T06:29:13.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.875 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.875 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T06:29:13.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T06:29:13.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-19T06:29:13.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.877 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.877 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.877 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.877 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T06:29:13.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T06:29:13.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.878 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.878 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.878 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.878 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.878 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T06:29:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T06:29:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T06:29:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T06:29:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.881 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.881 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.881 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T06:29:13.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T06:29:13.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.882 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.882 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.882 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T06:29:13.882 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T06:29:13.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.882 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:13.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.883 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-19T06:29:13.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-19T06:29:13.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.884 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.885 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-19T06:29:13.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-19T06:29:13.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.885 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.885 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.885 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.885 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-19T06:29:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-19T06:29:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-19T06:29:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-19T06:29:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-19T06:29:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-19T06:29:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-19T06:29:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-19T06:29:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-19T06:29:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-19T06:29:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-19T06:29:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-19T06:29:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T06:29:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.900 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.900 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-19T06:29:13.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-19T06:29:13.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.902 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.902 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.902 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.902 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.902 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.902 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.902 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-19T06:29:13.903 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-19T06:29:13.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-19T06:29:13.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.903 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.904 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-19T06:29:13.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-19T06:29:13.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-19T06:29:13.904 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.904 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.905 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-19T06:29:13.905 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.905 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-19T06:29:13.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-19T06:29:13.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.905 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-19T06:29:13.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.907 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.907 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.907 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.907 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-19T06:29:13.908 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.908 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.908 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-19T06:29:13.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-19T06:29:13.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.908 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.909 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.909 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.909 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.909 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-19T06:29:13.909 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.909 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.909 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.910 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.910 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.910 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.911 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.911 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.911 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.911 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.911 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.911 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.911 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.912 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.912 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.912 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.912 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.912 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.912 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.913 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.913 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.913 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.913 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.914 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.914 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.914 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.914 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.914 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.915 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.915 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.915 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.915 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.915 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.915 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.915 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.915 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-19T06:29:13.916 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-19T06:29:13.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.916 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.916 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-19T06:29:13.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.916 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.916 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.916 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T06:29:13.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.917 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-19T06:29:13.917 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-19T06:29:13.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.917 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-19T06:29:13.917 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.918 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T06:29:13.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.918 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.918 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-19T06:29:13.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-19T06:29:13.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-19T06:29:13.919 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.919 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.919 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-19T06:29:13.919 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.919 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-19T06:29:13.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-19T06:29:13.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.920 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-19T06:29:13.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T06:29:13.920 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.920 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.921 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-19T06:29:13.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-19T06:29:13.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:13.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.921 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.921 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.921 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T06:29:13.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.922 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:13.922 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:13.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.922 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.922 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:13.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T06:29:13.923 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.923 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.923 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-19T06:29:13.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-19T06:29:13.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-19T06:29:13.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.924 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T06:29:13.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.924 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-19T06:29:13.925 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.925 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.925 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.925 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.926 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.927 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.927 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.927 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.928 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.928 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.928 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.929 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.929 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.929 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.929 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.929 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.929 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.929 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.929 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.931 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.931 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.931 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.931 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-19T06:29:13.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-19T06:29:13.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-19T06:29:13.932 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.932 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.932 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.932 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-19T06:29:13.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-19T06:29:13.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-19T06:29:13.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.933 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.933 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.935 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.935 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.935 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.935 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-19T06:29:13.936 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-19T06:29:13.936 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-19T06:29:13.936 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.936 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-19T06:29:13.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-19T06:29:13.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.936 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.937 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-19T06:29:13.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.937 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T06:29:13.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.937 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.937 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-19T06:29:13.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-19T06:29:13.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-19T06:29:13.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.938 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T06:29:13.938 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.939 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-19T06:29:13.939 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-19T06:29:13.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.939 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.940 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-19T06:29:13.940 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.940 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:13.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:13.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:13.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.941 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-19T06:29:13.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-19T06:29:13.941 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T06:29:13.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.941 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.942 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.942 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.942 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.943 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.943 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.943 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.943 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.943 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.943 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.944 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.944 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.944 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.944 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.945 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.945 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.946 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.946 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.946 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.946 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.946 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.946 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.947 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.947 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.947 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.948 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-19T06:29:13.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-19T06:29:13.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.948 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-19T06:29:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-19T06:29:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-19T06:29:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-19T06:29:13.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T06:29:13.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.951 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-19T06:29:13.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-19T06:29:13.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:13.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-19T06:29:13.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-19T06:29:13.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T06:29:13.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.953 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-19T06:29:13.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-19T06:29:13.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.953 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.953 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.953 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.953 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.953 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-19T06:29:13.953 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.954 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.954 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.954 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-19T06:29:13.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-19T06:29:13.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-19T06:29:13.954 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.954 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T06:29:13.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-19T06:29:13.955 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.955 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.956 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.956 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.956 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.957 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.957 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.957 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.957 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.957 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.957 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.959 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.959 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.959 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.959 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.959 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.959 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.960 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.960 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.960 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.960 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.960 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.960 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.960 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.961 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.961 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.961 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.962 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.962 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.962 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-19T06:29:13.962 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.963 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.963 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.963 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.964 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.964 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.964 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.964 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.964 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.964 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.964 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.965 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.965 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.965 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.967 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.967 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.967 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.967 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.967 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.967 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.967 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.967 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.967 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.968 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.968 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.968 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.968 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.968 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.969 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-19T06:29:13.969 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.969 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.969 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.969 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.971 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-19T06:29:13.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-19T06:29:13.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:13.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:13.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-19T06:29:13.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.971 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T06:29:13.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.972 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.972 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.972 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.973 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.973 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.973 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.973 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.973 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.973 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.974 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.974 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.974 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.974 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.974 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.974 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.975 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.975 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.975 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.975 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.975 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.976 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.976 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.976 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.976 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.976 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.976 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.976 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.976 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.976 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.976 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.976 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.977 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.977 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.977 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.977 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.977 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-19T06:29:13.978 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.978 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.978 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.978 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-19T06:29:13.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-19T06:29:13.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-19T06:29:13.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.979 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-19T06:29:13.979 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T06:29:13.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.979 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.980 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.980 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.980 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.980 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.980 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.980 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.981 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.981 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.981 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.981 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.981 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.981 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.981 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.981 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.985 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.985 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.985 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.985 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.987 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-19T06:29:13.987 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T06:29:13.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.987 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.987 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-19T06:29:13.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-19T06:29:13.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.988 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-19T06:29:13.988 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.988 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.988 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-19T06:29:13.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.988 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-19T06:29:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-19T06:29:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-19T06:29:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-19T06:29:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-19T06:29:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-19T06:29:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T06:29:13.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T06:29:13.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.992 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-19T06:29:13.992 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.992 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:13.993 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:13.993 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:13.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:13.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.993 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:13.993 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.993 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.994 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.994 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.994 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.994 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.994 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.994 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.994 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.994 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.994 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.995 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.995 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:13.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:13.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.995 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.995 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.995 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.995 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.996 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.996 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.996 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:13.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:13.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:13.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:13.997 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.997 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:13.997 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:13.997 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:13.997 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:13.997 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:13.997 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:13.997 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:13.997 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:13.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.998 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.998 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.998 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:13.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:13.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:13.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:13.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:13.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:13.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:13.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:13.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:13.999 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:13.999 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:13.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:13.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:13.999 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.999 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:13.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:13.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:13.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-19T06:29:13.999 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:13.999 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.000 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.000 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.000 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.000 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.000 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.000 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.000 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.000 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.001 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.001 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.001 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.001 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.001 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.001 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.005 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.005 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.005 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.005 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.006 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.006 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.006 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.006 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.006 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.006 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-19T06:29:14.008 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.008 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.008 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.008 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.009 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.009 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.009 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.009 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.009 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.009 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.009 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.010 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.010 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.011 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.011 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.012 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.012 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.012 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.012 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.012 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.012 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.028 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.028 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.028 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.028 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.028 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.028 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.030 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.030 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.030 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.030 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.030 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.030 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.031 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.031 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.031 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.031 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.031 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-19T06:29:14.032 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.032 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.032 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.032 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.032 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.032 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.032 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.032 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.032 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.033 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.033 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.033 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.033 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.033 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.033 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.033 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.033 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.033 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.033 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.033 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.034 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.034 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.034 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.034 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.034 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.034 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.034 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.035 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.035 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.035 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.038 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.038 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.038 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.038 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-19T06:29:14.038 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.041 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.041 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.041 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.041 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.041 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.041 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.041 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.042 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.042 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.042 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.042 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.042 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.042 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.042 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.042 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.043 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.043 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.043 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.043 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.043 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.043 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.044 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.044 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.044 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.044 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.045 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:14.045 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:14.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.045 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.045 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:14.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.045 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.046 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.046 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-19T06:29:14.046 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-19T06:29:14.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:14.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-19T06:29:14.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-19T06:29:14.047 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.047 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.047 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-19T06:29:14.047 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T06:29:14.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.047 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-19T06:29:14.047 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.048 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.048 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.048 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-19T06:29:14.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-19T06:29:14.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.048 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.048 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.048 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.048 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.049 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T06:29:14.049 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.049 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.049 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-19T06:29:14.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-19T06:29:14.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.050 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.050 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.050 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.050 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.051 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.051 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.051 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.051 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.051 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.051 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.052 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.052 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.052 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.052 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.052 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.052 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.052 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.053 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.054 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.054 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.054 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.054 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.054 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.054 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.054 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.054 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.054 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.055 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.055 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.055 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.055 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.055 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.055 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.055 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.055 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.056 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.056 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.056 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-19T06:29:14.057 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-19T06:29:14.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.057 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-19T06:29:14.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.057 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-19T06:29:14.057 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.057 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.058 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:14.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:14.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.058 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.058 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:14.058 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.058 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.058 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-19T06:29:14.058 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.059 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.059 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-19T06:29:14.059 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.059 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.059 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.059 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-19T06:29:14.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-19T06:29:14.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.060 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.060 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.060 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.060 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.061 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-19T06:29:14.061 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-19T06:29:14.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.061 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.061 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.061 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.061 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T06:29:14.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T06:29:14.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.062 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.062 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.062 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.062 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.062 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.062 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.063 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.063 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.063 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.063 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.063 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.063 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.063 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.064 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.064 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.064 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.064 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.065 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.065 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.065 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.065 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.065 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.066 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.066 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.066 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.066 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.066 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.066 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.066 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.066 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.066 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.066 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.066 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.067 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.067 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.067 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.068 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.068 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.068 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.068 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.068 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.068 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-19T06:29:14.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-19T06:29:14.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.068 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.069 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.069 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.069 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.069 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T06:29:14.069 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T06:29:14.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-19T06:29:14.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.070 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.070 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.070 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.070 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.071 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-19T06:29:14.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-19T06:29:14.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:14.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.071 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-19T06:29:14.071 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.071 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.071 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.071 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.072 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.072 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.072 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-19T06:29:14.072 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-19T06:29:14.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.072 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-19T06:29:14.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T06:29:14.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.073 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.073 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-19T06:29:14.073 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-19T06:29:14.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-19T06:29:14.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.074 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.074 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.074 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T06:29:14.074 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.076 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-19T06:29:14.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-19T06:29:14.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-19T06:29:14.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.080 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.082 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.084 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.089 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.091 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.091 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.091 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.091 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.091 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.093 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.096 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.098 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.100 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.101 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.102 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T06:29:14.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.105 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.106 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.108 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.109 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.109 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.109 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.109 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.109 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.109 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.109 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.109 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-19T06:29:14.111 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.113 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.114 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.116 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.118 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.118 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.118 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-19T06:29:14.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-19T06:29:14.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.118 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-19T06:29:14.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-19T06:29:14.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.122 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.122 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.122 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.122 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.122 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.122 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.122 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.122 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.124 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-19T06:29:14.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-19T06:29:14.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.124 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.124 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.125 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-19T06:29:14.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-19T06:29:14.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.125 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.125 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.125 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.125 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.126 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.126 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.126 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.126 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-19T06:29:14.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-19T06:29:14.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.127 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.127 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.127 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-19T06:29:14.127 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-19T06:29:14.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.127 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.129 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-19T06:29:14.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.132 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.134 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.135 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.136 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.137 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-19T06:29:14.140 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.140 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.140 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.140 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.140 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.140 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.140 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.140 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.140 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.140 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.141 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.142 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.143 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.145 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.146 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.148 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.150 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.150 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.150 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.150 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-19T06:29:14.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:14.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:14.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.153 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.153 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-19T06:29:14.153 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.153 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.153 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.154 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.156 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.158 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.164 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.165 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.165 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.165 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.165 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.165 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.165 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.165 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.165 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.165 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-19T06:29:14.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.166 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.169 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.173 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.174 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-19T06:29:14.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.178 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-19T06:29:14.180 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.181 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.181 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.181 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.181 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.181 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.182 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.183 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.184 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.185 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.186 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.187 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.189 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.190 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-19T06:29:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-19T06:29:14.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-19T06:29:14.193 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.194 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.197 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.197 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-19T06:29:14.197 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-19T06:29:14.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.197 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.197 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.197 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:14.198 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.202 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.206 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.207 INFO:teuthology.orchestra.run.smithi005.stdout: "modules": [ 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard", 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: "iostat", 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: "nfs", 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: "restful", 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_schedule" 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.208 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.210 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.212 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.216 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-19T06:29:14.217 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.217 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.217 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-19T06:29:14.217 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-19T06:29:14.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.217 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.217 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.218 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-19T06:29:14.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.220 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-19T06:29:14.223 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T06:29:14.224 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.227 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T06:29:14.228 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.228 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.228 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-19T06:29:14.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-19T06:29:14.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.228 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T06:29:14.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-19T06:29:14.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.230 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:14.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.233 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.235 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.238 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T06:29:14.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.255 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.257 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.257 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-19T06:29:14.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-19T06:29:14.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.257 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T06:29:14.257 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.257 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.257 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.257 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.258 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.260 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.260 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-19T06:29:14.260 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.260 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.260 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-19T06:29:14.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-19T06:29:14.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.260 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.261 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T06:29:14.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.264 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.264 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-19T06:29:14.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.264 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.264 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-19T06:29:14.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-19T06:29:14.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-19T06:29:14.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-19T06:29:14.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-19T06:29:14.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-19T06:29:14.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-19T06:29:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-19T06:29:14.270 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-19T06:29:14.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-19T06:29:14.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.281 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.284 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.288 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.289 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.291 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-19T06:29:14.294 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.295 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.297 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.299 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-19T06:29:14.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-19T06:29:14.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-19T06:29:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.305 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.306 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.309 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-19T06:29:14.309 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-19T06:29:14.309 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.309 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T06:29:14.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.313 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-19T06:29:14.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-19T06:29:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.319 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.329 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.332 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.334 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.337 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.337 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.337 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.337 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.337 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T06:29:14.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T06:29:14.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T06:29:14.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.343 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T06:29:14.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T06:29:14.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T06:29:14.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.346 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.347 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T06:29:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T06:29:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.354 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-19T06:29:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.367 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.368 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.370 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.370 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.370 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.370 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.370 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.370 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.370 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-19T06:29:14.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-19T06:29:14.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-19T06:29:14.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.375 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.376 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.379 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-19T06:29:14.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.383 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.384 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.393 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.394 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.397 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.399 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.399 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.400 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-19T06:29:14.403 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-19T06:29:14.405 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-19T06:29:14.408 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.410 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.413 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.414 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-19T06:29:14.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.418 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T06:29:14.418 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.418 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-19T06:29:14.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-19T06:29:14.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-19T06:29:14.421 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.425 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.427 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.428 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.428 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.428 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.428 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.428 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.428 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.428 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.429 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-19T06:29:14.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-19T06:29:14.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.436 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.437 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.439 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.440 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-19T06:29:14.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.444 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.444 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.445 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.446 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-19T06:29:14.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-19T06:29:14.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-19T06:29:14.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-19T06:29:14.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.449 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-19T06:29:14.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.452 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-19T06:29:14.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-19T06:29:14.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-19T06:29:14.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T06:29:14.452 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T06:29:14.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.454 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.457 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.459 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.463 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.465 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.468 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.470 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.472 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.476 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.476 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.476 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-19T06:29:14.477 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.478 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.479 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.485 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.486 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:14.495 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-19T06:29:14.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.498 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.503 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.511 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.513 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.513 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.513 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.513 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.513 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.513 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.513 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.513 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.513 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.513 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.516 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-19T06:29:14.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-19T06:29:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-19T06:29:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.520 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.526 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.529 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.530 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.532 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-19T06:29:14.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-19T06:29:14.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.535 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.536 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.539 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.544 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.544 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.544 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.544 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.544 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.544 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.544 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.544 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.544 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.544 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.544 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.546 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.548 INFO:teuthology.orchestra.run.smithi005.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.549 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.553 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.556 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.557 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.568 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-19T06:29:14.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-19T06:29:14.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-19T06:29:14.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-19T06:29:14.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-19T06:29:14.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.575 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.576 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T06:29:14.576 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.576 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.576 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-19T06:29:14.576 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-19T06:29:14.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.576 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.579 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.580 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.586 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.587 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.592 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.593 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.594 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.597 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.598 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-19T06:29:14.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-19T06:29:14.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-19T06:29:14.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.606 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.607 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.612 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:14.612 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:14.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.612 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.612 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-19T06:29:14.612 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-19T06:29:14.612 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.612 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.613 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.614 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.618 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.619 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.620 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.628 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.633 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.634 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-19T06:29:14.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-19T06:29:14.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-19T06:29:14.637 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.638 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.640 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-19T06:29:14.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-19T06:29:14.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.649 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.650 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: "services": { 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard": "https://172.21.15.5:8443/" 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: "always_on_modules": { 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: "octopus": [ 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-19T06:29:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "pacific": [ 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-19T06:29:14.653 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: "quincy": [ 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-19T06:29:14.654 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: "reef": [ 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-19T06:29:14.655 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: "squid": [ 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.656 INFO:teuthology.orchestra.run.smithi005.stdout: "last_failure_osd_epoch": 84, 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: "active_clients": [ 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1880262708 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "libcephsqlite", 2024-09-19T06:29:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1002489810 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-19T06:29:14.658 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 73816063 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4023233039 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-19T06:29:14.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 505685129 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-19T06:29:14.660 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-19T06:29:14.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-19T06:29:14.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-19T06:29:14.661 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-19T06:29:14.661 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2263804142 2024-09-19T06:29:14.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.661 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-19T06:29:14.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:14.661 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-19T06:29:14.661 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:29:14.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:13 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2401896037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:29:14.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:13 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1172394669' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:29:14.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:13 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3492006035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:29:14.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:14 smithi157 ceph-mon[35346]: pgmap v819: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-19T06:29:14.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:14 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/3167533883' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T06:29:14.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:14 smithi157 ceph-mon[35346]: osdmap e125: 12 total, 12 up, 12 in 2024-09-19T06:29:14.663 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:13 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2401896037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:29:14.663 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:13 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1172394669' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:29:14.663 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:13 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3492006035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:29:14.663 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:14 smithi049 ceph-mon[34640]: pgmap v819: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-19T06:29:14.664 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:14 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/3167533883' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T06:29:14.664 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:14 smithi049 ceph-mon[34640]: osdmap e125: 12 total, 12 up, 12 in 2024-09-19T06:29:14.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.219+0000 7f3c3ac53640 1 -- 172.21.15.5:0/3167533883 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3c240a33c0 msgr2=0x7f3c240a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:14.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.219+0000 7f3c3ac53640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3c240a33c0 0x7f3c240a5880 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3c20031440 tx=0x7f3c20005a20 comp rx=0 tx=0).stop 2024-09-19T06:29:14.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.219+0000 7f3c3ac53640 1 -- 172.21.15.5:0/3167533883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c341009c0 msgr2=0x7f3c34198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.219+0000 7f3c3ac53640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c341009c0 0x7f3c34198350 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800d950 tx=0x7f3c2800de10 comp rx=0 tx=0).stop 2024-09-19T06:29:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.219+0000 7f3c3ac53640 1 -- 172.21.15.5:0/3167533883 shutdown_connections 2024-09-19T06:29:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.219+0000 7f3c3ac53640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3c240a33c0 0x7f3c240a5880 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.219+0000 7f3c3ac53640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c341069d0 0x7f3c3419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.219+0000 7f3c3ac53640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c34101360 0x7f3c34198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.219+0000 7f3c3ac53640 1 --2- 172.21.15.5:0/3167533883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c341009c0 0x7f3c34198350 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.219+0000 7f3c3ac53640 1 -- 172.21.15.5:0/3167533883 >> 172.21.15.5:0/3167533883 conn(0x7f3c34077a10 msgr2=0x7f3c340697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.219+0000 7f3c3ac53640 1 -- 172.21.15.5:0/3167533883 shutdown_connections 2024-09-19T06:29:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:13.219+0000 7f3c3ac53640 1 -- 172.21.15.5:0/3167533883 wait complete. 2024-09-19T06:29:14.728 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T06:29:14.729 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T06:29:14.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:14 smithi005 ceph-mon[30955]: pgmap v819: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-19T06:29:14.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:14 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/3167533883' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T06:29:14.730 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:14 smithi005 ceph-mon[30955]: osdmap e125: 12 total, 12 up, 12 in 2024-09-19T06:29:15.002 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:15.541 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:29:15.541 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:29:15.541 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2422, 2024-09-19T06:29:15.541 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2424, 2024-09-19T06:29:15.541 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 90524671, 2024-09-19T06:29:15.541 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 770197, 2024-09-19T06:29:15.541 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 9, 2024-09-19T06:29:15.541 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 4, 2024-09-19T06:29:15.541 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.247978493, 2024-09-19T06:29:15.541 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.063000835, 2024-09-19T06:29:15.542 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.102509326, 2024-09-19T06:29:15.542 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.064539756, 2024-09-19T06:29:15.542 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:15.542 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2422, 2024-09-19T06:29:15.542 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.069033108, 2024-09-19T06:29:15.542 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000028502 2024-09-19T06:29:15.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.542 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:15.542 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.542 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.542 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.543 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T06:29:15.543 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T06:29:15.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.543 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:29:15.543 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 3047, 2024-09-19T06:29:15.543 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2997, 2024-09-19T06:29:15.543 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 90676646, 2024-09-19T06:29:15.543 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1080343, 2024-09-19T06:29:15.543 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-19T06:29:15.543 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 5, 2024-09-19T06:29:15.543 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.280907220, 2024-09-19T06:29:15.543 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.068269370, 2024-09-19T06:29:15.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.162763385, 2024-09-19T06:29:15.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.054241851, 2024-09-19T06:29:15.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:15.544 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2996, 2024-09-19T06:29:15.544 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.085300933, 2024-09-19T06:29:15.544 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000028471 2024-09-19T06:29:15.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:15.544 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.544 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.544 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1959664, 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 283696 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 779, 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1115, 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 92475088, 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1014993, 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 10, 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 6, 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.151714297, 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.030486904, 2024-09-19T06:29:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.094262753, 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.018846496, 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1113, 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.043012153, 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000038645 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-19T06:29:15.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 769168 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 9639, 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.207256305, 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000021501 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:15.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.548 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 4580, 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 949, 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 3631, 2024-09-19T06:29:15.549 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-09-19T06:29:15.550 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 3224, 2024-09-19T06:29:15.550 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 0, 2024-09-19T06:29:15.550 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-09-19T06:29:15.550 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-09-19T06:29:15.550 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 1, 2024-09-19T06:29:15.550 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-09-19T06:29:15.550 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-09-19T06:29:15.550 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-09-19T06:29:15.550 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-09-19T06:29:15.550 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:29:15.550 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-09-19T06:29:15.550 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-09-19T06:29:15.551 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T06:29:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-09-19T06:29:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:29:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:29:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-09-19T06:29:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:29:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T06:29:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-09-19T06:29:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-09-19T06:29:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T06:29:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-09-19T06:29:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:29:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-09-19T06:29:15.553 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:29:15.554 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 32369, 2024-09-19T06:29:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 74, 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 4, 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 274323388, 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 274307495, 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-09-19T06:29:15.556 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 32369 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 2403, 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 4631, 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 2228, 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 83, 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 103, 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 20, 2024-09-19T06:29:15.557 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 4580, 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 5199, 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 619, 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 62140, 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215328 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-09-19T06:29:15.558 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.560 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-09-19T06:29:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-09-19T06:29:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-09-19T06:29:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-09-19T06:29:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-09-19T06:29:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-09-19T06:29:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-09-19T06:29:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.567 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.568 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:15.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 2, 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 5, 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 3, 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 4, 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 3689, 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:29:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:29:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-19T06:29:15.573 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:29:15.573 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-19T06:29:15.573 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:29:15.573 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:29:15.573 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:29:15.573 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:29:15.573 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:29:15.573 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:29:15.573 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:29:15.573 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-19T06:29:15.573 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:29:15.573 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 240225, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 81, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-19T06:29:15.574 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 8352, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 185, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 8605177, 2024-09-19T06:29:15.575 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 25668, 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 5700, 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 5700, 2024-09-19T06:29:15.576 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-09-19T06:29:15.577 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 5700, 2024-09-19T06:29:15.577 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 4.121917839, 2024-09-19T06:29:15.577 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000723143 2024-09-19T06:29:15.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.577 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-09-19T06:29:15.577 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-09-19T06:29:15.577 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 5700, 2024-09-19T06:29:15.577 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 5701 2024-09-19T06:29:15.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.577 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 5700, 2024-09-19T06:29:15.577 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 5700, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 3274, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 2424, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-09-19T06:29:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-09-19T06:29:15.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-09-19T06:29:15.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-09-19T06:29:15.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-09-19T06:29:15.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-09-19T06:29:15.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-09-19T06:29:15.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:29:15.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-09-19T06:29:15.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:29:15.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-09-19T06:29:15.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-09-19T06:29:15.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-09-19T06:29:15.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:29:15.580 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-09-19T06:29:15.580 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:29:15.580 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-09-19T06:29:15.580 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-09-19T06:29:15.580 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-09-19T06:29:15.580 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-09-19T06:29:15.580 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-09-19T06:29:15.580 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-09-19T06:29:15.580 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-09-19T06:29:15.580 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-09-19T06:29:15.580 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-09-19T06:29:15.580 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 125, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 42, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 11, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 14, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 3, 2024-09-19T06:29:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-09-19T06:29:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-19T06:29:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 6244, 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 273083944, 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 6244, 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 6244, 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 273083944, 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 5700, 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 270129089, 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 5700, 2024-09-19T06:29:15.585 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 270129089, 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:15.586 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 5700, 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 5700, 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 5700, 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 5700, 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-19T06:29:15.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:15.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:15.589 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x55aee74766e0": { 2024-09-19T06:29:15.589 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-19T06:29:15.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-19T06:29:15.589 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-19T06:29:15.589 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-19T06:29:15.589 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-19T06:29:15.589 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:15.589 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:15.589 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-19T06:29:15.589 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-19T06:29:15.589 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-19T06:29:15.590 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-19T06:29:15.590 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-19T06:29:15.590 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:15.590 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:15.590 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:15.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:15.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:15.590 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:29:16.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:16 smithi005 ceph-mon[30955]: pgmap v821: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-19T06:29:16.324 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-19T06:29:16.325 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-19T06:29:16.325 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-19T06:29:16.325 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:29:16.325 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:29:16.325 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T06:29:16.325 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T06:29:16.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:16 smithi049 ceph-mon[34640]: pgmap v821: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-19T06:29:16.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:16 smithi157 ceph-mon[35346]: pgmap v821: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-19T06:29:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:17.105 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:29:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:29:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 465, 2024-09-19T06:29:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 452, 2024-09-19T06:29:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1658000, 2024-09-19T06:29:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 279120, 2024-09-19T06:29:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-19T06:29:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-09-19T06:29:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.066154327, 2024-09-19T06:29:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.025717270, 2024-09-19T06:29:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.052966546, 2024-09-19T06:29:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:29:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 452, 2024-09-19T06:29:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.020972752, 2024-09-19T06:29:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000046399 2024-09-19T06:29:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1658000, 2024-09-19T06:29:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 279120 2024-09-19T06:29:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:29:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 3, 2024-09-19T06:29:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 346, 2024-09-19T06:29:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 480, 2024-09-19T06:29:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 296720, 2024-09-19T06:29:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-19T06:29:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-09-19T06:29:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.021731833, 2024-09-19T06:29:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.016147330, 2024-09-19T06:29:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002280206, 2024-09-19T06:29:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:29:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 343, 2024-09-19T06:29:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.015630910, 2024-09-19T06:29:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000045571 2024-09-19T06:29:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T06:29:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 296720 2024-09-19T06:29:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:29:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-09-19T06:29:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-09-19T06:29:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-09-19T06:29:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-09-19T06:29:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-09-19T06:29:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-09-19T06:29:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.000321781, 2024-09-19T06:29:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T06:29:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000224228, 2024-09-19T06:29:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:29:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-19T06:29:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:29:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:29:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148731, 2024-09-19T06:29:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 31, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:29:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-19T06:29:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 464, 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1589895, 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 464, 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 464, 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1589895, 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:17.120 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:29:17.901 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T06:29:17.901 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T06:29:17.901 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-19T06:29:17.902 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-19T06:29:17.902 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-19T06:29:17.902 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T06:29:17.902 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T06:29:18.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:18 smithi005 ceph-mon[30955]: pgmap v822: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 639 B/s wr, 15 op/s 2024-09-19T06:29:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:18.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:18 smithi049 ceph-mon[34640]: pgmap v822: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 639 B/s wr, 15 op/s 2024-09-19T06:29:18.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:18 smithi157 ceph-mon[35346]: pgmap v822: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 639 B/s wr, 15 op/s 2024-09-19T06:29:18.673 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:29:18.674 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:29:18.674 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2965, 2024-09-19T06:29:18.674 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2915, 2024-09-19T06:29:18.674 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 115957787, 2024-09-19T06:29:18.674 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1054854, 2024-09-19T06:29:18.674 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 7, 2024-09-19T06:29:18.674 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-09-19T06:29:18.674 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.357516511, 2024-09-19T06:29:18.674 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.103102659, 2024-09-19T06:29:18.674 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.190409800, 2024-09-19T06:29:18.675 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.064686213, 2024-09-19T06:29:18.675 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:18.675 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2914, 2024-09-19T06:29:18.675 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.090747452, 2024-09-19T06:29:18.675 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000031141 2024-09-19T06:29:18.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.675 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:18.675 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.675 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.675 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.675 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1960208, 2024-09-19T06:29:18.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 284976 2024-09-19T06:29:18.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.676 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:29:18.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2649, 2024-09-19T06:29:18.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2987, 2024-09-19T06:29:18.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 124901189, 2024-09-19T06:29:18.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1611667, 2024-09-19T06:29:18.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 9, 2024-09-19T06:29:18.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 7, 2024-09-19T06:29:18.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.279330284, 2024-09-19T06:29:18.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.060687704, 2024-09-19T06:29:18.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.143094564, 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.059292805, 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2985, 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.085313680, 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000028580 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-19T06:29:18.677 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 771136 2024-09-19T06:29:18.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.678 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:29:18.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 723, 2024-09-19T06:29:18.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 725, 2024-09-19T06:29:18.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 109236871, 2024-09-19T06:29:18.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 229926, 2024-09-19T06:29:18.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 8, 2024-09-19T06:29:18.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 5, 2024-09-19T06:29:18.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.131494135, 2024-09-19T06:29:18.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.016507945, 2024-09-19T06:29:18.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.091842573, 2024-09-19T06:29:18.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.017969646, 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 723, 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.020060616, 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000027746 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.679 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 9680, 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.212490355, 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000021951 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-09-19T06:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 13202, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 0, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 13202, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 13202, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 1, 2024-09-19T06:29:18.682 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-09-19T06:29:18.683 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-09-19T06:29:18.683 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 1, 2024-09-19T06:29:18.683 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-09-19T06:29:18.683 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-09-19T06:29:18.683 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-09-19T06:29:18.683 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-09-19T06:29:18.683 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:29:18.683 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-09-19T06:29:18.683 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-09-19T06:29:18.683 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-09-19T06:29:18.683 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-09-19T06:29:18.683 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:29:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-09-19T06:29:18.685 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-09-19T06:29:18.687 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-09-19T06:29:18.687 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-09-19T06:29:18.687 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:29:18.687 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:29:18.687 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-09-19T06:29:18.687 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-09-19T06:29:18.687 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:29:18.687 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:29:18.687 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:29:18.687 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-09-19T06:29:18.687 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-09-19T06:29:18.687 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 75269, 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-09-19T06:29:18.688 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 111, 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 8, 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 350717363, 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 350714537, 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 75269 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.689 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1718, 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 13373, 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 11655, 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 59, 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 92, 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 33, 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 13202, 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 13775, 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 573, 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-09-19T06:29:18.690 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 69832, 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215328 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.691 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-09-19T06:29:18.692 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.694 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-09-19T06:29:18.694 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.694 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.694 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.694 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-09-19T06:29:18.694 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.694 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.694 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.694 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-09-19T06:29:18.694 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.695 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.695 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.695 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-09-19T06:29:18.695 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.695 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.695 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.696 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-09-19T06:29:18.696 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.696 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.696 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.696 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-09-19T06:29:18.696 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.696 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.696 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.696 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-09-19T06:29:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.700 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-09-19T06:29:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.703 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:29:18.703 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:29:18.703 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-09-19T06:29:18.703 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.703 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.703 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:18.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.703 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-09-19T06:29:18.703 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 2, 2024-09-19T06:29:18.703 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 3, 2024-09-19T06:29:18.703 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 1, 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 2, 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 2626, 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:29:18.704 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:29:18.705 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:29:18.705 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:29:18.705 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:29:18.705 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:29:18.705 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:29:18.705 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:29:18.705 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:29:18.705 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:29:18.705 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:29:18.705 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-19T06:29:18.705 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:29:18.705 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-19T06:29:18.706 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 240472, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 80, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-19T06:29:18.707 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 8232, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 182, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 13286118, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 61102, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-19T06:29:18.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 5787, 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 5787, 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 5787, 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 4.103918126, 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000709161 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.709 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 5787, 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 5788 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 5787, 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 5787, 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 3355, 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 2430, 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-09-19T06:29:18.710 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-09-19T06:29:18.712 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 125, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-09-19T06:29:18.713 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 42, 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 11, 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 13, 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 2, 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-09-19T06:29:18.714 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-09-19T06:29:18.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 6333, 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 349495272, 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 6333, 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 6333, 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 349495272, 2024-09-19T06:29:18.716 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:18.717 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 5787, 2024-09-19T06:29:18.718 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 346523965, 2024-09-19T06:29:18.718 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 5787, 2024-09-19T06:29:18.718 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 346523965, 2024-09-19T06:29:18.718 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-19T06:29:18.718 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.718 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.718 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.718 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:18.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.718 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-09-19T06:29:18.718 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-19T06:29:18.718 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 5787, 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 5787, 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 5787, 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 5787, 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.719 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-19T06:29:18.720 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x55ba9c64e6e0": { 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-19T06:29:18.721 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:18.722 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:18.722 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-19T06:29:18.722 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-19T06:29:18.722 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-19T06:29:18.722 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-19T06:29:18.722 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-19T06:29:18.722 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:18.722 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:18.722 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:18.722 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:18.722 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:18.722 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:29:19.342 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-19T06:29:19.343 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-19T06:29:19.343 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-19T06:29:19.343 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:29:19.343 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:29:19.343 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T06:29:19.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T06:29:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:20.121 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-19T06:29:20.121 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:29:20.121 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 463, 2024-09-19T06:29:20.121 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 452, 2024-09-19T06:29:20.121 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1655600, 2024-09-19T06:29:20.121 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 279952, 2024-09-19T06:29:20.121 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-19T06:29:20.121 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-09-19T06:29:20.121 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.065701872, 2024-09-19T06:29:20.121 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.026067359, 2024-09-19T06:29:20.121 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.054270824, 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 451, 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.018216262, 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000040390 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1655600, 2024-09-19T06:29:20.122 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 279952 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 3, 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 347, 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 480, 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 296896, 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.023650227, 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.017981769, 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002339298, 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:29:20.123 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 344, 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.014925973, 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000043389 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 296896 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:29:20.124 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 4, 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 4, 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 2944, 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 448, 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.002374382, 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000034078, 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002255510, 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3, 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.004109485, 2024-09-19T06:29:20.125 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.001369828 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:20.126 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148732, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 31, 2024-09-19T06:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-19T06:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 466, 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1590155, 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 466, 2024-09-19T06:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-19T06:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-19T06:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 466, 2024-09-19T06:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1590155, 2024-09-19T06:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-19T06:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-19T06:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-19T06:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-19T06:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-19T06:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-19T06:29:20.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:20 smithi005 ceph-mon[30955]: pgmap v823: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 515 B/s wr, 12 op/s 2024-09-19T06:29:20.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:20 smithi049 ceph-mon[34640]: pgmap v823: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 515 B/s wr, 12 op/s 2024-09-19T06:29:20.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:20 smithi157 ceph-mon[35346]: pgmap v823: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 515 B/s wr, 12 op/s 2024-09-19T06:29:20.769 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T06:29:20.769 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T06:29:20.769 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-19T06:29:20.769 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-19T06:29:20.769 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-19T06:29:20.769 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T06:29:20.770 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T06:29:21.095 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:29:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-09-19T06:29:21.610 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:29:21.610 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 145185, 2024-09-19T06:29:21.610 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 128184, 2024-09-19T06:29:21.610 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 68436642, 2024-09-19T06:29:21.610 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 179429574, 2024-09-19T06:29:21.610 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 11, 2024-09-19T06:29:21.610 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 4, 2024-09-19T06:29:21.610 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 7.668857837, 2024-09-19T06:29:21.610 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 3.072467396, 2024-09-19T06:29:21.610 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 4.664774660, 2024-09-19T06:29:21.611 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.617623441, 2024-09-19T06:29:21.611 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:21.611 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 128188, 2024-09-19T06:29:21.611 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 7.161964693, 2024-09-19T06:29:21.611 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000055870 2024-09-19T06:29:21.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.611 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:21.611 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 221261, 2024-09-19T06:29:21.611 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.033153172, 2024-09-19T06:29:21.611 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000149 2024-09-19T06:29:21.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.612 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T06:29:21.612 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 654752 2024-09-19T06:29:21.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.612 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:29:21.612 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 136867, 2024-09-19T06:29:21.612 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 146374, 2024-09-19T06:29:21.612 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 38248279, 2024-09-19T06:29:21.612 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 207467151, 2024-09-19T06:29:21.612 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 19, 2024-09-19T06:29:21.612 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 7, 2024-09-19T06:29:21.613 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 8.322912596, 2024-09-19T06:29:21.613 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 3.051848371, 2024-09-19T06:29:21.613 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 4.941971031, 2024-09-19T06:29:21.613 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 1.145188171, 2024-09-19T06:29:21.613 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:21.613 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 146377, 2024-09-19T06:29:21.613 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 9.523168108, 2024-09-19T06:29:21.613 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000065059 2024-09-19T06:29:21.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.613 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:21.613 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 217907, 2024-09-19T06:29:21.613 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.033933240, 2024-09-19T06:29:21.614 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000155 2024-09-19T06:29:21.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.614 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 2825872, 2024-09-19T06:29:21.614 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 616352 2024-09-19T06:29:21.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.614 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:29:21.614 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 1103069, 2024-09-19T06:29:21.614 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1186237, 2024-09-19T06:29:21.614 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 327183408, 2024-09-19T06:29:21.614 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 795262140, 2024-09-19T06:29:21.614 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 478, 2024-09-19T06:29:21.614 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 10, 2024-09-19T06:29:21.615 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 52.753307590, 2024-09-19T06:29:21.615 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 20.883620343, 2024-09-19T06:29:21.615 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 38.044469514, 2024-09-19T06:29:21.615 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.838660615, 2024-09-19T06:29:21.615 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:21.615 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1186225, 2024-09-19T06:29:21.615 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 22.809978473, 2024-09-19T06:29:21.615 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000019229 2024-09-19T06:29:21.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2235458, 2024-09-19T06:29:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.312979148, 2024-09-19T06:29:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000140 2024-09-19T06:29:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 142144, 2024-09-19T06:29:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 160528 2024-09-19T06:29:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-09-19T06:29:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2024-09-19T06:29:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-09-19T06:29:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2024-09-19T06:29:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-09-19T06:29:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-09-19T06:29:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 95262, 2024-09-19T06:29:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 51.532852304, 2024-09-19T06:29:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000540959 2024-09-19T06:29:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2024-09-19T06:29:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-09-19T06:29:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-09-19T06:29:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 20147, 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 1.292591868, 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000064158 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 946942, 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 908280, 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 908280, 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 719.282329369, 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000791916 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2024-09-19T06:29:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 38662, 2024-09-19T06:29:21.619 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 17, 2024-09-19T06:29:21.619 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 4, 2024-09-19T06:29:21.619 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 3535, 2024-09-19T06:29:21.619 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 5, 2024-09-19T06:29:21.619 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2024-09-19T06:29:21.619 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 917, 2024-09-19T06:29:21.619 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 522, 2024-09-19T06:29:21.619 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 362, 2024-09-19T06:29:21.619 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 33, 2024-09-19T06:29:21.619 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 170, 2024-09-19T06:29:21.620 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 3572, 2024-09-19T06:29:21.620 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 3, 2024-09-19T06:29:21.620 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 3, 2024-09-19T06:29:21.620 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 6, 2024-09-19T06:29:21.620 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 1835412, 2024-09-19T06:29:21.620 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 1788361, 2024-09-19T06:29:21.620 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 13894, 2024-09-19T06:29:21.620 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 166, 2024-09-19T06:29:21.620 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:29:21.621 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2024-09-19T06:29:21.621 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 2, 2024-09-19T06:29:21.621 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 115, 2024-09-19T06:29:21.621 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2024-09-19T06:29:21.621 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 60, 2024-09-19T06:29:21.621 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 2999, 2024-09-19T06:29:21.621 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 53, 2024-09-19T06:29:21.621 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 1258, 2024-09-19T06:29:21.621 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2024-09-19T06:29:21.621 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 3162, 2024-09-19T06:29:21.621 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 4, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 234, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 6, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 3162, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 756, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 30, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 726, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 2406, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 14, 2024-09-19T06:29:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 46504, 2024-09-19T06:29:21.623 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 40448, 2024-09-19T06:29:21.623 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:29:21.623 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:29:21.623 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:29:21.623 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 62829, 2024-09-19T06:29:21.623 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 12300, 2024-09-19T06:29:21.623 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 789, 2024-09-19T06:29:21.623 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 34527 2024-09-19T06:29:21.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.623 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2024-09-19T06:29:21.623 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 37, 2024-09-19T06:29:21.623 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 5, 2024-09-19T06:29:21.624 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 3, 2024-09-19T06:29:21.624 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 168, 2024-09-19T06:29:21.624 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 301, 2024-09-19T06:29:21.624 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 23, 2024-09-19T06:29:21.624 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2024-09-19T06:29:21.624 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:29:21.624 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 9416, 2024-09-19T06:29:21.624 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 9393, 2024-09-19T06:29:21.624 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2024-09-19T06:29:21.624 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 30, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 70, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 5, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2024-09-19T06:29:21.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 104739, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 104561, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 104561, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 178, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 1951, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 236, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 235, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 235, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 1, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 1, 2024-09-19T06:29:21.626 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 3, 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 371828589, 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 372656596, 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 0, 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 46557, 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 169.079167270, 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.003631659 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 0 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 711, 2024-09-19T06:29:21.627 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 13016, 2024-09-19T06:29:21.628 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 12305, 2024-09-19T06:29:21.628 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 72, 2024-09-19T06:29:21.628 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 321, 2024-09-19T06:29:21.628 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 249, 2024-09-19T06:29:21.628 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 917, 2024-09-19T06:29:21.628 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 19694, 2024-09-19T06:29:21.628 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 18777, 2024-09-19T06:29:21.628 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 3, 2024-09-19T06:29:21.628 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 10333, 2024-09-19T06:29:21.628 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 10330, 2024-09-19T06:29:21.628 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 81212, 2024-09-19T06:29:21.628 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 231712 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 1041737, 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 23188, 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 946942, 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 703, 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 63184, 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 9978, 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 24.219974533, 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.002427337 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.629 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 818992, 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 270.707659951, 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000330537 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 36271, 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 222.944890526, 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.006146643 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.632 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2024-09-19T06:29:21.632 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.632 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.632 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.632 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2024-09-19T06:29:21.632 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.632 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.632 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.632 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2024-09-19T06:29:21.633 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.633 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.633 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.634 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2024-09-19T06:29:21.634 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 26, 2024-09-19T06:29:21.634 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.005987884, 2024-09-19T06:29:21.634 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000230303 2024-09-19T06:29:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2024-09-19T06:29:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 119, 2024-09-19T06:29:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.253567956, 2024-09-19T06:29:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.002130823 2024-09-19T06:29:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2024-09-19T06:29:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 26, 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 4.974392938, 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.191322805 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 5990, 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 5.162948961, 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000861928 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 25370, 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 58.913799243, 2024-09-19T06:29:21.636 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.002322183 2024-09-19T06:29:21.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.637 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2024-09-19T06:29:21.637 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1877, 2024-09-19T06:29:21.637 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 99.134050294, 2024-09-19T06:29:21.637 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.052815157 2024-09-19T06:29:21.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.637 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2024-09-19T06:29:21.637 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.637 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.637 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.637 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 34, 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.250405803, 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.007364876 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 20, 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 4.853602911, 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.242680145 2024-09-19T06:29:21.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 5, 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.021228479, 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.004245695 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.639 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.640 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.640 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2024-09-19T06:29:21.640 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 172, 2024-09-19T06:29:21.640 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.083335138, 2024-09-19T06:29:21.640 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000484506 2024-09-19T06:29:21.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.640 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2024-09-19T06:29:21.640 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 7, 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.021681239, 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.003097319 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.642 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2024-09-19T06:29:21.642 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 9393, 2024-09-19T06:29:21.642 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 27.734803513, 2024-09-19T06:29:21.642 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.002952709 2024-09-19T06:29:21.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.642 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:29:21.642 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:29:21.642 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2024-09-19T06:29:21.642 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.643 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.643 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.643 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:21.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.643 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2024-09-19T06:29:21.643 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 2, 2024-09-19T06:29:21.643 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 5, 2024-09-19T06:29:21.643 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 3, 2024-09-19T06:29:21.643 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 2, 2024-09-19T06:29:21.643 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2024-09-19T06:29:21.644 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 36333, 2024-09-19T06:29:21.644 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 18166, 2024-09-19T06:29:21.644 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 1539, 2024-09-19T06:29:21.644 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2024-09-19T06:29:21.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.644 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-09-19T06:29:21.644 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 360, 2024-09-19T06:29:21.644 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 360, 2024-09-19T06:29:21.644 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:29:21.644 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:29:21.644 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:29:21.645 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-09-19T06:29:21.646 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 495191, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 153, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:29:21.647 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 8232, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 182, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 2543400, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 12016, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-09-19T06:29:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 71751, 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 385933450, 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 31, 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 71751, 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 71751, 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 436.838298677, 2024-09-19T06:29:21.649 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.006088253 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 71751, 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 87813 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 71751, 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 4636, 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 67115, 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 3364, 2024-09-19T06:29:21.650 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 3253, 2024-09-19T06:29:21.651 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 1423, 2024-09-19T06:29:21.651 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 47950, 2024-09-19T06:29:21.651 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 1205, 2024-09-19T06:29:21.651 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2024-09-19T06:29:21.651 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2024-09-19T06:29:21.651 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2024-09-19T06:29:21.651 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 5, 2024-09-19T06:29:21.651 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 10079, 2024-09-19T06:29:21.651 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2024-09-19T06:29:21.651 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2024-09-19T06:29:21.651 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 3213, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 6506, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2024-09-19T06:29:21.652 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2024-09-19T06:29:21.653 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2024-09-19T06:29:21.654 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2024-09-19T06:29:21.654 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2024-09-19T06:29:21.654 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 125, 2024-09-19T06:29:21.654 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2024-09-19T06:29:21.654 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 44, 2024-09-19T06:29:21.654 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 12, 2024-09-19T06:29:21.654 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 12, 2024-09-19T06:29:21.654 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 0, 2024-09-19T06:29:21.654 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2024-09-19T06:29:21.654 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 8637, 2024-09-19T06:29:21.654 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 42, 2024-09-19T06:29:21.654 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 2136 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 1, 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 0 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2024-09-19T06:29:21.655 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 6, 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed_ops": 9393, 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 9393, 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 1385117, 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 309071242, 2024-09-19T06:29:21.656 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:21.657 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 1385117, 2024-09-19T06:29:21.657 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-09-19T06:29:21.657 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-09-19T06:29:21.657 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 1385117, 2024-09-19T06:29:21.657 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 309071242, 2024-09-19T06:29:21.657 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:21.657 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.657 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.657 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.657 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:21.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 71751, 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 386890079, 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 71751, 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 386890079, 2024-09-19T06:29:21.658 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:21.659 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:21.660 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 71751, 2024-09-19T06:29:21.660 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 71751, 2024-09-19T06:29:21.660 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 71751, 2024-09-19T06:29:21.660 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 71751, 2024-09-19T06:29:21.660 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:21.660 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.660 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.660 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.660 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:21.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.660 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 9393, 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 956917, 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 9393, 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 1393, 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 956917, 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:21.661 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x559db300a500": { 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 104739, 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 368462292, 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 104739, 2024-09-19T06:29:21.662 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-09-19T06:29:21.663 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-09-19T06:29:21.663 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 46557, 2024-09-19T06:29:21.663 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 368462292, 2024-09-19T06:29:21.663 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:21.663 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:21.663 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:21.663 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:21.663 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:21.663 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:21.663 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-09-19T06:29:22.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:22 smithi005 ceph-mon[30955]: pgmap v824: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-19T06:29:22.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:22 smithi049 ceph-mon[34640]: pgmap v824: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-19T06:29:22.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:22 smithi157 ceph-mon[35346]: pgmap v824: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-19T06:29:22.569 INFO:tasks.check_counter:Daemon mds.b mds.exported=60 2024-09-19T06:29:22.570 INFO:tasks.check_counter:Daemon mds.b mds.imported=53 2024-09-19T06:29:22.570 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-19T06:29:22.570 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=26 2024-09-19T06:29:22.570 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=20 2024-09-19T06:29:22.570 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T06:29:22.570 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T06:29:22.829 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:29:23.444 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-09-19T06:29:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:29:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 3089, 2024-09-19T06:29:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 3091, 2024-09-19T06:29:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 143486442, 2024-09-19T06:29:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 982312, 2024-09-19T06:29:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 8, 2024-09-19T06:29:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 4, 2024-09-19T06:29:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.279691882, 2024-09-19T06:29:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.071465176, 2024-09-19T06:29:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.122589101, 2024-09-19T06:29:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.066494658, 2024-09-19T06:29:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3089, 2024-09-19T06:29:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.074278782, 2024-09-19T06:29:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000024046 2024-09-19T06:29:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T06:29:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 1372, 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1650, 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 112958474, 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 1106789, 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 8, 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 6, 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.161629216, 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.033289007, 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.090456933, 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.028024900, 2024-09-19T06:29:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1648, 2024-09-19T06:29:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.039507690, 2024-09-19T06:29:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000023973 2024-09-19T06:29:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T06:29:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 671072 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 1473, 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1436, 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 97250036, 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 559980, 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 4, 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.165908589, 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.037137344, 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.107353491, 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.020792010, 2024-09-19T06:29:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1436, 2024-09-19T06:29:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.043468431, 2024-09-19T06:29:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000030270 2024-09-19T06:29:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 520544, 2024-09-19T06:29:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 234656 2024-09-19T06:29:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 9420, 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.308999340, 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000032802 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2024-09-19T06:29:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-09-19T06:29:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-09-19T06:29:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-09-19T06:29:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 0, 2024-09-19T06:29:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 0, 2024-09-19T06:29:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 0, 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 0, 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 0, 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 0, 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 0, 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 10197, 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 7003, 2024-09-19T06:29:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 3194, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 0, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 193, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 0, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 0, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 0, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 2, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 0, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 0, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 0, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 0, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2024-09-19T06:29:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 0, 2024-09-19T06:29:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 0, 2024-09-19T06:29:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2024-09-19T06:29:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 0, 2024-09-19T06:29:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 0, 2024-09-19T06:29:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 0, 2024-09-19T06:29:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 0, 2024-09-19T06:29:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2024-09-19T06:29:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 0, 2024-09-19T06:29:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2024-09-19T06:29:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 4, 2024-09-19T06:29:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 234, 2024-09-19T06:29:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 6, 2024-09-19T06:29:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T06:29:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2024-09-19T06:29:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:29:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:29:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 0, 2024-09-19T06:29:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:29:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T06:29:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 0, 2024-09-19T06:29:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 0, 2024-09-19T06:29:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T06:29:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 0, 2024-09-19T06:29:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:29:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:29:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:29:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 0, 2024-09-19T06:29:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 0, 2024-09-19T06:29:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 0, 2024-09-19T06:29:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 0 2024-09-19T06:29:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:29:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 0, 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 0, 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2024-09-19T06:29:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 0, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 0, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 0, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 99184, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 0, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 0, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 0, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 0, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 2, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 102, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 0, 2024-09-19T06:29:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 371828589, 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 372656596, 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 372656484, 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 99184 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 827, 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 12473, 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 11646, 2024-09-19T06:29:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 54, 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 201, 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 147, 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 10197, 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 13128, 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 2931, 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 0, 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 0, 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 0, 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 66548, 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 215328 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 0, 2024-09-19T06:29:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 0, 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 0, 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 0, 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 0, 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2024-09-19T06:29:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2024-09-19T06:29:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2024-09-19T06:29:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2024-09-19T06:29:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2024-09-19T06:29:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2024-09-19T06:29:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2024-09-19T06:29:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2024-09-19T06:29:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2024-09-19T06:29:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2024-09-19T06:29:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2024-09-19T06:29:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:29:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:29:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2024-09-19T06:29:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2024-09-19T06:29:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 2, 2024-09-19T06:29:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 2, 2024-09-19T06:29:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 0, 2024-09-19T06:29:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 0, 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 0, 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 0, 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 2573, 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:29:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:29:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-09-19T06:29:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 240217, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 80, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-09-19T06:29:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 8232, 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 182, 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 9105275, 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 45906, 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 5482, 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 26, 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 4, 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 5482, 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 5482, 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 3.798898164, 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000692976 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2024-09-19T06:29:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 5482, 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 5483 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 5482, 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 5482, 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 0, 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 2744, 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 0, 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 2735, 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 0, 2024-09-19T06:29:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 0, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 0, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 0, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 2, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 0, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2024-09-19T06:29:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2024-09-19T06:29:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 125, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2024-09-19T06:29:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 40, 2024-09-19T06:29:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 11, 2024-09-19T06:29:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 11, 2024-09-19T06:29:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 0, 2024-09-19T06:29:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2024-09-19T06:29:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 0, 2024-09-19T06:29:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 2, 2024-09-19T06:29:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 0 2024-09-19T06:29:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2024-09-19T06:29:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 0, 2024-09-19T06:29:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:29:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 0 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 0, 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed_ops": 0, 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 0, 2024-09-19T06:29:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2024-09-19T06:29:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:29:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-09-19T06:29:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 5930, 2024-09-19T06:29:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 353135641, 2024-09-19T06:29:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 5930, 2024-09-19T06:29:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-09-19T06:29:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-09-19T06:29:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 5930, 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 353135641, 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-09-19T06:29:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 5482, 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 351667329, 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 5482, 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 351667329, 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2024-09-19T06:29:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 5482, 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 5482, 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 5482, 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 5482, 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-09-19T06:29:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x556de48706e0": { 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-09-19T06:29:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:23.494 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-09-19T06:29:24.278 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T06:29:24.278 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T06:29:24.279 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-19T06:29:24.279 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:29:24.279 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:29:24.279 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T06:29:24.279 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T06:29:24.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:24 smithi005 ceph-mon[30955]: pgmap v825: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2024-09-19T06:29:24.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:24 smithi049 ceph-mon[34640]: pgmap v825: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2024-09-19T06:29:24.533 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:29:24.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:24 smithi157 ceph-mon[35346]: pgmap v825: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2024-09-19T06:29:25.052 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-09-19T06:29:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:29:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 51171, 2024-09-19T06:29:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 52066, 2024-09-19T06:29:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 17159791, 2024-09-19T06:29:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 129486590, 2024-09-19T06:29:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 41, 2024-09-19T06:29:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 7, 2024-09-19T06:29:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 3.073652772, 2024-09-19T06:29:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 1.186092416, 2024-09-19T06:29:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 1.667643192, 2024-09-19T06:29:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.414214112, 2024-09-19T06:29:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 52068, 2024-09-19T06:29:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 1.130457306, 2024-09-19T06:29:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000021711 2024-09-19T06:29:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 73163, 2024-09-19T06:29:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.012250835, 2024-09-19T06:29:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000167 2024-09-19T06:29:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1980688, 2024-09-19T06:29:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 307856 2024-09-19T06:29:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:29:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 56036, 2024-09-19T06:29:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 34592, 2024-09-19T06:29:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 35485547, 2024-09-19T06:29:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 123392771, 2024-09-19T06:29:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 49, 2024-09-19T06:29:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 7, 2024-09-19T06:29:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 2.932855579, 2024-09-19T06:29:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 1.218362261, 2024-09-19T06:29:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 1.743707350, 2024-09-19T06:29:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.296592222, 2024-09-19T06:29:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 34589, 2024-09-19T06:29:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.802352212, 2024-09-19T06:29:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000023196 2024-09-19T06:29:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 66932, 2024-09-19T06:29:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.009945104, 2024-09-19T06:29:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000148 2024-09-19T06:29:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T06:29:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 773424 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 476523, 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 548552, 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 143378847, 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 470029866, 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 13, 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 7, 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 25.859759595, 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 10.228170075, 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 18.790516419, 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.335200748, 2024-09-19T06:29:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 548548, 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 9.025885287, 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000016454 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 979780, 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.167905567, 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000171 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 4752, 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 17152 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 39786, 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 32.512467410, 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000817183 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2024-09-19T06:29:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 22737, 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 1.613399575, 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000070959 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 410215, 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 410208, 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 410208, 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 247.340117609, 2024-09-19T06:29:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000602962 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 7, 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 6, 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 0, 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 43, 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 4, 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 1933, 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 203, 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 1584, 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 146, 2024-09-19T06:29:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 1639, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 680, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 0, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 0, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 4, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 785676, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 764987, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 6, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 25, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 0, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 0, 2024-09-19T06:29:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 10, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 539, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 10, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 363, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 2299, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 4, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 234, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 6, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 2299, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:29:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 240, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 240, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 2059, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 0, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 34633, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 53118, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 52499, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 13634, 2024-09-19T06:29:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 692, 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 40579 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 10, 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 46, 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 31, 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 47, 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 30, 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 20, 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 11209, 2024-09-19T06:29:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 11189, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 10, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 4, 2024-09-19T06:29:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 75269, 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 0, 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 0, 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 75269, 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 0, 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 111, 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 0, 2024-09-19T06:29:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 0, 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 111, 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 8, 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 0, 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 4194304, 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 350717363, 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 0, 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 19603, 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 81.678126030, 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.004166613 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 0 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 1640, 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 13387, 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 11747, 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 70, 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 110, 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 40, 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 1933, 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 15246, 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 13313, 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 0, 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 11220, 2024-09-19T06:29:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 11220, 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 74424, 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 223520 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 453482, 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 14, 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 410215, 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 1125, 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 6320, 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 11017, 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 19.136030700, 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.001736954 2024-09-19T06:29:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 348918, 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 103.592478715, 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000296896 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3214, 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 1.449607858, 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000451029 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2024-09-19T06:29:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2024-09-19T06:29:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2024-09-19T06:29:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 6787, 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 4.696901549, 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000692043 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 26214, 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 84.053703030, 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.003206443 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2381, 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 5.283695875, 2024-09-19T06:29:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.002219107 2024-09-19T06:29:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2024-09-19T06:29:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2024-09-19T06:29:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 20, 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.092199475, 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.004609973 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 468, 2024-09-19T06:29:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.185333088, 2024-09-19T06:29:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000396010 2024-09-19T06:29:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2024-09-19T06:29:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2024-09-19T06:29:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2024-09-19T06:29:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2024-09-19T06:29:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 11189, 2024-09-19T06:29:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 28.850167319, 2024-09-19T06:29:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.002578440 2024-09-19T06:29:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:29:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:29:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2024-09-19T06:29:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2024-09-19T06:29:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 2, 2024-09-19T06:29:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 3, 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 1, 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 2, 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 49491, 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 24745, 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 1543, 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:29:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-09-19T06:29:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:29:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 451502, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 158, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-09-19T06:29:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 8232, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 182, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 5139299, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 21799, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-09-19T06:29:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 37243, 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 348697956, 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 33, 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 37243, 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 37243, 2024-09-19T06:29:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 135.336381569, 2024-09-19T06:29:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.003633874 2024-09-19T06:29:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2024-09-19T06:29:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2024-09-19T06:29:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 37243, 2024-09-19T06:29:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 38710 2024-09-19T06:29:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 37243, 2024-09-19T06:29:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 3600, 2024-09-19T06:29:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 33643, 2024-09-19T06:29:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2024-09-19T06:29:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 0, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 10, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 1289, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 20892, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 702, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 4, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 11283, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2024-09-19T06:29:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 2299, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 20, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2024-09-19T06:29:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2024-09-19T06:29:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 125, 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 42, 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 12, 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 17, 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 5, 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 1485, 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 18, 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 708 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 1, 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 0 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2024-09-19T06:29:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 6, 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed_ops": 11189, 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 11189, 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 583726, 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 143454493, 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 583726, 2024-09-19T06:29:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 583726, 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 143454493, 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 37243, 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 349838947, 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 37243, 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 349838947, 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 37243, 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 37243, 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 37243, 2024-09-19T06:29:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 37243, 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 11189, 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1141265, 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 11189, 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 1289, 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1141265, 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x5654a0a32500": { 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-09-19T06:29:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 75269, 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 346523059, 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 75269, 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 19603, 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 346523059, 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:25.102 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-09-19T06:29:25.771 INFO:tasks.check_counter:Daemon mds.h mds.exported=10 2024-09-19T06:29:25.771 INFO:tasks.check_counter:Daemon mds.h mds.imported=10 2024-09-19T06:29:25.771 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-19T06:29:25.771 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:29:25.771 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:29:25.772 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T06:29:25.772 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T06:29:26.027 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:29:26.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:26 smithi005 ceph-mon[30955]: pgmap v826: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 89 B/s wr, 10 op/s 2024-09-19T06:29:26.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:26 smithi049 ceph-mon[34640]: pgmap v826: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 89 B/s wr, 10 op/s 2024-09-19T06:29:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:26 smithi157 ceph-mon[35346]: pgmap v826: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 89 B/s wr, 10 op/s 2024-09-19T06:29:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-09-19T06:29:26.567 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:29:26.567 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 1664, 2024-09-19T06:29:26.567 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1666, 2024-09-19T06:29:26.567 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 172392755, 2024-09-19T06:29:26.567 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 529216, 2024-09-19T06:29:26.567 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 9, 2024-09-19T06:29:26.567 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 4, 2024-09-19T06:29:26.567 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.208080010, 2024-09-19T06:29:26.567 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.034206137, 2024-09-19T06:29:26.568 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.126453874, 2024-09-19T06:29:26.568 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.035714903, 2024-09-19T06:29:26.568 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:26.568 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1664, 2024-09-19T06:29:26.568 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.041690884, 2024-09-19T06:29:26.568 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000025054 2024-09-19T06:29:26.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.568 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:26.568 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.568 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.568 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.569 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T06:29:26.569 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T06:29:26.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.569 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:29:26.569 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 2149, 2024-09-19T06:29:26.569 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2097, 2024-09-19T06:29:26.569 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 147101868, 2024-09-19T06:29:26.569 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 796801, 2024-09-19T06:29:26.569 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 6, 2024-09-19T06:29:26.569 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 5, 2024-09-19T06:29:26.569 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.259064027, 2024-09-19T06:29:26.569 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.054998900, 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.177829060, 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.034801003, 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2097, 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.059443380, 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000028346 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.570 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1965056, 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 289904 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 3257, 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 3602, 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 168742354, 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 1806658, 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 8, 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 6, 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.352426807, 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.080810620, 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.164510617, 2024-09-19T06:29:26.571 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.077799511, 2024-09-19T06:29:26.572 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:26.572 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3599, 2024-09-19T06:29:26.572 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.104335112, 2024-09-19T06:29:26.572 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000028990 2024-09-19T06:29:26.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.572 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:26.572 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.572 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.572 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.572 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T06:29:26.572 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 771568 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 10997, 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.230543233, 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000020964 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:26.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 0, 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 0, 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2024-09-19T06:29:26.574 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.575 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.575 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.575 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2024-09-19T06:29:26.575 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 0, 2024-09-19T06:29:26.575 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 0, 2024-09-19T06:29:26.575 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 0, 2024-09-19T06:29:26.575 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 0, 2024-09-19T06:29:26.575 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 0, 2024-09-19T06:29:26.575 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2024-09-19T06:29:26.576 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 18601, 2024-09-19T06:29:26.576 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 3019, 2024-09-19T06:29:26.576 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 15582, 2024-09-19T06:29:26.576 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 0, 2024-09-19T06:29:26.576 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 14288, 2024-09-19T06:29:26.576 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 0, 2024-09-19T06:29:26.576 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 0, 2024-09-19T06:29:26.576 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 0, 2024-09-19T06:29:26.576 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 1, 2024-09-19T06:29:26.576 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 0, 2024-09-19T06:29:26.576 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 0, 2024-09-19T06:29:26.576 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 0, 2024-09-19T06:29:26.577 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 0, 2024-09-19T06:29:26.577 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:29:26.577 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2024-09-19T06:29:26.577 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 0, 2024-09-19T06:29:26.577 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 0, 2024-09-19T06:29:26.577 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2024-09-19T06:29:26.577 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 0, 2024-09-19T06:29:26.577 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 0, 2024-09-19T06:29:26.577 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 0, 2024-09-19T06:29:26.577 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 0, 2024-09-19T06:29:26.577 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2024-09-19T06:29:26.577 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 0, 2024-09-19T06:29:26.578 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 0, 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 0, 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 0, 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 0, 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 0, 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 0 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 0, 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 0, 2024-09-19T06:29:26.579 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 0, 2024-09-19T06:29:26.580 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 0, 2024-09-19T06:29:26.580 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 0, 2024-09-19T06:29:26.580 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 0, 2024-09-19T06:29:26.580 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2024-09-19T06:29:26.580 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:29:26.580 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 0, 2024-09-19T06:29:26.580 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 0, 2024-09-19T06:29:26.580 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2024-09-19T06:29:26.580 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2024-09-19T06:29:26.580 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2024-09-19T06:29:26.580 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:29:26.580 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:29:26.581 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2024-09-19T06:29:26.581 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2024-09-19T06:29:26.581 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:29:26.581 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:29:26.581 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:29:26.581 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 0, 2024-09-19T06:29:26.581 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2024-09-19T06:29:26.581 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 0, 2024-09-19T06:29:26.581 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2024-09-19T06:29:26.581 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2024-09-19T06:29:26.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 0, 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 0, 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 0, 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 131456, 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 0, 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 0, 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 0, 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 0, 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 141, 2024-09-19T06:29:26.582 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 12, 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 0, 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 51065483, 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 488665610, 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 488636054, 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 131456 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.583 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2024-09-19T06:29:26.584 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 1578, 2024-09-19T06:29:26.584 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 19184, 2024-09-19T06:29:26.584 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 17606, 2024-09-19T06:29:26.584 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 59, 2024-09-19T06:29:26.584 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 114, 2024-09-19T06:29:26.584 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 55, 2024-09-19T06:29:26.584 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 18601, 2024-09-19T06:29:26.584 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 19559, 2024-09-19T06:29:26.584 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 958, 2024-09-19T06:29:26.584 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 0, 2024-09-19T06:29:26.584 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 0, 2024-09-19T06:29:26.584 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 0, 2024-09-19T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 74804, 2024-09-19T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 215328 2024-09-19T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2024-09-19T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 0, 2024-09-19T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 0, 2024-09-19T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 0, 2024-09-19T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 0, 2024-09-19T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 0, 2024-09-19T06:29:26.586 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2024-09-19T06:29:26.586 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.586 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.586 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.586 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2024-09-19T06:29:26.586 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.586 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.586 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.586 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2024-09-19T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2024-09-19T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2024-09-19T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2024-09-19T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2024-09-19T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2024-09-19T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2024-09-19T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.590 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2024-09-19T06:29:26.590 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.590 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.590 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.590 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2024-09-19T06:29:26.590 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.590 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.590 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.591 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2024-09-19T06:29:26.591 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.591 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.592 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.592 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2024-09-19T06:29:26.592 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.592 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.592 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.592 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2024-09-19T06:29:26.592 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.593 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.593 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.593 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2024-09-19T06:29:26.593 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.593 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.593 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.593 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2024-09-19T06:29:26.593 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.593 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.594 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.594 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2024-09-19T06:29:26.594 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.594 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.594 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.594 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2024-09-19T06:29:26.594 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.594 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.594 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.595 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2024-09-19T06:29:26.595 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.595 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.595 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.595 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2024-09-19T06:29:26.595 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.595 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.595 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.596 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2024-09-19T06:29:26.596 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.596 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.596 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.596 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2024-09-19T06:29:26.596 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.596 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.596 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.597 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2024-09-19T06:29:26.597 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.597 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.597 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.597 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2024-09-19T06:29:26.597 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.597 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.597 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.597 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2024-09-19T06:29:26.598 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.598 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.598 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.598 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2024-09-19T06:29:26.598 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.598 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2024-09-19T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2024-09-19T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.600 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2024-09-19T06:29:26.600 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.600 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.600 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.600 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:29:26.600 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:29:26.600 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2024-09-19T06:29:26.600 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.600 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.600 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.601 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:26.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.601 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2024-09-19T06:29:26.601 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 1, 2024-09-19T06:29:26.601 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 3, 2024-09-19T06:29:26.601 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 2, 2024-09-19T06:29:26.601 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 2, 2024-09-19T06:29:26.601 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2024-09-19T06:29:26.601 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 0, 2024-09-19T06:29:26.601 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 0, 2024-09-19T06:29:26.601 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 3595, 2024-09-19T06:29:26.602 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2024-09-19T06:29:26.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.602 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-09-19T06:29:26.602 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:29:26.602 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-09-19T06:29:26.602 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:29:26.602 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:29:26.602 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:29:26.602 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:29:26.602 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:29:26.602 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:29:26.603 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:29:26.603 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:29:26.603 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:29:26.603 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:29:26.603 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:29:26.603 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:29:26.603 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:29:26.603 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-09-19T06:29:26.603 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:29:26.603 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-09-19T06:29:26.603 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:29:26.604 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:29:26.604 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:29:26.604 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:29:26.604 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:29:26.604 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:29:26.604 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:29:26.604 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-09-19T06:29:26.604 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:29:26.604 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:29:26.604 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:29:26.604 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-09-19T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-09-19T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-09-19T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 241783, 2024-09-19T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 84, 2024-09-19T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-09-19T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-09-19T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-09-19T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-09-19T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-09-19T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 8232, 2024-09-19T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 182, 2024-09-19T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-09-19T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-09-19T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 16873446, 2024-09-19T06:29:26.608 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 82306, 2024-09-19T06:29:26.608 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-09-19T06:29:26.608 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-09-19T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-09-19T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-09-19T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2024-09-19T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2024-09-19T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2024-09-19T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 6513, 2024-09-19T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 21, 2024-09-19T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 3, 2024-09-19T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 6513, 2024-09-19T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2024-09-19T06:29:26.610 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 6513, 2024-09-19T06:29:26.610 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 4.313884286, 2024-09-19T06:29:26.610 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000662349 2024-09-19T06:29:26.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.610 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2024-09-19T06:29:26.610 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2024-09-19T06:29:26.610 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 6513, 2024-09-19T06:29:26.610 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 6514 2024-09-19T06:29:26.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.610 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 6513, 2024-09-19T06:29:26.610 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 6513, 2024-09-19T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 0, 2024-09-19T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2024-09-19T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2024-09-19T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 3453, 2024-09-19T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 0, 2024-09-19T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 3058, 2024-09-19T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 0, 2024-09-19T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 0, 2024-09-19T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2024-09-19T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2024-09-19T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2024-09-19T06:29:26.612 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 0, 2024-09-19T06:29:26.612 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 0, 2024-09-19T06:29:26.612 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2024-09-19T06:29:26.612 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2024-09-19T06:29:26.612 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2024-09-19T06:29:26.612 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 1, 2024-09-19T06:29:26.612 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 0, 2024-09-19T06:29:26.612 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:29:26.612 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2024-09-19T06:29:26.612 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:29:26.612 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2024-09-19T06:29:26.613 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2024-09-19T06:29:26.613 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2024-09-19T06:29:26.613 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:29:26.613 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2024-09-19T06:29:26.613 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:29:26.613 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2024-09-19T06:29:26.613 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2024-09-19T06:29:26.613 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2024-09-19T06:29:26.613 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2024-09-19T06:29:26.613 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2024-09-19T06:29:26.614 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2024-09-19T06:29:26.614 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2024-09-19T06:29:26.614 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2024-09-19T06:29:26.614 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2024-09-19T06:29:26.614 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2024-09-19T06:29:26.614 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2024-09-19T06:29:26.614 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2024-09-19T06:29:26.614 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2024-09-19T06:29:26.614 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2024-09-19T06:29:26.614 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2024-09-19T06:29:26.614 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2024-09-19T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2024-09-19T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 125, 2024-09-19T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2024-09-19T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 42, 2024-09-19T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 11, 2024-09-19T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 12, 2024-09-19T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 1, 2024-09-19T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2024-09-19T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 0, 2024-09-19T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 2, 2024-09-19T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 0 2024-09-19T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2024-09-19T06:29:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 0, 2024-09-19T06:29:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:29:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2024-09-19T06:29:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2024-09-19T06:29:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2024-09-19T06:29:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 0 2024-09-19T06:29:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2024-09-19T06:29:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2024-09-19T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2024-09-19T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 0, 2024-09-19T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed_ops": 0, 2024-09-19T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 0, 2024-09-19T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2024-09-19T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-09-19T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:26.618 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 7066, 2024-09-19T06:29:26.618 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 487570068, 2024-09-19T06:29:26.618 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:26.618 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 7066, 2024-09-19T06:29:26.618 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-09-19T06:29:26.618 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-09-19T06:29:26.618 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 7066, 2024-09-19T06:29:26.618 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 487570068, 2024-09-19T06:29:26.618 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:26.618 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.618 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.619 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.619 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:26.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.619 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2024-09-19T06:29:26.619 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:26.619 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-09-19T06:29:26.619 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:26.619 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-09-19T06:29:26.619 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-09-19T06:29:26.619 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:26.619 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 6513, 2024-09-19T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 484478925, 2024-09-19T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 6513, 2024-09-19T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 484478925, 2024-09-19T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2024-09-19T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2024-09-19T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-09-19T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-09-19T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 6513, 2024-09-19T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 6513, 2024-09-19T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 6513, 2024-09-19T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 6513, 2024-09-19T06:29:26.622 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:26.622 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.622 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.622 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.622 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:26.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.622 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:29:26.622 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:26.622 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-09-19T06:29:26.622 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-09-19T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-09-19T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-09-19T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-09-19T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-09-19T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-09-19T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-09-19T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x55b52c3fa6e0": { 2024-09-19T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-09-19T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-09-19T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-09-19T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-09-19T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-09-19T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-09-19T06:29:26.625 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-09-19T06:29:26.625 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-09-19T06:29:26.625 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-09-19T06:29:26.625 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-09-19T06:29:26.625 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-09-19T06:29:26.625 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-09-19T06:29:26.625 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-09-19T06:29:26.625 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:26.625 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-09-19T06:29:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-09-19T06:29:27.379 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-19T06:29:27.379 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-19T06:29:27.379 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-19T06:29:27.379 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:29:27.379 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:29:27.379 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T06:29:27.379 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T06:29:27.700 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:29:28.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:28 smithi005 ceph-mon[30955]: pgmap v827: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 15 op/s 2024-09-19T06:29:28.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:28 smithi049 ceph-mon[34640]: pgmap v827: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 15 op/s 2024-09-19T06:29:28.397 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:28 smithi157 ceph-mon[35346]: pgmap v827: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 15 op/s 2024-09-19T06:29:28.752 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-09-19T06:29:28.753 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:29:28.753 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 552997, 2024-09-19T06:29:28.753 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 651912, 2024-09-19T06:29:28.753 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 169197034, 2024-09-19T06:29:28.753 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 632314414, 2024-09-19T06:29:28.753 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 17, 2024-09-19T06:29:28.753 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-09-19T06:29:28.753 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 29.939115324, 2024-09-19T06:29:28.753 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 12.135597182, 2024-09-19T06:29:28.753 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 20.932524978, 2024-09-19T06:29:28.753 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.671787497, 2024-09-19T06:29:28.754 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:28.754 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 651909, 2024-09-19T06:29:28.754 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 11.937662157, 2024-09-19T06:29:28.754 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000018311 2024-09-19T06:29:28.754 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.754 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:28.754 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1131147, 2024-09-19T06:29:28.754 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.184163800, 2024-09-19T06:29:28.754 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000162 2024-09-19T06:29:28.754 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.754 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1996560, 2024-09-19T06:29:28.755 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 329824 2024-09-19T06:29:28.755 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.755 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:29:28.755 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 34203, 2024-09-19T06:29:28.755 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 35864, 2024-09-19T06:29:28.755 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 10812842, 2024-09-19T06:29:28.755 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 162838825, 2024-09-19T06:29:28.755 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 14, 2024-09-19T06:29:28.755 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2024-09-19T06:29:28.755 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 2.532310966, 2024-09-19T06:29:28.755 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 1.073238674, 2024-09-19T06:29:28.755 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.895281789, 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.567958797, 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 35863, 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.009825660, 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000028157 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 26401, 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.004615222, 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000174 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-09-19T06:29:28.756 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 776960 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 189521, 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 155533, 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 65940720, 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 162280198, 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 67, 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 8, 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 8.991101514, 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 3.601953014, 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 5.693462228, 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.806949163, 2024-09-19T06:29:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 155532, 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.688562600, 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000017286 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 293239, 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.038811951, 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000132 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.758 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 80103, 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 42.580064844, 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000531566 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-09-19T06:29:28.759 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 35698, 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.000430408, 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000056037 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 455219, 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 453860, 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 453860, 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 274.726227255, 2024-09-19T06:29:28.760 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000605310 2024-09-19T06:29:28.761 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.761 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-09-19T06:29:28.761 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 1359, 2024-09-19T06:29:28.761 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 1, 2024-09-19T06:29:28.761 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-09-19T06:29:28.761 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 59, 2024-09-19T06:29:28.761 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 6, 2024-09-19T06:29:28.761 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-09-19T06:29:28.761 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 1828, 2024-09-19T06:29:28.761 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 201, 2024-09-19T06:29:28.761 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 1565, 2024-09-19T06:29:28.761 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 62, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 1540, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 1212, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 1, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 1, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 6, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 885616, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 823349, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 1352, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 56, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 8, 2024-09-19T06:29:28.762 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-09-19T06:29:28.763 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-09-19T06:29:28.763 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 25, 2024-09-19T06:29:28.763 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 922, 2024-09-19T06:29:28.763 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 11, 2024-09-19T06:29:28.763 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 354, 2024-09-19T06:29:28.763 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-09-19T06:29:28.763 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 2981, 2024-09-19T06:29:28.763 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-09-19T06:29:28.763 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 4, 2024-09-19T06:29:28.763 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 234, 2024-09-19T06:29:28.763 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 6, 2024-09-19T06:29:28.763 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 2981, 2024-09-19T06:29:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-09-19T06:29:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:29:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:29:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 260, 2024-09-19T06:29:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:29:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 260, 2024-09-19T06:29:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 2721, 2024-09-19T06:29:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 6, 2024-09-19T06:29:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 44388, 2024-09-19T06:29:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 70757, 2024-09-19T06:29:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:29:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 2, 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 71732, 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 20688, 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 1263, 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 60489 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 13, 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 38, 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 25, 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 75, 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 160, 2024-09-19T06:29:28.765 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 20, 2024-09-19T06:29:28.766 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-09-19T06:29:28.766 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:29:28.766 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 16748, 2024-09-19T06:29:28.766 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 16728, 2024-09-19T06:29:28.766 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-09-19T06:29:28.766 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-09-19T06:29:28.766 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-09-19T06:29:28.766 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:29:28.766 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:29:28.766 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-09-19T06:29:28.766 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-09-19T06:29:28.766 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 25, 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 6, 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 131456, 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 48873, 2024-09-19T06:29:28.767 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 36871, 2024-09-19T06:29:28.768 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 94585, 2024-09-19T06:29:28.768 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-09-19T06:29:28.768 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 12002, 2024-09-19T06:29:28.768 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 185, 2024-09-19T06:29:28.768 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 57, 2024-09-19T06:29:28.768 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 44, 2024-09-19T06:29:28.768 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 141, 2024-09-19T06:29:28.768 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 8, 2024-09-19T06:29:28.768 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-09-19T06:29:28.768 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 13, 2024-09-19T06:29:28.768 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 51065483, 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 488665610, 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 38929, 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 105.108856535, 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002700014 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 1537, 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 19269, 2024-09-19T06:29:28.769 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 17732, 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 75, 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 177, 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 102, 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 1828, 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 24230, 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 22402, 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 1, 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 16856, 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 16855, 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 79040, 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 223520 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.770 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 537468, 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 680, 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 455219, 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 1129, 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 19083, 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 16563, 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 25.440030430, 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001535955 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 356516, 2024-09-19T06:29:28.771 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 103.009431232, 2024-09-19T06:29:28.772 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000288933 2024-09-19T06:29:28.772 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.772 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-09-19T06:29:28.772 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.772 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.772 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.772 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.772 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-09-19T06:29:28.772 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.772 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.772 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 10396, 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 5.838110348, 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000561572 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-09-19T06:29:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.774 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.775 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-09-19T06:29:28.775 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.775 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.775 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.775 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.776 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 10428, 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 6.136503176, 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000588464 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 39351, 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 89.229478667, 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002267527 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3486, 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 7.307004844, 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002096100 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.778 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 20, 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.076233718, 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003811685 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.779 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.780 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 372, 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.124986148, 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000335984 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.781 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.782 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.782 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-09-19T06:29:28.782 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.782 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.782 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.782 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.782 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-09-19T06:29:28.782 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.782 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.782 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.782 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.783 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-09-19T06:29:28.783 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 16728, 2024-09-19T06:29:28.783 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 37.564448692, 2024-09-19T06:29:28.783 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002245603 2024-09-19T06:29:28.783 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.783 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:29:28.783 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:29:28.783 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-09-19T06:29:28.783 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.784 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.784 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.784 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:28.784 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.784 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-09-19T06:29:28.784 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2024-09-19T06:29:28.784 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 3, 2024-09-19T06:29:28.784 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 2, 2024-09-19T06:29:28.784 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2024-09-19T06:29:28.784 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-09-19T06:29:28.784 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 49078, 2024-09-19T06:29:28.785 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 49078, 2024-09-19T06:29:28.785 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 1566, 2024-09-19T06:29:28.785 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-09-19T06:29:28.785 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.785 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-09-19T06:29:28.785 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:29:28.785 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-09-19T06:29:28.785 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:29:28.785 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:29:28.785 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:29:28.786 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:29:28.786 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:29:28.786 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:29:28.786 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:29:28.786 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:29:28.786 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:29:28.786 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:29:28.786 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:29:28.786 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:29:28.786 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:29:28.786 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-09-19T06:29:28.786 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:29:28.787 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-09-19T06:29:28.787 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:29:28.787 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:29:28.787 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:29:28.787 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:29:28.787 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:29:28.787 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:29:28.787 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:29:28.787 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-09-19T06:29:28.787 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 466461, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 157, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-09-19T06:29:28.788 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8232, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 182, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 4998963, 2024-09-19T06:29:28.789 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 20875, 2024-09-19T06:29:28.790 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-09-19T06:29:28.790 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-09-19T06:29:28.790 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-09-19T06:29:28.790 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-09-19T06:29:28.790 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.790 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-09-19T06:29:28.790 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-09-19T06:29:28.790 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-09-19T06:29:28.790 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 66271, 2024-09-19T06:29:28.790 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 487709768, 2024-09-19T06:29:28.790 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 33, 2024-09-19T06:29:28.790 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 66271, 2024-09-19T06:29:28.791 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-09-19T06:29:28.791 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 66271, 2024-09-19T06:29:28.791 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 169.998112810, 2024-09-19T06:29:28.791 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002565196 2024-09-19T06:29:28.791 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.791 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-09-19T06:29:28.791 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-09-19T06:29:28.791 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 66271, 2024-09-19T06:29:28.791 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 68948 2024-09-19T06:29:28.791 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.791 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 66271, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 5332, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 60939, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 0, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 21, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 2349, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 41278, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 1308, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-09-19T06:29:28.792 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 6, 2024-09-19T06:29:28.793 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 16866, 2024-09-19T06:29:28.793 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-09-19T06:29:28.793 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-09-19T06:29:28.793 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-09-19T06:29:28.793 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 2981, 2024-09-19T06:29:28.793 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 42, 2024-09-19T06:29:28.793 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:29:28.793 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-09-19T06:29:28.793 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:29:28.793 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-09-19T06:29:28.793 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-09-19T06:29:28.793 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-09-19T06:29:28.794 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 125, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 44, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 12, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 12, 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-09-19T06:29:28.796 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-09-19T06:29:28.796 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 2778, 2024-09-19T06:29:28.796 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 3, 2024-09-19T06:29:28.796 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 1316 2024-09-19T06:29:28.796 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.796 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-09-19T06:29:28.796 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2024-09-19T06:29:28.796 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:29:28.796 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 6, 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 16728, 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 16728, 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-09-19T06:29:28.797 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.798 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:29:28.798 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:28.798 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T06:29:28.798 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:28.798 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 776717, 2024-09-19T06:29:28.798 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 176010729, 2024-09-19T06:29:28.798 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:28.798 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 776717, 2024-09-19T06:29:28.798 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T06:29:28.798 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T06:29:28.798 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 776717, 2024-09-19T06:29:28.798 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 176010729, 2024-09-19T06:29:28.799 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:28.799 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.799 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.799 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.799 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:28.799 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.799 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-09-19T06:29:28.799 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:28.799 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T06:29:28.799 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:28.799 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T06:29:28.800 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T06:29:28.800 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:28.800 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:28.800 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 66271, 2024-09-19T06:29:28.800 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 489410812, 2024-09-19T06:29:28.800 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 66271, 2024-09-19T06:29:28.800 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 489410812, 2024-09-19T06:29:28.800 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:28.800 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.800 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.800 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 66271, 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 66271, 2024-09-19T06:29:28.801 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 66271, 2024-09-19T06:29:28.802 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 66271, 2024-09-19T06:29:28.802 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:28.802 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.802 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.802 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.802 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:28.802 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.802 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:29:28.802 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:28.802 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T06:29:28.802 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 16728, 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1701088, 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 16728, 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2349, 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1701088, 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:28.803 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:28.804 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55616f056500": { 2024-09-19T06:29:28.804 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:28.804 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T06:29:28.804 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:28.804 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 131456, 2024-09-19T06:29:28.804 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 484471306, 2024-09-19T06:29:28.804 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:28.804 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 131456, 2024-09-19T06:29:28.804 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T06:29:28.804 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T06:29:28.804 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 38929, 2024-09-19T06:29:28.805 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 484471306, 2024-09-19T06:29:28.805 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:28.805 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:28.805 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:28.805 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:28.805 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:28.805 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:28.805 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-09-19T06:29:29.658 INFO:tasks.check_counter:Daemon mds.c mds.exported=25 2024-09-19T06:29:29.658 INFO:tasks.check_counter:Daemon mds.c mds.imported=11 2024-09-19T06:29:29.658 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-19T06:29:29.658 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:29:29.658 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:29:29.658 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T06:29:29.659 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T06:29:29.904 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:29:30.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:30 smithi157 ceph-mon[35346]: pgmap v828: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:29:30.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:30 smithi005 ceph-mon[30955]: pgmap v828: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:29:30.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:30 smithi049 ceph-mon[34640]: pgmap v828: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:29:30.424 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-09-19T06:29:30.424 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:29:30.424 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 25531, 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 24198, 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 7649077, 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 134679509, 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 73, 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 8, 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 1.785586303, 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.729259758, 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.795314123, 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.343589182, 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 24196, 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.645510916, 2024-09-19T06:29:30.425 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000026678 2024-09-19T06:29:30.426 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.426 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:30.426 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 25105, 2024-09-19T06:29:30.426 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.004288173, 2024-09-19T06:29:30.426 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000170 2024-09-19T06:29:30.426 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.426 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-19T06:29:30.426 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 976 2024-09-19T06:29:30.426 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.426 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:29:30.426 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 146472, 2024-09-19T06:29:30.426 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 118415, 2024-09-19T06:29:30.427 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 43115681, 2024-09-19T06:29:30.427 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 84257749, 2024-09-19T06:29:30.427 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 13, 2024-09-19T06:29:30.427 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2024-09-19T06:29:30.427 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 6.850654731, 2024-09-19T06:29:30.427 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 2.633154377, 2024-09-19T06:29:30.427 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 4.912730811, 2024-09-19T06:29:30.427 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.343887819, 2024-09-19T06:29:30.427 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:30.427 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 118415, 2024-09-19T06:29:30.427 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.513240189, 2024-09-19T06:29:30.427 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000021224 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 246636, 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.036421204, 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000147 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1988272, 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 957648 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 541423, 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 549915, 2024-09-19T06:29:30.428 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 160632735, 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 421082908, 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 18, 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 26.111435068, 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 10.485513046, 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 18.985301927, 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.296160141, 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 549912, 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 9.798282768, 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000017817 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.429 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1059550, 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.167706592, 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000158 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 7664, 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 157728 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T06:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T06:29:30.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 46927, 2024-09-19T06:29:30.431 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 18.171770534, 2024-09-19T06:29:30.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000387234 2024-09-19T06:29:30.431 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:30.431 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.431 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-09-19T06:29:30.431 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T06:29:30.431 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T06:29:30.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4958, 2024-09-19T06:29:30.431 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.295521785, 2024-09-19T06:29:30.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000059605 2024-09-19T06:29:30.431 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:30.432 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.432 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-09-19T06:29:30.432 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 489711, 2024-09-19T06:29:30.432 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 458164, 2024-09-19T06:29:30.432 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-09-19T06:29:30.432 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 458164, 2024-09-19T06:29:30.432 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 252.727356161, 2024-09-19T06:29:30.432 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000551608 2024-09-19T06:29:30.432 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.432 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-09-19T06:29:30.432 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 31547, 2024-09-19T06:29:30.432 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 11, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 2843, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 950, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 1689, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 204, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 1484, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 1052, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 2, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 2, 2024-09-19T06:29:30.433 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 4, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 942796, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 936079, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 5583, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 252, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 25, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 605, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 67, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 2875, 2024-09-19T06:29:30.434 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 803, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 4, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 234, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 6, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 803, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 360, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 360, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 443, 2024-09-19T06:29:30.435 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 36, 2024-09-19T06:29:30.436 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 19692, 2024-09-19T06:29:30.436 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 6739, 2024-09-19T06:29:30.436 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:29:30.436 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:29:30.436 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:29:30.436 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 22928, 2024-09-19T06:29:30.436 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 2594, 2024-09-19T06:29:30.436 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 336, 2024-09-19T06:29:30.436 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 6378 2024-09-19T06:29:30.436 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.437 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-09-19T06:29:30.437 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 2, 2024-09-19T06:29:30.437 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 201, 2024-09-19T06:29:30.437 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 153, 2024-09-19T06:29:30.437 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 337, 2024-09-19T06:29:30.437 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 81, 2024-09-19T06:29:30.437 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 30, 2024-09-19T06:29:30.437 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-09-19T06:29:30.438 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:29:30.438 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 1774, 2024-09-19T06:29:30.438 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 1744, 2024-09-19T06:29:30.438 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-09-19T06:29:30.438 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-09-19T06:29:30.438 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-09-19T06:29:30.438 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:29:30.438 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:29:30.438 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-09-19T06:29:30.438 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-09-19T06:29:30.438 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:29:30.438 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 25, 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 32369, 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 32369, 2024-09-19T06:29:30.439 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 74, 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 74, 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 4, 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 274323388, 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-09-19T06:29:30.440 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 24153, 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 115.392834075, 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.004777577 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 2414, 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 4831, 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 2417, 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 94, 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 218, 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 124, 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 2843, 2024-09-19T06:29:30.441 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 5691, 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 2848, 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 2, 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 2035, 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 2033, 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 75484, 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 223520 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 504399, 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 28994, 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 489711, 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 257, 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 57083, 2024-09-19T06:29:30.442 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1706, 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.213244757, 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001297329 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 439795, 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 123.033754483, 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000279752 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 10850, 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 107.666439540, 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.009923174 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.444 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.445 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.447 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1028, 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.711441855, 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000692064 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2601, 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 7.858468863, 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003021325 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-09-19T06:29:30.448 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 386, 2024-09-19T06:29:30.449 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 6.799583515, 2024-09-19T06:29:30.449 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.017615501 2024-09-19T06:29:30.449 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.449 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-09-19T06:29:30.449 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.449 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.449 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.449 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.449 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-09-19T06:29:30.449 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.449 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.449 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 30, 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.443698430, 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.014789947 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-09-19T06:29:30.450 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.451 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 24, 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.006477538, 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000269897 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.452 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1744, 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3.994247180, 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002290279 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.453 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:29:30.454 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:29:30.454 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-09-19T06:29:30.454 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.454 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.454 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.454 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:30.454 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.454 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-09-19T06:29:30.454 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-09-19T06:29:30.454 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 5, 2024-09-19T06:29:30.454 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 3, 2024-09-19T06:29:30.454 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 2, 2024-09-19T06:29:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-09-19T06:29:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 53260, 2024-09-19T06:29:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 26630, 2024-09-19T06:29:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 1548, 2024-09-19T06:29:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-09-19T06:29:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-09-19T06:29:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:29:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-09-19T06:29:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:29:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:29:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:29:30.456 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:29:30.456 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:29:30.456 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:29:30.456 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:29:30.456 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:29:30.456 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:29:30.456 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:29:30.456 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:29:30.456 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-09-19T06:29:30.457 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 569495, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 184, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:29:30.458 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8232, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 182, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 7434414, 2024-09-19T06:29:30.459 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 26796, 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 28842, 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 270521460, 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 33, 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 28842, 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-09-19T06:29:30.460 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 28842, 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 129.488839288, 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.004489592 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 28842, 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 29691 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 28842, 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 1349, 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 27493, 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-09-19T06:29:30.461 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-09-19T06:29:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 0, 2024-09-19T06:29:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 10, 2024-09-19T06:29:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 546, 2024-09-19T06:29:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 24699, 2024-09-19T06:29:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 362, 2024-09-19T06:29:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-09-19T06:29:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-09-19T06:29:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-09-19T06:29:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-09-19T06:29:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 1820, 2024-09-19T06:29:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-09-19T06:29:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 803, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 20, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-09-19T06:29:30.463 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-09-19T06:29:30.464 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 125, 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 43, 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 12, 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 16, 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 4, 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 1007, 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 0, 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 424 2024-09-19T06:29:30.465 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T06:29:30.466 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 6, 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 1744, 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 1744, 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 713422, 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 147154315, 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 713422, 2024-09-19T06:29:30.467 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T06:29:30.468 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T06:29:30.468 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 713422, 2024-09-19T06:29:30.468 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 147154315, 2024-09-19T06:29:30.468 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:30.468 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.468 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.468 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.468 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:30.468 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.468 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-09-19T06:29:30.469 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:30.469 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T06:29:30.469 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:30.469 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T06:29:30.469 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T06:29:30.469 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:30.469 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:30.469 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 28842, 2024-09-19T06:29:30.469 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 270699828, 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 28842, 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 270699828, 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T06:29:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 28842, 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 28842, 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 28842, 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 28842, 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:29:30.471 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 1744, 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 178368, 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 1744, 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 546, 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 178368, 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.472 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55bde1f4e500": { 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 32369, 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 270129084, 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 32369, 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T06:29:30.473 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 24153, 2024-09-19T06:29:30.474 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 270129084, 2024-09-19T06:29:30.474 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:30.474 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:30.474 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:30.474 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:30.474 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:30.474 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:30.474 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-09-19T06:29:31.299 INFO:tasks.check_counter:Daemon mds.f mds.exported=25 2024-09-19T06:29:31.300 INFO:tasks.check_counter:Daemon mds.f mds.imported=67 2024-09-19T06:29:31.300 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-19T06:29:31.300 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:29:31.300 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:29:31.300 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T06:29:31.300 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T06:29:31.543 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:29:32.051 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-09-19T06:29:32.051 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:29:32.051 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 142367, 2024-09-19T06:29:32.052 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 194540, 2024-09-19T06:29:32.052 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 40575929, 2024-09-19T06:29:32.052 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 191590541, 2024-09-19T06:29:32.052 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 67, 2024-09-19T06:29:32.052 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 7, 2024-09-19T06:29:32.052 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 9.014323915, 2024-09-19T06:29:32.052 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 3.931815665, 2024-09-19T06:29:32.052 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 5.242866471, 2024-09-19T06:29:32.052 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.608964287, 2024-09-19T06:29:32.052 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:32.052 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 194541, 2024-09-19T06:29:32.052 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3.843209149, 2024-09-19T06:29:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000019755 2024-09-19T06:29:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 307720, 2024-09-19T06:29:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.049822602, 2024-09-19T06:29:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000161 2024-09-19T06:29:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1999168, 2024-09-19T06:29:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 328368 2024-09-19T06:29:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:29:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 528940, 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 606464, 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 157653525, 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 616257771, 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 19, 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 7, 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 27.804182699, 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 11.322400567, 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 19.544947083, 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.494776631, 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 606459, 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 11.340133880, 2024-09-19T06:29:32.054 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000018698 2024-09-19T06:29:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1083365, 2024-09-19T06:29:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.177077061, 2024-09-19T06:29:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000163 2024-09-19T06:29:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-09-19T06:29:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 125056 2024-09-19T06:29:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:29:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 20363, 2024-09-19T06:29:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 20724, 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 6726158, 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 176471312, 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 13, 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 1.584863842, 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.600515287, 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.333755535, 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.544689567, 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 20723, 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.680213684, 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000032824 2024-09-19T06:29:32.056 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.057 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:32.057 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 84, 2024-09-19T06:29:32.057 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000016204, 2024-09-19T06:29:32.057 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000192 2024-09-19T06:29:32.057 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.057 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 6272, 2024-09-19T06:29:32.057 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 670352 2024-09-19T06:29:32.057 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.057 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-09-19T06:29:32.057 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-09-19T06:29:32.057 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-09-19T06:29:32.057 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.058 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-09-19T06:29:32.058 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T06:29:32.058 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T06:29:32.058 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 53224, 2024-09-19T06:29:32.058 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 36.019763916, 2024-09-19T06:29:32.058 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000676757 2024-09-19T06:29:32.058 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:32.058 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.058 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-09-19T06:29:32.058 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T06:29:32.058 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T06:29:32.058 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 26918, 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.384951165, 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000051450 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 445311, 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 445222, 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 445222, 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 285.542273368, 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000641348 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.059 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-09-19T06:29:32.060 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 89, 2024-09-19T06:29:32.060 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 11, 2024-09-19T06:29:32.060 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-09-19T06:29:32.060 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 104, 2024-09-19T06:29:32.060 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 11, 2024-09-19T06:29:32.060 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 5, 2024-09-19T06:29:32.060 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 2591, 2024-09-19T06:29:32.060 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 637, 2024-09-19T06:29:32.060 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 1879, 2024-09-19T06:29:32.060 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 75, 2024-09-19T06:29:32.060 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 1676, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 1228, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 2, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 2, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 6, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 833675, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 792467, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 73, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 80, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 213, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-09-19T06:29:32.061 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 40, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 969, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 19, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 1184, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 3015, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 4, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 234, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 6, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 3015, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:29:32.062 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 274, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 274, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 2741, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 1393, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 31023, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 47714, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 53505, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 16731, 2024-09-19T06:29:32.063 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 1150, 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 42626 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 25, 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 29, 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 20, 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 94, 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 149, 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 20, 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 11809, 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 11789, 2024-09-19T06:29:32.064 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 45, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 16, 2024-09-19T06:29:32.065 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 107408, 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 22503, 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 12291, 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 95117, 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 10212, 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 155, 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 27, 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 15, 2024-09-19T06:29:32.066 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 140, 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 8, 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 12, 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 18147016, 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 464097726, 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 25458, 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 86.003288593, 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003378242 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2024-09-19T06:29:32.067 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 2300, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 15142, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 12842, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 117, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 196, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 79, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 2591, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 20849, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 18258, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 2, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 11854, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 11852, 2024-09-19T06:29:32.068 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 80412, 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 223520 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 502667, 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 950, 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 445311, 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 1365, 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 9204, 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 11583, 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 19.494360552, 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001683014 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.069 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 341483, 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 101.435474816, 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000297043 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.070 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 11521, 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 5.900745758, 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000512173 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.071 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-09-19T06:29:32.072 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8929, 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 5.439298950, 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000609172 2024-09-19T06:29:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 56771, 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 89.373610030, 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001574282 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2449, 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 33.518222835, 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.013686493 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.075 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 20, 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.091878752, 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.004593937 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.076 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 93, 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.446362155, 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.004799593 2024-09-19T06:29:32.077 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 584, 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.169864433, 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000290863 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 11789, 2024-09-19T06:29:32.079 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 29.672455087, 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002516961 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 4, 2024-09-19T06:29:32.080 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 2, 2024-09-19T06:29:32.081 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 2, 2024-09-19T06:29:32.081 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-09-19T06:29:32.081 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 50999, 2024-09-19T06:29:32.081 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 25499, 2024-09-19T06:29:32.081 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 1551, 2024-09-19T06:29:32.081 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-09-19T06:29:32.081 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.081 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-09-19T06:29:32.081 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:29:32.081 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-09-19T06:29:32.081 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:29:32.082 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:29:32.083 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 472465, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 155, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:29:32.084 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8232, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 182, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 7181651, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 28358, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-09-19T06:29:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-09-19T06:29:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-09-19T06:29:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-09-19T06:29:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-09-19T06:29:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 47308, 2024-09-19T06:29:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 462833111, 2024-09-19T06:29:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 32, 2024-09-19T06:29:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 47308, 2024-09-19T06:29:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-09-19T06:29:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 47308, 2024-09-19T06:29:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 143.486583463, 2024-09-19T06:29:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003033030 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 47308, 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 49716 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 47308, 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 5167, 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 42141, 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 0, 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 21, 2024-09-19T06:29:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 2137, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 27595, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 1195, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 9, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 11925, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 3015, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 42, 2024-09-19T06:29:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-09-19T06:29:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-09-19T06:29:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 125, 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 44, 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 12, 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 12, 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 2574, 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 26, 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 1177 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:29:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 6, 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 11789, 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 11789, 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-09-19T06:29:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:29:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T06:29:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 691666, 2024-09-19T06:29:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 142669585, 2024-09-19T06:29:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 691666, 2024-09-19T06:29:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T06:29:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T06:29:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 691666, 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 142669585, 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T06:29:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 47308, 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 464035350, 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 47308, 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 464035350, 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-09-19T06:29:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 47308, 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 47308, 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 47308, 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 47308, 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 11789, 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1202537, 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 11789, 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2137, 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1202537, 2024-09-19T06:29:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55cbd06cc500": { 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 107408, 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 459903422, 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 107408, 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T06:29:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T06:29:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 25458, 2024-09-19T06:29:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 459903422, 2024-09-19T06:29:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:32.099 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-09-19T06:29:32.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:32 smithi157 ceph-mon[35346]: pgmap v829: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:29:32.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:32 smithi005 ceph-mon[30955]: pgmap v829: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:29:32.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:32 smithi049 ceph-mon[34640]: pgmap v829: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:29:32.935 INFO:tasks.check_counter:Daemon mds.i mds.exported=40 2024-09-19T06:29:32.935 INFO:tasks.check_counter:Daemon mds.i mds.imported=19 2024-09-19T06:29:32.935 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-19T06:29:32.935 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:29:32.935 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:29:32.935 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T06:29:32.935 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T06:29:33.181 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:29:33.681 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-09-19T06:29:33.681 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:29:33.681 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 4011, 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 4013, 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 168933069, 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 1275505, 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 8, 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.365255870, 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.085190663, 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.163621600, 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.092578612, 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4011, 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.112230630, 2024-09-19T06:29:33.682 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000027980 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 2063, 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 2013, 2024-09-19T06:29:33.683 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 174320153, 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 771300, 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.257609447, 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.052753937, 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.182666123, 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.032346949, 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2012, 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.055025584, 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000027348 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.684 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:33.685 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.685 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.685 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.685 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.685 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1962928, 2024-09-19T06:29:33.685 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 291376 2024-09-19T06:29:33.685 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.685 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:29:33.685 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 905, 2024-09-19T06:29:33.685 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1251, 2024-09-19T06:29:33.685 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 120397846, 2024-09-19T06:29:33.686 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 1063274, 2024-09-19T06:29:33.686 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 9, 2024-09-19T06:29:33.686 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2024-09-19T06:29:33.686 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.144664580, 2024-09-19T06:29:33.686 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.026465898, 2024-09-19T06:29:33.686 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.091477070, 2024-09-19T06:29:33.686 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.018254199, 2024-09-19T06:29:33.686 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:29:33.686 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1249, 2024-09-19T06:29:33.686 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.051552215, 2024-09-19T06:29:33.686 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000041274 2024-09-19T06:29:33.687 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.687 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:29:33.687 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.687 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.687 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.687 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.687 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-19T06:29:33.687 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 777392 2024-09-19T06:29:33.687 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.687 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-09-19T06:29:33.688 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-09-19T06:29:33.688 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-09-19T06:29:33.688 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.688 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-09-19T06:29:33.688 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T06:29:33.688 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T06:29:33.688 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 10848, 2024-09-19T06:29:33.688 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.221341185, 2024-09-19T06:29:33.688 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000020403 2024-09-19T06:29:33.688 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:33.688 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.688 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-09-19T06:29:33.689 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T06:29:33.689 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T06:29:33.689 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.689 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.689 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.689 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:33.689 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.689 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-09-19T06:29:33.689 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-09-19T06:29:33.689 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-09-19T06:29:33.690 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-09-19T06:29:33.690 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.690 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.690 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.690 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.690 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-09-19T06:29:33.690 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-09-19T06:29:33.690 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-09-19T06:29:33.690 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-09-19T06:29:33.690 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-09-19T06:29:33.690 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-09-19T06:29:33.690 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-09-19T06:29:33.691 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 14442, 2024-09-19T06:29:33.691 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 1078, 2024-09-19T06:29:33.691 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 13364, 2024-09-19T06:29:33.691 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-09-19T06:29:33.691 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 12901, 2024-09-19T06:29:33.691 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-09-19T06:29:33.691 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-09-19T06:29:33.691 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-09-19T06:29:33.691 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 1, 2024-09-19T06:29:33.691 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-09-19T06:29:33.691 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-09-19T06:29:33.692 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-09-19T06:29:33.692 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-09-19T06:29:33.692 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:29:33.692 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-09-19T06:29:33.692 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-09-19T06:29:33.692 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-09-19T06:29:33.692 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-09-19T06:29:33.692 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-09-19T06:29:33.692 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-09-19T06:29:33.692 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-09-19T06:29:33.692 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-09-19T06:29:33.693 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-09-19T06:29:33.693 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-09-19T06:29:33.693 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-09-19T06:29:33.693 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2024-09-19T06:29:33.693 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2024-09-19T06:29:33.693 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-09-19T06:29:33.693 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T06:29:33.693 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-09-19T06:29:33.693 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:29:33.693 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:29:33.693 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-09-19T06:29:33.693 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:29:33.694 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T06:29:33.694 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-09-19T06:29:33.694 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-09-19T06:29:33.694 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T06:29:33.694 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-09-19T06:29:33.694 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:29:33.694 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:29:33.694 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:29:33.694 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-09-19T06:29:33.694 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-09-19T06:29:33.694 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-09-19T06:29:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-09-19T06:29:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-09-19T06:29:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-09-19T06:29:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-09-19T06:29:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-09-19T06:29:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-09-19T06:29:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-09-19T06:29:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-09-19T06:29:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-09-19T06:29:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:29:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-09-19T06:29:33.696 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-09-19T06:29:33.696 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-09-19T06:29:33.696 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-09-19T06:29:33.696 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-09-19T06:29:33.696 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:29:33.696 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:29:33.696 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-09-19T06:29:33.696 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-09-19T06:29:33.696 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:29:33.696 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:29:33.696 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:29:33.696 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 107408, 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-09-19T06:29:33.697 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 140, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 10, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 18147016, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 464097726, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 464097127, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.698 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 107408 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 2358, 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 15069, 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 12711, 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 96, 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 184, 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 88, 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 14442, 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 15471, 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 1029, 2024-09-19T06:29:33.699 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-09-19T06:29:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-09-19T06:29:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-09-19T06:29:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 74020, 2024-09-19T06:29:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 215328 2024-09-19T06:29:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-09-19T06:29:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-09-19T06:29:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-09-19T06:29:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-09-19T06:29:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-09-19T06:29:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-09-19T06:29:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-09-19T06:29:33.701 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.701 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.701 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.701 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.701 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-09-19T06:29:33.701 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.701 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.701 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.701 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.701 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-09-19T06:29:33.701 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.701 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.702 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.702 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.702 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-09-19T06:29:33.702 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.702 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.702 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.702 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.702 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-09-19T06:29:33.702 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.702 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.702 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-09-19T06:29:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-09-19T06:29:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.704 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.705 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.705 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.705 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-09-19T06:29:33.705 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.705 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.705 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.705 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.706 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-09-19T06:29:33.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.706 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.706 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.706 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-09-19T06:29:33.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.706 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-09-19T06:29:33.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.708 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.708 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.708 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-09-19T06:29:33.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.708 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.708 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.708 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.711 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-09-19T06:29:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:29:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:29:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-09-19T06:29:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 4, 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 2, 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 3, 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 3129, 2024-09-19T06:29:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-09-19T06:29:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-09-19T06:29:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:29:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-09-19T06:29:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:29:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:29:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:29:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:29:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:29:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:29:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:29:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:29:33.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:29:33.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 241840, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 86, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8232, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 182, 2024-09-19T06:29:33.718 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-09-19T06:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-09-19T06:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 15609926, 2024-09-19T06:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 67014, 2024-09-19T06:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-09-19T06:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-09-19T06:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-09-19T06:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-09-19T06:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-09-19T06:29:33.720 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-09-19T06:29:33.720 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-09-19T06:29:33.720 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 6419, 2024-09-19T06:29:33.720 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 21, 2024-09-19T06:29:33.720 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 3, 2024-09-19T06:29:33.720 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 6419, 2024-09-19T06:29:33.720 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-09-19T06:29:33.720 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6419, 2024-09-19T06:29:33.720 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 4.432924700, 2024-09-19T06:29:33.720 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000690594 2024-09-19T06:29:33.720 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.720 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-09-19T06:29:33.721 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-09-19T06:29:33.721 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6419, 2024-09-19T06:29:33.721 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 6420 2024-09-19T06:29:33.721 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.721 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 6419, 2024-09-19T06:29:33.721 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 6419, 2024-09-19T06:29:33.721 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-09-19T06:29:33.721 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-09-19T06:29:33.721 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-09-19T06:29:33.721 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 3447, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 2970, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 1, 2024-09-19T06:29:33.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-09-19T06:29:33.723 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:29:33.723 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-09-19T06:29:33.723 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:29:33.723 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-09-19T06:29:33.723 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-09-19T06:29:33.723 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-09-19T06:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 125, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 42, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 11, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 12, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 1, 2024-09-19T06:29:33.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.726 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 6975, 2024-09-19T06:29:33.727 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 462992193, 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 6975, 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 6975, 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 462992193, 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-09-19T06:29:33.728 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 6419, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 459916338, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 6419, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 459916338, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 6419, 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 6419, 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 6419, 2024-09-19T06:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 6419, 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55b3fc6f66e0": { 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T06:29:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T06:29:33.734 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-09-19T06:29:34.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:34 smithi157 ceph-mon[35346]: pgmap v830: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-19T06:29:34.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:34 smithi005 ceph-mon[30955]: pgmap v830: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-19T06:29:34.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:34 smithi049 ceph-mon[34640]: pgmap v830: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-19T06:29:34.525 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-19T06:29:34.525 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-19T06:29:34.525 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-19T06:29:34.525 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:29:34.525 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:29:34.525 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-19T06:29:34.538 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-19T06:29:34.538 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T06:29:34.538 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:29:34.569 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T06:29:34.569 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:29:34.630 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T06:29:34.942 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.383+0000 7f89a9084640 1 -- 172.21.15.5:0/2443009434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a4103740 msgr2=0x7f89a4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.383+0000 7f89a9084640 1 --2- 172.21.15.5:0/2443009434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a4103740 0x7f89a4107b80 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f8998009930 tx=0x7f899802a420 comp rx=0 tx=0).stop 2024-09-19T06:29:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.383+0000 7f89a9084640 1 -- 172.21.15.5:0/2443009434 shutdown_connections 2024-09-19T06:29:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.383+0000 7f89a9084640 1 --2- 172.21.15.5:0/2443009434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a4108290 0x7f89a410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.383+0000 7f89a9084640 1 --2- 172.21.15.5:0/2443009434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a4103740 0x7f89a4107b80 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.383+0000 7f89a9084640 1 --2- 172.21.15.5:0/2443009434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a4102d90 0x7f89a4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.383+0000 7f89a9084640 1 -- 172.21.15.5:0/2443009434 >> 172.21.15.5:0/2443009434 conn(0x7f89a40fc7d0 msgr2=0x7f89a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.383+0000 7f89a9084640 1 -- 172.21.15.5:0/2443009434 shutdown_connections 2024-09-19T06:29:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.383+0000 7f89a9084640 1 -- 172.21.15.5:0/2443009434 wait complete. 2024-09-19T06:29:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.384+0000 7f89a9084640 1 Processor -- start 2024-09-19T06:29:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.384+0000 7f89a9084640 1 -- start start 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.384+0000 7f89a9084640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a4102d90 0x7f89a4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.384+0000 7f89a9084640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a4103740 0x7f89a4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.384+0000 7f89a9084640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a4108290 0x7f89a419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.384+0000 7f89a9084640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4111f00 con 0x7f89a4103740 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.384+0000 7f89a9084640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4111d80 con 0x7f89a4108290 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.384+0000 7f89a9084640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4112080 con 0x7f89a4102d90 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.384+0000 7f89a259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a4103740 0x7f89a4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.384+0000 7f89a2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a4102d90 0x7f89a4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.384+0000 7f89a259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a4103740 0x7f89a4198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40950/0 (socket says 172.21.15.5:40950) 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.384+0000 7f89a259c640 1 -- 172.21.15.5:0/1032197622 learned_addr learned my addr 172.21.15.5:0/1032197622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f89a359e640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a4108290 0x7f89a419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f89a2d9d640 1 -- 172.21.15.5:0/1032197622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a4108290 msgr2=0x7f89a419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f89a2d9d640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a4108290 0x7f89a419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f89a2d9d640 1 -- 172.21.15.5:0/1032197622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a4103740 msgr2=0x7f89a4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f89a2d9d640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a4103740 0x7f89a4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f89a2d9d640 1 -- 172.21.15.5:0/1032197622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a419d500 con 0x7f89a4102d90 2024-09-19T06:29:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f89a259c640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a4103740 0x7f89a4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:29:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f89a2d9d640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a4102d90 0x7f89a4198550 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f899400d6d0 tx=0x7f899400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f898bfff640 1 -- 172.21.15.5:0/1032197622 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994017070 con 0x7f89a4102d90 2024-09-19T06:29:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f89a9084640 1 -- 172.21.15.5:0/1032197622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a419d7f0 con 0x7f89a4102d90 2024-09-19T06:29:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f898bfff640 1 -- 172.21.15.5:0/1032197622 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8994009820 con 0x7f89a4102d90 2024-09-19T06:29:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f898bfff640 1 -- 172.21.15.5:0/1032197622 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994004c40 con 0x7f89a4102d90 2024-09-19T06:29:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.385+0000 7f89a9084640 1 -- 172.21.15.5:0/1032197622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a41a50d0 con 0x7f89a4102d90 2024-09-19T06:29:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.386+0000 7f89a9084640 1 -- 172.21.15.5:0/1032197622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a4103170 con 0x7f89a4102d90 2024-09-19T06:29:35.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.387+0000 7f898bfff640 1 -- 172.21.15.5:0/1032197622 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f89940099d0 con 0x7f89a4102d90 2024-09-19T06:29:35.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.388+0000 7f898bfff640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f898c0a33b0 0x7f898c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:35.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.388+0000 7f89a259c640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f898c0a33b0 0x7f898c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:35.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.388+0000 7f898bfff640 1 -- 172.21.15.5:0/1032197622 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8550+0+0 (secure 0 0 0) 0x7f89940cfc80 con 0x7f89a4102d90 2024-09-19T06:29:35.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.389+0000 7f89a259c640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f898c0a33b0 0x7f898c0a5870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f89a40fefc0 tx=0x7f8998005dd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:35.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.391+0000 7f898bfff640 1 -- 172.21.15.5:0/1032197622 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f899409c5e0 con 0x7f89a4102d90 2024-09-19T06:29:35.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.581+0000 7f89a9084640 1 -- 172.21.15.5:0/1032197622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f89a410a2a0 con 0x7f89a4102d90 2024-09-19T06:29:35.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.582+0000 7f898bfff640 1 -- 172.21.15.5:0/1032197622 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v125) ==== 81+0+923 (secure 0 0 0) 0x7f899409bee0 con 0x7f89a4102d90 2024-09-19T06:29:35.583 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3703679969 2024-09-20T06:03:20.622034+0000 2024-09-19T06:29:35.583 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/709970884 2024-09-20T06:03:20.622034+0000 2024-09-19T06:29:35.583 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1367598148 2024-09-20T05:54:15.334457+0000 2024-09-19T06:29:35.583 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2930274429 2024-09-20T05:54:15.334457+0000 2024-09-19T06:29:35.584 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/330367622 2024-09-20T05:54:15.334457+0000 2024-09-19T06:29:35.584 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2197596976 2024-09-20T06:03:20.622034+0000 2024-09-19T06:29:35.584 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/330367622 2024-09-20T05:54:15.334457+0000 2024-09-19T06:29:35.584 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1335130331 2024-09-20T06:03:20.622034+0000 2024-09-19T06:29:35.584 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/255461399 2024-09-20T05:54:38.703198+0000 2024-09-19T06:29:35.584 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1639646508 2024-09-20T06:03:20.622034+0000 2024-09-19T06:29:35.585 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1340147787 2024-09-20T05:54:38.703198+0000 2024-09-19T06:29:35.585 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/850678758 2024-09-20T05:54:15.334457+0000 2024-09-19T06:29:35.585 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/255461399 2024-09-20T05:54:38.703198+0000 2024-09-19T06:29:35.585 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2593868621 2024-09-20T05:54:38.703198+0000 2024-09-19T06:29:35.585 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4102275559 2024-09-20T05:54:38.703198+0000 2024-09-19T06:29:35.585 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/709970884 2024-09-20T06:03:20.622034+0000 2024-09-19T06:29:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:listed 16 entries 2024-09-19T06:29:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.584+0000 7f89a9084640 1 -- 172.21.15.5:0/1032197622 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f898c0a33b0 msgr2=0x7f898c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.584+0000 7f89a9084640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f898c0a33b0 0x7f898c0a5870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f89a40fefc0 tx=0x7f8998005dd0 comp rx=0 tx=0).stop 2024-09-19T06:29:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.584+0000 7f89a9084640 1 -- 172.21.15.5:0/1032197622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a4102d90 msgr2=0x7f89a4198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.584+0000 7f89a9084640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a4102d90 0x7f89a4198550 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f899400d6d0 tx=0x7f899400db90 comp rx=0 tx=0).stop 2024-09-19T06:29:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.584+0000 7f89a9084640 1 -- 172.21.15.5:0/1032197622 shutdown_connections 2024-09-19T06:29:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.584+0000 7f89a9084640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a4108290 0x7f89a419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.584+0000 7f89a9084640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a4103740 0x7f89a4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.584+0000 7f89a9084640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f898c0a33b0 0x7f898c0a5870 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:35.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.584+0000 7f89a9084640 1 --2- 172.21.15.5:0/1032197622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a4102d90 0x7f89a4198550 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:35.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.584+0000 7f89a9084640 1 -- 172.21.15.5:0/1032197622 >> 172.21.15.5:0/1032197622 conn(0x7f89a40fc7d0 msgr2=0x7f89a4105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:35.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.584+0000 7f89a9084640 1 -- 172.21.15.5:0/1032197622 shutdown_connections 2024-09-19T06:29:35.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:35.584+0000 7f89a9084640 1 -- 172.21.15.5:0/1032197622 wait complete. 2024-09-19T06:29:36.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:36 smithi005 ceph-mon[30955]: pgmap v831: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T06:29:36.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:36 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1032197622' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:29:36.337 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-19T06:29:36.337 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:29:36.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T06:29:36.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:36 smithi049 ceph-mon[34640]: pgmap v831: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T06:29:36.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:36 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1032197622' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:29:36.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:36 smithi157 ceph-mon[35346]: pgmap v831: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T06:29:36.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:36 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1032197622' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:29:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.220+0000 7f405fffe640 1 -- 172.21.15.5:0/3327807830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058108290 msgr2=0x7f405810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.220+0000 7f405fffe640 1 --2- 172.21.15.5:0/3327807830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058108290 0x7f405810ec30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f4050009cd0 tx=0x7f405002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:29:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.220+0000 7f405fffe640 1 -- 172.21.15.5:0/3327807830 shutdown_connections 2024-09-19T06:29:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.220+0000 7f405fffe640 1 --2- 172.21.15.5:0/3327807830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058108290 0x7f405810ec30 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.220+0000 7f405fffe640 1 --2- 172.21.15.5:0/3327807830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4058103740 0x7f4058107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.220+0000 7f405fffe640 1 --2- 172.21.15.5:0/3327807830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058102d90 0x7f4058103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.220+0000 7f405fffe640 1 -- 172.21.15.5:0/3327807830 >> 172.21.15.5:0/3327807830 conn(0x7f40580fc7d0 msgr2=0x7f40580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.220+0000 7f405fffe640 1 -- 172.21.15.5:0/3327807830 shutdown_connections 2024-09-19T06:29:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.220+0000 7f405fffe640 1 -- 172.21.15.5:0/3327807830 wait complete. 2024-09-19T06:29:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405fffe640 1 Processor -- start 2024-09-19T06:29:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405fffe640 1 -- start start 2024-09-19T06:29:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405fffe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058102d90 0x7f40581984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405fffe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4058103740 0x7f4058198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405fffe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058108290 0x7f405819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405fffe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4058111f40 con 0x7f4058108290 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405fffe640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4058111dc0 con 0x7f4058103740 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405fffe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f40581120c0 con 0x7f4058102d90 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405e59b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058108290 0x7f405819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405e59b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058108290 0x7f405819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40962/0 (socket says 172.21.15.5:40962) 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405e59b640 1 -- 172.21.15.5:0/2794020338 learned_addr learned my addr 172.21.15.5:0/2794020338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405dd9a640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058102d90 0x7f40581984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405d599640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4058103740 0x7f4058198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405e59b640 1 -- 172.21.15.5:0/2794020338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058102d90 msgr2=0x7f40581984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405e59b640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058102d90 0x7f40581984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405e59b640 1 -- 172.21.15.5:0/2794020338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4058103740 msgr2=0x7f4058198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405e59b640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4058103740 0x7f4058198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.221+0000 7f405e59b640 1 -- 172.21.15.5:0/2794020338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405819d490 con 0x7f4058108290 2024-09-19T06:29:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.222+0000 7f405d599640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4058103740 0x7f4058198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:29:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.222+0000 7f405e59b640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058108290 0x7f405819cdb0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f4050005cc0 tx=0x7f4050003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.222+0000 7f404effd640 1 -- 172.21.15.5:0/2794020338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050004450 con 0x7f4058108290 2024-09-19T06:29:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.222+0000 7f405fffe640 1 -- 172.21.15.5:0/2794020338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405819d720 con 0x7f4058108290 2024-09-19T06:29:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.222+0000 7f404effd640 1 -- 172.21.15.5:0/2794020338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4050031c90 con 0x7f4058108290 2024-09-19T06:29:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.222+0000 7f404effd640 1 -- 172.21.15.5:0/2794020338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405003b400 con 0x7f4058108290 2024-09-19T06:29:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.222+0000 7f405fffe640 1 -- 172.21.15.5:0/2794020338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40581a5060 con 0x7f4058108290 2024-09-19T06:29:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.223+0000 7f405fffe640 1 -- 172.21.15.5:0/2794020338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4058103170 con 0x7f4058108290 2024-09-19T06:29:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.224+0000 7f404effd640 1 -- 172.21.15.5:0/2794020338 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4050004680 con 0x7f4058108290 2024-09-19T06:29:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.225+0000 7f404effd640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f40440a33b0 0x7f40440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.225+0000 7f405dd9a640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f40440a33b0 0x7f40440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:37.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.225+0000 7f404effd640 1 -- 172.21.15.5:0/2794020338 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8550+0+0 (secure 0 0 0) 0x7f405002e030 con 0x7f4058108290 2024-09-19T06:29:37.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.225+0000 7f405dd9a640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f40440a33b0 0x7f40440a5870 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f405400bbe0 tx=0x7f405400b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.229+0000 7f404effd640 1 -- 172.21.15.5:0/2794020338 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f40500bd910 con 0x7f4058108290 2024-09-19T06:29:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.417+0000 7f405fffe640 1 -- 172.21.15.5:0/2794020338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f405810a2a0 con 0x7f4058108290 2024-09-19T06:29:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.418+0000 7f404effd640 1 -- 172.21.15.5:0/2794020338 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v125) ==== 81+0+923 (secure 0 0 0) 0x7f40500098b0 con 0x7f4058108290 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3703679969 2024-09-20T06:03:20.622034+0000 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/709970884 2024-09-20T06:03:20.622034+0000 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1367598148 2024-09-20T05:54:15.334457+0000 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2930274429 2024-09-20T05:54:15.334457+0000 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/330367622 2024-09-20T05:54:15.334457+0000 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2197596976 2024-09-20T06:03:20.622034+0000 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/330367622 2024-09-20T05:54:15.334457+0000 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1335130331 2024-09-20T06:03:20.622034+0000 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/255461399 2024-09-20T05:54:38.703198+0000 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1639646508 2024-09-20T06:03:20.622034+0000 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1340147787 2024-09-20T05:54:38.703198+0000 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/850678758 2024-09-20T05:54:15.334457+0000 2024-09-19T06:29:37.420 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/255461399 2024-09-20T05:54:38.703198+0000 2024-09-19T06:29:37.421 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2593868621 2024-09-20T05:54:38.703198+0000 2024-09-19T06:29:37.421 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4102275559 2024-09-20T05:54:38.703198+0000 2024-09-19T06:29:37.421 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/709970884 2024-09-20T06:03:20.622034+0000 2024-09-19T06:29:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:listed 16 entries 2024-09-19T06:29:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.420+0000 7f405fffe640 1 -- 172.21.15.5:0/2794020338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f40440a33b0 msgr2=0x7f40440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.420+0000 7f405fffe640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f40440a33b0 0x7f40440a5870 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f405400bbe0 tx=0x7f405400b390 comp rx=0 tx=0).stop 2024-09-19T06:29:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.420+0000 7f405fffe640 1 -- 172.21.15.5:0/2794020338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058108290 msgr2=0x7f405819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.420+0000 7f405fffe640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058108290 0x7f405819cdb0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f4050005cc0 tx=0x7f4050003c20 comp rx=0 tx=0).stop 2024-09-19T06:29:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.420+0000 7f405fffe640 1 -- 172.21.15.5:0/2794020338 shutdown_connections 2024-09-19T06:29:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.420+0000 7f405fffe640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058108290 0x7f405819cdb0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.420+0000 7f405fffe640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4058103740 0x7f4058198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.420+0000 7f405fffe640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f40440a33b0 0x7f40440a5870 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:37.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.420+0000 7f405fffe640 1 --2- 172.21.15.5:0/2794020338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058102d90 0x7f40581984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.420+0000 7f405fffe640 1 -- 172.21.15.5:0/2794020338 >> 172.21.15.5:0/2794020338 conn(0x7f40580fc7d0 msgr2=0x7f4058105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.421+0000 7f405fffe640 1 -- 172.21.15.5:0/2794020338 shutdown_connections 2024-09-19T06:29:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:37.421+0000 7f405fffe640 1 -- 172.21.15.5:0/2794020338 wait complete. 2024-09-19T06:29:38.122 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2024-09-19T06:29:38.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:29:38.123 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T06:29:38.161 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T06:29:38.161 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[80522]: fuse finished with error 0 and tester_r 0 2024-09-19T06:29:38.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:38 smithi005 ceph-mon[30955]: pgmap v832: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-19T06:29:38.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:38 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2794020338' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:29:38.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:38 smithi049 ceph-mon[34640]: pgmap v832: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-19T06:29:38.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:38 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2794020338' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:29:38.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:38 smithi157 ceph-mon[35346]: pgmap v832: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-19T06:29:38.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:38 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2794020338' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:29:40.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:40 smithi049 ceph-mon[34640]: pgmap v833: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T06:29:40.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:40 smithi157 ceph-mon[35346]: pgmap v833: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T06:29:40.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:40 smithi005 ceph-mon[30955]: pgmap v833: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T06:29:42.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:42 smithi049 ceph-mon[34640]: pgmap v834: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T06:29:42.431 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:42 smithi005 ceph-mon[30955]: pgmap v834: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T06:29:42.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:42 smithi157 ceph-mon[35346]: pgmap v834: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T06:29:44.263 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-09-19T06:29:44.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:29:44.264 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T06:29:44.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:29:44.292 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-19T06:29:44.378 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T06:29:44.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:29:44.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:29:44.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:44 smithi049 ceph-mon[34640]: pgmap v835: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-09-19T06:29:44.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:44 smithi005 ceph-mon[30955]: pgmap v835: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-09-19T06:29:44.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:29:44.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-09-19T06:29:44.508 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T06:29:44.520 INFO:tasks.cephadm:Teardown begin 2024-09-19T06:29:44.520 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T06:29:44.521 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:29:44.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:44 smithi157 ceph-mon[35346]: pgmap v835: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-09-19T06:29:44.783 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.297+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/683167363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6810c170 msgr2=0x7f3a6810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.297+0000 7f3a6ebbc640 1 --2- 172.21.15.5:0/683167363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6810c170 0x7f3a6810e610 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009ca0 tx=0x7f3a6002a950 comp rx=0 tx=0).stop 2024-09-19T06:29:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.297+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/683167363 shutdown_connections 2024-09-19T06:29:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.297+0000 7f3a6ebbc640 1 --2- 172.21.15.5:0/683167363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6810c170 0x7f3a6810e610 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.297+0000 7f3a6ebbc640 1 --2- 172.21.15.5:0/683167363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a68103660 0x7f3a6810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.297+0000 7f3a6ebbc640 1 --2- 172.21.15.5:0/683167363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a68102d40 0x7f3a68103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.297+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/683167363 >> 172.21.15.5:0/683167363 conn(0x7f3a680fc7a0 msgr2=0x7f3a680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.297+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/683167363 shutdown_connections 2024-09-19T06:29:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.298+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/683167363 wait complete. 2024-09-19T06:29:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.298+0000 7f3a6ebbc640 1 Processor -- start 2024-09-19T06:29:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.298+0000 7f3a6ebbc640 1 -- start start 2024-09-19T06:29:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.298+0000 7f3a6ebbc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a68102d40 0x7f3a6819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.298+0000 7f3a6ebbc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a68103660 0x7f3a6819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.298+0000 7f3a6ebbc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6810c170 0x7f3a6819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.298+0000 7f3a6ebbc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a68111e20 con 0x7f3a68102d40 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.298+0000 7f3a6ebbc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3a68111ca0 con 0x7f3a68103660 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.298+0000 7f3a6ebbc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3a68111fa0 con 0x7f3a6810c170 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.298+0000 7f3a6c958640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a68102d40 0x7f3a6819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.298+0000 7f3a6c958640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a68102d40 0x7f3a6819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56400/0 (socket says 172.21.15.5:56400) 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.298+0000 7f3a6c958640 1 -- 172.21.15.5:0/1454149014 learned_addr learned my addr 172.21.15.5:0/1454149014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a6d159640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6810c170 0x7f3a6819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a6c958640 1 -- 172.21.15.5:0/1454149014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6810c170 msgr2=0x7f3a6819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a6c958640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6810c170 0x7f3a6819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a6c958640 1 -- 172.21.15.5:0/1454149014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a68103660 msgr2=0x7f3a6819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a67fff640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a68103660 0x7f3a6819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a6c958640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a68103660 0x7f3a6819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a6c958640 1 -- 172.21.15.5:0/1454149014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6819f550 con 0x7f3a68102d40 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a6d159640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6810c170 0x7f3a6819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a67fff640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a68103660 0x7f3a6819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a6c958640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a68102d40 0x7f3a6819a5a0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00d770 tx=0x7f3a5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a65ffb640 1 -- 172.21.15.5:0/1454149014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c017070 con 0x7f3a68102d40 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/1454149014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6819f840 con 0x7f3a68102d40 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a65ffb640 1 -- 172.21.15.5:0/1454149014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a5c0054d0 con 0x7f3a68102d40 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a65ffb640 1 -- 172.21.15.5:0/1454149014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c004cf0 con 0x7f3a68102d40 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.299+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/1454149014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a681a7120 con 0x7f3a68102d40 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.300+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/1454149014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a68103120 con 0x7f3a68102d40 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.301+0000 7f3a65ffb640 1 -- 172.21.15.5:0/1454149014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3a5c004e90 con 0x7f3a68102d40 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.302+0000 7f3a65ffb640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3a580a3430 0x7f3a580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.302+0000 7f3a65ffb640 1 -- 172.21.15.5:0/1454149014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8550+0+0 (secure 0 0 0) 0x7f3a5c0cf7e0 con 0x7f3a68102d40 2024-09-19T06:29:45.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.302+0000 7f3a67fff640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3a580a3430 0x7f3a580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:45.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.303+0000 7f3a67fff640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3a580a3430 0x7f3a580a58f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3a54005fd0 tx=0x7f3a54005dc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:45.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.306+0000 7f3a65ffb640 1 -- 172.21.15.5:0/1454149014 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a5c09c1a0 con 0x7f3a68102d40 2024-09-19T06:29:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.518+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/1454149014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a681a7350 con 0x7f3a68102d40 2024-09-19T06:29:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.519+0000 7f3a65ffb640 1 -- 172.21.15.5:0/1454149014 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11970 (secure 0 0 0) 0x7f3a5c09baa0 con 0x7f3a68102d40 2024-09-19T06:29:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:29:45.521 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":66,"btime":"2024-09-19T06:28:43:618012+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:28:43.170395+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:29:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 66 2024-09-19T06:29:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.522+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/1454149014 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3a580a3430 msgr2=0x7f3a580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.522+0000 7f3a6ebbc640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3a580a3430 0x7f3a580a58f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3a54005fd0 tx=0x7f3a54005dc0 comp rx=0 tx=0).stop 2024-09-19T06:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.522+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/1454149014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a68102d40 msgr2=0x7f3a6819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.522+0000 7f3a6ebbc640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a68102d40 0x7f3a6819a5a0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00d770 tx=0x7f3a5c00dc30 comp rx=0 tx=0).stop 2024-09-19T06:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.522+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/1454149014 shutdown_connections 2024-09-19T06:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.522+0000 7f3a6ebbc640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f3a580a3430 0x7f3a580a58f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.522+0000 7f3a6ebbc640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6810c170 0x7f3a6819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.522+0000 7f3a6ebbc640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a68103660 0x7f3a6819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.522+0000 7f3a6ebbc640 1 --2- 172.21.15.5:0/1454149014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a68102d40 0x7f3a6819a5a0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.522+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/1454149014 >> 172.21.15.5:0/1454149014 conn(0x7f3a680fc7a0 msgr2=0x7f3a681008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.522+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/1454149014 shutdown_connections 2024-09-19T06:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:45.522+0000 7f3a6ebbc640 1 -- 172.21.15.5:0/1454149014 wait complete. 2024-09-19T06:29:46.126 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:29:46.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:46 smithi005 ceph-mon[30955]: pgmap v836: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T06:29:46.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:46 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1454149014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:29:46.387 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:46.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:46 smithi049 ceph-mon[34640]: pgmap v836: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T06:29:46.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:46 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1454149014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:29:46.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:46 smithi157 ceph-mon[35346]: pgmap v836: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T06:29:46.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:46 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1454149014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:29:46.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.899+0000 7f127129b640 1 -- 172.21.15.5:0/3052093363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126c100a80 msgr2=0x7f126c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:46.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.899+0000 7f127129b640 1 --2- 172.21.15.5:0/3052093363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126c100a80 0x7f126c100ee0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f1254009930 tx=0x7f125402a420 comp rx=0 tx=0).stop 2024-09-19T06:29:46.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.899+0000 7f127129b640 1 -- 172.21.15.5:0/3052093363 shutdown_connections 2024-09-19T06:29:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.899+0000 7f127129b640 1 --2- 172.21.15.5:0/3052093363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c101420 0x7f126c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.899+0000 7f127129b640 1 --2- 172.21.15.5:0/3052093363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126c100a80 0x7f126c100ee0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.899+0000 7f127129b640 1 --2- 172.21.15.5:0/3052093363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126c106a80 0x7f126c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.899+0000 7f127129b640 1 -- 172.21.15.5:0/3052093363 >> 172.21.15.5:0/3052093363 conn(0x7f126c0fc7a0 msgr2=0x7f126c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.899+0000 7f127129b640 1 -- 172.21.15.5:0/3052093363 shutdown_connections 2024-09-19T06:29:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.899+0000 7f127129b640 1 -- 172.21.15.5:0/3052093363 wait complete. 2024-09-19T06:29:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f127129b640 1 Processor -- start 2024-09-19T06:29:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f127129b640 1 -- start start 2024-09-19T06:29:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f127129b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126c100a80 0x7f126c102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f127129b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126c101420 0x7f126c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f127129b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c106a80 0x7f126c1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f127129b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f126c110d50 con 0x7f126c100a80 2024-09-19T06:29:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f127129b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f126c110bd0 con 0x7f126c101420 2024-09-19T06:29:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f127129b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f126c110ed0 con 0x7f126c106a80 2024-09-19T06:29:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f126affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126c100a80 0x7f126c102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f126affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126c100a80 0x7f126c102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56416/0 (socket says 172.21.15.5:56416) 2024-09-19T06:29:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f126affd640 1 -- 172.21.15.5:0/1352270997 learned_addr learned my addr 172.21.15.5:0/1352270997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f126b7fe640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c106a80 0x7f126c1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.900+0000 7f126a7fc640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126c101420 0x7f126c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.901+0000 7f126affd640 1 -- 172.21.15.5:0/1352270997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c106a80 msgr2=0x7f126c1030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.901+0000 7f126affd640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c106a80 0x7f126c1030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.901+0000 7f126affd640 1 -- 172.21.15.5:0/1352270997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126c101420 msgr2=0x7f126c102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.901+0000 7f126affd640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126c101420 0x7f126c102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.901+0000 7f126affd640 1 -- 172.21.15.5:0/1352270997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c1a68c0 con 0x7f126c100a80 2024-09-19T06:29:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.901+0000 7f126b7fe640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c106a80 0x7f126c1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.901+0000 7f126affd640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126c100a80 0x7f126c102650 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f126400eab0 tx=0x7f126400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:46.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.901+0000 7f1263fff640 1 -- 172.21.15.5:0/1352270997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126400ce40 con 0x7f126c100a80 2024-09-19T06:29:46.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.901+0000 7f1263fff640 1 -- 172.21.15.5:0/1352270997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1264011920 con 0x7f126c100a80 2024-09-19T06:29:46.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.901+0000 7f127129b640 1 -- 172.21.15.5:0/1352270997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c1a6bb0 con 0x7f126c100a80 2024-09-19T06:29:46.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.901+0000 7f1263fff640 1 -- 172.21.15.5:0/1352270997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264010a00 con 0x7f126c100a80 2024-09-19T06:29:46.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.901+0000 7f127129b640 1 -- 172.21.15.5:0/1352270997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c1a70c0 con 0x7f126c100a80 2024-09-19T06:29:46.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.902+0000 7f127129b640 1 -- 172.21.15.5:0/1352270997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126c100ee0 con 0x7f126c100a80 2024-09-19T06:29:46.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.903+0000 7f1263fff640 1 -- 172.21.15.5:0/1352270997 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1264011ad0 con 0x7f126c100a80 2024-09-19T06:29:46.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.904+0000 7f1263fff640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f12580a33b0 0x7f12580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:46.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.904+0000 7f1263fff640 1 -- 172.21.15.5:0/1352270997 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8550+0+0 (secure 0 0 0) 0x7f12640d0430 con 0x7f126c100a80 2024-09-19T06:29:46.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.904+0000 7f126a7fc640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f12580a33b0 0x7f12580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:46.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.905+0000 7f126a7fc640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f12580a33b0 0x7f12580a5870 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1254000c00 tx=0x7f1254032c80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:46.907+0000 7f1263fff640 1 -- 172.21.15.5:0/1352270997 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1264016030 con 0x7f126c100a80 2024-09-19T06:29:47.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.121+0000 7f127129b640 1 -- 172.21.15.5:0/1352270997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f126c103b80 con 0x7f126c100a80 2024-09-19T06:29:47.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.122+0000 7f1263fff640 1 -- 172.21.15.5:0/1352270997 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11970 (secure 0 0 0) 0x7f126c103b80 con 0x7f126c100a80 2024-09-19T06:29:47.123 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:29:47.123 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":66,"btime":"2024-09-19T06:28:43:618012+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/959534083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":959534083},{"type":"v1","addr":"172.21.15.5:6835","nonce":959534083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3191633882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3191633882},{"type":"v1","addr":"172.21.15.5:6837","nonce":3191633882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:01:29.531007+0000","modified":"2024-09-19T06:28:43.170395+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24448,"mds_3":24442,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/1310081153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":1310081153},{"type":"v1","addr":"172.21.15.5:6839","nonce":1310081153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/569764382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":569764382},{"type":"v1","addr":"172.21.15.5:6841","nonce":569764382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6833/2427602354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2427602354},{"type":"v1","addr":"172.21.15.49:6833","nonce":2427602354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/1048665474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1048665474},{"type":"v1","addr":"172.21.15.157:6833","nonce":1048665474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/2858664643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2858664643},{"type":"v1","addr":"172.21.15.49:6837","nonce":2858664643}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.157:6835/2070926870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2070926870},{"type":"v1","addr":"172.21.15.157:6835","nonce":2070926870}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/3264499329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3264499329},{"type":"v1","addr":"172.21.15.157:6837","nonce":3264499329}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/4294596962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4294596962},{"type":"v1","addr":"172.21.15.157:6839","nonce":4294596962}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6839/2359039211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2359039211},{"type":"v1","addr":"172.21.15.49:6839","nonce":2359039211}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25709":{"gid":25709,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6835/1729172575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1729172575},{"type":"v1","addr":"172.21.15.49:6835","nonce":1729172575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24437,24463,24442,24448,24473]},"id":1}]} 2024-09-19T06:29:47.124 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 66 2024-09-19T06:29:47.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.124+0000 7f127129b640 1 -- 172.21.15.5:0/1352270997 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f12580a33b0 msgr2=0x7f12580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:47.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.124+0000 7f127129b640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f12580a33b0 0x7f12580a5870 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1254000c00 tx=0x7f1254032c80 comp rx=0 tx=0).stop 2024-09-19T06:29:47.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.124+0000 7f127129b640 1 -- 172.21.15.5:0/1352270997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126c100a80 msgr2=0x7f126c102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:47.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.124+0000 7f127129b640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126c100a80 0x7f126c102650 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f126400eab0 tx=0x7f126400ef70 comp rx=0 tx=0).stop 2024-09-19T06:29:47.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.124+0000 7f127129b640 1 -- 172.21.15.5:0/1352270997 shutdown_connections 2024-09-19T06:29:47.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.124+0000 7f127129b640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f12580a33b0 0x7f12580a5870 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:47.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.124+0000 7f127129b640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c106a80 0x7f126c1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:47.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.124+0000 7f127129b640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126c101420 0x7f126c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:47.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.124+0000 7f127129b640 1 --2- 172.21.15.5:0/1352270997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126c100a80 0x7f126c102650 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:47.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.124+0000 7f127129b640 1 -- 172.21.15.5:0/1352270997 >> 172.21.15.5:0/1352270997 conn(0x7f126c0fc7a0 msgr2=0x7f126c104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:47.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.124+0000 7f127129b640 1 -- 172.21.15.5:0/1352270997 shutdown_connections 2024-09-19T06:29:47.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:47.124+0000 7f127129b640 1 -- 172.21.15.5:0/1352270997 wait complete. 2024-09-19T06:29:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:47 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1352270997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:29:47.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:47 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1352270997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:29:47.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:47 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1352270997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:29:47.730 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:29:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:48.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:48 smithi005 ceph-mon[30955]: pgmap v837: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-09-19T06:29:48.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:48 smithi049 ceph-mon[34640]: pgmap v837: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-09-19T06:29:48.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:48 smithi157 ceph-mon[35346]: pgmap v837: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-09-19T06:29:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.578+0000 7ff91909d640 1 -- 172.21.15.5:0/1780584448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff914102570 msgr2=0x7ff9141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.578+0000 7ff91909d640 1 --2- 172.21.15.5:0/1780584448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff914102570 0x7ff9141029b0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009900 tx=0x7ff8fc02a480 comp rx=0 tx=0).stop 2024-09-19T06:29:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.579+0000 7ff91909d640 1 -- 172.21.15.5:0/1780584448 shutdown_connections 2024-09-19T06:29:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.579+0000 7ff91909d640 1 --2- 172.21.15.5:0/1780584448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff914102ef0 0x7ff91410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.579+0000 7ff91909d640 1 --2- 172.21.15.5:0/1780584448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff914102570 0x7ff9141029b0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.579+0000 7ff91909d640 1 --2- 172.21.15.5:0/1780584448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914100e20 0x7ff914101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.579+0000 7ff91909d640 1 -- 172.21.15.5:0/1780584448 >> 172.21.15.5:0/1780584448 conn(0x7ff9140fc7d0 msgr2=0x7ff9140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.579+0000 7ff91909d640 1 -- 172.21.15.5:0/1780584448 shutdown_connections 2024-09-19T06:29:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.579+0000 7ff91909d640 1 -- 172.21.15.5:0/1780584448 wait complete. 2024-09-19T06:29:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.579+0000 7ff91909d640 1 Processor -- start 2024-09-19T06:29:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91909d640 1 -- start start 2024-09-19T06:29:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91909d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914100e20 0x7ff9141972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91909d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff914102570 0x7ff914197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91909d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff914102ef0 0x7ff91419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91909d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff914110e80 con 0x7ff914102ef0 2024-09-19T06:29:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91909d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff914110d00 con 0x7ff914102570 2024-09-19T06:29:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91909d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff914111000 con 0x7ff914100e20 2024-09-19T06:29:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff914102ef0 0x7ff91419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff914102570 0x7ff914197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff914102ef0 0x7ff91419bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56422/0 (socket says 172.21.15.5:56422) 2024-09-19T06:29:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91359e640 1 -- 172.21.15.5:0/2608863732 learned_addr learned my addr 172.21.15.5:0/2608863732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff912d9d640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914100e20 0x7ff9141972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91359e640 1 -- 172.21.15.5:0/2608863732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914100e20 msgr2=0x7ff9141972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91359e640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914100e20 0x7ff9141972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91359e640 1 -- 172.21.15.5:0/2608863732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff914102570 msgr2=0x7ff914197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91359e640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff914102570 0x7ff914197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.580+0000 7ff91359e640 1 -- 172.21.15.5:0/2608863732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91419c270 con 0x7ff914102ef0 2024-09-19T06:29:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.581+0000 7ff91359e640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff914102ef0 0x7ff91419bb90 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c970 tx=0x7ff90c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.581+0000 7ff90bfff640 1 -- 172.21.15.5:0/2608863732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c007bb0 con 0x7ff914102ef0 2024-09-19T06:29:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.581+0000 7ff90bfff640 1 -- 172.21.15.5:0/2608863732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff90c007d50 con 0x7ff914102ef0 2024-09-19T06:29:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.581+0000 7ff91909d640 1 -- 172.21.15.5:0/2608863732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91419c560 con 0x7ff914102ef0 2024-09-19T06:29:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.581+0000 7ff90bfff640 1 -- 172.21.15.5:0/2608863732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c018700 con 0x7ff914102ef0 2024-09-19T06:29:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.581+0000 7ff91909d640 1 -- 172.21.15.5:0/2608863732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9141a3e40 con 0x7ff914102ef0 2024-09-19T06:29:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.582+0000 7ff91909d640 1 -- 172.21.15.5:0/2608863732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff914101200 con 0x7ff914102ef0 2024-09-19T06:29:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.583+0000 7ff90bfff640 1 -- 172.21.15.5:0/2608863732 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff90c00fa20 con 0x7ff914102ef0 2024-09-19T06:29:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.584+0000 7ff90bfff640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9000a33b0 0x7ff9000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.584+0000 7ff90bfff640 1 -- 172.21.15.5:0/2608863732 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8550+0+0 (secure 0 0 0) 0x7ff90c0cfd50 con 0x7ff914102ef0 2024-09-19T06:29:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.584+0000 7ff912d9d640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9000a33b0 0x7ff9000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.585+0000 7ff912d9d640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9000a33b0 0x7ff9000a5870 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff904005c40 tx=0x7ff90400a570 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.588+0000 7ff90bfff640 1 -- 172.21.15.5:0/2608863732 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff90c0d3030 con 0x7ff914102ef0 2024-09-19T06:29:48.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.764+0000 7ff91909d640 1 -- 172.21.15.5:0/2608863732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff914109090 con 0x7ff914102ef0 2024-09-19T06:29:48.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.766+0000 7ff90bfff640 1 -- 172.21.15.5:0/2608863732 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+23166 (secure 0 0 0) 0x7ff90c014030 con 0x7ff914102ef0 2024-09-19T06:29:48.768 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-19T06:29:48.768 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":125,"fsid":"6b5d55e8-764b-11ef-bceb-c7b262605968","created":"2024-09-19T05:53:51.454068+0000","modified":"2024-09-19T06:29:13.575034+0000","last_up_change":"2024-09-19T06:00:38.837916+0000","last_in_change":"2024-09-19T06:00:15.609776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:56:55.213931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:01:25.648588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:01:27.760408+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":124,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f3c10d60-d354-407f-9cdf-099344599844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6803","nonce":3442981810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6805","nonce":3442981810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6809","nonce":3442981810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3442981810},{"type":"v1","addr":"172.21.15.5:6807","nonce":3442981810}]},"public_addr":"172.21.15.5:6803/3442981810","cluster_addr":"172.21.15.5:6805/3442981810","heartbeat_back_addr":"172.21.15.5:6809/3442981810","heartbeat_front_addr":"172.21.15.5:6807/3442981810","state":["exists","up"]},{"osd":1,"uuid":"5f40f636-7761-45ac-87f3-910a2fa2feb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448265014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6803","nonce":1448265014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6807","nonce":1448265014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1448265014},{"type":"v1","addr":"172.21.15.49:6805","nonce":1448265014}]},"public_addr":"172.21.15.49:6801/1448265014","cluster_addr":"172.21.15.49:6803/1448265014","heartbeat_back_addr":"172.21.15.49:6807/1448265014","heartbeat_front_addr":"172.21.15.49:6805/1448265014","state":["exists","up"]},{"osd":2,"uuid":"ad00940c-4fdd-4f5f-9eaa-b1f38bcac526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6801","nonce":2319626622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6803","nonce":2319626622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6807","nonce":2319626622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2319626622},{"type":"v1","addr":"172.21.15.157:6805","nonce":2319626622}]},"public_addr":"172.21.15.157:6801/2319626622","cluster_addr":"172.21.15.157:6803/2319626622","heartbeat_back_addr":"172.21.15.157:6807/2319626622","heartbeat_front_addr":"172.21.15.157:6805/2319626622","state":["exists","up"]},{"osd":3,"uuid":"63fec329-1b62-4c5e-b8ca-084a84d4d762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6811","nonce":1452491753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6813","nonce":1452491753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6817","nonce":1452491753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1452491753},{"type":"v1","addr":"172.21.15.5:6815","nonce":1452491753}]},"public_addr":"172.21.15.5:6811/1452491753","cluster_addr":"172.21.15.5:6813/1452491753","heartbeat_back_addr":"172.21.15.5:6817/1452491753","heartbeat_front_addr":"172.21.15.5:6815/1452491753","state":["exists","up"]},{"osd":4,"uuid":"ce5fe1ad-7143-4415-a8a9-0ce55e8a3f26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6809","nonce":3114952580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6811","nonce":3114952580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6815","nonce":3114952580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3114952580},{"type":"v1","addr":"172.21.15.49:6813","nonce":3114952580}]},"public_addr":"172.21.15.49:6809/3114952580","cluster_addr":"172.21.15.49:6811/3114952580","heartbeat_back_addr":"172.21.15.49:6815/3114952580","heartbeat_front_addr":"172.21.15.49:6813/3114952580","state":["exists","up"]},{"osd":5,"uuid":"21a7a7d3-ce42-4688-bfec-714be635eaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6809","nonce":3219063971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6811","nonce":3219063971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6815","nonce":3219063971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3219063971},{"type":"v1","addr":"172.21.15.157:6813","nonce":3219063971}]},"public_addr":"172.21.15.157:6809/3219063971","cluster_addr":"172.21.15.157:6811/3219063971","heartbeat_back_addr":"172.21.15.157:6815/3219063971","heartbeat_front_addr":"172.21.15.157:6813/3219063971","state":["exists","up"]},{"osd":6,"uuid":"cbef0e08-d6b0-4515-a2ee-2c81b048583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6819","nonce":2957795194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6821","nonce":2957795194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6825","nonce":2957795194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2957795194},{"type":"v1","addr":"172.21.15.5:6823","nonce":2957795194}]},"public_addr":"172.21.15.5:6819/2957795194","cluster_addr":"172.21.15.5:6821/2957795194","heartbeat_back_addr":"172.21.15.5:6825/2957795194","heartbeat_front_addr":"172.21.15.5:6823/2957795194","state":["exists","up"]},{"osd":7,"uuid":"d261ac55-532d-4aea-9b3e-ed2b4eeb1da6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6817","nonce":2959731400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6819","nonce":2959731400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6823","nonce":2959731400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2959731400},{"type":"v1","addr":"172.21.15.49:6821","nonce":2959731400}]},"public_addr":"172.21.15.49:6817/2959731400","cluster_addr":"172.21.15.49:6819/2959731400","heartbeat_back_addr":"172.21.15.49:6823/2959731400","heartbeat_front_addr":"172.21.15.49:6821/2959731400","state":["exists","up"]},{"osd":8,"uuid":"15d0eaf7-d604-4546-b393-4d6d39018f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6817","nonce":290026859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6819","nonce":290026859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6823","nonce":290026859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":290026859},{"type":"v1","addr":"172.21.15.157:6821","nonce":290026859}]},"public_addr":"172.21.15.157:6817/290026859","cluster_addr":"172.21.15.157:6819/290026859","heartbeat_back_addr":"172.21.15.157:6823/290026859","heartbeat_front_addr":"172.21.15.157:6821/290026859","state":["exists","up"]},{"osd":9,"uuid":"30c08537-f593-4299-bfa1-fa7ba8a95169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6827","nonce":4122448237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6829","nonce":4122448237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6833","nonce":4122448237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4122448237},{"type":"v1","addr":"172.21.15.5:6831","nonce":4122448237}]},"public_addr":"172.21.15.5:6827/4122448237","cluster_addr":"172.21.15.5:6829/4122448237","heartbeat_back_addr":"172.21.15.5:6833/4122448237","heartbeat_front_addr":"172.21.15.5:6831/4122448237","state":["exists","up"]},{"osd":10,"uuid":"68821e98-bd81-49ec-8cd6-51fb9880b553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6825","nonce":2551748002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6827","nonce":2551748002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6831","nonce":2551748002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2551748002},{"type":"v1","addr":"172.21.15.49:6829","nonce":2551748002}]},"public_addr":"172.21.15.49:6825/2551748002","cluster_addr":"172.21.15.49:6827/2551748002","heartbeat_back_addr":"172.21.15.49:6831/2551748002","heartbeat_front_addr":"172.21.15.49:6829/2551748002","state":["exists","up"]},{"osd":11,"uuid":"7e443b46-2697-4d0b-8452-868463a1ee1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6825","nonce":1996582544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6827","nonce":1996582544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6831","nonce":1996582544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1996582544},{"type":"v1","addr":"172.21.15.157:6829","nonce":1996582544}]},"public_addr":"172.21.15.157:6825/1996582544","cluster_addr":"172.21.15.157:6827/1996582544","heartbeat_back_addr":"172.21.15.157:6831/1996582544","heartbeat_front_addr":"172.21.15.157:6829/1996582544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:01.453592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:27.194705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:56:51.525385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:17.676749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:57:43.075555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:07.258154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:32.112583+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:58:57.306534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:21.320950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:59:47.164610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:12.548337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:00:37.262580+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3703679969":"2024-09-20T06:03:20.622034+0000","172.21.15.5:6801/709970884":"2024-09-20T06:03:20.622034+0000","172.21.15.5:0/1367598148":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2930274429":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6800/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/2197596976":"2024-09-20T06:03:20.622034+0000","172.21.15.5:6801/330367622":"2024-09-20T05:54:15.334457+0000","172.21.15.5:0/1335130331":"2024-09-20T06:03:20.622034+0000","172.21.15.5:6800/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/1639646508":"2024-09-20T06:03:20.622034+0000","172.21.15.5:0/1340147787":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/850678758":"2024-09-20T05:54:15.334457+0000","172.21.15.5:6801/255461399":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/2593868621":"2024-09-20T05:54:38.703198+0000","172.21.15.5:0/4102275559":"2024-09-20T05:54:38.703198+0000","172.21.15.5:6800/709970884":"2024-09-20T06:03:20.622034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:29:48.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.769+0000 7ff91909d640 1 -- 172.21.15.5:0/2608863732 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9000a33b0 msgr2=0x7ff9000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:48.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.769+0000 7ff91909d640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9000a33b0 0x7ff9000a5870 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff904005c40 tx=0x7ff90400a570 comp rx=0 tx=0).stop 2024-09-19T06:29:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.769+0000 7ff91909d640 1 -- 172.21.15.5:0/2608863732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff914102ef0 msgr2=0x7ff91419bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.769+0000 7ff91909d640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff914102ef0 0x7ff91419bb90 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c970 tx=0x7ff90c00ce30 comp rx=0 tx=0).stop 2024-09-19T06:29:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.769+0000 7ff91909d640 1 -- 172.21.15.5:0/2608863732 shutdown_connections 2024-09-19T06:29:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.769+0000 7ff91909d640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff914102ef0 0x7ff91419bb90 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.769+0000 7ff91909d640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff914102570 0x7ff914197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.769+0000 7ff91909d640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7ff9000a33b0 0x7ff9000a5870 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.769+0000 7ff91909d640 1 --2- 172.21.15.5:0/2608863732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914100e20 0x7ff9141972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.769+0000 7ff91909d640 1 -- 172.21.15.5:0/2608863732 >> 172.21.15.5:0/2608863732 conn(0x7ff9140fc7d0 msgr2=0x7ff9140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.769+0000 7ff91909d640 1 -- 172.21.15.5:0/2608863732 shutdown_connections 2024-09-19T06:29:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:48.769+0000 7ff91909d640 1 -- 172.21.15.5:0/2608863732 wait complete. 2024-09-19T06:29:49.351 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T06:29:49.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:49 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/2608863732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:29:49.610 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:49.628 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:49 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/2608863732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:29:49.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:49 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/2608863732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:29:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.080+0000 7fd1c90dd640 1 -- 172.21.15.5:0/891701836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c4103660 msgr2=0x7fd1c410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.080+0000 7fd1c90dd640 1 --2- 172.21.15.5:0/891701836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c4103660 0x7fd1c410bb50 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009900 tx=0x7fd1b802a480 comp rx=0 tx=0).stop 2024-09-19T06:29:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.080+0000 7fd1c90dd640 1 -- 172.21.15.5:0/891701836 shutdown_connections 2024-09-19T06:29:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.080+0000 7fd1c90dd640 1 --2- 172.21.15.5:0/891701836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c410c170 0x7fd1c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.080+0000 7fd1c90dd640 1 --2- 172.21.15.5:0/891701836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c4103660 0x7fd1c410bb50 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.080+0000 7fd1c90dd640 1 --2- 172.21.15.5:0/891701836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1c4102d40 0x7fd1c4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.080+0000 7fd1c90dd640 1 -- 172.21.15.5:0/891701836 >> 172.21.15.5:0/891701836 conn(0x7fd1c40fc7a0 msgr2=0x7fd1c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.080+0000 7fd1c90dd640 1 -- 172.21.15.5:0/891701836 shutdown_connections 2024-09-19T06:29:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.080+0000 7fd1c90dd640 1 -- 172.21.15.5:0/891701836 wait complete. 2024-09-19T06:29:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.080+0000 7fd1c90dd640 1 Processor -- start 2024-09-19T06:29:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c90dd640 1 -- start start 2024-09-19T06:29:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c90dd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1c4102d40 0x7fd1c41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c90dd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c4103660 0x7fd1c41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c90dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c410c170 0x7fd1c419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c90dd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4111e80 con 0x7fd1c410c170 2024-09-19T06:29:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c90dd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4111d00 con 0x7fd1c4103660 2024-09-19T06:29:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c90dd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4112000 con 0x7fd1c4102d40 2024-09-19T06:29:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c410c170 0x7fd1c419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c410c170 0x7fd1c419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56444/0 (socket says 172.21.15.5:56444) 2024-09-19T06:29:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c359e640 1 -- 172.21.15.5:0/1493670899 learned_addr learned my addr 172.21.15.5:0/1493670899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c2d9d640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1c4102d40 0x7fd1c41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c259c640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c4103660 0x7fd1c41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c359e640 1 -- 172.21.15.5:0/1493670899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1c4102d40 msgr2=0x7fd1c41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c359e640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1c4102d40 0x7fd1c41984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c359e640 1 -- 172.21.15.5:0/1493670899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c4103660 msgr2=0x7fd1c41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.081+0000 7fd1c359e640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c4103660 0x7fd1c41989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.082+0000 7fd1c359e640 1 -- 172.21.15.5:0/1493670899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c419d450 con 0x7fd1c410c170 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.082+0000 7fd1c359e640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c410c170 0x7fd1c419cd70 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00eaa0 tx=0x7fd1bc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.082+0000 7fd1abfff640 1 -- 172.21.15.5:0/1493670899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc004d20 con 0x7fd1c410c170 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.082+0000 7fd1abfff640 1 -- 172.21.15.5:0/1493670899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1bc005020 con 0x7fd1c410c170 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.082+0000 7fd1c90dd640 1 -- 172.21.15.5:0/1493670899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c419d740 con 0x7fd1c410c170 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.082+0000 7fd1abfff640 1 -- 172.21.15.5:0/1493670899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0116b0 con 0x7fd1c410c170 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.082+0000 7fd1c90dd640 1 -- 172.21.15.5:0/1493670899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c41a5020 con 0x7fd1c410c170 2024-09-19T06:29:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.083+0000 7fd1c90dd640 1 -- 172.21.15.5:0/1493670899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c4103120 con 0x7fd1c410c170 2024-09-19T06:29:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.084+0000 7fd1abfff640 1 -- 172.21.15.5:0/1493670899 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd1bc0051d0 con 0x7fd1c410c170 2024-09-19T06:29:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.085+0000 7fd1abfff640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1ac0a33b0 0x7fd1ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.085+0000 7fd1c2d9d640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1ac0a33b0 0x7fd1ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.085+0000 7fd1abfff640 1 -- 172.21.15.5:0/1493670899 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8550+0+0 (secure 0 0 0) 0x7fd1bc0d02e0 con 0x7fd1c410c170 2024-09-19T06:29:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.085+0000 7fd1c2d9d640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1ac0a33b0 0x7fd1ac0a5870 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009990 tx=0x7fd1b4009110 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.088+0000 7fd1abfff640 1 -- 172.21.15.5:0/1493670899 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd1bc016030 con 0x7fd1c410c170 2024-09-19T06:29:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.300+0000 7fd1c90dd640 1 -- 172.21.15.5:0/1493670899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd1c4107210 con 0x7fd1c410c170 2024-09-19T06:29:50.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:50 smithi005 ceph-mon[30955]: pgmap v838: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T06:29:50.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.302+0000 7fd1abfff640 1 -- 172.21.15.5:0/1493670899 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) ==== 161+0+0 (secure 0 0 0) 0x7fd1bc09cca0 con 0x7fd1c410c170 2024-09-19T06:29:50.305 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T06:29:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.306+0000 7fd1c90dd640 1 -- 172.21.15.5:0/1493670899 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1ac0a33b0 msgr2=0x7fd1ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.306+0000 7fd1c90dd640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1ac0a33b0 0x7fd1ac0a5870 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009990 tx=0x7fd1b4009110 comp rx=0 tx=0).stop 2024-09-19T06:29:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.306+0000 7fd1c90dd640 1 -- 172.21.15.5:0/1493670899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c410c170 msgr2=0x7fd1c419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.306+0000 7fd1c90dd640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c410c170 0x7fd1c419cd70 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00eaa0 tx=0x7fd1bc00eea0 comp rx=0 tx=0).stop 2024-09-19T06:29:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.306+0000 7fd1c90dd640 1 -- 172.21.15.5:0/1493670899 shutdown_connections 2024-09-19T06:29:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.306+0000 7fd1c90dd640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c410c170 0x7fd1c419cd70 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.306+0000 7fd1c90dd640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c4103660 0x7fd1c41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.306+0000 7fd1c90dd640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7fd1ac0a33b0 0x7fd1ac0a5870 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.306+0000 7fd1c90dd640 1 --2- 172.21.15.5:0/1493670899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1c4102d40 0x7fd1c41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.306+0000 7fd1c90dd640 1 -- 172.21.15.5:0/1493670899 >> 172.21.15.5:0/1493670899 conn(0x7fd1c40fc7a0 msgr2=0x7fd1c40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.307+0000 7fd1c90dd640 1 -- 172.21.15.5:0/1493670899 shutdown_connections 2024-09-19T06:29:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:50.307+0000 7fd1c90dd640 1 -- 172.21.15.5:0/1493670899 wait complete. 2024-09-19T06:29:50.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-mon[35346]: pgmap v838: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T06:29:50.551 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-i[65874]: ignoring --setuser ceph since I am not root 2024-09-19T06:29:50.551 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-i[65874]: ignoring --setgroup ceph since I am not root 2024-09-19T06:29:50.551 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-i[65874]: starting mds.i at 2024-09-19T06:29:50.552 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-l[64955]: ignoring --setuser ceph since I am not root 2024-09-19T06:29:50.552 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-l[64955]: ignoring --setgroup ceph since I am not root 2024-09-19T06:29:50.552 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-mds[64959]: -- 172.21.15.157:0/2312178950 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56418e2e4680 con 0x56418e2b9400 2024-09-19T06:29:50.552 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-mds[64959]: -- 172.21.15.157:0/2312178950 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56418e2dbc20 con 0x56418e2b9400 2024-09-19T06:29:50.552 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-l[64955]: starting mds.l at 2024-09-19T06:29:50.552 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-c[65576]: ignoring --setuser ceph since I am not root 2024-09-19T06:29:50.552 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-c[65576]: ignoring --setgroup ceph since I am not root 2024-09-19T06:29:50.553 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-c[65576]: starting mds.c at 2024-09-19T06:29:50.553 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f[65272]: ignoring --setuser ceph since I am not root 2024-09-19T06:29:50.553 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f[65272]: ignoring --setgroup ceph since I am not root 2024-09-19T06:29:50.553 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-mds[65276]: -- 172.21.15.157:0/1751317802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d4dbf51860 con 0x55d4dbf2e800 2024-09-19T06:29:50.553 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:29:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f[65272]: starting mds.f at 2024-09-19T06:29:50.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:50 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a[30951]: 2024-09-19T06:29:50.301+0000 7f860580f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T06:29:50.629 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:29:50 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-a[69721]: ignoring --setuser ceph since I am not root 2024-09-19T06:29:50.629 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:29:50 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-a[69721]: ignoring --setgroup ceph since I am not root 2024-09-19T06:29:50.629 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:29:50 smithi005 ceph-mds[69725]: -- 172.21.15.5:0/3979312035 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5595cd10b860 con 0x5595cd0e9400 2024-09-19T06:29:50.629 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:29:50 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-a[69721]: starting mds.a at 2024-09-19T06:29:50.630 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:29:50 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-g[69340]: ignoring --setuser ceph since I am not root 2024-09-19T06:29:50.630 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:29:50 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-g[69340]: ignoring --setgroup ceph since I am not root 2024-09-19T06:29:50.630 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:29:50 smithi005 ceph-mds[69356]: -- 172.21.15.5:0/2481752253 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563220729860 con 0x563220706800 2024-09-19T06:29:50.630 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:29:50 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-g[69340]: starting mds.g at 2024-09-19T06:29:50.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-mon[34640]: pgmap v838: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T06:29:50.646 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-h[64788]: ignoring --setuser ceph since I am not root 2024-09-19T06:29:50.646 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-h[64788]: ignoring --setgroup ceph since I am not root 2024-09-19T06:29:50.646 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-mds[64792]: -- 172.21.15.49:0/2268520979 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555bd6ee9860 con 0x555bd6ec7000 2024-09-19T06:29:50.647 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-h[64788]: starting mds.h at 2024-09-19T06:29:50.647 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e[64470]: ignoring --setuser ceph since I am not root 2024-09-19T06:29:50.647 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e[64470]: ignoring --setgroup ceph since I am not root 2024-09-19T06:29:50.647 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-mds[64474]: -- 172.21.15.49:0/857161741 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5591deb2f860 con 0x5591deb0d000 2024-09-19T06:29:50.647 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e[64470]: starting mds.e at 2024-09-19T06:29:50.648 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-k[64160]: ignoring --setuser ceph since I am not root 2024-09-19T06:29:50.648 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-k[64160]: ignoring --setgroup ceph since I am not root 2024-09-19T06:29:50.648 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-mds[64164]: -- 172.21.15.49:0/1021064167 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cacb979860 con 0x55cacb957400 2024-09-19T06:29:50.648 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-k[64160]: starting mds.k at 2024-09-19T06:29:50.648 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: ignoring --setuser ceph since I am not root 2024-09-19T06:29:50.648 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: ignoring --setgroup ceph since I am not root 2024-09-19T06:29:50.648 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-mds[65095]: -- 172.21.15.49:0/1133141673 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a9c6397860 con 0x55a9c6374800 2024-09-19T06:29:50.648 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:29:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: starting mds.b at 2024-09-19T06:29:51.058 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T06:29:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:51.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:51 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1493670899' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T06:29:51.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:51 smithi005 ceph-mon[30955]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T06:29:51.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:51 smithi005 ceph-mon[30955]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T06:29:51.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:51 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1493670899' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T06:29:51.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:51 smithi005 ceph-mon[30955]: osdmap e126: 12 total, 12 up, 12 in 2024-09-19T06:29:51.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:51 smithi005 ceph-mon[30955]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T06:29:51.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:51 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1493670899' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T06:29:51.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:51 smithi157 ceph-mon[35346]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T06:29:51.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:51 smithi157 ceph-mon[35346]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T06:29:51.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:51 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1493670899' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T06:29:51.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:51 smithi157 ceph-mon[35346]: osdmap e126: 12 total, 12 up, 12 in 2024-09-19T06:29:51.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:51 smithi157 ceph-mon[35346]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T06:29:51.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:51 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1493670899' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T06:29:51.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:51 smithi049 ceph-mon[34640]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T06:29:51.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:51 smithi049 ceph-mon[34640]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T06:29:51.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:51 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1493670899' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T06:29:51.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:51 smithi049 ceph-mon[34640]: osdmap e126: 12 total, 12 up, 12 in 2024-09-19T06:29:51.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:51 smithi049 ceph-mon[34640]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T06:29:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.789+0000 7f52363e7640 1 -- 172.21.15.5:0/1662670246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5230102790 msgr2=0x7f52301067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.789+0000 7f52363e7640 1 --2- 172.21.15.5:0/1662670246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5230102790 0x7f52301067e0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f5224009900 tx=0x7f522402a410 comp rx=0 tx=0).stop 2024-09-19T06:29:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.789+0000 7f52363e7640 1 -- 172.21.15.5:0/1662670246 shutdown_connections 2024-09-19T06:29:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.789+0000 7f52363e7640 1 --2- 172.21.15.5:0/1662670246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5230106f10 0x7f523010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.789+0000 7f52363e7640 1 --2- 172.21.15.5:0/1662670246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5230102790 0x7f52301067e0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.789+0000 7f52363e7640 1 --2- 172.21.15.5:0/1662670246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5230101de0 0x7f52301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.789+0000 7f52363e7640 1 -- 172.21.15.5:0/1662670246 >> 172.21.15.5:0/1662670246 conn(0x7f52300fd8c0 msgr2=0x7f52300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.790+0000 7f52363e7640 1 -- 172.21.15.5:0/1662670246 shutdown_connections 2024-09-19T06:29:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.790+0000 7f52363e7640 1 -- 172.21.15.5:0/1662670246 wait complete. 2024-09-19T06:29:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.790+0000 7f52363e7640 1 Processor -- start 2024-09-19T06:29:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.790+0000 7f52363e7640 1 -- start start 2024-09-19T06:29:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.790+0000 7f52363e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5230101de0 0x7f52300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f52363e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5230102790 0x7f5230074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f52363e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5230106f10 0x7f5230074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f52363e7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5230111e40 con 0x7f5230101de0 2024-09-19T06:29:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f52363e7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5230111cc0 con 0x7f5230106f10 2024-09-19T06:29:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f52363e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5230111fc0 con 0x7f5230102790 2024-09-19T06:29:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5230101de0 0x7f52300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5230101de0 0x7f52300744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56490/0 (socket says 172.21.15.5:56490) 2024-09-19T06:29:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522ffff640 1 -- 172.21.15.5:0/1334159891 learned_addr learned my addr 172.21.15.5:0/1334159891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f5234984640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5230106f10 0x7f5230074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522f7fe640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5230102790 0x7f5230074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522ffff640 1 -- 172.21.15.5:0/1334159891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5230102790 msgr2=0x7f5230074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522ffff640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5230102790 0x7f5230074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522ffff640 1 -- 172.21.15.5:0/1334159891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5230106f10 msgr2=0x7f5230074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522ffff640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5230106f10 0x7f5230074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522ffff640 1 -- 172.21.15.5:0/1334159891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52301a68e0 con 0x7f5230101de0 2024-09-19T06:29:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522ffff640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5230101de0 0x7f52300744f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f522000b4a0 tx=0x7f522000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522d7fa640 1 -- 172.21.15.5:0/1334159891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220014070 con 0x7f5230101de0 2024-09-19T06:29:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522d7fa640 1 -- 172.21.15.5:0/1334159891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5220010ac0 con 0x7f5230101de0 2024-09-19T06:29:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f52363e7640 1 -- 172.21.15.5:0/1334159891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52301a6bd0 con 0x7f5230101de0 2024-09-19T06:29:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.791+0000 7f522d7fa640 1 -- 172.21.15.5:0/1334159891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220017c30 con 0x7f5230101de0 2024-09-19T06:29:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.792+0000 7f52363e7640 1 -- 172.21.15.5:0/1334159891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52301a70e0 con 0x7f5230101de0 2024-09-19T06:29:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.792+0000 7f52363e7640 1 -- 172.21.15.5:0/1334159891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52301021c0 con 0x7f5230101de0 2024-09-19T06:29:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.794+0000 7f522d7fa640 1 -- 172.21.15.5:0/1334159891 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5220017070 con 0x7f5230101de0 2024-09-19T06:29:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.794+0000 7f522d7fa640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52180a33b0 0x7f52180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.794+0000 7f522f7fe640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52180a33b0 0x7f52180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.795+0000 7f522d7fa640 1 -- 172.21.15.5:0/1334159891 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8936+0+0 (secure 0 0 0) 0x7f52200cf430 con 0x7f5230101de0 2024-09-19T06:29:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.795+0000 7f522f7fe640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52180a33b0 0x7f52180a5870 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5224031440 tx=0x7f5224005b50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:51.799+0000 7f522d7fa640 1 -- 172.21.15.5:0/1334159891 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f522009b6d0 con 0x7f5230101de0 2024-09-19T06:29:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.011+0000 7f52363e7640 1 -- 172.21.15.5:0/1334159891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f523010a0f0 con 0x7f5230101de0 2024-09-19T06:29:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.313+0000 7f522d7fa640 1 -- 172.21.15.5:0/1334159891 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7f522009f9d0 con 0x7f5230101de0 2024-09-19T06:29:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.316+0000 7f52363e7640 1 -- 172.21.15.5:0/1334159891 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52180a33b0 msgr2=0x7f52180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.316+0000 7f52363e7640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52180a33b0 0x7f52180a5870 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5224031440 tx=0x7f5224005b50 comp rx=0 tx=0).stop 2024-09-19T06:29:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.316+0000 7f52363e7640 1 -- 172.21.15.5:0/1334159891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5230101de0 msgr2=0x7f52300744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.316+0000 7f52363e7640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5230101de0 0x7f52300744f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f522000b4a0 tx=0x7f522000b960 comp rx=0 tx=0).stop 2024-09-19T06:29:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.316+0000 7f52363e7640 1 -- 172.21.15.5:0/1334159891 shutdown_connections 2024-09-19T06:29:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.316+0000 7f52363e7640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5230106f10 0x7f5230074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.316+0000 7f52363e7640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52180a33b0 0x7f52180a5870 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.316+0000 7f52363e7640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5230102790 0x7f5230074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.316+0000 7f52363e7640 1 --2- 172.21.15.5:0/1334159891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5230101de0 0x7f52300744f0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.316+0000 7f52363e7640 1 -- 172.21.15.5:0/1334159891 >> 172.21.15.5:0/1334159891 conn(0x7f52300fd8c0 msgr2=0x7f52300fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.316+0000 7f52363e7640 1 -- 172.21.15.5:0/1334159891 shutdown_connections 2024-09-19T06:29:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:52.316+0000 7f52363e7640 1 -- 172.21.15.5:0/1334159891 wait complete. 2024-09-19T06:29:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: pgmap v840: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-19T06:29:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.49:6832/4231294365,v1:172.21.15.49:6834/4231294365] up:boot 2024-09-19T06:29:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.49:6833/2880440169,v1:172.21.15.49:6835/2880440169] up:boot 2024-09-19T06:29:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.49:6836/2811837150,v1:172.21.15.49:6838/2811837150] up:boot 2024-09-19T06:29:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.49:6837/4148570473,v1:172.21.15.49:6839/4148570473] up:boot 2024-09-19T06:29:52.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.157:6833/3046029726,v1:172.21.15.157:6835/3046029726] up:boot 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.157:6832/2572870892,v1:172.21.15.157:6834/2572870892] up:boot 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.157:6836/1297060068,v1:172.21.15.157:6838/1297060068] up:boot 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.157:6837/4261362078,v1:172.21.15.157:6839/4261362078] up:boot 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.5:6838/2118854487,v1:172.21.15.5:6840/2118854487] up:boot 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: mds.? [v2:172.21.15.5:6839/338959061,v1:172.21.15.5:6841/338959061] up:boot 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:29:52.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:29:52.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:29:52.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:29:52.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:52 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1334159891' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:29:52.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: pgmap v840: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-19T06:29:52.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.49:6832/4231294365,v1:172.21.15.49:6834/4231294365] up:boot 2024-09-19T06:29:52.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.49:6833/2880440169,v1:172.21.15.49:6835/2880440169] up:boot 2024-09-19T06:29:52.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.49:6836/2811837150,v1:172.21.15.49:6838/2811837150] up:boot 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.49:6837/4148570473,v1:172.21.15.49:6839/4148570473] up:boot 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.157:6833/3046029726,v1:172.21.15.157:6835/3046029726] up:boot 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.157:6832/2572870892,v1:172.21.15.157:6834/2572870892] up:boot 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.157:6836/1297060068,v1:172.21.15.157:6838/1297060068] up:boot 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.157:6837/4261362078,v1:172.21.15.157:6839/4261362078] up:boot 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.5:6838/2118854487,v1:172.21.15.5:6840/2118854487] up:boot 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: mds.? [v2:172.21.15.5:6839/338959061,v1:172.21.15.5:6841/338959061] up:boot 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:29:52.647 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:29:52.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:29:52.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:29:52.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:29:52.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:29:52.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:29:52.648 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:52 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1334159891' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:29:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: pgmap v840: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-19T06:29:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.49:6832/4231294365,v1:172.21.15.49:6834/4231294365] up:boot 2024-09-19T06:29:52.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.49:6833/2880440169,v1:172.21.15.49:6835/2880440169] up:boot 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.49:6836/2811837150,v1:172.21.15.49:6838/2811837150] up:boot 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.49:6837/4148570473,v1:172.21.15.49:6839/4148570473] up:boot 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.157:6833/3046029726,v1:172.21.15.157:6835/3046029726] up:boot 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.157:6832/2572870892,v1:172.21.15.157:6834/2572870892] up:boot 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.157:6836/1297060068,v1:172.21.15.157:6838/1297060068] up:boot 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.157:6837/4261362078,v1:172.21.15.157:6839/4261362078] up:boot 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.5:6838/2118854487,v1:172.21.15.5:6840/2118854487] up:boot 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: mds.? [v2:172.21.15.5:6839/338959061,v1:172.21.15.5:6841/338959061] up:boot 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:29:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:29:52.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:29:52.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:29:52.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:29:52.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:29:52.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: from='mgr.15240 172.21.15.5:0/1825398989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:29:52.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:52 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1334159891' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:29:53.041 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T06:29:53.297 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:53.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:53 smithi005 ceph-mon[30955]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T06:29:53.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:53 smithi005 ceph-mon[30955]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T06:29:53.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:53 smithi005 ceph-mon[30955]: Cluster is now healthy 2024-09-19T06:29:53.887 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:53 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/1334159891' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:29:53.887 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:53 smithi005 ceph-mon[30955]: fsmap 12 up:standby 2024-09-19T06:29:53.888 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:53 smithi157 ceph-mon[35346]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T06:29:53.888 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:53 smithi157 ceph-mon[35346]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T06:29:53.888 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:53 smithi157 ceph-mon[35346]: Cluster is now healthy 2024-09-19T06:29:53.888 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:53 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/1334159891' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:29:53.888 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:53 smithi157 ceph-mon[35346]: fsmap 12 up:standby 2024-09-19T06:29:53.888 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:53 smithi049 ceph-mon[34640]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T06:29:53.888 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:53 smithi049 ceph-mon[34640]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T06:29:53.888 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:53 smithi049 ceph-mon[34640]: Cluster is now healthy 2024-09-19T06:29:53.888 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:53 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/1334159891' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:29:53.888 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:53 smithi049 ceph-mon[34640]: fsmap 12 up:standby 2024-09-19T06:29:53.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.742+0000 7f505b73f640 1 -- 172.21.15.5:0/221784761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5054106a80 msgr2=0x7f5054106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:53.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.742+0000 7f505b73f640 1 --2- 172.21.15.5:0/221784761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5054106a80 0x7f5054106e60 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5050009930 tx=0x7f505002a420 comp rx=0 tx=0).stop 2024-09-19T06:29:53.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.742+0000 7f505b73f640 1 -- 172.21.15.5:0/221784761 shutdown_connections 2024-09-19T06:29:53.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.742+0000 7f505b73f640 1 --2- 172.21.15.5:0/221784761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5054101420 0x7f505410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:53.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.742+0000 7f505b73f640 1 --2- 172.21.15.5:0/221784761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5054100a80 0x7f5054100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:53.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.742+0000 7f505b73f640 1 --2- 172.21.15.5:0/221784761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5054106a80 0x7f5054106e60 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:53.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.742+0000 7f505b73f640 1 -- 172.21.15.5:0/221784761 >> 172.21.15.5:0/221784761 conn(0x7f50540fc7a0 msgr2=0x7f50540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:53.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.743+0000 7f505b73f640 1 -- 172.21.15.5:0/221784761 shutdown_connections 2024-09-19T06:29:53.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.743+0000 7f505b73f640 1 -- 172.21.15.5:0/221784761 wait complete. 2024-09-19T06:29:53.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.743+0000 7f505b73f640 1 Processor -- start 2024-09-19T06:29:53.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.743+0000 7f505b73f640 1 -- start start 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.743+0000 7f505b73f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5054100a80 0x7f5054199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.743+0000 7f505b73f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5054101420 0x7f50541998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.743+0000 7f505b73f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5054106a80 0x7f50541a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.743+0000 7f505b73f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5054110dd0 con 0x7f5054100a80 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.743+0000 7f505b73f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5054110c50 con 0x7f5054101420 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.743+0000 7f505b73f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5054110f50 con 0x7f5054106a80 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.743+0000 7f50594db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5054100a80 0x7f5054199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.744+0000 7f50594db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5054100a80 0x7f5054199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56518/0 (socket says 172.21.15.5:56518) 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.744+0000 7f50594db640 1 -- 172.21.15.5:0/916072819 learned_addr learned my addr 172.21.15.5:0/916072819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.744+0000 7f5058cda640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5054101420 0x7f50541998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.744+0000 7f5059cdc640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5054106a80 0x7f50541a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.744+0000 7f50594db640 1 -- 172.21.15.5:0/916072819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5054106a80 msgr2=0x7f50541a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.744+0000 7f50594db640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5054106a80 0x7f50541a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.744+0000 7f50594db640 1 -- 172.21.15.5:0/916072819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5054101420 msgr2=0x7f50541998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.744+0000 7f50594db640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5054101420 0x7f50541998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.744+0000 7f50594db640 1 -- 172.21.15.5:0/916072819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50541a4920 con 0x7f5054100a80 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.744+0000 7f50594db640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5054100a80 0x7f5054199360 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5050005940 tx=0x7f50500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.744+0000 7f504a7fc640 1 -- 172.21.15.5:0/916072819 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505003f030 con 0x7f5054100a80 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.744+0000 7f505b73f640 1 -- 172.21.15.5:0/916072819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50541a4af0 con 0x7f5054100a80 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.745+0000 7f504a7fc640 1 -- 172.21.15.5:0/916072819 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5050003cb0 con 0x7f5054100a80 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.745+0000 7f505b73f640 1 -- 172.21.15.5:0/916072819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50541a4fa0 con 0x7f5054100a80 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.745+0000 7f504a7fc640 1 -- 172.21.15.5:0/916072819 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050035070 con 0x7f5054100a80 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.746+0000 7f505b73f640 1 -- 172.21.15.5:0/916072819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5054100f60 con 0x7f5054100a80 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.747+0000 7f504a7fc640 1 -- 172.21.15.5:0/916072819 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5050042030 con 0x7f5054100a80 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.748+0000 7f504a7fc640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f50400a33b0 0x7f50400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.748+0000 7f5058cda640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f50400a33b0 0x7f50400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:53.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.748+0000 7f504a7fc640 1 -- 172.21.15.5:0/916072819 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8936+0+0 (secure 0 0 0) 0x7f50500f0780 con 0x7f5054100a80 2024-09-19T06:29:53.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.748+0000 7f5058cda640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f50400a33b0 0x7f50400a5870 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f505406b2e0 tx=0x7f5044009110 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:53.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.751+0000 7f504a7fc640 1 -- 172.21.15.5:0/916072819 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f50500bcf60 con 0x7f5054100a80 2024-09-19T06:29:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:53.928+0000 7f505b73f640 1 -- 172.21.15.5:0/916072819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5054108ff0 con 0x7f5054100a80 2024-09-19T06:29:54.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.590+0000 7f504a7fc640 1 -- 172.21.15.5:0/916072819 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) ==== 181+0+0 (secure 0 0 0) 0x7f50500bc860 con 0x7f5054100a80 2024-09-19T06:29:54.593 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-09-19T06:29:54.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.594+0000 7f505b73f640 1 -- 172.21.15.5:0/916072819 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f50400a33b0 msgr2=0x7f50400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:54.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.594+0000 7f505b73f640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f50400a33b0 0x7f50400a5870 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f505406b2e0 tx=0x7f5044009110 comp rx=0 tx=0).stop 2024-09-19T06:29:54.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.594+0000 7f505b73f640 1 -- 172.21.15.5:0/916072819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5054100a80 msgr2=0x7f5054199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:54.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.594+0000 7f505b73f640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5054100a80 0x7f5054199360 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5050005940 tx=0x7f50500042f0 comp rx=0 tx=0).stop 2024-09-19T06:29:54.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.595+0000 7f505b73f640 1 -- 172.21.15.5:0/916072819 shutdown_connections 2024-09-19T06:29:54.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.595+0000 7f505b73f640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f50400a33b0 0x7f50400a5870 secure :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f505406b2e0 tx=0x7f5044009110 comp rx=0 tx=0).stop 2024-09-19T06:29:54.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.595+0000 7f505b73f640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5054106a80 0x7f50541a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:54.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.595+0000 7f505b73f640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5054101420 0x7f50541998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:54.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.595+0000 7f505b73f640 1 --2- 172.21.15.5:0/916072819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5054100a80 0x7f5054199360 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:54.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.595+0000 7f505b73f640 1 -- 172.21.15.5:0/916072819 >> 172.21.15.5:0/916072819 conn(0x7f50540fc7a0 msgr2=0x7f50540fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:54.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.596+0000 7f505b73f640 1 -- 172.21.15.5:0/916072819 shutdown_connections 2024-09-19T06:29:54.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:54.597+0000 7f505b73f640 1 -- 172.21.15.5:0/916072819 wait complete. 2024-09-19T06:29:54.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:54 smithi005 ceph-mon[30955]: pgmap v841: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T06:29:54.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:54 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/916072819' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T06:29:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:54 smithi049 ceph-mon[34640]: pgmap v841: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T06:29:54.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:54 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/916072819' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T06:29:55.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:54 smithi157 ceph-mon[35346]: pgmap v841: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T06:29:55.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:54 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/916072819' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T06:29:55.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T06:29:55.488 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:29:55.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:55 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/916072819' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T06:29:55.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:55 smithi005 ceph-mon[30955]: osdmap e127: 12 total, 12 up, 12 in 2024-09-19T06:29:55.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:55 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/916072819' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T06:29:55.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:55 smithi049 ceph-mon[34640]: osdmap e127: 12 total, 12 up, 12 in 2024-09-19T06:29:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.976+0000 7f5296b74640 1 -- 172.21.15.5:0/2248071731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5290102790 msgr2=0x7f52901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.976+0000 7f5296b74640 1 --2- 172.21.15.5:0/2248071731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5290102790 0x7f52901067e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5280009900 tx=0x7f528002a410 comp rx=0 tx=0).stop 2024-09-19T06:29:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.976+0000 7f5296b74640 1 -- 172.21.15.5:0/2248071731 shutdown_connections 2024-09-19T06:29:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.976+0000 7f5296b74640 1 --2- 172.21.15.5:0/2248071731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5290106f10 0x7f529010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.976+0000 7f5296b74640 1 --2- 172.21.15.5:0/2248071731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5290102790 0x7f52901067e0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.976+0000 7f5296b74640 1 --2- 172.21.15.5:0/2248071731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5290101de0 0x7f52901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.976+0000 7f5296b74640 1 -- 172.21.15.5:0/2248071731 >> 172.21.15.5:0/2248071731 conn(0x7f52900fd8c0 msgr2=0x7f52900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.976+0000 7f5296b74640 1 -- 172.21.15.5:0/2248071731 shutdown_connections 2024-09-19T06:29:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.976+0000 7f5296b74640 1 -- 172.21.15.5:0/2248071731 wait complete. 2024-09-19T06:29:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.976+0000 7f5296b74640 1 Processor -- start 2024-09-19T06:29:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.976+0000 7f5296b74640 1 -- start start 2024-09-19T06:29:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5296b74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5290101de0 0x7f52901983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5296b74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5290102790 0x7f5290198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5296b74640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5290106f10 0x7f529019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5296b74640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5290111e90 con 0x7f5290102790 2024-09-19T06:29:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5296b74640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5290111d10 con 0x7f5290106f10 2024-09-19T06:29:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5296b74640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5290112010 con 0x7f5290101de0 2024-09-19T06:29:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5290102790 0x7f5290198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5290102790 0x7f5290198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35130/0 (socket says 172.21.15.5:35130) 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5287fff640 1 -- 172.21.15.5:0/136897459 learned_addr learned my addr 172.21.15.5:0/136897459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5295111640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5290106f10 0x7f529019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5294910640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5290101de0 0x7f52901983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5287fff640 1 -- 172.21.15.5:0/136897459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5290101de0 msgr2=0x7f52901983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5287fff640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5290101de0 0x7f52901983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5287fff640 1 -- 172.21.15.5:0/136897459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5290106f10 msgr2=0x7f529019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5287fff640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5290106f10 0x7f529019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5287fff640 1 -- 172.21.15.5:0/136897459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529019d380 con 0x7f5290102790 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5295111640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5290106f10 0x7f529019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5294910640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5290101de0 0x7f52901983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.977+0000 7f5287fff640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5290102790 0x7f5290198910 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5280005ab0 tx=0x7f5280003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.978+0000 7f5285ffb640 1 -- 172.21.15.5:0/136897459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528003f070 con 0x7f5290102790 2024-09-19T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.978+0000 7f5285ffb640 1 -- 172.21.15.5:0/136897459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5280031d60 con 0x7f5290102790 2024-09-19T06:29:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.978+0000 7f5296b74640 1 -- 172.21.15.5:0/136897459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529019d610 con 0x7f5290102790 2024-09-19T06:29:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.978+0000 7f5285ffb640 1 -- 172.21.15.5:0/136897459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280035030 con 0x7f5290102790 2024-09-19T06:29:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.978+0000 7f5296b74640 1 -- 172.21.15.5:0/136897459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52901a4f50 con 0x7f5290102790 2024-09-19T06:29:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.979+0000 7f5296b74640 1 -- 172.21.15.5:0/136897459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52901032c0 con 0x7f5290102790 2024-09-19T06:29:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.980+0000 7f5285ffb640 1 -- 172.21.15.5:0/136897459 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5280042070 con 0x7f5290102790 2024-09-19T06:29:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.981+0000 7f5285ffb640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52780a33b0 0x7f52780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:29:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.981+0000 7f5294910640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52780a33b0 0x7f52780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:29:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.981+0000 7f5285ffb640 1 -- 172.21.15.5:0/136897459 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8382+0+0 (secure 0 0 0) 0x7f52800f0640 con 0x7f5290102790 2024-09-19T06:29:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.981+0000 7f5294910640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52780a33b0 0x7f52780a5870 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f528c0058c0 tx=0x7f528c005850 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:29:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:55.985+0000 7f5285ffb640 1 -- 172.21.15.5:0/136897459 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f52800bd0b0 con 0x7f5290102790 2024-09-19T06:29:56.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:55 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/916072819' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T06:29:56.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:55 smithi157 ceph-mon[35346]: osdmap e127: 12 total, 12 up, 12 in 2024-09-19T06:29:56.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.164+0000 7f5296b74640 1 -- 172.21.15.5:0/136897459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f529010a150 con 0x7f5290102790 2024-09-19T06:29:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.602+0000 7f5285ffb640 1 -- 172.21.15.5:0/136897459 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v128) ==== 169+0+0 (secure 0 0 0) 0x7f52800bc9b0 con 0x7f5290102790 2024-09-19T06:29:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-09-19T06:29:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.607+0000 7f5296b74640 1 -- 172.21.15.5:0/136897459 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52780a33b0 msgr2=0x7f52780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.608+0000 7f5296b74640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52780a33b0 0x7f52780a5870 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f528c0058c0 tx=0x7f528c005850 comp rx=0 tx=0).stop 2024-09-19T06:29:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.608+0000 7f5296b74640 1 -- 172.21.15.5:0/136897459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5290102790 msgr2=0x7f5290198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:29:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.608+0000 7f5296b74640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5290102790 0x7f5290198910 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5280005ab0 tx=0x7f5280003cd0 comp rx=0 tx=0).stop 2024-09-19T06:29:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.608+0000 7f5296b74640 1 -- 172.21.15.5:0/136897459 shutdown_connections 2024-09-19T06:29:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.608+0000 7f5296b74640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5290106f10 0x7f529019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.608+0000 7f5296b74640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5290102790 0x7f5290198910 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.608+0000 7f5296b74640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.5:6800/2837801016,v1:172.21.15.5:6801/2837801016] conn(0x7f52780a33b0 0x7f52780a5870 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.608+0000 7f5296b74640 1 --2- 172.21.15.5:0/136897459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5290101de0 0x7f52901983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:29:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.608+0000 7f5296b74640 1 -- 172.21.15.5:0/136897459 >> 172.21.15.5:0/136897459 conn(0x7f52900fd8c0 msgr2=0x7f52900feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:29:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.608+0000 7f5296b74640 1 -- 172.21.15.5:0/136897459 shutdown_connections 2024-09-19T06:29:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-19T06:29:56.609+0000 7f5296b74640 1 -- 172.21.15.5:0/136897459 wait complete. 2024-09-19T06:29:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:56 smithi005 ceph-mon[30955]: pgmap v843: 65 pgs: 65 active+clean; 298 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:29:56.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:56 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/136897459' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T06:29:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:56 smithi049 ceph-mon[34640]: pgmap v843: 65 pgs: 65 active+clean; 298 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:29:56.896 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:56 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/136897459' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T06:29:57.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:56 smithi157 ceph-mon[35346]: pgmap v843: 65 pgs: 65 active+clean; 298 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:29:57.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:56 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/136897459' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T06:29:57.308 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:29:57.340 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:29:57.376 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:29:57.412 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T06:29:57.412 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T06:29:57.439 DEBUG:teuthology.orchestra.run.smithi049:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T06:29:57.460 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T06:29:57.476 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T06:29:57.476 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T06:29:57.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.a 2024-09-19T06:29:57.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:57 smithi049 ceph-mon[34640]: from='client.? 172.21.15.5:0/136897459' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T06:29:57.646 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:57 smithi049 ceph-mon[34640]: osdmap e128: 12 total, 12 up, 12 in 2024-09-19T06:29:57.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:57 smithi005 systemd[1]: Stopping Ceph mon.a for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:29:57.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:57 smithi005 ceph-mon[30955]: from='client.? 172.21.15.5:0/136897459' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T06:29:57.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:57 smithi005 ceph-mon[30955]: osdmap e128: 12 total, 12 up, 12 in 2024-09-19T06:29:58.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:57 smithi157 ceph-mon[35346]: from='client.? 172.21.15.5:0/136897459' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T06:29:58.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:29:57 smithi157 ceph-mon[35346]: osdmap e128: 12 total, 12 up, 12 in 2024-09-19T06:29:58.073 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:57 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a[30951]: 2024-09-19T06:29:57.808+0000 7f860b01a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:29:58.073 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:57 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a[30951]: 2024-09-19T06:29:57.808+0000 7f860b01a640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T06:29:58.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:58 smithi005 podman[135395]: 2024-09-19 06:29:58.072841045 +0000 UTC m=+0.385541585 container died 550e96d2d6843dafb63080c6849096e67598494006cedfd54fdd2e2d32350d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T06:29:58.774 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:58 smithi005 podman[135395]: 2024-09-19 06:29:58.521000829 +0000 UTC m=+0.833701369 container cleanup 550e96d2d6843dafb63080c6849096e67598494006cedfd54fdd2e2d32350d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T06:29:58.775 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:58 smithi005 bash[135395]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a 2024-09-19T06:29:59.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 19 06:29:58 smithi005 podman[135492]: 2024-09-19 06:29:58.772530678 +0000 UTC m=+0.692180073 container remove 550e96d2d6843dafb63080c6849096e67598494006cedfd54fdd2e2d32350d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T06:29:59.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T06:29:59.631 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:29:59.631 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T06:29:59.631 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T06:29:59.632 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.b 2024-09-19T06:30:00.011 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:29:59 smithi049 systemd[1]: Stopping Ceph mon.b for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:30:00.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:30:00 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-b[34636]: 2024-09-19T06:30:00.009+0000 7f62e9ef9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:30:00.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:30:00 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-b[34636]: 2024-09-19T06:30:00.009+0000 7f62e9ef9640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T06:30:00.396 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:30:00 smithi049 podman[75473]: 2024-09-19 06:30:00.163110125 +0000 UTC m=+0.282089654 container died 492ac721e6fc6af54cd66e4454e68f17a52210143ea06b4e65532363c87660cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:30:00.587 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:30:00.567+0000 7f3f34af6640 -1 client.15267: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-19T06:30:00.587 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:30:00.569+0000 7f3f34af6640 -1 client.15267: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-19T06:30:00.587 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: Exception in thread Thread-31: 2024-09-19T06:30:00.588 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: Traceback (most recent call last): 2024-09-19T06:30:00.588 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-09-19T06:30:00.588 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self.run() 2024-09-19T06:30:00.588 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-09-19T06:30:00.588 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self.function(*self.args, **self.kwargs) 2024-09-19T06:30:00.588 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-09-19T06:30:00.588 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: self.prune_snapshots(sched) 2024-09-19T06:30:00.588 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-09-19T06:30:00.678 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:30:00 smithi049 podman[75473]: 2024-09-19 06:30:00.484704214 +0000 UTC m=+0.603683795 container cleanup 492ac721e6fc6af54cd66e4454e68f17a52210143ea06b4e65532363c87660cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2) 2024-09-19T06:30:00.678 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:30:00 smithi049 bash[75473]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-b 2024-09-19T06:30:00.919 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T06:30:00.944 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:30:00 smithi049 podman[75486]: 2024-09-19 06:30:00.677201718 +0000 UTC m=+0.508936172 container remove 492ac721e6fc6af54cd66e4454e68f17a52210143ea06b4e65532363c87660cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T06:30:00.944 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:30:00 smithi049 systemd[1]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-19T06:30:00.944 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:30:00 smithi049 systemd[1]: Stopped Ceph mon.b for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:30:00.944 INFO:journalctl@ceph.mon.b.smithi049.stdout:Sep 19 06:30:00 smithi049 systemd[1]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.b.service: Consumed 19.582s CPU time. 2024-09-19T06:30:00.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:30:00.970 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T06:30:00.970 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T06:30:00.970 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.c 2024-09-19T06:30:01.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:30:01 smithi157 systemd[1]: Stopping Ceph mon.c for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:30:01.646 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:30:01 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-c[35342]: 2024-09-19T06:30:01.326+0000 7fc76531e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:30:01.647 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:30:01 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-c[35342]: 2024-09-19T06:30:01.326+0000 7fc76531e640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T06:30:01.647 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:30:01 smithi157 podman[74809]: 2024-09-19 06:30:01.348311336 +0000 UTC m=+0.143646513 container died 517a293e9503fad54b83dff27f5bc486bf1c147a5466728d5526c900ef64d169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T06:30:02.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:30:01 smithi157 podman[74809]: 2024-09-19 06:30:01.761045957 +0000 UTC m=+0.556381134 container cleanup 517a293e9503fad54b83dff27f5bc486bf1c147a5466728d5526c900ef64d169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T06:30:02.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:30:01 smithi157 bash[74809]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-c 2024-09-19T06:30:02.232 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T06:30:02.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:30:02 smithi157 podman[74821]: 2024-09-19 06:30:02.004369046 +0000 UTC m=+0.649327366 container remove 517a293e9503fad54b83dff27f5bc486bf1c147a5466728d5526c900ef64d169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T06:30:02.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:30:02 smithi157 systemd[1]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-19T06:30:02.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:30:02 smithi157 systemd[1]: Stopped Ceph mon.c for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:30:02.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 06:30:02 smithi157 systemd[1]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mon.c.service: Consumed 18.697s CPU time. 2024-09-19T06:30:02.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:30:02.287 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T06:30:02.287 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T06:30:02.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.x 2024-09-19T06:30:02.580 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:02 smithi005 systemd[1]: Stopping Ceph mgr.x for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:30:02.860 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:02 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x[31183]: 2024-09-19T06:30:02.577+0000 7f3fd2197640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:30:02.860 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:02 smithi005 podman[137928]: 2024-09-19 06:30:02.609012254 +0000 UTC m=+0.125366090 container died 67f13a9e350e9d53f5be01555dc1d2c319be6bae6ecee696cc5d097ed887e459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-19T06:30:03.129 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:02 smithi005 podman[137928]: 2024-09-19 06:30:02.978371638 +0000 UTC m=+0.494725519 container cleanup 67f13a9e350e9d53f5be01555dc1d2c319be6bae6ecee696cc5d097ed887e459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T06:30:03.129 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:02 smithi005 bash[137928]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x 2024-09-19T06:30:03.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T06:30:03.439 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:03 smithi005 podman[137940]: 2024-09-19 06:30:03.163441702 +0000 UTC m=+0.548371832 container remove 67f13a9e350e9d53f5be01555dc1d2c319be6bae6ecee696cc5d097ed887e459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T06:30:03.440 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:03 smithi005 systemd[1]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-19T06:30:03.440 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:03 smithi005 systemd[1]: Stopped Ceph mgr.x for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:30:03.440 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 19 06:30:03 smithi005 systemd[1]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.x.service: Consumed 1min 5.430s CPU time. 2024-09-19T06:30:03.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:30:03.457 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T06:30:03.457 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T06:30:03.458 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.y 2024-09-19T06:30:03.811 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:30:03 smithi049 systemd[1]: Stopping Ceph mgr.y for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:30:04.074 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:30:03 smithi049 podman[75592]: 2024-09-19 06:30:03.810937383 +0000 UTC m=+0.157289360 container died 9571f2bb9a57f07075d1e9cbdaa260b301bebef1fbed847a9583def821a381b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T06:30:04.396 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:30:04 smithi049 podman[75592]: 2024-09-19 06:30:04.180241541 +0000 UTC m=+0.526593559 container cleanup 9571f2bb9a57f07075d1e9cbdaa260b301bebef1fbed847a9583def821a381b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T06:30:04.396 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 19 06:30:04 smithi049 bash[75592]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-y 2024-09-19T06:30:04.555 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T06:30:04.592 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:30:04.593 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T06:30:04.593 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T06:30:04.593 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.z 2024-09-19T06:30:05.039 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:30:04 smithi157 systemd[1]: Stopping Ceph mgr.z for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:30:05.292 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:30:05 smithi157 podman[74934]: 2024-09-19 06:30:05.038255973 +0000 UTC m=+0.251300570 container died 48f555e37563d5177bd807409af4138fbf86a54f68de56735f4fbeed8753f787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:30:05.551 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:30:05 smithi157 podman[74934]: 2024-09-19 06:30:05.373652542 +0000 UTC m=+0.586697129 container cleanup 48f555e37563d5177bd807409af4138fbf86a54f68de56735f4fbeed8753f787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T06:30:05.551 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:30:05 smithi157 bash[74934]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z 2024-09-19T06:30:05.794 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T06:30:05.818 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:30:05 smithi157 podman[74946]: 2024-09-19 06:30:05.566644813 +0000 UTC m=+0.532959351 container remove 48f555e37563d5177bd807409af4138fbf86a54f68de56735f4fbeed8753f787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T06:30:05.818 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:30:05 smithi157 systemd[1]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-19T06:30:05.818 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:30:05 smithi157 systemd[1]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-19T06:30:05.818 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:30:05 smithi157 systemd[1]: Stopped Ceph mgr.z for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:30:05.818 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 06:30:05 smithi157 systemd[1]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mgr.z.service: Consumed 12.060s CPU time. 2024-09-19T06:30:05.830 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:30:05.830 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T06:30:05.830 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T06:30:05.830 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.0 2024-09-19T06:30:06.129 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:05 smithi005 systemd[1]: Stopping Ceph osd.0 for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:30:06.629 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:06 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0[42192]: 2024-09-19T06:30:06.258+0000 7fd864d7b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:30:06.629 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:06 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0[42192]: 2024-09-19T06:30:06.258+0000 7fd864d7b640 -1 osd.0 128 *** Got signal Terminated *** 2024-09-19T06:30:06.629 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:06 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0[42192]: 2024-09-19T06:30:06.258+0000 7fd864d7b640 -1 osd.0 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:30:11.910 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:11 smithi005 podman[138050]: 2024-09-19 06:30:11.634984123 +0000 UTC m=+5.601381622 container died 989bb7ea8e568138975a4c8d46c6057fa6b70bdc1934c9e9c6cc8740a728d8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T06:30:12.286 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:12 smithi005 podman[138050]: 2024-09-19 06:30:12.075289931 +0000 UTC m=+6.041687410 container cleanup 989bb7ea8e568138975a4c8d46c6057fa6b70bdc1934c9e9c6cc8740a728d8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T06:30:12.286 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:12 smithi005 bash[138050]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0 2024-09-19T06:30:12.629 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:12 smithi005 podman[138064]: 2024-09-19 06:30:12.285333887 +0000 UTC m=+0.644765582 container remove 989bb7ea8e568138975a4c8d46c6057fa6b70bdc1934c9e9c6cc8740a728d8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T06:30:12.965 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:12 smithi005 podman[138156]: 2024-09-19 06:30:12.603445342 +0000 UTC m=+0.016557573 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:30:12.965 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:12 smithi005 podman[138156]: 2024-09-19 06:30:12.753729099 +0000 UTC m=+0.166841312 container create 3adc42d66ddea060b7670505d421be6ac4099bcf4738cb88e95932415238a17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:30:12.965 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:12 smithi005 podman[138156]: 2024-09-19 06:30:12.964271203 +0000 UTC m=+0.377383434 container init 3adc42d66ddea060b7670505d421be6ac4099bcf4738cb88e95932415238a17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:30:13.379 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:12 smithi005 podman[138156]: 2024-09-19 06:30:12.967921917 +0000 UTC m=+0.381034129 container start 3adc42d66ddea060b7670505d421be6ac4099bcf4738cb88e95932415238a17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T06:30:13.379 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:13 smithi005 podman[138156]: 2024-09-19 06:30:13.081566049 +0000 UTC m=+0.494678267 container attach 3adc42d66ddea060b7670505d421be6ac4099bcf4738cb88e95932415238a17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T06:30:13.379 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 19 06:30:13 smithi005 podman[138156]: 2024-09-19 06:30:13.178155104 +0000 UTC m=+0.591267330 container died 3adc42d66ddea060b7670505d421be6ac4099bcf4738cb88e95932415238a17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T06:30:13.720 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T06:30:13.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:30:13.761 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T06:30:13.761 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T06:30:13.762 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.1 2024-09-19T06:30:14.061 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:13 smithi049 systemd[1]: Stopping Ceph osd.1 for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:30:14.396 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:14 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1[39135]: 2024-09-19T06:30:14.059+0000 7f53b34dc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:30:14.396 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:14 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1[39135]: 2024-09-19T06:30:14.059+0000 7f53b34dc640 -1 osd.1 128 *** Got signal Terminated *** 2024-09-19T06:30:14.396 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:14 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1[39135]: 2024-09-19T06:30:14.059+0000 7f53b34dc640 -1 osd.1 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:30:19.752 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:19 smithi049 podman[75718]: 2024-09-19 06:30:19.440499843 +0000 UTC m=+5.481814435 container died 644ad51420e3ab60b610ea0d1298d6b1473a2ccc48e6c6bccf94f7b5860bf9ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T06:30:20.146 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:19 smithi049 podman[75718]: 2024-09-19 06:30:19.867413568 +0000 UTC m=+5.908728146 container cleanup 644ad51420e3ab60b610ea0d1298d6b1473a2ccc48e6c6bccf94f7b5860bf9ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:30:20.146 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:19 smithi049 bash[75718]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1 2024-09-19T06:30:20.636 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:20 smithi049 podman[75734]: 2024-09-19 06:30:20.202080023 +0000 UTC m=+0.763656815 container remove 644ad51420e3ab60b610ea0d1298d6b1473a2ccc48e6c6bccf94f7b5860bf9ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T06:30:20.896 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:20 smithi049 podman[75823]: 2024-09-19 06:30:20.636172024 +0000 UTC m=+0.107663370 container create f2df2b322bd9dd3125a21a9965d72b8126a435409d56b96bba69a6f5ccf90099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T06:30:20.896 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:20 smithi049 podman[75823]: 2024-09-19 06:30:20.544752589 +0000 UTC m=+0.016243950 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:30:20.896 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:20 smithi049 podman[75823]: 2024-09-19 06:30:20.830769263 +0000 UTC m=+0.302260611 container init f2df2b322bd9dd3125a21a9965d72b8126a435409d56b96bba69a6f5ccf90099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T06:30:20.896 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:20 smithi049 podman[75823]: 2024-09-19 06:30:20.834620552 +0000 UTC m=+0.306111904 container start f2df2b322bd9dd3125a21a9965d72b8126a435409d56b96bba69a6f5ccf90099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T06:30:21.190 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:20 smithi049 podman[75823]: 2024-09-19 06:30:20.90515164 +0000 UTC m=+0.376642996 container attach f2df2b322bd9dd3125a21a9965d72b8126a435409d56b96bba69a6f5ccf90099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T06:30:21.472 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 19 06:30:21 smithi049 podman[75823]: 2024-09-19 06:30:21.189865502 +0000 UTC m=+0.661356856 container died f2df2b322bd9dd3125a21a9965d72b8126a435409d56b96bba69a6f5ccf90099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:30:21.818 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T06:30:21.858 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:30:21.859 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T06:30:21.859 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T06:30:21.859 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.2 2024-09-19T06:30:22.178 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:21 smithi157 systemd[1]: Stopping Ceph osd.2 for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:30:22.551 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:22 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2[40332]: 2024-09-19T06:30:22.176+0000 7f00bf660640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:30:22.551 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:22 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2[40332]: 2024-09-19T06:30:22.176+0000 7f00bf660640 -1 osd.2 128 *** Got signal Terminated *** 2024-09-19T06:30:22.551 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:22 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2[40332]: 2024-09-19T06:30:22.176+0000 7f00bf660640 -1 osd.2 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:30:27.854 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:27 smithi157 podman[75059]: 2024-09-19 06:30:27.587715264 +0000 UTC m=+5.532919298 container died 7400c3cdf67f8788b8b8c6e7d8d2c34d64d774f8aceeda49db94dbef76357108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T06:30:28.237 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:27 smithi157 podman[75059]: 2024-09-19 06:30:27.952010753 +0000 UTC m=+5.897214778 container cleanup 7400c3cdf67f8788b8b8c6e7d8d2c34d64d774f8aceeda49db94dbef76357108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T06:30:28.237 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:27 smithi157 bash[75059]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2 2024-09-19T06:30:28.545 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:28 smithi157 podman[75075]: 2024-09-19 06:30:28.236767864 +0000 UTC m=+0.645258795 container remove 7400c3cdf67f8788b8b8c6e7d8d2c34d64d774f8aceeda49db94dbef76357108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T06:30:28.799 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:28 smithi157 podman[75161]: 2024-09-19 06:30:28.447550386 +0000 UTC m=+0.016060947 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:30:28.799 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:28 smithi157 podman[75161]: 2024-09-19 06:30:28.546025741 +0000 UTC m=+0.114536335 container create 81980e803c6d87b81fbcb7cf50c7d18677d6484e42f9cf7688d00f4f9c07cdc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T06:30:28.800 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:28 smithi157 podman[75161]: 2024-09-19 06:30:28.798645514 +0000 UTC m=+0.367156071 container init 81980e803c6d87b81fbcb7cf50c7d18677d6484e42f9cf7688d00f4f9c07cdc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:30:29.051 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:28 smithi157 podman[75161]: 2024-09-19 06:30:28.802185295 +0000 UTC m=+0.370695848 container start 81980e803c6d87b81fbcb7cf50c7d18677d6484e42f9cf7688d00f4f9c07cdc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T06:30:29.051 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:28 smithi157 podman[75161]: 2024-09-19 06:30:28.88194607 +0000 UTC m=+0.450456635 container attach 81980e803c6d87b81fbcb7cf50c7d18677d6484e42f9cf7688d00f4f9c07cdc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T06:30:29.427 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 06:30:29 smithi157 podman[75161]: 2024-09-19 06:30:29.158752605 +0000 UTC m=+0.727263158 container died 81980e803c6d87b81fbcb7cf50c7d18677d6484e42f9cf7688d00f4f9c07cdc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-09-19T06:30:29.803 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T06:30:29.843 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:30:29.843 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T06:30:29.843 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T06:30:29.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.3 2024-09-19T06:30:30.129 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:29 smithi005 systemd[1]: Stopping Ceph osd.3 for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:30:30.629 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:30 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3[48677]: 2024-09-19T06:30:30.163+0000 7f92b7663640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:30:30.629 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:30 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3[48677]: 2024-09-19T06:30:30.163+0000 7f92b7663640 -1 osd.3 128 *** Got signal Terminated *** 2024-09-19T06:30:30.629 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:30 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3[48677]: 2024-09-19T06:30:30.163+0000 7f92b7663640 -1 osd.3 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:30:32.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:32 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:32.184+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:32.776 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:32 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:32.331+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:33.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:32 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:32.774+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:33.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:33 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:33.177+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:33.379 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:32 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:32.990+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:33.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:33 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:33.326+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:33.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:33 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:33.137+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:33.879 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:33 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3[48677]: 2024-09-19T06:30:33.515+0000 7f92b3c68640 -1 osd.3 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.381119+0000 front 2024-09-19T06:30:10.381080+0000 (oldest deadline 2024-09-19T06:30:33.281213+0000) 2024-09-19T06:30:34.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:33 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:33.756+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:34.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:33 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:33.856+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:34.379 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:34 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:34.017+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:34.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:34 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:34.209+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:34.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:34 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:34.125+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:34.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:34 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:34.337+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:34.880 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:34 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3[48677]: 2024-09-19T06:30:34.542+0000 7f92b3c68640 -1 osd.3 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.381119+0000 front 2024-09-19T06:30:10.381080+0000 (oldest deadline 2024-09-19T06:30:33.281213+0000) 2024-09-19T06:30:35.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:34 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:34.724+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:35.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:34 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:34.880+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:35.340 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:35 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:35.087+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:35.379 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:34 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:34.982+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:35.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:35 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:35.220+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:35.757 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:35 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:35.339+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:35.973 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:35 smithi005 podman[138270]: 2024-09-19 06:30:35.7161985 +0000 UTC m=+5.672960725 container died 957a727441011c01077908b222e54246a97be7cb6099cfcb08b2b634a175a97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T06:30:35.973 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:35 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:35.944+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:36.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:35 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:35.756+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:36.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:35 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:35.831+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:36.261 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:36 smithi005 podman[138270]: 2024-09-19 06:30:36.130028227 +0000 UTC m=+6.086790512 container cleanup 957a727441011c01077908b222e54246a97be7cb6099cfcb08b2b634a175a97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T06:30:36.261 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:36 smithi005 bash[138270]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3 2024-09-19T06:30:36.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:36 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:36.297+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:36.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:36 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:36.114+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:36.629 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:36 smithi005 podman[138285]: 2024-09-19 06:30:36.348213176 +0000 UTC m=+0.629809684 container remove 957a727441011c01077908b222e54246a97be7cb6099cfcb08b2b634a175a97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T06:30:36.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:36 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:36.259+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:37.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:36 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:36.746+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:37.118 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:36 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:36.900+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:37.119 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:36 smithi005 podman[138372]: 2024-09-19 06:30:36.798935351 +0000 UTC m=+0.114526657 container create e5f7568bb47d06cec1fe719dbfa6516ffe00992cbd2559220b7de394165f4e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:30:37.119 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:36 smithi005 podman[138372]: 2024-09-19 06:30:36.704060254 +0000 UTC m=+0.019651575 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:30:37.119 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:37 smithi005 podman[138372]: 2024-09-19 06:30:37.018337413 +0000 UTC m=+0.333928730 container init e5f7568bb47d06cec1fe719dbfa6516ffe00992cbd2559220b7de394165f4e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T06:30:37.119 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:37 smithi005 podman[138372]: 2024-09-19 06:30:37.021752711 +0000 UTC m=+0.337344018 container start e5f7568bb47d06cec1fe719dbfa6516ffe00992cbd2559220b7de394165f4e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:30:37.119 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:37 smithi005 podman[138372]: 2024-09-19 06:30:37.118207156 +0000 UTC m=+0.433798478 container attach e5f7568bb47d06cec1fe719dbfa6516ffe00992cbd2559220b7de394165f4e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2) 2024-09-19T06:30:37.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:36 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:36.787+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:37.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:37.229+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:37.379 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:37 smithi005 podman[138372]: 2024-09-19 06:30:37.220792939 +0000 UTC m=+0.536384257 container died e5f7568bb47d06cec1fe719dbfa6516ffe00992cbd2559220b7de394165f4e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T06:30:37.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:37.249+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:37.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:37.109+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:37.766 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 19 06:30:37 smithi005 podman[138396]: 2024-09-19 06:30:37.746984049 +0000 UTC m=+0.516955244 container remove e5f7568bb47d06cec1fe719dbfa6516ffe00992cbd2559220b7de394165f4e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T06:30:37.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T06:30:37.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:30:37.816 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T06:30:37.816 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T06:30:37.816 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.4 2024-09-19T06:30:38.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:37.773+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:38.129 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:37 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:37.917+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:38.137 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:37 smithi049 systemd[1]: Stopping Ceph osd.4 for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:30:38.137 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:37 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:37.771+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:38.396 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:38 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4[46049]: 2024-09-19T06:30:38.135+0000 7f0240652640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:30:38.396 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:38 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4[46049]: 2024-09-19T06:30:38.135+0000 7f0240652640 -1 osd.4 128 *** Got signal Terminated *** 2024-09-19T06:30:38.396 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:38 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4[46049]: 2024-09-19T06:30:38.135+0000 7f0240652640 -1 osd.4 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:30:38.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:38 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:38.226+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:38.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:38 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:38.083+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:38.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:38 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:38.190+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:39.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:38 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:38.811+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:39.193 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:38 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:38.821+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:39.194 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:39 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:39.035+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:39.229 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:38 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:38.935+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:39.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:39 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:39.192+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:39.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:39 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:39.228+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:40.146 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:40 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4[46049]: 2024-09-19T06:30:40.041+0000 7f023cc57640 -1 osd.4 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.215224+0000 front 2024-09-19T06:30:16.215281+0000 (oldest deadline 2024-09-19T06:30:39.715461+0000) 2024-09-19T06:30:40.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:39 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:39.793+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:40.185 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:39 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:39.894+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:40.200 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:39 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:39.825+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:40.201 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:40 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:40.018+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:40.471 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:40 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:40.199+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:40.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:40 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:40.183+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:40.645 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:40 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:40.377+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:41.037 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:40 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:40.777+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:41.146 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:41 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4[46049]: 2024-09-19T06:30:41.054+0000 7f023cc57640 -1 osd.4 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.215224+0000 front 2024-09-19T06:30:16.215281+0000 (oldest deadline 2024-09-19T06:30:39.715461+0000) 2024-09-19T06:30:41.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:40 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:40.808+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:41.189 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:40 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:40.909+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:41.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:41 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:41.179+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:41.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:41 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:41.179+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.999133+0000 front 2024-09-19T06:30:17.999259+0000 (oldest deadline 2024-09-19T06:30:40.899247+0000) 2024-09-19T06:30:41.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:41 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:41.036+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:41.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:41.188+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:41.775 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:41 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:41.415+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:42.048 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:41 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:41.728+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:42.075 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:41 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:41.773+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:42.225 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:41 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:41.907+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:42.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:42 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:42.047+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:42.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:42 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:42.131+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:42.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:42 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:42.131+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.999133+0000 front 2024-09-19T06:30:17.999259+0000 (oldest deadline 2024-09-19T06:30:40.899247+0000) 2024-09-19T06:30:42.396 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:42 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4[46049]: 2024-09-19T06:30:42.075+0000 7f023cc57640 -1 osd.4 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.215224+0000 front 2024-09-19T06:30:16.215281+0000 (oldest deadline 2024-09-19T06:30:39.715461+0000) 2024-09-19T06:30:42.513 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:42 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:42.224+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:42.821 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:42 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:42.425+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:43.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:42 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:42.730+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:43.092 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:42 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:42.819+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:43.128 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:42 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:42.873+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:43.396 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4[46049]: 2024-09-19T06:30:43.092+0000 7f023cc57640 -1 osd.4 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.215224+0000 front 2024-09-19T06:30:16.215281+0000 (oldest deadline 2024-09-19T06:30:39.715461+0000) 2024-09-19T06:30:43.519 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:43 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:43.234+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:43.519 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:43 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:43.234+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:43.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:43 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:43.089+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:43.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:43 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:43.124+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:43.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:43 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:43.124+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.999133+0000 front 2024-09-19T06:30:17.999259+0000 (oldest deadline 2024-09-19T06:30:40.899247+0000) 2024-09-19T06:30:43.845 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:43.417+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:43.845 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:43 smithi049 podman[75939]: 2024-09-19 06:30:43.515694565 +0000 UTC m=+5.483902054 container died 6cef94a9081d9332ddfa7e48d428473f30766d184463d230ce0cb62fa067a315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T06:30:44.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:43 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:43.721+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:44.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:43 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:43.843+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:44.146 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:43 smithi049 podman[75939]: 2024-09-19 06:30:43.935389567 +0000 UTC m=+5.903597051 container cleanup 6cef94a9081d9332ddfa7e48d428473f30766d184463d230ce0cb62fa067a315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T06:30:44.146 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:43 smithi049 bash[75939]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4 2024-09-19T06:30:44.233 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:43 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:43.888+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:44.396 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:44 smithi049 podman[75955]: 2024-09-19 06:30:44.144845497 +0000 UTC m=+0.629216025 container remove 6cef94a9081d9332ddfa7e48d428473f30766d184463d230ce0cb62fa067a315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T06:30:44.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:44 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:44.136+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:44.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:44 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:44.090+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:44.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:44 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:44.090+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.999133+0000 front 2024-09-19T06:30:17.999259+0000 (oldest deadline 2024-09-19T06:30:40.899247+0000) 2024-09-19T06:30:44.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:44 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:44.230+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:44.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:44 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:44.230+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:44.701 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:44 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:44.438+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:44.701 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:44 smithi049 podman[76041]: 2024-09-19 06:30:44.474912709 +0000 UTC m=+0.016219509 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:30:44.701 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:44 smithi049 podman[76041]: 2024-09-19 06:30:44.579014804 +0000 UTC m=+0.120321565 container create 08c4c81d7a37180c54824a8dea833c903f53c59257c173939c250e432df67894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T06:30:45.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:44 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:44.726+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:45.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:44 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:44.726+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:45.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:44 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:44.840+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:45.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:44 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:44.840+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:45.147 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:44 smithi049 podman[76041]: 2024-09-19 06:30:44.814622745 +0000 UTC m=+0.355929522 container init 08c4c81d7a37180c54824a8dea833c903f53c59257c173939c250e432df67894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T06:30:45.147 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:44 smithi049 podman[76041]: 2024-09-19 06:30:44.818081715 +0000 UTC m=+0.359388475 container start 08c4c81d7a37180c54824a8dea833c903f53c59257c173939c250e432df67894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T06:30:45.147 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:44 smithi049 podman[76041]: 2024-09-19 06:30:44.924340224 +0000 UTC m=+0.465646984 container attach 08c4c81d7a37180c54824a8dea833c903f53c59257c173939c250e432df67894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T06:30:45.147 INFO:journalctl@ceph.osd.4.smithi049.stdout:Sep 19 06:30:45 smithi049 podman[76041]: 2024-09-19 06:30:45.008537022 +0000 UTC m=+0.549843799 container died 08c4c81d7a37180c54824a8dea833c903f53c59257c173939c250e432df67894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T06:30:45.217 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:44 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:44.936+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:45.217 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:44 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:44.936+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:45.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:45 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:45.116+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:45.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:45 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:45.133+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:45.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:45 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:45.133+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.999133+0000 front 2024-09-19T06:30:17.999259+0000 (oldest deadline 2024-09-19T06:30:40.899247+0000) 2024-09-19T06:30:45.560 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:45 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:45.406+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:45.585 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T06:30:45.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:30:45.626 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T06:30:45.626 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T06:30:45.626 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.5 2024-09-19T06:30:45.628 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:45 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:45.215+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:45.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:45 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:45.215+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:45.922 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:45 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:45.746+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:45.923 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:45 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:45.746+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:45.923 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:45 smithi157 systemd[1]: Stopping Ceph osd.5 for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:30:45.923 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:45 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:45.921+0000 7fdd29c49640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:30:45.923 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:45 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:45.921+0000 7fdd29c49640 -1 osd.5 128 *** Got signal Terminated *** 2024-09-19T06:30:45.923 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:45 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:45.921+0000 7fdd29c49640 -1 osd.5 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:30:46.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:45 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:45.890+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:46.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:45 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:45.890+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:46.183 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:46 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:46.078+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:46.235 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:45 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:45.954+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:46.235 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:45 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:45.954+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:46.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:46 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:46.181+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:46.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:46 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:46.181+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.999133+0000 front 2024-09-19T06:30:17.999259+0000 (oldest deadline 2024-09-19T06:30:40.899247+0000) 2024-09-19T06:30:46.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:46 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:46.233+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:46.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:46 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:46.233+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:46.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:46 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:46.379+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:47.047 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:46 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:46.775+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:47.048 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:46 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:46.775+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:47.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:46 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:46.867+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:47.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:46 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:46.867+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:47.282 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:46 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:46.970+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:47.282 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:46 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:46.970+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:47.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:47 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:47.046+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:47.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:47 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:47.147+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:47.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:47 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:47.147+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.999133+0000 front 2024-09-19T06:30:17.999259+0000 (oldest deadline 2024-09-19T06:30:40.899247+0000) 2024-09-19T06:30:47.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:47 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:47.280+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:47.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:47 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:47.280+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:47.896 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:47 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:47.397+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:48.062 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:47 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:47.809+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:48.063 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:47 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:47.809+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:48.355 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:47 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:47.916+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:48.355 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:47 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:47.916+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:48.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:48 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:48.232+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:48.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:48 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:48.232+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:48.379 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:48 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:48.001+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:48.379 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:48 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:48.001+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:48.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:48 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:48.062+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:48.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:48 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:48.171+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:48.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:48 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:48.171+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.999133+0000 front 2024-09-19T06:30:17.999259+0000 (oldest deadline 2024-09-19T06:30:40.899247+0000) 2024-09-19T06:30:48.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:48 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:48.354+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:49.021 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:48 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:48.767+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:49.021 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:48 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:48.767+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:49.021 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:48 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:48.767+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:30:49.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:48 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:48.883+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:49.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:48 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:48.883+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:49.237 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:48 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:48.970+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:49.237 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:48 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:48.970+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:49.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:49 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:49.020+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:49.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:49 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:49.173+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:49.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:49 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:49.173+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.999133+0000 front 2024-09-19T06:30:17.999259+0000 (oldest deadline 2024-09-19T06:30:40.899247+0000) 2024-09-19T06:30:49.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:49 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:49.173+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.800189+0000 front 2024-09-19T06:30:26.800168+0000 (oldest deadline 2024-09-19T06:30:49.100318+0000) 2024-09-19T06:30:49.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:49 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:49.236+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:49.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:49 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:49.236+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:49.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:49 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:49.236+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:30:49.645 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:49 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:49.316+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:49.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:49 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:49.316+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:30:50.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:49 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:49.749+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:50.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:49 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:49.749+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:50.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:49 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:49.749+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:30:50.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:49 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:49.852+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:50.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:49 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:49.852+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:50.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:50 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:50.196+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:50.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:50 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:50.196+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:50.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:50 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:50.196+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:30:50.379 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:50 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:50.008+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:50.379 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:50 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:50.008+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:50.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:50.057+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:50.551 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:50.175+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.397934+0000 front 2024-09-19T06:30:10.398083+0000 (oldest deadline 2024-09-19T06:30:32.097981+0000) 2024-09-19T06:30:50.552 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:50.175+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.999133+0000 front 2024-09-19T06:30:17.999259+0000 (oldest deadline 2024-09-19T06:30:40.899247+0000) 2024-09-19T06:30:50.552 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5[47242]: 2024-09-19T06:30:50.175+0000 7fdd2624e640 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.800189+0000 front 2024-09-19T06:30:26.800168+0000 (oldest deadline 2024-09-19T06:30:49.100318+0000) 2024-09-19T06:30:50.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:50.328+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:50.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:50.328+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:30:51.048 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:50.762+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:51.049 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:50.762+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:51.049 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:50 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:50.762+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:30:51.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:50.863+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:51.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:50.863+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:51.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:50 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:50.863+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:30:51.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:51 smithi157 podman[75274]: 2024-09-19 06:30:51.294281891 +0000 UTC m=+5.477482547 container died 821df6f682be2244d3ddd5102dc6f7d77ae3c459224bf2907431442ad8c1bdc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T06:30:51.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:51 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:51.047+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:51.302 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:51 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:51.047+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:30:51.378 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:51 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:51.243+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:51.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:51 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:51.243+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:51.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:51 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:51.243+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:30:51.379 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:51 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:51.032+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:51.379 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:51 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:51.032+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:51.379 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:51 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:51.032+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:25.157700+0000 front 2024-09-19T06:30:25.157830+0000 (oldest deadline 2024-09-19T06:30:50.958438+0000) 2024-09-19T06:30:51.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:51 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:51.352+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:51.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:51 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:51.352+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:30:51.907 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:51 smithi157 podman[75274]: 2024-09-19 06:30:51.687871187 +0000 UTC m=+5.871071833 container cleanup 821df6f682be2244d3ddd5102dc6f7d77ae3c459224bf2907431442ad8c1bdc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T06:30:51.907 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:51 smithi157 bash[75274]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5 2024-09-19T06:30:51.907 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:51 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:51.746+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:51.907 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:51 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:51.746+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:51.907 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:51 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:51.746+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:30:52.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:51 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:51.828+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:52.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:51 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:51.828+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:52.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:51 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:51.828+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:30:52.295 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:51 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:51.994+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:52.295 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:51 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:51.994+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:52.295 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:51 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:51.994+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:25.157700+0000 front 2024-09-19T06:30:25.157830+0000 (oldest deadline 2024-09-19T06:30:50.958438+0000) 2024-09-19T06:30:52.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:51 smithi157 podman[75291]: 2024-09-19 06:30:51.905843821 +0000 UTC m=+0.607190511 container remove 821df6f682be2244d3ddd5102dc6f7d77ae3c459224bf2907431442ad8c1bdc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T06:30:52.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:52 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:51.999+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:52.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:52 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:51.999+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:30:52.602 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:52 smithi157 podman[75383]: 2024-09-19 06:30:52.214450977 +0000 UTC m=+0.015953213 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:30:52.602 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:52 smithi157 podman[75383]: 2024-09-19 06:30:52.314999893 +0000 UTC m=+0.116502114 container create 89888f7bb62096052d203b273b8fac4e7091acff7dc7af9b164cbf207157db22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T06:30:52.602 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:52 smithi157 podman[75383]: 2024-09-19 06:30:52.525569117 +0000 UTC m=+0.327071350 container init 89888f7bb62096052d203b273b8fac4e7091acff7dc7af9b164cbf207157db22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T06:30:52.602 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:52 smithi157 podman[75383]: 2024-09-19 06:30:52.528973142 +0000 UTC m=+0.330475364 container start 89888f7bb62096052d203b273b8fac4e7091acff7dc7af9b164cbf207157db22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T06:30:52.602 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:52 smithi157 podman[75383]: 2024-09-19 06:30:52.60054423 +0000 UTC m=+0.402046471 container attach 89888f7bb62096052d203b273b8fac4e7091acff7dc7af9b164cbf207157db22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-19T06:30:52.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:52 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:52.293+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:52.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:52 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:52.293+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:52.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:52 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:52.293+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:30:52.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:52 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:52.314+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:52.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:52 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:52.314+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:30:52.989 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 06:30:52 smithi157 podman[75383]: 2024-09-19 06:30:52.713230938 +0000 UTC m=+0.514733176 container died 89888f7bb62096052d203b273b8fac4e7091acff7dc7af9b164cbf207157db22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T06:30:52.990 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:52 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:52.790+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:52.990 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:52 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:52.790+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:52.990 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:52 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:52.790+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:30:53.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:52 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:52.830+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:53.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:52 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:52.830+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:53.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:52 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:52.830+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:30:53.255 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:52 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:52.988+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:53.255 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:52 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:52.988+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:30:53.276 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T06:30:53.292 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:52 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:52.969+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:53.292 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:52 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:52.969+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:53.292 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:52 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:52.969+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:25.157700+0000 front 2024-09-19T06:30:25.157830+0000 (oldest deadline 2024-09-19T06:30:50.958438+0000) 2024-09-19T06:30:53.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:30:53.315 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T06:30:53.315 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T06:30:53.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.6 2024-09-19T06:30:53.584 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:53 smithi005 systemd[1]: Stopping Ceph osd.6 for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:30:53.584 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:53 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:53.290+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:53.585 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:53 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:53.290+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:53.585 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:53 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:53.290+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:30:53.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:53 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:53.335+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:53.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:53 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:53.335+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:30:53.879 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:53 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:53.649+0000 7fa346927640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:30:53.879 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:53 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:53.649+0000 7fa346927640 -1 osd.6 128 *** Got signal Terminated *** 2024-09-19T06:30:53.879 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:53 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:53.649+0000 7fa346927640 -1 osd.6 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:30:54.051 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:53 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:53.969+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:54.051 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:53 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:53.969+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:30:54.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:53 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:53.799+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:54.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:53 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:53.799+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:54.052 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:53 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:53.799+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:30:54.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:53 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:53.792+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:54.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:53 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:53.792+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:54.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:53 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:53.792+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:30:54.300 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:53 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:53.981+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:54.300 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:53 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:53.981+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:54.300 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:53 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:53.981+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:25.157700+0000 front 2024-09-19T06:30:25.157830+0000 (oldest deadline 2024-09-19T06:30:50.958438+0000) 2024-09-19T06:30:54.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:54 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:54.298+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:54.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:54 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:54.298+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:54.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:54 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:54.299+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:30:54.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:54 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:54.330+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:54.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:54 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:54.330+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:30:55.051 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:54 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:54.988+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:55.051 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:54 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:54.988+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:30:55.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:54 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:54.752+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:55.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:54 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:54.752+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:55.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:54 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:54.752+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:30:55.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:54 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:54.755+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:55.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:54 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:54.755+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:55.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:54 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:54.755+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:30:55.280 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:55 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:55.006+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:55.280 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:55 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:55.006+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:55.280 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:55 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:55.006+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:25.157700+0000 front 2024-09-19T06:30:25.157830+0000 (oldest deadline 2024-09-19T06:30:50.958438+0000) 2024-09-19T06:30:55.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:55 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:55.278+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:55.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:55 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:55.278+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:55.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:55 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:55.278+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:30:55.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:55 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:55.324+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:55.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:55 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:55.324+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:30:56.014 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:55 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:55.724+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:56.014 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:55 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:55.724+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:56.014 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:55 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:55.724+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:30:56.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:55 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:55.756+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:56.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:55 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:55.756+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:56.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:55 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:55.756+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:30:56.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:56 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:56.013+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:56.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:56 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:56.013+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:30:56.318 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:56 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:56.002+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:56.318 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:56 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:56.002+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:56.318 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:56 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:56.002+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:25.157700+0000 front 2024-09-19T06:30:25.157830+0000 (oldest deadline 2024-09-19T06:30:50.958438+0000) 2024-09-19T06:30:56.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:56 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:56.316+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:56.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:56 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:56.316+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:56.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:56 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:56.316+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:30:56.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:56 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:56.290+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:56.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:56 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:56.290+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:30:57.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:56 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:56.706+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:57.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:56 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:56.706+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:57.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:56 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:56.706+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:30:57.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:56 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:56.773+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:57.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:56 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:56.773+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:57.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:56 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:56.773+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:30:57.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:56 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:56.773+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:30:57.324 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:57 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:57.050+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:57.325 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:57 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:57.050+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:30:57.331 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:56 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:56.986+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:57.331 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:56 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:56.986+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:57.331 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:56 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:56.986+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:25.157700+0000 front 2024-09-19T06:30:25.157830+0000 (oldest deadline 2024-09-19T06:30:50.958438+0000) 2024-09-19T06:30:57.331 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:56 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:56.986+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.059800+0000 front 2024-09-19T06:30:32.059827+0000 (oldest deadline 2024-09-19T06:30:56.760013+0000) 2024-09-19T06:30:57.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:57 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:57.330+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:57.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:57 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:57.330+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:57.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:57 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:57.330+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:30:57.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:57 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:57.323+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:57.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:57 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:57.323+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:30:58.027 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:58 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:58.026+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:58.027 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:58 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:58.026+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:30:58.027 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:57 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:57.713+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:58.028 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:57 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:57.713+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:58.028 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:57 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:57.713+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:30:58.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:57 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:57.750+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:58.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:57 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:57.750+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:58.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:57 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:57.750+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:30:58.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:57 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:57.750+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:30:58.311 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:57 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:57.937+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:08.854011+0000 front 2024-09-19T06:30:08.854069+0000 (oldest deadline 2024-09-19T06:30:32.954292+0000) 2024-09-19T06:30:58.311 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:57 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:57.937+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.756498+0000 front 2024-09-19T06:30:18.756446+0000 (oldest deadline 2024-09-19T06:30:44.056467+0000) 2024-09-19T06:30:58.311 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:57 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:57.937+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:25.157700+0000 front 2024-09-19T06:30:25.157830+0000 (oldest deadline 2024-09-19T06:30:50.958438+0000) 2024-09-19T06:30:58.311 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:57 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6[55640]: 2024-09-19T06:30:57.937+0000 7fa342f2c640 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.059800+0000 front 2024-09-19T06:30:32.059827+0000 (oldest deadline 2024-09-19T06:30:56.760013+0000) 2024-09-19T06:30:58.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:58 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:58.309+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:58.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:58 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:58.309+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:58.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:58 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:58.309+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:30:58.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:58 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:58.341+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:58.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:58 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:58.341+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:30:59.042 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:58 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:58.730+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:30:59.043 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:58 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:58.730+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:30:59.043 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:58 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:58.730+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:30:59.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:58 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:58.743+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:30:59.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:58 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:58.743+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:30:59.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:58 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:58.743+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:30:59.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:58 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:58.743+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:30:59.294 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:59 smithi005 podman[138487]: 2024-09-19 06:30:59.033931213 +0000 UTC m=+5.510142455 container died e87b3dbe7a4f7a66a0b3e5038fa3339c53040e83e1b901f6abfdcfff12d148c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-19T06:30:59.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:59 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:59.042+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:30:59.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:59 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:59.042+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:30:59.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:30:59 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:30:59.042+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:30:59.629 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:59 smithi005 podman[138487]: 2024-09-19 06:30:59.383482253 +0000 UTC m=+5.859693493 container cleanup e87b3dbe7a4f7a66a0b3e5038fa3339c53040e83e1b901f6abfdcfff12d148c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T06:30:59.629 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:59 smithi005 bash[138487]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6 2024-09-19T06:30:59.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:59 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:59.325+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:30:59.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:59 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:59.325+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:30:59.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:30:59 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:30:59.325+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:30:59.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:59 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:59.310+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:30:59.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:59 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:59.310+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:30:59.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:30:59 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:30:59.310+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.542365+0000 front 2024-09-19T06:30:32.542422+0000 (oldest deadline 2024-09-19T06:30:58.343111+0000) 2024-09-19T06:31:00.010 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:30:59 smithi005 podman[138502]: 2024-09-19 06:30:59.668775221 +0000 UTC m=+0.634968389 container remove e87b3dbe7a4f7a66a0b3e5038fa3339c53040e83e1b901f6abfdcfff12d148c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:31:00.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:59 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:59.707+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:00.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:59 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:59.707+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:00.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:30:59 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:30:59.707+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:00.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:59 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:59.786+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:00.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:59 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:59.786+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:00.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:59 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:59.786+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:00.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:30:59 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:30:59.786+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:00.305 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:31:00 smithi005 podman[138591]: 2024-09-19 06:30:59.910054048 +0000 UTC m=+0.016381884 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:31:00.305 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:31:00 smithi005 podman[138591]: 2024-09-19 06:31:00.018988998 +0000 UTC m=+0.125316868 container create 7fd8a889b494f02bfa6a21699c54f80a09206d57d252b5fb358eca3e7c5a4b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T06:31:00.306 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:31:00 smithi005 podman[138591]: 2024-09-19 06:31:00.213027228 +0000 UTC m=+0.319355065 container init 7fd8a889b494f02bfa6a21699c54f80a09206d57d252b5fb358eca3e7c5a4b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T06:31:00.306 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:31:00 smithi005 podman[138591]: 2024-09-19 06:31:00.216541707 +0000 UTC m=+0.322869537 container start 7fd8a889b494f02bfa6a21699c54f80a09206d57d252b5fb358eca3e7c5a4b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T06:31:00.306 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:31:00 smithi005 podman[138591]: 2024-09-19 06:31:00.304875748 +0000 UTC m=+0.411203582 container attach 7fd8a889b494f02bfa6a21699c54f80a09206d57d252b5fb358eca3e7c5a4b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:31:00.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:00 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:00.060+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:00.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:00 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:00.060+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:00.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:00 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:00.060+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:00.629 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 19 06:31:00 smithi005 podman[138591]: 2024-09-19 06:31:00.415754356 +0000 UTC m=+0.522082192 container died 7fd8a889b494f02bfa6a21699c54f80a09206d57d252b5fb358eca3e7c5a4b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T06:31:00.630 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:00.346+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:00.630 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:00.346+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:00.630 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:00 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:00.346+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:00.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:00 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:00.270+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:31:00.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:00 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:00.270+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:31:00.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:00 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:00.270+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.542365+0000 front 2024-09-19T06:30:32.542422+0000 (oldest deadline 2024-09-19T06:30:58.343111+0000) 2024-09-19T06:31:01.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T06:31:01.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:00 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:00.664+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:01.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:00 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:00.664+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:01.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:00 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:00.664+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:01.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:31:01.077 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T06:31:01.077 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T06:31:01.077 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.7 2024-09-19T06:31:01.106 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:00 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:00.760+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:01.106 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:00 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:00.760+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:01.106 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:00 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:00.760+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:01.106 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:00 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:00.760+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:01.387 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:01 smithi049 systemd[1]: Stopping Ceph osd.7 for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:31:01.387 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:01 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:01.276+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:31:01.387 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:01 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:01.276+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:31:01.387 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:01 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:01.276+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.542365+0000 front 2024-09-19T06:30:32.542422+0000 (oldest deadline 2024-09-19T06:30:58.343111+0000) 2024-09-19T06:31:01.388 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:01 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:01.385+0000 7fb38aff3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:31:01.388 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:01 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:01.385+0000 7fb38aff3640 -1 osd.7 128 *** Got signal Terminated *** 2024-09-19T06:31:01.388 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:01 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:01.385+0000 7fb38aff3640 -1 osd.7 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:31:01.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:01 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:01.054+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:01.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:01 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:01.054+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:01.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:01 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:01.054+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:01.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:01 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:01.375+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:01.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:01 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:01.375+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:01.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:01 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:01.375+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:02.018 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:01 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:01.663+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:02.018 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:01 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:01.663+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:02.018 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:01 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:01.663+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:02.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:01 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:01.737+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:02.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:01 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:01.737+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:02.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:01 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:01.737+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:02.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:01 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:01.737+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:02.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:02 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:02.017+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:02.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:02 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:02.017+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:02.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:02 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:02.017+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:02.433 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:02 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:02.235+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:31:02.433 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:02 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:02.235+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:31:02.433 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:02 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:02.235+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.542365+0000 front 2024-09-19T06:30:32.542422+0000 (oldest deadline 2024-09-19T06:30:58.343111+0000) 2024-09-19T06:31:02.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:02 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:02.396+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:02.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:02 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:02.396+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:02.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:02 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:02.396+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:02.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:02 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:02.396+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:03.008 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:02 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:02.645+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:03.009 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:02 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:02.645+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:03.009 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:02 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:02.645+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:03.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:02 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:02.755+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:03.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:02 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:02.755+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:03.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:02 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:02.755+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:03.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:02 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:02.755+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:03.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:03 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:03.006+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:03.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:03 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:03.006+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:03.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:03 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:03.006+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:03.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:03 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:03.278+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:31:03.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:03 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:03.278+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:31:03.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:03 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:03.278+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.542365+0000 front 2024-09-19T06:30:32.542422+0000 (oldest deadline 2024-09-19T06:30:58.343111+0000) 2024-09-19T06:31:03.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:03 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:03.422+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:03.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:03 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:03.422+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:03.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:03 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:03.422+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:03.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:03 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:03.422+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:04.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:03 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:03.603+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:04.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:03 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:03.603+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:04.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:03 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:03.603+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:04.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:03 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:03.799+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:04.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:03 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:03.799+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:04.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:03 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:03.799+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:04.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:03 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:03.799+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:04.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:04 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:04.050+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:04.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:04 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:04.050+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:04.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:04 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:04.050+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:04.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:04 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:04.269+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:31:04.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:04 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:04.269+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:31:04.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:04 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:04.269+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.542365+0000 front 2024-09-19T06:30:32.542422+0000 (oldest deadline 2024-09-19T06:30:58.343111+0000) 2024-09-19T06:31:04.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:04 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:04.269+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.644170+0000 front 2024-09-19T06:30:40.644110+0000 (oldest deadline 2024-09-19T06:31:03.544432+0000) 2024-09-19T06:31:04.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:04 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:04.423+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:04.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:04 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:04.423+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:04.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:04 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:04.423+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:04.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:04 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:04.423+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:04.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:04 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:04.423+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:05.026 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:04 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:04.576+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:05.026 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:04 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:04.576+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:05.027 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:04 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:04.576+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:05.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:04 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:04.753+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:05.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:04 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:04.753+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:05.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:04 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:04.753+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:05.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:04 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:04.753+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:05.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:04 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:04.753+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:05.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:05 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:05.025+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:05.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:05 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:05.025+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:05.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:05 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:05.025+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:05.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:05 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:05.375+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:05.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:05 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:05.375+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:05.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:05 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:05.375+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:05.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:05 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:05.375+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:05.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:05 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:05.375+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:05.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:05 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:05.276+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:31:05.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:05 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:05.276+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:31:05.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:05 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:05.276+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.542365+0000 front 2024-09-19T06:30:32.542422+0000 (oldest deadline 2024-09-19T06:30:58.343111+0000) 2024-09-19T06:31:05.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:05 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:05.276+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.644170+0000 front 2024-09-19T06:30:40.644110+0000 (oldest deadline 2024-09-19T06:31:03.544432+0000) 2024-09-19T06:31:05.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:05 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:05.529+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:05.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:05 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:05.529+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:05.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:05 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:05.529+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:06.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:05 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:05.776+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:06.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:05 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:05.776+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:06.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:05 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:05.776+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:06.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:05 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:05.776+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:06.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:05 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:05.776+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:06.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:06 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:06.046+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:06.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:06 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:06.046+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:06.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:06 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:06.046+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:06.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:06 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:06.046+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.066355+0000 front 2024-09-19T06:30:43.066260+0000 (oldest deadline 2024-09-19T06:31:05.966170+0000) 2024-09-19T06:31:06.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:06 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:06.351+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:06.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:06 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:06.351+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:06.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:06 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:06.351+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:06.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:06 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:06.351+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:06.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:06 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:06.351+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:06.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:06 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:06.280+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:17.538935+0000 front 2024-09-19T06:30:17.538965+0000 (oldest deadline 2024-09-19T06:30:39.839067+0000) 2024-09-19T06:31:06.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:06 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:06.280+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.341058+0000 front 2024-09-19T06:30:24.341029+0000 (oldest deadline 2024-09-19T06:30:48.441181+0000) 2024-09-19T06:31:06.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:06 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:06.280+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.542365+0000 front 2024-09-19T06:30:32.542422+0000 (oldest deadline 2024-09-19T06:30:58.343111+0000) 2024-09-19T06:31:06.646 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:06 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7[52992]: 2024-09-19T06:31:06.280+0000 7fb3875f8640 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.644170+0000 front 2024-09-19T06:30:40.644110+0000 (oldest deadline 2024-09-19T06:31:03.544432+0000) 2024-09-19T06:31:06.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:06 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:06.508+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:06.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:06 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:06.508+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:06.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:06 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:06.508+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:07.054 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:06 smithi049 podman[76146]: 2024-09-19 06:31:06.757816188 +0000 UTC m=+5.480916155 container stop 9ad847267e0cbc189911ea4a4975fc4c218f318a12807c5af864897728ee5b51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T06:31:07.054 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:06 smithi049 podman[76146]: 2024-09-19 06:31:06.783428321 +0000 UTC m=+5.506528294 container died 9ad847267e0cbc189911ea4a4975fc4c218f318a12807c5af864897728ee5b51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T06:31:07.055 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:06 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:06.761+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:07.055 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:06 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:06.761+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:07.055 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:06 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:06.761+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:07.055 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:06 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:06.761+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:07.055 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:06 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:06.761+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:07.362 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:07 smithi049 podman[76146]: 2024-09-19 06:31:07.160757692 +0000 UTC m=+5.883857657 container cleanup 9ad847267e0cbc189911ea4a4975fc4c218f318a12807c5af864897728ee5b51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:31:07.363 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:07 smithi049 bash[76146]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7 2024-09-19T06:31:07.543 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:07 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:07.090+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:07.544 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:07 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:07.090+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:07.544 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:07 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:07.090+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:07.544 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:07 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:07.090+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.066355+0000 front 2024-09-19T06:30:43.066260+0000 (oldest deadline 2024-09-19T06:31:05.966170+0000) 2024-09-19T06:31:07.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:07 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:07.350+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:07.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:07 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:07.350+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:07.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:07 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:07.350+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:07.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:07 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:07.350+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:07.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:07 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:07.350+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:07.645 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:07 smithi049 podman[76173]: 2024-09-19 06:31:07.362017382 +0000 UTC m=+0.596254000 container remove 9ad847267e0cbc189911ea4a4975fc4c218f318a12807c5af864897728ee5b51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:31:07.800 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:07 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:07.541+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:07.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:07 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:07.541+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:07.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:07 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:07.541+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:07.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:07 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:07.541+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:08.049 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:07 smithi049 podman[76261]: 2024-09-19 06:31:07.771028829 +0000 UTC m=+0.101931430 container create c71479de085b11aa69852bd5747192ed398f43e24e37689a43f54d84f0ab3b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T06:31:08.050 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:07 smithi049 podman[76261]: 2024-09-19 06:31:07.685571648 +0000 UTC m=+0.016474262 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:31:08.050 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:07 smithi049 podman[76261]: 2024-09-19 06:31:07.965203037 +0000 UTC m=+0.296105646 container init c71479de085b11aa69852bd5747192ed398f43e24e37689a43f54d84f0ab3b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T06:31:08.050 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:07 smithi049 podman[76261]: 2024-09-19 06:31:07.968504235 +0000 UTC m=+0.299406846 container start c71479de085b11aa69852bd5747192ed398f43e24e37689a43f54d84f0ab3b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:31:08.050 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:08 smithi049 podman[76261]: 2024-09-19 06:31:08.048510418 +0000 UTC m=+0.379413027 container attach c71479de085b11aa69852bd5747192ed398f43e24e37689a43f54d84f0ab3b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T06:31:08.050 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:07 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:07.760+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:08.050 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:07 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:07.760+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:08.050 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:07 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:07.760+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:08.050 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:07 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:07.760+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:08.050 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:07 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:07.760+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:08.396 INFO:journalctl@ceph.osd.7.smithi049.stdout:Sep 19 06:31:08 smithi049 podman[76261]: 2024-09-19 06:31:08.163706358 +0000 UTC m=+0.494608990 container died c71479de085b11aa69852bd5747192ed398f43e24e37689a43f54d84f0ab3b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T06:31:08.495 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:08 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:08.139+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:08.495 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:08 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:08.139+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:08.495 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:08 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:08.139+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:08.495 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:08 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:08.139+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.066355+0000 front 2024-09-19T06:30:43.066260+0000 (oldest deadline 2024-09-19T06:31:05.966170+0000) 2024-09-19T06:31:08.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:08 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:08.370+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:08.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:08 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:08.370+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:08.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:08 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:08.370+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:08.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:08 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:08.370+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:08.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:08 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:08.370+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:08.794 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:08 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:08.747+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:08.794 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:08 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:08.747+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:08.794 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:08 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:08.747+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:08.794 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:08 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:08.747+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:08.794 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:08 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:08.747+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:08.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:08 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:08.493+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:08.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:08 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:08.493+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:08.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:08 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:08.493+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:08.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:08 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:08.493+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:08.826 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T06:31:08.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:31:08.911 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T06:31:08.911 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T06:31:08.911 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.8 2024-09-19T06:31:09.225 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:08 smithi157 systemd[1]: Stopping Ceph osd.8 for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:31:09.225 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:09 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:09.141+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:09.225 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:09 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:09.141+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:09.225 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:09 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:09.141+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:09.225 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:09 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:09.141+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.066355+0000 front 2024-09-19T06:30:43.066260+0000 (oldest deadline 2024-09-19T06:31:05.966170+0000) 2024-09-19T06:31:09.225 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:09 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:09.222+0000 7fb6f1ef6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:31:09.225 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:09 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:09.222+0000 7fb6f1ef6640 -1 osd.8 128 *** Got signal Terminated *** 2024-09-19T06:31:09.225 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:09 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:09.222+0000 7fb6f1ef6640 -1 osd.8 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:31:09.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:09 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:09.351+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:09.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:09 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:09.351+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:09.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:09 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:09.351+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:09.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:09 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:09.351+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:09.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:09 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:09.351+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:09.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:09 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:09.538+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:09.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:09 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:09.538+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:09.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:09 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:09.538+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:09.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:09 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:09.538+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:10.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:09 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:09.718+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:10.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:09 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:09.718+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:10.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:09 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:09.718+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:10.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:09 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:09.718+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:10.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:09 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:09.718+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:10.494 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:10 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:10.107+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:10.494 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:10 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:10.107+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:10.494 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:10 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:10.107+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:10.494 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:10 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:10.107+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.066355+0000 front 2024-09-19T06:30:43.066260+0000 (oldest deadline 2024-09-19T06:31:05.966170+0000) 2024-09-19T06:31:10.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:10 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:10.344+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:10.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:10 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:10.344+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:10.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:10 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:10.344+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:10.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:10 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:10.344+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:10.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:10 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:10.344+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:10.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:10 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:10.491+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:10.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:10 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:10.491+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:10.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:10 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:10.491+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:10.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:10 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:10.491+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:11.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:10 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:10.723+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:11.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:10 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:10.723+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:11.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:10 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:10.723+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:11.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:10 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:10.723+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:11.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:10 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:10.723+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:11.533 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:11 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:11.097+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:11.533 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:11 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:11.097+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:11.533 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:11 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:11.097+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:11.533 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:11 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:11.097+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.066355+0000 front 2024-09-19T06:30:43.066260+0000 (oldest deadline 2024-09-19T06:31:05.966170+0000) 2024-09-19T06:31:11.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:11 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:11.355+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:11.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:11 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:11.355+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:11.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:11 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:11.355+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:11.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:11 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:11.355+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:11.629 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:11 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:11.355+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:11.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:11 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:11.531+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:11.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:11 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:11.531+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:11.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:11 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:11.531+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:11.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:11 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:11.531+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:12.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:11 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:11.756+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:12.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:11 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:11.756+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:12.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:11 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:11.756+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:12.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:11 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:11.756+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:12.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:11 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:11.756+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:12.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:12 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:12.049+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:12.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:12 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:12.049+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:12.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:12 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:12.049+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:12.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:12 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:12.049+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.066355+0000 front 2024-09-19T06:30:43.066260+0000 (oldest deadline 2024-09-19T06:31:05.966170+0000) 2024-09-19T06:31:12.656 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:12 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:12.382+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:12.657 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:12 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:12.382+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:12.657 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:12 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:12.382+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:12.657 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:12 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:12.382+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:12.657 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:12 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:12.382+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:12.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:12 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:12.520+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:12.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:12 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:12.520+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:12.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:12 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:12.520+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:12.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:12 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:12.520+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:13.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:12 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:12.728+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:13.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:12 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:12.728+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:13.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:12 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:12.728+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:13.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:12 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:12.728+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:13.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:12 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:12.728+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:13.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:13 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:13.022+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:13.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:13 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:13.022+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:13.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:13 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:13.022+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:13.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:13 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:13.022+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.066355+0000 front 2024-09-19T06:30:43.066260+0000 (oldest deadline 2024-09-19T06:31:05.966170+0000) 2024-09-19T06:31:13.685 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:13.424+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:13.685 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:13.424+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:13.686 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:13.424+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:13.686 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:13.424+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:13.686 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:13 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:13.424+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:13.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:13 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:13.480+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:13.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:13 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:13.480+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:13.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:13 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:13.480+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:13.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:13 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:13.480+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:14.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:13 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:13.754+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:14.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:13 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:13.754+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:14.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:13 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:13.754+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:14.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:13 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:13.754+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:14.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:13 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:13.754+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:14.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:14 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:14.007+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.960258+0000 front 2024-09-19T06:30:10.960217+0000 (oldest deadline 2024-09-19T06:30:32.660229+0000) 2024-09-19T06:31:14.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:14 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:14.007+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:24.962837+0000 front 2024-09-19T06:30:24.962801+0000 (oldest deadline 2024-09-19T06:30:50.763753+0000) 2024-09-19T06:31:14.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:14 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:14.007+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:34.264932+0000 front 2024-09-19T06:30:34.264907+0000 (oldest deadline 2024-09-19T06:30:58.965050+0000) 2024-09-19T06:31:14.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:14 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8[53747]: 2024-09-19T06:31:14.007+0000 7fb6ea4f3640 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.066355+0000 front 2024-09-19T06:30:43.066260+0000 (oldest deadline 2024-09-19T06:31:05.966170+0000) 2024-09-19T06:31:14.801 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:14 smithi157 podman[75499]: 2024-09-19 06:31:14.624931581 +0000 UTC m=+5.511803186 container died 145bede32c23dc85b9f9292408e9a6ce1600dc9dffde9503cbb59fbba68620b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T06:31:14.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:14 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:14.523+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:14.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:14 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:14.523+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:14.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:14 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:14.523+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:14.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:14 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:14.523+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:14.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:14 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:14.473+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:14.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:14 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:14.473+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:14.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:14 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:14.473+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:14.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:14 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:14.473+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:14.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:14 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:14.473+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:15.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:14 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:14.780+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:15.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:14 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:14.780+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:15.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:14 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:14.780+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:15.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:14 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:14.780+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:15.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:14 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:14.780+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:15.267 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:15 smithi157 podman[75499]: 2024-09-19 06:31:15.056641438 +0000 UTC m=+5.943513034 container cleanup 145bede32c23dc85b9f9292408e9a6ce1600dc9dffde9503cbb59fbba68620b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T06:31:15.267 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:15 smithi157 bash[75499]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8 2024-09-19T06:31:15.553 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:15 smithi157 podman[75514]: 2024-09-19 06:31:15.266196624 +0000 UTC m=+0.633389398 container remove 145bede32c23dc85b9f9292408e9a6ce1600dc9dffde9503cbb59fbba68620b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T06:31:15.553 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:15.482+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:15.553 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:15.482+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:15.554 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:15.482+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:15.554 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:15.482+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:15.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:15.512+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:15.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:15.512+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:15.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:15.512+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:15.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:15.512+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:15.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:15.512+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:15.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:15 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:15.512+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:49.657734+0000 front 2024-09-19T06:30:49.657806+0000 (oldest deadline 2024-09-19T06:31:14.957611+0000) 2024-09-19T06:31:15.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:15 smithi157 podman[75603]: 2024-09-19 06:31:15.572955887 +0000 UTC m=+0.015694753 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:31:15.987 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:15 smithi157 podman[75603]: 2024-09-19 06:31:15.675439033 +0000 UTC m=+0.118177879 container create 9a720400fcee2472ddd7f634542301c5763df7060276eadcacd7274957e6fdcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T06:31:15.987 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:15 smithi157 podman[75603]: 2024-09-19 06:31:15.902739965 +0000 UTC m=+0.345478827 container init 9a720400fcee2472ddd7f634542301c5763df7060276eadcacd7274957e6fdcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T06:31:15.987 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:15 smithi157 podman[75603]: 2024-09-19 06:31:15.906087956 +0000 UTC m=+0.348826809 container start 9a720400fcee2472ddd7f634542301c5763df7060276eadcacd7274957e6fdcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T06:31:15.987 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:15 smithi157 podman[75603]: 2024-09-19 06:31:15.985936542 +0000 UTC m=+0.428675398 container attach 9a720400fcee2472ddd7f634542301c5763df7060276eadcacd7274957e6fdcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:31:16.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:15.781+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:16.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:15.781+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:16.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:15.781+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:16.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:15.781+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:16.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:15.781+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:16.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:15 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:15.781+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:16.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 06:31:16 smithi157 podman[75603]: 2024-09-19 06:31:16.093110421 +0000 UTC m=+0.535849291 container died 9a720400fcee2472ddd7f634542301c5763df7060276eadcacd7274957e6fdcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-8-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T06:31:16.682 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:16 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:16.484+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:16.682 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:16 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:16.484+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:16.682 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:16 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:16.484+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:16.682 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:16 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:16.484+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:16.682 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:16 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:16.484+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:16.702 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T06:31:16.741 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:31:16.741 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T06:31:16.741 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T06:31:16.742 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.9 2024-09-19T06:31:16.772 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:16.511+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:16.772 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:16.511+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:16.772 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:16.511+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:16.772 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:16.511+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:16.772 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:16.511+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:16.772 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:16 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:16.511+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:49.657734+0000 front 2024-09-19T06:30:49.657806+0000 (oldest deadline 2024-09-19T06:31:14.957611+0000) 2024-09-19T06:31:17.046 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:16 smithi005 systemd[1]: Stopping Ceph osd.9 for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:31:17.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:16 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:16.736+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:17.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:16 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:16.736+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:17.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:16 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:16.736+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:17.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:16 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:16.736+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:17.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:16 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:16.736+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:17.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:16 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:16.736+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:17.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:17 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:17.044+0000 7f83cc93d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:31:17.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:17 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:17.044+0000 7f83cc93d640 -1 osd.9 128 *** Got signal Terminated *** 2024-09-19T06:31:17.379 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:17 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:17.044+0000 7f83cc93d640 -1 osd.9 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:31:17.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:17 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:17.513+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:17.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:17 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:17.513+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:17.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:17 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:17.513+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:17.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:17 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:17.513+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:17.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:17 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:17.513+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:17.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:17 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:17.511+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:17.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:17 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:17.511+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:17.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:17 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:17.511+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:17.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:17 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:17.511+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:17.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:17 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:17.511+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:17.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:17 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:17.511+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:49.657734+0000 front 2024-09-19T06:30:49.657806+0000 (oldest deadline 2024-09-19T06:31:14.957611+0000) 2024-09-19T06:31:18.108 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:17 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:17.738+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:18.108 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:17 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:17.738+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:18.108 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:17 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:17.738+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:18.108 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:17 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:17.738+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:18.108 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:17 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:17.738+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:18.108 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:17 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:17.738+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:18.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:18 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:18.489+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:18.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:18 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:18.489+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:18.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:18 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:18.489+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:18.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:18 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:18.489+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:18.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:18 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:18.489+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:18.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:18.467+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:18.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:18.467+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:18.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:18.467+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:18.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:18.467+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:18.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:18.467+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:18.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:18 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:18.467+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:49.657734+0000 front 2024-09-19T06:30:49.657806+0000 (oldest deadline 2024-09-19T06:31:14.957611+0000) 2024-09-19T06:31:19.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:18 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:18.751+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:19.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:18 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:18.751+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:19.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:18 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:18.751+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:19.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:18 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:18.751+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:19.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:18 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:18.751+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:19.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:18 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:18.751+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:19.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:19.442+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:19.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:19.442+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:19.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:19.442+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:19.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:19.442+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:19.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:19.442+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:19.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:19.455+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:19.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:19.455+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:19.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:19.455+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:19.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:19.455+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:19.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:19.455+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:19.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:19 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:19.455+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:49.657734+0000 front 2024-09-19T06:30:49.657806+0000 (oldest deadline 2024-09-19T06:31:14.957611+0000) 2024-09-19T06:31:20.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:19.798+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:20.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:19.798+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:20.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:19.798+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:20.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:19.798+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:20.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:19.798+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:20.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:19 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:19.798+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:20.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:20 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:20.413+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:20.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:20 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:20.413+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:20.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:20 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:20.413+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:20.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:20 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:20.413+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:20.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:20 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:20.413+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:20.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:20.462+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:20.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:20.462+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:20.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:20.462+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:20.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:20.462+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:20.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:20.462+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:20.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:20 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:20.462+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:49.657734+0000 front 2024-09-19T06:30:49.657806+0000 (oldest deadline 2024-09-19T06:31:14.957611+0000) 2024-09-19T06:31:21.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:20 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:20.825+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:21.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:20 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:20.825+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:21.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:20 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:20.825+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:21.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:20 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:20.825+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:21.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:20 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:20.825+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:21.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:20 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:20.825+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:21.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:20 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:20.825+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.203746+0000 front 2024-09-19T06:30:55.203866+0000 (oldest deadline 2024-09-19T06:31:19.903667+0000) 2024-09-19T06:31:21.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:21 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:21.411+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:21.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:21 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:21.411+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:21.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:21 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:21.411+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:21.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:21 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:21.411+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:21.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:21 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:21.411+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:21.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:21 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:21.411+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:21.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:21 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:21.483+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:10.450944+0000 front 2024-09-19T06:30:10.451083+0000 (oldest deadline 2024-09-19T06:30:32.151108+0000) 2024-09-19T06:31:21.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:21 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:21.483+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:16.852419+0000 front 2024-09-19T06:30:16.852444+0000 (oldest deadline 2024-09-19T06:30:42.653371+0000) 2024-09-19T06:31:21.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:21 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:21.483+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:22.654043+0000 front 2024-09-19T06:30:22.654059+0000 (oldest deadline 2024-09-19T06:30:48.554065+0000) 2024-09-19T06:31:21.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:21 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:21.483+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:32.655351+0000 front 2024-09-19T06:30:32.655250+0000 (oldest deadline 2024-09-19T06:31:01.456119+0000) 2024-09-19T06:31:21.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:21 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:21.483+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:41.456623+0000 front 2024-09-19T06:30:41.456565+0000 (oldest deadline 2024-09-19T06:31:03.756687+0000) 2024-09-19T06:31:21.879 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:21 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9[62681]: 2024-09-19T06:31:21.483+0000 7f83c4f3a640 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:49.657734+0000 front 2024-09-19T06:30:49.657806+0000 (oldest deadline 2024-09-19T06:31:14.957611+0000) 2024-09-19T06:31:22.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:21 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:21.861+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:22.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:21 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:21.861+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:22.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:21 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:21.861+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:22.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:21 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:21.861+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:22.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:21 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:21.861+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:22.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:21 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:21.861+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:22.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:21 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:21.861+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.203746+0000 front 2024-09-19T06:30:55.203866+0000 (oldest deadline 2024-09-19T06:31:19.903667+0000) 2024-09-19T06:31:22.687 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:22 smithi005 podman[138716]: 2024-09-19 06:31:22.428940882 +0000 UTC m=+5.485763213 container died 1e3a2822cc1fcf1614fe202e8e27e5c5ac5b9b5f8c0d2c2beb9a0273ad6f4f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:31:22.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:22 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:22.458+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:22.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:22 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:22.458+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:22.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:22 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:22.458+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:22.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:22 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:22.458+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:22.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:22 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:22.458+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:22.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:22 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:22.458+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:23.073 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:22 smithi005 podman[138716]: 2024-09-19 06:31:22.871251187 +0000 UTC m=+5.928073513 container cleanup 1e3a2822cc1fcf1614fe202e8e27e5c5ac5b9b5f8c0d2c2beb9a0273ad6f4f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T06:31:23.073 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:22 smithi005 bash[138716]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9 2024-09-19T06:31:23.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:22 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:22.853+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:23.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:22 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:22.853+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:23.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:22 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:22.853+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:23.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:22 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:22.853+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:23.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:22 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:22.853+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:23.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:22 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:22.853+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:23.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:22 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:22.853+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.203746+0000 front 2024-09-19T06:30:55.203866+0000 (oldest deadline 2024-09-19T06:31:19.903667+0000) 2024-09-19T06:31:23.378 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:23 smithi005 podman[138731]: 2024-09-19 06:31:23.072751007 +0000 UTC m=+0.637011280 container remove 1e3a2822cc1fcf1614fe202e8e27e5c5ac5b9b5f8c0d2c2beb9a0273ad6f4f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:31:23.742 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:23 smithi005 podman[138823]: 2024-09-19 06:31:23.481922906 +0000 UTC m=+0.102460958 container create 08194b1b7e0920374181e073ccf54c7ef13b5722634acd1ec9533335d6d672a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T06:31:23.742 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:23 smithi005 podman[138823]: 2024-09-19 06:31:23.395620008 +0000 UTC m=+0.016158078 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:31:23.742 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:23 smithi005 podman[138823]: 2024-09-19 06:31:23.726146068 +0000 UTC m=+0.346684131 container init 08194b1b7e0920374181e073ccf54c7ef13b5722634acd1ec9533335d6d672a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:31:23.742 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:23 smithi005 podman[138823]: 2024-09-19 06:31:23.729714482 +0000 UTC m=+0.350252537 container start 08194b1b7e0920374181e073ccf54c7ef13b5722634acd1ec9533335d6d672a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T06:31:23.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:23 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:23.452+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:23.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:23 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:23.452+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:23.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:23 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:23.452+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:23.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:23 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:23.452+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:23.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:23 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:23.452+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:23.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:23 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:23.452+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:24.129 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:23 smithi005 podman[138823]: 2024-09-19 06:31:23.859592578 +0000 UTC m=+0.480130635 container attach 08194b1b7e0920374181e073ccf54c7ef13b5722634acd1ec9533335d6d672a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T06:31:24.129 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 19 06:31:23 smithi005 podman[138823]: 2024-09-19 06:31:23.924843456 +0000 UTC m=+0.545381510 container died 08194b1b7e0920374181e073ccf54c7ef13b5722634acd1ec9533335d6d672a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T06:31:24.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:23 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:23.816+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:24.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:23 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:23.816+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:24.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:23 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:23.816+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:24.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:23 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:23.816+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:24.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:23 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:23.816+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:24.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:23 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:23.816+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:24.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:23 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:23.816+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.203746+0000 front 2024-09-19T06:30:55.203866+0000 (oldest deadline 2024-09-19T06:31:19.903667+0000) 2024-09-19T06:31:24.519 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T06:31:24.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:31:24.602 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T06:31:24.602 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T06:31:24.602 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.10 2024-09-19T06:31:24.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:24 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:24.474+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:24.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:24 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:24.474+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:24.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:24 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:24.474+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:24.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:24 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:24.474+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:24.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:24 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:24.474+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:24.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:24 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:24.474+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:24.896 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:24 smithi049 systemd[1]: Stopping Ceph osd.10 for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:31:24.896 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:24 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:24.852+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:24.896 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:24 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:24.852+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:24.896 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:24 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:24.852+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:24.896 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:24 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:24.852+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:24.897 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:24 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:24.852+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:24.897 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:24 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:24.852+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:24.897 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:24 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:24.852+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.203746+0000 front 2024-09-19T06:30:55.203866+0000 (oldest deadline 2024-09-19T06:31:19.903667+0000) 2024-09-19T06:31:25.396 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:24 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:24.902+0000 7fbd5b63c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:31:25.396 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:24 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:24.902+0000 7fbd5b63c640 -1 osd.10 128 *** Got signal Terminated *** 2024-09-19T06:31:25.397 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:24 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:24.902+0000 7fbd5b63c640 -1 osd.10 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:31:25.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:25 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:25.435+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:25.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:25 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:25.435+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:25.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:25 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:25.435+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:25.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:25 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:25.435+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:25.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:25 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:25.435+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:25.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:25 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:25.435+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:26.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:25 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:25.818+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:26.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:25 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:25.818+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:26.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:25 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:25.818+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:26.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:25 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:25.818+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:26.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:25 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:25.818+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:26.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:25 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:25.818+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:26.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:25 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:25.818+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.203746+0000 front 2024-09-19T06:30:55.203866+0000 (oldest deadline 2024-09-19T06:31:19.903667+0000) 2024-09-19T06:31:26.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:26 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:26.408+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:26.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:26 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:26.408+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:26.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:26 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:26.408+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:26.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:26 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:26.408+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:26.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:26 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:26.408+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:26.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:26 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:26.408+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:27.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:26 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:26.840+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:27.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:26 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:26.840+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:27.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:26 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:26.840+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:27.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:26 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:26.840+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:27.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:26 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:26.840+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:27.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:26 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:26.840+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:27.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:26 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:26.840+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.203746+0000 front 2024-09-19T06:30:55.203866+0000 (oldest deadline 2024-09-19T06:31:19.903667+0000) 2024-09-19T06:31:27.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:27 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:27.454+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:27.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:27 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:27.454+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:27.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:27 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:27.454+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:27.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:27 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:27.454+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:27.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:27 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:27.454+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:27.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:27 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:27.454+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:28.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:27 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:27.821+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:28.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:27 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:27.821+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:28.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:27 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:27.821+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:28.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:27 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:27.821+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:28.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:27 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:27.821+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:28.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:27 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:27.821+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:28.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:27 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:27.821+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.203746+0000 front 2024-09-19T06:30:55.203866+0000 (oldest deadline 2024-09-19T06:31:19.903667+0000) 2024-09-19T06:31:28.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:28 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:28.411+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:28.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:28 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:28.411+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:28.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:28 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:28.411+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:28.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:28 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:28.411+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:28.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:28 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:28.411+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:28.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:28 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:28.411+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:29.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:28 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:28.843+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:29.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:28 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:28.843+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:29.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:28 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:28.843+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:29.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:28 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:28.843+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:29.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:28 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:28.843+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:29.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:28 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:28.843+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:29.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:28 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:28.843+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.203746+0000 front 2024-09-19T06:30:55.203866+0000 (oldest deadline 2024-09-19T06:31:19.903667+0000) 2024-09-19T06:31:29.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:29 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:29.447+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:29.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:29 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:29.447+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:29.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:29 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:29.447+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:29.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:29 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:29.447+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:29.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:29 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:29.447+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:29.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:29 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:29.447+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:29.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:29 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:29.447+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-09-19T06:31:05.245506+0000 front 2024-09-19T06:31:05.245536+0000 (oldest deadline 2024-09-19T06:31:29.345440+0000) 2024-09-19T06:31:30.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:29 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:29.797+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:06.195095+0000 front 2024-09-19T06:30:06.195167+0000 (oldest deadline 2024-09-19T06:30:33.195784+0000) 2024-09-19T06:31:30.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:29 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:29.797+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.496917+0000 front 2024-09-19T06:30:18.497056+0000 (oldest deadline 2024-09-19T06:30:44.397099+0000) 2024-09-19T06:31:30.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:29 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:29.797+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.098474+0000 front 2024-09-19T06:30:26.098688+0000 (oldest deadline 2024-09-19T06:30:50.099013+0000) 2024-09-19T06:31:30.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:29 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:29.797+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-19T06:30:31.199943+0000 front 2024-09-19T06:30:31.199961+0000 (oldest deadline 2024-09-19T06:30:56.500081+0000) 2024-09-19T06:31:30.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:29 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:29.797+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:40.001300+0000 front 2024-09-19T06:30:40.001329+0000 (oldest deadline 2024-09-19T06:31:04.701456+0000) 2024-09-19T06:31:30.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:29 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:29.797+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.503309+0000 front 2024-09-19T06:30:50.503421+0000 (oldest deadline 2024-09-19T06:31:15.203185+0000) 2024-09-19T06:31:30.147 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:29 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10[59872]: 2024-09-19T06:31:29.797+0000 7fbd53c39640 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.203746+0000 front 2024-09-19T06:30:55.203866+0000 (oldest deadline 2024-09-19T06:31:19.903667+0000) 2024-09-19T06:31:30.738 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:30 smithi049 podman[76362]: 2024-09-19 06:31:30.44079325 +0000 UTC m=+5.635823905 container died b304c37a2bd778ed8a1125aee70652c9178f5e4f7e85498652fcd1e82e89dd65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T06:31:30.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:30 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:30.464+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:30.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:30 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:30.464+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:30.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:30 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:30.464+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:30.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:30 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:30.464+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:30.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:30 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:30.464+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:30.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:30 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:30.464+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:30.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:30 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:30.464+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-09-19T06:31:05.245506+0000 front 2024-09-19T06:31:05.245536+0000 (oldest deadline 2024-09-19T06:31:29.345440+0000) 2024-09-19T06:31:31.113 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:30 smithi049 podman[76362]: 2024-09-19 06:31:30.836421752 +0000 UTC m=+6.031452405 container cleanup b304c37a2bd778ed8a1125aee70652c9178f5e4f7e85498652fcd1e82e89dd65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T06:31:31.113 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:30 smithi049 bash[76362]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10 2024-09-19T06:31:31.396 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:31 smithi049 podman[76377]: 2024-09-19 06:31:31.11253156 +0000 UTC m=+0.662290419 container remove b304c37a2bd778ed8a1125aee70652c9178f5e4f7e85498652fcd1e82e89dd65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T06:31:31.749 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:31 smithi049 podman[76463]: 2024-09-19 06:31:31.454948265 +0000 UTC m=+0.111246129 container create 1663d0f918432585f00529149c487ed4e73a07c7e779329fec376e8bf83d6451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T06:31:31.749 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:31 smithi049 podman[76463]: 2024-09-19 06:31:31.36037942 +0000 UTC m=+0.016677303 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:31:31.749 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:31 smithi049 podman[76463]: 2024-09-19 06:31:31.665420508 +0000 UTC m=+0.321718372 container init 1663d0f918432585f00529149c487ed4e73a07c7e779329fec376e8bf83d6451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T06:31:31.749 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:31 smithi049 podman[76463]: 2024-09-19 06:31:31.669120002 +0000 UTC m=+0.325417866 container start 1663d0f918432585f00529149c487ed4e73a07c7e779329fec376e8bf83d6451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T06:31:31.750 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:31 smithi049 podman[76463]: 2024-09-19 06:31:31.748676647 +0000 UTC m=+0.404974513 container attach 1663d0f918432585f00529149c487ed4e73a07c7e779329fec376e8bf83d6451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10-deactivate, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:31:31.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:31 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:31.416+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:31.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:31 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:31.416+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:31.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:31 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:31.416+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:31.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:31 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:31.416+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:31.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:31 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:31.416+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:31.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:31 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:31.416+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:31.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:31 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:31.416+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-09-19T06:31:05.245506+0000 front 2024-09-19T06:31:05.245536+0000 (oldest deadline 2024-09-19T06:31:29.345440+0000) 2024-09-19T06:31:32.146 INFO:journalctl@ceph.osd.10.smithi049.stdout:Sep 19 06:31:31 smithi049 podman[76463]: 2024-09-19 06:31:31.857604724 +0000 UTC m=+0.513902606 container died 1663d0f918432585f00529149c487ed4e73a07c7e779329fec376e8bf83d6451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T06:31:32.433 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T06:31:32.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:31:32.474 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T06:31:32.475 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T06:31:32.475 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.11 2024-09-19T06:31:32.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:32 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:32.441+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:32.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:32 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:32.441+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:32.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:32 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:32.441+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:32.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:32 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:32.441+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:32.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:32 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:32.441+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:32.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:32 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:32.441+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:32.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:32 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:32.441+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-09-19T06:31:05.245506+0000 front 2024-09-19T06:31:05.245536+0000 (oldest deadline 2024-09-19T06:31:29.345440+0000) 2024-09-19T06:31:32.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:32 smithi157 systemd[1]: Stopping Ceph osd.11 for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:31:33.301 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:32 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:32.800+0000 7f7f7df79640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:31:33.301 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:32 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:32.800+0000 7f7f7df79640 -1 osd.11 128 *** Got signal Terminated *** 2024-09-19T06:31:33.301 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:32 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:32.800+0000 7f7f7df79640 -1 osd.11 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:31:33.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:33 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:33.490+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:33.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:33 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:33.490+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:33.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:33 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:33.490+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:33.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:33 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:33.490+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:33.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:33 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:33.490+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:33.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:33 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:33.490+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:33.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:33 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:33.490+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-09-19T06:31:05.245506+0000 front 2024-09-19T06:31:05.245536+0000 (oldest deadline 2024-09-19T06:31:29.345440+0000) 2024-09-19T06:31:34.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:34 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:34.539+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:34.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:34 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:34.539+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:34.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:34 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:34.539+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:34.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:34 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:34.539+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:34.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:34 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:34.539+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:34.805 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:34 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:34.539+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:34.805 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:34 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:34.539+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-09-19T06:31:05.245506+0000 front 2024-09-19T06:31:05.245536+0000 (oldest deadline 2024-09-19T06:31:29.345440+0000) 2024-09-19T06:31:35.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:35 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:35.526+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:35.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:35 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:35.526+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:35.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:35 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:35.526+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:35.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:35 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:35.526+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:35.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:35 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:35.526+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:35.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:35 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:35.526+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:35.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:35 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:35.526+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-09-19T06:31:05.245506+0000 front 2024-09-19T06:31:05.245536+0000 (oldest deadline 2024-09-19T06:31:29.345440+0000) 2024-09-19T06:31:37.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:36 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:36.574+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:37.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:36 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:36.574+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:37.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:36 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:36.574+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:37.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:36 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:36.574+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:37.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:36 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:36.574+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:37.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:36 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:36.574+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:37.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:36 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:36.574+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-09-19T06:31:05.245506+0000 front 2024-09-19T06:31:05.245536+0000 (oldest deadline 2024-09-19T06:31:29.345440+0000) 2024-09-19T06:31:37.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:36 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:36.574+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-09-19T06:31:12.846400+0000 front 2024-09-19T06:31:12.846428+0000 (oldest deadline 2024-09-19T06:31:36.346432+0000) 2024-09-19T06:31:38.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:37.607+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-19T06:30:09.737060+0000 front 2024-09-19T06:30:09.736957+0000 (oldest deadline 2024-09-19T06:30:32.036959+0000) 2024-09-19T06:31:38.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:37.607+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-09-19T06:30:18.438636+0000 front 2024-09-19T06:30:18.438932+0000 (oldest deadline 2024-09-19T06:30:43.738639+0000) 2024-09-19T06:31:38.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:37.607+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T06:30:26.639660+0000 front 2024-09-19T06:30:26.639752+0000 (oldest deadline 2024-09-19T06:30:48.339849+0000) 2024-09-19T06:31:38.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:37.607+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-09-19T06:30:43.042231+0000 front 2024-09-19T06:30:43.042258+0000 (oldest deadline 2024-09-19T06:31:07.142209+0000) 2024-09-19T06:31:38.052 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:37.607+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T06:30:50.543416+0000 front 2024-09-19T06:30:50.543495+0000 (oldest deadline 2024-09-19T06:31:15.843592+0000) 2024-09-19T06:31:38.052 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:37.607+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-19T06:30:55.844358+0000 front 2024-09-19T06:30:55.844477+0000 (oldest deadline 2024-09-19T06:31:20.544385+0000) 2024-09-19T06:31:38.052 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:37.607+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-09-19T06:31:05.245506+0000 front 2024-09-19T06:31:05.245536+0000 (oldest deadline 2024-09-19T06:31:29.345440+0000) 2024-09-19T06:31:38.052 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:37 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11[60458]: 2024-09-19T06:31:37.607+0000 7f7f7a57e640 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-09-19T06:31:12.846400+0000 front 2024-09-19T06:31:12.846428+0000 (oldest deadline 2024-09-19T06:31:36.346432+0000) 2024-09-19T06:31:38.642 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:38 smithi157 podman[75717]: 2024-09-19 06:31:38.340208244 +0000 UTC m=+5.669845690 container died 3efabb13c3be3658f42bd804a896282cec4b739cea744f3aab2ff741573624ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T06:31:38.642 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:38 smithi157 podman[75717]: 2024-09-19 06:31:38.641713456 +0000 UTC m=+5.971350890 container cleanup 3efabb13c3be3658f42bd804a896282cec4b739cea744f3aab2ff741573624ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T06:31:39.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:38 smithi157 bash[75717]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11 2024-09-19T06:31:39.051 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:38 smithi157 podman[75734]: 2024-09-19 06:31:38.851273746 +0000 UTC m=+0.506257089 container remove 3efabb13c3be3658f42bd804a896282cec4b739cea744f3aab2ff741573624ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T06:31:39.432 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:39 smithi157 podman[75821]: 2024-09-19 06:31:39.1605275 +0000 UTC m=+0.112033295 container create 50116b6d614b104494e5e9e3160b6ca0bbfb38cd377ddc9d5cbbd0f216dc53cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T06:31:39.432 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:39 smithi157 podman[75821]: 2024-09-19 06:31:39.064336355 +0000 UTC m=+0.015842105 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:31:39.432 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:39 smithi157 podman[75821]: 2024-09-19 06:31:39.431533423 +0000 UTC m=+0.383039182 container init 50116b6d614b104494e5e9e3160b6ca0bbfb38cd377ddc9d5cbbd0f216dc53cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T06:31:39.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:39 smithi157 podman[75821]: 2024-09-19 06:31:39.434851938 +0000 UTC m=+0.386357696 container start 50116b6d614b104494e5e9e3160b6ca0bbfb38cd377ddc9d5cbbd0f216dc53cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T06:31:39.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:39 smithi157 podman[75821]: 2024-09-19 06:31:39.523208363 +0000 UTC m=+0.474714125 container attach 50116b6d614b104494e5e9e3160b6ca0bbfb38cd377ddc9d5cbbd0f216dc53cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-19T06:31:39.801 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:39 smithi157 podman[75821]: 2024-09-19 06:31:39.625318673 +0000 UTC m=+0.576824435 container died 50116b6d614b104494e5e9e3160b6ca0bbfb38cd377ddc9d5cbbd0f216dc53cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T06:31:40.211 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:40 smithi157 podman[75844]: 2024-09-19 06:31:40.193199083 +0000 UTC m=+0.562263217 container remove 50116b6d614b104494e5e9e3160b6ca0bbfb38cd377ddc9d5cbbd0f216dc53cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-osd-11-deactivate, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T06:31:40.211 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:40 smithi157 systemd[1]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.11.service: Deactivated successfully. 2024-09-19T06:31:40.211 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:40 smithi157 systemd[1]: Stopped Ceph osd.11 for 6b5d55e8-764b-11ef-bceb-c7b262605968. 2024-09-19T06:31:40.211 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 06:31:40 smithi157 systemd[1]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.11.service: Consumed 1min 53.735s CPU time. 2024-09-19T06:31:40.212 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T06:31:40.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:31:40.249 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T06:31:40.249 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T06:31:40.250 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.a 2024-09-19T06:31:40.552 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:31:40 smithi005 systemd[1]: Stopping Ceph mds.a for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:31:40.878 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:31:40 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-a[69721]: 2024-09-19T06:31:40.547+0000 7fd03d347640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:31:40.879 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:31:40 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-a[69721]: 2024-09-19T06:31:40.547+0000 7fd03d347640 -1 mds.a *** got signal Terminated *** 2024-09-19T06:31:42.758 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:31:42 smithi005 podman[138928]: 2024-09-19 06:31:42.465108965 +0000 UTC m=+2.013360215 container died d95c9f55a0daee6fa6a485f162eb4ed33e88b44de7898d05899d44830e0f30b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T06:31:43.129 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:31:42 smithi005 podman[138928]: 2024-09-19 06:31:42.956688497 +0000 UTC m=+2.504939747 container cleanup d95c9f55a0daee6fa6a485f162eb4ed33e88b44de7898d05899d44830e0f30b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T06:31:43.129 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 19 06:31:42 smithi005 bash[138928]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-a 2024-09-19T06:31:43.346 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T06:31:43.381 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:31:43.382 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T06:31:43.382 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T06:31:43.382 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.d 2024-09-19T06:31:43.629 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:31:43 smithi005 systemd[1]: Stopping Ceph mds.d for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:31:44.129 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:31:43 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-d[68532]: 2024-09-19T06:31:43.736+0000 7faaf4066640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:31:44.129 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:31:43 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-d[68532]: 2024-09-19T06:31:43.736+0000 7faaf4066640 -1 mds.d *** got signal Terminated *** 2024-09-19T06:31:45.879 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:31:45 smithi005 podman[139048]: 2024-09-19 06:31:45.612895582 +0000 UTC m=+2.010814524 container died 3923c8f34b2c13eb6bcbb3a45f342b99a30d431867cb0dd4d63ed95429f03a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T06:31:46.263 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:31:45 smithi005 podman[139048]: 2024-09-19 06:31:45.986618095 +0000 UTC m=+2.384537036 container cleanup 3923c8f34b2c13eb6bcbb3a45f342b99a30d431867cb0dd4d63ed95429f03a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T06:31:46.263 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 19 06:31:45 smithi005 bash[139048]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-d 2024-09-19T06:31:46.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T06:31:46.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:31:46.437 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T06:31:46.437 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T06:31:46.437 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.g 2024-09-19T06:31:46.879 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:31:46 smithi005 systemd[1]: Stopping Ceph mds.g for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:31:46.879 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:31:46 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-g[69340]: 2024-09-19T06:31:46.765+0000 7f06e2e92640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:31:46.879 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:31:46 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-g[69340]: 2024-09-19T06:31:46.765+0000 7f06e2e92640 -1 mds.g *** got signal Terminated *** 2024-09-19T06:31:50.762 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:31:50 smithi005 podman[139183]: 2024-09-19 06:31:50.471895861 +0000 UTC m=+3.811255887 container died 270ffe4313e0dba4d6c9ef0b293794784beccf47e2c8ee491edc493a4c542111 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T06:31:51.100 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:31:50 smithi005 podman[139183]: 2024-09-19 06:31:50.848774066 +0000 UTC m=+4.188134087 container cleanup 270ffe4313e0dba4d6c9ef0b293794784beccf47e2c8ee491edc493a4c542111 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-g, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T06:31:51.101 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 19 06:31:50 smithi005 bash[139183]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-g 2024-09-19T06:31:51.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T06:31:51.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:31:51.267 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T06:31:51.267 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T06:31:51.267 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.j 2024-09-19T06:31:51.378 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:31:51 smithi005 systemd[1]: Stopping Ceph mds.j for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:31:51.879 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:31:51 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-j[68136]: 2024-09-19T06:31:51.585+0000 7fe51a533640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:31:51.901 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:31:51 smithi005 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-j[68136]: 2024-09-19T06:31:51.585+0000 7fe51a533640 -1 mds.j *** got signal Terminated *** 2024-09-19T06:31:54.379 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:31:54 smithi005 podman[139304]: 2024-09-19 06:31:54.100364212 +0000 UTC m=+2.613229367 container died 8b209cf01c7a97c1db2eab1dbf31580ab5a14741cac8d7e304315c00b746e449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T06:31:54.837 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:31:54 smithi005 podman[139304]: 2024-09-19 06:31:54.419799784 +0000 UTC m=+2.932664938 container cleanup 8b209cf01c7a97c1db2eab1dbf31580ab5a14741cac8d7e304315c00b746e449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T06:31:54.837 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:31:54 smithi005 bash[139304]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-j 2024-09-19T06:31:54.837 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 19 06:31:54 smithi005 podman[139316]: 2024-09-19 06:31:54.620889424 +0000 UTC m=+0.515756520 container remove 8b209cf01c7a97c1db2eab1dbf31580ab5a14741cac8d7e304315c00b746e449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T06:31:54.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T06:31:54.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:31:54.884 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T06:31:54.884 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T06:31:54.884 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.b 2024-09-19T06:31:55.171 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:31:54 smithi049 systemd[1]: Stopping Ceph mds.b for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:31:55.646 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:31:55 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:31:55.169+0000 7f6a1d154640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:31:55.646 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:31:55 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b[65091]: 2024-09-19T06:31:55.169+0000 7f6a1d154640 -1 mds.b *** got signal Terminated *** 2024-09-19T06:31:58.646 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:31:58 smithi049 podman[76578]: 2024-09-19 06:31:58.384108262 +0000 UTC m=+3.298987779 container died 855f6af864068389d939cc28e97ae6e139b7d69c18c9a6bf1cf3472977bc9ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T06:31:58.995 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:31:58 smithi049 podman[76578]: 2024-09-19 06:31:58.7350234 +0000 UTC m=+3.649902917 container cleanup 855f6af864068389d939cc28e97ae6e139b7d69c18c9a6bf1cf3472977bc9ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T06:31:58.995 INFO:journalctl@ceph.mds.b.smithi049.stdout:Sep 19 06:31:58 smithi049 bash[76578]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-b 2024-09-19T06:31:59.167 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T06:31:59.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:31:59.204 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T06:31:59.204 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T06:31:59.204 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.e 2024-09-19T06:31:59.531 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:31:59 smithi049 systemd[1]: Stopping Ceph mds.e for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:31:59.896 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:31:59 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e[64470]: 2024-09-19T06:31:59.530+0000 7f2cbf0de640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:31:59.896 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:31:59 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e[64470]: 2024-09-19T06:31:59.530+0000 7f2cbf0de640 -1 mds.e *** got signal Terminated *** 2024-09-19T06:32:02.646 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:32:02 smithi049 podman[76697]: 2024-09-19 06:32:02.382270495 +0000 UTC m=+2.953179861 container died d8d4fef20850f9954e44fcae320b7d63ad779ad4982066924079c1bd8c9b7d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:32:03.040 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:32:02 smithi049 podman[76697]: 2024-09-19 06:32:02.671679889 +0000 UTC m=+3.242589226 container cleanup d8d4fef20850f9954e44fcae320b7d63ad779ad4982066924079c1bd8c9b7d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T06:32:03.040 INFO:journalctl@ceph.mds.e.smithi049.stdout:Sep 19 06:32:02 smithi049 bash[76697]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-e 2024-09-19T06:32:03.174 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T06:32:03.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:32:03.209 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T06:32:03.209 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T06:32:03.209 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.h 2024-09-19T06:32:03.396 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:32:03 smithi049 systemd[1]: Stopping Ceph mds.h for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:32:03.896 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:32:03 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-h[64788]: 2024-09-19T06:32:03.525+0000 7f469ed85640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:32:03.896 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:32:03 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-h[64788]: 2024-09-19T06:32:03.525+0000 7f469ed85640 -1 mds.h *** got signal Terminated *** 2024-09-19T06:32:06.646 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:32:06 smithi049 podman[76817]: 2024-09-19 06:32:06.389076424 +0000 UTC m=+2.959437679 container died c5637008da4752bc17648beb2827202928ac0cc7349d0e8e0345be266953c530 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T06:32:07.018 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:32:06 smithi049 podman[76817]: 2024-09-19 06:32:06.758551508 +0000 UTC m=+3.328912751 container cleanup c5637008da4752bc17648beb2827202928ac0cc7349d0e8e0345be266953c530 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T06:32:07.018 INFO:journalctl@ceph.mds.h.smithi049.stdout:Sep 19 06:32:06 smithi049 bash[76817]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-h 2024-09-19T06:32:07.198 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T06:32:07.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:32:07.251 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T06:32:07.251 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T06:32:07.251 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.k 2024-09-19T06:32:07.547 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:32:07 smithi049 systemd[1]: Stopping Ceph mds.k for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:32:07.896 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:32:07 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-k[64160]: 2024-09-19T06:32:07.545+0000 7fe16fbae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:32:07.896 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:32:07 smithi049 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-k[64160]: 2024-09-19T06:32:07.545+0000 7fe16fbae640 -1 mds.k *** got signal Terminated *** 2024-09-19T06:32:10.646 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:32:10 smithi049 podman[76952]: 2024-09-19 06:32:10.386550012 +0000 UTC m=+2.930902098 container died bc1bd613a9967427c9dfbff367f39bc58e3e7fad9d4251cb6b68586b794856cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-k, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T06:32:11.013 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:32:10 smithi049 podman[76952]: 2024-09-19 06:32:10.761792556 +0000 UTC m=+3.306144635 container cleanup bc1bd613a9967427c9dfbff367f39bc58e3e7fad9d4251cb6b68586b794856cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T06:32:11.014 INFO:journalctl@ceph.mds.k.smithi049.stdout:Sep 19 06:32:10 smithi049 bash[76952]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-k 2024-09-19T06:32:11.144 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T06:32:11.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:32:11.179 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T06:32:11.179 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T06:32:11.179 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.c 2024-09-19T06:32:11.551 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:32:11 smithi157 systemd[1]: Stopping Ceph mds.c for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:32:12.051 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:32:11 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-c[65576]: 2024-09-19T06:32:11.741+0000 7f68cdfe0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:32:12.051 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:32:11 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-c[65576]: 2024-09-19T06:32:11.741+0000 7f68cdfe0640 -1 mds.c *** got signal Terminated *** 2024-09-19T06:32:14.702 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:32:14 smithi157 podman[75923]: 2024-09-19 06:32:14.426955314 +0000 UTC m=+3.055411930 container died 2cdb769b3739ebb7d118655d3d40eeeb285c62b67fba00c7119e6f899ff430a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-c, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T06:32:15.051 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:32:14 smithi157 podman[75923]: 2024-09-19 06:32:14.800117327 +0000 UTC m=+3.428573929 container cleanup 2cdb769b3739ebb7d118655d3d40eeeb285c62b67fba00c7119e6f899ff430a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T06:32:15.051 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 06:32:14 smithi157 bash[75923]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-c 2024-09-19T06:32:15.233 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T06:32:15.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:32:15.268 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T06:32:15.269 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T06:32:15.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.f 2024-09-19T06:32:15.614 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:32:15 smithi157 systemd[1]: Stopping Ceph mds.f for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:32:16.051 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:32:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f[65272]: 2024-09-19T06:32:15.611+0000 7f31b35a1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:32:16.051 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:32:15 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f[65272]: 2024-09-19T06:32:15.611+0000 7f31b35a1640 -1 mds.f *** got signal Terminated *** 2024-09-19T06:32:18.721 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:32:18 smithi157 podman[76057]: 2024-09-19 06:32:18.397932567 +0000 UTC m=+2.915454269 container died 0b945e16d702cc6a73ec2cff6e05752772e54c9d4356a58a0345e161fe2395ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:32:19.052 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:32:18 smithi157 podman[76057]: 2024-09-19 06:32:18.720528147 +0000 UTC m=+3.238049846 container cleanup 0b945e16d702cc6a73ec2cff6e05752772e54c9d4356a58a0345e161fe2395ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T06:32:19.052 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:32:18 smithi157 bash[76057]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f 2024-09-19T06:32:19.053 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 06:32:18 smithi157 podman[76068]: 2024-09-19 06:32:18.946933755 +0000 UTC m=+0.542501116 container remove 0b945e16d702cc6a73ec2cff6e05752772e54c9d4356a58a0345e161fe2395ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T06:32:19.193 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T06:32:19.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:32:19.227 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T06:32:19.228 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T06:32:19.228 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.i 2024-09-19T06:32:19.541 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:32:19 smithi157 systemd[1]: Stopping Ceph mds.i for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:32:19.801 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:32:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-i[65874]: 2024-09-19T06:32:19.539+0000 7f7011c38640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:32:19.801 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:32:19 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-i[65874]: 2024-09-19T06:32:19.539+0000 7f7011c38640 -1 mds.i *** got signal Terminated *** 2024-09-19T06:32:22.676 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:32:22 smithi157 podman[76180]: 2024-09-19 06:32:22.397530707 +0000 UTC m=+2.957963562 container died 052a38933963a477f2c2097209bd64b0ab27158bade7c6aa3184a4760269b2b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-i, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T06:32:23.017 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:32:22 smithi157 podman[76180]: 2024-09-19 06:32:22.757766289 +0000 UTC m=+3.318199136 container cleanup 052a38933963a477f2c2097209bd64b0ab27158bade7c6aa3184a4760269b2b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T06:32:23.018 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 06:32:22 smithi157 bash[76180]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-i 2024-09-19T06:32:23.178 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T06:32:23.210 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:32:23.210 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T06:32:23.211 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T06:32:23.211 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.l 2024-09-19T06:32:23.529 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:32:23 smithi157 systemd[1]: Stopping Ceph mds.l for 6b5d55e8-764b-11ef-bceb-c7b262605968... 2024-09-19T06:32:23.801 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:32:23 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-l[64955]: 2024-09-19T06:32:23.527+0000 7f5a4830d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:32:23.801 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:32:23 smithi157 ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-l[64955]: 2024-09-19T06:32:23.527+0000 7f5a4830d640 -1 mds.l *** got signal Terminated *** 2024-09-19T06:32:26.679 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:32:26 smithi157 podman[76302]: 2024-09-19 06:32:26.405758751 +0000 UTC m=+2.978663530 container died 45250dc07c390cbde5a89a2e3a82ddab7c8f0794b007a6d0103f70f30af08e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T06:32:27.046 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:32:26 smithi157 podman[76302]: 2024-09-19 06:32:26.76960116 +0000 UTC m=+3.342505948 container cleanup 45250dc07c390cbde5a89a2e3a82ddab7c8f0794b007a6d0103f70f30af08e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T06:32:27.046 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 06:32:26 smithi157 bash[76302]: ceph-6b5d55e8-764b-11ef-bceb-c7b262605968-mds-l 2024-09-19T06:32:27.176 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6b5d55e8-764b-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T06:32:27.207 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:32:27.208 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T06:32:27.208 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T06:32:27.419 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T06:32:30.020 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T06:32:30.233 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T06:32:32.751 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T06:32:32.951 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T06:32:35.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:32:35.478 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:32:35.506 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:32:35.533 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T06:32:35.536 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698/remote/smithi005/crash 2024-09-19T06:32:35.537 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/crash -- . 2024-09-19T06:32:35.583 INFO:teuthology.orchestra.run.smithi005.stderr:tar: /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T06:32:35.583 INFO:teuthology.orchestra.run.smithi005.stderr:tar: Error is not recoverable: exiting now 2024-09-19T06:32:35.585 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698/remote/smithi049/crash 2024-09-19T06:32:35.586 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/crash -- . 2024-09-19T06:32:35.625 INFO:teuthology.orchestra.run.smithi049.stderr:tar: /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T06:32:35.625 INFO:teuthology.orchestra.run.smithi049.stderr:tar: Error is not recoverable: exiting now 2024-09-19T06:32:35.626 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698/remote/smithi157/crash 2024-09-19T06:32:35.627 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/crash -- . 2024-09-19T06:32:35.662 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T06:32:35.662 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2024-09-19T06:32:35.663 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T06:32:35.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T06:32:35.704 INFO:teuthology.orchestra.run.smithi005.stdout:2024-09-19T06:12:58.881977+0000 mds.b (mds.0) 64 : cluster [WRN] Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-19T06:32:35.705 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-19T06:32:35.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[SEC\]' /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T06:32:35.785 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]' /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T06:32:35.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[WRN\]' /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T06:32:35.939 INFO:teuthology.orchestra.run.smithi005.stdout:2024-09-19T06:12:58.881977+0000 mds.b (mds.0) 64 : cluster [WRN] Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-19T06:32:35.940 INFO:tasks.cephadm:Compressing logs... 2024-09-19T06:32:35.941 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T06:32:35.984 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T06:32:35.986 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T06:32:36.016 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T06:32:36.016 INFO:teuthology.orchestra.run.smithi005.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T06:32:36.017 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T06:32:36.019 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T06:32:36.019 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T06:32:36.019 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T06:32:36.019 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T06:32:36.019 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T06:32:36.019 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T06:32:36.019 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T06:32:36.024 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log 2024-09-19T06:32:36.024 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T06:32:36.024 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T06:32:36.024 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T06:32:36.024 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T06:32:36.025 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T06:32:36.025 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T06:32:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T06:32:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T06:32:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T06:32:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.0.log: /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T06:32:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T06:32:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T06:32:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T06:32:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T06:32:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T06:32:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log 2024-09-19T06:32:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T06:32:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T06:32:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T06:32:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T06:32:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-19T06:32:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.j.log: /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T06:32:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 90.4% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T06:32:36.031 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T06:32:36.031 INFO:teuthology.orchestra.run.smithi005.stderr: 94.6% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T06:32:36.031 INFO:teuthology.orchestra.run.smithi005.stderr: 94.6% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T06:32:36.033 INFO:teuthology.orchestra.run.smithi049.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-09-19T06:32:36.033 INFO:teuthology.orchestra.run.smithi049.stderr: /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T06:32:36.033 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T06:32:36.033 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T06:32:36.033 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T06:32:36.033 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T06:32:36.033 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T06:32:36.033 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T06:32:36.034 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T06:32:36.034 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T06:32:36.034 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T06:32:36.034 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log 2024-09-19T06:32:36.034 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T06:32:36.034 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T06:32:36.034 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T06:32:36.034 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T06:32:36.034 INFO:teuthology.orchestra.run.smithi005.stderr: 84.8% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T06:32:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 82.5% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T06:32:36.151 INFO:teuthology.orchestra.run.smithi049.stderr: 92.6% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T06:32:36.162 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.7% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T06:32:36.314 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T06:32:36.441 INFO:teuthology.orchestra.run.smithi157.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T06:32:36.506 INFO:teuthology.orchestra.run.smithi157.stderr: 82.5% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T06:32:36.516 INFO:teuthology.orchestra.run.smithi157.stderr: 91.0% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T06:32:36.585 INFO:teuthology.orchestra.run.smithi049.stderr: 87.6% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T06:32:36.599 INFO:teuthology.orchestra.run.smithi157.stderr: 90.6% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T06:32:36.621 INFO:teuthology.orchestra.run.smithi049.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T06:32:36.626 INFO:teuthology.orchestra.run.smithi049.stderr: 90.6% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T06:32:36.677 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T06:32:36.848 INFO:teuthology.orchestra.run.smithi005.stderr: 94.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T06:32:37.051 INFO:teuthology.orchestra.run.smithi157.stderr: 87.6% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T06:32:38.196 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T06:32:38.889 INFO:teuthology.orchestra.run.smithi005.stderr: 90.3% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T06:32:45.509 INFO:teuthology.orchestra.run.smithi157.stderr: 91.6% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T06:32:57.814 INFO:teuthology.orchestra.run.smithi049.stderr: 91.6% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T06:33:19.680 INFO:teuthology.orchestra.run.smithi005.stderr: 95.2% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T06:33:22.998 INFO:teuthology.orchestra.run.smithi005.stderr: 94.9% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T06:33:32.282 INFO:teuthology.orchestra.run.smithi005.stderr: 93.7% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T06:33:49.026 INFO:teuthology.orchestra.run.smithi005.stderr: 93.9% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T06:33:54.809 INFO:teuthology.orchestra.run.smithi005.stderr: 93.6% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T06:34:09.957 INFO:teuthology.orchestra.run.smithi049.stderr: 95.1% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T06:34:15.191 INFO:teuthology.orchestra.run.smithi049.stderr: 93.8% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T06:34:15.448 INFO:teuthology.orchestra.run.smithi005.stderr: 93.8% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T06:34:17.727 INFO:teuthology.orchestra.run.smithi049.stderr: 94.9% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T06:34:24.381 INFO:teuthology.orchestra.run.smithi157.stderr: 93.8% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T06:34:59.947 INFO:teuthology.orchestra.run.smithi157.stderr: 94.8% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T06:35:01.691 INFO:teuthology.orchestra.run.smithi049.stderr: 93.7% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T06:35:05.433 INFO:teuthology.orchestra.run.smithi049.stderr: 93.8% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T06:35:18.750 INFO:teuthology.orchestra.run.smithi157.stderr: 93.9% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T06:36:05.236 INFO:teuthology.orchestra.run.smithi157.stderr: 93.7% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T06:36:08.919 INFO:teuthology.orchestra.run.smithi049.stderr: 93.8% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T06:36:30.925 INFO:teuthology.orchestra.run.smithi157.stderr: 93.8% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T06:41:26.151 INFO:teuthology.orchestra.run.smithi005.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-19T06:41:26.153 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-09-19T06:41:26.153 INFO:teuthology.orchestra.run.smithi005.stderr:real 8m50.156s 2024-09-19T06:41:26.153 INFO:teuthology.orchestra.run.smithi005.stderr:user 7m23.194s 2024-09-19T06:41:26.153 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m18.438s 2024-09-19T06:50:23.269 INFO:teuthology.orchestra.run.smithi049.stderr: 94.2% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T06:53:06.305 INFO:teuthology.orchestra.run.smithi049.stderr: 95.2% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T06:53:06.307 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-09-19T06:53:06.307 INFO:teuthology.orchestra.run.smithi049.stderr:real 20m30.309s 2024-09-19T06:53:06.307 INFO:teuthology.orchestra.run.smithi049.stderr:user 7m30.680s 2024-09-19T06:53:06.307 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m21.839s 2024-09-19T06:55:53.093 INFO:teuthology.orchestra.run.smithi157.stderr: 94.7% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T06:59:15.023 INFO:teuthology.orchestra.run.smithi157.stderr: 94.0% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T06:59:29.594 INFO:teuthology.orchestra.run.smithi157.stderr: 94.2% -- replaced with /var/log/ceph/6b5d55e8-764b-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T06:59:29.597 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-09-19T06:59:29.597 INFO:teuthology.orchestra.run.smithi157.stderr:real 26m53.596s 2024-09-19T06:59:29.597 INFO:teuthology.orchestra.run.smithi157.stderr:user 7m36.576s 2024-09-19T06:59:29.597 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m24.995s 2024-09-19T06:59:29.598 INFO:tasks.cephadm:Archiving logs... 2024-09-19T06:59:29.600 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698/remote/smithi005/log 2024-09-19T06:59:29.601 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:00:41.141 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698/remote/smithi049/log 2024-09-19T07:00:41.524 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:01:38.343 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698/remote/smithi157/log 2024-09-19T07:01:38.346 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:02:41.286 INFO:tasks.cephadm:Removing cluster... 2024-09-19T07:02:41.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 --force 2024-09-19T07:02:41.520 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T07:02:41.847 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 --force 2024-09-19T07:02:42.094 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T07:02:42.690 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6b5d55e8-764b-11ef-bceb-c7b262605968 --force 2024-09-19T07:02:43.332 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 6b5d55e8-764b-11ef-bceb-c7b262605968 2024-09-19T07:02:44.003 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T07:02:44.003 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T07:02:44.024 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T07:02:44.045 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T07:02:44.061 INFO:tasks.cephadm:Teardown complete 2024-09-19T07:02:44.061 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T07:02:44.076 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T07:02:44.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:02:44.079 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:02:44.087 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:02:44.213 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T07:02:44.223 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T07:02:44.223 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:02:44.226 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:02:44.229 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:02:44.241 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:02:44.243 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:02:44.244 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:02:44.268 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:02:44.268 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-09-19T07:02:44.269 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv01.front.sepia.ceph.com 3 7 377 102 +440us[ +425us] +/- 43ms 2024-09-19T07:02:44.269 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 -1051us[-1051us] +/- 71ms 2024-09-19T07:02:44.269 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 32 +324us[ +324us] +/- 69ms 2024-09-19T07:02:44.269 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:02:44.279 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:02:44.279 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-09-19T07:02:44.279 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +447us[ +438us] +/- 45ms 2024-09-19T07:02:44.279 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -1096us[-1096us] +/- 71ms 2024-09-19T07:02:44.279 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +261us[ +252us] +/- 69ms 2024-09-19T07:02:44.279 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:02:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:02:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-09-19T07:02:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 +450us[ +449us] +/- 44ms 2024-09-19T07:02:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -1073us[-1074us] +/- 71ms 2024-09-19T07:02:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +257us[ +256us] +/- 69ms 2024-09-19T07:02:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:02:44.288 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T07:02:44.298 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T07:02:44.298 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T07:02:44.312 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:02:44.482 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:02:44.601 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:02:44.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:02:44.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:02:44.774 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:02:44.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:02:44.809 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:02:44.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:02:44.837 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T07:02:44.850 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T07:02:44.859 INFO:teuthology.task.internal:Duration was 4539.674976 seconds 2024-09-19T07:02:44.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T07:02:44.868 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T07:02:44.868 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:02:44.871 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:02:44.874 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:02:44.932 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:02:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:02:44.949 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:02:45.362 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T07:02:45.362 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-09-19T07:02:45.362 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:02:45.387 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-09-19T07:02:45.387 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:02:45.426 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-09-19T07:02:45.426 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:02:45.453 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T07:02:45.453 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:02:45.456 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:02:45.469 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:02:45.521 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T07:02:45.521 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:02:45.530 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:02:45.541 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:02:47.394 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T07:02:47.406 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T07:02:47.406 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:02:47.433 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:02:47.476 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:02:47.520 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T07:02:47.532 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:02:47.534 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:02:47.537 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:02:47.569 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-09-19T07:02:47.582 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-09-19T07:02:47.583 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-09-19T07:02:47.619 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:02:47.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:02:47.645 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:02:47.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:02:47.662 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:02:47.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:02:47.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T07:02:47.690 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T07:02:47.691 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698/remote/smithi005 2024-09-19T07:02:47.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:02:47.770 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698/remote/smithi049 2024-09-19T07:02:47.770 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:02:47.812 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911698/remote/smithi157 2024-09-19T07:02:47.812 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:02:47.855 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T07:02:47.855 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:02:47.857 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:02:47.860 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:02:47.904 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T07:02:47.917 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T07:02:47.917 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T07:02:47.932 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T07:02:47.932 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:02:47.934 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:02:47.937 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:02:47.948 INFO:teuthology.orchestra.run.smithi005.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:02 /home/ubuntu/cephtest 2024-09-19T07:02:47.951 INFO:teuthology.orchestra.run.smithi049.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:02 /home/ubuntu/cephtest 2024-09-19T07:02:47.961 INFO:teuthology.orchestra.run.smithi157.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:02 /home/ubuntu/cephtest 2024-09-19T07:02:47.962 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T07:02:47.974 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T07:02:48.011 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} duration: 4539.674976110458 failure_reason: '"2024-09-19T06:12:58.881977+0000 mds.b (mds.0) 64 : cluster [WRN] Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/cf663d32-66ce-4414-bd62-494f4d2d56f9/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-09-19T07:02:48.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T07:02:48.099 INFO:teuthology.run:FAIL